From 754bbf7a25a8dda49b5d08ef0d0443bbf5af0e36 Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Sun, 7 Apr 2024 13:41:34 -0500 Subject: new repository --- assets/info/bash.info.gz | Bin 0 -> 144001 bytes assets/info/calc.info.gz | Bin 0 -> 483335 bytes assets/info/diffutils.info.gz | Bin 0 -> 61188 bytes assets/info/dir | 71 + assets/info/ediff.info.gz | Bin 0 -> 43341 bytes assets/info/eintr.info.gz | Bin 0 -> 207762 bytes assets/info/ert.info.gz | Bin 0 -> 21584 bytes assets/info/gawk.info.gz | Bin 0 -> 475091 bytes assets/info/git.info | 5681 + assets/info/grep.info.gz | Bin 0 -> 32302 bytes assets/info/gzip.info.gz | Bin 0 -> 15784 bytes assets/info/info.info.gz | Bin 0 -> 25998 bytes assets/info/latex2e.info | 21120 ++ assets/info/make.info-1.gz | Bin 0 -> 88166 bytes assets/info/make.info-2.gz | Bin 0 -> 80031 bytes assets/info/make.info.gz | Bin 0 -> 2651 bytes assets/info/python38.info | 352968 ++++++++++++++++++ assets/info/pythonpackagingguide.info | 15961 + assets/info/sed.info.gz | Bin 0 -> 54068 bytes assets/info/setuptools.info | 26390 ++ assets/info/sicp.info | 33394 ++ assets/info/standards.info | 6129 + assets/info/stow.info.gz | Bin 0 -> 29144 bytes assets/info/texinfo.info-1.gz | Bin 0 -> 87938 bytes assets/info/texinfo.info-2.gz | Bin 0 -> 99917 bytes assets/info/texinfo.info-3.gz | Bin 0 -> 69545 bytes assets/info/texinfo.info.gz | Bin 0 -> 5941 bytes assets/info/use-package.texi.files/doclicense.texi | 505 + assets/info/use-package.texi.files/docstyle.texi | 19 + .../info/use-package.texi.files/use-package.texi | 1903 + assets/info/wget.info.gz | Bin 0 -> 71860 bytes assets/info/zsh.info | 191 + assets/info/zsh.info-1 | 7487 + assets/info/zsh.info-2 | 6945 + assets/info/zsh.info-3 | 5810 + assets/info/zsh.info-4 | 7058 + assets/info/zsh.info-5 | 7230 + assets/info/zsh.info-6 | Bin 0 -> 289103 bytes assets/info/zsh.info-7 | Bin 0 -> 38011 bytes 39 files changed, 498862 insertions(+) create mode 100644 assets/info/bash.info.gz create mode 100644 assets/info/calc.info.gz create mode 100644 assets/info/diffutils.info.gz create mode 100644 assets/info/dir create mode 100644 assets/info/ediff.info.gz create mode 100644 assets/info/eintr.info.gz create mode 100644 assets/info/ert.info.gz create mode 100644 assets/info/gawk.info.gz create mode 100644 assets/info/git.info create mode 100644 assets/info/grep.info.gz create mode 100644 assets/info/gzip.info.gz create mode 100644 assets/info/info.info.gz create mode 100644 assets/info/latex2e.info create mode 100644 assets/info/make.info-1.gz create mode 100644 assets/info/make.info-2.gz create mode 100644 assets/info/make.info.gz create mode 100644 assets/info/python38.info create mode 100644 assets/info/pythonpackagingguide.info create mode 100644 assets/info/sed.info.gz create mode 100644 assets/info/setuptools.info create mode 100644 assets/info/sicp.info create mode 100644 assets/info/standards.info create mode 100644 assets/info/stow.info.gz create mode 100644 assets/info/texinfo.info-1.gz create mode 100644 assets/info/texinfo.info-2.gz create mode 100644 assets/info/texinfo.info-3.gz create mode 100644 assets/info/texinfo.info.gz create mode 100644 assets/info/use-package.texi.files/doclicense.texi create mode 100644 assets/info/use-package.texi.files/docstyle.texi create mode 100644 assets/info/use-package.texi.files/use-package.texi create mode 100644 assets/info/wget.info.gz create mode 100644 assets/info/zsh.info create mode 100644 assets/info/zsh.info-1 create mode 100644 assets/info/zsh.info-2 create mode 100644 assets/info/zsh.info-3 create mode 100644 assets/info/zsh.info-4 create mode 100644 assets/info/zsh.info-5 create mode 100644 assets/info/zsh.info-6 create mode 100644 assets/info/zsh.info-7 (limited to 'assets/info') diff --git a/assets/info/bash.info.gz b/assets/info/bash.info.gz new file mode 100644 index 00000000..e05630a5 Binary files /dev/null and b/assets/info/bash.info.gz differ diff --git a/assets/info/calc.info.gz b/assets/info/calc.info.gz new file mode 100644 index 00000000..e3d12f4d Binary files /dev/null and b/assets/info/calc.info.gz differ diff --git a/assets/info/diffutils.info.gz b/assets/info/diffutils.info.gz new file mode 100644 index 00000000..0f59bced Binary files /dev/null and b/assets/info/diffutils.info.gz differ diff --git a/assets/info/dir b/assets/info/dir new file mode 100644 index 00000000..2250e770 --- /dev/null +++ b/assets/info/dir @@ -0,0 +1,71 @@ +This is the file .../info/dir, which contains the +topmost node of the Info hierarchy, called (dir)Top. +The first time you invoke Info you start off looking at this node. + +File: dir, Node: Top This is the top of the INFO tree + + This (the Directory node) gives a menu of major topics. + Typing "q" exits, "H" lists all Info commands, "d" returns here, + "h" gives a primer for first-timers, + "mEmacs" visits the Emacs manual, etc. + + In Emacs, you can click mouse button 2 on a menu item or cross reference + to select it. + +* Menu: + +Lisp +* Emacs Lisp Intro: (eintr). A Simple Introduction to Emacs Lisp Programming. +* SICP: (sicp). Structure and Interpretation of Computer Programs +* ERT: (ert). Emacs Lisp regression testing tool. + +Emacs Features +* Calc: (calc). Advanced desk calculator and mathematical tool. +* Ediff: (ediff). A visual interface for comparing and merging programs. + +Development Tools +* Git: (git). A fast distributed revision control system +* Make: (make). Remake files automatically. + +LaTeX +* LaTeX2e: (latex2e). Unofficial LaTeX reference manual. + +Python +* Python 3.8.9: (python38). Python Reference +* Python Packaging Guide: (pythonpackagingguide). Python Packaging User Guide +* Python Packaging Authority: (setuptools). Python Packaging Authority + +Shells +* Bash: (bash). The GNU Bourne-Again SHell. +* ZSH: (zsh). The Z Shell Manual. + +Texinfo documentation system +* Texinfo: (texinfo). The GNU documentation format. +* install-info: (texinfo)Invoking install-info. Update info/dir entries. +* makeinfo: (texinfo)Invoking makeinfo. Translate Texinfo source. +* pdftexi2dvi: (texinfo)PDF Output. PDF output for Texinfo. +* texi2dvi: (texinfo)Format with texi2dvi. Print Texinfo documents. +* texi2pdf: (texinfo)PDF Output. PDF output for Texinfo. +* texindex: (texinfo)Format with tex/texindex. Sort Texinfo index files. + +Utilities +* Diffutils: (diffutils). Comparing and merging files. +* Grep: (grep). Print lines that match patterns. +* Gzip: (gzip). General (de)compression of files (lzw). +* Stow: (stow). GNU Stow. +* Wget: (wget). Non-interactive network downloader. +* awk: (gawk)Invoking Gawk. Text scanning and processing. +* cmp: (diffutils)Invoking cmp. Compare 2 files byte by byte. +* diff3: (diffutils)Invoking diff3. Compare 3 files line by line. +* diff: (diffutils)Invoking diff. Compare 2 files line by line. +* gunzip: (gzip)Overview. Compression / Decompression. +* gzexe: (gzip)Overview. Compress executables. +* patch: (diffutils)Invoking patch. Apply a patch to a file. +* sdiff: (diffutils)Invoking sdiff. Merge 2 files side-by-side. +* sed: (sed). Stream EDitor. +* zcat: (gzip)Overview. Decompression to stdout. +* zdiff: (gzip)Overview. Compare compressed files. +* zforce: (gzip)Overview. Force .gz extension on files. +* zgrep: (gzip)Overview. Search compressed files. +* zmore: (gzip)Overview. Decompression output by pages. + diff --git a/assets/info/ediff.info.gz b/assets/info/ediff.info.gz new file mode 100644 index 00000000..26601cba Binary files /dev/null and b/assets/info/ediff.info.gz differ diff --git a/assets/info/eintr.info.gz b/assets/info/eintr.info.gz new file mode 100644 index 00000000..152b38af Binary files /dev/null and b/assets/info/eintr.info.gz differ diff --git a/assets/info/ert.info.gz b/assets/info/ert.info.gz new file mode 100644 index 00000000..4d992c57 Binary files /dev/null and b/assets/info/ert.info.gz differ diff --git a/assets/info/gawk.info.gz b/assets/info/gawk.info.gz new file mode 100644 index 00000000..309b0cd9 Binary files /dev/null and b/assets/info/gawk.info.gz differ diff --git a/assets/info/git.info b/assets/info/git.info new file mode 100644 index 00000000..6c91b18a --- /dev/null +++ b/assets/info/git.info @@ -0,0 +1,5681 @@ +This is git.info, produced by makeinfo version 6.5 from +user-manual.texi. + +INFO-DIR-SECTION Development +START-INFO-DIR-ENTRY +* Git: (git). A fast distributed revision control system +END-INFO-DIR-ENTRY + + +File: git.info, Node: Top, Next: idm4, Up: (dir) + +Git User Manual +*************** + +* Menu: + +* : idm4. +* Repositories and Branches:: +* Exploring Git history:: +* Developing with Git:: +* Sharing development with others:: +* Rewriting history and maintaining patch series:: +* Advanced branch management:: +* Git concepts:: +* Submodules:: +* Low-level Git operations:: +* Hacking Git:: +* Git Glossary:: +* Git Quick Reference:: +* Notes and todo list for this manual:: + +— The Detailed Node Listing — + +Repositories and Branches + +* How to get a Git repository:: +* How to check out a different version of a project:: +* Understanding History; Commits:: +* Manipulating branches:: +* Examining an old version without creating a new branch:: +* Examining branches from a remote repository:: +* Naming branches, tags, and other references: Naming branches; tags; and other references. +* Updating a repository with git fetch:: +* Fetching branches from other repositories:: + +Exploring Git history + +* How to use bisect to find a regression:: +* Naming commits:: +* Creating tags:: +* Browsing revisions:: +* Generating diffs:: +* Viewing old file versions:: +* Examples:: + +Developing with Git + +* Telling Git your name:: +* Creating a new repository:: +* How to make a commit:: +* Creating good commit messages:: +* Ignoring files:: +* How to merge:: +* Resolving a merge:: +* Undoing a merge:: +* Fast-forward merges:: +* Fixing mistakes:: +* Ensuring good performance:: +* Ensuring reliability:: + +Sharing development with others + +* Getting updates with git pull:: +* Submitting patches to a project:: +* Importing patches to a project:: +* Public Git repositories:: +* How to get a Git repository with minimal history:: +* Examples: Examples <1>. + +Rewriting history and maintaining patch series + +* Creating the perfect patch series:: +* Keeping a patch series up to date using git rebase:: +* Rewriting a single commit:: +* Reordering or selecting from a patch series:: +* Using interactive rebases:: +* Other tools:: +* Problems with rewriting history:: +* Why bisecting merge commits can be harder than bisecting linear history:: + +Advanced branch management + +* Fetching individual branches:: +* git fetch and fast-forwards:: +* Forcing git fetch to do non-fast-forward updates:: +* Configuring remote-tracking branches:: + +Git concepts + +* The Object Database:: +* The index:: + +Submodules + +* Pitfalls with submodules:: + +Low-level Git operations + +* Object access and manipulation:: +* The Workflow:: +* Examining the data:: +* Merging multiple trees:: +* Merging multiple trees, continued: Merging multiple trees; continued. + +Hacking Git + +* Object storage format:: +* A birds-eye view of Git’s source code:: + +Git Glossary + +* Git explained:: + +Git Quick Reference + +* Creating a new repository: Creating a new repository <1>. +* Managing branches:: +* Exploring history:: +* Making changes:: +* Merging:: +* Sharing your changes:: +* Repository maintenance:: + +Notes and todo list for this manual + +* Todo list:: + + + +File: git.info, Node: idm4, Next: Repositories and Branches, Prev: Top, Up: Top + +Git is a fast distributed revision control system. + + This manual is designed to be readable by someone with basic UNIX +command-line skills, but no previous knowledge of Git. + + *note Repositories and Branches:: and *note Exploring Git history:: +explain how to fetch and study a project using git—read these chapters +to learn how to build and test a particular version of a software +project, search for regressions, and so on. + + People needing to do actual development will also want to read *note +Developing with Git:: and *note Sharing development with others::. + + Further chapters cover more specialized topics. + + Comprehensive reference documentation is available through the man +pages, or git-help(1) (git-help.html) command. For example, for the +command ‘git clone ’, you can either use: + + $ man git-clone + + or: + + $ git help clone + + With the latter, you can use the manual viewer of your choice; see +git-help(1) (git-help.html) for more information. + + See also *note Git Quick Reference:: for a brief overview of Git +commands, without any explanation. + + Finally, see *note Notes and todo list for this manual:: for ways +that you can help make this manual more complete. + + +File: git.info, Node: Repositories and Branches, Next: Exploring Git history, Prev: idm4, Up: Top + +1 Repositories and Branches +*************************** + +* Menu: + +* How to get a Git repository:: +* How to check out a different version of a project:: +* Understanding History; Commits:: +* Manipulating branches:: +* Examining an old version without creating a new branch:: +* Examining branches from a remote repository:: +* Naming branches, tags, and other references: Naming branches; tags; and other references. +* Updating a repository with git fetch:: +* Fetching branches from other repositories:: + + +File: git.info, Node: How to get a Git repository, Next: How to check out a different version of a project, Up: Repositories and Branches + +1.1 How to get a Git repository +=============================== + +It will be useful to have a Git repository to experiment with as you +read this manual. + + The best way to get one is by using the git-clone(1) (git-clone.html) +command to download a copy of an existing repository. If you don’t +already have a project in mind, here are some interesting examples: + + # Git itself (approx. 40MB download): + $ git clone git://git.kernel.org/pub/scm/git/git.git + # the Linux kernel (approx. 640MB download): + $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + + The initial clone may be time-consuming for a large project, but you +will only need to clone once. + + The clone command creates a new directory named after the project +(‘git’ or ‘linux’ in the examples above). After you cd into this +directory, you will see that it contains a copy of the project files, +called the working tree (*note [def_working_tree]::), together with a +special top-level directory named ‘.git’, which contains all the +information about the history of the project. + + +File: git.info, Node: How to check out a different version of a project, Next: Understanding History; Commits, Prev: How to get a Git repository, Up: Repositories and Branches + +1.2 How to check out a different version of a project +===================================================== + +Git is best thought of as a tool for storing the history of a collection +of files. It stores the history as a compressed collection of +interrelated snapshots of the project’s contents. In Git each such +version is called a commit (*note [def_commit]::). + + Those snapshots aren’t necessarily all arranged in a single line from +oldest to newest; instead, work may simultaneously proceed along +parallel lines of development, called branches (*note [def_branch]::), +which may merge and diverge. + + A single Git repository can track development on multiple branches. +It does this by keeping a list of heads (*note [def_head]::) which +reference the latest commit on each branch; the git-branch(1) +(git-branch.html) command shows you the list of branch heads: + + $ git branch + * master + + A freshly cloned repository contains a single branch head, by default +named "master", with the working directory initialized to the state of +the project referred to by that branch head. + + Most projects also use tags (*note [def_tag]::). Tags, like heads, +are references into the project’s history, and can be listed using the +git-tag(1) (git-tag.html) command: + + $ git tag -l + v2.6.11 + v2.6.11-tree + v2.6.12 + v2.6.12-rc2 + v2.6.12-rc3 + v2.6.12-rc4 + v2.6.12-rc5 + v2.6.12-rc6 + v2.6.13 + ... + + Tags are expected to always point at the same version of a project, +while heads are expected to advance as development progresses. + + Create a new branch head pointing to one of these versions and check +it out using git-switch(1) (git-switch.html): + + $ git switch -c new v2.6.13 + + The working directory then reflects the contents that the project had +when it was tagged v2.6.13, and git-branch(1) (git-branch.html) shows +two branches, with an asterisk marking the currently checked-out branch: + + $ git branch + master + * new + + If you decide that you’d rather see version 2.6.17, you can modify +the current branch to point at v2.6.17 instead, with + + $ git reset --hard v2.6.17 + + Note that if the current branch head was your only reference to a +particular point in history, then resetting that branch may leave you +with no way to find the history it used to point to; so use this command +carefully. + + +File: git.info, Node: Understanding History; Commits, Next: Manipulating branches, Prev: How to check out a different version of a project, Up: Repositories and Branches + +1.3 Understanding History: Commits +================================== + +Every change in the history of a project is represented by a commit. +The git-show(1) (git-show.html) command shows the most recent commit on +the current branch: + + $ git show + commit 17cf781661e6d38f737f15f53ab552f1e95960d7 + Author: Linus Torvalds + Date: Tue Apr 19 14:11:06 2005 -0700 + + Remove duplicate getenv(DB_ENVIRONMENT) call + + Noted by Tony Luck. + + diff --git a/init-db.c b/init-db.c + index 65898fa..b002dc6 100644 + --- a/init-db.c + +++ b/init-db.c + @@ -7,7 +7,7 @@ + + int main(int argc, char **argv) + { + - char *sha1_dir = getenv(DB_ENVIRONMENT), *path; + + char *sha1_dir, *path; + int len, i; + + if (mkdir(".git", 0755) < 0) { + + As you can see, a commit shows who made the latest change, what they +did, and why. + + Every commit has a 40-hexdigit id, sometimes called the "object name" +or the "SHA-1 id", shown on the first line of the ‘git show’ output. +You can usually refer to a commit by a shorter name, such as a tag or a +branch name, but this longer name can also be useful. Most importantly, +it is a globally unique name for this commit: so if you tell somebody +else the object name (for example in email), then you are guaranteed +that name will refer to the same commit in their repository that it does +in yours (assuming their repository has that commit at all). Since the +object name is computed as a hash over the contents of the commit, you +are guaranteed that the commit can never change without its name also +changing. + + In fact, in *note Git concepts:: we shall see that everything stored +in Git history, including file data and directory contents, is stored in +an object with a name that is a hash of its contents. + +* Menu: + +* Understanding history; commits, parents, and reachability: Understanding history; commits; parents; and reachability. +* Understanding history; History diagrams:: +* Understanding history; What is a branch?:: + + +File: git.info, Node: Understanding history; commits; parents; and reachability, Next: Understanding history; History diagrams, Up: Understanding History; Commits + +1.3.1 Understanding history: commits, parents, and reachability +--------------------------------------------------------------- + +Every commit (except the very first commit in a project) also has a +parent commit which shows what happened before this commit. Following +the chain of parents will eventually take you back to the beginning of +the project. + + However, the commits do not form a simple list; Git allows lines of +development to diverge and then reconverge, and the point where two +lines of development reconverge is called a "merge". The commit +representing a merge can therefore have more than one parent, with each +parent representing the most recent commit on one of the lines of +development leading to that point. + + The best way to see how this works is using the gitk(1) (gitk.html) +command; running gitk now on a Git repository and looking for merge +commits will help understand how Git organizes history. + + In the following, we say that commit X is "reachable" from commit Y +if commit X is an ancestor of commit Y. Equivalently, you could say that +Y is a descendant of X, or that there is a chain of parents leading from +commit Y to commit X. + + +File: git.info, Node: Understanding history; History diagrams, Next: Understanding history; What is a branch?, Prev: Understanding history; commits; parents; and reachability, Up: Understanding History; Commits + +1.3.2 Understanding history: History diagrams +--------------------------------------------- + +We will sometimes represent Git history using diagrams like the one +below. Commits are shown as "o", and the links between them with lines +drawn with - / and \. Time goes left to right: + + o--o--o <-- Branch A + / + o--o--o <-- master + \ + o--o--o <-- Branch B + + If we need to talk about a particular commit, the character "o" may +be replaced with another letter or number. + + +File: git.info, Node: Understanding history; What is a branch?, Prev: Understanding history; History diagrams, Up: Understanding History; Commits + +1.3.3 Understanding history: What is a branch? +---------------------------------------------- + +When we need to be precise, we will use the word "branch" to mean a line +of development, and "branch head" (or just "head") to mean a reference +to the most recent commit on a branch. In the example above, the branch +head named "A" is a pointer to one particular commit, but we refer to +the line of three commits leading up to that point as all being part of +"branch A". + + However, when no confusion will result, we often just use the term +"branch" both for branches and for branch heads. + + +File: git.info, Node: Manipulating branches, Next: Examining an old version without creating a new branch, Prev: Understanding History; Commits, Up: Repositories and Branches + +1.4 Manipulating branches +========================= + +Creating, deleting, and modifying branches is quick and easy; here’s a +summary of the commands: + +‘git branch’ + list all branches. + +‘git branch ’ + create a new branch named ‘’, referencing the same point in + history as the current branch. + +‘git branch ’ + create a new branch named ‘’, referencing ‘’, + which may be specified any way you like, including using a branch + name or a tag name. + +‘git branch -d ’ + delete the branch ‘’; if the branch is not fully merged in + its upstream branch or contained in the current branch, this + command will fail with a warning. + +‘git branch -D ’ + delete the branch ‘’ irrespective of its merged status. + +‘git switch ’ + make the current branch ‘’, updating the working directory + to reflect the version referenced by ‘’. + +‘git switch -c ’ + create a new branch ‘’ referencing ‘’, and check + it out. + + The special symbol "HEAD" can always be used to refer to the current +branch. In fact, Git uses a file named ‘HEAD’ in the ‘.git’ directory +to remember which branch is current: + + $ cat .git/HEAD + ref: refs/heads/master + + +File: git.info, Node: Examining an old version without creating a new branch, Next: Examining branches from a remote repository, Prev: Manipulating branches, Up: Repositories and Branches + +1.5 Examining an old version without creating a new branch +========================================================== + +The ‘git switch’ command normally expects a branch head, but will also +accept an arbitrary commit when invoked with –detach; for example, you +can check out the commit referenced by a tag: + + $ git switch --detach v2.6.17 + Note: checking out 'v2.6.17'. + + You are in 'detached HEAD' state. You can look around, make experimental + changes and commit them, and you can discard any commits you make in this + state without impacting any branches by performing another switch. + + If you want to create a new branch to retain commits you create, you may + do so (now or later) by using -c with the switch command again. Example: + + git switch -c new_branch_name + + HEAD is now at 427abfa Linux v2.6.17 + + The HEAD then refers to the SHA-1 of the commit instead of to a +branch, and git branch shows that you are no longer on a branch: + + $ cat .git/HEAD + 427abfa28afedffadfca9dd8b067eb6d36bac53f + $ git branch + * (detached from v2.6.17) + master + + In this case we say that the HEAD is "detached". + + This is an easy way to check out a particular version without having +to make up a name for the new branch. You can still create a new branch +(or tag) for this version later if you decide to. + + +File: git.info, Node: Examining branches from a remote repository, Next: Naming branches; tags; and other references, Prev: Examining an old version without creating a new branch, Up: Repositories and Branches + +1.6 Examining branches from a remote repository +=============================================== + +The "master" branch that was created at the time you cloned is a copy of +the HEAD in the repository that you cloned from. That repository may +also have had other branches, though, and your local repository keeps +branches which track each of those remote branches, called +remote-tracking branches, which you can view using the ‘-r’ option to +git-branch(1) (git-branch.html): + + $ git branch -r + origin/HEAD + origin/html + origin/maint + origin/man + origin/master + origin/next + origin/seen + origin/todo + + In this example, "origin" is called a remote repository, or "remote" +for short. The branches of this repository are called "remote branches" +from our point of view. The remote-tracking branches listed above were +created based on the remote branches at clone time and will be updated +by ‘git fetch’ (hence ‘git pull’) and ‘git push’. See *note Updating a +repository with git fetch:: for details. + + You might want to build on one of these remote-tracking branches on a +branch of your own, just as you would for a tag: + + $ git switch -c my-todo-copy origin/todo + + You can also check out ‘origin/todo’ directly to examine it or write +a one-off patch. See detached head (*note Examining an old version +without creating a new branch::). + + Note that the name "origin" is just the name that Git uses by default +to refer to the repository that you cloned from. + + +File: git.info, Node: Naming branches; tags; and other references, Next: Updating a repository with git fetch, Prev: Examining branches from a remote repository, Up: Repositories and Branches + +1.7 Naming branches, tags, and other references +=============================================== + +Branches, remote-tracking branches, and tags are all references to +commits. All references are named with a slash-separated path name +starting with ‘refs’; the names we’ve been using so far are actually +shorthand: + + • The branch ‘test’ is short for ‘refs/heads/test’. + + • The tag ‘v2.6.18’ is short for ‘refs/tags/v2.6.18’. + + • ‘origin/master’ is short for ‘refs/remotes/origin/master’. + + The full name is occasionally useful if, for example, there ever +exists a tag and a branch with the same name. + + (Newly created refs are actually stored in the ‘.git/refs’ directory, +under the path given by their name. However, for efficiency reasons +they may also be packed together in a single file; see git-pack-refs(1) +(git-pack-refs.html)). + + As another useful shortcut, the "HEAD" of a repository can be +referred to just using the name of that repository. So, for example, +"origin" is usually a shortcut for the HEAD branch in the repository +"origin". + + For the complete list of paths which Git checks for references, and +the order it uses to decide which to choose when there are multiple +references with the same shorthand name, see the "SPECIFYING REVISIONS" +section of gitrevisions(7) (gitrevisions.html). + + +File: git.info, Node: Updating a repository with git fetch, Next: Fetching branches from other repositories, Prev: Naming branches; tags; and other references, Up: Repositories and Branches + +1.8 Updating a repository with git fetch +======================================== + +After you clone a repository and commit a few changes of your own, you +may wish to check the original repository for updates. + + The ‘git-fetch’ command, with no arguments, will update all of the +remote-tracking branches to the latest version found in the original +repository. It will not touch any of your own branches—not even the +"master" branch that was created for you on clone. + + +File: git.info, Node: Fetching branches from other repositories, Prev: Updating a repository with git fetch, Up: Repositories and Branches + +1.9 Fetching branches from other repositories +============================================= + +You can also track branches from repositories other than the one you +cloned from, using git-remote(1) (git-remote.html): + + $ git remote add staging git://git.kernel.org/.../gregkh/staging.git + $ git fetch staging + ... + From git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging + * [new branch] master -> staging/master + * [new branch] staging-linus -> staging/staging-linus + * [new branch] staging-next -> staging/staging-next + + New remote-tracking branches will be stored under the shorthand name +that you gave ‘git remote add’, in this case ‘staging’: + + $ git branch -r + origin/HEAD -> origin/master + origin/master + staging/master + staging/staging-linus + staging/staging-next + + If you run ‘git fetch ’ later, the remote-tracking branches +for the named ‘’ will be updated. + + If you examine the file ‘.git/config’, you will see that Git has +added a new stanza: + + $ cat .git/config + ... + [remote "staging"] + url = git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git + fetch = +refs/heads/*:refs/remotes/staging/* + ... + + This is what causes Git to track the remote’s branches; you may +modify or delete these configuration options by editing ‘.git/config’ +with a text editor. (See the "CONFIGURATION FILE" section of +git-config(1) (git-config.html) for details.) + + +File: git.info, Node: Exploring Git history, Next: Developing with Git, Prev: Repositories and Branches, Up: Top + +2 Exploring Git history +*********************** + +Git is best thought of as a tool for storing the history of a collection +of files. It does this by storing compressed snapshots of the contents +of a file hierarchy, together with "commits" which show the +relationships between these snapshots. + + Git provides extremely flexible and fast tools for exploring the +history of a project. + + We start with one specialized tool that is useful for finding the +commit that introduced a bug into a project. + +* Menu: + +* How to use bisect to find a regression:: +* Naming commits:: +* Creating tags:: +* Browsing revisions:: +* Generating diffs:: +* Viewing old file versions:: +* Examples:: + + +File: git.info, Node: How to use bisect to find a regression, Next: Naming commits, Up: Exploring Git history + +2.1 How to use bisect to find a regression +========================================== + +Suppose version 2.6.18 of your project worked, but the version at +"master" crashes. Sometimes the best way to find the cause of such a +regression is to perform a brute-force search through the project’s +history to find the particular commit that caused the problem. The +git-bisect(1) (git-bisect.html) command can help you do this: + + $ git bisect start + $ git bisect good v2.6.18 + $ git bisect bad master + Bisecting: 3537 revisions left to test after this + [65934a9a028b88e83e2b0f8b36618fe503349f8e] BLOCK: Make USB storage depend on SCSI rather than selecting it [try #6] + + If you run ‘git branch’ at this point, you’ll see that Git has +temporarily moved you in "(no branch)". HEAD is now detached from any +branch and points directly to a commit (with commit id 65934) that is +reachable from "master" but not from v2.6.18. Compile and test it, and +see whether it crashes. Assume it does crash. Then: + + $ git bisect bad + Bisecting: 1769 revisions left to test after this + [7eff82c8b1511017ae605f0c99ac275a7e21b867] i2c-core: Drop useless bitmaskings + + checks out an older version. Continue like this, telling Git at each +stage whether the version it gives you is good or bad, and notice that +the number of revisions left to test is cut approximately in half each +time. + + After about 13 tests (in this case), it will output the commit id of +the guilty commit. You can then examine the commit with git-show(1) +(git-show.html), find out who wrote it, and mail them your bug report +with the commit id. Finally, run + + $ git bisect reset + + to return you to the branch you were on before. + + Note that the version which ‘git bisect’ checks out for you at each +point is just a suggestion, and you’re free to try a different version +if you think it would be a good idea. For example, occasionally you may +land on a commit that broke something unrelated; run + + $ git bisect visualize + + which will run gitk and label the commit it chose with a marker that +says "bisect". Choose a safe-looking commit nearby, note its commit id, +and check it out with: + + $ git reset --hard fb47ddb2db + + then test, run ‘bisect good’ or ‘bisect bad’ as appropriate, and +continue. + + Instead of ‘git bisect visualize’ and then ‘git reset --hard +fb47ddb2db’, you might just want to tell Git that you want to skip the +current commit: + + $ git bisect skip + + In this case, though, Git may not eventually be able to tell the +first bad one between some first skipped commits and a later bad commit. + + There are also ways to automate the bisecting process if you have a +test script that can tell a good from a bad commit. See git-bisect(1) +(git-bisect.html) for more information about this and other ‘git bisect’ +features. + + +File: git.info, Node: Naming commits, Next: Creating tags, Prev: How to use bisect to find a regression, Up: Exploring Git history + +2.2 Naming commits +================== + +We have seen several ways of naming commits already: + + • 40-hexdigit object name + + • branch name: refers to the commit at the head of the given branch + + • tag name: refers to the commit pointed to by the given tag (we’ve + seen branches and tags are special cases of references (*note + Naming branches; tags; and other references::)). + + • HEAD: refers to the head of the current branch + + There are many more; see the "SPECIFYING REVISIONS" section of the +gitrevisions(7) (gitrevisions.html) man page for the complete list of +ways to name revisions. Some examples: + + $ git show fb47ddb2 # the first few characters of the object name + # are usually enough to specify it uniquely + $ git show HEAD^ # the parent of the HEAD commit + $ git show HEAD^^ # the grandparent + $ git show HEAD~4 # the great-great-grandparent + + Recall that merge commits may have more than one parent; by default, +‘^’ and ‘~’ follow the first parent listed in the commit, but you can +also choose: + + $ git show HEAD^1 # show the first parent of HEAD + $ git show HEAD^2 # show the second parent of HEAD + + In addition to HEAD, there are several other special names for +commits: + + Merges (to be discussed later), as well as operations such as ‘git +reset’, which change the currently checked-out commit, generally set +ORIG_HEAD to the value HEAD had before the current operation. + + The ‘git fetch’ operation always stores the head of the last fetched +branch in FETCH_HEAD. For example, if you run ‘git fetch’ without +specifying a local branch as the target of the operation + + $ git fetch git://example.com/proj.git theirbranch + + the fetched commits will still be available from FETCH_HEAD. + + When we discuss merges we’ll also see the special name MERGE_HEAD, +which refers to the other branch that we’re merging in to the current +branch. + + The git-rev-parse(1) (git-rev-parse.html) command is a low-level +command that is occasionally useful for translating some name for a +commit to the object name for that commit: + + $ git rev-parse origin + e05db0fd4f31dde7005f075a84f96b360d05984b + + +File: git.info, Node: Creating tags, Next: Browsing revisions, Prev: Naming commits, Up: Exploring Git history + +2.3 Creating tags +================= + +We can also create a tag to refer to a particular commit; after running + + $ git tag stable-1 1b2e1d63ff + + You can use ‘stable-1’ to refer to the commit 1b2e1d63ff. + + This creates a "lightweight" tag. If you would also like to include +a comment with the tag, and possibly sign it cryptographically, then you +should create a tag object instead; see the git-tag(1) (git-tag.html) +man page for details. + + +File: git.info, Node: Browsing revisions, Next: Generating diffs, Prev: Creating tags, Up: Exploring Git history + +2.4 Browsing revisions +====================== + +The git-log(1) (git-log.html) command can show lists of commits. On its +own, it shows all commits reachable from the parent commit; but you can +also make more specific requests: + + $ git log v2.5.. # commits since (not reachable from) v2.5 + $ git log test..master # commits reachable from master but not test + $ git log master..test # ...reachable from test but not master + $ git log master...test # ...reachable from either test or master, + # but not both + $ git log --since="2 weeks ago" # commits from the last 2 weeks + $ git log Makefile # commits which modify Makefile + $ git log fs/ # ... which modify any file under fs/ + $ git log -S'foo()' # commits which add or remove any file data + # matching the string 'foo()' + + And of course you can combine all of these; the following finds +commits since v2.5 which touch the ‘Makefile’ or any file under ‘fs’: + + $ git log v2.5.. Makefile fs/ + + You can also ask git log to show patches: + + $ git log -p + + See the ‘--pretty’ option in the git-log(1) (git-log.html) man page +for more display options. + + Note that git log starts with the most recent commit and works +backwards through the parents; however, since Git history can contain +multiple independent lines of development, the particular order that +commits are listed in may be somewhat arbitrary. + + +File: git.info, Node: Generating diffs, Next: Viewing old file versions, Prev: Browsing revisions, Up: Exploring Git history + +2.5 Generating diffs +==================== + +You can generate diffs between any two versions using git-diff(1) +(git-diff.html): + + $ git diff master..test + + That will produce the diff between the tips of the two branches. If +you’d prefer to find the diff from their common ancestor to test, you +can use three dots instead of two: + + $ git diff master...test + + Sometimes what you want instead is a set of patches; for this you can +use git-format-patch(1) (git-format-patch.html): + + $ git format-patch master..test + + will generate a file with a patch for each commit reachable from test +but not from master. + + +File: git.info, Node: Viewing old file versions, Next: Examples, Prev: Generating diffs, Up: Exploring Git history + +2.6 Viewing old file versions +============================= + +You can always view an old version of a file by just checking out the +correct revision first. But sometimes it is more convenient to be able +to view an old version of a single file without checking anything out; +this command does that: + + $ git show v2.5:fs/locks.c + + Before the colon may be anything that names a commit, and after it +may be any path to a file tracked by Git. + + +File: git.info, Node: Examples, Prev: Viewing old file versions, Up: Exploring Git history + +2.7 Examples +============ + +* Menu: + +* Counting the number of commits on a branch:: +* Check whether two branches point at the same history:: +* Find first tagged version including a given fix:: +* Showing commits unique to a given branch:: +* Creating a changelog and tarball for a software release:: +* Finding commits referencing a file with given content:: + + +File: git.info, Node: Counting the number of commits on a branch, Next: Check whether two branches point at the same history, Up: Examples + +2.7.1 Counting the number of commits on a branch +------------------------------------------------ + +Suppose you want to know how many commits you’ve made on ‘mybranch’ +since it diverged from ‘origin’: + + $ git log --pretty=oneline origin..mybranch | wc -l + + Alternatively, you may often see this sort of thing done with the +lower-level command git-rev-list(1) (git-rev-list.html), which just +lists the SHA-1’s of all the given commits: + + $ git rev-list origin..mybranch | wc -l + + +File: git.info, Node: Check whether two branches point at the same history, Next: Find first tagged version including a given fix, Prev: Counting the number of commits on a branch, Up: Examples + +2.7.2 Check whether two branches point at the same history +---------------------------------------------------------- + +Suppose you want to check whether two branches point at the same point +in history. + + $ git diff origin..master + + will tell you whether the contents of the project are the same at the +two branches; in theory, however, it’s possible that the same project +contents could have been arrived at by two different historical routes. +You could compare the object names: + + $ git rev-list origin + e05db0fd4f31dde7005f075a84f96b360d05984b + $ git rev-list master + e05db0fd4f31dde7005f075a84f96b360d05984b + + Or you could recall that the ‘...’ operator selects all commits +reachable from either one reference or the other but not both; so + + $ git log origin...master + + will return no commits when the two branches are equal. + + +File: git.info, Node: Find first tagged version including a given fix, Next: Showing commits unique to a given branch, Prev: Check whether two branches point at the same history, Up: Examples + +2.7.3 Find first tagged version including a given fix +----------------------------------------------------- + +Suppose you know that the commit e05db0fd fixed a certain problem. +You’d like to find the earliest tagged release that contains that fix. + + Of course, there may be more than one answer—if the history branched +after commit e05db0fd, then there could be multiple "earliest" tagged +releases. + + You could just visually inspect the commits since e05db0fd: + + $ gitk e05db0fd.. + + or you can use git-name-rev(1) (git-name-rev.html), which will give +the commit a name based on any tag it finds pointing to one of the +commit’s descendants: + + $ git name-rev --tags e05db0fd + e05db0fd tags/v1.5.0-rc1^0~23 + + The git-describe(1) (git-describe.html) command does the opposite, +naming the revision using a tag on which the given commit is based: + + $ git describe e05db0fd + v1.5.0-rc0-260-ge05db0f + + but that may sometimes help you guess which tags might come after the +given commit. + + If you just want to verify whether a given tagged version contains a +given commit, you could use git-merge-base(1) (git-merge-base.html): + + $ git merge-base e05db0fd v1.5.0-rc1 + e05db0fd4f31dde7005f075a84f96b360d05984b + + The merge-base command finds a common ancestor of the given commits, +and always returns one or the other in the case where one is a +descendant of the other; so the above output shows that e05db0fd +actually is an ancestor of v1.5.0-rc1. + + Alternatively, note that + + $ git log v1.5.0-rc1..e05db0fd + + will produce empty output if and only if v1.5.0-rc1 includes +e05db0fd, because it outputs only commits that are not reachable from +v1.5.0-rc1. + + As yet another alternative, the git-show-branch(1) +(git-show-branch.html) command lists the commits reachable from its +arguments with a display on the left-hand side that indicates which +arguments that commit is reachable from. So, if you run something like + + $ git show-branch e05db0fd v1.5.0-rc0 v1.5.0-rc1 v1.5.0-rc2 + ! [e05db0fd] Fix warnings in sha1_file.c - use C99 printf format if + available + ! [v1.5.0-rc0] GIT v1.5.0 preview + ! [v1.5.0-rc1] GIT v1.5.0-rc1 + ! [v1.5.0-rc2] GIT v1.5.0-rc2 + ... + + then a line like + + + ++ [e05db0fd] Fix warnings in sha1_file.c - use C99 printf format if + available + + shows that e05db0fd is reachable from itself, from v1.5.0-rc1, and +from v1.5.0-rc2, and not from v1.5.0-rc0. + + +File: git.info, Node: Showing commits unique to a given branch, Next: Creating a changelog and tarball for a software release, Prev: Find first tagged version including a given fix, Up: Examples + +2.7.4 Showing commits unique to a given branch +---------------------------------------------- + +Suppose you would like to see all the commits reachable from the branch +head named ‘master’ but not from any other head in your repository. + + We can list all the heads in this repository with git-show-ref(1) +(git-show-ref.html): + + $ git show-ref --heads + bf62196b5e363d73353a9dcf094c59595f3153b7 refs/heads/core-tutorial + db768d5504c1bb46f63ee9d6e1772bd047e05bf9 refs/heads/maint + a07157ac624b2524a059a3414e99f6f44bebc1e7 refs/heads/master + 24dbc180ea14dc1aebe09f14c8ecf32010690627 refs/heads/tutorial-2 + 1e87486ae06626c2f31eaa63d26fc0fd646c8af2 refs/heads/tutorial-fixes + + We can get just the branch-head names, and remove ‘master’, with the +help of the standard utilities cut and grep: + + $ git show-ref --heads | cut -d' ' -f2 | grep -v '^refs/heads/master' + refs/heads/core-tutorial + refs/heads/maint + refs/heads/tutorial-2 + refs/heads/tutorial-fixes + + And then we can ask to see all the commits reachable from master but +not from these other heads: + + $ gitk master --not $( git show-ref --heads | cut -d' ' -f2 | + grep -v '^refs/heads/master' ) + + Obviously, endless variations are possible; for example, to see all +commits reachable from some head but not from any tag in the repository: + + $ gitk $( git show-ref --heads ) --not $( git show-ref --tags ) + + (See gitrevisions(7) (gitrevisions.html) for explanations of +commit-selecting syntax such as ‘--not’.) + + +File: git.info, Node: Creating a changelog and tarball for a software release, Next: Finding commits referencing a file with given content, Prev: Showing commits unique to a given branch, Up: Examples + +2.7.5 Creating a changelog and tarball for a software release +------------------------------------------------------------- + +The git-archive(1) (git-archive.html) command can create a tar or zip +archive from any version of a project; for example: + + $ git archive -o latest.tar.gz --prefix=project/ HEAD + + will use HEAD to produce a gzipped tar archive in which each filename +is preceded by ‘project/’. The output file format is inferred from the +output file extension if possible, see git-archive(1) (git-archive.html) +for details. + + Versions of Git older than 1.7.7 don’t know about the ‘tar.gz’ +format, you’ll need to use gzip explicitly: + + $ git archive --format=tar --prefix=project/ HEAD | gzip >latest.tar.gz + + If you’re releasing a new version of a software project, you may want +to simultaneously make a changelog to include in the release +announcement. + + Linus Torvalds, for example, makes new kernel releases by tagging +them, then running: + + $ release-script 2.6.12 2.6.13-rc6 2.6.13-rc7 + + where release-script is a shell script that looks like: + + #!/bin/sh + stable="$1" + last="$2" + new="$3" + echo "# git tag v$new" + echo "git archive --prefix=linux-$new/ v$new | gzip -9 > ../linux-$new.tar.gz" + echo "git diff v$stable v$new | gzip -9 > ../patch-$new.gz" + echo "git log --no-merges v$new ^v$last > ../ChangeLog-$new" + echo "git shortlog --no-merges v$new ^v$last > ../ShortLog" + echo "git diff --stat --summary -M v$last v$new > ../diffstat-$new" + + and then he just cut-and-pastes the output commands after verifying +that they look OK. + + +File: git.info, Node: Finding commits referencing a file with given content, Prev: Creating a changelog and tarball for a software release, Up: Examples + +2.7.6 Finding commits referencing a file with given content +----------------------------------------------------------- + +Somebody hands you a copy of a file, and asks which commits modified a +file such that it contained the given content either before or after the +commit. You can find out with this: + + $ git log --raw --abbrev=40 --pretty=oneline | + grep -B 1 `git hash-object filename` + + Figuring out why this works is left as an exercise to the (advanced) +student. The git-log(1) (git-log.html), git-diff-tree(1) +(git-diff-tree.html), and git-hash-object(1) (git-hash-object.html) man +pages may prove helpful. + + +File: git.info, Node: Developing with Git, Next: Sharing development with others, Prev: Exploring Git history, Up: Top + +3 Developing with Git +********************* + +* Menu: + +* Telling Git your name:: +* Creating a new repository:: +* How to make a commit:: +* Creating good commit messages:: +* Ignoring files:: +* How to merge:: +* Resolving a merge:: +* Undoing a merge:: +* Fast-forward merges:: +* Fixing mistakes:: +* Ensuring good performance:: +* Ensuring reliability:: + + +File: git.info, Node: Telling Git your name, Next: Creating a new repository, Up: Developing with Git + +3.1 Telling Git your name +========================= + +Before creating any commits, you should introduce yourself to Git. The +easiest way to do so is to use git-config(1) (git-config.html): + + $ git config --global user.name 'Your Name Comes Here' + $ git config --global user.email 'you@yourdomain.example.com' + + Which will add the following to a file named ‘.gitconfig’ in your +home directory: + + [user] + name = Your Name Comes Here + email = you@yourdomain.example.com + + See the "CONFIGURATION FILE" section of git-config(1) +(git-config.html) for details on the configuration file. The file is +plain text, so you can also edit it with your favorite editor. + + +File: git.info, Node: Creating a new repository, Next: How to make a commit, Prev: Telling Git your name, Up: Developing with Git + +3.2 Creating a new repository +============================= + +Creating a new repository from scratch is very easy: + + $ mkdir project + $ cd project + $ git init + + If you have some initial content (say, a tarball): + + $ tar xzvf project.tar.gz + $ cd project + $ git init + $ git add . # include everything below ./ in the first commit: + $ git commit + + +File: git.info, Node: How to make a commit, Next: Creating good commit messages, Prev: Creating a new repository, Up: Developing with Git + +3.3 How to make a commit +======================== + +Creating a new commit takes three steps: + + 1. Making some changes to the working directory using your favorite + editor. + + 2. Telling Git about your changes. + + 3. Creating the commit using the content you told Git about in step 2. + + In practice, you can interleave and repeat steps 1 and 2 as many +times as you want: in order to keep track of what you want committed at +step 3, Git maintains a snapshot of the tree’s contents in a special +staging area called "the index." + + At the beginning, the content of the index will be identical to that +of the HEAD. The command ‘git diff --cached’, which shows the difference +between the HEAD and the index, should therefore produce no output at +that point. + + Modifying the index is easy: + + To update the index with the contents of a new or modified file, use + + $ git add path/to/file + + To remove a file from the index and from the working tree, use + + $ git rm path/to/file + + After each step you can verify that + + $ git diff --cached + + always shows the difference between the HEAD and the index file—this +is what you’d commit if you created the commit now—and that + + $ git diff + + shows the difference between the working tree and the index file. + + Note that ‘git add’ always adds just the current contents of a file +to the index; further changes to the same file will be ignored unless +you run ‘git add’ on the file again. + + When you’re ready, just run + + $ git commit + + and Git will prompt you for a commit message and then create the new +commit. Check to make sure it looks like what you expected with + + $ git show + + As a special shortcut, + + $ git commit -a + + will update the index with any files that you’ve modified or removed +and create a commit, all in one step. + + A number of commands are useful for keeping track of what you’re +about to commit: + + $ git diff --cached # difference between HEAD and the index; what + # would be committed if you ran "commit" now. + $ git diff # difference between the index file and your + # working directory; changes that would not + # be included if you ran "commit" now. + $ git diff HEAD # difference between HEAD and working tree; what + # would be committed if you ran "commit -a" now. + $ git status # a brief per-file summary of the above. + + You can also use git-gui(1) (git-gui.html) to create commits, view +changes in the index and the working tree files, and individually select +diff hunks for inclusion in the index (by right-clicking on the diff +hunk and choosing "Stage Hunk For Commit"). + + +File: git.info, Node: Creating good commit messages, Next: Ignoring files, Prev: How to make a commit, Up: Developing with Git + +3.4 Creating good commit messages +================================= + +Though not required, it’s a good idea to begin the commit message with a +single short (less than 50 character) line summarizing the change, +followed by a blank line and then a more thorough description. The text +up to the first blank line in a commit message is treated as the commit +title, and that title is used throughout Git. For example, +git-format-patch(1) (git-format-patch.html) turns a commit into email, +and it uses the title on the Subject line and the rest of the commit in +the body. + + +File: git.info, Node: Ignoring files, Next: How to merge, Prev: Creating good commit messages, Up: Developing with Git + +3.5 Ignoring files +================== + +A project will often generate files that you do _not_ want to track with +Git. This typically includes files generated by a build process or +temporary backup files made by your editor. Of course, _not_ tracking +files with Git is just a matter of _not_ calling ‘git add’ on them. But +it quickly becomes annoying to have these untracked files lying around; +e.g. they make ‘git add .’ practically useless, and they keep showing +up in the output of ‘git status’. + + You can tell Git to ignore certain files by creating a file called +‘.gitignore’ in the top level of your working directory, with contents +such as: + + # Lines starting with '#' are considered comments. + # Ignore any file named foo.txt. + foo.txt + # Ignore (generated) html files, + *.html + # except foo.html which is maintained by hand. + !foo.html + # Ignore objects and archives. + *.[oa] + + See gitignore(5) (gitignore.html) for a detailed explanation of the +syntax. You can also place .gitignore files in other directories in +your working tree, and they will apply to those directories and their +subdirectories. The ‘.gitignore’ files can be added to your repository +like any other files (just run ‘git add .gitignore’ and ‘git commit’, as +usual), which is convenient when the exclude patterns (such as patterns +matching build output files) would also make sense for other users who +clone your repository. + + If you wish the exclude patterns to affect only certain repositories +(instead of every repository for a given project), you may instead put +them in a file in your repository named ‘.git/info/exclude’, or in any +file specified by the ‘core.excludesFile’ configuration variable. Some +Git commands can also take exclude patterns directly on the command +line. See gitignore(5) (gitignore.html) for the details. + + +File: git.info, Node: How to merge, Next: Resolving a merge, Prev: Ignoring files, Up: Developing with Git + +3.6 How to merge +================ + +You can rejoin two diverging branches of development using git-merge(1) +(git-merge.html): + + $ git merge branchname + + merges the development in the branch ‘branchname’ into the current +branch. + + A merge is made by combining the changes made in ‘branchname’ and the +changes made up to the latest commit in your current branch since their +histories forked. The work tree is overwritten by the result of the +merge when this combining is done cleanly, or overwritten by a +half-merged results when this combining results in conflicts. +Therefore, if you have uncommitted changes touching the same files as +the ones impacted by the merge, Git will refuse to proceed. Most of the +time, you will want to commit your changes before you can merge, and if +you don’t, then git-stash(1) (git-stash.html) can take these changes +away while you’re doing the merge, and reapply them afterwards. + + If the changes are independent enough, Git will automatically +complete the merge and commit the result (or reuse an existing commit in +case of fast-forward (*note Fast-forward merges::), see below). On the +other hand, if there are conflicts—for example, if the same file is +modified in two different ways in the remote branch and the local +branch—then you are warned; the output may look something like this: + + $ git merge next + 100% (4/4) done + Auto-merged file.txt + CONFLICT (content): Merge conflict in file.txt + Automatic merge failed; fix conflicts and then commit the result. + + Conflict markers are left in the problematic files, and after you +resolve the conflicts manually, you can update the index with the +contents and run Git commit, as you normally would when creating a new +file. + + If you examine the resulting commit using gitk, you will see that it +has two parents, one pointing to the top of the current branch, and one +to the top of the other branch. + + +File: git.info, Node: Resolving a merge, Next: Undoing a merge, Prev: How to merge, Up: Developing with Git + +3.7 Resolving a merge +===================== + +When a merge isn’t resolved automatically, Git leaves the index and the +working tree in a special state that gives you all the information you +need to help resolve the merge. + + Files with conflicts are marked specially in the index, so until you +resolve the problem and update the index, git-commit(1) +(git-commit.html) will fail: + + $ git commit + file.txt: needs merge + + Also, git-status(1) (git-status.html) will list those files as +"unmerged", and the files with conflicts will have conflict markers +added, like this: + + <<<<<<< HEAD:file.txt + Hello world + ======= + Goodbye + >>>>>>> 77976da35a11db4580b80ae27e8d65caf5208086:file.txt + + All you need to do is edit the files to resolve the conflicts, and +then + + $ git add file.txt + $ git commit + + Note that the commit message will already be filled in for you with +some information about the merge. Normally you can just use this +default message unchanged, but you may add additional commentary of your +own if desired. + + The above is all you need to know to resolve a simple merge. But Git +also provides more information to help resolve conflicts: + +* Menu: + +* Getting conflict-resolution help during a merge:: + + +File: git.info, Node: Getting conflict-resolution help during a merge, Up: Resolving a merge + +3.7.1 Getting conflict-resolution help during a merge +----------------------------------------------------- + +All of the changes that Git was able to merge automatically are already +added to the index file, so git-diff(1) (git-diff.html) shows only the +conflicts. It uses an unusual syntax: + + $ git diff + diff --cc file.txt + index 802992c,2b60207..0000000 + --- a/file.txt + +++ b/file.txt + @@@ -1,1 -1,1 +1,5 @@@ + ++<<<<<<< HEAD:file.txt + +Hello world + ++======= + + Goodbye + ++>>>>>>> 77976da35a11db4580b80ae27e8d65caf5208086:file.txt + + Recall that the commit which will be committed after we resolve this +conflict will have two parents instead of the usual one: one parent will +be HEAD, the tip of the current branch; the other will be the tip of the +other branch, which is stored temporarily in MERGE_HEAD. + + During the merge, the index holds three versions of each file. Each +of these three "file stages" represents a different version of the file: + + $ git show :1:file.txt # the file in a common ancestor of both branches + $ git show :2:file.txt # the version from HEAD. + $ git show :3:file.txt # the version from MERGE_HEAD. + + When you ask git-diff(1) (git-diff.html) to show the conflicts, it +runs a three-way diff between the conflicted merge results in the work +tree with stages 2 and 3 to show only hunks whose contents come from +both sides, mixed (in other words, when a hunk’s merge results come only +from stage 2, that part is not conflicting and is not shown. Same for +stage 3). + + The diff above shows the differences between the working-tree version +of file.txt and the stage 2 and stage 3 versions. So instead of +preceding each line by a single ‘+’ or ‘-’, it now uses two columns: the +first column is used for differences between the first parent and the +working directory copy, and the second for differences between the +second parent and the working directory copy. (See the "COMBINED DIFF +FORMAT" section of git-diff-files(1) (git-diff-files.html) for a details +of the format.) + + After resolving the conflict in the obvious way (but before updating +the index), the diff will look like: + + $ git diff + diff --cc file.txt + index 802992c,2b60207..0000000 + --- a/file.txt + +++ b/file.txt + @@@ -1,1 -1,1 +1,1 @@@ + - Hello world + -Goodbye + ++Goodbye world + + This shows that our resolved version deleted "Hello world" from the +first parent, deleted "Goodbye" from the second parent, and added +"Goodbye world", which was previously absent from both. + + Some special diff options allow diffing the working directory against +any of these stages: + + $ git diff -1 file.txt # diff against stage 1 + $ git diff --base file.txt # same as the above + $ git diff -2 file.txt # diff against stage 2 + $ git diff --ours file.txt # same as the above + $ git diff -3 file.txt # diff against stage 3 + $ git diff --theirs file.txt # same as the above. + + The git-log(1) (git-log.html) and gitk(1) (gitk.html) commands also +provide special help for merges: + + $ git log --merge + $ gitk --merge + + These will display all commits which exist only on HEAD or on +MERGE_HEAD, and which touch an unmerged file. + + You may also use git-mergetool(1) (git-mergetool.html), which lets +you merge the unmerged files using external tools such as Emacs or +kdiff3. + + Each time you resolve the conflicts in a file and update the index: + + $ git add file.txt + + the different stages of that file will be "collapsed", after which +‘git diff’ will (by default) no longer show diffs for that file. + + +File: git.info, Node: Undoing a merge, Next: Fast-forward merges, Prev: Resolving a merge, Up: Developing with Git + +3.8 Undoing a merge +=================== + +If you get stuck and decide to just give up and throw the whole mess +away, you can always return to the pre-merge state with + + $ git merge --abort + + Or, if you’ve already committed the merge that you want to throw +away, + + $ git reset --hard ORIG_HEAD + + However, this last command can be dangerous in some cases—never throw +away a commit you have already committed if that commit may itself have +been merged into another branch, as doing so may confuse further merges. + + +File: git.info, Node: Fast-forward merges, Next: Fixing mistakes, Prev: Undoing a merge, Up: Developing with Git + +3.9 Fast-forward merges +======================= + +There is one special case not mentioned above, which is treated +differently. Normally, a merge results in a merge commit, with two +parents, one pointing at each of the two lines of development that were +merged. + + However, if the current branch is an ancestor of the other—so every +commit present in the current branch is already contained in the other +branch—then Git just performs a "fast-forward"; the head of the current +branch is moved forward to point at the head of the merged-in branch, +without any new commits being created. + + +File: git.info, Node: Fixing mistakes, Next: Ensuring good performance, Prev: Fast-forward merges, Up: Developing with Git + +3.10 Fixing mistakes +==================== + +If you’ve messed up the working tree, but haven’t yet committed your +mistake, you can return the entire working tree to the last committed +state with + + $ git restore --staged --worktree :/ + + If you make a commit that you later wish you hadn’t, there are two +fundamentally different ways to fix the problem: + + 1. You can create a new commit that undoes whatever was done by the + old commit. This is the correct thing if your mistake has already + been made public. + + 2. You can go back and modify the old commit. You should never do + this if you have already made the history public; Git does not + normally expect the "history" of a project to change, and cannot + correctly perform repeated merges from a branch that has had its + history changed. + +* Menu: + +* Fixing a mistake with a new commit:: +* Fixing a mistake by rewriting history:: +* Checking out an old version of a file:: +* Temporarily setting aside work in progress:: + + +File: git.info, Node: Fixing a mistake with a new commit, Next: Fixing a mistake by rewriting history, Up: Fixing mistakes + +3.10.1 Fixing a mistake with a new commit +----------------------------------------- + +Creating a new commit that reverts an earlier change is very easy; just +pass the git-revert(1) (git-revert.html) command a reference to the bad +commit; for example, to revert the most recent commit: + + $ git revert HEAD + + This will create a new commit which undoes the change in HEAD. You +will be given a chance to edit the commit message for the new commit. + + You can also revert an earlier change, for example, the next-to-last: + + $ git revert HEAD^ + + In this case Git will attempt to undo the old change while leaving +intact any changes made since then. If more recent changes overlap with +the changes to be reverted, then you will be asked to fix conflicts +manually, just as in the case of resolving a merge (*note Resolving a +merge::). + + +File: git.info, Node: Fixing a mistake by rewriting history, Next: Checking out an old version of a file, Prev: Fixing a mistake with a new commit, Up: Fixing mistakes + +3.10.2 Fixing a mistake by rewriting history +-------------------------------------------- + +If the problematic commit is the most recent commit, and you have not +yet made that commit public, then you may just destroy it using ‘git +reset’ (*note Undoing a merge::). + + Alternatively, you can edit the working directory and update the +index to fix your mistake, just as if you were going to create a new +commit (*note How to make a commit::), then run + + $ git commit --amend + + which will replace the old commit by a new commit incorporating your +changes, giving you a chance to edit the old commit message first. + + Again, you should never do this to a commit that may already have +been merged into another branch; use git-revert(1) (git-revert.html) +instead in that case. + + It is also possible to replace commits further back in the history, +but this is an advanced topic to be left for another chapter (*note +Rewriting history and maintaining patch series::). + + +File: git.info, Node: Checking out an old version of a file, Next: Temporarily setting aside work in progress, Prev: Fixing a mistake by rewriting history, Up: Fixing mistakes + +3.10.3 Checking out an old version of a file +-------------------------------------------- + +In the process of undoing a previous bad change, you may find it useful +to check out an older version of a particular file using git-restore(1) +(git-restore.html). The command + + $ git restore --source=HEAD^ path/to/file + + replaces path/to/file by the contents it had in the commit HEAD^, and +also updates the index to match. It does not change branches. + + If you just want to look at an old version of the file, without +modifying the working directory, you can do that with git-show(1) +(git-show.html): + + $ git show HEAD^:path/to/file + + which will display the given version of the file. + + +File: git.info, Node: Temporarily setting aside work in progress, Prev: Checking out an old version of a file, Up: Fixing mistakes + +3.10.4 Temporarily setting aside work in progress +------------------------------------------------- + +While you are in the middle of working on something complicated, you +find an unrelated but obvious and trivial bug. You would like to fix it +before continuing. You can use git-stash(1) (git-stash.html) to save +the current state of your work, and after fixing the bug (or, optionally +after doing so on a different branch and then coming back), unstash the +work-in-progress changes. + + $ git stash push -m "work in progress for foo feature" + + This command will save your changes away to the ‘stash’, and reset +your working tree and the index to match the tip of your current branch. +Then you can make your fix as usual. + + ... edit and test ... + $ git commit -a -m "blorpl: typofix" + + After that, you can go back to what you were working on with ‘git +stash pop’: + + $ git stash pop + + +File: git.info, Node: Ensuring good performance, Next: Ensuring reliability, Prev: Fixing mistakes, Up: Developing with Git + +3.11 Ensuring good performance +============================== + +On large repositories, Git depends on compression to keep the history +information from taking up too much space on disk or in memory. Some +Git commands may automatically run git-gc(1) (git-gc.html), so you don’t +have to worry about running it manually. However, compressing a large +repository may take a while, so you may want to call ‘gc’ explicitly to +avoid automatic compression kicking in when it is not convenient. + + +File: git.info, Node: Ensuring reliability, Prev: Ensuring good performance, Up: Developing with Git + +3.12 Ensuring reliability +========================= + +* Menu: + +* Checking the repository for corruption:: +* Recovering lost changes:: + + +File: git.info, Node: Checking the repository for corruption, Next: Recovering lost changes, Up: Ensuring reliability + +3.12.1 Checking the repository for corruption +--------------------------------------------- + +The git-fsck(1) (git-fsck.html) command runs a number of +self-consistency checks on the repository, and reports on any problems. +This may take some time. + + $ git fsck + dangling commit 7281251ddd2a61e38657c827739c57015671a6b3 + dangling commit 2706a059f258c6b245f298dc4ff2ccd30ec21a63 + dangling commit 13472b7c4b80851a1bc551779171dcb03655e9b5 + dangling blob 218761f9d90712d37a9c5e36f406f92202db07eb + dangling commit bf093535a34a4d35731aa2bd90fe6b176302f14f + dangling commit 8e4bec7f2ddaa268bef999853c25755452100f8e + dangling tree d50bb86186bf27b681d25af89d3b5b68382e4085 + dangling tree b24c2473f1fd3d91352a624795be026d64c8841f + ... + + You will see informational messages on dangling objects. They are +objects that still exist in the repository but are no longer referenced +by any of your branches, and can (and will) be removed after a while +with ‘gc’. You can run ‘git fsck --no-dangling’ to suppress these +messages, and still view real errors. + + +File: git.info, Node: Recovering lost changes, Prev: Checking the repository for corruption, Up: Ensuring reliability + +3.12.2 Recovering lost changes +------------------------------ + +* Menu: + +* Reflogs:: +* Examining dangling objects:: + + +File: git.info, Node: Reflogs, Next: Examining dangling objects, Up: Recovering lost changes + +3.12.2.1 Reflogs +................ + +Say you modify a branch with ‘git reset --hard’ (*note Fixing +mistakes::), and then realize that the branch was the only reference you +had to that point in history. + + Fortunately, Git also keeps a log, called a "reflog", of all the +previous values of each branch. So in this case you can still find the +old history using, for example, + + $ git log master@{1} + + This lists the commits reachable from the previous version of the +‘master’ branch head. This syntax can be used with any Git command that +accepts a commit, not just with ‘git log’. Some other examples: + + $ git show master@{2} # See where the branch pointed 2, + $ git show master@{3} # 3, ... changes ago. + $ gitk master@{yesterday} # See where it pointed yesterday, + $ gitk master@{"1 week ago"} # ... or last week + $ git log --walk-reflogs master # show reflog entries for master + + A separate reflog is kept for the HEAD, so + + $ git show HEAD@{"1 week ago"} + + will show what HEAD pointed to one week ago, not what the current +branch pointed to one week ago. This allows you to see the history of +what you’ve checked out. + + The reflogs are kept by default for 30 days, after which they may be +pruned. See git-reflog(1) (git-reflog.html) and git-gc(1) (git-gc.html) +to learn how to control this pruning, and see the "SPECIFYING REVISIONS" +section of gitrevisions(7) (gitrevisions.html) for details. + + Note that the reflog history is very different from normal Git +history. While normal history is shared by every repository that works +on the same project, the reflog history is not shared: it tells you only +about how the branches in your local repository have changed over time. + + +File: git.info, Node: Examining dangling objects, Prev: Reflogs, Up: Recovering lost changes + +3.12.2.2 Examining dangling objects +................................... + +In some situations the reflog may not be able to save you. For example, +suppose you delete a branch, then realize you need the history it +contained. The reflog is also deleted; however, if you have not yet +pruned the repository, then you may still be able to find the lost +commits in the dangling objects that ‘git fsck’ reports. See *note +Dangling objects:: for the details. + + $ git fsck + dangling commit 7281251ddd2a61e38657c827739c57015671a6b3 + dangling commit 2706a059f258c6b245f298dc4ff2ccd30ec21a63 + dangling commit 13472b7c4b80851a1bc551779171dcb03655e9b5 + ... + + You can examine one of those dangling commits with, for example, + + $ gitk 7281251ddd --not --all + + which does what it sounds like: it says that you want to see the +commit history that is described by the dangling commit(s), but not the +history that is described by all your existing branches and tags. Thus +you get exactly the history reachable from that commit that is lost. +(And notice that it might not be just one commit: we only report the +"tip of the line" as being dangling, but there might be a whole deep and +complex commit history that was dropped.) + + If you decide you want the history back, you can always create a new +reference pointing to it, for example, a new branch: + + $ git branch recovered-branch 7281251ddd + + Other types of dangling objects (blobs and trees) are also possible, +and dangling objects can arise in other situations. + + +File: git.info, Node: Sharing development with others, Next: Rewriting history and maintaining patch series, Prev: Developing with Git, Up: Top + +4 Sharing development with others +********************************* + +* Menu: + +* Getting updates with git pull:: +* Submitting patches to a project:: +* Importing patches to a project:: +* Public Git repositories:: +* How to get a Git repository with minimal history:: +* Examples: Examples <1>. + + +File: git.info, Node: Getting updates with git pull, Next: Submitting patches to a project, Up: Sharing development with others + +4.1 Getting updates with git pull +================================= + +After you clone a repository and commit a few changes of your own, you +may wish to check the original repository for updates and merge them +into your own work. + + We have already seen how to keep remote-tracking branches up to date +(*note Updating a repository with git fetch::) with git-fetch(1) +(git-fetch.html), and how to merge two branches. So you can merge in +changes from the original repository’s master branch with: + + $ git fetch + $ git merge origin/master + + However, the git-pull(1) (git-pull.html) command provides a way to do +this in one step: + + $ git pull origin master + + In fact, if you have ‘master’ checked out, then this branch has been +configured by ‘git clone’ to get changes from the HEAD branch of the +origin repository. So often you can accomplish the above with just a +simple + + $ git pull + + This command will fetch changes from the remote branches to your +remote-tracking branches ‘origin/*’, and merge the default branch into +the current branch. + + More generally, a branch that is created from a remote-tracking +branch will pull by default from that branch. See the descriptions of +the ‘branch..remote’ and ‘branch..merge’ options in +git-config(1) (git-config.html), and the discussion of the ‘--track’ +option in git-checkout(1) (git-checkout.html), to learn how to control +these defaults. + + In addition to saving you keystrokes, ‘git pull’ also helps you by +producing a default commit message documenting the branch and repository +that you pulled from. + + (But note that no such commit will be created in the case of a +fast-forward (*note Fast-forward merges::); instead, your branch will +just be updated to point to the latest commit from the upstream branch.) + + The ‘git pull’ command can also be given ‘.’ as the "remote" +repository, in which case it just merges in a branch from the current +repository; so the commands + + $ git pull . branch + $ git merge branch + + are roughly equivalent. + + +File: git.info, Node: Submitting patches to a project, Next: Importing patches to a project, Prev: Getting updates with git pull, Up: Sharing development with others + +4.2 Submitting patches to a project +=================================== + +If you just have a few changes, the simplest way to submit them may just +be to send them as patches in email: + + First, use git-format-patch(1) (git-format-patch.html); for example: + + $ git format-patch origin + + will produce a numbered series of files in the current directory, one +for each patch in the current branch but not in ‘origin/HEAD’. + + ‘git format-patch’ can include an initial "cover letter". You can +insert commentary on individual patches after the three dash line which +‘format-patch’ places after the commit message but before the patch +itself. If you use ‘git notes’ to track your cover letter material, +‘git format-patch --notes’ will include the commit’s notes in a similar +manner. + + You can then import these into your mail client and send them by +hand. However, if you have a lot to send at once, you may prefer to use +the git-send-email(1) (git-send-email.html) script to automate the +process. Consult the mailing list for your project first to determine +their requirements for submitting patches. + + +File: git.info, Node: Importing patches to a project, Next: Public Git repositories, Prev: Submitting patches to a project, Up: Sharing development with others + +4.3 Importing patches to a project +================================== + +Git also provides a tool called git-am(1) (git-am.html) (am stands for +"apply mailbox"), for importing such an emailed series of patches. Just +save all of the patch-containing messages, in order, into a single +mailbox file, say ‘patches.mbox’, then run + + $ git am -3 patches.mbox + + Git will apply each patch in order; if any conflicts are found, it +will stop, and you can fix the conflicts as described in "Resolving a +merge (*note Resolving a merge::)". (The ‘-3’ option tells Git to +perform a merge; if you would prefer it just to abort and leave your +tree and index untouched, you may omit that option.) + + Once the index is updated with the results of the conflict +resolution, instead of creating a new commit, just run + + $ git am --continue + + and Git will create the commit for you and continue applying the +remaining patches from the mailbox. + + The final result will be a series of commits, one for each patch in +the original mailbox, with authorship and commit log message each taken +from the message containing each patch. + + +File: git.info, Node: Public Git repositories, Next: How to get a Git repository with minimal history, Prev: Importing patches to a project, Up: Sharing development with others + +4.4 Public Git repositories +=========================== + +Another way to submit changes to a project is to tell the maintainer of +that project to pull the changes from your repository using git-pull(1) +(git-pull.html). In the section "Getting updates with ‘git pull’ (*note +Getting updates with git pull::)" we described this as a way to get +updates from the "main" repository, but it works just as well in the +other direction. + + If you and the maintainer both have accounts on the same machine, +then you can just pull changes from each other’s repositories directly; +commands that accept repository URLs as arguments will also accept a +local directory name: + + $ git clone /path/to/repository + $ git pull /path/to/other/repository + + or an ssh URL: + + $ git clone ssh://yourhost/~you/repository + + For projects with few developers, or for synchronizing a few private +repositories, this may be all you need. + + However, the more common way to do this is to maintain a separate +public repository (usually on a different host) for others to pull +changes from. This is usually more convenient, and allows you to +cleanly separate private work in progress from publicly visible work. + + You will continue to do your day-to-day work in your personal +repository, but periodically "push" changes from your personal +repository into your public repository, allowing other developers to +pull from that repository. So the flow of changes, in a situation where +there is one other developer with a public repository, looks like this: + + you push + your personal repo ------------------> your public repo + ^ | + | | + | you pull | they pull + | | + | | + | they push V + their public repo <------------------- their repo + + We explain how to do this in the following sections. + +* Menu: + +* Setting up a public repository:: +* Exporting a Git repository via the Git protocol:: +* Exporting a git repository via HTTP:: +* Pushing changes to a public repository:: +* What to do when a push fails:: +* Setting up a shared repository:: +* Allowing web browsing of a repository:: + + +File: git.info, Node: Setting up a public repository, Next: Exporting a Git repository via the Git protocol, Up: Public Git repositories + +4.4.1 Setting up a public repository +------------------------------------ + +Assume your personal repository is in the directory ‘~/proj’. We first +create a new clone of the repository and tell ‘git daemon’ that it is +meant to be public: + + $ git clone --bare ~/proj proj.git + $ touch proj.git/git-daemon-export-ok + + The resulting directory proj.git contains a "bare" git repository—it +is just the contents of the ‘.git’ directory, without any files checked +out around it. + + Next, copy ‘proj.git’ to the server where you plan to host the public +repository. You can use scp, rsync, or whatever is most convenient. + + +File: git.info, Node: Exporting a Git repository via the Git protocol, Next: Exporting a git repository via HTTP, Prev: Setting up a public repository, Up: Public Git repositories + +4.4.2 Exporting a Git repository via the Git protocol +----------------------------------------------------- + +This is the preferred method. + + If someone else administers the server, they should tell you what +directory to put the repository in, and what ‘git://’ URL it will appear +at. You can then skip to the section "Pushing changes to a public +repository (*note Pushing changes to a public repository::)", below. + + Otherwise, all you need to do is start git-daemon(1) +(git-daemon.html); it will listen on port 9418. By default, it will +allow access to any directory that looks like a Git directory and +contains the magic file git-daemon-export-ok. Passing some directory +paths as ‘git daemon’ arguments will further restrict the exports to +those paths. + + You can also run ‘git daemon’ as an inetd service; see the +git-daemon(1) (git-daemon.html) man page for details. (See especially +the examples section.) + + +File: git.info, Node: Exporting a git repository via HTTP, Next: Pushing changes to a public repository, Prev: Exporting a Git repository via the Git protocol, Up: Public Git repositories + +4.4.3 Exporting a git repository via HTTP +----------------------------------------- + +The Git protocol gives better performance and reliability, but on a host +with a web server set up, HTTP exports may be simpler to set up. + + All you need to do is place the newly created bare Git repository in +a directory that is exported by the web server, and make some +adjustments to give web clients some extra information they need: + + $ mv proj.git /home/you/public_html/proj.git + $ cd proj.git + $ git --bare update-server-info + $ mv hooks/post-update.sample hooks/post-update + + (For an explanation of the last two lines, see +git-update-server-info(1) (git-update-server-info.html) and githooks(5) +(githooks.html).) + + Advertise the URL of ‘proj.git’. Anybody else should then be able to +clone or pull from that URL, for example with a command line like: + + $ git clone http://yourserver.com/~you/proj.git + + (See also setup-git-server-over-http +(howto/setup-git-server-over-http.html) for a slightly more +sophisticated setup using WebDAV which also allows pushing over HTTP.) + + +File: git.info, Node: Pushing changes to a public repository, Next: What to do when a push fails, Prev: Exporting a git repository via HTTP, Up: Public Git repositories + +4.4.4 Pushing changes to a public repository +-------------------------------------------- + +Note that the two techniques outlined above (exporting via http (*note +Exporting a git repository via HTTP::) or git (*note Exporting a Git +repository via the Git protocol::)) allow other maintainers to fetch +your latest changes, but they do not allow write access, which you will +need to update the public repository with the latest changes created in +your private repository. + + The simplest way to do this is using git-push(1) (git-push.html) and +ssh; to update the remote branch named ‘master’ with the latest state of +your branch named ‘master’, run + + $ git push ssh://yourserver.com/~you/proj.git master:master + + or just + + $ git push ssh://yourserver.com/~you/proj.git master + + As with ‘git fetch’, ‘git push’ will complain if this does not result +in a fast-forward (*note Fast-forward merges::); see the following +section for details on handling this case. + + Note that the target of a ‘push’ is normally a bare (*note +[def_bare_repository]::) repository. You can also push to a repository +that has a checked-out working tree, but a push to update the currently +checked-out branch is denied by default to prevent confusion. See the +description of the receive.denyCurrentBranch option in git-config(1) +(git-config.html) for details. + + As with ‘git fetch’, you may also set up configuration options to +save typing; so, for example: + + $ git remote add public-repo ssh://yourserver.com/~you/proj.git + + adds the following to ‘.git/config’: + + [remote "public-repo"] + url = yourserver.com:proj.git + fetch = +refs/heads/*:refs/remotes/example/* + + which lets you do the same push with just + + $ git push public-repo master + + See the explanations of the ‘remote..url’, +‘branch..remote’, and ‘remote..push’ options in +git-config(1) (git-config.html) for details. + + +File: git.info, Node: What to do when a push fails, Next: Setting up a shared repository, Prev: Pushing changes to a public repository, Up: Public Git repositories + +4.4.5 What to do when a push fails +---------------------------------- + +If a push would not result in a fast-forward (*note Fast-forward +merges::) of the remote branch, then it will fail with an error like: + + ! [rejected] master -> master (non-fast-forward) + error: failed to push some refs to '...' + hint: Updates were rejected because the tip of your current branch is behind + hint: its remote counterpart. Integrate the remote changes (e.g. + hint: 'git pull ...') before pushing again. + hint: See the 'Note about fast-forwards' in 'git push --help' for details. + + This can happen, for example, if you: + + • use ‘git reset --hard’ to remove already-published commits, or + + • use ‘git commit --amend’ to replace already-published commits (as + in *note Fixing a mistake by rewriting history::), or + + • use ‘git rebase’ to rebase any already-published commits (as in + *note Keeping a patch series up to date using git rebase::). + + You may force ‘git push’ to perform the update anyway by preceding +the branch name with a plus sign: + + $ git push ssh://yourserver.com/~you/proj.git +master + + Note the addition of the ‘+’ sign. Alternatively, you can use the +‘-f’ flag to force the remote update, as in: + + $ git push -f ssh://yourserver.com/~you/proj.git master + + Normally whenever a branch head in a public repository is modified, +it is modified to point to a descendant of the commit that it pointed to +before. By forcing a push in this situation, you break that convention. +(See *note Problems with rewriting history::.) + + Nevertheless, this is a common practice for people that need a simple +way to publish a work-in-progress patch series, and it is an acceptable +compromise as long as you warn other developers that this is how you +intend to manage the branch. + + It’s also possible for a push to fail in this way when other people +have the right to push to the same repository. In that case, the +correct solution is to retry the push after first updating your work: +either by a pull, or by a fetch followed by a rebase; see the next +section (*note Setting up a shared repository::) and gitcvs-migration(7) +(gitcvs-migration.html) for more. + + +File: git.info, Node: Setting up a shared repository, Next: Allowing web browsing of a repository, Prev: What to do when a push fails, Up: Public Git repositories + +4.4.6 Setting up a shared repository +------------------------------------ + +Another way to collaborate is by using a model similar to that commonly +used in CVS, where several developers with special rights all push to +and pull from a single shared repository. See gitcvs-migration(7) +(gitcvs-migration.html) for instructions on how to set this up. + + However, while there is nothing wrong with Git’s support for shared +repositories, this mode of operation is not generally recommended, +simply because the mode of collaboration that Git supports—by exchanging +patches and pulling from public repositories—has so many advantages over +the central shared repository: + + • Git’s ability to quickly import and merge patches allows a single + maintainer to process incoming changes even at very high rates. + And when that becomes too much, ‘git pull’ provides an easy way for + that maintainer to delegate this job to other maintainers while + still allowing optional review of incoming changes. + + • Since every developer’s repository has the same complete copy of + the project history, no repository is special, and it is trivial + for another developer to take over maintenance of a project, either + by mutual agreement, or because a maintainer becomes unresponsive + or difficult to work with. + + • The lack of a central group of "committers" means there is less + need for formal decisions about who is "in" and who is "out". + + +File: git.info, Node: Allowing web browsing of a repository, Prev: Setting up a shared repository, Up: Public Git repositories + +4.4.7 Allowing web browsing of a repository +------------------------------------------- + +The gitweb cgi script provides users an easy way to browse your +project’s revisions, file contents and logs without having to install +Git. Features like RSS/Atom feeds and blame/annotation details may +optionally be enabled. + + The git-instaweb(1) (git-instaweb.html) command provides a simple way +to start browsing the repository using gitweb. The default server when +using instaweb is lighttpd. + + See the file gitweb/INSTALL in the Git source tree and gitweb(1) +(gitweb.html) for instructions on details setting up a permanent +installation with a CGI or Perl capable server. + + +File: git.info, Node: How to get a Git repository with minimal history, Next: Examples <1>, Prev: Public Git repositories, Up: Sharing development with others + +4.5 How to get a Git repository with minimal history +==================================================== + +A shallow clone (*note [def_shallow_clone]::), with its truncated +history, is useful when one is interested only in recent history of a +project and getting full history from the upstream is expensive. + + A shallow clone (*note [def_shallow_clone]::) is created by +specifying the git-clone(1) (git-clone.html) ‘--depth’ switch. The +depth can later be changed with the git-fetch(1) (git-fetch.html) +‘--depth’ switch, or full history restored with ‘--unshallow’. + + Merging inside a shallow clone (*note [def_shallow_clone]::) will +work as long as a merge base is in the recent history. Otherwise, it +will be like merging unrelated histories and may have to result in huge +conflicts. This limitation may make such a repository unsuitable to be +used in merge based workflows. + + +File: git.info, Node: Examples <1>, Prev: How to get a Git repository with minimal history, Up: Sharing development with others + +4.6 Examples +============ + +* Menu: + +* Maintaining topic branches for a Linux subsystem maintainer:: + + +File: git.info, Node: Maintaining topic branches for a Linux subsystem maintainer, Up: Examples <1> + +4.6.1 Maintaining topic branches for a Linux subsystem maintainer +----------------------------------------------------------------- + +This describes how Tony Luck uses Git in his role as maintainer of the +IA64 architecture for the Linux kernel. + + He uses two public branches: + + • A "test" tree into which patches are initially placed so that they + can get some exposure when integrated with other ongoing + development. This tree is available to Andrew for pulling into -mm + whenever he wants. + + • A "release" tree into which tested patches are moved for final + sanity checking, and as a vehicle to send them upstream to Linus + (by sending him a "please pull" request.) + + He also uses a set of temporary branches ("topic branches"), each +containing a logical grouping of patches. + + To set this up, first create your work tree by cloning Linus’s public +tree: + + $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git work + $ cd work + + Linus’s tree will be stored in the remote-tracking branch named +origin/master, and can be updated using git-fetch(1) (git-fetch.html); +you can track other public trees using git-remote(1) (git-remote.html) +to set up a "remote" and git-fetch(1) (git-fetch.html) to keep them up +to date; see *note Repositories and Branches::. + + Now create the branches in which you are going to work; these start +out at the current tip of origin/master branch, and should be set up +(using the ‘--track’ option to git-branch(1) (git-branch.html)) to merge +changes in from Linus by default. + + $ git branch --track test origin/master + $ git branch --track release origin/master + + These can be easily kept up to date using git-pull(1) +(git-pull.html). + + $ git switch test && git pull + $ git switch release && git pull + + Important note! If you have any local changes in these branches, +then this merge will create a commit object in the history (with no +local changes Git will simply do a "fast-forward" merge). Many people +dislike the "noise" that this creates in the Linux history, so you +should avoid doing this capriciously in the ‘release’ branch, as these +noisy commits will become part of the permanent history when you ask +Linus to pull from the release branch. + + A few configuration variables (see git-config(1) (git-config.html)) +can make it easy to push both branches to your public tree. (See *note +Setting up a public repository::.) + + $ cat >> .git/config <&2 + exit 1 + ;; + esac + + ==== merge script ==== + # Merge a branch into either the test or release branch + + pname=$0 + + usage() + { + echo "usage: $pname branch test|release" 1>&2 + exit 1 + } + + git show-ref -q --verify -- refs/heads/"$1" || { + echo "Can't see branch <$1>" 1>&2 + usage + } + + case "$2" in + test|release) + if [ $(git log $2..$1 | wc -c) -eq 0 ] + then + echo $1 already merged into $2 1>&2 + exit 1 + fi + git checkout $2 && git pull . $1 + ;; + *) + usage + ;; + esac + + ==== status script ==== + # report on status of my ia64 Git tree + + gb=$(tput setab 2) + rb=$(tput setab 1) + restore=$(tput setab 9) + + if [ `git rev-list test..release | wc -c` -gt 0 ] + then + echo $rb Warning: commits in release that are not in test $restore + git log test..release + fi + + for branch in `git show-ref --heads | sed 's|^.*/||'` + do + if [ $branch = test -o $branch = release ] + then + continue + fi + + echo -n $gb ======= $branch ====== $restore " " + status= + for ref in test release origin/master + do + if [ `git rev-list $ref..$branch | wc -c` -gt 0 ] + then + status=$status${ref:0:1} + fi + done + case $status in + trl) + echo $rb Need to pull into test $restore + ;; + rl) + echo "In test" + ;; + l) + echo "Waiting for linus" + ;; + "") + echo $rb All done $restore + ;; + *) + echo $rb "<$status>" $restore + ;; + esac + git log origin/master..$branch | git shortlog + done + + +File: git.info, Node: Rewriting history and maintaining patch series, Next: Advanced branch management, Prev: Sharing development with others, Up: Top + +5 Rewriting history and maintaining patch series +************************************************ + +Normally commits are only added to a project, never taken away or +replaced. Git is designed with this assumption, and violating it will +cause Git’s merge machinery (for example) to do the wrong thing. + + However, there is a situation in which it can be useful to violate +this assumption. + +* Menu: + +* Creating the perfect patch series:: +* Keeping a patch series up to date using git rebase:: +* Rewriting a single commit:: +* Reordering or selecting from a patch series:: +* Using interactive rebases:: +* Other tools:: +* Problems with rewriting history:: +* Why bisecting merge commits can be harder than bisecting linear history:: + + +File: git.info, Node: Creating the perfect patch series, Next: Keeping a patch series up to date using git rebase, Up: Rewriting history and maintaining patch series + +5.1 Creating the perfect patch series +===================================== + +Suppose you are a contributor to a large project, and you want to add a +complicated feature, and to present it to the other developers in a way +that makes it easy for them to read your changes, verify that they are +correct, and understand why you made each change. + + If you present all of your changes as a single patch (or commit), +they may find that it is too much to digest all at once. + + If you present them with the entire history of your work, complete +with mistakes, corrections, and dead ends, they may be overwhelmed. + + So the ideal is usually to produce a series of patches such that: + + 1. Each patch can be applied in order. + + 2. Each patch includes a single logical change, together with a + message explaining the change. + + 3. No patch introduces a regression: after applying any initial part + of the series, the resulting project still compiles and works, and + has no bugs that it didn’t have before. + + 4. The complete series produces the same end result as your own + (probably much messier!) development process did. + + We will introduce some tools that can help you do this, explain how +to use them, and then explain some of the problems that can arise +because you are rewriting history. + + +File: git.info, Node: Keeping a patch series up to date using git rebase, Next: Rewriting a single commit, Prev: Creating the perfect patch series, Up: Rewriting history and maintaining patch series + +5.2 Keeping a patch series up to date using git rebase +====================================================== + +Suppose that you create a branch ‘mywork’ on a remote-tracking branch +‘origin’, and create some commits on top of it: + + $ git switch -c mywork origin + $ vi file.txt + $ git commit + $ vi otherfile.txt + $ git commit + ... + + You have performed no merges into mywork, so it is just a simple +linear sequence of patches on top of ‘origin’: + + o--o--O <-- origin + \ + a--b--c <-- mywork + + Some more interesting work has been done in the upstream project, and +‘origin’ has advanced: + + o--o--O--o--o--o <-- origin + \ + a--b--c <-- mywork + + At this point, you could use ‘pull’ to merge your changes back in; +the result would create a new merge commit, like this: + + o--o--O--o--o--o <-- origin + \ \ + a--b--c--m <-- mywork + + However, if you prefer to keep the history in mywork a simple series +of commits without any merges, you may instead choose to use +git-rebase(1) (git-rebase.html): + + $ git switch mywork + $ git rebase origin + + This will remove each of your commits from mywork, temporarily saving +them as patches (in a directory named ‘.git/rebase-apply’), update +mywork to point at the latest version of origin, then apply each of the +saved patches to the new mywork. The result will look like: + + o--o--O--o--o--o <-- origin + \ + a'--b'--c' <-- mywork + + In the process, it may discover conflicts. In that case it will stop +and allow you to fix the conflicts; after fixing conflicts, use ‘git +add’ to update the index with those contents, and then, instead of +running ‘git commit’, just run + + $ git rebase --continue + + and Git will continue applying the rest of the patches. + + At any point you may use the ‘--abort’ option to abort this process +and return mywork to the state it had before you started the rebase: + + $ git rebase --abort + + If you need to reorder or edit a number of commits in a branch, it +may be easier to use ‘git rebase -i’, which allows you to reorder and +squash commits, as well as marking them for individual editing during +the rebase. See *note Using interactive rebases:: for details, and +*note Reordering or selecting from a patch series:: for alternatives. + + +File: git.info, Node: Rewriting a single commit, Next: Reordering or selecting from a patch series, Prev: Keeping a patch series up to date using git rebase, Up: Rewriting history and maintaining patch series + +5.3 Rewriting a single commit +============================= + +We saw in *note Fixing a mistake by rewriting history:: that you can +replace the most recent commit using + + $ git commit --amend + + which will replace the old commit by a new commit incorporating your +changes, giving you a chance to edit the old commit message first. This +is useful for fixing typos in your last commit, or for adjusting the +patch contents of a poorly staged commit. + + If you need to amend commits from deeper in your history, you can use +interactive rebase’s ‘edit’ instruction (*note Using interactive +rebases::). + + +File: git.info, Node: Reordering or selecting from a patch series, Next: Using interactive rebases, Prev: Rewriting a single commit, Up: Rewriting history and maintaining patch series + +5.4 Reordering or selecting from a patch series +=============================================== + +Sometimes you want to edit a commit deeper in your history. One +approach is to use ‘git format-patch’ to create a series of patches and +then reset the state to before the patches: + + $ git format-patch origin + $ git reset --hard origin + + Then modify, reorder, or eliminate patches as needed before applying +them again with git-am(1) (git-am.html): + + $ git am *.patch + + +File: git.info, Node: Using interactive rebases, Next: Other tools, Prev: Reordering or selecting from a patch series, Up: Rewriting history and maintaining patch series + +5.5 Using interactive rebases +============================= + +You can also edit a patch series with an interactive rebase. This is +the same as reordering a patch series using ‘format-patch’ (*note +Reordering or selecting from a patch series::), so use whichever +interface you like best. + + Rebase your current HEAD on the last commit you want to retain as-is. +For example, if you want to reorder the last 5 commits, use: + + $ git rebase -i HEAD~5 + + This will open your editor with a list of steps to be taken to +perform your rebase. + + pick deadbee The oneline of this commit + pick fa1afe1 The oneline of the next commit + ... + + # Rebase c0ffeee..deadbee onto c0ffeee + # + # Commands: + # p, pick = use commit + # r, reword = use commit, but edit the commit message + # e, edit = use commit, but stop for amending + # s, squash = use commit, but meld into previous commit + # f, fixup = like "squash", but discard this commit's log message + # x, exec = run command (the rest of the line) using shell + # + # These lines can be re-ordered; they are executed from top to bottom. + # + # If you remove a line here THAT COMMIT WILL BE LOST. + # + # However, if you remove everything, the rebase will be aborted. + # + # Note that empty commits are commented out + + As explained in the comments, you can reorder commits, squash them +together, edit commit messages, etc. by editing the list. Once you are +satisfied, save the list and close your editor, and the rebase will +begin. + + The rebase will stop where ‘pick’ has been replaced with ‘edit’ or +when a step in the list fails to mechanically resolve conflicts and +needs your help. When you are done editing and/or resolving conflicts +you can continue with ‘git rebase --continue’. If you decide that +things are getting too hairy, you can always bail out with ‘git rebase +--abort’. Even after the rebase is complete, you can still recover the +original branch by using the reflog (*note Reflogs::). + + For a more detailed discussion of the procedure and additional tips, +see the "INTERACTIVE MODE" section of git-rebase(1) (git-rebase.html). + + +File: git.info, Node: Other tools, Next: Problems with rewriting history, Prev: Using interactive rebases, Up: Rewriting history and maintaining patch series + +5.6 Other tools +=============== + +There are numerous other tools, such as StGit, which exist for the +purpose of maintaining a patch series. These are outside of the scope +of this manual. + + +File: git.info, Node: Problems with rewriting history, Next: Why bisecting merge commits can be harder than bisecting linear history, Prev: Other tools, Up: Rewriting history and maintaining patch series + +5.7 Problems with rewriting history +=================================== + +The primary problem with rewriting the history of a branch has to do +with merging. Suppose somebody fetches your branch and merges it into +their branch, with a result something like this: + + o--o--O--o--o--o <-- origin + \ \ + t--t--t--m <-- their branch: + + Then suppose you modify the last three commits: + + o--o--o <-- new head of origin + / + o--o--O--o--o--o <-- old head of origin + + If we examined all this history together in one repository, it will +look like: + + o--o--o <-- new head of origin + / + o--o--O--o--o--o <-- old head of origin + \ \ + t--t--t--m <-- their branch: + + Git has no way of knowing that the new head is an updated version of +the old head; it treats this situation exactly the same as it would if +two developers had independently done the work on the old and new heads +in parallel. At this point, if someone attempts to merge the new head +in to their branch, Git will attempt to merge together the two (old and +new) lines of development, instead of trying to replace the old by the +new. The results are likely to be unexpected. + + You may still choose to publish branches whose history is rewritten, +and it may be useful for others to be able to fetch those branches in +order to examine or test them, but they should not attempt to pull such +branches into their own work. + + For true distributed development that supports proper merging, +published branches should never be rewritten. + + +File: git.info, Node: Why bisecting merge commits can be harder than bisecting linear history, Prev: Problems with rewriting history, Up: Rewriting history and maintaining patch series + +5.8 Why bisecting merge commits can be harder than bisecting linear history +=========================================================================== + +The git-bisect(1) (git-bisect.html) command correctly handles history +that includes merge commits. However, when the commit that it finds is +a merge commit, the user may need to work harder than usual to figure +out why that commit introduced a problem. + + Imagine this history: + + ---Z---o---X---...---o---A---C---D + \ / + o---o---Y---...---o---B + + Suppose that on the upper line of development, the meaning of one of +the functions that exists at Z is changed at commit X. The commits from +Z leading to A change both the function’s implementation and all calling +sites that exist at Z, as well as new calling sites they add, to be +consistent. There is no bug at A. + + Suppose that in the meantime on the lower line of development +somebody adds a new calling site for that function at commit Y. The +commits from Z leading to B all assume the old semantics of that +function and the callers and the callee are consistent with each other. +There is no bug at B, either. + + Suppose further that the two development lines merge cleanly at C, so +no conflict resolution is required. + + Nevertheless, the code at C is broken, because the callers added on +the lower line of development have not been converted to the new +semantics introduced on the upper line of development. So if all you +know is that D is bad, that Z is good, and that git-bisect(1) +(git-bisect.html) identifies C as the culprit, how will you figure out +that the problem is due to this change in semantics? + + When the result of a ‘git bisect’ is a non-merge commit, you should +normally be able to discover the problem by examining just that commit. +Developers can make this easy by breaking their changes into small +self-contained commits. That won’t help in the case above, however, +because the problem isn’t obvious from examination of any single commit; +instead, a global view of the development is required. To make matters +worse, the change in semantics in the problematic function may be just +one small part of the changes in the upper line of development. + + On the other hand, if instead of merging at C you had rebased the +history between Z to B on top of A, you would have gotten this linear +history: + + ---Z---o---X--...---o---A---o---o---Y*--...---o---B*--D* + + Bisecting between Z and D* would hit a single culprit commit Y*, and +understanding why Y* was broken would probably be easier. + + Partly for this reason, many experienced Git users, even when working +on an otherwise merge-heavy project, keep the history linear by rebasing +against the latest upstream version before publishing. + + +File: git.info, Node: Advanced branch management, Next: Git concepts, Prev: Rewriting history and maintaining patch series, Up: Top + +6 Advanced branch management +**************************** + +* Menu: + +* Fetching individual branches:: +* git fetch and fast-forwards:: +* Forcing git fetch to do non-fast-forward updates:: +* Configuring remote-tracking branches:: + + +File: git.info, Node: Fetching individual branches, Next: git fetch and fast-forwards, Up: Advanced branch management + +6.1 Fetching individual branches +================================ + +Instead of using git-remote(1) (git-remote.html), you can also choose +just to update one branch at a time, and to store it locally under an +arbitrary name: + + $ git fetch origin todo:my-todo-work + + The first argument, ‘origin’, just tells Git to fetch from the +repository you originally cloned from. The second argument tells Git to +fetch the branch named ‘todo’ from the remote repository, and to store +it locally under the name ‘refs/heads/my-todo-work’. + + You can also fetch branches from other repositories; so + + $ git fetch git://example.com/proj.git master:example-master + + will create a new branch named ‘example-master’ and store in it the +branch named ‘master’ from the repository at the given URL. If you +already have a branch named example-master, it will attempt to +fast-forward (*note Fast-forward merges::) to the commit given by +example.com’s master branch. In more detail: + + +File: git.info, Node: git fetch and fast-forwards, Next: Forcing git fetch to do non-fast-forward updates, Prev: Fetching individual branches, Up: Advanced branch management + +6.2 git fetch and fast-forwards +=============================== + +In the previous example, when updating an existing branch, ‘git fetch’ +checks to make sure that the most recent commit on the remote branch is +a descendant of the most recent commit on your copy of the branch before +updating your copy of the branch to point at the new commit. Git calls +this process a fast-forward (*note Fast-forward merges::). + + A fast-forward looks something like this: + + o--o--o--o <-- old head of the branch + \ + o--o--o <-- new head of the branch + + In some cases it is possible that the new head will *not* actually be +a descendant of the old head. For example, the developer may have +realized she made a serious mistake, and decided to backtrack, resulting +in a situation like: + + o--o--o--o--a--b <-- old head of the branch + \ + o--o--o <-- new head of the branch + + In this case, ‘git fetch’ will fail, and print out a warning. + + In that case, you can still force Git to update to the new head, as +described in the following section. However, note that in the situation +above this may mean losing the commits labeled ‘a’ and ‘b’, unless +you’ve already created a reference of your own pointing to them. + + +File: git.info, Node: Forcing git fetch to do non-fast-forward updates, Next: Configuring remote-tracking branches, Prev: git fetch and fast-forwards, Up: Advanced branch management + +6.3 Forcing git fetch to do non-fast-forward updates +==================================================== + +If git fetch fails because the new head of a branch is not a descendant +of the old head, you may force the update with: + + $ git fetch git://example.com/proj.git +master:refs/remotes/example/master + + Note the addition of the ‘+’ sign. Alternatively, you can use the +‘-f’ flag to force updates of all the fetched branches, as in: + + $ git fetch -f origin + + Be aware that commits that the old version of example/master pointed +at may be lost, as we saw in the previous section. + + +File: git.info, Node: Configuring remote-tracking branches, Prev: Forcing git fetch to do non-fast-forward updates, Up: Advanced branch management + +6.4 Configuring remote-tracking branches +======================================== + +We saw above that ‘origin’ is just a shortcut to refer to the repository +that you originally cloned from. This information is stored in Git +configuration variables, which you can see using git-config(1) +(git-config.html): + + $ git config -l + core.repositoryformatversion=0 + core.filemode=true + core.logallrefupdates=true + remote.origin.url=git://git.kernel.org/pub/scm/git/git.git + remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* + branch.master.remote=origin + branch.master.merge=refs/heads/master + + If there are other repositories that you also use frequently, you can +create similar configuration options to save typing; for example, + + $ git remote add example git://example.com/proj.git + + adds the following to ‘.git/config’: + + [remote "example"] + url = git://example.com/proj.git + fetch = +refs/heads/*:refs/remotes/example/* + + Also note that the above configuration can be performed by directly +editing the file ‘.git/config’ instead of using git-remote(1) +(git-remote.html). + + After configuring the remote, the following three commands will do +the same thing: + + $ git fetch git://example.com/proj.git +refs/heads/*:refs/remotes/example/* + $ git fetch example +refs/heads/*:refs/remotes/example/* + $ git fetch example + + See git-config(1) (git-config.html) for more details on the +configuration options mentioned above and git-fetch(1) (git-fetch.html) +for more details on the refspec syntax. + + +File: git.info, Node: Git concepts, Next: Submodules, Prev: Advanced branch management, Up: Top + +7 Git concepts +************** + +Git is built on a small number of simple but powerful ideas. While it +is possible to get things done without understanding them, you will find +Git much more intuitive if you do. + + We start with the most important, the object database (*note +[def_object_database]::) and the index (*note [def_index]::). + +* Menu: + +* The Object Database:: +* The index:: + + +File: git.info, Node: The Object Database, Next: The index, Up: Git concepts + +7.1 The Object Database +======================= + +We already saw in *note Understanding History: Commits: Understanding +History; Commits. that all commits are stored under a 40-digit "object +name". In fact, all the information needed to represent the history of +a project is stored in objects with such names. In each case the name +is calculated by taking the SHA-1 hash of the contents of the object. +The SHA-1 hash is a cryptographic hash function. What that means to us +is that it is impossible to find two different objects with the same +name. This has a number of advantages; among others: + + • Git can quickly determine whether two objects are identical or not, + just by comparing names. + + • Since object names are computed the same way in every repository, + the same content stored in two repositories will always be stored + under the same name. + + • Git can detect errors when it reads an object, by checking that the + object’s name is still the SHA-1 hash of its contents. + + (See *note Object storage format:: for the details of the object +formatting and SHA-1 calculation.) + + There are four different types of objects: "blob", "tree", "commit", +and "tag". + + • A "blob" object (*note [def_blob_object]::) is used to store file + data. + + • A "tree" object (*note [def_tree_object]::) ties one or more "blob" + objects into a directory structure. In addition, a tree object can + refer to other tree objects, thus creating a directory hierarchy. + + • A "commit" object (*note [def_commit_object]::) ties such directory + hierarchies together into a directed acyclic graph (*note + [def_DAG]::) of revisions—each commit contains the object name of + exactly one tree designating the directory hierarchy at the time of + the commit. In addition, a commit refers to "parent" commit + objects that describe the history of how we arrived at that + directory hierarchy. + + • A "tag" object (*note [def_tag_object]::) symbolically identifies + and can be used to sign other objects. It contains the object name + and type of another object, a symbolic name (of course!) and, + optionally, a signature. + + The object types in some more detail: + +* Menu: + +* Commit Object:: +* Tree Object:: +* Blob Object:: +* Trust:: +* Tag Object:: +* How Git stores objects efficiently; pack files:: +* Dangling objects:: +* Recovering from repository corruption:: + + +File: git.info, Node: Commit Object, Next: Tree Object, Up: The Object Database + +7.1.1 Commit Object +------------------- + +The "commit" object links a physical state of a tree with a description +of how we got there and why. Use the ‘--pretty=raw’ option to +git-show(1) (git-show.html) or git-log(1) (git-log.html) to examine your +favorite commit: + + $ git show -s --pretty=raw 2be7fcb476 + commit 2be7fcb4764f2dbcee52635b91fedb1b3dcf7ab4 + tree fb3a8bdd0ceddd019615af4d57a53f43d8cee2bf + parent 257a84d9d02e90447b149af58b271c19405edb6a + author Dave Watson 1187576872 -0400 + committer Junio C Hamano 1187591163 -0700 + + Fix misspelling of 'suppress' in docs + + Signed-off-by: Junio C Hamano + + As you can see, a commit is defined by: + + • a tree: The SHA-1 name of a tree object (as defined below), + representing the contents of a directory at a certain point in + time. + + • parent(s): The SHA-1 name(s) of some number of commits which + represent the immediately previous step(s) in the history of the + project. The example above has one parent; merge commits may have + more than one. A commit with no parents is called a "root" commit, + and represents the initial revision of a project. Each project + must have at least one root. A project can also have multiple + roots, though that isn’t common (or necessarily a good idea). + + • an author: The name of the person responsible for this change, + together with its date. + + • a committer: The name of the person who actually created the + commit, with the date it was done. This may be different from the + author, for example, if the author was someone who wrote a patch + and emailed it to the person who used it to create the commit. + + • a comment describing this commit. + + Note that a commit does not itself contain any information about what +actually changed; all changes are calculated by comparing the contents +of the tree referred to by this commit with the trees associated with +its parents. In particular, Git does not attempt to record file renames +explicitly, though it can identify cases where the existence of the same +file data at changing paths suggests a rename. (See, for example, the +‘-M’ option to git-diff(1) (git-diff.html)). + + A commit is usually created by git-commit(1) (git-commit.html), which +creates a commit whose parent is normally the current HEAD, and whose +tree is taken from the content currently stored in the index. + + +File: git.info, Node: Tree Object, Next: Blob Object, Prev: Commit Object, Up: The Object Database + +7.1.2 Tree Object +----------------- + +The ever-versatile git-show(1) (git-show.html) command can also be used +to examine tree objects, but git-ls-tree(1) (git-ls-tree.html) will give +you more details: + + $ git ls-tree fb3a8bdd0ce + 100644 blob 63c918c667fa005ff12ad89437f2fdc80926e21c .gitignore + 100644 blob 5529b198e8d14decbe4ad99db3f7fb632de0439d .mailmap + 100644 blob 6ff87c4664981e4397625791c8ea3bbb5f2279a3 COPYING + 040000 tree 2fb783e477100ce076f6bf57e4a6f026013dc745 Documentation + 100755 blob 3c0032cec592a765692234f1cba47dfdcc3a9200 GIT-VERSION-GEN + 100644 blob 289b046a443c0647624607d471289b2c7dcd470b INSTALL + 100644 blob 4eb463797adc693dc168b926b6932ff53f17d0b1 Makefile + 100644 blob 548142c327a6790ff8821d67c2ee1eff7a656b52 README + ... + + As you can see, a tree object contains a list of entries, each with a +mode, object type, SHA-1 name, and name, sorted by name. It represents +the contents of a single directory tree. + + The object type may be a blob, representing the contents of a file, +or another tree, representing the contents of a subdirectory. Since +trees and blobs, like all other objects, are named by the SHA-1 hash of +their contents, two trees have the same SHA-1 name if and only if their +contents (including, recursively, the contents of all subdirectories) +are identical. This allows Git to quickly determine the differences +between two related tree objects, since it can ignore any entries with +identical object names. + + (Note: in the presence of submodules, trees may also have commits as +entries. See *note Submodules:: for documentation.) + + Note that the files all have mode 644 or 755: Git actually only pays +attention to the executable bit. + + +File: git.info, Node: Blob Object, Next: Trust, Prev: Tree Object, Up: The Object Database + +7.1.3 Blob Object +----------------- + +You can use git-show(1) (git-show.html) to examine the contents of a +blob; take, for example, the blob in the entry for ‘COPYING’ from the +tree above: + + $ git show 6ff87c4664 + + Note that the only valid version of the GPL as far as this project + is concerned is _this_ particular version of the license (ie v2, not + v2.2 or v3.x or whatever), unless explicitly otherwise stated. + ... + + A "blob" object is nothing but a binary blob of data. It doesn’t +refer to anything else or have attributes of any kind. + + Since the blob is entirely defined by its data, if two files in a +directory tree (or in multiple different versions of the repository) +have the same contents, they will share the same blob object. The +object is totally independent of its location in the directory tree, and +renaming a file does not change the object that file is associated with. + + Note that any tree or blob object can be examined using git-show(1) +(git-show.html) with the : syntax. This can sometimes +be useful for browsing the contents of a tree that is not currently +checked out. + + +File: git.info, Node: Trust, Next: Tag Object, Prev: Blob Object, Up: The Object Database + +7.1.4 Trust +----------- + +If you receive the SHA-1 name of a blob from one source, and its +contents from another (possibly untrusted) source, you can still trust +that those contents are correct as long as the SHA-1 name agrees. This +is because the SHA-1 is designed so that it is infeasible to find +different contents that produce the same hash. + + Similarly, you need only trust the SHA-1 name of a top-level tree +object to trust the contents of the entire directory that it refers to, +and if you receive the SHA-1 name of a commit from a trusted source, +then you can easily verify the entire history of commits reachable +through parents of that commit, and all of those contents of the trees +referred to by those commits. + + So to introduce some real trust in the system, the only thing you +need to do is to digitally sign just _one_ special note, which includes +the name of a top-level commit. Your digital signature shows others +that you trust that commit, and the immutability of the history of +commits tells others that they can trust the whole history. + + In other words, you can easily validate a whole archive by just +sending out a single email that tells the people the name (SHA-1 hash) +of the top commit, and digitally sign that email using something like +GPG/PGP. + + To assist in this, Git also provides the tag object… + + +File: git.info, Node: Tag Object, Next: How Git stores objects efficiently; pack files, Prev: Trust, Up: The Object Database + +7.1.5 Tag Object +---------------- + +A tag object contains an object, object type, tag name, the name of the +person ("tagger") who created the tag, and a message, which may contain +a signature, as can be seen using git-cat-file(1) (git-cat-file.html): + + $ git cat-file tag v1.5.0 + object 437b1b20df4b356c9342dac8d38849f24ef44f27 + type commit + tag v1.5.0 + tagger Junio C Hamano 1171411200 +0000 + + GIT 1.5.0 + -----BEGIN PGP SIGNATURE----- + Version: GnuPG v1.4.6 (GNU/Linux) + + iD8DBQBF0lGqwMbZpPMRm5oRAuRiAJ9ohBLd7s2kqjkKlq1qqC57SbnmzQCdG4ui + nLE/L9aUXdWeTFPron96DLA= + =2E+0 + -----END PGP SIGNATURE----- + + See the git-tag(1) (git-tag.html) command to learn how to create and +verify tag objects. (Note that git-tag(1) (git-tag.html) can also be +used to create "lightweight tags", which are not tag objects at all, but +just simple references whose names begin with ‘refs/tags/’). + + +File: git.info, Node: How Git stores objects efficiently; pack files, Next: Dangling objects, Prev: Tag Object, Up: The Object Database + +7.1.6 How Git stores objects efficiently: pack files +---------------------------------------------------- + +Newly created objects are initially created in a file named after the +object’s SHA-1 hash (stored in ‘.git/objects’). + + Unfortunately this system becomes inefficient once a project has a +lot of objects. Try this on an old project: + + $ git count-objects + 6930 objects, 47620 kilobytes + + The first number is the number of objects which are kept in +individual files. The second is the amount of space taken up by those +"loose" objects. + + You can save space and make Git faster by moving these loose objects +in to a "pack file", which stores a group of objects in an efficient +compressed format; the details of how pack files are formatted can be +found in pack format (technical/pack-format.html). + + To put the loose objects into a pack, just run git repack: + + $ git repack + Counting objects: 6020, done. + Delta compression using up to 4 threads. + Compressing objects: 100% (6020/6020), done. + Writing objects: 100% (6020/6020), done. + Total 6020 (delta 4070), reused 0 (delta 0) + + This creates a single "pack file" in .git/objects/pack/ containing +all currently unpacked objects. You can then run + + $ git prune + + to remove any of the "loose" objects that are now contained in the +pack. This will also remove any unreferenced objects (which may be +created when, for example, you use ‘git reset’ to remove a commit). You +can verify that the loose objects are gone by looking at the +‘.git/objects’ directory or by running + + $ git count-objects + 0 objects, 0 kilobytes + + Although the object files are gone, any commands that refer to those +objects will work exactly as they did before. + + The git-gc(1) (git-gc.html) command performs packing, pruning, and +more for you, so is normally the only high-level command you need. + + +File: git.info, Node: Dangling objects, Next: Recovering from repository corruption, Prev: How Git stores objects efficiently; pack files, Up: The Object Database + +7.1.7 Dangling objects +---------------------- + +The git-fsck(1) (git-fsck.html) command will sometimes complain about +dangling objects. They are not a problem. + + The most common cause of dangling objects is that you’ve rebased a +branch, or you have pulled from somebody else who rebased a branch—see +*note Rewriting history and maintaining patch series::. In that case, +the old head of the original branch still exists, as does everything it +pointed to. The branch pointer itself just doesn’t, since you replaced +it with another one. + + There are also other situations that cause dangling objects. For +example, a "dangling blob" may arise because you did a ‘git add’ of a +file, but then, before you actually committed it and made it part of the +bigger picture, you changed something else in that file and committed +that *updated* thing—the old state that you added originally ends up not +being pointed to by any commit or tree, so it’s now a dangling blob +object. + + Similarly, when the "recursive" merge strategy runs, and finds that +there are criss-cross merges and thus more than one merge base (which is +fairly unusual, but it does happen), it will generate one temporary +midway tree (or possibly even more, if you had lots of criss-crossing +merges and more than two merge bases) as a temporary internal merge +base, and again, those are real objects, but the end result will not end +up pointing to them, so they end up "dangling" in your repository. + + Generally, dangling objects aren’t anything to worry about. They can +even be very useful: if you screw something up, the dangling objects can +be how you recover your old tree (say, you did a rebase, and realized +that you really didn’t want to—you can look at what dangling objects you +have, and decide to reset your head to some old dangling state). + + For commits, you can just use: + + $ gitk --not --all + + This asks for all the history reachable from the given commit but not +from any branch, tag, or other reference. If you decide it’s something +you want, you can always create a new reference to it, e.g., + + $ git branch recovered-branch + + For blobs and trees, you can’t do the same, but you can still examine +them. You can just do + + $ git show + + to show what the contents of the blob were (or, for a tree, basically +what the ‘ls’ for that directory was), and that may give you some idea +of what the operation was that left that dangling object. + + Usually, dangling blobs and trees aren’t very interesting. They’re +almost always the result of either being a half-way mergebase (the blob +will often even have the conflict markers from a merge in it, if you +have had conflicting merges that you fixed up by hand), or simply +because you interrupted a ‘git fetch’ with ^C or something like that, +leaving _some_ of the new objects in the object database, but just +dangling and useless. + + Anyway, once you are sure that you’re not interested in any dangling +state, you can just prune all unreachable objects: + + $ git prune + + and they’ll be gone. (You should only run ‘git prune’ on a quiescent +repository—it’s kind of like doing a filesystem fsck recovery: you don’t +want to do that while the filesystem is mounted. ‘git prune’ is +designed not to cause any harm in such cases of concurrent accesses to a +repository but you might receive confusing or scary messages.) + + +File: git.info, Node: Recovering from repository corruption, Prev: Dangling objects, Up: The Object Database + +7.1.8 Recovering from repository corruption +------------------------------------------- + +By design, Git treats data trusted to it with caution. However, even in +the absence of bugs in Git itself, it is still possible that hardware or +operating system errors could corrupt data. + + The first defense against such problems is backups. You can back up +a Git directory using clone, or just using cp, tar, or any other backup +mechanism. + + As a last resort, you can search for the corrupted objects and +attempt to replace them by hand. Back up your repository before +attempting this in case you corrupt things even more in the process. + + We’ll assume that the problem is a single missing or corrupted blob, +which is sometimes a solvable problem. (Recovering missing trees and +especially commits is *much* harder). + + Before starting, verify that there is corruption, and figure out +where it is with git-fsck(1) (git-fsck.html); this may be +time-consuming. + + Assume the output looks like this: + + $ git fsck --full --no-dangling + broken link from tree 2d9263c6d23595e7cb2a21e5ebbb53655278dff8 + to blob 4b9458b3786228369c63936db65827de3cc06200 + missing blob 4b9458b3786228369c63936db65827de3cc06200 + + Now you know that blob 4b9458b3 is missing, and that the tree +2d9263c6 points to it. If you could find just one copy of that missing +blob object, possibly in some other repository, you could move it into +‘.git/objects/4b/9458b3...’ and be done. Suppose you can’t. You can +still examine the tree that pointed to it with git-ls-tree(1) +(git-ls-tree.html), which might output something like: + + $ git ls-tree 2d9263c6d23595e7cb2a21e5ebbb53655278dff8 + 100644 blob 8d14531846b95bfa3564b58ccfb7913a034323b8 .gitignore + 100644 blob ebf9bf84da0aab5ed944264a5db2a65fe3a3e883 .mailmap + 100644 blob ca442d313d86dc67e0a2e5d584b465bd382cbf5c COPYING + ... + 100644 blob 4b9458b3786228369c63936db65827de3cc06200 myfile + ... + + So now you know that the missing blob was the data for a file named +‘myfile’. And chances are you can also identify the directory—let’s say +it’s in ‘somedirectory’. If you’re lucky the missing copy might be the +same as the copy you have checked out in your working tree at +‘somedirectory/myfile’; you can test whether that’s right with +git-hash-object(1) (git-hash-object.html): + + $ git hash-object -w somedirectory/myfile + + which will create and store a blob object with the contents of +somedirectory/myfile, and output the SHA-1 of that object. if you’re +extremely lucky it might be 4b9458b3786228369c63936db65827de3cc06200, in +which case you’ve guessed right, and the corruption is fixed! + + Otherwise, you need more information. How do you tell which version +of the file has been lost? + + The easiest way to do this is with: + + $ git log --raw --all --full-history -- somedirectory/myfile + + Because you’re asking for raw output, you’ll now get something like + + commit abc + Author: + Date: + ... + :100644 100644 4b9458b newsha M somedirectory/myfile + + + commit xyz + Author: + Date: + + ... + :100644 100644 oldsha 4b9458b M somedirectory/myfile + + This tells you that the immediately following version of the file was +"newsha", and that the immediately preceding version was "oldsha". You +also know the commit messages that went with the change from oldsha to +4b9458b and with the change from 4b9458b to newsha. + + If you’ve been committing small enough changes, you may now have a +good shot at reconstructing the contents of the in-between state +4b9458b. + + If you can do that, you can now recreate the missing object with + + $ git hash-object -w + + and your repository is good again! + + (Btw, you could have ignored the ‘fsck’, and started with doing a + + $ git log --raw --all + + and just looked for the sha of the missing object (4b9458b) in that +whole thing. It’s up to you—Git does *have* a lot of information, it is +just missing one particular blob version. + + +File: git.info, Node: The index, Prev: The Object Database, Up: Git concepts + +7.2 The index +============= + +The index is a binary file (generally kept in ‘.git/index’) containing a +sorted list of path names, each with permissions and the SHA-1 of a blob +object; git-ls-files(1) (git-ls-files.html) can show you the contents of +the index: + + $ git ls-files --stage + 100644 63c918c667fa005ff12ad89437f2fdc80926e21c 0 .gitignore + 100644 5529b198e8d14decbe4ad99db3f7fb632de0439d 0 .mailmap + 100644 6ff87c4664981e4397625791c8ea3bbb5f2279a3 0 COPYING + 100644 a37b2152bd26be2c2289e1f57a292534a51a93c7 0 Documentation/.gitignore + 100644 fbefe9a45b00a54b58d94d06eca48b03d40a50e0 0 Documentation/Makefile + ... + 100644 2511aef8d89ab52be5ec6a5e46236b4b6bcd07ea 0 xdiff/xtypes.h + 100644 2ade97b2574a9f77e7ae4002a4e07a6a38e46d07 0 xdiff/xutils.c + 100644 d5de8292e05e7c36c4b68857c1cf9855e3d2f70a 0 xdiff/xutils.h + + Note that in older documentation you may see the index called the +"current directory cache" or just the "cache". It has three important +properties: + + 1. The index contains all the information necessary to generate a + single (uniquely determined) tree object. + + For example, running git-commit(1) (git-commit.html) generates this + tree object from the index, stores it in the object database, and + uses it as the tree object associated with the new commit. + + 2. The index enables fast comparisons between the tree object it + defines and the working tree. + + It does this by storing some additional data for each entry (such + as the last modified time). This data is not displayed above, and + is not stored in the created tree object, but it can be used to + determine quickly which files in the working directory differ from + what was stored in the index, and thus save Git from having to read + all of the data from such files to look for changes. + + 3. It can efficiently represent information about merge conflicts + between different tree objects, allowing each pathname to be + associated with sufficient information about the trees involved + that you can create a three-way merge between them. + + We saw in *note Getting conflict-resolution help during a merge:: + that during a merge the index can store multiple versions of a + single file (called "stages"). The third column in the + git-ls-files(1) (git-ls-files.html) output above is the stage + number, and will take on values other than 0 for files with merge + conflicts. + + The index is thus a sort of temporary staging area, which is filled +with a tree which you are in the process of working on. + + If you blow the index away entirely, you generally haven’t lost any +information as long as you have the name of the tree that it described. + + +File: git.info, Node: Submodules, Next: Low-level Git operations, Prev: Git concepts, Up: Top + +8 Submodules +************ + +Large projects are often composed of smaller, self-contained modules. +For example, an embedded Linux distribution’s source tree would include +every piece of software in the distribution with some local +modifications; a movie player might need to build against a specific, +known-working version of a decompression library; several independent +programs might all share the same build scripts. + + With centralized revision control systems this is often accomplished +by including every module in one single repository. Developers can +check out all modules or only the modules they need to work with. They +can even modify files across several modules in a single commit while +moving things around or updating APIs and translations. + + Git does not allow partial checkouts, so duplicating this approach in +Git would force developers to keep a local copy of modules they are not +interested in touching. Commits in an enormous checkout would be slower +than you’d expect as Git would have to scan every directory for changes. +If modules have a lot of local history, clones would take forever. + + On the plus side, distributed revision control systems can much +better integrate with external sources. In a centralized model, a +single arbitrary snapshot of the external project is exported from its +own revision control and then imported into the local revision control +on a vendor branch. All the history is hidden. With distributed +revision control you can clone the entire external history and much more +easily follow development and re-merge local changes. + + Git’s submodule support allows a repository to contain, as a +subdirectory, a checkout of an external project. Submodules maintain +their own identity; the submodule support just stores the submodule +repository location and commit ID, so other developers who clone the +containing project ("superproject") can easily clone all the submodules +at the same revision. Partial checkouts of the superproject are +possible: you can tell Git to clone none, some or all of the submodules. + + The git-submodule(1) (git-submodule.html) command is available since +Git 1.5.3. Users with Git 1.5.2 can look up the submodule commits in +the repository and manually check them out; earlier versions won’t +recognize the submodules at all. + + To see how submodule support works, create four example repositories +that can be used later as a submodule: + + $ mkdir ~/git + $ cd ~/git + $ for i in a b c d + do + mkdir $i + cd $i + git init + echo "module $i" > $i.txt + git add $i.txt + git commit -m "Initial commit, submodule $i" + cd .. + done + + Now create the superproject and add all the submodules: + + $ mkdir super + $ cd super + $ git init + $ for i in a b c d + do + git submodule add ~/git/$i $i + done + + *Note* + + Do not use local URLs here if you plan to publish your + superproject! + + See what files ‘git submodule’ created: + + $ ls -a + . .. .git .gitmodules a b c d + + The ‘git submodule add ’ command does a couple of +things: + + • It clones the submodule from ‘’ to the given ‘’ under + the current directory and by default checks out the master branch. + + • It adds the submodule’s clone path to the gitmodules(5) + (gitmodules.html) file and adds this file to the index, ready to be + committed. + + • It adds the submodule’s current commit ID to the index, ready to be + committed. + + Commit the superproject: + + $ git commit -m "Add submodules a, b, c and d." + + Now clone the superproject: + + $ cd .. + $ git clone super cloned + $ cd cloned + + The submodule directories are there, but they’re empty: + + $ ls -a a + . .. + $ git submodule status + -d266b9873ad50488163457f025db7cdd9683d88b a + -e81d457da15309b4fef4249aba9b50187999670d b + -c1536a972b9affea0f16e0680ba87332dc059146 c + -d96249ff5d57de5de093e6baff9e0aafa5276a74 d + + *Note* + + The commit object names shown above would be different for you, but + they should match the HEAD commit object names of your + repositories. You can check it by running ‘git ls-remote ../a’. + + Pulling down the submodules is a two-step process. First run ‘git +submodule init’ to add the submodule repository URLs to ‘.git/config’: + + $ git submodule init + + Now use ‘git submodule update’ to clone the repositories and check +out the commits specified in the superproject: + + $ git submodule update + $ cd a + $ ls -a + . .. .git a.txt + + One major difference between ‘git submodule update’ and ‘git +submodule add’ is that ‘git submodule update’ checks out a specific +commit, rather than the tip of a branch. It’s like checking out a tag: +the head is detached, so you’re not working on a branch. + + $ git branch + * (detached from d266b98) + master + + If you want to make a change within a submodule and you have a +detached head, then you should create or checkout a branch, make your +changes, publish the change within the submodule, and then update the +superproject to reference the new commit: + + $ git switch master + + or + + $ git switch -c fix-up + + then + + $ echo "adding a line again" >> a.txt + $ git commit -a -m "Updated the submodule from within the superproject." + $ git push + $ cd .. + $ git diff + diff --git a/a b/a + index d266b98..261dfac 160000 + --- a/a + +++ b/a + @@ -1 +1 @@ + -Subproject commit d266b9873ad50488163457f025db7cdd9683d88b + +Subproject commit 261dfac35cb99d380eb966e102c1197139f7fa24 + $ git add a + $ git commit -m "Updated submodule a." + $ git push + + You have to run ‘git submodule update’ after ‘git pull’ if you want +to update submodules, too. + +* Menu: + +* Pitfalls with submodules:: + + +File: git.info, Node: Pitfalls with submodules, Up: Submodules + +8.1 Pitfalls with submodules +============================ + +Always publish the submodule change before publishing the change to the +superproject that references it. If you forget to publish the submodule +change, others won’t be able to clone the repository: + + $ cd ~/git/super/a + $ echo i added another line to this file >> a.txt + $ git commit -a -m "doing it wrong this time" + $ cd .. + $ git add a + $ git commit -m "Updated submodule a again." + $ git push + $ cd ~/git/cloned + $ git pull + $ git submodule update + error: pathspec '261dfac35cb99d380eb966e102c1197139f7fa24' did not match any file(s) known to git. + Did you forget to 'git add'? + Unable to checkout '261dfac35cb99d380eb966e102c1197139f7fa24' in submodule path 'a' + + In older Git versions it could be easily forgotten to commit new or +modified files in a submodule, which silently leads to similar problems +as not pushing the submodule changes. Starting with Git 1.7.0 both ‘git +status’ and ‘git diff’ in the superproject show submodules as modified +when they contain new or modified files to protect against accidentally +committing such a state. ‘git diff’ will also add a ‘-dirty’ to the +work tree side when generating patch output or used with the +‘--submodule’ option: + + $ git diff + diff --git a/sub b/sub + --- a/sub + +++ b/sub + @@ -1 +1 @@ + -Subproject commit 3f356705649b5d566d97ff843cf193359229a453 + +Subproject commit 3f356705649b5d566d97ff843cf193359229a453-dirty + $ git diff --submodule + Submodule sub 3f35670..3f35670-dirty: + + You also should not rewind branches in a submodule beyond commits +that were ever recorded in any superproject. + + It’s not safe to run ‘git submodule update’ if you’ve made and +committed changes within a submodule without checking out a branch +first. They will be silently overwritten: + + $ cat a.txt + module a + $ echo line added from private2 >> a.txt + $ git commit -a -m "line added inside private2" + $ cd .. + $ git submodule update + Submodule path 'a': checked out 'd266b9873ad50488163457f025db7cdd9683d88b' + $ cd a + $ cat a.txt + module a + + *Note* + + The changes are still visible in the submodule’s reflog. + + If you have uncommitted changes in your submodule working tree, ‘git +submodule update’ will not overwrite them. Instead, you get the usual +warning about not being able switch from a dirty branch. + + +File: git.info, Node: Low-level Git operations, Next: Hacking Git, Prev: Submodules, Up: Top + +9 Low-level Git operations +************************** + +Many of the higher-level commands were originally implemented as shell +scripts using a smaller core of low-level Git commands. These can still +be useful when doing unusual things with Git, or just as a way to +understand its inner workings. + +* Menu: + +* Object access and manipulation:: +* The Workflow:: +* Examining the data:: +* Merging multiple trees:: +* Merging multiple trees, continued: Merging multiple trees; continued. + + +File: git.info, Node: Object access and manipulation, Next: The Workflow, Up: Low-level Git operations + +9.1 Object access and manipulation +================================== + +The git-cat-file(1) (git-cat-file.html) command can show the contents of +any object, though the higher-level git-show(1) (git-show.html) is +usually more useful. + + The git-commit-tree(1) (git-commit-tree.html) command allows +constructing commits with arbitrary parents and trees. + + A tree can be created with git-write-tree(1) (git-write-tree.html) +and its data can be accessed by git-ls-tree(1) (git-ls-tree.html). Two +trees can be compared with git-diff-tree(1) (git-diff-tree.html). + + A tag is created with git-mktag(1) (git-mktag.html), and the +signature can be verified by git-verify-tag(1) (git-verify-tag.html), +though it is normally simpler to use git-tag(1) (git-tag.html) for both. + + +File: git.info, Node: The Workflow, Next: Examining the data, Prev: Object access and manipulation, Up: Low-level Git operations + +9.2 The Workflow +================ + +High-level operations such as git-commit(1) (git-commit.html) and +git-restore(1) (git-restore.html) work by moving data between the +working tree, the index, and the object database. Git provides +low-level operations which perform each of these steps individually. + + Generally, all Git operations work on the index file. Some +operations work *purely* on the index file (showing the current state of +the index), but most operations move data between the index file and +either the database or the working directory. Thus there are four main +combinations: + +* Menu: + +* working directory → index:: +* index → object database:: +* object database → index:: +* index → working directory:: +* Tying it all together:: + + +File: git.info, Node: working directory → index, Next: index → object database, Up: The Workflow + +9.2.1 working directory → index +------------------------------- + +The git-update-index(1) (git-update-index.html) command updates the +index with information from the working directory. You generally update +the index information by just specifying the filename you want to +update, like so: + + $ git update-index filename + + but to avoid common mistakes with filename globbing etc., the command +will not normally add totally new entries or remove old entries, i.e. +it will normally just update existing cache entries. + + To tell Git that yes, you really do realize that certain files no +longer exist, or that new files should be added, you should use the +‘--remove’ and ‘--add’ flags respectively. + + NOTE! A ‘--remove’ flag does _not_ mean that subsequent filenames +will necessarily be removed: if the files still exist in your directory +structure, the index will be updated with their new status, not removed. +The only thing ‘--remove’ means is that update-index will be considering +a removed file to be a valid thing, and if the file really does not +exist any more, it will update the index accordingly. + + As a special case, you can also do ‘git update-index --refresh’, +which will refresh the "stat" information of each index to match the +current stat information. It will _not_ update the object status +itself, and it will only update the fields that are used to quickly test +whether an object still matches its old backing store object. + + The previously introduced git-add(1) (git-add.html) is just a wrapper +for git-update-index(1) (git-update-index.html). + + +File: git.info, Node: index → object database, Next: object database → index, Prev: working directory → index, Up: The Workflow + +9.2.2 index → object database +----------------------------- + +You write your current index file to a "tree" object with the program + + $ git write-tree + + that doesn’t come with any options—it will just write out the current +index into the set of tree objects that describe that state, and it will +return the name of the resulting top-level tree. You can use that tree +to re-generate the index at any time by going in the other direction: + + +File: git.info, Node: object database → index, Next: index → working directory, Prev: index → object database, Up: The Workflow + +9.2.3 object database → index +----------------------------- + +You read a "tree" file from the object database, and use that to +populate (and overwrite—don’t do this if your index contains any unsaved +state that you might want to restore later!) your current index. +Normal operation is just + + $ git read-tree + + and your index file will now be equivalent to the tree that you saved +earlier. However, that is only your _index_ file: your working +directory contents have not been modified. + + +File: git.info, Node: index → working directory, Next: Tying it all together, Prev: object database → index, Up: The Workflow + +9.2.4 index → working directory +------------------------------- + +You update your working directory from the index by "checking out" +files. This is not a very common operation, since normally you’d just +keep your files updated, and rather than write to your working +directory, you’d tell the index files about the changes in your working +directory (i.e. ‘git update-index’). + + However, if you decide to jump to a new version, or check out +somebody else’s version, or just restore a previous tree, you’d populate +your index file with read-tree, and then you need to check out the +result with + + $ git checkout-index filename + + or, if you want to check out all of the index, use ‘-a’. + + NOTE! ‘git checkout-index’ normally refuses to overwrite old files, +so if you have an old version of the tree already checked out, you will +need to use the ‘-f’ flag (_before_ the ‘-a’ flag or the filename) to +_force_ the checkout. + + Finally, there are a few odds and ends which are not purely moving +from one representation to the other: + + +File: git.info, Node: Tying it all together, Prev: index → working directory, Up: The Workflow + +9.2.5 Tying it all together +--------------------------- + +To commit a tree you have instantiated with ‘git write-tree’, you’d +create a "commit" object that refers to that tree and the history behind +it—most notably the "parent" commits that preceded it in history. + + Normally a "commit" has one parent: the previous state of the tree +before a certain change was made. However, sometimes it can have two or +more parent commits, in which case we call it a "merge", due to the fact +that such a commit brings together ("merges") two or more previous +states represented by other commits. + + In other words, while a "tree" represents a particular directory +state of a working directory, a "commit" represents that state in time, +and explains how we got there. + + You create a commit object by giving it the tree that describes the +state at the time of the commit, and a list of parents: + + $ git commit-tree -p [(-p )...] + + and then giving the reason for the commit on stdin (either through +redirection from a pipe or file, or by just typing it at the tty). + + ‘git commit-tree’ will return the name of the object that represents +that commit, and you should save it away for later use. Normally, you’d +commit a new ‘HEAD’ state, and while Git doesn’t care where you save the +note about that state, in practice we tend to just write the result to +the file pointed at by ‘.git/HEAD’, so that we can always see what the +last committed state was. + + Here is a picture that illustrates how various pieces fit together: + + commit-tree + commit obj + +----+ + | | + | | + V V + +-----------+ + | Object DB | + | Backing | + | Store | + +-----------+ + ^ + write-tree | | + tree obj | | + | | read-tree + | | tree obj + V + +-----------+ + | Index | + | "cache" | + +-----------+ + update-index ^ + blob obj | | + | | + checkout-index -u | | checkout-index + stat | | blob obj + V + +-----------+ + | Working | + | Directory | + +-----------+ + + +File: git.info, Node: Examining the data, Next: Merging multiple trees, Prev: The Workflow, Up: Low-level Git operations + +9.3 Examining the data +====================== + +You can examine the data represented in the object database and the +index with various helper tools. For every object, you can use +git-cat-file(1) (git-cat-file.html) to examine details about the object: + + $ git cat-file -t + + shows the type of the object, and once you have the type (which is +usually implicit in where you find the object), you can use + + $ git cat-file blob|tree|commit|tag + + to show its contents. NOTE! Trees have binary content, and as a +result there is a special helper for showing that content, called ‘git +ls-tree’, which turns the binary content into a more easily readable +form. + + It’s especially instructive to look at "commit" objects, since those +tend to be small and fairly self-explanatory. In particular, if you +follow the convention of having the top commit name in ‘.git/HEAD’, you +can do + + $ git cat-file commit HEAD + + to see what the top commit was. + + +File: git.info, Node: Merging multiple trees, Next: Merging multiple trees; continued, Prev: Examining the data, Up: Low-level Git operations + +9.4 Merging multiple trees +========================== + +Git can help you perform a three-way merge, which can in turn be used +for a many-way merge by repeating the merge procedure several times. +The usual situation is that you only do one three-way merge (reconciling +two lines of history) and commit the result, but if you like to, you can +merge several branches in one go. + + To perform a three-way merge, you start with the two commits you want +to merge, find their closest common parent (a third commit), and compare +the trees corresponding to these three commits. + + To get the "base" for the merge, look up the common parent of two +commits: + + $ git merge-base + + This prints the name of a commit they are both based on. You should +now look up the tree objects of those commits, which you can easily do +with + + $ git cat-file commit | head -1 + + since the tree object information is always the first line in a +commit object. + + Once you know the three trees you are going to merge (the one +"original" tree, aka the common tree, and the two "result" trees, aka +the branches you want to merge), you do a "merge" read into the index. +This will complain if it has to throw away your old index contents, so +you should make sure that you’ve committed those—in fact you would +normally always do a merge against your last commit (which should thus +match what you have in your current index anyway). + + To do the merge, do + + $ git read-tree -m -u + + which will do all trivial merge operations for you directly in the +index file, and you can just write the result out with ‘git write-tree’. + + +File: git.info, Node: Merging multiple trees; continued, Prev: Merging multiple trees, Up: Low-level Git operations + +9.5 Merging multiple trees, continued +===================================== + +Sadly, many merges aren’t trivial. If there are files that have been +added, moved or removed, or if both branches have modified the same +file, you will be left with an index tree that contains "merge entries" +in it. Such an index tree can _NOT_ be written out to a tree object, +and you will have to resolve any such merge clashes using other tools +before you can write out the result. + + You can examine such index state with ‘git ls-files --unmerged’ +command. An example: + + $ git read-tree -m $orig HEAD $target + $ git ls-files --unmerged + 100644 263414f423d0e4d70dae8fe53fa34614ff3e2860 1 hello.c + 100644 06fa6a24256dc7e560efa5687fa84b51f0263c3a 2 hello.c + 100644 cc44c73eb783565da5831b4d820c962954019b69 3 hello.c + + Each line of the ‘git ls-files --unmerged’ output begins with the +blob mode bits, blob SHA-1, _stage number_, and the filename. The +_stage number_ is Git’s way to say which tree it came from: stage 1 +corresponds to the ‘$orig’ tree, stage 2 to the ‘HEAD’ tree, and stage 3 +to the ‘$target’ tree. + + Earlier we said that trivial merges are done inside ‘git read-tree +-m’. For example, if the file did not change from ‘$orig’ to ‘HEAD’ or +‘$target’, or if the file changed from ‘$orig’ to ‘HEAD’ and ‘$orig’ to +‘$target’ the same way, obviously the final outcome is what is in +‘HEAD’. What the above example shows is that file ‘hello.c’ was changed +from ‘$orig’ to ‘HEAD’ and ‘$orig’ to ‘$target’ in a different way. You +could resolve this by running your favorite 3-way merge program, e.g. +‘diff3’, ‘merge’, or Git’s own merge-file, on the blob objects from +these three stages yourself, like this: + + $ git cat-file blob 263414f >hello.c~1 + $ git cat-file blob 06fa6a2 >hello.c~2 + $ git cat-file blob cc44c73 >hello.c~3 + $ git merge-file hello.c~2 hello.c~1 hello.c~3 + + This would leave the merge result in ‘hello.c~2’ file, along with +conflict markers if there are conflicts. After verifying the merge +result makes sense, you can tell Git what the final merge result for +this file is by: + + $ mv -f hello.c~2 hello.c + $ git update-index hello.c + + When a path is in the "unmerged" state, running ‘git update-index’ +for that path tells Git to mark the path resolved. + + The above is the description of a Git merge at the lowest level, to +help you understand what conceptually happens under the hood. In +practice, nobody, not even Git itself, runs ‘git cat-file’ three times +for this. There is a ‘git merge-index’ program that extracts the stages +to temporary files and calls a "merge" script on it: + + $ git merge-index git-merge-one-file hello.c + + and that is what higher level ‘git merge -s resolve’ is implemented +with. + + +File: git.info, Node: Hacking Git, Next: Git Glossary, Prev: Low-level Git operations, Up: Top + +10 Hacking Git +************** + +This chapter covers internal details of the Git implementation which +probably only Git developers need to understand. + +* Menu: + +* Object storage format:: +* A birds-eye view of Git’s source code:: + + +File: git.info, Node: Object storage format, Next: A birds-eye view of Git’s source code, Up: Hacking Git + +10.1 Object storage format +========================== + +All objects have a statically determined "type" which identifies the +format of the object (i.e. how it is used, and how it can refer to +other objects). There are currently four different object types: +"blob", "tree", "commit", and "tag". + + Regardless of object type, all objects share the following +characteristics: they are all deflated with zlib, and have a header that +not only specifies their type, but also provides size information about +the data in the object. It’s worth noting that the SHA-1 hash that is +used to name the object is the hash of the original data plus this +header, so ‘sha1sum’ _file_ does not match the object name for _file_. + + As a result, the general consistency of an object can always be +tested independently of the contents or the type of the object: all +objects can be validated by verifying that (a) their hashes match the +content of the file and (b) the object successfully inflates to a stream +of bytes that forms a sequence of ‘ + ++ + + ’. + + The structured objects can further have their structure and +connectivity to other objects verified. This is generally done with the +‘git fsck’ program, which generates a full dependency graph of all +objects, and verifies their internal consistency (in addition to just +verifying their superficial consistency through the hash). + + +File: git.info, Node: A birds-eye view of Git’s source code, Prev: Object storage format, Up: Hacking Git + +10.2 A birds-eye view of Git’s source code +========================================== + +It is not always easy for new developers to find their way through Git’s +source code. This section gives you a little guidance to show where to +start. + + A good place to start is with the contents of the initial commit, +with: + + $ git switch --detach e83c5163 + + The initial revision lays the foundation for almost everything Git +has today, but is small enough to read in one sitting. + + Note that terminology has changed since that revision. For example, +the README in that revision uses the word "changeset" to describe what +we now call a commit (*note [def_commit_object]::). + + Also, we do not call it "cache" any more, but rather "index"; +however, the file is still called ‘cache.h’. Remark: Not much reason to +change it now, especially since there is no good single name for it +anyway, because it is basically _the_ header file which is included by +_all_ of Git’s C sources. + + If you grasp the ideas in that initial commit, you should check out a +more recent version and skim ‘cache.h’, ‘object.h’ and ‘commit.h’. + + In the early days, Git (in the tradition of UNIX) was a bunch of +programs which were extremely simple, and which you used in scripts, +piping the output of one into another. This turned out to be good for +initial development, since it was easier to test new things. However, +recently many of these parts have become builtins, and some of the core +has been "libified", i.e. put into libgit.a for performance, +portability reasons, and to avoid code duplication. + + By now, you know what the index is (and find the corresponding data +structures in ‘cache.h’), and that there are just a couple of object +types (blobs, trees, commits and tags) which inherit their common +structure from ‘struct object’, which is their first member (and thus, +you can cast e.g. ‘(struct object *)commit’ to achieve the _same_ as +‘&commit->object’, i.e. get at the object name and flags). + + Now is a good point to take a break to let this information sink in. + + Next step: get familiar with the object naming. Read *note Naming +commits::. There are quite a few ways to name an object (and not only +revisions!). All of these are handled in ‘sha1_name.c’. Just have a +quick look at the function ‘get_sha1()’. A lot of the special handling +is done by functions like ‘get_sha1_basic()’ or the likes. + + This is just to get you into the groove for the most libified part of +Git: the revision walker. + + Basically, the initial version of ‘git log’ was a shell script: + + $ git-rev-list --pretty $(git-rev-parse --default HEAD "$@") | \ + LESS=-S ${PAGER:-less} + + What does this mean? + + ‘git rev-list’ is the original version of the revision walker, which +_always_ printed a list of revisions to stdout. It is still functional, +and needs to, since most new Git commands start out as scripts using +‘git rev-list’. + + ‘git rev-parse’ is not as important any more; it was only used to +filter out options that were relevant for the different plumbing +commands that were called by the script. + + Most of what ‘git rev-list’ did is contained in ‘revision.c’ and +‘revision.h’. It wraps the options in a struct named ‘rev_info’, which +controls how and what revisions are walked, and more. + + The original job of ‘git rev-parse’ is now taken by the function +‘setup_revisions()’, which parses the revisions and the common +command-line options for the revision walker. This information is +stored in the struct ‘rev_info’ for later consumption. You can do your +own command-line option parsing after calling ‘setup_revisions()’. +After that, you have to call ‘prepare_revision_walk()’ for +initialization, and then you can get the commits one by one with the +function ‘get_revision()’. + + If you are interested in more details of the revision walking +process, just have a look at the first implementation of ‘cmd_log()’; +call ‘git show v1.3.0~155^2~4’ and scroll down to that function (note +that you no longer need to call ‘setup_pager()’ directly). + + Nowadays, ‘git log’ is a builtin, which means that it is _contained_ +in the command ‘git’. The source side of a builtin is + + • a function called ‘cmd_’, typically defined in + ‘builtin/’ (note that older versions of Git used to have it + in ‘builtin-.c’ instead), and declared in ‘builtin.h’. + + • an entry in the ‘commands[]’ array in ‘git.c’, and + + • an entry in ‘BUILTIN_OBJECTS’ in the ‘Makefile’. + + Sometimes, more than one builtin is contained in one source file. +For example, ‘cmd_whatchanged()’ and ‘cmd_log()’ both reside in +‘builtin/log.c’, since they share quite a bit of code. In that case, +the commands which are _not_ named like the ‘.c’ file in which they live +have to be listed in ‘BUILT_INS’ in the ‘Makefile’. + + ‘git log’ looks more complicated in C than it does in the original +script, but that allows for a much greater flexibility and performance. + + Here again it is a good point to take a pause. + + Lesson three is: study the code. Really, it is the best way to learn +about the organization of Git (after you know the basic concepts). + + So, think about something which you are interested in, say, "how can +I access a blob just knowing the object name of it?". The first step is +to find a Git command with which you can do it. In this example, it is +either ‘git show’ or ‘git cat-file’. + + For the sake of clarity, let’s stay with ‘git cat-file’, because it + + • is plumbing, and + + • was around even in the initial commit (it literally went only + through some 20 revisions as ‘cat-file.c’, was renamed to + ‘builtin/cat-file.c’ when made a builtin, and then saw less than 10 + versions). + + So, look into ‘builtin/cat-file.c’, search for ‘cmd_cat_file()’ and +look what it does. + + git_config(git_default_config); + if (argc != 3) + usage("git cat-file [-t|-s|-e|-p|] "); + if (get_sha1(argv[2], sha1)) + die("Not a valid object name %s", argv[2]); + + Let’s skip over the obvious details; the only really interesting part +here is the call to ‘get_sha1()’. It tries to interpret ‘argv[2]’ as an +object name, and if it refers to an object which is present in the +current repository, it writes the resulting SHA-1 into the variable +‘sha1’. + + Two things are interesting here: + + • ‘get_sha1()’ returns 0 on _success_. This might surprise some new + Git hackers, but there is a long tradition in UNIX to return + different negative numbers in case of different errors—and 0 on + success. + + • the variable ‘sha1’ in the function signature of ‘get_sha1()’ is + ‘unsigned char *’, but is actually expected to be a pointer to + ‘unsigned char[20]’. This variable will contain the 160-bit SHA-1 + of the given commit. Note that whenever a SHA-1 is passed as + ‘unsigned char *’, it is the binary representation, as opposed to + the ASCII representation in hex characters, which is passed as + ‘char *’. + + You will see both of these things throughout the code. + + Now, for the meat: + + case 0: + buf = read_object_with_reference(sha1, argv[1], &size, NULL); + + This is how you read a blob (actually, not only a blob, but any type +of object). To know how the function ‘read_object_with_reference()’ +actually works, find the source code for it (something like ‘git grep +read_object_with | grep ":[a-z]"’ in the Git repository), and read the +source. + + To find out how the result can be used, just read on in +‘cmd_cat_file()’: + + write_or_die(1, buf, size); + + Sometimes, you do not know where to look for a feature. In many such +cases, it helps to search through the output of ‘git log’, and then ‘git +show’ the corresponding commit. + + Example: If you know that there was some test case for ‘git bundle’, +but do not remember where it was (yes, you _could_ ‘git grep bundle t/’, +but that does not illustrate the point!): + + $ git log --no-merges t/ + + In the pager (‘less’), just search for "bundle", go a few lines back, +and see that it is in commit 18449ab0. Now just copy this object name, +and paste it into the command line + + $ git show 18449ab0 + + Voila. + + Another example: Find out what to do in order to make some script a +builtin: + + $ git log --no-merges --diff-filter=A builtin/*.c + + You see, Git is actually the best tool to find out about the source +of Git itself! + + +File: git.info, Node: Git Glossary, Next: Git Quick Reference, Prev: Hacking Git, Up: Top + +11 Git Glossary +*************** + +* Menu: + +* Git explained:: + + +File: git.info, Node: Git explained, Up: Git Glossary + +11.1 Git explained +================== + +alternate object database + Via the alternates mechanism, a repository (*note + [def_repository]::) can inherit part of its object database (*note + [def_object_database]::) from another object database, which is + called an "alternate". + +bare repository + A bare repository is normally an appropriately named directory + (*note [def_directory]::) with a ‘.git’ suffix that does not have a + locally checked-out copy of any of the files under revision + control. That is, all of the Git administrative and control files + that would normally be present in the hidden ‘.git’ sub-directory + are directly present in the ‘repository.git’ directory instead, and + no other files are present and checked out. Usually publishers of + public repositories make bare repositories available. + +blob object + Untyped object (*note [def_object]::), e.g. the contents of a + file. + +branch + A "branch" is a line of development. The most recent commit (*note + [def_commit]::) on a branch is referred to as the tip of that + branch. The tip of the branch is referenced by a branch head + (*note [def_head]::), which moves forward as additional development + is done on the branch. A single Git repository (*note + [def_repository]::) can track an arbitrary number of branches, but + your working tree (*note [def_working_tree]::) is associated with + just one of them (the "current" or "checked out" branch), and HEAD + (*note [def_HEAD]::) points to that branch. + +cache + Obsolete for: index (*note [def_index]::). + +chain + A list of objects, where each object (*note [def_object]::) in the + list contains a reference to its successor (for example, the + successor of a commit (*note [def_commit]::) could be one of its + parents (*note [def_parent]::)). + +changeset + BitKeeper/cvsps speak for "commit (*note [def_commit]::)". Since + Git does not store changes, but states, it really does not make + sense to use the term "changesets" with Git. + +checkout + The action of updating all or part of the working tree (*note + [def_working_tree]::) with a tree object (*note + [def_tree_object]::) or blob (*note [def_blob_object]::) from the + object database (*note [def_object_database]::), and updating the + index (*note [def_index]::) and HEAD (*note [def_HEAD]::) if the + whole working tree has been pointed at a new branch (*note + [def_branch]::). + +cherry-picking + In SCM (*note [def_SCM]::) jargon, "cherry pick" means to choose a + subset of changes out of a series of changes (typically commits) + and record them as a new series of changes on top of a different + codebase. In Git, this is performed by the "git cherry-pick" + command to extract the change introduced by an existing commit + (*note [def_commit]::) and to record it based on the tip of the + current branch (*note [def_branch]::) as a new commit. + +clean + A working tree (*note [def_working_tree]::) is clean, if it + corresponds to the revision (*note [def_revision]::) referenced by + the current head (*note [def_head]::). Also see "dirty (*note + [def_dirty]::)". + +commit + As a noun: A single point in the Git history; the entire history of + a project is represented as a set of interrelated commits. The + word "commit" is often used by Git in the same places other + revision control systems use the words "revision" or "version". + Also used as a short hand for commit object (*note + [def_commit_object]::). + + As a verb: The action of storing a new snapshot of the project’s + state in the Git history, by creating a new commit representing the + current state of the index (*note [def_index]::) and advancing HEAD + (*note [def_HEAD]::) to point at the new commit. + +commit object + An object (*note [def_object]::) which contains the information + about a particular revision (*note [def_revision]::), such as + parents (*note [def_parent]::), committer, author, date and the + tree object (*note [def_tree_object]::) which corresponds to the + top directory (*note [def_directory]::) of the stored revision. + +commit-ish (also committish) + A commit object (*note [def_commit_object]::) or an object (*note + [def_object]::) that can be recursively dereferenced to a commit + object. The following are all commit-ishes: a commit object, a tag + object (*note [def_tag_object]::) that points to a commit object, a + tag object that points to a tag object that points to a commit + object, etc. + +core Git + Fundamental data structures and utilities of Git. Exposes only + limited source code management tools. + +DAG + Directed acyclic graph. The commit objects (*note + [def_commit_object]::) form a directed acyclic graph, because they + have parents (directed), and the graph of commit objects is acyclic + (there is no chain (*note [def_chain]::) which begins and ends with + the same object (*note [def_object]::)). + +dangling object + An unreachable object (*note [def_unreachable_object]::) which is + not reachable (*note [def_reachable]::) even from other unreachable + objects; a dangling object has no references to it from any + reference or object (*note [def_object]::) in the repository (*note + [def_repository]::). + +detached HEAD + Normally the HEAD (*note [def_HEAD]::) stores the name of a branch + (*note [def_branch]::), and commands that operate on the history + HEAD represents operate on the history leading to the tip of the + branch the HEAD points at. However, Git also allows you to check + out (*note [def_checkout]::) an arbitrary commit (*note + [def_commit]::) that isn’t necessarily the tip of any particular + branch. The HEAD in such a state is called "detached". + + Note that commands that operate on the history of the current + branch (e.g. ‘git commit’ to build a new history on top of it) + still work while the HEAD is detached. They update the HEAD to + point at the tip of the updated history without affecting any + branch. Commands that update or inquire information _about_ the + current branch (e.g. ‘git branch --set-upstream-to’ that sets what + remote-tracking branch the current branch integrates with) + obviously do not work, as there is no (real) current branch to ask + about in this state. + +directory + The list you get with "ls" :-) + +dirty + A working tree (*note [def_working_tree]::) is said to be "dirty" + if it contains modifications which have not been committed (*note + [def_commit]::) to the current branch (*note [def_branch]::). + +evil merge + An evil merge is a merge (*note [def_merge]::) that introduces + changes that do not appear in any parent (*note [def_parent]::). + +fast-forward + A fast-forward is a special type of merge (*note [def_merge]::) + where you have a revision (*note [def_revision]::) and you are + "merging" another branch (*note [def_branch]::)’s changes that + happen to be a descendant of what you have. In such a case, you do + not make a new merge (*note [def_merge]::) commit (*note + [def_commit]::) but instead just update to his revision. This will + happen frequently on a remote-tracking branch (*note + [def_remote_tracking_branch]::) of a remote repository (*note + [def_repository]::). + +fetch + Fetching a branch (*note [def_branch]::) means to get the branch’s + head ref (*note [def_head_ref]::) from a remote repository (*note + [def_repository]::), to find out which objects are missing from the + local object database (*note [def_object_database]::), and to get + them, too. See also git-fetch(1) (git-fetch.html). + +file system + Linus Torvalds originally designed Git to be a user space file + system, i.e. the infrastructure to hold files and directories. + That ensured the efficiency and speed of Git. + +Git archive + Synonym for repository (*note [def_repository]::) (for arch + people). + +gitfile + A plain file ‘.git’ at the root of a working tree that points at + the directory that is the real repository. + +grafts + Grafts enables two otherwise different lines of development to be + joined together by recording fake ancestry information for commits. + This way you can make Git pretend the set of parents (*note + [def_parent]::) a commit (*note [def_commit]::) has is different + from what was recorded when the commit was created. Configured via + the ‘.git/info/grafts’ file. + + Note that the grafts mechanism is outdated and can lead to problems + transferring objects between repositories; see git-replace(1) + (git-replace.html) for a more flexible and robust system to do the + same thing. + +hash + In Git’s context, synonym for object name (*note + [def_object_name]::). + +head + A named reference (*note [def_ref]::) to the commit (*note + [def_commit]::) at the tip of a branch (*note [def_branch]::). + Heads are stored in a file in ‘$GIT_DIR/refs/heads/’ directory, + except when using packed refs. (See git-pack-refs(1) + (git-pack-refs.html).) + +HEAD + The current branch (*note [def_branch]::). In more detail: Your + working tree (*note [def_working_tree]::) is normally derived from + the state of the tree referred to by HEAD. HEAD is a reference to + one of the heads (*note [def_head]::) in your repository, except + when using a detached HEAD (*note [def_detached_HEAD]::), in which + case it directly references an arbitrary commit. + +head ref + A synonym for head (*note [def_head]::). + +hook + During the normal execution of several Git commands, call-outs are + made to optional scripts that allow a developer to add + functionality or checking. Typically, the hooks allow for a + command to be pre-verified and potentially aborted, and allow for a + post-notification after the operation is done. The hook scripts + are found in the ‘$GIT_DIR/hooks/’ directory, and are enabled by + simply removing the ‘.sample’ suffix from the filename. In earlier + versions of Git you had to make them executable. + +index + A collection of files with stat information, whose contents are + stored as objects. The index is a stored version of your working + tree (*note [def_working_tree]::). Truth be told, it can also + contain a second, and even a third version of a working tree, which + are used when merging (*note [def_merge]::). + +index entry + The information regarding a particular file, stored in the index + (*note [def_index]::). An index entry can be unmerged, if a merge + (*note [def_merge]::) was started, but not yet finished (i.e. if + the index contains multiple versions of that file). + +master + The default development branch (*note [def_branch]::). Whenever + you create a Git repository (*note [def_repository]::), a branch + named "master" is created, and becomes the active branch. In most + cases, this contains the local development, though that is purely + by convention and is not required. + +merge + As a verb: To bring the contents of another branch (*note + [def_branch]::) (possibly from an external repository (*note + [def_repository]::)) into the current branch. In the case where + the merged-in branch is from a different repository, this is done + by first fetching (*note [def_fetch]::) the remote branch and then + merging the result into the current branch. This combination of + fetch and merge operations is called a pull (*note [def_pull]::). + Merging is performed by an automatic process that identifies + changes made since the branches diverged, and then applies all + those changes together. In cases where changes conflict, manual + intervention may be required to complete the merge. + + As a noun: unless it is a fast-forward (*note + [def_fast_forward]::), a successful merge results in the creation + of a new commit (*note [def_commit]::) representing the result of + the merge, and having as parents (*note [def_parent]::) the tips of + the merged branches (*note [def_branch]::). This commit is + referred to as a "merge commit", or sometimes just a "merge". + +object + The unit of storage in Git. It is uniquely identified by the SHA-1 + (*note [def_SHA1]::) of its contents. Consequently, an object + cannot be changed. + +object database + Stores a set of "objects", and an individual object (*note + [def_object]::) is identified by its object name (*note + [def_object_name]::). The objects usually live in + ‘$GIT_DIR/objects/’. + +object identifier + Synonym for object name (*note [def_object_name]::). + +object name + The unique identifier of an object (*note [def_object]::). The + object name is usually represented by a 40 character hexadecimal + string. Also colloquially called SHA-1 (*note [def_SHA1]::). + +object type + One of the identifiers "commit (*note [def_commit_object]::)", + "tree (*note [def_tree_object]::)", "tag (*note + [def_tag_object]::)" or "blob (*note [def_blob_object]::)" + describing the type of an object (*note [def_object]::). + +octopus + To merge (*note [def_merge]::) more than two branches (*note + [def_branch]::). + +origin + The default upstream repository (*note [def_repository]::). Most + projects have at least one upstream project which they track. By + default _origin_ is used for that purpose. New upstream updates + will be fetched into remote-tracking branches (*note + [def_remote_tracking_branch]::) named + origin/name-of-upstream-branch, which you can see using ‘git branch + -r’. + +overlay + Only update and add files to the working directory, but don’t + delete them, similar to how _cp -R_ would update the contents in + the destination directory. This is the default mode in a checkout + (*note [def_checkout]::) when checking out files from the index + (*note [def_index]::) or a tree-ish (*note [def_tree-ish]::). In + contrast, no-overlay mode also deletes tracked files not present in + the source, similar to _rsync –delete_. + +pack + A set of objects which have been compressed into one file (to save + space or to transmit them efficiently). + +pack index + The list of identifiers, and other information, of the objects in a + pack (*note [def_pack]::), to assist in efficiently accessing the + contents of a pack. + +pathspec + Pattern used to limit paths in Git commands. + + Pathspecs are used on the command line of "git ls-files", "git + ls-tree", "git add", "git grep", "git diff", "git checkout", and + many other commands to limit the scope of operations to some subset + of the tree or worktree. See the documentation of each command for + whether paths are relative to the current directory or toplevel. + The pathspec syntax is as follows: + + • any path matches itself + + • the pathspec up to the last slash represents a directory + prefix. The scope of that pathspec is limited to that + subtree. + + • the rest of the pathspec is a pattern for the remainder of the + pathname. Paths relative to the directory prefix will be + matched against that pattern using fnmatch(3); in particular, + _*_ and _?_ _can_ match directory separators. + + For example, Documentation/*.jpg will match all .jpg files in the + Documentation subtree, including + Documentation/chapter_1/figure_1.jpg. + + A pathspec that begins with a colon ‘:’ has special meaning. In + the short form, the leading colon ‘:’ is followed by zero or more + "magic signature" letters (which optionally is terminated by + another colon ‘:’), and the remainder is the pattern to match + against the path. The "magic signature" consists of ASCII symbols + that are neither alphanumeric, glob, regex special characters nor + colon. The optional colon that terminates the "magic signature" + can be omitted if the pattern begins with a character that does not + belong to "magic signature" symbol set and is not a colon. + + In the long form, the leading colon ‘:’ is followed by an open + parenthesis ‘(’, a comma-separated list of zero or more "magic + words", and a close parentheses ‘)’, and the remainder is the + pattern to match against the path. + + A pathspec with only a colon means "there is no pathspec". This + form should not be combined with other pathspec. + + top + The magic word ‘top’ (magic signature: ‘/’) makes the pattern + match from the root of the working tree, even when you are + running the command from inside a subdirectory. + + literal + Wildcards in the pattern such as ‘*’ or ‘?’ are treated as + literal characters. + + icase + Case insensitive match. + + glob + Git treats the pattern as a shell glob suitable for + consumption by fnmatch(3) with the FNM_PATHNAME flag: + wildcards in the pattern will not match a / in the pathname. + For example, "Documentation/*.html" matches + "Documentation/git.html" but not "Documentation/ppc/ppc.html" + or "tools/perf/Documentation/perf.html". + + Two consecutive asterisks ("‘**’") in patterns matched against + full pathname may have special meaning: + + • A leading "‘**’" followed by a slash means match in all + directories. For example, "‘**/foo’" matches file or + directory "‘foo’" anywhere, the same as pattern "‘foo’". + "‘**/foo/bar’" matches file or directory "‘bar’" anywhere + that is directly under directory "‘foo’". + + • A trailing "‘/**’" matches everything inside. For + example, "‘abc/**’" matches all files inside directory + "abc", relative to the location of the ‘.gitignore’ file, + with infinite depth. + + • A slash followed by two consecutive asterisks then a + slash matches zero or more directories. For example, + "‘a/**/b’" matches "‘a/b’", "‘a/x/b’", "‘a/x/y/b’" and so + on. + + • Other consecutive asterisks are considered invalid. + + Glob magic is incompatible with literal magic. + + attr + After ‘attr:’ comes a space separated list of "attribute + requirements", all of which must be met in order for the path + to be considered a match; this is in addition to the usual + non-magic pathspec pattern matching. See gitattributes(5) + (gitattributes.html). + + Each of the attribute requirements for the path takes one of + these forms: + + • "‘ATTR’" requires that the attribute ‘ATTR’ be set. + + • "‘-ATTR’" requires that the attribute ‘ATTR’ be unset. + + • "‘ATTR=VALUE’" requires that the attribute ‘ATTR’ be set + to the string ‘VALUE’. + + • "‘!ATTR’" requires that the attribute ‘ATTR’ be + unspecified. + + Note that when matching against a tree object, attributes + are still obtained from working tree, not from the given + tree object. + + exclude + After a path matches any non-exclude pathspec, it will be run + through all exclude pathspecs (magic signature: ‘!’ or its + synonym ‘^’). If it matches, the path is ignored. When there + is no non-exclude pathspec, the exclusion is applied to the + result set as if invoked without any pathspec. + +parent + A commit object (*note [def_commit_object]::) contains a (possibly + empty) list of the logical predecessor(s) in the line of + development, i.e. its parents. + +pickaxe + The term pickaxe (*note [def_pickaxe]::) refers to an option to the + diffcore routines that help select changes that add or delete a + given text string. With the ‘--pickaxe-all’ option, it can be used + to view the full changeset (*note [def_changeset]::) that + introduced or removed, say, a particular line of text. See + git-diff(1) (git-diff.html). + +plumbing + Cute name for core Git (*note [def_core_git]::). + +porcelain + Cute name for programs and program suites depending on core Git + (*note [def_core_git]::), presenting a high level access to core + Git. Porcelains expose more of a SCM (*note [def_SCM]::) interface + than the plumbing (*note [def_plumbing]::). + +per-worktree ref + Refs that are per-worktree (*note [def_working_tree]::), rather + than global. This is presently only HEAD (*note [def_HEAD]::) and + any refs that start with ‘refs/bisect/’, but might later include + other unusual refs. + +pseudoref + Pseudorefs are a class of files under ‘$GIT_DIR’ which behave like + refs for the purposes of rev-parse, but which are treated specially + by git. Pseudorefs both have names that are all-caps, and always + start with a line consisting of a SHA-1 (*note [def_SHA1]::) + followed by whitespace. So, HEAD is not a pseudoref, because it is + sometimes a symbolic ref. They might optionally contain some + additional data. ‘MERGE_HEAD’ and ‘CHERRY_PICK_HEAD’ are examples. + Unlike per-worktree refs (*note [def_per_worktree_ref]::), these + files cannot be symbolic refs, and never have reflogs. They also + cannot be updated through the normal ref update machinery. + Instead, they are updated by directly writing to the files. + However, they can be read as if they were refs, so ‘git rev-parse + MERGE_HEAD’ will work. + +pull + Pulling a branch (*note [def_branch]::) means to fetch (*note + [def_fetch]::) it and merge (*note [def_merge]::) it. See also + git-pull(1) (git-pull.html). + +push + Pushing a branch (*note [def_branch]::) means to get the branch’s + head ref (*note [def_head_ref]::) from a remote repository (*note + [def_repository]::), find out if it is an ancestor to the branch’s + local head ref, and in that case, putting all objects, which are + reachable (*note [def_reachable]::) from the local head ref, and + which are missing from the remote repository, into the remote + object database (*note [def_object_database]::), and updating the + remote head ref. If the remote head (*note [def_head]::) is not an + ancestor to the local head, the push fails. + +reachable + All of the ancestors of a given commit (*note [def_commit]::) are + said to be "reachable" from that commit. More generally, one + object (*note [def_object]::) is reachable from another if we can + reach the one from the other by a chain (*note [def_chain]::) that + follows tags (*note [def_tag]::) to whatever they tag, commits + (*note [def_commit_object]::) to their parents or trees, and trees + (*note [def_tree_object]::) to the trees or blobs (*note + [def_blob_object]::) that they contain. + +rebase + To reapply a series of changes from a branch (*note [def_branch]::) + to a different base, and reset the head (*note [def_head]::) of + that branch to the result. + +ref + A name that begins with ‘refs/’ (e.g. ‘refs/heads/master’) that + points to an object name (*note [def_object_name]::) or another ref + (the latter is called a symbolic ref (*note [def_symref]::)). For + convenience, a ref can sometimes be abbreviated when used as an + argument to a Git command; see gitrevisions(7) (gitrevisions.html) + for details. Refs are stored in the repository (*note + [def_repository]::). + + The ref namespace is hierarchical. Different subhierarchies are + used for different purposes (e.g. the ‘refs/heads/’ hierarchy is + used to represent local branches). + + There are a few special-purpose refs that do not begin with + ‘refs/’. The most notable example is ‘HEAD’. + +reflog + A reflog shows the local "history" of a ref. In other words, it + can tell you what the 3rd last revision in _this_ repository was, + and what was the current state in _this_ repository, yesterday + 9:14pm. See git-reflog(1) (git-reflog.html) for details. + +refspec + A "refspec" is used by fetch (*note [def_fetch]::) and push (*note + [def_push]::) to describe the mapping between remote ref (*note + [def_ref]::) and local ref. + +remote repository + A repository (*note [def_repository]::) which is used to track the + same project but resides somewhere else. To communicate with + remotes, see fetch (*note [def_fetch]::) or push (*note + [def_push]::). + +remote-tracking branch + A ref (*note [def_ref]::) that is used to follow changes from + another repository (*note [def_repository]::). It typically looks + like _refs/remotes/foo/bar_ (indicating that it tracks a branch + named _bar_ in a remote named _foo_), and matches the + right-hand-side of a configured fetch refspec (*note + [def_refspec]::). A remote-tracking branch should not contain + direct modifications or have local commits made to it. + +repository + A collection of refs (*note [def_ref]::) together with an object + database (*note [def_object_database]::) containing all objects + which are reachable (*note [def_reachable]::) from the refs, + possibly accompanied by meta data from one or more porcelains + (*note [def_porcelain]::). A repository can share an object + database with other repositories via alternates mechanism (*note + [def_alternate_object_database]::). + +resolve + The action of fixing up manually what a failed automatic merge + (*note [def_merge]::) left behind. + +revision + Synonym for commit (*note [def_commit]::) (the noun). + +rewind + To throw away part of the development, i.e. to assign the head + (*note [def_head]::) to an earlier revision (*note + [def_revision]::). + +SCM + Source code management (tool). + +SHA-1 + "Secure Hash Algorithm 1"; a cryptographic hash function. In the + context of Git used as a synonym for object name (*note + [def_object_name]::). + +shallow clone + Mostly a synonym to shallow repository (*note + [def_shallow_repository]::) but the phrase makes it more explicit + that it was created by running ‘git clone --depth=...’ command. + +shallow repository + A shallow repository (*note [def_repository]::) has an incomplete + history some of whose commits (*note [def_commit]::) have parents + (*note [def_parent]::) cauterized away (in other words, Git is told + to pretend that these commits do not have the parents, even though + they are recorded in the commit object (*note + [def_commit_object]::)). This is sometimes useful when you are + interested only in the recent history of a project even though the + real history recorded in the upstream is much larger. A shallow + repository is created by giving the ‘--depth’ option to + git-clone(1) (git-clone.html), and its history can be later + deepened with git-fetch(1) (git-fetch.html). + +stash entry + An object (*note [def_object]::) used to temporarily store the + contents of a dirty (*note [def_dirty]::) working directory and the + index for future reuse. + +submodule + A repository (*note [def_repository]::) that holds the history of a + separate project inside another repository (the latter of which is + called superproject (*note [def_superproject]::)). + +superproject + A repository (*note [def_repository]::) that references + repositories of other projects in its working tree as submodules + (*note [def_submodule]::). The superproject knows about the names + of (but does not hold copies of) commit objects of the contained + submodules. + +symref + Symbolic reference: instead of containing the SHA-1 (*note + [def_SHA1]::) id itself, it is of the format _ref: refs/some/thing_ + and when referenced, it recursively dereferences to this reference. + _HEAD (*note [def_HEAD]::)_ is a prime example of a symref. + Symbolic references are manipulated with the git-symbolic-ref(1) + (git-symbolic-ref.html) command. + +tag + A ref (*note [def_ref]::) under ‘refs/tags/’ namespace that points + to an object of an arbitrary type (typically a tag points to either + a tag (*note [def_tag_object]::) or a commit object (*note + [def_commit_object]::)). In contrast to a head (*note + [def_head]::), a tag is not updated by the ‘commit’ command. A Git + tag has nothing to do with a Lisp tag (which would be called an + object type (*note [def_object_type]::) in Git’s context). A tag + is most typically used to mark a particular point in the commit + ancestry chain (*note [def_chain]::). + +tag object + An object (*note [def_object]::) containing a ref (*note + [def_ref]::) pointing to another object, which can contain a + message just like a commit object (*note [def_commit_object]::). + It can also contain a (PGP) signature, in which case it is called a + "signed tag object". + +topic branch + A regular Git branch (*note [def_branch]::) that is used by a + developer to identify a conceptual line of development. Since + branches are very easy and inexpensive, it is often desirable to + have several small branches that each contain very well defined + concepts or small incremental yet related changes. + +tree + Either a working tree (*note [def_working_tree]::), or a tree + object (*note [def_tree_object]::) together with the dependent blob + (*note [def_blob_object]::) and tree objects (i.e. a stored + representation of a working tree). + +tree object + An object (*note [def_object]::) containing a list of file names + and modes along with refs to the associated blob and/or tree + objects. A tree (*note [def_tree]::) is equivalent to a directory + (*note [def_directory]::). + +tree-ish (also treeish) + A tree object (*note [def_tree_object]::) or an object (*note + [def_object]::) that can be recursively dereferenced to a tree + object. Dereferencing a commit object (*note + [def_commit_object]::) yields the tree object corresponding to the + revision (*note [def_revision]::)’s top directory (*note + [def_directory]::). The following are all tree-ishes: a commit-ish + (*note [def_commit-ish]::), a tree object, a tag object (*note + [def_tag_object]::) that points to a tree object, a tag object that + points to a tag object that points to a tree object, etc. + +unmerged index + An index (*note [def_index]::) which contains unmerged index + entries (*note [def_index_entry]::). + +unreachable object + An object (*note [def_object]::) which is not reachable (*note + [def_reachable]::) from a branch (*note [def_branch]::), tag (*note + [def_tag]::), or any other reference. + +upstream branch + The default branch (*note [def_branch]::) that is merged into the + branch in question (or the branch in question is rebased onto). It + is configured via branch..remote and branch..merge. If + the upstream branch of _A_ is _origin/B_ sometimes we say "_A_ is + tracking _origin/B_". + +working tree + The tree of actual checked out files. The working tree normally + contains the contents of the HEAD (*note [def_HEAD]::) commit’s + tree, plus any local changes that you have made but not yet + committed. + + +File: git.info, Node: Git Quick Reference, Next: Notes and todo list for this manual, Prev: Git Glossary, Up: Top + +Appendix A Git Quick Reference +****************************** + +This is a quick summary of the major commands; the previous chapters +explain how these work in more detail. + +* Menu: + +* Creating a new repository: Creating a new repository <1>. +* Managing branches:: +* Exploring history:: +* Making changes:: +* Merging:: +* Sharing your changes:: +* Repository maintenance:: + + +File: git.info, Node: Creating a new repository <1>, Next: Managing branches, Up: Git Quick Reference + +A.1 Creating a new repository +============================= + +From a tarball: + + $ tar xzf project.tar.gz + $ cd project + $ git init + Initialized empty Git repository in .git/ + $ git add . + $ git commit + + From a remote repository: + + $ git clone git://example.com/pub/project.git + $ cd project + + +File: git.info, Node: Managing branches, Next: Exploring history, Prev: Creating a new repository <1>, Up: Git Quick Reference + +A.2 Managing branches +===================== + + $ git branch # list all local branches in this repo + $ git switch test # switch working directory to branch "test" + $ git branch new # create branch "new" starting at current HEAD + $ git branch -d new # delete branch "new" + + Instead of basing a new branch on current HEAD (the default), use: + + $ git branch new test # branch named "test" + $ git branch new v2.6.15 # tag named v2.6.15 + $ git branch new HEAD^ # commit before the most recent + $ git branch new HEAD^^ # commit before that + $ git branch new test~10 # ten commits before tip of branch "test" + + Create and switch to a new branch at the same time: + + $ git switch -c new v2.6.15 + + Update and examine branches from the repository you cloned from: + + $ git fetch # update + $ git branch -r # list + origin/master + origin/next + ... + $ git switch -c masterwork origin/master + + Fetch a branch from a different repository, and give it a new name in +your repository: + + $ git fetch git://example.com/project.git theirbranch:mybranch + $ git fetch git://example.com/project.git v2.6.15:mybranch + + Keep a list of repositories you work with regularly: + + $ git remote add example git://example.com/project.git + $ git remote # list remote repositories + example + origin + $ git remote show example # get details + * remote example + URL: git://example.com/project.git + Tracked remote branches + master + next + ... + $ git fetch example # update branches from example + $ git branch -r # list all remote branches + + +File: git.info, Node: Exploring history, Next: Making changes, Prev: Managing branches, Up: Git Quick Reference + +A.3 Exploring history +===================== + + $ gitk # visualize and browse history + $ git log # list all commits + $ git log src/ # ...modifying src/ + $ git log v2.6.15..v2.6.16 # ...in v2.6.16, not in v2.6.15 + $ git log master..test # ...in branch test, not in branch master + $ git log test..master # ...in branch master, but not in test + $ git log test...master # ...in one branch, not in both + $ git log -S'foo()' # ...where difference contain "foo()" + $ git log --since="2 weeks ago" + $ git log -p # show patches as well + $ git show # most recent commit + $ git diff v2.6.15..v2.6.16 # diff between two tagged versions + $ git diff v2.6.15..HEAD # diff with current head + $ git grep "foo()" # search working directory for "foo()" + $ git grep v2.6.15 "foo()" # search old tree for "foo()" + $ git show v2.6.15:a.txt # look at old version of a.txt + + Search for regressions: + + $ git bisect start + $ git bisect bad # current version is bad + $ git bisect good v2.6.13-rc2 # last known good revision + Bisecting: 675 revisions left to test after this + # test here, then: + $ git bisect good # if this revision is good, or + $ git bisect bad # if this revision is bad. + # repeat until done. + + +File: git.info, Node: Making changes, Next: Merging, Prev: Exploring history, Up: Git Quick Reference + +A.4 Making changes +================== + +Make sure Git knows who to blame: + + $ cat >>~/.gitconfig <<\EOF + [user] + name = Your Name Comes Here + email = you@yourdomain.example.com + EOF + + Select file contents to include in the next commit, then make the +commit: + + $ git add a.txt # updated file + $ git add b.txt # new file + $ git rm c.txt # old file + $ git commit + + Or, prepare and create the commit in one step: + + $ git commit d.txt # use latest content only of d.txt + $ git commit -a # use latest content of all tracked files + + +File: git.info, Node: Merging, Next: Sharing your changes, Prev: Making changes, Up: Git Quick Reference + +A.5 Merging +=========== + + $ git merge test # merge branch "test" into the current branch + $ git pull git://example.com/project.git master + # fetch and merge in remote branch + $ git pull . test # equivalent to git merge test + + +File: git.info, Node: Sharing your changes, Next: Repository maintenance, Prev: Merging, Up: Git Quick Reference + +A.6 Sharing your changes +======================== + +Importing or exporting patches: + + $ git format-patch origin..HEAD # format a patch for each commit + # in HEAD but not in origin + $ git am mbox # import patches from the mailbox "mbox" + + Fetch a branch in a different Git repository, then merge into the +current branch: + + $ git pull git://example.com/project.git theirbranch + + Store the fetched branch into a local branch before merging into the +current branch: + + $ git pull git://example.com/project.git theirbranch:mybranch + + After creating commits on a local branch, update the remote branch +with your commits: + + $ git push ssh://example.com/project.git mybranch:theirbranch + + When remote and local branch are both named "test": + + $ git push ssh://example.com/project.git test + + Shortcut version for a frequently used remote repository: + + $ git remote add example ssh://example.com/project.git + $ git push example test + + +File: git.info, Node: Repository maintenance, Prev: Sharing your changes, Up: Git Quick Reference + +A.7 Repository maintenance +========================== + +Check for corruption: + + $ git fsck + + Recompress, remove unused cruft: + + $ git gc + + +File: git.info, Node: Notes and todo list for this manual, Prev: Git Quick Reference, Up: Top + +Appendix B Notes and todo list for this manual +********************************************** + +* Menu: + +* Todo list:: + + +File: git.info, Node: Todo list, Up: Notes and todo list for this manual + +B.1 Todo list +============= + +This is a work in progress. + + The basic requirements: + + • It must be readable in order, from beginning to end, by someone + intelligent with a basic grasp of the UNIX command line, but + without any special knowledge of Git. If necessary, any other + prerequisites should be specifically mentioned as they arise. + + • Whenever possible, section headings should clearly describe the + task they explain how to do, in language that requires no more + knowledge than necessary: for example, "importing patches into a + project" rather than "the ‘git am’ command" + + Think about how to create a clear chapter dependency graph that will +allow people to get to important topics without necessarily reading +everything in between. + + Scan ‘Documentation/’ for other stuff left out; in particular: + + • howto’s + + • some of ‘technical/’? + + • hooks + + • list of commands in git(1) (git.html) + + Scan email archives for other stuff left out + + Scan man pages to see if any assume more background than this manual +provides. + + Add more good examples. Entire sections of just cookbook examples +might be a good idea; maybe make an "advanced examples" section a +standard end-of-chapter section? + + Include cross-references to the glossary, where appropriate. + + Add a section on working with other version control systems, +including CVS, Subversion, and just imports of series of release +tarballs. + + Write a chapter on using plumbing and writing scripts. + + Alternates, clone -reference, etc. + + More on recovery from repository corruption. See: +https://lore.kernel.org/git/Pine.LNX.4.64.0702272039540.12485@woody.linux-foundation.org/ +(https://lore.kernel.org/git/Pine.LNX.4.64.0702272039540.12485@woody.linux-foundation.org/) +https://lore.kernel.org/git/Pine.LNX.4.64.0702141033400.3604@woody.linux-foundation.org/ +(https://lore.kernel.org/git/Pine.LNX.4.64.0702141033400.3604@woody.linux-foundation.org/) + + + +Tag Table: +Node: Top212 +Node: idm43164 +Node: Repositories and Branches4465 +Node: How to get a Git repository5070 +Node: How to check out a different version of a project6331 +Node: Understanding History; Commits8890 +Node: Understanding history; commits; parents; and reachability11147 +Node: Understanding history; History diagrams12483 +Node: Understanding history; What is a branch?13224 +Node: Manipulating branches13963 +Node: Examining an old version without creating a new branch15544 +Node: Examining branches from a remote repository17105 +Node: Naming branches; tags; and other references18861 +Node: Updating a repository with git fetch20420 +Node: Fetching branches from other repositories21093 +Node: Exploring Git history22786 +Node: How to use bisect to find a regression23583 +Node: Naming commits26582 +Node: Creating tags28944 +Node: Browsing revisions29512 +Node: Generating diffs31126 +Node: Viewing old file versions31882 +Node: Examples32449 +Node: Counting the number of commits on a branch32902 +Node: Check whether two branches point at the same history33546 +Node: Find first tagged version including a given fix34613 +Node: Showing commits unique to a given branch37277 +Node: Creating a changelog and tarball for a software release39049 +Node: Finding commits referencing a file with given content40885 +Node: Developing with Git41679 +Node: Telling Git your name42152 +Node: Creating a new repository42961 +Node: How to make a commit43476 +Node: Creating good commit messages46371 +Node: Ignoring files47075 +Node: How to merge49098 +Node: Resolving a merge51148 +Node: Getting conflict-resolution help during a merge52517 +Node: Undoing a merge56290 +Node: Fast-forward merges56938 +Node: Fixing mistakes57648 +Node: Fixing a mistake with a new commit58787 +Node: Fixing a mistake by rewriting history59757 +Node: Checking out an old version of a file60906 +Node: Temporarily setting aside work in progress61784 +Node: Ensuring good performance62830 +Node: Ensuring reliability63453 +Node: Checking the repository for corruption63694 +Node: Recovering lost changes64907 +Node: Reflogs65147 +Node: Examining dangling objects67010 +Node: Sharing development with others68647 +Node: Getting updates with git pull69089 +Node: Submitting patches to a project71302 +Node: Importing patches to a project72606 +Node: Public Git repositories73901 +Node: Setting up a public repository76446 +Node: Exporting a Git repository via the Git protocol77232 +Node: Exporting a git repository via HTTP78351 +Node: Pushing changes to a public repository79643 +Node: What to do when a push fails81787 +Node: Setting up a shared repository84199 +Node: Allowing web browsing of a repository85848 +Node: How to get a Git repository with minimal history86654 +Node: Examples <1>87717 +Node: Maintaining topic branches for a Linux subsystem maintainer87952 +Node: Rewriting history and maintaining patch series96755 +Node: Creating the perfect patch series97644 +Node: Keeping a patch series up to date using git rebase99127 +Node: Rewriting a single commit101762 +Node: Reordering or selecting from a patch series102587 +Node: Using interactive rebases103261 +Node: Other tools105631 +Node: Problems with rewriting history105984 +Node: Why bisecting merge commits can be harder than bisecting linear history107811 +Node: Advanced branch management110808 +Node: Fetching individual branches111175 +Node: git fetch and fast-forwards112293 +Node: Forcing git fetch to do non-fast-forward updates113765 +Node: Configuring remote-tracking branches114557 +Node: Git concepts116297 +Node: The Object Database116786 +Node: Commit Object119304 +Node: Tree Object121890 +Node: Blob Object123746 +Node: Trust124995 +Node: Tag Object126433 +Node: How Git stores objects efficiently; pack files127511 +Node: Dangling objects129556 +Node: Recovering from repository corruption133244 +Node: The index137453 +Node: Submodules140334 +Node: Pitfalls with submodules146399 +Node: Low-level Git operations148958 +Node: Object access and manipulation149539 +Node: The Workflow150418 +Node: working directory → index151307 +Node: index → object database153010 +Node: object database → index153601 +Node: index → working directory154258 +Node: Tying it all together155462 +Node: Examining the data158326 +Node: Merging multiple trees159445 +Node: Merging multiple trees; continued161275 +Node: Hacking Git164309 +Node: Object storage format164641 +Node: A birds-eye view of Git’s source code166225 +Node: Git Glossary175147 +Node: Git explained175305 +Ref: [def_alternate_object_database]175403 +Ref: [def_bare_repository]175654 +Ref: [def_blob_object]176236 +Ref: [def_branch]176328 +Ref: [def_cache]176940 +Ref: [def_chain]176995 +Ref: [def_changeset]177248 +Ref: [def_checkout]177449 +Ref: [def_cherry-picking]177877 +Ref: [def_clean]178369 +Ref: [def_commit]178603 +Ref: [def_commit_object]179245 +Ref: [def_commit-ish]179604 +Ref: [def_core_git]180007 +Ref: [def_DAG]180129 +Ref: [def_dangling_object]180454 +Ref: [def_detached_HEAD]180779 +Ref: [def_directory]181857 +Ref: [def_dirty]181904 +Ref: [def_evil_merge]182120 +Ref: [def_fast_forward]182270 +Ref: [def_fetch]182850 +Ref: [def_file_system]183203 +Ref: [def_git_archive]183404 +Ref: [def_gitfile]183496 +Ref: [def_grafts]183627 +Ref: [def_hash]184255 +Ref: [def_head]184344 +Ref: [def_HEAD]184642 +Ref: [def_head_ref]185057 +Ref: [def_hook]185113 +Ref: [def_index]185665 +Ref: [def_index_entry]186003 +Ref: [def_master]186286 +Ref: [def_merge]186617 +Ref: [def_object]187767 +Ref: [def_object_database]187940 +Ref: [def_object_identifier]188168 +Ref: [def_object_name]188245 +Ref: [def_object_type]188464 +Ref: [def_octopus]188724 +Ref: [def_origin]188821 +Ref: [def_overlay]189229 +Ref: [def_pack]189709 +Ref: [def_pack_index]189832 +Ref: [def_pathspec]190013 +Ref: [def_parent]195356 +Ref: [def_pickaxe]195535 +Ref: [def_plumbing]195925 +Ref: [def_porcelain]195989 +Ref: [def_per_worktree_ref]196261 +Ref: [def_pseudoref]196519 +Ref: [def_pull]197410 +Ref: [def_push]197586 +Ref: [def_reachable]198210 +Ref: [def_rebase]198749 +Ref: [def_ref]198931 +Ref: [def_reflog]199697 +Ref: [def_refspec]199977 +Ref: [def_remote]200160 +Ref: [def_remote_tracking_branch]200399 +Ref: [def_repository]200876 +Ref: [def_resolve]201337 +Ref: [def_revision]201454 +Ref: [def_rewind]201523 +Ref: [def_SCM]201680 +Ref: [def_SHA1]201721 +Ref: [def_shallow_clone]201887 +Ref: [def_shallow_repository]202097 +Ref: [def_stash]202846 +Ref: [def_submodule]203029 +Ref: [def_superproject]203241 +Ref: [def_symref]203545 +Ref: [def_tag]203936 +Ref: [def_tag_object]204545 +Ref: [def_topic_branch]204854 +Ref: [def_tree]205198 +Ref: [def_tree_object]205450 +Ref: [def_tree-ish]205703 +Ref: [def_unmerged_index]206327 +Ref: [def_unreachable_object]206451 +Ref: [def_upstream_branch]206655 +Ref: [def_working_tree]206987 +Node: Git Quick Reference207223 +Node: Creating a new repository <1>207713 +Node: Managing branches208146 +Node: Exploring history210068 +Node: Making changes211701 +Node: Merging212411 +Node: Sharing your changes212786 +Node: Repository maintenance213906 +Node: Notes and todo list for this manual214157 +Node: Todo list214376 + +End Tag Table + + +Local Variables: +coding: utf-8 +End: diff --git a/assets/info/grep.info.gz b/assets/info/grep.info.gz new file mode 100644 index 00000000..a2bf786e Binary files /dev/null and b/assets/info/grep.info.gz differ diff --git a/assets/info/gzip.info.gz b/assets/info/gzip.info.gz new file mode 100644 index 00000000..192eb4d8 Binary files /dev/null and b/assets/info/gzip.info.gz differ diff --git a/assets/info/info.info.gz b/assets/info/info.info.gz new file mode 100644 index 00000000..3651c896 Binary files /dev/null and b/assets/info/info.info.gz differ diff --git a/assets/info/latex2e.info b/assets/info/latex2e.info new file mode 100644 index 00000000..a547df1e --- /dev/null +++ b/assets/info/latex2e.info @@ -0,0 +1,21120 @@ +This is latex2e.info, produced by makeinfo version 6.8 from +latex2e.texi. + +This document is an unofficial reference manual for LaTeX, a document +preparation system, version of January 2022. + + This manual was originally translated from 'LATEX.HLP' v1.0a in the +VMS Help Library. The pre-translation version was written by George D. +Greenwade of Sam Houston State University. The LaTeX 2.09 version was +written by Stephen Gilmore. The LaTeX2e version was adapted from this +by Torsten Martinsen. Karl Berry made further updates and additions, +and gratefully acknowledges using 'Hypertext Help with LaTeX', by +Sheldon Green, and 'LaTeX Command Summary' (for LaTeX 2.09) by L. Botway +and C. Biemesderfer (published by the TeX Users Group as 'TeXniques' +number 10), as reference material. We also gratefully acknowledge +additional material appearing in latex2e-reference by Martin Herbert +Dietze. (From these references no text was directly copied.) + + Copyright 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, +2017, 2018, 2019, 2020, 2021, 2022 Karl Berry. +Copyright 1988, 1994, 2007 Stephen Gilmore. +Copyright 1994, 1995, 1996 Torsten Martinsen. + + Permission is granted to make and distribute verbatim copies of this +manual provided the copyright notice and this permission notice are +preserved on all copies. + + Permission is granted to copy and distribute modified versions of +this manual under the conditions for verbatim copying, provided that the +entire resulting derived work is distributed under the terms of a +permission notice identical to this one. + + Permission is granted to copy and distribute translations of this +manual into another language, under the above conditions for modified +versions. +INFO-DIR-SECTION TeX +START-INFO-DIR-ENTRY +* LaTeX2e: (latex2e). Unofficial LaTeX reference manual. +END-INFO-DIR-ENTRY + + +File: latex2e.info, Node: Top, Next: About this document, Up: (dir) + +LaTeX2e: An unofficial reference manual +*************************************** + +This document is an unofficial reference manual (version of January +2022) for LaTeX2e, a document preparation system. + +* Menu: + +* About this document:: Bug reporting, etc. +* Overview:: What is LaTeX? +* Document classes:: Some of the various classes available. +* Fonts:: Italic, bold, typewriter, etc. +* Layout:: Controlling the page layout. +* Sectioning:: Parts, Chapters, Sections, etc. +* Cross references:: Automatic referencing. +* Environments:: Such as enumerate & itemize. +* Line breaking:: Influencing line breaks. +* Page breaking:: Influencing page breaks. +* Footnotes:: How to produce footnotes. +* Definitions:: Define your own commands, etc. +* Counters:: Internal counters used by LaTeX. +* Lengths:: The length commands. +* Making paragraphs:: Paragraph commands. +* Math formulas:: How to create mathematical formulas. +* Modes:: Paragraph, Math or LR modes. +* Page styles:: Various styles of page layout. +* Spaces:: Horizontal and vertical space. +* Boxes:: Making boxes. +* Color:: Defining and using colors. +* Graphics:: Importing graphics from outside LaTeX. +* Special insertions:: Inserting reserved and special characters. +* Splitting the input:: Dealing with big files by splitting. +* Front/back matter:: Tables of contents, glossaries, indexes. +* Letters:: The 'letter' class. +* Input/output:: User interaction. +* Command line interface:: Common command-line options. +* Document templates:: Starter templates for various document classes. +* Index:: General index. + + +File: latex2e.info, Node: About this document, Next: Overview, Prev: Top, Up: Top + +1 About this document +********************* + +This is an unofficial reference manual for the LaTeX2e document +preparation system, which is a macro package for the TeX typesetting +program (*note Overview::). + + This document's home page is ; it has separate +web pages for each topic. Alternatively. + has the entire document on a +single page. For other output formats, the sources, and plenty more +information, see . + + In this document, we will mostly just use 'LaTeX' rather than +'LaTeX2e', since the previous version of LaTeX (2.09) was frozen decades +ago. + + LaTeX is maintained by a group of volunteers +(). The official documentation written by +the LaTeX project is available from their web site. The present +document is completely unofficial and has not been written or reviewed +by the LaTeX maintainers. Do not send bug reports or anything else +about this document to them. Instead, please send all comments to +. + + This document is a reference, not a tutorial. There is a vast array +of other information available about LaTeX, at all levels. Here are a +few introductions. + + + Two pages of recommended references to LaTeX documentation. + + + Writing your first document, with a bit of both text and math. + + + A longer introduction to LaTeX, translated to many languages. + + + Introduction to the TeX system, including LaTeX, with further + references. + + +File: latex2e.info, Node: Overview, Next: Document classes, Prev: About this document, Up: Top + +2 Overview of LaTeX +******************* + +LaTeX is a system for typesetting documents. It was originally created +by Leslie Lamport in 1984, but has been maintained by a group of +volunteers for many years now (). It is +widely used, particularly but not exclusively for mathematical and +technical documents. + + A LaTeX user writes an input file containing text to be typeset along +with interspersed commands. The default encoding for the text is UTF-8 +(as of 2018). The commands specify, for example, how the text should be +formatted. + + LaTeX is implemented as a set of related so-called "macros" which use +Donald E. Knuth's TeX typesetting program or one of its derivatives, +collectively known as "engines". Thus, the user produces output, +typically PDF, by giving the input file to a TeX engine. (The following +sections describe all this in more detail.) + + The term LaTeX is also sometimes used to mean the language in which +the input document is marked up, that is, to mean the set of commands +available to a LaTeX user. + + The name LaTeX is short for "Lamport TeX". It is pronounced LAH-teck +or LAY-teck, or sometimes LAY-tecks. Inside a document, produce the +logo with '\LaTeX'. Where use of the logo is not sensible, such as in +plain text, write it as 'LaTeX'. + +* Menu: + +* Starting and ending:: The standard beginning and end of a document. +* Output files:: Files produced. +* TeX engines:: Programs that can compile TeX and LaTeX. +* LaTeX command syntax:: General syntax of LaTeX commands. +* Environment:: An area of the source with distinct behavior. +* CTAN:: Our repository. + + +File: latex2e.info, Node: Starting and ending, Next: Output files, Up: Overview + +2.1 Starting and ending +======================= + +LaTeX files have a simple global structure, with a standard beginning +and ending. This is a small example. + + \documentclass{article} + \begin{document} + Hello, \LaTeX\ world. + \end{document} + +Every LaTeX document has a '\begin{document}' line and an +'\end{document}' line. + +Here, the 'article' is the "document class". It is implemented in a +file 'article.cls'. You can use any document class on your system. A +few document classes are defined by LaTeX itself, and vast array of +others are widely available. *Note Document classes::. + + You can include other LaTeX commands between the '\documentclass' and +the '\begin{document}' commands. This area is called the "preamble". + + The '\begin{document}', '\end{document}' pair defines an +"environment"; the 'document' environment (and no others) is required in +all LaTeX documents (*note document::). LaTeX make available to you +many environments that are documented here (*note Environments::). Many +more are available to you from external packages, most importantly those +available at CTAN (*note CTAN::). + + The following sections discuss how to produce PDF or other output +from a LaTeX input file. + + +File: latex2e.info, Node: Output files, Next: TeX engines, Prev: Starting and ending, Up: Overview + +2.2 Output files +================ + +LaTeX produces a main output file and at least two auxiliary files. The +main output file's name ends in either '.dvi' or '.pdf'. + +'.dvi' + If LaTeX is invoked with the system command 'latex' then it + produces a DeVice Independent file, with extension '.dvi'. You can + view this file with a command such as 'xdvi', or convert it to a + PostScript '.ps' file with 'dvips' or to a Portable Document Format + '.pdf' file with 'dvipdfmx'. The contents of the file can be + dumped in human-readable form with 'dvitype'. A vast array of + other DVI utility programs are available + (). + +'.pdf' + If LaTeX is invoked via the system command 'pdflatex', among other + commands (*note TeX engines::), then the main output is a Portable + Document Format (PDF) file. Typically this is a self-contained + file, with all fonts and images included. + + LaTeX always produces at least two additional files. + +'.log' + This transcript file contains summary information such as a list of + loaded packages. It also includes diagnostic messages and perhaps + additional information for any errors. + +'.aux' + Auxiliary information is used by LaTeX for things such as cross + references. For example, the first time that LaTeX finds a forward + reference--a cross reference to something that has not yet appeared + in the source--it will appear in the output as a doubled question + mark '??'. When the referred-to spot does eventually appear in the + source then LaTeX writes its location information to this '.aux' + file. On the next invocation, LaTeX reads the location information + from this file and uses it to resolve the reference, replacing the + double question mark with the remembered location. + + LaTeX may produce yet more files, characterized by the filename +ending. These include a '.lof' file that is used to make a list of +figures, a '.lot' file used to make a list of tables, and a '.toc' file +used to make a table of contents (*note Table of contents etc.::). A +particular class may create others; the list is open-ended. + + +File: latex2e.info, Node: TeX engines, Next: LaTeX command syntax, Prev: Output files, Up: Overview + +2.3 TeX engines +=============== + +LaTeX is a large set of commands that is executed by a TeX program +(*note Overview::). Such a set of commands is called a "format", and is +embodied in a binary '.fmt' file, which can be read much more quickly +than the corresponding TeX source. + + This section gives a terse overview of the TeX programs that are +commonly available (see also *note Command line interface::). + +'latex' +'pdflatex' + In TeX Live (), if LaTeX is invoked via + either the system command 'latex' or 'pdflatex', then the pdfTeX + engine is run (). When invoked as + 'latex', the main output is a '.dvi' file; as 'pdflatex', the main + output is a '.pdf' file. + + pdfTeX incorporates the e-TeX extensions to Knuth's original + program (), including additional + programming features and bi-directional typesetting, and has plenty + of extensions of its own. e-TeX is available on its own as the + system command 'etex', but this is plain TeX (and produces '.dvi'). + + In other TeX distributions, 'latex' may invoke e-TeX rather than + pdfTeX. In any case, the e-TeX extensions can be assumed to be + available in LaTeX. + +'lualatex' + If LaTeX is invoked via the system command 'lualatex', the LuaTeX + engine is run (). This program allows + code written in the scripting language Lua () to + interact with TeX's typesetting. LuaTeX handles UTF-8 Unicode + input natively, can handle OpenType and TrueType fonts, and + produces a '.pdf' file by default. There is also 'dvilualatex' to + produce a '.dvi' file. + +'xelatex' + If LaTeX is invoked with the system command 'xelatex', the XeTeX + engine is run (). Like LuaTeX, XeTeX + natively supports UTF-8 Unicode and TrueType and OpenType fonts, + though the implementation is completely different, mainly using + external libraries instead of internal code. XeTeX produces a + '.pdf' file as output; it does not support DVI output. + + Internally, XeTeX creates an '.xdv' file, a variant of DVI, and + translates that to PDF using the ('x')'dvipdfmx' program, but this + process is automatic. The '.xdv' file is only useful for + debugging. + +'platex' +'uplatex' + These commands provide significant additional support for Japanese + and other languages; the 'u' variant supports Unicode. See + and . + + As of 2019, there is a companion '-dev' command and format for all of +the above: + +'dvilualatex-dev' +'latex-dev' +'lualatex-dev' +'pdflatex-dev' +'platex-dev' +'uplatex-dev' +'xelatex-dev' + These are candidates for an upcoming LaTeX release. The main + purpose is to find and address compatibility problems before an + official release. + + These '-dev' formats make it easy for anyone to help test documents + and code: you can run, say, 'pdflatex-dev' instead of 'pdflatex', + without changing anything else in your environment. Indeed, it is + easiest and most helpful to always run the '-dev' versions instead + of bothering to switch back and forth. During quiet times after a + release, the commands will be equivalent. + + These are not daily snapshots or untested development code. They + undergo the same extensive regression testing by the LaTeX team + before being released. + + For more information, see "The LaTeX release workflow and the LaTeX + 'dev' formats" by Frank Mittelbach, 'TUGboat' 40:2, + . + + +File: latex2e.info, Node: LaTeX command syntax, Next: Environment, Prev: TeX engines, Up: Overview + +2.4 LaTeX command syntax +======================== + +In the LaTeX input file, a command name starts with a backslash +character, '\'. The name itself then consists of either (a) a string of +letters or (b) a single non-letter. + + LaTeX commands names are case sensitive so that '\pagebreak' differs +from '\Pagebreak' (the latter is not a standard command). Most command +names are lowercase, but in any event you must enter all commands in the +same case as they are defined. + + A command may be followed by zero, one, or more arguments. These +arguments may be either required or optional. Required arguments are +contained in curly braces, '{...}'. Optional arguments are contained in +square brackets, '[...]'. Generally, but not universally, if the +command accepts an optional argument, it comes first, before any +required arguments. + + Inside of an optional argument, to use the character close square +bracket (']') hide it inside curly braces, as in '\item[closing bracket +{]}]'. Similarly, if an optional argument comes last, with no required +argument after it, then to make the first character of the following +text be an open square bracket, hide it inside curly braces. + + LaTeX has the convention that some commands have a '*' form that is +related to the form without a '*', such as '\chapter' and '\chapter*'. +The exact difference in behavior varies from command to command. + + This manual describes all accepted options and '*'-forms for the +commands it covers (barring unintentional omissions, a.k.a. bugs). + + As of the 2020-10-01 release of LaTeX, the 'expl3' and 'xparse' +packages are part of the LaTeX2e format. They provide a completely +different underlying programming language syntax. We won't try to cover +them in this document; see the related package documentation and other +LaTeX manuals. + + +File: latex2e.info, Node: Environment, Next: CTAN, Prev: LaTeX command syntax, Up: Overview + +2.5 Environment +=============== + +Synopsis: + + \begin{ENVIRONMENT-NAME} + ... + \end{ENVIRONMENT-NAME} + + An "environment" is an area of LaTeX source, inside of which there is +a distinct behavior. For instance, for poetry in LaTeX put the lines +between '\begin{verse}' and '\end{verse}'. + + \begin{verse} + There once was a man from Nantucket \\ + ... + \end{verse} + + *Note Environments::, for a list of environments. Particularly +notable is that every LaTeX document must have a 'document' environment, +a '\begin{document} ... \end{document}' pair. + + The ENVIRONMENT-NAME at the beginning must exactly match that at the +end. This includes the case where ENVIRONMENT-NAME ends in a +star ('*'); both the '\begin' and '\end' texts must include the star. + + Environments may have arguments, including optional arguments. This +example produces a table. The first argument is optional (and causes +the table to be aligned on its top row) while the second argument is +required (it specifies the formatting of columns). + + \begin{tabular}[t]{r|l} + ... rows of table ... + \end{tabular} + + +File: latex2e.info, Node: CTAN, Prev: Environment, Up: Overview + +2.6 CTAN: The Comprehensive TeX Archive Network +=============================================== + +The Comprehensive TeX Archive Network, CTAN, is the TeX and LaTeX +community's repository of free material. It is a set of Internet sites +around the world that offer material related to LaTeX for download. +Visit CTAN on the web at . + + This material is organized into packages, discrete bundles that +typically offer some coherent functionality and are maintained by one +person or a small number of people. For instance, many publishers have +a package that allows authors to format papers to that publisher's +specifications. + + In addition to the massive holdings, the 'ctan.org' web site offers +features such as search by name or by functionality. + + CTAN is not a single host, but instead is a set of hosts, one of +which is the so-called "master". The master host actively manages the +material, for instance, by accepting uploads of new or updated packages. +For many years, it has been hosted by the German TeX group, DANTE e.V. + + Other sites around the world help out by mirroring, that is, +automatically syncing their collections with the master site and then in +turn making their copies publicly available. This gives users close to +their location better access and relieves the load on the master site. +The list of mirrors is at . + + +File: latex2e.info, Node: Document classes, Next: Fonts, Prev: Overview, Up: Top + +3 Document classes +****************** + +The document's overall class is defined with this command, which is +normally the first command in a LaTeX source file. + + \documentclass[OPTIONS]{CLASS} + + The following document CLASS names are built into LaTeX. (Many other +document classes are available as separate packages; *note Overview::.) + +'article' + For a journal article, a presentation, and miscellaneous general + use. + +'book' + Full-length books, including chapters and possibly including front + matter, such as a preface, and back matter, such as an appendix + (*note Front/back matter::). + +'letter' + Mail, optionally including mailing labels (*note Letters::). + +'report' + For documents of length between an 'article' and a 'book', such as + technical reports or theses, which may contain several chapters. + +'slides' + For slide presentations--rarely used nowadays. The 'beamer' + package is perhaps the most prevalent + (). *Note beamer template::, for a + small template for a beamer document. + + Standard OPTIONS are described in the next section. + +* Menu: + +* Document class options:: Global options. +* Additional packages:: Bring in packages. +* Class and package construction:: Create new classes and packages. + + +File: latex2e.info, Node: Document class options, Next: Additional packages, Up: Document classes + +3.1 Document class options +========================== + +You can specify "global options" or "class options" to the +'\documentclass' command by enclosing them in square brackets. To +specify more than one OPTION, separate them with a comma. + + \documentclass[OPTION1,OPTION2,...]{CLASS} + + Here is the list of the standard class options. + + All of the standard classes except 'slides' accept the following +options for selecting the typeface size (default is '10pt'): + + 10pt 11pt 12pt + + All of the standard classes accept these options for selecting the +paper size (these show height by width): + +'a4paper' + 210 by 297mm (about 8.25 by 11.75 inches) + +'a5paper' + 148 by 210mm (about 5.8 by 8.3 inches) + +'b5paper' + 176 by 250mm (about 6.9 by 9.8 inches) + +'executivepaper' + 7.25 by 10.5 inches + +'legalpaper' + 8.5 by 14 inches + +'letterpaper' + 8.5 by 11 inches (the default) + + When using one of the engines pdfLaTeX, LuaLaTeX, or XeLaTeX (*note +TeX engines::), options other than 'letterpaper' set the print area but +you must also set the physical paper size. One way to do that is to put +'\pdfpagewidth=\paperwidth' and '\pdfpageheight=\paperheight' in your +document's preamble. + + The 'geometry' package provides flexible ways of setting the print +area and physical page size. + + Miscellaneous other options: + +'draft' +'final' + Mark ('draft') or do not mark ('final') overfull boxes with a black + box in the margin; default is 'final'. + +'fleqn' + Put displayed formulas flush left; default is centered. + +'landscape' + Selects landscape format; default is portrait. + +'leqno' + Put equation numbers on the left side of equations; default is the + right side. + +'openbib' + Use "open" bibliography format. + +'titlepage' +'notitlepage' + Specifies whether there is a separate page for the title + information and for the abstract also, if there is one. The + default for the 'report' class is 'titlepage', for the other + classes it is 'notitlepage'. + + The following options are not available with the 'slides' class. + +'onecolumn' +'twocolumn' + Typeset in one or two columns; default is 'onecolumn'. + +'oneside' +'twoside' + Selects one- or two-sided layout; default is 'oneside', except that + in the 'book' class the default is 'twoside'. + + For one-sided printing, the text is centered on the page. For + two-sided printing, the '\evensidemargin' ('\oddsidemargin') + parameter determines the distance on even (odd) numbered pages + between the left side of the page and the text's left margin, with + '\oddsidemargin' being 40% of the difference between '\paperwidth' + and '\textwidth', and '\evensidemargin' is the remainder. + +'openright' +'openany' + Determines if a chapter should start on a right-hand page; default + is 'openright' for 'book', and 'openany' for 'report'. + + The 'slides' class offers the option 'clock' for printing the time at +the bottom of each note. + + +File: latex2e.info, Node: Additional packages, Next: Class and package construction, Prev: Document class options, Up: Document classes + +3.2 Additional packages +======================= + +Load a package PKG, with the package options given in the +comma-separated list OPTIONS, as here. + + \usepackage[OPTIONS]{PKG}. + + To specify more than one package you can separate them with a comma, +as in '\usepackage{PKG1,PKG2,...}', or use multiple '\usepackage' +commands. + + Any options given in the '\documentclass' command that are unknown to +the selected document class are passed on to the packages loaded with +'\usepackage'. + + +File: latex2e.info, Node: Class and package construction, Prev: Additional packages, Up: Document classes + +3.3 Class and package construction +================================== + +You can create new document classes and new packages. For instance, if +your memos must satisfy some local requirements, such as a standard +header for each page, then you could create a new class 'smcmemo.cls' +and begin your documents with '\documentclass{smcmemo}'. + + What separates a package from a document class is that the commands +in a package are useful across classes while those in a document class +are specific to that class. Thus, a command to set page headers is for +a package while a command to make the page headers say 'Memo from the +SMC Math Department' is for a class. + + Inside of a class or package file you can use the at-sign '@' as a +character in command names without having to surround the code +containing that command with '\makeatletter' and '\makeatother'. *Note +\makeatletter & \makeatother::. This allow you to create commands that +users will not accidentally redefine. Another technique is to preface +class- or package-specific commands with some string to prevent your +class or package from interfering with others. For instance, the class +'smcmemo' might have commands '\smc@tolist', '\smc@fromlist', etc. + +* Menu: + +* Class and package structure:: Layout of the file. +* Class and package commands:: List of commands. + + +File: latex2e.info, Node: Class and package structure, Next: Class and package commands, Up: Class and package construction + +3.3.1 Class and package structure +--------------------------------- + +A class file or package file typically has four parts. + 1. In the "identification part", the file says that it is a LaTeX + package or class and describes itself, using the '\NeedsTeXFormat' + and '\ProvidesClass' or '\ProvidesPackage' commands. + + 2. The "preliminary declarations part" declares some commands and can + also load other files. Usually these commands will be those needed + for the code used in the next part. For example, an 'smcmemo' + class might be called with an option to read in a file with a list + of people for the to-head, as '\documentclass[mathto]{smcmemo}', + and therefore needs to define a command + '\newcommand{\setto}[1]{\def\@tolist{#1}}' used in that file. + + 3. In the "handle options part" the class or package declares and + processes its options. Class options allow a user to start their + document as '\documentclass[OPTION LIST]{CLASS NAME}', to modify + the behavior of the class. An example is when you declare + '\documentclass[11pt]{article}' to set the default document font + size. + + 4. Finally, in the "more declarations part" the class or package + usually does most of its work: declaring new variables, commands + and fonts, and loading other files. + + Here is a starting class file, which should be saved as 'stub.cls' +where LaTeX can find it, for example in the same directory as the '.tex' +file. + + \NeedsTeXFormat{LaTeX2e} + \ProvidesClass{stub}[2017/07/06 stub to start building classes from] + \DeclareOption*{\PassOptionsToClass{\CurrentOption}{article}} + \ProcessOptions\relax + \LoadClass{article} + +It identifies itself, handles the class options via the default of +passing them all to the 'article' class, and then loads the 'article' +class to provide the basis for this class's code. + + For more, see the official guide for class and package writers, the +Class Guide, at + (much of +the descriptions here derive from this document), or the tutorial +. + + +File: latex2e.info, Node: Class and package commands, Prev: Class and package structure, Up: Class and package construction + +3.3.2 Class and package commands +-------------------------------- + +These are the commands designed to help writers of classes or packages. + +'\AtBeginDvi{specials}' + Save in a box register things that are written to the '.dvi' file + at the beginning of the shipout of the first page of the document. + +'\AtEndOfClass{CODE}' +'\AtEndOfPackage{CODE}' + Hook to insert CODE to be executed when LaTeX finishes processing + the current class or package. You can use these hooks multiple + times; the 'code' will be executed in the order that you called it. + See also *note \AtBeginDocument::. + +'\CheckCommand{CMD}[NUM][DEFAULT]{DEFINITION}' +'\CheckCommand*{CMD}[NUM][DEFAULT]{DEFINITION}' + Like '\newcommand' (*note \newcommand & \renewcommand::) but does + not define CMD; instead it checks that the current definition of + CMD is exactly as given by DEFINITION and is or is not "long" as + expected. A long command is a command that accepts '\par' within + an argument. The CMD command is expected to be long with the + unstarred version of '\CheckCommand'. Raises an error when the + check fails. This allows you to check before you start redefining + 'cmd' yourself that no other package has already redefined this + command. + +'\ClassError{CLASS NAME}{ERROR TEXT}{HELP TEXT}' +'\PackageError{PACKAGE NAME}{ERROR TEXT}{HELP TEXT}' +'\ClassWarning{CLASS NAME}{WARNING TEXT}' +'\PackageWarning{PACKAGE NAME}{WARNING TEXT}' +'\ClassWarningNoLine{CLASS NAME}{WARNING TEXT}' +'\PackageWarningNoLine{PACKAGE NAME}{WARNING TEXT}' +'\ClassInfo{CLASS NAME}{INFO TEXT}' +'\PackageInfo{PACKAGE NAME}{INFO TEXT}' +'\ClassInfoNoLine{CLASS NAME}{INFO TEXT}' +'\PackageInfoNoLine{PACKAGE NAME}{INFO TEXT}' + Produce an error message, or warning or informational messages. + + For '\ClassError' and '\PackageError' the message is ERROR TEXT, + followed by TeX's '?' error prompt. If the user then asks for help + by typing 'h', they see the HELP TEXT. + + The four warning commands are similar except that they write + WARNING TEXT on the screen with no error prompt. The four info + commands write INFO TEXT only in the transcript file. The 'NoLine' + versions do not show the number of the line generating the message, + while the other versions do show that number. + + To format the messages, including the HELP TEXT: use '\protect' to + stop a command from expanding, get a line break with + '\MessageBreak', and get a space with '\space' when a space + character does not allow it, like after a command. Note that LaTeX + appends a period to the messages. + +'\CurrentOption' + Expands to the name of the currently-being-processed option. Can + only be used within the CODE argument of either '\DeclareOption' or + '\DeclareOption*'. + +'\DeclareOption{OPTION}{CODE}' +'\DeclareOption*{CODE}' + Make an option available to a user to invoke in their + '\documentclass' command. For example, the 'smcmemo' class could + have an option '\documentclass[logo]{smcmemo}' allowing users to + put the institutional logo on the first page. The class file must + contain '\DeclareOption{logo}{CODE}' (and later, + '\ProcessOptions'). + + If you request an option that has not been declared, by default + this will produce a warning like 'Unused global option(s): + [badoption].' Change this behavior with the starred version + '\DeclareOption*{CODE}'. For example, many classes extend an + existing class, using a command such as '\LoadClass{article}', and + for passing extra options to the underlying class use code such as + this. + + \DeclareOption*{% + \PassOptionsToClass{\CurrentOption}{article}% + } + + Another example is that the class 'smcmemo' may allow users to keep + lists of memo recipients in external files. Then the user could + invoke '\documentclass[math]{smcmemo}' and it will read the file + 'math.memo'. This code handles the file if it exists and otherwise + passes the option to the 'article' class. + + \DeclareOption*{\InputIfFileExists{\CurrentOption.memo}{}{% + \PassOptionsToClass{\CurrentOption}{article}}} + +'\DeclareRobustCommand{CMD}[NUM][DEFAULT]{DEFINITION}' +'\DeclareRobustCommand*{CMD}[NUM][DEFAULT]{DEFINITION}' + Like '\newcommand' and '\newcommand*' (*note \newcommand & + \renewcommand::) but these declare a robust command, even if some + code within the DEFINITION is fragile. (For a discussion of robust + and fragile commands *note \protect::.) Use this command to define + new robust commands or to redefine existing commands and make them + robust. Unlike '\newcommand' these do not give an error if macro + CMD already exists; instead, a log message is put into the + transcript file if a command is redefined. + + Commands defined this way are a bit less efficient than those + defined using '\newcommand' so unless the command's data is fragile + and the command is used within a moving argument, use + '\newcommand'. + + The 'etoolbox' package offers the commands '\newrobustcmd', + '\newrobustcmd*', as well as the commands '\renewrobustcmd', + '\renewrobustcmd*', and the commands '\providerobustcmd', and + '\providerobustcmd*'. These are similar to '\newcommand', + '\newcommand*', '\renewcommand', '\renewcommand*', + '\providecommand', and '\providecommand*', but define a robust CMD + with two advantages as compared to '\DeclareRobustCommand': + 1. They use the low-level e-TeX protection mechanism rather than + the higher level LaTeX '\protect' mechanism, so they do not + incur the slight loss of performance mentioned above, and + 2. They make the same distinction between '\new...', '\renew...', + and '\provide...', as the standard commands, so they do not + just make a log message when you redefine CMD that already + exists, in that case you need to use either '\renew...' or + '\provide...' or you get an error. + +'\IfFileExists{FILENAME}{TRUE CODE}{FALSE CODE}' +'\InputIfFileExists{FILENAME}{TRUE CODE}{FALSE CODE}' + Execute TRUE CODE if LaTeX finds the file 'FILE NAME' or FALSE CODE + otherwise. In the first case it executing TRUE CODE and then + inputs the file. Thus the command + + \IfFileExists{img.pdf}{% + \includegraphics{img.pdf}}{\typeout{!! img.pdf not found} + + will include the graphic 'img.pdf' if it is found and otherwise + give a warning. + + This command looks for the file in all search paths that LaTeX + uses, not only in the current directory. To look only in the + current directory do something like '\IfFileExists{./filename}{TRUE + CODE}{FALSE CODE}'. If you ask for a filename without a '.tex' + extension then LaTeX will first look for the file by appending the + '.tex'; for more on how LaTeX handles file extensions see *note + \input::. + +'\LoadClass[OPTIONS LIST]{CLASS NAME}[RELEASE DATE]' +'\LoadClassWithOptions{CLASS NAME}[RELEASE DATE]' + Load a class, as with '\documentclass[OPTIONS LIST]{CLASS + NAME}[RELEASE INFO]'. An example is + '\LoadClass[twoside]{article}'. + + The OPTIONS LIST, if present, is a comma-separated list. The + RELEASE DATE is optional. If present it must have the form + YYYY/MM/DD. + + If you request a RELEASE DATE and the date of the package installed + on your system is earlier, then you get a warning on the screen and + in the log like this. + + You have requested, on input line 4, version `2038/01/19' of + document class article, but only version `2014/09/29 v1.4h + Standard LaTeX document class' is available. + + The command version '\LoadClassWithOptions' uses the list of + options for the current class. This means it ignores any options + passed to it via '\PassOptionsToClass'. This is a convenience + command that lets you build classes on existing ones, such as the + standard 'article' class, without having to track which options + were passed. + +'\ExecuteOptions{OPTIONS-LIST}' + For each option OPTION in the OPTIONS-LIST, in order, this command + executes the command '\ds@OPTION'. If this command is not defined + then that option is silently ignored. + + It can be used to provide a default option list before + '\ProcessOptions'. For example, if in a class file you want the + default to be 11pt fonts then you could specify + '\ExecuteOptions{11pt}\ProcessOptions\relax'. + +'\NeedsTeXFormat{FORMAT}[FORMAT DATE]' + Specifies the format that this class must be run under. Often + issued as the first line of a class file, and most often used as: + '\NeedsTeXFormat{LaTeX2e}'. When a document using that class is + processed, the format name given here must match the format that is + actually being run (including that the FORMAT string is case + sensitive). If it does not match then execution stops with an + error like 'This file needs format `LaTeX2e' but this is `xxx'.' + + To specify a version of the format that you know to have certain + features, include the optional FORMAT DATE on which those features + were implemented. If present it must be in the form 'YYYY/MM/DD'. + If the format version installed on your system is earlier than + FORMAT DATE then you get a warning like this. + + You have requested release `2038/01/20' of LaTeX, but only + release `2016/02/01' is available. + +'\OptionNotUsed' + Adds the current option to the list of unused options. Can only be + used within the CODE argument of either '\DeclareOption' or + '\DeclareOption*'. + +'\PassOptionsToClass{OPTION LIST}{CLASS NAME}' +'\PassOptionsToPackage{OPTION LIST}{PACKAGE NAME}' + Adds the options in the comma-separated list OPTION LIST to the + options used by any future '\RequirePackage' or '\usepackage' + command for package PACKAGE NAME or the class CLASS NAME. + + The reason for these commands is: you may load a package any number + of times with no options but if you want options then you may only + supply them when you first load the package. Loading a package + with options more than once will get you an error like 'Option + clash for package foo.' (LaTeX throws an error even if there is no + conflict between the options.) + + If your own code is bringing in a package twice then you can + collapse that to once, for example replacing the two + '\RequirePackage[landscape]{geometry}' and + '\RequirePackage[margins=1in]{geometry}' with the single command + '\RequirePackage[landscape,margins=1in]{geometry}'. + + However, imagine that you are loading 'firstpkg' and inside that + package it loads 'secondpkg', and you need the second package to be + loaded with option 'draft'. Then before doing the first package + you must queue up the options for the second package, like this. + + \PassOptionsToPackage{draft}{secondpkg} + \RequirePackage{firstpkg} + + (If 'firstpkg.sty' loads an option in conflict with what you want + then you may have to alter its source.) + + These commands are useful for general users as well as class and + package writers. For instance, suppose a user wants to load the + 'graphicx' package with the option 'draft' and also wants to use a + class 'foo' that loads the 'graphicx' package, but without that + option. The user could start their LaTeX file with + '\PassOptionsToPackage{draft}{graphicx}\documentclass{foo}'. + +'\ProcessOptions' +'\ProcessOptions*\@OPTIONS' + Execute the code for each option that the user has invoked. + Include it in the class file as '\ProcessOptions\relax' (because of + the existence of the starred command). + + Options come in two types. "Local options" have been specified for + this particular package in the OPTIONS argument of + '\PassOptionsToPackage{OPTIONS}', '\usepackage[OPTIONS]', or + '\RequirePackage[OPTIONS]'. "Global options" are those given by + the class user in '\documentclass[OPTIONS]' (If an option is + specified both locally and globally then it is local.) + + When '\ProcessOptions' is called for a package 'pkg.sty', the + following happens: + 1. For each option OPTION so far declared with '\DeclareOption', + it looks to see if that option is either a global or a local + option for 'pkg'. If so then it executes the declared code. + This is done in the order in which these options were given in + 'pkg.sty'. + 2. For each remaining local option, it executes the command + '\ds@'OPTION if it has been defined somewhere (other than by a + '\DeclareOption'); otherwise, it executes the default option + code given in '\DeclareOption*'. If no default option code + has been declared then it gives an error message. This is + done in the order in which these options were specified. + + When '\ProcessOptions' is called for a class it works in the same + way except that all options are local, and the default CODE for + '\DeclareOption*' is '\OptionNotUsed' rather than an error. + + The starred version '\ProcessOptions*' executes the options in the + order specified in the calling commands, rather than in the order + of declaration in the class or package. For a package this means + that the global options are processed first. + +'\ProvidesClass{CLASS NAME}[RELEASE DATE BRIEF ADDITIONAL INFORMATION]' +'\ProvidesClass{CLASS NAME}[RELEASE DATE]' +'\ProvidesPackage{PACKAGE NAME}[RELEASE DATE BRIEF ADDITIONAL INFORMATION]' +'\ProvidesPackage{PACKAGE NAME}[RELEASE DATE]' + Identifies the class or package, printing a message to the screen + and the log file. + + When you load a class or package, for example with + '\documentclass{smcmemo}' or '\usepackage{test}', LaTeX inputs a + file. If the name of the file does not match the class or package + name declared in it then you get a warning. Thus, if you invoke + '\documentclass{smcmemo}', and the file 'smcmemo.cls' has the + statement '\ProvidesClass{xxx}' then you get a warning like 'You + have requested document class `smcmemo', but the document class + provides 'xxx'.' This warning does not prevent LaTeX from + processing the rest of the class file normally. + + If you include the optional argument then you must include a date, + before any spaces, of the form 'YYYY/MM/DD'. The rest of the + optional argument is free-form, although it traditionally + identifies the class, and is written to the screen during + compilation and to the log file. Thus, if your file 'smcmemo.cls' + contains the line '\ProvidesClass{smcmemo}[2008/06/01 v1.0 SMC memo + class]' and your document's first line is '\documentclass{smcmemo}' + then you will see 'Document Class: smcmemo 2008/06/01 v1.0 SMC memo + class'. + + The date in the optional argument allows class and package users to + ask to be warned if the version of the class or package is earlier + than RELEASE DATE. For instance, a user could enter + '\documentclass{smcmemo}[2018/10/12]' or + '\usepackage{foo}[[2017/07/07]]' to require a class or package with + certain features by specifying that it must be released no earlier + than the given date. (Although, in practice package users only + rarely include a date, and class users almost never do.) + +'\ProvidesFile{FILENAME}[ADDITIONAL INFORMATION]' + Declare a file other than the main class and package files, such as + configuration files or font definition files. Put this command in + that file and you get in the log a string like 'File: test.config + 2017/10/12 config file for test.cls' for FILENAME equal to + 'test.config' and ADDITIONAL INFORMATION equal to '2017/10/12 + config file for test.cls'. + +'\RequirePackage[OPTION LIST]{PACKAGE NAME}[RELEASE DATE]' +'\RequirePackageWithOptions{PACKAGE NAME}[RELEASE DATE]' + Load a package, like the command '\usepackage' (*note Additional + packages::). The LaTeX development team strongly recommends use of + these commands over Plain TeX's '\input'; see the Class Guide. An + example is '\RequirePackage[landscape,margin=1in]{geometry}'. + + The OPTION LIST, if present, is a comma-separated list. The + RELEASE DATE, if present, must have the form YYYY/MM/DD. If the + release date of the package as installed on your system is earlier + than RELEASE DATE then you get a warning like 'You have requested, + on input line 9, version `2017/07/03' of package jhtest, but only + version `2000/01/01' is available'. + + The '\RequirePackageWithOptions' version uses the list of options + for the current class. This means it ignores any options passed to + it via '\PassOptionsToClass'. This is a convenience command to + allow easily building classes on existing ones without having to + track which options were passed. + + The difference between '\usepackage' and '\RequirePackage' is + small. The '\usepackage' command is intended for the document file + while '\RequirePackage' is intended for package and class files. + Thus, using '\usepackage' before the '\documentclass' command + causes LaTeX to give error like '\usepackage before + \documentclass', but you can use '\RequirePackage' there. + + +File: latex2e.info, Node: Fonts, Next: Layout, Prev: Document classes, Up: Top + +4 Fonts +******* + +LaTeX comes with powerful font capacities. For one thing, its New Font +Selection Scheme allows you to work easily with the font families in +your document (for instance, see *note Font styles::). And, LaTeX +documents can use most fonts that are available today, including +versions of Times Roman, Helvetica, Courier, etc. (Note, though, that +many fonts do not have support for mathematics.) + + The first typeface in the TeX world was the Computer Modern family, +developed by Donald Knuth. It is the default for LaTeX documents and is +still the most widely used. But changing to another font often only +involves a few commands. For instance, putting the following in your +preamble gives you a Palatino-like font, which is handsome and more +readable online than many other fonts, while still allowing you to +typeset mathematics. (This example is from Michael Sharpe, +.) + + \usepackage[osf]{newpxtext} % osf for text, not math + \usepackage{cabin} % sans serif + \usepackage[varqu,varl]{inconsolata} % sans serif typewriter + \usepackage[bigdelims,vvarbb]{newpxmath} % bb from STIX + \usepackage[cal=boondoxo]{mathalfa} % mathcal + +In addition, the 'xelatex' or 'lualatex' engines allow you to use any +fonts on your system that are in OpenType or TrueType format (*note TeX +engines::). + + The LaTeX Font Catalogue () shows font +sample graphics and copy-and-pasteable source to use many fonts, +including many with support for mathematics. It aims to cover all Latin +alphabet free fonts available for easy use with LaTeX. + + More information is also available from the TeX Users Group, at +. + +* Menu: + +* fontenc package:: Encoding of the font. +* Font styles:: Select roman, italics, etc. +* Font sizes:: Select point size. +* Low-level font commands:: Select encoding, family, series, shape. + + +File: latex2e.info, Node: fontenc package, Next: Font styles, Up: Fonts + +4.1 'fontenc' package +===================== + +Synopsis: + + \usepackage[FONT_ENCODING]{fontenc} + + or + + \usepackage[FONT_ENCODING1, FONT_ENCODING2, ...]{fontenc} + + Specify the font encodings. A font encoding is a mapping of the +character codes to the font glyphs that are used to typeset your output. + + This package only applies if you use the 'pdflatex' engine (*note TeX +engines::). If you use the 'xelatex' or 'lualatex' engine then instead +use the 'fontspec' package. + + TeX's original font family, Computer Modern, has a limited character +set. For instance, to make common accented characters you must use +'\accent' (*note \accent::) but this disables hyphenation. TeX users +have agreed on a number of standards to access the larger sets of +characters provided by modern fonts. If you are using 'pdflatex' then +put this in the preamble + + \usepackage[T1]{fontenc} + +gives you support for the most widespread European languages, including +French, German, Italian, Polish, and others. In particular, if you have +words with accented letters then LaTeX will hyphenate them and your +output can be copied and pasted. (The optional second line allows you +to directly enter accented characters into your source file.) + + If you are using an encoding such as 'T1' and the characters appear +blurry or do not magnify well then your fonts may be bitmapped, +sometimes called raster or Type 3. You want vector fonts. Use a +package such as 'lmodern' or 'cm-super' to get a font that extends +LaTeX's default using vector fonts. + + For each FONT_ENCODING given as an option but not already declared, +this package loads the encoding definition files, named +'FONT_ENCODINGenc.def'. It also sets '\encodingdefault' to be the last +encoding in the option list. + + These are the common values for FONT_ENCODING. + +'OT1' + The original encoding for TeX. Limited to mostly English + characters. + +'OMS, OML' + Math symbols and math letters encoding. + +'T1' + TeX text extended. Sometimes called the Cork encoding for the + Users Group meeting where it was developed. Gives access to most + European accented characters. The most common option for this + package. + +'TS1' + Text Companion encoding. + +LaTeX's default is to load 'OML', 'T1', 'OT1', and then 'OMS', and set +the default to 'OT1'. + + Even if you do not use accented letters, you may need to specify a +font encoding if your font requires it. + + If you use 'T1' encoded fonts other than the default Computer Modern +family then you may need to load the package that selects your fonts +before loading 'fontenc', to prevent the system from loading any +'T1' encoded fonts from the default. + + The LaTeX team reserve encoding names starting with: 'T' for the +standard text encodings with 256 characters, 'TS' for symbols that +extend the corresponding T encodings, 'X' for test encodings, 'M' for +standard math encodings with 256 characters, 'A' for special +applications, 'OT' for standard text encodings with 128 characters, and +'OM' for standard math encodings with 128 characters ('O' stands for +'obsolete'). + + This package provides a number of commands, detailed below. Many of +them are encoding-specific, so if you have defined a command that works +for one encoding but the current encoding is different then the command +is not in effect. + +* Menu: + +* \DeclareFontEncoding:: Define an encoding. +* \DeclareTextAccent:: Define an accent in the encoding. +* \DeclareTextAccentDefault:: Fallback for using an accent in the encoding. +* \DeclareTextCommand & \ProvideTextCommand:: New encoding-specific command. +* \DeclareTextCommandDefault & \ProvideTextCommandDefault:: Fallback for encoding-specific commands. +* \DeclareTextComposite:: Directly access an accented glyph, in the encoding. +* \DeclareTextCompositeCommand:: Run code in slot, in the encoding. +* \DeclareTextSymbol:: Define a symbol in the encoding. +* \DeclareTextSymbolDefault:: Fallback for a symbol in the encoding. +* \LastDeclaredEncoding:: Save most recently declared encoding. +* \UseTextSymbol & \UseTextAccent:: Temporarily switch to another encoding. + + +File: latex2e.info, Node: \DeclareFontEncoding, Next: \DeclareTextAccent, Up: fontenc package + +4.1.1 '\DeclareFontEncoding' +---------------------------- + +Synopsis: + + \DeclareFontEncoding{ENCODING}{TEXT-SETTINGS}{MATH-SETTINGS} + + Declare the font encoding ENCODING. It also saves the value of +ENCODING in '\LastDeclaredEncoding' (*note \LastDeclaredEncoding::). + + The file 't1enc.def' contains this line (followed by many others). + + \DeclareFontEncoding{T1}{}{} + + The TEXT-SETTINGS are the commands that LaTeX will run every time it +switches from one encoding to another with the '\selectfont' and +'\fontencoding' commands. The MATH-SETTINGS are the commands that LaTeX +will use whenever the font is accessed as a math alphabet. + + LaTeX ignores any space characters inside TEXT-SETTINGS and +MATH-SETTINGS, to prevent unintended spaces in the output. + + If you invent an encoding you should pick a two or three letter name +starting with 'L' for 'local', or 'E' for 'experimental'. + + Note that output encoding files may be read several times by LaTeX so +using, e.g., '\newcommand' may cause an error. In addition, such files +should contain '\ProvidesFile' line (*note Class and package +commands::). + + Note also that you should use the '\...Default' commands only in a +package, not in the encoding definition files, since those files should +only contain declarations specific to that encoding. + + +File: latex2e.info, Node: \DeclareTextAccent, Next: \DeclareTextAccentDefault, Prev: \DeclareFontEncoding, Up: fontenc package + +4.1.2 '\DeclareTextAccent' +-------------------------- + +Synopsis: + + \DeclareTextAccent{CMD}{ENCODING}{SLOT} + + Define an accent, to be put on top of other glyphs, in the encoding +ENCODING at the location SLOT. + + A "slot" is the number identifying a glyph within a font. + + This line from 't1enc.def' declares that to make a circumflex accent +as in '\^A', the system will put the accent in slot 2 over the 'A' +character, which is represented in ASCII as 65. (This holds unless +there is a relevant 'DeclareTextComposite' or +'\DeclareTextCompositeCommand' declaration; *note +\DeclareTextComposite::.) + + \DeclareTextAccent{\^}{T1}{2} + + If CMD has already been defined then '\DeclareTextAccent' does not +give an error but it does log the redefinition in the transcript file. + + +File: latex2e.info, Node: \DeclareTextAccentDefault, Next: \DeclareTextCommand & \ProvideTextCommand, Prev: \DeclareTextAccent, Up: fontenc package + +4.1.3 '\DeclareTextAccentDefault' +--------------------------------- + +Synopsis: + + \DeclareTextAccentDefault{\CMD}{ENCODING} + + If there is an encoding-specific accent command \CMD but there is no +associated '\DeclareTextAccent' for that encoding then this command will +pick up the slack, by saying to use it as described for ENCODING. + + For example, to make the encoding 'OT1' be the default encoding for +the accent '\"', declare this. + + \DeclareTextAccentDefault{\"}{OT1} + +If you issue a '\"' when the current encoding does not have a definition +for that accent then LaTeX will use the definition from 'OT1' + + That is, this command is equivalent to this call (*note +\UseTextSymbol & \UseTextAccent::). + + \DeclareTextCommandDefault[1]{\CMD} + {\UseTextAccent{ENCODING}{\CMD}{#1}} + + Note that '\DeclareTextAccentDefault' works for any one-argument +'fontenc' command, not just the accent command. + + +File: latex2e.info, Node: \DeclareTextCommand & \ProvideTextCommand, Next: \DeclareTextCommandDefault & \ProvideTextCommandDefault, Prev: \DeclareTextAccentDefault, Up: fontenc package + +4.1.4 '\DeclareTextCommand' & '\ProvideTextCommand' +--------------------------------------------------- + +Synopsis, one of: + + \DeclareTextCommand{\CMD}{ENCODING}{DEFN} + \DeclareTextCommand{\CMD}{ENCODING}[NARGS]{DEFN} + \DeclareTextCommand{\CMD}{ENCODING}[NARGS][OPTARGDEFAULT]{DEFN} + + or one of: + + \ProvideTextCommand{\CMD}{ENCODING}{DEFN} + \ProvideTextCommand{\CMD}{ENCODING}[NARGS]{DEFN} + \ProvideTextCommand{\CMD}{ENCODING}[NARGS][OPTARGDEFAULT]{DEFN} + + Define the command '\CMD', which will be specific to one encoding. +The command name CMD must be preceded by a backslash, '\'. These +commands can only appear in the preamble. Redefining \CMD does not +cause an error. The defined command will be robust even if the code in +DEFN is fragile (*note \protect::). + + For example, the file 't1enc.def' contains this line. + + \DeclareTextCommand{\textperthousand}{T1}{\%\char 24 } + + With that, you can express parts per thousand. + + \usepackage[T1]{fontenc} % in preamble + ... + Legal limit is \( 0.8 \)\textperthousand. + +If you change the font encoding to 'OT1' then you get an error like +'LaTeX Error: Command \textperthousand unavailable in encoding OT1'. + + The '\ProvideTextCommand' variant does the same, except that it does +nothing if '\CMD' is already defined. The '\DeclareTextSymbol' command +is faster than this one for simple slot-to-glyph association (*note +\DeclareTextSymbol::) + + The optional NARGS and OPTARGDEFAULT arguments play the same role +here as in '\newcommand' (*note \newcommand & \renewcommand::). +Briefly, NARGS is an integer from 0 to 9 specifying the number of +arguments that the defined command '\CMD' takes. This number includes +any optional argument. Omitting this argument is the same as specifying +0, meaning that '\CMD' will have no arguments. And, if OPTARGDEFAULT is +present then the first argument of '\CMD' is optional, with default +value OPTARGDEFAULT (which may be the empty string). If OPTARGDEFAULT +is not present then '\CMD' does not take an optional argument. + + +File: latex2e.info, Node: \DeclareTextCommandDefault & \ProvideTextCommandDefault, Next: \DeclareTextComposite, Prev: \DeclareTextCommand & \ProvideTextCommand, Up: fontenc package + +4.1.5 '\DeclareTextCommandDefault' & '\ProvideTextCommandDefault ' +------------------------------------------------------------------ + +Synopsis: + + \DeclareTextCommandDefault{\CMD}{DEFN} + + or: + + \ProvideTextCommandDefault{\CMD}{DEFN} + + Give a default definition for '\CMD', for when that command is not +defined in the encoding currently in force. This default should only +use encodings known to be available. + + This makes '\copyright' available. + + \DeclareTextCommandDefault{\copyright}{\textcircled{c}} + +It uses only an encoding (OMS) that is always available. + + The '\DeclareTextCommandDefault' should not occur in the encoding +definition files since those files should declare only commands for use +when you select that encoding. It should instead be in a package. + + As with the related non-default commands, the +'\ProvideTextCommandDefault' has exactly the same behavior as +'\DeclareTextCommandDefault' except that it does nothing if '\CMD' is +already defined (*note \DeclareTextCommand & \ProvideTextCommand::). +So, packages can use it to provide fallbacks that other packages can +improve upon. + + +File: latex2e.info, Node: \DeclareTextComposite, Next: \DeclareTextCompositeCommand, Prev: \DeclareTextCommandDefault & \ProvideTextCommandDefault, Up: fontenc package + +4.1.6 '\DeclareTextComposite' +----------------------------- + +Synopsis: + + \DeclareTextComposite{\CMD}{ENCODING}{SIMPLE_OBJECT}{SLOT} + + Access an accented glyph directly, that is, without having to put an +accent over a separate character. + + This line from 't1enc.def' means that '\^o' will cause LaTeX to +typeset lowercase 'o' by taking the character directly from slot 224 in +the font. + + \DeclareTextComposite{\^}{T1}{o}{244} + + *Note fontenc package::, for a list of common encodings. The +SIMPLE_OBJECT should be a single character or a single command. The +SLOT argument is usually a positive integer represented in decimal +(although octal or hexadecimal are possible). Normally \CMD has already +been declared for this encoding, either with '\DeclareTextAccent' or +with a one-argument '\DeclareTextCommand'. In 't1enc.def', the above +line follows the '\DeclareTextAccent{\^}{T1}{2}' command. + + +File: latex2e.info, Node: \DeclareTextCompositeCommand, Next: \DeclareTextSymbol, Prev: \DeclareTextComposite, Up: fontenc package + +4.1.7 '\DeclareTextCompositeCommand' +------------------------------------ + +Synopsis: + + \DeclareTextCompositeCommand{\CMD}{ENCODING}{ARG}{CODE} + + A more general version of '\DeclareTextComposite' that runs arbitrary +code with '\CMD'. + + This allows accents on 'i' to act like accents on dotless i, '\i'. + + \DeclareTextCompositeCommand{\'}{OT1}{i}{\'\i} + + *Note fontenc package::, for a list of common encodings. Normally +'\CMD' will have already been declared with '\DeclareTextAccent' or as a +one argument '\DeclareTextCommand'. + + +File: latex2e.info, Node: \DeclareTextSymbol, Next: \DeclareTextSymbolDefault, Prev: \DeclareTextCompositeCommand, Up: fontenc package + +4.1.8 '\DeclareTextSymbol' +-------------------------- + +Synopsis: + + \DeclareTextSymbol{\CMD}{ENCODING}{SLOT} + + Define a symbol in the encoding ENCODING at the location SLOT. +Symbols defined in this way are for use in text, not mathematics. + + For example, this line from 't1enc.def' declares the number of the +glyph to use for «, the left guillemet. + + \DeclareTextSymbol{\guillemotleft}{T1}{19} + +The command '\DeclareTextCommand{\guillemotleft}{T1}{\char 19}' has the +same effect but is slower (*note \DeclareTextCommand & +\ProvideTextCommand::). + + *Note fontenc package::, for a list of common encodings. The SLOT +can be specified in decimal, or octal (as in ''023'), or hexadecimal (as +in '"13'), although decimal has the advantage that single quote or +double quote could be redefined by another package. + + If '\CMD' has already been defined then '\DeclareTextSymbol' does not +give an error but it does log the redefinition in the transcript file. + + +File: latex2e.info, Node: \DeclareTextSymbolDefault, Next: \LastDeclaredEncoding, Prev: \DeclareTextSymbol, Up: fontenc package + +4.1.9 '\DeclareTextSymbolDefault' +--------------------------------- + +Synopsis: + + \DeclareTextSymbolDefault{\CMD}{ENCODING} + + If there is an encoding-specific symbol command '\CMD' but there is +no associated '\DeclareTextSymbol' for that encoding, then this command +will pick up the slack, by saying to get the symbol as described for +ENCODING. + + For example, to declare that if the current encoding has no meaning +for '\textdollar' then use the one from 'OT1', declare this. + + \DeclareTextSymbolDefault{\textdollar}{OT1} + + That is, this command is equivalent to this call (*note +\UseTextSymbol & \UseTextAccent::). + + \DeclareTextCommandDefault{\CMD} + {\UseTextSymbol{ENCODING}{\CMD}} + + Note that '\DeclareTextSymbolDefault' can be used to define a default +for any zero-argument 'fontenc' command. + + +File: latex2e.info, Node: \LastDeclaredEncoding, Next: \UseTextSymbol & \UseTextAccent, Prev: \DeclareTextSymbolDefault, Up: fontenc package + +4.1.10 '\LastDeclaredEncoding' +------------------------------ + +Synopsis: + + \LastDeclaredEncoding + + Get the name of the most recently declared encoding. The +'\DeclareFontEncoding' command stores the name so that it can be +retrieved with this command (*note \DeclareFontEncoding::). + + This relies on '\LastDeclaredEncoding' rather than give the name of +the encoding explicitly. + + \DeclareFontEncoding{JH1}{}{} + \DeclareTextAccent{\'}{\LastDeclaredEncoding}{0} + + +File: latex2e.info, Node: \UseTextSymbol & \UseTextAccent, Prev: \LastDeclaredEncoding, Up: fontenc package + +4.1.11 '\UseTextSymbol' & '\UseTextAccent' +------------------------------------------ + +Synopsis: + + \UseTextSymbol{ENCODING}{\CMD} + + or: + + \UseTextAccent{ENCODING}{\CMD}{TEXT} + + Use a symbol or accent not from the current encoding. + + In general, to use a 'fontenc' command in an encoding where it is not +defined, and if the command has no arguments, then you can use it like +this: + + \UseTextSymbol{OT1}{\ss} + +which is equivalent to this (note the outer braces form a group, so +LaTeX reverts back to the prior encoding after the '\ss'): + + {\fontencoding{OT1}\selectfont\ss} + + Similarly, to use a 'fontenc' command in an encoding where it is not +defined, and if the command has one argument, you can use it like this: + + \UseTextAccent{OT1}{\'}{a} + +which is equivalent to this (again note the outer braces forming a +group): + + {fontencoding{OT1}\selectfont\'{\fontencoding{ENC_IN_USE}\selectfont a}} + +Here, ENC_IN_USE is the encoding in force before this sequence of +commands, so that 'a' is typeset using the current encoding and only the +accent is taken from 'OT1'. + + +File: latex2e.info, Node: Font styles, Next: Font sizes, Prev: fontenc package, Up: Fonts + +4.2 Font styles +=============== + +The following type style commands are supported by LaTeX. + + In the table below the listed commands, the '\text...' commands, are +used with an argument as in '\textit{TEXT}'. This is the preferred +form. But shown after it in parenthesis is the corresponding +"declaration form", which is often useful. This form takes no +arguments, as in '{\itshape TEXT}'. The scope of the declaration form +lasts until the next type style command or the end of the current group. +In addition, each has an environment form such as +'\begin{itshape}...\end{itshape}', which we'll describe further at the +end of the section. + + These commands, in any of the three forms, are cumulative; for +instance you can get bold sans serif by saying either of +'\sffamily\bfseries' or '\bfseries\sffamily'. + + One advantage of these commands is that they automatically insert +italic corrections if needed (*note \/::). Specifically, they insert +the italic correction unless the following character is in the list +'\nocorrlist', which by default consists of period and comma. To +suppress the automatic insertion of italic correction, use '\nocorr' at +the start or end of the command argument, such as '\textit{\nocorr +text}' or '\textsc{text \nocorr}'. + +'\textrm (\rmfamily)' + Roman. + +'\textit (\itshape)' + Italics. + +'\textmd (\mdseries)' + Medium weight (default). + +'\textbf (\bfseries)' + Boldface. + +'\textup (\upshape)' + Upright (default). + +'\textsl (\slshape)' + Slanted. + +'\textsf (\sffamily)' + Sans serif. + +'\textsc (\scshape)' + Small caps. + +'\texttt (\ttfamily)' + Typewriter. + +'\textnormal (\normalfont)' + Main document font. + + Although it also changes fonts, the '\emph{TEXT}' command is +semantic, for text to be emphasized, and should not be used as a +substitute for '\textit'. For example, '\emph{START TEXT \emph{MIDDLE +TEXT} END TEXT}' will result in the START TEXT and END TEXT in italics, +but MIDDLE TEXT will be in roman. + + LaTeX also provides the following commands, which unconditionally +switch to the given style, that is, are _not_ cumulative. They are used +as declarations: '{\CMD...}' instead of '\CMD{...}'. + + (The unconditional commands below are an older version of font +switching. The earlier commands are an improvement in most +circumstances. But sometimes an unconditional font switch is what is +needed.) + +'\bf' + Switch to bold face. + +'\cal' + Switch to calligraphic letters for math. + +'\it' + Italics. + +'\rm' + Roman. + +'\sc' + Small caps. + +'\sf' + Sans serif. + +'\sl' + Slanted (oblique). + +'\tt' + Typewriter (monospace, fixed-width). + + The '\em' command is the unconditional version of '\emph'. + + The following commands are for use in math mode. They are not +cumulative, so '\mathbf{\mathit{SYMBOL}}' does not create a boldface and +italic SYMBOL; instead, it will just be in italics. This is because +typically math symbols need consistent typographic treatment, regardless +of the surrounding environment. + +'\mathrm' + Roman, for use in math mode. + +'\mathbf' + Boldface, for use in math mode. + +'\mathsf' + Sans serif, for use in math mode. + +'\mathtt' + Typewriter, for use in math mode. + +'\mathit' +'(\mit)' + Italics, for use in math mode. + +'\mathnormal' + For use in math mode, e.g., inside another type style declaration. + +'\mathcal' + Calligraphic letters, for use in math mode. + + In addition, the command '\mathversion{bold}' can be used for +switching to bold letters and symbols in formulas. +'\mathversion{normal}' restores the default. + + Finally, the command '\oldstylenums{NUMERALS}' will typeset so-called +"old-style" numerals, which have differing heights and depths (and +sometimes widths) from the standard "lining" numerals, which all have +the same height as uppercase letters. LaTeX's default fonts support +this, and will respect '\textbf' (but not other styles; there are no +italic old-style numerals in Computer Modern). Many other fonts have +old-style numerals also; sometimes package options are provided to make +them the default. FAQ entry: . + + +File: latex2e.info, Node: Font sizes, Next: Low-level font commands, Prev: Font styles, Up: Fonts + +4.3 Font sizes +============== + +The following standard type size commands are supported by LaTeX. The +table shows the command name and the corresponding actual font size used +(in points) with the '10pt', '11pt', and '12pt' document size options, +respectively (*note Document class options::). + +Command '10pt' '11pt' '12pt' +-------------------------------------------------------- +'\tiny' 5 6 6 +'\scriptsize' 7 8 8 +'\footnotesize' 8 9 10 +'\small' 9 10 10.95 +'\normalsize' (default) 10 10.95 12 +'\large' 12 12 14.4 +'\Large' 14.4 14.4 17.28 +'\LARGE' 17.28 17.28 20.74 +'\huge' 20.74 20.74 24.88 +'\Huge' 24.88 24.88 24.88 + + The commands are listed here in declaration (not environment) form, +since that is how they are typically used. For example. + + \begin{quotation} \small + The Tao that can be named is not the eternal Tao. + \end{quotation} + +Here, the scope of the '\small' lasts until the end of the 'quotation' +environment. It would also end at the next type style command or the +end of the current group, so you could enclose it in curly braces +'{\small This text is typeset in the small font.}'. + + Trying to use these commands in math, as with '$\small mv^2/2$', +results in 'LaTeX Font Warning: Command \small invalid in math mode', +and the font size doesn't change. To work with a too-large formula, +often the best option is to use the 'displaymath' environment (*note +Math formulas::), or one of the environments from the 'amsmath' package. +For inline mathematics, such as in a table of formulas, an alternative +is something like '{\small $mv^2/2$}'. (Sometimes '\scriptsize' and +'\scriptstyle' are confused. Both change the font size, but the latter +also changes a number of other aspects of how mathematics is typeset. +*Note Math styles::.) + + An "environment form" of each of these commands is also defined; for +instance, '\begin{tiny}...\end{tiny}'. However, in practice this form +can easily lead to unwanted spaces at the beginning and/or end of the +environment without careful consideration, so it's generally less +error-prone to stick to the declaration form. + + (Aside: Technically, due to the way LaTeX defines '\begin' and +'\end', nearly every command that does not take an argument technically +has an environment form. But in almost all cases, it would only cause +confusion to use it. The reason for mentioning the environment form of +the font size declarations specifically is that this particular use is +not rare.) + + +File: latex2e.info, Node: Low-level font commands, Prev: Font sizes, Up: Fonts + +4.4 Low-level font commands +=========================== + +These commands are primarily intended for writers of macros and +packages. The commands listed here are only a subset of the available +ones. + +'\fontencoding{ENCODING}' + Select the font encoding, the encoding of the output font. There + are a large number of valid encodings. The most common are 'OT1', + Knuth's original encoding for Computer Modern (the default), and + 'T1', also known as the Cork encoding, which has support for the + accented characters used by the most widespread European languages + (German, French, Italian, Polish and others), which allows TeX to + hyphenate words containing accented letters. For more, see + . + +'\fontfamily{FAMILY}' + Select the font family. The web page + provides one way to browse through + many of the fonts easily used with LaTeX. Here are examples of + some common families. + + 'pag' Avant Garde + 'fvs' Bitstream Vera Sans + 'pbk' Bookman + 'bch' Charter + 'ccr' Computer Concrete + 'cmr' Computer Modern + 'cmss' Computer Modern Sans Serif + 'cmtt' Computer Modern Typewriter + 'pcr' Courier + 'phv' Helvetica + 'fi4' Inconsolata + 'lmr' Latin Modern + 'lmss' Latin Modern Sans + 'lmtt' Latin Modern Typewriter + 'pnc' New Century Schoolbook + 'ppl' Palatino + 'ptm' Times + 'uncl' Uncial + 'put' Utopia + 'pzc' Zapf Chancery + +'\fontseries{SERIES}' + Select the font series. A "series" combines a "weight" and a + "width". Typically, a font supports only a few of the possible + combinations. Some common combined series values include: + + 'm' Medium (normal) + 'b' Bold + 'c' Condensed + 'bc' Bold condensed + 'bx' Bold extended + + The possible values for weight, individually, are: + + 'ul' Ultra light + 'el' Extra light + 'l' Light + 'sl' Semi light + 'm' Medium (normal) + 'sb' Semi bold + 'b' Bold + 'eb' Extra bold + 'ub' Ultra bold + + The possible values for width, individually, are (the meaning and + relationship of these terms varies with individual typefaces): + + 'uc' Ultra condensed + 'ec' Extra condensed + 'c' Condensed + 'sc' Semi condensed + 'm' Medium + 'sx' Semi expanded + 'x' Expanded + 'ex' Extra expanded + 'ux' Ultra expanded + + When forming the SERIES string from the weight and width, drop the + 'm' that stands for medium weight or medium width, unless both + weight and width are 'm', in which case use just one (''m''). + +'\fontshape{SHAPE}' + Select font shape. Valid shapes are: + + 'n' Upright (normal) + 'it' Italic + 'sl' Slanted (oblique) + 'sc' Small caps + 'ui' Upright italics + 'ol' Outline + + The two last shapes are not available for most font families, and + small caps are often missing as well. + +'\fontsize{SIZE}{SKIP}' + Set the font size and the line spacing. The unit of both + parameters defaults to points ('pt'). The line spacing is the + nominal vertical space between lines, baseline to baseline. It is + stored in the parameter '\baselineskip'. The default + '\baselineskip' for the Computer Modern typeface is 1.2 times the + '\fontsize'. Changing '\baselineskip' directly is inadvisable + since its value is reset every time a size change happens; instead + use '\baselinestretch'. (*note \baselineskip & + \baselinestretch::). + +'\linespread{FACTOR}' + Equivalent to '\renewcommand{\baselinestretch}{FACTOR}', and + therefore must be followed by '\selectfont' to have any effect. + Best specified in the preamble. *Note \baselineskip & + \baselinestretch::, for using 'setspace' package instead. + +'\selectfont' + The effects of the font commands described above do not happen + until '\selectfont' is called, as in + '\fontfamily{FAMILYNAME}\selectfont'. It is often useful to put + this in a macro: + '\newcommand*{\myfont}{\fontfamily{FAMILYNAME}\selectfont}' + (*note \newcommand & \renewcommand::). + +'\usefont{ENC}{FAMILY}{SERIES}{SHAPE}' + The same as invoking '\fontencoding', '\fontfamily', '\fontseries' + and '\fontshape' with the given parameters, followed by + '\selectfont'. For example: + + \usefont{ot1}{cmr}{m}{n} + + +File: latex2e.info, Node: Layout, Next: Sectioning, Prev: Fonts, Up: Top + +5 Layout +******** + +Commands for controlling the general page layout. + +* Menu: + +* \onecolumn:: Use one-column layout. +* \twocolumn:: Use two-column layout. +* \flushbottom:: Make all text pages the same height. +* \raggedbottom:: Allow text pages of differing height. +* Page layout parameters:: '\headheight' '\footskip'. +* \baselineskip & \baselinestretch:: Space between lines. +* Floats:: Figures, tables, etc. + + +File: latex2e.info, Node: \onecolumn, Next: \twocolumn, Up: Layout + +5.1 '\onecolumn' +================ + +Synopsis: + + \onecolumn + + Start a new page and produce single-column output. If the document +is given the class option 'onecolumn' then this is the default behavior +(*note Document class options::). This command is fragile (*note +\protect::). + + +File: latex2e.info, Node: \twocolumn, Next: \flushbottom, Prev: \onecolumn, Up: Layout + +5.2 '\twocolumn' +================ + +Synopses: + + \twocolumn + \twocolumn[PRELIM ONE COLUMN TEXT] + + Start a new page and produce two-column output. If the document is +given the class option 'twocolumn' then this is the default (*note +Document class options::). This command is fragile (*note \protect::). + + If the optional PRELIM ONE COLUMN TEXT argument is present, it is +typeset in one-column mode before the two-column typesetting starts. + + These parameters control typesetting in two-column output: + +'\columnsep' + The distance between columns. The default is 35pt. Change it with + a command such as '\setlength{\columnsep}{40pt}'. You must change + it before the two column mode starts; in the preamble is a good + place. + +'\columnseprule' + The width of the rule between columns. The default is 0pt, meaning + that there is no rule. Otherwise, the rule appears halfway between + the two columns. Change it with a command such as + '\setlength{\columnseprule}{0.4pt}', before the two-column mode + starts. + +'\columnwidth' + The width of a single column. In one-column mode this is equal to + '\textwidth'. In two-column mode by default LaTeX sets the width + of each of the two columns, '\columnwidth', to be half of + '\textwidth' minus '\columnsep'. + + In a two-column document, the starred environments 'table*' and +'figure*' are two columns wide, whereas the unstarred environments +'table' and 'figure' take up only one column (*note figure:: and *note +table::). LaTeX places starred floats at the top of a page. The +following parameters control float behavior of two-column output. + +'\dbltopfraction' + The maximum fraction at the top of a two-column page that may be + occupied by two-column wide floats. The default is 0.7, meaning + that the height of a 'table*' or 'figure*' environment must not + exceed '0.7\textheight'. If the height of your starred float + environment exceeds this then you can take one of the following + actions to prevent it from floating all the way to the back of the + document: + + * Use the '[tp]' location specifier to tell LaTeX to try to put + the bulky float on a page by itself, as well as at the top of + a page. + + * Use the '[t!]' location specifier to override the effect of + '\dbltopfraction' for this particular float. + + * Increase the value of '\dbltopfraction' to a suitably large + number, to avoid going to float pages so soon. + + You can redefine it, as with '\renewcommand{\dbltopfraction}{0.9}'. + +'\dblfloatpagefraction' + For a float page of two-column wide floats, this is the minimum + fraction that must be occupied by floats, limiting the amount of + blank space. LaTeX's default is '0.5'. Change it with + '\renewcommand'. + +'\dblfloatsep' + On a float page of two-column wide floats, this length is the + distance between floats, at both the top and bottom of the page. + The default is '12pt plus2pt minus2pt' for a document set at '10pt' + or '11pt', and '14pt plus2pt minus4pt' for a document set at + '12pt'. + +'\dbltextfloatsep' + This length is the distance between a multi-column float at the top + or bottom of a page and the main text. The default is '20pt + plus2pt minus4pt'. + +'\dbltopnumber' + On a float page of two-column wide floats, this counter gives the + maximum number of floats allowed at the top of the page. The LaTeX + default is '2'. + + This example uses '\twocolumn''s optional argument of to create a +title that spans the two-column article: + + \documentclass[twocolumn]{article} + \newcommand{\authormark}[1]{\textsuperscript{#1}} + \begin{document} + \twocolumn[{% inside this optional argument goes one-column text + \centering + \LARGE The Title \\[1.5em] + \large Author One\authormark{1}, + Author Two\authormark{2}, + Author Three\authormark{1} \\[1em] + \normalsize + \begin{tabular}{p{.2\textwidth}@{\hspace{2em}}p{.2\textwidth}} + \authormark{1}Department one &\authormark{2}Department two \\ + School one &School two + \end{tabular}\\[3em] % space below title part + }] + + Two column text here. + + +File: latex2e.info, Node: \flushbottom, Next: \raggedbottom, Prev: \twocolumn, Up: Layout + +5.3 '\flushbottom' +================== + +Make all pages in the documents after this declaration have the same +height, by stretching the vertical space where necessary to fill out the +page. This is most often used when making two-sided documents since the +differences in facing pages can be glaring. + + If TeX cannot satisfactorily stretch the vertical space in a page +then you get a message like 'Underfull \vbox (badness 10000) has +occurred while \output is active'. If you get that, one option is to +change to '\raggedbottom' (*note \raggedbottom::). Alternatively, you +can adjust the 'textheight' to make compatible pages, or you can add +some vertical stretch glue between lines or between paragraphs, as in +'\setlength{\parskip}{0ex plus0.1ex}'. Your last option is to, in a +final editing stage, adjust the height of individual pages (*note +\enlargethispage::). + + The '\flushbottom' state is the default only if you select the +'twoside' document class option (*note Document class options::). + + +File: latex2e.info, Node: \raggedbottom, Next: Page layout parameters, Prev: \flushbottom, Up: Layout + +5.4 '\raggedbottom' +=================== + +Make all later pages the natural height of the material on that page; no +rubber vertical lengths will be stretched. Thus, in a two-sided +document the facing pages may be different heights. This command can go +at any point in the document body. See *note \flushbottom::. + + This is the default unless you select the 'twoside' document class +option (*note Document class options::). + + +File: latex2e.info, Node: Page layout parameters, Next: \baselineskip & \baselinestretch, Prev: \raggedbottom, Up: Layout + +5.5 Page layout parameters +========================== + +'\columnsep' +'\columnseprule' +'\columnwidth' + The distance between the two columns, the width of a rule between + the columns, and the width of the columns, when the document class + option 'twocolumn' is in effect (*note Document class options::). + *Note \twocolumn::. + +'\headheight' + Height of the box that contains the running head. The default in + the 'article', 'report', and 'book' classes is '12pt', at all type + sizes. + +'\headsep' + Vertical distance between the bottom of the header line and the top + of the main text. The default in the 'article' and 'report' + classes is '25pt'. In the 'book' class the default is: if the + document is set at 10pt then it is '0.25in', and at 11pt or 12pt it + is '0.275in'. + +'\footskip' + Distance from the baseline of the last line of text to the baseline + of the page footer. The default in the 'article' and 'report' + classes is '30pt'. In the 'book' class the default is: when the + type size is 10pt the default is '0.35in', while at 11pt it is + '0.38in', and at 12pt it is '30pt'. + +'\linewidth' + Width of the current line, decreased for each nested 'list' (*note + list::). That is, the nominal value for '\linewidth' is to equal + '\textwidth' but for each nested list the '\linewidth' is decreased + by the sum of that list's '\leftmargin' and '\rightmargin' (*note + itemize::). + +'\marginparpush' +'\marginsep' +'\marginparwidth' + The minimum vertical space between two marginal notes, the + horizontal space between the text body and the marginal notes, and + the horizontal width of the notes. + + Normally marginal notes appear on the outside of the page, but the + declaration '\reversemarginpar' changes that (and + '\normalmarginpar' changes it back). + + The defaults for '\marginparpush' in both 'book' and 'article' + classes are: '7pt' if the document is set at 12pt, and '5pt' if the + document is set at 11pt or 10pt. + + For '\marginsep', in 'article' class the default is '10pt' except + if the document is set at 10pt and in two-column mode where the + default is '11pt'. + + For '\marginsep' in 'book' class the default is '10pt' in + two-column mode and '7pt' in one-column mode. + + For '\marginparwidth' in both 'book' and 'article' classes, in + two-column mode the default is 60% of '\paperwidth - \textwidth', + while in one-column mode it is 50% of that distance. + +'\oddsidemargin' +'\evensidemargin' + The '\oddsidemargin' length is the extra distance between the left + side of the page and the text's left margin, on odd-numbered pages + when the document class option 'twoside' is chosen and on all pages + when 'oneside' is in effect. When 'twoside' is in effect, on + even-numbered pages the extra distance on the left is + '\evensidemargin'. + + LaTeX's default is that '\oddsidemargin' is 40% of the difference + between '\paperwidth' and '\textwidth', and '\evensidemargin' is + the remainder. + +'\paperheight' + The height of the paper, as distinct from the height of the print + area. Normally set with a document class option, as in + '\documentclass[a4paper]{article}' (*note Document class + options::). + +'\paperwidth' + The width of the paper, as distinct from the width of the print + area. Normally set with a document class option, as in + '\documentclass[a4paper]{article}' (*note Document class + options::). + +'\textheight' + The normal vertical height of the page body. If the document is + set at a nominal type size of 10pt then for an 'article' or + 'report' the default is '43\baselineskip', while for a 'book' it is + '41\baselineskip'. At a type size of 11pt the default is + '38\baselineskip' for all document classes. At 12pt it is + '36\baselineskip' for all classes. + +'\textwidth' + The full horizontal width of the entire page body. For an + 'article' or 'report' document, the default is '345pt' when the + chosen type size is 10pt, the default is '360pt' at 11pt, and it is + '390pt' at 12pt. For a 'book' document, the default is '4.5in' at + a type size of 10pt, and '5in' at 11pt or 12pt. + + In multi-column output, '\textwidth' remains the width of the + entire page body, while '\columnwidth' is the width of one column + (*note \twocolumn::). + + In lists (*note list::), '\textwidth' remains the width of the + entire page body (and '\columnwidth' the width of the entire + column), while '\linewidth' may decrease for nested lists. + + Inside a minipage (*note minipage::) or '\parbox' (*note + \parbox::), all the width-related parameters are set to the + specified width, and revert to their normal values at the end of + the 'minipage' or '\parbox'. + +'\hsize' + This entry is included for completeness: '\hsize' is the TeX + primitive parameter used when text is broken into lines. It should + not be used in normal LaTeX documents. + +'\topmargin' + Space between the top of the TeX page (one inch from the top of the + paper, by default) and the top of the header. The value is + computed based on many other parameters: '\paperheight - 2in - + \headheight - \headsep - \textheight - \footskip', and then divided + by two. + +'\topskip' + Minimum distance between the top of the page body and the baseline + of the first line of text. For the standard classes, the default + is the same as the font size, e.g., '10pt' at a type size of 10pt. + + +File: latex2e.info, Node: \baselineskip & \baselinestretch, Next: Floats, Prev: Page layout parameters, Up: Layout + +5.6 '\baselineskip' & '\baselinestretch' +======================================== + +The '\baselineskip' is a rubber length (*note Lengths::). It gives the +"leading", the normal distance between lines in a paragraph, from +baseline to baseline. + + Ordinarily document authors do not directly change '\baselineskip' +while writing. Instead, it is set by the low level font selection +command '\fontsize' (*note low level font commands fontsize::). The +'\baselineskip''s value is reset every time a font change happens and so +any direct change to '\baselineskip' would vanish the next time there +was a font switch. For how to influence line spacing, see the +discussion of '\baselinestretch' below. + + Usually, a font's size and baseline skip is assigned by the font +designer. These numbers are nominal in the sense that if, for instance, +a font's style file has the command '\fontsize{10pt}{12pt}' then that +does not mean that the characters in the font are 10pt tall; for +instance, parentheses and accented capitals may be taller. Nor does it +mean that if the lines are spaced less than 12pt apart then they risk +touching. Rather these numbers are typographic judgements. (Often, the +'\baselineskip' is about twenty percent larger than the font size.) + + The '\baselineskip' is not a property of each line but of the entire +paragraph. As a result, large text in the middle of a paragraph, such +as a single '{\Huge Q}', will be squashed into its line. TeX will make +sure it doesn't scrape up against the line above but won't change the +'\baselineskip' for that one line to make extra room above. For the +fix, use a '\strut' (*note \strut::). + + The value of '\baselineskip' that TeX uses for the paragraph is the +value in effect at the blank line or command that ends the paragraph +unit. So if a document contains this paragraph then its lines will be +scrunched together, compared to lines in surrounding paragraphs. + + Many people see a page break between text and a displayed equation as + bad style, so in effect the display is part of the paragraph. + Because this display is in footnotesize, the entire paragraph has the + baseline spacing matching that size. + {\footnotesize $$a+b = c$$} + + The process for making paragraphs is that when a new line is added, +if the depth of the previous line plus the height of the new line is +less than '\baselineskip' then TeX inserts vertical glue to make up the +difference. There are two fine points. The first is that if the lines +would be too close together, closer than '\lineskiplimit', then TeX +instead uses '\lineskip' as the interline glue. The second is that TeX +doesn't actually use the depth of the previous line. Instead it uses +'\prevdepth', which usually contains that depth. But at the beginning +of the paragraph (or any vertical list) or just after a rule, +'\prevdepth' has the value -1000pt and this special value tells TeX not +to insert any interline glue at the paragraph start. + + In the standard classes '\lineskiplimit' is 0pt and '\lineskip' is +1pt. By the prior paragraph then, the distance between lines can +approach zero but if it becomes zero (or less than zero) then the lines +jump to 1pt apart. + + Sometimes authors must, for editing purposes, put the document in +double space or one-and-a-half space. The right way to influence the +interline distance is via '\baselinestretch'. It scales +'\baselineskip', and has a default value of 1.0. It is a command, not a +length, so change the scale factor as in +'\renewcommand{\baselinestretch}{1.5}\selectfont'. + + The most straightforward way to change the line spacing for an entire +document is to put '\linespread{FACTOR}' in the preamble. For double +spacing, take FACTOR to be 1.6 and for one-and-a-half spacing use 1.3. +These number are rough: for instance, since the '\baselineskip' is about +1.2 times the font size, multiplying by 1.6 gives a baseline skip to +font size ratio of about 2. (The '\linespread' command is defined as +'\renewcommand{\baselinestretch}{FACTOR}' so it won't take effect until +a font setting happens. But that always takes place at the start of a +document, so there you don't need to follow it with '\selectfont'.) + + A simpler approach is the 'setspace' package. The basic example: + + \usepackage{setspace} + \doublespacing % or \onehalfspacing for 1.5 + +In the preamble these will start the document off with that sizing. But +you can also use these declarations in the document body to change the +spacing from that point forward, and consequently there is +'\singlespacing' to return the spacing to normal. In the document body, +a better practice than using the declarations is to use environments, +such as '\begin{doublespace} ... \end{doublespace}'. The package also +has commands to do arbitrary spacing: '\setstretch{FACTOR}' and +'\begin{spacing}{FACTOR} ... \end{spacing}'. This package also keeps +the line spacing single-spaced in places where that is typically +desirable, such as footnotes and figure captions. See the package +documentation. + + +File: latex2e.info, Node: Floats, Prev: \baselineskip & \baselinestretch, Up: Layout + +5.7 Floats +========== + +Some typographic elements, such as figures and tables, cannot be broken +across pages. They must be typeset outside of the normal flow of text, +for instance floating to the top of a later page. + + LaTeX can have a number of different classes of floating material. +The default is the two classes, 'figure' (*note figure::) and 'table' +(*note table::), but you can create a new class with the package +'float'. + + Within any one float class LaTeX always respects the order, so that +the first figure in a document source must be typeset before the second +figure. However, LaTeX may mix the classes, so it can happen that while +the first table appears in the source before the first figure, it +appears in the output after it. + + The placement of floats is subject to parameters, given below, that +limit the number of floats that can appear at the top of a page, and the +bottom, etc. If so many floats are queued that the limits prevent them +all from fitting on a page then LaTeX places what it can and defers the +rest to the next page. In this way, floats may end up being typeset far +from their place in the source. In particular, a float that is big may +migrate to the end of the document. In which event, because all floats +in a class must appear in sequential order, every following float in +that class also appears at the end. + + In addition to changing the parameters, for each float you can tweak +where the float placement algorithm tries to place it by using its +PLACEMENT argument. The possible values are a sequence of the letters +below. The default for both 'figure' and 'table', in both 'article' and +'book' classes, is 'tbp'. + +'t' + (Top)--at the top of a text page. + +'b' + (Bottom)--at the bottom of a text page. (However, 'b' is not + allowed for full-width floats ('figure*') with double-column + output. To ameliorate this, use the 'stfloats' or 'dblfloatfix' + package, but see the discussion at caveats in the FAQ: + . + +'h' + (Here)--at the position in the text where the 'figure' environment + appears. However, 'h' is not allowed by itself; 't' is + automatically added. + + To absolutely force a float to appear "here", you can + '\usepackage{float}' and use the 'H' specifier which it defines. + For further discussion, see the FAQ entry at + . + +'p' + (Page of floats)--on a separate "float page", which is a page + containing no text, only floats. + +'!' + Used in addition to one of the above; for this float only, LaTeX + ignores the restrictions on both the number of floats that can + appear and the relative amounts of float and non-float text on the + page. The '!' specifier does _not_ mean "put the float here"; see + above. + + Note: the order in which letters appear in the PLACEMENT argument +does not change the order in which LaTeX tries to place the float; for +instance, 'btp' has the same effect as 'tbp'. All that PLACEMENT does +is that if a letter is not present then the algorithm does not try that +location. Thus, LaTeX's default of 'tbp' is to try every location +except placing the float where it occurs in the source. + + To prevent LaTeX from moving floats to the end of the document or a +chapter you can use a '\clearpage' command to start a new page and +insert all pending floats. If a pagebreak is undesirable then you can +use the 'afterpage' package and issue '\afterpage{\clearpage}'. This +will wait until the current page is finished and then flush all +outstanding floats. + + LaTeX can typeset a float before where it appears in the source +(although on the same output page) if there is a 't' specifier in the +PLACEMENT parameter. If this is not desired, and deleting the 't' is +not acceptable as it keeps the float from being placed at the top of the +next page, then you can prevent it by either using the 'flafter' package +or using the command '\suppressfloats[t]', which causes floats for the +top position on this page to moved to the next page. + + Parameters relating to fractions of pages occupied by float and +non-float text (change them with '\renewcommand{PARAMETER}{DECIMAL +BETWEEN 0 AND 1}'): + +'\bottomfraction' + The maximum fraction of the page allowed to be occupied by floats + at the bottom; default '.3'. + +'\floatpagefraction' + The minimum fraction of a float page that must be occupied by + floats; default '.5'. + +'\textfraction' + Minimum fraction of a page that must be text; if floats take up too + much space to preserve this much text, floats will be moved to a + different page. The default is '.2'. + +'\topfraction' + Maximum fraction at the top of a page that may be occupied before + floats; default '.7'. + + Parameters relating to vertical space around floats (change them with +a command of the form '\setlength{PARAMETER}{LENGTH EXPRESSION}'): + +'\floatsep' + Space between floats at the top or bottom of a page; default '12pt + plus2pt minus2pt'. + +'\intextsep' + Space above and below a float in the middle of the main text; + default '12pt plus2pt minus2pt' for 10 point and 11 point + documents, and '14pt plus4pt minus4pt' for 12 point documents. + +'\textfloatsep' + Space between the last (first) float at the top (bottom) of a page; + default '20pt plus2pt minus4pt'. + + Counters relating to the number of floats on a page (change them with +a command of the form '\setcounter{CTRNAME}{NATURAL NUMBER}'): + +'bottomnumber' + Maximum number of floats that can appear at the bottom of a text + page; default 1. + +'dbltopnumber' + Maximum number of full-sized floats that can appear at the top of a + two-column page; default 2. + +'topnumber' + Maximum number of floats that can appear at the top of a text page; + default 2. + +'totalnumber' + Maximum number of floats that can appear on a text page; default 3. + + The principal TeX FAQ entry relating to floats + contains suggestions for relaxing +LaTeX's default parameters to reduce the problem of floats being pushed +to the end. A full explanation of the float placement algorithm is in +Frank Mittelbach's article "How to influence the position of float +environments like figure and table in LaTeX?" +(). + +* Menu: + +* \caption:: Make a caption for a floating environment. + + +File: latex2e.info, Node: \caption, Up: Floats + +5.7.1 '\caption' +---------------- + +Synopsis: + + \caption{CAPTION-TEXT} + +or + + \caption[SHORT-CAPTION-TEXT]{CAPTION-TEXT} + + Make a caption for a floating environment, such as a 'figure' or +'table' environment (*note figure:: or *note table::). + + In this example, LaTeX places a caption below the vertical blank +space that is left by the author for the later inclusion of a picture. + + \begin{figure} + \vspace*{1cm} + \caption{Alonzo Cushing, Battery A, 4th US Artillery.} + \label{fig:CushingPic} + \end{figure} + +The '\caption' command will label the CAPTION-TEXT with something like +'Figure 1:' for an article or 'Figure 1.1:' for a book. The text is +centered if it is shorter than the text width, or set as an unindented +paragraph if it takes more than one line. + + In addition to placing the CAPTION-TEXT in the output, the '\caption' +command also saves that information for use in a list of figures or list +of tables (*note Table of contents etc.::). + + Here the '\caption' command uses the optional SHORT-CAPTION-TEXT, so +that the shorter text appears in the list of tables, rather than the +longer CAPTION-TEXT. + + \begin{table} + \centering + \begin{tabular}{|*{3}{c}|} + \hline + 4 &9 &2 \\ + 3 &5 &7 \\ + 8 &1 &6 \\ + \hline + \end{tabular} + \caption[\textit{Lo Shu} magic square]{% + The \textit{Lo Shu} magic square, which is unique among + squares of order three up to rotation and reflection.} + \label{tab:LoShu} + \end{table} + +LaTeX will label the CAPTION-TEXT with something like 'Table 1:' for an +article or 'Table 1.1:' for a book. + + The caption can appear at the top of the 'figure' or 'table'. For +instance, that would happen in the prior example by putting the +'\caption' between the '\centering' and the '\begin{tabular}'. + + Different floating environments are numbered separately, by default. +It is '\caption' that updates the counter, and so any '\label' must come +after the '\caption'. The counter for the 'figure' environment is named +'figure', and similarly the counter for the 'table' environment is +'table'. + + The text that will be put in the list of figures or list of tables is +moving argument. If you get the LaTeX error '! Argument of \@caption +has an extra }' then you must put '\protect' in front of any fragile +commands. *Note \protect::. + + The 'caption' package has many options to adjust how the caption +appears, for example changing the font size, making the caption be +hanging text rather than set as a paragraph, or making the caption +always set as a paragraph rather than centered when it is short. + + +File: latex2e.info, Node: Sectioning, Next: Cross references, Prev: Layout, Up: Top + +6 Sectioning +************ + +Structure your text into divisions: parts, chapters, sections, etc. All +sectioning commands have the same form, one of: + + SECTIONING-COMMAND{TITLE} + SECTIONING-COMMAND*{TITLE} + SECTIONING-COMMAND[TOC-TITLE]{TITLE} + +For instance, declare the start of a subsection as with +'\subsection{Motivation}'. + + The table has each SECTIONING-COMMAND in LaTeX. All are available in +all of LaTeX's standard document classes 'book', 'report', +and 'article', except that '\chapter' is not available in 'article'. + +Sectioning unit Command Level +-------------------------------------------------------------------- +Part '\part' -1 ('book', 'report'), 0 + ('article') +Chapter '\chapter' 0 +Section '\section' 1 +Subsection '\subsection' 2 +Subsubsection '\subsubsection' 3 +Paragraph '\paragraph' 4 +Subparagraph '\subparagraph' 5 + + All these commands have a '*'-form that prints TITLE as usual but +does not number it and does not make an entry in the table of contents. +An example of using this is for an appendix in an 'article'. The input +'\appendix\section{Appendix}' gives the output 'A Appendix' (*note +\appendix::). You can lose the numbering 'A' by instead entering +'\section*{Appendix}' (articles often omit a table of contents and have +simple page headers so the other differences from the '\section' command +may not matter). + + The section title TITLE provides the heading in the main text, but it +may also appear in the table of contents and in the running head or foot +(*note Page styles::). You may not want the same text in these places +as in the main text. All of these commands have an optional argument +TOC-TITLE for these other places. + + The level number in the table above determines which sectional units +are numbered, and which appear in the table of contents. If the +sectioning command's LEVEL is less than or equal to the value of the +counter 'secnumdepth' then the titles for this sectioning command will +be numbered (*note Sectioning/secnumdepth::). And, if LEVEL is less +than or equal to the value of the counter 'tocdepth' then the table of +contents will have an entry for this sectioning unit (*note +Sectioning/tocdepth::). + + LaTeX expects that before you have a '\subsection' you will have a +'\section' and, in a 'book' class document, that before a '\section' you +will have a '\chapter'. Otherwise you can get something like a +subsection numbered '3.0.1'. + + LaTeX lets you change the appearance of the sectional units. As a +simple example, you can change the section numbering to upper-case +letters with '\renewcommand\thesection{\Alph{section}}' in the preamble +(*note \alph \Alph \arabic \roman \Roman \fnsymbol::). CTAN has many +packages that make this adjustment easier, notably 'titlesec'. + + Two counters relate to the appearance of headings made by sectioning +commands. + +'secnumdepth' + Controls which sectioning unit are numbered. Setting the counter + with '\setcounter{secnumdepth}{LEVEL}' will suppress numbering of + sectioning at any depth greater than LEVEL (*note \setcounter::). + See the above table for the level numbers. For instance, if the + 'secnumdepth' is 1 in an 'article' then a '\section{Introduction}' + command will produce output like '1 Introduction' while + '\subsection{Discussion}' will produce output like 'Discussion', + without the number. LaTeX's default 'secnumdepth' is 3 in + 'article' class and 2 in the 'book' and 'report' classes. + +'tocdepth' + Controls which sectioning units are listed in the table of + contents. The setting '\setcounter{tocdepth}{LEVEL}' makes the + sectioning units at LEVEL be the smallest ones listed (*note + \setcounter::). See the above table for the level numbers. For + instance, if 'tocdepth' is 1 then the table of contents will list + sections but not subsections. LaTeX's default 'tocdepth' is 3 in + 'article' class and 2 in the 'book' and 'report' classes. + +* Menu: + +* \part:: Start a part. +* \chapter:: Start a chapter. +* \section:: Start a section. +* \subsection:: Start a subsection. +* \subsubsection & \paragraph & \subparagraph:: Lower divisions. +* \appendix:: Start appendices. +* \frontmatter & \mainmatter & \backmatter:: The three parts of a book. +* \@startsection:: Sectional unit headings. + + +File: latex2e.info, Node: \part, Next: \chapter, Up: Sectioning + +6.1 '\part' +=========== + +Synopsis, one of: + + \part{TITLE} + \part*{TITLE} + \part[TOC-TITLE]{TITLE} + + Start a document part. The standard LaTeX classes 'book', 'report', +and 'article', all have this command. + + This produces a document part, in a book. + + \part{VOLUME I \\ + PERSONAL MEMOIRS OF U.\ S.\ GRANT} + \chapter{ANCESTRY--BIRTH--BOYHOOD.} + My family is American, and has been for generations, + in all its branches, direct and collateral. + + In each standard class the '\part' command outputs a part number such +as 'Part I', alone on its line, in boldface, and in large type. Then +LaTeX outputs TITLE, also alone on its line, in bold and in even larger +type. In class 'book', the LaTeX default puts each part alone on its +own page. If the book is two-sided then LaTeX will skip a page if +needed to have the new part on an odd-numbered page. In 'report' it is +again alone on a page, but LaTeX won't force it onto an odd-numbered +page. In an 'article' LaTeX does not put it on a fresh page, but +instead outputs the part number and part title onto the main document +page. + + The '*' form shows TITLE but it does not show the part number, does +not increment the 'part' counter, and produces no table of contents +entry. + + The optional argument TOC-TITLE will appear as the part title in the +table of contents (*note Table of contents etc.::) and in running +headers (*note Page styles::). If it is not present then TITLE will be +there. This example puts a line break in TITLE but omits the break in +the table of contents. + + \part[Up from the bottom; my life]{Up from the bottom\\ my life} + + For determining which sectional units are numbered and which appear +in the table of contents, the level number of a part is -1 (*note +Sectioning/secnumdepth::, and *note Sectioning/tocdepth::). + + In the class 'article', if a paragraph immediately follows the part +title then it is not indented. To get an indent you can use the package +'indentfirst'. + + One package to change the behavior of '\part' is 'titlesec'. See its +documentation on CTAN. + + +File: latex2e.info, Node: \chapter, Next: \section, Prev: \part, Up: Sectioning + +6.2 '\chapter' +============== + +Synopsis, one of: + + \chapter{TITLE} + \chapter*{TITLE} + \chapter[TOC-TITLE]{TITLE} + + Start a chapter. The standard LaTeX classes 'book' and 'report' have +this command but 'article' does not. + + This produces a chapter. + + \chapter{Loomings} + Call me Ishmael. + Some years ago---never mind how long precisely---having little or no + money in my purse, and nothing particular to interest me on shore, I + thought I would sail about a little and see the watery part of + the world. + + The LaTeX default starts each chapter on a fresh page, an +odd-numbered page if the document is two-sided. It produces a chapter +number such as 'Chapter 1' in large boldface type (the size is '\huge'). +It then puts TITLE on a fresh line, in boldface type that is still +larger (size '\Huge'). It also increments the 'chapter' counter, adds +an entry to the table of contents (*note Table of contents etc.::), and +sets the running header information (*note Page styles::). + + The '*' form shows TITLE on a fresh line, in boldface. But it does +not show the chapter number, does not increment the 'chapter' counter, +produces no table of contents entry, and does not affect the running +header. (If you use the page style 'headings' in a two-sided document +then the header will be from the prior chapter.) This example +illustrates. + + \chapter*{Preamble} + + The optional argument TOC-TITLE will appear as the chapter title in +the table of contents (*note Table of contents etc.::) and in running +headers (*note Page styles::). If it is not present then TITLE will be +there. This shows the full name in the chapter title, + + \chapter[Weyl]{Hermann Klaus Hugo (Peter) Weyl (1885--1955)} + +but only 'Weyl' on the contents page. This puts a line break in the +title but that doesn't work well with running headers so it omits the +break in the contents + + \chapter[Given it all; my story]{Given it all\\ my story} + + For determining which sectional units are numbered and which appear +in the table of contents, the level number of a chapter is 0 (*note +Sectioning/secnumdepth:: and *note Sectioning/tocdepth::). + + The paragraph that follows the chapter title is not indented, as is a +standard typographical practice. To get an indent use the package +'indentfirst'. + + You can change what is shown for the chapter number. To change it to +something like 'Lecture 1', put in the preamble either +'\renewcommand{\chaptername}{Lecture}' or this (*note \makeatletter & +\makeatother::). + + \makeatletter + \renewcommand{\@chapapp}{Lecture} + \makeatother + +To make this change because of the primary language for the document, +see the package 'babel'. + + In a two-sided document LaTeX puts a chapter on odd-numbered page, if +necessary leaving an even-numbered page that is blank except for any +running headers. To make that page completely blank, see *note +\clearpage & \cleardoublepage::. + + To change the behavior of the '\chapter' command, you can copy its +definition from the LaTeX format file and make adjustments. But there +are also many packages on CTAN that address this. One is 'titlesec'. +See its documentation, but the example below gives a sense of what it +can do. + + \usepackage{titlesec} % in preamble + \titleformat{\chapter} + {\Huge\bfseries} % format of title + {} % label, such as 1.2 for a subsection + {0pt} % length of separation between label and title + {} % before-code hook + +This omits the chapter number 'Chapter 1' from the page but unlike +'\chapter*' it keeps the chapter in the table of contents and the +running headers. + + +File: latex2e.info, Node: \section, Next: \subsection, Prev: \chapter, Up: Sectioning + +6.3 '\section' +============== + +Synopsis, one of: + + \section{TITLE} + \section*{TITLE} + \section[TOC-TITLE]{TITLE} + + Start a section. The standard LaTeX classes 'article', 'book', and +'report' all have this command. + + This produces a section. + + In this Part we tend to be more interested in the function, + in the input-output behavior, + than in the details of implementing that behavior. + + \section{Turing machines} + Despite this desire to downplay implementation, + we follow the approach of A~Turing that the + first step toward defining the set of computable functions + is to reflect on the details of what mechanisms can do. + + For the standard LaTeX classes 'book' and 'report' the default output +is like '1.2 TITLE' (for chapter 1, section 2), alone on its line and +flush left, in boldface and a larger type (the type size is '\Large'). +The same holds in 'article' except that there are no chapters in that +class so it looks like '2 TITLE'. + + The '*' form shows TITLE. But it does not show the section number, +does not increment the 'section' counter, produces no table of contents +entry, and does not affect the running header. (If you use the page +style 'headings' in a two-sided document then the header will be from +the prior section.) + + The optional argument TOC-TITLE will appear as the section title in +the table of contents (*note Table of contents etc.::) and in running +headers (*note Page styles::). If it is not present then TITLE will be +there. This shows the full name in the title of the section: + + \section[Elizabeth~II]{Elizabeth the Second, + by the Grace of God of the United Kingdom, + Canada and Her other Realms and Territories Queen, + Head of the Commonwealth, Defender of the Faith.} + +but only 'Elizabeth II' on the contents page and in the headers. This +has a line break in TITLE but that does not work with headers so it is +omitted from the contents and headers. + + \section[Truth is, I cheated; my life story]{Truth is, + I cheated\\my life story} + + For determining which sectional units are numbered and which appear +in the table of contents, the level number of a section is 1 (*note +Sectioning/secnumdepth:: and *note Sectioning/tocdepth::). + + The paragraph that follows the section title is not indented, as is a +standard typographical practice. One way to get an indent is to use the +package 'indentfirst'. + + In general, to change the behavior of the '\section' command, there +are a number of options. One is the '\@startsection' command (*note +\@startsection::). There are also many packages on CTAN that address +this, including 'titlesec'. See the documentation but the example below +gives a sense of what they can do. + + \usepackage{titlesec} % in preamble + \titleformat{\section} + {\normalfont\Large\bfseries} % format of title + {\makebox[1pc][r]{\thesection\hspace{1pc}}} % label + {0pt} % length of separation between label and title + {} % before-code hook + \titlespacing*{\section} + {-1pc}{18pt}{10pt}[10pc] + +That puts the section number in the margin. + + +File: latex2e.info, Node: \subsection, Next: \subsubsection & \paragraph & \subparagraph, Prev: \section, Up: Sectioning + +6.4 '\subsection' +================= + +Synopsis, one of: + + \subsection{TITLE} + \subsection*{TITLE} + \subsection[TOC-TITLE]{TITLE} + + Start a subsection. The standard LaTeX classes 'article', 'book', +and 'report' all have this command. + + This produces a subsection. + + We will show that there are more functions than Turing machines and that + therefore some functions have no associated machine. + + \subsection{Cardinality} We will begin with two paradoxes that + dramatize the challenge to our intuition posed by comparing the sizes of + infinite sets. + + For the standard LaTeX classes 'book' and 'report' the default output +is like '1.2.3 TITLE' (for chapter 1, section 2, subsection 3), alone on +its line and flush left, in boldface and a larger type (the type size is +'\large'). The same holds in 'article' except that there are no +chapters in that class so it looks like '2.3 TITLE'. + + The '*' form shows TITLE. But it does not show the subsection +number, does not increment the 'subsection' counter, and produces no +table of contents entry. + + The optional argument TOC-TITLE will appear as the subsection title +in the table of contents (*note Table of contents etc.::). If it is not +present then TITLE will be there. This shows the full text in the title +of the subsection: + + \subsection[$\alpha,\beta,\gamma$ paper]{\textit{The Origin of + Chemical Elements} by R.A.~Alpher, H.~Bethe, and G.~Gamow} + +but only 'α,β,γ paper' on the contents page. + + For determining which sectional units are numbered and which appear +in the table of contents, the level number of a subsection is 2 (*note +Sectioning/secnumdepth:: and *note Sectioning/tocdepth::). + + The paragraph that follows the subsection title is not indented, as +is a standard typographical practice. One way to get an indent is to +use the package 'indentfirst'. + + There are a number of ways to change the behavior of the +'\subsection' command. One is the '\@startsection' command (*note +\@startsection::). There are also many packages on CTAN that address +this, including 'titlesec'. See the documentation but the example below +gives a sense of what they can do. + + \usepackage{titlesec} % in preamble + \titleformat{\subsection}[runin] + {\normalfont\normalsize\bfseries} % format of the title + {\thesubsection} % label + {0.6em} % space between label and title + {} % before-code hook + +That puts the subsection number and TITLE in the first line of text. + + +File: latex2e.info, Node: \subsubsection & \paragraph & \subparagraph, Next: \appendix, Prev: \subsection, Up: Sectioning + +6.5 '\subsubsection', '\paragraph', '\subparagraph' +=================================================== + +Synopsis, one of: + + \subsubsection{TITLE} + \subsubsection*{TITLE} + \subsubsection[TOC-TITLE]{TITLE} + +or one of: + + \paragraph{TITLE} + \paragraph*{TITLE} + \paragraph[TOC-TITLE]{TITLE} + +or one of: + + \subparagraph{TITLE} + \subparagraph*{TITLE} + \subparagraph[TOC-TITLE]{TITLE} + + Start a subsubsection, paragraph, or subparagraph. The standard +LaTeX classes 'article', 'book', and 'report' all have these commands, +although they are not commonly used. + + This produces a subsubsection. + + \subsubsection{Piston ring compressors: structural performance} + Provide exterior/interior wall cladding assemblies + capable of withstanding the effects of load and stresses from + consumer-grade gasoline engine piston rings. + + The default output of each of the three does not change over the +standard LaTeX classes 'article', 'book', and 'report'. For +'\subsubsection' the TITLE is alone on its line, in boldface and normal +size type. For '\paragraph' the TITLE is inline with the text, not +indented, in boldface and normal size type. For '\subparagraph' the +TITLE is inline with the text, with a paragraph indent, in boldface and +normal size type (Because an 'article' has no chapters its +subsubsections are numbered and so it looks like '1.2.3 TITLE', for +section 1, subsection 2, and subsubsection 3. The other two divisions +are not numbered.) + + The '*' form shows TITLE. But it does not increment the associated +counter and produces no table of contents entry (and does not show the +number for '\subsubsection'). + + The optional argument TOC-TITLE will appear as the division title in +the table of contents (*note Table of contents etc.::). If it is not +present then TITLE will be there. + + For determining which sectional units are numbered and which appear +in the table of contents, the level number of a subsubsection is 3, of a +paragraph is 4, and of a subparagraph is 5 (*note +Sectioning/secnumdepth:: and *note Sectioning/tocdepth::). + + The paragraph that follows the subsubsection title is not indented, +as is a standard typographical practice. One way to get an indent is to +use the package 'indentfirst'. + + There are a number of ways to change the behavior of the these +commands. One is the '\@startsection' command (*note \@startsection::). +There are also many packages on CTAN that address this, including +'titlesec'. See the documentation on CTAN. + + +File: latex2e.info, Node: \appendix, Next: \frontmatter & \mainmatter & \backmatter, Prev: \subsubsection & \paragraph & \subparagraph, Up: Sectioning + +6.6 '\appendix' +=============== + +Synopsis: + + \appendix + + This does not directly produce any output. But in a book or report +it declares that subsequent '\chapter' commands start an appendix. In +an article it does the same, for '\section' commands. It also resets +the 'chapter' and 'section' counters to 0 in a book or report, and in an +article resets the 'section' and 'subsection' counters. + + In this book + + \chapter{One} ... + \chapter{Two} ... + ... + \appendix + \chapter{Three} ... + \chapter{Four} ... + +the first two will generate output numbered 'Chapter 1' and 'Chapter 2'. +After the '\appendix' the numbering will be 'Appendix A' and 'Appendix +B'. *Note Larger book template::, for another example. + + The 'appendix' package adds the command '\appendixpage' to put a +separate 'Appendices' in the document body before the first appendix, +and the command '\addappheadtotoc' to do the same in the table of +contents. You can reset the name 'Appendices' with a command like +'\renewcommand{\appendixname}{Specification}', as well as a number of +other features. See the documentation on CTAN. + + +File: latex2e.info, Node: \frontmatter & \mainmatter & \backmatter, Next: \@startsection, Prev: \appendix, Up: Sectioning + +6.7 '\frontmatter', '\mainmatter', '\backmatter' +================================================ + +Synopsis, one or more of: + + \frontmatter + ... + \mainmatter + ... + \backmatter + ... + + Format a 'book' class document differently according to which part of +the document is being produced. All three commands are optional. + + Traditionally, a book's front matter contains such things as the +title page, an abstract, a table of contents, a preface, a list of +notations, a list of figures, and a list of tables. (Some of these +front matter pages, such as the title page, are traditionally not +numbered.) The back matter may contain such things as a glossary, +notes, a bibliography, and an index. + + The '\frontmatter' command makes the pages numbered in lowercase +roman, and makes chapters not numbered, although each chapter's title +appears in the table of contents; if you use other sectioning commands +here, use the '*'-version (*note Sectioning::). + + The '\mainmatter' command changes the behavior back to the expected +version, and resets the page number. + + The '\backmatter' command leaves the page numbering alone but +switches the chapters back to being not numbered. + + *Note Larger book template::, for an example using these three +commands. + + +File: latex2e.info, Node: \@startsection, Prev: \frontmatter & \mainmatter & \backmatter, Up: Sectioning + +6.8 '\@startsection', typesetting sectional unit headings +========================================================= + +Synopsis: + + \@startsection{NAME}{LEVEL}{INDENT}{BEFORESKIP}{AFTERSKIP}{STYLE} + + Used to help redefine the behavior of commands that start sectioning +divisions such as '\section' or '\subsection'. + + Note that the 'titlesec' package makes manipulation of sectioning +easier. Further, while most requirements for sectioning commands can be +satisfied with '\@startsection', some cannot. For instance, in the +standard LaTeX 'book' and 'report' classes the commands '\chapter' and +'\report' are not constructed in this way. To make such a command you +may want to use the '\secdef' command. + + Technically, '\@startsection' has the form + + \@startsection{NAME} + {LEVEL} + {INDENT} + {BEFORESKIP} + {AFTERSKIP} + {STYLE}*[TOCTITLE]{TITLE} + +so that issuing + + \renewcommand{\section}{\@startsection{NAME} + {LEVEL} + {INDENT} + {BEFORESKIP} + {AFTERSKIP} + {STYLE}} + +redefines '\section' while keeping its standard calling form +'\section*[TOCTITLE]{TITLE}' (in which, for reminder, the star '*' is +optional). *Note Sectioning::. This implies that when you write a +command like '\renewcommand{\section}{...}', the '\@startsection{...}' +must come last in the definition. See the examples below. + +NAME + Name of the counter used to number the sectioning header. This + counter must be defined separately. Most commonly this is either + 'section', 'subsection', or 'paragraph'. Although in those cases + the counter name is the same as the sectioning command itself, you + don't have to use the same name. + + Then '\the'NAME displays the title number and '\'NAME'mark' is for + the page headers. See the third example below. + +LEVEL + An integer giving the depth of the sectioning command. *Note + Sectioning::, for the list of standard level numbers. + + If LEVEL is less than or equal to the value of the counter + 'secnumdepth' then titles for this sectioning command will be + numbered (*note Sectioning/secnumdepth::). For instance, if + 'secnumdepth' is 1 in an 'article' then the command + '\section{Introduction}' will produce output like "1 Introduction" + while '\subsection{Discussion}' will produce output like + "Discussion", without the number prefix. + + If LEVEL is less than or equal to the value of the counter TOCDEPTH + then the table of contents will have an entry for this sectioning + unit (*note Sectioning/tocdepth::). For instance, in an 'article', + if TOCDEPTH is 1 then the table of contents will list sections but + not subsections. + +INDENT + A length giving the indentation of all of the title lines with + respect to the left margin. To have the title flush with the + margin use '0pt'. A negative indentation such as '-\parindent' + will move the title into the left margin. + +BEFORESKIP + The absolute value of this length is the amount of vertical space + that is inserted before this sectioning unit's title. This space + will be discarded if the sectioning unit happens to start at the + beginning of a page. If this number is negative then the first + paragraph following the header is not indented, if it is + non-negative then the first paragraph is indented. (Note that the + negative of '1pt plus 2pt minus 3pt' is '-1pt plus -2pt minus + -3pt'.) + + For example, if BEFORESKIP is '-3.5ex plus -1ex minus -0.2ex' then + to start the new sectioning unit, LaTeX will add about 3.5 times + the height of a letter x in vertical space, and the first paragraph + in the section will not be indented. Using a rubber length, with + 'plus' and 'minus', is good practice here since it gives LaTeX more + flexibility in making up the page (*note Lengths::). + + The full accounting of the vertical space between the baseline of + the line prior to this sectioning unit's header and the baseline of + the header is that it is the sum of the '\parskip' of the text + font, the '\baselineskip' of the title font, and the absolute value + of the BEFORESKIP. This space is typically rubber so it may + stretch or shrink. (If the sectioning unit starts on a fresh page + so that the vertical space is discarded then the baseline of the + header text will be where LaTeX would put the baseline of the first + text line on that page.) + +AFTERSKIP + This is a length. If AFTERSKIP is non-negative then this is the + vertical space inserted after the sectioning unit's title header. + If it is negative then the title header becomes a run-in header, so + that it becomes part of the next paragraph. In this case the + absolute value of the length gives the horizontal space between the + end of the title and the beginning of the following paragraph. + (Note that the negative of '1pt plus 2pt minus 3pt' is '-1pt plus + -2pt minus -3pt'.) + + As with BEFORESKIP, using a rubber length, with 'plus' and 'minus' + components, is good practice here since it gives LaTeX more + flexibility in putting together the page. + + If 'afterskip' is non-negative then the full accounting of the + vertical space between the baseline of the sectioning unit's header + and the baseline of the first line of the following paragraph is + that it is the sum of the '\parskip' of the title font, the + '\baselineskip' of the text font, and the value of AFTER. That + space is typically rubber so it may stretch or shrink. (Note that + because the sign of 'afterskip' changes the sectioning unit + header's from standalone to run-in, you cannot use a negative + 'afterskip' to cancel part of the '\parskip'.) + +STYLE + Controls the styling of the title. See the examples below. + Typical commands to use here are '\centering', '\raggedright', + '\normalfont', '\hrule', or '\newpage'. The last command in STYLE + may be one that takes one argument, such as '\MakeUppercase' or + '\fbox' that takes one argument. The section title will be + supplied as the argument to this command. For instance, setting + STYLE to '\bfseries\MakeUppercase' would produce titles that are + bold and uppercase. + + These are LaTeX's defaults for the first three sectioning units that +are defined with '\@startsection', for the 'article', 'book', and +'report' classes. + + * For 'section': LEVEL is 1, INDENT is 0pt, BEFORESKIP is '-3.5ex + plus -1ex minus -0.2ex', AFTERSKIP is '2.3ex plus 0.2ex', and STYLE + is '\normalfont\Large\bfseries'. + + * For 'subsection': LEVEL is 2, INDENT is 0pt, BEFORESKIP is '-3.25ex + plus -1ex minus -0.2ex', AFTERSKIP is '1.5ex plus 0.2ex', and STYLE + is '\normalfont\large\bfseries'. + + * For 'subsubsection': LEVEL is 3, INDENT is 0pt, BEFORESKIP is + '-3.25ex plus -1ex minus -0.2ex', AFTERSKIP is '1.5ex plus 0.2ex', + and STYLE is '\normalfont\normalsize\bfseries'. + + Some examples follow. These go either in a package or class file or +in the preamble of a LaTeX document. If you put them in the preamble +they must go between a '\makeatletter' command and a '\makeatother'. +(Probably the error message 'You can't use `\spacefactor' in vertical +mode.' means that you forgot this.) *Note \makeatletter & +\makeatother::. + + This will put section titles in large boldface type, centered. It +says '\renewcommand' because LaTeX's standard classes have already +defined a '\section'. For the same reason it does not define a +'section' counter, or the commands '\thesection' and '\l@section'. + + \renewcommand\section{% + \@startsection{section}% *note NAME: \@startsection/name. + {1}% *note LEVEL: \@startsection/level. + {0pt}% *note INDENT: \@startsection/indent. + {-3.5ex plus -1ex minus -.2ex}% *note BEFORESKIP: \@startsection/beforeskip. + {2.3ex plus.2ex}% *note AFTERSKIP: \@startsection/afterskip. + {\centering\normalfont\Large\bfseries}% *note STYLE: \@startsection/style. + } + + This will put 'subsection' titles in small caps type, inline with the +paragraph. + + \renewcommand\subsection{% + \@startsection{subsection}% *note NAME: \@startsection/name. + {2}% *note LEVEL: \@startsection/level. + {0em}% *note INDENT: \@startsection/indent. + {-1ex plus 0.1ex minus -0.05ex}% *note BEFORESKIP: \@startsection/beforeskip. + {-1em plus 0.2em}% *note AFTERSKIP: \@startsection/afterskip. + {\scshape}% *note STYLE: \@startsection/style. + } + + The prior examples redefined existing sectional unit title commands. +This defines a new one, illustrating the needed counter and macros to +display that counter. + + \setcounter{secnumdepth}{6}% show counters this far down + \newcounter{subsubparagraph}[subparagraph]% counter for numbering + \renewcommand{\thesubsubparagraph}% how to display + {\thesubparagraph.\@arabic\c@subsubparagraph}% numbering + \newcommand{\subsubparagraph}{\@startsection + {subsubparagraph}% + {6}% + {0em}% + {\baselineskip}% + {0.5\baselineskip}% + {\normalfont\normalsize}} + \newcommand*\l@subsubparagraph{\@dottedtocline{6}{10em}{5em}}% for toc + \newcommand{\subsubparagraphmark}[1]{}% for page headers + + +File: latex2e.info, Node: Cross references, Next: Environments, Prev: Sectioning, Up: Top + +7 Cross references +****************** + +We often want something like 'See Theorem~31'. But by-hand typing the +31 is poor practice. Instead you should write a "label" such as +'\label{eq:GreensThm}' and then "reference" it, as with 'See +equation~\ref{eq:GreensThm}'. LaTeX will automatically work out the +number, put it into the output, and will change that number later if +needed. + + We will see this with Theorem~\ref{th:GreensThm}. % forward reference + ... + \begin{theorem} \label{th:GreensThm} + ... + \end{theorem} + ... + See Theorem~\ref{th:GreensThm} on page~\pageref{th:GreensThm}. + + LaTeX tracks cross reference information in a file having the +extension '.aux' and with the same base name as the file containing the +'\label'. So if '\label' is in 'calculus.tex' then the information is +in 'calculus.aux'. LaTeX puts the information in that file every time +it runs across a '\label'. + + The behavior described in the prior paragraph results in a quirk that +happens when your document has a "forward reference", a '\ref' that +appears before the associated '\label'. If this is the first time that +you are compiling the document then you will get 'LaTeX Warning: +Label(s) may have changed. Rerun to get cross references right' and in +the output the forward reference will appear as two question marks '??', +in boldface. A similar thing happens if you change some things so the +references changes; you get the same warning and the output contains the +old reference information. In both cases, resolve this by compiling the +document a second time. + + The 'cleveref' package enhances LaTeX's cross referencing features. +You can arrange that if you enter +'\begin{thm}\label{th:Nerode}...\end{thm}' then '\cref{th:Nerode}' will +output 'Theorem 3.21', without you having to enter the "Theorem." + +* Menu: + +* \label:: Assign a symbolic name to a piece of text. +* \pageref:: Refer to a page number. +* \ref:: Refer to a section, figure or similar. +* xr package:: References from another document. + + +File: latex2e.info, Node: \label, Next: \pageref, Up: Cross references + +7.1 '\label' +============ + +Synopsis: + + \label{KEY} + + Assign a reference number to KEY. In ordinary text '\label{KEY}' +assigns to KEY the number of the current sectional unit. Inside an +environment with numbering, such as a 'table' or 'theorem' environment, +'\label{KEY}' assigns to KEY the number of that environment. Retrieve +the assigned number with the '\ref{KEY}' command (*note \ref::). + + A key name can consist of any sequence of letters, digits, or common +punctuation characters. Upper and lowercase letters are distinguished, +as usual. + + A common convention is to use labels consisting of a prefix and a +suffix separated by a colon or period. Thus, '\label{fig:Post}' is a +label for a figure with a portrait of Emil Post. This helps to avoid +accidentally creating two labels with the same name, and makes your +source more readable. Some commonly-used prefixes: + +'ch' + for chapters + +'sec' +'subsec' + for lower-level sectioning commands + +'fig' + for figures + +'tab' + for tables + +'eq' + for equations + + In the auxiliary file the reference information is kept as the text +of a command of the form '\newlabel{LABEL}{{CURRENTLABEL}{PAGENUMBER}}'. +Here CURRENTLABEL is the current value of the macro '\@currentlabel' +that is usually updated whenever you call '\refstepcounter{COUNTER}'. + + Below, the key 'sec:test' will get the number of the current section +and the key 'fig:test' will get the number of the figure. +(Incidentally, put labels after captions in figures and tables.) + + \section{section name} + \label{sec:test} + This is Section~\ref{sec:test}. + \begin{figure} + ... + \caption{caption text} + \label{fig:test} + \end{figure} + See Figure~\ref{fig:test}. + + +File: latex2e.info, Node: \pageref, Next: \ref, Prev: \label, Up: Cross references + +7.2 '\pageref' +============== + +Synopsis: + + \pageref{KEY} + + Produce the page number of the place in the text where the +corresponding '\label'{KEY} command appears. + + If there is no '\label{KEY}' then you get something like 'LaTeX +Warning: Reference `th:GrensThm' on page 1 undefined on input line 11.' + + Below, the '\label{eq:main}' is used both for the formula number and +for the page number. (Note that the two references are forward +references so this document would need to be compiled twice to resolve +those.) + + The main result is formula~\ref{eq:main} on page~\pageref{eq:main}. + ... + \begin{equation} \label{eq:main} + \mathbf{P}=\mathbf{NP} + \end{equation} + + +File: latex2e.info, Node: \ref, Next: xr package, Prev: \pageref, Up: Cross references + +7.3 '\ref' +========== + +Synopsis: + + \ref{KEY} + + Produces the number of the sectional unit, equation, footnote, +figure, ..., of the corresponding '\label' command (*note \label::). It +does not produce any text, such as the word 'Section' or 'Figure', just +the bare number itself. + + If there is no '\label{KEY}' then you get something like 'LaTeX +Warning: Reference `th:GrensThm' on page 1 undefined on input line 11.' + + In this example the '\ref{popular}' produces '2'. Note that it is a +forward reference since it comes before '\label{popular}' so this +document would have to be compiled twice. + + The most widely-used format is item number~\ref{popular}. + \begin{enumerate} + \item Plain \TeX + \item \label{popular} \LaTeX + \item Con\TeX t + \end{enumerate} + + The 'cleveref' package includes text such as 'Theorem' in the +reference. See the documentation on CTAN. + + +File: latex2e.info, Node: xr package, Prev: \ref, Up: Cross references + +7.4 'xr' package +================ + +Synopsis: + + \usepackage{xr} + \externaldocument{DOCUMENT-BASENAME} + +or + + \usepackage{xr} + \externaldocument[REFERENCE-PREFIX]{DOCUMENT-BASENAME} + + Make cross references to the external document +'DOCUMENT-BASENAME.tex'. + + Here is an example. If 'lectures.tex' has this in the preamble + + \usepackage{xr} + \externaldocument{exercises} + \externaldocument[H-]{hints} + \externaldocument{answers} + +then it can use cross reference labels from the other three documents. +Suppose that 'exercises.tex' has an enumerated list containing this, + + \item \label{exer:EulersThm} What if every vertex has odd degree? + +and 'hints.tex' has an enumerated list with this, + + \item \label{exer:EulersThm} Distinguish the case of two vertices. + +and 'answers.tex' has an enumerated list with this, + + \item \label{ans:EulersThm} There is no Euler path, except if there + are exactly two vertices. + + After compiling the exercises, hints, and answers documents, entering +this in the body of 'lectures.tex' will result in the lectures getting +the reference numbers used in the other documents. + + See Exercise~\ref{exer:EulersThm}, with Hint~\ref{H-exer:EulersThm}. + The solution is Answer~\ref{ans:EulersThm}. + + The prefix 'H-' for the reference from the hints file is needed +because the label in the hints file is the same as the label in the +exercises file. Without that prefix, both references would get the +number from the later file. + + Note: if the document uses the 'hyperref' package then in place of +'xr', put '\usepackage{xr-hyper}' before the '\usepackage{hyperref}'. +Also, if any of the multiple documents uses 'hyperref' then they all +must use it. + + +File: latex2e.info, Node: Environments, Next: Line breaking, Prev: Cross references, Up: Top + +8 Environments +************** + +LaTeX provides many environments for delimiting certain behavior. An +environment begins with '\begin' and ends with '\end', like this: + + \begin{ENVIRONMENT-NAME} + ... + \end{ENVIRONMENT-NAME} + + The ENVIRONMENT-NAME at the beginning must exactly match that at the +end. For instance, the input '\begin{table*}...\end{table}' will cause +an error like: '! LaTeX Error: \begin{table*} on input line 5 ended by +\end{table}.' + + Environments are executed within a group. + +* Menu: + +* abstract:: Produce an abstract. +* array:: Math arrays. +* center:: Centered lines. +* description:: Labelled lists. +* displaymath:: Formulas that appear on their own line. +* document:: Enclose the whole document. +* enumerate:: Numbered lists. +* eqnarray:: Sequences of aligned equations. +* equation:: Displayed equation. +* figure:: Floating figures. +* filecontents:: Writing multiple files from the source. +* flushleft:: Flushed left lines. +* flushright:: Flushed right lines. +* itemize:: Bulleted lists. +* letter:: Letters. +* list:: Generic list environment. +* math:: In-line math. +* minipage:: Miniature page. +* picture:: Picture with text, arrows, lines and circles. +* quotation & quote:: Include a quotation. +* tabbing:: Align text arbitrarily. +* table:: Floating tables. +* tabular:: Align text in columns. +* thebibliography:: Bibliography or reference list. +* theorem:: Theorems, lemmas, etc. +* titlepage:: For hand crafted title pages. +* verbatim:: Simulating typed input. +* verse:: For poetry and other things. + + +File: latex2e.info, Node: abstract, Next: array, Up: Environments + +8.1 'abstract' +============== + +Synopsis: + + \begin{abstract} + ... + \end{abstract} + + Produce an abstract, possibly of multiple paragraphs. This +environment is only defined in the 'article' and 'report' document +classes (*note Document classes::). + + Using the example below in the 'article' class produces a displayed +paragraph. Document class option 'titlepage' causes the abstract to be +on a separate page (*note Document class options::); this is the default +only in the 'report' class. + + \begin{abstract} + We compare all known accounts of the proposal made by Porter Alexander + to Robert E Lee at the Appomattox Court House that the army continue + in a guerrilla war, which Lee refused. + \end{abstract} + + The next example produces a one column abstract in a two column +document (for a more flexible solution, use the package 'abstract'). + + \documentclass[twocolumn]{article} + ... + \begin{document} + \title{Babe Ruth as Cultural Progenitor: a Atavistic Approach} + \author{Smith \\ Jones \\ Robinson\thanks{Railroad tracking grant.}} + \twocolumn[ + \begin{@twocolumnfalse} + \maketitle + \begin{abstract} + Ruth was not just the Sultan of Swat, he was the entire swat + team. + \end{abstract} + \end{@twocolumnfalse} + ] + { % by-hand insert a footnote at page bottom + \renewcommand{\thefootnote}{\fnsymbol{footnote}} + \footnotetext[1]{Thanks for all the fish.} + } + + +File: latex2e.info, Node: array, Next: center, Prev: abstract, Up: Environments + +8.2 'array' +=========== + +Synopsis: + + \begin{array}{COLS} + COLUMN 1 ENTRY &COLUMN 2 ENTRY ... &COLUMN N ENTRY \\ + ... + \end{array} + +or: + + \begin{array}[POS]{COLS} + COLUMN 1 ENTRY &COLUMN 2 ENTRY ... &COLUMN N ENTRY \\ + ... + \end{array} + + Produce a mathematical array. This environment can only be used in +math mode (*note Modes::), and normally appears within a displayed +mathematics environment such as 'equation' (*note equation::). Inside +of each row the column entries are separated by an ampersand, ('&'). +Rows are terminated with double-backslashes (*note \\::). + + This example shows a three by three array. + + \begin{equation*} + \chi(x) = + \left| % vertical bar fence + \begin{array}{ccc} + x-a &-b &-c \\ + -d &x-e &-f \\ + -g &-h &x-i + \end{array} + \right| + \end{equation*} + + The required argument COLS describes the number of columns, their +alignment, and the formatting of the intercolumn regions. For instance, +'\begin{array}{rcl}...\end{array}' gives three columns: the first flush +right, the second centered, and the third flush left. See *note +tabular:: for the complete description of COLS and of the other common +features of the two environments, including the optional POS argument. + + There are two ways that 'array' diverges from 'tabular'. The first +is that 'array' entries are typeset in math mode, in textstyle (*note +Math styles::) except if the COLS definition specifies the column with +'p{...}', which causes the entry to be typeset in text mode. The second +is that, instead of 'tabular''s parameter '\tabcolsep', LaTeX's +intercolumn space in an 'array' is governed by '\arraycolsep', which +gives half the width between columns. The default for this is '5pt' so +that between two columns comes 10pt of space. + + To obtain arrays with braces the standard is to use the 'amsmath' +package. It comes with environments 'pmatrix' for an array surrounded +by parentheses '(...)', 'bmatrix' for an array surrounded by square +brackets '[...]', 'Bmatrix' for an array surrounded by curly +braces '{...}', 'vmatrix' for an array surrounded by vertical +bars '|...|', and 'Vmatrix' for an array surrounded by double vertical +bars '||...||', along with a number of other array constructs. + + The next example uses the 'amsmath' package. + + \usepackage{amsmath} % in preamble + + \begin{equation} + \begin{vmatrix}{cc} % array with vert lines + a &b \\ + c &d + \end{vmatrix}=ad-bc + \end{equation} + + There are many packages concerning arrays. The 'array' package has +many useful extensions, including more column types. The 'dcolumn' +package adds a column type to center on a decimal point. For both see +the documentation on CTAN. + + +File: latex2e.info, Node: center, Next: description, Prev: array, Up: Environments + +8.3 'center' +============ + +Synopsis: + + \begin{center} + LINE1 \\ + LINE2 \\ + ... + \end{center} + + Create a new paragraph consisting of a sequence of lines that are +centered within the left and right margins. Use double-backslash, '\\', +to get a line break (*note \\::). If some text is too long to fit on a +line then LaTeX will insert line breaks that avoid hyphenation and avoid +stretching or shrinking any interword space. + + This environment inserts space above and below the text body. See +*note \centering:: to avoid such space, for example inside a 'figure' +environment. + + This example produces three centered lines. There is extra vertical +space between the last two lines. + + \begin{center} + A Thesis Submitted in Partial Fufillment \\ + of the Requirements of \\[0.5ex] + the School of Environmental Engineering + \end{center} + + In this example, depending on the page's line width, LaTeX may choose +a line break for the part before the double backslash. If so, it will +center each of the two lines and if not it will center the single line. +Then LaTeX will break at the double backslash, and will center the +ending. + + \begin{center} + My father considered that anyone who went to chapel and didn't drink + alcohol was not to be tolerated.\\ + I grew up in that belief. ---Richard Burton + \end{center} + + A double backslash after the final line is optional. If present it +doesn't add any vertical space. + + In a two-column document the text is centered in a column, not in the +entire page. + +* Menu: + +* \centering:: Declaration form of the 'center' environment. + + +File: latex2e.info, Node: \centering, Up: center + +8.3.1 '\centering' +------------------ + +Synopsis: + + {\centering ... } + +or + + \begin{group} + \centering ... + \end{group} + + Center the material in its scope. It is most often used inside an +environment such as 'figure', or in a 'parbox'. + + This example's '\centering' declaration causes the graphic to be +horizontally centered. + + \begin{figure} + \centering + \includegraphics[width=0.6\textwidth]{ctan_lion.png} + \caption{CTAN Lion} \label{fig:CTANLion} + \end{figure} + +The scope of this '\centering' ends with the '\end{figure}'. + + Unlike the 'center' environment, the '\centering' command does not +add vertical space above and below the text. That's its advantage in +the above example; there is not an excess of space. + + It also does not start a new paragraph; it simply changes how LaTeX +formats paragraph units. If 'ww {\centering xx \\ yy} zz' is surrounded +by blank lines then LaTeX will create a paragraph whose first line 'ww +xx' is centered and whose second line, not centered, contains 'yy zz'. +Usually what is desired is for the scope of the declaration to contain a +blank line or the '\end' command of an environment such as 'figure' or +'table' that ends the paragraph unit. Thus, if '{\centering xx \\ +yy\par} zz' is surrounded by blank lines then it makes a new paragraph +with two centered lines 'xx' and 'yy', followed by a new paragraph with +'zz' that is formatted as usual. + + +File: latex2e.info, Node: description, Next: displaymath, Prev: center, Up: Environments + +8.4 'description' +================= + +Synopsis: + + \begin{description} + \item[LABEL OF FIRST ITEM] TEXT OF FIRST ITEM + \item[LABEL OF SECOND ITEM] TEXT OF SECOND ITEM + ... + \end{description} + + Environment to make a list of labeled items. Each item's LABEL is +typeset in bold and is flush left, so that long labels continue into the +first line of the item text. There must be at least one item; having +none causes the LaTeX error 'Something's wrong--perhaps a missing +\item'. + + This example shows the environment used for a sequence of +definitions. + + \begin{description} + \item[lama] A priest. + \item[llama] A beast. + \end{description} + +The labels 'lama' and 'llama' are output in boldface, with the left edge +on the left margin. + + Start list items with the '\item' command (*note \item::). Use the +optional labels, as in '\item[Main point]', because there is no sensible +default. Following the '\item' is optional text, which may contain +multiple paragraphs. + + Since the labels are in bold style, if the label text calls for a +font change given in argument style (see *note Font styles::) then it +will come out bold. For instance, if the label text calls for +typewriter with '\item[\texttt{label text}]' then it will appear in bold +typewriter, if that is available. The simplest way around this, in this +example to get non-bold typewriter, is to use declarative style: +'\item[{\tt label text}]'. Similarly, get the standard roman font with +'\item[{\rm label text}]'. + + For other major LaTeX labelled list environments, see *note itemize:: +and *note enumerate::. Unlike those environments, nesting 'description' +environments does not change the default label; it is boldface and flush +left at all levels. + + For information about list layout parameters, including the default +values, and for information about customizing list layout, see *note +list::. The package 'enumitem' is useful for customizing lists. + + This example changes the description labels to small caps. + + \renewcommand{\descriptionlabel}[1]{% + {\hspace{\labelsep}\textsc{#1}}} + + +File: latex2e.info, Node: displaymath, Next: document, Prev: description, Up: Environments + +8.5 'displaymath' +================= + +Synopsis: + + \begin{displaymath} + MATHEMATICAL TEXT + \end{displaymath} + + Environment to typeset the MATHEMATICAL TEXT on its own line, in +display style and centered. To make the text be flush-left use the +global option 'fleqn'; see *note Document class options::. + + In the 'displaymath' environment no equation number is added to the +math text. One way to get an equation number is to use the 'equation' +environment (*note equation::). + + LaTeX will not break the MATH TEXT across lines. + + Note that the 'amsmath' package has significantly more extensive +displayed equation facilities. For example, there are a number of ways +in that package for having math text broken across lines. + + The construct '\[MATH\]' is a synonym for the environment +'\begin{displaymath} MATH \end{displaymath}' but the latter is easier to +work with in the source; for instance, searching for a square bracket +may get false positives but the word 'displaymath' will likely be +unique. + + (Aside: The construct '$$MATH$$' from Plain TeX is sometimes +mistakenly used as a synonym for 'displaymath'. It is not a synonym, +and is not officially supported in LaTeX at all; '$$' doesn't support +the 'fleqn' option (*note Document class options::), has different +vertical spacing, and doesn't perform consistency checks.) + + The output from this example is centered and alone on its line. + + \begin{displaymath} + \int_1^2 x^2\,dx=7/3 + \end{displaymath} + +Also, the integral sign is larger than the inline version '\( \int_1^2 +x^2\,dx=7/3 \)' produces. + + +File: latex2e.info, Node: document, Next: enumerate, Prev: displaymath, Up: Environments + +8.6 'document' +============== + +The 'document' environment encloses the entire body of a document. It +is required in every LaTeX document. *Note Starting and ending::. + +* Menu: + +* \AtBeginDocument:: Hook for commands at the start of the document. +* \AtEndDocument:: Hook for commands at the end of the document. + + +File: latex2e.info, Node: \AtBeginDocument, Next: \AtEndDocument, Up: document + +8.6.1 '\AtBeginDocument' +------------------------ + +Synopsis: + + \AtBeginDocument{CODE} + + Save CODE and execute it when '\begin{document}' is executed, at the +very end of the preamble. The code is executed after the font selection +tables have been set up, so the normal font for the document is the +current font. However, the code is executed as part of the preamble so +you cannot do any typesetting with it. + + You can issue this command more than once; the successive code lines +will be executed in the order that you gave them. + + +File: latex2e.info, Node: \AtEndDocument, Prev: \AtBeginDocument, Up: document + +8.6.2 '\AtEndDocument' +---------------------- + +Synopsis: + + \AtEndDocument{CODE} + + Save CODE and execute it near the end of the document. Specifically, +it is executed when '\end{document}' is executed, before the final page +is finished and before any leftover floating environments are processed. +If you want some of the code to be executed after these two processes +then include a '\clearpage' at the appropriate point in CODE. + + You can issue this command more than once; the successive code lines +will be executed in the order that you gave them. + + +File: latex2e.info, Node: enumerate, Next: eqnarray, Prev: document, Up: Environments + +8.7 'enumerate' +=============== + +Synopsis: + + \begin{enumerate} + \item[OPTIONAL LABEL OF FIRST ITEM] TEXT OF FIRST ITEM + \item[OPTIONAL LABEL OF SECOND ITEM] TEXT OF SECOND ITEM + ... + \end{enumerate} + + Environment to produce a numbered list of items. The format of the +label numbering depends on the nesting level of this environment; see +below. The default top-level numbering is '1.', '2.', etc. Each +'enumerate' list environment must have at least one item; having none +causes the LaTeX error 'Something's wrong--perhaps a missing \item'. + + This example gives the first two finishers in the 1908 Olympic +marathon. As a top-level list the labels would come out as '1.' and +'2.'. + + \begin{enumerate} + \item Johnny Hayes (USA) + \item Charles Hefferon (RSA) + \end{enumerate} + + Start list items with the '\item' command (*note \item::). If you +give '\item' an optional argument by following it with square brackets, +as in '\item[Interstitial label]', then the next item will continue the +interrupted sequence (*note \item::). That is, you will get labels like +'1.', then 'Interstitial label', then '2.'. Following the '\item' is +optional text, which may contain multiple paragraphs. + + Enumerations may be nested within other 'enumerate' environments, or +within any paragraph-making environment such as 'itemize' (*note +itemize::), up to four levels deep. This gives LaTeX's default for the +format at each nesting level, where 1 is the top level, the outermost +level. + + 1. arabic number followed by a period: '1.', '2.', ... + 2. lowercase letter inside parentheses: '(a)', '(b)' ... + 3. lowercase roman numeral followed by a period: 'i.', 'ii.', ... + 4. uppercase letter followed by a period: 'A.', 'B.', ... + + The 'enumerate' environment uses the counters '\enumi' through +'\enumiv' (*note Counters::). + + For other major LaTeX labeled list environments, see *note +description:: and *note itemize::. For information about list layout +parameters, including the default values, and for information about +customizing list layout, see *note list::. The package 'enumitem' is +useful for customizing lists. + + To change the format of the label use '\renewcommand' (*note +\newcommand & \renewcommand::) on the commands '\labelenumi' through +'\labelenumiv'. For instance, this first level list will be labelled +with uppercase letters, in boldface, and without a trailing period. + + \renewcommand{\labelenumi}{\textbf{\Alph{enumi}}} + \begin{enumerate} + \item Shows as boldface A + \item Shows as boldface B + \end{enumerate} + + For a list of counter-labeling commands see *note \alph \Alph \arabic +\roman \Roman \fnsymbol::. + + +File: latex2e.info, Node: eqnarray, Next: equation, Prev: enumerate, Up: Environments + +8.8 'eqnarray' +============== + +The 'eqnarray' environment is obsolete. It has infelicities, including +spacing that is inconsistent with other mathematics elements. (See +"Avoid eqnarray!" by Lars Madsen +). New documents should +include the 'amsmath' package and use the displayed mathematics +environments provided there, such as the 'align' environment. We +include a description only for completeness and for working with old +documents. + + Synopsis: + + \begin{eqnarray} + FIRST FORMULA LEFT &FIRST FORMULA MIDDLE &FIRST FORMULA RIGHT \\ + ... + \end{eqnarray} + +or + + \begin{eqnarray*} + FIRST FORMULA LEFT &FIRST FORMULA MIDDLE &FIRST FORMULA RIGHT \\ + ... + \end{eqnarray*} + + Display a sequence of equations or inequalities. The left and right +sides are typeset in display mode, while the middle is typeset in text +mode. + + It is similar to a three-column 'array' environment, with items +within a row separated by an ampersand ('&'), and with rows separated by +double backslash '\\'). The starred form of line break ('\\*') can +also be used to separate equations, and will disallow a page break there +(*note \\::). + + The unstarred form 'eqnarray' places an equation number on every line +(using the 'equation' counter), unless that line contains a '\nonumber' +command. The starred form 'eqnarray*' omits equation numbering, while +otherwise being the same. + + The command '\lefteqn' is used for splitting long formulas across +lines. It typesets its argument in display style flush left in a box of +zero width. + + This example shows three lines. The first two lines make an +inequality, while the third line has not entry on the left side. + + \begin{eqnarray*} + \lefteqn{x_1+x_2+\cdots+x_n} \\ + &\leq &y_1+y_2+\cdots+y_n \\ + &= &z+y_3+\cdots+y_n + \end{eqnarray*} + + +File: latex2e.info, Node: equation, Next: figure, Prev: eqnarray, Up: Environments + +8.9 'equation' +============== + +Synopsis: + + \begin{equation} + MATHEMATICAL TEXT + \end{equation} + + The same as a 'displaymath' environment (*note displaymath::) except +that LaTeX puts an equation number flush to the right margin. The +equation number is generated using the 'equation' counter. + + You should have no blank lines between '\begin{equation}' and +'\begin{equation}', or LaTeX will tell you that there is a missing +dollar sign. + + The package 'amsmath' package has extensive displayed equation +facilities. New documents should include this package. + + +File: latex2e.info, Node: figure, Next: filecontents, Prev: equation, Up: Environments + +8.10 'figure' +============= + +Synopsis: + + \begin{figure}[PLACEMENT] + FIGURE BODY + \caption[LOFTITLE]{TITLE} % optional + \label{LABEL} % optional + \end{figure} + +or: + + \begin{figure*}[PLACEMENT] + FIGURE BODY + \caption[LOFTITLE]{TITLE} % optional + \label{LABEL} % optional + \end{figure*} + + Figures are for material that is not part of the normal text. An +example is material that you cannot have split between two pages, such +as a graphic. Because of this, LaTeX does not typeset figures in +sequence with normal text but instead "floats" them to a convenient +place, such as the top of a following page (*note Floats::). + + The FIGURE BODY can consist of imported graphics (*note Graphics::), +or text, LaTeX commands, etc. It is typeset in a 'parbox' of width +'\textwidth'. + + The possible values of PLACEMENT are 'h' for 'here', 't' for 'top', +'b' for 'bottom', and 'p' for 'on a separate page of floats'. For the +effect of these options on the float placement algorithm, see *note +Floats::. + + The starred form 'figure*' is used when a document is in +double-column mode (*note \twocolumn::). It produces a figure that +spans both columns, at the top of the page. To add the possibility of +placing at a page bottom see the discussion of PLACEMENT 'b' in *note +Floats::. + + The label is optional; it is used for cross references (*note Cross +references::). The optional '\caption' command specifies caption text +for the figure (*note \caption::). By default it is numbered. If +LOFTITLE is present, it is used in the list of figures instead of TITLE +(*note Table of contents etc.::). + + This example makes a figure out of a graphic. LaTeX will place that +graphic and its caption at the top of a page or, if it is pushed to the +end of the document, on a page of floats. + + \usepackage{graphicx} % in preamble + ... + \begin{figure}[t] + \centering + \includegraphics[width=0.5\textwidth]{CTANlion.png} + \caption{The CTAN lion, by Duane Bibby} + \end{figure} + + +File: latex2e.info, Node: filecontents, Next: flushleft, Prev: figure, Up: Environments + +8.11 'filecontents' +=================== + +Synopsis: + + \begin{filecontents}[OPTION]{FILENAME} + TEXT + \end{filecontents} + +or + + \begin{filecontents*}[OPTION]{FILENAME} + TEXT + \end{filecontents*} + + Create a file named FILENAME in the current directory (or the output +directory, if specified; *note output directory::) and write TEXT to it. +By default, an existing file is not overwritten. + + The unstarred version of the environment 'filecontents' prefixes the +content of the created file with a header of TeX comments; see the +example below. The starred version 'filecontents*' does not include the +header. + + The possible options are: + +'force' +'overwrite' + Overwrite an existing file. + +'noheader' + Omit the header. Equivalent to using 'filecontents*'. + +'nosearch' + Only check the current directory (and the output directory, if + specified) for an existing file, not the entire search path. + + These options were added in a 2019 release of LaTeX. + + This environment can be used anywhere in the preamble, although it +often appears before the '\documentclass' command. It is commonly used +to create a '.bib' or other such data file from the main document +source, to make the source file self-contained. Similarly, it can be +used to create a custom style or class file, again making the source +self-contained. + + For example, this document: + + \documentclass{article} + \begin{filecontents}{JH.sty} + \newcommand{\myname}{Jim Hef{}feron} + \end{filecontents} + \usepackage{JH} + \begin{document} + Article by \myname. + \end{document} + +produces this file 'JH.sty': + + %% LaTeX2e file `JH.sty' + %% generated by the `filecontents' environment + %% from source `test' on 2015/10/12. + %% + \newcommand{\myname}{Jim Hef{}feron} + + +File: latex2e.info, Node: flushleft, Next: flushright, Prev: filecontents, Up: Environments + +8.12 'flushleft' +================ + +Synopsis: + + \begin{flushleft} + LINE1 \\ + LINE2 \\ + ... + \end{flushleft} + + An environment that creates a paragraph whose lines are flush to the +left-hand margin, and ragged right. If you have lines that are too long +then LaTeX will linebreak them in a way that avoids hyphenation and +stretching or shrinking interword spaces. To force a new line use a +double backslash, '\\'. For the declaration form see *note +\raggedright::. + + This creates a box of text that is at most 3 inches wide, with the +text flush left and ragged right. + + \noindent\begin{minipage}{3in} + \begin{flushleft} + A long sentence that will be broken by \LaTeX{} + at a convenient spot. \\ + And, a fresh line forced by the double backslash. + \end{flushleft} + \end{minipage} + +* Menu: + +* \raggedright:: Declaration form of the 'flushleft' environment. + + +File: latex2e.info, Node: \raggedright, Up: flushleft + +8.12.1 '\raggedright' +--------------------- + +Synopses: + + {\raggedright ... } + +or + + \begin{ENVIRONMENT} \raggedright + ... + \end{ENVIRONMENT} + + A declaration which causes lines to be flush to the left margin and +ragged right. It can be used inside an ENVIRONMENT such as 'quote' or +in a 'parbox'. For the environment form see *note flushleft::. + + Unlike the 'flushleft' environment, the '\raggedright' command does +not start a new paragraph; it only changes how LaTeX formats paragraph +units. To affect a paragraph unit's format, the scope of the +declaration must contain the blank line or '\end' command that ends the +paragraph unit. + + Here '\raggedright' in each second column keeps LaTeX from doing very +awkward typesetting to fit the text into the narrow column. Note that +'\raggedright' is inside the curly braces '{...}' to delimit its effect. + + \begin{tabular}{rp{2in}} + Team alpha &{\raggedright This team does all the real work.} \\ + Team beta &{\raggedright This team ensures that the water + cooler is never empty.} \\ + \end{tabular} + + +File: latex2e.info, Node: flushright, Next: itemize, Prev: flushleft, Up: Environments + +8.13 'flushright' +================= + + \begin{flushright} + LINE1 \\ + LINE2 \\ + ... + \end{flushright} + + An environment that creates a paragraph whose lines are flush to the +right-hand margin and ragged left. If you have lines that are too long +to fit the margins then LaTeX will linebreak them in a way that avoids +hyphenation and stretching or shrinking inter-word spaces. To force a +new line use a double backslash, '\\'. For the declaration form +see *note \raggedleft::. + + For an example related to this environment, see *note flushleft::, +where one just have mutatis mutandis to replace 'flushleft' by +'flushright'. + +* Menu: + +* \raggedleft:: Declaration form of the 'flushright' environment. + + +File: latex2e.info, Node: \raggedleft, Up: flushright + +8.13.1 '\raggedleft' +-------------------- + +Synopses: + + {\raggedleft ... } + +or + + \begin{ENVIRONMENT} \raggedleft + ... + \end{ENVIRONMENT} + + A declaration which causes lines to be flush to the right margin and +ragged left. It can be used inside an ENVIRONMENT such as 'quote' or in +a 'parbox'. For the environment form see *note flushright::. + + Unlike the 'flushright' environment, the '\raggedleft' command does +not start a new paragraph; it only changes how LaTeX formats paragraph +units. To affect a paragraph unit's format, the scope of the +declaration must contain the blank line or '\end' command that ends the +paragraph unit. + + For an example related to this environment, see *note \raggedright::, +where one just have mutatis mutandis to replace '\raggedright' by +'\raggedleft'. + + +File: latex2e.info, Node: itemize, Next: letter, Prev: flushright, Up: Environments + +8.14 'itemize' +============== + +Synopsis: + + \begin{itemize} + \item[OPTIONAL LABEL OF FIRST ITEM] TEXT OF FIRST ITEM + \item[OPTIONAL LABEL OF SECOND ITEM] TEXT OF SECOND ITEM + ... + \end{itemize} + + Produce an "unordered list", sometimes called a bullet list. There +must be at least one '\item' within the environment; having none causes +the LaTeX error 'Something's wrong--perhaps a missing \item'. + + This gives a two-item list. + + \begin{itemize} + \item Pencil and watercolor sketch by Cassandra + \item Rice portrait + \end{itemize} + +With the default locale--without loading e.g. 'babel' package with +another language than USenglish--as a top-level list each label would +come out as a bullet, *. The format of the labeling depends on the +nesting level; see below. + + Start list items with the '\item' command (*note \item::). If you +give '\item' an optional argument by following it with square brackets, +as in '\item[OPTIONAL LABEL]', then by default OPTIONAL LABEL will +appear in bold and be flush right, so it could extend into the left +margin. For labels that are flush left see the *note description:: +environment. Following the '\item' is the text of the item, which may +be empty or contain multiple paragraphs. + + Unordered lists can be nested within one another, up to four levels +deep. They can also be nested within other paragraph-making +environments, such as 'enumerate' (*note enumerate::). + + The 'itemize' environment uses the commands '\labelitemi' through +'\labelitemiv' to produce the default label (note the convention of +lowercase roman numerals at the end of the command names that signify +the nesting level). These are the default marks at each level. + + 1. * (bullet, from '\textbullet') + 2. -- (bold en-dash, from '\normalfont\bfseries\textendash') + 3. * (asterisk, from '\textasteriskcentered') + 4. . (vertically centered dot, rendered here as a period, from + '\textperiodcentered') + + Change the labels with '\renewcommand'. For instance, this makes the +first level use diamonds. + + \renewcommand{\labelitemi}{$\diamond$} + + The distance between the left margin of the enclosing environment and +the left margin of the 'itemize' list is determined by the parameters +'\leftmargini' through '\leftmarginvi'. (This also uses the convention +of using lowercase roman numerals a the end of the command name to +denote the nesting level.) The defaults are: '2.5em' in level 1 ('2em' +in two-column mode), '2.2em' in level 2, '1.87em' in level 3, and +'1.7em' in level 4, with smaller values for more deeply nested levels. + + For other major LaTeX labeled list environments, see *note +description:: and *note enumerate::. The 'itemize', 'enumerate' and +'description' environment use the same list layout parameters. For a +description, including the default values, and for information about +customizing list layout, see *note list::. The package 'enumitem' is +useful for customizing lists. + + This example greatly reduces the margin space for outermost itemized +lists. + + \setlength{\leftmargini}{1.25em} % default 2.5em + + Especially for lists with short items, it may be desirable to elide +space between items. Here is an example defining an 'itemize*' +environment with no extra spacing between items, or between paragraphs +within a single item ('\parskip' is not list-specific, *note \parindent +& \parskip::): + + \newenvironment{itemize*}% + {\begin{itemize}% + \setlength{\itemsep}{0pt}% + \setlength{\parsep}{0pt}}% + \setlength{\parskip}{0pt}}% + {\end{itemize}} + + +File: latex2e.info, Node: letter, Next: list, Prev: itemize, Up: Environments + +8.15 'letter' environment: writing letters +========================================== + +This environment is used for creating letters. *Note Letters::. + + +File: latex2e.info, Node: list, Next: math, Prev: letter, Up: Environments + +8.16 'list' +=========== + +Synopsis: + + \begin{list}{LABELING}{SPACING} + \item[OPTIONAL LABEL OF FIRST ITEM] TEXT OF FIRST ITEM + \item[OPTIONAL LABEL OF SECOND ITEM] TEXT OF SECOND ITEM + ... + \end{list} + + An environment for constructing lists. + + Note that this environment does not typically appear in the document +body. Most lists created by LaTeX authors are the ones that come +standard: the 'description', 'enumerate', and 'itemize' environments +(*note description::, *note enumerate::, and *note itemize::). + + Instead, the 'list' environment is most often used in macros. For +example, many standard LaTeX environments that do not immediately appear +to be lists are in fact constructed using 'list', including 'quotation', +'quote', and 'center' (*note quotation & quote::, *note center::). + + This uses the 'list' environment to define a new custom environment. + + \newcounter{namedlistcounter} % number the items + \newenvironment{named} + {\begin{list} + {Item~\Roman{namedlistcounter}.} % labeling + {\usecounter{namedlistcounter} % set counter + \setlength{\leftmargin}{3.5em}} % set spacing + } + {\end{list}} + + \begin{named} + \item Shows as ``Item~I.'' + \item[Special label.] Shows as ``Special label.'' + \item Shows as ``Item~II.'' + \end{named} + + The mandatory first argument LABELING specifies the default labeling +of list items. It can contain text and LaTeX commands, as above where +it contains both 'Item' and '\Roman{...}'. LaTeX forms the label by +putting the LABELING argument in a box of width '\labelwidth'. If the +label is wider than that, the additional material extends to the right. +When making an instance of a 'list' you can override the default +labeling by giving '\item' an optional argument by including square +braces and the text, as in the above '\item[Special label.]'; *note +\item::. + + The mandatory second argument SPACING has a list of commands. This +list can be empty. A command that can go in here is +'\usecounter{COUNTERNAME}' (*note \usecounter::). Use this to tell +LaTeX to number the items using the given counter. The counter will be +reset to zero each time LaTeX enters the environment, and the counter is +incremented by one each time LaTeX encounters an '\item' that does not +have an optional argument. + + Another command that can go in SPACING is '\makelabel', which +constructs the label box. By default it puts the contents flush right. +Its only argument is the label, which it typesets in LR mode (*note +Modes::). One example of changing its definition is that to the above +'named' example, before the definition of the environment add +'\newcommand{\namedmakelabel}[1]{\textsc{#1}}', and between the +'\setlength' command and the parenthesis that closes the SPACING +argument also add '\let\makelabel\namedmakelabel'. Then the labels will +be typeset in small caps. Similarly, changing the second code line to +'\let\makelabel\fbox' puts the labels inside a framed box. Another +example of the '\makelabel' command is below, in the definition of the +'redlabel' environment. + + Also often in SPACING are commands to redefine the spacing for the +list. Below are the spacing parameters with their default values. +(Default values for derived environments such as 'itemize' can be +different than the values shown here.) See also the figure that follows +the list. Each is a length (*note Lengths::). The vertical spaces are +normally rubber lengths, with 'plus' and 'minus' components, to give TeX +flexibility in setting the page. Change each with a command such as +'\setlength{itemsep}{2pt plus1pt minus1pt}'. For some effects these +lengths should be zero or negative. + +'\itemindent' + Extra horizontal space indentation, beyond 'leftmargin', of the + first line each item. Its default value is '0pt'. + +'\itemsep' + Vertical space between items, beyond the '\parsep'. The defaults + for the first three levels in LaTeX's 'article', 'book', and + 'report' classes at 10 point size are: '4pt plus2pt minus1pt', + '\parsep' (that is, '2pt plus1pt minus1pt'), and '\topsep' (that + is, '2pt plus1pt minus1pt'). The defaults at 11 point are: '4.5pt + plus2pt minus1pt', '\parsep' (that is, '2pt plus1pt minus1pt'), and + '\topsep' (that is, '2pt plus1pt minus1pt'). The defaults at 12 + point are: '5pt plus2.5pt minus1pt', '\parsep' (that is, '2.5pt + plus1pt minus1pt'), and '\topsep' (that is, '2.5pt plus1pt + minus1pt'). + +'\labelsep' + Horizontal space between the label and text of an item. The + default for LaTeX's 'article', 'book', and 'report' classes is + '0.5em'. + +'\labelwidth' + Horizontal width. The box containing the label is nominally this + wide. If '\makelabel' returns text that is wider than this then + the first line of the item will be indented to make room for this + extra material. If '\makelabel' returns text of width less than or + equal to '\labelwidth' then LaTeX's default is that the label is + typeset flush right in a box of this width. + + The left edge of the label box is + '\leftmargin'+'\itemindent'-'\labelsep'-'\labelwidth' from the left + margin of the enclosing environment. + + The default for LaTeX's 'article', 'book', and 'report' classes at + the top level is '\leftmargini'-'\labelsep', (which is '2em' in one + column mode and '1.5em' in two column mode). At the second level + it is '\leftmarginii'-'\labelsep', and at the third level it is + '\leftmarginiii'-'\labelsep'. These definitions make the label's + left edge coincide with the left margin of the enclosing + environment. + +'\leftmargin' + Horizontal space between the left margin of the enclosing + environment (or the left margin of the page if this is a top-level + list), and the left margin of this list. It must be non-negative. + + In the standard LaTeX document classes, a top-level list has this + set to the value of '\leftmargini', while a list that is nested + inside a top-level list has this margin set to '\leftmarginii'. + More deeply nested lists get the values of '\leftmarginiii' through + '\leftmarginvi'. (Nesting greater than level five generates the + error message 'Too deeply nested'.) + + The defaults for the first three levels in LaTeX's 'article', + 'book', and 'report' classes are: '\leftmargini' is '2.5em' (in two + column mode, '2em'), '\leftmarginii' is '2.2em', and + '\leftmarginiii' is '1.87em'. + +'\listparindent' + Horizontal space of additional line indentation, beyond + '\leftmargin', for second and subsequent paragraphs within a list + item. A negative value makes this an "outdent". Its default value + is '0pt'. + +'\parsep' + Vertical space between paragraphs within an item. The defaults for + the first three levels in LaTeX's 'article', 'book', and 'report' + classes at 10 point size are: '4pt plus2pt minus1pt', '2pt plus1pt + minus1pt', and '0pt'. The defaults at 11 point size are: '4.5pt + plus2pt minus1pt', '2pt plus1pt minus1pt', and '0pt'. The defaults + at 12 point size are: '5pt plus2.5pt minus1pt', '2.5pt plus1pt + minus1pt', and '0pt'. + +'\partopsep' + Vertical space added, beyond '\topsep'+'\parskip', to the top and + bottom of the entire environment if the list instance is preceded + by a blank line. (A blank line in the LaTeX source before the list + changes spacing at both the top and bottom of the list; whether the + line following the list is blank does not matter.) + + The defaults for the first three levels in LaTeX's 'article', + 'book', and 'report' classes at 10 point size are: '2pt plus1 + minus1pt', '2pt plus1pt minus1pt', and '1pt plus0pt minus1pt'. The + defaults at 11 point are: '3pt plus1pt minus1pt', '3pt plus1pt + minus1pt', and '1pt plus0pt minus1pt'). The defaults at 12 point + are: '3pt plus2pt minus3pt', '3pt plus2pt minus2pt', and '1pt + plus0pt minus1pt'. + +'\rightmargin' + Horizontal space between the right margin of the list and the right + margin of the enclosing environment. Its default value is '0pt'. + It must be non-negative. + +'\topsep' + Vertical space added to both the top and bottom of the list, in + addition to '\parskip' (*note \parindent & \parskip::). The + defaults for the first three levels in LaTeX's 'article', 'book', + and 'report' classes at 10 point size are: '8pt plus2pt minus4pt', + '4pt plus2pt minus1pt', and '2pt plus1pt minus1pt'. The defaults + at 11 point are: '9pt plus3pt minus5pt', '4.5pt plus2pt minus1pt', + and '2pt plus1pt minus1pt'. The defaults at 12 point are: '10pt + plus4pt minus6pt', '5pt plus2.5pt minus1pt', and '2.5pt plus1pt + minus1pt'. + + This shows the horizontal and vertical distances. + +[image src="latex2e-figures/list.png" text="....................................................................... +..............Surrounding environment, preceding material.............. - + | v0 + |---h0--|-h1-| | + ..Label.. .....First item..................................... - + ............................................................ + ............................................................ - + | v1 + ........First item, second paragraph .................. - + |-h2-| | v2 + | + ..Label.. .....Second item.................................... - + ............................................................ + ............................................................ - +|-h3--|-h4-| |-h5-| |v3 + | +..............Surrounding environment, following material.............. - +......................................................................."] + + + The lengths shown are listed below. The key relationship is that the +right edge of the bracket for H1 equals the right edge of the bracket +for H4, so that the left edge of the label box is at H3+H4-(H0+H1). + +V0 + '\topsep' + '\parskip' if the list environment does not start a new + paragraph, and '\topsep'+'\parskip'+'\partopsep' if it does + +V1 + '\parsep' + +V2 + '\itemsep'+'\parsep' + +V3 + Same as V0. (This space is affected by whether a blank line + appears in the source above the environment; whether a blank line + appears in the source below the environment does not matter.) + +H0 + '\labelwidth' + +H1 + '\labelsep' + +H2 + '\listparindent' + +H3 + '\leftmargin' + +H4 + '\itemindent' + +H5 + '\rightmargin' + + The list's left and right margins, shown above as H3 and H5, are with +respect to the ones provided by the surrounding environment, or with +respect to the page margins for a top-level list. The line width used +for typesetting the list items is '\linewidth' (*note Page layout +parameters::). For instance, set the list's left margin to be one +quarter of the distance between the left and right margins of the +enclosing environment with '\setlength{\leftmargin}{0.25\linewidth}'. + + Page breaking in a list structure is controlled by the three +parameters below. For each, the LaTeX default is '-\@lowpenalty', that +is, '-51'. Because it is negative, it somewhat encourages a page break +at each spot. Change it with, e.g., '\@beginparpenalty=9999'; a value +of 10000 prohibits a page break. + +'\@beginparpenalty' + The page breaking penalty for breaking before the list (default + '-51'). + +'\@itempenalty' + The page breaking penalty for breaking before a list item (default + '-51'). + +'\@endparpenalty' + The page breaking penalty for breaking after a list (default + '-51'). + + The package 'enumitem' is useful for customizing lists. + + This example has the labels in red. They are numbered, and the left +edge of the label lines up with the left edge of the item text. *Note +\usecounter::. + + \usepackage{color} + \newcounter{cnt} + \newcommand{\makeredlabel}[1]{\textcolor{red}{#1.}} + \newenvironment{redlabel} + {\begin{list} + {\arabic{cnt}} + {\usecounter{cnt} + \setlength{\labelwidth}{0em} + \setlength{\labelsep}{0.5em} + \setlength{\leftmargin}{1.5em} + \setlength{\itemindent}{0.5em} % equals \labelwidth+\labelsep + \let\makelabel=\makeredlabel + } + } + {\end{list}} + +* Menu: + +* \item:: An entry in a list. +* trivlist:: A restricted form of 'list'. + + +File: latex2e.info, Node: \item, Next: trivlist, Up: list + +8.16.1 '\item': An entry in a list +---------------------------------- + +Synopsis: + + \item text of item + +or + + \item[OPTIONAL-LABEL] text of item + + An entry in a list. The entries are prefixed by a label, whose +default depends on the list type. + + Because the optional label is surrounded by square brackets '[...]', +if you have an item whose text starts with [, you have to hide the +bracket inside curly braces, as in: '\item {[} is an open square +bracket'; otherwise, LaTeX will think it marks the start of an optional +label. + + Similarly, if the item does have the optional label and you need a +close square bracket inside that label, you must hide it in the same +way: '\item[Close square bracket, {]}]'. *Note LaTeX command syntax::. + + In this example the enumerate list has two items that use the default +label and one that uses the optional label. + + \begin{enumerate} + \item Moe + \item[sometimes] Shemp + \item Larry + \end{enumerate} + + The first item is labelled '1.', the second item is labelled +'sometimes', and the third item is labelled '2.'. Because of the +optional label in the second item, the third item is not labelled '3.'. + + +File: latex2e.info, Node: trivlist, Prev: \item, Up: list + +8.16.2 'trivlist': A restricted form of 'list' +---------------------------------------------- + +Synopsis: + + \begin{trivlist} + ... + \end{trivlist} + + A restricted version of the list environment, in which margins are +not indented and an '\item' without an optional argument produces no +text. It is most often used in macros, to define an environment where +the '\item' command is part of the environment's definition. For +instance, the 'center' environment is defined essentially like this: + + \newenvironment{center} + {\begin{trivlist}\centering\item\relax} + {\end{trivlist}} + + Using 'trivlist' in this way allows the macro to inherit some common +code: combining vertical space of two adjacent environments; detecting +whether the text following the environment should be considered a new +paragraph or a continuation of the previous one; adjusting the left and +right margins for possible nested list environments. + + Specifically, 'trivlist' uses the current values of the list +parameters (*note list::), except that '\parsep' is set to the value of +'\parskip', and '\leftmargin', '\labelwidth', and '\itemindent' are set +to zero. + + This example outputs the items as two paragraphs, except that (by +default) they have no paragraph indent and are vertically separated. + + \begin{trivlist} + \item The \textit{Surprise} is not old; no one would call her old. + \item She has a bluff bow, lovely lines. + \end{trivlist} + + +File: latex2e.info, Node: math, Next: minipage, Prev: list, Up: Environments + +8.17 'math' +=========== + +Synopsis: + + \begin{math} + MATH + \end{math} + + The 'math' environment inserts given MATH material within the running +text. '\(...\)' and '$...$' are synonyms. *Note Math formulas::. + + +File: latex2e.info, Node: minipage, Next: picture, Prev: math, Up: Environments + +8.18 'minipage' +=============== + +Synopses: + + \begin{minipage}{WIDTH} + CONTENTS + \end{minipage} + +or + + \begin{minipage}[POSITION][HEIGHT][INNER-POS]{WIDTH} + CONTENTS + \end{minipage} + + Put CONTENTS into a box that is WIDTH wide. This is like a small +version of a page; it can contain its own footnotes, itemized lists, +etc. (There are some restrictions, including that it cannot have +floats.) This box will not be broken across pages. So 'minipage' is +similar to '\parbox' (*note \parbox::) but here you can have paragraphs. + + This example will be 3 inches wide, and has two paragraphs. + + \begin{minipage}{3in} + Stephen Kleene was a founder of the Theory of Computation. + + He was a student of Church, wrote three influential texts, + was President of the Association for Symbolic Logic, + and won the National Medal of Science. + \end{minipage} + +See below for a discussion of the paragraph indent inside a 'minipage'. + + The required argument WIDTH is a rigid length (*note Lengths::). It +gives the width of the box into which CONTENTS are typeset. + + There are three optional arguments, POSITION, HEIGHT, and INNER-POS. +You need not include all three. For example, get the default POSITION +and set the HEIGHT with '\begin{minipage}[c][2.54cm]{\columnwidth} +CONTENTS \end{minipage}'. (Get the natural height with an empty +argument, '[]'.) + + The optional argument POSITION governs how the 'minipage' vertically +aligns with the surrounding material. + +'c' + (synonym 'm') Default. Positions the 'minipage' so its vertical + center lines up with the center of the adjacent text line. + +'t' + Align the baseline of the top line in the 'minipage' with the + baseline of the surrounding text (plain TeX's '\vtop'). + +'b' + Align the baseline of the bottom line in the 'minipage' with the + baseline of the surrounding text (plain TeX's '\vbox'). + + To see the effects of these, contrast running this + + ---\begin{minipage}[c]{0.25in} + first\\ second\\ third + \end{minipage} + +with the results of changing 'c' to 'b' or 't'. + + The optional argument HEIGHT is a rigid length (*note Lengths::). It +sets the height of the 'minipage'. You can enter any value larger than, +or equal to, or smaller than the 'minipage''s natural height and LaTeX +will not give an error or warning. You can also set it to a height of +zero or a negative value. + + The final optional argument INNER-POS controls the placement of +CONTENTS inside the box. These are the possible values are (the default +is the value of POSITION). + +'t' + Place CONTENTS at the top of the box. + +'c' + Place it in the vertical center. + +'b' + Place it at the box bottom. + +'s' + Stretch CONTENTS out vertically; it must contain vertically + stretchable space. + + The INNER-POS argument makes sense when the HEIGHT option is set to a +value larger than the 'minipage''s natural height. To see the effect of +the options, run this example with the various choices in place of 'b'. + + Text before + \begin{center} + ---\begin{minipage}[c][3in][b]{0.25\textwidth} + first\\ second\\ third + \end{minipage} + \end{center} + Text after + + By default paragraphs are not indented in a 'minipage'. Change that +with a command such as '\setlength{\parindent}{1pc}' at the start of +CONTENTS. + + Footnotes in a 'minipage' environment are handled in a way that is +particularly useful for putting footnotes in figures or tables. A +'\footnote' or '\footnotetext' command puts the footnote at the bottom +of the minipage instead of at the bottom of the page, and it uses the +'\mpfootnote' counter instead of the ordinary 'footnote' counter (*note +Counters::). + + This puts the footnote at the bottom of the table, not the bottom of +the page. + + \begin{center} % center the minipage on the line + \begin{minipage}{2.5in} + \begin{center} % center the table inside the minipage + \begin{tabular}{ll} + \textsc{Monarch} &\textsc{Reign} \\ \hline + Elizabeth II &63 years\footnote{to date} \\ + Victoria &63 years \\ + George III &59 years + \end{tabular} + \end{center} + \end{minipage} + \end{center} + + If you nest minipages then there is an oddness when using footnotes. +Footnotes appear at the bottom of the text ended by the next +'\end{minipage}' which may not be their logical place. + + This puts a table containing data side by side with a map graphic. +They are vertically centered. + + % siunitx to have the S column specifier, + % which aligns numbers on their decimal point. + \usepackage{siunitx} + \newcommand*{\vcenteredhbox}[1]{\begin{tabular}{@{}c@{}}#1\end{tabular}} + ... + \begin{center} + \vcenteredhbox{\includegraphics[width=0.3\textwidth]{nyc.png}} + \hspace{0.1\textwidth} + \begin{minipage}{0.5\textwidth} + \begin{tabular}{r|S} + % \multicolumn to remove vertical bar between column headers + \multicolumn{1}{r}{Borough} & + % braces to prevent siunitx from misinterpreting the + % period as a decimal separator + {Pop. (million)} \\ \hline + The Bronx &1.5 \\ + Brooklyn &2.6 \\ + Manhattan &1.6 \\ + Queens &2.3 \\ + Staten Island &0.5 + \end{tabular} + \end{minipage} + \end{center} + + +File: latex2e.info, Node: picture, Next: quotation & quote, Prev: minipage, Up: Environments + +8.19 'picture' +============== + +Synopses: + \begin{picture}(WIDTH,HEIGHT) + PICTURE COMMAND + \end{picture} + +or + + \begin{picture}(WIDTH,HEIGHT)(XOFFSET,YOFFSET) + PICTURE COMMAND + \end{picture} + +Where there may be any number of PICTURE COMMAND's. + + An environment to create simple pictures containing lines, arrows, +boxes, circles, and text. This environment is not obsolete, but new +documents typically use much more powerful graphics creation systems, +such as TikZ, PSTricks, MetaPost, or Asymptote. None of these are +covered in this document; see CTAN. + + To start, here's an example showing the parallelogram law for adding +vectors. + + \setlength{\unitlength}{1cm} + \begin{picture}(6,6) % picture box will be 6cm wide by 6cm tall + \put(0,0){\vector(2,1){4}} % for every 2 over this vector goes 1 up + \put(2,1){\makebox(0,0)[l]{\ first leg}} + \put(4,2){\vector(1,2){2}} + \put(5,4){\makebox(0,0)[l]{\ second leg}} + \put(0,0){\vector(1,1){6}} + \put(3,3){\makebox(0,0)[r]{sum\ }} + \end{picture} + + The 'picture' environment has one required argument, a pair of +positive real numbers (WIDTH,HEIGHT). Multiply these by the value +'\unitlength' to get the nominal size of the output, i.e. the space that +LaTeX reserves on the output page. This nominal size need not be how +large the picture really is; LaTeX will draw things from the picture +outside the picture's box. + + This environment also has an optional argument (XOFFSET,YOFFSET). It +is used to shift the origin. Unlike most optional arguments, this one +is not contained in square brackets. As with the required argument, it +consists of a pair of two real numbers, but these may also be negative +or null. Multiply these by '\unitlength' to get the coordinates of the +point at the lower-left corner of the picture. + + For example, if '\unitlength' has been set to '1mm', the command + + \begin{picture}(100,200)(10,20) + +produces a box of width 100 millimeters and height 200 millimeters. The +picture's origin is the point (10mm,20mm) and so the lower-left corner +is there, and the upper-right corner is at (110mm,220mm). When you +first draw a picture you typically omit the optional argument, leaving +the origin at the lower-left corner. If you then want to modify your +picture by shifting everything, you can just add the appropriate +optional argument. + + Each PICTURE COMMAND tells LaTeX where to put something by providing +its position. A "position" is a pair such as '(2.4,-5)' giving the x- +and y-coordinates. A "coordinate" is a not a length, it is a real +number (it may have a decimal point or a minus sign). It specifies a +length in multiples of the unit length '\unitlength', so if +'\unitlength' has been set to '1cm', then the coordinate '2.54' +specifies a length of 2.54 centimeters. + + LaTeX's default for '\unitlength' is '1pt'. It is a rigid length +(*note Lengths::). Change it with the '\setlength' command (*note +\setlength::). Make this change only outside of a 'picture' +environment. + + The 'picture' environment supports using standard arithmetic +expressions as well as numbers. + + Coordinates are given with respect to an origin, which is by default +at the lower-left corner of the picture. Note that when a position +appears as an argument, as with '\put(1,2){...}', it is not enclosed in +braces since the parentheses serve to delimit the argument. Also, +unlike in some computer graphics systems, larger y-coordinates are +further up the page, for example, y = 1 is _above_ y = 0. + + There are four ways to put things in a picture: '\put', '\multiput', +'\qbezier', and '\graphpaper'. The most often used is '\put'. This + + \put(11.3,-0.3){...} + +places the object with its reference point at coordinates (11.3,-0.3). +The reference points for various objects will be described below. The +'\put' command creates an "LR box" (*note Modes::). Anything that can +go in an '\mbox' (*note \mbox & \makebox::) can go in the text argument +of the '\put' command. The reference point will be the lower left +corner of the box. In this picture + + \setlength{\unitlength}{1cm} + ...\begin{picture}(1,1) + \put(0,0){\line(1,0){1}} + \put(0,0){\line(1,1){1}} + \end{picture} + +the three dots are just slightly left of the point of the angle formed +by the two lines. (Also, '\line(1,1){1}' does not call for a line of +length one; rather the line has a change in the x coordinate of 1.) + + The '\multiput', 'qbezier', and 'graphpaper' commands are described +below. + + You can also use this environment to place arbitrary material at an +exact location. For example: + + \usepackage{color,graphicx} % in preamble + ... + \begin{center} + \setlength{\unitlength}{\textwidth} + \begin{picture}(1,1) % leave space, \textwidth wide and tall + \put(0,0){\includegraphics[width=\textwidth]{desertedisland.jpg}} + \put(0.25,0.35){\textcolor{red}{X Treasure here}} + \end{picture} + \end{center} + +The red X will be precisely a quarter of the '\textwidth' from the left +margin, and '0.35\textwidth' up from the bottom of the picture. Another +example of this usage is to put similar code in the page header to get +repeat material on each of a document's pages. + +* Menu: + +* \put:: Place an object at a specified place. +* \multiput:: Draw multiple instances of an object. +* \qbezier:: Draw a quadratic Bézier curve. +* \graphpaper:: Draw graph paper. +* \line:: Draw a straight line. +* \linethickness:: Set thickness of horizontal and vertical lines. +* \thinlines:: The default line thickness. +* \thicklines:: A heavier line thickness. +* \circle:: Draw a circle. +* \oval:: Draw an oval. +* \shortstack:: Make a stack of objects. +* \vector:: Draw a line with an arrow. +* \makebox (picture):: Draw a box of the specified size. +* \framebox (picture):: Draw a box with a frame around it. +* \frame:: Draw a frame around an object. +* \dashbox:: Draw a dashed box. + + +File: latex2e.info, Node: \put, Next: \multiput, Up: picture + +8.19.1 '\put' +------------- + +Synopsis: + + \put(XCOORD,YCOORD){CONTENT} + + Place CONTENT at the coordinate (XCOORD,YCOORD). See the discussion +of coordinates and '\unitlength' in *note picture::. The CONTENT is +processed in LR mode (*note Modes::) so it cannot contain line breaks. + + This includes the text into the 'picture'. + + \put(4.5,2.5){Apply the \textit{unpoke} move} + + The reference point, the location (4.5,2.5), is the lower left of the +text, at the bottom left of the 'A'. + + +File: latex2e.info, Node: \multiput, Next: \qbezier, Prev: \put, Up: picture + +8.19.2 '\multiput' +------------------ + +Synopsis: + + \multiput(X,Y)(DELTA_X,DELTA_Y){NUM-COPIES}{OBJ} + + Copy OBJ a total of NUM-COPIES times, with an increment of +DELTA_X,DELTA_Y. The OBJ first appears at position (x,y), then at +(x+\delta_x,y+\delta_y), and so on. + + This draws a simple grid with every fifth line in bold (see also +*note \graphpaper::). + + \begin{picture}(10,10) + \linethickness{0.05mm} + \multiput(0,0)(1,0){10}{\line(0,1){10}} + \multiput(0,0)(0,1){10}{\line(1,0){10}} + \linethickness{0.5mm} + \multiput(0,0)(5,0){3}{\line(0,1){10}} + \multiput(0,0)(0,5){3}{\line(1,0){10}} + \end{picture} + + +File: latex2e.info, Node: \qbezier, Next: \graphpaper, Prev: \multiput, Up: picture + +8.19.3 '\qbezier' +----------------- + +Synopsis: + + \qbezier(X1,Y1)(X2,Y2)(X3,Y3) + \qbezier[NUM](X1,Y1)(X2,Y2)(X3,Y3) + + Draw a quadratic Bezier curve whose control points are given by the +three required arguments '(X1,Y1)', '(X2,Y2)', and '(X3,Y3)'. That is, +the curve runs from (X1,Y1) to (X3,Y3), is quadratic, and is such that +the tangent line at (X1,Y1) passes through (X2,Y2), as does the tangent +line at (X3,Y3). + + This draws a curve from the coordinate (1,1) to (1,0). + + \qbezier(1,1)(1.25,0.75)(1,0) + +The curve's tangent line at (1,1) contains (1.25,0.75), as does the +curve's tangent line at (1,0). + + The optional argument NUM gives the number of calculated intermediate +points. The default is to draw a smooth curve whose maximum number of +points is '\qbeziermax' (change this value with '\renewcommand'). + + This draws a rectangle with a wavy top, using '\qbezier' for that +curve. + + \begin{picture}(8,4) + \put(0,0){\vector(1,0){8}} % x axis + \put(0,0){\vector(0,1){4}} % y axis + \put(2,0){\line(0,1){3}} % left side + \put(4,0){\line(0,1){3.5}} % right side + \qbezier(2,3)(2.5,2.9)(3,3.25) + \qbezier(3,3.25)(3.5,3.6)(4,3.5) + \thicklines % below here, lines are twice as thick + \put(2,3){\line(4,1){2}} + \put(4.5,2.5){\framebox{Trapezoidal Rule}} + \end{picture} + + +File: latex2e.info, Node: \graphpaper, Next: \line, Prev: \qbezier, Up: picture + +8.19.4 '\graphpaper' +-------------------- + +Synopsis: + + \graphpaper(X_INIT,Y_INIT)(X_DIMEN,Y_DIMEN) + \graphpaper[SPACING](X_INIT,Y_INIT)(X_DIMEN,Y_DIMEN) + + Draw a coordinate grid. Requires the 'graphpap' package. The grid's +origin is '(X_INIT,Y_INIT)'. Grid lines come every SPACING units (the +default is 10). The grid extends X_DIMEN units to the right and Y_DIMEN +units up. All arguments must be positive integers. + + This make a grid with seven vertical lines and eleven horizontal +lines. + + \usepackage{graphpap} % in preamble + ... + \begin{picture}(6,20) % in document body + \graphpaper[2](0,0)(12,20) + \end{picture} + +The lines are numbered every ten units. + + +File: latex2e.info, Node: \line, Next: \linethickness, Prev: \graphpaper, Up: picture + +8.19.5 '\line' +-------------- + +Synopsis: + + \line(X_RUN,Y_RISE){TRAVEL} + + Draw a line. It slopes such that it vertically rises Y_RISE for +every horizontal X_RUN. The TRAVEL is the total horizontal change--it +is not the length of the vector, it is the change in x. In the special +case of vertical lines, where (X_RUN,Y_RISE)=(0,1), the TRAVEL gives the +change in y. + + This draws a line starting at coordinates (1,3). + + \put(1,3){\line(2,5){4}} + +For every over 2, this line will go up 5. Because TRAVEL specifies that +this goes over 4, it must go up 10. Thus its endpoint is +(1,3)+(4,10)=(5,13). In particular, note that TRAVEL=4 is not the +length of the line, it is the change in x. + + The arguments X_RUN and Y_RISE are integers that can be positive, +negative, or zero. (If both are 0 then LaTeX treats the second as 1.) +With '\put(X_INIT,Y_INIT){\line(X_RUN,Y_RISE){TRAVEL}}', if X_RUN is +negative then the line's ending point has a first coordinate that is +less than X_INIT. If Y_RISE is negative then the line's ending point +has a second coordinate that is less than Y_INIT. + + If TRAVEL is negative then you get 'LaTeX Error: Bad \line or \vector +argument.' + + Standard LaTeX can only draw lines with a limited range of slopes +because these lines are made by putting together line segments from +pre-made fonts. The two numbers X_RUN and Y_RISE must have integer +values from -6 through 6. Also, they must be relatively prime, so that +(X_RUN,Y_RISE) can be (2,1) but not (4,2) (if you choose the latter then +instead of lines you get sequences of arrowheads; the solution is to +switch to the former). To get lines of arbitrary slope and plenty of +other shapes in a system like 'picture', see the package 'pict2e' +(). Another solution is to use a +full-featured graphics system such as TikZ, PSTricks, MetaPost, or +Asymptote. + + +File: latex2e.info, Node: \linethickness, Next: \thinlines, Prev: \line, Up: picture + +8.19.6 '\linethickness' +----------------------- + +Synopsis: + + \linethickness{DIM} + + Declares the thickness of subsequent horizontal and vertical lines in +a picture to be DIM, which must be a positive length (*note Lengths::). +It differs from '\thinlines' and '\thicklines' in that it does not +affect the thickness of slanted lines, circles, or ovals (*note +\oval::). + + +File: latex2e.info, Node: \thinlines, Next: \thicklines, Prev: \linethickness, Up: picture + +8.19.7 '\thinlines' +------------------- + +Declaration to set the thickness of subsequent lines, circles, and ovals +in a picture environment to be 0.4pt. This is the default thickness, so +this command is unnecessary unless the thickness has been changed with +either *note \linethickness:: or *note \thicklines::. + + +File: latex2e.info, Node: \thicklines, Next: \circle, Prev: \thinlines, Up: picture + +8.19.8 '\thicklines' +-------------------- + +Declaration to set the thickness of subsequent lines, circles, and ovals +in a picture environment to be 0.8pt. See also *note \linethickness:: +and *note \thinlines::. This command is illustrated in the Trapezoidal +Rule example of *note \qbezier::. + + +File: latex2e.info, Node: \circle, Next: \oval, Prev: \thicklines, Up: picture + +8.19.9 '\circle' +---------------- + +Synopsis: + + \circle{DIAMETER} + \circle*{DIAMETER} + + Produces a circle with a diameter as close as possible to the +specified one. The '*' form produces a filled-in circle. + + This draws a circle of radius 6, centered at '(5,7)'. + + \put(5,7){\circle{6}} + + The available radii for '\circle' are, in points, the even numbers +from 2 to 20, inclusive. For '\circle*' they are all the integers from +1 to 15. + + +File: latex2e.info, Node: \oval, Next: \shortstack, Prev: \circle, Up: picture + +8.19.10 '\oval' +--------------- + +Synopsis: + + \oval(WIDTH,HEIGHT) + \oval(WIDTH,HEIGHT)[PORTION] + + Produce a rectangle with rounded corners, hereinafter referred to as +an "oval". The optional argument PORTION allows you to produce only +half or a quarter of the oval. For half an oval take PORTION to be one +of these. + +'t' + top half +'b' + bottom half +'r' + right half +'l' + left half + + Produce only one quarter of the oval by setting PORTION to 'tr', +'br', 'bl', or 'tl'. + + This draws the top half of an oval that is 3 wide and 7 tall. + + \put(5,7){\oval(3,7)[t]} + +The (5,7) is the center of the entire oval, not just the center of the +top half. + + These shapes are not ellipses. They are rectangles whose corners are +made with quarter circles. These circles have a maximum radius of 20pt +(*note \circle:: for the sizes). Thus large ovals are just frames with +a small amount of corner rounding. + + +File: latex2e.info, Node: \shortstack, Next: \vector, Prev: \oval, Up: picture + +8.19.11 '\shortstack' +--------------------- + +Synopsis: + + \shortstack[POSITION]{LINE 1 \\ ... } + + Produce a vertical stack of objects. + + This labels the y axis by writing the word 'y' above the word 'axis'. + + \setlength{\unitlength}{1cm} + \begin{picture}(5,2.5)(-0.75,0) + \put(0,0){\vector(1,0){4}} % x axis + \put(0,0){\vector(0,1){2}} % y + \put(-0.2,2){\makebox(0,0)[r]{\shortstack[r]{$y$\\ axis}}} + \end{picture} + +For a short stack, the reference point is the lower left of the stack. +In the above example the '\makebox' (*note \mbox & \makebox::) puts the +stack flush right in a zero width box so in total the short stack sits +slightly to the left of the y axis. + + The valid positions are: + +'r' + Make objects flush right +'l' + Make objects flush left +'c' + Center objects (default) + + Separate objects into lines with '\\'. These stacks are short in +that, unlike in a 'tabular' or 'array' environment, here the rows are +not spaced out to be of even baseline skips. Thus, in +'\shortstack{X\\o\\o\\X}' the first and last rows are taller than the +middle two, and therefore the baseline skip between the two middle rows +is smaller than that between the third and last row. You can adjust row +heights and depths either by putting in the usual interline spacing with +'\shortstack{X\\ \strut o\\o\\X}' (*note \strut::), or explicitly, via +an zero-width box '\shortstack{X \\ \rule{0pt}{12pt} o\\o\\X}' or by +using '\\''s optional argument '\shortstack{X\\[2pt] o\\o\\X}'. + + The '\shortstack' command is also available outside the 'picture' +environment. + + +File: latex2e.info, Node: \vector, Next: \makebox (picture), Prev: \shortstack, Up: picture + +8.19.12 '\vector' +----------------- + +Synopsis: + + \vector(X_RUN,Y_RISE){TRAVEL} + + Draw a line ending in an arrow. The slope of that line is: it +vertically rises Y_RISE for every horizontal X_RUN. The TRAVEL is the +total horizontal change--it is not the length of the vector, it is the +change in x. In the special case of vertical vectors, if +(X_RUN,Y_RISE)=(0,1), then TRAVEL gives the change in y. + + For an example see *note picture::. + + For elaboration on X_RUN and Y_RISE see *note \line::. As there, the +values of X_RUN and Y_RISE are limited. For '\vector' you must chooses +integers between -4 and 4, inclusive. Also, the two you choose must be +relatively prime. Thus, '\vector(2,1){4}' is acceptable but +'\vector(4,2){4}' is not (if you use the latter then you get a sequence +of arrowheads). + + +File: latex2e.info, Node: \makebox (picture), Next: \framebox (picture), Prev: \vector, Up: picture + +8.19.13 '\makebox' (picture) +---------------------------- + +Synopsis: + + \makebox(REC-WIDTH,REC-HEIGHT){TEXT} + \makebox(REC-WIDTH,REC-HEIGHT)[POSITION]{TEXT} + + Make a box to hold TEXT. This command fits with the 'picture' +environment, although you can use it outside of there, because REC-WIDTH +and REC-HEIGHT are numbers specifying distances in terms of the +'\unitlength' (*note picture::). This command is similar to the normal +'\makebox' command (*note \mbox & \makebox::) except here that you must +specify the width and height. This command is fragile (*note +\protect::). + + This makes a box of length 3.5 times '\unitlength' and height 4 times +'\unitlength'. + + \put(1,2){\makebox(3.5,4){...}} + + The optional argument 'POSITION' specifies where in the box the TEXT +appears. The default is to center it, both horizontally and vertically. +To place it somewhere else, use a string with one or two of these +letters. + +'t' + Puts TEXT the top of the box. + +'b' + Put TEXT at the bottom. + +'l' + Put TEXT on the left. + +'r' + Put TEXT on the right. + + +File: latex2e.info, Node: \framebox (picture), Next: \frame, Prev: \makebox (picture), Up: picture + +8.19.14 '\framebox' (picture) +----------------------------- + +Synopsis: + + \framebox(REC-WIDTH,REC-HEIGHT){TEXT} + \framebox(REC-WIDTH,REC-HEIGHT)[POSITION]{TEXT} + + This is the same as *note \makebox (picture):: except that it puts a +frame around the outside of the box that it creates. The reference +point is the bottom left corner of the frame. This command fits with +the 'picture' environment, although you can use it outside of there, +because lengths are numbers specifying the distance in terms of the +'\unitlength' (*note picture::). This command is fragile (*note +\protect::). + + This example creates a frame 2.5 inches by 3 inches and puts the text +in the center. + + \setlength{\unitlength}{1in} + \framebox(2.5,3){test text} + + The required arguments are that the rectangle has overall width +RECT-WIDTH units and height RECT-HEIGHT units. + + The optional argument POSITION specifies the position of TEXT; see +*note \makebox (picture):: for the values that it can take. + + The rule has thickness '\fboxrule' and there is a blank space +'\fboxsep' between the frame and the contents of the box. + + For this command, you must specify the WIDTH and HEIGHT. If you want +to just put a frame around some contents whose dimension is determined +in some other way then either use '\fbox' (*note \fbox & \framebox::) or +'\frame' (*note \frame::). + + +File: latex2e.info, Node: \frame, Next: \dashbox, Prev: \framebox (picture), Up: picture + +8.19.15 '\frame' +---------------- + +Synopsis: + + \frame{CONTENTS} + + Puts a rectangular frame around CONTENTS. The reference point is the +bottom left corner of the frame. In contrast to '\framebox' (*note +\framebox (picture)::), this command puts no extra space is put between +the frame and the object. It is fragile (*note \protect::). + + +File: latex2e.info, Node: \dashbox, Prev: \frame, Up: picture + +8.19.16 '\dashbox' +------------------ + +Synopsis: + + \dashbox{DASH-LEN}(RECT-WIDTH,RECT-HEIGHT){TEXT} + \dashbox{DASH-LEN}(RECT-WIDTH,RECT-HEIGHT)[POSITION]{TEXT} + + Create a dashed rectangle around TEXT. This command fits with the +'picture' environment, although you can use it outside of there, because +lengths are numbers specifying the distance in terms of the +'\unitlength' (*note picture::). + + The required arguments are: dashes are DASH-LEN units long, with the +same length gap, and the rectangle has overall width RECT-WIDTH units +and height RECT-HEIGHT units. + + The optional argument POSITION specifies the position of TEXT; see +*note \makebox (picture):: for the values that it can take. + + This shows that you can use non-integer value for DASH-LEN. + + \put(0,0){\dashbox{0.1}(5,0.5){My hovercraft is full of eels.}} + +Each dash will be '0.1\unitlength' long, the box's width is +'5\unitlength' and its height is '0.5\unitlength'. + + As in that example, a dashed box looks best when RECT-WIDTH and +RECT-HEIGHT are multiples of the DASH-LEN. + + +File: latex2e.info, Node: quotation & quote, Next: tabbing, Prev: picture, Up: Environments + +8.20 'quotation' & 'quote' +========================== + +Synopsis: + + \begin{quotation} + TEXT + \end{quotation} + +or + + \begin{quote} + TEXT + \end{quote} + + Include a quotation. Both environments indent margins on both sides +by '\leftmargin' and the text is right-justified. + + They differ in how they treat paragraphs. In the 'quotation' +environment, paragraphs are indented by 1.5em and the space between +paragraphs is small, '0pt plus 1pt'. In the 'quote' environment, +paragraphs are not indented and there is vertical space between +paragraphs (it is the rubber length '\parsep'). + + \begin{quotation} \small\it + Four score and seven years ago + ... shall not perish from the earth. + \hspace{1em plus 1fill}---Abraham Lincoln + \end{quotation} + + +File: latex2e.info, Node: tabbing, Next: table, Prev: quotation & quote, Up: Environments + +8.21 'tabbing' +============== + +Synopsis: + + \begin{tabbing} + ROW1COL1 \= ROW1COL2 ... \\ + ROW2COL1 \> ROW2COL2 ... \\ + ... + \end{tabbing} + + Align text in columns, by setting tab stops and tabbing to them much +as was done on a typewriter. This is less often used than the +environments 'tabular' (*note tabular::) or 'array' (*note array::) +because in those the width of each column need not be constant and need +not be known in advance. + + This example has a first line where the tab stops are set to explicit +widths, ended by a '\kill' command (which is described below): + + \begin{tabbing} + \hspace{1.2in}\=\hspace{1in}\=\kill + Ship \>Guns \>Year \\ + \textit{Sophie} \>14 \>1800 \\ + \textit{Polychrest} \>24 \>1803 \\ + \textit{Lively} \>38 \>1804 \\ + \textit{Surprise} \>28 \>1805 \\ + \end{tabbing} + + Both the 'tabbing' environment and the more widely-used 'tabular' +environment put text in columns. The most important distinction is that +in 'tabular' the width of columns is determined automatically by LaTeX, +while in 'tabbing' the user sets the tab stops. Another distinction is +that 'tabular' generates a box, but 'tabbing' can be broken across +pages. Finally, while 'tabular' can be used in any mode, 'tabbing' can +be used only in paragraph mode and it always starts a new paragraph, +without indentation. + + Moreover, as shown in the example above, there is no need to use the +starred form of the '\hspace' command at the beginning of a tabbed row. +The right margin of the 'tabbing' environment is the end of line, so +that the width of the environment is '\linewidth'. + + The 'tabbing' environment contains a sequence of "tabbed rows". The +first tabbed row begins immediately after '\begin{tabbing}' and each row +ends with '\\' or '\kill'. The last row may omit the '\\' and end with +just '\end{tabbing}'. + + At any point the 'tabbing' environment has a "current tab stop +pattern", a sequence of N > 0 tab stops, numbered 0, 1, etc. These +create N corresponding columns. Tab stop 0 is always the left margin, +defined by the enclosing environment. Tab stop number I is set if it is +assigned a horizontal position on the page. Tab stop number I can only +be set if all the stops 0, ..., i-1 have already been set; normally +later stops are to the right of earlier ones. + + By default any text typeset in a 'tabbing' environment is typeset +ragged right and left-aligned on the current tab stop. Typesetting is +done in LR mode (*note Modes::). + + The following commands can be used inside a 'tabbing' environment. +They are all fragile (*note \protect::). + +'\\ (tabbing)' + End a tabbed line and typeset it. + +'\= (tabbing)' + Sets a tab stop at the current position. + +'\> (tabbing)' + Advances to the next tab stop. + +'\<' + Put following text to the left of the local margin (without + changing the margin). Can only be used at the start of the line. + +'\+' + Moves the left margin of the next and all the following commands + one tab stop to the right, beginning tabbed line if necessary. + +'\-' + Moves the left margin of the next and all the following commands + one tab stop to the left, beginning tabbed line if necessary. + +'\' (tabbing)' + Moves everything that you have typed so far in the current column, + i.e., everything from the most recent '\>', '\<', '\'', '\\', or + '\kill' command, to the previous column and aligned to the right, + flush against the current column's tab stop. + +'\` (tabbing)' + Allows you to put text flush right against any tab stop, including + tab stop 0. However, it can't move text to the right of the last + column because there's no tab stop there. The '\`' command moves + all the text that follows it, up to the '\\' or '\end{tabbing}' + command that ends the line, to the right margin of the 'tabbing' + environment. There must be no '\>' or '\'' command between the + '\`' and the '\\' or '\end{tabbing}' command that ends the line. + +'\a (tabbing)' + In a 'tabbing' environment, the commands '\=', '\'' and '\`' do not + produce accents as usual (*note Accents::). Instead, use the + commands '\a=', '\a'' and '\a`'. + +'\kill' + Sets tab stops without producing text. Works just like '\\' except + that it throws away the current line instead of producing output + for it. Any '\=', '\+' or '\-' commands in that line remain in + effect. + +'\poptabs' + Restores the tab stop positions saved by the last '\pushtabs'. + +'\pushtabs' + Saves all current tab stop positions. Useful for temporarily + changing tab stop positions in the middle of a 'tabbing' + environment. + +'\tabbingsep' + Distance of the text moved by '\'' to left of current tab stop. + + This example typesets a Pascal function: + + \begin{tabbing} + function \= fact(n : integer) : integer;\\ + \> begin \= \+ \\ + \> if \= n > 1 then \+ \\ + fact := n * fact(n-1) \- \\ + else \+ \\ + fact := 1; \-\- \\ + end;\\ + \end{tabbing} + +The output looks like this. + + function fact(n : integer) : integer; + begin + if n > 1 then + fact := n * fact(n-1); + else + fact := 1; + end; + +This example is just for illustration of the environment. To actually +typeset computer code in typewriter like this, a verbatim environment +(*note verbatim::) would normally be best. For pretty-printed code, +there are quite a few packages, including 'algorithm2e', 'fancyvrb', +'listings', and 'minted'. + + +File: latex2e.info, Node: table, Next: tabular, Prev: tabbing, Up: Environments + +8.22 'table' +============ + +Synopsis: + + \begin{table}[PLACEMENT] + TABLE BODY + \caption[LOFTITLE]{TITLE} % optional + \label{LABEL} % also optional + \end{table} + + A class of floats (*note Floats::). They cannot be split across +pages and so they are not typeset in sequence with the normal text but +instead are floated to a convenient place, such as the top of a +following page. + + This example 'table' environment contains a 'tabular' + + \begin{table} + \centering\small + \begin{tabular}{ll} + \multicolumn{1}{c}{\textit{Author}} + &\multicolumn{1}{c}{\textit{Piece}} \\ \hline + Bach &Cello Suite Number 1 \\ + Beethoven &Cello Sonata Number 3 \\ + Brahms &Cello Sonata Number 1 + \end{tabular} + \caption{Top cello pieces} + \label{tab:cello} + \end{table} + +but you can put many different kinds of content in a 'table': the TABLE +BODY may contain text, LaTeX commands, graphics, etc. It is typeset in +a 'parbox' of width '\textwidth'. + + For the possible values of PLACEMENT and their effect on the float +placement algorithm, see *note Floats::. + + The label is optional; it is used for cross references (*note Cross +references::). The '\caption' command is also optional. It specifies +caption text TITLE for the table (*note \caption::). By default it is +numbered. If its optional LOTTITLE is present then that text is used in +the list of tables instead of TITLE (*note Table of contents etc.::). + + In this example the table and caption will float to the bottom of a +page, unless it is pushed to a float page at the end. + + \begin{table}[b] + \centering + \begin{tabular}{r|p{2in}} \hline + One &The loneliest number \\ + Two &Can be as sad as one. + It's the loneliest number since the number one. + \end{tabular} + \caption{Cardinal virtues} + \label{tab:CardinalVirtues} + \end{table} + + +File: latex2e.info, Node: tabular, Next: thebibliography, Prev: table, Up: Environments + +8.23 'tabular' +============== + +Synopsis: + + \begin{tabular}[POS]{COLS} + COLUMN 1 ENTRY &COLUMN 2 ENTRY ... &COLUMN N ENTRY \\ + ... + \end{tabular} + +or + + \begin{tabular*}{WIDTH}[POS]{COLS} + COLUMN 1 ENTRY &COLUMN 2 ENTRY ... &COLUMN N ENTRY \\ + ... + \end{tabular*} + + Produce a table, a box consisting of a sequence of horizontal rows. +Each row consists of items that are aligned vertically in columns. This +illustrates many of the features. + + \begin{tabular}{l|l} + \textit{Player name} &\textit{Career home runs} \\ + \hline + Hank Aaron &755 \\ + Babe Ruth &714 + \end{tabular} + +The output will have two left-aligned columns with a vertical bar +between them. This is specified in 'tabular''s argument '{l|l}'. Put +the entries into different columns by separating them with an ampersand, +'&'. The end of each row is marked with a double backslash, '\\'. Put +a horizontal rule below a row, after a double backslash, with '\hline'. +After the last row the '\\' is optional, unless an '\hline' command +follows to put a rule below the table. + + The required and optional arguments to 'tabular' consist of: + +POS + Optional. Specifies the table's vertical position. The default is + to align the table so its vertical center matches the baseline of + the surrounding text. There are two other possible alignments: 't' + aligns the table so its top row matches the baseline of the + surrounding text, and 'b' aligns on the bottom row. + + This only has an effect if there is other text. In the common case + of a 'tabular' alone in a 'center' environment this option makes no + difference. + +COLS + Required. Specifies the formatting of columns. It consists of a + sequence of the following specifiers, corresponding to the types of + column and intercolumn material. + + 'l' + A column of left-aligned items. + + 'r' + A column of right-aligned items. + + 'c' + A column of centered items. + + '|' + A vertical line the full height and depth of the environment. + + '@{TEXT OR SPACE}' + Insert TEXT OR SPACE at this location in every row. The TEXT + OR SPACE material is typeset in LR mode. This text is fragile + (*note \protect::). + + If between two columns there is no @-expression then LaTeX's + 'book', 'article', and 'report' classes will put on either + side of each column a space of length '\tabcolsep', which by + default is 6pt. That is, by default adjacent columns are + separated by 12pt (so '\tabcolsep' is misleadingly named since + it is only half of the separation between tabular columns). + In addition, a space of 6pt also comes before the first column + and after the final column, unless you put a '@{...}' or '|' + there. + + If you override the default and use an @-expression then LaTeX + does not insert '\tabcolsep' so you must insert any desired + space yourself, as in '@{\hspace{1em}}'. + + An empty expression '@{}' will eliminate the space. In + particular, sometimes you want to eliminate the space before + the first column or after the last one, as in the example + below where the tabular lines need to lie on the left margin. + + \begin{flushleft} + \begin{tabular}{@{}l} + ... + \end{tabular} + \end{flushleft} + + The next example shows text, a decimal point between the + columns, arranged so the numbers in the table are aligned on + it. + + \begin{tabular}{r@{$.$}l} + $3$ &$14$ \\ + $9$ &$80665$ + \end{tabular} + + An '\extracolsep{WD}' command in an @-expression causes an + extra space of width WD to appear to the left of all + subsequent columns, until countermanded by another + '\extracolsep'. Unlike ordinary intercolumn space, this extra + space is not suppressed by an @-expression. An '\extracolsep' + command can be used only in an @-expression in the 'cols' + argument. Below, LaTeX inserts the right amount of + intercolumn space to make the entire table 4 inches wide. + + \begin{tabular*}{4in}{l@{\extracolsep{\fill}}l} + Seven times down, eight times up \ldots + &such is life! + \end{tabular*} + + To insert commands that are automatically executed before a + given column, load the 'array' package and use the '>{...}' + specifier. + + 'p{WD}' + Each item in the column is typeset in a parbox of width WD, as + if it were the argument of a '\parbox[t]{wd}{...}' command. + + A line break double backslash '\\' may not appear in the item, + except inside an environment like 'minipage', 'array', or + 'tabular', or inside an explicit '\parbox', or in the scope of + a '\centering', '\raggedright', or '\raggedleft' declaration + (when used in a 'p'-column element these declarations must + appear inside braces, as with '{\centering .. \\ ..}'). + Otherwise LaTeX will misinterpret the double backslash as + ending the tabular row. Instead, to get a line break in there + use '\newline' (*note \newline::). + + '*{NUM}{COLS}' + Equivalent to NUM copies of COLS, where NUM is a positive + integer and COLS is a list of specifiers. Thus the specifier + '\begin{tabular}{|*{3}{l|r}|}' is equivalent to the specifier + '\begin{tabular}{|l|rl|rl|r|}'. Note that COLS may contain + another '*'-expression. + +WIDTH + Required for 'tabular*', not allowed for 'tabular'. Specifies the + width of the 'tabular*' environment. The space between columns + should be rubber, as with '@{\extracolsep{\fill}}', to allow the + table to stretch or shrink to make the specified width, or else you + are likely to get the 'Underfull \hbox (badness 10000) in alignment + ...' warning. + + Parameters that control formatting: + +'\arrayrulewidth' + A length that is the thickness of the rule created by '|', + '\hline', and '\vline' in the 'tabular' and 'array' environments. + The default is '.4pt'. Change it as in + '\setlength{\arrayrulewidth}{0.8pt}'. + +'\arraystretch' + A factor by which the spacing between rows in the 'tabular' and + 'array' environments is multiplied. The default is '1', for no + scaling. Change it as '\renewcommand{\arraystretch}{1.2}'. + +'\doublerulesep' + A length that is the distance between the vertical rules produced + by the '||' specifier. The default is '2pt'. + +'\tabcolsep' + A length that is half of the space between columns. The default is + '6pt'. Change it with '\setlength'. + + The following commands can be used inside the body of a 'tabular' +environment, the first two inside an entry and the second two between +lines: + +* Menu: + +* \multicolumn:: Make an item spanning several columns. +* \vline:: Draw a vertical line. +* \cline:: Draw a horizontal line spanning some columns. +* \hline:: Draw a horizontal line spanning all columns. + + +File: latex2e.info, Node: \multicolumn, Next: \vline, Up: tabular + +8.23.1 '\multicolumn' +--------------------- + +Synopsis: + + \multicolumn{NUMCOLS}{COLS}{TEXT} + + Make an 'array' or 'tabular' entry that spans several columns. The +first argument NUMCOLS gives the number of columns to span. The second +argument COLS specifies the formatting of the entry, with 'c' for +centered, 'l' for flush left, or 'r' for flush right. The third +argument TEXT gives the contents of that entry. + + In this example, in the first row, the second and third columns are +spanned by the single heading 'Name'. + + \begin{tabular}{lccl} + \textit{ID} &\multicolumn{2}{c}{\textit{Name}} &\textit{Age} \\ + \hline + 978-0-393-03701-2 &O'Brian &Patrick &55 \\ + ... + \end{tabular} + + What counts as a column is: the column format specifier for the +'array' or 'tabular' environment is broken into parts, where each part +(except the first) begins with 'l', 'c', 'r', or 'p'. So from +'\begin{tabular}{|r|ccp{1.5in}|}' the parts are '|r|', 'c', 'c', +and 'p{1.5in}|'. + + The COLS argument overrides the 'array' or 'tabular' environment's +intercolumn area default adjoining this multicolumn entry. To affect +that area, this argument can contain vertical bars '|' indicating the +placement of vertical rules, and '@{...}' expressions. Thus if COLS is +'|c|' then this multicolumn entry will be centered and a vertical rule +will come in the intercolumn area before it and after it. This table +details the exact behavior. + + \begin{tabular}{|cc|c|c|} + \multicolumn{1}{r}{w} % entry one + &\multicolumn{1}{|r|}{x} % entry two + &\multicolumn{1}{|r}{y} % entry three + &z % entry four + \end{tabular} + +Before the first entry the output will not have a vertical rule because +the '\multicolumn' has the COLS specifier 'r' with no initial vertical +bar. Between entry one and entry two there will be a vertical rule; +although the first COLS does not have an ending vertical bar, the second +COLS does have a starting one. Between entry two and entry three there +is a single vertical rule; despite that the COLS in both of the +surrounding 'multicolumn''s call for a vertical rule, you only get one +rule. Between entry three and entry four there is no vertical rule; the +default calls for one but the COLS in the entry three '\multicolumn' +leaves it out, and that takes precedence. Finally, following entry four +there is a vertical rule because of the default. + + The number of spanned columns NUMCOLS can be 1. Besides giving the +ability to change the horizontal alignment, this also is useful to +override for one row the 'tabular' definition's default intercolumn area +specification, including the placement of vertical rules. + + In the example below, in the 'tabular' definition the first column is +specified to default to left justified but in the first row the entry is +centered with '\multicolumn{1}{c}{\textsc{Period}}'. Also in the first +row, the second and third columns are spanned by a single entry with +'\multicolumn{2}{c}{\textsc{Span}}', overriding the specification to +center those two columns on the page range en-dash. + + \begin{tabular}{l|r@{--}l} + \multicolumn{1}{c}{\textsc{Period}} + &\multicolumn{2}{c}{\textsc{Span}} \\ \hline + Baroque &1600 &1760 \\ + Classical &1730 &1820 \\ + Romantic &1780 &1910 \\ + Impressionistic &1875 &1925 + \end{tabular} + +Although the 'tabular' specification by default puts a vertical rule +between the first and second columns, no such vertical rule appears in +the first row here. That's because there is no vertical bar in the COLS +part of the first row's first '\multicolumn' command. + + +File: latex2e.info, Node: \vline, Next: \cline, Prev: \multicolumn, Up: tabular + +8.23.2 '\vline' +--------------- + +Draw a vertical line in a 'tabular' or 'array' environment extending the +full height and depth of an entry's row. Can also be used in an +@-expression, although its synonym vertical bar '|' is more common. +This command is rarely used in the body of a table; typically a table's +vertical lines are specified in 'tabular''s COLS argument and overridden +as needed with '\multicolumn' (*note tabular::). + + The example below illustrates some pitfalls. In the first row's +second entry the '\hfill' moves the '\vline' to the left edge of the +cell. But that is different than putting it halfway between the two +columns, so between the first and second columns there are two vertical +rules, with the one from the '{c|cc}' specifier coming before the one +produced by the '\vline\hfill'. In contrast, the first row's third +entry shows the usual way to put a vertical bar between two columns. In +the second row, the 'ghi' is the widest entry in its column so in the +'\vline\hfill' the '\hfill' has no effect and the vertical line in that +entry appears immediately next to the 'g', with no whitespace. + + \begin{tabular}{c|cc} + x &\vline\hfill y &\multicolumn{1}{|r}{z} \\ % row 1 + abc &def &\vline\hfill ghi % row 2 + \end{tabular} + + +File: latex2e.info, Node: \cline, Next: \hline, Prev: \vline, Up: tabular + +8.23.3 '\cline' +--------------- + +Synopsis: + + \cline{I-J} + + In an 'array' or 'tabular' environment, draw a horizontal rule +beginning in column I and ending in column J. The dash, '-', must +appear in the mandatory argument. To span a single column use the +number twice, as with '\cline{2-2}'. + + This example puts two horizontal lines between the first and second +rows, one line in the first column only, and the other spanning the +third and fourth columns. The two lines are side-by-side, at the same +height. + + \begin{tabular}{llrr} + a &b &c &d \\ \cline{1-1} \cline{3-4} + e &f &g &h + \end{tabular} + + +File: latex2e.info, Node: \hline, Prev: \cline, Up: tabular + +8.23.4 '\hline' +--------------- + +Draw a horizontal line the width of the enclosing 'tabular' or 'array' +environment. It's most commonly used to draw a line at the top, bottom, +and between the rows of a table. + + In this example the top of the table has two horizontal rules, one +above the other, that span both columns. The bottom of the table has a +single rule spanning both columns. Because of the '\hline', the +'tabular' second row's line ending double backslash '\\' is required. + + \begin{tabular}{ll} \hline\hline + Baseball &Red Sox \\ + Basketball &Celtics \\ \hline + \end{tabular} + + +File: latex2e.info, Node: thebibliography, Next: theorem, Prev: tabular, Up: Environments + +8.24 'thebibliography' +====================== + +Synopsis: + + \begin{thebibliography}{WIDEST-LABEL} + \bibitem[LABEL]{CITE_KEY} + ... + \end{thebibliography} + + Produce a bibliography or reference list. There are two ways to +produce bibliographic lists. This environment is suitable when you have +only a few references and can maintain the list by hand. *Note Using +BibTeX::, for a more sophisticated approach. + + This shows the environment with two entries. + + This work is based on \cite{latexdps}. + Together they are \cite{latexdps, texbook}. + ... + \begin{thebibliography}{9} + \bibitem{latexdps} + Leslie Lamport. + \textit{\LaTeX{}: a document preparation system}. + Addison-Wesley, Reading, Massachusetts, 1993. + \bibitem{texbook} + Donald Ervin Knuth. + \textit{The \TeX book}. + Addison-Wesley, Reading, Massachusetts, 1983. + \end{thebibliography} + +This styles the first reference as '[1] Leslie ...', and so that '... +based on \cite{latexdps}' produces the matching '... based on [1]'. The +second '\cite' produces '[1, 2]'. You must compile the document twice +to resolve these references. + + The mandatory argument WIDEST-LABEL is text that, when typeset, is as +wide as the widest item label produced by the '\bibitem' commands. The +tradition is to use '9' for bibliographies with less than 10 references, +'99' for ones with less than 100, etc. + + The bibliographic list is headed by a title such as 'Bibliography'. +To change it there are two cases. In the 'book' and 'report' classes, +where the top level sectioning is '\chapter' and the default title is +'Bibliography', that title is in the macro '\bibname'. For 'article', +where the class's top level sectioning is '\section' and the default is +'References', the title is in macro '\refname'. Change it by redefining +the command, as with '\renewcommand{\refname}{Cited references}', after +'\begin{document}'. + + Language support packages such as 'babel' will automatically redefine +'\refname' or '\bibname' to fit the selected language. + + *Note list::, for the list layout control parameters. + +* Menu: + +* \bibitem:: Specify a bibliography item. +* \cite:: Refer to a bibliography item. +* \nocite:: Include an item in the bibliography. +* Using BibTeX:: Automatic generation of bibliographies. + + +File: latex2e.info, Node: \bibitem, Next: \cite, Up: thebibliography + +8.24.1 '\bibitem' +----------------- + +Synopsis: + + \bibitem{CITE_KEY} + +or + + \bibitem[LABEL]{CITE_KEY} + + Generate an entry labeled by default by a number generated using the +'enumi' counter. The "citation key" CITE_KEY can be any string of +letters, numbers, and punctuation symbols (but not comma). + + *Note thebibliography::, for an example. + + When provided, the optional LABEL becomes the entry label and the +'enumi' counter is not incremented. With this + + \begin{thebibliography} + \bibitem[Lamport 1993]{latexdps} + Leslie Lamport. + \textit{\LaTeX{}: a document preparation system}. + Addison-Wesley, Reading, Massachusetts, 1993. + \bibitem{texbook} + Donald Ervin Knuth. + \textit{The \TeX book}. + Addison-Wesley, Reading, Massachusetts, 1983. + \end{thebibliography} + +the first entry will be styled as '[Lamport 1993] Leslie ...' (The +amount of horizontal space that LaTeX leaves for the label depends on +the WIDEST-LABEL argument of the 'thebibliography' environment; see +*note thebibliography::.) Similarly, '... based on \cite{latexdps}' +will produce '... based on [Lamport 1994]'. + + If you mix '\bibitem' entries having a LABEL with those that do not +then LaTeX will number the unlabelled ones sequentially. In the example +above the 'texbook' entry will appear as '[1] Donald ...', despite that +it is the second entry. + + If you use the same CITE_KEY twice then you get 'LaTeX Warning: There +were multiply-defined labels'. + + Under the hood, LaTeX remembers the CITE_KEY and LABEL information +because '\bibitem' writes it to the auxiliary file 'JOBNAME.aux' (*note +Jobname::). For instance, the above example causes the two +'\bibcite{latexdps}{Lamport, 1993}' and '\bibcite{texbook}{1}' to appear +in that file. The '.aux' file is read by the '\begin{document}' command +and then the information is available for '\cite' commands. This +explains why you need to run LaTeX twice to resolve references: once to +write it out and once to read it in. + + Because of this two-pass algorithm, when you add a '\bibitem' or +change its CITE_KEY you may get 'LaTeX Warning: Label(s) may have +changed. Rerun to get cross-references right'. Fix it by recompiling. + + +File: latex2e.info, Node: \cite, Next: \nocite, Prev: \bibitem, Up: thebibliography + +8.24.2 '\cite' +-------------- + +Synopsis: + + \cite{KEYS} + +or + + \cite[SUBCITE]{KEYS} + + Generate as output a citation to the references associated with KEYS. +The mandatory KEYS is a citation key, or a comma-separated list of +citation keys (*note \bibitem::). + + This + + The ultimate source is \cite{texbook}. + ... + \begin{thebibliography} + \bibitem{texbook} + Donald Ervin Knuth. + \textit{The \TeX book}. + Addison-Wesley, Reading, Massachusetts, 1983. + \end{thebibliography} + +produces output like '... source is [1]'. You can change the appearance +of the citation and of the reference by using bibliography styles if you +generate automatically the 'thebibliography' environment. More +information in *note Using BibTeX::. + + The optional argument SUBCITE is appended to the citation. For +example, 'See 14.3 in \cite[p.~314]{texbook}' might produce 'See 14.3 in +[1, p. 314]'. + + In addition to what appears in the output, '\cite' writes information +to the auxiliary file 'JOBNAME.aux' (*note Jobname::). For instance, +'\cite{latexdps}' writes '\citation{latexdps}' to that file. This +information is used by BibTeX to include in your reference list only +those works that you have actually cited; see *note \nocite:: also. + + If KEYS is not in your bibliography information then you get 'LaTeX +Warning: There were undefined references', and in the output the +citation shows as a boldface question mark between square brackets. +There are two possible causes. If you have mistyped something, as in +'\cite{texbok}' then you need to correct the spelling. On the other +hand, if you have just added or modified the bibliographic information +and so changed the '.aux' file (*note \bibitem::) then the fix may be to +run LaTeX again. + + +File: latex2e.info, Node: \nocite, Next: Using BibTeX, Prev: \cite, Up: thebibliography + +8.24.3 '\nocite' +---------------- + +Synopsis: + + \nocite{KEYS} + + Produces no output but writes KEYS to the auxiliary file +'JOBNAME.aux' (*note Jobname::). + + The mandatory argument KEYS is a comma-separated list of one or more +citation keys (*note \bibitem::). This information is used by BibTeX to +include these works in your reference list even though you have not +explicitly cited them (*note \cite::). + + +File: latex2e.info, Node: Using BibTeX, Prev: \nocite, Up: thebibliography + +8.24.4 Using BibTeX +------------------- + +As described in 'thebibliography' (*note thebibliography::), a +sophisticated approach to managing bibliographies is provided by the +BibTeX program. This is only an introduction; see the full +documentation on CTAN (*note CTAN::). + + With BibTeX, you don't use the 'thebibliography' environment directly +(*note thebibliography::). Instead, include these lines: + + \bibliographystyle{BIBSTYLE} + \bibliography{BIBFILE1, BIBFILE2, ...} + +The BIBSTYLE refers to a file 'BIBSTYLE.bst', which defines how your +citations will look. The standard BIBSTYLE's distributed with BibTeX +are: + +'alpha' + Labels are formed from name of author and year of publication. The + bibliographic items are sorted alphabetically. +'plain' + Labels are integers. Sort the bibliographic items alphabetically. +'unsrt' + Like 'plain', but entries are in order of citation. +'abbrv' + Like 'plain', but more compact labels. + +Many, many other BibTeX style files exist, tailored to the demands of +various publications. See the CTAN topic +. + + The '\bibliography' command is what actually produces the +bibliography. Its argument is a comma-separated list, referring to +files named 'BIBFILE1.bib', 'BIBFILE2.bib', ... These contain your +database in BibTeX format. This shows a typical couple of entries in +that format. + + @book{texbook, + title = {The {{\TeX}}book}, + author = {D.E. Knuth}, + isbn = {0201134489}, + series = {Computers \& typesetting}, + year = {1983}, + publisher = {Addison-Wesley} + } + @book{sexbook, + author = {W.H. Masters and V.E. Johnson}, + title = {Human Sexual Response}, + year = {1966}, + publisher = {Bantam Books} + } + + Only the bibliographic entries referred to via '\cite' and '\nocite' +will be listed in the document's bibliography. Thus you can keep all +your sources together in one file, or a small number of files, and rely +on BibTeX to include in this document only those that you used. + + With BibTeX, the KEYS argument to '\nocite' can also be the single +character '*'. This means to implicitly cite all entries from all given +bibliographies. + +* Menu: + +* BibTeX error messages:: + + +File: latex2e.info, Node: BibTeX error messages, Up: Using BibTeX + +8.24.4.1 BibTeX error messages +.............................. + +If you forget to use '\bibliography' or '\bibliographystyle' in your +document (or, less likely, any '\cite' or '\nocite' command), BibTeX +will issue an error message. Because BibTeX can be used with any +program, not just LaTeX, the error messages refer to the internal +commands read by BibTeX (from the '.aux' file), rather than the +user-level commands described above. + + Here is a table showing internal commands mentioned in the BibTeX +errors, and the corresponding user-level commands. + +'\bibdata' + '\bibliography' + +'\bibstyle' + '\bibliographystyle' + +'\citation' + '\cite', '\nocite' + + For example, if your document has no '\bibliographystyle' command, +BibTeX complains as follows: + + I found no \bibstyle command---while reading file DOCUMENT.aux + + +File: latex2e.info, Node: theorem, Next: titlepage, Prev: thebibliography, Up: Environments + +8.25 'theorem' +============== + +Synopsis: + + \begin{theorem} + THEOREM BODY + \end{theorem} + + Produces 'Theorem N' in boldface followed by THEOREM BODY in italics. +The numbering possibilities for N are described under '\newtheorem' +(*note \newtheorem::). + + \newtheorem{lem}{Lemma} % in preamble + \newtheorem{thm}{Theorem} + ... + \begin{lem} % in document body + text of lemma + \end{lem} + + The next result follows immediately. + \begin{thm}[Gauss] % put `Gauss' in parens after theorem head + text of theorem + \end{thm} + + Most new documents use the packages 'amsthm' and 'amsmath' from the +American Mathematical Society. Among other things these packages +include a large number of options for theorem environments, such as +styling options. + + +File: latex2e.info, Node: titlepage, Next: verbatim, Prev: theorem, Up: Environments + +8.26 'titlepage' +================ + +Synopsis: + + \begin{titlepage} + ... text and spacing ... + \end{titlepage} + + Create a title page, a page with no printed page number or heading +and with succeeding pages numbered starting with page one. + + In this example all formatting, including vertical spacing, is left +to the author. + + \begin{titlepage} + \vspace*{\stretch{1}} + \begin{center} + {\huge\bfseries Thesis \\[1ex] + title} \\[6.5ex] + {\large\bfseries Author name} \\ + \vspace{4ex} + Thesis submitted to \\[5pt] + \textit{University name} \\[2cm] + in partial fulfilment for the award of the degree of \\[2cm] + \textsc{\Large Doctor of Philosophy} \\[2ex] + \textsc{\large Mathematics} \\[12ex] + \vfill + Department of Mathematics \\ + Address \\ + \vfill + \today + \end{center} + \vspace{\stretch{2}} + \end{titlepage} + + To instead produce a standard title page without a 'titlepage' +environment, use '\maketitle' (*note \maketitle::). + + +File: latex2e.info, Node: verbatim, Next: verse, Prev: titlepage, Up: Environments + +8.27 'verbatim' +=============== + +Synopsis: + + \begin{verbatim} + LITERAL-TEXT + \end{verbatim} + + A paragraph-making environment in which LaTeX produces as output +exactly what you type as input. For instance inside LITERAL-TEXT the +backslash '\' character does not start commands, it produces a printed +'\', and carriage returns and blanks are taken literally. The output +appears in a monospaced typewriter-like font ('\tt'). + + \begin{verbatim} + Symbol swearing: %&$#?!. + \end{verbatim} + + The only restriction on 'literal-text' is that it cannot include the +string '\end{verbatim}'. + + You cannot use the verbatim environment in the argument to macros, +for instance in the argument to a '\section'. This is not the same as +commands being fragile (*note \protect::), instead it just cannot work, +as the 'verbatim' environment changes the catcode regime before +processing its contents, and restore it immediately afterward, +nevertheless with a macro argument the content of the argument has +already be converted to a token list along the catcode regime in effect +when the macro was called. However, the 'cprotect' package can help +with this. + + One common use of verbatim input is to typeset computer code. There +are packages that are an improvement the 'verbatim' environment. For +instance, one improvement is to allow the verbatim inclusion of external +files, or parts of those files. Such packages include 'listings', and +'minted'. + + A package that provides many more options for verbatim environments +is 'fancyvrb'. Another is 'verbatimbox'. + + For a list of all the relevant packages, see CTAN (*note CTAN::). + +* Menu: + +* \verb:: The macro form of the 'verbatim' environment. + + +File: latex2e.info, Node: \verb, Up: verbatim + +8.27.1 '\verb' +-------------- + +Synopsis: + + \verb CHAR LITERAL-TEXT CHAR + \verb* CHAR LITERAL-TEXT CHAR + + Typeset LITERAL-TEXT as it is input, including special characters and +spaces, using the typewriter ('\tt') font. + + This example shows two different invocations of '\verb'. + + This is \verb!literally! the biggest pumpkin ever. + And this is the best squash, \verb+literally!+ + +The first '\verb' has its LITERAL-TEXT surrounded with exclamation +point, '!'. The second instead uses plus, '+', because the exclamation +point is part of 'literal-text'. + + The single-character delimiter CHAR surrounds LITERAL-TEXT--it must +be the same character before and after. No spaces come between '\verb' +or '\verb*' and CHAR, or between CHAR and LITERAL-TEXT, or between +LITERAL-TEXT and the second occurrence of CHAR (the synopsis shows a +space only to distinguish one component from the other). The delimiter +must not appear in LITERAL-TEXT. The LITERAL-TEXT cannot include a line +break. + + The '*'-form differs only in that spaces are printed with a visible +space character. + + The output from this will include a visible space on both side of +word 'with': + + The command's first argument is \verb*!filename with extension! and ... + + For typesetting Internet addresses, urls, the package 'url' is a +better option than the '\verb' command, since it allows line breaks. + + For computer code there are many packages with advantages over +'\verb'. One is 'listings', another is 'minted'. + + You cannot use '\verb' in the argument to a macro, for instance in +the argument to a '\section'. It is not a question of '\verb' being +fragile (*note \protect::), instead it just cannot work, as the '\verb' +command changes the catcode regime before reading its argument, and +restore it immediately afterward, nevertheless with a macro argument the +content of the argument has already be converted to a token list along +the catcode regime in effect when the macro was called. However, the +'cprotect' package can help with this. + + +File: latex2e.info, Node: verse, Prev: verbatim, Up: Environments + +8.28 'verse' +============ + +Synopsis: + + \begin{verse} + LINE1 \\ + LINE2 \\ + ... + \end{verse} + + An environment for poetry. + + Here are two lines from Shakespeare's Romeo and Juliet. + + Then plainly know my heart's dear love is set \\ + On the fair daughter of rich Capulet. + + Separate the lines of each stanza with '\\', and use one or more +blank lines to separate the stanzas. + + \begin{verse} + \makebox[\linewidth][c]{\textit{Shut Not Your Doors} ---Walt Whitman} + \\[1\baselineskip] + Shut not your doors to me proud libraries, \\ + For that which was lacking on all your well-fill'd shelves, \\ + \qquad yet needed most, I bring, \\ + Forth from the war emerging, a book I have made, \\ + The words of my book nothing, the drift of it every thing, \\ + A book separate, not link'd with the rest nor felt by the intellect, \\ + But you ye untold latencies will thrill to every page. + \end{verse} + +The output has margins indented on the left and the right, paragraphs +are not indented, and the text is not right-justified. + + +File: latex2e.info, Node: Line breaking, Next: Page breaking, Prev: Environments, Up: Top + +9 Line breaking +*************** + +The first thing LaTeX does when processing ordinary text is to translate +your input file into a sequence of glyphs and spaces. To produce a +printed document, this sequence must be broken into lines (and these +lines must be broken into pages). + + LaTeX usually does the line (and page) breaking in the text body for +you but in some environments you manually force line breaks. + + A common workflow is to get a final version of the document content +before taking a final pass through and considering line breaks (and page +breaks). This differs from word processing, where you are formatting +text as you input it. Putting these off until the end prevents a lot of +fiddling with breaks that will change anyway. + +* Menu: + +* \\:: Start a new line. +* \obeycr & \restorecr:: Make each input line start a new output line. +* \newline:: Break the line +* \- (hyphenation):: Insert explicit hyphenation. +* \discretionary:: Explicit control of the hyphen character. +* \fussy & \sloppy:: Be more or less particular with line breaking. +* \hyphenation:: Tell LaTeX how to hyphenate a word. +* \linebreak & \nolinebreak:: Forcing & avoiding line breaks. + + +File: latex2e.info, Node: \\, Next: \obeycr & \restorecr, Up: Line breaking + +9.1 '\\' +======== + +Synopsis, one of: + + \\ + \\[MORESPACE] + +or one of: + + \\* + \\*[MORESPACE] + + End the current line. The optional argument MORESPACE specifies +extra vertical space to be inserted before the next line. This is a +rubber length (*note Lengths::) and can be negative. The text before +the line break is set at its normal length, that is, it is not stretched +to fill out the line width. This command is fragile (*note \protect::). + + \title{My story: \\[0.25in] + a tale of woe} + +The starred form, '\\*', tells LaTeX not to start a new page between the +two lines, by issuing a '\nobreak'. + + Explicit line breaks in the main text body are unusual in LaTeX. In +particular, don't start new paragraphs with '\\'. Instead leave a blank +line between the two paragraphs. And don't put in a sequence of '\\''s +to make vertical space. Instead use '\vspace{LENGTH}', or +'\leavevmode\vspace{LENGTH}', or '\vspace*{LENGTH}' if you want the +space to not be thrown out at the top of a new page (*note \vspace::). + + The '\\' command is mostly used outside of the main flow of text such +as in a 'tabular' or 'array' environment or in an equation environment. + + The '\\' command is a synonym for '\newline' (*note \newline::) under +ordinary circumstances (an example of an exception is the 'p{...}' +column in a 'tabular' environment; *note tabular::). + + The '\\' command is a macro, and its definition changes by context so +that its definition in normal text, a 'center' environment, a +'flushleft' environment, and a 'tabular' are all different. In normal +text when it forces a linebreak it is essentially a shorthand for +'\newline'. It does not end horizontal mode or end the paragraph, it +just inserts some glue and penalties so that when the paragraph does end +a linebreak will occur at that point, with the short line padded with +white space. + + You get 'LaTeX Error: There's no line here to end' if you use '\\' to +ask for a new line, rather than to end the current line. An example is +if you have '\begin{document}\\' or, more likely, something like this. + + \begin{center} + \begin{minipage}{0.5\textwidth} + \\ + In that vertical space put your mark. + \end{minipage} + \end{center} + +Fix it by replacing the double backslash with something like +'\vspace{\baselineskip}'. + + +File: latex2e.info, Node: \obeycr & \restorecr, Next: \newline, Prev: \\, Up: Line breaking + +9.2 '\obeycr' & '\restorecr' +============================ + +The '\obeycr' command makes a return in the input file ('^^M', +internally) the same as '\\', followed by '\relax'. So each new line in +the input will also be a new line in the output. The '\restorecr' +command restores normal line-breaking behavior. + + This is not the way to show verbatim text or computer code. Use +'verbatim' (*note verbatim::) instead. + + With LaTeX's usual defaults, this + + aaa + bbb + + \obeycr + ccc + ddd + eee + + \restorecr + fff + ggg + + hhh + iii + +produces output like this. + + aaa bbb + ccc + ddd + eee + + fff ggg + hhh iii + +The indents are paragraph indents. + + +File: latex2e.info, Node: \newline, Next: \- (hyphenation), Prev: \obeycr & \restorecr, Up: Line breaking + +9.3 '\newline' +============== + +In ordinary text, this ends a line in a way that does not right-justify +it, so the text before the end of line is not stretched. That is, in +paragraph mode (*note Modes::), the '\newline' command is equivalent to +double-backslash (*note \\::). This command is fragile (*note +\protect::). + + However, the two commands are different inside a 'tabular' or 'array' +environment. In a column with a specifier producing a paragraph box +such as typically 'p{...}', '\newline' will insert a line end inside of +the column; that is, it does not break the entire tabular row. To break +the entire row use '\\' or its equivalent '\tabularnewline'. + + This will print 'Name:' and 'Address:' as two lines in a single cell +of the table. + + \begin{tabular}{p{1in}@{\hspace{2in}}p{1in}} + Name: \newline Address: &Date: \\ \hline + \end{tabular} + +The 'Date:' will be baseline-aligned with 'Name:'. + + +File: latex2e.info, Node: \- (hyphenation), Next: \discretionary, Prev: \newline, Up: Line breaking + +9.4 '\-' (discretionary hyphen) +=============================== + +Tell LaTeX that it may hyphenate the word at that point. When you +insert '\-' commands in a word, the word will only be hyphenated at +those points and not at any of the other hyphenation points that LaTeX +might otherwise have chosen. This command is robust (*note \protect::). + + LaTeX is good at hyphenating and usually finds most of the correct +hyphenation points, while almost never using an incorrect one. The '\-' +command is for exceptional cases. + + For example, LaTeX does not ordinarily hyphenate words containing a +hyphen. Below, the long and hyphenated word means LaTeX has to put in +unacceptably large spaces to set the narrow column. + + \begin{tabular}{rp{1.75in}} + Isaac Asimov &The strain of + anti-intellectualism + % an\-ti-in\-tel\-lec\-tu\-al\-ism + has been a constant thread winding its way through our + political and cultural life, nurtured by + the false notion that democracy means that + `my ignorance is just as good as your knowledge'. + \end{tabular} + +Commenting out the third line and uncommenting the fourth makes a much +better fit. + + The '\-' command only allows LaTeX to break there, it does not +require that it break there. You can force a split with something like +'Hef-\linebreak feron'. Of course, if you later change the text then +this forced break may look very odd, so this approach requires care. + + +File: latex2e.info, Node: \discretionary, Next: \fussy & \sloppy, Prev: \- (hyphenation), Up: Line breaking + +9.5 '\discretionary' (generalized hyphenation point) +==================================================== + +Synopsis: + + \discretionary{PRE-BREAK}{POST-BREAK}{NO-BREAK} + + Handle word changes around hyphens. This command is not often used +in LaTeX documents. + + If a line break occurs at the point where '\discretionary' appears +then TeX puts PRE-BREAK at the end of the current line and puts +POST-BREAK at the start of the next line. If there is no line break +here then TeX puts NO-BREAK. + + In 'difficult' the three letters 'ffi' form a ligature. But TeX can +nonetheless break between the two 'f''s with this. + + di\discretionary{f-}{fi}{ffi}cult + + Note that users do not have to do this. It is typically handled +automatically by TeX's hyphenation algorithm. + + +File: latex2e.info, Node: \fussy & \sloppy, Next: \hyphenation, Prev: \discretionary, Up: Line breaking + +9.6 '\fussy' & '\sloppy' +======================== + +Declarations to make TeX more picky or less picky about line breaking. +Declaring '\fussy' usually avoids too much space between words, at the +cost of an occasional overfull box. Conversely, '\sloppy' avoids +overfull boxes while suffering loose interword spacing. + + The default is '\fussy'. Line breaking in a paragraph is controlled +by whichever declaration is current at the end of the paragraph, i.e., +at the blank line or '\par' or displayed equation ending that paragraph. +So to affect the line breaks, include that paragraph-ending material in +the scope of the command. + +* Menu: + +* sloppypar:: Environment version of \sloppy command. + + +File: latex2e.info, Node: sloppypar, Up: \fussy & \sloppy + +9.6.1 'sloppypar' +----------------- + +Synopsis: + + \begin{sloppypar} + ... paragraphs ... + \end{sloppypar} + + Typeset the paragraphs with '\sloppy' in effect (*note \fussy & +\sloppy::). Use this to locally adjust line breaking, to avoid +'Overfull box' or 'Underfull box' errors. + + The example is simple. + + \begin{sloppypar} + Her plan for the morning thus settled, she sat quietly down to her + book after breakfast, resolving to remain in the same place and the + same employment till the clock struck one; and from habitude very + little incommoded by the remarks and ejaculations of Mrs.\ Allen, + whose vacancy of mind and incapacity for thinking were such, that + as she never talked a great deal, so she could never be entirely + silent; and, therefore, while she sat at her work, if she lost her + needle or broke her thread, if she heard a carriage in the street, + or saw a speck upon her gown, she must observe it aloud, whether + there were anyone at leisure to answer her or not. + \end{sloppypar} + + +File: latex2e.info, Node: \hyphenation, Next: \linebreak & \nolinebreak, Prev: \fussy & \sloppy, Up: Line breaking + +9.7 '\hyphenation' +================== + +Synopsis: + + \hyphenation{WORD1 ...} + + Declares allowed hyphenation points within the words in the list. +The words in that list are separated by spaces. Show permitted points +for hyphenation with a dash character, '-'. + + Here is an example: + + \hyphenation{hat-er il-lit-e-ra-ti tru-th-i-ness} + + Use lowercase letters. TeX will only hyphenate if the word matches +exactly, no inflections are tried. Multiple '\hyphenation' commands +accumulate. + + +File: latex2e.info, Node: \linebreak & \nolinebreak, Prev: \hyphenation, Up: Line breaking + +9.8 '\linebreak' & '\nolinebreak' +================================= + +Synopses, one of: + + \linebreak + \linebreak[ZERO-TO-FOUR] + +or one of these. + + \nolinebreak + \nolinebreak[ZERO-TO-FOUR] + + Encourage or discourage a line break. The optional ZERO-TO-FOUR is +an integer lying between 0 and 4 that allows you to soften the +instruction. The default is 4, so that without the optional argument +these commands entirely force or prevent the break. But for instance, +'\nolinebreak[1]' is a suggestion that another place may be better. The +higher the number, the more insistent the request. Both commands are +fragile (*note \protect::). + + Here we tell LaTeX that a good place to put a linebreak is after the +standard legal text. + + \boilerplatelegal{} \linebreak[2] + We especially encourage applications from members of traditionally + underrepresented groups. + + When you issue '\linebreak', the spaces in the line are stretched out +so that the break point reaches the right margin. *Note \\:: and *note +\newline::, to have the spaces not stretched out. + + +File: latex2e.info, Node: Page breaking, Next: Footnotes, Prev: Line breaking, Up: Top + +10 Page breaking +**************** + +Ordinarily LaTeX automatically takes care of breaking output into pages +with its usual aplomb. But if you are writing commands, or tweaking the +final version of a document, then you may need to understand how to +influence its actions. + + LaTeX's algorithm for splitting a document into pages is more complex +than just waiting until there is enough material to fill a page and +outputting the result. Instead, LaTeX typesets more material than would +fit on the page and then chooses a break that is optimal in some way (it +has the smallest badness). An example of the advantage of this approach +is that if the page has some vertical space that can be stretched or +shrunk, such as with rubber lengths between paragraphs, then LaTeX can +use that to avoid widow lines (where a new page starts with the last +line of a paragraph; LaTeX can squeeze the extra line onto the first +page) and orphans (where the first line of paragraph is at the end of a +page; LaTeX can stretch the material of the first page so the extra line +falls on the second page). Another example is where LaTeX uses +available vertical shrinkage to fit on a page not just the header for a +new section but also the first two lines of that section. + + But LaTeX does not optimize over the entire document's set of page +breaks. So it can happen that the first page break is great but the +second one is lousy; to break the current page LaTeX doesn't look as far +ahead as the next page break. So occasionally you may want to influence +page breaks while preparing a final version of a document. + + *Note Layout::, for more material that is relevant to page breaking. + +* Menu: + +* \clearpage & \cleardoublepage:: Start a new page; eject floats. +* \newpage:: Start a new page. +* \enlargethispage:: Enlarge the current page a bit. +* \pagebreak & \nopagebreak:: Forcing & avoiding page breaks. + + +File: latex2e.info, Node: \clearpage & \cleardoublepage, Next: \newpage, Up: Page breaking + +10.1 '\clearpage' & '\cleardoublepage' +====================================== + +Synopsis: + + \clearpage + +or + + \cleardoublepage + + End the current page and output all of the pending floating figures +and tables (*note Floats::). If there are too many floats to fit on the +page then LaTeX will put in extra pages containing only floats. In +two-sided printing, '\cleardoublepage' also makes the next page of +content a right-hand page, an odd-numbered page, if necessary inserting +a blank page. The '\clearpage' command is robust while +'\cleardoublepage' is fragile (*note \protect::). + + LaTeX's page breaks are optimized so ordinarily you only use this +command in a document body to polish the final version, or inside +commands. + + The '\cleardoublepage' command will put in a blank page, but it will +have the running headers and footers. To get a really blank page, use +this command. + + \let\origdoublepage\cleardoublepage + \newcommand{\clearemptydoublepage}{% + \clearpage + {\pagestyle{empty}\origdoublepage}% + } + +If you want LaTeX's standard '\chapter' command to do this then add the +line '\let\cleardoublepage\clearemptydoublepage'. + + The command '\newpage' (*note \newpage::) also ends the current page, +but without clearing pending floats. And, if LaTeX is in two-column +mode then '\newpage' ends the current column while '\clearpage' and +'\cleardoublepage' end the current page. + + +File: latex2e.info, Node: \newpage, Next: \enlargethispage, Prev: \clearpage & \cleardoublepage, Up: Page breaking + +10.2 '\newpage' +=============== + +Synopsis: + + \newpage + + End the current page. This command is robust (*note \protect::). + + LaTeX's page breaks are optimized so ordinarily you only use this +command in a document body to polish the final version, or inside +commands. + + While the commands '\clearpage' and '\cleardoublepage' also end the +current page, in addition they clear pending floats (*note \clearpage & +\cleardoublepage::). And, if LaTeX is in two-column mode then +'\clearpage' and '\cleardoublepage' end the current page, possibly +leaving an empty column, while '\newpage' only ends the current column. + + In contrast with '\pagebreak' (*note \pagebreak & \nopagebreak::), +the '\newpage' command will cause the new page to start right where +requested. This + + Four score and seven years ago our fathers brought forth on this + continent, + \newpage + \noindent a new nation, conceived in Liberty, and dedicated to the + proposition that all men are created equal. + +makes a new page start after 'continent', and the cut-off line is not +right justified. In addition, '\newpage' does not vertically stretch +out the page, as '\pagebreak' does. + + +File: latex2e.info, Node: \enlargethispage, Next: \pagebreak & \nopagebreak, Prev: \newpage, Up: Page breaking + +10.3 '\enlargethispage' +======================= + +Synopsis, one of: + + \enlargethispage{size} + \enlargethispage*{size} + + Enlarge the '\textheight' for the current page. The required +argument SIZE must be a rigid length (*note Lengths::). It may be +positive or negative. This command is fragile (*note \protect::). + + A common strategy is to wait until you have the final text of a +document, and then pass through it tweaking line and page breaks. This +command allows you some page size leeway. + + This will allow one extra line on the current page. + + \enlargethispage{\baselineskip} + + The starred form '\enlargesthispage*' tries to squeeze the material +together on the page as much as possible, for the common use case of +getting one more line on the page. This is often used together with an +explicit '\pagebreak'. + + +File: latex2e.info, Node: \pagebreak & \nopagebreak, Prev: \enlargethispage, Up: Page breaking + +10.4 '\pagebreak' & '\nopagebreak' +================================== + +Synopses: + + \pagebreak + \pagebreak[ZERO-TO-FOUR] + +or + + \nopagebreak + \nopagebreak[ZERO-TO-FOUR] + + Encourage or discourage a page break. The optional ZERO-TO-FOUR is +an integer that allows you to soften the request. The default is 4, so +that without the optional argument these commands entirely force or +prevent the break. But for instance '\nopagebreak[1]' suggests to LaTeX +that another spot might be preferable. The higher the number, the more +insistent the request. Both commands are fragile (*note \protect::). + + LaTeX's page endings are optimized so ordinarily you only use this +command in a document body to polish the final version, or inside +commands. + + If you use these inside a paragraph, they apply to the point +following the line in which they appear. So this + + Four score and seven years ago our fathers brought forth on this + continent, + \pagebreak + a new nation, conceived in Liberty, and dedicated to the proposition + that all men are created equal. + +does not give a page break at 'continent', but instead at 'nation', +since that is where LaTeX breaks that line. In addition, with +'\pagebreak' the vertical space on the page is stretched out where +possible so that it extends to the normal bottom margin. This can look +strange, and if '\flushbottom' is in effect this can cause you to get +'Underfull \vbox (badness 10000) has occurred while \output is active'. +*Note \newpage::, for a command that does not have these effects. + + (There is an obsolete declaration '\samepage', which tries to only +allow a break between two paragraphs. There is a related environment +'samepage', also obsolete. Neither of these work reliably. For more on +keeping material on the same page, see the FAQ entry +.) + + +File: latex2e.info, Node: Footnotes, Next: Definitions, Prev: Page breaking, Up: Top + +11 Footnotes +************ + +Place a footnote at the bottom of the current page, as here. + + Noël Coward quipped that having to read a footnote is like having + to go downstairs to answer the door, while in the midst of making + love.\footnote{% + I wouldn't know, I don't read footnotes.} + + You can put multiple footnotes on a page. If the footnote text +becomes too long then it will flow to the next page. + + You can also produce footnotes by combining the '\footnotemark' and +the '\footnotetext' commands, which is useful in special circumstances. + + To make bibliographic references come out as footnotes you need to +include a bibliographic style with that behavior (*note Using BibTeX::). + +* Menu: + +* \footnote:: Insert a footnote. +* \footnotemark:: Insert footnote mark only. +* \footnotetext:: Insert footnote text only. +* Footnotes in section headings:: Chapter or section titles. +* Footnotes in a table:: Table footnotes. +* Footnotes of footnotes:: Multiple classes of footnotes. + + +File: latex2e.info, Node: \footnote, Next: \footnotemark, Up: Footnotes + +11.1 '\footnote' +================ + +Synopsis, one of: + + \footnote{TEXT} + \footnote[NUMBER]{TEXT} + + Place a footnote TEXT at the bottom of the current page. + + There are over a thousand footnotes in Gibbon's + \textit{Decline and Fall of the Roman Empire}.\footnote{% + After reading an early version with endnotes David Hume complained, + ``One is also plagued with his Notes, according to the present Method + of printing the Book'' and suggested that they ``only to be printed + at the Margin or the Bottom of the Page.''} + + The optional argument NUMBER allows you to specify the number of the +footnote. If you use this then LaTeX does not increment the 'footnote' +counter. + + By default, LaTeX uses arabic numbers as footnote markers. Change +this with something like +'\renewcommand{\thefootnote}{\fnsymbol{footnote}}', which uses a +sequence of symbols (*note \alph \Alph \arabic \roman \Roman +\fnsymbol::). To make this change global put that in the preamble. If +you make the change local then you may want to reset the counter with +'\setcounter{footnote}{0}'. + + LaTeX determines the spacing of footnotes with two parameters. + +'\footnoterule' + Produces the rule separating the main text on a page from the + page's footnotes. Default dimensions in the standard document + classes (except 'slides', where it does not appear) is: vertical + thickness of '0.4pt', and horizontal size of '0.4\columnwidth' + long. Change the rule with something like this. + + \renewcommand{\footnoterule}{% Kerns avoid vertical space + \kern -3pt % This -3 is negative + \hrule width \textwidth height 1pt % of the sum of this 1 + \kern 2pt} % and this 2 + +'\footnotesep' + The height of the strut placed at the beginning of the footnote + (*note \strut::). By default, this is set to the normal strut for + '\footnotesize' fonts (*note Font sizes::), therefore there is no + extra space between footnotes. This is '6.65pt' for '10pt', + '7.7pt' for '11pt', and '8.4pt' for '12pt'. Change it as with + '\setlength{\footnotesep}{11pt}'. + + The '\footnote' command is fragile (*note \protect::). + + LaTeX's default puts many restrictions on where you can use a +'\footnote'; for instance, you cannot use it in an argument to a +sectioning command such as '\chapter' (it can only be used in outer +paragraph mode; *note Modes::). There are some workarounds; see +following sections. + + In a 'minipage' environment the '\footnote' command uses the +'mpfootnote' counter instead of the 'footnote' counter, so they are +numbered independently. They are shown at the bottom of the +environment, not at the bottom of the page. And by default they are +shown alphabetically. *Note minipage:: and *note Footnotes in a +table::. + + +File: latex2e.info, Node: \footnotemark, Next: \footnotetext, Prev: \footnote, Up: Footnotes + +11.2 '\footnotemark' +==================== + +Synopsis, one of: + + \footnotemark + \footnotemark[NUMBER] + + Put the current footnote mark in the text. To specify associated +text for the footnote see *note \footnotetext::. The optional argument +NUMBER causes the command to use that number to determine the footnote +mark. This command can be used in inner paragraph mode (*note Modes::). + + If you use '\footnotemark' without the optional argument then it +increments the 'footnote' counter, but if you use the optional NUMBER +then it does not. The next example produces several consecutive +footnote markers referring to the same footnote. + + The first theorem\footnote{Due to Gauss.} + and the second theorem\footnotemark[\value{footnote}] + and the third theorem.\footnotemark[\value{footnote}] + + If there are intervening footnotes then you must remember the value +of the number of the common mark. This example gives the same +institutional affiliation to both the first and third authors ('\thanks' +is a version of '\footnote'), by-hand giving the number of the footnote. + + \title{A Treatise on the Binomial Theorem} + \author{J Moriarty\thanks{University of Leeds} + \and A C Doyle\thanks{Durham University} + \and S Holmes\footnotemark[1]} + \begin{document} + \maketitle + + This uses a counter to remember the footnote number. The third +sentence is followed by the same footnote marker as the first. + + \newcounter{footnoteValueSaver} + All babies are illogical.\footnote{% + Lewis Carroll.}\setcounter{footnoteValueSaver}{\value{footnote}} + Nobody is despised who can manage a crocodile.\footnote{% + Captain Hook.} + Illogical persons are despised.\footnotemark[\value{footnoteValueSaver}] + Therefore, anyone who can manage a crocodile is not a baby. + + This example accomplishes the same by using the package 'cleveref'. + + \usepackage{cleveref}[2012/02/15] % in preamble + \crefformat{footnote}{#2\footnotemark[#1]#3} + ... + The theorem is from Evers.\footnote{\label{fn:TE}Tinker, Evers, 1994.} + The corollary is from Chance.\footnote{Evers, Chance, 1990.} + But the key lemma is from Tinker.\cref{fn:TE} + + It will work with the package 'hyperref'. + + +File: latex2e.info, Node: \footnotetext, Next: Footnotes in section headings, Prev: \footnotemark, Up: Footnotes + +11.3 '\footnotetext' +==================== + +Synopsis, one of: + + \footnotetext{TEXT} + \footnotetext[NUMBER]{TEXT} + + Place TEXT at the bottom of the page as a footnote. It pairs with +'\footnotemark' (*note \footnotemark::) and can come anywhere after that +command, but must appear in outer paragraph mode (*note Modes::). The +optional argument NUMBER changes the number of the footnote mark. + + *Note \footnotemark:: and *note Footnotes in a table::, for usage +examples. + + +File: latex2e.info, Node: Footnotes in section headings, Next: Footnotes in a table, Prev: \footnotetext, Up: Footnotes + +11.4 Footnotes in section headings +================================== + +Putting a footnote in a section heading, as in: + + \section{Full sets\protect\footnote{This material due to ...}} + +causes the footnote to appear at the bottom of the page where the +section starts, as usual, but also at the bottom of the table of +contents, where it is not likely to be desired. The simplest way to +have it not appear on the table of contents is to use the optional +argument to '\section' + + \section[Please]{Please\footnote{% + Don't footnote in chapter and section headers!}} + +No '\protect' is needed in front of '\footnote' here because what gets +moved to the table of contents is the optional argument. + + +File: latex2e.info, Node: Footnotes in a table, Next: Footnotes of footnotes, Prev: Footnotes in section headings, Up: Footnotes + +11.5 Footnotes in a table +========================= + +Inside a 'tabular' or 'array' environment the '\footnote' command does +not work; there is a footnote mark in the table cell but the footnote +text does not appear. The solution is to use a 'minipage' environment +as here (*note minipage::). + + \begin{center} + \begin{minipage}{\textwidth} \centering + \begin{tabular}{l|l} + \textsc{Ship} &\textsc{Book} \\ \hline + \textit{HMS Sophie} &Master and Commander \\ + \textit{HMS Polychrest} &Post Captain \\ + \textit{HMS Lively} &Post Captain \\ + \textit{HMS Surprise} &A number of books\footnote{% + Starting with \textit{HMS Surprise}.} + \end{tabular} + \end{minipage} + \end{center} + + Inside a 'minipage', footnote marks are lowercase letters. Change +that with something like +'\renewcommand{\thempfootnote}{\arabic{mpfootnote}}' (*note \alph \Alph +\arabic \roman \Roman \fnsymbol::). + + The footnotes in the prior example appear at the bottom of the +'minipage'. To have them appear at the bottom of the main page, as part +of the regular footnote sequence, use the '\footnotemark' and +'\footnotetext' pair and make a new counter. + + \newcounter{mpFootnoteValueSaver} + \begin{center} + \begin{minipage}{\textwidth} + \setcounter{mpFootnoteValueSaver}{\value{footnote}} \centering + \begin{tabular}{l|l} + \textsc{Woman} &\textsc{Relationship} \\ \hline + Mona &Attached\footnotemark \\ + Diana Villiers &Eventual wife \\ + Christine Hatherleigh Wood &Fiance\footnotemark + \end{tabular} + \end{minipage}% percent sign keeps footnote text close to minipage + \stepcounter{mpFootnoteValueSaver}% + \footnotetext[\value{mpFootnoteValueSaver}]{% + Little is known other than her death.}% + \stepcounter{mpFootnoteValueSaver}% + \footnotetext[\value{mpFootnoteValueSaver}]{% + Relationship is unresolved in XXI.} + \end{center} + + For a floating 'table' environment (*note table::), use the +'tablefootnote' package. + + \usepackage{tablefootnote} % in preamble + ... + \begin{table} + \centering + \begin{tabular}{l|l} + \textsc{Date} &\textsc{Campaign} \\ \hline + 1862 &Fort Donelson \\ + 1863 &Vicksburg \\ + 1865 &Army of Northern Virginia\tablefootnote{% + Ending the war.} + \end{tabular} + \caption{Forces captured by US Grant} + \end{table} + +The footnote appears at the page bottom and is numbered in sequence with +other footnotes. + + +File: latex2e.info, Node: Footnotes of footnotes, Prev: Footnotes in a table, Up: Footnotes + +11.6 Footnotes of footnotes +=========================== + +Particularly in the humanities, authors can have multiple classes of +footnotes, including having footnotes of footnotes. The package +'bigfoot' extends LaTeX's default footnote mechanism in many ways, +including allow these two, as in this example. + + \usepackage{bigfoot} % in preamble + \DeclareNewFootnote{Default} + \DeclareNewFootnote{from}[alph] % create class \footnotefrom{} + ... + The third theorem is a partial converse of the + second.\footnotefrom{% + Noted in Wilson.\footnote{Second edition only.}} + + +File: latex2e.info, Node: Definitions, Next: Counters, Prev: Footnotes, Up: Top + +12 Definitions +************** + +LaTeX has support for making new commands of many different kinds. + +* Menu: + +* \newcommand & \renewcommand:: (Re)define a new command. +* \providecommand:: Define a new command, if name not used. +* \makeatletter & \makeatother:: Change the status of the at-sign character. +* \@ifstar:: Define your own commands with *-variants. +* \newcounter:: Define a new counter. +* \newlength:: Define a new length. +* \newsavebox:: Define a new box. +* \newenvironment & \renewenvironment:: Define a new environment. +* \newtheorem:: Define a new theorem-like environment. +* \newfont:: Define a new font name. +* \protect:: Using tricky commands. +* \ignorespaces & \ignorespacesafterend:: Discard extra spaces. +* xspace package:: Space after a macro, conditionally. + + +File: latex2e.info, Node: \newcommand & \renewcommand, Next: \providecommand, Up: Definitions + +12.1 '\newcommand' & '\renewcommand' +==================================== + +Synopses, one of: + + \newcommand{\CMD}{DEFN} + \newcommand{\CMD}[NARGS]{DEFN} + \newcommand{\CMD}[NARGS][OPTARGDEFAULT]{DEFN} + \newcommand*{\CMD}{DEFN} + \newcommand*{\CMD}[NARGS]{DEFN} + \newcommand*{\CMD}[NARGS][OPTARGDEFAULT]{DEFN} + +or one of these. + + \renewcommand{\CMD}[NARGS]{DEFN} + \renewcommand{\CMD}[NARGS]{DEFN} + \renewcommand{\CMD}[NARGS][OPTARGDEFAULT]{DEFN} + \renewcommand*{\CMD}{DEFN} + \renewcommand*{\CMD}[NARGS]{DEFN} + \renewcommand*{\CMD}[NARGS][OPTARGDEFAULT]{DEFN} + + Define or redefine a command (see also the discussion of +'\DeclareRobustCommand' in *note Class and package commands::). + + The starred form of these two forbids the arguments from containing +multiple paragraphs of text (in plain TeX terms, the commands are not +'\long'). With the default form, arguments can be multiple paragraphs. + + These are the parameters: + +CMD + + Required; '\CMD' is the command name. It must begin with a + backslash, '\', and must not begin with the four character string + '\end'. For '\newcommand', it must not be already defined. For + '\renewcommand', this name must already be defined. + +NARGS + Optional; an integer from 0 to 9, specifying the number of + arguments that the command takes, including any optional argument. + Omitting this argument is the same as specifying 0, meaning that + the command has no arguments. If you redefine a command, the new + version can have a different number of arguments than the old + version. + +OPTARGDEFAULT + Optional; if this argument is present then the first argument of + '\CMD' is optional, with default value OPTARGDEFAULT (which may be + the empty string). If OPTARGSDEFAULT is not present then '\CMD' + does not take an optional argument. + + That is, if '\CMD' is used with square brackets, as in + '\CMD[OPTVAL]{...}...', then within DEFN the parameter '#1' is set + to the value of OPTVAL. On the other hand, if '\CMD' is called + without the square brackets then within DEFN the parameter '#1' is + set to the value of OPTARGDEFAULT. In either case, the required + arguments start with '#2'. + + Omitting '[OPTARGDEFAULT]' is different from having the square + brackets with no contents, as in '[]'. The former sets '#1' to the + value of OPTARGDEFAULT; the latter sets '#1' to the empty string. + +DEFN + Required; the text to be substituted for every occurrence of + '\CMD'. The parameters '#1', '#2', ... '#NARGS' are replaced by + the values that you supply when you call the command (or by the + default value if there is an optional argument and you don't + exercise the option). + + TeX ignores blanks in the source following a control word (*note +Control sequences::), as in '\cmd '. If you actually want a space +there, one solution is to type '{}' after the command ('\cmd{} ', and +another solution is to use an explicit control space ('\cmd\ '). + + A simple example of defining a new command: '\newcommand{\RS}{Robin +Smith}' results in '\RS' being replaced by the longer text. Redefining +an existing command is similar: '\renewcommand{\qedsymbol}{{\small +QED}}'. + + If you try to define a command and the name has already been used +then you get something like 'LaTeX Error: Command \fred already defined. +Or name \end... illegal, see p.192 of the manual'. If you try to +redefine a command and the name has not yet been used then you get +something like 'LaTeX Error: \hank undefined'. + + Here the first definition creates a command with no arguments, and +the second, one with one required argument. + + \newcommand{\student}{Ms~O'Leary} + \newcommand{\defref}[1]{Definition~\ref{#1}} + +Use the first as in 'I highly recommend \student{} to you'. The second +has a variable argument, so that '\defref{def:basis}' expands to +'Definition~\ref{def:basis}', which ultimately expands to something like +'Definition~3.14'. + + Similarly, but with two required arguments: +'\newcommand{\nbym}[2]{$#1 \times #2$}' is invoked as '\nbym{2}{k}'. + + This example has an optional argument. + + \newcommand{\salutation}[1][Sir or Madam]{Dear #1:} + +Then '\salutation' gives 'Dear Sir or Madam:' while '\salutation[John]' +gives 'Dear John:'. And '\salutation[]' gives 'Dear :'. + + This example has an optional argument and two required arguments. + + \newcommand{\lawyers}[3][company]{#2, #3, and~#1} + I employ \lawyers[Howe]{Dewey}{Cheatem}. + +The output is 'I employ Dewey, Cheatem, and Howe'. The optional +argument, the 'Howe', is associated with '#1', while 'Dewey' and +'Cheatem' are associated with '#2' and '#3'. Because of the optional +argument, '\lawyers{Dewey}{Cheatem}' will give the output 'I employ +Dewey, Cheatem, and company'. + + The braces around DEFN do not define a group, that is, they do not +delimit the scope of the result of expanding DEFN. For example, with +'\newcommand{\shipname}[1]{\it #1}', in this sentence, + + The \shipname{Monitor} met the \shipname{Merrimac}. + +the words 'met the' would incorrectly be in italics. The solution is to +put another pair of braces inside the definition: +'\newcommand{\shipname}[1]{{\it #1}}'. + +* Menu: + +* Control sequences:: Control sequence, control word and control symbol. + + +File: latex2e.info, Node: Control sequences, Up: \newcommand & \renewcommand + +12.1.1 Control sequence, control word and control symbol +-------------------------------------------------------- + +When reading input TeX converts the sequences of read characters into a +sequence of "tokens". When TeX sees a backslash '\', it will handle the +following characters in a special way in order to make a "control +sequence" token. + + The control sequences fall into two categories: + + * "control word", when the control sequence is gathered from a '\' + followed by at least one ASCII letter, followed by at least one + blank. The sequence of at least one ASCII letter is called the + "control sequence name". + * "control symbol", when the control sequence is gathered from a '\' + followed by one non-letter character. + + Blanks after a control word are ignored and do not produce any +whitespace in the output (*note \newcommand & \renewcommand:: and *note +\(SPACE)::). + + Just as the '\relax' command does nothing, the following will print +'Hello!'--if you use the Emacs info viewer, turn on the whitespace-mode +minor mode to see the trailing spaces: + + Hel\relax + lo! + +This is because blanks after '\relax', including the newline are +ignored, and blanks at the beginning of a line are also ignored (*note +Leading blanks::). + + +File: latex2e.info, Node: \providecommand, Next: \makeatletter & \makeatother, Prev: \newcommand & \renewcommand, Up: Definitions + +12.2 '\providecommand' +====================== + +Synopses, one of: + + \providecommand{\CMD}{DEFN} + \providecommand{\CMD}[NARGS]{DEFN} + \providecommand{\CMD}[NARGS][OPTARGDEFAULT]{DEFN} + \providecommand*{\CMD}{DEFN} + \providecommand*{\CMD}[NARGS]{DEFN} + \providecommand*{\CMD}[NARGS][OPTARGDEFAULT]{DEFN} + + Defines a command, as long as no command of this name already exists. +If no command of this name already exists then this has the same effect +as '\newcommand'. If a command of this name already exists then this +definition does nothing. This is particularly useful in a file that may +be loaded more than once, such as a style file. *Note \newcommand & +\renewcommand::, for the description of the arguments. + + This example + + \providecommand{\myaffiliation}{Saint Michael's College} + \providecommand{\myaffiliation}{Lyc\'ee Henri IV} + From \myaffiliation. + +outputs 'From Saint Michael's College'. Unlike '\newcommand', the +repeated use of '\providecommand' does not give an error. + + +File: latex2e.info, Node: \makeatletter & \makeatother, Next: \@ifstar, Prev: \providecommand, Up: Definitions + +12.3 '\makeatletter' & '\makeatother' +===================================== + +Synopsis: + + \makeatletter + ... definition of commands with @ in their name .. + \makeatother + + Use this pair when you redefine LaTeX commands that are named with an +at-sign character ''@''. The '\makeatletter' declaration makes the +at-sign character have the category code of a letter, code 11. The +'\makeatother' declaration sets the category code of the at-sign to +code 12, its default value. + + As TeX reads characters, it assigns each one a category code, or +"catcode". For instance, it assigns the backslash character ''\'' the +catcode 0. Command names consist of a category 0 character, ordinarily +backslash, followed by letters, category 11 characters (except that a +command name can also consist of a category 0 character followed by a +single non-letter symbol). + + LaTeX's source code has the convention that some commands use '@' in +their name. These commands are mainly intended for package or class +writers. The convention prevents authors who are just using a package +or class from accidentally replacing such a command with one of their +own, because by default the at-sign has catcode 12. + + Use the pair '\makeatletter' and '\makeatother' inside a '.tex' file, +typically in the preamble, when you are defining or redefining commands +named with '@', by having them surround your definition. Don't use +these inside '.sty' or '.cls' files since the '\usepackage' and +'\documentclass' commands already arrange that the at-sign has the +character code of a letter, catcode 11. + + For a comprehensive list of macros with an at-sign in their names see +. + + In this example the class file has a command '\thesis@universityname' +that the user wants to change. These three lines should go in the +preamble, before the '\begin{document}'. + + \makeatletter + \renewcommand{\thesis@universityname}{Saint Michael's College} + \makeatother + + +File: latex2e.info, Node: \@ifstar, Next: \newcounter, Prev: \makeatletter & \makeatother, Up: Definitions + +12.4 '\@ifstar' +=============== + +Synopsis: + + \newcommand{\mycmd}{\@ifstar{\mycmd@star}{\mycmd@nostar}} + \newcommand{\mycmd@nostar}[NOSTAR-NUM-ARGS]{NOSTAR-BODY} + \newcommand{\mycmd@star}[STAR-NUM-ARGS]{STAR-BODY} + + Many standard LaTeX environments or commands have a variant with the +same name but ending with a star character '*', an asterisk. Examples +are the 'table' and 'table*' environments and the '\section' and +'\section*' commands. + + When defining environments, following this pattern is straightforward +because '\newenvironment' and '\renewenvironment' allow the environment +name to contain a star. So you just have to write +'\newenvironment{MYENV}' or '\newenvironment{MYENV*}' and continue the +definition as usual. For commands the situation is more complex as the +star not being a letter cannot be part of the command name. As in the +synopsis above, there will be a user-called command, given above as +'\mycmd', which peeks ahead to see if it is followed by a star. For +instance, LaTeX does not really have a '\section*' command; instead, the +'\section' command peeks ahead. This command does not accept arguments +but instead expands to one of two commands that do accept arguments. In +the synopsis these two are '\mycmd@nostar' and '\mycmd@star'. They +could take the same number of arguments or a different number, or no +arguments at all. As always, in a LaTeX document a command using an +at-sign '@' in its name must be enclosed inside a '\makeatletter ... +\makeatother' block (*note \makeatletter & \makeatother::). + + This example of '\@ifstar' defines the command '\ciel' and a variant +'\ciel*'. Both have one required argument. A call to '\ciel{blue}' +will return "not starry blue sky" while '\ciel*{night}' will return +"starry night sky". + + \makeatletter + \newcommand*{\ciel@unstarred}[1]{not starry #1 sky} + \newcommand*{\ciel@starred}[1]{starry #1 sky} + \newcommand*{\ciel}{\@ifstar{\ciel@starred}{\ciel@unstarred}} + \makeatother + + In the next example, the starred variant takes a different number of +arguments than the unstarred one. With this definition, Agent 007's +'``My name is \agentsecret*{Bond}, \agentsecret{James}{Bond}.''' is +equivalent to entering the commands '``My name is \textsc{Bond}, +\textit{James} textsc{Bond}.''' + + \newcommand*{\agentsecret@unstarred}[2]{\textit{#1} \textsc{#2}} + \newcommand*{\agentsecret@starred}[1]{\textsc{#1}} + \newcommand*{\agentsecret}{% + \@ifstar{\agentsecret@starred}{\agentsecret@unstarred}} + + After a command name, a star is handled similarly to an optional +argument. (This differs from environment names in which the star is +part of the name itself and as such could be in any position.) Thus, it +is technically possible to put any number of spaces between the command +and the star. Thus '\agentsecret*{Bond}' and '\agentsecret *{Bond}' are +equivalent. However, the standard practice is not to insert any such +spaces. + + There are two alternative ways to accomplish the work of '\@ifstar'. +(1) The 'suffix' package allows the construct +'\newcommand\mycommand{UNSTARRED-VARIANT}' followed by +'\WithSuffix\newcommand\mycommand*{STARRED-VARIANT}'. (2) LaTeX +provides the 'xparse' package, which allows this code: + + \NewDocumentCommand\foo{s}{\IfBooleanTF#1 + {STARRED-VARIANT}% + {UNSTARRED-VARIANT}% + } + + +File: latex2e.info, Node: \newcounter, Next: \newlength, Prev: \@ifstar, Up: Definitions + +12.5 '\newcounter': Allocating a counter +======================================== + +Synopsis, one of: + + \newcounter{COUNTERNAME} + \newcounter{COUNTERNAME}[SUPERCOUNTER] + + Globally defines a new counter named COUNTERNAME and initialize it to +zero (*note Counters::). + + The name COUNTERNAME must consist of letters only. It does not begin +with a backslash. This name must not already be in use by another +counter. + + When you use the optional argument '[SUPERCOUNTER]' then the counter +COUNTERNAME will be reset to zero whenever SUPERCOUNTER is incremented. +For example, ordinarily 'subsection' is numbered within 'section' so +that any time you increment SECTION, either with '\stepcounter' (*note +\stepcounter::) or '\refstepcounter' (*note \refstepcounter::), then +LaTeX will reset SUBSECTION to zero. + + This example + + \newcounter{asuper} \setcounter{asuper}{1} + \newcounter{asub}[asuper] \setcounter{asub}{3} % Note `asuper' + The value of asuper is \arabic{asuper} and of asub is \arabic{asub}. + \stepcounter{asuper} + Now asuper is \arabic{asuper} while asub is \arabic{asub}. + + produces 'The value of asuper is 1 and that of asub is 3' and 'Now +asuper is 2 while asub is 0'. + + If the counter already exists, for instance by entering 'asuper' +twice, then you get something like 'LaTeX Error: Command \c@asuper +already defined. Or name \end... illegal, see p.192 of the manual.'. + + If you use the optional argument then the super counter must already +exist. Entering '\newcounter{jh}[lh]' when 'lh' is not a defined +counter will get you 'LaTeX Error: No counter 'lh' defined.' + + +File: latex2e.info, Node: \newlength, Next: \newsavebox, Prev: \newcounter, Up: Definitions + +12.6 '\newlength' +================= + +Synopsis: + + \newlength{\LEN} + + Allocate a new length register (*note Lengths::). The required +argument '\LEN' has to be a control sequence (*note Control +sequences::), and as such must begin with a backslash, '\' under normal +circumstances. The new register holds rubber lengths such as '72.27pt' +or '1in plus.2in minus.1in' (a LaTeX length register is what plain TeX +calls a 'skip' register). The initial value is zero. The control +sequence '\LEN' must not be already defined. + + An example: + + \newlength{\graphichgt} + + If you forget the backslash then you get 'Missing control sequence +inserted'. If the control sequence already exists then you get +something like 'LaTeX Error: Command \graphichgt already defined. Or +name \end... illegal, see p.192 of the manual'. + + +File: latex2e.info, Node: \newsavebox, Next: \newenvironment & \renewenvironment, Prev: \newlength, Up: Definitions + +12.7 '\newsavebox' +================== + +Synopsis: + + \newsavebox{\CMD} + + Define \CMD, the string consisting of a backslash followed by CMD, to +refer to a new bin for storing material. These bins hold material that +has been typeset, to use multiple times or to measure or manipulate +(*note Boxes::). The bin name \CMD is required, must start with a +backslash, \, and must not already be a defined command. This command +is fragile (*note \protect::). + + This allocates a bin and then puts typeset material into it. + + \newsavebox{\logobox} + \savebox{\logobox}{LoGo} + Our logo is \usebox{\logobox}. + +The output is 'Our logo is LoGo'. + + If there is an already defined bin then you get something like 'LaTeX +Error: Command \logobox already defined. Or name \end... illegal, see +p.192 of the manual'. + + The allocation of a box is global. + + +File: latex2e.info, Node: \newenvironment & \renewenvironment, Next: \newtheorem, Prev: \newsavebox, Up: Definitions + +12.8 '\newenvironment' & '\renewenvironment' +============================================ + +Synopses, one of: + + \newenvironment{ENV}{BEGDEF}{ENDDEF} + \newenvironment{ENV}[NARGS]{BEGDEF}{ENDDEF} + \newenvironment{ENV}[NARGS][OPTARGDEFAULT]{BEGDEF}{ENDDEF} + \newenvironment*{ENV}{BEGDEF}{ENDDEF} + \newenvironment*{ENV}[NARGS]{BEGDEF}{ENDDEF} + \newenvironment*{ENV}[NARGS][OPTARGDEFAULT]{BEGDEF}{ENDDEF} + +or one of these. + + \renewenvironment{ENV}{BEGDEF}{ENDDEF} + \renewenvironment{ENV}[NARGS]{BEGDEF}{ENDDEF} + \renewenvironment{ENV}[NARGS][OPTARGDEFAULT]{BEGDEF}{ENDDEF} + \renewenvironment*{ENV}{BEGDEF}{ENDDEF} + \renewenvironment*{ENV}[NARGS]{BEGDEF}{ENDDEF} + \renewenvironment*{ENV}[NARGS][OPTARGDEFAULT]{BEGDEF}{ENDDEF} + + Define or redefine the environment ENV, that is, create the construct +'\begin{ENV} ... BODY ... \end{ENV}'. + + The starred form of these commands requires that the arguments not +contain multiple paragraphs of text. However, the body of these +environments can contain multiple paragraphs. + +ENV + Required; the environment name. It consists only of letters or the + '*' character, and thus does not begin with backslash, '\'. It + must not begin with the string 'end'. For '\newenvironment', the + name ENV must not be the name of an already existing environment, + and also the command '\ENV' must be undefined. For + '\renewenvironment', ENV must be the name of an existing + environment. + +NARGS + Optional; an integer from 0 to 9 denoting the number of arguments + of that the environment takes. When you use the environment these + arguments appear after the '\begin', as in '\begin{ENV}{ARG1} ... + {ARGN}'. Omitting this is equivalent to setting it to 0; the + environment will have no arguments. When redefining an + environment, the new version can have a different number of + arguments than the old version. + +OPTARGDEFAULT + Optional; if this is present then the first argument of the defined + environment is optional, with default value OPTARGDEFAULT (which + may be the empty string). If this is not in the definition then + the environment does not take an optional argument. + + That is, when OPTARGDEFAULT is present in the definition of the + environment then you can start the environment with square + brackets, as in '\begin{ENV}[OPTVAL]{...} ... \end{ENV}'. In this + case, within BEGDEFN the parameter '#1' is set to the value of + OPTVAL. If you call '\begin{ENV}' without square brackets, then + within BEGDEFN the parameter '#1' is set to the value of the + default OPTARGDEFAULT. In either case, any required arguments + start with '#2'. + + Omitting '[MYVAL]' in the call is different than having the square + brackets with no contents, as in '[]'. The former results in '#1' + expanding to OPTARGDEFAULT; the latter results in '#1' expanding to + the empty string. + +BEGDEF + Required; the text expanded at every occurrence of '\begin{ENV}'. + Within BEGDEF, the parameters '#1', '#2', ... '#NARGS', are + replaced by the values that you supply when you call the + environment; see the examples below. + +ENDDEF + Required; the text expanded at every occurrence of '\end{ENV}'. + This may not contain any parameters, that is, you cannot use '#1', + '#2', etc., here (but see the final example below). + + All environments, that is to say the BEGDEF code, the environment +body, and the ENDDEF code, are processed within a group. Thus, in the +first example below, the effect of the '\small' is limited to the quote +and does not extend to material following the environment. + + If you try to define an environment and the name has already been +used then you get something like 'LaTeX Error: Command \fred already +defined. Or name \end... illegal, see p.192 of the manual'. If you try +to redefine an environment and the name has not yet been used then you +get something like 'LaTeX Error: Environment hank undefined.'. + + This example gives an environment like LaTeX's 'quotation' except +that it will be set in smaller type. + + \newenvironment{smallquote}{% + \small\begin{quotation} + }{% + \end{quotation} + } + + This has an argument, which is set in boldface at the start of a +paragraph. + + \newenvironment{point}[1]{% + \noindent\textbf{#1} + }{% + } + + This one shows the use of a optional argument; it gives a quotation +environment that cites the author. + + \newenvironment{citequote}[1][Shakespeare]{% + \begin{quotation} + \noindent\textit{#1}: + }{% + \end{quotation} + } + +The author's name is optional, and defaults to 'Shakespeare'. In the +document, use the environment like this. + + \begin{citequote}[Lincoln] + ... + \end{citequote} + + The final example shows how to save the value of an argument to use +in ENDDEF, in this case in a box (*note \sbox & \savebox::). + + \newsavebox{\quoteauthor} + \newenvironment{citequote}[1][Shakespeare]{% + \sbox\quoteauthor{#1}% + \begin{quotation} + }{% + \hspace{1em plus 1fill}---\usebox{\quoteauthor} + \end{quotation} + } + + +File: latex2e.info, Node: \newtheorem, Next: \newfont, Prev: \newenvironment & \renewenvironment, Up: Definitions + +12.9 '\newtheorem' +================== + +Synopses: + + \newtheorem{NAME}{TITLE} + \newtheorem{NAME}{TITLE}[NUMBERED_WITHIN] + \newtheorem{NAME}[NUMBERED_LIKE]{TITLE} + + Define a new theorem-like environment. You can specify one of +NUMBERED_WITHIN and NUMBERED_LIKE, or neither, but not both. + + The first form, '\newtheorem{NAME}{TITLE}', creates an environment +that will be labelled with TITLE; see the first example below. + + The second form, '\newtheorem{NAME}{TITLE}[NUMBERED_WITHIN]', creates +an environment whose counter is subordinate to the existing counter +NUMBERED_WITHIN, so this counter will be reset when NUMBERED_WITHIN is +reset. See the second example below. + + The third form '\newtheorem{NAME}[NUMBERED_LIKE]{TITLE}', with +optional argument between the two required arguments, creates an +environment whose counter will share the previously defined counter +NUMBERED_LIKE. See the third example. + + This command creates a counter named NAME. In addition, unless the +optional argument NUMBERED_LIKE is used, inside of the theorem-like +environment the current '\ref' value will be that of +'\theNUMBERED_WITHIN' (*note \ref::). + + This declaration is global. It is fragile (*note \protect::). + + Arguments: + +NAME + The name of the environment. It is a string of letters. It must + not begin with a backslash, '\'. It must not be the name of an + existing environment, and the command name '\NAME' must not already + be defined. + +TITLE + The text to be printed at the beginning of the environment, before + the number. For example, 'Theorem'. + +NUMBERED_WITHIN + Optional; the name of an already defined counter, usually a + sectional unit such as 'chapter' or 'section'. When the + NUMBERED_WITHIN counter is reset then the NAME environment's + counter will also be reset. + + If this optional argument is not used then the command '\theNAME' + is set to '\arabic{NAME}'. + +NUMBERED_LIKE + Optional; the name of an already defined theorem-like environment. + The new environment will be numbered in sequence with + NUMBERED_LIKE. + + Without any optional arguments the environments are numbered +sequentially. The example below has a declaration in the preamble that +results in 'Definition 1' and 'Definition 2' in the output. + + \newtheorem{defn}{Definition} + \begin{document} + \section{...} + \begin{defn} + First def + \end{defn} + + \section{...} + \begin{defn} + Second def + \end{defn} + + This example has the same document body as the prior one. But here +'\newtheorem''s optional argument NUMBERED_WITHIN is given as 'section', +so the output is like 'Definition 1.1' and 'Definition 2.1'. + + \newtheorem{defn}{Definition}[section] + \begin{document} + \section{...} + \begin{defn} + First def + \end{defn} + + \section{...} + \begin{defn} + Second def + \end{defn} + + In the next example there are two declarations in the preamble, the +second of which calls for the new 'thm' environment to use the same +counter as 'defn'. It gives 'Definition 1.1', followed by 'Theorem 2.1' +and 'Definition 2.2'. + + \newtheorem{defn}{Definition}[section] + \newtheorem{thm}[defn]{Theorem} + \begin{document} + \section{...} + \begin{defn} + First def + \end{defn} + + \section{...} + \begin{thm} + First thm + \end{thm} + + \begin{defn} + Second def + \end{defn} + + +File: latex2e.info, Node: \newfont, Next: \protect, Prev: \newtheorem, Up: Definitions + +12.10 '\newfont' +================ + +This command is obsolete. This description is here only to help with +old documents. New documents should define fonts in families through +the New Font Selection Scheme which allows you to, for example, +associate a boldface with a roman (*note Fonts::). + + Synopsis: + + \newfont{\CMD}{FONT DESCRIPTION} + + Define a command '\CMD' that will change the current font. The +control sequence must not already be defined. It must begin with a +backslash, '\'. + + The FONT DESCRIPTION consists of a FONTNAME and an optional "at +clause". LaTeX will look on your system for a file named +'FONTNAME.tfm'. The at clause can have the form either 'at DIMEN' or +'scaled FACTOR', where a FACTOR of '1000' means no scaling. For LaTeX's +purposes, all this does is scale all the character and other font +dimensions relative to the font's design size, which is a value defined +in the '.tfm' file. + + This defines two equivalent fonts and typesets a few characters in +each. + + \newfont{\testfontat}{cmb10 at 11pt} + \newfont{\testfontscaled}{cmb10 scaled 1100} + \testfontat abc + \testfontscaled abc + + +File: latex2e.info, Node: \protect, Next: \ignorespaces & \ignorespacesafterend, Prev: \newfont, Up: Definitions + +12.11 '\protect' +================ + +All LaTeX commands are either "fragile" or "robust". A fragile command +can break when it is used in the argument to certain other commands. +Commands that contain data that LaTeX writes to an auxiliary file and +re-reads later are fragile. This includes material that goes into a +table of contents, list of figures, list of tables, etc. Fragile +commands also include line breaks, any command that has an optional +argument, and many more. To prevent such commands from breaking, one +solution is to preceded them with the command '\protect'. + + For example, when LaTeX runs the '\section{SECTION NAME}' command it +writes the SECTION NAME text to the '.aux' auxiliary file, moving it +there for use elsewhere in the document such as in the table of +contents. Any argument that is internally expanded by LaTeX without +typesetting it directly is referred to as a "moving argument". A +command is fragile if it can expand during this process into invalid TeX +code. Some examples of moving arguments are those that appear in the +'\caption{...}' command (*note figure::), in the '\thanks{...}' command +(*note \maketitle::), and in @-expressions in the 'tabular' and 'array' +environments (*note tabular::). + + If you get strange errors from commands used in moving arguments, try +preceding it with '\protect'. Every fragile commands must be protected +with their own '\protect'. + + Although usually a '\protect' command doesn't hurt, length commands +are robust and should not be preceded by a '\protect' command. Nor can +a '\protect' command be used in the argument to '\addtocounter' or +'\setcounter' command. + + In this example the '\caption' command gives a mysterious error about +an extra curly brace. Fix the problem by preceding each '\raisebox' +command with '\protect'. + + \begin{figure} + ... + \caption{Company headquarters of A\raisebox{1pt}{B}\raisebox{-1pt}{C}} + \end{figure} + + In the next example the '\tableofcontents' command gives an error +because the '\(..\)' in the section title expands to illegal TeX in the +'.toc' file. You can solve this by changing '\(..\)' to +'\protect\(..\protect\)'. + + \begin{document} + \tableofcontents + ... + \section{Einstein's \( e=mc^2 \)} + ... + + +File: latex2e.info, Node: \ignorespaces & \ignorespacesafterend, Next: xspace package, Prev: \protect, Up: Definitions + +12.12 '\ignorespaces & \ignorespacesafterend' +============================================= + +Synopsis: + + \ignorespaces + +or + + \ignorespacesafterend + + Both commands cause LaTeX to ignore blanks (that is, characters of +catcode 10 such as space or tabulation) after the end of the command up +to the first box or non-blank character. The first is a command from +plain TeX, and the second is LaTeX-specific. + + The '\ignorespaces' is often used when defining commands via +'\newcommand', or '\newenvironment', or '\def'. The example below +illustrates. It allows a user to show the points values for quiz +questions in the margin but it is inconvenient because, as shown in the +'enumerate' list, users must not put any space between the command and +the question text. + + \newcommand{\points}[1]{\makebox[0pt]{\makebox[10em][l]{#1~pts}} + \begin{enumerate} + \item\points{10}no extra space output here + \item\points{15} extra space between the number and the `extra' + \end{enumerate} + +The solution is to change to this. + + \newcommand{\points}[1]{% + \makebox[0pt]{\makebox[10em][l]{#1~pts}}\ignorespaces} + + A second example shows blanks being removed from the front of text. +The commands below allow a user to uniformly attach a title to names. +But, as given, if a title accidentally starts with a space then +'\fullname' will reproduce that. + + \newcommand{\honorific}[1]{\def\honorific{#1}} % remember title + \newcommand{\fullname}[1]{\honorific~#1} % put title before name + + \begin{tabular}{|l|} + \honorific{Mr/Ms} \fullname{Jones} \\ % no extra space here + \honorific{ Mr/Ms} \fullname{Jones} % extra space before title + \end{tabular} + +To fix this, change to +'\newcommand{\fullname}[1]{\ignorespaces\honorific~#1}'. + + The '\ignorespaces' is also often used in a '\newenvironment' at the +end of the BEGIN clause, as in '\begin{newenvironment}{ENV NAME}{... +\ignorespaces}{...}'. + + To strip blanks off the end of an environment use +'\ignorespacesafterend'. An example is that this will show a much +larger vertical space between the first and second environments than +between the second and third. + + \newenvironment{eq}{\begin{equation}}{\end{equation}} + \begin{eq} + e=mc^2 + \end{eq} + \begin{equation} + F=ma + \end{equation} + \begin{equation} + E=IR + \end{equation} + + Putting a comment character '%' immediately after the '\end{eq}' will +make the vertical space disappear, but that is inconvenient. The +solution is to change to +'\newenvironment{eq}{\begin{equation}}{\end{equation}\ignorespacesafterend}'. + + +File: latex2e.info, Node: xspace package, Prev: \ignorespaces & \ignorespacesafterend, Up: Definitions + +12.13 xspace package +==================== + +Synopsis: + + \usepackage{xspace} + ... + \newcommand{...}{...\xspace} + + The '\xspace' macro, when used at the end of a command definition, +adds a space unless the command is followed by certain punctuation +characters. + + After a command control sequence that is a control word (*note +Control sequences::, as opposed to control symbols such as '\$'), TeX +gobbles blank characters. Thus, in the first sentence below, the output +has 'Vermont' placed snugly against the period, without any intervening +space. + + \newcommand{\VT}{Vermont} + Our college is in \VT . + The \VT{} summers are nice. + + But because of the gobbling, the second sentence needs the empty +curly braces or else there would be no space separating 'Vermont' from +'summers'. (Many authors instead use a backslash-space '\ ' for this. +*Note \(SPACE)::.) + + The 'xspace' package provides '\xspace'. It is for writing commands +which are designed to be used mainly in text. It must be place at the +very end of the definition of these commands. It inserts a space after +that command unless what immediately follows is in a list of exceptions. +In this example, the empty braces are not needed. + + \newcommand{\VT}{Vermont\xspace} + Our college is in \VT . + \VT summers are nice. + + The default exception list contains the characters ',.'/?;:!~-)', the +open curly brace and the backslash-space command discussed above, and +the commands '\footnote' or '\footnotemark'. Add to that list as with +'\xspaceaddexceptions{\myfni \myfnii}' which adds '\myfni' and '\myfnii' +to the list, and remove from that list as with +'\xspaceremoveexception{!}'. + + A comment: many experts prefer not to use '\xspace'. Putting it in a +definition means that the command will usually get the spacing right. +But it isn't easy to predict when to enter empty braces because +'\xspace' will get it wrong, such as when it is followed by another +command, and so '\xspace' can make editing material harder and more +error-prone than instead always inserting the empty braces. + + +File: latex2e.info, Node: Counters, Next: Lengths, Prev: Definitions, Up: Top + +13 Counters +*********** + +Everything LaTeX numbers for you has a counter associated with it. The +name of the counter is often the same as the name of the environment or +command associated with the number, except that the counter's name has +no backslash '\'. Thus, associated with the '\chapter' command is the +'chapter' counter that keeps track of the chapter number. + + Below is a list of the counters used in LaTeX's standard document +classes to control numbering. + + part paragraph figure enumi + chapter subparagraph table enumii + section page footnote enumiii + subsection equation mpfootnote enumiv + subsubsection + + The 'mpfootnote' counter is used by the '\footnote' command inside of +a minipage (*note minipage::). The counters 'enumi' through 'enumiv' +are used in the 'enumerate' environment, for up to four levels of +nesting (*note enumerate::). + + Counters can have any integer value but they are typically positive. + + New counters are created with '\newcounter'. *Note \newcounter::. + +* Menu: + +* \alph \Alph \arabic \roman \Roman \fnsymbol:: Print value of a counter. +* \usecounter:: Use a specified counter in a list environment. +* \value:: Use the value of a counter in an expression. +* \setcounter:: Set the value of a counter. +* \addtocounter:: Add a quantity to a counter. +* \refstepcounter:: Add to a counter. +* \stepcounter:: Add to a counter, resetting subsidiary counters. +* \day & \month & \year:: Numeric date values. + + +File: latex2e.info, Node: \alph \Alph \arabic \roman \Roman \fnsymbol, Next: \usecounter, Up: Counters + +13.1 '\alph \Alph \arabic \roman \Roman \fnsymbol': Printing counters +===================================================================== + +Print the value of a counter, in a specified style. For instance, if +the counter COUNTER has the value 1 then a '\alph{COUNTER}' in your +source will result in a lowercase letter a appearing in the output. + + All of these commands take a single counter as an argument, for +instance, '\alph{enumi}'. Note that the counter name does not start +with a backslash. + +'\alph{COUNTER}' + Print the value of COUNTER in lowercase letters: 'a', 'b', ... If + the counter's value is less than 1 or more than 26 then you get + 'LaTeX Error: Counter too large.' + +'\Alph{COUNTER}' + Print in uppercase letters: 'A', 'B', ... If the counter's value + is less than 1 or more than 26 then you get 'LaTeX Error: Counter + too large.' + +'\arabic{COUNTER}' + Print in Arabic numbers such as '5' or '-2'. + +'\roman{COUNTER}' + Print in lowercase roman numerals: 'i', 'ii', ... If the counter's + value is less than 1 then you get no warning or error but LaTeX + does not print anything in the output. + +'\Roman{COUNTER}' + Print in uppercase roman numerals: 'I', 'II', ... If the counter's + value is less than 1 then you get no warning or error but LaTeX + does not print anything in the output. + +'\fnsymbol{COUNTER}' + Prints the value of COUNTER using a sequence of nine symbols that + are traditionally used for labeling footnotes. The value of + COUNTER should be between 1 and 9, inclusive. If the counter's + value is less than 0 or more than 9 then you get 'LaTeX Error: + Counter too large', while if it is 0 then you get no error or + warning but LaTeX does not output anything. + + Here are the symbols: + + Number Name Command Symbol + ----------------------------------------------------------------------------- + 1 asterisk '\ast' * + 2 dagger '\dagger' † + 3 ddagger '\ddagger' ‡ + 4 section-sign '\S' § + 5 paragraph-sign '\P' ¶ + 6 double-vert '\parallel' ‖ + 7 double-asterisk '\ast\ast' ** + 8 double-dagger '\dagger\dagger' †† + 9 double-ddagger '\ddagger\ddagger' ‡‡ + + +File: latex2e.info, Node: \usecounter, Next: \value, Prev: \alph \Alph \arabic \roman \Roman \fnsymbol, Up: Counters + +13.2 '\usecounter' +================== + +Synopsis: + + \usecounter{COUNTER} + + Used in the second argument of the 'list' environment (*note list::), +this declares that list items will be numbered by COUNTER. It +initializes COUNTER to zero, and arranges that when '\item' is called +without its optional argument then COUNTER is incremented by +'\refstepcounter', making its value be the current 'ref' value (*note +\ref::). This command is fragile (*note \protect::). + + Put in the document preamble, this example makes a new list +environment enumerated with TESTCOUNTER: + + \newcounter{testcounter} + \newenvironment{test}{% + \begin{list}{}{% + \usecounter{testcounter} + } + }{% + \end{list} + } + + +File: latex2e.info, Node: \value, Next: \setcounter, Prev: \usecounter, Up: Counters + +13.3 '\value' +============= + +Synopsis: + + \value{COUNTER} + + Expands to the value of the counter COUNTER. (Note that the name of +a counter does not begin with a backslash.) + + This example outputs 'Test counter is 6. Other counter is 5.'. + + \newcounter{test} \setcounter{test}{5} + \newcounter{other} \setcounter{other}{\value{test}} + \addtocounter{test}{1} + + Test counter is \arabic{test}. + Other counter is \arabic{other}. + + The '\value' command is not used for typesetting the value of the +counter. For that, see *note \alph \Alph \arabic \roman \Roman +\fnsymbol::. + + It is often used in '\setcounter' or '\addtocounter' but '\value' can +be used anywhere that LaTeX expects a number, such as in +'\hspace{\value{foo}\parindent}'. It must not be preceded by '\protect' +(*note \protect::). + + This example inserts '\hspace{4\parindent}'. + + \setcounter{myctr}{3} \addtocounter{myctr}{1} + \hspace{\value{myctr}\parindent} + + +File: latex2e.info, Node: \setcounter, Next: \addtocounter, Prev: \value, Up: Counters + +13.4 '\setcounter' +================== + +Synopsis: + + \setcounter{COUNTER}{VALUE} + + Globally set the counter COUNTER to have the value of the VALUE +argument, which must be an integer. Thus, you can set a counter's value +as '\setcounter{section}{5}'. Note that the counter name does not start +with a backslash. + + In this example if the counter 'theorem' has value 12 then the second +line will print 'XII'. + + \setcounter{exercise}{\value{theorem}} + Here it is in Roman: \Roman{exercise}. + + +File: latex2e.info, Node: \addtocounter, Next: \refstepcounter, Prev: \setcounter, Up: Counters + +13.5 '\addtocounter' +==================== + +Synopsis: + + \addtocounter{COUNTER}{VALUE} + + Globally increment COUNTER by the amount specified by the VALUE +argument, which may be negative. + + In this example the section value appears as 'VII'. + + \setcounter{section}{5} + \addtocounter{section}{2} + Here it is in Roman: \Roman{section}. + + +File: latex2e.info, Node: \refstepcounter, Next: \stepcounter, Prev: \addtocounter, Up: Counters + +13.6 '\refstepcounter' +====================== + +Synopsis: + + \refstepcounter{COUNTER} + + Globally increments the value of COUNTER by one, as does +'\stepcounter' (*note \stepcounter::). The difference is that this +command resets the value of any counter numbered within it. (For the +definition of "counters numbered within", *note \newcounter::.) + + In addition, this command also defines the current '\ref' value to be +the result of '\thecounter'. + + While the counter value is set globally, the '\ref' value is set +locally, i.e., inside the current group. + + +File: latex2e.info, Node: \stepcounter, Next: \day & \month & \year, Prev: \refstepcounter, Up: Counters + +13.7 '\stepcounter' +=================== + +Synopsis: + + \stepcounter{COUNTER} + + Globally adds one to COUNTER and resets all counters numbered within +it. (For the definition of "counters numbered within", *note +\newcounter::.) + + This command differs from '\refstepcounter' in that this one does not +influence references; that is, it does not define the current '\ref' +value to be the result of '\thecounter' (*note \refstepcounter::). + + +File: latex2e.info, Node: \day & \month & \year, Prev: \stepcounter, Up: Counters + +13.8 '\day' & '\month' & '\year' +================================ + +LaTeX defines the counter '\day' for the day of the month (nominally +with value between 1 and 31), '\month' for the month of the year +(nominally with value between 1 and 12), and '\year' for the year. When +TeX starts up, they are set from the current values on the system. The +related command '\today' produces a string representing the current day +(*note \today::). + + They counters are not updated as the job progresses so in principle +they could be incorrect by the end. In addition, TeX does no sanity +check: + + \day=-2 \month=13 \year=-4 \today + +gives no error or warning and results in the output '-2, -4' (the bogus +month value produces no output). + + *Note Command line input::, to force the date to a given value from +the command line. + + +File: latex2e.info, Node: Lengths, Next: Making paragraphs, Prev: Counters, Up: Top + +14 Lengths +********** + +A "length" is a measure of distance. Many LaTeX commands take a length +as an argument. + + Lengths come in two types. A "rigid length" such as '10pt' does not +contain a 'plus' or 'minus' component. (Plain TeX calls this a +"dimen".) A "rubber length" (what plain TeX calls a "skip" or "glue") +such as with '1cm plus0.05cm minus0.01cm' can contain either or both of +those components. In that rubber length, the '1cm' is the "natural +length" while the other two, the 'plus' and 'minus' components, allow +TeX to stretch or shrink the length to optimize placement. + + The illustrations below use these two commands. + + % make a black bar 10pt tall and #1 wide + \newcommand{\blackbar}[1]{\rule{#1}{10pt}} + + % Make a box around #2 that is #1 wide (excluding the border) + \newcommand{\showhbox}[2]{% + \fboxsep=0pt\fbox{\hbox to #1{#2}}} + +This example uses those commands to show a black bar 100 points long +between 'XXX' and 'YYY'. This length is rigid. + + XXX\showhbox{100pt}{\blackbar{100pt}}YYY + + As for rubber lengths, shrinking is simpler one: with '1cm minus +0.05cm', the natural length is 1cm but TeX can shrink it down as far as +0.95cm. Beyond that, TeX refuses to shrink any more. Thus, below the +first one works fine, producing a space of 98 points between the two +bars. + + XXX\showhbox{300pt}{% + \blackbar{101pt}\hspace{100pt minus 2pt}\blackbar{101pt}}YYY + + XXX\showhbox{300pt}{% + \blackbar{105pt}\hspace{100pt minus 1pt}\blackbar{105pt}}YYY + +But the second one gets a warning like 'Overfull \hbox (1.0pt too wide) +detected at line 17'. In the output the first 'Y' is overwritten by the +end of the black bar, because the box's material is wider than the 300pt +allocated, as TeX has refused to shrink the total to less than +309 points. + + Stretching is like shrinking except that if TeX is asked to stretch +beyond the given amount, it will do it. Here the first line is fine, +producing a space of 110 points between the bars. + + XXX\showhbox{300pt}{% + \blackbar{95pt}\hspace{100pt plus 10pt}\blackbar{95pt}}YYY + + XXX\showhbox{300pt}{% + \blackbar{95pt}\hspace{100pt plus 1pt}\blackbar{95pt}}YYY + +In the second line TeX needs a stretch of 10 points and only 1 point was +specified. TeX stretches the space to the required length but it gives +you a warning like 'Underfull \hbox (badness 10000) detected at line +22'. (We won't discuss badness.) + + You can put both stretch and shrink in the same length, as in '1ex +plus 0.05ex minus 0.02ex'. + + If TeX is setting two or more rubber lengths then it allocates the +stretch or shrink in proportion. + + XXX\showhbox{300pt}{% + \blackbar{100pt}% left + \hspace{0pt plus 50pt}\blackbar{80pt}\hspace{0pt plus 10pt}% middle + \blackbar{100pt}}YYY % right + +The left and right bars take up 100 points, so the middle needs another +100. The middle bar is 80 points so the two '\hspace''s must stretch +20 points. Because the two are 'plus 50pt' and 'plus 10pt', TeX gets +5/6 of the stretch from the first space and 1/6 from the second. + + The 'plus' or 'minus' component of a rubber length can contain a +"fill" component, as in '1in plus2fill'. This gives the length infinite +stretchability or shrinkability so that TeX could set it to any +distance. Here the two figures will be equal-spaced across the page. + + \begin{minipage}{\linewidth} + \hspace{0pt plus 1fill}\includegraphics{godel.png}% + \hspace{0pt plus 1fill}\includegraphics{einstein.png}% + \hspace{0pt plus 1fill} + \end{minipage} + + TeX has three levels of infinity for glue components: 'fil', 'fill', +and 'filll'. The later ones are more infinite than the earlier ones. +Ordinarily document authors only use the middle one (*note \hfill:: and +*note \vfill::). + + Multiplying a rubber length by a number turns it into a rigid length, +so that after '\setlength{\ylength}{1in plus 0.2in}' and +'\setlength{\zlength}{3\ylength}' then the value of '\zlength' is '3in'. + +* Menu: + +* Units of length:: The units that LaTeX knows. +* \setlength:: Set the value of a length. +* \addtolength:: Add a quantity to a length. +* \settodepth:: Set a length to the depth of something. +* \settoheight:: Set a length to the height of something. +* \settowidth:: Set a length to the width of something. +* \stretch:: Add infinite stretchability. +* Expressions:: Compute with lengths and integers. + + +File: latex2e.info, Node: Units of length, Next: \setlength, Up: Lengths + +14.1 Units of length +==================== + +TeX and LaTeX know about these units both inside and outside of math +mode. + +'pt' + Point, 1/72.27 inch. The (approximate) conversion to metric units + is 1point = .35146mm = .035146cm. + +'pc' + Pica, 12 pt + +'in' + Inch, 72.27 pt + +'bp' + Big point, 1/72 inch. This length is the definition of a point in + PostScript and many desktop publishing systems. + +'mm' + Millimeter, 2.845 pt + +'cm' + Centimeter, 10 mm + +'dd' + Didot point, 1.07 pt + +'cc' + Cicero, 12 dd + +'sp' + Scaled point, 1/65536 pt + + Three other units are defined according to the current font, rather +than being an absolute dimension. + +'ex' + The x-height of the current font "ex", traditionally the height of + the lowercase letter x, is often used for vertical lengths. + +'em' + Similarly "em", traditionally the width of the capital letter M, is + often used for horizontal lengths. This is also often the size of + the current font, e.g., a nominal 10pt font will have 1em = 10pt. + LaTeX has several commands to produce horizontal spaces based on + the em (*note \enspace & \quad & \qquad::). + +'mu' + Finally, in math mode, many definitions are expressed in terms of + the math unit "mu", defined by 1em = 18mu, where the em is taken + from the current math symbols family. *Note Spacing in math + mode::. + + Using these units can help make a definition work better across font +changes. For example, a definition of the vertical space between list +items given as '\setlength{\itemsep}{1ex plus 0.05ex minus 0.01ex}' is +more likely to still be reasonable if the font is changed than a +definition given in points. + + +File: latex2e.info, Node: \setlength, Next: \addtolength, Prev: Units of length, Up: Lengths + +14.2 '\setlength' +================= + +Synopsis: + + \setlength{\LEN}{AMOUNT} + + Set the length \LEN to AMOUNT. The length name '\LEN' has to be a +control sequence (*note Control sequences::), and as such must begin +with a backslash, '\' under normal circumstances. The AMOUNT can be a +rubber length (*note Lengths::). It can be positive, negative or zero, +and can be in any units that LaTeX understands (*note Units of +length::). + + Below, with LaTeX's defaults the first paragraph will be indented +while the second will not. + + I told the doctor I broke my leg in two places. + + \setlength{\parindent}{0em} + He said stop going to those places. + + If you did not declare \LEN with '\newlength', for example if you +mistype it as in +'\newlength{\specparindent}\setlength{\sepcparindent}{...}', then you +get an error like 'Undefined control sequence. \sepcindent'. +If you omit the backslash at the start of the length name then you get +an error like 'Missing number, treated as zero.'. + + +File: latex2e.info, Node: \addtolength, Next: \settodepth, Prev: \setlength, Up: Lengths + +14.3 '\addtolength' +=================== + +Synopsis: + + \addtolength{\LEN}{AMOUNT} + + Increment the length \LEN by AMOUNT. The length name '\LEN' has to +be a control sequence (*note Control sequences::), and as such must +begin with a backslash, '\' under normal circumstances. The AMOUNT is a +rubber length (*note Lengths::). It can be positive, negative or zero, +and can be in any units that LaTeX understands (*note Units of +length::). + + Below, if '\parskip' starts with the value '0pt plus 1pt' + + Doctor: how is the boy who swallowed the silver dollar? + \addtolength{\parskip}{1pt} + + Nurse: no change. + +then it has the value '1pt plus 1pt' for the second paragraph. + + If you did not declare \LEN with '\newlength', for example if you +mistype it as in +'\newlength{\specparindent}\addtolength{\sepcparindent}{...}', then you +get an error like 'Undefined control sequence. \sepcindent'. +If the AMOUNT uses some length that has not been declared, for instance +if for example you mistype the above as +'\addtolength{\specparindent}{0.6\praindent}', then you get something +like 'Undefined control sequence. \praindent'. If you leave +off the backslash at the start of \LEN, as in +'\addtolength{parindent}{1pt}', then you get something like 'You can't +use `the letter p' after \advance'. + + +File: latex2e.info, Node: \settodepth, Next: \settoheight, Prev: \addtolength, Up: Lengths + +14.4 '\settodepth' +================== + +Synopsis: + + \settodepth{\LEN}{TEXT} + + Set the length \LEN to the depth of box that LaTeX gets on +typesetting the TEXT argument. The length name '\LEN' has to be a +control sequence (*note Control sequences::), and as such must begin +with a backslash, '\' under normal circumstances. + + This will print how low the character descenders go. + + \newlength{\alphabetdepth} + \settodepth{\alphabetdepth}{abcdefghijklmnopqrstuvwxyz} + \the\alphabetdepth + + If you did not declare \LEN with '\newlength', if for example you +mistype the above as '\settodepth{\aplhabetdepth}{abc...}', then you get +something like 'Undefined control sequence. \aplhabetdepth'. +If you leave the backslash out of \LEN, as in +'\settodepth{alphabetdepth}{...}' then you get something like 'Missing +number, treated as zero. \setbox'. + + +File: latex2e.info, Node: \settoheight, Next: \settowidth, Prev: \settodepth, Up: Lengths + +14.5 '\settoheight' +=================== + +Synopsis: + + \settoheight{\LEN}{text} + + Sets the length \LEN to the height of box that LaTeX gets on +typesetting the 'text' argument. The length name '\LEN' has to be a +control sequence (*note Control sequences::), and as such must begin +with a backslash, '\' under normal circumstances. + + This will print how high the characters go. + + \newlength{\alphabetheight} + \settoheight{\alphabetheight}{abcdefghijklmnopqrstuvwxyz} + \the\alphabetheight + + If no such length \LEN has been declared with '\newlength', if for +example you mistype as '\settoheight{\aplhabetheight}{abc...}', then you +get something like 'Undefined control sequence. +\alphabetheight'. If you leave the backslash out of \LEN, as in +'\settoheight{alphabetheight}{...}' then you get something like 'Missing +number, treated as zero. \setbox'. + + +File: latex2e.info, Node: \settowidth, Next: \stretch, Prev: \settoheight, Up: Lengths + +14.6 '\settowidth' +================== + +Synopsis: + + \settowidth{\LEN}{TEXT} + + Set the length \LEN to the width of the box that LaTeX gets on +typesetting the TEXT argument. The length name '\LEN' has to be a +control sequence (*note Control sequences::), and as such must begin +with a backslash, '\' under normal circumstances. + + This prints the width of the lowercase ASCII alphabet. + + \newlength{\alphabetwidth} + \settowidth{\alphabetwidth}{abcdefghijklmnopqrstuvwxyz} + \the\alphabetwidth + + If no such length \LEN has been declared with '\newlength', if for +example you mistype the above as '\settowidth{\aplhabetwidth}{abc...}', +then you get something like 'Undefined control sequence. +\aplhabetwidth'. If you leave the backslash out of \LEN, as in +'\settoheight{alphabetwidth}{...}' then you get something like 'Missing +number, treated as zero. \setbox'. + + +File: latex2e.info, Node: \stretch, Next: Expressions, Prev: \settowidth, Up: Lengths + +14.7 '\stretch' +=============== + +Synopsis: + + \stretch{NUMBER} + + Produces a rubber length with zero natural length and NUMBER times +'\fill' units of stretchability (*note Lengths::). The NUMBER can be +positive or negative. This command is robust (*note \protect::). + + It works for both vertical and horizontal spacing. In this +horizontal example, LaTeX produces three tick marks, and the distance +between the first and second is half again as long as the distance +between the second and third. + + \rule{0.4pt}{1ex}\hspace{\stretch{1.5}}% + \rule{0.4pt}{1ex}\hspace{\stretch{1}}% + \rule{0.4pt}{1ex} + + In this vertical example, the 'We dedicate ...' will have three times +as much space under it as above it. + + \newenvironment{dedication}{% in document preamble + \clearpage\thispagestyle{empty}% + \vspace*{\stretch{1}} % stretchable space at top + \it + }{% + \vspace{\stretch{3}} % space at bot is 3x as at top + \clearpage + } + ... + \begin{dedication} % in document body + We dedicate this book to our wives. + \end{dedication} + + +File: latex2e.info, Node: Expressions, Prev: \stretch, Up: Lengths + +14.8 Expressions +================ + +Synopsis, one of: + + \numexpr EXPRESSION + \dimexpr EXPRESSION + \glueexpr EXPRESSION + \muglue EXPRESSION + + Any place where you may write an integer, or a TeX dimen, or TeX +glue, or muglue, you can instead write an expression to compute that +type of quantity. + + An example is that '\the\dimexpr\linewidth-4pt\relax' will produce as +output the length that is four points less than width of a line (the +only purpose of '\the' is to show the result in the document). +Analogously, '\romannumeral\numexpr6+3\relax' will produce 'ix', and +'\the\glueexpr 5pt plus 1pt * 2 \relax' will produce '10.0pt plus +2.0pt'. + + A convenience here over doing calculations by allocating registers +and then using '\advance', etc., is that the evaluation of expressions +does not involve assignments and can therefore be performed in places +where assignments are not allowed. The next example computes the width +of the '\parbox'. + + \newlength{\offset}\setlength{\offset}{2em} + \begin{center} + \parbox{\dimexpr\linewidth-\offset*3}{With malice toward none + with charity for all with firmness in the right as God gives us to see + the right let us strive on to finish the work we are in to bind up the + nation's wounds, to care for him who shall have borne the battle and + for his widow and his orphan \textasciitilde\ to do all which may + achieve and cherish a just and lasting peace among ourselves and with + all nations. ---Abraham Lincoln, Second Inaugural Address, from the + memorial} + \end{center} + + The EXPRESSION consists of one or more terms of the same type +(integer, dimension, etc.) that are added or subtracted. A term that +is a type of number, dimension, etc., consists of a factor of that type, +optionally multiplied or divided by factors. A factor of a type is +either a quantity of that type or a parenthesized subexpression. The +expression produces a result of the given type, so that '\numexpr' +produces an integer, '\dimexpr' produces a dimension, etc. + + In the quotation example above, changing to +'\dimexpr\linewidth-3*\offset' gives the error 'Illegal unit of measure +(pt inserted)'. This is because for '\dimexpr' and '\glueexpr', the +input consists of a dimension or glue value followed by an optional +multiplication factor, and not the other way around. Thus '\the\dimexpr +1pt*10\relax' is valid and produces '10.0pt', but '\the\dimexpr +10*1pt\relax' gives the 'Illegal unit' error. + + The expressions absorb tokens and carry out appropriate mathematics +up to a '\relax' (which will be absorbed), or up to the first non-valid +token. Thus, '\the\numexpr2+3px' will print '5px', because LaTeX reads +the '\numexpr2+3', which is made up of numbers, and then finds the +letter 'p', which cannot be part of a number. It therefore terminates +the expression and produces the '5', followed by the regular text 'px'. + + This termination behavior is useful in comparisons. In +'\ifnum\numexpr\parindent*2 < 10pt Yes\else No\fi', the less than sign +terminates the expression and the result is 'No' (in a standard LaTeX +article). + + Expressions may use the operators '+', '-', '*' and '/' along with +parentheses for subexpressions, '(...)'. In glue expressions the 'plus' +and 'minus' parts do not need parentheses to be affected by a factor. +So '\the\glueexpr 5pt plus 1pt * 2 \relax' results in '10pt plus 2pt'. + + TeX will coerce other numerical types in the same way as it does when +doing register assignment. Thus '\the\numexpr\dimexpr 1pt\relax\relax' +will result in '65536', which is '1pt' converted to scaled points (*note +'sp': units of length sp, TeX's internal unit) and then coerced into an +integer. With a '\glueexpr' here, the stretch and shrink would be +dropped. Going the other way, a '\numexpr' inside a '\dimexpr' or +'\glueexpr' will need appropriate units, as in '\the\dimexpr\numexpr 1 + +2\relax pt\relax', which produces '3.0pt'. + + The details of the arithmetic: each factor is checked to be in the +allowed range, numbers must be less than 2^{31} in absolute value, and +dimensions or glue components must be less than 2^{14} points, or 'mu', +or 'fil', etc. The arithmetic operations are performed individually, +except for a scaling operation (a multiplication immediately followed by +a division) which is done as one combined operation with a 64-bit +product as intermediate value. The result of each operation is again +checked to be in the allowed range. + + Finally, division and scaling take place with rounding (unlike TeX's +'\divide', which truncates). Thus '\the\dimexpr 5pt*(3/2)\relax' puts +'10.0pt' in the document, because it rounds '3/2' to '2', while +'\the\dimexpr 5pt*(4/3)\relax' produces '5.0pt'. + + +File: latex2e.info, Node: Making paragraphs, Next: Math formulas, Prev: Lengths, Up: Top + +15 Making paragraphs +******************** + +To start a paragraph, just type some text. To end the current +paragraph, put an empty line. This is three paragraphs, the separation +of which is made by two empty lines. + + It is a truth universally acknowledged, that a single man in possession + of a good fortune, must be in want of a wife. + + However little known the feelings or views of such a man may be on his + first entering a neighbourhood, this truth is so well fixed in the minds + of the surrounding families, that he is considered the rightful property + of some one or other of their daughters. + + ``My dear Mr. Bennet,'' said his lady to him one day, + ``have you heard that Netherfield Park is let at last?'' + + A paragraph separator is made of a sequence of at least one blank +line, at least one of which is not terminated by a comment. A blank +line is a line that is empty or made only of blank characters such as +space or tab. Comments in source code are started with a '%' and span +up to the end of line. In the following example the two columns are +identical: + + \documentclass[twocolumn]{article} + \begin{document} + First paragraph. + + Second paragraph. + \newpage + First paragraph. + + % separator lines may contain blank characters. + + Second paragraph. + \end{document} + + Once LaTeX has gathered all of a paragraph's contents it divides that +content into lines in a way that is optimized over the entire paragraph +(*note Line breaking::). + + There are places where a new paragraph is not permitted. Don't put a +blank line in math mode (*note Modes::); here the blank line before the +'\end{equation}' + + \begin{equation} + 2^{|S|} > |S| + + \end{equation} + +will get you the error 'Missing $ inserted'. Similarly, the blank line +in this '\section' argument + + \section{aaa + + bbb} + +gets 'Runaway argument? {aaa ! Paragraph ended before \@sect was +complete'. + +* Menu: + +* \par:: End the current paragraph. +* \indent & \noindent:: Go into horizontal mode, possibly with an indent. +* \parindent & \parskip:: Space added before paragraphs. +* Marginal notes:: Put remarks in the margin. + + +File: latex2e.info, Node: \par, Next: \indent & \noindent, Up: Making paragraphs + +15.1 '\par' +=========== + +Synopsis (note that while reading the input TeX converts any sequence of +one or more blank lines to a '\par', *note Making paragraphs::): + + \par + + End the current paragraph. The usual way to separate paragraphs is +with a blank line but the '\par' command is entirely equivalent. This +command is robust (*note \protect::). + + This example uses '\par' rather than a blank line simply for +readability. + + \newcommand{\syllabusLegalese}{% + \whatCheatingIs\par\whatHappensWhenICatchYou} + + In LR mode the '\par' command does nothing and is ignored. In +paragraph mode, the '\par' command terminates paragraph mode, switching +LaTeX to vertical mode (*note Modes::). + + You cannot use the '\par' command in a math mode. You also cannot +use it in the argument of many commands, such as the sectioning +commands, e.g. '\section' (*note Making paragraphs:: and *note +\newcommand & \renewcommand::). + + The '\par' command is not the same as the '\paragraph' command. The +latter is, like '\section' or '\subsection', a sectioning command used +by the LaTeX document standard classes (*note \subsubsection & +\paragraph & \subparagraph::). + + The '\par' command is not the same as '\newline' or the line break +double backslash, '\\'. The difference is that '\par' ends the +paragraph, not just the line, and also triggers the addition of the +between-paragraph vertical space '\parskip' (*note \parindent & +\parskip::). + + The output from this example + + xyz + + \setlength{\parindent}{3in} + \setlength{\parskip}{5in} + \noindent test\indent test1\par test2 + +is: after 'xyz' there is a vertical skip of 5 inches and then 'test' +appears, aligned with the left margin. On the same line, there is an +empty horizontal space of 3 inches and then 'test1' appears. Finally. +there is a vertical space of 5 inches, followed by a fresh paragraph +with a paragraph indent of 3 inches, and then LaTeX puts the text +'test2'. + + +File: latex2e.info, Node: \indent & \noindent, Next: \parindent & \parskip, Prev: \par, Up: Making paragraphs + +15.2 '\indent' & '\noindent' +============================ + +Synopsis: + + \indent + +or + + \noindent + + Go into horizontal mode (*note Modes::). The '\indent' command first +outputs an empty box whose width is '\parindent'. These commands are +robust (*note \protect::). + + Ordinarily you create a new paragraph by putting in a blank line. +*Note \par::, for the difference between this command and '\par'. To +start a paragraph without an indent, or to continue an interrupted +paragraph, use '\noindent'. + + In the middle of a paragraph the '\noindent' command has no effect, +because LaTeX is already in horizontal mode there. The '\indent' +command's only effect is to output a space. + + This example starts a fresh paragraph. + + ... end of the prior paragraph. + + \noindent This paragraph is not indented. + +and this continues an interrupted paragraph. + + The data + + \begin{center} + \begin{tabular}{rl} ... \end{tabular} + \end{center} + + \noindent shows this clearly. + + To omit indentation in the entire document put +'\setlength{\parindent}{0pt}' in the preamble. If you do that, you may +want to also set the length of spaces between paragraphs, '\parskip' +(*note \parindent & \parskip::). + + Default LaTeX styles have the first paragraph after a section that is +not indented, as is traditional typesetting in English. To change that, +look on CTAN for the package 'indentfirst'. + + +File: latex2e.info, Node: \parindent & \parskip, Next: Marginal notes, Prev: \indent & \noindent, Up: Making paragraphs + +15.3 '\parindent' & '\parskip' +============================== + +Synopsis: + + \setlength{\parindent}{HORIZONTAL LEN} + \setlength{\parskip}{VERTICAL LEN} + + Both are rubber lengths (*note Lengths::). They affect the +indentation of ordinary paragraphs, not paragraphs inside minipages +(*note minipage::), and the vertical space between paragraphs, +respectively. + + For example, if this is put in the preamble: + + \setlength{\parindent}{0em} + \setlength{\parskip}{1ex} + +The document will have paragraphs that are not indented, but instead are +vertically separated by about the height of a lowercase 'x'. + + In LaTeX standard class documents, the default value for '\parindent' +in one-column documents is '15pt' when the default text size is '10pt', +'17pt' for '11pt', and '1.5em' for '12pt'. In two-column documents it +is '1em'. (These values are set before LaTeX calls '\normalfont' so +'em' is derived from the default font, Computer Modern. If you use a +different font then to set '\parindent' to 1em matching that font, put +'\AtBeginDocument{\setlength{\parindent}{1em}}' in the preamble.) + + The default value for '\parskip' in LaTeX's standard document classes +is '0pt plus1pt'. + + +File: latex2e.info, Node: Marginal notes, Prev: \parindent & \parskip, Up: Making paragraphs + +15.4 Marginal notes +=================== + +Synopsis, one of: + + \marginpar{RIGHT} + \marginpar[LEFT]{RIGHT} + + Create a note in the margin. The first line of the note will have +the same baseline as the line in the text where the '\marginpar' occurs. + + The margin that LaTeX uses for the note depends on the current layout +(*note Document class options::) and also on '\reversemarginpar' (see +below). If you are using one-sided layout (document option 'oneside') +then it goes in the right margin. If you are using two-sided layout +(document option 'twoside') then it goes in the outside margin. If you +are in two-column layout (document option 'twocolumn') then it goes in +the nearest margin. + + If you declare '\reversemarginpar' then LaTeX will place subsequent +marginal notes in the opposite margin to that given in the prior +paragraph. Revert that to the default position with '\normalmarginpar'. + + When you specify the optional argument LEFT then it is used for a +note in the left margin, while the mandatory argument RIGHT is used for +a note in the right margin. + + Normally, a note's first word will not be hyphenated. You can enable +hyphenation there by beginning LEFT or RIGHT with '\hspace{0pt}'. + + These parameters affect the formatting of the note: + +'\marginparpush' + Minimum vertical space between notes; default '7pt' for '12pt' + documents, '5pt' else. See also *note page layout parameters + marginparpush::. + +'\marginparsep' + Horizontal space between the main text and the note; default '11pt' + for '10pt' documents, '10pt' else. + +'\marginparwidth' + Width of the note itself; default for a one-sided '10pt' document + is '90pt', '83pt' for '11pt', and '68pt' for '12pt'; '17pt' more in + each case for a two-sided document. In two column mode, the + default is '48pt'. + + The standard LaTeX routine for marginal notes does not prevent notes +from falling off the bottom of the page. + + +File: latex2e.info, Node: Math formulas, Next: Modes, Prev: Making paragraphs, Up: Top + +16 Math formulas +**************** + +Produce mathematical text by putting LaTeX into math mode or display +math mode (*note Modes::). This example shows both. + + The wave equation for \( u \) is + \begin{displaymath} + \frac{\partial^2u}{\partial t^2} = c^2\nabla^2u + \end{displaymath} + where \( \nabla^2 \) is the spatial Laplacian and \( c \) is constant. + +Math mode is for inline mathematics. In the above example it is invoked +by the starting '\(' and finished by the matching ending '\)'. Display +math mode is for displayed equations and here is invoked by the +'displaymath' environment. Note that any mathematical text whatever, +including mathematical text consisting of just one character, is handled +in math mode. + + When in math mode or display math mode, LaTeX handles many aspects of +your input text differently than in other text modes. For example, + + contrast x+y with \( x+y \) + +in math mode the letters are in italics and the spacing around the plus +sign is different. + + There are three ways to make inline formulas, to put LaTeX in math +mode. + + \( MATHEMATICAL MATERIAL \) + $ MATHEMATICAL MATERIAL $ + \begin{math} MATHEMATICAL MATERIAL \end{math} + +The first form is preferred and the second is quite common, but the +third form is rarely used. You can sometimes use one and sometimes +another, as in '\(x\) and $y$'. You can use these in paragraph mode or +in LR mode (*note Modes::). + + To make displayed formulas, put LaTeX into display math mode with +either: + + \begin{displaymath} + MATHEMATICAL MATERIAL + \end{displaymath} + +or + + \begin{equation} + MATHEMATICAL MATERIAL + \end{equation} + +(*note displaymath::, *note equation::). The only difference is that +with the 'equation' environment, LaTeX puts a formula number alongside +the formula. The construct '\[ MATH \]' is equivalent to +'\begin{displaymath} MATH \end{displaymath}'. These environments can +only be used in paragraph mode (*note Modes::). + + The American Mathematical Society has made freely available a set of +packages that greatly expand your options for writing mathematics, +'amsmath' and 'amssymb' (also be aware of the 'mathtools' package that +is an extension to, and loads, 'amsmath'). New documents that will have +mathematical text should use these packages. Descriptions of these +packages is outside the scope of this document; see their documentation +on CTAN. + +* Menu: + +* Subscripts & superscripts:: Also known as exponents or indices. +* Math symbols:: Various mathematical squiggles. +* Math functions:: Math function names like sin and exp. +* Math accents:: Accents in math. +* Over- and Underlining:: Things over or under formulas. +* Spacing in math mode:: Thick, medium, thin, and negative spaces. +* Math styles:: Determine the size of things. +* Math miscellany:: Stuff that doesn't fit anywhere else. + + +File: latex2e.info, Node: Subscripts & superscripts, Next: Math symbols, Up: Math formulas + +16.1 Subscripts & superscripts +============================== + +Synopsis (in math mode or display math mode), one of: + + BASE^EXP + BASE^{EXP} + +or, one of: + + BASE_EXP + BASE_{EXP} + + Make EXP appear as a superscript of BASE (with the caret +character, '^') or a subscript (with underscore, '_'). + + In this example the '0''s and '1''s are subscripts while the '2''s +are superscripts. + + \( (x_0+x_1)^2 \leq (x_0)^2+(x_1)^2 \) + + To have the subscript or superscript contain more than one character, +surround the expression with curly braces, as in 'e^{-2x}'. This +example's fourth line shows curly braces used to group an expression for +the exponent. + + \begin{displaymath} + (3^3)^3=27^3=19\,683 + \qquad + 3^{(3^3)}=3^{27}=7\,625\,597\,484\,987 + \end{displaymath} + + LaTeX knows how to handle a superscript on a superscript, or a +subscript on a subscript, or supers on subs, or subs on supers. So, +expressions such as 'e^{x^2}' and 'x_{i_0}' give correct output. Note +the use in those expressions of curly braces to give the BASE a +determined EXP. If you enter '\(3^3^3\)' then you get 'Double +superscript'. + + LaTeX does the right thing when something has both a subscript and a +superscript. In this example the integral has both. They come out in +the correct place without any author intervention. + + \begin{displaymath} + \int_{x=a}^b f'(x)\,dx = f(b)-f(a) + \end{displaymath} + +Note the parentheses around 'x=a' to make the entire expression a +subscript. + + To put a superscript or subscript before a symbol, use a construct +like '{}_t K^2'. The empty curly braces '{}' give the subscript +something to attach to and keeps it from accidentally attaching to a +prior symbols. + + Using the subscript or superscript character outside of math mode or +display math mode, as in 'the expression x^2', will get you the error +'Missing $ inserted'. + + A common reason to want subscripts outside of a mathematics mode is +to typeset chemical formulas. There are packages for that, such as +'mhchem'; see CTAN. + + +File: latex2e.info, Node: Math symbols, Next: Math functions, Prev: Subscripts & superscripts, Up: Math formulas + +16.2 Math symbols +================= + +LaTeX provides almost any mathematical or technical symbol that anyone +uses. For example, if you include '$\pi$' in your source, you will get +the pi symbol π. See the "Comprehensive LaTeX Symbol List" package at +. + + Here is a list of commonly-used symbols. It is by no means +exhaustive. Each symbol is described with a short phrase, and its +symbol class, which determines the spacing around it, is given in +parenthesis. Unless said otherwise, the commands for these symbols can +be used only in math mode. To redefine a command so that it can be used +whatever the current mode, see *note \ensuremath::. + +'\|' + ∥ Parallel (relation). Synonym: '\parallel'. + +'\aleph' + ℵ Aleph, transfinite cardinal (ordinary). + +'\alpha' + α Lowercase Greek letter alpha (ordinary). + +'\amalg' + ⨿ Disjoint union (binary) + +'\angle' + ∠ Geometric angle (ordinary). Similar: less-than sign '<' and + angle bracket '\langle'. + +'\approx' + ≈ Almost equal to (relation). + +'\ast' + ∗ Asterisk operator, convolution, six-pointed (binary). + Synonym: '*', which is often a superscript or subscript, as in the + Kleene star. Similar: '\star', which is five-pointed, and is + sometimes used as a general binary operation, and sometimes + reserved for cross-correlation. + +'\asymp' + ≍ Asymptotically equivalent (relation). + +'\backslash' + \ Backslash (ordinary). Similar: set minus '\setminus', and + '\textbackslash' for backslash outside of math mode. + +'\beta' + β Lowercase Greek letter beta (ordinary). + +'\bigcap' + ⋂ Variable-sized, or n-ary, intersection (operator). Similar: + binary intersection '\cap'. + +'\bigcirc' + ⚪ Circle, larger (binary). Similar: function composition '\circ'. + +'\bigcup' + ⋃ Variable-sized, or n-ary, union (operator). Similar: binary + union '\cup'. + +'\bigodot' + ⨀ Variable-sized, or n-ary, circled dot operator (operator). + +'\bigoplus' + ⨁ Variable-sized, or n-ary, circled plus operator (operator). + +'\bigotimes' + ⨂ Variable-sized, or n-ary, circled times operator (operator). + +'\bigtriangledown' + ▽ Variable-sized, or n-ary, open triangle pointing down (operator). + +'\bigtriangleup' + △ Variable-sized, or n-ary, open triangle pointing up (operator). + +'\bigsqcup' + ⨆ Variable-sized, or n-ary, square union (operator). + +'\biguplus' + ⨄ Variable-sized, or n-ary, union operator with a plus (operator). + (Note that the name has only one p.) + +'\bigvee' + ⋁ Variable-sized, or n-ary, logical-or (operator). + +'\bigwedge' + ⋀ Variable-sized, or n-ary, logical-and (operator). + +'\bot' + ⊥, Up tack, bottom, least element of a partially ordered set, or a + contradiction (ordinary). See also '\top'. + +'\bowtie' + ⋈ Natural join of two relations (relation). + +'\Box' + □ Modal operator for necessity; square open box (ordinary). Not + available in plain TeX. In LaTeX you need to load the 'amssymb' + package. + +'\bullet' + • Bullet (binary). Similar: multiplication dot '\cdot'. + +'\cap' + ∩ Intersection of two sets (binary). Similar: variable-sized + operator '\bigcap'. + +'\cdot' + ⋅ Multiplication (binary). Similar: Bullet dot '\bullet'. + +'\chi' + χ Lowercase Greek chi (ordinary). + +'\circ' + ∘ Function composition, ring operator (binary). Similar: + variable-sized operator '\bigcirc'. + +'\clubsuit' + ♣ Club card suit (ordinary). + +'\complement' + ∁, Set complement, used as a superscript as in '$S^\complement$' + (ordinary). Not available in plain TeX. In LaTeX you need to load + the 'amssymb' package. Also used: '$S^{\mathsf{c}}$' + or '$\bar{S}$'. + +'\cong' + ≅ Congruent (relation). + +'\coprod' + ∐ Coproduct (operator). + +'\cup' + ∪ Union of two sets (binary). Similar: variable-sized + operator '\bigcup'. + +'\dagger' + † Dagger relation (binary). + +'\dashv' + ⊣ Dash with vertical, reversed turnstile (relation). Similar: + turnstile '\vdash'. + +'\ddagger' + ‡ Double dagger relation (binary). + +'\Delta' + Δ Greek uppercase delta, used for increment (ordinary). + +'\delta' + δ Greek lowercase delta (ordinary). + +'\Diamond' + ◇ Large diamond operator (ordinary). Not available in plain TeX. + In LaTeX you need to load the 'amssymb' package. + +'\diamond' + ⋄ Diamond operator (binary). Similar: large diamond '\Diamond', + circle bullet '\bullet'. + +'\diamondsuit' + ♢ Diamond card suit (ordinary). + +'\div' + ÷ Division sign (binary). + +'\doteq' + ≐ Approaches the limit (relation). Similar: geometrically equal + to '\Doteq'. + +'\downarrow' + ↓ Down arrow, converges (relation). Similar: '\Downarrow' double + line down arrow. + +'\Downarrow' + ⇓ Double line down arrow (relation). Similar: '\downarrow' single + line down arrow. + +'\ell' + ℓ Lowercase cursive letter l (ordinary). + +'\emptyset' + ∅ Empty set symbol (ordinary). The variant form is '\varnothing'. + +'\epsilon' + ϵ Lowercase lunate epsilon (ordinary). Similar to Greek text + letter. More widely used in mathematics is the script small letter + epsilon '\varepsilon' ε. Related: the set membership relation + '\in' ∈. + +'\equiv' + ≡ Equivalence (relation). + +'\eta' + η Lowercase Greek letter (ordinary). + +'\exists' + ∃ Existential quantifier (ordinary). + +'\flat' + ♭ Musical flat (ordinary). + +'\forall' + ∀ Universal quantifier (ordinary). + +'\frown' + ⌢ Downward curving arc (ordinary). + +'\Gamma' + Γ uppercase Greek letter (ordinary). + +'\gamma' + γ Lowercase Greek letter (ordinary). + +'\ge' + ≥ Greater than or equal to (relation). This is a synonym + for '\geq'. + +'\geq' + ≥ Greater than or equal to (relation). This is a synonym + for '\ge'. + +'\gets' + ← Is assigned the value (relation). Synonym: '\leftarrow'. + +'\gg' + ≫ Much greater than (relation). Similar: much less than '\ll'. + +'\hbar' + ℏ Planck constant over two pi (ordinary). + +'\heartsuit' + ♡ Heart card suit (ordinary). + +'\hookleftarrow' + ↩ Hooked left arrow (relation). + +'\hookrightarrow' + ↪ Hooked right arrow (relation). + +'\iff' + ⟷ If and only if (relation). It is '\Longleftrightarrow' with a + '\thickmuskip' on either side. + +'\Im' + ℑ Imaginary part (ordinary). See: real part '\Re'. + +'\imath' + Dotless i; used when you are putting an accent on an i (*note Math + accents::). + +'\in' + ∈ Set element (relation). See also: lowercase lunate + epsilon '\epsilon'ϵ and small letter script epsilon '\varepsilon'. + +'\infty' + ∞ Infinity (ordinary). + +'\int' + ∫ Integral (operator). + +'\iota' + ι Lowercase Greek letter (ordinary). + +'\Join' + ⨝ Condensed bowtie symbol (relation). Not available in Plain TeX. + +'\jmath' + Dotless j; used when you are putting an accent on a j (*note Math + accents::). + +'\kappa' + κ Lowercase Greek letter (ordinary). + +'\Lambda' + Λ uppercase Greek letter (ordinary). + +'\lambda' + λ Lowercase Greek letter (ordinary). + +'\land' + ∧ Logical and (binary). Synonym: '\wedge'. See also + logical or '\lor'. + +'\langle' + ⟨ Left angle, or sequence, bracket (opening). Similar: + less-than '<'. Matches '\rangle'. + +'\lbrace' + { Left curly brace (opening). Synonym: '\{'. Matches '\rbrace'. + +'\lbrack' + [ Left square bracket (opening). Synonym: '['. Matches '\rbrack'. + +'\lceil' + ⌈ Left ceiling bracket, like a square bracket but with the bottom + shaved off (opening). Matches '\rceil'. + +'\le' + ≤ Less than or equal to (relation). This is a synonym for '\leq'. + +'\leadsto' + ⇝ Squiggly right arrow (relation). To get this symbol outside of + math mode you can put + '\newcommand*{\Leadsto}{\ensuremath{\leadsto}}' in the preamble and + then use '\Leadsto' instead. + +'\Leftarrow' + ⇐ Is implied by, double-line left arrow (relation). Similar: + single-line left arrow '\leftarrow'. + +'\leftarrow' + ← Single-line left arrow (relation). Synonym: '\gets'. Similar: + double-line left arrow '\Leftarrow'. + +'\leftharpoondown' + ↽ Single-line left harpoon, barb under bar (relation). + +'\leftharpoonup' + ↼ Single-line left harpoon, barb over bar (relation). + +'\Leftrightarrow' + ⇔ Bi-implication; double-line double-headed arrow (relation). + Similar: single-line double headed arrow '\leftrightarrow'. + +'\leftrightarrow' + ↔ Single-line double-headed arrow (relation). Similar: double-line + double headed arrow '\Leftrightarrow'. + +'\leq' + ≤ Less than or equal to (relation). This is a synonym for '\le'. + +'\lfloor' + ⌊ Left floor bracket (opening). Matches: '\floor'. + +'\lhd' + ◁ Arrowhead, that is, triangle, pointing left (binary). For the + normal subgroup symbol you should load 'amssymb' and + use '\vartriangleleft' (which is a relation and so gives better + spacing). + +'\ll' + ≪ Much less than (relation). Similar: much greater than '\gg'. + +'\lnot' + ¬ Logical negation (ordinary). Synonym: '\neg'. + +'\longleftarrow' + ⟵ Long single-line left arrow (relation). Similar: long + double-line left arrow '\Longleftarrow'. + +'\longleftrightarrow' + ⟷ Long single-line double-headed arrow (relation). Similar: long + double-line double-headed arrow '\Longleftrightarrow'. + +'\longmapsto' + ⟼ Long single-line left arrow starting with vertical bar + (relation). Similar: shorter version '\mapsto'. + +'\longrightarrow' + ⟶ Long single-line right arrow (relation). Similar: long + double-line right arrow '\Longrightarrow'. + +'\lor' + ∨ Logical or (binary). Synonym: '\vee'. See also + logical and '\land'. + +'\mapsto' + ↦ Single-line left arrow starting with vertical bar (relation). + Similar: longer version '\longmapsto'. + +'\mho' + ℧ Conductance, half-circle rotated capital omega (ordinary). + +'\mid' + ∣ Single-line vertical bar (relation). A typical use of '\mid' is + for a set '\{\, x \mid x\geq 5 \,\}'. + + Similar: '\vert' and '|' produce the same single-line vertical bar + symbol but without any spacing (they fall in class ordinary) and + you should not use them as relations but instead only as ordinals, + i.e., footnote symbols. For absolute value, see the entry + for '\vert' and for norm see the entry for '\Vert'. + +'\models' + ⊨ Entails, or satisfies; double turnstile, short double dash + (relation). Similar: long double dash '\vDash'. + +'\mp' + ∓ Minus or plus (relation). + +'\mu' + μ Lowercase Greek letter (ordinary). + +'\nabla' + ∇ Hamilton's del, or differential, operator (ordinary). + +'\natural' + ♮ Musical natural notation (ordinary). + +'\ne' + ≠ Not equal (relation). Synonym: '\neq'. + +'\nearrow' + ↗ North-east arrow (relation). + +'\neg' + ¬ Logical negation (ordinary). Synonym: '\lnot'. Sometimes + instead used for negation: '\sim'. + +'\neq' + ≠ Not equal (relation). Synonym: '\ne'. + +'\ni' + ∋ Reflected membership epsilon; has the member (relation). + Synonym: '\owns'. Similar: is a member of '\in'. + +'\not' + Long solidus, or slash, used to overstrike a following operator + (relation). + + Many negated operators are available that don't require '\not', + particularly with the 'amssymb' package. For example, '\notin' is + typographically preferable to '\not\in'. + +'\notin' + ∉ Not an element of (relation). Similar: not subset + of '\nsubseteq'. + +'\nu' + ν Lowercase Greek letter (ordinary). + +'\nwarrow' + ↖ North-west arrow (relation). + +'\odot' + ⊙ Dot inside a circle (binary). Similar: variable-sized + operator '\bigodot'. + +'\oint' + ∮ Contour integral, integral with circle in the middle (operator). + +'\Omega' + Ω uppercase Greek letter (ordinary). + +'\omega' + ω Lowercase Greek letter (ordinary). + +'\ominus' + ⊖ Minus sign, or dash, inside a circle (binary). + +'\oplus' + ⊕ Plus sign inside a circle (binary). Similar: variable-sized + operator '\bigoplus'. + +'\oslash' + ⊘ Solidus, or slash, inside a circle (binary). + +'\otimes' + ⊗ Times sign, or cross, inside a circle (binary). Similar: + variable-sized operator '\bigotimes'. + +'\owns' + ∋ Reflected membership epsilon; has the member (relation). + Synonym: '\ni'. Similar: is a member of '\in'. + +'\parallel' + ∥ Parallel (relation). Synonym: '\|'. + +'\partial' + ∂ Partial differential (ordinary). + +'\perp' + ⟂ Perpendicular (relation). Similar: '\bot' uses the same glyph + but the spacing is different because it is in the class ordinary. + +'\Phi' + Φ Uppercase Greek letter (ordinary). + +'\phi' + ϕ Lowercase Greek letter (ordinary). The variant form is + '\varphi' φ. + +'\Pi' + Π uppercase Greek letter (ordinary). + +'\pi' + π Lowercase Greek letter (ordinary). The variant form is + '\varpi' ϖ. + +'\pm' + ± Plus or minus (binary). + +'\prec' + ≺ Precedes (relation). Similar: less than '<'. + +'\preceq' + ⪯ Precedes or equals (relation). Similar: less than or + equals '\leq'. + +'\prime' + ′ Prime, or minute in a time expression (ordinary). Typically used + as a superscript: '$f^\prime$'; '$f^\prime$' and '$f'$' produce the + same result. An advantage of the second is that '$f'''$' produces + the desired symbol, that is, the same result as + '$f^{\prime\prime\prime}$', but uses rather less typing. You can + only use '\prime' in math mode. Using the right single quote ''' + in text mode produces a different character (apostrophe). + +'\prod' + ∏ Product (operator). + +'\propto' + ∝ Is proportional to (relation) + +'\Psi' + Ψ uppercase Greek letter (ordinary). + +'\psi' + ψ Lowercase Greek letter (ordinary). + +'\rangle' + ⟩ Right angle, or sequence, bracket (closing). Similar: greater + than '>'. Matches:'\langle'. + +'\rbrace' + } Right curly brace (closing). Synonym: '\}'. Matches '\lbrace'. + +'\rbrack' + ] Right square bracket (closing). Synonym: ']'. + Matches '\lbrack'. + +'\rceil' + ⌉ Right ceiling bracket (closing). Matches '\lceil'. + +'\Re' + ℜ Real part, real numbers, cursive capital R (ordinary). Related: + double-line, or blackboard bold, R '\mathbb{R}'; to access this, + load the 'amsfonts' package. + +'\restriction' + ↾, Restriction of a function (relation). Synonym: + '\upharpoonright'. Not available in plain TeX. In LaTeX you need + to load the 'amssymb' package. + +'\revemptyset' + ⦰, Reversed empty set symbol (ordinary). Related: '\varnothing'. + Not available in plain TeX. In LaTeX you need to load the 'stix' + package. + +'\rfloor' + ⌋ Right floor bracket, a right square bracket with the top cut off + (closing). Matches '\lfloor'. + +'\rhd' + ◁ Arrowhead, that is, triangle, pointing right (binary). For the + normal subgroup symbol you should instead load 'amssymb' and + use '\vartriangleright' (which is a relation and so gives better + spacing). + +'\rho' + ρ Lowercase Greek letter (ordinary). The variant form is + '\varrho' ϱ. + +'\Rightarrow' + ⇒ Implies, right-pointing double line arrow (relation). Similar: + right single-line arrow '\rightarrow'. + +'\rightarrow' + → Right-pointing single line arrow (relation). Synonym: '\to'. + Similar: right double line arrow '\Rightarrow'. + +'\rightharpoondown' + ⇁ Right-pointing harpoon with barb below the line (relation). + +'\rightharpoonup' + ⇀ Right-pointing harpoon with barb above the line (relation). + +'\rightleftharpoons' + ⇌ Right harpoon up above left harpoon down (relation). + +'\searrow' + ↘ Arrow pointing southeast (relation). + +'\setminus' + ⧵ Set difference, reverse solidus or reverse slash, like \ + (binary). Similar: backslash '\backslash' and also + '\textbackslash' outside of math mode. + +'\sharp' + ♯ Musical sharp (ordinary). + +'\Sigma' + Σ uppercase Greek letter (ordinary). + +'\sigma' + σ Lowercase Greek letter (ordinary). The variant form is + '\varsigma' ς. + +'\sim' + ∼ Similar, in a relation (relation). + +'\simeq' + ≃ Similar or equal to, in a relation (relation). + +'\smallint' + ∫ Integral sign that does not change to a larger size in a display + (operator). + +'\smile' + ⌣ Upward curving arc, smile (ordinary). + +'\spadesuit' + ♠ Spade card suit (ordinary). + +'\sqcap' + ⊓ Square intersection symbol (binary). Similar: + intersection 'cap'. + +'\sqcup' + ⊔ Square union symbol (binary). Similar: union 'cup'. Related: + variable-sized operator '\bigsqcup'. + +'\sqsubset' + ⊏, Square subset symbol (relation). Similar: subset '\subset'. + Not available in plain TeX. In LaTeX you need to load the + 'amssymb' package. + +'\sqsubseteq' + ⊑ Square subset or equal symbol (binary). Similar: subset or equal + to '\subseteq'. + +'\sqsupset' + ⊐, Square superset symbol (relation). Similar: superset '\supset'. + Not available in plain TeX. In LaTeX you need to load the + 'amssymb' package. + +'\sqsupseteq' + ⊒ Square superset or equal symbol (binary). Similar: superset or + equal '\supseteq'. + +'\star' + ⋆ Five-pointed star, sometimes used as a general binary operation + but sometimes reserved for cross-correlation (binary). Similar: + the synonyms asterisk '*' and '\ast', which are six-pointed, and + more often appear as a superscript or subscript, as with the Kleene + star. + +'\subset' + ⊂ Subset (occasionally, is implied by) (relation). + +'\subseteq' + ⊆ Subset or equal to (relation). + +'\succ' + ≻ Comes after, succeeds (relation). Similar: is less than '>'. + +'\succeq' + ⪰ Succeeds or is equal to (relation). Similar: less than or equal + to '\leq'. + +'\sum' + ∑ Summation (operator). Similar: Greek capital sigma '\Sigma'. + +'\supset' + ⊃ Superset (relation). + +'\supseteq' + ⊇ Superset or equal to (relation). + +'\surd' + √ Radical symbol (ordinary). The LaTeX command '\sqrt{...}' + typesets the square root of the argument, with a bar that extends + to cover the argument. + +'\swarrow' + ↙ Southwest-pointing arrow (relation). + +'\tau' + τ Lowercase Greek letter (ordinary). + +'\theta' + θ Lowercase Greek letter (ordinary). The variant form is + '\vartheta' ϑ. + +'\times' + × Primary school multiplication sign (binary). See also '\cdot'. + +'\to' + → Right-pointing single line arrow (relation). + Synonym: '\rightarrow'. + +'\top' + ⊤ Top, greatest element of a partially ordered set (ordinary). See + also '\bot'. + +'\triangle' + △ Triangle (ordinary). + +'\triangleleft' + ◁ Not-filled triangle pointing left (binary). Similar: '\lhd'. + For the normal subgroup symbol you should load 'amssymb' and + use '\vartriangleleft' (which is a relation and so gives better + spacing). + +'\triangleright' + ▷ Not-filled triangle pointing right (binary). For the normal + subgroup symbol you should instead load 'amssymb' and + use '\vartriangleright' (which is a relation and so gives better + spacing). + +'\unlhd' + ⊴ Left-pointing not-filled underlined arrowhead, that is, triangle, + with a line under (binary). For the normal subgroup symbol load + 'amssymb' and use '\vartrianglelefteq' (which is a relation and so + gives better spacing). + +'\unrhd' + ⊵ Right-pointing not-filled underlined arrowhead, that is, + triangle, with a line under (binary). For the normal subgroup + symbol load 'amssymb' and use '\vartrianglerighteq' (which is a + relation and so gives better spacing). + +'\Uparrow' + ⇑ Double-line upward-pointing arrow (relation). Similar: + single-line up-pointing arrow '\uparrow'. + +'\uparrow' + ↑ Single-line upward-pointing arrow, diverges (relation). Similar: + double-line up-pointing arrow '\Uparrow'. + +'\Updownarrow' + ⇕ Double-line upward-and-downward-pointing arrow (relation). + Similar: single-line upward-and-downward-pointing + arrow '\updownarrow'. + +'\updownarrow' + ↕ Single-line upward-and-downward-pointing arrow (relation). + Similar: double-line upward-and-downward-pointing + arrow '\Updownarrow'. + +'\upharpoonright' + ↾, Up harpoon, with barb on right side (relation). + Synonym: '\restriction'. Not available in plain TeX. In LaTeX you + need to load the 'amssymb' package. + +'\uplus' + ⊎ Multiset union, a union symbol with a plus symbol in the middle + (binary). Similar: union '\cup'. Related: variable-sized + operator '\biguplus'. + +'\Upsilon' + Υ uppercase Greek letter (ordinary). + +'\upsilon' + υ Lowercase Greek letter (ordinary). + +'\varepsilon' + ε Small letter script epsilon (ordinary). This is more widely used + in mathematics than the non-variant lunate epsilon form + '\epsilon' ϵ. Related: set membership '\in'. + +'\vanothing' + ∅, Empty set symbol. Similar: '\emptyset'. Related: + '\revemptyset'. Not available in plain TeX. In LaTeX you need to + load the 'amssymb' package. + +'\varphi' + φ Variant on the lowercase Greek letter (ordinary). The + non-variant form is '\phi' ϕ. + +'\varpi' + ϖ Variant on the lowercase Greek letter (ordinary). The + non-variant form is '\pi' π. + +'\varrho' + ϱ Variant on the lowercase Greek letter (ordinary). The + non-variant form is '\rho' ρ. + +'\varsigma' + ς Variant on the lowercase Greek letter (ordinary). The + non-variant form is '\sigma' σ. + +'\vartheta' + ϑ Variant on the lowercase Greek letter (ordinary). The + non-variant form is '\theta' θ. + +'\vdash' + ⊢ Provable; turnstile, vertical and a dash (relation). Similar: + turnstile rotated a half-circle '\dashv'. + +'\vee' + ∨ Logical or; a downwards v shape (binary). Related: logical + and '\wedge'. Similar: variable-sized operator '\bigvee'. + +'\Vert' + ‖ Vertical double bar (ordinary). *Note Delimiters::, for how to + use the 'mathtools' package to create flexibly-sized norm symbols. + +'\vert' + | Single line vertical bar (ordinary). For "such that", as in the + definition of a set, use '\mid' because it is a relation. *Note + Delimiters::, for how to use the 'mathtools' package to create + flexibly-sized absolute-value symbols. + +'\wedge' + ∧ Logical and (binary). Synonym: '\land'. See also logical or + '\vee'. Similar: variable-sized operator '\bigwedge'. + +'\wp' + ℘ Weierstrass p (ordinary). + +'\wr' + ≀ Wreath product (binary). + +'\Xi' + Ξ uppercase Greek letter (ordinary). + +'\xi' + ξ Lowercase Greek letter (ordinary). + +'\zeta' + ζ Lowercase Greek letter (ordinary). + + The following symbols are most often used in plain text but LaTeX +provides versions to use in mathematical text. + +'\mathdollar' + Dollar sign in math mode: $. + +'\mathparagraph' + Paragraph sign (pilcrow) in math mode, ¶. + +'\mathsection' + Section sign in math mode §. + +'\mathsterling' + Sterling sign in math mode: £. + +'\mathunderscore' + Underscore in math mode: _. + +* Menu: + +* Arrows:: List of arrows. +* \boldmath & \unboldmath:: Symbols in boldface. +* Blackboard bold:: Doublestruck characters. +* Calligraphic:: Cursive characters. +* Delimiters:: Parentheses, braces, etc. +* Dots:: Ellipses, etc. +* Greek letters:: List of Greek letters. + + +File: latex2e.info, Node: Arrows, Next: \boldmath & \unboldmath, Up: Math symbols + +16.2.1 Arrows +------------- + +These are the arrows that come with standard LaTeX. The 'latexsym' and +'amsfonts' packages contain many more. + +Symbol Command +--------------------------------------------------------------------------- +⇓ '\Downarrow' +↓ '\downarrow' +↩ '\hookleftarrow' +↪ '\hookrightarrow' +← '\leftarrow' +⇐ '\Leftarrow' +⇔ '\Leftrightarrow' +↔ '\leftrightarrow' +⟵ '\longleftarrow' +⟸ '\Longleftarrow' +⟷ '\longleftrightarrow' +⟺ '\Longleftrightarrow' +⟼ '\longmapsto' +⟹ '\Longrightarrow' +⟶ '\longrightarrow' +↦ '\mapsto' +↗ '\nearrow' +↖ '\nwarrow' +⇒ '\Rightarrow' +→ '\rightarrow', or '\to' +↘ '\searrow' +↙ '\swarrow' +↑ '\uparrow' +⇑ '\Uparrow' +↕ '\updownarrow' +⇕ '\Updownarrow' + + An example of the difference between '\to' and '\mapsto' is: '\( +f\colon D\to C \) given by \( n\mapsto n^2 \)'. + + For commutative diagrams there are a number of packages, including +'tikz-cd' and 'amscd'. + + +File: latex2e.info, Node: \boldmath & \unboldmath, Next: Blackboard bold, Prev: Arrows, Up: Math symbols + +16.2.2 '\boldmath' & '\unboldmath' +---------------------------------- + +Synopsis (used in paragraph mode or LR mode): + + \boldmath \( MATH \) + +or + + \unboldmath \( MATH \) + + Declarations to change the letters and symbols in MATH to be in a +bold font, or to countermand that and bring back the regular (non-bold) +default. They must be used when not in math mode or display math mode +(*note Modes::). Both commands are fragile (*note \protect::). + + In this example each '\boldmath' command takes place inside an +'\mbox', + + we have $\mbox{\boldmath \( v \)} = 5\cdot\mbox{\boldmath \( u \)$}$ + +which means '\boldmath' is only called in a text mode, here LR mode, and +explains why LaTeX must switch to math mode to set 'v' and 'u'. + + If you use either command inside math mode, as with 'Trouble: \( +\boldmath x \)', then you get something like 'LaTeX Font Warning: +Command \boldmath invalid in math mode on input line 11' and 'LaTeX Font +Warning: Command \mathversion invalid in math mode on input line 11'. + + There are many issues with '\boldmath'. New documents should use the +'bm' package provided by the LaTeX Project team. A complete description +is outside the scope of this document (see the full documentation on +CTAN) but even this small example + + \usepackage{bm} % in preamble + ... + we have $\bm{v} = 5\cdot\bm{u}$ + +shows that it is an improvement over '\boldmath'. + + +File: latex2e.info, Node: Blackboard bold, Next: Calligraphic, Prev: \boldmath & \unboldmath, Up: Math symbols + +16.2.3 Blackboard bold +---------------------- + +Synopsis: + + \usepackage{amssymb} % in preamble + ... + \mathbb{UPPERCASE-LETTER} + + Provide blackboard bold symbols, sometimes also known as doublestruck +letters, used to denote number sets such as the natural numbers, the +integers, etc. + + Here + + \( \forall n \in \mathbb{N}, n^2 \geq 0 \) + +the '\mathbb{N}' gives blackboard bold symbol ℕ, representing the +natural numbers. + + If you use other than an uppercase letter then you do not get an +error but you get strange results, including unexpected characters. + + There are packages that give access to symbols other than just the +capital letters; look on CTAN. + + +File: latex2e.info, Node: Calligraphic, Next: Delimiters, Prev: Blackboard bold, Up: Math symbols + +16.2.4 Calligraphic +------------------- + +Synopsis: + + \mathcal{UPPERCASE-LETTERS} + + Use a script-like font. + + In this example the graph identifier is output in a cursive font. + + Let the graph be \( \mathcal{G} \). + + If you use something other than an uppercase letter then you do not +get an error but you also do not get math calligraphic output. For +instance, '\mathcal{g}' outputs a close curly brace symbol. + + +File: latex2e.info, Node: Delimiters, Next: Dots, Prev: Calligraphic, Up: Math symbols + +16.2.5 Delimiters +----------------- + +Delimiters are parentheses, braces, or other characters used to mark the +start and end of subformulas. This formula has three sets of +parentheses delimiting the three subformulas. + + (z-z_0)^2 = (x-x_0)^2 + (y-y_0)^2 + +The delimiters do not need to match, so you can enter '\( [0,1) \)'. + + Here are the common delimiters: + +DelimiterCommand Name +------------------------------------------------------ +( '(' Left parenthesis +) ')' Right parenthesis +\} '{' or Left brace + '\lbrace' +\{ '}' or Right brace + '\rbrace' +[ '[' or Left bracket + '\lbrack' +] ']' or Right bracket + '\rbrack' +⌊ '\lfloor' Left floor bracket +⌋ '\rfloor' Right floor bracket +⌈ '\lceil' Left ceiling bracket +⌉ '\rceil' Right ceiling bracket +⟨ '\langle' Left angle bracket +⟩ '\rangle' Right angle bracket +/ '/' Slash, or forward slash +\ '\backslash' Reverse slash, or backslash +| '|' or Vertical bar + '\vert' +‖ '\|' or Double vertical bar + '\Vert' + + The 'mathtools' package allows you to create commands for paired +delimiters. For instance, if you put +'\DeclarePairedDelimiter\abs{\lvert}{\rvert}' in your preamble then you +get two commands for single-line vertical bars (they only work in math +mode). The starred form, such as '\abs*{\frac{22}{7}}', has the height +of the vertical bars match the height of the argument. The unstarred +form, such as '\abs{\frac{22}{7}}', has the bars fixed at a default +height. This form accepts an optional argument, as in '\abs[SIZE +COMMAND]{\frac{22}{7}}', where the height of the bars is given in SIZE +COMMAND, such as '\Bigg'. Using instead '\lVert' and '\rVert' as the +symbols will give you a norm symbol with the same behavior. + +* Menu: + +* \left & \right:: Automatically sized delimiters. +* \bigl & \bigr etc.:: Manually sized delimiters. + + +File: latex2e.info, Node: \left & \right, Next: \bigl & \bigr etc., Up: Delimiters + +16.2.5.1 '\left' & '\right' +........................... + +Synopsis: + + \left DELIMITER1 ... \right DELIMITER2 + + Make matching parentheses, braces, or other delimiters. LaTeX makes +the delimiters tall enough to just cover the size of the formula that +they enclose. + + This makes a unit vector surrounded by parentheses tall enough to +cover the entries. + + \begin{equation} + \left(\begin{array}{c} + 1 \\ + 0 \\ + \end{array}\right) + \end{equation} + + *Note Delimiters::, for a list of the common delimiters. + + Every '\left' must have a matching '\right'. In the above example, +leaving out the '\left(' gets the error message 'Extra \right'. Leaving +out the '\right)' gets 'You can't use `\eqno' in math mode'. + + However, DELIMITER1 and DELIMITER2 need not match. A common case is +that you want an unmatched brace, as below. Use a period, '.', as a +null delimiter. + + \begin{equation} + f(n)=\left\{\begin{array}{ll} + 1 &\mbox{--if \(n=0\)} \\ + f(n-1)+3n^2 &\mbox{--else} + \end{array}\right. + \end{equation} + +Note that to get a curly brace as a delimiter you must prefix it with a +backslash, '\{' (*note Reserved characters::). (The packages 'amsmath' +and 'mathtools' allow you to get the above construct through in a +'cases' environment.) + + The '\left ... \right' pair make a group. One consequence is that +the formula enclosed in the '\left ... \right' pair cannot have line +breaks in the output. This includes both manual line breaks and +LaTeX-generated automatic ones. In this example, LaTeX breaks the +equation to make the formula fit the margins. + + Lorem ipsum dolor sit amet + \( (a+b+c+d+e+f+g+h+i+j+k+l+m+n+o+p+q+r+s+t+u+v+w+x+y+z) \) + +But with '\left' and '\right' + + Lorem ipsum dolor sit amet + \( \left(a+b+c+d+e+f+g+h+i+j+k+l+m+n+o+p+q+r+s+t+u+v+w+x+y+z\right) \) + +LaTeX won't break the line, causing the formula to extend into the +margin. + + Because '\left ... \right' make a group, all the usual grouping rules +hold. Here, the value of '\testlength' set inside the equation will be +forgotten, and the output is '1.2pt'. + + \newlength{\testlength} \setlength{\testlength}{1.2pt} + \begin{equation} + \left( a+b=c \setlength{\testlength}{3.4pt} \right) + \the\testlength + \end{equation} + + The '\left ... \right' pair affect the horizontal spacing of the +enclosed formula, in two ways. The first is that in '\( \sin(x) = +\sin\left(x\right) \)' the one after the equals sign has more space +around the 'x'. That's because '\left( ... \right)' inserts an inner +node while '( ... )' inserts an opening node. The second way that the +pair affect the horizontal spacing is that because they form a group, +the enclosed subformula will be typeset at its natural width, with no +stretching or shrinking to make the line fit better. + + TeX scales the delimiters according to the height and depth of the +enclosed formula. Here LaTeX grows the brackets to extend the full +height of the integral. + + \begin{equation} + \left[ \int_{x=r_0}^{\infty} -G\frac{Mm}{r^2}\, dr \right] + \end{equation} + + Manual sizing is often better. For instance, although below the rule +has no depth, TeX will create delimiters that extend far below the rule. + + \begin{equation} + \left( \rule{1pt}{1cm} \right) + \end{equation} + +TeX can choose delimiters that are too small, as in '\( \left| |x|+|y| +\right| \)'. It can also choose delimiters that are too large, as here. + + \begin{equation} + \left( \sum_{0\leq i1/2}+ + \underbrace{1/5+1/6+1/7+1/8}_{>1/2}+\cdots + \end{displaymath} + + The superscript appears on top of the expression, and so can look + unconnected to the underbrace. + +'\overbrace{MATH}' + Put a brace over MATH, as with + '\overbrace{x+x+\cdots+x}^{\mbox{\(k\) times}}'. See also + '\underbrace'. + + The package 'mathtools' adds an over- and underbrace, as well as some +improvements on the braces. See the documentation on CTAN. + + +File: latex2e.info, Node: Spacing in math mode, Next: Math styles, Prev: Over- and Underlining, Up: Math formulas + +16.6 Spacing in math mode +========================= + +When typesetting mathematics, LaTeX puts in spacing according to the +normal rules for mathematics texts. If you enter 'y=m x' then LaTeX +ignores the space and in the output the m is next to the x, as y=mx. + + But LaTeX's rules occasionally need tweaking. For example, in an +integral the tradition is to put a small extra space between the 'f(x)' +and the 'dx', here done with the '\,' command: + + \int_0^1 f(x)\,dx + + LaTeX provides the commands that follow for use in math mode. Many +of these spacing definitions are expressed in terms of the math unit +"mu". It is defined as 1/18em, where the em is taken from the current +math symbols family (*note Units of length::). Thus, a '\thickspace' is +something like 5/18 times the width of a 'M'. + +'\;' + + Synonym: '\thickspace'. Normally '5.0mu plus 5.0mu'. With the + 'amsmath' package, or as of the 2020-10-01 LaTeX release, can be + used in text mode as well as math mode; otherwise, in math mode + only. + +'\negthickspace' + Normally '-5.0mu plus 2.0mu minus 4.0mu'. With the 'amsmath' + package, or as of the 2020-10-01 LaTeX release, can be used in text + mode as well as math mode; otherwise, in math mode only. + +'\:' +'\>' + Synonym: '\medspace'. Normally '4.0mu plus 2.0mu minus 4.0mu'. + With the 'amsmath' package, or as of the 2020-10-01 LaTeX release, + can be used in text mode as well as math mode; before that, in math + mode only. + +'\negmedspace' + Normally '-4.0mu plus 2.0mu minus 4.0mu'. With the 'amsmath' + package, or as of the 2020-10-01 LaTeX release, can be used in text + mode as well as math mode; before that, in math mode only. + +'\,' + Synonym: '\thinspace'. Normally '3mu', which is 1/6em. Can be + used in both math mode and text mode (*note \thinspace & + \negthinspace::). + + This space is widely used, for instance between the function and + the infinitesimal in an integral '\int f(x)\,dx' and, if an author + does this, before punctuation in a displayed equation. + + The antiderivative is + \begin{equation} + 3x^{-1/2}+3^{1/2}\,. + \end{equation} + +'\!' + Synonym: '\negthinspace'. A negative thin space. Normally '-3mu'. + With the 'amsmath' package, or as of the 2020-10-01 LaTeX release, + can be used in text mode as well as math mode; otherwise, the '\!' + command is math mode only but the '\negthinspace' command has + always also worked in text mode (*note \thinspace & + \negthinspace::). + +'\quad' + This is 18mu, that is, 1em. This is often used for space + surrounding equations or expressions, for instance for the space + between two equations inside a 'displaymath' environment. It is + available in both text and math mode. + +'\qquad' + A length of 2 quads, that is, 36mu = 2em. It is available in both + text and math mode. + +* Menu: + +* \smash:: Eliminate height or depth of a subformula. +* \phantom & \vphantom & \hphantom:: Make empty box same size as argument. +* \mathstrut:: Add some vertical space to a formula. + + +File: latex2e.info, Node: \smash, Next: \phantom & \vphantom & \hphantom, Up: Spacing in math mode + +16.6.1 '\smash' +--------------- + +Synopsis: + + \smash{SUBFORMULA} + + Typeset SUBFORMULA as if its height and depth were zero. + + In this example the exponential is so tall that without the '\smash' +command LaTeX would separate its line from the line above it, and the +uneven line spacing might be unsightly. + + To compute the tetration $\smash{2^{2^{2^2}}}$, + evaluate from the top down, as $2^{2^4}=2^{16}=65536$. + + (Because of the '\smash' the printed expression could run into the +line above so you may want to wait until the final version of the +document to make such adjustments.) + + This pictures the effect of '\smash' by using '\fbox' to surround the +box that LaTeX will put on the line. The '\blackbar' command makes a +bar extending from 10 points below the baseline to 20 points above. + + \newcommand{\blackbar}{\rule[-10pt]{5pt}{30pt}} + \fbox{\blackbar} + \fbox{\smash{\blackbar}} + + The first box that LaTeX places is 20 points high and 10 points deep. +But the second box is treated by LaTeX as having zero height and zero +depth, despite that the ink printed on the page still extends well above +and below the line. + + The '\smash' command appears often in mathematics to adjust the size +of an element that surrounds a subformula. Here the first radical +extends below the baseline while the second lies just on the baseline. + + \begin{equation} + \sqrt{\sum_{0\leq k< n} f(k)} + \sqrt{\vphantom{\sum}\smash{\sum_{0\leq k< n}} f(k)} + \end{equation} + + Note the use of '\vphantom' to give the '\sqrt' command an argument +with the height of the '\sum' (*note \phantom & \vphantom & +\hphantom::). + + While most often used in mathematics, the '\smash' command can appear +in other contexts. However, it doesn't change into horizontal mode. So +if it starts a paragraph then you should first put a '\leavevmode', as +in the bottom line below. + + xxx xxx xxx + + \smash{yyy} % no paragraph indent + + \leavevmode\smash{zzz} % usual paragraph indent + + The package 'mathtools' has operators that provide even finer control +over smashing a subformula box. + + +File: latex2e.info, Node: \phantom & \vphantom & \hphantom, Next: \mathstrut, Prev: \smash, Up: Spacing in math mode + +16.6.2 '\phantom' & '\vphantom' & '\hphantom' +--------------------------------------------- + +Synopsis: + + \phantom{SUBFORMULA} + + or + + \vphantom{SUBFORMULA} + + or + + \hphantom{SUBFORMULA} + + The '\phantom' command creates a box with the same height, depth, and +width as SUBFORMULA, but empty. That is, this command causes LaTeX to +typeset the space but not fill it with the material. Here LaTeX will +put a blank line that is the correct width for the answer, but will not +show that answer. + + \begin{displaymath} + \int x^2\,dx=\mbox{\underline{$\phantom{(1/3)x^3+C}$}} + \end{displaymath} + + The '\vphantom' variant produces an invisible box with the same +vertical size as SUBFORMULA, the same height and depth, but having zero +width. And '\hphantom' makes a box with the same width as SUBFORMULA +but with zero height and depth. + + In this example, the tower of exponents in the second summand +expression is so tall that TeX places this expression further down than +its default. Without adjustment, the two summand expressions would be +at different levels. The '\vphantom' in the first expression tells TeX +to leave as much vertical room as it does for the tower, so the two +expressions come out at the same level. + + \begin{displaymath} + \sum_{j\in\{0,\ldots\, 10\}\vphantom{3^{3^{3^j}}}} + \sum_{i\in\{0,\ldots\, 3^{3^{3^j}}\}} i\cdot j + \end{displaymath} + + These commands are often used in conjunction with '\smash'. *Note +\smash::, which includes another example of '\vphantom'. + + The three phantom commands appear often but note that LaTeX provides +a suite of other commands to work with box sizes that may be more +convenient, including '\makebox' (*note \mbox & \makebox::) as well as +'\settodepth' (*note \settodepth::), '\settoheight' (*note +\settoheight::), and '\settowidth' (*note \settowidth::). In addition, +the 'mathtools' package has many commands that offer fine-grained +control over spacing. + + All three commands produce an ordinary box, without any special +mathematics status. So to do something like attaching a superscript you +should give it such a status, for example with the '\operatorname' +command from the package 'amsmath'. + + While most often used in mathematics, these three can appear in other +contexts. However, they don't cause LaTeX to change into horizontal +mode. So if one of these starts a paragraph then you should prefix it +with '\leavevmode'. + + +File: latex2e.info, Node: \mathstrut, Prev: \phantom & \vphantom & \hphantom, Up: Spacing in math mode + +16.6.3 '\mathstrut' +------------------- + +Synopsis: + + \mathstrut + + The analogue of '\strut' for mathematics. *Note \strut::. + + The input '$\sqrt{x} + \sqrt{x^i}$' gives output where the second +radical is taller than the first. To add extra vertical space without +any horizontal space, so that the two have the same height, use +'$\sqrt{x\mathstrut} + \sqrt{x^i\mathstrut}$'. + + The '\mathstrut' command adds the vertical height of an open +parenthesis, '(', but no horizontal space. It is defined as +'\vphantom{(}', so see *note \phantom & \vphantom & \hphantom:: for +more. An advantage over '\strut' is that '\mathstrut' adds no depth, +which is often the right thing for formulas. Using the height of an +open parenthesis is just a convention; for complete control over the +amount of space, use '\rule' with a width of zero. *Note \rule::. + + +File: latex2e.info, Node: Math styles, Next: Math miscellany, Prev: Spacing in math mode, Up: Math formulas + +16.7 Math styles +================ + +TeX's rules for typesetting a formula depend on the context. For +example, inside a displayed equation, the input '\sum_{0\leq +i and '\' are equivalent +to backslash-space, '\ '. + + Please also note that in order to allow source code indentation, +under normal circumstances, TeX ignores leading blanks in a line. So +the following prints 'one word': + + one + word + +where the white space between 'one' and 'word' is produced by the +newline after 'one', not by the space before 'word'. + + +File: latex2e.info, Node: ~, Next: \thinspace & \negthinspace, Prev: \(SPACE), Up: Spaces + +19.7 '~' +======== + +Synopsis: + + BEFORE~AFTER + + The "tie" character, '~', produces a space between BEFORE and AFTER +at which the line will not be broken. By default the white space has +length 3.33333pt plus 1.66666pt minus 1.11111pt (*note Lengths::). + + Note that the word 'tie' has this meaning in the TeX/Texinfo +community; this differs from the typographic term "tie", which is a +diacritic in the shape of an arc, called a "tie-after" accent in 'The +TeXbook'. + + Here LaTeX will not break the line between the final two words: + + Thanks to Prof.~Lerman. + +In addition, despite the period, LaTeX does not use the end-of-sentence +spacing (*note \@::). + + Ties prevent the end of line separation of things where that could +cause confusion. They also still allow hyphenation (of either of the +tied words), so they are generally preferable to putting consecutive +words in an '\mbox' (*note \mbox & \makebox::). + + They are also matters of taste, sometimes alarmingly dogmatic taste, +among readers. Nevertheless, here are some usage models, many of them +from 'The TeXbook'. + + * Between an enumerator label and number, such as in references: + 'Chapter~12', or 'Theorem~\ref{th:Wilsons}', or + 'Figure~\ref{fig:KGraph}'. + + * When cases are enumerated inline: '(b)~Show that $f(x)$ is + (1)~continuous, and (2)~bounded'. + + * Between a number and its unit: '$745.7.8$~watts' (the 'siunitx' + package has a special facility for this) or '144~eggs'. This + includes between a month and day number in a date: 'October~12' or + '12~Oct'. In general, in any expressions where numbers and + abbreviations or symbols are separated by a space: 'AD~565', or + '2:50~pm', or 'Boeing~747', or '268~Plains Road', or + '\$$1.4$~billion'. Other common choices here are a thin space + (*note \thinspace & \negthinspace::) and no space at all. + + * When mathematical phrases are rendered in words: 'equals~$n$', or + 'less than~$\epsilon$', or 'given~$X$', or 'modulo~$p^e$ for all + large~$n$' (but compare 'is~$15$' with 'is $15$~times the height'). + Between mathematical symbols in apposition with nouns: + 'dimension~$d$' or 'function~$f(x)$' (but compare 'with length + $l$~or more'). When a symbol is a tightly bound object of a + preposition: 'of~$x$', or 'from $0$ to~$1$', or 'in common + with~$m$'. + + * Between symbols in series: '$1$,~$2$, or~$3$' or '$1$,~$2$, + \ldots,~$n$'. + + * Between a person's given names and between multiple surnames: + 'Donald~E. Knuth', or 'Luis~I. Trabb~Pardo', or 'Charles~XII'--but + you must give TeX places to break the line so you might do 'Charles + Louis Xavier~Joseph de~la Vall\'ee~Poussin'. + + +File: latex2e.info, Node: \thinspace & \negthinspace, Next: \/, Prev: ~, Up: Spaces + +19.8 '\thinspace' & '\negthinspace' +=================================== + +Synopsis, one of: + + \thinspace + \negthinspace + + These produce unbreakable and unstretchable spaces of 1/6em and +-1/6em, respectively. These are the text mode equivalents of '\,' and +'\!' (*note Spacing in math mode/\thinspace::). + + You can use '\,' as a synonym for '\thinspace' in text mode. + + One common use of '\thinspace' is as the space between nested quotes: + + Killick replied, ``I heard the Captain say, `Ahoy there.'\thinspace'' + +Another use is that some style guides call for a '\thinspace' between an +ellipsis and a sentence ending period (other style guides, think the +three dots and/or four dots are plenty). Another style-specific use is +between initials, as in 'D.\thinspace E.\ Knuth'. + + LaTeX provides a variety of similar spacing commands for math mode +(*note Spacing in math mode::). With the 'amsmath' package, or as of +the 2020-10-01 LaTeX release, they can be used in text mode as well as +math mode, including '\!' for '\negthinspace'; but otherwise, they are +available only in math mode. + + +File: latex2e.info, Node: \/, Next: \hrulefill & \dotfill, Prev: \thinspace & \negthinspace, Up: Spaces + +19.9 '\/' +========= + +Synopsis: + + BEFORE-CHARACTER\/AFTER-CHARACTER + + Insert an "italic correction", a small space defined by the font +designer for each character (possibly zero), to avoid the character +colliding with whatever follows. When you use '\/', LaTeX takes the +correction from the font metric file, scales it by any scaling that has +been applied to the font, and then inserts that much horizontal space. + + Here, were it not for the '\/', the BEFORE-CHARACTER italic f would +hit the AFTER-CHARACTER roman H + + \newcommand{\companylogo}{{\it f}\/H} + +because the italic letter f leans far to the right. + + If AFTER-CHARACTER is a period or comma then don't insert an italic +correction since those punctuation symbols are so low to the baseline +already. However, with semicolons or colons, as well as with normal +letters, the italic correction can help. It is typically used between a +switch from italic or slanted fonts to an upright font. + + When you use commands such as '\emph' and '\textit' and '\textsl' to +change fonts, LaTeX automatically inserts the italic correction when +needed (*note Font styles::). However, declarations such as '\em' and +'\itshape' and '\slshape' do not automatically insert italic +corrections. + + Upright characters can also have an italic correction. An example +where this is needed is the name 'pdf\/\TeX'. However, most upright +characters have a zero italic correction. Some font creators do not +include italic correction values even for italic fonts. + + Technically, LaTeX uses another font-specific value, the so-called +"slant parameter" (namely '\fontdimen1'), to determine whether to +possibly insert an italic correction, rather than tying the action to +particular font commands. + + There is no concept of italic correction in math mode; math spacing +is done in a different way. + + +File: latex2e.info, Node: \hrulefill & \dotfill, Next: \bigskip & \medskip & \smallskip, Prev: \/, Up: Spaces + +19.10 '\hrulefill' & '\dotfill' +=============================== + +Synopsis, one of: + + \hrulefill + \dotfill + + Produce an infinite horizontal rubber length (*note Lengths::) that +LaTeX fills with a rule (that is, a line) or with dots, instead of white +space. + + This outputs a line 2 inches long. + + Name:~\makebox[2in]{\hrulefill} + +This example, when placed between blank lines, creates a paragraph that +is left and right justified and where the middle is filled with evenly +spaced dots. + + \noindent John Aubrey, RN \dotfill{} Melbury Lodge + + To make the rule or dots go to the line's end use '\null' at the +start or end. + + To change the rule's thickness, copy the definition and adjust it, as +here + + \renewcommand{\hrulefill}{% + \leavevmode\leaders\hrule height 1pt\hfill\kern0pt } + +which changes the default thickness of 0.4pt to 1pt. Similarly, adjust +the dot spacing as with + + \renewcommand{\dotfill}{% + \leavevmode\cleaders\hbox to 1.00em{\hss .\hss }\hfill\kern0pt } + +which changes the default length of 0.33em to 1.00em. + + This example produces a line for a signature. + + \begin{minipage}{4cm} + \centering + \hrulefill\\ + Signed + \end{minipage} + +The line is 4cm long. + + +File: latex2e.info, Node: \bigskip & \medskip & \smallskip, Next: \bigbreak & \medbreak & \smallbreak, Prev: \hrulefill & \dotfill, Up: Spaces + +19.11 '\bigskip' & '\medskip' & '\smallskip' +============================================ + +Synopsis, one of: + + \bigskip + \medskip + \smallskip + + Produce an amount of vertical space, large or medium-sized or small. +These commands are fragile (*note \protect::). + + Here the skip suggests the passage of time (from The Golden Ocean by +O'Brian). + + Mr Saumarez would have something rude to say to him, no doubt: he + was at home again, and it was delightful. + + \bigskip + ``A hundred and fifty-seven miles and one third, in twenty-four hours,'' + said Peter. + + Each command is associated with a length defined in the document +class file. + +'\bigskip' + The same as '\vspace{\bigskipamount}', ordinarily about one line + space, with stretch and shrink. The default for the 'book' and + 'article' classes is '12pt plus 4pt minus 4pt'. + +'\medskip' + The same as '\vspace{\medskipamount}', ordinarily about half of a + line space, with stretch and shrink. The default for the 'book' + and 'article' classes is '6pt plus 2pt minus 2pt'. + +'\smallskip' + The same as '\vspace{\smallskipamount}', ordinarily about a quarter + of a line space, with stretch and shrink. The default for the + 'book' and 'article' classes is '3pt plus 1pt minus 1pt'. + + Because each command is a '\vspace', if you use it in mid-paragraph +then it will insert its vertical space between the line in which you use +it and the next line, not necessarily at the place that you use it. So +these are best between paragraphs. + + The commands '\bigbreak', '\medbreak', and '\smallbreak' are similar +but also suggest to LaTeX that this is a good place to put a page break +(*note \bigbreak & \medbreak & \smallbreak::. + + +File: latex2e.info, Node: \bigbreak & \medbreak & \smallbreak, Next: \strut, Prev: \bigskip & \medskip & \smallskip, Up: Spaces + +19.12 '\bigbreak' & '\medbreak' & '\smallbreak' +=============================================== + +Synopsis, one of: + + \bigbreak + \medbreak + \smallbreak + + Produce a vertical space that is big or medium-sized or small, and +suggest to LaTeX that this is a good place to break the page. (The +associated penalties are -200, -100, and -50.) + + *Note \bigskip & \medskip & \smallskip::, for more. These commands +produce the same vertical space but differ in that they also remove a +preceding vertical space if it is less than what they would insert (as +with '\addvspace'). In addition, they terminate a paragraph where they +are used: this example + + abc\bigbreak def ghi + + jkl mno pqr + +will output three paragraphs, the first ending in 'abc' and the second +starting, after an extra vertical space and a paragraph indent, with +'def'. + + +File: latex2e.info, Node: \strut, Next: \vspace, Prev: \bigbreak & \medbreak & \smallbreak, Up: Spaces + +19.13 '\strut' +============== + +Synopsis: + + \strut + + Ensure that the current line has height at least '0.7\baselineskip' +and depth at least '0.3\baselineskip'. Essentially, LaTeX inserts into +the line a rectangle having zero width, +'\rule[-0.3\baselineskip]{0pt}{\baselineskip}' (*note \rule::). The +'\baselineskip' changes with the current font or fontsize. + + In this example the '\strut' keeps the box inside the frame from +having zero height. + + \setlength{\fboxsep}{0pt}\framebox[2in]{\strut} + + This example has four lists. In the first there is a much bigger gap +between items 2 and 3 than there is between items 1 and 2. The second +list fixes that with a '\strut' at the end of its first item's second +line. + + \setlength{\fboxsep}{0pt} + \noindent\begin{minipage}[t]{0.2\linewidth} + \begin{enumerate} + \item \parbox[t]{15pt}{test \\ test} + \item test + \item test + \end{enumerate} + \end{minipage}% + \begin{minipage}[t]{0.2\linewidth} + \begin{enumerate} + \item \parbox[t]{15pt}{test \\ test\strut} + \item test + \item test + \end{enumerate} + \end{minipage}% + \begin{minipage}[t]{0.2\linewidth} + \begin{enumerate} + \item \fbox{\parbox[t]{15pt}{test \\ test}} + \item \fbox{test} + \item \fbox{test} + \end{enumerate} + \end{minipage}% + \begin{minipage}[t]{0.2\linewidth} + \begin{enumerate} + \item \fbox{\parbox[t]{15pt}{test \\ test\strut}} + \item \fbox{test} + \item \fbox{test} + \end{enumerate} + \end{minipage}% + +The final two lists use '\fbox' to show what's happening. The first +item '\parbox' of the third list goes only to the bottom of its second +'test', which happens not have any characters that descend below the +baseline. The fourth list adds the strut that gives the needed extra +below-baseline space. + + The '\strut' command is often useful in graphics, such as in 'TikZ' +or 'Asymptote'. For instance, you may have a command such as +'\graphnode{NODE-NAME}' that fits a circle around NODE-NAME. However, +unless you are careful the NODE-NAME's 'x' and 'y' will produce +different-diameter circles because the characters are different sizes. +A careful '\graphnode' might insert a '\strut', then NODE-NAME, and then +draw the circle. + + The general approach of using a zero width '\rule' is useful in many +circumstances. In this table, the zero-width rule keeps the top of the +first integral from hitting the '\hline'. Similarly, the second rule +keeps the second integral from hitting the first. + + \begin{tabular}{rl} + \textsc{Integral} &\textsc{Value} \\ + \hline + $\int_0^x t\, dt$ &$x^2/2$ \rule{0em}{2.5ex} \\ + $\int_0^x t^2\, dt$ &$x^3/3$ \rule{0em}{2.5ex} + \end{tabular} + +(Although the line-ending double backslash command has an available +optional argument to change the corresponding baseline skip, that won't +solve this issue. Changing the first double backslash to something like +'\\[2.5ex]' will put more room between the header line and the '\hline' +rule, and the integral would still hit the rule.) + + +File: latex2e.info, Node: \vspace, Next: \vfill, Prev: \strut, Up: Spaces + +19.14 '\vspace' +=============== + +Synopsis, one of: + + \vspace{LENGTH} + \vspace*{LENGTH} + + Add the vertical space LENGTH. The LENGTH can be positive, negative, +or zero. It is a rubber length--it may contain a 'plus' or 'minus' +component (*note Lengths::). + + This puts space between the two paragraphs. + + And I slept. + + \vspace{1ex plus 0.5ex} + The new day dawned cold. + +(*Note \bigskip & \medskip & \smallskip::, for common inter-paragraph +spaces.) + + The '*'-form inserts vertical space that is non-discardable. More +precisely, LaTeX discards vertical space at a page break and the +'*'-form causes the space to stay. This example leaves space between +the two questions. + + Question: Find the integral of \( 5x^4+5 \). + + \vspace*{2cm plus 0.5cm} + Question: Find the derivative of \( x^5+5x+9 \). + +That space will be present even if the page break happens to fall +between the questions. + + If you use '\vspace' in the middle of a paragraph (i.e., in +horizontal mode) then the space is inserted after the line containing +the '\vspace' command; it does not start a new paragraph at the +'\vspace' command. + + In this example the two questions will be evenly spaced vertically on +the page, with at least one inch of space below each. + + \begin{document} + 1) Who put the bomp in the bomp bah bomp bah bomp? + \vspace{1in plus 1fill} + + 2) Who put the ram in the rama lama ding dong? + \vspace{1in plus 1fill} + \end{document} + + +File: latex2e.info, Node: \vfill, Next: \addvspace, Prev: \vspace, Up: Spaces + +19.15 '\vfill' +============== + +Synopsis: + + \vfill + + End the current paragraph and insert a vertical rubber length that is +infinite, so it can stretch or shrink as far as needed (*note +Lengths::). + + It is often used in the same way as '\vspace{\fill}', except that +'\vfill' ends the current paragraph whereas '\vspace{\fill}' adds the +infinite vertical space below its line, irrespective of the paragraph +structure. In both cases that space will disappear at a page boundary; +to circumvent this see the starred option in *note \vspace::. + + In this example the page is filled, so the top and bottom lines +contain the text 'Lost Dog!' and the second 'Lost Dog!' is exactly +halfway between them. + + \begin{document} + Lost Dog! + \vfill + Lost Dog! % perfectly in the middle + \vfill + Lost Dog! + \end{document} + + +File: latex2e.info, Node: \addvspace, Prev: \vfill, Up: Spaces + +19.16 '\addvspace' +================== + +Synopsis: + + \addvspace{VERT-LENGTH} + + Add a vertical space of VERT-LENGTH. However, if there are two or +more '\addvspace''s in a sequence then together they only add the space +needed to make the natural length equal to the maximum of the +VERT-LENGTH's in that sequence. This command is fragile (*note +\protect::). The VERT-LENGTH is a rubber length (*note Lengths::). + + This example illustrates. The 'picture' draws a scale over which to +rules are placed. In a standard LaTeX article the length +'\baselineskip' is 12pt. As shown by the scale, the two rules are 22pt +apart: the sum of the '\baselineskip' and the 10pt from the first +'\addvspace'. + + \documentclass{article} + \usepackage{color} + \begin{document} + \setlength{\unitlength}{2pt}% + \noindent\begin{picture}(0,0)% + \multiput(0,0)(0,-1){25}{{\color{blue}\line(1,0){1}}} + \multiput(0,0)(0,-5){6}{{\color{red}\line(1,0){2}}} + \end{picture}% + \rule{0.25\linewidth}{0.1pt}% + \par\addvspace{10pt}% \addvspace{20pt}% + \par\noindent\rule{0.25\linewidth}{0.1pt}% + \end{document} + +Now uncomment the second '\addvspace'. It does not make the gap 20pt +longer; instead the gap is the sum of '\baselineskip' and 20pt. So +'\addvspace' in a sense does the opposite of its name--it makes sure +that multiple vertical spaces do not accumulate, but instead that only +the largest one is used. + + LaTeX uses this command to adjust the vertical space above or below +an environment that starts a new paragraph. For instance, a 'theorem' +environment begins and ends with '\addvspace' so that two consecutive +'theorem''s are separated by one vertical space, not two. + + A error 'Something's wrong--perhaps a missing \item' pointing to an +'\addvspace' means that you were not in vertical mode when you hit this +command. One way to change that is to precede '\addvspace' with a +'\par' command (*note \par::), as in the above example. + + +File: latex2e.info, Node: Boxes, Next: Color, Prev: Spaces, Up: Top + +20 Boxes +******** + +At its core, LaTeX puts things in boxes and then puts the boxes on a +page. So these commands are central. + + There are many packages on CTAN that are useful for manipulating +boxes. One useful adjunct to the commands here is 'adjustbox'. + +* Menu: + +* \mbox & \makebox:: Horizontal boxes. +* \fbox & \framebox:: Put a frame around a box. +* \parbox:: Box with text in paragraph mode. +* \raisebox:: Raise or lower text. +* \sbox & \savebox:: Like '\makebox' but save the text for later. +* lrbox:: Environment version of '\sbox'. +* \usebox:: Print saved text. + + +File: latex2e.info, Node: \mbox & \makebox, Next: \fbox & \framebox, Up: Boxes + +20.1 '\mbox' & '\makebox' +========================= + +Synopsis, one of: + + \mbox{TEXT} + \makebox{TEXT} + \makebox[WIDTH]{TEXT} + \makebox[WIDTH][POSITION]{TEXT} + + Create a box, a container for material. The TEXT is typeset in LR +mode (*note Modes::) so it is not broken into lines. The '\mbox' +command is robust, while '\makebox' is fragile (*note \protect::). + + Because 'text' is not broken into lines, you can use '\mbox' to +prevent hyphenation. In this example, LaTeX will not hyphenate the tank +name, 'T-34'. + + The soviet tank \mbox{T-34} is a symbol of victory against nazism. + + The first two command synopsis versions, '\mbox' and '\makebox', are +roughly equivalent. They create a box just wide enough to contain the +TEXT. (They are like plain TeX's '\hbox'.) + + In the third version the optional argument WIDTH specifies the width +of the box. Note that the space occupied by the text need not equal the +width of the box. For one thing, TEXT can be too small; this creates a +full-line box + + \makebox[\linewidth]{Chapter Exam} + +with 'Chapter Exam' centered. But TEXT can also be too wide for WIDTH. +See the example below of zero-width boxes. + + In the WIDTH argument you can use the following lengths that refer to +the dimension of the box that LaTeX gets on typesetting TEXT: '\depth', +'\height', '\width', '\totalheight' (this is the box's height plus its +depth). For example, to make a box with the text stretched to double +the natural size you can say this. + + \makebox[2\width]{Get a stretcher} + + For the fourth command synopsis version the optional argument +POSITION gives position of the text within the box. It may take the +following values: + +'c' + The TEXT is centered (default). + +'l' + The TEXT is flush left. + +'r' + Flush right. + +'s' + Stretch the interword space in TEXT across the entire WIDTH. The + TEXT must contain stretchable space for this to work. For + instance, this could head a press release: + '\noindent\makebox[\textwidth][s]{\large\hfil IMMEDIATE\hfil + RELEASE\hfil}' + + A common use of '\makebox' is to make zero-width text boxes. This +puts the value of the quiz questions to the left of those questions. + + \newcommand{\pts}[1]{\makebox[0em][r]{#1 points\hspace*{1em}}} + \pts{10}What is the air-speed velocity of an unladen swallow? + + \pts{90}An African or European swallow? + + + The right edge of the output '10 points ' (note the ending space +after 'points') will be just before the 'What'. You can use '\makebox' +similarly when making graphics, such as in 'TikZ' or 'Asymptote', where +you put the edge of the text at a known location, regardless of the +length of that text. + + For boxes with frames see *note \fbox & \framebox::. For colors +see *note Colored boxes::. + + There is a related version of '\makebox' that is used within the +'picture' environment, where the length is given in terms of +'\unitlength' (*note \makebox (picture)::). + + As TEXT is typeset in LR mode, neither a double backslash '\\' nor +'\par' will give you a new line; for instance '\makebox{abc def \\ ghi}' +outputs 'abc defghi' while '\makebox{abc def \par ghi}' outputs 'abc def +ghi', both on a single line. To get multiple lines see *note \parbox:: +and *note minipage::. + + +File: latex2e.info, Node: \fbox & \framebox, Next: \parbox, Prev: \mbox & \makebox, Up: Boxes + +20.2 '\fbox' & '\framebox' +========================== + +Synopses, one of: + + \fbox{TEXT} + \framebox{TEXT} + \framebox[WIDTH]{TEXT} + \framebox[WIDTH][POSITION]{TEXT} + + Create a box with an enclosing frame, four rules surrounding the +TEXT. These commands are the same as '\mbox' and '\makebox' except for +the frame (*note \mbox & \makebox::). The '\fbox' command is robust, +the '\framebox' command is fragile (*note \protect::). + + \fbox{Warning! No work shown, no credit given.} + +LaTeX puts the text into a box, the text cannot be hyphenated. Around +that box, separated from it by a small gap, are four rules making a +frame. + + The first two command invocations, '\fbox{...}' and '\framebox{...}', +are roughly the same. As to the third and fourth invocations, the +optional arguments allow you to specify the box width as WIDTH and the +position of the text inside that box as POSITION. *Note \mbox & +\makebox::, for the full description but here is an example creating an +empty box that is 1/4in wide. + + \setlength{\fboxsep}{0pt}\framebox[0.25in]{\strut}} + +The '\strut' ensures a total height of '\baselineskip' (*note \strut::). + + These parameters determine the frame layout. + +'\fboxrule' + The thickness of the rules around the enclosed box. The default is + 0.2pt. Change it with a command such as + '\setlength{\fboxrule}{0.8pt}' (*note \setlength::). + +'\fboxsep' + The distance from the frame to the enclosed box. The default is + 3pt. Change it with a command such as '\setlength{\fboxsep}{0pt}' + (*note \setlength::). Setting it to 0pt is useful sometimes: this + will put a frame around the picture with no white border. + + {\setlength{\fboxsep}{0pt}% + \framebox{% + \includegraphics[width=0.5\textwidth]{prudence.jpg}}} + + The extra curly braces keep the effect of the '\setlength' local. + + As with '\mbox' and '\makebox', LaTeX will not break lines in TEXT. +But this example has LaTeX break lines to make a paragraph, and then +frame the result. + + \framebox{% + \begin{minipage}{0.6\linewidth} + My dear, here we must run as fast as we can, just to stay in place. + And if you wish to go anywhere you must run twice as fast as that. + \end{minipage}} + + *Note Colored boxes::, for colors other than black and white. + + The 'picture' environment has a version of the '\framebox' command +where the units depend on 'picture''s '\unitlength' (*note \framebox +(picture)::). + + +File: latex2e.info, Node: \parbox, Next: \raisebox, Prev: \fbox & \framebox, Up: Boxes + +20.3 '\parbox' +============== + +Synopses, one of: + + \parbox{WIDTH}{CONTENTS} + \parbox[POSITION]{WIDTH}{CONTENTS} + \parbox[POSITION][HEIGHT]{WIDTH}{CONTENTS} + \parbox[POSITION][HEIGHT][INNER-POS]{WIDTH}{CONTENTS} + + Produce a box of text that is WIDTH wide. Use this command to make a +box of small pieces of text, of a single paragraph. This command is +fragile (*note \protect::). + + \begin{picture}(0,0) + ... + \put(1,2){\parbox{1.75in}{\raggedright Because the graph is a line on + this semilog paper, the relationship is + exponential.}} + \end{picture} + + The CONTENTS are processed in a text mode (*note Modes::) so LaTeX +will break lines to make a paragraph. But it won't make multiple +paragraphs; for that, use a 'minipage' environment (*note minipage::). + + The options for '\parbox' (except for CONTENTS) are the same as those +for 'minipage'. For convenience a summary of the options is here but +see *note minipage:: for a complete description. + + There are two required arguments. The WIDTH is a rigid length (*note +Lengths::). It sets the width of the box into which LaTeX typesets +CONTENTS. The CONTENTS is the text that is placed in that box. It +should not have any paragraph-making components. + + There are three optional arguments, POSITION, HEIGHT, and INNER-POS. +The POSITION gives the vertical alignment of the "parbox" with respect +to the surrounding material. The supported values are 'c' or 'm' to +make the vertical center of the parbox lines up with the center of the +adjacent text line (this is the default), or 't' to match the top line +of the parbox with the baseline of the surrounding material, or 'b' to +match the bottom line. + + The optional argument HEIGHT overrides the natural height of the box. + + The optional argument INNER-POS controls the placement of CONTENT +inside the 'parbox'. Its default is the value of POSITION. Its +possible values are: 't' to put the CONTENT at the top of the box, 'c' +to put it in the vertical center, 'b' to put it at the bottom of the +box, and 's' to stretch it out vertically (for this, the text must +contain vertically stretchable space). + + +File: latex2e.info, Node: \raisebox, Next: \sbox & \savebox, Prev: \parbox, Up: Boxes + +20.4 '\raisebox' +================ + +Synopsis, one of: + + \raisebox{DISTANCE}{TEXT} + \raisebox{DISTANCE}[HEIGHT]{TEXT} + \raisebox{DISTANCE}[HEIGHT][DEPTH]{TEXT} + + Raise or lower TEXT. This command is fragile (*note \protect::). + + This example makes a command for denoting the restriction of a +function by lowering the vertical bar symbol. + + \newcommand*\restricted[1]{\raisebox{-.5ex}{$|$}_{#1}} + $f\restricted{A}$ + + The first mandatory argument DISTANCE specifies how far to raise the +second mandatory argument TEXT. This is a rigid length (*note +Lengths::). If it is negative then it lowers TEXT. The TEXT is +processed in LR mode so it cannot contain line breaks (*note Modes::). + + The optional arguments HEIGHT and DEPTH are dimensions. If they are +specified, they override the natural height and depth of the box LaTeX +gets by typesetting TEXT. + + In the arguments DISTANCE, HEIGHT, and DEPTH you can use the +following lengths that refer to the dimension of the box that LaTeX gets +on typesetting TEXT: '\depth', '\height', '\width', '\totalheight' (this +is the box's height plus its depth). + + This will align two graphics on their top (*note Graphics::). + + \usepackage{graphicx,calc} % in preamble + ... + \begin{center} + \raisebox{1ex-\height}{% + \includegraphics[width=0.4\linewidth]{lion.png}} + \qquad + \raisebox{1ex-\height}{% + \includegraphics[width=0.4\linewidth]{meta.png}} + \end{center} + +The first '\height' is the height of 'lion.png' while the second is the +height of 'meta.png'. + + +File: latex2e.info, Node: \sbox & \savebox, Next: lrbox, Prev: \raisebox, Up: Boxes + +20.5 '\sbox' & '\savebox' +========================= + +Synopsis, one of: + + \sbox{BOX-CMD}{TEXT} + \savebox{BOX-CMD}{TEXT} + \savebox{BOX-CMD}[WIDTH]{TEXT} + \savebox{BOX-CMD}[WIDTH][POS]{TEXT} + + Typeset TEXT just as with '\makebox' (*note \mbox & \makebox::) +except that LaTeX does not output it but instead saves it in a box +register referred to by a variable named BOX-CMD. The variable name +BOX-CMD begins with a backslash, '\'. You must have previously +allocated the box register BOX-CMD with '\newsavebox' (*note +\newsavebox::). The '\sbox' command is robust while '\savebox' is +fragile (*note \protect::). + + This creates and uses a box register. + + \newsavebox{\fullname} + \sbox{\fullname}{John Jacob Jingleheimer Schmidt} + ... + \usebox{\fullname}! His name is my name, too! + Whenever we go out, the people always shout! + There goes \usebox{\fullname}! Ya da da da da da da. + +One advantage of using and reusing a box register over a '\newcommand' +macro variable is efficiency, that LaTeX need not repeatedly retypeset +the contents. See the example below. + + The first two command invocations, '\sbox{BOX-CMD}{TEXT}' and +'\savebox{BOX-CMD}{TEXT}', are roughly equivalent. As to the third and +fourth, the optional arguments allow you to specify the box width as +WIDTH, and the position of the text inside that box as POSITION. *Note +\mbox & \makebox::, for the full description. + + In the '\sbox' and '\savebox' commands the TEXT is typeset in LR mode +so it does not have line breaks (*note Modes::). If you use these then +LaTeX doesn't give you an error but it ignores what you want: if you +enter '\sbox{\newreg}{test \\ test}' and '\usebox{\newreg}' then you get +'testtest', while if you enter '\sbox{\newreg}{test \par test}' and +'\usebox{\newreg}' then you get 'test test', but no error or warning. +To fix this use a '\parbox' or 'minipage' as here. + + \newsavebox{\areg} + \savebox{\areg}{% + \begin{minipage}{\linewidth} + \begin{enumerate} + \item First item + \item Second item + \end{enumerate} + \end{minipage}} + ... + \usebox{\areg} + + As an example of the efficiency of reusing a register's contents, +this puts the same picture on each page of the document by putting it in +the header. LaTeX only typesets it once. + + \usepackage{graphicx} % all this in the preamble + \newsavebox{\sealreg} + \savebox{\sealreg}{% + \setlength{\unitlength}{1in}% + \begin{picture}(0,0)% + \put(1.5,-2.5){% + \begin{tabular}{c} + \includegraphics[height=2in]{companylogo.png} \\ + Office of the President + \end{tabular}} + \end{picture}% + } + \markright{\usebox{\sealreg}} + \pagestyle{headings} + +The 'picture' environment is good for fine-tuning the placement. + + If the register '\noreg' has not already been defined then you get +something like 'Undefined control sequence. \noreg'. + + +File: latex2e.info, Node: lrbox, Next: \usebox, Prev: \sbox & \savebox, Up: Boxes + +20.6 'lrbox' +============ + +Synopsis: + + \begin{lrbox}{BOX-CMD} + TEXT + \end{lrbox} + + This is the environment form of the '\sbox' and '\savebox' commands, +and is equivalent to them. *Note \sbox & \savebox::, for the full +description. + + The TEXT inside the environment is saved in the box register referred +to by variable 'BOX-CMD'. The variable name BOX-CMD must begin with a +backslash, '\'. You must allocate this box register in advance with +'\newsavebox' (*note \newsavebox::). In this example the environment is +convenient for entering the 'tabular'. + + \newsavebox{\jhreg} + \begin{lrbox}{\jhreg} + \begin{tabular}{c} + \includegraphics[height=1in]{jh.png} \\ + Jim Hef{}feron + \end{tabular} + \end{lrbox} + ... + \usebox{\jhreg} + + +File: latex2e.info, Node: \usebox, Prev: lrbox, Up: Boxes + +20.7 '\usebox' +============== + +Synopsis: + + \usebox{BOX-CMD} + + Produce the box most recently saved in the box register BOX-CMD by +the commands '\sbox' or '\savebox', or the 'lrbox' environment. For +more information and examples, *note \sbox & \savebox::. (Note that the +variable name BOX-CMD starts with a backslash, '\'.) This command is +robust (*note \protect::). + + +File: latex2e.info, Node: Color, Next: Graphics, Prev: Boxes, Up: Top + +21 Color +******** + +You can add color to text, rules, etc. You can also have color in a box +or on an entire page and write text on top of it. + + Color support comes as an additional package. So put +'\usepackage{color}' in your document preamble to use the commands +described here. + + Many other packages also supplement LaTeX's color abilities. +Particularly worth mentioning is 'xcolor', which is widely used and +significantly extends the capabilities described here, including adding +'HTML' and 'Hsb' color models. + +* Menu: + +* Color package options:: Options when you load the standard package. +* Color models:: How colors are represented. +* Commands for color:: The available commands. + + +File: latex2e.info, Node: Color package options, Next: Color models, Up: Color + +21.1 'color' package options +============================ + +Synopsis (must be in the document preamble): + + \usepackage[COMMA-SEPARATED OPTION LIST]{color} + + When you load the 'color' package there are two kinds of available +options. + + The first specifies the "printer driver". LaTeX doesn't contain +information about different output systems but instead depends on +information stored in a file. Normally you should not specify the +driver option in the document, and instead rely on your system's +default. One advantage of this is that it makes the document portable +across systems. For completeness we include a list of the drivers. The +currently relevant ones are: 'dvipdfmx', 'dvips', 'dvisvgm', 'luatex', +'pdftex', 'xetex'. The two 'xdvi' and 'oztex' are essentially aliases +for 'dvips' (and 'xdvi' is monochrome). Ones that should not be used +for new systems are: 'dvipdf', 'dvipdfm', 'dviwin', 'dvipsone', 'emtex', +'pctexps', 'pctexwin', 'pctexhp', 'pctex32', 'truetex', 'tcidvi', 'vtex' +(and 'dviwindo' is an alias for 'dvipsone'). + + The second kind of options, beyond the drivers, are below. + +'monochrome' + Disable the color commands, so that they do not generate errors but + do not generate color either. + +'dvipsnames' + Make available a list of 68 color names that are often used, + particularly in legacy documents. These color names were + originally provided by the 'dvips' driver, giving the option name. + +'nodvipsnames' + Do not load that list of color names, saving LaTeX a tiny amount of + memory space. + + +File: latex2e.info, Node: Color models, Next: Commands for color, Prev: Color package options, Up: Color + +21.2 Color models +================= + +A "color model" is a way of representing colors. LaTeX's capabilities +depend on the printer driver. However, the 'pdftex', 'xetex', and +'luatex' printer drivers are today by far the most commonly used. The +models below work for those drivers. All but one of these is also +supported by essentially all other printer drivers used today. + + Note that color combination can be additive or subtractive. Additive +mixes colors of light, so that for instance combining full intensities +of red, green, and blue produces white. Subtractive mixes pigments, +such as with inks, so that combining full intensity of cyan, magenta, +and yellow makes black. + +'cmyk' + A comma-separated list with four real numbers between 0 and 1, + inclusive. The first number is the intensity of cyan, the second + is magenta, and the others are yellow and black. A number value of + 0 means minimal intensity, while a 1 is for full intensity. This + model is often used in color printing. It is a subtractive model. + +'gray' + A single real number between 0 and 1, inclusive. The colors are + shades of grey. The number 0 produces black while 1 gives white. + +'rgb' + A comma-separated list with three real numbers between 0 and 1, + inclusive. The first number is the intensity of the red component, + the second is green, and the third the blue. A number value of 0 + means that none of that component is added in, while a 1 means full + intensity. This is an additive model. + +'RGB' + ('pdftex', 'xetex', 'luatex' drivers) A comma-separated list with + three integers between 0 and 255, inclusive. This model is a + convenience for using 'rgb' since outside of LaTeX colors are often + described in a red-green-blue model using numbers in this range. + The values entered here are converted to the 'rgb' model by + dividing by 255. + +'named' + Colors are accessed by name, such as 'PrussianBlue'. The list of + names depends on the driver, but all support the names 'black', + 'blue', 'cyan', 'green', 'magenta', 'red', 'white', and 'yellow' + (See the 'dvipsnames' option in *note Color package options::). + + +File: latex2e.info, Node: Commands for color, Prev: Color models, Up: Color + +21.3 Commands for color +======================= + +These are the commands available with the 'color' package. + +* Menu: + +* Define colors:: Give a color a name. +* Colored text:: Text or rules in color. +* Colored boxes:: A box of color, to write over. +* Colored pages:: A whole page colored. + + +File: latex2e.info, Node: Define colors, Next: Colored text, Up: Commands for color + +21.3.1 Define colors +-------------------- + +Synopsis: + + \definecolor{NAME}{MODEL}{SPECIFICATION} + + Give the name NAME to the color. For example, after this + + \definecolor{silver}{rgb}{0.75,0.75,0.74} + +you can use that color name with 'Hi ho, \textcolor{silver}{Silver}!'. + + This example gives the color a more abstract name, so it could change +and not be misleading. + + \definecolor{logocolor}{RGB}{145,92,131} % RGB needs pdflatex + \newcommand{\logo}{\textcolor{logocolor}{Bob's Big Bagels}} + + Often a document's colors are defined in the preamble, or in the +class or style, rather than in the document body. + + +File: latex2e.info, Node: Colored text, Next: Colored boxes, Prev: Define colors, Up: Commands for color + +21.3.2 Colored text +------------------- + +Synopses: + + \textcolor{NAME}{...} + \textcolor[COLOR MODEL]{COLOR SPECIFICATION}{...} + +or + + \color{NAME} + \color[COLOR MODEL]{SPECIFICATION} + + The affected text gets the color. This line + + \textcolor{magenta}{My name is Ozymandias, king of kings:} + Look on my works, ye Mighty, and despair! + +causes the first half to be in magenta while the rest is in black. You +can use a color declared with '\definecolor' in exactly the same way +that we just used the builtin color 'magenta'. + + \definecolor{MidlifeCrisisRed}{rgb}{1.0,0.11,0.0} + I'm thinking about getting a \textcolor{MidlifeCrisisRed}{sports car}. + + The two '\textcolor' and '\color' differ in that the first is a +command form, enclosing the text to be colored as an argument. Often +this form is more convenient, or at least more explicit. The second +form is a declaration, as in 'The moon is made of {\color{green} green} +cheese', so it is in effect until the end of the current group or +environment. This is sometimes useful when writing macros or as below +where it colors everything inside the 'center' environment, including +the vertical and horizontal lines. + + \begin{center} \color{blue} + \begin{tabular}{l|r} + UL &UR \\ \hline + LL &LR + \end{tabular} + \end{center} + + You can use color in equations. A document might have this +definition in the preamble + + \definecolor{highlightcolor}{RGB}{225,15,0} + +and then contain this equation. + + \begin{equation} + \int_a^b \textcolor{highlightcolor}{f'(x)}\,dx=f(b)-f(a) + \end{equation} + + Typically the colors used in a document are declared in a class or +style but sometimes you want a one-off. Those are the second forms in +the synopses. + + Colors of \textcolor[rgb]{0.33,0.14,0.47}{Purple} and + {\color[rgb]{0.72,0.60,0.37}Gold} for the team. + + The format of COLOR SPECIFICATION depends on the color model (*note +Color models::). For instance, while 'rgb' takes three numbers, 'gray' +takes only one. + + The selection was \textcolor[gray]{0.5}{grayed out}. + + Colors inside colors do not combine. Thus + + \textcolor{green}{kind of \textcolor{blue}{blue}} + +has a final word that is blue, not a combination of blue and green. + + +File: latex2e.info, Node: Colored boxes, Next: Colored pages, Prev: Colored text, Up: Commands for color + +21.3.3 Colored boxes +-------------------- + +Synopses: + + \colorbox{NAME}{...} + \colorbox[MODEL NAME]{BOX BACKGROUND COLOR}{...} + +or + + \fcolorbox{FRAME COLOR}{BOX BACKGROUND COLOR}{...} + \fcolorbox[MODEL NAME]{FRAME COLOR}{BOX BACKGROUND COLOR}{...} + + Make a box with the stated background color. The '\fcolorbox' +command puts a frame around the box. For instance this + + Name:~\colorbox{cyan}{\makebox[5cm][l]{\strut}} + +makes a cyan-colored box that is five centimeters long and gets its +depth and height from the '\strut' (so the depth is '-.3\baselineskip' +and the height is '\baselineskip'). This puts white text on a blue +background. + + \colorbox{blue}{\textcolor{white}{Welcome to the machine.}} + + The '\fcolorbox' commands use the same parameters as '\fbox' (*note +\fbox & \framebox::), '\fboxrule' and '\fboxsep', to set the thickness +of the rule and the boundary between the box interior and the +surrounding rule. LaTeX's defaults are '0.4pt' and '3pt', respectively. + + This example changes the thickness of the border to 0.8 points. Note +that it is surrounded by curly braces so that the change ends at the end +of the second line. + + {\setlength{\fboxrule}{0.8pt} + \fcolorbox{black}{red}{Under no circumstances turn this knob.}} + + +File: latex2e.info, Node: Colored pages, Prev: Colored boxes, Up: Commands for color + +21.3.4 Colored pages +-------------------- + +Synopses: + + \pagecolor{NAME} + \pagecolor[COLOR MODEL]{COLOR SPECIFICATION} + \nopagecolor + + The first two set the background of the page, and all subsequent +pages, to the color. For an explanation of the specification in the +second form *note Colored text::. The third returns the background to +normal, which is a transparent background. (If that is not supported +use '\pagecolor{white}', although that will make a white background +rather than the default transparent background.) + + ... + \pagecolor{cyan} + ... + \nopagecolor + + +File: latex2e.info, Node: Graphics, Next: Special insertions, Prev: Color, Up: Top + +22 Graphics +*********** + +You can use graphics such as PNG or PDF files in your LaTeX document. +You need an additional package, which comes standard with LaTeX. This +example is the short how-to. + + \include{graphicx} % goes in the preamble + ... + \includegraphics[width=0.5\linewidth]{plot.pdf} + + To use the commands described here your document preamble must +contain either '\usepackage{graphicx}' or '\usepackage{graphics}'. Most +of the time, 'graphicx' is the better choice. + + Graphics come in two main types, raster and vector. LaTeX can use +both. In raster graphics the file contains an entry for each location +in an array, describing what color it is. An example is a photograph in +JPG format. In vector graphics, the file contains a list of +instructions such as 'draw a circle with this radius and that center'. +An example is a line drawing produced by the Asymptote program, in PDF +format. Generally vector graphics are more useful because you can +rescale their size without pixelation or other problems, and because +they often have a smaller size. + + There are systems particularly well-suited to make graphics for a +LaTeX document. For example, these allow you to use the same fonts as +in your document. LaTeX comes with a 'picture' environment (*note +picture::) that has simple capabilities. Besides that, there are other +ways to include the graphic-making commands in the document. Two such +systems are the PSTricks and TikZ packages. There are also systems +external to LaTeX, that generate a graphic that you include using the +commands of this chapter. Two that use a programming language are +Asymptote and MetaPost. One that uses a graphical interface is Xfig. +Full description of these systems is outside the scope of this document; +see their documentation on CTAN. + +* Menu: + +* Graphics package options:: Options when you load the package. +* Graphics package configuration:: Where to look for files, which file types. +* Commands for graphics:: The available commands. + + +File: latex2e.info, Node: Graphics package options, Next: Graphics package configuration, Up: Graphics + +22.1 'graphics' package options +=============================== + +Synopsis (must be in the document preamble): + + \usepackage[COMMA-SEPARATED OPTION LIST]{graphics} + +or + + \usepackage[COMMA-SEPARATED OPTION LIST]{graphicx} + + The 'graphicx' package has a format for optional arguments to the +'\includegraphics' command that is convenient (it is the key-value +format), so it is the better choice for new documents. When you load +the 'graphics' or 'graphicx' package with '\usepackage' there are two +kinds of available options. + + The first is that LaTeX does not contain information about different +output systems but instead depends on information stored in a "printer +driver" file. Normally you should not specify the driver option in the +document, and instead rely on your system's default. One advantage of +this is that it makes the document portable across systems. + + For completeness here is a list of the drivers. The currently +relevant ones are: 'dvipdfmx', 'dvips', 'dvisvgm', 'luatex', 'pdftex', +'xetex'. The two 'xdvi' and 'oztex' are essentially aliases for 'dvips' +(and 'xdvi' is monochrome). Ones that should not be used for new +systems are: 'dvipdf', 'dvipdfm', 'dviwin', 'dvipsone', 'emtex', +'pctexps', 'pctexwin', 'pctexhp', 'pctex32', 'truetex', 'tcidvi', 'vtex' +(and 'dviwindo' is an alias for 'dvipsone'). These are stored in files +with a '.def' extension, such as 'pdftex.def'. + + The second kind of options are below. + +'demo' + Instead of an image file, LaTeX puts in a 150 pt by 100 pt + rectangle (unless another size is specified in the + '\includegraphics' command). + +'draft' + For each graphic file, it is not shown but instead its file name is + printed in a box of the correct size. In order to determine the + size, the file must be present. + +'final' + (Default) Override any previous 'draft' option, so that the + document shows the contents of the graphic files. + +'hiderotate' + Do not show rotated text. (This allows for the possibility that a + previewer does not have the capability to rotate text.) + +'hidescale' + Do not show scaled text. (This allows for the possibility that a + previewer does not have the capability to scale.) + +'hiresbb' + In a PS or EPS file the graphic size may be specified in two ways. + The '%%BoundingBox' lines describe the graphic size using integer + multiples of a PostScript point, that is, integer multiples of 1/72 + inch. A later addition to the PostScript language allows decimal + multiples, such as 1.23, in '%%HiResBoundingBox' lines. This + option has LaTeX to read the size from the latter. + + +File: latex2e.info, Node: Graphics package configuration, Next: Commands for graphics, Prev: Graphics package options, Up: Graphics + +22.2 'graphics' package configuration +===================================== + +These commands configure the way LaTeX searches the file system for the +graphic. + + The behavior of file system search code is necessarily platform +dependent. In this document we cover GNU/Linux, Macintosh, and Windows, +as those systems are typically configured. For other situations consult +the documentation in 'grfguide.pdf', or the LaTeX source, or your TeX +distribution's documentation. + +* Menu: + +* \graphicspath:: Directories to search. +* \DeclareGraphicsExtensions:: File types, such as JPG or EPS. +* \DeclareGraphicsRule:: How to handle file types. + + +File: latex2e.info, Node: \graphicspath, Next: \DeclareGraphicsExtensions, Up: Graphics package configuration + +22.2.1 '\graphicspath' +---------------------- + +Synopsis: + + \graphicspath{LIST OF DIRECTORIES INSIDE CURLY BRACES} + + Declare a list of directories to search for graphics files. This +allows you to later say something like '\includegraphics{lion.png}' +instead of having to give its path. + + LaTeX always looks for graphic files first in the current directory +(and the output directory, if specified; *note output directory::). The +declaration below tells the system to then look in the subdirectory +'pix', and then '../pix'. + + \usepackage{graphicx} % or graphics; put in preamble + ... + \graphicspath{ {pix/} {../pix/} } + + The '\graphicspath' declaration is optional. If you don't include it +then LaTeX's default is to search all of the places that it usually +looks for a file (it uses LaTeX's '\input@path'). In particular, in +this case one of the places it looks is the current directory. + + Enclose each directory name in curly braces; for example, above it +says ''{pix}''. Do this even if there is only one directory. Each +directory name must end in a forward slash, '/'. This is true even on +Windows, where good practice is to use forward slashes for all the +directory separators since it makes the document portable to other +platforms. If you have spaces in your directory name then use double +quotes, as with '{"my docs/"}'. Getting one of these rules wrong will +cause LaTeX to report 'Error: File `FILENAME' not found'. + + Basically, the algorithm is that with this example, after looking in +the current directory, + + \graphicspath{ {pix/} {../pix/} } + ... + \usepackage{lion.png} + +for each of the listed directories, LaTeX concatenates it with the +filename and searches for the result, checking for 'pix/lion.png' and +then '../pix/lion.png'. This algorithm means that the '\graphicspath' +command does not recursively search subdirectories: if you issue +'\graphicspath{{a/}}' and the graphic is in 'a/b/lion.png' then LaTeX +will not find it. It also means that you can use absolute paths such as +'\graphicspath{{/home/jim/logos/}}' or +'\graphicspath{{C:/Users/Albert/Pictures/}}'. However, using these +means that the document is not portable. (You could preserve +portability by adjusting your TeX system settings configuration file +parameter 'TEXINPUTS'; see the documentation of your system.) + + You can use '\graphicspath' anywhere in the document. You can use it +more than once. Show its value with +'\makeatletter\typeout{\Ginput@path}\makeatother'. + + The directories are taken with respect to the base file. That is, +suppose that you are working on a document based on 'book/book.tex' and +it contains '\include{chapters/chap1}'. If in 'chap1.tex' you put +'\graphicspath{{plots/}}' then LaTeX will not search for graphics in +'book/chapters/plots', but instead in 'book/plots'. + + +File: latex2e.info, Node: \DeclareGraphicsExtensions, Next: \DeclareGraphicsRule, Prev: \graphicspath, Up: Graphics package configuration + +22.2.2 '\DeclareGraphicsExtensions' +----------------------------------- + +Synopses: + + \DeclareGraphicsExtensions{COMMA-SEPARATED LIST OF FILE EXTENSIONS} + + Declare the filename extensions to try. This allows you to specify +the order in which to choose graphic formats when you include graphic +files by giving the filename without the extension, as in +'\includegraphics{functionplot}'. + + In this example, LaTeX will find files in the PNG format before PDF +files. + + \DeclareGraphicsExtensions{.png,PNG,.pdf,.PDF} + ... + \includegraphics{lion} % will find lion.png before lion.pdf + +Because the filename 'lion' does not have a period, LaTeX uses the +extension list. For each directory in the graphics path (*note +\graphicspath::), LaTeX will try the extensions in the order given. If +it does not find such a file after trying all the directories and +extensions then it reports '! LaTeX Error: File `'lion'' not found'. +Note that you must include the periods at the start of the extensions. + + Because GNU/Linux and Macintosh filenames are case sensitive, the +list of file extensions is case sensitive on those platforms. The +Windows platform is not case sensitive. + + You are not required to include '\DeclareGraphicsExtensions' in your +document; the printer driver has a sensible default. For example, the +most recent 'pdftex.def' has this extension list. + + .pdf,.png,.jpg,.mps,.jpeg,.jbig2,.jb2,.PDF,.PNG,.JPG,.JPEG,.JBIG2,.JB2 + + To change the order, use the 'grfext' package. + + You can use this command anywhere in the document. You can use it +more than once. Show its value with +'\makeatletter\typeout{\Gin@extensions}\makeatother'. + + +File: latex2e.info, Node: \DeclareGraphicsRule, Prev: \DeclareGraphicsExtensions, Up: Graphics package configuration + +22.2.3 '\DeclareGraphicsRule' +----------------------------- + +Synopsis: + + \DeclareGraphicsRule{EXTENSION}{TYPE}{SIZE-FILE EXTENSION}{COMMAND} + + Declare how to handle graphic files whose names end in EXTENSION. + + This example declares that all files with names of the form +'filename-without-dot.mps' will be treated as output from MetaPost, +meaning that the printer driver will use its MetaPost-handling code to +input the file. + + \DeclareGraphicsRule{.mps}{mps}{.mps}{} + + This + + \DeclareGraphicsRule{*}{mps}{*}{} + +tells LaTeX that it should handle as MetaPost output any file with an +extension not covered by another rule, so it covers 'filename.1', +'filename.2', etc. + + This describes the four arguments. + +EXTENSION + The file extension to which this rule applies. The extension is + anything after and including the first dot in the filename. Use + the Kleene star, '*', to denote the default behavior for all + undeclared extensions. + +TYPE + The type of file involved. This type is a string that must be + defined in the printer driver. For instance, files with extensions + '.ps', '.eps', or '.ps.gz' may all be classed as type 'eps'. All + files of the same type will be input with the same internal command + by the printer driver. For example, the file types that 'pdftex' + recognizes are: 'jpg', 'jbig2', 'mps', 'pdf', 'png', 'tif'. + +SIZE-FILE EXTENSION + The extension of the file to be read to determine the size of the + graphic, if there is such a file. It may be the same as EXTENSION + but it may be different. + + As an example, consider a PostScript graphic. To make it smaller, + it might be compressed into a '.ps.gz' file. Compressed files are + not easily read by LaTeX so you can put the bounding box + information in a separate file. If SIZE-FILE EXTENSION is empty + then you must specify size information in the arguments of + '\includegraphics'. + + If the driver file has a procedure for reading size files for + 'type' then that will be used, otherwise it will use the procedure + for reading '.eps' files. (Thus you may specify the size of bitmap + files in a file with a PostScript style '%%BoundingBox' line if no + other format is available.) + +COMMAND + A command that will be applied to the file. This is very often + left empty. This command must start with a single backward quote. + Thus, '\DeclareGraphicsRule{.eps.gz}{eps}{.eps.bb}{`gunzip -c #1}' + specifies that any file with the extension '.eps.gz' should be + treated as an 'eps' file, with the BoundingBox information stored + in the file with extension '.eps.bb', and that the command 'gunzip + -c' will run on your platform to decompresses the file. + + Such a command is specific to your platform. In addition, your TeX + system must allow you to run external commands; as a security + measure modern systems restrict running commands unless you + explicitly allow it. See the documentation for your TeX + distribution. + + +File: latex2e.info, Node: Commands for graphics, Prev: Graphics package configuration, Up: Graphics + +22.3 Commands for graphics +========================== + +These are the commands available with the 'graphics' and 'graphicx' +packages. + +* Menu: + +* \includegraphics:: Using a graphic in your document. +* \rotatebox:: Rotating boxes, including graphics. +* \scalebox:: Scaling boxes, including graphics. +* \resizebox:: Scaling boxes, including graphics, to a set size. + + +File: latex2e.info, Node: \includegraphics, Next: \rotatebox, Up: Commands for graphics + +22.3.1 '\includegraphics' +------------------------- + +Synopses for 'graphics' package: + + \includegraphics{FILENAME} + \includegraphics[URX,URY]{FILENAME} + \includegraphics[LLX,LLY][URX,URY]{FILENAME} + \includegraphics*{FILENAME} + \includegraphics*[URX,URY]{FILENAME} + \includegraphics*[LLX,LLY][URX,URY]{FILENAME} + + Synopses for 'graphicx' package: + + \includegraphics{FILENAME} + \includegraphics[KEY-VALUE LIST]{FILENAME} + \includegraphics*{FILENAME} + \includegraphics*[KEY-VALUE LIST]{FILENAME} + + Include a graphics file. The starred form '\includegraphics*' will +clip the graphic to the size specified, while for the unstarred form any +part of the graphic that is outside the box of the specified size will +over-print the surrounding area. + + This + + \usepackage{graphicx} % in preamble + ... + \begin{center} + \includegraphics{plot.pdf} + \end{center} + +will incorporate into the document the graphic in 'plot.pdf', centered +and at its nominal size. You can also give a path to the file, as with +'\includegraphics{graphics/plot.pdf}'. To specify a list of locations +to search for the file, *note \graphicspath::. + + If your filename includes spaces then put it in double quotes. An +example is '\includegraphics{"sister picture.jpg"}'. + + The '\includegraphics{FILENAME}' command decides on the type of +graphic by splitting FILENAME on the first dot. You can instead use +FILENAME with no dot, as in '\includegraphics{turing}', and then LaTeX +tries a sequence of extensions such as '.png' and '.pdf' until it finds +a file with that extension (*note \DeclareGraphicsExtensions::). + + If your file name contains dots before the extension then you can +hide them with curly braces, as in +'\includegraphics{{plot.2018.03.12.a}.pdf}'. Or, if you use the +'graphicx' package then you can use the options 'type' and 'ext'; see +below. This and other filename issues are also handled with the package +'grffile'. + + This example puts a graphic in a 'figure' environment so LaTeX can +move it to the next page if fitting it on the current page is awkward +(*note figure::). + + \begin{figure} + \centering + \includegraphics[width=3cm]{lungxray.jpg} + \caption{The evidence is overwhelming: don't smoke.} \label{fig:xray} + \end{figure} + + This places a graphic that will not float, so it is sure to appear at +this point in the document even if makes LaTeX stretch the text or +resort to blank areas on the page. It will be centered and will have a +caption. + + \usepackage{caption} % in preamble + ... + \begin{center} + \includegraphics{pix/nix.png} + \captionof{figure}{The spirit of the night} \label{pix:nix} % optional + \end{center} + + This example puts a box with a graphic side by side with one having +text, with the two vertically centered. + + \newcommand*{\vcenteredhbox}[1]{\begin{tabular}{@{}c@{}}#1\end{tabular}} + ... + \begin{center} + \vcenteredhbox{\includegraphics[width=0.4\textwidth]{plot}} + \hspace{1em} + \vcenteredhbox{\begin{minipage}{0.4\textwidth} + \begin{displaymath} + f(x)=x\cdot \sin (1/x) + \end{displaymath} + \end{minipage}} + \end{center} + + If you use the 'graphics' package then the only options involve the +size of the graphic (but see *note \rotatebox:: and *note \scalebox::). +When one optional argument is present then it is '[URX,URY]' and it +gives the coordinates of the top right corner of the image, as a pair of +TeX dimensions (*note Units of length::). If the units are omitted they +default to 'bp'. In this case, the lower left corner of the image is +assumed to be at (0,0). If two optional arguments are present then the +leading one is '[LLX,LLY]', specifying the coordinates of the image's +lower left. Thus, '\includegraphics[1in,0.618in]{...}' calls for the +graphic to be placed so it is 1 inch wide and 0.618 inches tall and so +its origin is at (0,0). + + The 'graphicx' package gives you many more options. Specify them in +a key-value form, as here. + + \begin{center} + \includegraphics[width=1in,angle=90]{lion} + \hspace{2em} + \includegraphics[angle=90,width=1in]{lion} + \end{center} + +The options are read left-to-right. So the first graphic above is made +one inch wide and then rotated, while the second is rotated and then +made one inch wide. Thus, unless the graphic is perfectly square, the +two will end with different widths and heights. + + There are many options. The primary ones are listed first. + + Note that a graphic is placed by LaTeX into a box, which is +traditionally referred to as its "bounding box" (distinct from the +PostScript BoundingBox described below). The graphic's printed area may +go beyond this box, or sit inside this box, but when LaTeX makes up a +page it puts together boxes and this is the box allocated for the +graphic. + +'width' + The graphic will be shown so its bounding box is this width. An + example is '\includegraphics[width=1in]{plot}'. You can use the + standard TeX dimensions (*note Units of length::) and also + convenient is '\linewidth', or in a two-column document, + '\columnwidth' (*note Page layout parameters::). An example is + that by using the 'calc' package you can make the graphic be 1 cm + narrower than the width of the text with + '\includegraphics[width=\linewidth-1.0cm]{hefferon.jpg}'. + +'height' + The graphic will be shown so its bounding box is this height. You + can use the standard TeX dimensions (*note Units of length::), and + also convenient are '\pageheight' and '\textheight' (*note Page + layout parameters::). For instance, the command + '\includegraphics[height=0.25\textheight]{godel}' will make the + graphic a quarter of the height of the text area. + +'totalheight' + The graphic will be shown so its bounding box has this height plus + depth. This differs from the height if the graphic was rotated. + For instance, if it has been rotated by -90 then it will have zero + height but a large depth. + +'keepaspectratio' + If set to 'true', or just specified as here + + \includegraphics[...,keepaspectratio,...]{...} + + and you give as options both 'width' and 'height' (or + 'totalheight'), then LaTeX will make the graphic is as large as + possible without distortion. That is, LaTeX will ensure that + neither is the graphic wider than 'width' nor taller than 'height' + (or 'totalheight'). + +'scale' + Factor by which to scale the graphic. To make a graphic twice its + nominal size, enter '\includegraphics[scale=2.0]{...}'. This + number may be any value; a number between 0 and 1 will shrink the + graphic and a negative number will reflect it. + +'angle' + Rotate the graphic. The angle is taken in degrees and + counterclockwise. The graphic is rotated about its 'origin'; see + that option. For a complete description of how rotated material is + typeset, *note \rotatebox::. + +'origin' + The point of the graphic about which the rotation happens. + Possible values are any string containing one or two of: 'l' for + left, 'r' for right, 'b' for bottom, 'c' for center, 't' for top, + and 'B' for baseline. Thus, entering the command + '\includegraphics[angle=180,origin=c]{moon}' will turn the picture + upside down about that picture's center, while the command + '\includegraphics[angle=180,origin=lB]{LeBateau}' will turn its + picture upside down about its left baseline. (The character 'c' + gives the horizontal center in 'bc' or 'tc', but gives the vertical + center in 'lc' or 'rc'.) The default is 'lB'. + + To rotate about an arbitrary point, *note \rotatebox::. + + These are lesser-used options. + +'viewport' + Pick out a subregion of the graphic to show. Takes four arguments, + separated by spaces and given in TeX dimensions, as with + '\includegraphics[.., viewport=0in 0in 1in 0.618in]{...}'. When + the unit is omitted, the dimensions default to big points, 'bp'. + They are taken relative to the origin specified by the bounding + box. See also the 'trim' option. + +'trim' + Gives parts of the graphic to not show. Takes four arguments, + separated by spaces, that are given in TeX dimensions, as with + '\includegraphics[.., trim= 0in 0.1in 0.2in 0.3in, ...]{...}'. + These give the amounts of the graphic not to show, that is, LaTeX + will crop the picture by 0 inches on the left, 0.1 inches on the + bottom, 0.2 inches on the right, and 0.3 inches on the top. See + also the 'viewport' option. + +'clip' + If set to 'true', or just specified as here + + \includegraphics[...,clip,...]{...} + + then the graphic is cropped to the bounding box. This is the same + as using the starred form of the command, + '\includegraphics*[...]{...}'. + +'page' + Give the page number of a multi-page PDF file. The default is + 'page=1'. + +'pagebox' + Specifies which bounding box to use for PDF files from among + 'mediabox', 'cropbox', 'bleedbox', 'trimbox', or 'artbox'. PDF + files do not have the BoundingBox that PostScript files have, but + may specify up to four predefined rectangles. The MediaBox gives + the boundaries of the physical medium. The CropBox is the region + to which the contents of the page are to be clipped when displayed. + The BleedBox is the region to which the contents of the page should + be clipped in production. The TrimBox is the intended dimensions + of the finished page. The ArtBox is the extent of the page's + meaningful content. The driver will set the image size based on + CropBox if present, otherwise it will not use one of the others, + with a driver-defined order of preference. MediaBox is always + present. + +'interpolate' + Enable or disable interpolation of raster images by the viewer. + Can be set with 'interpolate=true' or just specified as here. + + \includegraphics[...,interpolate,...]{...} + +'quiet' + Do not write information to the log. You can set it with + 'quiet=true' or just specified it with + '\includegraphics[...,quiet,...]{...}', + +'draft' + If you set it with 'draft=true' or just specify it with + + \includegraphics[...,draft,...]{...} + + then the graphic will not appear in the document, possibly saving + color printer ink. Instead, LaTeX will put an empty box of the + correct size with the filename printed in it. + + These options address the bounding box for Encapsulated PostScript +graphic files, which have a size specified with a line '%%BoundingBox' +that appears in the file. It has four values, giving the lower x +coordinate, lower y coordinate, upper x coordinate, and upper y +coordinate. The units are PostScript points, equivalent to TeX's big +points, 1/72 inch. For example, if an '.eps' file has the line +'%%BoundingBox 10 20 40 80' then its natural size is 30/72 inch wide by +60/72 inch tall. + +'bb' + Specify the bounding box of the displayed region. The argument is + four dimensions separated by spaces, as with '\includegraphics[.., + bb= 0in 0in 1in 0.618in]{...}'. Usually '\includegraphics' reads + the BoundingBox numbers from the EPS file automatically, so this + option is only useful if the bounding box is missing from that file + or if you want to change it. + +'bbllx, bblly, bburx, bbury' + Set the bounding box. These four are obsolete, but are retained + for compatibility with old packages. + +'natwidth, natheight' + An alternative for 'bb'. Setting + + \includegraphics[...,natwidth=1in,natheight=0.618in,...]{...} + + is the same as setting 'bb=0 0 1in 0.618in'. + +'hiresbb' + If set to 'true', or just specified as with + + \includegraphics[...,hiresbb,...]{...} + + then LaTeX will look for '%%HiResBoundingBox' lines instead of + '%%BoundingBox' lines. (The 'BoundingBox' lines use only natural + numbers while the 'HiResBoundingBox' lines use decimals; both use + units equivalent to TeX's big points, 1/72 inch.) To override a + prior setting of 'true', you can set it to 'false'. + + These following options allow a user to override LaTeX's method of +choosing the graphic type based on the filename extension. An example +is that '\includegraphics[type=png,ext=.xxx,read=.xxx]{lion}' will read +the file 'lion.xxx' as though it were 'lion.png'. For more on these, +*note \DeclareGraphicsRule::. + +'type' + Specify the graphics type. + +'ext' + Specify the graphics extension. Only use this in conjunction with + the option 'type'. + +'read' + Specify the file extension of the read file. Only use this in + conjunction with the option 'type'. + +'command' + Specify a command to be applied to this file. Only use this in + conjunction with the option 'type'. *Note Command line options::, + for a discussion of enabling the '\write18' functionality to run + external commands. + + +File: latex2e.info, Node: \rotatebox, Next: \scalebox, Prev: \includegraphics, Up: Commands for graphics + +22.3.2 '\rotatebox' +------------------- + +Synopsis if you use the 'graphics' package: + + \rotatebox{ANGLE}{MATERIAL} + + Synopses if you use the 'graphicx' package: + + \rotatebox{ANGLE}{MATERIAL} + \rotatebox[KEY-VALUE LIST]{ANGLE}{MATERIAL} + + Put MATERIAL in a box and rotate it ANGLE degrees counterclockwise. + + This example rotates the table column heads forty-five degrees. + + \begin{tabular}{ll} + \rotatebox{45}{Character} &\rotatebox{45}{NATO phonetic} \\ + A &AL-FAH \\ + B &BRAH-VOH + \end{tabular} + + The MATERIAL can be anything that goes in a box, including a graphic. + + \rotatebox[origin=c]{45}{\includegraphics[width=1in]{lion}} + + To place the rotated material, the first step is that LaTeX sets +MATERIAL in a box, with a reference point on the left baseline. The +second step is the rotation, by default about the reference point. The +third step is that LaTeX computes a box to bound the rotated material. +Fourth, LaTeX moves this box horizontally so that the left edge of this +new bounding box coincides with the left edge of the box from the first +step (they need not coincide vertically). This new bounding box, in its +new position, is what LaTeX uses as the box when typesetting this +material. + + If you use the 'graphics' package then the rotation is about the +reference point of the box. If you use the 'graphicx' package then +these are the options that can go in the KEY-VALUE LIST, but note that +you can get the same effect without needing this package, except for the +'x' and 'y' options (*note \includegraphics::). + +'origin' + The point of the MATERIAL's box about which the rotation happens. + Possible value is any string containing one or two of: 'l' for + left, 'r' for right, 'b' for bottom, 'c' for center, 't' for top, + and 'B' for baseline. Thus, the first line here + + \rotatebox[origin=c]{180}{moon} + \rotatebox[origin=lB]{180}{LeBateau} + + will turn the picture upside down from the center while the second + will turn its picture upside down about its left baseline. (The + character 'c' gives the horizontal center in 'bc' or 'tc' but gives + the vertical center in 'lc' or 'rc', and gives both in 'c'.) The + default is 'lB'. + +'x, y' + Specify an arbitrary point of rotation with '\rotatebox[x=TeX + DIMENSION,y=TeX DIMENSION]{...}' (*note Units of length::). These + give the offset from the box's reference point. + +'units' + This key allows you to change the default of degrees + counterclockwise. Setting 'units=-360' changes the direction to + degrees clockwise and setting 'units=6.283185' changes to radians + counterclockwise. + + +File: latex2e.info, Node: \scalebox, Next: \resizebox, Prev: \rotatebox, Up: Commands for graphics + +22.3.3 '\scalebox' +------------------ + +Synopses: + + \scalebox{HORIZONTAL FACTOR}{MATERIAL} + \scalebox{HORIZONTAL FACTOR}[VERTICAL FACTOR]{MATERIAL} + \reflectbox{MATERIAL} + + Scale the MATERIAL. + + This example halves the size, both horizontally and vertically, of +the first text and doubles the size of the second. + + \scalebox{0.5}{DRINK ME} and \scalebox{2.0}{Eat Me} + + If you do not specify the optional VERTICAL FACTOR then it defaults +to the same value as the HORIZONTAL FACTOR. + + You can use this command to resize a graphic, as here. + + \scalebox{0.5}{\includegraphics{lion}} + +If you use the 'graphicx' package then you can accomplish the same thing +with optional arguments to '\includegraphics' (*note +\includegraphics::). + + The '\reflectbox' command abbreviates '\scalebox{-1}[1]{MATERIAL}'. +Thus, 'Able was I\reflectbox{Able was I}' will show the phrase 'Able was +I' immediately followed by its mirror reflection against a vertical +axis. + + +File: latex2e.info, Node: \resizebox, Prev: \scalebox, Up: Commands for graphics + +22.3.4 '\resizebox' +------------------- + +Synopses: + + \resizebox{HORIZONTAL LENGTH}{VERTICAL LENGTH}{MATERIAL} + \resizebox*{HORIZONTAL LENGTH}{VERTICAL LENGTH}{MATERIAL} + + Given a size, such as '3cm', transform MATERIAL to make it that size. +If either HORIZONTAL LENGTH or VERTICAL LENGTH is an exclamation +point '!' then the other argument is used to determine a scale factor +for both directions. + + This example makes the graphic be a half inch wide and scales it +vertically by the same factor to keep it from being distorted. + + \resizebox{0.5in}{!}{\includegraphics{lion}} + + The unstarred form '\resizebox' takes VERTICAL LENGTH to be the box's +height while the starred form '\resizebox*' takes it to be height+depth. +For instance, make the text have a height+depth of a quarter-inch with +'\resizebox*{!}{0.25in}{\parbox{3.5in}{This box has both height and +depth.}}'. + + You can use '\depth', '\height', '\totalheight', and '\width' to +refer to the original size of the box. Thus, make the text two inches +wide but keep the original height with '\resizebox{2in}{\height}{Two +inches}'. + + +File: latex2e.info, Node: Special insertions, Next: Splitting the input, Prev: Graphics, Up: Top + +23 Special insertions +********************* + +LaTeX provides commands for inserting characters that have a special +meaning do not correspond to simple characters you can type. + +* Menu: + +* Reserved characters:: Inserting '# $ % & { } _ ~ ^ \' +* Upper and lower case:: Make text upper or lower case. +* Symbols by font position:: Inserting font symbols by number. +* Text symbols:: Inserting other non-letter symbols in text. +* Accents:: Inserting accents. +* Additional Latin letters:: Inserting other non-English characters. +* inputenc package:: Set the input file text encoding. +* \rule:: Inserting lines and rectangles. +* \today:: Inserting today's date. + + +File: latex2e.info, Node: Reserved characters, Next: Upper and lower case, Up: Special insertions + +23.1 Reserved characters +======================== + +LaTeX sets aside the following characters for special purposes. For +example, the percent sign '%' is for comments. They are called +"reserved characters" or "special characters". They are all discussed +elsewhere in this manual. + + # $ % & { } _ ~ ^ \ + + If you want a reserved character to be printed as itself, in the text +body font, for all but the final three characters in that list simply +put a backslash '\' in front of the character. Thus, typing '\$1.23' +will produce '$1.23' in your output. + + As to the last three characters, to get a tilde in the text body font +use '\~{}' (omitting the curly braces would result in the next character +receiving a tilde accent). Similarly, to get a text body font +circumflex use '\^{}'. To get a backslash in the font of the text body, +enter '\textbackslash{}'. + + To produce the reserved characters in a typewriter font use '\verb!!' +as below (the double backslash '\\' is only there to split the lines in +the output). + + \begin{center} + \# \$ \% \& \{ \} \_ \~{} \^{} \textbackslash \\ + \verb!# $ % & { } _ ~ ^ \! + \end{center} + + +File: latex2e.info, Node: Upper and lower case, Next: Symbols by font position, Prev: Reserved characters, Up: Special insertions + +23.2 Upper and lower case +========================= + +Synopsis: + + \uppercase{TEXT} + \lowercase{TEXT} + \MakeUppercase{TEXT} + \MakeLowercase{TEXT} + + Change the case of characters. The TeX primitive commands +'\uppercase' and '\lowercase' are set up by default to work only with +the 26 letters a-z and A-Z. The LaTeX commands '\MakeUppercase' and +'\MakeLowercase' commands also change characters accessed by commands +such as '\ae' or '\aa'. The commands '\MakeUppercase' and +'\MakeLowercase' are robust but they have moving arguments (*note +\protect::). + + These commands do not change the case of letters used in the name of +a command within TEXT. But they do change the case of every other Latin +letter inside the argument TEXT. Thus, '\MakeUppercase{Let $y=f(x)$'} +produces 'LET Y=F(X)'. Another example is that the name of an +environment will be changed, so that '\MakeUppercase{\begin{tabular} ... +\end{tabular}}' will produce an error because the first half is changed +to '\begin{TABULAR}'. + + LaTeX uses the same fixed table for changing case throughout a +document, The table used is designed for the font encoding T1; this +works well with the standard TeX fonts for all Latin alphabets but will +cause problems when using other alphabets. + + To change the case of text that results from a macro inside TEXT you +need to do expansion. Here the '\Schoolname' produces 'COLLEGE OF +MATHEMATICS'. + + \newcommand{\schoolname}{College of Mathematics} + \newcommand{\Schoolname}{\expandafter\MakeUppercase + \expandafter{\schoolname}} + + The 'textcase' package brings some of the missing feature of the +standard LaTeX commands '\MakeUppercase' and '\MakeLowerCase'. + + To uppercase only the first letter of words, you can use the package +'mfirstuc'. + + Handling all the casing rules specified by Unicode, e.g., for +non-Latin scripts, is a much bigger job than anything envisioned in the +original TeX and LaTeX. It has been implemented in the 'expl3' package +as of 2020. The article "Case changing: From TeX primitives to the +Unicode algorithm", (Joseph Wright, 'TUGboat' 41:1, +), gives a good +overview of the topic, past and present. + + +File: latex2e.info, Node: Symbols by font position, Next: Text symbols, Prev: Upper and lower case, Up: Special insertions + +23.3 Symbols by font position +============================= + +You can access any character of the current font using its number with +the '\symbol' command. For example, the visible space character used in +the '\verb*' command has the code decimal 32 in the standard Computer +Modern typewriter font, so it can be typed as '\symbol{32}'. + + You can also specify numbers in octal (base 8) by using a ''' prefix, +or hexadecimal (base 16) with a '"' prefix, so the visible space at 32 +decimal could also be written as '\symbol{'40}' or '\symbol{"20}'. + + +File: latex2e.info, Node: Text symbols, Next: Accents, Prev: Symbols by font position, Up: Special insertions + +23.4 Text symbols +================= + +LaTeX provides commands to generate a number of non-letter symbols in +running text. Some of these, especially the more obscure ones, are not +available in OT1. As of the LaTeX February 2020 release, all symbols +are available by default; before that, it was necessary to use the +'textcomp' package for some (technically, those in the 'TS1' font +encoding). + +'\copyright' +'\textcopyright' + (C) The copyright symbol. + +'\dag' + † The dagger symbol (in text). + +'\ddag' + ‡ The double dagger symbol (in text). + +'\LaTeX' + The LaTeX logo. + +'\LaTeXe' + The LaTeX2e logo. + +'\guillemotleft («)' +'\guillemotright (»)' +'\guilsinglleft (‹)' +'\guilsinglright (›)' + «, », ‹, › Double and single angle quotation marks, commonly used + in French. + +'\ldots' +'\dots' +'\textellipsis' + ... An ellipsis (three dots at the baseline): '\ldots' and '\dots' + also work in math mode. + +'\lq' + ' Left (opening) quote. + +'\P' +'\textparagraph' + ¶ Paragraph sign (pilcrow). + +'\pounds' +'\textsterling' + £ English pounds sterling. + +'\quotedblbase („)' +'\quotesinglbase (‚)' + „ and ‚ Double and single quotation marks on the baseline. + +'\rq' + ' Right (closing) quote. + +'\S' +'\textsection' + § Section sign. + +'\TeX' + The TeX logo. + +'\textasciicircum' + ^ ASCII circumflex. + +'\textasciitilde' + ~ ASCII tilde. + +'\textasteriskcentered' + * Centered asterisk. + +'\textbackslash' + \ Backslash. + +'\textbar' + | Vertical bar. + +'\textbardbl' + ⏸ Double vertical bar. + +'\textbigcircle' + ◯, Big circle symbol. + +'\textbraceleft' + { Left brace. + +'\textbraceright' + } Right brace. + +'\textbullet' + * Bullet. + +'\textcircled{LETTER}' + Ⓐ, Circle around LETTER. + +'\textcompwordmark' +'\textcapitalcompwordmark' +'\textascendercompwordmark' + Used to separate letters that would normally ligature. For + example, 'f\textcompwordmark i' produces 'fi' without a ligature. + This is most useful in non-English languages. The + '\textcapitalcompwordmark' form has the cap height of the font + while the '\textascendercompwordmark' form has the ascender height. + +'\textdagger' + † Dagger. + +'\textdaggerdbl' + ‡ Double dagger. + +'\textdollar (or '\$')' + $ Dollar sign. + +'\textemdash (or '---')' + -- Em-dash. Used for punctuation, usually similar to commas or + parentheses, as in ''The playoffs---if you're lucky enough to make + the playoffs---are more like a sprint.'' Conventions for spacing + around em-dashes vary widely. + +'\textendash (or '--')' + - En-dash. Used for ranges, as in ''see pages 12--14''. + +'\texteuro' + + The Euro currency symbol: €. + + For an alternative glyph design, try the 'eurosym' package; also, + most fonts nowadays come with their own Euro symbol (Unicode + U+20AC). + +'\textexclamdown (or '!`')' + ¡ Upside down exclamation point. + +'\textfiguredash' + Dash used between numerals, Unicode U+2012. Defined in the June + 2021 release of LaTeX. When used in pdfTeX, approximated by an + en-dash; with a Unicode engine, either typesets the glyph if + available in the current font, or writes the usual "Missing + character" warning to the log file. + +'\textgreater' + > Greater than symbol. + +'\texthorizontalbar' + Horizontal bar character, Unicode U+2015. Defined in the June 2021 + release of LaTeX. Behavior as with '\textfiguredash' above; the + pdfTeX approximation is an em-dash. + +'\textless' + < Less than symbol. + +'\textleftarrow' + ←, Left arrow. + +'\textnonbreakinghyphen' + Non-breaking hyphen character, Unicode U+2011. Defined in the June + 2021 release of LaTeX. Behavior as with '\textfiguredash' above; + the pdfTeX approximation is a regular ASCII hyphen (with breaks + disallowed after). + +'\textordfeminine' +'\textordmasculine' + ª, º Feminine and masculine ordinal symbols. + +'\textperiodcentered' + · Centered period. + +'\textquestiondown (or '?`')' + ¿ Upside down question mark. + +'\textquotedblleft (or '``')' + " Double left quote. + +'\textquotedblright (or '''')' + " Double right quote. + +'\textquoteleft (or '`')' + ' Single left quote. + +'\textquoteright (or ''')' + ' Single right quote. + +'\textquotesingle' + ', Straight single quote. (From TS1 encoding.) + +'\textquotestraightbase' +'\textquotestraightdblbase' + Single and double straight quotes on the baseline. + +'\textregistered' + (R) Registered symbol. + +'\textrightarrow' + →, Right arrow. + +'\textthreequartersemdash' + ﹘, "Three-quarters" em-dash, between en-dash and em-dash. + +'\texttrademark' + ™ Trademark symbol. + +'\texttwelveudash' + ﹘, "Two-thirds" em-dash, between en-dash and em-dash. + +'\textunderscore' + _ Underscore. + +'\textvisiblespace' + ␣, Visible space symbol. + + +File: latex2e.info, Node: Accents, Next: Additional Latin letters, Prev: Text symbols, Up: Special insertions + +23.5 Accents +============ + +LaTeX has wide support for many of the world's scripts and languages, +provided through the core 'babel' package, which supports pdfLaTeX, +XeLaTeX and LuaLaTeX. The 'polyglossia' package provides similar +support with the latter two engines. + + This section does not cover that support. It only lists the core +LaTeX commands for creating accented characters. The '\capital...' +commands shown here produce alternative forms for use with capital +letters. These are not available with OT1. + + Below, to make them easier to find, the accents are all illustrated +with lowercase 'o'. + + Note that '\i' produces a dotless i, and '\j' produces a dotless j. +These are often used in place of their dotted counterparts when they are +accented. + +'\"' +'\capitaldieresis' + ö Umlaut (dieresis). + +'\'' +'\capitalacute' + ó Acute accent. + +'\.' + ȯ Dot accent. + +'\=' +'\capitalmacron' + ō Macron (overbar) accent. + +'\^' +'\capitalcircumflex' + ô Circumflex (hat) accent. + +'\`' +'\capitalgrave' + ò Grave accent. + +'\~' +'\capitaltilde' + ñ Tilde accent. + +'\b' + o̲ Bar accent underneath. + + Related to this, '\underbar{TEXT}' produces a bar under TEXT. The + argument is always processed in LR mode (*note Modes::). The bar + is always a fixed position under the baseline, thus crossing + through descenders. See also '\underline' in *note Math + miscellany::. + +'\c' +'\capitalcedilla' + ç Cedilla accent underneath. + +'\d' +'\capitaldotaccent' + ọ Dot accent underneath. + +'\H' +'\capitalhungarumlaut' + ő Long Hungarian umlaut accent. + +'\k' +'\capitalogonek' + ǫ Ogonek. Not available in the OT1 encoding. + +'\r' +'\capitalring' + o̊ Ring accent. + +'\t' +'\capitaltie' +'\newtie' +'\capitalnewtie' + Tie-after accent (used for transliterating from Cyrillic, such as + in the ALA-LC romanization). It expects that the argument has two + characters. The '\newtie' form is centered in its box. + +'\u' +'\capitalbreve' + ŏ Breve accent. + +'\v' +'\capitalcaron' + ǒ Háček (check, caron) accent. + +* Menu: + +* \accent:: Low level command to produce an accented character. + + +File: latex2e.info, Node: \accent, Up: Accents + +23.5.1 '\accent' +---------------- + +Synopsis: + + \accent NUMBER CHARACTER + + A TeX primitive command used to generate accented characters from +accent marks and letters. The accent mark is selected by NUMBER, a +numeric argument, followed by a space and then a CHARACTER argument to +construct the accented character in the current font. + + These are accented 'e' characters. + + \accent18 e + \accent20 e + \accent21 e + \accent22 e + \accent23 e + +The first is a grave, the second a caron, the third a breve, the fourth +a macron, and the fifth a ring above. + + The position of the accent is determined by the font designer and so +the outcome of '\accent' use may differ between fonts. In LaTeX it is +desirable to have glyphs for accented characters rather than building +them using '\accent'. Using glyphs that already contain the accented +characters (as in T1 encoding) allows correct hyphenation whereas +'\accent' disables hyphenation (specifically with OT1 font encoding +where accented glyphs are absent). + + There can be an optional font change between NUMBER and CHARACTER. +Note also that this command sets the '\spacefactor' to 1000 (*note +\spacefactor::). + + An unavoidable characteristic of some Cyrillic letters and the +majority of accented Cyrillic letters is that they must be assembled +from multiple elements (accents, modifiers, etc.) while '\accent' +provides for a single accent mark and a single letter combination. +There are also cases where accents must appear between letters that +\accent does not support. Still other cases exist where the letters I +and J have dots above their lowercase counterparts that conflict with +dotted accent marks. The use of '\accent' in these cases will not work +as it cannot analyze upper/lower case. + + +File: latex2e.info, Node: Additional Latin letters, Next: inputenc package, Prev: Accents, Up: Special insertions + +23.6 Additional Latin letters +============================= + +Here are the basic LaTeX commands for inserting letters beyond A-Z that +extend the Latin alphabet, used primarily in languages other than +English. + +'\aa' +'\AA' + å and Å. + +'\ae' +'\AE' + æ and Æ. + +'\dh' +'\DH' + Icelandic letter eth: ð and Ð. Not available with OT1 encoding, you + need the 'fontenc' package to select an alternate font encoding, + such as T1. + +'\dj' +'\DJ' + Crossed d and D, a.k.a. capital and small letter d with stroke. + Not available with OT1 encoding, you need the 'fontenc' package to + select an alternate font encoding, such as T1. + +'\ij' +'\IJ' + ij and IJ (except somewhat closer together than appears here). + +'\l' +'\L' + ł and Ł. + +'\ng' +'\NG' + Lappish letter eng, also used in phonetics. + +'\o' +'\O' + ø and Ø. + +'\oe' +'\OE' + œ and Œ. + +'\ss' +'\SS' + ß and SS. + +'\th' +'\TH' + Icelandic letter thorn: þ and Þ. Not available with OT1 encoding, + you need the 'fontenc' package to select an alternate font + encoding, such as T1. + + +File: latex2e.info, Node: inputenc package, Next: \rule, Prev: Additional Latin letters, Up: Special insertions + +23.7 'inputenc' package +======================= + +Synopsis: + + \usepackage[ENCODING-NAME]{inputenc} + + Declare the input file's text encoding to be ENCODING-NAME. The +default, if this package is not loaded, is UTF-8. Technically, +specifying the encoding name is optional, but in practice it is not +useful to omit it. + + In a computer file, the characters are stored according to a scheme +called the "encoding". There are many different encodings. The +simplest is ASCII, which supports 95 printable characters, not enough +for most of the world's languages. For instance, to typeset the +a-umlaut character 'ä' in an ASCII-encoded LaTeX source file, the +sequence '\"a' is used. This would make source files for anything but +English hard to read; even for English, often a more extensive encoding +is more convenient. + + The modern encoding standard, in some ways a union of the others, is +UTF-8, one of the representations of Unicode. This is the default for +LaTeX since 2018. + + The 'inputenc' package is how LaTeX knows what encoding is used. For +instance, the following command explicitly says that the input file is +UTF-8 (note the lack of a dash). + + \usepackage[utf8]{inputenc} + + Caution: use 'inputenc' only with the pdfTeX engine (*note TeX +engines::). (The XeTeX and LuaTeX engines assume that the input file is +UTF-8 encoded.) If you invoke LaTeX with either the 'xelatex' command +or the 'lualatex' command, and try to declare a non-UTF-8 encoding with +'inputenc', such as 'latin1', then you will get the error 'inputenc is +not designed for xetex or luatex'. + + An 'inputenc' package error such as 'Invalid UTF-8 byte "96' means +that some of the material in the input file does not follow the encoding +scheme. Often these errors come from copying material from a document +that uses a different encoding than the input file; this one is a left +single quote from a web page using 'latin1' inside a LaTeX input file +that uses UTF-8. The simplest solution is to replace the non-UTF-8 +character with its UTF-8 equivalent, or use a LaTeX equivalent command +or character. + + In some documents, such as a collection of journal articles from a +variety of authors, changing the encoding in mid-document may be +necessary. Use the command '\inputencoding{ENCODING-NAME}'. The most +common values for ENCODING-NAME are: 'ascii', 'latin1', 'latin2', +'latin3', 'latin4', 'latin5', 'latin9', 'latin10', and 'utf8'. + + +File: latex2e.info, Node: \rule, Next: \today, Prev: inputenc package, Up: Special insertions + +23.8 '\rule' +============ + +Synopsis, one of: + + \rule{WIDTH}{THICKNESS} + \rule[RAISE]{WIDTH}{THICKNESS} + + Produce a "rule", a filled-in rectangle. + + This example produces a rectangular blob, sometimes called a Halmos +symbol, or just "qed", often used to mark the end of a proof: + + \newcommand{\qedsymbol}{\rule{0.4em}{2ex}} + +The 'amsthm' package includes this command, with a somewhat +different-looking symbol. + + The mandatory arguments give the horizontal WIDTH and vertical +THICKNESS of the rectangle. They are rigid lengths (*note Lengths::). +The optional argument RAISE is also a rigid length, and tells LaTeX how +much to raise the rule above the baseline, or lower it if the length is +negative. + + This produces a line, a rectangle that is wide but not tall. + + \noindent\rule{\textwidth}{0.4pt} + +The line is the width of the page and 0.4 points tall. This line +thickness is common in LaTeX. + + A rule that has zero width, or zero thickness, will not show up in +the output, but can cause LaTeX to change the output around it. *Note +\strut::, for examples. + + +File: latex2e.info, Node: \today, Prev: \rule, Up: Special insertions + +23.9 '\today' +============= + +Synopsis: + + \today + + Produce today's date in the format 'MONTH DD, YYYY'. An example of a +date in that format is 'July 4, 1976'. + + Multilingual packages such as 'babel' or 'polyglossia', or classes +such as 'lettre', will localize '\today'. For example, the following +will output '4 juillet 1976': + + \year=1976 \month=7 \day=4 + \documentclass{minimal} + \usepackage[french]{babel} + \begin{document} + \today + \end{document} + +'\today' uses the counters '\day', '\month', and '\year' (*note \day & +\month & \year::). + + A number of package on CTAN work with dates. One is 'datetime' +package which can produce a wide variety of date formats, including ISO +standards. + + The date is not updated as the LaTeX process runs, so in principle +the date could be incorrect by the time the program finishes. + + +File: latex2e.info, Node: Splitting the input, Next: Front/back matter, Prev: Special insertions, Up: Top + +24 Splitting the input +********************** + +LaTeX lets you split a large document into several smaller ones. This +can simplify editing or allow multiple authors to work on the document. +It can also speed processing. + + Regardless of how many separate files you use, there is always one +"root file", on which LaTeX compilation starts. This shows such a file +with five included files. + + \documentclass{book} + \includeonly{ % comment out lines below to omit compiling + pref, + chap1, + chap2, + append, + bib + } + \begin{document} + \frontmatter + \include{pref} + \mainmatter + \include{chap1} + \include{chap2} + \appendix + \include{append} + \backmatter + \include{bib} + \end{document} + +This will bring in material from 'pref.tex', 'chap1.tex', 'chap2.tex', +'append.tex', and 'bib.tex'. If you compile this file, and then comment +out all of the lines inside '\includeonly{...}' except for 'chap1', and +compile again, then LaTeX will only process the material in the first +chapter. Thus, your output will appear more quickly and be shorter to +print. However, the advantage of the '\includeonly' command is that +LaTeX will retain the page numbers and all of the cross reference +information from the other parts of the document so these will appear in +your output correctly. + + *Note Larger book template::, for another example of '\includeonly'. + +* Menu: + +* \endinput:: Stop including material from a file. +* \include & \includeonly:: Conditionally include files. +* \input:: Unconditionally include a file. + + +File: latex2e.info, Node: \endinput, Next: \include & \includeonly, Up: Splitting the input + +24.1 '\endinput' +================ + +Synopsis: + + \endinput + + When you '\include{filename}', inside 'filename.tex' the material +after '\endinput' will not be included. This command is optional; if +'filename.tex' has no '\endinput' then LaTeX will read all of the file. + + For example, suppose that a document's root file has '\input{chap1}' +and this is 'chap1.tex'. + + \chapter{One} + This material will appear in the document. + \endinput + This will not appear. + + This can be useful for putting documentation or comments at the end +of a file, or for avoiding junk characters that can be added if the file +is transmitted in the body of an email. It is also useful for +debugging: one strategy to localize errors is to put '\endinput' halfway +through the included file and see if the error disappears. Now, knowing +which half contains the error, moving '\endinput' to halfway through +that area further narrows down the location. This process rapidly finds +the offending line. + + After reading '\endinput', LaTeX continues to read to the end of the +line, so something can follow this command and be read nonetheless. +This allows you, for instance, to close an '\if...' with a '\fi'. + + +File: latex2e.info, Node: \include & \includeonly, Next: \input, Prev: \endinput, Up: Splitting the input + +24.2 '\include' & '\includeonly' +================================ + +Synopsis: + + \includeonly{ % in document preamble + ... + FILENAME, + ... + } + ... + \include{FILENAME} % in document body + + Bring material from the external file 'FILENAME.tex' into a LaTeX +document. + + The '\include' command does three things: it executes '\clearpage' +(*note \clearpage & \cleardoublepage::), then it inputs the material +from 'FILENAME.tex' into the document, and then it does another +'\clearpage'. This command can only appear in the document body. + + The '\includeonly' command controls which files will be read by LaTeX +under subsequent '\include' commands. Its list of filenames is +comma-separated. It must appear in the preamble or even earlier, e.g., +the command line; it can't appear in the document body. + + This example root document, 'constitution.tex', brings in three +files, 'preamble.tex', 'articles.tex', and 'amendments.tex'. + + \documentclass{book} + \includeonly{ + preamble, + articles, + amendments + } + \begin{document} + \include{preamble} + \include{articles} + \include{amendments} + \end{document} + +The file 'preamble.tex' contains no special code; you have just +excerpted the chapter from 'consitution.tex' and put it in a separate +file just for editing convenience. + + \chapter{Preamble} + We the People of the United States, + in Order to form a more perfect Union, ... + +Running LaTeX on 'constitution.tex' makes the material from the three +files appear in the document but also generates the auxiliary files +'preamble.aux', 'articles.aux', and 'amendments.aux'. These contain +information such as page numbers and cross-references (*note Cross +references::). If you now comment out '\includeonly''s lines with +'preamble' and 'amendments' and run LaTeX again then the resulting +document shows only the material from 'articles.tex', not the material +from 'preamble.tex' or 'amendments.tex'. Nonetheless, all of the +auxiliary information from the omitted files is still there, including +the starting page number of the chapter. + + If the document preamble does not have '\includeonly' then LaTeX will +include all the files you call for with '\include' commands. + + The '\include' command makes a new page. To avoid that, see *note +\input:: (which, however, does not retain the auxiliary information). + + *Note Larger book template::, for another example using '\include' +and '\includeonly'. That example also uses '\input' for some material +that will not necessarily start on a new page. + + File names can involve paths. + + \documentclass{book} + \includeonly{ + chapters/chap1, + } + \begin{document} + \include{chapters/chap1} + \end{document} + + To make your document portable across distributions and platforms you +should avoid spaces in the file names. The tradition is to instead use +dashes or underscores. Nevertheless, for the name 'amo amas amat', this +works under TeX Live on GNU/Linux: + + \documentclass{book} + \includeonly{ + "amo\space amas\space amat" + } + \begin{document} + \include{"amo\space amas\space amat"} + \end{document} + + and this works under MiKTeX on Windows: + + \documentclass{book} + \includeonly{ + {"amo amas amat"} + } + \begin{document} + \include{{"amo amas amat"}} + \end{document} + + You cannot use '\include' inside a file that is being included or you +get 'LaTeX Error: \include cannot be nested.' The '\include' command +cannot appear in the document preamble; you will get 'LaTeX Error: +Missing \begin{document}'. + + If a file that you '\include' does not exist, for instance if you +'\include{athiesm}' but you meant '\include{atheism}', then LaTeX does +not give you an error but will warn you 'No file athiesm.tex.' (It will +also create 'athiesm.aux'.) + + If you '\include' the root file in itself then you first get 'LaTeX +Error: Can be used only in preamble.' Later runs get 'TeX capacity +exceeded, sorry [text input levels=15]'. To fix this, you must remove +the inclusion '\include{ROOT}' but also delete the file 'ROOT.aux' and +rerun LaTeX. + + +File: latex2e.info, Node: \input, Prev: \include & \includeonly, Up: Splitting the input + +24.3 '\input' +============= + +Synopsis: + + \input{FILENAME} + + LaTeX processes the file as if its contents were inserted in the +current file. For a more sophisticated inclusion mechanism see *note +\include & \includeonly::. + + If FILENAME does not end in '.tex' then LaTeX first tries the +filename with that extension; this is the usual case. If FILENAME ends +with '.tex' then LaTeX looks for the filename as it is. + + For example, this + + \input{macros} + +will cause LaTeX to first look for 'macros.tex'. If it finds that file +then it processes its contents as thought they had been copy-pasted in. +If there is no file of the name 'macros.tex' then LaTeX tries the name +'macros', without an extension. (This may vary by distribution.) + + To make your document portable across distributions and platforms you +should avoid spaces in the file names. The tradition is to instead use +dashes or underscores. Nevertheless, for the name 'amo amas amat', this +works under TeX Live on GNU/Linux: + + \input{"amo\space amas\space amat"} + + and this works under MiKTeX on Windows: + + \input{{"amo amas amat"}} + + +File: latex2e.info, Node: Front/back matter, Next: Letters, Prev: Splitting the input, Up: Top + +25 Front/back matter +******************** + +* Menu: + +* Table of contents etc.:: Table of contents, list of figures, list of tables. +* Indexes:: Generate an index. +* Glossaries:: Generate a glossary. + + +File: latex2e.info, Node: Table of contents etc., Next: Indexes, Up: Front/back matter + +25.1 Table of contents, list of figures, list of tables +======================================================= + +Synopsis, one of: + + \tableofcontents + \listoffigures + \listoftables + + Produce a table of contents, or list of figures, or list of tables. +Put the command in the input file where you want the table or list to +go. You do not type the entries; for example, typically the table of +contents entries are automatically generated from the sectioning +commands '\chapter', etc. + + This example illustrates the first command, '\tableofcontents'. +LaTeX will produce a table of contents on the book's first page. + + \documentclass{book} + % \setcounter{tocdepth}{1} + \begin{document} + \tableofcontents\newpage + ... + \chapter{...} + ... + \section{...} + ... + \subsection{...} + ... + \end{document} + +Uncommenting the second line would cause that table to contain chapter +and section listings but not subsection listings, because the '\section' +command has level 1. *Note Sectioning::, for level numbers of the +sectioning units. For more on the 'tocdepth' *note +Sectioning/tocdepth::. + + Another example of the use of '\tableofcontents' is in *note Larger +book template::. + + If you want a page break after the table of contents, write a +'\newpage' command after the '\tableofcontents' command, as above. + + To make the table of contents, LaTeX stores the information in an +auxiliary file named 'ROOT-FILE.toc' (*note Splitting the input::). For +example, this LaTeX file 'test.tex' + + \documentclass{article} + \begin{document} + \tableofcontents\newpage + \section{First section} + \subsection{First subsection} + ... + +writes these lines to 'test.toc'. + + \contentsline {section}{\numberline {1}First section}{2} + \contentsline {subsection}{\numberline {1.1}First subsection}{2} + +Each line contains a single command, '\contentsline' (*note +\contentsline::). The first argument, the 'section' or 'subsection', is +the sectioning unit. The second argument has two components. The hook +'\numberline' determines how the sectioning number, '1' or '1.1', +appears in the table of contents (*note \numberline::). The remainder +of the second argument of '\contentsline', 'First section' or 'First +subsection', is the sectioning title text. Finally, the third argument, +'2', is the page number on which this sectioning unit starts. + + To typeset these lines, the document class provides '\l@SECTION-UNIT' +commands such as '\l@section{TEXT}{PAGENUMBER}' and +'\l@subsection{TEXT}{PAGENUMBER}'. These commands often use the +'\@dottedtocline' command (*note \@dottedtocline::). + + A consequence of LaTeX's strategy of using auxiliary files is that to +get the correct information in the document you must run LaTeX twice, +once to store the information and the second time to retrieve it. In +the ordinary course of writing a document authors run LaTeX a number of +times, but you may notice that the first time that you compile a new +document, the table of contents page will be empty except for its +'Contents' header. Just run LaTeX again. + + The commands '\listoffigures' and '\listoftables' produce a list of +figures and a list of tables. Their information is stored in files with +extension '.lof' and '.lot'. They work the same way as +'\tableofcontents' but the latter is more common, so we use it for most +examples. + + You can manually add material to the table of contents, the list of +figures, and the list of tables. For instance, add a line about a +section to the table of contents with +'\addcontentsline{toc}{section}{TEXT}'. (*note \addcontentsline::). +Add arbitrary material, that is, non-line material, with +'\addtocontents', as with the command +'\addtocontents{lof}{\protect\vspace{2ex}}', which adds vertical space +to the list of figures (*note \addtocontents::). + + Lines in the table of contents, the list of figures, and the list of +tables, have four parts. First is an indent. Next is a box into which +sectioning numbers are placed, and then the third box holds the title +text, such as 'First section'. Finally there is a box up against the +right margin, inside of which LaTeX puts the page number box. For the +indent and the width of the number box, *note \@dottedtocline::. The +right margin box has width '\@tocrmarg' and the page number is flush +right in that space, inside a box of width '\@pnumwidth'. By default +'\@tocrmarg' is '2.55em' and '\@pnumwidth' is '1.55em'. Change these as +with '\renewcommand{\@tocrmarg}{3.5em}'. + + CTAN has many packages for the table of contents and lists of figures +and tables (*note CTAN::). The package 'tocloft' is convenient for +adjusting some aspects of the default such as spacing. And, +'tocbibbind' will automatically add the bibliography, index, etc. to +the table of contents. + + To change the header for the table of contents page, do something +like these commands before you call '\tableofcontents', etc. + + \renewcommand{\contentsname}{Table of Contents} + \renewcommand{\listfigurename}{Plots} + \renewcommand{\listtablename}{Specifications} + + + Internationalization packages such as 'babel' or 'polyglossia' will +change these headers depending on the chosen base language. + +* Menu: + +* \@dottedtocline:: Format entry line in table of contents, etc. +* \addcontentsline:: Add an entry to table of contents, etc. +* \addtocontents:: Add text directly to table of contents file, etc. +* \contentsline:: Set line in table of contents, etc. +* \nofiles:: Prevent writing to auxiliary files. +* \numberline:: Put its number argument flush left in a box. + + +File: latex2e.info, Node: \@dottedtocline, Next: \addcontentsline, Up: Table of contents etc. + +25.1.1 '\@dottedtocline' +------------------------ + +Synopsis: + + \@dottedtocline{SECTION-LEVEL-NUM}{INDENT}{NUMWIDTH}{TEXT}{PAGENUMBER} + + Used internally by LaTeX to format an entry line in the table of +contents, list of figures, or list of tables. Authors do not directly +enter '\@dottedtocline' commands. + + This command is typically used by '\l@section', '\l@subsection', +etc., to format the content lines. For example, the 'article.cls' file +contains these definitions: + + \newcommand*\l@section{\@dottedtocline{1}{1.5em}{2.3em}} + \newcommand*\l@subsection{\@dottedtocline{2}{3.8em}{3.2em}} + \newcommand*\l@subsubsection{\@dottedtocline{3}{7.0em}{4.1em}} + +In this example, '\@dottedcline' appears to have been given only three +arguments. But tracing the internal code shows that it picks up the +final TEXT and PAGENUMBER arguments in the synopsis from a call to +'\contentsline' (*note \contentsline::). + + Between the box for the title text of a section and the right margin +box, these '\@dottedtocline' commands insert "leaders", that is, +evenly-spaced dots. The dot-to-dot space is given by the command +'\@dotsep'. By default it is 4.5 (it is in math units, aka. 'mu', which +are '1/18' em. Change it using '\renewcommand', as in +'\renewcommand{\@dotsep}{3.5}'. + + In the standard 'book' class, LaTeX does not use dotted leaders for +the Part and Chapter table entries, and in the standard 'article' class +it does not use dotted leaders for Section entries. + + +File: latex2e.info, Node: \addcontentsline, Next: \addtocontents, Prev: \@dottedtocline, Up: Table of contents etc. + +25.1.2 '\addcontentsline' +------------------------- + +Synopsis: + + \addcontentsline{EXT}{UNIT}{TEXT} + + Add an entry to the auxiliary file with extension EXT. + + The following will result in an 'Appendices' line in the table of +contents. + + \addcontentsline{toc}{section}{\protect\textbf{Appendices}} + +It will appear at the same indentation level as the sections, will be in +boldface, and will be assigned the page number associated with the point +where the command appears in the input file. + + The '\addcontentsline' command writes information to the file +'ROOT-NAME.EXT', where ROOT-NAME is the file name of the root file +(*note Splitting the input::). It writes that information as the text +of the command '\contentsline{UNIT}{TEXT}{NUM}', where 'NUM' is the +current value of counter 'UNIT' (*note \contentsline::). The most +common case is the table of contents and there NUM is the page number of +the first page of UNIT. + + This command is invoked by the sectioning commands '\chapter', etc. +(*note Sectioning::), and also by '\caption' inside a float environment +(*note Floats::). But it is also directly used by authors. For +example, an author writing a book whose style is to have an unnumbered +preface may use the starred '\chapter*'. But that command leaves out +table of contents information, which can be entered manually, as here. + + \chapter*{Preface} + \addcontentsline{toc}{chapter}{\protect\numberline{}Preface} + +In the 'ROOT-NAME.toc' file LaTeX will put the line '\contentsline +{chapter}{\numberline {}Preface}{3}'; note that the page number '3' is +automatically generated by the system, not entered manually. + + All of the arguments for '\addcontentsline' are required. + +EXT + Typically one of the strings 'toc' for the table of contents, 'lof' + for the list of figures, or 'lot' for the list of tables. The + filename extension of the information file. + +UNIT + A string that depends on the value of the EXT argument, typically + one of: + + 'toc' + For the table of contents, this is the name of a sectional + unit: 'part', 'chapter', 'section', 'subsection', etc. + + 'lof' + For the list of figures: 'figure'. + + 'lot' + For the list of tables: 'table'. + +TEXT + The text of the entry. You must '\protect' any fragile commands + (*note \protect::) used in it. + + The '\addcontentsline' command has an interaction with '\include' +(*note \include & \includeonly::). If you use them at the same level, +as with '\addcontentsline{...}{...}{...}\include{...}' then lines in the +table of contents can come out in the wrong order. The solution is to +move '\addcontentsline' into the file being included. + + If you use a UNIT that LaTeX does not recognize, as with the typo +here + + \addcontentsline{toc}{setcion}{\protect\textbf{Appendices}} + +then you don't get an error but the formatting in the table of contents +will not make sense. + + +File: latex2e.info, Node: \addtocontents, Next: \contentsline, Prev: \addcontentsline, Up: Table of contents etc. + +25.1.3 '\addtocontents' +----------------------- + +Synopsis: + + \addtocontents{EXT}{TEXT} + + Add TEXT, which may be text or formatting commands, directly to the +auxiliary file with extension EXT. This is most commonly used for the +table of contents so that is the discussion here, but it also applies to +the list of figures and list of tables. + + This will put some vertical space in the table of contents after the +'Contents' header. + + \tableofcontents\newpage + \addtocontents{toc}{\protect\vspace*{3ex}} + +This puts the word 'Page', in boldface, above the column of page numbers +and after the header. + + \tableofcontents + \addtocontents{toc}{~\hfill\textbf{Page}\par} + \chapter{...} + +This adds a line announcing work by a new author. + + \addtocontents{toc}{% + \protect\vspace{2ex} + \textbf{Chapters by N. Other Author}\par} + + The difference between '\addtocontents' and '\addcontentsline' is +that the latter is strictly for lines, such as with a line giving the +page number for the start of a new subset of the chapters. As the above +examples show, '\addtocontents' is for material such as spacing. + + The '\addtocontents' command has two arguments. Both are required. + +EXT + Typically one of: 'toc' for the table of contents, 'lof' for the + list of figures, or 'lot' for the list of tables. The extension of + the file holding the information. + +TEXT + The text, and possibly commands, to be written. + + The sectioning commands such as '\chapter' use the '\addcontentsline' +command to store information. This command creates lines in the '.toc' +auxiliary file containing the '\contentsline' command (*note +\addcontentsline::). In contrast, the command '\addtocontents' puts +material directly in that file. + + The '\addtocontents' command has an interaction with '\include' +(*note \include & \includeonly::). If you use them at the same level, +as with '\addtocontents{...}{...}\include{...}' then lines in the table +of contents can come out in the wrong order. The solution is to move +'\addtocontents' into the file being included. + + +File: latex2e.info, Node: \contentsline, Next: \nofiles, Prev: \addtocontents, Up: Table of contents etc. + +25.1.4 '\contentsline' +---------------------- + +Synopsis: + + \contentsline{UNIT}{TEXT}{PAGENUMBER} + + Used internally by LaTeX to typeset an entry of the table of +contents, list of figures, or list of tables (*note Table of contents +etc.::). Authors do not directly enter '\contentsline' commands. + + Usually adding material to these lists is done automatically by the +commands '\chapter', '\section', etc. for the table of contents, or by +the '\caption' command inside of a '\figure' or '\table' environment +(*note figure:: and *note table::). Thus, where the root file is +'thesis.tex', and contains the declaration '\tableofcontents', the +command '\chapter{Chapter One}' produces something like this in the file +'thesis.toc'. + + \contentsline {chapter}{\numberline {1}Chapter One}{3} + + If the file contains the declaration '\listoffigures' then a figure +environment involving '\caption{Test}' will produce something like this +in 'thesis.lof'. + + \contentsline {figure}{\numberline {1.1}{\ignorespaces Test}}{6} + + To manually add material, use +'\addcontentsline{FILETYPE}{UNIT}{TEXT}', where FILETYPE is 'toc', +'lof', or 'lot' (*note \addcontentsline::). + + For manipulating how the '\contentline' material is typeset, see the +'tocloft' package. + + Note that the 'hyperref' package changes the definition of +'\contentsline' (and '\addcontentsline') to add more arguments, to make +hyperlinks. This is the source of the error 'Argument of \contentsline +has an extra }' when one adds/remove the use of package 'hyperref' and a +compilation was already run. Fix this error by deleting the '.toc' or +'.lof' or '.lot' file, and running LaTeX again. + + +File: latex2e.info, Node: \nofiles, Next: \numberline, Prev: \contentsline, Up: Table of contents etc. + +25.1.5 '\nofiles' +----------------- + +Synopsis: + + \nofiles + + Prevent LaTeX from writing any auxiliary files. The only output will +be the '.log' and '.pdf' (or '.dvi') files. This command must go in the +preamble. + + Because of the '\nofiles' command this example will not produce a +'.toc' file. + + \documentclass{book} + \nofiles + \begin{document} + \tableofcontents\newpage + \chapter{...} + ... + +LaTeX will not erase any existing auxiliary files, so if you insert the +'\nofiles' command after you have run the file and gotten a '.toc' then +the table of contents page will continue to show the old information. + + +File: latex2e.info, Node: \numberline, Prev: \nofiles, Up: Table of contents etc. + +25.1.6 '\numberline' +-------------------- + +Synopsis: + + \numberline{NUMBER} + + Typeset its argument flush left in a box. This is used in a +'\contentsline' command to typeset the section number (*note +\contentsline::). + + For example, this line in a '.toc' file causes the '1.1' to be +typeset flush left. + + \contentsline {subsection}{\numberline {1.1}Motivation}{2} + + By default, LaTeX typesets the section numbers in a box of length +'\@tempdima'. That length is set by the commands '\l@section', +'\l@subsection', etc. Put section numbers inside a natural-width box +with '\renewcommand{\numberline}[1]{#1~}' before '\tableofcontents'. + + This command is fragile so you may need to precede it with '\protect' +(*note \protect::). An example is the use of '\protect' in this +command, + + \addcontentsline{toc}{section}{\protect\numberline{}Summary} + +to get the '\numberline' into the '\contentsline' command in the '.toc' +file: '\contentsline {section}{\numberline {}Summary}{6}' (the page +number '6' is automatically added by LaTeX; *note \addcontentsline::). + + +File: latex2e.info, Node: Indexes, Next: Glossaries, Prev: Table of contents etc., Up: Front/back matter + +25.2 Indexes +============ + +If you tell LaTeX what terms you want to appear in an index then it can +produce that index, alphabetized and with the page numbers automatically +maintained. This illustrates the basics. + + \documentclass{article} + \usepackage{makeidx} % Provide indexing commands + \makeindex + % \usepackage{showidx} % Show marginal notes of index entries + ... + \begin{document} + ... + Wilson's Theorem\index{Wilson's Theorem} + says that a number $n>1$ is prime if and only if the factorial + of $n-1$ is congruent to $-1$ + modulo~$n$.\index{congruence!and Wilson's Theorem} + ... + \printindex + \end{document} + +As that shows, declare index entries with the '\index' command (*note +\index::). When you run LaTeX, the '\index' writes its information, +such as 'Wilson's Theorem' and the page number, to an auxiliary file +whose name ends in '.idx'. Next, to alphabetize and do other +manipulations, run an external command, typically 'makeindex' (*note +makeindex::), which writes a file whose name ends in '.ind'. Finally, +'\printindex' brings this manipulated information into the output (*note +\printindex::). + + Thus, if the above example is in the file 'numth.tex' then running +'pdflatex numth' will save index entry and page number information to +'numth.idx'. Then running 'makeindex numth' will alphabetize and save +the results to 'numth.ind'. Finally, again running 'pdflatex numth' +will show the desired index, at the place where the '\printindex' +command is in the source file. + + There are many options for the output. An example is that the +exclamation point in '\index{congruence!and Wilson's Theorem}' produces +a main entry of 'congruence' with a subentry of 'and Wilson's Theorem'. +For more, *note makeindex::. + + The '\makeindex' and '\printindex' commands are independent. Leaving +out the '\makeindex' will stop LaTeX from saving the index entries to +the auxiliary file. Leaving out the '\printindex' will cause LaTeX to +not show the index in the document output. + + There are many packages in the area of indexing. The 'showidx' +package causes each index entries to be shown in the margin on the page +where the '\index' appears. This can help in preparing the index. The +'multind' package, among others, supports multiple indexes. See also +the TeX FAQ entry on this topic, , +and the CTAN topic, . + +* Menu: + +* Produce the index manually:: Alphabetize entries yourself. +* \index:: Declare an index entry. +* makeindex:: Alphabetize index entries automatically. +* \printindex:: Put the index here. + + +File: latex2e.info, Node: Produce the index manually, Next: \index, Up: Indexes + +25.2.1 Produce the index manually +--------------------------------- + +Documents that are small and static can have a manually produced index. +This will make a separate page labeled 'Index', in twocolumn format. + + \begin{theindex} + \item acorn squash, 1 + \subitem maple baked, 2 + \indexspace + \item bacon, 3 + \subitem maple baked, 4 + \end{theindex} + + Note that the author must enter the page numbers, which is tedious +and which will result in wrong numbers if the document changes. This is +why in most cases automated methods such as 'makeindex' are best. *Note +Indexes::. + + However we cover the commands for completeness, and because the +automated methods are based on these commands. There are three levels +of entries. As the example shows, a main entry uses '\item', subentries +use '\subitem', and the lowest level uses '\subsubitem'. Blank lines +between entries have no effect. The example above includes +'\indexspace' to produce vertical space in the output that some index +styles use before the first entry starting with a new letter. + + +File: latex2e.info, Node: \index, Next: makeindex, Prev: Produce the index manually, Up: Indexes + +25.2.2 '\index' +--------------- + +Synopsis: + + \index{INDEX-ENTRY-STRING} + + Declare an entry in the index. This command is fragile (*note +\protect::). + + For example, as described in *note Indexes::, one way to get an index +from what's below is to compile the document with 'pdflatex test', then +process the index entries with 'makeindex test', and then compile again +with 'pdflatex test'. + + % file test.tex + ... + W~Ackermann (1896--1962).\index{Ackermann} + ... + Ackermann function\index{Ackermann!function} + ... + rate of growth\index{Ackermann!function!growth rate} + +All three index entries will get a page number, such as 'Ackermann, 22'. +LaTeX will format the second as a subitem of the first, on the line +below it and indented, and the third as a subitem of the second. Three +levels deep is as far as you can nest subentries. (If you add +'\index{Ackermann!function!growth rate!comparison}' then 'makeindex' +says 'Scanning input file test.idx....done (4 entries accepted, 1 +rejected)' and the fourth level is silently missing from the index.) + + If you enter a second '\index' with the same INDEX-ENTRY-STRING then +you will get a single index entry with two page numbers (unless they +happen to fall on the same page). Thus, adding 'as for +Ackermann.\index{Ackermann}' later in the same document as above will +give an index entry like 'Ackermann, 22, 151'. Also, you can enter the +index entries in any order, so for instance '\index{Ackermann!function}' +could come before '\index{Ackermann}'. + + Get a page range in the output, like 'Hilbert, 23--27', as here. + + W~Ackermann (1896--1962).\index{Ackermann} + ... + D~Hilbert (1862--1943)\index{Ackermann!Hilbert|(} + ... + disapproved of his marriage.\index{Ackermann!Hilbert|)} + +If the beginning and ending of the page range are equal then the system +just gives a single page number, not a range. + + If you index subentries but not a main entry, as with +'\index{Jones!program}' and '\index{Jones!results}', then the output is +the item 'Jones' with no comma or page number, followed by two subitems, +like 'program, 50' and 'results, 51'. + + Generate a index entry that says 'see' by using a vertical bar +character: '\index{Ackermann!function|see{P\'eter's function}}'. You +can instead get 'see also' with 'seealso'. (The text 'see' is defined +by '\seename', and 'see also' by '\alsoname'. You can redefine these +either by using an internationalization package such as 'babel' or +'polyglossia', or directly as with '\renewcommand{\alsoname}{Also +see}'.) + + The 'see' feature is part of a more general functionality. After the +vertical bar you can put the name of a one-input command, as in +'\index{group|textit}' (note the missing backslash on the '\textit' +command) and the system will apply that command to the page number, here +giving something like '\textit{7}'. You can define your own one-input +commands, such as '\newcommand{\definedpage}[1]{{\color{blue}#1}}' and +then '\index{Ackermann!function|definedpage}' will give a blue page +number (*note Color::). Another, less practical, example is this, + + \newcommand\indexownpage[1]{#1, \thepage} + ... Epimenides.\index{self-reference|indexownpage} + +which creates an entry citing the page number of its own index listing. + + The two functions just described combine, as here + + \index{Ackermann!function|(definedpage} + ... + \index{Ackermann!function|)} + +which outputs an index entry like 'function, 23--27' where the page +number range is in blue. + + Consider an index entry such as 'α-ring'. Entering it as +'$\alpha$-ring' will cause it to be alphabetized according to the dollar +sign. You can instead enter it using an at-sign, as +'\index{alpha-ring@$\alpha$-ring}'. If you specify an entry with an +at-sign separating two strings, 'POS@TEXT', then POS gives the +alphabetical position of the entry while TEXT produces the text of the +entry. Another example is that '\index{Saint Michael's College@SMC}' +produces an index entry 'SMC' alphabetized into a different location +than its spelling would naturally give it. + + To put a '!', or '@', or '|', or '"' character in an index entry, +escape it by preceding it with a double quote, '"'. (The double quote +gets deleted before alphabetization.) + + A number of packages on CTAN have additional functionality beyond +that provided by 'makeidx'. One is 'index', which allows for multiple +indices and contains a command '\index*{INDEX-ENTRY-STRING}' that prints +the INDEX-ENTRY-STRING as well as indexing it. + + The '\index' command writes the indexing information to the file +'ROOT-NAME.idx' file. Specifically, it writes text of the command +'\indexentry{INDEX-ENTRY-STRING}{PAGE-NUM}', where PAGE-NUM is the value +of the '\thepage' counter. On occasion, when the '\printindex' command +is confused, you have to delete this file to start with a fresh slate. + + If you omit the closing brace of an '\index' command then you get a +message like this. + + Runaway argument? {Ackermann!function + ! Paragraph ended before \@wrindex was complete. + + +File: latex2e.info, Node: makeindex, Next: \printindex, Prev: \index, Up: Indexes + +25.2.3 'makeindex' +------------------ + +Synopsis, one of: + + makeindex FILENAME + makeindex -s STYLE-FILE FILENAME + makeindex OPTIONS FILENAME0 ... + + Sort, and otherwise process, the index information in the auxiliary +file FILENAME. This is a command line program. It takes one or more +raw index files, 'FILENAME.idx' files, and produces the actual index +file, the 'FILENAME.ind' file that is input by '\printindex' (*note +\printindex::). + + The first form of the command suffices for many uses. The second +allows you to format the index by using an "index style file", a '.isty' +file. The third form is the most general; see the full documentation on +CTAN. + + This is a simple '.isty' file. + + % book.isty + % $ makeindex -s book.isty -p odd book.idx + % creates the index as book.ind, starting on an odd page. + preamble + "\\pagestyle{empty} + \\small + \\begin{theindex} + \\thispagestyle{empty}" + + postamble + "\n + \\end{theindex}" + + The description here covers only some of the index formatting +possibilities in STYLE-FILE. For a full list see the documentation on +CTAN. + + A style file consists of a list of pairs: SPECIFIER and ATTRIBUTE. +These can appear in the file in any order. All of the ATTRIBUTEs are +strings, except where noted. Strings are surrounded with double quotes, +'"', and the maximum length of a string is 144 characters. The '\n' is +for a newline and '\t' is for a tab. Backslashes are escaped with +another backslash, '\\'. If a line begins with a percent sign, '%', +then it is a comment. + +'preamble' + Preamble of the output index file. Defines the context in which + the index is formatted. Default: '"\\begin{theindex}\n"'. + +'postamble' + Postamble of the output index file. Default: + '"\n\n\\end{theindex}\n"'. + +'group_skip' + Traditionally index items are broken into groups, typically a group + for entries starting with letter 'a', etc. This specifier gives + what is inserted when a new group begins. Default: '"\n\n + \\indexspace\n"' ('\indexspace' is a command inserting a rubber + length, by default '10pt plus5pt minus3pt'). + +'lethead_flag' + An integer. It governs what is inserted for a new group or letter. + If it is 0 (which is the default) then other than 'group_skip' + nothing will be inserted before the group. If it is positive then + at a new letter the 'lethead_prefix' and 'lethead_suffix' will be + inserted, with that letter in uppercase between them. If it is + negative then what will be inserted is the letter in lowercase. + The default is 0. + +'lethead_prefix' + If a new group begins with a different letter then this is the + prefix inserted before the new letter header. Default: '""' + +'lethead_suffix' + If a group begins with a different letter then this is the suffix + inserted after the new letter header. Default: '""'. + +'item_0' + What is put between two level 0 items. Default: '"\n \\item "'. + +'item_1' + Put between two level 1 items. Default: '"\n \\subitem "'. + +'item_2' + put between two level 2 items. Default: '"\n \\subsubitem "'. + +'item_01' + What is put between a level 0 item and a level 1 item. Default: + '"\n \\subitem "'. + +'item_x1' + What is put between a level 0 item and a level 1 item in the case + that the level 0 item doesn't have any page numbers (as in + '\index{aaa|see{bbb}}'). Default: '"\n \\subitem "'. + +'item_12' + What is put between a level 1 item and a level 2 item. Default: + '"\n \\subsubitem "'. + +'item_x2' + What is put between a level 1 item and a level 2 item, if the + level 1 item doesn't have page numbers. Default: '"\n \\subsubitem + "'. + +'delim_0' + Delimiter put between a level 0 key and its first page number. + Default: a comma followed by a blank, '", "'. + +'delim_1' + Delimiter put between a level 1 key and its first page number. + Default: a comma followed by a blank, '", "'. + +'delim_2' + Delimiter between a level 2 key and its first page number. + Default: a comma followed by a blank, '", "'. + +'delim_n' + Delimiter between two page numbers for the same key (at any level). + Default: a comma followed by a blank, '", "'. + +'delim_r' + What is put between the starting and ending page numbers of a + range. Default: '"--"'. + +'line_max' + An integer. Maximum length of an index entry's line in the output, + beyond which the line wraps. Default: '72'. + +'indent_space' + What is inserted at the start of a wrapped line. Default: + '"\t\t"'. + +'indent_length' + A number. The length of the wrapped line indentation. The default + 'indent_space' is two tabs and each tab is eight spaces so the + default here is '16'. + +'page_precedence' + A document may have pages numbered in different ways. For example, + a book may have front matter pages numbered in lowercase roman + while main matter pages are in arabic. This string specifies the + order in which they will appear in the index. The 'makeindex' + command supports five different types of numerals: lowercase roman + 'r', and numeric or arabic 'n', and lowercase alphabetic 'a', and + uppercase roman 'R', and uppercase alphabetic 'A'. Default: + '"rnaRA"'. + + There are a number of other programs that do the job 'makeindex' +does. One is 'xindy' (), which does +internationalization and can process indexes for documents marked up +using LaTeX and a number of other languages. It is written in Lisp, +highly configurable, both in markup terms and in terms of the collating +order of the text, as described in its documentation. + + A more recent indexing program supporting Unicode is 'xindex', +written in Lua (). + + +File: latex2e.info, Node: \printindex, Prev: makeindex, Up: Indexes + +25.2.4 '\printindex' +-------------------- + +Synopsis: + + \printindex + + Place the index into the output. + + To get an index you must first include +'\usepackage{makeidx}\makeindex' in the document preamble and compile +the document, then run the system command 'makeindex', and then compile +the document again. *Note Indexes::, for further discussion and an +example of the use of '\printindex'. + + +File: latex2e.info, Node: Glossaries, Prev: Indexes, Up: Front/back matter + +25.3 Glossaries +=============== + +Synopsis: + + \usepackage{glossaries} \makeglossaries + ... + \newglossaryentry{LABEL}{SETTINGS} + ... + \gls{LABEL}. + ... + \printglossaries + + The 'glossaries' package allows you to make glossaries, including +multiple glossaries, as well as lists of acronyms. + + To get the output from this example, compile the document (for +instance with 'pdflatex filename'), then run the command line command +'makeglossaries filename', and then compile the document again. + + \documentclass{...} + \usepackage{glossaries} \makeglossaries + \newglossaryentry{tm}{% + name={Turing machine}, + description={A model of a machine that computes. The model is simple + but can compute anything any existing device can compute. + It is the standard model used in Computer Science.}, + } + \begin{document} + Everything begins with the definition of a \gls{tm}. + ... + \printglossaries + \end{document} + +That gives two things. In the main text it outputs '... definition of a +Turing machine'. In addition, in a separate sectional unit headed +'Glossary' there appears a description list. In boldface it says +'Turing machine' and the rest of the item says in normal type 'A model +of a machine ... Computer Science'. + + The command '\makeglossary' opens the file that will contain the +entry information, 'ROOT-FILE.glo'. Put the '\printglossaries' command +where you want the glossaries to appear in your document. + + The 'glossaries' package is very powerful. For instance, besides the +commands '\newglossaryentry' and '\gls', there are similar commands for +a list of acronyms. See the package documentations on CTAN. + +* Menu: + +* \newglossaryentry:: Declare the content of a glossary entry. +* \gls:: Give a page reference for a glossary entry. + + +File: latex2e.info, Node: \newglossaryentry, Next: \gls, Up: Glossaries + +25.3.1 '\newglossaryentry' +-------------------------- + +Synopsis, one of: + + \newglossaryentry{LABEL} + { + name={NAME}, + description={DESCRIPTION}, + OTHER OPTIONS, ... + } + + or + + \longnewglossaryentry{LABEL} + { + name={NAME}, + OTHER OPTIONS ..., + } + {DESCRIPTION} + + Declare a new entry for a glossary. The LABEL must be unique for the +document. The settings associated with the label are pairs: +'KEY=VALUE'. + + This puts the blackboard bold symbol for the real numbers ℝ, in the +glossary. + + \newglossaryentry{R} + { + name={\ensuremath{\mathbb{R}}}, + description={the real numbers}, + } + + Use the second command form if the DESCRIPTION spans more than one +paragraph. + + For a full list of KEYs see the package documentation on CTAN but +here are a few. + +'name' + (Required.) The word, phrase, or symbol that you are defining. + +'description' + (Required.) The description that will appear in the glossary. If + this has more than one paragraph then you must use the second + command form given in the synopsis. + +'plural' + The plural form of NAME. Refer to the plural form using '\glspl' + or '\Glspl' (*note \gls::). + +'sort' + How to place this entry in the list of entries that the glossary + holds. + +'symbol' + A symbol, such as a mathematical symbol, besides the name. + + +File: latex2e.info, Node: \gls, Prev: \newglossaryentry, Up: Glossaries + +25.3.2 '\gls' +------------- + +Synopsis, one of: + + \gls{LABEL} + \glspl{LABEL} + \Gls{LABEL} + \Glspl{LABEL} + + Refer to a glossary entry. The entries are declared with +'\newglossaryentry' (*note \newglossaryentry::). + + This + + \newglossaryentry{N}{% + name={the natural numbers}, + description={The numbers $0$, $1$, $2$, $\ldots$\@}, + symbol={\ensuremath{\mathbb{N}}}, + } + ... + Consider \gls{N}. + +gives the output 'Consider the natural numbers'. + + The second command form '\glspl{LABEL}' produces the plural of NAME +(by default it tries adding an 's'). The third form capitalizes the +first letter of NAME, as does the fourth form, which also takes the +plural. + + +File: latex2e.info, Node: Letters, Next: Input/output, Prev: Front/back matter, Up: Top + +26 Letters +********** + +Synopsis: + + \documentclass{letter} + \address{SENDERS ADDRESS} % return address + \signature{SENDER NAME} + \begin{document} + \begin{letter}{RECIPIENT ADDRESS} + \opening{SALUTATION} + LETTER BODY + \closing{CLOSING TEXT} + \end{letter} + ... + \end{document} + + Produce one or more letters. + + Each letter is in a separate 'letter' environment, whose argument +RECIPIENT ADDRESS often contains multiple lines separated with a double +backslash, ('\\'). For example, you might have: + + \begin{letter}{Ninon de l'Enclos \\ + l'h\^otel Sagonne} + ... + \end{letter} + + The start of the 'letter' environment resets the page number to 1, +and the footnote number to 1 also. + + The SENDER ADDRESS and SENDER NAME are common to all of the letters, +whether there is one or more, so these are best put in the preamble. As +with the recipient address, often SENDER ADDRESS contains multiple lines +separated by a double backslash ('\\'). LaTeX will put the SENDER NAME +under the closing, after a vertical space for the traditional +hand-written signature. + + Each 'letter' environment body begins with a required '\opening' +command such as '\opening{Dear Madam or Sir:}'. The LETTER BODY text is +ordinary LaTeX so it can contain everything from enumerated lists to +displayed math, except that commands such as '\chapter' that make no +sense in a letter are turned off. Each 'letter' environment body +typically ends with a '\closing' command such as '\closing{Yours,}'. + + Additional material may come after the '\closing'. You can say who +is receiving a copy of the letter with a command like '\cc{the Boss \\ +the Boss's Boss}'. There's a similar '\encl' command for a list of +enclosures. And, you can add a postscript with '\ps'. + + LaTeX's default is to indent the sender name and the closing above it +by a length of '\longindentation'. By default this is '0.5\textwidth'. +To make them flush left, put '\setlength{\longindentation}{0em}' in your +preamble. + + To set a fixed date use something like +'\renewcommand{\today}{1958-Oct-12}'. If put in your preamble then it +will apply to all the letters. + + This example shows only one 'letter' environment. The three lines +marked as optional are typically omitted. + + \documentclass{letter} + \address{Sender's street \\ Sender's town} + \signature{Sender's name \\ Sender's title} + % optional: \location{Mailbox 13} + % optional: \telephone{(102) 555-0101} + \begin{document} + \begin{letter}{Recipient's name \\ Recipient's address} + \opening{Sir:} + % optional: \thispagestyle{firstpage} + I am not interested in entering a business arrangement with you. + \closing{Your most humble, etc.,} + \end{letter} + \end{document} + + These commands are used with the 'letter' class. + +* Menu: + +* \address:: Sender's return address. +* \cc:: Carbon copy list. +* \closing:: Saying goodbye. +* \encl:: List of enclosed material. +* \location:: Sender's organizational location. +* \makelabels:: Make address labels. +* \name:: Sender's name, for the return address. +* \opening:: Saying hello. +* \ps:: Adding a postscript. +* \signature:: Sender's signature. +* \telephone:: Sender's phone number. + + +File: latex2e.info, Node: \address, Next: \cc, Up: Letters + +26.1 '\address' +=============== + +Synopsis: + + \address{SENDERS ADDRESS} + + Specify the return address, as it appears on the letter and on the +envelope. Separate multiple lines in SENDERS ADDRESS with a double +backslash, '\\'. + + Because it can apply to multiple letters this declaration is often +put in the preamble. However, it can go anywhere, including inside an +individual 'letter' environment. + + This command is optional: if you do not use it then the letter is +formatted with some blank space on top, for copying onto pre-printed +letterhead paper. If you do use the '\address' declaration then it is +formatted as a personal letter. + + Here is an example. + + \address{Stephen Maturin \\ + The Grapes of the Savoy} + + +File: latex2e.info, Node: \cc, Next: \closing, Prev: \address, Up: Letters + +26.2 '\cc' +========== + +Synopsis: + + \cc{NAME0 \\ + ... } + + Produce a list of names to which copies of the letter were sent. +This command is optional. If it appears then typically it comes after +'\closing'. Put the names on different lines by separating them with a +double backslash, '\\', as in: + + \cc{President \\ + Vice President} + + +File: latex2e.info, Node: \closing, Next: \encl, Prev: \cc, Up: Letters + +26.3 '\closing' +=============== + +Synopsis: + + \closing{TEXT} + + Produce the letter's closing. This is optional, but usual. It +appears at the end of a letter, above a handwritten signature. For +example: + + \closing{Regards,} + + +File: latex2e.info, Node: \encl, Next: \location, Prev: \closing, Up: Letters + +26.4 '\encl' +============ + +Synopsis: + + \encl{FIRST ENCLOSED OBJECT \\ + ... } + + Produce a list of things included with the letter. This command is +optional; when it is used, it typically is put after '\closing'. +Separate multiple lines with a double backslash, '\\'. + + \encl{License \\ + Passport} + + +File: latex2e.info, Node: \location, Next: \makelabels, Prev: \encl, Up: Letters + +26.5 '\location' +================ + +Synopsis: + + \location{TEXT} + + The TEXT appears centered at the bottom of the page. It only appears +if the page style is 'firstpage'. + + +File: latex2e.info, Node: \makelabels, Next: \name, Prev: \location, Up: Letters + +26.6 '\makelabels' +================== + +Synopsis: + + \makelabels % in preamble + + Optional, for a document that contains 'letter' environments. If you +just put '\makelabels' in the preamble then at the end of the document +you will get a sheet with labels for all the recipients, one for each +letter environment, that you can copy to a sheet of peel-off address +labels. + + Customize the labels by redefining the commands '\startlabels', +'\mlabel', and '\returnaddress' (and perhaps '\name') in the preamble. +The command '\startlabels' sets the width, height, number of columns, +etc., of the page onto which the labels are printed. The command +'\mlabel{RETURN ADDRESS}{RECIPIENT ADDRESS}' produces the two labels (or +one, if you choose to ignore the RETURN ADDRESS) for each letter +environment. The first argument, RETURN ADDRESS, is the value returned +by the macro '\returnaddress'. The second argument, RECIPIENT ADDRESS, +is the value passed in the argument to the 'letter' environment. By +default '\mlabel' ignores the first argument, the RETURN ADDRESS, +causing the default behavior described in the prior paragraph. + + This illustrates customization. Its output includes a page with two +columns having two labels each. + + \documentclass{letter} + \renewcommand*{\returnaddress}{Fred McGuilicuddy \\ + Oshkosh, Mineola 12305} + \newcommand*\originalMlabel{} + \let\originalMlabel\mlabel + \def\mlabel#1#2{\originalMlabel{}{#1}\originalMlabel{}{#2}} + \makelabels + ... + \begin{document} + \begin{letter}{A Einstein \\ + 112 Mercer Street \\ + Princeton, New Jersey, USA 08540} + ... + \end{letter} + \begin{letter}{K G\"odel \\ + 145 Linden Lane \\ + Princeton, New Jersey, USA 08540} + ... + \end{letter} + \end{document} + +The first column contains the return address twice. The second column +contains the address for each recipient. + + The package 'envlab' makes formatting the labels easier, with +standard sizes already provided. The preamble lines +'\usepackage[personalenvelope]{envlab}' and '\makelabels' are all that +you need to print envelopes. + + +File: latex2e.info, Node: \name, Next: \opening, Prev: \makelabels, Up: Letters + +26.7 '\name' +============ + +Synopsis: + + \name{NAME} + + Optional. Sender's name, used for printing on the envelope together +with the return address. + + +File: latex2e.info, Node: \opening, Next: \ps, Prev: \name, Up: Letters + +26.8 '\opening' +=============== + +Synopsis: + + \opening{SALUTATION} + + Required. Follows the '\begin{letter}{...}'. The argument +SALUTATION is mandatory. For instance: + + \opening{Dear John:} + + +File: latex2e.info, Node: \ps, Next: \signature, Prev: \opening, Up: Letters + +26.9 '\ps' +========== + +Synopsis: + + \ps{TEXT} + + Add a postscript. This command is optional and usually is used after +'\closing'. + + \ps{P.S. After you have read this letter, burn it. Or eat it.} + + +File: latex2e.info, Node: \signature, Next: \telephone, Prev: \ps, Up: Letters + +26.10 '\signature' +================== + +Synopsis: + + \signature{FIRST LINE \\ + ... } + + The sender's name. This command is optional, although its inclusion +is usual. + + The argument text appears at the end of the letter, after the +closing. LaTeX leaves some vertical space for a handwritten signature. +Separate multiple lines with a double backslash, '\\'. For example: + + \signature{J Fred Muggs \\ + White House} + + LaTeX's default for the vertical space from the '\closing' text down +to the '\signature' text is '6\medskipamount', which is six times +'\medskipamount' (where '\medskipamount' is equal to a '\parskip', which +in turn is defined by default here to 0.7em). + + This command is usually in the preamble, to apply to all the letters +in the document. To have it apply to one letter only, put it inside a +'letter' environment and before the '\closing'. + + You can include a graphic in the signature as here. + + \signature{\vspace{-6\medskipamount}\includegraphics{sig.png}\\ + My name} + +For this you must put '\usepackage{graphicx}' in the preamble (*note +Graphics::). + + +File: latex2e.info, Node: \telephone, Prev: \signature, Up: Letters + +26.11 '\telephone' +================== + +Synopsis: + + \telephone{NUMBER} + + The sender's telephone number. This is typically in the preamble, +where it applies to all letters. This only appears if the 'firstpage' +pagestyle is selected. If so, it appears on the lower right of the +page. + + +File: latex2e.info, Node: Input/output, Next: Command line interface, Prev: Letters, Up: Top + +27 Input/output +*************** + +LaTeX uses the ability to write to a file and later read it back in to +build document components such as a table of contents or index. You can +also read a file that other programs written, or write a file for others +to read. You can communicate with users through the terminal. And, you +can issue instructions for the operating system. + +* Menu: + +* \openin & \openout:: Open a file. +* \read:: Read text from a file. +* \typein:: Read text from the terminal. +* \typeout:: Write text to the terminal. +* \write:: Write text to a file or terminal. + + +File: latex2e.info, Node: \openin & \openout, Next: \read, Up: Input/output + +27.1 '\openin' & '\openout' +=========================== + +Synopsis: + + \openin NUMBER=FILENAME + + or: + + \openout NUMBER=FILENAME + + Open a file for reading material, or for writing it. In most +engines, the NUMBER must be between 0 and 15, as in '\openin3'; in +LuaLaTeX, NUMBER can be between 0 and 127. + + Here TeX opens the file 'presidents.tex' for reading. + + \newread\presidentsfile + \openin\presidentsfile=presidents + \typeout{presidentsfile is \the\presidentsfile} + \read\presidentsfile to\presidentline + \typeout{\presidentline} + +The '\newread' command allocates input stream numbers from 0 to 15 +(there is also a '\newwrite'). The '\presidentsfile' is more memorable +but under the hood it is still a number; the first '\typeout' gives +something like 'presidentsfile is 1'. In addition, '\newread' keeps +track of the allocation so that if you use too many then you get an +error like '! No room for a new \read'. The second '\typeout' gives the +first line of the file, something like '1 Washington, George'. + + Ordinarily TeX will not try to open the file until the next page +shipout. To change this, use '\immediate\openin NUMBER=FILENAME' or +'\immediate\openout NUMBER=FILENAME'. + + Close files with '\closein NUMBER' and '\closeout NUMBER'. + + How LaTeX handles filenames varies among distributions, and even can +vary among versions of a distribution. If the file does not have an +extension then TeX will add a '.tex'. This creates 'presidents.tex', +writes one line to it, and closes it. + + \newwrite\presidentsfile + \openout\presidentsfile=presidents + \write\presidentsfile{1 Washington, George} + \closeout\presidentsfile + +But filenames with a period can cause trouble: if TeX finds a FILENAME +of 'presidents.dat' it could look first for 'presidents.dat.tex' and +later for 'presidents.dat', or it could do the opposite. Your +distribution's documentation should say more, and if you find something +that works for you then you are good, but to ensure complete portability +the best thing is to use file names containing only the twenty six ASCII +letters (not case-sensitive) and the ten digits, along with underscore +and dash, and in particular with no dot or space. + + For '\openin', if TeX cannot find the file then it does not give an +error. It just considers that the stream is not open (test for this +with '\ifeof'; one recourse is the command '\InputIfFileExists', *note +Class and package commands::). If you try to use the same number twice, +LaTeX won't give you an error. If you try to use a bad number then you +get an error message like '! Bad number (16). = l.30 +\openin16=test.jh'. + + +File: latex2e.info, Node: \read, Next: \typein, Prev: \openin & \openout, Up: Input/output + +27.2 '\read' +============ + +Synopsis: + + \read NUMBER toMACRO + + Make the command MACRO contain the next line of input from text +stream NUMBER, as in '\read5 to\data'. + + This opens the file 'email.tex' for reading, puts the contents of the +first line into the command '\email', and then closes the file. + + \newread\recipientfile + \openin\recipientfile=email + \read\recipientfile to\email + \typeout{Email address: \email} + \closein\recipientfile + + If NUMBER is outside the range from 0 to 15 or if no file of that +number is open, or if the file has ended, then '\read' will take input +from the terminal (or exit if interaction is not allowed, e.g., +'\nonstopmode'; *note interaction modes::). (However, the natural way +in LaTeX to take input from the terminal is '\typein' (*note \typein::.) + + To read an entire file as additional LaTeX source, use '\input' +(*note \input::) or '\include' (*note \include & \includeonly::). + + A common reason to want to read from a data file is to do mail +merges. CTAN has a number of packages for that; one is 'datatool'. + + +File: latex2e.info, Node: \typein, Next: \typeout, Prev: \read, Up: Input/output + +27.3 '\typein' +============== + +Synopsis, one of: + + \typein{PROMPT-MSG} + \typein[CMD]{PROMPT-MSG} + + Print PROMPT-MSG on the terminal and cause LaTeX to stop and wait for +you to type a line of input. This line of input ends when you hit the +return key. + + For example, this + + As long as I live I shall never forget \typein{Enter student name:} + +coupled with this command line interaction + + Enter student name: + + \@typein=Aphra Behn + +gives the output '... never forget Aphra Behn'. + + The first command version, '\typein{PROMPT-MSG}', causes the input +you typed to be processed as if it had been included in the input file +in place of the '\typein' command. + + In the second command version the optional argument 'CMD' argument +must be a command name, that is, it must begin with a backslash, \. +This command name is then defined or redefined to be the input that you +typed. For example, this + + \typein[\student]{Enter student name:} + \typeout{Recommendation for \student .} + +gives this output on the command line, + + Enter student name: + + \student=John Dee + Recommendation for John Dee. + +where the user has entered 'John Dee.' + + +File: latex2e.info, Node: \typeout, Next: \write, Prev: \typein, Up: Input/output + +27.4 '\typeout' +=============== + +Synopsis: + + \typeout{MSG} + + Print 'msg' on the terminal and in the 'log' file. + + This + + \newcommand{\student}{John Dee} + \typeout{Recommendation for \student .} + +outputs 'Recommendation for John Dee'. Like what happens here with +'\student', commands that are defined with '\newcommand' or +'\renewcommand' (among others) are replaced by their definitions before +being printed. + + LaTeX's usual rules for treating multiple spaces as a single space +and ignoring spaces after a command name apply to 'msg'. Use the +command '\space' to get a single space, independent of surrounding +spaces. Use '^^J' to get a newline. Get a percent character with +'\csname @percentchar\endcsname'. + + This command can be useful for simple debugging, as here: + + \newlength{\jhlength} + \setlength{\jhlength}{5pt} + \typeout{The length is \the\jhlength.} + +produces on the command line 'The length is 5.0pt'. + + +File: latex2e.info, Node: \write, Prev: \typeout, Up: Input/output + +27.5 '\write' +============= + +Synopsis: + + \write NUMBER{STRING} + + Write STRING to the log file, to the terminal, or to a file opened by +'\openout'. For instance, '\write6' writes to text stream number 6. + + If the following appears in 'BASEFILE.tex' then it opens +'BASEFILE.jh', writes 'Hello World!' and a newline to it, and closes +that file. + + \newwrite\myfile + \immediate\openout\myfile=\jobname.jh % \jobname is root file basename + ... + \immediate\write\myfile{Hello world!} + ... + \immediate\closeout\myfile + +The '\newwrite' allocates a stream number, giving it a symbolic name to +make life easier, so that 'stream \newwrite\myfile\the\myfile' produces +something like 'stream 3'. Then '\openout' associates the stream number +with the given file name. TeX ultimately executed '\write3' which puts +the string in the file. + + Typically NUMBER is between 0 and 15, because typically LaTeX authors +follow the prior example and the number is allocated by the system. If +NUMBER is outside the range from 0 to 15 or if it is not associated with +an open file then LaTeX writes STRING to the log file. If NUMBER is +positive then in addition LaTeX writes STRING to the terminal. + + Thus, 'test \write-1{Hello World!}' puts 'Hello World!' followed by a +newline in the log file. (This is what the '\wlog' command does; *note +\wlog::). And '\write100{Hello World!}' puts the same in the log file +but also puts 'Hello World!' followed by a newline in the terminal +output. (But 16, 17, and 18 are special as NUMBER; see below.) + + In LuaTeX, instead of 16 output streams there are 256 (*note TeX +engines::). + + Use '\write\@auxout{STRING}' to write to the current '.aux' file, +which is associated with either the root file or with the current +include file; and use '\write\@mainaux{STRING}' to write to the main +'.aux'. These symbolic names are defined by LaTeX. + + By default LaTeX does not write STRING to the file right away. This +is because, for example, you may need '\write' to save the current page +number, but when TeX comes across a '\write' it typically does not know +what the page number is, since it has not yet done the page breaking. +So, you use '\write' in one of three contexts: + + \immediate\write\@auxout{STRING} %1 + \write\@auxout{STRING} %2 + \protected@write\@auxout{}{STRING} %3 + + 1. With the first, LaTeX writes STRING to the file immediately. Any + macros in STRING are fully expanded (just as in '\edef') so to + prevent expansion you must use '\noexpand', 'toks', etc., except + that you should use '#' instead of '##'). + + 2. With the second, STRING is stored on the current list of things (as + a TeX "whatsit" item) and kept until the page is shipped out and + likewise the macros are unexpanded until '\shipout'. At + '\shipout', STRING is fully expanded. + + 3. The third, '\protected@write', is like the second except that you + can use '\protect' to avoid expansion. The extra first argument + allows you to locally insert extra definitions to make more macros + protected or to have some other special definition for the write. + + As a simple example of expansion with '\write', STRING here contains +a control sequence '\triplex' which we've defined to be the text 'XYZ': + + \newwrite\jhfile + \openout\jhfile=test.jh + \newcommand{\triplex}{XYZ} + \write\jhfile{test \triplex test} + +This results in the file 'test.jh' containing the text 'test XYZtest' +followed by a newline. + + The cases where NUMBER is 16, 17, or 18 are special. Because of +'\write''s behavior when NUMBER is outside the range from 0 to 15 +described above, in Plain TeX '\write16' and '\write17' were sometimes +used to write to the log file and the terminal; however, in LaTeX, the +natural way to do that is with '\typeout' (*note \typeout::). The +'\write18' command is even more special; modern TeX systems use it for +giving commands to the operating system (*note \write18::). + + Ordinarily '\write' outputs a single line. You can include a newline +with '^^J'. Thus, this produces two lines in the log file: + + \wlog{Parallel lines have a lot in common.^^JBut they never meet.} + + A common case where authors need to write their own file is for +answers to exercises, or another situation where you want to write out +verbatim, without expanding the macros. CTAN has a number of packages +for this; one is 'answers'. + +* Menu: + +* \write and security:: Security. +* \message:: Write text to the log file and terminal. +* \wlog:: Write text to the log file. +* \write18:: Run an operating system command. + + +File: latex2e.info, Node: \write and security, Next: \message, Up: \write + +27.5.1 '\write' and security +---------------------------- + +The ability to write files raises security issues. If you compiled a +downloaded LaTeX file and it overwrote your password file then you would +be justifiably troubled. + + Thus, by default TeX systems only allow you to open files for writing +that are in the current directory or output directory, if specified +(*note output directory::), or in a subdirectory of those. So, this +code + + \newwrite\jhfile + \openout\jhfile=../test.jh + +gives an error like: + Not writing to ../test.jh (openout_any = p). + ! I can't write on file `../test.jh' + + You can get just such an error when using commands such as +'\include{../filename}' because LaTeX will try to open +'../filename.aux'. The simplest solution is to put the included files +in the same directory as the root file, or in subdirectories. + + +File: latex2e.info, Node: \message, Next: \wlog, Prev: \write and security, Up: \write + +27.5.2 '\message' +----------------- + +Synopsis: + + \message{STRING} + + Write STRING to the log file and the terminal. + + Typically, LaTeX authors use '\typeout' (*note \typeout::). It +allows you to use '\protect' on any fragile commands in STRING (*note +\protect::). But '\typeout' always inserts a newline at the end of +STRING while '\message' does not, so the latter can be useful. + + With this example document body. + + before\message{One Two}\message{Three}\message{Four^^JI} + \message{declare a thumb war.}After + +under some circumstances (see below) LaTeX writes the following to both +the terminal and the log file. + + One Two Three Four + I declare a thumb war. + +The '^^J' produces a newline. Also, in the output document, between +'before' and 'After' will be a single space (from the end of line +following 'I}'). + + While '\message' allows you more control over formatting, a gotcha is +that LaTeX may mess up that formatting because it inserts line breaks +depending on what it has already written. Contrast this document body, +where the 'Two' has moved, to the one given above. + + before\message{One}\message{Two Three}\message{Four^^JI} + \message{declare a thumb war.}After + + This can happen: when LaTeX is outputting the messages to the +terminal, now the message with 'One' is shorter and it fits at the end +of the output terminal line, and so LaTeX breaks the line between it and +the 'Two Three'. That line break appears also in the log file. This +line break insertion can depend on, for instance, the length of the full +path names of included files. So producing finely-formatted lines in a +way that is portable is hard, likely requiring starting your message at +the beginning of a line. + + +File: latex2e.info, Node: \wlog, Next: \write18, Prev: \message, Up: \write + +27.5.3 '\wlog' +-------------- + +Synopsis: + + \wlog{STRING} + + Write STRING to the log file. + + \wlog{Did you hear about the mathematician who hates negatives?} + \wlog{He'll stop at nothing to avoid them.} + + Ordinarily STRING appears in a single separate line. Use '^^J' to +insert a newline. + + \wlog{Helvetica and Times Roman walk into a bar.} + \wlog{The barman says,^^JWe don't serve your type.} + + +File: latex2e.info, Node: \write18, Prev: \wlog, Up: \write + +27.5.4 '\write18' +----------------- + +Synopsis: + + \write18{SHELL_COMMAND} + + Issue a command to the operating system shell. The operating system +runs the command and LaTeX's execution is blocked until that finishes. + + This sequence (on Unix) + + \usepackage{graphicx} % in preamble + ... + \newcommand{\fignum}{1} + \immediate\write18{cd pix && asy figure\fignum} + \includegraphics{pix\figure\fignum.pdf} + +will run Asymptote (the 'asy' program) on 'pix/figure1.asy', so that the +document can later read in the resulting graphic (*note +\includegraphics::). Like any '\write', here LaTeX expands macros in +SHELL_COMMAND so that '\fignum' is replaced by '1'. + + Another example is that you can automatically run BibTeX at the start +of each LaTeX run (*note Using BibTeX::) by including +'\immediate\write18{bibtex8 \jobname}' as the first line of the file. +Note that '\jobname' expands to the basename of the root file unless the +'--jobname' option is passed on the command line, in which case this is +the option argument. + + You sometimes need to do a multi-step process to get the information +that you want. This will insert into the input a list of all PDF files +in the current directory (but see 'texosquery' below): + + \immediate\write18{ls *.pdf > tmp.dat} + \input{tmp.dat} + + The standard behavior of any '\write' is to wait until a page is +being shipped out before expanding the macros or writing to the stream +(*note \write::). But sometimes you want it done now. For this, use +'\immediate\write18{SHELL_COMMAND}'. + + There are obvious security issues with allowing system commands +inside a LaTeX file. If you download a file off the net and it contains +commands to delete all your files then you would be unhappy. The +standard settings in modern distributions turn off full shell access. +You can turn it on, if you are sure the shell commands are safe, by +compiling with 'latex --enable-write18 FILENAME' (*note Command line +options::). (The '--shell-escape' option is a synonym, in TeX Live.) + + In the place of full shell access, modern distributions by default +use a restricted version that allows some commands to work, such as +those that run Metafont to generate missing fonts, even if you do not +use the 'enable-write18' option. By default this list of allowed +commands is short and features only commands that are under the control +of the distribution maintainers (*note Command line options::). + + The SHELL_COMMAND text is always passed to '/bin/sh' on Unix-like +operating systems, and the DOS command interpreter 'cmd.exe' on Windows. +Any different shell set by the user, and the 'SHELL' environment +variable, is ignored. + + If what you need is system information, such as the operating system +name, locale information, or directory contents, take a look at the +'texosquery' package, which provides a convenient and secure interface +for this, unlike the above examples using the raw '\write18': +. + + LaTeX provides a package 'shellesc' on top of the primitive +'\write18' command. Its primary purpose is to provide a command +'\ShellEscape' which works identically on all TeX engines; LuaTeX +intentionally did not retain '\write18' as a way to invoke a shell +command, so some engine-specific code is needed. The 'shellesc' package +also provides a command '\DelayedShellEscape', executed at '\output' +time, for the same reason. + + +File: latex2e.info, Node: Command line interface, Next: Document templates, Prev: Input/output, Up: Top + +28 Command line interface +************************* + +Synopsis (from a terminal command line): + + pdflatex OPTIONS ARGUMENT + + Run LaTeX on ARGUMENT. In place of 'pdflatex' you can also use (for +PDF output) 'xelatex' or 'lualatex', or (for DVI output) 'latex' or +'dvilualatex', among others (*note TeX engines::). + + For example, this will run LaTeX on the file 'thesis.tex', creating +the output 'thesis.pdf'. + + pdflatex thesis + +Note that '.tex' is the default file name extension. + + pdfTeX is an extension of the original TeX program, as are XeTeX and +LuaTeX (*note TeX engines::). The first two are completely backward +compatible and the latter, almost so. Perhaps the most fundamental new +feature for all three is that the original TeX output its own DVI +format, while the newer ones can output directly to PDF. This allows +them to take advantage of the extra features in PDF such as hyperlinks, +support for modern image formats such as JPG and PNG, and ubiquitous +viewing programs. In short, if you run 'pdflatex' or 'xelatex' or +'lualatex' then you will by default get PDF and have access to all its +modern features. If you run 'latex', or 'dvilualatex', then you will +get DVI. The description here assumes pdfLaTeX. + + *Note Command line options::, for a selection of the most useful +command line options. As to ARGUMENT, the usual case is that it does +not begin with a backslash, so the system takes it to be the name of a +file and it compiles that file. If ARGUMENT begins with a backslash +then the system will interpret it as a line of LaTeX input, which can be +used for special effects (*note Command line input::). + + If you gave no arguments or options then 'pdflatex' prompts for input +from the terminal. You can escape from this by entering 'CTRL-D'. + + If LaTeX finds an error in your document then by default it stops and +asks you about it. *Note Recovering from errors::, for an outline of +what to do. + +* Menu: + +* Command line options:: Commonly used command line options. +* Command line input:: Specify LaTeX code on the command line. +* Jobname:: How TeX sets the current job name. +* Recovering from errors:: When something goes wrong. + + +File: latex2e.info, Node: Command line options, Next: Command line input, Up: Command line interface + +28.1 Command line options +========================= + +These are the command-line options relevant to ordinary document +authoring. For a full list, try running 'latex --help' from the command +line. + + With many implementations you can specify command line options by +prefixing them with '-' or '--'. This is the case for both TeX Live +(including MacTeX) and MiKTeX. We will use both conventions +interchangeably. If an option takes a value, it can be specified either +as a separate argument ('--foo val'), or as one argument with an '=' +sign ('--foo=val'), but there can be no spaces around the '='. We will +generally use the '=' syntax. + +'-version' + Show the current version, like 'pdfTeX 3.14159265-2.6-1.40.16 (TeX + Live 2015/Debian)' along with a small amount of additional + information, and exit. + +'-help' + Give a brief usage message that is useful as a prompt and exit. + +'-interaction=MODE' + TeX compiles a document in one of four interaction modes: + 'batchmode', 'nonstopmode', 'scrollmode', 'errorstopmode'. In + "errorstopmode" (the default), TeX stops at each error and asks for + user intervention. In "batchmode" it prints nothing on the + terminal, errors are scrolled as if the user hit 'RETURN' at every + error, and missing files cause the job to abort. In "nonstopmode", + diagnostic message appear on the terminal but as in batch mode + there is no user interaction. In "scrollmode", TeX stops for + missing files or keyboard input, but nothing else. + + For instance, starting LaTeX with this command line + + pdflatex -interaction=batchmode FILENAME + + eliminates most terminal output. + +'-jobname=STRING' + Set the value of TeX's "jobname" to the string. The log file and + output file will then be named 'STRING.log' and 'STRING.pdf'. + *note Jobname::. + +'-output-directory=DIRECTORY' + Write files in the directory DIRECTORY. It must already exist. + This applies to all external files created by TeX or LaTeX, such as + the '.log' file for the run, the '.aux', '.toc', etc., files + created by LaTeX, as well as the main '.pdf' or '.dvi' output file + itself. + + When specified, the output directory DIRECTORY is also + automatically checked first for any file that it is input, so that + the external files can be read back in, if desired. The true + current directory (in which LaTeX was run) remains unchanged, and + is also checked for input files. + +'--enable-write18' +'--disable-write18' +'--shell-escape' +'--no-shell-escape' + Enable or disable '\write18{SHELL_COMMAND}' (*note \write18::). + The first two options are supported by both TeX Live and MiKTeX, + while the second two are synonyms supported by TeX Live. + + Enabling this functionality has major security implications, since + it allows a LaTeX file to run any command whatsoever. Thus, by + default, unrestricted '\write18' is not allowed. (The default for + TeX Live, MacTeX, and MiKTeX is to allow the execution of a limited + number of TeX-related programs, which they distribute.) + + For example, if you invoke LaTeX with the option 'no-shell-escape', + and in your document you call '\write18{ls -l}', then you do not + get an error but the log file says 'runsystem(ls -l)...disabled'. + +'-halt-on-error' + Stop processing at the first error. + +'-file-line-error' +'-no-file-line-error' + Enable or disable 'FILENAME:LINENO:ERROR'-style error messages. + These are only available with TeX Live or MacTeX. + + +File: latex2e.info, Node: Command line input, Next: Jobname, Prev: Command line options, Up: Command line interface + +28.2 Command line input +======================= + +As part of the command line invocation + + LATEX-ENGINE OPTIONS ARGUMENT + +you can specify arbitrary LaTeX input by starting ARGUMENT with a +backslash. (All the engines support this.) This allows you to do some +special effects. + + For example, this file (which uses the 'hyperref' package for +hyperlinks) can produce two kinds of output, one to be read on physical +paper and one to be read online. + + \ifdefined\paperversion % in preamble + \newcommand{\urlcolor}{black} + \else + \newcommand{\urlcolor}{blue} + \fi + \usepackage[colorlinks=true,urlcolor=\urlcolor]{hyperref} + ... + \href{https://www.ctan.org}{CTAN} % in body + ... + +Compiling this document 'book.tex' with the command line 'pdflatex book' +will give the 'CTAN' link in blue. But compiling it with + + pdflatex "\def\paperversion{}\input book.tex" + +has the link in black. We use double quotes to prevent interpretation +of the symbols by the command line shell. (This usually works on both +Unix and Windows systems, but there are many peculiarities to shell +quoting, so read your system documentation if need be.) + + In a similar way, from the single file 'main.tex' you can compile two +different versions. + + pdflatex -jobname=students "\def\student{}\input{main}" + pdflatex -jobname=teachers "\def\teachers{}\input{main}" + +The 'jobname' option is there because otherwise both files would be +called 'main.pdf' and the second would overwrite the first. (*note +Jobname::.) + + In this example we use the command line to select which parts of a +document to include. For a book named 'mybook.tex' and structured like +this. + + \documentclass{book} + \begin{document} + ... + \include{chap1} + \include{chap2} + ... + \end{document} + +the command line + + pdflatex "\includeonly{chap1}\input{mybook}" + +will give output that has the first chapter but no other chapter. *Note +Splitting the input::. + + +File: latex2e.info, Node: Jobname, Next: Recovering from errors, Prev: Command line input, Up: Command line interface + +28.3 Jobname +============ + +Running LaTeX creates a number of files, including the main PDF (or DVI) +output but also including others. These files are named with the +so-called "jobname". The most common case is also the simplest, where +for instance the command 'pdflatex thesis' creates 'thesis.pdf' and also +'thesis.log' and 'thesis.aux'. Here the job name is 'thesis'. + + In general, LaTeX is invoked as 'LATEX-ENGINE OPTIONS ARGUMENT', +where LATEX-ENGINE is 'pdflatex', 'lualatex', etc. (*note TeX +engines::). If ARGUMENT does not start with a backslash, as is the case +above with 'thesis', then TeX considers it to be the name of the file to +input as the main document. This file is referred to as the "root file" +(*note Splitting the input::, and *note \input::). The name of that +root file, without the '.tex' extension if any, is the jobname. If +ARGUMENT does start with a backslash, or if TeX is in interactive mode, +then it waits for the first '\input' command, and the jobname is the +argument to '\input'. + + There are two more possibilities for the jobname. It can be directly +specified with the '-jobname' option, as in 'pdflatex -jobname=myname' +(*note Command line input:: for a real example). + + The final possibility is 'texput', which is the final fallback +default if no other name is available to TeX. That is, if no '-jobname' +option was specified, and the compilation stops before any input file is +met, then the log file will be named 'texput.log'. + + A special case of this is that in LaTeX versions of (approximately) +2020 or later, the jobname is also 'texput' if the first '\input' occurs +as a result of being called by either '\documentclass' or +'\RequirePackage'. So this will produce a file named 'texput.pdf': + + pdflatex "\documentclass{minimal}\begin{document}Hello!\end{document}" + + However, this special case only applies to those two commands. Thus, +with + + pdflatex "\documentclass{article}\usepackage{lipsum}\input{thesis}" + +the output file is 'lipsum.pdf', as '\usepackage' calls '\input'. + + Within the document, the macro '\jobname' expands to the jobname. +(When you run LaTeX on a file whose name contains spaces, the string +returned by '\jobname' contains matching start and end quotes.) In the +expansion of that macro, all characters are of catcode 12 (other) except +that spaces are category 10, including letters that are normally +catcode 11. + + Because of this catcode situation, using the jobname in a conditional +can become complicated. One solution is to use the macro '\IfBeginWith' +from the 'xstring' package in its star variant, which is insensitive to +catcode. For example, in the following text the footnote "Including +Respublica Bananensis Francorum." is only present if the task name +starts with 'my-doc'. + + If a democracy is just a regime where citizens vote then + all banana republics \IfBeginWith*{\jobname}{my-doc}% + {\footnote{Including Respublica Bananensis Francorum.}}{} are + democracies. + + Manipulating the value of '\jobname' inside of a document does not +change the name of the output file or the log file. + + +File: latex2e.info, Node: Recovering from errors, Prev: Jobname, Up: Command line interface + +28.4 Recovering from errors +=========================== + +If LaTeX finds an error in your document then it gives you an error +message and prompts you with a question mark, '?'. For instance, +running LaTeX on this file + + \newcommand{\NP}{\ensuremath{\textbf{NP}}} + The \PN{} problem is a million dollar one. + +causes it show this, and wait for input. + + ! Undefined control sequence. + l.5 The \PN + {} problem is a million dollar one. + ? + +The simplest thing is to enter 'x' and 'RETURN' and fix the typo. You +could instead enter '?' and 'RETURN' to see other options. + + There are two other error scenarios. The first is that you forgot to +include the '\end{document}' or misspelled it. In this case LaTeX gives +you a '*' prompt. You can get back to the command line by typing +'\stop' and 'RETURN'; this command does its best to exit LaTeX +immediately, whatever state it may be in. + + The last scenario is that you mistyped the filename. For instance, +instead of 'pdflatex test' you might type 'pdflatex tste'. + + ! I can't find file `tste'. + <*> tste + + (Press Enter to retry, or Control-D to exit) + Please type another input file name: + +The simplest thing is to enter 'CTRL d' (holding the Control and d keys +down at the same time), and then retype the correct command line. + + +File: latex2e.info, Node: Document templates, Next: Index, Prev: Command line interface, Up: Top + +Appendix A Document templates +***************************** + +Although illustrative material, perhaps these document templates will be +useful. Additional template resources are listed at +. + +* Menu: + +* beamer template:: +* article template:: +* book template:: +* Larger book template:: + + +File: latex2e.info, Node: beamer template, Next: article template, Up: Document templates + +A.1 'beamer' template +===================== + +The 'beamer' class creates presentation slides. It has a vast array of +features, but here is a basic template: + +\documentclass{beamer} + +\title{Beamer Class template} +\author{Alex Author} +\date{July 31, 2020} + +\begin{document} + +\maketitle + +% without [fragile], any {verbatim} code gets mysterious errors. +\begin{frame}[fragile] + \frametitle{First Slide} + +\begin{verbatim} + This is \verbatim! +\end{verbatim} + +\end{frame} + +\end{document} + + The Beamer package on CTAN: . + + +File: latex2e.info, Node: article template, Next: book template, Prev: beamer template, Up: Document templates + +A.2 'article' template +====================== + +A simple template for an article. + +\documentclass{article} +\title{Article Class Template} +\author{Alex Author} + +\begin{document} +\maketitle + +\section{First section} +Some text. + +\subsection{First section, first subsection} +Additional text. + +\section{Second section} +Some more text. + +\end{document} + + +File: latex2e.info, Node: book template, Next: Larger book template, Prev: article template, Up: Document templates + +A.3 'book' template +=================== + +This is a straightforward template for a book. *Note Larger book +template::, for a more elaborate one. + +\documentclass{book} +\title{Book Class Template} +\author{Alex Author} + +\begin{document} +\maketitle + +\chapter{First} +Some text. + +\chapter{Second} +Some other text. + +\section{A subtopic} +The end. + +\end{document} + + +File: latex2e.info, Node: Larger book template, Prev: book template, Up: Document templates + +A.4 Larger 'book' template +========================== + +This is a somewhat elaborate template for a book. *Note book +template::, for a simpler one. + + This template uses '\frontmatter', '\mainmatter', and '\backmatter' +to control the typography of the three main areas of a book (*note +\frontmatter & \mainmatter & \backmatter::). The book has a +bibliography and an index. + + Also notable is that it uses '\include' and '\includeonly' (*note +Splitting the input::). While you are working on a chapter you can +comment out all the other chapter entries from the argument to +'\includeonly'. That will speed up compilation without losing any +information such as cross-references. (Material that does not need to +come on a new page is brought in with '\input' instead of '\include'. +You don't get the cross-reference benefit with '\input'.) + +\documentclass[titlepage]{book} +\usepackage{makeidx}\makeindex + +\title{Book Class Template} +\author{Alex Author} + +\includeonly{% +% frontcover, + preface, + chap1, +% appenA, + } + +\begin{document} +\frontmatter +\include{frontcover} + % maybe comment out while drafting: +\maketitle \input{dedication} \input{copyright} +\tableofcontents +\include{preface} + +\mainmatter +\include{chap1} +... +\appendix +\include{appenA} +... + +\backmatter +\bibliographystyle{apalike} +\addcontentsline{toc}{chapter}{Bibliography} +\bibliography +\addcontentsline{toc}{chapter}{Index} +\printindex + +\include{backcover} +\end{document} + + +File: latex2e.info, Node: Index, Prev: Document templates, Up: Top + +Index +***** + +[index] +* Menu: + +* $ inline math: math. (line 12) +* $$...$$ plain TeX display math: displaymath. (line 32) +* & for table cells: tabular. (line 32) +* * prompt: Recovering from errors. + (line 23) +* *, to \nocite all keys: Using BibTeX. (line 61) +* *-form of environment commands: \newenvironment & \renewenvironment. + (line 27) +* *-form of sectioning commands: Sectioning. (line 31) +* *-form, defining new commands: \newcommand & \renewcommand. + (line 27) +* --disable-write18 command-line option: Command line options. + (line 61) +* --enable-write18 command-line option: Command line options. + (line 61) +* --file-line-error command-line option: Command line options. + (line 82) +* --halt-on-error command-line option: Command line options. + (line 79) +* --help command-line option: Command line options. + (line 23) +* --interaction command-line option: Command line options. + (line 26) +* --jobname command-line option: Command line options. + (line 43) +* --no-file-line-error command-line option: Command line options. + (line 82) +* --no-shell-escape command-line option: Command line options. + (line 61) +* --output-directory command-line option: Command line options. + (line 48) +* --shell-escape command-line option: Command line options. + (line 61) +* --version command-line option: Command line options. + (line 18) +* -1, write stream number: \write. (line 30) +* .aux file: Output files. (line 33) +* .aux file and BibTeX commands: BibTeX error messages. + (line 6) +* .dvi file: Output files. (line 10) +* .fmt file: TeX engines. (line 6) +* .glo file: Glossaries. (line 43) +* .idx file: Indexes. (line 6) +* .idx file <1>: makeindex. (line 6) +* .ind file: makeindex. (line 6) +* .isty file: makeindex. (line 18) +* .lof file: Output files. (line 43) +* .lof file <1>: Table of contents etc.. + (line 6) +* .lof file <2>: \contentsline. (line 6) +* .log file: Output files. (line 28) +* .lot file: Output files. (line 43) +* .lot file <1>: Table of contents etc.. + (line 6) +* .lot file <2>: \contentsline. (line 6) +* .pdf file: Output files. (line 20) +* .tex, default extension: Command line interface. + (line 19) +* .toc file: Output files. (line 43) +* .toc file <1>: Table of contents etc.. + (line 6) +* .toc file <2>: \contentsline. (line 6) +* .xdv file: TeX engines. (line 42) +* /bin/sh, used by \write18: \write18. (line 60) +* 10pt option: Document class options. + (line 14) +* 11pt option: Document class options. + (line 14) +* 12pt option: Document class options. + (line 14) +* : for math: Colon character & \colon. + (line 6) +* [...] for optional arguments: LaTeX command syntax. + (line 6) +* \ (backslash-space): \(SPACE). (line 6) +* \ character starting commands: LaTeX command syntax. + (line 6) +* \!: Spacing in math mode. + (line 61) +* \" (umlaut accent): Accents. (line 25) +* \#: Reserved characters. (line 13) +* \$: Reserved characters. (line 13) +* \%: Reserved characters. (line 13) +* \&: Reserved characters. (line 13) +* \' (acute accent): Accents. (line 29) +* \' (tabbing): tabbing. (line 87) +* \(...\) inline math: math. (line 12) +* \*: \*. (line 6) +* \+: tabbing. (line 79) +* \,: Spacing in math mode. + (line 47) +* \-: tabbing. (line 83) +* \- (hyphenation): \- (hyphenation). (line 6) +* \. (dot-over accent): Accents. (line 32) +* \/: \/. (line 6) +* \:: Spacing in math mode. + (line 36) +* \;: Spacing in math mode. + (line 23) +* \<: tabbing. (line 75) +* \= (macron accent): Accents. (line 36) +* \= (tabbing): tabbing. (line 69) +* \>: tabbing. (line 73) +* \> <1>: Spacing in math mode. + (line 36) +* \> (tabbing): tabbing. (line 72) +* \@: \@. (line 6) +* \@auxout: \write. (line 45) +* \@beginparpenalty: list. (line 243) +* \@dottedtocline: \@dottedtocline. (line 6) +* \@endparpenalty: list. (line 251) +* \@fnsymbol: \footnote. (line 24) +* \@ifstar: \@ifstar. (line 6) +* \@itempenalty: list. (line 247) +* \@mainaux: \write. (line 45) +* \@startsection: \@startsection. (line 6) +* \a (tabbing): tabbing. (line 102) +* \a' (acute accent in tabbing): tabbing. (line 103) +* \a= (macron accent in tabbing): tabbing. (line 103) +* \aa (å): Additional Latin letters. + (line 12) +* \AA (Å): Additional Latin letters. + (line 12) +* \accent: \accent. (line 6) +* \acute: Math accents. (line 10) +* \addcontentsline: \addcontentsline. (line 6) +* \address: \address. (line 6) +* \addtocontents{EXT}{TEXT}: \addtocontents. (line 6) +* \addtocounter: \addtocounter. (line 6) +* \addtolength: \addtolength. (line 6) +* \addvspace: \addvspace. (line 6) +* \ae (æ): Additional Latin letters. + (line 16) +* \AE (Æ): Additional Latin letters. + (line 16) +* \aleph: Math symbols. (line 21) +* \Alph example: enumerate. (line 61) +* \alpha: Math symbols. (line 24) +* \alph{COUNTER}: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 14) +* \Alph{COUNTER}: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 19) +* \alsoname: \index. (line 58) +* \amalg: Math symbols. (line 27) +* \and for \author: \maketitle. (line 43) +* \angle: Math symbols. (line 30) +* \appendix: \appendix. (line 6) +* \approx: Math symbols. (line 34) +* \arabic{COUNTER}: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 24) +* \arccos: Math functions. (line 9) +* \arcsin: Math functions. (line 12) +* \arctan: Math functions. (line 15) +* \arg: Math functions. (line 18) +* \arraycolsep: array. (line 51) +* \arrayrulewidth: tabular. (line 157) +* \arraystretch: tabular. (line 163) +* \ast: Math symbols. (line 37) +* \asymp: Math symbols. (line 44) +* \AtBeginDocument: \AtBeginDocument. (line 6) +* \AtBeginDvi: Class and package commands. + (line 9) +* \AtEndDocument: \AtEndDocument. (line 6) +* \AtEndOfClass: Class and package commands. + (line 14) +* \AtEndOfPackage: Class and package commands. + (line 14) +* \author{NAME1 \and NAME2 \and ...}: \maketitle. (line 42) +* \a` (grave accent in tabbing): tabbing. (line 103) +* \b (bar-under accent): Accents. (line 51) +* \backmatter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* \backslash: Math symbols. (line 47) +* \bar: Math accents. (line 13) +* \baselineskip: Low-level font commands. + (line 101) +* \baselineskip <1>: \baselineskip & \baselinestretch. + (line 6) +* \baselinestretch: \baselineskip & \baselinestretch. + (line 6) +* \begin: Environments. (line 6) +* \beta: Math symbols. (line 51) +* \bf: Font styles. (line 75) +* \bfseries: Font styles. (line 40) +* \bibdata: BibTeX error messages. + (line 16) +* \bibitem: \bibitem. (line 6) +* \bibliography: Using BibTeX. (line 6) +* \bibliography and internal \bibdata: BibTeX error messages. + (line 17) +* \bibliographystyle: Using BibTeX. (line 6) +* \bibliographystyle and internal \bibstyle: BibTeX error messages. + (line 20) +* \bibname: thebibliography. (line 44) +* \bibstyle: BibTeX error messages. + (line 19) +* \bigcap: Math symbols. (line 54) +* \bigcirc: Math symbols. (line 58) +* \bigcup: Math symbols. (line 61) +* \bigodot: Math symbols. (line 65) +* \bigoplus: Math symbols. (line 68) +* \bigotimes: Math symbols. (line 71) +* \bigskip: \bigskip & \medskip & \smallskip. + (line 28) +* \bigskipamount: \bigskip & \medskip & \smallskip. + (line 29) +* \bigsqcup: Math symbols. (line 80) +* \bigtriangledown: Math symbols. (line 74) +* \bigtriangleup: Math symbols. (line 77) +* \biguplus: Math symbols. (line 83) +* \bigvee: Math symbols. (line 87) +* \bigwedge: Math symbols. (line 90) +* \bmod: Math functions. (line 21) +* \boldmath: \boldmath & \unboldmath. + (line 6) +* \boldmath <1>: \boldmath & \unboldmath. + (line 14) +* \bot: Math symbols. (line 93) +* \bottomfraction: Floats. (line 94) +* \bottomfraction <1>: Floats. (line 95) +* \bowtie: Math symbols. (line 97) +* \Box: Math symbols. (line 100) +* \breve: Math accents. (line 16) +* \bullet: Math symbols. (line 105) +* \c (cedilla accent): Accents. (line 61) +* \cal: Font styles. (line 78) +* \cap: Math symbols. (line 108) +* \capitalacute: Accents. (line 29) +* \capitalbreve: Accents. (line 89) +* \capitalcaron: Accents. (line 93) +* \capitalcedilla: Accents. (line 61) +* \capitalcircumflex: Accents. (line 40) +* \capitaldieresis: Accents. (line 25) +* \capitaldotaccent: Accents. (line 65) +* \capitalgrave: Accents. (line 44) +* \capitalhungarumlaut: Accents. (line 69) +* \capitalmacron: Accents. (line 36) +* \capitalnewtie: Accents. (line 83) +* \capitalogonek: Accents. (line 73) +* \capitalring: Accents. (line 77) +* \capitaltie: Accents. (line 83) +* \capitaltilde: Accents. (line 48) +* \caption: \caption. (line 6) +* \caption <1>: figure. (line 43) +* \caption <2>: table. (line 42) +* \cc: \cc. (line 6) +* \cdot: Math symbols. (line 112) +* \cdots: Dots. (line 17) +* \centering: \centering. (line 6) +* \chapter: Sectioning. (line 6) +* \chapter <1>: \chapter. (line 6) +* \check: Math accents. (line 19) +* \CheckCommand: Class and package commands. + (line 21) +* \CheckCommand*: Class and package commands. + (line 21) +* \chi: Math symbols. (line 115) +* \circ: Math symbols. (line 118) +* \circle: \circle. (line 6) +* \citation: BibTeX error messages. + (line 22) +* \cite: \cite. (line 6) +* \cite and internal \citation: BibTeX error messages. + (line 23) +* \ClassError: Class and package commands. + (line 41) +* \ClassInfo: Class and package commands. + (line 41) +* \ClassInfoNoLine: Class and package commands. + (line 41) +* \ClassWarning: Class and package commands. + (line 41) +* \ClassWarningNoLine: Class and package commands. + (line 41) +* \cleardoublepage: \clearpage & \cleardoublepage. + (line 6) +* \clearpage: \clearpage & \cleardoublepage. + (line 6) +* \cline: \cline. (line 6) +* \closein: \openin & \openout. (line 6) +* \closeout: \openin & \openout. (line 6) +* \closing: \closing. (line 6) +* \clubsuit: Math symbols. (line 122) +* \colon: Colon character & \colon. + (line 6) +* \columnsep: \twocolumn. (line 20) +* \columnsep <1>: Page layout parameters. + (line 6) +* \columnsep <2>: Page layout parameters. + (line 9) +* \columnseprule: \twocolumn. (line 26) +* \columnseprule <1>: Page layout parameters. + (line 7) +* \columnseprule <2>: Page layout parameters. + (line 9) +* \columnwidth: \twocolumn. (line 33) +* \columnwidth <1>: Page layout parameters. + (line 8) +* \columnwidth <2>: Page layout parameters. + (line 9) +* \complement: Math symbols. (line 125) +* \cong: Math symbols. (line 131) +* \contentsline: Table of contents etc.. + (line 62) +* \contentsline <1>: \addcontentsline. (line 10) +* \contentsline <2>: \contentsline. (line 6) +* \coprod: Math symbols. (line 134) +* \copyright: Text symbols. (line 13) +* \cos: Math functions. (line 24) +* \cosh: Math functions. (line 27) +* \cot: Math functions. (line 30) +* \coth: Math functions. (line 33) +* \csc: Math functions. (line 36) +* \cup: Math symbols. (line 137) +* \CurrentOption: Class and package commands. + (line 60) +* \d (dot-under accent): Accents. (line 65) +* \dag: Text symbols. (line 17) +* \dagger: Math symbols. (line 141) +* \dashbox: \dashbox. (line 6) +* \dashv: Math symbols. (line 144) +* \date{TEXT}: \maketitle. (line 50) +* \day: \day & \month & \year. + (line 6) +* \dblfloatpagefraction: \twocolumn. (line 66) +* \dblfloatsep: \twocolumn. (line 72) +* \dbltextfloatsep: \twocolumn. (line 79) +* \dbltopfraction: \twocolumn. (line 45) +* \dbltopnumber: \twocolumn. (line 84) +* \ddag: Text symbols. (line 20) +* \ddagger: Math symbols. (line 148) +* \ddot: Math accents. (line 22) +* \ddots: Dots. (line 21) +* \DeclareFontEncoding: \DeclareFontEncoding. + (line 6) +* \DeclareGraphicsExtensions: \DeclareGraphicsExtensions. + (line 6) +* \DeclareGraphicsRule: \DeclareGraphicsRule. + (line 6) +* \DeclareOption: Class and package commands. + (line 66) +* \DeclareOption*: Class and package commands. + (line 66) +* \DeclareRobustCommand: Class and package commands. + (line 96) +* \DeclareRobustCommand*: Class and package commands. + (line 96) +* \DeclareTextAccent: \DeclareTextAccent. (line 6) +* \DeclareTextAccent <1>: \DeclareTextAccentDefault. + (line 6) +* \DeclareTextAccentDefault: \DeclareTextAccentDefault. + (line 6) +* \DeclareTextCommand: \DeclareTextCommand & \ProvideTextCommand. + (line 6) +* \DeclareTextCommand <1>: \DeclareTextCommandDefault & \ProvideTextCommandDefault. + (line 6) +* \DeclareTextCommandDefault: \DeclareTextCommandDefault & \ProvideTextCommandDefault. + (line 6) +* \DeclareTextComposite: \DeclareTextComposite. + (line 6) +* \DeclareTextCompositeCommand: \DeclareTextCompositeCommand. + (line 6) +* \DeclareTextSymbol: \DeclareTextCommand & \ProvideTextCommand. + (line 37) +* \DeclareTextSymbol <1>: \DeclareTextSymbol. (line 6) +* \DeclareTextSymbol <2>: \DeclareTextSymbolDefault. + (line 6) +* \DeclareTextSymbolDefault: \DeclareTextSymbolDefault. + (line 6) +* \deg: Math functions. (line 39) +* \DelayedShellEscape: \write18. (line 71) +* \Delta: Math symbols. (line 151) +* \delta: Math symbols. (line 154) +* \det: Math functions. (line 42) +* \dh (ð): Additional Latin letters. + (line 20) +* \DH (Ð): Additional Latin letters. + (line 20) +* \Diamond: Math symbols. (line 157) +* \diamond: Math symbols. (line 161) +* \diamondsuit: Math symbols. (line 165) +* \dim: Math functions. (line 45) +* \displaystyle: Math styles. (line 29) +* \div: Math symbols. (line 168) +* \dj: Additional Latin letters. + (line 26) +* \DJ: Additional Latin letters. + (line 26) +* \documentclass: Document classes. (line 6) +* \documentclass, and texput jobname: Jobname. (line 32) +* \dot: Math accents. (line 25) +* \doteq: Math symbols. (line 171) +* \dotfill: \hrulefill & \dotfill. + (line 6) +* \dots: Text symbols. (line 37) +* \doublerulesep: tabular. (line 168) +* \downarrow: Math symbols. (line 175) +* \Downarrow: Math symbols. (line 179) +* \ell: Math symbols. (line 183) +* \emph: Font styles. (line 60) +* \emptyset: Math symbols. (line 186) +* \encl: \encl. (line 6) +* \end: Environments. (line 6) +* \endinput: \endinput. (line 6) +* \enlargethispage: \enlargethispage. (line 6) +* \enspace: \enspace & \quad & \qquad. + (line 6) +* \enumi: enumerate. (line 47) +* \enumii: enumerate. (line 47) +* \enumiii: enumerate. (line 47) +* \enumiv: enumerate. (line 47) +* \epsilon: Math symbols. (line 189) +* \equiv: Math symbols. (line 195) +* \eta: Math symbols. (line 198) +* \evensidemargin: Document class options. + (line 84) +* \evensidemargin <1>: Page layout parameters. + (line 67) +* \evensidemargin <2>: Page layout parameters. + (line 68) +* \ExecuteOptions: Class and package commands. + (line 172) +* \exists: Math symbols. (line 201) +* \exp: Math functions. (line 48) +* \externaldocument: xr package. (line 6) +* \extracolsep: tabular. (line 108) +* \fbox: \fbox & \framebox. (line 6) +* \fboxrule: \framebox (picture). (line 31) +* \fboxrule <1>: \fbox & \framebox. (line 38) +* \fboxrule <2>: \fbox & \framebox. (line 37) +* \fboxsep: \framebox (picture). (line 31) +* \fboxsep <1>: \fbox & \framebox. (line 43) +* \fboxsep <2>: \fbox & \framebox. (line 42) +* \fill: \hfill. (line 18) +* \flat: Math symbols. (line 204) +* \floatpagefraction: Floats. (line 98) +* \floatpagefraction <1>: Floats. (line 99) +* \floatsep: Floats. (line 114) +* \floatsep <1>: Floats. (line 115) +* \flushbottom: \flushbottom. (line 6) +* \fnsymbol, and footnotes: \footnote. (line 24) +* \fnsymbol{COUNTER}: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 37) +* \fontdimen1: \/. (line 40) +* \fontencoding: Low-level font commands. + (line 11) +* \fontfamily: Low-level font commands. + (line 21) +* \fontseries: Low-level font commands. + (line 48) +* \fontshape: Low-level font commands. + (line 88) +* \fontsize: Low-level font commands. + (line 101) +* \footnote: \footnote. (line 6) +* \footnotemark: \footnotemark. (line 6) +* \footnoterule: \footnote. (line 34) +* \footnotesep: \footnote. (line 46) +* \footnotesize: Font sizes. (line 11) +* \footnotetext: \footnotetext. (line 6) +* \footskip: Page layout parameters. + (line 26) +* \footskip <1>: Page layout parameters. + (line 27) +* \forall: Math symbols. (line 207) +* \frac: \frac. (line 6) +* \frame: \frame. (line 6) +* \framebox: \framebox (picture). (line 6) +* \framebox <1>: \fbox & \framebox. (line 6) +* \frenchspacing: \frenchspacing & \nonfrenchspacing. + (line 6) +* \frontmatter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* \frown: Math symbols. (line 210) +* \fussy: \fussy & \sloppy. (line 6) +* \Gamma: Math symbols. (line 213) +* \gamma: Math symbols. (line 216) +* \gcd: Math functions. (line 51) +* \ge: Math symbols. (line 219) +* \geq: Math symbols. (line 223) +* \gets: Math symbols. (line 227) +* \gg: Math symbols. (line 230) +* \gls: \gls. (line 6) +* \graphicspath: \graphicspath. (line 6) +* \graphpaper: \graphpaper. (line 6) +* \grave: Math accents. (line 28) +* \guillemotleft («): Text symbols. (line 29) +* \guillemotright (»): Text symbols. (line 30) +* \guilsinglleft (‹): Text symbols. (line 31) +* \guilsinglright (›): Text symbols. (line 32) +* \H (Hungarian umlaut accent): Accents. (line 69) +* \hat: Math accents. (line 31) +* \hbar: Math symbols. (line 233) +* \headheight: Page layout parameters. + (line 14) +* \headheight <1>: Page layout parameters. + (line 15) +* \headsep: Page layout parameters. + (line 19) +* \headsep <1>: Page layout parameters. + (line 20) +* \heartsuit: Math symbols. (line 236) +* \hfill: \hfill. (line 6) +* \hline: \hline. (line 6) +* \hom: Math functions. (line 54) +* \hookleftarrow: Math symbols. (line 239) +* \hookrightarrow: Math symbols. (line 242) +* \hrulefill: \hrulefill & \dotfill. + (line 6) +* \hsize: Page layout parameters. + (line 119) +* \hsize <1>: Page layout parameters. + (line 120) +* \hspace: \hspace. (line 6) +* \hss: \hss. (line 6) +* \huge: Font sizes. (line 11) +* \Huge: Font sizes. (line 11) +* \hyphenation: \hyphenation. (line 6) +* \i (dotless i): Accents. (line 19) +* \IfBeginWith* macro from xstring: Jobname. (line 53) +* \iff: Math symbols. (line 245) +* \IfFileExists: Class and package commands. + (line 128) +* \ignorespaces: \ignorespaces & \ignorespacesafterend. + (line 6) +* \ignorespacesafterend: \ignorespaces & \ignorespacesafterend. + (line 6) +* \ij (ij): Additional Latin letters. + (line 32) +* \IJ (IJ): Additional Latin letters. + (line 32) +* \Im: Math symbols. (line 249) +* \imath: Math symbols. (line 252) +* \immediate\write: \write. (line 60) +* \in: Math symbols. (line 256) +* \include: \include & \includeonly. + (line 6) +* \includegraphics: \includegraphics. (line 6) +* \includeonly: \include & \includeonly. + (line 6) +* \indent: \indent & \noindent. (line 6) +* \index: Indexes. (line 6) +* \index <1>: \index. (line 6) +* \indexentry: \index. (line 108) +* \indexspace: Produce the index manually. + (line 22) +* \indexspace <1>: makeindex. (line 59) +* \inf: Math functions. (line 57) +* \infty: Math symbols. (line 260) +* \input: \input. (line 6) +* \inputencoding: inputenc package. (line 50) +* \InputIfFileExists: Class and package commands. + (line 128) +* \int: Math symbols. (line 263) +* \intextsep: Floats. (line 118) +* \intextsep <1>: Floats. (line 119) +* \iota: Math symbols. (line 266) +* \it: Font styles. (line 81) +* \item: description. (line 31) +* \item <1>: enumerate. (line 29) +* \item <2>: itemize. (line 6) +* \item <3>: itemize. (line 30) +* \item <4>: Produce the index manually. + (line 22) +* \itemindent: list. (line 84) +* \itemsep: list. (line 88) +* \itshape: Font styles. (line 34) +* \j (dotless j): Accents. (line 19) +* \jmath: Math symbols. (line 272) +* \jobname: Jobname. (line 6) +* \jobname <1>: Jobname. (line 46) +* \Join: Math symbols. (line 269) +* \k (ogonek): Accents. (line 73) +* \kappa: Math symbols. (line 276) +* \ker: Math functions. (line 60) +* \kill: tabbing. (line 107) +* \l (ł): Additional Latin letters. + (line 36) +* \L (Ł): Additional Latin letters. + (line 36) +* \l@chapter: \contentsline. (line 6) +* \l@section: \contentsline. (line 6) +* \l@subsection: \contentsline. (line 6) +* \label: \label. (line 6) +* \labelenumi: enumerate. (line 56) +* \labelenumii: enumerate. (line 56) +* \labelenumiii: enumerate. (line 56) +* \labelenumiv: enumerate. (line 56) +* \labelitemi: itemize. (line 42) +* \labelitemii: itemize. (line 42) +* \labelitemiii: itemize. (line 42) +* \labelitemiv: itemize. (line 42) +* \labelsep: list. (line 100) +* \labelwidth: list. (line 105) +* \Lambda: Math symbols. (line 279) +* \lambda: Math symbols. (line 282) +* \land: Math symbols. (line 285) +* \langle: Math symbols. (line 289) +* \large: Font sizes. (line 11) +* \Large: Font sizes. (line 11) +* \LARGE: Font sizes. (line 11) +* \LastDeclaredEncoding: \LastDeclaredEncoding. + (line 6) +* \LaTeX: Text symbols. (line 23) +* \LaTeXe: Text symbols. (line 26) +* \lbrace: Math symbols. (line 293) +* \lbrack: Math symbols. (line 296) +* \lceil: Math symbols. (line 299) +* \ldots: Dots. (line 24) +* \ldots <1>: Text symbols. (line 36) +* \le: Math symbols. (line 303) +* \leadsto: Math symbols. (line 306) +* \left: \left & \right. (line 6) +* \Leftarrow: Math symbols. (line 312) +* \leftarrow: Math symbols. (line 316) +* \lefteqn: eqnarray. (line 44) +* \leftharpoondown: Math symbols. (line 320) +* \leftharpoonup: Math symbols. (line 323) +* \leftmargin: itemize. (line 58) +* \leftmargin <1>: list. (line 125) +* \leftmargini: itemize. (line 58) +* \leftmarginii: itemize. (line 58) +* \leftmarginiii: itemize. (line 58) +* \leftmarginiv: itemize. (line 58) +* \leftmarginv: itemize. (line 58) +* \leftmarginvi: itemize. (line 58) +* \Leftrightarrow: Math symbols. (line 326) +* \leftrightarrow: Math symbols. (line 330) +* \leq: Math symbols. (line 334) +* \lfloor: Math symbols. (line 337) +* \lg: Math functions. (line 63) +* \lhd: Math symbols. (line 340) +* \lim: Math functions. (line 66) +* \liminf: Math functions. (line 69) +* \limsup: Math functions. (line 72) +* \line: \line. (line 6) +* \linebreak: \linebreak & \nolinebreak. + (line 6) +* \lineskip: \baselineskip & \baselinestretch. + (line 45) +* \lineskiplimit: \baselineskip & \baselinestretch. + (line 45) +* \linespread: Low-level font commands. + (line 112) +* \linespread <1>: \baselineskip & \baselinestretch. + (line 6) +* \linethickness: \linethickness. (line 6) +* \linewidth: Page layout parameters. + (line 33) +* \linewidth <1>: Page layout parameters. + (line 34) +* \listoffigures: Table of contents etc.. + (line 6) +* \listoffigures <1>: \contentsline. (line 6) +* \listoftables: Table of contents etc.. + (line 6) +* \listoftables <1>: \contentsline. (line 6) +* \listparindent: list. (line 142) +* \ll: Math symbols. (line 346) +* \ln: Math functions. (line 75) +* \lnot: Math symbols. (line 349) +* \LoadClass: Class and package commands. + (line 148) +* \LoadClassWithOptions: Class and package commands. + (line 148) +* \location: \location. (line 6) +* \log: Math functions. (line 78) +* \long: \newcommand & \renewcommand. + (line 27) +* \longleftarrow: Math symbols. (line 352) +* \longleftrightarrow: Math symbols. (line 356) +* \longmapsto: Math symbols. (line 360) +* \longrightarrow: Math symbols. (line 364) +* \lor: Math symbols. (line 368) +* \lq: Text symbols. (line 42) +* \mainmatter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* \makebox: \mbox & \makebox. (line 6) +* \makebox (for picture): \makebox (picture). (line 6) +* \makeglossary: Glossaries. (line 6) +* \makeglossary <1>: Glossaries. (line 43) +* \makeindex: Indexes. (line 6) +* \makelabel: list. (line 61) +* \makelabels: \makelabels. (line 6) +* \maketitle: \maketitle. (line 6) +* \mapsto: Math symbols. (line 372) +* \marginpar: Marginal notes. (line 6) +* \marginparpush: Page layout parameters. + (line 40) +* \marginparpush <1>: Page layout parameters. + (line 43) +* \marginparpush <2>: Marginal notes. (line 35) +* \marginparsep: Marginal notes. (line 40) +* \marginparwidth: Page layout parameters. + (line 42) +* \marginparwidth <1>: Page layout parameters. + (line 43) +* \marginparwidth <2>: Marginal notes. (line 44) +* \marginsep: Page layout parameters. + (line 41) +* \marginsep <1>: Page layout parameters. + (line 43) +* \markboth{LEFT-HEAD}{RIGHT-HEAD}: \pagestyle. (line 75) +* \markright{RIGHT}: \pagestyle. (line 84) +* \mathbf: Font styles. (line 111) +* \mathcal: Font styles. (line 127) +* \mathdollar: Math symbols. (line 832) +* \mathnormal: Font styles. (line 124) +* \mathparagraph: Math symbols. (line 835) +* \mathring: Math accents. (line 34) +* \mathrm: Font styles. (line 108) +* \mathsection: Math symbols. (line 838) +* \mathsf: Font styles. (line 114) +* \mathsterling: Math symbols. (line 841) +* \mathstrut: \mathstrut. (line 6) +* \mathtt: Font styles. (line 117) +* \mathunderscore: Math symbols. (line 844) +* \mathversion: Font styles. (line 129) +* \max: Math functions. (line 81) +* \mbox: \mbox & \makebox. (line 6) +* \mdseries: Font styles. (line 37) +* \medskip: \bigskip & \medskip & \smallskip. + (line 33) +* \medskipamount: \bigskip & \medskip & \smallskip. + (line 34) +* \medspace: Spacing in math mode. + (line 36) +* \message: \message. (line 6) +* \mho: Math symbols. (line 376) +* \mid: Math symbols. (line 379) +* \min: Math functions. (line 84) +* \models: Math symbols. (line 389) +* \month: \day & \month & \year. + (line 6) +* \mp: Math symbols. (line 393) +* \mu: Math symbols. (line 396) +* \multicolumn: \multicolumn. (line 6) +* \multiput: \multiput. (line 6) +* \nabla: Math symbols. (line 399) +* \name: \name. (line 6) +* \natural: Math symbols. (line 402) +* \ne: Math symbols. (line 405) +* \nearrow: Math symbols. (line 408) +* \NeedsTeXFormat: Class and package commands. + (line 182) +* \neg: Math symbols. (line 411) +* \negmedspace: Spacing in math mode. + (line 42) +* \negthickspace: Spacing in math mode. + (line 30) +* \negthinspace: Spacing in math mode. + (line 61) +* \negthinspace <1>: \thinspace & \negthinspace. + (line 6) +* \neq: Math symbols. (line 415) +* \newcommand: \newcommand & \renewcommand. + (line 6) +* \newcounter: \newcounter. (line 6) +* \newenvironment: \newenvironment & \renewenvironment. + (line 6) +* \newfont: \newfont. (line 6) +* \newglossaryentry: \newglossaryentry. (line 6) +* \newlength: \newlength. (line 6) +* \newline: \newline. (line 6) +* \: \(SPACE). (line 6) +* \newpage: \newpage. (line 6) +* \newsavebox: \newsavebox. (line 6) +* \newtheorem: \newtheorem. (line 6) +* \newtie: Accents. (line 83) +* \newwrite: \write. (line 24) +* \ng: Additional Latin letters. + (line 40) +* \NG: Additional Latin letters. + (line 40) +* \ni: Math symbols. (line 418) +* \nocite: \nocite. (line 6) +* \nocite and internal \citation: BibTeX error messages. + (line 23) +* \nocite {*}, for all keys: Using BibTeX. (line 61) +* \nocorr: Font styles. (line 22) +* \nocorrlist: Font styles. (line 22) +* \nofiles: \nofiles. (line 6) +* \noindent: \indent & \noindent. (line 6) +* \nolinebreak: \linebreak & \nolinebreak. + (line 6) +* \nonfrenchspacing: \frenchspacing & \nonfrenchspacing. + (line 6) +* \nonumber: eqnarray. (line 39) +* \nopagebreak: \pagebreak & \nopagebreak. + (line 6) +* \normalfont: Font styles. (line 58) +* \normalmarginpar: Marginal notes. (line 22) +* \normalsfcodes: \normalsfcodes. (line 6) +* \normalsize: Font sizes. (line 11) +* \not: Math symbols. (line 422) +* \notin: Math symbols. (line 430) +* \nu: Math symbols. (line 434) +* \numberline: \numberline. (line 6) +* \nwarrow: Math symbols. (line 437) +* \o (ø): Additional Latin letters. + (line 44) +* \O (Ø): Additional Latin letters. + (line 44) +* \obeycr: \obeycr & \restorecr. + (line 6) +* \oddsidemargin: Document class options. + (line 84) +* \oddsidemargin <1>: Page layout parameters. + (line 66) +* \oddsidemargin <2>: Page layout parameters. + (line 68) +* \odot: Math symbols. (line 440) +* \oe (œ): Additional Latin letters. + (line 48) +* \OE (Œ): Additional Latin letters. + (line 48) +* \oint: Math symbols. (line 444) +* \oldstylenums: Font styles. (line 133) +* \Omega: Math symbols. (line 447) +* \omega: Math symbols. (line 450) +* \ominus: Math symbols. (line 453) +* \onecolumn: \onecolumn. (line 6) +* \openin: \openin & \openout. (line 6) +* \opening: \opening. (line 6) +* \openout: \openin & \openout. (line 6) +* \oplus: Math symbols. (line 456) +* \OptionNotUsed: Class and package commands. + (line 200) +* \oslash: Math symbols. (line 460) +* \otimes: Math symbols. (line 463) +* \oval: \oval. (line 6) +* \overbrace{MATH}: Over- and Underlining. + (line 39) +* \overline{TEXT}: Over- and Underlining. + (line 20) +* \owns: Math symbols. (line 467) +* \P: Text symbols. (line 45) +* \PackageError: Class and package commands. + (line 41) +* \PackageInfo: Class and package commands. + (line 41) +* \PackageInfoNoLine: Class and package commands. + (line 41) +* \PackageWarning: Class and package commands. + (line 41) +* \PackageWarningNoLine: Class and package commands. + (line 41) +* \pagebreak: \pagebreak & \nopagebreak. + (line 6) +* \pagenumbering: \pagenumbering. (line 6) +* \pageref: \pageref. (line 6) +* \pagestyle: \pagestyle. (line 6) +* \paperheight: Page layout parameters. + (line 79) +* \paperheight <1>: Page layout parameters. + (line 80) +* \paperwidth: Page layout parameters. + (line 85) +* \paperwidth <1>: Page layout parameters. + (line 86) +* \par: \par. (line 6) +* \paragraph: Sectioning. (line 6) +* \paragraph <1>: \subsubsection & \paragraph & \subparagraph. + (line 6) +* \parallel: Math symbols. (line 471) +* \parbox: \parbox. (line 6) +* \parindent: minipage. (line 103) +* \parindent <1>: \indent & \noindent. (line 6) +* \parindent <2>: \parindent & \parskip. + (line 6) +* \parsep: list. (line 148) +* \parskip: \parindent & \parskip. + (line 6) +* \parskip example: itemize. (line 78) +* \part: Sectioning. (line 6) +* \part <1>: \part. (line 6) +* \partial: Math symbols. (line 474) +* \partopsep: list. (line 157) +* \PassOptionsToClass: Class and package commands. + (line 206) +* \PassOptionsToPackage: Class and package commands. + (line 206) +* \pdfpageheight: Document class options. + (line 40) +* \pdfpagewidth: Document class options. + (line 40) +* \perp: Math symbols. (line 477) +* \Phi: Math symbols. (line 481) +* \phi: Math symbols. (line 484) +* \Pi: Math symbols. (line 488) +* \pi: Math symbols. (line 491) +* \pm: Math symbols. (line 495) +* \pmod: Math functions. (line 87) +* \poptabs: tabbing. (line 113) +* \poptabs <1>: tabbing. (line 114) +* \pounds: Text symbols. (line 49) +* \Pr: Math functions. (line 90) +* \prec: Math symbols. (line 498) +* \preceq: Math symbols. (line 501) +* \prevdepth: \baselineskip & \baselinestretch. + (line 45) +* \prime: Math symbols. (line 505) +* \printglossaries: Glossaries. (line 6) +* \printglossaries <1>: Glossaries. (line 43) +* \printindex: \printindex. (line 6) +* \ProcessOptions: Class and package commands. + (line 243) +* \ProcessOptions*: Class and package commands. + (line 243) +* \prod: Math symbols. (line 514) +* \propto: Math symbols. (line 517) +* \protect: \protect. (line 6) +* \protected@write: \write. (line 70) +* \providecommand: \providecommand. (line 6) +* \ProvidesClass: Class and package commands. + (line 281) +* \ProvidesFile: Class and package commands. + (line 314) +* \ProvidesPackage: Class and package commands. + (line 281) +* \ProvideTextCommand: \DeclareTextCommand & \ProvideTextCommand. + (line 6) +* \ProvideTextCommand <1>: \DeclareTextCommandDefault & \ProvideTextCommandDefault. + (line 6) +* \ProvideTextCommandDefault: \DeclareTextCommandDefault & \ProvideTextCommandDefault. + (line 6) +* \ps: \ps. (line 6) +* \Psi: Math symbols. (line 520) +* \psi: Math symbols. (line 523) +* \pushtabs: tabbing. (line 116) +* \put: \put. (line 6) +* \qbezier: \qbezier. (line 6) +* \qquad: Spacing in math mode. + (line 75) +* \qquad <1>: \enspace & \quad & \qquad. + (line 6) +* \quad: Spacing in math mode. + (line 69) +* \quad <1>: \enspace & \quad & \qquad. + (line 6) +* \quotedblbase („): Text symbols. (line 53) +* \quotesinglbase (‚): Text symbols. (line 54) +* \r (ring accent): Accents. (line 77) +* \raggedbottom: \raggedbottom. (line 6) +* \raggedleft: \raggedleft. (line 6) +* \raggedright: \raggedright. (line 6) +* \raisebox: \raisebox. (line 6) +* \rangle: Math symbols. (line 526) +* \rbrace: Math symbols. (line 530) +* \rbrack: Math symbols. (line 533) +* \rceil: Math symbols. (line 537) +* \Re: Math symbols. (line 540) +* \read: \read. (line 6) +* \ref: \ref. (line 6) +* \reflectbox: \scalebox. (line 6) +* \refname: thebibliography. (line 44) +* \refstepcounter: \refstepcounter. (line 6) +* \renewenvironment: \newenvironment & \renewenvironment. + (line 6) +* \RequirePackage: Class and package commands. + (line 323) +* \RequirePackage, and texput jobname: Jobname. (line 32) +* \RequirePackageWithOptions: Class and package commands. + (line 323) +* \resizebox: \resizebox. (line 6) +* \restorecr: \obeycr & \restorecr. + (line 6) +* \restriction: Math symbols. (line 545) +* \revemptyset: Math symbols. (line 550) +* \reversemarginpar: Marginal notes. (line 22) +* \rfloor: Math symbols. (line 555) +* \rhd: Math symbols. (line 559) +* \rho: Math symbols. (line 565) +* \right: \left & \right. (line 6) +* \Rightarrow: Math symbols. (line 569) +* \rightarrow: Math symbols. (line 573) +* \rightharpoondown: Math symbols. (line 577) +* \rightharpoonup: Math symbols. (line 580) +* \rightleftharpoons: Math symbols. (line 583) +* \rightmargin: list. (line 172) +* \rm: Font styles. (line 84) +* \rmfamily: Font styles. (line 31) +* \roman{COUNTER}: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 27) +* \Roman{COUNTER}: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 32) +* \rotatebox: \rotatebox. (line 6) +* \rq: Text symbols. (line 57) +* \rule: \rule. (line 6) +* \S: Text symbols. (line 60) +* \samepage: \pagebreak & \nopagebreak. + (line 44) +* \savebox: \sbox & \savebox. (line 6) +* \sbox: \sbox & \savebox. (line 6) +* \sc: Font styles. (line 87) +* \scalebox: \scalebox. (line 6) +* \scriptscriptstyle: Math styles. (line 29) +* \scriptsize: Font sizes. (line 11) +* \scriptstyle: Math styles. (line 29) +* \scshape: Font styles. (line 52) +* \searrow: Math symbols. (line 586) +* \sec: Math functions. (line 93) +* \section: Sectioning. (line 6) +* \section <1>: \section. (line 6) +* \seename: \index. (line 58) +* \selectfont: Low-level font commands. + (line 118) +* \setcounter: \setcounter. (line 6) +* \setlength: \setlength. (line 6) +* \setminus: Math symbols. (line 589) +* \settodepth: \settodepth. (line 6) +* \settoheight: \settoheight. (line 6) +* \settowidth: \settowidth. (line 6) +* \sf: Font styles. (line 90) +* \sffamily: Font styles. (line 49) +* \sharp: Math symbols. (line 594) +* \ShellEscape: \write18. (line 71) +* \shipout and expansion: \write. (line 65) +* \shortstack: \shortstack. (line 6) +* \Sigma: Math symbols. (line 597) +* \sigma: Math symbols. (line 600) +* \signature: \signature. (line 11) +* \sim: Math symbols. (line 604) +* \simeq: Math symbols. (line 607) +* \sin: Math functions. (line 96) +* \sinh: Math functions. (line 99) +* \sl: Font styles. (line 93) +* \sloppy: \fussy & \sloppy. (line 6) +* \slshape: Font styles. (line 46) +* \small: Font sizes. (line 11) +* \smallint: Math symbols. (line 610) +* \smallskip: \bigskip & \medskip & \smallskip. + (line 38) +* \smallskipamount: \bigskip & \medskip & \smallskip. + (line 39) +* \smile: Math symbols. (line 614) +* \: \(SPACE). (line 6) +* \ <1>: \(SPACE). (line 6) +* \space: \include & \includeonly. + (line 88) +* \space <1>: \input. (line 32) +* \space <2>: \typeout. (line 22) +* \spacefactor: \spacefactor. (line 10) +* \spadesuit: Math symbols. (line 617) +* \sqcap: Math symbols. (line 620) +* \sqcup: Math symbols. (line 624) +* \sqrt: \sqrt. (line 6) +* \sqsubset: Math symbols. (line 628) +* \sqsubseteq: Math symbols. (line 633) +* \sqsupset: Math symbols. (line 637) +* \sqsupseteq: Math symbols. (line 642) +* \ss (ß): Additional Latin letters. + (line 52) +* \SS (SS): Additional Latin letters. + (line 52) +* \stackrel: \stackrel. (line 6) +* \star: Math symbols. (line 646) +* \stepcounter: \stepcounter. (line 6) +* \stop: Recovering from errors. + (line 23) +* \stretch: \stretch. (line 6) +* \strut: \strut. (line 6) +* \subitem: Produce the index manually. + (line 22) +* \subparagraph: Sectioning. (line 6) +* \subparagraph <1>: \subsubsection & \paragraph & \subparagraph. + (line 6) +* \subsection: Sectioning. (line 6) +* \subsection <1>: \subsection. (line 6) +* \subset: Math symbols. (line 653) +* \subseteq: Math symbols. (line 656) +* \subsubitem: Produce the index manually. + (line 22) +* \subsubsection: Sectioning. (line 6) +* \subsubsection <1>: \subsubsection & \paragraph & \subparagraph. + (line 6) +* \succ: Math symbols. (line 659) +* \succeq: Math symbols. (line 662) +* \sum: Math symbols. (line 666) +* \sup: Math functions. (line 102) +* \suppressfloats: Floats. (line 87) +* \supset: Math symbols. (line 669) +* \supseteq: Math symbols. (line 672) +* \surd: Math symbols. (line 675) +* \swarrow: Math symbols. (line 680) +* \symbol: Symbols by font position. + (line 6) +* \t (tie-after accent): Accents. (line 83) +* \: \(SPACE). (line 6) +* \tabbingsep: tabbing. (line 121) +* \tabcolsep: tabular. (line 172) +* \tableofcontents: Table of contents etc.. + (line 6) +* \tableofcontents <1>: \contentsline. (line 6) +* \tan: Math functions. (line 105) +* \tanh: Math functions. (line 108) +* \tau: Math symbols. (line 683) +* \telephone: \telephone. (line 6) +* \TeX: Text symbols. (line 64) +* \textascendercompwordmark: Text symbols. (line 102) +* \textasciicircum: Text symbols. (line 67) +* \textasciitilde: Text symbols. (line 70) +* \textasteriskcentered: Text symbols. (line 73) +* \textbackslash: Reserved characters. (line 18) +* \textbackslash <1>: Text symbols. (line 76) +* \textbar: Text symbols. (line 79) +* \textbardbl: Text symbols. (line 82) +* \textbf: Font styles. (line 40) +* \textbigcircle: Text symbols. (line 85) +* \textbraceleft: Text symbols. (line 88) +* \textbraceright: Text symbols. (line 91) +* \textbullet: Text symbols. (line 94) +* \textcapitalcompwordmark: Text symbols. (line 101) +* \textcircled{LETTER}: Text symbols. (line 97) +* \textcompwordmark: Text symbols. (line 100) +* \textcopyright: Text symbols. (line 14) +* \textdagger: Text symbols. (line 109) +* \textdaggerdbl: Text symbols. (line 112) +* \textdollar (or \$): Text symbols. (line 115) +* \textellipsis: Text symbols. (line 38) +* \textemdash (or ---): Text symbols. (line 118) +* \textendash (or --): Text symbols. (line 124) +* \texteuro: Text symbols. (line 127) +* \textexclamdown (or !`): Text symbols. (line 135) +* \textfiguredash: Text symbols. (line 138) +* \textfloatsep: Floats. (line 123) +* \textfloatsep <1>: Floats. (line 124) +* \textfraction: Floats. (line 102) +* \textfraction <1>: Floats. (line 103) +* \textgreater: Text symbols. (line 145) +* \textheight: Page layout parameters. + (line 91) +* \textheight <1>: Page layout parameters. + (line 92) +* \texthorizontalbar: Text symbols. (line 148) +* \textit: Font styles. (line 34) +* \textleftarrow: Text symbols. (line 156) +* \textless: Text symbols. (line 153) +* \textmd: Font styles. (line 37) +* \textnonbreakinghyphen: Text symbols. (line 159) +* \textnormal: Font styles. (line 58) +* \textordfeminine: Text symbols. (line 165) +* \textordmasculine: Text symbols. (line 166) +* \textparagraph: Text symbols. (line 46) +* \textperiodcentered: Text symbols. (line 169) +* \textquestiondown (or ?`): Text symbols. (line 172) +* \textquotedblleft (or ``): Text symbols. (line 175) +* \textquotedblright (or ''): Text symbols. (line 178) +* \textquoteleft (or `): Text symbols. (line 181) +* \textquoteright (or '): Text symbols. (line 184) +* \textquotesingle: Text symbols. (line 187) +* \textquotestraightbase: Text symbols. (line 190) +* \textquotestraightdblbase: Text symbols. (line 191) +* \textregistered: Text symbols. (line 194) +* \textrightarrow: Text symbols. (line 197) +* \textrm: Font styles. (line 31) +* \textsc: Font styles. (line 52) +* \textsection: Text symbols. (line 61) +* \textsf: Font styles. (line 49) +* \textsl: Font styles. (line 46) +* \textsterling: Text symbols. (line 50) +* \textstyle: Math styles. (line 29) +* \textthreequartersemdash: Text symbols. (line 200) +* \texttrademark: Text symbols. (line 203) +* \texttt: Font styles. (line 55) +* \texttwelveudash: Text symbols. (line 206) +* \textunderscore: Text symbols. (line 209) +* \textup: Font styles. (line 43) +* \textvisiblespace: Text symbols. (line 212) +* \textwidth: Page layout parameters. + (line 99) +* \textwidth <1>: Page layout parameters. + (line 100) +* \th (þ): Additional Latin letters. + (line 56) +* \TH (Þ): Additional Latin letters. + (line 56) +* \thanks{TEXT}: \maketitle. (line 56) +* \theta: Math symbols. (line 686) +* \thicklines: \thicklines. (line 6) +* \thickspace: Spacing in math mode. + (line 23) +* \thinlines: \thinlines. (line 6) +* \thinspace: Spacing in math mode. + (line 47) +* \thinspace <1>: \thinspace & \negthinspace. + (line 6) +* \thispagestyle: \thispagestyle. (line 6) +* \tilde: Math accents. (line 37) +* \times: Math symbols. (line 690) +* \tiny: Font sizes. (line 11) +* \title{TEXT}: \maketitle. (line 63) +* \to: Math symbols. (line 693) +* \today: \today. (line 6) +* \top: Math symbols. (line 697) +* \topfraction: Floats. (line 107) +* \topfraction <1>: Floats. (line 108) +* \topmargin: Page layout parameters. + (line 124) +* \topsep: list. (line 177) +* \topskip: Page layout parameters. + (line 131) +* \topskip <1>: Page layout parameters. + (line 132) +* \triangle: Math symbols. (line 701) +* \triangleleft: Math symbols. (line 704) +* \triangleright: Math symbols. (line 710) +* \tt: Font styles. (line 96) +* \ttfamily: Font styles. (line 55) +* \twocolumn: \twocolumn. (line 6) +* \typein: \typein. (line 6) +* \typeout: \typeout. (line 6) +* \u (breve accent): Accents. (line 89) +* \unboldmath: \boldmath & \unboldmath. + (line 6) +* \unboldmath <1>: \boldmath & \unboldmath. + (line 14) +* \underbar: Accents. (line 53) +* \underbrace{MATH}: Over- and Underlining. + (line 25) +* \underline{TEXT}: Over- and Underlining. + (line 9) +* \unitlength: picture. (line 28) +* \unlhd: Math symbols. (line 716) +* \unrhd: Math symbols. (line 722) +* \Uparrow: Math symbols. (line 728) +* \uparrow: Math symbols. (line 732) +* \Updownarrow: Math symbols. (line 736) +* \updownarrow: Math symbols. (line 741) +* \upharpoonright: Math symbols. (line 746) +* \uplus: Math symbols. (line 751) +* \upshape: Font styles. (line 43) +* \Upsilon: Math symbols. (line 756) +* \upsilon: Math symbols. (line 759) +* \usebox: \usebox. (line 6) +* \usecounter: \usecounter. (line 6) +* \usefont: Low-level font commands. + (line 126) +* \usepackage: Additional packages. (line 6) +* \UseTextAccent: \UseTextSymbol & \UseTextAccent. + (line 6) +* \UseTextSymbol: \UseTextSymbol & \UseTextAccent. + (line 6) +* \v (breve accent): Accents. (line 93) +* \value: \value. (line 6) +* \vanothing: Math symbols. (line 767) +* \varepsilon: Math symbols. (line 762) +* \varphi: Math symbols. (line 772) +* \varpi: Math symbols. (line 776) +* \varrho: Math symbols. (line 780) +* \varsigma: Math symbols. (line 784) +* \vartheta: Math symbols. (line 788) +* \vbox (plain TeX): minipage. (line 57) +* \vdash: Math symbols. (line 792) +* \vdots: Dots. (line 33) +* \vec: Math accents. (line 40) +* \vector: \vector. (line 6) +* \vee: Math symbols. (line 796) +* \verb: \verb. (line 6) +* \Vert: Math symbols. (line 800) +* \vert: Math symbols. (line 804) +* \vfill: \vfill. (line 6) +* \vline: \vline. (line 6) +* \vspace: \vspace. (line 6) +* \vtop plain TeX: minipage. (line 53) +* \wedge: Math symbols. (line 810) +* \widehat: Math accents. (line 43) +* \widetilde: Math accents. (line 46) +* \wlog: \wlog. (line 6) +* \wp: Math symbols. (line 814) +* \wr: Math symbols. (line 817) +* \write: \write. (line 6) +* \write and security: \write and security. (line 6) +* \write streams 16, 17, 18: \write. (line 86) +* \write18: \write18. (line 6) +* \write18, enabling: Command line options. + (line 61) +* \Xi: Math symbols. (line 820) +* \xi: Math symbols. (line 823) +* \xspace: xspace package. (line 6) +* \year: \day & \month & \year. + (line 6) +* \zeta: Math symbols. (line 826) +* \[...\] display math: displaymath. (line 26) +* \\ (for center): center. (line 16) +* \\ (for eqnarray): eqnarray. (line 29) +* \\ (for flushright): flushright. (line 12) +* \\ (for \shortstack objects): \shortstack. (line 35) +* \\ (tabbing): tabbing. (line 66) +* \\ for flushleft: flushleft. (line 14) +* \\ for letters: Letters. (line 48) +* \\ for tabular: tabular. (line 36) +* \\ for verse: verse. (line 21) +* \\ for \author: \maketitle. (line 43) +* \\ for \title: \maketitle. (line 64) +* \\ force line break: \\. (line 6) +* \\* (for eqnarray): eqnarray. (line 35) +* \^: Reserved characters. (line 18) +* \^ (circumflex accent): Accents. (line 40) +* \_: Reserved characters. (line 13) +* \` (grave accent): Accents. (line 44) +* \` (tabbing): tabbing. (line 93) +* \{: Reserved characters. (line 13) +* \|: Math symbols. (line 18) +* \}: Reserved characters. (line 13) +* \~: Reserved characters. (line 18) +* \~ (tilde accent): Accents. (line 48) +* ^ superscript: Subscripts & superscripts. + (line 6) +* ^^J, in \write: \write. (line 94) +* _ subscript: Subscripts & superscripts. + (line 6) +* {...} for required arguments: LaTeX command syntax. + (line 6) +* ~: ~. (line 6) +* a4paper option: Document class options. + (line 19) +* a5paper option: Document class options. + (line 19) +* abstract environment: abstract. (line 6) +* abstract package: abstract. (line 27) +* abstracts: abstract. (line 6) +* accents: Accents. (line 6) +* accents, defining: \DeclareFontEncoding. + (line 6) +* accents, defining <1>: \DeclareTextAccent. (line 6) +* accents, defining <2>: \DeclareTextAccentDefault. + (line 6) +* accents, defining <3>: \DeclareTextComposite. + (line 6) +* accents, defining <4>: \DeclareTextCompositeCommand. + (line 6) +* accents, defining <5>: \DeclareTextSymbolDefault. + (line 6) +* accents, mathematical: Math accents. (line 6) +* accessing any character of a font: Symbols by font position. + (line 6) +* acronyms, list of: Glossaries. (line 6) +* acute accent: Accents. (line 29) +* acute accent, math: Math accents. (line 11) +* additional packages, loading: Additional packages. (line 6) +* adjustbox package: Boxes. (line 9) +* ae ligature: Additional Latin letters. + (line 16) +* algorithm2e package: tabbing. (line 146) +* align environment, from amsmath: eqnarray. (line 6) +* aligning equations: eqnarray. (line 6) +* alignment via tabbing: tabbing. (line 6) +* amscd package: Arrows. (line 41) +* amsfonts package: Math formulas. (line 61) +* amsfonts package <1>: Arrows. (line 6) +* amsmath package: array. (line 55) +* amsmath package <1>: array. (line 63) +* amsmath package <2>: displaymath. (line 22) +* amsmath package <3>: equation. (line 20) +* amsmath package <4>: theorem. (line 28) +* amsmath package <5>: Math formulas. (line 61) +* amsmath package <6>: \left & \right. (line 30) +* amsmath package <7>: \bigl & \bigr etc.. (line 67) +* amsmath package <8>: Dots. (line 36) +* amsmath package <9>: Math functions. (line 111) +* amsmath package <10>: Spacing in math mode. + (line 23) +* amsmath package <11>: \phantom & \vphantom & \hphantom. + (line 56) +* amsmath package <12>: Colon character & \colon. + (line 15) +* amsmath package <13>: Colon character & \colon. + (line 19) +* amsmath package <14>: \thinspace & \negthinspace. + (line 26) +* amsmath package, replacing eqnarray: eqnarray. (line 6) +* amsthm package: theorem. (line 28) +* amsthm package <1>: \rule. (line 18) +* answers package: \write. (line 99) +* appendices: \appendix. (line 6) +* appendix: \appendix. (line 6) +* appendix package: \appendix. (line 29) +* aring: Additional Latin letters. + (line 12) +* array environment: array. (line 6) +* array package: array. (line 74) +* arrays, math: array. (line 6) +* arrow, left, in text: Text symbols. (line 157) +* arrow, right, in text: Text symbols. (line 198) +* arrows: Arrows. (line 6) +* article class: Document classes. (line 11) +* ascender height: Text symbols. (line 103) +* ASCII circumflex, in text: Text symbols. (line 68) +* ASCII tilde, in text: Text symbols. (line 71) +* asterisk, centered, in text: Text symbols. (line 74) +* Asymptote package: \line. (line 35) +* Asymptote package <1>: \strut. (line 62) +* Asymptote package <2>: \mbox & \makebox. (line 74) +* Asymptote package <3>: \write18. (line 13) +* at clause, in font definitions: \newfont. (line 19) +* at-sign: \@. (line 6) +* author, for titlepage: \maketitle. (line 43) +* auxiliary file: Output files. (line 33) +* b5paper option: Document class options. + (line 19) +* babel package: \chapter. (line 71) +* babel package <1>: thebibliography. (line 53) +* babel package <2>: Accents. (line 6) +* babel package <3>: \today. (line 13) +* babel package <4>: Table of contents etc.. + (line 123) +* babel package <5>: \index. (line 58) +* back matter of a book: \frontmatter & \mainmatter & \backmatter. + (line 6) +* background, colored: Colored pages. (line 6) +* backslash, in text: Text symbols. (line 77) +* bar, double vertical, in text: Text symbols. (line 83) +* bar, vertical, in text: Text symbols. (line 80) +* bar-over accent: Accents. (line 36) +* bar-over accent, math: Math accents. (line 14) +* bar-under accent: Accents. (line 51) +* basics of LaTeX: Overview. (line 6) +* batchmode: Command line options. + (line 27) +* beamer package: beamer template. (line 6) +* beamer template and class: beamer template. (line 6) +* beginning of document hook: \AtBeginDocument. (line 6) +* bibliography format, open: Document class options. + (line 67) +* bibliography, creating (automatically): Using BibTeX. (line 6) +* bibliography, creating (manually): thebibliography. (line 6) +* BibTeX error messages: BibTeX error messages. + (line 6) +* bibTeX, using: Using BibTeX. (line 6) +* big circle symbols, in text: Text symbols. (line 86) +* big point: Units of length. (line 20) +* bigfoot package: Footnotes of footnotes. + (line 6) +* black boxes, omitting: Document class options. + (line 53) +* blackboard bold: Blackboard bold. (line 6) +* bm package: \boldmath & \unboldmath. + (line 32) +* bold font: Font styles. (line 76) +* bold math: Font styles. (line 129) +* bold typewriter, avoiding: description. (line 36) +* boldface mathematics: \boldmath & \unboldmath. + (line 6) +* book class: Document classes. (line 11) +* book, back matter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* book, end matter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* book, front matter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* book, main matter: \frontmatter & \mainmatter & \backmatter. + (line 6) +* bottomnumber: Floats. (line 130) +* bottomnumber <1>: Floats. (line 131) +* bounding box: \includegraphics. (line 121) +* box: \mbox & \makebox. (line 6) +* box, allocating new: \newsavebox. (line 6) +* box, bounding: \includegraphics. (line 121) +* box, colored: Colored boxes. (line 6) +* box, save: \sbox & \savebox. (line 6) +* box, use saved box: \usebox. (line 6) +* boxes: Boxes. (line 6) +* bp: Units of length. (line 19) +* brace, left, in text: Text symbols. (line 89) +* brace, right, in text: Text symbols. (line 92) +* braces: Delimiters. (line 6) +* brackets: Delimiters. (line 6) +* breaking lines: Line breaking. (line 6) +* breaking pages: Page breaking. (line 6) +* breaks, multiplication discretionary: \*. (line 6) +* breve accent: Accents. (line 89) +* breve accent, math: Math accents. (line 17) +* bug reporting: About this document. (line 24) +* bullet lists: itemize. (line 6) +* bullet symbol: Math symbols. (line 106) +* bullet, in text: Text symbols. (line 95) +* bulleted lists: itemize. (line 6) +* calligraphic fonts: Calligraphic. (line 6) +* calligraphic letters for math: Font styles. (line 79) +* cap height: Text symbols. (line 103) +* caption package: \caption. (line 72) +* captions: \caption. (line 6) +* caron accent: Accents. (line 93) +* catcode: \makeatletter & \makeatother. + (line 18) +* category code, character: \makeatletter & \makeatother. + (line 18) +* cc: Units of length. (line 32) +* cc list, in letters: \cc. (line 6) +* cedilla accent: Accents. (line 61) +* center environment: center. (line 6) +* centered asterisk, in text: Text symbols. (line 74) +* centered equations: Document class options. + (line 57) +* centered period, in text: Text symbols. (line 170) +* centering text, declaration for: \centering. (line 6) +* centering text, environment for: center. (line 6) +* centimeter: Units of length. (line 27) +* changing case of characters: Upper and lower case. + (line 6) +* chapter: Sectioning. (line 6) +* chapter <1>: \chapter. (line 6) +* chapter counter: Counters. (line 15) +* character category code: \makeatletter & \makeatother. + (line 18) +* character encoding: inputenc package. (line 15) +* character, invisible: \phantom & \vphantom & \hphantom. + (line 6) +* character, invisible <1>: \mathstrut. (line 6) +* characters, accented: Accents. (line 6) +* characters, case of: Upper and lower case. + (line 6) +* characters, non-English: Additional Latin letters. + (line 6) +* characters, reserved: Reserved characters. (line 6) +* characters, special: Reserved characters. (line 6) +* check accent: Accents. (line 93) +* check accent, math: Math accents. (line 20) +* cicero: Units of length. (line 33) +* circle symbol, big, in text: Text symbols. (line 86) +* circled letter, in text: Text symbols. (line 98) +* circumflex accent: Accents. (line 40) +* circumflex accent, math: Math accents. (line 32) +* circumflex, ASCII, in text: Text symbols. (line 68) +* citation key: \bibitem. (line 15) +* class and package commands: Class and package commands. + (line 6) +* class and package difference: Class and package construction. + (line 15) +* class and package structure: Class and package structure. + (line 6) +* class file example: Class and package structure. + (line 39) +* class file layout: Class and package structure. + (line 6) +* class options: Document class options. + (line 6) +* class options <1>: Class and package structure. + (line 6) +* class options <2>: Class and package commands. + (line 66) +* classes of documents: Document classes. (line 6) +* cleveref package: Cross references. (line 38) +* cleveref package <1>: \ref. (line 29) +* cleveref package <2>: \footnotemark. (line 48) +* CLI: Command line interface. + (line 6) +* clock option to slides class: Document class options. + (line 99) +* closing letters: \closing. (line 6) +* closing quote: Text symbols. (line 58) +* cm: Units of length. (line 26) +* cm-super package: fontenc package. (line 36) +* cmd.exe, used by \write18: \write18. (line 60) +* code, typesetting: verbatim. (line 6) +* colon character: Colon character & \colon. + (line 6) +* color: Color. (line 6) +* color <1>: Define colors. (line 6) +* color <2>: Colored text. (line 6) +* color <3>: Colored boxes. (line 6) +* color <4>: Colored pages. (line 6) +* color models: Color models. (line 6) +* color package commands: Commands for color. (line 6) +* color package options: Color package options. + (line 6) +* color, define: Define colors. (line 6) +* colored boxes: Colored boxes. (line 6) +* colored page: Colored pages. (line 6) +* colored text: Colored text. (line 6) +* command line interface: Command line interface. + (line 6) +* command syntax: LaTeX command syntax. + (line 6) +* commands, class and package: Class and package commands. + (line 6) +* commands, defining new ones: \newcommand & \renewcommand. + (line 6) +* commands, defining new ones <1>: \providecommand. (line 6) +* commands, document class: Class and package construction. + (line 6) +* commands, graphics package: Commands for graphics. + (line 6) +* commands, ignore spaces: \ignorespaces & \ignorespacesafterend. + (line 6) +* commands, ignore spaces <1>: xspace package. (line 6) +* commands, redefining: \newcommand & \renewcommand. + (line 6) +* commands, run from LaTeX: \write18. (line 6) +* commands, star-variants: \@ifstar. (line 6) +* composite word mark, in text: Text symbols. (line 103) +* comprehensive package: Math symbols. (line 6) +* computer programs, typesetting: verbatim. (line 6) +* configuration, graphics package: Graphics package configuration. + (line 6) +* contents file: Output files. (line 43) +* copyright symbol: Text symbols. (line 15) +* counters, a list of: Counters. (line 6) +* counters, defining new: \newcounter. (line 6) +* counters, getting value of: \value. (line 6) +* counters, printing: \alph \Alph \arabic \roman \Roman \fnsymbol. + (line 6) +* counters, setting: \setcounter. (line 6) +* cprotect package: verbatim. (line 25) +* cprotect package <1>: \verb. (line 45) +* creating pictures: picture. (line 6) +* creating tables: table. (line 6) +* credit footnote: \maketitle. (line 57) +* cross references: Cross references. (line 6) +* cross references, resolving: Output files. (line 33) +* cross referencing with page number: \pageref. (line 6) +* cross referencing, across documents: xr package. (line 6) +* cross referencing, symbolic: \ref. (line 6) +* CTAN: CTAN. (line 6) +* curly braces: Delimiters. (line 6) +* currency, dollar: Text symbols. (line 116) +* currency, euro: Text symbols. (line 128) +* dagger, double, in text: Text symbols. (line 113) +* dagger, in text: Text symbols. (line 18) +* dagger, in text <1>: Text symbols. (line 110) +* DANTE e.V.: CTAN. (line 20) +* datatool package: \read. (line 31) +* date, for titlepage: \maketitle. (line 51) +* date, today's: \today. (line 6) +* datetime package: \today. (line 27) +* dbltopnumber: Floats. (line 134) +* dbltopnumber <1>: Floats. (line 135) +* dcolumn package: array. (line 74) +* dd: Units of length. (line 29) +* declaration form of font size commands: Font sizes. (line 24) +* declaration form of font style commands: Font styles. (line 8) +* define color: Define colors. (line 6) +* defining a new command: \newcommand & \renewcommand. + (line 6) +* defining a new command <1>: \providecommand. (line 6) +* defining new environments: \newenvironment & \renewenvironment. + (line 6) +* defining new fonts: \newfont. (line 6) +* defining new theorems: \newtheorem. (line 6) +* definitions: Definitions. (line 6) +* delayed \write: \write. (line 65) +* delimiters: Delimiters. (line 6) +* delimiters, paired: \left & \right. (line 6) +* delim_0: makeindex. (line 109) +* delim_1: makeindex. (line 113) +* delim_2: makeindex. (line 117) +* delim_n: makeindex. (line 121) +* delim_r: makeindex. (line 125) +* description: \newglossaryentry. (line 46) +* description environment: description. (line 6) +* description lists, creating: description. (line 6) +* design size, in font definitions: \newfont. (line 19) +* didot point: Units of length. (line 30) +* dieresis accent: Accents. (line 25) +* difference between class and package: Class and package construction. + (line 15) +* dimen plain TeX: Lengths. (line 9) +* directory listings, from system: \write18. (line 65) +* discretionary breaks, multiplication: \*. (line 6) +* discretionary hyphenation: \discretionary. (line 6) +* display math mode: Modes. (line 31) +* display style: Math styles. (line 18) +* displaying quoted text with paragraph indentation: quotation & quote. + (line 6) +* displaying quoted text without paragraph indentation: quotation & quote. + (line 6) +* displaymath environment: displaymath. (line 6) +* displaymath environment <1>: Math formulas. (line 6) +* document class commands: Class and package construction. + (line 6) +* document class options: Document class options. + (line 6) +* document class, defined: Starting and ending. (line 17) +* document classes: Document classes. (line 6) +* document environment: document. (line 6) +* document root name: Jobname. (line 6) +* document templates: Document templates. (line 6) +* dollar sign: Text symbols. (line 116) +* dot accent: Accents. (line 32) +* dot over accent, math: Math accents. (line 26) +* dot-over accent: Accents. (line 32) +* dot-under accent: Accents. (line 65) +* dotless i: Accents. (line 19) +* dotless i, math: Math symbols. (line 253) +* dotless j: Accents. (line 19) +* dotless j, math: Math symbols. (line 273) +* dots: Dots. (line 6) +* double angle quotation marks: Text symbols. (line 33) +* double dagger, in text: Text symbols. (line 21) +* double dagger, in text <1>: Text symbols. (line 113) +* double dot accent, math: Math accents. (line 23) +* double guillemets: Text symbols. (line 33) +* double left quote: Text symbols. (line 176) +* double low-9 quotation mark: Text symbols. (line 55) +* double quote, straight base: Text symbols. (line 192) +* double right quote: Text symbols. (line 179) +* double spacing: \baselineskip & \baselinestretch. + (line 6) +* double vertical bar, in text: Text symbols. (line 83) +* doublestruck: Blackboard bold. (line 6) +* draft option: Document class options. + (line 49) +* dvilualatex-dev: TeX engines. (line 63) +* dvipdfmx command: Output files. (line 10) +* dvips command: Output files. (line 10) +* dvitype command: Output files. (line 10) +* e-dash: Text symbols. (line 125) +* e-TeX: TeX engines. (line 16) +* ellipses: Dots. (line 6) +* ellipsis: Text symbols. (line 39) +* em: Units of length. (line 46) +* em <1>: Units of length. (line 45) +* em-dash: Text symbols. (line 119) +* em-dash, three-quarters: Text symbols. (line 201) +* em-dash, two-thirds: Text symbols. (line 207) +* emphasis: Font styles. (line 60) +* enclosure list: \encl. (line 6) +* encoding, font: \DeclareFontEncoding. + (line 6) +* encoding, of input files: inputenc package. (line 15) +* end matter of a book: \frontmatter & \mainmatter & \backmatter. + (line 6) +* end of document hook: \AtEndDocument. (line 6) +* ending and starting: Starting and ending. (line 6) +* engines, TeX: TeX engines. (line 6) +* enlarge current page: \enlargethispage. (line 6) +* enumerate environment: enumerate. (line 6) +* enumi counter: Counters. (line 15) +* enumii counter: Counters. (line 15) +* enumiii counter: Counters. (line 15) +* enumitem package: itemize. (line 66) +* enumitem package <1>: list. (line 255) +* enumiv counter: Counters. (line 15) +* environment: Starting and ending. (line 25) +* environment form of font size commands: Font sizes. (line 47) +* environment, abstract: abstract. (line 6) +* environment, array: array. (line 6) +* environment, center: center. (line 6) +* environment, description: description. (line 6) +* environment, displaymath: displaymath. (line 6) +* environment, displaymath <1>: Math formulas. (line 6) +* environment, document: document. (line 6) +* environment, enumerate: enumerate. (line 6) +* environment, eqnarray: eqnarray. (line 6) +* environment, equation: equation. (line 6) +* environment, equation <1>: Math formulas. (line 6) +* environment, figure: figure. (line 6) +* environment, filecontents: filecontents. (line 6) +* environment, filecontents*: filecontents. (line 6) +* environment, flushleft: flushleft. (line 6) +* environment, flushright: flushright. (line 6) +* environment, itemize: itemize. (line 6) +* environment, letter: letter. (line 6) +* environment, list: list. (line 6) +* environment, math: math. (line 6) +* environment, math <1>: Math formulas. (line 6) +* environment, minipage: minipage. (line 6) +* environment, picture: picture. (line 6) +* environment, quotation: quotation & quote. (line 6) +* environment, quote: quotation & quote. (line 6) +* environment, sloppypar: sloppypar. (line 6) +* environment, tabbing: tabbing. (line 6) +* environment, table: table. (line 6) +* environment, tabular: tabular. (line 6) +* environment, thebibliography: thebibliography. (line 6) +* environment, theindex: Produce the index manually. + (line 9) +* environment, theorem: theorem. (line 6) +* environment, theorem-like: \newtheorem. (line 6) +* environment, titlepage: titlepage. (line 6) +* environment, verbatim: verbatim. (line 6) +* environment, verse: verse. (line 6) +* environments: Environments. (line 6) +* environments, defining: \newenvironment & \renewenvironment. + (line 6) +* envlab package: \makelabels. (line 55) +* EPS files: \includegraphics. (line 6) +* eqnarray environment: eqnarray. (line 6) +* equation counter: Counters. (line 15) +* equation environment: equation. (line 6) +* equation environment <1>: Math formulas. (line 6) +* equation number, cross referencing: \ref. (line 6) +* equation numbers, left vs. right: Document class options. + (line 63) +* equation numbers, omitting: eqnarray. (line 39) +* equations, aligning: eqnarray. (line 6) +* equations, environment for: equation. (line 6) +* equations, flush left vs. centered: Document class options. + (line 57) +* error messages, from BibTeX: BibTeX error messages. + (line 6) +* errorstopmode: Command line options. + (line 27) +* es-zet German letter: Additional Latin letters. + (line 52) +* etex command: TeX engines. (line 16) +* eth, Icelandic letter: Additional Latin letters. + (line 20) +* etoolbox package: Class and package commands. + (line 110) +* euro symbol: Text symbols. (line 128) +* eurosym package: Text symbols. (line 128) +* eurosym package <1>: Text symbols. (line 129) +* ex: Units of length. (line 42) +* ex <1>: Units of length. (line 41) +* exclamation point, upside-down: Text symbols. (line 136) +* executivepaper option: Document class options. + (line 19) +* expl3 package: LaTeX command syntax. + (line 35) +* expl3 package <1>: Upper and lower case. + (line 48) +* exponent: Subscripts & superscripts. + (line 6) +* expressions: Expressions. (line 6) +* extended Latin: Additional Latin letters. + (line 6) +* external commands: \write18. (line 6) +* external files, writing: filecontents. (line 6) +* fallback jobname: Jobname. (line 27) +* families, of fonts: Low-level font commands. + (line 21) +* fancyhdr package: Page styles. (line 12) +* fancyhdr package <1>: \pagestyle. (line 13) +* fancyvrb package: tabbing. (line 146) +* fancyvrb package <1>: verbatim. (line 41) +* feminine ordinal symbol: Text symbols. (line 167) +* figure counter: Counters. (line 15) +* figure dash character: Text symbols. (line 139) +* figure environment: figure. (line 6) +* figure number, cross referencing: \ref. (line 6) +* figures, footnotes in: minipage. (line 107) +* figures, inserting: figure. (line 6) +* file, closing: \openin & \openout. (line 6) +* file, opening: \openin & \openout. (line 6) +* file, reading: \read. (line 6) +* file, root: Splitting the input. (line 10) +* file, root <1>: Jobname. (line 6) +* filecontents environment: filecontents. (line 6) +* filecontents* environment: filecontents. (line 6) +* filename for current job: Command line options. + (line 43) +* final option: Document class options. + (line 49) +* first-latex-doc document: About this document. (line 36) +* fixed-width font: Font styles. (line 97) +* flafter package: Floats. (line 82) +* fleqn option: Document class options. + (line 49) +* float package: Floats. (line 52) +* float page: Floats. (line 58) +* flush left equations: Document class options. + (line 57) +* flushing floats and starting a page: \clearpage & \cleardoublepage. + (line 6) +* flushleft environment: flushleft. (line 6) +* flushright environment: flushright. (line 6) +* font catalogue: Low-level font commands. + (line 21) +* font commands, low-level: Low-level font commands. + (line 6) +* font dimension, slant: \/. (line 40) +* font encoding: fontenc package. (line 6) +* font encoding <1>: \DeclareTextAccent. (line 6) +* font encoding, declaring: \DeclareFontEncoding. + (line 6) +* font size: Low-level font commands. + (line 101) +* font sizes: Font sizes. (line 6) +* font styles: Font styles. (line 6) +* font symbols, by number: Symbols by font position. + (line 6) +* fontenc: fontenc package. (line 6) +* fonts: Fonts. (line 6) +* fonts, new commands for: \newfont. (line 6) +* fonts, script: Calligraphic. (line 6) +* fontspec package: fontenc package. (line 17) +* footer style: \pagestyle. (line 6) +* footer, parameters for: Page layout parameters. + (line 6) +* footnote counter: Counters. (line 15) +* footnote number, cross referencing: \ref. (line 6) +* footnote parameters: \footnote. (line 34) +* footnote, in a table: Footnotes in a table. + (line 6) +* footnote, in section headings: Footnotes in section headings. + (line 6) +* footnote, of a footnote: Footnotes of footnotes. + (line 6) +* footnotes in figures: minipage. (line 107) +* footnotes, creating: Footnotes. (line 6) +* footnotes, in a minipage: \footnote. (line 62) +* footnotes, symbols instead of numbers: \footnote. (line 24) +* force option for filecontents: filecontents. (line 31) +* format files, TeX: TeX engines. (line 6) +* formulas, environment for: equation. (line 6) +* formulas, math: Math formulas. (line 6) +* forward reference: Cross references. (line 27) +* forward references, resolving: Output files. (line 33) +* fraction: \frac. (line 6) +* fragile commands: \protect. (line 6) +* frame rule width: \fbox & \framebox. (line 38) +* frame, line width: \fbox & \framebox. (line 38) +* frame, separation from contents: \fbox & \framebox. (line 43) +* French quotation marks: Text symbols. (line 33) +* front matter of a book: \frontmatter & \mainmatter & \backmatter. + (line 6) +* functions, math: Math functions. (line 6) +* geometry package: Document class options. + (line 40) +* geometry package <1>: Document class options. + (line 44) +* global options: Document class options. + (line 6) +* global options <1>: Additional packages. (line 15) +* glossaries: Glossaries. (line 6) +* glossary: Glossaries. (line 6) +* glossary, entries: \newglossaryentry. (line 6) +* glossary, entry reference: \gls. (line 6) +* glue plain TeX: Lengths. (line 9) +* glue register, plain TeX: \newlength. (line 6) +* graphics: Graphics. (line 6) +* graphics <1>: Graphics package configuration. + (line 6) +* graphics <2>: \includegraphics. (line 6) +* graphics package: Graphics. (line 6) +* graphics package <1>: Graphics package configuration. + (line 6) +* graphics package <2>: \includegraphics. (line 6) +* graphics package commands: Commands for graphics. + (line 6) +* graphics package options: Graphics package options. + (line 6) +* graphics packages: \line. (line 35) +* graphics, resizing: \scalebox. (line 6) +* graphics, resizing <1>: \resizebox. (line 6) +* graphics, scaling: \scalebox. (line 6) +* graphics, scaling <1>: \resizebox. (line 6) +* graphpap package: \graphpaper. (line 11) +* grave accent: Accents. (line 44) +* grave accent, math: Math accents. (line 29) +* greater than symbol, in text: Text symbols. (line 146) +* greek letters: Math symbols. (line 6) +* Greek letters: Greek letters. (line 6) +* grfext package: \DeclareGraphicsExtensions. + (line 39) +* group, and environments: Environments. (line 18) +* group_skip: makeindex. (line 58) +* hacek accent: Accents. (line 93) +* Halmos symbol: \rule. (line 13) +* hard space: ~. (line 6) +* hat accent: Accents. (line 40) +* hat accent, math: Math accents. (line 32) +* header style: \pagestyle. (line 6) +* header, parameters for: Page layout parameters. + (line 6) +* hello, world: Starting and ending. (line 6) +* here, putting floats: Floats. (line 52) +* home page for manual: About this document. (line 6) +* horizontal bar character: Text symbols. (line 149) +* horizontal paragraph indentation: \parindent & \parskip. + (line 6) +* horizontal space: \hss. (line 6) +* horizontal space, stretchable: \hss. (line 6) +* horizontal spacing: \phantom & \vphantom & \hphantom. + (line 6) +* hungarian umlaut accent: Accents. (line 69) +* hyperref package: \footnotemark. (line 48) +* hyperref package <1>: \footnotemark. (line 57) +* hyperref package <2>: \pagenumbering. (line 45) +* hyperref package <3>: \contentsline. (line 37) +* hyperref package <4>: Command line input. (line 14) +* hyphen character, non-breaking: Text symbols. (line 160) +* hyphenation, defining: \hyphenation. (line 6) +* hyphenation, discretionary: \discretionary. (line 6) +* hyphenation, forcing: \- (hyphenation). (line 6) +* hyphenation, preventing: \mbox & \makebox. (line 6) +* háček accent, math: Math accents. (line 20) +* Icelandic eth: Additional Latin letters. + (line 20) +* Icelandic thorn: Additional Latin letters. + (line 56) +* idx file: \index. (line 108) +* ij letter, Dutch: Additional Latin letters. + (line 32) +* immediate \write: \write. (line 60) +* implementations of TeX: TeX engines. (line 6) +* importing graphics: \includegraphics. (line 6) +* in: Units of length. (line 16) +* inch: Units of length. (line 17) +* including graphics: \includegraphics. (line 6) +* indent, forcing: \indent & \noindent. (line 6) +* indentation of paragraphs, in minipage: minipage. (line 103) +* indentfirst package: \part. (line 50) +* indentfirst package <1>: \chapter. (line 58) +* indentfirst package <2>: \section. (line 60) +* indentfirst package <3>: \subsection. (line 48) +* indentfirst package <4>: \subsubsection & \paragraph & \subparagraph. + (line 59) +* indentfirst package <5>: \indent & \noindent. (line 48) +* indent_length: makeindex. (line 137) +* indent_space: makeindex. (line 133) +* index entries, subentries: \index. (line 26) +* index entries, 'see' and 'see also': \index. (line 58) +* index entry: \index. (line 6) +* index package: \index. (line 103) +* index, multiple: Indexes. (line 51) +* index, page range: \index. (line 42) +* index, printing: \printindex. (line 6) +* index, processing: makeindex. (line 6) +* index, producing manually: Produce the index manually. + (line 6) +* index, style file: makeindex. (line 18) +* indexes: Indexes. (line 6) +* infinite horizontal stretch: \hfill. (line 6) +* infinite vertical stretch: \vfill. (line 6) +* inline formulas: math. (line 6) +* inner paragraph mode: Modes. (line 51) +* input file: Splitting the input. (line 6) +* input, on command line: Command line input. (line 6) +* input/output: Input/output. (line 6) +* input/output, to terminal: Input/output. (line 6) +* inputenc: inputenc package. (line 6) +* inserting figures: figure. (line 6) +* insertions of special characters: Special insertions. (line 6) +* interface, command line: Command line interface. + (line 6) +* interline space: \baselineskip & \baselinestretch. + (line 6) +* internal vertical mode: Modes. (line 41) +* invisible character: \phantom & \vphantom & \hphantom. + (line 6) +* invisible character <1>: \mathstrut. (line 6) +* italic correction: \/. (line 6) +* italic font: Font styles. (line 82) +* itemize environment: itemize. (line 6) +* item_0: makeindex. (line 82) +* item_01: makeindex. (line 91) +* item_1: makeindex. (line 85) +* item_12: makeindex. (line 100) +* item_2: makeindex. (line 88) +* item_x1: makeindex. (line 95) +* item_x2: makeindex. (line 104) +* jobname: Command line options. + (line 43) +* jobname <1>: Jobname. (line 6) +* JPEG files: \includegraphics. (line 6) +* JPG files: \includegraphics. (line 6) +* justification, ragged left: \raggedleft. (line 6) +* justification, ragged right: \raggedright. (line 6) +* Knuth, Donald E.: Overview. (line 6) +* label: Cross references. (line 6) +* labelled lists, creating: description. (line 6) +* Lamport TeX: Overview. (line 27) +* Lamport, Leslie: Overview. (line 6) +* landscape option: Document class options. + (line 49) +* landscape orientation: Document class options. + (line 60) +* latex: TeX engines. (line 14) +* latex command: Output files. (line 10) +* LaTeX format (.fmt) files: TeX engines. (line 6) +* LaTeX logo: Text symbols. (line 24) +* LaTeX overview: Overview. (line 6) +* LaTeX Project team: About this document. (line 20) +* LaTeX vs. LaTeX2e: About this document. (line 16) +* latex-dev: TeX engines. (line 64) +* latex-doc-ptr document: About this document. (line 33) +* LaTeX2e logo: Text symbols. (line 27) +* LaTeX3 syntax: LaTeX command syntax. + (line 35) +* email address: About this document. (line 20) +* latexsym package: Arrows. (line 6) +* Latin letters, additional: Additional Latin letters. + (line 6) +* layout commands: Layout. (line 6) +* layout, page parameters for: Page layout parameters. + (line 6) +* leaders, dots in table of contents: \@dottedtocline. (line 27) +* leading: \baselineskip & \baselinestretch. + (line 6) +* left angle quotation marks: Text symbols. (line 33) +* left arrow, in text: Text symbols. (line 157) +* left brace, in text: Text symbols. (line 89) +* left quote: Text symbols. (line 43) +* left quote, double: Text symbols. (line 176) +* left quote, single: Text symbols. (line 182) +* left-hand equation numbers: Document class options. + (line 63) +* left-justifying text: \raggedright. (line 6) +* left-justifying text, environment for: flushleft. (line 6) +* left-to-right mode: Modes. (line 18) +* legalpaper option: Document class options. + (line 19) +* lengths, adding to: \addtolength. (line 6) +* lengths, allocating new: \newlength. (line 6) +* lengths, defining and using: Lengths. (line 6) +* lengths, setting: \setlength. (line 6) +* leqno option: Document class options. + (line 49) +* less than symbol, in text: Text symbols. (line 154) +* lethead_flag: makeindex. (line 65) +* lethead_prefix: makeindex. (line 74) +* lethead_suffix: makeindex. (line 78) +* letter class: Document classes. (line 11) +* letter environment: letter. (line 6) +* letterpaper option: Document class options. + (line 19) +* letters, accented: Accents. (line 6) +* letters, additional Latin: Additional Latin letters. + (line 6) +* letters, ending: \closing. (line 6) +* letters, starting: \opening. (line 6) +* letters, writing: Letters. (line 6) +* line break, forcing: \\. (line 6) +* line breaking: Line breaking. (line 6) +* line breaks, changing: \fussy & \sloppy. (line 6) +* line breaks, forcing: \linebreak & \nolinebreak. + (line 6) +* line breaks, multiplication discretionary: \*. (line 6) +* line breaks, preventing: \linebreak & \nolinebreak. + (line 6) +* lines in tables: tabular. (line 6) +* line_max: makeindex. (line 129) +* lining numerals: Font styles. (line 133) +* lining text up in tables: tabular. (line 6) +* lining text up using tab stops: tabbing. (line 6) +* list environment: list. (line 6) +* list items, specifying counter: \usecounter. (line 6) +* list of figures file: Output files. (line 43) +* list of tables file: Output files. (line 43) +* listings package: tabbing. (line 146) +* listings package <1>: verbatim. (line 35) +* listings package <2>: \verb. (line 42) +* lists of items: itemize. (line 6) +* lists of items, generic: list. (line 6) +* lists of items, numbered: enumerate. (line 6) +* lmodern package: fontenc package. (line 36) +* loading additional packages: Additional packages. (line 6) +* locale information, from system: \write18. (line 65) +* log file: Output files. (line 28) +* log file, writing to: \write. (line 30) +* logo, LaTeX: Text symbols. (line 24) +* logo, LaTeX2e: Text symbols. (line 27) +* logo, TeX: Text symbols. (line 65) +* long command: Class and package commands. + (line 23) +* low-9 quotation marks, single and double: Text symbols. (line 55) +* low-level font commands: Low-level font commands. + (line 6) +* lowercase: Upper and lower case. + (line 6) +* LR box: picture. (line 93) +* LR mode: Modes. (line 18) +* lrbox: lrbox. (line 6) +* lshort document: About this document. (line 39) +* lualatex: TeX engines. (line 32) +* lualatex-dev: TeX engines. (line 65) +* LuaTeX: TeX engines. (line 33) +* LuaTeX, 256 output streams in: \write. (line 42) +* m-width: Units of length. (line 46) +* macron accent: Accents. (line 36) +* macron accent, math: Math accents. (line 14) +* macros2e package: \makeatletter & \makeatother. + (line 38) +* Madsen, Lars: eqnarray. (line 6) +* mail merges: \read. (line 31) +* main matter of a book: \frontmatter & \mainmatter & \backmatter. + (line 6) +* make a box: \mbox & \makebox. (line 6) +* makeidx package: \printindex. (line 12) +* makeindex program: makeindex. (line 6) +* makeindex, program: makeindex. (line 6) +* makeindex, style file: makeindex. (line 18) +* making a title page: titlepage. (line 6) +* making paragraphs: Making paragraphs. (line 6) +* marginal notes: Marginal notes. (line 6) +* masculine ordinal symbol: Text symbols. (line 167) +* matching brackets: \left & \right. (line 6) +* matching parentheses: \left & \right. (line 6) +* math accents: Math accents. (line 6) +* math environment: math. (line 6) +* math environment <1>: Math formulas. (line 6) +* math formulas: Math formulas. (line 6) +* math functions: Math functions. (line 6) +* math miscellany: Math miscellany. (line 6) +* math mode: Modes. (line 28) +* math mode, entering: Math formulas. (line 6) +* math mode, spacing: Spacing in math mode. + (line 6) +* math mode, spacing <1>: \phantom & \vphantom & \hphantom. + (line 6) +* math mode, spacing <2>: \mathstrut. (line 6) +* math mode, vertical space: \smash. (line 6) +* math styles: Math styles. (line 6) +* math symbols: Math symbols. (line 6) +* math, arrows: Arrows. (line 6) +* math, bold: Font styles. (line 129) +* mathtools package: Math formulas. (line 61) +* mathtools package <1>: Delimiters. (line 41) +* mathtools package <2>: \left & \right. (line 30) +* mathtools package <3>: Over- and Underlining. + (line 44) +* mathtools package <4>: \smash. (line 60) +* mathtools package <5>: \phantom & \vphantom & \hphantom. + (line 48) +* MetaPost package: \line. (line 35) +* mfirstuc package: Upper and lower case. + (line 45) +* mhchem package: Subscripts & superscripts. + (line 62) +* millimeter: Units of length. (line 24) +* minipage environment: minipage. (line 6) +* minipage, creating a: minipage. (line 6) +* minted package: tabbing. (line 146) +* minted package <1>: verbatim. (line 35) +* minted package <2>: \verb. (line 42) +* mirrors of CTAN: CTAN. (line 25) +* mm: Units of length. (line 23) +* modes: Modes. (line 6) +* monospace font: Font styles. (line 97) +* moving arguments: \protect. (line 19) +* mpfootnote counter: \footnote. (line 62) +* mpfootnote counter <1>: Counters. (line 15) +* mu: Units of length. (line 52) +* mu, math unit: Units of length. (line 53) +* multicolumn text: \twocolumn. (line 6) +* multilingual support: Accents. (line 6) +* multind package: Indexes. (line 51) +* multiple indexes: Indexes. (line 51) +* multiplication, discretionary: \*. (line 6) +* name: \newglossaryentry. (line 43) +* name of document root: Jobname. (line 6) +* NBSP: ~. (line 6) +* nested \include, not allowed: \include & \includeonly. + (line 106) +* new class commands: Class and package construction. + (line 6) +* new command, check: Class and package commands. + (line 21) +* new command, definition: Class and package commands. + (line 96) +* new commands, defining: \newcommand & \renewcommand. + (line 6) +* new commands, defining <1>: \providecommand. (line 6) +* new line, output as input: \obeycr & \restorecr. + (line 6) +* new line, starting: \\. (line 6) +* new line, starting (paragraph mode): \newline. (line 6) +* new page, starting: \newpage. (line 6) +* newline, in \write: \write. (line 94) +* noheader option for filecontents: filecontents. (line 34) +* non-breaking hyphen character: Text symbols. (line 160) +* non-English characters: Additional Latin letters. + (line 6) +* nonstopmode: Command line options. + (line 27) +* nosearch option for filecontents: filecontents. (line 37) +* notes in the margin: Marginal notes. (line 6) +* notitlepage option: Document class options. + (line 49) +* null delimiter: \left & \right. (line 6) +* numbered items, specifying counter: \usecounter. (line 6) +* numerals, old-style: Font styles. (line 133) +* oblique font: Font styles. (line 94) +* oe ligature: Additional Latin letters. + (line 48) +* ogonek: Accents. (line 73) +* old-style numerals: Font styles. (line 133) +* one-column output: \onecolumn. (line 6) +* onecolumn option: Document class options. + (line 78) +* oneside option: Document class options. + (line 78) +* open a file: \openin & \openout. (line 6) +* openany option: Document class options. + (line 78) +* openbib option: Document class options. + (line 49) +* opening quote: Text symbols. (line 43) +* openright option: Document class options. + (line 78) +* OpenType fonts: TeX engines. (line 6) +* operating system information: \write18. (line 65) +* options, class: Class and package commands. + (line 66) +* options, color package: Color package options. + (line 6) +* options, command line: Command line options. + (line 6) +* options, document class: Document class options. + (line 6) +* options, document class <1>: Class and package structure. + (line 6) +* options, global: Additional packages. (line 15) +* options, graphics package: Graphics package options. + (line 6) +* options, package: Class and package structure. + (line 6) +* options, package <1>: Class and package commands. + (line 66) +* ordinals, feminine and masculine: Text symbols. (line 167) +* oslash: Additional Latin letters. + (line 44) +* OT1: fontenc package. (line 6) +* outer paragraph mode: Modes. (line 51) +* output directory for all external files: Command line options. + (line 48) +* overbar accent: Accents. (line 36) +* overdot accent, math: Math accents. (line 26) +* overlining: Over- and Underlining. + (line 6) +* overview of LaTeX: Overview. (line 6) +* overwrite option for filecontents: filecontents. (line 31) +* package file layout: Class and package structure. + (line 6) +* package options: Class and package structure. + (line 6) +* package options <1>: Class and package commands. + (line 66) +* package, abstract: abstract. (line 27) +* package, adjustbox: Boxes. (line 9) +* package, algorithm2e: tabbing. (line 146) +* package, amscd: Arrows. (line 41) +* package, amsfonts: Math formulas. (line 61) +* package, amsfonts <1>: Arrows. (line 6) +* package, amsmath: array. (line 55) +* package, amsmath <1>: array. (line 63) +* package, amsmath <2>: displaymath. (line 22) +* package, amsmath <3>: equation. (line 20) +* package, amsmath <4>: theorem. (line 28) +* package, amsmath <5>: Math formulas. (line 61) +* package, amsmath <6>: \left & \right. (line 30) +* package, amsmath <7>: \bigl & \bigr etc.. (line 67) +* package, amsmath <8>: Dots. (line 36) +* package, amsmath <9>: Math functions. (line 111) +* package, amsmath <10>: Spacing in math mode. + (line 23) +* package, amsmath <11>: \phantom & \vphantom & \hphantom. + (line 56) +* package, amsmath <12>: Colon character & \colon. + (line 15) +* package, amsmath <13>: Colon character & \colon. + (line 19) +* package, amsmath <14>: \thinspace & \negthinspace. + (line 26) +* package, amsthm: theorem. (line 28) +* package, amsthm <1>: \rule. (line 18) +* package, answers: \write. (line 99) +* package, appendix: \appendix. (line 29) +* package, array: array. (line 74) +* package, Asymptote: \line. (line 35) +* package, Asymptote <1>: \strut. (line 62) +* package, Asymptote <2>: \mbox & \makebox. (line 74) +* package, Asymptote <3>: \write18. (line 13) +* package, babel: \chapter. (line 71) +* package, babel <1>: thebibliography. (line 53) +* package, babel <2>: Accents. (line 6) +* package, babel <3>: \today. (line 13) +* package, babel <4>: Table of contents etc.. + (line 123) +* package, babel <5>: \index. (line 58) +* package, beamer: beamer template. (line 6) +* package, bigfoot: Footnotes of footnotes. + (line 6) +* package, bm: \boldmath & \unboldmath. + (line 32) +* package, caption: \caption. (line 72) +* package, cleveref: Cross references. (line 38) +* package, cleveref <1>: \ref. (line 29) +* package, cleveref <2>: \footnotemark. (line 48) +* package, cm-super: fontenc package. (line 36) +* package, comprehensive: Math symbols. (line 6) +* package, cprotect: verbatim. (line 25) +* package, cprotect <1>: \verb. (line 45) +* package, datatool: \read. (line 31) +* package, datetime: \today. (line 27) +* package, dcolumn: array. (line 74) +* package, enumitem: itemize. (line 66) +* package, enumitem <1>: list. (line 255) +* package, envlab: \makelabels. (line 55) +* package, etoolbox: Class and package commands. + (line 110) +* package, eurosym: Text symbols. (line 128) +* package, eurosym <1>: Text symbols. (line 129) +* package, expl3: LaTeX command syntax. + (line 35) +* package, expl3 <1>: Upper and lower case. + (line 48) +* package, fancyhdr: Page styles. (line 12) +* package, fancyhdr <1>: \pagestyle. (line 13) +* package, fancyvrb: tabbing. (line 146) +* package, fancyvrb <1>: verbatim. (line 41) +* package, flafter: Floats. (line 82) +* package, float: Floats. (line 52) +* package, fontspec: fontenc package. (line 17) +* package, geometry: Document class options. + (line 40) +* package, geometry <1>: Document class options. + (line 44) +* package, graphpap: \graphpaper. (line 11) +* package, grfext: \DeclareGraphicsExtensions. + (line 39) +* package, hyperref: \footnotemark. (line 48) +* package, hyperref <1>: \footnotemark. (line 57) +* package, hyperref <2>: \pagenumbering. (line 45) +* package, hyperref <3>: \contentsline. (line 37) +* package, hyperref <4>: Command line input. (line 14) +* package, indentfirst: \part. (line 50) +* package, indentfirst <1>: \chapter. (line 58) +* package, indentfirst <2>: \section. (line 60) +* package, indentfirst <3>: \subsection. (line 48) +* package, indentfirst <4>: \subsubsection & \paragraph & \subparagraph. + (line 59) +* package, indentfirst <5>: \indent & \noindent. (line 48) +* package, index: \index. (line 103) +* package, latexsym: Arrows. (line 6) +* package, listings: tabbing. (line 146) +* package, listings <1>: verbatim. (line 35) +* package, listings <2>: \verb. (line 42) +* package, lmodern: fontenc package. (line 36) +* package, macros2e: \makeatletter & \makeatother. + (line 38) +* package, makeidx: \printindex. (line 12) +* package, mathtools: Math formulas. (line 61) +* package, mathtools <1>: Delimiters. (line 41) +* package, mathtools <2>: \left & \right. (line 30) +* package, mathtools <3>: Over- and Underlining. + (line 44) +* package, mathtools <4>: \smash. (line 60) +* package, mathtools <5>: \phantom & \vphantom & \hphantom. + (line 48) +* package, MetaPost: \line. (line 35) +* package, mfirstuc: Upper and lower case. + (line 45) +* package, mhchem: Subscripts & superscripts. + (line 62) +* package, minted: tabbing. (line 146) +* package, minted <1>: verbatim. (line 35) +* package, minted <2>: \verb. (line 42) +* package, multind: Indexes. (line 51) +* package, pict2e: \line. (line 35) +* package, polyglossia: Accents. (line 6) +* package, polyglossia <1>: \today. (line 13) +* package, polyglossia <2>: Table of contents etc.. + (line 124) +* package, polyglossia <3>: \index. (line 58) +* package, PSTricks: \line. (line 35) +* package, setspace: \baselineskip & \baselinestretch. + (line 79) +* package, shellesc: \write18. (line 71) +* package, showidx: Indexes. (line 51) +* package, siunitx: minipage. (line 137) +* package, siunitx <1>: ~. (line 42) +* package, suffix: \@ifstar. (line 64) +* package, tablefootnote: Footnotes in a table. + (line 53) +* package, texosquery: \write18. (line 65) +* package, textcase: Upper and lower case. + (line 42) +* package, TikZ: \line. (line 35) +* package, TikZ <1>: \strut. (line 62) +* package, TikZ <2>: \mbox & \makebox. (line 73) +* package, tikz-cd: Arrows. (line 41) +* package, titlesec: Sectioning. (line 60) +* package, titlesec <1>: \part. (line 54) +* package, titlesec <2>: \chapter. (line 79) +* package, titlesec <3>: \section. (line 64) +* package, titlesec <4>: \subsection. (line 52) +* package, titlesec <5>: \subsubsection & \paragraph & \subparagraph. + (line 63) +* package, tocbibbind: Table of contents etc.. + (line 110) +* package, tocloft: Table of contents etc.. + (line 110) +* package, tocloft <1>: \contentsline. (line 34) +* package, ulem: Over- and Underlining. + (line 15) +* package, unicode-math: Greek letters. (line 37) +* package, url: \verb. (line 39) +* package, verbatimbox: verbatim. (line 41) +* package, xparse: LaTeX command syntax. + (line 35) +* package, xparse <1>: \@ifstar. (line 64) +* package, xr: xr package. (line 6) +* package, xr-hyper: xr package. (line 6) +* package, xspace: xspace package. (line 6) +* package, xstring: Jobname. (line 53) +* packages, loading additional: Additional packages. (line 6) +* page break, forcing: \pagebreak & \nopagebreak. + (line 6) +* page break, preventing: \pagebreak & \nopagebreak. + (line 6) +* page breaking: Page breaking. (line 6) +* page counter: Counters. (line 15) +* page layout parameters: Page layout parameters. + (line 6) +* page number, cross referencing: \pageref. (line 6) +* page numbering style: \pagenumbering. (line 6) +* page style, this page: \thispagestyle. (line 6) +* page styles: Page styles. (line 6) +* page, colored: Colored pages. (line 6) +* page_precedence: makeindex. (line 142) +* paired delimiters: \left & \right. (line 6) +* paragraph: Sectioning. (line 6) +* paragraph <1>: \subsubsection & \paragraph & \subparagraph. + (line 6) +* paragraph counter: Counters. (line 15) +* paragraph indentation: \parindent & \parskip. + (line 6) +* paragraph indentation, in minipage: minipage. (line 103) +* paragraph indentations in quoted text: quotation & quote. (line 6) +* paragraph indentations in quoted text, omitting: quotation & quote. + (line 6) +* paragraph mode: Modes. (line 12) +* paragraph mode <1>: \parbox. (line 6) +* paragraph symbol: Text symbols. (line 47) +* paragraph, ending: \par. (line 6) +* paragraph, in a box: \parbox. (line 6) +* paragraphs: Making paragraphs. (line 6) +* parameters, for footnotes: \footnote. (line 34) +* parameters, page layout: Page layout parameters. + (line 6) +* parent directories, cannot write to: \write and security. (line 22) +* parentheses: Delimiters. (line 6) +* part: Sectioning. (line 6) +* part <1>: \part. (line 6) +* part counter: Counters. (line 15) +* pattern, current tab stops, tabbing: tabbing. (line 51) +* pc: Units of length. (line 13) +* PDF graphic files: \includegraphics. (line 6) +* pdflatex: TeX engines. (line 15) +* pdflatex command: Output files. (line 20) +* pdflatex-dev: TeX engines. (line 66) +* pdfTeX: Output files. (line 20) +* pdfTeX engine: TeX engines. (line 16) +* period, abbreviation-ending: \@. (line 6) +* period, centered, in text: Text symbols. (line 170) +* period, sentence-ending: \@. (line 6) +* period, spacing after: \@. (line 6) +* pica: Units of length. (line 14) +* pict2e package: \line. (line 35) +* picture environment: picture. (line 6) +* pictures, creating: picture. (line 6) +* pilcrow: Text symbols. (line 47) +* placement of floats: Floats. (line 31) +* platex: TeX engines. (line 54) +* platex-dev: TeX engines. (line 67) +* plural: \newglossaryentry. (line 51) +* PNG files: \includegraphics. (line 6) +* poetry, an environment for: verse. (line 6) +* point: Units of length. (line 10) +* polish l: Additional Latin letters. + (line 36) +* polyglossia package: Accents. (line 6) +* polyglossia package <1>: \today. (line 13) +* polyglossia package <2>: Table of contents etc.. + (line 124) +* polyglossia package <3>: \index. (line 58) +* portrait orientation: Document class options. + (line 60) +* position, in picture: picture. (line 64) +* positional parameter: \newcommand & \renewcommand. + (line 54) +* postamble: makeindex. (line 54) +* PostScript point: Units of length. (line 20) +* postscript, in letters: \ps. (line 6) +* pounds symbol: Text symbols. (line 51) +* preamble: makeindex. (line 50) +* preamble, defined: Starting and ending. (line 22) +* prerelease testing: TeX engines. (line 70) +* prompt, *: Recovering from errors. + (line 23) +* pronunciation: Overview. (line 27) +* PSTricks package: \line. (line 35) +* pt: Units of length. (line 9) +* quad: Spacing in math mode. + (line 69) +* question mark, upside-down: Text symbols. (line 173) +* quotation environment: quotation & quote. (line 6) +* quotation marks, French: Text symbols. (line 33) +* quote environment: quotation & quote. (line 6) +* quote, single straight: Text symbols. (line 188) +* quote, straight base: Text symbols. (line 192) +* quoted text with paragraph indentation, displaying: quotation & quote. + (line 6) +* quoted text without paragraph indentation, displaying: quotation & quote. + (line 6) +* radical: \sqrt. (line 6) +* ragged left text: \raggedleft. (line 6) +* ragged left text, environment for: flushright. (line 6) +* ragged right text: \raggedright. (line 6) +* ragged right text, environment for: flushleft. (line 6) +* read a file: \read. (line 6) +* redefining environments: \newenvironment & \renewenvironment. + (line 6) +* reference, forward: Cross references. (line 27) +* references, resolving forward: Output files. (line 33) +* registered symbol: Text symbols. (line 195) +* relation, text above: \stackrel. (line 6) +* release candidates: TeX engines. (line 70) +* remarks in the margin: Marginal notes. (line 6) +* report class: Document classes. (line 11) +* reporting bugs: About this document. (line 24) +* reserved characters: Reserved characters. (line 6) +* resizing: \scalebox. (line 6) +* resizing <1>: \resizebox. (line 6) +* restricted shell access: \write18. (line 53) +* right angle quotation marks: Text symbols. (line 33) +* right arrow, in text: Text symbols. (line 198) +* right brace, in text: Text symbols. (line 92) +* right quote: Text symbols. (line 58) +* right quote, double: Text symbols. (line 179) +* right quote, single: Text symbols. (line 185) +* right-hand equation numbers: Document class options. + (line 63) +* right-justifying text: \raggedleft. (line 6) +* right-justifying text, environment for: flushright. (line 6) +* rigid lengths: Lengths. (line 9) +* ring accent: Accents. (line 77) +* ring accent, math: Math accents. (line 35) +* robust commands: \protect. (line 6) +* roman font: Font styles. (line 85) +* root file: Splitting the input. (line 10) +* root file <1>: Jobname. (line 6) +* roots: \sqrt. (line 6) +* rotating graphics: \rotatebox. (line 6) +* rotating text: \rotatebox. (line 6) +* rotation: \rotatebox. (line 6) +* row, tabbing: tabbing. (line 46) +* rubber lengths: Lengths. (line 9) +* rubber lengths, defining new: \newlength. (line 6) +* running header and footer: Page layout parameters. + (line 6) +* running header and footer style: \pagestyle. (line 6) +* samepage environment: \pagebreak & \nopagebreak. + (line 44) +* sans serif font: Font styles. (line 91) +* scaled point: Units of length. (line 36) +* scaling: \scalebox. (line 6) +* scaling <1>: \resizebox. (line 6) +* script fonts: Calligraphic. (line 6) +* script letters for math: Font styles. (line 79) +* script style: Math styles. (line 23) +* scriptscript style: Math styles. (line 26) +* scrollmode: Command line options. + (line 27) +* secnumdepth: Sectioning. (line 69) +* secnumdepth counter: Sectioning. (line 70) +* section: Sectioning. (line 6) +* section <1>: \section. (line 6) +* section counter: Counters. (line 15) +* section number, cross referencing: \ref. (line 6) +* section numbers, printing: Sectioning. (line 70) +* section symbol: Text symbols. (line 62) +* section, redefining: \@startsection. (line 6) +* sectioning commands: Sectioning. (line 6) +* sectioning, part: \part. (line 6) +* security and \write: \write and security. (line 6) +* see and see also index entries: \index. (line 58) +* self-contained sources: filecontents. (line 42) +* series, of fonts: Low-level font commands. + (line 48) +* setspace package: \baselineskip & \baselinestretch. + (line 79) +* setting counters: \setcounter. (line 6) +* sh, used by \write18: \write18. (line 60) +* shapes, of fonts: Low-level font commands. + (line 88) +* sharp S letters: Additional Latin letters. + (line 52) +* shell access: \write18. (line 6) +* shell escape: Command line options. + (line 61) +* SHELL, environment variables: \write18. (line 60) +* shellesc package: \write18. (line 71) +* showidx package: Indexes. (line 51) +* simulating typed text: verbatim. (line 6) +* single angle quotation marks: Text symbols. (line 33) +* single guillemets: Text symbols. (line 33) +* single left quote: Text symbols. (line 182) +* single low-9 quotation mark: Text symbols. (line 55) +* single quote, straight: Text symbols. (line 188) +* single right quote: Text symbols. (line 185) +* siunitx package: minipage. (line 137) +* siunitx package <1>: ~. (line 42) +* sizes of text: Font sizes. (line 6) +* skip plain TeX: Lengths. (line 9) +* skip register, plain TeX: \newlength. (line 6) +* slanted font: Font styles. (line 94) +* slides class: Document classes. (line 11) +* sloppypar environment: sloppypar. (line 6) +* sloppypar environment <1>: sloppypar. (line 6) +* slot, font: \DeclareTextAccent. (line 13) +* small caps font: Font styles. (line 88) +* sort: \newglossaryentry. (line 55) +* source files, making self-contained: filecontents. (line 42) +* sp: Units of length. (line 35) +* space between lines: \baselineskip & \baselinestretch. + (line 6) +* space, inserting horizontal: \hss. (line 6) +* space, inserting vertical: \addvspace. (line 6) +* space, negative thin: \thinspace & \negthinspace. + (line 6) +* space, thin: \thinspace & \negthinspace. + (line 6) +* space, unbreakable: ~. (line 6) +* space, vertical: \vspace. (line 6) +* spaces: Spaces. (line 6) +* spaces, ignore around commands: \ignorespaces & \ignorespacesafterend. + (line 6) +* spaces, ignore around commands <1>: xspace package. (line 6) +* spacing within math mode: Spacing in math mode. + (line 6) +* spacing, inter-sentence: \frenchspacing & \nonfrenchspacing. + (line 6) +* spacing, inter-sentence <1>: \normalsfcodes. (line 6) +* spacing, math mode: \phantom & \vphantom & \hphantom. + (line 6) +* spacing, math mode <1>: \mathstrut. (line 6) +* Spanish ordinals, feminine and masculine: Text symbols. (line 167) +* special characters: Reserved characters. (line 6) +* special characters <1>: Additional Latin letters. + (line 6) +* special insertions: Special insertions. (line 6) +* specifier, float placement: Floats. (line 31) +* splitting the input file: Splitting the input. (line 6) +* square root: \sqrt. (line 6) +* stack math: \stackrel. (line 6) +* star-variants, commands: \@ifstar. (line 6) +* starred form, defining new commands: \newcommand & \renewcommand. + (line 27) +* starting a new page: \newpage. (line 6) +* starting a new page and clearing floats: \clearpage & \cleardoublepage. + (line 6) +* starting and ending: Starting and ending. (line 6) +* starting on a right-hand page: \clearpage & \cleardoublepage. + (line 6) +* sterling symbol: Text symbols. (line 51) +* straight double quote, base: Text symbols. (line 192) +* straight quote, base: Text symbols. (line 192) +* straight single quote: Text symbols. (line 188) +* stretch, infinite horizontal: \hfill. (line 6) +* stretch, infinite vertical: \vfill. (line 6) +* stretch, omitting vertical: \raggedbottom. (line 6) +* strut: \footnote. (line 47) +* strut <1>: \strut. (line 6) +* strut, math: \mathstrut. (line 6) +* styles of text: Font styles. (line 6) +* styles, page: Page styles. (line 6) +* subparagraph: Sectioning. (line 6) +* subparagraph <1>: \subsubsection & \paragraph & \subparagraph. + (line 6) +* subparagraph counter: Counters. (line 15) +* subscript: Subscripts & superscripts. + (line 6) +* subsection: Sectioning. (line 6) +* subsection <1>: \subsection. (line 6) +* subsection counter: Counters. (line 15) +* subsubsection: Sectioning. (line 6) +* subsubsection <1>: \subsubsection & \paragraph & \subparagraph. + (line 6) +* subsubsection counter: Counters. (line 15) +* suffix package: \@ifstar. (line 64) +* superscript: Subscripts & superscripts. + (line 6) +* symbol: \newglossaryentry. (line 59) +* symbol, defining: \DeclareTextSymbol. (line 6) +* symbols, arrows: Arrows. (line 6) +* symbols, boldface: \boldmath & \unboldmath. + (line 6) +* symbols, math: Math symbols. (line 6) +* symbols, text: Text symbols. (line 6) +* system commands, run from LaTeX: \write18. (line 6) +* system information: \write18. (line 65) +* T1: fontenc package. (line 6) +* tab stops, using: tabbing. (line 6) +* tabbing environment: tabbing. (line 6) +* table counter: Counters. (line 15) +* table environment: table. (line 6) +* table of contents: \contentsline. (line 6) +* table of contents entry, create dotted line: \@dottedtocline. + (line 6) +* table of contents entry, manually adding: \addcontentsline. (line 6) +* table of contents file: Output files. (line 43) +* table of contents, avoiding footnotes: Footnotes in section headings. + (line 6) +* table of contents, creating: Table of contents etc.. + (line 6) +* table of contents, sectioning numbers printed: Sectioning. (line 81) +* tablefootnote package: Footnotes in a table. + (line 53) +* tables, creating: table. (line 6) +* tabular environment: tabular. (line 6) +* template (simple), article: article template. (line 6) +* template, beamer: beamer template. (line 6) +* template, book: book template. (line 6) +* template, book <1>: Larger book template. + (line 6) +* templates, document: Document templates. (line 6) +* terminal input/output: Input/output. (line 6) +* terminal, writing to: \write. (line 30) +* TeX format (.fmt) files: TeX engines. (line 6) +* TeX logo: Text symbols. (line 65) +* texosquery package: \write18. (line 65) +* texput, jobname default: Jobname. (line 27) +* text style: Math styles. (line 21) +* text symbols: Text symbols. (line 6) +* text, resizing: \scalebox. (line 6) +* text, resizing <1>: \resizebox. (line 6) +* text, scaling: \scalebox. (line 6) +* text, scaling <1>: \resizebox. (line 6) +* textcase package: Upper and lower case. + (line 42) +* textcomp package: Text symbols. (line 6) +* thanks, for titlepage: \maketitle. (line 57) +* thebibliography environment: thebibliography. (line 6) +* theindex: Produce the index manually. + (line 6) +* theindex environment: Produce the index manually. + (line 9) +* theorem environment: theorem. (line 6) +* theorem-like environment: \newtheorem. (line 6) +* theorems, defining: \newtheorem. (line 6) +* theorems, typesetting: theorem. (line 6) +* thin space: Spacing in math mode. + (line 47) +* thin space <1>: \thinspace & \negthinspace. + (line 6) +* thin space, negative: Spacing in math mode. + (line 61) +* thin space, negative <1>: \thinspace & \negthinspace. + (line 6) +* thorn, Icelandic letter: Additional Latin letters. + (line 56) +* three-quarters em-dash: Text symbols. (line 201) +* tie: ~. (line 6) +* tie-after accent: Accents. (line 83) +* TikZ package: \line. (line 35) +* TikZ package <1>: \strut. (line 62) +* TikZ package <2>: \mbox & \makebox. (line 73) +* tikz-cd package: Arrows. (line 41) +* tilde accent: Accents. (line 48) +* tilde accent, math: Math accents. (line 38) +* tilde, ASCII, in text: Text symbols. (line 71) +* title page, separate or run-in: Document class options. + (line 71) +* title pages, creating: titlepage. (line 6) +* title, for titlepage: \maketitle. (line 64) +* titlepage environment: titlepage. (line 6) +* titlepage option: Document class options. + (line 49) +* titles, making: \maketitle. (line 6) +* titlesec package: Sectioning. (line 60) +* titlesec package <1>: \part. (line 54) +* titlesec package <2>: \chapter. (line 79) +* titlesec package <3>: \section. (line 64) +* titlesec package <4>: \subsection. (line 52) +* titlesec package <5>: \subsubsection & \paragraph & \subparagraph. + (line 63) +* tocbibbind package: Table of contents etc.. + (line 110) +* tocdepth: Sectioning. (line 80) +* tocdepth counter: Sectioning. (line 81) +* tocloft package: Table of contents etc.. + (line 110) +* tocloft package <1>: \contentsline. (line 34) +* today's date: \today. (line 6) +* tombstone: \rule. (line 13) +* topmargin: Page layout parameters. + (line 125) +* topnumber: Floats. (line 138) +* topnumber <1>: Floats. (line 139) +* totalnumber: Floats. (line 142) +* totalnumber <1>: Floats. (line 143) +* trademark symbol: Text symbols. (line 204) +* transcript file: Output files. (line 28) +* TrueType fonts: TeX engines. (line 6) +* TS1 encoding: Text symbols. (line 6) +* two-column output: \twocolumn. (line 6) +* two-thirds em-dash: Text symbols. (line 207) +* twocolumn option: Document class options. + (line 78) +* twoside option: Document class options. + (line 78) +* type styles: Font styles. (line 6) +* typed text, simulating: verbatim. (line 6) +* typeface sizes: Font sizes. (line 6) +* typefaces: Fonts. (line 6) +* typewriter font: Font styles. (line 97) +* typewriter labels in lists: description. (line 36) +* ulem package: Over- and Underlining. + (line 15) +* umlaut accent: Accents. (line 25) +* unbreakable space: ~. (line 6) +* underbar: Accents. (line 53) +* underlining: Over- and Underlining. + (line 6) +* underscore, in text: Text symbols. (line 210) +* Unicode: inputenc package. (line 15) +* Unicode input, native: TeX engines. (line 6) +* unicode-math package: Greek letters. (line 37) +* units, of length: Units of length. (line 6) +* unofficial nature of this manual: About this document. (line 20) +* unordered lists: itemize. (line 6) +* uplatex: TeX engines. (line 55) +* uplatex-dev: TeX engines. (line 68) +* uppercase: Upper and lower case. + (line 6) +* url package: \verb. (line 39) +* using BibTeX: Using BibTeX. (line 6) +* UTF-8, default LaTeX input encoding: Overview. (line 12) +* UTF-8, engine support for: TeX engines. (line 6) +* UTF-8, font support for: fontenc package. (line 6) +* variables, a list of: Counters. (line 6) +* vector symbol, math: Math accents. (line 41) +* verbatim environment: verbatim. (line 6) +* verbatim text: verbatim. (line 6) +* verbatim text, inline: \verb. (line 6) +* verbatimbox package: verbatim. (line 41) +* verse environment: verse. (line 6) +* vertical bar, double, in text: Text symbols. (line 83) +* vertical bar, in text: Text symbols. (line 80) +* vertical mode: Modes. (line 36) +* vertical space: \vspace. (line 6) +* vertical space <1>: \addvspace. (line 6) +* vertical space before paragraphs: \parindent & \parskip. + (line 6) +* vertical spacing: \phantom & \vphantom & \hphantom. + (line 6) +* vertical spacing <1>: \mathstrut. (line 6) +* vertical spacing, math mode: \smash. (line 6) +* visible space: \verb. (line 31) +* visible space symbol, in text: Text symbols. (line 213) +* weights, of fonts: Low-level font commands. + (line 58) +* whatsit item: \write. (line 65) +* white space: Spaces. (line 6) +* wide hat accent, math: Math accents. (line 44) +* wide tilde accent, math: Math accents. (line 47) +* widths, of fonts: Low-level font commands. + (line 70) +* Wright, Joseph: Upper and lower case. + (line 48) +* writing external files: filecontents. (line 6) +* writing letters: Letters. (line 6) +* x-height: Units of length. (line 42) +* xdvi command: Output files. (line 10) +* xdvipdfmx: TeX engines. (line 42) +* xelatex: TeX engines. (line 41) +* xelatex-dev: TeX engines. (line 69) +* XeTeX: TeX engines. (line 42) +* xindex program: makeindex. (line 159) +* xindy program: makeindex. (line 152) +* xparse package: LaTeX command syntax. + (line 35) +* xparse package <1>: \@ifstar. (line 64) +* xr package: xr package. (line 6) +* xr-hyper package: xr package. (line 6) +* xspace package: xspace package. (line 6) +* xstring package: Jobname. (line 53) + + + +Tag Table: +Node: Top1861 +Node: About this document3668 +Node: Overview5406 +Node: Starting and ending7186 +Ref: Starting & ending7321 +Node: Output files8498 +Ref: output files dvi8777 +Ref: output files pdf9289 +Ref: output files log9614 +Ref: output files aux9811 +Node: TeX engines10780 +Ref: tex engines latex11316 +Ref: tex engines lualatex12151 +Ref: tex engines xelatex12612 +Node: LaTeX command syntax14572 +Node: Environment16499 +Node: CTAN17726 +Node: Document classes19182 +Ref: document classes article19622 +Ref: document classes book19710 +Ref: document classes letter19895 +Ref: document classes report19971 +Ref: document classes slides20123 +Node: Document class options20576 +Node: Additional packages23658 +Node: Class and package construction24289 +Node: Class and package structure25739 +Node: Class and package commands28034 +Node: Fonts45639 +Ref: Typefaces45742 +Node: fontenc package47704 +Node: \DeclareFontEncoding51956 +Node: \DeclareTextAccent53376 +Node: \DeclareTextAccentDefault54295 +Node: \DeclareTextCommand & \ProvideTextCommand55370 +Node: \DeclareTextCommandDefault & \ProvideTextCommandDefault57622 +Node: \DeclareTextComposite58934 +Node: \DeclareTextCompositeCommand60019 +Node: \DeclareTextSymbol60701 +Node: \DeclareTextSymbolDefault61810 +Node: \LastDeclaredEncoding62770 +Node: \UseTextSymbol & \UseTextAccent63394 +Node: Font styles64604 +Node: Font sizes68834 +Node: Low-level font commands71713 +Ref: low level font commands fontencoding71997 +Ref: low level font commands fontfamily72553 +Ref: low level font commands fontseries73316 +Ref: low level font commands fontshape74448 +Ref: low level font commands fontsize74765 +Ref: low level font commands linespread75342 +Ref: low level font commands selectfont75623 +Ref: low level font commands usefont75949 +Node: Layout76192 +Node: \onecolumn76758 +Node: \twocolumn77117 +Ref: twocolumn columnsep77740 +Ref: twocolumn columnseprule77984 +Ref: twocolumn columnwidth78284 +Ref: twocolumn dbltopfraction78867 +Ref: twocolumn dblfloatpagefraction79833 +Ref: twocolumn dblfloatsep80071 +Ref: twocolumn dbltextfloatsep80380 +Ref: twocolumn dbltopnumber80560 +Node: \flushbottom81509 +Node: \raggedbottom82609 +Node: Page layout parameters83145 +Ref: page layout parameters columnsep83374 +Ref: page layout parameters columnseprule83374 +Ref: page layout parameters columnwidth83374 +Ref: page layout parameters headheight83628 +Ref: page layout parameters headsep83795 +Ref: page layout parameters footskip84107 +Ref: page layout parameters linewidth84441 +Ref: page layout parameters marginparpush84794 +Ref: page layout parameters marginsep84794 +Ref: page layout parameters marginparwidth84794 +Ref: page layout parameters oddsidemargin85834 +Ref: page layout parameters evensidemargin85834 +Ref: page layout parameters paperheight86379 +Ref: page layout parameters paperwidth86605 +Ref: page layout parameters textheight86829 +Ref: page layout parameters textwidth87218 +Ref: page layout parameters hsize88156 +Ref: page layout parameters topmargin88353 +Ref: page layout parameters topskip88657 +Node: \baselineskip & \baselinestretch88873 +Node: Floats94043 +Ref: floats bottomfraction98381 +Ref: floats floatpagefraction98508 +Ref: floats textfraction98619 +Ref: floats topfraction98821 +Ref: floats floatsep99073 +Ref: floats intextsep99183 +Ref: floats textfloatsep99398 +Ref: floats bottomnumber99662 +Ref: floats dbltopnumber99770 +Ref: floats topnumber99889 +Ref: floats totalnumber99993 +Node: \caption100610 +Node: Sectioning103336 +Ref: sectioning secnumdepth106445 +Ref: Sectioning/secnumdepth106445 +Ref: sectioning tocdepth107071 +Ref: Sectioning/tocdepth107071 +Node: \part108111 +Node: \chapter110283 +Node: \section114056 +Node: \subsection117323 +Node: \subsubsection & \paragraph & \subparagraph120037 +Node: \appendix122694 +Node: \frontmatter & \mainmatter & \backmatter123987 +Node: \@startsection125396 +Ref: startsection name126885 +Ref: \@startsection/name126885 +Ref: startsection level127339 +Ref: \@startsection/level127339 +Ref: startsection indent128220 +Ref: \@startsection/indent128220 +Ref: startsection beforeskip128483 +Ref: \@startsection/beforeskip128483 +Ref: startsection afterskip130004 +Ref: \@startsection/afterskip130004 +Ref: startsection style131315 +Ref: \@startsection/style131315 +Node: Cross references135016 +Node: \label137165 +Node: \pageref138985 +Node: \ref139775 +Node: xr package140770 +Node: Environments142591 +Node: abstract144593 +Node: array146182 +Node: center149095 +Node: \centering150848 +Node: description152345 +Node: displaymath154562 +Node: document156258 +Node: \AtBeginDocument156688 +Node: \AtEndDocument157312 +Node: enumerate157956 +Ref: enumerate enumi159819 +Ref: enumerate enumii159819 +Ref: enumerate enumiii159819 +Ref: enumerate enumiv159819 +Ref: enumerate labelenumi160217 +Ref: enumerate labelenumii160217 +Ref: enumerate labelenumiii160217 +Ref: enumerate labelenumiv160217 +Node: eqnarray160756 +Node: equation162752 +Node: figure163420 +Node: filecontents165593 +Node: flushleft167502 +Node: \raggedright168529 +Node: flushright169727 +Node: \raggedleft170551 +Node: itemize171422 +Ref: itemize labelitemi172970 +Ref: itemize labelitemii172970 +Ref: itemize labelitemiii172970 +Ref: itemize labelitemiv172970 +Ref: itemize leftmargin173629 +Ref: itemize leftmargini173629 +Ref: itemize leftmarginii173629 +Ref: itemize leftmarginiii173629 +Ref: itemize leftmarginiv173629 +Ref: itemize leftmarginv173629 +Ref: itemize leftmarginvi173629 +Node: letter175111 +Node: list175349 +Ref: list makelabel177796 +Ref: list itemindent179181 +Ref: list itemsep179318 +Ref: list labelsep179971 +Ref: list labelwidth180134 +Ref: list leftmargin181145 +Ref: list listparindent181998 +Ref: list parsep182229 +Ref: list partopsep182697 +Ref: list rightmargin183495 +Ref: list topsep183680 +Ref: list beginparpenalty187256 +Ref: list itempenalty187355 +Ref: list endparpenalty187459 +Node: \item188330 +Node: trivlist189576 +Node: math191104 +Node: minipage191410 +Node: picture197007 +Node: \put203237 +Node: \multiput203802 +Node: \qbezier204541 +Node: \graphpaper206014 +Node: \line206808 +Node: \linethickness208782 +Node: \thinlines209247 +Node: \thicklines209658 +Node: \circle210043 +Node: \oval210585 +Node: \shortstack211600 +Node: \vector213298 +Node: \makebox (picture)214212 +Node: \framebox (picture)215394 +Node: \frame216867 +Node: \dashbox217307 +Node: quotation & quote218442 +Node: tabbing219338 +Node: table225241 +Node: tabular227324 +Ref: tabular arrayrulewidth233661 +Ref: tabular arraystrech233901 +Ref: tabular doublerulesep234122 +Ref: tabular tabcolsep234258 +Node: \multicolumn234779 +Node: \vline238656 +Node: \cline240047 +Node: \hline240757 +Node: thebibliography241439 +Node: \bibitem243922 +Node: \cite246224 +Node: \nocite248092 +Node: Using BibTeX248600 +Node: BibTeX error messages250980 +Node: theorem251885 +Node: titlepage252807 +Node: verbatim254090 +Node: \verb255903 +Node: verse257996 +Node: Line breaking259223 +Node: \\260589 +Node: \obeycr & \restorecr263019 +Node: \newline263830 +Node: \- (hyphenation)264871 +Node: \discretionary266511 +Node: \fussy & \sloppy267402 +Node: sloppypar268217 +Node: \hyphenation269363 +Node: \linebreak & \nolinebreak269983 +Node: Page breaking271163 +Node: \clearpage & \cleardoublepage273193 +Node: \newpage274713 +Node: \enlargethispage276011 +Node: \pagebreak & \nopagebreak276967 +Node: Footnotes278938 +Node: \footnote280084 +Ref: footnote footnoterule281352 +Ref: footnote footnotesep281963 +Node: \footnotemark283029 +Node: \footnotetext285386 +Node: Footnotes in section headings285988 +Node: Footnotes in a table286821 +Node: Footnotes of footnotes289752 +Node: Definitions290450 +Node: \newcommand & \renewcommand291497 +Node: Control sequences296932 +Node: \providecommand298286 +Node: \makeatletter & \makeatother299449 +Node: \@ifstar301551 +Node: \newcounter305036 +Node: \newlength306756 +Node: \newsavebox307679 +Node: \newenvironment & \renewenvironment308658 +Node: \newtheorem313982 +Node: \newfont317554 +Node: \protect318787 +Node: \ignorespaces & \ignorespacesafterend321175 +Node: xspace package323925 +Node: Counters326124 +Node: \alph \Alph \arabic \roman \Roman \fnsymbol327825 +Node: \usecounter330460 +Node: \value331319 +Node: \setcounter332372 +Node: \addtocounter332968 +Node: \refstepcounter333422 +Node: \stepcounter334091 +Node: \day & \month & \year334644 +Node: Lengths335553 +Node: Units of length340131 +Ref: units of length pt340334 +Ref: units of length pc340451 +Ref: units of length in340474 +Ref: units of length bp340500 +Ref: units of length mm340631 +Ref: units of length cm340663 +Ref: units of length dd340692 +Ref: units of length cc340724 +Ref: units of length sp340749 +Ref: Lengths/ex340891 +Ref: units of length ex340891 +Ref: Lengths/em341034 +Ref: units of length em341034 +Ref: units of length mu341375 +Node: \setlength341904 +Node: \addtolength343016 +Node: \settodepth344439 +Node: \settoheight345427 +Node: \settowidth346425 +Node: \stretch347430 +Node: Expressions348632 +Node: Making paragraphs353446 +Node: \par355761 +Node: \indent & \noindent357807 +Node: \parindent & \parskip359342 +Node: Marginal notes360671 +Ref: marginal notes marginparpush362067 +Ref: marginal notes marginparsep362241 +Ref: marginal notes marginparwidth362373 +Node: Math formulas362722 +Node: Subscripts & superscripts365770 +Node: Math symbols367931 +Node: Arrows391702 +Node: \boldmath & \unboldmath392890 +Node: Blackboard bold394409 +Node: Calligraphic395211 +Node: Delimiters395742 +Node: \left & \right397929 +Node: \bigl & \bigr etc.401966 +Node: Dots404736 +Ref: ellipses cdots405167 +Ref: ellipses ddots405312 +Ref: ellipses ldots405393 +Ref: ellipses vdots405810 +Node: Greek letters406994 +Node: Math functions408673 +Node: Math accents410317 +Node: Over- and Underlining411216 +Node: Spacing in math mode413043 +Ref: spacing in math mode thickspace413974 +Ref: spacing in math mode medspace414425 +Ref: Spacing in math mode/\thinspace414881 +Ref: spacing in math mode thinspace414881 +Ref: spacing in math mode negthinspace415362 +Ref: spacing in math mode quad415735 +Ref: spacing in math mode qquad415991 +Node: \smash416329 +Node: \phantom & \vphantom & \hphantom418546 +Node: \mathstrut421120 +Node: Math styles422082 +Node: Math miscellany424816 +Node: Colon character & \colon425313 +Node: \*426006 +Node: \frac426586 +Node: \sqrt426957 +Node: \stackrel427543 +Node: Modes427816 +Ref: modes paragraph mode428266 +Ref: modes lr mode428578 +Ref: modes math mode429184 +Ref: modes vertical mode429519 +Ref: modes internal vertical mode429737 +Ref: modes inner paragraph mode430260 +Ref: modes outer paragraph mode430260 +Node: \ensuremath430674 +Node: Page styles431379 +Node: \maketitle432142 +Node: \pagenumbering435160 +Node: \pagestyle437147 +Node: \thispagestyle440655 +Node: Spaces441609 +Node: \enspace & \quad & \qquad442929 +Node: \hspace443844 +Node: \hfill445701 +Node: \hss446765 +Node: \spacefactor447459 +Node: \@450866 +Ref: \AT450986 +Node: \frenchspacing & \nonfrenchspacing452926 +Ref: \frenchspacing453044 +Node: \normalsfcodes453846 +Node: \(SPACE)454114 +Ref: Leading blanks456054 +Node: ~456367 +Node: \thinspace & \negthinspace459177 +Node: \/460375 +Node: \hrulefill & \dotfill462333 +Node: \bigskip & \medskip & \smallskip463693 +Ref: bigskip464511 +Ref: medskip464715 +Ref: smallskip464924 +Node: \bigbreak & \medbreak & \smallbreak465585 +Node: \strut466571 +Node: \vspace469808 +Node: \vfill471369 +Node: \addvspace472297 +Node: Boxes474343 +Node: \mbox & \makebox475049 +Ref: mbox makebox depth476317 +Ref: mbox makebox height476317 +Ref: mbox makebox width476317 +Ref: mbox makebox totalheight476317 +Node: \fbox & \framebox478412 +Ref: fbox framebox fboxrule479719 +Ref: fbox framebox fboxsep479909 +Node: \parbox481011 +Node: \raisebox483317 +Ref: raisebox depth484290 +Ref: raisebox height484290 +Ref: raisebox width484290 +Ref: raisebox totalheight484290 +Node: \sbox & \savebox484991 +Node: lrbox488077 +Node: \usebox488963 +Node: Color489402 +Node: Color package options490201 +Node: Color models491847 +Ref: color models cmyk492644 +Ref: color models gray493007 +Ref: color models rgb493156 +Ref: color models RGB493493 +Ref: color models named493868 +Node: Commands for color494156 +Node: Define colors494571 +Node: Colored text495296 +Node: Colored boxes497693 +Node: Colored pages499082 +Node: Graphics499775 +Node: Graphics package options501901 +Node: Graphics package configuration504654 +Node: \graphicspath505456 +Node: \DeclareGraphicsExtensions508414 +Node: \DeclareGraphicsRule510232 +Node: Commands for graphics513411 +Node: \includegraphics513916 +Ref: includegraphics width518979 +Ref: includegraphics height519512 +Ref: includegraphics totalheight519918 +Ref: includegraphics keepaspectratio520182 +Ref: includegraphics viewport521868 +Ref: includegraphics trim522263 +Ref: includegraphics clip522719 +Ref: includegraphics page522979 +Ref: includegraphics pagebox523070 +Ref: includegraphics interpolate523935 +Ref: includegraphics quiet524140 +Ref: includegraphics draft524301 +Ref: includegraphics bb525106 +Ref: includegraphics bbllx525504 +Ref: includegraphics bblly525504 +Ref: includegraphics bburx525504 +Ref: includegraphics bbury525504 +Ref: includegraphics natwidth525646 +Ref: includegraphics natheight525646 +Ref: includegraphics hiresbb525832 +Ref: includegraphics type526594 +Ref: includegraphics ext526634 +Ref: includegraphics read526737 +Ref: includegraphics command526854 +Node: \rotatebox527100 +Node: \scalebox529954 +Node: \resizebox531034 +Node: Special insertions532228 +Node: Reserved characters533096 +Node: Upper and lower case534356 +Node: Symbols by font position536742 +Node: Text symbols537421 +Node: Accents542405 +Node: \accent544690 +Node: Additional Latin letters546516 +Ref: Non-English characters546698 +Node: inputenc package547717 +Ref: \inputencoding549933 +Node: \rule550269 +Node: \today551457 +Node: Splitting the input552393 +Node: \endinput554135 +Node: \include & \includeonly555438 +Node: \input559735 +Node: Front/back matter560950 +Node: Table of contents etc.561283 +Node: \@dottedtocline567044 +Node: \addcontentsline568631 +Node: \addtocontents571694 +Node: \contentsline573911 +Node: \nofiles575689 +Node: \numberline576440 +Node: Indexes577604 +Node: Produce the index manually580398 +Node: \index581562 +Node: makeindex586773 +Ref: makeindex preamble588441 +Ref: makeindex postamble588587 +Ref: makeindex group skip588683 +Ref: makeindex letheadflag589023 +Ref: makeindex lethead prefix589484 +Ref: makeindex lethead suffix589636 +Ref: makeindex item 0589784 +Ref: makeindex item 1589864 +Ref: makeindex item 2589939 +Ref: makeindex item 01590017 +Ref: makeindex item x1590122 +Ref: makeindex item 12590327 +Ref: makeindex item x2590435 +Ref: makeindex delim 0590595 +Ref: makeindex delim 1590725 +Ref: makeindex delim 2590855 +Ref: makeindex delim n590981 +Ref: makeindex delim r591116 +Ref: makeindex line max591224 +Ref: makeindex indent space591359 +Ref: makeindex indent length591454 +Ref: makeindex page precedence591639 +Node: \printindex592676 +Node: Glossaries593149 +Node: \newglossaryentry595116 +Node: \gls596583 +Node: Letters597377 +Node: \address600995 +Node: \cc601806 +Node: \closing602248 +Node: \encl602562 +Node: \location602976 +Node: \makelabels603240 +Node: \name605557 +Node: \opening605798 +Node: \ps606079 +Node: \signature606368 +Node: \telephone607596 +Node: Input/output607961 +Node: \openin & \openout608691 +Node: \read611451 +Node: \typein612638 +Node: \typeout613898 +Node: \write614938 +Node: \write and security619665 +Node: \message620609 +Node: \wlog622438 +Node: \write18622939 +Node: Command line interface626430 +Ref: Command line626594 +Node: Command line options628747 +Ref: interaction modes629750 +Ref: output directory630704 +Node: Command line input632403 +Node: Jobname634520 +Node: Recovering from errors637763 +Node: Document templates639191 +Node: beamer template639625 +Node: article template640267 +Node: book template640730 +Node: Larger book template641209 +Node: Index642751 +Ref: Command Index642837 + +End Tag Table + + +Local Variables: +coding: utf-8 +End: diff --git a/assets/info/make.info-1.gz b/assets/info/make.info-1.gz new file mode 100644 index 00000000..b177a721 Binary files /dev/null and b/assets/info/make.info-1.gz differ diff --git a/assets/info/make.info-2.gz b/assets/info/make.info-2.gz new file mode 100644 index 00000000..e4f05146 Binary files /dev/null and b/assets/info/make.info-2.gz differ diff --git a/assets/info/make.info.gz b/assets/info/make.info.gz new file mode 100644 index 00000000..b16d30d4 Binary files /dev/null and b/assets/info/make.info.gz differ diff --git a/assets/info/python38.info b/assets/info/python38.info new file mode 100644 index 00000000..953a8de4 --- /dev/null +++ b/assets/info/python38.info @@ -0,0 +1,352968 @@ +This is python.info, produced by makeinfo version 6.5 from python.texi. + + Python 3.8.9, September 01, 2021 + + unknown + + Copyright © 2001-2021, Python Software Foundation + +INFO-DIR-SECTION Miscellaneous +START-INFO-DIR-ENTRY +* python: (python.info). One line description of project +END-INFO-DIR-ENTRY + + + Generated by Sphinx 2.4.4. + + +File: python.info, Node: Top, Next: What’s New in Python, Up: (dir) + +Python +****** + + Python 3.8.9, September 01, 2021 + + unknown + + Copyright © 2001-2021, Python Software Foundation + +* Menu: + +* What’s New in Python:: +* The Python Tutorial:: +* Python Setup and Usage:: +* The Python Language Reference:: +* The Python Standard Library:: +* Extending and Embedding the Python Interpreter:: +* Python/C API Reference Manual:: +* Distributing Python Modules:: +* Installing Python Modules:: +* Python HOWTOs:: +* Python Frequently Asked Questions:: +* Glossary:: +* About these documents:: +* Dealing with Bugs:: +* Copyright:: +* History and License:: +* Distributing Python Modules (Legacy version): Distributing Python Modules Legacy version. +* Installing Python Modules (Legacy version): Installing Python Modules Legacy version. +* Python Module Index:: +* Index:: + + — The Detailed Node Listing — + +What’s New in Python + +* What’s New In Python 3.8: What’s New In Python 3 8. +* What’s New In Python 3.7: What’s New In Python 3 7. +* What’s New In Python 3.6: What’s New In Python 3 6. +* What’s New In Python 3.5: What’s New In Python 3 5. +* What’s New In Python 3.4: What’s New In Python 3 4. +* What’s New In Python 3.3: What’s New In Python 3 3. +* What’s New In Python 3.2: What’s New In Python 3 2. +* What’s New In Python 3.1: What’s New In Python 3 1. +* What’s New In Python 3.0: What’s New In Python 3 0. +* What’s New in Python 2.7: What’s New in Python 2 7. +* What’s New in Python 2.6: What’s New in Python 2 6. +* What’s New in Python 2.5: What’s New in Python 2 5. +* What’s New in Python 2.4: What’s New in Python 2 4. +* What’s New in Python 2.3: What’s New in Python 2 3. +* What’s New in Python 2.2: What’s New in Python 2 2. +* What’s New in Python 2.1: What’s New in Python 2 1. +* What’s New in Python 2.0: What’s New in Python 2 0. +* Changelog:: + +What’s New In Python 3.8 + +* Summary – Release highlights:: +* New Features:: +* Other Language Changes:: +* New Modules:: +* Improved Modules:: +* Optimizations:: +* Build and C API Changes:: +* Deprecated:: +* API and Feature Removals:: +* Porting to Python 3.8: Porting to Python 3 8. +* Notable changes in Python 3.8.1: Notable changes in Python 3 8 1. +* Notable changes in Python 3.8.2: Notable changes in Python 3 8 2. +* Notable changes in Python 3.8.3: Notable changes in Python 3 8 3. +* Notable changes in Python 3.8.8: Notable changes in Python 3 8 8. +* Notable changes in Python 3.8.9: Notable changes in Python 3 8 9. + +New Features + +* Assignment expressions:: +* Positional-only parameters:: +* Parallel filesystem cache for compiled bytecode files:: +* Debug build uses the same ABI as release build:: +* f-strings support = for self-documenting expressions and debugging:: +* PEP 578; Python Runtime Audit Hooks: PEP 578 Python Runtime Audit Hooks. +* PEP 587; Python Initialization Configuration: PEP 587 Python Initialization Configuration. +* Vectorcall; a fast calling protocol for CPython: Vectorcall a fast calling protocol for CPython. +* Pickle protocol 5 with out-of-band data buffers:: + +Improved Modules + +* ast:: +* asyncio:: +* builtins:: +* collections:: +* cProfile:: +* csv:: +* curses:: +* ctypes:: +* datetime:: +* functools:: +* gc:: +* gettext:: +* gzip:: +* IDLE and idlelib:: +* inspect:: +* io:: +* itertools:: +* json.tool: json tool. +* logging:: +* math:: +* mmap:: +* multiprocessing:: +* os:: +* os.path: os path. +* pathlib:: +* pickle:: +* plistlib:: +* pprint:: +* py_compile:: +* shlex:: +* shutil:: +* socket:: +* ssl:: +* statistics:: +* sys:: +* tarfile:: +* threading:: +* tokenize:: +* tkinter:: +* time:: +* typing:: +* unicodedata:: +* unittest:: +* venv:: +* weakref:: +* xml:: +* xmlrpc:: + +Porting to Python 3.8 + +* Changes in Python behavior:: +* Changes in the Python API:: +* Changes in the C API:: +* CPython bytecode changes:: +* Demos and Tools:: + +What’s New In Python 3.7 + +* Summary – Release Highlights:: +* New Features: New Features<2>. +* Other Language Changes: Other Language Changes<2>. +* New Modules: New Modules<2>. +* Improved Modules: Improved Modules<2>. +* C API Changes:: +* Build Changes:: +* Optimizations: Optimizations<2>. +* Other CPython Implementation Changes:: +* Deprecated Python Behavior:: +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods. +* Deprecated functions and types of the C API:: +* Platform Support Removals:: +* API and Feature Removals: API and Feature Removals<2>. +* Module Removals:: +* Windows-only Changes:: +* Porting to Python 3.7: Porting to Python 3 7. +* Notable changes in Python 3.7.1: Notable changes in Python 3 7 1. +* Notable changes in Python 3.7.2: Notable changes in Python 3 7 2. +* Notable changes in Python 3.7.6: Notable changes in Python 3 7 6. +* Notable changes in Python 3.7.10: Notable changes in Python 3 7 10. + +New Features + +* PEP 563; Postponed Evaluation of Annotations: PEP 563 Postponed Evaluation of Annotations. +* PEP 538; Legacy C Locale Coercion: PEP 538 Legacy C Locale Coercion. +* PEP 540; Forced UTF-8 Runtime Mode: PEP 540 Forced UTF-8 Runtime Mode. +* PEP 553; Built-in breakpoint(): PEP 553 Built-in breakpoint. +* PEP 539; New C API for Thread-Local Storage: PEP 539 New C API for Thread-Local Storage. +* PEP 562; Customization of Access to Module Attributes: PEP 562 Customization of Access to Module Attributes. +* PEP 564; New Time Functions With Nanosecond Resolution: PEP 564 New Time Functions With Nanosecond Resolution. +* PEP 565; Show DeprecationWarning in __main__: PEP 565 Show DeprecationWarning in __main__. +* PEP 560; Core Support for typing module and Generic Types: PEP 560 Core Support for typing module and Generic Types. +* PEP 552; Hash-based .pyc Files: PEP 552 Hash-based pyc Files. +* PEP 545; Python Documentation Translations: PEP 545 Python Documentation Translations. +* Development Runtime Mode; -X dev: Development Runtime Mode -X dev. + +New Modules + +* contextvars:: +* dataclasses:: +* importlib.resources: importlib resources. + +Improved Modules + +* argparse:: +* asyncio: asyncio<2>. +* binascii:: +* calendar:: +* collections: collections<2>. +* compileall:: +* concurrent.futures: concurrent futures. +* contextlib:: +* cProfile: cProfile<2>. +* crypt:: +* datetime: datetime<2>. +* dbm:: +* decimal:: +* dis:: +* distutils:: +* enum:: +* functools: functools<2>. +* gc: gc<2>. +* hmac:: +* http.client: http client. +* http.server: http server. +* idlelib and IDLE:: +* importlib:: +* io: io<2>. +* ipaddress:: +* itertools: itertools<2>. +* locale:: +* logging: logging<2>. +* math: math<2>. +* mimetypes:: +* msilib:: +* multiprocessing: multiprocessing<2>. +* os: os<2>. +* pathlib: pathlib<2>. +* pdb:: +* py_compile: py_compile<2>. +* pydoc:: +* queue:: +* re:: +* signal:: +* socket: socket<2>. +* socketserver:: +* sqlite3:: +* ssl: ssl<2>. +* string:: +* subprocess:: +* sys: sys<2>. +* time: time<2>. +* tkinter: tkinter<2>. +* tracemalloc:: +* types:: +* unicodedata: unicodedata<2>. +* unittest: unittest<2>. +* unittest.mock: unittest mock. +* urllib.parse: urllib parse. +* uu:: +* uuid:: +* warnings:: +* xml.etree: xml etree. +* xmlrpc.server: xmlrpc server. +* zipapp:: +* zipfile:: + +Deprecated Python modules, functions and methods + +* aifc:: +* asyncio: asyncio<3>. +* collections: collections<3>. +* dbm: dbm<2>. +* enum: enum<2>. +* gettext: gettext<2>. +* importlib: importlib<2>. +* locale: locale<2>. +* macpath:: +* threading: threading<2>. +* socket: socket<3>. +* ssl: ssl<3>. +* sunau:: +* sys: sys<3>. +* wave:: + +Porting to Python 3.7 + +* Changes in Python Behavior:: +* Changes in the Python API: Changes in the Python API<2>. +* Changes in the C API: Changes in the C API<2>. +* CPython bytecode changes: CPython bytecode changes<2>. +* Windows-only Changes: Windows-only Changes<2>. +* Other CPython implementation changes:: + +What’s New In Python 3.6 + +* Summary – Release highlights: Summary – Release highlights<2>. +* New Features: New Features<3>. +* Other Language Changes: Other Language Changes<3>. +* New Modules: New Modules<3>. +* Improved Modules: Improved Modules<3>. +* Optimizations: Optimizations<3>. +* Build and C API Changes: Build and C API Changes<2>. +* Other Improvements:: +* Deprecated: Deprecated<2>. +* Removed:: +* Porting to Python 3.6: Porting to Python 3 6. +* Notable changes in Python 3.6.2: Notable changes in Python 3 6 2. +* Notable changes in Python 3.6.4: Notable changes in Python 3 6 4. +* Notable changes in Python 3.6.5: Notable changes in Python 3 6 5. +* Notable changes in Python 3.6.7: Notable changes in Python 3 6 7. +* Notable changes in Python 3.6.10: Notable changes in Python 3 6 10. +* Notable changes in Python 3.6.13: Notable changes in Python 3 6 13. + +New Features + +* PEP 498; Formatted string literals: PEP 498 Formatted string literals. +* PEP 526; Syntax for variable annotations: PEP 526 Syntax for variable annotations. +* PEP 515; Underscores in Numeric Literals: PEP 515 Underscores in Numeric Literals. +* PEP 525; Asynchronous Generators: PEP 525 Asynchronous Generators. +* PEP 530; Asynchronous Comprehensions: PEP 530 Asynchronous Comprehensions. +* PEP 487; Simpler customization of class creation: PEP 487 Simpler customization of class creation. +* PEP 487; Descriptor Protocol Enhancements: PEP 487 Descriptor Protocol Enhancements. +* PEP 519; Adding a file system path protocol: PEP 519 Adding a file system path protocol. +* PEP 495; Local Time Disambiguation: PEP 495 Local Time Disambiguation. +* PEP 529; Change Windows filesystem encoding to UTF-8: PEP 529 Change Windows filesystem encoding to UTF-8. +* PEP 528; Change Windows console encoding to UTF-8: PEP 528 Change Windows console encoding to UTF-8. +* PEP 520; Preserving Class Attribute Definition Order: PEP 520 Preserving Class Attribute Definition Order. +* PEP 468; Preserving Keyword Argument Order: PEP 468 Preserving Keyword Argument Order. +* New dict implementation:: +* PEP 523; Adding a frame evaluation API to CPython: PEP 523 Adding a frame evaluation API to CPython. +* PYTHONMALLOC environment variable:: +* DTrace and SystemTap probing support:: + +New Modules + +* secrets:: + +Improved Modules + +* array:: +* ast: ast<2>. +* asyncio: asyncio<4>. +* binascii: binascii<2>. +* cmath:: +* collections: collections<4>. +* concurrent.futures: concurrent futures<2>. +* contextlib: contextlib<2>. +* datetime: datetime<3>. +* decimal: decimal<2>. +* distutils: distutils<2>. +* email:: +* encodings:: +* enum: enum<3>. +* faulthandler:: +* fileinput:: +* hashlib:: +* http.client: http client<2>. +* idlelib and IDLE: idlelib and IDLE<2>. +* importlib: importlib<3>. +* inspect: inspect<2>. +* json:: +* logging: logging<3>. +* math: math<3>. +* multiprocessing: multiprocessing<3>. +* os: os<3>. +* pathlib: pathlib<3>. +* pdb: pdb<2>. +* pickle: pickle<2>. +* pickletools:: +* pydoc: pydoc<2>. +* random:: +* re: re<2>. +* readline:: +* rlcompleter:: +* shlex: shlex<2>. +* site:: +* sqlite3: sqlite3<2>. +* socket: socket<4>. +* socketserver: socketserver<2>. +* ssl: ssl<4>. +* statistics: statistics<2>. +* struct:: +* subprocess: subprocess<2>. +* sys: sys<4>. +* telnetlib:: +* time: time<3>. +* timeit:: +* tkinter: tkinter<3>. +* traceback:: +* tracemalloc: tracemalloc<2>. +* typing: typing<2>. +* unicodedata: unicodedata<3>. +* unittest.mock: unittest mock<2>. +* urllib.request: urllib request. +* urllib.robotparser: urllib robotparser. +* venv: venv<2>. +* warnings: warnings<2>. +* winreg:: +* winsound:: +* xmlrpc.client: xmlrpc client. +* zipfile: zipfile<2>. +* zlib:: + +Deprecated + +* New Keywords:: +* Deprecated Python behavior:: +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods<2>. +* Deprecated functions and types of the C API: Deprecated functions and types of the C API<2>. +* Deprecated Build Options:: + +Deprecated Python modules, functions and methods + +* asynchat:: +* asyncore:: +* dbm: dbm<3>. +* distutils: distutils<3>. +* grp:: +* importlib: importlib<4>. +* os: os<4>. +* re: re<3>. +* ssl: ssl<5>. +* tkinter: tkinter<4>. +* venv: venv<3>. + +Removed + +* API and Feature Removals: API and Feature Removals<3>. + +Porting to Python 3.6 + +* Changes in ‘python’ Command Behavior:: +* Changes in the Python API: Changes in the Python API<3>. +* Changes in the C API: Changes in the C API<3>. +* CPython bytecode changes: CPython bytecode changes<3>. + +Notable changes in Python 3.6.2 + +* New make regen-all build target:: +* Removal of make touch build target:: + +What’s New In Python 3.5 + +* Summary – Release highlights: Summary – Release highlights<3>. +* New Features: New Features<4>. +* Other Language Changes: Other Language Changes<4>. +* New Modules: New Modules<4>. +* Improved Modules: Improved Modules<4>. +* Other module-level changes:: +* Optimizations: Optimizations<4>. +* Build and C API Changes: Build and C API Changes<3>. +* Deprecated: Deprecated<3>. +* Removed: Removed<2>. +* Porting to Python 3.5: Porting to Python 3 5. +* Notable changes in Python 3.5.4: Notable changes in Python 3 5 4. + +New Features + +* PEP 492 - Coroutines with async and await syntax:: +* PEP 465 - A dedicated infix operator for matrix multiplication:: +* PEP 448 - Additional Unpacking Generalizations:: +* PEP 461 - percent formatting support for bytes and bytearray:: +* PEP 484 - Type Hints:: +* PEP 471 - os.scandir() function – a better and faster directory iterator: PEP 471 - os scandir function – a better and faster directory iterator. +* PEP 475; Retry system calls failing with EINTR: PEP 475 Retry system calls failing with EINTR. +* PEP 479; Change StopIteration handling inside generators: PEP 479 Change StopIteration handling inside generators. +* PEP 485; A function for testing approximate equality: PEP 485 A function for testing approximate equality. +* PEP 486; Make the Python Launcher aware of virtual environments: PEP 486 Make the Python Launcher aware of virtual environments. +* PEP 488; Elimination of PYO files: PEP 488 Elimination of PYO files. +* PEP 489; Multi-phase extension module initialization: PEP 489 Multi-phase extension module initialization. + +New Modules + +* typing: typing<3>. +* zipapp: zipapp<2>. + +Improved Modules + +* argparse: argparse<2>. +* asyncio: asyncio<5>. +* bz2:: +* cgi:: +* cmath: cmath<2>. +* code:: +* collections: collections<5>. +* collections.abc: collections abc. +* compileall: compileall<2>. +* concurrent.futures: concurrent futures<3>. +* configparser:: +* contextlib: contextlib<3>. +* csv: csv<2>. +* curses: curses<2>. +* dbm: dbm<4>. +* difflib:: +* distutils: distutils<4>. +* doctest:: +* email: email<2>. +* enum: enum<4>. +* faulthandler: faulthandler<2>. +* functools: functools<3>. +* glob:: +* gzip: gzip<2>. +* heapq:: +* http:: +* http.client: http client<3>. +* idlelib and IDLE: idlelib and IDLE<3>. +* imaplib:: +* imghdr:: +* importlib: importlib<5>. +* inspect: inspect<3>. +* io: io<3>. +* ipaddress: ipaddress<2>. +* json: json<2>. +* linecache:: +* locale: locale<3>. +* logging: logging<4>. +* lzma:: +* math: math<4>. +* multiprocessing: multiprocessing<4>. +* operator:: +* os: os<5>. +* pathlib: pathlib<4>. +* pickle: pickle<3>. +* poplib:: +* re: re<4>. +* readline: readline<2>. +* selectors:: +* shutil: shutil<2>. +* signal: signal<2>. +* smtpd:: +* smtplib:: +* sndhdr:: +* socket: socket<5>. +* ssl: ssl<6>. +* sqlite3: sqlite3<3>. +* subprocess: subprocess<3>. +* sys: sys<5>. +* sysconfig:: +* tarfile: tarfile<2>. +* threading: threading<3>. +* time: time<4>. +* timeit: timeit<2>. +* tkinter: tkinter<5>. +* traceback: traceback<2>. +* types: types<2>. +* unicodedata: unicodedata<4>. +* unittest: unittest<3>. +* unittest.mock: unittest mock<3>. +* urllib:: +* wsgiref:: +* xmlrpc: xmlrpc<2>. +* xml.sax: xml sax. +* zipfile: zipfile<3>. + +ssl + +* Memory BIO Support:: +* Application-Layer Protocol Negotiation Support:: +* Other Changes:: + +Deprecated + +* New Keywords: New Keywords<2>. +* Deprecated Python Behavior: Deprecated Python Behavior<2>. +* Unsupported Operating Systems:: +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods<3>. + +Removed + +* API and Feature Removals: API and Feature Removals<4>. + +Porting to Python 3.5 + +* Changes in Python behavior: Changes in Python behavior<2>. +* Changes in the Python API: Changes in the Python API<4>. +* Changes in the C API: Changes in the C API<4>. + +Notable changes in Python 3.5.4 + +* New make regen-all build target: New make regen-all build target<2>. +* Removal of make touch build target: Removal of make touch build target<2>. + +What’s New In Python 3.4 + +* Summary – Release Highlights: Summary – Release Highlights<2>. +* New Features: New Features<5>. +* New Modules: New Modules<5>. +* Improved Modules: Improved Modules<5>. +* CPython Implementation Changes:: +* Deprecated: Deprecated<4>. +* Removed: Removed<3>. +* Porting to Python 3.4: Porting to Python 3 4. +* Changed in 3.4.3: Changed in 3 4 3. + +New Features + +* PEP 453; Explicit Bootstrapping of PIP in Python Installations: PEP 453 Explicit Bootstrapping of PIP in Python Installations. +* PEP 446; Newly Created File Descriptors Are Non-Inheritable: PEP 446 Newly Created File Descriptors Are Non-Inheritable. +* Improvements to Codec Handling:: +* PEP 451; A ModuleSpec Type for the Import System: PEP 451 A ModuleSpec Type for the Import System. +* Other Language Changes: Other Language Changes<5>. + +PEP 453: Explicit Bootstrapping of PIP in Python Installations + +* Bootstrapping pip By Default:: +* Documentation Changes:: + +New Modules + +* asyncio: asyncio<6>. +* ensurepip:: +* enum: enum<5>. +* pathlib: pathlib<5>. +* selectors: selectors<2>. +* statistics: statistics<3>. +* tracemalloc: tracemalloc<3>. + +Improved Modules + +* abc:: +* aifc: aifc<2>. +* argparse: argparse<3>. +* audioop:: +* base64:: +* collections: collections<6>. +* colorsys:: +* contextlib: contextlib<4>. +* dbm: dbm<5>. +* dis: dis<2>. +* doctest: doctest<2>. +* email: email<3>. +* filecmp:: +* functools: functools<4>. +* gc: gc<3>. +* glob: glob<2>. +* hashlib: hashlib<2>. +* hmac: hmac<2>. +* html:: +* http: http<2>. +* idlelib and IDLE: idlelib and IDLE<4>. +* importlib: importlib<6>. +* inspect: inspect<4>. +* ipaddress: ipaddress<3>. +* logging: logging<5>. +* marshal:: +* mmap: mmap<2>. +* multiprocessing: multiprocessing<5>. +* operator: operator<2>. +* os: os<6>. +* pdb: pdb<3>. +* pickle: pickle<4>. +* plistlib: plistlib<2>. +* poplib: poplib<2>. +* pprint: pprint<2>. +* pty:: +* pydoc: pydoc<3>. +* re: re<5>. +* resource:: +* select:: +* shelve:: +* shutil: shutil<3>. +* smtpd: smtpd<2>. +* smtplib: smtplib<2>. +* socket: socket<6>. +* sqlite3: sqlite3<4>. +* ssl: ssl<7>. +* stat:: +* struct: struct<2>. +* subprocess: subprocess<4>. +* sunau: sunau<2>. +* sys: sys<6>. +* tarfile: tarfile<3>. +* textwrap:: +* threading: threading<4>. +* traceback: traceback<3>. +* types: types<3>. +* urllib: urllib<2>. +* unittest: unittest<4>. +* venv: venv<4>. +* wave: wave<2>. +* weakref: weakref<2>. +* xml.etree: xml etree<2>. +* zipfile: zipfile<4>. + +CPython Implementation Changes + +* PEP 445; Customization of CPython Memory Allocators: PEP 445 Customization of CPython Memory Allocators. +* PEP 442; Safe Object Finalization: PEP 442 Safe Object Finalization. +* PEP 456; Secure and Interchangeable Hash Algorithm: PEP 456 Secure and Interchangeable Hash Algorithm. +* PEP 436; Argument Clinic: PEP 436 Argument Clinic. +* Other Build and C API Changes:: +* Other Improvements: Other Improvements<2>. +* Significant Optimizations:: + +Deprecated + +* Deprecations in the Python API:: +* Deprecated Features:: + +Removed + +* Operating Systems No Longer Supported:: +* API and Feature Removals: API and Feature Removals<5>. +* Code Cleanups:: + +Porting to Python 3.4 + +* Changes in ‘python’ Command Behavior: Changes in ‘python’ Command Behavior<2>. +* Changes in the Python API: Changes in the Python API<5>. +* Changes in the C API: Changes in the C API<5>. + +Changed in 3.4.3 + +* PEP 476; Enabling certificate verification by default for stdlib http clients: PEP 476 Enabling certificate verification by default for stdlib http clients. + +What’s New In Python 3.3 + +* Summary – Release highlights: Summary – Release highlights<4>. +* PEP 405; Virtual Environments: PEP 405 Virtual Environments. +* PEP 420; Implicit Namespace Packages: PEP 420 Implicit Namespace Packages. +* PEP 3118; New memoryview implementation and buffer protocol documentation: PEP 3118 New memoryview implementation and buffer protocol documentation. +* PEP 393; Flexible String Representation: PEP 393 Flexible String Representation. +* PEP 397; Python Launcher for Windows: PEP 397 Python Launcher for Windows. +* PEP 3151; Reworking the OS and IO exception hierarchy: PEP 3151 Reworking the OS and IO exception hierarchy. +* PEP 380; Syntax for Delegating to a Subgenerator: PEP 380 Syntax for Delegating to a Subgenerator. +* PEP 409; Suppressing exception context: PEP 409 Suppressing exception context. +* PEP 414; Explicit Unicode literals: PEP 414 Explicit Unicode literals. +* PEP 3155; Qualified name for classes and functions: PEP 3155 Qualified name for classes and functions. +* PEP 412; Key-Sharing Dictionary: PEP 412 Key-Sharing Dictionary. +* PEP 362; Function Signature Object: PEP 362 Function Signature Object. +* PEP 421; Adding sys.implementation: PEP 421 Adding sys implementation. +* Using importlib as the Implementation of Import:: +* Other Language Changes: Other Language Changes<6>. +* A Finer-Grained Import Lock:: +* Builtin functions and types:: +* New Modules: New Modules<6>. +* Improved Modules: Improved Modules<6>. +* Optimizations: Optimizations<5>. +* Build and C API Changes: Build and C API Changes<4>. +* Deprecated: Deprecated<5>. +* Porting to Python 3.3: Porting to Python 3 3. + +PEP 3118: New memoryview implementation and buffer protocol documentation + +* Features:: +* API changes:: + +PEP 393: Flexible String Representation + +* Functionality:: +* Performance and resource usage:: + +PEP 421: Adding sys.implementation + +* SimpleNamespace:: + +Using importlib as the Implementation of Import + +* New APIs:: +* Visible Changes:: + +New Modules + +* faulthandler: faulthandler<3>. +* ipaddress: ipaddress<4>. +* lzma: lzma<2>. + +Improved Modules + +* abc: abc<2>. +* array: array<2>. +* base64: base64<2>. +* binascii: binascii<3>. +* bz2: bz2<2>. +* codecs:: +* collections: collections<7>. +* contextlib: contextlib<5>. +* crypt: crypt<2>. +* curses: curses<3>. +* datetime: datetime<4>. +* decimal: decimal<3>. +* email: email<4>. +* ftplib:: +* functools: functools<5>. +* gc: gc<4>. +* hmac: hmac<3>. +* http: http<3>. +* html: html<2>. +* imaplib: imaplib<2>. +* inspect: inspect<5>. +* io: io<4>. +* itertools: itertools<3>. +* logging: logging<6>. +* math: math<5>. +* mmap: mmap<3>. +* multiprocessing: multiprocessing<6>. +* nntplib:: +* os: os<7>. +* pdb: pdb<4>. +* pickle: pickle<5>. +* pydoc: pydoc<4>. +* re: re<6>. +* sched:: +* select: select<2>. +* shlex: shlex<3>. +* shutil: shutil<4>. +* signal: signal<3>. +* smtpd: smtpd<3>. +* smtplib: smtplib<3>. +* socket: socket<7>. +* socketserver: socketserver<3>. +* sqlite3: sqlite3<5>. +* ssl: ssl<8>. +* stat: stat<2>. +* struct: struct<3>. +* subprocess: subprocess<5>. +* sys: sys<7>. +* tarfile: tarfile<4>. +* tempfile:: +* textwrap: textwrap<2>. +* threading: threading<5>. +* time: time<5>. +* types: types<4>. +* unittest: unittest<5>. +* urllib: urllib<3>. +* webbrowser:: +* xml.etree.ElementTree: xml etree ElementTree. +* zlib: zlib<2>. + +decimal + +* Features: Features<2>. +* API changes: API changes<2>. + +email + +* Policy Framework:: +* Provisional Policy with New Header API:: +* Other API Changes:: + +Deprecated + +* Unsupported Operating Systems: Unsupported Operating Systems<2>. +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods<4>. +* Deprecated functions and types of the C API: Deprecated functions and types of the C API<3>. +* Deprecated features:: + +Porting to Python 3.3 + +* Porting Python code:: +* Porting C code:: +* Building C extensions:: +* Command Line Switch Changes:: + +What’s New In Python 3.2 + +* PEP 384; Defining a Stable ABI: PEP 384 Defining a Stable ABI. +* PEP 389; Argparse Command Line Parsing Module: PEP 389 Argparse Command Line Parsing Module. +* PEP 391; Dictionary Based Configuration for Logging: PEP 391 Dictionary Based Configuration for Logging. +* PEP 3148; The concurrent.futures module: PEP 3148 The concurrent futures module. +* PEP 3147; PYC Repository Directories: PEP 3147 PYC Repository Directories. +* PEP 3149; ABI Version Tagged .so Files: PEP 3149 ABI Version Tagged so Files. +* PEP 3333; Python Web Server Gateway Interface v1.0.1: PEP 3333 Python Web Server Gateway Interface v1 0 1. +* Other Language Changes: Other Language Changes<7>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules. +* Multi-threading:: +* Optimizations: Optimizations<6>. +* Unicode:: +* Codecs:: +* Documentation:: +* IDLE:: +* Code Repository:: +* Build and C API Changes: Build and C API Changes<5>. +* Porting to Python 3.2: Porting to Python 3 2. + +New, Improved, and Deprecated Modules + +* email: email<5>. +* elementtree:: +* functools: functools<6>. +* itertools: itertools<4>. +* collections: collections<8>. +* threading: threading<6>. +* datetime and time:: +* math: math<6>. +* abc: abc<3>. +* io: io<5>. +* reprlib:: +* logging: logging<7>. +* csv: csv<3>. +* contextlib: contextlib<6>. +* decimal and fractions:: +* ftp:: +* popen:: +* select: select<3>. +* gzip and zipfile:: +* tarfile: tarfile<5>. +* hashlib: hashlib<3>. +* ast: ast<3>. +* os: os<8>. +* shutil: shutil<5>. +* sqlite3: sqlite3<6>. +* html: html<3>. +* socket: socket<8>. +* ssl: ssl<9>. +* nntp:: +* certificates:: +* imaplib: imaplib<3>. +* http.client: http client<4>. +* unittest: unittest<6>. +* random: random<2>. +* poplib: poplib<3>. +* asyncore: asyncore<2>. +* tempfile: tempfile<2>. +* inspect: inspect<6>. +* pydoc: pydoc<5>. +* dis: dis<3>. +* dbm: dbm<6>. +* ctypes: ctypes<2>. +* site: site<2>. +* sysconfig: sysconfig<2>. +* pdb: pdb<5>. +* configparser: configparser<2>. +* urllib.parse: urllib parse<2>. +* mailbox:: +* turtledemo:: + +What’s New In Python 3.1 + +* PEP 372; Ordered Dictionaries: PEP 372 Ordered Dictionaries. +* PEP 378; Format Specifier for Thousands Separator: PEP 378 Format Specifier for Thousands Separator. +* Other Language Changes: Other Language Changes<8>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules<2>. +* Optimizations: Optimizations<7>. +* IDLE: IDLE<2>. +* Build and C API Changes: Build and C API Changes<6>. +* Porting to Python 3.1: Porting to Python 3 1. + +What’s New In Python 3.0 + +* Common Stumbling Blocks:: +* Overview Of Syntax Changes:: +* Changes Already Present In Python 2.6: Changes Already Present In Python 2 6. +* Library Changes:: +* PEP 3101; A New Approach To String Formatting: PEP 3101 A New Approach To String Formatting. +* Changes To Exceptions:: +* Miscellaneous Other Changes:: +* Build and C API Changes: Build and C API Changes<7>. +* Performance:: +* Porting To Python 3.0: Porting To Python 3 0. + +Common Stumbling Blocks + +* Print Is A Function:: +* Views And Iterators Instead Of Lists:: +* Ordering Comparisons:: +* Integers:: +* Text Vs. Data Instead Of Unicode Vs. 8-bit: Text Vs Data Instead Of Unicode Vs 8-bit. + +Overview Of Syntax Changes + +* New Syntax:: +* Changed Syntax:: +* Removed Syntax:: + +Miscellaneous Other Changes + +* Operators And Special Methods:: +* Builtins:: + +What’s New in Python 2.7 + +* The Future for Python 2.x: The Future for Python 2 x. +* Changes to the Handling of Deprecation Warnings:: +* Python 3.1 Features: Python 3 1 Features. +* PEP 372; Adding an Ordered Dictionary to collections: PEP 372 Adding an Ordered Dictionary to collections. +* PEP 378; Format Specifier for Thousands Separator: PEP 378 Format Specifier for Thousands Separator<2>. +* PEP 389; The argparse Module for Parsing Command Lines: PEP 389 The argparse Module for Parsing Command Lines. +* PEP 391; Dictionary-Based Configuration For Logging: PEP 391 Dictionary-Based Configuration For Logging. +* PEP 3106; Dictionary Views: PEP 3106 Dictionary Views. +* PEP 3137; The memoryview Object: PEP 3137 The memoryview Object. +* Other Language Changes: Other Language Changes<9>. +* New and Improved Modules:: +* Build and C API Changes: Build and C API Changes<8>. +* Other Changes and Fixes:: +* Porting to Python 2.7: Porting to Python 2 7. +* New Features Added to Python 2.7 Maintenance Releases: New Features Added to Python 2 7 Maintenance Releases. +* Acknowledgements:: + +Other Language Changes + +* Interpreter Changes:: +* Optimizations: Optimizations<8>. + +New and Improved Modules + +* New module; importlib: New module importlib. +* New module; sysconfig: New module sysconfig. +* ttk; Themed Widgets for Tk: ttk Themed Widgets for Tk. +* Updated module; unittest: Updated module unittest. +* Updated module; ElementTree 1.3: Updated module ElementTree 1 3. + +Build and C API Changes + +* Capsules:: +* Port-Specific Changes; Windows: Port-Specific Changes Windows. +* Port-Specific Changes; Mac OS X: Port-Specific Changes Mac OS X. +* Port-Specific Changes; FreeBSD: Port-Specific Changes FreeBSD. + +New Features Added to Python 2.7 Maintenance Releases + +* Two new environment variables for debug mode:: +* PEP 434; IDLE Enhancement Exception for All Branches: PEP 434 IDLE Enhancement Exception for All Branches. +* PEP 466; Network Security Enhancements for Python 2.7: PEP 466 Network Security Enhancements for Python 2 7. +* PEP 477; Backport ensurepip (PEP 453) to Python 2.7: PEP 477 Backport ensurepip PEP 453 to Python 2 7. +* PEP 476; Enabling certificate verification by default for stdlib http clients: PEP 476 Enabling certificate verification by default for stdlib http clients<2>. +* PEP 493; HTTPS verification migration tools for Python 2.7: PEP 493 HTTPS verification migration tools for Python 2 7. +* New make regen-all build target: New make regen-all build target<3>. +* Removal of make touch build target: Removal of make touch build target<3>. + +PEP 477: Backport ensurepip (PEP 453) to Python 2.7 + +* Bootstrapping pip By Default: Bootstrapping pip By Default<2>. +* Documentation Changes: Documentation Changes<2>. + +What’s New in Python 2.6 + +* Python 3.0: Python 3 0. +* Changes to the Development Process:: +* PEP 343; The ‘with’ statement: PEP 343 The ‘with’ statement. +* PEP 366; Explicit Relative Imports From a Main Module: PEP 366 Explicit Relative Imports From a Main Module. +* PEP 370; Per-user site-packages Directory: PEP 370 Per-user site-packages Directory. +* PEP 371; The multiprocessing Package: PEP 371 The multiprocessing Package. +* PEP 3101; Advanced String Formatting: PEP 3101 Advanced String Formatting. +* PEP 3105; print As a Function: PEP 3105 print As a Function. +* PEP 3110; Exception-Handling Changes: PEP 3110 Exception-Handling Changes. +* PEP 3112; Byte Literals: PEP 3112 Byte Literals. +* PEP 3116; New I/O Library: PEP 3116 New I/O Library. +* PEP 3118; Revised Buffer Protocol: PEP 3118 Revised Buffer Protocol. +* PEP 3119; Abstract Base Classes: PEP 3119 Abstract Base Classes. +* PEP 3127; Integer Literal Support and Syntax: PEP 3127 Integer Literal Support and Syntax. +* PEP 3129; Class Decorators: PEP 3129 Class Decorators. +* PEP 3141; A Type Hierarchy for Numbers: PEP 3141 A Type Hierarchy for Numbers. +* Other Language Changes: Other Language Changes<10>. +* New and Improved Modules: New and Improved Modules<2>. +* Deprecations and Removals:: +* Build and C API Changes: Build and C API Changes<9>. +* Porting to Python 2.6: Porting to Python 2 6. +* Acknowledgements: Acknowledgements<2>. + +Changes to the Development Process + +* New Issue Tracker; Roundup: New Issue Tracker Roundup. +* New Documentation Format; reStructuredText Using Sphinx: New Documentation Format reStructuredText Using Sphinx. + +PEP 343: The ‘with’ statement + +* Writing Context Managers:: +* The contextlib module:: + +PEP 3141: A Type Hierarchy for Numbers + +* The fractions Module:: + +Other Language Changes + +* Optimizations: Optimizations<9>. +* Interpreter Changes: Interpreter Changes<2>. + +New and Improved Modules + +* The ast module:: +* The future_builtins module:: +* The json module; JavaScript Object Notation: The json module JavaScript Object Notation. +* The plistlib module; A Property-List Parser: The plistlib module A Property-List Parser. +* ctypes Enhancements:: +* Improved SSL Support:: + +Build and C API Changes + +* Port-Specific Changes; Windows: Port-Specific Changes Windows<2>. +* Port-Specific Changes; Mac OS X: Port-Specific Changes Mac OS X<2>. +* Port-Specific Changes; IRIX: Port-Specific Changes IRIX. + +What’s New in Python 2.5 + +* PEP 308; Conditional Expressions: PEP 308 Conditional Expressions. +* PEP 309; Partial Function Application: PEP 309 Partial Function Application. +* PEP 314; Metadata for Python Software Packages v1.1: PEP 314 Metadata for Python Software Packages v1 1. +* PEP 328; Absolute and Relative Imports: PEP 328 Absolute and Relative Imports. +* PEP 338; Executing Modules as Scripts: PEP 338 Executing Modules as Scripts. +* PEP 341; Unified try/except/finally: PEP 341 Unified try/except/finally. +* PEP 342; New Generator Features: PEP 342 New Generator Features. +* PEP 343; The ‘with’ statement: PEP 343 The ‘with’ statement<2>. +* PEP 352; Exceptions as New-Style Classes: PEP 352 Exceptions as New-Style Classes. +* PEP 353; Using ssize_t as the index type: PEP 353 Using ssize_t as the index type. +* PEP 357; The ‘__index__’ method: PEP 357 The ‘__index__’ method. +* Other Language Changes: Other Language Changes<11>. +* New, Improved, and Removed Modules: New Improved and Removed Modules. +* Build and C API Changes: Build and C API Changes<10>. +* Porting to Python 2.5: Porting to Python 2 5. +* Acknowledgements: Acknowledgements<3>. + +PEP 343: The ‘with’ statement + +* Writing Context Managers: Writing Context Managers<2>. +* The contextlib module: The contextlib module<2>. + +Other Language Changes + +* Interactive Interpreter Changes:: +* Optimizations: Optimizations<10>. + +New, Improved, and Removed Modules + +* The ctypes package:: +* The ElementTree package:: +* The hashlib package:: +* The sqlite3 package:: +* The wsgiref package:: + +Build and C API Changes + +* Port-Specific Changes:: + +What’s New in Python 2.4 + +* PEP 218; Built-In Set Objects: PEP 218 Built-In Set Objects. +* PEP 237; Unifying Long Integers and Integers: PEP 237 Unifying Long Integers and Integers. +* PEP 289; Generator Expressions: PEP 289 Generator Expressions. +* PEP 292; Simpler String Substitutions: PEP 292 Simpler String Substitutions. +* PEP 318; Decorators for Functions and Methods: PEP 318 Decorators for Functions and Methods. +* PEP 322; Reverse Iteration: PEP 322 Reverse Iteration. +* PEP 324; New subprocess Module: PEP 324 New subprocess Module. +* PEP 327; Decimal Data Type: PEP 327 Decimal Data Type. +* PEP 328; Multi-line Imports: PEP 328 Multi-line Imports. +* PEP 331; Locale-Independent Float/String Conversions: PEP 331 Locale-Independent Float/String Conversions. +* Other Language Changes: Other Language Changes<12>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules<3>. +* Build and C API Changes: Build and C API Changes<11>. +* Porting to Python 2.4: Porting to Python 2 4. +* Acknowledgements: Acknowledgements<4>. + +PEP 327: Decimal Data Type + +* Why is Decimal needed?:: +* The Decimal type:: +* The Context type:: + +Other Language Changes + +* Optimizations: Optimizations<11>. + +New, Improved, and Deprecated Modules + +* cookielib:: +* doctest: doctest<3>. + +Build and C API Changes + +* Port-Specific Changes: Port-Specific Changes<2>. + +What’s New in Python 2.3 + +* PEP 218; A Standard Set Datatype: PEP 218 A Standard Set Datatype. +* PEP 255; Simple Generators: PEP 255 Simple Generators. +* PEP 263; Source Code Encodings: PEP 263 Source Code Encodings. +* PEP 273; Importing Modules from ZIP Archives: PEP 273 Importing Modules from ZIP Archives. +* PEP 277; Unicode file name support for Windows NT: PEP 277 Unicode file name support for Windows NT. +* PEP 278; Universal Newline Support: PEP 278 Universal Newline Support. +* PEP 279; enumerate(): PEP 279 enumerate. +* PEP 282; The logging Package: PEP 282 The logging Package. +* PEP 285; A Boolean Type: PEP 285 A Boolean Type. +* PEP 293; Codec Error Handling Callbacks: PEP 293 Codec Error Handling Callbacks. +* PEP 301; Package Index and Metadata for Distutils: PEP 301 Package Index and Metadata for Distutils. +* PEP 302; New Import Hooks: PEP 302 New Import Hooks. +* PEP 305; Comma-separated Files: PEP 305 Comma-separated Files. +* PEP 307; Pickle Enhancements: PEP 307 Pickle Enhancements. +* Extended Slices:: +* Other Language Changes: Other Language Changes<13>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules<4>. +* Pymalloc; A Specialized Object Allocator: Pymalloc A Specialized Object Allocator. +* Build and C API Changes: Build and C API Changes<12>. +* Other Changes and Fixes: Other Changes and Fixes<2>. +* Porting to Python 2.3: Porting to Python 2 3. +* Acknowledgements: Acknowledgements<5>. + +Other Language Changes + +* String Changes:: +* Optimizations: Optimizations<12>. + +New, Improved, and Deprecated Modules + +* Date/Time Type:: +* The optparse Module:: + +Build and C API Changes + +* Port-Specific Changes: Port-Specific Changes<3>. + +What’s New in Python 2.2 + +* Introduction:: +* PEPs 252 and 253; Type and Class Changes: PEPs 252 and 253 Type and Class Changes. +* PEP 234; Iterators: PEP 234 Iterators. +* PEP 255; Simple Generators: PEP 255 Simple Generators<2>. +* PEP 237; Unifying Long Integers and Integers: PEP 237 Unifying Long Integers and Integers<2>. +* PEP 238; Changing the Division Operator: PEP 238 Changing the Division Operator. +* Unicode Changes:: +* PEP 227; Nested Scopes: PEP 227 Nested Scopes. +* New and Improved Modules: New and Improved Modules<3>. +* Interpreter Changes and Fixes:: +* Other Changes and Fixes: Other Changes and Fixes<3>. +* Acknowledgements: Acknowledgements<6>. + +PEPs 252 and 253: Type and Class Changes + +* Old and New Classes:: +* Descriptors:: +* Multiple Inheritance; The Diamond Rule: Multiple Inheritance The Diamond Rule. +* Attribute Access:: +* Related Links:: + +What’s New in Python 2.1 + +* Introduction: Introduction<2>. +* PEP 227; Nested Scopes: PEP 227 Nested Scopes<2>. +* PEP 236; __future__ Directives: PEP 236 __future__ Directives. +* PEP 207; Rich Comparisons: PEP 207 Rich Comparisons. +* PEP 230; Warning Framework: PEP 230 Warning Framework. +* PEP 229; New Build System: PEP 229 New Build System. +* PEP 205; Weak References: PEP 205 Weak References. +* PEP 232; Function Attributes: PEP 232 Function Attributes. +* PEP 235; Importing Modules on Case-Insensitive Platforms: PEP 235 Importing Modules on Case-Insensitive Platforms. +* PEP 217; Interactive Display Hook: PEP 217 Interactive Display Hook. +* PEP 208; New Coercion Model: PEP 208 New Coercion Model. +* PEP 241; Metadata in Python Packages: PEP 241 Metadata in Python Packages. +* New and Improved Modules: New and Improved Modules<4>. +* Other Changes and Fixes: Other Changes and Fixes<4>. +* Acknowledgements: Acknowledgements<7>. + +What’s New in Python 2.0 + +* Introduction: Introduction<3>. +* What About Python 1.6?: What About Python 1 6?. +* New Development Process:: +* Unicode: Unicode<2>. +* List Comprehensions:: +* Augmented Assignment:: +* String Methods:: +* Garbage Collection of Cycles:: +* Other Core Changes:: +* Porting to 2.0: Porting to 2 0. +* Extending/Embedding Changes:: +* Distutils; Making Modules Easy to Install: Distutils Making Modules Easy to Install. +* XML Modules:: +* Module changes:: +* New modules:: +* IDLE Improvements:: +* Deleted and Deprecated Modules:: +* Acknowledgements: Acknowledgements<8>. + +Other Core Changes + +* Minor Language Changes:: +* Changes to Built-in Functions:: + +XML Modules + +* SAX2 Support:: +* DOM Support:: +* Relationship to PyXML:: + +The Python Tutorial + +* Whetting Your Appetite:: +* Using the Python Interpreter:: +* An Informal Introduction to Python:: +* More Control Flow Tools:: +* Data Structures:: +* Modules:: +* Input and Output:: +* Errors and Exceptions:: +* Classes:: +* Brief Tour of the Standard Library:: +* Brief Tour of the Standard Library — Part II:: +* Virtual Environments and Packages:: +* What Now?:: +* Interactive Input Editing and History Substitution:: +* Floating Point Arithmetic; Issues and Limitations: Floating Point Arithmetic Issues and Limitations. +* Appendix:: + +Using the Python Interpreter + +* Invoking the Interpreter:: +* The Interpreter and Its Environment:: + +Invoking the Interpreter + +* Argument Passing:: +* Interactive Mode:: + +The Interpreter and Its Environment + +* Source Code Encoding:: + +An Informal Introduction to Python + +* Using Python as a Calculator:: +* First Steps Towards Programming:: + +Using Python as a Calculator + +* Numbers:: +* Strings:: +* Lists:: + +More Control Flow Tools + +* if Statements:: +* for Statements:: +* The range() Function: The range Function. +* break and continue Statements, and else Clauses on Loops: break and continue Statements and else Clauses on Loops. +* pass Statements:: +* Defining Functions:: +* More on Defining Functions:: +* Intermezzo; Coding Style: Intermezzo Coding Style. + +More on Defining Functions + +* Default Argument Values:: +* Keyword Arguments:: +* Special parameters:: +* Arbitrary Argument Lists:: +* Unpacking Argument Lists:: +* Lambda Expressions:: +* Documentation Strings:: +* Function Annotations:: + +Special parameters + +* Positional-or-Keyword Arguments:: +* Positional-Only Parameters:: +* Keyword-Only Arguments:: +* Function Examples:: +* Recap:: + +Data Structures + +* More on Lists:: +* The del statement:: +* Tuples and Sequences:: +* Sets:: +* Dictionaries:: +* Looping Techniques:: +* More on Conditions:: +* Comparing Sequences and Other Types:: + +More on Lists + +* Using Lists as Stacks:: +* Using Lists as Queues:: +* List Comprehensions: List Comprehensions<2>. +* Nested List Comprehensions:: + +Modules + +* More on Modules:: +* Standard Modules:: +* The dir() Function: The dir Function. +* Packages:: + +More on Modules + +* Executing modules as scripts:: +* The Module Search Path:: +* “Compiled” Python files:: + +Packages + +* Importing * From a Package:: +* Intra-package References:: +* Packages in Multiple Directories:: + +Input and Output + +* Fancier Output Formatting:: +* Reading and Writing Files:: + +Fancier Output Formatting + +* Formatted String Literals:: +* The String format() Method: The String format Method. +* Manual String Formatting:: +* Old string formatting:: + +Reading and Writing Files + +* Methods of File Objects:: +* Saving structured data with json:: + +Errors and Exceptions + +* Syntax Errors:: +* Exceptions:: +* Handling Exceptions:: +* Raising Exceptions:: +* User-defined Exceptions:: +* Defining Clean-up Actions:: +* Predefined Clean-up Actions:: + +Classes + +* A Word About Names and Objects:: +* Python Scopes and Namespaces:: +* A First Look at Classes:: +* Random Remarks:: +* Inheritance:: +* Private Variables:: +* Odds and Ends:: +* Iterators:: +* Generators:: +* Generator Expressions:: + +Python Scopes and Namespaces + +* Scopes and Namespaces Example:: + +A First Look at Classes + +* Class Definition Syntax:: +* Class Objects:: +* Instance Objects:: +* Method Objects:: +* Class and Instance Variables:: + +Inheritance + +* Multiple Inheritance:: + +Brief Tour of the Standard Library + +* Operating System Interface:: +* File Wildcards:: +* Command Line Arguments:: +* Error Output Redirection and Program Termination:: +* String Pattern Matching:: +* Mathematics:: +* Internet Access:: +* Dates and Times:: +* Data Compression:: +* Performance Measurement:: +* Quality Control:: +* Batteries Included:: + +Brief Tour of the Standard Library — Part II + +* Output Formatting:: +* Templating:: +* Working with Binary Data Record Layouts:: +* Multi-threading: Multi-threading<2>. +* Logging:: +* Weak References:: +* Tools for Working with Lists:: +* Decimal Floating Point Arithmetic:: + +Virtual Environments and Packages + +* Introduction: Introduction<4>. +* Creating Virtual Environments:: +* Managing Packages with pip:: + +Interactive Input Editing and History Substitution + +* Tab Completion and History Editing:: +* Alternatives to the Interactive Interpreter:: + +Floating Point Arithmetic: Issues and Limitations + +* Representation Error:: + +Appendix + +* Interactive Mode: Interactive Mode<2>. + +Interactive Mode + +* Error Handling:: +* Executable Python Scripts:: +* The Interactive Startup File:: +* The Customization Modules:: + +Python Setup and Usage + +* Command line and environment:: +* Using Python on Unix platforms:: +* Using Python on Windows:: +* Using Python on a Macintosh:: +* Editors and IDEs:: + +Command line and environment + +* Command line:: +* Environment variables:: + +Command line + +* Interface options:: +* Generic options:: +* Miscellaneous options:: +* Options you shouldn’t use:: + +Environment variables + +* Debug-mode variables:: + +Using Python on Unix platforms + +* Getting and installing the latest version of Python:: +* Building Python:: +* Python-related paths and files:: +* Miscellaneous:: + +Getting and installing the latest version of Python + +* On Linux:: +* On FreeBSD and OpenBSD:: +* On OpenSolaris:: + +Using Python on Windows + +* The full installer:: +* The Microsoft Store package:: +* The nuget.org packages: The nuget org packages. +* The embeddable package:: +* Alternative bundles:: +* Configuring Python:: +* UTF-8 mode:: +* Python Launcher for Windows:: +* Finding modules:: +* Additional modules:: +* Compiling Python on Windows:: +* Other Platforms:: + +The full installer + +* Installation steps:: +* Removing the MAX_PATH Limitation:: +* Installing Without UI:: +* Installing Without Downloading:: +* Modifying an install:: + +The Microsoft Store package + +* Known Issues:: + +The embeddable package + +* Python Application:: +* Embedding Python:: + +Configuring Python + +* Excursus; Setting environment variables: Excursus Setting environment variables. +* Finding the Python executable:: + +Python Launcher for Windows + +* Getting started:: +* Shebang Lines:: +* Arguments in shebang lines:: +* Customization:: +* Diagnostics:: + +Getting started + +* From the command-line:: +* Virtual environments:: +* From a script:: +* From file associations:: + +Customization + +* Customization via INI files:: +* Customizing default Python versions:: + +Additional modules + +* PyWin32:: +* cx_Freeze:: +* WConio:: + +Using Python on a Macintosh + +* Getting and Installing MacPython:: +* The IDE:: +* Installing Additional Python Packages:: +* GUI Programming on the Mac:: +* Distributing Python Applications on the Mac:: +* Other Resources:: + +Getting and Installing MacPython + +* How to run a Python script:: +* Running scripts with a GUI:: +* Configuration:: + +The Python Language Reference + +* Introduction: Introduction<5>. +* Lexical analysis:: +* Data model:: +* Execution model:: +* The import system:: +* Expressions:: +* Simple statements:: +* Compound statements:: +* Top-level components:: +* Full Grammar specification:: + +Introduction + +* Alternate Implementations:: +* Notation:: + +Lexical analysis + +* Line structure:: +* Other tokens:: +* Identifiers and keywords:: +* Literals:: +* Operators:: +* Delimiters:: + +Line structure + +* Logical lines:: +* Physical lines:: +* Comments:: +* Encoding declarations:: +* Explicit line joining:: +* Implicit line joining:: +* Blank lines:: +* Indentation:: +* Whitespace between tokens:: + +Identifiers and keywords + +* Keywords:: +* Reserved classes of identifiers:: + +Literals + +* String and Bytes literals:: +* String literal concatenation:: +* Formatted string literals:: +* Numeric literals:: +* Integer literals:: +* Floating point literals:: +* Imaginary literals:: + +Data model + +* Objects, values and types: Objects values and types. +* The standard type hierarchy:: +* Special method names:: +* Coroutines:: + +Special method names + +* Basic customization:: +* Customizing attribute access:: +* Customizing class creation:: +* Customizing instance and subclass checks:: +* Emulating generic types:: +* Emulating callable objects:: +* Emulating container types:: +* Emulating numeric types:: +* With Statement Context Managers:: +* Special method lookup:: + +Customizing attribute access + +* Customizing module attribute access:: +* Implementing Descriptors:: +* Invoking Descriptors:: +* __slots__:: + +__slots__ + +* Notes on using __slots__:: + +Customizing class creation + +* Metaclasses:: +* Resolving MRO entries:: +* Determining the appropriate metaclass:: +* Preparing the class namespace:: +* Executing the class body:: +* Creating the class object:: +* Uses for metaclasses:: + +Coroutines + +* Awaitable Objects:: +* Coroutine Objects:: +* Asynchronous Iterators:: +* Asynchronous Context Managers:: + +Execution model + +* Structure of a program:: +* Naming and binding:: +* Exceptions: Exceptions<2>. + +Naming and binding + +* Binding of names:: +* Resolution of names:: +* Builtins and restricted execution:: +* Interaction with dynamic features:: + +The import system + +* importlib: importlib<7>. +* Packages: Packages<2>. +* Searching:: +* Loading:: +* The Path Based Finder:: +* Replacing the standard import system:: +* Package Relative Imports:: +* Special considerations for __main__:: +* Open issues:: +* References:: + +Packages + +* Regular packages:: +* Namespace packages:: + +Searching + +* The module cache:: +* Finders and loaders:: +* Import hooks:: +* The meta path:: + +Loading + +* Loaders:: +* Submodules:: +* Module spec:: +* Import-related module attributes:: +* module.__path__: module __path__. +* Module reprs:: +* Cached bytecode invalidation:: + +The Path Based Finder + +* Path entry finders:: +* Path entry finder protocol:: + +Special considerations for __main__ + +* __main__.__spec__: __main__ __spec__. + +Expressions + +* Arithmetic conversions:: +* Atoms:: +* Primaries:: +* Await expression:: +* The power operator:: +* Unary arithmetic and bitwise operations:: +* Binary arithmetic operations:: +* Shifting operations:: +* Binary bitwise operations:: +* Comparisons:: +* Boolean operations:: +* Assignment expressions: Assignment expressions<2>. +* Conditional expressions:: +* Lambdas:: +* Expression lists:: +* Evaluation order:: +* Operator precedence:: + +Atoms + +* Identifiers (Names): Identifiers Names. +* Literals: Literals<2>. +* Parenthesized forms:: +* Displays for lists, sets and dictionaries: Displays for lists sets and dictionaries. +* List displays:: +* Set displays:: +* Dictionary displays:: +* Generator expressions:: +* Yield expressions:: + +Yield expressions + +* Generator-iterator methods:: +* Examples:: +* Asynchronous generator functions:: +* Asynchronous generator-iterator methods:: + +Primaries + +* Attribute references:: +* Subscriptions:: +* Slicings:: +* Calls:: + +Comparisons + +* Value comparisons:: +* Membership test operations:: +* Identity comparisons:: + +Simple statements + +* Expression statements:: +* Assignment statements:: +* The assert statement:: +* The pass statement:: +* The del statement: The del statement<2>. +* The return statement:: +* The yield statement:: +* The raise statement:: +* The break statement:: +* The continue statement:: +* The import statement:: +* The global statement:: +* The nonlocal statement:: + +Assignment statements + +* Augmented assignment statements:: +* Annotated assignment statements:: + +The import statement + +* Future statements:: + +Compound statements + +* The if statement:: +* The while statement:: +* The for statement:: +* The try statement:: +* The with statement:: +* Function definitions:: +* Class definitions:: +* Coroutines: Coroutines<2>. + +Coroutines + +* Coroutine function definition:: +* The async for statement:: +* The async with statement:: + +Top-level components + +* Complete Python programs:: +* File input:: +* Interactive input:: +* Expression input:: + +The Python Standard Library + +* Introduction: Introduction<6>. +* Built-in Functions:: +* Built-in Constants:: +* Built-in Types:: +* Built-in Exceptions:: +* Text Processing Services:: +* Binary Data Services:: +* Data Types:: +* Numeric and Mathematical Modules:: +* Functional Programming Modules:: +* File and Directory Access:: +* Data Persistence:: +* Data Compression and Archiving:: +* File Formats:: +* Cryptographic Services:: +* Generic Operating System Services:: +* Concurrent Execution:: +* Networking and Interprocess Communication:: +* Internet Data Handling:: +* Structured Markup Processing Tools:: +* Internet Protocols and Support:: +* Multimedia Services:: +* Internationalization:: +* Program Frameworks:: +* Graphical User Interfaces with Tk:: +* Development Tools:: +* Debugging and Profiling:: +* Software Packaging and Distribution:: +* Python Runtime Services:: +* Custom Python Interpreters:: +* Importing Modules:: +* Python Language Services:: +* Miscellaneous Services:: +* MS Windows Specific Services:: +* Unix Specific Services:: +* Superseded Modules:: +* Undocumented Modules:: + +Introduction + +* Notes on availability:: + +Built-in Constants + +* Constants added by the site module:: + +Built-in Types + +* Truth Value Testing:: +* Boolean Operations — and, or, not: Boolean Operations — and or not. +* Comparisons: Comparisons<2>. +* Numeric Types — int, float, complex: Numeric Types — int float complex. +* Iterator Types:: +* Sequence Types — list, tuple, range: Sequence Types — list tuple range. +* Text Sequence Type — str:: +* Binary Sequence Types — bytes, bytearray, memoryview: Binary Sequence Types — bytes bytearray memoryview. +* Set Types — set, frozenset: Set Types — set frozenset. +* Mapping Types — dict:: +* Context Manager Types:: +* Other Built-in Types:: +* Special Attributes:: + +Numeric Types — int, float, complex + +* Bitwise Operations on Integer Types:: +* Additional Methods on Integer Types:: +* Additional Methods on Float:: +* Hashing of numeric types:: + +Iterator Types + +* Generator Types:: + +Sequence Types — list, tuple, range + +* Common Sequence Operations:: +* Immutable Sequence Types:: +* Mutable Sequence Types:: +* Lists: Lists<2>. +* Tuples:: +* Ranges:: + +Text Sequence Type — str + +* String Methods: String Methods<2>. +* printf-style String Formatting:: + +Binary Sequence Types — bytes, bytearray, memoryview + +* Bytes Objects:: +* Bytearray Objects:: +* Bytes and Bytearray Operations:: +* printf-style Bytes Formatting:: +* Memory Views:: + +Mapping Types — dict + +* Dictionary view objects:: + +Other Built-in Types + +* Modules: Modules<2>. +* Classes and Class Instances:: +* Functions:: +* Methods:: +* Code Objects:: +* Type Objects:: +* The Null Object:: +* The Ellipsis Object:: +* The NotImplemented Object:: +* Boolean Values:: +* Internal Objects:: + +Built-in Exceptions + +* Base classes:: +* Concrete exceptions:: +* Warnings:: +* Exception hierarchy:: + +Concrete exceptions + +* OS exceptions:: + +Text Processing Services + +* string — Common string operations:: +* re — Regular expression operations:: +* difflib — Helpers for computing deltas:: +* textwrap — Text wrapping and filling:: +* unicodedata — Unicode Database:: +* stringprep — Internet String Preparation:: +* readline — GNU readline interface:: +* rlcompleter — Completion function for GNU readline:: + +string — Common string operations + +* String constants:: +* Custom String Formatting:: +* Format String Syntax:: +* Template strings:: +* Helper functions:: + +Format String Syntax + +* Format Specification Mini-Language:: +* Format examples:: + +re — Regular expression operations + +* Regular Expression Syntax:: +* Module Contents:: +* Regular Expression Objects:: +* Match Objects:: +* Regular Expression Examples:: + +Regular Expression Examples + +* Checking for a Pair:: +* Simulating scanf(): Simulating scanf. +* search() vs. match(): search vs match. +* Making a Phonebook:: +* Text Munging:: +* Finding all Adverbs:: +* Finding all Adverbs and their Positions:: +* Raw String Notation:: +* Writing a Tokenizer:: + +difflib — Helpers for computing deltas + +* SequenceMatcher Objects:: +* SequenceMatcher Examples:: +* Differ Objects:: +* Differ Example:: +* A command-line interface to difflib:: + +readline — GNU readline interface + +* Init file:: +* Line buffer:: +* History file:: +* History list:: +* Startup hooks:: +* Completion:: +* Example:: + +rlcompleter — Completion function for GNU readline + +* Completer Objects:: + +Binary Data Services + +* struct — Interpret bytes as packed binary data:: +* codecs — Codec registry and base classes:: + +struct — Interpret bytes as packed binary data + +* Functions and Exceptions:: +* Format Strings:: +* Classes: Classes<2>. + +Format Strings + +* Byte Order, Size, and Alignment: Byte Order Size and Alignment. +* Format Characters:: +* Examples: Examples<2>. + +codecs — Codec registry and base classes + +* Codec Base Classes:: +* Encodings and Unicode:: +* Standard Encodings:: +* Python Specific Encodings:: +* encodings.idna — Internationalized Domain Names in Applications: encodings idna — Internationalized Domain Names in Applications. +* encodings.mbcs — Windows ANSI codepage: encodings mbcs — Windows ANSI codepage. +* encodings.utf_8_sig — UTF-8 codec with BOM signature: encodings utf_8_sig — UTF-8 codec with BOM signature. + +Codec Base Classes + +* Error Handlers:: +* Stateless Encoding and Decoding:: +* Incremental Encoding and Decoding:: +* Stream Encoding and Decoding:: + +Incremental Encoding and Decoding + +* IncrementalEncoder Objects:: +* IncrementalDecoder Objects:: + +Stream Encoding and Decoding + +* StreamWriter Objects:: +* StreamReader Objects:: +* StreamReaderWriter Objects:: +* StreamRecoder Objects:: + +Python Specific Encodings + +* Text Encodings:: +* Binary Transforms:: +* Text Transforms:: + +Data Types + +* datetime — Basic date and time types:: +* calendar — General calendar-related functions:: +* collections — Container datatypes:: +* collections.abc — Abstract Base Classes for Containers: collections abc — Abstract Base Classes for Containers. +* heapq — Heap queue algorithm:: +* bisect — Array bisection algorithm:: +* array — Efficient arrays of numeric values:: +* weakref — Weak references:: +* types — Dynamic type creation and names for built-in types:: +* copy — Shallow and deep copy operations:: +* pprint — Data pretty printer:: +* reprlib — Alternate repr() implementation: reprlib — Alternate repr implementation. +* enum — Support for enumerations:: + +datetime — Basic date and time types + +* Aware and Naive Objects:: +* Constants:: +* Available Types:: +* timedelta Objects:: +* date Objects:: +* datetime Objects:: +* time Objects:: +* tzinfo Objects:: +* timezone Objects:: +* strftime() and strptime() Behavior: strftime and strptime Behavior. + +Available Types + +* Common Properties:: +* Determining if an Object is Aware or Naive:: + +timedelta Objects + +* Examples of usage; timedelta: Examples of usage timedelta. + +date Objects + +* Examples of Usage; date: Examples of Usage date. + +datetime Objects + +* Examples of Usage; datetime: Examples of Usage datetime. + +time Objects + +* Examples of Usage; time: Examples of Usage time. + +strftime() and strptime() Behavior + +* strftime() and strptime() Format Codes: strftime and strptime Format Codes. +* Technical Detail:: + +collections — Container datatypes + +* ChainMap objects:: +* Counter objects:: +* deque objects:: +* defaultdict objects:: +* namedtuple() Factory Function for Tuples with Named Fields: namedtuple Factory Function for Tuples with Named Fields. +* OrderedDict objects:: +* UserDict objects:: +* UserList objects:: +* UserString objects:: + +ChainMap objects + +* ChainMap Examples and Recipes:: + +deque objects + +* deque Recipes:: + +defaultdict objects + +* defaultdict Examples:: + +OrderedDict objects + +* OrderedDict Examples and Recipes:: + +collections.abc — Abstract Base Classes for Containers + +* Collections Abstract Base Classes:: + +heapq — Heap queue algorithm + +* Basic Examples:: +* Priority Queue Implementation Notes:: +* Theory:: + +bisect — Array bisection algorithm + +* Searching Sorted Lists:: +* Other Examples:: + +weakref — Weak references + +* Weak Reference Objects:: +* Example: Example<2>. +* Finalizer Objects:: +* Comparing finalizers with __del__() methods: Comparing finalizers with __del__ methods. + +types — Dynamic type creation and names for built-in types + +* Dynamic Type Creation:: +* Standard Interpreter Types:: +* Additional Utility Classes and Functions:: +* Coroutine Utility Functions:: + +pprint — Data pretty printer + +* PrettyPrinter Objects:: +* Example: Example<3>. + +reprlib — Alternate repr() implementation + +* Repr Objects:: +* Subclassing Repr Objects:: + +enum — Support for enumerations + +* Module Contents: Module Contents<2>. +* Creating an Enum:: +* Programmatic access to enumeration members and their attributes:: +* Duplicating enum members and values:: +* Ensuring unique enumeration values:: +* Using automatic values:: +* Iteration:: +* Comparisons: Comparisons<3>. +* Allowed members and attributes of enumerations:: +* Restricted Enum subclassing:: +* Pickling:: +* Functional API:: +* Derived Enumerations:: +* When to use __new__() vs. __init__(): When to use __new__ vs __init__. +* Interesting examples:: +* How are Enums different?:: + +Derived Enumerations + +* IntEnum:: +* IntFlag:: +* Flag:: +* Others:: + +Interesting examples + +* Omitting values:: +* OrderedEnum:: +* DuplicateFreeEnum:: +* Planet:: +* TimePeriod:: + +Omitting values + +* Using auto:: +* Using object:: +* Using a descriptive string:: +* Using a custom __new__(): Using a custom __new__. + +How are Enums different? + +* Enum Classes:: +* Enum Members (aka instances): Enum Members aka instances. +* Finer Points:: + +Finer Points + +* Supported __dunder__ names:: +* Supported _sunder_ names:: +* Enum member type:: +* Boolean value of Enum classes and members:: +* Enum classes with methods:: +* Combining members of Flag:: + +Numeric and Mathematical Modules + +* numbers — Numeric abstract base classes:: +* math — Mathematical functions:: +* cmath — Mathematical functions for complex numbers:: +* decimal — Decimal fixed point and floating point arithmetic:: +* fractions — Rational numbers:: +* random — Generate pseudo-random numbers:: +* statistics — Mathematical statistics functions:: + +numbers — Numeric abstract base classes + +* The numeric tower:: +* Notes for type implementors:: + +Notes for type implementors + +* Adding More Numeric ABCs:: +* Implementing the arithmetic operations:: + +math — Mathematical functions + +* Number-theoretic and representation functions:: +* Power and logarithmic functions:: +* Trigonometric functions:: +* Angular conversion:: +* Hyperbolic functions:: +* Special functions:: +* Constants: Constants<2>. + +cmath — Mathematical functions for complex numbers + +* Conversions to and from polar coordinates:: +* Power and logarithmic functions: Power and logarithmic functions<2>. +* Trigonometric functions: Trigonometric functions<2>. +* Hyperbolic functions: Hyperbolic functions<2>. +* Classification functions:: +* Constants: Constants<3>. + +decimal — Decimal fixed point and floating point arithmetic + +* Quick-start Tutorial:: +* Decimal objects:: +* Context objects:: +* Constants: Constants<4>. +* Rounding modes:: +* Signals:: +* Floating Point Notes:: +* Working with threads:: +* Recipes:: +* Decimal FAQ:: + +Decimal objects + +* Logical operands:: + +Floating Point Notes + +* Mitigating round-off error with increased precision:: +* Special values:: + +random — Generate pseudo-random numbers + +* Bookkeeping functions:: +* Functions for integers:: +* Functions for sequences:: +* Real-valued distributions:: +* Alternative Generator:: +* Notes on Reproducibility:: +* Examples and Recipes:: + +statistics — Mathematical statistics functions + +* Averages and measures of central location:: +* Measures of spread:: +* Function details:: +* Exceptions: Exceptions<3>. +* NormalDist objects:: + +NormalDist objects + +* NormalDist Examples and Recipes:: + +Functional Programming Modules + +* itertools — Functions creating iterators for efficient looping:: +* functools — Higher-order functions and operations on callable objects:: +* operator — Standard operators as functions:: + +itertools — Functions creating iterators for efficient looping + +* Itertool functions:: +* Itertools Recipes:: + +functools — Higher-order functions and operations on callable objects + +* partial Objects:: + +operator — Standard operators as functions + +* Mapping Operators to Functions:: +* In-place Operators:: + +File and Directory Access + +* pathlib — Object-oriented filesystem paths:: +* os.path — Common pathname manipulations: os path — Common pathname manipulations. +* fileinput — Iterate over lines from multiple input streams:: +* stat — Interpreting stat() results: stat — Interpreting stat results. +* filecmp — File and Directory Comparisons:: +* tempfile — Generate temporary files and directories:: +* glob — Unix style pathname pattern expansion:: +* fnmatch — Unix filename pattern matching:: +* linecache — Random access to text lines:: +* shutil — High-level file operations:: + +pathlib — Object-oriented filesystem paths + +* Basic use:: +* Pure paths:: +* Concrete paths:: +* Correspondence to tools in the os module:: + +Pure paths + +* General properties:: +* Operators: Operators<2>. +* Accessing individual parts:: +* Methods and properties:: + +Concrete paths + +* Methods: Methods<2>. + +filecmp — File and Directory Comparisons + +* The dircmp class:: + +tempfile — Generate temporary files and directories + +* Examples: Examples<3>. +* Deprecated functions and variables:: + +shutil — High-level file operations + +* Directory and files operations:: +* Archiving operations:: +* Querying the size of the output terminal:: + +Directory and files operations + +* Platform-dependent efficient copy operations:: +* copytree example:: +* rmtree example:: + +Archiving operations + +* Archiving example:: +* Archiving example with base_dir:: + +Data Persistence + +* pickle — Python object serialization:: +* copyreg — Register pickle support functions:: +* shelve — Python object persistence:: +* marshal — Internal Python object serialization:: +* dbm — Interfaces to Unix “databases”:: +* sqlite3 — DB-API 2.0 interface for SQLite databases: sqlite3 — DB-API 2 0 interface for SQLite databases. + +pickle — Python object serialization + +* Relationship to other Python modules:: +* Data stream format:: +* Module Interface:: +* What can be pickled and unpickled?:: +* Pickling Class Instances:: +* Custom Reduction for Types, Functions, and Other Objects: Custom Reduction for Types Functions and Other Objects. +* Out-of-band Buffers:: +* Restricting Globals:: +* Performance: Performance<2>. +* Examples: Examples<4>. + +Relationship to other Python modules + +* Comparison with marshal:: +* Comparison with json:: + +Pickling Class Instances + +* Persistence of External Objects:: +* Dispatch Tables:: +* Handling Stateful Objects:: + +Out-of-band Buffers + +* Provider API:: +* Consumer API:: +* Example: Example<4>. + +copyreg — Register pickle support functions + +* Example: Example<5>. + +shelve — Python object persistence + +* Restrictions:: +* Example: Example<6>. + +dbm — Interfaces to Unix “databases” + +* dbm.gnu — GNU’s reinterpretation of dbm: dbm gnu — GNU’s reinterpretation of dbm. +* dbm.ndbm — Interface based on ndbm: dbm ndbm — Interface based on ndbm. +* dbm.dumb — Portable DBM implementation: dbm dumb — Portable DBM implementation. + +sqlite3 — DB-API 2.0 interface for SQLite databases + +* Module functions and constants:: +* Connection Objects:: +* Cursor Objects:: +* Row Objects:: +* Exceptions: Exceptions<4>. +* SQLite and Python types:: +* Controlling Transactions:: +* Using sqlite3 efficiently:: + +SQLite and Python types + +* Introduction: Introduction<7>. +* Using adapters to store additional Python types in SQLite databases:: +* Converting SQLite values to custom Python types:: +* Default adapters and converters:: + +Using adapters to store additional Python types in SQLite databases + +* Letting your object adapt itself:: +* Registering an adapter callable:: + +Using sqlite3 efficiently + +* Using shortcut methods:: +* Accessing columns by name instead of by index:: +* Using the connection as a context manager:: + +Data Compression and Archiving + +* zlib — Compression compatible with gzip:: +* gzip — Support for gzip files:: +* bz2 — Support for bzip2 compression:: +* lzma — Compression using the LZMA algorithm:: +* zipfile — Work with ZIP archives:: +* tarfile — Read and write tar archive files:: + +gzip — Support for gzip files + +* Examples of usage:: +* Command Line Interface:: + +Command Line Interface + +* Command line options:: + +bz2 — Support for bzip2 compression + +* (De)compression of files: De compression of files. +* Incremental (de)compression: Incremental de compression. +* One-shot (de)compression: One-shot de compression. +* Examples of usage: Examples of usage<2>. + +lzma — Compression using the LZMA algorithm + +* Reading and writing compressed files:: +* Compressing and decompressing data in memory:: +* Miscellaneous: Miscellaneous<2>. +* Specifying custom filter chains:: +* Examples: Examples<5>. + +zipfile — Work with ZIP archives + +* ZipFile Objects:: +* Path Objects:: +* PyZipFile Objects:: +* ZipInfo Objects:: +* Command-Line Interface:: +* Decompression pitfalls:: + +Command-Line Interface + +* Command-line options:: + +Decompression pitfalls + +* From file itself:: +* File System limitations:: +* Resources limitations:: +* Interruption:: +* Default behaviors of extraction:: + +tarfile — Read and write tar archive files + +* TarFile Objects:: +* TarInfo Objects:: +* Command-Line Interface: Command-Line Interface<2>. +* Examples: Examples<6>. +* Supported tar formats:: +* Unicode issues:: + +Command-Line Interface + +* Command-line options: Command-line options<2>. + +File Formats + +* csv — CSV File Reading and Writing:: +* configparser — Configuration file parser:: +* netrc — netrc file processing:: +* xdrlib — Encode and decode XDR data:: +* plistlib — Generate and parse Mac OS X .plist files: plistlib — Generate and parse Mac OS X plist files. + +csv — CSV File Reading and Writing + +* Module Contents: Module Contents<3>. +* Dialects and Formatting Parameters:: +* Reader Objects:: +* Writer Objects:: +* Examples: Examples<7>. + +configparser — Configuration file parser + +* Quick Start:: +* Supported Datatypes:: +* Fallback Values:: +* Supported INI File Structure:: +* Interpolation of values:: +* Mapping Protocol Access:: +* Customizing Parser Behaviour:: +* Legacy API Examples:: +* ConfigParser Objects:: +* RawConfigParser Objects:: +* Exceptions: Exceptions<5>. + +netrc — netrc file processing + +* netrc Objects:: + +xdrlib — Encode and decode XDR data + +* Packer Objects:: +* Unpacker Objects:: +* Exceptions: Exceptions<6>. + +plistlib — Generate and parse Mac OS X .plist files + +* Examples: Examples<8>. + +Cryptographic Services + +* hashlib — Secure hashes and message digests:: +* hmac — Keyed-Hashing for Message Authentication:: +* secrets — Generate secure random numbers for managing secrets:: + +hashlib — Secure hashes and message digests + +* Hash algorithms:: +* SHAKE variable length digests:: +* Key derivation:: +* BLAKE2:: + +BLAKE2 + +* Creating hash objects:: +* Constants: Constants<5>. +* Examples: Examples<9>. +* Credits:: + +Examples + +* Simple hashing:: +* Using different digest sizes:: +* Keyed hashing:: +* Randomized hashing:: +* Personalization:: +* Tree mode:: + +secrets — Generate secure random numbers for managing secrets + +* Random numbers:: +* Generating tokens:: +* Other functions:: +* Recipes and best practices:: + +Generating tokens + +* How many bytes should tokens use?:: + +Generic Operating System Services + +* os — Miscellaneous operating system interfaces:: +* io — Core tools for working with streams:: +* time — Time access and conversions:: +* argparse — Parser for command-line options, arguments and sub-commands: argparse — Parser for command-line options arguments and sub-commands. +* getopt — C-style parser for command line options:: +* logging — Logging facility for Python:: +* logging.config — Logging configuration: logging config — Logging configuration. +* logging.handlers — Logging handlers: logging handlers — Logging handlers. +* getpass — Portable password input:: +* curses — Terminal handling for character-cell displays:: +* curses.textpad — Text input widget for curses programs: curses textpad — Text input widget for curses programs. +* curses.ascii — Utilities for ASCII characters: curses ascii — Utilities for ASCII characters. +* curses.panel — A panel stack extension for curses: curses panel — A panel stack extension for curses. +* platform — Access to underlying platform’s identifying data:: +* errno — Standard errno system symbols:: +* ctypes — A foreign function library for Python:: + +os — Miscellaneous operating system interfaces + +* File Names, Command Line Arguments, and Environment Variables: File Names Command Line Arguments and Environment Variables. +* Process Parameters:: +* File Object Creation:: +* File Descriptor Operations:: +* Files and Directories:: +* Process Management:: +* Interface to the scheduler:: +* Miscellaneous System Information:: +* Random numbers: Random numbers<2>. + +File Descriptor Operations + +* Querying the size of a terminal:: +* Inheritance of File Descriptors:: + +Files and Directories + +* Linux extended attributes:: + +io — Core tools for working with streams + +* Overview:: +* High-level Module Interface:: +* Class hierarchy:: +* Performance: Performance<3>. + +Overview + +* Text I/O:: +* Binary I/O:: +* Raw I/O:: + +Class hierarchy + +* I/O Base Classes:: +* Raw File I/O:: +* Buffered Streams:: +* Text I/O: Text I/O<2>. + +Performance + +* Binary I/O: Binary I/O<2>. +* Text I/O: Text I/O<3>. +* Multi-threading: Multi-threading<3>. +* Reentrancy:: + +time — Time access and conversions + +* Functions: Functions<2>. +* Clock ID Constants:: +* Timezone Constants:: + +argparse — Parser for command-line options, arguments and sub-commands + +* Example: Example<7>. +* ArgumentParser objects:: +* The add_argument() method: The add_argument method. +* The parse_args() method: The parse_args method. +* Other utilities:: +* Upgrading optparse code:: + +Example + +* Creating a parser:: +* Adding arguments:: +* Parsing arguments:: + +ArgumentParser objects + +* prog:: +* usage:: +* description:: +* epilog:: +* parents:: +* formatter_class:: +* prefix_chars:: +* fromfile_prefix_chars:: +* argument_default:: +* allow_abbrev:: +* conflict_handler:: +* add_help:: + +The add_argument() method + +* name or flags:: +* action:: +* nargs:: +* const:: +* default:: +* type:: +* choices:: +* required:: +* help:: +* metavar:: +* dest:: +* Action classes:: + +The parse_args() method + +* Option value syntax:: +* Invalid arguments:: +* Arguments containing -:: +* Argument abbreviations (prefix matching): Argument abbreviations prefix matching. +* Beyond sys.argv: Beyond sys argv. +* The Namespace object:: + +Other utilities + +* Sub-commands:: +* FileType objects:: +* Argument groups:: +* Mutual exclusion:: +* Parser defaults:: +* Printing help:: +* Partial parsing:: +* Customizing file parsing:: +* Exiting methods:: +* Intermixed parsing:: + +logging — Logging facility for Python + +* Logger Objects:: +* Logging Levels:: +* Handler Objects:: +* Formatter Objects:: +* Filter Objects:: +* LogRecord Objects:: +* LogRecord attributes:: +* LoggerAdapter Objects:: +* Thread Safety:: +* Module-Level Functions:: +* Module-Level Attributes:: +* Integration with the warnings module:: + +logging.config — Logging configuration + +* Configuration functions:: +* Configuration dictionary schema:: +* Configuration file format:: + +Configuration dictionary schema + +* Dictionary Schema Details:: +* Incremental Configuration:: +* Object connections:: +* User-defined objects:: +* Access to external objects:: +* Access to internal objects:: +* Import resolution and custom importers:: + +logging.handlers — Logging handlers + +* StreamHandler:: +* FileHandler:: +* NullHandler:: +* WatchedFileHandler:: +* BaseRotatingHandler:: +* RotatingFileHandler:: +* TimedRotatingFileHandler:: +* SocketHandler:: +* DatagramHandler:: +* SysLogHandler:: +* NTEventLogHandler:: +* SMTPHandler:: +* MemoryHandler:: +* HTTPHandler:: +* QueueHandler:: +* QueueListener:: + +curses — Terminal handling for character-cell displays + +* Functions: Functions<3>. +* Window Objects:: +* Constants: Constants<6>. + +curses.textpad — Text input widget for curses programs + +* Textbox objects:: + +curses.panel — A panel stack extension for curses + +* Functions: Functions<4>. +* Panel Objects:: + +platform — Access to underlying platform’s identifying data + +* Cross Platform:: +* Java Platform:: +* Windows Platform:: +* Mac OS Platform:: +* Unix Platforms:: + +ctypes — A foreign function library for Python + +* ctypes tutorial:: +* ctypes reference:: + +ctypes tutorial + +* Loading dynamic link libraries:: +* Accessing functions from loaded dlls:: +* Calling functions:: +* Fundamental data types:: +* Calling functions, continued: Calling functions continued. +* Calling functions with your own custom data types:: +* Specifying the required argument types (function prototypes): Specifying the required argument types function prototypes. +* Return types:: +* Passing pointers (or; passing parameters by reference): Passing pointers or passing parameters by reference. +* Structures and unions:: +* Structure/union alignment and byte order:: +* Bit fields in structures and unions:: +* Arrays:: +* Pointers:: +* Type conversions:: +* Incomplete Types:: +* Callback functions:: +* Accessing values exported from dlls:: +* Surprises:: +* Variable-sized data types:: + +ctypes reference + +* Finding shared libraries:: +* Loading shared libraries:: +* Foreign functions:: +* Function prototypes:: +* Utility functions:: +* Data types:: +* Fundamental data types: Fundamental data types<2>. +* Structured data types:: +* Arrays and pointers:: + +Concurrent Execution + +* threading — Thread-based parallelism:: +* multiprocessing — Process-based parallelism:: +* multiprocessing.shared_memory — Provides shared memory for direct access across processes: multiprocessing shared_memory — Provides shared memory for direct access across processes. +* The concurrent package:: +* concurrent.futures — Launching parallel tasks: concurrent futures — Launching parallel tasks. +* subprocess — Subprocess management:: +* sched — Event scheduler:: +* queue — A synchronized queue class:: +* contextvars — Context Variables:: +* _thread — Low-level threading API:: +* _dummy_thread — Drop-in replacement for the _thread module:: +* dummy_threading — Drop-in replacement for the threading module:: + +threading — Thread-based parallelism + +* Thread-Local Data:: +* Thread Objects:: +* Lock Objects:: +* RLock Objects:: +* Condition Objects:: +* Semaphore Objects:: +* Event Objects:: +* Timer Objects:: +* Barrier Objects:: +* Using locks, conditions, and semaphores in the with statement: Using locks conditions and semaphores in the with statement. + +Semaphore Objects + +* Semaphore Example:: + +multiprocessing — Process-based parallelism + +* Introduction: Introduction<8>. +* Reference:: +* Programming guidelines:: +* Examples: Examples<10>. + +Introduction + +* The Process class:: +* Contexts and start methods:: +* Exchanging objects between processes:: +* Synchronization between processes:: +* Sharing state between processes:: +* Using a pool of workers:: + +Reference + +* Process and exceptions:: +* Pipes and Queues:: +* Miscellaneous: Miscellaneous<3>. +* Connection Objects: Connection Objects<2>. +* Synchronization primitives:: +* Shared ctypes Objects:: +* Managers:: +* Proxy Objects:: +* Process Pools:: +* Listeners and Clients:: +* Authentication keys:: +* Logging: Logging<2>. +* The multiprocessing.dummy module: The multiprocessing dummy module. + +Shared ctypes Objects + +* The multiprocessing.sharedctypes module: The multiprocessing sharedctypes module. + +Managers + +* Customized managers:: +* Using a remote manager:: + +Proxy Objects + +* Cleanup:: + +Listeners and Clients + +* Address Formats:: + +Programming guidelines + +* All start methods:: +* The spawn and forkserver start methods:: + +concurrent.futures — Launching parallel tasks + +* Executor Objects:: +* ThreadPoolExecutor:: +* ProcessPoolExecutor:: +* Future Objects:: +* Module Functions:: +* Exception classes:: + +ThreadPoolExecutor + +* ThreadPoolExecutor Example:: + +ProcessPoolExecutor + +* ProcessPoolExecutor Example:: + +subprocess — Subprocess management + +* Using the subprocess Module:: +* Security Considerations:: +* Popen Objects:: +* Windows Popen Helpers:: +* Older high-level API:: +* Replacing Older Functions with the subprocess Module:: +* Legacy Shell Invocation Functions:: +* Notes:: + +Using the subprocess Module + +* Frequently Used Arguments:: +* Popen Constructor:: +* Exceptions: Exceptions<7>. + +Windows Popen Helpers + +* Windows Constants:: + +Replacing Older Functions with the subprocess Module + +* Replacing /bin/sh shell command substitution:: +* Replacing shell pipeline:: +* Replacing os.system(): Replacing os system. +* Replacing the os.spawn family: Replacing the os spawn family. +* Replacing os.popen(), os.popen2(), os.popen3(): Replacing os popen os popen2 os popen3. +* Replacing functions from the popen2 module:: + +Notes + +* Converting an argument sequence to a string on Windows:: + +sched — Event scheduler + +* Scheduler Objects:: + +queue — A synchronized queue class + +* Queue Objects:: +* SimpleQueue Objects:: + +contextvars — Context Variables + +* Context Variables:: +* Manual Context Management:: +* asyncio support:: + +Networking and Interprocess Communication + +* asyncio — Asynchronous I/O:: +* socket — Low-level networking interface:: +* ssl — TLS/SSL wrapper for socket objects:: +* select — Waiting for I/O completion:: +* selectors — High-level I/O multiplexing:: +* asyncore — Asynchronous socket handler:: +* asynchat — Asynchronous socket command/response handler:: +* signal — Set handlers for asynchronous events:: +* mmap — Memory-mapped file support:: + +asyncio — Asynchronous I/O + +* Coroutines and Tasks:: +* Streams:: +* Synchronization Primitives:: +* Subprocesses:: +* Queues:: +* Exceptions: Exceptions<9>. +* Event Loop:: +* Futures:: +* Transports and Protocols:: +* Policies:: +* Platform Support:: +* High-level API Index:: +* Low-level API Index:: +* Developing with asyncio:: + +Coroutines and Tasks + +* Coroutines: Coroutines<3>. +* Awaitables:: +* Running an asyncio Program:: +* Creating Tasks:: +* Sleeping:: +* Running Tasks Concurrently:: +* Shielding From Cancellation:: +* Timeouts:: +* Waiting Primitives:: +* Scheduling From Other Threads:: +* Introspection:: +* Task Object:: +* Generator-based Coroutines:: + +Streams + +* StreamReader:: +* StreamWriter:: +* Examples: Examples<11>. + +Examples + +* TCP echo client using streams:: +* TCP echo server using streams:: +* Get HTTP headers:: +* Register an open socket to wait for data using streams:: + +Synchronization Primitives + +* Lock:: +* Event:: +* Condition:: +* Semaphore:: +* BoundedSemaphore:: + +Subprocesses + +* Creating Subprocesses:: +* Constants: Constants<7>. +* Interacting with Subprocesses:: + +Interacting with Subprocesses + +* Subprocess and Threads:: +* Examples: Examples<12>. + +Queues + +* Queue:: +* Priority Queue:: +* LIFO Queue:: +* Exceptions: Exceptions<8>. +* Examples: Examples<13>. + +Event Loop + +* Event Loop Methods:: +* Callback Handles:: +* Server Objects:: +* Event Loop Implementations:: +* Examples: Examples<14>. + +Event Loop Methods + +* Running and stopping the loop:: +* Scheduling callbacks:: +* Scheduling delayed callbacks:: +* Creating Futures and Tasks:: +* Opening network connections:: +* Creating network servers:: +* Transferring files:: +* TLS Upgrade:: +* Watching file descriptors:: +* Working with socket objects directly:: +* DNS:: +* Working with pipes:: +* Unix signals:: +* Executing code in thread or process pools:: +* Error Handling API:: +* Enabling debug mode:: +* Running Subprocesses:: + +Examples + +* Hello World with call_soon(): Hello World with call_soon. +* Display the current date with call_later(): Display the current date with call_later. +* Watch a file descriptor for read events:: +* Set signal handlers for SIGINT and SIGTERM:: + +Futures + +* Future Functions:: +* Future Object:: + +Transports and Protocols + +* Transports:: +* Protocols:: +* Examples: Examples<15>. + +Transports + +* Transports Hierarchy:: +* Base Transport:: +* Read-only Transports:: +* Write-only Transports:: +* Datagram Transports:: +* Subprocess Transports:: + +Protocols + +* Base Protocols:: +* Base Protocol:: +* Streaming Protocols:: +* Buffered Streaming Protocols:: +* Datagram Protocols:: +* Subprocess Protocols:: + +Examples + +* TCP Echo Server:: +* TCP Echo Client:: +* UDP Echo Server:: +* UDP Echo Client:: +* Connecting Existing Sockets:: +* loop.subprocess_exec() and SubprocessProtocol: loop subprocess_exec and SubprocessProtocol. + +Policies + +* Getting and Setting the Policy:: +* Policy Objects:: +* Process Watchers:: +* Custom Policies:: + +Platform Support + +* All Platforms:: +* Windows:: +* macOS:: + +Windows + +* Subprocess Support on Windows:: + +High-level API Index + +* Tasks:: +* Queues: Queues<2>. +* Subprocesses: Subprocesses<2>. +* Streams: Streams<2>. +* Synchronization:: +* Exceptions: Exceptions<10>. + +Low-level API Index + +* Obtaining the Event Loop:: +* Event Loop Methods: Event Loop Methods<2>. +* Transports: Transports<2>. +* Protocols: Protocols<2>. +* Event Loop Policies:: + +Developing with asyncio + +* Debug Mode:: +* Concurrency and Multithreading:: +* Running Blocking Code:: +* Logging: Logging<3>. +* Detect never-awaited coroutines:: +* Detect never-retrieved exceptions:: + +socket — Low-level networking interface + +* Socket families:: +* Module contents:: +* Socket Objects:: +* Notes on socket timeouts:: +* Example: Example<8>. + +Module contents + +* Exceptions: Exceptions<11>. +* Constants: Constants<8>. +* Functions: Functions<5>. + +Functions + +* Creating sockets:: +* Other functions: Other functions<2>. + +Notes on socket timeouts + +* Timeouts and the connect method:: +* Timeouts and the accept method:: + +ssl — TLS/SSL wrapper for socket objects + +* Functions, Constants, and Exceptions: Functions Constants and Exceptions. +* SSL Sockets:: +* SSL Contexts:: +* Certificates:: +* Examples: Examples<16>. +* Notes on non-blocking sockets:: +* Memory BIO Support: Memory BIO Support<2>. +* SSL session:: +* Security considerations:: +* TLS 1.3: TLS 1 3. +* LibreSSL support:: + +Functions, Constants, and Exceptions + +* Socket creation:: +* Context creation:: +* Exceptions: Exceptions<12>. +* Random generation:: +* Certificate handling:: +* Constants: Constants<9>. + +Certificates + +* Certificate chains:: +* CA certificates:: +* Combined key and certificate:: +* Self-signed certificates:: + +Examples + +* Testing for SSL support:: +* Client-side operation:: +* Server-side operation:: + +Security considerations + +* Best defaults:: +* Manual settings:: +* Multi-processing:: + +Manual settings + +* Verifying certificates:: +* Protocol versions:: +* Cipher selection:: + +select — Waiting for I/O completion + +* /dev/poll Polling Objects:: +* Edge and Level Trigger Polling (epoll) Objects: Edge and Level Trigger Polling epoll Objects. +* Polling Objects:: +* Kqueue Objects:: +* Kevent Objects:: + +selectors — High-level I/O multiplexing + +* Introduction: Introduction<9>. +* Classes: Classes<3>. +* Examples: Examples<17>. + +asyncore — Asynchronous socket handler + +* asyncore Example basic HTTP client:: +* asyncore Example basic echo server:: + +asynchat — Asynchronous socket command/response handler + +* asynchat Example:: + +signal — Set handlers for asynchronous events + +* General rules:: +* Module contents: Module contents<2>. +* Example: Example<9>. +* Note on SIGPIPE:: + +General rules + +* Execution of Python signal handlers:: +* Signals and threads:: + +mmap — Memory-mapped file support + +* MADV_* Constants:: + +Internet Data Handling + +* email — An email and MIME handling package:: +* json — JSON encoder and decoder:: +* mailcap — Mailcap file handling:: +* mailbox — Manipulate mailboxes in various formats:: +* mimetypes — Map filenames to MIME types:: +* base64 — Base16, Base32, Base64, Base85 Data Encodings: base64 — Base16 Base32 Base64 Base85 Data Encodings. +* binhex — Encode and decode binhex4 files:: +* binascii — Convert between binary and ASCII:: +* quopri — Encode and decode MIME quoted-printable data:: +* uu — Encode and decode uuencode files:: + +email — An email and MIME handling package + +* email.message; Representing an email message: email message Representing an email message. +* email.parser; Parsing email messages: email parser Parsing email messages. +* email.generator; Generating MIME documents: email generator Generating MIME documents. +* email.policy; Policy Objects: email policy Policy Objects. +* email.errors; Exception and Defect classes: email errors Exception and Defect classes. +* email.headerregistry; Custom Header Objects: email headerregistry Custom Header Objects. +* email.contentmanager; Managing MIME Content: email contentmanager Managing MIME Content. +* email; Examples: email Examples. +* email.message.Message; Representing an email message using the compat32 API: email message Message Representing an email message using the compat32 API. +* email.mime; Creating email and MIME objects from scratch: email mime Creating email and MIME objects from scratch. +* email.header; Internationalized headers: email header Internationalized headers. +* email.charset; Representing character sets: email charset Representing character sets. +* email.encoders; Encoders: email encoders Encoders. +* email.utils; Miscellaneous utilities: email utils Miscellaneous utilities. +* email.iterators; Iterators: email iterators Iterators. + +email.parser: Parsing email messages + +* FeedParser API:: +* Parser API:: +* Additional notes:: + +email.contentmanager: Managing MIME Content + +* Content Manager Instances:: + +json — JSON encoder and decoder + +* Basic Usage:: +* Encoders and Decoders:: +* Exceptions: Exceptions<13>. +* Standard Compliance and Interoperability:: +* Command Line Interface: Command Line Interface<2>. + +Standard Compliance and Interoperability + +* Character Encodings:: +* Infinite and NaN Number Values:: +* Repeated Names Within an Object:: +* Top-level Non-Object, Non-Array Values: Top-level Non-Object Non-Array Values. +* Implementation Limitations:: + +Command Line Interface + +* Command line options: Command line options<2>. + +mailbox — Manipulate mailboxes in various formats + +* Mailbox objects:: +* Message objects:: +* Exceptions: Exceptions<14>. +* Examples: Examples<18>. + +Mailbox objects + +* Maildir:: +* mbox:: +* MH:: +* Babyl:: +* MMDF:: + +Message objects + +* MaildirMessage:: +* mboxMessage:: +* MHMessage:: +* BabylMessage:: +* MMDFMessage:: + +mimetypes — Map filenames to MIME types + +* MimeTypes Objects:: + +binhex — Encode and decode binhex4 files + +* Notes: Notes<2>. + +Structured Markup Processing Tools + +* html — HyperText Markup Language support:: +* html.parser — Simple HTML and XHTML parser: html parser — Simple HTML and XHTML parser. +* html.entities — Definitions of HTML general entities: html entities — Definitions of HTML general entities. +* XML Processing Modules:: +* xml.etree.ElementTree — The ElementTree XML API: xml etree ElementTree — The ElementTree XML API. +* xml.dom — The Document Object Model API: xml dom — The Document Object Model API. +* xml.dom.minidom — Minimal DOM implementation: xml dom minidom — Minimal DOM implementation. +* xml.dom.pulldom — Support for building partial DOM trees: xml dom pulldom — Support for building partial DOM trees. +* xml.sax — Support for SAX2 parsers: xml sax — Support for SAX2 parsers. +* xml.sax.handler — Base classes for SAX handlers: xml sax handler — Base classes for SAX handlers. +* xml.sax.saxutils — SAX Utilities: xml sax saxutils — SAX Utilities. +* xml.sax.xmlreader — Interface for XML parsers: xml sax xmlreader — Interface for XML parsers. +* xml.parsers.expat — Fast XML parsing using Expat: xml parsers expat — Fast XML parsing using Expat. + +html.parser — Simple HTML and XHTML parser + +* Example HTML Parser Application:: +* HTMLParser Methods:: +* Examples: Examples<19>. + +XML Processing Modules + +* XML vulnerabilities:: +* The defusedxml Package:: + +xml.etree.ElementTree — The ElementTree XML API + +* Tutorial:: +* XPath support:: +* Reference: Reference<2>. +* XInclude support:: +* Reference: Reference<3>. + +Tutorial + +* XML tree and elements:: +* Parsing XML:: +* Pull API for non-blocking parsing:: +* Finding interesting elements:: +* Modifying an XML File:: +* Building XML documents:: +* Parsing XML with Namespaces:: +* Additional resources:: + +XPath support + +* Example: Example<10>. +* Supported XPath syntax:: + +Reference + +* Functions: Functions<6>. + +XInclude support + +* Example: Example<11>. + +Reference + +* Functions: Functions<7>. +* Element Objects:: +* ElementTree Objects:: +* QName Objects:: +* TreeBuilder Objects:: +* XMLParser Objects:: +* XMLPullParser Objects:: +* Exceptions: Exceptions<15>. + +xml.dom — The Document Object Model API + +* Module Contents: Module Contents<4>. +* Objects in the DOM:: +* Conformance:: + +Objects in the DOM + +* DOMImplementation Objects:: +* Node Objects:: +* NodeList Objects:: +* DocumentType Objects:: +* Document Objects:: +* Element Objects: Element Objects<2>. +* Attr Objects:: +* NamedNodeMap Objects:: +* Comment Objects:: +* Text and CDATASection Objects:: +* ProcessingInstruction Objects:: +* Exceptions: Exceptions<16>. + +Conformance + +* Type Mapping:: +* Accessor Methods:: + +xml.dom.minidom — Minimal DOM implementation + +* DOM Objects:: +* DOM Example:: +* minidom and the DOM standard:: + +xml.dom.pulldom — Support for building partial DOM trees + +* DOMEventStream Objects:: + +xml.sax — Support for SAX2 parsers + +* SAXException Objects:: + +xml.sax.handler — Base classes for SAX handlers + +* ContentHandler Objects:: +* DTDHandler Objects:: +* EntityResolver Objects:: +* ErrorHandler Objects:: + +xml.sax.xmlreader — Interface for XML parsers + +* XMLReader Objects:: +* IncrementalParser Objects:: +* Locator Objects:: +* InputSource Objects:: +* The Attributes Interface:: +* The AttributesNS Interface:: + +xml.parsers.expat — Fast XML parsing using Expat + +* XMLParser Objects: XMLParser Objects<2>. +* ExpatError Exceptions:: +* Example: Example<12>. +* Content Model Descriptions:: +* Expat error constants:: + +Internet Protocols and Support + +* webbrowser — Convenient Web-browser controller:: +* cgi — Common Gateway Interface support:: +* cgitb — Traceback manager for CGI scripts:: +* wsgiref — WSGI Utilities and Reference Implementation:: +* urllib — URL handling modules:: +* urllib.request — Extensible library for opening URLs: urllib request — Extensible library for opening URLs. +* urllib.response — Response classes used by urllib: urllib response — Response classes used by urllib. +* urllib.parse — Parse URLs into components: urllib parse — Parse URLs into components. +* urllib.error — Exception classes raised by urllib.request: urllib error — Exception classes raised by urllib request. +* urllib.robotparser — Parser for robots.txt: urllib robotparser — Parser for robots txt. +* http — HTTP modules:: +* http.client — HTTP protocol client: http client — HTTP protocol client. +* ftplib — FTP protocol client:: +* poplib — POP3 protocol client:: +* imaplib — IMAP4 protocol client:: +* nntplib — NNTP protocol client:: +* smtplib — SMTP protocol client:: +* smtpd — SMTP Server:: +* telnetlib — Telnet client:: +* uuid — UUID objects according to RFC 4122:: +* socketserver — A framework for network servers:: +* http.server — HTTP servers: http server — HTTP servers. +* http.cookies — HTTP state management: http cookies — HTTP state management. +* http.cookiejar — Cookie handling for HTTP clients: http cookiejar — Cookie handling for HTTP clients. +* xmlrpc — XMLRPC server and client modules:: +* xmlrpc.client — XML-RPC client access: xmlrpc client — XML-RPC client access. +* xmlrpc.server — Basic XML-RPC servers: xmlrpc server — Basic XML-RPC servers. +* ipaddress — IPv4/IPv6 manipulation library:: + +webbrowser — Convenient Web-browser controller + +* Browser Controller Objects:: + +cgi — Common Gateway Interface support + +* Introduction: Introduction<10>. +* Using the cgi module:: +* Higher Level Interface:: +* Functions: Functions<8>. +* Caring about security:: +* Installing your CGI script on a Unix system:: +* Testing your CGI script:: +* Debugging CGI scripts:: +* Common problems and solutions:: + +wsgiref — WSGI Utilities and Reference Implementation + +* wsgiref.util – WSGI environment utilities: wsgiref util – WSGI environment utilities. +* wsgiref.headers – WSGI response header tools: wsgiref headers – WSGI response header tools. +* wsgiref.simple_server – a simple WSGI HTTP server: wsgiref simple_server – a simple WSGI HTTP server. +* wsgiref.validate — WSGI conformance checker: wsgiref validate — WSGI conformance checker. +* wsgiref.handlers – server/gateway base classes: wsgiref handlers – server/gateway base classes. +* Examples: Examples<20>. + +urllib.request — Extensible library for opening URLs + +* Request Objects:: +* OpenerDirector Objects:: +* BaseHandler Objects:: +* HTTPRedirectHandler Objects:: +* HTTPCookieProcessor Objects:: +* ProxyHandler Objects:: +* HTTPPasswordMgr Objects:: +* HTTPPasswordMgrWithPriorAuth Objects:: +* AbstractBasicAuthHandler Objects:: +* HTTPBasicAuthHandler Objects:: +* ProxyBasicAuthHandler Objects:: +* AbstractDigestAuthHandler Objects:: +* HTTPDigestAuthHandler Objects:: +* ProxyDigestAuthHandler Objects:: +* HTTPHandler Objects:: +* HTTPSHandler Objects:: +* FileHandler Objects:: +* DataHandler Objects:: +* FTPHandler Objects:: +* CacheFTPHandler Objects:: +* UnknownHandler Objects:: +* HTTPErrorProcessor Objects:: +* Examples: Examples<21>. +* Legacy interface:: +* urllib.request Restrictions: urllib request Restrictions. + +urllib.parse — Parse URLs into components + +* URL Parsing:: +* Parsing ASCII Encoded Bytes:: +* Structured Parse Results:: +* URL Quoting:: + +http — HTTP modules + +* HTTP status codes:: + +http.client — HTTP protocol client + +* HTTPConnection Objects:: +* HTTPResponse Objects:: +* Examples: Examples<22>. +* HTTPMessage Objects:: + +ftplib — FTP protocol client + +* FTP Objects:: +* FTP_TLS Objects:: + +poplib — POP3 protocol client + +* POP3 Objects:: +* POP3 Example:: + +imaplib — IMAP4 protocol client + +* IMAP4 Objects:: +* IMAP4 Example:: + +nntplib — NNTP protocol client + +* NNTP Objects:: +* Utility functions: Utility functions<2>. + +NNTP Objects + +* Attributes:: +* Methods: Methods<3>. + +smtplib — SMTP protocol client + +* SMTP Objects:: +* SMTP Example:: + +smtpd — SMTP Server + +* SMTPServer Objects:: +* DebuggingServer Objects:: +* PureProxy Objects:: +* MailmanProxy Objects:: +* SMTPChannel Objects:: + +telnetlib — Telnet client + +* Telnet Objects:: +* Telnet Example:: + +uuid — UUID objects according to RFC 4122 + +* Example: Example<13>. + +socketserver — A framework for network servers + +* Server Creation Notes:: +* Server Objects: Server Objects<2>. +* Request Handler Objects:: +* Examples: Examples<23>. + +Examples + +* socketserver.TCPServer Example: socketserver TCPServer Example. +* socketserver.UDPServer Example: socketserver UDPServer Example. +* Asynchronous Mixins:: + +http.cookies — HTTP state management + +* Cookie Objects:: +* Morsel Objects:: +* Example: Example<14>. + +http.cookiejar — Cookie handling for HTTP clients + +* CookieJar and FileCookieJar Objects:: +* FileCookieJar subclasses and co-operation with web browsers:: +* CookiePolicy Objects:: +* DefaultCookiePolicy Objects:: +* Cookie Objects: Cookie Objects<2>. +* Examples: Examples<24>. + +xmlrpc.client — XML-RPC client access + +* ServerProxy Objects:: +* DateTime Objects:: +* Binary Objects:: +* Fault Objects:: +* ProtocolError Objects:: +* MultiCall Objects:: +* Convenience Functions:: +* Example of Client Usage:: +* Example of Client and Server Usage:: + +xmlrpc.server — Basic XML-RPC servers + +* SimpleXMLRPCServer Objects:: +* CGIXMLRPCRequestHandler:: +* Documenting XMLRPC server:: +* DocXMLRPCServer Objects:: +* DocCGIXMLRPCRequestHandler:: + +SimpleXMLRPCServer Objects + +* SimpleXMLRPCServer Example:: + +ipaddress — IPv4/IPv6 manipulation library + +* Convenience factory functions:: +* IP Addresses:: +* IP Network definitions:: +* Interface objects:: +* Other Module Level Functions:: +* Custom Exceptions:: + +IP Addresses + +* Address objects:: +* Conversion to Strings and Integers:: +* Operators: Operators<3>. + +Operators + +* Comparison operators:: +* Arithmetic operators:: + +IP Network definitions + +* Prefix, net mask and host mask: Prefix net mask and host mask. +* Network objects:: +* Operators: Operators<4>. + +Operators + +* Logical operators:: +* Iteration: Iteration<2>. +* Networks as containers of addresses:: + +Interface objects + +* Operators: Operators<5>. + +Operators + +* Logical operators: Logical operators<2>. + +Multimedia Services + +* audioop — Manipulate raw audio data:: +* aifc — Read and write AIFF and AIFC files:: +* sunau — Read and write Sun AU files:: +* wave — Read and write WAV files:: +* chunk — Read IFF chunked data:: +* colorsys — Conversions between color systems:: +* imghdr — Determine the type of an image:: +* sndhdr — Determine type of sound file:: +* ossaudiodev — Access to OSS-compatible audio devices:: + +sunau — Read and write Sun AU files + +* AU_read Objects:: +* AU_write Objects:: + +wave — Read and write WAV files + +* Wave_read Objects:: +* Wave_write Objects:: + +ossaudiodev — Access to OSS-compatible audio devices + +* Audio Device Objects:: +* Mixer Device Objects:: + +Internationalization + +* gettext — Multilingual internationalization services:: +* locale — Internationalization services:: + +gettext — Multilingual internationalization services + +* GNU gettext API:: +* Class-based API:: +* Internationalizing your programs and modules:: +* Acknowledgements: Acknowledgements<9>. + +Class-based API + +* The NullTranslations class:: +* The GNUTranslations class:: +* Solaris message catalog support:: +* The Catalog constructor:: + +Internationalizing your programs and modules + +* Localizing your module:: +* Localizing your application:: +* Changing languages on the fly:: +* Deferred translations:: + +locale — Internationalization services + +* Background, details, hints, tips and caveats: Background details hints tips and caveats. +* For extension writers and programs that embed Python:: +* Access to message catalogs:: + +Program Frameworks + +* turtle — Turtle graphics:: +* cmd — Support for line-oriented command interpreters:: +* shlex — Simple lexical analysis:: + +turtle — Turtle graphics + +* Introduction: Introduction<11>. +* Overview of available Turtle and Screen methods:: +* Methods of RawTurtle/Turtle and corresponding functions:: +* Methods of TurtleScreen/Screen and corresponding functions:: +* Public classes:: +* Help and configuration:: +* turtledemo — Demo scripts:: +* Changes since Python 2.6: Changes since Python 2 6. +* Changes since Python 3.0: Changes since Python 3 0. + +Overview of available Turtle and Screen methods + +* Turtle methods:: +* Methods of TurtleScreen/Screen:: + +Methods of RawTurtle/Turtle and corresponding functions + +* Turtle motion:: +* Tell Turtle’s state:: +* Settings for measurement:: +* Pen control:: +* Turtle state:: +* Using events:: +* Special Turtle methods:: +* Compound shapes:: + +Pen control + +* Drawing state:: +* Color control:: +* Filling:: +* More drawing control:: + +Turtle state + +* Visibility:: +* Appearance:: + +Methods of TurtleScreen/Screen and corresponding functions + +* Window control:: +* Animation control:: +* Using screen events:: +* Input methods:: +* Settings and special methods:: +* Methods specific to Screen, not inherited from TurtleScreen: Methods specific to Screen not inherited from TurtleScreen. + +Help and configuration + +* How to use help:: +* Translation of docstrings into different languages:: +* How to configure Screen and Turtles:: + +cmd — Support for line-oriented command interpreters + +* Cmd Objects:: +* Cmd Example:: + +shlex — Simple lexical analysis + +* shlex Objects:: +* Parsing Rules:: +* Improved Compatibility with Shells:: + +Graphical User Interfaces with Tk + +* tkinter — Python interface to Tcl/Tk:: +* tkinter.ttk — Tk themed widgets: tkinter ttk — Tk themed widgets. +* tkinter.tix — Extension widgets for Tk: tkinter tix — Extension widgets for Tk. +* tkinter.scrolledtext — Scrolled Text Widget: tkinter scrolledtext — Scrolled Text Widget. +* IDLE: IDLE<3>. +* Other Graphical User Interface Packages:: + +tkinter — Python interface to Tcl/Tk + +* Tkinter Modules:: +* Tkinter Life Preserver:: +* A (Very) Quick Look at Tcl/Tk: A Very Quick Look at Tcl/Tk. +* Mapping Basic Tk into Tkinter:: +* How Tk and Tkinter are Related:: +* Handy Reference:: +* File Handlers:: + +Tkinter Life Preserver + +* How To Use This Section:: +* A Simple Hello World Program:: + +Handy Reference + +* Setting Options:: +* The Packer:: +* Packer Options:: +* Coupling Widget Variables:: +* The Window Manager:: +* Tk Option Data Types:: +* Bindings and Events:: +* The index Parameter:: +* Images:: + +tkinter.ttk — Tk themed widgets + +* Using Ttk:: +* Ttk Widgets:: +* Widget:: +* Combobox:: +* Spinbox:: +* Notebook:: +* Progressbar:: +* Separator:: +* Sizegrip:: +* Treeview:: +* Ttk Styling:: + +Widget + +* Standard Options:: +* Scrollable Widget Options:: +* Label Options:: +* Compatibility Options:: +* Widget States:: +* ttk.Widget: ttk Widget. + +Combobox + +* Options:: +* Virtual events:: +* ttk.Combobox: ttk Combobox. + +Spinbox + +* Options: Options<2>. +* Virtual events: Virtual events<2>. +* ttk.Spinbox: ttk Spinbox. + +Notebook + +* Options: Options<3>. +* Tab Options:: +* Tab Identifiers:: +* Virtual Events:: +* ttk.Notebook: ttk Notebook. + +Progressbar + +* Options: Options<4>. +* ttk.Progressbar: ttk Progressbar. + +Separator + +* Options: Options<5>. + +Sizegrip + +* Platform-specific notes:: +* Bugs:: + +Treeview + +* Options: Options<6>. +* Item Options:: +* Tag Options:: +* Column Identifiers:: +* Virtual Events: Virtual Events<2>. +* ttk.Treeview: ttk Treeview. + +Ttk Styling + +* Layouts:: + +tkinter.tix — Extension widgets for Tk + +* Using Tix:: +* Tix Widgets:: +* Tix Commands:: + +Tix Widgets + +* Basic Widgets:: +* File Selectors:: +* Hierarchical ListBox:: +* Tabular ListBox:: +* Manager Widgets:: +* Image Types:: +* Miscellaneous Widgets:: +* Form Geometry Manager:: + +IDLE + +* Menus:: +* Editing and navigation:: +* Startup and code execution:: +* Help and preferences:: + +Menus + +* File menu (Shell and Editor): File menu Shell and Editor. +* Edit menu (Shell and Editor): Edit menu Shell and Editor. +* Format menu (Editor window only): Format menu Editor window only. +* Run menu (Editor window only): Run menu Editor window only. +* Shell menu (Shell window only): Shell menu Shell window only. +* Debug menu (Shell window only): Debug menu Shell window only. +* Options menu (Shell and Editor): Options menu Shell and Editor. +* Window menu (Shell and Editor): Window menu Shell and Editor. +* Help menu (Shell and Editor): Help menu Shell and Editor. +* Context Menus:: + +Editing and navigation + +* Editor windows:: +* Key bindings:: +* Automatic indentation:: +* Completions:: +* Calltips:: +* Code Context:: +* Python Shell window:: +* Text colors:: + +Startup and code execution + +* Command line usage:: +* Startup failure:: +* Running user code:: +* User output in Shell:: +* Developing tkinter applications:: +* Running without a subprocess:: + +Help and preferences + +* Help sources:: +* Setting preferences:: +* IDLE on macOS:: +* Extensions:: + +Development Tools + +* typing — Support for type hints:: +* pydoc — Documentation generator and online help system:: +* doctest — Test interactive Python examples:: +* unittest — Unit testing framework:: +* unittest.mock — mock object library: unittest mock — mock object library. +* unittest.mock — getting started: unittest mock — getting started. +* 2to3 - Automated Python 2 to 3 code translation:: +* test — Regression tests package for Python:: +* test.support — Utilities for the Python test suite: test support — Utilities for the Python test suite. +* test.support.script_helper — Utilities for the Python execution tests: test support script_helper — Utilities for the Python execution tests. + +typing — Support for type hints + +* Type aliases:: +* NewType:: +* Callable:: +* Generics:: +* User-defined generic types:: +* The Any type:: +* Nominal vs structural subtyping:: +* Classes, functions, and decorators: Classes functions and decorators. + +doctest — Test interactive Python examples + +* Simple Usage; Checking Examples in Docstrings: Simple Usage Checking Examples in Docstrings. +* Simple Usage; Checking Examples in a Text File: Simple Usage Checking Examples in a Text File. +* How It Works:: +* Basic API:: +* Unittest API:: +* Advanced API:: +* Debugging:: +* Soapbox:: + +How It Works + +* Which Docstrings Are Examined?:: +* How are Docstring Examples Recognized?:: +* What’s the Execution Context?:: +* What About Exceptions?:: +* Option Flags:: +* Directives:: +* Warnings: Warnings<2>. + +Advanced API + +* DocTest Objects:: +* Example Objects:: +* DocTestFinder objects:: +* DocTestParser objects:: +* DocTestRunner objects:: +* OutputChecker objects:: + +unittest — Unit testing framework + +* Basic example:: +* Command-Line Interface: Command-Line Interface<3>. +* Test Discovery:: +* Organizing test code:: +* Re-using old test code:: +* Skipping tests and expected failures:: +* Distinguishing test iterations using subtests:: +* Classes and functions:: +* Class and Module Fixtures:: +* Signal Handling:: + +Command-Line Interface + +* Command-line options: Command-line options<3>. + +Classes and functions + +* Test cases:: +* Grouping tests:: +* Loading and running tests:: + +Test cases + +* Deprecated aliases:: + +Loading and running tests + +* load_tests Protocol:: + +Class and Module Fixtures + +* setUpClass and tearDownClass:: +* setUpModule and tearDownModule:: + +unittest.mock — mock object library + +* Quick Guide:: +* The Mock Class:: +* The patchers:: +* MagicMock and magic method support:: +* Helpers:: + +The Mock Class + +* Calling:: +* Deleting Attributes:: +* Mock names and the name attribute:: +* Attaching Mocks as Attributes:: + +The patchers + +* patch:: +* patch.object: patch object. +* patch.dict: patch dict. +* patch.multiple: patch multiple. +* patch methods; start and stop: patch methods start and stop. +* patch builtins:: +* TEST_PREFIX:: +* Nesting Patch Decorators:: +* Where to patch:: +* Patching Descriptors and Proxy Objects:: + +MagicMock and magic method support + +* Mocking Magic Methods:: +* Magic Mock:: + +Helpers + +* sentinel:: +* DEFAULT:: +* call:: +* create_autospec:: +* ANY:: +* FILTER_DIR:: +* mock_open:: +* Autospeccing:: +* Sealing mocks:: + +unittest.mock — getting started + +* Using Mock:: +* Patch Decorators:: +* Further Examples:: + +Using Mock + +* Mock Patching Methods:: +* Mock for Method Calls on an Object:: +* Mocking Classes:: +* Naming your mocks:: +* Tracking all Calls:: +* Setting Return Values and Attributes:: +* Raising exceptions with mocks:: +* Side effect functions and iterables:: +* Mocking asynchronous iterators:: +* Mocking asynchronous context manager:: +* Creating a Mock from an Existing Object:: + +Further Examples + +* Mocking chained calls:: +* Partial mocking:: +* Mocking a Generator Method:: +* Applying the same patch to every test method:: +* Mocking Unbound Methods:: +* Checking multiple calls with mock:: +* Coping with mutable arguments:: +* Nesting Patches:: +* Mocking a dictionary with MagicMock:: +* Mock subclasses and their attributes:: +* Mocking imports with patch.dict: Mocking imports with patch dict. +* Tracking order of calls and less verbose call assertions:: +* More complex argument matching:: + +2to3 - Automated Python 2 to 3 code translation + +* Using 2to3:: +* Fixers:: +* lib2to3 - 2to3’s library:: + +test — Regression tests package for Python + +* Writing Unit Tests for the test package:: +* Running tests using the command-line interface:: + +Debugging and Profiling + +* Audit events table:: +* bdb — Debugger framework:: +* faulthandler — Dump the Python traceback:: +* pdb — The Python Debugger:: +* The Python Profilers:: +* timeit — Measure execution time of small code snippets:: +* trace — Trace or track Python statement execution:: +* tracemalloc — Trace memory allocations:: + +faulthandler — Dump the Python traceback + +* Dumping the traceback:: +* Fault handler state:: +* Dumping the tracebacks after a timeout:: +* Dumping the traceback on a user signal:: +* Issue with file descriptors:: +* Example: Example<15>. + +pdb — The Python Debugger + +* Debugger Commands:: + +The Python Profilers + +* Introduction to the profilers:: +* Instant User’s Manual:: +* profile and cProfile Module Reference:: +* The Stats Class:: +* What Is Deterministic Profiling?:: +* Limitations:: +* Calibration:: +* Using a custom timer:: + +timeit — Measure execution time of small code snippets + +* Basic Examples: Basic Examples<2>. +* Python Interface:: +* Command-Line Interface: Command-Line Interface<4>. +* Examples: Examples<25>. + +trace — Trace or track Python statement execution + +* Command-Line Usage:: +* Programmatic Interface:: + +Command-Line Usage + +* Main options:: +* Modifiers:: +* Filters:: + +tracemalloc — Trace memory allocations + +* Examples: Examples<26>. +* API:: + +Examples + +* Display the top 10:: +* Compute differences:: +* Get the traceback of a memory block:: +* Pretty top:: + +API + +* Functions: Functions<9>. +* DomainFilter:: +* Filter:: +* Frame:: +* Snapshot:: +* Statistic:: +* StatisticDiff:: +* Trace:: +* Traceback:: + +Software Packaging and Distribution + +* distutils — Building and installing Python modules:: +* ensurepip — Bootstrapping the pip installer:: +* venv — Creation of virtual environments:: +* zipapp — Manage executable Python zip archives:: + +ensurepip — Bootstrapping the pip installer + +* Command line interface:: +* Module API:: + +venv — Creation of virtual environments + +* Creating virtual environments:: +* API: API<2>. +* An example of extending EnvBuilder:: + +zipapp — Manage executable Python zip archives + +* Basic Example:: +* Command-Line Interface: Command-Line Interface<5>. +* Python API:: +* Examples: Examples<27>. +* Specifying the Interpreter:: +* Creating Standalone Applications with zipapp:: +* The Python Zip Application Archive Format:: + +Creating Standalone Applications with zipapp + +* Making a Windows executable:: +* Caveats:: + +Python Runtime Services + +* sys — System-specific parameters and functions:: +* sysconfig — Provide access to Python’s configuration information:: +* builtins — Built-in objects:: +* __main__ — Top-level script environment:: +* warnings — Warning control:: +* dataclasses — Data Classes:: +* contextlib — Utilities for with-statement contexts:: +* abc — Abstract Base Classes:: +* atexit — Exit handlers:: +* traceback — Print or retrieve a stack traceback:: +* __future__ — Future statement definitions:: +* gc — Garbage Collector interface:: +* inspect — Inspect live objects:: +* site — Site-specific configuration hook:: + +sysconfig — Provide access to Python’s configuration information + +* Configuration variables:: +* Installation paths:: +* Other functions: Other functions<3>. +* Using sysconfig as a script:: + +warnings — Warning control + +* Warning Categories:: +* The Warnings Filter:: +* Temporarily Suppressing Warnings:: +* Testing Warnings:: +* Updating Code For New Versions of Dependencies:: +* Available Functions:: +* Available Context Managers:: + +The Warnings Filter + +* Describing Warning Filters:: +* Default Warning Filter:: +* Overriding the default filter:: + +dataclasses — Data Classes + +* Module-level decorators, classes, and functions: Module-level decorators classes and functions. +* Post-init processing:: +* Class variables:: +* Init-only variables:: +* Frozen instances:: +* Inheritance: Inheritance<2>. +* Default factory functions:: +* Mutable default values:: +* Exceptions: Exceptions<17>. + +contextlib — Utilities for with-statement contexts + +* Utilities:: +* Examples and Recipes: Examples and Recipes<2>. +* Single use, reusable and reentrant context managers: Single use reusable and reentrant context managers. + +Examples and Recipes + +* Supporting a variable number of context managers:: +* Catching exceptions from __enter__ methods:: +* Cleaning up in an __enter__ implementation:: +* Replacing any use of try-finally and flag variables:: +* Using a context manager as a function decorator:: + +Single use, reusable and reentrant context managers + +* Reentrant context managers:: +* Reusable context managers:: + +atexit — Exit handlers + +* atexit Example:: + +traceback — Print or retrieve a stack traceback + +* TracebackException Objects:: +* StackSummary Objects:: +* FrameSummary Objects:: +* Traceback Examples:: + +inspect — Inspect live objects + +* Types and members:: +* Retrieving source code:: +* Introspecting callables with the Signature object:: +* Classes and functions: Classes and functions<2>. +* The interpreter stack:: +* Fetching attributes statically:: +* Current State of Generators and Coroutines:: +* Code Objects Bit Flags:: +* Command Line Interface: Command Line Interface<3>. + +site — Site-specific configuration hook + +* Readline configuration:: +* Module contents: Module contents<3>. +* Command Line Interface: Command Line Interface<4>. + +Custom Python Interpreters + +* code — Interpreter base classes:: +* codeop — Compile Python code:: + +code — Interpreter base classes + +* Interactive Interpreter Objects:: +* Interactive Console Objects:: + +Importing Modules + +* zipimport — Import modules from Zip archives:: +* pkgutil — Package extension utility:: +* modulefinder — Find modules used by a script:: +* runpy — Locating and executing Python modules:: +* importlib — The implementation of import:: +* Using importlib.metadata: Using importlib metadata. + +zipimport — Import modules from Zip archives + +* zipimporter Objects:: +* Examples: Examples<28>. + +modulefinder — Find modules used by a script + +* Example usage of ModuleFinder:: + +importlib — The implementation of import + +* Introduction: Introduction<12>. +* Functions: Functions<10>. +* importlib.abc – Abstract base classes related to import: importlib abc – Abstract base classes related to import. +* importlib.resources – Resources: importlib resources – Resources. +* importlib.machinery – Importers and path hooks: importlib machinery – Importers and path hooks. +* importlib.util – Utility code for importers: importlib util – Utility code for importers. +* Examples: Examples<29>. + +Examples + +* Importing programmatically:: +* Checking if a module can be imported:: +* Importing a source file directly:: +* Setting up an importer:: +* Approximating importlib.import_module(): Approximating importlib import_module. + +Using importlib.metadata + +* Overview: Overview<2>. +* Functional API: Functional API<2>. +* Distributions:: +* Extending the search algorithm:: + +Functional API + +* Entry points:: +* Distribution metadata:: +* Distribution versions:: +* Distribution files:: +* Distribution requirements:: + +Python Language Services + +* parser — Access Python parse trees:: +* ast — Abstract Syntax Trees:: +* symtable — Access to the compiler’s symbol tables:: +* symbol — Constants used with Python parse trees:: +* token — Constants used with Python parse trees:: +* keyword — Testing for Python keywords:: +* tokenize — Tokenizer for Python source:: +* tabnanny — Detection of ambiguous indentation:: +* pyclbr — Python module browser support:: +* py_compile — Compile Python source files:: +* compileall — Byte-compile Python libraries:: +* dis — Disassembler for Python bytecode:: +* pickletools — Tools for pickle developers:: + +parser — Access Python parse trees + +* Creating ST Objects:: +* Converting ST Objects:: +* Queries on ST Objects:: +* Exceptions and Error Handling:: +* ST Objects:: +* Example; Emulation of compile(): Example Emulation of compile. + +ast — Abstract Syntax Trees + +* Node classes:: +* Abstract Grammar:: +* ast Helpers:: + +symtable — Access to the compiler’s symbol tables + +* Generating Symbol Tables:: +* Examining Symbol Tables:: + +tokenize — Tokenizer for Python source + +* Tokenizing Input:: +* Command-Line Usage: Command-Line Usage<2>. +* Examples: Examples<30>. + +pyclbr — Python module browser support + +* Function Objects:: +* Class Objects: Class Objects<2>. + +compileall — Byte-compile Python libraries + +* Command-line use:: +* Public functions:: + +dis — Disassembler for Python bytecode + +* Bytecode analysis:: +* Analysis functions:: +* Python Bytecode Instructions:: +* Opcode collections:: + +pickletools — Tools for pickle developers + +* Command line usage: Command line usage<2>. +* Programmatic Interface: Programmatic Interface<2>. + +Command line usage + +* Command line options: Command line options<3>. + +Miscellaneous Services + +* formatter — Generic output formatting:: + +formatter — Generic output formatting + +* The Formatter Interface:: +* Formatter Implementations:: +* The Writer Interface:: +* Writer Implementations:: + +MS Windows Specific Services + +* msilib — Read and write Microsoft Installer files:: +* msvcrt — Useful routines from the MS VC++ runtime:: +* winreg — Windows registry access:: +* winsound — Sound-playing interface for Windows:: + +msilib — Read and write Microsoft Installer files + +* Database Objects:: +* View Objects:: +* Summary Information Objects:: +* Record Objects:: +* Errors:: +* CAB Objects:: +* Directory Objects:: +* Features: Features<3>. +* GUI classes:: +* Precomputed tables:: + +msvcrt — Useful routines from the MS VC++ runtime + +* File Operations:: +* Console I/O:: +* Other Functions:: + +winreg — Windows registry access + +* Functions: Functions<11>. +* Constants: Constants<10>. +* Registry Handle Objects:: + +Constants + +* HKEY_* Constants:: +* Access Rights:: +* Value Types:: + +Access Rights + +* 64-bit Specific:: + +Unix Specific Services + +* posix — The most common POSIX system calls:: +* pwd — The password database:: +* spwd — The shadow password database:: +* grp — The group database:: +* crypt — Function to check Unix passwords:: +* termios — POSIX style tty control:: +* tty — Terminal control functions:: +* pty — Pseudo-terminal utilities:: +* fcntl — The fcntl and ioctl system calls:: +* pipes — Interface to shell pipelines:: +* resource — Resource usage information:: +* nis — Interface to Sun’s NIS (Yellow Pages): nis — Interface to Sun’s NIS Yellow Pages. +* syslog — Unix syslog library routines:: + +posix — The most common POSIX system calls + +* Large File Support:: +* Notable Module Contents:: + +crypt — Function to check Unix passwords + +* Hashing Methods:: +* Module Attributes:: +* Module Functions: Module Functions<2>. +* Examples: Examples<31>. + +termios — POSIX style tty control + +* Example: Example<16>. + +pty — Pseudo-terminal utilities + +* Example: Example<17>. + +pipes — Interface to shell pipelines + +* Template Objects:: + +resource — Resource usage information + +* Resource Limits:: +* Resource Usage:: + +syslog — Unix syslog library routines + +* Examples: Examples<32>. + +Examples + +* Simple example:: + +Superseded Modules + +* optparse — Parser for command line options:: +* imp — Access the import internals:: + +optparse — Parser for command line options + +* Background:: +* Tutorial: Tutorial<2>. +* Reference Guide:: +* Option Callbacks:: +* Extending optparse:: + +Background + +* Terminology:: +* What are options for?:: +* What are positional arguments for?:: + +Tutorial + +* Understanding option actions:: +* The store action:: +* Handling boolean (flag) options: Handling boolean flag options. +* Other actions:: +* Default values:: +* Generating help:: +* Printing a version string:: +* How optparse handles errors:: +* Putting it all together:: + +Generating help + +* Grouping Options:: + +Reference Guide + +* Creating the parser:: +* Populating the parser:: +* Defining options:: +* Option attributes:: +* Standard option actions:: +* Standard option types:: +* Parsing arguments: Parsing arguments<2>. +* Querying and manipulating your option parser:: +* Conflicts between options:: +* Cleanup: Cleanup<2>. +* Other methods:: + +Option Callbacks + +* Defining a callback option:: +* How callbacks are called:: +* Raising errors in a callback:: +* Callback example 1; trivial callback: Callback example 1 trivial callback. +* Callback example 2; check option order: Callback example 2 check option order. +* Callback example 3; check option order (generalized): Callback example 3 check option order generalized. +* Callback example 4; check arbitrary condition: Callback example 4 check arbitrary condition. +* Callback example 5; fixed arguments: Callback example 5 fixed arguments. +* Callback example 6; variable arguments: Callback example 6 variable arguments. + +Extending optparse + +* Adding new types:: +* Adding new actions:: + +imp — Access the import internals + +* Examples: Examples<33>. + +Undocumented Modules + +* Platform specific modules:: + +Extending and Embedding the Python Interpreter + +* Recommended third party tools:: +* Creating extensions without third party tools:: +* Embedding the CPython runtime in a larger application:: + +Creating extensions without third party tools + +* Extending Python with C or C++:: +* Defining Extension Types; Tutorial: Defining Extension Types Tutorial. +* Defining Extension Types; Assorted Topics: Defining Extension Types Assorted Topics. +* Building C and C++ Extensions:: +* Building C and C++ Extensions on Windows:: + +Extending Python with C or C++ + +* A Simple Example:: +* Intermezzo; Errors and Exceptions: Intermezzo Errors and Exceptions. +* Back to the Example:: +* The Module’s Method Table and Initialization Function:: +* Compilation and Linkage:: +* Calling Python Functions from C:: +* Extracting Parameters in Extension Functions:: +* Keyword Parameters for Extension Functions:: +* Building Arbitrary Values:: +* Reference Counts:: +* Writing Extensions in C++:: +* Providing a C API for an Extension Module:: + +Reference Counts + +* Reference Counting in Python:: +* Ownership Rules:: +* Thin Ice:: +* NULL Pointers:: + +Defining Extension Types: Tutorial + +* The Basics:: +* Adding data and methods to the Basic example:: +* Providing finer control over data attributes:: +* Supporting cyclic garbage collection:: +* Subclassing other types:: + +Defining Extension Types: Assorted Topics + +* Finalization and De-allocation:: +* Object Presentation:: +* Attribute Management:: +* Object Comparison:: +* Abstract Protocol Support:: +* Weak Reference Support:: +* More Suggestions:: + +Attribute Management + +* Generic Attribute Management:: +* Type-specific Attribute Management:: + +Building C and C++ Extensions + +* Building C and C++ Extensions with distutils:: +* Distributing your extension modules:: + +Building C and C++ Extensions on Windows + +* A Cookbook Approach:: +* Differences Between Unix and Windows:: +* Using DLLs in Practice:: + +Embedding the CPython runtime in a larger application + +* Embedding Python in Another Application:: + +Embedding Python in Another Application + +* Very High Level Embedding:: +* Beyond Very High Level Embedding; An overview: Beyond Very High Level Embedding An overview. +* Pure Embedding:: +* Extending Embedded Python:: +* Embedding Python in C++:: +* Compiling and Linking under Unix-like systems:: + +Python/C API Reference Manual + +* Introduction: Introduction<13>. +* Stable Application Binary Interface:: +* The Very High Level Layer:: +* Reference Counting:: +* Exception Handling:: +* Utilities: Utilities<2>. +* Abstract Objects Layer:: +* Concrete Objects Layer:: +* Initialization, Finalization, and Threads: Initialization Finalization and Threads. +* Python Initialization Configuration:: +* Memory Management:: +* Object Implementation Support:: +* API and ABI Versioning:: + +Introduction + +* Coding standards:: +* Include Files:: +* Useful macros:: +* Objects, Types and Reference Counts: Objects Types and Reference Counts. +* Exceptions: Exceptions<18>. +* Embedding Python: Embedding Python<2>. +* Debugging Builds:: + +Objects, Types and Reference Counts + +* Reference Counts: Reference Counts<2>. +* Types:: + +Reference Counts + +* Reference Count Details:: + +Exception Handling + +* Printing and clearing:: +* Raising exceptions:: +* Issuing warnings:: +* Querying the error indicator:: +* Signal Handling: Signal Handling<2>. +* Exception Classes:: +* Exception Objects:: +* Unicode Exception Objects:: +* Recursion Control:: +* Standard Exceptions:: +* Standard Warning Categories:: + +Utilities + +* Operating System Utilities:: +* System Functions:: +* Process Control:: +* Importing Modules: Importing Modules<2>. +* Data marshalling support:: +* Parsing arguments and building values:: +* String conversion and formatting:: +* Reflection:: +* Codec registry and support functions:: + +Parsing arguments and building values + +* Parsing arguments: Parsing arguments<3>. +* Building values:: + +Parsing arguments + +* Strings and buffers:: +* Numbers: Numbers<2>. +* Other objects:: +* API Functions:: + +Codec registry and support functions + +* Codec lookup API:: +* Registry API for Unicode encoding error handlers:: + +Abstract Objects Layer + +* Object Protocol:: +* Number Protocol:: +* Sequence Protocol:: +* Mapping Protocol:: +* Iterator Protocol:: +* Buffer Protocol:: +* Old Buffer Protocol:: + +Buffer Protocol + +* Buffer structure:: +* Buffer request types:: +* Complex arrays:: +* Buffer-related functions:: + +Buffer request types + +* request-independent fields:: +* readonly, format: readonly format. +* shape, strides, suboffsets: shape strides suboffsets. +* contiguity requests:: +* compound requests:: + +Complex arrays + +* NumPy-style; shape and strides: NumPy-style shape and strides. +* PIL-style; shape, strides and suboffsets: PIL-style shape strides and suboffsets. + +Concrete Objects Layer + +* Fundamental Objects:: +* Numeric Objects:: +* Sequence Objects:: +* Container Objects:: +* Function Objects: Function Objects<2>. +* Other Objects:: + +Fundamental Objects + +* Type Objects: Type Objects<2>. +* The None Object:: + +Type Objects + +* Creating Heap-Allocated Types:: + +Numeric Objects + +* Integer Objects:: +* Boolean Objects:: +* Floating Point Objects:: +* Complex Number Objects:: + +Complex Number Objects + +* Complex Numbers as C Structures:: +* Complex Numbers as Python Objects:: + +Sequence Objects + +* Bytes Objects: Bytes Objects<2>. +* Byte Array Objects:: +* Unicode Objects and Codecs:: +* Tuple Objects:: +* Struct Sequence Objects:: +* List Objects:: + +Byte Array Objects + +* Type check macros:: +* Direct API functions:: +* Macros:: + +Unicode Objects and Codecs + +* Unicode Objects:: +* Built-in Codecs:: +* Methods and Slot Functions:: + +Unicode Objects + +* Unicode Type:: +* Unicode Character Properties:: +* Creating and accessing Unicode strings:: +* Deprecated Py_UNICODE APIs:: +* Locale Encoding:: +* File System Encoding:: +* wchar_t Support:: + +Built-in Codecs + +* Generic Codecs:: +* UTF-8 Codecs:: +* UTF-32 Codecs:: +* UTF-16 Codecs:: +* UTF-7 Codecs:: +* Unicode-Escape Codecs:: +* Raw-Unicode-Escape Codecs:: +* Latin-1 Codecs:: +* ASCII Codecs:: +* Character Map Codecs:: +* MBCS codecs for Windows:: +* Methods & Slots:: + +Container Objects + +* Dictionary Objects:: +* Set Objects:: + +Function Objects + +* Function Objects: Function Objects<3>. +* Instance Method Objects:: +* Method Objects: Method Objects<2>. +* Cell Objects:: +* Code Objects: Code Objects<2>. + +Other Objects + +* File Objects:: +* Module Objects:: +* Iterator Objects:: +* Descriptor Objects:: +* Slice Objects:: +* Ellipsis Object:: +* MemoryView objects:: +* Weak Reference Objects: Weak Reference Objects<2>. +* Capsules: Capsules<2>. +* Generator Objects:: +* Coroutine Objects: Coroutine Objects<2>. +* Context Variables Objects:: +* DateTime Objects: DateTime Objects<2>. + +Module Objects + +* Initializing C modules:: +* Module lookup:: + +Initializing C modules + +* Single-phase initialization:: +* Multi-phase initialization:: +* Low-level module creation functions:: +* Support functions:: + +Initialization, Finalization, and Threads + +* Before Python Initialization:: +* Global configuration variables:: +* Initializing and finalizing the interpreter:: +* Process-wide parameters:: +* Thread State and the Global Interpreter Lock:: +* Sub-interpreter support:: +* Asynchronous Notifications:: +* Profiling and Tracing:: +* Advanced Debugger Support:: +* Thread Local Storage Support:: + +Thread State and the Global Interpreter Lock + +* Releasing the GIL from extension code:: +* Non-Python created threads:: +* Cautions about fork(): Cautions about fork. +* High-level API:: +* Low-level API:: + +Sub-interpreter support + +* Bugs and caveats:: + +Thread Local Storage Support + +* Thread Specific Storage (TSS) API: Thread Specific Storage TSS API. +* Thread Local Storage (TLS) API: Thread Local Storage TLS API. + +Thread Specific Storage (TSS) API + +* Dynamic Allocation:: +* Methods: Methods<4>. + +Python Initialization Configuration + +* PyWideStringList:: +* PyStatus:: +* PyPreConfig:: +* Preinitialization with PyPreConfig:: +* PyConfig:: +* Initialization with PyConfig:: +* Isolated Configuration:: +* Python Configuration:: +* Path Configuration:: +* Py_RunMain(): Py_RunMain. +* Multi-Phase Initialization Private Provisional API:: + +Memory Management + +* Overview: Overview<3>. +* Raw Memory Interface:: +* Memory Interface:: +* Object allocators:: +* Default Memory Allocators:: +* Customize Memory Allocators:: +* The pymalloc allocator:: +* tracemalloc C API:: +* Examples: Examples<34>. + +The pymalloc allocator + +* Customize pymalloc Arena Allocator:: + +Object Implementation Support + +* Allocating Objects on the Heap:: +* Common Object Structures:: +* Type Objects: Type Objects<3>. +* Number Object Structures:: +* Mapping Object Structures:: +* Sequence Object Structures:: +* Buffer Object Structures:: +* Async Object Structures:: +* Slot Type typedefs:: +* Examples: Examples<35>. +* Supporting Cyclic Garbage Collection:: + +Type Objects + +* Quick Reference:: +* PyTypeObject Definition:: +* PyObject Slots:: +* PyVarObject Slots:: +* PyTypeObject Slots:: +* Heap Types:: + +Quick Reference + +* “tp slots”:: +* sub-slots:: +* slot typedefs:: + +Distributing Python Modules + +* Key terms:: +* Open source licensing and collaboration:: +* Installing the tools:: +* Reading the Python Packaging User Guide:: +* How do I…?:: + +How do I…? + +* … choose a name for my project?:: +* … create and distribute binary extensions?:: + +Installing Python Modules + +* Key terms: Key terms<2>. +* Basic usage:: +* How do I …?:: +* Common installation issues:: + +How do I …? + +* … install pip in versions of Python prior to Python 3.4?: … install pip in versions of Python prior to Python 3 4?. +* … install packages just for the current user?:: +* … install scientific Python packages?:: +* … work with multiple versions of Python installed in parallel?:: + +Common installation issues + +* Installing into the system Python on Linux:: +* Pip not installed:: +* Installing binary extensions:: + +Python HOWTOs + +* Porting Python 2 Code to Python 3:: +* Porting Extension Modules to Python 3:: +* Curses Programming with Python:: +* Descriptor HowTo Guide:: +* Functional Programming HOWTO:: +* Logging HOWTO:: +* Logging Cookbook:: +* Regular Expression HOWTO:: +* Socket Programming HOWTO:: +* Sorting HOW TO:: +* Unicode HOWTO:: +* HOWTO Fetch Internet Resources Using The urllib Package:: +* Argparse Tutorial:: +* An introduction to the ipaddress module:: +* Argument Clinic How-To:: +* Instrumenting CPython with DTrace and SystemTap:: + +Porting Python 2 Code to Python 3 + +* The Short Explanation:: +* Details:: + +Details + +* Drop support for Python 2.6 and older: Drop support for Python 2 6 and older. +* Make sure you specify the proper version support in your setup.py file: Make sure you specify the proper version support in your setup py file. +* Have good test coverage:: +* Learn the differences between Python 2 & 3:: +* Update your code:: +* Prevent compatibility regressions:: +* Check which dependencies block your transition:: +* Update your setup.py file to denote Python 3 compatibility: Update your setup py file to denote Python 3 compatibility. +* Use continuous integration to stay compatible:: +* Consider using optional static type checking:: + +Update your code + +* Division:: +* Text versus binary data:: +* Use feature detection instead of version detection:: + +Curses Programming with Python + +* What is curses?:: +* Starting and ending a curses application:: +* Windows and Pads:: +* Displaying Text:: +* User Input:: +* For More Information:: + +What is curses? + +* The Python curses module:: + +Displaying Text + +* Attributes and Color:: + +Descriptor HowTo Guide + +* Abstract:: +* Definition and Introduction:: +* Descriptor Protocol:: +* Invoking Descriptors: Invoking Descriptors<2>. +* Descriptor Example:: +* Properties:: +* Functions and Methods:: +* Static Methods and Class Methods:: + +Functional Programming HOWTO + +* Introduction: Introduction<14>. +* Iterators: Iterators<2>. +* Generator expressions and list comprehensions:: +* Generators: Generators<2>. +* Built-in functions:: +* The itertools module:: +* The functools module:: +* Small functions and the lambda expression:: +* Revision History and Acknowledgements:: +* References: References<2>. + +Introduction + +* Formal provability:: +* Modularity:: +* Ease of debugging and testing:: +* Composability:: + +Iterators + +* Data Types That Support Iterators:: + +Generators + +* Passing values into a generator:: + +The itertools module + +* Creating new iterators:: +* Calling functions on elements:: +* Selecting elements:: +* Combinatoric functions:: +* Grouping elements:: + +The functools module + +* The operator module:: + +References + +* General:: +* Python-specific:: +* Python documentation:: + +Logging HOWTO + +* Basic Logging Tutorial:: +* Advanced Logging Tutorial:: +* Logging Levels: Logging Levels<2>. +* Useful Handlers:: +* Exceptions raised during logging:: +* Using arbitrary objects as messages:: +* Optimization:: + +Basic Logging Tutorial + +* When to use logging:: +* A simple example:: +* Logging to a file:: +* Logging from multiple modules:: +* Logging variable data:: +* Changing the format of displayed messages:: +* Displaying the date/time in messages:: +* Next Steps:: + +Advanced Logging Tutorial + +* Logging Flow:: +* Loggers:: +* Handlers:: +* Formatters:: +* Configuring Logging:: +* What happens if no configuration is provided:: +* Configuring Logging for a Library:: + +Logging Levels + +* Custom Levels:: + +Logging Cookbook + +* Using logging in multiple modules:: +* Logging from multiple threads:: +* Multiple handlers and formatters:: +* Logging to multiple destinations:: +* Configuration server example:: +* Dealing with handlers that block:: +* Sending and receiving logging events across a network:: +* Adding contextual information to your logging output:: +* Logging to a single file from multiple processes:: +* Using file rotation:: +* Use of alternative formatting styles:: +* Customizing LogRecord:: +* Subclassing QueueHandler - a ZeroMQ example:: +* Subclassing QueueListener - a ZeroMQ example:: +* An example dictionary-based configuration:: +* Using a rotator and namer to customize log rotation processing:: +* A more elaborate multiprocessing example:: +* Inserting a BOM into messages sent to a SysLogHandler:: +* Implementing structured logging:: +* Customizing handlers with dictConfig(): Customizing handlers with dictConfig. +* Using particular formatting styles throughout your application:: +* Configuring filters with dictConfig(): Configuring filters with dictConfig. +* Customized exception formatting:: +* Speaking logging messages:: +* Buffering logging messages and outputting them conditionally:: +* Formatting times using UTC (GMT) via configuration: Formatting times using UTC GMT via configuration. +* Using a context manager for selective logging:: +* A CLI application starter template:: +* A Qt GUI for logging:: + +Adding contextual information to your logging output + +* Using LoggerAdapters to impart contextual information:: +* Using Filters to impart contextual information:: + +Using LoggerAdapters to impart contextual information + +* Using objects other than dicts to pass contextual information:: + +Logging to a single file from multiple processes + +* Using concurrent.futures.ProcessPoolExecutor: Using concurrent futures ProcessPoolExecutor. + +Using particular formatting styles throughout your application + +* Using LogRecord factories:: +* Using custom message objects:: + +Regular Expression HOWTO + +* Introduction: Introduction<15>. +* Simple Patterns:: +* Using Regular Expressions:: +* More Pattern Power:: +* Modifying Strings:: +* Common Problems:: +* Feedback:: + +Simple Patterns + +* Matching Characters:: +* Repeating Things:: + +Using Regular Expressions + +* Compiling Regular Expressions:: +* The Backslash Plague:: +* Performing Matches:: +* Module-Level Functions: Module-Level Functions<2>. +* Compilation Flags:: + +More Pattern Power + +* More Metacharacters:: +* Grouping:: +* Non-capturing and Named Groups:: +* Lookahead Assertions:: + +Modifying Strings + +* Splitting Strings:: +* Search and Replace:: + +Common Problems + +* Use String Methods:: +* match() versus search(): match versus search. +* Greedy versus Non-Greedy:: +* Using re.VERBOSE: Using re VERBOSE. + +Socket Programming HOWTO + +* Sockets:: +* Creating a Socket:: +* Using a Socket:: +* Disconnecting:: +* Non-blocking Sockets:: + +Sockets + +* History:: + +Creating a Socket + +* IPC:: + +Using a Socket + +* Binary Data:: + +Disconnecting + +* When Sockets Die:: + +Sorting HOW TO + +* Sorting Basics:: +* Key Functions:: +* Operator Module Functions:: +* Ascending and Descending:: +* Sort Stability and Complex Sorts:: +* The Old Way Using Decorate-Sort-Undecorate:: +* The Old Way Using the cmp Parameter:: +* Odd and Ends:: + +Unicode HOWTO + +* Introduction to Unicode:: +* Python’s Unicode Support:: +* Reading and Writing Unicode Data:: +* Acknowledgements: Acknowledgements<10>. + +Introduction to Unicode + +* Definitions:: +* Encodings:: +* References: References<3>. + +Python’s Unicode Support + +* The String Type:: +* Converting to Bytes:: +* Unicode Literals in Python Source Code:: +* Unicode Properties:: +* Comparing Strings:: +* Unicode Regular Expressions:: +* References: References<4>. + +Reading and Writing Unicode Data + +* Unicode filenames:: +* Tips for Writing Unicode-aware Programs:: +* References: References<5>. + +Tips for Writing Unicode-aware Programs + +* Converting Between File Encodings:: +* Files in an Unknown Encoding:: + +HOWTO Fetch Internet Resources Using The urllib Package + +* Introduction: Introduction<16>. +* Fetching URLs:: +* Handling Exceptions: Handling Exceptions<2>. +* info and geturl:: +* Openers and Handlers:: +* Basic Authentication:: +* Proxies:: +* Sockets and Layers:: +* Footnotes:: + +Fetching URLs + +* Data:: +* Headers:: + +Handling Exceptions + +* URLError:: +* HTTPError:: +* Wrapping it Up:: + +HTTPError + +* Error Codes:: + +Wrapping it Up + +* Number 1:: +* Number 2:: + +Argparse Tutorial + +* Concepts:: +* The basics:: +* Introducing Positional arguments:: +* Introducing Optional arguments:: +* Combining Positional and Optional arguments:: +* Getting a little more advanced:: +* Conclusion:: + +Introducing Optional arguments + +* Short options:: + +Getting a little more advanced + +* Conflicting options:: + +An introduction to the ipaddress module + +* Creating Address/Network/Interface objects:: +* Inspecting Address/Network/Interface Objects:: +* Networks as lists of Addresses:: +* Comparisons: Comparisons<4>. +* Using IP Addresses with other modules:: +* Getting more detail when instance creation fails:: + +Creating Address/Network/Interface objects + +* A Note on IP Versions:: +* IP Host Addresses:: +* Defining Networks:: +* Host Interfaces:: + +Argument Clinic How-To + +* The Goals Of Argument Clinic:: +* Basic Concepts And Usage:: +* Converting Your First Function:: +* Advanced Topics:: + +Advanced Topics + +* Symbolic default values:: +* Renaming the C functions and variables generated by Argument Clinic:: +* Converting functions using PyArg_UnpackTuple:: +* Optional Groups:: +* Using real Argument Clinic converters, instead of “legacy converters”: Using real Argument Clinic converters instead of “legacy converters”. +* Py_buffer:: +* Advanced converters:: +* Parameter default values:: +* The NULL default value:: +* Expressions specified as default values:: +* Using a return converter:: +* Cloning existing functions:: +* Calling Python code:: +* Using a “self converter”:: +* Writing a custom converter:: +* Writing a custom return converter:: +* METH_O and METH_NOARGS:: +* tp_new and tp_init functions:: +* Changing and redirecting Clinic’s output:: +* The #ifdef trick:: +* Using Argument Clinic in Python files:: + +Instrumenting CPython with DTrace and SystemTap + +* Enabling the static markers:: +* Static DTrace probes:: +* Static SystemTap markers:: +* Available static markers:: +* SystemTap Tapsets:: +* Examples: Examples<36>. + +Python Frequently Asked Questions + +* General Python FAQ:: +* Programming FAQ:: +* Design and History FAQ:: +* Library and Extension FAQ:: +* Extending/Embedding FAQ:: +* Python on Windows FAQ:: +* Graphic User Interface FAQ:: +* “Why is Python Installed on my Computer?” FAQ:: + +General Python FAQ + +* General Information:: +* Python in the real world:: + +General Information + +* What is Python?:: +* What is the Python Software Foundation?:: +* Are there copyright restrictions on the use of Python?:: +* Why was Python created in the first place?:: +* What is Python good for?:: +* How does the Python version numbering scheme work?:: +* How do I obtain a copy of the Python source?:: +* How do I get documentation on Python?:: +* I’ve never programmed before. Is there a Python tutorial?: I’ve never programmed before Is there a Python tutorial?. +* Is there a newsgroup or mailing list devoted to Python?:: +* How do I get a beta test version of Python?:: +* How do I submit bug reports and patches for Python?:: +* Are there any published articles about Python that I can reference?:: +* Are there any books on Python?:: +* Where in the world is www.python.org located?: Where in the world is www python org located?. +* Why is it called Python?:: +* Do I have to like “Monty Python’s Flying Circus”?:: + +Python in the real world + +* How stable is Python?:: +* How many people are using Python?:: +* Have any significant projects been done in Python?:: +* What new developments are expected for Python in the future?:: +* Is it reasonable to propose incompatible changes to Python?:: +* Is Python a good language for beginning programmers?:: + +Programming FAQ + +* General Questions:: +* Core Language:: +* Numbers and strings:: +* Performance: Performance<4>. +* Sequences (Tuples/Lists): Sequences Tuples/Lists. +* Objects:: +* Modules: Modules<3>. + +General Questions + +* Is there a source code level debugger with breakpoints, single-stepping, etc.?: Is there a source code level debugger with breakpoints single-stepping etc ?. +* Are there tools to help find bugs or perform static analysis?:: +* How can I create a stand-alone binary from a Python script?:: +* Are there coding standards or a style guide for Python programs?:: + +Core Language + +* Why am I getting an UnboundLocalError when the variable has a value?:: +* What are the rules for local and global variables in Python?:: +* Why do lambdas defined in a loop with different values all return the same result?:: +* How do I share global variables across modules?:: +* What are the “best practices” for using import in a module?:: +* Why are default values shared between objects?:: +* How can I pass optional or keyword parameters from one function to another?:: +* What is the difference between arguments and parameters?:: +* Why did changing list ‘y’ also change list ‘x’?:: +* How do I write a function with output parameters (call by reference)?: How do I write a function with output parameters call by reference ?. +* How do you make a higher order function in Python?:: +* How do I copy an object in Python?:: +* How can I find the methods or attributes of an object?:: +* How can my code discover the name of an object?:: +* What’s up with the comma operator’s precedence?:: +* Is there an equivalent of C’s “?;” ternary operator?: Is there an equivalent of C’s “? ” ternary operator?. +* Is it possible to write obfuscated one-liners in Python?:: +* What does the slash(/) in the parameter list of a function mean?: What does the slash / in the parameter list of a function mean?. + +Numbers and strings + +* How do I specify hexadecimal and octal integers?:: +* Why does -22 // 10 return -3?:: +* How do I convert a string to a number?:: +* How do I convert a number to a string?:: +* How do I modify a string in place?:: +* How do I use strings to call functions/methods?:: +* Is there an equivalent to Perl’s chomp() for removing trailing newlines from strings?: Is there an equivalent to Perl’s chomp for removing trailing newlines from strings?. +* Is there a scanf() or sscanf() equivalent?: Is there a scanf or sscanf equivalent?. +* What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean?:: + +Performance + +* My program is too slow. How do I speed it up?: My program is too slow How do I speed it up?. +* What is the most efficient way to concatenate many strings together?:: + +Sequences (Tuples/Lists) + +* How do I convert between tuples and lists?:: +* What’s a negative index?:: +* How do I iterate over a sequence in reverse order?:: +* How do you remove duplicates from a list?:: +* How do you remove multiple items from a list:: +* How do you make an array in Python?:: +* How do I create a multidimensional list?:: +* How do I apply a method to a sequence of objects?:: +* Why does a_tuple[i] += [‘item’] raise an exception when the addition works?:: +* I want to do a complicated sort; can you do a Schwartzian Transform in Python?: I want to do a complicated sort can you do a Schwartzian Transform in Python?. +* How can I sort one list by values from another list?:: + +Objects + +* What is a class?:: +* What is a method?:: +* What is self?:: +* How do I check if an object is an instance of a given class or of a subclass of it?:: +* What is delegation?:: +* How do I call a method defined in a base class from a derived class that overrides it?:: +* How can I organize my code to make it easier to change the base class?:: +* How do I create static class data and static class methods?:: +* How can I overload constructors (or methods) in Python?: How can I overload constructors or methods in Python?. +* I try to use __spam and I get an error about _SomeClassName__spam.: I try to use __spam and I get an error about _SomeClassName__spam. +* My class defines __del__ but it is not called when I delete the object.: My class defines __del__ but it is not called when I delete the object. +* How do I get a list of all instances of a given class?:: +* Why does the result of id() appear to be not unique?: Why does the result of id appear to be not unique?. + +Modules + +* How do I create a .pyc file?: How do I create a pyc file?. +* How do I find the current module name?:: +* How can I have modules that mutually import each other?:: +* __import__(‘x.y.z’) returns ; how do I get z?: __import__ ‘x y z’ returns ; how do I get z?. +* When I edit an imported module and reimport it, the changes don’t show up. Why does this happen?: When I edit an imported module and reimport it the changes don’t show up Why does this happen?. + +Design and History FAQ + +* Why does Python use indentation for grouping of statements?:: +* Why am I getting strange results with simple arithmetic operations?:: +* Why are floating-point calculations so inaccurate?:: +* Why are Python strings immutable?:: +* Why must ‘self’ be used explicitly in method definitions and calls?:: +* Why can’t I use an assignment in an expression?:: +* Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))?: Why does Python use methods for some functionality e g list index but functions for other e g len list ?. +* Why is join() a string method instead of a list or tuple method?: Why is join a string method instead of a list or tuple method?. +* How fast are exceptions?:: +* Why isn’t there a switch or case statement in Python?:: +* Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?:: +* Why can’t lambda expressions contain statements?:: +* Can Python be compiled to machine code, C or some other language?: Can Python be compiled to machine code C or some other language?. +* How does Python manage memory?:: +* Why doesn’t CPython use a more traditional garbage collection scheme?:: +* Why isn’t all memory freed when CPython exits?:: +* Why are there separate tuple and list data types?:: +* How are lists implemented in CPython?:: +* How are dictionaries implemented in CPython?:: +* Why must dictionary keys be immutable?:: +* Why doesn’t list.sort() return the sorted list?: Why doesn’t list sort return the sorted list?. +* How do you specify and enforce an interface spec in Python?:: +* Why is there no goto?:: +* Why can’t raw strings (r-strings) end with a backslash?: Why can’t raw strings r-strings end with a backslash?. +* Why doesn’t Python have a “with” statement for attribute assignments?:: +* Why are colons required for the if/while/def/class statements?:: +* Why does Python allow commas at the end of lists and tuples?:: + +Library and Extension FAQ + +* General Library Questions:: +* Common tasks:: +* Threads:: +* Input and Output: Input and Output<2>. +* Network/Internet Programming:: +* Databases:: +* Mathematics and Numerics:: + +General Library Questions + +* How do I find a module or application to perform task X?:: +* Where is the math.py (socket.py, regex.py, etc.) source file?: Where is the math py socket py regex py etc source file?. +* How do I make a Python script executable on Unix?:: +* Is there a curses/termcap package for Python?:: +* Is there an equivalent to C’s onexit() in Python?: Is there an equivalent to C’s onexit in Python?. +* Why don’t my signal handlers work?:: + +Common tasks + +* How do I test a Python program or component?:: +* How do I create documentation from doc strings?:: +* How do I get a single keypress at a time?:: + +Threads + +* How do I program using threads?:: +* None of my threads seem to run; why?: None of my threads seem to run why?. +* How do I parcel out work among a bunch of worker threads?:: +* What kinds of global value mutation are thread-safe?:: +* Can’t we get rid of the Global Interpreter Lock?:: + +Input and Output + +* How do I delete a file? (And other file questions…): How do I delete a file? And other file questions…. +* How do I copy a file?:: +* How do I read (or write) binary data?: How do I read or write binary data?. +* I can’t seem to use os.read() on a pipe created with os.popen(); why?: I can’t seem to use os read on a pipe created with os popen ; why?. +* How do I access the serial (RS232) port?: How do I access the serial RS232 port?. +* Why doesn’t closing sys.stdout (stdin, stderr) really close it?: Why doesn’t closing sys stdout stdin stderr really close it?. + +Network/Internet Programming + +* What WWW tools are there for Python?:: +* How can I mimic CGI form submission (METHOD=POST)?: How can I mimic CGI form submission METHOD=POST ?. +* What module should I use to help with generating HTML?:: +* How do I send mail from a Python script?:: +* How do I avoid blocking in the connect() method of a socket?: How do I avoid blocking in the connect method of a socket?. + +Databases + +* Are there any interfaces to database packages in Python?:: +* How do you implement persistent objects in Python?:: + +Mathematics and Numerics + +* How do I generate random numbers in Python?:: + +Extending/Embedding FAQ + +* Can I create my own functions in C?:: +* Can I create my own functions in C++?:: +* Writing C is hard; are there any alternatives?:: +* How can I execute arbitrary Python statements from C?:: +* How can I evaluate an arbitrary Python expression from C?:: +* How do I extract C values from a Python object?:: +* How do I use Py_BuildValue() to create a tuple of arbitrary length?: How do I use Py_BuildValue to create a tuple of arbitrary length?. +* How do I call an object’s method from C?:: +* How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?: How do I catch the output from PyErr_Print or anything that prints to stdout/stderr ?. +* How do I access a module written in Python from C?:: +* How do I interface to C++ objects from Python?:: +* I added a module using the Setup file and the make fails; why?:: +* How do I debug an extension?:: +* I want to compile a Python module on my Linux system, but some files are missing. Why?: I want to compile a Python module on my Linux system but some files are missing Why?. +* How do I tell “incomplete input” from “invalid input”?:: +* How do I find undefined g++ symbols __builtin_new or __pure_virtual?:: +* Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?: Can I create an object class with some methods implemented in C and others in Python e g through inheritance ?. + +Python on Windows FAQ + +* How do I run a Python program under Windows?:: +* How do I make Python scripts executable?:: +* Why does Python sometimes take so long to start?:: +* How do I make an executable from a Python script?:: +* Is a *.pyd file the same as a DLL?: Is a * pyd file the same as a DLL?. +* How can I embed Python into a Windows application?:: +* How do I keep editors from inserting tabs into my Python source?:: +* How do I check for a keypress without blocking?:: + +Graphic User Interface FAQ + +* General GUI Questions:: +* What platform-independent GUI toolkits exist for Python?:: +* What platform-specific GUI toolkits exist for Python?:: +* Tkinter questions:: + +What platform-independent GUI toolkits exist for Python? + +* Tkinter:: +* wxWidgets:: +* Qt:: +* Gtk+:: +* Kivy:: +* FLTK:: +* OpenGL:: + +Tkinter questions + +* How do I freeze Tkinter applications?:: +* Can I have Tk events handled while waiting for I/O?:: +* I can’t get key bindings to work in Tkinter; why?: I can’t get key bindings to work in Tkinter why?. + +“Why is Python Installed on my Computer?” FAQ + +* What is Python?: What is Python?<2>. +* Why is Python installed on my machine?:: +* Can I delete Python?:: + +About these documents + +* Contributors to the Python Documentation:: + +Dealing with Bugs + +* Documentation bugs:: +* Using the Python issue tracker:: +* Getting started contributing to Python yourself:: + +History and License + +* History of the software:: +* Terms and conditions for accessing or otherwise using Python:: +* Licenses and Acknowledgements for Incorporated Software:: + +Terms and conditions for accessing or otherwise using Python + +* PSF LICENSE AGREEMENT FOR PYTHON 3.8.9: PSF LICENSE AGREEMENT FOR PYTHON 3 8 9. +* BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0: BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 0. +* CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1: CNRI LICENSE AGREEMENT FOR PYTHON 1 6 1. +* CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2: CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 2. +* ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3.8.9 DOCUMENTATION: ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3 8 9 DOCUMENTATION. + +Licenses and Acknowledgements for Incorporated Software + +* Mersenne Twister:: +* Sockets: Sockets<2>. +* Asynchronous socket services:: +* Cookie management:: +* Execution tracing:: +* UUencode and UUdecode functions:: +* XML Remote Procedure Calls:: +* test_epoll:: +* Select kqueue:: +* SipHash24:: +* strtod and dtoa:: +* OpenSSL:: +* expat:: +* libffi:: +* zlib: zlib<3>. +* cfuhash:: +* libmpdec:: +* W3C C14N test suite:: + +Distributing Python Modules (Legacy version) + +* An Introduction to Distutils:: +* Writing the Setup Script:: +* Writing the Setup Configuration File:: +* Creating a Source Distribution:: +* Creating Built Distributions:: +* Distutils Examples:: +* Extending Distutils:: +* Command Reference:: +* API Reference:: + +An Introduction to Distutils + +* Concepts & Terminology:: +* A Simple Example: A Simple Example<2>. +* General Python terminology:: +* Distutils-specific terminology:: + +Writing the Setup Script + +* Listing whole packages:: +* Listing individual modules:: +* Describing extension modules:: +* Relationships between Distributions and Packages:: +* Installing Scripts:: +* Installing Package Data:: +* Installing Additional Files:: +* Additional meta-data:: +* Debugging the setup script:: + +Describing extension modules + +* Extension names and packages:: +* Extension source files:: +* Preprocessor options:: +* Library options:: +* Other options:: + +Creating a Source Distribution + +* Specifying the files to distribute:: +* Manifest-related options:: + +Creating Built Distributions + +* Creating RPM packages:: +* Creating Windows Installers:: +* Cross-compiling on Windows:: +* Vista User Access Control (UAC): Vista User Access Control UAC. + +Cross-compiling on Windows + +* The Postinstallation script:: + +Distutils Examples + +* Pure Python distribution (by module): Pure Python distribution by module. +* Pure Python distribution (by package): Pure Python distribution by package. +* Single extension module:: +* Checking a package:: +* Reading the metadata:: + +Extending Distutils + +* Integrating new commands:: +* Adding new distribution types:: + +Command Reference + +* Installing modules; the install command family: Installing modules the install command family. +* Creating a source distribution; the sdist command: Creating a source distribution the sdist command. + +Installing modules: the install command family + +* install_data:: +* install_scripts:: + +API Reference + +* distutils.core — Core Distutils functionality: distutils core — Core Distutils functionality. +* distutils.ccompiler — CCompiler base class: distutils ccompiler — CCompiler base class. +* distutils.unixccompiler — Unix C Compiler: distutils unixccompiler — Unix C Compiler. +* distutils.msvccompiler — Microsoft Compiler: distutils msvccompiler — Microsoft Compiler. +* distutils.bcppcompiler — Borland Compiler: distutils bcppcompiler — Borland Compiler. +* distutils.cygwincompiler — Cygwin Compiler: distutils cygwincompiler — Cygwin Compiler. +* distutils.archive_util — Archiving utilities: distutils archive_util — Archiving utilities. +* distutils.dep_util — Dependency checking: distutils dep_util — Dependency checking. +* distutils.dir_util — Directory tree operations: distutils dir_util — Directory tree operations. +* distutils.file_util — Single file operations: distutils file_util — Single file operations. +* distutils.util — Miscellaneous other utility functions: distutils util — Miscellaneous other utility functions. +* distutils.dist — The Distribution class: distutils dist — The Distribution class. +* distutils.extension — The Extension class: distutils extension — The Extension class. +* distutils.debug — Distutils debug mode: distutils debug — Distutils debug mode. +* distutils.errors — Distutils exceptions: distutils errors — Distutils exceptions. +* distutils.fancy_getopt — Wrapper around the standard getopt module: distutils fancy_getopt — Wrapper around the standard getopt module. +* distutils.filelist — The FileList class: distutils filelist — The FileList class. +* distutils.log — Simple PEP 282-style logging: distutils log — Simple PEP 282-style logging. +* distutils.spawn — Spawn a sub-process: distutils spawn — Spawn a sub-process. +* distutils.sysconfig — System configuration information: distutils sysconfig — System configuration information. +* distutils.text_file — The TextFile class: distutils text_file — The TextFile class. +* distutils.version — Version number classes: distutils version — Version number classes. +* distutils.cmd — Abstract base class for Distutils commands: distutils cmd — Abstract base class for Distutils commands. +* Creating a new Distutils command:: +* distutils.command — Individual Distutils commands: distutils command — Individual Distutils commands. +* distutils.command.bdist — Build a binary installer: distutils command bdist — Build a binary installer. +* distutils.command.bdist_packager — Abstract base class for packagers: distutils command bdist_packager — Abstract base class for packagers. +* distutils.command.bdist_dumb — Build a “dumb” installer: distutils command bdist_dumb — Build a “dumb” installer. +* distutils.command.bdist_msi — Build a Microsoft Installer binary package: distutils command bdist_msi — Build a Microsoft Installer binary package. +* distutils.command.bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM: distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM. +* distutils.command.bdist_wininst — Build a Windows installer: distutils command bdist_wininst — Build a Windows installer. +* distutils.command.sdist — Build a source distribution: distutils command sdist — Build a source distribution. +* distutils.command.build — Build all files of a package: distutils command build — Build all files of a package. +* distutils.command.build_clib — Build any C libraries in a package: distutils command build_clib — Build any C libraries in a package. +* distutils.command.build_ext — Build any extensions in a package: distutils command build_ext — Build any extensions in a package. +* distutils.command.build_py — Build the .py/.pyc files of a package: distutils command build_py — Build the py/ pyc files of a package. +* distutils.command.build_scripts — Build the scripts of a package: distutils command build_scripts — Build the scripts of a package. +* distutils.command.clean — Clean a package build area: distutils command clean — Clean a package build area. +* distutils.command.config — Perform package configuration: distutils command config — Perform package configuration. +* distutils.command.install — Install a package: distutils command install — Install a package. +* distutils.command.install_data — Install data files from a package: distutils command install_data — Install data files from a package. +* distutils.command.install_headers — Install C/C++ header files from a package: distutils command install_headers — Install C/C++ header files from a package. +* distutils.command.install_lib — Install library files from a package: distutils command install_lib — Install library files from a package. +* distutils.command.install_scripts — Install script files from a package: distutils command install_scripts — Install script files from a package. +* distutils.command.register — Register a module with the Python Package Index: distutils command register — Register a module with the Python Package Index. +* distutils.command.check — Check the meta-data of a package: distutils command check — Check the meta-data of a package. + +Installing Python Modules (Legacy version) + +* Introduction: Introduction<17>. +* Standard Build and Install:: +* Alternate Installation:: +* Custom Installation:: +* Distutils Configuration Files:: +* Building Extensions; Tips and Tricks: Building Extensions Tips and Tricks. + +Introduction + +* Distutils based source distributions:: + +Standard Build and Install + +* Platform variations:: +* Splitting the job up:: +* How building works:: +* How installation works:: + +Alternate Installation + +* Alternate installation; the user scheme: Alternate installation the user scheme. +* Alternate installation; the home scheme: Alternate installation the home scheme. +* Alternate installation; Unix (the prefix scheme): Alternate installation Unix the prefix scheme. +* Alternate installation; Windows (the prefix scheme): Alternate installation Windows the prefix scheme. + +Custom Installation + +* Modifying Python’s Search Path:: + +Distutils Configuration Files + +* Location and names of config files:: +* Syntax of config files:: + +Building Extensions: Tips and Tricks + +* Tweaking compiler/linker flags:: +* Using non-Microsoft compilers on Windows:: + +Using non-Microsoft compilers on Windows + +* Borland/CodeGear C++:: +* GNU C / Cygwin / MinGW:: + +GNU C / Cygwin / MinGW + +* Older Versions of Python and MinGW:: + + + +File: python.info, Node: What’s New in Python, Next: The Python Tutorial, Prev: Top, Up: Top + +1 What’s New in Python +********************** + +The “What’s New in Python” series of essays takes tours through the most +important changes between major Python versions. They are a “must read” +for anyone wishing to stay up-to-date after a new release. + +* Menu: + +* What’s New In Python 3.8: What’s New In Python 3 8. +* What’s New In Python 3.7: What’s New In Python 3 7. +* What’s New In Python 3.6: What’s New In Python 3 6. +* What’s New In Python 3.5: What’s New In Python 3 5. +* What’s New In Python 3.4: What’s New In Python 3 4. +* What’s New In Python 3.3: What’s New In Python 3 3. +* What’s New In Python 3.2: What’s New In Python 3 2. +* What’s New In Python 3.1: What’s New In Python 3 1. +* What’s New In Python 3.0: What’s New In Python 3 0. +* What’s New in Python 2.7: What’s New in Python 2 7. +* What’s New in Python 2.6: What’s New in Python 2 6. +* What’s New in Python 2.5: What’s New in Python 2 5. +* What’s New in Python 2.4: What’s New in Python 2 4. +* What’s New in Python 2.3: What’s New in Python 2 3. +* What’s New in Python 2.2: What’s New in Python 2 2. +* What’s New in Python 2.1: What’s New in Python 2 1. +* What’s New in Python 2.0: What’s New in Python 2 0. +* Changelog:: + + +File: python.info, Node: What’s New In Python 3 8, Next: What’s New In Python 3 7, Up: What’s New in Python + +1.1 What’s New In Python 3.8 +============================ + + +Editor: Raymond Hettinger + +This article explains the new features in Python 3.8, compared to 3.7. +For full details, see the *note changelog: 14c. + +Python 3.8 was released on October 14th, 2019. + +* Menu: + +* Summary – Release highlights:: +* New Features:: +* Other Language Changes:: +* New Modules:: +* Improved Modules:: +* Optimizations:: +* Build and C API Changes:: +* Deprecated:: +* API and Feature Removals:: +* Porting to Python 3.8: Porting to Python 3 8. +* Notable changes in Python 3.8.1: Notable changes in Python 3 8 1. +* Notable changes in Python 3.8.2: Notable changes in Python 3 8 2. +* Notable changes in Python 3.8.3: Notable changes in Python 3 8 3. +* Notable changes in Python 3.8.8: Notable changes in Python 3 8 8. +* Notable changes in Python 3.8.9: Notable changes in Python 3 8 9. + + +File: python.info, Node: Summary – Release highlights, Next: New Features, Up: What’s New In Python 3 8 + +1.1.1 Summary – Release highlights +---------------------------------- + + +File: python.info, Node: New Features, Next: Other Language Changes, Prev: Summary – Release highlights, Up: What’s New In Python 3 8 + +1.1.2 New Features +------------------ + +* Menu: + +* Assignment expressions:: +* Positional-only parameters:: +* Parallel filesystem cache for compiled bytecode files:: +* Debug build uses the same ABI as release build:: +* f-strings support = for self-documenting expressions and debugging:: +* PEP 578; Python Runtime Audit Hooks: PEP 578 Python Runtime Audit Hooks. +* PEP 587; Python Initialization Configuration: PEP 587 Python Initialization Configuration. +* Vectorcall; a fast calling protocol for CPython: Vectorcall a fast calling protocol for CPython. +* Pickle protocol 5 with out-of-band data buffers:: + + +File: python.info, Node: Assignment expressions, Next: Positional-only parameters, Up: New Features + +1.1.2.1 Assignment expressions +.............................. + +There is new syntax ‘:=’ that assigns values to variables as part of a +larger expression. It is affectionately known as “the walrus operator” +due to its resemblance to the eyes and tusks of a walrus(1). + +In this example, the assignment expression helps avoid calling *note +len(): 150. twice: + + if (n := len(a)) > 10: + print(f"List is too long ({n} elements, expected <= 10)") + +A similar benefit arises during regular expression matching where match +objects are needed twice, once to test whether a match occurred and +another to extract a subgroup: + + discount = 0.0 + if (mo := re.search(r'(\d+)% discount', advertisement)): + discount = float(mo.group(1)) / 100.0 + +The operator is also useful with while-loops that compute a value to +test loop termination and then need that same value again in the body of +the loop: + + # Loop over fixed length blocks + while (block := f.read(256)) != '': + process(block) + +Another motivating use case arises in list comprehensions where a value +computed in a filtering condition is also needed in the expression body: + + [clean_name.title() for name in names + if (clean_name := normalize('NFC', name)) in allowed_names] + +Try to limit use of the walrus operator to clean cases that reduce +complexity and improve readability. + +See PEP 572(2) for a full description. + +(Contributed by Emily Morehouse in bpo-35224(3).) + + ---------- Footnotes ---------- + + (1) +https://en.wikipedia.org/wiki/Walrus#/media/File:Pacific_Walrus_-_Bull_(8247646168).jpg + + (2) https://www.python.org/dev/peps/pep-0572 + + (3) https://bugs.python.org/issue35224 + + +File: python.info, Node: Positional-only parameters, Next: Parallel filesystem cache for compiled bytecode files, Prev: Assignment expressions, Up: New Features + +1.1.2.2 Positional-only parameters +.................................. + +There is a new function parameter syntax ‘/’ to indicate that some +function parameters must be specified positionally and cannot be used as +keyword arguments. This is the same notation shown by ‘help()’ for C +functions annotated with Larry Hastings’ Argument Clinic(1) tool. + +In the following example, parameters `a' and `b' are positional-only, +while `c' or `d' can be positional or keyword, and `e' or `f' are +required to be keywords: + + def f(a, b, /, c, d, *, e, f): + print(a, b, c, d, e, f) + +The following is a valid call: + + f(10, 20, 30, d=40, e=50, f=60) + +However, these are invalid calls: + + f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument + f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument + +One use case for this notation is that it allows pure Python functions +to fully emulate behaviors of existing C coded functions. For example, +the built-in *note divmod(): 152. function does not accept keyword +arguments: + + def divmod(a, b, /): + "Emulate the built in divmod() function" + return (a // b, a % b) + +Another use case is to preclude keyword arguments when the parameter +name is not helpful. For example, the builtin *note len(): 150. +function has the signature ‘len(obj, /)’. This precludes awkward calls +such as: + + len(obj='hello') # The "obj" keyword argument impairs readability + +A further benefit of marking a parameter as positional-only is that it +allows the parameter name to be changed in the future without risk of +breaking client code. For example, in the *note statistics: f5. module, +the parameter name `dist' may be changed in the future. This was made +possible with the following function specification: + + def quantiles(dist, /, *, n=4, method='exclusive') + ... + +Since the parameters to the left of ‘/’ are not exposed as possible +keywords, the parameters names remain available for use in ‘**kwargs’: + + >>> def f(a, b, /, **kwargs): + ... print(a, b, kwargs) + ... + >>> f(10, 20, a=1, b=2, c=3) # a and b are used in two ways + 10 20 {'a': 1, 'b': 2, 'c': 3} + +This greatly simplifies the implementation of functions and methods that +need to accept arbitrary keyword arguments. For example, here is an +excerpt from code in the *note collections: 1e. module: + + class Counter(dict): + + def __init__(self, iterable=None, /, **kwds): + # Note "iterable" is a possible keyword argument + +See PEP 570(2) for a full description. + +(Contributed by Pablo Galindo in bpo-36540(3).) + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/howto/clinic.html + + (2) https://www.python.org/dev/peps/pep-0570 + + (3) https://bugs.python.org/issue36540 + + +File: python.info, Node: Parallel filesystem cache for compiled bytecode files, Next: Debug build uses the same ABI as release build, Prev: Positional-only parameters, Up: New Features + +1.1.2.3 Parallel filesystem cache for compiled bytecode files +............................................................. + +The new *note PYTHONPYCACHEPREFIX: 154. setting (also available as *note +-X: 155. ‘pycache_prefix’) configures the implicit bytecode cache to use +a separate parallel filesystem tree, rather than the default +‘__pycache__’ subdirectories within each source directory. + +The location of the cache is reported in *note sys.pycache_prefix: 156. +(*note None: 157. indicates the default location in ‘__pycache__’ +subdirectories). + +(Contributed by Carl Meyer in bpo-33499(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33499 + + +File: python.info, Node: Debug build uses the same ABI as release build, Next: f-strings support = for self-documenting expressions and debugging, Prev: Parallel filesystem cache for compiled bytecode files, Up: New Features + +1.1.2.4 Debug build uses the same ABI as release build +...................................................... + +Python now uses the same ABI whether it’s built in release or debug +mode. On Unix, when Python is built in debug mode, it is now possible +to load C extensions built in release mode and C extensions built using +the stable ABI. + +Release builds and debug builds are now ABI compatible: defining the +‘Py_DEBUG’ macro no longer implies the ‘Py_TRACE_REFS’ macro, which +introduces the only ABI incompatibility. The ‘Py_TRACE_REFS’ macro, +which adds the ‘sys.getobjects()’ function and the *note PYTHONDUMPREFS: +159. environment variable, can be set using the new ‘./configure +--with-trace-refs’ build option. (Contributed by Victor Stinner in +bpo-36465(1).) + +On Unix, C extensions are no longer linked to libpython except on +Android and Cygwin. It is now possible for a statically linked Python +to load a C extension built using a shared library Python. (Contributed +by Victor Stinner in bpo-21536(2).) + +On Unix, when Python is built in debug mode, import now also looks for C +extensions compiled in release mode and for C extensions compiled with +the stable ABI. (Contributed by Victor Stinner in bpo-36722(3).) + +To embed Python into an application, a new ‘--embed’ option must be +passed to ‘python3-config --libs --embed’ to get ‘-lpython3.8’ (link the +application to libpython). To support both 3.8 and older, try +‘python3-config --libs --embed’ first and fallback to ‘python3-config +--libs’ (without ‘--embed’) if the previous command fails. + +Add a pkg-config ‘python-3.8-embed’ module to embed Python into an +application: ‘pkg-config python-3.8-embed --libs’ includes +‘-lpython3.8’. To support both 3.8 and older, try ‘pkg-config +python-X.Y-embed --libs’ first and fallback to ‘pkg-config python-X.Y +--libs’ (without ‘--embed’) if the previous command fails (replace ‘X.Y’ +with the Python version). + +On the other hand, ‘pkg-config python3.8 --libs’ no longer contains +‘-lpython3.8’. C extensions must not be linked to libpython (except on +Android and Cygwin, whose cases are handled by the script); this change +is backward incompatible on purpose. (Contributed by Victor Stinner in +bpo-36721(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36465 + + (2) https://bugs.python.org/issue21536 + + (3) https://bugs.python.org/issue36722 + + (4) https://bugs.python.org/issue36721 + + +File: python.info, Node: f-strings support = for self-documenting expressions and debugging, Next: PEP 578 Python Runtime Audit Hooks, Prev: Debug build uses the same ABI as release build, Up: New Features + +1.1.2.5 f-strings support ‘=’ for self-documenting expressions and debugging +............................................................................ + +Added an ‘=’ specifier to *note f-string: 15b.s. An f-string such as +‘f'{expr=}'’ will expand to the text of the expression, an equal sign, +then the representation of the evaluated expression. For example: + + >>> user = 'eric_idle' + >>> member_since = date(1975, 7, 31) + >>> f'{user=} {member_since=}' + "user='eric_idle' member_since=datetime.date(1975, 7, 31)" + +The usual *note f-string format specifiers: 15c. allow more control over +how the result of the expression is displayed: + + >>> delta = date.today() - member_since + >>> f'{user=!s} {delta.days=:,d}' + 'user=eric_idle delta.days=16,075' + +The ‘=’ specifier will display the whole expression so that calculations +can be shown: + + >>> print(f'{theta=} {cos(radians(theta))=:.3f}') + theta=30 cos(radians(theta))=0.866 + +(Contributed by Eric V. Smith and Larry Hastings in bpo-36817(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36817 + + +File: python.info, Node: PEP 578 Python Runtime Audit Hooks, Next: PEP 587 Python Initialization Configuration, Prev: f-strings support = for self-documenting expressions and debugging, Up: New Features + +1.1.2.6 PEP 578: Python Runtime Audit Hooks +........................................... + +The PEP adds an Audit Hook and Verified Open Hook. Both are available +from Python and native code, allowing applications and frameworks +written in pure Python code to take advantage of extra notifications, +while also allowing embedders or system administrators to deploy builds +of Python where auditing is always enabled. + +See PEP 578(1) for full details. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0578 + + +File: python.info, Node: PEP 587 Python Initialization Configuration, Next: Vectorcall a fast calling protocol for CPython, Prev: PEP 578 Python Runtime Audit Hooks, Up: New Features + +1.1.2.7 PEP 587: Python Initialization Configuration +.................................................... + +The PEP 587(1) adds a new C API to configure the Python Initialization +providing finer control on the whole configuration and better error +reporting. + +New structures: + + * *note PyConfig: 15f. + + * *note PyPreConfig: 160. + + * *note PyStatus: 161. + + * *note PyWideStringList: 162. + +New functions: + + * *note PyConfig_Clear(): 163. + + * *note PyConfig_InitIsolatedConfig(): 164. + + * *note PyConfig_InitPythonConfig(): 165. + + * *note PyConfig_Read(): 166. + + * *note PyConfig_SetArgv(): 167. + + * *note PyConfig_SetBytesArgv(): 168. + + * *note PyConfig_SetBytesString(): 169. + + * *note PyConfig_SetString(): 16a. + + * *note PyPreConfig_InitIsolatedConfig(): 16b. + + * *note PyPreConfig_InitPythonConfig(): 16c. + + * *note PyStatus_Error(): 16d. + + * *note PyStatus_Exception(): 16e. + + * *note PyStatus_Exit(): 16f. + + * *note PyStatus_IsError(): 170. + + * *note PyStatus_IsExit(): 171. + + * *note PyStatus_NoMemory(): 172. + + * *note PyStatus_Ok(): 173. + + * *note PyWideStringList_Append(): 174. + + * *note PyWideStringList_Insert(): 175. + + * *note Py_BytesMain(): 176. + + * *note Py_ExitStatusException(): 177. + + * *note Py_InitializeFromConfig(): 178. + + * *note Py_PreInitialize(): 179. + + * *note Py_PreInitializeFromArgs(): 17a. + + * *note Py_PreInitializeFromBytesArgs(): 17b. + + * *note Py_RunMain(): 17c. + +This PEP also adds ‘_PyRuntimeState.preconfig’ (*note PyPreConfig: 160. +type) and ‘PyInterpreterState.config’ (*note PyConfig: 15f. type) fields +to these internal structures. ‘PyInterpreterState.config’ becomes the +new reference configuration, replacing global configuration variables +and other private variables. + +See *note Python Initialization Configuration: 17d. for the +documentation. + +See PEP 587(2) for a full description. + +(Contributed by Victor Stinner in bpo-36763(3).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0587 + + (2) https://www.python.org/dev/peps/pep-0587 + + (3) https://bugs.python.org/issue36763 + + +File: python.info, Node: Vectorcall a fast calling protocol for CPython, Next: Pickle protocol 5 with out-of-band data buffers, Prev: PEP 587 Python Initialization Configuration, Up: New Features + +1.1.2.8 Vectorcall: a fast calling protocol for CPython +....................................................... + +The “vectorcall” protocol is added to the Python/C API. It is meant to +formalize existing optimizations which were already done for various +classes. Any extension type implementing a callable can use this +protocol. + +This is currently provisional. The aim is to make it fully public in +Python 3.9. + +See PEP 590(1) for a full description. + +(Contributed by Jeroen Demeyer and Mark Shannon in bpo-36974(2).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0590 + + (2) https://bugs.python.org/issue36974 + + +File: python.info, Node: Pickle protocol 5 with out-of-band data buffers, Prev: Vectorcall a fast calling protocol for CPython, Up: New Features + +1.1.2.9 Pickle protocol 5 with out-of-band data buffers +....................................................... + +When *note pickle: ca. is used to transfer large data between Python +processes in order to take advantage of multi-core or multi-machine +processing, it is important to optimize the transfer by reducing memory +copies, and possibly by applying custom techniques such as +data-dependent compression. + +The *note pickle: ca. protocol 5 introduces support for out-of-band +buffers where PEP 3118(1)-compatible data can be transmitted separately +from the main pickle stream, at the discretion of the communication +layer. + +See PEP 574(2) for a full description. + +(Contributed by Antoine Pitrou in bpo-36785(3).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3118 + + (2) https://www.python.org/dev/peps/pep-0574 + + (3) https://bugs.python.org/issue36785 + + +File: python.info, Node: Other Language Changes, Next: New Modules, Prev: New Features, Up: What’s New In Python 3 8 + +1.1.3 Other Language Changes +---------------------------- + + * A *note continue: 181. statement was illegal in the *note finally: + 182. clause due to a problem with the implementation. In Python + 3.8 this restriction was lifted. (Contributed by Serhiy Storchaka + in bpo-32489(1).) + + * The *note bool: 183, *note int: 184, and *note fractions.Fraction: + 185. types now have an *note as_integer_ratio(): 186. method like + that found in *note float: 187. and *note decimal.Decimal: 188. + This minor API extension makes it possible to write ‘numerator, + denominator = x.as_integer_ratio()’ and have it work across + multiple numeric types. (Contributed by Lisa Roach in bpo-33073(2) + and Raymond Hettinger in bpo-37819(3).) + + * Constructors of *note int: 184, *note float: 187. and *note + complex: 189. will now use the *note __index__(): 18a. special + method, if available and the corresponding method *note __int__(): + 18b, *note __float__(): 18c. or *note __complex__(): 18d. is not + available. (Contributed by Serhiy Storchaka in bpo-20092(4).) + + * Added support of ‘\N{name}’ escapes in *note regular expressions: + dd.: + + >>> notice = 'Copyright © 2019' + >>> copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})') + >>> int(copyright_year_pattern.search(notice).group(1)) + 2019 + + (Contributed by Jonathan Eunice and Serhiy Storchaka in + bpo-30688(5).) + + * Dict and dictviews are now iterable in reversed insertion order + using *note reversed(): 18e. (Contributed by Rémi Lapeyre in + bpo-33462(6).) + + * The syntax allowed for keyword names in function calls was further + restricted. In particular, ‘f((keyword)=arg)’ is no longer + allowed. It was never intended to permit more than a bare name on + the left-hand side of a keyword argument assignment term. + (Contributed by Benjamin Peterson in bpo-34641(7).) + + * Generalized iterable unpacking in *note yield: 18f. and *note + return: 190. statements no longer requires enclosing parentheses. + This brings the `yield' and `return' syntax into better agreement + with normal assignment syntax: + + >>> def parse(family): + lastname, *members = family.split() + return lastname.upper(), *members + + >>> parse('simpsons homer marge bart lisa sally') + ('SIMPSONS', 'homer', 'marge', 'bart', 'lisa', 'sally') + + (Contributed by David Cuthbert and Jordan Chapman in bpo-32117(8).) + + * When a comma is missed in code such as ‘[(10, 20) (30, 40)]’, the + compiler displays a *note SyntaxWarning: 191. with a helpful + suggestion. This improves on just having a *note TypeError: 192. + indicating that the first tuple was not callable. (Contributed by + Serhiy Storchaka in bpo-15248(9).) + + * Arithmetic operations between subclasses of *note datetime.date: + 193. or *note datetime.datetime: 194. and *note datetime.timedelta: + 195. objects now return an instance of the subclass, rather than + the base class. This also affects the return type of operations + whose implementation (directly or indirectly) uses *note + datetime.timedelta: 195. arithmetic, such as *note astimezone(): + 196. (Contributed by Paul Ganssle in bpo-32417(10).) + + * When the Python interpreter is interrupted by Ctrl-C (SIGINT) and + the resulting *note KeyboardInterrupt: 197. exception is not + caught, the Python process now exits via a SIGINT signal or with + the correct exit code such that the calling process can detect that + it died due to a Ctrl-C. Shells on POSIX and Windows use this to + properly terminate scripts in interactive sessions. (Contributed + by Google via Gregory P. Smith in bpo-1054041(11).) + + * Some advanced styles of programming require updating the *note + types.CodeType: 198. object for an existing function. Since code + objects are immutable, a new code object needs to be created, one + that is modeled on the existing code object. With 19 parameters, + this was somewhat tedious. Now, the new ‘replace()’ method makes + it possible to create a clone with a few altered parameters. + + Here’s an example that alters the *note statistics.mean(): 199. + function to prevent the `data' parameter from being used as a + keyword argument: + + >>> from statistics import mean + >>> mean(data=[10, 20, 90]) + 40 + >>> mean.__code__ = mean.__code__.replace(co_posonlyargcount=1) + >>> mean(data=[10, 20, 90]) + Traceback (most recent call last): + ... + TypeError: mean() got some positional-only arguments passed as keyword arguments: 'data' + + (Contributed by Victor Stinner in bpo-37032(12).) + + * For integers, the three-argument form of the *note pow(): 19a. + function now permits the exponent to be negative in the case where + the base is relatively prime to the modulus. It then computes a + modular inverse to the base when the exponent is ‘-1’, and a + suitable power of that inverse for other negative exponents. For + example, to compute the modular multiplicative inverse(13) of 38 + modulo 137, write: + + >>> pow(38, -1, 137) + 119 + >>> 119 * 38 % 137 + 1 + + Modular inverses arise in the solution of linear Diophantine + equations(14). For example, to find integer solutions for ‘4258𝑥 + + 147𝑦 = 369’, first rewrite as ‘4258𝑥 ≡ 369 (mod 147)’ then solve: + + >>> x = 369 * pow(4258, -1, 147) % 147 + >>> y = (4258 * x - 369) // -147 + >>> 4258 * x + 147 * y + 369 + + (Contributed by Mark Dickinson in bpo-36027(15).) + + * Dict comprehensions have been synced-up with dict literals so that + the key is computed first and the value second: + + >>> # Dict comprehension + >>> cast = {input('role? '): input('actor? ') for i in range(2)} + role? King Arthur + actor? Chapman + role? Black Knight + actor? Cleese + + >>> # Dict literal + >>> cast = {input('role? '): input('actor? ')} + role? Sir Robin + actor? Eric Idle + + The guaranteed execution order is helpful with assignment + expressions because variables assigned in the key expression will + be available in the value expression: + + >>> names = ['Martin von Löwis', 'Łukasz Langa', 'Walter Dörwald'] + >>> {(n := normalize('NFC', name)).casefold() : n for name in names} + {'martin von löwis': 'Martin von Löwis', + 'łukasz langa': 'Łukasz Langa', + 'walter dörwald': 'Walter Dörwald'} + + (Contributed by Jörn Heissler in bpo-35224(16).) + + * The *note object.__reduce__(): 19b. method can now return a tuple + from two to six elements long. Formerly, five was the limit. The + new, optional sixth element is a callable with a ‘(obj, state)’ + signature. This allows the direct control over the state-updating + behavior of a specific object. If not `None', this callable will + have priority over the object’s *note __setstate__(): 19c. method. + (Contributed by Pierre Glaser and Olivier Grisel in bpo-35900(17).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32489 + + (2) https://bugs.python.org/issue33073 + + (3) https://bugs.python.org/issue37819 + + (4) https://bugs.python.org/issue20092 + + (5) https://bugs.python.org/issue30688 + + (6) https://bugs.python.org/issue33462 + + (7) https://bugs.python.org/issue34641 + + (8) https://bugs.python.org/issue32117 + + (9) https://bugs.python.org/issue15248 + + (10) https://bugs.python.org/issue32417 + + (11) https://bugs.python.org/issue1054041 + + (12) https://bugs.python.org/issue37032 + + (13) https://en.wikipedia.org/wiki/Modular_multiplicative_inverse + + (14) https://en.wikipedia.org/wiki/Diophantine_equation + + (15) https://bugs.python.org/issue36027 + + (16) https://bugs.python.org/issue35224 + + (17) https://bugs.python.org/issue35900 + + +File: python.info, Node: New Modules, Next: Improved Modules, Prev: Other Language Changes, Up: What’s New In Python 3 8 + +1.1.4 New Modules +----------------- + + * The new ‘importlib.metadata’ module provides (provisional) support + for reading metadata from third-party packages. For example, it + can extract an installed package’s version number, list of entry + points, and more: + + >>> # Note following example requires that the popular "requests" + >>> # package has been installed. + >>> + >>> from importlib.metadata import version, requires, files + >>> version('requests') + '2.22.0' + >>> list(requires('requests')) + ['chardet (<3.1.0,>=3.0.2)'] + >>> list(files('requests'))[:5] + [PackagePath('requests-2.22.0.dist-info/INSTALLER'), + PackagePath('requests-2.22.0.dist-info/LICENSE'), + PackagePath('requests-2.22.0.dist-info/METADATA'), + PackagePath('requests-2.22.0.dist-info/RECORD'), + PackagePath('requests-2.22.0.dist-info/WHEEL')] + + (Contributed by Barry Warsaw and Jason R. Coombs in bpo-34632(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34632 + + +File: python.info, Node: Improved Modules, Next: Optimizations, Prev: New Modules, Up: What’s New In Python 3 8 + +1.1.5 Improved Modules +---------------------- + +* Menu: + +* ast:: +* asyncio:: +* builtins:: +* collections:: +* cProfile:: +* csv:: +* curses:: +* ctypes:: +* datetime:: +* functools:: +* gc:: +* gettext:: +* gzip:: +* IDLE and idlelib:: +* inspect:: +* io:: +* itertools:: +* json.tool: json tool. +* logging:: +* math:: +* mmap:: +* multiprocessing:: +* os:: +* os.path: os path. +* pathlib:: +* pickle:: +* plistlib:: +* pprint:: +* py_compile:: +* shlex:: +* shutil:: +* socket:: +* ssl:: +* statistics:: +* sys:: +* tarfile:: +* threading:: +* tokenize:: +* tkinter:: +* time:: +* typing:: +* unicodedata:: +* unittest:: +* venv:: +* weakref:: +* xml:: +* xmlrpc:: + + +File: python.info, Node: ast, Next: asyncio, Up: Improved Modules + +1.1.5.1 ast +........... + +AST nodes now have ‘end_lineno’ and ‘end_col_offset’ attributes, which +give the precise location of the end of the node. (This only applies to +nodes that have ‘lineno’ and ‘col_offset’ attributes.) + +New function *note ast.get_source_segment(): 1a0. returns the source +code for a specific AST node. + +(Contributed by Ivan Levkivskyi in bpo-33416(1).) + +The *note ast.parse(): 1a1. function has some new flags: + + * ‘type_comments=True’ causes it to return the text of PEP 484(2) and + PEP 526(3) type comments associated with certain AST nodes; + + * ‘mode='func_type'’ can be used to parse PEP 484(4) “signature type + comments” (returned for function definition AST nodes); + + * ‘feature_version=(3, N)’ allows specifying an earlier Python 3 + version. For example, ‘feature_version=(3, 4)’ will treat *note + async: 1a2. and *note await: 1a3. as non-reserved words. + +(Contributed by Guido van Rossum in bpo-35766(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33416 + + (2) https://www.python.org/dev/peps/pep-0484 + + (3) https://www.python.org/dev/peps/pep-0526 + + (4) https://www.python.org/dev/peps/pep-0484 + + (5) https://bugs.python.org/issue35766 + + +File: python.info, Node: asyncio, Next: builtins, Prev: ast, Up: Improved Modules + +1.1.5.2 asyncio +............... + +*note asyncio.run(): 1a5. has graduated from the provisional to stable +API. This function can be used to execute a *note coroutine: 1a6. and +return the result while automatically managing the event loop. For +example: + + import asyncio + + async def main(): + await asyncio.sleep(0) + return 42 + + asyncio.run(main()) + +This is `roughly' equivalent to: + + import asyncio + + async def main(): + await asyncio.sleep(0) + return 42 + + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + try: + loop.run_until_complete(main()) + finally: + asyncio.set_event_loop(None) + loop.close() + +The actual implementation is significantly more complex. Thus, *note +asyncio.run(): 1a5. should be the preferred way of running asyncio +programs. + +(Contributed by Yury Selivanov in bpo-32314(1).) + +Running ‘python -m asyncio’ launches a natively async REPL. This allows +rapid experimentation with code that has a top-level *note await: 1a3. +There is no longer a need to directly call ‘asyncio.run()’ which would +spawn a new event loop on every invocation: + + $ python -m asyncio + asyncio REPL 3.8.0 + Use "await" directly instead of "asyncio.run()". + Type "help", "copyright", "credits" or "license" for more information. + >>> import asyncio + >>> await asyncio.sleep(10, result='hello') + hello + +(Contributed by Yury Selivanov in bpo-37028(2).) + +The exception *note asyncio.CancelledError: 1a7. now inherits from *note +BaseException: 1a8. rather than *note Exception: 1a9. and no longer +inherits from *note concurrent.futures.CancelledError: 1aa. +(Contributed by Yury Selivanov in bpo-32528(3).) + +On Windows, the default event loop is now *note ProactorEventLoop: 1ab. +(Contributed by Victor Stinner in bpo-34687(4).) + +*note ProactorEventLoop: 1ab. now also supports UDP. (Contributed by +Adam Meily and Andrew Svetlov in bpo-29883(5).) + +*note ProactorEventLoop: 1ab. can now be interrupted by *note +KeyboardInterrupt: 197. (“CTRL+C”). (Contributed by Vladimir Matveev in +bpo-23057(6).) + +Added *note asyncio.Task.get_coro(): 1ac. for getting the wrapped +coroutine within an *note asyncio.Task: 1ad. (Contributed by Alex +Grönholm in bpo-36999(7).) + +Asyncio tasks can now be named, either by passing the ‘name’ keyword +argument to *note asyncio.create_task(): 1ae. or the *note +create_task(): 1af. event loop method, or by calling the *note +set_name(): 1b0. method on the task object. The task name is visible in +the ‘repr()’ output of *note asyncio.Task: 1ad. and can also be +retrieved using the *note get_name(): 1b1. method. (Contributed by Alex +Grönholm in bpo-34270(8).) + +Added support for Happy Eyeballs(9) to *note +asyncio.loop.create_connection(): 1b2. To specify the behavior, two new +parameters have been added: `happy_eyeballs_delay' and `interleave'. +The Happy Eyeballs algorithm improves responsiveness in applications +that support IPv4 and IPv6 by attempting to simultaneously connect using +both. (Contributed by twisteroid ambassador in bpo-33530(10).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32314 + + (2) https://bugs.python.org/issue37028 + + (3) https://bugs.python.org/issue32528 + + (4) https://bugs.python.org/issue34687 + + (5) https://bugs.python.org/issue29883 + + (6) https://bugs.python.org/issue23057 + + (7) https://bugs.python.org/issue36999 + + (8) https://bugs.python.org/issue34270 + + (9) https://en.wikipedia.org/wiki/Happy_Eyeballs + + (10) https://bugs.python.org/issue33530 + + +File: python.info, Node: builtins, Next: collections, Prev: asyncio, Up: Improved Modules + +1.1.5.3 builtins +................ + +The *note compile(): 1b4. built-in has been improved to accept the +‘ast.PyCF_ALLOW_TOP_LEVEL_AWAIT’ flag. With this new flag passed, *note +compile(): 1b4. will allow top-level ‘await’, ‘async for’ and ‘async +with’ constructs that are usually considered invalid syntax. +Asynchronous code object marked with the ‘CO_COROUTINE’ flag may then be +returned. (Contributed by Matthias Bussonnier in bpo-34616(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34616 + + +File: python.info, Node: collections, Next: cProfile, Prev: builtins, Up: Improved Modules + +1.1.5.4 collections +................... + +The *note _asdict(): 1b6. method for *note collections.namedtuple(): +1b7. now returns a *note dict: 1b8. instead of a *note +collections.OrderedDict: 1b9. This works because regular dicts have +guaranteed ordering since Python 3.7. If the extra features of +‘OrderedDict’ are required, the suggested remediation is to cast the +result to the desired type: ‘OrderedDict(nt._asdict())’. (Contributed +by Raymond Hettinger in bpo-35864(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35864 + + +File: python.info, Node: cProfile, Next: csv, Prev: collections, Up: Improved Modules + +1.1.5.5 cProfile +................ + +The *note cProfile.Profile: 1bb. class can now be used as a context +manager. Profile a block of code by running: + + import cProfile + + with cProfile.Profile() as profiler: + # code to be profiled + ... + +(Contributed by Scott Sanderson in bpo-29235(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29235 + + +File: python.info, Node: csv, Next: curses, Prev: cProfile, Up: Improved Modules + +1.1.5.6 csv +........... + +The *note csv.DictReader: 1bd. now returns instances of *note dict: 1b8. +instead of a *note collections.OrderedDict: 1b9. The tool is now faster +and uses less memory while still preserving the field order. +(Contributed by Michael Selik in bpo-34003(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34003 + + +File: python.info, Node: curses, Next: ctypes, Prev: csv, Up: Improved Modules + +1.1.5.7 curses +.............. + +Added a new variable holding structured version information for the +underlying ncurses library: *note ncurses_version: 1bf. (Contributed by +Serhiy Storchaka in bpo-31680(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31680 + + +File: python.info, Node: ctypes, Next: datetime, Prev: curses, Up: Improved Modules + +1.1.5.8 ctypes +.............. + +On Windows, *note CDLL: 1c1. and subclasses now accept a `winmode' +parameter to specify flags for the underlying ‘LoadLibraryEx’ call. The +default flags are set to only load DLL dependencies from trusted +locations, including the path where the DLL is stored (if a full or +partial path is used to load the initial DLL) and paths added by *note +add_dll_directory(): 1c2. (Contributed by Steve Dower in bpo-36085(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36085 + + +File: python.info, Node: datetime, Next: functools, Prev: ctypes, Up: Improved Modules + +1.1.5.9 datetime +................ + +Added new alternate constructors *note datetime.date.fromisocalendar(): +1c4. and *note datetime.datetime.fromisocalendar(): 1c5, which construct +‘date’ and *note datetime: 31. objects respectively from ISO year, week +number, and weekday; these are the inverse of each class’s ‘isocalendar’ +method. (Contributed by Paul Ganssle in bpo-36004(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36004 + + +File: python.info, Node: functools, Next: gc, Prev: datetime, Up: Improved Modules + +1.1.5.10 functools +.................. + +*note functools.lru_cache(): 1c7. can now be used as a straight +decorator rather than as a function returning a decorator. So both of +these are now supported: + + @lru_cache + def f(x): + ... + + @lru_cache(maxsize=256) + def f(x): + ... + +(Contributed by Raymond Hettinger in bpo-36772(1).) + +Added a new *note functools.cached_property(): 1c8. decorator, for +computed properties cached for the life of the instance. + + import functools + import statistics + + class Dataset: + def __init__(self, sequence_of_numbers): + self.data = sequence_of_numbers + + @functools.cached_property + def variance(self): + return statistics.variance(self.data) + +(Contributed by Carl Meyer in bpo-21145(2)) + +Added a new *note functools.singledispatchmethod(): 1c9. decorator that +converts methods into *note generic functions: 1ca. using *note single +dispatch: 1cb.: + + from functools import singledispatchmethod + from contextlib import suppress + + class TaskManager: + + def __init__(self, tasks): + self.tasks = list(tasks) + + @singledispatchmethod + def discard(self, value): + with suppress(ValueError): + self.tasks.remove(value) + + @discard.register(list) + def _(self, tasks): + targets = set(tasks) + self.tasks = [x for x in self.tasks if x not in targets] + +(Contributed by Ethan Smith in bpo-32380(3)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36772 + + (2) https://bugs.python.org/issue21145 + + (3) https://bugs.python.org/issue32380 + + +File: python.info, Node: gc, Next: gettext, Prev: functools, Up: Improved Modules + +1.1.5.11 gc +........... + +*note get_objects(): 1cd. can now receive an optional `generation' +parameter indicating a generation to get objects from. (Contributed by +Pablo Galindo in bpo-36016(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36016 + + +File: python.info, Node: gettext, Next: gzip, Prev: gc, Up: Improved Modules + +1.1.5.12 gettext +................ + +Added *note pgettext(): 1cf. and its variants. (Contributed by Franz +Glasner, Éric Araujo, and Cheryl Sabella in bpo-2504(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2504 + + +File: python.info, Node: gzip, Next: IDLE and idlelib, Prev: gettext, Up: Improved Modules + +1.1.5.13 gzip +............. + +Added the `mtime' parameter to *note gzip.compress(): 1d1. for +reproducible output. (Contributed by Guo Ci Teo in bpo-34898(1).) + +A *note BadGzipFile: 1d2. exception is now raised instead of *note +OSError: 1d3. for certain types of invalid or corrupt gzip files. +(Contributed by Filip Gruszczyński, Michele Orrù, and Zackery Spytz in +bpo-6584(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34898 + + (2) https://bugs.python.org/issue6584 + + +File: python.info, Node: IDLE and idlelib, Next: inspect, Prev: gzip, Up: Improved Modules + +1.1.5.14 IDLE and idlelib +......................... + +Output over N lines (50 by default) is squeezed down to a button. N can +be changed in the PyShell section of the General page of the Settings +dialog. Fewer, but possibly extra long, lines can be squeezed by right +clicking on the output. Squeezed output can be expanded in place by +double-clicking the button or into the clipboard or a separate window by +right-clicking the button. (Contributed by Tal Einat in +bpo-1529353(1).) + +Add “Run Customized” to the Run menu to run a module with customized +settings. Any command line arguments entered are added to sys.argv. +They also re-appear in the box for the next customized run. One can +also suppress the normal Shell main module restart. (Contributed by +Cheryl Sabella, Terry Jan Reedy, and others in bpo-5680(2) and +bpo-37627(3).) + +Added optional line numbers for IDLE editor windows. Windows open +without line numbers unless set otherwise in the General tab of the +configuration dialog. Line numbers for an existing window are shown and +hidden in the Options menu. (Contributed by Tal Einat and Saimadhav +Heblikar in bpo-17535(4).) + +OS native encoding is now used for converting between Python strings and +Tcl objects. This allows IDLE to work with emoji and other non-BMP +characters. These characters can be displayed or copied and pasted to +or from the clipboard. Converting strings from Tcl to Python and back +now never fails. (Many people worked on this for eight years but the +problem was finally solved by Serhiy Storchaka in bpo-13153(5).) + +New in 3.8.1: + +Add option to toggle cursor blink off. (Contributed by Zackery Spytz in +bpo-4603(6).) + +Escape key now closes IDLE completion windows. (Contributed by Johnny +Najera in bpo-38944(7).) + +The changes above have been backported to 3.7 maintenance releases. + +Add keywords to module name completion list. (Contributed by Terry J. +Reedy in bpo-37765(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1529353 + + (2) https://bugs.python.org/issue5680 + + (3) https://bugs.python.org/issue37627 + + (4) https://bugs.python.org/issue17535 + + (5) https://bugs.python.org/issue13153 + + (6) https://bugs.python.org/issue4603 + + (7) https://bugs.python.org/issue38944 + + (8) https://bugs.python.org/issue37765 + + +File: python.info, Node: inspect, Next: io, Prev: IDLE and idlelib, Up: Improved Modules + +1.1.5.15 inspect +................ + +The *note inspect.getdoc(): 1d6. function can now find docstrings for +‘__slots__’ if that attribute is a *note dict: 1b8. where the values are +docstrings. This provides documentation options similar to what we +already have for *note property(): 1d7, *note classmethod(): 1d8, and +*note staticmethod(): 1d9.: + + class AudioClip: + __slots__ = {'bit_rate': 'expressed in kilohertz to one decimal place', + 'duration': 'in seconds, rounded up to an integer'} + def __init__(self, bit_rate, duration): + self.bit_rate = round(bit_rate / 1000.0, 1) + self.duration = ceil(duration) + +(Contributed by Raymond Hettinger in bpo-36326(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36326 + + +File: python.info, Node: io, Next: itertools, Prev: inspect, Up: Improved Modules + +1.1.5.16 io +........... + +In development mode (*note -X: 155. ‘env’) and in debug build, the *note +io.IOBase: 1db. finalizer now logs the exception if the ‘close()’ method +fails. The exception is ignored silently by default in release build. +(Contributed by Victor Stinner in bpo-18748(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18748 + + +File: python.info, Node: itertools, Next: json tool, Prev: io, Up: Improved Modules + +1.1.5.17 itertools +.................. + +The *note itertools.accumulate(): 1dd. function added an option +`initial' keyword argument to specify an initial value: + + >>> from itertools import accumulate + >>> list(accumulate([10, 5, 30, 15], initial=1000)) + [1000, 1010, 1015, 1045, 1060] + +(Contributed by Lisa Roach in bpo-34659(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34659 + + +File: python.info, Node: json tool, Next: logging, Prev: itertools, Up: Improved Modules + +1.1.5.18 json.tool +.................. + +Add option ‘--json-lines’ to parse every input line as a separate JSON +object. (Contributed by Weipeng Hong in bpo-31553(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31553 + + +File: python.info, Node: logging, Next: math, Prev: json tool, Up: Improved Modules + +1.1.5.19 logging +................ + +Added a `force' keyword argument to *note logging.basicConfig(): 1e0. +When set to true, any existing handlers attached to the root logger are +removed and closed before carrying out the configuration specified by +the other arguments. + +This solves a long-standing problem. Once a logger or `basicConfig()' +had been called, subsequent calls to `basicConfig()' were silently +ignored. This made it difficult to update, experiment with, or teach +the various logging configuration options using the interactive prompt +or a Jupyter notebook. + +(Suggested by Raymond Hettinger, implemented by Dong-hee Na, and +reviewed by Vinay Sajip in bpo-33897(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33897 + + +File: python.info, Node: math, Next: mmap, Prev: logging, Up: Improved Modules + +1.1.5.20 math +............. + +Added new function *note math.dist(): 1e2. for computing Euclidean +distance between two points. (Contributed by Raymond Hettinger in +bpo-33089(1).) + +Expanded the *note math.hypot(): 1e3. function to handle multiple +dimensions. Formerly, it only supported the 2-D case. (Contributed by +Raymond Hettinger in bpo-33089(2).) + +Added new function, *note math.prod(): 1e4, as analogous function to +*note sum(): 1e5. that returns the product of a ‘start’ value (default: +1) times an iterable of numbers: + + >>> prior = 0.8 + >>> likelihoods = [0.625, 0.84, 0.30] + >>> math.prod(likelihoods, start=prior) + 0.126 + +(Contributed by Pablo Galindo in bpo-35606(3).) + +Added two new combinatoric functions *note math.perm(): 1e6. and *note +math.comb(): 1e7.: + + >>> math.perm(10, 3) # Permutations of 10 things taken 3 at a time + 720 + >>> math.comb(10, 3) # Combinations of 10 things taken 3 at a time + 120 + +(Contributed by Yash Aggarwal, Keller Fuchs, Serhiy Storchaka, and +Raymond Hettinger in bpo-37128(4), bpo-37178(5), and bpo-35431(6).) + +Added a new function *note math.isqrt(): 1e8. for computing accurate +integer square roots without conversion to floating point. The new +function supports arbitrarily large integers. It is faster than +‘floor(sqrt(n))’ but slower than *note math.sqrt(): 1e9.: + + >>> r = 650320427 + >>> s = r ** 2 + >>> isqrt(s - 1) # correct + 650320426 + >>> floor(sqrt(s - 1)) # incorrect + 650320427 + +(Contributed by Mark Dickinson in bpo-36887(7).) + +The function *note math.factorial(): 1ea. no longer accepts arguments +that are not int-like. (Contributed by Pablo Galindo in bpo-33083(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33089 + + (2) https://bugs.python.org/issue33089 + + (3) https://bugs.python.org/issue35606 + + (4) https://bugs.python.org/issue37128 + + (5) https://bugs.python.org/issue37178 + + (6) https://bugs.python.org/issue35431 + + (7) https://bugs.python.org/issue36887 + + (8) https://bugs.python.org/issue33083 + + +File: python.info, Node: mmap, Next: multiprocessing, Prev: math, Up: Improved Modules + +1.1.5.21 mmap +............. + +The *note mmap.mmap: 1ec. class now has an *note madvise(): 1ed. method +to access the ‘madvise()’ system call. (Contributed by Zackery Spytz in +bpo-32941(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32941 + + +File: python.info, Node: multiprocessing, Next: os, Prev: mmap, Up: Improved Modules + +1.1.5.22 multiprocessing +........................ + +Added new *note multiprocessing.shared_memory: bc. module. (Contributed +by Davin Potts in bpo-35813(1).) + +On macOS, the `spawn' start method is now used by default. (Contributed +by Victor Stinner in bpo-33725(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35813 + + (2) https://bugs.python.org/issue33725 + + +File: python.info, Node: os, Next: os path, Prev: multiprocessing, Up: Improved Modules + +1.1.5.23 os +........... + +Added new function *note add_dll_directory(): 1c2. on Windows for +providing additional search paths for native dependencies when importing +extension modules or loading DLLs using *note ctypes: 2b. (Contributed +by Steve Dower in bpo-36085(1).) + +A new *note os.memfd_create(): 1f0. function was added to wrap the +‘memfd_create()’ syscall. (Contributed by Zackery Spytz and Christian +Heimes in bpo-26836(2).) + +On Windows, much of the manual logic for handling reparse points +(including symlinks and directory junctions) has been delegated to the +operating system. Specifically, *note os.stat(): 1f1. will now traverse +anything supported by the operating system, while *note os.lstat(): 1f2. +will only open reparse points that identify as “name surrogates” while +others are opened as for *note os.stat(): 1f1. In all cases, +‘stat_result.st_mode’ will only have ‘S_IFLNK’ set for symbolic links +and not other kinds of reparse points. To identify other kinds of +reparse point, check the new ‘stat_result.st_reparse_tag’ attribute. + +On Windows, *note os.readlink(): 1f3. is now able to read directory +junctions. Note that *note islink(): 1f4. will return ‘False’ for +directory junctions, and so code that checks ‘islink’ first will +continue to treat junctions as directories, while code that handles +errors from *note os.readlink(): 1f3. may now treat junctions as links. + +(Contributed by Steve Dower in bpo-37834(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36085 + + (2) https://bugs.python.org/issue26836 + + (3) https://bugs.python.org/issue37834 + + +File: python.info, Node: os path, Next: pathlib, Prev: os, Up: Improved Modules + +1.1.5.24 os.path +................ + +*note os.path: c5. functions that return a boolean result like *note +exists(): 1f6, *note lexists(): 1f7, *note isdir(): 1f8, *note isfile(): +1f9, *note islink(): 1f4, and *note ismount(): 1fa. now return ‘False’ +instead of raising *note ValueError: 1fb. or its subclasses *note +UnicodeEncodeError: 1fc. and *note UnicodeDecodeError: 1fd. for paths +that contain characters or bytes unrepresentable at the OS level. +(Contributed by Serhiy Storchaka in bpo-33721(1).) + +*note expanduser(): 1fe. on Windows now prefers the ‘USERPROFILE’ +environment variable and does not use ‘HOME’, which is not normally set +for regular user accounts. (Contributed by Anthony Sottile in +bpo-36264(2).) + +*note isdir(): 1f8. on Windows no longer returns ‘True’ for a link to a +non-existent directory. + +*note realpath(): 1ff. on Windows now resolves reparse points, including +symlinks and directory junctions. + +(Contributed by Steve Dower in bpo-37834(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33721 + + (2) https://bugs.python.org/issue36264 + + (3) https://bugs.python.org/issue37834 + + +File: python.info, Node: pathlib, Next: pickle, Prev: os path, Up: Improved Modules + +1.1.5.25 pathlib +................ + +*note pathlib.Path: 201. methods that return a boolean result like *note +exists(): 202, *note is_dir(): 203, *note is_file(): 204, *note +is_mount(): 205, *note is_symlink(): 206, *note is_block_device(): 207, +*note is_char_device(): 208, *note is_fifo(): 209, *note is_socket(): +20a. now return ‘False’ instead of raising *note ValueError: 1fb. or its +subclass *note UnicodeEncodeError: 1fc. for paths that contain +characters unrepresentable at the OS level. (Contributed by Serhiy +Storchaka in bpo-33721(1).) + +Added *note pathlib.Path.link_to(): 20b. which creates a hard link +pointing to a path. (Contributed by Joannah Nanjekye in bpo-26978(2)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33721 + + (2) https://bugs.python.org/issue26978 + + +File: python.info, Node: pickle, Next: plistlib, Prev: pathlib, Up: Improved Modules + +1.1.5.26 pickle +............... + +*note pickle: ca. extensions subclassing the C-optimized *note Pickler: +20d. can now override the pickling logic of functions and classes by +defining the special *note reducer_override(): 20e. method. +(Contributed by Pierre Glaser and Olivier Grisel in bpo-35900(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35900 + + +File: python.info, Node: plistlib, Next: pprint, Prev: pickle, Up: Improved Modules + +1.1.5.27 plistlib +................. + +Added new *note plistlib.UID: 210. and enabled support for reading and +writing NSKeyedArchiver-encoded binary plists. (Contributed by Jon +Janzen in bpo-26707(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26707 + + +File: python.info, Node: pprint, Next: py_compile, Prev: plistlib, Up: Improved Modules + +1.1.5.28 pprint +............... + +The *note pprint: d2. module added a `sort_dicts' parameter to several +functions. By default, those functions continue to sort dictionaries +before rendering or printing. However, if `sort_dicts' is set to false, +the dictionaries retain the order that keys were inserted. This can be +useful for comparison to JSON inputs during debugging. + +In addition, there is a convenience new function, *note pprint.pp(): +212. that is like *note pprint.pprint(): 213. but with `sort_dicts' +defaulting to ‘False’: + + >>> from pprint import pprint, pp + >>> d = dict(source='input.txt', operation='filter', destination='output.txt') + >>> pp(d, width=40) # Original order + {'source': 'input.txt', + 'operation': 'filter', + 'destination': 'output.txt'} + >>> pprint(d, width=40) # Keys sorted alphabetically + {'destination': 'output.txt', + 'operation': 'filter', + 'source': 'input.txt'} + +(Contributed by Rémi Lapeyre in bpo-30670(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30670 + + +File: python.info, Node: py_compile, Next: shlex, Prev: pprint, Up: Improved Modules + +1.1.5.29 py_compile +................... + +*note py_compile.compile(): 215. now supports silent mode. (Contributed +by Joannah Nanjekye in bpo-22640(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22640 + + +File: python.info, Node: shlex, Next: shutil, Prev: py_compile, Up: Improved Modules + +1.1.5.30 shlex +.............. + +The new *note shlex.join(): 217. function acts as the inverse of *note +shlex.split(): 218. (Contributed by Bo Bayles in bpo-32102(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32102 + + +File: python.info, Node: shutil, Next: socket, Prev: shlex, Up: Improved Modules + +1.1.5.31 shutil +............... + +*note shutil.copytree(): 21a. now accepts a new ‘dirs_exist_ok’ keyword +argument. (Contributed by Josh Bronson in bpo-20849(1).) + +*note shutil.make_archive(): 21b. now defaults to the modern pax +(POSIX.1-2001) format for new archives to improve portability and +standards conformance, inherited from the corresponding change to the +*note tarfile: 101. module. (Contributed by C.A.M. Gerlach in +bpo-30661(2).) + +*note shutil.rmtree(): 21c. on Windows now removes directory junctions +without recursively removing their contents first. (Contributed by +Steve Dower in bpo-37834(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20849 + + (2) https://bugs.python.org/issue30661 + + (3) https://bugs.python.org/issue37834 + + +File: python.info, Node: socket, Next: ssl, Prev: shutil, Up: Improved Modules + +1.1.5.32 socket +............... + +Added *note create_server(): 21e. and *note has_dualstack_ipv6(): 21f. +convenience functions to automate the necessary tasks usually involved +when creating a server socket, including accepting both IPv4 and IPv6 +connections on the same socket. (Contributed by Giampaolo Rodolà in +bpo-17561(1).) + +The *note socket.if_nameindex(): 220, *note socket.if_nametoindex(): +221, and *note socket.if_indextoname(): 222. functions have been +implemented on Windows. (Contributed by Zackery Spytz in bpo-37007(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17561 + + (2) https://bugs.python.org/issue37007 + + +File: python.info, Node: ssl, Next: statistics, Prev: socket, Up: Improved Modules + +1.1.5.33 ssl +............ + +Added *note post_handshake_auth: 224. to enable and *note +verify_client_post_handshake(): 225. to initiate TLS 1.3 post-handshake +authentication. (Contributed by Christian Heimes in bpo-34670(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34670 + + +File: python.info, Node: statistics, Next: sys, Prev: ssl, Up: Improved Modules + +1.1.5.34 statistics +................... + +Added *note statistics.fmean(): 227. as a faster, floating point variant +of *note statistics.mean(): 199. (Contributed by Raymond Hettinger and +Steven D’Aprano in bpo-35904(1).) + +Added *note statistics.geometric_mean(): 228. (Contributed by Raymond +Hettinger in bpo-27181(2).) + +Added *note statistics.multimode(): 229. that returns a list of the most +common values. (Contributed by Raymond Hettinger in bpo-35892(3).) + +Added *note statistics.quantiles(): 22a. that divides data or a +distribution in to equiprobable intervals (e.g. quartiles, deciles, or +percentiles). (Contributed by Raymond Hettinger in bpo-36546(4).) + +Added *note statistics.NormalDist: 22b, a tool for creating and +manipulating normal distributions of a random variable. (Contributed by +Raymond Hettinger in bpo-36018(5).) + + >>> temperature_feb = NormalDist.from_samples([4, 12, -3, 2, 7, 14]) + >>> temperature_feb.mean + 6.0 + >>> temperature_feb.stdev + 6.356099432828281 + + >>> temperature_feb.cdf(3) # Chance of being under 3 degrees + 0.3184678262814532 + >>> # Relative chance of being 7 degrees versus 10 degrees + >>> temperature_feb.pdf(7) / temperature_feb.pdf(10) + 1.2039930378537762 + + >>> el_niño = NormalDist(4, 2.5) + >>> temperature_feb += el_niño # Add in a climate effect + >>> temperature_feb + NormalDist(mu=10.0, sigma=6.830080526611674) + + >>> temperature_feb * (9/5) + 32 # Convert to Fahrenheit + NormalDist(mu=50.0, sigma=12.294144947901014) + >>> temperature_feb.samples(3) # Generate random samples + [7.672102882379219, 12.000027119750287, 4.647488369766392] + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35904 + + (2) https://bugs.python.org/issue27181 + + (3) https://bugs.python.org/issue35892 + + (4) https://bugs.python.org/issue36546 + + (5) https://bugs.python.org/issue36018 + + +File: python.info, Node: sys, Next: tarfile, Prev: statistics, Up: Improved Modules + +1.1.5.35 sys +............ + +Add new *note sys.unraisablehook(): 22d. function which can be +overridden to control how “unraisable exceptions” are handled. It is +called when an exception has occurred but there is no way for Python to +handle it. For example, when a destructor raises an exception or during +garbage collection (*note gc.collect(): 22e.). (Contributed by Victor +Stinner in bpo-36829(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36829 + + +File: python.info, Node: tarfile, Next: threading, Prev: sys, Up: Improved Modules + +1.1.5.36 tarfile +................ + +The *note tarfile: 101. module now defaults to the modern pax +(POSIX.1-2001) format for new archives, instead of the previous +GNU-specific one. This improves cross-platform portability with a +consistent encoding (UTF-8) in a standardized and extensible format, and +offers several other benefits. (Contributed by C.A.M. Gerlach in +bpo-36268(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36268 + + +File: python.info, Node: threading, Next: tokenize, Prev: tarfile, Up: Improved Modules + +1.1.5.37 threading +.................. + +Add a new *note threading.excepthook(): 231. function which handles +uncaught *note threading.Thread.run(): 232. exception. It can be +overridden to control how uncaught *note threading.Thread.run(): 232. +exceptions are handled. (Contributed by Victor Stinner in +bpo-1230540(1).) + +Add a new *note threading.get_native_id(): 233. function and a *note +native_id: 234. attribute to the *note threading.Thread: 235. class. +These return the native integral Thread ID of the current thread +assigned by the kernel. This feature is only available on certain +platforms, see *note get_native_id: 233. for more information. +(Contributed by Jake Tesler in bpo-36084(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1230540 + + (2) https://bugs.python.org/issue36084 + + +File: python.info, Node: tokenize, Next: tkinter, Prev: threading, Up: Improved Modules + +1.1.5.38 tokenize +................. + +The *note tokenize: 111. module now implicitly emits a ‘NEWLINE’ token +when provided with input that does not have a trailing new line. This +behavior now matches what the C tokenizer does internally. (Contributed +by Ammar Askar in bpo-33899(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33899 + + +File: python.info, Node: tkinter, Next: time, Prev: tokenize, Up: Improved Modules + +1.1.5.39 tkinter +................ + +Added methods ‘selection_from()’, ‘selection_present()’, +‘selection_range()’ and ‘selection_to()’ in the ‘tkinter.Spinbox’ class. +(Contributed by Juliette Monsel in bpo-34829(1).) + +Added method ‘moveto()’ in the ‘tkinter.Canvas’ class. (Contributed by +Juliette Monsel in bpo-23831(2).) + +The ‘tkinter.PhotoImage’ class now has ‘transparency_get()’ and +‘transparency_set()’ methods. (Contributed by Zackery Spytz in +bpo-25451(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34829 + + (2) https://bugs.python.org/issue23831 + + (3) https://bugs.python.org/issue25451 + + +File: python.info, Node: time, Next: typing, Prev: tkinter, Up: Improved Modules + +1.1.5.40 time +............. + +Added new clock *note CLOCK_UPTIME_RAW: 239. for macOS 10.12. +(Contributed by Joannah Nanjekye in bpo-35702(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35702 + + +File: python.info, Node: typing, Next: unicodedata, Prev: time, Up: Improved Modules + +1.1.5.41 typing +............... + +The *note typing: 119. module incorporates several new features: + + * A dictionary type with per-key types. See PEP 589(1) and *note + typing.TypedDict: 23b. TypedDict uses only string keys. By + default, every key is required to be present. Specify + “total=False” to allow keys to be optional: + + class Location(TypedDict, total=False): + lat_long: tuple + grid_square: str + xy_coordinate: tuple + + * Literal types. See PEP 586(2) and *note typing.Literal: 23c. + Literal types indicate that a parameter or return value is + constrained to one or more specific literal values: + + def get_status(port: int) -> Literal['connected', 'disconnected']: + ... + + * “Final” variables, functions, methods and classes. See PEP 591(3), + *note typing.Final: 23d. and *note typing.final(): 23e. The final + qualifier instructs a static type checker to restrict subclassing, + overriding, or reassignment: + + pi: Final[float] = 3.1415926536 + + * Protocol definitions. See PEP 544(4), *note typing.Protocol: 23f. + and *note typing.runtime_checkable(): 240. Simple ABCs like *note + typing.SupportsInt: 241. are now ‘Protocol’ subclasses. + + * New protocol class *note typing.SupportsIndex: 242. + + * New functions *note typing.get_origin(): 243. and *note + typing.get_args(): 244. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0589 + + (2) https://www.python.org/dev/peps/pep-0586 + + (3) https://www.python.org/dev/peps/pep-0591 + + (4) https://www.python.org/dev/peps/pep-0544 + + +File: python.info, Node: unicodedata, Next: unittest, Prev: typing, Up: Improved Modules + +1.1.5.42 unicodedata +.................... + +The *note unicodedata: 11a. module has been upgraded to use the Unicode +12.1.0(1) release. + +New function *note is_normalized(): 246. can be used to verify a string +is in a specific normal form, often much faster than by actually +normalizing the string. (Contributed by Max Belanger, David Euresti, +and Greg Price in bpo-32285(2) and bpo-37966(3)). + + ---------- Footnotes ---------- + + (1) http://blog.unicode.org/2019/05/unicode-12-1-en.html + + (2) https://bugs.python.org/issue32285 + + (3) https://bugs.python.org/issue37966 + + +File: python.info, Node: unittest, Next: venv, Prev: unicodedata, Up: Improved Modules + +1.1.5.43 unittest +................. + +Added *note AsyncMock: 248. to support an asynchronous version of *note +Mock: 249. Appropriate new assert functions for testing have been added +as well. (Contributed by Lisa Roach in bpo-26467(1)). + +Added *note addModuleCleanup(): 24a. and *note addClassCleanup(): 24b. +to unittest to support cleanups for ‘setUpModule()’ and *note +setUpClass(): 24c. (Contributed by Lisa Roach in bpo-24412(2).) + +Several mock assert functions now also print a list of actual calls upon +failure. (Contributed by Petter Strandmark in bpo-35047(3).) + +*note unittest: 11b. module gained support for coroutines to be used as +test cases with *note unittest.IsolatedAsyncioTestCase: 24d. +(Contributed by Andrew Svetlov in bpo-32972(4).) + +Example: + + import unittest + + + class TestRequest(unittest.IsolatedAsyncioTestCase): + + async def asyncSetUp(self): + self.connection = await AsyncConnection() + + async def test_get(self): + response = await self.connection.get("https://example.com") + self.assertEqual(response.status_code, 200) + + async def asyncTearDown(self): + await self.connection.close() + + + if __name__ == "__main__": + unittest.main() + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26467 + + (2) https://bugs.python.org/issue24412 + + (3) https://bugs.python.org/issue35047 + + (4) https://bugs.python.org/issue32972 + + +File: python.info, Node: venv, Next: weakref, Prev: unittest, Up: Improved Modules + +1.1.5.44 venv +............. + +*note venv: 125. now includes an ‘Activate.ps1’ script on all platforms +for activating virtual environments under PowerShell Core 6.1. +(Contributed by Brett Cannon in bpo-32718(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32718 + + +File: python.info, Node: weakref, Next: xml, Prev: venv, Up: Improved Modules + +1.1.5.45 weakref +................ + +The proxy objects returned by *note weakref.proxy(): 250. now support +the matrix multiplication operators ‘@’ and ‘@=’ in addition to the +other numeric operators. (Contributed by Mark Dickinson in +bpo-36669(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36669 + + +File: python.info, Node: xml, Next: xmlrpc, Prev: weakref, Up: Improved Modules + +1.1.5.46 xml +............ + +As mitigation against DTD and external entity retrieval, the *note +xml.dom.minidom: 135. and *note xml.sax: 13b. modules no longer process +external entities by default. (Contributed by Christian Heimes in +bpo-17239(1).) + +The ‘.find*()’ methods in the *note xml.etree.ElementTree: 137. module +support wildcard searches like ‘{*}tag’ which ignores the namespace and +‘{namespace}*’ which returns all tags in the given namespace. +(Contributed by Stefan Behnel in bpo-28238(2).) + +The *note xml.etree.ElementTree: 137. module provides a new function +‘–xml.etree.ElementTree.canonicalize()’ that implements C14N 2.0. +(Contributed by Stefan Behnel in bpo-13611(3).) + +The target object of *note xml.etree.ElementTree.XMLParser: 252. can +receive namespace declaration events through the new callback methods +‘start_ns()’ and ‘end_ns()’. Additionally, the *note +xml.etree.ElementTree.TreeBuilder: 253. target can be configured to +process events about comments and processing instructions to include +them in the generated tree. (Contributed by Stefan Behnel in +bpo-36676(4) and bpo-36673(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17239 + + (2) https://bugs.python.org/issue28238 + + (3) https://bugs.python.org/issue13611 + + (4) https://bugs.python.org/issue36676 + + (5) https://bugs.python.org/issue36673 + + +File: python.info, Node: xmlrpc, Prev: xml, Up: Improved Modules + +1.1.5.47 xmlrpc +............... + +*note xmlrpc.client.ServerProxy: 255. now supports an optional `headers' +keyword argument for a sequence of HTTP headers to be sent with each +request. Among other things, this makes it possible to upgrade from +default basic authentication to faster session authentication. +(Contributed by Cédric Krier in bpo-35153(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35153 + + +File: python.info, Node: Optimizations, Next: Build and C API Changes, Prev: Improved Modules, Up: What’s New In Python 3 8 + +1.1.6 Optimizations +------------------- + + * The *note subprocess: f9. module can now use the *note + os.posix_spawn(): 257. function in some cases for better + performance. Currently, it is only used on macOS and Linux (using + glibc 2.24 or newer) if all these conditions are met: + + * `close_fds' is false; + + * `preexec_fn', `pass_fds', `cwd' and `start_new_session' + parameters are not set; + + * the `executable' path contains a directory. + + (Contributed by Joannah Nanjekye and Victor Stinner in + bpo-35537(1).) + + * *note shutil.copyfile(): 258, *note shutil.copy(): 259, *note + shutil.copy2(): 25a, *note shutil.copytree(): 21a. and *note + shutil.move(): 25b. use platform-specific “fast-copy” syscalls on + Linux and macOS in order to copy the file more efficiently. + “fast-copy” means that the copying operation occurs within the + kernel, avoiding the use of userspace buffers in Python as in + “‘outfd.write(infd.read())’”. On Windows *note shutil.copyfile(): + 258. uses a bigger default buffer size (1 MiB instead of 16 KiB) + and a *note memoryview(): 25c.-based variant of *note + shutil.copyfileobj(): 25d. is used. The speedup for copying a 512 + MiB file within the same partition is about +26% on Linux, +50% on + macOS and +40% on Windows. Also, much less CPU cycles are + consumed. See *note Platform-dependent efficient copy operations: + 25e. section. (Contributed by Giampaolo Rodolà in bpo-33671(2).) + + * *note shutil.copytree(): 21a. uses *note os.scandir(): 25f. + function and all copy functions depending from it use cached *note + os.stat(): 1f1. values. The speedup for copying a directory with + 8000 files is around +9% on Linux, +20% on Windows and +30% on a + Windows SMB share. Also the number of *note os.stat(): 1f1. + syscalls is reduced by 38% making *note shutil.copytree(): 21a. + especially faster on network filesystems. (Contributed by + Giampaolo Rodolà in bpo-33695(3).) + + * The default protocol in the *note pickle: ca. module is now + Protocol 4, first introduced in Python 3.4. It offers better + performance and smaller size compared to Protocol 3 available since + Python 3.0. + + * Removed one ‘Py_ssize_t’ member from ‘PyGC_Head’. All GC tracked + objects (e.g. tuple, list, dict) size is reduced 4 or 8 bytes. + (Contributed by Inada Naoki in bpo-33597(4).) + + * *note uuid.UUID: 260. now uses ‘__slots__’ to reduce its memory + footprint. (Contributed by Wouter Bolsterlee and Tal Einat in + bpo-30977(5)) + + * Improved performance of *note operator.itemgetter(): 261. by 33%. + Optimized argument handling and added a fast path for the common + case of a single non-negative integer index into a tuple (which is + the typical use case in the standard library). (Contributed by + Raymond Hettinger in bpo-35664(6).) + + * Sped-up field lookups in *note collections.namedtuple(): 1b7. They + are now more than two times faster, making them the fastest form of + instance variable lookup in Python. (Contributed by Raymond + Hettinger, Pablo Galindo, and Joe Jevnik, Serhiy Storchaka in + bpo-32492(7).) + + * The *note list: 262. constructor does not overallocate the internal + item buffer if the input iterable has a known length (the input + implements ‘__len__’). This makes the created list 12% smaller on + average. (Contributed by Raymond Hettinger and Pablo Galindo in + bpo-33234(8).) + + * Doubled the speed of class variable writes. When a non-dunder + attribute was updated, there was an unnecessary call to update + slots. (Contributed by Stefan Behnel, Pablo Galindo Salgado, + Raymond Hettinger, Neil Schemenauer, and Serhiy Storchaka in + bpo-36012(9).) + + * Reduced an overhead of converting arguments passed to many builtin + functions and methods. This sped up calling some simple builtin + functions and methods up to 20–50%. (Contributed by Serhiy + Storchaka in bpo-23867(10), bpo-35582(11) and bpo-36127(12).) + + * ‘LOAD_GLOBAL’ instruction now uses new “per opcode cache” + mechanism. It is about 40% faster now. (Contributed by Yury + Selivanov and Inada Naoki in bpo-26219(13).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35537 + + (2) https://bugs.python.org/issue33671 + + (3) https://bugs.python.org/issue33695 + + (4) https://bugs.python.org/issue33597 + + (5) https://bugs.python.org/issue30977 + + (6) https://bugs.python.org/issue35664 + + (7) https://bugs.python.org/issue32492 + + (8) https://bugs.python.org/issue33234 + + (9) https://bugs.python.org/issue36012 + + (10) https://bugs.python.org/issue23867 + + (11) https://bugs.python.org/issue35582 + + (12) https://bugs.python.org/issue36127 + + (13) https://bugs.python.org/issue26219 + + +File: python.info, Node: Build and C API Changes, Next: Deprecated, Prev: Optimizations, Up: What’s New In Python 3 8 + +1.1.7 Build and C API Changes +----------------------------- + + * Default *note sys.abiflags: 264. became an empty string: the ‘m’ + flag for pymalloc became useless (builds with and without pymalloc + are ABI compatible) and so has been removed. (Contributed by + Victor Stinner in bpo-36707(1).) + + Example of changes: + + * Only ‘python3.8’ program is installed, ‘python3.8m’ program is + gone. + + * Only ‘python3.8-config’ script is installed, + ‘python3.8m-config’ script is gone. + + * The ‘m’ flag has been removed from the suffix of dynamic + library filenames: extension modules in the standard library + as well as those produced and installed by third-party + packages, like those downloaded from PyPI. On Linux, for + example, the Python 3.7 suffix + ‘.cpython-37m-x86_64-linux-gnu.so’ became + ‘.cpython-38-x86_64-linux-gnu.so’ in Python 3.8. + + * The header files have been reorganized to better separate the + different kinds of APIs: + + * ‘Include/*.h’ should be the portable public stable C API. + + * ‘Include/cpython/*.h’ should be the unstable C API specific to + CPython; public API, with some private API prefixed by ‘_Py’ + or ‘_PY’. + + * ‘Include/internal/*.h’ is the private internal C API very + specific to CPython. This API comes with no backward + compatibility warranty and should not be used outside CPython. + It is only exposed for very specific needs like debuggers and + profiles which has to access to CPython internals without + calling functions. This API is now installed by ‘make + install’. + + (Contributed by Victor Stinner in bpo-35134(2) and bpo-35081(3), + work initiated by Eric Snow in Python 3.7.) + + * Some macros have been converted to static inline functions: + parameter types and return type are well defined, they don’t have + issues specific to macros, variables have a local scopes. + Examples: + + * *note Py_INCREF(): 265, *note Py_DECREF(): 266. + + * *note Py_XINCREF(): 267, *note Py_XDECREF(): 268. + + * ‘PyObject_INIT()’, ‘PyObject_INIT_VAR()’ + + * Private functions: ‘_PyObject_GC_TRACK()’, + ‘_PyObject_GC_UNTRACK()’, ‘_Py_Dealloc()’ + + (Contributed by Victor Stinner in bpo-35059(4).) + + * The ‘PyByteArray_Init()’ and ‘PyByteArray_Fini()’ functions have + been removed. They did nothing since Python 2.7.4 and Python + 3.2.0, were excluded from the limited API (stable ABI), and were + not documented. (Contributed by Victor Stinner in bpo-35713(5).) + + * The result of ‘PyExceptionClass_Name()’ is now of type ‘const char + *’ rather of ‘char *’. (Contributed by Serhiy Storchaka in + bpo-33818(6).) + + * The duality of ‘Modules/Setup.dist’ and ‘Modules/Setup’ has been + removed. Previously, when updating the CPython source tree, one + had to manually copy ‘Modules/Setup.dist’ (inside the source tree) + to ‘Modules/Setup’ (inside the build tree) in order to reflect any + changes upstream. This was of a small benefit to packagers at the + expense of a frequent annoyance to developers following CPython + development, as forgetting to copy the file could produce build + failures. + + Now the build system always reads from ‘Modules/Setup’ inside the + source tree. People who want to customize that file are encouraged + to maintain their changes in a git fork of CPython or as patch + files, as they would do for any other change to the source tree. + + (Contributed by Antoine Pitrou in bpo-32430(7).) + + * Functions that convert Python number to C integer like *note + PyLong_AsLong(): 269. and argument parsing functions like *note + PyArg_ParseTuple(): 26a. with integer converting format units like + ‘'i'’ will now use the *note __index__(): 18a. special method + instead of *note __int__(): 18b, if available. The deprecation + warning will be emitted for objects with the ‘__int__()’ method but + without the ‘__index__()’ method (like *note Decimal: 188. and + *note Fraction: 185.). *note PyNumber_Check(): 26b. will now + return ‘1’ for objects implementing ‘__index__()’. *note + PyNumber_Long(): 26c, *note PyNumber_Float(): 26d. and *note + PyFloat_AsDouble(): 26e. also now use the ‘__index__()’ method if + available. (Contributed by Serhiy Storchaka in bpo-36048(8) and + bpo-20092(9).) + + * Heap-allocated type objects will now increase their reference count + in *note PyObject_Init(): 26f. (and its parallel macro + ‘PyObject_INIT’) instead of in *note PyType_GenericAlloc(): 270. + Types that modify instance allocation or deallocation may need to + be adjusted. (Contributed by Eddie Elizondo in bpo-35810(10).) + + * The new function *note PyCode_NewWithPosOnlyArgs(): 271. allows to + create code objects like *note PyCode_New(): 272, but with an extra + `posonlyargcount' parameter for indicating the number of + positional-only arguments. (Contributed by Pablo Galindo in + bpo-37221(11).) + + * *note Py_SetPath(): 273. now sets *note sys.executable: 274. to the + program full path (*note Py_GetProgramFullPath(): 275.) rather than + to the program name (*note Py_GetProgramName(): 276.). + (Contributed by Victor Stinner in bpo-38234(12).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36707 + + (2) https://bugs.python.org/issue35134 + + (3) https://bugs.python.org/issue35081 + + (4) https://bugs.python.org/issue35059 + + (5) https://bugs.python.org/issue35713 + + (6) https://bugs.python.org/issue33818 + + (7) https://bugs.python.org/issue32430 + + (8) https://bugs.python.org/issue36048 + + (9) https://bugs.python.org/issue20092 + + (10) https://bugs.python.org/issue35810 + + (11) https://bugs.python.org/issue37221 + + (12) https://bugs.python.org/issue38234 + + +File: python.info, Node: Deprecated, Next: API and Feature Removals, Prev: Build and C API Changes, Up: What’s New In Python 3 8 + +1.1.8 Deprecated +---------------- + + * The distutils ‘bdist_wininst’ command is now deprecated, use + ‘bdist_wheel’ (wheel packages) instead. (Contributed by Victor + Stinner in bpo-37481(1).) + + * Deprecated methods ‘getchildren()’ and ‘getiterator()’ in the *note + ElementTree: 137. module now emit a *note DeprecationWarning: 278. + instead of *note PendingDeprecationWarning: 279. They will be + removed in Python 3.9. (Contributed by Serhiy Storchaka in + bpo-29209(2).) + + * Passing an object that is not an instance of *note + concurrent.futures.ThreadPoolExecutor: 27a. to *note + loop.set_default_executor(): 27b. is deprecated and will be + prohibited in Python 3.9. (Contributed by Elvis Pranskevichus in + bpo-34075(3).) + + * The *note __getitem__(): 27c. methods of *note + xml.dom.pulldom.DOMEventStream: 27d, *note + wsgiref.util.FileWrapper: 27e. and *note fileinput.FileInput: 27f. + have been deprecated. + + Implementations of these methods have been ignoring their `index' + parameter, and returning the next item instead. (Contributed by + Berker Peksag in bpo-9372(4).) + + * The *note typing.NamedTuple: 280. class has deprecated the + ‘_field_types’ attribute in favor of the ‘__annotations__’ + attribute which has the same information. (Contributed by Raymond + Hettinger in bpo-36320(5).) + + * *note ast: 8. classes ‘Num’, ‘Str’, ‘Bytes’, ‘NameConstant’ and + ‘Ellipsis’ are considered deprecated and will be removed in future + Python versions. ‘Constant’ should be used instead. (Contributed + by Serhiy Storchaka in bpo-32892(6).) + + * *note ast.NodeVisitor: 281. methods ‘visit_Num()’, ‘visit_Str()’, + ‘visit_Bytes()’, ‘visit_NameConstant()’ and ‘visit_Ellipsis()’ are + deprecated now and will not be called in future Python versions. + Add the ‘visit_Constant()’ method to handle all constant nodes. + (Contributed by Serhiy Storchaka in bpo-36917(7).) + + * The *note asyncio.coroutine(): 282. *note decorator: 283. is + deprecated and will be removed in version 3.10. Instead of + ‘@asyncio.coroutine’, use *note async def: 284. instead. + (Contributed by Andrew Svetlov in bpo-36921(8).) + + * In *note asyncio: a, the explicit passing of a `loop' argument has + been deprecated and will be removed in version 3.10 for the + following: *note asyncio.sleep(): 285, *note asyncio.gather(): 286, + *note asyncio.shield(): 287, *note asyncio.wait_for(): 288, *note + asyncio.wait(): 289, *note asyncio.as_completed(): 28a, *note + asyncio.Task: 1ad, *note asyncio.Lock: 28b, *note asyncio.Event: + 28c, *note asyncio.Condition: 28d, *note asyncio.Semaphore: 28e, + *note asyncio.BoundedSemaphore: 28f, *note asyncio.Queue: 290, + *note asyncio.create_subprocess_exec(): 291, and *note + asyncio.create_subprocess_shell(): 292. + + * The explicit passing of coroutine objects to *note asyncio.wait(): + 289. has been deprecated and will be removed in version 3.11. + (Contributed by Yury Selivanov in bpo-34790(9).) + + * The following functions and methods are deprecated in the *note + gettext: 89. module: *note lgettext(): 293, *note ldgettext(): 294, + *note lngettext(): 295. and *note ldngettext(): 296. They return + encoded bytes, and it’s possible that you will get unexpected + Unicode-related exceptions if there are encoding problems with the + translated strings. It’s much better to use alternatives which + return Unicode strings in Python 3. These functions have been + broken for a long time. + + Function *note bind_textdomain_codeset(): 297, methods *note + output_charset(): 298. and *note set_output_charset(): 299, and the + `codeset' parameter of functions *note translation(): 29a. and + *note install(): 29b. are also deprecated, since they are only used + for the ‘l*gettext()’ functions. (Contributed by Serhiy Storchaka + in bpo-33710(10).) + + * The ‘isAlive()’ method of *note threading.Thread: 235. has been + deprecated. (Contributed by Dong-hee Na in bpo-35283(11).) + + * Many builtin and extension functions that take integer arguments + will now emit a deprecation warning for *note Decimal: 188.s, *note + Fraction: 185.s and any other objects that can be converted to + integers only with a loss (e.g. that have the *note __int__(): + 18b. method but do not have the *note __index__(): 18a. method). + In future version they will be errors. (Contributed by Serhiy + Storchaka in bpo-36048(12).) + + * Deprecated passing the following arguments as keyword arguments: + + - `func' in *note functools.partialmethod(): 29c, *note + weakref.finalize(): 29d, *note profile.Profile.runcall(): 29e, + ‘cProfile.Profile.runcall()’, *note bdb.Bdb.runcall(): 29f, + *note trace.Trace.runfunc(): 2a0. and *note curses.wrapper(): + 2a1. + + - `function' in *note unittest.TestCase.addCleanup(): 2a2. + + - `fn' in the *note submit(): 2a3. method of *note + concurrent.futures.ThreadPoolExecutor: 27a. and *note + concurrent.futures.ProcessPoolExecutor: 2a4. + + - `callback' in *note contextlib.ExitStack.callback(): 2a5, + ‘contextlib.AsyncExitStack.callback()’ and *note + contextlib.AsyncExitStack.push_async_callback(): 2a6. + + - `c' and `typeid' in the ‘create()’ method of + ‘multiprocessing.managers.Server’ and + ‘multiprocessing.managers.SharedMemoryServer’. + + - `obj' in *note weakref.finalize(): 29d. + + In future releases of Python, they will be *note positional-only: + 2a7. (Contributed by Serhiy Storchaka in bpo-36492(13).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue37481 + + (2) https://bugs.python.org/issue29209 + + (3) https://bugs.python.org/issue34075 + + (4) https://bugs.python.org/issue9372 + + (5) https://bugs.python.org/issue36320 + + (6) https://bugs.python.org/issue32892 + + (7) https://bugs.python.org/issue36917 + + (8) https://bugs.python.org/issue36921 + + (9) https://bugs.python.org/issue34790 + + (10) https://bugs.python.org/issue33710 + + (11) https://bugs.python.org/issue35283 + + (12) https://bugs.python.org/issue36048 + + (13) https://bugs.python.org/issue36492 + + +File: python.info, Node: API and Feature Removals, Next: Porting to Python 3 8, Prev: Deprecated, Up: What’s New In Python 3 8 + +1.1.9 API and Feature Removals +------------------------------ + +The following features and APIs have been removed from Python 3.8: + + * Starting with Python 3.3, importing ABCs from *note collections: + 1e. was deprecated, and importing should be done from *note + collections.abc: 1f. Being able to import from collections was + marked for removal in 3.8, but has been delayed to 3.9. (See + bpo-36952(1).) + + * The ‘macpath’ module, deprecated in Python 3.7, has been removed. + (Contributed by Victor Stinner in bpo-35471(2).) + + * The function ‘platform.popen()’ has been removed, after having been + deprecated since Python 3.3: use *note os.popen(): 2a9. instead. + (Contributed by Victor Stinner in bpo-35345(3).) + + * The function ‘time.clock()’ has been removed, after having been + deprecated since Python 3.3: use *note time.perf_counter(): 2aa. or + *note time.process_time(): 2ab. instead, depending on your + requirements, to have well-defined behavior. (Contributed by + Matthias Bussonnier in bpo-36895(4).) + + * The ‘pyvenv’ script has been removed in favor of ‘python3.8 -m + venv’ to help eliminate confusion as to what Python interpreter the + ‘pyvenv’ script is tied to. (Contributed by Brett Cannon in + bpo-25427(5).) + + * ‘parse_qs’, ‘parse_qsl’, and ‘escape’ are removed from the *note + cgi: 16. module. They are deprecated in Python 3.2 or older. They + should be imported from the ‘urllib.parse’ and ‘html’ modules + instead. + + * ‘filemode’ function is removed from the *note tarfile: 101. module. + It is not documented and deprecated since Python 3.3. + + * The *note XMLParser: 252. constructor no longer accepts the `html' + argument. It never had an effect and was deprecated in Python 3.4. + All other parameters are now *note keyword-only: 2ac. (Contributed + by Serhiy Storchaka in bpo-29209(6).) + + * Removed the ‘doctype()’ method of *note XMLParser: 252. + (Contributed by Serhiy Storchaka in bpo-29209(7).) + + * “unicode_internal” codec is removed. (Contributed by Inada Naoki + in bpo-36297(8).) + + * The ‘Cache’ and ‘Statement’ objects of the *note sqlite3: f2. + module are not exposed to the user. (Contributed by Aviv Palivoda + in bpo-30262(9).) + + * The ‘bufsize’ keyword argument of *note fileinput.input(): 2ad. and + *note fileinput.FileInput(): 27f. which was ignored and deprecated + since Python 3.6 has been removed. bpo-36952(10) (Contributed by + Matthias Bussonnier.) + + * The functions ‘sys.set_coroutine_wrapper()’ and + ‘sys.get_coroutine_wrapper()’ deprecated in Python 3.7 have been + removed; bpo-36933(11) (Contributed by Matthias Bussonnier.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue36952 + + (2) https://bugs.python.org/issue35471 + + (3) https://bugs.python.org/issue35345 + + (4) https://bugs.python.org/issue36895 + + (5) https://bugs.python.org/issue25427 + + (6) https://bugs.python.org/issue29209 + + (7) https://bugs.python.org/issue29209 + + (8) https://bugs.python.org/issue36297 + + (9) https://bugs.python.org/issue30262 + + (10) https://bugs.python.org/issue36952 + + (11) https://bugs.python.org/issue36933 + + +File: python.info, Node: Porting to Python 3 8, Next: Notable changes in Python 3 8 1, Prev: API and Feature Removals, Up: What’s New In Python 3 8 + +1.1.10 Porting to Python 3.8 +---------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code. + +* Menu: + +* Changes in Python behavior:: +* Changes in the Python API:: +* Changes in the C API:: +* CPython bytecode changes:: +* Demos and Tools:: + + +File: python.info, Node: Changes in Python behavior, Next: Changes in the Python API, Up: Porting to Python 3 8 + +1.1.10.1 Changes in Python behavior +................................... + + * Yield expressions (both ‘yield’ and ‘yield from’ clauses) are now + disallowed in comprehensions and generator expressions (aside from + the iterable expression in the leftmost ‘for’ clause). + (Contributed by Serhiy Storchaka in bpo-10544(1).) + + * The compiler now produces a *note SyntaxWarning: 191. when identity + checks (‘is’ and ‘is not’) are used with certain types of literals + (e.g. strings, numbers). These can often work by accident in + CPython, but are not guaranteed by the language spec. The warning + advises users to use equality tests (‘==’ and ‘!=’) instead. + (Contributed by Serhiy Storchaka in bpo-34850(2).) + + * The CPython interpreter can swallow exceptions in some + circumstances. In Python 3.8 this happens in fewer cases. In + particular, exceptions raised when getting the attribute from the + type dictionary are no longer ignored. (Contributed by Serhiy + Storchaka in bpo-35459(3).) + + * Removed ‘__str__’ implementations from builtin types *note bool: + 183, *note int: 184, *note float: 187, *note complex: 189. and few + classes from the standard library. They now inherit ‘__str__()’ + from *note object: 2b0. As result, defining the ‘__repr__()’ + method in the subclass of these classes will affect their string + representation. (Contributed by Serhiy Storchaka in bpo-36793(4).) + + * On AIX, *note sys.platform: 2b1. doesn’t contain the major version + anymore. It is always ‘'aix'’, instead of ‘'aix3'’ .. ‘'aix7'’. + Since older Python versions include the version number, so it is + recommended to always use ‘sys.platform.startswith('aix')’. + (Contributed by M. Felt in bpo-36588(5).) + + * *note PyEval_AcquireLock(): 2b2. and *note PyEval_AcquireThread(): + 2b3. now terminate the current thread if called while the + interpreter is finalizing, making them consistent with *note + PyEval_RestoreThread(): 2b4, *note Py_END_ALLOW_THREADS(): 2b5, and + *note PyGILState_Ensure(): 2b6. If this behavior is not desired, + guard the call by checking ‘_Py_IsFinalizing()’ or + ‘sys.is_finalizing()’. (Contributed by Joannah Nanjekye in + bpo-36475(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10544 + + (2) https://bugs.python.org/issue34850 + + (3) https://bugs.python.org/issue35459 + + (4) https://bugs.python.org/issue36793 + + (5) https://bugs.python.org/issue36588 + + (6) https://bugs.python.org/issue36475 + + +File: python.info, Node: Changes in the Python API, Next: Changes in the C API, Prev: Changes in Python behavior, Up: Porting to Python 3 8 + +1.1.10.2 Changes in the Python API +.................................. + + * The *note os.getcwdb(): 2b8. function now uses the UTF-8 encoding + on Windows, rather than the ANSI code page: see PEP 529(1) for the + rationale. The function is no longer deprecated on Windows. + (Contributed by Victor Stinner in bpo-37412(2).) + + * *note subprocess.Popen: 2b9. can now use *note os.posix_spawn(): + 257. in some cases for better performance. On Windows Subsystem + for Linux and QEMU User Emulation, the ‘Popen’ constructor using + *note os.posix_spawn(): 257. no longer raises an exception on + errors like “missing program”. Instead the child process fails + with a non-zero ‘returncode’. (Contributed by Joannah Nanjekye and + Victor Stinner in bpo-35537(3).) + + * The `preexec_fn' argument of * *note subprocess.Popen: 2b9. is no + longer compatible with subinterpreters. The use of the parameter + in a subinterpreter now raises *note RuntimeError: 2ba. + (Contributed by Eric Snow in bpo-34651(4), modified by Christian + Heimes in bpo-37951(5).) + + * The ‘imap.IMAP4.logout()’ method no longer silently ignores + arbitrary exceptions. (Contributed by Victor Stinner in + bpo-36348(6).) + + * The function ‘platform.popen()’ has been removed, after having been + deprecated since Python 3.3: use *note os.popen(): 2a9. instead. + (Contributed by Victor Stinner in bpo-35345(7).) + + * The *note statistics.mode(): 2bb. function no longer raises an + exception when given multimodal data. Instead, it returns the + first mode encountered in the input data. (Contributed by Raymond + Hettinger in bpo-35892(8).) + + * The *note selection(): 2bc. method of the *note + tkinter.ttk.Treeview: 2bd. class no longer takes arguments. Using + it with arguments for changing the selection was deprecated in + Python 3.6. Use specialized methods like *note selection_set(): + 2be. for changing the selection. (Contributed by Serhiy Storchaka + in bpo-31508(9).) + + * The ‘writexml()’, ‘toxml()’ and ‘toprettyxml()’ methods of *note + xml.dom.minidom: 135, and the ‘write()’ method of ‘xml.etree’, now + preserve the attribute order specified by the user. (Contributed + by Diego Rojas and Raymond Hettinger in bpo-34160(10).) + + * A *note dbm.dumb: 33. database opened with flags ‘'r'’ is now + read-only. *note dbm.dumb.open(): 2bf. with flags ‘'r'’ and ‘'w'’ + no longer creates a database if it does not exist. (Contributed by + Serhiy Storchaka in bpo-32749(11).) + + * The ‘doctype()’ method defined in a subclass of *note XMLParser: + 252. will no longer be called and will emit a *note RuntimeWarning: + 2c0. instead of a *note DeprecationWarning: 278. Define the *note + doctype(): 2c1. method on a target for handling an XML doctype + declaration. (Contributed by Serhiy Storchaka in bpo-29209(12).) + + * A *note RuntimeError: 2ba. is now raised when the custom metaclass + doesn’t provide the ‘__classcell__’ entry in the namespace passed + to ‘type.__new__’. A *note DeprecationWarning: 278. was emitted in + Python 3.6–3.7. (Contributed by Serhiy Storchaka in + bpo-23722(13).) + + * The ‘cProfile.Profile’ class can now be used as a context manager. + (Contributed by Scott Sanderson in bpo-29235(14).) + + * *note shutil.copyfile(): 258, *note shutil.copy(): 259, *note + shutil.copy2(): 25a, *note shutil.copytree(): 21a. and *note + shutil.move(): 25b. use platform-specific “fast-copy” syscalls (see + *note Platform-dependent efficient copy operations: 25e. section). + + * *note shutil.copyfile(): 258. default buffer size on Windows was + changed from 16 KiB to 1 MiB. + + * The ‘PyGC_Head’ struct has changed completely. All code that + touched the struct member should be rewritten. (See + bpo-33597(15).) + + * The *note PyInterpreterState: 2c2. struct has been moved into the + “internal” header files (specifically + Include/internal/pycore_pystate.h). An opaque ‘PyInterpreterState’ + is still available as part of the public API (and stable ABI). The + docs indicate that none of the struct’s fields are public, so we + hope no one has been using them. However, if you do rely on one or + more of those private fields and have no alternative then please + open a BPO issue. We’ll work on helping you adjust (possibly + including adding accessor functions to the public API). (See + bpo-35886(16).) + + * The *note mmap.flush(): 2c3. method now returns ‘None’ on success + and raises an exception on error under all platforms. Previously, + its behavior was platform-dependent: a nonzero value was returned + on success; zero was returned on error under Windows. A zero value + was returned on success; an exception was raised on error under + Unix. (Contributed by Berker Peksag in bpo-2122(17).) + + * *note xml.dom.minidom: 135. and *note xml.sax: 13b. modules no + longer process external entities by default. (Contributed by + Christian Heimes in bpo-17239(18).) + + * Deleting a key from a read-only *note dbm: 32. database (*note + dbm.dumb: 33, *note dbm.gnu: 34. or *note dbm.ndbm: 35.) raises + ‘error’ (*note dbm.dumb.error: 2c4, *note dbm.gnu.error: 2c5. or + *note dbm.ndbm.error: 2c6.) instead of *note KeyError: 2c7. + (Contributed by Xiang Zhang in bpo-33106(19).) + + * Simplified AST for literals. All constants will be represented as + ‘ast.Constant’ instances. Instantiating old classes ‘Num’, ‘Str’, + ‘Bytes’, ‘NameConstant’ and ‘Ellipsis’ will return an instance of + ‘Constant’. (Contributed by Serhiy Storchaka in bpo-32892(20).) + + * *note expanduser(): 1fe. on Windows now prefers the ‘USERPROFILE’ + environment variable and does not use ‘HOME’, which is not normally + set for regular user accounts. (Contributed by Anthony Sottile in + bpo-36264(21).) + + * The exception *note asyncio.CancelledError: 1a7. now inherits from + *note BaseException: 1a8. rather than *note Exception: 1a9. and no + longer inherits from *note concurrent.futures.CancelledError: 1aa. + (Contributed by Yury Selivanov in bpo-32528(22).) + + * The function *note asyncio.wait_for(): 288. now correctly waits for + cancellation when using an instance of *note asyncio.Task: 1ad. + Previously, upon reaching `timeout', it was cancelled and + immediately returned. (Contributed by Elvis Pranskevichus in + bpo-32751(23).) + + * The function *note asyncio.BaseTransport.get_extra_info(): 2c8. now + returns a safe to use socket object when ‘socket’ is passed to the + `name' parameter. (Contributed by Yury Selivanov in + bpo-37027(24).) + + * *note asyncio.BufferedProtocol: 2c9. has graduated to the stable + API. + + * DLL dependencies for extension modules and DLLs loaded with *note + ctypes: 2b. on Windows are now resolved more securely. Only the + system paths, the directory containing the DLL or PYD file, and + directories added with *note add_dll_directory(): 1c2. are searched + for load-time dependencies. Specifically, ‘PATH’ and the current + working directory are no longer used, and modifications to these + will no longer have any effect on normal DLL resolution. If your + application relies on these mechanisms, you should check for *note + add_dll_directory(): 1c2. and if it exists, use it to add your DLLs + directory while loading your library. Note that Windows 7 users + will need to ensure that Windows Update KB2533623 has been + installed (this is also verified by the installer). (Contributed + by Steve Dower in bpo-36085(25).) + + * The header files and functions related to pgen have been removed + after its replacement by a pure Python implementation. + (Contributed by Pablo Galindo in bpo-36623(26).) + + * *note types.CodeType: 198. has a new parameter in the second + position of the constructor (`posonlyargcount') to support + positional-only arguments defined in PEP 570(27). The first + argument (`argcount') now represents the total number of positional + arguments (including positional-only arguments). The new + ‘replace()’ method of *note types.CodeType: 198. can be used to + make the code future-proof. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0529 + + (2) https://bugs.python.org/issue37412 + + (3) https://bugs.python.org/issue35537 + + (4) https://bugs.python.org/issue34651 + + (5) https://bugs.python.org/issue37951 + + (6) https://bugs.python.org/issue36348 + + (7) https://bugs.python.org/issue35345 + + (8) https://bugs.python.org/issue35892 + + (9) https://bugs.python.org/issue31508 + + (10) https://bugs.python.org/issue34160 + + (11) https://bugs.python.org/issue32749 + + (12) https://bugs.python.org/issue29209 + + (13) https://bugs.python.org/issue23722 + + (14) https://bugs.python.org/issue29235 + + (15) https://bugs.python.org/issue33597 + + (16) https://bugs.python.org/issue35886 + + (17) https://bugs.python.org/issue2122 + + (18) https://bugs.python.org/issue17239 + + (19) https://bugs.python.org/issue33106 + + (20) https://bugs.python.org/issue32892 + + (21) https://bugs.python.org/issue36264 + + (22) https://bugs.python.org/issue32528 + + (23) https://bugs.python.org/issue32751 + + (24) https://bugs.python.org/issue37027 + + (25) https://bugs.python.org/issue36085 + + (26) https://bugs.python.org/issue36623 + + (27) https://www.python.org/dev/peps/pep-0570 + + +File: python.info, Node: Changes in the C API, Next: CPython bytecode changes, Prev: Changes in the Python API, Up: Porting to Python 3 8 + +1.1.10.3 Changes in the C API +............................. + + * The *note PyCompilerFlags: 2cc. structure got a new + `cf_feature_version' field. It should be initialized to + ‘PY_MINOR_VERSION’. The field is ignored by default, and is used + if and only if ‘PyCF_ONLY_AST’ flag is set in `cf_flags'. + (Contributed by Guido van Rossum in bpo-35766(1).) + + * The ‘PyEval_ReInitThreads()’ function has been removed from the C + API. It should not be called explicitly: use *note + PyOS_AfterFork_Child(): 2cd. instead. (Contributed by Victor + Stinner in bpo-36728(2).) + + * On Unix, C extensions are no longer linked to libpython except on + Android and Cygwin. When Python is embedded, ‘libpython’ must not + be loaded with ‘RTLD_LOCAL’, but ‘RTLD_GLOBAL’ instead. + Previously, using ‘RTLD_LOCAL’, it was already not possible to load + C extensions which were not linked to ‘libpython’, like C + extensions of the standard library built by the ‘*shared*’ section + of ‘Modules/Setup’. (Contributed by Victor Stinner in + bpo-21536(3).) + + * Use of ‘#’ variants of formats in parsing or building value (e.g. + *note PyArg_ParseTuple(): 26a, *note Py_BuildValue(): 2ce, *note + PyObject_CallFunction(): 2cf, etc.) without ‘PY_SSIZE_T_CLEAN’ + defined raises ‘DeprecationWarning’ now. It will be removed in + 3.10 or 4.0. Read *note Parsing arguments and building values: + 2d0. for detail. (Contributed by Inada Naoki in bpo-36381(4).) + + * Instances of heap-allocated types (such as those created with *note + PyType_FromSpec(): 2d1.) hold a reference to their type object. + Increasing the reference count of these type objects has been moved + from *note PyType_GenericAlloc(): 270. to the more low-level + functions, *note PyObject_Init(): 26f. and ‘PyObject_INIT()’. This + makes types created through *note PyType_FromSpec(): 2d1. behave + like other classes in managed code. + + Statically allocated types are not affected. + + For the vast majority of cases, there should be no side effect. + However, types that manually increase the reference count after + allocating an instance (perhaps to work around the bug) may now + become immortal. To avoid this, these classes need to call + Py_DECREF on the type object during instance deallocation. + + To correctly port these types into 3.8, please apply the following + changes: + + * Remove *note Py_INCREF: 265. on the type object after + allocating an instance - if any. This may happen after + calling *note PyObject_New(): 2d2, *note PyObject_NewVar(): + 2d3, *note PyObject_GC_New(): 2d4, *note PyObject_GC_NewVar(): + 2d5, or any other custom allocator that uses *note + PyObject_Init(): 26f. or ‘PyObject_INIT()’. + + Example: + + static foo_struct * + foo_new(PyObject *type) { + foo_struct *foo = PyObject_GC_New(foo_struct, (PyTypeObject *) type); + if (foo == NULL) + return NULL; + #if PY_VERSION_HEX < 0x03080000 + // Workaround for Python issue 35810; no longer necessary in Python 3.8 + PY_INCREF(type) + #endif + return foo; + } + + * Ensure that all custom ‘tp_dealloc’ functions of + heap-allocated types decrease the type’s reference count. + + Example: + + static void + foo_dealloc(foo_struct *instance) { + PyObject *type = Py_TYPE(instance); + PyObject_GC_Del(instance); + #if PY_VERSION_HEX >= 0x03080000 + // This was not needed before Python 3.8 (Python issue 35810) + Py_DECREF(type); + #endif + } + + (Contributed by Eddie Elizondo in bpo-35810(5).) + + * The ‘Py_DEPRECATED()’ macro has been implemented for MSVC. The + macro now must be placed before the symbol name. + + Example: + + Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void); + + (Contributed by Zackery Spytz in bpo-33407(6).) + + * The interpreter does not pretend to support binary compatibility of + extension types across feature releases, anymore. A *note + PyTypeObject: 2d6. exported by a third-party extension module is + supposed to have all the slots expected in the current Python + version, including *note tp_finalize: 2d7. (*note + Py_TPFLAGS_HAVE_FINALIZE: 2d8. is not checked anymore before + reading *note tp_finalize: 2d7.). + + (Contributed by Antoine Pitrou in bpo-32388(7).) + + * The functions ‘PyNode_AddChild()’ and ‘PyParser_AddToken()’ now + accept two additional ‘int’ arguments `end_lineno' and + `end_col_offset'. + + * The ‘libpython38.a’ file to allow MinGW tools to link directly + against ‘python38.dll’ is no longer included in the regular Windows + distribution. If you require this file, it may be generated with + the ‘gendef’ and ‘dlltool’ tools, which are part of the MinGW + binutils package: + + gendef - python38.dll > tmp.def + dlltool --dllname python38.dll --def tmp.def --output-lib libpython38.a + + The location of an installed ‘pythonXY.dll’ will depend on the + installation options and the version and language of Windows. See + *note Using Python on Windows: 2d9. for more information. The + resulting library should be placed in the same directory as + ‘pythonXY.lib’, which is generally the ‘libs’ directory under your + Python installation. + + (Contributed by Steve Dower in bpo-37351(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35766 + + (2) https://bugs.python.org/issue36728 + + (3) https://bugs.python.org/issue21536 + + (4) https://bugs.python.org/issue36381 + + (5) https://bugs.python.org/issue35810 + + (6) https://bugs.python.org/issue33407 + + (7) https://bugs.python.org/issue32388 + + (8) https://bugs.python.org/issue37351 + + +File: python.info, Node: CPython bytecode changes, Next: Demos and Tools, Prev: Changes in the C API, Up: Porting to Python 3 8 + +1.1.10.4 CPython bytecode changes +................................. + + * The interpreter loop has been simplified by moving the logic of + unrolling the stack of blocks into the compiler. The compiler + emits now explicit instructions for adjusting the stack of values + and calling the cleaning-up code for *note break: 2db, *note + continue: 181. and *note return: 190. + + Removed opcodes ‘BREAK_LOOP’, ‘CONTINUE_LOOP’, ‘SETUP_LOOP’ and + ‘SETUP_EXCEPT’. Added new opcodes *note ROT_FOUR: 2dc, *note + BEGIN_FINALLY: 2dd, *note CALL_FINALLY: 2de. and *note POP_FINALLY: + 2df. Changed the behavior of *note END_FINALLY: 2e0. and *note + WITH_CLEANUP_START: 2e1. + + (Contributed by Mark Shannon, Antoine Pitrou and Serhiy Storchaka + in bpo-17611(1).) + + * Added new opcode *note END_ASYNC_FOR: 2e2. for handling exceptions + raised when awaiting a next item in an *note async for: 2e3. loop. + (Contributed by Serhiy Storchaka in bpo-33041(2).) + + * The *note MAP_ADD: 2e4. now expects the value as the first element + in the stack and the key as the second element. This change was + made so the key is always evaluated before the value in dictionary + comprehensions, as proposed by PEP 572(3). (Contributed by Jörn + Heissler in bpo-35224(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17611 + + (2) https://bugs.python.org/issue33041 + + (3) https://www.python.org/dev/peps/pep-0572 + + (4) https://bugs.python.org/issue35224 + + +File: python.info, Node: Demos and Tools, Prev: CPython bytecode changes, Up: Porting to Python 3 8 + +1.1.10.5 Demos and Tools +........................ + +Added a benchmark script for timing various ways to access variables: +‘Tools/scripts/var_access_benchmark.py’. (Contributed by Raymond +Hettinger in bpo-35884(1).) + +Here’s a summary of performance improvements since Python 3.3: + + Python version 3.3 3.4 3.5 3.6 3.7 3.8 + -------------- --- --- --- --- --- --- + + Variable and attribute read access: + read_local 4.0 7.1 7.1 5.4 5.1 3.9 + read_nonlocal 5.3 7.1 8.1 5.8 5.4 4.4 + read_global 13.3 15.5 19.0 14.3 13.6 7.6 + read_builtin 20.0 21.1 21.6 18.5 19.0 7.5 + read_classvar_from_class 20.5 25.6 26.5 20.7 19.5 18.4 + read_classvar_from_instance 18.5 22.8 23.5 18.8 17.1 16.4 + read_instancevar 26.8 32.4 33.1 28.0 26.3 25.4 + read_instancevar_slots 23.7 27.8 31.3 20.8 20.8 20.2 + read_namedtuple 68.5 73.8 57.5 45.0 46.8 18.4 + read_boundmethod 29.8 37.6 37.9 29.6 26.9 27.7 + + Variable and attribute write access: + write_local 4.6 8.7 9.3 5.5 5.3 4.3 + write_nonlocal 7.3 10.5 11.1 5.6 5.5 4.7 + write_global 15.9 19.7 21.2 18.0 18.0 15.8 + write_classvar 81.9 92.9 96.0 104.6 102.1 39.2 + write_instancevar 36.4 44.6 45.8 40.0 38.9 35.5 + write_instancevar_slots 28.7 35.6 36.1 27.3 26.6 25.7 + + Data structure read access: + read_list 19.2 24.2 24.5 20.8 20.8 19.0 + read_deque 19.9 24.7 25.5 20.2 20.6 19.8 + read_dict 19.7 24.3 25.7 22.3 23.0 21.0 + read_strdict 17.9 22.6 24.3 19.5 21.2 18.9 + + Data structure write access: + write_list 21.2 27.1 28.5 22.5 21.6 20.0 + write_deque 23.8 28.7 30.1 22.7 21.8 23.5 + write_dict 25.9 31.4 33.3 29.3 29.2 24.7 + write_strdict 22.9 28.4 29.9 27.5 25.2 23.1 + + Stack (or queue) operations: + list_append_pop 144.2 93.4 112.7 75.4 74.2 50.8 + deque_append_pop 30.4 43.5 57.0 49.4 49.2 42.5 + deque_append_popleft 30.8 43.7 57.3 49.7 49.7 42.8 + + Timing loop: + loop_overhead 0.3 0.5 0.6 0.4 0.3 0.3 + +The benchmarks were measured on an Intel® Core™ i7-4960HQ processor(2) +running the macOS 64-bit builds found at python.org(3). The benchmark +script displays timings in nanoseconds. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue35884 + + (2) +https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html + + (3) https://www.python.org/downloads/mac-osx/ + + +File: python.info, Node: Notable changes in Python 3 8 1, Next: Notable changes in Python 3 8 2, Prev: Porting to Python 3 8, Up: What’s New In Python 3 8 + +1.1.11 Notable changes in Python 3.8.1 +-------------------------------------- + +Due to significant security concerns, the `reuse_address' parameter of +*note asyncio.loop.create_datagram_endpoint(): 2e7. is no longer +supported. This is because of the behavior of the socket option +‘SO_REUSEADDR’ in UDP. For more details, see the documentation for +‘loop.create_datagram_endpoint()’. (Contributed by Kyle Stanley, +Antoine Pitrou, and Yury Selivanov in bpo-37228(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue37228 + + +File: python.info, Node: Notable changes in Python 3 8 2, Next: Notable changes in Python 3 8 3, Prev: Notable changes in Python 3 8 1, Up: What’s New In Python 3 8 + +1.1.12 Notable changes in Python 3.8.2 +-------------------------------------- + +Fixed a regression with the ‘ignore’ callback of *note +shutil.copytree(): 21a. The argument types are now str and List[str] +again. (Contributed by Manuel Barkhau and Giampaolo Rodola in +bpo-39390(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue39390 + + +File: python.info, Node: Notable changes in Python 3 8 3, Next: Notable changes in Python 3 8 8, Prev: Notable changes in Python 3 8 2, Up: What’s New In Python 3 8 + +1.1.13 Notable changes in Python 3.8.3 +-------------------------------------- + +The constant values of future flags in the *note __future__: 0. module +are updated in order to prevent collision with compiler flags. +Previously ‘PyCF_ALLOW_TOP_LEVEL_AWAIT’ was clashing with +‘CO_FUTURE_DIVISION’. (Contributed by Batuhan Taskaya in bpo-39562(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue39562 + + +File: python.info, Node: Notable changes in Python 3 8 8, Next: Notable changes in Python 3 8 9, Prev: Notable changes in Python 3 8 3, Up: What’s New In Python 3 8 + +1.1.14 Notable changes in Python 3.8.8 +-------------------------------------- + +Earlier Python versions allowed using both ‘;’ and ‘&’ as query +parameter separators in *note urllib.parse.parse_qs(): 2eb. and *note +urllib.parse.parse_qsl(): 2ec. Due to security concerns, and to conform +with newer W3C recommendations, this has been changed to allow only a +single separator key, with ‘&’ as the default. This change also affects +*note cgi.parse(): 2ed. and *note cgi.parse_multipart(): 2ee. as they +use the affected functions internally. For more details, please see +their respective documentation. (Contributed by Adam Goldschmidt, +Senthil Kumaran and Ken Jin in bpo-42967(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue42967 + + +File: python.info, Node: Notable changes in Python 3 8 9, Prev: Notable changes in Python 3 8 8, Up: What’s New In Python 3 8 + +1.1.15 Notable changes in Python 3.8.9 +-------------------------------------- + +A security fix alters the *note ftplib.FTP: 2f0. behavior to not trust +the IPv4 address sent from the remote server when setting up a passive +data channel. We reuse the ftp server IP address instead. For unusual +code requiring the old behavior, set a ‘trust_server_pasv_ipv4_address’ +attribute on your FTP instance to ‘True’. (See bpo-43285(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue43285 + + +File: python.info, Node: What’s New In Python 3 7, Next: What’s New In Python 3 6, Prev: What’s New In Python 3 8, Up: What’s New in Python + +1.2 What’s New In Python 3.7 +============================ + + +Editor: Elvis Pranskevichus <> + +This article explains the new features in Python 3.7, compared to 3.6. +Python 3.7 was released on June 27, 2018. For full details, see the +*note changelog: 14c. + +* Menu: + +* Summary – Release Highlights:: +* New Features: New Features<2>. +* Other Language Changes: Other Language Changes<2>. +* New Modules: New Modules<2>. +* Improved Modules: Improved Modules<2>. +* C API Changes:: +* Build Changes:: +* Optimizations: Optimizations<2>. +* Other CPython Implementation Changes:: +* Deprecated Python Behavior:: +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods. +* Deprecated functions and types of the C API:: +* Platform Support Removals:: +* API and Feature Removals: API and Feature Removals<2>. +* Module Removals:: +* Windows-only Changes:: +* Porting to Python 3.7: Porting to Python 3 7. +* Notable changes in Python 3.7.1: Notable changes in Python 3 7 1. +* Notable changes in Python 3.7.2: Notable changes in Python 3 7 2. +* Notable changes in Python 3.7.6: Notable changes in Python 3 7 6. +* Notable changes in Python 3.7.10: Notable changes in Python 3 7 10. + + +File: python.info, Node: Summary – Release Highlights, Next: New Features<2>, Up: What’s New In Python 3 7 + +1.2.1 Summary – Release Highlights +---------------------------------- + +New syntax features: + + * *note PEP 563: 2f4, postponed evaluation of type annotations. + +Backwards incompatible syntax changes: + + * *note async: 1a2. and *note await: 1a3. are now reserved keywords. + +New library modules: + + * *note contextvars: 25.: *note PEP 567 – Context Variables: 2f5. + + * *note dataclasses: 30.: *note PEP 557 – Data Classes: 2f6. + + * *note importlib.resources: 2f7. + +New built-in features: + + * *note PEP 553: 2f8, the new *note breakpoint(): 2f9. function. + +Python data model improvements: + + * *note PEP 562: 2fa, customization of access to module attributes. + + * *note PEP 560: 2fb, core support for typing module and generic + types. + + * the insertion-order preservation nature of *note dict: 2fc. objects + has been declared(1) to be an official part of the Python language + spec. + +Significant improvements in the standard library: + + * The *note asyncio: a. module has received new features, significant + *note usability and performance improvements: 2fd. + + * The *note time: 10a. module gained support for *note functions with + nanosecond resolution: 2fe. + +CPython implementation improvements: + + * Avoiding the use of ASCII as a default text encoding: + + * *note PEP 538: 2ff, legacy C locale coercion + + * *note PEP 540: 300, forced UTF-8 runtime mode + + * *note PEP 552: 301, deterministic .pycs + + * *note the new development runtime mode: 302. + + * *note PEP 565: 303, improved *note DeprecationWarning: 278. + handling + +C API improvements: + + * *note PEP 539: 304, new C API for thread-local storage + +Documentation improvements: + + * *note PEP 545: 305, Python documentation translations + + * New documentation translations: Japanese(2), French(3), and + Korean(4). + +This release features notable performance improvements in many areas. +The *note Optimizations: 306. section lists them in detail. + +For a list of changes that may affect compatibility with previous Python +releases please refer to the *note Porting to Python 3.7: 307. section. + + ---------- Footnotes ---------- + + (1) +https://mail.python.org/pipermail/python-dev/2017-December/151283.html + + (2) https://docs.python.org/ja/ + + (3) https://docs.python.org/fr/ + + (4) https://docs.python.org/ko/ + + +File: python.info, Node: New Features<2>, Next: Other Language Changes<2>, Prev: Summary – Release Highlights, Up: What’s New In Python 3 7 + +1.2.2 New Features +------------------ + +* Menu: + +* PEP 563; Postponed Evaluation of Annotations: PEP 563 Postponed Evaluation of Annotations. +* PEP 538; Legacy C Locale Coercion: PEP 538 Legacy C Locale Coercion. +* PEP 540; Forced UTF-8 Runtime Mode: PEP 540 Forced UTF-8 Runtime Mode. +* PEP 553; Built-in breakpoint(): PEP 553 Built-in breakpoint. +* PEP 539; New C API for Thread-Local Storage: PEP 539 New C API for Thread-Local Storage. +* PEP 562; Customization of Access to Module Attributes: PEP 562 Customization of Access to Module Attributes. +* PEP 564; New Time Functions With Nanosecond Resolution: PEP 564 New Time Functions With Nanosecond Resolution. +* PEP 565; Show DeprecationWarning in __main__: PEP 565 Show DeprecationWarning in __main__. +* PEP 560; Core Support for typing module and Generic Types: PEP 560 Core Support for typing module and Generic Types. +* PEP 552; Hash-based .pyc Files: PEP 552 Hash-based pyc Files. +* PEP 545; Python Documentation Translations: PEP 545 Python Documentation Translations. +* Development Runtime Mode; -X dev: Development Runtime Mode -X dev. + + +File: python.info, Node: PEP 563 Postponed Evaluation of Annotations, Next: PEP 538 Legacy C Locale Coercion, Up: New Features<2> + +1.2.2.1 PEP 563: Postponed Evaluation of Annotations +.................................................... + +The advent of type hints in Python uncovered two glaring usability +issues with the functionality of annotations added in PEP 3107(1) and +refined further in PEP 526(2): + + * annotations could only use names which were already available in + the current scope, in other words they didn’t support forward + references of any kind; and + + * annotating source code had adverse effects on startup time of + Python programs. + +Both of these issues are fixed by postponing the evaluation of +annotations. Instead of compiling code which executes expressions in +annotations at their definition time, the compiler stores the annotation +in a string form equivalent to the AST of the expression in question. +If needed, annotations can be resolved at runtime using *note +typing.get_type_hints(): 30a. In the common case where this is not +required, the annotations are cheaper to store (since short strings are +interned by the interpreter) and make startup time faster. + +Usability-wise, annotations now support forward references, making the +following syntax valid: + + class C: + @classmethod + def from_string(cls, source: str) -> C: + ... + + def validate_b(self, obj: B) -> bool: + ... + + class B: + ... + +Since this change breaks compatibility, the new behavior needs to be +enabled on a per-module basis in Python 3.7 using a *note __future__: 0. +import: + + from __future__ import annotations + +It will become the default in Python 3.10. + +See also +........ + +PEP 563(3) – Postponed evaluation of annotations + + PEP written and implemented by Łukasz Langa. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3107 + + (2) https://www.python.org/dev/peps/pep-0526 + + (3) https://www.python.org/dev/peps/pep-0563 + + +File: python.info, Node: PEP 538 Legacy C Locale Coercion, Next: PEP 540 Forced UTF-8 Runtime Mode, Prev: PEP 563 Postponed Evaluation of Annotations, Up: New Features<2> + +1.2.2.2 PEP 538: Legacy C Locale Coercion +......................................... + +An ongoing challenge within the Python 3 series has been determining a +sensible default strategy for handling the “7-bit ASCII” text encoding +assumption currently implied by the use of the default C or POSIX locale +on non-Windows platforms. + +PEP 538(1) updates the default interpreter command line interface to +automatically coerce that locale to an available UTF-8 based locale as +described in the documentation of the new *note PYTHONCOERCECLOCALE: +30c. environment variable. Automatically setting ‘LC_CTYPE’ this way +means that both the core interpreter and locale-aware C extensions (such +as *note readline: de.) will assume the use of UTF-8 as the default text +encoding, rather than ASCII. + +The platform support definition in PEP 11(2) has also been updated to +limit full text handling support to suitably configured non-ASCII based +locales. + +As part of this change, the default error handler for *note stdin: 30d. +and *note stdout: 30e. is now ‘surrogateescape’ (rather than ‘strict’) +when using any of the defined coercion target locales (currently +‘C.UTF-8’, ‘C.utf8’, and ‘UTF-8’). The default error handler for *note +stderr: 30f. continues to be ‘backslashreplace’, regardless of locale. + +Locale coercion is silent by default, but to assist in debugging +potentially locale related integration problems, explicit warnings +(emitted directly on *note stderr: 30f.) can be requested by setting +‘PYTHONCOERCECLOCALE=warn’. This setting will also cause the Python +runtime to emit a warning if the legacy C locale remains active when the +core interpreter is initialized. + +While PEP 538(3)’s locale coercion has the benefit of also affecting +extension modules (such as GNU ‘readline’), as well as child processes +(including those running non-Python applications and older versions of +Python), it has the downside of requiring that a suitable target locale +be present on the running system. To better handle the case where no +suitable target locale is available (as occurs on RHEL/CentOS 7, for +example), Python 3.7 also implements *note PEP 540; Forced UTF-8 Runtime +Mode: 300. + +See also +........ + +PEP 538(4) – Coercing the legacy C locale to a UTF-8 based locale + + PEP written and implemented by Nick Coghlan. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0538 + + (2) https://www.python.org/dev/peps/pep-0011 + + (3) https://www.python.org/dev/peps/pep-0538 + + (4) https://www.python.org/dev/peps/pep-0538 + + +File: python.info, Node: PEP 540 Forced UTF-8 Runtime Mode, Next: PEP 553 Built-in breakpoint, Prev: PEP 538 Legacy C Locale Coercion, Up: New Features<2> + +1.2.2.3 PEP 540: Forced UTF-8 Runtime Mode +.......................................... + +The new *note -X: 155. ‘utf8’ command line option and *note PYTHONUTF8: +311. environment variable can be used to enable the CPython `UTF-8 +mode'. + +When in UTF-8 mode, CPython ignores the locale settings, and uses the +UTF-8 encoding by default. The error handlers for *note sys.stdin: 30d. +and *note sys.stdout: 30e. streams are set to ‘surrogateescape’. + +The forced UTF-8 mode can be used to change the text handling behavior +in an embedded Python interpreter without changing the locale settings +of an embedding application. + +While PEP 540(1)’s UTF-8 mode has the benefit of working regardless of +which locales are available on the running system, it has the downside +of having no effect on extension modules (such as GNU ‘readline’), child +processes running non-Python applications, and child processes running +older versions of Python. To reduce the risk of corrupting text data +when communicating with such components, Python 3.7 also implements +*note PEP 540; Forced UTF-8 Runtime Mode: 300.). + +The UTF-8 mode is enabled by default when the locale is ‘C’ or ‘POSIX’, +and the PEP 538(2) locale coercion feature fails to change it to a UTF-8 +based alternative (whether that failure is due to +‘PYTHONCOERCECLOCALE=0’ being set, ‘LC_ALL’ being set, or the lack of a +suitable target locale). + +See also +........ + +PEP 540(3) – Add a new UTF-8 mode + + PEP written and implemented by Victor Stinner + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0540 + + (2) https://www.python.org/dev/peps/pep-0538 + + (3) https://www.python.org/dev/peps/pep-0540 + + +File: python.info, Node: PEP 553 Built-in breakpoint, Next: PEP 539 New C API for Thread-Local Storage, Prev: PEP 540 Forced UTF-8 Runtime Mode, Up: New Features<2> + +1.2.2.4 PEP 553: Built-in ‘breakpoint()’ +........................................ + +Python 3.7 includes the new built-in *note breakpoint(): 2f9. function +as an easy and consistent way to enter the Python debugger. + +Built-in ‘breakpoint()’ calls *note sys.breakpointhook(): 313. By +default, the latter imports *note pdb: c9. and then calls +‘pdb.set_trace()’, but by binding ‘sys.breakpointhook()’ to the function +of your choosing, ‘breakpoint()’ can enter any debugger. Additionally, +the environment variable *note PYTHONBREAKPOINT: 314. can be set to the +callable of your debugger of choice. Set ‘PYTHONBREAKPOINT=0’ to +completely disable built-in ‘breakpoint()’. + +See also +........ + +PEP 553(1) – Built-in breakpoint() + + PEP written and implemented by Barry Warsaw + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0553 + + +File: python.info, Node: PEP 539 New C API for Thread-Local Storage, Next: PEP 562 Customization of Access to Module Attributes, Prev: PEP 553 Built-in breakpoint, Up: New Features<2> + +1.2.2.5 PEP 539: New C API for Thread-Local Storage +................................................... + +While Python provides a C API for thread-local storage support; the +existing *note Thread Local Storage (TLS) API: 316. has used ‘int’ to +represent TLS keys across all platforms. This has not generally been a +problem for officially-support platforms, but that is neither +POSIX-compliant, nor portable in any practical sense. + +PEP 539(1) changes this by providing a new *note Thread Specific Storage +(TSS) API: 317. to CPython which supersedes use of the existing TLS API +within the CPython interpreter, while deprecating the existing API. The +TSS API uses a new type *note Py_tss_t: 318. instead of ‘int’ to +represent TSS keys–an opaque type the definition of which may depend on +the underlying TLS implementation. Therefore, this will allow to build +CPython on platforms where the native TLS key is defined in a way that +cannot be safely cast to ‘int’. + +Note that on platforms where the native TLS key is defined in a way that +cannot be safely cast to ‘int’, all functions of the existing TLS API +will be no-op and immediately return failure. This indicates clearly +that the old API is not supported on platforms where it cannot be used +reliably, and that no effort will be made to add such support. + +See also +........ + +PEP 539(2) – A New C-API for Thread-Local Storage in CPython + + PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0539 + + (2) https://www.python.org/dev/peps/pep-0539 + + +File: python.info, Node: PEP 562 Customization of Access to Module Attributes, Next: PEP 564 New Time Functions With Nanosecond Resolution, Prev: PEP 539 New C API for Thread-Local Storage, Up: New Features<2> + +1.2.2.6 PEP 562: Customization of Access to Module Attributes +............................................................. + +Python 3.7 allows defining *note __getattr__(): 31a. on modules and will +call it whenever a module attribute is otherwise not found. Defining +*note __dir__(): 31b. on modules is now also allowed. + +A typical example of where this may be useful is module attribute +deprecation and lazy loading. + +See also +........ + +PEP 562(1) – Module ‘__getattr__’ and ‘__dir__’ + + PEP written and implemented by Ivan Levkivskyi + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0562 + + +File: python.info, Node: PEP 564 New Time Functions With Nanosecond Resolution, Next: PEP 565 Show DeprecationWarning in __main__, Prev: PEP 562 Customization of Access to Module Attributes, Up: New Features<2> + +1.2.2.7 PEP 564: New Time Functions With Nanosecond Resolution +.............................................................. + +The resolution of clocks in modern systems can exceed the limited +precision of a floating point number returned by the *note time.time(): +31d. function and its variants. To avoid loss of precision, PEP 564(1) +adds six new “nanosecond” variants of the existing timer functions to +the *note time: 10a. module: + + * *note time.clock_gettime_ns(): 31e. + + * *note time.clock_settime_ns(): 31f. + + * *note time.monotonic_ns(): 320. + + * *note time.perf_counter_ns(): 321. + + * *note time.process_time_ns(): 322. + + * *note time.time_ns(): 323. + +The new functions return the number of nanoseconds as an integer value. + +Measurements(2) show that on Linux and Windows the resolution of *note +time.time_ns(): 323. is approximately 3 times better than that of *note +time.time(): 31d. + +See also +........ + +PEP 564(3) – Add new time functions with nanosecond resolution + + PEP written and implemented by Victor Stinner + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0564 + + (2) +https://www.python.org/dev/peps/pep-0564/#annex-clocks-resolution-in-python + + (3) https://www.python.org/dev/peps/pep-0564 + + +File: python.info, Node: PEP 565 Show DeprecationWarning in __main__, Next: PEP 560 Core Support for typing module and Generic Types, Prev: PEP 564 New Time Functions With Nanosecond Resolution, Up: New Features<2> + +1.2.2.8 PEP 565: Show DeprecationWarning in ‘__main__’ +...................................................... + +The default handling of *note DeprecationWarning: 278. has been changed +such that these warnings are once more shown by default, but only when +the code triggering them is running directly in the *note __main__: 1. +module. As a result, developers of single file scripts and those using +Python interactively should once again start seeing deprecation warnings +for the APIs they use, but deprecation warnings triggered by imported +application, library and framework modules will continue to be hidden by +default. + +As a result of this change, the standard library now allows developers +to choose between three different deprecation warning behaviours: + + * *note FutureWarning: 325.: always displayed by default, recommended + for warnings intended to be seen by application end users (e.g. + for deprecated application configuration settings). + + * *note DeprecationWarning: 278.: displayed by default only in *note + __main__: 1. and when running tests, recommended for warnings + intended to be seen by other Python developers where a version + upgrade may result in changed behaviour or an error. + + * *note PendingDeprecationWarning: 279.: displayed by default only + when running tests, intended for cases where a future version + upgrade will change the warning category to *note + DeprecationWarning: 278. or *note FutureWarning: 325. + +Previously both *note DeprecationWarning: 278. and *note +PendingDeprecationWarning: 279. were only visible when running tests, +which meant that developers primarily writing single file scripts or +using Python interactively could be surprised by breaking changes in the +APIs they used. + +See also +........ + +PEP 565(1) – Show DeprecationWarning in ‘__main__’ + + PEP written and implemented by Nick Coghlan + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0565 + + +File: python.info, Node: PEP 560 Core Support for typing module and Generic Types, Next: PEP 552 Hash-based pyc Files, Prev: PEP 565 Show DeprecationWarning in __main__, Up: New Features<2> + +1.2.2.9 PEP 560: Core Support for ‘typing’ module and Generic Types +................................................................... + +Initially PEP 484(1) was designed in such way that it would not +introduce `any' changes to the core CPython interpreter. Now type hints +and the *note typing: 119. module are extensively used by the community, +so this restriction is removed. The PEP introduces two special methods +*note __class_getitem__(): 327. and ‘__mro_entries__’, these methods are +now used by most classes and special constructs in *note typing: 119. +As a result, the speed of various operations with types increased up to +7 times, the generic types can be used without metaclass conflicts, and +several long standing bugs in *note typing: 119. module are fixed. + +See also +........ + +PEP 560(2) – Core support for typing module and generic types + + PEP written and implemented by Ivan Levkivskyi + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + (2) https://www.python.org/dev/peps/pep-0560 + + +File: python.info, Node: PEP 552 Hash-based pyc Files, Next: PEP 545 Python Documentation Translations, Prev: PEP 560 Core Support for typing module and Generic Types, Up: New Features<2> + +1.2.2.10 PEP 552: Hash-based .pyc Files +....................................... + +Python has traditionally checked the up-to-dateness of bytecode cache +files (i.e., ‘.pyc’ files) by comparing the source metadata +(last-modified timestamp and size) with source metadata saved in the +cache file header when it was generated. While effective, this +invalidation method has its drawbacks. When filesystem timestamps are +too coarse, Python can miss source updates, leading to user confusion. +Additionally, having a timestamp in the cache file is problematic for +build reproducibility(1) and content-based build systems. + +PEP 552(2) extends the pyc format to allow the hash of the source file +to be used for invalidation instead of the source timestamp. Such +‘.pyc’ files are called “hash-based”. By default, Python still uses +timestamp-based invalidation and does not generate hash-based ‘.pyc’ +files at runtime. Hash-based ‘.pyc’ files may be generated with *note +py_compile: d7. or *note compileall: 21. + +Hash-based ‘.pyc’ files come in two variants: checked and unchecked. +Python validates checked hash-based ‘.pyc’ files against the +corresponding source files at runtime but doesn’t do so for unchecked +hash-based pycs. Unchecked hash-based ‘.pyc’ files are a useful +performance optimization for environments where a system external to +Python (e.g., the build system) is responsible for keeping ‘.pyc’ files +up-to-date. + +See *note Cached bytecode invalidation: 329. for more information. + +See also +........ + +PEP 552(3) – Deterministic pycs + + PEP written and implemented by Benjamin Peterson + + ---------- Footnotes ---------- + + (1) https://reproducible-builds.org/ + + (2) https://www.python.org/dev/peps/pep-0552 + + (3) https://www.python.org/dev/peps/pep-0552 + + +File: python.info, Node: PEP 545 Python Documentation Translations, Next: Development Runtime Mode -X dev, Prev: PEP 552 Hash-based pyc Files, Up: New Features<2> + +1.2.2.11 PEP 545: Python Documentation Translations +................................................... + +PEP 545(1) describes the process of creating and maintaining Python +documentation translations. + +Three new translations have been added: + + - Japanese: ‘https://docs.python.org/ja/’ + + - French: ‘https://docs.python.org/fr/’ + + - Korean: ‘https://docs.python.org/ko/’ + +See also +........ + +PEP 545(2) – Python Documentation Translations + + PEP written and implemented by Julien Palard, Inada Naoki, and + Victor Stinner. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0545 + + (2) https://www.python.org/dev/peps/pep-0545 + + +File: python.info, Node: Development Runtime Mode -X dev, Prev: PEP 545 Python Documentation Translations, Up: New Features<2> + +1.2.2.12 Development Runtime Mode: -X dev +......................................... + +The new *note -X: 155. ‘dev’ command line option or the new *note +PYTHONDEVMODE: 32c. environment variable can be used to enable CPython’s +`development mode'. When in development mode, CPython performs +additional runtime checks that are too expensive to be enabled by +default. See *note -X: 155. ‘dev’ documentation for the full +description of the effects of this mode. + + +File: python.info, Node: Other Language Changes<2>, Next: New Modules<2>, Prev: New Features<2>, Up: What’s New In Python 3 7 + +1.2.3 Other Language Changes +---------------------------- + + * An *note await: 1a3. expression and comprehensions containing an + *note async for: 2e3. clause were illegal in the expressions in + *note formatted string literals: 15c. due to a problem with the + implementation. In Python 3.7 this restriction was lifted. + + * More than 255 arguments can now be passed to a function, and a + function can now have more than 255 parameters. (Contributed by + Serhiy Storchaka in bpo-12844(1) and bpo-18896(2).) + + * *note bytes.fromhex(): 32e. and *note bytearray.fromhex(): 32f. now + ignore all ASCII whitespace, not only spaces. (Contributed by + Robert Xiao in bpo-28927(3).) + + * *note str: 330, *note bytes: 331, and *note bytearray: 332. gained + support for the new *note isascii(): 333. method, which can be used + to test if a string or bytes contain only the ASCII characters. + (Contributed by INADA Naoki in bpo-32677(4).) + + * *note ImportError: 334. now displays module name and module + ‘__file__’ path when ‘from ... import ...’ fails. (Contributed by + Matthias Bussonnier in bpo-29546(5).) + + * Circular imports involving absolute imports with binding a + submodule to a name are now supported. (Contributed by Serhiy + Storchaka in bpo-30024(6).) + + * ‘object.__format__(x, '')’ is now equivalent to ‘str(x)’ rather + than ‘format(str(self), '')’. (Contributed by Serhiy Storchaka in + bpo-28974(7).) + + * In order to better support dynamic creation of stack traces, *note + types.TracebackType: 335. can now be instantiated from Python code, + and the ‘tb_next’ attribute on *note tracebacks: 336. is now + writable. (Contributed by Nathaniel J. Smith in bpo-30579(8).) + + * When using the *note -m: 337. switch, ‘sys.path[0]’ is now eagerly + expanded to the full starting directory path, rather than being + left as the empty directory (which allows imports from the + `current' working directory at the time when an import occurs) + (Contributed by Nick Coghlan in bpo-33053(9).) + + * The new *note -X: 155. ‘importtime’ option or the *note + PYTHONPROFILEIMPORTTIME: 338. environment variable can be used to + show the timing of each module import. (Contributed by Victor + Stinner in bpo-31415(10).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12844 + + (2) https://bugs.python.org/issue18896 + + (3) https://bugs.python.org/issue28927 + + (4) https://bugs.python.org/issue32677 + + (5) https://bugs.python.org/issue29546 + + (6) https://bugs.python.org/issue30024 + + (7) https://bugs.python.org/issue28974 + + (8) https://bugs.python.org/issue30579 + + (9) https://bugs.python.org/issue33053 + + (10) https://bugs.python.org/issue31415 + + +File: python.info, Node: New Modules<2>, Next: Improved Modules<2>, Prev: Other Language Changes<2>, Up: What’s New In Python 3 7 + +1.2.4 New Modules +----------------- + +* Menu: + +* contextvars:: +* dataclasses:: +* importlib.resources: importlib resources. + + +File: python.info, Node: contextvars, Next: dataclasses, Up: New Modules<2> + +1.2.4.1 contextvars +................... + +The new *note contextvars: 25. module and a set of *note new C APIs: +33b. introduce support for `context variables'. Context variables are +conceptually similar to thread-local variables. Unlike TLS, context +variables support asynchronous code correctly. + +The *note asyncio: a. and *note decimal: 36. modules have been updated +to use and support context variables out of the box. Particularly the +active decimal context is now stored in a context variable, which allows +decimal operations to work with the correct context in asynchronous +code. + +See also +........ + +PEP 567(1) – Context Variables + + PEP written and implemented by Yury Selivanov + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0567 + + +File: python.info, Node: dataclasses, Next: importlib resources, Prev: contextvars, Up: New Modules<2> + +1.2.4.2 dataclasses +................... + +The new *note dataclass(): 33d. decorator provides a way to declare +`data classes'. A data class describes its attributes using class +variable annotations. Its constructor and other magic methods, such as +*note __repr__(): 33e, *note __eq__(): 33f, and *note __hash__(): 340. +are generated automatically. + +Example: + + @dataclass + class Point: + x: float + y: float + z: float = 0.0 + + p = Point(1.5, 2.5) + print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" + +See also +........ + +PEP 557(1) – Data Classes + + PEP written and implemented by Eric V. Smith + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0557 + + +File: python.info, Node: importlib resources, Prev: dataclasses, Up: New Modules<2> + +1.2.4.3 importlib.resources +........................... + +The new *note importlib.resources: 9e. module provides several new APIs +and one new ABC for access to, opening, and reading `resources' inside +packages. Resources are roughly similar to files inside packages, but +they needn’t be actual files on the physical file system. Module +loaders can provide a ‘get_resource_reader()’ function which returns a +*note importlib.abc.ResourceReader: 342. instance to support this new +API. Built-in file path loaders and zip file loaders both support this. + +Contributed by Barry Warsaw and Brett Cannon in bpo-32248(1). + +See also +........ + +importlib_resources(2) – a PyPI backport for earlier Python versions. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32248 + + (2) http://importlib-resources.readthedocs.io/en/latest/ + + +File: python.info, Node: Improved Modules<2>, Next: C API Changes, Prev: New Modules<2>, Up: What’s New In Python 3 7 + +1.2.5 Improved Modules +---------------------- + +* Menu: + +* argparse:: +* asyncio: asyncio<2>. +* binascii:: +* calendar:: +* collections: collections<2>. +* compileall:: +* concurrent.futures: concurrent futures. +* contextlib:: +* cProfile: cProfile<2>. +* crypt:: +* datetime: datetime<2>. +* dbm:: +* decimal:: +* dis:: +* distutils:: +* enum:: +* functools: functools<2>. +* gc: gc<2>. +* hmac:: +* http.client: http client. +* http.server: http server. +* idlelib and IDLE:: +* importlib:: +* io: io<2>. +* ipaddress:: +* itertools: itertools<2>. +* locale:: +* logging: logging<2>. +* math: math<2>. +* mimetypes:: +* msilib:: +* multiprocessing: multiprocessing<2>. +* os: os<2>. +* pathlib: pathlib<2>. +* pdb:: +* py_compile: py_compile<2>. +* pydoc:: +* queue:: +* re:: +* signal:: +* socket: socket<2>. +* socketserver:: +* sqlite3:: +* ssl: ssl<2>. +* string:: +* subprocess:: +* sys: sys<2>. +* time: time<2>. +* tkinter: tkinter<2>. +* tracemalloc:: +* types:: +* unicodedata: unicodedata<2>. +* unittest: unittest<2>. +* unittest.mock: unittest mock. +* urllib.parse: urllib parse. +* uu:: +* uuid:: +* warnings:: +* xml.etree: xml etree. +* xmlrpc.server: xmlrpc server. +* zipapp:: +* zipfile:: + + +File: python.info, Node: argparse, Next: asyncio<2>, Up: Improved Modules<2> + +1.2.5.1 argparse +................ + +The new *note ArgumentParser.parse_intermixed_args(): 345. method allows +intermixing options and positional arguments. (Contributed by paul.j3 +in bpo-14191(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14191 + + +File: python.info, Node: asyncio<2>, Next: binascii, Prev: argparse, Up: Improved Modules<2> + +1.2.5.2 asyncio +............... + +The *note asyncio: a. module has received many new features, usability +and *note performance improvements: 347. Notable changes include: + + * The new *note provisional: 348. *note asyncio.run(): 1a5. function + can be used to run a coroutine from synchronous code by + automatically creating and destroying the event loop. (Contributed + by Yury Selivanov in bpo-32314(1).) + + * asyncio gained support for *note contextvars: 25. *note + loop.call_soon(): 349, *note loop.call_soon_threadsafe(): 34a, + *note loop.call_later(): 34b, *note loop.call_at(): 34c, and *note + Future.add_done_callback(): 34d. have a new optional keyword-only + `context' parameter. *note Tasks: 1ad. now track their context + automatically. See PEP 567(2) for more details. (Contributed by + Yury Selivanov in bpo-32436(3).) + + * The new *note asyncio.create_task(): 1ae. function has been added + as a shortcut to ‘asyncio.get_event_loop().create_task()’. + (Contributed by Andrew Svetlov in bpo-32311(4).) + + * The new *note loop.start_tls(): 34e. method can be used to upgrade + an existing connection to TLS. (Contributed by Yury Selivanov in + bpo-23749(5).) + + * The new *note loop.sock_recv_into(): 34f. method allows reading + data from a socket directly into a provided buffer making it + possible to reduce data copies. (Contributed by Antoine Pitrou in + bpo-31819(6).) + + * The new *note asyncio.current_task(): 350. function returns the + currently running *note Task: 1ad. instance, and the new *note + asyncio.all_tasks(): 351. function returns a set of all existing + ‘Task’ instances in a given loop. The *note Task.current_task(): + 352. and *note Task.all_tasks(): 353. methods have been deprecated. + (Contributed by Andrew Svetlov in bpo-32250(7).) + + * The new `provisional' *note BufferedProtocol: 2c9. class allows + implementing streaming protocols with manual control over the + receive buffer. (Contributed by Yury Selivanov in bpo-32251(8).) + + * The new *note asyncio.get_running_loop(): 354. function returns the + currently running loop, and raises a *note RuntimeError: 2ba. if no + loop is running. This is in contrast with *note + asyncio.get_event_loop(): 355, which will `create' a new event loop + if none is running. (Contributed by Yury Selivanov in + bpo-32269(9).) + + * The new *note StreamWriter.wait_closed(): 356. coroutine method + allows waiting until the stream writer is closed. The new *note + StreamWriter.is_closing(): 357. method can be used to determine if + the writer is closing. (Contributed by Andrew Svetlov in + bpo-32391(10).) + + * The new *note loop.sock_sendfile(): 358. coroutine method allows + sending files using *note os.sendfile: 359. when possible. + (Contributed by Andrew Svetlov in bpo-32410(11).) + + * The new *note Future.get_loop(): 35a. and ‘Task.get_loop()’ methods + return the instance of the loop on which a task or a future were + created. *note Server.get_loop(): 35b. allows doing the same for + *note asyncio.Server: 35c. objects. (Contributed by Yury Selivanov + in bpo-32415(12) and Srinivas Reddy Thatiparthy in bpo-32418(13).) + + * It is now possible to control how instances of *note + asyncio.Server: 35c. begin serving. Previously, the server would + start serving immediately when created. The new `start_serving' + keyword argument to *note loop.create_server(): 35d. and *note + loop.create_unix_server(): 35e, as well as *note + Server.start_serving(): 35f, and *note Server.serve_forever(): 360. + can be used to decouple server instantiation and serving. The new + *note Server.is_serving(): 361. method returns ‘True’ if the server + is serving. *note Server: 35c. objects are now asynchronous + context managers: + + srv = await loop.create_server(...) + + async with srv: + # some code + + # At this point, srv is closed and no longer accepts new connections. + + (Contributed by Yury Selivanov in bpo-32662(14).) + + * Callback objects returned by *note loop.call_later(): 34b. gained + the new *note when(): 362. method which returns an absolute + scheduled callback timestamp. (Contributed by Andrew Svetlov in + bpo-32741(15).) + + * The *note loop.create_datagram_endpoint(): 2e7. method gained + support for Unix sockets. (Contributed by Quentin Dawans in + bpo-31245(16).) + + * The *note asyncio.open_connection(): 363, *note + asyncio.start_server(): 364. functions, *note + loop.create_connection(): 1b2, *note loop.create_server(): 35d, + *note loop.create_accepted_socket(): 365. methods and their + corresponding UNIX socket variants now accept the + `ssl_handshake_timeout' keyword argument. (Contributed by Neil + Aspinall in bpo-29970(17).) + + * The new *note Handle.cancelled(): 366. method returns ‘True’ if the + callback was cancelled. (Contributed by Marat Sharafutdinov in + bpo-31943(18).) + + * The asyncio source has been converted to use the *note async: + 1a2./*note await: 1a3. syntax. (Contributed by Andrew Svetlov in + bpo-32193(19).) + + * The new *note ReadTransport.is_reading(): 367. method can be used + to determine the reading state of the transport. Additionally, + calls to *note ReadTransport.resume_reading(): 368. and *note + ReadTransport.pause_reading(): 369. are now idempotent. + (Contributed by Yury Selivanov in bpo-32356(20).) + + * Loop methods which accept socket paths now support passing *note + path-like objects: 36a. (Contributed by Yury Selivanov in + bpo-32066(21).) + + * In *note asyncio: a. TCP sockets on Linux are now created with + ‘TCP_NODELAY’ flag set by default. (Contributed by Yury Selivanov + and Victor Stinner in bpo-27456(22).) + + * Exceptions occurring in cancelled tasks are no longer logged. + (Contributed by Yury Selivanov in bpo-30508(23).) + + * New ‘WindowsSelectorEventLoopPolicy’ and + ‘WindowsProactorEventLoopPolicy’ classes. (Contributed by Yury + Selivanov in bpo-33792(24).) + +Several ‘asyncio’ APIs have been *note deprecated: 36b. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32314 + + (2) https://www.python.org/dev/peps/pep-0567 + + (3) https://bugs.python.org/issue32436 + + (4) https://bugs.python.org/issue32311 + + (5) https://bugs.python.org/issue23749 + + (6) https://bugs.python.org/issue31819 + + (7) https://bugs.python.org/issue32250 + + (8) https://bugs.python.org/issue32251 + + (9) https://bugs.python.org/issue32269 + + (10) https://bugs.python.org/issue32391 + + (11) https://bugs.python.org/issue32410 + + (12) https://bugs.python.org/issue32415 + + (13) https://bugs.python.org/issue32418 + + (14) https://bugs.python.org/issue32662 + + (15) https://bugs.python.org/issue32741 + + (16) https://bugs.python.org/issue31245 + + (17) https://bugs.python.org/issue29970 + + (18) https://bugs.python.org/issue31943 + + (19) https://bugs.python.org/issue32193 + + (20) https://bugs.python.org/issue32356 + + (21) https://bugs.python.org/issue32066 + + (22) https://bugs.python.org/issue27456 + + (23) https://bugs.python.org/issue30508 + + (24) https://bugs.python.org/issue33792 + + +File: python.info, Node: binascii, Next: calendar, Prev: asyncio<2>, Up: Improved Modules<2> + +1.2.5.3 binascii +................ + +The *note b2a_uu(): 36d. function now accepts an optional `backtick' +keyword argument. When it’s true, zeros are represented by ‘'`'’ +instead of spaces. (Contributed by Xiang Zhang in bpo-30103(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30103 + + +File: python.info, Node: calendar, Next: collections<2>, Prev: binascii, Up: Improved Modules<2> + +1.2.5.4 calendar +................ + +The *note HTMLCalendar: 36f. class has new class attributes which ease +the customization of CSS classes in the produced HTML calendar. +(Contributed by Oz Tiram in bpo-30095(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30095 + + +File: python.info, Node: collections<2>, Next: compileall, Prev: calendar, Up: Improved Modules<2> + +1.2.5.5 collections +................... + +‘collections.namedtuple()’ now supports default values. (Contributed by +Raymond Hettinger in bpo-32320(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32320 + + +File: python.info, Node: compileall, Next: concurrent futures, Prev: collections<2>, Up: Improved Modules<2> + +1.2.5.6 compileall +.................. + +*note compileall.compile_dir(): 372. learned the new `invalidation_mode' +parameter, which can be used to enable *note hash-based .pyc +invalidation: 301. The invalidation mode can also be specified on the +command line using the new ‘--invalidation-mode’ argument. (Contributed +by Benjamin Peterson in bpo-31650(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31650 + + +File: python.info, Node: concurrent futures, Next: contextlib, Prev: compileall, Up: Improved Modules<2> + +1.2.5.7 concurrent.futures +.......................... + +*note ProcessPoolExecutor: 2a4. and *note ThreadPoolExecutor: 27a. now +support the new `initializer' and `initargs' constructor arguments. +(Contributed by Antoine Pitrou in bpo-21423(1).) + +The *note ProcessPoolExecutor: 2a4. can now take the multiprocessing +context via the new `mp_context' argument. (Contributed by Thomas +Moreau in bpo-31540(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21423 + + (2) https://bugs.python.org/issue31540 + + +File: python.info, Node: contextlib, Next: cProfile<2>, Prev: concurrent futures, Up: Improved Modules<2> + +1.2.5.8 contextlib +.................. + +The new *note nullcontext(): 375. is a simpler and faster no-op context +manager than *note ExitStack: 376. (Contributed by Jesse-Bakker in +bpo-10049(1).) + +The new *note asynccontextmanager(): 377, *note +AbstractAsyncContextManager: 378, and *note AsyncExitStack: 379. have +been added to complement their synchronous counterparts. (Contributed +by Jelle Zijlstra in bpo-29679(2) and bpo-30241(3), and by Alexander +Mohr and Ilya Kulakov in bpo-29302(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10049 + + (2) https://bugs.python.org/issue29679 + + (3) https://bugs.python.org/issue30241 + + (4) https://bugs.python.org/issue29302 + + +File: python.info, Node: cProfile<2>, Next: crypt, Prev: contextlib, Up: Improved Modules<2> + +1.2.5.9 cProfile +................ + +The *note cProfile: 28. command line now accepts ‘-m module_name’ as an +alternative to script path. (Contributed by Sanyam Khurana in +bpo-21862(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21862 + + +File: python.info, Node: crypt, Next: datetime<2>, Prev: cProfile<2>, Up: Improved Modules<2> + +1.2.5.10 crypt +.............. + +The *note crypt: 29. module now supports the Blowfish hashing method. +(Contributed by Serhiy Storchaka in bpo-31664(1).) + +The *note mksalt(): 37c. function now allows specifying the number of +rounds for hashing. (Contributed by Serhiy Storchaka in bpo-31702(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31664 + + (2) https://bugs.python.org/issue31702 + + +File: python.info, Node: datetime<2>, Next: dbm, Prev: crypt, Up: Improved Modules<2> + +1.2.5.11 datetime +................. + +The new *note datetime.fromisoformat(): 37e. method constructs a *note +datetime: 194. object from a string in one of the formats output by +*note datetime.isoformat(): 37f. (Contributed by Paul Ganssle in +bpo-15873(1).) + +The *note tzinfo: 380. class now supports sub-minute offsets. +(Contributed by Alexander Belopolsky in bpo-5288(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15873 + + (2) https://bugs.python.org/issue5288 + + +File: python.info, Node: dbm, Next: decimal, Prev: datetime<2>, Up: Improved Modules<2> + +1.2.5.12 dbm +............ + +*note dbm.dumb: 33. now supports reading read-only files and no longer +writes the index file when it is not changed. + + +File: python.info, Node: decimal, Next: dis, Prev: dbm, Up: Improved Modules<2> + +1.2.5.13 decimal +................ + +The *note decimal: 36. module now uses *note context variables: 2f5. to +store the decimal context. (Contributed by Yury Selivanov in +bpo-32630(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32630 + + +File: python.info, Node: dis, Next: distutils, Prev: decimal, Up: Improved Modules<2> + +1.2.5.14 dis +............ + +The *note dis(): 384. function is now able to disassemble nested code +objects (the code of comprehensions, generator expressions and nested +functions, and the code used for building nested classes). The maximum +depth of disassembly recursion is controlled by the new `depth' +parameter. (Contributed by Serhiy Storchaka in bpo-11822(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11822 + + +File: python.info, Node: distutils, Next: enum, Prev: dis, Up: Improved Modules<2> + +1.2.5.15 distutils +.................. + +‘README.rst’ is now included in the list of distutils standard READMEs +and therefore included in source distributions. (Contributed by Ryan +Gonzalez in bpo-11913(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11913 + + +File: python.info, Node: enum, Next: functools<2>, Prev: distutils, Up: Improved Modules<2> + +1.2.5.16 enum +............. + +The *note Enum: 387. learned the new ‘_ignore_’ class property, which +allows listing the names of properties which should not become enum +members. (Contributed by Ethan Furman in bpo-31801(1).) + +In Python 3.8, attempting to check for non-Enum objects in ‘Enum’ +classes will raise a *note TypeError: 192. (e.g. ‘1 in Color’); +similarly, attempting to check for non-Flag objects in a ‘Flag’ member +will raise *note TypeError: 192. (e.g. ‘1 in Perm.RW’); currently, both +operations return *note False: 388. instead and are deprecated. +(Contributed by Ethan Furman in bpo-33217(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31801 + + (2) https://bugs.python.org/issue33217 + + +File: python.info, Node: functools<2>, Next: gc<2>, Prev: enum, Up: Improved Modules<2> + +1.2.5.17 functools +.................. + +*note functools.singledispatch(): 38a. now supports registering +implementations using type annotations. (Contributed by Łukasz Langa in +bpo-32227(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32227 + + +File: python.info, Node: gc<2>, Next: hmac, Prev: functools<2>, Up: Improved Modules<2> + +1.2.5.18 gc +........... + +The new *note gc.freeze(): 38c. function allows freezing all objects +tracked by the garbage collector and excluding them from future +collections. This can be used before a POSIX ‘fork()’ call to make the +GC copy-on-write friendly or to speed up collection. The new *note +gc.unfreeze(): 38d. functions reverses this operation. Additionally, +*note gc.get_freeze_count(): 38e. can be used to obtain the number of +frozen objects. (Contributed by Li Zekun in bpo-31558(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31558 + + +File: python.info, Node: hmac, Next: http client, Prev: gc<2>, Up: Improved Modules<2> + +1.2.5.19 hmac +............. + +The *note hmac: 8f. module now has an optimized one-shot *note digest(): +390. function, which is up to three times faster than ‘HMAC()’. +(Contributed by Christian Heimes in bpo-32433(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32433 + + +File: python.info, Node: http client, Next: http server, Prev: hmac, Up: Improved Modules<2> + +1.2.5.20 http.client +.................... + +*note HTTPConnection: 392. and *note HTTPSConnection: 393. now support +the new `blocksize' argument for improved upload throughput. +(Contributed by Nir Soffer in bpo-31945(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31945 + + +File: python.info, Node: http server, Next: idlelib and IDLE, Prev: http client, Up: Improved Modules<2> + +1.2.5.21 http.server +.................... + +*note SimpleHTTPRequestHandler: 395. now supports the HTTP +‘If-Modified-Since’ header. The server returns the 304 response status +if the target file was not modified after the time specified in the +header. (Contributed by Pierre Quentel in bpo-29654(1).) + +*note SimpleHTTPRequestHandler: 395. accepts the new `directory' +argument, in addition to the new ‘--directory’ command line argument. +With this parameter, the server serves the specified directory, by +default it uses the current working directory. (Contributed by Stéphane +Wirtel and Julien Palard in bpo-28707(2).) + +The new *note ThreadingHTTPServer: 396. class uses threads to handle +requests using ‘ThreadingMixin’. It is used when ‘http.server’ is run +with ‘-m’. (Contributed by Julien Palard in bpo-31639(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29654 + + (2) https://bugs.python.org/issue28707 + + (3) https://bugs.python.org/issue31639 + + +File: python.info, Node: idlelib and IDLE, Next: importlib, Prev: http server, Up: Improved Modules<2> + +1.2.5.22 idlelib and IDLE +......................... + +Multiple fixes for autocompletion. (Contributed by Louie Lu in +bpo-15786(1).) + +Module Browser (on the File menu, formerly called Class Browser), now +displays nested functions and classes in addition to top-level functions +and classes. (Contributed by Guilherme Polo, Cheryl Sabella, and Terry +Jan Reedy in bpo-1612262(2).) + +The Settings dialog (Options, Configure IDLE) has been partly rewritten +to improve both appearance and function. (Contributed by Cheryl Sabella +and Terry Jan Reedy in multiple issues.) + +The font sample now includes a selection of non-Latin characters so that +users can better see the effect of selecting a particular font. +(Contributed by Terry Jan Reedy in bpo-13802(3).) The sample can be +edited to include other characters. (Contributed by Serhiy Storchaka in +bpo-31860(4).) + +The IDLE features formerly implemented as extensions have been +reimplemented as normal features. Their settings have been moved from +the Extensions tab to other dialog tabs. (Contributed by Charles +Wohlganger and Terry Jan Reedy in bpo-27099(5).) + +Editor code context option revised. Box displays all context lines up +to maxlines. Clicking on a context line jumps the editor to that line. +Context colors for custom themes is added to Highlights tab of Settings +dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in +bpo-33642(6), bpo-33768(7), and bpo-33679(8).) + +On Windows, a new API call tells Windows that tk scales for DPI. On +Windows 8.1+ or 10, with DPI compatibility properties of the Python +binary unchanged, and a monitor resolution greater than 96 DPI, this +should make text and lines sharper. It should otherwise have no effect. +(Contributed by Terry Jan Reedy in bpo-33656(9).) + +New in 3.7.1: + +Output over N lines (50 by default) is squeezed down to a button. N can +be changed in the PyShell section of the General page of the Settings +dialog. Fewer, but possibly extra long, lines can be squeezed by right +clicking on the output. Squeezed output can be expanded in place by +double-clicking the button or into the clipboard or a separate window by +right-clicking the button. (Contributed by Tal Einat in +bpo-1529353(10).) + +The changes above have been backported to 3.6 maintenance releases. + +NEW in 3.7.4: + +Add “Run Customized” to the Run menu to run a module with customized +settings. Any command line arguments entered are added to sys.argv. +They re-appear in the box for the next customized run. One can also +suppress the normal Shell main module restart. (Contributed by Cheryl +Sabella, Terry Jan Reedy, and others in bpo-5680(11) and bpo-37627(12).) + +New in 3.7.5: + +Add optional line numbers for IDLE editor windows. Windows open without +line numbers unless set otherwise in the General tab of the +configuration dialog. Line numbers for an existing window are shown and +hidden in the Options menu. (Contributed by Tal Einat and Saimadhav +Heblikar in bpo-17535(13).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15786 + + (2) https://bugs.python.org/issue1612262 + + (3) https://bugs.python.org/issue13802 + + (4) https://bugs.python.org/issue31860 + + (5) https://bugs.python.org/issue27099 + + (6) https://bugs.python.org/issue33642 + + (7) https://bugs.python.org/issue33768 + + (8) https://bugs.python.org/issue33679 + + (9) https://bugs.python.org/issue33656 + + (10) https://bugs.python.org/issue1529353 + + (11) https://bugs.python.org/issue5680 + + (12) https://bugs.python.org/issue37627 + + (13) https://bugs.python.org/issue17535 + + +File: python.info, Node: importlib, Next: io<2>, Prev: idlelib and IDLE, Up: Improved Modules<2> + +1.2.5.23 importlib +.................. + +The *note importlib.abc.ResourceReader: 342. ABC was introduced to +support the loading of resources from packages. See also *note +importlib.resources: 2f7. (Contributed by Barry Warsaw, Brett Cannon in +bpo-32248(1).) + +*note importlib.reload(): 399. now raises *note ModuleNotFoundError: +39a. if the module lacks a spec. (Contributed by Garvit Khatri in +bpo-29851(2).) + +‘importlib.find_spec()’ now raises *note ModuleNotFoundError: 39a. +instead of *note AttributeError: 39b. if the specified parent module is +not a package (i.e. lacks a ‘__path__’ attribute). (Contributed by +Milan Oberkirch in bpo-30436(3).) + +The new ‘importlib.source_hash()’ can be used to compute the hash of the +passed source. A *note hash-based .pyc file: 301. embeds the value +returned by this function. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32248 + + (2) https://bugs.python.org/issue29851 + + (3) https://bugs.python.org/issue30436 + + +File: python.info, Node: io<2>, Next: ipaddress, Prev: importlib, Up: Improved Modules<2> + +1.2.5.24 io +........... + +The new *note TextIOWrapper.reconfigure(): 39d. method can be used to +reconfigure the text stream with the new settings. (Contributed by +Antoine Pitrou in bpo-30526(1) and INADA Naoki in bpo-15216(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30526 + + (2) https://bugs.python.org/issue15216 + + +File: python.info, Node: ipaddress, Next: itertools<2>, Prev: io<2>, Up: Improved Modules<2> + +1.2.5.25 ipaddress +.................. + +The new ‘subnet_of()’ and ‘supernet_of()’ methods of *note +ipaddress.IPv6Network: 39f. and *note ipaddress.IPv4Network: 3a0. can be +used for network containment tests. (Contributed by Michel Albert and +Cheryl Sabella in bpo-20825(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20825 + + +File: python.info, Node: itertools<2>, Next: locale, Prev: ipaddress, Up: Improved Modules<2> + +1.2.5.26 itertools +.................. + +*note itertools.islice(): 3a2. now accepts *note integer-like objects: +18a. as start, stop, and slice arguments. (Contributed by Will Roberts +in bpo-30537(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30537 + + +File: python.info, Node: locale, Next: logging<2>, Prev: itertools<2>, Up: Improved Modules<2> + +1.2.5.27 locale +............... + +The new `monetary' argument to *note locale.format_string(): 3a4. can be +used to make the conversion use monetary thousands separators and +grouping strings. (Contributed by Garvit in bpo-10379(1).) + +The *note locale.getpreferredencoding(): 3a5. function now always +returns ‘'UTF-8'’ on Android or when in the *note forced UTF-8 mode: +300. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10379 + + +File: python.info, Node: logging<2>, Next: math<2>, Prev: locale, Up: Improved Modules<2> + +1.2.5.28 logging +................ + +*note Logger: 3a7. instances can now be pickled. (Contributed by Vinay +Sajip in bpo-30520(1).) + +The new *note StreamHandler.setStream(): 3a8. method can be used to +replace the logger stream after handler creation. (Contributed by Vinay +Sajip in bpo-30522(2).) + +It is now possible to specify keyword arguments to handler constructors +in configuration passed to *note logging.config.fileConfig(): 3a9. +(Contributed by Preston Landers in bpo-31080(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30520 + + (2) https://bugs.python.org/issue30522 + + (3) https://bugs.python.org/issue31080 + + +File: python.info, Node: math<2>, Next: mimetypes, Prev: logging<2>, Up: Improved Modules<2> + +1.2.5.29 math +............. + +The new *note math.remainder(): 3ab. function implements the IEEE +754-style remainder operation. (Contributed by Mark Dickinson in +bpo-29962(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29962 + + +File: python.info, Node: mimetypes, Next: msilib, Prev: math<2>, Up: Improved Modules<2> + +1.2.5.30 mimetypes +.................. + +The MIME type of .bmp has been changed from ‘'image/x-ms-bmp'’ to +‘'image/bmp'’. (Contributed by Nitish Chandra in bpo-22589(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22589 + + +File: python.info, Node: msilib, Next: multiprocessing<2>, Prev: mimetypes, Up: Improved Modules<2> + +1.2.5.31 msilib +............... + +The new *note Database.Close(): 3ae. method can be used to close the MSI +(last-in, first-out) database. (Contributed by Berker Peksag in +bpo-20486(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20486 + + +File: python.info, Node: multiprocessing<2>, Next: os<2>, Prev: msilib, Up: Improved Modules<2> + +1.2.5.32 multiprocessing +........................ + +The new *note Process.close(): 3b0. method explicitly closes the process +object and releases all resources associated with it. *note ValueError: +1fb. is raised if the underlying process is still running. (Contributed +by Antoine Pitrou in bpo-30596(1).) + +The new *note Process.kill(): 3b1. method can be used to terminate the +process using the ‘SIGKILL’ signal on Unix. (Contributed by Vitor +Pereira in bpo-30794(2).) + +Non-daemonic threads created by *note Process: 3b2. are now joined on +process exit. (Contributed by Antoine Pitrou in bpo-18966(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30596 + + (2) https://bugs.python.org/issue30794 + + (3) https://bugs.python.org/issue18966 + + +File: python.info, Node: os<2>, Next: pathlib<2>, Prev: multiprocessing<2>, Up: Improved Modules<2> + +1.2.5.33 os +........... + +*note os.fwalk(): 3b4. now accepts the `path' argument as *note bytes: +331. (Contributed by Serhiy Storchaka in bpo-28682(1).) + +*note os.scandir(): 25f. gained support for *note file descriptors: 3b5. +(Contributed by Serhiy Storchaka in bpo-25996(2).) + +The new *note register_at_fork(): 3b6. function allows registering +Python callbacks to be executed at process fork. (Contributed by +Antoine Pitrou in bpo-16500(3).) + +Added *note os.preadv(): 3b7. (combine the functionality of *note +os.readv(): 3b8. and *note os.pread(): 3b9.) and *note os.pwritev(): +3ba. functions (combine the functionality of *note os.writev(): 3bb. and +*note os.pwrite(): 3bc.). (Contributed by Pablo Galindo in +bpo-31368(4).) + +The mode argument of *note os.makedirs(): 3bd. no longer affects the +file permission bits of newly-created intermediate-level directories. +(Contributed by Serhiy Storchaka in bpo-19930(5).) + +*note os.dup2(): 3be. now returns the new file descriptor. Previously, +‘None’ was always returned. (Contributed by Benjamin Peterson in +bpo-32441(6).) + +The structure returned by *note os.stat(): 1f1. now contains the *note +st_fstype: 3bf. attribute on Solaris and its derivatives. (Contributed +by Jesús Cea Avión in bpo-32659(7).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28682 + + (2) https://bugs.python.org/issue25996 + + (3) https://bugs.python.org/issue16500 + + (4) https://bugs.python.org/issue31368 + + (5) https://bugs.python.org/issue19930 + + (6) https://bugs.python.org/issue32441 + + (7) https://bugs.python.org/issue32659 + + +File: python.info, Node: pathlib<2>, Next: pdb, Prev: os<2>, Up: Improved Modules<2> + +1.2.5.34 pathlib +................ + +The new *note Path.is_mount(): 205. method is now available on POSIX +systems and can be used to determine whether a path is a mount point. +(Contributed by Cooper Ry Lees in bpo-30897(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30897 + + +File: python.info, Node: pdb, Next: py_compile<2>, Prev: pathlib<2>, Up: Improved Modules<2> + +1.2.5.35 pdb +............ + +*note pdb.set_trace(): 3c2. now takes an optional `header' keyword-only +argument. If given, it is printed to the console just before debugging +begins. (Contributed by Barry Warsaw in bpo-31389(1).) + +*note pdb: c9. command line now accepts ‘-m module_name’ as an +alternative to script file. (Contributed by Mario Corchero in +bpo-32206(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31389 + + (2) https://bugs.python.org/issue32206 + + +File: python.info, Node: py_compile<2>, Next: pydoc, Prev: pdb, Up: Improved Modules<2> + +1.2.5.36 py_compile +................... + +*note py_compile.compile(): 215. – and by extension, *note compileall: +21. – now respects the ‘SOURCE_DATE_EPOCH’ environment variable by +unconditionally creating ‘.pyc’ files for hash-based validation. This +allows for guaranteeing reproducible builds(1) of ‘.pyc’ files when they +are created eagerly. (Contributed by Bernhard M. Wiedemann in +bpo-29708(2).) + + ---------- Footnotes ---------- + + (1) https://reproducible-builds.org/ + + (2) https://bugs.python.org/issue29708 + + +File: python.info, Node: pydoc, Next: queue, Prev: py_compile<2>, Up: Improved Modules<2> + +1.2.5.37 pydoc +.............. + +The pydoc server can now bind to an arbitrary hostname specified by the +new ‘-n’ command-line argument. (Contributed by Feanil Patel in +bpo-31128(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31128 + + +File: python.info, Node: queue, Next: re, Prev: pydoc, Up: Improved Modules<2> + +1.2.5.38 queue +.............. + +The new *note SimpleQueue: 3c6. class is an unbounded FIFO (last-in, +first-out) queue. (Contributed by Antoine Pitrou in bpo-14976(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14976 + + +File: python.info, Node: re, Next: signal, Prev: queue, Up: Improved Modules<2> + +1.2.5.39 re +........... + +The flags *note re.ASCII: 3c8, *note re.LOCALE: 3c9. and ‘re.UNICODE’ +can be set within the scope of a group. (Contributed by Serhiy +Storchaka in bpo-31690(1).) + +*note re.split(): 3ca. now supports splitting on a pattern like ‘r'\b'’, +‘'^$'’ or ‘(?=-)’ that matches an empty string. (Contributed by Serhiy +Storchaka in bpo-25054(2).) + +Regular expressions compiled with the *note re.LOCALE: 3c9. flag no +longer depend on the locale at compile time. Locale settings are +applied only when the compiled regular expression is used. (Contributed +by Serhiy Storchaka in bpo-30215(3).) + +*note FutureWarning: 325. is now emitted if a regular expression +contains character set constructs that will change semantically in the +future, such as nested sets and set operations. (Contributed by Serhiy +Storchaka in bpo-30349(4).) + +Compiled regular expression and match objects can now be copied using +*note copy.copy(): 3cb. and *note copy.deepcopy(): 3cc. (Contributed by +Serhiy Storchaka in bpo-10076(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31690 + + (2) https://bugs.python.org/issue25054 + + (3) https://bugs.python.org/issue30215 + + (4) https://bugs.python.org/issue30349 + + (5) https://bugs.python.org/issue10076 + + +File: python.info, Node: signal, Next: socket<2>, Prev: re, Up: Improved Modules<2> + +1.2.5.40 signal +............... + +The new `warn_on_full_buffer' argument to the *note +signal.set_wakeup_fd(): 3ce. function makes it possible to specify +whether Python prints a warning on stderr when the wakeup buffer +overflows. (Contributed by Nathaniel J. Smith in bpo-30050(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30050 + + +File: python.info, Node: socket<2>, Next: socketserver, Prev: signal, Up: Improved Modules<2> + +1.2.5.41 socket +............... + +The new *note socket.getblocking(): 3d0. method returns ‘True’ if the +socket is in blocking mode and ‘False’ otherwise. (Contributed by Yury +Selivanov in bpo-32373(1).) + +The new *note socket.close(): 3d1. function closes the passed socket +file descriptor. This function should be used instead of *note +os.close(): 3d2. for better compatibility across platforms. +(Contributed by Christian Heimes in bpo-32454(2).) + +The *note socket: ef. module now exposes the ‘socket.TCP_CONGESTION’ +(Linux 2.6.13), ‘socket.TCP_USER_TIMEOUT’ (Linux 2.6.37), and +‘socket.TCP_NOTSENT_LOWAT’ (Linux 3.12) constants. (Contributed by Omar +Sandoval in bpo-26273(3) and Nathaniel J. Smith in bpo-29728(4).) + +Support for *note socket.AF_VSOCK: 3d3. sockets has been added to allow +communication between virtual machines and their hosts. (Contributed by +Cathy Avery in bpo-27584(5).) + +Sockets now auto-detect family, type and protocol from file descriptor +by default. (Contributed by Christian Heimes in bpo-28134(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32373 + + (2) https://bugs.python.org/issue32454 + + (3) https://bugs.python.org/issue26273 + + (4) https://bugs.python.org/issue29728 + + (5) https://bugs.python.org/issue27584 + + (6) https://bugs.python.org/issue28134 + + +File: python.info, Node: socketserver, Next: sqlite3, Prev: socket<2>, Up: Improved Modules<2> + +1.2.5.42 socketserver +..................... + +‘socketserver.ThreadingMixIn.server_close()’ now waits until all +non-daemon threads complete. ‘socketserver.ForkingMixIn.server_close()’ +now waits until all child processes complete. + +Add a new ‘socketserver.ForkingMixIn.block_on_close’ class attribute to +*note socketserver.ForkingMixIn: 3d5. and *note +socketserver.ThreadingMixIn: 3d6. classes. Set the class attribute to +‘False’ to get the pre-3.7 behaviour. + + +File: python.info, Node: sqlite3, Next: ssl<2>, Prev: socketserver, Up: Improved Modules<2> + +1.2.5.43 sqlite3 +................ + +*note sqlite3.Connection: 3d8. now exposes the *note backup(): 3d9. +method when the underlying SQLite library is at version 3.6.11 or +higher. (Contributed by Lele Gaifax in bpo-27645(1).) + +The `database' argument of *note sqlite3.connect(): 3da. now accepts any +*note path-like object: 36a, instead of just a string. (Contributed by +Anders Lorentsen in bpo-31843(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27645 + + (2) https://bugs.python.org/issue31843 + + +File: python.info, Node: ssl<2>, Next: string, Prev: sqlite3, Up: Improved Modules<2> + +1.2.5.44 ssl +............ + +The *note ssl: f3. module now uses OpenSSL’s builtin API instead of +*note match_hostname(): 3dc. to check a host name or an IP address. +Values are validated during TLS handshake. Any certificate validation +error including failing the host name check now raises *note +SSLCertVerificationError: 3dd. and aborts the handshake with a proper +TLS Alert message. The new exception contains additional information. +Host name validation can be customized with *note +SSLContext.hostname_checks_common_name: 3de. (Contributed by Christian +Heimes in bpo-31399(1).) + + Note: The improved host name check requires a `libssl' + implementation compatible with OpenSSL 1.0.2 or 1.1. Consequently, + OpenSSL 0.9.8 and 1.0.1 are no longer supported (see *note Platform + Support Removals: 3df. for more details). The ssl module is mostly + compatible with LibreSSL 2.7.2 and newer. + +The ‘ssl’ module no longer sends IP addresses in SNI TLS extension. +(Contributed by Christian Heimes in bpo-32185(2).) + +*note match_hostname(): 3dc. no longer supports partial wildcards like +‘www*.example.org’. (Contributed by Mandeep Singh in bpo-23033(3) and +Christian Heimes in bpo-31399(4).) + +The default cipher suite selection of the ‘ssl’ module now uses a +blacklist approach rather than a hard-coded whitelist. Python no longer +re-enables ciphers that have been blocked by OpenSSL security updates. +Default cipher suite selection can be configured at compile time. +(Contributed by Christian Heimes in bpo-31429(5).) + +Validation of server certificates containing internationalized domain +names (IDNs) is now supported. As part of this change, the *note +SSLSocket.server_hostname: 3e0. attribute now stores the expected +hostname in A-label form (‘"xn--pythn-mua.org"’), rather than the +U-label form (‘"pythön.org"’). (Contributed by Nathaniel J. Smith and +Christian Heimes in bpo-28414(6).) + +The ‘ssl’ module has preliminary and experimental support for TLS 1.3 +and OpenSSL 1.1.1. At the time of Python 3.7.0 release, OpenSSL 1.1.1 +is still under development and TLS 1.3 hasn’t been finalized yet. The +TLS 1.3 handshake and protocol behaves slightly differently than TLS 1.2 +and earlier, see *note TLS 1.3: 3e1. (Contributed by Christian Heimes +in bpo-32947(7), bpo-20995(8), bpo-29136(9), bpo-30622(10) and +bpo-33618(11)) + +*note SSLSocket: 3e2. and *note SSLObject: 3e3. no longer have a public +constructor. Direct instantiation was never a documented and supported +feature. Instances must be created with *note SSLContext: 3e4. methods +*note wrap_socket(): 3e5. and *note wrap_bio(): 3e6. (Contributed by +Christian Heimes in bpo-32951(12)) + +OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol +version are available as *note SSLContext.minimum_version: 3e7. and +*note SSLContext.maximum_version: 3e8. Supported protocols are +indicated by several new flags, such as *note HAS_TLSv1_1: 3e9. +(Contributed by Christian Heimes in bpo-32609(13).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31399 + + (2) https://bugs.python.org/issue32185 + + (3) https://bugs.python.org/issue23033 + + (4) https://bugs.python.org/issue31399 + + (5) https://bugs.python.org/issue31429 + + (6) https://bugs.python.org/issue28414 + + (7) https://bugs.python.org/issue32947 + + (8) https://bugs.python.org/issue20995 + + (9) https://bugs.python.org/issue29136 + + (10) https://bugs.python.org/issue30622 + + (11) https://bugs.python.org/issue33618 + + (12) https://bugs.python.org/issue32951 + + (13) https://bugs.python.org/issue32609 + + +File: python.info, Node: string, Next: subprocess, Prev: ssl<2>, Up: Improved Modules<2> + +1.2.5.45 string +............... + +*note string.Template: 3eb. now lets you to optionally modify the +regular expression pattern for braced placeholders and non-braced +placeholders separately. (Contributed by Barry Warsaw in +bpo-1198569(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1198569 + + +File: python.info, Node: subprocess, Next: sys<2>, Prev: string, Up: Improved Modules<2> + +1.2.5.46 subprocess +................... + +The *note subprocess.run(): 3ed. function accepts the new +`capture_output' keyword argument. When true, stdout and stderr will be +captured. This is equivalent to passing *note subprocess.PIPE: 3ee. as +`stdout' and `stderr' arguments. (Contributed by Bo Bayles in +bpo-32102(1).) + +The ‘subprocess.run’ function and the *note subprocess.Popen: 2b9. +constructor now accept the `text' keyword argument as an alias to +`universal_newlines'. (Contributed by Andrew Clegg in bpo-31756(2).) + +On Windows the default for `close_fds' was changed from ‘False’ to +‘True’ when redirecting the standard handles. It’s now possible to set +`close_fds' to true when redirecting the standard handles. See *note +subprocess.Popen: 2b9. This means that `close_fds' now defaults to +‘True’ on all supported platforms. (Contributed by Segev Finer in +bpo-19764(3).) + +The subprocess module is now more graceful when handling *note +KeyboardInterrupt: 197. during *note subprocess.call(): 3ef, *note +subprocess.run(): 3ed, or in a *note Popen: 2b9. context manager. It +now waits a short amount of time for the child to exit, before +continuing the handling of the ‘KeyboardInterrupt’ exception. +(Contributed by Gregory P. Smith in bpo-25942(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32102 + + (2) https://bugs.python.org/issue31756 + + (3) https://bugs.python.org/issue19764 + + (4) https://bugs.python.org/issue25942 + + +File: python.info, Node: sys<2>, Next: time<2>, Prev: subprocess, Up: Improved Modules<2> + +1.2.5.47 sys +............ + +The new *note sys.breakpointhook(): 313. hook function is called by the +built-in *note breakpoint(): 2f9. (Contributed by Barry Warsaw in +bpo-31353(1).) + +On Android, the new *note sys.getandroidapilevel(): 3f1. returns the +build-time Android API version. (Contributed by Victor Stinner in +bpo-28740(2).) + +The new *note sys.get_coroutine_origin_tracking_depth(): 3f2. function +returns the current coroutine origin tracking depth, as set by the new +*note sys.set_coroutine_origin_tracking_depth(): 3f3. *note asyncio: a. +has been converted to use this new API instead of the deprecated +‘sys.set_coroutine_wrapper()’. (Contributed by Nathaniel J. Smith in +bpo-32591(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31353 + + (2) https://bugs.python.org/issue28740 + + (3) https://bugs.python.org/issue32591 + + +File: python.info, Node: time<2>, Next: tkinter<2>, Prev: sys<2>, Up: Improved Modules<2> + +1.2.5.48 time +............. + +PEP 564(1) adds six new functions with nanosecond resolution to the +*note time: 10a. module: + + * *note time.clock_gettime_ns(): 31e. + + * *note time.clock_settime_ns(): 31f. + + * *note time.monotonic_ns(): 320. + + * *note time.perf_counter_ns(): 321. + + * *note time.process_time_ns(): 322. + + * *note time.time_ns(): 323. + +New clock identifiers have been added: + + * *note time.CLOCK_BOOTTIME: 3f5. (Linux): Identical to *note + time.CLOCK_MONOTONIC: 3f6, except it also includes any time that + the system is suspended. + + * *note time.CLOCK_PROF: 3f7. (FreeBSD, NetBSD and OpenBSD): + High-resolution per-process CPU timer. + + * *note time.CLOCK_UPTIME: 3f8. (FreeBSD, OpenBSD): Time whose + absolute value is the time the system has been running and not + suspended, providing accurate uptime measurement. + +The new *note time.thread_time(): 3f9. and *note time.thread_time_ns(): +3fa. functions can be used to get per-thread CPU time measurements. +(Contributed by Antoine Pitrou in bpo-32025(2).) + +The new *note time.pthread_getcpuclockid(): 3fb. function returns the +clock ID of the thread-specific CPU-time clock. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0564 + + (2) https://bugs.python.org/issue32025 + + +File: python.info, Node: tkinter<2>, Next: tracemalloc, Prev: time<2>, Up: Improved Modules<2> + +1.2.5.49 tkinter +................ + +The new *note tkinter.ttk.Spinbox: 3fd. class is now available. +(Contributed by Alan Moore in bpo-32585(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32585 + + +File: python.info, Node: tracemalloc, Next: types, Prev: tkinter<2>, Up: Improved Modules<2> + +1.2.5.50 tracemalloc +.................... + +*note tracemalloc.Traceback: 3ff. behaves more like regular tracebacks, +sorting the frames from oldest to most recent. *note +Traceback.format(): 400. now accepts negative `limit', truncating the +result to the ‘abs(limit)’ oldest frames. To get the old behaviour, use +the new `most_recent_first' argument to ‘Traceback.format()’. +(Contributed by Jesse Bakker in bpo-32121(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32121 + + +File: python.info, Node: types, Next: unicodedata<2>, Prev: tracemalloc, Up: Improved Modules<2> + +1.2.5.51 types +.............. + +The new *note WrapperDescriptorType: 402, *note MethodWrapperType: 403, +*note MethodDescriptorType: 404, and *note ClassMethodDescriptorType: +405. classes are now available. (Contributed by Manuel Krebber and +Guido van Rossum in bpo-29377(1), and Serhiy Storchaka in bpo-32265(2).) + +The new *note types.resolve_bases(): 406. function resolves MRO entries +dynamically as specified by PEP 560(3). (Contributed by Ivan Levkivskyi +in bpo-32717(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29377 + + (2) https://bugs.python.org/issue32265 + + (3) https://www.python.org/dev/peps/pep-0560 + + (4) https://bugs.python.org/issue32717 + + +File: python.info, Node: unicodedata<2>, Next: unittest<2>, Prev: types, Up: Improved Modules<2> + +1.2.5.52 unicodedata +.................... + +The internal *note unicodedata: 11a. database has been upgraded to use +Unicode 11(1). (Contributed by Benjamin Peterson.) + + ---------- Footnotes ---------- + + (1) http://www.unicode.org/versions/Unicode11.0.0/ + + +File: python.info, Node: unittest<2>, Next: unittest mock, Prev: unicodedata<2>, Up: Improved Modules<2> + +1.2.5.53 unittest +................. + +The new ‘-k’ command-line option allows filtering tests by a name +substring or a Unix shell-like pattern. For example, ‘python -m +unittest -k foo’ runs ‘foo_tests.SomeTest.test_something’, +‘bar_tests.SomeTest.test_foo’, but not +‘bar_tests.FooTest.test_something’. (Contributed by Jonas Haag in +bpo-32071(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32071 + + +File: python.info, Node: unittest mock, Next: urllib parse, Prev: unittest<2>, Up: Improved Modules<2> + +1.2.5.54 unittest.mock +...................... + +The *note sentinel: 40a. attributes now preserve their identity when +they are *note copied: 26. or *note pickled: ca. (Contributed by Serhiy +Storchaka in bpo-20804(1).) + +The new *note seal(): 40b. function allows sealing *note Mock: 249. +instances, which will disallow further creation of attribute mocks. The +seal is applied recursively to all attributes that are themselves mocks. +(Contributed by Mario Corchero in bpo-30541(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20804 + + (2) https://bugs.python.org/issue30541 + + +File: python.info, Node: urllib parse, Next: uu, Prev: unittest mock, Up: Improved Modules<2> + +1.2.5.55 urllib.parse +..................... + +*note urllib.parse.quote(): 40d. has been updated from RFC 2396(1) to +RFC 3986(2), adding ‘~’ to the set of characters that are never quoted +by default. (Contributed by Christian Theune and Ratnadeep Debnath in +bpo-16285(3).) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2396.html + + (2) https://tools.ietf.org/html/rfc3986.html + + (3) https://bugs.python.org/issue16285 + + +File: python.info, Node: uu, Next: uuid, Prev: urllib parse, Up: Improved Modules<2> + +1.2.5.56 uu +........... + +The *note uu.encode(): 40f. function now accepts an optional `backtick' +keyword argument. When it’s true, zeros are represented by ‘'`'’ +instead of spaces. (Contributed by Xiang Zhang in bpo-30103(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30103 + + +File: python.info, Node: uuid, Next: warnings, Prev: uu, Up: Improved Modules<2> + +1.2.5.57 uuid +............. + +The new *note UUID.is_safe: 411. attribute relays information from the +platform about whether generated UUIDs are generated with a +multiprocessing-safe method. (Contributed by Barry Warsaw in +bpo-22807(1).) + +*note uuid.getnode(): 412. now prefers universally administered MAC +addresses over locally administered MAC addresses. This makes a better +guarantee for global uniqueness of UUIDs returned from *note +uuid.uuid1(): 413. If only locally administered MAC addresses are +available, the first such one found is returned. (Contributed by Barry +Warsaw in bpo-32107(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22807 + + (2) https://bugs.python.org/issue32107 + + +File: python.info, Node: warnings, Next: xml etree, Prev: uuid, Up: Improved Modules<2> + +1.2.5.58 warnings +................. + +The initialization of the default warnings filters has changed as +follows: + + * warnings enabled via command line options (including those for + *note -b: 415. and the new CPython-specific *note -X: 155. ‘dev’ + option) are always passed to the warnings machinery via the *note + sys.warnoptions: 416. attribute. + + * warnings filters enabled via the command line or the environment + now have the following order of precedence: + + * the ‘BytesWarning’ filter for *note -b: 415. (or ‘-bb’) + + * any filters specified with the *note -W: 417. option + + * any filters specified with the *note PYTHONWARNINGS: 418. + environment variable + + * any other CPython specific filters (e.g. the ‘default’ + filter added for the new ‘-X dev’ mode) + + * any implicit filters defined directly by the warnings + machinery + + * in CPython debug builds, all warnings are now displayed by default + (the implicit filter list is empty) + +(Contributed by Nick Coghlan and Victor Stinner in bpo-20361(1), +bpo-32043(2), and bpo-32230(3).) + +Deprecation warnings are once again shown by default in single-file +scripts and at the interactive prompt. See *note PEP 565; Show +DeprecationWarning in __main__: 303. for details. (Contributed by Nick +Coghlan in bpo-31975(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20361 + + (2) https://bugs.python.org/issue32043 + + (3) https://bugs.python.org/issue32230 + + (4) https://bugs.python.org/issue31975 + + +File: python.info, Node: xml etree, Next: xmlrpc server, Prev: warnings, Up: Improved Modules<2> + +1.2.5.59 xml.etree +.................. + +*note ElementPath: 41a. predicates in the ‘find()’ methods can now +compare text of the current node with ‘[. = "text"]’, not only text in +children. Predicates also allow adding spaces for better readability. +(Contributed by Stefan Behnel in bpo-31648(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31648 + + +File: python.info, Node: xmlrpc server, Next: zipapp, Prev: xml etree, Up: Improved Modules<2> + +1.2.5.60 xmlrpc.server +...................... + +‘SimpleXMLRPCDispatcher.register_function’ can now be used as a +decorator. (Contributed by Xiang Zhang in bpo-7769(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7769 + + +File: python.info, Node: zipapp, Next: zipfile, Prev: xmlrpc server, Up: Improved Modules<2> + +1.2.5.61 zipapp +............... + +Function *note create_archive(): 41d. now accepts an optional `filter' +argument to allow the user to select which files should be included in +the archive. (Contributed by Irmen de Jong in bpo-31072(1).) + +Function *note create_archive(): 41d. now accepts an optional +`compressed' argument to generate a compressed archive. A command line +option ‘--compress’ has also been added to support compression. +(Contributed by Zhiming Wang in bpo-31638(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31072 + + (2) https://bugs.python.org/issue31638 + + +File: python.info, Node: zipfile, Prev: zipapp, Up: Improved Modules<2> + +1.2.5.62 zipfile +................ + +*note ZipFile: 41f. now accepts the new `compresslevel' parameter to +control the compression level. (Contributed by Bo Bayles in +bpo-21417(1).) + +Subdirectories in archives created by ‘ZipFile’ are now stored in +alphabetical order. (Contributed by Bernhard M. Wiedemann in +bpo-30693(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21417 + + (2) https://bugs.python.org/issue30693 + + +File: python.info, Node: C API Changes, Next: Build Changes, Prev: Improved Modules<2>, Up: What’s New In Python 3 7 + +1.2.6 C API Changes +------------------- + +A new API for thread-local storage has been implemented. See *note PEP +539; New C API for Thread-Local Storage: 304. for an overview and *note +Thread Specific Storage (TSS) API: 317. for a complete reference. +(Contributed by Masayuki Yamamoto in bpo-25658(1).) + +The new *note context variables: 2f5. functionality exposes a number of +*note new C APIs: 33b. + +The new *note PyImport_GetModule(): 421. function returns the previously +imported module with the given name. (Contributed by Eric Snow in +bpo-28411(2).) + +The new *note Py_RETURN_RICHCOMPARE: 422. macro eases writing rich +comparison functions. (Contributed by Petr Victorin in bpo-23699(3).) + +The new *note Py_UNREACHABLE: 423. macro can be used to mark unreachable +code paths. (Contributed by Barry Warsaw in bpo-31338(4).) + +The *note tracemalloc: 114. now exposes a C API through the new *note +PyTraceMalloc_Track(): 424. and *note PyTraceMalloc_Untrack(): 425. +functions. (Contributed by Victor Stinner in bpo-30054(5).) + +The new ‘import__find__load__start()’ and ‘import__find__load__done()’ +static markers can be used to trace module imports. (Contributed by +Christian Heimes in bpo-31574(6).) + +The fields ‘name’ and ‘doc’ of structures *note PyMemberDef: 426, *note +PyGetSetDef: 427, *note PyStructSequence_Field: 428, *note +PyStructSequence_Desc: 429, and ‘wrapperbase’ are now of type ‘const +char *’ rather of ‘char *’. (Contributed by Serhiy Storchaka in +bpo-28761(7).) + +The result of *note PyUnicode_AsUTF8AndSize(): 42a. and *note +PyUnicode_AsUTF8(): 42b. is now of type ‘const char *’ rather of ‘char +*’. (Contributed by Serhiy Storchaka in bpo-28769(8).) + +The result of *note PyMapping_Keys(): 42c, *note PyMapping_Values(): +42d. and *note PyMapping_Items(): 42e. is now always a list, rather than +a list or a tuple. (Contributed by Oren Milman in bpo-28280(9).) + +Added functions *note PySlice_Unpack(): 42f. and *note +PySlice_AdjustIndices(): 430. (Contributed by Serhiy Storchaka in +bpo-27867(10).) + +*note PyOS_AfterFork(): 431. is deprecated in favour of the new +functions *note PyOS_BeforeFork(): 432, *note PyOS_AfterFork_Parent(): +433. and *note PyOS_AfterFork_Child(): 2cd. (Contributed by Antoine +Pitrou in bpo-16500(11).) + +The ‘PyExc_RecursionErrorInst’ singleton that was part of the public API +has been removed as its members being never cleared may cause a segfault +during finalization of the interpreter. Contributed by Xavier de Gaye +in bpo-22898(12) and bpo-30697(13). + +Added C API support for timezones with timezone constructors *note +PyTimeZone_FromOffset(): 434. and *note PyTimeZone_FromOffsetAndName(): +435, and access to the UTC singleton with *note PyDateTime_TimeZone_UTC: +436. Contributed by Paul Ganssle in bpo-10381(14). + +The type of results of ‘PyThread_start_new_thread()’ and +‘PyThread_get_thread_ident()’, and the `id' parameter of *note +PyThreadState_SetAsyncExc(): 437. changed from ‘long’ to ‘unsigned +long’. (Contributed by Serhiy Storchaka in bpo-6532(15).) + +*note PyUnicode_AsWideCharString(): 438. now raises a *note ValueError: +1fb. if the second argument is ‘NULL’ and the ‘wchar_t*’ string contains +null characters. (Contributed by Serhiy Storchaka in bpo-30708(16).) + +Changes to the startup sequence and the management of dynamic memory +allocators mean that the long documented requirement to call *note +Py_Initialize(): 439. before calling most C API functions is now relied +on more heavily, and failing to abide by it may lead to segfaults in +embedding applications. See the *note Porting to Python 3.7: 307. +section in this document and the *note Before Python Initialization: +43a. section in the C API documentation for more details. + +The new *note PyInterpreterState_GetID(): 43b. returns the unique ID for +a given interpreter. (Contributed by Eric Snow in bpo-29102(17).) + +*note Py_DecodeLocale(): 43c, *note Py_EncodeLocale(): 43d. now use the +UTF-8 encoding when the *note UTF-8 mode: 300. is enabled. (Contributed +by Victor Stinner in bpo-29240(18).) + +*note PyUnicode_DecodeLocaleAndSize(): 43e. and *note +PyUnicode_EncodeLocale(): 43f. now use the current locale encoding for +‘surrogateescape’ error handler. (Contributed by Victor Stinner in +bpo-29240(19).) + +The `start' and `end' parameters of *note PyUnicode_FindChar(): 440. are +now adjusted to behave like string slices. (Contributed by Xiang Zhang +in bpo-28822(20).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25658 + + (2) https://bugs.python.org/issue28411 + + (3) https://bugs.python.org/issue23699 + + (4) https://bugs.python.org/issue31338 + + (5) https://bugs.python.org/issue30054 + + (6) https://bugs.python.org/issue31574 + + (7) https://bugs.python.org/issue28761 + + (8) https://bugs.python.org/issue28769 + + (9) https://bugs.python.org/issue28280 + + (10) https://bugs.python.org/issue27867 + + (11) https://bugs.python.org/issue16500 + + (12) https://bugs.python.org/issue22898 + + (13) https://bugs.python.org/issue30697 + + (14) https://bugs.python.org/issue10381 + + (15) https://bugs.python.org/issue6532 + + (16) https://bugs.python.org/issue30708 + + (17) https://bugs.python.org/issue29102 + + (18) https://bugs.python.org/issue29240 + + (19) https://bugs.python.org/issue29240 + + (20) https://bugs.python.org/issue28822 + + +File: python.info, Node: Build Changes, Next: Optimizations<2>, Prev: C API Changes, Up: What’s New In Python 3 7 + +1.2.7 Build Changes +------------------- + +Support for building ‘--without-threads’ has been removed. The *note +threading: 109. module is now always available. (Contributed by Antoine +Pitrou in bpo-31370(1).). + +A full copy of libffi is no longer bundled for use when building the +*note _ctypes: 2b. module on non-OSX UNIX platforms. An installed copy +of libffi is now required when building ‘_ctypes’ on such platforms. +(Contributed by Zachary Ware in bpo-27979(2).) + +The Windows build process no longer depends on Subversion to pull in +external sources, a Python script is used to download zipfiles from +GitHub instead. If Python 3.6 is not found on the system (via ‘py +-3.6’), NuGet is used to download a copy of 32-bit Python for this +purpose. (Contributed by Zachary Ware in bpo-30450(3).) + +The *note ssl: f3. module requires OpenSSL 1.0.2 or 1.1 compatible +libssl. OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is +no longer supported. LibreSSL is temporarily not supported as well. +LibreSSL releases up to version 2.6.4 are missing required OpenSSL 1.0.2 +APIs. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31370 + + (2) https://bugs.python.org/issue27979 + + (3) https://bugs.python.org/issue30450 + + +File: python.info, Node: Optimizations<2>, Next: Other CPython Implementation Changes, Prev: Build Changes, Up: What’s New In Python 3 7 + +1.2.8 Optimizations +------------------- + +The overhead of calling many methods of various standard library classes +implemented in C has been significantly reduced by porting more code to +use the ‘METH_FASTCALL’ convention. (Contributed by Victor Stinner in +bpo-29300(1), bpo-29507(2), bpo-29452(3), and bpo-29286(4).) + +Various optimizations have reduced Python startup time by 10% on Linux +and up to 30% on macOS. (Contributed by Victor Stinner, INADA Naoki in +bpo-29585(5), and Ivan Levkivskyi in bpo-31333(6).) + +Method calls are now up to 20% faster due to the bytecode changes which +avoid creating bound method instances. (Contributed by Yury Selivanov +and INADA Naoki in bpo-26110(7).) The *note asyncio: a. module received +a number of notable optimizations for commonly used functions: + + * The *note asyncio.get_event_loop(): 355. function has been + reimplemented in C to make it up to 15 times faster. (Contributed + by Yury Selivanov in bpo-32296(8).) + + * *note asyncio.Future: 443. callback management has been optimized. + (Contributed by Yury Selivanov in bpo-32348(9).) + + * *note asyncio.gather(): 286. is now up to 15% faster. (Contributed + by Yury Selivanov in bpo-32355(10).) + + * *note asyncio.sleep(): 285. is now up to 2 times faster when the + `delay' argument is zero or negative. (Contributed by Andrew + Svetlov in bpo-32351(11).) + + * The performance overhead of asyncio debug mode has been reduced. + (Contributed by Antoine Pitrou in bpo-31970(12).) + +As a result of *note PEP 560 work: 2fb, the import time of *note typing: +119. has been reduced by a factor of 7, and many typing operations are +now faster. (Contributed by Ivan Levkivskyi in bpo-32226(13).) + +*note sorted(): 444. and *note list.sort(): 445. have been optimized for +common cases to be up to 40-75% faster. (Contributed by Elliot +Gorokhovsky in bpo-28685(14).) + +*note dict.copy(): 446. is now up to 5.5 times faster. (Contributed by +Yury Selivanov in bpo-31179(15).) + +*note hasattr(): 447. and *note getattr(): 448. are now about 4 times +faster when `name' is not found and `obj' does not override *note +object.__getattr__(): 31a. or *note object.__getattribute__(): 449. +(Contributed by INADA Naoki in bpo-32544(16).) + +Searching for certain Unicode characters (like Ukrainian capital “Є”) in +a string was up to 25 times slower than searching for other characters. +It is now only 3 times slower in the worst case. (Contributed by Serhiy +Storchaka in bpo-24821(17).) + +The *note collections.namedtuple(): 1b7. factory has been reimplemented +to make the creation of named tuples 4 to 6 times faster. (Contributed +by Jelle Zijlstra with further improvements by INADA Naoki, Serhiy +Storchaka, and Raymond Hettinger in bpo-28638(18).) + +‘date.fromordinal()’ and ‘date.fromtimestamp()’ are now up to 30% faster +in the common case. (Contributed by Paul Ganssle in bpo-32403(19).) + +The *note os.fwalk(): 3b4. function is now up to 2 times faster thanks +to the use of *note os.scandir(): 25f. (Contributed by Serhiy Storchaka +in bpo-25996(20).) + +The speed of the *note shutil.rmtree(): 21c. function has been improved +by 20–40% thanks to the use of the *note os.scandir(): 25f. function. +(Contributed by Serhiy Storchaka in bpo-28564(21).) + +Optimized case-insensitive matching and searching of *note regular +expressions: dd. Searching some patterns can now be up to 20 times +faster. (Contributed by Serhiy Storchaka in bpo-30285(22).) + +*note re.compile(): 44a. now converts ‘flags’ parameter to int object if +it is ‘RegexFlag’. It is now as fast as Python 3.5, and faster than +Python 3.6 by about 10% depending on the pattern. (Contributed by INADA +Naoki in bpo-31671(23).) + +The *note modify(): 44b. methods of classes *note +selectors.EpollSelector: 44c, *note selectors.PollSelector: 44d. and +*note selectors.DevpollSelector: 44e. may be around 10% faster under +heavy loads. (Contributed by Giampaolo Rodola’ in bpo-30014(24)) + +Constant folding has been moved from the peephole optimizer to the new +AST optimizer, which is able perform optimizations more consistently. +(Contributed by Eugene Toder and INADA Naoki in bpo-29469(25) and +bpo-11549(26).) + +Most functions and methods in *note abc: 4. have been rewritten in C. +This makes creation of abstract base classes, and calling *note +isinstance(): 44f. and *note issubclass(): 450. on them 1.5x faster. +This also reduces Python start-up time by up to 10%. (Contributed by +Ivan Levkivskyi and INADA Naoki in bpo-31333(27)) + +Significant speed improvements to alternate constructors for *note +datetime.date: 193. and *note datetime.datetime: 194. by using fast-path +constructors when not constructing subclasses. (Contributed by Paul +Ganssle in bpo-32403(28)) + +The speed of comparison of *note array.array: 451. instances has been +improved considerably in certain cases. It is now from 10x to 70x +faster when comparing arrays holding values of the same integer type. +(Contributed by Adrian Wielgosik in bpo-24700(29).) + +The *note math.erf(): 452. and *note math.erfc(): 453. functions now use +the (faster) C library implementation on most platforms. (Contributed +by Serhiy Storchaka in bpo-26121(30).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29300 + + (2) https://bugs.python.org/issue29507 + + (3) https://bugs.python.org/issue29452 + + (4) https://bugs.python.org/issue29286 + + (5) https://bugs.python.org/issue29585 + + (6) https://bugs.python.org/issue31333 + + (7) https://bugs.python.org/issue26110 + + (8) https://bugs.python.org/issue32296 + + (9) https://bugs.python.org/issue32348 + + (10) https://bugs.python.org/issue32355 + + (11) https://bugs.python.org/issue32351 + + (12) https://bugs.python.org/issue31970 + + (13) https://bugs.python.org/issue32226 + + (14) https://bugs.python.org/issue28685 + + (15) https://bugs.python.org/issue31179 + + (16) https://bugs.python.org/issue32544 + + (17) https://bugs.python.org/issue24821 + + (18) https://bugs.python.org/issue28638 + + (19) https://bugs.python.org/issue32403 + + (20) https://bugs.python.org/issue25996 + + (21) https://bugs.python.org/issue28564 + + (22) https://bugs.python.org/issue30285 + + (23) https://bugs.python.org/issue31671 + + (24) https://bugs.python.org/issue30014 + + (25) https://bugs.python.org/issue29469 + + (26) https://bugs.python.org/issue11549 + + (27) https://bugs.python.org/issue31333 + + (28) https://bugs.python.org/issue32403 + + (29) https://bugs.python.org/issue24700 + + (30) https://bugs.python.org/issue26121 + + +File: python.info, Node: Other CPython Implementation Changes, Next: Deprecated Python Behavior, Prev: Optimizations<2>, Up: What’s New In Python 3 7 + +1.2.9 Other CPython Implementation Changes +------------------------------------------ + + * Trace hooks may now opt out of receiving the ‘line’ and opt into + receiving the ‘opcode’ events from the interpreter by setting the + corresponding new ‘f_trace_lines’ and ‘f_trace_opcodes’ attributes + on the frame being traced. (Contributed by Nick Coghlan in + bpo-31344(1).) + + * Fixed some consistency problems with namespace package module + attributes. Namespace module objects now have an ‘__file__’ that + is set to ‘None’ (previously unset), and their ‘__spec__.origin’ is + also set to ‘None’ (previously the string ‘"namespace"’). See + bpo-32305(2). Also, the namespace module object’s + ‘__spec__.loader’ is set to the same value as ‘__loader__’ + (previously, the former was set to ‘None’). See bpo-32303(3). + + * The *note locals(): 455. dictionary now displays in the lexical + order that variables were defined. Previously, the order was + undefined. (Contributed by Raymond Hettinger in bpo-32690(4).) + + * The *note distutils: 39. ‘upload’ command no longer tries to change + CR end-of-line characters to CRLF. This fixes a corruption issue + with sdists that ended with a byte equivalent to CR. (Contributed + by Bo Bayles in bpo-32304(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31344 + + (2) https://bugs.python.org/issue32305 + + (3) https://bugs.python.org/issue32303 + + (4) https://bugs.python.org/issue32690 + + (5) https://bugs.python.org/issue32304 + + +File: python.info, Node: Deprecated Python Behavior, Next: Deprecated Python modules functions and methods, Prev: Other CPython Implementation Changes, Up: What’s New In Python 3 7 + +1.2.10 Deprecated Python Behavior +--------------------------------- + +Yield expressions (both ‘yield’ and ‘yield from’ clauses) are now +deprecated in comprehensions and generator expressions (aside from the +iterable expression in the leftmost ‘for’ clause). This ensures that +comprehensions always immediately return a container of the appropriate +type (rather than potentially returning a *note generator iterator: 457. +object), while generator expressions won’t attempt to interleave their +implicit output with the output from any explicit yield expressions. In +Python 3.7, such expressions emit *note DeprecationWarning: 278. when +compiled, in Python 3.8 this will be a *note SyntaxError: 458. +(Contributed by Serhiy Storchaka in bpo-10544(1).) + +Returning a subclass of *note complex: 189. from *note +object.__complex__(): 18d. is deprecated and will be an error in future +Python versions. This makes ‘__complex__()’ consistent with *note +object.__int__(): 18b. and *note object.__float__(): 18c. (Contributed +by Serhiy Storchaka in bpo-28894(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10544 + + (2) https://bugs.python.org/issue28894 + + +File: python.info, Node: Deprecated Python modules functions and methods, Next: Deprecated functions and types of the C API, Prev: Deprecated Python Behavior, Up: What’s New In Python 3 7 + +1.2.11 Deprecated Python modules, functions and methods +------------------------------------------------------- + +* Menu: + +* aifc:: +* asyncio: asyncio<3>. +* collections: collections<3>. +* dbm: dbm<2>. +* enum: enum<2>. +* gettext: gettext<2>. +* importlib: importlib<2>. +* locale: locale<2>. +* macpath:: +* threading: threading<2>. +* socket: socket<3>. +* ssl: ssl<3>. +* sunau:: +* sys: sys<3>. +* wave:: + + +File: python.info, Node: aifc, Next: asyncio<3>, Up: Deprecated Python modules functions and methods + +1.2.11.1 aifc +............. + +‘aifc.openfp()’ has been deprecated and will be removed in Python 3.9. +Use *note aifc.open(): 45b. instead. (Contributed by Brian Curtin in +bpo-31985(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31985 + + +File: python.info, Node: asyncio<3>, Next: collections<3>, Prev: aifc, Up: Deprecated Python modules functions and methods + +1.2.11.2 asyncio +................ + +Support for directly ‘await’-ing instances of *note asyncio.Lock: 28b. +and other asyncio synchronization primitives has been deprecated. An +asynchronous context manager must be used in order to acquire and +release the synchronization resource. (Contributed by Andrew Svetlov in +bpo-32253(1).) + +The *note asyncio.Task.current_task(): 352. and *note +asyncio.Task.all_tasks(): 353. methods have been deprecated. +(Contributed by Andrew Svetlov in bpo-32250(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue32253 + + (2) https://bugs.python.org/issue32250 + + +File: python.info, Node: collections<3>, Next: dbm<2>, Prev: asyncio<3>, Up: Deprecated Python modules functions and methods + +1.2.11.3 collections +.................... + +In Python 3.8, the abstract base classes in *note collections.abc: 1f. +will no longer be exposed in the regular *note collections: 1e. module. +This will help create a clearer distinction between the concrete classes +and the abstract base classes. (Contributed by Serhiy Storchaka in +bpo-25988(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25988 + + +File: python.info, Node: dbm<2>, Next: enum<2>, Prev: collections<3>, Up: Deprecated Python modules functions and methods + +1.2.11.4 dbm +............ + +*note dbm.dumb: 33. now supports reading read-only files and no longer +writes the index file when it is not changed. A deprecation warning is +now emitted if the index file is missing and recreated in the ‘'r'’ and +‘'w'’ modes (this will be an error in future Python releases). +(Contributed by Serhiy Storchaka in bpo-28847(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28847 + + +File: python.info, Node: enum<2>, Next: gettext<2>, Prev: dbm<2>, Up: Deprecated Python modules functions and methods + +1.2.11.5 enum +............. + +In Python 3.8, attempting to check for non-Enum objects in ‘Enum’ +classes will raise a *note TypeError: 192. (e.g. ‘1 in Color’); +similarly, attempting to check for non-Flag objects in a ‘Flag’ member +will raise *note TypeError: 192. (e.g. ‘1 in Perm.RW’); currently, both +operations return *note False: 388. instead. (Contributed by Ethan +Furman in bpo-33217(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33217 + + +File: python.info, Node: gettext<2>, Next: importlib<2>, Prev: enum<2>, Up: Deprecated Python modules functions and methods + +1.2.11.6 gettext +................ + +Using non-integer value for selecting a plural form in *note gettext: +89. is now deprecated. It never correctly worked. (Contributed by +Serhiy Storchaka in bpo-28692(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28692 + + +File: python.info, Node: importlib<2>, Next: locale<2>, Prev: gettext<2>, Up: Deprecated Python modules functions and methods + +1.2.11.7 importlib +.................. + +Methods *note MetaPathFinder.find_module(): 462. (replaced by *note +MetaPathFinder.find_spec(): 463.) and *note +PathEntryFinder.find_loader(): 464. (replaced by *note +PathEntryFinder.find_spec(): 465.) both deprecated in Python 3.4 now +emit *note DeprecationWarning: 278. (Contributed by Matthias Bussonnier +in bpo-29576(1)) + +The *note importlib.abc.ResourceLoader: 466. ABC has been deprecated in +favour of *note importlib.abc.ResourceReader: 342. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29576 + + +File: python.info, Node: locale<2>, Next: macpath, Prev: importlib<2>, Up: Deprecated Python modules functions and methods + +1.2.11.8 locale +............... + +*note locale.format(): 468. has been deprecated, use *note +locale.format_string(): 3a4. instead. (Contributed by Garvit in +bpo-10379(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10379 + + +File: python.info, Node: macpath, Next: threading<2>, Prev: locale<2>, Up: Deprecated Python modules functions and methods + +1.2.11.9 macpath +................ + +The ‘macpath’ is now deprecated and will be removed in Python 3.8. +(Contributed by Chi Hsuan Yen in bpo-9850(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9850 + + +File: python.info, Node: threading<2>, Next: socket<3>, Prev: macpath, Up: Deprecated Python modules functions and methods + +1.2.11.10 threading +................... + +*note dummy_threading: 68. and *note _dummy_thread: 2. have been +deprecated. It is no longer possible to build Python with threading +disabled. Use *note threading: 109. instead. (Contributed by Antoine +Pitrou in bpo-31370(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31370 + + +File: python.info, Node: socket<3>, Next: ssl<3>, Prev: threading<2>, Up: Deprecated Python modules functions and methods + +1.2.11.11 socket +................ + +The silent argument value truncation in *note socket.htons(): 46c. and +*note socket.ntohs(): 46d. has been deprecated. In future versions of +Python, if the passed argument is larger than 16 bits, an exception will +be raised. (Contributed by Oren Milman in bpo-28332(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28332 + + +File: python.info, Node: ssl<3>, Next: sunau, Prev: socket<3>, Up: Deprecated Python modules functions and methods + +1.2.11.12 ssl +............. + +*note ssl.wrap_socket(): 46f. is deprecated. Use *note +ssl.SSLContext.wrap_socket(): 3e5. instead. (Contributed by Christian +Heimes in bpo-28124(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28124 + + +File: python.info, Node: sunau, Next: sys<3>, Prev: ssl<3>, Up: Deprecated Python modules functions and methods + +1.2.11.13 sunau +............... + +*note sunau.openfp(): 471. has been deprecated and will be removed in +Python 3.9. Use *note sunau.open(): 472. instead. (Contributed by +Brian Curtin in bpo-31985(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31985 + + +File: python.info, Node: sys<3>, Next: wave, Prev: sunau, Up: Deprecated Python modules functions and methods + +1.2.11.14 sys +............. + +Deprecated ‘sys.set_coroutine_wrapper()’ and +‘sys.get_coroutine_wrapper()’. + +The undocumented ‘sys.callstats()’ function has been deprecated and will +be removed in a future Python version. (Contributed by Victor Stinner +in bpo-28799(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28799 + + +File: python.info, Node: wave, Prev: sys<3>, Up: Deprecated Python modules functions and methods + +1.2.11.15 wave +.............. + +*note wave.openfp(): 475. has been deprecated and will be removed in +Python 3.9. Use *note wave.open(): 476. instead. (Contributed by Brian +Curtin in bpo-31985(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31985 + + +File: python.info, Node: Deprecated functions and types of the C API, Next: Platform Support Removals, Prev: Deprecated Python modules functions and methods, Up: What’s New In Python 3 7 + +1.2.12 Deprecated functions and types of the C API +-------------------------------------------------- + +Function *note PySlice_GetIndicesEx(): 478. is deprecated and replaced +with a macro if ‘Py_LIMITED_API’ is not set or set to a value in the +range between ‘0x03050400’ and ‘0x03060000’ (not inclusive), or is +‘0x03060100’ or higher. (Contributed by Serhiy Storchaka in +bpo-27867(1).) + +*note PyOS_AfterFork(): 431. has been deprecated. Use *note +PyOS_BeforeFork(): 432, *note PyOS_AfterFork_Parent(): 433. or *note +PyOS_AfterFork_Child(): 2cd. instead. (Contributed by Antoine Pitrou in +bpo-16500(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27867 + + (2) https://bugs.python.org/issue16500 + + +File: python.info, Node: Platform Support Removals, Next: API and Feature Removals<2>, Prev: Deprecated functions and types of the C API, Up: What’s New In Python 3 7 + +1.2.13 Platform Support Removals +-------------------------------- + + * FreeBSD 9 and older are no longer officially supported. + + * For full Unicode support, including within extension modules, *nix + platforms are now expected to provide at least one of ‘C.UTF-8’ + (full locale), ‘C.utf8’ (full locale) or ‘UTF-8’ (‘LC_CTYPE’-only + locale) as an alternative to the legacy ‘ASCII’-based ‘C’ locale. + + * OpenSSL 0.9.8 and 1.0.1 are no longer supported, which means + building CPython 3.7 with SSL/TLS support on older platforms still + using these versions requires custom build options that link to a + more recent version of OpenSSL. + + Notably, this issue affects the Debian 8 (aka “jessie”) and Ubuntu + 14.04 (aka “Trusty”) LTS Linux distributions, as they still use + OpenSSL 1.0.1 by default. + + Debian 9 (“stretch”) and Ubuntu 16.04 (“xenial”), as well as recent + releases of other LTS Linux releases (e.g. RHEL/CentOS 7.5, SLES + 12-SP3), use OpenSSL 1.0.2 or later, and remain supported in the + default build configuration. + + CPython’s own CI configuration file(1) provides an example of using + the SSL compatibility testing infrastructure(2) in CPython’s test + suite to build and link against OpenSSL 1.1.0 rather than an + outdated system provided OpenSSL. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/.travis.yml + + (2) +https://github.com/python/cpython/tree/3.8/Tools/ssl/multissltests.py + + +File: python.info, Node: API and Feature Removals<2>, Next: Module Removals, Prev: Platform Support Removals, Up: What’s New In Python 3 7 + +1.2.14 API and Feature Removals +------------------------------- + +The following features and APIs have been removed from Python 3.7: + + * The ‘os.stat_float_times()’ function has been removed. It was + introduced in Python 2.3 for backward compatibility with Python + 2.2, and was deprecated since Python 3.1. + + * Unknown escapes consisting of ‘'\'’ and an ASCII letter in + replacement templates for *note re.sub(): 47b. were deprecated in + Python 3.5, and will now cause an error. + + * Removed support of the `exclude' argument in *note + tarfile.TarFile.add(): 47c. It was deprecated in Python 2.7 and + 3.2. Use the `filter' argument instead. + + * The ‘splitunc()’ function in the ‘ntpath’ module was deprecated in + Python 3.1, and has now been removed. Use the *note splitdrive(): + 47d. function instead. + + * *note collections.namedtuple(): 1b7. no longer supports the + `verbose' parameter or ‘_source’ attribute which showed the + generated source code for the named tuple class. This was part of + an optimization designed to speed-up class creation. (Contributed + by Jelle Zijlstra with further improvements by INADA Naoki, Serhiy + Storchaka, and Raymond Hettinger in bpo-28638(1).) + + * Functions *note bool(): 183, *note float(): 187, *note list(): 262. + and *note tuple(): 47e. no longer take keyword arguments. The + first argument of *note int(): 184. can now be passed only as + positional argument. + + * Removed previously deprecated in Python 2.4 classes ‘Plist’, ‘Dict’ + and ‘_InternalDict’ in the *note plistlib: cf. module. Dict values + in the result of functions *note readPlist(): 47f. and *note + readPlistFromBytes(): 480. are now normal dicts. You no longer can + use attribute access to access items of these dictionaries. + + * The ‘asyncio.windows_utils.socketpair()’ function has been removed. + Use the *note socket.socketpair(): 481. function instead, it is + available on all platforms since Python 3.5. + ‘asyncio.windows_utils.socketpair’ was just an alias to + ‘socket.socketpair’ on Python 3.5 and newer. + + * *note asyncio: a. no longer exports the *note selectors: e6. and + ‘_overlapped’ modules as ‘asyncio.selectors’ and + ‘asyncio._overlapped’. Replace ‘from asyncio import selectors’ + with ‘import selectors’. + + * Direct instantiation of *note ssl.SSLSocket: 3e2. and *note + ssl.SSLObject: 3e3. objects is now prohibited. The constructors + were never documented, tested, or designed as public constructors. + Users were supposed to use *note ssl.wrap_socket(): 46f. or *note + ssl.SSLContext: 3e4. (Contributed by Christian Heimes in + bpo-32951(2).) + + * The unused *note distutils: 39. ‘install_misc’ command has been + removed. (Contributed by Eric N. Vander Weele in bpo-29218(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28638 + + (2) https://bugs.python.org/issue32951 + + (3) https://bugs.python.org/issue29218 + + +File: python.info, Node: Module Removals, Next: Windows-only Changes, Prev: API and Feature Removals<2>, Up: What’s New In Python 3 7 + +1.2.15 Module Removals +---------------------- + +The ‘fpectl’ module has been removed. It was never enabled by default, +never worked correctly on x86-64, and it changed the Python ABI in ways +that caused unexpected breakage of C extensions. (Contributed by +Nathaniel J. Smith in bpo-29137(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue29137 + + +File: python.info, Node: Windows-only Changes, Next: Porting to Python 3 7, Prev: Module Removals, Up: What’s New In Python 3 7 + +1.2.16 Windows-only Changes +--------------------------- + +The python launcher, (py.exe), can accept 32 & 64 bit specifiers +`without' having to specify a minor version as well. So ‘py -3-32’ and +‘py -3-64’ become valid as well as ‘py -3.7-32’, also the -`m'-64 and +-`m.n'-64 forms are now accepted to force 64 bit python even if 32 bit +would have otherwise been used. If the specified version is not +available py.exe will error exit. (Contributed by Steve Barnes in +bpo-30291(1).) + +The launcher can be run as ‘py -0’ to produce a list of the installed +pythons, `with default marked with an asterisk'. Running ‘py -0p’ will +include the paths. If py is run with a version specifier that cannot be +matched it will also print the `short form' list of available +specifiers. (Contributed by Steve Barnes in bpo-30362(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30291 + + (2) https://bugs.python.org/issue30362 + + +File: python.info, Node: Porting to Python 3 7, Next: Notable changes in Python 3 7 1, Prev: Windows-only Changes, Up: What’s New In Python 3 7 + +1.2.17 Porting to Python 3.7 +---------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code. + +* Menu: + +* Changes in Python Behavior:: +* Changes in the Python API: Changes in the Python API<2>. +* Changes in the C API: Changes in the C API<2>. +* CPython bytecode changes: CPython bytecode changes<2>. +* Windows-only Changes: Windows-only Changes<2>. +* Other CPython implementation changes:: + + +File: python.info, Node: Changes in Python Behavior, Next: Changes in the Python API<2>, Up: Porting to Python 3 7 + +1.2.17.1 Changes in Python Behavior +................................... + + * *note async: 1a2. and *note await: 1a3. names are now reserved + keywords. Code using these names as identifiers will now raise a + *note SyntaxError: 458. (Contributed by Jelle Zijlstra in + bpo-30406(1).) + + * PEP 479(2) is enabled for all code in Python 3.7, meaning that + *note StopIteration: 486. exceptions raised directly or indirectly + in coroutines and generators are transformed into *note + RuntimeError: 2ba. exceptions. (Contributed by Yury Selivanov in + bpo-32670(3).) + + * *note object.__aiter__(): 487. methods can no longer be declared as + asynchronous. (Contributed by Yury Selivanov in bpo-31709(4).) + + * Due to an oversight, earlier Python versions erroneously accepted + the following syntax: + + f(1 for x in [1],) + + class C(1 for x in [1]): + pass + + Python 3.7 now correctly raises a *note SyntaxError: 458, as a + generator expression always needs to be directly inside a set of + parentheses and cannot have a comma on either side, and the + duplication of the parentheses can be omitted only on calls. + (Contributed by Serhiy Storchaka in bpo-32012(5) and bpo-32023(6).) + + * When using the *note -m: 337. switch, the initial working directory + is now added to *note sys.path: 488, rather than an empty string + (which dynamically denoted the current working directory at the + time of each import). Any programs that are checking for the empty + string, or otherwise relying on the previous behaviour, will need + to be updated accordingly (e.g. by also checking for ‘os.getcwd()’ + or ‘os.path.dirname(__main__.__file__)’, depending on why the code + was checking for the empty string in the first place). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue30406 + + (2) https://www.python.org/dev/peps/pep-0479 + + (3) https://bugs.python.org/issue32670 + + (4) https://bugs.python.org/issue31709 + + (5) https://bugs.python.org/issue32012 + + (6) https://bugs.python.org/issue32023 + + +File: python.info, Node: Changes in the Python API<2>, Next: Changes in the C API<2>, Prev: Changes in Python Behavior, Up: Porting to Python 3 7 + +1.2.17.2 Changes in the Python API +.................................. + + * ‘socketserver.ThreadingMixIn.server_close()’ now waits until all + non-daemon threads complete. Set the new + ‘socketserver.ThreadingMixIn.block_on_close’ class attribute to + ‘False’ to get the pre-3.7 behaviour. (Contributed by Victor + Stinner in bpo-31233(1) and bpo-33540(2).) + + * ‘socketserver.ForkingMixIn.server_close()’ now waits until all + child processes complete. Set the new + ‘socketserver.ForkingMixIn.block_on_close’ class attribute to + ‘False’ to get the pre-3.7 behaviour. (Contributed by Victor + Stinner in bpo-31151(3) and bpo-33540(4).) + + * The *note locale.localeconv(): 48a. function now temporarily sets + the ‘LC_CTYPE’ locale to the value of ‘LC_NUMERIC’ in some cases. + (Contributed by Victor Stinner in bpo-31900(5).) + + * *note pkgutil.walk_packages(): 48b. now raises a *note ValueError: + 1fb. if `path' is a string. Previously an empty list was returned. + (Contributed by Sanyam Khurana in bpo-24744(6).) + + * A format string argument for *note string.Formatter.format(): 48c. + is now *note positional-only: 2a7. Passing it as a keyword + argument was deprecated in Python 3.5. (Contributed by Serhiy + Storchaka in bpo-29193(7).) + + * Attributes *note key: 48d, *note value: 48e. and *note coded_value: + 48f. of class *note http.cookies.Morsel: 490. are now read-only. + Assigning to them was deprecated in Python 3.5. Use the *note + set(): 491. method for setting them. (Contributed by Serhiy + Storchaka in bpo-29192(8).) + + * The `mode' argument of *note os.makedirs(): 3bd. no longer affects + the file permission bits of newly-created intermediate-level + directories. To set their file permission bits you can set the + umask before invoking ‘makedirs()’. (Contributed by Serhiy + Storchaka in bpo-19930(9).) + + * The *note struct.Struct.format: 492. type is now *note str: 330. + instead of *note bytes: 331. (Contributed by Victor Stinner in + bpo-21071(10).) + + * *note parse_multipart(): 2ee. now accepts the `encoding' and + `errors' arguments and returns the same results as ‘FieldStorage’: + for non-file fields, the value associated to a key is a list of + strings, not bytes. (Contributed by Pierre Quentel in + bpo-29979(11).) + + * Due to internal changes in *note socket: ef, calling *note + socket.fromshare(): 493. on a socket created by *note socket.share: + 494. in older Python versions is not supported. + + * ‘repr’ for *note BaseException: 1a8. has changed to not include the + trailing comma. Most exceptions are affected by this change. + (Contributed by Serhiy Storchaka in bpo-30399(12).) + + * ‘repr’ for *note datetime.timedelta: 195. has changed to include + the keyword arguments in the output. (Contributed by Utkarsh + Upadhyay in bpo-30302(13).) + + * Because *note shutil.rmtree(): 21c. is now implemented using the + *note os.scandir(): 25f. function, the user specified handler + `onerror' is now called with the first argument ‘os.scandir’ + instead of ‘os.listdir’ when listing the directory is failed. + + * Support for nested sets and set operations in regular expressions + as in Unicode Technical Standard #18(14) might be added in the + future. This would change the syntax. To facilitate this future + change a *note FutureWarning: 325. will be raised in ambiguous + cases for the time being. That include sets starting with a + literal ‘'['’ or containing literal character sequences ‘'--'’, + ‘'&&'’, ‘'~~'’, and ‘'||'’. To avoid a warning, escape them with a + backslash. (Contributed by Serhiy Storchaka in bpo-30349(15).) + + * The result of splitting a string on a *note regular expression: dd. + that could match an empty string has been changed. For example + splitting on ‘r'\s*'’ will now split not only on whitespaces as it + did previously, but also on empty strings before all non-whitespace + characters and just before the end of the string. The previous + behavior can be restored by changing the pattern to ‘r'\s+'’. A + *note FutureWarning: 325. was emitted for such patterns since + Python 3.5. + + For patterns that match both empty and non-empty strings, the + result of searching for all matches may also be changed in other + cases. For example in the string ‘'a\n\n'’, the pattern + ‘r'(?m)^\s*?$'’ will not only match empty strings at positions 2 + and 3, but also the string ‘'\n'’ at positions 2–3. To match only + blank lines, the pattern should be rewritten as + ‘r'(?m)^[^\S\n]*$'’. + + *note re.sub(): 47b. now replaces empty matches adjacent to a + previous non-empty match. For example ‘re.sub('x*', '-', 'abxd')’ + returns now ‘'-a-b--d-'’ instead of ‘'-a-b-d-'’ (the first minus + between ‘b’ and ‘d’ replaces ‘x’, and the second minus replaces an + empty string between ‘x’ and ‘d’). + + (Contributed by Serhiy Storchaka in bpo-25054(16) and + bpo-32308(17).) + + * Change *note re.escape(): 495. to only escape regex special + characters instead of escaping all characters other than ASCII + letters, numbers, and ‘'_'’. (Contributed by Serhiy Storchaka in + bpo-29995(18).) + + * *note tracemalloc.Traceback: 3ff. frames are now sorted from oldest + to most recent to be more consistent with *note traceback: 113. + (Contributed by Jesse Bakker in bpo-32121(19).) + + * On OSes that support *note socket.SOCK_NONBLOCK: 496. or *note + socket.SOCK_CLOEXEC: 497. bit flags, the *note socket.type: 498. no + longer has them applied. Therefore, checks like ‘if sock.type == + socket.SOCK_STREAM’ work as expected on all platforms. + (Contributed by Yury Selivanov in bpo-32331(20).) + + * On Windows the default for the `close_fds' argument of *note + subprocess.Popen: 2b9. was changed from *note False: 388. to *note + True: 499. when redirecting the standard handles. If you + previously depended on handles being inherited when using *note + subprocess.Popen: 2b9. with standard io redirection, you will have + to pass ‘close_fds=False’ to preserve the previous behaviour, or + use *note STARTUPINFO.lpAttributeList: 49a. + + * *note importlib.machinery.PathFinder.invalidate_caches(): 49b. – + which implicitly affects *note importlib.invalidate_caches(): 49c. + – now deletes entries in *note sys.path_importer_cache: 49d. which + are set to ‘None’. (Contributed by Brett Cannon in bpo-33169(21).) + + * In *note asyncio: a, *note loop.sock_recv(): 49e, *note + loop.sock_sendall(): 49f, *note loop.sock_accept(): 4a0, *note + loop.getaddrinfo(): 4a1, *note loop.getnameinfo(): 4a2. have been + changed to be proper coroutine methods to match their + documentation. Previously, these methods returned *note + asyncio.Future: 443. instances. (Contributed by Yury Selivanov in + bpo-32327(22).) + + * *note asyncio.Server.sockets: 4a3. now returns a copy of the + internal list of server sockets, instead of returning it directly. + (Contributed by Yury Selivanov in bpo-32662(23).) + + * *note Struct.format: 492. is now a *note str: 330. instance instead + of a *note bytes: 331. instance. (Contributed by Victor Stinner in + bpo-21071(24).) + + * *note argparse: 6. subparsers can now be made mandatory by passing + ‘required=True’ to *note ArgumentParser.add_subparsers(): 4a4. + (Contributed by Anthony Sottile in bpo-26510(25).) + + * *note ast.literal_eval(): 4a5. is now stricter. Addition and + subtraction of arbitrary numbers are no longer allowed. + (Contributed by Serhiy Storchaka in bpo-31778(26).) + + * *note Calendar.itermonthdates: 4a6. will now consistently raise an + exception when a date falls outside of the ‘0001-01-01’ through + ‘9999-12-31’ range. To support applications that cannot tolerate + such exceptions, the new *note Calendar.itermonthdays3: 4a7. and + *note Calendar.itermonthdays4: 4a8. can be used. The new methods + return tuples and are not restricted by the range supported by + *note datetime.date: 193. (Contributed by Alexander Belopolsky in + bpo-28292(27).) + + * *note collections.ChainMap: 4a9. now preserves the order of the + underlying mappings. (Contributed by Raymond Hettinger in + bpo-32792(28).) + + * The ‘submit()’ method of *note + concurrent.futures.ThreadPoolExecutor: 27a. and *note + concurrent.futures.ProcessPoolExecutor: 2a4. now raises a *note + RuntimeError: 2ba. if called during interpreter shutdown. + (Contributed by Mark Nemec in bpo-33097(29).) + + * The *note configparser.ConfigParser: 4aa. constructor now uses + ‘read_dict()’ to process the default values, making its behavior + consistent with the rest of the parser. Non-string keys and values + in the defaults dictionary are now being implicitly converted to + strings. (Contributed by James Tocknell in bpo-23835(30).) + + * Several undocumented internal imports were removed. One example is + that ‘os.errno’ is no longer available; use ‘import errno’ directly + instead. Note that such undocumented internal imports may be + removed any time without notice, even in micro version releases. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31233 + + (2) https://bugs.python.org/issue33540 + + (3) https://bugs.python.org/issue31151 + + (4) https://bugs.python.org/issue33540 + + (5) https://bugs.python.org/issue31900 + + (6) https://bugs.python.org/issue24744 + + (7) https://bugs.python.org/issue29193 + + (8) https://bugs.python.org/issue29192 + + (9) https://bugs.python.org/issue19930 + + (10) https://bugs.python.org/issue21071 + + (11) https://bugs.python.org/issue29979 + + (12) https://bugs.python.org/issue30399 + + (13) https://bugs.python.org/issue30302 + + (14) https://unicode.org/reports/tr18/ + + (15) https://bugs.python.org/issue30349 + + (16) https://bugs.python.org/issue25054 + + (17) https://bugs.python.org/issue32308 + + (18) https://bugs.python.org/issue29995 + + (19) https://bugs.python.org/issue32121 + + (20) https://bugs.python.org/issue32331 + + (21) https://bugs.python.org/issue33169 + + (22) https://bugs.python.org/issue32327 + + (23) https://bugs.python.org/issue32662 + + (24) https://bugs.python.org/issue21071 + + (25) https://bugs.python.org/issue26510 + + (26) https://bugs.python.org/issue31778 + + (27) https://bugs.python.org/issue28292 + + (28) https://bugs.python.org/issue32792 + + (29) https://bugs.python.org/issue33097 + + (30) https://bugs.python.org/issue23835 + + +File: python.info, Node: Changes in the C API<2>, Next: CPython bytecode changes<2>, Prev: Changes in the Python API<2>, Up: Porting to Python 3 7 + +1.2.17.3 Changes in the C API +............................. + +The function *note PySlice_GetIndicesEx(): 478. is considered unsafe for +resizable sequences. If the slice indices are not instances of *note +int: 184, but objects that implement the ‘__index__()’ method, the +sequence can be resized after passing its length to +‘PySlice_GetIndicesEx()’. This can lead to returning indices out of the +length of the sequence. For avoiding possible problems use new +functions *note PySlice_Unpack(): 42f. and *note +PySlice_AdjustIndices(): 430. (Contributed by Serhiy Storchaka in +bpo-27867(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27867 + + +File: python.info, Node: CPython bytecode changes<2>, Next: Windows-only Changes<2>, Prev: Changes in the C API<2>, Up: Porting to Python 3 7 + +1.2.17.4 CPython bytecode changes +................................. + +There are two new opcodes: *note LOAD_METHOD: 4ad. and *note +CALL_METHOD: 4ae. (Contributed by Yury Selivanov and INADA Naoki in +bpo-26110(1).) + +The ‘STORE_ANNOTATION’ opcode has been removed. (Contributed by Mark +Shannon in bpo-32550(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26110 + + (2) https://bugs.python.org/issue32550 + + +File: python.info, Node: Windows-only Changes<2>, Next: Other CPython implementation changes, Prev: CPython bytecode changes<2>, Up: Porting to Python 3 7 + +1.2.17.5 Windows-only Changes +............................. + +The file used to override *note sys.path: 488. is now called +‘._pth’ instead of ‘'sys.path'’. See *note Finding +modules: 4b0. for more information. (Contributed by Steve Dower in +bpo-28137(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28137 + + +File: python.info, Node: Other CPython implementation changes, Prev: Windows-only Changes<2>, Up: Porting to Python 3 7 + +1.2.17.6 Other CPython implementation changes +............................................. + +In preparation for potential future changes to the public CPython +runtime initialization API (see PEP 432(1) for an initial, but somewhat +outdated, draft), CPython’s internal startup and configuration +management logic has been significantly refactored. While these updates +are intended to be entirely transparent to both embedding applications +and users of the regular CPython CLI, they’re being mentioned here as +the refactoring changes the internal order of various operations during +interpreter startup, and hence may uncover previously latent defects, +either in embedding applications, or in CPython itself. (Initially +contributed by Nick Coghlan and Eric Snow as part of bpo-22257(2), and +further updated by Nick, Eric, and Victor Stinner in a number of other +issues). Some known details affected: + + * *note PySys_AddWarnOptionUnicode(): 4b2. is not currently usable by + embedding applications due to the requirement to create a Unicode + object prior to calling ‘Py_Initialize’. Use *note + PySys_AddWarnOption(): 4b3. instead. + + * warnings filters added by an embedding application with *note + PySys_AddWarnOption(): 4b3. should now more consistently take + precedence over the default filters set by the interpreter + +Due to changes in the way the default warnings filters are configured, +setting *note Py_BytesWarningFlag: 4b4. to a value greater than one is +no longer sufficient to both emit *note BytesWarning: 4b5. messages and +have them converted to exceptions. Instead, the flag must be set (to +cause the warnings to be emitted in the first place), and an explicit +‘error::BytesWarning’ warnings filter added to convert them to +exceptions. + +Due to a change in the way docstrings are handled by the compiler, the +implicit ‘return None’ in a function body consisting solely of a +docstring is now marked as occurring on the same line as the docstring, +not on the function’s header line. + +The current exception state has been moved from the frame object to the +co-routine. This simplified the interpreter and fixed a couple of +obscure bugs caused by having swap exception state when entering or +exiting a generator. (Contributed by Mark Shannon in bpo-25612(3).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0432 + + (2) https://bugs.python.org/issue22257 + + (3) https://bugs.python.org/issue25612 + + +File: python.info, Node: Notable changes in Python 3 7 1, Next: Notable changes in Python 3 7 2, Prev: Porting to Python 3 7, Up: What’s New In Python 3 7 + +1.2.18 Notable changes in Python 3.7.1 +-------------------------------------- + +Starting in 3.7.1, *note Py_Initialize(): 439. now consistently reads +and respects all of the same environment settings as *note Py_Main(): +4b7. (in earlier Python versions, it respected an ill-defined subset of +those environment variables, while in Python 3.7.0 it didn’t read any of +them due to bpo-34247(1)). If this behavior is unwanted, set *note +Py_IgnoreEnvironmentFlag: 4b8. to 1 before calling *note +Py_Initialize(): 439. + +In 3.7.1 the C API for Context Variables *note was updated: 4b9. to use +*note PyObject: 4ba. pointers. See also bpo-34762(2). + +In 3.7.1 the *note tokenize: 111. module now implicitly emits a +‘NEWLINE’ token when provided with input that does not have a trailing +new line. This behavior now matches what the C tokenizer does +internally. (Contributed by Ammar Askar in bpo-33899(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34247 + + (2) https://bugs.python.org/issue34762 + + (3) https://bugs.python.org/issue33899 + + +File: python.info, Node: Notable changes in Python 3 7 2, Next: Notable changes in Python 3 7 6, Prev: Notable changes in Python 3 7 1, Up: What’s New In Python 3 7 + +1.2.19 Notable changes in Python 3.7.2 +-------------------------------------- + +In 3.7.2, *note venv: 125. on Windows no longer copies the original +binaries, but creates redirector scripts named ‘python.exe’ and +‘pythonw.exe’ instead. This resolves a long standing issue where all +virtual environments would have to be upgraded or recreated with each +Python update. However, note that this release will still require +recreation of virtual environments in order to get the new scripts. + + +File: python.info, Node: Notable changes in Python 3 7 6, Next: Notable changes in Python 3 7 10, Prev: Notable changes in Python 3 7 2, Up: What’s New In Python 3 7 + +1.2.20 Notable changes in Python 3.7.6 +-------------------------------------- + +Due to significant security concerns, the `reuse_address' parameter of +*note asyncio.loop.create_datagram_endpoint(): 2e7. is no longer +supported. This is because of the behavior of the socket option +‘SO_REUSEADDR’ in UDP. For more details, see the documentation for +‘loop.create_datagram_endpoint()’. (Contributed by Kyle Stanley, +Antoine Pitrou, and Yury Selivanov in bpo-37228(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue37228 + + +File: python.info, Node: Notable changes in Python 3 7 10, Prev: Notable changes in Python 3 7 6, Up: What’s New In Python 3 7 + +1.2.21 Notable changes in Python 3.7.10 +--------------------------------------- + +Earlier Python versions allowed using both ‘;’ and ‘&’ as query +parameter separators in *note urllib.parse.parse_qs(): 2eb. and *note +urllib.parse.parse_qsl(): 2ec. Due to security concerns, and to conform +with newer W3C recommendations, this has been changed to allow only a +single separator key, with ‘&’ as the default. This change also affects +*note cgi.parse(): 2ed. and *note cgi.parse_multipart(): 2ee. as they +use the affected functions internally. For more details, please see +their respective documentation. (Contributed by Adam Goldschmidt, +Senthil Kumaran and Ken Jin in bpo-42967(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue42967 + + +File: python.info, Node: What’s New In Python 3 6, Next: What’s New In Python 3 5, Prev: What’s New In Python 3 7, Up: What’s New in Python + +1.3 What’s New In Python 3.6 +============================ + + +Editors: Elvis Pranskevichus <>, Yury Selivanov +<> + +This article explains the new features in Python 3.6, compared to 3.5. +Python 3.6 was released on December 23, 2016.  See the changelog(1) for +a full list of changes. + +See also +........ + +PEP 494(2) - Python 3.6 Release Schedule + +* Menu: + +* Summary – Release highlights: Summary – Release highlights<2>. +* New Features: New Features<3>. +* Other Language Changes: Other Language Changes<3>. +* New Modules: New Modules<3>. +* Improved Modules: Improved Modules<3>. +* Optimizations: Optimizations<3>. +* Build and C API Changes: Build and C API Changes<2>. +* Other Improvements:: +* Deprecated: Deprecated<2>. +* Removed:: +* Porting to Python 3.6: Porting to Python 3 6. +* Notable changes in Python 3.6.2: Notable changes in Python 3 6 2. +* Notable changes in Python 3.6.4: Notable changes in Python 3 6 4. +* Notable changes in Python 3.6.5: Notable changes in Python 3 6 5. +* Notable changes in Python 3.6.7: Notable changes in Python 3 6 7. +* Notable changes in Python 3.6.10: Notable changes in Python 3 6 10. +* Notable changes in Python 3.6.13: Notable changes in Python 3 6 13. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3.6/whatsnew/changelog.html + + (2) https://www.python.org/dev/peps/pep-0494 + + +File: python.info, Node: Summary – Release highlights<2>, Next: New Features<3>, Up: What’s New In Python 3 6 + +1.3.1 Summary – Release highlights +---------------------------------- + +New syntax features: + + * *note PEP 498: 4c1, formatted string literals. + + * *note PEP 515: 4c2, underscores in numeric literals. + + * *note PEP 526: 4c3, syntax for variable annotations. + + * *note PEP 525: 4c4, asynchronous generators. + + * *note PEP 530: 4c5.: asynchronous comprehensions. + +New library modules: + + * *note secrets: e4.: *note PEP 506 – Adding A Secrets Module To The + Standard Library: 4c6. + +CPython implementation improvements: + + * The *note dict: 2fc. type has been reimplemented to use a *note + more compact representation: 4c7. based on a proposal by Raymond + Hettinger(1) and similar to the PyPy dict implementation(2). This + resulted in dictionaries using 20% to 25% less memory when compared + to Python 3.5. + + * Customization of class creation has been simplified with the *note + new protocol: 4c8. + + * The class attribute definition order is *note now preserved: 4c9. + + * The order of elements in ‘**kwargs’ now *note corresponds to the + order: 4ca. in which keyword arguments were passed to the function. + + * DTrace and SystemTap *note probing support: 4cb. has been added. + + * The new *note PYTHONMALLOC: 4cc. environment variable can now be + used to debug the interpreter memory allocation and access errors. + +Significant improvements in the standard library: + + * The *note asyncio: a. module has received new features, significant + usability and performance improvements, and a fair amount of bug + fixes. Starting with Python 3.6 the ‘asyncio’ module is no longer + provisional and its API is considered stable. + + * A new *note file system path protocol: 4cd. has been implemented to + support *note path-like objects: 36a. All standard library + functions operating on paths have been updated to work with the new + protocol. + + * The *note datetime: 31. module has gained support for *note Local + Time Disambiguation: 4ce. + + * The *note typing: 119. module received a number of *note + improvements: 4cf. + + * The *note tracemalloc: 114. module has been significantly reworked + and is now used to provide better output for *note ResourceWarning: + 4d0. as well as provide better diagnostics for memory allocation + errors. See the *note PYTHONMALLOC section: 4cc. for more + information. + +Security improvements: + + * The new *note secrets: e4. module has been added to simplify the + generation of cryptographically strong pseudo-random numbers + suitable for managing secrets such as account authentication, + tokens, and similar. + + * On Linux, *note os.urandom(): 4d1. now blocks until the system + urandom entropy pool is initialized to increase the security. See + the PEP 524(3) for the rationale. + + * The *note hashlib: 8d. and *note ssl: f3. modules now support + OpenSSL 1.1.0. + + * The default settings and feature set of the *note ssl: f3. module + have been improved. + + * The *note hashlib: 8d. module received support for the BLAKE2, + SHA-3 and SHAKE hash algorithms and the *note scrypt(): 4d2. key + derivation function. + +Windows improvements: + + * *note PEP 528: 4d3. and *note PEP 529: 4d4, Windows filesystem and + console encoding changed to UTF-8. + + * The ‘py.exe’ launcher, when used interactively, no longer prefers + Python 2 over Python 3 when the user doesn’t specify a version (via + command line arguments or a config file). Handling of shebang + lines remains unchanged - “python” refers to Python 2 in that case. + + * ‘python.exe’ and ‘pythonw.exe’ have been marked as long-path aware, + which means that the 260 character path limit may no longer apply. + See *note removing the MAX_PATH limitation: 4d5. for details. + + * A ‘._pth’ file can be added to force isolated mode and fully + specify all search paths to avoid registry and environment lookup. + See *note the documentation: 4b0. for more information. + + * A ‘python36.zip’ file now works as a landmark to infer *note + PYTHONHOME: 4d6. See *note the documentation: 4b0. for more + information. + + ---------- Footnotes ---------- + + (1) +https://mail.python.org/pipermail/python-dev/2012-December/123028.html + + (2) +https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html + + (3) https://www.python.org/dev/peps/pep-0524 + + +File: python.info, Node: New Features<3>, Next: Other Language Changes<3>, Prev: Summary – Release highlights<2>, Up: What’s New In Python 3 6 + +1.3.2 New Features +------------------ + +* Menu: + +* PEP 498; Formatted string literals: PEP 498 Formatted string literals. +* PEP 526; Syntax for variable annotations: PEP 526 Syntax for variable annotations. +* PEP 515; Underscores in Numeric Literals: PEP 515 Underscores in Numeric Literals. +* PEP 525; Asynchronous Generators: PEP 525 Asynchronous Generators. +* PEP 530; Asynchronous Comprehensions: PEP 530 Asynchronous Comprehensions. +* PEP 487; Simpler customization of class creation: PEP 487 Simpler customization of class creation. +* PEP 487; Descriptor Protocol Enhancements: PEP 487 Descriptor Protocol Enhancements. +* PEP 519; Adding a file system path protocol: PEP 519 Adding a file system path protocol. +* PEP 495; Local Time Disambiguation: PEP 495 Local Time Disambiguation. +* PEP 529; Change Windows filesystem encoding to UTF-8: PEP 529 Change Windows filesystem encoding to UTF-8. +* PEP 528; Change Windows console encoding to UTF-8: PEP 528 Change Windows console encoding to UTF-8. +* PEP 520; Preserving Class Attribute Definition Order: PEP 520 Preserving Class Attribute Definition Order. +* PEP 468; Preserving Keyword Argument Order: PEP 468 Preserving Keyword Argument Order. +* New dict implementation:: +* PEP 523; Adding a frame evaluation API to CPython: PEP 523 Adding a frame evaluation API to CPython. +* PYTHONMALLOC environment variable:: +* DTrace and SystemTap probing support:: + + +File: python.info, Node: PEP 498 Formatted string literals, Next: PEP 526 Syntax for variable annotations, Up: New Features<3> + +1.3.2.1 PEP 498: Formatted string literals +.......................................... + +PEP 498(1) introduces a new kind of string literals: `f-strings', or +*note formatted string literals: 15c. + +Formatted string literals are prefixed with ‘'f'’ and are similar to the +format strings accepted by *note str.format(): 4da. They contain +replacement fields surrounded by curly braces. The replacement fields +are expressions, which are evaluated at run time, and then formatted +using the *note format(): 4db. protocol: + + >>> name = "Fred" + >>> f"He said his name is {name}." + 'He said his name is Fred.' + >>> width = 10 + >>> precision = 4 + >>> value = decimal.Decimal("12.34567") + >>> f"result: {value:{width}.{precision}}" # nested fields + 'result: 12.35' + +See also +........ + +PEP 498(2) – Literal String Interpolation. + + PEP written and implemented by Eric V. Smith. + +*note Feature documentation: 15c. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0498 + + (2) https://www.python.org/dev/peps/pep-0498 + + +File: python.info, Node: PEP 526 Syntax for variable annotations, Next: PEP 515 Underscores in Numeric Literals, Prev: PEP 498 Formatted string literals, Up: New Features<3> + +1.3.2.2 PEP 526: Syntax for variable annotations +................................................ + +PEP 484(1) introduced the standard for type annotations of function +parameters, a.k.a. type hints. This PEP adds syntax to Python for +annotating the types of variables including class variables and instance +variables: + + primes: List[int] = [] + + captain: str # Note: no initial value! + + class Starship: + stats: Dict[str, int] = {} + +Just as for function annotations, the Python interpreter does not attach +any particular meaning to variable annotations and only stores them in +the ‘__annotations__’ attribute of a class or module. + +In contrast to variable declarations in statically typed languages, the +goal of annotation syntax is to provide an easy way to specify +structured type metadata for third party tools and libraries via the +abstract syntax tree and the ‘__annotations__’ attribute. + +See also +........ + +PEP 526(2) – Syntax for variable annotations. + + PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa + Roach, and Guido van Rossum. Implemented by Ivan Levkivskyi. + +Tools that use or will use the new syntax: mypy(3), pytype(4), PyCharm, +etc. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + (2) https://www.python.org/dev/peps/pep-0526 + + (3) http://www.mypy-lang.org/ + + (4) https://github.com/google/pytype + + +File: python.info, Node: PEP 515 Underscores in Numeric Literals, Next: PEP 525 Asynchronous Generators, Prev: PEP 526 Syntax for variable annotations, Up: New Features<3> + +1.3.2.3 PEP 515: Underscores in Numeric Literals +................................................ + +PEP 515(1) adds the ability to use underscores in numeric literals for +improved readability. For example: + + >>> 1_000_000_000_000_000 + 1000000000000000 + >>> 0x_FF_FF_FF_FF + 4294967295 + +Single underscores are allowed between digits and after any base +specifier. Leading, trailing, or multiple underscores in a row are not +allowed. + +The *note string formatting: 4de. language also now has support for the +‘'_'’ option to signal the use of an underscore for a thousands +separator for floating point presentation types and for integer +presentation type ‘'d'’. For integer presentation types ‘'b'’, ‘'o'’, +‘'x'’, and ‘'X'’, underscores will be inserted every 4 digits: + + >>> '{:_}'.format(1000000) + '1_000_000' + >>> '{:_x}'.format(0xFFFFFFFF) + 'ffff_ffff' + +See also +........ + +PEP 515(2) – Underscores in Numeric Literals + + PEP written by Georg Brandl and Serhiy Storchaka. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0515 + + (2) https://www.python.org/dev/peps/pep-0515 + + +File: python.info, Node: PEP 525 Asynchronous Generators, Next: PEP 530 Asynchronous Comprehensions, Prev: PEP 515 Underscores in Numeric Literals, Up: New Features<3> + +1.3.2.4 PEP 525: Asynchronous Generators +........................................ + +PEP 492(1) introduced support for native coroutines and ‘async’ / +‘await’ syntax to Python 3.5. A notable limitation of the Python 3.5 +implementation is that it was not possible to use ‘await’ and ‘yield’ in +the same function body. In Python 3.6 this restriction has been lifted, +making it possible to define `asynchronous generators': + + async def ticker(delay, to): + """Yield numbers from 0 to *to* every *delay* seconds.""" + for i in range(to): + yield i + await asyncio.sleep(delay) + +The new syntax allows for faster and more concise code. + +See also +........ + +PEP 525(2) – Asynchronous Generators + + PEP written and implemented by Yury Selivanov. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0492 + + (2) https://www.python.org/dev/peps/pep-0525 + + +File: python.info, Node: PEP 530 Asynchronous Comprehensions, Next: PEP 487 Simpler customization of class creation, Prev: PEP 525 Asynchronous Generators, Up: New Features<3> + +1.3.2.5 PEP 530: Asynchronous Comprehensions +............................................ + +PEP 530(1) adds support for using ‘async for’ in list, set, dict +comprehensions and generator expressions: + + result = [i async for i in aiter() if i % 2] + +Additionally, ‘await’ expressions are supported in all kinds of +comprehensions: + + result = [await fun() for fun in funcs if await condition()] + +See also +........ + +PEP 530(2) – Asynchronous Comprehensions + + PEP written and implemented by Yury Selivanov. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0530 + + (2) https://www.python.org/dev/peps/pep-0530 + + +File: python.info, Node: PEP 487 Simpler customization of class creation, Next: PEP 487 Descriptor Protocol Enhancements, Prev: PEP 530 Asynchronous Comprehensions, Up: New Features<3> + +1.3.2.6 PEP 487: Simpler customization of class creation +........................................................ + +It is now possible to customize subclass creation without using a +metaclass. The new ‘__init_subclass__’ classmethod will be called on +the base class whenever a new subclass is created: + + class PluginBase: + subclasses = [] + + def __init_subclass__(cls, **kwargs): + super().__init_subclass__(**kwargs) + cls.subclasses.append(cls) + + class Plugin1(PluginBase): + pass + + class Plugin2(PluginBase): + pass + +In order to allow zero-argument *note super(): 4e2. calls to work +correctly from *note __init_subclass__(): 4e3. implementations, custom +metaclasses must ensure that the new ‘__classcell__’ namespace entry is +propagated to ‘type.__new__’ (as described in *note Creating the class +object: 4e4.). + +See also +........ + +PEP 487(1) – Simpler customization of class creation + + PEP written and implemented by Martin Teichmann. + +*note Feature documentation: 4e5. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0487 + + +File: python.info, Node: PEP 487 Descriptor Protocol Enhancements, Next: PEP 519 Adding a file system path protocol, Prev: PEP 487 Simpler customization of class creation, Up: New Features<3> + +1.3.2.7 PEP 487: Descriptor Protocol Enhancements +................................................. + +PEP 487(1) extends the descriptor protocol to include the new optional +*note __set_name__(): 4e8. method. Whenever a new class is defined, the +new method will be called on all descriptors included in the definition, +providing them with a reference to the class being defined and the name +given to the descriptor within the class namespace. In other words, +instances of descriptors can now know the attribute name of the +descriptor in the owner class: + + class IntField: + def __get__(self, instance, owner): + return instance.__dict__[self.name] + + def __set__(self, instance, value): + if not isinstance(value, int): + raise ValueError(f'expecting integer in {self.name}') + instance.__dict__[self.name] = value + + # this is the new initializer: + def __set_name__(self, owner, name): + self.name = name + + class Model: + int_field = IntField() + +See also +........ + +PEP 487(2) – Simpler customization of class creation + + PEP written and implemented by Martin Teichmann. + +*note Feature documentation: 4e9. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0487 + + (2) https://www.python.org/dev/peps/pep-0487 + + +File: python.info, Node: PEP 519 Adding a file system path protocol, Next: PEP 495 Local Time Disambiguation, Prev: PEP 487 Descriptor Protocol Enhancements, Up: New Features<3> + +1.3.2.8 PEP 519: Adding a file system path protocol +................................................... + +File system paths have historically been represented as *note str: 330. +or *note bytes: 331. objects. This has led to people who write code +which operate on file system paths to assume that such objects are only +one of those two types (an *note int: 184. representing a file +descriptor does not count as that is not a file path). Unfortunately +that assumption prevents alternative object representations of file +system paths like *note pathlib: c8. from working with pre-existing +code, including Python’s standard library. + +To fix this situation, a new interface represented by *note os.PathLike: +4eb. has been defined. By implementing the *note __fspath__(): 4ec. +method, an object signals that it represents a path. An object can then +provide a low-level representation of a file system path as a *note str: +330. or *note bytes: 331. object. This means an object is considered +*note path-like: 36a. if it implements *note os.PathLike: 4eb. or is a +*note str: 330. or *note bytes: 331. object which represents a file +system path. Code can use *note os.fspath(): 4ed, *note os.fsdecode(): +4ee, or *note os.fsencode(): 4ef. to explicitly get a *note str: 330. +and/or *note bytes: 331. representation of a path-like object. + +The built-in *note open(): 4f0. function has been updated to accept +*note os.PathLike: 4eb. objects, as have all relevant functions in the +*note os: c4. and *note os.path: c5. modules, and most other functions +and classes in the standard library. The *note os.DirEntry: 4f1. class +and relevant classes in *note pathlib: c8. have also been updated to +implement *note os.PathLike: 4eb. + +The hope is that updating the fundamental functions for operating on +file system paths will lead to third-party code to implicitly support +all *note path-like objects: 36a. without any code changes, or at least +very minimal ones (e.g. calling *note os.fspath(): 4ed. at the +beginning of code before operating on a path-like object). + +Here are some examples of how the new interface allows for *note +pathlib.Path: 201. to be used more easily and transparently with +pre-existing code: + + >>> import pathlib + >>> with open(pathlib.Path("README")) as f: + ... contents = f.read() + ... + >>> import os.path + >>> os.path.splitext(pathlib.Path("some_file.txt")) + ('some_file', '.txt') + >>> os.path.join("/a/b", pathlib.Path("c")) + '/a/b/c' + >>> import os + >>> os.fspath(pathlib.Path("some_file.txt")) + 'some_file.txt' + +(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle +Zijlstra.) + +See also +........ + +PEP 519(1) – Adding a file system path protocol + + PEP written by Brett Cannon and Koos Zevenhoven. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0519 + + +File: python.info, Node: PEP 495 Local Time Disambiguation, Next: PEP 529 Change Windows filesystem encoding to UTF-8, Prev: PEP 519 Adding a file system path protocol, Up: New Features<3> + +1.3.2.9 PEP 495: Local Time Disambiguation +.......................................... + +In most world locations, there have been and will be times when local +clocks are moved back. In those times, intervals are introduced in +which local clocks show the same time twice in the same day. In these +situations, the information displayed on a local clock (or stored in a +Python datetime instance) is insufficient to identify a particular +moment in time. + +PEP 495(1) adds the new `fold' attribute to instances of *note +datetime.datetime: 194. and *note datetime.time: 4f3. classes to +differentiate between two moments in time for which local times are the +same: + + >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc) + >>> for i in range(4): + ... u = u0 + i*HOUR + ... t = u.astimezone(Eastern) + ... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold) + ... + 04:00:00 UTC = 00:00:00 EDT 0 + 05:00:00 UTC = 01:00:00 EDT 0 + 06:00:00 UTC = 01:00:00 EST 1 + 07:00:00 UTC = 02:00:00 EST 0 + +The values of the *note fold: 4f4. attribute have the value ‘0’ for all +instances except those that represent the second (chronologically) +moment in time in an ambiguous case. + +See also +........ + +PEP 495(2) – Local Time Disambiguation + + PEP written by Alexander Belopolsky and Tim Peters, implementation + by Alexander Belopolsky. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0495 + + (2) https://www.python.org/dev/peps/pep-0495 + + +File: python.info, Node: PEP 529 Change Windows filesystem encoding to UTF-8, Next: PEP 528 Change Windows console encoding to UTF-8, Prev: PEP 495 Local Time Disambiguation, Up: New Features<3> + +1.3.2.10 PEP 529: Change Windows filesystem encoding to UTF-8 +............................................................. + +Representing filesystem paths is best performed with str (Unicode) +rather than bytes. However, there are some situations where using bytes +is sufficient and correct. + +Prior to Python 3.6, data loss could result when using bytes paths on +Windows. With this change, using bytes to represent paths is now +supported on Windows, provided those bytes are encoded with the encoding +returned by *note sys.getfilesystemencoding(): 4f6, which now defaults +to ‘'utf-8'’. + +Applications that do not use str to represent paths should use *note +os.fsencode(): 4ef. and *note os.fsdecode(): 4ee. to ensure their bytes +are correctly encoded. To revert to the previous behaviour, set *note +PYTHONLEGACYWINDOWSFSENCODING: 4f7. or call *note +sys._enablelegacywindowsfsencoding(): 4f8. + +See PEP 529(1) for more information and discussion of code modifications +that may be required. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0529 + + +File: python.info, Node: PEP 528 Change Windows console encoding to UTF-8, Next: PEP 520 Preserving Class Attribute Definition Order, Prev: PEP 529 Change Windows filesystem encoding to UTF-8, Up: New Features<3> + +1.3.2.11 PEP 528: Change Windows console encoding to UTF-8 +.......................................................... + +The default console on Windows will now accept all Unicode characters +and provide correctly read str objects to Python code. ‘sys.stdin’, +‘sys.stdout’ and ‘sys.stderr’ now default to utf-8 encoding. + +This change only applies when using an interactive console, and not when +redirecting files or pipes. To revert to the previous behaviour for +interactive console use, set *note PYTHONLEGACYWINDOWSSTDIO: 4fa. + +See also +........ + +PEP 528(1) – Change Windows console encoding to UTF-8 + + PEP written and implemented by Steve Dower. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0528 + + +File: python.info, Node: PEP 520 Preserving Class Attribute Definition Order, Next: PEP 468 Preserving Keyword Argument Order, Prev: PEP 528 Change Windows console encoding to UTF-8, Up: New Features<3> + +1.3.2.12 PEP 520: Preserving Class Attribute Definition Order +............................................................. + +Attributes in a class definition body have a natural ordering: the same +order in which the names appear in the source. This order is now +preserved in the new class’s *note __dict__: 4fc. attribute. + +Also, the effective default class `execution' namespace (returned from +*note type.__prepare__(): 4fd.) is now an insertion-order-preserving +mapping. + +See also +........ + +PEP 520(1) – Preserving Class Attribute Definition Order + + PEP written and implemented by Eric Snow. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0520 + + +File: python.info, Node: PEP 468 Preserving Keyword Argument Order, Next: New dict implementation, Prev: PEP 520 Preserving Class Attribute Definition Order, Up: New Features<3> + +1.3.2.13 PEP 468: Preserving Keyword Argument Order +................................................... + +‘**kwargs’ in a function signature is now guaranteed to be an +insertion-order-preserving mapping. + +See also +........ + +PEP 468(1) – Preserving Keyword Argument Order + + PEP written and implemented by Eric Snow. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0468 + + +File: python.info, Node: New dict implementation, Next: PEP 523 Adding a frame evaluation API to CPython, Prev: PEP 468 Preserving Keyword Argument Order, Up: New Features<3> + +1.3.2.14 New dict implementation +................................ + +The *note dict: 2fc. type now uses a “compact” representation based on a +proposal by Raymond Hettinger(1) which was first implemented by PyPy(2). +The memory usage of the new *note dict(): 1b8. is between 20% and 25% +smaller compared to Python 3.5. + +The order-preserving aspect of this new implementation is considered an +implementation detail and should not be relied upon (this may change in +the future, but it is desired to have this new dict implementation in +the language for a few releases before changing the language spec to +mandate order-preserving semantics for all current and future Python +implementations; this also helps preserve backwards-compatibility with +older versions of the language where random iteration order is still in +effect, e.g. Python 3.5). + +(Contributed by INADA Naoki in bpo-27350(3). Idea originally suggested +by Raymond Hettinger(4).) + + ---------- Footnotes ---------- + + (1) +https://mail.python.org/pipermail/python-dev/2012-December/123028.html + + (2) +https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html + + (3) https://bugs.python.org/issue27350 + + (4) +https://mail.python.org/pipermail/python-dev/2012-December/123028.html + + +File: python.info, Node: PEP 523 Adding a frame evaluation API to CPython, Next: PYTHONMALLOC environment variable, Prev: New dict implementation, Up: New Features<3> + +1.3.2.15 PEP 523: Adding a frame evaluation API to CPython +.......................................................... + +While Python provides extensive support to customize how code executes, +one place it has not done so is in the evaluation of frame objects. If +you wanted some way to intercept frame evaluation in Python there really +wasn’t any way without directly manipulating function pointers for +defined functions. + +PEP 523(1) changes this by providing an API to make frame evaluation +pluggable at the C level. This will allow for tools such as debuggers +and JITs to intercept frame evaluation before the execution of Python +code begins. This enables the use of alternative evaluation +implementations for Python code, tracking frame evaluation, etc. + +This API is not part of the limited C API and is marked as private to +signal that usage of this API is expected to be limited and only +applicable to very select, low-level use-cases. Semantics of the API +will change with Python as necessary. + +See also +........ + +PEP 523(2) – Adding a frame evaluation API to CPython + + PEP written by Brett Cannon and Dino Viehland. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0523 + + (2) https://www.python.org/dev/peps/pep-0523 + + +File: python.info, Node: PYTHONMALLOC environment variable, Next: DTrace and SystemTap probing support, Prev: PEP 523 Adding a frame evaluation API to CPython, Up: New Features<3> + +1.3.2.16 PYTHONMALLOC environment variable +.......................................... + +The new *note PYTHONMALLOC: 503. environment variable allows setting the +Python memory allocators and installing debug hooks. + +It is now possible to install debug hooks on Python memory allocators on +Python compiled in release mode using ‘PYTHONMALLOC=debug’. Effects of +debug hooks: + + * Newly allocated memory is filled with the byte ‘0xCB’ + + * Freed memory is filled with the byte ‘0xDB’ + + * Detect violations of the Python memory allocator API. For example, + *note PyObject_Free(): 504. called on a memory block allocated by + *note PyMem_Malloc(): 505. + + * Detect writes before the start of a buffer (buffer underflows) + + * Detect writes after the end of a buffer (buffer overflows) + + * Check that the *note GIL: 506. is held when allocator functions of + *note PYMEM_DOMAIN_OBJ: 507. (ex: *note PyObject_Malloc(): 508.) + and *note PYMEM_DOMAIN_MEM: 509. (ex: *note PyMem_Malloc(): 505.) + domains are called. + +Checking if the GIL is held is also a new feature of Python 3.6. + +See the *note PyMem_SetupDebugHooks(): 50a. function for debug hooks on +Python memory allocators. + +It is now also possible to force the usage of the ‘malloc()’ allocator +of the C library for all Python memory allocations using +‘PYTHONMALLOC=malloc’. This is helpful when using external memory +debuggers like Valgrind on a Python compiled in release mode. + +On error, the debug hooks on Python memory allocators now use the *note +tracemalloc: 114. module to get the traceback where a memory block was +allocated. + +Example of fatal error on buffer overflow using ‘python3.6 -X +tracemalloc=5’ (store 5 frames in traces): + + Debug memory block at address p=0x7fbcd41666f8: API 'o' + 4 bytes originally requested + The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. + The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): + at tail+0: 0x02 *** OUCH + at tail+1: 0xfb + at tail+2: 0xfb + at tail+3: 0xfb + at tail+4: 0xfb + at tail+5: 0xfb + at tail+6: 0xfb + at tail+7: 0xfb + The block was made by call #1233329 to debug malloc/realloc. + Data at p: 1a 2b 30 00 + + Memory block allocated at (most recent call first): + File "test/test_bytes.py", line 323 + File "unittest/case.py", line 600 + File "unittest/case.py", line 648 + File "unittest/suite.py", line 122 + File "unittest/suite.py", line 84 + + Fatal Python error: bad trailing pad byte + + Current thread 0x00007fbcdbd32700 (most recent call first): + File "test/test_bytes.py", line 323 in test_hex + File "unittest/case.py", line 600 in run + File "unittest/case.py", line 648 in __call__ + File "unittest/suite.py", line 122 in run + File "unittest/suite.py", line 84 in __call__ + File "unittest/suite.py", line 122 in run + File "unittest/suite.py", line 84 in __call__ + ... + +(Contributed by Victor Stinner in bpo-26516(1) and bpo-26564(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26516 + + (2) https://bugs.python.org/issue26564 + + +File: python.info, Node: DTrace and SystemTap probing support, Prev: PYTHONMALLOC environment variable, Up: New Features<3> + +1.3.2.17 DTrace and SystemTap probing support +............................................. + +Python can now be built ‘--with-dtrace’ which enables static markers for +the following events in the interpreter: + + * function call/return + + * garbage collection started/finished + + * line of code executed. + +This can be used to instrument running interpreters in production, +without the need to recompile specific debug builds or providing +application-specific profiling/debugging code. + +More details in *note Instrumenting CPython with DTrace and SystemTap: +50c. + +The current implementation is tested on Linux and macOS. Additional +markers may be added in the future. + +(Contributed by Łukasz Langa in bpo-21590(1), based on patches by Jesús +Cea Avión, David Malcolm, and Nikhil Benesch.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21590 + + +File: python.info, Node: Other Language Changes<3>, Next: New Modules<3>, Prev: New Features<3>, Up: What’s New In Python 3 6 + +1.3.3 Other Language Changes +---------------------------- + +Some smaller changes made to the core Python language are: + + * A ‘global’ or ‘nonlocal’ statement must now textually appear before + the first use of the affected name in the same scope. Previously + this was a *note SyntaxWarning: 191. + + * It is now possible to set a *note special method: 50e. to ‘None’ to + indicate that the corresponding operation is not available. For + example, if a class sets *note __iter__(): 50f. to ‘None’, the + class is not iterable. (Contributed by Andrew Barnert and Ivan + Levkivskyi in bpo-25958(1).) + + * Long sequences of repeated traceback lines are now abbreviated as + ‘"[Previous line repeated {count} more times]"’ (see *note + traceback: 510. for an example). (Contributed by Emanuel Barry in + bpo-26823(2).) + + * Import now raises the new exception *note ModuleNotFoundError: 39a. + (subclass of *note ImportError: 334.) when it cannot find a module. + Code that currently checks for ImportError (in try-except) will + still work. (Contributed by Eric Snow in bpo-15767(3).) + + * Class methods relying on zero-argument ‘super()’ will now work + correctly when called from metaclass methods during class creation. + (Contributed by Martin Teichmann in bpo-23722(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25958 + + (2) https://bugs.python.org/issue26823 + + (3) https://bugs.python.org/issue15767 + + (4) https://bugs.python.org/issue23722 + + +File: python.info, Node: New Modules<3>, Next: Improved Modules<3>, Prev: Other Language Changes<3>, Up: What’s New In Python 3 6 + +1.3.4 New Modules +----------------- + +* Menu: + +* secrets:: + + +File: python.info, Node: secrets, Up: New Modules<3> + +1.3.4.1 secrets +............... + +The main purpose of the new *note secrets: e4. module is to provide an +obvious way to reliably generate cryptographically strong pseudo-random +values suitable for managing secrets, such as account authentication, +tokens, and similar. + + Warning: Note that the pseudo-random generators in the *note + random: dc. module should `NOT' be used for security purposes. Use + *note secrets: e4. on Python 3.6+ and *note os.urandom(): 4d1. on + Python 3.5 and earlier. + +See also +........ + +PEP 506(1) – Adding A Secrets Module To The Standard Library + + PEP written and implemented by Steven D’Aprano. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0506 + + +File: python.info, Node: Improved Modules<3>, Next: Optimizations<3>, Prev: New Modules<3>, Up: What’s New In Python 3 6 + +1.3.5 Improved Modules +---------------------- + +* Menu: + +* array:: +* ast: ast<2>. +* asyncio: asyncio<4>. +* binascii: binascii<2>. +* cmath:: +* collections: collections<4>. +* concurrent.futures: concurrent futures<2>. +* contextlib: contextlib<2>. +* datetime: datetime<3>. +* decimal: decimal<2>. +* distutils: distutils<2>. +* email:: +* encodings:: +* enum: enum<3>. +* faulthandler:: +* fileinput:: +* hashlib:: +* http.client: http client<2>. +* idlelib and IDLE: idlelib and IDLE<2>. +* importlib: importlib<3>. +* inspect: inspect<2>. +* json:: +* logging: logging<3>. +* math: math<3>. +* multiprocessing: multiprocessing<3>. +* os: os<3>. +* pathlib: pathlib<3>. +* pdb: pdb<2>. +* pickle: pickle<2>. +* pickletools:: +* pydoc: pydoc<2>. +* random:: +* re: re<2>. +* readline:: +* rlcompleter:: +* shlex: shlex<2>. +* site:: +* sqlite3: sqlite3<2>. +* socket: socket<4>. +* socketserver: socketserver<2>. +* ssl: ssl<4>. +* statistics: statistics<2>. +* struct:: +* subprocess: subprocess<2>. +* sys: sys<4>. +* telnetlib:: +* time: time<3>. +* timeit:: +* tkinter: tkinter<3>. +* traceback:: +* tracemalloc: tracemalloc<2>. +* typing: typing<2>. +* unicodedata: unicodedata<3>. +* unittest.mock: unittest mock<2>. +* urllib.request: urllib request. +* urllib.robotparser: urllib robotparser. +* venv: venv<2>. +* warnings: warnings<2>. +* winreg:: +* winsound:: +* xmlrpc.client: xmlrpc client. +* zipfile: zipfile<2>. +* zlib:: + + +File: python.info, Node: array, Next: ast<2>, Up: Improved Modules<3> + +1.3.5.1 array +............. + +Exhausted iterators of *note array.array: 451. will now stay exhausted +even if the iterated array is extended. This is consistent with the +behavior of other mutable sequences. + +Contributed by Serhiy Storchaka in bpo-26492(1). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26492 + + +File: python.info, Node: ast<2>, Next: asyncio<4>, Prev: array, Up: Improved Modules<3> + +1.3.5.2 ast +........... + +The new ‘ast.Constant’ AST node has been added. It can be used by +external AST optimizers for the purposes of constant folding. + +Contributed by Victor Stinner in bpo-26146(1). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26146 + + +File: python.info, Node: asyncio<4>, Next: binascii<2>, Prev: ast<2>, Up: Improved Modules<3> + +1.3.5.3 asyncio +............... + +Starting with Python 3.6 the ‘asyncio’ module is no longer provisional +and its API is considered stable. + +Notable changes in the *note asyncio: a. module since Python 3.5.0 (all +backported to 3.5.x due to the provisional status): + + * The *note get_event_loop(): 355. function has been changed to + always return the currently running loop when called from + coroutines and callbacks. (Contributed by Yury Selivanov in + bpo-28613(1).) + + * The *note ensure_future(): 517. function and all functions that use + it, such as *note loop.run_until_complete(): 518, now accept all + kinds of *note awaitable objects: 519. (Contributed by Yury + Selivanov.) + + * New *note run_coroutine_threadsafe(): 51a. function to submit + coroutines to event loops from other threads. (Contributed by + Vincent Michel.) + + * New *note Transport.is_closing(): 51b. method to check if the + transport is closing or closed. (Contributed by Yury Selivanov.) + + * The *note loop.create_server(): 35d. method can now accept a list + of hosts. (Contributed by Yann Sionneau.) + + * New *note loop.create_future(): 51c. method to create Future + objects. This allows alternative event loop implementations, such + as uvloop(2), to provide a faster *note asyncio.Future: 443. + implementation. (Contributed by Yury Selivanov in bpo-27041(3).) + + * New *note loop.get_exception_handler(): 51d. method to get the + current exception handler. (Contributed by Yury Selivanov in + bpo-27040(4).) + + * New *note StreamReader.readuntil(): 51e. method to read data from + the stream until a separator bytes sequence appears. (Contributed + by Mark Korenberg.) + + * The performance of *note StreamReader.readexactly(): 51f. has been + improved. (Contributed by Mark Korenberg in bpo-28370(5).) + + * The *note loop.getaddrinfo(): 4a1. method is optimized to avoid + calling the system ‘getaddrinfo’ function if the address is already + resolved. (Contributed by A. Jesse Jiryu Davis.) + + * The *note loop.stop(): 520. method has been changed to stop the + loop immediately after the current iteration. Any new callbacks + scheduled as a result of the last iteration will be discarded. + (Contributed by Guido van Rossum in bpo-25593(6).) + + * ‘Future.set_exception’ will now raise *note TypeError: 192. when + passed an instance of the *note StopIteration: 486. exception. + (Contributed by Chris Angelico in bpo-26221(7).) + + * New *note loop.connect_accepted_socket(): 365. method to be used by + servers that accept connections outside of asyncio, but that use + asyncio to handle them. (Contributed by Jim Fulton in + bpo-27392(8).) + + * ‘TCP_NODELAY’ flag is now set for all TCP transports by default. + (Contributed by Yury Selivanov in bpo-27456(9).) + + * New *note loop.shutdown_asyncgens(): 521. to properly close pending + asynchronous generators before closing the loop. (Contributed by + Yury Selivanov in bpo-28003(10).) + + * *note Future: 443. and *note Task: 1ad. classes now have an + optimized C implementation which makes asyncio code up to 30% + faster. (Contributed by Yury Selivanov and INADA Naoki in + bpo-26081(11) and bpo-28544(12).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28613 + + (2) https://github.com/MagicStack/uvloop + + (3) https://bugs.python.org/issue27041 + + (4) https://bugs.python.org/issue27040 + + (5) https://bugs.python.org/issue28370 + + (6) https://bugs.python.org/issue25593 + + (7) https://bugs.python.org/issue26221 + + (8) https://bugs.python.org/issue27392 + + (9) https://bugs.python.org/issue27456 + + (10) https://bugs.python.org/issue28003 + + (11) https://bugs.python.org/issue26081 + + (12) https://bugs.python.org/issue28544 + + +File: python.info, Node: binascii<2>, Next: cmath, Prev: asyncio<4>, Up: Improved Modules<3> + +1.3.5.4 binascii +................ + +The *note b2a_base64(): 523. function now accepts an optional `newline' +keyword argument to control whether the newline character is appended to +the return value. (Contributed by Victor Stinner in bpo-25357(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25357 + + +File: python.info, Node: cmath, Next: collections<4>, Prev: binascii<2>, Up: Improved Modules<3> + +1.3.5.5 cmath +............. + +The new *note cmath.tau: 525. (`τ') constant has been added. +(Contributed by Lisa Roach in bpo-12345(1), see PEP 628(2) for details.) + +New constants: *note cmath.inf: 526. and *note cmath.nan: 527. to match +*note math.inf: 528. and *note math.nan: 529, and also *note cmath.infj: +52a. and *note cmath.nanj: 52b. to match the format used by complex +repr. (Contributed by Mark Dickinson in bpo-23229(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12345 + + (2) https://www.python.org/dev/peps/pep-0628 + + (3) https://bugs.python.org/issue23229 + + +File: python.info, Node: collections<4>, Next: concurrent futures<2>, Prev: cmath, Up: Improved Modules<3> + +1.3.5.6 collections +................... + +The new *note Collection: 52d. abstract base class has been added to +represent sized iterable container classes. (Contributed by Ivan +Levkivskyi, docs by Neil Girdhar in bpo-27598(1).) + +The new *note Reversible: 52e. abstract base class represents iterable +classes that also provide the *note __reversed__(): 52f. method. +(Contributed by Ivan Levkivskyi in bpo-25987(2).) + +The new *note AsyncGenerator: 530. abstract base class represents +asynchronous generators. (Contributed by Yury Selivanov in +bpo-28720(3).) + +The *note namedtuple(): 1b7. function now accepts an optional keyword +argument `module', which, when specified, is used for the ‘__module__’ +attribute of the returned named tuple class. (Contributed by Raymond +Hettinger in bpo-17941(4).) + +The `verbose' and `rename' arguments for *note namedtuple(): 1b7. are +now keyword-only. (Contributed by Raymond Hettinger in bpo-25628(5).) + +Recursive *note collections.deque: 531. instances can now be pickled. +(Contributed by Serhiy Storchaka in bpo-26482(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27598 + + (2) https://bugs.python.org/issue25987 + + (3) https://bugs.python.org/issue28720 + + (4) https://bugs.python.org/issue17941 + + (5) https://bugs.python.org/issue25628 + + (6) https://bugs.python.org/issue26482 + + +File: python.info, Node: concurrent futures<2>, Next: contextlib<2>, Prev: collections<4>, Up: Improved Modules<3> + +1.3.5.7 concurrent.futures +.......................... + +The *note ThreadPoolExecutor: 27a. class constructor now accepts an +optional `thread_name_prefix' argument to make it possible to customize +the names of the threads created by the pool. (Contributed by Gregory +P. Smith in bpo-27664(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27664 + + +File: python.info, Node: contextlib<2>, Next: datetime<3>, Prev: concurrent futures<2>, Up: Improved Modules<3> + +1.3.5.8 contextlib +.................. + +The *note contextlib.AbstractContextManager: 534. class has been added +to provide an abstract base class for context managers. It provides a +sensible default implementation for ‘__enter__()’ which returns ‘self’ +and leaves ‘__exit__()’ an abstract method. A matching class has been +added to the *note typing: 119. module as *note typing.ContextManager: +535. (Contributed by Brett Cannon in bpo-25609(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25609 + + +File: python.info, Node: datetime<3>, Next: decimal<2>, Prev: contextlib<2>, Up: Improved Modules<3> + +1.3.5.9 datetime +................ + +The *note datetime: 194. and *note time: 4f3. classes have the new +‘fold’ attribute used to disambiguate local time when necessary. Many +functions in the *note datetime: 31. have been updated to support local +time disambiguation. See *note Local Time Disambiguation: 4ce. section +for more information. (Contributed by Alexander Belopolsky in +bpo-24773(1).) + +The *note datetime.strftime(): 537. and *note date.strftime(): 538. +methods now support ISO 8601 date directives ‘%G’, ‘%u’ and ‘%V’. +(Contributed by Ashley Anderson in bpo-12006(2).) + +The *note datetime.isoformat(): 37f. function now accepts an optional +`timespec' argument that specifies the number of additional components +of the time value to include. (Contributed by Alessandro Cucci and +Alexander Belopolsky in bpo-19475(3).) + +The *note datetime.combine(): 539. now accepts an optional `tzinfo' +argument. (Contributed by Alexander Belopolsky in bpo-27661(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24773 + + (2) https://bugs.python.org/issue12006 + + (3) https://bugs.python.org/issue19475 + + (4) https://bugs.python.org/issue27661 + + +File: python.info, Node: decimal<2>, Next: distutils<2>, Prev: datetime<3>, Up: Improved Modules<3> + +1.3.5.10 decimal +................ + +New *note Decimal.as_integer_ratio(): 53b. method that returns a pair +‘(n, d)’ of integers that represent the given *note Decimal: 188. +instance as a fraction, in lowest terms and with a positive denominator: + + >>> Decimal('-3.14').as_integer_ratio() + (-157, 50) + +(Contributed by Stefan Krah amd Mark Dickinson in bpo-25928(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25928 + + +File: python.info, Node: distutils<2>, Next: email, Prev: decimal<2>, Up: Improved Modules<3> + +1.3.5.11 distutils +.................. + +The ‘default_format’ attribute has been removed from +‘distutils.command.sdist.sdist’ and the ‘formats’ attribute defaults to +‘['gztar']’. Although not anticipated, any code relying on the presence +of ‘default_format’ may need to be adapted. See bpo-27819(1) for more +details. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27819 + + +File: python.info, Node: email, Next: encodings, Prev: distutils<2>, Up: Improved Modules<3> + +1.3.5.12 email +.............. + +The new email API, enabled via the `policy' keyword to various +constructors, is no longer provisional. The *note email: 69. +documentation has been reorganized and rewritten to focus on the new +API, while retaining the old documentation for the legacy API. +(Contributed by R. David Murray in bpo-24277(1).) + +The *note email.mime: 73. classes now all accept an optional `policy' +keyword. (Contributed by Berker Peksag in bpo-27331(2).) + +The *note DecodedGenerator: 53e. now supports the `policy' keyword. + +There is a new *note policy: 75. attribute, *note message_factory: 53f, +that controls what class is used by default when the parser creates new +message objects. For the *note email.policy.compat32: 540. policy this +is *note Message: 541, for the new policies it is *note EmailMessage: +542. (Contributed by R. David Murray in bpo-20476(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24277 + + (2) https://bugs.python.org/issue27331 + + (3) https://bugs.python.org/issue20476 + + +File: python.info, Node: encodings, Next: enum<3>, Prev: email, Up: Improved Modules<3> + +1.3.5.13 encodings +.................. + +On Windows, added the ‘'oem'’ encoding to use ‘CP_OEMCP’, and the +‘'ansi'’ alias for the existing ‘'mbcs'’ encoding, which uses the +‘CP_ACP’ code page. (Contributed by Steve Dower in bpo-27959(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27959 + + +File: python.info, Node: enum<3>, Next: faulthandler, Prev: encodings, Up: Improved Modules<3> + +1.3.5.14 enum +............. + +Two new enumeration base classes have been added to the *note enum: 7b. +module: *note Flag: 545. and ‘IntFlags’. Both are used to define +constants that can be combined using the bitwise operators. +(Contributed by Ethan Furman in bpo-23591(1).) + +Many standard library modules have been updated to use the ‘IntFlags’ +class for their constants. + +The new *note enum.auto: 546. value can be used to assign values to enum +members automatically: + + >>> from enum import Enum, auto + >>> class Color(Enum): + ... red = auto() + ... blue = auto() + ... green = auto() + ... + >>> list(Color) + [, , ] + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23591 + + +File: python.info, Node: faulthandler, Next: fileinput, Prev: enum<3>, Up: Improved Modules<3> + +1.3.5.15 faulthandler +..................... + +On Windows, the *note faulthandler: 7d. module now installs a handler +for Windows exceptions: see *note faulthandler.enable(): 548. +(Contributed by Victor Stinner in bpo-23848(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23848 + + +File: python.info, Node: fileinput, Next: hashlib, Prev: faulthandler, Up: Improved Modules<3> + +1.3.5.16 fileinput +.................. + +*note hook_encoded(): 54a. now supports the `errors' argument. +(Contributed by Joseph Hackman in bpo-25788(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25788 + + +File: python.info, Node: hashlib, Next: http client<2>, Prev: fileinput, Up: Improved Modules<3> + +1.3.5.17 hashlib +................ + +*note hashlib: 8d. supports OpenSSL 1.1.0. The minimum recommend +version is 1.0.2. (Contributed by Christian Heimes in bpo-26470(1).) + +BLAKE2 hash functions were added to the module. *note blake2b(): 54c. +and *note blake2s(): 54d. are always available and support the full +feature set of BLAKE2. (Contributed by Christian Heimes in bpo-26798(2) +based on code by Dmitry Chestnykh and Samuel Neves. Documentation +written by Dmitry Chestnykh.) + +The SHA-3 hash functions ‘sha3_224()’, ‘sha3_256()’, ‘sha3_384()’, +‘sha3_512()’, and SHAKE hash functions ‘shake_128()’ and ‘shake_256()’ +were added. (Contributed by Christian Heimes in bpo-16113(3). Keccak +Code Package by Guido Bertoni, Joan Daemen, Michaël Peeters, Gilles Van +Assche, and Ronny Van Keer.) + +The password-based key derivation function *note scrypt(): 4d2. is now +available with OpenSSL 1.1.0 and newer. (Contributed by Christian +Heimes in bpo-27928(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26470 + + (2) https://bugs.python.org/issue26798 + + (3) https://bugs.python.org/issue16113 + + (4) https://bugs.python.org/issue27928 + + +File: python.info, Node: http client<2>, Next: idlelib and IDLE<2>, Prev: hashlib, Up: Improved Modules<3> + +1.3.5.18 http.client +.................... + +*note HTTPConnection.request(): 54f. and *note endheaders(): 550. both +now support chunked encoding request bodies. (Contributed by Demian +Brecht and Rolf Krahl in bpo-12319(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12319 + + +File: python.info, Node: idlelib and IDLE<2>, Next: importlib<3>, Prev: http client<2>, Up: Improved Modules<3> + +1.3.5.19 idlelib and IDLE +......................... + +The idlelib package is being modernized and refactored to make IDLE look +and work better and to make the code easier to understand, test, and +improve. Part of making IDLE look better, especially on Linux and Mac, +is using ttk widgets, mostly in the dialogs. As a result, IDLE no +longer runs with tcl/tk 8.4. It now requires tcl/tk 8.5 or 8.6. We +recommend running the latest release of either. + +‘Modernizing’ includes renaming and consolidation of idlelib modules. +The renaming of files with partial uppercase names is similar to the +renaming of, for instance, Tkinter and TkFont to tkinter and +tkinter.font in 3.0. As a result, imports of idlelib files that worked +in 3.5 will usually not work in 3.6. At least a module name change will +be needed (see idlelib/README.txt), sometimes more. (Name changes +contributed by Al Swiegart and Terry Reedy in bpo-24225(1). Most +idlelib patches since have been and will be part of the process.) + +In compensation, the eventual result with be that some idlelib classes +will be easier to use, with better APIs and docstrings explaining them. +Additional useful information will be added to idlelib when available. + +New in 3.6.2: + +Multiple fixes for autocompletion. (Contributed by Louie Lu in +bpo-15786(2).) + +New in 3.6.3: + +Module Browser (on the File menu, formerly called Class Browser), now +displays nested functions and classes in addition to top-level functions +and classes. (Contributed by Guilherme Polo, Cheryl Sabella, and Terry +Jan Reedy in bpo-1612262(3).) + +The IDLE features formerly implemented as extensions have been +reimplemented as normal features. Their settings have been moved from +the Extensions tab to other dialog tabs. (Contributed by Charles +Wohlganger and Terry Jan Reedy in bpo-27099(4).) + +The Settings dialog (Options, Configure IDLE) has been partly rewritten +to improve both appearance and function. (Contributed by Cheryl Sabella +and Terry Jan Reedy in multiple issues.) + +New in 3.6.4: + +The font sample now includes a selection of non-Latin characters so that +users can better see the effect of selecting a particular font. +(Contributed by Terry Jan Reedy in bpo-13802(5).) The sample can be +edited to include other characters. (Contributed by Serhiy Storchaka in +bpo-31860(6).) + +New in 3.6.6: + +Editor code context option revised. Box displays all context lines up +to maxlines. Clicking on a context line jumps the editor to that line. +Context colors for custom themes is added to Highlights tab of Settings +dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in +bpo-33642(7), bpo-33768(8), and bpo-33679(9).) + +On Windows, a new API call tells Windows that tk scales for DPI. On +Windows 8.1+ or 10, with DPI compatibility properties of the Python +binary unchanged, and a monitor resolution greater than 96 DPI, this +should make text and lines sharper. It should otherwise have no effect. +(Contributed by Terry Jan Reedy in bpo-33656(10).) + +New in 3.6.7: + +Output over N lines (50 by default) is squeezed down to a button. N can +be changed in the PyShell section of the General page of the Settings +dialog. Fewer, but possibly extra long, lines can be squeezed by right +clicking on the output. Squeezed output can be expanded in place by +double-clicking the button or into the clipboard or a separate window by +right-clicking the button. (Contributed by Tal Einat in +bpo-1529353(11).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24225 + + (2) https://bugs.python.org/issue15786 + + (3) https://bugs.python.org/issue1612262 + + (4) https://bugs.python.org/issue27099 + + (5) https://bugs.python.org/issue13802 + + (6) https://bugs.python.org/issue31860 + + (7) https://bugs.python.org/issue33642 + + (8) https://bugs.python.org/issue33768 + + (9) https://bugs.python.org/issue33679 + + (10) https://bugs.python.org/issue33656 + + (11) https://bugs.python.org/issue1529353 + + +File: python.info, Node: importlib<3>, Next: inspect<2>, Prev: idlelib and IDLE<2>, Up: Improved Modules<3> + +1.3.5.20 importlib +.................. + +Import now raises the new exception *note ModuleNotFoundError: 39a. +(subclass of *note ImportError: 334.) when it cannot find a module. +Code that current checks for ‘ImportError’ (in try-except) will still +work. (Contributed by Eric Snow in bpo-15767(1).) + +*note importlib.util.LazyLoader: 553. now calls *note create_module(): +554. on the wrapped loader, removing the restriction that *note +importlib.machinery.BuiltinImporter: 555. and *note +importlib.machinery.ExtensionFileLoader: 556. couldn’t be used with +*note importlib.util.LazyLoader: 553. + +*note importlib.util.cache_from_source(): 557, *note +importlib.util.source_from_cache(): 558, and *note +importlib.util.spec_from_file_location(): 559. now accept a *note +path-like object: 36a. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15767 + + +File: python.info, Node: inspect<2>, Next: json, Prev: importlib<3>, Up: Improved Modules<3> + +1.3.5.21 inspect +................ + +The *note inspect.signature(): 55b. function now reports the implicit +‘.0’ parameters generated by the compiler for comprehension and +generator expression scopes as if they were positional-only parameters +called ‘implicit0’. (Contributed by Jelle Zijlstra in bpo-19611(1).) + +To reduce code churn when upgrading from Python 2.7 and the legacy *note +inspect.getargspec(): 55c. API, the previously documented deprecation of +*note inspect.getfullargspec(): 55d. has been reversed. While this +function is convenient for single/source Python 2/3 code bases, the +richer *note inspect.signature(): 55b. interface remains the recommended +approach for new code. (Contributed by Nick Coghlan in bpo-27172(2)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19611 + + (2) https://bugs.python.org/issue27172 + + +File: python.info, Node: json, Next: logging<3>, Prev: inspect<2>, Up: Improved Modules<3> + +1.3.5.22 json +............. + +*note json.load(): 55f. and *note json.loads(): 560. now support binary +input. Encoded JSON should be represented using either UTF-8, UTF-16, +or UTF-32. (Contributed by Serhiy Storchaka in bpo-17909(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17909 + + +File: python.info, Node: logging<3>, Next: math<3>, Prev: json, Up: Improved Modules<3> + +1.3.5.23 logging +................ + +The new *note WatchedFileHandler.reopenIfNeeded(): 562. method has been +added to add the ability to check if the log file needs to be reopened. +(Contributed by Marian Horban in bpo-24884(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24884 + + +File: python.info, Node: math<3>, Next: multiprocessing<3>, Prev: logging<3>, Up: Improved Modules<3> + +1.3.5.24 math +............. + +The tau (`τ') constant has been added to the *note math: b1. and *note +cmath: 19. modules. (Contributed by Lisa Roach in bpo-12345(1), see PEP +628(2) for details.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12345 + + (2) https://www.python.org/dev/peps/pep-0628 + + +File: python.info, Node: multiprocessing<3>, Next: os<3>, Prev: math<3>, Up: Improved Modules<3> + +1.3.5.25 multiprocessing +........................ + +*note Proxy Objects: 565. returned by ‘multiprocessing.Manager()’ can +now be nested. (Contributed by Davin Potts in bpo-6766(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6766 + + +File: python.info, Node: os<3>, Next: pathlib<3>, Prev: multiprocessing<3>, Up: Improved Modules<3> + +1.3.5.26 os +........... + +See the summary of *note PEP 519: 4cd. for details on how the *note os: +c4. and *note os.path: c5. modules now support *note path-like objects: +36a. + +*note scandir(): 25f. now supports *note bytes: 331. paths on Windows. + +A new *note close(): 567. method allows explicitly closing a *note +scandir(): 25f. iterator. The *note scandir(): 25f. iterator now +supports the *note context manager: 568. protocol. If a ‘scandir()’ +iterator is neither exhausted nor explicitly closed a *note +ResourceWarning: 4d0. will be emitted in its destructor. (Contributed +by Serhiy Storchaka in bpo-25994(1).) + +On Linux, *note os.urandom(): 4d1. now blocks until the system urandom +entropy pool is initialized to increase the security. See the PEP +524(2) for the rationale. + +The Linux ‘getrandom()’ syscall (get random bytes) is now exposed as the +new *note os.getrandom(): 569. function. (Contributed by Victor +Stinner, part of the PEP 524(3)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25994 + + (2) https://www.python.org/dev/peps/pep-0524 + + (3) https://www.python.org/dev/peps/pep-0524 + + +File: python.info, Node: pathlib<3>, Next: pdb<2>, Prev: os<3>, Up: Improved Modules<3> + +1.3.5.27 pathlib +................ + +*note pathlib: c8. now supports *note path-like objects: 36a. +(Contributed by Brett Cannon in bpo-27186(1).) + +See the summary of *note PEP 519: 4cd. for details. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27186 + + +File: python.info, Node: pdb<2>, Next: pickle<2>, Prev: pathlib<3>, Up: Improved Modules<3> + +1.3.5.28 pdb +............ + +The *note Pdb: 56c. class constructor has a new optional `readrc' +argument to control whether ‘.pdbrc’ files should be read. + + +File: python.info, Node: pickle<2>, Next: pickletools, Prev: pdb<2>, Up: Improved Modules<3> + +1.3.5.29 pickle +............... + +Objects that need ‘__new__’ called with keyword arguments can now be +pickled using *note pickle protocols: 56e. older than protocol version +4. Protocol version 4 already supports this case. (Contributed by +Serhiy Storchaka in bpo-24164(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24164 + + +File: python.info, Node: pickletools, Next: pydoc<2>, Prev: pickle<2>, Up: Improved Modules<3> + +1.3.5.30 pickletools +.................... + +*note pickletools.dis(): 570. now outputs the implicit memo index for +the ‘MEMOIZE’ opcode. (Contributed by Serhiy Storchaka in +bpo-25382(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25382 + + +File: python.info, Node: pydoc<2>, Next: random, Prev: pickletools, Up: Improved Modules<3> + +1.3.5.31 pydoc +.............. + +The *note pydoc: d9. module has learned to respect the ‘MANPAGER’ +environment variable. (Contributed by Matthias Klose in bpo-8637(1).) + +*note help(): 572. and *note pydoc: d9. can now list named tuple fields +in the order they were defined rather than alphabetically. (Contributed +by Raymond Hettinger in bpo-24879(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8637 + + (2) https://bugs.python.org/issue24879 + + +File: python.info, Node: random, Next: re<2>, Prev: pydoc<2>, Up: Improved Modules<3> + +1.3.5.32 random +............... + +The new *note choices(): 574. function returns a list of elements of +specified size from the given population with optional weights. +(Contributed by Raymond Hettinger in bpo-18844(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18844 + + +File: python.info, Node: re<2>, Next: readline, Prev: random, Up: Improved Modules<3> + +1.3.5.33 re +........... + +Added support of modifier spans in regular expressions. Examples: +‘'(?i:p)ython'’ matches ‘'python'’ and ‘'Python'’, but not ‘'PYTHON'’; +‘'(?i)g(?-i:v)r'’ matches ‘'GvR'’ and ‘'gvr'’, but not ‘'GVR'’. +(Contributed by Serhiy Storchaka in bpo-433028(1).) + +Match object groups can be accessed by ‘__getitem__’, which is +equivalent to ‘group()’. So ‘mo['name']’ is now equivalent to +‘mo.group('name')’. (Contributed by Eric Smith in bpo-24454(2).) + +‘Match’ objects now support *note index-like objects: 18a. as group +indices. (Contributed by Jeroen Demeyer and Xiang Zhang in +bpo-27177(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue433028 + + (2) https://bugs.python.org/issue24454 + + (3) https://bugs.python.org/issue27177 + + +File: python.info, Node: readline, Next: rlcompleter, Prev: re<2>, Up: Improved Modules<3> + +1.3.5.34 readline +................. + +Added *note set_auto_history(): 577. to enable or disable automatic +addition of input to the history list. (Contributed by Tyler Crompton +in bpo-26870(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26870 + + +File: python.info, Node: rlcompleter, Next: shlex<2>, Prev: readline, Up: Improved Modules<3> + +1.3.5.35 rlcompleter +.................... + +Private and special attribute names now are omitted unless the prefix +starts with underscores. A space or a colon is added after some +completed keywords. (Contributed by Serhiy Storchaka in bpo-25011(1) +and bpo-25209(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25011 + + (2) https://bugs.python.org/issue25209 + + +File: python.info, Node: shlex<2>, Next: site, Prev: rlcompleter, Up: Improved Modules<3> + +1.3.5.36 shlex +.............. + +The *note shlex: 57a. has much *note improved shell compatibility: 57b. +through the new `punctuation_chars' argument to control which characters +are treated as punctuation. (Contributed by Vinay Sajip in +bpo-1521950(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1521950 + + +File: python.info, Node: site, Next: sqlite3<2>, Prev: shlex<2>, Up: Improved Modules<3> + +1.3.5.37 site +............. + +When specifying paths to add to *note sys.path: 488. in a ‘.pth’ file, +you may now specify file paths on top of directories (e.g. zip files). +(Contributed by Wolfgang Langner in bpo-26587(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26587 + + +File: python.info, Node: sqlite3<2>, Next: socket<4>, Prev: site, Up: Improved Modules<3> + +1.3.5.38 sqlite3 +................ + +*note sqlite3.Cursor.lastrowid: 57e. now supports the ‘REPLACE’ +statement. (Contributed by Alex LordThorsen in bpo-16864(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16864 + + +File: python.info, Node: socket<4>, Next: socketserver<2>, Prev: sqlite3<2>, Up: Improved Modules<3> + +1.3.5.39 socket +............... + +The *note ioctl(): 580. function now supports the *note +SIO_LOOPBACK_FAST_PATH: 581. control code. (Contributed by Daniel +Stokes in bpo-26536(1).) + +The *note getsockopt(): 582. constants ‘SO_DOMAIN’, ‘SO_PROTOCOL’, +‘SO_PEERSEC’, and ‘SO_PASSSEC’ are now supported. (Contributed by +Christian Heimes in bpo-26907(2).) + +The *note setsockopt(): 583. now supports the ‘setsockopt(level, +optname, None, optlen: int)’ form. (Contributed by Christian Heimes in +bpo-27744(3).) + +The socket module now supports the address family *note AF_ALG: 584. to +interface with Linux Kernel crypto API. ‘ALG_*’, ‘SOL_ALG’ and *note +sendmsg_afalg(): 585. were added. (Contributed by Christian Heimes in +bpo-27744(4) with support from Victor Stinner.) + +New Linux constants ‘TCP_USER_TIMEOUT’ and ‘TCP_CONGESTION’ were added. +(Contributed by Omar Sandoval, issue:‘26273’). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26536 + + (2) https://bugs.python.org/issue26907 + + (3) https://bugs.python.org/issue27744 + + (4) https://bugs.python.org/issue27744 + + +File: python.info, Node: socketserver<2>, Next: ssl<4>, Prev: socket<4>, Up: Improved Modules<3> + +1.3.5.40 socketserver +..................... + +Servers based on the *note socketserver: f0. module, including those +defined in *note http.server: 97, *note xmlrpc.server: 140. and *note +wsgiref.simple_server: 12f, now support the *note context manager: 568. +protocol. (Contributed by Aviv Palivoda in bpo-26404(1).) + +The ‘wfile’ attribute of *note StreamRequestHandler: 587. classes now +implements the *note io.BufferedIOBase: 588. writable interface. In +particular, calling *note write(): 589. is now guaranteed to send the +data in full. (Contributed by Martin Panter in bpo-26721(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26404 + + (2) https://bugs.python.org/issue26721 + + +File: python.info, Node: ssl<4>, Next: statistics<2>, Prev: socketserver<2>, Up: Improved Modules<3> + +1.3.5.41 ssl +............ + +*note ssl: f3. supports OpenSSL 1.1.0. The minimum recommend version is +1.0.2. (Contributed by Christian Heimes in bpo-26470(1).) + +3DES has been removed from the default cipher suites and ChaCha20 +Poly1305 cipher suites have been added. (Contributed by Christian +Heimes in bpo-27850(2) and bpo-27766(3).) + +*note SSLContext: 3e4. has better default configuration for options and +ciphers. (Contributed by Christian Heimes in bpo-28043(4).) + +SSL session can be copied from one client-side connection to another +with the new *note SSLSession: 58b. class. TLS session resumption can +speed up the initial handshake, reduce latency and improve performance +(Contributed by Christian Heimes in bpo-19500(5) based on a draft by +Alex Warhawk.) + +The new *note get_ciphers(): 58c. method can be used to get a list of +enabled ciphers in order of cipher priority. + +All constants and flags have been converted to *note IntEnum: 58d. and +‘IntFlags’. (Contributed by Christian Heimes in bpo-28025(6).) + +Server and client-side specific TLS protocols for *note SSLContext: 3e4. +were added. (Contributed by Christian Heimes in bpo-28085(7).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26470 + + (2) https://bugs.python.org/issue27850 + + (3) https://bugs.python.org/issue27766 + + (4) https://bugs.python.org/issue28043 + + (5) https://bugs.python.org/issue19500 + + (6) https://bugs.python.org/issue28025 + + (7) https://bugs.python.org/issue28085 + + +File: python.info, Node: statistics<2>, Next: struct, Prev: ssl<4>, Up: Improved Modules<3> + +1.3.5.42 statistics +................... + +A new *note harmonic_mean(): 58f. function has been added. (Contributed +by Steven D’Aprano in bpo-27181(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27181 + + +File: python.info, Node: struct, Next: subprocess<2>, Prev: statistics<2>, Up: Improved Modules<3> + +1.3.5.43 struct +............... + +*note struct: f8. now supports IEEE 754 half-precision floats via the +‘'e'’ format specifier. (Contributed by Eli Stevens, Mark Dickinson in +bpo-11734(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11734 + + +File: python.info, Node: subprocess<2>, Next: sys<4>, Prev: struct, Up: Improved Modules<3> + +1.3.5.44 subprocess +................... + +*note subprocess.Popen: 2b9. destructor now emits a *note +ResourceWarning: 4d0. warning if the child process is still running. +Use the context manager protocol (‘with proc: ...’) or explicitly call +the *note wait(): 592. method to read the exit status of the child +process. (Contributed by Victor Stinner in bpo-26741(1).) + +The *note subprocess.Popen: 2b9. constructor and all functions that pass +arguments through to it now accept `encoding' and `errors' arguments. +Specifying either of these will enable text mode for the `stdin', +`stdout' and `stderr' streams. (Contributed by Steve Dower in +bpo-6135(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26741 + + (2) https://bugs.python.org/issue6135 + + +File: python.info, Node: sys<4>, Next: telnetlib, Prev: subprocess<2>, Up: Improved Modules<3> + +1.3.5.45 sys +............ + +The new *note getfilesystemencodeerrors(): 594. function returns the +name of the error mode used to convert between Unicode filenames and +bytes filenames. (Contributed by Steve Dower in bpo-27781(1).) + +On Windows the return value of the *note getwindowsversion(): 595. +function now includes the `platform_version' field which contains the +accurate major version, minor version and build number of the current +operating system, rather than the version that is being emulated for the +process (Contributed by Steve Dower in bpo-27932(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27781 + + (2) https://bugs.python.org/issue27932 + + +File: python.info, Node: telnetlib, Next: time<3>, Prev: sys<4>, Up: Improved Modules<3> + +1.3.5.46 telnetlib +.................. + +*note Telnet: 597. is now a context manager (contributed by Stéphane +Wirtel in bpo-25485(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25485 + + +File: python.info, Node: time<3>, Next: timeit, Prev: telnetlib, Up: Improved Modules<3> + +1.3.5.47 time +............. + +The *note struct_time: 599. attributes ‘tm_gmtoff’ and ‘tm_zone’ are now +available on all platforms. + + +File: python.info, Node: timeit, Next: tkinter<3>, Prev: time<3>, Up: Improved Modules<3> + +1.3.5.48 timeit +............... + +The new *note Timer.autorange(): 59b. convenience method has been added +to call *note Timer.timeit(): 59c. repeatedly so that the total run time +is greater or equal to 200 milliseconds. (Contributed by Steven +D’Aprano in bpo-6422(1).) + +*note timeit: 10b. now warns when there is substantial (4x) variance +between best and worst times. (Contributed by Serhiy Storchaka in +bpo-23552(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6422 + + (2) https://bugs.python.org/issue23552 + + +File: python.info, Node: tkinter<3>, Next: traceback, Prev: timeit, Up: Improved Modules<3> + +1.3.5.49 tkinter +................ + +Added methods ‘trace_add()’, ‘trace_remove()’ and ‘trace_info()’ in the +‘tkinter.Variable’ class. They replace old methods ‘trace_variable()’, +‘trace()’, ‘trace_vdelete()’ and ‘trace_vinfo()’ that use obsolete Tcl +commands and might not work in future versions of Tcl. (Contributed by +Serhiy Storchaka in bpo-22115(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22115 + + +File: python.info, Node: traceback, Next: tracemalloc<2>, Prev: tkinter<3>, Up: Improved Modules<3> + +1.3.5.50 traceback +.................. + +Both the traceback module and the interpreter’s builtin exception +display now abbreviate long sequences of repeated lines in tracebacks as +shown in the following example: + + >>> def f(): f() + ... + >>> f() + Traceback (most recent call last): + File "", line 1, in + File "", line 1, in f + File "", line 1, in f + File "", line 1, in f + [Previous line repeated 995 more times] + RecursionError: maximum recursion depth exceeded + +(Contributed by Emanuel Barry in bpo-26823(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26823 + + +File: python.info, Node: tracemalloc<2>, Next: typing<2>, Prev: traceback, Up: Improved Modules<3> + +1.3.5.51 tracemalloc +.................... + +The *note tracemalloc: 114. module now supports tracing memory +allocations in multiple different address spaces. + +The new *note DomainFilter: 5a0. filter class has been added to filter +block traces by their address space (domain). + +(Contributed by Victor Stinner in bpo-26588(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26588 + + +File: python.info, Node: typing<2>, Next: unicodedata<3>, Prev: tracemalloc<2>, Up: Improved Modules<3> + +1.3.5.52 typing +............... + +Since the *note typing: 119. module is *note provisional: 348, all +changes introduced in Python 3.6 have also been backported to Python +3.5.x. + +The *note typing: 119. module has a much improved support for generic +type aliases. For example ‘Dict[str, Tuple[S, T]]’ is now a valid type +annotation. (Contributed by Guido van Rossum in Github #195(1).) + +The *note typing.ContextManager: 535. class has been added for +representing *note contextlib.AbstractContextManager: 534. (Contributed +by Brett Cannon in bpo-25609(2).) + +The *note typing.Collection: 5a2. class has been added for representing +*note collections.abc.Collection: 52d. (Contributed by Ivan Levkivskyi +in bpo-27598(3).) + +The *note typing.ClassVar: 5a3. type construct has been added to mark +class variables. As introduced in PEP 526(4), a variable annotation +wrapped in ClassVar indicates that a given attribute is intended to be +used as a class variable and should not be set on instances of that +class. (Contributed by Ivan Levkivskyi in Github #280(5).) + +A new *note TYPE_CHECKING: 5a4. constant that is assumed to be ‘True’ by +the static type checkers, but is ‘False’ at runtime. (Contributed by +Guido van Rossum in Github #230(6).) + +A new *note NewType(): 5a5. helper function has been added to create +lightweight distinct types for annotations: + + from typing import NewType + + UserId = NewType('UserId', int) + some_id = UserId(524313) + +The static type checker will treat the new type as if it were a subclass +of the original type. (Contributed by Ivan Levkivskyi in Github +#189(7).) + + ---------- Footnotes ---------- + + (1) https://github.com/python/typing/pull/195 + + (2) https://bugs.python.org/issue25609 + + (3) https://bugs.python.org/issue27598 + + (4) https://www.python.org/dev/peps/pep-0526 + + (5) https://github.com/python/typing/pull/280 + + (6) https://github.com/python/typing/issues/230 + + (7) https://github.com/python/typing/issues/189 + + +File: python.info, Node: unicodedata<3>, Next: unittest mock<2>, Prev: typing<2>, Up: Improved Modules<3> + +1.3.5.53 unicodedata +.................... + +The *note unicodedata: 11a. module now uses data from Unicode 9.0.0(1). +(Contributed by Benjamin Peterson.) + + ---------- Footnotes ---------- + + (1) http://unicode.org/versions/Unicode9.0.0/ + + +File: python.info, Node: unittest mock<2>, Next: urllib request, Prev: unicodedata<3>, Up: Improved Modules<3> + +1.3.5.54 unittest.mock +...................... + +The *note Mock: 249. class has the following improvements: + + * Two new methods, *note Mock.assert_called(): 5a8. and *note + Mock.assert_called_once(): 5a9. to check if the mock object was + called. (Contributed by Amit Saha in bpo-26323(1).) + + * The *note Mock.reset_mock(): 5aa. method now has two optional + keyword only arguments: `return_value' and `side_effect'. + (Contributed by Kushal Das in bpo-21271(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26323 + + (2) https://bugs.python.org/issue21271 + + +File: python.info, Node: urllib request, Next: urllib robotparser, Prev: unittest mock<2>, Up: Improved Modules<3> + +1.3.5.55 urllib.request +....................... + +If a HTTP request has a file or iterable body (other than a bytes +object) but no ‘Content-Length’ header, rather than throwing an error, +‘AbstractHTTPHandler’ now falls back to use chunked transfer encoding. +(Contributed by Demian Brecht and Rolf Krahl in bpo-12319(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12319 + + +File: python.info, Node: urllib robotparser, Next: venv<2>, Prev: urllib request, Up: Improved Modules<3> + +1.3.5.56 urllib.robotparser +........................... + +*note RobotFileParser: 5ad. now supports the ‘Crawl-delay’ and +‘Request-rate’ extensions. (Contributed by Nikolay Bogoychev in +bpo-16099(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16099 + + +File: python.info, Node: venv<2>, Next: warnings<2>, Prev: urllib robotparser, Up: Improved Modules<3> + +1.3.5.57 venv +............. + +*note venv: 125. accepts a new parameter ‘--prompt’. This parameter +provides an alternative prefix for the virtual environment. (Proposed +by Łukasz Balcerzak and ported to 3.6 by Stéphane Wirtel in +bpo-22829(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22829 + + +File: python.info, Node: warnings<2>, Next: winreg, Prev: venv<2>, Up: Improved Modules<3> + +1.3.5.58 warnings +................. + +A new optional `source' parameter has been added to the *note +warnings.warn_explicit(): 5b0. function: the destroyed object which +emitted a *note ResourceWarning: 4d0. A `source' attribute has also +been added to ‘warnings.WarningMessage’ (contributed by Victor Stinner +in bpo-26568(1) and bpo-26567(2)). + +When a *note ResourceWarning: 4d0. warning is logged, the *note +tracemalloc: 114. module is now used to try to retrieve the traceback +where the destroyed object was allocated. + +Example with the script ‘example.py’: + + import warnings + + def func(): + return open(__file__) + + f = func() + f = None + +Output of the command ‘python3.6 -Wd -X tracemalloc=5 example.py’: + + example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'> + f = None + Object allocated at (most recent call first): + File "example.py", lineno 4 + return open(__file__) + File "example.py", lineno 6 + f = func() + +The “Object allocated at” traceback is new and is only displayed if +*note tracemalloc: 114. is tracing Python memory allocations and if the +*note warnings: 126. module was already imported. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26568 + + (2) https://bugs.python.org/issue26567 + + +File: python.info, Node: winreg, Next: winsound, Prev: warnings<2>, Up: Improved Modules<3> + +1.3.5.59 winreg +............... + +Added the 64-bit integer type *note REG_QWORD: 5b2. (Contributed by +Clement Rouault in bpo-23026(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23026 + + +File: python.info, Node: winsound, Next: xmlrpc client, Prev: winreg, Up: Improved Modules<3> + +1.3.5.60 winsound +................. + +Allowed keyword arguments to be passed to *note Beep: 5b4, *note +MessageBeep: 5b5, and *note PlaySound: 5b6. (bpo-27982(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27982 + + +File: python.info, Node: xmlrpc client, Next: zipfile<2>, Prev: winsound, Up: Improved Modules<3> + +1.3.5.61 xmlrpc.client +...................... + +The *note xmlrpc.client: 13f. module now supports unmarshalling +additional data types used by the Apache XML-RPC implementation for +numerics and ‘None’. (Contributed by Serhiy Storchaka in bpo-26885(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26885 + + +File: python.info, Node: zipfile<2>, Next: zlib, Prev: xmlrpc client, Up: Improved Modules<3> + +1.3.5.62 zipfile +................ + +A new *note ZipInfo.from_file(): 5b9. class method allows making a *note +ZipInfo: 5ba. instance from a filesystem file. A new *note +ZipInfo.is_dir(): 5bb. method can be used to check if the *note ZipInfo: +5ba. instance represents a directory. (Contributed by Thomas Kluyver in +bpo-26039(1).) + +The *note ZipFile.open(): 5bc. method can now be used to write data into +a ZIP file, as well as for extracting data. (Contributed by Thomas +Kluyver in bpo-26039(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26039 + + (2) https://bugs.python.org/issue26039 + + +File: python.info, Node: zlib, Prev: zipfile<2>, Up: Improved Modules<3> + +1.3.5.63 zlib +............. + +The *note compress(): 5be. and *note decompress(): 5bf. functions now +accept keyword arguments. (Contributed by Aviv Palivoda in bpo-26243(1) +and Xiang Zhang in bpo-16764(2) respectively.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26243 + + (2) https://bugs.python.org/issue16764 + + +File: python.info, Node: Optimizations<3>, Next: Build and C API Changes<2>, Prev: Improved Modules<3>, Up: What’s New In Python 3 6 + +1.3.6 Optimizations +------------------- + + * The Python interpreter now uses a 16-bit wordcode instead of + bytecode which made a number of opcode optimizations possible. + (Contributed by Demur Rumed with input and reviews from Serhiy + Storchaka and Victor Stinner in bpo-26647(1) and bpo-28050(2).) + + * The *note asyncio.Future: 443. class now has an optimized C + implementation. (Contributed by Yury Selivanov and INADA Naoki in + bpo-26081(3).) + + * The *note asyncio.Task: 1ad. class now has an optimized C + implementation. (Contributed by Yury Selivanov in bpo-28544(4).) + + * Various implementation improvements in the *note typing: 119. + module (such as caching of generic types) allow up to 30 times + performance improvements and reduced memory footprint. + + * The ASCII decoder is now up to 60 times as fast for error handlers + ‘surrogateescape’, ‘ignore’ and ‘replace’ (Contributed by Victor + Stinner in bpo-24870(5)). + + * The ASCII and the Latin1 encoders are now up to 3 times as fast for + the error handler ‘surrogateescape’ (Contributed by Victor Stinner + in bpo-25227(6)). + + * The UTF-8 encoder is now up to 75 times as fast for error handlers + ‘ignore’, ‘replace’, ‘surrogateescape’, ‘surrogatepass’ + (Contributed by Victor Stinner in bpo-25267(7)). + + * The UTF-8 decoder is now up to 15 times as fast for error handlers + ‘ignore’, ‘replace’ and ‘surrogateescape’ (Contributed by Victor + Stinner in bpo-25301(8)). + + * ‘bytes % args’ is now up to 2 times faster. (Contributed by Victor + Stinner in bpo-25349(9)). + + * ‘bytearray % args’ is now between 2.5 and 5 times faster. + (Contributed by Victor Stinner in bpo-25399(10)). + + * Optimize *note bytes.fromhex(): 32e. and *note bytearray.fromhex(): + 32f.: they are now between 2x and 3.5x faster. (Contributed by + Victor Stinner in bpo-25401(11)). + + * Optimize ‘bytes.replace(b'', b'.')’ and ‘bytearray.replace(b'', + b'.')’: up to 80% faster. (Contributed by Josh Snider in + bpo-26574(12)). + + * Allocator functions of the *note PyMem_Malloc(): 505. domain (*note + PYMEM_DOMAIN_MEM: 509.) now use the *note pymalloc memory + allocator: 5c1. instead of ‘malloc()’ function of the C library. + The pymalloc allocator is optimized for objects smaller or equal to + 512 bytes with a short lifetime, and use ‘malloc()’ for larger + memory blocks. (Contributed by Victor Stinner in bpo-26249(13)). + + * *note pickle.load(): 5c2. and *note pickle.loads(): 5c3. are now up + to 10% faster when deserializing many small objects (Contributed by + Victor Stinner in bpo-27056(14)). + + * Passing *note keyword arguments: 5c4. to a function has an overhead + in comparison with passing *note positional arguments: 5c5. Now in + extension functions implemented with using Argument Clinic this + overhead is significantly decreased. (Contributed by Serhiy + Storchaka in bpo-27574(15)). + + * Optimized *note glob(): 5c6. and *note iglob(): 5c7. functions in + the *note glob: 8a. module; they are now about 3–6 times faster. + (Contributed by Serhiy Storchaka in bpo-25596(16)). + + * Optimized globbing in *note pathlib: c8. by using *note + os.scandir(): 25f.; it is now about 1.5–4 times faster. + (Contributed by Serhiy Storchaka in bpo-26032(17)). + + * *note xml.etree.ElementTree: 137. parsing, iteration and deepcopy + performance has been significantly improved. (Contributed by + Serhiy Storchaka in bpo-25638(18), bpo-25873(19), and + bpo-25869(20).) + + * Creation of *note fractions.Fraction: 185. instances from floats + and decimals is now 2 to 3 times faster. (Contributed by Serhiy + Storchaka in bpo-25971(21).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26647 + + (2) https://bugs.python.org/issue28050 + + (3) https://bugs.python.org/issue26081 + + (4) https://bugs.python.org/issue28544 + + (5) https://bugs.python.org/issue24870 + + (6) https://bugs.python.org/issue25227 + + (7) https://bugs.python.org/issue25267 + + (8) https://bugs.python.org/issue25301 + + (9) https://bugs.python.org/issue25349 + + (10) https://bugs.python.org/issue25399 + + (11) https://bugs.python.org/issue25401 + + (12) https://bugs.python.org/issue26574 + + (13) https://bugs.python.org/issue26249 + + (14) https://bugs.python.org/issue27056 + + (15) https://bugs.python.org/issue27574 + + (16) https://bugs.python.org/issue25596 + + (17) https://bugs.python.org/issue26032 + + (18) https://bugs.python.org/issue25638 + + (19) https://bugs.python.org/issue25873 + + (20) https://bugs.python.org/issue25869 + + (21) https://bugs.python.org/issue25971 + + +File: python.info, Node: Build and C API Changes<2>, Next: Other Improvements, Prev: Optimizations<3>, Up: What’s New In Python 3 6 + +1.3.7 Build and C API Changes +----------------------------- + + * Python now requires some C99 support in the toolchain to build. + Most notably, Python now uses standard integer types and macros in + place of custom macros like ‘PY_LONG_LONG’. For more information, + see PEP 7(1) and bpo-17884(2). + + * Cross-compiling CPython with the Android NDK and the Android API + level set to 21 (Android 5.0 Lollipop) or greater runs + successfully. While Android is not yet a supported platform, the + Python test suite runs on the Android emulator with only about 16 + tests failures. See the Android meta-issue bpo-26865(3). + + * The ‘--enable-optimizations’ configure flag has been added. + Turning it on will activate expensive optimizations like PGO. + (Original patch by Alecsandru Patrascu of Intel in bpo-26359(4).) + + * The *note GIL: 506. must now be held when allocator functions of + *note PYMEM_DOMAIN_OBJ: 507. (ex: *note PyObject_Malloc(): 508.) + and *note PYMEM_DOMAIN_MEM: 509. (ex: *note PyMem_Malloc(): 505.) + domains are called. + + * New *note Py_FinalizeEx(): 5c9. API which indicates if flushing + buffered data failed. (Contributed by Martin Panter in + bpo-5319(5).) + + * *note PyArg_ParseTupleAndKeywords(): 5ca. now supports *note + positional-only parameters: 2a7. Positional-only parameters are + defined by empty names. (Contributed by Serhiy Storchaka in + bpo-26282(6)). + + * ‘PyTraceback_Print’ method now abbreviates long sequences of + repeated lines as ‘"[Previous line repeated {count} more times]"’. + (Contributed by Emanuel Barry in bpo-26823(7).) + + * The new *note PyErr_SetImportErrorSubclass(): 5cb. function allows + for specifying a subclass of *note ImportError: 334. to raise. + (Contributed by Eric Snow in bpo-15767(8).) + + * The new *note PyErr_ResourceWarning(): 5cc. function can be used to + generate a *note ResourceWarning: 4d0. providing the source of the + resource allocation. (Contributed by Victor Stinner in + bpo-26567(9).) + + * The new *note PyOS_FSPath(): 5cd. function returns the file system + representation of a *note path-like object: 36a. (Contributed by + Brett Cannon in bpo-27186(10).) + + * The *note PyUnicode_FSConverter(): 5ce. and *note + PyUnicode_FSDecoder(): 5cf. functions will now accept *note + path-like objects: 36a. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0007 + + (2) https://bugs.python.org/issue17884 + + (3) https://bugs.python.org/issue26865 + + (4) https://bugs.python.org/issue26359 + + (5) https://bugs.python.org/issue5319 + + (6) https://bugs.python.org/issue26282 + + (7) https://bugs.python.org/issue26823 + + (8) https://bugs.python.org/issue15767 + + (9) https://bugs.python.org/issue26567 + + (10) https://bugs.python.org/issue27186 + + +File: python.info, Node: Other Improvements, Next: Deprecated<2>, Prev: Build and C API Changes<2>, Up: What’s New In Python 3 6 + +1.3.8 Other Improvements +------------------------ + + * When *note –version: 5d1. (short form: *note -V: 5d2.) is supplied + twice, Python prints *note sys.version: 5d3. for detailed + information. + + $ ./python -VV + Python 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04) + [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] + + +File: python.info, Node: Deprecated<2>, Next: Removed, Prev: Other Improvements, Up: What’s New In Python 3 6 + +1.3.9 Deprecated +---------------- + +* Menu: + +* New Keywords:: +* Deprecated Python behavior:: +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods<2>. +* Deprecated functions and types of the C API: Deprecated functions and types of the C API<2>. +* Deprecated Build Options:: + + +File: python.info, Node: New Keywords, Next: Deprecated Python behavior, Up: Deprecated<2> + +1.3.9.1 New Keywords +.................... + +‘async’ and ‘await’ are not recommended to be used as variable, class, +function or module names. Introduced by PEP 492(1) in Python 3.5, they +will become proper keywords in Python 3.7. Starting in Python 3.6, the +use of ‘async’ or ‘await’ as names will generate a *note +DeprecationWarning: 278. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0492 + + +File: python.info, Node: Deprecated Python behavior, Next: Deprecated Python modules functions and methods<2>, Prev: New Keywords, Up: Deprecated<2> + +1.3.9.2 Deprecated Python behavior +.................................. + +Raising the *note StopIteration: 486. exception inside a generator will +now generate a *note DeprecationWarning: 278, and will trigger a *note +RuntimeError: 2ba. in Python 3.7. See *note PEP 479; Change +StopIteration handling inside generators: 5d7. for details. + +The *note __aiter__(): 487. method is now expected to return an +asynchronous iterator directly instead of returning an awaitable as +previously. Doing the former will trigger a *note DeprecationWarning: +278. Backward compatibility will be removed in Python 3.7. +(Contributed by Yury Selivanov in bpo-27243(1).) + +A backslash-character pair that is not a valid escape sequence now +generates a *note DeprecationWarning: 278. Although this will +eventually become a *note SyntaxError: 458, that will not be for several +Python releases. (Contributed by Emanuel Barry in bpo-27364(2).) + +When performing a relative import, falling back on ‘__name__’ and +‘__path__’ from the calling module when ‘__spec__’ or ‘__package__’ are +not defined now raises an *note ImportWarning: 5d8. (Contributed by +Rose Ames in bpo-25791(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27243 + + (2) https://bugs.python.org/issue27364 + + (3) https://bugs.python.org/issue25791 + + +File: python.info, Node: Deprecated Python modules functions and methods<2>, Next: Deprecated functions and types of the C API<2>, Prev: Deprecated Python behavior, Up: Deprecated<2> + +1.3.9.3 Deprecated Python modules, functions and methods +........................................................ + +* Menu: + +* asynchat:: +* asyncore:: +* dbm: dbm<3>. +* distutils: distutils<3>. +* grp:: +* importlib: importlib<4>. +* os: os<4>. +* re: re<3>. +* ssl: ssl<5>. +* tkinter: tkinter<4>. +* venv: venv<3>. + + +File: python.info, Node: asynchat, Next: asyncore, Up: Deprecated Python modules functions and methods<2> + +1.3.9.4 asynchat +................ + +The *note asynchat: 9. has been deprecated in favor of *note asyncio: a. +(Contributed by Mariatta in bpo-25002(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25002 + + +File: python.info, Node: asyncore, Next: dbm<3>, Prev: asynchat, Up: Deprecated Python modules functions and methods<2> + +1.3.9.5 asyncore +................ + +The *note asyncore: b. has been deprecated in favor of *note asyncio: a. +(Contributed by Mariatta in bpo-25002(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25002 + + +File: python.info, Node: dbm<3>, Next: distutils<3>, Prev: asyncore, Up: Deprecated Python modules functions and methods<2> + +1.3.9.6 dbm +........... + +Unlike other *note dbm: 32. implementations, the *note dbm.dumb: 33. +module creates databases with the ‘'rw'’ mode and allows modifying the +database opened with the ‘'r'’ mode. This behavior is now deprecated +and will be removed in 3.8. (Contributed by Serhiy Storchaka in +bpo-21708(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21708 + + +File: python.info, Node: distutils<3>, Next: grp, Prev: dbm<3>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.7 distutils +................. + +The undocumented ‘extra_path’ argument to the ‘Distribution’ constructor +is now considered deprecated and will raise a warning if set. Support +for this parameter will be removed in a future Python release. See +bpo-27919(1) for details. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue27919 + + +File: python.info, Node: grp, Next: importlib<4>, Prev: distutils<3>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.8 grp +........... + +The support of non-integer arguments in *note getgrgid(): 5df. has been +deprecated. (Contributed by Serhiy Storchaka in bpo-26129(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26129 + + +File: python.info, Node: importlib<4>, Next: os<4>, Prev: grp, Up: Deprecated Python modules functions and methods<2> + +1.3.9.9 importlib +................. + +The *note importlib.machinery.SourceFileLoader.load_module(): 5e1. and +*note importlib.machinery.SourcelessFileLoader.load_module(): 5e2. +methods are now deprecated. They were the only remaining +implementations of *note importlib.abc.Loader.load_module(): 5e3. in +*note importlib: 9b. that had not been deprecated in previous versions +of Python in favour of *note importlib.abc.Loader.exec_module(): 5e4. + +The *note importlib.machinery.WindowsRegistryFinder: 5e5. class is now +deprecated. As of 3.6.0, it is still added to *note sys.meta_path: 5e6. +by default (on Windows), but this may change in future releases. + + +File: python.info, Node: os<4>, Next: re<3>, Prev: importlib<4>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.10 os +........... + +Undocumented support of general *note bytes-like objects: 5e8. as paths +in *note os: c4. functions, *note compile(): 1b4. and similar functions +is now deprecated. (Contributed by Serhiy Storchaka in bpo-25791(1) and +bpo-26754(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25791 + + (2) https://bugs.python.org/issue26754 + + +File: python.info, Node: re<3>, Next: ssl<5>, Prev: os<4>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.11 re +........... + +Support for inline flags ‘(?letters)’ in the middle of the regular +expression has been deprecated and will be removed in a future Python +version. Flags at the start of a regular expression are still allowed. +(Contributed by Serhiy Storchaka in bpo-22493(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22493 + + +File: python.info, Node: ssl<5>, Next: tkinter<4>, Prev: re<3>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.12 ssl +............ + +OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported. +In the future the *note ssl: f3. module will require at least OpenSSL +1.0.2 or 1.1.0. + +SSL-related arguments like ‘certfile’, ‘keyfile’ and ‘check_hostname’ in +*note ftplib: 84, *note http.client: 94, *note imaplib: 98, *note +poplib: d0, and *note smtplib: ed. have been deprecated in favor of +‘context’. (Contributed by Christian Heimes in bpo-28022(1).) + +A couple of protocols and functions of the *note ssl: f3. module are now +deprecated. Some features will no longer be available in future +versions of OpenSSL. Other features are deprecated in favor of a +different API. (Contributed by Christian Heimes in bpo-28022(2) and +bpo-26470(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue28022 + + (2) https://bugs.python.org/issue28022 + + (3) https://bugs.python.org/issue26470 + + +File: python.info, Node: tkinter<4>, Next: venv<3>, Prev: ssl<5>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.13 tkinter +................ + +The *note tkinter.tix: 10e. module is now deprecated. *note tkinter: +10c. users should use *note tkinter.ttk: 10f. instead. + + +File: python.info, Node: venv<3>, Prev: tkinter<4>, Up: Deprecated Python modules functions and methods<2> + +1.3.9.14 venv +............. + +The ‘pyvenv’ script has been deprecated in favour of ‘python3 -m venv’. +This prevents confusion as to what Python interpreter ‘pyvenv’ is +connected to and thus what Python interpreter will be used by the +virtual environment. (Contributed by Brett Cannon in bpo-25154(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue25154 + + +File: python.info, Node: Deprecated functions and types of the C API<2>, Next: Deprecated Build Options, Prev: Deprecated Python modules functions and methods<2>, Up: Deprecated<2> + +1.3.9.15 Deprecated functions and types of the C API +.................................................... + +Undocumented functions ‘PyUnicode_AsEncodedObject()’, +‘PyUnicode_AsDecodedObject()’, ‘PyUnicode_AsEncodedUnicode()’ and +‘PyUnicode_AsDecodedUnicode()’ are deprecated now. Use the *note +generic codec based API: 5ee. instead. + + +File: python.info, Node: Deprecated Build Options, Prev: Deprecated functions and types of the C API<2>, Up: Deprecated<2> + +1.3.9.16 Deprecated Build Options +................................. + +The ‘--with-system-ffi’ configure flag is now on by default on non-macOS +UNIX platforms. It may be disabled by using ‘--without-system-ffi’, but +using the flag is deprecated and will not be accepted in Python 3.7. +macOS is unaffected by this change. Note that many OS distributors +already use the ‘--with-system-ffi’ flag when building their system +Python. + + +File: python.info, Node: Removed, Next: Porting to Python 3 6, Prev: Deprecated<2>, Up: What’s New In Python 3 6 + +1.3.10 Removed +-------------- + +* Menu: + +* API and Feature Removals: API and Feature Removals<3>. + + +File: python.info, Node: API and Feature Removals<3>, Up: Removed + +1.3.10.1 API and Feature Removals +................................. + + * Unknown escapes consisting of ‘'\'’ and an ASCII letter in regular + expressions will now cause an error. In replacement templates for + *note re.sub(): 47b. they are still allowed, but deprecated. The + *note re.LOCALE: 3c9. flag can now only be used with binary + patterns. + + * ‘inspect.getmoduleinfo()’ was removed (was deprecated since CPython + 3.3). *note inspect.getmodulename(): 5f2. should be used for + obtaining the module name for a given path. (Contributed by Yury + Selivanov in bpo-13248(1).) + + * ‘traceback.Ignore’ class and ‘traceback.usage’, + ‘traceback.modname’, ‘traceback.fullmodname’, + ‘traceback.find_lines_from_code’, ‘traceback.find_lines’, + ‘traceback.find_strings’, ‘traceback.find_executable_lines’ methods + were removed from the *note traceback: 113. module. They were + undocumented methods deprecated since Python 3.2 and equivalent + functionality is available from private methods. + + * The ‘tk_menuBar()’ and ‘tk_bindForTraversal()’ dummy methods in + *note tkinter: 10c. widget classes were removed (corresponding Tk + commands were obsolete since Tk 4.0). + + * The *note open(): 5bc. method of the *note zipfile.ZipFile: 41f. + class no longer supports the ‘'U'’ mode (was deprecated since + Python 3.4). Use *note io.TextIOWrapper: 5f3. for reading + compressed text files in *note universal newlines: 5f4. mode. + + * The undocumented ‘IN’, ‘CDROM’, ‘DLFCN’, ‘TYPES’, ‘CDIO’, and + ‘STROPTS’ modules have been removed. They had been available in + the platform specific ‘Lib/plat-*/’ directories, but were + chronically out of date, inconsistently available across platforms, + and unmaintained. The script that created these modules is still + available in the source distribution at Tools/scripts/h2py.py(2). + + * The deprecated ‘asynchat.fifo’ class has been removed. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13248 + + (2) https://github.com/python/cpython/tree/3.8/Tools/scripts/h2py.py + + +File: python.info, Node: Porting to Python 3 6, Next: Notable changes in Python 3 6 2, Prev: Removed, Up: What’s New In Python 3 6 + +1.3.11 Porting to Python 3.6 +---------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code. + +* Menu: + +* Changes in ‘python’ Command Behavior:: +* Changes in the Python API: Changes in the Python API<3>. +* Changes in the C API: Changes in the C API<3>. +* CPython bytecode changes: CPython bytecode changes<3>. + + +File: python.info, Node: Changes in ‘python’ Command Behavior, Next: Changes in the Python API<3>, Up: Porting to Python 3 6 + +1.3.11.1 Changes in ‘python’ Command Behavior +............................................. + + * The output of a special Python build with defined ‘COUNT_ALLOCS’, + ‘SHOW_ALLOC_COUNT’ or ‘SHOW_TRACK_COUNT’ macros is now off by + default. It can be re-enabled using the ‘-X showalloccount’ + option. It now outputs to ‘stderr’ instead of ‘stdout’. + (Contributed by Serhiy Storchaka in bpo-23034(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23034 + + +File: python.info, Node: Changes in the Python API<3>, Next: Changes in the C API<3>, Prev: Changes in ‘python’ Command Behavior, Up: Porting to Python 3 6 + +1.3.11.2 Changes in the Python API +.................................. + + * *note open(): 4f0. will no longer allow combining the ‘'U'’ mode + flag with ‘'+'’. (Contributed by Jeff Balogh and John O’Connor in + bpo-2091(1).) + + * *note sqlite3: f2. no longer implicitly commits an open transaction + before DDL statements. + + * On Linux, *note os.urandom(): 4d1. now blocks until the system + urandom entropy pool is initialized to increase the security. + + * When *note importlib.abc.Loader.exec_module(): 5e4. is defined, + *note importlib.abc.Loader.create_module(): 554. must also be + defined. + + * *note PyErr_SetImportError(): 5f8. now sets *note TypeError: 192. + when its `msg' argument is not set. Previously only ‘NULL’ was + returned. + + * The format of the ‘co_lnotab’ attribute of code objects changed to + support a negative line number delta. By default, Python does not + emit bytecode with a negative line number delta. Functions using + ‘frame.f_lineno’, ‘PyFrame_GetLineNumber()’ or ‘PyCode_Addr2Line()’ + are not affected. Functions directly decoding ‘co_lnotab’ should + be updated to use a signed 8-bit integer type for the line number + delta, but this is only required to support applications using a + negative line number delta. See ‘Objects/lnotab_notes.txt’ for the + ‘co_lnotab’ format and how to decode it, and see the PEP 511(2) for + the rationale. + + * The functions in the *note compileall: 21. module now return + booleans instead of ‘1’ or ‘0’ to represent success or failure, + respectively. Thanks to booleans being a subclass of integers, + this should only be an issue if you were doing identity checks for + ‘1’ or ‘0’. See bpo-25768(3). + + * Reading the ‘port’ attribute of *note urllib.parse.urlsplit(): 5f9. + and *note urlparse(): 5fa. results now raises *note ValueError: + 1fb. for out-of-range values, rather than returning *note None: + 157. See bpo-20059(4). + + * The *note imp: 9a. module now raises a *note DeprecationWarning: + 278. instead of *note PendingDeprecationWarning: 279. + + * The following modules have had missing APIs added to their + ‘__all__’ attributes to match the documented APIs: *note calendar: + 15, *note cgi: 16, *note csv: 2a, *note ElementTree: 137, *note + enum: 7b, *note fileinput: 80, *note ftplib: 84, *note logging: aa, + *note mailbox: ae, *note mimetypes: b2, *note optparse: c3, *note + plistlib: cf, *note smtpd: ec, *note subprocess: f9, *note tarfile: + 101, *note threading: 109. and *note wave: 127. This means they + will export new symbols when ‘import *’ is used. (Contributed by + Joel Taddei and Jacek Kołodziej in bpo-23883(5).) + + * When performing a relative import, if ‘__package__’ does not + compare equal to ‘__spec__.parent’ then *note ImportWarning: 5d8. + is raised. (Contributed by Brett Cannon in bpo-25791(6).) + + * When a relative import is performed and no parent package is known, + then *note ImportError: 334. will be raised. Previously, *note + SystemError: 5fb. could be raised. (Contributed by Brett Cannon in + bpo-18018(7).) + + * Servers based on the *note socketserver: f0. module, including + those defined in *note http.server: 97, *note xmlrpc.server: 140. + and *note wsgiref.simple_server: 12f, now only catch exceptions + derived from *note Exception: 1a9. Therefore if a request handler + raises an exception like *note SystemExit: 5fc. or *note + KeyboardInterrupt: 197, *note handle_error(): 5fd. is no longer + called, and the exception will stop a single-threaded server. + (Contributed by Martin Panter in bpo-23430(8).) + + * *note spwd.getspnam(): 5fe. now raises a *note PermissionError: + 5ff. instead of *note KeyError: 2c7. if the user doesn’t have + privileges. + + * The *note socket.socket.close(): 600. method now raises an + exception if an error (e.g. ‘EBADF’) was reported by the + underlying system call. (Contributed by Martin Panter in + bpo-26685(9).) + + * The `decode_data' argument for the *note smtpd.SMTPChannel: 601. + and *note smtpd.SMTPServer: 602. constructors is now ‘False’ by + default. This means that the argument passed to *note + process_message(): 603. is now a bytes object by default, and + ‘process_message()’ will be passed keyword arguments. Code that + has already been updated in accordance with the deprecation warning + generated by 3.5 will not be affected. + + * All optional arguments of the *note dump(): 604, *note dumps(): + 605, *note load(): 55f. and *note loads(): 560. functions and *note + JSONEncoder: 606. and *note JSONDecoder: 607. class constructors in + the *note json: a4. module are now *note keyword-only: 2ac. + (Contributed by Serhiy Storchaka in bpo-18726(10).) + + * Subclasses of *note type: 608. which don’t override ‘type.__new__’ + may no longer use the one-argument form to get the type of an + object. + + * As part of PEP 487(11), the handling of keyword arguments passed to + *note type: 608. (other than the metaclass hint, ‘metaclass’) is + now consistently delegated to *note object.__init_subclass__(): + 4e3. This means that ‘type.__new__()’ and ‘type.__init__()’ both + now accept arbitrary keyword arguments, but *note + object.__init_subclass__(): 4e3. (which is called from + ‘type.__new__()’) will reject them by default. Custom metaclasses + accepting additional keyword arguments will need to adjust their + calls to ‘type.__new__()’ (whether direct or via *note super: 4e2.) + accordingly. + + * In ‘distutils.command.sdist.sdist’, the ‘default_format’ attribute + has been removed and is no longer honored. Instead, the gzipped + tarfile format is the default on all platforms and no + platform-specific selection is made. In environments where + distributions are built on Windows and zip distributions are + required, configure the project with a ‘setup.cfg’ file containing + the following: + + [sdist] + formats=zip + + This behavior has also been backported to earlier Python versions + by Setuptools 26.0.0. + + * In the *note urllib.request: 120. module and the *note + http.client.HTTPConnection.request(): 54f. method, if no + Content-Length header field has been specified and the request body + is a file object, it is now sent with HTTP 1.1 chunked encoding. + If a file object has to be sent to a HTTP 1.0 server, the + Content-Length value now has to be specified by the caller. + (Contributed by Demian Brecht and Rolf Krahl with tweaks from + Martin Panter in bpo-12319(12).) + + * The *note DictReader: 1bd. now returns rows of type *note + OrderedDict: 1b9. (Contributed by Steve Holden in bpo-27842(13).) + + * The *note crypt.METHOD_CRYPT: 609. will no longer be added to + ‘crypt.methods’ if unsupported by the platform. (Contributed by + Victor Stinner in bpo-25287(14).) + + * The `verbose' and `rename' arguments for *note namedtuple(): 1b7. + are now keyword-only. (Contributed by Raymond Hettinger in + bpo-25628(15).) + + * On Linux, *note ctypes.util.find_library(): 60a. now looks in + ‘LD_LIBRARY_PATH’ for shared libraries. (Contributed by Vinay + Sajip in bpo-9998(16).) + + * The *note imaplib.IMAP4: 60b. class now handles flags containing + the ‘']'’ character in messages sent from the server to improve + real-world compatibility. (Contributed by Lita Cho in + bpo-21815(17).) + + * The ‘mmap.write()’ function now returns the number of bytes written + like other write methods. (Contributed by Jakub Stasiak in + bpo-26335(18).) + + * The *note pkgutil.iter_modules(): 60c. and *note + pkgutil.walk_packages(): 48b. functions now return *note + ModuleInfo: 60d. named tuples. (Contributed by Ramchandra Apte in + bpo-17211(19).) + + * *note re.sub(): 47b. now raises an error for invalid numerical + group references in replacement templates even if the pattern is + not found in the string. The error message for invalid group + references now includes the group index and the position of the + reference. (Contributed by SilentGhost, Serhiy Storchaka in + bpo-25953(20).) + + * *note zipfile.ZipFile: 41f. will now raise *note + NotImplementedError: 60e. for unrecognized compression values. + Previously a plain *note RuntimeError: 2ba. was raised. + Additionally, calling *note ZipFile: 41f. methods on a closed + ZipFile or calling the *note write(): 60f. method on a ZipFile + created with mode ‘'r'’ will raise a *note ValueError: 1fb. + Previously, a *note RuntimeError: 2ba. was raised in those + scenarios. + + * when custom metaclasses are combined with zero-argument *note + super(): 4e2. or direct references from methods to the implicit + ‘__class__’ closure variable, the implicit ‘__classcell__’ + namespace entry must now be passed up to ‘type.__new__’ for + initialisation. Failing to do so will result in a *note + DeprecationWarning: 278. in Python 3.6 and a *note RuntimeError: + 2ba. in Python 3.8. + + * With the introduction of *note ModuleNotFoundError: 39a, import + system consumers may start expecting import system replacements to + raise that more specific exception when appropriate, rather than + the less-specific *note ImportError: 334. To provide future + compatibility with such consumers, implementors of alternative + import systems that completely replace *note __import__(): 610. + will need to update their implementations to raise the new subclass + when a module can’t be found at all. Implementors of compliant + plugins to the default import system shouldn’t need to make any + changes, as the default import system will raise the new subclass + when appropriate. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2091 + + (2) https://www.python.org/dev/peps/pep-0511 + + (3) https://bugs.python.org/issue25768 + + (4) https://bugs.python.org/issue20059 + + (5) https://bugs.python.org/issue23883 + + (6) https://bugs.python.org/issue25791 + + (7) https://bugs.python.org/issue18018 + + (8) https://bugs.python.org/issue23430 + + (9) https://bugs.python.org/issue26685 + + (10) https://bugs.python.org/issue18726 + + (11) https://www.python.org/dev/peps/pep-0487 + + (12) https://bugs.python.org/issue12319 + + (13) https://bugs.python.org/issue27842 + + (14) https://bugs.python.org/issue25287 + + (15) https://bugs.python.org/issue25628 + + (16) https://bugs.python.org/issue9998 + + (17) https://bugs.python.org/issue21815 + + (18) https://bugs.python.org/issue26335 + + (19) https://bugs.python.org/issue17211 + + (20) https://bugs.python.org/issue25953 + + +File: python.info, Node: Changes in the C API<3>, Next: CPython bytecode changes<3>, Prev: Changes in the Python API<3>, Up: Porting to Python 3 6 + +1.3.11.3 Changes in the C API +............................. + + * The *note PyMem_Malloc(): 505. allocator family now uses the *note + pymalloc allocator: 5c1. rather than the system ‘malloc()’. + Applications calling *note PyMem_Malloc(): 505. without holding the + GIL can now crash. Set the *note PYTHONMALLOC: 503. environment + variable to ‘debug’ to validate the usage of memory allocators in + your application. See bpo-26249(1). + + * *note Py_Exit(): 612. (and the main interpreter) now override the + exit status with 120 if flushing buffered data failed. See + bpo-5319(2). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26249 + + (2) https://bugs.python.org/issue5319 + + +File: python.info, Node: CPython bytecode changes<3>, Prev: Changes in the C API<3>, Up: Porting to Python 3 6 + +1.3.11.4 CPython bytecode changes +................................. + +There have been several major changes to the *note bytecode: 614. in +Python 3.6. + + * The Python interpreter now uses a 16-bit wordcode instead of + bytecode. (Contributed by Demur Rumed with input and reviews from + Serhiy Storchaka and Victor Stinner in bpo-26647(1) and + bpo-28050(2).) + + * The new *note FORMAT_VALUE: 615. and *note BUILD_STRING: 616. + opcodes as part of the *note formatted string literal: 4c1. + implementation. (Contributed by Eric Smith in bpo-25483(3) and + Serhiy Storchaka in bpo-27078(4).) + + * The new *note BUILD_CONST_KEY_MAP: 617. opcode to optimize the + creation of dictionaries with constant keys. (Contributed by + Serhiy Storchaka in bpo-27140(5).) + + * The function call opcodes have been heavily reworked for better + performance and simpler implementation. The *note MAKE_FUNCTION: + 618, *note CALL_FUNCTION: 619, *note CALL_FUNCTION_KW: 61a. and + *note BUILD_MAP_UNPACK_WITH_CALL: 61b. opcodes have been modified, + the new *note CALL_FUNCTION_EX: 61c. and *note + BUILD_TUPLE_UNPACK_WITH_CALL: 61d. have been added, and + ‘CALL_FUNCTION_VAR’, ‘CALL_FUNCTION_VAR_KW’ and ‘MAKE_CLOSURE’ + opcodes have been removed. (Contributed by Demur Rumed in + bpo-27095(6), and Serhiy Storchaka in bpo-27213(7), bpo-28257(8).) + + * The new *note SETUP_ANNOTATIONS: 61e. and ‘STORE_ANNOTATION’ + opcodes have been added to support the new *note variable + annotation: 61f. syntax. (Contributed by Ivan Levkivskyi in + bpo-27985(9).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue26647 + + (2) https://bugs.python.org/issue28050 + + (3) https://bugs.python.org/issue25483 + + (4) https://bugs.python.org/issue27078 + + (5) https://bugs.python.org/issue27140 + + (6) https://bugs.python.org/issue27095 + + (7) https://bugs.python.org/issue27213 + + (8) https://bugs.python.org/issue28257 + + (9) https://bugs.python.org/issue27985 + + +File: python.info, Node: Notable changes in Python 3 6 2, Next: Notable changes in Python 3 6 4, Prev: Porting to Python 3 6, Up: What’s New In Python 3 6 + +1.3.12 Notable changes in Python 3.6.2 +-------------------------------------- + +* Menu: + +* New make regen-all build target:: +* Removal of make touch build target:: + + +File: python.info, Node: New make regen-all build target, Next: Removal of make touch build target, Up: Notable changes in Python 3 6 2 + +1.3.12.1 New ‘make regen-all’ build target +.......................................... + +To simplify cross-compilation, and to ensure that CPython can reliably +be compiled without requiring an existing version of Python to already +be available, the autotools-based build system no longer attempts to +implicitly recompile generated files based on file modification times. + +Instead, a new ‘make regen-all’ command has been added to force +regeneration of these files when desired (e.g. after an initial version +of Python has already been built based on the pregenerated versions). + +More selective regeneration targets are also defined - see +Makefile.pre.in(1) for details. + +(Contributed by Victor Stinner in bpo-23404(2).) + +New in version 3.6.2. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Makefile.pre.in + + (2) https://bugs.python.org/issue23404 + + +File: python.info, Node: Removal of make touch build target, Prev: New make regen-all build target, Up: Notable changes in Python 3 6 2 + +1.3.12.2 Removal of ‘make touch’ build target +............................................. + +The ‘make touch’ build target previously used to request implicit +regeneration of generated files by updating their modification times has +been removed. + +It has been replaced by the new ‘make regen-all’ target. + +(Contributed by Victor Stinner in bpo-23404(1).) + +Changed in version 3.6.2. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23404 + + +File: python.info, Node: Notable changes in Python 3 6 4, Next: Notable changes in Python 3 6 5, Prev: Notable changes in Python 3 6 2, Up: What’s New In Python 3 6 + +1.3.13 Notable changes in Python 3.6.4 +-------------------------------------- + +The ‘PyExc_RecursionErrorInst’ singleton that was part of the public API +has been removed as its members being never cleared may cause a segfault +during finalization of the interpreter. (Contributed by Xavier de Gaye +in bpo-22898(1) and bpo-30697(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22898 + + (2) https://bugs.python.org/issue30697 + + +File: python.info, Node: Notable changes in Python 3 6 5, Next: Notable changes in Python 3 6 7, Prev: Notable changes in Python 3 6 4, Up: What’s New In Python 3 6 + +1.3.14 Notable changes in Python 3.6.5 +-------------------------------------- + +The *note locale.localeconv(): 48a. function now sets temporarily the +‘LC_CTYPE’ locale to the ‘LC_NUMERIC’ locale in some cases. +(Contributed by Victor Stinner in bpo-31900(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31900 + + +File: python.info, Node: Notable changes in Python 3 6 7, Next: Notable changes in Python 3 6 10, Prev: Notable changes in Python 3 6 5, Up: What’s New In Python 3 6 + +1.3.15 Notable changes in Python 3.6.7 +-------------------------------------- + +In 3.6.7 the *note tokenize: 111. module now implicitly emits a +‘NEWLINE’ token when provided with input that does not have a trailing +new line. This behavior now matches what the C tokenizer does +internally. (Contributed by Ammar Askar in bpo-33899(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue33899 + + +File: python.info, Node: Notable changes in Python 3 6 10, Next: Notable changes in Python 3 6 13, Prev: Notable changes in Python 3 6 7, Up: What’s New In Python 3 6 + +1.3.16 Notable changes in Python 3.6.10 +--------------------------------------- + +Due to significant security concerns, the `reuse_address' parameter of +*note asyncio.loop.create_datagram_endpoint(): 2e7. is no longer +supported. This is because of the behavior of the socket option +‘SO_REUSEADDR’ in UDP. For more details, see the documentation for +‘loop.create_datagram_endpoint()’. (Contributed by Kyle Stanley, +Antoine Pitrou, and Yury Selivanov in bpo-37228(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue37228 + + +File: python.info, Node: Notable changes in Python 3 6 13, Prev: Notable changes in Python 3 6 10, Up: What’s New In Python 3 6 + +1.3.17 Notable changes in Python 3.6.13 +--------------------------------------- + +Earlier Python versions allowed using both ‘;’ and ‘&’ as query +parameter separators in *note urllib.parse.parse_qs(): 2eb. and *note +urllib.parse.parse_qsl(): 2ec. Due to security concerns, and to conform +with newer W3C recommendations, this has been changed to allow only a +single separator key, with ‘&’ as the default. This change also affects +*note cgi.parse(): 2ed. and *note cgi.parse_multipart(): 2ee. as they +use the affected functions internally. For more details, please see +their respective documentation. (Contributed by Adam Goldschmidt, +Senthil Kumaran and Ken Jin in bpo-42967(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue42967 + + +File: python.info, Node: What’s New In Python 3 5, Next: What’s New In Python 3 4, Prev: What’s New In Python 3 6, Up: What’s New in Python + +1.4 What’s New In Python 3.5 +============================ + + +Editors: Elvis Pranskevichus <>, Yury Selivanov +<> + +This article explains the new features in Python 3.5, compared to 3.4. +Python 3.5 was released on September 13, 2015.  See the changelog(1) +for a full list of changes. + +See also +........ + +PEP 478(2) - Python 3.5 Release Schedule + +* Menu: + +* Summary – Release highlights: Summary – Release highlights<3>. +* New Features: New Features<4>. +* Other Language Changes: Other Language Changes<4>. +* New Modules: New Modules<4>. +* Improved Modules: Improved Modules<4>. +* Other module-level changes:: +* Optimizations: Optimizations<4>. +* Build and C API Changes: Build and C API Changes<3>. +* Deprecated: Deprecated<3>. +* Removed: Removed<2>. +* Porting to Python 3.5: Porting to Python 3 5. +* Notable changes in Python 3.5.4: Notable changes in Python 3 5 4. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3.5/whatsnew/changelog.html + + (2) https://www.python.org/dev/peps/pep-0478 + + +File: python.info, Node: Summary – Release highlights<3>, Next: New Features<4>, Up: What’s New In Python 3 5 + +1.4.1 Summary – Release highlights +---------------------------------- + +New syntax features: + + * *note PEP 492: 62b, coroutines with async and await syntax. + + * *note PEP 465: 62c, a new matrix multiplication operator: ‘a @ b’. + + * *note PEP 448: 62d, additional unpacking generalizations. + +New library modules: + + * *note typing: 119.: *note PEP 484 – Type Hints: 62e. + + * *note zipapp: 141.: *note PEP 441 Improving Python ZIP Application + Support: 62f. + +New built-in features: + + * ‘bytes % args’, ‘bytearray % args’: *note PEP 461: 630. – Adding + ‘%’ formatting to bytes and bytearray. + + * New *note bytes.hex(): 631, *note bytearray.hex(): 632. and *note + memoryview.hex(): 633. methods. (Contributed by Arnon Yaari in + bpo-9951(1).) + + * *note memoryview: 25c. now supports tuple indexing (including + multi-dimensional). (Contributed by Antoine Pitrou in + bpo-23632(2).) + + * Generators have a new ‘gi_yieldfrom’ attribute, which returns the + object being iterated by ‘yield from’ expressions. (Contributed by + Benno Leslie and Yury Selivanov in bpo-24450(3).) + + * A new *note RecursionError: 634. exception is now raised when + maximum recursion depth is reached. (Contributed by Georg Brandl + in bpo-19235(4).) + +CPython implementation improvements: + + * When the ‘LC_TYPE’ locale is the POSIX locale (‘C’ locale), *note + sys.stdin: 30d. and *note sys.stdout: 30e. now use the + ‘surrogateescape’ error handler, instead of the ‘strict’ error + handler. (Contributed by Victor Stinner in bpo-19977(5).) + + * ‘.pyo’ files are no longer used and have been replaced by a more + flexible scheme that includes the optimization level explicitly in + ‘.pyc’ name. (See *note PEP 488 overview: 635.) + + * Builtin and extension modules are now initialized in a multi-phase + process, which is similar to how Python modules are loaded. (See + *note PEP 489 overview: 636.) + +Significant improvements in the standard library: + + * *note collections.OrderedDict: 1b9. is now *note implemented in C: + 637, which makes it 4 to 100 times faster. + + * The *note ssl: f3. module gained *note support for Memory BIO: 638, + which decouples SSL protocol handling from network IO. + + * The new *note os.scandir(): 25f. function provides a *note better + and significantly faster way: 639. of directory traversal. + + * *note functools.lru_cache(): 1c7. has been mostly *note + reimplemented in C: 63a, yielding much better performance. + + * The new *note subprocess.run(): 3ed. function provides a *note + streamlined way to run subprocesses: 63b. + + * The *note traceback: 113. module has been significantly *note + enhanced: 63c. for improved performance and developer convenience. + +Security improvements: + + * SSLv3 is now disabled throughout the standard library. It can + still be enabled by instantiating a *note ssl.SSLContext: 3e4. + manually. (See bpo-22638(6) for more details; this change was + backported to CPython 3.4 and 2.7.) + + * HTTP cookie parsing is now stricter, in order to protect against + potential injection attacks. (Contributed by Antoine Pitrou in + bpo-22796(7).) + +Windows improvements: + + * A new installer for Windows has replaced the old MSI. See *note + Using Python on Windows: 2d9. for more information. + + * Windows builds now use Microsoft Visual C++ 14.0, and extension + modules should use the same. + +Please read on for a comprehensive list of user-facing changes, +including many other smaller improvements, CPython optimizations, +deprecations, and potential porting issues. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9951 + + (2) https://bugs.python.org/issue23632 + + (3) https://bugs.python.org/issue24450 + + (4) https://bugs.python.org/issue19235 + + (5) https://bugs.python.org/issue19977 + + (6) https://bugs.python.org/issue22638 + + (7) https://bugs.python.org/issue22796 + + +File: python.info, Node: New Features<4>, Next: Other Language Changes<4>, Prev: Summary – Release highlights<3>, Up: What’s New In Python 3 5 + +1.4.2 New Features +------------------ + +* Menu: + +* PEP 492 - Coroutines with async and await syntax:: +* PEP 465 - A dedicated infix operator for matrix multiplication:: +* PEP 448 - Additional Unpacking Generalizations:: +* PEP 461 - percent formatting support for bytes and bytearray:: +* PEP 484 - Type Hints:: +* PEP 471 - os.scandir() function – a better and faster directory iterator: PEP 471 - os scandir function – a better and faster directory iterator. +* PEP 475; Retry system calls failing with EINTR: PEP 475 Retry system calls failing with EINTR. +* PEP 479; Change StopIteration handling inside generators: PEP 479 Change StopIteration handling inside generators. +* PEP 485; A function for testing approximate equality: PEP 485 A function for testing approximate equality. +* PEP 486; Make the Python Launcher aware of virtual environments: PEP 486 Make the Python Launcher aware of virtual environments. +* PEP 488; Elimination of PYO files: PEP 488 Elimination of PYO files. +* PEP 489; Multi-phase extension module initialization: PEP 489 Multi-phase extension module initialization. + + +File: python.info, Node: PEP 492 - Coroutines with async and await syntax, Next: PEP 465 - A dedicated infix operator for matrix multiplication, Up: New Features<4> + +1.4.2.1 PEP 492 - Coroutines with async and await syntax +........................................................ + +PEP 492(1) greatly improves support for asynchronous programming in +Python by adding *note awaitable objects: 519, *note coroutine +functions: 63f, *note asynchronous iteration: 640, and *note +asynchronous context managers: 641. + +Coroutine functions are declared using the new *note async def: 284. +syntax: + + >>> async def coro(): + ... return 'spam' + +Inside a coroutine function, the new *note await: 1a3. expression can be +used to suspend coroutine execution until the result is available. Any +object can be `awaited', as long as it implements the *note awaitable: +519. protocol by defining the *note __await__(): 642. method. + +PEP 492 also adds *note async for: 2e3. statement for convenient +iteration over asynchronous iterables. + +An example of a rudimentary HTTP client written using the new syntax: + + import asyncio + + async def http_get(domain): + reader, writer = await asyncio.open_connection(domain, 80) + + writer.write(b'\r\n'.join([ + b'GET / HTTP/1.1', + b'Host: %b' % domain.encode('latin-1'), + b'Connection: close', + b'', b'' + ])) + + async for line in reader: + print('>>>', line) + + writer.close() + + loop = asyncio.get_event_loop() + try: + loop.run_until_complete(http_get('example.com')) + finally: + loop.close() + +Similarly to asynchronous iteration, there is a new syntax for +asynchronous context managers. The following script: + + import asyncio + + async def coro(name, lock): + print('coro {}: waiting for lock'.format(name)) + async with lock: + print('coro {}: holding the lock'.format(name)) + await asyncio.sleep(1) + print('coro {}: releasing the lock'.format(name)) + + loop = asyncio.get_event_loop() + lock = asyncio.Lock() + coros = asyncio.gather(coro(1, lock), coro(2, lock)) + try: + loop.run_until_complete(coros) + finally: + loop.close() + +will output: + + coro 2: waiting for lock + coro 2: holding the lock + coro 1: waiting for lock + coro 2: releasing the lock + coro 1: holding the lock + coro 1: releasing the lock + +Note that both *note async for: 2e3. and *note async with: 643. can only +be used inside a coroutine function declared with *note async def: 284. + +Coroutine functions are intended to be run inside a compatible event +loop, such as the *note asyncio loop: 644. + + Note: + Changed in version 3.5.2: Starting with CPython 3.5.2, ‘__aiter__’ + can directly return *note asynchronous iterators: 645. Returning + an *note awaitable: 519. object will result in a *note + PendingDeprecationWarning: 279. + + See more details in the *note Asynchronous Iterators: 646. + documentation section. + +See also +........ + +PEP 492(2) – Coroutines with async and await syntax + + PEP written and implemented by Yury Selivanov. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0492 + + (2) https://www.python.org/dev/peps/pep-0492 + + +File: python.info, Node: PEP 465 - A dedicated infix operator for matrix multiplication, Next: PEP 448 - Additional Unpacking Generalizations, Prev: PEP 492 - Coroutines with async and await syntax, Up: New Features<4> + +1.4.2.2 PEP 465 - A dedicated infix operator for matrix multiplication +...................................................................... + +PEP 465(1) adds the ‘@’ infix operator for matrix multiplication. +Currently, no builtin Python types implement the new operator, however, +it can be implemented by defining *note __matmul__(): 648, *note +__rmatmul__(): 649, and *note __imatmul__(): 64a. for regular, +reflected, and in-place matrix multiplication. The semantics of these +methods is similar to that of methods defining other infix arithmetic +operators. + +Matrix multiplication is a notably common operation in many fields of +mathematics, science, engineering, and the addition of ‘@’ allows +writing cleaner code: + + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +instead of: + + S = dot((dot(H, beta) - r).T, + dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) + +NumPy 1.10 has support for the new operator: + + >>> import numpy + + >>> x = numpy.ones(3) + >>> x + array([ 1., 1., 1.]) + + >>> m = numpy.eye(3) + >>> m + array([[ 1., 0., 0.], + [ 0., 1., 0.], + [ 0., 0., 1.]]) + + >>> x @ m + array([ 1., 1., 1.]) + +See also +........ + +PEP 465(2) – A dedicated infix operator for matrix multiplication + + PEP written by Nathaniel J. Smith; implemented by Benjamin + Peterson. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0465 + + (2) https://www.python.org/dev/peps/pep-0465 + + +File: python.info, Node: PEP 448 - Additional Unpacking Generalizations, Next: PEP 461 - percent formatting support for bytes and bytearray, Prev: PEP 465 - A dedicated infix operator for matrix multiplication, Up: New Features<4> + +1.4.2.3 PEP 448 - Additional Unpacking Generalizations +...................................................... + +PEP 448(1) extends the allowed uses of the ‘*’ iterable unpacking +operator and ‘**’ dictionary unpacking operator. It is now possible to +use an arbitrary number of unpackings in *note function calls: 64c.: + + >>> print(*[1], *[2], 3, *[4, 5]) + 1 2 3 4 5 + + >>> def fn(a, b, c, d): + ... print(a, b, c, d) + ... + + >>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4}) + 1 2 3 4 + +Similarly, tuple, list, set, and dictionary displays allow multiple +unpackings (see *note Expression lists: 64d. and *note Dictionary +displays: 64e.): + + >>> *range(4), 4 + (0, 1, 2, 3, 4) + + >>> [*range(4), 4] + [0, 1, 2, 3, 4] + + >>> {*range(4), 4, *(5, 6, 7)} + {0, 1, 2, 3, 4, 5, 6, 7} + + >>> {'x': 1, **{'y': 2}} + {'x': 1, 'y': 2} + +See also +........ + +PEP 448(2) – Additional Unpacking Generalizations + + PEP written by Joshua Landau; implemented by Neil Girdhar, Thomas + Wouters, and Joshua Landau. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0448 + + (2) https://www.python.org/dev/peps/pep-0448 + + +File: python.info, Node: PEP 461 - percent formatting support for bytes and bytearray, Next: PEP 484 - Type Hints, Prev: PEP 448 - Additional Unpacking Generalizations, Up: New Features<4> + +1.4.2.4 PEP 461 - percent formatting support for bytes and bytearray +.................................................................... + +PEP 461(1) adds support for the ‘%’ *note interpolation operator: 650. +to *note bytes: 331. and *note bytearray: 332. + +While interpolation is usually thought of as a string operation, there +are cases where interpolation on ‘bytes’ or ‘bytearrays’ makes sense, +and the work needed to make up for this missing functionality detracts +from the overall readability of the code. This issue is particularly +important when dealing with wire format protocols, which are often a +mixture of binary and ASCII compatible text. + +Examples: + + >>> b'Hello %b!' % b'World' + b'Hello World!' + + >>> b'x=%i y=%f' % (1, 2.5) + b'x=1 y=2.500000' + +Unicode is not allowed for ‘%b’, but it is accepted by ‘%a’ (equivalent +of ‘repr(obj).encode('ascii', 'backslashreplace')’): + + >>> b'Hello %b!' % 'World' + Traceback (most recent call last): + File "", line 1, in + TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' + + >>> b'price: %a' % '10€' + b"price: '10\\u20ac'" + +Note that ‘%s’ and ‘%r’ conversion types, although supported, should +only be used in codebases that need compatibility with Python 2. + +See also +........ + +PEP 461(2) – Adding % formatting to bytes and bytearray + + PEP written by Ethan Furman; implemented by Neil Schemenauer and + Ethan Furman. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0461 + + (2) https://www.python.org/dev/peps/pep-0461 + + +File: python.info, Node: PEP 484 - Type Hints, Next: PEP 471 - os scandir function – a better and faster directory iterator, Prev: PEP 461 - percent formatting support for bytes and bytearray, Up: New Features<4> + +1.4.2.5 PEP 484 - Type Hints +............................ + +Function annotation syntax has been a Python feature since version 3.0 ( +PEP 3107(1)), however the semantics of annotations has been left +undefined. + +Experience has shown that the majority of function annotation uses were +to provide type hints to function parameters and return values. It +became evident that it would be beneficial for Python users, if the +standard library included the base definitions and tools for type +annotations. + +PEP 484(2) introduces a *note provisional module: 348. to provide these +standard definitions and tools, along with some conventions for +situations where annotations are not available. + +For example, here is a simple function whose argument and return type +are declared in the annotations: + + def greeting(name: str) -> str: + return 'Hello ' + name + +While these annotations are available at runtime through the usual +‘__annotations__’ attribute, `no automatic type checking happens at +runtime'. Instead, it is assumed that a separate off-line type checker +(e.g. mypy(3)) will be used for on-demand source code analysis. + +The type system supports unions, generic types, and a special type named +*note Any: 652. which is consistent with (i.e. assignable to and from) +all types. + +See also +........ + + * *note typing: 119. module documentation + + * + PEP 484(4) – Type Hints + + PEP written by Guido van Rossum, Jukka Lehtosalo, and Łukasz + Langa; implemented by Guido van Rossum. + + * + PEP 483(5) – The Theory of Type Hints + + PEP written by Guido van Rossum + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3107 + + (2) https://www.python.org/dev/peps/pep-0484 + + (3) http://mypy-lang.org + + (4) https://www.python.org/dev/peps/pep-0484 + + (5) https://www.python.org/dev/peps/pep-0483 + + +File: python.info, Node: PEP 471 - os scandir function – a better and faster directory iterator, Next: PEP 475 Retry system calls failing with EINTR, Prev: PEP 484 - Type Hints, Up: New Features<4> + +1.4.2.6 PEP 471 - os.scandir() function – a better and faster directory iterator +................................................................................ + +PEP 471(1) adds a new directory iteration function, *note os.scandir(): +25f, to the standard library. Additionally, *note os.walk(): 654. is +now implemented using ‘scandir’, which makes it 3 to 5 times faster on +POSIX systems and 7 to 20 times faster on Windows systems. This is +largely achieved by greatly reducing the number of calls to *note +os.stat(): 1f1. required to walk a directory tree. + +Additionally, ‘scandir’ returns an iterator, as opposed to returning a +list of file names, which improves memory efficiency when iterating over +very large directories. + +The following example shows a simple use of *note os.scandir(): 25f. to +display all the files (excluding directories) in the given `path' that +don’t start with ‘'.'’. The *note entry.is_file(): 655. call will +generally not make an additional system call: + + for entry in os.scandir(path): + if not entry.name.startswith('.') and entry.is_file(): + print(entry.name) + +See also +........ + +PEP 471(2) – os.scandir() function – a better and faster directory iterator + + PEP written and implemented by Ben Hoyt with the help of Victor + Stinner. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0471 + + (2) https://www.python.org/dev/peps/pep-0471 + + +File: python.info, Node: PEP 475 Retry system calls failing with EINTR, Next: PEP 479 Change StopIteration handling inside generators, Prev: PEP 471 - os scandir function – a better and faster directory iterator, Up: New Features<4> + +1.4.2.7 PEP 475: Retry system calls failing with EINTR +...................................................... + +An *note errno.EINTR: 658. error code is returned whenever a system +call, that is waiting for I/O, is interrupted by a signal. Previously, +Python would raise *note InterruptedError: 659. in such cases. This +meant that, when writing a Python application, the developer had two +choices: + + 1. Ignore the ‘InterruptedError’. + + 2. Handle the ‘InterruptedError’ and attempt to restart the + interrupted system call at every call site. + +The first option makes an application fail intermittently. The second +option adds a large amount of boilerplate that makes the code nearly +unreadable. Compare: + + print("Hello World") + +and: + + while True: + try: + print("Hello World") + break + except InterruptedError: + continue + +PEP 475(1) implements automatic retry of system calls on ‘EINTR’. This +removes the burden of dealing with ‘EINTR’ or *note InterruptedError: +659. in user code in most situations and makes Python programs, +including the standard library, more robust. Note that the system call +is only retried if the signal handler does not raise an exception. + +Below is a list of functions which are now retried when interrupted by a +signal: + + * *note open(): 4f0. and *note io.open(): 65a.; + + * functions of the *note faulthandler: 7d. module; + + * *note os: c4. functions: *note fchdir(): 65b, *note fchmod(): 65c, + *note fchown(): 65d, *note fdatasync(): 65e, *note fstat(): 65f, + *note fstatvfs(): 660, *note fsync(): 661, *note ftruncate(): 662, + *note mkfifo(): 663, *note mknod(): 664, *note open(): 665, *note + posix_fadvise(): 666, *note posix_fallocate(): 667, *note pread(): + 3b9, *note pwrite(): 3bc, *note read(): 668, *note readv(): 3b8, + *note sendfile(): 359, *note wait3(): 669, *note wait4(): 66a, + *note wait(): 66b, *note waitid(): 66c, *note waitpid(): 66d, *note + write(): 66e, *note writev(): 3bb.; + + * special cases: *note os.close(): 3d2. and *note os.dup2(): 3be. now + ignore *note EINTR: 658. errors; the syscall is not retried (see + the PEP for the rationale); + + * *note select: e5. functions: *note devpoll.poll(): 66f, *note + epoll.poll(): 670, *note kqueue.control(): 671, *note poll.poll(): + 672, *note select(): 673.; + + * methods of the *note socket: 674. class: *note accept(): 675, *note + connect(): 676. (except for non-blocking sockets), *note recv(): + 677, *note recvfrom(): 678, *note recvmsg(): 679, *note send(): + 67a, *note sendall(): 67b, *note sendmsg(): 67c, *note sendto(): + 67d.; + + * *note signal.sigtimedwait(): 67e. and *note signal.sigwaitinfo(): + 67f.; + + * *note time.sleep(): 680. + +See also +........ + +PEP 475(2) – Retry system calls failing with EINTR + + PEP and implementation written by Charles-François Natali and + Victor Stinner, with the help of Antoine Pitrou (the French + connection). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0475 + + (2) https://www.python.org/dev/peps/pep-0475 + + +File: python.info, Node: PEP 479 Change StopIteration handling inside generators, Next: PEP 485 A function for testing approximate equality, Prev: PEP 475 Retry system calls failing with EINTR, Up: New Features<4> + +1.4.2.8 PEP 479: Change StopIteration handling inside generators +................................................................ + +The interaction of generators and *note StopIteration: 486. in Python +3.4 and earlier was sometimes surprising, and could conceal obscure +bugs. Previously, ‘StopIteration’ raised accidentally inside a +generator function was interpreted as the end of the iteration by the +loop construct driving the generator. + +PEP 479(1) changes the behavior of generators: when a ‘StopIteration’ +exception is raised inside a generator, it is replaced with a *note +RuntimeError: 2ba. before it exits the generator frame. The main goal +of this change is to ease debugging in the situation where an unguarded +*note next(): 682. call raises ‘StopIteration’ and causes the iteration +controlled by the generator to terminate silently. This is particularly +pernicious in combination with the ‘yield from’ construct. + +This is a backwards incompatible change, so to enable the new behavior, +a *note __future__: 683. import is necessary: + + >>> from __future__ import generator_stop + + >>> def gen(): + ... next(iter([])) + ... yield + ... + >>> next(gen()) + Traceback (most recent call last): + File "", line 2, in gen + StopIteration + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + RuntimeError: generator raised StopIteration + +Without a ‘__future__’ import, a *note PendingDeprecationWarning: 279. +will be raised whenever a *note StopIteration: 486. exception is raised +inside a generator. + +See also +........ + +PEP 479(2) – Change StopIteration handling inside generators + + PEP written by Chris Angelico and Guido van Rossum. Implemented by + Chris Angelico, Yury Selivanov and Nick Coghlan. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0479 + + (2) https://www.python.org/dev/peps/pep-0479 + + +File: python.info, Node: PEP 485 A function for testing approximate equality, Next: PEP 486 Make the Python Launcher aware of virtual environments, Prev: PEP 479 Change StopIteration handling inside generators, Up: New Features<4> + +1.4.2.9 PEP 485: A function for testing approximate equality +............................................................ + +PEP 485(1) adds the *note math.isclose(): 686. and *note +cmath.isclose(): 687. functions which tell whether two values are +approximately equal or “close” to each other. Whether or not two values +are considered close is determined according to given absolute and +relative tolerances. Relative tolerance is the maximum allowed +difference between ‘isclose’ arguments, relative to the larger absolute +value: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, rel_tol=1e-5) + True + >>> math.isclose(a, b, rel_tol=1e-6) + False + +It is also possible to compare two values using absolute tolerance, +which must be a non-negative value: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, abs_tol=0.00003) + True + >>> math.isclose(a, b, abs_tol=0.00001) + False + +See also +........ + +PEP 485(2) – A function for testing approximate equality + + PEP written by Christopher Barker; implemented by Chris Barker and + Tal Einat. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0485 + + (2) https://www.python.org/dev/peps/pep-0485 + + +File: python.info, Node: PEP 486 Make the Python Launcher aware of virtual environments, Next: PEP 488 Elimination of PYO files, Prev: PEP 485 A function for testing approximate equality, Up: New Features<4> + +1.4.2.10 PEP 486: Make the Python Launcher aware of virtual environments +........................................................................ + +PEP 486(1) makes the Windows launcher (see PEP 397(2)) aware of an +active virtual environment. When the default interpreter would be used +and the ‘VIRTUAL_ENV’ environment variable is set, the interpreter in +the virtual environment will be used. + +See also +........ + +PEP 486(3) – Make the Python Launcher aware of virtual environments + + PEP written and implemented by Paul Moore. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0486 + + (2) https://www.python.org/dev/peps/pep-0397 + + (3) https://www.python.org/dev/peps/pep-0486 + + +File: python.info, Node: PEP 488 Elimination of PYO files, Next: PEP 489 Multi-phase extension module initialization, Prev: PEP 486 Make the Python Launcher aware of virtual environments, Up: New Features<4> + +1.4.2.11 PEP 488: Elimination of PYO files +.......................................... + +PEP 488(1) does away with the concept of ‘.pyo’ files. This means that +‘.pyc’ files represent both unoptimized and optimized bytecode. To +prevent the need to constantly regenerate bytecode files, ‘.pyc’ files +now have an optional ‘opt-’ tag in their name when the bytecode is +optimized. This has the side-effect of no more bytecode file name +clashes when running under either *note -O: 68b. or *note -OO: 68c. +Consequently, bytecode files generated from *note -O: 68b, and *note +-OO: 68c. may now exist simultaneously. *note +importlib.util.cache_from_source(): 557. has an updated API to help with +this change. + +See also +........ + +PEP 488(2) – Elimination of PYO files + + PEP written and implemented by Brett Cannon. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0488 + + (2) https://www.python.org/dev/peps/pep-0488 + + +File: python.info, Node: PEP 489 Multi-phase extension module initialization, Prev: PEP 488 Elimination of PYO files, Up: New Features<4> + +1.4.2.12 PEP 489: Multi-phase extension module initialization +............................................................. + +PEP 489(1) updates extension module initialization to take advantage of +the two step module loading mechanism introduced by PEP 451(2) in Python +3.4. + +This change brings the import semantics of extension modules that opt-in +to using the new mechanism much closer to those of Python source and +bytecode modules, including the ability to use any valid identifier as a +module name, rather than being restricted to ASCII. + +See also +........ + +PEP 489(3) – Multi-phase extension module initialization + + PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; + implemented by Petr Viktorin. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0489 + + (2) https://www.python.org/dev/peps/pep-0451 + + (3) https://www.python.org/dev/peps/pep-0489 + + +File: python.info, Node: Other Language Changes<4>, Next: New Modules<4>, Prev: New Features<4>, Up: What’s New In Python 3 5 + +1.4.3 Other Language Changes +---------------------------- + +Some smaller changes made to the core Python language are: + + * Added the ‘"namereplace"’ error handlers. The ‘"backslashreplace"’ + error handlers now work with decoding and translating. + (Contributed by Serhiy Storchaka in bpo-19676(1) and bpo-22286(2).) + + * The *note -b: 415. option now affects comparisons of *note bytes: + 331. with *note int: 184. (Contributed by Serhiy Storchaka in + bpo-23681(3).) + + * New Kazakh ‘kz1048’ and Tajik ‘koi8_t’ *note codecs: 68f. + (Contributed by Serhiy Storchaka in bpo-22682(4) and bpo-22681(5).) + + * Property docstrings are now writable. This is especially useful + for *note collections.namedtuple(): 1b7. docstrings. (Contributed + by Berker Peksag in bpo-24064(6).) + + * Circular imports involving relative imports are now supported. + (Contributed by Brett Cannon and Antoine Pitrou in bpo-17636(7).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19676 + + (2) https://bugs.python.org/issue22286 + + (3) https://bugs.python.org/issue23681 + + (4) https://bugs.python.org/issue22682 + + (5) https://bugs.python.org/issue22681 + + (6) https://bugs.python.org/issue24064 + + (7) https://bugs.python.org/issue17636 + + +File: python.info, Node: New Modules<4>, Next: Improved Modules<4>, Prev: Other Language Changes<4>, Up: What’s New In Python 3 5 + +1.4.4 New Modules +----------------- + +* Menu: + +* typing: typing<3>. +* zipapp: zipapp<2>. + + +File: python.info, Node: typing<3>, Next: zipapp<2>, Up: New Modules<4> + +1.4.4.1 typing +.............. + +The new *note typing: 119. *note provisional: 348. module provides +standard definitions and tools for function type annotations. See *note +Type Hints: 62e. for more information. + + +File: python.info, Node: zipapp<2>, Prev: typing<3>, Up: New Modules<4> + +1.4.4.2 zipapp +.............. + +The new *note zipapp: 141. module (specified in PEP 441(1)) provides an +API and command line tool for creating executable Python Zip +Applications, which were introduced in Python 2.6 in bpo-1739468(2), but +which were not well publicized, either at the time or since. + +With the new module, bundling your application is as simple as putting +all the files, including a ‘__main__.py’ file, into a directory ‘myapp’ +and running: + + $ python -m zipapp myapp + $ python myapp.pyz + +The module implementation has been contributed by Paul Moore in +bpo-23491(3). + +See also +........ + +PEP 441(4) – Improving Python ZIP Application Support + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0441 + + (2) https://bugs.python.org/issue1739468 + + (3) https://bugs.python.org/issue23491 + + (4) https://www.python.org/dev/peps/pep-0441 + + +File: python.info, Node: Improved Modules<4>, Next: Other module-level changes, Prev: New Modules<4>, Up: What’s New In Python 3 5 + +1.4.5 Improved Modules +---------------------- + +* Menu: + +* argparse: argparse<2>. +* asyncio: asyncio<5>. +* bz2:: +* cgi:: +* cmath: cmath<2>. +* code:: +* collections: collections<5>. +* collections.abc: collections abc. +* compileall: compileall<2>. +* concurrent.futures: concurrent futures<3>. +* configparser:: +* contextlib: contextlib<3>. +* csv: csv<2>. +* curses: curses<2>. +* dbm: dbm<4>. +* difflib:: +* distutils: distutils<4>. +* doctest:: +* email: email<2>. +* enum: enum<4>. +* faulthandler: faulthandler<2>. +* functools: functools<3>. +* glob:: +* gzip: gzip<2>. +* heapq:: +* http:: +* http.client: http client<3>. +* idlelib and IDLE: idlelib and IDLE<3>. +* imaplib:: +* imghdr:: +* importlib: importlib<5>. +* inspect: inspect<3>. +* io: io<3>. +* ipaddress: ipaddress<2>. +* json: json<2>. +* linecache:: +* locale: locale<3>. +* logging: logging<4>. +* lzma:: +* math: math<4>. +* multiprocessing: multiprocessing<4>. +* operator:: +* os: os<5>. +* pathlib: pathlib<4>. +* pickle: pickle<3>. +* poplib:: +* re: re<4>. +* readline: readline<2>. +* selectors:: +* shutil: shutil<2>. +* signal: signal<2>. +* smtpd:: +* smtplib:: +* sndhdr:: +* socket: socket<5>. +* ssl: ssl<6>. +* sqlite3: sqlite3<3>. +* subprocess: subprocess<3>. +* sys: sys<5>. +* sysconfig:: +* tarfile: tarfile<2>. +* threading: threading<3>. +* time: time<4>. +* timeit: timeit<2>. +* tkinter: tkinter<5>. +* traceback: traceback<2>. +* types: types<2>. +* unicodedata: unicodedata<4>. +* unittest: unittest<3>. +* unittest.mock: unittest mock<3>. +* urllib:: +* wsgiref:: +* xmlrpc: xmlrpc<2>. +* xml.sax: xml sax. +* zipfile: zipfile<3>. + + +File: python.info, Node: argparse<2>, Next: asyncio<5>, Up: Improved Modules<4> + +1.4.5.1 argparse +................ + +The *note ArgumentParser: 695. class now allows disabling *note +abbreviated usage: 696. of long options by setting *note allow_abbrev: +697. to ‘False’. (Contributed by Jonathan Paugh, Steven Bethard, paul +j3 and Daniel Eriksson in bpo-14910(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14910 + + +File: python.info, Node: asyncio<5>, Next: bz2, Prev: argparse<2>, Up: Improved Modules<4> + +1.4.5.2 asyncio +............... + +Since the *note asyncio: a. module is *note provisional: 348, all +changes introduced in Python 3.5 have also been backported to Python +3.4.x. + +Notable changes in the *note asyncio: a. module since Python 3.4.0: + + * New debugging APIs: *note loop.set_debug(): 699. and *note + loop.get_debug(): 69a. methods. (Contributed by Victor Stinner.) + + * The proactor event loop now supports SSL. (Contributed by Antoine + Pitrou and Victor Stinner in bpo-22560(1).) + + * A new *note loop.is_closed(): 69b. method to check if the event + loop is closed. (Contributed by Victor Stinner in bpo-21326(2).) + + * A new *note loop.create_task(): 1af. to conveniently create and + schedule a new *note Task: 1ad. for a coroutine. The ‘create_task’ + method is also used by all asyncio functions that wrap coroutines + into tasks, such as *note asyncio.wait(): 289, *note + asyncio.gather(): 286, etc. (Contributed by Victor Stinner.) + + * A new *note transport.get_write_buffer_limits(): 69c. method to + inquire for `high-' and `low-' water limits of the flow control. + (Contributed by Victor Stinner.) + + * The ‘async()’ function is deprecated in favor of *note + ensure_future(): 517. (Contributed by Yury Selivanov.) + + * New *note loop.set_task_factory(): 69d. and *note + loop.get_task_factory(): 69e. methods to customize the task factory + that *note loop.create_task(): 1af. method uses. (Contributed by + Yury Selivanov.) + + * New *note Queue.join(): 69f. and *note Queue.task_done(): 6a0. + queue methods. (Contributed by Victor Stinner.) + + * The ‘JoinableQueue’ class was removed, in favor of the *note + asyncio.Queue: 290. class. (Contributed by Victor Stinner.) + +Updates in 3.5.1: + + * The *note ensure_future(): 517. function and all functions that use + it, such as *note loop.run_until_complete(): 518, now accept all + kinds of *note awaitable objects: 519. (Contributed by Yury + Selivanov.) + + * New *note run_coroutine_threadsafe(): 51a. function to submit + coroutines to event loops from other threads. (Contributed by + Vincent Michel.) + + * New *note Transport.is_closing(): 51b. method to check if the + transport is closing or closed. (Contributed by Yury Selivanov.) + + * The *note loop.create_server(): 35d. method can now accept a list + of hosts. (Contributed by Yann Sionneau.) + +Updates in 3.5.2: + + * New *note loop.create_future(): 51c. method to create Future + objects. This allows alternative event loop implementations, such + as uvloop(3), to provide a faster *note asyncio.Future: 443. + implementation. (Contributed by Yury Selivanov.) + + * New *note loop.get_exception_handler(): 51d. method to get the + current exception handler. (Contributed by Yury Selivanov.) + + * New *note StreamReader.readuntil(): 51e. method to read data from + the stream until a separator bytes sequence appears. (Contributed + by Mark Korenberg.) + + * The *note loop.create_connection(): 1b2. and *note + loop.create_server(): 35d. methods are optimized to avoid calling + the system ‘getaddrinfo’ function if the address is already + resolved. (Contributed by A. Jesse Jiryu Davis.) + + * The *note loop.sock_connect(sock, address): 6a1. no longer requires + the `address' to be resolved prior to the call. (Contributed by A. + Jesse Jiryu Davis.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22560 + + (2) https://bugs.python.org/issue21326 + + (3) https://github.com/MagicStack/uvloop + + +File: python.info, Node: bz2, Next: cgi, Prev: asyncio<5>, Up: Improved Modules<4> + +1.4.5.3 bz2 +........... + +The *note BZ2Decompressor.decompress: 6a3. method now accepts an +optional `max_length' argument to limit the maximum size of decompressed +data. (Contributed by Nikolaus Rath in bpo-15955(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15955 + + +File: python.info, Node: cgi, Next: cmath<2>, Prev: bz2, Up: Improved Modules<4> + +1.4.5.4 cgi +........... + +The ‘FieldStorage’ class now supports the *note context manager: 568. +protocol. (Contributed by Berker Peksag in bpo-20289(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20289 + + +File: python.info, Node: cmath<2>, Next: code, Prev: cgi, Up: Improved Modules<4> + +1.4.5.5 cmath +............. + +A new function *note isclose(): 687. provides a way to test for +approximate equality. (Contributed by Chris Barker and Tal Einat in +bpo-24270(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24270 + + +File: python.info, Node: code, Next: collections<5>, Prev: cmath<2>, Up: Improved Modules<4> + +1.4.5.6 code +............ + +The *note InteractiveInterpreter.showtraceback(): 6a7. method now prints +the full chained traceback, just like the interactive interpreter. +(Contributed by Claudiu Popa in bpo-17442(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17442 + + +File: python.info, Node: collections<5>, Next: collections abc, Prev: code, Up: Improved Modules<4> + +1.4.5.7 collections +................... + +The *note OrderedDict: 1b9. class is now implemented in C, which makes +it 4 to 100 times faster. (Contributed by Eric Snow in bpo-16991(1).) + +‘OrderedDict.items()’, ‘OrderedDict.keys()’, ‘OrderedDict.values()’ +views now support *note reversed(): 18e. iteration. (Contributed by +Serhiy Storchaka in bpo-19505(2).) + +The *note deque: 531. class now defines *note index(): 6a9, *note +insert(): 6aa, and *note copy(): 6ab, and supports the ‘+’ and ‘*’ +operators. This allows deques to be recognized as a *note +MutableSequence: 6ac. and improves their substitutability for lists. +(Contributed by Raymond Hettinger in bpo-23704(3).) + +Docstrings produced by *note namedtuple(): 1b7. can now be updated: + + Point = namedtuple('Point', ['x', 'y']) + Point.__doc__ += ': Cartesian coodinate' + Point.x.__doc__ = 'abscissa' + Point.y.__doc__ = 'ordinate' + +(Contributed by Berker Peksag in bpo-24064(4).) + +The *note UserString: 6ad. class now implements the *note +__getnewargs__(): 6ae, *note __rmod__(): 6af, *note casefold(): 6b0, +*note format_map(): 6b1, *note isprintable(): 6b2, and *note +maketrans(): 6b3. methods to match the corresponding methods of *note +str: 330. (Contributed by Joe Jevnik in bpo-22189(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16991 + + (2) https://bugs.python.org/issue19505 + + (3) https://bugs.python.org/issue23704 + + (4) https://bugs.python.org/issue24064 + + (5) https://bugs.python.org/issue22189 + + +File: python.info, Node: collections abc, Next: compileall<2>, Prev: collections<5>, Up: Improved Modules<4> + +1.4.5.8 collections.abc +....................... + +The ‘Sequence.index()’ method now accepts `start' and `stop' arguments +to match the corresponding methods of *note tuple: 47e, *note list: 262, +etc. (Contributed by Devin Jeanpierre in bpo-23086(1).) + +A new *note Generator: 6b5. abstract base class. (Contributed by Stefan +Behnel in bpo-24018(2).) + +New *note Awaitable: 6b6, *note Coroutine: 6b7, *note AsyncIterator: +6b8, and *note AsyncIterable: 6b9. abstract base classes. (Contributed +by Yury Selivanov in bpo-24184(3).) + +For earlier Python versions, a backport of the new ABCs is available in +an external PyPI package(4). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23086 + + (2) https://bugs.python.org/issue24018 + + (3) https://bugs.python.org/issue24184 + + (4) https://pypi.org/project/backports_abc + + +File: python.info, Node: compileall<2>, Next: concurrent futures<3>, Prev: collections abc, Up: Improved Modules<4> + +1.4.5.9 compileall +.................. + +A new *note compileall: 21. option, ‘-j `N'’, allows running `N' workers +simultaneously to perform parallel bytecode compilation. The *note +compile_dir(): 372. function has a corresponding ‘workers’ parameter. +(Contributed by Claudiu Popa in bpo-16104(1).) + +Another new option, ‘-r’, allows controlling the maximum recursion level +for subdirectories. (Contributed by Claudiu Popa in bpo-19628(2).) + +The ‘-q’ command line option can now be specified more than once, in +which case all output, including errors, will be suppressed. The +corresponding ‘quiet’ parameter in *note compile_dir(): 372, *note +compile_file(): 6bb, and *note compile_path(): 6bc. can now accept an +integer value indicating the level of output suppression. (Contributed +by Thomas Kluyver in bpo-21338(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16104 + + (2) https://bugs.python.org/issue19628 + + (3) https://bugs.python.org/issue21338 + + +File: python.info, Node: concurrent futures<3>, Next: configparser, Prev: compileall<2>, Up: Improved Modules<4> + +1.4.5.10 concurrent.futures +........................... + +The *note Executor.map(): 6be. method now accepts a `chunksize' argument +to allow batching of tasks to improve performance when *note +ProcessPoolExecutor(): 2a4. is used. (Contributed by Dan O’Reilly in +bpo-11271(1).) + +The number of workers in the *note ThreadPoolExecutor: 27a. constructor +is optional now. The default value is 5 times the number of CPUs. +(Contributed by Claudiu Popa in bpo-21527(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11271 + + (2) https://bugs.python.org/issue21527 + + +File: python.info, Node: configparser, Next: contextlib<3>, Prev: concurrent futures<3>, Up: Improved Modules<4> + +1.4.5.11 configparser +..................... + +*note configparser: 23. now provides a way to customize the conversion +of values by specifying a dictionary of converters in the *note +ConfigParser: 4aa. constructor, or by defining them as methods in +‘ConfigParser’ subclasses. Converters defined in a parser instance are +inherited by its section proxies. + +Example: + + >>> import configparser + >>> conv = {} + >>> conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()] + >>> cfg = configparser.ConfigParser(converters=conv) + >>> cfg.read_string(""" + ... [s] + ... list = a b c d e f g + ... """) + >>> cfg.get('s', 'list') + 'a b c d e f g' + >>> cfg.getlist('s', 'list') + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] + >>> section = cfg['s'] + >>> section.getlist('list') + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] + +(Contributed by Łukasz Langa in bpo-18159(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18159 + + +File: python.info, Node: contextlib<3>, Next: csv<2>, Prev: configparser, Up: Improved Modules<4> + +1.4.5.12 contextlib +................... + +The new *note redirect_stderr(): 6c1. *note context manager: 568. +(similar to *note redirect_stdout(): 6c2.) makes it easier for utility +scripts to handle inflexible APIs that write their output to *note +sys.stderr: 30f. and don’t provide any options to redirect it: + + >>> import contextlib, io, logging + >>> f = io.StringIO() + >>> with contextlib.redirect_stderr(f): + ... logging.warning('warning') + ... + >>> f.getvalue() + 'WARNING:root:warning\n' + +(Contributed by Berker Peksag in bpo-22389(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22389 + + +File: python.info, Node: csv<2>, Next: curses<2>, Prev: contextlib<3>, Up: Improved Modules<4> + +1.4.5.13 csv +............ + +The *note writerow(): 6c4. method now supports arbitrary iterables, not +just sequences. (Contributed by Serhiy Storchaka in bpo-23171(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23171 + + +File: python.info, Node: curses<2>, Next: dbm<4>, Prev: csv<2>, Up: Improved Modules<4> + +1.4.5.14 curses +............... + +The new *note update_lines_cols(): 6c6. function updates the ‘LINES’ and +‘COLS’ environment variables. This is useful for detecting manual +screen resizing. (Contributed by Arnon Yaari in bpo-4254(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4254 + + +File: python.info, Node: dbm<4>, Next: difflib, Prev: curses<2>, Up: Improved Modules<4> + +1.4.5.15 dbm +............ + +*note dumb.open: 2bf. always creates a new database when the flag has +the value ‘"n"’. (Contributed by Claudiu Popa in bpo-18039(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18039 + + +File: python.info, Node: difflib, Next: distutils<4>, Prev: dbm<4>, Up: Improved Modules<4> + +1.4.5.16 difflib +................ + +The charset of HTML documents generated by *note HtmlDiff.make_file(): +6c9. can now be customized by using a new `charset' keyword-only +argument. The default charset of HTML document changed from +‘"ISO-8859-1"’ to ‘"utf-8"’. (Contributed by Berker Peksag in +bpo-2052(1).) + +The *note diff_bytes(): 6ca. function can now compare lists of byte +strings. This fixes a regression from Python 2. (Contributed by Terry +J. Reedy and Greg Ward in bpo-17445(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2052 + + (2) https://bugs.python.org/issue17445 + + +File: python.info, Node: distutils<4>, Next: doctest, Prev: difflib, Up: Improved Modules<4> + +1.4.5.17 distutils +.................. + +Both the ‘build’ and ‘build_ext’ commands now accept a ‘-j’ option to +enable parallel building of extension modules. (Contributed by Antoine +Pitrou in bpo-5309(1).) + +The *note distutils: 39. module now supports ‘xz’ compression, and can +be enabled by passing ‘xztar’ as an argument to ‘bdist --format’. +(Contributed by Serhiy Storchaka in bpo-16314(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5309 + + (2) https://bugs.python.org/issue16314 + + +File: python.info, Node: doctest, Next: email<2>, Prev: distutils<4>, Up: Improved Modules<4> + +1.4.5.18 doctest +................ + +The *note DocTestSuite(): 6cd. function returns an empty *note +unittest.TestSuite: 6ce. if `module' contains no docstrings, instead of +raising *note ValueError: 1fb. (Contributed by Glenn Jones in +bpo-15916(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15916 + + +File: python.info, Node: email<2>, Next: enum<4>, Prev: doctest, Up: Improved Modules<4> + +1.4.5.19 email +.............. + +A new policy option *note Policy.mangle_from_: 6d0. controls whether or +not lines that start with ‘"From "’ in email bodies are prefixed with a +‘">"’ character by generators. The default is ‘True’ for *note +compat32: 540. and ‘False’ for all other policies. (Contributed by +Milan Oberkirch in bpo-20098(1).) + +A new *note Message.get_content_disposition(): 6d1. method provides easy +access to a canonical value for the ‘Content-Disposition’ header. +(Contributed by Abhilash Raj in bpo-21083(2).) + +A new policy option *note EmailPolicy.utf8: 6d2. can be set to ‘True’ to +encode email headers using the UTF-8 charset instead of using encoded +words. This allows ‘Messages’ to be formatted according to RFC 6532(3) +and used with an SMTP server that supports the RFC 6531(4) ‘SMTPUTF8’ +extension. (Contributed by R. David Murray in bpo-24211(5).) + +The *note mime.text.MIMEText: 6d3. constructor now accepts a *note +charset.Charset: 6d4. instance. (Contributed by Claude Paroz and Berker +Peksag in bpo-16324(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20098 + + (2) https://bugs.python.org/issue21083 + + (3) https://tools.ietf.org/html/rfc6532.html + + (4) https://tools.ietf.org/html/rfc6531.html + + (5) https://bugs.python.org/issue24211 + + (6) https://bugs.python.org/issue16324 + + +File: python.info, Node: enum<4>, Next: faulthandler<2>, Prev: email<2>, Up: Improved Modules<4> + +1.4.5.20 enum +............. + +The *note Enum: 387. callable has a new parameter `start' to specify the +initial number of enum values if only `names' are provided: + + >>> Animal = enum.Enum('Animal', 'cat dog', start=10) + >>> Animal.cat + + >>> Animal.dog + + +(Contributed by Ethan Furman in bpo-21706(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21706 + + +File: python.info, Node: faulthandler<2>, Next: functools<3>, Prev: enum<4>, Up: Improved Modules<4> + +1.4.5.21 faulthandler +..................... + +The *note enable(): 548, *note register(): 6d7, *note dump_traceback(): +6d8. and *note dump_traceback_later(): 6d9. functions now accept file +descriptors in addition to file-like objects. (Contributed by Wei Wu in +bpo-23566(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23566 + + +File: python.info, Node: functools<3>, Next: glob, Prev: faulthandler<2>, Up: Improved Modules<4> + +1.4.5.22 functools +.................. + +Most of the *note lru_cache(): 1c7. machinery is now implemented in C, +making it significantly faster. (Contributed by Matt Joiner, Alexey +Kachayev, and Serhiy Storchaka in bpo-14373(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14373 + + +File: python.info, Node: glob, Next: gzip<2>, Prev: functools<3>, Up: Improved Modules<4> + +1.4.5.23 glob +............. + +The *note iglob(): 5c7. and *note glob(): 5c6. functions now support +recursive search in subdirectories, using the ‘"**"’ pattern. +(Contributed by Serhiy Storchaka in bpo-13968(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13968 + + +File: python.info, Node: gzip<2>, Next: heapq, Prev: glob, Up: Improved Modules<4> + +1.4.5.24 gzip +............. + +The `mode' argument of the *note GzipFile: 6dd. constructor now accepts +‘"x"’ to request exclusive creation. (Contributed by Tim Heaney in +bpo-19222(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19222 + + +File: python.info, Node: heapq, Next: http, Prev: gzip<2>, Up: Improved Modules<4> + +1.4.5.25 heapq +.............. + +Element comparison in *note merge(): 6df. can now be customized by +passing a *note key function: 6e0. in a new optional `key' keyword +argument, and a new optional `reverse' keyword argument can be used to +reverse element comparison: + + >>> import heapq + >>> a = ['9', '777', '55555'] + >>> b = ['88', '6666'] + >>> list(heapq.merge(a, b, key=len)) + ['9', '88', '777', '6666', '55555'] + >>> list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True)) + ['55555', '6666', '777', '88', '9'] + +(Contributed by Raymond Hettinger in bpo-13742(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13742 + + +File: python.info, Node: http, Next: http client<3>, Prev: heapq, Up: Improved Modules<4> + +1.4.5.26 http +............. + +A new *note HTTPStatus: 6e2. enum that defines a set of HTTP status +codes, reason phrases and long descriptions written in English. +(Contributed by Demian Brecht in bpo-21793(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21793 + + +File: python.info, Node: http client<3>, Next: idlelib and IDLE<3>, Prev: http, Up: Improved Modules<4> + +1.4.5.27 http.client +.................... + +*note HTTPConnection.getresponse(): 6e4. now raises a *note +RemoteDisconnected: 6e5. exception when a remote server connection is +closed unexpectedly. Additionally, if a *note ConnectionError: 6e6. (of +which ‘RemoteDisconnected’ is a subclass) is raised, the client socket +is now closed automatically, and will reconnect on the next request: + + import http.client + conn = http.client.HTTPConnection('www.python.org') + for retries in range(3): + try: + conn.request('GET', '/') + resp = conn.getresponse() + except http.client.RemoteDisconnected: + pass + +(Contributed by Martin Panter in bpo-3566(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue3566 + + +File: python.info, Node: idlelib and IDLE<3>, Next: imaplib, Prev: http client<3>, Up: Improved Modules<4> + +1.4.5.28 idlelib and IDLE +......................... + +Since idlelib implements the IDLE shell and editor and is not intended +for import by other programs, it gets improvements with every release. +See ‘Lib/idlelib/NEWS.txt’ for a cumulative list of changes since 3.4.0, +as well as changes made in future 3.5.x releases. This file is also +available from the IDLE Help ‣ About IDLE dialog. + + +File: python.info, Node: imaplib, Next: imghdr, Prev: idlelib and IDLE<3>, Up: Improved Modules<4> + +1.4.5.29 imaplib +................ + +The *note IMAP4: 60b. class now supports the *note context manager: 568. +protocol. When used in a *note with: 6e9. statement, the IMAP4 ‘LOGOUT’ +command will be called automatically at the end of the block. +(Contributed by Tarek Ziadé and Serhiy Storchaka in bpo-4972(1).) + +The *note imaplib: 98. module now supports RFC 5161(2) (ENABLE +Extension) and RFC 6855(3) (UTF-8 Support) via the *note IMAP4.enable(): +6ea. method. A new *note IMAP4.utf8_enabled: 6eb. attribute tracks +whether or not RFC 6855(4) support is enabled. (Contributed by Milan +Oberkirch, R. David Murray, and Maciej Szulik in bpo-21800(5).) + +The *note imaplib: 98. module now automatically encodes non-ASCII string +usernames and passwords using UTF-8, as recommended by the RFCs. +(Contributed by Milan Oberkirch in bpo-21800(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4972 + + (2) https://tools.ietf.org/html/rfc5161.html + + (3) https://tools.ietf.org/html/rfc6855.html + + (4) https://tools.ietf.org/html/rfc6855.html + + (5) https://bugs.python.org/issue21800 + + (6) https://bugs.python.org/issue21800 + + +File: python.info, Node: imghdr, Next: importlib<5>, Prev: imaplib, Up: Improved Modules<4> + +1.4.5.30 imghdr +............... + +The *note what(): 6ed. function now recognizes the OpenEXR(1) format +(contributed by Martin Vignali and Claudiu Popa in bpo-20295(2)), and +the WebP(3) format (contributed by Fabrice Aneche and Claudiu Popa in +bpo-20197(4).) + + ---------- Footnotes ---------- + + (1) http://www.openexr.com + + (2) https://bugs.python.org/issue20295 + + (3) https://en.wikipedia.org/wiki/WebP + + (4) https://bugs.python.org/issue20197 + + +File: python.info, Node: importlib<5>, Next: inspect<3>, Prev: imghdr, Up: Improved Modules<4> + +1.4.5.31 importlib +.................. + +The *note util.LazyLoader: 553. class allows for lazy loading of modules +in applications where startup time is important. (Contributed by Brett +Cannon in bpo-17621(1).) + +The *note abc.InspectLoader.source_to_code(): 6ef. method is now a +static method. This makes it easier to initialize a module object with +code compiled from a string by running ‘exec(code, module.__dict__)’. +(Contributed by Brett Cannon in bpo-21156(2).) + +The new *note util.module_from_spec(): 6f0. function is now the +preferred way to create a new module. As opposed to creating a *note +types.ModuleType: 6f1. instance directly, this new function will set the +various import-controlled attributes based on the passed-in spec object. +(Contributed by Brett Cannon in bpo-20383(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17621 + + (2) https://bugs.python.org/issue21156 + + (3) https://bugs.python.org/issue20383 + + +File: python.info, Node: inspect<3>, Next: io<3>, Prev: importlib<5>, Up: Improved Modules<4> + +1.4.5.32 inspect +................ + +Both the *note Signature: 6f3. and *note Parameter: 6f4. classes are now +picklable and hashable. (Contributed by Yury Selivanov in bpo-20726(1) +and bpo-20334(2).) + +A new *note BoundArguments.apply_defaults(): 6f5. method provides a way +to set default values for missing arguments: + + >>> def foo(a, b='ham', *args): pass + >>> ba = inspect.signature(foo).bind('spam') + >>> ba.apply_defaults() + >>> ba.arguments + OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())]) + +(Contributed by Yury Selivanov in bpo-24190(3).) + +A new class method *note Signature.from_callable(): 6f6. makes +subclassing of *note Signature: 6f3. easier. (Contributed by Yury +Selivanov and Eric Snow in bpo-17373(4).) + +The *note signature(): 55b. function now accepts a `follow_wrapped' +optional keyword argument, which, when set to ‘False’, disables +automatic following of ‘__wrapped__’ links. (Contributed by Yury +Selivanov in bpo-20691(5).) + +A set of new functions to inspect *note coroutine functions: 63f. and +*note coroutine objects: 1a6. has been added: *note iscoroutine(): 6f7, +*note iscoroutinefunction(): 6f8, *note isawaitable(): 6f9, *note +getcoroutinelocals(): 6fa, and *note getcoroutinestate(): 6fb. +(Contributed by Yury Selivanov in bpo-24017(6) and bpo-24400(7).) + +The *note stack(): 6fc, *note trace(): 6fd, *note getouterframes(): 6fe, +and *note getinnerframes(): 6ff. functions now return a list of named +tuples. (Contributed by Daniel Shahaf in bpo-16808(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20726 + + (2) https://bugs.python.org/issue20334 + + (3) https://bugs.python.org/issue24190 + + (4) https://bugs.python.org/issue17373 + + (5) https://bugs.python.org/issue20691 + + (6) https://bugs.python.org/issue24017 + + (7) https://bugs.python.org/issue24400 + + (8) https://bugs.python.org/issue16808 + + +File: python.info, Node: io<3>, Next: ipaddress<2>, Prev: inspect<3>, Up: Improved Modules<4> + +1.4.5.33 io +........... + +A new *note BufferedIOBase.readinto1(): 701. method, that uses at most +one call to the underlying raw stream’s *note RawIOBase.read(): 702. or +*note RawIOBase.readinto(): 703. methods. (Contributed by Nikolaus Rath +in bpo-20578(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20578 + + +File: python.info, Node: ipaddress<2>, Next: json<2>, Prev: io<3>, Up: Improved Modules<4> + +1.4.5.34 ipaddress +.................. + +Both the *note IPv4Network: 3a0. and *note IPv6Network: 39f. classes now +accept an ‘(address, netmask)’ tuple argument, so as to easily construct +network objects from existing addresses: + + >>> import ipaddress + >>> ipaddress.IPv4Network(('127.0.0.0', 8)) + IPv4Network('127.0.0.0/8') + >>> ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0')) + IPv4Network('127.0.0.0/8') + +(Contributed by Peter Moody and Antoine Pitrou in bpo-16531(1).) + +A new ‘reverse_pointer’ attribute for the *note IPv4Network: 3a0. and +*note IPv6Network: 39f. classes returns the name of the reverse DNS PTR +record: + + >>> import ipaddress + >>> addr = ipaddress.IPv4Address('127.0.0.1') + >>> addr.reverse_pointer + '1.0.0.127.in-addr.arpa' + >>> addr6 = ipaddress.IPv6Address('::1') + >>> addr6.reverse_pointer + '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa' + +(Contributed by Leon Weber in bpo-20480(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16531 + + (2) https://bugs.python.org/issue20480 + + +File: python.info, Node: json<2>, Next: linecache, Prev: ipaddress<2>, Up: Improved Modules<4> + +1.4.5.35 json +............. + +The *note json.tool: a5. command line interface now preserves the order +of keys in JSON objects passed in input. The new ‘--sort-keys’ option +can be used to sort the keys alphabetically. (Contributed by Berker +Peksag in bpo-21650(1).) + +JSON decoder now raises *note JSONDecodeError: 706. instead of *note +ValueError: 1fb. to provide better context information about the error. +(Contributed by Serhiy Storchaka in bpo-19361(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21650 + + (2) https://bugs.python.org/issue19361 + + +File: python.info, Node: linecache, Next: locale<3>, Prev: json<2>, Up: Improved Modules<4> + +1.4.5.36 linecache +.................. + +A new *note lazycache(): 708. function can be used to capture +information about a non-file-based module to permit getting its lines +later via *note getline(): 709. This avoids doing I/O until a line is +actually needed, without having to carry the module globals around +indefinitely. (Contributed by Robert Collins in bpo-17911(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17911 + + +File: python.info, Node: locale<3>, Next: logging<4>, Prev: linecache, Up: Improved Modules<4> + +1.4.5.37 locale +............... + +A new *note delocalize(): 70b. function can be used to convert a string +into a normalized number string, taking the ‘LC_NUMERIC’ settings into +account: + + >>> import locale + >>> locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8') + 'de_DE.UTF-8' + >>> locale.delocalize('1.234,56') + '1234.56' + >>> locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8') + 'en_US.UTF-8' + >>> locale.delocalize('1,234.56') + '1234.56' + +(Contributed by Cédric Krier in bpo-13918(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13918 + + +File: python.info, Node: logging<4>, Next: lzma, Prev: locale<3>, Up: Improved Modules<4> + +1.4.5.38 logging +................ + +All logging methods (*note Logger: 3a7. *note log(): 70d, *note +exception(): 70e, *note critical(): 70f, *note debug(): 710, etc.), now +accept exception instances as an `exc_info' argument, in addition to +boolean values and exception tuples: + + >>> import logging + >>> try: + ... 1/0 + ... except ZeroDivisionError as ex: + ... logging.error('exception', exc_info=ex) + ERROR:root:exception + +(Contributed by Yury Selivanov in bpo-20537(1).) + +The *note handlers.HTTPHandler: 711. class now accepts an optional *note +ssl.SSLContext: 3e4. instance to configure SSL settings used in an HTTP +connection. (Contributed by Alex Gaynor in bpo-22788(2).) + +The *note handlers.QueueListener: 712. class now takes a +`respect_handler_level' keyword argument which, if set to ‘True’, will +pass messages to handlers taking handler levels into account. +(Contributed by Vinay Sajip.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20537 + + (2) https://bugs.python.org/issue22788 + + +File: python.info, Node: lzma, Next: math<4>, Prev: logging<4>, Up: Improved Modules<4> + +1.4.5.39 lzma +............. + +The *note LZMADecompressor.decompress(): 714. method now accepts an +optional `max_length' argument to limit the maximum size of decompressed +data. (Contributed by Martin Panter in bpo-15955(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15955 + + +File: python.info, Node: math<4>, Next: multiprocessing<4>, Prev: lzma, Up: Improved Modules<4> + +1.4.5.40 math +............. + +Two new constants have been added to the *note math: b1. module: *note +inf: 528. and *note nan: 529. (Contributed by Mark Dickinson in +bpo-23185(1).) + +A new function *note isclose(): 686. provides a way to test for +approximate equality. (Contributed by Chris Barker and Tal Einat in +bpo-24270(2).) + +A new *note gcd(): 716. function has been added. The *note +fractions.gcd(): 717. function is now deprecated. (Contributed by Mark +Dickinson and Serhiy Storchaka in bpo-22486(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23185 + + (2) https://bugs.python.org/issue24270 + + (3) https://bugs.python.org/issue22486 + + +File: python.info, Node: multiprocessing<4>, Next: operator, Prev: math<4>, Up: Improved Modules<4> + +1.4.5.41 multiprocessing +........................ + +*note sharedctypes.synchronized(): 719. objects now support the *note +context manager: 568. protocol. (Contributed by Charles-François Natali +in bpo-21565(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21565 + + +File: python.info, Node: operator, Next: os<5>, Prev: multiprocessing<4>, Up: Improved Modules<4> + +1.4.5.42 operator +................. + +*note attrgetter(): 71b, *note itemgetter(): 261, and *note +methodcaller(): 71c. objects now support pickling. (Contributed by Josh +Rosenberg and Serhiy Storchaka in bpo-22955(1).) + +New *note matmul(): 71d. and *note imatmul(): 71e. functions to perform +matrix multiplication. (Contributed by Benjamin Peterson in +bpo-21176(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22955 + + (2) https://bugs.python.org/issue21176 + + +File: python.info, Node: os<5>, Next: pathlib<4>, Prev: operator, Up: Improved Modules<4> + +1.4.5.43 os +........... + +The new *note scandir(): 25f. function returning an iterator of *note +DirEntry: 4f1. objects has been added. If possible, *note scandir(): +25f. extracts file attributes while scanning a directory, removing the +need to perform subsequent system calls to determine file type or +attributes, which may significantly improve performance. (Contributed +by Ben Hoyt with the help of Victor Stinner in bpo-22524(1).) + +On Windows, a new *note stat_result.st_file_attributes: 720. attribute +is now available. It corresponds to the ‘dwFileAttributes’ member of +the ‘BY_HANDLE_FILE_INFORMATION’ structure returned by +‘GetFileInformationByHandle()’. (Contributed by Ben Hoyt in +bpo-21719(2).) + +The *note urandom(): 4d1. function now uses the ‘getrandom()’ syscall on +Linux 3.17 or newer, and ‘getentropy()’ on OpenBSD 5.6 and newer, +removing the need to use ‘/dev/urandom’ and avoiding failures due to +potential file descriptor exhaustion. (Contributed by Victor Stinner in +bpo-22181(3).) + +New *note get_blocking(): 721. and *note set_blocking(): 722. functions +allow getting and setting a file descriptor’s blocking mode (*note +O_NONBLOCK: 723.) (Contributed by Victor Stinner in bpo-22054(4).) + +The *note truncate(): 724. and *note ftruncate(): 662. functions are now +supported on Windows. (Contributed by Steve Dower in bpo-23668(5).) + +There is a new *note os.path.commonpath(): 725. function returning the +longest common sub-path of each passed pathname. Unlike the *note +os.path.commonprefix(): 726. function, it always returns a valid path: + + >>> os.path.commonprefix(['/usr/lib', '/usr/local/lib']) + '/usr/l' + + >>> os.path.commonpath(['/usr/lib', '/usr/local/lib']) + '/usr' + +(Contributed by Rafik Draoui and Serhiy Storchaka in bpo-10395(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22524 + + (2) https://bugs.python.org/issue21719 + + (3) https://bugs.python.org/issue22181 + + (4) https://bugs.python.org/issue22054 + + (5) https://bugs.python.org/issue23668 + + (6) https://bugs.python.org/issue10395 + + +File: python.info, Node: pathlib<4>, Next: pickle<3>, Prev: os<5>, Up: Improved Modules<4> + +1.4.5.44 pathlib +................ + +The new *note Path.samefile(): 728. method can be used to check whether +the path points to the same file as another path, which can be either +another *note Path: 201. object, or a string: + + >>> import pathlib + >>> p1 = pathlib.Path('/etc/hosts') + >>> p2 = pathlib.Path('/etc/../etc/hosts') + >>> p1.samefile(p2) + True + +(Contributed by Vajrasky Kok and Antoine Pitrou in bpo-19775(1).) + +The *note Path.mkdir(): 729. method now accepts a new optional +`exist_ok' argument to match ‘mkdir -p’ and *note os.makedirs(): 3bd. +functionality. (Contributed by Berker Peksag in bpo-21539(2).) + +There is a new *note Path.expanduser(): 72a. method to expand ‘~’ and +‘~user’ prefixes. (Contributed by Serhiy Storchaka and Claudiu Popa in +bpo-19776(3).) + +A new *note Path.home(): 72b. class method can be used to get a *note +Path: 201. instance representing the user’s home directory. +(Contributed by Victor Salgado and Mayank Tripathi in bpo-19777(4).) + +New *note Path.write_text(): 72c, *note Path.read_text(): 72d, *note +Path.write_bytes(): 72e, *note Path.read_bytes(): 72f. methods to +simplify read/write operations on files. + +The following code snippet will create or rewrite existing file +‘~/spam42’: + + >>> import pathlib + >>> p = pathlib.Path('~/spam42') + >>> p.expanduser().write_text('ham') + 3 + +(Contributed by Christopher Welborn in bpo-20218(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19775 + + (2) https://bugs.python.org/issue21539 + + (3) https://bugs.python.org/issue19776 + + (4) https://bugs.python.org/issue19777 + + (5) https://bugs.python.org/issue20218 + + +File: python.info, Node: pickle<3>, Next: poplib, Prev: pathlib<4>, Up: Improved Modules<4> + +1.4.5.45 pickle +............... + +Nested objects, such as unbound methods or nested classes, can now be +pickled using *note pickle protocols: 56e. older than protocol version +4. Protocol version 4 already supports these cases. (Contributed by +Serhiy Storchaka in bpo-23611(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23611 + + +File: python.info, Node: poplib, Next: re<4>, Prev: pickle<3>, Up: Improved Modules<4> + +1.4.5.46 poplib +............... + +A new *note POP3.utf8(): 732. command enables RFC 6856(1) +(Internationalized Email) support, if a POP server supports it. +(Contributed by Milan OberKirch in bpo-21804(2).) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc6856.html + + (2) https://bugs.python.org/issue21804 + + +File: python.info, Node: re<4>, Next: readline<2>, Prev: poplib, Up: Improved Modules<4> + +1.4.5.47 re +........... + +References and conditional references to groups with fixed length are +now allowed in lookbehind assertions: + + >>> import re + >>> pat = re.compile(r'(a|b).(?<=\1)c') + >>> pat.match('aac') + <_sre.SRE_Match object; span=(0, 3), match='aac'> + >>> pat.match('bbc') + <_sre.SRE_Match object; span=(0, 3), match='bbc'> + +(Contributed by Serhiy Storchaka in bpo-9179(1).) + +The number of capturing groups in regular expressions is no longer +limited to 100. (Contributed by Serhiy Storchaka in bpo-22437(2).) + +The *note sub(): 47b. and *note subn(): 734. functions now replace +unmatched groups with empty strings instead of raising an exception. +(Contributed by Serhiy Storchaka in bpo-1519638(3).) + +The *note re.error: 735. exceptions have new attributes, *note msg: 736, +*note pattern: 737, *note pos: 738, *note lineno: 739, and *note colno: +73a, that provide better context information about the error: + + >>> re.compile(""" + ... (?x) + ... .++ + ... """) + Traceback (most recent call last): + ... + sre_constants.error: multiple repeat at position 16 (line 3, column 7) + +(Contributed by Serhiy Storchaka in bpo-22578(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9179 + + (2) https://bugs.python.org/issue22437 + + (3) https://bugs.python.org/issue1519638 + + (4) https://bugs.python.org/issue22578 + + +File: python.info, Node: readline<2>, Next: selectors, Prev: re<4>, Up: Improved Modules<4> + +1.4.5.48 readline +................. + +A new *note append_history_file(): 73c. function can be used to append +the specified number of trailing elements in history to the given file. +(Contributed by Bruno Cauet in bpo-22940(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22940 + + +File: python.info, Node: selectors, Next: shutil<2>, Prev: readline<2>, Up: Improved Modules<4> + +1.4.5.49 selectors +.................. + +The new *note DevpollSelector: 44e. supports efficient ‘/dev/poll’ +polling on Solaris. (Contributed by Giampaolo Rodola’ in bpo-18931(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18931 + + +File: python.info, Node: shutil<2>, Next: signal<2>, Prev: selectors, Up: Improved Modules<4> + +1.4.5.50 shutil +............... + +The *note move(): 25b. function now accepts a `copy_function' argument, +allowing, for example, the *note copy(): 259. function to be used +instead of the default *note copy2(): 25a. if there is a need to ignore +file metadata when moving. (Contributed by Claudiu Popa in +bpo-19840(1).) + +The *note make_archive(): 21b. function now supports the `xztar' format. +(Contributed by Serhiy Storchaka in bpo-5411(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19840 + + (2) https://bugs.python.org/issue5411 + + +File: python.info, Node: signal<2>, Next: smtpd, Prev: shutil<2>, Up: Improved Modules<4> + +1.4.5.51 signal +............... + +On Windows, the *note set_wakeup_fd(): 3ce. function now also supports +socket handles. (Contributed by Victor Stinner in bpo-22018(1).) + +Various ‘SIG*’ constants in the *note signal: ea. module have been +converted into *note Enums: 7b. This allows meaningful names to be +printed during debugging, instead of integer “magic numbers”. +(Contributed by Giampaolo Rodola’ in bpo-21076(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22018 + + (2) https://bugs.python.org/issue21076 + + +File: python.info, Node: smtpd, Next: smtplib, Prev: signal<2>, Up: Improved Modules<4> + +1.4.5.52 smtpd +.............. + +Both the *note SMTPServer: 602. and *note SMTPChannel: 601. classes now +accept a `decode_data' keyword argument to determine if the ‘DATA’ +portion of the SMTP transaction is decoded using the ‘"utf-8"’ codec or +is instead provided to the *note SMTPServer.process_message(): 603. +method as a byte string. The default is ‘True’ for backward +compatibility reasons, but will change to ‘False’ in Python 3.6. If +`decode_data' is set to ‘False’, the ‘process_message’ method must be +prepared to accept keyword arguments. (Contributed by Maciej Szulik in +bpo-19662(1).) + +The *note SMTPServer: 602. class now advertises the ‘8BITMIME’ extension +( RFC 6152(2)) if `decode_data' has been set ‘True’. If the client +specifies ‘BODY=8BITMIME’ on the ‘MAIL’ command, it is passed to *note +SMTPServer.process_message(): 603. via the `mail_options' keyword. +(Contributed by Milan Oberkirch and R. David Murray in bpo-21795(3).) + +The *note SMTPServer: 602. class now also supports the ‘SMTPUTF8’ +extension ( RFC 6531(4): Internationalized Email). If the client +specified ‘SMTPUTF8 BODY=8BITMIME’ on the ‘MAIL’ command, they are +passed to *note SMTPServer.process_message(): 603. via the +`mail_options' keyword. It is the responsibility of the +‘process_message’ method to correctly handle the ‘SMTPUTF8’ data. +(Contributed by Milan Oberkirch in bpo-21725(5).) + +It is now possible to provide, directly or via name resolution, IPv6 +addresses in the *note SMTPServer: 602. constructor, and have it +successfully connect. (Contributed by Milan Oberkirch in bpo-14758(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19662 + + (2) https://tools.ietf.org/html/rfc6152.html + + (3) https://bugs.python.org/issue21795 + + (4) https://tools.ietf.org/html/rfc6531.html + + (5) https://bugs.python.org/issue21725 + + (6) https://bugs.python.org/issue14758 + + +File: python.info, Node: smtplib, Next: sndhdr, Prev: smtpd, Up: Improved Modules<4> + +1.4.5.53 smtplib +................ + +A new *note SMTP.auth(): 742. method provides a convenient way to +implement custom authentication mechanisms. (Contributed by Milan +Oberkirch in bpo-15014(1).) + +The *note SMTP.set_debuglevel(): 743. method now accepts an additional +debuglevel (2), which enables timestamps in debug messages. +(Contributed by Gavin Chappell and Maciej Szulik in bpo-16914(2).) + +Both the *note SMTP.sendmail(): 744. and *note SMTP.send_message(): 745. +methods now support RFC 6531(3) (SMTPUTF8). (Contributed by Milan +Oberkirch and R. David Murray in bpo-22027(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15014 + + (2) https://bugs.python.org/issue16914 + + (3) https://tools.ietf.org/html/rfc6531.html + + (4) https://bugs.python.org/issue22027 + + +File: python.info, Node: sndhdr, Next: socket<5>, Prev: smtplib, Up: Improved Modules<4> + +1.4.5.54 sndhdr +............... + +The *note what(): 747. and *note whathdr(): 748. functions now return a +*note namedtuple(): 1b7. (Contributed by Claudiu Popa in bpo-18615(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18615 + + +File: python.info, Node: socket<5>, Next: ssl<6>, Prev: sndhdr, Up: Improved Modules<4> + +1.4.5.55 socket +............... + +Functions with timeouts now use a monotonic clock, instead of a system +clock. (Contributed by Victor Stinner in bpo-22043(1).) + +A new *note socket.sendfile(): 74a. method allows sending a file over a +socket by using the high-performance *note os.sendfile(): 359. function +on UNIX, resulting in uploads being from 2 to 3 times faster than when +using plain *note socket.send(): 67a. (Contributed by Giampaolo Rodola’ +in bpo-17552(2).) + +The *note socket.sendall(): 67b. method no longer resets the socket +timeout every time bytes are received or sent. The socket timeout is +now the maximum total duration to send all data. (Contributed by Victor +Stinner in bpo-23853(3).) + +The `backlog' argument of the *note socket.listen(): 74b. method is now +optional. By default it is set to *note SOMAXCONN: 74c. or to ‘128’, +whichever is less. (Contributed by Charles-François Natali in +bpo-21455(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22043 + + (2) https://bugs.python.org/issue17552 + + (3) https://bugs.python.org/issue23853 + + (4) https://bugs.python.org/issue21455 + + +File: python.info, Node: ssl<6>, Next: sqlite3<3>, Prev: socket<5>, Up: Improved Modules<4> + +1.4.5.56 ssl +............ + +* Menu: + +* Memory BIO Support:: +* Application-Layer Protocol Negotiation Support:: +* Other Changes:: + + +File: python.info, Node: Memory BIO Support, Next: Application-Layer Protocol Negotiation Support, Up: ssl<6> + +1.4.5.57 Memory BIO Support +........................... + +(Contributed by Geert Jansen in bpo-21965(1).) + +The new *note SSLObject: 3e3. class has been added to provide SSL +protocol support for cases when the network I/O capabilities of *note +SSLSocket: 3e2. are not necessary or are suboptimal. ‘SSLObject’ +represents an SSL protocol instance, but does not implement any network +I/O methods, and instead provides a memory buffer interface. The new +*note MemoryBIO: 74f. class can be used to pass data between Python and +an SSL protocol instance. + +The memory BIO SSL support is primarily intended to be used in +frameworks implementing asynchronous I/O for which *note SSLSocket: +3e2.’s readiness model (“select/poll”) is inefficient. + +A new *note SSLContext.wrap_bio(): 3e6. method can be used to create a +new ‘SSLObject’ instance. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21965 + + +File: python.info, Node: Application-Layer Protocol Negotiation Support, Next: Other Changes, Prev: Memory BIO Support, Up: ssl<6> + +1.4.5.58 Application-Layer Protocol Negotiation Support +....................................................... + +(Contributed by Benjamin Peterson in bpo-20188(1).) + +Where OpenSSL support is present, the *note ssl: f3. module now +implements the `Application-Layer Protocol Negotiation' TLS extension as +described in RFC 7301(2). + +The new *note SSLContext.set_alpn_protocols(): 751. can be used to +specify which protocols a socket should advertise during the TLS +handshake. + +The new *note SSLSocket.selected_alpn_protocol(): 752. returns the +protocol that was selected during the TLS handshake. The *note +HAS_ALPN: 753. flag indicates whether ALPN support is present. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20188 + + (2) https://tools.ietf.org/html/rfc7301.html + + +File: python.info, Node: Other Changes, Prev: Application-Layer Protocol Negotiation Support, Up: ssl<6> + +1.4.5.59 Other Changes +...................... + +There is a new *note SSLSocket.version(): 755. method to query the +actual protocol version in use. (Contributed by Antoine Pitrou in +bpo-20421(1).) + +The *note SSLSocket: 3e2. class now implements a ‘SSLSocket.sendfile()’ +method. (Contributed by Giampaolo Rodola’ in bpo-17552(2).) + +The ‘SSLSocket.send()’ method now raises either the *note +ssl.SSLWantReadError: 756. or *note ssl.SSLWantWriteError: 757. +exception on a non-blocking socket if the operation would block. +Previously, it would return ‘0’. (Contributed by Nikolaus Rath in +bpo-20951(3).) + +The *note cert_time_to_seconds(): 758. function now interprets the input +time as UTC and not as local time, per RFC 5280(4). Additionally, the +return value is always an *note int: 184. (Contributed by Akira Li in +bpo-19940(5).) + +New ‘SSLObject.shared_ciphers()’ and *note SSLSocket.shared_ciphers(): +759. methods return the list of ciphers sent by the client during the +handshake. (Contributed by Benjamin Peterson in bpo-23186(6).) + +The *note SSLSocket.do_handshake(): 75a, *note SSLSocket.read(): 75b, +‘SSLSocket.shutdown()’, and *note SSLSocket.write(): 75c. methods of the +*note SSLSocket: 3e2. class no longer reset the socket timeout every +time bytes are received or sent. The socket timeout is now the maximum +total duration of the method. (Contributed by Victor Stinner in +bpo-23853(7).) + +The *note match_hostname(): 3dc. function now supports matching of IP +addresses. (Contributed by Antoine Pitrou in bpo-23239(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20421 + + (2) https://bugs.python.org/issue17552 + + (3) https://bugs.python.org/issue20951 + + (4) https://tools.ietf.org/html/rfc5280.html + + (5) https://bugs.python.org/issue19940 + + (6) https://bugs.python.org/issue23186 + + (7) https://bugs.python.org/issue23853 + + (8) https://bugs.python.org/issue23239 + + +File: python.info, Node: sqlite3<3>, Next: subprocess<3>, Prev: ssl<6>, Up: Improved Modules<4> + +1.4.5.60 sqlite3 +................ + +The *note Row: 75e. class now fully supports the sequence protocol, in +particular *note reversed(): 18e. iteration and slice indexing. +(Contributed by Claudiu Popa in bpo-10203(1); by Lucas Sinclair, Jessica +McKellar, and Serhiy Storchaka in bpo-13583(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10203 + + (2) https://bugs.python.org/issue13583 + + +File: python.info, Node: subprocess<3>, Next: sys<5>, Prev: sqlite3<3>, Up: Improved Modules<4> + +1.4.5.61 subprocess +................... + +The new *note run(): 3ed. function has been added. It runs the +specified command and returns a *note CompletedProcess: 760. object, +which describes a finished process. The new API is more consistent and +is the recommended approach to invoking subprocesses in Python code that +does not need to maintain compatibility with earlier Python versions. +(Contributed by Thomas Kluyver in bpo-23342(1).) + +Examples: + + >>> subprocess.run(["ls", "-l"]) # doesn't capture output + CompletedProcess(args=['ls', '-l'], returncode=0) + + >>> subprocess.run("exit 1", shell=True, check=True) + Traceback (most recent call last): + ... + subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 + + >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE) + CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, + stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n') + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23342 + + +File: python.info, Node: sys<5>, Next: sysconfig, Prev: subprocess<3>, Up: Improved Modules<4> + +1.4.5.62 sys +............ + +A new ‘set_coroutine_wrapper()’ function allows setting a global hook +that will be called whenever a *note coroutine object: 1a6. is created +by an *note async def: 284. function. A corresponding +‘get_coroutine_wrapper()’ can be used to obtain a currently set wrapper. +Both functions are *note provisional: 348, and are intended for +debugging purposes only. (Contributed by Yury Selivanov in +bpo-24017(1).) + +A new *note is_finalizing(): 762. function can be used to check if the +Python interpreter is *note shutting down: 763. (Contributed by Antoine +Pitrou in bpo-22696(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24017 + + (2) https://bugs.python.org/issue22696 + + +File: python.info, Node: sysconfig, Next: tarfile<2>, Prev: sys<5>, Up: Improved Modules<4> + +1.4.5.63 sysconfig +.................. + +The name of the user scripts directory on Windows now includes the first +two components of the Python version. (Contributed by Paul Moore in +bpo-23437(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23437 + + +File: python.info, Node: tarfile<2>, Next: threading<3>, Prev: sysconfig, Up: Improved Modules<4> + +1.4.5.64 tarfile +................ + +The `mode' argument of the *note open(): 766. function now accepts ‘"x"’ +to request exclusive creation. (Contributed by Berker Peksag in +bpo-21717(1).) + +The *note TarFile.extractall(): 767. and *note TarFile.extract(): 768. +methods now take a keyword argument `numeric_owner'. If set to ‘True’, +the extracted files and directories will be owned by the numeric ‘uid’ +and ‘gid’ from the tarfile. If set to ‘False’ (the default, and the +behavior in versions prior to 3.5), they will be owned by the named user +and group in the tarfile. (Contributed by Michael Vogt and Eric Smith +in bpo-23193(2).) + +The *note TarFile.list(): 769. now accepts an optional `members' keyword +argument that can be set to a subset of the list returned by *note +TarFile.getmembers(): 76a. (Contributed by Serhiy Storchaka in +bpo-21549(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21717 + + (2) https://bugs.python.org/issue23193 + + (3) https://bugs.python.org/issue21549 + + +File: python.info, Node: threading<3>, Next: time<4>, Prev: tarfile<2>, Up: Improved Modules<4> + +1.4.5.65 threading +.................. + +Both the *note Lock.acquire(): 76c. and *note RLock.acquire(): 76d. +methods now use a monotonic clock for timeout management. (Contributed +by Victor Stinner in bpo-22043(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22043 + + +File: python.info, Node: time<4>, Next: timeit<2>, Prev: threading<3>, Up: Improved Modules<4> + +1.4.5.66 time +............. + +The *note monotonic(): 76f. function is now always available. +(Contributed by Victor Stinner in bpo-22043(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22043 + + +File: python.info, Node: timeit<2>, Next: tkinter<5>, Prev: time<4>, Up: Improved Modules<4> + +1.4.5.67 timeit +............... + +A new command line option ‘-u’ or ‘--unit=`U'’ can be used to specify +the time unit for the timer output. Supported options are ‘usec’, +‘msec’, or ‘sec’. (Contributed by Julian Gindi in bpo-18983(1).) + +The *note timeit(): 771. function has a new `globals' parameter for +specifying the namespace in which the code will be running. +(Contributed by Ben Roberts in bpo-2527(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18983 + + (2) https://bugs.python.org/issue2527 + + +File: python.info, Node: tkinter<5>, Next: traceback<2>, Prev: timeit<2>, Up: Improved Modules<4> + +1.4.5.68 tkinter +................ + +The ‘tkinter._fix’ module used for setting up the Tcl/Tk environment on +Windows has been replaced by a private function in the ‘_tkinter’ module +which makes no permanent changes to environment variables. (Contributed +by Zachary Ware in bpo-20035(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20035 + + +File: python.info, Node: traceback<2>, Next: types<2>, Prev: tkinter<5>, Up: Improved Modules<4> + +1.4.5.69 traceback +.................. + +New *note walk_stack(): 774. and *note walk_tb(): 775. functions to +conveniently traverse frame and traceback objects. (Contributed by +Robert Collins in bpo-17911(1).) + +New lightweight classes: *note TracebackException: 776, *note +StackSummary: 777, and *note FrameSummary: 778. (Contributed by Robert +Collins in bpo-17911(2).) + +Both the *note print_tb(): 779. and *note print_stack(): 77a. functions +now support negative values for the `limit' argument. (Contributed by +Dmitry Kazakov in bpo-22619(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17911 + + (2) https://bugs.python.org/issue17911 + + (3) https://bugs.python.org/issue22619 + + +File: python.info, Node: types<2>, Next: unicodedata<4>, Prev: traceback<2>, Up: Improved Modules<4> + +1.4.5.70 types +.............. + +A new *note coroutine(): 77c. function to transform *note generator: +457. and *note generator-like: 6b5. objects into *note awaitables: 519. +(Contributed by Yury Selivanov in bpo-24017(1).) + +A new type called *note CoroutineType: 77d, which is used for *note +coroutine: 1a6. objects created by *note async def: 284. functions. +(Contributed by Yury Selivanov in bpo-24400(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue24017 + + (2) https://bugs.python.org/issue24400 + + +File: python.info, Node: unicodedata<4>, Next: unittest<3>, Prev: types<2>, Up: Improved Modules<4> + +1.4.5.71 unicodedata +.................... + +The *note unicodedata: 11a. module now uses data from Unicode 8.0.0(1). + + ---------- Footnotes ---------- + + (1) http://unicode.org/versions/Unicode8.0.0/ + + +File: python.info, Node: unittest<3>, Next: unittest mock<3>, Prev: unicodedata<4>, Up: Improved Modules<4> + +1.4.5.72 unittest +................. + +The *note TestLoader.loadTestsFromModule(): 780. method now accepts a +keyword-only argument `pattern' which is passed to ‘load_tests’ as the +third argument. Found packages are now checked for ‘load_tests’ +regardless of whether their path matches `pattern', because it is +impossible for a package name to match the default pattern. +(Contributed by Robert Collins and Barry A. Warsaw in bpo-16662(1).) + +Unittest discovery errors now are exposed in the *note +TestLoader.errors: 781. attribute of the *note TestLoader: 782. +instance. (Contributed by Robert Collins in bpo-19746(2).) + +A new command line option ‘--locals’ to show local variables in +tracebacks. (Contributed by Robert Collins in bpo-22936(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16662 + + (2) https://bugs.python.org/issue19746 + + (3) https://bugs.python.org/issue22936 + + +File: python.info, Node: unittest mock<3>, Next: urllib, Prev: unittest<3>, Up: Improved Modules<4> + +1.4.5.73 unittest.mock +...................... + +The *note Mock: 249. class has the following improvements: + + * The class constructor has a new `unsafe' parameter, which causes + mock objects to raise *note AttributeError: 39b. on attribute names + starting with ‘"assert"’. (Contributed by Kushal Das in + bpo-21238(1).) + + * A new *note Mock.assert_not_called(): 784. method to check if the + mock object was called. (Contributed by Kushal Das in + bpo-21262(2).) + +The *note MagicMock: 785. class now supports *note __truediv__(): 786, +*note __divmod__(): 787. and *note __matmul__(): 648. operators. +(Contributed by Johannes Baiter in bpo-20968(3), and Håkan Lövdahl in +bpo-23581(4) and bpo-23568(5).) + +It is no longer necessary to explicitly pass ‘create=True’ to the *note +patch(): 788. function when patching builtin names. (Contributed by +Kushal Das in bpo-17660(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21238 + + (2) https://bugs.python.org/issue21262 + + (3) https://bugs.python.org/issue20968 + + (4) https://bugs.python.org/issue23581 + + (5) https://bugs.python.org/issue23568 + + (6) https://bugs.python.org/issue17660 + + +File: python.info, Node: urllib, Next: wsgiref, Prev: unittest mock<3>, Up: Improved Modules<4> + +1.4.5.74 urllib +............... + +A new *note request.HTTPPasswordMgrWithPriorAuth: 78a. class allows HTTP +Basic Authentication credentials to be managed so as to eliminate +unnecessary ‘401’ response handling, or to unconditionally send +credentials on the first request in order to communicate with servers +that return a ‘404’ response instead of a ‘401’ if the ‘Authorization’ +header is not sent. (Contributed by Matej Cepl in bpo-19494(1) and +Akshit Khurana in bpo-7159(2).) + +A new `quote_via' argument for the *note parse.urlencode(): 78b. +function provides a way to control the encoding of query parts if +needed. (Contributed by Samwyse and Arnon Yaari in bpo-13866(3).) + +The *note request.urlopen(): 78c. function accepts an *note +ssl.SSLContext: 3e4. object as a `context' argument, which will be used +for the HTTPS connection. (Contributed by Alex Gaynor in bpo-22366(4).) + +The *note parse.urljoin(): 78d. was updated to use the RFC 3986(5) +semantics for the resolution of relative URLs, rather than RFC 1808(6) +and RFC 2396(7). (Contributed by Demian Brecht and Senthil Kumaran in +bpo-22118(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19494 + + (2) https://bugs.python.org/issue7159 + + (3) https://bugs.python.org/issue13866 + + (4) https://bugs.python.org/issue22366 + + (5) https://tools.ietf.org/html/rfc3986.html + + (6) https://tools.ietf.org/html/rfc1808.html + + (7) https://tools.ietf.org/html/rfc2396.html + + (8) https://bugs.python.org/issue22118 + + +File: python.info, Node: wsgiref, Next: xmlrpc<2>, Prev: urllib, Up: Improved Modules<4> + +1.4.5.75 wsgiref +................ + +The `headers' argument of the *note headers.Headers: 78f. class +constructor is now optional. (Contributed by Pablo Torres Navarrete and +SilentGhost in bpo-5800(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5800 + + +File: python.info, Node: xmlrpc<2>, Next: xml sax, Prev: wsgiref, Up: Improved Modules<4> + +1.4.5.76 xmlrpc +............... + +The *note client.ServerProxy: 255. class now supports the *note context +manager: 568. protocol. (Contributed by Claudiu Popa in bpo-20627(1).) + +The *note client.ServerProxy: 255. constructor now accepts an optional +*note ssl.SSLContext: 3e4. instance. (Contributed by Alex Gaynor in +bpo-22960(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue20627 + + (2) https://bugs.python.org/issue22960 + + +File: python.info, Node: xml sax, Next: zipfile<3>, Prev: xmlrpc<2>, Up: Improved Modules<4> + +1.4.5.77 xml.sax +................ + +SAX parsers now support a character stream of the *note +xmlreader.InputSource: 792. object. (Contributed by Serhiy Storchaka in +bpo-2175(1).) + +*note parseString(): 793. now accepts a *note str: 330. instance. +(Contributed by Serhiy Storchaka in bpo-10590(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2175 + + (2) https://bugs.python.org/issue10590 + + +File: python.info, Node: zipfile<3>, Prev: xml sax, Up: Improved Modules<4> + +1.4.5.78 zipfile +................ + +ZIP output can now be written to unseekable streams. (Contributed by +Serhiy Storchaka in bpo-23252(1).) + +The `mode' argument of *note ZipFile.open(): 5bc. method now accepts +‘"x"’ to request exclusive creation. (Contributed by Serhiy Storchaka +in bpo-21717(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23252 + + (2) https://bugs.python.org/issue21717 + + +File: python.info, Node: Other module-level changes, Next: Optimizations<4>, Prev: Improved Modules<4>, Up: What’s New In Python 3 5 + +1.4.6 Other module-level changes +-------------------------------- + +Many functions in the *note mmap: b3, *note ossaudiodev: c6, *note +socket: ef, *note ssl: f3, and *note codecs: 1c. modules now accept +writable *note bytes-like objects: 5e8. (Contributed by Serhiy +Storchaka in bpo-23001(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23001 + + +File: python.info, Node: Optimizations<4>, Next: Build and C API Changes<3>, Prev: Other module-level changes, Up: What’s New In Python 3 5 + +1.4.7 Optimizations +------------------- + +The *note os.walk(): 654. function has been sped up by 3 to 5 times on +POSIX systems, and by 7 to 20 times on Windows. This was done using the +new *note os.scandir(): 25f. function, which exposes file information +from the underlying ‘readdir’ or ‘FindFirstFile’/‘FindNextFile’ system +calls. (Contributed by Ben Hoyt with help from Victor Stinner in +bpo-23605(1).) + +Construction of ‘bytes(int)’ (filled by zero bytes) is faster and uses +less memory for large objects. ‘calloc()’ is used instead of ‘malloc()’ +to allocate memory for these objects. (Contributed by Victor Stinner in +bpo-21233(2).) + +Some operations on *note ipaddress: a2. *note IPv4Network: 3a0. and +*note IPv6Network: 39f. have been massively sped up, such as *note +subnets(): 797, *note supernet(): 798, *note summarize_address_range(): +799, *note collapse_addresses(): 79a. The speed up can range from 3 to +15 times. (Contributed by Antoine Pitrou, Michel Albert, and Markus in +bpo-21486(3), bpo-21487(4), bpo-20826(5), bpo-23266(6).) + +Pickling of *note ipaddress: a2. objects was optimized to produce +significantly smaller output. (Contributed by Serhiy Storchaka in +bpo-23133(7).) + +Many operations on *note io.BytesIO: 79b. are now 50% to 100% faster. +(Contributed by Serhiy Storchaka in bpo-15381(8) and David Wilson in +bpo-22003(9).) + +The *note marshal.dumps(): 79c. function is now faster: 65–85% with +versions 3 and 4, 20–25% with versions 0 to 2 on typical data, and up to +5 times in best cases. (Contributed by Serhiy Storchaka in +bpo-20416(10) and bpo-23344(11).) + +The UTF-32 encoder is now 3 to 7 times faster. (Contributed by Serhiy +Storchaka in bpo-15027(12).) + +Regular expressions are now parsed up to 10% faster. (Contributed by +Serhiy Storchaka in bpo-19380(13).) + +The *note json.dumps(): 605. function was optimized to run with +‘ensure_ascii=False’ as fast as with ‘ensure_ascii=True’. (Contributed +by Naoki Inada in bpo-23206(14).) + +The *note PyObject_IsInstance(): 79d. and *note PyObject_IsSubclass(): +79e. functions have been sped up in the common case that the second +argument has *note type: 608. as its metaclass. (Contributed Georg +Brandl by in bpo-22540(15).) + +Method caching was slightly improved, yielding up to 5% performance +improvement in some benchmarks. (Contributed by Antoine Pitrou in +bpo-22847(16).) + +Objects from the *note random: dc. module now use 50% less memory on +64-bit builds. (Contributed by Serhiy Storchaka in bpo-23488(17).) + +The *note property(): 1d7. getter calls are up to 25% faster. +(Contributed by Joe Jevnik in bpo-23910(18).) + +Instantiation of *note fractions.Fraction: 185. is now up to 30% faster. +(Contributed by Stefan Behnel in bpo-22464(19).) + +String methods *note find(): 79f, *note rfind(): 7a0, *note split(): +7a1, *note partition(): 7a2. and the *note in: 7a3. string operator are +now significantly faster for searching 1-character substrings. +(Contributed by Serhiy Storchaka in bpo-23573(20).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23605 + + (2) https://bugs.python.org/issue21233 + + (3) https://bugs.python.org/issue21486 + + (4) https://bugs.python.org/issue21487 + + (5) https://bugs.python.org/issue20826 + + (6) https://bugs.python.org/issue23266 + + (7) https://bugs.python.org/issue23133 + + (8) https://bugs.python.org/issue15381 + + (9) https://bugs.python.org/issue22003 + + (10) https://bugs.python.org/issue20416 + + (11) https://bugs.python.org/issue23344 + + (12) https://bugs.python.org/issue15027 + + (13) https://bugs.python.org/issue19380 + + (14) https://bugs.python.org/issue23206 + + (15) https://bugs.python.org/issue22540 + + (16) https://bugs.python.org/issue22847 + + (17) https://bugs.python.org/issue23488 + + (18) https://bugs.python.org/issue23910 + + (19) https://bugs.python.org/issue22464 + + (20) https://bugs.python.org/issue23573 + + +File: python.info, Node: Build and C API Changes<3>, Next: Deprecated<3>, Prev: Optimizations<4>, Up: What’s New In Python 3 5 + +1.4.8 Build and C API Changes +----------------------------- + +New ‘calloc’ functions were added: + + * *note PyMem_RawCalloc(): 7a5, + + * *note PyMem_Calloc(): 7a6, + + * *note PyObject_Calloc(): 7a7. + +(Contributed by Victor Stinner in bpo-21233(1).) + +New encoding/decoding helper functions: + + * *note Py_DecodeLocale(): 43c. (replaced ‘_Py_char2wchar()’), + + * *note Py_EncodeLocale(): 43d. (replaced ‘_Py_wchar2char()’). + +(Contributed by Victor Stinner in bpo-18395(2).) + +A new *note PyCodec_NameReplaceErrors(): 7a8. function to replace the +unicode encode error with ‘\N{...}’ escapes. (Contributed by Serhiy +Storchaka in bpo-19676(3).) + +A new *note PyErr_FormatV(): 7a9. function similar to *note +PyErr_Format(): 7aa, but accepts a ‘va_list’ argument. (Contributed by +Antoine Pitrou in bpo-18711(4).) + +A new ‘PyExc_RecursionError’ exception. (Contributed by Georg Brandl in +bpo-19235(5).) + +New *note PyModule_FromDefAndSpec(): 7ab, *note +PyModule_FromDefAndSpec2(): 7ac, and *note PyModule_ExecDef(): 7ad. +functions introduced by PEP 489(6) – multi-phase extension module +initialization. (Contributed by Petr Viktorin in bpo-24268(7).) + +New *note PyNumber_MatrixMultiply(): 7ae. and *note +PyNumber_InPlaceMatrixMultiply(): 7af. functions to perform matrix +multiplication. (Contributed by Benjamin Peterson in bpo-21176(8). See +also PEP 465(9) for details.) + +The *note PyTypeObject.tp_finalize: 2d7. slot is now part of the stable +ABI. + +Windows builds now require Microsoft Visual C++ 14.0, which is available +as part of Visual Studio 2015(10). + +Extension modules now include a platform information tag in their +filename on some platforms (the tag is optional, and CPython will import +extensions without it, although if the tag is present and mismatched, +the extension won’t be loaded): + + * On Linux, extension module filenames end with + ‘.cpython-m--.pyd’: + + * ‘’ is the major number of the Python version; for + Python 3.5 this is ‘3’. + + * ‘’ is the minor number of the Python version; for + Python 3.5 this is ‘5’. + + * ‘’ is the hardware architecture the extension + module was built to run on. It’s most commonly either ‘i386’ + for 32-bit Intel platforms or ‘x86_64’ for 64-bit Intel (and + AMD) platforms. + + * ‘’ is always ‘linux-gnu’, except for extensions built to + talk to the 32-bit ABI on 64-bit platforms, in which case it + is ‘linux-gnu32’ (and ‘’ will be ‘x86_64’). + + * On Windows, extension module filenames end with + ‘.cp-.pyd’: + + * ‘’ is the major number of the Python version; for + Python 3.5 this is ‘3’. + + * ‘’ is the minor number of the Python version; for + Python 3.5 this is ‘5’. + + * ‘’ is the platform the extension module was built + for, either ‘win32’ for Win32, ‘win_amd64’ for Win64, + ‘win_ia64’ for Windows Itanium 64, and ‘win_arm’ for Windows + on ARM. + + * If built in debug mode, ‘’ will be ‘_d’, otherwise it + will be blank. + + * On OS X platforms, extension module filenames now end with + ‘-darwin.so’. + + * On all other platforms, extension module filenames are the same as + they were with Python 3.4. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue21233 + + (2) https://bugs.python.org/issue18395 + + (3) https://bugs.python.org/issue19676 + + (4) https://bugs.python.org/issue18711 + + (5) https://bugs.python.org/issue19235 + + (6) https://www.python.org/dev/peps/pep-0489 + + (7) https://bugs.python.org/issue24268 + + (8) https://bugs.python.org/issue21176 + + (9) https://www.python.org/dev/peps/pep-0465 + + (10) https://www.visualstudio.com/ + + +File: python.info, Node: Deprecated<3>, Next: Removed<2>, Prev: Build and C API Changes<3>, Up: What’s New In Python 3 5 + +1.4.9 Deprecated +---------------- + +* Menu: + +* New Keywords: New Keywords<2>. +* Deprecated Python Behavior: Deprecated Python Behavior<2>. +* Unsupported Operating Systems:: +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods<3>. + + +File: python.info, Node: New Keywords<2>, Next: Deprecated Python Behavior<2>, Up: Deprecated<3> + +1.4.9.1 New Keywords +.................... + +‘async’ and ‘await’ are not recommended to be used as variable, class, +function or module names. Introduced by PEP 492(1) in Python 3.5, they +will become proper keywords in Python 3.7. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0492 + + +File: python.info, Node: Deprecated Python Behavior<2>, Next: Unsupported Operating Systems, Prev: New Keywords<2>, Up: Deprecated<3> + +1.4.9.2 Deprecated Python Behavior +.................................. + +Raising the *note StopIteration: 486. exception inside a generator will +now generate a silent *note PendingDeprecationWarning: 279, which will +become a non-silent deprecation warning in Python 3.6 and will trigger a +*note RuntimeError: 2ba. in Python 3.7. See *note PEP 479; Change +StopIteration handling inside generators: 5d7. for details. + + +File: python.info, Node: Unsupported Operating Systems, Next: Deprecated Python modules functions and methods<3>, Prev: Deprecated Python Behavior<2>, Up: Deprecated<3> + +1.4.9.3 Unsupported Operating Systems +..................................... + +Windows XP is no longer supported by Microsoft, thus, per PEP 11(1), +CPython 3.5 is no longer officially supported on this OS. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0011 + + +File: python.info, Node: Deprecated Python modules functions and methods<3>, Prev: Unsupported Operating Systems, Up: Deprecated<3> + +1.4.9.4 Deprecated Python modules, functions and methods +........................................................ + +The *note formatter: 82. module has now graduated to full deprecation +and is still slated for removal in Python 3.6. + +The ‘asyncio.async()’ function is deprecated in favor of *note +ensure_future(): 517. + +The *note smtpd: ec. module has in the past always decoded the DATA +portion of email messages using the ‘utf-8’ codec. This can now be +controlled by the new `decode_data' keyword to *note SMTPServer: 602. +The default value is ‘True’, but this default is deprecated. Specify +the `decode_data' keyword with an appropriate value to avoid the +deprecation warning. + +Directly assigning values to the *note key: 48d, *note value: 48e. and +*note coded_value: 48f. of *note http.cookies.Morsel: 490. objects is +deprecated. Use the *note set(): 491. method instead. In addition, the +undocumented `LegalChars' parameter of *note set(): 491. is deprecated, +and is now ignored. + +Passing a format string as keyword argument `format_string' to the *note +format(): 48c. method of the *note string.Formatter: 7b5. class has been +deprecated. (Contributed by Serhiy Storchaka in bpo-23671(1).) + +The ‘platform.dist()’ and ‘platform.linux_distribution()’ functions are +now deprecated. Linux distributions use too many different ways of +describing themselves, so the functionality is left to a package. +(Contributed by Vajrasky Kok and Berker Peksag in bpo-1322(2).) + +The previously undocumented ‘from_function’ and ‘from_builtin’ methods +of *note inspect.Signature: 6f3. are deprecated. Use the new *note +Signature.from_callable(): 6f6. method instead. (Contributed by Yury +Selivanov in bpo-24248(3).) + +The *note inspect.getargspec(): 55c. function is deprecated and +scheduled to be removed in Python 3.6. (See bpo-20438(4) for details.) + +The *note inspect: a0. *note getfullargspec(): 55d, *note getcallargs(): +7b6, and *note formatargspec(): 7b7. functions are deprecated in favor +of the *note inspect.signature(): 55b. API. (Contributed by Yury +Selivanov in bpo-20438(5).) + +*note getargvalues(): 7b8. and *note formatargvalues(): 7b9. functions +were inadvertently marked as deprecated with the release of Python +3.5.0. + +Use of *note re.LOCALE: 3c9. flag with str patterns or *note re.ASCII: +3c8. is now deprecated. (Contributed by Serhiy Storchaka in +bpo-22407(6).) + +Use of unrecognized special sequences consisting of ‘'\'’ and an ASCII +letter in regular expression patterns and replacement patterns now +raises a deprecation warning and will be forbidden in Python 3.6. +(Contributed by Serhiy Storchaka in bpo-23622(7).) + +The undocumented and unofficial `use_load_tests' default argument of the +*note unittest.TestLoader.loadTestsFromModule(): 780. method now is +deprecated and ignored. (Contributed by Robert Collins and Barry A. +Warsaw in bpo-16662(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23671 + + (2) https://bugs.python.org/issue1322 + + (3) https://bugs.python.org/issue24248 + + (4) https://bugs.python.org/issue20438 + + (5) https://bugs.python.org/issue20438 + + (6) https://bugs.python.org/issue22407 + + (7) https://bugs.python.org/issue23622 + + (8) https://bugs.python.org/issue16662 + + +File: python.info, Node: Removed<2>, Next: Porting to Python 3 5, Prev: Deprecated<3>, Up: What’s New In Python 3 5 + +1.4.10 Removed +-------------- + +* Menu: + +* API and Feature Removals: API and Feature Removals<4>. + + +File: python.info, Node: API and Feature Removals<4>, Up: Removed<2> + +1.4.10.1 API and Feature Removals +................................. + +The following obsolete and previously deprecated APIs and features have +been removed: + + * The ‘__version__’ attribute has been dropped from the email + package. The email code hasn’t been shipped separately from the + stdlib for a long time, and the ‘__version__’ string was not + updated in the last few releases. + + * The internal ‘Netrc’ class in the *note ftplib: 84. module was + deprecated in 3.4, and has now been removed. (Contributed by Matt + Chaput in bpo-6623(1).) + + * The concept of ‘.pyo’ files has been removed. + + * The JoinableQueue class in the provisional *note asyncio: a. module + was deprecated in 3.4.4 and is now removed. (Contributed by A. + Jesse Jiryu Davis in bpo-23464(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6623 + + (2) https://bugs.python.org/issue23464 + + +File: python.info, Node: Porting to Python 3 5, Next: Notable changes in Python 3 5 4, Prev: Removed<2>, Up: What’s New In Python 3 5 + +1.4.11 Porting to Python 3.5 +---------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code. + +* Menu: + +* Changes in Python behavior: Changes in Python behavior<2>. +* Changes in the Python API: Changes in the Python API<4>. +* Changes in the C API: Changes in the C API<4>. + + +File: python.info, Node: Changes in Python behavior<2>, Next: Changes in the Python API<4>, Up: Porting to Python 3 5 + +1.4.11.1 Changes in Python behavior +................................... + + * Due to an oversight, earlier Python versions erroneously accepted + the following syntax: + + f(1 for x in [1], *args) + f(1 for x in [1], **kwargs) + + Python 3.5 now correctly raises a *note SyntaxError: 458, as + generator expressions must be put in parentheses if not a sole + argument to a function. + + +File: python.info, Node: Changes in the Python API<4>, Next: Changes in the C API<4>, Prev: Changes in Python behavior<2>, Up: Porting to Python 3 5 + +1.4.11.2 Changes in the Python API +.................................. + + * PEP 475(1): System calls are now retried when interrupted by a + signal instead of raising *note InterruptedError: 659. if the + Python signal handler does not raise an exception. + + * Before Python 3.5, a *note datetime.time: 4f3. object was + considered to be false if it represented midnight in UTC. This + behavior was considered obscure and error-prone and has been + removed in Python 3.5. See bpo-13936(2) for full details. + + * The ‘ssl.SSLSocket.send()’ method now raises either *note + ssl.SSLWantReadError: 756. or *note ssl.SSLWantWriteError: 757. on + a non-blocking socket if the operation would block. Previously, it + would return ‘0’. (Contributed by Nikolaus Rath in bpo-20951(3).) + + * The ‘__name__’ attribute of generators is now set from the function + name, instead of being set from the code name. Use + ‘gen.gi_code.co_name’ to retrieve the code name. Generators also + have a new ‘__qualname__’ attribute, the qualified name, which is + now used for the representation of a generator (‘repr(gen)’). + (Contributed by Victor Stinner in bpo-21205(4).) + + * The deprecated “strict” mode and argument of *note HTMLParser: 7bf, + ‘HTMLParser.error()’, and the ‘HTMLParserError’ exception have been + removed. (Contributed by Ezio Melotti in bpo-15114(5).) The + `convert_charrefs' argument of *note HTMLParser: 7bf. is now ‘True’ + by default. (Contributed by Berker Peksag in bpo-21047(6).) + + * Although it is not formally part of the API, it is worth noting for + porting purposes (ie: fixing tests) that error messages that were + previously of the form “‘sometype’ does not support the buffer + protocol” are now of the form “a *note bytes-like object: 5e8. is + required, not ‘sometype’”. (Contributed by Ezio Melotti in + bpo-16518(7).) + + * If the current directory is set to a directory that no longer + exists then *note FileNotFoundError: 7c0. will no longer be raised + and instead *note find_spec(): 7c1. will return ‘None’ `without' + caching ‘None’ in *note sys.path_importer_cache: 49d, which is + different than the typical case (bpo-22834(8)). + + * HTTP status code and messages from *note http.client: 94. and *note + http.server: 97. were refactored into a common *note HTTPStatus: + 6e2. enum. The values in *note http.client: 94. and *note + http.server: 97. remain available for backwards compatibility. + (Contributed by Demian Brecht in bpo-21793(9).) + + * When an import loader defines + ‘importlib.machinery.Loader.exec_module()’ it is now expected to + also define ‘create_module()’ (raises a *note DeprecationWarning: + 278. now, will be an error in Python 3.6). If the loader inherits + from *note importlib.abc.Loader: 7c2. then there is nothing to do, + else simply define ‘create_module()’ to return ‘None’. + (Contributed by Brett Cannon in bpo-23014(10).) + + * The *note re.split(): 3ca. function always ignored empty pattern + matches, so the ‘"x*"’ pattern worked the same as ‘"x+"’, and the + ‘"\b"’ pattern never worked. Now *note re.split(): 3ca. raises a + warning if the pattern could match an empty string. For + compatibility, use patterns that never match an empty string (e.g. + ‘"x+"’ instead of ‘"x*"’). Patterns that could only match an empty + string (such as ‘"\b"’) now raise an error. (Contributed by Serhiy + Storchaka in bpo-22818(11).) + + * The *note http.cookies.Morsel: 490. dict-like interface has been + made self consistent: morsel comparison now takes the *note key: + 48d. and *note value: 48e. into account, *note copy(): 7c3. now + results in a *note Morsel: 490. instance rather than a *note dict: + 1b8, and *note update(): 7c4. will now raise an exception if any of + the keys in the update dictionary are invalid. In addition, the + undocumented `LegalChars' parameter of *note set(): 491. is + deprecated and is now ignored. (Contributed by Demian Brecht in + bpo-2211(12).) + + * PEP 488(13) has removed ‘.pyo’ files from Python and introduced the + optional ‘opt-’ tag in ‘.pyc’ file names. The *note + importlib.util.cache_from_source(): 557. has gained an + `optimization' parameter to help control the ‘opt-’ tag. Because + of this, the `debug_override' parameter of the function is now + deprecated. ‘.pyo’ files are also no longer supported as a file + argument to the Python interpreter and thus serve no purpose when + distributed on their own (i.e. sourceless code distribution). Due + to the fact that the magic number for bytecode has changed in + Python 3.5, all old ‘.pyo’ files from previous versions of Python + are invalid regardless of this PEP. + + * The *note socket: ef. module now exports the *note + CAN_RAW_FD_FRAMES: 7c5. constant on linux 3.6 and greater. + + * The *note ssl.cert_time_to_seconds(): 758. function now interprets + the input time as UTC and not as local time, per RFC 5280(14). + Additionally, the return value is always an *note int: 184. + (Contributed by Akira Li in bpo-19940(15).) + + * The ‘pygettext.py’ Tool now uses the standard +NNNN format for + timezones in the POT-Creation-Date header. + + * The *note smtplib: ed. module now uses *note sys.stderr: 30f. + instead of the previous module-level ‘stderr’ variable for debug + output. If your (test) program depends on patching the + module-level variable to capture the debug output, you will need to + update it to capture sys.stderr instead. + + * The *note str.startswith(): 7c6. and *note str.endswith(): 7c7. + methods no longer return ‘True’ when finding the empty string and + the indexes are completely out of range. (Contributed by Serhiy + Storchaka in bpo-24284(16).) + + * The *note inspect.getdoc(): 1d6. function now returns documentation + strings inherited from base classes. Documentation strings no + longer need to be duplicated if the inherited documentation is + appropriate. To suppress an inherited string, an empty string must + be specified (or the documentation may be filled in). This change + affects the output of the *note pydoc: d9. module and the *note + help(): 572. function. (Contributed by Serhiy Storchaka in + bpo-15582(17).) + + * Nested *note functools.partial(): 7c8. calls are now flattened. If + you were relying on the previous behavior, you can now either add + an attribute to a *note functools.partial(): 7c8. object or you can + create a subclass of *note functools.partial(): 7c8. (Contributed + by Alexander Belopolsky in bpo-7830(18).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0475 + + (2) https://bugs.python.org/issue13936 + + (3) https://bugs.python.org/issue20951 + + (4) https://bugs.python.org/issue21205 + + (5) https://bugs.python.org/issue15114 + + (6) https://bugs.python.org/issue21047 + + (7) https://bugs.python.org/issue16518 + + (8) https://bugs.python.org/issue22834 + + (9) https://bugs.python.org/issue21793 + + (10) https://bugs.python.org/issue23014 + + (11) https://bugs.python.org/issue22818 + + (12) https://bugs.python.org/issue2211 + + (13) https://www.python.org/dev/peps/pep-0488 + + (14) https://tools.ietf.org/html/rfc5280.html + + (15) https://bugs.python.org/issue19940 + + (16) https://bugs.python.org/issue24284 + + (17) https://bugs.python.org/issue15582 + + (18) https://bugs.python.org/issue7830 + + +File: python.info, Node: Changes in the C API<4>, Prev: Changes in the Python API<4>, Up: Porting to Python 3 5 + +1.4.11.3 Changes in the C API +............................. + + * The undocumented ‘format’ member of the (non-public) + ‘PyMemoryViewObject’ structure has been removed. All extensions + relying on the relevant parts in ‘memoryobject.h’ must be rebuilt. + + * The ‘PyMemAllocator’ structure was renamed to *note + PyMemAllocatorEx: 7ca. and a new ‘calloc’ field was added. + + * Removed non-documented macro ‘PyObject_REPR’ which leaked + references. Use format character ‘%R’ in *note + PyUnicode_FromFormat(): 7cb.-like functions to format the *note + repr(): 7cc. of the object. (Contributed by Serhiy Storchaka in + bpo-22453(1).) + + * Because the lack of the ‘__module__’ attribute breaks pickling and + introspection, a deprecation warning is now raised for builtin + types without the ‘__module__’ attribute. This would be an + AttributeError in the future. (Contributed by Serhiy Storchaka in + bpo-20204(2).) + + * As part of the PEP 492(3) implementation, the ‘tp_reserved’ slot of + *note PyTypeObject: 2d6. was replaced with a ‘tp_as_async’ slot. + Refer to *note Coroutine Objects: 7cd. for new types, structures + and functions. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue22453 + + (2) https://bugs.python.org/issue20204 + + (3) https://www.python.org/dev/peps/pep-0492 + + +File: python.info, Node: Notable changes in Python 3 5 4, Prev: Porting to Python 3 5, Up: What’s New In Python 3 5 + +1.4.12 Notable changes in Python 3.5.4 +-------------------------------------- + +* Menu: + +* New make regen-all build target: New make regen-all build target<2>. +* Removal of make touch build target: Removal of make touch build target<2>. + + +File: python.info, Node: New make regen-all build target<2>, Next: Removal of make touch build target<2>, Up: Notable changes in Python 3 5 4 + +1.4.12.1 New ‘make regen-all’ build target +.......................................... + +To simplify cross-compilation, and to ensure that CPython can reliably +be compiled without requiring an existing version of Python to already +be available, the autotools-based build system no longer attempts to +implicitly recompile generated files based on file modification times. + +Instead, a new ‘make regen-all’ command has been added to force +regeneration of these files when desired (e.g. after an initial version +of Python has already been built based on the pregenerated versions). + +More selective regeneration targets are also defined - see +Makefile.pre.in(1) for details. + +(Contributed by Victor Stinner in bpo-23404(2).) + +New in version 3.5.4. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Makefile.pre.in + + (2) https://bugs.python.org/issue23404 + + +File: python.info, Node: Removal of make touch build target<2>, Prev: New make regen-all build target<2>, Up: Notable changes in Python 3 5 4 + +1.4.12.2 Removal of ‘make touch’ build target +............................................. + +The ‘make touch’ build target previously used to request implicit +regeneration of generated files by updating their modification times has +been removed. + +It has been replaced by the new ‘make regen-all’ target. + +(Contributed by Victor Stinner in bpo-23404(1).) + +Changed in version 3.5.4. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23404 + + +File: python.info, Node: What’s New In Python 3 4, Next: What’s New In Python 3 3, Prev: What’s New In Python 3 5, Up: What’s New in Python + +1.5 What’s New In Python 3.4 +============================ + + +Author: R. David Murray <> (Editor) + +This article explains the new features in Python 3.4, compared to 3.3. +Python 3.4 was released on March 16, 2014. For full details, see the +changelog(1). + +See also +........ + +PEP 429(2) – Python 3.4 Release Schedule + +* Menu: + +* Summary – Release Highlights: Summary – Release Highlights<2>. +* New Features: New Features<5>. +* New Modules: New Modules<5>. +* Improved Modules: Improved Modules<5>. +* CPython Implementation Changes:: +* Deprecated: Deprecated<4>. +* Removed: Removed<3>. +* Porting to Python 3.4: Porting to Python 3 4. +* Changed in 3.4.3: Changed in 3 4 3. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3.4/whatsnew/changelog.html + + (2) https://www.python.org/dev/peps/pep-0429 + + +File: python.info, Node: Summary – Release Highlights<2>, Next: New Features<5>, Up: What’s New In Python 3 4 + +1.5.1 Summary – Release Highlights +---------------------------------- + +New syntax features: + + * No new syntax features were added in Python 3.4. + +Other new features: + + * *note pip should always be available: 7d4. ( PEP 453(1)). + + * *note Newly created file descriptors are non-inheritable: 7d5. ( + PEP 446(2)). + + * command line option for *note isolated mode: 7d6. (bpo-16499(3)). + + * *note improvements in the handling of codecs: 7d7. that are not + text encodings (multiple issues). + + * *note A ModuleSpec Type: 7d8. for the Import System ( PEP 451(4)). + (Affects importer authors.) + + * The *note marshal: b0. format has been made *note more compact and + efficient: 7d9. (bpo-16475(5)). + +New library modules: + + * *note asyncio: a.: *note New provisional API for asynchronous IO: + 7da. ( PEP 3156(6)). + + * *note ensurepip: 7a.: *note Bootstrapping the pip installer: 7db. ( + PEP 453(7)). + + * *note enum: 7b.: *note Support for enumeration types: 7dc. ( PEP + 435(8)). + + * *note pathlib: c8.: *note Object-oriented filesystem paths: 7dd. ( + PEP 428(9)). + + * *note selectors: e6.: *note High-level and efficient I/O + multiplexing: 7de, built upon the *note select: e5. module + primitives (part of PEP 3156(10)). + + * *note statistics: f5.: A basic *note numerically stable statistics + library: 7df. ( PEP 450(11)). + + * *note tracemalloc: 114.: *note Trace Python memory allocations: + 7e0. ( PEP 454(12)). + +Significantly improved library modules: + + * *note Single-dispatch generic functions: 7e1. in *note functools: + 85. ( PEP 443(13)). + + * New *note pickle: ca. *note protocol 4: 7e2. ( PEP 3154(14)). + + * *note multiprocessing: b7. now has *note an option to avoid using + os.fork on Unix: 7e3. (bpo-8713(15)). + + * *note email: 69. has a new submodule, *note contentmanager: 6b, and + a new *note Message: 541. subclass (‘EmailMessage’) that *note + simplify MIME handling: 7e4. (bpo-18891(16)). + + * The *note inspect: a0. and *note pydoc: d9. modules are now capable + of correct introspection of a much wider variety of callable + objects, which improves the output of the Python *note help(): 572. + system. + + * The *note ipaddress: a2. module API has been declared stable + +Security improvements: + + * *note Secure and interchangeable hash algorithm: 7e5. ( PEP + 456(17)). + + * *note Make newly created file descriptors non-inheritable: 7d5. ( + PEP 446(18)) to avoid leaking file descriptors to child processes. + + * New command line option for *note isolated mode: 7d6, + (bpo-16499(19)). + + * *note multiprocessing: b7. now has *note an option to avoid using + os.fork on Unix: 7e3. `spawn' and `forkserver' are more secure + because they avoid sharing data with child processes. + + * *note multiprocessing: b7. child processes on Windows no longer + inherit all of the parent’s inheritable handles, only the necessary + ones. + + * A new *note hashlib.pbkdf2_hmac(): 7e6. function provides the + PKCS#5 password-based key derivation function 2(20). + + * *note TLSv1.1 and TLSv1.2 support: 7e7. for *note ssl: f3. + + * *note Retrieving certificates from the Windows system cert store + support: 7e8. for *note ssl: f3. + + * *note Server-side SNI (Server Name Indication) support: 7e9. for + *note ssl: f3. + + * The *note ssl.SSLContext: 3e4. class has a *note lot of + improvements: 7ea. + + * All modules in the standard library that support SSL now support + server certificate verification, including hostname matching (*note + ssl.match_hostname(): 3dc.) and CRLs (Certificate Revocation lists, + see *note ssl.SSLContext.load_verify_locations(): 7eb.). + +CPython implementation improvements: + + * *note Safe object finalization: 7ec. ( PEP 442(21)). + + * Leveraging PEP 442(22), in most cases *note module globals are no + longer set to None during finalization: 7ec. (bpo-18214(23)). + + * *note Configurable memory allocators: 7ed. ( PEP 445(24)). + + * *note Argument Clinic: 7ee. ( PEP 436(25)). + +Please read on for a comprehensive list of user-facing changes, +including many other smaller improvements, CPython optimizations, +deprecations, and potential porting issues. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0453 + + (2) https://www.python.org/dev/peps/pep-0446 + + (3) https://bugs.python.org/issue16499 + + (4) https://www.python.org/dev/peps/pep-0451 + + (5) https://bugs.python.org/issue16475 + + (6) https://www.python.org/dev/peps/pep-3156 + + (7) https://www.python.org/dev/peps/pep-0453 + + (8) https://www.python.org/dev/peps/pep-0435 + + (9) https://www.python.org/dev/peps/pep-0428 + + (10) https://www.python.org/dev/peps/pep-3156 + + (11) https://www.python.org/dev/peps/pep-0450 + + (12) https://www.python.org/dev/peps/pep-0454 + + (13) https://www.python.org/dev/peps/pep-0443 + + (14) https://www.python.org/dev/peps/pep-3154 + + (15) https://bugs.python.org/issue8713 + + (16) https://bugs.python.org/issue18891 + + (17) https://www.python.org/dev/peps/pep-0456 + + (18) https://www.python.org/dev/peps/pep-0446 + + (19) https://bugs.python.org/issue16499 + + (20) https://en.wikipedia.org/wiki/PBKDF2 + + (21) https://www.python.org/dev/peps/pep-0442 + + (22) https://www.python.org/dev/peps/pep-0442 + + (23) https://bugs.python.org/issue18214 + + (24) https://www.python.org/dev/peps/pep-0445 + + (25) https://www.python.org/dev/peps/pep-0436 + + +File: python.info, Node: New Features<5>, Next: New Modules<5>, Prev: Summary – Release Highlights<2>, Up: What’s New In Python 3 4 + +1.5.2 New Features +------------------ + +* Menu: + +* PEP 453; Explicit Bootstrapping of PIP in Python Installations: PEP 453 Explicit Bootstrapping of PIP in Python Installations. +* PEP 446; Newly Created File Descriptors Are Non-Inheritable: PEP 446 Newly Created File Descriptors Are Non-Inheritable. +* Improvements to Codec Handling:: +* PEP 451; A ModuleSpec Type for the Import System: PEP 451 A ModuleSpec Type for the Import System. +* Other Language Changes: Other Language Changes<5>. + + +File: python.info, Node: PEP 453 Explicit Bootstrapping of PIP in Python Installations, Next: PEP 446 Newly Created File Descriptors Are Non-Inheritable, Up: New Features<5> + +1.5.2.1 PEP 453: Explicit Bootstrapping of PIP in Python Installations +...................................................................... + +* Menu: + +* Bootstrapping pip By Default:: +* Documentation Changes:: + + +File: python.info, Node: Bootstrapping pip By Default, Next: Documentation Changes, Up: PEP 453 Explicit Bootstrapping of PIP in Python Installations + +1.5.2.2 Bootstrapping pip By Default +.................................... + +The new *note ensurepip: 7a. module (defined in PEP 453(1)) provides a +standard cross-platform mechanism to bootstrap the pip installer into +Python installations and virtual environments. The version of ‘pip’ +included with Python 3.4.0 is ‘pip’ 1.5.4, and future 3.4.x maintenance +releases will update the bundled version to the latest version of ‘pip’ +that is available at the time of creating the release candidate. + +By default, the commands ‘pipX’ and ‘pipX.Y’ will be installed on all +platforms (where X.Y stands for the version of the Python installation), +along with the ‘pip’ Python package and its dependencies. On Windows +and in virtual environments on all platforms, the unversioned ‘pip’ +command will also be installed. On other platforms, the system wide +unversioned ‘pip’ command typically refers to the separately installed +Python 2 version. + +The ‘pyvenv’ command line utility and the *note venv: 125. module make +use of the *note ensurepip: 7a. module to make ‘pip’ readily available +in virtual environments. When using the command line utility, ‘pip’ is +installed by default, while when using the *note venv: 125. module *note +API: 7f2. installation of ‘pip’ must be requested explicitly. + +For CPython *note source builds on POSIX systems: 7f3, the ‘make +install’ and ‘make altinstall’ commands bootstrap ‘pip’ by default. +This behaviour can be controlled through configure options, and +overridden through Makefile options. + +On Windows and Mac OS X, the CPython installers now default to +installing ‘pip’ along with CPython itself (users may opt out of +installing it during the installation process). Window users will need +to opt in to the automatic ‘PATH’ modifications to have ‘pip’ available +from the command line by default, otherwise it can still be accessed +through the Python launcher for Windows as ‘py -m pip’. + +As discussed in the PEP(2), platform packagers may choose not to install +these commands by default, as long as, when invoked, they provide clear +and simple directions on how to install them on that platform (usually +using the system package manager). + + Note: To avoid conflicts between parallel Python 2 and Python 3 + installations, only the versioned ‘pip3’ and ‘pip3.4’ commands are + bootstrapped by default when ‘ensurepip’ is invoked directly - the + ‘--default-pip’ option is needed to also request the unversioned + ‘pip’ command. ‘pyvenv’ and the Windows installer ensure that the + unqualified ‘pip’ command is made available in those environments, + and ‘pip’ can always be invoked via the ‘-m’ switch rather than + directly to avoid ambiguity on systems with multiple Python + installations. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0453 + + (2) +https://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors + + +File: python.info, Node: Documentation Changes, Prev: Bootstrapping pip By Default, Up: PEP 453 Explicit Bootstrapping of PIP in Python Installations + +1.5.2.3 Documentation Changes +............................. + +As part of this change, the *note Installing Python Modules: 7f5. and +*note Distributing Python Modules: 7f6. sections of the documentation +have been completely redesigned as short getting started and FAQ +documents. Most packaging documentation has now been moved out to the +Python Packaging Authority maintained Python Packaging User Guide(1) and +the documentation of the individual projects. + +However, as this migration is currently still incomplete, the legacy +versions of those guides remaining available as *note Installing Python +Modules (Legacy version): 7f7. and *note Distributing Python Modules +(Legacy version): 7f8. + +See also +........ + +PEP 453(2) – Explicit bootstrapping of pip in Python installations + + PEP written by Donald Stufft and Nick Coghlan, implemented by + Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily. + + ---------- Footnotes ---------- + + (1) https://packaging.python.org + + (2) https://www.python.org/dev/peps/pep-0453 + + +File: python.info, Node: PEP 446 Newly Created File Descriptors Are Non-Inheritable, Next: Improvements to Codec Handling, Prev: PEP 453 Explicit Bootstrapping of PIP in Python Installations, Up: New Features<5> + +1.5.2.4 PEP 446: Newly Created File Descriptors Are Non-Inheritable +................................................................... + +PEP 446(1) makes newly created file descriptors *note non-inheritable: +7fa. In general, this is the behavior an application will want: when +launching a new process, having currently open files also open in the +new process can lead to all sorts of hard to find bugs, and potentially +to security issues. + +However, there are occasions when inheritance is desired. To support +these cases, the following new functions and methods are available: + + * *note os.get_inheritable(): 7fb, *note os.set_inheritable(): 7fc. + + * *note os.get_handle_inheritable(): 7fd, *note + os.set_handle_inheritable(): 7fe. + + * *note socket.socket.get_inheritable(): 7ff, *note + socket.socket.set_inheritable(): 800. + +See also +........ + +PEP 446(2) – Make newly created file descriptors non-inheritable + + PEP written and implemented by Victor Stinner. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0446 + + (2) https://www.python.org/dev/peps/pep-0446 + + +File: python.info, Node: Improvements to Codec Handling, Next: PEP 451 A ModuleSpec Type for the Import System, Prev: PEP 446 Newly Created File Descriptors Are Non-Inheritable, Up: New Features<5> + +1.5.2.5 Improvements to Codec Handling +...................................... + +Since it was first introduced, the *note codecs: 1c. module has always +been intended to operate as a type-neutral dynamic encoding and decoding +system. However, its close coupling with the Python text model, +especially the type restricted convenience methods on the builtin *note +str: 330, *note bytes: 331. and *note bytearray: 332. types, has +historically obscured that fact. + +As a key step in clarifying the situation, the *note codecs.encode(): +802. and *note codecs.decode(): 803. convenience functions are now +properly documented in Python 2.7, 3.3 and 3.4. These functions have +existed in the *note codecs: 1c. module (and have been covered by the +regression test suite) since Python 2.4, but were previously only +discoverable through runtime introspection. + +Unlike the convenience methods on *note str: 330, *note bytes: 331. and +*note bytearray: 332, the *note codecs: 1c. convenience functions +support arbitrary codecs in both Python 2 and Python 3, rather than +being limited to Unicode text encodings (in Python 3) or ‘basestring’ +<-> ‘basestring’ conversions (in Python 2). + +In Python 3.4, the interpreter is able to identify the known non-text +encodings provided in the standard library and direct users towards +these general purpose convenience functions when appropriate: + + >>> b"abcdef".decode("hex") + Traceback (most recent call last): + File "", line 1, in + LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs + + >>> "hello".encode("rot13") + Traceback (most recent call last): + File "", line 1, in + LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs + + >>> open("foo.txt", encoding="hex") + Traceback (most recent call last): + File "", line 1, in + LookupError: 'hex' is not a text encoding; use codecs.open() to handle arbitrary codecs + +In a related change, whenever it is feasible without breaking backwards +compatibility, exceptions raised during encoding and decoding operations +are wrapped in a chained exception of the same type that mentions the +name of the codec responsible for producing the error: + + >>> import codecs + + >>> codecs.decode(b"abcdefgh", "hex") + Traceback (most recent call last): + File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode + return (binascii.a2b_hex(input), len(input)) + binascii.Error: Non-hexadecimal digit found + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found) + + >>> codecs.encode("hello", "bz2") + Traceback (most recent call last): + File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode + return (bz2.compress(input), len(input)) + File "/usr/lib/python3.4/bz2.py", line 498, in compress + return comp.compress(data) + comp.flush() + TypeError: 'str' does not support the buffer interface + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface) + +Finally, as the examples above show, these improvements have permitted +the restoration of the convenience aliases for the non-Unicode codecs +that were themselves restored in Python 3.2. This means that encoding +binary data to and from its hexadecimal representation (for example) can +now be written as: + + >>> from codecs import encode, decode + >>> encode(b"hello", "hex") + b'68656c6c6f' + >>> decode(b"68656c6c6f", "hex") + b'hello' + +The binary and text transforms provided in the standard library are +detailed in *note Binary Transforms: 804. and *note Text Transforms: +805. + +(Contributed by Nick Coghlan in bpo-7475(1), bpo-17827(2), bpo-17828(3) +and bpo-19619(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7475 + + (2) https://bugs.python.org/issue17827 + + (3) https://bugs.python.org/issue17828 + + (4) https://bugs.python.org/issue19619 + + +File: python.info, Node: PEP 451 A ModuleSpec Type for the Import System, Next: Other Language Changes<5>, Prev: Improvements to Codec Handling, Up: New Features<5> + +1.5.2.6 PEP 451: A ModuleSpec Type for the Import System +........................................................ + +PEP 451(1) provides an encapsulation of the information about a module +that the import machinery will use to load it (that is, a module +specification). This helps simplify both the import implementation and +several import-related APIs. The change is also a stepping stone for +several future import-related improvements(2). + +The public-facing changes from the PEP are entirely backward-compatible. +Furthermore, they should be transparent to everyone but importer +authors. Key finder and loader methods have been deprecated, but they +will continue working. New importers should use the new methods +described in the PEP. Existing importers should be updated to implement +the new methods. See the *note Deprecated: 807. section for a list of +methods that should be replaced and their replacements. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0451 + + (2) +https://mail.python.org/pipermail/python-dev/2013-November/130111.html + + +File: python.info, Node: Other Language Changes<5>, Prev: PEP 451 A ModuleSpec Type for the Import System, Up: New Features<5> + +1.5.2.7 Other Language Changes +.............................. + +Some smaller changes made to the core Python language are: + + * Unicode database updated to UCD version 6.3. + + * *note min(): 809. and *note max(): 80a. now accept a `default' + keyword-only argument that can be used to specify the value they + return if the iterable they are evaluating has no elements. + (Contributed by Julian Berman in bpo-18111(1).) + + * Module objects are now *note weakref: 128.’able. + + * Module ‘__file__’ attributes (and related values) should now always + contain absolute paths by default, with the sole exception of + ‘__main__.__file__’ when a script has been executed directly using + a relative path. (Contributed by Brett Cannon in bpo-18416(2).) + + * All the UTF-* codecs (except UTF-7) now reject surrogates during + both encoding and decoding unless the ‘surrogatepass’ error handler + is used, with the exception of the UTF-16 decoder (which accepts + valid surrogate pairs) and the UTF-16 encoder (which produces them + while encoding non-BMP characters). (Contributed by Victor + Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in bpo-12892(3).) + + * New German EBCDIC *note codec: 68f. ‘cp273’. (Contributed by + Michael Bierenfeld and Andrew Kuchling in bpo-1097797(4).) + + * New Ukrainian *note codec: 68f. ‘cp1125’. (Contributed by Serhiy + Storchaka in bpo-19668(5).) + + * *note bytes: 331.join() and *note bytearray: 332.join() now accept + arbitrary buffer objects as arguments. (Contributed by Antoine + Pitrou in bpo-15958(6).) + + * The *note int: 184. constructor now accepts any object that has an + ‘__index__’ method for its `base' argument. (Contributed by Mark + Dickinson in bpo-16772(7).) + + * Frame objects now have a *note clear(): 80b. method that clears all + references to local variables from the frame. (Contributed by + Antoine Pitrou in bpo-17934(8).) + + * *note memoryview: 25c. is now registered as a *note Sequence: 1f, + and supports the *note reversed(): 18e. builtin. (Contributed by + Nick Coghlan and Claudiu Popa in bpo-18690(9) and bpo-19078(10).) + + * Signatures reported by *note help(): 572. have been modified and + improved in several cases as a result of the introduction of + Argument Clinic and other changes to the *note inspect: a0. and + *note pydoc: d9. modules. + + * *note __length_hint__(): 80c. is now part of the formal language + specification (see PEP 424(11)). (Contributed by Armin Ronacher in + bpo-16148(12).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18111 + + (2) https://bugs.python.org/issue18416 + + (3) https://bugs.python.org/issue12892 + + (4) https://bugs.python.org/issue1097797 + + (5) https://bugs.python.org/issue19668 + + (6) https://bugs.python.org/issue15958 + + (7) https://bugs.python.org/issue16772 + + (8) https://bugs.python.org/issue17934 + + (9) https://bugs.python.org/issue18690 + + (10) https://bugs.python.org/issue19078 + + (11) https://www.python.org/dev/peps/pep-0424 + + (12) https://bugs.python.org/issue16148 + + +File: python.info, Node: New Modules<5>, Next: Improved Modules<5>, Prev: New Features<5>, Up: What’s New In Python 3 4 + +1.5.3 New Modules +----------------- + +* Menu: + +* asyncio: asyncio<6>. +* ensurepip:: +* enum: enum<5>. +* pathlib: pathlib<5>. +* selectors: selectors<2>. +* statistics: statistics<3>. +* tracemalloc: tracemalloc<3>. + + +File: python.info, Node: asyncio<6>, Next: ensurepip, Up: New Modules<5> + +1.5.3.1 asyncio +............... + +The new *note asyncio: a. module (defined in PEP 3156(1)) provides a +standard pluggable event loop model for Python, providing solid +asynchronous IO support in the standard library, and making it easier +for other event loop implementations to interoperate with the standard +library and each other. + +For Python 3.4, this module is considered a *note provisional API: 348. + +See also +........ + +PEP 3156(2) – Asynchronous IO Support Rebooted: the “asyncio” Module + + PEP written and implementation led by Guido van Rossum. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3156 + + (2) https://www.python.org/dev/peps/pep-3156 + + +File: python.info, Node: ensurepip, Next: enum<5>, Prev: asyncio<6>, Up: New Modules<5> + +1.5.3.2 ensurepip +................. + +The new *note ensurepip: 7a. module is the primary infrastructure for +the PEP 453(1) implementation. In the normal course of events end users +will not need to interact with this module, but it can be used to +manually bootstrap ‘pip’ if the automated bootstrapping into an +installation or virtual environment was declined. + +*note ensurepip: 7a. includes a bundled copy of ‘pip’, up-to-date as of +the first release candidate of the release of CPython with which it +ships (this applies to both maintenance releases and feature releases). +‘ensurepip’ does not access the internet. If the installation has +Internet access, after ‘ensurepip’ is run the bundled ‘pip’ can be used +to upgrade ‘pip’ to a more recent release than the bundled one. (Note +that such an upgraded version of ‘pip’ is considered to be a separately +installed package and will not be removed if Python is uninstalled.) + +The module is named `ensure'pip because if called when ‘pip’ is already +installed, it does nothing. It also has an ‘--upgrade’ option that will +cause it to install the bundled copy of ‘pip’ if the existing installed +version of ‘pip’ is older than the bundled copy. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0453 + + +File: python.info, Node: enum<5>, Next: pathlib<5>, Prev: ensurepip, Up: New Modules<5> + +1.5.3.3 enum +............ + +The new *note enum: 7b. module (defined in PEP 435(1)) provides a +standard implementation of enumeration types, allowing other modules +(such as *note socket: ef.) to provide more informative error messages +and better debugging support by replacing opaque integer constants with +backwards compatible enumeration values. + +See also +........ + +PEP 435(2) – Adding an Enum type to the Python standard library + + PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman, + implemented by Ethan Furman. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0435 + + (2) https://www.python.org/dev/peps/pep-0435 + + +File: python.info, Node: pathlib<5>, Next: selectors<2>, Prev: enum<5>, Up: New Modules<5> + +1.5.3.4 pathlib +............... + +The new *note pathlib: c8. module offers classes representing filesystem +paths with semantics appropriate for different operating systems. Path +classes are divided between `pure paths', which provide purely +computational operations without I/O, and `concrete paths', which +inherit from pure paths but also provide I/O operations. + +For Python 3.4, this module is considered a *note provisional API: 348. + +See also +........ + +PEP 428(1) – The pathlib module – object-oriented filesystem paths + + PEP written and implemented by Antoine Pitrou. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0428 + + +File: python.info, Node: selectors<2>, Next: statistics<3>, Prev: pathlib<5>, Up: New Modules<5> + +1.5.3.5 selectors +................. + +The new *note selectors: e6. module (created as part of implementing PEP +3156(1)) allows high-level and efficient I/O multiplexing, built upon +the *note select: e5. module primitives. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3156 + + +File: python.info, Node: statistics<3>, Next: tracemalloc<3>, Prev: selectors<2>, Up: New Modules<5> + +1.5.3.6 statistics +.................. + +The new *note statistics: f5. module (defined in PEP 450(1)) offers some +core statistics functionality directly in the standard library. This +module supports calculation of the mean, median, mode, variance and +standard deviation of a data series. + +See also +........ + +PEP 450(2) – Adding A Statistics Module To The Standard Library + + PEP written and implemented by Steven D’Aprano + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0450 + + (2) https://www.python.org/dev/peps/pep-0450 + + +File: python.info, Node: tracemalloc<3>, Prev: statistics<3>, Up: New Modules<5> + +1.5.3.7 tracemalloc +................... + +The new *note tracemalloc: 114. module (defined in PEP 454(1)) is a +debug tool to trace memory blocks allocated by Python. It provides the +following information: + + * Trace where an object was allocated + + * Statistics on allocated memory blocks per filename and per line + number: total size, number and average size of allocated memory + blocks + + * Compute the differences between two snapshots to detect memory + leaks + +See also +........ + +PEP 454(2) – Add a new tracemalloc module to trace Python memory allocations + + PEP written and implemented by Victor Stinner + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0454 + + (2) https://www.python.org/dev/peps/pep-0454 + + +File: python.info, Node: Improved Modules<5>, Next: CPython Implementation Changes, Prev: New Modules<5>, Up: What’s New In Python 3 4 + +1.5.4 Improved Modules +---------------------- + +* Menu: + +* abc:: +* aifc: aifc<2>. +* argparse: argparse<3>. +* audioop:: +* base64:: +* collections: collections<6>. +* colorsys:: +* contextlib: contextlib<4>. +* dbm: dbm<5>. +* dis: dis<2>. +* doctest: doctest<2>. +* email: email<3>. +* filecmp:: +* functools: functools<4>. +* gc: gc<3>. +* glob: glob<2>. +* hashlib: hashlib<2>. +* hmac: hmac<2>. +* html:: +* http: http<2>. +* idlelib and IDLE: idlelib and IDLE<4>. +* importlib: importlib<6>. +* inspect: inspect<4>. +* ipaddress: ipaddress<3>. +* logging: logging<5>. +* marshal:: +* mmap: mmap<2>. +* multiprocessing: multiprocessing<5>. +* operator: operator<2>. +* os: os<6>. +* pdb: pdb<3>. +* pickle: pickle<4>. +* plistlib: plistlib<2>. +* poplib: poplib<2>. +* pprint: pprint<2>. +* pty:: +* pydoc: pydoc<3>. +* re: re<5>. +* resource:: +* select:: +* shelve:: +* shutil: shutil<3>. +* smtpd: smtpd<2>. +* smtplib: smtplib<2>. +* socket: socket<6>. +* sqlite3: sqlite3<4>. +* ssl: ssl<7>. +* stat:: +* struct: struct<2>. +* subprocess: subprocess<4>. +* sunau: sunau<2>. +* sys: sys<6>. +* tarfile: tarfile<3>. +* textwrap:: +* threading: threading<4>. +* traceback: traceback<3>. +* types: types<3>. +* urllib: urllib<2>. +* unittest: unittest<4>. +* venv: venv<4>. +* wave: wave<2>. +* weakref: weakref<2>. +* xml.etree: xml etree<2>. +* zipfile: zipfile<4>. + + +File: python.info, Node: abc, Next: aifc<2>, Up: Improved Modules<5> + +1.5.4.1 abc +........... + +New function *note abc.get_cache_token(): 817. can be used to know when +to invalidate caches that are affected by changes in the object graph. +(Contributed by Łukasz Langa in bpo-16832(1).) + +New class *note ABC: 818. has *note ABCMeta: 819. as its meta class. +Using ‘ABC’ as a base class has essentially the same effect as +specifying ‘metaclass=abc.ABCMeta’, but is simpler to type and easier to +read. (Contributed by Bruno Dupuis in bpo-16049(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16832 + + (2) https://bugs.python.org/issue16049 + + +File: python.info, Node: aifc<2>, Next: argparse<3>, Prev: abc, Up: Improved Modules<5> + +1.5.4.2 aifc +............ + +The *note getparams(): 81b. method now returns a namedtuple rather than +a plain tuple. (Contributed by Claudiu Popa in bpo-17818(1).) + +*note aifc.open(): 45b. now supports the context management protocol: +when used in a *note with: 6e9. block, the *note close(): 81c. method of +the returned object will be called automatically at the end of the +block. (Contributed by Serhiy Storchacha in bpo-16486(2).) + +The *note writeframesraw(): 81d. and *note writeframes(): 81e. methods +now accept any *note bytes-like object: 5e8. (Contributed by Serhiy +Storchaka in bpo-8311(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17818 + + (2) https://bugs.python.org/issue16486 + + (3) https://bugs.python.org/issue8311 + + +File: python.info, Node: argparse<3>, Next: audioop, Prev: aifc<2>, Up: Improved Modules<5> + +1.5.4.3 argparse +................ + +The *note FileType: 820. class now accepts `encoding' and `errors' +arguments, which are passed through to *note open(): 4f0. (Contributed +by Lucas Maystre in bpo-11175(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11175 + + +File: python.info, Node: audioop, Next: base64, Prev: argparse<3>, Up: Improved Modules<5> + +1.5.4.4 audioop +............... + +*note audioop: d. now supports 24-bit samples. (Contributed by Serhiy +Storchaka in bpo-12866(1).) + +New *note byteswap(): 822. function converts big-endian samples to +little-endian and vice versa. (Contributed by Serhiy Storchaka in +bpo-19641(2).) + +All *note audioop: d. functions now accept any *note bytes-like object: +5e8. Strings are not accepted: they didn’t work before, now they raise +an error right away. (Contributed by Serhiy Storchaka in bpo-16685(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12866 + + (2) https://bugs.python.org/issue19641 + + (3) https://bugs.python.org/issue16685 + + +File: python.info, Node: base64, Next: collections<6>, Prev: audioop, Up: Improved Modules<5> + +1.5.4.5 base64 +.............. + +The encoding and decoding functions in *note base64: e. now accept any +*note bytes-like object: 5e8. in cases where it previously required a +*note bytes: 331. or *note bytearray: 332. instance. (Contributed by +Nick Coghlan in bpo-17839(1).) + +New functions *note a85encode(): 824, *note a85decode(): 825, *note +b85encode(): 826, and *note b85decode(): 827. provide the ability to +encode and decode binary data from and to ‘Ascii85’ and the +git/mercurial ‘Base85’ formats, respectively. The ‘a85’ functions have +options that can be used to make them compatible with the variants of +the ‘Ascii85’ encoding, including the Adobe variant. (Contributed by +Martin Morrison, the Mercurial project, Serhiy Storchaka, and Antoine +Pitrou in bpo-17618(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17839 + + (2) https://bugs.python.org/issue17618 + + +File: python.info, Node: collections<6>, Next: colorsys, Prev: base64, Up: Improved Modules<5> + +1.5.4.6 collections +................... + +The *note ChainMap.new_child(): 829. method now accepts an `m' argument +specifying the child map to add to the chain. This allows an existing +mapping and/or a custom mapping type to be used for the child. +(Contributed by Vinay Sajip in bpo-16613(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16613 + + +File: python.info, Node: colorsys, Next: contextlib<4>, Prev: collections<6>, Up: Improved Modules<5> + +1.5.4.7 colorsys +................ + +The number of digits in the coefficients for the RGB — YIQ conversions +have been expanded so that they match the FCC NTSC versions. The change +in results should be less than 1% and may better match results found +elsewhere. (Contributed by Brian Landers and Serhiy Storchaka in +bpo-14323(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14323 + + +File: python.info, Node: contextlib<4>, Next: dbm<5>, Prev: colorsys, Up: Improved Modules<5> + +1.5.4.8 contextlib +.................. + +The new *note contextlib.suppress: 82c. context manager helps to clarify +the intent of code that deliberately suppresses exceptions from a single +statement. (Contributed by Raymond Hettinger in bpo-15806(1) and Zero +Piraeus in bpo-19266(2).) + +The new *note contextlib.redirect_stdout(): 6c2. context manager makes +it easier for utility scripts to handle inflexible APIs that write their +output to *note sys.stdout: 30e. and don’t provide any options to +redirect it. Using the context manager, the *note sys.stdout: 30e. +output can be redirected to any other stream or, in conjunction with +*note io.StringIO: 82d, to a string. The latter can be especially +useful, for example, to capture output from a function that was written +to implement a command line interface. It is recommended only for +utility scripts because it affects the global state of *note sys.stdout: +30e. (Contributed by Raymond Hettinger in bpo-15805(3).) + +The *note contextlib: 24. documentation has also been updated to include +a *note discussion: 82e. of the differences between single use, reusable +and reentrant context managers. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15806 + + (2) https://bugs.python.org/issue19266 + + (3) https://bugs.python.org/issue15805 + + +File: python.info, Node: dbm<5>, Next: dis<2>, Prev: contextlib<4>, Up: Improved Modules<5> + +1.5.4.9 dbm +........... + +*note dbm.open(): 830. objects now support the context management +protocol. When used in a *note with: 6e9. statement, the ‘close’ method +of the database object will be called automatically at the end of the +block. (Contributed by Claudiu Popa and Nick Coghlan in bpo-19282(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19282 + + +File: python.info, Node: dis<2>, Next: doctest<2>, Prev: dbm<5>, Up: Improved Modules<5> + +1.5.4.10 dis +............ + +Functions *note show_code(): 832, *note dis(): 384, *note distb(): 833, +and *note disassemble(): 834. now accept a keyword-only `file' argument +that controls where they write their output. + +The *note dis: 38. module is now built around an *note Instruction: 835. +class that provides object oriented access to the details of each +individual bytecode operation. + +A new method, *note get_instructions(): 836, provides an iterator that +emits the Instruction stream for a given piece of Python code. Thus it +is now possible to write a program that inspects and manipulates a +bytecode object in ways different from those provided by the *note dis: +38. module itself. For example: + + >>> import dis + >>> for instr in dis.get_instructions(lambda x: x + 1): + ... print(instr.opname) + LOAD_FAST + LOAD_CONST + BINARY_ADD + RETURN_VALUE + +The various display tools in the *note dis: 38. module have been +rewritten to use these new components. + +In addition, a new application-friendly class *note Bytecode: 837. +provides an object-oriented API for inspecting bytecode in both in +human-readable form and for iterating over instructions. The *note +Bytecode: 837. constructor takes the same arguments that +‘get_instruction()’ does (plus an optional `current_offset'), and the +resulting object can be iterated to produce *note Instruction: 835. +objects. But it also has a *note dis: 838. method, equivalent to +calling *note dis: 384. on the constructor argument, but returned as a +multi-line string: + + >>> bytecode = dis.Bytecode(lambda x: x + 1, current_offset=3) + >>> for instr in bytecode: + ... print('{} ({})'.format(instr.opname, instr.opcode)) + LOAD_FAST (124) + LOAD_CONST (100) + BINARY_ADD (23) + RETURN_VALUE (83) + >>> bytecode.dis().splitlines() + [' 1 0 LOAD_FAST 0 (x)', + ' --> 3 LOAD_CONST 1 (1)', + ' 6 BINARY_ADD', + ' 7 RETURN_VALUE'] + +*note Bytecode: 837. also has a class method, *note from_traceback(): +839, that provides the ability to manipulate a traceback (that is, +‘print(Bytecode.from_traceback(tb).dis())’ is equivalent to +‘distb(tb)’). + +(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in +bpo-11816(1) and Claudiu Popa in bpo-17916(2).) + +New function *note stack_effect(): 83a. computes the effect on the +Python stack of a given opcode and argument, information that is not +otherwise available. (Contributed by Larry Hastings in bpo-19722(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11816 + + (2) https://bugs.python.org/issue17916 + + (3) https://bugs.python.org/issue19722 + + +File: python.info, Node: doctest<2>, Next: email<3>, Prev: dis<2>, Up: Improved Modules<5> + +1.5.4.11 doctest +................ + +A new *note option flag: 83c, *note FAIL_FAST: 83d, halts test running +as soon as the first failure is detected. (Contributed by R. David +Murray and Daniel Urban in bpo-16522(1).) + +The *note doctest: 67. command line interface now uses *note argparse: +6, and has two new options, ‘-o’ and ‘-f’. ‘-o’ allows *note doctest +options: 83c. to be specified on the command line, and ‘-f’ is a +shorthand for ‘-o FAIL_FAST’ (to parallel the similar option supported +by the *note unittest: 11b. CLI). (Contributed by R. David Murray in +bpo-11390(2).) + +*note doctest: 67. will now find doctests in extension module ‘__doc__’ +strings. (Contributed by Zachary Ware in bpo-3158(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16522 + + (2) https://bugs.python.org/issue11390 + + (3) https://bugs.python.org/issue3158 + + +File: python.info, Node: email<3>, Next: filecmp, Prev: doctest<2>, Up: Improved Modules<5> + +1.5.4.12 email +.............. + +*note as_string(): 83f. now accepts a `policy' argument to override the +default policy of the message when generating a string representation of +it. This means that ‘as_string’ can now be used in more circumstances, +instead of having to create and use a *note generator: 6e. in order to +pass formatting parameters to its ‘flatten’ method. (Contributed by R. +David Murray in bpo-18600(1).) + +New method *note as_bytes(): 840. added to produce a bytes +representation of the message in a fashion similar to how ‘as_string’ +produces a string representation. It does not accept the `maxheaderlen' +argument, but does accept the `unixfrom' and `policy' arguments. The +*note Message: 541. *note __bytes__(): 841. method calls it, meaning +that ‘bytes(mymsg)’ will now produce the intuitive result: a bytes +object containing the fully formatted message. (Contributed by R. David +Murray in bpo-18600(2).) + +The *note Message.set_param(): 842. message now accepts a `replace' +keyword argument. When specified, the associated header will be updated +without changing its location in the list of headers. For backward +compatibility, the default is ‘False’. (Contributed by R. David Murray +in bpo-18891(3).) A pair of new subclasses of *note Message: 541. have +been added (*note EmailMessage: 542. and *note MIMEPart: 843.), along +with a new sub-module, *note contentmanager: 6b. and a new *note policy: +75. attribute *note content_manager: 844. All documentation is +currently in the new module, which is being added as part of email’s new +*note provisional API: 348. These classes provide a number of new +methods that make extracting content from and inserting content into +email messages much easier. For details, see the *note contentmanager: +6b. documentation and the *note email; Examples: 845. These API +additions complete the bulk of the work that was planned as part of the +email6 project. The currently provisional API is scheduled to become +final in Python 3.5 (possibly with a few minor additions in the area of +error handling). (Contributed by R. David Murray in bpo-18891(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18600 + + (2) https://bugs.python.org/issue18600 + + (3) https://bugs.python.org/issue18891 + + (4) https://bugs.python.org/issue18891 + + +File: python.info, Node: filecmp, Next: functools<4>, Prev: email<3>, Up: Improved Modules<5> + +1.5.4.13 filecmp +................ + +A new *note clear_cache(): 847. function provides the ability to clear +the *note filecmp: 7f. comparison cache, which uses *note os.stat(): +1f1. information to determine if the file has changed since the last +compare. This can be used, for example, if the file might have been +changed and re-checked in less time than the resolution of a particular +filesystem’s file modification time field. (Contributed by Mark Levitt +in bpo-18149(1).) + +New module attribute *note DEFAULT_IGNORES: 848. provides the list of +directories that are used as the default value for the `ignore' +parameter of the *note dircmp(): 849. function. (Contributed by Eli +Bendersky in bpo-15442(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18149 + + (2) https://bugs.python.org/issue15442 + + +File: python.info, Node: functools<4>, Next: gc<3>, Prev: filecmp, Up: Improved Modules<5> + +1.5.4.14 functools +.................. + +The new *note partialmethod(): 29c. descriptor brings partial argument +application to descriptors, just as *note partial(): 7c8. provides for +normal callables. The new descriptor also makes it easier to get +arbitrary callables (including *note partial(): 7c8. instances) to +behave like normal instance methods when included in a class definition. +(Contributed by Alon Horev and Nick Coghlan in bpo-4331(1).) The new +*note singledispatch(): 38a. decorator brings support for +single-dispatch generic functions to the Python standard library. Where +object oriented programming focuses on grouping multiple operations on a +common set of data into a class, a generic function focuses on grouping +multiple implementations of an operation that allows it to work with +`different' kinds of data. + +See also +........ + +PEP 443(2) – Single-dispatch generic functions + + PEP written and implemented by Łukasz Langa. + +*note total_ordering(): 84b. now supports a return value of *note +NotImplemented: 84c. from the underlying comparison function. +(Contributed by Katie Miller in bpo-10042(3).) + +A pure-python version of the *note partial(): 7c8. function is now in +the stdlib; in CPython it is overridden by the C accelerated version, +but it is available for other implementations to use. (Contributed by +Brian Thorne in bpo-12428(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4331 + + (2) https://www.python.org/dev/peps/pep-0443 + + (3) https://bugs.python.org/issue10042 + + (4) https://bugs.python.org/issue12428 + + +File: python.info, Node: gc<3>, Next: glob<2>, Prev: functools<4>, Up: Improved Modules<5> + +1.5.4.15 gc +........... + +New function *note get_stats(): 84e. returns a list of three +per-generation dictionaries containing the collections statistics since +interpreter startup. (Contributed by Antoine Pitrou in bpo-16351(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16351 + + +File: python.info, Node: glob<2>, Next: hashlib<2>, Prev: gc<3>, Up: Improved Modules<5> + +1.5.4.16 glob +............. + +A new function *note escape(): 850. provides a way to escape special +characters in a filename so that they do not become part of the globbing +expansion but are instead matched literally. (Contributed by Serhiy +Storchaka in bpo-8402(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8402 + + +File: python.info, Node: hashlib<2>, Next: hmac<2>, Prev: glob<2>, Up: Improved Modules<5> + +1.5.4.17 hashlib +................ + +A new *note hashlib.pbkdf2_hmac(): 7e6. function provides the PKCS#5 +password-based key derivation function 2(1). (Contributed by Christian +Heimes in bpo-18582(2).) + +The *note name: 852. attribute of *note hashlib: 8d. hash objects is now +a formally supported interface. It has always existed in CPython’s +*note hashlib: 8d. (although it did not return lower case names for all +supported hashes), but it was not a public interface and so some other +Python implementations have not previously supported it. (Contributed +by Jason R. Coombs in bpo-18532(3).) + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/PBKDF2 + + (2) https://bugs.python.org/issue18582 + + (3) https://bugs.python.org/issue18532 + + +File: python.info, Node: hmac<2>, Next: html, Prev: hashlib<2>, Up: Improved Modules<5> + +1.5.4.18 hmac +............. + +*note hmac: 8f. now accepts ‘bytearray’ as well as ‘bytes’ for the `key' +argument to the *note new(): 854. function, and the `msg' parameter to +both the *note new(): 854. function and the *note update(): 855. method +now accepts any type supported by the *note hashlib: 8d. module. +(Contributed by Jonas Borgström in bpo-18240(1).) + +The `digestmod' argument to the *note hmac.new(): 854. function may now +be any hash digest name recognized by *note hashlib: 8d. In addition, +the current behavior in which the value of `digestmod' defaults to ‘MD5’ +is deprecated: in a future version of Python there will be no default +value. (Contributed by Christian Heimes in bpo-17276(2).) + +With the addition of *note block_size: 856. and *note name: 857. +attributes (and the formal documentation of the *note digest_size: 858. +attribute), the *note hmac: 8f. module now conforms fully to the PEP +247(3) API. (Contributed by Christian Heimes in bpo-18775(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18240 + + (2) https://bugs.python.org/issue17276 + + (3) https://www.python.org/dev/peps/pep-0247 + + (4) https://bugs.python.org/issue18775 + + +File: python.info, Node: html, Next: http<2>, Prev: hmac<2>, Up: Improved Modules<5> + +1.5.4.19 html +............. + +New function *note unescape(): 85a. function converts HTML5 character +references to the corresponding Unicode characters. (Contributed by +Ezio Melotti in bpo-2927(1).) + +*note HTMLParser: 7bf. accepts a new keyword argument `convert_charrefs' +that, when ‘True’, automatically converts all character references. For +backward-compatibility, its value defaults to ‘False’, but it will +change to ‘True’ in a future version of Python, so you are invited to +set it explicitly and update your code to use this new feature. +(Contributed by Ezio Melotti in bpo-13633(2).) + +The `strict' argument of *note HTMLParser: 7bf. is now deprecated. +(Contributed by Ezio Melotti in bpo-15114(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2927 + + (2) https://bugs.python.org/issue13633 + + (3) https://bugs.python.org/issue15114 + + +File: python.info, Node: http<2>, Next: idlelib and IDLE<4>, Prev: html, Up: Improved Modules<5> + +1.5.4.20 http +............. + +*note send_error(): 85c. now accepts an optional additional `explain' +parameter which can be used to provide an extended error description, +overriding the hardcoded default if there is one. This extended error +description will be formatted using the ‘error_message_format’ attribute +and sent as the body of the error response. (Contributed by Karl Cow in +bpo-12921(1).) + +The *note http.server: 97. *note command line interface: 85d. now has a +‘-b/--bind’ option that causes the server to listen on a specific +address. (Contributed by Malte Swart in bpo-17764(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12921 + + (2) https://bugs.python.org/issue17764 + + +File: python.info, Node: idlelib and IDLE<4>, Next: importlib<6>, Prev: http<2>, Up: Improved Modules<5> + +1.5.4.21 idlelib and IDLE +......................... + +Since idlelib implements the IDLE shell and editor and is not intended +for import by other programs, it gets improvements with every release. +See ‘Lib/idlelib/NEWS.txt’ for a cumulative list of changes since 3.3.0, +as well as changes made in future 3.4.x releases. This file is also +available from the IDLE Help ‣ About IDLE dialog. + + +File: python.info, Node: importlib<6>, Next: inspect<4>, Prev: idlelib and IDLE<4>, Up: Improved Modules<5> + +1.5.4.22 importlib +.................. + +The *note InspectLoader: 860. ABC defines a new method, *note +source_to_code(): 6ef. that accepts source data and a path and returns a +code object. The default implementation is equivalent to ‘compile(data, +path, 'exec', dont_inherit=True)’. (Contributed by Eric Snow and Brett +Cannon in bpo-15627(1).) + +*note InspectLoader: 860. also now has a default implementation for the +*note get_code(): 861. method. However, it will normally be desirable +to override the default implementation for performance reasons. +(Contributed by Brett Cannon in bpo-18072(2).) + +The *note reload(): 399. function has been moved from *note imp: 9a. to +*note importlib: 9b. as part of the *note imp: 9a. module deprecation. +(Contributed by Berker Peksag in bpo-18193(3).) + +*note importlib.util: 9f. now has a *note MAGIC_NUMBER: 862. attribute +providing access to the bytecode version number. This replaces the +*note get_magic(): 863. function in the deprecated *note imp: 9a. +module. (Contributed by Brett Cannon in bpo-18192(4).) + +New *note importlib.util: 9f. functions *note cache_from_source(): 557. +and *note source_from_cache(): 558. replace the same-named functions in +the deprecated *note imp: 9a. module. (Contributed by Brett Cannon in +bpo-18194(5).) + +The *note importlib: 9b. bootstrap ‘NamespaceLoader’ now conforms to the +*note InspectLoader: 860. ABC, which means that ‘runpy’ and ‘python -m’ +can now be used with namespace packages. (Contributed by Brett Cannon +in bpo-18058(6).) + +*note importlib.util: 9f. has a new function *note decode_source(): 864. +that decodes source from bytes using universal newline processing. This +is useful for implementing *note InspectLoader.get_source(): 865. +methods. + +*note importlib.machinery.ExtensionFileLoader: 556. now has a *note +get_filename(): 866. method. This was inadvertently omitted in the +original implementation. (Contributed by Eric Snow in bpo-19152(7).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15627 + + (2) https://bugs.python.org/issue18072 + + (3) https://bugs.python.org/issue18193 + + (4) https://bugs.python.org/issue18192 + + (5) https://bugs.python.org/issue18194 + + (6) https://bugs.python.org/issue18058 + + (7) https://bugs.python.org/issue19152 + + +File: python.info, Node: inspect<4>, Next: ipaddress<3>, Prev: importlib<6>, Up: Improved Modules<5> + +1.5.4.23 inspect +................ + +The *note inspect: a0. module now offers a basic *note command line +interface: 868. to quickly display source code and other information for +modules, classes and functions. (Contributed by Claudiu Popa and Nick +Coghlan in bpo-18626(1).) + +*note unwrap(): 869. makes it easy to unravel wrapper function chains +created by *note functools.wraps(): 86a. (and any other API that sets +the ‘__wrapped__’ attribute on a wrapper function). (Contributed by +Daniel Urban, Aaron Iles and Nick Coghlan in bpo-13266(2).) + +As part of the implementation of the new *note enum: 7b. module, the +*note inspect: a0. module now has substantially better support for +custom ‘__dir__’ methods and dynamic class attributes provided through +metaclasses. (Contributed by Ethan Furman in bpo-18929(3) and +bpo-19030(4).) + +*note getfullargspec(): 55d. and *note getargspec(): 55c. now use the +*note signature(): 55b. API. This allows them to support a much broader +range of callables, including those with ‘__signature__’ attributes, +those with metadata provided by argument clinic, *note +functools.partial(): 7c8. objects and more. Note that, unlike *note +signature(): 55b, these functions still ignore ‘__wrapped__’ attributes, +and report the already bound first argument for bound methods, so it is +still necessary to update your code to use *note signature(): 55b. +directly if those features are desired. (Contributed by Yury Selivanov +in bpo-17481(5).) + +*note signature(): 55b. now supports duck types of CPython functions, +which adds support for functions compiled with Cython. (Contributed by +Stefan Behnel and Yury Selivanov in bpo-17159(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18626 + + (2) https://bugs.python.org/issue13266 + + (3) https://bugs.python.org/issue18929 + + (4) https://bugs.python.org/issue19030 + + (5) https://bugs.python.org/issue17481 + + (6) https://bugs.python.org/issue17159 + + +File: python.info, Node: ipaddress<3>, Next: logging<5>, Prev: inspect<4>, Up: Improved Modules<5> + +1.5.4.24 ipaddress +.................. + +*note ipaddress: a2. was added to the standard library in Python 3.3 as +a *note provisional API: 348. With the release of Python 3.4, this +qualification has been removed: *note ipaddress: a2. is now considered a +stable API, covered by the normal standard library requirements to +maintain backwards compatibility. + +A new *note is_global: 86c. property is ‘True’ if an address is globally +routeable. (Contributed by Peter Moody in bpo-17400(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17400 + + +File: python.info, Node: logging<5>, Next: marshal, Prev: ipaddress<3>, Up: Improved Modules<5> + +1.5.4.25 logging +................ + +The *note TimedRotatingFileHandler: 86e. has a new `atTime' parameter +that can be used to specify the time of day when rollover should happen. +(Contributed by Ronald Oussoren in bpo-9556(1).) + +*note SocketHandler: 86f. and *note DatagramHandler: 870. now support +Unix domain sockets (by setting `port' to ‘None’). (Contributed by +Vinay Sajip in commit ce46195b56a9.) + +*note fileConfig(): 3a9. now accepts a *note +configparser.RawConfigParser: 871. subclass instance for the `fname' +parameter. This facilitates using a configuration file when logging +configuration is just a part of the overall application configuration, +or where the application modifies the configuration before passing it to +*note fileConfig(): 3a9. (Contributed by Vinay Sajip in bpo-16110(2).) + +Logging configuration data received from a socket via the *note +logging.config.listen(): 872. function can now be validated before being +processed by supplying a verification function as the argument to the +new `verify' keyword argument. (Contributed by Vinay Sajip in +bpo-15452(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9556 + + (2) https://bugs.python.org/issue16110 + + (3) https://bugs.python.org/issue15452 + + +File: python.info, Node: marshal, Next: mmap<2>, Prev: logging<5>, Up: Improved Modules<5> + +1.5.4.26 marshal +................ + +The default *note marshal: b0. version has been bumped to 3. The code +implementing the new version restores the Python2 behavior of recording +only one copy of interned strings and preserving the interning on +deserialization, and extends this “one copy” ability to any object type +(including handling recursive references). This reduces both the size +of ‘.pyc’ files and the amount of memory a module occupies in memory +when it is loaded from a ‘.pyc’ (or ‘.pyo’) file. (Contributed by +Kristján Valur Jónsson in bpo-16475(1), with additional speedups by +Antoine Pitrou in bpo-19219(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16475 + + (2) https://bugs.python.org/issue19219 + + +File: python.info, Node: mmap<2>, Next: multiprocessing<5>, Prev: marshal, Up: Improved Modules<5> + +1.5.4.27 mmap +............. + +mmap objects can now be *note weakref: 128.ed. (Contributed by Valerie +Lambert in bpo-4885(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4885 + + +File: python.info, Node: multiprocessing<5>, Next: operator<2>, Prev: mmap<2>, Up: Improved Modules<5> + +1.5.4.28 multiprocessing +........................ + +On Unix two new *note start methods: 876, ‘spawn’ and ‘forkserver’, have +been added for starting processes using *note multiprocessing: b7. +These make the mixing of processes with threads more robust, and the +‘spawn’ method matches the semantics that multiprocessing has always +used on Windows. New function *note get_all_start_methods(): 877. +reports all start methods available on the platform, *note +get_start_method(): 878. reports the current start method, and *note +set_start_method(): 879. sets the start method. (Contributed by Richard +Oudkerk in bpo-8713(1).) + +*note multiprocessing: b7. also now has the concept of a ‘context’, +which determines how child processes are created. New function *note +get_context(): 87a. returns a context that uses a specified start +method. It has the same API as the *note multiprocessing: b7. module +itself, so you can use it to create *note Pool: 87b.s and other objects +that will operate within that context. This allows a framework and an +application or different parts of the same application to use +multiprocessing without interfering with each other. (Contributed by +Richard Oudkerk in bpo-18999(2).) + +Except when using the old `fork' start method, child processes no longer +inherit unneeded handles/file descriptors from their parents (part of +bpo-8713(3)). + +*note multiprocessing: b7. now relies on *note runpy: e2. (which +implements the ‘-m’ switch) to initialise ‘__main__’ appropriately in +child processes when using the ‘spawn’ or ‘forkserver’ start methods. +This resolves some edge cases where combining multiprocessing, the ‘-m’ +command line switch, and explicit relative imports could cause obscure +failures in child processes. (Contributed by Nick Coghlan in +bpo-19946(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8713 + + (2) https://bugs.python.org/issue18999 + + (3) https://bugs.python.org/issue8713 + + (4) https://bugs.python.org/issue19946 + + +File: python.info, Node: operator<2>, Next: os<6>, Prev: multiprocessing<5>, Up: Improved Modules<5> + +1.5.4.29 operator +................. + +New function *note length_hint(): 87d. provides an implementation of the +specification for how the *note __length_hint__(): 80c. special method +should be used, as part of the PEP 424(1) formal specification of this +language feature. (Contributed by Armin Ronacher in bpo-16148(2).) + +There is now a pure-python version of the *note operator: c2. module +available for reference and for use by alternate implementations of +Python. (Contributed by Zachary Ware in bpo-16694(3).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0424 + + (2) https://bugs.python.org/issue16148 + + (3) https://bugs.python.org/issue16694 + + +File: python.info, Node: os<6>, Next: pdb<3>, Prev: operator<2>, Up: Improved Modules<5> + +1.5.4.30 os +........... + +There are new functions to get and set the *note inheritable flag: 7fa. +of a file descriptor (*note os.get_inheritable(): 7fb, *note +os.set_inheritable(): 7fc.) or a Windows handle (*note +os.get_handle_inheritable(): 7fd, *note os.set_handle_inheritable(): +7fe.). + +New function *note cpu_count(): 87f. reports the number of CPUs +available on the platform on which Python is running (or ‘None’ if the +count can’t be determined). The *note multiprocessing.cpu_count(): 880. +function is now implemented in terms of this function). (Contributed by +Trent Nelson, Yogesh Chaudhari, Victor Stinner, and Charles-François +Natali in bpo-17914(1).) + +*note os.path.samestat(): 881. is now available on the Windows platform +(and the *note os.path.samefile(): 882. implementation is now shared +between Unix and Windows). (Contributed by Brian Curtin in +bpo-11939(2).) + +*note os.path.ismount(): 1fa. now recognizes volumes mounted below a +drive root on Windows. (Contributed by Tim Golden in bpo-9035(3).) + +*note os.open(): 665. supports two new flags on platforms that provide +them, *note O_PATH: 883. (un-opened file descriptor), and *note +O_TMPFILE: 884. (unnamed temporary file; as of 3.4.0 release available +only on Linux systems with a kernel version of 3.11 or newer that have +uapi headers). (Contributed by Christian Heimes in bpo-18673(4) and +Benjamin Peterson, respectively.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17914 + + (2) https://bugs.python.org/issue11939 + + (3) https://bugs.python.org/issue9035 + + (4) https://bugs.python.org/issue18673 + + +File: python.info, Node: pdb<3>, Next: pickle<4>, Prev: os<6>, Up: Improved Modules<5> + +1.5.4.31 pdb +............ + +*note pdb: c9. has been enhanced to handle generators, *note yield: 18f, +and ‘yield from’ in a more useful fashion. This is especially helpful +when debugging *note asyncio: a. based programs. (Contributed by Andrew +Svetlov and Xavier de Gaye in bpo-16596(1).) + +The ‘print’ command has been removed from *note pdb: c9, restoring +access to the Python *note print(): 886. function from the pdb command +line. Python2’s ‘pdb’ did not have a ‘print’ command; instead, entering +‘print’ executed the ‘print’ statement. In Python3 ‘print’ was +mistakenly made an alias for the pdb *note p: 887. command. ‘p’, +however, prints the ‘repr’ of its argument, not the ‘str’ like the +Python2 ‘print’ command did. Worse, the Python3 ‘pdb print’ command +shadowed the Python3 ‘print’ function, making it inaccessible at the +‘pdb’ prompt. (Contributed by Connor Osborn in bpo-18764(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16596 + + (2) https://bugs.python.org/issue18764 + + +File: python.info, Node: pickle<4>, Next: plistlib<2>, Prev: pdb<3>, Up: Improved Modules<5> + +1.5.4.32 pickle +............... + +*note pickle: ca. now supports (but does not use by default) a new +pickle protocol, protocol 4. This new protocol addresses a number of +issues that were present in previous protocols, such as the +serialization of nested classes, very large strings and containers, and +classes whose *note __new__(): 889. method takes keyword-only arguments. +It also provides some efficiency improvements. + +See also +........ + +PEP 3154(1) – Pickle protocol 4 + + PEP written by Antoine Pitrou and implemented by Alexandre + Vassalotti. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3154 + + +File: python.info, Node: plistlib<2>, Next: poplib<2>, Prev: pickle<4>, Up: Improved Modules<5> + +1.5.4.33 plistlib +................. + +*note plistlib: cf. now has an API that is similar to the standard +pattern for stdlib serialization protocols, with new *note load(): 88b, +*note dump(): 88c, *note loads(): 88d, and *note dumps(): 88e. +functions. (The older API is now deprecated.) In addition to the +already supported XML plist format (*note FMT_XML: 88f.), it also now +supports the binary plist format (*note FMT_BINARY: 890.). (Contributed +by Ronald Oussoren and others in bpo-14455(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14455 + + +File: python.info, Node: poplib<2>, Next: pprint<2>, Prev: plistlib<2>, Up: Improved Modules<5> + +1.5.4.34 poplib +............... + +Two new methods have been added to *note poplib: d0.: *note capa(): 892, +which returns the list of capabilities advertised by the POP server, and +*note stls(): 893, which switches a clear-text POP3 session into an +encrypted POP3 session if the POP server supports it. (Contributed by +Lorenzo Catucci in bpo-4473(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4473 + + +File: python.info, Node: pprint<2>, Next: pty, Prev: poplib<2>, Up: Improved Modules<5> + +1.5.4.35 pprint +............... + +The *note pprint: d2. module’s *note PrettyPrinter: 895. class and its +*note pformat(): 896, and *note pprint(): 213. functions have a new +option, `compact', that controls how the output is formatted. Currently +setting `compact' to ‘True’ means that sequences will be printed with as +many sequence elements as will fit within `width' on each (indented) +line. (Contributed by Serhiy Storchaka in bpo-19132(1).) + +Long strings are now wrapped using Python’s normal line continuation +syntax. (Contributed by Antoine Pitrou in bpo-17150(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19132 + + (2) https://bugs.python.org/issue17150 + + +File: python.info, Node: pty, Next: pydoc<3>, Prev: pprint<2>, Up: Improved Modules<5> + +1.5.4.36 pty +............ + +*note pty.spawn(): 898. now returns the status value from *note +os.waitpid(): 66d. on the child process, instead of ‘None’. +(Contributed by Gregory P. Smith.) + + +File: python.info, Node: pydoc<3>, Next: re<5>, Prev: pty, Up: Improved Modules<5> + +1.5.4.37 pydoc +.............. + +The *note pydoc: d9. module is now based directly on the *note +inspect.signature(): 55b. introspection API, allowing it to provide +signature information for a wider variety of callable objects. This +change also means that ‘__wrapped__’ attributes are now taken into +account when displaying help information. (Contributed by Larry +Hastings in bpo-19674(1).) + +The *note pydoc: d9. module no longer displays the ‘self’ parameter for +already bound methods. Instead, it aims to always display the exact +current signature of the supplied callable. (Contributed by Larry +Hastings in bpo-20710(2).) + +In addition to the changes that have been made to *note pydoc: d9. +directly, its handling of custom ‘__dir__’ methods and various +descriptor behaviours has also been improved substantially by the +underlying changes in the *note inspect: a0. module. + +As the *note help(): 572. builtin is based on *note pydoc: d9, the above +changes also affect the behaviour of *note help(): 572. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19674 + + (2) https://bugs.python.org/issue20710 + + +File: python.info, Node: re<5>, Next: resource, Prev: pydoc<3>, Up: Improved Modules<5> + +1.5.4.38 re +........... + +New *note fullmatch(): 89b. function and ‘regex.fullmatch()’ method +anchor the pattern at both ends of the string to match. This provides a +way to be explicit about the goal of the match, which avoids a class of +subtle bugs where ‘$’ characters get lost during code changes or the +addition of alternatives to an existing regular expression. +(Contributed by Matthew Barnett in bpo-16203(1).) + +The repr of *note regex objects: 89c. now includes the pattern and the +flags; the repr of *note match objects: 89d. now includes the start, +end, and the part of the string that matched. (Contributed by Hugo +Lopes Tavares and Serhiy Storchaka in bpo-13592(2) and bpo-17087(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16203 + + (2) https://bugs.python.org/issue13592 + + (3) https://bugs.python.org/issue17087 + + +File: python.info, Node: resource, Next: select, Prev: re<5>, Up: Improved Modules<5> + +1.5.4.39 resource +................. + +New *note prlimit(): 89f. function, available on Linux platforms with a +kernel version of 2.6.36 or later and glibc of 2.13 or later, provides +the ability to query or set the resource limits for processes other than +the one making the call. (Contributed by Christian Heimes in +bpo-16595(1).) + +On Linux kernel version 2.6.36 or later, there are also some new Linux +specific constants: *note RLIMIT_MSGQUEUE: 8a0, *note RLIMIT_NICE: 8a1, +*note RLIMIT_RTPRIO: 8a2, *note RLIMIT_RTTIME: 8a3, and *note +RLIMIT_SIGPENDING: 8a4. (Contributed by Christian Heimes in +bpo-19324(2).) + +On FreeBSD version 9 and later, there some new FreeBSD specific +constants: *note RLIMIT_SBSIZE: 8a5, *note RLIMIT_SWAP: 8a6, and *note +RLIMIT_NPTS: 8a7. (Contributed by Claudiu Popa in bpo-19343(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16595 + + (2) https://bugs.python.org/issue19324 + + (3) https://bugs.python.org/issue19343 + + +File: python.info, Node: select, Next: shelve, Prev: resource, Up: Improved Modules<5> + +1.5.4.40 select +............... + +*note epoll: 8a9. objects now support the context management protocol. +When used in a *note with: 6e9. statement, the *note close(): 8aa. +method will be called automatically at the end of the block. +(Contributed by Serhiy Storchaka in bpo-16488(1).) + +*note devpoll: 8ab. objects now have *note fileno(): 8ac. and *note +close(): 8ad. methods, as well as a new attribute *note closed: 8ae. +(Contributed by Victor Stinner in bpo-18794(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16488 + + (2) https://bugs.python.org/issue18794 + + +File: python.info, Node: shelve, Next: shutil<3>, Prev: select, Up: Improved Modules<5> + +1.5.4.41 shelve +............... + +*note Shelf: 8b0. instances may now be used in *note with: 6e9. +statements, and will be automatically closed at the end of the ‘with’ +block. (Contributed by Filip Gruszczyński in bpo-13896(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13896 + + +File: python.info, Node: shutil<3>, Next: smtpd<2>, Prev: shelve, Up: Improved Modules<5> + +1.5.4.42 shutil +............... + +*note copyfile(): 258. now raises a specific *note Error: 8b2. subclass, +*note SameFileError: 8b3, when the source and destination are the same +file, which allows an application to take appropriate action on this +specific error. (Contributed by Atsuo Ishimoto and Hynek Schlawack in +bpo-1492704(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1492704 + + +File: python.info, Node: smtpd<2>, Next: smtplib<2>, Prev: shutil<3>, Up: Improved Modules<5> + +1.5.4.43 smtpd +.............. + +The *note SMTPServer: 602. and *note SMTPChannel: 601. classes now +accept a `map' keyword argument which, if specified, is passed in to +*note asynchat.async_chat: 8b5. as its `map' argument. This allows an +application to avoid affecting the global socket map. (Contributed by +Vinay Sajip in bpo-11959(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11959 + + +File: python.info, Node: smtplib<2>, Next: socket<6>, Prev: smtpd<2>, Up: Improved Modules<5> + +1.5.4.44 smtplib +................ + +*note SMTPException: 8b7. is now a subclass of *note OSError: 1d3, which +allows both socket level errors and SMTP protocol level errors to be +caught in one try/except statement by code that only cares whether or +not an error occurred. (Contributed by Ned Jackson Lovely in +bpo-2118(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2118 + + +File: python.info, Node: socket<6>, Next: sqlite3<4>, Prev: smtplib<2>, Up: Improved Modules<5> + +1.5.4.45 socket +............... + +The socket module now supports the *note CAN_BCM: 8b9. protocol on +platforms that support it. (Contributed by Brian Thorne in +bpo-15359(1).) + +Socket objects have new methods to get or set their *note inheritable +flag: 7fa, *note get_inheritable(): 7ff. and *note set_inheritable(): +800. + +The ‘socket.AF_*’ and ‘socket.SOCK_*’ constants are now enumeration +values using the new *note enum: 7b. module. This allows meaningful +names to be printed during debugging, instead of integer “magic +numbers”. + +The *note AF_LINK: 8ba. constant is now available on BSD and OSX. + +*note inet_pton(): 8bb. and *note inet_ntop(): 8bc. are now supported on +Windows. (Contributed by Atsuo Ishimoto in bpo-7171(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15359 + + (2) https://bugs.python.org/issue7171 + + +File: python.info, Node: sqlite3<4>, Next: ssl<7>, Prev: socket<6>, Up: Improved Modules<5> + +1.5.4.46 sqlite3 +................ + +A new boolean parameter to the *note connect(): 3da. function, `uri', +can be used to indicate that the `database' parameter is a ‘uri’ (see +the SQLite URI documentation(1)). (Contributed by poq in bpo-13773(2).) + + ---------- Footnotes ---------- + + (1) https://www.sqlite.org/uri.html + + (2) https://bugs.python.org/issue13773 + + +File: python.info, Node: ssl<7>, Next: stat, Prev: sqlite3<4>, Up: Improved Modules<5> + +1.5.4.47 ssl +............ + +*note PROTOCOL_TLSv1_1: 8bf. and *note PROTOCOL_TLSv1_2: 8c0. (TLSv1.1 +and TLSv1.2 support) have been added; support for these protocols is +only available if Python is linked with OpenSSL 1.0.1 or later. +(Contributed by Michele Orrù and Antoine Pitrou in bpo-16692(1).) New +function *note create_default_context(): 8c1. provides a standard way to +obtain an *note SSLContext: 3e4. whose settings are intended to be a +reasonable balance between compatibility and security. These settings +are more stringent than the defaults provided by the *note SSLContext: +3e4. constructor, and may be adjusted in the future, without prior +deprecation, if best-practice security requirements change. The new +recommended best practice for using stdlib libraries that support SSL is +to use *note create_default_context(): 8c1. to obtain an *note +SSLContext: 3e4. object, modify it if needed, and then pass it as the +`context' argument of the appropriate stdlib API. (Contributed by +Christian Heimes in bpo-19689(2).) + +*note SSLContext: 3e4. method *note load_verify_locations(): 7eb. +accepts a new optional argument `cadata', which can be used to provide +PEM or DER encoded certificates directly via strings or bytes, +respectively. (Contributed by Christian Heimes in bpo-18138(3).) + +New function *note get_default_verify_paths(): 8c2. returns a named +tuple of the paths and environment variables that the *note +set_default_verify_paths(): 8c3. method uses to set OpenSSL’s default +‘cafile’ and ‘capath’. This can be an aid in debugging default +verification issues. (Contributed by Christian Heimes in bpo-18143(4).) + +*note SSLContext: 3e4. has a new method, *note cert_store_stats(): 8c4, +that reports the number of loaded ‘X.509’ certs, ‘X.509 CA’ certs, and +certificate revocation lists (‘crl’s), as well as a *note +get_ca_certs(): 8c5. method that returns a list of the loaded ‘CA’ +certificates. (Contributed by Christian Heimes in bpo-18147(5).) + +If OpenSSL 0.9.8 or later is available, *note SSLContext: 3e4. has a new +attribute *note verify_flags: 8c6. that can be used to control the +certificate verification process by setting it to some combination of +the new constants *note VERIFY_DEFAULT: 8c7, *note +VERIFY_CRL_CHECK_LEAF: 8c8, *note VERIFY_CRL_CHECK_CHAIN: 8c9, or *note +VERIFY_X509_STRICT: 8ca. OpenSSL does not do any CRL verification by +default. (Contributed by Christien Heimes in bpo-8813(6).) + +New *note SSLContext: 3e4. method *note load_default_certs(): 8cb. loads +a set of default “certificate authority” (CA) certificates from default +locations, which vary according to the platform. It can be used to load +both TLS web server authentication certificates (‘purpose=’*note +SERVER_AUTH: 8cc.) for a client to use to verify a server, and +certificates for a server to use in verifying client certificates +(‘purpose=’*note CLIENT_AUTH: 8cd.). (Contributed by Christian Heimes +in bpo-19292(7).) Two new windows-only functions, *note +enum_certificates(): 8ce. and *note enum_crls(): 8cf. provide the +ability to retrieve certificates, certificate information, and CRLs from +the Windows cert store. (Contributed by Christian Heimes in +bpo-17134(8).) Support for server-side SNI (Server Name Indication) +using the new *note ssl.SSLContext.set_servername_callback(): 8d0. +method. (Contributed by Daniel Black in bpo-8109(9).) + +The dictionary returned by *note SSLSocket.getpeercert(): 8d1. contains +additional ‘X509v3’ extension items: ‘crlDistributionPoints’, +‘calIssuers’, and ‘OCSP’ URIs. (Contributed by Christian Heimes in +bpo-18379(10).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16692 + + (2) https://bugs.python.org/issue19689 + + (3) https://bugs.python.org/issue18138 + + (4) https://bugs.python.org/issue18143 + + (5) https://bugs.python.org/issue18147 + + (6) https://bugs.python.org/issue8813 + + (7) https://bugs.python.org/issue19292 + + (8) https://bugs.python.org/issue17134 + + (9) https://bugs.python.org/issue8109 + + (10) https://bugs.python.org/issue18379 + + +File: python.info, Node: stat, Next: struct<2>, Prev: ssl<7>, Up: Improved Modules<5> + +1.5.4.48 stat +............. + +The *note stat: f4. module is now backed by a C implementation in +‘_stat’. A C implementation is required as most of the values aren’t +standardized and are platform-dependent. (Contributed by Christian +Heimes in bpo-11016(1).) + +The module supports new *note ST_MODE: 8d3. flags, *note S_IFDOOR: 8d4, +*note S_IFPORT: 8d5, and *note S_IFWHT: 8d6. (Contributed by Christian +Hiemes in bpo-11016(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11016 + + (2) https://bugs.python.org/issue11016 + + +File: python.info, Node: struct<2>, Next: subprocess<4>, Prev: stat, Up: Improved Modules<5> + +1.5.4.49 struct +............... + +New function *note iter_unpack: 8d8. and a new *note +struct.Struct.iter_unpack(): 8d9. method on compiled formats provide +streamed unpacking of a buffer containing repeated instances of a given +format of data. (Contributed by Antoine Pitrou in bpo-17804(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17804 + + +File: python.info, Node: subprocess<4>, Next: sunau<2>, Prev: struct<2>, Up: Improved Modules<5> + +1.5.4.50 subprocess +................... + +*note check_output(): 8db. now accepts an `input' argument that can be +used to provide the contents of ‘stdin’ for the command that is run. +(Contributed by Zack Weinberg in bpo-16624(1).) + +‘getstatus()’ and *note getstatusoutput(): 8dc. now work on Windows. +This change was actually inadvertently made in 3.3.4. (Contributed by +Tim Golden in bpo-10197(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16624 + + (2) https://bugs.python.org/issue10197 + + +File: python.info, Node: sunau<2>, Next: sys<6>, Prev: subprocess<4>, Up: Improved Modules<5> + +1.5.4.51 sunau +.............. + +The ‘getparams()’ method now returns a namedtuple rather than a plain +tuple. (Contributed by Claudiu Popa in bpo-18901(1).) + +*note sunau.open(): 472. now supports the context management protocol: +when used in a *note with: 6e9. block, the ‘close’ method of the +returned object will be called automatically at the end of the block. +(Contributed by Serhiy Storchaka in bpo-18878(2).) + +*note AU_write.setsampwidth(): 8de. now supports 24 bit samples, thus +adding support for writing 24 sample using the module. (Contributed by +Serhiy Storchaka in bpo-19261(3).) + +The *note writeframesraw(): 8df. and *note writeframes(): 8e0. methods +now accept any *note bytes-like object: 5e8. (Contributed by Serhiy +Storchaka in bpo-8311(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18901 + + (2) https://bugs.python.org/issue18878 + + (3) https://bugs.python.org/issue19261 + + (4) https://bugs.python.org/issue8311 + + +File: python.info, Node: sys<6>, Next: tarfile<3>, Prev: sunau<2>, Up: Improved Modules<5> + +1.5.4.52 sys +............ + +New function *note sys.getallocatedblocks(): 8e2. returns the current +number of blocks allocated by the interpreter. (In CPython with the +default ‘--with-pymalloc’ setting, this is allocations made through the +*note PyObject_Malloc(): 508. API.) This can be useful for tracking +memory leaks, especially if automated via a test suite. (Contributed by +Antoine Pitrou in bpo-13390(1).) + +When the Python interpreter starts in *note interactive mode: 8e3, it +checks for an *note __interactivehook__: 8e4. attribute on the *note +sys: fd. module. If the attribute exists, its value is called with no +arguments just before interactive mode is started. The check is made +after the *note PYTHONSTARTUP: 8e5. file is read, so it can be set +there. The *note site: eb. module *note sets it: 8e6. to a function +that enables tab completion and history saving (in ‘~/.python-history’) +if the platform supports *note readline: de. If you do not want this +(new) behavior, you can override it in *note PYTHONSTARTUP: 8e5, +‘sitecustomize’, or ‘usercustomize’ by deleting this attribute from +*note sys: fd. (or setting it to some other callable). (Contributed by +Éric Araujo and Antoine Pitrou in bpo-5845(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13390 + + (2) https://bugs.python.org/issue5845 + + +File: python.info, Node: tarfile<3>, Next: textwrap, Prev: sys<6>, Up: Improved Modules<5> + +1.5.4.53 tarfile +................ + +The *note tarfile: 101. module now supports a simple *note Command-Line +Interface: 8e8. when called as a script directly or via ‘-m’. This can +be used to create and extract tarfile archives. (Contributed by Berker +Peksag in bpo-13477(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13477 + + +File: python.info, Node: textwrap, Next: threading<4>, Prev: tarfile<3>, Up: Improved Modules<5> + +1.5.4.54 textwrap +................. + +The *note TextWrapper: 8ea. class has two new attributes/constructor +arguments: *note max_lines: 8eb, which limits the number of lines in the +output, and *note placeholder: 8ec, which is a string that will appear +at the end of the output if it has been truncated because of +`max_lines'. Building on these capabilities, a new convenience function +*note shorten(): 8ed. collapses all of the whitespace in the input to +single spaces and produces a single line of a given `width' that ends +with the `placeholder' (by default, ‘[...]’). (Contributed by Antoine +Pitrou and Serhiy Storchaka in bpo-18585(1) and bpo-18725(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18585 + + (2) https://bugs.python.org/issue18725 + + +File: python.info, Node: threading<4>, Next: traceback<3>, Prev: textwrap, Up: Improved Modules<5> + +1.5.4.55 threading +.................. + +The *note Thread: 235. object representing the main thread can be +obtained from the new *note main_thread(): 8ef. function. In normal +conditions this will be the thread from which the Python interpreter was +started. (Contributed by Andrew Svetlov in bpo-18882(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18882 + + +File: python.info, Node: traceback<3>, Next: types<3>, Prev: threading<4>, Up: Improved Modules<5> + +1.5.4.56 traceback +.................. + +A new *note traceback.clear_frames(): 8f1. function takes a traceback +object and clears the local variables in all of the frames it +references, reducing the amount of memory consumed. (Contributed by +Andrew Kuchling in bpo-1565525(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1565525 + + +File: python.info, Node: types<3>, Next: urllib<2>, Prev: traceback<3>, Up: Improved Modules<5> + +1.5.4.57 types +.............. + +A new *note DynamicClassAttribute(): 8f3. descriptor provides a way to +define an attribute that acts normally when looked up through an +instance object, but which is routed to the `class' ‘__getattr__’ when +looked up through the class. This allows one to have properties active +on a class, and have virtual attributes on the class with the same name +(see ‘Enum’ for an example). (Contributed by Ethan Furman in +bpo-19030(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19030 + + +File: python.info, Node: urllib<2>, Next: unittest<4>, Prev: types<3>, Up: Improved Modules<5> + +1.5.4.58 urllib +............... + +*note urllib.request: 120. now supports ‘data:’ URLs via the *note +DataHandler: 8f5. class. (Contributed by Mathias Panzenböck in +bpo-16423(1).) + +The http method that will be used by a *note Request: 8f6. class can now +be specified by setting a *note method: 8f7. class attribute on the +subclass. (Contributed by Jason R Coombs in bpo-18978(2).) + +*note Request: 8f6. objects are now reusable: if the *note full_url: +8f8. or *note data: 8f9. attributes are modified, all relevant internal +properties are updated. This means, for example, that it is now +possible to use the same *note Request: 8f6. object in more than one +*note OpenerDirector.open(): 8fa. call with different `data' arguments, +or to modify a *note Request: 8f6.’s ‘url’ rather than recomputing it +from scratch. There is also a new *note remove_header(): 8fb. method +that can be used to remove headers from a *note Request: 8f6. +(Contributed by Alexey Kachayev in bpo-16464(3), Daniel Wozniak in +bpo-17485(4), and Damien Brecht and Senthil Kumaran in bpo-17272(5).) + +*note HTTPError: 8fc. objects now have a *note headers: 8fd. attribute +that provides access to the HTTP response headers associated with the +error. (Contributed by Berker Peksag in bpo-15701(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16423 + + (2) https://bugs.python.org/issue18978 + + (3) https://bugs.python.org/issue16464 + + (4) https://bugs.python.org/issue17485 + + (5) https://bugs.python.org/issue17272 + + (6) https://bugs.python.org/issue15701 + + +File: python.info, Node: unittest<4>, Next: venv<4>, Prev: urllib<2>, Up: Improved Modules<5> + +1.5.4.59 unittest +................. + +The *note TestCase: 8ff. class has a new method, *note subTest(): 900, +that produces a context manager whose *note with: 6e9. block becomes a +“sub-test”. This context manager allows a test method to dynamically +generate subtests by, say, calling the ‘subTest’ context manager inside +a loop. A single test method can thereby produce an indefinite number +of separately-identified and separately-counted tests, all of which will +run even if one or more of them fail. For example: + + class NumbersTest(unittest.TestCase): + def test_even(self): + for i in range(6): + with self.subTest(i=i): + self.assertEqual(i % 2, 0) + +will result in six subtests, each identified in the unittest verbose +output with a label consisting of the variable name ‘i’ and a particular +value for that variable (‘i=0’, ‘i=1’, etc). See *note Distinguishing +test iterations using subtests: 901. for the full version of this +example. (Contributed by Antoine Pitrou in bpo-16997(1).) + +*note unittest.main(): 902. now accepts an iterable of test names for +`defaultTest', where previously it only accepted a single test name as a +string. (Contributed by Jyrki Pulliainen in bpo-15132(2).) + +If *note SkipTest: 903. is raised during test discovery (that is, at the +module level in the test file), it is now reported as a skip instead of +an error. (Contributed by Zach Ware in bpo-16935(3).) + +*note discover(): 904. now sorts the discovered files to provide +consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani +in bpo-16709(4).) + +*note TestSuite: 6ce. now drops references to tests as soon as the test +has been run, if the test is successful. On Python interpreters that do +garbage collection, this allows the tests to be garbage collected if +nothing else is holding a reference to the test. It is possible to +override this behavior by creating a *note TestSuite: 6ce. subclass that +defines a custom ‘_removeTestAtIndex’ method. (Contributed by Tom +Wardill, Matt McClure, and Andrew Svetlov in bpo-11798(5).) + +A new test assertion context-manager, *note assertLogs(): 905, will +ensure that a given block of code emits a log message using the *note +logging: aa. module. By default the message can come from any logger +and have a priority of ‘INFO’ or higher, but both the logger name and an +alternative minimum logging level may be specified. The object returned +by the context manager can be queried for the *note LogRecord: 906.s +and/or formatted messages that were logged. (Contributed by Antoine +Pitrou in bpo-18937(6).) + +Test discovery now works with namespace packages (Contributed by Claudiu +Popa in bpo-17457(7).) + +*note unittest.mock: 11c. objects now inspect their specification +signatures when matching calls, which means an argument can now be +matched by either position or name, instead of only by position. +(Contributed by Antoine Pitrou in bpo-17015(8).) + +‘mock_open()’ objects now have ‘readline’ and ‘readlines’ methods. +(Contributed by Toshio Kuratomi in bpo-17467(9).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16997 + + (2) https://bugs.python.org/issue15132 + + (3) https://bugs.python.org/issue16935 + + (4) https://bugs.python.org/issue16709 + + (5) https://bugs.python.org/issue11798 + + (6) https://bugs.python.org/issue18937 + + (7) https://bugs.python.org/issue17457 + + (8) https://bugs.python.org/issue17015 + + (9) https://bugs.python.org/issue17467 + + +File: python.info, Node: venv<4>, Next: wave<2>, Prev: unittest<4>, Up: Improved Modules<5> + +1.5.4.60 venv +............. + +*note venv: 125. now includes activation scripts for the ‘csh’ and +‘fish’ shells. (Contributed by Andrew Svetlov in bpo-15417(1).) + +*note EnvBuilder: 908. and the *note create(): 909. convenience function +take a new keyword argument `with_pip', which defaults to ‘False’, that +controls whether or not *note EnvBuilder: 908. ensures that ‘pip’ is +installed in the virtual environment. (Contributed by Nick Coghlan in +bpo-19552(2) as part of the PEP 453(3) implementation.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15417 + + (2) https://bugs.python.org/issue19552 + + (3) https://www.python.org/dev/peps/pep-0453 + + +File: python.info, Node: wave<2>, Next: weakref<2>, Prev: venv<4>, Up: Improved Modules<5> + +1.5.4.61 wave +............. + +The ‘getparams()’ method now returns a namedtuple rather than a plain +tuple. (Contributed by Claudiu Popa in bpo-17487(1).) + +*note wave.open(): 476. now supports the context management protocol. +(Contributed by Claudiu Popa in bpo-17616(2).) + +*note wave: 127. can now *note write output to unseekable files: 90b. +(Contributed by David Jones, Guilherme Polo, and Serhiy Storchaka in +bpo-5202(3).) + +The *note writeframesraw(): 90c. and *note writeframes(): 90d. methods +now accept any *note bytes-like object: 5e8. (Contributed by Serhiy +Storchaka in bpo-8311(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17487 + + (2) https://bugs.python.org/issue17616 + + (3) https://bugs.python.org/issue5202 + + (4) https://bugs.python.org/issue8311 + + +File: python.info, Node: weakref<2>, Next: xml etree<2>, Prev: wave<2>, Up: Improved Modules<5> + +1.5.4.62 weakref +................ + +New *note WeakMethod: 90f. class simulates weak references to bound +methods. (Contributed by Antoine Pitrou in bpo-14631(1).) + +New *note finalize: 29d. class makes it possible to register a callback +to be invoked when an object is garbage collected, without needing to +carefully manage the lifecycle of the weak reference itself. +(Contributed by Richard Oudkerk in bpo-15528(2).) + +The callback, if any, associated with a *note ref: 910. is now exposed +via the *note __callback__: 911. attribute. (Contributed by Mark +Dickinson in bpo-17643(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14631 + + (2) https://bugs.python.org/issue15528 + + (3) https://bugs.python.org/issue17643 + + +File: python.info, Node: xml etree<2>, Next: zipfile<4>, Prev: weakref<2>, Up: Improved Modules<5> + +1.5.4.63 xml.etree +.................. + +A new parser, *note XMLPullParser: 913, allows a non-blocking +applications to parse XML documents. An example can be seen at *note +Pull API for non-blocking parsing: 914. (Contributed by Antoine Pitrou +in bpo-17741(1).) + +The *note xml.etree.ElementTree: 137. *note tostring(): 915. and *note +tostringlist(): 916. functions, and the *note ElementTree: 917. *note +write(): 918. method, now have a `short_empty_elements' *note +keyword-only parameter: 2ac. providing control over whether elements +with no content are written in abbreviated (‘’) or expanded +(‘’) form. (Contributed by Ariel Poliak and Serhiy Storchaka +in bpo-14377(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17741 + + (2) https://bugs.python.org/issue14377 + + +File: python.info, Node: zipfile<4>, Prev: xml etree<2>, Up: Improved Modules<5> + +1.5.4.64 zipfile +................ + +The *note writepy(): 91a. method of the *note PyZipFile: 91b. class has +a new `filterfunc' option that can be used to control which directories +and files are added to the archive. For example, this could be used to +exclude test files from the archive. (Contributed by Christian Tismer +in bpo-19274(1).) + +The `allowZip64' parameter to *note ZipFile: 41f. and ‘PyZipfile’ is now +‘True’ by default. (Contributed by William Mallard in bpo-17201(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue19274 + + (2) https://bugs.python.org/issue17201 + + +File: python.info, Node: CPython Implementation Changes, Next: Deprecated<4>, Prev: Improved Modules<5>, Up: What’s New In Python 3 4 + +1.5.5 CPython Implementation Changes +------------------------------------ + +* Menu: + +* PEP 445; Customization of CPython Memory Allocators: PEP 445 Customization of CPython Memory Allocators. +* PEP 442; Safe Object Finalization: PEP 442 Safe Object Finalization. +* PEP 456; Secure and Interchangeable Hash Algorithm: PEP 456 Secure and Interchangeable Hash Algorithm. +* PEP 436; Argument Clinic: PEP 436 Argument Clinic. +* Other Build and C API Changes:: +* Other Improvements: Other Improvements<2>. +* Significant Optimizations:: + + +File: python.info, Node: PEP 445 Customization of CPython Memory Allocators, Next: PEP 442 Safe Object Finalization, Up: CPython Implementation Changes + +1.5.5.1 PEP 445: Customization of CPython Memory Allocators +........................................................... + +PEP 445(1) adds new C level interfaces to customize memory allocation in +the CPython interpreter. + +See also +........ + +PEP 445(2) – Add new APIs to customize Python memory allocators + + PEP written and implemented by Victor Stinner. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0445 + + (2) https://www.python.org/dev/peps/pep-0445 + + +File: python.info, Node: PEP 442 Safe Object Finalization, Next: PEP 456 Secure and Interchangeable Hash Algorithm, Prev: PEP 445 Customization of CPython Memory Allocators, Up: CPython Implementation Changes + +1.5.5.2 PEP 442: Safe Object Finalization +......................................... + +PEP 442(1) removes the current limitations and quirks of object +finalization in CPython. With it, objects with *note __del__(): 91f. +methods, as well as generators with *note finally: 182. clauses, can be +finalized when they are part of a reference cycle. + +As part of this change, module globals are no longer forcibly set to +*note None: 157. during interpreter shutdown in most cases, instead +relying on the normal operation of the cyclic garbage collector. This +avoids a whole class of interpreter-shutdown-time errors, usually +involving ‘__del__’ methods, that have plagued Python since the cyclic +GC was first introduced. + +See also +........ + +PEP 442(2) – Safe object finalization + + PEP written and implemented by Antoine Pitrou. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0442 + + (2) https://www.python.org/dev/peps/pep-0442 + + +File: python.info, Node: PEP 456 Secure and Interchangeable Hash Algorithm, Next: PEP 436 Argument Clinic, Prev: PEP 442 Safe Object Finalization, Up: CPython Implementation Changes + +1.5.5.3 PEP 456: Secure and Interchangeable Hash Algorithm +.......................................................... + +PEP 456(1) follows up on earlier security fix work done on Python’s hash +algorithm to address certain DOS attacks to which public facing APIs +backed by dictionary lookups may be subject. (See bpo-14621(2) for the +start of the current round of improvements.) The PEP unifies CPython’s +hash code to make it easier for a packager to substitute a different +hash algorithm, and switches Python’s default implementation to a +SipHash implementation on platforms that have a 64 bit data type. Any +performance differences in comparison with the older FNV algorithm are +trivial. + +The PEP adds additional fields to the *note sys.hash_info: 921. named +tuple to describe the hash algorithm in use by the currently executing +binary. Otherwise, the PEP does not alter any existing CPython APIs. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0456 + + (2) https://bugs.python.org/issue14621 + + +File: python.info, Node: PEP 436 Argument Clinic, Next: Other Build and C API Changes, Prev: PEP 456 Secure and Interchangeable Hash Algorithm, Up: CPython Implementation Changes + +1.5.5.4 PEP 436: Argument Clinic +................................ + +“Argument Clinic” ( PEP 436(1)) is now part of the CPython build process +and can be used to simplify the process of defining and maintaining +accurate signatures for builtins and standard library extension modules +implemented in C. + +Some standard library extension modules have been converted to use +Argument Clinic in Python 3.4, and *note pydoc: d9. and *note inspect: +a0. have been updated accordingly. + +It is expected that signature metadata for programmatic introspection +will be added to additional callables implemented in C as part of Python +3.4 maintenance releases. + + Note: The Argument Clinic PEP is not fully up to date with the + state of the implementation. This has been deemed acceptable by + the release manager and core development team in this case, as + Argument Clinic will not be made available as a public API for + third party use in Python 3.4. + +See also +........ + +PEP 436(2) – The Argument Clinic DSL + + PEP written and implemented by Larry Hastings. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0436 + + (2) https://www.python.org/dev/peps/pep-0436 + + +File: python.info, Node: Other Build and C API Changes, Next: Other Improvements<2>, Prev: PEP 436 Argument Clinic, Up: CPython Implementation Changes + +1.5.5.5 Other Build and C API Changes +..................................... + + * The new *note PyType_GetSlot(): 924. function has been added to the + stable ABI, allowing retrieval of function pointers from named type + slots when using the limited API. (Contributed by Martin von Löwis + in bpo-17162(1).) + + * The new *note Py_SetStandardStreamEncoding(): 925. + pre-initialization API allows applications embedding the CPython + interpreter to reliably force a particular encoding and error + handler for the standard streams. (Contributed by Bastien Montagne + and Nick Coghlan in bpo-16129(2).) + + * Most Python C APIs that don’t mutate string arguments are now + correctly marked as accepting ‘const char *’ rather than ‘char *’. + (Contributed by Serhiy Storchaka in bpo-1772673(3).) + + * A new shell version of ‘python-config’ can be used even when a + python interpreter is not available (for example, in cross + compilation scenarios). + + * *note PyUnicode_FromFormat(): 7cb. now supports width and precision + specifications for ‘%s’, ‘%A’, ‘%U’, ‘%V’, ‘%S’, and ‘%R’. + (Contributed by Ysj Ray and Victor Stinner in bpo-7330(4).) + + * New function *note PyStructSequence_InitType2(): 926. supplements + the existing *note PyStructSequence_InitType(): 927. function. The + difference is that it returns ‘0’ on success and ‘-1’ on failure. + + * The CPython source can now be compiled using the address sanity + checking features of recent versions of GCC and clang: the false + alarms in the small object allocator have been silenced. + (Contributed by Dhiru Kholia in bpo-18596(5).) + + * The Windows build now uses Address Space Layout Randomization(6) + and Data Execution Prevention(7). (Contributed by Christian Heimes + in bpo-16632(8).) + + * New function *note PyObject_LengthHint(): 928. is the C API + equivalent of *note operator.length_hint(): 87d. (Contributed by + Armin Ronacher in bpo-16148(9).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17162 + + (2) https://bugs.python.org/issue16129 + + (3) https://bugs.python.org/issue1772673 + + (4) https://bugs.python.org/issue7330 + + (5) https://bugs.python.org/issue18596 + + (6) https://en.wikipedia.org/wiki/Address_space_layout_randomization + + (7) https://en.wikipedia.org/wiki/Data_Execution_Prevention + + (8) https://bugs.python.org/issue16632 + + (9) https://bugs.python.org/issue16148 + + +File: python.info, Node: Other Improvements<2>, Next: Significant Optimizations, Prev: Other Build and C API Changes, Up: CPython Implementation Changes + +1.5.5.6 Other Improvements +.......................... + + * The *note python: 92b. command has a new *note option: 92c, ‘-I’, + which causes it to run in “isolated mode”, which means that *note + sys.path: 488. contains neither the script’s directory nor the + user’s ‘site-packages’ directory, and all ‘PYTHON*’ environment + variables are ignored (it implies both ‘-s’ and ‘-E’). Other + restrictions may also be applied in the future, with the goal being + to isolate the execution of a script from the user’s environment. + This is appropriate, for example, when Python is used to run a + system script. On most POSIX systems it can and should be used in + the ‘#!’ line of system scripts. (Contributed by Christian Heimes + in bpo-16499(1).) + + * Tab-completion is now enabled by default in the interactive + interpreter on systems that support *note readline: de. History is + also enabled by default, and is written to (and read from) the file + ‘~/.python-history’. (Contributed by Antoine Pitrou and Éric + Araujo in bpo-5845(2).) + + * Invoking the Python interpreter with ‘--version’ now outputs the + version to standard output instead of standard error + (bpo-18338(3)). Similar changes were made to *note argparse: 6. + (bpo-18920(4)) and other modules that have script-like invocation + capabilities (bpo-18922(5)). + + * The CPython Windows installer now adds ‘.py’ to the ‘PATHEXT’ + variable when extensions are registered, allowing users to run a + python script at the windows command prompt by just typing its name + without the ‘.py’ extension. (Contributed by Paul Moore in + bpo-18569(6).) + + * A new ‘make’ target coverage-report(7) will build python, run the + test suite, and generate an HTML coverage report for the C codebase + using ‘gcov’ and lcov(8). + + * The ‘-R’ option to the *note python regression test suite: 92d. now + also checks for memory allocation leaks, using *note + sys.getallocatedblocks(): 8e2. (Contributed by Antoine Pitrou in + bpo-13390(9).) + + * ‘python -m’ now works with namespace packages. + + * The *note stat: f4. module is now implemented in C, which means it + gets the values for its constants from the C header files, instead + of having the values hard-coded in the python module as was + previously the case. + + * Loading multiple python modules from a single OS module (‘.so’, + ‘.dll’) now works correctly (previously it silently returned the + first python module in the file). (Contributed by Václav Šmilauer + in bpo-16421(10).) + + * A new opcode, *note LOAD_CLASSDEREF: 92e, has been added to fix a + bug in the loading of free variables in class bodies that could be + triggered by certain uses of *note __prepare__: 4fd. (Contributed + by Benjamin Peterson in bpo-17853(11).) + + * A number of MemoryError-related crashes were identified and fixed + by Victor Stinner using his PEP 445(12)-based ‘pyfailmalloc’ tool + (bpo-18408(13), bpo-18520(14)). + + * The ‘pyvenv’ command now accepts a ‘--copies’ option to use copies + rather than symlinks even on systems where symlinks are the + default. (Contributed by Vinay Sajip in bpo-18807(15).) + + * The ‘pyvenv’ command also accepts a ‘--without-pip’ option to + suppress the otherwise-automatic bootstrapping of pip into the + virtual environment. (Contributed by Nick Coghlan in bpo-19552(16) + as part of the PEP 453(17) implementation.) + + * The encoding name is now optional in the value set for the *note + PYTHONIOENCODING: 92f. environment variable. This makes it + possible to set just the error handler, without changing the + default encoding. (Contributed by Serhiy Storchaka in + bpo-18818(18).) + + * The *note bz2: 14, *note lzma: ad, and *note gzip: 8c. module + ‘open’ functions now support ‘x’ (exclusive creation) mode. + (Contributed by Tim Heaney and Vajrasky Kok in bpo-19201(19), + bpo-19222(20), and bpo-19223(21).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16499 + + (2) https://bugs.python.org/issue5845 + + (3) https://bugs.python.org/issue18338 + + (4) https://bugs.python.org/issue18920 + + (5) https://bugs.python.org/issue18922 + + (6) https://bugs.python.org/issue18569 + + (7) +https://devguide.python.org/coverage/#measuring-coverage-of-c-code-with-gcov-and-lcov + + (8) http://ltp.sourceforge.net/coverage/lcov.php + + (9) https://bugs.python.org/issue13390 + + (10) https://bugs.python.org/issue16421 + + (11) https://bugs.python.org/issue17853 + + (12) https://www.python.org/dev/peps/pep-0445 + + (13) https://bugs.python.org/issue18408 + + (14) https://bugs.python.org/issue18520 + + (15) https://bugs.python.org/issue18807 + + (16) https://bugs.python.org/issue19552 + + (17) https://www.python.org/dev/peps/pep-0453 + + (18) https://bugs.python.org/issue18818 + + (19) https://bugs.python.org/issue19201 + + (20) https://bugs.python.org/issue19222 + + (21) https://bugs.python.org/issue19223 + + +File: python.info, Node: Significant Optimizations, Prev: Other Improvements<2>, Up: CPython Implementation Changes + +1.5.5.7 Significant Optimizations +................................. + + * The UTF-32 decoder is now 3x to 4x faster. (Contributed by Serhiy + Storchaka in bpo-14625(1).) + + * The cost of hash collisions for sets is now reduced. Each hash + table probe now checks a series of consecutive, adjacent key/hash + pairs before continuing to make random probes through the hash + table. This exploits cache locality to make collision resolution + less expensive. The collision resolution scheme can be described + as a hybrid of linear probing and open addressing. The number of + additional linear probes defaults to nine. This can be changed at + compile-time by defining LINEAR_PROBES to be any value. Set + LINEAR_PROBES=0 to turn-off linear probing entirely. (Contributed + by Raymond Hettinger in bpo-18771(2).) + + * The interpreter starts about 30% faster. A couple of measures lead + to the speedup. The interpreter loads fewer modules on startup, + e.g. the *note re: dd, *note collections: 1e. and *note locale: + a9. modules and their dependencies are no longer imported by + default. The marshal module has been improved to load compiled + Python code faster. (Contributed by Antoine Pitrou, Christian + Heimes and Victor Stinner in bpo-19219(3), bpo-19218(4), + bpo-19209(5), bpo-19205(6) and bpo-9548(7).) + + * *note bz2.BZ2File: 931. is now as fast or faster than the Python2 + version for most cases. *note lzma.LZMAFile: 932. has also been + optimized. (Contributed by Serhiy Storchaka and Nadeem Vawda in + bpo-16034(8).) + + * *note random.getrandbits(): 933. is 20%-40% faster for small + integers (the most common use case). (Contributed by Serhiy + Storchaka in bpo-16674(9).) + + * By taking advantage of the new storage format for strings, pickling + of strings is now significantly faster. (Contributed by Victor + Stinner and Antoine Pitrou in bpo-15596(10).) + + * A performance issue in ‘io.FileIO.readall()’ has been solved. This + particularly affects Windows, and significantly speeds up the case + of piping significant amounts of data through *note subprocess: f9. + (Contributed by Richard Oudkerk in bpo-15758(11).) + + * *note html.escape(): 934. is now 10x faster. (Contributed by Matt + Bryant in bpo-18020(12).) + + * On Windows, the native ‘VirtualAlloc’ is now used instead of the + CRT ‘malloc’ in ‘obmalloc’. Artificial benchmarks show about a 3% + memory savings. + + * *note os.urandom(): 4d1. now uses a lazily-opened persistent file + descriptor so as to avoid using many file descriptors when run in + parallel from multiple threads. (Contributed by Antoine Pitrou in + bpo-18756(13).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14625 + + (2) https://bugs.python.org/issue18771 + + (3) https://bugs.python.org/issue19219 + + (4) https://bugs.python.org/issue19218 + + (5) https://bugs.python.org/issue19209 + + (6) https://bugs.python.org/issue19205 + + (7) https://bugs.python.org/issue9548 + + (8) https://bugs.python.org/issue16034 + + (9) https://bugs.python.org/issue16674 + + (10) https://bugs.python.org/issue15596 + + (11) https://bugs.python.org/issue15758 + + (12) https://bugs.python.org/issue18020 + + (13) https://bugs.python.org/issue18756 + + +File: python.info, Node: Deprecated<4>, Next: Removed<3>, Prev: CPython Implementation Changes, Up: What’s New In Python 3 4 + +1.5.6 Deprecated +---------------- + +This section covers various APIs and other features that have been +deprecated in Python 3.4, and will be removed in Python 3.5 or later. +In most (but not all) cases, using the deprecated APIs will produce a +*note DeprecationWarning: 278. when the interpreter is run with +deprecation warnings enabled (for example, by using ‘-Wd’). + +* Menu: + +* Deprecations in the Python API:: +* Deprecated Features:: + + +File: python.info, Node: Deprecations in the Python API, Next: Deprecated Features, Up: Deprecated<4> + +1.5.6.1 Deprecations in the Python API +...................................... + + * As mentioned in *note PEP 451; A ModuleSpec Type for the Import + System: 7d8, a number of *note importlib: 9b. methods and functions + are deprecated: *note importlib.find_loader(): 937. is replaced by + *note importlib.util.find_spec(): 938.; *note + importlib.machinery.PathFinder.find_module(): 939. is replaced by + *note importlib.machinery.PathFinder.find_spec(): 93a.; *note + importlib.abc.MetaPathFinder.find_module(): 462. is replaced by + *note importlib.abc.MetaPathFinder.find_spec(): 463.; *note + importlib.abc.PathEntryFinder.find_loader(): 464. and *note + find_module(): 93b. are replaced by *note + importlib.abc.PathEntryFinder.find_spec(): 465.; all of the + ‘xxxLoader’ ABC ‘load_module’ methods (*note + importlib.abc.Loader.load_module(): 5e3, *note + importlib.abc.InspectLoader.load_module(): 93c, *note + importlib.abc.FileLoader.load_module(): 93d, *note + importlib.abc.SourceLoader.load_module(): 93e.) should no longer be + implemented, instead loaders should implement an ‘exec_module’ + method (*note importlib.abc.Loader.exec_module(): 5e4, *note + importlib.abc.InspectLoader.exec_module(): 93f. *note + importlib.abc.SourceLoader.exec_module(): 940.) and let the import + system take care of the rest; and *note + importlib.abc.Loader.module_repr(): 941, *note + importlib.util.module_for_loader(): 942, *note + importlib.util.set_loader(): 943, and *note + importlib.util.set_package(): 944. are no longer needed because + their functions are now handled automatically by the import system. + + * The *note imp: 9a. module is pending deprecation. To keep + compatibility with Python 2/3 code bases, the module’s removal is + currently not scheduled. + + * The *note formatter: 82. module is pending deprecation and is + slated for removal in Python 3.6. + + * ‘MD5’ as the default `digestmod' for the *note hmac.new(): 854. + function is deprecated. Python 3.6 will require an explicit digest + name or constructor as `digestmod' argument. + + * The internal ‘Netrc’ class in the *note ftplib: 84. module has been + documented as deprecated in its docstring for quite some time. It + now emits a *note DeprecationWarning: 278. and will be removed + completely in Python 3.5. + + * The undocumented `endtime' argument to *note + subprocess.Popen.wait(): 592. should not have been exposed and is + hopefully not in use; it is deprecated and will mostly likely be + removed in Python 3.5. + + * The `strict' argument of *note HTMLParser: 7bf. is deprecated. + + * The *note plistlib: cf. *note readPlist(): 47f, *note writePlist(): + 945, *note readPlistFromBytes(): 480, and *note + writePlistToBytes(): 946. functions are deprecated in favor of the + corresponding new functions *note load(): 88b, *note dump(): 88c, + *note loads(): 88d, and *note dumps(): 88e. *note Data(): 947. is + deprecated in favor of just using the *note bytes: 331. + constructor. + + * The *note sysconfig: fe. key ‘SO’ is deprecated, it has been + replaced by ‘EXT_SUFFIX’. + + * The ‘U’ mode accepted by various ‘open’ functions is deprecated. + In Python3 it does not do anything useful, and should be replaced + by appropriate uses of *note io.TextIOWrapper: 5f3. (if needed) and + its `newline' argument. + + * The `parser' argument of *note xml.etree.ElementTree.iterparse(): + 948. has been deprecated, as has the `html' argument of *note + XMLParser(): 252. To prepare for the removal of the latter, all + arguments to ‘XMLParser’ should be passed by keyword. + + +File: python.info, Node: Deprecated Features, Prev: Deprecations in the Python API, Up: Deprecated<4> + +1.5.6.2 Deprecated Features +........................... + + * Running *note IDLE: 94a. with the ‘-n’ flag (no subprocess) is + deprecated. However, the feature will not be removed until + bpo-18823(1) is resolved. + + * The site module adding a “site-python” directory to sys.path, if it + exists, is deprecated (bpo-19375(2)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18823 + + (2) https://bugs.python.org/issue19375 + + +File: python.info, Node: Removed<3>, Next: Porting to Python 3 4, Prev: Deprecated<4>, Up: What’s New In Python 3 4 + +1.5.7 Removed +------------- + +* Menu: + +* Operating Systems No Longer Supported:: +* API and Feature Removals: API and Feature Removals<5>. +* Code Cleanups:: + + +File: python.info, Node: Operating Systems No Longer Supported, Next: API and Feature Removals<5>, Up: Removed<3> + +1.5.7.1 Operating Systems No Longer Supported +............................................. + +Support for the following operating systems has been removed from the +source and build tools: + + * OS/2 (bpo-16135(1)). + + * Windows 2000 (changeset e52df05b496a). + + * Windows systems where ‘COMSPEC’ points to ‘command.com’ + (bpo-14470(2)). + + * VMS (bpo-16136(3)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16135 + + (2) https://bugs.python.org/issue14470 + + (3) https://bugs.python.org/issue16136 + + +File: python.info, Node: API and Feature Removals<5>, Next: Code Cleanups, Prev: Operating Systems No Longer Supported, Up: Removed<3> + +1.5.7.2 API and Feature Removals +................................ + +The following obsolete and previously deprecated APIs and features have +been removed: + + * The unmaintained ‘Misc/TextMate’ and ‘Misc/vim’ directories have + been removed (see the devguide(1) for suggestions on what to use + instead). + + * The ‘SO’ makefile macro is removed (it was replaced by the + ‘SHLIB_SUFFIX’ and ‘EXT_SUFFIX’ macros) (bpo-16754(2)). + + * The ‘PyThreadState.tick_counter’ field has been removed; its value + has been meaningless since Python 3.2, when the “new GIL” was + introduced (bpo-19199(3)). + + * ‘PyLoader’ and ‘PyPycLoader’ have been removed from *note + importlib: 9b. (Contributed by Taras Lyapun in bpo-15641(4).) + + * The `strict' argument to *note HTTPConnection: 392. and *note + HTTPSConnection: 393. has been removed. HTTP 0.9-style “Simple + Responses” are no longer supported. + + * The deprecated *note urllib.request.Request: 8f6. getter and setter + methods ‘add_data’, ‘has_data’, ‘get_data’, ‘get_type’, ‘get_host’, + ‘get_selector’, ‘set_proxy’, ‘get_origin_req_host’, and + ‘is_unverifiable’ have been removed (use direct attribute access + instead). + + * Support for loading the deprecated ‘TYPE_INT64’ has been removed + from *note marshal: b0. (Contributed by Dan Riti in bpo-15480(5).) + + * *note inspect.Signature: 6f3.: positional-only parameters are now + required to have a valid name. + + * *note object.__format__(): 94e. no longer accepts non-empty format + strings, it now raises a *note TypeError: 192. instead. Using a + non-empty string has been deprecated since Python 3.2. This change + has been made to prevent a situation where previously working (but + incorrect) code would start failing if an object gained a + __format__ method, which means that your code may now raise a *note + TypeError: 192. if you are using an ‘'s'’ format code with objects + that do not have a __format__ method that handles it. See + bpo-7994(6) for background. + + * ‘difflib.SequenceMatcher.isbjunk()’ and + ‘difflib.SequenceMatcher.isbpopular()’ were deprecated in 3.2, and + have now been removed: use ‘x in sm.bjunk’ and ‘x in sm.bpopular’, + where `sm' is a *note SequenceMatcher: 94f. object (bpo-13248(7)). + + ---------- Footnotes ---------- + + (1) https://devguide.python.org + + (2) https://bugs.python.org/issue16754 + + (3) https://bugs.python.org/issue19199 + + (4) https://bugs.python.org/issue15641 + + (5) https://bugs.python.org/issue15480 + + (6) https://bugs.python.org/issue7994 + + (7) https://bugs.python.org/issue13248 + + +File: python.info, Node: Code Cleanups, Prev: API and Feature Removals<5>, Up: Removed<3> + +1.5.7.3 Code Cleanups +..................... + + * The unused and undocumented internal ‘Scanner’ class has been + removed from the *note pydoc: d9. module. + + * The private and effectively unused ‘_gestalt’ module has been + removed, along with the private *note platform: ce. functions + ‘_mac_ver_lookup’, ‘_mac_ver_gstalt’, and ‘_bcd2str’, which would + only have ever been called on badly broken OSX systems (see + bpo-18393(1)). + + * The hardcoded copies of certain *note stat: f4. constants that were + included in the *note tarfile: 101. module namespace have been + removed. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue18393 + + +File: python.info, Node: Porting to Python 3 4, Next: Changed in 3 4 3, Prev: Removed<3>, Up: What’s New In Python 3 4 + +1.5.8 Porting to Python 3.4 +--------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code. + +* Menu: + +* Changes in ‘python’ Command Behavior: Changes in ‘python’ Command Behavior<2>. +* Changes in the Python API: Changes in the Python API<5>. +* Changes in the C API: Changes in the C API<5>. + + +File: python.info, Node: Changes in ‘python’ Command Behavior<2>, Next: Changes in the Python API<5>, Up: Porting to Python 3 4 + +1.5.8.1 Changes in ‘python’ Command Behavior +............................................ + + * In a posix shell, setting the ‘PATH’ environment variable to an + empty value is equivalent to not setting it at all. However, + setting *note PYTHONPATH: 953. to an empty value was `not' + equivalent to not setting it at all: setting *note PYTHONPATH: 953. + to an empty value was equivalent to setting it to ‘.’, which leads + to confusion when reasoning by analogy to how ‘PATH’ works. The + behavior now conforms to the posix convention for ‘PATH’. + + * The [X refs, Y blocks] output of a debug (‘--with-pydebug’) build + of the CPython interpreter is now off by default. It can be + re-enabled using the ‘-X showrefcount’ option. (Contributed by + Ezio Melotti in bpo-17323(1).) + + * The python command and most stdlib scripts (as well as *note + argparse: 6.) now output ‘--version’ information to ‘stdout’ + instead of ‘stderr’ (for issue list see *note Other Improvements: + 92a. above). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17323 + + +File: python.info, Node: Changes in the Python API<5>, Next: Changes in the C API<5>, Prev: Changes in ‘python’ Command Behavior<2>, Up: Porting to Python 3 4 + +1.5.8.2 Changes in the Python API +................................. + + * The ABCs defined in *note importlib.abc: 9c. now either raise the + appropriate exception or return a default value instead of raising + *note NotImplementedError: 60e. blindly. This will only affect + code calling *note super(): 4e2. and falling through all the way to + the ABCs. For compatibility, catch both *note NotImplementedError: + 60e. or the appropriate exception as needed. + + * The module type now initializes the *note __package__: 955. and + *note __loader__: 956. attributes to ‘None’ by default. To + determine if these attributes were set in a backwards-compatible + fashion, use e.g. ‘getattr(module, '__loader__', None) is not + None’. (bpo-17115(1).) + + * *note importlib.util.module_for_loader(): 942. now sets + ‘__loader__’ and ‘__package__’ unconditionally to properly support + reloading. If this is not desired then you will need to set these + attributes manually. You can use ‘importlib.util.module_to_load()’ + for module management. + + * Import now resets relevant attributes (e.g. ‘__name__’, + ‘__loader__’, ‘__package__’, ‘__file__’, ‘__cached__’) + unconditionally when reloading. Note that this restores a pre-3.3 + behavior in that it means a module is re-found when re-loaded + (bpo-19413(2)). + + * Frozen packages no longer set ‘__path__’ to a list containing the + package name, they now set it to an empty list. The previous + behavior could cause the import system to do the wrong thing on + submodule imports if there was also a directory with the same name + as the frozen package. The correct way to determine if a module is + a package or not is to use ‘hasattr(module, '__path__')’ + (bpo-18065(3)). + + * Frozen modules no longer define a ‘__file__’ attribute. It’s + semantically incorrect for frozen modules to set the attribute as + they are not loaded from any explicit location. If you must know + that a module comes from frozen code then you can see if the + module’s ‘__spec__.location’ is set to ‘'frozen'’, check if the + loader is a subclass of *note importlib.machinery.FrozenImporter: + 957, or if Python 2 compatibility is necessary you can use + ‘imp.is_frozen()’. + + * *note py_compile.compile(): 215. now raises *note FileExistsError: + 958. if the file path it would write to is a symlink or a + non-regular file. This is to act as a warning that import will + overwrite those files with a regular file regardless of what type + of file path they were originally. + + * *note importlib.abc.SourceLoader.get_source(): 959. no longer + raises *note ImportError: 334. when the source code being loaded + triggers a *note SyntaxError: 458. or *note UnicodeDecodeError: + 1fd. As *note ImportError: 334. is meant to be raised only when + source code cannot be found but it should, it was felt to be + over-reaching/overloading of that meaning when the source code is + found but improperly structured. If you were catching ImportError + before and wish to continue to ignore syntax or decoding issues, + catch all three exceptions now. + + * *note functools.update_wrapper(): 95a. and *note functools.wraps(): + 86a. now correctly set the ‘__wrapped__’ attribute to the function + being wrapped, even if that function also had its ‘__wrapped__’ + attribute set. This means ‘__wrapped__’ attributes now correctly + link a stack of decorated functions rather than every ‘__wrapped__’ + attribute in the chain referring to the innermost function. + Introspection libraries that assumed the previous behaviour was + intentional can use *note inspect.unwrap(): 869. to access the + first function in the chain that has no ‘__wrapped__’ attribute. + + * *note inspect.getfullargspec(): 55d. has been reimplemented on top + of *note inspect.signature(): 55b. and hence handles a much wider + variety of callable objects than it did in the past. It is + expected that additional builtin and extension module callables + will gain signature metadata over the course of the Python 3.4 + series. Code that assumes that *note inspect.getfullargspec(): + 55d. will fail on non-Python callables may need to be adjusted + accordingly. + + * *note importlib.machinery.PathFinder: 95b. now passes on the + current working directory to objects in *note sys.path_hooks: 95c. + for the empty string. This results in *note + sys.path_importer_cache: 49d. never containing ‘''’, thus iterating + through *note sys.path_importer_cache: 49d. based on *note + sys.path: 488. will not find all keys. A module’s ‘__file__’ when + imported in the current working directory will also now have an + absolute path, including when using ‘-m’ with the interpreter + (except for ‘__main__.__file__’ when a script has been executed + directly using a relative path) (Contributed by Brett Cannon in + bpo-18416(4)). is specified on the command-line) (bpo-18416(5)). + + * The removal of the `strict' argument to *note HTTPConnection: 392. + and *note HTTPSConnection: 393. changes the meaning of the + remaining arguments if you are specifying them positionally rather + than by keyword. If you’ve been paying attention to deprecation + warnings your code should already be specifying any additional + arguments via keywords. + + * Strings between ‘from __future__ import ...’ statements now + `always' raise a *note SyntaxError: 458. Previously if there was + no leading docstring, an interstitial string would sometimes be + ignored. This brings CPython into compliance with the language + spec; Jython and PyPy already were. (bpo-17434(6)). + + * *note ssl.SSLSocket.getpeercert(): 8d1. and *note + ssl.SSLSocket.do_handshake(): 75a. now raise an *note OSError: 1d3. + with ‘ENOTCONN’ when the ‘SSLSocket’ is not connected, instead of + the previous behavior of raising an *note AttributeError: 39b. In + addition, *note getpeercert(): 8d1. will raise a *note ValueError: + 1fb. if the handshake has not yet been done. + + * *note base64.b32decode(): 95d. now raises a *note binascii.Error: + 95e. when the input string contains non-b32-alphabet characters, + instead of a *note TypeError: 192. This particular *note + TypeError: 192. was missed when the other *note TypeError: 192.s + were converted. (Contributed by Serhiy Storchaka in bpo-18011(7).) + Note: this change was also inadvertently applied in Python 3.3.3. + + * The ‘file’ attribute is now automatically closed when the creating + ‘cgi.FieldStorage’ instance is garbage collected. If you were + pulling the file object out separately from the ‘cgi.FieldStorage’ + instance and not keeping the instance alive, then you should either + store the entire ‘cgi.FieldStorage’ instance or read the contents + of the file before the ‘cgi.FieldStorage’ instance is garbage + collected. + + * Calling ‘read’ or ‘write’ on a closed SSL socket now raises an + informative *note ValueError: 1fb. rather than the previous more + mysterious *note AttributeError: 39b. (bpo-9177(8)). + + * *note slice.indices(): 95f. no longer produces an *note + OverflowError: 960. for huge values. As a consequence of this fix, + *note slice.indices(): 95f. now raises a *note ValueError: 1fb. if + given a negative length; previously it returned nonsense values + (bpo-14794(9)). + + * The *note complex: 189. constructor, unlike the *note cmath: 19. + functions, was incorrectly accepting *note float: 187. values if an + object’s ‘__complex__’ special method returned one. This now + raises a *note TypeError: 192. (bpo-16290(10).) + + * The *note int: 184. constructor in 3.2 and 3.3 erroneously accepts + *note float: 187. values for the `base' parameter. It is unlikely + anyone was doing this, but if so, it will now raise a *note + TypeError: 192. (bpo-16772(11)). + + * Defaults for keyword-only arguments are now evaluated `after' + defaults for regular keyword arguments, instead of before. + Hopefully no one wrote any code that depends on the previous buggy + behavior (bpo-16967(12)). + + * Stale thread states are now cleared after *note fork(): 961. This + may cause some system resources to be released that previously were + incorrectly kept perpetually alive (for example, database + connections kept in thread-local storage). (bpo-17094(13).) + + * Parameter names in ‘__annotations__’ dicts are now mangled + properly, similarly to ‘__kwdefaults__’. (Contributed by Yury + Selivanov in bpo-20625(14).) + + * *note hashlib.hash.name: 852. now always returns the identifier in + lower case. Previously some builtin hashes had uppercase names, + but now that it is a formal public interface the naming has been + made consistent (bpo-18532(15)). + + * Because *note unittest.TestSuite: 6ce. now drops references to + tests after they are run, test harnesses that re-use a *note + TestSuite: 6ce. to re-run a set of tests may fail. Test suites + should not be re-used in this fashion since it means state is + retained between test runs, breaking the test isolation that *note + unittest: 11b. is designed to provide. However, if the lack of + isolation is considered acceptable, the old behavior can be + restored by creating a *note TestSuite: 6ce. subclass that defines + a ‘_removeTestAtIndex’ method that does nothing (see *note + TestSuite.__iter__(): 962.) (bpo-11798(16)). + + * *note unittest: 11b. now uses *note argparse: 6. for command line + parsing. There are certain invalid command forms that used to work + that are no longer allowed; in theory this should not cause + backward compatibility issues since the disallowed command forms + didn’t make any sense and are unlikely to be in use. + + * The *note re.split(): 3ca, *note re.findall(): 963, and *note + re.sub(): 47b. functions, and the ‘group()’ and ‘groups()’ methods + of ‘match’ objects now always return a `bytes' object when the + string to be matched is a *note bytes-like object: 5e8. Previously + the return type matched the input type, so if your code was + depending on the return value being, say, a ‘bytearray’, you will + need to change your code. + + * *note audioop: d. functions now raise an error immediately if + passed string input, instead of failing randomly later on + (bpo-16685(17)). + + * The new `convert_charrefs' argument to *note HTMLParser: 7bf. + currently defaults to ‘False’ for backward compatibility, but will + eventually be changed to default to ‘True’. It is recommended that + you add this keyword, with the appropriate value, to any *note + HTMLParser: 7bf. calls in your code (bpo-13633(18)). + + * Since the `digestmod' argument to the *note hmac.new(): 854. + function will in the future have no default, all calls to *note + hmac.new(): 854. should be changed to explicitly specify a + `digestmod' (bpo-17276(19)). + + * Calling *note sysconfig.get_config_var(): 964. with the ‘SO’ key, + or looking ‘SO’ up in the results of a call to *note + sysconfig.get_config_vars(): 965. is deprecated. This key should + be replaced by ‘EXT_SUFFIX’ or ‘SHLIB_SUFFIX’, depending on the + context (bpo-19555(20)). + + * Any calls to ‘open’ functions that specify ‘U’ should be modified. + ‘U’ is ineffective in Python3 and will eventually raise an error if + used. Depending on the function, the equivalent of its old Python2 + behavior can be achieved using either a `newline' argument, or if + necessary by wrapping the stream in *note TextIOWrapper: 5f3. to + use its `newline' argument (bpo-15204(21)). + + * If you use ‘pyvenv’ in a script and desire that pip `not' be + installed, you must add ‘--without-pip’ to your command invocation. + + * The default behavior of *note json.dump(): 604. and *note + json.dumps(): 605. when an indent is specified has changed: it no + longer produces trailing spaces after the item separating commas at + the ends of lines. This will matter only if you have tests that + are doing white-space-sensitive comparisons of such output + (bpo-16333(22)). + + * *note doctest: 67. now looks for doctests in extension module + ‘__doc__’ strings, so if your doctest test discovery includes + extension modules that have things that look like doctests in them + you may see test failures you’ve never seen before when running + your tests (bpo-3158(23)). + + * The *note collections.abc: 1f. module has been slightly refactored + as part of the Python startup improvements. As a consequence of + this, it is no longer the case that importing *note collections: + 1e. automatically imports *note collections.abc: 1f. If your + program depended on the (undocumented) implicit import, you will + need to add an explicit ‘import collections.abc’ (bpo-20784(24)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue17115 + + (2) https://bugs.python.org/issue19413 + + (3) https://bugs.python.org/issue18065 + + (4) https://bugs.python.org/issue18416 + + (5) https://bugs.python.org/issue18416 + + (6) https://bugs.python.org/issue17434 + + (7) https://bugs.python.org/issue18011 + + (8) https://bugs.python.org/issue9177 + + (9) https://bugs.python.org/issue14794 + + (10) https://bugs.python.org/issue16290 + + (11) https://bugs.python.org/issue16772 + + (12) https://bugs.python.org/issue16967 + + (13) https://bugs.python.org/issue17094 + + (14) https://bugs.python.org/issue20625 + + (15) https://bugs.python.org/issue18532 + + (16) https://bugs.python.org/issue11798 + + (17) https://bugs.python.org/issue16685 + + (18) https://bugs.python.org/issue13633 + + (19) https://bugs.python.org/issue17276 + + (20) https://bugs.python.org/issue19555 + + (21) https://bugs.python.org/issue15204 + + (22) https://bugs.python.org/issue16333 + + (23) https://bugs.python.org/issue3158 + + (24) https://bugs.python.org/issue20784 + + +File: python.info, Node: Changes in the C API<5>, Prev: Changes in the Python API<5>, Up: Porting to Python 3 4 + +1.5.8.3 Changes in the C API +............................ + + * *note PyEval_EvalFrameEx(): 967, *note PyObject_Repr(): 968, and + *note PyObject_Str(): 969, along with some other internal C APIs, + now include a debugging assertion that ensures they are not used in + situations where they may silently discard a currently active + exception. In cases where discarding the active exception is + expected and desired (for example, because it has already been + saved locally with *note PyErr_Fetch(): 96a. or is being + deliberately replaced with a different exception), an explicit + *note PyErr_Clear(): 96b. call will be needed to avoid triggering + the assertion when invoking these operations (directly or + indirectly) and running against a version of Python that is + compiled with assertions enabled. + + * *note PyErr_SetImportError(): 5f8. now sets *note TypeError: 192. + when its `msg' argument is not set. Previously only ‘NULL’ was + returned with no exception set. + + * The result of the *note PyOS_ReadlineFunctionPointer: 96c. callback + must now be a string allocated by *note PyMem_RawMalloc(): 96d. or + *note PyMem_RawRealloc(): 96e, or ‘NULL’ if an error occurred, + instead of a string allocated by *note PyMem_Malloc(): 505. or + *note PyMem_Realloc(): 96f. (bpo-16742(1)) + + * *note PyThread_set_key_value(): 970. now always set the value. In + Python 3.3, the function did nothing if the key already exists (if + the current value is a non-‘NULL’ pointer). + + * The ‘f_tstate’ (thread state) field of the *note PyFrameObject: + 971. structure has been removed to fix a bug: see bpo-14432(2) for + the rationale. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue16742 + + (2) https://bugs.python.org/issue14432 + + +File: python.info, Node: Changed in 3 4 3, Prev: Porting to Python 3 4, Up: What’s New In Python 3 4 + +1.5.9 Changed in 3.4.3 +---------------------- + +* Menu: + +* PEP 476; Enabling certificate verification by default for stdlib http clients: PEP 476 Enabling certificate verification by default for stdlib http clients. + + +File: python.info, Node: PEP 476 Enabling certificate verification by default for stdlib http clients, Up: Changed in 3 4 3 + +1.5.9.1 PEP 476: Enabling certificate verification by default for stdlib http clients +..................................................................................... + +*note http.client: 94. and modules which use it, such as *note +urllib.request: 120. and *note xmlrpc.client: 13f, will now verify that +the server presents a certificate which is signed by a CA in the +platform trust store and whose hostname matches the hostname being +requested by default, significantly improving security for many +applications. + +For applications which require the old previous behavior, they can pass +an alternate context: + + import urllib.request + import ssl + + # This disables all verification + context = ssl._create_unverified_context() + + # This allows using a specific certificate for the host, which doesn't need + # to be in the trust store + context = ssl.create_default_context(cafile="/path/to/file.crt") + + urllib.request.urlopen("https://invalid-cert", context=context) + + +File: python.info, Node: What’s New In Python 3 3, Next: What’s New In Python 3 2, Prev: What’s New In Python 3 4, Up: What’s New in Python + +1.6 What’s New In Python 3.3 +============================ + +This article explains the new features in Python 3.3, compared to 3.2. +Python 3.3 was released on September 29, 2012. For full details, see +the changelog(1). + +See also +........ + +PEP 398(2) - Python 3.3 Release Schedule + +* Menu: + +* Summary – Release highlights: Summary – Release highlights<4>. +* PEP 405; Virtual Environments: PEP 405 Virtual Environments. +* PEP 420; Implicit Namespace Packages: PEP 420 Implicit Namespace Packages. +* PEP 3118; New memoryview implementation and buffer protocol documentation: PEP 3118 New memoryview implementation and buffer protocol documentation. +* PEP 393; Flexible String Representation: PEP 393 Flexible String Representation. +* PEP 397; Python Launcher for Windows: PEP 397 Python Launcher for Windows. +* PEP 3151; Reworking the OS and IO exception hierarchy: PEP 3151 Reworking the OS and IO exception hierarchy. +* PEP 380; Syntax for Delegating to a Subgenerator: PEP 380 Syntax for Delegating to a Subgenerator. +* PEP 409; Suppressing exception context: PEP 409 Suppressing exception context. +* PEP 414; Explicit Unicode literals: PEP 414 Explicit Unicode literals. +* PEP 3155; Qualified name for classes and functions: PEP 3155 Qualified name for classes and functions. +* PEP 412; Key-Sharing Dictionary: PEP 412 Key-Sharing Dictionary. +* PEP 362; Function Signature Object: PEP 362 Function Signature Object. +* PEP 421; Adding sys.implementation: PEP 421 Adding sys implementation. +* Using importlib as the Implementation of Import:: +* Other Language Changes: Other Language Changes<6>. +* A Finer-Grained Import Lock:: +* Builtin functions and types:: +* New Modules: New Modules<6>. +* Improved Modules: Improved Modules<6>. +* Optimizations: Optimizations<5>. +* Build and C API Changes: Build and C API Changes<4>. +* Deprecated: Deprecated<5>. +* Porting to Python 3.3: Porting to Python 3 3. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3.3/whatsnew/changelog.html + + (2) https://www.python.org/dev/peps/pep-0398 + + +File: python.info, Node: Summary – Release highlights<4>, Next: PEP 405 Virtual Environments, Up: What’s New In Python 3 3 + +1.6.1 Summary – Release highlights +---------------------------------- + +New syntax features: + + * New ‘yield from’ expression for *note generator delegation: 978. + + * The ‘u'unicode'’ syntax is accepted again for *note str: 330. + objects. + +New library modules: + + * *note faulthandler: 7d. (helps debugging low-level crashes) + + * *note ipaddress: a2. (high-level objects representing IP addresses + and masks) + + * *note lzma: ad. (compress data using the XZ / LZMA algorithm) + + * *note unittest.mock: 11c. (replace parts of your system under test + with mock objects) + + * *note venv: 125. (Python *note virtual environments: 979, as in the + popular ‘virtualenv’ package) + +New built-in features: + + * Reworked *note I/O exception hierarchy: 97a. + +Implementation improvements: + + * Rewritten *note import machinery: 97b. based on *note importlib: + 9b. + + * More compact *note unicode strings: 97c. + + * More compact *note attribute dictionaries: 97d. + +Significantly Improved Library Modules: + + * C Accelerator for the *note decimal: 97e. module. + + * Better unicode handling in the *note email: 97f. module (*note + provisional: 980.). + +Security improvements: + + * Hash randomization is switched on by default. + +Please read on for a comprehensive list of user-facing changes. + + +File: python.info, Node: PEP 405 Virtual Environments, Next: PEP 420 Implicit Namespace Packages, Prev: Summary – Release highlights<4>, Up: What’s New In Python 3 3 + +1.6.2 PEP 405: Virtual Environments +----------------------------------- + +Virtual environments help create separate Python setups while sharing a +system-wide base install, for ease of maintenance. Virtual environments +have their own set of private site packages (i.e. locally-installed +libraries), and are optionally segregated from the system-wide site +packages. Their concept and implementation are inspired by the popular +‘virtualenv’ third-party package, but benefit from tighter integration +with the interpreter core. + +This PEP adds the *note venv: 125. module for programmatic access, and +the ‘pyvenv’ script for command-line access and administration. The +Python interpreter checks for a ‘pyvenv.cfg’, file whose existence +signals the base of a virtual environment’s directory tree. + +See also +........ + +PEP 405(1) - Python Virtual Environments + + PEP written by Carl Meyer; implementation by Carl Meyer and Vinay + Sajip + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0405 + + +File: python.info, Node: PEP 420 Implicit Namespace Packages, Next: PEP 3118 New memoryview implementation and buffer protocol documentation, Prev: PEP 405 Virtual Environments, Up: What’s New In Python 3 3 + +1.6.3 PEP 420: Implicit Namespace Packages +------------------------------------------ + +Native support for package directories that don’t require ‘__init__.py’ +marker files and can automatically span multiple path segments (inspired +by various third party approaches to namespace packages, as described in +PEP 420(1)) + +See also +........ + +PEP 420(2) - Implicit Namespace Packages + + PEP written by Eric V. Smith; implementation by Eric V. Smith and + Barry Warsaw + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0420 + + (2) https://www.python.org/dev/peps/pep-0420 + + +File: python.info, Node: PEP 3118 New memoryview implementation and buffer protocol documentation, Next: PEP 393 Flexible String Representation, Prev: PEP 420 Implicit Namespace Packages, Up: What’s New In Python 3 3 + +1.6.4 PEP 3118: New memoryview implementation and buffer protocol documentation +------------------------------------------------------------------------------- + +The implementation of PEP 3118(1) has been significantly improved. + +The new memoryview implementation comprehensively fixes all ownership +and lifetime issues of dynamically allocated fields in the Py_buffer +struct that led to multiple crash reports. Additionally, several +functions that crashed or returned incorrect results for non-contiguous +or multi-dimensional input have been fixed. + +The memoryview object now has a PEP-3118 compliant getbufferproc() that +checks the consumer’s request type. Many new features have been added, +most of them work in full generality for non-contiguous arrays and +arrays with suboffsets. + +The documentation has been updated, clearly spelling out +responsibilities for both exporters and consumers. Buffer request flags +are grouped into basic and compound flags. The memory layout of +non-contiguous and multi-dimensional NumPy-style arrays is explained. + +* Menu: + +* Features:: +* API changes:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3118 + + +File: python.info, Node: Features, Next: API changes, Up: PEP 3118 New memoryview implementation and buffer protocol documentation + +1.6.4.1 Features +................ + + * All native single character format specifiers in struct module + syntax (optionally prefixed with ‘@’) are now supported. + + * With some restrictions, the cast() method allows changing of format + and shape of C-contiguous arrays. + + * Multi-dimensional list representations are supported for any array + type. + + * Multi-dimensional comparisons are supported for any array type. + + * One-dimensional memoryviews of hashable (read-only) types with + formats B, b or c are now hashable. (Contributed by Antoine Pitrou + in bpo-13411(1).) + + * Arbitrary slicing of any 1-D arrays type is supported. For + example, it is now possible to reverse a memoryview in O(1) by + using a negative step. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13411 + + +File: python.info, Node: API changes, Prev: Features, Up: PEP 3118 New memoryview implementation and buffer protocol documentation + +1.6.4.2 API changes +................... + + * The maximum number of dimensions is officially limited to 64. + + * The representation of empty shape, strides and suboffsets is now an + empty tuple instead of ‘None’. + + * Accessing a memoryview element with format ‘B’ (unsigned bytes) now + returns an integer (in accordance with the struct module syntax). + For returning a bytes object the view must be cast to ‘c’ first. + + * memoryview comparisons now use the logical structure of the + operands and compare all array elements by value. All format + strings in struct module syntax are supported. Views with + unrecognised format strings are still permitted, but will always + compare as unequal, regardless of view contents. + + * For further changes see *note Build and C API Changes: 987. and + *note Porting C code: 988. + +(Contributed by Stefan Krah in bpo-10181(1).) + +See also +........ + +PEP 3118(2) - Revising the Buffer Protocol + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10181 + + (2) https://www.python.org/dev/peps/pep-3118 + + +File: python.info, Node: PEP 393 Flexible String Representation, Next: PEP 397 Python Launcher for Windows, Prev: PEP 3118 New memoryview implementation and buffer protocol documentation, Up: What’s New In Python 3 3 + +1.6.5 PEP 393: Flexible String Representation +--------------------------------------------- + +The Unicode string type is changed to support multiple internal +representations, depending on the character with the largest Unicode +ordinal (1, 2, or 4 bytes) in the represented string. This allows a +space-efficient representation in common cases, but gives access to full +UCS-4 on all systems. For compatibility with existing APIs, several +representations may exist in parallel; over time, this compatibility +should be phased out. + +On the Python side, there should be no downside to this change. + +On the C API side, PEP 393(1) is fully backward compatible. The legacy +API should remain available at least five years. Applications using the +legacy API will not fully benefit of the memory reduction, or - worse - +may use a bit more memory, because Python may have to maintain two +versions of each string (in the legacy format and in the new efficient +storage). + +* Menu: + +* Functionality:: +* Performance and resource usage:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + +File: python.info, Node: Functionality, Next: Performance and resource usage, Up: PEP 393 Flexible String Representation + +1.6.5.1 Functionality +..................... + +Changes introduced by PEP 393(1) are the following: + + * Python now always supports the full range of Unicode code points, + including non-BMP ones (i.e. from ‘U+0000’ to ‘U+10FFFF’). The + distinction between narrow and wide builds no longer exists and + Python now behaves like a wide build, even under Windows. + + * With the death of narrow builds, the problems specific to narrow + builds have also been fixed, for example: + + * *note len(): 150. now always returns 1 for non-BMP characters, + so ‘len('\U0010FFFF') == 1’; + + * surrogate pairs are not recombined in string literals, so + ‘'\uDBFF\uDFFF' != '\U0010FFFF'’; + + * indexing or slicing non-BMP characters returns the expected + value, so ‘'\U0010FFFF'[0]’ now returns ‘'\U0010FFFF'’ and not + ‘'\uDBFF'’; + + * all other functions in the standard library now correctly + handle non-BMP code points. + + * The value of *note sys.maxunicode: 98b. is now always ‘1114111’ + (‘0x10FFFF’ in hexadecimal). The ‘PyUnicode_GetMax()’ function + still returns either ‘0xFFFF’ or ‘0x10FFFF’ for backward + compatibility, and it should not be used with the new Unicode API + (see bpo-13054(2)). + + * The ‘./configure’ flag ‘--with-wide-unicode’ has been removed. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + (2) https://bugs.python.org/issue13054 + + +File: python.info, Node: Performance and resource usage, Prev: Functionality, Up: PEP 393 Flexible String Representation + +1.6.5.2 Performance and resource usage +...................................... + +The storage of Unicode strings now depends on the highest code point in +the string: + + * pure ASCII and Latin1 strings (‘U+0000-U+00FF’) use 1 byte per code + point; + + * BMP strings (‘U+0000-U+FFFF’) use 2 bytes per code point; + + * non-BMP strings (‘U+10000-U+10FFFF’) use 4 bytes per code point. + +The net effect is that for most applications, memory usage of string +storage should decrease significantly - especially compared to former +wide unicode builds - as, in many cases, strings will be pure ASCII even +in international contexts (because many strings store non-human language +data, such as XML fragments, HTTP headers, JSON-encoded data, etc.). We +also hope that it will, for the same reasons, increase CPU cache +efficiency on non-trivial applications. The memory usage of Python 3.3 +is two to three times smaller than Python 3.2, and a little bit better +than Python 2.7, on a Django benchmark (see the PEP for details). + +See also +........ + +PEP 393(1) - Flexible String Representation + + PEP written by Martin von Löwis; implementation by Torsten Becker + and Martin von Löwis. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + +File: python.info, Node: PEP 397 Python Launcher for Windows, Next: PEP 3151 Reworking the OS and IO exception hierarchy, Prev: PEP 393 Flexible String Representation, Up: What’s New In Python 3 3 + +1.6.6 PEP 397: Python Launcher for Windows +------------------------------------------ + +The Python 3.3 Windows installer now includes a ‘py’ launcher +application that can be used to launch Python applications in a version +independent fashion. + +This launcher is invoked implicitly when double-clicking ‘*.py’ files. +If only a single Python version is installed on the system, that version +will be used to run the file. If multiple versions are installed, the +most recent version is used by default, but this can be overridden by +including a Unix-style “shebang line” in the Python script. + +The launcher can also be used explicitly from the command line as the +‘py’ application. Running ‘py’ follows the same version selection rules +as implicitly launching scripts, but a more specific version can be +selected by passing appropriate arguments (such as ‘-3’ to request +Python 3 when Python 2 is also installed, or ‘-2.6’ to specifically +request an earlier Python version when a more recent version is +installed). + +In addition to the launcher, the Windows installer now includes an +option to add the newly installed Python to the system PATH. +(Contributed by Brian Curtin in bpo-3561(1).) + +See also +........ + +PEP 397(2) - Python Launcher for Windows + + PEP written by Mark Hammond and Martin v. Löwis; implementation by + Vinay Sajip. + +Launcher documentation: *note Python Launcher for Windows: 98f. + +Installer PATH modification: *note Finding the Python executable: 990. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue3561 + + (2) https://www.python.org/dev/peps/pep-0397 + + +File: python.info, Node: PEP 3151 Reworking the OS and IO exception hierarchy, Next: PEP 380 Syntax for Delegating to a Subgenerator, Prev: PEP 397 Python Launcher for Windows, Up: What’s New In Python 3 3 + +1.6.7 PEP 3151: Reworking the OS and IO exception hierarchy +----------------------------------------------------------- + +The hierarchy of exceptions raised by operating system errors is now +both simplified and finer-grained. + +You don’t have to worry anymore about choosing the appropriate exception +type between *note OSError: 1d3, *note IOError: 992, *note +EnvironmentError: 993, *note WindowsError: 994, ‘mmap.error’, *note +socket.error: 995. or *note select.error: 996. All these exception +types are now only one: *note OSError: 1d3. The other names are kept as +aliases for compatibility reasons. + +Also, it is now easier to catch a specific error condition. Instead of +inspecting the ‘errno’ attribute (or ‘args[0]’) for a particular +constant from the *note errno: 7c. module, you can catch the adequate +*note OSError: 1d3. subclass. The available subclasses are the +following: + + * *note BlockingIOError: 997. + + * *note ChildProcessError: 998. + + * *note ConnectionError: 6e6. + + * *note FileExistsError: 958. + + * *note FileNotFoundError: 7c0. + + * *note InterruptedError: 659. + + * *note IsADirectoryError: 999. + + * *note NotADirectoryError: 99a. + + * *note PermissionError: 5ff. + + * *note ProcessLookupError: 99b. + + * *note TimeoutError: 99c. + +And the *note ConnectionError: 6e6. itself has finer-grained subclasses: + + * *note BrokenPipeError: 99d. + + * *note ConnectionAbortedError: 99e. + + * *note ConnectionRefusedError: 99f. + + * *note ConnectionResetError: 9a0. + +Thanks to the new exceptions, common usages of the *note errno: 7c. can +now be avoided. For example, the following code written for Python 3.2: + + from errno import ENOENT, EACCES, EPERM + + try: + with open("document.txt") as f: + content = f.read() + except IOError as err: + if err.errno == ENOENT: + print("document.txt file is missing") + elif err.errno in (EACCES, EPERM): + print("You are not allowed to read document.txt") + else: + raise + +can now be written without the *note errno: 7c. import and without +manual inspection of exception attributes: + + try: + with open("document.txt") as f: + content = f.read() + except FileNotFoundError: + print("document.txt file is missing") + except PermissionError: + print("You are not allowed to read document.txt") + +See also +........ + +PEP 3151(1) - Reworking the OS and IO Exception Hierarchy + + PEP written and implemented by Antoine Pitrou + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3151 + + +File: python.info, Node: PEP 380 Syntax for Delegating to a Subgenerator, Next: PEP 409 Suppressing exception context, Prev: PEP 3151 Reworking the OS and IO exception hierarchy, Up: What’s New In Python 3 3 + +1.6.8 PEP 380: Syntax for Delegating to a Subgenerator +------------------------------------------------------ + +PEP 380 adds the ‘yield from’ expression, allowing a *note generator: +9a2. to delegate part of its operations to another generator. This +allows a section of code containing *note yield: 18f. to be factored out +and placed in another generator. Additionally, the subgenerator is +allowed to return with a value, and the value is made available to the +delegating generator. + +While designed primarily for use in delegating to a subgenerator, the +‘yield from’ expression actually allows delegation to arbitrary +subiterators. + +For simple iterators, ‘yield from iterable’ is essentially just a +shortened form of ‘for item in iterable: yield item’: + + >>> def g(x): + ... yield from range(x, 0, -1) + ... yield from range(x) + ... + >>> list(g(5)) + [5, 4, 3, 2, 1, 0, 1, 2, 3, 4] + +However, unlike an ordinary loop, ‘yield from’ allows subgenerators to +receive sent and thrown values directly from the calling scope, and +return a final value to the outer generator: + + >>> def accumulate(): + ... tally = 0 + ... while 1: + ... next = yield + ... if next is None: + ... return tally + ... tally += next + ... + >>> def gather_tallies(tallies): + ... while 1: + ... tally = yield from accumulate() + ... tallies.append(tally) + ... + >>> tallies = [] + >>> acc = gather_tallies(tallies) + >>> next(acc) # Ensure the accumulator is ready to accept values + >>> for i in range(4): + ... acc.send(i) + ... + >>> acc.send(None) # Finish the first tally + >>> for i in range(5): + ... acc.send(i) + ... + >>> acc.send(None) # Finish the second tally + >>> tallies + [6, 10] + +The main principle driving this change is to allow even generators that +are designed to be used with the ‘send’ and ‘throw’ methods to be split +into multiple subgenerators as easily as a single large function can be +split into multiple subfunctions. + +See also +........ + +PEP 380(1) - Syntax for Delegating to a Subgenerator + + PEP written by Greg Ewing; implementation by Greg Ewing, integrated + into 3.3 by Renaud Blanch, Ryan Kelly and Nick Coghlan; + documentation by Zbigniew Jędrzejewski-Szmek and Nick Coghlan + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0380 + + +File: python.info, Node: PEP 409 Suppressing exception context, Next: PEP 414 Explicit Unicode literals, Prev: PEP 380 Syntax for Delegating to a Subgenerator, Up: What’s New In Python 3 3 + +1.6.9 PEP 409: Suppressing exception context +-------------------------------------------- + +PEP 409 introduces new syntax that allows the display of the chained +exception context to be disabled. This allows cleaner error messages in +applications that convert between exception types: + + >>> class D: + ... def __init__(self, extra): + ... self._extra_attributes = extra + ... def __getattr__(self, attr): + ... try: + ... return self._extra_attributes[attr] + ... except KeyError: + ... raise AttributeError(attr) from None + ... + >>> D({}).x + Traceback (most recent call last): + File "", line 1, in + File "", line 8, in __getattr__ + AttributeError: x + +Without the ‘from None’ suffix to suppress the cause, the original +exception would be displayed by default: + + >>> class C: + ... def __init__(self, extra): + ... self._extra_attributes = extra + ... def __getattr__(self, attr): + ... try: + ... return self._extra_attributes[attr] + ... except KeyError: + ... raise AttributeError(attr) + ... + >>> C({}).x + Traceback (most recent call last): + File "", line 6, in __getattr__ + KeyError: 'x' + + During handling of the above exception, another exception occurred: + + Traceback (most recent call last): + File "", line 1, in + File "", line 8, in __getattr__ + AttributeError: x + +No debugging capability is lost, as the original exception context +remains available if needed (for example, if an intervening library has +incorrectly suppressed valuable underlying details): + + >>> try: + ... D({}).x + ... except AttributeError as exc: + ... print(repr(exc.__context__)) + ... + KeyError('x',) + +See also +........ + +PEP 409(1) - Suppressing exception context + + PEP written by Ethan Furman; implemented by Ethan Furman and Nick + Coghlan. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0409 + + +File: python.info, Node: PEP 414 Explicit Unicode literals, Next: PEP 3155 Qualified name for classes and functions, Prev: PEP 409 Suppressing exception context, Up: What’s New In Python 3 3 + +1.6.10 PEP 414: Explicit Unicode literals +----------------------------------------- + +To ease the transition from Python 2 for Unicode aware Python +applications that make heavy use of Unicode literals, Python 3.3 once +again supports the “‘u’” prefix for string literals. This prefix has no +semantic significance in Python 3, it is provided solely to reduce the +number of purely mechanical changes in migrating to Python 3, making it +easier for developers to focus on the more significant semantic changes +(such as the stricter default separation of binary and text data). + +See also +........ + +PEP 414(1) - Explicit Unicode literals + + PEP written by Armin Ronacher. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0414 + + +File: python.info, Node: PEP 3155 Qualified name for classes and functions, Next: PEP 412 Key-Sharing Dictionary, Prev: PEP 414 Explicit Unicode literals, Up: What’s New In Python 3 3 + +1.6.11 PEP 3155: Qualified name for classes and functions +--------------------------------------------------------- + +Functions and class objects have a new ‘__qualname__’ attribute +representing the “path” from the module top-level to their definition. +For global functions and classes, this is the same as ‘__name__’. For +other functions and classes, it provides better information about where +they were actually defined, and how they might be accessible from the +global scope. + +Example with (non-bound) methods: + + >>> class C: + ... def meth(self): + ... pass + >>> C.meth.__name__ + 'meth' + >>> C.meth.__qualname__ + 'C.meth' + +Example with nested classes: + + >>> class C: + ... class D: + ... def meth(self): + ... pass + ... + >>> C.D.__name__ + 'D' + >>> C.D.__qualname__ + 'C.D' + >>> C.D.meth.__name__ + 'meth' + >>> C.D.meth.__qualname__ + 'C.D.meth' + +Example with nested functions: + + >>> def outer(): + ... def inner(): + ... pass + ... return inner + ... + >>> outer().__name__ + 'inner' + >>> outer().__qualname__ + 'outer..inner' + +The string representation of those objects is also changed to include +the new, more precise information: + + >>> str(C.D) + "" + >>> str(C.D.meth) + '' + +See also +........ + +PEP 3155(1) - Qualified name for classes and functions + + PEP written and implemented by Antoine Pitrou. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3155 + + +File: python.info, Node: PEP 412 Key-Sharing Dictionary, Next: PEP 362 Function Signature Object, Prev: PEP 3155 Qualified name for classes and functions, Up: What’s New In Python 3 3 + +1.6.12 PEP 412: Key-Sharing Dictionary +-------------------------------------- + +Dictionaries used for the storage of objects’ attributes are now able to +share part of their internal storage between each other (namely, the +part which stores the keys and their respective hashes). This reduces +the memory consumption of programs creating many instances of +non-builtin types. + +See also +........ + +PEP 412(1) - Key-Sharing Dictionary + + PEP written and implemented by Mark Shannon. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0412 + + +File: python.info, Node: PEP 362 Function Signature Object, Next: PEP 421 Adding sys implementation, Prev: PEP 412 Key-Sharing Dictionary, Up: What’s New In Python 3 3 + +1.6.13 PEP 362: Function Signature Object +----------------------------------------- + +A new function *note inspect.signature(): 55b. makes introspection of +python callables easy and straightforward. A broad range of callables +is supported: python functions, decorated or not, classes, and *note +functools.partial(): 7c8. objects. New classes *note inspect.Signature: +6f3, *note inspect.Parameter: 6f4. and *note inspect.BoundArguments: +9a8. hold information about the call signatures, such as, annotations, +default values, parameters kinds, and bound arguments, which +considerably simplifies writing decorators and any code that validates +or amends calling signatures or arguments. + +See also +........ + +PEP 362(1): - Function Signature Object + + PEP written by Brett Cannon, Yury Selivanov, Larry Hastings, Jiwon + Seo; implemented by Yury Selivanov. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0362 + + +File: python.info, Node: PEP 421 Adding sys implementation, Next: Using importlib as the Implementation of Import, Prev: PEP 362 Function Signature Object, Up: What’s New In Python 3 3 + +1.6.14 PEP 421: Adding sys.implementation +----------------------------------------- + +A new attribute on the *note sys: fd. module exposes details specific to +the implementation of the currently running interpreter. The initial +set of attributes on *note sys.implementation: 9aa. are ‘name’, +‘version’, ‘hexversion’, and ‘cache_tag’. + +The intention of ‘sys.implementation’ is to consolidate into one +namespace the implementation-specific data used by the standard library. +This allows different Python implementations to share a single standard +library code base much more easily. In its initial state, +‘sys.implementation’ holds only a small portion of the +implementation-specific data. Over time that ratio will shift in order +to make the standard library more portable. + +One example of improved standard library portability is ‘cache_tag’. As +of Python 3.3, ‘sys.implementation.cache_tag’ is used by *note +importlib: 9b. to support PEP 3147(1) compliance. Any Python +implementation that uses ‘importlib’ for its built-in import system may +use ‘cache_tag’ to control the caching behavior for modules. + +* Menu: + +* SimpleNamespace:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: SimpleNamespace, Up: PEP 421 Adding sys implementation + +1.6.14.1 SimpleNamespace +........................ + +The implementation of ‘sys.implementation’ also introduces a new type to +Python: *note types.SimpleNamespace: 9ac. In contrast to a +mapping-based namespace, like *note dict: 1b8, ‘SimpleNamespace’ is +attribute-based, like *note object: 2b0. However, unlike ‘object’, +‘SimpleNamespace’ instances are writable. This means that you can add, +remove, and modify the namespace through normal attribute access. + +See also +........ + +PEP 421(1) - Adding sys.implementation + + PEP written and implemented by Eric Snow. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0421 + + +File: python.info, Node: Using importlib as the Implementation of Import, Next: Other Language Changes<6>, Prev: PEP 421 Adding sys implementation, Up: What’s New In Python 3 3 + +1.6.15 Using importlib as the Implementation of Import +------------------------------------------------------ + +bpo-2377(1) - Replace __import__ w/ importlib.__import__ bpo-13959(2) - +Re-implement parts of *note imp: 9a. in pure Python bpo-14605(3) - Make +import machinery explicit bpo-14646(4) - Require loaders set __loader__ +and __package__ + +The *note __import__(): 610. function is now powered by *note +importlib.__import__(): 9ae. This work leads to the completion of +“phase 2” of PEP 302(5). There are multiple benefits to this change. +First, it has allowed for more of the machinery powering import to be +exposed instead of being implicit and hidden within the C code. It also +provides a single implementation for all Python VMs supporting Python +3.3 to use, helping to end any VM-specific deviations in import +semantics. And finally it eases the maintenance of import, allowing for +future growth to occur. + +For the common user, there should be no visible change in semantics. +For those whose code currently manipulates import or calls import +programmatically, the code changes that might possibly be required are +covered in the *note Porting Python code: 9af. section of this document. + +* Menu: + +* New APIs:: +* Visible Changes:: + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2377 + + (2) https://bugs.python.org/issue13959 + + (3) https://bugs.python.org/issue14605 + + (4) https://bugs.python.org/issue14646 + + (5) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: New APIs, Next: Visible Changes, Up: Using importlib as the Implementation of Import + +1.6.15.1 New APIs +................. + +One of the large benefits of this work is the exposure of what goes into +making the import statement work. That means the various importers that +were once implicit are now fully exposed as part of the *note importlib: +9b. package. + +The abstract base classes defined in *note importlib.abc: 9c. have been +expanded to properly delineate between *note meta path finders: 9b1. and +*note path entry finders: 9b2. by introducing *note +importlib.abc.MetaPathFinder: 9b3. and *note +importlib.abc.PathEntryFinder: 9b4, respectively. The old ABC of *note +importlib.abc.Finder: 9b5. is now only provided for +backwards-compatibility and does not enforce any method requirements. + +In terms of finders, *note importlib.machinery.FileFinder: 9b6. exposes +the mechanism used to search for source and bytecode files of a module. +Previously this class was an implicit member of *note sys.path_hooks: +95c. + +For loaders, the new abstract base class *note importlib.abc.FileLoader: +9b7. helps write a loader that uses the file system as the storage +mechanism for a module’s code. The loader for source files (*note +importlib.machinery.SourceFileLoader: 9b8.), sourceless bytecode files +(*note importlib.machinery.SourcelessFileLoader: 9b9.), and extension +modules (*note importlib.machinery.ExtensionFileLoader: 556.) are now +available for direct use. + +*note ImportError: 334. now has ‘name’ and ‘path’ attributes which are +set when there is relevant data to provide. The message for failed +imports will also provide the full name of the module now instead of +just the tail end of the module’s name. + +The *note importlib.invalidate_caches(): 49c. function will now call the +method with the same name on all finders cached in *note +sys.path_importer_cache: 49d. to help clean up any stored state as +necessary. + + +File: python.info, Node: Visible Changes, Prev: New APIs, Up: Using importlib as the Implementation of Import + +1.6.15.2 Visible Changes +........................ + +For potential required changes to code, see the *note Porting Python +code: 9af. section. + +Beyond the expanse of what *note importlib: 9b. now exposes, there are +other visible changes to import. The biggest is that *note +sys.meta_path: 5e6. and *note sys.path_hooks: 95c. now store all of the +meta path finders and path entry hooks used by import. Previously the +finders were implicit and hidden within the C code of import instead of +being directly exposed. This means that one can now easily remove or +change the order of the various finders to fit one’s needs. + +Another change is that all modules have a ‘__loader__’ attribute, +storing the loader used to create the module. PEP 302(1) has been +updated to make this attribute mandatory for loaders to implement, so in +the future once 3rd-party loaders have been updated people will be able +to rely on the existence of the attribute. Until such time, though, +import is setting the module post-load. + +Loaders are also now expected to set the ‘__package__’ attribute from +PEP 366(2). Once again, import itself is already setting this on all +loaders from *note importlib: 9b. and import itself is setting the +attribute post-load. + +‘None’ is now inserted into *note sys.path_importer_cache: 49d. when no +finder can be found on *note sys.path_hooks: 95c. Since *note +imp.NullImporter: 9bb. is not directly exposed on *note sys.path_hooks: +95c. it could no longer be relied upon to always be available to use as +a value representing no finder found. + +All other changes relate to semantic changes which should be taken into +consideration when updating code for Python 3.3, and thus should be read +about in the *note Porting Python code: 9af. section of this document. + +(Implementation by Brett Cannon) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0302 + + (2) https://www.python.org/dev/peps/pep-0366 + + +File: python.info, Node: Other Language Changes<6>, Next: A Finer-Grained Import Lock, Prev: Using importlib as the Implementation of Import, Up: What’s New In Python 3 3 + +1.6.16 Other Language Changes +----------------------------- + +Some smaller changes made to the core Python language are: + + * Added support for Unicode name aliases and named sequences. Both + *note unicodedata.lookup(): 9bd. and ‘'\N{...}'’ now resolve name + aliases, and *note unicodedata.lookup(): 9bd. resolves named + sequences too. + + (Contributed by Ezio Melotti in bpo-12753(1).) + + * Unicode database updated to UCD version 6.1.0 + + * Equality comparisons on *note range(): 9be. objects now return a + result reflecting the equality of the underlying sequences + generated by those range objects. (bpo-13201(2)) + + * The ‘count()’, ‘find()’, ‘rfind()’, ‘index()’ and ‘rindex()’ + methods of *note bytes: 331. and *note bytearray: 332. objects now + accept an integer between 0 and 255 as their first argument. + + (Contributed by Petri Lehtinen in bpo-12170(3).) + + * The ‘rjust()’, ‘ljust()’, and ‘center()’ methods of *note bytes: + 331. and *note bytearray: 332. now accept a *note bytearray: 332. + for the ‘fill’ argument. (Contributed by Petri Lehtinen in + bpo-12380(4).) + + * New methods have been added to *note list: 262. and *note + bytearray: 332.: ‘copy()’ and ‘clear()’ (bpo-10516(5)). + Consequently, *note MutableSequence: 6ac. now also defines a + ‘clear()’ method (bpo-11388(6)). + + * Raw bytes literals can now be written ‘rb"..."’ as well as + ‘br"..."’. + + (Contributed by Antoine Pitrou in bpo-13748(7).) + + * *note dict.setdefault(): 9bf. now does only one lookup for the + given key, making it atomic when used with built-in types. + + (Contributed by Filip Gruszczyński in bpo-13521(8).) + + * The error messages produced when a function call does not match the + function signature have been significantly improved. + + (Contributed by Benjamin Peterson.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12753 + + (2) https://bugs.python.org/issue13201 + + (3) https://bugs.python.org/issue12170 + + (4) https://bugs.python.org/issue12380 + + (5) https://bugs.python.org/issue10516 + + (6) https://bugs.python.org/issue11388 + + (7) https://bugs.python.org/issue13748 + + (8) https://bugs.python.org/issue13521 + + +File: python.info, Node: A Finer-Grained Import Lock, Next: Builtin functions and types, Prev: Other Language Changes<6>, Up: What’s New In Python 3 3 + +1.6.17 A Finer-Grained Import Lock +---------------------------------- + +Previous versions of CPython have always relied on a global import lock. +This led to unexpected annoyances, such as deadlocks when importing a +module would trigger code execution in a different thread as a +side-effect. Clumsy workarounds were sometimes employed, such as the +*note PyImport_ImportModuleNoBlock(): 9c1. C API function. + +In Python 3.3, importing a module takes a per-module lock. This +correctly serializes importation of a given module from multiple threads +(preventing the exposure of incompletely initialized modules), while +eliminating the aforementioned annoyances. + +(Contributed by Antoine Pitrou in bpo-9260(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9260 + + +File: python.info, Node: Builtin functions and types, Next: New Modules<6>, Prev: A Finer-Grained Import Lock, Up: What’s New In Python 3 3 + +1.6.18 Builtin functions and types +---------------------------------- + + * *note open(): 4f0. gets a new `opener' parameter: the underlying + file descriptor for the file object is then obtained by calling + `opener' with (`file', `flags'). It can be used to use custom + flags like *note os.O_CLOEXEC: 9c3. for example. The ‘'x'’ mode + was added: open for exclusive creation, failing if the file already + exists. + + * *note print(): 886.: added the `flush' keyword argument. If the + `flush' keyword argument is true, the stream is forcibly flushed. + + * *note hash(): 9c4.: hash randomization is enabled by default, see + *note object.__hash__(): 340. and *note PYTHONHASHSEED: 9c5. + + * The *note str: 330. type gets a new *note casefold(): 6b0. method: + return a casefolded copy of the string, casefolded strings may be + used for caseless matching. For example, ‘'ß'.casefold()’ returns + ‘'ss'’. + + * The sequence documentation has been substantially rewritten to + better explain the binary/text sequence distinction and to provide + specific documentation sections for the individual builtin sequence + types (bpo-4966(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4966 + + +File: python.info, Node: New Modules<6>, Next: Improved Modules<6>, Prev: Builtin functions and types, Up: What’s New In Python 3 3 + +1.6.19 New Modules +------------------ + +* Menu: + +* faulthandler: faulthandler<3>. +* ipaddress: ipaddress<4>. +* lzma: lzma<2>. + + +File: python.info, Node: faulthandler<3>, Next: ipaddress<4>, Up: New Modules<6> + +1.6.19.1 faulthandler +..................... + +This new debug module *note faulthandler: 7d. contains functions to dump +Python tracebacks explicitly, on a fault (a crash like a segmentation +fault), after a timeout, or on a user signal. Call *note +faulthandler.enable(): 548. to install fault handlers for the ‘SIGSEGV’, +‘SIGFPE’, ‘SIGABRT’, ‘SIGBUS’, and ‘SIGILL’ signals. You can also +enable them at startup by setting the *note PYTHONFAULTHANDLER: 9c8. +environment variable or by using *note -X: 155. ‘faulthandler’ command +line option. + +Example of a segmentation fault on Linux: + + $ python -q -X faulthandler + >>> import ctypes + >>> ctypes.string_at(0) + Fatal Python error: Segmentation fault + + Current thread 0x00007fb899f39700: + File "/home/python/cpython/Lib/ctypes/__init__.py", line 486 in string_at + File "", line 1 in + Segmentation fault + + +File: python.info, Node: ipaddress<4>, Next: lzma<2>, Prev: faulthandler<3>, Up: New Modules<6> + +1.6.19.2 ipaddress +.................. + +The new *note ipaddress: a2. module provides tools for creating and +manipulating objects representing IPv4 and IPv6 addresses, networks and +interfaces (i.e. an IP address associated with a specific IP subnet). + +(Contributed by Google and Peter Moody in PEP 3144(1).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3144 + + +File: python.info, Node: lzma<2>, Prev: ipaddress<4>, Up: New Modules<6> + +1.6.19.3 lzma +............. + +The newly-added *note lzma: ad. module provides data compression and +decompression using the LZMA algorithm, including support for the ‘.xz’ +and ‘.lzma’ file formats. + +(Contributed by Nadeem Vawda and Per Øyvind Karlsen in bpo-6715(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6715 + + +File: python.info, Node: Improved Modules<6>, Next: Optimizations<5>, Prev: New Modules<6>, Up: What’s New In Python 3 3 + +1.6.20 Improved Modules +----------------------- + +* Menu: + +* abc: abc<2>. +* array: array<2>. +* base64: base64<2>. +* binascii: binascii<3>. +* bz2: bz2<2>. +* codecs:: +* collections: collections<7>. +* contextlib: contextlib<5>. +* crypt: crypt<2>. +* curses: curses<3>. +* datetime: datetime<4>. +* decimal: decimal<3>. +* email: email<4>. +* ftplib:: +* functools: functools<5>. +* gc: gc<4>. +* hmac: hmac<3>. +* http: http<3>. +* html: html<2>. +* imaplib: imaplib<2>. +* inspect: inspect<5>. +* io: io<4>. +* itertools: itertools<3>. +* logging: logging<6>. +* math: math<5>. +* mmap: mmap<3>. +* multiprocessing: multiprocessing<6>. +* nntplib:: +* os: os<7>. +* pdb: pdb<4>. +* pickle: pickle<5>. +* pydoc: pydoc<4>. +* re: re<6>. +* sched:: +* select: select<2>. +* shlex: shlex<3>. +* shutil: shutil<4>. +* signal: signal<3>. +* smtpd: smtpd<3>. +* smtplib: smtplib<3>. +* socket: socket<7>. +* socketserver: socketserver<3>. +* sqlite3: sqlite3<5>. +* ssl: ssl<8>. +* stat: stat<2>. +* struct: struct<3>. +* subprocess: subprocess<5>. +* sys: sys<7>. +* tarfile: tarfile<4>. +* tempfile:: +* textwrap: textwrap<2>. +* threading: threading<5>. +* time: time<5>. +* types: types<4>. +* unittest: unittest<5>. +* urllib: urllib<3>. +* webbrowser:: +* xml.etree.ElementTree: xml etree ElementTree. +* zlib: zlib<2>. + + +File: python.info, Node: abc<2>, Next: array<2>, Up: Improved Modules<6> + +1.6.20.1 abc +............ + +Improved support for abstract base classes containing descriptors +composed with abstract methods. The recommended approach to declaring +abstract descriptors is now to provide ‘__isabstractmethod__’ as a +dynamically updated property. The built-in descriptors have been +updated accordingly. + + * *note abc.abstractproperty: 9cd. has been deprecated, use + *note property: 1d7. with *note abc.abstractmethod(): 9ce. + instead. + + * *note abc.abstractclassmethod: 9cf. has been deprecated, use + *note classmethod: 1d8. with *note abc.abstractmethod(): 9ce. + instead. + + * *note abc.abstractstaticmethod: 9d0. has been deprecated, use + *note staticmethod: 1d9. with *note abc.abstractmethod(): 9ce. + instead. + +(Contributed by Darren Dale in bpo-11610(1).) + +*note abc.ABCMeta.register(): 9d1. now returns the registered subclass, +which means it can now be used as a class decorator (bpo-10868(2)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11610 + + (2) https://bugs.python.org/issue10868 + + +File: python.info, Node: array<2>, Next: base64<2>, Prev: abc<2>, Up: Improved Modules<6> + +1.6.20.2 array +.............. + +The *note array: 7. module supports the ‘long long’ type using ‘q’ and +‘Q’ type codes. + +(Contributed by Oren Tirosh and Hirokazu Yamamoto in bpo-1172711(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1172711 + + +File: python.info, Node: base64<2>, Next: binascii<3>, Prev: array<2>, Up: Improved Modules<6> + +1.6.20.3 base64 +............... + +ASCII-only Unicode strings are now accepted by the decoding functions of +the *note base64: e. modern interface. For example, +‘base64.b64decode('YWJj')’ returns ‘b'abc'’. (Contributed by Catalin +Iacob in bpo-13641(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13641 + + +File: python.info, Node: binascii<3>, Next: bz2<2>, Prev: base64<2>, Up: Improved Modules<6> + +1.6.20.4 binascii +................. + +In addition to the binary objects they normally accept, the ‘a2b_’ +functions now all also accept ASCII-only strings as input. (Contributed +by Antoine Pitrou in bpo-13637(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13637 + + +File: python.info, Node: bz2<2>, Next: codecs, Prev: binascii<3>, Up: Improved Modules<6> + +1.6.20.5 bz2 +............ + +The *note bz2: 14. module has been rewritten from scratch. In the +process, several new features have been added: + + * New *note bz2.open(): 9d6. function: open a bzip2-compressed file + in binary or text mode. + + * *note bz2.BZ2File: 931. can now read from and write to arbitrary + file-like objects, by means of its constructor’s `fileobj' + argument. + + (Contributed by Nadeem Vawda in bpo-5863(1).) + + * *note bz2.BZ2File: 931. and *note bz2.decompress(): 9d7. can now + decompress multi-stream inputs (such as those produced by the + ‘pbzip2’ tool). *note bz2.BZ2File: 931. can now also be used to + create this type of file, using the ‘'a'’ (append) mode. + + (Contributed by Nir Aides in bpo-1625(2).) + + * *note bz2.BZ2File: 931. now implements all of the *note + io.BufferedIOBase: 588. API, except for the ‘detach()’ and + ‘truncate()’ methods. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5863 + + (2) https://bugs.python.org/issue1625 + + +File: python.info, Node: codecs, Next: collections<7>, Prev: bz2<2>, Up: Improved Modules<6> + +1.6.20.6 codecs +............... + +The *note mbcs: 78. codec has been rewritten to handle correctly +‘replace’ and ‘ignore’ error handlers on all Windows versions. The +*note mbcs: 78. codec now supports all error handlers, instead of only +‘replace’ to encode and ‘ignore’ to decode. + +A new Windows-only codec has been added: ‘cp65001’ (bpo-13216(1)). It +is the Windows code page 65001 (Windows UTF-8, ‘CP_UTF8’). For example, +it is used by ‘sys.stdout’ if the console output code page is set to +cp65001 (e.g., using ‘chcp 65001’ command). + +Multibyte CJK decoders now resynchronize faster. They only ignore the +first byte of an invalid byte sequence. For example, +‘b'\xff\n'.decode('gb2312', 'replace')’ now returns a ‘\n’ after the +replacement character. + +(bpo-12016(2)) + +Incremental CJK codec encoders are no longer reset at each call to their +encode() methods. For example: + + >>> import codecs + >>> encoder = codecs.getincrementalencoder('hz')('strict') + >>> b''.join(encoder.encode(x) for x in '\u52ff\u65bd\u65bc\u4eba\u3002 Bye.') + b'~{NpJ)l6HK!#~} Bye.' + +This example gives ‘b'~{Np~}~{J)~}~{l6~}~{HK~}~{!#~} Bye.'’ with older +Python versions. + +(bpo-12100(3)) + +The ‘unicode_internal’ codec has been deprecated. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13216 + + (2) https://bugs.python.org/issue12016 + + (3) https://bugs.python.org/issue12100 + + +File: python.info, Node: collections<7>, Next: contextlib<5>, Prev: codecs, Up: Improved Modules<6> + +1.6.20.7 collections +.................... + +Addition of a new *note ChainMap: 4a9. class to allow treating a number +of mappings as a single unit. (Written by Raymond Hettinger for +bpo-11089(1), made public in bpo-11297(2).) + +The abstract base classes have been moved in a new *note +collections.abc: 1f. module, to better differentiate between the +abstract and the concrete collections classes. Aliases for ABCs are +still present in the *note collections: 1e. module to preserve existing +imports. (bpo-11085(3)) + +The *note Counter: 9da. class now supports the unary ‘+’ and ‘-’ +operators, as well as the in-place operators ‘+=’, ‘-=’, ‘|=’, and ‘&=’. +(Contributed by Raymond Hettinger in bpo-13121(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11089 + + (2) https://bugs.python.org/issue11297 + + (3) https://bugs.python.org/issue11085 + + (4) https://bugs.python.org/issue13121 + + +File: python.info, Node: contextlib<5>, Next: crypt<2>, Prev: collections<7>, Up: Improved Modules<6> + +1.6.20.8 contextlib +................... + +*note ExitStack: 376. now provides a solid foundation for programmatic +manipulation of context managers and similar cleanup functionality. +Unlike the previous ‘contextlib.nested’ API (which was deprecated and +removed), the new API is designed to work correctly regardless of +whether context managers acquire their resources in their ‘__init__’ +method (for example, file objects) or in their ‘__enter__’ method (for +example, synchronisation objects from the *note threading: 109. module). + +(bpo-13585(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13585 + + +File: python.info, Node: crypt<2>, Next: curses<3>, Prev: contextlib<5>, Up: Improved Modules<6> + +1.6.20.9 crypt +.............. + +Addition of salt and modular crypt format (hashing method) and the *note +mksalt(): 37c. function to the *note crypt: 29. module. + +(bpo-10924(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10924 + + +File: python.info, Node: curses<3>, Next: datetime<4>, Prev: crypt<2>, Up: Improved Modules<6> + +1.6.20.10 curses +................ + + * If the *note curses: 2c. module is linked to the ncursesw + library, use Unicode functions when Unicode strings or + characters are passed (e.g. ‘waddwstr()’), and bytes + functions otherwise (e.g. ‘waddstr()’). + + * Use the locale encoding instead of ‘utf-8’ to encode Unicode + strings. + + * ‘curses.window’ has a new *note curses.window.encoding: 9de. + attribute. + + * The ‘curses.window’ class has a new *note get_wch(): 9df. + method to get a wide character + + * The *note curses: 2c. module has a new *note unget_wch(): 9e0. + function to push a wide character so the next *note get_wch(): + 9df. will return it + +(Contributed by Iñigo Serna in bpo-6755(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6755 + + +File: python.info, Node: datetime<4>, Next: decimal<3>, Prev: curses<3>, Up: Improved Modules<6> + +1.6.20.11 datetime +.................. + + * Equality comparisons between naive and aware *note datetime: + 194. instances now return *note False: 388. instead of raising + *note TypeError: 192. (bpo-15006(1)). + + * New *note datetime.datetime.timestamp(): 9e2. method: Return + POSIX timestamp corresponding to the *note datetime: 194. + instance. + + * The *note datetime.datetime.strftime(): 537. method supports + formatting years older than 1000. + + * The *note datetime.datetime.astimezone(): 196. method can now + be called without arguments to convert datetime instance to + the system timezone. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15006 + + +File: python.info, Node: decimal<3>, Next: email<4>, Prev: datetime<4>, Up: Improved Modules<6> + +1.6.20.12 decimal +................. + +bpo-7652(1) - integrate fast native decimal arithmetic. + + C-module and libmpdec written by Stefan Krah. + +The new C version of the decimal module integrates the high speed +libmpdec library for arbitrary precision correctly-rounded decimal +floating point arithmetic. libmpdec conforms to IBM’s General Decimal +Arithmetic Specification. + +Performance gains range from 10x for database applications to 100x for +numerically intensive applications. These numbers are expected gains +for standard precisions used in decimal floating point arithmetic. +Since the precision is user configurable, the exact figures may vary. +For example, in integer bignum arithmetic the differences can be +significantly higher. + +The following table is meant as an illustration. Benchmarks are +available at ‘http://www.bytereef.org/mpdecimal/quickstart.html’. + + decimal.py _decimal speedup + + --------------------------------------------------------------------- + + pi 42.02s 0.345s 120x + + + telco 172.19s 5.68s 30x + + + psycopg 3.57s 0.29s 12x + + +* Menu: + +* Features: Features<2>. +* API changes: API changes<2>. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7652 + + +File: python.info, Node: Features<2>, Next: API changes<2>, Up: decimal<3> + +1.6.20.13 Features +.................. + + * The *note FloatOperation: 9e5. signal optionally enables stricter + semantics for mixing floats and Decimals. + + * If Python is compiled without threads, the C version automatically + disables the expensive thread local context machinery. In this + case, the variable *note HAVE_THREADS: 9e6. is set to ‘False’. + + +File: python.info, Node: API changes<2>, Prev: Features<2>, Up: decimal<3> + +1.6.20.14 API changes +..................... + + * The C module has the following context limits, depending on the + machine architecture: + + 32-bit 64-bit + + ------------------------------------------------------------------------------------- + + ‘MAX_PREC’ ‘425000000’ ‘999999999999999999’ + + + ‘MAX_EMAX’ ‘425000000’ ‘999999999999999999’ + + + ‘MIN_EMIN’ ‘-425000000’ ‘-999999999999999999’ + + + * In the context templates (*note DefaultContext: 9e8, *note + BasicContext: 9e9. and *note ExtendedContext: 9ea.) the magnitude + of ‘Emax’ and ‘Emin’ has changed to ‘999999’. + + * The *note Decimal: 188. constructor in decimal.py does not observe + the context limits and converts values with arbitrary exponents or + precision exactly. Since the C version has internal limits, the + following scheme is used: If possible, values are converted + exactly, otherwise *note InvalidOperation: 9eb. is raised and the + result is NaN. In the latter case it is always possible to use + *note create_decimal(): 9ec. in order to obtain a rounded or + inexact value. + + * The power function in decimal.py is always correctly-rounded. In + the C version, it is defined in terms of the correctly-rounded + *note exp(): 9ed. and *note ln(): 9ee. functions, but the final + result is only “almost always correctly rounded”. + + * In the C version, the context dictionary containing the signals is + a *note MutableMapping: 9ef. For speed reasons, ‘flags’ and + ‘traps’ always refer to the same *note MutableMapping: 9ef. that + the context was initialized with. If a new signal dictionary is + assigned, ‘flags’ and ‘traps’ are updated with the new values, but + they do not reference the RHS dictionary. + + * Pickling a *note Context: 9f0. produces a different output in order + to have a common interchange format for the Python and C versions. + + * The order of arguments in the *note Context: 9f0. constructor has + been changed to match the order displayed by *note repr(): 7cc. + + * The ‘watchexp’ parameter in the *note quantize(): 9f1. method is + deprecated. + + +File: python.info, Node: email<4>, Next: ftplib, Prev: decimal<3>, Up: Improved Modules<6> + +1.6.20.15 email +............... + +* Menu: + +* Policy Framework:: +* Provisional Policy with New Header API:: +* Other API Changes:: + + +File: python.info, Node: Policy Framework, Next: Provisional Policy with New Header API, Up: email<4> + +1.6.20.16 Policy Framework +.......................... + +The email package now has a *note policy: 75. framework. A *note +Policy: 9f4. is an object with several methods and properties that +control how the email package behaves. The primary policy for Python +3.3 is the *note Compat32: 9f5. policy, which provides backward +compatibility with the email package in Python 3.2. A ‘policy’ can be +specified when an email message is parsed by a *note parser: 74, or when +a *note Message: 541. object is created, or when an email is serialized +using a *note generator: 6e. Unless overridden, a policy passed to a +‘parser’ is inherited by all the ‘Message’ object and sub-objects +created by the ‘parser’. By default a ‘generator’ will use the policy +of the ‘Message’ object it is serializing. The default policy is *note +compat32: 540. + +The minimum set of controls implemented by all ‘policy’ objects are: + + max_line_length The maximum length, excluding the linesep character(s), + individual lines may have when a ‘Message’ is serialized. + Defaults to 78. + + + linesep The character used to separate individual lines when a + ‘Message’ is serialized. Defaults to ‘\n’. + + + cte_type ‘7bit’ or ‘8bit’. ‘8bit’ applies only to a ‘Bytes’ + ‘generator’, and means that non-ASCII may be used where + allowed by the protocol (or where it exists in the + original input). + + + raise_on_defect Causes a ‘parser’ to raise error when defects are + encountered instead of adding them to the ‘Message’ + object’s ‘defects’ list. + + +A new policy instance, with new settings, is created using the *note +clone(): 9f6. method of policy objects. ‘clone’ takes any of the above +controls as keyword arguments. Any control not specified in the call +retains its default value. Thus you can create a policy that uses +‘\r\n’ linesep characters like this: + + mypolicy = compat32.clone(linesep='\r\n') + +Policies can be used to make the generation of messages in the format +needed by your application simpler. Instead of having to remember to +specify ‘linesep='\r\n'’ in all the places you call a ‘generator’, you +can specify it once, when you set the policy used by the ‘parser’ or the +‘Message’, whichever your program uses to create ‘Message’ objects. On +the other hand, if you need to generate messages in multiple forms, you +can still specify the parameters in the appropriate ‘generator’ call. +Or you can have custom policy instances for your different cases, and +pass those in when you create the ‘generator’. + + +File: python.info, Node: Provisional Policy with New Header API, Next: Other API Changes, Prev: Policy Framework, Up: email<4> + +1.6.20.17 Provisional Policy with New Header API +................................................ + +While the policy framework is worthwhile all by itself, the main +motivation for introducing it is to allow the creation of new policies +that implement new features for the email package in a way that +maintains backward compatibility for those who do not use the new +policies. Because the new policies introduce a new API, we are +releasing them in Python 3.3 as a *note provisional policy: 980. +Backwards incompatible changes (up to and including removal of the code) +may occur if deemed necessary by the core developers. + +The new policies are instances of *note EmailPolicy: 9f8, and add the +following additional controls: + + refold_source Controls whether or not headers parsed by a + *note parser: 74. are refolded by the + *note generator: 6e. It can be ‘none’, ‘long’, or ‘all’. + The default is ‘long’, which means that source headers + with a line longer than ‘max_line_length’ get refolded. + ‘none’ means no line get refolded, and ‘all’ means that + all lines get refolded. + + + header_factory A callable that take a ‘name’ and ‘value’ and produces a + custom header object. + + +The ‘header_factory’ is the key to the new features provided by the new +policies. When one of the new policies is used, any header retrieved +from a ‘Message’ object is an object produced by the ‘header_factory’, +and any time you set a header on a ‘Message’ it becomes an object +produced by ‘header_factory’. All such header objects have a ‘name’ +attribute equal to the header name. Address and Date headers have +additional attributes that give you access to the parsed data of the +header. This means you can now do things like this: + + >>> m = Message(policy=SMTP) + >>> m['To'] = 'Éric ' + >>> m['to'] + 'Éric ' + >>> m['to'].addresses + (Address(display_name='Éric', username='foo', domain='example.com'),) + >>> m['to'].addresses[0].username + 'foo' + >>> m['to'].addresses[0].display_name + 'Éric' + >>> m['Date'] = email.utils.localtime() + >>> m['Date'].datetime + datetime.datetime(2012, 5, 25, 21, 39, 24, 465484, tzinfo=datetime.timezone(datetime.timedelta(-1, 72000), 'EDT')) + >>> m['Date'] + 'Fri, 25 May 2012 21:44:27 -0400' + >>> print(m) + To: =?utf-8?q?=C3=89ric?= + Date: Fri, 25 May 2012 21:44:27 -0400 + +You will note that the unicode display name is automatically encoded as +‘utf-8’ when the message is serialized, but that when the header is +accessed directly, you get the unicode version. This eliminates any +need to deal with the *note email.header: 6f. *note decode_header(): +9f9. or *note make_header(): 9fa. functions. + +You can also create addresses from parts: + + >>> m['cc'] = [Group('pals', [Address('Bob', 'bob', 'example.com'), + ... Address('Sally', 'sally', 'example.com')]), + ... Address('Bonzo', addr_spec='bonz@laugh.com')] + >>> print(m) + To: =?utf-8?q?=C3=89ric?= + Date: Fri, 25 May 2012 21:44:27 -0400 + cc: pals: Bob , Sally ;, Bonzo + +Decoding to unicode is done automatically: + + >>> m2 = message_from_string(str(m)) + >>> m2['to'] + 'Éric ' + +When you parse a message, you can use the ‘addresses’ and ‘groups’ +attributes of the header objects to access the groups and individual +addresses: + + >>> m2['cc'].addresses + (Address(display_name='Bob', username='bob', domain='example.com'), Address(display_name='Sally', username='sally', domain='example.com'), Address(display_name='Bonzo', username='bonz', domain='laugh.com')) + >>> m2['cc'].groups + (Group(display_name='pals', addresses=(Address(display_name='Bob', username='bob', domain='example.com'), Address(display_name='Sally', username='sally', domain='example.com')), Group(display_name=None, addresses=(Address(display_name='Bonzo', username='bonz', domain='laugh.com'),)) + +In summary, if you use one of the new policies, header manipulation +works the way it ought to: your application works with unicode strings, +and the email package transparently encodes and decodes the unicode to +and from the RFC standard Content Transfer Encodings. + + +File: python.info, Node: Other API Changes, Prev: Provisional Policy with New Header API, Up: email<4> + +1.6.20.18 Other API Changes +........................... + +New *note BytesHeaderParser: 9fc, added to the *note parser: 74. module +to complement *note HeaderParser: 9fd. and complete the Bytes API. + +New utility functions: + + * *note format_datetime(): 9fe.: given a *note datetime: 194, + produce a string formatted for use in an email header. + + * *note parsedate_to_datetime(): 9ff.: given a date string from + an email header, convert it into an aware *note datetime: 194, + or a naive *note datetime: 194. if the offset is ‘-0000’. + + * *note localtime(): a00.: With no argument, returns the current + local time as an aware *note datetime: 194. using the local + *note timezone: a01. Given an aware *note datetime: 194, + converts it into an aware *note datetime: 194. using the local + *note timezone: a01. + + +File: python.info, Node: ftplib, Next: functools<5>, Prev: email<4>, Up: Improved Modules<6> + +1.6.20.19 ftplib +................ + + * *note ftplib.FTP: 2f0. now accepts a ‘source_address’ keyword + argument to specify the ‘(host, port)’ to use as the source address + in the bind call when creating the outgoing socket. (Contributed + by Giampaolo Rodolà in bpo-8594(1).) + + * The *note FTP_TLS: a03. class now provides a new *note ccc(): a04. + function to revert control channel back to plaintext. This can be + useful to take advantage of firewalls that know how to handle NAT + with non-secure FTP without opening fixed ports. (Contributed by + Giampaolo Rodolà in bpo-12139(2).) + + * Added *note ftplib.FTP.mlsd(): a05. method which provides a + parsable directory listing format and deprecates *note + ftplib.FTP.nlst(): a06. and *note ftplib.FTP.dir(): a07. + (Contributed by Giampaolo Rodolà in bpo-11072(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8594 + + (2) https://bugs.python.org/issue12139 + + (3) https://bugs.python.org/issue11072 + + +File: python.info, Node: functools<5>, Next: gc<4>, Prev: ftplib, Up: Improved Modules<6> + +1.6.20.20 functools +................... + +The *note functools.lru_cache(): 1c7. decorator now accepts a ‘typed’ +keyword argument (that defaults to ‘False’ to ensure that it caches +values of different types that compare equal in separate cache slots. +(Contributed by Raymond Hettinger in bpo-13227(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13227 + + +File: python.info, Node: gc<4>, Next: hmac<3>, Prev: functools<5>, Up: Improved Modules<6> + +1.6.20.21 gc +............ + +It is now possible to register callbacks invoked by the garbage +collector before and after collection using the new *note callbacks: +a0a. list. + + +File: python.info, Node: hmac<3>, Next: http<3>, Prev: gc<4>, Up: Improved Modules<6> + +1.6.20.22 hmac +.............. + +A new *note compare_digest(): a0c. function has been added to prevent +side channel attacks on digests through timing analysis. (Contributed +by Nick Coghlan and Christian Heimes in bpo-15061(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15061 + + +File: python.info, Node: http<3>, Next: html<2>, Prev: hmac<3>, Up: Improved Modules<6> + +1.6.20.23 http +.............. + +*note http.server.BaseHTTPRequestHandler: a0e. now buffers the headers +and writes them all at once when *note end_headers(): a0f. is called. A +new method *note flush_headers(): a10. can be used to directly manage +when the accumulated headers are sent. (Contributed by Andrew Schaaf in +bpo-3709(1).) + +*note http.server: 97. now produces valid ‘HTML 4.01 strict’ output. +(Contributed by Ezio Melotti in bpo-13295(2).) + +*note http.client.HTTPResponse: a11. now has a *note readinto(): a12. +method, which means it can be used as an *note io.RawIOBase: a13. class. +(Contributed by John Kuhn in bpo-13464(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue3709 + + (2) https://bugs.python.org/issue13295 + + (3) https://bugs.python.org/issue13464 + + +File: python.info, Node: html<2>, Next: imaplib<2>, Prev: http<3>, Up: Improved Modules<6> + +1.6.20.24 html +.............. + +*note html.parser.HTMLParser: 7bf. is now able to parse broken markup +without raising errors, therefore the `strict' argument of the +constructor and the ‘HTMLParseError’ exception are now deprecated. The +ability to parse broken markup is the result of a number of bug fixes +that are also available on the latest bug fix releases of Python +2.7/3.2. (Contributed by Ezio Melotti in bpo-15114(1), and +bpo-14538(2), bpo-13993(3), bpo-13960(4), bpo-13358(5), bpo-1745761(6), +bpo-755670(7), bpo-13357(8), bpo-12629(9), bpo-1200313(10), +bpo-670664(11), bpo-13273(12), bpo-12888(13), bpo-7311(14).) + +A new *note html5: a15. dictionary that maps HTML5 named character +references to the equivalent Unicode character(s) (e.g. ‘html5['gt;'] +== '>'’) has been added to the *note html.entities: 91. module. The +dictionary is now also used by *note HTMLParser: 7bf. (Contributed by +Ezio Melotti in bpo-11113(15) and bpo-15156(16).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue15114 + + (2) https://bugs.python.org/issue14538 + + (3) https://bugs.python.org/issue13993 + + (4) https://bugs.python.org/issue13960 + + (5) https://bugs.python.org/issue13358 + + (6) https://bugs.python.org/issue1745761 + + (7) https://bugs.python.org/issue755670 + + (8) https://bugs.python.org/issue13357 + + (9) https://bugs.python.org/issue12629 + + (10) https://bugs.python.org/issue1200313 + + (11) https://bugs.python.org/issue670664 + + (12) https://bugs.python.org/issue13273 + + (13) https://bugs.python.org/issue12888 + + (14) https://bugs.python.org/issue7311 + + (15) https://bugs.python.org/issue11113 + + (16) https://bugs.python.org/issue15156 + + +File: python.info, Node: imaplib<2>, Next: inspect<5>, Prev: html<2>, Up: Improved Modules<6> + +1.6.20.25 imaplib +................. + +The *note IMAP4_SSL: a17. constructor now accepts an SSLContext +parameter to control parameters of the secure channel. + +(Contributed by Sijin Joseph in bpo-8808(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8808 + + +File: python.info, Node: inspect<5>, Next: io<4>, Prev: imaplib<2>, Up: Improved Modules<6> + +1.6.20.26 inspect +................. + +A new *note getclosurevars(): a19. function has been added. This +function reports the current binding of all names referenced from the +function body and where those names were resolved, making it easier to +verify correct internal state when testing code that relies on stateful +closures. + +(Contributed by Meador Inge and Nick Coghlan in bpo-13062(1).) + +A new *note getgeneratorlocals(): a1a. function has been added. This +function reports the current binding of local variables in the +generator’s stack frame, making it easier to verify correct internal +state when testing generators. + +(Contributed by Meador Inge in bpo-15153(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13062 + + (2) https://bugs.python.org/issue15153 + + +File: python.info, Node: io<4>, Next: itertools<3>, Prev: inspect<5>, Up: Improved Modules<6> + +1.6.20.27 io +............ + +The *note open(): 65a. function has a new ‘'x'’ mode that can be used to +exclusively create a new file, and raise a *note FileExistsError: 958. +if the file already exists. It is based on the C11 ‘x’ mode to fopen(). + +(Contributed by David Townshend in bpo-12760(1).) + +The constructor of the *note TextIOWrapper: 5f3. class has a new +`write_through' optional argument. If `write_through' is ‘True’, calls +to ‘write()’ are guaranteed not to be buffered: any data written on the +*note TextIOWrapper: 5f3. object is immediately handled to its +underlying binary buffer. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12760 + + +File: python.info, Node: itertools<3>, Next: logging<6>, Prev: io<4>, Up: Improved Modules<6> + +1.6.20.28 itertools +................... + +*note accumulate(): 1dd. now takes an optional ‘func’ argument for +providing a user-supplied binary function. + + +File: python.info, Node: logging<6>, Next: math<5>, Prev: itertools<3>, Up: Improved Modules<6> + +1.6.20.29 logging +................. + +The *note basicConfig(): 1e0. function now supports an optional +‘handlers’ argument taking an iterable of handlers to be added to the +root logger. + +A class level attribute ‘append_nul’ has been added to *note +SysLogHandler: a1e. to allow control of the appending of the ‘NUL’ +(‘\000’) byte to syslog records, since for some daemons it is required +while for others it is passed through to the log. + + +File: python.info, Node: math<5>, Next: mmap<3>, Prev: logging<6>, Up: Improved Modules<6> + +1.6.20.30 math +.............. + +The *note math: b1. module has a new function, *note log2(): a20, which +returns the base-2 logarithm of `x'. + +(Written by Mark Dickinson in bpo-11888(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11888 + + +File: python.info, Node: mmap<3>, Next: multiprocessing<6>, Prev: math<5>, Up: Improved Modules<6> + +1.6.20.31 mmap +.............. + +The *note read(): a22. method is now more compatible with other +file-like objects: if the argument is omitted or specified as ‘None’, it +returns the bytes from the current file position to the end of the +mapping. (Contributed by Petri Lehtinen in bpo-12021(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12021 + + +File: python.info, Node: multiprocessing<6>, Next: nntplib, Prev: mmap<3>, Up: Improved Modules<6> + +1.6.20.32 multiprocessing +......................... + +The new *note multiprocessing.connection.wait(): a24. function allows +polling multiple objects (such as connections, sockets and pipes) with a +timeout. (Contributed by Richard Oudkerk in bpo-12328(1).) + +‘multiprocessing.Connection’ objects can now be transferred over +multiprocessing connections. (Contributed by Richard Oudkerk in +bpo-4892(2).) + +*note multiprocessing.Process: 3b2. now accepts a ‘daemon’ keyword +argument to override the default behavior of inheriting the ‘daemon’ +flag from the parent process (bpo-6064(3)). + +New attribute *note multiprocessing.Process.sentinel: a25. allows a +program to wait on multiple *note Process: 3b2. objects at one time +using the appropriate OS primitives (for example, *note select: e5. on +posix systems). + +New methods *note multiprocessing.pool.Pool.starmap(): a26. and *note +starmap_async(): a27. provide *note itertools.starmap(): a28. +equivalents to the existing *note multiprocessing.pool.Pool.map(): a29. +and *note map_async(): a2a. functions. (Contributed by Hynek Schlawack +in bpo-12708(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12328 + + (2) https://bugs.python.org/issue4892 + + (3) https://bugs.python.org/issue6064 + + (4) https://bugs.python.org/issue12708 + + +File: python.info, Node: nntplib, Next: os<7>, Prev: multiprocessing<6>, Up: Improved Modules<6> + +1.6.20.33 nntplib +................. + +The *note nntplib.NNTP: a2c. class now supports the context management +protocol to unconditionally consume *note socket.error: 995. exceptions +and to close the NNTP connection when done: + + >>> from nntplib import NNTP + >>> with NNTP('news.gmane.org') as n: + ... n.group('gmane.comp.python.committers') + ... + ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') + >>> + +(Contributed by Giampaolo Rodolà in bpo-9795(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9795 + + +File: python.info, Node: os<7>, Next: pdb<4>, Prev: nntplib, Up: Improved Modules<6> + +1.6.20.34 os +............ + + * The *note os: c4. module has a new *note pipe2(): a2e. function + that makes it possible to create a pipe with *note O_CLOEXEC: 9c3. + or *note O_NONBLOCK: 723. flags set atomically. This is especially + useful to avoid race conditions in multi-threaded programs. + + * The *note os: c4. module has a new *note sendfile(): 359. function + which provides an efficient “zero-copy” way for copying data from + one file (or socket) descriptor to another. The phrase “zero-copy” + refers to the fact that all of the copying of data between the two + descriptors is done entirely by the kernel, with no copying of data + into userspace buffers. *note sendfile(): 359. can be used to + efficiently copy data from a file on disk to a network socket, e.g. + for downloading a file. + + (Patch submitted by Ross Lagerwall and Giampaolo Rodolà in + bpo-10882(1).) + + * To avoid race conditions like symlink attacks and issues with + temporary files and directories, it is more reliable (and also + faster) to manipulate file descriptors instead of file names. + Python 3.3 enhances existing functions and introduces new functions + to work on file descriptors (bpo-4761(2), bpo-10755(3) and + bpo-14626(4)). + + - The *note os: c4. module has a new *note fwalk(): 3b4. + function similar to *note walk(): 654. except that it also + yields file descriptors referring to the directories visited. + This is especially useful to avoid symlink races. + + - The following functions get new optional `dir_fd' (*note paths + relative to directory descriptors: a2f.) and/or + `follow_symlinks' (*note not following symlinks: a30.): *note + access(): a31, *note chflags(): a32, *note chmod(): a33, *note + chown(): a34, *note link(): a35, *note lstat(): 1f2, *note + mkdir(): a36, *note mkfifo(): 663, *note mknod(): 664, *note + open(): 665, *note readlink(): 1f3, *note remove(): a37, *note + rename(): a38, *note replace(): a39, *note rmdir(): a3a, *note + stat(): 1f1, *note symlink(): a3b, *note unlink(): a3c, *note + utime(): a3d. Platform support for using these parameters can + be checked via the sets *note os.supports_dir_fd: a3e. and + ‘os.supports_follows_symlinks’. + + - The following functions now support a file descriptor for + their path argument: *note chdir(): a3f, *note chmod(): a33, + *note chown(): a34, *note execve(): a40, *note listdir(): a41, + *note pathconf(): a42, *note exists(): 1f6, *note stat(): 1f1, + *note statvfs(): a43, *note utime(): a3d. Platform support + for this can be checked via the *note os.supports_fd: a44. + set. + + * *note access(): a31. accepts an ‘effective_ids’ keyword argument to + turn on using the effective uid/gid rather than the real uid/gid in + the access check. Platform support for this can be checked via the + *note supports_effective_ids: a45. set. + + * The *note os: c4. module has two new functions: *note + getpriority(): a46. and *note setpriority(): a47. They can be used + to get or set process niceness/priority in a fashion similar to + *note os.nice(): a48. but extended to all processes instead of just + the current one. + + (Patch submitted by Giampaolo Rodolà in bpo-10784(5).) + + * The new *note os.replace(): a39. function allows cross-platform + renaming of a file with overwriting the destination. With *note + os.rename(): a38, an existing destination file is overwritten under + POSIX, but raises an error under Windows. (Contributed by Antoine + Pitrou in bpo-8828(6).) + + * The stat family of functions (*note stat(): 1f1, *note fstat(): + 65f, and *note lstat(): 1f2.) now support reading a file’s + timestamps with nanosecond precision. Symmetrically, *note + utime(): a3d. can now write file timestamps with nanosecond + precision. (Contributed by Larry Hastings in bpo-14127(7).) + + * The new *note os.get_terminal_size(): a49. function queries the + size of the terminal attached to a file descriptor. See also *note + shutil.get_terminal_size(): a4a. (Contributed by Zbigniew + Jędrzejewski-Szmek in bpo-13609(8).) + + * New functions to support Linux extended attributes (bpo-12720(9)): + *note getxattr(): a4b, *note listxattr(): a4c, *note removexattr(): + a4d, *note setxattr(): a4e. + + * New interface to the scheduler. These functions control how a + process is allocated CPU time by the operating system. New + functions: *note sched_get_priority_max(): a4f, *note + sched_get_priority_min(): a50, *note sched_getaffinity(): a51, + *note sched_getparam(): a52, *note sched_getscheduler(): a53, *note + sched_rr_get_interval(): a54, *note sched_setaffinity(): a55, *note + sched_setparam(): a56, *note sched_setscheduler(): a57, *note + sched_yield(): a58, + + * New functions to control the file system: + + * *note posix_fadvise(): 666.: Announces an intention to access + data in a specific pattern thus allowing the kernel to make + optimizations. + + * *note posix_fallocate(): 667.: Ensures that enough disk space + is allocated for a file. + + * *note sync(): a59.: Force write of everything to disk. + + * Additional new posix functions: + + * *note lockf(): a5a.: Apply, test or remove a POSIX lock on an + open file descriptor. + + * *note pread(): 3b9.: Read from a file descriptor at an offset, + the file offset remains unchanged. + + * *note pwrite(): 3bc.: Write to a file descriptor from an + offset, leaving the file offset unchanged. + + * *note readv(): 3b8.: Read from a file descriptor into a number + of writable buffers. + + * *note truncate(): 724.: Truncate the file corresponding to + `path', so that it is at most `length' bytes in size. + + * *note waitid(): 66c.: Wait for the completion of one or more + child processes. + + * *note writev(): 3bb.: Write the contents of `buffers' to a + file descriptor, where `buffers' is an arbitrary sequence of + buffers. + + * *note getgrouplist(): a5b. (bpo-9344(10)): Return list of + group ids that specified user belongs to. + + * *note times(): a5c. and *note uname(): a5d.: Return type changed + from a tuple to a tuple-like object with named attributes. + + * Some platforms now support additional constants for the *note + lseek(): a5e. function, such as ‘os.SEEK_HOLE’ and ‘os.SEEK_DATA’. + + * New constants *note RTLD_LAZY: a5f, *note RTLD_NOW: a60, *note + RTLD_GLOBAL: a61, *note RTLD_LOCAL: a62, *note RTLD_NODELETE: a63, + *note RTLD_NOLOAD: a64, and *note RTLD_DEEPBIND: a65. are available + on platforms that support them. These are for use with the *note + sys.setdlopenflags(): a66. function, and supersede the similar + constants defined in *note ctypes: 2b. and ‘DLFCN’. (Contributed + by Victor Stinner in bpo-13226(11).) + + * *note os.symlink(): a3b. now accepts (and ignores) the + ‘target_is_directory’ keyword argument on non-Windows platforms, to + ease cross-platform support. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10882 + + (2) https://bugs.python.org/issue4761 + + (3) https://bugs.python.org/issue10755 + + (4) https://bugs.python.org/issue14626 + + (5) https://bugs.python.org/issue10784 + + (6) https://bugs.python.org/issue8828 + + (7) https://bugs.python.org/issue14127 + + (8) https://bugs.python.org/issue13609 + + (9) https://bugs.python.org/issue12720 + + (10) https://bugs.python.org/issue9344 + + (11) https://bugs.python.org/issue13226 + + +File: python.info, Node: pdb<4>, Next: pickle<5>, Prev: os<7>, Up: Improved Modules<6> + +1.6.20.35 pdb +............. + +Tab-completion is now available not only for command names, but also +their arguments. For example, for the ‘break’ command, function and +file names are completed. + +(Contributed by Georg Brandl in bpo-14210(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14210 + + +File: python.info, Node: pickle<5>, Next: pydoc<4>, Prev: pdb<4>, Up: Improved Modules<6> + +1.6.20.36 pickle +................ + +*note pickle.Pickler: 20d. objects now have an optional *note +dispatch_table: a69. attribute allowing per-pickler reduction functions +to be set. + +(Contributed by Richard Oudkerk in bpo-14166(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14166 + + +File: python.info, Node: pydoc<4>, Next: re<6>, Prev: pickle<5>, Up: Improved Modules<6> + +1.6.20.37 pydoc +............... + +The Tk GUI and the ‘serve()’ function have been removed from the *note +pydoc: d9. module: ‘pydoc -g’ and ‘serve()’ have been deprecated in +Python 3.2. + + +File: python.info, Node: re<6>, Next: sched, Prev: pydoc<4>, Up: Improved Modules<6> + +1.6.20.38 re +............ + +*note str: 330. regular expressions now support ‘\u’ and ‘\U’ escapes. + +(Contributed by Serhiy Storchaka in bpo-3665(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue3665 + + +File: python.info, Node: sched, Next: select<2>, Prev: re<6>, Up: Improved Modules<6> + +1.6.20.39 sched +............... + + * *note run(): a6d. now accepts a `blocking' parameter which when set + to false makes the method execute the scheduled events due to + expire soonest (if any) and then return immediately. This is + useful in case you want to use the *note scheduler: a6e. in + non-blocking applications. (Contributed by Giampaolo Rodolà in + bpo-13449(1).) + + * *note scheduler: a6e. class can now be safely used in + multi-threaded environments. (Contributed by Josiah Carlson and + Giampaolo Rodolà in bpo-8684(2).) + + * `timefunc' and `delayfunct' parameters of *note scheduler: a6e. + class constructor are now optional and defaults to *note + time.time(): 31d. and *note time.sleep(): 680. respectively. + (Contributed by Chris Clark in bpo-13245(3).) + + * *note enter(): a6f. and *note enterabs(): a70. `argument' parameter + is now optional. (Contributed by Chris Clark in bpo-13245(4).) + + * *note enter(): a6f. and *note enterabs(): a70. now accept a + `kwargs' parameter. (Contributed by Chris Clark in bpo-13245(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13449 + + (2) https://bugs.python.org/issue8684 + + (3) https://bugs.python.org/issue13245 + + (4) https://bugs.python.org/issue13245 + + (5) https://bugs.python.org/issue13245 + + +File: python.info, Node: select<2>, Next: shlex<3>, Prev: sched, Up: Improved Modules<6> + +1.6.20.40 select +................ + +Solaris and derivative platforms have a new class *note select.devpoll: +8ab. for high performance asynchronous sockets via ‘/dev/poll’. +(Contributed by Jesús Cea Avión in bpo-6397(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6397 + + +File: python.info, Node: shlex<3>, Next: shutil<4>, Prev: select<2>, Up: Improved Modules<6> + +1.6.20.41 shlex +............... + +The previously undocumented helper function ‘quote’ from the *note +pipes: cc. modules has been moved to the *note shlex: e8. module and +documented. *note quote(): a73. properly escapes all characters in a +string that might be otherwise given special meaning by the shell. + + +File: python.info, Node: shutil<4>, Next: signal<3>, Prev: shlex<3>, Up: Improved Modules<6> + +1.6.20.42 shutil +................ + + * New functions: + + * *note disk_usage(): a75.: provides total, used and free disk + space statistics. (Contributed by Giampaolo Rodolà in + bpo-12442(1).) + + * *note chown(): a76.: allows one to change user and/or group of + the given path also specifying the user/group names and not + only their numeric ids. (Contributed by Sandro Tosi in + bpo-12191(2).) + + * *note shutil.get_terminal_size(): a4a.: returns the size of + the terminal window to which the interpreter is attached. + (Contributed by Zbigniew Jędrzejewski-Szmek in bpo-13609(3).) + + * *note copy2(): 25a. and *note copystat(): a77. now preserve file + timestamps with nanosecond precision on platforms that support it. + They also preserve file “extended attributes” on Linux. + (Contributed by Larry Hastings in bpo-14127(4) and bpo-15238(5).) + + * Several functions now take an optional ‘symlinks’ argument: when + that parameter is true, symlinks aren’t dereferenced and the + operation instead acts on the symlink itself (or creates one, if + relevant). (Contributed by Hynek Schlawack in bpo-12715(6).) + + * When copying files to a different file system, *note move(): 25b. + now handles symlinks the way the posix ‘mv’ command does, + recreating the symlink rather than copying the target file + contents. (Contributed by Jonathan Niehof in bpo-9993(7).) *note + move(): 25b. now also returns the ‘dst’ argument as its result. + + * *note rmtree(): 21c. is now resistant to symlink attacks on + platforms which support the new ‘dir_fd’ parameter in *note + os.open(): 665. and *note os.unlink(): a3c. (Contributed by Martin + von Löwis and Hynek Schlawack in bpo-4489(8).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12442 + + (2) https://bugs.python.org/issue12191 + + (3) https://bugs.python.org/issue13609 + + (4) https://bugs.python.org/issue14127 + + (5) https://bugs.python.org/issue15238 + + (6) https://bugs.python.org/issue12715 + + (7) https://bugs.python.org/issue9993 + + (8) https://bugs.python.org/issue4489 + + +File: python.info, Node: signal<3>, Next: smtpd<3>, Prev: shutil<4>, Up: Improved Modules<6> + +1.6.20.43 signal +................ + + * The *note signal: ea. module has new functions: + + * *note pthread_sigmask(): a79.: fetch and/or change the signal + mask of the calling thread (Contributed by Jean-Paul Calderone + in bpo-8407(1)); + + * *note pthread_kill(): a7a.: send a signal to a thread; + + * *note sigpending(): a7b.: examine pending functions; + + * *note sigwait(): a7c.: wait a signal; + + * *note sigwaitinfo(): 67f.: wait for a signal, returning + detailed information about it; + + * *note sigtimedwait(): 67e.: like *note sigwaitinfo(): 67f. but + with a timeout. + + * The signal handler writes the signal number as a single byte + instead of a nul byte into the wakeup file descriptor. So it is + possible to wait more than one signal and know which signals were + raised. + + * *note signal.signal(): a7d. and *note signal.siginterrupt(): a7e. + raise an OSError, instead of a RuntimeError: OSError has an errno + attribute. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8407 + + +File: python.info, Node: smtpd<3>, Next: smtplib<3>, Prev: signal<3>, Up: Improved Modules<6> + +1.6.20.44 smtpd +............... + +The *note smtpd: ec. module now supports RFC 5321(1) (extended SMTP) and +RFC 1870(2) (size extension). Per the standard, these extensions are +enabled if and only if the client initiates the session with an ‘EHLO’ +command. + +(Initial ‘ELHO’ support by Alberto Trevino. Size extension by Juhana +Jauhiainen. Substantial additional work on the patch contributed by +Michele Orrù and Dan Boswell. bpo-8739(3)) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc5321.html + + (2) https://tools.ietf.org/html/rfc1870.html + + (3) https://bugs.python.org/issue8739 + + +File: python.info, Node: smtplib<3>, Next: socket<7>, Prev: smtpd<3>, Up: Improved Modules<6> + +1.6.20.45 smtplib +................. + +The *note SMTP: a81, *note SMTP_SSL: a82, and *note LMTP: a83. classes +now accept a ‘source_address’ keyword argument to specify the ‘(host, +port)’ to use as the source address in the bind call when creating the +outgoing socket. (Contributed by Paulo Scardine in bpo-11281(1).) + +*note SMTP: a81. now supports the context management protocol, allowing +an ‘SMTP’ instance to be used in a ‘with’ statement. (Contributed by +Giampaolo Rodolà in bpo-11289(2).) + +The *note SMTP_SSL: a82. constructor and the *note starttls(): a84. +method now accept an SSLContext parameter to control parameters of the +secure channel. (Contributed by Kasun Herath in bpo-8809(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11281 + + (2) https://bugs.python.org/issue11289 + + (3) https://bugs.python.org/issue8809 + + +File: python.info, Node: socket<7>, Next: socketserver<3>, Prev: smtplib<3>, Up: Improved Modules<6> + +1.6.20.46 socket +................ + + * The *note socket: 674. class now exposes additional methods to + process ancillary data when supported by the underlying platform: + + * *note sendmsg(): 67c. + + * *note recvmsg(): 679. + + * *note recvmsg_into(): a86. + + (Contributed by David Watson in bpo-6560(1), based on an earlier + patch by Heiko Wundram) + + * The *note socket: 674. class now supports the PF_CAN protocol + family (‘https://en.wikipedia.org/wiki/Socketcan’), on Linux + (‘https://lwn.net/Articles/253425’). + + (Contributed by Matthias Fuchs, updated by Tiago Gonçalves in + bpo-10141(2).) + + * The *note socket: 674. class now supports the PF_RDS protocol + family (‘https://en.wikipedia.org/wiki/Reliable_Datagram_Sockets’ + and ‘https://oss.oracle.com/projects/rds/’). + + * The *note socket: 674. class now supports the ‘PF_SYSTEM’ protocol + family on OS X. (Contributed by Michael Goderbauer in + bpo-13777(3).) + + * New function *note sethostname(): a87. allows the hostname to be + set on unix systems if the calling process has sufficient + privileges. (Contributed by Ross Lagerwall in bpo-10866(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6560 + + (2) https://bugs.python.org/issue10141 + + (3) https://bugs.python.org/issue13777 + + (4) https://bugs.python.org/issue10866 + + +File: python.info, Node: socketserver<3>, Next: sqlite3<5>, Prev: socket<7>, Up: Improved Modules<6> + +1.6.20.47 socketserver +...................... + +*note BaseServer: a89. now has an overridable method *note +service_actions(): a8a. that is called by the *note serve_forever(): +a8b. method in the service loop. *note ForkingMixIn: 3d5. now uses this +to clean up zombie child processes. (Contributed by Justin Warkentin in +bpo-11109(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11109 + + +File: python.info, Node: sqlite3<5>, Next: ssl<8>, Prev: socketserver<3>, Up: Improved Modules<6> + +1.6.20.48 sqlite3 +................. + +New *note sqlite3.Connection: 3d8. method *note set_trace_callback(): +a8d. can be used to capture a trace of all sql commands processed by +sqlite. (Contributed by Torsten Landschoff in bpo-11688(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11688 + + +File: python.info, Node: ssl<8>, Next: stat<2>, Prev: sqlite3<5>, Up: Improved Modules<6> + +1.6.20.49 ssl +............. + + * The *note ssl: f3. module has two new random generation functions: + + * *note RAND_bytes(): a8f.: generate cryptographically strong + pseudo-random bytes. + + * *note RAND_pseudo_bytes(): a90.: generate pseudo-random bytes. + + (Contributed by Victor Stinner in bpo-12049(1).) + + * The *note ssl: f3. module now exposes a finer-grained exception + hierarchy in order to make it easier to inspect the various kinds + of errors. (Contributed by Antoine Pitrou in bpo-11183(2).) + + * *note load_cert_chain(): a91. now accepts a `password' argument to + be used if the private key is encrypted. (Contributed by Adam + Simpkins in bpo-12803(3).) + + * Diffie-Hellman key exchange, both regular and Elliptic Curve-based, + is now supported through the *note load_dh_params(): a92. and *note + set_ecdh_curve(): a93. methods. (Contributed by Antoine Pitrou in + bpo-13626(4) and bpo-13627(5).) + + * SSL sockets have a new *note get_channel_binding(): a94. method + allowing the implementation of certain authentication mechanisms + such as SCRAM-SHA-1-PLUS. (Contributed by Jacek Konieczny in + bpo-12551(6).) + + * You can query the SSL compression algorithm used by an SSL socket, + thanks to its new *note compression(): a95. method. The new + attribute *note OP_NO_COMPRESSION: a96. can be used to disable + compression. (Contributed by Antoine Pitrou in bpo-13634(7).) + + * Support has been added for the Next Protocol Negotiation extension + using the *note ssl.SSLContext.set_npn_protocols(): a97. method. + (Contributed by Colin Marc in bpo-14204(8).) + + * SSL errors can now be introspected more easily thanks to *note + library: a98. and *note reason: a99. attributes. (Contributed by + Antoine Pitrou in bpo-14837(9).) + + * The *note get_server_certificate(): a9a. function now supports + IPv6. (Contributed by Charles-François Natali in bpo-11811(10).) + + * New attribute *note OP_CIPHER_SERVER_PREFERENCE: a9b. allows + setting SSLv3 server sockets to use the server’s cipher ordering + preference rather than the client’s (bpo-13635(11)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12049 + + (2) https://bugs.python.org/issue11183 + + (3) https://bugs.python.org/issue12803 + + (4) https://bugs.python.org/issue13626 + + (5) https://bugs.python.org/issue13627 + + (6) https://bugs.python.org/issue12551 + + (7) https://bugs.python.org/issue13634 + + (8) https://bugs.python.org/issue14204 + + (9) https://bugs.python.org/issue14837 + + (10) https://bugs.python.org/issue11811 + + (11) https://bugs.python.org/issue13635 + + +File: python.info, Node: stat<2>, Next: struct<3>, Prev: ssl<8>, Up: Improved Modules<6> + +1.6.20.50 stat +.............. + +The undocumented tarfile.filemode function has been moved to *note +stat.filemode(): a9d. It can be used to convert a file’s mode to a +string of the form ‘-rwxrwxrwx’. + +(Contributed by Giampaolo Rodolà in bpo-14807(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14807 + + +File: python.info, Node: struct<3>, Next: subprocess<5>, Prev: stat<2>, Up: Improved Modules<6> + +1.6.20.51 struct +................ + +The *note struct: f8. module now supports ‘ssize_t’ and ‘size_t’ via the +new codes ‘n’ and ‘N’, respectively. (Contributed by Antoine Pitrou in +bpo-3163(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue3163 + + +File: python.info, Node: subprocess<5>, Next: sys<7>, Prev: struct<3>, Up: Improved Modules<6> + +1.6.20.52 subprocess +.................... + +Command strings can now be bytes objects on posix platforms. +(Contributed by Victor Stinner in bpo-8513(1).) + +A new constant *note DEVNULL: aa0. allows suppressing output in a +platform-independent fashion. (Contributed by Ross Lagerwall in +bpo-5870(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8513 + + (2) https://bugs.python.org/issue5870 + + +File: python.info, Node: sys<7>, Next: tarfile<4>, Prev: subprocess<5>, Up: Improved Modules<6> + +1.6.20.53 sys +............. + +The *note sys: fd. module has a new *note thread_info: aa2. *note named +tuple: aa3. holding information about the thread implementation +(bpo-11223(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue11223 + + +File: python.info, Node: tarfile<4>, Next: tempfile, Prev: sys<7>, Up: Improved Modules<6> + +1.6.20.54 tarfile +................. + +*note tarfile: 101. now supports ‘lzma’ encoding via the *note lzma: ad. +module. (Contributed by Lars Gustäbel in bpo-5689(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5689 + + +File: python.info, Node: tempfile, Next: textwrap<2>, Prev: tarfile<4>, Up: Improved Modules<6> + +1.6.20.55 tempfile +.................. + +*note tempfile.SpooledTemporaryFile: aa6.’s ‘truncate()’ method now +accepts a ‘size’ parameter. (Contributed by Ryan Kelly in bpo-9957(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9957 + + +File: python.info, Node: textwrap<2>, Next: threading<5>, Prev: tempfile, Up: Improved Modules<6> + +1.6.20.56 textwrap +.................. + +The *note textwrap: 108. module has a new *note indent(): aa8. that +makes it straightforward to add a common prefix to selected lines in a +block of text (bpo-13857(1)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13857 + + +File: python.info, Node: threading<5>, Next: time<5>, Prev: textwrap<2>, Up: Improved Modules<6> + +1.6.20.57 threading +................... + +*note threading.Condition: aaa, *note threading.Semaphore: aab, *note +threading.BoundedSemaphore: aac, *note threading.Event: aad, and *note +threading.Timer: aae, all of which used to be factory functions +returning a class instance, are now classes and may be subclassed. +(Contributed by Éric Araujo in bpo-10968(1).) + +The *note threading.Thread: 235. constructor now accepts a ‘daemon’ +keyword argument to override the default behavior of inheriting the +‘daemon’ flag value from the parent thread (bpo-6064(2)). + +The formerly private function ‘_thread.get_ident’ is now available as +the public function *note threading.get_ident(): aaf. This eliminates +several cases of direct access to the ‘_thread’ module in the stdlib. +Third party code that used ‘_thread.get_ident’ should likewise be +changed to use the new public interface. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10968 + + (2) https://bugs.python.org/issue6064 + + +File: python.info, Node: time<5>, Next: types<4>, Prev: threading<5>, Up: Improved Modules<6> + +1.6.20.58 time +.............. + +The PEP 418(1) added new functions to the *note time: 10a. module: + + * *note get_clock_info(): ab1.: Get information on a clock. + + * *note monotonic(): 76f.: Monotonic clock (cannot go backward), not + affected by system clock updates. + + * *note perf_counter(): 2aa.: Performance counter with the highest + available resolution to measure a short duration. + + * *note process_time(): 2ab.: Sum of the system and user CPU time of + the current process. + +Other new functions: + + * *note clock_getres(): ab2, *note clock_gettime(): ab3. and *note + clock_settime(): ab4. functions with ‘CLOCK_xxx’ constants. + (Contributed by Victor Stinner in bpo-10278(2).) + +To improve cross platform consistency, *note sleep(): 680. now raises a +*note ValueError: 1fb. when passed a negative sleep value. Previously +this was an error on posix, but produced an infinite sleep on Windows. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0418 + + (2) https://bugs.python.org/issue10278 + + +File: python.info, Node: types<4>, Next: unittest<5>, Prev: time<5>, Up: Improved Modules<6> + +1.6.20.59 types +............... + +Add a new *note types.MappingProxyType: ab6. class: Read-only proxy of a +mapping. (bpo-14386(1)) + +The new functions *note types.new_class(): ab7. and *note +types.prepare_class(): ab8. provide support for PEP 3115(2) compliant +dynamic type creation. (bpo-14588(3)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14386 + + (2) https://www.python.org/dev/peps/pep-3115 + + (3) https://bugs.python.org/issue14588 + + +File: python.info, Node: unittest<5>, Next: urllib<3>, Prev: types<4>, Up: Improved Modules<6> + +1.6.20.60 unittest +.................. + +*note assertRaises(): aba, *note assertRaisesRegex(): abb, *note +assertWarns(): abc, and *note assertWarnsRegex(): abd. now accept a +keyword argument `msg' when used as context managers. (Contributed by +Ezio Melotti and Winston Ewert in bpo-10775(1).) + +*note unittest.TestCase.run(): abe. now returns the *note TestResult: +abf. object. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10775 + + +File: python.info, Node: urllib<3>, Next: webbrowser, Prev: unittest<5>, Up: Improved Modules<6> + +1.6.20.61 urllib +................ + +The *note Request: 8f6. class, now accepts a `method' argument used by +*note get_method(): ac1. to determine what HTTP method should be used. +For example, this will send a ‘'HEAD'’ request: + + >>> urlopen(Request('https://www.python.org', method='HEAD')) + +(bpo-1673007(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1673007 + + +File: python.info, Node: webbrowser, Next: xml etree ElementTree, Prev: urllib<3>, Up: Improved Modules<6> + +1.6.20.62 webbrowser +.................... + +The *note webbrowser: 129. module supports more “browsers”: Google +Chrome (named ‘chrome’, ‘chromium’, ‘chrome-browser’ or +‘chromium-browser’ depending on the version and operating system), and +the generic launchers ‘xdg-open’, from the FreeDesktop.org project, and +‘gvfs-open’, which is the default URI handler for GNOME 3. (The former +contributed by Arnaud Calmettes in bpo-13620(1), the latter by Matthias +Klose in bpo-14493(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue13620 + + (2) https://bugs.python.org/issue14493 + + +File: python.info, Node: xml etree ElementTree, Next: zlib<2>, Prev: webbrowser, Up: Improved Modules<6> + +1.6.20.63 xml.etree.ElementTree +............................... + +The *note xml.etree.ElementTree: 137. module now imports its C +accelerator by default; there is no longer a need to explicitly import +‘xml.etree.cElementTree’ (this module stays for backwards compatibility, +but is now deprecated). In addition, the ‘iter’ family of methods of +*note Element: ac4. has been optimized (rewritten in C). The module’s +documentation has also been greatly improved with added examples and a +more detailed reference. + + +File: python.info, Node: zlib<2>, Prev: xml etree ElementTree, Up: Improved Modules<6> + +1.6.20.64 zlib +.............. + +New attribute *note zlib.Decompress.eof: ac6. makes it possible to +distinguish between a properly-formed compressed stream and an +incomplete or truncated one. (Contributed by Nadeem Vawda in +bpo-12646(1).) + +New attribute *note zlib.ZLIB_RUNTIME_VERSION: ac7. reports the version +string of the underlying ‘zlib’ library that is loaded at runtime. +(Contributed by Torsten Landschoff in bpo-12306(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12646 + + (2) https://bugs.python.org/issue12306 + + +File: python.info, Node: Optimizations<5>, Next: Build and C API Changes<4>, Prev: Improved Modules<6>, Up: What’s New In Python 3 3 + +1.6.21 Optimizations +-------------------- + +Major performance enhancements have been added: + + * Thanks to PEP 393(1), some operations on Unicode strings have been + optimized: + + * the memory footprint is divided by 2 to 4 depending on the + text + + * encode an ASCII string to UTF-8 doesn’t need to encode + characters anymore, the UTF-8 representation is shared with + the ASCII representation + + * the UTF-8 encoder has been optimized + + * repeating a single ASCII letter and getting a substring of an + ASCII string is 4 times faster + + * UTF-8 is now 2x to 4x faster. UTF-16 encoding is now up to 10x + faster. + + (Contributed by Serhiy Storchaka, bpo-14624(2), bpo-14738(3) and + bpo-15026(4).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + (2) https://bugs.python.org/issue14624 + + (3) https://bugs.python.org/issue14738 + + (4) https://bugs.python.org/issue15026 + + +File: python.info, Node: Build and C API Changes<4>, Next: Deprecated<5>, Prev: Optimizations<5>, Up: What’s New In Python 3 3 + +1.6.22 Build and C API Changes +------------------------------ + +Changes to Python’s build process and to the C API include: + + * New PEP 3118(1) related function: + + * *note PyMemoryView_FromMemory(): ac9. + + * PEP 393(2) added new Unicode types, macros and functions: + + * High-level API: + + * *note PyUnicode_CopyCharacters(): aca. + + * *note PyUnicode_FindChar(): 440. + + * *note PyUnicode_GetLength(): acb, *note + PyUnicode_GET_LENGTH: acc. + + * *note PyUnicode_New(): acd. + + * *note PyUnicode_Substring(): ace. + + * *note PyUnicode_ReadChar(): acf, *note + PyUnicode_WriteChar(): ad0. + + * Low-level API: + + * *note Py_UCS1: ad1, *note Py_UCS2: ad2, *note Py_UCS4: + ad3. types + + * *note PyASCIIObject: ad4. and *note + PyCompactUnicodeObject: ad5. structures + + * *note PyUnicode_READY: ad6. + + * *note PyUnicode_FromKindAndData(): ad7. + + * *note PyUnicode_AsUCS4(): ad8, *note + PyUnicode_AsUCS4Copy(): ad9. + + * *note PyUnicode_DATA: ada, *note PyUnicode_1BYTE_DATA: + adb, *note PyUnicode_2BYTE_DATA: adc, *note + PyUnicode_4BYTE_DATA: add. + + * *note PyUnicode_KIND: ade. with ‘PyUnicode_Kind’ enum: + *note PyUnicode_WCHAR_KIND: adf, *note + PyUnicode_1BYTE_KIND: ae0, *note PyUnicode_2BYTE_KIND: + ae1, *note PyUnicode_4BYTE_KIND: ae2. + + * *note PyUnicode_READ: ae3, *note PyUnicode_READ_CHAR: + ae4, *note PyUnicode_WRITE: ae5. + + * *note PyUnicode_MAX_CHAR_VALUE: ae6. + + * *note PyArg_ParseTuple: 26a. now accepts a *note bytearray: 332. + for the ‘c’ format (bpo-12380(3)). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3118 + + (2) https://www.python.org/dev/peps/pep-0393 + + (3) https://bugs.python.org/issue12380 + + +File: python.info, Node: Deprecated<5>, Next: Porting to Python 3 3, Prev: Build and C API Changes<4>, Up: What’s New In Python 3 3 + +1.6.23 Deprecated +----------------- + +* Menu: + +* Unsupported Operating Systems: Unsupported Operating Systems<2>. +* Deprecated Python modules, functions and methods: Deprecated Python modules functions and methods<4>. +* Deprecated functions and types of the C API: Deprecated functions and types of the C API<3>. +* Deprecated features:: + + +File: python.info, Node: Unsupported Operating Systems<2>, Next: Deprecated Python modules functions and methods<4>, Up: Deprecated<5> + +1.6.23.1 Unsupported Operating Systems +...................................... + +OS/2 and VMS are no longer supported due to the lack of a maintainer. + +Windows 2000 and Windows platforms which set ‘COMSPEC’ to ‘command.com’ +are no longer supported due to maintenance burden. + +OSF support, which was deprecated in 3.2, has been completely removed. + + +File: python.info, Node: Deprecated Python modules functions and methods<4>, Next: Deprecated functions and types of the C API<3>, Prev: Unsupported Operating Systems<2>, Up: Deprecated<5> + +1.6.23.2 Deprecated Python modules, functions and methods +......................................................... + + * Passing a non-empty string to ‘object.__format__()’ is deprecated, + and will produce a *note TypeError: 192. in Python 3.4 + (bpo-9856(1)). + + * The ‘unicode_internal’ codec has been deprecated because of the PEP + 393(2), use UTF-8, UTF-16 (‘utf-16-le’ or ‘utf-16-be’), or UTF-32 + (‘utf-32-le’ or ‘utf-32-be’) + + * *note ftplib.FTP.nlst(): a06. and *note ftplib.FTP.dir(): a07.: use + *note ftplib.FTP.mlsd(): a05. + + * ‘platform.popen()’: use the *note subprocess: f9. module. Check + especially the *note Replacing Older Functions with the subprocess + Module: aea. section (bpo-11377(3)). + + * bpo-13374(4): The Windows bytes API has been deprecated in the + *note os: c4. module. Use Unicode filenames, instead of bytes + filenames, to not depend on the ANSI code page anymore and to + support any filename. + + * bpo-13988(5): The ‘xml.etree.cElementTree’ module is deprecated. + The accelerator is used automatically whenever available. + + * The behaviour of ‘time.clock()’ depends on the platform: use the + new *note time.perf_counter(): 2aa. or *note time.process_time(): + 2ab. function instead, depending on your requirements, to have a + well defined behaviour. + + * The ‘os.stat_float_times()’ function is deprecated. + + * *note abc: 4. module: + + * *note abc.abstractproperty: 9cd. has been deprecated, use + *note property: 1d7. with *note abc.abstractmethod(): 9ce. + instead. + + * *note abc.abstractclassmethod: 9cf. has been deprecated, use + *note classmethod: 1d8. with *note abc.abstractmethod(): 9ce. + instead. + + * *note abc.abstractstaticmethod: 9d0. has been deprecated, use + *note staticmethod: 1d9. with *note abc.abstractmethod(): 9ce. + instead. + + * *note importlib: 9b. package: + + * *note importlib.abc.SourceLoader.path_mtime(): aeb. is now + deprecated in favour of *note + importlib.abc.SourceLoader.path_stats(): aec. as bytecode + files now store both the modification time and size of the + source file the bytecode file was compiled from. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9856 + + (2) https://www.python.org/dev/peps/pep-0393 + + (3) https://bugs.python.org/issue11377 + + (4) https://bugs.python.org/issue13374 + + (5) https://bugs.python.org/issue13988 + + +File: python.info, Node: Deprecated functions and types of the C API<3>, Next: Deprecated features, Prev: Deprecated Python modules functions and methods<4>, Up: Deprecated<5> + +1.6.23.3 Deprecated functions and types of the C API +.................................................... + +The *note Py_UNICODE: aee. has been deprecated by PEP 393(1) and will be +removed in Python 4. All functions using this type are deprecated: + +Unicode functions and methods using *note Py_UNICODE: aee. and *note +Py_UNICODE*: aee. types: + + * *note PyUnicode_FromUnicode: aef.: use *note + PyUnicode_FromWideChar(): af0. or *note + PyUnicode_FromKindAndData(): ad7. + + * *note PyUnicode_AS_UNICODE: af1, *note PyUnicode_AsUnicode(): af2, + *note PyUnicode_AsUnicodeAndSize(): af3.: use *note + PyUnicode_AsWideCharString(): 438. + + * *note PyUnicode_AS_DATA: af4.: use *note PyUnicode_DATA: ada. with + *note PyUnicode_READ: ae3. and *note PyUnicode_WRITE: ae5. + + * *note PyUnicode_GET_SIZE: af5, *note PyUnicode_GetSize(): af6.: use + *note PyUnicode_GET_LENGTH: acc. or *note PyUnicode_GetLength(): + acb. + + * *note PyUnicode_GET_DATA_SIZE: af7.: use ‘PyUnicode_GET_LENGTH(str) + * PyUnicode_KIND(str)’ (only work on ready strings) + + * *note PyUnicode_AsUnicodeCopy(): af8.: use *note + PyUnicode_AsUCS4Copy(): ad9. or *note PyUnicode_AsWideCharString(): + 438. + + * ‘PyUnicode_GetMax()’ + +Functions and macros manipulating Py_UNICODE* strings: + + * ‘Py_UNICODE_strlen’: use *note PyUnicode_GetLength(): acb. or *note + PyUnicode_GET_LENGTH: acc. + + * ‘Py_UNICODE_strcat’: use *note PyUnicode_CopyCharacters(): aca. or + *note PyUnicode_FromFormat(): 7cb. + + * ‘Py_UNICODE_strcpy’, ‘Py_UNICODE_strncpy’, ‘Py_UNICODE_COPY’: use + *note PyUnicode_CopyCharacters(): aca. or *note + PyUnicode_Substring(): ace. + + * ‘Py_UNICODE_strcmp’: use *note PyUnicode_Compare(): af9. + + * ‘Py_UNICODE_strncmp’: use *note PyUnicode_Tailmatch(): afa. + + * ‘Py_UNICODE_strchr’, ‘Py_UNICODE_strrchr’: use *note + PyUnicode_FindChar(): 440. + + * ‘Py_UNICODE_FILL’: use *note PyUnicode_Fill(): afb. + + * ‘Py_UNICODE_MATCH’ + +Encoders: + + * *note PyUnicode_Encode(): afc.: use ‘PyUnicode_AsEncodedObject()’ + + * *note PyUnicode_EncodeUTF7(): afd. + + * *note PyUnicode_EncodeUTF8(): afe.: use *note PyUnicode_AsUTF8(): + 42b. or *note PyUnicode_AsUTF8String(): aff. + + * *note PyUnicode_EncodeUTF32(): b00. + + * *note PyUnicode_EncodeUTF16(): b01. + + * *note PyUnicode_EncodeUnicodeEscape(): b02. use *note + PyUnicode_AsUnicodeEscapeString(): b03. + + * *note PyUnicode_EncodeRawUnicodeEscape(): b04. use *note + PyUnicode_AsRawUnicodeEscapeString(): b05. + + * *note PyUnicode_EncodeLatin1(): b06.: use *note + PyUnicode_AsLatin1String(): b07. + + * *note PyUnicode_EncodeASCII(): b08.: use *note + PyUnicode_AsASCIIString(): b09. + + * *note PyUnicode_EncodeCharmap(): b0a. + + * *note PyUnicode_TranslateCharmap(): b0b. + + * *note PyUnicode_EncodeMBCS(): b0c.: use *note + PyUnicode_AsMBCSString(): b0d. or *note PyUnicode_EncodeCodePage(): + b0e. (with ‘CP_ACP’ code_page) + + * ‘PyUnicode_EncodeDecimal()’, *note + PyUnicode_TransformDecimalToASCII(): b0f. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + +File: python.info, Node: Deprecated features, Prev: Deprecated functions and types of the C API<3>, Up: Deprecated<5> + +1.6.23.4 Deprecated features +............................ + +The *note array: 7. module’s ‘'u'’ format code is now deprecated and +will be removed in Python 4 together with the rest of the (*note +Py_UNICODE: aee.) API. + + +File: python.info, Node: Porting to Python 3 3, Prev: Deprecated<5>, Up: What’s New In Python 3 3 + +1.6.24 Porting to Python 3.3 +---------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code. + +* Menu: + +* Porting Python code:: +* Porting C code:: +* Building C extensions:: +* Command Line Switch Changes:: + + +File: python.info, Node: Porting Python code, Next: Porting C code, Up: Porting to Python 3 3 + +1.6.24.1 Porting Python code +............................ + + * Hash randomization is enabled by default. Set the *note + PYTHONHASHSEED: 9c5. environment variable to ‘0’ to disable hash + randomization. See also the *note object.__hash__(): 340. method. + + * bpo-12326(1): On Linux, sys.platform doesn’t contain the major + version anymore. It is now always ‘linux’, instead of ‘linux2’ or + ‘linux3’ depending on the Linux version used to build Python. + Replace sys.platform == ‘linux2’ with + sys.platform.startswith(‘linux’), or directly sys.platform == + ‘linux’ if you don’t need to support older Python versions. + + * bpo-13847(2), bpo-14180(3): *note time: 10a. and *note datetime: + 31.: *note OverflowError: 960. is now raised instead of *note + ValueError: 1fb. if a timestamp is out of range. *note OSError: + 1d3. is now raised if C functions ‘gmtime()’ or ‘localtime()’ + failed. + + * The default finders used by import now utilize a cache of what is + contained within a specific directory. If you create a Python + source file or sourceless bytecode file, make sure to call *note + importlib.invalidate_caches(): 49c. to clear out the cache for the + finders to notice the new file. + + * *note ImportError: 334. now uses the full name of the module that + was attempted to be imported. Doctests that check ImportErrors’ + message will need to be updated to use the full name of the module + instead of just the tail of the name. + + * The `index' argument to *note __import__(): 610. now defaults to 0 + instead of -1 and no longer support negative values. It was an + oversight when PEP 328(4) was implemented that the default value + remained -1. If you need to continue to perform a relative import + followed by an absolute import, then perform the relative import + using an index of 1, followed by another import using an index of + 0. It is preferred, though, that you use *note + importlib.import_module(): b13. rather than call *note + __import__(): 610. directly. + + * *note __import__(): 610. no longer allows one to use an index value + other than 0 for top-level modules. E.g. ‘__import__('sys', + level=1)’ is now an error. + + * Because *note sys.meta_path: 5e6. and *note sys.path_hooks: 95c. + now have finders on them by default, you will most likely want to + use ‘list.insert()’ instead of ‘list.append()’ to add to those + lists. + + * Because ‘None’ is now inserted into *note sys.path_importer_cache: + 49d, if you are clearing out entries in the dictionary of paths + that do not have a finder, you will need to remove keys paired with + values of ‘None’ `and' *note imp.NullImporter: 9bb. to be + backwards-compatible. This will lead to extra overhead on older + versions of Python that re-insert ‘None’ into *note + sys.path_importer_cache: 49d. where it represents the use of + implicit finders, but semantically it should not change anything. + + * *note importlib.abc.Finder: 9b5. no longer specifies a + ‘find_module()’ abstract method that must be implemented. If you + were relying on subclasses to implement that method, make sure to + check for the method’s existence first. You will probably want to + check for ‘find_loader()’ first, though, in the case of working + with *note path entry finders: 9b2. + + * *note pkgutil: cd. has been converted to use *note importlib: 9b. + internally. This eliminates many edge cases where the old + behaviour of the PEP 302(5) import emulation failed to match the + behaviour of the real import system. The import emulation itself + is still present, but is now deprecated. The *note + pkgutil.iter_importers(): b14. and *note pkgutil.walk_packages(): + 48b. functions special case the standard import hooks so they are + still supported even though they do not provide the non-standard + ‘iter_modules()’ method. + + * A longstanding RFC-compliance bug (bpo-1079(6)) in the parsing done + by *note email.header.decode_header(): 9f9. has been fixed. Code + that uses the standard idiom to convert encoded headers into + unicode (‘str(make_header(decode_header(h))’) will see no change, + but code that looks at the individual tuples returned by + decode_header will see that whitespace that precedes or follows + ‘ASCII’ sections is now included in the ‘ASCII’ section. Code that + builds headers using ‘make_header’ should also continue to work + without change, since ‘make_header’ continues to add whitespace + between ‘ASCII’ and non-‘ASCII’ sections if it is not already + present in the input strings. + + * *note email.utils.formataddr(): b15. now does the correct content + transfer encoding when passed non-‘ASCII’ display names. Any code + that depended on the previous buggy behavior that preserved the + non-‘ASCII’ unicode in the formatted output string will need to be + changed (bpo-1690608(7)). + + * *note poplib.POP3.quit(): b16. may now raise protocol errors like + all other ‘poplib’ methods. Code that assumes ‘quit’ does not + raise *note poplib.error_proto: b17. errors may need to be changed + if errors on ‘quit’ are encountered by a particular application + (bpo-11291(8)). + + * The ‘strict’ argument to *note email.parser.Parser: b18, deprecated + since Python 2.4, has finally been removed. + + * The deprecated method ‘unittest.TestCase.assertSameElements’ has + been removed. + + * The deprecated variable ‘time.accept2dyear’ has been removed. + + * The deprecated ‘Context._clamp’ attribute has been removed from the + *note decimal: 36. module. It was previously replaced by the + public attribute ‘clamp’. (See bpo-8540(9).) + + * The undocumented internal helper class ‘SSLFakeFile’ has been + removed from *note smtplib: ed, since its functionality has long + been provided directly by *note socket.socket.makefile(): b19. + + * Passing a negative value to *note time.sleep(): 680. on Windows now + raises an error instead of sleeping forever. It has always raised + an error on posix. + + * The ‘ast.__version__’ constant has been removed. If you need to + make decisions affected by the AST version, use *note + sys.version_info: b1a. to make the decision. + + * Code that used to work around the fact that the *note threading: + 109. module used factory functions by subclassing the private + classes will need to change to subclass the now-public classes. + + * The undocumented debugging machinery in the threading module has + been removed, simplifying the code. This should have no effect on + production code, but is mentioned here in case any application + debug frameworks were interacting with it (bpo-13550(10)). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue12326 + + (2) https://bugs.python.org/issue13847 + + (3) https://bugs.python.org/issue14180 + + (4) https://www.python.org/dev/peps/pep-0328 + + (5) https://www.python.org/dev/peps/pep-0302 + + (6) https://bugs.python.org/issue1079 + + (7) https://bugs.python.org/issue1690608 + + (8) https://bugs.python.org/issue11291 + + (9) https://bugs.python.org/issue8540 + + (10) https://bugs.python.org/issue13550 + + +File: python.info, Node: Porting C code, Next: Building C extensions, Prev: Porting Python code, Up: Porting to Python 3 3 + +1.6.24.2 Porting C code +....................... + + * In the course of changes to the buffer API the undocumented + ‘smalltable’ member of the *note Py_buffer: b1b. structure has been + removed and the layout of the ‘PyMemoryViewObject’ has changed. + + All extensions relying on the relevant parts in ‘memoryobject.h’ or + ‘object.h’ must be rebuilt. + + * Due to *note PEP 393: 97c, the *note Py_UNICODE: aee. type and all + functions using this type are deprecated (but will stay available + for at least five years). If you were using low-level Unicode APIs + to construct and access unicode objects and you want to benefit of + the memory footprint reduction provided by PEP 393(1), you have to + convert your code to the new *note Unicode API: b1c. + + However, if you only have been using high-level functions such as + *note PyUnicode_Concat(): b1d, *note PyUnicode_Join(): b1e. or + *note PyUnicode_FromFormat(): 7cb, your code will automatically + take advantage of the new unicode representations. + + * *note PyImport_GetMagicNumber(): b1f. now returns ‘-1’ upon + failure. + + * As a negative value for the `level' argument to *note __import__(): + 610. is no longer valid, the same now holds for *note + PyImport_ImportModuleLevel(): b20. This also means that the value + of `level' used by *note PyImport_ImportModuleEx(): b21. is now ‘0’ + instead of ‘-1’. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + +File: python.info, Node: Building C extensions, Next: Command Line Switch Changes, Prev: Porting C code, Up: Porting to Python 3 3 + +1.6.24.3 Building C extensions +.............................. + + * The range of possible file names for C extensions has been + narrowed. Very rarely used spellings have been suppressed: under + POSIX, files named ‘xxxmodule.so’, ‘xxxmodule.abi3.so’ and + ‘xxxmodule.cpython-*.so’ are no longer recognized as implementing + the ‘xxx’ module. If you had been generating such files, you have + to switch to the other spellings (i.e., remove the ‘module’ string + from the file names). + + (implemented in bpo-14040(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue14040 + + +File: python.info, Node: Command Line Switch Changes, Prev: Building C extensions, Up: Porting to Python 3 3 + +1.6.24.4 Command Line Switch Changes +.................................... + + * The -Q command-line flag and related artifacts have been removed. + Code checking sys.flags.division_warning will need updating. + + (bpo-10998(1), contributed by Éric Araujo.) + + * When ‘python’ is started with *note -S: b24, ‘import site’ will no + longer add site-specific paths to the module search paths. In + previous versions, it did. + + (bpo-11591(2), contributed by Carl Meyer with editions by Éric + Araujo.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10998 + + (2) https://bugs.python.org/issue11591 + + +File: python.info, Node: What’s New In Python 3 2, Next: What’s New In Python 3 1, Prev: What’s New In Python 3 3, Up: What’s New in Python + +1.7 What’s New In Python 3.2 +============================ + + +Author: Raymond Hettinger + +This article explains the new features in Python 3.2 as compared to 3.1. +It focuses on a few highlights and gives a few examples. For full +details, see the Misc/NEWS(1) file. + +See also +........ + +PEP 392(2) - Python 3.2 Release Schedule + +* Menu: + +* PEP 384; Defining a Stable ABI: PEP 384 Defining a Stable ABI. +* PEP 389; Argparse Command Line Parsing Module: PEP 389 Argparse Command Line Parsing Module. +* PEP 391; Dictionary Based Configuration for Logging: PEP 391 Dictionary Based Configuration for Logging. +* PEP 3148; The concurrent.futures module: PEP 3148 The concurrent futures module. +* PEP 3147; PYC Repository Directories: PEP 3147 PYC Repository Directories. +* PEP 3149; ABI Version Tagged .so Files: PEP 3149 ABI Version Tagged so Files. +* PEP 3333; Python Web Server Gateway Interface v1.0.1: PEP 3333 Python Web Server Gateway Interface v1 0 1. +* Other Language Changes: Other Language Changes<7>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules. +* Multi-threading:: +* Optimizations: Optimizations<6>. +* Unicode:: +* Codecs:: +* Documentation:: +* IDLE:: +* Code Repository:: +* Build and C API Changes: Build and C API Changes<5>. +* Porting to Python 3.2: Porting to Python 3 2. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/blob/076ca6c3c8df3030307e548d9be792ce3c1c6eea/Misc/NEWS + + (2) https://www.python.org/dev/peps/pep-0392 + + +File: python.info, Node: PEP 384 Defining a Stable ABI, Next: PEP 389 Argparse Command Line Parsing Module, Up: What’s New In Python 3 2 + +1.7.1 PEP 384: Defining a Stable ABI +------------------------------------ + +In the past, extension modules built for one Python version were often +not usable with other Python versions. Particularly on Windows, every +feature release of Python required rebuilding all extension modules that +one wanted to use. This requirement was the result of the free access +to Python interpreter internals that extension modules could use. + +With Python 3.2, an alternative approach becomes available: extension +modules which restrict themselves to a limited API (by defining +Py_LIMITED_API) cannot use many of the internals, but are constrained to +a set of API functions that are promised to be stable for several +releases. As a consequence, extension modules built for 3.2 in that +mode will also work with 3.3, 3.4, and so on. Extension modules that +make use of details of memory structures can still be built, but will +need to be recompiled for every feature release. + +See also +........ + +PEP 384(1) - Defining a Stable ABI + + PEP written by Martin von Löwis. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0384 + + +File: python.info, Node: PEP 389 Argparse Command Line Parsing Module, Next: PEP 391 Dictionary Based Configuration for Logging, Prev: PEP 384 Defining a Stable ABI, Up: What’s New In Python 3 2 + +1.7.2 PEP 389: Argparse Command Line Parsing Module +--------------------------------------------------- + +A new module for command line parsing, *note argparse: 6, was introduced +to overcome the limitations of *note optparse: c3. which did not provide +support for positional arguments (not just options), subcommands, +required options and other common patterns of specifying and validating +options. + +This module has already had widespread success in the community as a +third-party module. Being more fully featured than its predecessor, the +*note argparse: 6. module is now the preferred module for command-line +processing. The older module is still being kept available because of +the substantial amount of legacy code that depends on it. + +Here’s an annotated example parser showing features like limiting +results to a set of choices, specifying a `metavar' in the help screen, +validating that one or more positional arguments is present, and making +a required option: + + import argparse + parser = argparse.ArgumentParser( + description = 'Manage servers', # main description for help + epilog = 'Tested on Solaris and Linux') # displayed after help + parser.add_argument('action', # argument name + choices = ['deploy', 'start', 'stop'], # three allowed values + help = 'action on each target') # help msg + parser.add_argument('targets', + metavar = 'HOSTNAME', # var name used in help msg + nargs = '+', # require one or more targets + help = 'url for target machines') # help msg explanation + parser.add_argument('-u', '--user', # -u or --user option + required = True, # make it a required argument + help = 'login as user') + +Example of calling the parser on a command string: + + >>> cmd = 'deploy sneezy.example.com sleepy.example.com -u skycaptain' + >>> result = parser.parse_args(cmd.split()) + >>> result.action + 'deploy' + >>> result.targets + ['sneezy.example.com', 'sleepy.example.com'] + >>> result.user + 'skycaptain' + +Example of the parser’s automatically generated help: + + >>> parser.parse_args('-h'.split()) + + usage: manage_cloud.py [-h] -u USER + {deploy,start,stop} HOSTNAME [HOSTNAME ...] + + Manage servers + + positional arguments: + {deploy,start,stop} action on each target + HOSTNAME url for target machines + + optional arguments: + -h, --help show this help message and exit + -u USER, --user USER login as user + + Tested on Solaris and Linux + +An especially nice *note argparse: 6. feature is the ability to define +subparsers, each with their own argument patterns and help displays: + + import argparse + parser = argparse.ArgumentParser(prog='HELM') + subparsers = parser.add_subparsers() + + parser_l = subparsers.add_parser('launch', help='Launch Control') # first subgroup + parser_l.add_argument('-m', '--missiles', action='store_true') + parser_l.add_argument('-t', '--torpedos', action='store_true') + + parser_m = subparsers.add_parser('move', help='Move Vessel', # second subgroup + aliases=('steer', 'turn')) # equivalent names + parser_m.add_argument('-c', '--course', type=int, required=True) + parser_m.add_argument('-s', '--speed', type=int, default=0) + + $ ./helm.py --help # top level help (launch and move) + $ ./helm.py launch --help # help for launch options + $ ./helm.py launch --missiles # set missiles=True and torpedos=False + $ ./helm.py steer --course 180 --speed 5 # set movement parameters + +See also +........ + +PEP 389(1) - New Command Line Parsing Module + + PEP written by Steven Bethard. + +*note Upgrading optparse code: b29. for details on the differences from +*note optparse: c3. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0389 + + +File: python.info, Node: PEP 391 Dictionary Based Configuration for Logging, Next: PEP 3148 The concurrent futures module, Prev: PEP 389 Argparse Command Line Parsing Module, Up: What’s New In Python 3 2 + +1.7.3 PEP 391: Dictionary Based Configuration for Logging +--------------------------------------------------------- + +The *note logging: aa. module provided two kinds of configuration, one +style with function calls for each option or another style driven by an +external file saved in a ‘ConfigParser’ format. Those options did not +provide the flexibility to create configurations from JSON or YAML +files, nor did they support incremental configuration, which is needed +for specifying logger options from a command line. + +To support a more flexible style, the module now offers *note +logging.config.dictConfig(): b2b. for specifying logging configuration +with plain Python dictionaries. The configuration options include +formatters, handlers, filters, and loggers. Here’s a working example of +a configuration dictionary: + + {"version": 1, + "formatters": {"brief": {"format": "%(levelname)-8s: %(name)-15s: %(message)s"}, + "full": {"format": "%(asctime)s %(name)-15s %(levelname)-8s %(message)s"} + }, + "handlers": {"console": { + "class": "logging.StreamHandler", + "formatter": "brief", + "level": "INFO", + "stream": "ext://sys.stdout"}, + "console_priority": { + "class": "logging.StreamHandler", + "formatter": "full", + "level": "ERROR", + "stream": "ext://sys.stderr"} + }, + "root": {"level": "DEBUG", "handlers": ["console", "console_priority"]}} + +If that dictionary is stored in a file called ‘conf.json’, it can be +loaded and called with code like this: + + >>> import json, logging.config + >>> with open('conf.json') as f: + ... conf = json.load(f) + ... + >>> logging.config.dictConfig(conf) + >>> logging.info("Transaction completed normally") + INFO : root : Transaction completed normally + >>> logging.critical("Abnormal termination") + 2011-02-17 11:14:36,694 root CRITICAL Abnormal termination + +See also +........ + +PEP 391(1) - Dictionary Based Configuration for Logging + + PEP written by Vinay Sajip. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0391 + + +File: python.info, Node: PEP 3148 The concurrent futures module, Next: PEP 3147 PYC Repository Directories, Prev: PEP 391 Dictionary Based Configuration for Logging, Up: What’s New In Python 3 2 + +1.7.4 PEP 3148: The ‘concurrent.futures’ module +----------------------------------------------- + +Code for creating and managing concurrency is being collected in a new +top-level namespace, `concurrent'. Its first member is a `futures' +package which provides a uniform high-level interface for managing +threads and processes. + +The design for *note concurrent.futures: 22. was inspired by the +`java.util.concurrent' package. In that model, a running call and its +result are represented by a *note Future: b2d. object that abstracts +features common to threads, processes, and remote procedure calls. That +object supports status checks (running or done), timeouts, +cancellations, adding callbacks, and access to results or exceptions. + +The primary offering of the new module is a pair of executor classes for +launching and managing calls. The goal of the executors is to make it +easier to use existing tools for making parallel calls. They save the +effort needed to setup a pool of resources, launch the calls, create a +results queue, add time-out handling, and limit the total number of +threads, processes, or remote procedure calls. + +Ideally, each application should share a single executor across multiple +components so that process and thread limits can be centrally managed. +This solves the design challenge that arises when each component has its +own competing strategy for resource management. + +Both classes share a common interface with three methods: *note +submit(): 2a3. for scheduling a callable and returning a *note Future: +b2d. object; *note map(): 6be. for scheduling many asynchronous calls at +a time, and *note shutdown(): b2e. for freeing resources. The class is +a *note context manager: 568. and can be used in a *note with: 6e9. +statement to assure that resources are automatically released when +currently pending futures are done executing. + +A simple of example of *note ThreadPoolExecutor: 27a. is a launch of +four parallel threads for copying files: + + import concurrent.futures, shutil + with concurrent.futures.ThreadPoolExecutor(max_workers=4) as e: + e.submit(shutil.copy, 'src1.txt', 'dest1.txt') + e.submit(shutil.copy, 'src2.txt', 'dest2.txt') + e.submit(shutil.copy, 'src3.txt', 'dest3.txt') + e.submit(shutil.copy, 'src3.txt', 'dest4.txt') + +See also +........ + +PEP 3148(1) - Futures – Execute Computations Asynchronously + + PEP written by Brian Quinlan. + +*note Code for Threaded Parallel URL reads: b2f, an example using +threads to fetch multiple web pages in parallel. + +*note Code for computing prime numbers in parallel: b30, an example +demonstrating *note ProcessPoolExecutor: 2a4. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3148 + + +File: python.info, Node: PEP 3147 PYC Repository Directories, Next: PEP 3149 ABI Version Tagged so Files, Prev: PEP 3148 The concurrent futures module, Up: What’s New In Python 3 2 + +1.7.5 PEP 3147: PYC Repository Directories +------------------------------------------ + +Python’s scheme for caching bytecode in `.pyc' files did not work well +in environments with multiple Python interpreters. If one interpreter +encountered a cached file created by another interpreter, it would +recompile the source and overwrite the cached file, thus losing the +benefits of caching. + +The issue of “pyc fights” has become more pronounced as it has become +commonplace for Linux distributions to ship with multiple versions of +Python. These conflicts also arise with CPython alternatives such as +Unladen Swallow. + +To solve this problem, Python’s import machinery has been extended to +use distinct filenames for each interpreter. Instead of Python 3.2 and +Python 3.3 and Unladen Swallow each competing for a file called +“mymodule.pyc”, they will now look for “mymodule.cpython-32.pyc”, +“mymodule.cpython-33.pyc”, and “mymodule.unladen10.pyc”. And to prevent +all of these new files from cluttering source directories, the `pyc' +files are now collected in a “__pycache__” directory stored under the +package directory. + +Aside from the filenames and target directories, the new scheme has a +few aspects that are visible to the programmer: + + * Imported modules now have a *note __cached__: b32. attribute which + stores the name of the actual file that was imported: + + >>> import collections + >>> collections.__cached__ + 'c:/py32/lib/__pycache__/collections.cpython-32.pyc' + + * The tag that is unique to each interpreter is accessible from the + *note imp: 9a. module: + + >>> import imp + >>> imp.get_tag() + 'cpython-32' + + * Scripts that try to deduce source filename from the imported file + now need to be smarter. It is no longer sufficient to simply strip + the “c” from a “.pyc” filename. Instead, use the new functions in + the *note imp: 9a. module: + + >>> imp.source_from_cache('c:/py32/lib/__pycache__/collections.cpython-32.pyc') + 'c:/py32/lib/collections.py' + >>> imp.cache_from_source('c:/py32/lib/collections.py') + 'c:/py32/lib/__pycache__/collections.cpython-32.pyc' + + * The *note py_compile: d7. and *note compileall: 21. modules have + been updated to reflect the new naming convention and target + directory. The command-line invocation of `compileall' has new + options: ‘-i’ for specifying a list of files and directories to + compile and ‘-b’ which causes bytecode files to be written to their + legacy location rather than `__pycache__'. + + * The *note importlib.abc: 9c. module has been updated with new *note + abstract base classes: b33. for loading bytecode files. The + obsolete ABCs, ‘PyLoader’ and ‘PyPycLoader’, have been deprecated + (instructions on how to stay Python 3.1 compatible are included + with the documentation). + +See also +........ + +PEP 3147(1) - PYC Repository Directories + + PEP written by Barry Warsaw. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: PEP 3149 ABI Version Tagged so Files, Next: PEP 3333 Python Web Server Gateway Interface v1 0 1, Prev: PEP 3147 PYC Repository Directories, Up: What’s New In Python 3 2 + +1.7.6 PEP 3149: ABI Version Tagged .so Files +-------------------------------------------- + +The PYC repository directory allows multiple bytecode cache files to be +co-located. This PEP implements a similar mechanism for shared object +files by giving them a common directory and distinct names for each +version. + +The common directory is “pyshared” and the file names are made distinct +by identifying the Python implementation (such as CPython, PyPy, Jython, +etc.), the major and minor version numbers, and optional build flags +(such as “d” for debug, “m” for pymalloc, “u” for wide-unicode). For an +arbitrary package “foo”, you may see these files when the distribution +package is installed: + + /usr/share/pyshared/foo.cpython-32m.so + /usr/share/pyshared/foo.cpython-33md.so + +In Python itself, the tags are accessible from functions in the *note +sysconfig: fe. module: + + >>> import sysconfig + >>> sysconfig.get_config_var('SOABI') # find the version tag + 'cpython-32mu' + >>> sysconfig.get_config_var('EXT_SUFFIX') # find the full filename extension + '.cpython-32mu.so' + +See also +........ + +PEP 3149(1) - ABI Version Tagged .so Files + + PEP written by Barry Warsaw. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3149 + + +File: python.info, Node: PEP 3333 Python Web Server Gateway Interface v1 0 1, Next: Other Language Changes<7>, Prev: PEP 3149 ABI Version Tagged so Files, Up: What’s New In Python 3 2 + +1.7.7 PEP 3333: Python Web Server Gateway Interface v1.0.1 +---------------------------------------------------------- + +This informational PEP clarifies how bytes/text issues are to be handled +by the WSGI protocol. The challenge is that string handling in Python 3 +is most conveniently handled with the *note str: 330. type even though +the HTTP protocol is itself bytes oriented. + +The PEP differentiates so-called `native strings' that are used for +request/response headers and metadata versus `byte strings' which are +used for the bodies of requests and responses. + +The `native strings' are always of type *note str: 330. but are +restricted to code points between `U+0000' through `U+00FF' which are +translatable to bytes using `Latin-1' encoding. These strings are used +for the keys and values in the environment dictionary and for response +headers and statuses in the ‘start_response()’ function. They must +follow RFC 2616(1) with respect to encoding. That is, they must either +be `ISO-8859-1' characters or use RFC 2047(2) MIME encoding. + +For developers porting WSGI applications from Python 2, here are the +salient points: + + * If the app already used strings for headers in Python 2, no change + is needed. + + * If instead, the app encoded output headers or decoded input + headers, then the headers will need to be re-encoded to Latin-1. + For example, an output header encoded in utf-8 was using + ‘h.encode('utf-8')’ now needs to convert from bytes to native + strings using ‘h.encode('utf-8').decode('latin-1')’. + + * Values yielded by an application or sent using the ‘write()’ method + must be byte strings. The ‘start_response()’ function and environ + must use native strings. The two cannot be mixed. + +For server implementers writing CGI-to-WSGI pathways or other CGI-style +protocols, the users must to be able access the environment using native +strings even though the underlying platform may have a different +convention. To bridge this gap, the *note wsgiref: 12c. module has a +new function, *note wsgiref.handlers.read_environ(): b36. for +transcoding CGI variables from *note os.environ: b37. into native +strings and returning a new dictionary. + +See also +........ + +PEP 3333(3) - Python Web Server Gateway Interface v1.0.1 + + PEP written by Phillip Eby. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2616.html + + (2) https://tools.ietf.org/html/rfc2047.html + + (3) https://www.python.org/dev/peps/pep-3333 + + +File: python.info, Node: Other Language Changes<7>, Next: New Improved and Deprecated Modules, Prev: PEP 3333 Python Web Server Gateway Interface v1 0 1, Up: What’s New In Python 3 2 + +1.7.8 Other Language Changes +---------------------------- + +Some smaller changes made to the core Python language are: + + * String formatting for *note format(): 4db. and *note str.format(): + 4da. gained new capabilities for the format character `#'. + Previously, for integers in binary, octal, or hexadecimal, it + caused the output to be prefixed with ‘0b’, ‘0o’, or ‘0x’ + respectively. Now it can also handle floats, complex, and Decimal, + causing the output to always have a decimal point even when no + digits follow it. + + >>> format(20, '#o') + '0o24' + >>> format(12.34, '#5.0f') + ' 12.' + + (Suggested by Mark Dickinson and implemented by Eric Smith in + bpo-7094(1).) + + * There is also a new *note str.format_map(): 6b1. method that + extends the capabilities of the existing *note str.format(): 4da. + method by accepting arbitrary *note mapping: b39. objects. This + new method makes it possible to use string formatting with any of + Python’s many dictionary-like objects such as *note defaultdict: + b3a, *note Shelf: 8b0, *note ConfigParser: 4aa, or *note dbm: 32. + It is also useful with custom *note dict: 1b8. subclasses that + normalize keys before look-up or that supply a *note __missing__(): + b3b. method for unknown keys: + + >>> import shelve + >>> d = shelve.open('tmp.shl') + >>> 'The {project_name} status is {status} as of {date}'.format_map(d) + 'The testing project status is green as of February 15, 2011' + + >>> class LowerCasedDict(dict): + ... def __getitem__(self, key): + ... return dict.__getitem__(self, key.lower()) + >>> lcd = LowerCasedDict(part='widgets', quantity=10) + >>> 'There are {QUANTITY} {Part} in stock'.format_map(lcd) + 'There are 10 widgets in stock' + + >>> class PlaceholderDict(dict): + ... def __missing__(self, key): + ... return '<{}>'.format(key) + >>> 'Hello {name}, welcome to {location}'.format_map(PlaceholderDict()) + 'Hello , welcome to ' + + (Suggested by Raymond Hettinger and implemented by Eric Smith in + bpo-6081(2).) + + * The interpreter can now be started with a quiet option, ‘-q’, to + prevent the copyright and version information from being displayed + in the interactive mode. The option can be introspected using the + *note sys.flags: b3c. attribute: + + $ python -q + >>> sys.flags + sys.flags(debug=0, division_warning=0, inspect=0, interactive=0, + optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, + ignore_environment=0, verbose=0, bytes_warning=0, quiet=1) + + (Contributed by Marcin Wojdyr in bpo-1772833(3)). + + * The *note hasattr(): 447. function works by calling *note + getattr(): 448. and detecting whether an exception is raised. This + technique allows it to detect methods created dynamically by *note + __getattr__(): 31a. or *note __getattribute__(): 449. which would + otherwise be absent from the class dictionary. Formerly, `hasattr' + would catch any exception, possibly masking genuine errors. Now, + `hasattr' has been tightened to only catch *note AttributeError: + 39b. and let other exceptions pass through: + + >>> class A: + ... @property + ... def f(self): + ... return 1 // 0 + ... + >>> a = A() + >>> hasattr(a, 'f') + Traceback (most recent call last): + ... + ZeroDivisionError: integer division or modulo by zero + + (Discovered by Yury Selivanov and fixed by Benjamin Peterson; + bpo-9666(4).) + + * The *note str(): 330. of a float or complex number is now the same + as its *note repr(): 7cc. Previously, the *note str(): 330. form + was shorter but that just caused confusion and is no longer needed + now that the shortest possible *note repr(): 7cc. is displayed by + default: + + >>> import math + >>> repr(math.pi) + '3.141592653589793' + >>> str(math.pi) + '3.141592653589793' + + (Proposed and implemented by Mark Dickinson; bpo-9337(5).) + + * *note memoryview: 25c. objects now have a *note release(): b3d. + method and they also now support the context management protocol. + This allows timely release of any resources that were acquired when + requesting a buffer from the original object. + + >>> with memoryview(b'abcdefgh') as v: + ... print(v.tolist()) + [97, 98, 99, 100, 101, 102, 103, 104] + + (Added by Antoine Pitrou; bpo-9757(6).) + + * Previously it was illegal to delete a name from the local namespace + if it occurs as a free variable in a nested block: + + def outer(x): + def inner(): + return x + inner() + del x + + This is now allowed. Remember that the target of an *note except: + b3e. clause is cleared, so this code which used to work with Python + 2.6, raised a *note SyntaxError: 458. with Python 3.1 and now works + again: + + def f(): + def print_error(): + print(e) + try: + something + except Exception as e: + print_error() + # implicit "del e" here + + (See bpo-4617(7).) + + * The internal ‘structsequence’ tool now creates subclasses of tuple. + This means that C structures like those returned by *note + os.stat(): 1f1, *note time.gmtime(): b3f, and *note + sys.version_info: b1a. now work like a *note named tuple: aa3. and + now work with functions and methods that expect a tuple as an + argument. This is a big step forward in making the C structures as + flexible as their pure Python counterparts: + + >>> import sys + >>> isinstance(sys.version_info, tuple) + True + >>> 'Version %d.%d.%d %s(%d)' % sys.version_info + 'Version 3.2.0 final(0)' + + (Suggested by Arfrever Frehtes Taifersar Arahesis and implemented + by Benjamin Peterson in bpo-8413(8).) + + * Warnings are now easier to control using the *note PYTHONWARNINGS: + 418. environment variable as an alternative to using ‘-W’ at the + command line: + + $ export PYTHONWARNINGS='ignore::RuntimeWarning::,once::UnicodeWarning::' + + (Suggested by Barry Warsaw and implemented by Philip Jenvey in + bpo-7301(9).) + + * A new warning category, *note ResourceWarning: 4d0, has been added. + It is emitted when potential issues with resource consumption or + cleanup are detected. It is silenced by default in normal release + builds but can be enabled through the means provided by the *note + warnings: 126. module, or on the command line. + + A *note ResourceWarning: 4d0. is issued at interpreter shutdown if + the *note gc.garbage: b40. list isn’t empty, and if *note + gc.DEBUG_UNCOLLECTABLE: b41. is set, all uncollectable objects are + printed. This is meant to make the programmer aware that their + code contains object finalization issues. + + A *note ResourceWarning: 4d0. is also issued when a *note file + object: b42. is destroyed without having been explicitly closed. + While the deallocator for such object ensures it closes the + underlying operating system resource (usually, a file descriptor), + the delay in deallocating the object could produce various issues, + especially under Windows. Here is an example of enabling the + warning from the command line: + + $ python -q -Wdefault + >>> f = open("foo", "wb") + >>> del f + __main__:1: ResourceWarning: unclosed file <_io.BufferedWriter name='foo'> + + (Added by Antoine Pitrou and Georg Brandl in bpo-10093(10) and + bpo-477863(11).) + + * *note range: 9be. objects now support `index' and `count' methods. + This is part of an effort to make more objects fully implement the + ‘collections.Sequence’ *note abstract base class: b33. As a + result, the language will have a more uniform API. In addition, + *note range: 9be. objects now support slicing and negative indices, + even with values larger than *note sys.maxsize: b43. This makes + `range' more interoperable with lists: + + >>> range(0, 100, 2).count(10) + 1 + >>> range(0, 100, 2).index(10) + 5 + >>> range(0, 100, 2)[5] + 10 + >>> range(0, 100, 2)[0:5] + range(0, 10, 2) + + (Contributed by Daniel Stutzbach in bpo-9213(12), by Alexander + Belopolsky in bpo-2690(13), and by Nick Coghlan in bpo-10889(14).) + + * The *note callable(): b44. builtin function from Py2.x was + resurrected. It provides a concise, readable alternative to using + an *note abstract base class: b33. in an expression like + ‘isinstance(x, collections.Callable)’: + + >>> callable(max) + True + >>> callable(20) + False + + (See bpo-10518(15).) + + * Python’s import mechanism can now load modules installed in + directories with non-ASCII characters in the path name. This + solved an aggravating problem with home directories for users with + non-ASCII characters in their usernames. + + (Required extensive work by Victor Stinner in bpo-9425(16).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7094 + + (2) https://bugs.python.org/issue6081 + + (3) https://bugs.python.org/issue1772833 + + (4) https://bugs.python.org/issue9666 + + (5) https://bugs.python.org/issue9337 + + (6) https://bugs.python.org/issue9757 + + (7) https://bugs.python.org/issue4617 + + (8) https://bugs.python.org/issue8413 + + (9) https://bugs.python.org/issue7301 + + (10) https://bugs.python.org/issue10093 + + (11) https://bugs.python.org/issue477863 + + (12) https://bugs.python.org/issue9213 + + (13) https://bugs.python.org/issue2690 + + (14) https://bugs.python.org/issue10889 + + (15) https://bugs.python.org/issue10518 + + (16) https://bugs.python.org/issue9425 + + +File: python.info, Node: New Improved and Deprecated Modules, Next: Multi-threading, Prev: Other Language Changes<7>, Up: What’s New In Python 3 2 + +1.7.9 New, Improved, and Deprecated Modules +------------------------------------------- + +Python’s standard library has undergone significant maintenance efforts +and quality improvements. + +The biggest news for Python 3.2 is that the *note email: 69. package, +*note mailbox: ae. module, and *note nntplib: c0. modules now work +correctly with the bytes/text model in Python 3. For the first time, +there is correct handling of messages with mixed encodings. + +Throughout the standard library, there has been more careful attention +to encodings and text versus bytes issues. In particular, interactions +with the operating system are now better able to exchange non-ASCII data +using the Windows MBCS encoding, locale-aware encodings, or UTF-8. + +Another significant win is the addition of substantially better support +for `SSL' connections and security certificates. + +In addition, more classes now implement a *note context manager: 568. to +support convenient and reliable resource clean-up using a *note with: +6e9. statement. + +* Menu: + +* email: email<5>. +* elementtree:: +* functools: functools<6>. +* itertools: itertools<4>. +* collections: collections<8>. +* threading: threading<6>. +* datetime and time:: +* math: math<6>. +* abc: abc<3>. +* io: io<5>. +* reprlib:: +* logging: logging<7>. +* csv: csv<3>. +* contextlib: contextlib<6>. +* decimal and fractions:: +* ftp:: +* popen:: +* select: select<3>. +* gzip and zipfile:: +* tarfile: tarfile<5>. +* hashlib: hashlib<3>. +* ast: ast<3>. +* os: os<8>. +* shutil: shutil<5>. +* sqlite3: sqlite3<6>. +* html: html<3>. +* socket: socket<8>. +* ssl: ssl<9>. +* nntp:: +* certificates:: +* imaplib: imaplib<3>. +* http.client: http client<4>. +* unittest: unittest<6>. +* random: random<2>. +* poplib: poplib<3>. +* asyncore: asyncore<2>. +* tempfile: tempfile<2>. +* inspect: inspect<6>. +* pydoc: pydoc<5>. +* dis: dis<3>. +* dbm: dbm<6>. +* ctypes: ctypes<2>. +* site: site<2>. +* sysconfig: sysconfig<2>. +* pdb: pdb<5>. +* configparser: configparser<2>. +* urllib.parse: urllib parse<2>. +* mailbox:: +* turtledemo:: + + +File: python.info, Node: email<5>, Next: elementtree, Up: New Improved and Deprecated Modules + +1.7.9.1 email +............. + +The usability of the *note email: 69. package in Python 3 has been +mostly fixed by the extensive efforts of R. David Murray. The problem +was that emails are typically read and stored in the form of *note +bytes: 331. rather than *note str: 330. text, and they may contain +multiple encodings within a single email. So, the email package had to +be extended to parse and generate email messages in bytes format. + + * New functions *note message_from_bytes(): b47. and *note + message_from_binary_file(): b48, and new classes *note + BytesFeedParser: b49. and *note BytesParser: b4a. allow binary + message data to be parsed into model objects. + + * Given bytes input to the model, *note get_payload(): b4b. will by + default decode a message body that has a + ‘Content-Transfer-Encoding’ of `8bit' using the charset specified + in the MIME headers and return the resulting string. + + * Given bytes input to the model, *note Generator: b4c. will convert + message bodies that have a ‘Content-Transfer-Encoding’ of `8bit' to + instead have a `7bit' ‘Content-Transfer-Encoding’. + + Headers with unencoded non-ASCII bytes are deemed to be RFC + 2047(1)-encoded using the `unknown-8bit' character set. + + * A new class *note BytesGenerator: b4d. produces bytes as output, + preserving any unchanged non-ASCII data that was present in the + input used to build the model, including message bodies with a + ‘Content-Transfer-Encoding’ of `8bit'. + + * The *note smtplib: ed. *note SMTP: a81. class now accepts a byte + string for the `msg' argument to the *note sendmail(): 744. method, + and a new method, *note send_message(): 745. accepts a *note + Message: 541. object and can optionally obtain the `from_addr' and + `to_addrs' addresses directly from the object. + +(Proposed and implemented by R. David Murray, bpo-4661(2) and +bpo-10321(3).) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2047.html + + (2) https://bugs.python.org/issue4661 + + (3) https://bugs.python.org/issue10321 + + +File: python.info, Node: elementtree, Next: functools<6>, Prev: email<5>, Up: New Improved and Deprecated Modules + +1.7.9.2 elementtree +................... + +The *note xml.etree.ElementTree: 137. package and its +‘xml.etree.cElementTree’ counterpart have been updated to version 1.3. + +Several new and useful functions and methods have been added: + + * *note xml.etree.ElementTree.fromstringlist(): b4f. which builds an + XML document from a sequence of fragments + + * *note xml.etree.ElementTree.register_namespace(): b50. for + registering a global namespace prefix + + * *note xml.etree.ElementTree.tostringlist(): 916. for string + representation including all sublists + + * *note xml.etree.ElementTree.Element.extend(): b51. for appending a + sequence of zero or more elements + + * *note xml.etree.ElementTree.Element.iterfind(): b52. searches an + element and subelements + + * *note xml.etree.ElementTree.Element.itertext(): b53. creates a text + iterator over an element and its subelements + + * *note xml.etree.ElementTree.TreeBuilder.end(): b54. closes the + current element + + * *note xml.etree.ElementTree.TreeBuilder.doctype(): 2c1. handles a + doctype declaration + +Two methods have been deprecated: + + * ‘xml.etree.ElementTree.getchildren()’ use ‘list(elem)’ instead. + + * ‘xml.etree.ElementTree.getiterator()’ use ‘Element.iter’ instead. + +For details of the update, see Introducing ElementTree(1) on Fredrik +Lundh’s website. + +(Contributed by Florent Xicluna and Fredrik Lundh, bpo-6472(2).) + + ---------- Footnotes ---------- + + (1) http://effbot.org/zone/elementtree-13-intro.htm + + (2) https://bugs.python.org/issue6472 + + +File: python.info, Node: functools<6>, Next: itertools<4>, Prev: elementtree, Up: New Improved and Deprecated Modules + +1.7.9.3 functools +................. + + * The *note functools: 85. module includes a new decorator for + caching function calls. *note functools.lru_cache(): 1c7. can save + repeated queries to an external resource whenever the results are + expected to be the same. + + For example, adding a caching decorator to a database query + function can save database accesses for popular searches: + + >>> import functools + >>> @functools.lru_cache(maxsize=300) + ... def get_phone_number(name): + ... c = conn.cursor() + ... c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,)) + ... return c.fetchone()[0] + + >>> for name in user_requests: + ... get_phone_number(name) # cached lookup + + To help with choosing an effective cache size, the wrapped function + is instrumented for tracking cache statistics: + + >>> get_phone_number.cache_info() + CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300) + + If the phonelist table gets updated, the outdated contents of the + cache can be cleared with: + + >>> get_phone_number.cache_clear() + + (Contributed by Raymond Hettinger and incorporating design ideas + from Jim Baker, Miki Tebeka, and Nick Coghlan; see recipe + 498245(1), recipe 577479(2), bpo-10586(3), and bpo-10593(4).) + + * The *note functools.wraps(): 86a. decorator now adds a + ‘__wrapped__’ attribute pointing to the original callable function. + This allows wrapped functions to be introspected. It also copies + ‘__annotations__’ if defined. And now it also gracefully skips + over missing attributes such as ‘__doc__’ which might not be + defined for the wrapped callable. + + In the above example, the cache can be removed by recovering the + original function: + + >>> get_phone_number = get_phone_number.__wrapped__ # uncached function + + (By Nick Coghlan and Terrence Cole; bpo-9567(5), bpo-3445(6), and + bpo-8814(7).) + + * To help write classes with rich comparison methods, a new decorator + *note functools.total_ordering(): 84b. will use existing equality + and inequality methods to fill in the remaining methods. + + For example, supplying `__eq__' and `__lt__' will enable *note + total_ordering(): 84b. to fill-in `__le__', `__gt__' and `__ge__': + + @total_ordering + class Student: + def __eq__(self, other): + return ((self.lastname.lower(), self.firstname.lower()) == + (other.lastname.lower(), other.firstname.lower())) + + def __lt__(self, other): + return ((self.lastname.lower(), self.firstname.lower()) < + (other.lastname.lower(), other.firstname.lower())) + + With the `total_ordering' decorator, the remaining comparison + methods are filled in automatically. + + (Contributed by Raymond Hettinger.) + + * To aid in porting programs from Python 2, the *note + functools.cmp_to_key(): b56. function converts an old-style + comparison function to modern *note key function: 6e0.: + + >>> # locale-aware sort order + >>> sorted(iterable, key=cmp_to_key(locale.strcoll)) + + For sorting examples and a brief sorting tutorial, see the Sorting + HowTo(8) tutorial. + + (Contributed by Raymond Hettinger.) + + ---------- Footnotes ---------- + + (1) https://code.activestate.com/recipes/498245 + + (2) https://code.activestate.com/recipes/577479 + + (3) https://bugs.python.org/issue10586 + + (4) https://bugs.python.org/issue10593 + + (5) https://bugs.python.org/issue9567 + + (6) https://bugs.python.org/issue3445 + + (7) https://bugs.python.org/issue8814 + + (8) https://wiki.python.org/moin/HowTo/Sorting/ + + +File: python.info, Node: itertools<4>, Next: collections<8>, Prev: functools<6>, Up: New Improved and Deprecated Modules + +1.7.9.4 itertools +................. + + * The *note itertools: a3. module has a new *note accumulate(): 1dd. + function modeled on APL’s `scan' operator and Numpy’s `accumulate' + function: + + >>> from itertools import accumulate + >>> list(accumulate([8, 2, 50])) + [8, 10, 60] + + >>> prob_dist = [0.1, 0.4, 0.2, 0.3] + >>> list(accumulate(prob_dist)) # cumulative probability distribution + [0.1, 0.5, 0.7, 1.0] + + For an example using *note accumulate(): 1dd, see the *note + examples for the random module: b58. + + (Contributed by Raymond Hettinger and incorporating design + suggestions from Mark Dickinson.) + + +File: python.info, Node: collections<8>, Next: threading<6>, Prev: itertools<4>, Up: New Improved and Deprecated Modules + +1.7.9.5 collections +................... + + * The *note collections.Counter: 9da. class now has two forms of + in-place subtraction, the existing `-=' operator for saturating + subtraction(1) and the new *note subtract(): b5a. method for + regular subtraction. The former is suitable for multisets(2) which + only have positive counts, and the latter is more suitable for use + cases that allow negative counts: + + >>> from collections import Counter + >>> tally = Counter(dogs=5, cats=3) + >>> tally -= Counter(dogs=2, cats=8) # saturating subtraction + >>> tally + Counter({'dogs': 3}) + + >>> tally = Counter(dogs=5, cats=3) + >>> tally.subtract(dogs=2, cats=8) # regular subtraction + >>> tally + Counter({'dogs': 3, 'cats': -5}) + + (Contributed by Raymond Hettinger.) + + * The *note collections.OrderedDict: 1b9. class has a new method + *note move_to_end(): b5b. which takes an existing key and moves it + to either the first or last position in the ordered sequence. + + The default is to move an item to the last position. This is + equivalent of renewing an entry with ‘od[k] = od.pop(k)’. + + A fast move-to-end operation is useful for resequencing entries. + For example, an ordered dictionary can be used to track order of + access by aging entries from the oldest to the most recently + accessed. + + >>> from collections import OrderedDict + >>> d = OrderedDict.fromkeys(['a', 'b', 'X', 'd', 'e']) + >>> list(d) + ['a', 'b', 'X', 'd', 'e'] + >>> d.move_to_end('X') + >>> list(d) + ['a', 'b', 'd', 'e', 'X'] + + (Contributed by Raymond Hettinger.) + + * The *note collections.deque: 531. class grew two new methods *note + count(): b5c. and *note reverse(): b5d. that make them more + substitutable for *note list: 262. objects: + + >>> from collections import deque + >>> d = deque('simsalabim') + >>> d.count('s') + 2 + >>> d.reverse() + >>> d + deque(['m', 'i', 'b', 'a', 'l', 'a', 's', 'm', 'i', 's']) + + (Contributed by Raymond Hettinger.) + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Saturation_arithmetic + + (2) https://en.wikipedia.org/wiki/Multiset + + +File: python.info, Node: threading<6>, Next: datetime and time, Prev: collections<8>, Up: New Improved and Deprecated Modules + +1.7.9.6 threading +................. + +The *note threading: 109. module has a new *note Barrier: b5f. +synchronization class for making multiple threads wait until all of them +have reached a common barrier point. Barriers are useful for making +sure that a task with multiple preconditions does not run until all of +the predecessor tasks are complete. + +Barriers can work with an arbitrary number of threads. This is a +generalization of a Rendezvous(1) which is defined for only two threads. + +Implemented as a two-phase cyclic barrier, *note Barrier: b5f. objects +are suitable for use in loops. The separate `filling' and `draining' +phases assure that all threads get released (drained) before any one of +them can loop back and re-enter the barrier. The barrier fully resets +after each cycle. + +Example of using barriers: + + from threading import Barrier, Thread + + def get_votes(site): + ballots = conduct_election(site) + all_polls_closed.wait() # do not count until all polls are closed + totals = summarize(ballots) + publish(site, totals) + + all_polls_closed = Barrier(len(sites)) + for site in sites: + Thread(target=get_votes, args=(site,)).start() + +In this example, the barrier enforces a rule that votes cannot be +counted at any polling site until all polls are closed. Notice how a +solution with a barrier is similar to one with *note +threading.Thread.join(): b60, but the threads stay alive and continue to +do work (summarizing ballots) after the barrier point is crossed. + +If any of the predecessor tasks can hang or be delayed, a barrier can be +created with an optional `timeout' parameter. Then if the timeout +period elapses before all the predecessor tasks reach the barrier point, +all waiting threads are released and a *note BrokenBarrierError: b61. +exception is raised: + + def get_votes(site): + ballots = conduct_election(site) + try: + all_polls_closed.wait(timeout=midnight - time.now()) + except BrokenBarrierError: + lockbox = seal_ballots(ballots) + queue.put(lockbox) + else: + totals = summarize(ballots) + publish(site, totals) + +In this example, the barrier enforces a more robust rule. If some +election sites do not finish before midnight, the barrier times-out and +the ballots are sealed and deposited in a queue for later handling. + +See Barrier Synchronization Patterns(2) for more examples of how +barriers can be used in parallel computing. Also, there is a simple but +thorough explanation of barriers in The Little Book of Semaphores(3), +`section 3.6'. + +(Contributed by Kristján Valur Jónsson with an API review by Jeffrey +Yasskin in bpo-8777(4).) + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Synchronous_rendezvous + + (2) +http://osl.cs.illinois.edu/media/papers/karmani-2009-barrier_synchronization_pattern.pdf + + (3) https://greenteapress.com/semaphores/LittleBookOfSemaphores.pdf + + (4) https://bugs.python.org/issue8777 + + +File: python.info, Node: datetime and time, Next: math<6>, Prev: threading<6>, Up: New Improved and Deprecated Modules + +1.7.9.7 datetime and time +......................... + + * The *note datetime: 31. module has a new type *note timezone: a01. + that implements the *note tzinfo: 380. interface by returning a + fixed UTC offset and timezone name. This makes it easier to create + timezone-aware datetime objects: + + >>> from datetime import datetime, timezone + + >>> datetime.now(timezone.utc) + datetime.datetime(2010, 12, 8, 21, 4, 2, 923754, tzinfo=datetime.timezone.utc) + + >>> datetime.strptime("01/01/2000 12:00 +0000", "%m/%d/%Y %H:%M %z") + datetime.datetime(2000, 1, 1, 12, 0, tzinfo=datetime.timezone.utc) + + * Also, *note timedelta: 195. objects can now be multiplied by *note + float: 187. and divided by *note float: 187. and *note int: 184. + objects. And *note timedelta: 195. objects can now divide one + another. + + * The *note datetime.date.strftime(): 538. method is no longer + restricted to years after 1900. The new supported year range is + from 1000 to 9999 inclusive. + + * Whenever a two-digit year is used in a time tuple, the + interpretation has been governed by ‘time.accept2dyear’. The + default is ‘True’ which means that for a two-digit year, the + century is guessed according to the POSIX rules governing the ‘%y’ + strptime format. + + Starting with Py3.2, use of the century guessing heuristic will + emit a *note DeprecationWarning: 278. Instead, it is recommended + that ‘time.accept2dyear’ be set to ‘False’ so that large date + ranges can be used without guesswork: + + >>> import time, warnings + >>> warnings.resetwarnings() # remove the default warning filters + + >>> time.accept2dyear = True # guess whether 11 means 11 or 2011 + >>> time.asctime((11, 1, 1, 12, 34, 56, 4, 1, 0)) + Warning (from warnings module): + ... + DeprecationWarning: Century info guessed for a 2-digit year. + 'Fri Jan 1 12:34:56 2011' + + >>> time.accept2dyear = False # use the full range of allowable dates + >>> time.asctime((11, 1, 1, 12, 34, 56, 4, 1, 0)) + 'Fri Jan 1 12:34:56 11' + + Several functions now have significantly expanded date ranges. + When ‘time.accept2dyear’ is false, the *note time.asctime(): b63. + function will accept any year that fits in a C int, while the *note + time.mktime(): b64. and *note time.strftime(): b65. functions will + accept the full range supported by the corresponding operating + system functions. + +(Contributed by Alexander Belopolsky and Victor Stinner in +bpo-1289118(1), bpo-5094(2), bpo-6641(3), bpo-2706(4), bpo-1777412(5), +bpo-8013(6), and bpo-10827(7).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1289118 + + (2) https://bugs.python.org/issue5094 + + (3) https://bugs.python.org/issue6641 + + (4) https://bugs.python.org/issue2706 + + (5) https://bugs.python.org/issue1777412 + + (6) https://bugs.python.org/issue8013 + + (7) https://bugs.python.org/issue10827 + + +File: python.info, Node: math<6>, Next: abc<3>, Prev: datetime and time, Up: New Improved and Deprecated Modules + +1.7.9.8 math +............ + +The *note math: b1. module has been updated with six new functions +inspired by the C99 standard. + +The *note isfinite(): b67. function provides a reliable and fast way to +detect special values. It returns ‘True’ for regular numbers and +‘False’ for `Nan' or `Infinity': + + >>> from math import isfinite + >>> [isfinite(x) for x in (123, 4.56, float('Nan'), float('Inf'))] + [True, True, False, False] + +The *note expm1(): b68. function computes ‘e**x-1’ for small values of +`x' without incurring the loss of precision that usually accompanies the +subtraction of nearly equal quantities: + + >>> from math import expm1 + >>> expm1(0.013671875) # more accurate way to compute e**x-1 for a small x + 0.013765762467652909 + +The *note erf(): 452. function computes a probability integral or +Gaussian error function(1). The complementary error function, *note +erfc(): 453, is ‘1 - erf(x)’: + + >>> from math import erf, erfc, sqrt + >>> erf(1.0/sqrt(2.0)) # portion of normal distribution within 1 standard deviation + 0.682689492137086 + >>> erfc(1.0/sqrt(2.0)) # portion of normal distribution outside 1 standard deviation + 0.31731050786291404 + >>> erf(1.0/sqrt(2.0)) + erfc(1.0/sqrt(2.0)) + 1.0 + +The *note gamma(): b69. function is a continuous extension of the +factorial function. See ‘https://en.wikipedia.org/wiki/Gamma_function’ +for details. Because the function is related to factorials, it grows +large even for small values of `x', so there is also a *note lgamma(): +b6a. function for computing the natural logarithm of the gamma function: + + >>> from math import gamma, lgamma + >>> gamma(7.0) # six factorial + 720.0 + >>> lgamma(801.0) # log(800 factorial) + 4551.950730698041 + +(Contributed by Mark Dickinson.) + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Error_function + + +File: python.info, Node: abc<3>, Next: io<5>, Prev: math<6>, Up: New Improved and Deprecated Modules + +1.7.9.9 abc +........... + +The *note abc: 4. module now supports *note abstractclassmethod(): 9cf. +and *note abstractstaticmethod(): 9d0. + +These tools make it possible to define an *note abstract base class: +b33. that requires a particular *note classmethod(): 1d8. or *note +staticmethod(): 1d9. to be implemented: + + class Temperature(metaclass=abc.ABCMeta): + @abc.abstractclassmethod + def from_fahrenheit(cls, t): + ... + @abc.abstractclassmethod + def from_celsius(cls, t): + ... + +(Patch submitted by Daniel Urban; bpo-5867(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5867 + + +File: python.info, Node: io<5>, Next: reprlib, Prev: abc<3>, Up: New Improved and Deprecated Modules + +1.7.9.10 io +........... + +The *note io.BytesIO: 79b. has a new method, *note getbuffer(): b6d, +which provides functionality similar to *note memoryview(): 25c. It +creates an editable view of the data without making a copy. The +buffer’s random access and support for slice notation are well-suited to +in-place editing: + + >>> REC_LEN, LOC_START, LOC_LEN = 34, 7, 11 + + >>> def change_location(buffer, record_number, location): + ... start = record_number * REC_LEN + LOC_START + ... buffer[start: start+LOC_LEN] = location + + >>> import io + + >>> byte_stream = io.BytesIO( + ... b'G3805 storeroom Main chassis ' + ... b'X7899 shipping Reserve cog ' + ... b'L6988 receiving Primary sprocket' + ... ) + >>> buffer = byte_stream.getbuffer() + >>> change_location(buffer, 1, b'warehouse ') + >>> change_location(buffer, 0, b'showroom ') + >>> print(byte_stream.getvalue()) + b'G3805 showroom Main chassis ' + b'X7899 warehouse Reserve cog ' + b'L6988 receiving Primary sprocket' + +(Contributed by Antoine Pitrou in bpo-5506(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5506 + + +File: python.info, Node: reprlib, Next: logging<7>, Prev: io<5>, Up: New Improved and Deprecated Modules + +1.7.9.11 reprlib +................ + +When writing a *note __repr__(): 33e. method for a custom container, it +is easy to forget to handle the case where a member refers back to the +container itself. Python’s builtin objects such as *note list: 262. and +*note set: b6f. handle self-reference by displaying “…” in the recursive +part of the representation string. + +To help write such *note __repr__(): 33e. methods, the *note reprlib: +df. module has a new decorator, *note recursive_repr(): b70, for +detecting recursive calls to *note __repr__(): 33e. and substituting a +placeholder string instead: + + >>> class MyList(list): + ... @recursive_repr() + ... def __repr__(self): + ... return '<' + '|'.join(map(repr, self)) + '>' + ... + >>> m = MyList('abc') + >>> m.append(m) + >>> m.append('x') + >>> print(m) + <'a'|'b'|'c'|...|'x'> + +(Contributed by Raymond Hettinger in bpo-9826(1) and bpo-9840(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9826 + + (2) https://bugs.python.org/issue9840 + + +File: python.info, Node: logging<7>, Next: csv<3>, Prev: reprlib, Up: New Improved and Deprecated Modules + +1.7.9.12 logging +................ + +In addition to dictionary-based configuration described above, the *note +logging: aa. package has many other improvements. + +The logging documentation has been augmented by a *note basic tutorial: +b72, an *note advanced tutorial: b73, and a *note cookbook: b74. of +logging recipes. These documents are the fastest way to learn about +logging. + +The *note logging.basicConfig(): 1e0. set-up function gained a `style' +argument to support three different types of string formatting. It +defaults to “%” for traditional %-formatting, can be set to “{” for the +new *note str.format(): 4da. style, or can be set to “$” for the +shell-style formatting provided by *note string.Template: 3eb. The +following three configurations are equivalent: + + >>> from logging import basicConfig + >>> basicConfig(style='%', format="%(name)s -> %(levelname)s: %(message)s") + >>> basicConfig(style='{', format="{name} -> {levelname} {message}") + >>> basicConfig(style='$', format="$name -> $levelname: $message") + +If no configuration is set-up before a logging event occurs, there is +now a default configuration using a *note StreamHandler: b75. directed +to *note sys.stderr: 30f. for events of ‘WARNING’ level or higher. +Formerly, an event occurring before a configuration was set-up would +either raise an exception or silently drop the event depending on the +value of ‘logging.raiseExceptions’. The new default handler is stored +in *note logging.lastResort: b76. + +The use of filters has been simplified. Instead of creating a *note +Filter: b77. object, the predicate can be any Python callable that +returns ‘True’ or ‘False’. + +There were a number of other improvements that add flexibility and +simplify configuration. See the module documentation for a full listing +of changes in Python 3.2. + + +File: python.info, Node: csv<3>, Next: contextlib<6>, Prev: logging<7>, Up: New Improved and Deprecated Modules + +1.7.9.13 csv +............ + +The *note csv: 2a. module now supports a new dialect, *note +unix_dialect: b79, which applies quoting for all fields and a +traditional Unix style with ‘'\n'’ as the line terminator. The +registered dialect name is ‘unix’. + +The *note csv.DictWriter: b7a. has a new method, *note writeheader(): +b7b. for writing-out an initial row to document the field names: + + >>> import csv, sys + >>> w = csv.DictWriter(sys.stdout, ['name', 'dept'], dialect='unix') + >>> w.writeheader() + "name","dept" + >>> w.writerows([ + ... {'name': 'tom', 'dept': 'accounting'}, + ... {'name': 'susan', 'dept': 'Salesl'}]) + "tom","accounting" + "susan","sales" + +(New dialect suggested by Jay Talbot in bpo-5975(1), and the new method +suggested by Ed Abraham in bpo-1537721(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5975 + + (2) https://bugs.python.org/issue1537721 + + +File: python.info, Node: contextlib<6>, Next: decimal and fractions, Prev: csv<3>, Up: New Improved and Deprecated Modules + +1.7.9.14 contextlib +................... + +There is a new and slightly mind-blowing tool *note ContextDecorator: +b7d. that is helpful for creating a *note context manager: 568. that +does double duty as a function decorator. + +As a convenience, this new functionality is used by *note +contextmanager(): b7e. so that no extra effort is needed to support both +roles. + +The basic idea is that both context managers and function decorators can +be used for pre-action and post-action wrappers. Context managers wrap +a group of statements using a *note with: 6e9. statement, and function +decorators wrap a group of statements enclosed in a function. So, +occasionally there is a need to write a pre-action or post-action +wrapper that can be used in either role. + +For example, it is sometimes useful to wrap functions or groups of +statements with a logger that can track the time of entry and time of +exit. Rather than writing both a function decorator and a context +manager for the task, the *note contextmanager(): b7e. provides both +capabilities in a single definition: + + from contextlib import contextmanager + import logging + + logging.basicConfig(level=logging.INFO) + + @contextmanager + def track_entry_and_exit(name): + logging.info('Entering: %s', name) + yield + logging.info('Exiting: %s', name) + +Formerly, this would have only been usable as a context manager: + + with track_entry_and_exit('widget loader'): + print('Some time consuming activity goes here') + load_widget() + +Now, it can be used as a decorator as well: + + @track_entry_and_exit('widget loader') + def activity(): + print('Some time consuming activity goes here') + load_widget() + +Trying to fulfill two roles at once places some limitations on the +technique. Context managers normally have the flexibility to return an +argument usable by a *note with: 6e9. statement, but there is no +parallel for function decorators. + +In the above example, there is not a clean way for the +`track_entry_and_exit' context manager to return a logging instance for +use in the body of enclosed statements. + +(Contributed by Michael Foord in bpo-9110(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9110 + + +File: python.info, Node: decimal and fractions, Next: ftp, Prev: contextlib<6>, Up: New Improved and Deprecated Modules + +1.7.9.15 decimal and fractions +.............................. + +Mark Dickinson crafted an elegant and efficient scheme for assuring that +different numeric datatypes will have the same hash value whenever their +actual values are equal (bpo-8188(1)): + + assert hash(Fraction(3, 2)) == hash(1.5) == \ + hash(Decimal("1.5")) == hash(complex(1.5, 0)) + +Some of the hashing details are exposed through a new attribute, *note +sys.hash_info: 921, which describes the bit width of the hash value, the +prime modulus, the hash values for `infinity' and `nan', and the +multiplier used for the imaginary part of a number: + + >>> sys.hash_info + sys.hash_info(width=64, modulus=2305843009213693951, inf=314159, nan=0, imag=1000003) + +An early decision to limit the inter-operability of various numeric +types has been relaxed. It is still unsupported (and ill-advised) to +have implicit mixing in arithmetic expressions such as ‘Decimal('1.1') + +float('1.1')’ because the latter loses information in the process of +constructing the binary float. However, since existing floating point +value can be converted losslessly to either a decimal or rational +representation, it makes sense to add them to the constructor and to +support mixed-type comparisons. + + * The *note decimal.Decimal: 188. constructor now accepts *note + float: 187. objects directly so there in no longer a need to use + the *note from_float(): b80. method (bpo-8257(2)). + + * Mixed type comparisons are now fully supported so that *note + Decimal: 188. objects can be directly compared with *note float: + 187. and *note fractions.Fraction: 185. (bpo-2531(3) and + bpo-8188(4)). + +Similar changes were made to *note fractions.Fraction: 185. so that the +*note from_float(): b81. and *note from_decimal(): b82. methods are no +longer needed (bpo-8294(5)): + + >>> from decimal import Decimal + >>> from fractions import Fraction + >>> Decimal(1.1) + Decimal('1.100000000000000088817841970012523233890533447265625') + >>> Fraction(1.1) + Fraction(2476979795053773, 2251799813685248) + +Another useful change for the *note decimal: 36. module is that the +‘Context.clamp’ attribute is now public. This is useful in creating +contexts that correspond to the decimal interchange formats specified in +IEEE 754 (see bpo-8540(6)). + +(Contributed by Mark Dickinson and Raymond Hettinger.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8188 + + (2) https://bugs.python.org/issue8257 + + (3) https://bugs.python.org/issue2531 + + (4) https://bugs.python.org/issue8188 + + (5) https://bugs.python.org/issue8294 + + (6) https://bugs.python.org/issue8540 + + +File: python.info, Node: ftp, Next: popen, Prev: decimal and fractions, Up: New Improved and Deprecated Modules + +1.7.9.16 ftp +............ + +The *note ftplib.FTP: 2f0. class now supports the context management +protocol to unconditionally consume *note socket.error: 995. exceptions +and to close the FTP connection when done: + + >>> from ftplib import FTP + >>> with FTP("ftp1.at.proftpd.org") as ftp: + ftp.login() + ftp.dir() + + '230 Anonymous login ok, restrictions apply.' + dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 . + dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 .. + dr-xr-xr-x 5 ftp ftp 4096 May 6 10:43 CentOS + dr-xr-xr-x 3 ftp ftp 18 Jul 10 2008 Fedora + +Other file-like objects such as *note mmap.mmap: 1ec. and *note +fileinput.input(): 2ad. also grew auto-closing context managers: + + with fileinput.input(files=('log1.txt', 'log2.txt')) as f: + for line in f: + process(line) + +(Contributed by Tarek Ziadé and Giampaolo Rodolà in bpo-4972(1), and by +Georg Brandl in bpo-8046(2) and bpo-1286(3).) + +The *note FTP_TLS: a03. class now accepts a `context' parameter, which +is a *note ssl.SSLContext: 3e4. object allowing bundling SSL +configuration options, certificates and private keys into a single +(potentially long-lived) structure. + +(Contributed by Giampaolo Rodolà; bpo-8806(4).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4972 + + (2) https://bugs.python.org/issue8046 + + (3) https://bugs.python.org/issue1286 + + (4) https://bugs.python.org/issue8806 + + +File: python.info, Node: popen, Next: select<3>, Prev: ftp, Up: New Improved and Deprecated Modules + +1.7.9.17 popen +.............. + +The *note os.popen(): 2a9. and *note subprocess.Popen(): 2b9. functions +now support *note with: 6e9. statements for auto-closing of the file +descriptors. + +(Contributed by Antoine Pitrou and Brian Curtin in bpo-7461(1) and +bpo-10554(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7461 + + (2) https://bugs.python.org/issue10554 + + +File: python.info, Node: select<3>, Next: gzip and zipfile, Prev: popen, Up: New Improved and Deprecated Modules + +1.7.9.18 select +............... + +The *note select: e5. module now exposes a new, constant attribute, +*note PIPE_BUF: b86, which gives the minimum number of bytes which are +guaranteed not to block when *note select.select(): 673. says a pipe is +ready for writing. + + >>> import select + >>> select.PIPE_BUF + 512 + +(Available on Unix systems. Patch by Sébastien Sablé in bpo-9862(1)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9862 + + +File: python.info, Node: gzip and zipfile, Next: tarfile<5>, Prev: select<3>, Up: New Improved and Deprecated Modules + +1.7.9.19 gzip and zipfile +......................... + +*note gzip.GzipFile: 6dd. now implements the *note io.BufferedIOBase: +588. *note abstract base class: b33. (except for ‘truncate()’). It also +has a *note peek(): b88. method and supports unseekable as well as +zero-padded file objects. + +The *note gzip: 8c. module also gains the *note compress(): 1d1. and +*note decompress(): b89. functions for easier in-memory compression and +decompression. Keep in mind that text needs to be encoded as *note +bytes: 331. before compressing and decompressing: + + >>> import gzip + >>> s = 'Three shall be the number thou shalt count, ' + >>> s += 'and the number of the counting shall be three' + >>> b = s.encode() # convert to utf-8 + >>> len(b) + 89 + >>> c = gzip.compress(b) + >>> len(c) + 77 + >>> gzip.decompress(c).decode()[:42] # decompress and convert to text + 'Three shall be the number thou shalt count' + +(Contributed by Anand B. Pillai in bpo-3488(1); and by Antoine Pitrou, +Nir Aides and Brian Curtin in bpo-9962(2), bpo-1675951(3), bpo-7471(4) +and bpo-2846(5).) + +Also, the ‘zipfile.ZipExtFile’ class was reworked internally to +represent files stored inside an archive. The new implementation is +significantly faster and can be wrapped in an *note io.BufferedReader: +b8a. object for more speedups. It also solves an issue where +interleaved calls to `read' and `readline' gave the wrong results. + +(Patch submitted by Nir Aides in bpo-7610(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue3488 + + (2) https://bugs.python.org/issue9962 + + (3) https://bugs.python.org/issue1675951 + + (4) https://bugs.python.org/issue7471 + + (5) https://bugs.python.org/issue2846 + + (6) https://bugs.python.org/issue7610 + + +File: python.info, Node: tarfile<5>, Next: hashlib<3>, Prev: gzip and zipfile, Up: New Improved and Deprecated Modules + +1.7.9.20 tarfile +................ + +The *note TarFile: b8c. class can now be used as a context manager. In +addition, its *note add(): 47c. method has a new option, `filter', that +controls which files are added to the archive and allows the file +metadata to be edited. + +The new `filter' option replaces the older, less flexible `exclude' +parameter which is now deprecated. If specified, the optional `filter' +parameter needs to be a *note keyword argument: 5c4. The user-supplied +filter function accepts a *note TarInfo: b8d. object and returns an +updated *note TarInfo: b8d. object, or if it wants the file to be +excluded, the function can return ‘None’: + + >>> import tarfile, glob + + >>> def myfilter(tarinfo): + ... if tarinfo.isfile(): # only save real files + ... tarinfo.uname = 'monty' # redact the user name + ... return tarinfo + + >>> with tarfile.open(name='myarchive.tar.gz', mode='w:gz') as tf: + ... for filename in glob.glob('*.txt'): + ... tf.add(filename, filter=myfilter) + ... tf.list() + -rw-r--r-- monty/501 902 2011-01-26 17:59:11 annotations.txt + -rw-r--r-- monty/501 123 2011-01-26 17:59:11 general_questions.txt + -rw-r--r-- monty/501 3514 2011-01-26 17:59:11 prion.txt + -rw-r--r-- monty/501 124 2011-01-26 17:59:11 py_todo.txt + -rw-r--r-- monty/501 1399 2011-01-26 17:59:11 semaphore_notes.txt + +(Proposed by Tarek Ziadé and implemented by Lars Gustäbel in +bpo-6856(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6856 + + +File: python.info, Node: hashlib<3>, Next: ast<3>, Prev: tarfile<5>, Up: New Improved and Deprecated Modules + +1.7.9.21 hashlib +................ + +The *note hashlib: 8d. module has two new constant attributes listing +the hashing algorithms guaranteed to be present in all implementations +and those available on the current implementation: + + >>> import hashlib + + >>> hashlib.algorithms_guaranteed + {'sha1', 'sha224', 'sha384', 'sha256', 'sha512', 'md5'} + + >>> hashlib.algorithms_available + {'md2', 'SHA256', 'SHA512', 'dsaWithSHA', 'mdc2', 'SHA224', 'MD4', 'sha256', + 'sha512', 'ripemd160', 'SHA1', 'MDC2', 'SHA', 'SHA384', 'MD2', + 'ecdsa-with-SHA1','md4', 'md5', 'sha1', 'DSA-SHA', 'sha224', + 'dsaEncryption', 'DSA', 'RIPEMD160', 'sha', 'MD5', 'sha384'} + +(Suggested by Carl Chenet in bpo-7418(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7418 + + +File: python.info, Node: ast<3>, Next: os<8>, Prev: hashlib<3>, Up: New Improved and Deprecated Modules + +1.7.9.22 ast +............ + +The *note ast: 8. module has a wonderful a general-purpose tool for +safely evaluating expression strings using the Python literal syntax. +The *note ast.literal_eval(): 4a5. function serves as a secure +alternative to the builtin *note eval(): b90. function which is easily +abused. Python 3.2 adds *note bytes: 331. and *note set: b6f. literals +to the list of supported types: strings, bytes, numbers, tuples, lists, +dicts, sets, booleans, and ‘None’. + + >>> from ast import literal_eval + + >>> request = "{'req': 3, 'func': 'pow', 'args': (2, 0.5)}" + >>> literal_eval(request) + {'args': (2, 0.5), 'req': 3, 'func': 'pow'} + + >>> request = "os.system('do something harmful')" + >>> literal_eval(request) + Traceback (most recent call last): + ... + ValueError: malformed node or string: <_ast.Call object at 0x101739a10> + +(Implemented by Benjamin Peterson and Georg Brandl.) + + +File: python.info, Node: os<8>, Next: shutil<5>, Prev: ast<3>, Up: New Improved and Deprecated Modules + +1.7.9.23 os +........... + +Different operating systems use various encodings for filenames and +environment variables. The *note os: c4. module provides two new +functions, *note fsencode(): 4ef. and *note fsdecode(): 4ee, for +encoding and decoding filenames: + + >>> import os + >>> filename = 'Sehenswürdigkeiten' + >>> os.fsencode(filename) + b'Sehensw\xc3\xbcrdigkeiten' + +Some operating systems allow direct access to encoded bytes in the +environment. If so, the *note os.supports_bytes_environ: b92. constant +will be true. + +For direct access to encoded environment variables (if available), use +the new *note os.getenvb(): b93. function or use *note os.environb: b94. +which is a bytes version of *note os.environ: b37. + +(Contributed by Victor Stinner.) + + +File: python.info, Node: shutil<5>, Next: sqlite3<6>, Prev: os<8>, Up: New Improved and Deprecated Modules + +1.7.9.24 shutil +............... + +The *note shutil.copytree(): 21a. function has two new options: + + * `ignore_dangling_symlinks': when ‘symlinks=False’ so that the + function copies a file pointed to by a symlink, not the symlink + itself. This option will silence the error raised if the file + doesn’t exist. + + * `copy_function': is a callable that will be used to copy files. + *note shutil.copy2(): 25a. is used by default. + +(Contributed by Tarek Ziadé.) + +In addition, the *note shutil: e9. module now supports *note archiving +operations: b96. for zipfiles, uncompressed tarfiles, gzipped tarfiles, +and bzipped tarfiles. And there are functions for registering +additional archiving file formats (such as xz compressed tarfiles or +custom formats). + +The principal functions are *note make_archive(): 21b. and *note +unpack_archive(): b97. By default, both operate on the current +directory (which can be set by *note os.chdir(): a3f.) and on any +sub-directories. The archive filename needs to be specified with a full +pathname. The archiving step is non-destructive (the original files are +left unchanged). + + >>> import shutil, pprint + + >>> os.chdir('mydata') # change to the source directory + >>> f = shutil.make_archive('/var/backup/mydata', + ... 'zip') # archive the current directory + >>> f # show the name of archive + '/var/backup/mydata.zip' + >>> os.chdir('tmp') # change to an unpacking + >>> shutil.unpack_archive('/var/backup/mydata.zip') # recover the data + + >>> pprint.pprint(shutil.get_archive_formats()) # display known formats + [('bztar', "bzip2'ed tar-file"), + ('gztar', "gzip'ed tar-file"), + ('tar', 'uncompressed tar file'), + ('zip', 'ZIP file')] + + >>> shutil.register_archive_format( # register a new archive format + ... name='xz', + ... function=xz.compress, # callable archiving function + ... extra_args=[('level', 8)], # arguments to the function + ... description='xz compression' + ... ) + +(Contributed by Tarek Ziadé.) + + +File: python.info, Node: sqlite3<6>, Next: html<3>, Prev: shutil<5>, Up: New Improved and Deprecated Modules + +1.7.9.25 sqlite3 +................ + +The *note sqlite3: f2. module was updated to pysqlite version 2.6.0. It +has two new capabilities. + + * The ‘sqlite3.Connection.in_transit’ attribute is true if there is + an active transaction for uncommitted changes. + + * The *note sqlite3.Connection.enable_load_extension(): b99. and + *note sqlite3.Connection.load_extension(): b9a. methods allows you + to load SQLite extensions from “.so” files. One well-known + extension is the fulltext-search extension distributed with SQLite. + +(Contributed by R. David Murray and Shashwat Anand; bpo-8845(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8845 + + +File: python.info, Node: html<3>, Next: socket<8>, Prev: sqlite3<6>, Up: New Improved and Deprecated Modules + +1.7.9.26 html +............. + +A new *note html: 90. module was introduced with only a single function, +*note escape(): 934, which is used for escaping reserved characters from +HTML markup: + + >>> import html + >>> html.escape('x > 2 && x < 7') + 'x > 2 && x < 7' + + +File: python.info, Node: socket<8>, Next: ssl<9>, Prev: html<3>, Up: New Improved and Deprecated Modules + +1.7.9.27 socket +............... + +The *note socket: ef. module has two new improvements. + + * Socket objects now have a *note detach(): b9d. method which puts + the socket into closed state without actually closing the + underlying file descriptor. The latter can then be reused for + other purposes. (Added by Antoine Pitrou; bpo-8524(1).) + + * *note socket.create_connection(): b9e. now supports the context + management protocol to unconditionally consume *note socket.error: + 995. exceptions and to close the socket when done. (Contributed by + Giampaolo Rodolà; bpo-9794(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8524 + + (2) https://bugs.python.org/issue9794 + + +File: python.info, Node: ssl<9>, Next: nntp, Prev: socket<8>, Up: New Improved and Deprecated Modules + +1.7.9.28 ssl +............ + +The *note ssl: f3. module added a number of features to satisfy common +requirements for secure (encrypted, authenticated) internet connections: + + * A new class, *note SSLContext: 3e4, serves as a container for + persistent SSL data, such as protocol settings, certificates, + private keys, and various other options. It includes a *note + wrap_socket(): 3e5. for creating an SSL socket from an SSL context. + + * A new function, *note ssl.match_hostname(): 3dc, supports server + identity verification for higher-level protocols by implementing + the rules of HTTPS (from RFC 2818(1)) which are also suitable for + other protocols. + + * The *note ssl.wrap_socket(): 46f. constructor function now takes a + `ciphers' argument. The `ciphers' string lists the allowed + encryption algorithms using the format described in the OpenSSL + documentation(2). + + * When linked against recent versions of OpenSSL, the *note ssl: f3. + module now supports the Server Name Indication extension to the TLS + protocol, allowing multiple “virtual hosts” using different + certificates on a single IP port. This extension is only supported + in client mode, and is activated by passing the `server_hostname' + argument to *note ssl.SSLContext.wrap_socket(): 3e5. + + * Various options have been added to the *note ssl: f3. module, such + as *note OP_NO_SSLv2: ba0. which disables the insecure and obsolete + SSLv2 protocol. + + * The extension now loads all the OpenSSL ciphers and digest + algorithms. If some SSL certificates cannot be verified, they are + reported as an “unknown algorithm” error. + + * The version of OpenSSL being used is now accessible using the + module attributes *note ssl.OPENSSL_VERSION: ba1. (a string), *note + ssl.OPENSSL_VERSION_INFO: ba2. (a 5-tuple), and *note + ssl.OPENSSL_VERSION_NUMBER: ba3. (an integer). + +(Contributed by Antoine Pitrou in bpo-8850(3), bpo-1589(4), bpo-8322(5), +bpo-5639(6), bpo-4870(7), bpo-8484(8), and bpo-8321(9).) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2818.html + + (2) +https://www.openssl.org/docs/manmaster/man1/ciphers.html#CIPHER-LIST-FORMAT + + (3) https://bugs.python.org/issue8850 + + (4) https://bugs.python.org/issue1589 + + (5) https://bugs.python.org/issue8322 + + (6) https://bugs.python.org/issue5639 + + (7) https://bugs.python.org/issue4870 + + (8) https://bugs.python.org/issue8484 + + (9) https://bugs.python.org/issue8321 + + +File: python.info, Node: nntp, Next: certificates, Prev: ssl<9>, Up: New Improved and Deprecated Modules + +1.7.9.29 nntp +............. + +The *note nntplib: c0. module has a revamped implementation with better +bytes and text semantics as well as more practical APIs. These +improvements break compatibility with the nntplib version in Python 3.1, +which was partly dysfunctional in itself. + +Support for secure connections through both implicit (using *note +nntplib.NNTP_SSL: ba5.) and explicit (using *note +nntplib.NNTP.starttls(): ba6.) TLS has also been added. + +(Contributed by Antoine Pitrou in bpo-9360(1) and Andrew Vant in +bpo-1926(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9360 + + (2) https://bugs.python.org/issue1926 + + +File: python.info, Node: certificates, Next: imaplib<3>, Prev: nntp, Up: New Improved and Deprecated Modules + +1.7.9.30 certificates +..................... + +*note http.client.HTTPSConnection: 393, *note +urllib.request.HTTPSHandler: ba8. and *note urllib.request.urlopen(): +78c. now take optional arguments to allow for server certificate +checking against a set of Certificate Authorities, as recommended in +public uses of HTTPS. + +(Added by Antoine Pitrou, bpo-9003(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9003 + + +File: python.info, Node: imaplib<3>, Next: http client<4>, Prev: certificates, Up: New Improved and Deprecated Modules + +1.7.9.31 imaplib +................ + +Support for explicit TLS on standard IMAP4 connections has been added +through the new *note imaplib.IMAP4.starttls: baa. method. + +(Contributed by Lorenzo M. Catucci and Antoine Pitrou, bpo-4471(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4471 + + +File: python.info, Node: http client<4>, Next: unittest<6>, Prev: imaplib<3>, Up: New Improved and Deprecated Modules + +1.7.9.32 http.client +.................... + +There were a number of small API improvements in the *note http.client: +94. module. The old-style HTTP 0.9 simple responses are no longer +supported and the `strict' parameter is deprecated in all classes. + +The *note HTTPConnection: 392. and *note HTTPSConnection: 393. classes +now have a `source_address' parameter for a (host, port) tuple +indicating where the HTTP connection is made from. + +Support for certificate checking and HTTPS virtual hosts were added to +*note HTTPSConnection: 393. + +The *note request(): 54f. method on connection objects allowed an +optional `body' argument so that a *note file object: b42. could be used +to supply the content of the request. Conveniently, the `body' argument +now also accepts an *note iterable: bac. object so long as it includes +an explicit ‘Content-Length’ header. This extended interface is much +more flexible than before. + +To establish an HTTPS connection through a proxy server, there is a new +*note set_tunnel(): bad. method that sets the host and port for HTTP +Connect tunneling. + +To match the behavior of *note http.server: 97, the HTTP client library +now also encodes headers with ISO-8859-1 (Latin-1) encoding. It was +already doing that for incoming headers, so now the behavior is +consistent for both incoming and outgoing traffic. (See work by Armin +Ronacher in bpo-10980(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10980 + + +File: python.info, Node: unittest<6>, Next: random<2>, Prev: http client<4>, Up: New Improved and Deprecated Modules + +1.7.9.33 unittest +................. + +The unittest module has a number of improvements supporting test +discovery for packages, easier experimentation at the interactive +prompt, new testcase methods, improved diagnostic messages for test +failures, and better method names. + + * The command-line call ‘python -m unittest’ can now accept file + paths instead of module names for running specific tests + (bpo-10620(1)). The new test discovery can find tests within + packages, locating any test importable from the top-level + directory. The top-level directory can be specified with the ‘-t’ + option, a pattern for matching files with ‘-p’, and a directory to + start discovery with ‘-s’: + + $ python -m unittest discover -s my_proj_dir -p _test.py + + (Contributed by Michael Foord.) + + * Experimentation at the interactive prompt is now easier because the + ‘unittest.case.TestCase’ class can now be instantiated without + arguments: + + >>> from unittest import TestCase + >>> TestCase().assertEqual(pow(2, 3), 8) + + (Contributed by Michael Foord.) + + * The *note unittest: 11b. module has two new methods, *note + assertWarns(): abc. and *note assertWarnsRegex(): abd. to verify + that a given warning type is triggered by the code under test: + + with self.assertWarns(DeprecationWarning): + legacy_function('XYZ') + + (Contributed by Antoine Pitrou, bpo-9754(2).) + + Another new method, *note assertCountEqual(): baf. is used to + compare two iterables to determine if their element counts are + equal (whether the same elements are present with the same number + of occurrences regardless of order): + + def test_anagram(self): + self.assertCountEqual('algorithm', 'logarithm') + + (Contributed by Raymond Hettinger.) + + * A principal feature of the unittest module is an effort to produce + meaningful diagnostics when a test fails. When possible, the + failure is recorded along with a diff of the output. This is + especially helpful for analyzing log files of failed test runs. + However, since diffs can sometime be voluminous, there is a new + *note maxDiff: bb0. attribute that sets maximum length of diffs + displayed. + + * In addition, the method names in the module have undergone a number + of clean-ups. + + For example, *note assertRegex(): bb1. is the new name for + ‘assertRegexpMatches()’ which was misnamed because the test uses + *note re.search(): bb2, not *note re.match(): bb3. Other methods + using regular expressions are now named using short form “Regex” in + preference to “Regexp” – this matches the names used in other + unittest implementations, matches Python’s old name for the *note + re: dd. module, and it has unambiguous camel-casing. + + (Contributed by Raymond Hettinger and implemented by Ezio Melotti.) + + * To improve consistency, some long-standing method aliases are being + deprecated in favor of the preferred names: + + Old Name Preferred Name + + ----------------------------------------------------------------------- + + ‘assert_()’ *note assertTrue(): bb4. + + + ‘assertEquals()’ *note assertEqual(): bb5. + + + ‘assertNotEquals()’ *note assertNotEqual(): bb6. + + + ‘assertAlmostEquals()’ *note assertAlmostEqual(): bb7. + + + ‘assertNotAlmostEquals()’ *note assertNotAlmostEqual(): bb8. + + + Likewise, the ‘TestCase.fail*’ methods deprecated in Python 3.1 are + expected to be removed in Python 3.3. Also see the *note + Deprecated aliases: bb9. section in the *note unittest: 11b. + documentation. + + (Contributed by Ezio Melotti; bpo-9424(3).) + + * The ‘assertDictContainsSubset()’ method was deprecated because it + was misimplemented with the arguments in the wrong order. This + created hard-to-debug optical illusions where tests like + ‘TestCase().assertDictContainsSubset({'a':1, 'b':2}, {'a':1})’ + would fail. + + (Contributed by Raymond Hettinger.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10620 + + (2) https://bugs.python.org/issue9754 + + (3) https://bugs.python.org/issue9424 + + +File: python.info, Node: random<2>, Next: poplib<3>, Prev: unittest<6>, Up: New Improved and Deprecated Modules + +1.7.9.34 random +............... + +The integer methods in the *note random: dc. module now do a better job +of producing uniform distributions. Previously, they computed +selections with ‘int(n*random())’ which had a slight bias whenever `n' +was not a power of two. Now, multiple selections are made from a range +up to the next power of two and a selection is kept only when it falls +within the range ‘0 <= x < n’. The functions and methods affected are +*note randrange(): bbb, *note randint(): bbc, *note choice(): bbd, *note +shuffle(): bbe. and *note sample(): bbf. + +(Contributed by Raymond Hettinger; bpo-9025(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9025 + + +File: python.info, Node: poplib<3>, Next: asyncore<2>, Prev: random<2>, Up: New Improved and Deprecated Modules + +1.7.9.35 poplib +............... + +*note POP3_SSL: bc1. class now accepts a `context' parameter, which is a +*note ssl.SSLContext: 3e4. object allowing bundling SSL configuration +options, certificates and private keys into a single (potentially +long-lived) structure. + +(Contributed by Giampaolo Rodolà; bpo-8807(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8807 + + +File: python.info, Node: asyncore<2>, Next: tempfile<2>, Prev: poplib<3>, Up: New Improved and Deprecated Modules + +1.7.9.36 asyncore +................. + +*note asyncore.dispatcher: bc3. now provides a *note handle_accepted(): +bc4. method returning a ‘(sock, addr)’ pair which is called when a +connection has actually been established with a new remote endpoint. +This is supposed to be used as a replacement for old *note +handle_accept(): bc5. and avoids the user to call *note accept(): bc6. +directly. + +(Contributed by Giampaolo Rodolà; bpo-6706(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6706 + + +File: python.info, Node: tempfile<2>, Next: inspect<6>, Prev: asyncore<2>, Up: New Improved and Deprecated Modules + +1.7.9.37 tempfile +................. + +The *note tempfile: 103. module has a new context manager, *note +TemporaryDirectory: bc8. which provides easy deterministic cleanup of +temporary directories: + + with tempfile.TemporaryDirectory() as tmpdirname: + print('created temporary dir:', tmpdirname) + +(Contributed by Neil Schemenauer and Nick Coghlan; bpo-5178(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5178 + + +File: python.info, Node: inspect<6>, Next: pydoc<5>, Prev: tempfile<2>, Up: New Improved and Deprecated Modules + +1.7.9.38 inspect +................ + + * The *note inspect: a0. module has a new function *note + getgeneratorstate(): bca. to easily identify the current state of a + generator-iterator: + + >>> from inspect import getgeneratorstate + >>> def gen(): + ... yield 'demo' + >>> g = gen() + >>> getgeneratorstate(g) + 'GEN_CREATED' + >>> next(g) + 'demo' + >>> getgeneratorstate(g) + 'GEN_SUSPENDED' + >>> next(g, None) + >>> getgeneratorstate(g) + 'GEN_CLOSED' + + (Contributed by Rodolpho Eckhardt and Nick Coghlan, bpo-10220(1).) + + * To support lookups without the possibility of activating a dynamic + attribute, the *note inspect: a0. module has a new function, *note + getattr_static(): bcb. Unlike *note hasattr(): 447, this is a true + read-only search, guaranteed not to change state while it is + searching: + + >>> class A: + ... @property + ... def f(self): + ... print('Running') + ... return 10 + ... + >>> a = A() + >>> getattr(a, 'f') + Running + 10 + >>> inspect.getattr_static(a, 'f') + + + (Contributed by Michael Foord.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10220 + + +File: python.info, Node: pydoc<5>, Next: dis<3>, Prev: inspect<6>, Up: New Improved and Deprecated Modules + +1.7.9.39 pydoc +.............. + +The *note pydoc: d9. module now provides a much-improved Web server +interface, as well as a new command-line option ‘-b’ to automatically +open a browser window to display that server: + + $ pydoc3.2 -b + +(Contributed by Ron Adam; bpo-2001(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2001 + + +File: python.info, Node: dis<3>, Next: dbm<6>, Prev: pydoc<5>, Up: New Improved and Deprecated Modules + +1.7.9.40 dis +............ + +The *note dis: 38. module gained two new functions for inspecting code, +*note code_info(): bce. and *note show_code(): 832. Both provide +detailed code object information for the supplied function, method, +source code string or code object. The former returns a string and the +latter prints it: + + >>> import dis, random + >>> dis.show_code(random.choice) + Name: choice + Filename: /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/random.py + Argument count: 2 + Kw-only arguments: 0 + Number of locals: 3 + Stack size: 11 + Flags: OPTIMIZED, NEWLOCALS, NOFREE + Constants: + 0: 'Choose a random element from a non-empty sequence.' + 1: 'Cannot choose from an empty sequence' + Names: + 0: _randbelow + 1: len + 2: ValueError + 3: IndexError + Variable names: + 0: self + 1: seq + 2: i + +In addition, the *note dis(): 384. function now accepts string arguments +so that the common idiom ‘dis(compile(s, '', 'eval'))’ can be shortened +to ‘dis(s)’: + + >>> dis('3*x+1 if x%2==1 else x//2') + 1 0 LOAD_NAME 0 (x) + 3 LOAD_CONST 0 (2) + 6 BINARY_MODULO + 7 LOAD_CONST 1 (1) + 10 COMPARE_OP 2 (==) + 13 POP_JUMP_IF_FALSE 28 + 16 LOAD_CONST 2 (3) + 19 LOAD_NAME 0 (x) + 22 BINARY_MULTIPLY + 23 LOAD_CONST 1 (1) + 26 BINARY_ADD + 27 RETURN_VALUE + >> 28 LOAD_NAME 0 (x) + 31 LOAD_CONST 0 (2) + 34 BINARY_FLOOR_DIVIDE + 35 RETURN_VALUE + +Taken together, these improvements make it easier to explore how CPython +is implemented and to see for yourself what the language syntax does +under-the-hood. + +(Contributed by Nick Coghlan in bpo-9147(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9147 + + +File: python.info, Node: dbm<6>, Next: ctypes<2>, Prev: dis<3>, Up: New Improved and Deprecated Modules + +1.7.9.41 dbm +............ + +All database modules now support the ‘get()’ and ‘setdefault()’ methods. + +(Suggested by Ray Allen in bpo-9523(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9523 + + +File: python.info, Node: ctypes<2>, Next: site<2>, Prev: dbm<6>, Up: New Improved and Deprecated Modules + +1.7.9.42 ctypes +............... + +A new type, *note ctypes.c_ssize_t: bd1. represents the C ‘ssize_t’ +datatype. + + +File: python.info, Node: site<2>, Next: sysconfig<2>, Prev: ctypes<2>, Up: New Improved and Deprecated Modules + +1.7.9.43 site +............. + +The *note site: eb. module has three new functions useful for reporting +on the details of a given Python installation. + + * *note getsitepackages(): bd3. lists all global site-packages + directories. + + * *note getuserbase(): bd4. reports on the user’s base directory + where data can be stored. + + * *note getusersitepackages(): bd5. reveals the user-specific + site-packages directory path. + + >>> import site + >>> site.getsitepackages() + ['/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages', + '/Library/Frameworks/Python.framework/Versions/3.2/lib/site-python', + '/Library/Python/3.2/site-packages'] + >>> site.getuserbase() + '/Users/raymondhettinger/Library/Python/3.2' + >>> site.getusersitepackages() + '/Users/raymondhettinger/Library/Python/3.2/lib/python/site-packages' + +Conveniently, some of site’s functionality is accessible directly from +the command-line: + + $ python -m site --user-base + /Users/raymondhettinger/.local + $ python -m site --user-site + /Users/raymondhettinger/.local/lib/python3.2/site-packages + +(Contributed by Tarek Ziadé in bpo-6693(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6693 + + +File: python.info, Node: sysconfig<2>, Next: pdb<5>, Prev: site<2>, Up: New Improved and Deprecated Modules + +1.7.9.44 sysconfig +.................. + +The new *note sysconfig: fe. module makes it straightforward to discover +installation paths and configuration variables that vary across +platforms and installations. + +The module offers access simple access functions for platform and +version information: + + * *note get_platform(): bd7. returning values like `linux-i586' or + `macosx-10.6-ppc'. + + * *note get_python_version(): bd8. returns a Python version string + such as “3.2”. + +It also provides access to the paths and variables corresponding to one +of seven named schemes used by *note distutils: 39. Those include +`posix_prefix', `posix_home', `posix_user', `nt', `nt_user', `os2', +`os2_home': + + * *note get_paths(): bd9. makes a dictionary containing installation + paths for the current installation scheme. + + * *note get_config_vars(): 965. returns a dictionary of platform + specific variables. + +There is also a convenient command-line interface: + + C:\Python32>python -m sysconfig + Platform: "win32" + Python version: "3.2" + Current installation scheme: "nt" + + Paths: + data = "C:\Python32" + include = "C:\Python32\Include" + platinclude = "C:\Python32\Include" + platlib = "C:\Python32\Lib\site-packages" + platstdlib = "C:\Python32\Lib" + purelib = "C:\Python32\Lib\site-packages" + scripts = "C:\Python32\Scripts" + stdlib = "C:\Python32\Lib" + + Variables: + BINDIR = "C:\Python32" + BINLIBDEST = "C:\Python32\Lib" + EXE = ".exe" + INCLUDEPY = "C:\Python32\Include" + LIBDEST = "C:\Python32\Lib" + SO = ".pyd" + VERSION = "32" + abiflags = "" + base = "C:\Python32" + exec_prefix = "C:\Python32" + platbase = "C:\Python32" + prefix = "C:\Python32" + projectbase = "C:\Python32" + py_version = "3.2" + py_version_nodot = "32" + py_version_short = "3.2" + srcdir = "C:\Python32" + userbase = "C:\Documents and Settings\Raymond\Application Data\Python" + +(Moved out of Distutils by Tarek Ziadé.) + + +File: python.info, Node: pdb<5>, Next: configparser<2>, Prev: sysconfig<2>, Up: New Improved and Deprecated Modules + +1.7.9.45 pdb +............ + +The *note pdb: c9. debugger module gained a number of usability +improvements: + + * ‘pdb.py’ now has a ‘-c’ option that executes commands as given in a + ‘.pdbrc’ script file. + + * A ‘.pdbrc’ script file can contain ‘continue’ and ‘next’ commands + that continue debugging. + + * The ‘Pdb’ class constructor now accepts a `nosigint' argument. + + * New commands: ‘l(list)’, ‘ll(long list)’ and ‘source’ for listing + source code. + + * New commands: ‘display’ and ‘undisplay’ for showing or hiding the + value of an expression if it has changed. + + * New command: ‘interact’ for starting an interactive interpreter + containing the global and local names found in the current scope. + + * Breakpoints can be cleared by breakpoint number. + +(Contributed by Georg Brandl, Antonio Cuni and Ilya Sandler.) + + +File: python.info, Node: configparser<2>, Next: urllib parse<2>, Prev: pdb<5>, Up: New Improved and Deprecated Modules + +1.7.9.46 configparser +..................... + +The *note configparser: 23. module was modified to improve usability and +predictability of the default parser and its supported INI syntax. The +old ‘ConfigParser’ class was removed in favor of ‘SafeConfigParser’ +which has in turn been renamed to *note ConfigParser: 4aa. Support for +inline comments is now turned off by default and section or option +duplicates are not allowed in a single configuration source. + +Config parsers gained a new API based on the mapping protocol: + + >>> parser = ConfigParser() + >>> parser.read_string(""" + ... [DEFAULT] + ... location = upper left + ... visible = yes + ... editable = no + ... color = blue + ... + ... [main] + ... title = Main Menu + ... color = green + ... + ... [options] + ... title = Options + ... """) + >>> parser['main']['color'] + 'green' + >>> parser['main']['editable'] + 'no' + >>> section = parser['options'] + >>> section['title'] + 'Options' + >>> section['title'] = 'Options (editable: %(editable)s)' + >>> section['title'] + 'Options (editable: no)' + +The new API is implemented on top of the classical API, so custom parser +subclasses should be able to use it without modifications. + +The INI file structure accepted by config parsers can now be customized. +Users can specify alternative option/value delimiters and comment +prefixes, change the name of the `DEFAULT' section or switch the +interpolation syntax. + +There is support for pluggable interpolation including an additional +interpolation handler *note ExtendedInterpolation: bdc.: + + >>> parser = ConfigParser(interpolation=ExtendedInterpolation()) + >>> parser.read_dict({'buildout': {'directory': '/home/ambv/zope9'}, + ... 'custom': {'prefix': '/usr/local'}}) + >>> parser.read_string(""" + ... [buildout] + ... parts = + ... zope9 + ... instance + ... find-links = + ... ${buildout:directory}/downloads/dist + ... + ... [zope9] + ... recipe = plone.recipe.zope9install + ... location = /opt/zope + ... + ... [instance] + ... recipe = plone.recipe.zope9instance + ... zope9-location = ${zope9:location} + ... zope-conf = ${custom:prefix}/etc/zope.conf + ... """) + >>> parser['buildout']['find-links'] + '\n/home/ambv/zope9/downloads/dist' + >>> parser['instance']['zope-conf'] + '/usr/local/etc/zope.conf' + >>> instance = parser['instance'] + >>> instance['zope-conf'] + '/usr/local/etc/zope.conf' + >>> instance['zope9-location'] + '/opt/zope' + +A number of smaller features were also introduced, like support for +specifying encoding in read operations, specifying fallback values for +get-functions, or reading directly from dictionaries and strings. + +(All changes contributed by Łukasz Langa.) + + +File: python.info, Node: urllib parse<2>, Next: mailbox, Prev: configparser<2>, Up: New Improved and Deprecated Modules + +1.7.9.47 urllib.parse +..................... + +A number of usability improvements were made for the *note urllib.parse: +11f. module. + +The *note urlparse(): 5fa. function now supports IPv6(1) addresses as +described in RFC 2732(2): + + >>> import urllib.parse + >>> urllib.parse.urlparse('http://[dead:beef:cafe:5417:affe:8FA3:deaf:feed]/foo/') + ParseResult(scheme='http', + netloc='[dead:beef:cafe:5417:affe:8FA3:deaf:feed]', + path='/foo/', + params='', + query='', + fragment='') + +The *note urldefrag(): bde. function now returns a *note named tuple: +aa3.: + + >>> r = urllib.parse.urldefrag('http://python.org/about/#target') + >>> r + DefragResult(url='http://python.org/about/', fragment='target') + >>> r[0] + 'http://python.org/about/' + >>> r.fragment + 'target' + +And, the *note urlencode(): 78b. function is now much more flexible, +accepting either a string or bytes type for the `query' argument. If it +is a string, then the `safe', `encoding', and `error' parameters are +sent to *note quote_plus(): bdf. for encoding: + + >>> urllib.parse.urlencode([ + ... ('type', 'telenovela'), + ... ('name', '¿Dónde Está Elisa?')], + ... encoding='latin-1') + 'type=telenovela&name=%BFD%F3nde+Est%E1+Elisa%3F' + +As detailed in *note Parsing ASCII Encoded Bytes: be0, all the *note +urllib.parse: 11f. functions now accept ASCII-encoded byte strings as +input, so long as they are not mixed with regular strings. If +ASCII-encoded byte strings are given as parameters, the return types +will also be an ASCII-encoded byte strings: + + >>> urllib.parse.urlparse(b'http://www.python.org:80/about/') + ParseResultBytes(scheme=b'http', netloc=b'www.python.org:80', + path=b'/about/', params=b'', query=b'', fragment=b'') + +(Work by Nick Coghlan, Dan Mahn, and Senthil Kumaran in bpo-2987(3), +bpo-5468(4), and bpo-9873(5).) + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/IPv6 + + (2) https://tools.ietf.org/html/rfc2732.html + + (3) https://bugs.python.org/issue2987 + + (4) https://bugs.python.org/issue5468 + + (5) https://bugs.python.org/issue9873 + + +File: python.info, Node: mailbox, Next: turtledemo, Prev: urllib parse<2>, Up: New Improved and Deprecated Modules + +1.7.9.48 mailbox +................ + +Thanks to a concerted effort by R. David Murray, the *note mailbox: ae. +module has been fixed for Python 3.2. The challenge was that mailbox +had been originally designed with a text interface, but email messages +are best represented with *note bytes: 331. because various parts of a +message may have different encodings. + +The solution harnessed the *note email: 69. package’s binary support for +parsing arbitrary email messages. In addition, the solution required a +number of API changes. + +As expected, the *note add(): be2. method for *note mailbox.Mailbox: +be3. objects now accepts binary input. + +*note StringIO: 82d. and text file input are deprecated. Also, string +input will fail early if non-ASCII characters are used. Previously it +would fail when the email was processed in a later step. + +There is also support for binary output. The *note get_file(): be4. +method now returns a file in the binary mode (where it used to +incorrectly set the file to text-mode). There is also a new *note +get_bytes(): be5. method that returns a *note bytes: 331. representation +of a message corresponding to a given `key'. + +It is still possible to get non-binary output using the old API’s *note +get_string(): be6. method, but that approach is not very useful. +Instead, it is best to extract messages from a *note Message: be7. +object or to load them from binary input. + +(Contributed by R. David Murray, with efforts from Steffen Daode +Nurpmeso and an initial patch by Victor Stinner in bpo-9124(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue9124 + + +File: python.info, Node: turtledemo, Prev: mailbox, Up: New Improved and Deprecated Modules + +1.7.9.49 turtledemo +................... + +The demonstration code for the *note turtle: 116. module was moved from +the `Demo' directory to main library. It includes over a dozen sample +scripts with lively displays. Being on *note sys.path: 488, it can now +be run directly from the command-line: + + $ python -m turtledemo + +(Moved from the Demo directory by Alexander Belopolsky in bpo-10199(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10199 + + +File: python.info, Node: Multi-threading, Next: Optimizations<6>, Prev: New Improved and Deprecated Modules, Up: What’s New In Python 3 2 + +1.7.10 Multi-threading +---------------------- + + * The mechanism for serializing execution of concurrently running + Python threads (generally known as the *note GIL: bea. or Global + Interpreter Lock) has been rewritten. Among the objectives were + more predictable switching intervals and reduced overhead due to + lock contention and the number of ensuing system calls. The notion + of a “check interval” to allow thread switches has been abandoned + and replaced by an absolute duration expressed in seconds. This + parameter is tunable through *note sys.setswitchinterval(): beb. + It currently defaults to 5 milliseconds. + + Additional details about the implementation can be read from a + python-dev mailing-list message(1) (however, “priority requests” as + exposed in this message have not been kept for inclusion). + + (Contributed by Antoine Pitrou.) + + * Regular and recursive locks now accept an optional `timeout' + argument to their *note acquire(): 76c. method. (Contributed by + Antoine Pitrou; bpo-7316(2).) + + * Similarly, *note threading.Semaphore.acquire(): bec. also gained a + `timeout' argument. (Contributed by Torsten Landschoff; + bpo-850728(3).) + + * Regular and recursive lock acquisitions can now be interrupted by + signals on platforms using Pthreads. This means that Python + programs that deadlock while acquiring locks can be successfully + killed by repeatedly sending SIGINT to the process (by pressing + ‘Ctrl+C’ in most shells). (Contributed by Reid Kleckner; + bpo-8844(4).) + + ---------- Footnotes ---------- + + (1) +https://mail.python.org/pipermail/python-dev/2009-October/093321.html + + (2) https://bugs.python.org/issue7316 + + (3) https://bugs.python.org/issue850728 + + (4) https://bugs.python.org/issue8844 + + +File: python.info, Node: Optimizations<6>, Next: Unicode, Prev: Multi-threading, Up: What’s New In Python 3 2 + +1.7.11 Optimizations +-------------------- + +A number of small performance enhancements have been added: + + * Python’s peephole optimizer now recognizes patterns such ‘x in {1, + 2, 3}’ as being a test for membership in a set of constants. The + optimizer recasts the *note set: b6f. as a *note frozenset: bee. + and stores the pre-built constant. + + Now that the speed penalty is gone, it is practical to start + writing membership tests using set-notation. This style is both + semantically clear and operationally fast: + + extension = name.rpartition('.')[2] + if extension in {'xml', 'html', 'xhtml', 'css'}: + handle(name) + + (Patch and additional tests contributed by Dave Malcolm; + bpo-6690(1)). + + * Serializing and unserializing data using the *note pickle: ca. + module is now several times faster. + + (Contributed by Alexandre Vassalotti, Antoine Pitrou and the + Unladen Swallow team in bpo-9410(2) and bpo-3873(3).) + + * The Timsort algorithm(4) used in *note list.sort(): 445. and *note + sorted(): 444. now runs faster and uses less memory when called + with a *note key function: 6e0. Previously, every element of a + list was wrapped with a temporary object that remembered the key + value associated with each element. Now, two arrays of keys and + values are sorted in parallel. This saves the memory consumed by + the sort wrappers, and it saves time lost to delegating + comparisons. + + (Patch by Daniel Stutzbach in bpo-9915(5).) + + * JSON decoding performance is improved and memory consumption is + reduced whenever the same string is repeated for multiple keys. + Also, JSON encoding now uses the C speedups when the ‘sort_keys’ + argument is true. + + (Contributed by Antoine Pitrou in bpo-7451(6) and by Raymond + Hettinger and Antoine Pitrou in bpo-10314(7).) + + * Recursive locks (created with the *note threading.RLock(): bef. + API) now benefit from a C implementation which makes them as fast + as regular locks, and between 10x and 15x faster than their + previous pure Python implementation. + + (Contributed by Antoine Pitrou; bpo-3001(8).) + + * The fast-search algorithm in stringlib is now used by the + ‘split()’, ‘rsplit()’, ‘splitlines()’ and ‘replace()’ methods on + *note bytes: 331, *note bytearray: 332. and *note str: 330. + objects. Likewise, the algorithm is also used by ‘rfind()’, + ‘rindex()’, ‘rsplit()’ and ‘rpartition()’. + + (Patch by Florent Xicluna in bpo-7622(9) and bpo-7462(10).) + + * Integer to string conversions now work two “digits” at a time, + reducing the number of division and modulo operations. + + (bpo-6713(11) by Gawain Bolton, Mark Dickinson, and Victor + Stinner.) + +There were several other minor optimizations. Set differencing now runs +faster when one operand is much larger than the other (patch by Andress +Bennetts in bpo-8685(12)). The ‘array.repeat()’ method has a faster +implementation (bpo-1569291(13) by Alexander Belopolsky). The +‘BaseHTTPRequestHandler’ has more efficient buffering (bpo-3709(14) by +Andrew Schaaf). The *note operator.attrgetter(): 71b. function has been +sped-up (bpo-10160(15) by Christos Georgiou). And ‘ConfigParser’ loads +multi-line arguments a bit faster (bpo-7113(16) by Łukasz Langa). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6690 + + (2) https://bugs.python.org/issue9410 + + (3) https://bugs.python.org/issue3873 + + (4) https://en.wikipedia.org/wiki/Timsort + + (5) https://bugs.python.org/issue9915 + + (6) https://bugs.python.org/issue7451 + + (7) https://bugs.python.org/issue10314 + + (8) https://bugs.python.org/issue3001 + + (9) https://bugs.python.org/issue7622 + + (10) https://bugs.python.org/issue7462 + + (11) https://bugs.python.org/issue6713 + + (12) https://bugs.python.org/issue8685 + + (13) https://bugs.python.org/issue1569291 + + (14) https://bugs.python.org/issue3709 + + (15) https://bugs.python.org/issue10160 + + (16) https://bugs.python.org/issue7113 + + +File: python.info, Node: Unicode, Next: Codecs, Prev: Optimizations<6>, Up: What’s New In Python 3 2 + +1.7.12 Unicode +-------------- + +Python has been updated to Unicode 6.0.0(1). The update to the standard +adds over 2,000 new characters including emoji(2) symbols which are +important for mobile phones. + +In addition, the updated standard has altered the character properties +for two Kannada characters (U+0CF1, U+0CF2) and one New Tai Lue numeric +character (U+19DA), making the former eligible for use in identifiers +while disqualifying the latter. For more information, see Unicode +Character Database Changes(3). + + ---------- Footnotes ---------- + + (1) http://unicode.org/versions/Unicode6.0.0/ + + (2) https://en.wikipedia.org/wiki/Emoji + + (3) http://www.unicode.org/versions/Unicode6.0.0/#Database_Changes + + +File: python.info, Node: Codecs, Next: Documentation, Prev: Unicode, Up: What’s New In Python 3 2 + +1.7.13 Codecs +------------- + +Support was added for `cp720' Arabic DOS encoding (bpo-1616979(1)). + +MBCS encoding no longer ignores the error handler argument. In the +default strict mode, it raises an *note UnicodeDecodeError: 1fd. when it +encounters an undecodable byte sequence and an *note UnicodeEncodeError: +1fc. for an unencodable character. + +The MBCS codec supports ‘'strict'’ and ‘'ignore'’ error handlers for +decoding, and ‘'strict'’ and ‘'replace'’ for encoding. + +To emulate Python3.1 MBCS encoding, select the ‘'ignore'’ handler for +decoding and the ‘'replace'’ handler for encoding. + +On Mac OS X, Python decodes command line arguments with ‘'utf-8'’ rather +than the locale encoding. + +By default, *note tarfile: 101. uses ‘'utf-8'’ encoding on Windows +(instead of ‘'mbcs'’) and the ‘'surrogateescape'’ error handler on all +operating systems. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1616979 + + +File: python.info, Node: Documentation, Next: IDLE, Prev: Codecs, Up: What’s New In Python 3 2 + +1.7.14 Documentation +-------------------- + +The documentation continues to be improved. + + * A table of quick links has been added to the top of lengthy + sections such as *note Built-in Functions: bf3. In the case of + *note itertools: a3, the links are accompanied by tables of + cheatsheet-style summaries to provide an overview and memory jog + without having to read all of the docs. + + * In some cases, the pure Python source code can be a helpful adjunct + to the documentation, so now many modules now feature quick links + to the latest version of the source code. For example, the *note + functools: 85. module documentation has a quick link at the top + labeled: + + `Source code' Lib/functools.py(1). + + (Contributed by Raymond Hettinger; see rationale(2).) + + * The docs now contain more examples and recipes. In particular, + *note re: dd. module has an extensive section, *note Regular + Expression Examples: bf4. Likewise, the *note itertools: a3. + module continues to be updated with new *note Itertools Recipes: + bf5. + + * The *note datetime: 31. module now has an auxiliary implementation + in pure Python. No functionality was changed. This just provides + an easier-to-read alternate implementation. + + (Contributed by Alexander Belopolsky in bpo-9528(3).) + + * The unmaintained ‘Demo’ directory has been removed. Some demos + were integrated into the documentation, some were moved to the + ‘Tools/demo’ directory, and others were removed altogether. + + (Contributed by Georg Brandl in bpo-7962(4).) + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/functools.py + + (2) +https://rhettinger.wordpress.com/2011/01/28/open-your-source-more/ + + (3) https://bugs.python.org/issue9528 + + (4) https://bugs.python.org/issue7962 + + +File: python.info, Node: IDLE, Next: Code Repository, Prev: Documentation, Up: What’s New In Python 3 2 + +1.7.15 IDLE +----------- + + * The format menu now has an option to clean source files by + stripping trailing whitespace. + + (Contributed by Raymond Hettinger; bpo-5150(1).) + + * IDLE on Mac OS X now works with both Carbon AquaTk and Cocoa + AquaTk. + + (Contributed by Kevin Walzer, Ned Deily, and Ronald Oussoren; + bpo-6075(2).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5150 + + (2) https://bugs.python.org/issue6075 + + +File: python.info, Node: Code Repository, Next: Build and C API Changes<5>, Prev: IDLE, Up: What’s New In Python 3 2 + +1.7.16 Code Repository +---------------------- + +In addition to the existing Subversion code repository at +‘http://svn.python.org’ there is now a Mercurial(1) repository at +‘https://hg.python.org/’. + +After the 3.2 release, there are plans to switch to Mercurial as the +primary repository. This distributed version control system should make +it easier for members of the community to create and share external +changesets. See PEP 385(2) for details. + +To learn to use the new version control system, see the Quick Start(3) +or the Guide to Mercurial Workflows(4). + + ---------- Footnotes ---------- + + (1) https://www.mercurial-scm.org/ + + (2) https://www.python.org/dev/peps/pep-0385 + + (3) https://www.mercurial-scm.org/wiki/QuickStart + + (4) https://www.mercurial-scm.org/guide + + +File: python.info, Node: Build and C API Changes<5>, Next: Porting to Python 3 2, Prev: Code Repository, Up: What’s New In Python 3 2 + +1.7.17 Build and C API Changes +------------------------------ + +Changes to Python’s build process and to the C API include: + + * The `idle', `pydoc' and `2to3' scripts are now installed with a + version-specific suffix on ‘make altinstall’ (bpo-10679(1)). + + * The C functions that access the Unicode Database now accept and + return characters from the full Unicode range, even on narrow + unicode builds (Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and + others). A visible difference in Python is that *note + unicodedata.numeric(): bf9. now returns the correct value for large + code points, and *note repr(): 7cc. may consider more characters as + printable. + + (Reported by Bupjoe Lee and fixed by Amaury Forgeot D’Arc; + bpo-5127(2).) + + * Computed gotos are now enabled by default on supported compilers + (which are detected by the configure script). They can still be + disabled selectively by specifying ‘--without-computed-gotos’. + + (Contributed by Antoine Pitrou; bpo-9203(3).) + + * The option ‘--with-wctype-functions’ was removed. The built-in + unicode database is now used for all functions. + + (Contributed by Amaury Forgeot D’Arc; bpo-9210(4).) + + * Hash values are now values of a new type, ‘Py_hash_t’, which is + defined to be the same size as a pointer. Previously they were of + type long, which on some 64-bit operating systems is still only 32 + bits long. As a result of this fix, *note set: b6f. and *note + dict: 1b8. can now hold more than ‘2**32’ entries on builds with + 64-bit pointers (previously, they could grow to that size but their + performance degraded catastrophically). + + (Suggested by Raymond Hettinger and implemented by Benjamin + Peterson; bpo-9778(5).) + + * A new macro ‘Py_VA_COPY’ copies the state of the variable argument + list. It is equivalent to C99 `va_copy' but available on all + Python platforms (bpo-2443(6)). + + * A new C API function *note PySys_SetArgvEx(): bfa. allows an + embedded interpreter to set *note sys.argv: bfb. without also + modifying *note sys.path: 488. (bpo-5753(7)). + + * ‘PyEval_CallObject’ is now only available in macro form. The + function declaration, which was kept for backwards compatibility + reasons, is now removed – the macro was introduced in 1997 + (bpo-8276(8)). + + * There is a new function *note PyLong_AsLongLongAndOverflow(): bfc. + which is analogous to *note PyLong_AsLongAndOverflow(): bfd. They + both serve to convert Python *note int: 184. into a native + fixed-width type while providing detection of cases where the + conversion won’t fit (bpo-7767(9)). + + * The *note PyUnicode_CompareWithASCIIString(): bfe. function now + returns `not equal' if the Python string is `NUL' terminated. + + * There is a new function *note PyErr_NewExceptionWithDoc(): bff. + that is like *note PyErr_NewException(): c00. but allows a + docstring to be specified. This lets C exceptions have the same + self-documenting capabilities as their pure Python counterparts + (bpo-7033(10)). + + * When compiled with the ‘--with-valgrind’ option, the pymalloc + allocator will be automatically disabled when running under + Valgrind. This gives improved memory leak detection when running + under Valgrind, while taking advantage of pymalloc at other times + (bpo-2422(11)). + + * Removed the ‘O?’ format from the `PyArg_Parse' functions. The + format is no longer used and it had never been documented + (bpo-8837(12)). + +There were a number of other small changes to the C-API. See the +Misc/NEWS(13) file for a complete list. + +Also, there were a number of updates to the Mac OS X build, see +Mac/BuildScript/README.txt(14) for details. For users running a +32/64-bit build, there is a known problem with the default Tcl/Tk on Mac +OS X 10.6. Accordingly, we recommend installing an updated alternative +such as ActiveState Tcl/Tk 8.5.9(15). See +‘https://www.python.org/download/mac/tcltk/’ for additional details. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue10679 + + (2) https://bugs.python.org/issue5127 + + (3) https://bugs.python.org/issue9203 + + (4) https://bugs.python.org/issue9210 + + (5) https://bugs.python.org/issue9778 + + (6) https://bugs.python.org/issue2443 + + (7) https://bugs.python.org/issue5753 + + (8) https://bugs.python.org/issue8276 + + (9) https://bugs.python.org/issue7767 + + (10) https://bugs.python.org/issue7033 + + (11) https://bugs.python.org/issue2422 + + (12) https://bugs.python.org/issue8837 + + (13) https://github.com/python/cpython/tree/3.8/Misc/NEWS + + (14) +https://github.com/python/cpython/tree/3.8/Mac/BuildScript/README.txt + + (15) https://www.activestate.com/activetcl/downloads + + +File: python.info, Node: Porting to Python 3 2, Prev: Build and C API Changes<5>, Up: What’s New In Python 3 2 + +1.7.18 Porting to Python 3.2 +---------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code: + + * The *note configparser: 23. module has a number of clean-ups. The + major change is to replace the old ‘ConfigParser’ class with + long-standing preferred alternative ‘SafeConfigParser’. In + addition there are a number of smaller incompatibilities: + + * The interpolation syntax is now validated on *note get(): c02. + and *note set(): c03. operations. In the default + interpolation scheme, only two tokens with percent signs are + valid: ‘%(name)s’ and ‘%%’, the latter being an escaped + percent sign. + + * The *note set(): c03. and *note add_section(): c04. methods + now verify that values are actual strings. Formerly, + unsupported types could be introduced unintentionally. + + * Duplicate sections or options from a single source now raise + either *note DuplicateSectionError: c05. or *note + DuplicateOptionError: c06. Formerly, duplicates would + silently overwrite a previous entry. + + * Inline comments are now disabled by default so now the `;' + character can be safely used in values. + + * Comments now can be indented. Consequently, for `;' or `#' to + appear at the start of a line in multiline values, it has to + be interpolated. This keeps comment prefix characters in + values from being mistaken as comments. + + * ‘""’ is now a valid value and is no longer automatically + converted to an empty string. For empty strings, use ‘"option + ="’ in a line. + + * The *note nntplib: c0. module was reworked extensively, meaning + that its APIs are often incompatible with the 3.1 APIs. + + * *note bytearray: 332. objects can no longer be used as filenames; + instead, they should be converted to *note bytes: 331. + + * The ‘array.tostring()’ and ‘array.fromstring()’ have been renamed + to ‘array.tobytes()’ and ‘array.frombytes()’ for clarity. The old + names have been deprecated. (See bpo-8990(1).) + + * ‘PyArg_Parse*()’ functions: + + * “t#” format has been removed: use “s#” or “s*” instead + + * “w” and “w#” formats has been removed: use “w*” instead + + * The ‘PyCObject’ type, deprecated in 3.1, has been removed. To wrap + opaque C pointers in Python objects, the *note PyCapsule: c07. API + should be used instead; the new type has a well-defined interface + for passing typing safety information and a less complicated + signature for calling a destructor. + + * The ‘sys.setfilesystemencoding()’ function was removed because it + had a flawed design. + + * The *note random.seed(): c08. function and method now salt string + seeds with an sha512 hash function. To access the previous version + of `seed' in order to reproduce Python 3.1 sequences, set the + `version' argument to `1', ‘random.seed(s, version=1)’. + + * The previously deprecated ‘string.maketrans()’ function has been + removed in favor of the static methods *note bytes.maketrans(): + c09. and *note bytearray.maketrans(): c0a. This change solves the + confusion around which types were supported by the *note string: + f6. module. Now, *note str: 330, *note bytes: 331, and *note + bytearray: 332. each have their own `maketrans' and `translate' + methods with intermediate translation tables of the appropriate + type. + + (Contributed by Georg Brandl; bpo-5675(2).) + + * The previously deprecated ‘contextlib.nested()’ function has been + removed in favor of a plain *note with: 6e9. statement which can + accept multiple context managers. The latter technique is faster + (because it is built-in), and it does a better job finalizing + multiple context managers when one of them raises an exception: + + with open('mylog.txt') as infile, open('a.out', 'w') as outfile: + for line in infile: + if '' in line: + outfile.write(line) + + (Contributed by Georg Brandl and Mattias Brändström; appspot issue + 53094(3).) + + * *note struct.pack(): c0b. now only allows bytes for the ‘s’ string + pack code. Formerly, it would accept text arguments and implicitly + encode them to bytes using UTF-8. This was problematic because it + made assumptions about the correct encoding and because a + variable-length encoding can fail when writing to fixed length + segment of a structure. + + Code such as ‘struct.pack('<6sHHBBB', 'GIF87a', x, y)’ should be + rewritten with to use bytes instead of text, + ‘struct.pack('<6sHHBBB', b'GIF87a', x, y)’. + + (Discovered by David Beazley and fixed by Victor Stinner; + bpo-10783(4).) + + * The *note xml.etree.ElementTree: 137. class now raises an *note + xml.etree.ElementTree.ParseError: c0c. when a parse fails. + Previously it raised an *note xml.parsers.expat.ExpatError: c0d. + + * The new, longer *note str(): 330. value on floats may break + doctests which rely on the old output format. + + * In *note subprocess.Popen: 2b9, the default value for `close_fds' + is now ‘True’ under Unix; under Windows, it is ‘True’ if the three + standard streams are set to ‘None’, ‘False’ otherwise. Previously, + `close_fds' was always ‘False’ by default, which produced difficult + to solve bugs or race conditions when open file descriptors would + leak into the child process. + + * Support for legacy HTTP 0.9 has been removed from *note + urllib.request: 120. and *note http.client: 94. Such support is + still present on the server side (in *note http.server: 97.). + + (Contributed by Antoine Pitrou, bpo-10711(5).) + + * SSL sockets in timeout mode now raise *note socket.timeout: c0e. + when a timeout occurs, rather than a generic *note SSLError: c0f. + + (Contributed by Antoine Pitrou, bpo-10272(6).) + + * The misleading functions *note PyEval_AcquireLock(): 2b2. and *note + PyEval_ReleaseLock(): c10. have been officially deprecated. The + thread-state aware APIs (such as *note PyEval_SaveThread(): c11. + and *note PyEval_RestoreThread(): 2b4.) should be used instead. + + * Due to security risks, ‘asyncore.handle_accept()’ has been + deprecated, and a new function, ‘asyncore.handle_accepted()’, was + added to replace it. + + (Contributed by Giampaolo Rodola in bpo-6706(7).) + + * Due to the new *note GIL: bea. implementation, *note + PyEval_InitThreads(): c12. cannot be called before *note + Py_Initialize(): 439. anymore. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8990 + + (2) https://bugs.python.org/issue5675 + + (3) https://codereview.appspot.com/53094 + + (4) https://bugs.python.org/issue10783 + + (5) https://bugs.python.org/issue10711 + + (6) https://bugs.python.org/issue10272 + + (7) https://bugs.python.org/issue6706 + + +File: python.info, Node: What’s New In Python 3 1, Next: What’s New In Python 3 0, Prev: What’s New In Python 3 2, Up: What’s New in Python + +1.8 What’s New In Python 3.1 +============================ + + +Author: Raymond Hettinger + +This article explains the new features in Python 3.1, compared to 3.0. + +* Menu: + +* PEP 372; Ordered Dictionaries: PEP 372 Ordered Dictionaries. +* PEP 378; Format Specifier for Thousands Separator: PEP 378 Format Specifier for Thousands Separator. +* Other Language Changes: Other Language Changes<8>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules<2>. +* Optimizations: Optimizations<7>. +* IDLE: IDLE<2>. +* Build and C API Changes: Build and C API Changes<6>. +* Porting to Python 3.1: Porting to Python 3 1. + + +File: python.info, Node: PEP 372 Ordered Dictionaries, Next: PEP 378 Format Specifier for Thousands Separator, Up: What’s New In Python 3 1 + +1.8.1 PEP 372: Ordered Dictionaries +----------------------------------- + +Regular Python dictionaries iterate over key/value pairs in arbitrary +order. Over the years, a number of authors have written alternative +implementations that remember the order that the keys were originally +inserted. Based on the experiences from those implementations, a new +*note collections.OrderedDict: 1b9. class has been introduced. + +The OrderedDict API is substantially the same as regular dictionaries +but will iterate over keys and values in a guaranteed order depending on +when a key was first inserted. If a new entry overwrites an existing +entry, the original insertion position is left unchanged. Deleting an +entry and reinserting it will move it to the end. + +The standard library now supports use of ordered dictionaries in several +modules. The *note configparser: 23. module uses them by default. This +lets configuration files be read, modified, and then written back in +their original order. The `_asdict()' method for *note +collections.namedtuple(): 1b7. now returns an ordered dictionary with +the values appearing in the same order as the underlying tuple indices. +The *note json: a4. module is being built-out with an +`object_pairs_hook' to allow OrderedDicts to be built by the decoder. +Support was also added for third-party tools like PyYAML(1). + +See also +........ + +PEP 372(2) - Ordered Dictionaries + + PEP written by Armin Ronacher and Raymond Hettinger. + Implementation written by Raymond Hettinger. + + ---------- Footnotes ---------- + + (1) http://pyyaml.org/ + + (2) https://www.python.org/dev/peps/pep-0372 + + +File: python.info, Node: PEP 378 Format Specifier for Thousands Separator, Next: Other Language Changes<8>, Prev: PEP 372 Ordered Dictionaries, Up: What’s New In Python 3 1 + +1.8.2 PEP 378: Format Specifier for Thousands Separator +------------------------------------------------------- + +The built-in *note format(): 4db. function and the *note str.format(): +4da. method use a mini-language that now includes a simple, non-locale +aware way to format a number with a thousands separator. That provides +a way to humanize a program’s output, improving its professional +appearance and readability: + + >>> format(1234567, ',d') + '1,234,567' + >>> format(1234567.89, ',.2f') + '1,234,567.89' + >>> format(12345.6 + 8901234.12j, ',f') + '12,345.600000+8,901,234.120000j' + >>> format(Decimal('1234567.89'), ',f') + '1,234,567.89' + +The supported types are *note int: 184, *note float: 187, *note complex: +189. and *note decimal.Decimal: 188. + +Discussions are underway about how to specify alternative separators +like dots, spaces, apostrophes, or underscores. Locale-aware +applications should use the existing `n' format specifier which already +has some support for thousands separators. + +See also +........ + +PEP 378(1) - Format Specifier for Thousands Separator + + PEP written by Raymond Hettinger and implemented by Eric Smith and + Mark Dickinson. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0378 + + +File: python.info, Node: Other Language Changes<8>, Next: New Improved and Deprecated Modules<2>, Prev: PEP 378 Format Specifier for Thousands Separator, Up: What’s New In Python 3 1 + +1.8.3 Other Language Changes +---------------------------- + +Some smaller changes made to the core Python language are: + + * Directories and zip archives containing a ‘__main__.py’ file can + now be executed directly by passing their name to the interpreter. + The directory/zipfile is automatically inserted as the first entry + in sys.path. (Suggestion and initial patch by Andy Chu; revised + patch by Phillip J. Eby and Nick Coghlan; bpo-1739468(1).) + + * The *note int(): 184. type gained a ‘bit_length’ method that + returns the number of bits necessary to represent its argument in + binary: + + >>> n = 37 + >>> bin(37) + '0b100101' + >>> n.bit_length() + 6 + >>> n = 2**123-1 + >>> n.bit_length() + 123 + >>> (n+1).bit_length() + 124 + + (Contributed by Fredrik Johansson, Victor Stinner, Raymond + Hettinger, and Mark Dickinson; bpo-3439(2).) + + * The fields in *note format(): 4db. strings can now be automatically + numbered: + + >>> 'Sir {} of {}'.format('Gallahad', 'Camelot') + 'Sir Gallahad of Camelot' + + Formerly, the string would have required numbered fields such as: + ‘'Sir {0} of {1}'’. + + (Contributed by Eric Smith; bpo-5237(3).) + + * The ‘string.maketrans()’ function is deprecated and is replaced by + new static methods, *note bytes.maketrans(): c09. and *note + bytearray.maketrans(): c0a. This change solves the confusion + around which types were supported by the *note string: f6. module. + Now, *note str: 330, *note bytes: 331, and *note bytearray: 332. + each have their own `maketrans' and `translate' methods with + intermediate translation tables of the appropriate type. + + (Contributed by Georg Brandl; bpo-5675(4).) + + * The syntax of the *note with: 6e9. statement now allows multiple + context managers in a single statement: + + >>> with open('mylog.txt') as infile, open('a.out', 'w') as outfile: + ... for line in infile: + ... if '' in line: + ... outfile.write(line) + + With the new syntax, the ‘contextlib.nested()’ function is no + longer needed and is now deprecated. + + (Contributed by Georg Brandl and Mattias Brändström; appspot issue + 53094(5).) + + * ‘round(x, n)’ now returns an integer if `x' is an integer. + Previously it returned a float: + + >>> round(1123, -2) + 1100 + + (Contributed by Mark Dickinson; bpo-4707(6).) + + * Python now uses David Gay’s algorithm for finding the shortest + floating point representation that doesn’t change its value. This + should help mitigate some of the confusion surrounding binary + floating point numbers. + + The significance is easily seen with a number like ‘1.1’ which does + not have an exact equivalent in binary floating point. Since there + is no exact equivalent, an expression like ‘float('1.1')’ evaluates + to the nearest representable value which is ‘0x1.199999999999ap+0’ + in hex or ‘1.100000000000000088817841970012523233890533447265625’ + in decimal. That nearest value was and still is used in subsequent + floating point calculations. + + What is new is how the number gets displayed. Formerly, Python + used a simple approach. The value of ‘repr(1.1)’ was computed as + ‘format(1.1, '.17g')’ which evaluated to ‘'1.1000000000000001'’. + The advantage of using 17 digits was that it relied on IEEE-754 + guarantees to assure that ‘eval(repr(1.1))’ would round-trip + exactly to its original value. The disadvantage is that many + people found the output to be confusing (mistaking intrinsic + limitations of binary floating point representation as being a + problem with Python itself). + + The new algorithm for ‘repr(1.1)’ is smarter and returns ‘'1.1'’. + Effectively, it searches all equivalent string representations + (ones that get stored with the same underlying float value) and + returns the shortest representation. + + The new algorithm tends to emit cleaner representations when + possible, but it does not change the underlying values. So, it is + still the case that ‘1.1 + 2.2 != 3.3’ even though the + representations may suggest otherwise. + + The new algorithm depends on certain features in the underlying + floating point implementation. If the required features are not + found, the old algorithm will continue to be used. Also, the text + pickle protocols assure cross-platform portability by using the old + algorithm. + + (Contributed by Eric Smith and Mark Dickinson; bpo-1580(7)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1739468 + + (2) https://bugs.python.org/issue3439 + + (3) https://bugs.python.org/issue5237 + + (4) https://bugs.python.org/issue5675 + + (5) https://codereview.appspot.com/53094 + + (6) https://bugs.python.org/issue4707 + + (7) https://bugs.python.org/issue1580 + + +File: python.info, Node: New Improved and Deprecated Modules<2>, Next: Optimizations<7>, Prev: Other Language Changes<8>, Up: What’s New In Python 3 1 + +1.8.4 New, Improved, and Deprecated Modules +------------------------------------------- + + * Added a *note collections.Counter: 9da. class to support convenient + counting of unique items in a sequence or iterable: + + >>> Counter(['red', 'blue', 'red', 'green', 'blue', 'blue']) + Counter({'blue': 3, 'red': 2, 'green': 1}) + + (Contributed by Raymond Hettinger; bpo-1696199(1).) + + * Added a new module, *note tkinter.ttk: 10f. for access to the Tk + themed widget set. The basic idea of ttk is to separate, to the + extent possible, the code implementing a widget’s behavior from the + code implementing its appearance. + + (Contributed by Guilherme Polo; bpo-2983(2).) + + * The *note gzip.GzipFile: 6dd. and *note bz2.BZ2File: 931. classes + now support the context management protocol: + + >>> # Automatically close file after writing + >>> with gzip.GzipFile(filename, "wb") as f: + ... f.write(b"xxx") + + (Contributed by Antoine Pitrou.) + + * The *note decimal: 36. module now supports methods for creating a + decimal object from a binary *note float: 187. The conversion is + exact but can sometimes be surprising: + + >>> Decimal.from_float(1.1) + Decimal('1.100000000000000088817841970012523233890533447265625') + + The long decimal result shows the actual binary fraction being + stored for `1.1'. The fraction has many digits because `1.1' + cannot be exactly represented in binary. + + (Contributed by Raymond Hettinger and Mark Dickinson.) + + * The *note itertools: a3. module grew two new functions. The *note + itertools.combinations_with_replacement(): c19. function is one of + four for generating combinatorics including permutations and + Cartesian products. The *note itertools.compress(): c1a. function + mimics its namesake from APL. Also, the existing *note + itertools.count(): c1b. function now has an optional `step' + argument and can accept any type of counting sequence including + *note fractions.Fraction: 185. and *note decimal.Decimal: 188.: + + >>> [p+q for p,q in combinations_with_replacement('LOVE', 2)] + ['LL', 'LO', 'LV', 'LE', 'OO', 'OV', 'OE', 'VV', 'VE', 'EE'] + + >>> list(compress(data=range(10), selectors=[0,0,1,1,0,1,0,1,0,0])) + [2, 3, 5, 7] + + >>> c = count(start=Fraction(1,2), step=Fraction(1,6)) + >>> [next(c), next(c), next(c), next(c)] + [Fraction(1, 2), Fraction(2, 3), Fraction(5, 6), Fraction(1, 1)] + + (Contributed by Raymond Hettinger.) + + * *note collections.namedtuple(): 1b7. now supports a keyword + argument `rename' which lets invalid fieldnames be automatically + converted to positional names in the form _0, _1, etc. This is + useful when the field names are being created by an external source + such as a CSV header, SQL field list, or user input: + + >>> query = input() + SELECT region, dept, count(*) FROM main GROUPBY region, dept + + >>> cursor.execute(query) + >>> query_fields = [desc[0] for desc in cursor.description] + >>> UserQuery = namedtuple('UserQuery', query_fields, rename=True) + >>> pprint.pprint([UserQuery(*row) for row in cursor]) + [UserQuery(region='South', dept='Shipping', _2=185), + UserQuery(region='North', dept='Accounting', _2=37), + UserQuery(region='West', dept='Sales', _2=419)] + + (Contributed by Raymond Hettinger; bpo-1818(3).) + + * The *note re.sub(): 47b, *note re.subn(): 734. and *note + re.split(): 3ca. functions now accept a flags parameter. + + (Contributed by Gregory Smith.) + + * The *note logging: aa. module now implements a simple *note + logging.NullHandler: c1c. class for applications that are not using + logging but are calling library code that does. Setting-up a null + handler will suppress spurious warnings such as “No handlers could + be found for logger foo”: + + >>> h = logging.NullHandler() + >>> logging.getLogger("foo").addHandler(h) + + (Contributed by Vinay Sajip; bpo-4384(4)). + + * The *note runpy: e2. module which supports the ‘-m’ command line + switch now supports the execution of packages by looking for and + executing a ‘__main__’ submodule when a package name is supplied. + + (Contributed by Andi Vajda; bpo-4195(5).) + + * The *note pdb: c9. module can now access and display source code + loaded via *note zipimport: 143. (or any other conformant PEP + 302(6) loader). + + (Contributed by Alexander Belopolsky; bpo-4201(7).) + + * *note functools.partial: 7c8. objects can now be pickled. + + (Suggested by Antoine Pitrou and Jesse Noller. Implemented by Jack + Diederich; bpo-5228(8).) + + * Add *note pydoc: d9. help topics for symbols so that ‘help('@')’ + works as expected in the interactive environment. + + (Contributed by David Laban; bpo-4739(9).) + + * The *note unittest: 11b. module now supports skipping individual + tests or classes of tests. And it supports marking a test as an + expected failure, a test that is known to be broken, but shouldn’t + be counted as a failure on a TestResult: + + class TestGizmo(unittest.TestCase): + + @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") + def test_gizmo_on_windows(self): + ... + + @unittest.expectedFailure + def test_gimzo_without_required_library(self): + ... + + Also, tests for exceptions have been builtout to work with context + managers using the *note with: 6e9. statement: + + def test_division_by_zero(self): + with self.assertRaises(ZeroDivisionError): + x / 0 + + In addition, several new assertion methods were added including + ‘assertSetEqual()’, ‘assertDictEqual()’, + ‘assertDictContainsSubset()’, ‘assertListEqual()’, + ‘assertTupleEqual()’, ‘assertSequenceEqual()’, + ‘assertRaisesRegexp()’, ‘assertIsNone()’, and ‘assertIsNotNone()’. + + (Contributed by Benjamin Peterson and Antoine Pitrou.) + + * The *note io: a1. module has three new constants for the ‘seek()’ + method ‘SEEK_SET’, ‘SEEK_CUR’, and ‘SEEK_END’. + + * The *note sys.version_info: b1a. tuple is now a named tuple: + + >>> sys.version_info + sys.version_info(major=3, minor=1, micro=0, releaselevel='alpha', serial=2) + + (Contributed by Ross Light; bpo-4285(10).) + + * The *note nntplib: c0. and *note imaplib: 98. modules now support + IPv6. + + (Contributed by Derek Morr; bpo-1655(11) and bpo-1664(12).) + + * The *note pickle: ca. module has been adapted for better + interoperability with Python 2.x when used with protocol 2 or + lower. The reorganization of the standard library changed the + formal reference for many objects. For example, ‘__builtin__.set’ + in Python 2 is called ‘builtins.set’ in Python 3. This change + confounded efforts to share data between different versions of + Python. But now when protocol 2 or lower is selected, the pickler + will automatically use the old Python 2 names for both loading and + dumping. This remapping is turned-on by default but can be + disabled with the `fix_imports' option: + + >>> s = {1, 2, 3} + >>> pickle.dumps(s, protocol=0) + b'c__builtin__\nset\np0\n((lp1\nL1L\naL2L\naL3L\natp2\nRp3\n.' + >>> pickle.dumps(s, protocol=0, fix_imports=False) + b'cbuiltins\nset\np0\n((lp1\nL1L\naL2L\naL3L\natp2\nRp3\n.' + + An unfortunate but unavoidable side-effect of this change is that + protocol 2 pickles produced by Python 3.1 won’t be readable with + Python 3.0. The latest pickle protocol, protocol 3, should be used + when migrating data between Python 3.x implementations, as it + doesn’t attempt to remain compatible with Python 2.x. + + (Contributed by Alexandre Vassalotti and Antoine Pitrou, + bpo-6137(13).) + + * A new module, *note importlib: 9b. was added. It provides a + complete, portable, pure Python reference implementation of the + *note import: c1d. statement and its counterpart, the *note + __import__(): 610. function. It represents a substantial step + forward in documenting and defining the actions that take place + during imports. + + (Contributed by Brett Cannon.) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1696199 + + (2) https://bugs.python.org/issue2983 + + (3) https://bugs.python.org/issue1818 + + (4) https://bugs.python.org/issue4384 + + (5) https://bugs.python.org/issue4195 + + (6) https://www.python.org/dev/peps/pep-0302 + + (7) https://bugs.python.org/issue4201 + + (8) https://bugs.python.org/issue5228 + + (9) https://bugs.python.org/issue4739 + + (10) https://bugs.python.org/issue4285 + + (11) https://bugs.python.org/issue1655 + + (12) https://bugs.python.org/issue1664 + + (13) https://bugs.python.org/issue6137 + + +File: python.info, Node: Optimizations<7>, Next: IDLE<2>, Prev: New Improved and Deprecated Modules<2>, Up: What’s New In Python 3 1 + +1.8.5 Optimizations +------------------- + +Major performance enhancements have been added: + + * The new I/O library (as defined in PEP 3116(1)) was mostly written + in Python and quickly proved to be a problematic bottleneck in + Python 3.0. In Python 3.1, the I/O library has been entirely + rewritten in C and is 2 to 20 times faster depending on the task at + hand. The pure Python version is still available for + experimentation purposes through the ‘_pyio’ module. + + (Contributed by Amaury Forgeot d’Arc and Antoine Pitrou.) + + * Added a heuristic so that tuples and dicts containing only + untrackable objects are not tracked by the garbage collector. This + can reduce the size of collections and therefore the garbage + collection overhead on long-running programs, depending on their + particular use of datatypes. + + (Contributed by Antoine Pitrou, bpo-4688(2).) + + * Enabling a configure option named ‘--with-computed-gotos’ on + compilers that support it (notably: gcc, SunPro, icc), the bytecode + evaluation loop is compiled with a new dispatch mechanism which + gives speedups of up to 20%, depending on the system, the compiler, + and the benchmark. + + (Contributed by Antoine Pitrou along with a number of other + participants, bpo-4753(3)). + + * The decoding of UTF-8, UTF-16 and LATIN-1 is now two to four times + faster. + + (Contributed by Antoine Pitrou and Amaury Forgeot d’Arc, + bpo-4868(4).) + + * The *note json: a4. module now has a C extension to substantially + improve its performance. In addition, the API was modified so that + json works only with *note str: 330, not with *note bytes: 331. + That change makes the module closely match the JSON + specification(5) which is defined in terms of Unicode. + + (Contributed by Bob Ippolito and converted to Py3.1 by Antoine + Pitrou and Benjamin Peterson; bpo-4136(6).) + + * Unpickling now interns the attribute names of pickled objects. + This saves memory and allows pickles to be smaller. + + (Contributed by Jake McGuire and Antoine Pitrou; bpo-5084(7).) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3116 + + (2) https://bugs.python.org/issue4688 + + (3) https://bugs.python.org/issue4753 + + (4) https://bugs.python.org/issue4868 + + (5) http://json.org/ + + (6) https://bugs.python.org/issue4136 + + (7) https://bugs.python.org/issue5084 + + +File: python.info, Node: IDLE<2>, Next: Build and C API Changes<6>, Prev: Optimizations<7>, Up: What’s New In Python 3 1 + +1.8.6 IDLE +---------- + + * IDLE’s format menu now provides an option to strip trailing + whitespace from a source file. + + (Contributed by Roger D. Serwy; bpo-5150(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5150 + + +File: python.info, Node: Build and C API Changes<6>, Next: Porting to Python 3 1, Prev: IDLE<2>, Up: What’s New In Python 3 1 + +1.8.7 Build and C API Changes +----------------------------- + +Changes to Python’s build process and to the C API include: + + * Integers are now stored internally either in base 2**15 or in base + 2**30, the base being determined at build time. Previously, they + were always stored in base 2**15. Using base 2**30 gives + significant performance improvements on 64-bit machines, but + benchmark results on 32-bit machines have been mixed. Therefore, + the default is to use base 2**30 on 64-bit machines and base 2**15 + on 32-bit machines; on Unix, there’s a new configure option + ‘--enable-big-digits’ that can be used to override this default. + + Apart from the performance improvements this change should be + invisible to end users, with one exception: for testing and + debugging purposes there’s a new *note sys.int_info: c21. that + provides information about the internal format, giving the number + of bits per digit and the size in bytes of the C type used to store + each digit: + + >>> import sys + >>> sys.int_info + sys.int_info(bits_per_digit=30, sizeof_digit=4) + + (Contributed by Mark Dickinson; bpo-4258(1).) + + * The *note PyLong_AsUnsignedLongLong(): c22. function now handles a + negative `pylong' by raising *note OverflowError: 960. instead of + *note TypeError: 192. + + (Contributed by Mark Dickinson and Lisandro Dalcrin; bpo-5175(2).) + + * Deprecated ‘PyNumber_Int()’. Use *note PyNumber_Long(): 26c. + instead. + + (Contributed by Mark Dickinson; bpo-4910(3).) + + * Added a new *note PyOS_string_to_double(): c23. function to replace + the deprecated functions ‘PyOS_ascii_strtod()’ and + ‘PyOS_ascii_atof()’. + + (Contributed by Mark Dickinson; bpo-5914(4).) + + * Added *note PyCapsule: c07. as a replacement for the ‘PyCObject’ + API. The principal difference is that the new type has a well + defined interface for passing typing safety information and a less + complicated signature for calling a destructor. The old type had a + problematic API and is now deprecated. + + (Contributed by Larry Hastings; bpo-5630(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4258 + + (2) https://bugs.python.org/issue5175 + + (3) https://bugs.python.org/issue4910 + + (4) https://bugs.python.org/issue5914 + + (5) https://bugs.python.org/issue5630 + + +File: python.info, Node: Porting to Python 3 1, Prev: Build and C API Changes<6>, Up: What’s New In Python 3 1 + +1.8.8 Porting to Python 3.1 +--------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code: + + * The new floating point string representations can break existing + doctests. For example: + + def e(): + '''Compute the base of natural logarithms. + + >>> e() + 2.7182818284590451 + + ''' + return sum(1/math.factorial(x) for x in reversed(range(30))) + + doctest.testmod() + + ********************************************************************** + Failed example: + e() + Expected: + 2.7182818284590451 + Got: + 2.718281828459045 + ********************************************************************** + + * The automatic name remapping in the pickle module for protocol 2 or + lower can make Python 3.1 pickles unreadable in Python 3.0. One + solution is to use protocol 3. Another solution is to set the + `fix_imports' option to ‘False’. See the discussion above for more + details. + + +File: python.info, Node: What’s New In Python 3 0, Next: What’s New in Python 2 7, Prev: What’s New In Python 3 1, Up: What’s New in Python + +1.9 What’s New In Python 3.0 +============================ + + +Author: Guido van Rossum + +This article explains the new features in Python 3.0, compared to 2.6. +Python 3.0, also known as “Python 3000” or “Py3K”, is the first ever +`intentionally backwards incompatible' Python release. There are more +changes than in a typical release, and more that are important for all +Python users. Nevertheless, after digesting the changes, you’ll find +that Python really hasn’t changed all that much – by and large, we’re +mostly fixing well-known annoyances and warts, and removing a lot of old +cruft. + +This article doesn’t attempt to provide a complete specification of all +new features, but instead tries to give a convenient overview. For full +details, you should refer to the documentation for Python 3.0, and/or +the many PEPs referenced in the text. If you want to understand the +complete implementation and design rationale for a particular feature, +PEPs usually have more details than the regular documentation; but note +that PEPs usually are not kept up-to-date once a feature has been fully +implemented. + +Due to time constraints this document is not as complete as it should +have been. As always for a new release, the ‘Misc/NEWS’ file in the +source distribution contains a wealth of detailed information about +every small thing that was changed. + +* Menu: + +* Common Stumbling Blocks:: +* Overview Of Syntax Changes:: +* Changes Already Present In Python 2.6: Changes Already Present In Python 2 6. +* Library Changes:: +* PEP 3101; A New Approach To String Formatting: PEP 3101 A New Approach To String Formatting. +* Changes To Exceptions:: +* Miscellaneous Other Changes:: +* Build and C API Changes: Build and C API Changes<7>. +* Performance:: +* Porting To Python 3.0: Porting To Python 3 0. + + +File: python.info, Node: Common Stumbling Blocks, Next: Overview Of Syntax Changes, Up: What’s New In Python 3 0 + +1.9.1 Common Stumbling Blocks +----------------------------- + +This section lists those few changes that are most likely to trip you up +if you’re used to Python 2.5. + +* Menu: + +* Print Is A Function:: +* Views And Iterators Instead Of Lists:: +* Ordering Comparisons:: +* Integers:: +* Text Vs. Data Instead Of Unicode Vs. 8-bit: Text Vs Data Instead Of Unicode Vs 8-bit. + + +File: python.info, Node: Print Is A Function, Next: Views And Iterators Instead Of Lists, Up: Common Stumbling Blocks + +1.9.1.1 Print Is A Function +........................... + +The ‘print’ statement has been replaced with a *note print(): 886. +function, with keyword arguments to replace most of the special syntax +of the old ‘print’ statement ( PEP 3105(1)). Examples: + + Old: print "The answer is", 2*2 + New: print("The answer is", 2*2) + + Old: print x, # Trailing comma suppresses newline + New: print(x, end=" ") # Appends a space instead of a newline + + Old: print # Prints a newline + New: print() # You must call the function! + + Old: print >>sys.stderr, "fatal error" + New: print("fatal error", file=sys.stderr) + + Old: print (x, y) # prints repr((x, y)) + New: print((x, y)) # Not the same as print(x, y)! + +You can also customize the separator between items, e.g.: + + print("There are <", 2**32, "> possibilities!", sep="") + +which produces: + + There are <4294967296> possibilities! + +Note: + + * The *note print(): 886. function doesn’t support the “softspace” + feature of the old ‘print’ statement. For example, in Python 2.x, + ‘print "A\n", "B"’ would write ‘"A\nB\n"’; but in Python 3.0, + ‘print("A\n", "B")’ writes ‘"A\n B\n"’. + + * Initially, you’ll be finding yourself typing the old ‘print x’ a + lot in interactive mode. Time to retrain your fingers to type + ‘print(x)’ instead! + + * When using the ‘2to3’ source-to-source conversion tool, all ‘print’ + statements are automatically converted to *note print(): 886. + function calls, so this is mostly a non-issue for larger projects. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3105 + + +File: python.info, Node: Views And Iterators Instead Of Lists, Next: Ordering Comparisons, Prev: Print Is A Function, Up: Common Stumbling Blocks + +1.9.1.2 Views And Iterators Instead Of Lists +............................................ + +Some well-known APIs no longer return lists: + + * *note dict: 1b8. methods *note dict.keys(): c2a, *note + dict.items(): c2b. and *note dict.values(): c2c. return “views” + instead of lists. For example, this no longer works: ‘k = + d.keys(); k.sort()’. Use ‘k = sorted(d)’ instead (this works in + Python 2.5 too and is just as efficient). + + * Also, the ‘dict.iterkeys()’, ‘dict.iteritems()’ and + ‘dict.itervalues()’ methods are no longer supported. + + * *note map(): c2d. and *note filter(): c2e. return iterators. If + you really need a list and the input sequences are all of equal + length, a quick fix is to wrap *note map(): c2d. in *note list(): + 262, e.g. ‘list(map(...))’, but a better fix is often to use a + list comprehension (especially when the original code uses *note + lambda: c2f.), or rewriting the code so it doesn’t need a list at + all. Particularly tricky is *note map(): c2d. invoked for the side + effects of the function; the correct transformation is to use a + regular *note for: c30. loop (since creating a list would just be + wasteful). + + If the input sequences are not of equal length, *note map(): c2d. + will stop at the termination of the shortest of the sequences. For + full compatibility with *note map(): c2d. from Python 2.x, also + wrap the sequences in *note itertools.zip_longest(): c31, e.g. + ‘map(func, *sequences)’ becomes ‘list(map(func, + itertools.zip_longest(*sequences)))’. + + * *note range(): 9be. now behaves like ‘xrange()’ used to behave, + except it works with values of arbitrary size. The latter no + longer exists. + + * *note zip(): c32. now returns an iterator. + + +File: python.info, Node: Ordering Comparisons, Next: Integers, Prev: Views And Iterators Instead Of Lists, Up: Common Stumbling Blocks + +1.9.1.3 Ordering Comparisons +............................ + +Python 3.0 has simplified the rules for ordering comparisons: + + * The ordering comparison operators (‘<’, ‘<=’, ‘>=’, ‘>’) raise a + TypeError exception when the operands don’t have a meaningful + natural ordering. Thus, expressions like ‘1 < ''’, ‘0 > None’ or + ‘len <= len’ are no longer valid, and e.g. ‘None < None’ raises + *note TypeError: 192. instead of returning ‘False’. A corollary is + that sorting a heterogeneous list no longer makes sense – all the + elements must be comparable to each other. Note that this does not + apply to the ‘==’ and ‘!=’ operators: objects of different + incomparable types always compare unequal to each other. + + * ‘builtin.sorted()’ and *note list.sort(): 445. no longer accept the + `cmp' argument providing a comparison function. Use the `key' + argument instead. N.B. the `key' and `reverse' arguments are now + “keyword-only”. + + * The ‘cmp()’ function should be treated as gone, and the ‘__cmp__()’ + special method is no longer supported. Use *note __lt__(): c34. + for sorting, *note __eq__(): 33f. with *note __hash__(): 340, and + other rich comparisons as needed. (If you really need the ‘cmp()’ + functionality, you could use the expression ‘(a > b) - (a < b)’ as + the equivalent for ‘cmp(a, b)’.) + + +File: python.info, Node: Integers, Next: Text Vs Data Instead Of Unicode Vs 8-bit, Prev: Ordering Comparisons, Up: Common Stumbling Blocks + +1.9.1.4 Integers +................ + + * PEP 237(1): Essentially, ‘long’ renamed to *note int: 184. That + is, there is only one built-in integral type, named *note int: + 184.; but it behaves mostly like the old ‘long’ type. + + * PEP 238(2): An expression like ‘1/2’ returns a float. Use ‘1//2’ + to get the truncating behavior. (The latter syntax has existed for + years, at least since Python 2.2.) + + * The ‘sys.maxint’ constant was removed, since there is no longer a + limit to the value of integers. However, *note sys.maxsize: b43. + can be used as an integer larger than any practical list or string + index. It conforms to the implementation’s “natural” integer size + and is typically the same as ‘sys.maxint’ in previous releases on + the same platform (assuming the same build options). + + * The *note repr(): 7cc. of a long integer doesn’t include the + trailing ‘L’ anymore, so code that unconditionally strips that + character will chop off the last digit instead. (Use *note str(): + 330. instead.) + + * Octal literals are no longer of the form ‘0720’; use ‘0o720’ + instead. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0237 + + (2) https://www.python.org/dev/peps/pep-0238 + + +File: python.info, Node: Text Vs Data Instead Of Unicode Vs 8-bit, Prev: Integers, Up: Common Stumbling Blocks + +1.9.1.5 Text Vs. Data Instead Of Unicode Vs. 8-bit +.................................................. + +Everything you thought you knew about binary data and Unicode has +changed. + + * Python 3.0 uses the concepts of `text' and (binary) `data' instead + of Unicode strings and 8-bit strings. All text is Unicode; however + `encoded' Unicode is represented as binary data. The type used to + hold text is *note str: 330, the type used to hold data is *note + bytes: 331. The biggest difference with the 2.x situation is that + any attempt to mix text and data in Python 3.0 raises *note + TypeError: 192, whereas if you were to mix Unicode and 8-bit + strings in Python 2.x, it would work if the 8-bit string happened + to contain only 7-bit (ASCII) bytes, but you would get *note + UnicodeDecodeError: 1fd. if it contained non-ASCII values. This + value-specific behavior has caused numerous sad faces over the + years. + + * As a consequence of this change in philosophy, pretty much all code + that uses Unicode, encodings or binary data most likely has to + change. The change is for the better, as in the 2.x world there + were numerous bugs having to do with mixing encoded and unencoded + text. To be prepared in Python 2.x, start using ‘unicode’ for all + unencoded text, and *note str: 330. for binary or encoded data + only. Then the ‘2to3’ tool will do most of the work for you. + + * You can no longer use ‘u"..."’ literals for Unicode text. However, + you must use ‘b"..."’ literals for binary data. + + * As the *note str: 330. and *note bytes: 331. types cannot be mixed, + you must always explicitly convert between them. Use *note + str.encode(): c37. to go from *note str: 330. to *note bytes: 331, + and *note bytes.decode(): c38. to go from *note bytes: 331. to + *note str: 330. You can also use ‘bytes(s, encoding=...)’ and + ‘str(b, encoding=...)’, respectively. + + * Like *note str: 330, the *note bytes: 331. type is immutable. + There is a separate `mutable' type to hold buffered binary data, + *note bytearray: 332. Nearly all APIs that accept *note bytes: + 331. also accept *note bytearray: 332. The mutable API is based on + ‘collections.MutableSequence’. + + * All backslashes in raw string literals are interpreted literally. + This means that ‘'\U'’ and ‘'\u'’ escapes in raw strings are not + treated specially. For example, ‘r'\u20ac'’ is a string of 6 + characters in Python 3.0, whereas in 2.6, ‘ur'\u20ac'’ was the + single “euro” character. (Of course, this change only affects raw + string literals; the euro character is ‘'\u20ac'’ in Python 3.0.) + + * The built-in ‘basestring’ abstract type was removed. Use *note + str: 330. instead. The *note str: 330. and *note bytes: 331. types + don’t have functionality enough in common to warrant a shared base + class. The ‘2to3’ tool (see below) replaces every occurrence of + ‘basestring’ with *note str: 330. + + * Files opened as text files (still the default mode for *note + open(): 4f0.) always use an encoding to map between strings (in + memory) and bytes (on disk). Binary files (opened with a ‘b’ in + the mode argument) always use bytes in memory. This means that if + a file is opened using an incorrect mode or encoding, I/O will + likely fail loudly, instead of silently producing incorrect data. + It also means that even Unix users will have to specify the correct + mode (text or binary) when opening a file. There is a + platform-dependent default encoding, which on Unixy platforms can + be set with the ‘LANG’ environment variable (and sometimes also + with some other platform-specific locale-related environment + variables). In many cases, but not all, the system default is + UTF-8; you should never count on this default. Any application + reading or writing more than pure ASCII text should probably have a + way to override the encoding. There is no longer any need for + using the encoding-aware streams in the *note codecs: 1c. module. + + * The initial values of *note sys.stdin: 30d, *note sys.stdout: 30e. + and *note sys.stderr: 30f. are now unicode-only text files (i.e., + they are instances of *note io.TextIOBase: c39.). To read and + write bytes data with these streams, you need to use their *note + io.TextIOBase.buffer: c3a. attribute. + + * Filenames are passed to and returned from APIs as (Unicode) + strings. This can present platform-specific problems because on + some platforms filenames are arbitrary byte strings. (On the other + hand, on Windows filenames are natively stored as Unicode.) As a + work-around, most APIs (e.g. *note open(): 4f0. and many functions + in the *note os: c4. module) that take filenames accept *note + bytes: 331. objects as well as strings, and a few APIs have a way + to ask for a *note bytes: 331. return value. Thus, *note + os.listdir(): a41. returns a list of *note bytes: 331. instances if + the argument is a *note bytes: 331. instance, and *note + os.getcwdb(): 2b8. returns the current working directory as a *note + bytes: 331. instance. Note that when *note os.listdir(): a41. + returns a list of strings, filenames that cannot be decoded + properly are omitted rather than raising *note UnicodeError: c3b. + + * Some system APIs like *note os.environ: b37. and *note sys.argv: + bfb. can also present problems when the bytes made available by the + system is not interpretable using the default encoding. Setting + the ‘LANG’ variable and rerunning the program is probably the best + approach. + + * PEP 3138(1): The *note repr(): 7cc. of a string no longer escapes + non-ASCII characters. It still escapes control characters and code + points with non-printable status in the Unicode standard, however. + + * PEP 3120(2): The default source encoding is now UTF-8. + + * PEP 3131(3): Non-ASCII letters are now allowed in identifiers. + (However, the standard library remains ASCII-only with the + exception of contributor names in comments.) + + * The ‘StringIO’ and ‘cStringIO’ modules are gone. Instead, import + the *note io: a1. module and use *note io.StringIO: 82d. or *note + io.BytesIO: 79b. for text and data respectively. + + * See also the *note Unicode HOWTO: c3c, which was updated for Python + 3.0. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3138 + + (2) https://www.python.org/dev/peps/pep-3120 + + (3) https://www.python.org/dev/peps/pep-3131 + + +File: python.info, Node: Overview Of Syntax Changes, Next: Changes Already Present In Python 2 6, Prev: Common Stumbling Blocks, Up: What’s New In Python 3 0 + +1.9.2 Overview Of Syntax Changes +-------------------------------- + +This section gives a brief overview of every `syntactic' change in +Python 3.0. + +* Menu: + +* New Syntax:: +* Changed Syntax:: +* Removed Syntax:: + + +File: python.info, Node: New Syntax, Next: Changed Syntax, Up: Overview Of Syntax Changes + +1.9.2.1 New Syntax +.................. + + * PEP 3107(1): Function argument and return value annotations. This + provides a standardized way of annotating a function’s parameters + and return value. There are no semantics attached to such + annotations except that they can be introspected at runtime using + the ‘__annotations__’ attribute. The intent is to encourage + experimentation through metaclasses, decorators or frameworks. + + * PEP 3102(2): Keyword-only arguments. Named parameters occurring + after ‘*args’ in the parameter list `must' be specified using + keyword syntax in the call. You can also use a bare ‘*’ in the + parameter list to indicate that you don’t accept a variable-length + argument list, but you do have keyword-only arguments. + + * Keyword arguments are allowed after the list of base classes in a + class definition. This is used by the new convention for + specifying a metaclass (see next section), but can be used for + other purposes as well, as long as the metaclass supports it. + + * PEP 3104(3): *note nonlocal: c3f. statement. Using ‘nonlocal x’ + you can now assign directly to a variable in an outer (but + non-global) scope. ‘nonlocal’ is a new reserved word. + + * PEP 3132(4): Extended Iterable Unpacking. You can now write things + like ‘a, b, *rest = some_sequence’. And even ‘*rest, a = stuff’. + The ‘rest’ object is always a (possibly empty) list; the right-hand + side may be any iterable. Example: + + (a, *rest, b) = range(5) + + This sets `a' to ‘0’, `b' to ‘4’, and `rest' to ‘[1, 2, 3]’. + + * Dictionary comprehensions: ‘{k: v for k, v in stuff}’ means the + same thing as ‘dict(stuff)’ but is more flexible. (This is PEP + 274(5) vindicated. :-) + + * Set literals, e.g. ‘{1, 2}’. Note that ‘{}’ is an empty + dictionary; use ‘set()’ for an empty set. Set comprehensions are + also supported; e.g., ‘{x for x in stuff}’ means the same thing as + ‘set(stuff)’ but is more flexible. + + * New octal literals, e.g. ‘0o720’ (already in 2.6). The old octal + literals (‘0720’) are gone. + + * New binary literals, e.g. ‘0b1010’ (already in 2.6), and there is + a new corresponding built-in function, *note bin(): c40. + + * Bytes literals are introduced with a leading ‘b’ or ‘B’, and there + is a new corresponding built-in function, *note bytes(): 331. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3107 + + (2) https://www.python.org/dev/peps/pep-3102 + + (3) https://www.python.org/dev/peps/pep-3104 + + (4) https://www.python.org/dev/peps/pep-3132 + + (5) https://www.python.org/dev/peps/pep-0274 + + +File: python.info, Node: Changed Syntax, Next: Removed Syntax, Prev: New Syntax, Up: Overview Of Syntax Changes + +1.9.2.2 Changed Syntax +...................... + + * PEP 3109(1) and PEP 3134(2): new *note raise: c42. statement + syntax: ‘raise [`expr' [from `expr']]’. See below. + + * ‘as’ and *note with: 6e9. are now reserved words. (Since 2.6, + actually.) + + * ‘True’, ‘False’, and ‘None’ are reserved words. (2.6 partially + enforced the restrictions on ‘None’ already.) + + * Change from *note except: b3e. `exc', `var' to ‘except’ `exc' ‘as’ + `var'. See PEP 3110(3). + + * PEP 3115(4): New Metaclass Syntax. Instead of: + + class C: + __metaclass__ = M + ... + + you must now use: + + class C(metaclass=M): + ... + + The module-global ‘__metaclass__’ variable is no longer supported. + (It was a crutch to make it easier to default to new-style classes + without deriving every class from *note object: 2b0.) + + * List comprehensions no longer support the syntactic form ‘[... for + `var' in `item1', `item2', ...]’. Use ‘[... for `var' in (`item1', + `item2', ...)]’ instead. Also note that list comprehensions have + different semantics: they are closer to syntactic sugar for a + generator expression inside a *note list(): 262. constructor, and + in particular the loop control variables are no longer leaked into + the surrounding scope. + + * The `ellipsis' (‘...’) can be used as an atomic expression + anywhere. (Previously it was only allowed in slices.) Also, it + `must' now be spelled as ‘...’. (Previously it could also be + spelled as ‘. . .’, by a mere accident of the grammar.) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3109 + + (2) https://www.python.org/dev/peps/pep-3134 + + (3) https://www.python.org/dev/peps/pep-3110 + + (4) https://www.python.org/dev/peps/pep-3115 + + +File: python.info, Node: Removed Syntax, Prev: Changed Syntax, Up: Overview Of Syntax Changes + +1.9.2.3 Removed Syntax +...................... + + * PEP 3113(1): Tuple parameter unpacking removed. You can no longer + write ‘def foo(a, (b, c)): ...’. Use ‘def foo(a, b_c): b, c = b_c’ + instead. + + * Removed backticks (use *note repr(): 7cc. instead). + + * Removed ‘<>’ (use ‘!=’ instead). + + * Removed keyword: *note exec(): c44. is no longer a keyword; it + remains as a function. (Fortunately the function syntax was also + accepted in 2.x.) Also note that *note exec(): c44. no longer + takes a stream argument; instead of ‘exec(f)’ you can use + ‘exec(f.read())’. + + * Integer literals no longer support a trailing ‘l’ or ‘L’. + + * String literals no longer support a leading ‘u’ or ‘U’. + + * The *note from: c45. `module' *note import: c1d. ‘*’ syntax is only + allowed at the module level, no longer inside functions. + + * The only acceptable syntax for relative imports is ‘from + .[`module'] import `name'’. All *note import: c1d. forms not + starting with ‘.’ are interpreted as absolute imports. ( PEP + 328(2)) + + * Classic classes are gone. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3113 + + (2) https://www.python.org/dev/peps/pep-0328 + + +File: python.info, Node: Changes Already Present In Python 2 6, Next: Library Changes, Prev: Overview Of Syntax Changes, Up: What’s New In Python 3 0 + +1.9.3 Changes Already Present In Python 2.6 +------------------------------------------- + +Since many users presumably make the jump straight from Python 2.5 to +Python 3.0, this section reminds the reader of new features that were +originally designed for Python 3.0 but that were back-ported to Python +2.6. The corresponding sections in *note What’s New in Python 2.6: c47. +should be consulted for longer descriptions. + + * *note PEP 343; The ‘with’ statement: c48. The *note with: 6e9. + statement is now a standard feature and no longer needs to be + imported from the *note __future__: 0. Also check out *note + Writing Context Managers: c49. and *note The contextlib module: + c4a. + + * *note PEP 366; Explicit Relative Imports From a Main Module: c4b. + This enhances the usefulness of the *note -m: 337. option when the + referenced module lives in a package. + + * *note PEP 370; Per-user site-packages Directory: c4c. + + * *note PEP 371; The multiprocessing Package: c4d. + + * *note PEP 3101; Advanced String Formatting: c4e. Note: the 2.6 + description mentions the *note format(): 4db. method for both 8-bit + and Unicode strings. In 3.0, only the *note str: 330. type (text + strings with Unicode support) supports this method; the *note + bytes: 331. type does not. The plan is to eventually make this the + only API for string formatting, and to start deprecating the ‘%’ + operator in Python 3.1. + + * *note PEP 3105; print As a Function: c4f. This is now a standard + feature and no longer needs to be imported from *note __future__: + 0. More details were given above. + + * *note PEP 3110; Exception-Handling Changes: c50. The *note except: + b3e. `exc' ‘as’ `var' syntax is now standard and ‘except’ `exc', + `var' is no longer supported. (Of course, the ‘as’ `var' part is + still optional.) + + * *note PEP 3112; Byte Literals: c51. The ‘b"..."’ string literal + notation (and its variants like ‘b'...'’, ‘b"""..."""’, and + ‘br"..."’) now produces a literal of type *note bytes: 331. + + * *note PEP 3116; New I/O Library: c52. The *note io: a1. module is + now the standard way of doing file I/O. The built-in *note open(): + 4f0. function is now an alias for *note io.open(): 65a. and has + additional keyword arguments `encoding', `errors', `newline' and + `closefd'. Also note that an invalid `mode' argument now raises + *note ValueError: 1fb, not *note IOError: 992. The binary file + object underlying a text file object can be accessed as ‘f.buffer’ + (but beware that the text object maintains a buffer of itself in + order to speed up the encoding and decoding operations). + + * *note PEP 3118; Revised Buffer Protocol: c53. The old builtin + ‘buffer()’ is now really gone; the new builtin *note memoryview(): + 25c. provides (mostly) similar functionality. + + * *note PEP 3119; Abstract Base Classes: c54. The *note abc: 4. + module and the ABCs defined in the *note collections: 1e. module + plays a somewhat more prominent role in the language now, and + built-in collection types like *note dict: 1b8. and *note list: + 262. conform to the ‘collections.MutableMapping’ and + ‘collections.MutableSequence’ ABCs, respectively. + + * *note PEP 3127; Integer Literal Support and Syntax: c55. As + mentioned above, the new octal literal notation is the only one + supported, and binary literals have been added. + + * *note PEP 3129; Class Decorators: c56. + + * *note PEP 3141; A Type Hierarchy for Numbers: c57. The *note + numbers: c1. module is another new use of ABCs, defining Python’s + “numeric tower”. Also note the new *note fractions: 83. module + which implements *note numbers.Rational: c58. + + +File: python.info, Node: Library Changes, Next: PEP 3101 A New Approach To String Formatting, Prev: Changes Already Present In Python 2 6, Up: What’s New In Python 3 0 + +1.9.4 Library Changes +--------------------- + +Due to time constraints, this document does not exhaustively cover the +very extensive changes to the standard library. PEP 3108(1) is the +reference for the major changes to the library. Here’s a capsule +review: + + * Many old modules were removed. Some, like ‘gopherlib’ (no longer + used) and ‘md5’ (replaced by *note hashlib: 8d.), were already + deprecated by PEP 4(2). Others were removed as a result of the + removal of support for various platforms such as Irix, BeOS and Mac + OS 9 (see PEP 11(3)). Some modules were also selected for removal + in Python 3.0 due to lack of use or because a better replacement + exists. See PEP 3108(4) for an exhaustive list. + + * The ‘bsddb3’ package was removed because its presence in the core + standard library has proved over time to be a particular burden for + the core developers due to testing instability and Berkeley DB’s + release schedule. However, the package is alive and well, + externally maintained at + ‘https://www.jcea.es/programacion/pybsddb.htm’. + + * Some modules were renamed because their old name disobeyed PEP + 8(5), or for various other reasons. Here’s the list: + + Old Name New Name + + -------------------------------------------------------- + + _winreg winreg + + + ConfigParser configparser + + + copy_reg copyreg + + + Queue queue + + + SocketServer socketserver + + + markupbase _markupbase + + + repr reprlib + + + test.test_support test.support + + + * A common pattern in Python 2.x is to have one version of a module + implemented in pure Python, with an optional accelerated version + implemented as a C extension; for example, *note pickle: ca. and + ‘cPickle’. This places the burden of importing the accelerated + version and falling back on the pure Python version on each user of + these modules. In Python 3.0, the accelerated versions are + considered implementation details of the pure Python versions. + Users should always import the standard version, which attempts to + import the accelerated version and falls back to the pure Python + version. The *note pickle: ca. / ‘cPickle’ pair received this + treatment. The *note profile: d3. module is on the list for 3.1. + The ‘StringIO’ module has been turned into a class in the *note io: + a1. module. + + * Some related modules have been grouped into packages, and usually + the submodule names have been simplified. The resulting new + packages are: + + * *note dbm: 32. (‘anydbm’, ‘dbhash’, *note dbm: 32, ‘dumbdbm’, + ‘gdbm’, ‘whichdb’). + + * *note html: 90. (‘HTMLParser’, ‘htmlentitydefs’). + + * *note http: 93. (‘httplib’, ‘BaseHTTPServer’, ‘CGIHTTPServer’, + ‘SimpleHTTPServer’, ‘Cookie’, ‘cookielib’). + + * *note tkinter: 10c. (all ‘Tkinter’-related modules except + *note turtle: 116.). The target audience of *note turtle: + 116. doesn’t really care about *note tkinter: 10c. Also note + that as of Python 2.6, the functionality of *note turtle: 116. + has been greatly enhanced. + + * *note urllib: 11d. (*note urllib: 11d, ‘urllib2’, ‘urlparse’, + ‘robotparse’). + + * ‘xmlrpc’ (‘xmlrpclib’, ‘DocXMLRPCServer’, + ‘SimpleXMLRPCServer’). + +Some other changes to standard library modules, not covered by PEP +3108(6): + + * Killed ‘sets’. Use the built-in *note set(): b6f. class. + + * Cleanup of the *note sys: fd. module: removed ‘sys.exitfunc()’, + ‘sys.exc_clear()’, ‘sys.exc_type’, ‘sys.exc_value’, + ‘sys.exc_traceback’. (Note that *note sys.last_type: c5a. etc. + remain.) + + * Cleanup of the *note array.array: 451. type: the ‘read()’ and + ‘write()’ methods are gone; use ‘fromfile()’ and ‘tofile()’ + instead. Also, the ‘'c'’ typecode for array is gone – use either + ‘'b'’ for bytes or ‘'u'’ for Unicode characters. + + * Cleanup of the *note operator: c2. module: removed + ‘sequenceIncludes()’ and ‘isCallable()’. + + * Cleanup of the ‘thread’ module: ‘acquire_lock()’ and + ‘release_lock()’ are gone; use ‘acquire()’ and ‘release()’ instead. + + * Cleanup of the *note random: dc. module: removed the ‘jumpahead()’ + API. + + * The ‘new’ module is gone. + + * The functions ‘os.tmpnam()’, ‘os.tempnam()’ and ‘os.tmpfile()’ have + been removed in favor of the *note tempfile: 103. module. + + * The *note tokenize: 111. module has been changed to work with + bytes. The main entry point is now *note tokenize.tokenize(): c5b, + instead of generate_tokens. + + * ‘string.letters’ and its friends (‘string.lowercase’ and + ‘string.uppercase’) are gone. Use *note string.ascii_letters: c5c. + etc. instead. (The reason for the removal is that + ‘string.letters’ and friends had locale-specific behavior, which is + a bad idea for such attractively-named global “constants”.) + + * Renamed module ‘__builtin__’ to *note builtins: 13. (removing the + underscores, adding an ‘s’). The ‘__builtins__’ variable found in + most global namespaces is unchanged. To modify a builtin, you + should use *note builtins: 13, not ‘__builtins__’! + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3108 + + (2) https://www.python.org/dev/peps/pep-0004 + + (3) https://www.python.org/dev/peps/pep-0011 + + (4) https://www.python.org/dev/peps/pep-3108 + + (5) https://www.python.org/dev/peps/pep-0008 + + (6) https://www.python.org/dev/peps/pep-3108 + + +File: python.info, Node: PEP 3101 A New Approach To String Formatting, Next: Changes To Exceptions, Prev: Library Changes, Up: What’s New In Python 3 0 + +1.9.5 `PEP 3101': A New Approach To String Formatting +----------------------------------------------------- + + * A new system for built-in string formatting operations replaces the + ‘%’ string formatting operator. (However, the ‘%’ operator is + still supported; it will be deprecated in Python 3.1 and removed + from the language at some later time.) Read PEP 3101(1) for the + full scoop. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3101 + + +File: python.info, Node: Changes To Exceptions, Next: Miscellaneous Other Changes, Prev: PEP 3101 A New Approach To String Formatting, Up: What’s New In Python 3 0 + +1.9.6 Changes To Exceptions +--------------------------- + +The APIs for raising and catching exception have been cleaned up and new +powerful features added: + + * PEP 352(1): All exceptions must be derived (directly or indirectly) + from *note BaseException: 1a8. This is the root of the exception + hierarchy. This is not new as a recommendation, but the + `requirement' to inherit from *note BaseException: 1a8. is new. + (Python 2.6 still allowed classic classes to be raised, and placed + no restriction on what you can catch.) As a consequence, string + exceptions are finally truly and utterly dead. + + * Almost all exceptions should actually derive from *note Exception: + 1a9.; *note BaseException: 1a8. should only be used as a base class + for exceptions that should only be handled at the top level, such + as *note SystemExit: 5fc. or *note KeyboardInterrupt: 197. The + recommended idiom for handling all exceptions except for this + latter category is to use *note except: b3e. *note Exception: 1a9. + + * ‘StandardError’ was removed. + + * Exceptions no longer behave as sequences. Use the ‘args’ attribute + instead. + + * PEP 3109(2): Raising exceptions. You must now use ‘raise + `Exception'(`args')’ instead of ‘raise `Exception', `args'’. + Additionally, you can no longer explicitly specify a traceback; + instead, if you `have' to do this, you can assign directly to the + ‘__traceback__’ attribute (see below). + + * PEP 3110(3): Catching exceptions. You must now use ‘except + `SomeException' as `variable'’ instead of ‘except `SomeException', + `variable'’. Moreover, the `variable' is explicitly deleted when + the *note except: b3e. block is left. + + * PEP 3134(4): Exception chaining. There are two cases: implicit + chaining and explicit chaining. Implicit chaining happens when an + exception is raised in an *note except: b3e. or *note finally: 182. + handler block. This usually happens due to a bug in the handler + block; we call this a `secondary' exception. In this case, the + original exception (that was being handled) is saved as the + ‘__context__’ attribute of the secondary exception. Explicit + chaining is invoked with this syntax: + + raise SecondaryException() from primary_exception + + (where `primary_exception' is any expression that produces an + exception object, probably an exception that was previously + caught). In this case, the primary exception is stored on the + ‘__cause__’ attribute of the secondary exception. The traceback + printed when an unhandled exception occurs walks the chain of + ‘__cause__’ and ‘__context__’ attributes and prints a separate + traceback for each component of the chain, with the primary + exception at the top. (Java users may recognize this behavior.) + + * PEP 3134(5): Exception objects now store their traceback as the + ‘__traceback__’ attribute. This means that an exception object now + contains all the information pertaining to an exception, and there + are fewer reasons to use *note sys.exc_info(): c5f. (though the + latter is not removed). + + * A few exception messages are improved when Windows fails to load an + extension module. For example, ‘error code 193’ is now ‘%1 is not + a valid Win32 application’. Strings now deal with non-English + locales. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0352 + + (2) https://www.python.org/dev/peps/pep-3109 + + (3) https://www.python.org/dev/peps/pep-3110 + + (4) https://www.python.org/dev/peps/pep-3134 + + (5) https://www.python.org/dev/peps/pep-3134 + + +File: python.info, Node: Miscellaneous Other Changes, Next: Build and C API Changes<7>, Prev: Changes To Exceptions, Up: What’s New In Python 3 0 + +1.9.7 Miscellaneous Other Changes +--------------------------------- + +* Menu: + +* Operators And Special Methods:: +* Builtins:: + + +File: python.info, Node: Operators And Special Methods, Next: Builtins, Up: Miscellaneous Other Changes + +1.9.7.1 Operators And Special Methods +..................................... + + * ‘!=’ now returns the opposite of ‘==’, unless ‘==’ returns *note + NotImplemented: 84c. + + * The concept of “unbound methods” has been removed from the + language. When referencing a method as a class attribute, you now + get a plain function object. + + * ‘__getslice__()’, ‘__setslice__()’ and ‘__delslice__()’ were + killed. The syntax ‘a[i:j]’ now translates to + ‘a.__getitem__(slice(i, j))’ (or *note __setitem__(): c62. or *note + __delitem__(): c63, when used as an assignment or deletion target, + respectively). + + * PEP 3114(1): the standard *note next(): 682. method has been + renamed to *note __next__(): c64. + + * The ‘__oct__()’ and ‘__hex__()’ special methods are removed – *note + oct(): c65. and *note hex(): c66. use *note __index__(): 18a. now + to convert the argument to an integer. + + * Removed support for ‘__members__’ and ‘__methods__’. + + * The function attributes named ‘func_X’ have been renamed to use the + ‘__X__’ form, freeing up these names in the function attribute + namespace for user-defined attributes. To wit, ‘func_closure’, + ‘func_code’, ‘func_defaults’, ‘func_dict’, ‘func_doc’, + ‘func_globals’, ‘func_name’ were renamed to ‘__closure__’, + ‘__code__’, ‘__defaults__’, *note __dict__: 4fc, ‘__doc__’, + ‘__globals__’, *note __name__: c67, respectively. + + * ‘__nonzero__()’ is now *note __bool__(): c68. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3114 + + +File: python.info, Node: Builtins, Prev: Operators And Special Methods, Up: Miscellaneous Other Changes + +1.9.7.2 Builtins +................ + + * PEP 3135(1): New *note super(): 4e2. You can now invoke *note + super(): 4e2. without arguments and (assuming this is in a regular + instance method defined inside a *note class: c6a. statement) the + right class and instance will automatically be chosen. With + arguments, the behavior of *note super(): 4e2. is unchanged. + + * PEP 3111(2): ‘raw_input()’ was renamed to *note input(): c6b. That + is, the new *note input(): c6b. function reads a line from *note + sys.stdin: 30d. and returns it with the trailing newline stripped. + It raises *note EOFError: c6c. if the input is terminated + prematurely. To get the old behavior of *note input(): c6b, use + ‘eval(input())’. + + * A new built-in function *note next(): 682. was added to call the + *note __next__(): c64. method on an object. + + * The *note round(): c6d. function rounding strategy and return type + have changed. Exact halfway cases are now rounded to the nearest + even result instead of away from zero. (For example, ‘round(2.5)’ + now returns ‘2’ rather than ‘3’.) ‘round(x[, n])’ now delegates to + ‘x.__round__([n])’ instead of always returning a float. It + generally returns an integer when called with a single argument and + a value of the same type as ‘x’ when called with two arguments. + + * Moved ‘intern()’ to *note sys.intern(): c6e. + + * Removed: ‘apply()’. Instead of ‘apply(f, args)’ use ‘f(*args)’. + + * Removed *note callable(): b44. Instead of ‘callable(f)’ you can + use ‘isinstance(f, collections.Callable)’. The + ‘operator.isCallable()’ function is also gone. + + * Removed ‘coerce()’. This function no longer serves a purpose now + that classic classes are gone. + + * Removed ‘execfile()’. Instead of ‘execfile(fn)’ use + ‘exec(open(fn).read())’. + + * Removed the ‘file’ type. Use *note open(): 4f0. There are now + several different kinds of streams that open can return in the + *note io: a1. module. + + * Removed ‘reduce()’. Use *note functools.reduce(): c6f. if you + really need it; however, 99 percent of the time an explicit *note + for: c30. loop is more readable. + + * Removed ‘reload()’. Use *note imp.reload(): c70. + + * Removed. ‘dict.has_key()’ – use the *note in: 7a3. operator + instead. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3135 + + (2) https://www.python.org/dev/peps/pep-3111 + + +File: python.info, Node: Build and C API Changes<7>, Next: Performance, Prev: Miscellaneous Other Changes, Up: What’s New In Python 3 0 + +1.9.8 Build and C API Changes +----------------------------- + +Due to time constraints, here is a `very' incomplete list of changes to +the C API. + + * Support for several platforms was dropped, including but not + limited to Mac OS 9, BeOS, RISCOS, Irix, and Tru64. + + * PEP 3118(1): New Buffer API. + + * PEP 3121(2): Extension Module Initialization & Finalization. + + * PEP 3123(3): Making *note PyObject_HEAD: c72. conform to standard + C. + + * No more C API support for restricted execution. + + * ‘PyNumber_Coerce()’, ‘PyNumber_CoerceEx()’, ‘PyMember_Get()’, and + ‘PyMember_Set()’ C APIs are removed. + + * New C API *note PyImport_ImportModuleNoBlock(): 9c1, works like + *note PyImport_ImportModule(): c73. but won’t block on the import + lock (returning an error instead). + + * Renamed the boolean conversion C-level slot and method: + ‘nb_nonzero’ is now ‘nb_bool’. + + * Removed ‘METH_OLDARGS’ and ‘WITH_CYCLE_GC’ from the C API. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3118 + + (2) https://www.python.org/dev/peps/pep-3121 + + (3) https://www.python.org/dev/peps/pep-3123 + + +File: python.info, Node: Performance, Next: Porting To Python 3 0, Prev: Build and C API Changes<7>, Up: What’s New In Python 3 0 + +1.9.9 Performance +----------------- + +The net result of the 3.0 generalizations is that Python 3.0 runs the +pystone benchmark around 10% slower than Python 2.5. Most likely the +biggest cause is the removal of special-casing for small integers. +There’s room for improvement, but it will happen after 3.0 is released! + + +File: python.info, Node: Porting To Python 3 0, Prev: Performance, Up: What’s New In Python 3 0 + +1.9.10 Porting To Python 3.0 +---------------------------- + +For porting existing Python 2.5 or 2.6 source code to Python 3.0, the +best strategy is the following: + + 0. (Prerequisite:) Start with excellent test coverage. + + 1. Port to Python 2.6. This should be no more work than the average + port from Python 2.x to Python 2.(x+1). Make sure all your tests + pass. + + 2. (Still using 2.6:) Turn on the ‘-3’ command line switch. This + enables warnings about features that will be removed (or change) in + 3.0. Run your test suite again, and fix code that you get warnings + about until there are no warnings left, and all your tests still + pass. + + 3. Run the ‘2to3’ source-to-source translator over your source code + tree. (See *note 2to3 - Automated Python 2 to 3 code translation: + c76. for more on this tool.) Run the result of the translation + under Python 3.0. Manually fix up any remaining issues, fixing + problems until all tests pass again. + +It is not recommended to try to write source code that runs unchanged +under both Python 2.6 and 3.0; you’d have to use a very contorted coding +style, e.g. avoiding ‘print’ statements, metaclasses, and much more. +If you are maintaining a library that needs to support both Python 2.6 +and Python 3.0, the best approach is to modify step 3 above by editing +the 2.6 version of the source code and running the ‘2to3’ translator +again, rather than editing the 3.0 version of the source code. + +For porting C extensions to Python 3.0, please see *note Porting +Extension Modules to Python 3: c77. + + +File: python.info, Node: What’s New in Python 2 7, Next: What’s New in Python 2 6, Prev: What’s New In Python 3 0, Up: What’s New in Python + +1.10 What’s New in Python 2.7 +============================= + + +Author: A.M. Kuchling (amk at amk.ca) + +This article explains the new features in Python 2.7. Python 2.7 was +released on July 3, 2010. + +Numeric handling has been improved in many ways, for both floating-point +numbers and for the *note Decimal: 188. class. There are some useful +additions to the standard library, such as a greatly enhanced *note +unittest: 11b. module, the *note argparse: 6. module for parsing +command-line options, convenient *note OrderedDict: 1b9. and *note +Counter: 9da. classes in the *note collections: 1e. module, and many +other improvements. + +Python 2.7 is planned to be the last of the 2.x releases, so we worked +on making it a good release for the long term. To help with porting to +Python 3, several new features from the Python 3.x series have been +included in 2.7. + +This article doesn’t attempt to provide a complete specification of the +new features, but instead provides a convenient overview. For full +details, you should refer to the documentation for Python 2.7 at +‘https://docs.python.org’. If you want to understand the rationale for +the design and implementation, refer to the PEP for a particular new +feature or the issue on ‘https://bugs.python.org’ in which a change was +discussed. Whenever possible, “What’s New in Python” links to the +bug/patch item for each change. + +* Menu: + +* The Future for Python 2.x: The Future for Python 2 x. +* Changes to the Handling of Deprecation Warnings:: +* Python 3.1 Features: Python 3 1 Features. +* PEP 372; Adding an Ordered Dictionary to collections: PEP 372 Adding an Ordered Dictionary to collections. +* PEP 378; Format Specifier for Thousands Separator: PEP 378 Format Specifier for Thousands Separator<2>. +* PEP 389; The argparse Module for Parsing Command Lines: PEP 389 The argparse Module for Parsing Command Lines. +* PEP 391; Dictionary-Based Configuration For Logging: PEP 391 Dictionary-Based Configuration For Logging. +* PEP 3106; Dictionary Views: PEP 3106 Dictionary Views. +* PEP 3137; The memoryview Object: PEP 3137 The memoryview Object. +* Other Language Changes: Other Language Changes<9>. +* New and Improved Modules:: +* Build and C API Changes: Build and C API Changes<8>. +* Other Changes and Fixes:: +* Porting to Python 2.7: Porting to Python 2 7. +* New Features Added to Python 2.7 Maintenance Releases: New Features Added to Python 2 7 Maintenance Releases. +* Acknowledgements:: + + +File: python.info, Node: The Future for Python 2 x, Next: Changes to the Handling of Deprecation Warnings, Up: What’s New in Python 2 7 + +1.10.1 The Future for Python 2.x +-------------------------------- + +Python 2.7 is the last major release in the 2.x series, as the Python +maintainers have shifted the focus of their new feature development +efforts to the Python 3.x series. This means that while Python 2 +continues to receive bug fixes, and to be updated to build correctly on +new hardware and versions of supported operated systems, there will be +no new full feature releases for the language or standard library. + +However, while there is a large common subset between Python 2.7 and +Python 3, and many of the changes involved in migrating to that common +subset, or directly to Python 3, can be safely automated, some other +changes (notably those associated with Unicode handling) may require +careful consideration, and preferably robust automated regression test +suites, to migrate effectively. + +This means that Python 2.7 will remain in place for a long time, +providing a stable and supported base platform for production systems +that have not yet been ported to Python 3. The full expected lifecycle +of the Python 2.7 series is detailed in PEP 373(1). + +Some key consequences of the long-term significance of 2.7 are: + + * As noted above, the 2.7 release has a much longer period of + maintenance when compared to earlier 2.x versions. Python 2.7 is + currently expected to remain supported by the core development team + (receiving security updates and other bug fixes) until at least + 2020 (10 years after its initial release, compared to the more + typical support period of 18–24 months). + + * As the Python 2.7 standard library ages, making effective use of + the Python Package Index (either directly or via a redistributor) + becomes more important for Python 2 users. In addition to a wide + variety of third party packages for various tasks, the available + packages include backports of new modules and features from the + Python 3 standard library that are compatible with Python 2, as + well as various tools and libraries that can make it easier to + migrate to Python 3. The Python Packaging User Guide(2) provides + guidance on downloading and installing software from the Python + Package Index. + + * While the preferred approach to enhancing Python 2 is now the + publication of new packages on the Python Package Index, this + approach doesn’t necessarily work in all cases, especially those + related to network security. In exceptional cases that cannot be + handled adequately by publishing new or updated packages on PyPI, + the Python Enhancement Proposal process may be used to make the + case for adding new features directly to the Python 2 standard + library. Any such additions, and the maintenance releases where + they were added, will be noted in the *note New Features Added to + Python 2.7 Maintenance Releases: c7c. section below. + +For projects wishing to migrate from Python 2 to Python 3, or for +library and framework developers wishing to support users on both Python +2 and Python 3, there are a variety of tools and guides available to +help decide on a suitable approach and manage some of the technical +details involved. The recommended starting point is the *note Porting +Python 2 Code to Python 3: c7d. HOWTO guide. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0373 + + (2) https://packaging.python.org + + +File: python.info, Node: Changes to the Handling of Deprecation Warnings, Next: Python 3 1 Features, Prev: The Future for Python 2 x, Up: What’s New in Python 2 7 + +1.10.2 Changes to the Handling of Deprecation Warnings +------------------------------------------------------ + +For Python 2.7, a policy decision was made to silence warnings only of +interest to developers by default. *note DeprecationWarning: 278. and +its descendants are now ignored unless otherwise requested, preventing +users from seeing warnings triggered by an application. This change was +also made in the branch that became Python 3.2. (Discussed on +stdlib-sig and carried out in bpo-7319(1).) + +In previous releases, *note DeprecationWarning: 278. messages were +enabled by default, providing Python developers with a clear indication +of where their code may break in a future major version of Python. + +However, there are increasingly many users of Python-based applications +who are not directly involved in the development of those applications. +*note DeprecationWarning: 278. messages are irrelevant to such users, +making them worry about an application that’s actually working correctly +and burdening application developers with responding to these concerns. + +You can re-enable display of *note DeprecationWarning: 278. messages by +running Python with the *note -Wdefault: 417. (short form: *note -Wd: +417.) switch, or by setting the *note PYTHONWARNINGS: 418. environment +variable to ‘"default"’ (or ‘"d"’) before running Python. Python code +can also re-enable them by calling ‘warnings.simplefilter('default')’. + +The ‘unittest’ module also automatically reenables deprecation warnings +when running tests. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7319 + + +File: python.info, Node: Python 3 1 Features, Next: PEP 372 Adding an Ordered Dictionary to collections, Prev: Changes to the Handling of Deprecation Warnings, Up: What’s New in Python 2 7 + +1.10.3 Python 3.1 Features +-------------------------- + +Much as Python 2.6 incorporated features from Python 3.0, version 2.7 +incorporates some of the new features in Python 3.1. The 2.x series +continues to provide tools for migrating to the 3.x series. + +A partial list of 3.1 features that were backported to 2.7: + + * The syntax for set literals (‘{1,2,3}’ is a mutable set). + + * Dictionary and set comprehensions (‘{i: i*2 for i in range(3)}’). + + * Multiple context managers in a single *note with: 6e9. statement. + + * A new version of the *note io: a1. library, rewritten in C for + performance. + + * The ordered-dictionary type described in *note PEP 372; Adding an + Ordered Dictionary to collections: c80. + + * The new ‘","’ format specifier described in *note PEP 378; Format + Specifier for Thousands Separator: c81. + + * The *note memoryview: 25c. object. + + * A small subset of the *note importlib: 9b. module, *note described + below: c82. + + * The *note repr(): 7cc. of a float ‘x’ is shorter in many cases: + it’s now based on the shortest decimal string that’s guaranteed to + round back to ‘x’. As in previous versions of Python, it’s + guaranteed that ‘float(repr(x))’ recovers ‘x’. + + * Float-to-string and string-to-float conversions are correctly + rounded. The *note round(): c6d. function is also now correctly + rounded. + + * The *note PyCapsule: c07. type, used to provide a C API for + extension modules. + + * The *note PyLong_AsLongAndOverflow(): bfd. C API function. + +Other new Python3-mode warnings include: + + * ‘operator.isCallable()’ and ‘operator.sequenceIncludes()’, which + are not supported in 3.x, now trigger warnings. + + * The ‘-3’ switch now automatically enables the ‘-Qwarn’ switch that + causes warnings about using classic division with integers and long + integers. + + +File: python.info, Node: PEP 372 Adding an Ordered Dictionary to collections, Next: PEP 378 Format Specifier for Thousands Separator<2>, Prev: Python 3 1 Features, Up: What’s New in Python 2 7 + +1.10.4 PEP 372: Adding an Ordered Dictionary to collections +----------------------------------------------------------- + +Regular Python dictionaries iterate over key/value pairs in arbitrary +order. Over the years, a number of authors have written alternative +implementations that remember the order that the keys were originally +inserted. Based on the experiences from those implementations, 2.7 +introduces a new *note OrderedDict: 1b9. class in the *note collections: +1e. module. + +The *note OrderedDict: 1b9. API provides the same interface as regular +dictionaries but iterates over keys and values in a guaranteed order +depending on when a key was first inserted: + + >>> from collections import OrderedDict + >>> d = OrderedDict([('first', 1), + ... ('second', 2), + ... ('third', 3)]) + >>> d.items() + [('first', 1), ('second', 2), ('third', 3)] + +If a new entry overwrites an existing entry, the original insertion +position is left unchanged: + + >>> d['second'] = 4 + >>> d.items() + [('first', 1), ('second', 4), ('third', 3)] + +Deleting an entry and reinserting it will move it to the end: + + >>> del d['second'] + >>> d['second'] = 5 + >>> d.items() + [('first', 1), ('third', 3), ('second', 5)] + +The *note popitem(): c84. method has an optional `last' argument that +defaults to ‘True’. If `last' is true, the most recently added key is +returned and removed; if it’s false, the oldest key is selected: + + >>> od = OrderedDict([(x,0) for x in range(20)]) + >>> od.popitem() + (19, 0) + >>> od.popitem() + (18, 0) + >>> od.popitem(last=False) + (0, 0) + >>> od.popitem(last=False) + (1, 0) + +Comparing two ordered dictionaries checks both the keys and values, and +requires that the insertion order was the same: + + >>> od1 = OrderedDict([('first', 1), + ... ('second', 2), + ... ('third', 3)]) + >>> od2 = OrderedDict([('third', 3), + ... ('first', 1), + ... ('second', 2)]) + >>> od1 == od2 + False + >>> # Move 'third' key to the end + >>> del od2['third']; od2['third'] = 3 + >>> od1 == od2 + True + +Comparing an *note OrderedDict: 1b9. with a regular dictionary ignores +the insertion order and just compares the keys and values. + +How does the *note OrderedDict: 1b9. work? It maintains a doubly-linked +list of keys, appending new keys to the list as they’re inserted. A +secondary dictionary maps keys to their corresponding list node, so +deletion doesn’t have to traverse the entire linked list and therefore +remains O(1). + +The standard library now supports use of ordered dictionaries in several +modules. + + * The ‘ConfigParser’ module uses them by default, meaning that + configuration files can now be read, modified, and then written + back in their original order. + + * The *note _asdict(): 1b6. method for *note + collections.namedtuple(): 1b7. now returns an ordered dictionary + with the values appearing in the same order as the underlying tuple + indices. + + * The *note json: a4. module’s *note JSONDecoder: 607. class + constructor was extended with an `object_pairs_hook' parameter to + allow ‘OrderedDict’ instances to be built by the decoder. Support + was also added for third-party tools like PyYAML(1). + +See also +........ + +PEP 372(2) - Adding an ordered dictionary to collections + + PEP written by Armin Ronacher and Raymond Hettinger; implemented by + Raymond Hettinger. + + ---------- Footnotes ---------- + + (1) http://pyyaml.org/ + + (2) https://www.python.org/dev/peps/pep-0372 + + +File: python.info, Node: PEP 378 Format Specifier for Thousands Separator<2>, Next: PEP 389 The argparse Module for Parsing Command Lines, Prev: PEP 372 Adding an Ordered Dictionary to collections, Up: What’s New in Python 2 7 + +1.10.5 PEP 378: Format Specifier for Thousands Separator +-------------------------------------------------------- + +To make program output more readable, it can be useful to add separators +to large numbers, rendering them as 18,446,744,073,709,551,616 instead +of 18446744073709551616. + +The fully general solution for doing this is the *note locale: a9. +module, which can use different separators (“,” in North America, “.” in +Europe) and different grouping sizes, but *note locale: a9. is +complicated to use and unsuitable for multi-threaded applications where +different threads are producing output for different locales. + +Therefore, a simple comma-grouping mechanism has been added to the +mini-language used by the *note str.format(): 4da. method. When +formatting a floating-point number, simply include a comma between the +width and the precision: + + >>> '{:20,.2f}'.format(18446744073709551616.0) + '18,446,744,073,709,551,616.00' + +When formatting an integer, include the comma after the width: + + >>> '{:20,d}'.format(18446744073709551616) + '18,446,744,073,709,551,616' + +This mechanism is not adaptable at all; commas are always used as the +separator and the grouping is always into three-digit groups. The +comma-formatting mechanism isn’t as general as the *note locale: a9. +module, but it’s easier to use. + +See also +........ + +PEP 378(1) - Format Specifier for Thousands Separator + + PEP written by Raymond Hettinger; implemented by Eric Smith. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0378 + + +File: python.info, Node: PEP 389 The argparse Module for Parsing Command Lines, Next: PEP 391 Dictionary-Based Configuration For Logging, Prev: PEP 378 Format Specifier for Thousands Separator<2>, Up: What’s New in Python 2 7 + +1.10.6 PEP 389: The argparse Module for Parsing Command Lines +------------------------------------------------------------- + +The *note argparse: 6. module for parsing command-line arguments was +added as a more powerful replacement for the *note optparse: c3. module. + +This means Python now supports three different modules for parsing +command-line arguments: *note getopt: 87, *note optparse: c3, and *note +argparse: 6. The *note getopt: 87. module closely resembles the C +library’s ‘getopt()’ function, so it remains useful if you’re writing a +Python prototype that will eventually be rewritten in C. *note optparse: +c3. becomes redundant, but there are no plans to remove it because there +are many scripts still using it, and there’s no automated way to update +these scripts. (Making the *note argparse: 6. API consistent with *note +optparse: c3.’s interface was discussed but rejected as too messy and +difficult.) + +In short, if you’re writing a new script and don’t need to worry about +compatibility with earlier versions of Python, use *note argparse: 6. +instead of *note optparse: c3. + +Here’s an example: + + import argparse + + parser = argparse.ArgumentParser(description='Command-line example.') + + # Add optional switches + parser.add_argument('-v', action='store_true', dest='is_verbose', + help='produce verbose output') + parser.add_argument('-o', action='store', dest='output', + metavar='FILE', + help='direct output to FILE instead of stdout') + parser.add_argument('-C', action='store', type=int, dest='context', + metavar='NUM', default=0, + help='display NUM lines of added context') + + # Allow any number of additional arguments. + parser.add_argument(nargs='*', action='store', dest='inputs', + help='input filenames (default is stdin)') + + args = parser.parse_args() + print args.__dict__ + +Unless you override it, ‘-h’ and ‘--help’ switches are automatically +added, and produce neatly formatted output: + + -> ./python.exe argparse-example.py --help + usage: argparse-example.py [-h] [-v] [-o FILE] [-C NUM] [inputs [inputs ...]] + + Command-line example. + + positional arguments: + inputs input filenames (default is stdin) + + optional arguments: + -h, --help show this help message and exit + -v produce verbose output + -o FILE direct output to FILE instead of stdout + -C NUM display NUM lines of added context + +As with *note optparse: c3, the command-line switches and arguments are +returned as an object with attributes named by the `dest' parameters: + + -> ./python.exe argparse-example.py -v + {'output': None, + 'is_verbose': True, + 'context': 0, + 'inputs': []} + + -> ./python.exe argparse-example.py -v -o /tmp/output -C 4 file1 file2 + {'output': '/tmp/output', + 'is_verbose': True, + 'context': 4, + 'inputs': ['file1', 'file2']} + +*note argparse: 6. has much fancier validation than *note optparse: c3.; +you can specify an exact number of arguments as an integer, 0 or more +arguments by passing ‘'*'’, 1 or more by passing ‘'+'’, or an optional +argument with ‘'?'’. A top-level parser can contain sub-parsers to +define subcommands that have different sets of switches, as in ‘svn +commit’, ‘svn checkout’, etc. You can specify an argument’s type as +*note FileType: 820, which will automatically open files for you and +understands that ‘'-'’ means standard input or output. + +See also +........ + +*note argparse: 6. documentation + + The documentation page of the argparse module. + +*note Upgrading optparse code: b29. + + Part of the Python documentation, describing how to convert code + that uses *note optparse: c3. + +PEP 389(1) - argparse - New Command Line Parsing Module + + PEP written and implemented by Steven Bethard. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0389 + + +File: python.info, Node: PEP 391 Dictionary-Based Configuration For Logging, Next: PEP 3106 Dictionary Views, Prev: PEP 389 The argparse Module for Parsing Command Lines, Up: What’s New in Python 2 7 + +1.10.7 PEP 391: Dictionary-Based Configuration For Logging +---------------------------------------------------------- + +The *note logging: aa. module is very flexible; applications can define +a tree of logging subsystems, and each logger in this tree can filter +out certain messages, format them differently, and direct messages to a +varying number of handlers. + +All this flexibility can require a lot of configuration. You can write +Python statements to create objects and set their properties, but a +complex set-up requires verbose but boring code. *note logging: aa. +also supports a ‘fileConfig()’ function that parses a file, but the file +format doesn’t support configuring filters, and it’s messier to generate +programmatically. + +Python 2.7 adds a ‘dictConfig()’ function that uses a dictionary to +configure logging. There are many ways to produce a dictionary from +different sources: construct one with code; parse a file containing +JSON; or use a YAML parsing library if one is installed. For more +information see *note Configuration functions: c88. + +The following example configures two loggers, the root logger and a +logger named “network”. Messages sent to the root logger will be sent +to the system log using the syslog protocol, and messages to the +“network” logger will be written to a ‘network.log’ file that will be +rotated once the log reaches 1MB. + + import logging + import logging.config + + configdict = { + 'version': 1, # Configuration schema in use; must be 1 for now + 'formatters': { + 'standard': { + 'format': ('%(asctime)s %(name)-15s ' + '%(levelname)-8s %(message)s')}}, + + 'handlers': {'netlog': {'backupCount': 10, + 'class': 'logging.handlers.RotatingFileHandler', + 'filename': '/logs/network.log', + 'formatter': 'standard', + 'level': 'INFO', + 'maxBytes': 1000000}, + 'syslog': {'class': 'logging.handlers.SysLogHandler', + 'formatter': 'standard', + 'level': 'ERROR'}}, + + # Specify all the subordinate loggers + 'loggers': { + 'network': { + 'handlers': ['netlog'] + } + }, + # Specify properties of the root logger + 'root': { + 'handlers': ['syslog'] + }, + } + + # Set up configuration + logging.config.dictConfig(configdict) + + # As an example, log two error messages + logger = logging.getLogger('/') + logger.error('Database not found') + + netlogger = logging.getLogger('network') + netlogger.error('Connection failed') + +Three smaller enhancements to the *note logging: aa. module, all +implemented by Vinay Sajip, are: + + * The *note SysLogHandler: a1e. class now supports syslogging over + TCP. The constructor has a `socktype' parameter giving the type of + socket to use, either *note socket.SOCK_DGRAM: c89. for UDP or + *note socket.SOCK_STREAM: c8a. for TCP. The default protocol + remains UDP. + + * *note Logger: 3a7. instances gained a *note getChild(): c8b. method + that retrieves a descendant logger using a relative path. For + example, once you retrieve a logger by doing ‘log = + getLogger('app')’, calling ‘log.getChild('network.listen')’ is + equivalent to ‘getLogger('app.network.listen')’. + + * The *note LoggerAdapter: c8c. class gained an ‘isEnabledFor()’ + method that takes a `level' and returns whether the underlying + logger would process a message of that level of importance. + +See also +........ + +PEP 391(1) - Dictionary-Based Configuration For Logging + + PEP written and implemented by Vinay Sajip. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0391 + + +File: python.info, Node: PEP 3106 Dictionary Views, Next: PEP 3137 The memoryview Object, Prev: PEP 391 Dictionary-Based Configuration For Logging, Up: What’s New in Python 2 7 + +1.10.8 PEP 3106: Dictionary Views +--------------------------------- + +The dictionary methods *note keys(): c2a, *note values(): c2c, and *note +items(): c2b. are different in Python 3.x. They return an object called +a `view' instead of a fully materialized list. + +It’s not possible to change the return values of *note keys(): c2a, +*note values(): c2c, and *note items(): c2b. in Python 2.7 because too +much code would break. Instead the 3.x versions were added under the +new names ‘viewkeys()’, ‘viewvalues()’, and ‘viewitems()’. + + >>> d = dict((i*10, chr(65+i)) for i in range(26)) + >>> d + {0: 'A', 130: 'N', 10: 'B', 140: 'O', 20: ..., 250: 'Z'} + >>> d.viewkeys() + dict_keys([0, 130, 10, 140, 20, 150, 30, ..., 250]) + +Views can be iterated over, but the key and item views also behave like +sets. The ‘&’ operator performs intersection, and ‘|’ performs a union: + + >>> d1 = dict((i*10, chr(65+i)) for i in range(26)) + >>> d2 = dict((i**.5, i) for i in range(1000)) + >>> d1.viewkeys() & d2.viewkeys() + set([0.0, 10.0, 20.0, 30.0]) + >>> d1.viewkeys() | range(0, 30) + set([0, 1, 130, 3, 4, 5, 6, ..., 120, 250]) + +The view keeps track of the dictionary and its contents change as the +dictionary is modified: + + >>> vk = d.viewkeys() + >>> vk + dict_keys([0, 130, 10, ..., 250]) + >>> d[260] = '&' + >>> vk + dict_keys([0, 130, 260, 10, ..., 250]) + +However, note that you can’t add or remove keys while you’re iterating +over the view: + + >>> for k in vk: + ... d[k*2] = k + ... + Traceback (most recent call last): + File "", line 1, in + RuntimeError: dictionary changed size during iteration + +You can use the view methods in Python 2.x code, and the 2to3 converter +will change them to the standard *note keys(): c2a, *note values(): c2c, +and *note items(): c2b. methods. + +See also +........ + +PEP 3106(1) - Revamping dict.keys(), .values() and .items() + + PEP written by Guido van Rossum. Backported to 2.7 by Alexandre + Vassalotti; bpo-1967(2). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3106 + + (2) https://bugs.python.org/issue1967 + + +File: python.info, Node: PEP 3137 The memoryview Object, Next: Other Language Changes<9>, Prev: PEP 3106 Dictionary Views, Up: What’s New in Python 2 7 + +1.10.9 PEP 3137: The memoryview Object +-------------------------------------- + +The *note memoryview: 25c. object provides a view of another object’s +memory content that matches the *note bytes: 331. type’s interface. + + >>> import string + >>> m = memoryview(string.letters) + >>> m + + >>> len(m) # Returns length of underlying object + 52 + >>> m[0], m[25], m[26] # Indexing returns one byte + ('a', 'z', 'A') + >>> m2 = m[0:26] # Slicing returns another memoryview + >>> m2 + + +The content of the view can be converted to a string of bytes or a list +of integers: + + >>> m2.tobytes() + 'abcdefghijklmnopqrstuvwxyz' + >>> m2.tolist() + [97, 98, 99, 100, 101, 102, 103, ... 121, 122] + >>> + +*note memoryview: 25c. objects allow modifying the underlying object if +it’s a mutable object. + + >>> m2[0] = 75 + Traceback (most recent call last): + File "", line 1, in + TypeError: cannot modify read-only memory + >>> b = bytearray(string.letters) # Creating a mutable object + >>> b + bytearray(b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ') + >>> mb = memoryview(b) + >>> mb[0] = '*' # Assign to view, changing the bytearray. + >>> b[0:5] # The bytearray has been changed. + bytearray(b'*bcde') + >>> + +See also +........ + +PEP 3137(1) - Immutable Bytes and Mutable Buffer + + PEP written by Guido van Rossum. Implemented by Travis Oliphant, + Antoine Pitrou and others. Backported to 2.7 by Antoine Pitrou; + bpo-2396(2). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3137 + + (2) https://bugs.python.org/issue2396 + + +File: python.info, Node: Other Language Changes<9>, Next: New and Improved Modules, Prev: PEP 3137 The memoryview Object, Up: What’s New in Python 2 7 + +1.10.10 Other Language Changes +------------------------------ + +Some smaller changes made to the core Python language are: + + * The syntax for set literals has been backported from Python 3.x. + Curly brackets are used to surround the contents of the resulting + mutable set; set literals are distinguished from dictionaries by + not containing colons and values. ‘{}’ continues to represent an + empty dictionary; use ‘set()’ for an empty set. + + >>> {1, 2, 3, 4, 5} + set([1, 2, 3, 4, 5]) + >>> set() # empty set + set([]) + >>> {} # empty dict + {} + + Backported by Alexandre Vassalotti; bpo-2335(1). + + * Dictionary and set comprehensions are another feature backported + from 3.x, generalizing list/generator comprehensions to use the + literal syntax for sets and dictionaries. + + >>> {x: x*x for x in range(6)} + {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25} + >>> {('a'*x) for x in range(6)} + set(['', 'a', 'aa', 'aaa', 'aaaa', 'aaaaa']) + + Backported by Alexandre Vassalotti; bpo-2333(2). + + * The *note with: 6e9. statement can now use multiple context + managers in one statement. Context managers are processed from + left to right and each one is treated as beginning a new ‘with’ + statement. This means that: + + with A() as a, B() as b: + ... suite of statements ... + + is equivalent to: + + with A() as a: + with B() as b: + ... suite of statements ... + + The ‘contextlib.nested()’ function provides a very similar + function, so it’s no longer necessary and has been deprecated. + + (Proposed in ‘https://codereview.appspot.com/53094’; implemented by + Georg Brandl.) + + * Conversions between floating-point numbers and strings are now + correctly rounded on most platforms. These conversions occur in + many different places: *note str(): 330. on floats and complex + numbers; the *note float: 187. and *note complex: 189. + constructors; numeric formatting; serializing and deserializing + floats and complex numbers using the *note marshal: b0, *note + pickle: ca. and *note json: a4. modules; parsing of float and + imaginary literals in Python code; and *note Decimal: 188.-to-float + conversion. + + Related to this, the *note repr(): 7cc. of a floating-point number + `x' now returns a result based on the shortest decimal string + that’s guaranteed to round back to `x' under correct rounding (with + round-half-to-even rounding mode). Previously it gave a string + based on rounding x to 17 decimal digits. + + The rounding library responsible for this improvement works on + Windows and on Unix platforms using the gcc, icc, or suncc + compilers. There may be a small number of platforms where correct + operation of this code cannot be guaranteed, so the code is not + used on such systems. You can find out which code is being used by + checking *note sys.float_repr_style: c90, which will be ‘short’ if + the new code is in use and ‘legacy’ if it isn’t. + + Implemented by Eric Smith and Mark Dickinson, using David Gay’s + ‘dtoa.c’ library; bpo-7117(3). + + * Conversions from long integers and regular integers to floating + point now round differently, returning the floating-point number + closest to the number. This doesn’t matter for small integers that + can be converted exactly, but for large numbers that will + unavoidably lose precision, Python 2.7 now approximates more + closely. For example, Python 2.6 computed the following: + + >>> n = 295147905179352891391 + >>> float(n) + 2.9514790517935283e+20 + >>> n - long(float(n)) + 65535L + + Python 2.7’s floating-point result is larger, but much closer to + the true value: + + >>> n = 295147905179352891391 + >>> float(n) + 2.9514790517935289e+20 + >>> n - long(float(n)) + -1L + + (Implemented by Mark Dickinson; bpo-3166(4).) + + Integer division is also more accurate in its rounding behaviours. + (Also implemented by Mark Dickinson; bpo-1811(5).) + + * Implicit coercion for complex numbers has been removed; the + interpreter will no longer ever attempt to call a ‘__coerce__()’ + method on complex objects. (Removed by Meador Inge and Mark + Dickinson; bpo-5211(6).) + + * The *note str.format(): 4da. method now supports automatic + numbering of the replacement fields. This makes using *note + str.format(): 4da. more closely resemble using ‘%s’ formatting: + + >>> '{}:{}:{}'.format(2009, 04, 'Sunday') + '2009:4:Sunday' + >>> '{}:{}:{day}'.format(2009, 4, day='Sunday') + '2009:4:Sunday' + + The auto-numbering takes the fields from left to right, so the + first ‘{...}’ specifier will use the first argument to *note + str.format(): 4da, the next specifier will use the next argument, + and so on. You can’t mix auto-numbering and explicit numbering – + either number all of your specifier fields or none of them – but + you can mix auto-numbering and named fields, as in the second + example above. (Contributed by Eric Smith; bpo-5237(7).) + + Complex numbers now correctly support usage with *note format(): + 4db, and default to being right-aligned. Specifying a precision or + comma-separation applies to both the real and imaginary parts of + the number, but a specified field width and alignment is applied to + the whole of the resulting ‘1.5+3j’ output. (Contributed by Eric + Smith; bpo-1588(8) and bpo-7988(9).) + + The ‘F’ format code now always formats its output using uppercase + characters, so it will now produce ‘INF’ and ‘NAN’. (Contributed + by Eric Smith; bpo-3382(10).) + + A low-level change: the *note object.__format__(): 94e. method now + triggers a *note PendingDeprecationWarning: 279. if it’s passed a + format string, because the *note __format__(): 94e. method for + *note object: 2b0. converts the object to a string representation + and formats that. Previously the method silently applied the + format string to the string representation, but that could hide + mistakes in Python code. If you’re supplying formatting + information such as an alignment or precision, presumably you’re + expecting the formatting to be applied in some object-specific way. + (Fixed by Eric Smith; bpo-7994(11).) + + * The *note int(): 184. and ‘long()’ types gained a ‘bit_length’ + method that returns the number of bits necessary to represent its + argument in binary: + + >>> n = 37 + >>> bin(n) + '0b100101' + >>> n.bit_length() + 6 + >>> n = 2**123-1 + >>> n.bit_length() + 123 + >>> (n+1).bit_length() + 124 + + (Contributed by Fredrik Johansson and Victor Stinner; + bpo-3439(12).) + + * The *note import: c1d. statement will no longer try an absolute + import if a relative import (e.g. ‘from .os import sep’) fails. + This fixes a bug, but could possibly break certain ‘import’ + statements that were only working by accident. (Fixed by Meador + Inge; bpo-7902(13).) + + * It’s now possible for a subclass of the built-in ‘unicode’ type to + override the ‘__unicode__()’ method. (Implemented by Victor + Stinner; bpo-1583863(14).) + + * The *note bytearray: 332. type’s *note translate(): c91. method now + accepts ‘None’ as its first argument. (Fixed by Georg Brandl; + bpo-4759(15).) + + * When using ‘@classmethod’ and ‘@staticmethod’ to wrap methods as + class or static methods, the wrapper object now exposes the wrapped + function as their ‘__func__’ attribute. (Contributed by Amaury + Forgeot d’Arc, after a suggestion by George Sakkis; bpo-5982(16).) + + * When a restricted set of attributes were set using ‘__slots__’, + deleting an unset attribute would not raise *note AttributeError: + 39b. as you would expect. Fixed by Benjamin Peterson; + bpo-7604(17).) + + * Two new encodings are now supported: “cp720”, used primarily for + Arabic text; and “cp858”, a variant of CP 850 that adds the euro + symbol. (CP720 contributed by Alexander Belchenko and Amaury + Forgeot d’Arc in bpo-1616979(18); CP858 contributed by Tim Hatch in + bpo-8016(19).) + + * The ‘file’ object will now set the ‘filename’ attribute on the + *note IOError: 992. exception when trying to open a directory on + POSIX platforms (noted by Jan Kaliszewski; bpo-4764(20)), and now + explicitly checks for and forbids writing to read-only file objects + instead of trusting the C library to catch and report the error + (fixed by Stefan Krah; bpo-5677(21)). + + * The Python tokenizer now translates line endings itself, so the + *note compile(): 1b4. built-in function now accepts code using any + line-ending convention. Additionally, it no longer requires that + the code end in a newline. + + * Extra parentheses in function definitions are illegal in Python + 3.x, meaning that you get a syntax error from ‘def f((x)): pass’. + In Python3-warning mode, Python 2.7 will now warn about this odd + usage. (Noted by James Lingard; bpo-7362(22).) + + * It’s now possible to create weak references to old-style class + objects. New-style classes were always weak-referenceable. (Fixed + by Antoine Pitrou; bpo-8268(23).) + + * When a module object is garbage-collected, the module’s dictionary + is now only cleared if no one else is holding a reference to the + dictionary (bpo-7140(24)). + +* Menu: + +* Interpreter Changes:: +* Optimizations: Optimizations<8>. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2335 + + (2) https://bugs.python.org/issue2333 + + (3) https://bugs.python.org/issue7117 + + (4) https://bugs.python.org/issue3166 + + (5) https://bugs.python.org/issue1811 + + (6) https://bugs.python.org/issue5211 + + (7) https://bugs.python.org/issue5237 + + (8) https://bugs.python.org/issue1588 + + (9) https://bugs.python.org/issue7988 + + (10) https://bugs.python.org/issue3382 + + (11) https://bugs.python.org/issue7994 + + (12) https://bugs.python.org/issue3439 + + (13) https://bugs.python.org/issue7902 + + (14) https://bugs.python.org/issue1583863 + + (15) https://bugs.python.org/issue4759 + + (16) https://bugs.python.org/issue5982 + + (17) https://bugs.python.org/issue7604 + + (18) https://bugs.python.org/issue1616979 + + (19) https://bugs.python.org/issue8016 + + (20) https://bugs.python.org/issue4764 + + (21) https://bugs.python.org/issue5677 + + (22) https://bugs.python.org/issue7362 + + (23) https://bugs.python.org/issue8268 + + (24) https://bugs.python.org/issue7140 + + +File: python.info, Node: Interpreter Changes, Next: Optimizations<8>, Up: Other Language Changes<9> + +1.10.10.1 Interpreter Changes +............................. + +A new environment variable, *note PYTHONWARNINGS: 418, allows +controlling warnings. It should be set to a string containing warning +settings, equivalent to those used with the *note -W: 417. switch, +separated by commas. (Contributed by Brian Curtin; bpo-7301(1).) + +For example, the following setting will print warnings every time they +occur, but turn warnings from the ‘Cookie’ module into an error. (The +exact syntax for setting an environment variable varies across operating +systems and shells.) + + export PYTHONWARNINGS=all,error:::Cookie:0 + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue7301 + + +File: python.info, Node: Optimizations<8>, Prev: Interpreter Changes, Up: Other Language Changes<9> + +1.10.10.2 Optimizations +....................... + +Several performance enhancements have been added: + + * A new opcode was added to perform the initial setup for *note with: + 6e9. statements, looking up the *note __enter__(): c95. and *note + __exit__(): c96. methods. (Contributed by Benjamin Peterson.) + + * The garbage collector now performs better for one common usage + pattern: when many objects are being allocated without deallocating + any of them. This would previously take quadratic time for garbage + collection, but now the number of full garbage collections is + reduced as the number of objects on the heap grows. The new logic + only performs a full garbage collection pass when the middle + generation has been collected 10 times and when the number of + survivor objects from the middle generation exceeds 10% of the + number of objects in the oldest generation. (Suggested by Martin + von Löwis and implemented by Antoine Pitrou; bpo-4074(1).) + + * The garbage collector tries to avoid tracking simple containers + which can’t be part of a cycle. In Python 2.7, this is now true + for tuples and dicts containing atomic types (such as ints, + strings, etc.). Transitively, a dict containing tuples of atomic + types won’t be tracked either. This helps reduce the cost of each + garbage collection by decreasing the number of objects to be + considered and traversed by the collector. (Contributed by Antoine + Pitrou; bpo-4688(2).) + + * Long integers are now stored internally either in base 2**15 or in + base 2**30, the base being determined at build time. Previously, + they were always stored in base 2**15. Using base 2**30 gives + significant performance improvements on 64-bit machines, but + benchmark results on 32-bit machines have been mixed. Therefore, + the default is to use base 2**30 on 64-bit machines and base 2**15 + on 32-bit machines; on Unix, there’s a new configure option + ‘--enable-big-digits’ that can be used to override this default. + + Apart from the performance improvements this change should be + invisible to end users, with one exception: for testing and + debugging purposes there’s a new structseq ‘sys.long_info’ that + provides information about the internal format, giving the number + of bits per digit and the size in bytes of the C type used to store + each digit: + + >>> import sys + >>> sys.long_info + sys.long_info(bits_per_digit=30, sizeof_digit=4) + + (Contributed by Mark Dickinson; bpo-4258(3).) + + Another set of changes made long objects a few bytes smaller: 2 + bytes smaller on 32-bit systems and 6 bytes on 64-bit. + (Contributed by Mark Dickinson; bpo-5260(4).) + + * The division algorithm for long integers has been made faster by + tightening the inner loop, doing shifts instead of multiplications, + and fixing an unnecessary extra iteration. Various benchmarks show + speedups of between 50% and 150% for long integer divisions and + modulo operations. (Contributed by Mark Dickinson; bpo-5512(5).) + Bitwise operations are also significantly faster (initial patch by + Gregory Smith; bpo-1087418(6)). + + * The implementation of ‘%’ checks for the left-side operand being a + Python string and special-cases it; this results in a 1–3% + performance increase for applications that frequently use ‘%’ with + strings, such as templating libraries. (Implemented by Collin + Winter; bpo-5176(7).) + + * List comprehensions with an ‘if’ condition are compiled into faster + bytecode. (Patch by Antoine Pitrou, back-ported to 2.7 by Jeffrey + Yasskin; bpo-4715(8).) + + * Converting an integer or long integer to a decimal string was made + faster by special-casing base 10 instead of using a generalized + conversion function that supports arbitrary bases. (Patch by + Gawain Bolton; bpo-6713(9).) + + * The ‘split()’, ‘replace()’, ‘rindex()’, ‘rpartition()’, and + ‘rsplit()’ methods of string-like types (strings, Unicode strings, + and *note bytearray: 332. objects) now use a fast reverse-search + algorithm instead of a character-by-character scan. This is + sometimes faster by a factor of 10. (Added by Florent Xicluna; + bpo-7462(10) and bpo-7622(11).) + + * The *note pickle: ca. and ‘cPickle’ modules now automatically + intern the strings used for attribute names, reducing memory usage + of the objects resulting from unpickling. (Contributed by Jake + McGuire; bpo-5084(12).) + + * The ‘cPickle’ module now special-cases dictionaries, nearly halving + the time required to pickle them. (Contributed by Collin Winter; + bpo-5670(13).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4074 + + (2) https://bugs.python.org/issue4688 + + (3) https://bugs.python.org/issue4258 + + (4) https://bugs.python.org/issue5260 + + (5) https://bugs.python.org/issue5512 + + (6) https://bugs.python.org/issue1087418 + + (7) https://bugs.python.org/issue5176 + + (8) https://bugs.python.org/issue4715 + + (9) https://bugs.python.org/issue6713 + + (10) https://bugs.python.org/issue7462 + + (11) https://bugs.python.org/issue7622 + + (12) https://bugs.python.org/issue5084 + + (13) https://bugs.python.org/issue5670 + + +File: python.info, Node: New and Improved Modules, Next: Build and C API Changes<8>, Prev: Other Language Changes<9>, Up: What’s New in Python 2 7 + +1.10.11 New and Improved Modules +-------------------------------- + +As in every release, Python’s standard library received a number of +enhancements and bug fixes. Here’s a partial list of the most notable +changes, sorted alphabetically by module name. Consult the ‘Misc/NEWS’ +file in the source tree for a more complete list of changes, or look +through the Subversion logs for all the details. + + * The *note bdb: f. module’s base debugging class *note Bdb: c98. + gained a feature for skipping modules. The constructor now takes + an iterable containing glob-style patterns such as ‘django.*’; the + debugger will not step into stack frames from a module that matches + one of these patterns. (Contributed by Maru Newby after a + suggestion by Senthil Kumaran; bpo-5142(1).) + + * The *note binascii: 10. module now supports the buffer API, so it + can be used with *note memoryview: 25c. instances and other similar + buffer objects. (Backported from 3.x by Florent Xicluna; + bpo-7703(2).) + + * Updated module: the ‘bsddb’ module has been updated from + 4.7.2devel9 to version 4.8.4 of the pybsddb package(3). The new + version features better Python 3.x compatibility, various bug + fixes, and adds several new BerkeleyDB flags and methods. (Updated + by Jesús Cea Avión; bpo-8156(4). The pybsddb changelog can be read + at ‘http://hg.jcea.es/pybsddb/file/tip/ChangeLog’.) + + * The *note bz2: 14. module’s *note BZ2File: 931. now supports the + context management protocol, so you can write ‘with + bz2.BZ2File(...) as f:’. (Contributed by Hagen Fürstenau; + bpo-3860(5).) + + * New class: the *note Counter: 9da. class in the *note collections: + 1e. module is useful for tallying data. *note Counter: 9da. + instances behave mostly like dictionaries but return zero for + missing keys instead of raising a *note KeyError: 2c7.: + + >>> from collections import Counter + >>> c = Counter() + >>> for letter in 'here is a sample of english text': + ... c[letter] += 1 + ... + >>> c + Counter({' ': 6, 'e': 5, 's': 3, 'a': 2, 'i': 2, 'h': 2, + 'l': 2, 't': 2, 'g': 1, 'f': 1, 'm': 1, 'o': 1, 'n': 1, + 'p': 1, 'r': 1, 'x': 1}) + >>> c['e'] + 5 + >>> c['z'] + 0 + + There are three additional *note Counter: 9da. methods. *note + most_common(): c99. returns the N most common elements and their + counts. *note elements(): c9a. returns an iterator over the + contained elements, repeating each element as many times as its + count. *note subtract(): b5a. takes an iterable and subtracts one + for each element instead of adding; if the argument is a dictionary + or another ‘Counter’, the counts are subtracted. + + >>> c.most_common(5) + [(' ', 6), ('e', 5), ('s', 3), ('a', 2), ('i', 2)] + >>> c.elements() -> + 'a', 'a', ' ', ' ', ' ', ' ', ' ', ' ', + 'e', 'e', 'e', 'e', 'e', 'g', 'f', 'i', 'i', + 'h', 'h', 'm', 'l', 'l', 'o', 'n', 'p', 's', + 's', 's', 'r', 't', 't', 'x' + >>> c['e'] + 5 + >>> c.subtract('very heavy on the letter e') + >>> c['e'] # Count is now lower + -1 + + Contributed by Raymond Hettinger; bpo-1696199(6). + + New class: *note OrderedDict: 1b9. is described in the earlier + section *note PEP 372; Adding an Ordered Dictionary to collections: + c80. + + New method: The *note deque: 531. data type now has a *note + count(): b5c. method that returns the number of contained elements + equal to the supplied argument `x', and a *note reverse(): b5d. + method that reverses the elements of the deque in-place. *note + deque: 531. also exposes its maximum length as the read-only *note + maxlen: c9b. attribute. (Both features added by Raymond + Hettinger.) + + The *note namedtuple: 1b7. class now has an optional `rename' + parameter. If `rename' is true, field names that are invalid + because they’ve been repeated or aren’t legal Python identifiers + will be renamed to legal names that are derived from the field’s + position within the list of fields: + + >>> from collections import namedtuple + >>> T = namedtuple('T', ['field1', '$illegal', 'for', 'field2'], rename=True) + >>> T._fields + ('field1', '_1', '_2', 'field2') + + (Added by Raymond Hettinger; bpo-1818(7).) + + Finally, the ‘Mapping’ abstract base class now returns *note + NotImplemented: 84c. if a mapping is compared to another type that + isn’t a ‘Mapping’. (Fixed by Daniel Stutzbach; bpo-8729(8).) + + * Constructors for the parsing classes in the ‘ConfigParser’ module + now take an `allow_no_value' parameter, defaulting to false; if + true, options without values will be allowed. For example: + + >>> import ConfigParser, StringIO + >>> sample_config = """ + ... [mysqld] + ... user = mysql + ... pid-file = /var/run/mysqld/mysqld.pid + ... skip-bdb + ... """ + >>> config = ConfigParser.RawConfigParser(allow_no_value=True) + >>> config.readfp(StringIO.StringIO(sample_config)) + >>> config.get('mysqld', 'user') + 'mysql' + >>> print config.get('mysqld', 'skip-bdb') + None + >>> print config.get('mysqld', 'unknown') + Traceback (most recent call last): + ... + NoOptionError: No option 'unknown' in section: 'mysqld' + + (Contributed by Mats Kindahl; bpo-7005(9).) + + * Deprecated function: ‘contextlib.nested()’, which allows handling + more than one context manager with a single *note with: 6e9. + statement, has been deprecated, because the ‘with’ statement now + supports multiple context managers. + + * The ‘cookielib’ module now ignores cookies that have an invalid + version field, one that doesn’t contain an integer value. (Fixed + by John J. Lee; bpo-3924(10).) + + * The *note copy: 26. module’s *note deepcopy(): 3cc. function will + now correctly copy bound instance methods. (Implemented by Robert + Collins; bpo-1515(11).) + + * The *note ctypes: 2b. module now always converts ‘None’ to a C + ‘NULL’ pointer for arguments declared as pointers. (Changed by + Thomas Heller; bpo-4606(12).) The underlying libffi library(13) + has been updated to version 3.0.9, containing various fixes for + different platforms. (Updated by Matthias Klose; bpo-8142(14).) + + * New method: the *note datetime: 31. module’s *note timedelta: 195. + class gained a *note total_seconds(): c9c. method that returns the + number of seconds in the duration. (Contributed by Brian Quinlan; + bpo-5788(15).) + + * New method: the *note Decimal: 188. class gained a *note + from_float(): b80. class method that performs an exact conversion + of a floating-point number to a *note Decimal: 188. This exact + conversion strives for the closest decimal approximation to the + floating-point representation’s value; the resulting decimal value + will therefore still include the inaccuracy, if any. For example, + ‘Decimal.from_float(0.1)’ returns + ‘Decimal('0.1000000000000000055511151231257827021181583404541015625')’. + (Implemented by Raymond Hettinger; bpo-4796(16).) + + Comparing instances of *note Decimal: 188. with floating-point + numbers now produces sensible results based on the numeric values + of the operands. Previously such comparisons would fall back to + Python’s default rules for comparing objects, which produced + arbitrary results based on their type. Note that you still cannot + combine ‘Decimal’ and floating-point in other operations such as + addition, since you should be explicitly choosing how to convert + between float and *note Decimal: 188. (Fixed by Mark Dickinson; + bpo-2531(17).) + + The constructor for *note Decimal: 188. now accepts floating-point + numbers (added by Raymond Hettinger; bpo-8257(18)) and non-European + Unicode characters such as Arabic-Indic digits (contributed by Mark + Dickinson; bpo-6595(19)). + + Most of the methods of the *note Context: 9f0. class now accept + integers as well as *note Decimal: 188. instances; the only + exceptions are the *note canonical(): c9d. and *note + is_canonical(): c9e. methods. (Patch by Juan José Conti; + bpo-7633(20).) + + When using *note Decimal: 188. instances with a string’s *note + format(): 4da. method, the default alignment was previously + left-alignment. This has been changed to right-alignment, which is + more sensible for numeric types. (Changed by Mark Dickinson; + bpo-6857(21).) + + Comparisons involving a signaling NaN value (or ‘sNAN’) now signal + ‘InvalidOperation’ instead of silently returning a true or false + value depending on the comparison operator. Quiet NaN values (or + ‘NaN’) are now hashable. (Fixed by Mark Dickinson; bpo-7279(22).) + + * The *note difflib: 37. module now produces output that is more + compatible with modern ‘diff’/‘patch’ tools through one small + change, using a tab character instead of spaces as a separator in + the header giving the filename. (Fixed by Anatoly Techtonik; + bpo-7585(23).) + + * The Distutils ‘sdist’ command now always regenerates the ‘MANIFEST’ + file, since even if the ‘MANIFEST.in’ or ‘setup.py’ files haven’t + been modified, the user might have created some new files that + should be included. (Fixed by Tarek Ziadé; bpo-8688(24).) + + * The *note doctest: 67. module’s ‘IGNORE_EXCEPTION_DETAIL’ flag will + now ignore the name of the module containing the exception being + tested. (Patch by Lennart Regebro; bpo-7490(25).) + + * The *note email: 69. module’s *note Message: 541. class will now + accept a Unicode-valued payload, automatically converting the + payload to the encoding specified by ‘output_charset’. (Added by + R. David Murray; bpo-1368247(26).) + + * The *note Fraction: 185. class now accepts a single float or *note + Decimal: 188. instance, or two rational numbers, as arguments to + its constructor. (Implemented by Mark Dickinson; rationals added + in bpo-5812(27), and float/decimal in bpo-8294(28).) + + Ordering comparisons (‘<’, ‘<=’, ‘>’, ‘>=’) between fractions and + complex numbers now raise a *note TypeError: 192. This fixes an + oversight, making the *note Fraction: 185. match the other numeric + types. + + * New class: *note FTP_TLS: a03. in the *note ftplib: 84. module + provides secure FTP connections using TLS encapsulation of + authentication as well as subsequent control and data transfers. + (Contributed by Giampaolo Rodola; bpo-2054(29).) + + The *note storbinary(): c9f. method for binary uploads can now + restart uploads thanks to an added `rest' parameter (patch by Pablo + Mouzo; bpo-6845(30).) + + * New class decorator: *note total_ordering(): 84b. in the *note + functools: 85. module takes a class that defines an *note __eq__(): + 33f. method and one of *note __lt__(): c34, *note __le__(): ca0, + *note __gt__(): ca1, or *note __ge__(): ca2, and generates the + missing comparison methods. Since the ‘__cmp__()’ method is being + deprecated in Python 3.x, this decorator makes it easier to define + ordered classes. (Added by Raymond Hettinger; bpo-5479(31).) + + New function: *note cmp_to_key(): b56. will take an old-style + comparison function that expects two arguments and return a new + callable that can be used as the `key' parameter to functions such + as *note sorted(): 444, *note min(): 809. and *note max(): 80a, + etc. The primary intended use is to help with making code + compatible with Python 3.x. (Added by Raymond Hettinger.) + + * New function: the *note gc: 86. module’s *note is_tracked(): ca3. + returns true if a given instance is tracked by the garbage + collector, false otherwise. (Contributed by Antoine Pitrou; + bpo-4688(32).) + + * The *note gzip: 8c. module’s *note GzipFile: 6dd. now supports the + context management protocol, so you can write ‘with + gzip.GzipFile(...) as f:’ (contributed by Hagen Fürstenau; + bpo-3860(33)), and it now implements the *note io.BufferedIOBase: + 588. ABC, so you can wrap it with *note io.BufferedReader: b8a. for + faster processing (contributed by Nir Aides; bpo-7471(34)). It’s + also now possible to override the modification time recorded in a + gzipped file by providing an optional timestamp to the constructor. + (Contributed by Jacques Frechet; bpo-4272(35).) + + Files in gzip format can be padded with trailing zero bytes; the + *note gzip: 8c. module will now consume these trailing bytes. + (Fixed by Tadek Pietraszek and Brian Curtin; bpo-2846(36).) + + * New attribute: the *note hashlib: 8d. module now has an + ‘algorithms’ attribute containing a tuple naming the supported + algorithms. In Python 2.7, ‘hashlib.algorithms’ contains ‘('md5', + 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')’. (Contributed by + Carl Chenet; bpo-7418(37).) + + * The default ‘HTTPResponse’ class used by the ‘httplib’ module now + supports buffering, resulting in much faster reading of HTTP + responses. (Contributed by Kristján Valur Jónsson; bpo-4879(38).) + + The ‘HTTPConnection’ and ‘HTTPSConnection’ classes now support a + `source_address' parameter, a ‘(host, port)’ 2-tuple giving the + source address that will be used for the connection. (Contributed + by Eldon Ziegler; bpo-3972(39).) + + * The ‘ihooks’ module now supports relative imports. Note that + ‘ihooks’ is an older module for customizing imports, superseded by + the ‘imputil’ module added in Python 2.0. (Relative import support + added by Neil Schemenauer.) + + * The *note imaplib: 98. module now supports IPv6 addresses. + (Contributed by Derek Morr; bpo-1655(40).) + + * New function: the *note inspect: a0. module’s *note getcallargs(): + 7b6. takes a callable and its positional and keyword arguments, and + figures out which of the callable’s parameters will receive each + argument, returning a dictionary mapping argument names to their + values. For example: + + >>> from inspect import getcallargs + >>> def f(a, b=1, *pos, **named): + ... pass + >>> getcallargs(f, 1, 2, 3) + {'a': 1, 'b': 2, 'pos': (3,), 'named': {}} + >>> getcallargs(f, a=2, x=4) + {'a': 2, 'b': 1, 'pos': (), 'named': {'x': 4}} + >>> getcallargs(f) + Traceback (most recent call last): + ... + TypeError: f() takes at least 1 argument (0 given) + + Contributed by George Sakkis; bpo-3135(41). + + * Updated module: The *note io: a1. library has been upgraded to the + version shipped with Python 3.1. For 3.1, the I/O library was + entirely rewritten in C and is 2 to 20 times faster depending on + the task being performed. The original Python version was renamed + to the ‘_pyio’ module. + + One minor resulting change: the *note io.TextIOBase: c39. class now + has an ‘errors’ attribute giving the error setting used for + encoding and decoding errors (one of ‘'strict'’, ‘'replace'’, + ‘'ignore'’). + + The *note io.FileIO: ca4. class now raises an *note OSError: 1d3. + when passed an invalid file descriptor. (Implemented by Benjamin + Peterson; bpo-4991(42).) The *note truncate(): ca5. method now + preserves the file position; previously it would change the file + position to the end of the new file. (Fixed by Pascal Chambon; + bpo-6939(43).) + + * New function: ‘itertools.compress(data, selectors)’ takes two + iterators. Elements of `data' are returned if the corresponding + value in `selectors' is true: + + itertools.compress('ABCDEF', [1,0,1,0,1,1]) => + A, C, E, F + + New function: ‘itertools.combinations_with_replacement(iter, r)’ + returns all the possible `r'-length combinations of elements from + the iterable `iter'. Unlike *note combinations(): ca6, individual + elements can be repeated in the generated combinations: + + itertools.combinations_with_replacement('abc', 2) => + ('a', 'a'), ('a', 'b'), ('a', 'c'), + ('b', 'b'), ('b', 'c'), ('c', 'c') + + Note that elements are treated as unique depending on their + position in the input, not their actual values. + + The *note itertools.count(): c1b. function now has a `step' + argument that allows incrementing by values other than 1. *note + count(): c1b. also now allows keyword arguments, and using + non-integer values such as floats or *note Decimal: 188. instances. + (Implemented by Raymond Hettinger; bpo-5032(44).) + + *note itertools.combinations(): ca6. and *note itertools.product(): + ca7. previously raised *note ValueError: 1fb. for values of `r' + larger than the input iterable. This was deemed a specification + error, so they now return an empty iterator. (Fixed by Raymond + Hettinger; bpo-4816(45).) + + * Updated module: The *note json: a4. module was upgraded to version + 2.0.9 of the simplejson package, which includes a C extension that + makes encoding and decoding faster. (Contributed by Bob Ippolito; + bpo-4136(46).) + + To support the new *note collections.OrderedDict: 1b9. type, *note + json.load(): 55f. now has an optional `object_pairs_hook' parameter + that will be called with any object literal that decodes to a list + of pairs. (Contributed by Raymond Hettinger; bpo-5381(47).) + + * The *note mailbox: ae. module’s *note Maildir: ca8. class now + records the timestamp on the directories it reads, and only + re-reads them if the modification time has subsequently changed. + This improves performance by avoiding unneeded directory scans. + (Fixed by A.M. Kuchling and Antoine Pitrou; bpo-1607951(48), + bpo-6896(49).) + + * New functions: the *note math: b1. module gained *note erf(): 452. + and *note erfc(): 453. for the error function and the complementary + error function, *note expm1(): b68. which computes ‘e**x - 1’ with + more precision than using *note exp(): ca9. and subtracting 1, + *note gamma(): b69. for the Gamma function, and *note lgamma(): + b6a. for the natural log of the Gamma function. (Contributed by + Mark Dickinson and nirinA raseliarison; bpo-3366(50).) + + * The *note multiprocessing: b7. module’s ‘Manager*’ classes can now + be passed a callable that will be called whenever a subprocess is + started, along with a set of arguments that will be passed to the + callable. (Contributed by lekma; bpo-5585(51).) + + The ‘Pool’ class, which controls a pool of worker processes, now + has an optional `maxtasksperchild' parameter. Worker processes + will perform the specified number of tasks and then exit, causing + the ‘Pool’ to start a new worker. This is useful if tasks may leak + memory or other resources, or if some tasks will cause the worker + to become very large. (Contributed by Charles Cazabon; + bpo-6963(52).) + + * The *note nntplib: c0. module now supports IPv6 addresses. + (Contributed by Derek Morr; bpo-1664(53).) + + * New functions: the *note os: c4. module wraps the following POSIX + system calls: *note getresgid(): caa. and *note getresuid(): cab, + which return the real, effective, and saved GIDs and UIDs; *note + setresgid(): cac. and *note setresuid(): cad, which set real, + effective, and saved GIDs and UIDs to new values; *note + initgroups(): cae, which initialize the group access list for the + current process. (GID/UID functions contributed by Travis H.; + bpo-6508(54). Support for initgroups added by Jean-Paul Calderone; + bpo-7333(55).) + + The *note os.fork(): 961. function now re-initializes the import + lock in the child process; this fixes problems on Solaris when + *note fork(): 961. is called from a thread. (Fixed by Zsolt + Cserna; bpo-7242(56).) + + * In the *note os.path: c5. module, the *note normpath(): caf. and + *note abspath(): cb0. functions now preserve Unicode; if their + input path is a Unicode string, the return value is also a Unicode + string. (*note normpath(): caf. fixed by Matt Giuca in + bpo-5827(57); *note abspath(): cb0. fixed by Ezio Melotti in + bpo-3426(58).) + + * The *note pydoc: d9. module now has help for the various symbols + that Python uses. You can now do ‘help('<<')’ or ‘help('@')’, for + example. (Contributed by David Laban; bpo-4739(59).) + + * The *note re: dd. module’s *note split(): 3ca, *note sub(): 47b, + and *note subn(): 734. now accept an optional `flags' argument, for + consistency with the other functions in the module. (Added by + Gregory P. Smith.) + + * New function: *note run_path(): cb1. in the *note runpy: e2. module + will execute the code at a provided `path' argument. `path' can be + the path of a Python source file (‘example.py’), a compiled + bytecode file (‘example.pyc’), a directory (‘./package/’), or a zip + archive (‘example.zip’). If a directory or zip path is provided, + it will be added to the front of ‘sys.path’ and the module *note + __main__: 1. will be imported. It’s expected that the directory or + zip contains a ‘__main__.py’; if it doesn’t, some other + ‘__main__.py’ might be imported from a location later in + ‘sys.path’. This makes more of the machinery of *note runpy: e2. + available to scripts that want to mimic the way Python’s command + line processes an explicit path name. (Added by Nick Coghlan; + bpo-6816(60).) + + * New function: in the *note shutil: e9. module, *note + make_archive(): 21b. takes a filename, archive type (zip or + tar-format), and a directory path, and creates an archive + containing the directory’s contents. (Added by Tarek Ziadé.) + + *note shutil: e9.’s *note copyfile(): 258. and *note copytree(): + 21a. functions now raise a ‘SpecialFileError’ exception when asked + to copy a named pipe. Previously the code would treat named pipes + like a regular file by opening them for reading, and this would + block indefinitely. (Fixed by Antoine Pitrou; bpo-3002(61).) + + * The *note signal: ea. module no longer re-installs the signal + handler unless this is truly necessary, which fixes a bug that + could make it impossible to catch the EINTR signal robustly. + (Fixed by Charles-Francois Natali; bpo-8354(62).) + + * New functions: in the *note site: eb. module, three new functions + return various site- and user-specific paths. *note + getsitepackages(): bd3. returns a list containing all global + site-packages directories, *note getusersitepackages(): bd5. + returns the path of the user’s site-packages directory, and *note + getuserbase(): bd4. returns the value of the ‘USER_BASE’ + environment variable, giving the path to a directory that can be + used to store data. (Contributed by Tarek Ziadé; bpo-6693(63).) + + The *note site: eb. module now reports exceptions occurring when + the ‘sitecustomize’ module is imported, and will no longer catch + and swallow the *note KeyboardInterrupt: 197. exception. (Fixed by + Victor Stinner; bpo-3137(64).) + + * The *note create_connection(): b9e. function gained a + `source_address' parameter, a ‘(host, port)’ 2-tuple giving the + source address that will be used for the connection. (Contributed + by Eldon Ziegler; bpo-3972(65).) + + The *note recv_into(): cb2. and *note recvfrom_into(): cb3. methods + will now write into objects that support the buffer API, most + usefully the *note bytearray: 332. and *note memoryview: 25c. + objects. (Implemented by Antoine Pitrou; bpo-8104(66).) + + * The ‘SocketServer’ module’s ‘TCPServer’ class now supports socket + timeouts and disabling the Nagle algorithm. The + ‘disable_nagle_algorithm’ class attribute defaults to ‘False’; if + overridden to be true, new request connections will have the + TCP_NODELAY option set to prevent buffering many small sends into a + single TCP packet. The ‘timeout’ class attribute can hold a + timeout in seconds that will be applied to the request socket; if + no request is received within that time, ‘handle_timeout()’ will be + called and ‘handle_request()’ will return. (Contributed by + Kristján Valur Jónsson; bpo-6192(67) and bpo-6267(68).) + + * Updated module: the *note sqlite3: f2. module has been updated to + version 2.6.0 of the pysqlite package(69). Version 2.6.0 includes + a number of bugfixes, and adds the ability to load SQLite + extensions from shared libraries. Call the + ‘enable_load_extension(True)’ method to enable extensions, and then + call *note load_extension(): b9a. to load a particular shared + library. (Updated by Gerhard Häring.) + + * The *note ssl: f3. module’s *note SSLSocket: 3e2. objects now + support the buffer API, which fixed a test suite failure (fix by + Antoine Pitrou; bpo-7133(70)) and automatically set OpenSSL’s + ‘SSL_MODE_AUTO_RETRY’, which will prevent an error code being + returned from ‘recv()’ operations that trigger an SSL renegotiation + (fix by Antoine Pitrou; bpo-8222(71)). + + The *note ssl.wrap_socket(): 46f. constructor function now takes a + `ciphers' argument that’s a string listing the encryption + algorithms to be allowed; the format of the string is described in + the OpenSSL documentation(72). (Added by Antoine Pitrou; + bpo-8322(73).) + + Another change makes the extension load all of OpenSSL’s ciphers + and digest algorithms so that they’re all available. Some SSL + certificates couldn’t be verified, reporting an “unknown algorithm” + error. (Reported by Beda Kosata, and fixed by Antoine Pitrou; + bpo-8484(74).) + + The version of OpenSSL being used is now available as the module + attributes *note ssl.OPENSSL_VERSION: ba1. (a string), *note + ssl.OPENSSL_VERSION_INFO: ba2. (a 5-tuple), and *note + ssl.OPENSSL_VERSION_NUMBER: ba3. (an integer). (Added by Antoine + Pitrou; bpo-8321(75).) + + * The *note struct: f8. module will no longer silently ignore + overflow errors when a value is too large for a particular integer + format code (one of ‘bBhHiIlLqQ’); it now always raises a *note + struct.error: cb4. exception. (Changed by Mark Dickinson; + bpo-1523(76).) The *note pack(): c0b. function will also attempt + to use *note __index__(): 18a. to convert and pack non-integers + before trying the *note __int__(): 18b. method or reporting an + error. (Changed by Mark Dickinson; bpo-8300(77).) + + * New function: the *note subprocess: f9. module’s *note + check_output(): 8db. runs a command with a specified set of + arguments and returns the command’s output as a string when the + command runs without error, or raises a *note CalledProcessError: + cb5. exception otherwise. + + >>> subprocess.check_output(['df', '-h', '.']) + 'Filesystem Size Used Avail Capacity Mounted on\n + /dev/disk0s2 52G 49G 3.0G 94% /\n' + + >>> subprocess.check_output(['df', '-h', '/bogus']) + ... + subprocess.CalledProcessError: Command '['df', '-h', '/bogus']' returned non-zero exit status 1 + + (Contributed by Gregory P. Smith.) + + The *note subprocess: f9. module will now retry its internal system + calls on receiving an ‘EINTR’ signal. (Reported by several people; + final patch by Gregory P. Smith in bpo-1068268(78).) + + * New function: *note is_declared_global(): cb6. in the *note + symtable: fc. module returns true for variables that are explicitly + declared to be global, false for ones that are implicitly global. + (Contributed by Jeremy Hylton.) + + * The *note syslog: ff. module will now use the value of + ‘sys.argv[0]’ as the identifier instead of the previous default + value of ‘'python'’. (Changed by Sean Reifschneider; + bpo-8451(79).) + + * The ‘sys.version_info’ value is now a named tuple, with attributes + named ‘major’, ‘minor’, ‘micro’, ‘releaselevel’, and ‘serial’. + (Contributed by Ross Light; bpo-4285(80).) + + *note sys.getwindowsversion(): 595. also returns a named tuple, + with attributes named ‘major’, ‘minor’, ‘build’, *note platform: + ce, ‘service_pack’, ‘service_pack_major’, ‘service_pack_minor’, + ‘suite_mask’, and ‘product_type’. (Contributed by Brian Curtin; + bpo-7766(81).) + + * The *note tarfile: 101. module’s default error handling has + changed, to no longer suppress fatal errors. The default error + level was previously 0, which meant that errors would only result + in a message being written to the debug log, but because the debug + log is not activated by default, these errors go unnoticed. The + default error level is now 1, which raises an exception if there’s + an error. (Changed by Lars Gustäbel; bpo-7357(82).) + + *note tarfile: 101. now supports filtering the *note TarInfo: b8d. + objects being added to a tar file. When you call *note add(): 47c, + you may supply an optional `filter' argument that’s a callable. + The `filter' callable will be passed the *note TarInfo: b8d. for + every file being added, and can modify and return it. If the + callable returns ‘None’, the file will be excluded from the + resulting archive. This is more powerful than the existing + `exclude' argument, which has therefore been deprecated. (Added by + Lars Gustäbel; bpo-6856(83).) The *note TarFile: b8c. class also + now supports the context management protocol. (Added by Lars + Gustäbel; bpo-7232(84).) + + * The *note wait(): cb7. method of the *note threading.Event: aad. + class now returns the internal flag on exit. This means the method + will usually return true because *note wait(): cb7. is supposed to + block until the internal flag becomes true. The return value will + only be false if a timeout was provided and the operation timed + out. (Contributed by Tim Lesher; bpo-1674032(85).) + + * The Unicode database provided by the *note unicodedata: 11a. module + is now used internally to determine which characters are numeric, + whitespace, or represent line breaks. The database also includes + information from the ‘Unihan.txt’ data file (patch by Anders + Chrigström and Amaury Forgeot d’Arc; bpo-1571184(86)) and has been + updated to version 5.2.0 (updated by Florent Xicluna; + bpo-8024(87)). + + * The ‘urlparse’ module’s ‘urlsplit()’ now handles unknown URL + schemes in a fashion compliant with RFC 3986(88): if the URL is of + the form ‘"://..."’, the text before the ‘://’ is + treated as the scheme, even if it’s a made-up scheme that the + module doesn’t know about. This change may break code that worked + around the old behaviour. For example, Python 2.6.4 or 2.5 will + return the following: + + >>> import urlparse + >>> urlparse.urlsplit('invented://host/filename?query') + ('invented', '', '//host/filename?query', '', '') + + Python 2.7 (and Python 2.6.5) will return: + + >>> import urlparse + >>> urlparse.urlsplit('invented://host/filename?query') + ('invented', 'host', '/filename?query', '', '') + + (Python 2.7 actually produces slightly different output, since it + returns a named tuple instead of a standard tuple.) + + The ‘urlparse’ module also supports IPv6 literal addresses as + defined by RFC 2732(89) (contributed by Senthil Kumaran; + bpo-2987(90)). + + >>> urlparse.urlparse('http://[1080::8:800:200C:417A]/foo') + ParseResult(scheme='http', netloc='[1080::8:800:200C:417A]', + path='/foo', params='', query='', fragment='') + + * New class: the *note WeakSet: cb8. class in the *note weakref: 128. + module is a set that only holds weak references to its elements; + elements will be removed once there are no references pointing to + them. (Originally implemented in Python 3.x by Raymond Hettinger, + and backported to 2.7 by Michael Foord.) + + * The ElementTree library, ‘xml.etree’, no longer escapes ampersands + and angle brackets when outputting an XML processing instruction + (which looks like ‘’) or comment + (which looks like ‘’). (Patch by Neil Muller; + bpo-2746(91).) + + * The XML-RPC client and server, provided by the ‘xmlrpclib’ and + ‘SimpleXMLRPCServer’ modules, have improved performance by + supporting HTTP/1.1 keep-alive and by optionally using gzip + encoding to compress the XML being exchanged. The gzip compression + is controlled by the ‘encode_threshold’ attribute of + ‘SimpleXMLRPCRequestHandler’, which contains a size in bytes; + responses larger than this will be compressed. (Contributed by + Kristján Valur Jónsson; bpo-6267(92).) + + * The *note zipfile: 142. module’s *note ZipFile: 41f. now supports + the context management protocol, so you can write ‘with + zipfile.ZipFile(...) as f:’. (Contributed by Brian Curtin; + bpo-5511(93).) + + *note zipfile: 142. now also supports archiving empty directories + and extracts them correctly. (Fixed by Kuba Wieczorek; + bpo-4710(94).) Reading files out of an archive is faster, and + interleaving *note read(): cb9. and ‘readline()’ now works + correctly. (Contributed by Nir Aides; bpo-7610(95).) + + The *note is_zipfile(): cba. function now accepts a file object, in + addition to the path names accepted in earlier versions. + (Contributed by Gabriel Genellina; bpo-4756(96).) + + The *note writestr(): cbb. method now has an optional + `compress_type' parameter that lets you override the default + compression method specified in the *note ZipFile: 41f. + constructor. (Contributed by Ronald Oussoren; bpo-6003(97).) + +* Menu: + +* New module; importlib: New module importlib. +* New module; sysconfig: New module sysconfig. +* ttk; Themed Widgets for Tk: ttk Themed Widgets for Tk. +* Updated module; unittest: Updated module unittest. +* Updated module; ElementTree 1.3: Updated module ElementTree 1 3. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5142 + + (2) https://bugs.python.org/issue7703 + + (3) https://www.jcea.es/programacion/pybsddb.htm + + (4) https://bugs.python.org/issue8156 + + (5) https://bugs.python.org/issue3860 + + (6) https://bugs.python.org/issue1696199 + + (7) https://bugs.python.org/issue1818 + + (8) https://bugs.python.org/issue8729 + + (9) https://bugs.python.org/issue7005 + + (10) https://bugs.python.org/issue3924 + + (11) https://bugs.python.org/issue1515 + + (12) https://bugs.python.org/issue4606 + + (13) https://sourceware.org/libffi/ + + (14) https://bugs.python.org/issue8142 + + (15) https://bugs.python.org/issue5788 + + (16) https://bugs.python.org/issue4796 + + (17) https://bugs.python.org/issue2531 + + (18) https://bugs.python.org/issue8257 + + (19) https://bugs.python.org/issue6595 + + (20) https://bugs.python.org/issue7633 + + (21) https://bugs.python.org/issue6857 + + (22) https://bugs.python.org/issue7279 + + (23) https://bugs.python.org/issue7585 + + (24) https://bugs.python.org/issue8688 + + (25) https://bugs.python.org/issue7490 + + (26) https://bugs.python.org/issue1368247 + + (27) https://bugs.python.org/issue5812 + + (28) https://bugs.python.org/issue8294 + + (29) https://bugs.python.org/issue2054 + + (30) https://bugs.python.org/issue6845 + + (31) https://bugs.python.org/issue5479 + + (32) https://bugs.python.org/issue4688 + + (33) https://bugs.python.org/issue3860 + + (34) https://bugs.python.org/issue7471 + + (35) https://bugs.python.org/issue4272 + + (36) https://bugs.python.org/issue2846 + + (37) https://bugs.python.org/issue7418 + + (38) https://bugs.python.org/issue4879 + + (39) https://bugs.python.org/issue3972 + + (40) https://bugs.python.org/issue1655 + + (41) https://bugs.python.org/issue3135 + + (42) https://bugs.python.org/issue4991 + + (43) https://bugs.python.org/issue6939 + + (44) https://bugs.python.org/issue5032 + + (45) https://bugs.python.org/issue4816 + + (46) https://bugs.python.org/issue4136 + + (47) https://bugs.python.org/issue5381 + + (48) https://bugs.python.org/issue1607951 + + (49) https://bugs.python.org/issue6896 + + (50) https://bugs.python.org/issue3366 + + (51) https://bugs.python.org/issue5585 + + (52) https://bugs.python.org/issue6963 + + (53) https://bugs.python.org/issue1664 + + (54) https://bugs.python.org/issue6508 + + (55) https://bugs.python.org/issue7333 + + (56) https://bugs.python.org/issue7242 + + (57) https://bugs.python.org/issue5827 + + (58) https://bugs.python.org/issue3426 + + (59) https://bugs.python.org/issue4739 + + (60) https://bugs.python.org/issue6816 + + (61) https://bugs.python.org/issue3002 + + (62) https://bugs.python.org/issue8354 + + (63) https://bugs.python.org/issue6693 + + (64) https://bugs.python.org/issue3137 + + (65) https://bugs.python.org/issue3972 + + (66) https://bugs.python.org/issue8104 + + (67) https://bugs.python.org/issue6192 + + (68) https://bugs.python.org/issue6267 + + (69) https://github.com/ghaering/pysqlite + + (70) https://bugs.python.org/issue7133 + + (71) https://bugs.python.org/issue8222 + + (72) +https://www.openssl.org/docs/manmaster/man1/ciphers.html#CIPHER-LIST-FORMAT + + (73) https://bugs.python.org/issue8322 + + (74) https://bugs.python.org/issue8484 + + (75) https://bugs.python.org/issue8321 + + (76) https://bugs.python.org/issue1523 + + (77) https://bugs.python.org/issue8300 + + (78) https://bugs.python.org/issue1068268 + + (79) https://bugs.python.org/issue8451 + + (80) https://bugs.python.org/issue4285 + + (81) https://bugs.python.org/issue7766 + + (82) https://bugs.python.org/issue7357 + + (83) https://bugs.python.org/issue6856 + + (84) https://bugs.python.org/issue7232 + + (85) https://bugs.python.org/issue1674032 + + (86) https://bugs.python.org/issue1571184 + + (87) https://bugs.python.org/issue8024 + + (88) https://tools.ietf.org/html/rfc3986.html + + (89) https://tools.ietf.org/html/rfc2732.html + + (90) https://bugs.python.org/issue2987 + + (91) https://bugs.python.org/issue2746 + + (92) https://bugs.python.org/issue6267 + + (93) https://bugs.python.org/issue5511 + + (94) https://bugs.python.org/issue4710 + + (95) https://bugs.python.org/issue7610 + + (96) https://bugs.python.org/issue4756 + + (97) https://bugs.python.org/issue6003 + + +File: python.info, Node: New module importlib, Next: New module sysconfig, Up: New and Improved Modules + +1.10.11.1 New module: importlib +............................... + +Python 3.1 includes the *note importlib: 9b. package, a +re-implementation of the logic underlying Python’s *note import: c1d. +statement. *note importlib: 9b. is useful for implementors of Python +interpreters and to users who wish to write new importers that can +participate in the import process. Python 2.7 doesn’t contain the +complete *note importlib: 9b. package, but instead has a tiny subset +that contains a single function, *note import_module(): b13. + +‘import_module(name, package=None)’ imports a module. `name' is a +string containing the module or package’s name. It’s possible to do +relative imports by providing a string that begins with a ‘.’ character, +such as ‘..utils.errors’. For relative imports, the `package' argument +must be provided and is the name of the package that will be used as the +anchor for the relative import. *note import_module(): b13. both +inserts the imported module into ‘sys.modules’ and returns the module +object. + +Here are some examples: + + >>> from importlib import import_module + >>> anydbm = import_module('anydbm') # Standard absolute import + >>> anydbm + + >>> # Relative import + >>> file_util = import_module('..file_util', 'distutils.command') + >>> file_util + + +*note importlib: 9b. was implemented by Brett Cannon and introduced in +Python 3.1. + + +File: python.info, Node: New module sysconfig, Next: ttk Themed Widgets for Tk, Prev: New module importlib, Up: New and Improved Modules + +1.10.11.2 New module: sysconfig +............................... + +The *note sysconfig: fe. module has been pulled out of the Distutils +package, becoming a new top-level module in its own right. *note +sysconfig: fe. provides functions for getting information about Python’s +build process: compiler switches, installation paths, the platform name, +and whether Python is running from its source directory. + +Some of the functions in the module are: + + * *note get_config_var(): 964. returns variables from Python’s + Makefile and the ‘pyconfig.h’ file. + + * *note get_config_vars(): 965. returns a dictionary containing all + of the configuration variables. + + * *note get_path(): cbe. returns the configured path for a particular + type of module: the standard library, site-specific modules, + platform-specific modules, etc. + + * *note is_python_build(): cbf. returns true if you’re running a + binary from a Python source tree, and false otherwise. + +Consult the *note sysconfig: fe. documentation for more details and for +a complete list of functions. + +The Distutils package and *note sysconfig: fe. are now maintained by +Tarek Ziadé, who has also started a Distutils2 package (source +repository at ‘https://hg.python.org/distutils2/’) for developing a +next-generation version of Distutils. + + +File: python.info, Node: ttk Themed Widgets for Tk, Next: Updated module unittest, Prev: New module sysconfig, Up: New and Improved Modules + +1.10.11.3 ttk: Themed Widgets for Tk +.................................... + +Tcl/Tk 8.5 includes a set of themed widgets that re-implement basic Tk +widgets but have a more customizable appearance and can therefore more +closely resemble the native platform’s widgets. This widget set was +originally called Tile, but was renamed to Ttk (for “themed Tk”) on +being added to Tcl/Tck release 8.5. + +To learn more, read the ‘ttk’ module documentation. You may also wish +to read the Tcl/Tk manual page describing the Ttk theme engine, +available at ‘https://www.tcl.tk/man/tcl8.5/TkCmd/ttk_intro.htm’. Some +screenshots of the Python/Ttk code in use are at +‘https://code.google.com/archive/p/python-ttk/wikis/Screenshots.wiki’. + +The ‘ttk’ module was written by Guilherme Polo and added in bpo-2983(1). +An alternate version called ‘Tile.py’, written by Martin Franklin and +maintained by Kevin Walzer, was proposed for inclusion in bpo-2618(2), +but the authors argued that Guilherme Polo’s work was more +comprehensive. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2983 + + (2) https://bugs.python.org/issue2618 + + +File: python.info, Node: Updated module unittest, Next: Updated module ElementTree 1 3, Prev: ttk Themed Widgets for Tk, Up: New and Improved Modules + +1.10.11.4 Updated module: unittest +.................................. + +The *note unittest: 11b. module was greatly enhanced; many new features +were added. Most of these features were implemented by Michael Foord, +unless otherwise noted. The enhanced version of the module is +downloadable separately for use with Python versions 2.4 to 2.6, +packaged as the ‘unittest2’ package, from +‘https://pypi.org/project/unittest2’. + +When used from the command line, the module can automatically discover +tests. It’s not as fancy as py.test(1) or nose(2), but provides a +simple way to run tests kept within a set of package directories. For +example, the following command will search the ‘test/’ subdirectory for +any importable test files named ‘test*.py’: + + python -m unittest discover -s test + +Consult the *note unittest: 11b. module documentation for more details. +(Developed in bpo-6001(3).) + +The *note main(): 902. function supports some other new options: + + * *note -b: cc3. or ‘--buffer’ will buffer the standard output and + standard error streams during each test. If the test passes, any + resulting output will be discarded; on failure, the buffered output + will be displayed. + + * *note -c: cc4. or ‘--catch’ will cause the control-C interrupt to + be handled more gracefully. Instead of interrupting the test + process immediately, the currently running test will be completed + and then the partial results up to the interruption will be + reported. If you’re impatient, a second press of control-C will + cause an immediate interruption. + + This control-C handler tries to avoid causing problems when the + code being tested or the tests being run have defined a signal + handler of their own, by noticing that a signal handler was already + set and calling it. If this doesn’t work for you, there’s a *note + removeHandler(): cc5. decorator that can be used to mark tests that + should have the control-C handling disabled. + + * *note -f: cc6. or ‘--failfast’ makes test execution stop + immediately when a test fails instead of continuing to execute + further tests. (Suggested by Cliff Dyer and implemented by Michael + Foord; bpo-8074(4).) + +The progress messages now show ‘x’ for expected failures and ‘u’ for +unexpected successes when run in verbose mode. (Contributed by Benjamin +Peterson.) + +Test cases can raise the *note SkipTest: 903. exception to skip a test +(bpo-1034053(5)). + +The error messages for *note assertEqual(): bb5, *note assertTrue(): +bb4, and *note assertFalse(): cc7. failures now provide more +information. If you set the *note longMessage: cc8. attribute of your +*note TestCase: 8ff. classes to true, both the standard error message +and any additional message you provide will be printed for failures. +(Added by Michael Foord; bpo-5663(6).) + +The *note assertRaises(): aba. method now returns a context handler when +called without providing a callable object to run. For example, you can +write this: + + with self.assertRaises(KeyError): + {}['foo'] + +(Implemented by Antoine Pitrou; bpo-4444(7).) + +Module- and class-level setup and teardown fixtures are now supported. +Modules can contain ‘setUpModule()’ and ‘tearDownModule()’ functions. +Classes can have *note setUpClass(): 24c. and *note tearDownClass(): +cc9. methods that must be defined as class methods (using ‘@classmethod’ +or equivalent). These functions and methods are invoked when the test +runner switches to a test case in a different module or class. + +The methods *note addCleanup(): 2a2. and *note doCleanups(): cca. were +added. *note addCleanup(): 2a2. lets you add cleanup functions that +will be called unconditionally (after *note setUp(): ccb. if *note +setUp(): ccb. fails, otherwise after *note tearDown(): ccc.). This +allows for much simpler resource allocation and deallocation during +tests (bpo-5679(8)). + +A number of new methods were added that provide more specialized tests. +Many of these methods were written by Google engineers for use in their +test suites; Gregory P. Smith, Michael Foord, and GvR worked on merging +them into Python’s version of *note unittest: 11b. + + * *note assertIsNone(): ccd. and *note assertIsNotNone(): cce. take + one expression and verify that the result is or is not ‘None’. + + * *note assertIs(): ccf. and *note assertIsNot(): cd0. take two + values and check whether the two values evaluate to the same object + or not. (Added by Michael Foord; bpo-2578(9).) + + * *note assertIsInstance(): cd1. and *note assertNotIsInstance(): + cd2. check whether the resulting object is an instance of a + particular class, or of one of a tuple of classes. (Added by Georg + Brandl; bpo-7031(10).) + + * *note assertGreater(): cd3, *note assertGreaterEqual(): cd4, *note + assertLess(): cd5, and *note assertLessEqual(): cd6. compare two + quantities. + + * *note assertMultiLineEqual(): cd7. compares two strings, and if + they’re not equal, displays a helpful comparison that highlights + the differences in the two strings. This comparison is now used by + default when Unicode strings are compared with *note assertEqual(): + bb5. + + * ‘assertRegexpMatches()’ and ‘assertNotRegexpMatches()’ checks + whether the first argument is a string matching or not matching the + regular expression provided as the second argument (bpo-8038(11)). + + * ‘assertRaisesRegexp()’ checks whether a particular exception is + raised, and then also checks that the string representation of the + exception matches the provided regular expression. + + * *note assertIn(): cd8. and *note assertNotIn(): cd9. tests whether + `first' is or is not in `second'. + + * ‘assertItemsEqual()’ tests whether two provided sequences contain + the same elements. + + * *note assertSetEqual(): cda. compares whether two sets are equal, + and only reports the differences between the sets in case of error. + + * Similarly, *note assertListEqual(): cdb. and *note + assertTupleEqual(): cdc. compare the specified types and explain + any differences without necessarily printing their full values; + these methods are now used by default when comparing lists and + tuples using *note assertEqual(): bb5. More generally, *note + assertSequenceEqual(): cdd. compares two sequences and can + optionally check whether both sequences are of a particular type. + + * *note assertDictEqual(): cde. compares two dictionaries and reports + the differences; it’s now used by default when you compare two + dictionaries using *note assertEqual(): bb5. + ‘assertDictContainsSubset()’ checks whether all of the key/value + pairs in `first' are found in `second'. + + * *note assertAlmostEqual(): bb7. and *note assertNotAlmostEqual(): + bb8. test whether `first' and `second' are approximately equal. + This method can either round their difference to an + optionally-specified number of `places' (the default is 7) and + compare it to zero, or require the difference to be smaller than a + supplied `delta' value. + + * *note loadTestsFromName(): cdf. properly honors the *note + suiteClass: ce0. attribute of the *note TestLoader: 782. (Fixed by + Mark Roddy; bpo-6866(12).) + + * A new hook lets you extend the *note assertEqual(): bb5. method to + handle new data types. The *note addTypeEqualityFunc(): ce1. + method takes a type object and a function. The function will be + used when both of the objects being compared are of the specified + type. This function should compare the two objects and raise an + exception if they don’t match; it’s a good idea for the function to + provide additional information about why the two objects aren’t + matching, much as the new sequence comparison methods do. + +*note unittest.main(): 902. now takes an optional ‘exit’ argument. If +false, *note main(): 902. doesn’t call *note sys.exit(): ce2, allowing +*note main(): 902. to be used from the interactive interpreter. +(Contributed by J. Pablo Fernández; bpo-3379(13).) + +*note TestResult: abf. has new *note startTestRun(): ce3. and *note +stopTestRun(): ce4. methods that are called immediately before and after +a test run. (Contributed by Robert Collins; bpo-5728(14).) + +With all these changes, the ‘unittest.py’ was becoming awkwardly large, +so the module was turned into a package and the code split into several +files (by Benjamin Peterson). This doesn’t affect how the module is +imported or used. + +See also +........ + +‘http://www.voidspace.org.uk/python/articles/unittest2.shtml’ + + Describes the new features, how to use them, and the rationale for + various design decisions. (By Michael Foord.) + + ---------- Footnotes ---------- + + (1) http://pytest.org + + (2) https://nose.readthedocs.io/ + + (3) https://bugs.python.org/issue6001 + + (4) https://bugs.python.org/issue8074 + + (5) https://bugs.python.org/issue1034053 + + (6) https://bugs.python.org/issue5663 + + (7) https://bugs.python.org/issue4444 + + (8) https://bugs.python.org/issue5679 + + (9) https://bugs.python.org/issue2578 + + (10) https://bugs.python.org/issue7031 + + (11) https://bugs.python.org/issue8038 + + (12) https://bugs.python.org/issue6866 + + (13) https://bugs.python.org/issue3379 + + (14) https://bugs.python.org/issue5728 + + +File: python.info, Node: Updated module ElementTree 1 3, Prev: Updated module unittest, Up: New and Improved Modules + +1.10.11.5 Updated module: ElementTree 1.3 +......................................... + +The version of the ElementTree library included with Python was updated +to version 1.3. Some of the new features are: + + * The various parsing functions now take a `parser' keyword argument + giving an *note XMLParser: 252. instance that will be used. This + makes it possible to override the file’s internal encoding: + + p = ET.XMLParser(encoding='utf-8') + t = ET.XML("""""", parser=p) + + Errors in parsing XML now raise a ‘ParseError’ exception, whose + instances have a ‘position’ attribute containing a (`line', + `column') tuple giving the location of the problem. + + * ElementTree’s code for converting trees to a string has been + significantly reworked, making it roughly twice as fast in many + cases. The *note ElementTree.write(): 918. and ‘Element.write()’ + methods now have a `method' parameter that can be “xml” (the + default), “html”, or “text”. HTML mode will output empty elements + as ‘’ instead of ‘’, and text mode will skip + over elements and only output the text chunks. If you set the + ‘tag’ attribute of an element to ‘None’ but leave its children in + place, the element will be omitted when the tree is written out, so + you don’t need to do more extensive rearrangement to remove a + single element. + + Namespace handling has also been improved. All ‘xmlns:’ + declarations are now output on the root element, not scattered + throughout the resulting XML. You can set the default namespace for + a tree by setting the ‘default_namespace’ attribute and can + register new prefixes with *note register_namespace(): b50. In XML + mode, you can use the true/false `xml_declaration' parameter to + suppress the XML declaration. + + * New *note Element: ac4. method: *note extend(): b51. appends the + items from a sequence to the element’s children. Elements + themselves behave like sequences, so it’s easy to move children + from one element to another: + + from xml.etree import ElementTree as ET + + t = ET.XML(""" + 1 2 3 + """) + new = ET.XML('') + new.extend(t) + + # Outputs 1... + print ET.tostring(new) + + * New ‘Element’ method: *note iter(): ce7. yields the children of the + element as a generator. It’s also possible to write ‘for child in + elem:’ to loop over an element’s children. The existing method + ‘getiterator()’ is now deprecated, as is ‘getchildren()’ which + constructs and returns a list of children. + + * New ‘Element’ method: *note itertext(): b53. yields all chunks of + text that are descendants of the element. For example: + + t = ET.XML(""" + 1 2 3 + """) + + # Outputs ['\n ', '1', ' ', '2', ' ', '3', '\n'] + print list(t.itertext()) + + * Deprecated: using an element as a Boolean (i.e., ‘if elem:’) would + return true if the element had any children, or false if there were + no children. This behaviour is confusing – ‘None’ is false, but so + is a childless element? – so it will now trigger a *note + FutureWarning: 325. In your code, you should be explicit: write + ‘len(elem) != 0’ if you’re interested in the number of children, or + ‘elem is not None’. + +Fredrik Lundh develops ElementTree and produced the 1.3 version; you can +read his article describing 1.3 at +‘http://effbot.org/zone/elementtree-13-intro.htm’. Florent Xicluna +updated the version included with Python, after discussions on +python-dev and in bpo-6472(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue6472 + + +File: python.info, Node: Build and C API Changes<8>, Next: Other Changes and Fixes, Prev: New and Improved Modules, Up: What’s New in Python 2 7 + +1.10.12 Build and C API Changes +------------------------------- + +Changes to Python’s build process and to the C API include: + + * The latest release of the GNU Debugger, GDB 7, can be scripted + using Python(1). When you begin debugging an executable program P, + GDB will look for a file named ‘P-gdb.py’ and automatically read + it. Dave Malcolm contributed a ‘python-gdb.py’ that adds a number + of commands useful when debugging Python itself. For example, + ‘py-up’ and ‘py-down’ go up or down one Python stack frame, which + usually corresponds to several C stack frames. ‘py-print’ prints + the value of a Python variable, and ‘py-bt’ prints the Python stack + trace. (Added as a result of bpo-8032(2).) + + * If you use the ‘.gdbinit’ file provided with Python, the “pyo” + macro in the 2.7 version now works correctly when the thread being + debugged doesn’t hold the GIL; the macro now acquires it before + printing. (Contributed by Victor Stinner; bpo-3632(3).) + + * *note Py_AddPendingCall(): ce9. is now thread-safe, letting any + worker thread submit notifications to the main Python thread. This + is particularly useful for asynchronous IO operations. + (Contributed by Kristján Valur Jónsson; bpo-4293(4).) + + * New function: *note PyCode_NewEmpty(): cea. creates an empty code + object; only the filename, function name, and first line number are + required. This is useful for extension modules that are attempting + to construct a more useful traceback stack. Previously such + extensions needed to call *note PyCode_New(): 272, which had many + more arguments. (Added by Jeffrey Yasskin.) + + * New function: *note PyErr_NewExceptionWithDoc(): bff. creates a new + exception class, just as the existing *note PyErr_NewException(): + c00. does, but takes an extra ‘char *’ argument containing the + docstring for the new exception class. (Added by ‘lekma’ on the + Python bug tracker; bpo-7033(5).) + + * New function: *note PyFrame_GetLineNumber(): ceb. takes a frame + object and returns the line number that the frame is currently + executing. Previously code would need to get the index of the + bytecode instruction currently executing, and then look up the line + number corresponding to that address. (Added by Jeffrey Yasskin.) + + * New functions: *note PyLong_AsLongAndOverflow(): bfd. and *note + PyLong_AsLongLongAndOverflow(): bfc. approximates a Python long + integer as a C ‘long’ or ‘long long’. If the number is too large + to fit into the output type, an `overflow' flag is set and returned + to the caller. (Contributed by Case Van Horsen; bpo-7528(6) and + bpo-7767(7).) + + * New function: stemming from the rewrite of string-to-float + conversion, a new *note PyOS_string_to_double(): c23. function was + added. The old ‘PyOS_ascii_strtod()’ and ‘PyOS_ascii_atof()’ + functions are now deprecated. + + * New function: *note PySys_SetArgvEx(): bfa. sets the value of + ‘sys.argv’ and can optionally update ‘sys.path’ to include the + directory containing the script named by ‘sys.argv[0]’ depending on + the value of an `updatepath' parameter. + + This function was added to close a security hole for applications + that embed Python. The old function, *note PySys_SetArgv(): cec, + would always update ‘sys.path’, and sometimes it would add the + current directory. This meant that, if you ran an application + embedding Python in a directory controlled by someone else, + attackers could put a Trojan-horse module in the directory (say, a + file named ‘os.py’) that your application would then import and + run. + + If you maintain a C/C++ application that embeds Python, check + whether you’re calling *note PySys_SetArgv(): cec. and carefully + consider whether the application should be using *note + PySys_SetArgvEx(): bfa. with `updatepath' set to false. + + Security issue reported as CVE-2008-5983(8); discussed in + bpo-5753(9), and fixed by Antoine Pitrou. + + * New macros: the Python header files now define the following + macros: ‘Py_ISALNUM’, ‘Py_ISALPHA’, ‘Py_ISDIGIT’, ‘Py_ISLOWER’, + ‘Py_ISSPACE’, ‘Py_ISUPPER’, ‘Py_ISXDIGIT’, ‘Py_TOLOWER’, and + ‘Py_TOUPPER’. All of these functions are analogous to the C + standard macros for classifying characters, but ignore the current + locale setting, because in several places Python needs to analyze + characters in a locale-independent way. (Added by Eric Smith; + bpo-5793(10).) + + * Removed function: ‘PyEval_CallObject’ is now only available as a + macro. A function version was being kept around to preserve ABI + linking compatibility, but that was in 1997; it can certainly be + deleted by now. (Removed by Antoine Pitrou; bpo-8276(11).) + + * New format codes: the ‘PyFormat_FromString()’, + ‘PyFormat_FromStringV()’, and *note PyErr_Format(): 7aa. functions + now accept ‘%lld’ and ‘%llu’ format codes for displaying C’s ‘long + long’ types. (Contributed by Mark Dickinson; bpo-7228(12).) + + * The complicated interaction between threads and process forking has + been changed. Previously, the child process created by *note + os.fork(): 961. might fail because the child is created with only a + single thread running, the thread performing the *note os.fork(): + 961. If other threads were holding a lock, such as Python’s import + lock, when the fork was performed, the lock would still be marked + as “held” in the new process. But in the child process nothing + would ever release the lock, since the other threads weren’t + replicated, and the child process would no longer be able to + perform imports. + + Python 2.7 acquires the import lock before performing an *note + os.fork(): 961, and will also clean up any locks created using the + *note threading: 109. module. C extension modules that have + internal locks, or that call ‘fork()’ themselves, will not benefit + from this clean-up. + + (Fixed by Thomas Wouters; bpo-1590864(13).) + + * The *note Py_Finalize(): ced. function now calls the internal + ‘threading._shutdown()’ function; this prevents some exceptions + from being raised when an interpreter shuts down. (Patch by Adam + Olsen; bpo-1722344(14).) + + * When using the *note PyMemberDef: 426. structure to define + attributes of a type, Python will no longer let you try to delete + or set a ‘T_STRING_INPLACE’ attribute. + + * Global symbols defined by the *note ctypes: 2b. module are now + prefixed with ‘Py’, or with ‘_ctypes’. (Implemented by Thomas + Heller; bpo-3102(15).) + + * New configure option: the ‘--with-system-expat’ switch allows + building the ‘pyexpat’ module to use the system Expat library. + (Contributed by Arfrever Frehtes Taifersar Arahesis; bpo-7609(16).) + + * New configure option: the ‘--with-valgrind’ option will now disable + the pymalloc allocator, which is difficult for the Valgrind + memory-error detector to analyze correctly. Valgrind will + therefore be better at detecting memory leaks and overruns. + (Contributed by James Henstridge; bpo-2422(17).) + + * New configure option: you can now supply an empty string to + ‘--with-dbmliborder=’ in order to disable all of the various DBM + modules. (Added by Arfrever Frehtes Taifersar Arahesis; + bpo-6491(18).) + + * The ‘configure’ script now checks for floating-point rounding bugs + on certain 32-bit Intel chips and defines a ‘X87_DOUBLE_ROUNDING’ + preprocessor definition. No code currently uses this definition, + but it’s available if anyone wishes to use it. (Added by Mark + Dickinson; bpo-2937(19).) + + ‘configure’ also now sets a ‘LDCXXSHARED’ Makefile variable for + supporting C++ linking. (Contributed by Arfrever Frehtes Taifersar + Arahesis; bpo-1222585(20).) + + * The build process now creates the necessary files for pkg-config + support. (Contributed by Clinton Roy; bpo-3585(21).) + + * The build process now supports Subversion 1.7. (Contributed by + Arfrever Frehtes Taifersar Arahesis; bpo-6094(22).) + +* Menu: + +* Capsules:: +* Port-Specific Changes; Windows: Port-Specific Changes Windows. +* Port-Specific Changes; Mac OS X: Port-Specific Changes Mac OS X. +* Port-Specific Changes; FreeBSD: Port-Specific Changes FreeBSD. + + ---------- Footnotes ---------- + + (1) https://sourceware.org/gdb/current/onlinedocs/gdb/Python.html + + (2) https://bugs.python.org/issue8032 + + (3) https://bugs.python.org/issue3632 + + (4) https://bugs.python.org/issue4293 + + (5) https://bugs.python.org/issue7033 + + (6) https://bugs.python.org/issue7528 + + (7) https://bugs.python.org/issue7767 + + (8) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983 + + (9) https://bugs.python.org/issue5753 + + (10) https://bugs.python.org/issue5793 + + (11) https://bugs.python.org/issue8276 + + (12) https://bugs.python.org/issue7228 + + (13) https://bugs.python.org/issue1590864 + + (14) https://bugs.python.org/issue1722344 + + (15) https://bugs.python.org/issue3102 + + (16) https://bugs.python.org/issue7609 + + (17) https://bugs.python.org/issue2422 + + (18) https://bugs.python.org/issue6491 + + (19) https://bugs.python.org/issue2937 + + (20) https://bugs.python.org/issue1222585 + + (21) https://bugs.python.org/issue3585 + + (22) https://bugs.python.org/issue6094 + + +File: python.info, Node: Capsules, Next: Port-Specific Changes Windows, Up: Build and C API Changes<8> + +1.10.12.1 Capsules +.................. + +Python 3.1 adds a new C datatype, *note PyCapsule: c07, for providing a +C API to an extension module. A capsule is essentially the holder of a +C ‘void *’ pointer, and is made available as a module attribute; for +example, the *note socket: ef. module’s API is exposed as ‘socket.CAPI’, +and *note unicodedata: 11a. exposes ‘ucnhash_CAPI’. Other extensions +can import the module, access its dictionary to get the capsule object, +and then get the ‘void *’ pointer, which will usually point to an array +of pointers to the module’s various API functions. + +There is an existing data type already used for this, ‘PyCObject’, but +it doesn’t provide type safety. Evil code written in pure Python could +cause a segmentation fault by taking a ‘PyCObject’ from module A and +somehow substituting it for the ‘PyCObject’ in module B. Capsules know +their own name, and getting the pointer requires providing the name: + + void *vtable; + + if (!PyCapsule_IsValid(capsule, "mymodule.CAPI") { + PyErr_SetString(PyExc_ValueError, "argument type invalid"); + return NULL; + } + + vtable = PyCapsule_GetPointer(capsule, "mymodule.CAPI"); + +You are assured that ‘vtable’ points to whatever you’re expecting. If a +different capsule was passed in, *note PyCapsule_IsValid(): cf0. would +detect the mismatched name and return false. Refer to *note Providing a +C API for an Extension Module: cf1. for more information on using these +objects. + +Python 2.7 now uses capsules internally to provide various +extension-module APIs, but the ‘PyCObject_AsVoidPtr()’ was modified to +handle capsules, preserving compile-time compatibility with the +‘CObject’ interface. Use of ‘PyCObject_AsVoidPtr()’ will signal a *note +PendingDeprecationWarning: 279, which is silent by default. + +Implemented in Python 3.1 and backported to 2.7 by Larry Hastings; +discussed in bpo-5630(1). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5630 + + +File: python.info, Node: Port-Specific Changes Windows, Next: Port-Specific Changes Mac OS X, Prev: Capsules, Up: Build and C API Changes<8> + +1.10.12.2 Port-Specific Changes: Windows +........................................ + + * The *note msvcrt: b6. module now contains some constants from the + ‘crtassem.h’ header file: ‘CRT_ASSEMBLY_VERSION’, + ‘VC_ASSEMBLY_PUBLICKEYTOKEN’, and ‘LIBRARIES_ASSEMBLY_NAME_PREFIX’. + (Contributed by David Cournapeau; bpo-4365(1).) + + * The ‘_winreg’ module for accessing the registry now implements the + ‘CreateKeyEx()’ and ‘DeleteKeyEx()’ functions, extended versions of + previously-supported functions that take several extra arguments. + The ‘DisableReflectionKey()’, ‘EnableReflectionKey()’, and + ‘QueryReflectionKey()’ were also tested and documented. + (Implemented by Brian Curtin: bpo-7347(2).) + + * The new ‘_beginthreadex()’ API is used to start threads, and the + native thread-local storage functions are now used. (Contributed + by Kristján Valur Jónsson; bpo-3582(3).) + + * The *note os.kill(): cf3. function now works on Windows. The + signal value can be the constants ‘CTRL_C_EVENT’, + ‘CTRL_BREAK_EVENT’, or any integer. The first two constants will + send ‘Control-C’ and ‘Control-Break’ keystroke events to + subprocesses; any other value will use the ‘TerminateProcess()’ + API. (Contributed by Miki Tebeka; bpo-1220212(4).) + + * The *note os.listdir(): a41. function now correctly fails for an + empty path. (Fixed by Hirokazu Yamamoto; bpo-5913(5).) + + * The ‘mimelib’ module will now read the MIME database from the + Windows registry when initializing. (Patch by Gabriel Genellina; + bpo-4969(6).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4365 + + (2) https://bugs.python.org/issue7347 + + (3) https://bugs.python.org/issue3582 + + (4) https://bugs.python.org/issue1220212 + + (5) https://bugs.python.org/issue5913 + + (6) https://bugs.python.org/issue4969 + + +File: python.info, Node: Port-Specific Changes Mac OS X, Next: Port-Specific Changes FreeBSD, Prev: Port-Specific Changes Windows, Up: Build and C API Changes<8> + +1.10.12.3 Port-Specific Changes: Mac OS X +......................................... + + * The path ‘/Library/Python/2.7/site-packages’ is now appended to + ‘sys.path’, in order to share added packages between the system + installation and a user-installed copy of the same version. + (Changed by Ronald Oussoren; bpo-4865(1).) + + Changed in version 2.7.13: As of 2.7.13, this change was + removed. ‘/Library/Python/2.7/site-packages’, the + site-packages directory used by the Apple-supplied system + Python 2.7 is no longer appended to ‘sys.path’ for + user-installed Pythons such as from the python.org installers. + As of macOS 10.12, Apple changed how the system site-packages + directory is configured, which could cause installation of pip + components, like setuptools, to fail. Packages installed for + the system Python will no longer be shared with user-installed + Pythons. (bpo-28440(2)) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue4865 + + (2) https://bugs.python.org/issue28440 + + +File: python.info, Node: Port-Specific Changes FreeBSD, Prev: Port-Specific Changes Mac OS X, Up: Build and C API Changes<8> + +1.10.12.4 Port-Specific Changes: FreeBSD +........................................ + + * FreeBSD 7.1’s ‘SO_SETFIB’ constant, used with + ‘getsockopt()’/‘setsockopt()’ to select an alternate routing table, + is now available in the *note socket: ef. module. (Added by Kyle + VanderBeek; bpo-8235(1).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue8235 + + +File: python.info, Node: Other Changes and Fixes, Next: Porting to Python 2 7, Prev: Build and C API Changes<8>, Up: What’s New in Python 2 7 + +1.10.13 Other Changes and Fixes +------------------------------- + + * Two benchmark scripts, ‘iobench’ and ‘ccbench’, were added to the + ‘Tools’ directory. ‘iobench’ measures the speed of the built-in + file I/O objects returned by *note open(): 4f0. while performing + various operations, and ‘ccbench’ is a concurrency benchmark that + tries to measure computing throughput, thread switching latency, + and IO processing bandwidth when performing several tasks using a + varying number of threads. + + * The ‘Tools/i18n/msgfmt.py’ script now understands plural forms in + ‘.po’ files. (Fixed by Martin von Löwis; bpo-5464(1).) + + * When importing a module from a ‘.pyc’ or ‘.pyo’ file with an + existing ‘.py’ counterpart, the ‘co_filename’ attributes of the + resulting code objects are overwritten when the original filename + is obsolete. This can happen if the file has been renamed, moved, + or is accessed through different paths. (Patch by Ziga Seilnacht + and Jean-Paul Calderone; bpo-1180193(2).) + + * The ‘regrtest.py’ script now takes a ‘--randseed=’ switch that + takes an integer that will be used as the random seed for the ‘-r’ + option that executes tests in random order. The ‘-r’ option also + reports the seed that was used (Added by Collin Winter.) + + * Another ‘regrtest.py’ switch is ‘-j’, which takes an integer + specifying how many tests run in parallel. This allows reducing + the total runtime on multi-core machines. This option is + compatible with several other options, including the ‘-R’ switch + which is known to produce long runtimes. (Added by Antoine Pitrou, + bpo-6152(3).) This can also be used with a new ‘-F’ switch that + runs selected tests in a loop until they fail. (Added by Antoine + Pitrou; bpo-7312(4).) + + * When executed as a script, the ‘py_compile.py’ module now accepts + ‘'-'’ as an argument, which will read standard input for the list + of filenames to be compiled. (Contributed by Piotr Ożarowski; + bpo-8233(5).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue5464 + + (2) https://bugs.python.org/issue1180193 + + (3) https://bugs.python.org/issue6152 + + (4) https://bugs.python.org/issue7312 + + (5) https://bugs.python.org/issue8233 + + +File: python.info, Node: Porting to Python 2 7, Next: New Features Added to Python 2 7 Maintenance Releases, Prev: Other Changes and Fixes, Up: What’s New in Python 2 7 + +1.10.14 Porting to Python 2.7 +----------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code: + + * The *note range(): 9be. function processes its arguments more + consistently; it will now call *note __int__(): 18b. on non-float, + non-integer arguments that are supplied to it. (Fixed by Alexander + Belopolsky; bpo-1533(1).) + + * The string *note format(): 4db. method changed the default + precision used for floating-point and complex numbers from 6 + decimal places to 12, which matches the precision used by *note + str(): 330. (Changed by Eric Smith; bpo-5920(2).) + + * Because of an optimization for the *note with: 6e9. statement, the + special methods *note __enter__(): c95. and *note __exit__(): c96. + must belong to the object’s type, and cannot be directly attached + to the object’s instance. This affects new-style classes (derived + from *note object: 2b0.) and C extension types. (bpo-6101(3).) + + * Due to a bug in Python 2.6, the `exc_value' parameter to *note + __exit__(): c96. methods was often the string representation of the + exception, not an instance. This was fixed in 2.7, so `exc_value' + will be an instance as expected. (Fixed by Florent Xicluna; + bpo-7853(4).) + + * When a restricted set of attributes were set using ‘__slots__’, + deleting an unset attribute would not raise *note AttributeError: + 39b. as you would expect. Fixed by Benjamin Peterson; + bpo-7604(5).) + +In the standard library: + + * Operations with *note datetime: 194. instances that resulted in a + year falling outside the supported range didn’t always raise *note + OverflowError: 960. Such errors are now checked more carefully and + will now raise the exception. (Reported by Mark Leander, patch by + Anand B. Pillai and Alexander Belopolsky; bpo-7150(6).) + + * When using *note Decimal: 188. instances with a string’s *note + format(): 4db. method, the default alignment was previously + left-alignment. This has been changed to right-alignment, which + might change the output of your programs. (Changed by Mark + Dickinson; bpo-6857(7).) + + Comparisons involving a signaling NaN value (or ‘sNAN’) now signal + *note InvalidOperation: 9eb. instead of silently returning a true + or false value depending on the comparison operator. Quiet NaN + values (or ‘NaN’) are now hashable. (Fixed by Mark Dickinson; + bpo-7279(8).) + + * The ElementTree library, ‘xml.etree’, no longer escapes ampersands + and angle brackets when outputting an XML processing instruction + (which looks like ‘’) or comment + (which looks like ‘’). (Patch by Neil Muller; + bpo-2746(9).) + + * The ‘readline()’ method of ‘StringIO’ objects now does nothing when + a negative length is requested, as other file-like objects do. + (bpo-7348(10)). + + * The *note syslog: ff. module will now use the value of + ‘sys.argv[0]’ as the identifier instead of the previous default + value of ‘'python'’. (Changed by Sean Reifschneider; + bpo-8451(11).) + + * The *note tarfile: 101. module’s default error handling has + changed, to no longer suppress fatal errors. The default error + level was previously 0, which meant that errors would only result + in a message being written to the debug log, but because the debug + log is not activated by default, these errors go unnoticed. The + default error level is now 1, which raises an exception if there’s + an error. (Changed by Lars Gustäbel; bpo-7357(12).) + + * The ‘urlparse’ module’s ‘urlsplit()’ now handles unknown URL + schemes in a fashion compliant with RFC 3986(13): if the URL is of + the form ‘"://..."’, the text before the ‘://’ is + treated as the scheme, even if it’s a made-up scheme that the + module doesn’t know about. This change may break code that worked + around the old behaviour. For example, Python 2.6.4 or 2.5 will + return the following: + + >>> import urlparse + >>> urlparse.urlsplit('invented://host/filename?query') + ('invented', '', '//host/filename?query', '', '') + + Python 2.7 (and Python 2.6.5) will return: + + >>> import urlparse + >>> urlparse.urlsplit('invented://host/filename?query') + ('invented', 'host', '/filename?query', '', '') + + (Python 2.7 actually produces slightly different output, since it + returns a named tuple instead of a standard tuple.) + +For C extensions: + + * C extensions that use integer format codes with the ‘PyArg_Parse*’ + family of functions will now raise a *note TypeError: 192. + exception instead of triggering a *note DeprecationWarning: 278. + (bpo-5080(14)). + + * Use the new *note PyOS_string_to_double(): c23. function instead of + the old ‘PyOS_ascii_strtod()’ and ‘PyOS_ascii_atof()’ functions, + which are now deprecated. + +For applications that embed Python: + + * The *note PySys_SetArgvEx(): bfa. function was added, letting + applications close a security hole when the existing *note + PySys_SetArgv(): cec. function was used. Check whether you’re + calling *note PySys_SetArgv(): cec. and carefully consider whether + the application should be using *note PySys_SetArgvEx(): bfa. with + `updatepath' set to false. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1533 + + (2) https://bugs.python.org/issue5920 + + (3) https://bugs.python.org/issue6101 + + (4) https://bugs.python.org/issue7853 + + (5) https://bugs.python.org/issue7604 + + (6) https://bugs.python.org/issue7150 + + (7) https://bugs.python.org/issue6857 + + (8) https://bugs.python.org/issue7279 + + (9) https://bugs.python.org/issue2746 + + (10) https://bugs.python.org/issue7348 + + (11) https://bugs.python.org/issue8451 + + (12) https://bugs.python.org/issue7357 + + (13) https://tools.ietf.org/html/rfc3986.html + + (14) https://bugs.python.org/issue5080 + + +File: python.info, Node: New Features Added to Python 2 7 Maintenance Releases, Next: Acknowledgements, Prev: Porting to Python 2 7, Up: What’s New in Python 2 7 + +1.10.15 New Features Added to Python 2.7 Maintenance Releases +------------------------------------------------------------- + +New features may be added to Python 2.7 maintenance releases when the +situation genuinely calls for it. Any such additions must go through +the Python Enhancement Proposal process, and make a compelling case for +why they can’t be adequately addressed by either adding the new feature +solely to Python 3, or else by publishing it on the Python Package +Index. + +In addition to the specific proposals listed below, there is a general +exemption allowing new ‘-3’ warnings to be added in any Python 2.7 +maintenance release. + +* Menu: + +* Two new environment variables for debug mode:: +* PEP 434; IDLE Enhancement Exception for All Branches: PEP 434 IDLE Enhancement Exception for All Branches. +* PEP 466; Network Security Enhancements for Python 2.7: PEP 466 Network Security Enhancements for Python 2 7. +* PEP 477; Backport ensurepip (PEP 453) to Python 2.7: PEP 477 Backport ensurepip PEP 453 to Python 2 7. +* PEP 476; Enabling certificate verification by default for stdlib http clients: PEP 476 Enabling certificate verification by default for stdlib http clients<2>. +* PEP 493; HTTPS verification migration tools for Python 2.7: PEP 493 HTTPS verification migration tools for Python 2 7. +* New make regen-all build target: New make regen-all build target<3>. +* Removal of make touch build target: Removal of make touch build target<3>. + + +File: python.info, Node: Two new environment variables for debug mode, Next: PEP 434 IDLE Enhancement Exception for All Branches, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.1 Two new environment variables for debug mode +...................................................... + +In debug mode, the ‘[xxx refs]’ statistic is not written by default, the +‘PYTHONSHOWREFCOUNT’ environment variable now must also be set. +(Contributed by Victor Stinner; bpo-31733(1).) + +When Python is compiled with ‘COUNT_ALLOC’ defined, allocation counts +are no longer dumped by default anymore: the ‘PYTHONSHOWALLOCCOUNT’ +environment variable must now also be set. Moreover, allocation counts +are now dumped into stderr, rather than stdout. (Contributed by Victor +Stinner; bpo-31692(2).) + +New in version 2.7.15. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue31733 + + (2) https://bugs.python.org/issue31692 + + +File: python.info, Node: PEP 434 IDLE Enhancement Exception for All Branches, Next: PEP 466 Network Security Enhancements for Python 2 7, Prev: Two new environment variables for debug mode, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.2 PEP 434: IDLE Enhancement Exception for All Branches +.............................................................. + +PEP 434(1) describes a general exemption for changes made to the IDLE +development environment shipped along with Python. This exemption makes +it possible for the IDLE developers to provide a more consistent user +experience across all supported versions of Python 2 and 3. + +For details of any IDLE changes, refer to the NEWS file for the specific +release. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0434 + + +File: python.info, Node: PEP 466 Network Security Enhancements for Python 2 7, Next: PEP 477 Backport ensurepip PEP 453 to Python 2 7, Prev: PEP 434 IDLE Enhancement Exception for All Branches, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.3 PEP 466: Network Security Enhancements for Python 2.7 +............................................................... + +PEP 466(1) describes a number of network security enhancement proposals +that have been approved for inclusion in Python 2.7 maintenance +releases, with the first of those changes appearing in the Python 2.7.7 +release. + +PEP 466(2) related features added in Python 2.7.7: + + * *note hmac.compare_digest(): a0c. was backported from Python 3 to + make a timing attack resistant comparison operation available to + Python 2 applications. (Contributed by Alex Gaynor; bpo-21306(3).) + + * OpenSSL 1.0.1g was upgraded in the official Windows installers + published on python.org. (Contributed by Zachary Ware; + bpo-21462(4).) + +PEP 466(5) related features added in Python 2.7.8: + + * *note hashlib.pbkdf2_hmac(): 7e6. was backported from Python 3 to + make a hashing algorithm suitable for secure password storage + broadly available to Python 2 applications. (Contributed by Alex + Gaynor; bpo-21304(6).) + + * OpenSSL 1.0.1h was upgraded for the official Windows installers + published on python.org. (contributed by Zachary Ware in + bpo-21671(7) for CVE-2014-0224) + +PEP 466(8) related features added in Python 2.7.9: + + * Most of Python 3.4’s *note ssl: f3. module was backported. This + means *note ssl: f3. now supports Server Name Indication, TLS1.x + settings, access to the platform certificate store, the *note + SSLContext: 3e4. class, and other features. (Contributed by Alex + Gaynor and David Reid; bpo-21308(9).) + + Refer to the “Version added: 2.7.9” notes in the module + documentation for specific details. + + * *note os.urandom(): 4d1. was changed to cache a file descriptor to + ‘/dev/urandom’ instead of reopening ‘/dev/urandom’ on every call. + (Contributed by Alex Gaynor; bpo-21305(10).) + + * *note hashlib.algorithms_guaranteed: cfc. and *note + hashlib.algorithms_available: cfd. were backported from Python 3 to + make it easier for Python 2 applications to select the strongest + available hash algorithm. (Contributed by Alex Gaynor in + bpo-21307(11)) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0466 + + (2) https://www.python.org/dev/peps/pep-0466 + + (3) https://bugs.python.org/issue21306 + + (4) https://bugs.python.org/issue21462 + + (5) https://www.python.org/dev/peps/pep-0466 + + (6) https://bugs.python.org/issue21304 + + (7) https://bugs.python.org/issue21671 + + (8) https://www.python.org/dev/peps/pep-0466 + + (9) https://bugs.python.org/issue21308 + + (10) https://bugs.python.org/issue21305 + + (11) https://bugs.python.org/issue21307 + + +File: python.info, Node: PEP 477 Backport ensurepip PEP 453 to Python 2 7, Next: PEP 476 Enabling certificate verification by default for stdlib http clients<2>, Prev: PEP 466 Network Security Enhancements for Python 2 7, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.4 PEP 477: Backport ensurepip (PEP 453) to Python 2.7 +............................................................. + +PEP 477(1) approves the inclusion of the PEP 453(2) ensurepip module and +the improved documentation that was enabled by it in the Python 2.7 +maintenance releases, appearing first in the Python 2.7.9 release. + +* Menu: + +* Bootstrapping pip By Default: Bootstrapping pip By Default<2>. +* Documentation Changes: Documentation Changes<2>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0477 + + (2) https://www.python.org/dev/peps/pep-0453 + + +File: python.info, Node: Bootstrapping pip By Default<2>, Next: Documentation Changes<2>, Up: PEP 477 Backport ensurepip PEP 453 to Python 2 7 + +1.10.15.5 Bootstrapping pip By Default +...................................... + +The new *note ensurepip: 7a. module (defined in PEP 453(1)) provides a +standard cross-platform mechanism to bootstrap the pip installer into +Python installations. The version of ‘pip’ included with Python 2.7.9 +is ‘pip’ 1.5.6, and future 2.7.x maintenance releases will update the +bundled version to the latest version of ‘pip’ that is available at the +time of creating the release candidate. + +By default, the commands ‘pip’, ‘pipX’ and ‘pipX.Y’ will be installed on +all platforms (where X.Y stands for the version of the Python +installation), along with the ‘pip’ Python package and its dependencies. + +For CPython *note source builds on POSIX systems: 7f3, the ‘make +install’ and ‘make altinstall’ commands do not bootstrap ‘pip’ by +default. This behaviour can be controlled through configure options, +and overridden through Makefile options. + +On Windows and Mac OS X, the CPython installers now default to +installing ‘pip’ along with CPython itself (users may opt out of +installing it during the installation process). Window users will need +to opt in to the automatic ‘PATH’ modifications to have ‘pip’ available +from the command line by default, otherwise it can still be accessed +through the Python launcher for Windows as ‘py -m pip’. + +As discussed in the PEP(2), platform packagers may choose not to install +these commands by default, as long as, when invoked, they provide clear +and simple directions on how to install them on that platform (usually +using the system package manager). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0453 + + (2) +https://www.python.org/dev/peps/pep-0477/#disabling-ensurepip-by-downstream-distributors + + +File: python.info, Node: Documentation Changes<2>, Prev: Bootstrapping pip By Default<2>, Up: PEP 477 Backport ensurepip PEP 453 to Python 2 7 + +1.10.15.6 Documentation Changes +............................... + +As part of this change, the *note Installing Python Modules: 7f5. and +*note Distributing Python Modules: 7f6. sections of the documentation +have been completely redesigned as short getting started and FAQ +documents. Most packaging documentation has now been moved out to the +Python Packaging Authority maintained Python Packaging User Guide(1) and +the documentation of the individual projects. + +However, as this migration is currently still incomplete, the legacy +versions of those guides remaining available as *note Installing Python +Modules (Legacy version): 7f7. and *note Distributing Python Modules +(Legacy version): 7f8. + +See also +........ + +PEP 453(2) – Explicit bootstrapping of pip in Python installations + + PEP written by Donald Stufft and Nick Coghlan, implemented by + Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily. + + ---------- Footnotes ---------- + + (1) http://packaging.python.org + + (2) https://www.python.org/dev/peps/pep-0453 + + +File: python.info, Node: PEP 476 Enabling certificate verification by default for stdlib http clients<2>, Next: PEP 493 HTTPS verification migration tools for Python 2 7, Prev: PEP 477 Backport ensurepip PEP 453 to Python 2 7, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.7 PEP 476: Enabling certificate verification by default for stdlib http clients +....................................................................................... + +PEP 476(1) updated ‘httplib’ and modules which use it, such as ‘urllib2’ +and ‘xmlrpclib’, to now verify that the server presents a certificate +which is signed by a Certificate Authority in the platform trust store +and whose hostname matches the hostname being requested by default, +significantly improving security for many applications. This change was +made in the Python 2.7.9 release. + +For applications which require the old previous behavior, they can pass +an alternate context: + + import urllib2 + import ssl + + # This disables all verification + context = ssl._create_unverified_context() + + # This allows using a specific certificate for the host, which doesn't need + # to be in the trust store + context = ssl.create_default_context(cafile="/path/to/file.crt") + + urllib2.urlopen("https://invalid-cert", context=context) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0476 + + +File: python.info, Node: PEP 493 HTTPS verification migration tools for Python 2 7, Next: New make regen-all build target<3>, Prev: PEP 476 Enabling certificate verification by default for stdlib http clients<2>, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.8 PEP 493: HTTPS verification migration tools for Python 2.7 +.................................................................... + +PEP 493(1) provides additional migration tools to support a more +incremental infrastructure upgrade process for environments containing +applications and services relying on the historically permissive +processing of server certificates when establishing client HTTPS +connections. These additions were made in the Python 2.7.12 release. + +These tools are intended for use in cases where affected applications +and services can’t be modified to explicitly pass a more permissive SSL +context when establishing the connection. + +For applications and services which can’t be modified at all, the new +‘PYTHONHTTPSVERIFY’ environment variable may be set to ‘0’ to revert an +entire Python process back to the default permissive behaviour of Python +2.7.8 and earlier. + +For cases where the connection establishment code can’t be modified, but +the overall application can be, the new +‘ssl._https_verify_certificates()’ function can be used to adjust the +default behaviour at runtime. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0493 + + +File: python.info, Node: New make regen-all build target<3>, Next: Removal of make touch build target<3>, Prev: PEP 493 HTTPS verification migration tools for Python 2 7, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.9 New ‘make regen-all’ build target +........................................... + +To simplify cross-compilation, and to ensure that CPython can reliably +be compiled without requiring an existing version of Python to already +be available, the autotools-based build system no longer attempts to +implicitly recompile generated files based on file modification times. + +Instead, a new ‘make regen-all’ command has been added to force +regeneration of these files when desired (e.g. after an initial version +of Python has already been built based on the pregenerated versions). + +More selective regeneration targets are also defined - see +Makefile.pre.in(1) for details. + +(Contributed by Victor Stinner in bpo-23404(2).) + +New in version 2.7.14. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Makefile.pre.in + + (2) https://bugs.python.org/issue23404 + + +File: python.info, Node: Removal of make touch build target<3>, Prev: New make regen-all build target<3>, Up: New Features Added to Python 2 7 Maintenance Releases + +1.10.15.10 Removal of ‘make touch’ build target +............................................... + +The ‘make touch’ build target previously used to request implicit +regeneration of generated files by updating their modification times has +been removed. + +It has been replaced by the new ‘make regen-all’ target. + +(Contributed by Victor Stinner in bpo-23404(1).) + +Changed in version 2.7.14. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue23404 + + +File: python.info, Node: Acknowledgements, Prev: New Features Added to Python 2 7 Maintenance Releases, Up: What’s New in Python 2 7 + +1.10.16 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Nick Coghlan, Philip Jenvey, Ryan Lovett, R. David Murray, Hugh +Secker-Walker. + + +File: python.info, Node: What’s New in Python 2 6, Next: What’s New in Python 2 5, Prev: What’s New in Python 2 7, Up: What’s New in Python + +1.11 What’s New in Python 2.6 +============================= + + +Author: A.M. Kuchling (amk at amk.ca) + +This article explains the new features in Python 2.6, released on +October 1 2008. The release schedule is described in PEP 361(1). + +The major theme of Python 2.6 is preparing the migration path to Python +3.0, a major redesign of the language. Whenever possible, Python 2.6 +incorporates new features and syntax from 3.0 while remaining compatible +with existing code by not removing older features or syntax. When it’s +not possible to do that, Python 2.6 tries to do what it can, adding +compatibility functions in a ‘future_builtins’ module and a ‘-3’ switch +to warn about usages that will become unsupported in 3.0. + +Some significant new packages have been added to the standard library, +such as the *note multiprocessing: b7. and *note json: a4. modules, but +there aren’t many new features that aren’t related to Python 3.0 in some +way. + +Python 2.6 also sees a number of improvements and bugfixes throughout +the source. A search through the change logs finds there were 259 +patches applied and 612 bugs fixed between Python 2.5 and 2.6. Both +figures are likely to be underestimates. + +This article doesn’t attempt to provide a complete specification of the +new features, but instead provides a convenient overview. For full +details, you should refer to the documentation for Python 2.6. If you +want to understand the rationale for the design and implementation, +refer to the PEP for a particular new feature. Whenever possible, +“What’s New in Python” links to the bug/patch item for each change. + +* Menu: + +* Python 3.0: Python 3 0. +* Changes to the Development Process:: +* PEP 343; The ‘with’ statement: PEP 343 The ‘with’ statement. +* PEP 366; Explicit Relative Imports From a Main Module: PEP 366 Explicit Relative Imports From a Main Module. +* PEP 370; Per-user site-packages Directory: PEP 370 Per-user site-packages Directory. +* PEP 371; The multiprocessing Package: PEP 371 The multiprocessing Package. +* PEP 3101; Advanced String Formatting: PEP 3101 Advanced String Formatting. +* PEP 3105; print As a Function: PEP 3105 print As a Function. +* PEP 3110; Exception-Handling Changes: PEP 3110 Exception-Handling Changes. +* PEP 3112; Byte Literals: PEP 3112 Byte Literals. +* PEP 3116; New I/O Library: PEP 3116 New I/O Library. +* PEP 3118; Revised Buffer Protocol: PEP 3118 Revised Buffer Protocol. +* PEP 3119; Abstract Base Classes: PEP 3119 Abstract Base Classes. +* PEP 3127; Integer Literal Support and Syntax: PEP 3127 Integer Literal Support and Syntax. +* PEP 3129; Class Decorators: PEP 3129 Class Decorators. +* PEP 3141; A Type Hierarchy for Numbers: PEP 3141 A Type Hierarchy for Numbers. +* Other Language Changes: Other Language Changes<10>. +* New and Improved Modules: New and Improved Modules<2>. +* Deprecations and Removals:: +* Build and C API Changes: Build and C API Changes<9>. +* Porting to Python 2.6: Porting to Python 2 6. +* Acknowledgements: Acknowledgements<2>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0361 + + +File: python.info, Node: Python 3 0, Next: Changes to the Development Process, Up: What’s New in Python 2 6 + +1.11.1 Python 3.0 +----------------- + +The development cycle for Python versions 2.6 and 3.0 was synchronized, +with the alpha and beta releases for both versions being made on the +same days. The development of 3.0 has influenced many features in 2.6. + +Python 3.0 is a far-ranging redesign of Python that breaks compatibility +with the 2.x series. This means that existing Python code will need +some conversion in order to run on Python 3.0. However, not all the +changes in 3.0 necessarily break compatibility. In cases where new +features won’t cause existing code to break, they’ve been backported to +2.6 and are described in this document in the appropriate place. Some +of the 3.0-derived features are: + + * A *note __complex__(): 18d. method for converting objects to a + complex number. + + * Alternate syntax for catching exceptions: ‘except TypeError as + exc’. + + * The addition of *note functools.reduce(): c6f. as a synonym for the + built-in ‘reduce()’ function. + +Python 3.0 adds several new built-in functions and changes the semantics +of some existing builtins. Functions that are new in 3.0 such as *note +bin(): c40. have simply been added to Python 2.6, but existing builtins +haven’t been changed; instead, the ‘future_builtins’ module has versions +with the new 3.0 semantics. Code written to be compatible with 3.0 can +do ‘from future_builtins import hex, map’ as necessary. + +A new command-line switch, ‘-3’, enables warnings about features that +will be removed in Python 3.0. You can run code with this switch to see +how much work will be necessary to port code to 3.0. The value of this +switch is available to Python code as the boolean variable +‘sys.py3kwarning’, and to C extension code as ‘Py_Py3kWarningFlag’. + +See also +........ + +The 3xxx series of PEPs, which contains proposals for Python 3.0. PEP +3000(1) describes the development process for Python 3.0. Start with +PEP 3100(2) that describes the general goals for Python 3.0, and then +explore the higher-numbered PEPS that propose specific features. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3000 + + (2) https://www.python.org/dev/peps/pep-3100 + + +File: python.info, Node: Changes to the Development Process, Next: PEP 343 The ‘with’ statement, Prev: Python 3 0, Up: What’s New in Python 2 6 + +1.11.2 Changes to the Development Process +----------------------------------------- + +While 2.6 was being developed, the Python development process underwent +two significant changes: we switched from SourceForge’s issue tracker to +a customized Roundup installation, and the documentation was converted +from LaTeX to reStructuredText. + +* Menu: + +* New Issue Tracker; Roundup: New Issue Tracker Roundup. +* New Documentation Format; reStructuredText Using Sphinx: New Documentation Format reStructuredText Using Sphinx. + + +File: python.info, Node: New Issue Tracker Roundup, Next: New Documentation Format reStructuredText Using Sphinx, Up: Changes to the Development Process + +1.11.2.1 New Issue Tracker: Roundup +................................... + +For a long time, the Python developers had been growing increasingly +annoyed by SourceForge’s bug tracker. SourceForge’s hosted solution +doesn’t permit much customization; for example, it wasn’t possible to +customize the life cycle of issues. + +The infrastructure committee of the Python Software Foundation therefore +posted a call for issue trackers, asking volunteers to set up different +products and import some of the bugs and patches from SourceForge. Four +different trackers were examined: Jira(1), Launchpad(2), Roundup(3), and +Trac(4). The committee eventually settled on Jira and Roundup as the +two candidates. Jira is a commercial product that offers no-cost hosted +instances to free-software projects; Roundup is an open-source project +that requires volunteers to administer it and a server to host it. + +After posting a call for volunteers, a new Roundup installation was set +up at ‘https://bugs.python.org’. One installation of Roundup can host +multiple trackers, and this server now also hosts issue trackers for +Jython and for the Python web site. It will surely find other uses in +the future. Where possible, this edition of “What’s New in Python” +links to the bug/patch item for each change. + +Hosting of the Python bug tracker is kindly provided by Upfront +Systems(5) of Stellenbosch, South Africa. Martin von Löwis put a lot of +effort into importing existing bugs and patches from SourceForge; his +scripts for this import operation are at +‘http://svn.python.org/view/tracker/importer/’ and may be useful to +other projects wishing to move from SourceForge to Roundup. + +See also +........ + +‘https://bugs.python.org’ + + The Python bug tracker. + +‘http://bugs.jython.org’: + + The Jython bug tracker. + +‘http://roundup.sourceforge.net/’ + + Roundup downloads and documentation. + +‘http://svn.python.org/view/tracker/importer/’ + + Martin von Löwis’s conversion scripts. + + ---------- Footnotes ---------- + + (1) https://www.atlassian.com/software/jira/ + + (2) https://launchpad.net/ + + (3) http://roundup.sourceforge.net/ + + (4) https://trac.edgewall.org/ + + (5) http://www.upfrontsoftware.co.za + + +File: python.info, Node: New Documentation Format reStructuredText Using Sphinx, Prev: New Issue Tracker Roundup, Up: Changes to the Development Process + +1.11.2.2 New Documentation Format: reStructuredText Using Sphinx +................................................................ + +The Python documentation was written using LaTeX since the project +started around 1989. In the 1980s and early 1990s, most documentation +was printed out for later study, not viewed online. LaTeX was widely +used because it provided attractive printed output while remaining +straightforward to write once the basic rules of the markup were +learned. + +Today LaTeX is still used for writing publications destined for +printing, but the landscape for programming tools has shifted. We no +longer print out reams of documentation; instead, we browse through it +online and HTML has become the most important format to support. +Unfortunately, converting LaTeX to HTML is fairly complicated and Fred +L. Drake Jr., the long-time Python documentation editor, spent a lot of +time maintaining the conversion process. Occasionally people would +suggest converting the documentation into SGML and later XML, but +performing a good conversion is a major task and no one ever committed +the time required to finish the job. + +During the 2.6 development cycle, Georg Brandl put a lot of effort into +building a new toolchain for processing the documentation. The +resulting package is called Sphinx, and is available from +‘http://sphinx-doc.org/’. + +Sphinx concentrates on HTML output, producing attractively styled and +modern HTML; printed output is still supported through conversion to +LaTeX. The input format is reStructuredText, a markup syntax supporting +custom extensions and directives that is commonly used in the Python +community. + +Sphinx is a standalone package that can be used for writing, and almost +two dozen other projects (listed on the Sphinx web site(1)) have adopted +Sphinx as their documentation tool. + +See also +........ + +Documenting Python(2) + + Describes how to write for Python’s documentation. + +Sphinx(3) + + Documentation and code for the Sphinx toolchain. + +Docutils(4) + + The underlying reStructuredText parser and toolset. + + ---------- Footnotes ---------- + + (1) https://www.sphinx-doc.org/en/master/examples.html + + (2) https://devguide.python.org/documenting/ + + (3) http://sphinx-doc.org/ + + (4) http://docutils.sourceforge.net + + +File: python.info, Node: PEP 343 The ‘with’ statement, Next: PEP 366 Explicit Relative Imports From a Main Module, Prev: Changes to the Development Process, Up: What’s New in Python 2 6 + +1.11.3 PEP 343: The ‘with’ statement +------------------------------------ + +The previous version, Python 2.5, added the ‘*note with: 6e9.’ statement +as an optional feature, to be enabled by a ‘from __future__ import +with_statement’ directive. In 2.6 the statement no longer needs to be +specially enabled; this means that ‘with’ is now always a keyword. The +rest of this section is a copy of the corresponding section from the +“What’s New in Python 2.5” document; if you’re familiar with the +‘‘with’’ statement from Python 2.5, you can skip this section. + +The ‘*note with: 6e9.’ statement clarifies code that previously would +use ‘try...finally’ blocks to ensure that clean-up code is executed. In +this section, I’ll discuss the statement as it will commonly be used. +In the next section, I’ll examine the implementation details and show +how to write objects for use with this statement. + +The ‘*note with: 6e9.’ statement is a control-flow structure whose basic +structure is: + + with expression [as variable]: + with-block + +The expression is evaluated, and it should result in an object that +supports the context management protocol (that is, has *note +__enter__(): c95. and *note __exit__(): c96. methods). + +The object’s *note __enter__(): c95. is called before `with-block' is +executed and therefore can run set-up code. It also may return a value +that is bound to the name `variable', if given. (Note carefully that +`variable' is `not' assigned the result of `expression'.) + +After execution of the `with-block' is finished, the object’s *note +__exit__(): c96. method is called, even if the block raised an +exception, and can therefore run clean-up code. + +Some standard Python objects now support the context management protocol +and can be used with the ‘*note with: 6e9.’ statement. File objects are +one example: + + with open('/etc/passwd', 'r') as f: + for line in f: + print line + ... more processing code ... + +After this statement has executed, the file object in `f' will have been +automatically closed, even if the *note for: c30. loop raised an +exception part-way through the block. + + Note: In this case, `f' is the same object created by *note open(): + 4f0, because ‘file.__enter__()’ returns `self'. + +The *note threading: 109. module’s locks and condition variables also +support the ‘*note with: 6e9.’ statement: + + lock = threading.Lock() + with lock: + # Critical section of code + ... + +The lock is acquired before the block is executed and always released +once the block is complete. + +The ‘localcontext()’ function in the *note decimal: 36. module makes it +easy to save and restore the current decimal context, which encapsulates +the desired precision and rounding characteristics for computations: + + from decimal import Decimal, Context, localcontext + + # Displays with default precision of 28 digits + v = Decimal('578') + print v.sqrt() + + with localcontext(Context(prec=16)): + # All code in this block uses a precision of 16 digits. + # The original context is restored on exiting the block. + print v.sqrt() + +* Menu: + +* Writing Context Managers:: +* The contextlib module:: + + +File: python.info, Node: Writing Context Managers, Next: The contextlib module, Up: PEP 343 The ‘with’ statement + +1.11.3.1 Writing Context Managers +................................. + +Under the hood, the ‘*note with: 6e9.’ statement is fairly complicated. +Most people will only use ‘‘with’’ in company with existing objects and +don’t need to know these details, so you can skip the rest of this +section if you like. Authors of new objects will need to understand the +details of the underlying implementation and should keep reading. + +A high-level explanation of the context management protocol is: + + * The expression is evaluated and should result in an object called a + “context manager”. The context manager must have *note + __enter__(): c95. and *note __exit__(): c96. methods. + + * The context manager’s *note __enter__(): c95. method is called. + The value returned is assigned to `VAR'. If no ‘as VAR’ clause is + present, the value is simply discarded. + + * The code in `BLOCK' is executed. + + * If `BLOCK' raises an exception, the context manager’s *note + __exit__(): c96. method is called with three arguments, the + exception details (‘type, value, traceback’, the same values + returned by *note sys.exc_info(): c5f, which can also be ‘None’ if + no exception occurred). The method’s return value controls whether + an exception is re-raised: any false value re-raises the exception, + and ‘True’ will result in suppressing it. You’ll only rarely want + to suppress the exception, because if you do the author of the code + containing the ‘*note with: 6e9.’ statement will never realize + anything went wrong. + + * If `BLOCK' didn’t raise an exception, the *note __exit__(): c96. + method is still called, but `type', `value', and `traceback' are + all ‘None’. + +Let’s think through an example. I won’t present detailed code but will +only sketch the methods necessary for a database that supports +transactions. + +(For people unfamiliar with database terminology: a set of changes to +the database are grouped into a transaction. Transactions can be either +committed, meaning that all the changes are written into the database, +or rolled back, meaning that the changes are all discarded and the +database is unchanged. See any database textbook for more information.) + +Let’s assume there’s an object representing a database connection. Our +goal will be to let the user write code like this: + + db_connection = DatabaseConnection() + with db_connection as cursor: + cursor.execute('insert into ...') + cursor.execute('delete from ...') + # ... more operations ... + +The transaction should be committed if the code in the block runs +flawlessly or rolled back if there’s an exception. Here’s the basic +interface for ‘DatabaseConnection’ that I’ll assume: + + class DatabaseConnection: + # Database interface + def cursor(self): + "Returns a cursor object and starts a new transaction" + def commit(self): + "Commits current transaction" + def rollback(self): + "Rolls back current transaction" + +The *note __enter__(): c95. method is pretty easy, having only to start +a new transaction. For this application the resulting cursor object +would be a useful result, so the method will return it. The user can +then add ‘as cursor’ to their ‘*note with: 6e9.’ statement to bind the +cursor to a variable name. + + class DatabaseConnection: + ... + def __enter__(self): + # Code to start a new transaction + cursor = self.cursor() + return cursor + +The *note __exit__(): c96. method is the most complicated because it’s +where most of the work has to be done. The method has to check if an +exception occurred. If there was no exception, the transaction is +committed. The transaction is rolled back if there was an exception. + +In the code below, execution will just fall off the end of the function, +returning the default value of ‘None’. ‘None’ is false, so the +exception will be re-raised automatically. If you wished, you could be +more explicit and add a *note return: 190. statement at the marked +location. + + class DatabaseConnection: + ... + def __exit__(self, type, value, tb): + if tb is None: + # No exception, so commit + self.commit() + else: + # Exception occurred, so rollback. + self.rollback() + # return False + + +File: python.info, Node: The contextlib module, Prev: Writing Context Managers, Up: PEP 343 The ‘with’ statement + +1.11.3.2 The contextlib module +.............................. + +The *note contextlib: 24. module provides some functions and a decorator +that are useful when writing objects for use with the ‘*note with: 6e9.’ +statement. + +The decorator is called ‘contextmanager()’, and lets you write a single +generator function instead of defining a new class. The generator +should yield exactly one value. The code up to the *note yield: 18f. +will be executed as the *note __enter__(): c95. method, and the value +yielded will be the method’s return value that will get bound to the +variable in the ‘*note with: 6e9.’ statement’s ‘as’ clause, if any. The +code after the ‘yield’ will be executed in the *note __exit__(): c96. +method. Any exception raised in the block will be raised by the ‘yield’ +statement. + +Using this decorator, our database example from the previous section +could be written as: + + from contextlib import contextmanager + + @contextmanager + def db_transaction(connection): + cursor = connection.cursor() + try: + yield cursor + except: + connection.rollback() + raise + else: + connection.commit() + + db = DatabaseConnection() + with db_transaction(db) as cursor: + ... + +The *note contextlib: 24. module also has a ‘nested(mgr1, mgr2, ...)’ +function that combines a number of context managers so you don’t need to +write nested ‘*note with: 6e9.’ statements. In this example, the single +‘‘with’’ statement both starts a database transaction and acquires a +thread lock: + + lock = threading.Lock() + with nested (db_transaction(db), lock) as (cursor, locked): + ... + +Finally, the ‘closing()’ function returns its argument so that it can be +bound to a variable, and calls the argument’s ‘.close()’ method at the +end of the block. + + import urllib, sys + from contextlib import closing + + with closing(urllib.urlopen('http://www.yahoo.com')) as f: + for line in f: + sys.stdout.write(line) + +See also +........ + +PEP 343(1) - The “with” statement + + PEP written by Guido van Rossum and Nick Coghlan; implemented by + Mike Bland, Guido van Rossum, and Neal Norwitz. The PEP shows the + code generated for a ‘*note with: 6e9.’ statement, which can be + helpful in learning how the statement works. + +The documentation for the *note contextlib: 24. module. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0343 + + +File: python.info, Node: PEP 366 Explicit Relative Imports From a Main Module, Next: PEP 370 Per-user site-packages Directory, Prev: PEP 343 The ‘with’ statement, Up: What’s New in Python 2 6 + +1.11.4 PEP 366: Explicit Relative Imports From a Main Module +------------------------------------------------------------ + +Python’s *note -m: 337. switch allows running a module as a script. +When you ran a module that was located inside a package, relative +imports didn’t work correctly. + +The fix for Python 2.6 adds a *note __package__: 955. attribute to +modules. When this attribute is present, relative imports will be +relative to the value of this attribute instead of the *note __name__: +d12. attribute. + +PEP 302-style importers can then set *note __package__: 955. as +necessary. The *note runpy: e2. module that implements the *note -m: +337. switch now does this, so relative imports will now work correctly +in scripts running from inside a package. + + +File: python.info, Node: PEP 370 Per-user site-packages Directory, Next: PEP 371 The multiprocessing Package, Prev: PEP 366 Explicit Relative Imports From a Main Module, Up: What’s New in Python 2 6 + +1.11.5 PEP 370: Per-user ‘site-packages’ Directory +-------------------------------------------------- + +When you run Python, the module search path ‘sys.path’ usually includes +a directory whose path ends in ‘"site-packages"’. This directory is +intended to hold locally-installed packages available to all users using +a machine or a particular site installation. + +Python 2.6 introduces a convention for user-specific site directories. +The directory varies depending on the platform: + + * Unix and Mac OS X: ‘~/.local/’ + + * Windows: ‘%APPDATA%/Python’ + +Within this directory, there will be version-specific subdirectories, +such as ‘lib/python2.6/site-packages’ on Unix/Mac OS and +‘Python26/site-packages’ on Windows. + +If you don’t like the default directory, it can be overridden by an +environment variable. *note PYTHONUSERBASE: d14. sets the root +directory used for all Python versions supporting this feature. On +Windows, the directory for application-specific data can be changed by +setting the ‘APPDATA’ environment variable. You can also modify the +‘site.py’ file for your Python installation. + +The feature can be disabled entirely by running Python with the *note +-s: d15. option or setting the *note PYTHONNOUSERSITE: d16. environment +variable. + +See also +........ + +PEP 370(1) - Per-user ‘site-packages’ Directory + + PEP written and implemented by Christian Heimes. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0370 + + +File: python.info, Node: PEP 371 The multiprocessing Package, Next: PEP 3101 Advanced String Formatting, Prev: PEP 370 Per-user site-packages Directory, Up: What’s New in Python 2 6 + +1.11.6 PEP 371: The ‘multiprocessing’ Package +--------------------------------------------- + +The new *note multiprocessing: b7. package lets Python programs create +new processes that will perform a computation and return a result to the +parent. The parent and child processes can communicate using queues and +pipes, synchronize their operations using locks and semaphores, and can +share simple arrays of data. + +The *note multiprocessing: b7. module started out as an exact emulation +of the *note threading: 109. module using processes instead of threads. +That goal was discarded along the path to Python 2.6, but the general +approach of the module is still similar. The fundamental class is the +‘Process’, which is passed a callable object and a collection of +arguments. The ‘start()’ method sets the callable running in a +subprocess, after which you can call the ‘is_alive()’ method to check +whether the subprocess is still running and the ‘join()’ method to wait +for the process to exit. + +Here’s a simple example where the subprocess will calculate a factorial. +The function doing the calculation is written strangely so that it takes +significantly longer when the input argument is a multiple of 4. + + import time + from multiprocessing import Process, Queue + + + def factorial(queue, N): + "Compute a factorial." + # If N is a multiple of 4, this function will take much longer. + if (N % 4) == 0: + time.sleep(.05 * N/4) + + # Calculate the result + fact = 1L + for i in range(1, N+1): + fact = fact * i + + # Put the result on the queue + queue.put(fact) + + if __name__ == '__main__': + queue = Queue() + + N = 5 + + p = Process(target=factorial, args=(queue, N)) + p.start() + p.join() + + result = queue.get() + print 'Factorial', N, '=', result + +A *note Queue: d18. is used to communicate the result of the factorial. +The *note Queue: d18. object is stored in a global variable. The child +process will use the value of the variable when the child was created; +because it’s a *note Queue: d18, parent and child can use the object to +communicate. (If the parent were to change the value of the global +variable, the child’s value would be unaffected, and vice versa.) + +Two other classes, ‘Pool’ and ‘Manager’, provide higher-level +interfaces. ‘Pool’ will create a fixed number of worker processes, and +requests can then be distributed to the workers by calling ‘apply()’ or +‘apply_async()’ to add a single request, and *note map(): c2d. or +‘map_async()’ to add a number of requests. The following code uses a +‘Pool’ to spread requests across 5 worker processes and retrieve a list +of results: + + from multiprocessing import Pool + + def factorial(N, dictionary): + "Compute a factorial." + ... + p = Pool(5) + result = p.map(factorial, range(1, 1000, 10)) + for v in result: + print v + +This produces the following output: + + 1 + 39916800 + 51090942171709440000 + 8222838654177922817725562880000000 + 33452526613163807108170062053440751665152000000000 + ... + +The other high-level interface, the ‘Manager’ class, creates a separate +server process that can hold master copies of Python data structures. +Other processes can then access and modify these data structures using +proxy objects. The following example creates a shared dictionary by +calling the *note dict(): 1b8. method; the worker processes then insert +values into the dictionary. (Locking is not done for you automatically, +which doesn’t matter in this example. ‘Manager’’s methods also include +‘Lock()’, ‘RLock()’, and ‘Semaphore()’ to create shared locks.) + + import time + from multiprocessing import Pool, Manager + + def factorial(N, dictionary): + "Compute a factorial." + # Calculate the result + fact = 1L + for i in range(1, N+1): + fact = fact * i + + # Store result in dictionary + dictionary[N] = fact + + if __name__ == '__main__': + p = Pool(5) + mgr = Manager() + d = mgr.dict() # Create shared dictionary + + # Run tasks using the pool + for N in range(1, 1000, 10): + p.apply_async(factorial, (N, d)) + + # Mark pool as closed -- no more tasks can be added. + p.close() + + # Wait for tasks to exit + p.join() + + # Output results + for k, v in sorted(d.items()): + print k, v + +This will produce the output: + + 1 1 + 11 39916800 + 21 51090942171709440000 + 31 8222838654177922817725562880000000 + 41 33452526613163807108170062053440751665152000000000 + 51 15511187532873822802242430164693032110632597200169861120000... + +See also +........ + +The documentation for the *note multiprocessing: b7. module. + +PEP 371(1) - Addition of the multiprocessing package + + PEP written by Jesse Noller and Richard Oudkerk; implemented by + Richard Oudkerk and Jesse Noller. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0371 + + +File: python.info, Node: PEP 3101 Advanced String Formatting, Next: PEP 3105 print As a Function, Prev: PEP 371 The multiprocessing Package, Up: What’s New in Python 2 6 + +1.11.7 PEP 3101: Advanced String Formatting +------------------------------------------- + +In Python 3.0, the ‘%’ operator is supplemented by a more powerful +string formatting method, *note format(): 4db. Support for the *note +str.format(): 4da. method has been backported to Python 2.6. + +In 2.6, both 8-bit and Unicode strings have a ‘.format()’ method that +treats the string as a template and takes the arguments to be formatted. +The formatting template uses curly brackets (‘{’, ‘}’) as special +characters: + + >>> # Substitute positional argument 0 into the string. + >>> "User ID: {0}".format("root") + 'User ID: root' + >>> # Use the named keyword arguments + >>> "User ID: {uid} Last seen: {last_login}".format( + ... uid="root", + ... last_login = "5 Mar 2008 07:20") + 'User ID: root Last seen: 5 Mar 2008 07:20' + +Curly brackets can be escaped by doubling them: + + >>> "Empty dict: {{}}".format() + "Empty dict: {}" + +Field names can be integers indicating positional arguments, such as +‘{0}’, ‘{1}’, etc. or names of keyword arguments. You can also supply +compound field names that read attributes or access dictionary keys: + + >>> import sys + >>> print 'Platform: {0.platform}\nPython version: {0.version}'.format(sys) + Platform: darwin + Python version: 2.6a1+ (trunk:61261M, Mar 5 2008, 20:29:41) + [GCC 4.0.1 (Apple Computer, Inc. build 5367)]' + + >>> import mimetypes + >>> 'Content-type: {0[.mp4]}'.format(mimetypes.types_map) + 'Content-type: video/mp4' + +Note that when using dictionary-style notation such as ‘[.mp4]’, you +don’t need to put any quotation marks around the string; it will look up +the value using ‘.mp4’ as the key. Strings beginning with a number will +be converted to an integer. You can’t write more complicated +expressions inside a format string. + +So far we’ve shown how to specify which field to substitute into the +resulting string. The precise formatting used is also controllable by +adding a colon followed by a format specifier. For example: + + >>> # Field 0: left justify, pad to 15 characters + >>> # Field 1: right justify, pad to 6 characters + >>> fmt = '{0:15} ${1:>6}' + >>> fmt.format('Registration', 35) + 'Registration $ 35' + >>> fmt.format('Tutorial', 50) + 'Tutorial $ 50' + >>> fmt.format('Banquet', 125) + 'Banquet $ 125' + +Format specifiers can reference other fields through nesting: + + >>> fmt = '{0:{1}}' + >>> width = 15 + >>> fmt.format('Invoice #1234', width) + 'Invoice #1234 ' + >>> width = 35 + >>> fmt.format('Invoice #1234', width) + 'Invoice #1234 ' + +The alignment of a field within the desired width can be specified: + +Character Effect + +---------------------------------------------------------------------- + +< (default) Left-align + + +> Right-align + + +^ Center + + += (For numeric types only) Pad after the sign. + + +Format specifiers can also include a presentation type, which controls +how the value is formatted. For example, floating-point numbers can be +formatted as a general number or in exponential notation: + + >>> '{0:g}'.format(3.75) + '3.75' + >>> '{0:e}'.format(3.75) + '3.750000e+00' + +A variety of presentation types are available. Consult the 2.6 +documentation for a *note complete list: d1a.; here’s a sample: + +‘b’ Binary. Outputs the number in base 2. + + +‘c’ Character. Converts the integer to the corresponding Unicode character + before printing. + + +‘d’ Decimal Integer. Outputs the number in base 10. + + +‘o’ Octal format. Outputs the number in base 8. + + +‘x’ Hex format. Outputs the number in base 16, using lower-case letters for + the digits above 9. + + +‘e’ Exponent notation. Prints the number in scientific notation using the + letter ‘e’ to indicate the exponent. + + +‘g’ General format. This prints the number as a fixed-point number, unless + the number is too large, in which case it switches to ‘e’ exponent + notation. + + +‘n’ Number. This is the same as ‘g’ (for floats) or ‘d’ (for integers), + except that it uses the current locale setting to insert the appropriate + number separator characters. + + +‘%’ Percentage. Multiplies the number by 100 and displays in fixed (‘f’) + format, followed by a percent sign. + + +Classes and types can define a *note __format__(): 94e. method to +control how they’re formatted. It receives a single argument, the +format specifier: + + def __format__(self, format_spec): + if isinstance(format_spec, unicode): + return unicode(str(self)) + else: + return str(self) + +There’s also a *note format(): 4db. builtin that will format a single +value. It calls the type’s *note __format__(): 94e. method with the +provided specifier: + + >>> format(75.6564, '.2f') + '75.66' + +See also +........ + +*note Format String Syntax: d1a. + + The reference documentation for format fields. + +PEP 3101(1) - Advanced String Formatting + + PEP written by Talin. Implemented by Eric Smith. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3101 + + +File: python.info, Node: PEP 3105 print As a Function, Next: PEP 3110 Exception-Handling Changes, Prev: PEP 3101 Advanced String Formatting, Up: What’s New in Python 2 6 + +1.11.8 PEP 3105: ‘print’ As a Function +-------------------------------------- + +The ‘print’ statement becomes the *note print(): 886. function in Python +3.0. Making *note print(): 886. a function makes it possible to replace +the function by doing ‘def print(...)’ or importing a new function from +somewhere else. + +Python 2.6 has a ‘__future__’ import that removes ‘print’ as language +syntax, letting you use the functional form instead. For example: + + >>> from __future__ import print_function + >>> print('# of entries', len(dictionary), file=sys.stderr) + +The signature of the new function is: + + def print(*args, sep=' ', end='\n', file=None) + +The parameters are: + + * `args': positional arguments whose values will be printed out. + + * `sep': the separator, which will be printed between arguments. + + * `end': the ending text, which will be printed after all of the + arguments have been output. + + * `file': the file object to which the output will be sent. + +See also +........ + +PEP 3105(1) - Make print a function + + PEP written by Georg Brandl. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3105 + + +File: python.info, Node: PEP 3110 Exception-Handling Changes, Next: PEP 3112 Byte Literals, Prev: PEP 3105 print As a Function, Up: What’s New in Python 2 6 + +1.11.9 PEP 3110: Exception-Handling Changes +------------------------------------------- + +One error that Python programmers occasionally make is writing the +following code: + + try: + ... + except TypeError, ValueError: # Wrong! + ... + +The author is probably trying to catch both *note TypeError: 192. and +*note ValueError: 1fb. exceptions, but this code actually does something +different: it will catch *note TypeError: 192. and bind the resulting +exception object to the local name ‘"ValueError"’. The *note +ValueError: 1fb. exception will not be caught at all. The correct code +specifies a tuple of exceptions: + + try: + ... + except (TypeError, ValueError): + ... + +This error happens because the use of the comma here is ambiguous: does +it indicate two different nodes in the parse tree, or a single node +that’s a tuple? + +Python 3.0 makes this unambiguous by replacing the comma with the word +“as”. To catch an exception and store the exception object in the +variable ‘exc’, you must write: + + try: + ... + except TypeError as exc: + ... + +Python 3.0 will only support the use of “as”, and therefore interprets +the first example as catching two different exceptions. Python 2.6 +supports both the comma and “as”, so existing code will continue to +work. We therefore suggest using “as” when writing new Python code that +will only be executed with 2.6. + +See also +........ + +PEP 3110(1) - Catching Exceptions in Python 3000 + + PEP written and implemented by Collin Winter. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3110 + + +File: python.info, Node: PEP 3112 Byte Literals, Next: PEP 3116 New I/O Library, Prev: PEP 3110 Exception-Handling Changes, Up: What’s New in Python 2 6 + +1.11.10 PEP 3112: Byte Literals +------------------------------- + +Python 3.0 adopts Unicode as the language’s fundamental string type and +denotes 8-bit literals differently, either as ‘b'string'’ or using a +*note bytes: 331. constructor. For future compatibility, Python 2.6 +adds *note bytes: 331. as a synonym for the *note str: 330. type, and it +also supports the ‘b''’ notation. + +The 2.6 *note str: 330. differs from 3.0’s *note bytes: 331. type in +various ways; most notably, the constructor is completely different. In +3.0, ‘bytes([65, 66, 67])’ is 3 elements long, containing the bytes +representing ‘ABC’; in 2.6, ‘bytes([65, 66, 67])’ returns the 12-byte +string representing the *note str(): 330. of the list. + +The primary use of *note bytes: 331. in 2.6 will be to write tests of +object type such as ‘isinstance(x, bytes)’. This will help the 2to3 +converter, which can’t tell whether 2.x code intends strings to contain +either characters or 8-bit bytes; you can now use either *note bytes: +331. or *note str: 330. to represent your intention exactly, and the +resulting code will also be correct in Python 3.0. + +There’s also a ‘__future__’ import that causes all string literals to +become Unicode strings. This means that ‘\u’ escape sequences can be +used to include Unicode characters: + + from __future__ import unicode_literals + + s = ('\u751f\u3080\u304e\u3000\u751f\u3054' + '\u3081\u3000\u751f\u305f\u307e\u3054') + + print len(s) # 12 Unicode characters + +At the C level, Python 3.0 will rename the existing 8-bit string type, +called ‘PyStringObject’ in Python 2.x, to *note PyBytesObject: d1e. +Python 2.6 uses ‘#define’ to support using the names *note +PyBytesObject(): d1e, *note PyBytes_Check(): d1f, *note +PyBytes_FromStringAndSize(): d20, and all the other functions and macros +used with strings. + +Instances of the *note bytes: 331. type are immutable just as strings +are. A new *note bytearray: 332. type stores a mutable sequence of +bytes: + + >>> bytearray([65, 66, 67]) + bytearray(b'ABC') + >>> b = bytearray(u'\u21ef\u3244', 'utf-8') + >>> b + bytearray(b'\xe2\x87\xaf\xe3\x89\x84') + >>> b[0] = '\xe3' + >>> b + bytearray(b'\xe3\x87\xaf\xe3\x89\x84') + >>> unicode(str(b), 'utf-8') + u'\u31ef \u3244' + +Byte arrays support most of the methods of string types, such as +‘startswith()’/‘endswith()’, ‘find()’/‘rfind()’, and some of the methods +of lists, such as ‘append()’, ‘pop()’, and ‘reverse()’. + + >>> b = bytearray('ABC') + >>> b.append('d') + >>> b.append(ord('e')) + >>> b + bytearray(b'ABCde') + +There’s also a corresponding C API, with *note PyByteArray_FromObject(): +d21, *note PyByteArray_FromStringAndSize(): d22, and various other +functions. + +See also +........ + +PEP 3112(1) - Bytes literals in Python 3000 + + PEP written by Jason Orendorff; backported to 2.6 by Christian + Heimes. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3112 + + +File: python.info, Node: PEP 3116 New I/O Library, Next: PEP 3118 Revised Buffer Protocol, Prev: PEP 3112 Byte Literals, Up: What’s New in Python 2 6 + +1.11.11 PEP 3116: New I/O Library +--------------------------------- + +Python’s built-in file objects support a number of methods, but +file-like objects don’t necessarily support all of them. Objects that +imitate files usually support ‘read()’ and ‘write()’, but they may not +support *note readline(): de, for example. Python 3.0 introduces a +layered I/O library in the *note io: a1. module that separates buffering +and text-handling features from the fundamental read and write +operations. + +There are three levels of abstract base classes provided by the *note +io: a1. module: + + * ‘RawIOBase’ defines raw I/O operations: ‘read()’, ‘readinto()’, + ‘write()’, ‘seek()’, ‘tell()’, ‘truncate()’, and ‘close()’. Most + of the methods of this class will often map to a single system + call. There are also ‘readable()’, ‘writable()’, and ‘seekable()’ + methods for determining what operations a given object will allow. + + Python 3.0 has concrete implementations of this class for files and + sockets, but Python 2.6 hasn’t restructured its file and socket + objects in this way. + + * ‘BufferedIOBase’ is an abstract base class that buffers data in + memory to reduce the number of system calls used, making I/O + processing more efficient. It supports all of the methods of + ‘RawIOBase’, and adds a ‘raw’ attribute holding the underlying raw + object. + + There are five concrete classes implementing this ABC. + ‘BufferedWriter’ and ‘BufferedReader’ are for objects that support + write-only or read-only usage that have a ‘seek()’ method for + random access. ‘BufferedRandom’ objects support read and write + access upon the same underlying stream, and ‘BufferedRWPair’ is for + objects such as TTYs that have both read and write operations + acting upon unconnected streams of data. The ‘BytesIO’ class + supports reading, writing, and seeking over an in-memory buffer. + + * ‘TextIOBase’: Provides functions for reading and writing strings + (remember, strings will be Unicode in Python 3.0), and supporting + *note universal newlines: 5f4. ‘TextIOBase’ defines the *note + readline(): de. method and supports iteration upon objects. + + There are two concrete implementations. ‘TextIOWrapper’ wraps a + buffered I/O object, supporting all of the methods for text I/O and + adding a ‘buffer’ attribute for access to the underlying object. + ‘StringIO’ simply buffers everything in memory without ever writing + anything to disk. + + (In Python 2.6, *note io.StringIO: 82d. is implemented in pure + Python, so it’s pretty slow. You should therefore stick with the + existing ‘StringIO’ module or ‘cStringIO’ for now. At some point + Python 3.0’s *note io: a1. module will be rewritten into C for + speed, and perhaps the C implementation will be backported to the + 2.x releases.) + +In Python 2.6, the underlying implementations haven’t been restructured +to build on top of the *note io: a1. module’s classes. The module is +being provided to make it easier to write code that’s forward-compatible +with 3.0, and to save developers the effort of writing their own +implementations of buffering and text I/O. + +See also +........ + +PEP 3116(1) - New I/O + + PEP written by Daniel Stutzbach, Mike Verdone, and Guido van + Rossum. Code by Guido van Rossum, Georg Brandl, Walter Doerwald, + Jeremy Hylton, Martin von Löwis, Tony Lownds, and others. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3116 + + +File: python.info, Node: PEP 3118 Revised Buffer Protocol, Next: PEP 3119 Abstract Base Classes, Prev: PEP 3116 New I/O Library, Up: What’s New in Python 2 6 + +1.11.12 PEP 3118: Revised Buffer Protocol +----------------------------------------- + +The buffer protocol is a C-level API that lets Python types exchange +pointers into their internal representations. A memory-mapped file can +be viewed as a buffer of characters, for example, and this lets another +module such as *note re: dd. treat memory-mapped files as a string of +characters to be searched. + +The primary users of the buffer protocol are numeric-processing packages +such as NumPy, which expose the internal representation of arrays so +that callers can write data directly into an array instead of going +through a slower API. This PEP updates the buffer protocol in light of +experience from NumPy development, adding a number of new features such +as indicating the shape of an array or locking a memory region. + +The most important new C API function is ‘PyObject_GetBuffer(PyObject +*obj, Py_buffer *view, int flags)’, which takes an object and a set of +flags, and fills in the ‘Py_buffer’ structure with information about the +object’s memory representation. Objects can use this operation to lock +memory in place while an external caller could be modifying the +contents, so there’s a corresponding ‘PyBuffer_Release(Py_buffer *view)’ +to indicate that the external caller is done. + +The `flags' argument to *note PyObject_GetBuffer(): d25. specifies +constraints upon the memory returned. Some examples are: + + * ‘PyBUF_WRITABLE’ indicates that the memory must be writable. + + * ‘PyBUF_LOCK’ requests a read-only or exclusive lock on the + memory. + + * ‘PyBUF_C_CONTIGUOUS’ and ‘PyBUF_F_CONTIGUOUS’ requests a + C-contiguous (last dimension varies the fastest) or + Fortran-contiguous (first dimension varies the fastest) array + layout. + +Two new argument codes for *note PyArg_ParseTuple(): 26a, ‘s*’ and ‘z*’, +return locked buffer objects for a parameter. + +See also +........ + +PEP 3118(1) - Revising the buffer protocol + + PEP written by Travis Oliphant and Carl Banks; implemented by + Travis Oliphant. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3118 + + +File: python.info, Node: PEP 3119 Abstract Base Classes, Next: PEP 3127 Integer Literal Support and Syntax, Prev: PEP 3118 Revised Buffer Protocol, Up: What’s New in Python 2 6 + +1.11.13 PEP 3119: Abstract Base Classes +--------------------------------------- + +Some object-oriented languages such as Java support interfaces, +declaring that a class has a given set of methods or supports a given +access protocol. Abstract Base Classes (or ABCs) are an equivalent +feature for Python. The ABC support consists of an *note abc: 4. module +containing a metaclass called ‘ABCMeta’, special handling of this +metaclass by the *note isinstance(): 44f. and *note issubclass(): 450. +builtins, and a collection of basic ABCs that the Python developers +think will be widely useful. Future versions of Python will probably +add more ABCs. + +Let’s say you have a particular class and wish to know whether it +supports dictionary-style access. The phrase “dictionary-style” is +vague, however. It probably means that accessing items with ‘obj[1]’ +works. Does it imply that setting items with ‘obj[2] = value’ works? +Or that the object will have ‘keys()’, ‘values()’, and ‘items()’ +methods? What about the iterative variants such as ‘iterkeys()’? *note +copy(): 26. and ‘update()’? Iterating over the object with *note +iter(): d27.? + +The Python 2.6 *note collections: 1e. module includes a number of +different ABCs that represent these distinctions. ‘Iterable’ indicates +that a class defines *note __iter__(): 50f, and ‘Container’ means the +class defines a *note __contains__(): d28. method and therefore supports +‘x in y’ expressions. The basic dictionary interface of getting items, +setting items, and ‘keys()’, ‘values()’, and ‘items()’, is defined by +the ‘MutableMapping’ ABC. + +You can derive your own classes from a particular ABC to indicate they +support that ABC’s interface: + + import collections + + class Storage(collections.MutableMapping): + ... + +Alternatively, you could write the class without deriving from the +desired ABC and instead register the class by calling the ABC’s +‘register()’ method: + + import collections + + class Storage: + ... + + collections.MutableMapping.register(Storage) + +For classes that you write, deriving from the ABC is probably clearer. +The ‘register()’ method is useful when you’ve written a new ABC that can +describe an existing type or class, or if you want to declare that some +third-party class implements an ABC. For example, if you defined a +‘PrintableType’ ABC, it’s legal to do: + + # Register Python's types + PrintableType.register(int) + PrintableType.register(float) + PrintableType.register(str) + +Classes should obey the semantics specified by an ABC, but Python can’t +check this; it’s up to the class author to understand the ABC’s +requirements and to implement the code accordingly. + +To check whether an object supports a particular interface, you can now +write: + + def func(d): + if not isinstance(d, collections.MutableMapping): + raise ValueError("Mapping object expected, not %r" % d) + +Don’t feel that you must now begin writing lots of checks as in the +above example. Python has a strong tradition of duck-typing, where +explicit type-checking is never done and code simply calls methods on an +object, trusting that those methods will be there and raising an +exception if they aren’t. Be judicious in checking for ABCs and only do +it where it’s absolutely necessary. + +You can write your own ABCs by using ‘abc.ABCMeta’ as the metaclass in a +class definition: + + from abc import ABCMeta, abstractmethod + + class Drawable(): + __metaclass__ = ABCMeta + + @abstractmethod + def draw(self, x, y, scale=1.0): + pass + + def draw_doubled(self, x, y): + self.draw(x, y, scale=2.0) + + + class Square(Drawable): + def draw(self, x, y, scale): + ... + +In the ‘Drawable’ ABC above, the ‘draw_doubled()’ method renders the +object at twice its size and can be implemented in terms of other +methods described in ‘Drawable’. Classes implementing this ABC +therefore don’t need to provide their own implementation of +‘draw_doubled()’, though they can do so. An implementation of ‘draw()’ +is necessary, though; the ABC can’t provide a useful generic +implementation. + +You can apply the ‘@abstractmethod’ decorator to methods such as +‘draw()’ that must be implemented; Python will then raise an exception +for classes that don’t define the method. Note that the exception is +only raised when you actually try to create an instance of a subclass +lacking the method: + + >>> class Circle(Drawable): + ... pass + ... + >>> c = Circle() + Traceback (most recent call last): + File "", line 1, in + TypeError: Can't instantiate abstract class Circle with abstract methods draw + >>> + +Abstract data attributes can be declared using the ‘@abstractproperty’ +decorator: + + from abc import abstractproperty + ... + + @abstractproperty + def readonly(self): + return self._x + +Subclasses must then define a ‘readonly()’ property. + +See also +........ + +PEP 3119(1) - Introducing Abstract Base Classes + + PEP written by Guido van Rossum and Talin. Implemented by Guido + van Rossum. Backported to 2.6 by Benjamin Aranguren, with Alex + Martelli. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3119 + + +File: python.info, Node: PEP 3127 Integer Literal Support and Syntax, Next: PEP 3129 Class Decorators, Prev: PEP 3119 Abstract Base Classes, Up: What’s New in Python 2 6 + +1.11.14 PEP 3127: Integer Literal Support and Syntax +---------------------------------------------------- + +Python 3.0 changes the syntax for octal (base-8) integer literals, +prefixing them with “0o” or “0O” instead of a leading zero, and adds +support for binary (base-2) integer literals, signalled by a “0b” or +“0B” prefix. + +Python 2.6 doesn’t drop support for a leading 0 signalling an octal +number, but it does add support for “0o” and “0b”: + + >>> 0o21, 2*8 + 1 + (17, 17) + >>> 0b101111 + 47 + +The *note oct(): c65. builtin still returns numbers prefixed with a +leading zero, and a new *note bin(): c40. builtin returns the binary +representation for a number: + + >>> oct(42) + '052' + >>> future_builtins.oct(42) + '0o52' + >>> bin(173) + '0b10101101' + +The *note int(): 184. and ‘long()’ builtins will now accept the “0o” and +“0b” prefixes when base-8 or base-2 are requested, or when the `base' +argument is zero (signalling that the base used should be determined +from the string): + + >>> int ('0o52', 0) + 42 + >>> int('1101', 2) + 13 + >>> int('0b1101', 2) + 13 + >>> int('0b1101', 0) + 13 + +See also +........ + +PEP 3127(1) - Integer Literal Support and Syntax + + PEP written by Patrick Maupin; backported to 2.6 by Eric Smith. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3127 + + +File: python.info, Node: PEP 3129 Class Decorators, Next: PEP 3141 A Type Hierarchy for Numbers, Prev: PEP 3127 Integer Literal Support and Syntax, Up: What’s New in Python 2 6 + +1.11.15 PEP 3129: Class Decorators +---------------------------------- + +Decorators have been extended from functions to classes. It’s now legal +to write: + + @foo + @bar + class A: + pass + +This is equivalent to: + + class A: + pass + + A = foo(bar(A)) + +See also +........ + +PEP 3129(1) - Class Decorators + + PEP written by Collin Winter. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3129 + + +File: python.info, Node: PEP 3141 A Type Hierarchy for Numbers, Next: Other Language Changes<10>, Prev: PEP 3129 Class Decorators, Up: What’s New in Python 2 6 + +1.11.16 PEP 3141: A Type Hierarchy for Numbers +---------------------------------------------- + +Python 3.0 adds several abstract base classes for numeric types inspired +by Scheme’s numeric tower. These classes were backported to 2.6 as the +*note numbers: c1. module. + +The most general ABC is ‘Number’. It defines no operations at all, and +only exists to allow checking if an object is a number by doing +‘isinstance(obj, Number)’. + +‘Complex’ is a subclass of ‘Number’. Complex numbers can undergo the +basic operations of addition, subtraction, multiplication, division, and +exponentiation, and you can retrieve the real and imaginary parts and +obtain a number’s conjugate. Python’s built-in complex type is an +implementation of ‘Complex’. + +‘Real’ further derives from ‘Complex’, and adds operations that only +work on real numbers: ‘floor()’, ‘trunc()’, rounding, taking the +remainder mod N, floor division, and comparisons. + +‘Rational’ numbers derive from ‘Real’, have ‘numerator’ and +‘denominator’ properties, and can be converted to floats. Python 2.6 +adds a simple rational-number class, ‘Fraction’, in the *note fractions: +83. module. (It’s called ‘Fraction’ instead of ‘Rational’ to avoid a +name clash with *note numbers.Rational: c58.) + +‘Integral’ numbers derive from ‘Rational’, and can be shifted left and +right with ‘<<’ and ‘>>’, combined using bitwise operations such as ‘&’ +and ‘|’, and can be used as array indexes and slice boundaries. + +In Python 3.0, the PEP slightly redefines the existing builtins *note +round(): c6d, *note math.floor(): d2c, *note math.ceil(): d2d, and adds +a new one, *note math.trunc(): d2e, that’s been backported to Python +2.6. *note math.trunc(): d2e. rounds toward zero, returning the closest +‘Integral’ that’s between the function’s argument and zero. + +See also +........ + +PEP 3141(1) - A Type Hierarchy for Numbers + + PEP written by Jeffrey Yasskin. + +Scheme’s numerical tower(2), from the Guile manual. + +Scheme’s number datatypes(3) from the R5RS Scheme specification. + +* Menu: + +* The fractions Module:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3141 + + (2) +https://www.gnu.org/software/guile/manual/html_node/Numerical-Tower.html#Numerical-Tower + + (3) +http://schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-9.html#%_sec_6.2 + + +File: python.info, Node: The fractions Module, Up: PEP 3141 A Type Hierarchy for Numbers + +1.11.16.1 The ‘fractions’ Module +................................ + +To fill out the hierarchy of numeric types, the *note fractions: 83. +module provides a rational-number class. Rational numbers store their +values as a numerator and denominator forming a fraction, and can +exactly represent numbers such as ‘2/3’ that floating-point numbers can +only approximate. + +The ‘Fraction’ constructor takes two ‘Integral’ values that will be the +numerator and denominator of the resulting fraction. + + >>> from fractions import Fraction + >>> a = Fraction(2, 3) + >>> b = Fraction(2, 5) + >>> float(a), float(b) + (0.66666666666666663, 0.40000000000000002) + >>> a+b + Fraction(16, 15) + >>> a/b + Fraction(5, 3) + +For converting floating-point numbers to rationals, the float type now +has an ‘as_integer_ratio()’ method that returns the numerator and +denominator for a fraction that evaluates to the same floating-point +value: + + >>> (2.5) .as_integer_ratio() + (5, 2) + >>> (3.1415) .as_integer_ratio() + (7074029114692207L, 2251799813685248L) + >>> (1./3) .as_integer_ratio() + (6004799503160661L, 18014398509481984L) + +Note that values that can only be approximated by floating-point +numbers, such as 1./3, are not simplified to the number being +approximated; the fraction attempts to match the floating-point value +`exactly'. + +The *note fractions: 83. module is based upon an implementation by +Sjoerd Mullender that was in Python’s ‘Demo/classes/’ directory for a +long time. This implementation was significantly updated by Jeffrey +Yasskin. + + +File: python.info, Node: Other Language Changes<10>, Next: New and Improved Modules<2>, Prev: PEP 3141 A Type Hierarchy for Numbers, Up: What’s New in Python 2 6 + +1.11.17 Other Language Changes +------------------------------ + +Some smaller changes made to the core Python language are: + + * Directories and zip archives containing a ‘__main__.py’ file can + now be executed directly by passing their name to the interpreter. + The directory or zip archive is automatically inserted as the first + entry in sys.path. (Suggestion and initial patch by Andy Chu, + subsequently revised by Phillip J. Eby and Nick Coghlan; + bpo-1739468(1).) + + * The *note hasattr(): 447. function was catching and ignoring all + errors, under the assumption that they meant a *note __getattr__(): + 31a. method was failing somehow and the return value of *note + hasattr(): 447. would therefore be ‘False’. This logic shouldn’t + be applied to *note KeyboardInterrupt: 197. and *note SystemExit: + 5fc, however; Python 2.6 will no longer discard such exceptions + when *note hasattr(): 447. encounters them. (Fixed by Benjamin + Peterson; bpo-2196(2).) + + * When calling a function using the ‘**’ syntax to provide keyword + arguments, you are no longer required to use a Python dictionary; + any mapping will now work: + + >>> def f(**kw): + ... print sorted(kw) + ... + >>> ud=UserDict.UserDict() + >>> ud['a'] = 1 + >>> ud['b'] = 'string' + >>> f(**ud) + ['a', 'b'] + + (Contributed by Alexander Belopolsky; bpo-1686487(3).) + + It’s also become legal to provide keyword arguments after a ‘*args’ + argument to a function call. + + >>> def f(*args, **kw): + ... print args, kw + ... + >>> f(1,2,3, *(4,5,6), keyword=13) + (1, 2, 3, 4, 5, 6) {'keyword': 13} + + Previously this would have been a syntax error. (Contributed by + Amaury Forgeot d’Arc; bpo-3473(4).) + + * A new builtin, ‘next(iterator, [default])’ returns the next item + from the specified iterator. If the `default' argument is + supplied, it will be returned if `iterator' has been exhausted; + otherwise, the *note StopIteration: 486. exception will be raised. + (Backported in bpo-2719(5).) + + * Tuples now have ‘index()’ and ‘count()’ methods matching the list + type’s ‘index()’ and ‘count()’ methods: + + >>> t = (0,1,2,3,4,0,1,2) + >>> t.index(3) + 3 + >>> t.count(0) + 2 + + (Contributed by Raymond Hettinger) + + * The built-in types now have improved support for extended slicing + syntax, accepting various combinations of ‘(start, stop, step)’. + Previously, the support was partial and certain corner cases + wouldn’t work. (Implemented by Thomas Wouters.) + + * Properties now have three attributes, ‘getter’, ‘setter’ and + ‘deleter’, that are decorators providing useful shortcuts for + adding a getter, setter or deleter function to an existing + property. You would use them like this: + + class C(object): + @property + def x(self): + return self._x + + @x.setter + def x(self, value): + self._x = value + + @x.deleter + def x(self): + del self._x + + class D(C): + @C.x.getter + def x(self): + return self._x * 2 + + @x.setter + def x(self, value): + self._x = value / 2 + + * Several methods of the built-in set types now accept multiple + iterables: ‘intersection()’, ‘intersection_update()’, ‘union()’, + ‘update()’, ‘difference()’ and ‘difference_update()’. + + >>> s=set('1234567890') + >>> s.intersection('abc123', 'cdf246') # Intersection between all inputs + set(['2']) + >>> s.difference('246', '789') + set(['1', '0', '3', '5']) + + (Contributed by Raymond Hettinger.) + + * Many floating-point features were added. The *note float(): 187. + function will now turn the string ‘nan’ into an IEEE 754 Not A + Number value, and ‘+inf’ and ‘-inf’ into positive or negative + infinity. This works on any platform with IEEE 754 semantics. + (Contributed by Christian Heimes; bpo-1635(6).) + + Other functions in the *note math: b1. module, ‘isinf()’ and + ‘isnan()’, return true if their floating-point argument is infinite + or Not A Number. (bpo-1640(7)) + + Conversion functions were added to convert floating-point numbers + into hexadecimal strings (bpo-3008(8)). These functions convert + floats to and from a string representation without introducing + rounding errors from the conversion between decimal and binary. + Floats have a *note hex(): c66. method that returns a string + representation, and the ‘float.fromhex()’ method converts a string + back into a number: + + >>> a = 3.75 + >>> a.hex() + '0x1.e000000000000p+1' + >>> float.fromhex('0x1.e000000000000p+1') + 3.75 + >>> b=1./3 + >>> b.hex() + '0x1.5555555555555p-2' + + * A numerical nicety: when creating a complex number from two floats + on systems that support signed zeros (-0 and +0), the *note + complex(): 189. constructor will now preserve the sign of the zero. + (Fixed by Mark T. Dickinson; bpo-1507(9).) + + * Classes that inherit a *note __hash__(): 340. method from a parent + class can set ‘__hash__ = None’ to indicate that the class isn’t + hashable. This will make ‘hash(obj)’ raise a *note TypeError: 192. + and the class will not be indicated as implementing the ‘Hashable’ + ABC. + + You should do this when you’ve defined a ‘__cmp__()’ or *note + __eq__(): 33f. method that compares objects by their value rather + than by identity. All objects have a default hash method that uses + ‘id(obj)’ as the hash value. There’s no tidy way to remove the + *note __hash__(): 340. method inherited from a parent class, so + assigning ‘None’ was implemented as an override. At the C level, + extensions can set ‘tp_hash’ to *note + PyObject_HashNotImplemented(): d31. (Fixed by Nick Coghlan and + Amaury Forgeot d’Arc; bpo-2235(10).) + + * The *note GeneratorExit: d32. exception now subclasses *note + BaseException: 1a8. instead of *note Exception: 1a9. This means + that an exception handler that does ‘except Exception:’ will not + inadvertently catch *note GeneratorExit: d32. (Contributed by Chad + Austin; bpo-1537(11).) + + * Generator objects now have a ‘gi_code’ attribute that refers to the + original code object backing the generator. (Contributed by Collin + Winter; bpo-1473257(12).) + + * The *note compile(): 1b4. built-in function now accepts keyword + arguments as well as positional parameters. (Contributed by Thomas + Wouters; bpo-1444529(13).) + + * The *note complex(): 189. constructor now accepts strings + containing parenthesized complex numbers, meaning that + ‘complex(repr(cplx))’ will now round-trip values. For example, + ‘complex('(3+4j)')’ now returns the value (3+4j). + (bpo-1491866(14)) + + * The string ‘translate()’ method now accepts ‘None’ as the + translation table parameter, which is treated as the identity + transformation. This makes it easier to carry out operations that + only delete characters. (Contributed by Bengt Richter and + implemented by Raymond Hettinger; bpo-1193128(15).) + + * The built-in *note dir(): d33. function now checks for a *note + __dir__(): 31b. method on the objects it receives. This method + must return a list of strings containing the names of valid + attributes for the object, and lets the object control the value + that *note dir(): d33. produces. Objects that have *note + __getattr__(): 31a. or *note __getattribute__(): 449. methods can + use this to advertise pseudo-attributes they will honor. + (bpo-1591665(16)) + + * Instance method objects have new attributes for the object and + function comprising the method; the new synonym for ‘im_self’ is + ‘__self__’, and ‘im_func’ is also available as ‘__func__’. The old + names are still supported in Python 2.6, but are gone in 3.0. + + * An obscure change: when you use the *note locals(): 455. function + inside a *note class: c6a. statement, the resulting dictionary no + longer returns free variables. (Free variables, in this case, are + variables referenced in the ‘class’ statement that aren’t + attributes of the class.) + +* Menu: + +* Optimizations: Optimizations<9>. +* Interpreter Changes: Interpreter Changes<2>. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1739468 + + (2) https://bugs.python.org/issue2196 + + (3) https://bugs.python.org/issue1686487 + + (4) https://bugs.python.org/issue3473 + + (5) https://bugs.python.org/issue2719 + + (6) https://bugs.python.org/issue1635 + + (7) https://bugs.python.org/issue1640 + + (8) https://bugs.python.org/issue3008 + + (9) https://bugs.python.org/issue1507 + + (10) https://bugs.python.org/issue2235 + + (11) https://bugs.python.org/issue1537 + + (12) https://bugs.python.org/issue1473257 + + (13) https://bugs.python.org/issue1444529 + + (14) https://bugs.python.org/issue1491866 + + (15) https://bugs.python.org/issue1193128 + + (16) https://bugs.python.org/issue1591665 + + +File: python.info, Node: Optimizations<9>, Next: Interpreter Changes<2>, Up: Other Language Changes<10> + +1.11.17.1 Optimizations +....................... + + * The *note warnings: 126. module has been rewritten in C. This makes + it possible to invoke warnings from the parser, and may also make + the interpreter’s startup faster. (Contributed by Neal Norwitz and + Brett Cannon; bpo-1631171(1).) + + * Type objects now have a cache of methods that can reduce the work + required to find the correct method implementation for a particular + class; once cached, the interpreter doesn’t need to traverse base + classes to figure out the right method to call. The cache is + cleared if a base class or the class itself is modified, so the + cache should remain correct even in the face of Python’s dynamic + nature. (Original optimization implemented by Armin Rigo, updated + for Python 2.6 by Kevin Jacobs; bpo-1700288(2).) + + By default, this change is only applied to types that are included + with the Python core. Extension modules may not necessarily be + compatible with this cache, so they must explicitly add + ‘Py_TPFLAGS_HAVE_VERSION_TAG’ to the module’s ‘tp_flags’ field to + enable the method cache. (To be compatible with the method cache, + the extension module’s code must not directly access and modify the + ‘tp_dict’ member of any of the types it implements. Most modules + don’t do this, but it’s impossible for the Python interpreter to + determine that. See bpo-1878(3) for some discussion.) + + * Function calls that use keyword arguments are significantly faster + by doing a quick pointer comparison, usually saving the time of a + full string comparison. (Contributed by Raymond Hettinger, after + an initial implementation by Antoine Pitrou; bpo-1819(4).) + + * All of the functions in the *note struct: f8. module have been + rewritten in C, thanks to work at the Need For Speed sprint. + (Contributed by Raymond Hettinger.) + + * Some of the standard built-in types now set a bit in their type + objects. This speeds up checking whether an object is a subclass + of one of these types. (Contributed by Neal Norwitz.) + + * Unicode strings now use faster code for detecting whitespace and + line breaks; this speeds up the ‘split()’ method by about 25% and + ‘splitlines()’ by 35%. (Contributed by Antoine Pitrou.) Memory + usage is reduced by using pymalloc for the Unicode string’s data. + + * The ‘with’ statement now stores the *note __exit__(): c96. method + on the stack, producing a small speedup. (Implemented by Jeffrey + Yasskin.) + + * To reduce memory usage, the garbage collector will now clear + internal free lists when garbage-collecting the highest generation + of objects. This may return memory to the operating system sooner. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1631171 + + (2) https://bugs.python.org/issue1700288 + + (3) https://bugs.python.org/issue1878 + + (4) https://bugs.python.org/issue1819 + + +File: python.info, Node: Interpreter Changes<2>, Prev: Optimizations<9>, Up: Other Language Changes<10> + +1.11.17.2 Interpreter Changes +............................. + +Two command-line options have been reserved for use by other Python +implementations. The *note -J: d37. switch has been reserved for use by +Jython for Jython-specific options, such as switches that are passed to +the underlying JVM. *note -X: 155. has been reserved for options +specific to a particular implementation of Python such as CPython, +Jython, or IronPython. If either option is used with Python 2.6, the +interpreter will report that the option isn’t currently used. + +Python can now be prevented from writing ‘.pyc’ or ‘.pyo’ files by +supplying the *note -B: d38. switch to the Python interpreter, or by +setting the *note PYTHONDONTWRITEBYTECODE: d39. environment variable +before running the interpreter. This setting is available to Python +programs as the ‘sys.dont_write_bytecode’ variable, and Python code can +change the value to modify the interpreter’s behaviour. (Contributed by +Neal Norwitz and Georg Brandl.) + +The encoding used for standard input, output, and standard error can be +specified by setting the *note PYTHONIOENCODING: 92f. environment +variable before running the interpreter. The value should be a string +in the form ‘’ or ‘:’. The `encoding' +part specifies the encoding’s name, e.g. ‘utf-8’ or ‘latin-1’; the +optional `errorhandler' part specifies what to do with characters that +can’t be handled by the encoding, and should be one of “error”, +“ignore”, or “replace”. (Contributed by Martin von Löwis.) + + +File: python.info, Node: New and Improved Modules<2>, Next: Deprecations and Removals, Prev: Other Language Changes<10>, Up: What’s New in Python 2 6 + +1.11.18 New and Improved Modules +-------------------------------- + +As in every release, Python’s standard library received a number of +enhancements and bug fixes. Here’s a partial list of the most notable +changes, sorted alphabetically by module name. Consult the ‘Misc/NEWS’ +file in the source tree for a more complete list of changes, or look +through the Subversion logs for all the details. + + * The *note asyncore: b. and *note asynchat: 9. modules are being + actively maintained again, and a number of patches and bugfixes + were applied. (Maintained by Josiah Carlson; see bpo-1736190(1) + for one patch.) + + * The ‘bsddb’ module also has a new maintainer, Jesús Cea Avión, and + the package is now available as a standalone package. The web page + for the package is www.jcea.es/programacion/pybsddb.htm(2). The + plan is to remove the package from the standard library in Python + 3.0, because its pace of releases is much more frequent than + Python’s. + + The ‘bsddb.dbshelve’ module now uses the highest pickling protocol + available, instead of restricting itself to protocol 1. + (Contributed by W. Barnes.) + + * The *note cgi: 16. module will now read variables from the query + string of an HTTP POST request. This makes it possible to use form + actions with URLs that include query strings such as + “/cgi-bin/add.py?category=1”. (Contributed by Alexandre Fiori and + Nubis; bpo-1817(3).) + + The ‘parse_qs()’ and ‘parse_qsl()’ functions have been relocated + from the *note cgi: 16. module to the ‘urlparse’ module. The + versions still available in the *note cgi: 16. module will trigger + *note PendingDeprecationWarning: 279. messages in 2.6 + (bpo-600362(4)). + + * The *note cmath: 19. module underwent extensive revision, + contributed by Mark Dickinson and Christian Heimes. Five new + functions were added: + + * ‘polar()’ converts a complex number to polar form, returning + the modulus and argument of the complex number. + + * ‘rect()’ does the opposite, turning a modulus, argument pair + back into the corresponding complex number. + + * ‘phase()’ returns the argument (also called the angle) of a + complex number. + + * ‘isnan()’ returns True if either the real or imaginary part of + its argument is a NaN. + + * ‘isinf()’ returns True if either the real or imaginary part of + its argument is infinite. + + The revisions also improved the numerical soundness of the *note + cmath: 19. module. For all functions, the real and imaginary parts + of the results are accurate to within a few units of least + precision (ulps) whenever possible. See bpo-1381(5) for the + details. The branch cuts for ‘asinh()’, ‘atanh()’: and ‘atan()’ + have also been corrected. + + The tests for the module have been greatly expanded; nearly 2000 + new test cases exercise the algebraic functions. + + On IEEE 754 platforms, the *note cmath: 19. module now handles IEEE + 754 special values and floating-point exceptions in a manner + consistent with Annex ‘G’ of the C99 standard. + + * A new data type in the *note collections: 1e. module: + ‘namedtuple(typename, fieldnames)’ is a factory function that + creates subclasses of the standard tuple whose fields are + accessible by name as well as index. For example: + + >>> var_type = collections.namedtuple('variable', + ... 'id name type size') + >>> # Names are separated by spaces or commas. + >>> # 'id, name, type, size' would also work. + >>> var_type._fields + ('id', 'name', 'type', 'size') + + >>> var = var_type(1, 'frequency', 'int', 4) + >>> print var[0], var.id # Equivalent + 1 1 + >>> print var[2], var.type # Equivalent + int int + >>> var._asdict() + {'size': 4, 'type': 'int', 'id': 1, 'name': 'frequency'} + >>> v2 = var._replace(name='amplitude') + >>> v2 + variable(id=1, name='amplitude', type='int', size=4) + + Several places in the standard library that returned tuples have + been modified to return ‘namedtuple’ instances. For example, the + ‘Decimal.as_tuple()’ method now returns a named tuple with ‘sign’, + ‘digits’, and ‘exponent’ fields. + + (Contributed by Raymond Hettinger.) + + * Another change to the *note collections: 1e. module is that the + ‘deque’ type now supports an optional `maxlen' parameter; if + supplied, the deque’s size will be restricted to no more than + `maxlen' items. Adding more items to a full deque causes old items + to be discarded. + + >>> from collections import deque + >>> dq=deque(maxlen=3) + >>> dq + deque([], maxlen=3) + >>> dq.append(1); dq.append(2); dq.append(3) + >>> dq + deque([1, 2, 3], maxlen=3) + >>> dq.append(4) + >>> dq + deque([2, 3, 4], maxlen=3) + + (Contributed by Raymond Hettinger.) + + * The ‘Cookie’ module’s ‘Morsel’ objects now support an ‘httponly’ + attribute. In some browsers. cookies with this attribute set + cannot be accessed or manipulated by JavaScript code. (Contributed + by Arvin Schnell; bpo-1638033(6).) + + * A new window method in the *note curses: 2c. module, ‘chgat()’, + changes the display attributes for a certain number of characters + on a single line. (Contributed by Fabian Kreutz.) + + # Boldface text starting at y=0,x=21 + # and affecting the rest of the line. + stdscr.chgat(0, 21, curses.A_BOLD) + + The ‘Textbox’ class in the *note curses.textpad: 2f. module now + supports editing in insert mode as well as overwrite mode. Insert + mode is enabled by supplying a true value for the `insert_mode' + parameter when creating the ‘Textbox’ instance. + + * The *note datetime: 31. module’s ‘strftime()’ methods now support a + ‘%f’ format code that expands to the number of microseconds in the + object, zero-padded on the left to six places. (Contributed by + Skip Montanaro; bpo-1158(7).) + + * The *note decimal: 36. module was updated to version 1.66 of the + General Decimal Specification(8). New features include some + methods for some basic mathematical functions such as ‘exp()’ and + ‘log10()’: + + >>> Decimal(1).exp() + Decimal("2.718281828459045235360287471") + >>> Decimal("2.7182818").ln() + Decimal("0.9999999895305022877376682436") + >>> Decimal(1000).log10() + Decimal("3") + + The ‘as_tuple()’ method of ‘Decimal’ objects now returns a named + tuple with ‘sign’, ‘digits’, and ‘exponent’ fields. + + (Implemented by Facundo Batista and Mark Dickinson. Named tuple + support added by Raymond Hettinger.) + + * The *note difflib: 37. module’s ‘SequenceMatcher’ class now returns + named tuples representing matches, with ‘a’, ‘b’, and ‘size’ + attributes. (Contributed by Raymond Hettinger.) + + * An optional ‘timeout’ parameter, specifying a timeout measured in + seconds, was added to the *note ftplib.FTP: 2f0. class constructor + as well as the ‘connect()’ method. (Added by Facundo Batista.) + Also, the ‘FTP’ class’s ‘storbinary()’ and ‘storlines()’ now take + an optional `callback' parameter that will be called with each + block of data after the data has been sent. (Contributed by Phil + Schwartz; bpo-1221598(9).) + + * The ‘reduce()’ built-in function is also available in the *note + functools: 85. module. In Python 3.0, the builtin has been dropped + and ‘reduce()’ is only available from *note functools: 85.; + currently there are no plans to drop the builtin in the 2.x series. + (Patched by Christian Heimes; bpo-1739906(10).) + + * When possible, the *note getpass: 88. module will now use + ‘/dev/tty’ to print a prompt message and read the password, falling + back to standard error and standard input. If the password may be + echoed to the terminal, a warning is printed before the prompt is + displayed. (Contributed by Gregory P. Smith.) + + * The *note glob.glob(): 5c6. function can now return Unicode + filenames if a Unicode path was used and Unicode filenames are + matched within the directory. (bpo-1001604(11)) + + * A new function in the *note heapq: 8e. module, ‘merge(iter1, iter2, + ...)’, takes any number of iterables returning data in sorted + order, and returns a new generator that returns the contents of all + the iterators, also in sorted order. For example: + + >>> list(heapq.merge([1, 3, 5, 9], [2, 8, 16])) + [1, 2, 3, 5, 8, 9, 16] + + Another new function, ‘heappushpop(heap, item)’, pushes `item' onto + `heap', then pops off and returns the smallest item. This is more + efficient than making a call to ‘heappush()’ and then ‘heappop()’. + + *note heapq: 8e. is now implemented to only use less-than + comparison, instead of the less-than-or-equal comparison it + previously used. This makes *note heapq: 8e.’s usage of a type + match the *note list.sort(): 445. method. (Contributed by Raymond + Hettinger.) + + * An optional ‘timeout’ parameter, specifying a timeout measured in + seconds, was added to the ‘httplib.HTTPConnection’ and + ‘HTTPSConnection’ class constructors. (Added by Facundo Batista.) + + * Most of the *note inspect: a0. module’s functions, such as + ‘getmoduleinfo()’ and ‘getargs()’, now return named tuples. In + addition to behaving like tuples, the elements of the return value + can also be accessed as attributes. (Contributed by Raymond + Hettinger.) + + Some new functions in the module include ‘isgenerator()’, + ‘isgeneratorfunction()’, and ‘isabstract()’. + + * The *note itertools: a3. module gained several new functions. + + ‘izip_longest(iter1, iter2, ...[, fillvalue])’ makes tuples from + each of the elements; if some of the iterables are shorter than + others, the missing values are set to `fillvalue'. For example: + + >>> tuple(itertools.izip_longest([1,2,3], [1,2,3,4,5])) + ((1, 1), (2, 2), (3, 3), (None, 4), (None, 5)) + + ‘product(iter1, iter2, ..., [repeat=N])’ returns the Cartesian + product of the supplied iterables, a set of tuples containing every + possible combination of the elements returned from each iterable. + + >>> list(itertools.product([1,2,3], [4,5,6])) + [(1, 4), (1, 5), (1, 6), + (2, 4), (2, 5), (2, 6), + (3, 4), (3, 5), (3, 6)] + + The optional `repeat' keyword argument is used for taking the + product of an iterable or a set of iterables with themselves, + repeated `N' times. With a single iterable argument, `N'-tuples + are returned: + + >>> list(itertools.product([1,2], repeat=3)) + [(1, 1, 1), (1, 1, 2), (1, 2, 1), (1, 2, 2), + (2, 1, 1), (2, 1, 2), (2, 2, 1), (2, 2, 2)] + + With two iterables, `2N'-tuples are returned. + + >>> list(itertools.product([1,2], [3,4], repeat=2)) + [(1, 3, 1, 3), (1, 3, 1, 4), (1, 3, 2, 3), (1, 3, 2, 4), + (1, 4, 1, 3), (1, 4, 1, 4), (1, 4, 2, 3), (1, 4, 2, 4), + (2, 3, 1, 3), (2, 3, 1, 4), (2, 3, 2, 3), (2, 3, 2, 4), + (2, 4, 1, 3), (2, 4, 1, 4), (2, 4, 2, 3), (2, 4, 2, 4)] + + ‘combinations(iterable, r)’ returns sub-sequences of length `r' + from the elements of `iterable'. + + >>> list(itertools.combinations('123', 2)) + [('1', '2'), ('1', '3'), ('2', '3')] + >>> list(itertools.combinations('123', 3)) + [('1', '2', '3')] + >>> list(itertools.combinations('1234', 3)) + [('1', '2', '3'), ('1', '2', '4'), + ('1', '3', '4'), ('2', '3', '4')] + + ‘permutations(iter[, r])’ returns all the permutations of length + `r' of the iterable’s elements. If `r' is not specified, it will + default to the number of elements produced by the iterable. + + >>> list(itertools.permutations([1,2,3,4], 2)) + [(1, 2), (1, 3), (1, 4), + (2, 1), (2, 3), (2, 4), + (3, 1), (3, 2), (3, 4), + (4, 1), (4, 2), (4, 3)] + + ‘itertools.chain(*iterables)’ is an existing function in *note + itertools: a3. that gained a new constructor in Python 2.6. + ‘itertools.chain.from_iterable(iterable)’ takes a single iterable + that should return other iterables. ‘chain()’ will then return all + the elements of the first iterable, then all the elements of the + second, and so on. + + >>> list(itertools.chain.from_iterable([[1,2,3], [4,5,6]])) + [1, 2, 3, 4, 5, 6] + + (All contributed by Raymond Hettinger.) + + * The *note logging: aa. module’s ‘FileHandler’ class and its + subclasses ‘WatchedFileHandler’, ‘RotatingFileHandler’, and + ‘TimedRotatingFileHandler’ now have an optional `delay' parameter + to their constructors. If `delay' is true, opening of the log file + is deferred until the first ‘emit()’ call is made. (Contributed by + Vinay Sajip.) + + ‘TimedRotatingFileHandler’ also has a `utc' constructor parameter. + If the argument is true, UTC time will be used in determining when + midnight occurs and in generating filenames; otherwise local time + will be used. + + * Several new functions were added to the *note math: b1. module: + + * *note isinf(): d3b. and *note isnan(): d3c. determine whether + a given float is a (positive or negative) infinity or a NaN + (Not a Number), respectively. + + * *note copysign(): d3d. copies the sign bit of an IEEE 754 + number, returning the absolute value of `x' combined with the + sign bit of `y'. For example, ‘math.copysign(1, -0.0)’ + returns -1.0. (Contributed by Christian Heimes.) + + * *note factorial(): 1ea. computes the factorial of a number. + (Contributed by Raymond Hettinger; bpo-2138(12).) + + * *note fsum(): d3e. adds up the stream of numbers from an + iterable, and is careful to avoid loss of precision through + using partial sums. (Contributed by Jean Brouwers, Raymond + Hettinger, and Mark Dickinson; bpo-2819(13).) + + * *note acosh(): d3f, *note asinh(): d40. and *note atanh(): + d41. compute the inverse hyperbolic functions. + + * *note log1p(): d42. returns the natural logarithm of `1+x' + (base `e'). + + * ‘trunc()’ rounds a number toward zero, returning the closest + ‘Integral’ that’s between the function’s argument and zero. + Added as part of the backport of *note PEP 3141’s type + hierarchy for numbers: c57. + + * The *note math: b1. module has been improved to give more + consistent behaviour across platforms, especially with respect to + handling of floating-point exceptions and IEEE 754 special values. + + Whenever possible, the module follows the recommendations of the + C99 standard about 754’s special values. For example, ‘sqrt(-1.)’ + should now give a *note ValueError: 1fb. across almost all + platforms, while ‘sqrt(float('NaN'))’ should return a NaN on all + IEEE 754 platforms. Where Annex ‘F’ of the C99 standard recommends + signaling ‘divide-by-zero’ or ‘invalid’, Python will raise *note + ValueError: 1fb. Where Annex ‘F’ of the C99 standard recommends + signaling ‘overflow’, Python will raise *note OverflowError: 960. + (See bpo-711019(14) and bpo-1640(15).) + + (Contributed by Christian Heimes and Mark Dickinson.) + + * *note mmap: 1ec. objects now have a ‘rfind()’ method that searches + for a substring beginning at the end of the string and searching + backwards. The ‘find()’ method also gained an `end' parameter + giving an index at which to stop searching. (Contributed by John + Lenton.) + + * The *note operator: c2. module gained a ‘methodcaller()’ function + that takes a name and an optional set of arguments, returning a + callable that will call the named function on any arguments passed + to it. For example: + + >>> # Equivalent to lambda s: s.replace('old', 'new') + >>> replacer = operator.methodcaller('replace', 'old', 'new') + >>> replacer('old wine in old bottles') + 'new wine in new bottles' + + (Contributed by Georg Brandl, after a suggestion by Gregory + Petrosyan.) + + The ‘attrgetter()’ function now accepts dotted names and performs + the corresponding attribute lookups: + + >>> inst_name = operator.attrgetter( + ... '__class__.__name__') + >>> inst_name('') + 'str' + >>> inst_name(help) + '_Helper' + + (Contributed by Georg Brandl, after a suggestion by Barry Warsaw.) + + * The *note os: c4. module now wraps several new system calls. + ‘fchmod(fd, mode)’ and ‘fchown(fd, uid, gid)’ change the mode and + ownership of an opened file, and ‘lchmod(path, mode)’ changes the + mode of a symlink. (Contributed by Georg Brandl and Christian + Heimes.) + + ‘chflags()’ and ‘lchflags()’ are wrappers for the corresponding + system calls (where they’re available), changing the flags set on a + file. Constants for the flag values are defined in the *note stat: + f4. module; some possible values include ‘UF_IMMUTABLE’ to signal + the file may not be changed and ‘UF_APPEND’ to indicate that data + can only be appended to the file. (Contributed by M. Levinson.) + + ‘os.closerange(low, high)’ efficiently closes all file descriptors + from `low' to `high', ignoring any errors and not including `high' + itself. This function is now used by the *note subprocess: f9. + module to make starting processes faster. (Contributed by Georg + Brandl; bpo-1663329(16).) + + * The ‘os.environ’ object’s ‘clear()’ method will now unset the + environment variables using *note os.unsetenv(): d43. in addition + to clearing the object’s keys. (Contributed by Martin Horcicka; + bpo-1181(17).) + + * The *note os.walk(): 654. function now has a ‘followlinks’ + parameter. If set to True, it will follow symlinks pointing to + directories and visit the directory’s contents. For backward + compatibility, the parameter’s default value is false. Note that + the function can fall into an infinite recursion if there’s a + symlink that points to a parent directory. (bpo-1273829(18)) + + * In the *note os.path: c5. module, the ‘splitext()’ function has + been changed to not split on leading period characters. This + produces better results when operating on Unix’s dot-files. For + example, ‘os.path.splitext('.ipython')’ now returns ‘('.ipython', + '')’ instead of ‘('', '.ipython')’. (bpo-1115886(19)) + + A new function, ‘os.path.relpath(path, start='.')’, returns a + relative path from the ‘start’ path, if it’s supplied, or from the + current working directory to the destination ‘path’. (Contributed + by Richard Barran; bpo-1339796(20).) + + On Windows, *note os.path.expandvars(): d44. will now expand + environment variables given in the form “%var%”, and “~user” will + be expanded into the user’s home directory path. (Contributed by + Josiah Carlson; bpo-957650(21).) + + * The Python debugger provided by the *note pdb: c9. module gained a + new command: “run” restarts the Python program being debugged and + can optionally take new command-line arguments for the program. + (Contributed by Rocky Bernstein; bpo-1393667(22).) + + * The *note pdb.post_mortem(): d45. function, used to begin debugging + a traceback, will now use the traceback returned by *note + sys.exc_info(): c5f. if no traceback is supplied. (Contributed by + Facundo Batista; bpo-1106316(23).) + + * The *note pickletools: cb. module now has an ‘optimize()’ function + that takes a string containing a pickle and removes some unused + opcodes, returning a shorter pickle that contains the same data + structure. (Contributed by Raymond Hettinger.) + + * A ‘get_data()’ function was added to the *note pkgutil: cd. module + that returns the contents of resource files included with an + installed Python package. For example: + + >>> import pkgutil + >>> print pkgutil.get_data('test', 'exception_hierarchy.txt') + BaseException + +-- SystemExit + +-- KeyboardInterrupt + +-- GeneratorExit + +-- Exception + +-- StopIteration + +-- StandardError + ... + + (Contributed by Paul Moore; bpo-2439(24).) + + * The ‘pyexpat’ module’s ‘Parser’ objects now allow setting their + ‘buffer_size’ attribute to change the size of the buffer used to + hold character data. (Contributed by Achim Gaedke; bpo-1137(25).) + + * The ‘Queue’ module now provides queue variants that retrieve + entries in different orders. The ‘PriorityQueue’ class stores + queued items in a heap and retrieves them in priority order, and + ‘LifoQueue’ retrieves the most recently added entries first, + meaning that it behaves like a stack. (Contributed by Raymond + Hettinger.) + + * The *note random: dc. module’s ‘Random’ objects can now be pickled + on a 32-bit system and unpickled on a 64-bit system, and vice + versa. Unfortunately, this change also means that Python 2.6’s + ‘Random’ objects can’t be unpickled correctly on earlier versions + of Python. (Contributed by Shawn Ligocki; bpo-1727780(26).) + + The new ‘triangular(low, high, mode)’ function returns random + numbers following a triangular distribution. The returned values + are between `low' and `high', not including `high' itself, and with + `mode' as the most frequently occurring value in the distribution. + (Contributed by Wladmir van der Laan and Raymond Hettinger; + bpo-1681432(27).) + + * Long regular expression searches carried out by the *note re: dd. + module will check for signals being delivered, so time-consuming + searches can now be interrupted. (Contributed by Josh Hoyt and + Ralf Schmitt; bpo-846388(28).) + + The regular expression module is implemented by compiling bytecodes + for a tiny regex-specific virtual machine. Untrusted code could + create malicious strings of bytecode directly and cause crashes, so + Python 2.6 includes a verifier for the regex bytecode. + (Contributed by Guido van Rossum from work for Google App Engine; + bpo-3487(29).) + + * The *note rlcompleter: e1. module’s ‘Completer.complete()’ method + will now ignore exceptions triggered while evaluating a name. + (Fixed by Lorenz Quack; bpo-2250(30).) + + * The *note sched: e3. module’s ‘scheduler’ instances now have a + read-only *note queue: da. attribute that returns the contents of + the scheduler’s queue, represented as a list of named tuples with + the fields ‘(time, priority, action, argument)’. (Contributed by + Raymond Hettinger; bpo-1861(31).) + + * The *note select: e5. module now has wrapper functions for the + Linux ‘epoll()’ and BSD ‘kqueue()’ system calls. ‘modify()’ method + was added to the existing ‘poll’ objects; ‘pollobj.modify(fd, + eventmask)’ takes a file descriptor or file object and an event + mask, modifying the recorded event mask for that file. + (Contributed by Christian Heimes; bpo-1657(32).) + + * The *note shutil.copytree(): 21a. function now has an optional + `ignore' argument that takes a callable object. This callable will + receive each directory path and a list of the directory’s contents, + and returns a list of names that will be ignored, not copied. + + The *note shutil: e9. module also provides an ‘ignore_patterns()’ + function for use with this new parameter. ‘ignore_patterns()’ + takes an arbitrary number of glob-style patterns and returns a + callable that will ignore any files and directories that match any + of these patterns. The following example copies a directory tree, + but skips both ‘.svn’ directories and Emacs backup files, which + have names ending with ‘~’: + + shutil.copytree('Doc/library', '/tmp/library', + ignore=shutil.ignore_patterns('*~', '.svn')) + + (Contributed by Tarek Ziadé; bpo-2663(33).) + + * Integrating signal handling with GUI handling event loops like + those used by Tkinter or GTk+ has long been a problem; most + software ends up polling, waking up every fraction of a second to + check if any GUI events have occurred. The *note signal: ea. + module can now make this more efficient. Calling + ‘signal.set_wakeup_fd(fd)’ sets a file descriptor to be used; when + a signal is received, a byte is written to that file descriptor. + There’s also a C-level function, *note PySignal_SetWakeupFd(): d46, + for setting the descriptor. + + Event loops will use this by opening a pipe to create two + descriptors, one for reading and one for writing. The writable + descriptor will be passed to ‘set_wakeup_fd()’, and the readable + descriptor will be added to the list of descriptors monitored by + the event loop via ‘select()’ or ‘poll()’. On receiving a signal, + a byte will be written and the main event loop will be woken up, + avoiding the need to poll. + + (Contributed by Adam Olsen; bpo-1583(34).) + + The ‘siginterrupt()’ function is now available from Python code, + and allows changing whether signals can interrupt system calls or + not. (Contributed by Ralf Schmitt.) + + The ‘setitimer()’ and ‘getitimer()’ functions have also been added + (where they’re available). ‘setitimer()’ allows setting interval + timers that will cause a signal to be delivered to the process + after a specified time, measured in wall-clock time, consumed + process time, or combined process+system time. (Contributed by + Guilherme Polo; bpo-2240(35).) + + * The *note smtplib: ed. module now supports SMTP over SSL thanks to + the addition of the ‘SMTP_SSL’ class. This class supports an + interface identical to the existing ‘SMTP’ class. (Contributed by + Monty Taylor.) Both class constructors also have an optional + ‘timeout’ parameter that specifies a timeout for the initial + connection attempt, measured in seconds. (Contributed by Facundo + Batista.) + + An implementation of the LMTP protocol ( RFC 2033(36)) was also + added to the module. LMTP is used in place of SMTP when + transferring e-mail between agents that don’t manage a mail queue. + (LMTP implemented by Leif Hedstrom; bpo-957003(37).) + + ‘SMTP.starttls()’ now complies with RFC 3207(38) and forgets any + knowledge obtained from the server not obtained from the TLS + negotiation itself. (Patch contributed by Bill Fenner; + bpo-829951(39).) + + * The *note socket: ef. module now supports TIPC + (‘http://tipc.sourceforge.net/’), a high-performance non-IP-based + protocol designed for use in clustered environments. TIPC + addresses are 4- or 5-tuples. (Contributed by Alberto Bertogli; + bpo-1646(40).) + + A new function, ‘create_connection()’, takes an address and + connects to it using an optional timeout value, returning the + connected socket object. This function also looks up the address’s + type and connects to it using IPv4 or IPv6 as appropriate. + Changing your code to use ‘create_connection()’ instead of + ‘socket(socket.AF_INET, ...)’ may be all that’s required to make + your code work with IPv6. + + * The base classes in the ‘SocketServer’ module now support calling a + ‘handle_timeout()’ method after a span of inactivity specified by + the server’s ‘timeout’ attribute. (Contributed by Michael + Pomraning.) The ‘serve_forever()’ method now takes an optional + poll interval measured in seconds, controlling how often the server + will check for a shutdown request. (Contributed by Pedro Werneck + and Jeffrey Yasskin; bpo-742598(41), bpo-1193577(42).) + + * The *note sqlite3: f2. module, maintained by Gerhard Häring, has + been updated from version 2.3.2 in Python 2.5 to version 2.4.1. + + * The *note struct: f8. module now supports the C99 ‘_Bool’ type, + using the format character ‘'?'’. (Contributed by David Remahl.) + + * The ‘Popen’ objects provided by the *note subprocess: f9. module + now have ‘terminate()’, ‘kill()’, and ‘send_signal()’ methods. On + Windows, ‘send_signal()’ only supports the ‘SIGTERM’ signal, and + all these methods are aliases for the Win32 API function + ‘TerminateProcess()’. (Contributed by Christian Heimes.) + + * A new variable in the *note sys: fd. module, ‘float_info’, is an + object containing information derived from the ‘float.h’ file about + the platform’s floating-point support. Attributes of this object + include ‘mant_dig’ (number of digits in the mantissa), ‘epsilon’ + (smallest difference between 1.0 and the next largest value + representable), and several others. (Contributed by Christian + Heimes; bpo-1534(43).) + + Another new variable, ‘dont_write_bytecode’, controls whether + Python writes any ‘.pyc’ or ‘.pyo’ files on importing a module. If + this variable is true, the compiled files are not written. The + variable is initially set on start-up by supplying the *note -B: + d38. switch to the Python interpreter, or by setting the *note + PYTHONDONTWRITEBYTECODE: d39. environment variable before running + the interpreter. Python code can subsequently change the value of + this variable to control whether bytecode files are written or not. + (Contributed by Neal Norwitz and Georg Brandl.) + + Information about the command-line arguments supplied to the Python + interpreter is available by reading attributes of a named tuple + available as ‘sys.flags’. For example, the ‘verbose’ attribute is + true if Python was executed in verbose mode, ‘debug’ is true in + debugging mode, etc. These attributes are all read-only. + (Contributed by Christian Heimes.) + + A new function, ‘getsizeof()’, takes a Python object and returns + the amount of memory used by the object, measured in bytes. + Built-in objects return correct results; third-party extensions may + not, but can define a ‘__sizeof__()’ method to return the object’s + size. (Contributed by Robert Schuppenies; bpo-2898(44).) + + It’s now possible to determine the current profiler and tracer + functions by calling *note sys.getprofile(): d47. and *note + sys.gettrace(): d48. (Contributed by Georg Brandl; bpo-1648(45).) + + * The *note tarfile: 101. module now supports POSIX.1-2001 (pax) + tarfiles in addition to the POSIX.1-1988 (ustar) and GNU tar + formats that were already supported. The default format is GNU + tar; specify the ‘format’ parameter to open a file using a + different format: + + tar = tarfile.open("output.tar", "w", + format=tarfile.PAX_FORMAT) + + The new ‘encoding’ and ‘errors’ parameters specify an encoding and + an error handling scheme for character conversions. ‘'strict'’, + ‘'ignore'’, and ‘'replace'’ are the three standard ways Python can + handle errors,; ‘'utf-8'’ is a special value that replaces bad + characters with their UTF-8 representation. (Character conversions + occur because the PAX format supports Unicode filenames, defaulting + to UTF-8 encoding.) + + The ‘TarFile.add()’ method now accepts an ‘exclude’ argument that’s + a function that can be used to exclude certain filenames from an + archive. The function must take a filename and return true if the + file should be excluded or false if it should be archived. The + function is applied to both the name initially passed to ‘add()’ + and to the names of files in recursively-added directories. + + (All changes contributed by Lars Gustäbel). + + * An optional ‘timeout’ parameter was added to the *note + telnetlib.Telnet: 597. class constructor, specifying a timeout + measured in seconds. (Added by Facundo Batista.) + + * The *note tempfile.NamedTemporaryFile: d49. class usually deletes + the temporary file it created when the file is closed. This + behaviour can now be changed by passing ‘delete=False’ to the + constructor. (Contributed by Damien Miller; bpo-1537850(46).) + + A new class, ‘SpooledTemporaryFile’, behaves like a temporary file + but stores its data in memory until a maximum size is exceeded. On + reaching that limit, the contents will be written to an on-disk + temporary file. (Contributed by Dustin J. Mitchell.) + + The ‘NamedTemporaryFile’ and ‘SpooledTemporaryFile’ classes both + work as context managers, so you can write ‘with + tempfile.NamedTemporaryFile() as tmp: ...’. (Contributed by + Alexander Belopolsky; bpo-2021(47).) + + * The ‘test.test_support’ module gained a number of context managers + useful for writing tests. ‘EnvironmentVarGuard()’ is a context + manager that temporarily changes environment variables and + automatically restores them to their old values. + + Another context manager, ‘TransientResource’, can surround calls to + resources that may or may not be available; it will catch and + ignore a specified list of exceptions. For example, a network test + may ignore certain failures when connecting to an external web + site: + + with test_support.TransientResource(IOError, + errno=errno.ETIMEDOUT): + f = urllib.urlopen('https://sf.net') + ... + + Finally, ‘check_warnings()’ resets the ‘warning’ module’s warning + filters and returns an object that will record all warning messages + triggered (bpo-3781(48)): + + with test_support.check_warnings() as wrec: + warnings.simplefilter("always") + # ... code that triggers a warning ... + assert str(wrec.message) == "function is outdated" + assert len(wrec.warnings) == 1, "Multiple warnings raised" + + (Contributed by Brett Cannon.) + + * The *note textwrap: 108. module can now preserve existing + whitespace at the beginnings and ends of the newly-created lines by + specifying ‘drop_whitespace=False’ as an argument: + + >>> S = """This sentence has a bunch of + ... extra whitespace.""" + >>> print textwrap.fill(S, width=15) + This sentence + has a bunch + of extra + whitespace. + >>> print textwrap.fill(S, drop_whitespace=False, width=15) + This sentence + has a bunch + of extra + whitespace. + >>> + + (Contributed by Dwayne Bailey; bpo-1581073(49).) + + * The *note threading: 109. module API is being changed to use + properties such as ‘daemon’ instead of ‘setDaemon()’ and + ‘isDaemon()’ methods, and some methods have been renamed to use + underscores instead of camel-case; for example, the ‘activeCount()’ + method is renamed to ‘active_count()’. Both the 2.6 and 3.0 + versions of the module support the same properties and renamed + methods, but don’t remove the old methods. No date has been set + for the deprecation of the old APIs in Python 3.x; the old APIs + won’t be removed in any 2.x version. (Carried out by several + people, most notably Benjamin Peterson.) + + The *note threading: 109. module’s ‘Thread’ objects gained an + ‘ident’ property that returns the thread’s identifier, a nonzero + integer. (Contributed by Gregory P. Smith; bpo-2871(50).) + + * The *note timeit: 10b. module now accepts callables as well as + strings for the statement being timed and for the setup code. Two + convenience functions were added for creating ‘Timer’ instances: + ‘repeat(stmt, setup, time, repeat, number)’ and ‘timeit(stmt, + setup, time, number)’ create an instance and call the corresponding + method. (Contributed by Erik Demaine; bpo-1533909(51).) + + * The ‘Tkinter’ module now accepts lists and tuples for options, + separating the elements by spaces before passing the resulting + value to Tcl/Tk. (Contributed by Guilherme Polo; bpo-2906(52).) + + * The *note turtle: 116. module for turtle graphics was greatly + enhanced by Gregor Lingl. New features in the module include: + + * Better animation of turtle movement and rotation. + + * Control over turtle movement using the new ‘delay()’, + ‘tracer()’, and ‘speed()’ methods. + + * The ability to set new shapes for the turtle, and to define a + new coordinate system. + + * Turtles now have an ‘undo()’ method that can roll back + actions. + + * Simple support for reacting to input events such as mouse and + keyboard activity, making it possible to write simple games. + + * A ‘turtle.cfg’ file can be used to customize the starting + appearance of the turtle’s screen. + + * The module’s docstrings can be replaced by new docstrings that + have been translated into another language. + + (bpo-1513695(53)) + + * An optional ‘timeout’ parameter was added to the ‘urllib.urlopen()’ + function and the ‘urllib.ftpwrapper’ class constructor, as well as + the ‘urllib2.urlopen()’ function. The parameter specifies a + timeout measured in seconds. For example: + + >>> u = urllib2.urlopen("http://slow.example.com", + timeout=3) + Traceback (most recent call last): + ... + urllib2.URLError: + >>> + + (Added by Facundo Batista.) + + * The Unicode database provided by the *note unicodedata: 11a. module + has been updated to version 5.1.0. (Updated by Martin von Löwis; + bpo-3811(54).) + + * The *note warnings: 126. module’s ‘formatwarning()’ and + ‘showwarning()’ gained an optional `line' argument that can be used + to supply the line of source code. (Added as part of + bpo-1631171(55), which re-implemented part of the *note warnings: + 126. module in C code.) + + A new function, ‘catch_warnings()’, is a context manager intended + for testing purposes that lets you temporarily modify the warning + filters and then restore their original values (bpo-3781(56)). + + * The XML-RPC ‘SimpleXMLRPCServer’ and ‘DocXMLRPCServer’ classes can + now be prevented from immediately opening and binding to their + socket by passing ‘False’ as the `bind_and_activate' constructor + parameter. This can be used to modify the instance’s + ‘allow_reuse_address’ attribute before calling the ‘server_bind()’ + and ‘server_activate()’ methods to open the socket and begin + listening for connections. (Contributed by Peter Parente; + bpo-1599845(57).) + + ‘SimpleXMLRPCServer’ also has a ‘_send_traceback_header’ attribute; + if true, the exception and formatted traceback are returned as HTTP + headers “X-Exception” and “X-Traceback”. This feature is for + debugging purposes only and should not be used on production + servers because the tracebacks might reveal passwords or other + sensitive information. (Contributed by Alan McIntyre as part of + his project for Google’s Summer of Code 2007.) + + * The ‘xmlrpclib’ module no longer automatically converts *note + datetime.date: 193. and *note datetime.time: 4f3. to the + ‘xmlrpclib.DateTime’ type; the conversion semantics were not + necessarily correct for all applications. Code using ‘xmlrpclib’ + should convert ‘date’ and *note time: 4f3. instances. + (bpo-1330538(58)) The code can also handle dates before 1900 + (contributed by Ralf Schmitt; bpo-2014(59)) and 64-bit integers + represented by using ‘’ in XML-RPC responses (contributed by + Riku Lindblad; bpo-2985(60)). + + * The *note zipfile: 142. module’s ‘ZipFile’ class now has + ‘extract()’ and ‘extractall()’ methods that will unpack a single + file or all the files in the archive to the current directory, or + to a specified directory: + + z = zipfile.ZipFile('python-251.zip') + + # Unpack a single file, writing it relative + # to the /tmp directory. + z.extract('Python/sysmodule.c', '/tmp') + + # Unpack all the files in the archive. + z.extractall() + + (Contributed by Alan McIntyre; bpo-467924(61).) + + The *note open(): 4f0, ‘read()’ and ‘extract()’ methods can now + take either a filename or a ‘ZipInfo’ object. This is useful when + an archive accidentally contains a duplicated filename. + (Contributed by Graham Horler; bpo-1775025(62).) + + Finally, *note zipfile: 142. now supports using Unicode filenames + for archived files. (Contributed by Alexey Borzenkov; + bpo-1734346(63).) + +* Menu: + +* The ast module:: +* The future_builtins module:: +* The json module; JavaScript Object Notation: The json module JavaScript Object Notation. +* The plistlib module; A Property-List Parser: The plistlib module A Property-List Parser. +* ctypes Enhancements:: +* Improved SSL Support:: + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1736190 + + (2) https://www.jcea.es/programacion/pybsddb.htm + + (3) https://bugs.python.org/issue1817 + + (4) https://bugs.python.org/issue600362 + + (5) https://bugs.python.org/issue1381 + + (6) https://bugs.python.org/issue1638033 + + (7) https://bugs.python.org/issue1158 + + (8) http://speleotrove.com/decimal/decarith.html + + (9) https://bugs.python.org/issue1221598 + + (10) https://bugs.python.org/issue1739906 + + (11) https://bugs.python.org/issue1001604 + + (12) https://bugs.python.org/issue2138 + + (13) https://bugs.python.org/issue2819 + + (14) https://bugs.python.org/issue711019 + + (15) https://bugs.python.org/issue1640 + + (16) https://bugs.python.org/issue1663329 + + (17) https://bugs.python.org/issue1181 + + (18) https://bugs.python.org/issue1273829 + + (19) https://bugs.python.org/issue1115886 + + (20) https://bugs.python.org/issue1339796 + + (21) https://bugs.python.org/issue957650 + + (22) https://bugs.python.org/issue1393667 + + (23) https://bugs.python.org/issue1106316 + + (24) https://bugs.python.org/issue2439 + + (25) https://bugs.python.org/issue1137 + + (26) https://bugs.python.org/issue1727780 + + (27) https://bugs.python.org/issue1681432 + + (28) https://bugs.python.org/issue846388 + + (29) https://bugs.python.org/issue3487 + + (30) https://bugs.python.org/issue2250 + + (31) https://bugs.python.org/issue1861 + + (32) https://bugs.python.org/issue1657 + + (33) https://bugs.python.org/issue2663 + + (34) https://bugs.python.org/issue1583 + + (35) https://bugs.python.org/issue2240 + + (36) https://tools.ietf.org/html/rfc2033.html + + (37) https://bugs.python.org/issue957003 + + (38) https://tools.ietf.org/html/rfc3207.html + + (39) https://bugs.python.org/issue829951 + + (40) https://bugs.python.org/issue1646 + + (41) https://bugs.python.org/issue742598 + + (42) https://bugs.python.org/issue1193577 + + (43) https://bugs.python.org/issue1534 + + (44) https://bugs.python.org/issue2898 + + (45) https://bugs.python.org/issue1648 + + (46) https://bugs.python.org/issue1537850 + + (47) https://bugs.python.org/issue2021 + + (48) https://bugs.python.org/issue3781 + + (49) https://bugs.python.org/issue1581073 + + (50) https://bugs.python.org/issue2871 + + (51) https://bugs.python.org/issue1533909 + + (52) https://bugs.python.org/issue2906 + + (53) https://bugs.python.org/issue1513695 + + (54) https://bugs.python.org/issue3811 + + (55) https://bugs.python.org/issue1631171 + + (56) https://bugs.python.org/issue3781 + + (57) https://bugs.python.org/issue1599845 + + (58) https://bugs.python.org/issue1330538 + + (59) https://bugs.python.org/issue2014 + + (60) https://bugs.python.org/issue2985 + + (61) https://bugs.python.org/issue467924 + + (62) https://bugs.python.org/issue1775025 + + (63) https://bugs.python.org/issue1734346 + + +File: python.info, Node: The ast module, Next: The future_builtins module, Up: New and Improved Modules<2> + +1.11.18.1 The ‘ast’ module +.......................... + +The *note ast: 8. module provides an Abstract Syntax Tree representation +of Python code, and Armin Ronacher contributed a set of helper functions +that perform a variety of common tasks. These will be useful for HTML +templating packages, code analyzers, and similar tools that process +Python code. + +The ‘parse()’ function takes an expression and returns an AST. The +‘dump()’ function outputs a representation of a tree, suitable for +debugging: + + import ast + + t = ast.parse(""" + d = {} + for i in 'abcdefghijklm': + d[i + i] = ord(i) - ord('a') + 1 + print d + """) + print ast.dump(t) + +This outputs a deeply nested tree: + + Module(body=[ + Assign(targets=[ + Name(id='d', ctx=Store()) + ], value=Dict(keys=[], values=[])) + For(target=Name(id='i', ctx=Store()), + iter=Str(s='abcdefghijklm'), body=[ + Assign(targets=[ + Subscript(value= + Name(id='d', ctx=Load()), + slice= + Index(value= + BinOp(left=Name(id='i', ctx=Load()), op=Add(), + right=Name(id='i', ctx=Load()))), ctx=Store()) + ], value= + BinOp(left= + BinOp(left= + Call(func= + Name(id='ord', ctx=Load()), args=[ + Name(id='i', ctx=Load()) + ], keywords=[], starargs=None, kwargs=None), + op=Sub(), right=Call(func= + Name(id='ord', ctx=Load()), args=[ + Str(s='a') + ], keywords=[], starargs=None, kwargs=None)), + op=Add(), right=Num(n=1))) + ], orelse=[]) + Print(dest=None, values=[ + Name(id='d', ctx=Load()) + ], nl=True) + ]) + +The ‘literal_eval()’ method takes a string or an AST representing a +literal expression, parses and evaluates it, and returns the resulting +value. A literal expression is a Python expression containing only +strings, numbers, dictionaries, etc. but no statements or function +calls. If you need to evaluate an expression but cannot accept the +security risk of using an *note eval(): b90. call, ‘literal_eval()’ will +handle it safely: + + >>> literal = '("a", "b", {2:4, 3:8, 1:2})' + >>> print ast.literal_eval(literal) + ('a', 'b', {1: 2, 2: 4, 3: 8}) + >>> print ast.literal_eval('"a" + "b"') + Traceback (most recent call last): + ... + ValueError: malformed string + +The module also includes ‘NodeVisitor’ and ‘NodeTransformer’ classes for +traversing and modifying an AST, and functions for common +transformations such as changing line numbers. + + +File: python.info, Node: The future_builtins module, Next: The json module JavaScript Object Notation, Prev: The ast module, Up: New and Improved Modules<2> + +1.11.18.2 The ‘future_builtins’ module +...................................... + +Python 3.0 makes many changes to the repertoire of built-in functions, +and most of the changes can’t be introduced in the Python 2.x series +because they would break compatibility. The ‘future_builtins’ module +provides versions of these built-in functions that can be imported when +writing 3.0-compatible code. + +The functions in this module currently include: + + * ‘ascii(obj)’: equivalent to *note repr(): 7cc. In Python 3.0, + *note repr(): 7cc. will return a Unicode string, while *note + ascii(): d4c. will return a pure ASCII bytestring. + + * ‘filter(predicate, iterable)’, ‘map(func, iterable1, ...)’: the 3.0 + versions return iterators, unlike the 2.x builtins which return + lists. + + * ‘hex(value)’, ‘oct(value)’: instead of calling the ‘__hex__()’ or + ‘__oct__()’ methods, these versions will call the *note + __index__(): 18a. method and convert the result to hexadecimal or + octal. *note oct(): c65. will use the new ‘0o’ notation for its + result. + + +File: python.info, Node: The json module JavaScript Object Notation, Next: The plistlib module A Property-List Parser, Prev: The future_builtins module, Up: New and Improved Modules<2> + +1.11.18.3 The ‘json’ module: JavaScript Object Notation +....................................................... + +The new *note json: a4. module supports the encoding and decoding of +Python types in JSON (Javascript Object Notation). JSON is a +lightweight interchange format often used in web applications. For more +information about JSON, see ‘http://www.json.org’. + +*note json: a4. comes with support for decoding and encoding most +built-in Python types. The following example encodes and decodes a +dictionary: + + >>> import json + >>> data = {"spam": "foo", "parrot": 42} + >>> in_json = json.dumps(data) # Encode the data + >>> in_json + '{"parrot": 42, "spam": "foo"}' + >>> json.loads(in_json) # Decode into a Python object + {"spam": "foo", "parrot": 42} + +It’s also possible to write your own decoders and encoders to support +more types. Pretty-printing of the JSON strings is also supported. + +*note json: a4. (originally called simplejson) was written by Bob +Ippolito. + + +File: python.info, Node: The plistlib module A Property-List Parser, Next: ctypes Enhancements, Prev: The json module JavaScript Object Notation, Up: New and Improved Modules<2> + +1.11.18.4 The ‘plistlib’ module: A Property-List Parser +....................................................... + +The ‘.plist’ format is commonly used on Mac OS X to store basic data +types (numbers, strings, lists, and dictionaries) by serializing them +into an XML-based format. It resembles the XML-RPC serialization of +data types. + +Despite being primarily used on Mac OS X, the format has nothing +Mac-specific about it and the Python implementation works on any +platform that Python supports, so the *note plistlib: cf. module has +been promoted to the standard library. + +Using the module is simple: + + import sys + import plistlib + import datetime + + # Create data structure + data_struct = dict(lastAccessed=datetime.datetime.now(), + version=1, + categories=('Personal','Shared','Private')) + + # Create string containing XML. + plist_str = plistlib.writePlistToString(data_struct) + new_struct = plistlib.readPlistFromString(plist_str) + print data_struct + print new_struct + + # Write data structure to a file and read it back. + plistlib.writePlist(data_struct, '/tmp/customizations.plist') + new_struct = plistlib.readPlist('/tmp/customizations.plist') + + # read/writePlist accepts file-like objects as well as paths. + plistlib.writePlist(data_struct, sys.stdout) + + +File: python.info, Node: ctypes Enhancements, Next: Improved SSL Support, Prev: The plistlib module A Property-List Parser, Up: New and Improved Modules<2> + +1.11.18.5 ctypes Enhancements +............................. + +Thomas Heller continued to maintain and enhance the *note ctypes: 2b. +module. + +*note ctypes: 2b. now supports a ‘c_bool’ datatype that represents the +C99 ‘bool’ type. (Contributed by David Remahl; bpo-1649190(1).) + +The *note ctypes: 2b. string, buffer and array types have improved +support for extended slicing syntax, where various combinations of +‘(start, stop, step)’ are supplied. (Implemented by Thomas Wouters.) + +All *note ctypes: 2b. data types now support ‘from_buffer()’ and +‘from_buffer_copy()’ methods that create a ctypes instance based on a +provided buffer object. ‘from_buffer_copy()’ copies the contents of the +object, while ‘from_buffer()’ will share the same memory area. + +A new calling convention tells *note ctypes: 2b. to clear the ‘errno’ or +Win32 LastError variables at the outset of each wrapped call. +(Implemented by Thomas Heller; bpo-1798(2).) + +You can now retrieve the Unix ‘errno’ variable after a function call. +When creating a wrapped function, you can supply ‘use_errno=True’ as a +keyword parameter to the ‘DLL()’ function and then call the module-level +methods ‘set_errno()’ and ‘get_errno()’ to set and retrieve the error +value. + +The Win32 LastError variable is similarly supported by the ‘DLL()’, +‘OleDLL()’, and ‘WinDLL()’ functions. You supply ‘use_last_error=True’ +as a keyword parameter and then call the module-level methods +‘set_last_error()’ and ‘get_last_error()’. + +The ‘byref()’ function, used to retrieve a pointer to a ctypes instance, +now has an optional `offset' parameter that is a byte count that will be +added to the returned pointer. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1649190 + + (2) https://bugs.python.org/issue1798 + + +File: python.info, Node: Improved SSL Support, Prev: ctypes Enhancements, Up: New and Improved Modules<2> + +1.11.18.6 Improved SSL Support +.............................. + +Bill Janssen made extensive improvements to Python 2.6’s support for the +Secure Sockets Layer by adding a new module, *note ssl: f3, that’s built +atop the OpenSSL(1) library. This new module provides more control over +the protocol negotiated, the X.509 certificates used, and has better +support for writing SSL servers (as opposed to clients) in Python. The +existing SSL support in the *note socket: ef. module hasn’t been removed +and continues to work, though it will be removed in Python 3.0. + +To use the new module, you must first create a TCP connection in the +usual way and then pass it to the *note ssl.wrap_socket(): 46f. +function. It’s possible to specify whether a certificate is required, +and to obtain certificate info by calling the ‘getpeercert()’ method. + +See also +........ + +The documentation for the *note ssl: f3. module. + + ---------- Footnotes ---------- + + (1) https://www.openssl.org/ + + +File: python.info, Node: Deprecations and Removals, Next: Build and C API Changes<9>, Prev: New and Improved Modules<2>, Up: What’s New in Python 2 6 + +1.11.19 Deprecations and Removals +--------------------------------- + + * String exceptions have been removed. Attempting to use them raises + a *note TypeError: 192. + + * Changes to the *note Exception: 1a9. interface as dictated by PEP + 352(1) continue to be made. For 2.6, the ‘message’ attribute is + being deprecated in favor of the ‘args’ attribute. + + * (3.0-warning mode) Python 3.0 will feature a reorganized standard + library that will drop many outdated modules and rename others. + Python 2.6 running in 3.0-warning mode will warn about these + modules when they are imported. + + The list of deprecated modules is: ‘audiodev’, ‘bgenlocations’, + ‘buildtools’, ‘bundlebuilder’, ‘Canvas’, ‘compiler’, ‘dircache’, + ‘dl’, ‘fpformat’, ‘gensuitemodule’, ‘ihooks’, ‘imageop’, ‘imgfile’, + ‘linuxaudiodev’, ‘mhlib’, ‘mimetools’, ‘multifile’, ‘new’, ‘pure’, + ‘statvfs’, ‘sunaudiodev’, ‘test.testall’, and ‘toaiff’. + + * The ‘gopherlib’ module has been removed. + + * The ‘MimeWriter’ module and ‘mimify’ module have been deprecated; + use the *note email: 69. package instead. + + * The ‘md5’ module has been deprecated; use the *note hashlib: 8d. + module instead. + + * The ‘posixfile’ module has been deprecated; *note fcntl.lockf(): + d52. provides better locking. + + * The ‘popen2’ module has been deprecated; use the *note subprocess: + f9. module. + + * The ‘rgbimg’ module has been removed. + + * The ‘sets’ module has been deprecated; it’s better to use the + built-in *note set: b6f. and *note frozenset: bee. types. + + * The ‘sha’ module has been deprecated; use the *note hashlib: 8d. + module instead. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0352 + + +File: python.info, Node: Build and C API Changes<9>, Next: Porting to Python 2 6, Prev: Deprecations and Removals, Up: What’s New in Python 2 6 + +1.11.20 Build and C API Changes +------------------------------- + +Changes to Python’s build process and to the C API include: + + * Python now must be compiled with C89 compilers (after 19 years!). + This means that the Python source tree has dropped its own + implementations of ‘memmove()’ and ‘strerror()’, which are in the + C89 standard library. + + * Python 2.6 can be built with Microsoft Visual Studio 2008 (version + 9.0), and this is the new default compiler. See the ‘PCbuild’ + directory for the build files. (Implemented by Christian Heimes.) + + * On Mac OS X, Python 2.6 can be compiled as a 4-way universal build. + The ‘configure’ script can take a + ‘--with-universal-archs=[32-bit|64-bit|all]’ switch, controlling + whether the binaries are built for 32-bit architectures (x86, + PowerPC), 64-bit (x86-64 and PPC-64), or both. (Contributed by + Ronald Oussoren.) + + * The BerkeleyDB module now has a C API object, available as + ‘bsddb.db.api’. This object can be used by other C extensions that + wish to use the ‘bsddb’ module for their own purposes. + (Contributed by Duncan Grisby.) + + * The new buffer interface, previously described in *note the PEP + 3118 section: d24, adds *note PyObject_GetBuffer(): d25. and *note + PyBuffer_Release(): d54, as well as a few other functions. + + * Python’s use of the C stdio library is now thread-safe, or at least + as thread-safe as the underlying library is. A long-standing + potential bug occurred if one thread closed a file object while + another thread was reading from or writing to the object. In 2.6 + file objects have a reference count, manipulated by the + ‘PyFile_IncUseCount()’ and ‘PyFile_DecUseCount()’ functions. File + objects can’t be closed unless the reference count is zero. + ‘PyFile_IncUseCount()’ should be called while the GIL is still + held, before carrying out an I/O operation using the ‘FILE *’ + pointer, and ‘PyFile_DecUseCount()’ should be called immediately + after the GIL is re-acquired. (Contributed by Antoine Pitrou and + Gregory P. Smith.) + + * Importing modules simultaneously in two different threads no longer + deadlocks; it will now raise an *note ImportError: 334. A new API + function, *note PyImport_ImportModuleNoBlock(): 9c1, will look for + a module in ‘sys.modules’ first, then try to import it after + acquiring an import lock. If the import lock is held by another + thread, an *note ImportError: 334. is raised. (Contributed by + Christian Heimes.) + + * Several functions return information about the platform’s + floating-point support. *note PyFloat_GetMax(): d55. returns the + maximum representable floating point value, and *note + PyFloat_GetMin(): d56. returns the minimum positive value. *note + PyFloat_GetInfo(): d57. returns an object containing more + information from the ‘float.h’ file, such as ‘"mant_dig"’ (number + of digits in the mantissa), ‘"epsilon"’ (smallest difference + between 1.0 and the next largest value representable), and several + others. (Contributed by Christian Heimes; bpo-1534(1).) + + * C functions and methods that use *note PyComplex_AsCComplex(): d58. + will now accept arguments that have a *note __complex__(): 18d. + method. In particular, the functions in the *note cmath: 19. + module will now accept objects with this method. This is a + backport of a Python 3.0 change. (Contributed by Mark Dickinson; + bpo-1675423(2).) + + * Python’s C API now includes two functions for case-insensitive + string comparisons, ‘PyOS_stricmp(char*, char*)’ and + ‘PyOS_strnicmp(char*, char*, Py_ssize_t)’. (Contributed by + Christian Heimes; bpo-1635(3).) + + * Many C extensions define their own little macro for adding integers + and strings to the module’s dictionary in the ‘init*’ function. + Python 2.6 finally defines standard macros for adding values to a + module, *note PyModule_AddStringMacro: d59. and + ‘PyModule_AddIntMacro()’. (Contributed by Christian Heimes.) + + * Some macros were renamed in both 3.0 and 2.6 to make it clearer + that they are macros, not functions. ‘Py_Size()’ became + ‘Py_SIZE()’, ‘Py_Type()’ became ‘Py_TYPE()’, and ‘Py_Refcnt()’ + became ‘Py_REFCNT()’. The mixed-case macros are still available in + Python 2.6 for backward compatibility. (bpo-1629(4)) + + * Distutils now places C extensions it builds in a different + directory when running on a debug version of Python. (Contributed + by Collin Winter; bpo-1530959(5).) + + * Several basic data types, such as integers and strings, maintain + internal free lists of objects that can be re-used. The data + structures for these free lists now follow a naming convention: the + variable is always named ‘free_list’, the counter is always named + ‘numfree’, and a macro ‘Py_MAXFREELIST’ is always + defined. + + * A new Makefile target, “make patchcheck”, prepares the Python + source tree for making a patch: it fixes trailing whitespace in all + modified ‘.py’ files, checks whether the documentation has been + changed, and reports whether the ‘Misc/ACKS’ and ‘Misc/NEWS’ files + have been updated. (Contributed by Brett Cannon.) + + Another new target, “make profile-opt”, compiles a Python binary + using GCC’s profile-guided optimization. It compiles Python with + profiling enabled, runs the test suite to obtain a set of profiling + results, and then compiles using these results for optimization. + (Contributed by Gregory P. Smith.) + +* Menu: + +* Port-Specific Changes; Windows: Port-Specific Changes Windows<2>. +* Port-Specific Changes; Mac OS X: Port-Specific Changes Mac OS X<2>. +* Port-Specific Changes; IRIX: Port-Specific Changes IRIX. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1534 + + (2) https://bugs.python.org/issue1675423 + + (3) https://bugs.python.org/issue1635 + + (4) https://bugs.python.org/issue1629 + + (5) https://bugs.python.org/issue1530959 + + +File: python.info, Node: Port-Specific Changes Windows<2>, Next: Port-Specific Changes Mac OS X<2>, Up: Build and C API Changes<9> + +1.11.20.1 Port-Specific Changes: Windows +........................................ + + * The support for Windows 95, 98, ME and NT4 has been dropped. + Python 2.6 requires at least Windows 2000 SP4. + + * The new default compiler on Windows is Visual Studio 2008 (version + 9.0). The build directories for Visual Studio 2003 (version 7.1) + and 2005 (version 8.0) were moved into the PC/ directory. The new + ‘PCbuild’ directory supports cross compilation for X64, debug + builds and Profile Guided Optimization (PGO). PGO builds are + roughly 10% faster than normal builds. (Contributed by Christian + Heimes with help from Amaury Forgeot d’Arc and Martin von Löwis.) + + * The *note msvcrt: b6. module now supports both the normal and wide + char variants of the console I/O API. The ‘getwch()’ function reads + a keypress and returns a Unicode value, as does the ‘getwche()’ + function. The ‘putwch()’ function takes a Unicode character and + writes it to the console. (Contributed by Christian Heimes.) + + * *note os.path.expandvars(): d44. will now expand environment + variables in the form “%var%”, and “~user” will be expanded into + the user’s home directory path. (Contributed by Josiah Carlson; + bpo-957650(1).) + + * The *note socket: ef. module’s socket objects now have an ‘ioctl()’ + method that provides a limited interface to the ‘WSAIoctl()’ system + interface. + + * The ‘_winreg’ module now has a function, + ‘ExpandEnvironmentStrings()’, that expands environment variable + references such as ‘%NAME%’ in an input string. The handle objects + provided by this module now support the context protocol, so they + can be used in *note with: 6e9. statements. (Contributed by + Christian Heimes.) + + ‘_winreg’ also has better support for x64 systems, exposing the + ‘DisableReflectionKey()’, ‘EnableReflectionKey()’, and + ‘QueryReflectionKey()’ functions, which enable and disable registry + reflection for 32-bit processes running on 64-bit systems. + (bpo-1753245(2)) + + * The *note msilib: b5. module’s ‘Record’ object gained + ‘GetInteger()’ and ‘GetString()’ methods that return field values + as an integer or a string. (Contributed by Floris Bruynooghe; + bpo-2125(3).) + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue957650 + + (2) https://bugs.python.org/issue1753245 + + (3) https://bugs.python.org/issue2125 + + +File: python.info, Node: Port-Specific Changes Mac OS X<2>, Next: Port-Specific Changes IRIX, Prev: Port-Specific Changes Windows<2>, Up: Build and C API Changes<9> + +1.11.20.2 Port-Specific Changes: Mac OS X +......................................... + + * When compiling a framework build of Python, you can now specify the + framework name to be used by providing the ‘--with-framework-name=’ + option to the ‘configure’ script. + + * The ‘macfs’ module has been removed. This in turn required the + ‘macostools.touched()’ function to be removed because it depended + on the ‘macfs’ module. (bpo-1490190(1)) + + * Many other Mac OS modules have been deprecated and will be removed + in Python 3.0: ‘_builtinSuites’, ‘aepack’, ‘aetools’, ‘aetypes’, + ‘applesingle’, ‘appletrawmain’, ‘appletrunner’, ‘argvemulator’, + ‘Audio_mac’, ‘autoGIL’, ‘Carbon’, ‘cfmfile’, ‘CodeWarrior’, + ‘ColorPicker’, ‘EasyDialogs’, ‘Explorer’, ‘Finder’, ‘FrameWork’, + ‘findertools’, ‘ic’, ‘icglue’, ‘icopen’, ‘macerrors’, ‘MacOS’, + ‘macfs’, ‘macostools’, ‘macresource’, ‘MiniAEFrame’, ‘Nav’, + ‘Netscape’, ‘OSATerminology’, ‘pimp’, ‘PixMapWrapper’, ‘StdSuites’, + ‘SystemEvents’, ‘Terminal’, and ‘terminalcommand’. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1490190 + + +File: python.info, Node: Port-Specific Changes IRIX, Prev: Port-Specific Changes Mac OS X<2>, Up: Build and C API Changes<9> + +1.11.20.3 Port-Specific Changes: IRIX +..................................... + +A number of old IRIX-specific modules were deprecated and will be +removed in Python 3.0: ‘al’ and ‘AL’, ‘cd’, ‘cddb’, ‘cdplayer’, ‘CL’ and +‘cl’, ‘DEVICE’, ‘ERRNO’, ‘FILE’, ‘FL’ and ‘fl’, ‘flp’, ‘fm’, ‘GET’, +‘GLWS’, ‘GL’ and ‘gl’, ‘IN’, ‘IOCTL’, ‘jpeg’, ‘panelparser’, ‘readcd’, +‘SV’ and ‘sv’, ‘torgb’, ‘videoreader’, and ‘WAIT’. + + +File: python.info, Node: Porting to Python 2 6, Next: Acknowledgements<2>, Prev: Build and C API Changes<9>, Up: What’s New in Python 2 6 + +1.11.21 Porting to Python 2.6 +----------------------------- + +This section lists previously described changes and other bugfixes that +may require changes to your code: + + * Classes that aren’t supposed to be hashable should set ‘__hash__ = + None’ in their definitions to indicate the fact. + + * String exceptions have been removed. Attempting to use them raises + a *note TypeError: 192. + + * The *note __init__(): d5e. method of *note collections.deque: 531. + now clears any existing contents of the deque before adding + elements from the iterable. This change makes the behavior match + ‘list.__init__()’. + + * *note object.__init__(): d5e. previously accepted arbitrary + arguments and keyword arguments, ignoring them. In Python 2.6, + this is no longer allowed and will result in a *note TypeError: + 192. This will affect *note __init__(): d5e. methods that end up + calling the corresponding method on *note object: 2b0. (perhaps + through using *note super(): 4e2.). See bpo-1683368(1) for + discussion. + + * The ‘Decimal’ constructor now accepts leading and trailing + whitespace when passed a string. Previously it would raise an + ‘InvalidOperation’ exception. On the other hand, the + ‘create_decimal()’ method of ‘Context’ objects now explicitly + disallows extra whitespace, raising a ‘ConversionSyntax’ exception. + + * Due to an implementation accident, if you passed a file path to the + built-in *note __import__(): 610. function, it would actually + import the specified file. This was never intended to work, + however, and the implementation now explicitly checks for this case + and raises an *note ImportError: 334. + + * C API: the *note PyImport_Import(): d5f. and *note + PyImport_ImportModule(): c73. functions now default to absolute + imports, not relative imports. This will affect C extensions that + import other modules. + + * C API: extension data types that shouldn’t be hashable should + define their ‘tp_hash’ slot to *note PyObject_HashNotImplemented(): + d31. + + * The *note socket: ef. module exception *note socket.error: 995. now + inherits from *note IOError: 992. Previously it wasn’t a subclass + of ‘StandardError’ but now it is, through *note IOError: 992. + (Implemented by Gregory P. Smith; bpo-1706815(2).) + + * The ‘xmlrpclib’ module no longer automatically converts *note + datetime.date: 193. and *note datetime.time: 4f3. to the + ‘xmlrpclib.DateTime’ type; the conversion semantics were not + necessarily correct for all applications. Code using ‘xmlrpclib’ + should convert ‘date’ and *note time: 4f3. instances. + (bpo-1330538(3)) + + * (3.0-warning mode) The *note Exception: 1a9. class now warns when + accessed using slicing or index access; having *note Exception: + 1a9. behave like a tuple is being phased out. + + * (3.0-warning mode) inequality comparisons between two dictionaries + or two objects that don’t implement comparison methods are reported + as warnings. ‘dict1 == dict2’ still works, but ‘dict1 < dict2’ is + being phased out. + + Comparisons between cells, which are an implementation detail of + Python’s scoping rules, also cause warnings because such + comparisons are forbidden entirely in 3.0. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue1683368 + + (2) https://bugs.python.org/issue1706815 + + (3) https://bugs.python.org/issue1330538 + + +File: python.info, Node: Acknowledgements<2>, Prev: Porting to Python 2 6, Up: What’s New in Python 2 6 + +1.11.22 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Georg Brandl, Steve Brown, Nick Coghlan, Ralph Corderoy, Jim +Jewett, Kent Johnson, Chris Lambacher, Martin Michlmayr, Antoine Pitrou, +Brian Warner. + + +File: python.info, Node: What’s New in Python 2 5, Next: What’s New in Python 2 4, Prev: What’s New in Python 2 6, Up: What’s New in Python + +1.12 What’s New in Python 2.5 +============================= + + +Author: A.M. Kuchling + +This article explains the new features in Python 2.5. The final release +of Python 2.5 is scheduled for August 2006; PEP 356(1) describes the +planned release schedule. + +The changes in Python 2.5 are an interesting mix of language and library +improvements. The library enhancements will be more important to +Python’s user community, I think, because several widely-useful packages +were added. New modules include ElementTree for XML processing +(‘xml.etree’), the SQLite database module (‘sqlite’), and the *note +ctypes: 2b. module for calling C functions. + +The language changes are of middling significance. Some pleasant new +features were added, but most of them aren’t features that you’ll use +every day. Conditional expressions were finally added to the language +using a novel syntax; see section *note PEP 308; Conditional +Expressions: d64. The new ‘*note with: 6e9.’ statement will make +writing cleanup code easier (section *note PEP 343; The ‘with’ +statement: d65.). Values can now be passed into generators (section +*note PEP 342; New Generator Features: d66.). Imports are now visible +as either absolute or relative (section *note PEP 328; Absolute and +Relative Imports: d67.). Some corner cases of exception handling are +handled better (section *note PEP 341; Unified try/except/finally: +d68.). All these improvements are worthwhile, but they’re improvements +to one specific language feature or another; none of them are broad +modifications to Python’s semantics. + +As well as the language and library additions, other improvements and +bugfixes were made throughout the source tree. A search through the SVN +change logs finds there were 353 patches applied and 458 bugs fixed +between Python 2.4 and 2.5. (Both figures are likely to be +underestimates.) + +This article doesn’t try to be a complete specification of the new +features; instead changes are briefly introduced using helpful examples. +For full details, you should always refer to the documentation for +Python 2.5 at ‘https://docs.python.org’. If you want to understand the +complete implementation and design rationale, refer to the PEP for a +particular new feature. + +Comments, suggestions, and error reports for this document are welcome; +please e-mail them to the author or open a bug in the Python bug +tracker. + +* Menu: + +* PEP 308; Conditional Expressions: PEP 308 Conditional Expressions. +* PEP 309; Partial Function Application: PEP 309 Partial Function Application. +* PEP 314; Metadata for Python Software Packages v1.1: PEP 314 Metadata for Python Software Packages v1 1. +* PEP 328; Absolute and Relative Imports: PEP 328 Absolute and Relative Imports. +* PEP 338; Executing Modules as Scripts: PEP 338 Executing Modules as Scripts. +* PEP 341; Unified try/except/finally: PEP 341 Unified try/except/finally. +* PEP 342; New Generator Features: PEP 342 New Generator Features. +* PEP 343; The ‘with’ statement: PEP 343 The ‘with’ statement<2>. +* PEP 352; Exceptions as New-Style Classes: PEP 352 Exceptions as New-Style Classes. +* PEP 353; Using ssize_t as the index type: PEP 353 Using ssize_t as the index type. +* PEP 357; The ‘__index__’ method: PEP 357 The ‘__index__’ method. +* Other Language Changes: Other Language Changes<11>. +* New, Improved, and Removed Modules: New Improved and Removed Modules. +* Build and C API Changes: Build and C API Changes<10>. +* Porting to Python 2.5: Porting to Python 2 5. +* Acknowledgements: Acknowledgements<3>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0356 + + +File: python.info, Node: PEP 308 Conditional Expressions, Next: PEP 309 Partial Function Application, Up: What’s New in Python 2 5 + +1.12.1 PEP 308: Conditional Expressions +--------------------------------------- + +For a long time, people have been requesting a way to write conditional +expressions, which are expressions that return value A or value B +depending on whether a Boolean value is true or false. A conditional +expression lets you write a single assignment statement that has the +same effect as the following: + + if condition: + x = true_value + else: + x = false_value + +There have been endless tedious discussions of syntax on both python-dev +and comp.lang.python. A vote was even held that found the majority of +voters wanted conditional expressions in some form, but there was no +syntax that was preferred by a clear majority. Candidates included C’s +‘cond ? true_v : false_v’, ‘if cond then true_v else false_v’, and 16 +other variations. + +Guido van Rossum eventually chose a surprising syntax: + + x = true_value if condition else false_value + +Evaluation is still lazy as in existing Boolean expressions, so the +order of evaluation jumps around a bit. The `condition' expression in +the middle is evaluated first, and the `true_value' expression is +evaluated only if the condition was true. Similarly, the `false_value' +expression is only evaluated when the condition is false. + +This syntax may seem strange and backwards; why does the condition go in +the `middle' of the expression, and not in the front as in C’s ‘c ? x : +y’? The decision was checked by applying the new syntax to the modules +in the standard library and seeing how the resulting code read. In many +cases where a conditional expression is used, one value seems to be the +‘common case’ and one value is an ‘exceptional case’, used only on rarer +occasions when the condition isn’t met. The conditional syntax makes +this pattern a bit more obvious: + + contents = ((doc + '\n') if doc else '') + +I read the above statement as meaning “here `contents' is usually +assigned a value of ‘doc+'\n'’; sometimes `doc' is empty, in which +special case an empty string is returned.” I doubt I will use +conditional expressions very often where there isn’t a clear common and +uncommon case. + +There was some discussion of whether the language should require +surrounding conditional expressions with parentheses. The decision was +made to `not' require parentheses in the Python language’s grammar, but +as a matter of style I think you should always use them. Consider these +two statements: + + # First version -- no parens + level = 1 if logging else 0 + + # Second version -- with parens + level = (1 if logging else 0) + +In the first version, I think a reader’s eye might group the statement +into ‘level = 1’, ‘if logging’, ‘else 0’, and think that the condition +decides whether the assignment to `level' is performed. The second +version reads better, in my opinion, because it makes it clear that the +assignment is always performed and the choice is being made between two +values. + +Another reason for including the brackets: a few odd combinations of +list comprehensions and lambdas could look like incorrect conditional +expressions. See PEP 308(1) for some examples. If you put parentheses +around your conditional expressions, you won’t run into this case. + +See also +........ + +PEP 308(2) - Conditional Expressions + + PEP written by Guido van Rossum and Raymond D. Hettinger; + implemented by Thomas Wouters. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0308 + + (2) https://www.python.org/dev/peps/pep-0308 + + +File: python.info, Node: PEP 309 Partial Function Application, Next: PEP 314 Metadata for Python Software Packages v1 1, Prev: PEP 308 Conditional Expressions, Up: What’s New in Python 2 5 + +1.12.2 PEP 309: Partial Function Application +-------------------------------------------- + +The *note functools: 85. module is intended to contain tools for +functional-style programming. + +One useful tool in this module is the ‘partial()’ function. For +programs written in a functional style, you’ll sometimes want to +construct variants of existing functions that have some of the +parameters filled in. Consider a Python function ‘f(a, b, c)’; you +could create a new function ‘g(b, c)’ that was equivalent to ‘f(1, b, +c)’. This is called “partial function application”. + +‘partial()’ takes the arguments ‘(function, arg1, arg2, ... +kwarg1=value1, kwarg2=value2)’. The resulting object is callable, so +you can just call it to invoke `function' with the filled-in arguments. + +Here’s a small but realistic example: + + import functools + + def log (message, subsystem): + "Write the contents of 'message' to the specified subsystem." + print '%s: %s' % (subsystem, message) + ... + + server_log = functools.partial(log, subsystem='server') + server_log('Unable to open socket') + +Here’s another example, from a program that uses PyGTK. Here a +context-sensitive pop-up menu is being constructed dynamically. The +callback provided for the menu option is a partially applied version of +the ‘open_item()’ method, where the first argument has been provided. + + ... + class Application: + def open_item(self, path): + ... + def init (self): + open_func = functools.partial(self.open_item, item_path) + popup_menu.append( ("Open", open_func, 1) ) + +Another function in the *note functools: 85. module is the +‘update_wrapper(wrapper, wrapped)’ function that helps you write +well-behaved decorators. ‘update_wrapper()’ copies the name, module, +and docstring attribute to a wrapper function so that tracebacks inside +the wrapped function are easier to understand. For example, you might +write: + + def my_decorator(f): + def wrapper(*args, **kwds): + print 'Calling decorated function' + return f(*args, **kwds) + functools.update_wrapper(wrapper, f) + return wrapper + +‘wraps()’ is a decorator that can be used inside your own decorators to +copy the wrapped function’s information. An alternate version of the +previous example would be: + + def my_decorator(f): + @functools.wraps(f) + def wrapper(*args, **kwds): + print 'Calling decorated function' + return f(*args, **kwds) + return wrapper + +See also +........ + +PEP 309(1) - Partial Function Application + + PEP proposed and written by Peter Harris; implemented by Hye-Shik + Chang and Nick Coghlan, with adaptations by Raymond Hettinger. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0309 + + +File: python.info, Node: PEP 314 Metadata for Python Software Packages v1 1, Next: PEP 328 Absolute and Relative Imports, Prev: PEP 309 Partial Function Application, Up: What’s New in Python 2 5 + +1.12.3 PEP 314: Metadata for Python Software Packages v1.1 +---------------------------------------------------------- + +Some simple dependency support was added to Distutils. The ‘setup()’ +function now has ‘requires’, ‘provides’, and ‘obsoletes’ keyword +parameters. When you build a source distribution using the ‘sdist’ +command, the dependency information will be recorded in the ‘PKG-INFO’ +file. + +Another new keyword parameter is ‘download_url’, which should be set to +a URL for the package’s source code. This means it’s now possible to +look up an entry in the package index, determine the dependencies for a +package, and download the required packages. + + VERSION = '1.0' + setup(name='PyPackage', + version=VERSION, + requires=['numarray', 'zlib (>=1.1.4)'], + obsoletes=['OldPackage'] + download_url=('http://www.example.com/pypackage/dist/pkg-%s.tar.gz' + % VERSION), + ) + +Another new enhancement to the Python package index at +‘https://pypi.org’ is storing source and binary archives for a package. +The new ‘upload’ Distutils command will upload a package to the +repository. + +Before a package can be uploaded, you must be able to build a +distribution using the ‘sdist’ Distutils command. Once that works, you +can run ‘python setup.py upload’ to add your package to the PyPI +archive. Optionally you can GPG-sign the package by supplying the +‘--sign’ and ‘--identity’ options. + +Package uploading was implemented by Martin von Löwis and Richard Jones. + +See also +........ + +PEP 314(1) - Metadata for Python Software Packages v1.1 + + PEP proposed and written by A.M. Kuchling, Richard Jones, and Fred + Drake; implemented by Richard Jones and Fred Drake. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0314 + + +File: python.info, Node: PEP 328 Absolute and Relative Imports, Next: PEP 338 Executing Modules as Scripts, Prev: PEP 314 Metadata for Python Software Packages v1 1, Up: What’s New in Python 2 5 + +1.12.4 PEP 328: Absolute and Relative Imports +--------------------------------------------- + +The simpler part of PEP 328(1) was implemented in Python 2.4: +parentheses could now be used to enclose the names imported from a +module using the ‘from ... import ...’ statement, making it easier to +import many different names. + +The more complicated part has been implemented in Python 2.5: importing +a module can be specified to use absolute or package-relative imports. +The plan is to move toward making absolute imports the default in future +versions of Python. + +Let’s say you have a package directory like this: + + pkg/ + pkg/__init__.py + pkg/main.py + pkg/string.py + +This defines a package named ‘pkg’ containing the ‘pkg.main’ and +‘pkg.string’ submodules. + +Consider the code in the ‘main.py’ module. What happens if it executes +the statement ‘import string’? In Python 2.4 and earlier, it will first +look in the package’s directory to perform a relative import, finds +‘pkg/string.py’, imports the contents of that file as the ‘pkg.string’ +module, and that module is bound to the name ‘string’ in the ‘pkg.main’ +module’s namespace. + +That’s fine if ‘pkg.string’ was what you wanted. But what if you wanted +Python’s standard *note string: f6. module? There’s no clean way to +ignore ‘pkg.string’ and look for the standard module; generally you had +to look at the contents of ‘sys.modules’, which is slightly unclean. +Holger Krekel’s ‘py.std’ package provides a tidier way to perform +imports from the standard library, ‘import py; py.std.string.join()’, +but that package isn’t available on all Python installations. + +Reading code which relies on relative imports is also less clear, +because a reader may be confused about which module, *note string: f6. +or ‘pkg.string’, is intended to be used. Python users soon learned not +to duplicate the names of standard library modules in the names of their +packages’ submodules, but you can’t protect against having your +submodule’s name being used for a new module added in a future version +of Python. + +In Python 2.5, you can switch *note import: c1d.’s behaviour to absolute +imports using a ‘from __future__ import absolute_import’ directive. +This absolute-import behaviour will become the default in a future +version (probably Python 2.7). Once absolute imports are the default, +‘import string’ will always find the standard library’s version. It’s +suggested that users should begin using absolute imports as much as +possible, so it’s preferable to begin writing ‘from pkg import string’ +in your code. + +Relative imports are still possible by adding a leading period to the +module name when using the ‘from ... import’ form: + + # Import names from pkg.string + from .string import name1, name2 + # Import pkg.string + from . import string + +This imports the *note string: f6. module relative to the current +package, so in ‘pkg.main’ this will import `name1' and `name2' from +‘pkg.string’. Additional leading periods perform the relative import +starting from the parent of the current package. For example, code in +the ‘A.B.C’ module can do: + + from . import D # Imports A.B.D + from .. import E # Imports A.E + from ..F import G # Imports A.F.G + +Leading periods cannot be used with the ‘import modname’ form of the +import statement, only the ‘from ... import’ form. + +See also +........ + +PEP 328(2) - Imports: Multi-Line and Absolute/Relative + + PEP written by Aahz; implemented by Thomas Wouters. + +‘https://pylib.readthedocs.io/’ + + The py library by Holger Krekel, which contains the ‘py.std’ + package. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0328 + + (2) https://www.python.org/dev/peps/pep-0328 + + +File: python.info, Node: PEP 338 Executing Modules as Scripts, Next: PEP 341 Unified try/except/finally, Prev: PEP 328 Absolute and Relative Imports, Up: What’s New in Python 2 5 + +1.12.5 PEP 338: Executing Modules as Scripts +-------------------------------------------- + +The *note -m: 337. switch added in Python 2.4 to execute a module as a +script gained a few more abilities. Instead of being implemented in C +code inside the Python interpreter, the switch now uses an +implementation in a new module, *note runpy: e2. + +The *note runpy: e2. module implements a more sophisticated import +mechanism so that it’s now possible to run modules in a package such as +‘pychecker.checker’. The module also supports alternative import +mechanisms such as the *note zipimport: 143. module. This means you can +add a .zip archive’s path to ‘sys.path’ and then use the *note -m: 337. +switch to execute code from the archive. + +See also +........ + +PEP 338(1) - Executing modules as scripts + + PEP written and implemented by Nick Coghlan. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0338 + + +File: python.info, Node: PEP 341 Unified try/except/finally, Next: PEP 342 New Generator Features, Prev: PEP 338 Executing Modules as Scripts, Up: What’s New in Python 2 5 + +1.12.6 PEP 341: Unified try/except/finally +------------------------------------------ + +Until Python 2.5, the *note try: d72. statement came in two flavours. +You could use a *note finally: 182. block to ensure that code is always +executed, or one or more *note except: b3e. blocks to catch specific +exceptions. You couldn’t combine both ‘except’ blocks and a ‘finally’ +block, because generating the right bytecode for the combined version +was complicated and it wasn’t clear what the semantics of the combined +statement should be. + +Guido van Rossum spent some time working with Java, which does support +the equivalent of combining *note except: b3e. blocks and a *note +finally: 182. block, and this clarified what the statement should mean. +In Python 2.5, you can now write: + + try: + block-1 ... + except Exception1: + handler-1 ... + except Exception2: + handler-2 ... + else: + else-block + finally: + final-block + +The code in `block-1' is executed. If the code raises an exception, the +various *note except: b3e. blocks are tested: if the exception is of +class ‘Exception1’, `handler-1' is executed; otherwise if it’s of class +‘Exception2’, `handler-2' is executed, and so forth. If no exception is +raised, the `else-block' is executed. + +No matter what happened previously, the `final-block' is executed once +the code block is complete and any raised exceptions handled. Even if +there’s an error in an exception handler or the `else-block' and a new +exception is raised, the code in the `final-block' is still run. + +See also +........ + +PEP 341(1) - Unifying try-except and try-finally + + PEP written by Georg Brandl; implementation by Thomas Lee. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0341 + + +File: python.info, Node: PEP 342 New Generator Features, Next: PEP 343 The ‘with’ statement<2>, Prev: PEP 341 Unified try/except/finally, Up: What’s New in Python 2 5 + +1.12.7 PEP 342: New Generator Features +-------------------------------------- + +Python 2.5 adds a simple way to pass values `into' a generator. As +introduced in Python 2.3, generators only produce output; once a +generator’s code was invoked to create an iterator, there was no way to +pass any new information into the function when its execution is +resumed. Sometimes the ability to pass in some information would be +useful. Hackish solutions to this include making the generator’s code +look at a global variable and then changing the global variable’s value, +or passing in some mutable object that callers then modify. + +To refresh your memory of basic generators, here’s a simple example: + + def counter (maximum): + i = 0 + while i < maximum: + yield i + i += 1 + +When you call ‘counter(10)’, the result is an iterator that returns the +values from 0 up to 9. On encountering the *note yield: 18f. statement, +the iterator returns the provided value and suspends the function’s +execution, preserving the local variables. Execution resumes on the +following call to the iterator’s *note next(): 682. method, picking up +after the ‘yield’ statement. + +In Python 2.3, *note yield: 18f. was a statement; it didn’t return any +value. In 2.5, ‘yield’ is now an expression, returning a value that can +be assigned to a variable or otherwise operated on: + + val = (yield i) + +I recommend that you always put parentheses around a *note yield: 18f. +expression when you’re doing something with the returned value, as in +the above example. The parentheses aren’t always necessary, but it’s +easier to always add them instead of having to remember when they’re +needed. + +( PEP 342(1) explains the exact rules, which are that a *note yield: +18f.-expression must always be parenthesized except when it occurs at +the top-level expression on the right-hand side of an assignment. This +means you can write ‘val = yield i’ but have to use parentheses when +there’s an operation, as in ‘val = (yield i) + 12’.) + +Values are sent into a generator by calling its ‘send(value)’ method. +The generator’s code is then resumed and the *note yield: 18f. +expression returns the specified `value'. If the regular *note next(): +682. method is called, the ‘yield’ returns *note None: 157. + +Here’s the previous example, modified to allow changing the value of the +internal counter. + + def counter (maximum): + i = 0 + while i < maximum: + val = (yield i) + # If value provided, change counter + if val is not None: + i = val + else: + i += 1 + +And here’s an example of changing the counter: + + >>> it = counter(10) + >>> print it.next() + 0 + >>> print it.next() + 1 + >>> print it.send(8) + 8 + >>> print it.next() + 9 + >>> print it.next() + Traceback (most recent call last): + File "t.py", line 15, in ? + print it.next() + StopIteration + +*note yield: 18f. will usually return *note None: 157, so you should +always check for this case. Don’t just use its value in expressions +unless you’re sure that the ‘send()’ method will be the only method used +to resume your generator function. + +In addition to ‘send()’, there are two other new methods on generators: + + * ‘throw(type, value=None, traceback=None)’ is used to raise an + exception inside the generator; the exception is raised by the + *note yield: 18f. expression where the generator’s execution is + paused. + + * ‘close()’ raises a new *note GeneratorExit: d32. exception inside + the generator to terminate the iteration. On receiving this + exception, the generator’s code must either raise *note + GeneratorExit: d32. or *note StopIteration: 486. Catching the + *note GeneratorExit: d32. exception and returning a value is + illegal and will trigger a *note RuntimeError: 2ba.; if the + function raises some other exception, that exception is propagated + to the caller. ‘close()’ will also be called by Python’s garbage + collector when the generator is garbage-collected. + + If you need to run cleanup code when a *note GeneratorExit: d32. + occurs, I suggest using a ‘try: ... finally:’ suite instead of + catching *note GeneratorExit: d32. + +The cumulative effect of these changes is to turn generators from +one-way producers of information into both producers and consumers. + +Generators also become `coroutines', a more generalized form of +subroutines. Subroutines are entered at one point and exited at another +point (the top of the function, and a *note return: 190. statement), but +coroutines can be entered, exited, and resumed at many different points +(the *note yield: 18f. statements). We’ll have to figure out patterns +for using coroutines effectively in Python. + +The addition of the ‘close()’ method has one side effect that isn’t +obvious. ‘close()’ is called when a generator is garbage-collected, so +this means the generator’s code gets one last chance to run before the +generator is destroyed. This last chance means that ‘try...finally’ +statements in generators can now be guaranteed to work; the *note +finally: 182. clause will now always get a chance to run. The syntactic +restriction that you couldn’t mix *note yield: 18f. statements with a +‘try...finally’ suite has therefore been removed. This seems like a +minor bit of language trivia, but using generators and ‘try...finally’ +is actually necessary in order to implement the *note with: 6e9. +statement described by PEP 343(2). I’ll look at this new statement in +the following section. + +Another even more esoteric effect of this change: previously, the +‘gi_frame’ attribute of a generator was always a frame object. It’s now +possible for ‘gi_frame’ to be ‘None’ once the generator has been +exhausted. + +See also +........ + +PEP 342(3) - Coroutines via Enhanced Generators + + PEP written by Guido van Rossum and Phillip J. Eby; implemented by + Phillip J. Eby. Includes examples of some fancier uses of + generators as coroutines. + + Earlier versions of these features were proposed in PEP 288(4) by + Raymond Hettinger and PEP 325(5) by Samuele Pedroni. + +‘https://en.wikipedia.org/wiki/Coroutine’ + + The Wikipedia entry for coroutines. + +‘http://www.sidhe.org/~dan/blog/archives/000178.html’ + + An explanation of coroutines from a Perl point of view, written by + Dan Sugalski. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0342 + + (2) https://www.python.org/dev/peps/pep-0343 + + (3) https://www.python.org/dev/peps/pep-0342 + + (4) https://www.python.org/dev/peps/pep-0288 + + (5) https://www.python.org/dev/peps/pep-0325 + + +File: python.info, Node: PEP 343 The ‘with’ statement<2>, Next: PEP 352 Exceptions as New-Style Classes, Prev: PEP 342 New Generator Features, Up: What’s New in Python 2 5 + +1.12.8 PEP 343: The ‘with’ statement +------------------------------------ + +The ‘*note with: 6e9.’ statement clarifies code that previously would +use ‘try...finally’ blocks to ensure that clean-up code is executed. In +this section, I’ll discuss the statement as it will commonly be used. +In the next section, I’ll examine the implementation details and show +how to write objects for use with this statement. + +The ‘*note with: 6e9.’ statement is a new control-flow structure whose +basic structure is: + + with expression [as variable]: + with-block + +The expression is evaluated, and it should result in an object that +supports the context management protocol (that is, has *note +__enter__(): c95. and *note __exit__(): c96. methods. + +The object’s *note __enter__(): c95. is called before `with-block' is +executed and therefore can run set-up code. It also may return a value +that is bound to the name `variable', if given. (Note carefully that +`variable' is `not' assigned the result of `expression'.) + +After execution of the `with-block' is finished, the object’s *note +__exit__(): c96. method is called, even if the block raised an +exception, and can therefore run clean-up code. + +To enable the statement in Python 2.5, you need to add the following +directive to your module: + + from __future__ import with_statement + +The statement will always be enabled in Python 2.6. + +Some standard Python objects now support the context management protocol +and can be used with the ‘*note with: 6e9.’ statement. File objects are +one example: + + with open('/etc/passwd', 'r') as f: + for line in f: + print line + ... more processing code ... + +After this statement has executed, the file object in `f' will have been +automatically closed, even if the *note for: c30. loop raised an +exception part-way through the block. + + Note: In this case, `f' is the same object created by *note open(): + 4f0, because ‘file.__enter__()’ returns `self'. + +The *note threading: 109. module’s locks and condition variables also +support the ‘*note with: 6e9.’ statement: + + lock = threading.Lock() + with lock: + # Critical section of code + ... + +The lock is acquired before the block is executed and always released +once the block is complete. + +The new ‘localcontext()’ function in the *note decimal: 36. module makes +it easy to save and restore the current decimal context, which +encapsulates the desired precision and rounding characteristics for +computations: + + from decimal import Decimal, Context, localcontext + + # Displays with default precision of 28 digits + v = Decimal('578') + print v.sqrt() + + with localcontext(Context(prec=16)): + # All code in this block uses a precision of 16 digits. + # The original context is restored on exiting the block. + print v.sqrt() + +* Menu: + +* Writing Context Managers: Writing Context Managers<2>. +* The contextlib module: The contextlib module<2>. + + +File: python.info, Node: Writing Context Managers<2>, Next: The contextlib module<2>, Up: PEP 343 The ‘with’ statement<2> + +1.12.8.1 Writing Context Managers +................................. + +Under the hood, the ‘*note with: 6e9.’ statement is fairly complicated. +Most people will only use ‘‘with’’ in company with existing objects and +don’t need to know these details, so you can skip the rest of this +section if you like. Authors of new objects will need to understand the +details of the underlying implementation and should keep reading. + +A high-level explanation of the context management protocol is: + + * The expression is evaluated and should result in an object called a + “context manager”. The context manager must have *note + __enter__(): c95. and *note __exit__(): c96. methods. + + * The context manager’s *note __enter__(): c95. method is called. + The value returned is assigned to `VAR'. If no ‘'as VAR'’ clause is + present, the value is simply discarded. + + * The code in `BLOCK' is executed. + + * If `BLOCK' raises an exception, the ‘__exit__(type, value, + traceback)’ is called with the exception details, the same values + returned by *note sys.exc_info(): c5f. The method’s return value + controls whether the exception is re-raised: any false value + re-raises the exception, and ‘True’ will result in suppressing it. + You’ll only rarely want to suppress the exception, because if you + do the author of the code containing the ‘*note with: 6e9.’ + statement will never realize anything went wrong. + + * If `BLOCK' didn’t raise an exception, the *note __exit__(): c96. + method is still called, but `type', `value', and `traceback' are + all ‘None’. + +Let’s think through an example. I won’t present detailed code but will +only sketch the methods necessary for a database that supports +transactions. + +(For people unfamiliar with database terminology: a set of changes to +the database are grouped into a transaction. Transactions can be either +committed, meaning that all the changes are written into the database, +or rolled back, meaning that the changes are all discarded and the +database is unchanged. See any database textbook for more information.) + +Let’s assume there’s an object representing a database connection. Our +goal will be to let the user write code like this: + + db_connection = DatabaseConnection() + with db_connection as cursor: + cursor.execute('insert into ...') + cursor.execute('delete from ...') + # ... more operations ... + +The transaction should be committed if the code in the block runs +flawlessly or rolled back if there’s an exception. Here’s the basic +interface for ‘DatabaseConnection’ that I’ll assume: + + class DatabaseConnection: + # Database interface + def cursor (self): + "Returns a cursor object and starts a new transaction" + def commit (self): + "Commits current transaction" + def rollback (self): + "Rolls back current transaction" + +The *note __enter__(): c95. method is pretty easy, having only to start +a new transaction. For this application the resulting cursor object +would be a useful result, so the method will return it. The user can +then add ‘as cursor’ to their ‘*note with: 6e9.’ statement to bind the +cursor to a variable name. + + class DatabaseConnection: + ... + def __enter__ (self): + # Code to start a new transaction + cursor = self.cursor() + return cursor + +The *note __exit__(): c96. method is the most complicated because it’s +where most of the work has to be done. The method has to check if an +exception occurred. If there was no exception, the transaction is +committed. The transaction is rolled back if there was an exception. + +In the code below, execution will just fall off the end of the function, +returning the default value of ‘None’. ‘None’ is false, so the +exception will be re-raised automatically. If you wished, you could be +more explicit and add a *note return: 190. statement at the marked +location. + + class DatabaseConnection: + ... + def __exit__ (self, type, value, tb): + if tb is None: + # No exception, so commit + self.commit() + else: + # Exception occurred, so rollback. + self.rollback() + # return False + + +File: python.info, Node: The contextlib module<2>, Prev: Writing Context Managers<2>, Up: PEP 343 The ‘with’ statement<2> + +1.12.8.2 The contextlib module +.............................. + +The new *note contextlib: 24. module provides some functions and a +decorator that are useful for writing objects for use with the ‘*note +with: 6e9.’ statement. + +The decorator is called ‘contextmanager()’, and lets you write a single +generator function instead of defining a new class. The generator +should yield exactly one value. The code up to the *note yield: 18f. +will be executed as the *note __enter__(): c95. method, and the value +yielded will be the method’s return value that will get bound to the +variable in the ‘*note with: 6e9.’ statement’s ‘as’ clause, if any. The +code after the *note yield: 18f. will be executed in the *note +__exit__(): c96. method. Any exception raised in the block will be +raised by the ‘yield’ statement. + +Our database example from the previous section could be written using +this decorator as: + + from contextlib import contextmanager + + @contextmanager + def db_transaction (connection): + cursor = connection.cursor() + try: + yield cursor + except: + connection.rollback() + raise + else: + connection.commit() + + db = DatabaseConnection() + with db_transaction(db) as cursor: + ... + +The *note contextlib: 24. module also has a ‘nested(mgr1, mgr2, ...)’ +function that combines a number of context managers so you don’t need to +write nested ‘*note with: 6e9.’ statements. In this example, the single +‘‘with’’ statement both starts a database transaction and acquires a +thread lock: + + lock = threading.Lock() + with nested (db_transaction(db), lock) as (cursor, locked): + ... + +Finally, the ‘closing(object)’ function returns `object' so that it can +be bound to a variable, and calls ‘object.close’ at the end of the +block. + + import urllib, sys + from contextlib import closing + + with closing(urllib.urlopen('http://www.yahoo.com')) as f: + for line in f: + sys.stdout.write(line) + +See also +........ + +PEP 343(1) - The “with” statement + + PEP written by Guido van Rossum and Nick Coghlan; implemented by + Mike Bland, Guido van Rossum, and Neal Norwitz. The PEP shows the + code generated for a ‘*note with: 6e9.’ statement, which can be + helpful in learning how the statement works. + +The documentation for the *note contextlib: 24. module. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0343 + + +File: python.info, Node: PEP 352 Exceptions as New-Style Classes, Next: PEP 353 Using ssize_t as the index type, Prev: PEP 343 The ‘with’ statement<2>, Up: What’s New in Python 2 5 + +1.12.9 PEP 352: Exceptions as New-Style Classes +----------------------------------------------- + +Exception classes can now be new-style classes, not just classic +classes, and the built-in *note Exception: 1a9. class and all the +standard built-in exceptions (*note NameError: d7b, *note ValueError: +1fb, etc.) are now new-style classes. + +The inheritance hierarchy for exceptions has been rearranged a bit. In +2.5, the inheritance relationships are: + + BaseException # New in Python 2.5 + |- KeyboardInterrupt + |- SystemExit + |- Exception + |- (all other current built-in exceptions) + +This rearrangement was done because people often want to catch all +exceptions that indicate program errors. *note KeyboardInterrupt: 197. +and *note SystemExit: 5fc. aren’t errors, though, and usually represent +an explicit action such as the user hitting ‘Control-C’ or code calling +*note sys.exit(): ce2. A bare ‘except:’ will catch all exceptions, so +you commonly need to list *note KeyboardInterrupt: 197. and *note +SystemExit: 5fc. in order to re-raise them. The usual pattern is: + + try: + ... + except (KeyboardInterrupt, SystemExit): + raise + except: + # Log error... + # Continue running program... + +In Python 2.5, you can now write ‘except Exception’ to achieve the same +result, catching all the exceptions that usually indicate errors but +leaving *note KeyboardInterrupt: 197. and *note SystemExit: 5fc. alone. +As in previous versions, a bare ‘except:’ still catches all exceptions. + +The goal for Python 3.0 is to require any class raised as an exception +to derive from *note BaseException: 1a8. or some descendant of *note +BaseException: 1a8, and future releases in the Python 2.x series may +begin to enforce this constraint. Therefore, I suggest you begin making +all your exception classes derive from *note Exception: 1a9. now. It’s +been suggested that the bare ‘except:’ form should be removed in Python +3.0, but Guido van Rossum hasn’t decided whether to do this or not. + +Raising of strings as exceptions, as in the statement ‘raise "Error +occurred"’, is deprecated in Python 2.5 and will trigger a warning. The +aim is to be able to remove the string-exception feature in a few +releases. + +See also +........ + +PEP 352(1) - Required Superclass for Exceptions + + PEP written by Brett Cannon and Guido van Rossum; implemented by + Brett Cannon. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0352 + + +File: python.info, Node: PEP 353 Using ssize_t as the index type, Next: PEP 357 The ‘__index__’ method, Prev: PEP 352 Exceptions as New-Style Classes, Up: What’s New in Python 2 5 + +1.12.10 PEP 353: Using ssize_t as the index type +------------------------------------------------ + +A wide-ranging change to Python’s C API, using a new ‘Py_ssize_t’ type +definition instead of ‘int’, will permit the interpreter to handle more +data on 64-bit platforms. This change doesn’t affect Python’s capacity +on 32-bit platforms. + +Various pieces of the Python interpreter used C’s ‘int’ type to store +sizes or counts; for example, the number of items in a list or tuple +were stored in an ‘int’. The C compilers for most 64-bit platforms +still define ‘int’ as a 32-bit type, so that meant that lists could only +hold up to ‘2**31 - 1’ = 2147483647 items. (There are actually a few +different programming models that 64-bit C compilers can use – see +‘http://www.unix.org/version2/whatsnew/lp64_wp.html’ for a discussion – +but the most commonly available model leaves ‘int’ as 32 bits.) + +A limit of 2147483647 items doesn’t really matter on a 32-bit platform +because you’ll run out of memory before hitting the length limit. Each +list item requires space for a pointer, which is 4 bytes, plus space for +a *note PyObject: 4ba. representing the item. 2147483647*4 is already +more bytes than a 32-bit address space can contain. + +It’s possible to address that much memory on a 64-bit platform, however. +The pointers for a list that size would only require 16 GiB of space, so +it’s not unreasonable that Python programmers might construct lists that +large. Therefore, the Python interpreter had to be changed to use some +type other than ‘int’, and this will be a 64-bit type on 64-bit +platforms. The change will cause incompatibilities on 64-bit machines, +so it was deemed worth making the transition now, while the number of +64-bit users is still relatively small. (In 5 or 10 years, we may `all' +be on 64-bit machines, and the transition would be more painful then.) + +This change most strongly affects authors of C extension modules. +Python strings and container types such as lists and tuples now use +‘Py_ssize_t’ to store their size. Functions such as *note +PyList_Size(): d7e. now return ‘Py_ssize_t’. Code in extension modules +may therefore need to have some variables changed to ‘Py_ssize_t’. + +The *note PyArg_ParseTuple(): 26a. and *note Py_BuildValue(): 2ce. +functions have a new conversion code, ‘n’, for ‘Py_ssize_t’. *note +PyArg_ParseTuple(): 26a.’s ‘s#’ and ‘t#’ still output ‘int’ by default, +but you can define the macro ‘PY_SSIZE_T_CLEAN’ before including +‘Python.h’ to make them return ‘Py_ssize_t’. + +PEP 353(1) has a section on conversion guidelines that extension authors +should read to learn about supporting 64-bit platforms. + +See also +........ + +PEP 353(2) - Using ssize_t as the index type + + PEP written and implemented by Martin von Löwis. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0353 + + (2) https://www.python.org/dev/peps/pep-0353 + + +File: python.info, Node: PEP 357 The ‘__index__’ method, Next: Other Language Changes<11>, Prev: PEP 353 Using ssize_t as the index type, Up: What’s New in Python 2 5 + +1.12.11 PEP 357: The ‘__index__’ method +--------------------------------------- + +The NumPy developers had a problem that could only be solved by adding a +new special method, *note __index__(): 18a. When using slice notation, +as in ‘[start:stop:step]’, the values of the `start', `stop', and `step' +indexes must all be either integers or long integers. NumPy defines a +variety of specialized integer types corresponding to unsigned and +signed integers of 8, 16, 32, and 64 bits, but there was no way to +signal that these types could be used as slice indexes. + +Slicing can’t just use the existing *note __int__(): 18b. method because +that method is also used to implement coercion to integers. If slicing +used *note __int__(): 18b, floating-point numbers would also become +legal slice indexes and that’s clearly an undesirable behaviour. + +Instead, a new special method called *note __index__(): 18a. was added. +It takes no arguments and returns an integer giving the slice index to +use. For example: + + class C: + def __index__ (self): + return self.value + +The return value must be either a Python integer or long integer. The +interpreter will check that the type returned is correct, and raises a +*note TypeError: 192. if this requirement isn’t met. + +A corresponding ‘nb_index’ slot was added to the C-level *note +PyNumberMethods: d81. structure to let C extensions implement this +protocol. ‘PyNumber_Index(obj)’ can be used in extension code to call +the *note __index__(): 18a. function and retrieve its result. + +See also +........ + +PEP 357(1) - Allowing Any Object to be Used for Slicing + + PEP written and implemented by Travis Oliphant. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0357 + + +File: python.info, Node: Other Language Changes<11>, Next: New Improved and Removed Modules, Prev: PEP 357 The ‘__index__’ method, Up: What’s New in Python 2 5 + +1.12.12 Other Language Changes +------------------------------ + +Here are all of the changes that Python 2.5 makes to the core Python +language. + + * The *note dict: 1b8. type has a new hook for letting subclasses + provide a default value when a key isn’t contained in the + dictionary. When a key isn’t found, the dictionary’s + ‘__missing__(key)’ method will be called. This hook is used to + implement the new ‘defaultdict’ class in the *note collections: 1e. + module. The following example defines a dictionary that returns + zero for any missing key: + + class zerodict (dict): + def __missing__ (self, key): + return 0 + + d = zerodict({1:1, 2:2}) + print d[1], d[2] # Prints 1, 2 + print d[3], d[4] # Prints 0, 0 + + * Both 8-bit and Unicode strings have new ‘partition(sep)’ and + ‘rpartition(sep)’ methods that simplify a common use case. + + The ‘find(S)’ method is often used to get an index which is then + used to slice the string and obtain the pieces that are before and + after the separator. ‘partition(sep)’ condenses this pattern into + a single method call that returns a 3-tuple containing the + substring before the separator, the separator itself, and the + substring after the separator. If the separator isn’t found, the + first element of the tuple is the entire string and the other two + elements are empty. ‘rpartition(sep)’ also returns a 3-tuple but + starts searching from the end of the string; the ‘r’ stands for + ‘reverse’. + + Some examples: + + >>> ('http://www.python.org').partition('://') + ('http', '://', 'www.python.org') + >>> ('file:/usr/share/doc/index.html').partition('://') + ('file:/usr/share/doc/index.html', '', '') + >>> (u'Subject: a quick question').partition(':') + (u'Subject', u':', u' a quick question') + >>> 'www.python.org'.rpartition('.') + ('www.python', '.', 'org') + >>> 'www.python.org'.rpartition(':') + ('', '', 'www.python.org') + + (Implemented by Fredrik Lundh following a suggestion by Raymond + Hettinger.) + + * The ‘startswith()’ and ‘endswith()’ methods of string types now + accept tuples of strings to check for. + + def is_image_file (filename): + return filename.endswith(('.gif', '.jpg', '.tiff')) + + (Implemented by Georg Brandl following a suggestion by Tom Lynn.) + + * The *note min(): 809. and *note max(): 80a. built-in functions + gained a ‘key’ keyword parameter analogous to the ‘key’ argument + for ‘sort()’. This parameter supplies a function that takes a + single argument and is called for every value in the list; *note + min(): 809./*note max(): 80a. will return the element with the + smallest/largest return value from this function. For example, to + find the longest string in a list, you can do: + + L = ['medium', 'longest', 'short'] + # Prints 'longest' + print max(L, key=len) + # Prints 'short', because lexicographically 'short' has the largest value + print max(L) + + (Contributed by Steven Bethard and Raymond Hettinger.) + + * Two new built-in functions, *note any(): d84. and *note all(): d85, + evaluate whether an iterator contains any true or false values. + *note any(): d84. returns *note True: 499. if any value returned by + the iterator is true; otherwise it will return *note False: 388. + *note all(): d85. returns *note True: 499. only if all of the + values returned by the iterator evaluate as true. (Suggested by + Guido van Rossum, and implemented by Raymond Hettinger.) + + * The result of a class’s *note __hash__(): 340. method can now be + either a long integer or a regular integer. If a long integer is + returned, the hash of that value is taken. In earlier versions the + hash value was required to be a regular integer, but in 2.5 the + *note id(): d86. built-in was changed to always return non-negative + numbers, and users often seem to use ‘id(self)’ in *note + __hash__(): 340. methods (though this is discouraged). + + * ASCII is now the default encoding for modules. It’s now a syntax + error if a module contains string literals with 8-bit characters + but doesn’t have an encoding declaration. In Python 2.4 this + triggered a warning, not a syntax error. See PEP 263(1) for how to + declare a module’s encoding; for example, you might add a line like + this near the top of the source file: + + # -*- coding: latin1 -*- + + * A new warning, *note UnicodeWarning: d87, is triggered when you + attempt to compare a Unicode string and an 8-bit string that can’t + be converted to Unicode using the default ASCII encoding. The + result of the comparison is false: + + >>> chr(128) == unichr(128) # Can't convert chr(128) to Unicode + __main__:1: UnicodeWarning: Unicode equal comparison failed + to convert both arguments to Unicode - interpreting them + as being unequal + False + >>> chr(127) == unichr(127) # chr(127) can be converted + True + + Previously this would raise a *note UnicodeDecodeError: 1fd. + exception, but in 2.5 this could result in puzzling problems when + accessing a dictionary. If you looked up ‘unichr(128)’ and + ‘chr(128)’ was being used as a key, you’d get a *note + UnicodeDecodeError: 1fd. exception. Other changes in 2.5 resulted + in this exception being raised instead of suppressed by the code in + ‘dictobject.c’ that implements dictionaries. + + Raising an exception for such a comparison is strictly correct, but + the change might have broken code, so instead *note UnicodeWarning: + d87. was introduced. + + (Implemented by Marc-André Lemburg.) + + * One error that Python programmers sometimes make is forgetting to + include an ‘__init__.py’ module in a package directory. Debugging + this mistake can be confusing, and usually requires running Python + with the *note -v: d88. switch to log all the paths searched. In + Python 2.5, a new *note ImportWarning: 5d8. warning is triggered + when an import would have picked up a directory as a package but no + ‘__init__.py’ was found. This warning is silently ignored by + default; provide the *note -Wd: 417. option when running the Python + executable to display the warning message. (Implemented by Thomas + Wouters.) + + * The list of base classes in a class definition can now be empty. + As an example, this is now legal: + + class C(): + pass + + (Implemented by Brett Cannon.) + +* Menu: + +* Interactive Interpreter Changes:: +* Optimizations: Optimizations<10>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0263 + + +File: python.info, Node: Interactive Interpreter Changes, Next: Optimizations<10>, Up: Other Language Changes<11> + +1.12.12.1 Interactive Interpreter Changes +......................................... + +In the interactive interpreter, ‘quit’ and ‘exit’ have long been strings +so that new users get a somewhat helpful message when they try to quit: + + >>> quit + 'Use Ctrl-D (i.e. EOF) to exit.' + +In Python 2.5, ‘quit’ and ‘exit’ are now objects that still produce +string representations of themselves, but are also callable. Newbies +who try ‘quit()’ or ‘exit()’ will now exit the interpreter as they +expect. (Implemented by Georg Brandl.) + +The Python executable now accepts the standard long options *note –help: +d8b. and *note –version: 5d1.; on Windows, it also accepts the *note /?: +d8c. option for displaying a help message. (Implemented by Georg +Brandl.) + + +File: python.info, Node: Optimizations<10>, Prev: Interactive Interpreter Changes, Up: Other Language Changes<11> + +1.12.12.2 Optimizations +....................... + +Several of the optimizations were developed at the NeedForSpeed sprint, +an event held in Reykjavik, Iceland, from May 21–28 2006. The sprint +focused on speed enhancements to the CPython implementation and was +funded by EWT LLC with local support from CCP Games. Those +optimizations added at this sprint are specially marked in the following +list. + + * When they were introduced in Python 2.4, the built-in *note set: + b6f. and *note frozenset: bee. types were built on top of Python’s + dictionary type. In 2.5 the internal data structure has been + customized for implementing sets, and as a result sets will use a + third less memory and are somewhat faster. (Implemented by Raymond + Hettinger.) + + * The speed of some Unicode operations, such as finding substrings, + string splitting, and character map encoding and decoding, has been + improved. (Substring search and splitting improvements were added + by Fredrik Lundh and Andrew Dalke at the NeedForSpeed sprint. + Character maps were improved by Walter Dörwald and Martin von + Löwis.) + + * The ‘long(str, base)’ function is now faster on long digit strings + because fewer intermediate results are calculated. The peak is for + strings of around 800–1000 digits where the function is 6 times + faster. (Contributed by Alan McIntyre and committed at the + NeedForSpeed sprint.) + + * It’s now illegal to mix iterating over a file with ‘for line in + file’ and calling the file object’s ‘read()’/*note readline(): + de./‘readlines()’ methods. Iteration uses an internal buffer and + the ‘read*()’ methods don’t use that buffer. Instead they would + return the data following the buffer, causing the data to appear + out of order. Mixing iteration and these methods will now trigger + a *note ValueError: 1fb. from the ‘read*()’ method. (Implemented + by Thomas Wouters.) + + * The *note struct: f8. module now compiles structure format strings + into an internal representation and caches this representation, + yielding a 20% speedup. (Contributed by Bob Ippolito at the + NeedForSpeed sprint.) + + * The *note re: dd. module got a 1 or 2% speedup by switching to + Python’s allocator functions instead of the system’s ‘malloc()’ and + ‘free()’. (Contributed by Jack Diederich at the NeedForSpeed + sprint.) + + * The code generator’s peephole optimizer now performs simple + constant folding in expressions. If you write something like ‘a = + 2+3’, the code generator will do the arithmetic and produce code + corresponding to ‘a = 5’. (Proposed and implemented by Raymond + Hettinger.) + + * Function calls are now faster because code objects now keep the + most recently finished frame (a “zombie frame”) in an internal + field of the code object, reusing it the next time the code object + is invoked. (Original patch by Michael Hudson, modified by Armin + Rigo and Richard Jones; committed at the NeedForSpeed sprint.) + Frame objects are also slightly smaller, which may improve cache + locality and reduce memory usage a bit. (Contributed by Neal + Norwitz.) + + * Python’s built-in exceptions are now new-style classes, a change + that speeds up instantiation considerably. Exception handling in + Python 2.5 is therefore about 30% faster than in 2.4. (Contributed + by Richard Jones, Georg Brandl and Sean Reifschneider at the + NeedForSpeed sprint.) + + * Importing now caches the paths tried, recording whether they exist + or not so that the interpreter makes fewer ‘open()’ and ‘stat()’ + calls on startup. (Contributed by Martin von Löwis and Georg + Brandl.) + + +File: python.info, Node: New Improved and Removed Modules, Next: Build and C API Changes<10>, Prev: Other Language Changes<11>, Up: What’s New in Python 2 5 + +1.12.13 New, Improved, and Removed Modules +------------------------------------------ + +The standard library received many enhancements and bug fixes in Python +2.5. Here’s a partial list of the most notable changes, sorted +alphabetically by module name. Consult the ‘Misc/NEWS’ file in the +source tree for a more complete list of changes, or look through the SVN +logs for all the details. + + * The *note audioop: d. module now supports the a-LAW encoding, and + the code for u-LAW encoding has been improved. (Contributed by + Lars Immisch.) + + * The *note codecs: 1c. module gained support for incremental codecs. + The ‘codec.lookup()’ function now returns a ‘CodecInfo’ instance + instead of a tuple. ‘CodecInfo’ instances behave like a 4-tuple to + preserve backward compatibility but also have the attributes + ‘encode’, ‘decode’, ‘incrementalencoder’, ‘incrementaldecoder’, + ‘streamwriter’, and ‘streamreader’. Incremental codecs can receive + input and produce output in multiple chunks; the output is the same + as if the entire input was fed to the non-incremental codec. See + the *note codecs: 1c. module documentation for details. (Designed + and implemented by Walter Dörwald.) + + * The *note collections: 1e. module gained a new type, ‘defaultdict’, + that subclasses the standard *note dict: 1b8. type. The new type + mostly behaves like a dictionary but constructs a default value + when a key isn’t present, automatically adding it to the dictionary + for the requested key value. + + The first argument to ‘defaultdict’’s constructor is a factory + function that gets called whenever a key is requested but not + found. This factory function receives no arguments, so you can use + built-in type constructors such as *note list(): 262. or *note + int(): 184. For example, you can make an index of words based on + their initial letter like this: + + words = """Nel mezzo del cammin di nostra vita + mi ritrovai per una selva oscura + che la diritta via era smarrita""".lower().split() + + index = defaultdict(list) + + for w in words: + init_letter = w[0] + index[init_letter].append(w) + + Printing ‘index’ results in the following output: + + defaultdict(, {'c': ['cammin', 'che'], 'e': ['era'], + 'd': ['del', 'di', 'diritta'], 'm': ['mezzo', 'mi'], + 'l': ['la'], 'o': ['oscura'], 'n': ['nel', 'nostra'], + 'p': ['per'], 's': ['selva', 'smarrita'], + 'r': ['ritrovai'], 'u': ['una'], 'v': ['vita', 'via']} + + (Contributed by Guido van Rossum.) + + * The ‘deque’ double-ended queue type supplied by the *note + collections: 1e. module now has a ‘remove(value)’ method that + removes the first occurrence of `value' in the queue, raising *note + ValueError: 1fb. if the value isn’t found. (Contributed by Raymond + Hettinger.) + + * New module: The *note contextlib: 24. module contains helper + functions for use with the new ‘*note with: 6e9.’ statement. See + section *note The contextlib module: d77. for more about this + module. + + * New module: The *note cProfile: 28. module is a C implementation of + the existing *note profile: d3. module that has much lower + overhead. The module’s interface is the same as *note profile: + d3.: you run ‘cProfile.run('main()')’ to profile a function, can + save profile data to a file, etc. It’s not yet known if the + Hotshot profiler, which is also written in C but doesn’t match the + *note profile: d3. module’s interface, will continue to be + maintained in future versions of Python. (Contributed by Armin + Rigo.) + + Also, the *note pstats: d4. module for analyzing the data measured + by the profiler now supports directing the output to any file + object by supplying a `stream' argument to the ‘Stats’ constructor. + (Contributed by Skip Montanaro.) + + * The *note csv: 2a. module, which parses files in comma-separated + value format, received several enhancements and a number of + bugfixes. You can now set the maximum size in bytes of a field by + calling the ‘csv.field_size_limit(new_limit)’ function; omitting + the `new_limit' argument will return the currently-set limit. The + ‘reader’ class now has a ‘line_num’ attribute that counts the + number of physical lines read from the source; records can span + multiple physical lines, so ‘line_num’ is not the same as the + number of records read. + + The CSV parser is now stricter about multi-line quoted fields. + Previously, if a line ended within a quoted field without a + terminating newline character, a newline would be inserted into the + returned field. This behavior caused problems when reading files + that contained carriage return characters within fields, so the + code was changed to return the field without inserting newlines. + As a consequence, if newlines embedded within fields are important, + the input should be split into lines in a manner that preserves the + newline characters. + + (Contributed by Skip Montanaro and Andrew McNamara.) + + * The *note datetime: 194. class in the *note datetime: 31. module + now has a ‘strptime(string, format)’ method for parsing date + strings, contributed by Josh Spoerri. It uses the same format + characters as *note time.strptime(): d91. and *note + time.strftime(): b65.: + + from datetime import datetime + + ts = datetime.strptime('10:13:15 2006-03-07', + '%H:%M:%S %Y-%m-%d') + + * The ‘SequenceMatcher.get_matching_blocks()’ method in the *note + difflib: 37. module now guarantees to return a minimal list of + blocks describing matching subsequences. Previously, the algorithm + would occasionally break a block of matching elements into two list + entries. (Enhancement by Tim Peters.) + + * The *note doctest: 67. module gained a ‘SKIP’ option that keeps an + example from being executed at all. This is intended for code + snippets that are usage examples intended for the reader and aren’t + actually test cases. + + An `encoding' parameter was added to the ‘testfile()’ function and + the ‘DocFileSuite’ class to specify the file’s encoding. This + makes it easier to use non-ASCII characters in tests contained + within a docstring. (Contributed by Bjorn Tillenius.) + + * The *note email: 69. package has been updated to version 4.0. + (Contributed by Barry Warsaw.) + + * The *note fileinput: 80. module was made more flexible. Unicode + filenames are now supported, and a `mode' parameter that defaults + to ‘"r"’ was added to the *note input(): c6b. function to allow + opening files in binary or *note universal newlines: 5f4. mode. + Another new parameter, `openhook', lets you use a function other + than *note open(): 4f0. to open the input files. Once you’re + iterating over the set of files, the ‘FileInput’ object’s new + ‘fileno()’ returns the file descriptor for the currently opened + file. (Contributed by Georg Brandl.) + + * In the *note gc: 86. module, the new ‘get_count()’ function returns + a 3-tuple containing the current collection counts for the three GC + generations. This is accounting information for the garbage + collector; when these counts reach a specified threshold, a garbage + collection sweep will be made. The existing *note gc.collect(): + 22e. function now takes an optional `generation' argument of 0, 1, + or 2 to specify which generation to collect. (Contributed by Barry + Warsaw.) + + * The ‘nsmallest()’ and ‘nlargest()’ functions in the *note heapq: + 8e. module now support a ‘key’ keyword parameter similar to the one + provided by the *note min(): 809./*note max(): 80a. functions and + the ‘sort()’ methods. For example: + + >>> import heapq + >>> L = ["short", 'medium', 'longest', 'longer still'] + >>> heapq.nsmallest(2, L) # Return two lowest elements, lexicographically + ['longer still', 'longest'] + >>> heapq.nsmallest(2, L, key=len) # Return two shortest elements + ['short', 'medium'] + + (Contributed by Raymond Hettinger.) + + * The *note itertools.islice(): 3a2. function now accepts ‘None’ for + the start and step arguments. This makes it more compatible with + the attributes of slice objects, so that you can now write the + following: + + s = slice(5) # Create slice object + itertools.islice(iterable, s.start, s.stop, s.step) + + (Contributed by Raymond Hettinger.) + + * The *note format(): 4db. function in the *note locale: a9. module + has been modified and two new functions were added, + ‘format_string()’ and ‘currency()’. + + The *note format(): 4db. function’s `val' parameter could + previously be a string as long as no more than one %char specifier + appeared; now the parameter must be exactly one %char specifier + with no surrounding text. An optional `monetary' parameter was + also added which, if ‘True’, will use the locale’s rules for + formatting currency in placing a separator between groups of three + digits. + + To format strings with multiple %char specifiers, use the new + ‘format_string()’ function that works like *note format(): 4db. but + also supports mixing %char specifiers with arbitrary text. + + A new ‘currency()’ function was also added that formats a number + according to the current locale’s settings. + + (Contributed by Georg Brandl.) + + * The *note mailbox: ae. module underwent a massive rewrite to add + the capability to modify mailboxes in addition to reading them. A + new set of classes that include ‘mbox’, ‘MH’, and ‘Maildir’ are + used to read mailboxes, and have an ‘add(message)’ method to add + messages, ‘remove(key)’ to remove messages, and ‘lock()’/‘unlock()’ + to lock/unlock the mailbox. The following example converts a + maildir-format mailbox into an mbox-format one: + + import mailbox + + # 'factory=None' uses email.Message.Message as the class representing + # individual messages. + src = mailbox.Maildir('maildir', factory=None) + dest = mailbox.mbox('/tmp/mbox') + + for msg in src: + dest.add(msg) + + (Contributed by Gregory K. Johnson. Funding was provided by + Google’s 2005 Summer of Code.) + + * New module: the *note msilib: b5. module allows creating Microsoft + Installer ‘.msi’ files and CAB files. Some support for reading the + ‘.msi’ database is also included. (Contributed by Martin von + Löwis.) + + * The *note nis: bf. module now supports accessing domains other than + the system default domain by supplying a `domain' argument to the + *note nis.match(): d92. and *note nis.maps(): d93. functions. + (Contributed by Ben Bell.) + + * The *note operator: c2. module’s ‘itemgetter()’ and ‘attrgetter()’ + functions now support multiple fields. A call such as + ‘operator.attrgetter('a', 'b')’ will return a function that + retrieves the ‘a’ and ‘b’ attributes. Combining this new feature + with the ‘sort()’ method’s ‘key’ parameter lets you easily sort + lists using multiple fields. (Contributed by Raymond Hettinger.) + + * The *note optparse: c3. module was updated to version 1.5.1 of the + Optik library. The ‘OptionParser’ class gained an ‘epilog’ + attribute, a string that will be printed after the help message, + and a ‘destroy()’ method to break reference cycles created by the + object. (Contributed by Greg Ward.) + + * The *note os: c4. module underwent several changes. The + ‘stat_float_times’ variable now defaults to true, meaning that + *note os.stat(): 1f1. will now return time values as floats. (This + doesn’t necessarily mean that *note os.stat(): 1f1. will return + times that are precise to fractions of a second; not all systems + support such precision.) + + Constants named *note os.SEEK_SET: d94, *note os.SEEK_CUR: d95, and + *note os.SEEK_END: d96. have been added; these are the parameters + to the *note os.lseek(): a5e. function. Two new constants for + locking are *note os.O_SHLOCK: d97. and *note os.O_EXLOCK: d98. + + Two new functions, ‘wait3()’ and ‘wait4()’, were added. They’re + similar the ‘waitpid()’ function which waits for a child process to + exit and returns a tuple of the process ID and its exit status, but + ‘wait3()’ and ‘wait4()’ return additional information. ‘wait3()’ + doesn’t take a process ID as input, so it waits for any child + process to exit and returns a 3-tuple of `process-id', + `exit-status', `resource-usage' as returned from the *note + resource.getrusage(): d99. function. ‘wait4(pid)’ does take a + process ID. (Contributed by Chad J. Schroeder.) + + On FreeBSD, the *note os.stat(): 1f1. function now returns times + with nanosecond resolution, and the returned object now has + ‘st_gen’ and ‘st_birthtime’. The ‘st_flags’ attribute is also + available, if the platform supports it. (Contributed by Antti + Louko and Diego Pettenò.) + + * The Python debugger provided by the *note pdb: c9. module can now + store lists of commands to execute when a breakpoint is reached and + execution stops. Once breakpoint #1 has been created, enter + ‘commands 1’ and enter a series of commands to be executed, + finishing the list with ‘end’. The command list can include + commands that resume execution, such as ‘continue’ or ‘next’. + (Contributed by Grégoire Dooms.) + + * The *note pickle: ca. and ‘cPickle’ modules no longer accept a + return value of ‘None’ from the *note __reduce__(): 19b. method; + the method must return a tuple of arguments instead. The ability + to return ‘None’ was deprecated in Python 2.4, so this completes + the removal of the feature. + + * The *note pkgutil: cd. module, containing various utility functions + for finding packages, was enhanced to support PEP 302(1)’s import + hooks and now also works for packages stored in ZIP-format + archives. (Contributed by Phillip J. Eby.) + + * The pybench benchmark suite by Marc-André Lemburg is now included + in the ‘Tools/pybench’ directory. The pybench suite is an + improvement on the commonly used ‘pystone.py’ program because + pybench provides a more detailed measurement of the interpreter’s + speed. It times particular operations such as function calls, + tuple slicing, method lookups, and numeric operations, instead of + performing many different operations and reducing the result to a + single number as ‘pystone.py’ does. + + * The ‘pyexpat’ module now uses version 2.0 of the Expat parser. + (Contributed by Trent Mick.) + + * The *note Queue: d18. class provided by the ‘Queue’ module gained + two new methods. ‘join()’ blocks until all items in the queue have + been retrieved and all processing work on the items have been + completed. Worker threads call the other new method, + ‘task_done()’, to signal that processing for an item has been + completed. (Contributed by Raymond Hettinger.) + + * The old ‘regex’ and ‘regsub’ modules, which have been deprecated + ever since Python 2.0, have finally been deleted. Other deleted + modules: ‘statcache’, ‘tzparse’, ‘whrandom’. + + * Also deleted: the ‘lib-old’ directory, which includes ancient + modules such as ‘dircmp’ and ‘ni’, was removed. ‘lib-old’ wasn’t + on the default ‘sys.path’, so unless your programs explicitly added + the directory to ‘sys.path’, this removal shouldn’t affect your + code. + + * The *note rlcompleter: e1. module is no longer dependent on + importing the *note readline: de. module and therefore now works on + non-Unix platforms. (Patch from Robert Kiendl.) + + * The ‘SimpleXMLRPCServer’ and ‘DocXMLRPCServer’ classes now have a + ‘rpc_paths’ attribute that constrains XML-RPC operations to a + limited set of URL paths; the default is to allow only ‘'/'’ and + ‘'/RPC2'’. Setting ‘rpc_paths’ to ‘None’ or an empty tuple + disables this path checking. + + * The *note socket: ef. module now supports ‘AF_NETLINK’ sockets on + Linux, thanks to a patch from Philippe Biondi. Netlink sockets are + a Linux-specific mechanism for communications between a user-space + process and kernel code; an introductory article about them is at + ‘https://www.linuxjournal.com/article/7356’. In Python code, + netlink addresses are represented as a tuple of 2 integers, ‘(pid, + group_mask)’. + + Two new methods on socket objects, ‘recv_into(buffer)’ and + ‘recvfrom_into(buffer)’, store the received data in an object that + supports the buffer protocol instead of returning the data as a + string. This means you can put the data directly into an array or + a memory-mapped file. + + Socket objects also gained ‘getfamily()’, ‘gettype()’, and + ‘getproto()’ accessor methods to retrieve the family, type, and + protocol values for the socket. + + * New module: the *note spwd: f1. module provides functions for + accessing the shadow password database on systems that support + shadow passwords. + + * The *note struct: f8. is now faster because it compiles format + strings into ‘Struct’ objects with ‘pack()’ and ‘unpack()’ methods. + This is similar to how the *note re: dd. module lets you create + compiled regular expression objects. You can still use the + module-level ‘pack()’ and ‘unpack()’ functions; they’ll create + ‘Struct’ objects and cache them. Or you can use ‘Struct’ instances + directly: + + s = struct.Struct('ih3s') + + data = s.pack(1972, 187, 'abc') + year, number, name = s.unpack(data) + + You can also pack and unpack data to and from buffer objects + directly using the ‘pack_into(buffer, offset, v1, v2, ...)’ and + ‘unpack_from(buffer, offset)’ methods. This lets you store data + directly into an array or a memory-mapped file. + + (‘Struct’ objects were implemented by Bob Ippolito at the + NeedForSpeed sprint. Support for buffer objects was added by + Martin Blais, also at the NeedForSpeed sprint.) + + * The Python developers switched from CVS to Subversion during the + 2.5 development process. Information about the exact build version + is available as the ‘sys.subversion’ variable, a 3-tuple of + ‘(interpreter-name, branch-name, revision-range)’. For example, at + the time of writing my copy of 2.5 was reporting ‘('CPython', + 'trunk', '45313:45315')’. + + This information is also available to C extensions via the *note + Py_GetBuildInfo(): d9a. function that returns a string of build + information like this: ‘"trunk:45355:45356M, Apr 13 2006, + 07:42:19"’. (Contributed by Barry Warsaw.) + + * Another new function, *note sys._current_frames(): d9b, returns the + current stack frames for all running threads as a dictionary + mapping thread identifiers to the topmost stack frame currently + active in that thread at the time the function is called. + (Contributed by Tim Peters.) + + * The ‘TarFile’ class in the *note tarfile: 101. module now has an + ‘extractall()’ method that extracts all members from the archive + into the current working directory. It’s also possible to set a + different directory as the extraction target, and to unpack only a + subset of the archive’s members. + + The compression used for a tarfile opened in stream mode can now be + autodetected using the mode ‘'r|*'’. (Contributed by Lars + Gustäbel.) + + * The *note threading: 109. module now lets you set the stack size + used when new threads are created. The ‘stack_size([*size*])’ + function returns the currently configured stack size, and supplying + the optional `size' parameter sets a new value. Not all platforms + support changing the stack size, but Windows, POSIX threading, and + OS/2 all do. (Contributed by Andrew MacIntyre.) + + * The *note unicodedata: 11a. module has been updated to use version + 4.1.0 of the Unicode character database. Version 3.2.0 is required + by some specifications, so it’s still available as *note + unicodedata.ucd_3_2_0: d9c. + + * New module: the *note uuid: 124. module generates universally + unique identifiers (UUIDs) according to RFC 4122(2). The RFC + defines several different UUID versions that are generated from a + starting string, from system properties, or purely randomly. This + module contains a ‘UUID’ class and functions named ‘uuid1()’, + ‘uuid3()’, ‘uuid4()’, and ‘uuid5()’ to generate different versions + of UUID. (Version 2 UUIDs are not specified in RFC 4122(3) and are + not supported by this module.) + + >>> import uuid + >>> # make a UUID based on the host ID and current time + >>> uuid.uuid1() + UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + + >>> # make a UUID using an MD5 hash of a namespace UUID and a name + >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') + UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + + >>> # make a random UUID + >>> uuid.uuid4() + UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + + >>> # make a UUID using a SHA-1 hash of a namespace UUID and a name + >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') + UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + + (Contributed by Ka-Ping Yee.) + + * The *note weakref: 128. module’s ‘WeakKeyDictionary’ and + ‘WeakValueDictionary’ types gained new methods for iterating over + the weak references contained in the dictionary. ‘iterkeyrefs()’ + and ‘keyrefs()’ methods were added to ‘WeakKeyDictionary’, and + ‘itervaluerefs()’ and ‘valuerefs()’ were added to + ‘WeakValueDictionary’. (Contributed by Fred L. Drake, Jr.) + + * The *note webbrowser: 129. module received a number of + enhancements. It’s now usable as a script with ‘python -m + webbrowser’, taking a URL as the argument; there are a number of + switches to control the behaviour (‘-n’ for a new browser window, + ‘-t’ for a new tab). New module-level functions, ‘open_new()’ and + ‘open_new_tab()’, were added to support this. The module’s *note + open(): 4f0. function supports an additional feature, an + `autoraise' parameter that signals whether to raise the open window + when possible. A number of additional browsers were added to the + supported list such as Firefox, Opera, Konqueror, and elinks. + (Contributed by Oleg Broytmann and Georg Brandl.) + + * The ‘xmlrpclib’ module now supports returning *note datetime: 194. + objects for the XML-RPC date type. Supply ‘use_datetime=True’ to + the ‘loads()’ function or the ‘Unmarshaller’ class to enable this + feature. (Contributed by Skip Montanaro.) + + * The *note zipfile: 142. module now supports the ZIP64 version of + the format, meaning that a .zip archive can now be larger than 4 + GiB and can contain individual files larger than 4 GiB. + (Contributed by Ronald Oussoren.) + + * The *note zlib: 144. module’s ‘Compress’ and ‘Decompress’ objects + now support a *note copy(): 26. method that makes a copy of the + object’s internal state and returns a new ‘Compress’ or + ‘Decompress’ object. (Contributed by Chris AtLee.) + +* Menu: + +* The ctypes package:: +* The ElementTree package:: +* The hashlib package:: +* The sqlite3 package:: +* The wsgiref package:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0302 + + (2) https://tools.ietf.org/html/rfc4122.html + + (3) https://tools.ietf.org/html/rfc4122.html + + +File: python.info, Node: The ctypes package, Next: The ElementTree package, Up: New Improved and Removed Modules + +1.12.13.1 The ctypes package +............................ + +The *note ctypes: 2b. package, written by Thomas Heller, has been added +to the standard library. *note ctypes: 2b. lets you call arbitrary +functions in shared libraries or DLLs. Long-time users may remember the +‘dl’ module, which provides functions for loading shared libraries and +calling functions in them. The *note ctypes: 2b. package is much +fancier. + +To load a shared library or DLL, you must create an instance of the +‘CDLL’ class and provide the name or path of the shared library or DLL. +Once that’s done, you can call arbitrary functions by accessing them as +attributes of the ‘CDLL’ object. + + import ctypes + + libc = ctypes.CDLL('libc.so.6') + result = libc.printf("Line of output\n") + +Type constructors for the various C types are provided: ‘c_int()’, +‘c_float()’, ‘c_double()’, ‘c_char_p()’ (equivalent to ‘char *’), and so +forth. Unlike Python’s types, the C versions are all mutable; you can +assign to their ‘value’ attribute to change the wrapped value. Python +integers and strings will be automatically converted to the +corresponding C types, but for other types you must call the correct +type constructor. (And I mean `must'; getting it wrong will often +result in the interpreter crashing with a segmentation fault.) + +You shouldn’t use ‘c_char_p()’ with a Python string when the C function +will be modifying the memory area, because Python strings are supposed +to be immutable; breaking this rule will cause puzzling bugs. When you +need a modifiable memory area, use ‘create_string_buffer()’: + + s = "this is a string" + buf = ctypes.create_string_buffer(s) + libc.strfry(buf) + +C functions are assumed to return integers, but you can set the +‘restype’ attribute of the function object to change this: + + >>> libc.atof('2.71828') + -1783957616 + >>> libc.atof.restype = ctypes.c_double + >>> libc.atof('2.71828') + 2.71828 + +*note ctypes: 2b. also provides a wrapper for Python’s C API as the +‘ctypes.pythonapi’ object. This object does `not' release the global +interpreter lock before calling a function, because the lock must be +held when calling into the interpreter’s code. There’s a ‘py_object()’ +type constructor that will create a *note PyObject *: 4ba. pointer. A +simple usage: + + import ctypes + + d = {} + ctypes.pythonapi.PyObject_SetItem(ctypes.py_object(d), + ctypes.py_object("abc"), ctypes.py_object(1)) + # d is now {'abc', 1}. + +Don’t forget to use ‘py_object()’; if it’s omitted you end up with a +segmentation fault. + +*note ctypes: 2b. has been around for a while, but people still write +and distribution hand-coded extension modules because you can’t rely on +*note ctypes: 2b. being present. Perhaps developers will begin to write +Python wrappers atop a library accessed through *note ctypes: 2b. +instead of extension modules, now that *note ctypes: 2b. is included +with core Python. + +See also +........ + +‘http://starship.python.net/crew/theller/ctypes/’ + + The ctypes web page, with a tutorial, reference, and FAQ. + +The documentation for the *note ctypes: 2b. module. + + +File: python.info, Node: The ElementTree package, Next: The hashlib package, Prev: The ctypes package, Up: New Improved and Removed Modules + +1.12.13.2 The ElementTree package +................................. + +A subset of Fredrik Lundh’s ElementTree library for processing XML has +been added to the standard library as ‘xml.etree’. The available +modules are ‘ElementTree’, ‘ElementPath’, and ‘ElementInclude’ from +ElementTree 1.2.6. The ‘cElementTree’ accelerator module is also +included. + +The rest of this section will provide a brief overview of using +ElementTree. Full documentation for ElementTree is available at +‘http://effbot.org/zone/element-index.htm’. + +ElementTree represents an XML document as a tree of element nodes. The +text content of the document is stored as the ‘text’ and ‘tail’ +attributes of (This is one of the major differences between ElementTree +and the Document Object Model; in the DOM there are many different types +of node, including ‘TextNode’.) + +The most commonly used parsing function is ‘parse()’, that takes either +a string (assumed to contain a filename) or a file-like object and +returns an ‘ElementTree’ instance: + + from xml.etree import ElementTree as ET + + tree = ET.parse('ex-1.xml') + + feed = urllib.urlopen( + 'http://planet.python.org/rss10.xml') + tree = ET.parse(feed) + +Once you have an ‘ElementTree’ instance, you can call its ‘getroot()’ +method to get the root ‘Element’ node. + +There’s also an ‘XML()’ function that takes a string literal and returns +an ‘Element’ node (not an ‘ElementTree’). This function provides a tidy +way to incorporate XML fragments, approaching the convenience of an XML +literal: + + svg = ET.XML(""" + """) + svg.set('height', '320px') + svg.append(elem1) + +Each XML element supports some dictionary-like and some list-like access +methods. Dictionary-like operations are used to access attribute +values, and list-like operations are used to access child nodes. + +Operation Result + +------------------------------------------------------------------------------------- + +‘elem[n]’ Returns n’th child element. + + +‘elem[m:n]’ Returns list of m’th through n’th child + elements. + + +‘len(elem)’ Returns number of child elements. + + +‘list(elem)’ Returns list of child elements. + + +‘elem.append(elem2)’ Adds `elem2' as a child. + + +‘elem.insert(index, elem2)’ Inserts `elem2' at the specified location. + + +‘del elem[n]’ Deletes n’th child element. + + +‘elem.keys()’ Returns list of attribute names. + + +‘elem.get(name)’ Returns value of attribute `name'. + + +‘elem.set(name, value)’ Sets new value for attribute `name'. + + +‘elem.attrib’ Retrieves the dictionary containing + attributes. + + +‘del elem.attrib[name]’ Deletes attribute `name'. + + +Comments and processing instructions are also represented as ‘Element’ +nodes. To check if a node is a comment or processing instructions: + + if elem.tag is ET.Comment: + ... + elif elem.tag is ET.ProcessingInstruction: + ... + +To generate XML output, you should call the ‘ElementTree.write()’ +method. Like ‘parse()’, it can take either a string or a file-like +object: + + # Encoding is US-ASCII + tree.write('output.xml') + + # Encoding is UTF-8 + f = open('output.xml', 'w') + tree.write(f, encoding='utf-8') + +(Caution: the default encoding used for output is ASCII. For general XML +work, where an element’s name may contain arbitrary Unicode characters, +ASCII isn’t a very useful encoding because it will raise an exception if +an element’s name contains any characters with values greater than 127. +Therefore, it’s best to specify a different encoding such as UTF-8 that +can handle any Unicode character.) + +This section is only a partial description of the ElementTree +interfaces. Please read the package’s official documentation for more +details. + +See also +........ + +‘http://effbot.org/zone/element-index.htm’ + + Official documentation for ElementTree. + + +File: python.info, Node: The hashlib package, Next: The sqlite3 package, Prev: The ElementTree package, Up: New Improved and Removed Modules + +1.12.13.3 The hashlib package +............................. + +A new *note hashlib: 8d. module, written by Gregory P. Smith, has been +added to replace the ‘md5’ and ‘sha’ modules. *note hashlib: 8d. adds +support for additional secure hashes (SHA-224, SHA-256, SHA-384, and +SHA-512). When available, the module uses OpenSSL for fast platform +optimized implementations of algorithms. + +The old ‘md5’ and ‘sha’ modules still exist as wrappers around hashlib +to preserve backwards compatibility. The new module’s interface is very +close to that of the old modules, but not identical. The most +significant difference is that the constructor functions for creating +new hashing objects are named differently. + + # Old versions + h = md5.md5() + h = md5.new() + + # New version + h = hashlib.md5() + + # Old versions + h = sha.sha() + h = sha.new() + + # New version + h = hashlib.sha1() + + # Hash that weren't previously available + h = hashlib.sha224() + h = hashlib.sha256() + h = hashlib.sha384() + h = hashlib.sha512() + + # Alternative form + h = hashlib.new('md5') # Provide algorithm as a string + +Once a hash object has been created, its methods are the same as before: +‘update(string)’ hashes the specified string into the current digest +state, ‘digest()’ and ‘hexdigest()’ return the digest value as a binary +string or a string of hex digits, and *note copy(): 26. returns a new +hashing object with the same digest state. + +See also +........ + +The documentation for the *note hashlib: 8d. module. + + +File: python.info, Node: The sqlite3 package, Next: The wsgiref package, Prev: The hashlib package, Up: New Improved and Removed Modules + +1.12.13.4 The sqlite3 package +............................. + +The pysqlite module (‘http://www.pysqlite.org’), a wrapper for the +SQLite embedded database, has been added to the standard library under +the package name *note sqlite3: f2. + +SQLite is a C library that provides a lightweight disk-based database +that doesn’t require a separate server process and allows accessing the +database using a nonstandard variant of the SQL query language. Some +applications can use SQLite for internal data storage. It’s also +possible to prototype an application using SQLite and then port the code +to a larger database such as PostgreSQL or Oracle. + +pysqlite was written by Gerhard Häring and provides a SQL interface +compliant with the DB-API 2.0 specification described by PEP 249(1). + +If you’re compiling the Python source yourself, note that the source +tree doesn’t include the SQLite code, only the wrapper module. You’ll +need to have the SQLite libraries and headers installed before compiling +Python, and the build process will compile the module when the necessary +headers are available. + +To use the module, you must first create a ‘Connection’ object that +represents the database. Here the data will be stored in the +‘/tmp/example’ file: + + conn = sqlite3.connect('/tmp/example') + +You can also supply the special name ‘:memory:’ to create a database in +RAM. + +Once you have a ‘Connection’, you can create a ‘Cursor’ object and call +its ‘execute()’ method to perform SQL commands: + + c = conn.cursor() + + # Create table + c.execute('''create table stocks + (date text, trans text, symbol text, + qty real, price real)''') + + # Insert a row of data + c.execute("""insert into stocks + values ('2006-01-05','BUY','RHAT',100,35.14)""") + +Usually your SQL operations will need to use values from Python +variables. You shouldn’t assemble your query using Python’s string +operations because doing so is insecure; it makes your program +vulnerable to an SQL injection attack. + +Instead, use the DB-API’s parameter substitution. Put ‘?’ as a +placeholder wherever you want to use a value, and then provide a tuple +of values as the second argument to the cursor’s ‘execute()’ method. +(Other database modules may use a different placeholder, such as ‘%s’ or +‘:1’.) For example: + + # Never do this -- insecure! + symbol = 'IBM' + c.execute("... where symbol = '%s'" % symbol) + + # Do this instead + t = (symbol,) + c.execute('select * from stocks where symbol=?', t) + + # Larger example + for t in (('2006-03-28', 'BUY', 'IBM', 1000, 45.00), + ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00), + ('2006-04-06', 'SELL', 'IBM', 500, 53.00), + ): + c.execute('insert into stocks values (?,?,?,?,?)', t) + +To retrieve data after executing a SELECT statement, you can either +treat the cursor as an iterator, call the cursor’s ‘fetchone()’ method +to retrieve a single matching row, or call ‘fetchall()’ to get a list of +the matching rows. + +This example uses the iterator form: + + >>> c = conn.cursor() + >>> c.execute('select * from stocks order by price') + >>> for row in c: + ... print row + ... + (u'2006-01-05', u'BUY', u'RHAT', 100, 35.140000000000001) + (u'2006-03-28', u'BUY', u'IBM', 1000, 45.0) + (u'2006-04-06', u'SELL', u'IBM', 500, 53.0) + (u'2006-04-05', u'BUY', u'MSOFT', 1000, 72.0) + >>> + +For more information about the SQL dialect supported by SQLite, see +‘https://www.sqlite.org’. + +See also +........ + +‘http://www.pysqlite.org’ + + The pysqlite web page. + +‘https://www.sqlite.org’ + + The SQLite web page; the documentation describes the syntax and the + available data types for the supported SQL dialect. + +The documentation for the *note sqlite3: f2. module. + +PEP 249(2) - Database API Specification 2.0 + + PEP written by Marc-André Lemburg. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0249 + + (2) https://www.python.org/dev/peps/pep-0249 + + +File: python.info, Node: The wsgiref package, Prev: The sqlite3 package, Up: New Improved and Removed Modules + +1.12.13.5 The wsgiref package +............................. + +The Web Server Gateway Interface (WSGI) v1.0 defines a standard +interface between web servers and Python web applications and is +described in PEP 333(1). The *note wsgiref: 12c. package is a reference +implementation of the WSGI specification. + +The package includes a basic HTTP server that will run a WSGI +application; this server is useful for debugging but isn’t intended for +production use. Setting up a server takes only a few lines of code: + + from wsgiref import simple_server + + wsgi_app = ... + + host = '' + port = 8000 + httpd = simple_server.make_server(host, port, wsgi_app) + httpd.serve_forever() + +See also +........ + +‘http://www.wsgi.org’ + + A central web site for WSGI-related resources. + +PEP 333(2) - Python Web Server Gateway Interface v1.0 + + PEP written by Phillip J. Eby. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0333 + + (2) https://www.python.org/dev/peps/pep-0333 + + +File: python.info, Node: Build and C API Changes<10>, Next: Porting to Python 2 5, Prev: New Improved and Removed Modules, Up: What’s New in Python 2 5 + +1.12.14 Build and C API Changes +------------------------------- + +Changes to Python’s build process and to the C API include: + + * The Python source tree was converted from CVS to Subversion, in a + complex migration procedure that was supervised and flawlessly + carried out by Martin von Löwis. The procedure was developed as + PEP 347(1). + + * Coverity, a company that markets a source code analysis tool called + Prevent, provided the results of their examination of the Python + source code. The analysis found about 60 bugs that were quickly + fixed. Many of the bugs were refcounting problems, often occurring + in error-handling code. See ‘https://scan.coverity.com’ for the + statistics. + + * The largest change to the C API came from PEP 353(2), which + modifies the interpreter to use a ‘Py_ssize_t’ type definition + instead of ‘int’. See the earlier section *note PEP 353; Using + ssize_t as the index type: d7c. for a discussion of this change. + + * The design of the bytecode compiler has changed a great deal, no + longer generating bytecode by traversing the parse tree. Instead + the parse tree is converted to an abstract syntax tree (or AST), + and it is the abstract syntax tree that’s traversed to produce the + bytecode. + + It’s possible for Python code to obtain AST objects by using the + *note compile(): 1b4. built-in and specifying ‘_ast.PyCF_ONLY_AST’ + as the value of the `flags' parameter: + + from _ast import PyCF_ONLY_AST + ast = compile("""a=0 + for i in range(10): + a += i + """, "", 'exec', PyCF_ONLY_AST) + + assignment = ast.body[0] + for_loop = ast.body[1] + + No official documentation has been written for the AST code yet, + but PEP 339(3) discusses the design. To start learning about the + code, read the definition of the various AST nodes in + ‘Parser/Python.asdl’. A Python script reads this file and + generates a set of C structure definitions in + ‘Include/Python-ast.h’. The ‘PyParser_ASTFromString()’ and + ‘PyParser_ASTFromFile()’, defined in ‘Include/pythonrun.h’, take + Python source as input and return the root of an AST representing + the contents. This AST can then be turned into a code object by + ‘PyAST_Compile()’. For more information, read the source code, and + then ask questions on python-dev. + + The AST code was developed under Jeremy Hylton’s management, and + implemented by (in alphabetical order) Brett Cannon, Nick Coghlan, + Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Tim + Peters, Armin Rigo, and Neil Schemenauer, plus the participants in + a number of AST sprints at conferences such as PyCon. + + * Evan Jones’s patch to obmalloc, first described in a talk at PyCon + DC 2005, was applied. Python 2.4 allocated small objects in + 256K-sized arenas, but never freed arenas. With this patch, Python + will free arenas when they’re empty. The net effect is that on + some platforms, when you allocate many objects, Python’s memory + usage may actually drop when you delete them and the memory may be + returned to the operating system. (Implemented by Evan Jones, and + reworked by Tim Peters.) + + Note that this change means extension modules must be more careful + when allocating memory. Python’s API has many different functions + for allocating memory that are grouped into families. For example, + *note PyMem_Malloc(): 505, *note PyMem_Realloc(): 96f, and *note + PyMem_Free(): da9. are one family that allocates raw memory, while + *note PyObject_Malloc(): 508, *note PyObject_Realloc(): daa, and + *note PyObject_Free(): 504. are another family that’s supposed to + be used for creating Python objects. + + Previously these different families all reduced to the platform’s + ‘malloc()’ and ‘free()’ functions. This meant it didn’t matter if + you got things wrong and allocated memory with the ‘PyMem()’ + function but freed it with the *note PyObject(): 4ba. function. + With 2.5’s changes to obmalloc, these families now do different + things and mismatches will probably result in a segfault. You + should carefully test your C extension modules with Python 2.5. + + * The built-in set types now have an official C API. Call *note + PySet_New(): dab. and *note PyFrozenSet_New(): dac. to create a new + set, *note PySet_Add(): dad. and *note PySet_Discard(): dae. to add + and remove elements, and *note PySet_Contains(): daf. and *note + PySet_Size(): db0. to examine the set’s state. (Contributed by + Raymond Hettinger.) + + * C code can now obtain information about the exact revision of the + Python interpreter by calling the *note Py_GetBuildInfo(): d9a. + function that returns a string of build information like this: + ‘"trunk:45355:45356M, Apr 13 2006, 07:42:19"’. (Contributed by + Barry Warsaw.) + + * Two new macros can be used to indicate C functions that are local + to the current file so that a faster calling convention can be + used. ‘Py_LOCAL(type)’ declares the function as returning a value + of the specified `type' and uses a fast-calling qualifier. + ‘Py_LOCAL_INLINE(type)’ does the same thing and also requests the + function be inlined. If ‘PY_LOCAL_AGGRESSIVE()’ is defined before + ‘python.h’ is included, a set of more aggressive optimizations are + enabled for the module; you should benchmark the results to find + out if these optimizations actually make the code faster. + (Contributed by Fredrik Lundh at the NeedForSpeed sprint.) + + * ‘PyErr_NewException(name, base, dict)’ can now accept a tuple of + base classes as its `base' argument. (Contributed by Georg + Brandl.) + + * The ‘PyErr_Warn()’ function for issuing warnings is now deprecated + in favour of ‘PyErr_WarnEx(category, message, stacklevel)’ which + lets you specify the number of stack frames separating this + function and the caller. A `stacklevel' of 1 is the function + calling *note PyErr_WarnEx(): db1, 2 is the function above that, + and so forth. (Added by Neal Norwitz.) + + * The CPython interpreter is still written in C, but the code can now + be compiled with a C++ compiler without errors. (Implemented by + Anthony Baxter, Martin von Löwis, Skip Montanaro.) + + * The ‘PyRange_New()’ function was removed. It was never documented, + never used in the core code, and had dangerously lax error + checking. In the unlikely case that your extensions were using it, + you can replace it by something like the following: + + range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", + start, stop, step); + +* Menu: + +* Port-Specific Changes:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0347 + + (2) https://www.python.org/dev/peps/pep-0353 + + (3) https://www.python.org/dev/peps/pep-0339 + + +File: python.info, Node: Port-Specific Changes, Up: Build and C API Changes<10> + +1.12.14.1 Port-Specific Changes +............................... + + * MacOS X (10.3 and higher): dynamic loading of modules now uses the + ‘dlopen()’ function instead of MacOS-specific functions. + + * MacOS X: an ‘--enable-universalsdk’ switch was added to the + ‘configure’ script that compiles the interpreter as a universal + binary able to run on both PowerPC and Intel processors. + (Contributed by Ronald Oussoren; bpo-2573(1).) + + * Windows: ‘.dll’ is no longer supported as a filename extension for + extension modules. ‘.pyd’ is now the only filename extension that + will be searched for. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue2573 + + +File: python.info, Node: Porting to Python 2 5, Next: Acknowledgements<3>, Prev: Build and C API Changes<10>, Up: What’s New in Python 2 5 + +1.12.15 Porting to Python 2.5 +----------------------------- + +This section lists previously described changes that may require changes +to your code: + + * ASCII is now the default encoding for modules. It’s now a syntax + error if a module contains string literals with 8-bit characters + but doesn’t have an encoding declaration. In Python 2.4 this + triggered a warning, not a syntax error. + + * Previously, the ‘gi_frame’ attribute of a generator was always a + frame object. Because of the PEP 342(1) changes described in + section *note PEP 342; New Generator Features: d66, it’s now + possible for ‘gi_frame’ to be ‘None’. + + * A new warning, *note UnicodeWarning: d87, is triggered when you + attempt to compare a Unicode string and an 8-bit string that can’t + be converted to Unicode using the default ASCII encoding. + Previously such comparisons would raise a *note UnicodeDecodeError: + 1fd. exception. + + * Library: the *note csv: 2a. module is now stricter about multi-line + quoted fields. If your files contain newlines embedded within + fields, the input should be split into lines in a manner which + preserves the newline characters. + + * Library: the *note locale: a9. module’s *note format(): 4db. + function’s would previously accept any string as long as no more + than one %char specifier appeared. In Python 2.5, the argument + must be exactly one %char specifier with no surrounding text. + + * Library: The *note pickle: ca. and ‘cPickle’ modules no longer + accept a return value of ‘None’ from the *note __reduce__(): 19b. + method; the method must return a tuple of arguments instead. The + modules also no longer accept the deprecated `bin' keyword + parameter. + + * Library: The ‘SimpleXMLRPCServer’ and ‘DocXMLRPCServer’ classes now + have a ‘rpc_paths’ attribute that constrains XML-RPC operations to + a limited set of URL paths; the default is to allow only ‘'/'’ and + ‘'/RPC2'’. Setting ‘rpc_paths’ to ‘None’ or an empty tuple + disables this path checking. + + * C API: Many functions now use ‘Py_ssize_t’ instead of ‘int’ to + allow processing more data on 64-bit machines. Extension code may + need to make the same change to avoid warnings and to support + 64-bit machines. See the earlier section *note PEP 353; Using + ssize_t as the index type: d7c. for a discussion of this change. + + * C API: The obmalloc changes mean that you must be careful to not + mix usage of the ‘PyMem_*()’ and ‘PyObject_*()’ families of + functions. Memory allocated with one family’s ‘*_Malloc()’ must be + freed with the corresponding family’s ‘*_Free()’ function. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0342 + + +File: python.info, Node: Acknowledgements<3>, Prev: Porting to Python 2 5, Up: What’s New in Python 2 5 + +1.12.16 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Georg Brandl, Nick Coghlan, Phillip J. Eby, Lars Gustäbel, +Raymond Hettinger, Ralf W. Grosse-Kunstleve, Kent Johnson, Iain Lowe, +Martin von Löwis, Fredrik Lundh, Andrew McNamara, Skip Montanaro, +Gustavo Niemeyer, Paul Prescod, James Pryor, Mike Rovner, Scott Weikart, +Barry Warsaw, Thomas Wouters. + + +File: python.info, Node: What’s New in Python 2 4, Next: What’s New in Python 2 3, Prev: What’s New in Python 2 5, Up: What’s New in Python + +1.13 What’s New in Python 2.4 +============================= + + +Author: A.M. Kuchling + +This article explains the new features in Python 2.4.1, released on +March 30, 2005. + +Python 2.4 is a medium-sized release. It doesn’t introduce as many +changes as the radical Python 2.2, but introduces more features than the +conservative 2.3 release. The most significant new language features +are function decorators and generator expressions; most other changes +are to the standard library. + +According to the CVS change logs, there were 481 patches applied and 502 +bugs fixed between Python 2.3 and 2.4. Both figures are likely to be +underestimates. + +This article doesn’t attempt to provide a complete specification of +every single new feature, but instead provides a brief introduction to +each feature. For full details, you should refer to the documentation +for Python 2.4, such as the Python Library Reference and the Python +Reference Manual. Often you will be referred to the PEP for a +particular new feature for explanations of the implementation and design +rationale. + +* Menu: + +* PEP 218; Built-In Set Objects: PEP 218 Built-In Set Objects. +* PEP 237; Unifying Long Integers and Integers: PEP 237 Unifying Long Integers and Integers. +* PEP 289; Generator Expressions: PEP 289 Generator Expressions. +* PEP 292; Simpler String Substitutions: PEP 292 Simpler String Substitutions. +* PEP 318; Decorators for Functions and Methods: PEP 318 Decorators for Functions and Methods. +* PEP 322; Reverse Iteration: PEP 322 Reverse Iteration. +* PEP 324; New subprocess Module: PEP 324 New subprocess Module. +* PEP 327; Decimal Data Type: PEP 327 Decimal Data Type. +* PEP 328; Multi-line Imports: PEP 328 Multi-line Imports. +* PEP 331; Locale-Independent Float/String Conversions: PEP 331 Locale-Independent Float/String Conversions. +* Other Language Changes: Other Language Changes<12>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules<3>. +* Build and C API Changes: Build and C API Changes<11>. +* Porting to Python 2.4: Porting to Python 2 4. +* Acknowledgements: Acknowledgements<4>. + + +File: python.info, Node: PEP 218 Built-In Set Objects, Next: PEP 237 Unifying Long Integers and Integers, Up: What’s New in Python 2 4 + +1.13.1 PEP 218: Built-In Set Objects +------------------------------------ + +Python 2.3 introduced the ‘sets’ module. C implementations of set data +types have now been added to the Python core as two new built-in types, +‘set(iterable)’ and ‘frozenset(iterable)’. They provide high speed +operations for membership testing, for eliminating duplicates from +sequences, and for mathematical operations like unions, intersections, +differences, and symmetric differences. + + >>> a = set('abracadabra') # form a set from a string + >>> 'z' in a # fast membership testing + False + >>> a # unique letters in a + set(['a', 'r', 'b', 'c', 'd']) + >>> ''.join(a) # convert back into a string + 'arbcd' + + >>> b = set('alacazam') # form a second set + >>> a - b # letters in a but not in b + set(['r', 'd', 'b']) + >>> a | b # letters in either a or b + set(['a', 'c', 'r', 'd', 'b', 'm', 'z', 'l']) + >>> a & b # letters in both a and b + set(['a', 'c']) + >>> a ^ b # letters in a or b but not both + set(['r', 'd', 'b', 'm', 'z', 'l']) + + >>> a.add('z') # add a new element + >>> a.update('wxy') # add multiple new elements + >>> a + set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'x', 'z']) + >>> a.remove('x') # take one element out + >>> a + set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'z']) + +The *note frozenset(): bee. type is an immutable version of *note set(): +b6f. Since it is immutable and hashable, it may be used as a dictionary +key or as a member of another set. + +The ‘sets’ module remains in the standard library, and may be useful if +you wish to subclass the ‘Set’ or ‘ImmutableSet’ classes. There are +currently no plans to deprecate the module. + +See also +........ + +PEP 218(1) - Adding a Built-In Set Object Type + + Originally proposed by Greg Wilson and ultimately implemented by + Raymond Hettinger. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0218 + + +File: python.info, Node: PEP 237 Unifying Long Integers and Integers, Next: PEP 289 Generator Expressions, Prev: PEP 218 Built-In Set Objects, Up: What’s New in Python 2 4 + +1.13.2 PEP 237: Unifying Long Integers and Integers +--------------------------------------------------- + +The lengthy transition process for this PEP, begun in Python 2.2, takes +another step forward in Python 2.4. In 2.3, certain integer operations +that would behave differently after int/long unification triggered *note +FutureWarning: 325. warnings and returned values limited to 32 or 64 +bits (depending on your platform). In 2.4, these expressions no longer +produce a warning and instead produce a different result that’s usually +a long integer. + +The problematic expressions are primarily left shifts and lengthy +hexadecimal and octal constants. For example, ‘2 << 32’ results in a +warning in 2.3, evaluating to 0 on 32-bit platforms. In Python 2.4, +this expression now returns the correct answer, 8589934592. + +See also +........ + +PEP 237(1) - Unifying Long Integers and Integers + + Original PEP written by Moshe Zadka and GvR. The changes for 2.4 + were implemented by Kalle Svensson. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0237 + + +File: python.info, Node: PEP 289 Generator Expressions, Next: PEP 292 Simpler String Substitutions, Prev: PEP 237 Unifying Long Integers and Integers, Up: What’s New in Python 2 4 + +1.13.3 PEP 289: Generator Expressions +------------------------------------- + +The iterator feature introduced in Python 2.2 and the *note itertools: +a3. module make it easier to write programs that loop through large data +sets without having the entire data set in memory at one time. List +comprehensions don’t fit into this picture very well because they +produce a Python list object containing all of the items. This +unavoidably pulls all of the objects into memory, which can be a problem +if your data set is very large. When trying to write a +functionally-styled program, it would be natural to write something +like: + + links = [link for link in get_all_links() if not link.followed] + for link in links: + ... + +instead of + + for link in get_all_links(): + if link.followed: + continue + ... + +The first form is more concise and perhaps more readable, but if you’re +dealing with a large number of link objects you’d have to write the +second form to avoid having all link objects in memory at the same time. + +Generator expressions work similarly to list comprehensions but don’t +materialize the entire list; instead they create a generator that will +return elements one by one. The above example could be written as: + + links = (link for link in get_all_links() if not link.followed) + for link in links: + ... + +Generator expressions always have to be written inside parentheses, as +in the above example. The parentheses signalling a function call also +count, so if you want to create an iterator that will be immediately +passed to a function you could write: + + print sum(obj.count for obj in list_all_objects()) + +Generator expressions differ from list comprehensions in various small +ways. Most notably, the loop variable (`obj' in the above example) is +not accessible outside of the generator expression. List comprehensions +leave the variable assigned to its last value; future versions of Python +will change this, making list comprehensions match generator expressions +in this respect. + +See also +........ + +PEP 289(1) - Generator Expressions + + Proposed by Raymond Hettinger and implemented by Jiwon Seo with + early efforts steered by Hye-Shik Chang. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0289 + + +File: python.info, Node: PEP 292 Simpler String Substitutions, Next: PEP 318 Decorators for Functions and Methods, Prev: PEP 289 Generator Expressions, Up: What’s New in Python 2 4 + +1.13.4 PEP 292: Simpler String Substitutions +-------------------------------------------- + +Some new classes in the standard library provide an alternative +mechanism for substituting variables into strings; this style of +substitution may be better for applications where untrained users need +to edit templates. + +The usual way of substituting variables by name is the ‘%’ operator: + + >>> '%(page)i: %(title)s' % {'page':2, 'title': 'The Best of Times'} + '2: The Best of Times' + +When writing the template string, it can be easy to forget the ‘i’ or +‘s’ after the closing parenthesis. This isn’t a big problem if the +template is in a Python module, because you run the code, get an +“Unsupported format character” *note ValueError: 1fb, and fix the +problem. However, consider an application such as Mailman where +template strings or translations are being edited by users who aren’t +aware of the Python language. The format string’s syntax is complicated +to explain to such users, and if they make a mistake, it’s difficult to +provide helpful feedback to them. + +PEP 292 adds a ‘Template’ class to the *note string: f6. module that +uses ‘$’ to indicate a substitution: + + >>> import string + >>> t = string.Template('$page: $title') + >>> t.substitute({'page':2, 'title': 'The Best of Times'}) + '2: The Best of Times' + +If a key is missing from the dictionary, the ‘substitute()’ method will +raise a *note KeyError: 2c7. There’s also a ‘safe_substitute()’ method +that ignores missing keys: + + >>> t = string.Template('$page: $title') + >>> t.safe_substitute({'page':3}) + '3: $title' + +See also +........ + +PEP 292(1) - Simpler String Substitutions + + Written and implemented by Barry Warsaw. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0292 + + +File: python.info, Node: PEP 318 Decorators for Functions and Methods, Next: PEP 322 Reverse Iteration, Prev: PEP 292 Simpler String Substitutions, Up: What’s New in Python 2 4 + +1.13.5 PEP 318: Decorators for Functions and Methods +---------------------------------------------------- + +Python 2.2 extended Python’s object model by adding static methods and +class methods, but it didn’t extend Python’s syntax to provide any new +way of defining static or class methods. Instead, you had to write a +*note def: dbe. statement in the usual way, and pass the resulting +method to a *note staticmethod(): 1d9. or *note classmethod(): 1d8. +function that would wrap up the function as a method of the new type. +Your code would look like this: + + class C: + def meth (cls): + ... + + meth = classmethod(meth) # Rebind name to wrapped-up class method + +If the method was very long, it would be easy to miss or forget the +*note classmethod(): 1d8. invocation after the function body. + +The intention was always to add some syntax to make such definitions +more readable, but at the time of 2.2’s release a good syntax was not +obvious. Today a good syntax `still' isn’t obvious but users are asking +for easier access to the feature; a new syntactic feature has been added +to meet this need. + +The new feature is called “function decorators”. The name comes from +the idea that *note classmethod(): 1d8, *note staticmethod(): 1d9, and +friends are storing additional information on a function object; they’re +`decorating' functions with more details. + +The notation borrows from Java and uses the ‘'@'’ character as an +indicator. Using the new syntax, the example above would be written: + + class C: + + @classmethod + def meth (cls): + ... + +The ‘@classmethod’ is shorthand for the ‘meth=classmethod(meth)’ +assignment. More generally, if you have the following: + + @A + @B + @C + def f (): + ... + +It’s equivalent to the following pre-decorator code: + + def f(): ... + f = A(B(C(f))) + +Decorators must come on the line before a function definition, one +decorator per line, and can’t be on the same line as the def statement, +meaning that ‘@A def f(): ...’ is illegal. You can only decorate +function definitions, either at the module level or inside a class; you +can’t decorate class definitions. + +A decorator is just a function that takes the function to be decorated +as an argument and returns either the same function or some new object. +The return value of the decorator need not be callable (though it +typically is), unless further decorators will be applied to the result. +It’s easy to write your own decorators. The following simple example +just sets an attribute on the function object: + + >>> def deco(func): + ... func.attr = 'decorated' + ... return func + ... + >>> @deco + ... def f(): pass + ... + >>> f + + >>> f.attr + 'decorated' + >>> + +As a slightly more realistic example, the following decorator checks +that the supplied argument is an integer: + + def require_int (func): + def wrapper (arg): + assert isinstance(arg, int) + return func(arg) + + return wrapper + + @require_int + def p1 (arg): + print arg + + @require_int + def p2(arg): + print arg*2 + +An example in PEP 318(1) contains a fancier version of this idea that +lets you both specify the required type and check the returned type. + +Decorator functions can take arguments. If arguments are supplied, your +decorator function is called with only those arguments and must return a +new decorator function; this function must take a single function and +return a function, as previously described. In other words, ‘@A @B +@C(args)’ becomes: + + def f(): ... + _deco = C(args) + f = A(B(_deco(f))) + +Getting this right can be slightly brain-bending, but it’s not too +difficult. + +A small related change makes the ‘func_name’ attribute of functions +writable. This attribute is used to display function names in +tracebacks, so decorators should change the name of any new function +that’s constructed and returned. + +See also +........ + +PEP 318(2) - Decorators for Functions, Methods and Classes + + Written by Kevin D. Smith, Jim Jewett, and Skip Montanaro. Several + people wrote patches implementing function decorators, but the one + that was actually checked in was patch #979728, written by Mark + Russell. + +‘https://wiki.python.org/moin/PythonDecoratorLibrary’ + + This Wiki page contains several examples of decorators. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0318 + + (2) https://www.python.org/dev/peps/pep-0318 + + +File: python.info, Node: PEP 322 Reverse Iteration, Next: PEP 324 New subprocess Module, Prev: PEP 318 Decorators for Functions and Methods, Up: What’s New in Python 2 4 + +1.13.6 PEP 322: Reverse Iteration +--------------------------------- + +A new built-in function, ‘reversed(seq)’, takes a sequence and returns +an iterator that loops over the elements of the sequence in reverse +order. + + >>> for i in reversed(xrange(1,4)): + ... print i + ... + 3 + 2 + 1 + +Compared to extended slicing, such as ‘range(1,4)[::-1]’, *note +reversed(): 18e. is easier to read, runs faster, and uses substantially +less memory. + +Note that *note reversed(): 18e. only accepts sequences, not arbitrary +iterators. If you want to reverse an iterator, first convert it to a +list with *note list(): 262. + + >>> input = open('/etc/passwd', 'r') + >>> for line in reversed(list(input)): + ... print line + ... + root:*:0:0:System Administrator:/var/root:/bin/tcsh + ... + +See also +........ + +PEP 322(1) - Reverse Iteration + + Written and implemented by Raymond Hettinger. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0322 + + +File: python.info, Node: PEP 324 New subprocess Module, Next: PEP 327 Decimal Data Type, Prev: PEP 322 Reverse Iteration, Up: What’s New in Python 2 4 + +1.13.7 PEP 324: New subprocess Module +------------------------------------- + +The standard library provides a number of ways to execute a subprocess, +offering different features and different levels of complexity. +‘os.system(command)’ is easy to use, but slow (it runs a shell process +which executes the command) and dangerous (you have to be careful about +escaping the shell’s metacharacters). The ‘popen2’ module offers +classes that can capture standard output and standard error from the +subprocess, but the naming is confusing. The *note subprocess: f9. +module cleans this up, providing a unified interface that offers all the +features you might need. + +Instead of ‘popen2’’s collection of classes, *note subprocess: f9. +contains a single class called ‘Popen’ whose constructor supports a +number of different keyword arguments. + + class Popen(args, bufsize=0, executable=None, + stdin=None, stdout=None, stderr=None, + preexec_fn=None, close_fds=False, shell=False, + cwd=None, env=None, universal_newlines=False, + startupinfo=None, creationflags=0): + +`args' is commonly a sequence of strings that will be the arguments to +the program executed as the subprocess. (If the `shell' argument is +true, `args' can be a string which will then be passed on to the shell +for interpretation, just as *note os.system(): dc1. does.) + +`stdin', `stdout', and `stderr' specify what the subprocess’s input, +output, and error streams will be. You can provide a file object or a +file descriptor, or you can use the constant ‘subprocess.PIPE’ to create +a pipe between the subprocess and the parent. + +The constructor has a number of handy options: + + * `close_fds' requests that all file descriptors be closed before + running the subprocess. + + * `cwd' specifies the working directory in which the subprocess will + be executed (defaulting to whatever the parent’s working directory + is). + + * `env' is a dictionary specifying environment variables. + + * `preexec_fn' is a function that gets called before the child is + started. + + * `universal_newlines' opens the child’s input and output using + Python’s *note universal newlines: 5f4. feature. + +Once you’ve created the ‘Popen’ instance, you can call its ‘wait()’ +method to pause until the subprocess has exited, ‘poll()’ to check if +it’s exited without pausing, or ‘communicate(data)’ to send the string +`data' to the subprocess’s standard input. ‘communicate(data)’ then +reads any data that the subprocess has sent to its standard output or +standard error, returning a tuple ‘(stdout_data, stderr_data)’. + +‘call()’ is a shortcut that passes its arguments along to the ‘Popen’ +constructor, waits for the command to complete, and returns the status +code of the subprocess. It can serve as a safer analog to *note +os.system(): dc1.: + + sts = subprocess.call(['dpkg', '-i', '/tmp/new-package.deb']) + if sts == 0: + # Success + ... + else: + # dpkg returned an error + ... + +The command is invoked without use of the shell. If you really do want +to use the shell, you can add ‘shell=True’ as a keyword argument and +provide a string instead of a sequence: + + sts = subprocess.call('dpkg -i /tmp/new-package.deb', shell=True) + +The PEP takes various examples of shell and Python code and shows how +they’d be translated into Python code that uses *note subprocess: f9. +Reading this section of the PEP is highly recommended. + +See also +........ + +PEP 324(1) - subprocess - New process module + + Written and implemented by Peter Åstrand, with assistance from + Fredrik Lundh and others. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0324 + + +File: python.info, Node: PEP 327 Decimal Data Type, Next: PEP 328 Multi-line Imports, Prev: PEP 324 New subprocess Module, Up: What’s New in Python 2 4 + +1.13.8 PEP 327: Decimal Data Type +--------------------------------- + +Python has always supported floating-point (FP) numbers, based on the +underlying C ‘double’ type, as a data type. However, while most +programming languages provide a floating-point type, many people (even +programmers) are unaware that floating-point numbers don’t represent +certain decimal fractions accurately. The new ‘Decimal’ type can +represent these fractions accurately, up to a user-specified precision +limit. + +* Menu: + +* Why is Decimal needed?:: +* The Decimal type:: +* The Context type:: + + +File: python.info, Node: Why is Decimal needed?, Next: The Decimal type, Up: PEP 327 Decimal Data Type + +1.13.8.1 Why is Decimal needed? +............................... + +The limitations arise from the representation used for floating-point +numbers. FP numbers are made up of three components: + + * The sign, which is positive or negative. + + * The mantissa, which is a single-digit binary number followed by a + fractional part. For example, ‘1.01’ in base-2 notation is ‘1 + + 0/2 + 1/4’, or 1.25 in decimal notation. + + * The exponent, which tells where the decimal point is located in the + number represented. + +For example, the number 1.25 has positive sign, a mantissa value of 1.01 +(in binary), and an exponent of 0 (the decimal point doesn’t need to be +shifted). The number 5 has the same sign and mantissa, but the exponent +is 2 because the mantissa is multiplied by 4 (2 to the power of the +exponent 2); 1.25 * 4 equals 5. + +Modern systems usually provide floating-point support that conforms to a +standard called IEEE 754. C’s ‘double’ type is usually implemented as a +64-bit IEEE 754 number, which uses 52 bits of space for the mantissa. +This means that numbers can only be specified to 52 bits of precision. +If you’re trying to represent numbers whose expansion repeats endlessly, +the expansion is cut off after 52 bits. Unfortunately, most software +needs to produce output in base 10, and common fractions in base 10 are +often repeating decimals in binary. For example, 1.1 decimal is binary +‘1.0001100110011 ...’; .1 = 1/16 + 1/32 + 1/256 plus an infinite number +of additional terms. IEEE 754 has to chop off that infinitely repeated +decimal after 52 digits, so the representation is slightly inaccurate. + +Sometimes you can see this inaccuracy when the number is printed: + + >>> 1.1 + 1.1000000000000001 + +The inaccuracy isn’t always visible when you print the number because +the FP-to-decimal-string conversion is provided by the C library, and +most C libraries try to produce sensible output. Even if it’s not +displayed, however, the inaccuracy is still there and subsequent +operations can magnify the error. + +For many applications this doesn’t matter. If I’m plotting points and +displaying them on my monitor, the difference between 1.1 and +1.1000000000000001 is too small to be visible. Reports often limit +output to a certain number of decimal places, and if you round the +number to two or three or even eight decimal places, the error is never +apparent. However, for applications where it does matter, it’s a lot of +work to implement your own custom arithmetic routines. + +Hence, the ‘Decimal’ type was created. + + +File: python.info, Node: The Decimal type, Next: The Context type, Prev: Why is Decimal needed?, Up: PEP 327 Decimal Data Type + +1.13.8.2 The ‘Decimal’ type +........................... + +A new module, *note decimal: 36, was added to Python’s standard library. +It contains two classes, ‘Decimal’ and ‘Context’. ‘Decimal’ instances +represent numbers, and ‘Context’ instances are used to wrap up various +settings such as the precision and default rounding mode. + +‘Decimal’ instances are immutable, like regular Python integers and FP +numbers; once it’s been created, you can’t change the value an instance +represents. ‘Decimal’ instances can be created from integers or +strings: + + >>> import decimal + >>> decimal.Decimal(1972) + Decimal("1972") + >>> decimal.Decimal("1.1") + Decimal("1.1") + +You can also provide tuples containing the sign, the mantissa +represented as a tuple of decimal digits, and the exponent: + + >>> decimal.Decimal((1, (1, 4, 7, 5), -2)) + Decimal("-14.75") + +Cautionary note: the sign bit is a Boolean value, so 0 is positive and 1 +is negative. + +Converting from floating-point numbers poses a bit of a problem: should +the FP number representing 1.1 turn into the decimal number for exactly +1.1, or for 1.1 plus whatever inaccuracies are introduced? The decision +was to dodge the issue and leave such a conversion out of the API. +Instead, you should convert the floating-point number into a string +using the desired precision and pass the string to the ‘Decimal’ +constructor: + + >>> f = 1.1 + >>> decimal.Decimal(str(f)) + Decimal("1.1") + >>> decimal.Decimal('%.12f' % f) + Decimal("1.100000000000") + +Once you have ‘Decimal’ instances, you can perform the usual +mathematical operations on them. One limitation: exponentiation +requires an integer exponent: + + >>> a = decimal.Decimal('35.72') + >>> b = decimal.Decimal('1.73') + >>> a+b + Decimal("37.45") + >>> a-b + Decimal("33.99") + >>> a*b + Decimal("61.7956") + >>> a/b + Decimal("20.64739884393063583815028902") + >>> a ** 2 + Decimal("1275.9184") + >>> a**b + Traceback (most recent call last): + ... + decimal.InvalidOperation: x ** (non-integer) + +You can combine ‘Decimal’ instances with integers, but not with +floating-point numbers: + + >>> a + 4 + Decimal("39.72") + >>> a + 4.5 + Traceback (most recent call last): + ... + TypeError: You can interact Decimal only with int, long or Decimal data types. + >>> + +‘Decimal’ numbers can be used with the *note math: b1. and *note cmath: +19. modules, but note that they’ll be immediately converted to +floating-point numbers before the operation is performed, resulting in a +possible loss of precision and accuracy. You’ll also get back a regular +floating-point number and not a ‘Decimal’. + + >>> import math, cmath + >>> d = decimal.Decimal('123456789012.345') + >>> math.sqrt(d) + 351364.18288201344 + >>> cmath.sqrt(-d) + 351364.18288201344j + +‘Decimal’ instances have a ‘sqrt()’ method that returns a ‘Decimal’, but +if you need other things such as trigonometric functions you’ll have to +implement them. + + >>> d.sqrt() + Decimal("351364.1828820134592177245001") + + +File: python.info, Node: The Context type, Prev: The Decimal type, Up: PEP 327 Decimal Data Type + +1.13.8.3 The ‘Context’ type +........................... + +Instances of the ‘Context’ class encapsulate several settings for +decimal operations: + + * ‘prec’ is the precision, the number of decimal places. + + * ‘rounding’ specifies the rounding mode. The *note decimal: 36. + module has constants for the various possibilities: ‘ROUND_DOWN’, + ‘ROUND_CEILING’, ‘ROUND_HALF_EVEN’, and various others. + + * ‘traps’ is a dictionary specifying what happens on encountering + certain error conditions: either an exception is raised or a value + is returned. Some examples of error conditions are division by + zero, loss of precision, and overflow. + +There’s a thread-local default context available by calling +‘getcontext()’; you can change the properties of this context to alter +the default precision, rounding, or trap handling. The following +example shows the effect of changing the precision of the default +context: + + >>> decimal.getcontext().prec + 28 + >>> decimal.Decimal(1) / decimal.Decimal(7) + Decimal("0.1428571428571428571428571429") + >>> decimal.getcontext().prec = 9 + >>> decimal.Decimal(1) / decimal.Decimal(7) + Decimal("0.142857143") + +The default action for error conditions is selectable; the module can +either return a special value such as infinity or not-a-number, or +exceptions can be raised: + + >>> decimal.Decimal(1) / decimal.Decimal(0) + Traceback (most recent call last): + ... + decimal.DivisionByZero: x / 0 + >>> decimal.getcontext().traps[decimal.DivisionByZero] = False + >>> decimal.Decimal(1) / decimal.Decimal(0) + Decimal("Infinity") + >>> + +The ‘Context’ instance also has various methods for formatting numbers +such as ‘to_eng_string()’ and ‘to_sci_string()’. + +For more information, see the documentation for the *note decimal: 36. +module, which includes a quick-start tutorial and a reference. + +See also +........ + +PEP 327(1) - Decimal Data Type + + Written by Facundo Batista and implemented by Facundo Batista, Eric + Price, Raymond Hettinger, Aahz, and Tim Peters. + +‘http://www.lahey.com/float.htm’ + + The article uses Fortran code to illustrate many of the problems + that floating-point inaccuracy can cause. + +‘http://speleotrove.com/decimal/’ + + A description of a decimal-based representation. This + representation is being proposed as a standard, and underlies the + new Python decimal type. Much of this material was written by Mike + Cowlishaw, designer of the Rexx language. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0327 + + +File: python.info, Node: PEP 328 Multi-line Imports, Next: PEP 331 Locale-Independent Float/String Conversions, Prev: PEP 327 Decimal Data Type, Up: What’s New in Python 2 4 + +1.13.9 PEP 328: Multi-line Imports +---------------------------------- + +One language change is a small syntactic tweak aimed at making it easier +to import many names from a module. In a ‘from module import names’ +statement, `names' is a sequence of names separated by commas. If the +sequence is very long, you can either write multiple imports from the +same module, or you can use backslashes to escape the line endings like +this: + + from SimpleXMLRPCServer import SimpleXMLRPCServer,\ + SimpleXMLRPCRequestHandler,\ + CGIXMLRPCRequestHandler,\ + resolve_dotted_attribute + +The syntactic change in Python 2.4 simply allows putting the names +within parentheses. Python ignores newlines within a parenthesized +expression, so the backslashes are no longer needed: + + from SimpleXMLRPCServer import (SimpleXMLRPCServer, + SimpleXMLRPCRequestHandler, + CGIXMLRPCRequestHandler, + resolve_dotted_attribute) + +The PEP also proposes that all *note import: c1d. statements be absolute +imports, with a leading ‘.’ character to indicate a relative import. +This part of the PEP was not implemented for Python 2.4, but was +completed for Python 2.5. + +See also +........ + +PEP 328(1) - Imports: Multi-Line and Absolute/Relative + + Written by Aahz. Multi-line imports were implemented by Dima + Dorfman. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0328 + + +File: python.info, Node: PEP 331 Locale-Independent Float/String Conversions, Next: Other Language Changes<12>, Prev: PEP 328 Multi-line Imports, Up: What’s New in Python 2 4 + +1.13.10 PEP 331: Locale-Independent Float/String Conversions +------------------------------------------------------------ + +The *note locale: a9. modules lets Python software select various +conversions and display conventions that are localized to a particular +country or language. However, the module was careful to not change the +numeric locale because various functions in Python’s implementation +required that the numeric locale remain set to the ‘'C'’ locale. Often +this was because the code was using the C library’s ‘atof()’ function. + +Not setting the numeric locale caused trouble for extensions that used +third-party C libraries, however, because they wouldn’t have the correct +locale set. The motivating example was GTK+, whose user interface +widgets weren’t displaying numbers in the current locale. + +The solution described in the PEP is to add three new functions to the +Python API that perform ASCII-only conversions, ignoring the locale +setting: + + * ‘PyOS_ascii_strtod(str, ptr)’ and ‘PyOS_ascii_atof(str, ptr)’ both + convert a string to a C ‘double’. + + * ‘PyOS_ascii_formatd(buffer, buf_len, format, d)’ converts a + ‘double’ to an ASCII string. + +The code for these functions came from the GLib library +(‘https://developer.gnome.org/glib/stable/’), whose developers kindly +relicensed the relevant functions and donated them to the Python +Software Foundation. The *note locale: a9. module can now change the +numeric locale, letting extensions such as GTK+ produce the correct +results. + +See also +........ + +PEP 331(1) - Locale-Independent Float/String Conversions + + Written by Christian R. Reis, and implemented by Gustavo Carneiro. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0331 + + +File: python.info, Node: Other Language Changes<12>, Next: New Improved and Deprecated Modules<3>, Prev: PEP 331 Locale-Independent Float/String Conversions, Up: What’s New in Python 2 4 + +1.13.11 Other Language Changes +------------------------------ + +Here are all of the changes that Python 2.4 makes to the core Python +language. + + * Decorators for functions and methods were added ( PEP 318(1)). + + * Built-in *note set(): b6f. and *note frozenset(): bee. types were + added ( PEP 218(2)). Other new built-ins include the + ‘reversed(seq)’ function ( PEP 322(3)). + + * Generator expressions were added ( PEP 289(4)). + + * Certain numeric expressions no longer return values restricted to + 32 or 64 bits ( PEP 237(5)). + + * You can now put parentheses around the list of names in a ‘from + module import names’ statement ( PEP 328(6)). + + * The *note dict.update(): dc9. method now accepts the same argument + forms as the *note dict: 1b8. constructor. This includes any + mapping, any iterable of key/value pairs, and keyword arguments. + (Contributed by Raymond Hettinger.) + + * The string methods ‘ljust()’, ‘rjust()’, and ‘center()’ now take an + optional argument for specifying a fill character other than a + space. (Contributed by Raymond Hettinger.) + + * Strings also gained an ‘rsplit()’ method that works like the + ‘split()’ method but splits from the end of the string. + (Contributed by Sean Reifschneider.) + + >>> 'www.python.org'.split('.', 1) + ['www', 'python.org'] + 'www.python.org'.rsplit('.', 1) + ['www.python', 'org'] + + * Three keyword parameters, `cmp', `key', and `reverse', were added + to the ‘sort()’ method of lists. These parameters make some common + usages of ‘sort()’ simpler. All of these parameters are optional. + + For the `cmp' parameter, the value should be a comparison function + that takes two parameters and returns -1, 0, or +1 depending on how + the parameters compare. This function will then be used to sort + the list. Previously this was the only parameter that could be + provided to ‘sort()’. + + `key' should be a single-parameter function that takes a list + element and returns a comparison key for the element. The list is + then sorted using the comparison keys. The following example sorts + a list case-insensitively: + + >>> L = ['A', 'b', 'c', 'D'] + >>> L.sort() # Case-sensitive sort + >>> L + ['A', 'D', 'b', 'c'] + >>> # Using 'key' parameter to sort list + >>> L.sort(key=lambda x: x.lower()) + >>> L + ['A', 'b', 'c', 'D'] + >>> # Old-fashioned way + >>> L.sort(cmp=lambda x,y: cmp(x.lower(), y.lower())) + >>> L + ['A', 'b', 'c', 'D'] + + The last example, which uses the `cmp' parameter, is the old way to + perform a case-insensitive sort. It works but is slower than using + a `key' parameter. Using `key' calls ‘lower()’ method once for + each element in the list while using `cmp' will call it twice for + each comparison, so using `key' saves on invocations of the + ‘lower()’ method. + + For simple key functions and comparison functions, it is often + possible to avoid a *note lambda: c2f. expression by using an + unbound method instead. For example, the above case-insensitive + sort is best written as: + + >>> L.sort(key=str.lower) + >>> L + ['A', 'b', 'c', 'D'] + + Finally, the `reverse' parameter takes a Boolean value. If the + value is true, the list will be sorted into reverse order. Instead + of ‘L.sort(); L.reverse()’, you can now write + ‘L.sort(reverse=True)’. + + The results of sorting are now guaranteed to be stable. This means + that two entries with equal keys will be returned in the same order + as they were input. For example, you can sort a list of people by + name, and then sort the list by age, resulting in a list sorted by + age where people with the same age are in name-sorted order. + + (All changes to ‘sort()’ contributed by Raymond Hettinger.) + + * There is a new built-in function ‘sorted(iterable)’ that works like + the in-place *note list.sort(): 445. method but can be used in + expressions. The differences are: + + * the input may be any iterable; + + * a newly formed copy is sorted, leaving the original intact; and + + * the expression returns the new sorted copy + + >>> L = [9,7,8,3,2,4,1,6,5] + >>> [10+i for i in sorted(L)] # usable in a list comprehension + [11, 12, 13, 14, 15, 16, 17, 18, 19] + >>> L # original is left unchanged + [9,7,8,3,2,4,1,6,5] + >>> sorted('Monty Python') # any iterable may be an input + [' ', 'M', 'P', 'h', 'n', 'n', 'o', 'o', 't', 't', 'y', 'y'] + + >>> # List the contents of a dict sorted by key values + >>> colormap = dict(red=1, blue=2, green=3, black=4, yellow=5) + >>> for k, v in sorted(colormap.iteritems()): + ... print k, v + ... + black 4 + blue 2 + green 3 + red 1 + yellow 5 + + (Contributed by Raymond Hettinger.) + + * Integer operations will no longer trigger an ‘OverflowWarning’. + The ‘OverflowWarning’ warning will disappear in Python 2.5. + + * The interpreter gained a new switch, *note -m: 337, that takes a + name, searches for the corresponding module on ‘sys.path’, and runs + the module as a script. For example, you can now run the Python + profiler with ‘python -m profile’. (Contributed by Nick Coghlan.) + + * The ‘eval(expr, globals, locals)’ and ‘execfile(filename, globals, + locals)’ functions and the ‘exec’ statement now accept any mapping + type for the `locals' parameter. Previously this had to be a + regular Python dictionary. (Contributed by Raymond Hettinger.) + + * The *note zip(): c32. built-in function and ‘itertools.izip()’ now + return an empty list if called with no arguments. Previously they + raised a *note TypeError: 192. exception. This makes them more + suitable for use with variable length argument lists: + + >>> def transpose(array): + ... return zip(*array) + ... + >>> transpose([(1,2,3), (4,5,6)]) + [(1, 4), (2, 5), (3, 6)] + >>> transpose([]) + [] + + (Contributed by Raymond Hettinger.) + + * Encountering a failure while importing a module no longer leaves a + partially-initialized module object in ‘sys.modules’. The + incomplete module object left behind would fool further imports of + the same module into succeeding, leading to confusing errors. + (Fixed by Tim Peters.) + + * *note None: 157. is now a constant; code that binds a new value to + the name ‘None’ is now a syntax error. (Contributed by Raymond + Hettinger.) + +* Menu: + +* Optimizations: Optimizations<11>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0318 + + (2) https://www.python.org/dev/peps/pep-0218 + + (3) https://www.python.org/dev/peps/pep-0322 + + (4) https://www.python.org/dev/peps/pep-0289 + + (5) https://www.python.org/dev/peps/pep-0237 + + (6) https://www.python.org/dev/peps/pep-0328 + + +File: python.info, Node: Optimizations<11>, Up: Other Language Changes<12> + +1.13.11.1 Optimizations +....................... + + * The inner loops for list and tuple slicing were optimized and now + run about one-third faster. The inner loops for dictionaries were + also optimized, resulting in performance boosts for ‘keys()’, + ‘values()’, ‘items()’, ‘iterkeys()’, ‘itervalues()’, and + ‘iteritems()’. (Contributed by Raymond Hettinger.) + + * The machinery for growing and shrinking lists was optimized for + speed and for space efficiency. Appending and popping from lists + now runs faster due to more efficient code paths and less frequent + use of the underlying system ‘realloc()’. List comprehensions also + benefit. ‘list.extend()’ was also optimized and no longer converts + its argument into a temporary list before extending the base list. + (Contributed by Raymond Hettinger.) + + * *note list(): 262, *note tuple(): 47e, *note map(): c2d, *note + filter(): c2e, and *note zip(): c32. now run several times faster + with non-sequence arguments that supply a *note __len__(): dcb. + method. (Contributed by Raymond Hettinger.) + + * The methods ‘list.__getitem__()’, ‘dict.__getitem__()’, and + ‘dict.__contains__()’ are now implemented as ‘method_descriptor’ + objects rather than ‘wrapper_descriptor’ objects. This form of + access doubles their performance and makes them more suitable for + use as arguments to functionals: ‘map(mydict.__getitem__, + keylist)’. (Contributed by Raymond Hettinger.) + + * Added a new opcode, ‘LIST_APPEND’, that simplifies the generated + bytecode for list comprehensions and speeds them up by about a + third. (Contributed by Raymond Hettinger.) + + * The peephole bytecode optimizer has been improved to produce + shorter, faster bytecode; remarkably, the resulting bytecode is + more readable. (Enhanced by Raymond Hettinger.) + + * String concatenations in statements of the form ‘s = s + "abc"’ and + ‘s += "abc"’ are now performed more efficiently in certain + circumstances. This optimization won’t be present in other Python + implementations such as Jython, so you shouldn’t rely on it; using + the ‘join()’ method of strings is still recommended when you want + to efficiently glue a large number of strings together. + (Contributed by Armin Rigo.) + +The net result of the 2.4 optimizations is that Python 2.4 runs the +pystone benchmark around 5% faster than Python 2.3 and 35% faster than +Python 2.2. (pystone is not a particularly good benchmark, but it’s the +most commonly used measurement of Python’s performance. Your own +applications may show greater or smaller benefits from Python 2.4.) + + +File: python.info, Node: New Improved and Deprecated Modules<3>, Next: Build and C API Changes<11>, Prev: Other Language Changes<12>, Up: What’s New in Python 2 4 + +1.13.12 New, Improved, and Deprecated Modules +--------------------------------------------- + +As usual, Python’s standard library received a number of enhancements +and bug fixes. Here’s a partial list of the most notable changes, +sorted alphabetically by module name. Consult the ‘Misc/NEWS’ file in +the source tree for a more complete list of changes, or look through the +CVS logs for all the details. + + * The *note asyncore: b. module’s ‘loop()’ function now has a `count' + parameter that lets you perform a limited number of passes through + the polling loop. The default is still to loop forever. + + * The *note base64: e. module now has more complete RFC 3548(1) + support for Base64, Base32, and Base16 encoding and decoding, + including optional case folding and optional alternative alphabets. + (Contributed by Barry Warsaw.) + + * The *note bisect: 12. module now has an underlying C implementation + for improved performance. (Contributed by Dmitry Vasiliev.) + + * The CJKCodecs collections of East Asian codecs, maintained by + Hye-Shik Chang, was integrated into 2.4. The new encodings are: + + * Chinese (PRC): gb2312, gbk, gb18030, big5hkscs, hz + + * Chinese (ROC): big5, cp950 + + * + Japanese: cp932, euc-jis-2004, euc-jp, euc-jisx0213, iso-2022-jp, + + iso-2022-jp-1, iso-2022-jp-2, iso-2022-jp-3, iso-2022-jp-ext, + iso-2022-jp-2004, shift-jis, shift-jisx0213, shift-jis-2004 + + * Korean: cp949, euc-kr, johab, iso-2022-kr + + * Some other new encodings were added: HP Roman8, ISO_8859-11, + ISO_8859-16, PCTP-154, and TIS-620. + + * The UTF-8 and UTF-16 codecs now cope better with receiving partial + input. Previously the ‘StreamReader’ class would try to read more + data, making it impossible to resume decoding from the stream. The + ‘read()’ method will now return as much data as it can and future + calls will resume decoding where previous ones left off. + (Implemented by Walter Dörwald.) + + * There is a new *note collections: 1e. module for various + specialized collection datatypes. Currently it contains just one + type, ‘deque’, a double-ended queue that supports efficiently + adding and removing elements from either end: + + >>> from collections import deque + >>> d = deque('ghi') # make a new deque with three items + >>> d.append('j') # add a new entry to the right side + >>> d.appendleft('f') # add a new entry to the left side + >>> d # show the representation of the deque + deque(['f', 'g', 'h', 'i', 'j']) + >>> d.pop() # return and remove the rightmost item + 'j' + >>> d.popleft() # return and remove the leftmost item + 'f' + >>> list(d) # list the contents of the deque + ['g', 'h', 'i'] + >>> 'h' in d # search the deque + True + + Several modules, such as the ‘Queue’ and *note threading: 109. + modules, now take advantage of *note collections.deque: 531. for + improved performance. (Contributed by Raymond Hettinger.) + + * The ‘ConfigParser’ classes have been enhanced slightly. The + ‘read()’ method now returns a list of the files that were + successfully parsed, and the *note set(): b6f. method raises *note + TypeError: 192. if passed a `value' argument that isn’t a string. + (Contributed by John Belmonte and David Goodger.) + + * The *note curses: 2c. module now supports the ncurses extension + ‘use_default_colors()’. On platforms where the terminal supports + transparency, this makes it possible to use a transparent + background. (Contributed by Jörg Lehmann.) + + * The *note difflib: 37. module now includes an ‘HtmlDiff’ class that + creates an HTML table showing a side by side comparison of two + versions of a text. (Contributed by Dan Gass.) + + * The *note email: 69. package was updated to version 3.0, which + dropped various deprecated APIs and removes support for Python + versions earlier than 2.3. The 3.0 version of the package uses a + new incremental parser for MIME messages, available in the + ‘email.FeedParser’ module. The new parser doesn’t require reading + the entire message into memory, and doesn’t raise exceptions if a + message is malformed; instead it records any problems in the + ‘defect’ attribute of the message. (Developed by Anthony Baxter, + Barry Warsaw, Thomas Wouters, and others.) + + * The *note heapq: 8e. module has been converted to C. The resulting + tenfold improvement in speed makes the module suitable for handling + high volumes of data. In addition, the module has two new + functions ‘nlargest()’ and ‘nsmallest()’ that use heaps to find the + N largest or smallest values in a dataset without the expense of a + full sort. (Contributed by Raymond Hettinger.) + + * The ‘httplib’ module now contains constants for HTTP status codes + defined in various HTTP-related RFC documents. Constants have + names such as ‘OK’, ‘CREATED’, ‘CONTINUE’, and ‘MOVED_PERMANENTLY’; + use pydoc to get a full list. (Contributed by Andrew Eland.) + + * The *note imaplib: 98. module now supports IMAP’s THREAD command + (contributed by Yves Dionne) and new ‘deleteacl()’ and ‘myrights()’ + methods (contributed by Arnaud Mazin). + + * The *note itertools: a3. module gained a ‘groupby(iterable[, + *func*])’ function. `iterable' is something that can be iterated + over to return a stream of elements, and the optional `func' + parameter is a function that takes an element and returns a key + value; if omitted, the key is simply the element itself. + ‘groupby()’ then groups the elements into subsequences which have + matching values of the key, and returns a series of 2-tuples + containing the key value and an iterator over the subsequence. + + Here’s an example to make this clearer. The `key' function simply + returns whether a number is even or odd, so the result of + ‘groupby()’ is to return consecutive runs of odd or even numbers. + + >>> import itertools + >>> L = [2, 4, 6, 7, 8, 9, 11, 12, 14] + >>> for key_val, it in itertools.groupby(L, lambda x: x % 2): + ... print key_val, list(it) + ... + 0 [2, 4, 6] + 1 [7] + 0 [8] + 1 [9, 11] + 0 [12, 14] + >>> + + ‘groupby()’ is typically used with sorted input. The logic for + ‘groupby()’ is similar to the Unix ‘uniq’ filter which makes it + handy for eliminating, counting, or identifying duplicate elements: + + >>> word = 'abracadabra' + >>> letters = sorted(word) # Turn string into a sorted list of letters + >>> letters + ['a', 'a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'r', 'r'] + >>> for k, g in itertools.groupby(letters): + ... print k, list(g) + ... + a ['a', 'a', 'a', 'a', 'a'] + b ['b', 'b'] + c ['c'] + d ['d'] + r ['r', 'r'] + >>> # List unique letters + >>> [k for k, g in groupby(letters)] + ['a', 'b', 'c', 'd', 'r'] + >>> # Count letter occurrences + >>> [(k, len(list(g))) for k, g in groupby(letters)] + [('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)] + + (Contributed by Hye-Shik Chang.) + + * *note itertools: a3. also gained a function named ‘tee(iterator, + N)’ that returns `N' independent iterators that replicate + `iterator'. If `N' is omitted, the default is 2. + + >>> L = [1,2,3] + >>> i1, i2 = itertools.tee(L) + >>> i1,i2 + (, ) + >>> list(i1) # Run the first iterator to exhaustion + [1, 2, 3] + >>> list(i2) # Run the second iterator to exhaustion + [1, 2, 3] + + Note that ‘tee()’ has to keep copies of the values returned by the + iterator; in the worst case, it may need to keep all of them. This + should therefore be used carefully if the leading iterator can run + far ahead of the trailing iterator in a long stream of inputs. If + the separation is large, then you might as well use *note list(): + 262. instead. When the iterators track closely with one another, + ‘tee()’ is ideal. Possible applications include bookmarking, + windowing, or lookahead iterators. (Contributed by Raymond + Hettinger.) + + * A number of functions were added to the *note locale: a9. module, + such as ‘bind_textdomain_codeset()’ to specify a particular + encoding and a family of ‘l*gettext()’ functions that return + messages in the chosen encoding. (Contributed by Gustavo + Niemeyer.) + + * Some keyword arguments were added to the *note logging: aa. + package’s ‘basicConfig()’ function to simplify log configuration. + The default behavior is to log messages to standard error, but + various keyword arguments can be specified to log to a particular + file, change the logging format, or set the logging level. For + example: + + import logging + logging.basicConfig(filename='/var/log/application.log', + level=0, # Log all messages + format='%(levelname):%(process):%(thread):%(message)') + + Other additions to the *note logging: aa. package include a + ‘log(level, msg)’ convenience method, as well as a + ‘TimedRotatingFileHandler’ class that rotates its log files at a + timed interval. The module already had ‘RotatingFileHandler’, + which rotated logs once the file exceeded a certain size. Both + classes derive from a new ‘BaseRotatingHandler’ class that can be + used to implement other rotating handlers. + + (Changes implemented by Vinay Sajip.) + + * The *note marshal: b0. module now shares interned strings on + unpacking a data structure. This may shrink the size of certain + pickle strings, but the primary effect is to make ‘.pyc’ files + significantly smaller. (Contributed by Martin von Löwis.) + + * The *note nntplib: c0. module’s ‘NNTP’ class gained ‘description()’ + and ‘descriptions()’ methods to retrieve newsgroup descriptions for + a single group or for a range of groups. (Contributed by Jürgen A. + Erhard.) + + * Two new functions were added to the *note operator: c2. module, + ‘attrgetter(attr)’ and ‘itemgetter(index)’. Both functions return + callables that take a single argument and return the corresponding + attribute or item; these callables make excellent data extractors + when used with *note map(): c2d. or *note sorted(): 444. For + example: + + >>> L = [('c', 2), ('d', 1), ('a', 4), ('b', 3)] + >>> map(operator.itemgetter(0), L) + ['c', 'd', 'a', 'b'] + >>> map(operator.itemgetter(1), L) + [2, 1, 4, 3] + >>> sorted(L, key=operator.itemgetter(1)) # Sort list by second tuple item + [('d', 1), ('c', 2), ('b', 3), ('a', 4)] + + (Contributed by Raymond Hettinger.) + + * The *note optparse: c3. module was updated in various ways. The + module now passes its messages through *note gettext.gettext(): + dcd, making it possible to internationalize Optik’s help and error + messages. Help messages for options can now include the string + ‘'%default'’, which will be replaced by the option’s default value. + (Contributed by Greg Ward.) + + * The long-term plan is to deprecate the ‘rfc822’ module in some + future Python release in favor of the *note email: 69. package. To + this end, the ‘email.Utils.formatdate()’ function has been changed + to make it usable as a replacement for ‘rfc822.formatdate()’. You + may want to write new e-mail processing code with this in mind. + (Change implemented by Anthony Baxter.) + + * A new ‘urandom(n)’ function was added to the *note os: c4. module, + returning a string containing `n' bytes of random data. This + function provides access to platform-specific sources of randomness + such as ‘/dev/urandom’ on Linux or the Windows CryptoAPI. + (Contributed by Trevor Perrin.) + + * Another new function: ‘os.path.lexists(path)’ returns true if the + file specified by `path' exists, whether or not it’s a symbolic + link. This differs from the existing ‘os.path.exists(path)’ + function, which returns false if `path' is a symlink that points to + a destination that doesn’t exist. (Contributed by Beni + Cherniavsky.) + + * A new ‘getsid()’ function was added to the *note posix: d1. module + that underlies the *note os: c4. module. (Contributed by J. + Raynor.) + + * The *note poplib: d0. module now supports POP over SSL. + (Contributed by Hector Urtubia.) + + * The *note profile: d3. module can now profile C extension + functions. (Contributed by Nick Bastin.) + + * The *note random: dc. module has a new method called + ‘getrandbits(N)’ that returns a long integer `N' bits in length. + The existing ‘randrange()’ method now uses ‘getrandbits()’ where + appropriate, making generation of arbitrarily large random numbers + more efficient. (Contributed by Raymond Hettinger.) + + * The regular expression language accepted by the *note re: dd. + module was extended with simple conditional expressions, written as + ‘(?(group)A|B)’. `group' is either a numeric group ID or a group + name defined with ‘(?P...)’ earlier in the expression. If + the specified group matched, the regular expression pattern `A' + will be tested against the string; if the group didn’t match, the + pattern `B' will be used instead. (Contributed by Gustavo + Niemeyer.) + + * The *note re: dd. module is also no longer recursive, thanks to a + massive amount of work by Gustavo Niemeyer. In a recursive regular + expression engine, certain patterns result in a large amount of C + stack space being consumed, and it was possible to overflow the + stack. For example, if you matched a 30000-byte string of ‘a’ + characters against the expression ‘(a|b)+’, one stack frame was + consumed per character. Python 2.3 tried to check for stack + overflow and raise a *note RuntimeError: 2ba. exception, but + certain patterns could sidestep the checking and if you were + unlucky Python could segfault. Python 2.4’s regular expression + engine can match this pattern without problems. + + * The *note signal: ea. module now performs tighter error-checking on + the parameters to the *note signal.signal(): a7d. function. For + example, you can’t set a handler on the ‘SIGKILL’ signal; previous + versions of Python would quietly accept this, but 2.4 will raise a + *note RuntimeError: 2ba. exception. + + * Two new functions were added to the *note socket: ef. module. + ‘socketpair()’ returns a pair of connected sockets and + ‘getservbyport(port)’ looks up the service name for a given port + number. (Contributed by Dave Cole and Barry Warsaw.) + + * The ‘sys.exitfunc()’ function has been deprecated. Code should be + using the existing *note atexit: c. module, which correctly handles + calling multiple exit functions. Eventually ‘sys.exitfunc()’ will + become a purely internal interface, accessed only by *note atexit: + c. + + * The *note tarfile: 101. module now generates GNU-format tar files + by default. (Contributed by Lars Gustäbel.) + + * The *note threading: 109. module now has an elegantly simple way to + support thread-local data. The module contains a ‘local’ class + whose attribute values are local to different threads. + + import threading + + data = threading.local() + data.number = 42 + data.url = ('www.python.org', 80) + + Other threads can assign and retrieve their own values for the + ‘number’ and ‘url’ attributes. You can subclass ‘local’ to + initialize attributes or to add methods. (Contributed by Jim + Fulton.) + + * The *note timeit: 10b. module now automatically disables periodic + garbage collection during the timing loop. This change makes + consecutive timings more comparable. (Contributed by Raymond + Hettinger.) + + * The *note weakref: 128. module now supports a wider variety of + objects including Python functions, class instances, sets, + frozensets, deques, arrays, files, sockets, and regular expression + pattern objects. (Contributed by Raymond Hettinger.) + + * The ‘xmlrpclib’ module now supports a multi-call extension for + transmitting multiple XML-RPC calls in a single HTTP operation. + (Contributed by Brian Quinlan.) + + * The ‘mpz’, ‘rotor’, and ‘xreadlines’ modules have been removed. + +* Menu: + +* cookielib:: +* doctest: doctest<3>. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc3548.html + + +File: python.info, Node: cookielib, Next: doctest<3>, Up: New Improved and Deprecated Modules<3> + +1.13.12.1 cookielib +................... + +The ‘cookielib’ library supports client-side handling for HTTP cookies, +mirroring the ‘Cookie’ module’s server-side cookie support. Cookies are +stored in cookie jars; the library transparently stores cookies offered +by the web server in the cookie jar, and fetches the cookie from the jar +when connecting to the server. As in web browsers, policy objects +control whether cookies are accepted or not. + +In order to store cookies across sessions, two implementations of cookie +jars are provided: one that stores cookies in the Netscape format so +applications can use the Mozilla or Lynx cookie files, and one that +stores cookies in the same format as the Perl libwww library. + +‘urllib2’ has been changed to interact with ‘cookielib’: +‘HTTPCookieProcessor’ manages a cookie jar that is used when accessing +URLs. + +This module was contributed by John J. Lee. + + +File: python.info, Node: doctest<3>, Prev: cookielib, Up: New Improved and Deprecated Modules<3> + +1.13.12.2 doctest +................. + +The *note doctest: 67. module underwent considerable refactoring thanks +to Edward Loper and Tim Peters. Testing can still be as simple as +running *note doctest.testmod(): dd0, but the refactorings allow +customizing the module’s operation in various ways + +The new ‘DocTestFinder’ class extracts the tests from a given object’s +docstrings: + + def f (x, y): + """>>> f(2,2) + 4 + >>> f(3,2) + 6 + """ + return x*y + + finder = doctest.DocTestFinder() + + # Get list of DocTest instances + tests = finder.find(f) + +The new ‘DocTestRunner’ class then runs individual tests and can produce +a summary of the results: + + runner = doctest.DocTestRunner() + for t in tests: + tried, failed = runner.run(t) + + runner.summarize(verbose=1) + +The above example produces the following output: + + 1 items passed all tests: + 2 tests in f + 2 tests in 1 items. + 2 passed and 0 failed. + Test passed. + +‘DocTestRunner’ uses an instance of the ‘OutputChecker’ class to compare +the expected output with the actual output. This class takes a number +of different flags that customize its behaviour; ambitious users can +also write a completely new subclass of ‘OutputChecker’. + +The default output checker provides a number of handy features. For +example, with the *note doctest.ELLIPSIS: dd1. option flag, an ellipsis +(‘...’) in the expected output matches any substring, making it easier +to accommodate outputs that vary in minor ways: + + def o (n): + """>>> o(1) + <__main__.C instance at 0x...> + >>> + """ + +Another special string, ‘’, matches a blank line: + + def p (n): + """>>> p(1) + + >>> + """ + +Another new capability is producing a diff-style display of the output +by specifying the *note doctest.REPORT_UDIFF: dd2. (unified diffs), +*note doctest.REPORT_CDIFF: dd3. (context diffs), or *note +doctest.REPORT_NDIFF: dd4. (delta-style) option flags. For example: + + def g (n): + """>>> g(4) + here + is + a + lengthy + >>>""" + L = 'here is a rather lengthy list of words'.split() + for word in L[:n]: + print word + +Running the above function’s tests with *note doctest.REPORT_UDIFF: dd2. +specified, you get the following output: + + ********************************************************************** + File "t.py", line 15, in g + Failed example: + g(4) + Differences (unified diff with -expected +actual): + @@ -2,3 +2,3 @@ + is + a + -lengthy + +rather + ********************************************************************** + + +File: python.info, Node: Build and C API Changes<11>, Next: Porting to Python 2 4, Prev: New Improved and Deprecated Modules<3>, Up: What’s New in Python 2 4 + +1.13.13 Build and C API Changes +------------------------------- + +Some of the changes to Python’s build process and to the C API are: + + * Three new convenience macros were added for common return values + from extension functions: *note Py_RETURN_NONE: dd6, *note + Py_RETURN_TRUE: dd7, and *note Py_RETURN_FALSE: dd8. (Contributed + by Brett Cannon.) + + * Another new macro, ‘Py_CLEAR(obj)’, decreases the reference count + of `obj' and sets `obj' to the null pointer. (Contributed by Jim + Fulton.) + + * A new function, ‘PyTuple_Pack(N, obj1, obj2, ..., objN)’, + constructs tuples from a variable length argument list of Python + objects. (Contributed by Raymond Hettinger.) + + * A new function, ‘PyDict_Contains(d, k)’, implements fast dictionary + lookups without masking exceptions raised during the look-up + process. (Contributed by Raymond Hettinger.) + + * The ‘Py_IS_NAN(X)’ macro returns 1 if its float or double argument + `X' is a NaN. (Contributed by Tim Peters.) + + * C code can avoid unnecessary locking by using the new *note + PyEval_ThreadsInitialized(): dd9. function to tell if any thread + operations have been performed. If this function returns false, no + lock operations are needed. (Contributed by Nick Coghlan.) + + * A new function, *note PyArg_VaParseTupleAndKeywords(): dda, is the + same as *note PyArg_ParseTupleAndKeywords(): 5ca. but takes a + ‘va_list’ instead of a number of arguments. (Contributed by Greg + Chapman.) + + * A new method flag, ‘METH_COEXISTS’, allows a function defined in + slots to co-exist with a *note PyCFunction: ddb. having the same + name. This can halve the access time for a method such as + ‘set.__contains__()’. (Contributed by Raymond Hettinger.) + + * Python can now be built with additional profiling for the + interpreter itself, intended as an aid to people developing the + Python core. Providing ‘--enable-profiling’ to the ‘configure’ + script will let you profile the interpreter with ‘gprof’, and + providing the ‘--with-tsc’ switch enables profiling using the + Pentium’s Time-Stamp-Counter register. Note that the ‘--with-tsc’ + switch is slightly misnamed, because the profiling feature also + works on the PowerPC platform, though that processor architecture + doesn’t call that register “the TSC register”. (Contributed by + Jeremy Hylton.) + + * The ‘tracebackobject’ type has been renamed to ‘PyTracebackObject’. + +* Menu: + +* Port-Specific Changes: Port-Specific Changes<2>. + + +File: python.info, Node: Port-Specific Changes<2>, Up: Build and C API Changes<11> + +1.13.13.1 Port-Specific Changes +............................... + + * The Windows port now builds under MSVC++ 7.1 as well as version 6. + (Contributed by Martin von Löwis.) + + +File: python.info, Node: Porting to Python 2 4, Next: Acknowledgements<4>, Prev: Build and C API Changes<11>, Up: What’s New in Python 2 4 + +1.13.14 Porting to Python 2.4 +----------------------------- + +This section lists previously described changes that may require changes +to your code: + + * Left shifts and hexadecimal/octal constants that are too large no + longer trigger a *note FutureWarning: 325. and return a value + limited to 32 or 64 bits; instead they return a long integer. + + * Integer operations will no longer trigger an ‘OverflowWarning’. + The ‘OverflowWarning’ warning will disappear in Python 2.5. + + * The *note zip(): c32. built-in function and ‘itertools.izip()’ now + return an empty list instead of raising a *note TypeError: 192. + exception if called with no arguments. + + * You can no longer compare the ‘date’ and *note datetime: 194. + instances provided by the *note datetime: 31. module. Two + instances of different classes will now always be unequal, and + relative comparisons (‘<’, ‘>’) will raise a *note TypeError: 192. + + * ‘dircache.listdir()’ now passes exceptions to the caller instead of + returning empty lists. + + * ‘LexicalHandler.startDTD()’ used to receive the public and system + IDs in the wrong order. This has been corrected; applications + relying on the wrong order need to be fixed. + + * *note fcntl.ioctl(): dde. now warns if the `mutate' argument is + omitted and relevant. + + * The *note tarfile: 101. module now generates GNU-format tar files + by default. + + * Encountering a failure while importing a module no longer leaves a + partially-initialized module object in ‘sys.modules’. + + * *note None: 157. is now a constant; code that binds a new value to + the name ‘None’ is now a syntax error. + + * The ‘signals.signal()’ function now raises a *note RuntimeError: + 2ba. exception for certain illegal values; previously these errors + would pass silently. For example, you can no longer set a handler + on the ‘SIGKILL’ signal. + + +File: python.info, Node: Acknowledgements<4>, Prev: Porting to Python 2 4, Up: What’s New in Python 2 4 + +1.13.15 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Koray Can, Hye-Shik Chang, Michael Dyck, Raymond Hettinger, +Brian Hurt, Hamish Lawson, Fredrik Lundh, Sean Reifschneider, Sadruddin +Rejeb. + + +File: python.info, Node: What’s New in Python 2 3, Next: What’s New in Python 2 2, Prev: What’s New in Python 2 4, Up: What’s New in Python + +1.14 What’s New in Python 2.3 +============================= + + +Author: A.M. Kuchling + +This article explains the new features in Python 2.3. Python 2.3 was +released on July 29, 2003. + +The main themes for Python 2.3 are polishing some of the features added +in 2.2, adding various small but useful enhancements to the core +language, and expanding the standard library. The new object model +introduced in the previous version has benefited from 18 months of +bugfixes and from optimization efforts that have improved the +performance of new-style classes. A few new built-in functions have +been added such as *note sum(): 1e5. and *note enumerate(): de3. The +*note in: 7a3. operator can now be used for substring searches (e.g. +‘"ab" in "abc"’ returns *note True: 499.). + +Some of the many new library features include Boolean, set, heap, and +date/time data types, the ability to import modules from ZIP-format +archives, metadata support for the long-awaited Python catalog, an +updated version of IDLE, and modules for logging messages, wrapping +text, parsing CSV files, processing command-line options, using +BerkeleyDB databases… the list of new and enhanced modules is lengthy. + +This article doesn’t attempt to provide a complete specification of the +new features, but instead provides a convenient overview. For full +details, you should refer to the documentation for Python 2.3, such as +the Python Library Reference and the Python Reference Manual. If you +want to understand the complete implementation and design rationale, +refer to the PEP for a particular new feature. + +* Menu: + +* PEP 218; A Standard Set Datatype: PEP 218 A Standard Set Datatype. +* PEP 255; Simple Generators: PEP 255 Simple Generators. +* PEP 263; Source Code Encodings: PEP 263 Source Code Encodings. +* PEP 273; Importing Modules from ZIP Archives: PEP 273 Importing Modules from ZIP Archives. +* PEP 277; Unicode file name support for Windows NT: PEP 277 Unicode file name support for Windows NT. +* PEP 278; Universal Newline Support: PEP 278 Universal Newline Support. +* PEP 279; enumerate(): PEP 279 enumerate. +* PEP 282; The logging Package: PEP 282 The logging Package. +* PEP 285; A Boolean Type: PEP 285 A Boolean Type. +* PEP 293; Codec Error Handling Callbacks: PEP 293 Codec Error Handling Callbacks. +* PEP 301; Package Index and Metadata for Distutils: PEP 301 Package Index and Metadata for Distutils. +* PEP 302; New Import Hooks: PEP 302 New Import Hooks. +* PEP 305; Comma-separated Files: PEP 305 Comma-separated Files. +* PEP 307; Pickle Enhancements: PEP 307 Pickle Enhancements. +* Extended Slices:: +* Other Language Changes: Other Language Changes<13>. +* New, Improved, and Deprecated Modules: New Improved and Deprecated Modules<4>. +* Pymalloc; A Specialized Object Allocator: Pymalloc A Specialized Object Allocator. +* Build and C API Changes: Build and C API Changes<12>. +* Other Changes and Fixes: Other Changes and Fixes<2>. +* Porting to Python 2.3: Porting to Python 2 3. +* Acknowledgements: Acknowledgements<5>. + + +File: python.info, Node: PEP 218 A Standard Set Datatype, Next: PEP 255 Simple Generators, Up: What’s New in Python 2 3 + +1.14.1 PEP 218: A Standard Set Datatype +--------------------------------------- + +The new ‘sets’ module contains an implementation of a set datatype. The +‘Set’ class is for mutable sets, sets that can have members added and +removed. The ‘ImmutableSet’ class is for sets that can’t be modified, +and instances of ‘ImmutableSet’ can therefore be used as dictionary +keys. Sets are built on top of dictionaries, so the elements within a +set must be hashable. + +Here’s a simple example: + + >>> import sets + >>> S = sets.Set([1,2,3]) + >>> S + Set([1, 2, 3]) + >>> 1 in S + True + >>> 0 in S + False + >>> S.add(5) + >>> S.remove(3) + >>> S + Set([1, 2, 5]) + >>> + +The union and intersection of sets can be computed with the ‘union()’ +and ‘intersection()’ methods; an alternative notation uses the bitwise +operators ‘&’ and ‘|’. Mutable sets also have in-place versions of +these methods, ‘union_update()’ and ‘intersection_update()’. + + >>> S1 = sets.Set([1,2,3]) + >>> S2 = sets.Set([4,5,6]) + >>> S1.union(S2) + Set([1, 2, 3, 4, 5, 6]) + >>> S1 | S2 # Alternative notation + Set([1, 2, 3, 4, 5, 6]) + >>> S1.intersection(S2) + Set([]) + >>> S1 & S2 # Alternative notation + Set([]) + >>> S1.union_update(S2) + >>> S1 + Set([1, 2, 3, 4, 5, 6]) + >>> + +It’s also possible to take the symmetric difference of two sets. This +is the set of all elements in the union that aren’t in the intersection. +Another way of putting it is that the symmetric difference contains all +elements that are in exactly one set. Again, there’s an alternative +notation (‘^’), and an in-place version with the ungainly name +‘symmetric_difference_update()’. + + >>> S1 = sets.Set([1,2,3,4]) + >>> S2 = sets.Set([3,4,5,6]) + >>> S1.symmetric_difference(S2) + Set([1, 2, 5, 6]) + >>> S1 ^ S2 + Set([1, 2, 5, 6]) + >>> + +There are also ‘issubset()’ and ‘issuperset()’ methods for checking +whether one set is a subset or superset of another: + + >>> S1 = sets.Set([1,2,3]) + >>> S2 = sets.Set([2,3]) + >>> S2.issubset(S1) + True + >>> S1.issubset(S2) + False + >>> S1.issuperset(S2) + True + >>> + +See also +........ + +PEP 218(1) - Adding a Built-In Set Object Type + + PEP written by Greg V. Wilson. Implemented by Greg V. Wilson, Alex + Martelli, and GvR. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0218 + + +File: python.info, Node: PEP 255 Simple Generators, Next: PEP 263 Source Code Encodings, Prev: PEP 218 A Standard Set Datatype, Up: What’s New in Python 2 3 + +1.14.2 PEP 255: Simple Generators +--------------------------------- + +In Python 2.2, generators were added as an optional feature, to be +enabled by a ‘from __future__ import generators’ directive. In 2.3 +generators no longer need to be specially enabled, and are now always +present; this means that *note yield: 18f. is now always a keyword. The +rest of this section is a copy of the description of generators from the +“What’s New in Python 2.2” document; if you read it back when Python 2.2 +came out, you can skip the rest of this section. + +You’re doubtless familiar with how function calls work in Python or C. +When you call a function, it gets a private namespace where its local +variables are created. When the function reaches a *note return: 190. +statement, the local variables are destroyed and the resulting value is +returned to the caller. A later call to the same function will get a +fresh new set of local variables. But, what if the local variables +weren’t thrown away on exiting a function? What if you could later +resume the function where it left off? This is what generators provide; +they can be thought of as resumable functions. + +Here’s the simplest example of a generator function: + + def generate_ints(N): + for i in range(N): + yield i + +A new keyword, *note yield: 18f, was introduced for generators. Any +function containing a ‘yield’ statement is a generator function; this is +detected by Python’s bytecode compiler which compiles the function +specially as a result. + +When you call a generator function, it doesn’t return a single value; +instead it returns a generator object that supports the iterator +protocol. On executing the *note yield: 18f. statement, the generator +outputs the value of ‘i’, similar to a *note return: 190. statement. +The big difference between ‘yield’ and a ‘return’ statement is that on +reaching a ‘yield’ the generator’s state of execution is suspended and +local variables are preserved. On the next call to the generator’s +‘.next()’ method, the function will resume executing immediately after +the ‘yield’ statement. (For complicated reasons, the ‘yield’ statement +isn’t allowed inside the *note try: d72. block of a ‘try’…‘finally’ +statement; read PEP 255(1) for a full explanation of the interaction +between ‘yield’ and exceptions.) + +Here’s a sample usage of the ‘generate_ints()’ generator: + + >>> gen = generate_ints(3) + >>> gen + + >>> gen.next() + 0 + >>> gen.next() + 1 + >>> gen.next() + 2 + >>> gen.next() + Traceback (most recent call last): + File "stdin", line 1, in ? + File "stdin", line 2, in generate_ints + StopIteration + +You could equally write ‘for i in generate_ints(5)’, or ‘a,b,c = +generate_ints(3)’. + +Inside a generator function, the *note return: 190. statement can only +be used without a value, and signals the end of the procession of +values; afterwards the generator cannot return any further values. +‘return’ with a value, such as ‘return 5’, is a syntax error inside a +generator function. The end of the generator’s results can also be +indicated by raising *note StopIteration: 486. manually, or by just +letting the flow of execution fall off the bottom of the function. + +You could achieve the effect of generators manually by writing your own +class and storing all the local variables of the generator as instance +variables. For example, returning a list of integers could be done by +setting ‘self.count’ to 0, and having the *note next(): 682. method +increment ‘self.count’ and return it. However, for a moderately +complicated generator, writing a corresponding class would be much +messier. ‘Lib/test/test_generators.py’ contains a number of more +interesting examples. The simplest one implements an in-order traversal +of a tree using generators recursively. + + # A recursive generator that generates Tree leaves in in-order. + def inorder(t): + if t: + for x in inorder(t.left): + yield x + yield t.label + for x in inorder(t.right): + yield x + +Two other examples in ‘Lib/test/test_generators.py’ produce solutions +for the N-Queens problem (placing $N$ queens on an $NxN$ chess board so +that no queen threatens another) and the Knight’s Tour (a route that +takes a knight to every square of an $NxN$ chessboard without visiting +any square twice). + +The idea of generators comes from other programming languages, +especially Icon (‘https://www.cs.arizona.edu/icon/’), where the idea of +generators is central. In Icon, every expression and function call +behaves like a generator. One example from “An Overview of the Icon +Programming Language” at +‘https://www.cs.arizona.edu/icon/docs/ipd266.htm’ gives an idea of what +this looks like: + + sentence := "Store it in the neighboring harbor" + if (i := find("or", sentence)) > 5 then write(i) + +In Icon the ‘find()’ function returns the indexes at which the substring +“or” is found: 3, 23, 33. In the *note if: de7. statement, ‘i’ is first +assigned a value of 3, but 3 is less than 5, so the comparison fails, +and Icon retries it with the second value of 23. 23 is greater than 5, +so the comparison now succeeds, and the code prints the value 23 to the +screen. + +Python doesn’t go nearly as far as Icon in adopting generators as a +central concept. Generators are considered part of the core Python +language, but learning or using them isn’t compulsory; if they don’t +solve any problems that you have, feel free to ignore them. One novel +feature of Python’s interface as compared to Icon’s is that a +generator’s state is represented as a concrete object (the iterator) +that can be passed around to other functions or stored in a data +structure. + +See also +........ + +PEP 255(2) - Simple Generators + + Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland. + Implemented mostly by Neil Schemenauer and Tim Peters, with other + fixes from the Python Labs crew. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0255 + + (2) https://www.python.org/dev/peps/pep-0255 + + +File: python.info, Node: PEP 263 Source Code Encodings, Next: PEP 273 Importing Modules from ZIP Archives, Prev: PEP 255 Simple Generators, Up: What’s New in Python 2 3 + +1.14.3 PEP 263: Source Code Encodings +------------------------------------- + +Python source files can now be declared as being in different character +set encodings. Encodings are declared by including a specially +formatted comment in the first or second line of the source file. For +example, a UTF-8 file can be declared with: + + #!/usr/bin/env python + # -*- coding: UTF-8 -*- + +Without such an encoding declaration, the default encoding used is 7-bit +ASCII. Executing or importing modules that contain string literals with +8-bit characters and have no encoding declaration will result in a *note +DeprecationWarning: 278. being signalled by Python 2.3; in 2.4 this will +be a syntax error. + +The encoding declaration only affects Unicode string literals, which +will be converted to Unicode using the specified encoding. Note that +Python identifiers are still restricted to ASCII characters, so you +can’t have variable names that use characters outside of the usual +alphanumerics. + +See also +........ + +PEP 263(1) - Defining Python Source Code Encodings + + Written by Marc-André Lemburg and Martin von Löwis; implemented by + Suzuki Hisao and Martin von Löwis. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0263 + + +File: python.info, Node: PEP 273 Importing Modules from ZIP Archives, Next: PEP 277 Unicode file name support for Windows NT, Prev: PEP 263 Source Code Encodings, Up: What’s New in Python 2 3 + +1.14.4 PEP 273: Importing Modules from ZIP Archives +--------------------------------------------------- + +The new *note zipimport: 143. module adds support for importing modules +from a ZIP-format archive. You don’t need to import the module +explicitly; it will be automatically imported if a ZIP archive’s +filename is added to ‘sys.path’. For example: + + amk@nyman:~/src/python$ unzip -l /tmp/example.zip + Archive: /tmp/example.zip + Length Date Time Name + -------- ---- ---- ---- + 8467 11-26-02 22:30 jwzthreading.py + -------- ------- + 8467 1 file + amk@nyman:~/src/python$ ./python + Python 2.3 (#1, Aug 1 2003, 19:54:32) + >>> import sys + >>> sys.path.insert(0, '/tmp/example.zip') # Add .zip file to front of path + >>> import jwzthreading + >>> jwzthreading.__file__ + '/tmp/example.zip/jwzthreading.py' + >>> + +An entry in ‘sys.path’ can now be the filename of a ZIP archive. The +ZIP archive can contain any kind of files, but only files named ‘*.py’, +‘*.pyc’, or ‘*.pyo’ can be imported. If an archive only contains ‘*.py’ +files, Python will not attempt to modify the archive by adding the +corresponding ‘*.pyc’ file, meaning that if a ZIP archive doesn’t +contain ‘*.pyc’ files, importing may be rather slow. + +A path within the archive can also be specified to only import from a +subdirectory; for example, the path ‘/tmp/example.zip/lib/’ would only +import from the ‘lib/’ subdirectory within the archive. + +See also +........ + +PEP 273(1) - Import Modules from Zip Archives + + Written by James C. Ahlstrom, who also provided an implementation. + Python 2.3 follows the specification in PEP 273(2), but uses an + implementation written by Just van Rossum that uses the import + hooks described in PEP 302(3). See section *note PEP 302; New + Import Hooks: deb. for a description of the new import hooks. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0273 + + (2) https://www.python.org/dev/peps/pep-0273 + + (3) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: PEP 277 Unicode file name support for Windows NT, Next: PEP 278 Universal Newline Support, Prev: PEP 273 Importing Modules from ZIP Archives, Up: What’s New in Python 2 3 + +1.14.5 PEP 277: Unicode file name support for Windows NT +-------------------------------------------------------- + +On Windows NT, 2000, and XP, the system stores file names as Unicode +strings. Traditionally, Python has represented file names as byte +strings, which is inadequate because it renders some file names +inaccessible. + +Python now allows using arbitrary Unicode strings (within the +limitations of the file system) for all functions that expect file +names, most notably the *note open(): 4f0. built-in function. If a +Unicode string is passed to *note os.listdir(): a41, Python now returns +a list of Unicode strings. A new function, ‘os.getcwdu()’, returns the +current directory as a Unicode string. + +Byte strings still work as file names, and on Windows Python will +transparently convert them to Unicode using the ‘mbcs’ encoding. + +Other systems also allow Unicode strings as file names but convert them +to byte strings before passing them to the system, which can cause a +*note UnicodeError: c3b. to be raised. Applications can test whether +arbitrary Unicode strings are supported as file names by checking *note +os.path.supports_unicode_filenames: ded, a Boolean value. + +Under MacOS, *note os.listdir(): a41. may now return Unicode filenames. + +See also +........ + +PEP 277(1) - Unicode file name support for Windows NT + + Written by Neil Hodgson; implemented by Neil Hodgson, Martin von + Löwis, and Mark Hammond. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0277 + + +File: python.info, Node: PEP 278 Universal Newline Support, Next: PEP 279 enumerate, Prev: PEP 277 Unicode file name support for Windows NT, Up: What’s New in Python 2 3 + +1.14.6 PEP 278: Universal Newline Support +----------------------------------------- + +The three major operating systems used today are Microsoft Windows, +Apple’s Macintosh OS, and the various Unix derivatives. A minor +irritation of cross-platform work is that these three platforms all use +different characters to mark the ends of lines in text files. Unix uses +the linefeed (ASCII character 10), MacOS uses the carriage return (ASCII +character 13), and Windows uses a two-character sequence of a carriage +return plus a newline. + +Python’s file objects can now support end of line conventions other than +the one followed by the platform on which Python is running. Opening a +file with the mode ‘'U'’ or ‘'rU'’ will open a file for reading in *note +universal newlines: 5f4. mode. All three line ending conventions will +be translated to a ‘'\n'’ in the strings returned by the various file +methods such as ‘read()’ and *note readline(): de. + +Universal newline support is also used when importing modules and when +executing a file with the ‘execfile()’ function. This means that Python +modules can be shared between all three operating systems without +needing to convert the line-endings. + +This feature can be disabled when compiling Python by specifying the +‘--without-universal-newlines’ switch when running Python’s ‘configure’ +script. + +See also +........ + +PEP 278(1) - Universal Newline Support + + Written and implemented by Jack Jansen. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0278 + + +File: python.info, Node: PEP 279 enumerate, Next: PEP 282 The logging Package, Prev: PEP 278 Universal Newline Support, Up: What’s New in Python 2 3 + +1.14.7 PEP 279: enumerate() +--------------------------- + +A new built-in function, *note enumerate(): de3, will make certain loops +a bit clearer. ‘enumerate(thing)’, where `thing' is either an iterator +or a sequence, returns an iterator that will return ‘(0, thing[0])’, +‘(1, thing[1])’, ‘(2, thing[2])’, and so forth. + +A common idiom to change every element of a list looks like this: + + for i in range(len(L)): + item = L[i] + # ... compute some result based on item ... + L[i] = result + +This can be rewritten using *note enumerate(): de3. as: + + for i, item in enumerate(L): + # ... compute some result based on item ... + L[i] = result + +See also +........ + +PEP 279(1) - The enumerate() built-in function + + Written and implemented by Raymond D. Hettinger. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0279 + + +File: python.info, Node: PEP 282 The logging Package, Next: PEP 285 A Boolean Type, Prev: PEP 279 enumerate, Up: What’s New in Python 2 3 + +1.14.8 PEP 282: The logging Package +----------------------------------- + +A standard package for writing logs, *note logging: aa, has been added +to Python 2.3. It provides a powerful and flexible mechanism for +generating logging output which can then be filtered and processed in +various ways. A configuration file written in a standard format can be +used to control the logging behavior of a program. Python includes +handlers that will write log records to standard error or to a file or +socket, send them to the system log, or even e-mail them to a particular +address; of course, it’s also possible to write your own handler +classes. + +The ‘Logger’ class is the primary class. Most application code will +deal with one or more ‘Logger’ objects, each one used by a particular +subsystem of the application. Each ‘Logger’ is identified by a name, +and names are organized into a hierarchy using ‘.’ as the component +separator. For example, you might have ‘Logger’ instances named +‘server’, ‘server.auth’ and ‘server.network’. The latter two instances +are below ‘server’ in the hierarchy. This means that if you turn up the +verbosity for ‘server’ or direct ‘server’ messages to a different +handler, the changes will also apply to records logged to ‘server.auth’ +and ‘server.network’. There’s also a root ‘Logger’ that’s the parent of +all other loggers. + +For simple uses, the *note logging: aa. package contains some +convenience functions that always use the root log: + + import logging + + logging.debug('Debugging information') + logging.info('Informational message') + logging.warning('Warning:config file %s not found', 'server.conf') + logging.error('Error occurred') + logging.critical('Critical error -- shutting down') + +This produces the following output: + + WARNING:root:Warning:config file server.conf not found + ERROR:root:Error occurred + CRITICAL:root:Critical error -- shutting down + +In the default configuration, informational and debugging messages are +suppressed and the output is sent to standard error. You can enable the +display of informational and debugging messages by calling the +‘setLevel()’ method on the root logger. + +Notice the ‘warning()’ call’s use of string formatting operators; all of +the functions for logging messages take the arguments ‘(msg, arg1, arg2, +...)’ and log the string resulting from ‘msg % (arg1, arg2, ...)’. + +There’s also an ‘exception()’ function that records the most recent +traceback. Any of the other functions will also record the traceback if +you specify a true value for the keyword argument `exc_info'. + + def f(): + try: 1/0 + except: logging.exception('Problem recorded') + + f() + +This produces the following output: + + ERROR:root:Problem recorded + Traceback (most recent call last): + File "t.py", line 6, in f + 1/0 + ZeroDivisionError: integer division or modulo by zero + +Slightly more advanced programs will use a logger other than the root +logger. The ‘getLogger(name)’ function is used to get a particular log, +creating it if it doesn’t exist yet. ‘getLogger(None)’ returns the root +logger. + + log = logging.getLogger('server') + ... + log.info('Listening on port %i', port) + ... + log.critical('Disk full') + ... + +Log records are usually propagated up the hierarchy, so a message logged +to ‘server.auth’ is also seen by ‘server’ and ‘root’, but a ‘Logger’ can +prevent this by setting its ‘propagate’ attribute to *note False: 388. + +There are more classes provided by the *note logging: aa. package that +can be customized. When a ‘Logger’ instance is told to log a message, +it creates a ‘LogRecord’ instance that is sent to any number of +different ‘Handler’ instances. Loggers and handlers can also have an +attached list of filters, and each filter can cause the ‘LogRecord’ to +be ignored or can modify the record before passing it along. When +they’re finally output, ‘LogRecord’ instances are converted to text by a +‘Formatter’ class. All of these classes can be replaced by your own +specially-written classes. + +With all of these features the *note logging: aa. package should provide +enough flexibility for even the most complicated applications. This is +only an incomplete overview of its features, so please see the package’s +reference documentation for all of the details. Reading PEP 282(1) will +also be helpful. + +See also +........ + +PEP 282(2) - A Logging System + + Written by Vinay Sajip and Trent Mick; implemented by Vinay Sajip. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0282 + + (2) https://www.python.org/dev/peps/pep-0282 + + +File: python.info, Node: PEP 285 A Boolean Type, Next: PEP 293 Codec Error Handling Callbacks, Prev: PEP 282 The logging Package, Up: What’s New in Python 2 3 + +1.14.9 PEP 285: A Boolean Type +------------------------------ + +A Boolean type was added to Python 2.3. Two new constants were added to +the ‘__builtin__’ module, *note True: 499. and *note False: 388. (*note +True: 499. and *note False: 388. constants were added to the built-ins +in Python 2.2.1, but the 2.2.1 versions are simply set to integer values +of 1 and 0 and aren’t a different type.) + +The type object for this new type is named *note bool: 183.; the +constructor for it takes any Python value and converts it to *note True: +499. or *note False: 388. + + >>> bool(1) + True + >>> bool(0) + False + >>> bool([]) + False + >>> bool( (1,) ) + True + +Most of the standard library modules and built-in functions have been +changed to return Booleans. + + >>> obj = [] + >>> hasattr(obj, 'append') + True + >>> isinstance(obj, list) + True + >>> isinstance(obj, tuple) + False + +Python’s Booleans were added with the primary goal of making code +clearer. For example, if you’re reading a function and encounter the +statement ‘return 1’, you might wonder whether the ‘1’ represents a +Boolean truth value, an index, or a coefficient that multiplies some +other quantity. If the statement is ‘return True’, however, the meaning +of the return value is quite clear. + +Python’s Booleans were `not' added for the sake of strict type-checking. +A very strict language such as Pascal would also prevent you performing +arithmetic with Booleans, and would require that the expression in an +*note if: de7. statement always evaluate to a Boolean result. Python is +not this strict and never will be, as PEP 285(1) explicitly says. This +means you can still use any expression in an ‘if’ statement, even ones +that evaluate to a list or tuple or some random object. The Boolean +type is a subclass of the *note int: 184. class so that arithmetic using +a Boolean still works. + + >>> True + 1 + 2 + >>> False + 1 + 1 + >>> False * 75 + 0 + >>> True * 75 + 75 + +To sum up *note True: 499. and *note False: 388. in a sentence: they’re +alternative ways to spell the integer values 1 and 0, with the single +difference that *note str(): 330. and *note repr(): 7cc. return the +strings ‘'True'’ and ‘'False'’ instead of ‘'1'’ and ‘'0'’. + +See also +........ + +PEP 285(2) - Adding a bool type + + Written and implemented by GvR. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0285 + + (2) https://www.python.org/dev/peps/pep-0285 + + +File: python.info, Node: PEP 293 Codec Error Handling Callbacks, Next: PEP 301 Package Index and Metadata for Distutils, Prev: PEP 285 A Boolean Type, Up: What’s New in Python 2 3 + +1.14.10 PEP 293: Codec Error Handling Callbacks +----------------------------------------------- + +When encoding a Unicode string into a byte string, unencodable +characters may be encountered. So far, Python has allowed specifying +the error processing as either “strict” (raising *note UnicodeError: +c3b.), “ignore” (skipping the character), or “replace” (using a question +mark in the output string), with “strict” being the default behavior. +It may be desirable to specify alternative processing of such errors, +such as inserting an XML character reference or HTML entity reference +into the converted string. + +Python now has a flexible framework to add different processing +strategies. New error handlers can be added with *note +codecs.register_error(): df5, and codecs then can access the error +handler with *note codecs.lookup_error(): df6. An equivalent C API has +been added for codecs written in C. The error handler gets the necessary +state information such as the string being converted, the position in +the string where the error was detected, and the target encoding. The +handler can then either raise an exception or return a replacement +string. + +Two additional error handlers have been implemented using this +framework: “backslashreplace” uses Python backslash quoting to represent +unencodable characters and “xmlcharrefreplace” emits XML character +references. + +See also +........ + +PEP 293(1) - Codec Error Handling Callbacks + + Written and implemented by Walter Dörwald. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0293 + + +File: python.info, Node: PEP 301 Package Index and Metadata for Distutils, Next: PEP 302 New Import Hooks, Prev: PEP 293 Codec Error Handling Callbacks, Up: What’s New in Python 2 3 + +1.14.11 PEP 301: Package Index and Metadata for Distutils +--------------------------------------------------------- + +Support for the long-requested Python catalog makes its first appearance +in 2.3. + +The heart of the catalog is the new Distutils ‘register’ command. +Running ‘python setup.py register’ will collect the metadata describing +a package, such as its name, version, maintainer, description, &c., and +send it to a central catalog server. The resulting catalog is available +from ‘https://pypi.org’. + +To make the catalog a bit more useful, a new optional `classifiers' +keyword argument has been added to the Distutils ‘setup()’ function. A +list of Trove(1)-style strings can be supplied to help classify the +software. + +Here’s an example ‘setup.py’ with classifiers, written to be compatible +with older versions of the Distutils: + + from distutils import core + kw = {'name': "Quixote", + 'version': "0.5.1", + 'description': "A highly Pythonic Web application framework", + # ... + } + + if (hasattr(core, 'setup_keywords') and + 'classifiers' in core.setup_keywords): + kw['classifiers'] = \ + ['Topic :: Internet :: WWW/HTTP :: Dynamic Content', + 'Environment :: No Input/Output (Daemon)', + 'Intended Audience :: Developers'], + + core.setup(**kw) + +The full list of classifiers can be obtained by running ‘python setup.py +register --list-classifiers’. + +See also +........ + +PEP 301(2) - Package Index and Metadata for Distutils + + Written and implemented by Richard Jones. + + ---------- Footnotes ---------- + + (1) http://catb.org/~esr/trove/ + + (2) https://www.python.org/dev/peps/pep-0301 + + +File: python.info, Node: PEP 302 New Import Hooks, Next: PEP 305 Comma-separated Files, Prev: PEP 301 Package Index and Metadata for Distutils, Up: What’s New in Python 2 3 + +1.14.12 PEP 302: New Import Hooks +--------------------------------- + +While it’s been possible to write custom import hooks ever since the +‘ihooks’ module was introduced in Python 1.3, no one has ever been +really happy with it because writing new import hooks is difficult and +messy. There have been various proposed alternatives such as the +‘imputil’ and ‘iu’ modules, but none of them has ever gained much +acceptance, and none of them were easily usable from C code. + +PEP 302(1) borrows ideas from its predecessors, especially from Gordon +McMillan’s ‘iu’ module. Three new items are added to the *note sys: fd. +module: + + * ‘sys.path_hooks’ is a list of callable objects; most often they’ll + be classes. Each callable takes a string containing a path and + either returns an importer object that will handle imports from + this path or raises an *note ImportError: 334. exception if it + can’t handle this path. + + * ‘sys.path_importer_cache’ caches importer objects for each path, so + ‘sys.path_hooks’ will only need to be traversed once for each path. + + * ‘sys.meta_path’ is a list of importer objects that will be + traversed before ‘sys.path’ is checked. This list is initially + empty, but user code can add objects to it. Additional built-in + and frozen modules can be imported by an object added to this list. + +Importer objects must have a single method, ‘find_module(fullname, +path=None)’. `fullname' will be a module or package name, e.g. +‘string’ or ‘distutils.core’. ‘find_module()’ must return a loader +object that has a single method, ‘load_module(fullname)’, that creates +and returns the corresponding module object. + +Pseudo-code for Python’s new import logic, therefore, looks something +like this (simplified a bit; see PEP 302(2) for the full details): + + for mp in sys.meta_path: + loader = mp(fullname) + if loader is not None: + = loader.load_module(fullname) + + for path in sys.path: + for hook in sys.path_hooks: + try: + importer = hook(path) + except ImportError: + # ImportError, so try the other path hooks + pass + else: + loader = importer.find_module(fullname) + = loader.load_module(fullname) + + # Not found! + raise ImportError + +See also +........ + +PEP 302(3) - New Import Hooks + + Written by Just van Rossum and Paul Moore. Implemented by Just van + Rossum. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0302 + + (2) https://www.python.org/dev/peps/pep-0302 + + (3) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: PEP 305 Comma-separated Files, Next: PEP 307 Pickle Enhancements, Prev: PEP 302 New Import Hooks, Up: What’s New in Python 2 3 + +1.14.13 PEP 305: Comma-separated Files +-------------------------------------- + +Comma-separated files are a format frequently used for exporting data +from databases and spreadsheets. Python 2.3 adds a parser for +comma-separated files. + +Comma-separated format is deceptively simple at first glance: + + Costs,150,200,3.95 + +Read a line and call ‘line.split(',')’: what could be simpler? But toss +in string data that can contain commas, and things get more complicated: + + "Costs",150,200,3.95,"Includes taxes, shipping, and sundry items" + +A big ugly regular expression can parse this, but using the new *note +csv: 2a. package is much simpler: + + import csv + + input = open('datafile', 'rb') + reader = csv.reader(input) + for line in reader: + print line + +The ‘reader()’ function takes a number of different options. The field +separator isn’t limited to the comma and can be changed to any +character, and so can the quoting and line-ending characters. + +Different dialects of comma-separated files can be defined and +registered; currently there are two dialects, both used by Microsoft +Excel. A separate *note csv.writer: dfc. class will generate +comma-separated files from a succession of tuples or lists, quoting +strings that contain the delimiter. + +See also +........ + +PEP 305(1) - CSV File API + + Written and implemented by Kevin Altis, Dave Cole, Andrew McNamara, + Skip Montanaro, Cliff Wells. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0305 + + +File: python.info, Node: PEP 307 Pickle Enhancements, Next: Extended Slices, Prev: PEP 305 Comma-separated Files, Up: What’s New in Python 2 3 + +1.14.14 PEP 307: Pickle Enhancements +------------------------------------ + +The *note pickle: ca. and ‘cPickle’ modules received some attention +during the 2.3 development cycle. In 2.2, new-style classes could be +pickled without difficulty, but they weren’t pickled very compactly; PEP +307(1) quotes a trivial example where a new-style class results in a +pickled string three times longer than that for a classic class. + +The solution was to invent a new pickle protocol. The *note +pickle.dumps(): dff. function has supported a text-or-binary flag for a +long time. In 2.3, this flag is redefined from a Boolean to an integer: +0 is the old text-mode pickle format, 1 is the old binary format, and +now 2 is a new 2.3-specific format. A new constant, *note +pickle.HIGHEST_PROTOCOL: e00, can be used to select the fanciest +protocol available. + +Unpickling is no longer considered a safe operation. 2.2’s *note +pickle: ca. provided hooks for trying to prevent unsafe classes from +being unpickled (specifically, a ‘__safe_for_unpickling__’ attribute), +but none of this code was ever audited and therefore it’s all been +ripped out in 2.3. You should not unpickle untrusted data in any +version of Python. + +To reduce the pickling overhead for new-style classes, a new interface +for customizing pickling was added using three special methods: *note +__getstate__(): e01, *note __setstate__(): 19c, and *note +__getnewargs__(): 6ae. Consult PEP 307(2) for the full semantics of +these methods. + +As a way to compress pickles yet further, it’s now possible to use +integer codes instead of long strings to identify pickled classes. The +Python Software Foundation will maintain a list of standardized codes; +there’s also a range of codes for private use. Currently no codes have +been specified. + +See also +........ + +PEP 307(3) - Extensions to the pickle protocol + + Written and implemented by Guido van Rossum and Tim Peters. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0307 + + (2) https://www.python.org/dev/peps/pep-0307 + + (3) https://www.python.org/dev/peps/pep-0307 + + +File: python.info, Node: Extended Slices, Next: Other Language Changes<13>, Prev: PEP 307 Pickle Enhancements, Up: What’s New in Python 2 3 + +1.14.15 Extended Slices +----------------------- + +Ever since Python 1.4, the slicing syntax has supported an optional +third “step” or “stride” argument. For example, these are all legal +Python syntax: ‘L[1:10:2]’, ‘L[:-1:1]’, ‘L[::-1]’. This was added to +Python at the request of the developers of Numerical Python, which uses +the third argument extensively. However, Python’s built-in list, tuple, +and string sequence types have never supported this feature, raising a +*note TypeError: 192. if you tried it. Michael Hudson contributed a +patch to fix this shortcoming. + +For example, you can now easily extract the elements of a list that have +even indexes: + + >>> L = range(10) + >>> L[::2] + [0, 2, 4, 6, 8] + +Negative values also work to make a copy of the same list in reverse +order: + + >>> L[::-1] + [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] + +This also works for tuples, arrays, and strings: + + >>> s='abcd' + >>> s[::2] + 'ac' + >>> s[::-1] + 'dcba' + +If you have a mutable sequence such as a list or an array you can assign +to or delete an extended slice, but there are some differences between +assignment to extended and regular slices. Assignment to a regular +slice can be used to change the length of the sequence: + + >>> a = range(3) + >>> a + [0, 1, 2] + >>> a[1:3] = [4, 5, 6] + >>> a + [0, 4, 5, 6] + +Extended slices aren’t this flexible. When assigning to an extended +slice, the list on the right hand side of the statement must contain the +same number of items as the slice it is replacing: + + >>> a = range(4) + >>> a + [0, 1, 2, 3] + >>> a[::2] + [0, 2] + >>> a[::2] = [0, -1] + >>> a + [0, 1, -1, 3] + >>> a[::2] = [0,1,2] + Traceback (most recent call last): + File "", line 1, in ? + ValueError: attempt to assign sequence of size 3 to extended slice of size 2 + +Deletion is more straightforward: + + >>> a = range(4) + >>> a + [0, 1, 2, 3] + >>> a[::2] + [0, 2] + >>> del a[::2] + >>> a + [1, 3] + +One can also now pass slice objects to the *note __getitem__(): 27c. +methods of the built-in sequences: + + >>> range(10).__getitem__(slice(0, 5, 2)) + [0, 2, 4] + +Or use slice objects directly in subscripts: + + >>> range(10)[slice(0, 5, 2)] + [0, 2, 4] + +To simplify implementing sequences that support extended slicing, slice +objects now have a method ‘indices(length)’ which, given the length of a +sequence, returns a ‘(start, stop, step)’ tuple that can be passed +directly to *note range(): 9be. ‘indices()’ handles omitted and +out-of-bounds indices in a manner consistent with regular slices (and +this innocuous phrase hides a welter of confusing details!). The method +is intended to be used like this: + + class FakeSeq: + ... + def calc_item(self, i): + ... + def __getitem__(self, item): + if isinstance(item, slice): + indices = item.indices(len(self)) + return FakeSeq([self.calc_item(i) for i in range(*indices)]) + else: + return self.calc_item(i) + +From this example you can also see that the built-in *note slice: e04. +object is now the type object for the slice type, and is no longer a +function. This is consistent with Python 2.2, where *note int: 184, +*note str: 330, etc., underwent the same change. + + +File: python.info, Node: Other Language Changes<13>, Next: New Improved and Deprecated Modules<4>, Prev: Extended Slices, Up: What’s New in Python 2 3 + +1.14.16 Other Language Changes +------------------------------ + +Here are all of the changes that Python 2.3 makes to the core Python +language. + + * The *note yield: 18f. statement is now always a keyword, as + described in section *note PEP 255; Simple Generators: de6. of this + document. + + * A new built-in function *note enumerate(): de3. was added, as + described in section *note PEP 279; enumerate(): df0. of this + document. + + * Two new constants, *note True: 499. and *note False: 388. were + added along with the built-in *note bool: 183. type, as described + in section *note PEP 285; A Boolean Type: df3. of this document. + + * The *note int(): 184. type constructor will now return a long + integer instead of raising an *note OverflowError: 960. when a + string or floating-point number is too large to fit into an + integer. This can lead to the paradoxical result that + ‘isinstance(int(expression), int)’ is false, but that seems + unlikely to cause problems in practice. + + * Built-in types now support the extended slicing syntax, as + described in section *note Extended Slices: e03. of this document. + + * A new built-in function, ‘sum(iterable, start=0)’, adds up the + numeric items in the iterable object and returns their sum. *note + sum(): 1e5. only accepts numbers, meaning that you can’t use it to + concatenate a bunch of strings. (Contributed by Alex Martelli.) + + * ‘list.insert(pos, value)’ used to insert `value' at the front of + the list when `pos' was negative. The behaviour has now been + changed to be consistent with slice indexing, so when `pos' is -1 + the value will be inserted before the last element, and so forth. + + * ‘list.index(value)’, which searches for `value' within the list and + returns its index, now takes optional `start' and `stop' arguments + to limit the search to only part of the list. + + * Dictionaries have a new method, ‘pop(key[, *default*])’, that + returns the value corresponding to `key' and removes that key/value + pair from the dictionary. If the requested key isn’t present in + the dictionary, `default' is returned if it’s specified and *note + KeyError: 2c7. raised if it isn’t. + + >>> d = {1:2} + >>> d + {1: 2} + >>> d.pop(4) + Traceback (most recent call last): + File "stdin", line 1, in ? + KeyError: 4 + >>> d.pop(1) + 2 + >>> d.pop(1) + Traceback (most recent call last): + File "stdin", line 1, in ? + KeyError: 'pop(): dictionary is empty' + >>> d + {} + >>> + + There’s also a new class method, ‘dict.fromkeys(iterable, value)’, + that creates a dictionary with keys taken from the supplied + iterator `iterable' and all values set to `value', defaulting to + ‘None’. + + (Patches contributed by Raymond Hettinger.) + + Also, the *note dict(): 1b8. constructor now accepts keyword + arguments to simplify creating small dictionaries: + + >>> dict(red=1, blue=2, green=3, black=4) + {'blue': 2, 'black': 4, 'green': 3, 'red': 1} + + (Contributed by Just van Rossum.) + + * The *note assert: e06. statement no longer checks the ‘__debug__’ + flag, so you can no longer disable assertions by assigning to + ‘__debug__’. Running Python with the *note -O: 68b. switch will + still generate code that doesn’t execute any assertions. + + * Most type objects are now callable, so you can use them to create + new objects such as functions, classes, and modules. (This means + that the ‘new’ module can be deprecated in a future Python version, + because you can now use the type objects available in the *note + types: 118. module.) For example, you can create a new module + object with the following code: + + >>> import types + >>> m = types.ModuleType('abc','docstring') + >>> m + + >>> m.__doc__ + 'docstring' + + * A new warning, *note PendingDeprecationWarning: 279. was added to + indicate features which are in the process of being deprecated. + The warning will `not' be printed by default. To check for use of + features that will be deprecated in the future, supply *note + -Walways;;PendingDeprecationWarning;;: 417. on the command line or + use *note warnings.filterwarnings(): e07. + + * The process of deprecating string-based exceptions, as in ‘raise + "Error occurred"’, has begun. Raising a string will now trigger + *note PendingDeprecationWarning: 279. + + * Using ‘None’ as a variable name will now result in a *note + SyntaxWarning: 191. warning. In a future version of Python, ‘None’ + may finally become a keyword. + + * The ‘xreadlines()’ method of file objects, introduced in Python + 2.1, is no longer necessary because files now behave as their own + iterator. ‘xreadlines()’ was originally introduced as a faster way + to loop over all the lines in a file, but now you can simply write + ‘for line in file_obj’. File objects also have a new read-only + ‘encoding’ attribute that gives the encoding used by the file; + Unicode strings written to the file will be automatically converted + to bytes using the given encoding. + + * The method resolution order used by new-style classes has changed, + though you’ll only notice the difference if you have a really + complicated inheritance hierarchy. Classic classes are unaffected + by this change. Python 2.2 originally used a topological sort of a + class’s ancestors, but 2.3 now uses the C3 algorithm as described + in the paper "A Monotonic Superclass Linearization for Dylan"(1). + To understand the motivation for this change, read Michele + Simionato’s article "Python 2.3 Method Resolution Order"(2), or + read the thread on python-dev starting with the message at + ‘https://mail.python.org/pipermail/python-dev/2002-October/029035.html’. + Samuele Pedroni first pointed out the problem and also implemented + the fix by coding the C3 algorithm. + + * Python runs multithreaded programs by switching between threads + after executing N bytecodes. The default value for N has been + increased from 10 to 100 bytecodes, speeding up single-threaded + applications by reducing the switching overhead. Some + multithreaded applications may suffer slower response time, but + that’s easily fixed by setting the limit back to a lower number + using ‘sys.setcheckinterval(N)’. The limit can be retrieved with + the new *note sys.getcheckinterval(): e08. function. + + * One minor but far-reaching change is that the names of extension + types defined by the modules included with Python now contain the + module and a ‘'.'’ in front of the type name. For example, in + Python 2.2, if you created a socket and printed its ‘__class__’, + you’d get this output: + + >>> s = socket.socket() + >>> s.__class__ + + + In 2.3, you get this: + + >>> s.__class__ + + + * One of the noted incompatibilities between old- and new-style + classes has been removed: you can now assign to the *note __name__: + c67. and *note __bases__: e09. attributes of new-style classes. + There are some restrictions on what can be assigned to *note + __bases__: e09. along the lines of those relating to assigning to + an instance’s *note __class__: e0a. attribute. + +* Menu: + +* String Changes:: +* Optimizations: Optimizations<12>. + + ---------- Footnotes ---------- + + (1) http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.3910 + + (2) http://www.phyast.pitt.edu/~micheles/mro.html + + +File: python.info, Node: String Changes, Next: Optimizations<12>, Up: Other Language Changes<13> + +1.14.16.1 String Changes +........................ + + * The *note in: 7a3. operator now works differently for strings. + Previously, when evaluating ‘X in Y’ where `X' and `Y' are strings, + `X' could only be a single character. That’s now changed; `X' can + be a string of any length, and ‘X in Y’ will return *note True: + 499. if `X' is a substring of `Y'. If `X' is the empty string, the + result is always *note True: 499. + + >>> 'ab' in 'abcd' + True + >>> 'ad' in 'abcd' + False + >>> '' in 'abcd' + True + + Note that this doesn’t tell you where the substring starts; if you + need that information, use the ‘find()’ string method. + + * The ‘strip()’, ‘lstrip()’, and ‘rstrip()’ string methods now have + an optional argument for specifying the characters to strip. The + default is still to remove all whitespace characters: + + >>> ' abc '.strip() + 'abc' + >>> '><><><>'.strip('<>') + 'abc' + >>> '><><><>\n'.strip('<>') + 'abc<><><>\n' + >>> u'\u4000\u4001abc\u4000'.strip(u'\u4000') + u'\u4001abc' + >>> + + (Suggested by Simon Brunning and implemented by Walter Dörwald.) + + * The ‘startswith()’ and ‘endswith()’ string methods now accept + negative numbers for the `start' and `end' parameters. + + * Another new string method is ‘zfill()’, originally a function in + the *note string: f6. module. ‘zfill()’ pads a numeric string with + zeros on the left until it’s the specified width. Note that the + ‘%’ operator is still more flexible and powerful than ‘zfill()’. + + >>> '45'.zfill(4) + '0045' + >>> '12345'.zfill(4) + '12345' + >>> 'goofy'.zfill(6) + '0goofy' + + (Contributed by Walter Dörwald.) + + * A new type object, ‘basestring’, has been added. Both 8-bit + strings and Unicode strings inherit from this type, so + ‘isinstance(obj, basestring)’ will return *note True: 499. for + either kind of string. It’s a completely abstract type, so you + can’t create ‘basestring’ instances. + + * Interned strings are no longer immortal and will now be + garbage-collected in the usual way when the only reference to them + is from the internal dictionary of interned strings. (Implemented + by Oren Tirosh.) + + +File: python.info, Node: Optimizations<12>, Prev: String Changes, Up: Other Language Changes<13> + +1.14.16.2 Optimizations +....................... + + * The creation of new-style class instances has been made much + faster; they’re now faster than classic classes! + + * The ‘sort()’ method of list objects has been extensively rewritten + by Tim Peters, and the implementation is significantly faster. + + * Multiplication of large long integers is now much faster thanks to + an implementation of Karatsuba multiplication, an algorithm that + scales better than the O(n*n) required for the grade-school + multiplication algorithm. (Original patch by Christopher A. Craig, + and significantly reworked by Tim Peters.) + + * The ‘SET_LINENO’ opcode is now gone. This may provide a small + speed increase, depending on your compiler’s idiosyncrasies. See + section *note Other Changes and Fixes: e0d. for a longer + explanation. (Removed by Michael Hudson.) + + * ‘xrange()’ objects now have their own iterator, making ‘for i in + xrange(n)’ slightly faster than ‘for i in range(n)’. (Patch by + Raymond Hettinger.) + + * A number of small rearrangements have been made in various hotspots + to improve performance, such as inlining a function or removing + some code. (Implemented mostly by GvR, but lots of people have + contributed single changes.) + +The net result of the 2.3 optimizations is that Python 2.3 runs the +pystone benchmark around 25% faster than Python 2.2. + + +File: python.info, Node: New Improved and Deprecated Modules<4>, Next: Pymalloc A Specialized Object Allocator, Prev: Other Language Changes<13>, Up: What’s New in Python 2 3 + +1.14.17 New, Improved, and Deprecated Modules +--------------------------------------------- + +As usual, Python’s standard library received a number of enhancements +and bug fixes. Here’s a partial list of the most notable changes, +sorted alphabetically by module name. Consult the ‘Misc/NEWS’ file in +the source tree for a more complete list of changes, or look through the +CVS logs for all the details. + + * The *note array: 7. module now supports arrays of Unicode + characters using the ‘'u'’ format character. Arrays also now + support using the ‘+=’ assignment operator to add another array’s + contents, and the ‘*=’ assignment operator to repeat an array. + (Contributed by Jason Orendorff.) + + * The ‘bsddb’ module has been replaced by version 4.1.6 of the + PyBSDDB(1) package, providing a more complete interface to the + transactional features of the BerkeleyDB library. + + The old version of the module has been renamed to ‘bsddb185’ and is + no longer built automatically; you’ll have to edit ‘Modules/Setup’ + to enable it. Note that the new ‘bsddb’ package is intended to be + compatible with the old module, so be sure to file bugs if you + discover any incompatibilities. When upgrading to Python 2.3, if + the new interpreter is compiled with a new version of the + underlying BerkeleyDB library, you will almost certainly have to + convert your database files to the new version. You can do this + fairly easily with the new scripts ‘db2pickle.py’ and + ‘pickle2db.py’ which you will find in the distribution’s + ‘Tools/scripts’ directory. If you’ve already been using the + PyBSDDB package and importing it as ‘bsddb3’, you will have to + change your ‘import’ statements to import it as ‘bsddb’. + + * The new *note bz2: 14. module is an interface to the bz2 data + compression library. bz2-compressed data is usually smaller than + corresponding *note zlib: 144.-compressed data. (Contributed by + Gustavo Niemeyer.) + + * A set of standard date/time types has been added in the new *note + datetime: 31. module. See the following section for more details. + + * The Distutils ‘Extension’ class now supports an extra constructor + argument named `depends' for listing additional source files that + an extension depends on. This lets Distutils recompile the module + if any of the dependency files are modified. For example, if + ‘sampmodule.c’ includes the header file ‘sample.h’, you would + create the ‘Extension’ object like this: + + ext = Extension("samp", + sources=["sampmodule.c"], + depends=["sample.h"]) + + Modifying ‘sample.h’ would then cause the module to be recompiled. + (Contributed by Jeremy Hylton.) + + * Other minor changes to Distutils: it now checks for the ‘CC’, + ‘CFLAGS’, ‘CPP’, ‘LDFLAGS’, and ‘CPPFLAGS’ environment variables, + using them to override the settings in Python’s configuration + (contributed by Robert Weber). + + * Previously the *note doctest: 67. module would only search the + docstrings of public methods and functions for test cases, but it + now also examines private ones as well. The ‘DocTestSuite()’ + function creates a *note unittest.TestSuite: 6ce. object from a set + of *note doctest: 67. tests. + + * The new ‘gc.get_referents(object)’ function returns a list of all + the objects referenced by `object'. + + * The *note getopt: 87. module gained a new function, ‘gnu_getopt()’, + that supports the same arguments as the existing *note getopt(): + 87. function but uses GNU-style scanning mode. The existing *note + getopt(): 87. stops processing options as soon as a non-option + argument is encountered, but in GNU-style mode processing + continues, meaning that options and arguments can be mixed. For + example: + + >>> getopt.getopt(['-f', 'filename', 'output', '-v'], 'f:v') + ([('-f', 'filename')], ['output', '-v']) + >>> getopt.gnu_getopt(['-f', 'filename', 'output', '-v'], 'f:v') + ([('-f', 'filename'), ('-v', '')], ['output']) + + (Contributed by Peter Åstrand.) + + * The *note grp: 8b, *note pwd: d6, and *note resource: e0. modules + now return enhanced tuples: + + >>> import grp + >>> g = grp.getgrnam('amk') + >>> g.gr_name, g.gr_gid + ('amk', 500) + + * The *note gzip: 8c. module can now handle files exceeding 2 GiB. + + * The new *note heapq: 8e. module contains an implementation of a + heap queue algorithm. A heap is an array-like data structure that + keeps items in a partially sorted order such that, for every index + `k', ‘heap[k] <= heap[2*k+1]’ and ‘heap[k] <= heap[2*k+2]’. This + makes it quick to remove the smallest item, and inserting a new + item while maintaining the heap property is O(lg n). (See + ‘https://xlinux.nist.gov/dads//HTML/priorityque.html’ for more + information about the priority queue data structure.) + + The *note heapq: 8e. module provides ‘heappush()’ and ‘heappop()’ + functions for adding and removing items while maintaining the heap + property on top of some other mutable Python sequence type. Here’s + an example that uses a Python list: + + >>> import heapq + >>> heap = [] + >>> for item in [3, 7, 5, 11, 1]: + ... heapq.heappush(heap, item) + ... + >>> heap + [1, 3, 5, 11, 7] + >>> heapq.heappop(heap) + 1 + >>> heapq.heappop(heap) + 3 + >>> heap + [5, 7, 11] + + (Contributed by Kevin O’Connor.) + + * The IDLE integrated development environment has been updated using + the code from the IDLEfork project + (‘http://idlefork.sourceforge.net’). The most notable feature is + that the code being developed is now executed in a subprocess, + meaning that there’s no longer any need for manual ‘reload()’ + operations. IDLE’s core code has been incorporated into the + standard library as the ‘idlelib’ package. + + * The *note imaplib: 98. module now supports IMAP over SSL. + (Contributed by Piers Lauder and Tino Lange.) + + * The *note itertools: a3. contains a number of useful functions for + use with iterators, inspired by various functions provided by the + ML and Haskell languages. For example, + ‘itertools.ifilter(predicate, iterator)’ returns all elements in + the iterator for which the function ‘predicate()’ returns *note + True: 499, and ‘itertools.repeat(obj, N)’ returns ‘obj’ `N' times. + There are a number of other functions in the module; see the + package’s reference documentation for details. (Contributed by + Raymond Hettinger.) + + * Two new functions in the *note math: b1. module, ‘degrees(rads)’ + and ‘radians(degs)’, convert between radians and degrees. Other + functions in the *note math: b1. module such as *note math.sin(): + e0f. and *note math.cos(): e10. have always required input values + measured in radians. Also, an optional `base' argument was added + to *note math.log(): e11. to make it easier to compute logarithms + for bases other than ‘e’ and ‘10’. (Contributed by Raymond + Hettinger.) + + * Several new POSIX functions (‘getpgid()’, ‘killpg()’, ‘lchown()’, + ‘loadavg()’, ‘major()’, ‘makedev()’, ‘minor()’, and ‘mknod()’) were + added to the *note posix: d1. module that underlies the *note os: + c4. module. (Contributed by Gustavo Niemeyer, Geert Jansen, and + Denis S. Otkidach.) + + * In the *note os: c4. module, the ‘*stat()’ family of functions can + now report fractions of a second in a timestamp. Such time stamps + are represented as floats, similar to the value returned by *note + time.time(): 31d. + + During testing, it was found that some applications will break if + time stamps are floats. For compatibility, when using the tuple + interface of the ‘stat_result’ time stamps will be represented as + integers. When using named fields (a feature first introduced in + Python 2.2), time stamps are still represented as integers, unless + ‘os.stat_float_times()’ is invoked to enable float return values: + + >>> os.stat("/tmp").st_mtime + 1034791200 + >>> os.stat_float_times(True) + >>> os.stat("/tmp").st_mtime + 1034791200.6335014 + + In Python 2.4, the default will change to always returning floats. + + Application developers should enable this feature only if all their + libraries work properly when confronted with floating point time + stamps, or if they use the tuple API. If used, the feature should + be activated on an application level instead of trying to enable it + on a per-use basis. + + * The *note optparse: c3. module contains a new parser for + command-line arguments that can convert option values to a + particular Python type and will automatically generate a usage + message. See the following section for more details. + + * The old and never-documented ‘linuxaudiodev’ module has been + deprecated, and a new version named *note ossaudiodev: c6. has been + added. The module was renamed because the OSS sound drivers can be + used on platforms other than Linux, and the interface has also been + tidied and brought up to date in various ways. (Contributed by + Greg Ward and Nicholas FitzRoy-Dale.) + + * The new *note platform: ce. module contains a number of functions + that try to determine various properties of the platform you’re + running on. There are functions for getting the architecture, CPU + type, the Windows OS version, and even the Linux distribution + version. (Contributed by Marc-André Lemburg.) + + * The parser objects provided by the ‘pyexpat’ module can now + optionally buffer character data, resulting in fewer calls to your + character data handler and therefore faster performance. Setting + the parser object’s ‘buffer_text’ attribute to *note True: 499. + will enable buffering. + + * The ‘sample(population, k)’ function was added to the *note random: + dc. module. `population' is a sequence or ‘xrange’ object + containing the elements of a population, and ‘sample()’ chooses `k' + elements from the population without replacing chosen elements. + `k' can be any value up to ‘len(population)’. For example: + + >>> days = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'St', 'Sn'] + >>> random.sample(days, 3) # Choose 3 elements + ['St', 'Sn', 'Th'] + >>> random.sample(days, 7) # Choose 7 elements + ['Tu', 'Th', 'Mo', 'We', 'St', 'Fr', 'Sn'] + >>> random.sample(days, 7) # Choose 7 again + ['We', 'Mo', 'Sn', 'Fr', 'Tu', 'St', 'Th'] + >>> random.sample(days, 8) # Can't choose eight + Traceback (most recent call last): + File "", line 1, in ? + File "random.py", line 414, in sample + raise ValueError, "sample larger than population" + ValueError: sample larger than population + >>> random.sample(xrange(1,10000,2), 10) # Choose ten odd nos. under 10000 + [3407, 3805, 1505, 7023, 2401, 2267, 9733, 3151, 8083, 9195] + + The *note random: dc. module now uses a new algorithm, the Mersenne + Twister, implemented in C. It’s faster and more extensively studied + than the previous algorithm. + + (All changes contributed by Raymond Hettinger.) + + * The *note readline: de. module also gained a number of new + functions: ‘get_history_item()’, ‘get_current_history_length()’, + and ‘redisplay()’. + + * The ‘rexec’ and ‘Bastion’ modules have been declared dead, and + attempts to import them will fail with a *note RuntimeError: 2ba. + New-style classes provide new ways to break out of the restricted + execution environment provided by ‘rexec’, and no one has interest + in fixing them or time to do so. If you have applications using + ‘rexec’, rewrite them to use something else. + + (Sticking with Python 2.2 or 2.1 will not make your applications + any safer because there are known bugs in the ‘rexec’ module in + those versions. To repeat: if you’re using ‘rexec’, stop using it + immediately.) + + * The ‘rotor’ module has been deprecated because the algorithm it + uses for encryption is not believed to be secure. If you need + encryption, use one of the several AES Python modules that are + available separately. + + * The *note shutil: e9. module gained a ‘move(src, dest)’ function + that recursively moves a file or directory to a new location. + + * Support for more advanced POSIX signal handling was added to the + *note signal: ea. but then removed again as it proved impossible to + make it work reliably across platforms. + + * The *note socket: ef. module now supports timeouts. You can call + the ‘settimeout(t)’ method on a socket object to set a timeout of + `t' seconds. Subsequent socket operations that take longer than + `t' seconds to complete will abort and raise a *note + socket.timeout: c0e. exception. + + The original timeout implementation was by Tim O’Malley. Michael + Gilfix integrated it into the Python *note socket: ef. module and + shepherded it through a lengthy review. After the code was checked + in, Guido van Rossum rewrote parts of it. (This is a good example + of a collaborative development process in action.) + + * On Windows, the *note socket: ef. module now ships with Secure + Sockets Layer (SSL) support. + + * The value of the C ‘PYTHON_API_VERSION’ macro is now exposed at the + Python level as ‘sys.api_version’. The current exception can be + cleared by calling the new ‘sys.exc_clear()’ function. + + * The new *note tarfile: 101. module allows reading from and writing + to ‘tar’-format archive files. (Contributed by Lars Gustäbel.) + + * The new *note textwrap: 108. module contains functions for wrapping + strings containing paragraphs of text. The ‘wrap(text, width)’ + function takes a string and returns a list containing the text + split into lines of no more than the chosen width. The ‘fill(text, + width)’ function returns a single string, reformatted to fit into + lines no longer than the chosen width. (As you can guess, ‘fill()’ + is built on top of ‘wrap()’. For example: + + >>> import textwrap + >>> paragraph = "Not a whit, we defy augury: ... more text ..." + >>> textwrap.wrap(paragraph, 60) + ["Not a whit, we defy augury: there's a special providence in", + "the fall of a sparrow. If it be now, 'tis not to come; if it", + ...] + >>> print textwrap.fill(paragraph, 35) + Not a whit, we defy augury: there's + a special providence in the fall of + a sparrow. If it be now, 'tis not + to come; if it be not to come, it + will be now; if it be not now, yet + it will come: the readiness is all. + >>> + + The module also contains a ‘TextWrapper’ class that actually + implements the text wrapping strategy. Both the ‘TextWrapper’ + class and the ‘wrap()’ and ‘fill()’ functions support a number of + additional keyword arguments for fine-tuning the formatting; + consult the module’s documentation for details. (Contributed by + Greg Ward.) + + * The ‘thread’ and *note threading: 109. modules now have companion + modules, ‘dummy_thread’ and *note dummy_threading: 68, that provide + a do-nothing implementation of the ‘thread’ module’s interface for + platforms where threads are not supported. The intention is to + simplify thread-aware modules (ones that `don’t' rely on threads to + run) by putting the following code at the top: + + try: + import threading as _threading + except ImportError: + import dummy_threading as _threading + + In this example, ‘_threading’ is used as the module name to make it + clear that the module being used is not necessarily the actual + *note threading: 109. module. Code can call functions and use + classes in ‘_threading’ whether or not threads are supported, + avoiding an *note if: de7. statement and making the code slightly + clearer. This module will not magically make multithreaded code + run without threads; code that waits for another thread to return + or to do something will simply hang forever. + + * The *note time: 10a. module’s ‘strptime()’ function has long been + an annoyance because it uses the platform C library’s ‘strptime()’ + implementation, and different platforms sometimes have odd bugs. + Brett Cannon contributed a portable implementation that’s written + in pure Python and should behave identically on all platforms. + + * The new *note timeit: 10b. module helps measure how long snippets + of Python code take to execute. The ‘timeit.py’ file can be run + directly from the command line, or the module’s ‘Timer’ class can + be imported and used directly. Here’s a short example that figures + out whether it’s faster to convert an 8-bit string to Unicode by + appending an empty Unicode string to it or by using the ‘unicode()’ + function: + + import timeit + + timer1 = timeit.Timer('unicode("abc")') + timer2 = timeit.Timer('"abc" + u""') + + # Run three trials + print timer1.repeat(repeat=3, number=100000) + print timer2.repeat(repeat=3, number=100000) + + # On my laptop this outputs: + # [0.36831796169281006, 0.37441694736480713, 0.35304892063140869] + # [0.17574405670166016, 0.18193507194519043, 0.17565798759460449] + + * The ‘Tix’ module has received various bug fixes and updates for the + current version of the Tix package. + + * The ‘Tkinter’ module now works with a thread-enabled version of + Tcl. Tcl’s threading model requires that widgets only be accessed + from the thread in which they’re created; accesses from another + thread can cause Tcl to panic. For certain Tcl interfaces, + ‘Tkinter’ will now automatically avoid this when a widget is + accessed from a different thread by marshalling a command, passing + it to the correct thread, and waiting for the results. Other + interfaces can’t be handled automatically but ‘Tkinter’ will now + raise an exception on such an access so that you can at least find + out about the problem. See + ‘https://mail.python.org/pipermail/python-dev/2002-December/031107.html’ + for a more detailed explanation of this change. (Implemented by + Martin von Löwis.) + + * Calling Tcl methods through ‘_tkinter’ no longer returns only + strings. Instead, if Tcl returns other objects those objects are + converted to their Python equivalent, if one exists, or wrapped + with a ‘_tkinter.Tcl_Obj’ object if no Python equivalent exists. + This behavior can be controlled through the ‘wantobjects()’ method + of ‘tkapp’ objects. + + When using ‘_tkinter’ through the ‘Tkinter’ module (as most Tkinter + applications will), this feature is always activated. It should + not cause compatibility problems, since Tkinter would always + convert string results to Python types where possible. + + If any incompatibilities are found, the old behavior can be + restored by setting the ‘wantobjects’ variable in the ‘Tkinter’ + module to false before creating the first ‘tkapp’ object. + + import Tkinter + Tkinter.wantobjects = 0 + + Any breakage caused by this change should be reported as a bug. + + * The ‘UserDict’ module has a new ‘DictMixin’ class which defines all + dictionary methods for classes that already have a minimum mapping + interface. This greatly simplifies writing classes that need to be + substitutable for dictionaries, such as the classes in the *note + shelve: e7. module. + + Adding the mix-in as a superclass provides the full dictionary + interface whenever the class defines *note __getitem__(): 27c, + *note __setitem__(): c62, *note __delitem__(): c63, and ‘keys()’. + For example: + + >>> import UserDict + >>> class SeqDict(UserDict.DictMixin): + ... """Dictionary lookalike implemented with lists.""" + ... def __init__(self): + ... self.keylist = [] + ... self.valuelist = [] + ... def __getitem__(self, key): + ... try: + ... i = self.keylist.index(key) + ... except ValueError: + ... raise KeyError + ... return self.valuelist[i] + ... def __setitem__(self, key, value): + ... try: + ... i = self.keylist.index(key) + ... self.valuelist[i] = value + ... except ValueError: + ... self.keylist.append(key) + ... self.valuelist.append(value) + ... def __delitem__(self, key): + ... try: + ... i = self.keylist.index(key) + ... except ValueError: + ... raise KeyError + ... self.keylist.pop(i) + ... self.valuelist.pop(i) + ... def keys(self): + ... return list(self.keylist) + ... + >>> s = SeqDict() + >>> dir(s) # See that other dictionary methods are implemented + ['__cmp__', '__contains__', '__delitem__', '__doc__', '__getitem__', + '__init__', '__iter__', '__len__', '__module__', '__repr__', + '__setitem__', 'clear', 'get', 'has_key', 'items', 'iteritems', + 'iterkeys', 'itervalues', 'keylist', 'keys', 'pop', 'popitem', + 'setdefault', 'update', 'valuelist', 'values'] + + (Contributed by Raymond Hettinger.) + + * The DOM implementation in *note xml.dom.minidom: 135. can now + generate XML output in a particular encoding by providing an + optional encoding argument to the ‘toxml()’ and ‘toprettyxml()’ + methods of DOM nodes. + + * The ‘xmlrpclib’ module now supports an XML-RPC extension for + handling nil data values such as Python’s ‘None’. Nil values are + always supported on unmarshalling an XML-RPC response. To generate + requests containing ‘None’, you must supply a true value for the + `allow_none' parameter when creating a ‘Marshaller’ instance. + + * The new ‘DocXMLRPCServer’ module allows writing self-documenting + XML-RPC servers. Run it in demo mode (as a program) to see it in + action. Pointing the Web browser to the RPC server produces + pydoc-style documentation; pointing xmlrpclib to the server allows + invoking the actual methods. (Contributed by Brian Quinlan.) + + * Support for internationalized domain names (RFCs 3454, 3490, 3491, + and 3492) has been added. The “idna” encoding can be used to + convert between a Unicode domain name and the ASCII-compatible + encoding (ACE) of that name. + + >{}>{}> u"www.Alliancefrançaise.nu".encode("idna") + 'www.xn--alliancefranaise-npb.nu' + + The *note socket: ef. module has also been extended to + transparently convert Unicode hostnames to the ACE version before + passing them to the C library. Modules that deal with hostnames + such as ‘httplib’ and *note ftplib: 84.) also support Unicode host + names; ‘httplib’ also sends HTTP ‘Host’ headers using the ACE + version of the domain name. *note urllib: 11d. supports Unicode + URLs with non-ASCII host names as long as the ‘path’ part of the + URL is ASCII only. + + To implement this change, the *note stringprep: f7. module, the + ‘mkstringprep’ tool and the ‘punycode’ encoding have been added. + +* Menu: + +* Date/Time Type:: +* The optparse Module:: + + ---------- Footnotes ---------- + + (1) http://pybsddb.sourceforge.net + + +File: python.info, Node: Date/Time Type, Next: The optparse Module, Up: New Improved and Deprecated Modules<4> + +1.14.17.1 Date/Time Type +........................ + +Date and time types suitable for expressing timestamps were added as the +*note datetime: 31. module. The types don’t support different calendars +or many fancy features, and just stick to the basics of representing +time. + +The three primary types are: ‘date’, representing a day, month, and +year; *note time: 4f3, consisting of hour, minute, and second; and *note +datetime: 194, which contains all the attributes of both ‘date’ and +*note time: 4f3. There’s also a ‘timedelta’ class representing +differences between two points in time, and time zone logic is +implemented by classes inheriting from the abstract ‘tzinfo’ class. + +You can create instances of ‘date’ and *note time: 4f3. by either +supplying keyword arguments to the appropriate constructor, e.g. +‘datetime.date(year=1972, month=10, day=15)’, or by using one of a +number of class methods. For example, the ‘date.today()’ class method +returns the current local date. + +Once created, instances of the date/time classes are all immutable. +There are a number of methods for producing formatted strings from +objects: + + >>> import datetime + >>> now = datetime.datetime.now() + >>> now.isoformat() + '2002-12-30T21:27:03.994956' + >>> now.ctime() # Only available on date, datetime + 'Mon Dec 30 21:27:03 2002' + >>> now.strftime('%Y %d %b') + '2002 30 Dec' + +The ‘replace()’ method allows modifying one or more fields of a ‘date’ +or *note datetime: 194. instance, returning a new instance: + + >>> d = datetime.datetime.now() + >>> d + datetime.datetime(2002, 12, 30, 22, 15, 38, 827738) + >>> d.replace(year=2001, hour = 12) + datetime.datetime(2001, 12, 30, 12, 15, 38, 827738) + >>> + +Instances can be compared, hashed, and converted to strings (the result +is the same as that of ‘isoformat()’). ‘date’ and *note datetime: 194. +instances can be subtracted from each other, and added to ‘timedelta’ +instances. The largest missing feature is that there’s no standard +library support for parsing strings and getting back a ‘date’ or *note +datetime: 194. + +For more information, refer to the module’s reference documentation. +(Contributed by Tim Peters.) + + +File: python.info, Node: The optparse Module, Prev: Date/Time Type, Up: New Improved and Deprecated Modules<4> + +1.14.17.2 The optparse Module +............................. + +The *note getopt: 87. module provides simple parsing of command-line +arguments. The new *note optparse: c3. module (originally named Optik) +provides more elaborate command-line parsing that follows the Unix +conventions, automatically creates the output for ‘--help’, and can +perform different actions for different options. + +You start by creating an instance of ‘OptionParser’ and telling it what +your program’s options are. + + import sys + from optparse import OptionParser + + op = OptionParser() + op.add_option('-i', '--input', + action='store', type='string', dest='input', + help='set input filename') + op.add_option('-l', '--length', + action='store', type='int', dest='length', + help='set maximum length of output') + +Parsing a command line is then done by calling the ‘parse_args()’ +method. + + options, args = op.parse_args(sys.argv[1:]) + print options + print args + +This returns an object containing all of the option values, and a list +of strings containing the remaining arguments. + +Invoking the script with the various arguments now works as you’d expect +it to. Note that the length argument is automatically converted to an +integer. + + $ ./python opt.py -i data arg1 + + ['arg1'] + $ ./python opt.py --input=data --length=4 + + [] + $ + +The help message is automatically generated for you: + + $ ./python opt.py --help + usage: opt.py [options] + + options: + -h, --help show this help message and exit + -iINPUT, --input=INPUT + set input filename + -lLENGTH, --length=LENGTH + set maximum length of output + $ + +See the module’s documentation for more details. + +Optik was written by Greg Ward, with suggestions from the readers of the +Getopt SIG. + + +File: python.info, Node: Pymalloc A Specialized Object Allocator, Next: Build and C API Changes<12>, Prev: New Improved and Deprecated Modules<4>, Up: What’s New in Python 2 3 + +1.14.18 Pymalloc: A Specialized Object Allocator +------------------------------------------------ + +Pymalloc, a specialized object allocator written by Vladimir Marangozov, +was a feature added to Python 2.1. Pymalloc is intended to be faster +than the system ‘malloc()’ and to have less memory overhead for +allocation patterns typical of Python programs. The allocator uses C’s +‘malloc()’ function to get large pools of memory and then fulfills +smaller memory requests from these pools. + +In 2.1 and 2.2, pymalloc was an experimental feature and wasn’t enabled +by default; you had to explicitly enable it when compiling Python by +providing the ‘--with-pymalloc’ option to the ‘configure’ script. In +2.3, pymalloc has had further enhancements and is now enabled by +default; you’ll have to supply ‘--without-pymalloc’ to disable it. + +This change is transparent to code written in Python; however, pymalloc +may expose bugs in C extensions. Authors of C extension modules should +test their code with pymalloc enabled, because some incorrect code may +cause core dumps at runtime. + +There’s one particularly common error that causes problems. There are a +number of memory allocation functions in Python’s C API that have +previously just been aliases for the C library’s ‘malloc()’ and +‘free()’, meaning that if you accidentally called mismatched functions +the error wouldn’t be noticeable. When the object allocator is enabled, +these functions aren’t aliases of ‘malloc()’ and ‘free()’ any more, and +calling the wrong function to free memory may get you a core dump. For +example, if memory was allocated using *note PyObject_Malloc(): 508, it +has to be freed using *note PyObject_Free(): 504, not ‘free()’. A few +modules included with Python fell afoul of this and had to be fixed; +doubtless there are more third-party modules that will have the same +problem. + +As part of this change, the confusing multiple interfaces for allocating +memory have been consolidated down into two API families. Memory +allocated with one family must not be manipulated with functions from +the other family. There is one family for allocating chunks of memory +and another family of functions specifically for allocating Python +objects. + + * To allocate and free an undistinguished chunk of memory use the + “raw memory” family: *note PyMem_Malloc(): 505, *note + PyMem_Realloc(): 96f, and *note PyMem_Free(): da9. + + * The “object memory” family is the interface to the pymalloc + facility described above and is biased towards a large number of + “small” allocations: *note PyObject_Malloc(): 508, *note + PyObject_Realloc(): daa, and *note PyObject_Free(): 504. + + * To allocate and free Python objects, use the “object” family *note + PyObject_New(): 2d2, *note PyObject_NewVar(): 2d3, and *note + PyObject_Del(): e16. + +Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides +debugging features to catch memory overwrites and doubled frees in both +extension modules and in the interpreter itself. To enable this +support, compile a debugging version of the Python interpreter by +running ‘configure’ with ‘--with-pydebug’. + +To aid extension writers, a header file ‘Misc/pymemcompat.h’ is +distributed with the source to Python 2.3 that allows Python extensions +to use the 2.3 interfaces to memory allocation while compiling against +any version of Python since 1.5.2. You would copy the file from +Python’s source distribution and bundle it with the source of your +extension. + +See also +........ + +‘https://hg.python.org/cpython/file/default/Objects/obmalloc.c’ + + For the full details of the pymalloc implementation, see the + comments at the top of the file ‘Objects/obmalloc.c’ in the Python + source code. The above link points to the file within the + python.org SVN browser. + + +File: python.info, Node: Build and C API Changes<12>, Next: Other Changes and Fixes<2>, Prev: Pymalloc A Specialized Object Allocator, Up: What’s New in Python 2 3 + +1.14.19 Build and C API Changes +------------------------------- + +Changes to Python’s build process and to the C API include: + + * The cycle detection implementation used by the garbage collection + has proven to be stable, so it’s now been made mandatory. You can + no longer compile Python without it, and the ‘--with-cycle-gc’ + switch to ‘configure’ has been removed. + + * Python can now optionally be built as a shared library + (‘libpython2.3.so’) by supplying ‘--enable-shared’ when running + Python’s ‘configure’ script. (Contributed by Ondrej Palkovsky.) + + * The ‘DL_EXPORT’ and ‘DL_IMPORT’ macros are now deprecated. + Initialization functions for Python extension modules should now be + declared using the new macro ‘PyMODINIT_FUNC’, while the Python + core will generally use the ‘PyAPI_FUNC’ and ‘PyAPI_DATA’ macros. + + * The interpreter can be compiled without any docstrings for the + built-in functions and modules by supplying ‘--without-doc-strings’ + to the ‘configure’ script. This makes the Python executable about + 10% smaller, but will also mean that you can’t get help for + Python’s built-ins. (Contributed by Gustavo Niemeyer.) + + * The ‘PyArg_NoArgs()’ macro is now deprecated, and code that uses it + should be changed. For Python 2.2 and later, the method definition + table can specify the *note METH_NOARGS: e18. flag, signalling that + there are no arguments, and the argument checking can then be + removed. If compatibility with pre-2.2 versions of Python is + important, the code could use ‘PyArg_ParseTuple(args, "")’ instead, + but this will be slower than using *note METH_NOARGS: e18. + + * *note PyArg_ParseTuple(): 26a. accepts new format characters for + various sizes of unsigned integers: ‘B’ for ‘unsigned char’, ‘H’ + for ‘unsigned short int’, ‘I’ for ‘unsigned int’, and ‘K’ for + ‘unsigned long long’. + + * A new function, ‘PyObject_DelItemString(mapping, char *key)’ was + added as shorthand for ‘PyObject_DelItem(mapping, + PyString_New(key))’. + + * File objects now manage their internal string buffer differently, + increasing it exponentially when needed. This results in the + benchmark tests in ‘Lib/test/test_bufio.py’ speeding up + considerably (from 57 seconds to 1.7 seconds, according to one + measurement). + + * It’s now possible to define class and static methods for a C + extension type by setting either the *note METH_CLASS: e19. or + *note METH_STATIC: e1a. flags in a method’s *note PyMethodDef: e1b. + structure. + + * Python now includes a copy of the Expat XML parser’s source code, + removing any dependence on a system version or local installation + of Expat. + + * If you dynamically allocate type objects in your extension, you + should be aware of a change in the rules relating to the + ‘__module__’ and *note __name__: c67. attributes. In summary, you + will want to ensure the type’s dictionary contains a ‘'__module__'’ + key; making the module name the part of the type name leading up to + the final period will no longer have the desired effect. For more + detail, read the API reference documentation or the source. + +* Menu: + +* Port-Specific Changes: Port-Specific Changes<3>. + + +File: python.info, Node: Port-Specific Changes<3>, Up: Build and C API Changes<12> + +1.14.19.1 Port-Specific Changes +............................... + +Support for a port to IBM’s OS/2 using the EMX runtime environment was +merged into the main Python source tree. EMX is a POSIX emulation layer +over the OS/2 system APIs. The Python port for EMX tries to support all +the POSIX-like capability exposed by the EMX runtime, and mostly +succeeds; ‘fork()’ and *note fcntl(): 7e. are restricted by the +limitations of the underlying emulation layer. The standard OS/2 port, +which uses IBM’s Visual Age compiler, also gained support for +case-sensitive import semantics as part of the integration of the EMX +port into CVS. (Contributed by Andrew MacIntyre.) + +On MacOS, most toolbox modules have been weaklinked to improve backward +compatibility. This means that modules will no longer fail to load if a +single routine is missing on the current OS version. Instead calling +the missing routine will raise an exception. (Contributed by Jack +Jansen.) + +The RPM spec files, found in the ‘Misc/RPM/’ directory in the Python +source distribution, were updated for 2.3. (Contributed by Sean +Reifschneider.) + +Other new platforms now supported by Python include AtheOS +(‘http://www.atheos.cx/’), GNU/Hurd, and OpenVMS. + + +File: python.info, Node: Other Changes and Fixes<2>, Next: Porting to Python 2 3, Prev: Build and C API Changes<12>, Up: What’s New in Python 2 3 + +1.14.20 Other Changes and Fixes +------------------------------- + +As usual, there were a bunch of other improvements and bugfixes +scattered throughout the source tree. A search through the CVS change +logs finds there were 523 patches applied and 514 bugs fixed between +Python 2.2 and 2.3. Both figures are likely to be underestimates. + +Some of the more notable changes are: + + * If the *note PYTHONINSPECT: e1e. environment variable is set, the + Python interpreter will enter the interactive prompt after running + a Python program, as if Python had been invoked with the *note -i: + e1f. option. The environment variable can be set before running + the Python interpreter, or it can be set by the Python program as + part of its execution. + + * The ‘regrtest.py’ script now provides a way to allow “all resources + except `foo'.” A resource name passed to the ‘-u’ option can now be + prefixed with a hyphen (‘'-'’) to mean “remove this resource.” For + example, the option ‘‘-uall,-bsddb’’ could be used to enable the + use of all resources except ‘bsddb’. + + * The tools used to build the documentation now work under Cygwin as + well as Unix. + + * The ‘SET_LINENO’ opcode has been removed. Back in the mists of + time, this opcode was needed to produce line numbers in tracebacks + and support trace functions (for, e.g., *note pdb: c9.). Since + Python 1.5, the line numbers in tracebacks have been computed using + a different mechanism that works with “python -O”. For Python 2.3 + Michael Hudson implemented a similar scheme to determine when to + call the trace function, removing the need for ‘SET_LINENO’ + entirely. + + It would be difficult to detect any resulting difference from + Python code, apart from a slight speed up when Python is run + without *note -O: 68b. + + C extensions that access the ‘f_lineno’ field of frame objects + should instead call ‘PyCode_Addr2Line(f->f_code, f->f_lasti)’. + This will have the added effect of making the code work as desired + under “python -O” in earlier versions of Python. + + A nifty new feature is that trace functions can now assign to the + ‘f_lineno’ attribute of frame objects, changing the line that will + be executed next. A ‘jump’ command has been added to the *note + pdb: c9. debugger taking advantage of this new feature. + (Implemented by Richie Hindle.) + + +File: python.info, Node: Porting to Python 2 3, Next: Acknowledgements<5>, Prev: Other Changes and Fixes<2>, Up: What’s New in Python 2 3 + +1.14.21 Porting to Python 2.3 +----------------------------- + +This section lists previously described changes that may require changes +to your code: + + * *note yield: 18f. is now always a keyword; if it’s used as a + variable name in your code, a different name must be chosen. + + * For strings `X' and `Y', ‘X in Y’ now works if `X' is more than one + character long. + + * The *note int(): 184. type constructor will now return a long + integer instead of raising an *note OverflowError: 960. when a + string or floating-point number is too large to fit into an + integer. + + * If you have Unicode strings that contain 8-bit characters, you must + declare the file’s encoding (UTF-8, Latin-1, or whatever) by adding + a comment to the top of the file. See section *note PEP 263; + Source Code Encodings: de9. for more information. + + * Calling Tcl methods through ‘_tkinter’ no longer returns only + strings. Instead, if Tcl returns other objects those objects are + converted to their Python equivalent, if one exists, or wrapped + with a ‘_tkinter.Tcl_Obj’ object if no Python equivalent exists. + + * Large octal and hex literals such as ‘0xffffffff’ now trigger a + *note FutureWarning: 325. Currently they’re stored as 32-bit + numbers and result in a negative value, but in Python 2.4 they’ll + become positive long integers. + + There are a few ways to fix this warning. If you really need a + positive number, just add an ‘L’ to the end of the literal. If + you’re trying to get a 32-bit integer with low bits set and have + previously used an expression such as ‘~(1 << 31)’, it’s probably + clearest to start with all bits set and clear the desired upper + bits. For example, to clear just the top bit (bit 31), you could + write ‘0xffffffffL &~(1L<<31)’. + + * You can no longer disable assertions by assigning to ‘__debug__’. + + * The Distutils ‘setup()’ function has gained various new keyword + arguments such as `depends'. Old versions of the Distutils will + abort if passed unknown keywords. A solution is to check for the + presence of the new ‘get_distutil_options()’ function in your + ‘setup.py’ and only uses the new keywords with a version of the + Distutils that supports them: + + from distutils import core + + kw = {'sources': 'foo.c', ...} + if hasattr(core, 'get_distutil_options'): + kw['depends'] = ['foo.h'] + ext = Extension(**kw) + + * Using ‘None’ as a variable name will now result in a *note + SyntaxWarning: 191. warning. + + * Names of extension types defined by the modules included with + Python now contain the module and a ‘'.'’ in front of the type + name. + + +File: python.info, Node: Acknowledgements<5>, Prev: Porting to Python 2 3, Up: What’s New in Python 2 3 + +1.14.22 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Jeff Bauer, Simon Brunning, Brett Cannon, Michael Chermside, +Andrew Dalke, Scott David Daniels, Fred L. Drake, Jr., David Fraser, +Kelly Gerber, Raymond Hettinger, Michael Hudson, Chris Lambert, Detlef +Lannert, Martin von Löwis, Andrew MacIntyre, Lalo Martins, Chad Netzer, +Gustavo Niemeyer, Neal Norwitz, Hans Nowak, Chris Reedy, Francesco +Ricciardi, Vinay Sajip, Neil Schemenauer, Roman Suzi, Jason Tishler, +Just van Rossum. + + +File: python.info, Node: What’s New in Python 2 2, Next: What’s New in Python 2 1, Prev: What’s New in Python 2 3, Up: What’s New in Python + +1.15 What’s New in Python 2.2 +============================= + + +Author: A.M. Kuchling + +* Menu: + +* Introduction:: +* PEPs 252 and 253; Type and Class Changes: PEPs 252 and 253 Type and Class Changes. +* PEP 234; Iterators: PEP 234 Iterators. +* PEP 255; Simple Generators: PEP 255 Simple Generators<2>. +* PEP 237; Unifying Long Integers and Integers: PEP 237 Unifying Long Integers and Integers<2>. +* PEP 238; Changing the Division Operator: PEP 238 Changing the Division Operator. +* Unicode Changes:: +* PEP 227; Nested Scopes: PEP 227 Nested Scopes. +* New and Improved Modules: New and Improved Modules<3>. +* Interpreter Changes and Fixes:: +* Other Changes and Fixes: Other Changes and Fixes<3>. +* Acknowledgements: Acknowledgements<6>. + + +File: python.info, Node: Introduction, Next: PEPs 252 and 253 Type and Class Changes, Up: What’s New in Python 2 2 + +1.15.1 Introduction +------------------- + +This article explains the new features in Python 2.2.2, released on +October 14, 2002. Python 2.2.2 is a bugfix release of Python 2.2, +originally released on December 21, 2001. + +Python 2.2 can be thought of as the “cleanup release”. There are some +features such as generators and iterators that are completely new, but +most of the changes, significant and far-reaching though they may be, +are aimed at cleaning up irregularities and dark corners of the language +design. + +This article doesn’t attempt to provide a complete specification of the +new features, but instead provides a convenient overview. For full +details, you should refer to the documentation for Python 2.2, such as +the Python Library Reference(1) and the Python Reference Manual(2). If +you want to understand the complete implementation and design rationale +for a change, refer to the PEP for a particular new feature. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/2.2/lib/lib.html + + (2) https://docs.python.org/2.2/ref/ref.html + + +File: python.info, Node: PEPs 252 and 253 Type and Class Changes, Next: PEP 234 Iterators, Prev: Introduction, Up: What’s New in Python 2 2 + +1.15.2 PEPs 252 and 253: Type and Class Changes +----------------------------------------------- + +The largest and most far-reaching changes in Python 2.2 are to Python’s +model of objects and classes. The changes should be backward +compatible, so it’s likely that your code will continue to run +unchanged, but the changes provide some amazing new capabilities. +Before beginning this, the longest and most complicated section of this +article, I’ll provide an overview of the changes and offer some +comments. + +A long time ago I wrote a Web page listing flaws in Python’s design. +One of the most significant flaws was that it’s impossible to subclass +Python types implemented in C. In particular, it’s not possible to +subclass built-in types, so you can’t just subclass, say, lists in order +to add a single useful method to them. The ‘UserList’ module provides a +class that supports all of the methods of lists and that can be +subclassed further, but there’s lots of C code that expects a regular +Python list and won’t accept a ‘UserList’ instance. + +Python 2.2 fixes this, and in the process adds some exciting new +capabilities. A brief summary: + + * You can subclass built-in types such as lists and even integers, + and your subclasses should work in every place that requires the + original type. + + * It’s now possible to define static and class methods, in addition + to the instance methods available in previous versions of Python. + + * It’s also possible to automatically call methods on accessing or + setting an instance attribute by using a new mechanism called + `properties'. Many uses of *note __getattr__(): 31a. can be + rewritten to use properties instead, making the resulting code + simpler and faster. As a small side benefit, attributes can now + have docstrings, too. + + * The list of legal attributes for an instance can be limited to a + particular set using `slots', making it possible to safeguard + against typos and perhaps make more optimizations possible in + future versions of Python. + +Some users have voiced concern about all these changes. Sure, they say, +the new features are neat and lend themselves to all sorts of tricks +that weren’t possible in previous versions of Python, but they also make +the language more complicated. Some people have said that they’ve +always recommended Python for its simplicity, and feel that its +simplicity is being lost. + +Personally, I think there’s no need to worry. Many of the new features +are quite esoteric, and you can write a lot of Python code without ever +needed to be aware of them. Writing a simple class is no more difficult +than it ever was, so you don’t need to bother learning or teaching them +unless they’re actually needed. Some very complicated tasks that were +previously only possible from C will now be possible in pure Python, and +to my mind that’s all for the better. + +I’m not going to attempt to cover every single corner case and small +change that were required to make the new features work. Instead this +section will paint only the broad strokes. See section *note Related +Links: e27, “Related Links”, for further sources of information about +Python 2.2’s new object model. + +* Menu: + +* Old and New Classes:: +* Descriptors:: +* Multiple Inheritance; The Diamond Rule: Multiple Inheritance The Diamond Rule. +* Attribute Access:: +* Related Links:: + + +File: python.info, Node: Old and New Classes, Next: Descriptors, Up: PEPs 252 and 253 Type and Class Changes + +1.15.2.1 Old and New Classes +............................ + +First, you should know that Python 2.2 really has two kinds of classes: +classic or old-style classes, and new-style classes. The old-style +class model is exactly the same as the class model in earlier versions +of Python. All the new features described in this section apply only to +new-style classes. This divergence isn’t intended to last forever; +eventually old-style classes will be dropped, possibly in Python 3.0. + +So how do you define a new-style class? You do it by subclassing an +existing new-style class. Most of Python’s built-in types, such as +integers, lists, dictionaries, and even files, are new-style classes +now. A new-style class named *note object: 2b0, the base class for all +built-in types, has also been added so if no built-in type is suitable, +you can just subclass *note object: 2b0.: + + class C(object): + def __init__ (self): + ... + ... + +This means that *note class: c6a. statements that don’t have any base +classes are always classic classes in Python 2.2. (Actually you can +also change this by setting a module-level variable named +‘__metaclass__’ — see PEP 253(1) for the details — but it’s easier to +just subclass *note object: 2b0.) + +The type objects for the built-in types are available as built-ins, +named using a clever trick. Python has always had built-in functions +named *note int(): 184, *note float(): 187, and *note str(): 330. In +2.2, they aren’t functions any more, but type objects that behave as +factories when called. + + >>> int + + >>> int('123') + 123 + +To make the set of types complete, new type objects such as *note +dict(): 1b8. and ‘file()’ have been added. Here’s a more interesting +example, adding a ‘lock()’ method to file objects: + + class LockableFile(file): + def lock (self, operation, length=0, start=0, whence=0): + import fcntl + return fcntl.lockf(self.fileno(), operation, + length, start, whence) + +The now-obsolete ‘posixfile’ module contained a class that emulated all +of a file object’s methods and also added a ‘lock()’ method, but this +class couldn’t be passed to internal functions that expected a built-in +file, something which is possible with our new ‘LockableFile’. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0253 + + +File: python.info, Node: Descriptors, Next: Multiple Inheritance The Diamond Rule, Prev: Old and New Classes, Up: PEPs 252 and 253 Type and Class Changes + +1.15.2.2 Descriptors +.................... + +In previous versions of Python, there was no consistent way to discover +what attributes and methods were supported by an object. There were +some informal conventions, such as defining ‘__members__’ and +‘__methods__’ attributes that were lists of names, but often the author +of an extension type or a class wouldn’t bother to define them. You +could fall back on inspecting the *note __dict__: 4fc. of an object, but +when class inheritance or an arbitrary *note __getattr__(): 31a. hook +were in use this could still be inaccurate. + +The one big idea underlying the new class model is that an API for +describing the attributes of an object using `descriptors' has been +formalized. Descriptors specify the value of an attribute, stating +whether it’s a method or a field. With the descriptor API, static +methods and class methods become possible, as well as more exotic +constructs. + +Attribute descriptors are objects that live inside class objects, and +have a few attributes of their own: + + * *note __name__: c67. is the attribute’s name. + + * ‘__doc__’ is the attribute’s docstring. + + * ‘__get__(object)’ is a method that retrieves the attribute value + from `object'. + + * ‘__set__(object, value)’ sets the attribute on `object' to `value'. + + * ‘__delete__(object, value)’ deletes the `value' attribute of + `object'. + +For example, when you write ‘obj.x’, the steps that Python actually +performs are: + + descriptor = obj.__class__.x + descriptor.__get__(obj) + +For methods, ‘descriptor.__get__()’ returns a temporary object that’s +callable, and wraps up the instance and the method to be called on it. +This is also why static methods and class methods are now possible; they +have descriptors that wrap up just the method, or the method and the +class. As a brief explanation of these new kinds of methods, static +methods aren’t passed the instance, and therefore resemble regular +functions. Class methods are passed the class of the object, but not +the object itself. Static and class methods are defined like this: + + class C(object): + def f(arg1, arg2): + ... + f = staticmethod(f) + + def g(cls, arg1, arg2): + ... + g = classmethod(g) + +The *note staticmethod(): 1d9. function takes the function ‘f()’, and +returns it wrapped up in a descriptor so it can be stored in the class +object. You might expect there to be special syntax for creating such +methods (‘def static f’, ‘defstatic f()’, or something like that) but no +such syntax has been defined yet; that’s been left for future versions +of Python. + +More new features, such as slots and properties, are also implemented as +new kinds of descriptors, and it’s not difficult to write a descriptor +class that does something novel. For example, it would be possible to +write a descriptor class that made it possible to write Eiffel-style +preconditions and postconditions for a method. A class that used this +feature might be defined like this: + + from eiffel import eiffelmethod + + class C(object): + def f(self, arg1, arg2): + # The actual function + ... + def pre_f(self): + # Check preconditions + ... + def post_f(self): + # Check postconditions + ... + + f = eiffelmethod(f, pre_f, post_f) + +Note that a person using the new ‘eiffelmethod()’ doesn’t have to +understand anything about descriptors. This is why I think the new +features don’t increase the basic complexity of the language. There +will be a few wizards who need to know about it in order to write +‘eiffelmethod()’ or the ZODB or whatever, but most users will just write +code on top of the resulting libraries and ignore the implementation +details. + + +File: python.info, Node: Multiple Inheritance The Diamond Rule, Next: Attribute Access, Prev: Descriptors, Up: PEPs 252 and 253 Type and Class Changes + +1.15.2.3 Multiple Inheritance: The Diamond Rule +............................................... + +Multiple inheritance has also been made more useful through changing the +rules under which names are resolved. Consider this set of classes +(diagram taken from PEP 253(1) by Guido van Rossum): + + class A: + ^ ^ def save(self): ... + / \ + / \ + / \ + / \ + class B class C: + ^ ^ def save(self): ... + \ / + \ / + \ / + \ / + class D + +The lookup rule for classic classes is simple but not very smart; the +base classes are searched depth-first, going from left to right. A +reference to ‘D.save()’ will search the classes ‘D’, ‘B’, and then ‘A’, +where ‘save()’ would be found and returned. ‘C.save()’ would never be +found at all. This is bad, because if ‘C’’s ‘save()’ method is saving +some internal state specific to ‘C’, not calling it will result in that +state never getting saved. + +New-style classes follow a different algorithm that’s a bit more +complicated to explain, but does the right thing in this situation. +(Note that Python 2.3 changes this algorithm to one that produces the +same results in most cases, but produces more useful results for really +complicated inheritance graphs.) + + 1. List all the base classes, following the classic lookup rule and + include a class multiple times if it’s visited repeatedly. In the + above example, the list of visited classes is [‘D’, ‘B’, ‘A’, ‘C’, + ‘A’]. + + 2. Scan the list for duplicated classes. If any are found, remove all + but one occurrence, leaving the `last' one in the list. In the + above example, the list becomes [‘D’, ‘B’, ‘C’, ‘A’] after dropping + duplicates. + +Following this rule, referring to ‘D.save()’ will return ‘C.save()’, +which is the behaviour we’re after. This lookup rule is the same as the +one followed by Common Lisp. A new built-in function, *note super(): +4e2, provides a way to get at a class’s superclasses without having to +reimplement Python’s algorithm. The most commonly used form will be +‘super(class, obj)’, which returns a bound superclass object (not the +actual class object). This form will be used in methods to call a +method in the superclass; for example, ‘D’’s ‘save()’ method would look +like this: + + class D (B,C): + def save (self): + # Call superclass .save() + super(D, self).save() + # Save D's private information here + ... + +*note super(): 4e2. can also return unbound superclass objects when +called as ‘super(class)’ or ‘super(class1, class2)’, but this probably +won’t often be useful. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0253 + + +File: python.info, Node: Attribute Access, Next: Related Links, Prev: Multiple Inheritance The Diamond Rule, Up: PEPs 252 and 253 Type and Class Changes + +1.15.2.4 Attribute Access +......................... + +A fair number of sophisticated Python classes define hooks for attribute +access using *note __getattr__(): 31a.; most commonly this is done for +convenience, to make code more readable by automatically mapping an +attribute access such as ‘obj.parent’ into a method call such as +‘obj.get_parent’. Python 2.2 adds some new ways of controlling +attribute access. + +First, ‘__getattr__(attr_name)’ is still supported by new-style classes, +and nothing about it has changed. As before, it will be called when an +attempt is made to access ‘obj.foo’ and no attribute named ‘foo’ is +found in the instance’s dictionary. + +New-style classes also support a new method, +‘__getattribute__(attr_name)’. The difference between the two methods +is that *note __getattribute__(): 449. is `always' called whenever any +attribute is accessed, while the old *note __getattr__(): 31a. is only +called if ‘foo’ isn’t found in the instance’s dictionary. + +However, Python 2.2’s support for `properties' will often be a simpler +way to trap attribute references. Writing a *note __getattr__(): 31a. +method is complicated because to avoid recursion you can’t use regular +attribute accesses inside them, and instead have to mess around with the +contents of *note __dict__: 4fc. *note __getattr__(): 31a. methods also +end up being called by Python when it checks for other methods such as +*note __repr__(): 33e. or ‘__coerce__()’, and so have to be written with +this in mind. Finally, calling a function on every attribute access +results in a sizable performance loss. + +*note property: 1d7. is a new built-in type that packages up three +functions that get, set, or delete an attribute, and a docstring. For +example, if you want to define a ‘size’ attribute that’s computed, but +also settable, you could write: + + class C(object): + def get_size (self): + result = ... computation ... + return result + def set_size (self, size): + ... compute something based on the size + and set internal state appropriately ... + + # Define a property. The 'delete this attribute' + # method is defined as None, so the attribute + # can't be deleted. + size = property(get_size, set_size, + None, + "Storage size of this instance") + +That is certainly clearer and easier to write than a pair of *note +__getattr__(): 31a./*note __setattr__(): e2c. methods that check for the +‘size’ attribute and handle it specially while retrieving all other +attributes from the instance’s *note __dict__: 4fc. Accesses to ‘size’ +are also the only ones which have to perform the work of calling a +function, so references to other attributes run at their usual speed. + +Finally, it’s possible to constrain the list of attributes that can be +referenced on an object using the new *note __slots__: e2d. class +attribute. Python objects are usually very dynamic; at any time it’s +possible to define a new attribute on an instance by just doing +‘obj.new_attr=1’. A new-style class can define a class attribute named +*note __slots__: e2d. to limit the legal attributes to a particular set +of names. An example will make this clear: + + >>> class C(object): + ... __slots__ = ('template', 'name') + ... + >>> obj = C() + >>> print obj.template + None + >>> obj.template = 'Test' + >>> print obj.template + Test + >>> obj.newattr = None + Traceback (most recent call last): + File "", line 1, in ? + AttributeError: 'C' object has no attribute 'newattr' + +Note how you get an *note AttributeError: 39b. on the attempt to assign +to an attribute not listed in *note __slots__: e2d. + + +File: python.info, Node: Related Links, Prev: Attribute Access, Up: PEPs 252 and 253 Type and Class Changes + +1.15.2.5 Related Links +...................... + +This section has just been a quick overview of the new features, giving +enough of an explanation to start you programming, but many details have +been simplified or ignored. Where should you go to get a more complete +picture? + +‘https://docs.python.org/dev/howto/descriptor.html’ is a lengthy +tutorial introduction to the descriptor features, written by Guido van +Rossum. If my description has whetted your appetite, go read this +tutorial next, because it goes into much more detail about the new +features while still remaining quite easy to read. + +Next, there are two relevant PEPs, PEP 252(1) and PEP 253(2). PEP +252(3) is titled “Making Types Look More Like Classes”, and covers the +descriptor API. PEP 253(4) is titled “Subtyping Built-in Types”, and +describes the changes to type objects that make it possible to subtype +built-in objects. PEP 253(5) is the more complicated PEP of the two, +and at a few points the necessary explanations of types and meta-types +may cause your head to explode. Both PEPs were written and implemented +by Guido van Rossum, with substantial assistance from the rest of the +Zope Corp. team. + +Finally, there’s the ultimate authority: the source code. Most of the +machinery for the type handling is in ‘Objects/typeobject.c’, but you +should only resort to it after all other avenues have been exhausted, +including posting a question to python-list or python-dev. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0252 + + (2) https://www.python.org/dev/peps/pep-0253 + + (3) https://www.python.org/dev/peps/pep-0252 + + (4) https://www.python.org/dev/peps/pep-0253 + + (5) https://www.python.org/dev/peps/pep-0253 + + +File: python.info, Node: PEP 234 Iterators, Next: PEP 255 Simple Generators<2>, Prev: PEPs 252 and 253 Type and Class Changes, Up: What’s New in Python 2 2 + +1.15.3 PEP 234: Iterators +------------------------- + +Another significant addition to 2.2 is an iteration interface at both +the C and Python levels. Objects can define how they can be looped over +by callers. + +In Python versions up to 2.1, the usual way to make ‘for item in obj’ +work is to define a *note __getitem__(): 27c. method that looks +something like this: + + def __getitem__(self, index): + return + +*note __getitem__(): 27c. is more properly used to define an indexing +operation on an object so that you can write ‘obj[5]’ to retrieve the +sixth element. It’s a bit misleading when you’re using this only to +support *note for: c30. loops. Consider some file-like object that +wants to be looped over; the `index' parameter is essentially +meaningless, as the class probably assumes that a series of *note +__getitem__(): 27c. calls will be made with `index' incrementing by one +each time. In other words, the presence of the *note __getitem__(): +27c. method doesn’t mean that using ‘file[5]’ to randomly access the +sixth element will work, though it really should. + +In Python 2.2, iteration can be implemented separately, and *note +__getitem__(): 27c. methods can be limited to classes that really do +support random access. The basic idea of iterators is simple. A new +built-in function, ‘iter(obj)’ or ‘iter(C, sentinel)’, is used to get an +iterator. ‘iter(obj)’ returns an iterator for the object `obj', while +‘iter(C, sentinel)’ returns an iterator that will invoke the callable +object `C' until it returns `sentinel' to signal that the iterator is +done. + +Python classes can define an *note __iter__(): 50f. method, which should +create and return a new iterator for the object; if the object is its +own iterator, this method can just return ‘self’. In particular, +iterators will usually be their own iterators. Extension types +implemented in C can implement a *note tp_iter: e30. function in order +to return an iterator, and extension types that want to behave as +iterators can define a *note tp_iternext: e31. function. + +So, after all this, what do iterators actually do? They have one +required method, *note next(): 682, which takes no arguments and returns +the next value. When there are no more values to be returned, calling +*note next(): 682. should raise the *note StopIteration: 486. exception. + + >>> L = [1,2,3] + >>> i = iter(L) + >>> print i + + >>> i.next() + 1 + >>> i.next() + 2 + >>> i.next() + 3 + >>> i.next() + Traceback (most recent call last): + File "", line 1, in ? + StopIteration + >>> + +In 2.2, Python’s *note for: c30. statement no longer expects a sequence; +it expects something for which *note iter(): d27. will return an +iterator. For backward compatibility and convenience, an iterator is +automatically constructed for sequences that don’t implement *note +__iter__(): 50f. or a *note tp_iter: e30. slot, so ‘for i in [1,2,3]’ +will still work. Wherever the Python interpreter loops over a sequence, +it’s been changed to use the iterator protocol. This means you can do +things like this: + + >>> L = [1,2,3] + >>> i = iter(L) + >>> a,b,c = i + >>> a,b,c + (1, 2, 3) + +Iterator support has been added to some of Python’s basic types. +Calling *note iter(): d27. on a dictionary will return an iterator which +loops over its keys: + + >>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6, + ... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12} + >>> for key in m: print key, m[key] + ... + Mar 3 + Feb 2 + Aug 8 + Sep 9 + May 5 + Jun 6 + Jul 7 + Jan 1 + Apr 4 + Nov 11 + Dec 12 + Oct 10 + +That’s just the default behaviour. If you want to iterate over keys, +values, or key/value pairs, you can explicitly call the ‘iterkeys()’, +‘itervalues()’, or ‘iteritems()’ methods to get an appropriate iterator. +In a minor related change, the *note in: 7a3. operator now works on +dictionaries, so ‘key in dict’ is now equivalent to ‘dict.has_key(key)’. + +Files also provide an iterator, which calls the *note readline(): de. +method until there are no more lines in the file. This means you can +now read each line of a file using code like this: + + for line in file: + # do something for each line + ... + +Note that you can only go forward in an iterator; there’s no way to get +the previous element, reset the iterator, or make a copy of it. An +iterator object could provide such additional capabilities, but the +iterator protocol only requires a *note next(): 682. method. + +See also +........ + +PEP 234(1) - Iterators + + Written by Ka-Ping Yee and GvR; implemented by the Python Labs + crew, mostly by GvR and Tim Peters. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0234 + + +File: python.info, Node: PEP 255 Simple Generators<2>, Next: PEP 237 Unifying Long Integers and Integers<2>, Prev: PEP 234 Iterators, Up: What’s New in Python 2 2 + +1.15.4 PEP 255: Simple Generators +--------------------------------- + +Generators are another new feature, one that interacts with the +introduction of iterators. + +You’re doubtless familiar with how function calls work in Python or C. +When you call a function, it gets a private namespace where its local +variables are created. When the function reaches a *note return: 190. +statement, the local variables are destroyed and the resulting value is +returned to the caller. A later call to the same function will get a +fresh new set of local variables. But, what if the local variables +weren’t thrown away on exiting a function? What if you could later +resume the function where it left off? This is what generators provide; +they can be thought of as resumable functions. + +Here’s the simplest example of a generator function: + + def generate_ints(N): + for i in range(N): + yield i + +A new keyword, *note yield: 18f, was introduced for generators. Any +function containing a ‘yield’ statement is a generator function; this is +detected by Python’s bytecode compiler which compiles the function +specially as a result. Because a new keyword was introduced, generators +must be explicitly enabled in a module by including a ‘from __future__ +import generators’ statement near the top of the module’s source code. +In Python 2.3 this statement will become unnecessary. + +When you call a generator function, it doesn’t return a single value; +instead it returns a generator object that supports the iterator +protocol. On executing the *note yield: 18f. statement, the generator +outputs the value of ‘i’, similar to a *note return: 190. statement. +The big difference between ‘yield’ and a ‘return’ statement is that on +reaching a ‘yield’ the generator’s state of execution is suspended and +local variables are preserved. On the next call to the generator’s +‘next()’ method, the function will resume executing immediately after +the ‘yield’ statement. (For complicated reasons, the ‘yield’ statement +isn’t allowed inside the ‘try’ block of a *note try: d72.…*note finally: +182. statement; read PEP 255(1) for a full explanation of the +interaction between ‘yield’ and exceptions.) + +Here’s a sample usage of the ‘generate_ints()’ generator: + + >>> gen = generate_ints(3) + >>> gen + + >>> gen.next() + 0 + >>> gen.next() + 1 + >>> gen.next() + 2 + >>> gen.next() + Traceback (most recent call last): + File "", line 1, in ? + File "", line 2, in generate_ints + StopIteration + +You could equally write ‘for i in generate_ints(5)’, or ‘a,b,c = +generate_ints(3)’. + +Inside a generator function, the *note return: 190. statement can only +be used without a value, and signals the end of the procession of +values; afterwards the generator cannot return any further values. +‘return’ with a value, such as ‘return 5’, is a syntax error inside a +generator function. The end of the generator’s results can also be +indicated by raising *note StopIteration: 486. manually, or by just +letting the flow of execution fall off the bottom of the function. + +You could achieve the effect of generators manually by writing your own +class and storing all the local variables of the generator as instance +variables. For example, returning a list of integers could be done by +setting ‘self.count’ to 0, and having the *note next(): 682. method +increment ‘self.count’ and return it. However, for a moderately +complicated generator, writing a corresponding class would be much +messier. ‘Lib/test/test_generators.py’ contains a number of more +interesting examples. The simplest one implements an in-order traversal +of a tree using generators recursively. + + # A recursive generator that generates Tree leaves in in-order. + def inorder(t): + if t: + for x in inorder(t.left): + yield x + yield t.label + for x in inorder(t.right): + yield x + +Two other examples in ‘Lib/test/test_generators.py’ produce solutions +for the N-Queens problem (placing $N$ queens on an $NxN$ chess board so +that no queen threatens another) and the Knight’s Tour (a route that +takes a knight to every square of an $NxN$ chessboard without visiting +any square twice). + +The idea of generators comes from other programming languages, +especially Icon (‘https://www.cs.arizona.edu/icon/’), where the idea of +generators is central. In Icon, every expression and function call +behaves like a generator. One example from “An Overview of the Icon +Programming Language” at +‘https://www.cs.arizona.edu/icon/docs/ipd266.htm’ gives an idea of what +this looks like: + + sentence := "Store it in the neighboring harbor" + if (i := find("or", sentence)) > 5 then write(i) + +In Icon the ‘find()’ function returns the indexes at which the substring +“or” is found: 3, 23, 33. In the *note if: de7. statement, ‘i’ is first +assigned a value of 3, but 3 is less than 5, so the comparison fails, +and Icon retries it with the second value of 23. 23 is greater than 5, +so the comparison now succeeds, and the code prints the value 23 to the +screen. + +Python doesn’t go nearly as far as Icon in adopting generators as a +central concept. Generators are considered a new part of the core +Python language, but learning or using them isn’t compulsory; if they +don’t solve any problems that you have, feel free to ignore them. One +novel feature of Python’s interface as compared to Icon’s is that a +generator’s state is represented as a concrete object (the iterator) +that can be passed around to other functions or stored in a data +structure. + +See also +........ + +PEP 255(2) - Simple Generators + + Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland. + Implemented mostly by Neil Schemenauer and Tim Peters, with other + fixes from the Python Labs crew. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0255 + + (2) https://www.python.org/dev/peps/pep-0255 + + +File: python.info, Node: PEP 237 Unifying Long Integers and Integers<2>, Next: PEP 238 Changing the Division Operator, Prev: PEP 255 Simple Generators<2>, Up: What’s New in Python 2 2 + +1.15.5 PEP 237: Unifying Long Integers and Integers +--------------------------------------------------- + +In recent versions, the distinction between regular integers, which are +32-bit values on most machines, and long integers, which can be of +arbitrary size, was becoming an annoyance. For example, on platforms +that support files larger than ‘2**32’ bytes, the ‘tell()’ method of +file objects has to return a long integer. However, there were various +bits of Python that expected plain integers and would raise an error if +a long integer was provided instead. For example, in Python 1.5, only +regular integers could be used as a slice index, and ‘'abc'[1L:]’ would +raise a *note TypeError: 192. exception with the message ‘slice index +must be int’. + +Python 2.2 will shift values from short to long integers as required. +The ‘L’ suffix is no longer needed to indicate a long integer literal, +as now the compiler will choose the appropriate type. (Using the ‘L’ +suffix will be discouraged in future 2.x versions of Python, triggering +a warning in Python 2.4, and probably dropped in Python 3.0.) Many +operations that used to raise an *note OverflowError: 960. will now +return a long integer as their result. For example: + + >>> 1234567890123 + 1234567890123L + >>> 2 ** 64 + 18446744073709551616L + +In most cases, integers and long integers will now be treated +identically. You can still distinguish them with the *note type(): 608. +built-in function, but that’s rarely needed. + +See also +........ + +PEP 237(1) - Unifying Long Integers and Integers + + Written by Moshe Zadka and Guido van Rossum. Implemented mostly by + Guido van Rossum. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0237 + + +File: python.info, Node: PEP 238 Changing the Division Operator, Next: Unicode Changes, Prev: PEP 237 Unifying Long Integers and Integers<2>, Up: What’s New in Python 2 2 + +1.15.6 PEP 238: Changing the Division Operator +---------------------------------------------- + +The most controversial change in Python 2.2 heralds the start of an +effort to fix an old design flaw that’s been in Python from the +beginning. Currently Python’s division operator, ‘/’, behaves like C’s +division operator when presented with two integer arguments: it returns +an integer result that’s truncated down when there would be a fractional +part. For example, ‘3/2’ is 1, not 1.5, and ‘(-1)/2’ is -1, not -0.5. +This means that the results of division can vary unexpectedly depending +on the type of the two operands and because Python is dynamically typed, +it can be difficult to determine the possible types of the operands. + +(The controversy is over whether this is `really' a design flaw, and +whether it’s worth breaking existing code to fix this. It’s caused +endless discussions on python-dev, and in July 2001 erupted into a storm +of acidly sarcastic postings on ‘comp.lang.python’. I won’t argue for +either side here and will stick to describing what’s implemented in 2.2. +Read PEP 238(1) for a summary of arguments and counter-arguments.) + +Because this change might break code, it’s being introduced very +gradually. Python 2.2 begins the transition, but the switch won’t be +complete until Python 3.0. + +First, I’ll borrow some terminology from PEP 238(2). “True division” is +the division that most non-programmers are familiar with: 3/2 is 1.5, +1/4 is 0.25, and so forth. “Floor division” is what Python’s ‘/’ +operator currently does when given integer operands; the result is the +floor of the value returned by true division. “Classic division” is the +current mixed behaviour of ‘/’; it returns the result of floor division +when the operands are integers, and returns the result of true division +when one of the operands is a floating-point number. + +Here are the changes 2.2 introduces: + + * A new operator, ‘//’, is the floor division operator. (Yes, we + know it looks like C++’s comment symbol.) ‘//’ `always' performs + floor division no matter what the types of its operands are, so ‘1 + // 2’ is 0 and ‘1.0 // 2.0’ is also 0.0. + + ‘//’ is always available in Python 2.2; you don’t need to enable it + using a ‘__future__’ statement. + + * By including a ‘from __future__ import division’ in a module, the + ‘/’ operator will be changed to return the result of true division, + so ‘1/2’ is 0.5. Without the ‘__future__’ statement, ‘/’ still + means classic division. The default meaning of ‘/’ will not change + until Python 3.0. + + * Classes can define methods called *note __truediv__(): 786. and + *note __floordiv__(): e35. to overload the two division operators. + At the C level, there are also slots in the *note PyNumberMethods: + d81. structure so extension types can define the two operators. + + * Python 2.2 supports some command-line arguments for testing whether + code will work with the changed division semantics. Running python + with ‘-Q warn’ will cause a warning to be issued whenever division + is applied to two integers. You can use this to find code that’s + affected by the change and fix it. By default, Python 2.2 will + simply perform classic division without a warning; the warning will + be turned on by default in Python 2.3. + +See also +........ + +PEP 238(3) - Changing the Division Operator + + Written by Moshe Zadka and Guido van Rossum. Implemented by Guido + van Rossum.. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0238 + + (2) https://www.python.org/dev/peps/pep-0238 + + (3) https://www.python.org/dev/peps/pep-0238 + + +File: python.info, Node: Unicode Changes, Next: PEP 227 Nested Scopes, Prev: PEP 238 Changing the Division Operator, Up: What’s New in Python 2 2 + +1.15.7 Unicode Changes +---------------------- + +Python’s Unicode support has been enhanced a bit in 2.2. Unicode +strings are usually stored as UCS-2, as 16-bit unsigned integers. +Python 2.2 can also be compiled to use UCS-4, 32-bit unsigned integers, +as its internal encoding by supplying ‘--enable-unicode=ucs4’ to the +configure script. (It’s also possible to specify ‘--disable-unicode’ to +completely disable Unicode support.) + +When built to use UCS-4 (a “wide Python”), the interpreter can natively +handle Unicode characters from U+000000 to U+110000, so the range of +legal values for the ‘unichr()’ function is expanded accordingly. Using +an interpreter compiled to use UCS-2 (a “narrow Python”), values greater +than 65535 will still cause ‘unichr()’ to raise a *note ValueError: 1fb. +exception. This is all described in PEP 261(1), “Support for ‘wide’ +Unicode characters”; consult it for further details. + +Another change is simpler to explain. Since their introduction, Unicode +strings have supported an ‘encode()’ method to convert the string to a +selected encoding such as UTF-8 or Latin-1. A symmetric +‘decode([*encoding*])’ method has been added to 8-bit strings (though +not to Unicode strings) in 2.2. ‘decode()’ assumes that the string is +in the specified encoding and decodes it, returning whatever is returned +by the codec. + +Using this new feature, codecs have been added for tasks not directly +related to Unicode. For example, codecs have been added for +uu-encoding, MIME’s base64 encoding, and compression with the *note +zlib: 144. module: + + >>> s = """Here is a lengthy piece of redundant, overly verbose, + ... and repetitive text. + ... """ + >>> data = s.encode('zlib') + >>> data + 'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...' + >>> data.decode('zlib') + 'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n' + >>> print s.encode('uu') + begin 666 + M2&5R92!I=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X* + + end + >>> "sheesh".encode('rot-13') + 'furrfu' + +To convert a class instance to Unicode, a ‘__unicode__()’ method can be +defined by a class, analogous to *note __str__(): e37. + +‘encode()’, ‘decode()’, and ‘__unicode__()’ were implemented by +Marc-André Lemburg. The changes to support using UCS-4 internally were +implemented by Fredrik Lundh and Martin von Löwis. + +See also +........ + +PEP 261(2) - Support for ‘wide’ Unicode characters + + Written by Paul Prescod. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0261 + + (2) https://www.python.org/dev/peps/pep-0261 + + +File: python.info, Node: PEP 227 Nested Scopes, Next: New and Improved Modules<3>, Prev: Unicode Changes, Up: What’s New in Python 2 2 + +1.15.8 PEP 227: Nested Scopes +----------------------------- + +In Python 2.1, statically nested scopes were added as an optional +feature, to be enabled by a ‘from __future__ import nested_scopes’ +directive. In 2.2 nested scopes no longer need to be specially enabled, +and are now always present. The rest of this section is a copy of the +description of nested scopes from my “What’s New in Python 2.1” +document; if you read it when 2.1 came out, you can skip the rest of +this section. + +The largest change introduced in Python 2.1, and made complete in 2.2, +is to Python’s scoping rules. In Python 2.0, at any given time there +are at most three namespaces used to look up variable names: local, +module-level, and the built-in namespace. This often surprised people +because it didn’t match their intuitive expectations. For example, a +nested recursive function definition doesn’t work: + + def f(): + ... + def g(value): + ... + return g(value-1) + 1 + ... + +The function ‘g()’ will always raise a *note NameError: d7b. exception, +because the binding of the name ‘g’ isn’t in either its local namespace +or in the module-level namespace. This isn’t much of a problem in +practice (how often do you recursively define interior functions like +this?), but this also made using the *note lambda: c2f. expression +clumsier, and this was a problem in practice. In code which uses +‘lambda’ you can often find local variables being copied by passing them +as the default values of arguments. + + def find(self, name): + "Return list of any entries equal to 'name'" + L = filter(lambda x, name=name: x == name, + self.list_attribute) + return L + +The readability of Python code written in a strongly functional style +suffers greatly as a result. + +The most significant change to Python 2.2 is that static scoping has +been added to the language to fix this problem. As a first effect, the +‘name=name’ default argument is now unnecessary in the above example. +Put simply, when a given variable name is not assigned a value within a +function (by an assignment, or the *note def: dbe, *note class: c6a, or +*note import: c1d. statements), references to the variable will be +looked up in the local namespace of the enclosing scope. A more +detailed explanation of the rules, and a dissection of the +implementation, can be found in the PEP. + +This change may cause some compatibility problems for code where the +same variable name is used both at the module level and as a local +variable within a function that contains further function definitions. +This seems rather unlikely though, since such code would have been +pretty confusing to read in the first place. + +One side effect of the change is that the ‘from module import *’ and +‘exec’ statements have been made illegal inside a function scope under +certain conditions. The Python reference manual has said all along that +‘from module import *’ is only legal at the top level of a module, but +the CPython interpreter has never enforced this before. As part of the +implementation of nested scopes, the compiler which turns Python source +into bytecodes has to generate different code to access variables in a +containing scope. ‘from module import *’ and ‘exec’ make it impossible +for the compiler to figure this out, because they add names to the local +namespace that are unknowable at compile time. Therefore, if a function +contains function definitions or *note lambda: c2f. expressions with +free variables, the compiler will flag this by raising a *note +SyntaxError: 458. exception. + +To make the preceding explanation a bit clearer, here’s an example: + + x = 1 + def f(): + # The next line is a syntax error + exec 'x=2' + def g(): + return x + +Line 4 containing the ‘exec’ statement is a syntax error, since ‘exec’ +would define a new local variable named ‘x’ whose value should be +accessed by ‘g()’. + +This shouldn’t be much of a limitation, since ‘exec’ is rarely used in +most Python code (and when it is used, it’s often a sign of a poor +design anyway). + +See also +........ + +PEP 227(1) - Statically Nested Scopes + + Written and implemented by Jeremy Hylton. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0227 + + +File: python.info, Node: New and Improved Modules<3>, Next: Interpreter Changes and Fixes, Prev: PEP 227 Nested Scopes, Up: What’s New in Python 2 2 + +1.15.9 New and Improved Modules +------------------------------- + + * The ‘xmlrpclib’ module was contributed to the standard library by + Fredrik Lundh, providing support for writing XML-RPC clients. + XML-RPC is a simple remote procedure call protocol built on top of + HTTP and XML. For example, the following snippet retrieves a list + of RSS channels from the O’Reilly Network, and then lists the + recent headlines for one channel: + + import xmlrpclib + s = xmlrpclib.Server( + 'http://www.oreillynet.com/meerkat/xml-rpc/server.php') + channels = s.meerkat.getChannels() + # channels is a list of dictionaries, like this: + # [{'id': 4, 'title': 'Freshmeat Daily News'} + # {'id': 190, 'title': '32Bits Online'}, + # {'id': 4549, 'title': '3DGamers'}, ... ] + + # Get the items for one channel + items = s.meerkat.getItems( {'channel': 4} ) + + # 'items' is another list of dictionaries, like this: + # [{'link': 'http://freshmeat.net/releases/52719/', + # 'description': 'A utility which converts HTML to XSL FO.', + # 'title': 'html2fo 0.3 (Default)'}, ... ] + + The ‘SimpleXMLRPCServer’ module makes it easy to create + straightforward XML-RPC servers. See + ‘http://xmlrpc.scripting.com/’ for more information about XML-RPC. + + * The new *note hmac: 8f. module implements the HMAC algorithm + described by RFC 2104(1). (Contributed by Gerhard Häring.) + + * Several functions that originally returned lengthy tuples now + return pseudo-sequences that still behave like tuples but also have + mnemonic attributes such as memberst_mtime or ‘tm_year’. The + enhanced functions include *note stat(): f4, ‘fstat()’, + ‘statvfs()’, and ‘fstatvfs()’ in the *note os: c4. module, and + ‘localtime()’, ‘gmtime()’, and ‘strptime()’ in the *note time: 10a. + module. + + For example, to obtain a file’s size using the old tuples, you’d + end up writing something like ‘file_size = + os.stat(filename)[stat.ST_SIZE]’, but now this can be written more + clearly as ‘file_size = os.stat(filename).st_size’. + + The original patch for this feature was contributed by Nick + Mathewson. + + * The Python profiler has been extensively reworked and various + errors in its output have been corrected. (Contributed by Fred L. + Drake, Jr. and Tim Peters.) + + * The *note socket: ef. module can be compiled to support IPv6; + specify the ‘--enable-ipv6’ option to Python’s configure script. + (Contributed by Jun-ichiro “itojun” Hagino.) + + * Two new format characters were added to the *note struct: f8. + module for 64-bit integers on platforms that support the C ‘long + long’ type. ‘q’ is for a signed 64-bit integer, and ‘Q’ is for an + unsigned one. The value is returned in Python’s long integer type. + (Contributed by Tim Peters.) + + * In the interpreter’s interactive mode, there’s a new built-in + function *note help(): 572. that uses the *note pydoc: d9. module + introduced in Python 2.1 to provide interactive help. + ‘help(object)’ displays any available help text about `object'. + *note help(): 572. with no argument puts you in an online help + utility, where you can enter the names of functions, classes, or + modules to read their help text. (Contributed by Guido van Rossum, + using Ka-Ping Yee’s *note pydoc: d9. module.) + + * Various bugfixes and performance improvements have been made to the + SRE engine underlying the *note re: dd. module. For example, the + *note re.sub(): 47b. and *note re.split(): 3ca. functions have been + rewritten in C. Another contributed patch speeds up certain Unicode + character ranges by a factor of two, and a new ‘finditer()’ method + that returns an iterator over all the non-overlapping matches in a + given string. (SRE is maintained by Fredrik Lundh. The BIGCHARSET + patch was contributed by Martin von Löwis.) + + * The *note smtplib: ed. module now supports RFC 2487(2), “Secure + SMTP over TLS”, so it’s now possible to encrypt the SMTP traffic + between a Python program and the mail transport agent being handed + a message. *note smtplib: ed. also supports SMTP authentication. + (Contributed by Gerhard Häring.) + + * The *note imaplib: 98. module, maintained by Piers Lauder, has + support for several new extensions: the NAMESPACE extension defined + in RFC 2342(3), SORT, GETACL and SETACL. (Contributed by Anthony + Baxter and Michel Pelletier.) + + * The ‘rfc822’ module’s parsing of email addresses is now compliant + with RFC 2822(4), an update to RFC 822(5). (The module’s name is + `not' going to be changed to ‘rfc2822’.) A new package, *note + email: 69, has also been added for parsing and generating e-mail + messages. (Contributed by Barry Warsaw, and arising out of his + work on Mailman.) + + * The *note difflib: 37. module now contains a new ‘Differ’ class for + producing human-readable lists of changes (a “delta”) between two + sequences of lines of text. There are also two generator + functions, ‘ndiff()’ and ‘restore()’, which respectively return a + delta from two sequences, or one of the original sequences from a + delta. (Grunt work contributed by David Goodger, from ndiff.py + code by Tim Peters who then did the generatorization.) + + * New constants ‘ascii_letters’, ‘ascii_lowercase’, and + ‘ascii_uppercase’ were added to the *note string: f6. module. + There were several modules in the standard library that used + ‘string.letters’ to mean the ranges A-Za-z, but that assumption is + incorrect when locales are in use, because ‘string.letters’ varies + depending on the set of legal characters defined by the current + locale. The buggy modules have all been fixed to use + ‘ascii_letters’ instead. (Reported by an unknown person; fixed by + Fred L. Drake, Jr.) + + * The *note mimetypes: b2. module now makes it easier to use + alternative MIME-type databases by the addition of a ‘MimeTypes’ + class, which takes a list of filenames to be parsed. (Contributed + by Fred L. Drake, Jr.) + + * A ‘Timer’ class was added to the *note threading: 109. module that + allows scheduling an activity to happen at some future time. + (Contributed by Itamar Shtull-Trauring.) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2104.html + + (2) https://tools.ietf.org/html/rfc2487.html + + (3) https://tools.ietf.org/html/rfc2342.html + + (4) https://tools.ietf.org/html/rfc2822.html + + (5) https://tools.ietf.org/html/rfc822.html + + +File: python.info, Node: Interpreter Changes and Fixes, Next: Other Changes and Fixes<3>, Prev: New and Improved Modules<3>, Up: What’s New in Python 2 2 + +1.15.10 Interpreter Changes and Fixes +------------------------------------- + +Some of the changes only affect people who deal with the Python +interpreter at the C level because they’re writing Python extension +modules, embedding the interpreter, or just hacking on the interpreter +itself. If you only write Python code, none of the changes described +here will affect you very much. + + * Profiling and tracing functions can now be implemented in C, which + can operate at much higher speeds than Python-based functions and + should reduce the overhead of profiling and tracing. This will be + of interest to authors of development environments for Python. Two + new C functions were added to Python’s API, *note + PyEval_SetProfile(): e3b. and *note PyEval_SetTrace(): e3c. The + existing *note sys.setprofile(): e3d. and *note sys.settrace(): + e3e. functions still exist, and have simply been changed to use the + new C-level interface. (Contributed by Fred L. Drake, Jr.) + + * Another low-level API, primarily of interest to implementors of + Python debuggers and development tools, was added. *note + PyInterpreterState_Head(): e3f. and *note + PyInterpreterState_Next(): e40. let a caller walk through all the + existing interpreter objects; *note + PyInterpreterState_ThreadHead(): e41. and *note + PyThreadState_Next(): e42. allow looping over all the thread states + for a given interpreter. (Contributed by David Beazley.) + + * The C-level interface to the garbage collector has been changed to + make it easier to write extension types that support garbage + collection and to debug misuses of the functions. Various + functions have slightly different semantics, so a bunch of + functions had to be renamed. Extensions that use the old API will + still compile but will `not' participate in garbage collection, so + updating them for 2.2 should be considered fairly high priority. + + To upgrade an extension module to the new API, perform the + following steps: + + * Rename ‘Py_TPFLAGS_GC()’ to ‘PyTPFLAGS_HAVE_GC()’. + + * + Use *note PyObject_GC_New(): 2d4. or *note PyObject_GC_NewVar(): 2d5. to allocate + + objects, and *note PyObject_GC_Del(): e43. to deallocate them. + + * + Rename ‘PyObject_GC_Init()’ to *note PyObject_GC_Track(): e44. and + + ‘PyObject_GC_Fini()’ to *note PyObject_GC_UnTrack(): e45. + + * Remove ‘PyGC_HEAD_SIZE()’ from object size calculations. + + * Remove calls to ‘PyObject_AS_GC()’ and ‘PyObject_FROM_GC()’. + + * A new ‘et’ format sequence was added to *note PyArg_ParseTuple(): + 26a.; ‘et’ takes both a parameter and an encoding name, and + converts the parameter to the given encoding if the parameter turns + out to be a Unicode string, or leaves it alone if it’s an 8-bit + string, assuming it to already be in the desired encoding. This + differs from the ‘es’ format character, which assumes that 8-bit + strings are in Python’s default ASCII encoding and converts them to + the specified new encoding. (Contributed by M.-A. Lemburg, and + used for the MBCS support on Windows described in the following + section.) + + * A different argument parsing function, *note PyArg_UnpackTuple(): + e46, has been added that’s simpler and presumably faster. Instead + of specifying a format string, the caller simply gives the minimum + and maximum number of arguments expected, and a set of pointers to + *note PyObject*: 4ba. variables that will be filled in with + argument values. + + * Two new flags *note METH_NOARGS: e18. and *note METH_O: e47. are + available in method definition tables to simplify implementation of + methods with no arguments or a single untyped argument. Calling + such methods is more efficient than calling a corresponding method + that uses *note METH_VARARGS: e48. Also, the old ‘METH_OLDARGS’ + style of writing C methods is now officially deprecated. + + * Two new wrapper functions, *note PyOS_snprintf(): e49. and *note + PyOS_vsnprintf(): e4a. were added to provide cross-platform + implementations for the relatively new ‘snprintf()’ and + ‘vsnprintf()’ C lib APIs. In contrast to the standard ‘sprintf()’ + and ‘vsprintf()’ functions, the Python versions check the bounds of + the buffer used to protect against buffer overruns. (Contributed + by M.-A. Lemburg.) + + * The *note _PyTuple_Resize(): e4b. function has lost an unused + parameter, so now it takes 2 parameters instead of 3. The third + argument was never used, and can simply be discarded when porting + code from earlier versions to Python 2.2. + + +File: python.info, Node: Other Changes and Fixes<3>, Next: Acknowledgements<6>, Prev: Interpreter Changes and Fixes, Up: What’s New in Python 2 2 + +1.15.11 Other Changes and Fixes +------------------------------- + +As usual there were a bunch of other improvements and bugfixes scattered +throughout the source tree. A search through the CVS change logs finds +there were 527 patches applied and 683 bugs fixed between Python 2.1 and +2.2; 2.2.1 applied 139 patches and fixed 143 bugs; 2.2.2 applied 106 +patches and fixed 82 bugs. These figures are likely to be +underestimates. + +Some of the more notable changes are: + + * The code for the MacOS port for Python, maintained by Jack Jansen, + is now kept in the main Python CVS tree, and many changes have been + made to support MacOS X. + + The most significant change is the ability to build Python as a + framework, enabled by supplying the ‘--enable-framework’ option to + the configure script when compiling Python. According to Jack + Jansen, “This installs a self-contained Python installation plus + the OS X framework “glue” into + ‘/Library/Frameworks/Python.framework’ (or another location of + choice). For now there is little immediate added benefit to this + (actually, there is the disadvantage that you have to change your + PATH to be able to find Python), but it is the basis for creating a + full-blown Python application, porting the MacPython IDE, possibly + using Python as a standard OSA scripting language and much more.” + + Most of the MacPython toolbox modules, which interface to MacOS + APIs such as windowing, QuickTime, scripting, etc. have been + ported to OS X, but they’ve been left commented out in ‘setup.py’. + People who want to experiment with these modules can uncomment them + manually. + + * Keyword arguments passed to built-in functions that don’t take them + now cause a *note TypeError: 192. exception to be raised, with the + message “`function' takes no keyword arguments”. + + * Weak references, added in Python 2.1 as an extension module, are + now part of the core because they’re used in the implementation of + new-style classes. The *note ReferenceError: e4d. exception has + therefore moved from the *note weakref: 128. module to become a + built-in exception. + + * A new script, ‘Tools/scripts/cleanfuture.py’ by Tim Peters, + automatically removes obsolete ‘__future__’ statements from Python + source code. + + * An additional `flags' argument has been added to the built-in + function *note compile(): 1b4, so the behaviour of ‘__future__’ + statements can now be correctly observed in simulated shells, such + as those presented by IDLE and other development environments. + This is described in PEP 264(1). (Contributed by Michael Hudson.) + + * The new license introduced with Python 1.6 wasn’t GPL-compatible. + This is fixed by some minor textual changes to the 2.2 license, so + it’s now legal to embed Python inside a GPLed program again. Note + that Python itself is not GPLed, but instead is under a license + that’s essentially equivalent to the BSD license, same as it always + was. The license changes were also applied to the Python 2.0.1 and + 2.1.1 releases. + + * When presented with a Unicode filename on Windows, Python will now + convert it to an MBCS encoded string, as used by the Microsoft file + APIs. As MBCS is explicitly used by the file APIs, Python’s choice + of ASCII as the default encoding turns out to be an annoyance. On + Unix, the locale’s character set is used if + ‘locale.nl_langinfo(CODESET)’ is available. (Windows support was + contributed by Mark Hammond with assistance from Marc-André + Lemburg. Unix support was added by Martin von Löwis.) + + * Large file support is now enabled on Windows. (Contributed by Tim + Peters.) + + * The ‘Tools/scripts/ftpmirror.py’ script now parses a ‘.netrc’ file, + if you have one. (Contributed by Mike Romberg.) + + * Some features of the object returned by the ‘xrange()’ function are + now deprecated, and trigger warnings when they’re accessed; they’ll + disappear in Python 2.3. ‘xrange’ objects tried to pretend they + were full sequence types by supporting slicing, sequence + multiplication, and the *note in: 7a3. operator, but these features + were rarely used and therefore buggy. The ‘tolist()’ method and + the ‘start’, ‘stop’, and ‘step’ attributes are also being + deprecated. At the C level, the fourth argument to the + ‘PyRange_New()’ function, ‘repeat’, has also been deprecated. + + * There were a bunch of patches to the dictionary implementation, + mostly to fix potential core dumps if a dictionary contains objects + that sneakily changed their hash value, or mutated the dictionary + they were contained in. For a while python-dev fell into a gentle + rhythm of Michael Hudson finding a case that dumped core, Tim + Peters fixing the bug, Michael finding another case, and round and + round it went. + + * On Windows, Python can now be compiled with Borland C thanks to a + number of patches contributed by Stephen Hansen, though the result + isn’t fully functional yet. (But this `is' progress…) + + * Another Windows enhancement: Wise Solutions generously offered + PythonLabs use of their InstallerMaster 8.1 system. Earlier + PythonLabs Windows installers used Wise 5.0a, which was beginning + to show its age. (Packaged up by Tim Peters.) + + * Files ending in ‘.pyw’ can now be imported on Windows. ‘.pyw’ is a + Windows-only thing, used to indicate that a script needs to be run + using PYTHONW.EXE instead of PYTHON.EXE in order to prevent a DOS + console from popping up to display the output. This patch makes it + possible to import such scripts, in case they’re also usable as + modules. (Implemented by David Bolen.) + + * On platforms where Python uses the C ‘dlopen()’ function to load + extension modules, it’s now possible to set the flags used by + ‘dlopen()’ using the *note sys.getdlopenflags(): e4e. and *note + sys.setdlopenflags(): a66. functions. (Contributed by Bram Stolk.) + + * The *note pow(): 19a. built-in function no longer supports 3 + arguments when floating-point numbers are supplied. ‘pow(x, y, z)’ + returns ‘(x**y) % z’, but this is never useful for floating point + numbers, and the final result varies unpredictably depending on the + platform. A call such as ‘pow(2.0, 8.0, 7.0)’ will now raise a + *note TypeError: 192. exception. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0264 + + +File: python.info, Node: Acknowledgements<6>, Prev: Other Changes and Fixes<3>, Up: What’s New in Python 2 2 + +1.15.12 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Fred Bremmer, Keith Briggs, Andrew Dalke, Fred L. Drake, Jr., +Carel Fellinger, David Goodger, Mark Hammond, Stephen Hansen, Michael +Hudson, Jack Jansen, Marc-André Lemburg, Martin von Löwis, Fredrik +Lundh, Michael McLay, Nick Mathewson, Paul Moore, Gustavo Niemeyer, Don +O’Donnell, Joonas Paalasma, Tim Peters, Jens Quade, Tom Reinhardt, Neil +Schemenauer, Guido van Rossum, Greg Ward, Edward Welbourne. + + +File: python.info, Node: What’s New in Python 2 1, Next: What’s New in Python 2 0, Prev: What’s New in Python 2 2, Up: What’s New in Python + +1.16 What’s New in Python 2.1 +============================= + + +Author: A.M. Kuchling + +* Menu: + +* Introduction: Introduction<2>. +* PEP 227; Nested Scopes: PEP 227 Nested Scopes<2>. +* PEP 236; __future__ Directives: PEP 236 __future__ Directives. +* PEP 207; Rich Comparisons: PEP 207 Rich Comparisons. +* PEP 230; Warning Framework: PEP 230 Warning Framework. +* PEP 229; New Build System: PEP 229 New Build System. +* PEP 205; Weak References: PEP 205 Weak References. +* PEP 232; Function Attributes: PEP 232 Function Attributes. +* PEP 235; Importing Modules on Case-Insensitive Platforms: PEP 235 Importing Modules on Case-Insensitive Platforms. +* PEP 217; Interactive Display Hook: PEP 217 Interactive Display Hook. +* PEP 208; New Coercion Model: PEP 208 New Coercion Model. +* PEP 241; Metadata in Python Packages: PEP 241 Metadata in Python Packages. +* New and Improved Modules: New and Improved Modules<4>. +* Other Changes and Fixes: Other Changes and Fixes<4>. +* Acknowledgements: Acknowledgements<7>. + + +File: python.info, Node: Introduction<2>, Next: PEP 227 Nested Scopes<2>, Up: What’s New in Python 2 1 + +1.16.1 Introduction +------------------- + +This article explains the new features in Python 2.1. While there +aren’t as many changes in 2.1 as there were in Python 2.0, there are +still some pleasant surprises in store. 2.1 is the first release to be +steered through the use of Python Enhancement Proposals, or PEPs, so +most of the sizable changes have accompanying PEPs that provide more +complete documentation and a design rationale for the change. This +article doesn’t attempt to document the new features completely, but +simply provides an overview of the new features for Python programmers. +Refer to the Python 2.1 documentation, or to the specific PEP, for more +details about any new feature that particularly interests you. + +One recent goal of the Python development team has been to accelerate +the pace of new releases, with a new release coming every 6 to 9 months. +2.1 is the first release to come out at this faster pace, with the first +alpha appearing in January, 3 months after the final version of 2.0 was +released. + +The final release of Python 2.1 was made on April 17, 2001. + + +File: python.info, Node: PEP 227 Nested Scopes<2>, Next: PEP 236 __future__ Directives, Prev: Introduction<2>, Up: What’s New in Python 2 1 + +1.16.2 PEP 227: Nested Scopes +----------------------------- + +The largest change in Python 2.1 is to Python’s scoping rules. In +Python 2.0, at any given time there are at most three namespaces used to +look up variable names: local, module-level, and the built-in namespace. +This often surprised people because it didn’t match their intuitive +expectations. For example, a nested recursive function definition +doesn’t work: + + def f(): + ... + def g(value): + ... + return g(value-1) + 1 + ... + +The function ‘g()’ will always raise a *note NameError: d7b. exception, +because the binding of the name ‘g’ isn’t in either its local namespace +or in the module-level namespace. This isn’t much of a problem in +practice (how often do you recursively define interior functions like +this?), but this also made using the *note lambda: c2f. expression +clumsier, and this was a problem in practice. In code which uses *note +lambda: c2f. you can often find local variables being copied by passing +them as the default values of arguments. + + def find(self, name): + "Return list of any entries equal to 'name'" + L = filter(lambda x, name=name: x == name, + self.list_attribute) + return L + +The readability of Python code written in a strongly functional style +suffers greatly as a result. + +The most significant change to Python 2.1 is that static scoping has +been added to the language to fix this problem. As a first effect, the +‘name=name’ default argument is now unnecessary in the above example. +Put simply, when a given variable name is not assigned a value within a +function (by an assignment, or the *note def: dbe, *note class: c6a, or +*note import: c1d. statements), references to the variable will be +looked up in the local namespace of the enclosing scope. A more +detailed explanation of the rules, and a dissection of the +implementation, can be found in the PEP. + +This change may cause some compatibility problems for code where the +same variable name is used both at the module level and as a local +variable within a function that contains further function definitions. +This seems rather unlikely though, since such code would have been +pretty confusing to read in the first place. + +One side effect of the change is that the ‘from module import *’ and +‘exec’ statements have been made illegal inside a function scope under +certain conditions. The Python reference manual has said all along that +‘from module import *’ is only legal at the top level of a module, but +the CPython interpreter has never enforced this before. As part of the +implementation of nested scopes, the compiler which turns Python source +into bytecodes has to generate different code to access variables in a +containing scope. ‘from module import *’ and ‘exec’ make it impossible +for the compiler to figure this out, because they add names to the local +namespace that are unknowable at compile time. Therefore, if a function +contains function definitions or *note lambda: c2f. expressions with +free variables, the compiler will flag this by raising a *note +SyntaxError: 458. exception. + +To make the preceding explanation a bit clearer, here’s an example: + + x = 1 + def f(): + # The next line is a syntax error + exec 'x=2' + def g(): + return x + +Line 4 containing the ‘exec’ statement is a syntax error, since ‘exec’ +would define a new local variable named ‘x’ whose value should be +accessed by ‘g()’. + +This shouldn’t be much of a limitation, since ‘exec’ is rarely used in +most Python code (and when it is used, it’s often a sign of a poor +design anyway). + +Compatibility concerns have led to nested scopes being introduced +gradually; in Python 2.1, they aren’t enabled by default, but can be +turned on within a module by using a future statement as described in +PEP 236(1). (See the following section for further discussion of PEP +236(2).) In Python 2.2, nested scopes will become the default and there +will be no way to turn them off, but users will have had all of 2.1’s +lifetime to fix any breakage resulting from their introduction. + +See also +........ + +PEP 227(3) - Statically Nested Scopes + + Written and implemented by Jeremy Hylton. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0236 + + (2) https://www.python.org/dev/peps/pep-0236 + + (3) https://www.python.org/dev/peps/pep-0227 + + +File: python.info, Node: PEP 236 __future__ Directives, Next: PEP 207 Rich Comparisons, Prev: PEP 227 Nested Scopes<2>, Up: What’s New in Python 2 1 + +1.16.3 PEP 236: __future__ Directives +------------------------------------- + +The reaction to nested scopes was widespread concern about the dangers +of breaking code with the 2.1 release, and it was strong enough to make +the Pythoneers take a more conservative approach. This approach +consists of introducing a convention for enabling optional functionality +in release N that will become compulsory in release N+1. + +The syntax uses a ‘from...import’ statement using the reserved module +name *note __future__: 0. Nested scopes can be enabled by the following +statement: + + from __future__ import nested_scopes + +While it looks like a normal *note import: c1d. statement, it’s not; +there are strict rules on where such a future statement can be put. +They can only be at the top of a module, and must precede any Python +code or regular ‘import’ statements. This is because such statements +can affect how the Python bytecode compiler parses code and generates +bytecode, so they must precede any statement that will result in +bytecodes being produced. + +See also +........ + +PEP 236(1) - Back to the *note __future__: 0. + + Written by Tim Peters, and primarily implemented by Jeremy Hylton. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0236 + + +File: python.info, Node: PEP 207 Rich Comparisons, Next: PEP 230 Warning Framework, Prev: PEP 236 __future__ Directives, Up: What’s New in Python 2 1 + +1.16.4 PEP 207: Rich Comparisons +-------------------------------- + +In earlier versions, Python’s support for implementing comparisons on +user-defined classes and extension types was quite simple. Classes +could implement a ‘__cmp__()’ method that was given two instances of a +class, and could only return 0 if they were equal or +1 or -1 if they +weren’t; the method couldn’t raise an exception or return anything other +than a Boolean value. Users of Numeric Python often found this model +too weak and restrictive, because in the number-crunching programs that +numeric Python is used for, it would be more useful to be able to +perform elementwise comparisons of two matrices, returning a matrix +containing the results of a given comparison for each element. If the +two matrices are of different sizes, then the compare has to be able to +raise an exception to signal the error. + +In Python 2.1, rich comparisons were added in order to support this +need. Python classes can now individually overload each of the ‘<’, +‘<=’, ‘>’, ‘>=’, ‘==’, and ‘!=’ operations. The new magic method names +are: + +Operation Method name + +------------------------------------- + +‘<’ *note __lt__(): c34. + + +‘<=’ *note __le__(): ca0. + + +‘>’ *note __gt__(): ca1. + + +‘>=’ *note __ge__(): ca2. + + +‘==’ *note __eq__(): 33f. + + +‘!=’ *note __ne__(): e56. + + +(The magic methods are named after the corresponding Fortran operators +‘.LT.’. ‘.LE.’, &c. Numeric programmers are almost certainly quite +familiar with these names and will find them easy to remember.) + +Each of these magic methods is of the form ‘method(self, other)’, where +‘self’ will be the object on the left-hand side of the operator, while +‘other’ will be the object on the right-hand side. For example, the +expression ‘A < B’ will cause ‘A.__lt__(B)’ to be called. + +Each of these magic methods can return anything at all: a Boolean, a +matrix, a list, or any other Python object. Alternatively they can +raise an exception if the comparison is impossible, inconsistent, or +otherwise meaningless. + +The built-in ‘cmp(A,B)’ function can use the rich comparison machinery, +and now accepts an optional argument specifying which comparison +operation to use; this is given as one of the strings ‘"<"’, ‘"<="’, +‘">"’, ‘">="’, ‘"=="’, or ‘"!="’. If called without the optional third +argument, ‘cmp()’ will only return -1, 0, or +1 as in previous versions +of Python; otherwise it will call the appropriate method and can return +any Python object. + +There are also corresponding changes of interest to C programmers; +there’s a new slot ‘tp_richcmp’ in type objects and an API for +performing a given rich comparison. I won’t cover the C API here, but +will refer you to PEP 207(1), or to 2.1’s C API documentation, for the +full list of related functions. + +See also +........ + +PEP 207(2) - Rich Comparisons + + Written by Guido van Rossum, heavily based on earlier work by David + Ascher, and implemented by Guido van Rossum. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0207 + + (2) https://www.python.org/dev/peps/pep-0207 + + +File: python.info, Node: PEP 230 Warning Framework, Next: PEP 229 New Build System, Prev: PEP 207 Rich Comparisons, Up: What’s New in Python 2 1 + +1.16.5 PEP 230: Warning Framework +--------------------------------- + +Over its 10 years of existence, Python has accumulated a certain number +of obsolete modules and features along the way. It’s difficult to know +when a feature is safe to remove, since there’s no way of knowing how +much code uses it — perhaps no programs depend on the feature, or +perhaps many do. To enable removing old features in a more structured +way, a warning framework was added. When the Python developers want to +get rid of a feature, it will first trigger a warning in the next +version of Python. The following Python version can then drop the +feature, and users will have had a full release cycle to remove uses of +the old feature. + +Python 2.1 adds the warning framework to be used in this scheme. It +adds a *note warnings: 126. module that provide functions to issue +warnings, and to filter out warnings that you don’t want to be +displayed. Third-party modules can also use this framework to deprecate +old features that they no longer wish to support. + +For example, in Python 2.1 the ‘regex’ module is deprecated, so +importing it causes a warning to be printed: + + >>> import regex + __main__:1: DeprecationWarning: the regex module + is deprecated; please use the re module + >>> + +Warnings can be issued by calling the *note warnings.warn(): e58. +function: + + warnings.warn("feature X no longer supported") + +The first parameter is the warning message; an additional optional +parameters can be used to specify a particular warning category. + +Filters can be added to disable certain warnings; a regular expression +pattern can be applied to the message or to the module name in order to +suppress a warning. For example, you may have a program that uses the +‘regex’ module and not want to spare the time to convert it to use the +*note re: dd. module right now. The warning can be suppressed by +calling + + import warnings + warnings.filterwarnings(action = 'ignore', + message='.*regex module is deprecated', + category=DeprecationWarning, + module = '__main__') + +This adds a filter that will apply only to warnings of the class *note +DeprecationWarning: 278. triggered in the *note __main__: 1. module, and +applies a regular expression to only match the message about the ‘regex’ +module being deprecated, and will cause such warnings to be ignored. +Warnings can also be printed only once, printed every time the offending +code is executed, or turned into exceptions that will cause the program +to stop (unless the exceptions are caught in the usual way, of course). + +Functions were also added to Python’s C API for issuing warnings; refer +to PEP 230 or to Python’s API documentation for the details. + +See also +........ + +PEP 5(1) - Guidelines for Language Evolution + + Written by Paul Prescod, to specify procedures to be followed when + removing old features from Python. The policy described in this + PEP hasn’t been officially adopted, but the eventual policy + probably won’t be too different from Prescod’s proposal. + +PEP 230(2) - Warning Framework + + Written and implemented by Guido van Rossum. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0005 + + (2) https://www.python.org/dev/peps/pep-0230 + + +File: python.info, Node: PEP 229 New Build System, Next: PEP 205 Weak References, Prev: PEP 230 Warning Framework, Up: What’s New in Python 2 1 + +1.16.6 PEP 229: New Build System +-------------------------------- + +When compiling Python, the user had to go in and edit the +‘Modules/Setup’ file in order to enable various additional modules; the +default set is relatively small and limited to modules that compile on +most Unix platforms. This means that on Unix platforms with many more +features, most notably Linux, Python installations often don’t contain +all useful modules they could. + +Python 2.0 added the Distutils, a set of modules for distributing and +installing extensions. In Python 2.1, the Distutils are used to compile +much of the standard library of extension modules, autodetecting which +ones are supported on the current machine. It’s hoped that this will +make Python installations easier and more featureful. + +Instead of having to edit the ‘Modules/Setup’ file in order to enable +modules, a ‘setup.py’ script in the top directory of the Python source +distribution is run at build time, and attempts to discover which +modules can be enabled by examining the modules and header files on the +system. If a module is configured in ‘Modules/Setup’, the ‘setup.py’ +script won’t attempt to compile that module and will defer to the +‘Modules/Setup’ file’s contents. This provides a way to specific any +strange command-line flags or libraries that are required for a specific +platform. + +In another far-reaching change to the build mechanism, Neil Schemenauer +restructured things so Python now uses a single makefile that isn’t +recursive, instead of makefiles in the top directory and in each of the +‘Python/’, ‘Parser/’, ‘Objects/’, and ‘Modules/’ subdirectories. This +makes building Python faster and also makes hacking the Makefiles +clearer and simpler. + +See also +........ + +PEP 229(1) - Using Distutils to Build Python + + Written and implemented by A.M. Kuchling. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0229 + + +File: python.info, Node: PEP 205 Weak References, Next: PEP 232 Function Attributes, Prev: PEP 229 New Build System, Up: What’s New in Python 2 1 + +1.16.7 PEP 205: Weak References +------------------------------- + +Weak references, available through the *note weakref: 128. module, are a +minor but useful new data type in the Python programmer’s toolbox. + +Storing a reference to an object (say, in a dictionary or a list) has +the side effect of keeping that object alive forever. There are a few +specific cases where this behaviour is undesirable, object caches being +the most common one, and another being circular references in data +structures such as trees. + +For example, consider a memoizing function that caches the results of +another function ‘f(x)’ by storing the function’s argument and its +result in a dictionary: + + _cache = {} + def memoize(x): + if _cache.has_key(x): + return _cache[x] + + retval = f(x) + + # Cache the returned object + _cache[x] = retval + + return retval + +This version works for simple things such as integers, but it has a side +effect; the ‘_cache’ dictionary holds a reference to the return values, +so they’ll never be deallocated until the Python process exits and +cleans up. This isn’t very noticeable for integers, but if ‘f()’ +returns an object, or a data structure that takes up a lot of memory, +this can be a problem. + +Weak references provide a way to implement a cache that won’t keep +objects alive beyond their time. If an object is only accessible +through weak references, the object will be deallocated and the weak +references will now indicate that the object it referred to no longer +exists. A weak reference to an object `obj' is created by calling ‘wr = +weakref.ref(obj)’. The object being referred to is returned by calling +the weak reference as if it were a function: ‘wr()’. It will return the +referenced object, or ‘None’ if the object no longer exists. + +This makes it possible to write a ‘memoize()’ function whose cache +doesn’t keep objects alive, by storing weak references in the cache. + + _cache = {} + def memoize(x): + if _cache.has_key(x): + obj = _cache[x]() + # If weak reference object still exists, + # return it + if obj is not None: return obj + + retval = f(x) + + # Cache a weak reference + _cache[x] = weakref.ref(retval) + + return retval + +The *note weakref: 128. module also allows creating proxy objects which +behave like weak references — an object referenced only by proxy objects +is deallocated – but instead of requiring an explicit call to retrieve +the object, the proxy transparently forwards all operations to the +object as long as the object still exists. If the object is +deallocated, attempting to use a proxy will cause a +‘weakref.ReferenceError’ exception to be raised. + + proxy = weakref.proxy(obj) + proxy.attr # Equivalent to obj.attr + proxy.meth() # Equivalent to obj.meth() + del obj + proxy.attr # raises weakref.ReferenceError + +See also +........ + +PEP 205(1) - Weak References + + Written and implemented by Fred L. Drake, Jr. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0205 + + +File: python.info, Node: PEP 232 Function Attributes, Next: PEP 235 Importing Modules on Case-Insensitive Platforms, Prev: PEP 205 Weak References, Up: What’s New in Python 2 1 + +1.16.8 PEP 232: Function Attributes +----------------------------------- + +In Python 2.1, functions can now have arbitrary information attached to +them. People were often using docstrings to hold information about +functions and methods, because the ‘__doc__’ attribute was the only way +of attaching any information to a function. For example, in the Zope +Web application server, functions are marked as safe for public access +by having a docstring, and in John Aycock’s SPARK parsing framework, +docstrings hold parts of the BNF grammar to be parsed. This overloading +is unfortunate, since docstrings are really intended to hold a +function’s documentation; for example, it means you can’t properly +document functions intended for private use in Zope. + +Arbitrary attributes can now be set and retrieved on functions using the +regular Python syntax: + + def f(): pass + + f.publish = 1 + f.secure = 1 + f.grammar = "A ::= B (C D)*" + +The dictionary containing attributes can be accessed as the function’s +*note __dict__: 4fc. Unlike the *note __dict__: 4fc. attribute of class +instances, in functions you can actually assign a new dictionary to +*note __dict__: 4fc, though the new value is restricted to a regular +Python dictionary; you `can’t' be tricky and set it to a ‘UserDict’ +instance, or any other random object that behaves like a mapping. + +See also +........ + +PEP 232(1) - Function Attributes + + Written and implemented by Barry Warsaw. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0232 + + +File: python.info, Node: PEP 235 Importing Modules on Case-Insensitive Platforms, Next: PEP 217 Interactive Display Hook, Prev: PEP 232 Function Attributes, Up: What’s New in Python 2 1 + +1.16.9 PEP 235: Importing Modules on Case-Insensitive Platforms +--------------------------------------------------------------- + +Some operating systems have filesystems that are case-insensitive, MacOS +and Windows being the primary examples; on these systems, it’s +impossible to distinguish the filenames ‘FILE.PY’ and ‘file.py’, even +though they do store the file’s name in its original case (they’re +case-preserving, too). + +In Python 2.1, the *note import: c1d. statement will work to simulate +case-sensitivity on case-insensitive platforms. Python will now search +for the first case-sensitive match by default, raising an *note +ImportError: 334. if no such file is found, so ‘import file’ will not +import a module named ‘FILE.PY’. Case-insensitive matching can be +requested by setting the *note PYTHONCASEOK: e5d. environment variable +before starting the Python interpreter. + + +File: python.info, Node: PEP 217 Interactive Display Hook, Next: PEP 208 New Coercion Model, Prev: PEP 235 Importing Modules on Case-Insensitive Platforms, Up: What’s New in Python 2 1 + +1.16.10 PEP 217: Interactive Display Hook +----------------------------------------- + +When using the Python interpreter interactively, the output of commands +is displayed using the built-in *note repr(): 7cc. function. In Python +2.1, the variable *note sys.displayhook(): e5f. can be set to a callable +object which will be called instead of *note repr(): 7cc. For example, +you can set it to a special pretty-printing function: + + >>> # Create a recursive data structure + ... L = [1,2,3] + >>> L.append(L) + >>> L # Show Python's default output + [1, 2, 3, [...]] + >>> # Use pprint.pprint() as the display function + ... import sys, pprint + >>> sys.displayhook = pprint.pprint + >>> L + [1, 2, 3, ] + >>> + +See also +........ + +PEP 217(1) - Display Hook for Interactive Use + + Written and implemented by Moshe Zadka. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0217 + + +File: python.info, Node: PEP 208 New Coercion Model, Next: PEP 241 Metadata in Python Packages, Prev: PEP 217 Interactive Display Hook, Up: What’s New in Python 2 1 + +1.16.11 PEP 208: New Coercion Model +----------------------------------- + +How numeric coercion is done at the C level was significantly modified. +This will only affect the authors of C extensions to Python, allowing +them more flexibility in writing extension types that support numeric +operations. + +Extension types can now set the type flag ‘Py_TPFLAGS_CHECKTYPES’ in +their ‘PyTypeObject’ structure to indicate that they support the new +coercion model. In such extension types, the numeric slot functions can +no longer assume that they’ll be passed two arguments of the same type; +instead they may be passed two arguments of differing types, and can +then perform their own internal coercion. If the slot function is +passed a type it can’t handle, it can indicate the failure by returning +a reference to the ‘Py_NotImplemented’ singleton value. The numeric +functions of the other type will then be tried, and perhaps they can +handle the operation; if the other type also returns +‘Py_NotImplemented’, then a *note TypeError: 192. will be raised. +Numeric methods written in Python can also return ‘Py_NotImplemented’, +causing the interpreter to act as if the method did not exist (perhaps +raising a *note TypeError: 192, perhaps trying another object’s numeric +methods). + +See also +........ + +PEP 208(1) - Reworking the Coercion Model + + Written and implemented by Neil Schemenauer, heavily based upon + earlier work by Marc-André Lemburg. Read this to understand the + fine points of how numeric operations will now be processed at the + C level. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0208 + + +File: python.info, Node: PEP 241 Metadata in Python Packages, Next: New and Improved Modules<4>, Prev: PEP 208 New Coercion Model, Up: What’s New in Python 2 1 + +1.16.12 PEP 241: Metadata in Python Packages +-------------------------------------------- + +A common complaint from Python users is that there’s no single catalog +of all the Python modules in existence. T. Middleton’s Vaults of +Parnassus at ‘http://www.vex.net/parnassus/’ are the largest catalog of +Python modules, but registering software at the Vaults is optional, and +many people don’t bother. + +As a first small step toward fixing the problem, Python software +packaged using the Distutils ‘sdist’ command will include a file named +‘PKG-INFO’ containing information about the package such as its name, +version, and author (metadata, in cataloguing terminology). PEP 241(1) +contains the full list of fields that can be present in the ‘PKG-INFO’ +file. As people began to package their software using Python 2.1, more +and more packages will include metadata, making it possible to build +automated cataloguing systems and experiment with them. With the result +experience, perhaps it’ll be possible to design a really good catalog +and then build support for it into Python 2.2. For example, the +Distutils ‘sdist’ and ‘bdist_*’ commands could support an ‘upload’ +option that would automatically upload your package to a catalog server. + +You can start creating packages containing ‘PKG-INFO’ even if you’re not +using Python 2.1, since a new release of the Distutils will be made for +users of earlier Python versions. Version 1.0.2 of the Distutils +includes the changes described in PEP 241(2), as well as various +bugfixes and enhancements. It will be available from the Distutils SIG +at ‘https://www.python.org/community/sigs/current/distutils-sig/’. + +See also +........ + +PEP 241(3) - Metadata for Python Software Packages + + Written and implemented by A.M. Kuchling. + +PEP 243(4) - Module Repository Upload Mechanism + + Written by Sean Reifschneider, this draft PEP describes a proposed + mechanism for uploading Python packages to a central server. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0241 + + (2) https://www.python.org/dev/peps/pep-0241 + + (3) https://www.python.org/dev/peps/pep-0241 + + (4) https://www.python.org/dev/peps/pep-0243 + + +File: python.info, Node: New and Improved Modules<4>, Next: Other Changes and Fixes<4>, Prev: PEP 241 Metadata in Python Packages, Up: What’s New in Python 2 1 + +1.16.13 New and Improved Modules +-------------------------------- + + * Ka-Ping Yee contributed two new modules: ‘inspect.py’, a module for + getting information about live Python code, and ‘pydoc.py’, a + module for interactively converting docstrings to HTML or text. As + a bonus, ‘Tools/scripts/pydoc’, which is now automatically + installed, uses ‘pydoc.py’ to display documentation given a Python + module, package, or class name. For example, ‘pydoc xml.dom’ + displays the following: + + Python Library Documentation: package xml.dom in xml + + NAME + xml.dom - W3C Document Object Model implementation for Python. + + FILE + /usr/local/lib/python2.1/xml/dom/__init__.pyc + + DESCRIPTION + The Python mapping of the Document Object Model is documented in the + Python Library Reference in the section on the xml.dom package. + + This package contains the following modules: + ... + + ‘pydoc’ also includes a Tk-based interactive help browser. ‘pydoc’ + quickly becomes addictive; try it out! + + * Two different modules for unit testing were added to the standard + library. The *note doctest: 67. module, contributed by Tim Peters, + provides a testing framework based on running embedded examples in + docstrings and comparing the results against the expected output. + PyUnit, contributed by Steve Purcell, is a unit testing framework + inspired by JUnit, which was in turn an adaptation of Kent Beck’s + Smalltalk testing framework. See ‘http://pyunit.sourceforge.net/’ + for more information about PyUnit. + + * The *note difflib: 37. module contains a class, ‘SequenceMatcher’, + which compares two sequences and computes the changes required to + transform one sequence into the other. For example, this module + can be used to write a tool similar to the Unix ‘diff’ program, and + in fact the sample program ‘Tools/scripts/ndiff.py’ demonstrates + how to write such a script. + + * *note curses.panel: 2e, a wrapper for the panel library, part of + ncurses and of SYSV curses, was contributed by Thomas Gellekum. + The panel library provides windows with the additional feature of + depth. Windows can be moved higher or lower in the depth ordering, + and the panel library figures out where panels overlap and which + sections are visible. + + * The PyXML package has gone through a few releases since Python 2.0, + and Python 2.1 includes an updated version of the *note xml: 133. + package. Some of the noteworthy changes include support for Expat + 1.2 and later versions, the ability for Expat parsers to handle + files in any encoding supported by Python, and various bugfixes for + SAX, DOM, and the ‘minidom’ module. + + * Ping also contributed another hook for handling uncaught + exceptions. *note sys.excepthook(): e63. can be set to a callable + object. When an exception isn’t caught by any *note try: + d72.…*note except: b3e. blocks, the exception will be passed to + *note sys.excepthook(): e63, which can then do whatever it likes. + At the Ninth Python Conference, Ping demonstrated an application + for this hook: printing an extended traceback that not only lists + the stack frames, but also lists the function arguments and the + local variables for each frame. + + * Various functions in the *note time: 10a. module, such as + ‘asctime()’ and ‘localtime()’, require a floating point argument + containing the time in seconds since the epoch. The most common + use of these functions is to work with the current time, so the + floating point argument has been made optional; when a value isn’t + provided, the current time will be used. For example, log file + entries usually need a string containing the current time; in + Python 2.1, ‘time.asctime()’ can be used, instead of the lengthier + ‘time.asctime(time.localtime(time.time()))’ that was previously + required. + + This change was proposed and implemented by Thomas Wouters. + + * The *note ftplib: 84. module now defaults to retrieving files in + passive mode, because passive mode is more likely to work from + behind a firewall. This request came from the Debian bug tracking + system, since other Debian packages use *note ftplib: 84. to + retrieve files and then don’t work from behind a firewall. It’s + deemed unlikely that this will cause problems for anyone, because + Netscape defaults to passive mode and few people complain, but if + passive mode is unsuitable for your application or network setup, + call ‘set_pasv(0)’ on FTP objects to disable passive mode. + + * Support for raw socket access has been added to the *note socket: + ef. module, contributed by Grant Edwards. + + * The *note pstats: d4. module now contains a simple interactive + statistics browser for displaying timing profiles for Python + programs, invoked when the module is run as a script. Contributed + by Eric S. Raymond. + + * A new implementation-dependent function, ‘sys._getframe([depth])’, + has been added to return a given frame object from the current call + stack. *note sys._getframe(): e64. returns the frame at the top of + the call stack; if the optional integer argument `depth' is + supplied, the function returns the frame that is `depth' calls + below the top of the stack. For example, ‘sys._getframe(1)’ + returns the caller’s frame object. + + This function is only present in CPython, not in Jython or the .NET + implementation. Use it for debugging, and resist the temptation to + put it into production code. + + +File: python.info, Node: Other Changes and Fixes<4>, Next: Acknowledgements<7>, Prev: New and Improved Modules<4>, Up: What’s New in Python 2 1 + +1.16.14 Other Changes and Fixes +------------------------------- + +There were relatively few smaller changes made in Python 2.1 due to the +shorter release cycle. A search through the CVS change logs turns up +117 patches applied, and 136 bugs fixed; both figures are likely to be +underestimates. Some of the more notable changes are: + + * A specialized object allocator is now optionally available, that + should be faster than the system ‘malloc()’ and have less memory + overhead. The allocator uses C’s ‘malloc()’ function to get large + pools of memory, and then fulfills smaller memory requests from + these pools. It can be enabled by providing the ‘--with-pymalloc’ + option to the ‘configure’ script; see ‘Objects/obmalloc.c’ for the + implementation details. + + Authors of C extension modules should test their code with the + object allocator enabled, because some incorrect code may break, + causing core dumps at runtime. There are a bunch of memory + allocation functions in Python’s C API that have previously been + just aliases for the C library’s ‘malloc()’ and ‘free()’, meaning + that if you accidentally called mismatched functions, the error + wouldn’t be noticeable. When the object allocator is enabled, + these functions aren’t aliases of ‘malloc()’ and ‘free()’ any more, + and calling the wrong function to free memory will get you a core + dump. For example, if memory was allocated using ‘PyMem_New()’, it + has to be freed using ‘PyMem_Del()’, not ‘free()’. A few modules + included with Python fell afoul of this and had to be fixed; + doubtless there are more third-party modules that will have the + same problem. + + The object allocator was contributed by Vladimir Marangozov. + + * The speed of line-oriented file I/O has been improved because + people often complain about its lack of speed, and because it’s + often been used as a naïve benchmark. The *note readline(): de. + method of file objects has therefore been rewritten to be much + faster. The exact amount of the speedup will vary from platform to + platform depending on how slow the C library’s ‘getc()’ was, but is + around 66%, and potentially much faster on some particular + operating systems. Tim Peters did much of the benchmarking and + coding for this change, motivated by a discussion in + comp.lang.python. + + A new module and method for file objects was also added, + contributed by Jeff Epler. The new method, ‘xreadlines()’, is + similar to the existing ‘xrange()’ built-in. ‘xreadlines()’ + returns an opaque sequence object that only supports being iterated + over, reading a line on every iteration but not reading the entire + file into memory as the existing ‘readlines()’ method does. You’d + use it like this: + + for line in sys.stdin.xreadlines(): + # ... do something for each line ... + ... + + For a fuller discussion of the line I/O changes, see the python-dev + summary for January 1–15, 2001 at + ‘https://mail.python.org/pipermail/python-dev/2001-January/’. + + * A new method, ‘popitem()’, was added to dictionaries to enable + destructively iterating through the contents of a dictionary; this + can be faster for large dictionaries because there’s no need to + construct a list containing all the keys or values. ‘D.popitem()’ + removes a random ‘(key, value)’ pair from the dictionary ‘D’ and + returns it as a 2-tuple. This was implemented mostly by Tim Peters + and Guido van Rossum, after a suggestion and preliminary patch by + Moshe Zadka. + + * Modules can now control which names are imported when ‘from module + import *’ is used, by defining an ‘__all__’ attribute containing a + list of names that will be imported. One common complaint is that + if the module imports other modules such as *note sys: fd. or *note + string: f6, ‘from module import *’ will add them to the importing + module’s namespace. To fix this, simply list the public names in + ‘__all__’: + + # List public names + __all__ = ['Database', 'open'] + + A stricter version of this patch was first suggested and + implemented by Ben Wolfson, but after some python-dev discussion, a + weaker final version was checked in. + + * Applying *note repr(): 7cc. to strings previously used octal + escapes for non-printable characters; for example, a newline was + ‘'\012'’. This was a vestigial trace of Python’s C ancestry, but + today octal is of very little practical use. Ka-Ping Yee suggested + using hex escapes instead of octal ones, and using the ‘\n’, ‘\t’, + ‘\r’ escapes for the appropriate characters, and implemented this + new formatting. + + * Syntax errors detected at compile-time can now raise exceptions + containing the filename and line number of the error, a pleasant + side effect of the compiler reorganization done by Jeremy Hylton. + + * C extensions which import other modules have been changed to use + ‘PyImport_ImportModule()’, which means that they will use any + import hooks that have been installed. This is also encouraged for + third-party extensions that need to import some other module from C + code. + + * The size of the Unicode character database was shrunk by another + 340K thanks to Fredrik Lundh. + + * Some new ports were contributed: MacOS X (by Steven Majewski), + Cygwin (by Jason Tishler); RISCOS (by Dietmar Schwertberger); + Unixware 7 (by Billy G. Allie). + +And there’s the usual list of minor bugfixes, minor memory leaks, +docstring edits, and other tweaks, too lengthy to be worth itemizing; +see the CVS logs for the full details if you want them. + + +File: python.info, Node: Acknowledgements<7>, Prev: Other Changes and Fixes<4>, Up: What’s New in Python 2 1 + +1.16.15 Acknowledgements +------------------------ + +The author would like to thank the following people for offering +suggestions on various drafts of this article: Graeme Cross, David +Goodger, Jay Graves, Michael Hudson, Marc-André Lemburg, Fredrik Lundh, +Neil Schemenauer, Thomas Wouters. + + +File: python.info, Node: What’s New in Python 2 0, Next: Changelog, Prev: What’s New in Python 2 1, Up: What’s New in Python + +1.17 What’s New in Python 2.0 +============================= + + +Author: A.M. Kuchling and Moshe Zadka + +* Menu: + +* Introduction: Introduction<3>. +* What About Python 1.6?: What About Python 1 6?. +* New Development Process:: +* Unicode: Unicode<2>. +* List Comprehensions:: +* Augmented Assignment:: +* String Methods:: +* Garbage Collection of Cycles:: +* Other Core Changes:: +* Porting to 2.0: Porting to 2 0. +* Extending/Embedding Changes:: +* Distutils; Making Modules Easy to Install: Distutils Making Modules Easy to Install. +* XML Modules:: +* Module changes:: +* New modules:: +* IDLE Improvements:: +* Deleted and Deprecated Modules:: +* Acknowledgements: Acknowledgements<8>. + + +File: python.info, Node: Introduction<3>, Next: What About Python 1 6?, Up: What’s New in Python 2 0 + +1.17.1 Introduction +------------------- + +A new release of Python, version 2.0, was released on October 16, 2000. +This article covers the exciting new features in 2.0, highlights some +other useful changes, and points out a few incompatible changes that may +require rewriting code. + +Python’s development never completely stops between releases, and a +steady flow of bug fixes and improvements are always being submitted. A +host of minor fixes, a few optimizations, additional docstrings, and +better error messages went into 2.0; to list them all would be +impossible, but they’re certainly significant. Consult the +publicly-available CVS logs if you want to see the full list. This +progress is due to the five developers working for PythonLabs are now +getting paid to spend their days fixing bugs, and also due to the +improved communication resulting from moving to SourceForge. + + +File: python.info, Node: What About Python 1 6?, Next: New Development Process, Prev: Introduction<3>, Up: What’s New in Python 2 0 + +1.17.2 What About Python 1.6? +----------------------------- + +Python 1.6 can be thought of as the Contractual Obligations Python +release. After the core development team left CNRI in May 2000, CNRI +requested that a 1.6 release be created, containing all the work on +Python that had been performed at CNRI. Python 1.6 therefore represents +the state of the CVS tree as of May 2000, with the most significant new +feature being Unicode support. Development continued after May, of +course, so the 1.6 tree received a few fixes to ensure that it’s +forward-compatible with Python 2.0. 1.6 is therefore part of Python’s +evolution, and not a side branch. + +So, should you take much interest in Python 1.6? Probably not. The +1.6final and 2.0beta1 releases were made on the same day (September 5, +2000), the plan being to finalize Python 2.0 within a month or so. If +you have applications to maintain, there seems little point in breaking +things by moving to 1.6, fixing them, and then having another round of +breakage within a month by moving to 2.0; you’re better off just going +straight to 2.0. Most of the really interesting features described in +this document are only in 2.0, because a lot of work was done between +May and September. + + +File: python.info, Node: New Development Process, Next: Unicode<2>, Prev: What About Python 1 6?, Up: What’s New in Python 2 0 + +1.17.3 New Development Process +------------------------------ + +The most important change in Python 2.0 may not be to the code at all, +but to how Python is developed: in May 2000 the Python developers began +using the tools made available by SourceForge for storing source code, +tracking bug reports, and managing the queue of patch submissions. To +report bugs or submit patches for Python 2.0, use the bug tracking and +patch manager tools available from Python’s project page, located at +‘https://sourceforge.net/projects/python/’. + +The most important of the services now hosted at SourceForge is the +Python CVS tree, the version-controlled repository containing the source +code for Python. Previously, there were roughly 7 or so people who had +write access to the CVS tree, and all patches had to be inspected and +checked in by one of the people on this short list. Obviously, this +wasn’t very scalable. By moving the CVS tree to SourceForge, it became +possible to grant write access to more people; as of September 2000 +there were 27 people able to check in changes, a fourfold increase. +This makes possible large-scale changes that wouldn’t be attempted if +they’d have to be filtered through the small group of core developers. +For example, one day Peter Schneider-Kamp took it into his head to drop +K&R C compatibility and convert the C source for Python to ANSI C. After +getting approval on the python-dev mailing list, he launched into a +flurry of checkins that lasted about a week, other developers joined in +to help, and the job was done. If there were only 5 people with write +access, probably that task would have been viewed as “nice, but not +worth the time and effort needed” and it would never have gotten done. + +The shift to using SourceForge’s services has resulted in a remarkable +increase in the speed of development. Patches now get submitted, +commented on, revised by people other than the original submitter, and +bounced back and forth between people until the patch is deemed worth +checking in. Bugs are tracked in one central location and can be +assigned to a specific person for fixing, and we can count the number of +open bugs to measure progress. This didn’t come without a cost: +developers now have more e-mail to deal with, more mailing lists to +follow, and special tools had to be written for the new environment. +For example, SourceForge sends default patch and bug notification e-mail +messages that are completely unhelpful, so Ka-Ping Yee wrote an HTML +screen-scraper that sends more useful messages. + +The ease of adding code caused a few initial growing pains, such as code +was checked in before it was ready or without getting clear agreement +from the developer group. The approval process that has emerged is +somewhat similar to that used by the Apache group. Developers can vote ++1, +0, -0, or -1 on a patch; +1 and -1 denote acceptance or rejection, +while +0 and -0 mean the developer is mostly indifferent to the change, +though with a slight positive or negative slant. The most significant +change from the Apache model is that the voting is essentially advisory, +letting Guido van Rossum, who has Benevolent Dictator For Life status, +know what the general opinion is. He can still ignore the result of a +vote, and approve or reject a change even if the community disagrees +with him. + +Producing an actual patch is the last step in adding a new feature, and +is usually easy compared to the earlier task of coming up with a good +design. Discussions of new features can often explode into lengthy +mailing list threads, making the discussion hard to follow, and no one +can read every posting to python-dev. Therefore, a relatively formal +process has been set up to write Python Enhancement Proposals (PEPs), +modelled on the Internet RFC process. PEPs are draft documents that +describe a proposed new feature, and are continually revised until the +community reaches a consensus, either accepting or rejecting the +proposal. Quoting from the introduction to PEP 1(1), “PEP Purpose and +Guidelines”: + + PEP stands for Python Enhancement Proposal. A PEP is a design + document providing information to the Python community, or + describing a new feature for Python. The PEP should provide a + concise technical specification of the feature and a rationale for + the feature. + + We intend PEPs to be the primary mechanisms for proposing new + features, for collecting community input on an issue, and for + documenting the design decisions that have gone into Python. The + PEP author is responsible for building consensus within the + community and documenting dissenting opinions. + +Read the rest of PEP 1(2) for the details of the PEP editorial process, +style, and format. PEPs are kept in the Python CVS tree on SourceForge, +though they’re not part of the Python 2.0 distribution, and are also +available in HTML form from ‘https://www.python.org/dev/peps/’. As of +September 2000, there are 25 PEPS, ranging from PEP 201(3), “Lockstep +Iteration”, to PEP 225, “Elementwise/Objectwise Operators”. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0001 + + (2) https://www.python.org/dev/peps/pep-0001 + + (3) https://www.python.org/dev/peps/pep-0201 + + +File: python.info, Node: Unicode<2>, Next: List Comprehensions, Prev: New Development Process, Up: What’s New in Python 2 0 + +1.17.4 Unicode +-------------- + +The largest new feature in Python 2.0 is a new fundamental data type: +Unicode strings. Unicode uses 16-bit numbers to represent characters +instead of the 8-bit number used by ASCII, meaning that 65,536 distinct +characters can be supported. + +The final interface for Unicode support was arrived at through countless +often-stormy discussions on the python-dev mailing list, and mostly +implemented by Marc-André Lemburg, based on a Unicode string type +implementation by Fredrik Lundh. A detailed explanation of the +interface was written up as PEP 100(1), “Python Unicode Integration”. +This article will simply cover the most significant points about the +Unicode interfaces. + +In Python source code, Unicode strings are written as ‘u"string"’. +Arbitrary Unicode characters can be written using a new escape sequence, +‘\uHHHH’, where `HHHH' is a 4-digit hexadecimal number from 0000 to +FFFF. The existing ‘\xHHHH’ escape sequence can also be used, and octal +escapes can be used for characters up to U+01FF, which is represented by +‘\777’. + +Unicode strings, just like regular strings, are an immutable sequence +type. They can be indexed and sliced, but not modified in place. +Unicode strings have an ‘encode( [encoding] )’ method that returns an +8-bit string in the desired encoding. Encodings are named by strings, +such as ‘'ascii'’, ‘'utf-8'’, ‘'iso-8859-1'’, or whatever. A codec API +is defined for implementing and registering new encodings that are then +available throughout a Python program. If an encoding isn’t specified, +the default encoding is usually 7-bit ASCII, though it can be changed +for your Python installation by calling the +‘sys.setdefaultencoding(encoding)’ function in a customized version of +‘site.py’. + +Combining 8-bit and Unicode strings always coerces to Unicode, using the +default ASCII encoding; the result of ‘'a' + u'bc'’ is ‘u'abc'’. + +New built-in functions have been added, and existing built-ins modified +to support Unicode: + + * ‘unichr(ch)’ returns a Unicode string 1 character long, containing + the character `ch'. + + * ‘ord(u)’, where `u' is a 1-character regular or Unicode string, + returns the number of the character as an integer. + + * ‘unicode(string [, encoding] [, errors] )’ creates a Unicode string + from an 8-bit string. ‘encoding’ is a string naming the encoding + to use. The ‘errors’ parameter specifies the treatment of + characters that are invalid for the current encoding; passing + ‘'strict'’ as the value causes an exception to be raised on any + encoding error, while ‘'ignore'’ causes errors to be silently + ignored and ‘'replace'’ uses U+FFFD, the official replacement + character, in case of any problems. + + * The ‘exec’ statement, and various built-ins such as ‘eval()’, + ‘getattr()’, and ‘setattr()’ will also accept Unicode strings as + well as regular strings. (It’s possible that the process of fixing + this missed some built-ins; if you find a built-in function that + accepts strings but doesn’t accept Unicode strings at all, please + report it as a bug.) + +A new module, *note unicodedata: 11a, provides an interface to Unicode +character properties. For example, ‘unicodedata.category(u'A')’ returns +the 2-character string ‘Lu’, the ‘L’ denoting it’s a letter, and ‘u’ +meaning that it’s uppercase. ‘unicodedata.bidirectional(u'\u0660')’ +returns ‘AN’, meaning that U+0660 is an Arabic number. + +The *note codecs: 1c. module contains functions to look up existing +encodings and register new ones. Unless you want to implement a new +encoding, you’ll most often use the ‘codecs.lookup(encoding)’ function, +which returns a 4-element tuple: ‘(encode_func, decode_func, +stream_reader, stream_writer)’. + + * `encode_func' is a function that takes a Unicode string, and + returns a 2-tuple ‘(string, length)’. `string' is an 8-bit string + containing a portion (perhaps all) of the Unicode string converted + into the given encoding, and `length' tells you how much of the + Unicode string was converted. + + * `decode_func' is the opposite of `encode_func', taking an 8-bit + string and returning a 2-tuple ‘(ustring, length)’, consisting of + the resulting Unicode string `ustring' and the integer `length' + telling how much of the 8-bit string was consumed. + + * `stream_reader' is a class that supports decoding input from a + stream. `stream_reader(file_obj)' returns an object that supports + the ‘read()’, *note readline(): de, and ‘readlines()’ methods. + These methods will all translate from the given encoding and return + Unicode strings. + + * `stream_writer', similarly, is a class that supports encoding + output to a stream. `stream_writer(file_obj)' returns an object + that supports the ‘write()’ and ‘writelines()’ methods. These + methods expect Unicode strings, translating them to the given + encoding on output. + +For example, the following code writes a Unicode string into a file, +encoding it as UTF-8: + + import codecs + + unistr = u'\u0660\u2000ab ...' + + (UTF8_encode, UTF8_decode, + UTF8_streamreader, UTF8_streamwriter) = codecs.lookup('UTF-8') + + output = UTF8_streamwriter( open( '/tmp/output', 'wb') ) + output.write( unistr ) + output.close() + +The following code would then read UTF-8 input from the file: + + input = UTF8_streamreader( open( '/tmp/output', 'rb') ) + print repr(input.read()) + input.close() + +Unicode-aware regular expressions are available through the *note re: +dd. module, which has a new underlying implementation called SRE written +by Fredrik Lundh of Secret Labs AB. + +A ‘-U’ command line option was added which causes the Python compiler to +interpret all string literals as Unicode string literals. This is +intended to be used in testing and future-proofing your Python code, +since some future version of Python may drop support for 8-bit strings +and provide only Unicode strings. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0100 + + +File: python.info, Node: List Comprehensions, Next: Augmented Assignment, Prev: Unicode<2>, Up: What’s New in Python 2 0 + +1.17.5 List Comprehensions +-------------------------- + +Lists are a workhorse data type in Python, and many programs manipulate +a list at some point. Two common operations on lists are to loop over +them, and either pick out the elements that meet a certain criterion, or +apply some function to each element. For example, given a list of +strings, you might want to pull out all the strings containing a given +substring, or strip off trailing whitespace from each line. + +The existing *note map(): c2d. and *note filter(): c2e. functions can be +used for this purpose, but they require a function as one of their +arguments. This is fine if there’s an existing built-in function that +can be passed directly, but if there isn’t, you have to create a little +function to do the required work, and Python’s scoping rules make the +result ugly if the little function needs additional information. Take +the first example in the previous paragraph, finding all the strings in +the list containing a given substring. You could write the following to +do it: + + # Given the list L, make a list of all strings + # containing the substring S. + sublist = filter( lambda s, substring=S: + string.find(s, substring) != -1, + L) + +Because of Python’s scoping rules, a default argument is used so that +the anonymous function created by the *note lambda: c2f. expression +knows what substring is being searched for. List comprehensions make +this cleaner: + + sublist = [ s for s in L if string.find(s, S) != -1 ] + +List comprehensions have the form: + + [ expression for expr in sequence1 + for expr2 in sequence2 ... + for exprN in sequenceN + if condition ] + +The ‘for’…‘in’ clauses contain the sequences to be iterated over. The +sequences do not have to be the same length, because they are `not' +iterated over in parallel, but from left to right; this is explained +more clearly in the following paragraphs. The elements of the generated +list will be the successive values of `expression'. The final ‘if’ +clause is optional; if present, `expression' is only evaluated and added +to the result if `condition' is true. + +To make the semantics very clear, a list comprehension is equivalent to +the following Python code: + + for expr1 in sequence1: + for expr2 in sequence2: + ... + for exprN in sequenceN: + if (condition): + # Append the value of + # the expression to the + # resulting list. + +This means that when there are multiple ‘for’…‘in’ clauses, the +resulting list will be equal to the product of the lengths of all the +sequences. If you have two lists of length 3, the output list is 9 +elements long: + + seq1 = 'abc' + seq2 = (1,2,3) + >>> [ (x,y) for x in seq1 for y in seq2] + [('a', 1), ('a', 2), ('a', 3), ('b', 1), ('b', 2), ('b', 3), ('c', 1), + ('c', 2), ('c', 3)] + +To avoid introducing an ambiguity into Python’s grammar, if `expression' +is creating a tuple, it must be surrounded with parentheses. The first +list comprehension below is a syntax error, while the second one is +correct: + + # Syntax error + [ x,y for x in seq1 for y in seq2] + # Correct + [ (x,y) for x in seq1 for y in seq2] + +The idea of list comprehensions originally comes from the functional +programming language Haskell (‘https://www.haskell.org’). Greg Ewing +argued most effectively for adding them to Python and wrote the initial +list comprehension patch, which was then discussed for a seemingly +endless time on the python-dev mailing list and kept up-to-date by Skip +Montanaro. + + +File: python.info, Node: Augmented Assignment, Next: String Methods, Prev: List Comprehensions, Up: What’s New in Python 2 0 + +1.17.6 Augmented Assignment +--------------------------- + +Augmented assignment operators, another long-requested feature, have +been added to Python 2.0. Augmented assignment operators include ‘+=’, +‘-=’, ‘*=’, and so forth. For example, the statement ‘a += 2’ +increments the value of the variable ‘a’ by 2, equivalent to the +slightly lengthier ‘a = a + 2’. + +The full list of supported assignment operators is ‘+=’, ‘-=’, ‘*=’, +‘/=’, ‘%=’, ‘**=’, ‘&=’, ‘|=’, ‘^=’, ‘>>=’, and ‘<<=’. Python classes +can override the augmented assignment operators by defining methods +named *note __iadd__(): e6f, *note __isub__(): e70, etc. For example, +the following ‘Number’ class stores a number and supports using += to +create a new instance with an incremented value. + + class Number: + def __init__(self, value): + self.value = value + def __iadd__(self, increment): + return Number( self.value + increment) + + n = Number(5) + n += 3 + print n.value + +The *note __iadd__(): e6f. special method is called with the value of +the increment, and should return a new instance with an appropriately +modified value; this return value is bound as the new value of the +variable on the left-hand side. + +Augmented assignment operators were first introduced in the C +programming language, and most C-derived languages, such as ‘awk’, C++, +Java, Perl, and PHP also support them. The augmented assignment patch +was implemented by Thomas Wouters. + + +File: python.info, Node: String Methods, Next: Garbage Collection of Cycles, Prev: Augmented Assignment, Up: What’s New in Python 2 0 + +1.17.7 String Methods +--------------------- + +Until now string-manipulation functionality was in the *note string: f6. +module, which was usually a front-end for the ‘strop’ module written in +C. The addition of Unicode posed a difficulty for the ‘strop’ module, +because the functions would all need to be rewritten in order to accept +either 8-bit or Unicode strings. For functions such as +‘string.replace()’, which takes 3 string arguments, that means eight +possible permutations, and correspondingly complicated code. + +Instead, Python 2.0 pushes the problem onto the string type, making +string manipulation functionality available through methods on both +8-bit strings and Unicode strings. + + >>> 'andrew'.capitalize() + 'Andrew' + >>> 'hostname'.replace('os', 'linux') + 'hlinuxtname' + >>> 'moshe'.find('sh') + 2 + +One thing that hasn’t changed, a noteworthy April Fools’ joke +notwithstanding, is that Python strings are immutable. Thus, the string +methods return new strings, and do not modify the string on which they +operate. + +The old *note string: f6. module is still around for backwards +compatibility, but it mostly acts as a front-end to the new string +methods. + +Two methods which have no parallel in pre-2.0 versions, although they +did exist in JPython for quite some time, are ‘startswith()’ and +‘endswith()’. ‘s.startswith(t)’ is equivalent to ‘s[:len(t)] == t’, +while ‘s.endswith(t)’ is equivalent to ‘s[-len(t):] == t’. + +One other method which deserves special mention is ‘join()’. The +‘join()’ method of a string receives one parameter, a sequence of +strings, and is equivalent to the ‘string.join()’ function from the old +*note string: f6. module, with the arguments reversed. In other words, +‘s.join(seq)’ is equivalent to the old ‘string.join(seq, s)’. + + +File: python.info, Node: Garbage Collection of Cycles, Next: Other Core Changes, Prev: String Methods, Up: What’s New in Python 2 0 + +1.17.8 Garbage Collection of Cycles +----------------------------------- + +The C implementation of Python uses reference counting to implement +garbage collection. Every Python object maintains a count of the number +of references pointing to itself, and adjusts the count as references +are created or destroyed. Once the reference count reaches zero, the +object is no longer accessible, since you need to have a reference to an +object to access it, and if the count is zero, no references exist any +longer. + +Reference counting has some pleasant properties: it’s easy to understand +and implement, and the resulting implementation is portable, fairly +fast, and reacts well with other libraries that implement their own +memory handling schemes. The major problem with reference counting is +that it sometimes doesn’t realise that objects are no longer accessible, +resulting in a memory leak. This happens when there are cycles of +references. + +Consider the simplest possible cycle, a class instance which has a +reference to itself: + + instance = SomeClass() + instance.myself = instance + +After the above two lines of code have been executed, the reference +count of ‘instance’ is 2; one reference is from the variable named +‘'instance'’, and the other is from the ‘myself’ attribute of the +instance. + +If the next line of code is ‘del instance’, what happens? The reference +count of ‘instance’ is decreased by 1, so it has a reference count of 1; +the reference in the ‘myself’ attribute still exists. Yet the instance +is no longer accessible through Python code, and it could be deleted. +Several objects can participate in a cycle if they have references to +each other, causing all of the objects to be leaked. + +Python 2.0 fixes this problem by periodically executing a cycle +detection algorithm which looks for inaccessible cycles and deletes the +objects involved. A new *note gc: 86. module provides functions to +perform a garbage collection, obtain debugging statistics, and tuning +the collector’s parameters. + +Running the cycle detection algorithm takes some time, and therefore +will result in some additional overhead. It is hoped that after we’ve +gotten experience with the cycle collection from using 2.0, Python 2.1 +will be able to minimize the overhead with careful tuning. It’s not yet +obvious how much performance is lost, because benchmarking this is +tricky and depends crucially on how often the program creates and +destroys objects. The detection of cycles can be disabled when Python +is compiled, if you can’t afford even a tiny speed penalty or suspect +that the cycle collection is buggy, by specifying the +‘--without-cycle-gc’ switch when running the ‘configure’ script. + +Several people tackled this problem and contributed to a solution. An +early implementation of the cycle detection approach was written by Toby +Kelsey. The current algorithm was suggested by Eric Tiedemann during a +visit to CNRI, and Guido van Rossum and Neil Schemenauer wrote two +different implementations, which were later integrated by Neil. Lots of +other people offered suggestions along the way; the March 2000 archives +of the python-dev mailing list contain most of the relevant discussion, +especially in the threads titled “Reference cycle collection for Python” +and “Finalization again”. + + +File: python.info, Node: Other Core Changes, Next: Porting to 2 0, Prev: Garbage Collection of Cycles, Up: What’s New in Python 2 0 + +1.17.9 Other Core Changes +------------------------- + +Various minor changes have been made to Python’s syntax and built-in +functions. None of the changes are very far-reaching, but they’re handy +conveniences. + +* Menu: + +* Minor Language Changes:: +* Changes to Built-in Functions:: + + +File: python.info, Node: Minor Language Changes, Next: Changes to Built-in Functions, Up: Other Core Changes + +1.17.9.1 Minor Language Changes +............................... + +A new syntax makes it more convenient to call a given function with a +tuple of arguments and/or a dictionary of keyword arguments. In Python +1.5 and earlier, you’d use the ‘apply()’ built-in function: ‘apply(f, +args, kw)’ calls the function ‘f()’ with the argument tuple `args' and +the keyword arguments in the dictionary `kw'. ‘apply()’ is the same in +2.0, but thanks to a patch from Greg Ewing, ‘f(*args, **kw)’ is a +shorter and clearer way to achieve the same effect. This syntax is +symmetrical with the syntax for defining functions: + + def f(*args, **kw): + # args is a tuple of positional args, + # kw is a dictionary of keyword args + ... + +The ‘print’ statement can now have its output directed to a file-like +object by following the ‘print’ with ‘>> file’, similar to the +redirection operator in Unix shells. Previously you’d either have to +use the ‘write()’ method of the file-like object, which lacks the +convenience and simplicity of ‘print’, or you could assign a new value +to ‘sys.stdout’ and then restore the old value. For sending output to +standard error, it’s much easier to write this: + + print >> sys.stderr, "Warning: action field not supplied" + +Modules can now be renamed on importing them, using the syntax ‘import +module as name’ or ‘from module import name as othername’. The patch +was submitted by Thomas Wouters. + +A new format style is available when using the ‘%’ operator; ‘%r’ will +insert the *note repr(): 7cc. of its argument. This was also added from +symmetry considerations, this time for symmetry with the existing ‘%s’ +format style, which inserts the *note str(): 330. of its argument. For +example, ‘'%r %s' % ('abc', 'abc')’ returns a string containing ‘'abc' +abc’. + +Previously there was no way to implement a class that overrode Python’s +built-in *note in: 7a3. operator and implemented a custom version. ‘obj +in seq’ returns true if `obj' is present in the sequence `seq'; Python +computes this by simply trying every index of the sequence until either +`obj' is found or an *note IndexError: e75. is encountered. Moshe Zadka +contributed a patch which adds a *note __contains__(): d28. magic method +for providing a custom implementation for ‘in’. Additionally, new +built-in objects written in C can define what ‘in’ means for them via a +new slot in the sequence protocol. + +Earlier versions of Python used a recursive algorithm for deleting +objects. Deeply nested data structures could cause the interpreter to +fill up the C stack and crash; Christian Tismer rewrote the deletion +logic to fix this problem. On a related note, comparing recursive +objects recursed infinitely and crashed; Jeremy Hylton rewrote the code +to no longer crash, producing a useful result instead. For example, +after this code: + + a = [] + b = [] + a.append(a) + b.append(b) + +The comparison ‘a==b’ returns true, because the two recursive data +structures are isomorphic. See the thread “trashcan and PR#7” in the +April 2000 archives of the python-dev mailing list for the discussion +leading up to this implementation, and some useful relevant links. Note +that comparisons can now also raise exceptions. In earlier versions of +Python, a comparison operation such as ‘cmp(a,b)’ would always produce +an answer, even if a user-defined ‘__cmp__()’ method encountered an +error, since the resulting exception would simply be silently swallowed. + +Work has been done on porting Python to 64-bit Windows on the Itanium +processor, mostly by Trent Mick of ActiveState. (Confusingly, +‘sys.platform’ is still ‘'win32'’ on Win64 because it seems that for +ease of porting, MS Visual C++ treats code as 32 bit on Itanium.) +PythonWin also supports Windows CE; see the Python CE page at +‘http://pythonce.sourceforge.net/’ for more information. + +Another new platform is Darwin/MacOS X; initial support for it is in +Python 2.0. Dynamic loading works, if you specify “configure –with-dyld +–with-suffix=.x”. Consult the README in the Python source distribution +for more instructions. + +An attempt has been made to alleviate one of Python’s warts, the +often-confusing *note NameError: d7b. exception when code refers to a +local variable before the variable has been assigned a value. For +example, the following code raises an exception on the ‘print’ statement +in both 1.5.2 and 2.0; in 1.5.2 a *note NameError: d7b. exception is +raised, while 2.0 raises a new *note UnboundLocalError: e76. exception. +*note UnboundLocalError: e76. is a subclass of *note NameError: d7b, so +any existing code that expects *note NameError: d7b. to be raised should +still work. + + def f(): + print "i=",i + i = i + 1 + f() + +Two new exceptions, *note TabError: e77. and *note IndentationError: +e78, have been introduced. They’re both subclasses of *note +SyntaxError: 458, and are raised when Python code is found to be +improperly indented. + + +File: python.info, Node: Changes to Built-in Functions, Prev: Minor Language Changes, Up: Other Core Changes + +1.17.9.2 Changes to Built-in Functions +...................................... + +A new built-in, ‘zip(seq1, seq2, ...)’, has been added. *note zip(): +c32. returns a list of tuples where each tuple contains the i-th element +from each of the argument sequences. The difference between *note +zip(): c32. and ‘map(None, seq1, seq2)’ is that *note map(): c2d. pads +the sequences with ‘None’ if the sequences aren’t all of the same +length, while *note zip(): c32. truncates the returned list to the +length of the shortest argument sequence. + +The *note int(): 184. and ‘long()’ functions now accept an optional +“base” parameter when the first argument is a string. ‘int('123', 10)’ +returns 123, while ‘int('123', 16)’ returns 291. ‘int(123, 16)’ raises +a *note TypeError: 192. exception with the message “can’t convert +non-string with explicit base”. + +A new variable holding more detailed version information has been added +to the *note sys: fd. module. ‘sys.version_info’ is a tuple ‘(major, +minor, micro, level, serial)’ For example, in a hypothetical 2.0.1beta1, +‘sys.version_info’ would be ‘(2, 0, 1, 'beta', 1)’. `level' is a string +such as ‘"alpha"’, ‘"beta"’, or ‘"final"’ for a final release. + +Dictionaries have an odd new method, ‘setdefault(key, default)’, which +behaves similarly to the existing ‘get()’ method. However, if the key +is missing, ‘setdefault()’ both returns the value of `default' as +‘get()’ would do, and also inserts it into the dictionary as the value +for `key'. Thus, the following lines of code: + + if dict.has_key( key ): return dict[key] + else: + dict[key] = [] + return dict[key] + +can be reduced to a single ‘return dict.setdefault(key, [])’ statement. + +The interpreter sets a maximum recursion depth in order to catch runaway +recursion before filling the C stack and causing a core dump or GPF.. +Previously this limit was fixed when you compiled Python, but in 2.0 the +maximum recursion depth can be read and modified using *note +sys.getrecursionlimit(): e7a. and *note sys.setrecursionlimit(): e7b. +The default value is 1000, and a rough maximum value for a given +platform can be found by running a new script, +‘Misc/find_recursionlimit.py’. + + +File: python.info, Node: Porting to 2 0, Next: Extending/Embedding Changes, Prev: Other Core Changes, Up: What’s New in Python 2 0 + +1.17.10 Porting to 2.0 +---------------------- + +New Python releases try hard to be compatible with previous releases, +and the record has been pretty good. However, some changes are +considered useful enough, usually because they fix initial design +decisions that turned out to be actively mistaken, that breaking +backward compatibility can’t always be avoided. This section lists the +changes in Python 2.0 that may cause old Python code to break. + +The change which will probably break the most code is tightening up the +arguments accepted by some methods. Some methods would take multiple +arguments and treat them as a tuple, particularly various list methods +such as ‘append()’ and ‘insert()’. In earlier versions of Python, if +‘L’ is a list, ‘L.append( 1,2 )’ appends the tuple ‘(1,2)’ to the list. +In Python 2.0 this causes a *note TypeError: 192. exception to be +raised, with the message: ‘append requires exactly 1 argument; 2 given’. +The fix is to simply add an extra set of parentheses to pass both values +as a tuple: ‘L.append( (1,2) )’. + +The earlier versions of these methods were more forgiving because they +used an old function in Python’s C interface to parse their arguments; +2.0 modernizes them to use ‘PyArg_ParseTuple()’, the current argument +parsing function, which provides more helpful error messages and treats +multi-argument calls as errors. If you absolutely must use 2.0 but +can’t fix your code, you can edit ‘Objects/listobject.c’ and define the +preprocessor symbol ‘NO_STRICT_LIST_APPEND’ to preserve the old +behaviour; this isn’t recommended. + +Some of the functions in the *note socket: ef. module are still +forgiving in this way. For example, ‘socket.connect( ('hostname', 25) +)()’ is the correct form, passing a tuple representing an IP address, +but ‘socket.connect( 'hostname', 25 )()’ also works. +‘socket.connect_ex()’ and ‘socket.bind()’ are similarly easy-going. +2.0alpha1 tightened these functions up, but because the documentation +actually used the erroneous multiple argument form, many people wrote +code which would break with the stricter checking. GvR backed out the +changes in the face of public reaction, so for the *note socket: ef. +module, the documentation was fixed and the multiple argument form is +simply marked as deprecated; it `will' be tightened up again in a future +Python version. + +The ‘\x’ escape in string literals now takes exactly 2 hex digits. +Previously it would consume all the hex digits following the ‘x’ and +take the lowest 8 bits of the result, so ‘\x123456’ was equivalent to +‘\x56’. + +The *note AttributeError: 39b. and *note NameError: d7b. exceptions have +a more friendly error message, whose text will be something like ‘'Spam' +instance has no attribute 'eggs'’ or ‘name 'eggs' is not defined’. +Previously the error message was just the missing attribute name ‘eggs’, +and code written to take advantage of this fact will break in 2.0. + +Some work has been done to make integers and long integers a bit more +interchangeable. In 1.5.2, large-file support was added for Solaris, to +allow reading files larger than 2 GiB; this made the ‘tell()’ method of +file objects return a long integer instead of a regular integer. Some +code would subtract two file offsets and attempt to use the result to +multiply a sequence or slice a string, but this raised a *note +TypeError: 192. In 2.0, long integers can be used to multiply or slice +a sequence, and it’ll behave as you’d intuitively expect it to; ‘3L * +'abc'’ produces ‘abcabcabc’, and ‘(0,1,2,3)[2L:4L]’ produces (2,3). +Long integers can also be used in various contexts where previously only +integers were accepted, such as in the ‘seek()’ method of file objects, +and in the formats supported by the ‘%’ operator (‘%d’, ‘%i’, ‘%x’, +etc.). For example, ‘"%d" % 2L**64’ will produce the string +‘18446744073709551616’. + +The subtlest long integer change of all is that the *note str(): 330. of +a long integer no longer has a trailing ‘L’ character, though *note +repr(): 7cc. still includes it. The ‘L’ annoyed many people who wanted +to print long integers that looked just like regular integers, since +they had to go out of their way to chop off the character. This is no +longer a problem in 2.0, but code which does ‘str(longval)[:-1]’ and +assumes the ‘L’ is there, will now lose the final digit. + +Taking the *note repr(): 7cc. of a float now uses a different formatting +precision than *note str(): 330. *note repr(): 7cc. uses ‘%.17g’ format +string for C’s ‘sprintf()’, while *note str(): 330. uses ‘%.12g’ as +before. The effect is that *note repr(): 7cc. may occasionally show +more decimal places than *note str(): 330, for certain numbers. For +example, the number 8.1 can’t be represented exactly in binary, so +‘repr(8.1)’ is ‘'8.0999999999999996'’, while str(8.1) is ‘'8.1'’. + +The ‘-X’ command-line option, which turned all standard exceptions into +strings instead of classes, has been removed; the standard exceptions +will now always be classes. The ‘exceptions’ module containing the +standard exceptions was translated from Python to a built-in C module, +written by Barry Warsaw and Fredrik Lundh. + + +File: python.info, Node: Extending/Embedding Changes, Next: Distutils Making Modules Easy to Install, Prev: Porting to 2 0, Up: What’s New in Python 2 0 + +1.17.11 Extending/Embedding Changes +----------------------------------- + +Some of the changes are under the covers, and will only be apparent to +people writing C extension modules or embedding a Python interpreter in +a larger application. If you aren’t dealing with Python’s C API, you +can safely skip this section. + +The version number of the Python C API was incremented, so C extensions +compiled for 1.5.2 must be recompiled in order to work with 2.0. On +Windows, it’s not possible for Python 2.0 to import a third party +extension built for Python 1.5.x due to how Windows DLLs work, so Python +will raise an exception and the import will fail. + +Users of Jim Fulton’s ExtensionClass module will be pleased to find out +that hooks have been added so that ExtensionClasses are now supported by +*note isinstance(): 44f. and *note issubclass(): 450. This means you no +longer have to remember to write code such as ‘if type(obj) == +myExtensionClass’, but can use the more natural ‘if isinstance(obj, +myExtensionClass)’. + +The ‘Python/importdl.c’ file, which was a mass of #ifdefs to support +dynamic loading on many different platforms, was cleaned up and +reorganised by Greg Stein. ‘importdl.c’ is now quite small, and +platform-specific code has been moved into a bunch of +‘Python/dynload_*.c’ files. Another cleanup: there were also a number +of ‘my*.h’ files in the Include/ directory that held various portability +hacks; they’ve been merged into a single file, ‘Include/pyport.h’. + +Vladimir Marangozov’s long-awaited malloc restructuring was completed, +to make it easy to have the Python interpreter use a custom allocator +instead of C’s standard ‘malloc()’. For documentation, read the +comments in ‘Include/pymem.h’ and ‘Include/objimpl.h’. For the lengthy +discussions during which the interface was hammered out, see the Web +archives of the ‘patches’ and ‘python-dev’ lists at python.org. + +Recent versions of the GUSI development environment for MacOS support +POSIX threads. Therefore, Python’s POSIX threading support now works on +the Macintosh. Threading support using the user-space GNU ‘pth’ library +was also contributed. + +Threading support on Windows was enhanced, too. Windows supports thread +locks that use kernel objects only in case of contention; in the common +case when there’s no contention, they use simpler functions which are an +order of magnitude faster. A threaded version of Python 1.5.2 on NT is +twice as slow as an unthreaded version; with the 2.0 changes, the +difference is only 10%. These improvements were contributed by Yakov +Markovitch. + +Python 2.0’s source now uses only ANSI C prototypes, so compiling Python +now requires an ANSI C compiler, and can no longer be done using a +compiler that only supports K&R C. + +Previously the Python virtual machine used 16-bit numbers in its +bytecode, limiting the size of source files. In particular, this +affected the maximum size of literal lists and dictionaries in Python +source; occasionally people who are generating Python code would run +into this limit. A patch by Charles G. Waldman raises the limit from +‘2^16’ to ‘2^{32}’. + +Three new convenience functions intended for adding constants to a +module’s dictionary at module initialization time were added: +‘PyModule_AddObject()’, ‘PyModule_AddIntConstant()’, and +‘PyModule_AddStringConstant()’. Each of these functions takes a module +object, a null-terminated C string containing the name to be added, and +a third argument for the value to be assigned to the name. This third +argument is, respectively, a Python object, a C long, or a C string. + +A wrapper API was added for Unix-style signal handlers. ‘PyOS_getsig()’ +gets a signal handler and ‘PyOS_setsig()’ will set a new handler. + + +File: python.info, Node: Distutils Making Modules Easy to Install, Next: XML Modules, Prev: Extending/Embedding Changes, Up: What’s New in Python 2 0 + +1.17.12 Distutils: Making Modules Easy to Install +------------------------------------------------- + +Before Python 2.0, installing modules was a tedious affair – there was +no way to figure out automatically where Python is installed, or what +compiler options to use for extension modules. Software authors had to +go through an arduous ritual of editing Makefiles and configuration +files, which only really work on Unix and leave Windows and MacOS +unsupported. Python users faced wildly differing installation +instructions which varied between different extension packages, which +made administering a Python installation something of a chore. + +The SIG for distribution utilities, shepherded by Greg Ward, has created +the Distutils, a system to make package installation much easier. They +form the *note distutils: 39. package, a new part of Python’s standard +library. In the best case, installing a Python module from source will +require the same steps: first you simply mean unpack the tarball or zip +archive, and the run “‘python setup.py install’”. The platform will be +automatically detected, the compiler will be recognized, C extension +modules will be compiled, and the distribution installed into the proper +directory. Optional command-line arguments provide more control over +the installation process, the distutils package offers many places to +override defaults – separating the build from the install, building or +installing in non-default directories, and more. + +In order to use the Distutils, you need to write a ‘setup.py’ script. +For the simple case, when the software contains only .py files, a +minimal ‘setup.py’ can be just a few lines long: + + from distutils.core import setup + setup (name = "foo", version = "1.0", + py_modules = ["module1", "module2"]) + +The ‘setup.py’ file isn’t much more complicated if the software consists +of a few packages: + + from distutils.core import setup + setup (name = "foo", version = "1.0", + packages = ["package", "package.subpackage"]) + +A C extension can be the most complicated case; here’s an example taken +from the PyXML package: + + from distutils.core import setup, Extension + + expat_extension = Extension('xml.parsers.pyexpat', + define_macros = [('XML_NS', None)], + include_dirs = [ 'extensions/expat/xmltok', + 'extensions/expat/xmlparse' ], + sources = [ 'extensions/pyexpat.c', + 'extensions/expat/xmltok/xmltok.c', + 'extensions/expat/xmltok/xmlrole.c', ] + ) + setup (name = "PyXML", version = "0.5.4", + ext_modules =[ expat_extension ] ) + +The Distutils can also take care of creating source and binary +distributions. The “sdist” command, run by “‘python setup.py sdist’’, +builds a source distribution such as ‘foo-1.0.tar.gz’. Adding new +commands isn’t difficult, “bdist_rpm” and “bdist_wininst” commands have +already been contributed to create an RPM distribution and a Windows +installer for the software, respectively. Commands to create other +distribution formats such as Debian packages and Solaris ‘.pkg’ files +are in various stages of development. + +All this is documented in a new manual, `Distributing Python Modules', +that joins the basic set of Python documentation. + + +File: python.info, Node: XML Modules, Next: Module changes, Prev: Distutils Making Modules Easy to Install, Up: What’s New in Python 2 0 + +1.17.13 XML Modules +------------------- + +Python 1.5.2 included a simple XML parser in the form of the ‘xmllib’ +module, contributed by Sjoerd Mullender. Since 1.5.2’s release, two +different interfaces for processing XML have become common: SAX2 +(version 2 of the Simple API for XML) provides an event-driven interface +with some similarities to ‘xmllib’, and the DOM (Document Object Model) +provides a tree-based interface, transforming an XML document into a +tree of nodes that can be traversed and modified. Python 2.0 includes a +SAX2 interface and a stripped-down DOM interface as part of the *note +xml: 133. package. Here we will give a brief overview of these new +interfaces; consult the Python documentation or the source code for +complete details. The Python XML SIG is also working on improved +documentation. + +* Menu: + +* SAX2 Support:: +* DOM Support:: +* Relationship to PyXML:: + + +File: python.info, Node: SAX2 Support, Next: DOM Support, Up: XML Modules + +1.17.13.1 SAX2 Support +...................... + +SAX defines an event-driven interface for parsing XML. To use SAX, you +must write a SAX handler class. Handler classes inherit from various +classes provided by SAX, and override various methods that will then be +called by the XML parser. For example, the ‘startElement()’ and +‘endElement()’ methods are called for every starting and end tag +encountered by the parser, the ‘characters()’ method is called for every +chunk of character data, and so forth. + +The advantage of the event-driven approach is that the whole document +doesn’t have to be resident in memory at any one time, which matters if +you are processing really huge documents. However, writing the SAX +handler class can get very complicated if you’re trying to modify the +document structure in some elaborate way. + +For example, this little example program defines a handler that prints a +message for every starting and ending tag, and then parses the file +‘hamlet.xml’ using it: + + from xml import sax + + class SimpleHandler(sax.ContentHandler): + def startElement(self, name, attrs): + print 'Start of element:', name, attrs.keys() + + def endElement(self, name): + print 'End of element:', name + + # Create a parser object + parser = sax.make_parser() + + # Tell it what handler to use + handler = SimpleHandler() + parser.setContentHandler( handler ) + + # Parse a file! + parser.parse( 'hamlet.xml' ) + +For more information, consult the Python documentation, or the XML HOWTO +at ‘http://pyxml.sourceforge.net/topics/howto/xml-howto.html’. + + +File: python.info, Node: DOM Support, Next: Relationship to PyXML, Prev: SAX2 Support, Up: XML Modules + +1.17.13.2 DOM Support +..................... + +The Document Object Model is a tree-based representation for an XML +document. A top-level ‘Document’ instance is the root of the tree, and +has a single child which is the top-level ‘Element’ instance. This +‘Element’ has children nodes representing character data and any +sub-elements, which may have further children of their own, and so +forth. Using the DOM you can traverse the resulting tree any way you +like, access element and attribute values, insert and delete nodes, and +convert the tree back into XML. + +The DOM is useful for modifying XML documents, because you can create a +DOM tree, modify it by adding new nodes or rearranging subtrees, and +then produce a new XML document as output. You can also construct a DOM +tree manually and convert it to XML, which can be a more flexible way of +producing XML output than simply writing ‘’…‘’ to a file. + +The DOM implementation included with Python lives in the *note +xml.dom.minidom: 135. module. It’s a lightweight implementation of the +Level 1 DOM with support for XML namespaces. The ‘parse()’ and +‘parseString()’ convenience functions are provided for generating a DOM +tree: + + from xml.dom import minidom + doc = minidom.parse('hamlet.xml') + +‘doc’ is a ‘Document’ instance. ‘Document’, like all the other DOM +classes such as ‘Element’ and ‘Text’, is a subclass of the ‘Node’ base +class. All the nodes in a DOM tree therefore support certain common +methods, such as ‘toxml()’ which returns a string containing the XML +representation of the node and its children. Each class also has +special methods of its own; for example, ‘Element’ and ‘Document’ +instances have a method to find all child elements with a given tag +name. Continuing from the previous 2-line example: + + perslist = doc.getElementsByTagName( 'PERSONA' ) + print perslist[0].toxml() + print perslist[1].toxml() + +For the `Hamlet' XML file, the above few lines output: + + CLAUDIUS, king of Denmark. + HAMLET, son to the late, and nephew to the present king. + +The root element of the document is available as ‘doc.documentElement’, +and its children can be easily modified by deleting, adding, or removing +nodes: + + root = doc.documentElement + + # Remove the first child + root.removeChild( root.childNodes[0] ) + + # Move the new first child to the end + root.appendChild( root.childNodes[0] ) + + # Insert the new first child (originally, + # the third child) before the 20th child. + root.insertBefore( root.childNodes[0], root.childNodes[20] ) + +Again, I will refer you to the Python documentation for a complete +listing of the different ‘Node’ classes and their various methods. + + +File: python.info, Node: Relationship to PyXML, Prev: DOM Support, Up: XML Modules + +1.17.13.3 Relationship to PyXML +............................... + +The XML Special Interest Group has been working on XML-related Python +code for a while. Its code distribution, called PyXML, is available +from the SIG’s Web pages at +‘https://www.python.org/community/sigs/current/xml-sig’. The PyXML +distribution also used the package name ‘xml’. If you’ve written +programs that used PyXML, you’re probably wondering about its +compatibility with the 2.0 *note xml: 133. package. + +The answer is that Python 2.0’s *note xml: 133. package isn’t compatible +with PyXML, but can be made compatible by installing a recent version +PyXML. Many applications can get by with the XML support that is +included with Python 2.0, but more complicated applications will require +that the full PyXML package will be installed. When installed, PyXML +versions 0.6.0 or greater will replace the *note xml: 133. package +shipped with Python, and will be a strict superset of the standard +package, adding a bunch of additional features. Some of the additional +features in PyXML include: + + * 4DOM, a full DOM implementation from FourThought, Inc. + + * The xmlproc validating parser, written by Lars Marius Garshol. + + * The ‘sgmlop’ parser accelerator module, written by Fredrik Lundh. + + +File: python.info, Node: Module changes, Next: New modules, Prev: XML Modules, Up: What’s New in Python 2 0 + +1.17.14 Module changes +---------------------- + +Lots of improvements and bugfixes were made to Python’s extensive +standard library; some of the affected modules include *note readline: +de, ‘ConfigParser’, *note cgi: 16, *note calendar: 15, *note posix: d1, +*note readline: de, ‘xmllib’, *note aifc: 5, ‘chunk, wave’, *note +random: dc, *note shelve: e7, and *note nntplib: c0. Consult the CVS +logs for the exact patch-by-patch details. + +Brian Gallew contributed OpenSSL support for the *note socket: ef. +module. OpenSSL is an implementation of the Secure Socket Layer, which +encrypts the data being sent over a socket. When compiling Python, you +can edit ‘Modules/Setup’ to include SSL support, which adds an +additional function to the *note socket: ef. module: ‘socket.ssl(socket, +keyfile, certfile)’, which takes a socket object and returns an SSL +socket. The ‘httplib’ and *note urllib: 11d. modules were also changed +to support ‘https://’ URLs, though no one has implemented FTP or SMTP +over SSL. + +The ‘httplib’ module has been rewritten by Greg Stein to support +HTTP/1.1. Backward compatibility with the 1.5 version of ‘httplib’ is +provided, though using HTTP/1.1 features such as pipelining will require +rewriting code to use a different set of interfaces. + +The ‘Tkinter’ module now supports Tcl/Tk version 8.1, 8.2, or 8.3, and +support for the older 7.x versions has been dropped. The Tkinter module +now supports displaying Unicode strings in Tk widgets. Also, Fredrik +Lundh contributed an optimization which makes operations like +‘create_line’ and ‘create_polygon’ much faster, especially when using +lots of coordinates. + +The *note curses: 2c. module has been greatly extended, starting from +Oliver Andrich’s enhanced version, to provide many additional functions +from ncurses and SYSV curses, such as colour, alternative character set +support, pads, and mouse support. This means the module is no longer +compatible with operating systems that only have BSD curses, but there +don’t seem to be any currently maintained OSes that fall into this +category. + +As mentioned in the earlier discussion of 2.0’s Unicode support, the +underlying implementation of the regular expressions provided by the +*note re: dd. module has been changed. SRE, a new regular expression +engine written by Fredrik Lundh and partially funded by Hewlett Packard, +supports matching against both 8-bit strings and Unicode strings. + + +File: python.info, Node: New modules, Next: IDLE Improvements, Prev: Module changes, Up: What’s New in Python 2 0 + +1.17.15 New modules +------------------- + +A number of new modules were added. We’ll simply list them with brief +descriptions; consult the 2.0 documentation for the details of a +particular module. + + * *note atexit: c.: For registering functions to be called before the + Python interpreter exits. Code that currently sets ‘sys.exitfunc’ + directly should be changed to use the *note atexit: c. module + instead, importing *note atexit: c. and calling *note + atexit.register(): e85. with the function to be called on exit. + (Contributed by Skip Montanaro.) + + * *note codecs: 1c, ‘encodings’, *note unicodedata: 11a.: Added as + part of the new Unicode support. + + * *note filecmp: 7f.: Supersedes the old ‘cmp’, ‘cmpcache’ and + ‘dircmp’ modules, which have now become deprecated. (Contributed + by Gordon MacMillan and Moshe Zadka.) + + * *note gettext: 89.: This module provides internationalization + (I18N) and localization (L10N) support for Python programs by + providing an interface to the GNU gettext message catalog library. + (Integrated by Barry Warsaw, from separate contributions by Martin + von Löwis, Peter Funk, and James Henstridge.) + + * ‘linuxaudiodev’: Support for the ‘/dev/audio’ device on Linux, a + twin to the existing ‘sunaudiodev’ module. (Contributed by Peter + Bosch, with fixes by Jeremy Hylton.) + + * *note mmap: b3.: An interface to memory-mapped files on both + Windows and Unix. A file’s contents can be mapped directly into + memory, at which point it behaves like a mutable string, so its + contents can be read and modified. They can even be passed to + functions that expect ordinary strings, such as the *note re: dd. + module. (Contributed by Sam Rushing, with some extensions by A.M. + Kuchling.) + + * ‘pyexpat’: An interface to the Expat XML parser. (Contributed by + Paul Prescod.) + + * ‘robotparser’: Parse a ‘robots.txt’ file, which is used for writing + Web spiders that politely avoid certain areas of a Web site. The + parser accepts the contents of a ‘robots.txt’ file, builds a set of + rules from it, and can then answer questions about the fetchability + of a given URL. (Contributed by Skip Montanaro.) + + * *note tabnanny: 100.: A module/script to check Python source code + for ambiguous indentation. (Contributed by Tim Peters.) + + * ‘UserString’: A base class useful for deriving objects that behave + like strings. + + * *note webbrowser: 129.: A module that provides a platform + independent way to launch a web browser on a specific URL. For each + platform, various browsers are tried in a specific order. The user + can alter which browser is launched by setting the `BROWSER' + environment variable. (Originally inspired by Eric S. Raymond’s + patch to *note urllib: 11d. which added similar functionality, but + the final module comes from code originally implemented by Fred + Drake as ‘Tools/idle/BrowserControl.py’, and adapted for the + standard library by Fred.) + + * ‘_winreg’: An interface to the Windows registry. ‘_winreg’ is an + adaptation of functions that have been part of PythonWin since + 1995, but has now been added to the core distribution, and enhanced + to support Unicode. ‘_winreg’ was written by Bill Tutt and Mark + Hammond. + + * *note zipfile: 142.: A module for reading and writing ZIP-format + archives. These are archives produced by ‘PKZIP’ on DOS/Windows or + ‘zip’ on Unix, not to be confused with ‘gzip’-format files (which + are supported by the *note gzip: 8c. module) (Contributed by James + C. Ahlstrom.) + + * ‘imputil’: A module that provides a simpler way for writing + customized import hooks, in comparison to the existing ‘ihooks’ + module. (Implemented by Greg Stein, with much discussion on + python-dev along the way.) + + +File: python.info, Node: IDLE Improvements, Next: Deleted and Deprecated Modules, Prev: New modules, Up: What’s New in Python 2 0 + +1.17.16 IDLE Improvements +------------------------- + +IDLE is the official Python cross-platform IDE, written using Tkinter. +Python 2.0 includes IDLE 0.6, which adds a number of new features and +improvements. A partial list: + + * UI improvements and optimizations, especially in the area of syntax + highlighting and auto-indentation. + + * The class browser now shows more information, such as the top level + functions in a module. + + * Tab width is now a user settable option. When opening an existing + Python file, IDLE automatically detects the indentation + conventions, and adapts. + + * There is now support for calling browsers on various platforms, + used to open the Python documentation in a browser. + + * IDLE now has a command line, which is largely similar to the + vanilla Python interpreter. + + * Call tips were added in many places. + + * IDLE can now be installed as a package. + + * In the editor window, there is now a line/column bar at the bottom. + + * Three new keystroke commands: Check module (‘Alt-F5’), Import + module (‘F5’) and Run script (‘Ctrl-F5’). + + +File: python.info, Node: Deleted and Deprecated Modules, Next: Acknowledgements<8>, Prev: IDLE Improvements, Up: What’s New in Python 2 0 + +1.17.17 Deleted and Deprecated Modules +-------------------------------------- + +A few modules have been dropped because they’re obsolete, or because +there are now better ways to do the same thing. The ‘stdwin’ module is +gone; it was for a platform-independent windowing toolkit that’s no +longer developed. + +A number of modules have been moved to the ‘lib-old’ subdirectory: +‘cmp’, ‘cmpcache’, ‘dircmp’, ‘dump’, ‘find’, ‘grep’, ‘packmail’, ‘poly’, +‘util’, ‘whatsound’, ‘zmod’. If you have code which relies on a module +that’s been moved to ‘lib-old’, you can simply add that directory to +‘sys.path’ to get them back, but you’re encouraged to update any code +that uses these modules. + + +File: python.info, Node: Acknowledgements<8>, Prev: Deleted and Deprecated Modules, Up: What’s New in Python 2 0 + +1.17.18 Acknowledgements +------------------------ + +The authors would like to thank the following people for offering +suggestions on various drafts of this article: David Bolen, Mark +Hammond, Gregg Hauser, Jeremy Hylton, Fredrik Lundh, Detlef Lannert, +Aahz Maruch, Skip Montanaro, Vladimir Marangozov, Tobias Polzin, Guido +van Rossum, Neil Schemenauer, and Russ Schmidt. + +The “Changelog” is an HTML version of the file built(1) from the +contents of the Misc/NEWS.d(2) directory tree, which contains `all' +nontrivial changes to Python for the current version. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/blurb + + (2) https://github.com/python/cpython/tree/3.8/Misc/NEWS.d + + +File: python.info, Node: Changelog, Prev: What’s New in Python 2 0, Up: What’s New in Python + +1.18 Changelog +============== + +`The NEWS file is not available.' + + +File: python.info, Node: The Python Tutorial, Next: Python Setup and Usage, Prev: What’s New in Python, Up: Top + +2 The Python Tutorial +********************* + +Python is an easy to learn, powerful programming language. It has +efficient high-level data structures and a simple but effective approach +to object-oriented programming. Python’s elegant syntax and dynamic +typing, together with its interpreted nature, make it an ideal language +for scripting and rapid application development in many areas on most +platforms. + +The Python interpreter and the extensive standard library are freely +available in source or binary form for all major platforms from the +Python Web site, ‘https://www.python.org/’, and may be freely +distributed. The same site also contains distributions of and pointers +to many free third party Python modules, programs and tools, and +additional documentation. + +The Python interpreter is easily extended with new functions and data +types implemented in C or C++ (or other languages callable from C). +Python is also suitable as an extension language for customizable +applications. + +This tutorial introduces the reader informally to the basic concepts and +features of the Python language and system. It helps to have a Python +interpreter handy for hands-on experience, but all examples are +self-contained, so the tutorial can be read off-line as well. + +For a description of standard objects and modules, see *note The Python +Standard Library: e8e. *note The Python Language Reference: e8f. gives +a more formal definition of the language. To write extensions in C or +C++, read *note Extending and Embedding the Python Interpreter: e90. and +*note Python/C API Reference Manual: e91. There are also several books +covering Python in depth. + +This tutorial does not attempt to be comprehensive and cover every +single feature, or even every commonly used feature. Instead, it +introduces many of Python’s most noteworthy features, and will give you +a good idea of the language’s flavor and style. After reading it, you +will be able to read and write Python modules and programs, and you will +be ready to learn more about the various Python library modules +described in *note The Python Standard Library: e8e. + +The *note Glossary: e92. is also worth going through. + +* Menu: + +* Whetting Your Appetite:: +* Using the Python Interpreter:: +* An Informal Introduction to Python:: +* More Control Flow Tools:: +* Data Structures:: +* Modules:: +* Input and Output:: +* Errors and Exceptions:: +* Classes:: +* Brief Tour of the Standard Library:: +* Brief Tour of the Standard Library — Part II:: +* Virtual Environments and Packages:: +* What Now?:: +* Interactive Input Editing and History Substitution:: +* Floating Point Arithmetic; Issues and Limitations: Floating Point Arithmetic Issues and Limitations. +* Appendix:: + + +File: python.info, Node: Whetting Your Appetite, Next: Using the Python Interpreter, Up: The Python Tutorial + +2.1 Whetting Your Appetite +========================== + +If you do much work on computers, eventually you find that there’s some +task you’d like to automate. For example, you may wish to perform a +search-and-replace over a large number of text files, or rename and +rearrange a bunch of photo files in a complicated way. Perhaps you’d +like to write a small custom database, or a specialized GUI application, +or a simple game. + +If you’re a professional software developer, you may have to work with +several C/C++/Java libraries but find the usual +write/compile/test/re-compile cycle is too slow. Perhaps you’re writing +a test suite for such a library and find writing the testing code a +tedious task. Or maybe you’ve written a program that could use an +extension language, and you don’t want to design and implement a whole +new language for your application. + +Python is just the language for you. + +You could write a Unix shell script or Windows batch files for some of +these tasks, but shell scripts are best at moving around files and +changing text data, not well-suited for GUI applications or games. You +could write a C/C++/Java program, but it can take a lot of development +time to get even a first-draft program. Python is simpler to use, +available on Windows, Mac OS X, and Unix operating systems, and will +help you get the job done more quickly. + +Python is simple to use, but it is a real programming language, offering +much more structure and support for large programs than shell scripts or +batch files can offer. On the other hand, Python also offers much more +error checking than C, and, being a `very-high-level language', it has +high-level data types built in, such as flexible arrays and +dictionaries. Because of its more general data types Python is +applicable to a much larger problem domain than Awk or even Perl, yet +many things are at least as easy in Python as in those languages. + +Python allows you to split your program into modules that can be reused +in other Python programs. It comes with a large collection of standard +modules that you can use as the basis of your programs — or as examples +to start learning to program in Python. Some of these modules provide +things like file I/O, system calls, sockets, and even interfaces to +graphical user interface toolkits like Tk. + +Python is an interpreted language, which can save you considerable time +during program development because no compilation and linking is +necessary. The interpreter can be used interactively, which makes it +easy to experiment with features of the language, to write throw-away +programs, or to test functions during bottom-up program development. It +is also a handy desk calculator. + +Python enables programs to be written compactly and readably. Programs +written in Python are typically much shorter than equivalent C, C++, or +Java programs, for several reasons: + + * the high-level data types allow you to express complex operations + in a single statement; + + * statement grouping is done by indentation instead of beginning and + ending brackets; + + * no variable or argument declarations are necessary. + +Python is `extensible': if you know how to program in C it is easy to +add a new built-in function or module to the interpreter, either to +perform critical operations at maximum speed, or to link Python programs +to libraries that may only be available in binary form (such as a +vendor-specific graphics library). Once you are really hooked, you can +link the Python interpreter into an application written in C and use it +as an extension or command language for that application. + +By the way, the language is named after the BBC show “Monty Python’s +Flying Circus” and has nothing to do with reptiles. Making references +to Monty Python skits in documentation is not only allowed, it is +encouraged! + +Now that you are all excited about Python, you’ll want to examine it in +some more detail. Since the best way to learn a language is to use it, +the tutorial invites you to play with the Python interpreter as you +read. + +In the next chapter, the mechanics of using the interpreter are +explained. This is rather mundane information, but essential for trying +out the examples shown later. + +The rest of the tutorial introduces various features of the Python +language and system through examples, beginning with simple expressions, +statements and data types, through functions and modules, and finally +touching upon advanced concepts like exceptions and user-defined +classes. + + +File: python.info, Node: Using the Python Interpreter, Next: An Informal Introduction to Python, Prev: Whetting Your Appetite, Up: The Python Tutorial + +2.2 Using the Python Interpreter +================================ + +* Menu: + +* Invoking the Interpreter:: +* The Interpreter and Its Environment:: + + +File: python.info, Node: Invoking the Interpreter, Next: The Interpreter and Its Environment, Up: Using the Python Interpreter + +2.2.1 Invoking the Interpreter +------------------------------ + +The Python interpreter is usually installed as +‘/usr/local/bin/python3.8’ on those machines where it is available; +putting ‘/usr/local/bin’ in your Unix shell’s search path makes it +possible to start it by typing the command: + + python3.8 + +to the shell. (1) Since the choice of the directory where the +interpreter lives is an installation option, other places are possible; +check with your local Python guru or system administrator. (E.g., +‘/usr/local/python’ is a popular alternative location.) + +On Windows machines where you have installed Python from the *note +Microsoft Store: e9b, the ‘python3.8’ command will be available. If you +have the *note py.exe launcher: 98f. installed, you can use the ‘py’ +command. See *note Excursus; Setting environment variables: e9c. for +other ways to launch Python. + +Typing an end-of-file character (‘Control-D’ on Unix, ‘Control-Z’ on +Windows) at the primary prompt causes the interpreter to exit with a +zero exit status. If that doesn’t work, you can exit the interpreter by +typing the following command: ‘quit()’. + +The interpreter’s line-editing features include interactive editing, +history substitution and code completion on systems that support the GNU +Readline(2) library. Perhaps the quickest check to see whether command +line editing is supported is typing ‘Control-P’ to the first Python +prompt you get. If it beeps, you have command line editing; see +Appendix *note Interactive Input Editing and History Substitution: e9d. +for an introduction to the keys. If nothing appears to happen, or if +‘^P’ is echoed, command line editing isn’t available; you’ll only be +able to use backspace to remove characters from the current line. + +The interpreter operates somewhat like the Unix shell: when called with +standard input connected to a tty device, it reads and executes commands +interactively; when called with a file name argument or with a file as +standard input, it reads and executes a `script' from that file. + +A second way of starting the interpreter is ‘python -c command [arg] +...’, which executes the statement(s) in `command', analogous to the +shell’s *note -c: e9e. option. Since Python statements often contain +spaces or other characters that are special to the shell, it is usually +advised to quote `command' in its entirety with single quotes. + +Some Python modules are also useful as scripts. These can be invoked +using ‘python -m module [arg] ...’, which executes the source file for +`module' as if you had spelled out its full name on the command line. + +When a script file is used, it is sometimes useful to be able to run the +script and enter interactive mode afterwards. This can be done by +passing *note -i: e1f. before the script. + +All command line options are described in *note Command line and +environment: e9f. + +* Menu: + +* Argument Passing:: +* Interactive Mode:: + + ---------- Footnotes ---------- + + (1) (1) On Unix, the Python 3.x interpreter is by default not +installed with the executable named ‘python’, so that it does not +conflict with a simultaneously installed Python 2.x executable. + + (2) https://tiswww.case.edu/php/chet/readline/rltop.html + + +File: python.info, Node: Argument Passing, Next: Interactive Mode, Up: Invoking the Interpreter + +2.2.1.1 Argument Passing +........................ + +When known to the interpreter, the script name and additional arguments +thereafter are turned into a list of strings and assigned to the ‘argv’ +variable in the ‘sys’ module. You can access this list by executing +‘import sys’. The length of the list is at least one; when no script +and no arguments are given, ‘sys.argv[0]’ is an empty string. When the +script name is given as ‘'-'’ (meaning standard input), ‘sys.argv[0]’ is +set to ‘'-'’. When *note -c: e9e. `command' is used, ‘sys.argv[0]’ is +set to ‘'-c'’. When *note -m: 337. `module' is used, ‘sys.argv[0]’ is +set to the full name of the located module. Options found after *note +-c: e9e. `command' or *note -m: 337. `module' are not consumed by the +Python interpreter’s option processing but left in ‘sys.argv’ for the +command or module to handle. + + +File: python.info, Node: Interactive Mode, Prev: Argument Passing, Up: Invoking the Interpreter + +2.2.1.2 Interactive Mode +........................ + +When commands are read from a tty, the interpreter is said to be in +`interactive mode'. In this mode it prompts for the next command with +the `primary prompt', usually three greater-than signs (‘>>>’); for +continuation lines it prompts with the `secondary prompt', by default +three dots (‘...’). The interpreter prints a welcome message stating +its version number and a copyright notice before printing the first +prompt: + + $ python3.8 + Python 3.8 (default, Sep 16 2015, 09:25:04) + [GCC 4.8.2] on linux + Type "help", "copyright", "credits" or "license" for more information. + >>> + +Continuation lines are needed when entering a multi-line construct. As +an example, take a look at this *note if: de7. statement: + + >>> the_world_is_flat = True + >>> if the_world_is_flat: + ... print("Be careful not to fall off!") + ... + Be careful not to fall off! + +For more on interactive mode, see *note Interactive Mode: ea3. + + +File: python.info, Node: The Interpreter and Its Environment, Prev: Invoking the Interpreter, Up: Using the Python Interpreter + +2.2.2 The Interpreter and Its Environment +----------------------------------------- + +* Menu: + +* Source Code Encoding:: + + +File: python.info, Node: Source Code Encoding, Up: The Interpreter and Its Environment + +2.2.2.1 Source Code Encoding +............................ + +By default, Python source files are treated as encoded in UTF-8. In +that encoding, characters of most languages in the world can be used +simultaneously in string literals, identifiers and comments — although +the standard library only uses ASCII characters for identifiers, a +convention that any portable code should follow. To display all these +characters properly, your editor must recognize that the file is UTF-8, +and it must use a font that supports all the characters in the file. + +To declare an encoding other than the default one, a special comment +line should be added as the `first' line of the file. The syntax is as +follows: + + # -*- coding: encoding -*- + +where `encoding' is one of the valid *note codecs: 1c. supported by +Python. + +For example, to declare that Windows-1252 encoding is to be used, the +first line of your source code file should be: + + # -*- coding: cp1252 -*- + +One exception to the `first line' rule is when the source code starts +with a *note UNIX “shebang” line: ea8. In this case, the encoding +declaration should be added as the second line of the file. For +example: + + #!/usr/bin/env python3 + # -*- coding: cp1252 -*- + + +File: python.info, Node: An Informal Introduction to Python, Next: More Control Flow Tools, Prev: Using the Python Interpreter, Up: The Python Tutorial + +2.3 An Informal Introduction to Python +====================================== + +In the following examples, input and output are distinguished by the +presence or absence of prompts (*note >>>: eac. and *note …: ead.): to +repeat the example, you must type everything after the prompt, when the +prompt appears; lines that do not begin with a prompt are output from +the interpreter. Note that a secondary prompt on a line by itself in an +example means you must type a blank line; this is used to end a +multi-line command. + +Many of the examples in this manual, even those entered at the +interactive prompt, include comments. Comments in Python start with the +hash character, ‘#’, and extend to the end of the physical line. A +comment may appear at the start of a line or following whitespace or +code, but not within a string literal. A hash character within a string +literal is just a hash character. Since comments are to clarify code +and are not interpreted by Python, they may be omitted when typing in +examples. + +Some examples: + + # this is the first comment + spam = 1 # and this is the second comment + # ... and now a third! + text = "# This is not a comment because it's inside quotes." + +* Menu: + +* Using Python as a Calculator:: +* First Steps Towards Programming:: + + +File: python.info, Node: Using Python as a Calculator, Next: First Steps Towards Programming, Up: An Informal Introduction to Python + +2.3.1 Using Python as a Calculator +---------------------------------- + +Let’s try some simple Python commands. Start the interpreter and wait +for the primary prompt, ‘>>>’. (It shouldn’t take long.) + +* Menu: + +* Numbers:: +* Strings:: +* Lists:: + + +File: python.info, Node: Numbers, Next: Strings, Up: Using Python as a Calculator + +2.3.1.1 Numbers +............... + +The interpreter acts as a simple calculator: you can type an expression +at it and it will write the value. Expression syntax is +straightforward: the operators ‘+’, ‘-’, ‘*’ and ‘/’ work just like in +most other languages (for example, Pascal or C); parentheses (‘()’) can +be used for grouping. For example: + + >>> 2 + 2 + 4 + >>> 50 - 5*6 + 20 + >>> (50 - 5*6) / 4 + 5.0 + >>> 8 / 5 # division always returns a floating point number + 1.6 + +The integer numbers (e.g. ‘2’, ‘4’, ‘20’) have type *note int: 184, the +ones with a fractional part (e.g. ‘5.0’, ‘1.6’) have type *note float: +187. We will see more about numeric types later in the tutorial. + +Division (‘/’) always returns a float. To do *note floor division: eb2. +and get an integer result (discarding any fractional result) you can use +the ‘//’ operator; to calculate the remainder you can use ‘%’: + + >>> 17 / 3 # classic division returns a float + 5.666666666666667 + >>> + >>> 17 // 3 # floor division discards the fractional part + 5 + >>> 17 % 3 # the % operator returns the remainder of the division + 2 + >>> 5 * 3 + 2 # result * divisor + remainder + 17 + +With Python, it is possible to use the ‘**’ operator to calculate powers +(1): + + >>> 5 ** 2 # 5 squared + 25 + >>> 2 ** 7 # 2 to the power of 7 + 128 + +The equal sign (‘=’) is used to assign a value to a variable. +Afterwards, no result is displayed before the next interactive prompt: + + >>> width = 20 + >>> height = 5 * 9 + >>> width * height + 900 + +If a variable is not “defined” (assigned a value), trying to use it will +give you an error: + + >>> n # try to access an undefined variable + Traceback (most recent call last): + File "", line 1, in + NameError: name 'n' is not defined + +There is full support for floating point; operators with mixed type +operands convert the integer operand to floating point: + + >>> 4 * 3.75 - 1 + 14.0 + +In interactive mode, the last printed expression is assigned to the +variable ‘_’. This means that when you are using Python as a desk +calculator, it is somewhat easier to continue calculations, for example: + + >>> tax = 12.5 / 100 + >>> price = 100.50 + >>> price * tax + 12.5625 + >>> price + _ + 113.0625 + >>> round(_, 2) + 113.06 + +This variable should be treated as read-only by the user. Don’t +explicitly assign a value to it — you would create an independent local +variable with the same name masking the built-in variable with its magic +behavior. + +In addition to *note int: 184. and *note float: 187, Python supports +other types of numbers, such as *note Decimal: 188. and *note Fraction: +185. Python also has built-in support for *note complex numbers: eb3, +and uses the ‘j’ or ‘J’ suffix to indicate the imaginary part (e.g. +‘3+5j’). + + ---------- Footnotes ---------- + + (1) (1) Since ‘**’ has higher precedence than ‘-’, ‘-3**2’ will be +interpreted as ‘-(3**2)’ and thus result in ‘-9’. To avoid this and get +‘9’, you can use ‘(-3)**2’. + + +File: python.info, Node: Strings, Next: Lists, Prev: Numbers, Up: Using Python as a Calculator + +2.3.1.2 Strings +............... + +Besides numbers, Python can also manipulate strings, which can be +expressed in several ways. They can be enclosed in single quotes +(‘'...'’) or double quotes (‘"..."’) with the same result (1). ‘\’ can +be used to escape quotes: + + >>> 'spam eggs' # single quotes + 'spam eggs' + >>> 'doesn\'t' # use \' to escape the single quote... + "doesn't" + >>> "doesn't" # ...or use double quotes instead + "doesn't" + >>> '"Yes," they said.' + '"Yes," they said.' + >>> "\"Yes,\" they said." + '"Yes," they said.' + >>> '"Isn\'t," they said.' + '"Isn\'t," they said.' + +In the interactive interpreter, the output string is enclosed in quotes +and special characters are escaped with backslashes. While this might +sometimes look different from the input (the enclosing quotes could +change), the two strings are equivalent. The string is enclosed in +double quotes if the string contains a single quote and no double +quotes, otherwise it is enclosed in single quotes. The *note print(): +886. function produces a more readable output, by omitting the enclosing +quotes and by printing escaped and special characters: + + >>> '"Isn\'t," they said.' + '"Isn\'t," they said.' + >>> print('"Isn\'t," they said.') + "Isn't," they said. + >>> s = 'First line.\nSecond line.' # \n means newline + >>> s # without print(), \n is included in the output + 'First line.\nSecond line.' + >>> print(s) # with print(), \n produces a new line + First line. + Second line. + +If you don’t want characters prefaced by ‘\’ to be interpreted as +special characters, you can use `raw strings' by adding an ‘r’ before +the first quote: + + >>> print('C:\some\name') # here \n means newline! + C:\some + ame + >>> print(r'C:\some\name') # note the r before the quote + C:\some\name + +String literals can span multiple lines. One way is using +triple-quotes: ‘"""..."""’ or ‘'''...'''’. End of lines are +automatically included in the string, but it’s possible to prevent this +by adding a ‘\’ at the end of the line. The following example: + + print("""\ + Usage: thingy [OPTIONS] + -h Display this usage message + -H hostname Hostname to connect to + """) + +produces the following output (note that the initial newline is not +included): + + Usage: thingy [OPTIONS] + -h Display this usage message + -H hostname Hostname to connect to + +Strings can be concatenated (glued together) with the ‘+’ operator, and +repeated with ‘*’: + + >>> # 3 times 'un', followed by 'ium' + >>> 3 * 'un' + 'ium' + 'unununium' + +Two or more `string literals' (i.e. the ones enclosed between quotes) +next to each other are automatically concatenated. + + >>> 'Py' 'thon' + 'Python' + +This feature is particularly useful when you want to break long strings: + + >>> text = ('Put several strings within parentheses ' + ... 'to have them joined together.') + >>> text + 'Put several strings within parentheses to have them joined together.' + +This only works with two literals though, not with variables or +expressions: + + >>> prefix = 'Py' + >>> prefix 'thon' # can't concatenate a variable and a string literal + File "", line 1 + prefix 'thon' + ^ + SyntaxError: invalid syntax + >>> ('un' * 3) 'ium' + File "", line 1 + ('un' * 3) 'ium' + ^ + SyntaxError: invalid syntax + +If you want to concatenate variables or a variable and a literal, use +‘+’: + + >>> prefix + 'thon' + 'Python' + +Strings can be `indexed' (subscripted), with the first character having +index 0. There is no separate character type; a character is simply a +string of size one: + + >>> word = 'Python' + >>> word[0] # character in position 0 + 'P' + >>> word[5] # character in position 5 + 'n' + +Indices may also be negative numbers, to start counting from the right: + + >>> word[-1] # last character + 'n' + >>> word[-2] # second-last character + 'o' + >>> word[-6] + 'P' + +Note that since -0 is the same as 0, negative indices start from -1. + +In addition to indexing, `slicing' is also supported. While indexing is +used to obtain individual characters, `slicing' allows you to obtain +substring: + + >>> word[0:2] # characters from position 0 (included) to 2 (excluded) + 'Py' + >>> word[2:5] # characters from position 2 (included) to 5 (excluded) + 'tho' + +Note how the start is always included, and the end always excluded. +This makes sure that ‘s[:i] + s[i:]’ is always equal to ‘s’: + + >>> word[:2] + word[2:] + 'Python' + >>> word[:4] + word[4:] + 'Python' + +Slice indices have useful defaults; an omitted first index defaults to +zero, an omitted second index defaults to the size of the string being +sliced. + + >>> word[:2] # character from the beginning to position 2 (excluded) + 'Py' + >>> word[4:] # characters from position 4 (included) to the end + 'on' + >>> word[-2:] # characters from the second-last (included) to the end + 'on' + +One way to remember how slices work is to think of the indices as +pointing `between' characters, with the left edge of the first character +numbered 0. Then the right edge of the last character of a string of +`n' characters has index `n', for example: + + +---+---+---+---+---+---+ + | P | y | t | h | o | n | + +---+---+---+---+---+---+ + 0 1 2 3 4 5 6 + -6 -5 -4 -3 -2 -1 + +The first row of numbers gives the position of the indices 0…6 in the +string; the second row gives the corresponding negative indices. The +slice from `i' to `j' consists of all characters between the edges +labeled `i' and `j', respectively. + +For non-negative indices, the length of a slice is the difference of the +indices, if both are within bounds. For example, the length of +‘word[1:3]’ is 2. + +Attempting to use an index that is too large will result in an error: + + >>> word[42] # the word only has 6 characters + Traceback (most recent call last): + File "", line 1, in + IndexError: string index out of range + +However, out of range slice indexes are handled gracefully when used for +slicing: + + >>> word[4:42] + 'on' + >>> word[42:] + '' + +Python strings cannot be changed — they are *note immutable: eb6. +Therefore, assigning to an indexed position in the string results in an +error: + + >>> word[0] = 'J' + Traceback (most recent call last): + File "", line 1, in + TypeError: 'str' object does not support item assignment + >>> word[2:] = 'py' + Traceback (most recent call last): + File "", line 1, in + TypeError: 'str' object does not support item assignment + +If you need a different string, you should create a new one: + + >>> 'J' + word[1:] + 'Jython' + >>> word[:2] + 'py' + 'Pypy' + +The built-in function *note len(): 150. returns the length of a string: + + >>> s = 'supercalifragilisticexpialidocious' + >>> len(s) + 34 + +See also +........ + +*note Text Sequence Type — str: eb7. + + Strings are examples of `sequence types', and support the common + operations supported by such types. + +*note String Methods: eb8. + + Strings support a large number of methods for basic transformations + and searching. + +*note Formatted string literals: 15c. + + String literals that have embedded expressions. + +*note Format String Syntax: d1a. + + Information about string formatting with *note str.format(): 4da. + +*note printf-style String Formatting: eb9. + + The old formatting operations invoked when strings are the left + operand of the ‘%’ operator are described in more detail here. + + ---------- Footnotes ---------- + + (1) (2) Unlike other languages, special characters such as ‘\n’ have +the same meaning with both single (‘'...'’) and double (‘"..."’) quotes. +The only difference between the two is that within single quotes you +don’t need to escape ‘"’ (but you have to escape ‘\'’) and vice versa. + + +File: python.info, Node: Lists, Prev: Strings, Up: Using Python as a Calculator + +2.3.1.3 Lists +............. + +Python knows a number of `compound' data types, used to group together +other values. The most versatile is the `list', which can be written as +a list of comma-separated values (items) between square brackets. Lists +might contain items of different types, but usually the items all have +the same type. + + >>> squares = [1, 4, 9, 16, 25] + >>> squares + [1, 4, 9, 16, 25] + +Like strings (and all other built-in *note sequence: ebc. types), lists +can be indexed and sliced: + + >>> squares[0] # indexing returns the item + 1 + >>> squares[-1] + 25 + >>> squares[-3:] # slicing returns a new list + [9, 16, 25] + +All slice operations return a new list containing the requested +elements. This means that the following slice returns a *note shallow +copy: ebd. of the list: + + >>> squares[:] + [1, 4, 9, 16, 25] + +Lists also support operations like concatenation: + + >>> squares + [36, 49, 64, 81, 100] + [1, 4, 9, 16, 25, 36, 49, 64, 81, 100] + +Unlike strings, which are *note immutable: eb6, lists are a *note +mutable: ebe. type, i.e. it is possible to change their content: + + >>> cubes = [1, 8, 27, 65, 125] # something's wrong here + >>> 4 ** 3 # the cube of 4 is 64, not 65! + 64 + >>> cubes[3] = 64 # replace the wrong value + >>> cubes + [1, 8, 27, 64, 125] + +You can also add new items at the end of the list, by using the +‘append()’ `method' (we will see more about methods later): + + >>> cubes.append(216) # add the cube of 6 + >>> cubes.append(7 ** 3) # and the cube of 7 + >>> cubes + [1, 8, 27, 64, 125, 216, 343] + +Assignment to slices is also possible, and this can even change the size +of the list or clear it entirely: + + >>> letters = ['a', 'b', 'c', 'd', 'e', 'f', 'g'] + >>> letters + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] + >>> # replace some values + >>> letters[2:5] = ['C', 'D', 'E'] + >>> letters + ['a', 'b', 'C', 'D', 'E', 'f', 'g'] + >>> # now remove them + >>> letters[2:5] = [] + >>> letters + ['a', 'b', 'f', 'g'] + >>> # clear the list by replacing all the elements with an empty list + >>> letters[:] = [] + >>> letters + [] + +The built-in function *note len(): 150. also applies to lists: + + >>> letters = ['a', 'b', 'c', 'd'] + >>> len(letters) + 4 + +It is possible to nest lists (create lists containing other lists), for +example: + + >>> a = ['a', 'b', 'c'] + >>> n = [1, 2, 3] + >>> x = [a, n] + >>> x + [['a', 'b', 'c'], [1, 2, 3]] + >>> x[0] + ['a', 'b', 'c'] + >>> x[0][1] + 'b' + + +File: python.info, Node: First Steps Towards Programming, Prev: Using Python as a Calculator, Up: An Informal Introduction to Python + +2.3.2 First Steps Towards Programming +------------------------------------- + +Of course, we can use Python for more complicated tasks than adding two +and two together. For instance, we can write an initial sub-sequence of +the Fibonacci series(1) as follows: + + >>> # Fibonacci series: + ... # the sum of two elements defines the next + ... a, b = 0, 1 + >>> while a < 10: + ... print(a) + ... a, b = b, a+b + ... + 0 + 1 + 1 + 2 + 3 + 5 + 8 + +This example introduces several new features. + + * The first line contains a `multiple assignment': the variables ‘a’ + and ‘b’ simultaneously get the new values 0 and 1. On the last + line this is used again, demonstrating that the expressions on the + right-hand side are all evaluated first before any of the + assignments take place. The right-hand side expressions are + evaluated from the left to the right. + + * The *note while: ec1. loop executes as long as the condition (here: + ‘a < 10’) remains true. In Python, like in C, any non-zero integer + value is true; zero is false. The condition may also be a string + or list value, in fact any sequence; anything with a non-zero + length is true, empty sequences are false. The test used in the + example is a simple comparison. The standard comparison operators + are written the same as in C: ‘<’ (less than), ‘>’ (greater than), + ‘==’ (equal to), ‘<=’ (less than or equal to), ‘>=’ (greater than + or equal to) and ‘!=’ (not equal to). + + * The `body' of the loop is `indented': indentation is Python’s way + of grouping statements. At the interactive prompt, you have to + type a tab or space(s) for each indented line. In practice you + will prepare more complicated input for Python with a text editor; + all decent text editors have an auto-indent facility. When a + compound statement is entered interactively, it must be followed by + a blank line to indicate completion (since the parser cannot guess + when you have typed the last line). Note that each line within a + basic block must be indented by the same amount. + + * The *note print(): 886. function writes the value of the + argument(s) it is given. It differs from just writing the + expression you want to write (as we did earlier in the calculator + examples) in the way it handles multiple arguments, floating point + quantities, and strings. Strings are printed without quotes, and a + space is inserted between items, so you can format things nicely, + like this: + + >>> i = 256*256 + >>> print('The value of i is', i) + The value of i is 65536 + + The keyword argument `end' can be used to avoid the newline after + the output, or end the output with a different string: + + >>> a, b = 0, 1 + >>> while a < 1000: + ... print(a, end=',') + ... a, b = b, a+b + ... + 0,1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987, + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Fibonacci_number + + +File: python.info, Node: More Control Flow Tools, Next: Data Structures, Prev: An Informal Introduction to Python, Up: The Python Tutorial + +2.4 More Control Flow Tools +=========================== + +Besides the *note while: ec1. statement just introduced, Python uses the +usual flow control statements known from other languages, with some +twists. + +* Menu: + +* if Statements:: +* for Statements:: +* The range() Function: The range Function. +* break and continue Statements, and else Clauses on Loops: break and continue Statements and else Clauses on Loops. +* pass Statements:: +* Defining Functions:: +* More on Defining Functions:: +* Intermezzo; Coding Style: Intermezzo Coding Style. + + +File: python.info, Node: if Statements, Next: for Statements, Up: More Control Flow Tools + +2.4.1 ‘if’ Statements +--------------------- + +Perhaps the most well-known statement type is the *note if: de7. +statement. For example: + + >>> x = int(input("Please enter an integer: ")) + Please enter an integer: 42 + >>> if x < 0: + ... x = 0 + ... print('Negative changed to zero') + ... elif x == 0: + ... print('Zero') + ... elif x == 1: + ... print('Single') + ... else: + ... print('More') + ... + More + +There can be zero or more *note elif: ec7. parts, and the *note else: +ec8. part is optional. The keyword ‘‘elif’’ is short for ‘else if’, and +is useful to avoid excessive indentation. An ‘if’ … ‘elif’ … ‘elif’ … +sequence is a substitute for the ‘switch’ or ‘case’ statements found in +other languages. + + +File: python.info, Node: for Statements, Next: The range Function, Prev: if Statements, Up: More Control Flow Tools + +2.4.2 ‘for’ Statements +---------------------- + +The *note for: c30. statement in Python differs a bit from what you may +be used to in C or Pascal. Rather than always iterating over an +arithmetic progression of numbers (like in Pascal), or giving the user +the ability to define both the iteration step and halting condition (as +C), Python’s ‘for’ statement iterates over the items of any sequence (a +list or a string), in the order that they appear in the sequence. For +example (no pun intended): + + >>> # Measure some strings: + ... words = ['cat', 'window', 'defenestrate'] + >>> for w in words: + ... print(w, len(w)) + ... + cat 3 + window 6 + defenestrate 12 + +Code that modifies a collection while iterating over that same +collection can be tricky to get right. Instead, it is usually more +straight-forward to loop over a copy of the collection or to create a +new collection: + + # Strategy: Iterate over a copy + for user, status in users.copy().items(): + if status == 'inactive': + del users[user] + + # Strategy: Create a new collection + active_users = {} + for user, status in users.items(): + if status == 'active': + active_users[user] = status + + +File: python.info, Node: The range Function, Next: break and continue Statements and else Clauses on Loops, Prev: for Statements, Up: More Control Flow Tools + +2.4.3 The ‘range()’ Function +---------------------------- + +If you do need to iterate over a sequence of numbers, the built-in +function *note range(): 9be. comes in handy. It generates arithmetic +progressions: + + >>> for i in range(5): + ... print(i) + ... + 0 + 1 + 2 + 3 + 4 + +The given end point is never part of the generated sequence; ‘range(10)’ +generates 10 values, the legal indices for items of a sequence of length +10. It is possible to let the range start at another number, or to +specify a different increment (even negative; sometimes this is called +the ‘step’): + + range(5, 10) + 5, 6, 7, 8, 9 + + range(0, 10, 3) + 0, 3, 6, 9 + + range(-10, -100, -30) + -10, -40, -70 + +To iterate over the indices of a sequence, you can combine *note +range(): 9be. and *note len(): 150. as follows: + + >>> a = ['Mary', 'had', 'a', 'little', 'lamb'] + >>> for i in range(len(a)): + ... print(i, a[i]) + ... + 0 Mary + 1 had + 2 a + 3 little + 4 lamb + +In most such cases, however, it is convenient to use the *note +enumerate(): de3. function, see *note Looping Techniques: ecd. + +A strange thing happens if you just print a range: + + >>> print(range(10)) + range(0, 10) + +In many ways the object returned by *note range(): 9be. behaves as if it +is a list, but in fact it isn’t. It is an object which returns the +successive items of the desired sequence when you iterate over it, but +it doesn’t really make the list, thus saving space. + +We say such an object is *note iterable: bac, that is, suitable as a +target for functions and constructs that expect something from which +they can obtain successive items until the supply is exhausted. We have +seen that the *note for: c30. statement is such a construct, while an +example of a function that takes an iterable is *note sum(): 1e5.: + + >>> sum(range(4)) # 0 + 1 + 2 + 3 + 6 + +Later we will see more functions that return iterables and take +iterables as arguments. Lastly, maybe you are curious about how to get +a list from a range. Here is the solution: + + >>> list(range(4)) + [0, 1, 2, 3] + +In chapter *note Data Structures: ece, we will discuss in more detail +about *note list(): 262. + + +File: python.info, Node: break and continue Statements and else Clauses on Loops, Next: pass Statements, Prev: The range Function, Up: More Control Flow Tools + +2.4.4 ‘break’ and ‘continue’ Statements, and ‘else’ Clauses on Loops +-------------------------------------------------------------------- + +The *note break: 2db. statement, like in C, breaks out of the innermost +enclosing *note for: c30. or *note while: ec1. loop. + +Loop statements may have an ‘else’ clause; it is executed when the loop +terminates through exhaustion of the iterable (with *note for: c30.) or +when the condition becomes false (with *note while: ec1.), but not when +the loop is terminated by a *note break: 2db. statement. This is +exemplified by the following loop, which searches for prime numbers: + + >>> for n in range(2, 10): + ... for x in range(2, n): + ... if n % x == 0: + ... print(n, 'equals', x, '*', n//x) + ... break + ... else: + ... # loop fell through without finding a factor + ... print(n, 'is a prime number') + ... + 2 is a prime number + 3 is a prime number + 4 equals 2 * 2 + 5 is a prime number + 6 equals 2 * 3 + 7 is a prime number + 8 equals 2 * 4 + 9 equals 3 * 3 + +(Yes, this is the correct code. Look closely: the ‘else’ clause belongs +to the *note for: c30. loop, `not' the *note if: de7. statement.) + +When used with a loop, the ‘else’ clause has more in common with the +‘else’ clause of a *note try: d72. statement than it does with that of +*note if: de7. statements: a *note try: d72. statement’s ‘else’ clause +runs when no exception occurs, and a loop’s ‘else’ clause runs when no +‘break’ occurs. For more on the ‘try’ statement and exceptions, see +*note Handling Exceptions: ed1. + +The *note continue: 181. statement, also borrowed from C, continues with +the next iteration of the loop: + + >>> for num in range(2, 10): + ... if num % 2 == 0: + ... print("Found an even number", num) + ... continue + ... print("Found an odd number", num) + Found an even number 2 + Found an odd number 3 + Found an even number 4 + Found an odd number 5 + Found an even number 6 + Found an odd number 7 + Found an even number 8 + Found an odd number 9 + + +File: python.info, Node: pass Statements, Next: Defining Functions, Prev: break and continue Statements and else Clauses on Loops, Up: More Control Flow Tools + +2.4.5 ‘pass’ Statements +----------------------- + +The *note pass: ed4. statement does nothing. It can be used when a +statement is required syntactically but the program requires no action. +For example: + + >>> while True: + ... pass # Busy-wait for keyboard interrupt (Ctrl+C) + ... + +This is commonly used for creating minimal classes: + + >>> class MyEmptyClass: + ... pass + ... + +Another place *note pass: ed4. can be used is as a place-holder for a +function or conditional body when you are working on new code, allowing +you to keep thinking at a more abstract level. The ‘pass’ is silently +ignored: + + >>> def initlog(*args): + ... pass # Remember to implement this! + ... + + +File: python.info, Node: Defining Functions, Next: More on Defining Functions, Prev: pass Statements, Up: More Control Flow Tools + +2.4.6 Defining Functions +------------------------ + +We can create a function that writes the Fibonacci series to an +arbitrary boundary: + + >>> def fib(n): # write Fibonacci series up to n + ... """Print a Fibonacci series up to n.""" + ... a, b = 0, 1 + ... while a < n: + ... print(a, end=' ') + ... a, b = b, a+b + ... print() + ... + >>> # Now call the function we just defined: + ... fib(2000) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 + +The keyword *note def: dbe. introduces a function `definition'. It must +be followed by the function name and the parenthesized list of formal +parameters. The statements that form the body of the function start at +the next line, and must be indented. + +The first statement of the function body can optionally be a string +literal; this string literal is the function’s documentation string, or +`docstring'. (More about docstrings can be found in the section *note +Documentation Strings: ed7.) There are tools which use docstrings to +automatically produce online or printed documentation, or to let the +user interactively browse through code; it’s good practice to include +docstrings in code that you write, so make a habit of it. + +The `execution' of a function introduces a new symbol table used for the +local variables of the function. More precisely, all variable +assignments in a function store the value in the local symbol table; +whereas variable references first look in the local symbol table, then +in the local symbol tables of enclosing functions, then in the global +symbol table, and finally in the table of built-in names. Thus, global +variables and variables of enclosing functions cannot be directly +assigned a value within a function (unless, for global variables, named +in a *note global: ed8. statement, or, for variables of enclosing +functions, named in a *note nonlocal: c3f. statement), although they may +be referenced. + +The actual parameters (arguments) to a function call are introduced in +the local symbol table of the called function when it is called; thus, +arguments are passed using `call by value' (where the `value' is always +an object `reference', not the value of the object). (1) When a +function calls another function, or calls itself recursively, a new +local symbol table is created for that call. + +A function definition associates the function name with the function +object in the current symbol table. The interpreter recognizes the +object pointed to by that name as a user-defined function. Other names +can also point to that same function object and can also be used to +access the function: + + >>> fib + + >>> f = fib + >>> f(100) + 0 1 1 2 3 5 8 13 21 34 55 89 + +Coming from other languages, you might object that ‘fib’ is not a +function but a procedure since it doesn’t return a value. In fact, even +functions without a *note return: 190. statement do return a value, +albeit a rather boring one. This value is called ‘None’ (it’s a +built-in name). Writing the value ‘None’ is normally suppressed by the +interpreter if it would be the only value written. You can see it if +you really want to using *note print(): 886.: + + >>> fib(0) + >>> print(fib(0)) + None + +It is simple to write a function that returns a list of the numbers of +the Fibonacci series, instead of printing it: + + >>> def fib2(n): # return Fibonacci series up to n + ... """Return a list containing the Fibonacci series up to n.""" + ... result = [] + ... a, b = 0, 1 + ... while a < n: + ... result.append(a) # see below + ... a, b = b, a+b + ... return result + ... + >>> f100 = fib2(100) # call it + >>> f100 # write the result + [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89] + +This example, as usual, demonstrates some new Python features: + + * The *note return: 190. statement returns with a value from a + function. ‘return’ without an expression argument returns ‘None’. + Falling off the end of a function also returns ‘None’. + + * The statement ‘result.append(a)’ calls a `method' of the list + object ‘result’. A method is a function that ‘belongs’ to an + object and is named ‘obj.methodname’, where ‘obj’ is some object + (this may be an expression), and ‘methodname’ is the name of a + method that is defined by the object’s type. Different types + define different methods. Methods of different types may have the + same name without causing ambiguity. (It is possible to define + your own object types and methods, using `classes', see *note + Classes: ed9.) The method ‘append()’ shown in the example is + defined for list objects; it adds a new element at the end of the + list. In this example it is equivalent to ‘result = result + [a]’, + but more efficient. + + ---------- Footnotes ---------- + + (1) (1) Actually, `call by object reference' would be a better +description, since if a mutable object is passed, the caller will see +any changes the callee makes to it (items inserted into a list). + + +File: python.info, Node: More on Defining Functions, Next: Intermezzo Coding Style, Prev: Defining Functions, Up: More Control Flow Tools + +2.4.7 More on Defining Functions +-------------------------------- + +It is also possible to define functions with a variable number of +arguments. There are three forms, which can be combined. + +* Menu: + +* Default Argument Values:: +* Keyword Arguments:: +* Special parameters:: +* Arbitrary Argument Lists:: +* Unpacking Argument Lists:: +* Lambda Expressions:: +* Documentation Strings:: +* Function Annotations:: + + +File: python.info, Node: Default Argument Values, Next: Keyword Arguments, Up: More on Defining Functions + +2.4.7.1 Default Argument Values +............................... + +The most useful form is to specify a default value for one or more +arguments. This creates a function that can be called with fewer +arguments than it is defined to allow. For example: + + def ask_ok(prompt, retries=4, reminder='Please try again!'): + while True: + ok = input(prompt) + if ok in ('y', 'ye', 'yes'): + return True + if ok in ('n', 'no', 'nop', 'nope'): + return False + retries = retries - 1 + if retries < 0: + raise ValueError('invalid user response') + print(reminder) + +This function can be called in several ways: + + * giving only the mandatory argument: ‘ask_ok('Do you really want to + quit?')’ + + * giving one of the optional arguments: ‘ask_ok('OK to overwrite the + file?', 2)’ + + * or even giving all arguments: ‘ask_ok('OK to overwrite the file?', + 2, 'Come on, only yes or no!')’ + +This example also introduces the *note in: 7a3. keyword. This tests +whether or not a sequence contains a certain value. + +The default values are evaluated at the point of function definition in +the `defining' scope, so that + + i = 5 + + def f(arg=i): + print(arg) + + i = 6 + f() + +will print ‘5’. + +`Important warning:' The default value is evaluated only once. This +makes a difference when the default is a mutable object such as a list, +dictionary, or instances of most classes. For example, the following +function accumulates the arguments passed to it on subsequent calls: + + def f(a, L=[]): + L.append(a) + return L + + print(f(1)) + print(f(2)) + print(f(3)) + +This will print + + [1] + [1, 2] + [1, 2, 3] + +If you don’t want the default to be shared between subsequent calls, you +can write the function like this instead: + + def f(a, L=None): + if L is None: + L = [] + L.append(a) + return L + + +File: python.info, Node: Keyword Arguments, Next: Special parameters, Prev: Default Argument Values, Up: More on Defining Functions + +2.4.7.2 Keyword Arguments +......................... + +Functions can also be called using *note keyword arguments: 5c4. of the +form ‘kwarg=value’. For instance, the following function: + + def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'): + print("-- This parrot wouldn't", action, end=' ') + print("if you put", voltage, "volts through it.") + print("-- Lovely plumage, the", type) + print("-- It's", state, "!") + +accepts one required argument (‘voltage’) and three optional arguments +(‘state’, ‘action’, and ‘type’). This function can be called in any of +the following ways: + + parrot(1000) # 1 positional argument + parrot(voltage=1000) # 1 keyword argument + parrot(voltage=1000000, action='VOOOOOM') # 2 keyword arguments + parrot(action='VOOOOOM', voltage=1000000) # 2 keyword arguments + parrot('a million', 'bereft of life', 'jump') # 3 positional arguments + parrot('a thousand', state='pushing up the daisies') # 1 positional, 1 keyword + +but all the following calls would be invalid: + + parrot() # required argument missing + parrot(voltage=5.0, 'dead') # non-keyword argument after a keyword argument + parrot(110, voltage=220) # duplicate value for the same argument + parrot(actor='John Cleese') # unknown keyword argument + +In a function call, keyword arguments must follow positional arguments. +All the keyword arguments passed must match one of the arguments +accepted by the function (e.g. ‘actor’ is not a valid argument for the +‘parrot’ function), and their order is not important. This also +includes non-optional arguments (e.g. ‘parrot(voltage=1000)’ is valid +too). No argument may receive a value more than once. Here’s an +example that fails due to this restriction: + + >>> def function(a): + ... pass + ... + >>> function(0, a=0) + Traceback (most recent call last): + File "", line 1, in + TypeError: function() got multiple values for keyword argument 'a' + +When a final formal parameter of the form ‘**name’ is present, it +receives a dictionary (see *note Mapping Types — dict: 2fc.) containing +all keyword arguments except for those corresponding to a formal +parameter. This may be combined with a formal parameter of the form +‘*name’ (described in the next subsection) which receives a *note tuple: +ee0. containing the positional arguments beyond the formal parameter +list. (‘*name’ must occur before ‘**name’.) For example, if we define +a function like this: + + def cheeseshop(kind, *arguments, **keywords): + print("-- Do you have any", kind, "?") + print("-- I'm sorry, we're all out of", kind) + for arg in arguments: + print(arg) + print("-" * 40) + for kw in keywords: + print(kw, ":", keywords[kw]) + +It could be called like this: + + cheeseshop("Limburger", "It's very runny, sir.", + "It's really very, VERY runny, sir.", + shopkeeper="Michael Palin", + client="John Cleese", + sketch="Cheese Shop Sketch") + +and of course it would print: + + -- Do you have any Limburger ? + -- I'm sorry, we're all out of Limburger + It's very runny, sir. + It's really very, VERY runny, sir. + ---------------------------------------- + shopkeeper : Michael Palin + client : John Cleese + sketch : Cheese Shop Sketch + +Note that the order in which the keyword arguments are printed is +guaranteed to match the order in which they were provided in the +function call. + + +File: python.info, Node: Special parameters, Next: Arbitrary Argument Lists, Prev: Keyword Arguments, Up: More on Defining Functions + +2.4.7.3 Special parameters +.......................... + +By default, arguments may be passed to a Python function either by +position or explicitly by keyword. For readability and performance, it +makes sense to restrict the way arguments can be passed so that a +developer need only look at the function definition to determine if +items are passed by position, by position or keyword, or by keyword. + +A function definition may look like: + + def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2): + ----------- ---------- ---------- + | | | + | Positional or keyword | + | - Keyword only + -- Positional only + +where ‘/’ and ‘*’ are optional. If used, these symbols indicate the +kind of parameter by how the arguments may be passed to the function: +positional-only, positional-or-keyword, and keyword-only. Keyword +parameters are also referred to as named parameters. + +* Menu: + +* Positional-or-Keyword Arguments:: +* Positional-Only Parameters:: +* Keyword-Only Arguments:: +* Function Examples:: +* Recap:: + + +File: python.info, Node: Positional-or-Keyword Arguments, Next: Positional-Only Parameters, Up: Special parameters + +2.4.7.4 Positional-or-Keyword Arguments +....................................... + +If ‘/’ and ‘*’ are not present in the function definition, arguments may +be passed to a function by position or by keyword. + + +File: python.info, Node: Positional-Only Parameters, Next: Keyword-Only Arguments, Prev: Positional-or-Keyword Arguments, Up: Special parameters + +2.4.7.5 Positional-Only Parameters +.................................. + +Looking at this in a bit more detail, it is possible to mark certain +parameters as `positional-only'. If `positional-only', the parameters’ +order matters, and the parameters cannot be passed by keyword. +Positional-only parameters are placed before a ‘/’ (forward-slash). The +‘/’ is used to logically separate the positional-only parameters from +the rest of the parameters. If there is no ‘/’ in the function +definition, there are no positional-only parameters. + +Parameters following the ‘/’ may be `positional-or-keyword' or +`keyword-only'. + + +File: python.info, Node: Keyword-Only Arguments, Next: Function Examples, Prev: Positional-Only Parameters, Up: Special parameters + +2.4.7.6 Keyword-Only Arguments +.............................. + +To mark parameters as `keyword-only', indicating the parameters must be +passed by keyword argument, place an ‘*’ in the arguments list just +before the first `keyword-only' parameter. + + +File: python.info, Node: Function Examples, Next: Recap, Prev: Keyword-Only Arguments, Up: Special parameters + +2.4.7.7 Function Examples +......................... + +Consider the following example function definitions paying close +attention to the markers ‘/’ and ‘*’: + + >>> def standard_arg(arg): + ... print(arg) + ... + >>> def pos_only_arg(arg, /): + ... print(arg) + ... + >>> def kwd_only_arg(*, arg): + ... print(arg) + ... + >>> def combined_example(pos_only, /, standard, *, kwd_only): + ... print(pos_only, standard, kwd_only) + +The first function definition, ‘standard_arg’, the most familiar form, +places no restrictions on the calling convention and arguments may be +passed by position or keyword: + + >>> standard_arg(2) + 2 + + >>> standard_arg(arg=2) + 2 + +The second function ‘pos_only_arg’ is restricted to only use positional +parameters as there is a ‘/’ in the function definition: + + >>> pos_only_arg(1) + 1 + + >>> pos_only_arg(arg=1) + Traceback (most recent call last): + File "", line 1, in + TypeError: pos_only_arg() got an unexpected keyword argument 'arg' + +The third function ‘kwd_only_args’ only allows keyword arguments as +indicated by a ‘*’ in the function definition: + + >>> kwd_only_arg(3) + Traceback (most recent call last): + File "", line 1, in + TypeError: kwd_only_arg() takes 0 positional arguments but 1 was given + + >>> kwd_only_arg(arg=3) + 3 + +And the last uses all three calling conventions in the same function +definition: + + >>> combined_example(1, 2, 3) + Traceback (most recent call last): + File "", line 1, in + TypeError: combined_example() takes 2 positional arguments but 3 were given + + >>> combined_example(1, 2, kwd_only=3) + 1 2 3 + + >>> combined_example(1, standard=2, kwd_only=3) + 1 2 3 + + >>> combined_example(pos_only=1, standard=2, kwd_only=3) + Traceback (most recent call last): + File "", line 1, in + TypeError: combined_example() got an unexpected keyword argument 'pos_only' + +Finally, consider this function definition which has a potential +collision between the positional argument ‘name’ and ‘**kwds’ which has +‘name’ as a key: + + def foo(name, **kwds): + return 'name' in kwds + +There is no possible call that will make it return ‘True’ as the keyword +‘'name'’ will always bind to the first parameter. For example: + + >>> foo(1, **{'name': 2}) + Traceback (most recent call last): + File "", line 1, in + TypeError: foo() got multiple values for argument 'name' + >>> + +But using ‘/’ (positional only arguments), it is possible since it +allows ‘name’ as a positional argument and ‘'name'’ as a key in the +keyword arguments: + + def foo(name, /, **kwds): + return 'name' in kwds + >>> foo(1, **{'name': 2}) + True + +In other words, the names of positional-only parameters can be used in +‘**kwds’ without ambiguity. + + +File: python.info, Node: Recap, Prev: Function Examples, Up: Special parameters + +2.4.7.8 Recap +............. + +The use case will determine which parameters to use in the function +definition: + + def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2): + +As guidance: + + * Use positional-only if you want the name of the parameters to not + be available to the user. This is useful when parameter names have + no real meaning, if you want to enforce the order of the arguments + when the function is called or if you need to take some positional + parameters and arbitrary keywords. + + * Use keyword-only when names have meaning and the function + definition is more understandable by being explicit with names or + you want to prevent users relying on the position of the argument + being passed. + + * For an API, use positional-only to prevent breaking API changes if + the parameter’s name is modified in the future. + + +File: python.info, Node: Arbitrary Argument Lists, Next: Unpacking Argument Lists, Prev: Special parameters, Up: More on Defining Functions + +2.4.7.9 Arbitrary Argument Lists +................................ + +Finally, the least frequently used option is to specify that a function +can be called with an arbitrary number of arguments. These arguments +will be wrapped up in a tuple (see *note Tuples and Sequences: ee0.). +Before the variable number of arguments, zero or more normal arguments +may occur. + + def write_multiple_items(file, separator, *args): + file.write(separator.join(args)) + +Normally, these ‘variadic’ arguments will be last in the list of formal +parameters, because they scoop up all remaining input arguments that are +passed to the function. Any formal parameters which occur after the +‘*args’ parameter are ‘keyword-only’ arguments, meaning that they can +only be used as keywords rather than positional arguments. + + >>> def concat(*args, sep="/"): + ... return sep.join(args) + ... + >>> concat("earth", "mars", "venus") + 'earth/mars/venus' + >>> concat("earth", "mars", "venus", sep=".") + 'earth.mars.venus' + + +File: python.info, Node: Unpacking Argument Lists, Next: Lambda Expressions, Prev: Arbitrary Argument Lists, Up: More on Defining Functions + +2.4.7.10 Unpacking Argument Lists +................................. + +The reverse situation occurs when the arguments are already in a list or +tuple but need to be unpacked for a function call requiring separate +positional arguments. For instance, the built-in *note range(): 9be. +function expects separate `start' and `stop' arguments. If they are not +available separately, write the function call with the ‘*’-operator to +unpack the arguments out of a list or tuple: + + >>> list(range(3, 6)) # normal call with separate arguments + [3, 4, 5] + >>> args = [3, 6] + >>> list(range(*args)) # call with arguments unpacked from a list + [3, 4, 5] + +In the same fashion, dictionaries can deliver keyword arguments with the +‘**’-operator: + + >>> def parrot(voltage, state='a stiff', action='voom'): + ... print("-- This parrot wouldn't", action, end=' ') + ... print("if you put", voltage, "volts through it.", end=' ') + ... print("E's", state, "!") + ... + >>> d = {"voltage": "four million", "state": "bleedin' demised", "action": "VOOM"} + >>> parrot(**d) + -- This parrot wouldn't VOOM if you put four million volts through it. E's bleedin' demised ! + + +File: python.info, Node: Lambda Expressions, Next: Documentation Strings, Prev: Unpacking Argument Lists, Up: More on Defining Functions + +2.4.7.11 Lambda Expressions +........................... + +Small anonymous functions can be created with the *note lambda: c2f. +keyword. This function returns the sum of its two arguments: ‘lambda a, +b: a+b’. Lambda functions can be used wherever function objects are +required. They are syntactically restricted to a single expression. +Semantically, they are just syntactic sugar for a normal function +definition. Like nested function definitions, lambda functions can +reference variables from the containing scope: + + >>> def make_incrementor(n): + ... return lambda x: x + n + ... + >>> f = make_incrementor(42) + >>> f(0) + 42 + >>> f(1) + 43 + +The above example uses a lambda expression to return a function. +Another use is to pass a small function as an argument: + + >>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')] + >>> pairs.sort(key=lambda pair: pair[1]) + >>> pairs + [(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')] + + +File: python.info, Node: Documentation Strings, Next: Function Annotations, Prev: Lambda Expressions, Up: More on Defining Functions + +2.4.7.12 Documentation Strings +.............................. + +Here are some conventions about the content and formatting of +documentation strings. + +The first line should always be a short, concise summary of the object’s +purpose. For brevity, it should not explicitly state the object’s name +or type, since these are available by other means (except if the name +happens to be a verb describing a function’s operation). This line +should begin with a capital letter and end with a period. + +If there are more lines in the documentation string, the second line +should be blank, visually separating the summary from the rest of the +description. The following lines should be one or more paragraphs +describing the object’s calling conventions, its side effects, etc. + +The Python parser does not strip indentation from multi-line string +literals in Python, so tools that process documentation have to strip +indentation if desired. This is done using the following convention. +The first non-blank line `after' the first line of the string determines +the amount of indentation for the entire documentation string. (We +can’t use the first line since it is generally adjacent to the string’s +opening quotes so its indentation is not apparent in the string +literal.) Whitespace “equivalent” to this indentation is then stripped +from the start of all lines of the string. Lines that are indented less +should not occur, but if they occur all their leading whitespace should +be stripped. Equivalence of whitespace should be tested after expansion +of tabs (to 8 spaces, normally). + +Here is an example of a multi-line docstring: + + >>> def my_function(): + ... """Do nothing, but document it. + ... + ... No, really, it doesn't do anything. + ... """ + ... pass + ... + >>> print(my_function.__doc__) + Do nothing, but document it. + + No, really, it doesn't do anything. + + +File: python.info, Node: Function Annotations, Prev: Documentation Strings, Up: More on Defining Functions + +2.4.7.13 Function Annotations +............................. + +*note Function annotations: ef0. are completely optional metadata +information about the types used by user-defined functions (see PEP +3107(1) and PEP 484(2) for more information). + +*note Annotations: ef1. are stored in the ‘__annotations__’ attribute of +the function as a dictionary and have no effect on any other part of the +function. Parameter annotations are defined by a colon after the +parameter name, followed by an expression evaluating to the value of the +annotation. Return annotations are defined by a literal ‘->’, followed +by an expression, between the parameter list and the colon denoting the +end of the *note def: dbe. statement. The following example has a +required argument, an optional argument, and the return value annotated: + + >>> def f(ham: str, eggs: str = 'eggs') -> str: + ... print("Annotations:", f.__annotations__) + ... print("Arguments:", ham, eggs) + ... return ham + ' and ' + eggs + ... + >>> f('spam') + Annotations: {'ham': , 'return': , 'eggs': } + Arguments: spam eggs + 'spam and eggs' + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3107 + + (2) https://www.python.org/dev/peps/pep-0484 + + +File: python.info, Node: Intermezzo Coding Style, Prev: More on Defining Functions, Up: More Control Flow Tools + +2.4.8 Intermezzo: Coding Style +------------------------------ + +Now that you are about to write longer, more complex pieces of Python, +it is a good time to talk about `coding style'. Most languages can be +written (or more concise, `formatted') in different styles; some are +more readable than others. Making it easy for others to read your code +is always a good idea, and adopting a nice coding style helps +tremendously for that. + +For Python, PEP 8(1) has emerged as the style guide that most projects +adhere to; it promotes a very readable and eye-pleasing coding style. +Every Python developer should read it at some point; here are the most +important points extracted for you: + + * Use 4-space indentation, and no tabs. + + 4 spaces are a good compromise between small indentation (allows + greater nesting depth) and large indentation (easier to read). + Tabs introduce confusion, and are best left out. + + * Wrap lines so that they don’t exceed 79 characters. + + This helps users with small displays and makes it possible to have + several code files side-by-side on larger displays. + + * Use blank lines to separate functions and classes, and larger + blocks of code inside functions. + + * When possible, put comments on a line of their own. + + * Use docstrings. + + * Use spaces around operators and after commas, but not directly + inside bracketing constructs: ‘a = f(1, 2) + g(3, 4)’. + + * Name your classes and functions consistently; the convention is to + use ‘UpperCamelCase’ for classes and ‘lowercase_with_underscores’ + for functions and methods. Always use ‘self’ as the name for the + first method argument (see *note A First Look at Classes: ef4. for + more on classes and methods). + + * Don’t use fancy encodings if your code is meant to be used in + international environments. Python’s default, UTF-8, or even plain + ASCII work best in any case. + + * Likewise, don’t use non-ASCII characters in identifiers if there is + only the slightest chance people speaking a different language will + read or maintain the code. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0008 + + +File: python.info, Node: Data Structures, Next: Modules, Prev: More Control Flow Tools, Up: The Python Tutorial + +2.5 Data Structures +=================== + +This chapter describes some things you’ve learned about already in more +detail, and adds some new things as well. + +* Menu: + +* More on Lists:: +* The del statement:: +* Tuples and Sequences:: +* Sets:: +* Dictionaries:: +* Looping Techniques:: +* More on Conditions:: +* Comparing Sequences and Other Types:: + + +File: python.info, Node: More on Lists, Next: The del statement, Up: Data Structures + +2.5.1 More on Lists +------------------- + +The list data type has some more methods. Here are all of the methods +of list objects: + + -- Method: list.append (x) + + Add an item to the end of the list. Equivalent to ‘a[len(a):] = + [x]’. + + -- Method: list.extend (iterable) + + Extend the list by appending all the items from the iterable. + Equivalent to ‘a[len(a):] = iterable’. + + -- Method: list.insert (i, x) + + Insert an item at a given position. The first argument is the + index of the element before which to insert, so ‘a.insert(0, x)’ + inserts at the front of the list, and ‘a.insert(len(a), x)’ is + equivalent to ‘a.append(x)’. + + -- Method: list.remove (x) + + Remove the first item from the list whose value is equal to `x'. + It raises a *note ValueError: 1fb. if there is no such item. + + -- Method: list.pop ([i]) + + Remove the item at the given position in the list, and return it. + If no index is specified, ‘a.pop()’ removes and returns the last + item in the list. (The square brackets around the `i' in the + method signature denote that the parameter is optional, not that + you should type square brackets at that position. You will see + this notation frequently in the Python Library Reference.) + + -- Method: list.clear () + + Remove all items from the list. Equivalent to ‘del a[:]’. + + -- Method: list.index (x[, start[, end]]) + + Return zero-based index in the list of the first item whose value + is equal to `x'. Raises a *note ValueError: 1fb. if there is no + such item. + + The optional arguments `start' and `end' are interpreted as in the + slice notation and are used to limit the search to a particular + subsequence of the list. The returned index is computed relative + to the beginning of the full sequence rather than the `start' + argument. + + -- Method: list.count (x) + + Return the number of times `x' appears in the list. + + -- Method: list.sort (*, key=None, reverse=False) + + Sort the items of the list in place (the arguments can be used for + sort customization, see *note sorted(): 444. for their + explanation). + + -- Method: list.reverse () + + Reverse the elements of the list in place. + + -- Method: list.copy () + + Return a shallow copy of the list. Equivalent to ‘a[:]’. + +An example that uses most of the list methods: + + >>> fruits = ['orange', 'apple', 'pear', 'banana', 'kiwi', 'apple', 'banana'] + >>> fruits.count('apple') + 2 + >>> fruits.count('tangerine') + 0 + >>> fruits.index('banana') + 3 + >>> fruits.index('banana', 4) # Find next banana starting a position 4 + 6 + >>> fruits.reverse() + >>> fruits + ['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange'] + >>> fruits.append('grape') + >>> fruits + ['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange', 'grape'] + >>> fruits.sort() + >>> fruits + ['apple', 'apple', 'banana', 'banana', 'grape', 'kiwi', 'orange', 'pear'] + >>> fruits.pop() + 'pear' + +You might have noticed that methods like ‘insert’, ‘remove’ or ‘sort’ +that only modify the list have no return value printed – they return the +default ‘None’. (1) This is a design principle for all mutable data +structures in Python. + +Another thing you might notice is that not all data can be sorted or +compared. For instance, ‘[None, 'hello', 10]’ doesn’t sort because +integers can’t be compared to strings and `None' can’t be compared to +other types. Also, there are some types that don’t have a defined +ordering relation. For example, ‘3+4j < 5+7j’ isn’t a valid comparison. + +* Menu: + +* Using Lists as Stacks:: +* Using Lists as Queues:: +* List Comprehensions: List Comprehensions<2>. +* Nested List Comprehensions:: + + ---------- Footnotes ---------- + + (1) (1) Other languages may return the mutated object, which allows +method chaining, such as ‘d->insert("a")->remove("b")->sort();’. + + +File: python.info, Node: Using Lists as Stacks, Next: Using Lists as Queues, Up: More on Lists + +2.5.1.1 Using Lists as Stacks +............................. + +The list methods make it very easy to use a list as a stack, where the +last element added is the first element retrieved (“last-in, +first-out”). To add an item to the top of the stack, use ‘append()’. +To retrieve an item from the top of the stack, use ‘pop()’ without an +explicit index. For example: + + >>> stack = [3, 4, 5] + >>> stack.append(6) + >>> stack.append(7) + >>> stack + [3, 4, 5, 6, 7] + >>> stack.pop() + 7 + >>> stack + [3, 4, 5, 6] + >>> stack.pop() + 6 + >>> stack.pop() + 5 + >>> stack + [3, 4] + + +File: python.info, Node: Using Lists as Queues, Next: List Comprehensions<2>, Prev: Using Lists as Stacks, Up: More on Lists + +2.5.1.2 Using Lists as Queues +............................. + +It is also possible to use a list as a queue, where the first element +added is the first element retrieved (“first-in, first-out”); however, +lists are not efficient for this purpose. While appends and pops from +the end of list are fast, doing inserts or pops from the beginning of a +list is slow (because all of the other elements have to be shifted by +one). + +To implement a queue, use *note collections.deque: 531. which was +designed to have fast appends and pops from both ends. For example: + + >>> from collections import deque + >>> queue = deque(["Eric", "John", "Michael"]) + >>> queue.append("Terry") # Terry arrives + >>> queue.append("Graham") # Graham arrives + >>> queue.popleft() # The first to arrive now leaves + 'Eric' + >>> queue.popleft() # The second to arrive now leaves + 'John' + >>> queue # Remaining queue in order of arrival + deque(['Michael', 'Terry', 'Graham']) + + +File: python.info, Node: List Comprehensions<2>, Next: Nested List Comprehensions, Prev: Using Lists as Queues, Up: More on Lists + +2.5.1.3 List Comprehensions +........................... + +List comprehensions provide a concise way to create lists. Common +applications are to make new lists where each element is the result of +some operations applied to each member of another sequence or iterable, +or to create a subsequence of those elements that satisfy a certain +condition. + +For example, assume we want to create a list of squares, like: + + >>> squares = [] + >>> for x in range(10): + ... squares.append(x**2) + ... + >>> squares + [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] + +Note that this creates (or overwrites) a variable named ‘x’ that still +exists after the loop completes. We can calculate the list of squares +without any side effects using: + + squares = list(map(lambda x: x**2, range(10))) + +or, equivalently: + + squares = [x**2 for x in range(10)] + +which is more concise and readable. + +A list comprehension consists of brackets containing an expression +followed by a ‘for’ clause, then zero or more ‘for’ or ‘if’ clauses. +The result will be a new list resulting from evaluating the expression +in the context of the ‘for’ and ‘if’ clauses which follow it. For +example, this listcomp combines the elements of two lists if they are +not equal: + + >>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] + [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)] + +and it’s equivalent to: + + >>> combs = [] + >>> for x in [1,2,3]: + ... for y in [3,1,4]: + ... if x != y: + ... combs.append((x, y)) + ... + >>> combs + [(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)] + +Note how the order of the *note for: c30. and *note if: de7. statements +is the same in both these snippets. + +If the expression is a tuple (e.g. the ‘(x, y)’ in the previous +example), it must be parenthesized. + + >>> vec = [-4, -2, 0, 2, 4] + >>> # create a new list with the values doubled + >>> [x*2 for x in vec] + [-8, -4, 0, 4, 8] + >>> # filter the list to exclude negative numbers + >>> [x for x in vec if x >= 0] + [0, 2, 4] + >>> # apply a function to all the elements + >>> [abs(x) for x in vec] + [4, 2, 0, 2, 4] + >>> # call a method on each element + >>> freshfruit = [' banana', ' loganberry ', 'passion fruit '] + >>> [weapon.strip() for weapon in freshfruit] + ['banana', 'loganberry', 'passion fruit'] + >>> # create a list of 2-tuples like (number, square) + >>> [(x, x**2) for x in range(6)] + [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)] + >>> # the tuple must be parenthesized, otherwise an error is raised + >>> [x, x**2 for x in range(6)] + File "", line 1, in + [x, x**2 for x in range(6)] + ^ + SyntaxError: invalid syntax + >>> # flatten a list using a listcomp with two 'for' + >>> vec = [[1,2,3], [4,5,6], [7,8,9]] + >>> [num for elem in vec for num in elem] + [1, 2, 3, 4, 5, 6, 7, 8, 9] + +List comprehensions can contain complex expressions and nested +functions: + + >>> from math import pi + >>> [str(round(pi, i)) for i in range(1, 6)] + ['3.1', '3.14', '3.142', '3.1416', '3.14159'] + + +File: python.info, Node: Nested List Comprehensions, Prev: List Comprehensions<2>, Up: More on Lists + +2.5.1.4 Nested List Comprehensions +.................................. + +The initial expression in a list comprehension can be any arbitrary +expression, including another list comprehension. + +Consider the following example of a 3x4 matrix implemented as a list of +3 lists of length 4: + + >>> matrix = [ + ... [1, 2, 3, 4], + ... [5, 6, 7, 8], + ... [9, 10, 11, 12], + ... ] + +The following list comprehension will transpose rows and columns: + + >>> [[row[i] for row in matrix] for i in range(4)] + [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]] + +As we saw in the previous section, the nested listcomp is evaluated in +the context of the *note for: c30. that follows it, so this example is +equivalent to: + + >>> transposed = [] + >>> for i in range(4): + ... transposed.append([row[i] for row in matrix]) + ... + >>> transposed + [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]] + +which, in turn, is the same as: + + >>> transposed = [] + >>> for i in range(4): + ... # the following 3 lines implement the nested listcomp + ... transposed_row = [] + ... for row in matrix: + ... transposed_row.append(row[i]) + ... transposed.append(transposed_row) + ... + >>> transposed + [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]] + +In the real world, you should prefer built-in functions to complex flow +statements. The *note zip(): c32. function would do a great job for +this use case: + + >>> list(zip(*matrix)) + [(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)] + +See *note Unpacking Argument Lists: ee9. for details on the asterisk in +this line. + + +File: python.info, Node: The del statement, Next: Tuples and Sequences, Prev: More on Lists, Up: Data Structures + +2.5.2 The ‘del’ statement +------------------------- + +There is a way to remove an item from a list given its index instead of +its value: the *note del: f02. statement. This differs from the ‘pop()’ +method which returns a value. The ‘del’ statement can also be used to +remove slices from a list or clear the entire list (which we did earlier +by assignment of an empty list to the slice). For example: + + >>> a = [-1, 1, 66.25, 333, 333, 1234.5] + >>> del a[0] + >>> a + [1, 66.25, 333, 333, 1234.5] + >>> del a[2:4] + >>> a + [1, 66.25, 1234.5] + >>> del a[:] + >>> a + [] + +*note del: f02. can also be used to delete entire variables: + + >>> del a + +Referencing the name ‘a’ hereafter is an error (at least until another +value is assigned to it). We’ll find other uses for *note del: f02. +later. + + +File: python.info, Node: Tuples and Sequences, Next: Sets, Prev: The del statement, Up: Data Structures + +2.5.3 Tuples and Sequences +-------------------------- + +We saw that lists and strings have many common properties, such as +indexing and slicing operations. They are two examples of `sequence' +data types (see *note Sequence Types — list, tuple, range: f04.). Since +Python is an evolving language, other sequence data types may be added. +There is also another standard sequence data type: the `tuple'. + +A tuple consists of a number of values separated by commas, for +instance: + + >>> t = 12345, 54321, 'hello!' + >>> t[0] + 12345 + >>> t + (12345, 54321, 'hello!') + >>> # Tuples may be nested: + ... u = t, (1, 2, 3, 4, 5) + >>> u + ((12345, 54321, 'hello!'), (1, 2, 3, 4, 5)) + >>> # Tuples are immutable: + ... t[0] = 88888 + Traceback (most recent call last): + File "", line 1, in + TypeError: 'tuple' object does not support item assignment + >>> # but they can contain mutable objects: + ... v = ([1, 2, 3], [3, 2, 1]) + >>> v + ([1, 2, 3], [3, 2, 1]) + +As you see, on output tuples are always enclosed in parentheses, so that +nested tuples are interpreted correctly; they may be input with or +without surrounding parentheses, although often parentheses are +necessary anyway (if the tuple is part of a larger expression). It is +not possible to assign to the individual items of a tuple, however it is +possible to create tuples which contain mutable objects, such as lists. + +Though tuples may seem similar to lists, they are often used in +different situations and for different purposes. Tuples are *note +immutable: eb6, and usually contain a heterogeneous sequence of elements +that are accessed via unpacking (see later in this section) or indexing +(or even by attribute in the case of *note namedtuples: 1b7.). Lists +are *note mutable: ebe, and their elements are usually homogeneous and +are accessed by iterating over the list. + +A special problem is the construction of tuples containing 0 or 1 items: +the syntax has some extra quirks to accommodate these. Empty tuples are +constructed by an empty pair of parentheses; a tuple with one item is +constructed by following a value with a comma (it is not sufficient to +enclose a single value in parentheses). Ugly, but effective. For +example: + + >>> empty = () + >>> singleton = 'hello', # <-- note trailing comma + >>> len(empty) + 0 + >>> len(singleton) + 1 + >>> singleton + ('hello',) + +The statement ‘t = 12345, 54321, 'hello!'’ is an example of `tuple +packing': the values ‘12345’, ‘54321’ and ‘'hello!'’ are packed together +in a tuple. The reverse operation is also possible: + + >>> x, y, z = t + +This is called, appropriately enough, `sequence unpacking' and works for +any sequence on the right-hand side. Sequence unpacking requires that +there are as many variables on the left side of the equals sign as there +are elements in the sequence. Note that multiple assignment is really +just a combination of tuple packing and sequence unpacking. + + +File: python.info, Node: Sets, Next: Dictionaries, Prev: Tuples and Sequences, Up: Data Structures + +2.5.4 Sets +---------- + +Python also includes a data type for `sets'. A set is an unordered +collection with no duplicate elements. Basic uses include membership +testing and eliminating duplicate entries. Set objects also support +mathematical operations like union, intersection, difference, and +symmetric difference. + +Curly braces or the *note set(): b6f. function can be used to create +sets. Note: to create an empty set you have to use ‘set()’, not ‘{}’; +the latter creates an empty dictionary, a data structure that we discuss +in the next section. + +Here is a brief demonstration: + + >>> basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'} + >>> print(basket) # show that duplicates have been removed + {'orange', 'banana', 'pear', 'apple'} + >>> 'orange' in basket # fast membership testing + True + >>> 'crabgrass' in basket + False + + >>> # Demonstrate set operations on unique letters from two words + ... + >>> a = set('abracadabra') + >>> b = set('alacazam') + >>> a # unique letters in a + {'a', 'r', 'b', 'c', 'd'} + >>> a - b # letters in a but not in b + {'r', 'd', 'b'} + >>> a | b # letters in a or b or both + {'a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'} + >>> a & b # letters in both a and b + {'a', 'c'} + >>> a ^ b # letters in a or b but not both + {'r', 'd', 'b', 'm', 'z', 'l'} + +Similarly to *note list comprehensions: efe, set comprehensions are also +supported: + + >>> a = {x for x in 'abracadabra' if x not in 'abc'} + >>> a + {'r', 'd'} + + +File: python.info, Node: Dictionaries, Next: Looping Techniques, Prev: Sets, Up: Data Structures + +2.5.5 Dictionaries +------------------ + +Another useful data type built into Python is the `dictionary' (see +*note Mapping Types — dict: 2fc.). Dictionaries are sometimes found in +other languages as “associative memories” or “associative arrays”. +Unlike sequences, which are indexed by a range of numbers, dictionaries +are indexed by `keys', which can be any immutable type; strings and +numbers can always be keys. Tuples can be used as keys if they contain +only strings, numbers, or tuples; if a tuple contains any mutable object +either directly or indirectly, it cannot be used as a key. You can’t +use lists as keys, since lists can be modified in place using index +assignments, slice assignments, or methods like ‘append()’ and +‘extend()’. + +It is best to think of a dictionary as a set of `key: value' pairs, with +the requirement that the keys are unique (within one dictionary). A +pair of braces creates an empty dictionary: ‘{}’. Placing a +comma-separated list of key:value pairs within the braces adds initial +key:value pairs to the dictionary; this is also the way dictionaries are +written on output. + +The main operations on a dictionary are storing a value with some key +and extracting the value given the key. It is also possible to delete a +key:value pair with ‘del’. If you store using a key that is already in +use, the old value associated with that key is forgotten. It is an +error to extract a value using a non-existent key. + +Performing ‘list(d)’ on a dictionary returns a list of all the keys used +in the dictionary, in insertion order (if you want it sorted, just use +‘sorted(d)’ instead). To check whether a single key is in the +dictionary, use the *note in: 7a3. keyword. + +Here is a small example using a dictionary: + + >>> tel = {'jack': 4098, 'sape': 4139} + >>> tel['guido'] = 4127 + >>> tel + {'jack': 4098, 'sape': 4139, 'guido': 4127} + >>> tel['jack'] + 4098 + >>> del tel['sape'] + >>> tel['irv'] = 4127 + >>> tel + {'jack': 4098, 'guido': 4127, 'irv': 4127} + >>> list(tel) + ['jack', 'guido', 'irv'] + >>> sorted(tel) + ['guido', 'irv', 'jack'] + >>> 'guido' in tel + True + >>> 'jack' not in tel + False + +The *note dict(): 1b8. constructor builds dictionaries directly from +sequences of key-value pairs: + + >>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)]) + {'sape': 4139, 'guido': 4127, 'jack': 4098} + +In addition, dict comprehensions can be used to create dictionaries from +arbitrary key and value expressions: + + >>> {x: x**2 for x in (2, 4, 6)} + {2: 4, 4: 16, 6: 36} + +When the keys are simple strings, it is sometimes easier to specify +pairs using keyword arguments: + + >>> dict(sape=4139, guido=4127, jack=4098) + {'sape': 4139, 'guido': 4127, 'jack': 4098} + + +File: python.info, Node: Looping Techniques, Next: More on Conditions, Prev: Dictionaries, Up: Data Structures + +2.5.6 Looping Techniques +------------------------ + +When looping through dictionaries, the key and corresponding value can +be retrieved at the same time using the ‘items()’ method. + + >>> knights = {'gallahad': 'the pure', 'robin': 'the brave'} + >>> for k, v in knights.items(): + ... print(k, v) + ... + gallahad the pure + robin the brave + +When looping through a sequence, the position index and corresponding +value can be retrieved at the same time using the *note enumerate(): +de3. function. + + >>> for i, v in enumerate(['tic', 'tac', 'toe']): + ... print(i, v) + ... + 0 tic + 1 tac + 2 toe + +To loop over two or more sequences at the same time, the entries can be +paired with the *note zip(): c32. function. + + >>> questions = ['name', 'quest', 'favorite color'] + >>> answers = ['lancelot', 'the holy grail', 'blue'] + >>> for q, a in zip(questions, answers): + ... print('What is your {0}? It is {1}.'.format(q, a)) + ... + What is your name? It is lancelot. + What is your quest? It is the holy grail. + What is your favorite color? It is blue. + +To loop over a sequence in reverse, first specify the sequence in a +forward direction and then call the *note reversed(): 18e. function. + + >>> for i in reversed(range(1, 10, 2)): + ... print(i) + ... + 9 + 7 + 5 + 3 + 1 + +To loop over a sequence in sorted order, use the *note sorted(): 444. +function which returns a new sorted list while leaving the source +unaltered. + + >>> basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana'] + >>> for f in sorted(set(basket)): + ... print(f) + ... + apple + banana + orange + pear + +It is sometimes tempting to change a list while you are looping over it; +however, it is often simpler and safer to create a new list instead. + + >>> import math + >>> raw_data = [56.2, float('NaN'), 51.7, 55.3, 52.5, float('NaN'), 47.8] + >>> filtered_data = [] + >>> for value in raw_data: + ... if not math.isnan(value): + ... filtered_data.append(value) + ... + >>> filtered_data + [56.2, 51.7, 55.3, 52.5, 47.8] + + +File: python.info, Node: More on Conditions, Next: Comparing Sequences and Other Types, Prev: Looping Techniques, Up: Data Structures + +2.5.7 More on Conditions +------------------------ + +The conditions used in ‘while’ and ‘if’ statements can contain any +operators, not just comparisons. + +The comparison operators ‘in’ and ‘not in’ check whether a value occurs +(does not occur) in a sequence. The operators ‘is’ and ‘is not’ compare +whether two objects are really the same object; this only matters for +mutable objects like lists. All comparison operators have the same +priority, which is lower than that of all numerical operators. + +Comparisons can be chained. For example, ‘a < b == c’ tests whether ‘a’ +is less than ‘b’ and moreover ‘b’ equals ‘c’. + +Comparisons may be combined using the Boolean operators ‘and’ and ‘or’, +and the outcome of a comparison (or of any other Boolean expression) may +be negated with ‘not’. These have lower priorities than comparison +operators; between them, ‘not’ has the highest priority and ‘or’ the +lowest, so that ‘A and not B or C’ is equivalent to ‘(A and (not B)) or +C’. As always, parentheses can be used to express the desired +composition. + +The Boolean operators ‘and’ and ‘or’ are so-called `short-circuit' +operators: their arguments are evaluated from left to right, and +evaluation stops as soon as the outcome is determined. For example, if +‘A’ and ‘C’ are true but ‘B’ is false, ‘A and B and C’ does not evaluate +the expression ‘C’. When used as a general value and not as a Boolean, +the return value of a short-circuit operator is the last evaluated +argument. + +It is possible to assign the result of a comparison or other Boolean +expression to a variable. For example, + + >>> string1, string2, string3 = '', 'Trondheim', 'Hammer Dance' + >>> non_null = string1 or string2 or string3 + >>> non_null + 'Trondheim' + +Note that in Python, unlike C, assignment inside expressions must be +done explicitly with the *note walrus operator: f0c. ‘:=’. This avoids +a common class of problems encountered in C programs: typing ‘=’ in an +expression when ‘==’ was intended. + + +File: python.info, Node: Comparing Sequences and Other Types, Prev: More on Conditions, Up: Data Structures + +2.5.8 Comparing Sequences and Other Types +----------------------------------------- + +Sequence objects typically may be compared to other objects with the +same sequence type. The comparison uses `lexicographical' ordering: +first the first two items are compared, and if they differ this +determines the outcome of the comparison; if they are equal, the next +two items are compared, and so on, until either sequence is exhausted. +If two items to be compared are themselves sequences of the same type, +the lexicographical comparison is carried out recursively. If all items +of two sequences compare equal, the sequences are considered equal. If +one sequence is an initial sub-sequence of the other, the shorter +sequence is the smaller (lesser) one. Lexicographical ordering for +strings uses the Unicode code point number to order individual +characters. Some examples of comparisons between sequences of the same +type: + + (1, 2, 3) < (1, 2, 4) + [1, 2, 3] < [1, 2, 4] + 'ABC' < 'C' < 'Pascal' < 'Python' + (1, 2, 3, 4) < (1, 2, 4) + (1, 2) < (1, 2, -1) + (1, 2, 3) == (1.0, 2.0, 3.0) + (1, 2, ('aa', 'ab')) < (1, 2, ('abc', 'a'), 4) + +Note that comparing objects of different types with ‘<’ or ‘>’ is legal +provided that the objects have appropriate comparison methods. For +example, mixed numeric types are compared according to their numeric +value, so 0 equals 0.0, etc. Otherwise, rather than providing an +arbitrary ordering, the interpreter will raise a *note TypeError: 192. +exception. + + +File: python.info, Node: Modules, Next: Input and Output, Prev: Data Structures, Up: The Python Tutorial + +2.6 Modules +=========== + +If you quit from the Python interpreter and enter it again, the +definitions you have made (functions and variables) are lost. +Therefore, if you want to write a somewhat longer program, you are +better off using a text editor to prepare the input for the interpreter +and running it with that file as input instead. This is known as +creating a `script'. As your program gets longer, you may want to split +it into several files for easier maintenance. You may also want to use +a handy function that you’ve written in several programs without copying +its definition into each program. + +To support this, Python has a way to put definitions in a file and use +them in a script or in an interactive instance of the interpreter. Such +a file is called a `module'; definitions from a module can be `imported' +into other modules or into the `main' module (the collection of +variables that you have access to in a script executed at the top level +and in calculator mode). + +A module is a file containing Python definitions and statements. The +file name is the module name with the suffix ‘.py’ appended. Within a +module, the module’s name (as a string) is available as the value of the +global variable ‘__name__’. For instance, use your favorite text editor +to create a file called ‘fibo.py’ in the current directory with the +following contents: + + # Fibonacci numbers module + + def fib(n): # write Fibonacci series up to n + a, b = 0, 1 + while a < n: + print(a, end=' ') + a, b = b, a+b + print() + + def fib2(n): # return Fibonacci series up to n + result = [] + a, b = 0, 1 + while a < n: + result.append(a) + a, b = b, a+b + return result + +Now enter the Python interpreter and import this module with the +following command: + + >>> import fibo + +This does not enter the names of the functions defined in ‘fibo’ +directly in the current symbol table; it only enters the module name +‘fibo’ there. Using the module name you can access the functions: + + >>> fibo.fib(1000) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 + >>> fibo.fib2(100) + [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89] + >>> fibo.__name__ + 'fibo' + +If you intend to use a function often you can assign it to a local name: + + >>> fib = fibo.fib + >>> fib(500) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 + +* Menu: + +* More on Modules:: +* Standard Modules:: +* The dir() Function: The dir Function. +* Packages:: + + +File: python.info, Node: More on Modules, Next: Standard Modules, Up: Modules + +2.6.1 More on Modules +--------------------- + +A module can contain executable statements as well as function +definitions. These statements are intended to initialize the module. +They are executed only the `first' time the module name is encountered +in an import statement. (1) (They are also run if the file is executed +as a script.) + +Each module has its own private symbol table, which is used as the +global symbol table by all functions defined in the module. Thus, the +author of a module can use global variables in the module without +worrying about accidental clashes with a user’s global variables. On +the other hand, if you know what you are doing you can touch a module’s +global variables with the same notation used to refer to its functions, +‘modname.itemname’. + +Modules can import other modules. It is customary but not required to +place all *note import: c1d. statements at the beginning of a module (or +script, for that matter). The imported module names are placed in the +importing module’s global symbol table. + +There is a variant of the *note import: c1d. statement that imports +names from a module directly into the importing module’s symbol table. +For example: + + >>> from fibo import fib, fib2 + >>> fib(500) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 + +This does not introduce the module name from which the imports are taken +in the local symbol table (so in the example, ‘fibo’ is not defined). + +There is even a variant to import all names that a module defines: + + >>> from fibo import * + >>> fib(500) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 + +This imports all names except those beginning with an underscore (‘_’). +In most cases Python programmers do not use this facility since it +introduces an unknown set of names into the interpreter, possibly hiding +some things you have already defined. + +Note that in general the practice of importing ‘*’ from a module or +package is frowned upon, since it often causes poorly readable code. +However, it is okay to use it to save typing in interactive sessions. + +If the module name is followed by ‘as’, then the name following ‘as’ is +bound directly to the imported module. + + >>> import fibo as fib + >>> fib.fib(500) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 + +This is effectively importing the module in the same way that ‘import +fibo’ will do, with the only difference of it being available as ‘fib’. + +It can also be used when utilising *note from: c45. with similar +effects: + + >>> from fibo import fib as fibonacci + >>> fibonacci(500) + 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 + + Note: For efficiency reasons, each module is only imported once per + interpreter session. Therefore, if you change your modules, you + must restart the interpreter – or, if it’s just one module you want + to test interactively, use *note importlib.reload(): 399, e.g. + ‘import importlib; importlib.reload(modulename)’. + +* Menu: + +* Executing modules as scripts:: +* The Module Search Path:: +* “Compiled” Python files:: + + ---------- Footnotes ---------- + + (1) (1) In fact function definitions are also ‘statements’ that are +‘executed’; the execution of a module-level function definition enters +the function name in the module’s global symbol table. + + +File: python.info, Node: Executing modules as scripts, Next: The Module Search Path, Up: More on Modules + +2.6.1.1 Executing modules as scripts +.................................... + +When you run a Python module with + + python fibo.py + +the code in the module will be executed, just as if you imported it, but +with the ‘__name__’ set to ‘"__main__"’. That means that by adding this +code at the end of your module: + + if __name__ == "__main__": + import sys + fib(int(sys.argv[1])) + +you can make the file usable as a script as well as an importable +module, because the code that parses the command line only runs if the +module is executed as the “main” file: + + $ python fibo.py 50 + 0 1 1 2 3 5 8 13 21 34 + +If the module is imported, the code is not run: + + >>> import fibo + >>> + +This is often used either to provide a convenient user interface to a +module, or for testing purposes (running the module as a script executes +a test suite). + + +File: python.info, Node: The Module Search Path, Next: “Compiled” Python files, Prev: Executing modules as scripts, Up: More on Modules + +2.6.1.2 The Module Search Path +.............................. + +When a module named ‘spam’ is imported, the interpreter first searches +for a built-in module with that name. If not found, it then searches +for a file named ‘spam.py’ in a list of directories given by the +variable *note sys.path: 488. *note sys.path: 488. is initialized from +these locations: + + * The directory containing the input script (or the current directory + when no file is specified). + + * *note PYTHONPATH: 953. (a list of directory names, with the same + syntax as the shell variable ‘PATH’). + + * The installation-dependent default. + + Note: On file systems which support symlinks, the directory + containing the input script is calculated after the symlink is + followed. In other words the directory containing the symlink is + `not' added to the module search path. + +After initialization, Python programs can modify *note sys.path: 488. +The directory containing the script being run is placed at the beginning +of the search path, ahead of the standard library path. This means that +scripts in that directory will be loaded instead of modules of the same +name in the library directory. This is an error unless the replacement +is intended. See section *note Standard Modules: f18. for more +information. + + +File: python.info, Node: “Compiled” Python files, Prev: The Module Search Path, Up: More on Modules + +2.6.1.3 “Compiled” Python files +............................... + +To speed up loading modules, Python caches the compiled version of each +module in the ‘__pycache__’ directory under the name +‘module.`version'.pyc’, where the version encodes the format of the +compiled file; it generally contains the Python version number. For +example, in CPython release 3.3 the compiled version of spam.py would be +cached as ‘__pycache__/spam.cpython-33.pyc’. This naming convention +allows compiled modules from different releases and different versions +of Python to coexist. + +Python checks the modification date of the source against the compiled +version to see if it’s out of date and needs to be recompiled. This is +a completely automatic process. Also, the compiled modules are +platform-independent, so the same library can be shared among systems +with different architectures. + +Python does not check the cache in two circumstances. First, it always +recompiles and does not store the result for the module that’s loaded +directly from the command line. Second, it does not check the cache if +there is no source module. To support a non-source (compiled only) +distribution, the compiled module must be in the source directory, and +there must not be a source module. + +Some tips for experts: + + * You can use the *note -O: 68b. or *note -OO: 68c. switches on the + Python command to reduce the size of a compiled module. The ‘-O’ + switch removes assert statements, the ‘-OO’ switch removes both + assert statements and __doc__ strings. Since some programs may + rely on having these available, you should only use this option if + you know what you’re doing. “Optimized” modules have an ‘opt-’ tag + and are usually smaller. Future releases may change the effects of + optimization. + + * A program doesn’t run any faster when it is read from a ‘.pyc’ file + than when it is read from a ‘.py’ file; the only thing that’s + faster about ‘.pyc’ files is the speed with which they are loaded. + + * The module *note compileall: 21. can create .pyc files for all + modules in a directory. + + * There is more detail on this process, including a flow chart of the + decisions, in PEP 3147(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: Standard Modules, Next: The dir Function, Prev: More on Modules, Up: Modules + +2.6.2 Standard Modules +---------------------- + +Python comes with a library of standard modules, described in a separate +document, the Python Library Reference (“Library Reference” hereafter). +Some modules are built into the interpreter; these provide access to +operations that are not part of the core of the language but are +nevertheless built in, either for efficiency or to provide access to +operating system primitives such as system calls. The set of such +modules is a configuration option which also depends on the underlying +platform. For example, the *note winreg: 12a. module is only provided +on Windows systems. One particular module deserves some attention: +*note sys: fd, which is built into every Python interpreter. The +variables ‘sys.ps1’ and ‘sys.ps2’ define the strings used as primary and +secondary prompts: + + >>> import sys + >>> sys.ps1 + '>>> ' + >>> sys.ps2 + '... ' + >>> sys.ps1 = 'C> ' + C> print('Yuck!') + Yuck! + C> + +These two variables are only defined if the interpreter is in +interactive mode. + +The variable ‘sys.path’ is a list of strings that determines the +interpreter’s search path for modules. It is initialized to a default +path taken from the environment variable *note PYTHONPATH: 953, or from +a built-in default if *note PYTHONPATH: 953. is not set. You can modify +it using standard list operations: + + >>> import sys + >>> sys.path.append('/ufs/guido/lib/python') + + +File: python.info, Node: The dir Function, Next: Packages, Prev: Standard Modules, Up: Modules + +2.6.3 The ‘dir()’ Function +-------------------------- + +The built-in function *note dir(): d33. is used to find out which names +a module defines. It returns a sorted list of strings: + + >>> import fibo, sys + >>> dir(fibo) + ['__name__', 'fib', 'fib2'] + >>> dir(sys) + ['__displayhook__', '__doc__', '__excepthook__', '__loader__', '__name__', + '__package__', '__stderr__', '__stdin__', '__stdout__', + '_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe', + '_home', '_mercurial', '_xoptions', 'abiflags', 'api_version', 'argv', + 'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder', + 'call_tracing', 'callstats', 'copyright', 'displayhook', + 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix', + 'executable', 'exit', 'flags', 'float_info', 'float_repr_style', + 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', + 'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit', + 'getrefcount', 'getsizeof', 'getswitchinterval', 'gettotalrefcount', + 'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info', + 'intern', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', + 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1', + 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit', + 'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout', + 'thread_info', 'version', 'version_info', 'warnoptions'] + +Without arguments, *note dir(): d33. lists the names you have defined +currently: + + >>> a = [1, 2, 3, 4, 5] + >>> import fibo + >>> fib = fibo.fib + >>> dir() + ['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys'] + +Note that it lists all types of names: variables, modules, functions, +etc. + +*note dir(): d33. does not list the names of built-in functions and +variables. If you want a list of those, they are defined in the +standard module *note builtins: 13.: + + >>> import builtins + >>> dir(builtins) + ['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', + 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', + 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', + 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', + 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', + 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', + 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', + 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', + 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', + 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', + 'NotImplementedError', 'OSError', 'OverflowError', + 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', + 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', + 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', + 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', + 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', + 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', + 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__', + '__debug__', '__doc__', '__import__', '__name__', '__package__', 'abs', + 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', + 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', + 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', + 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', + 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', + 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', + 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', + 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', + 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', + 'zip'] + + +File: python.info, Node: Packages, Prev: The dir Function, Up: Modules + +2.6.4 Packages +-------------- + +Packages are a way of structuring Python’s module namespace by using +“dotted module names”. For example, the module name ‘A.B’ designates a +submodule named ‘B’ in a package named ‘A’. Just like the use of +modules saves the authors of different modules from having to worry +about each other’s global variable names, the use of dotted module names +saves the authors of multi-module packages like NumPy or Pillow from +having to worry about each other’s module names. + +Suppose you want to design a collection of modules (a “package”) for the +uniform handling of sound files and sound data. There are many +different sound file formats (usually recognized by their extension, for +example: ‘.wav’, ‘.aiff’, ‘.au’), so you may need to create and maintain +a growing collection of modules for the conversion between the various +file formats. There are also many different operations you might want +to perform on sound data (such as mixing, adding echo, applying an +equalizer function, creating an artificial stereo effect), so in +addition you will be writing a never-ending stream of modules to perform +these operations. Here’s a possible structure for your package +(expressed in terms of a hierarchical filesystem): + + sound/ Top-level package + __init__.py Initialize the sound package + formats/ Subpackage for file format conversions + __init__.py + wavread.py + wavwrite.py + aiffread.py + aiffwrite.py + auread.py + auwrite.py + ... + effects/ Subpackage for sound effects + __init__.py + echo.py + surround.py + reverse.py + ... + filters/ Subpackage for filters + __init__.py + equalizer.py + vocoder.py + karaoke.py + ... + +When importing the package, Python searches through the directories on +‘sys.path’ looking for the package subdirectory. + +The ‘__init__.py’ files are required to make Python treat directories +containing the file as packages. This prevents directories with a +common name, such as ‘string’, unintentionally hiding valid modules that +occur later on the module search path. In the simplest case, +‘__init__.py’ can just be an empty file, but it can also execute +initialization code for the package or set the ‘__all__’ variable, +described later. + +Users of the package can import individual modules from the package, for +example: + + import sound.effects.echo + +This loads the submodule ‘sound.effects.echo’. It must be referenced +with its full name. + + sound.effects.echo.echofilter(input, output, delay=0.7, atten=4) + +An alternative way of importing the submodule is: + + from sound.effects import echo + +This also loads the submodule ‘echo’, and makes it available without its +package prefix, so it can be used as follows: + + echo.echofilter(input, output, delay=0.7, atten=4) + +Yet another variation is to import the desired function or variable +directly: + + from sound.effects.echo import echofilter + +Again, this loads the submodule ‘echo’, but this makes its function +‘echofilter()’ directly available: + + echofilter(input, output, delay=0.7, atten=4) + +Note that when using ‘from package import item’, the item can be either +a submodule (or subpackage) of the package, or some other name defined +in the package, like a function, class or variable. The ‘import’ +statement first tests whether the item is defined in the package; if +not, it assumes it is a module and attempts to load it. If it fails to +find it, an *note ImportError: 334. exception is raised. + +Contrarily, when using syntax like ‘import item.subitem.subsubitem’, +each item except for the last must be a package; the last item can be a +module or a package but can’t be a class or function or variable defined +in the previous item. + +* Menu: + +* Importing * From a Package:: +* Intra-package References:: +* Packages in Multiple Directories:: + + +File: python.info, Node: Importing * From a Package, Next: Intra-package References, Up: Packages + +2.6.4.1 Importing * From a Package +.................................. + +Now what happens when the user writes ‘from sound.effects import *’? +Ideally, one would hope that this somehow goes out to the filesystem, +finds which submodules are present in the package, and imports them all. +This could take a long time and importing sub-modules might have +unwanted side-effects that should only happen when the sub-module is +explicitly imported. + +The only solution is for the package author to provide an explicit index +of the package. The *note import: c1d. statement uses the following +convention: if a package’s ‘__init__.py’ code defines a list named +‘__all__’, it is taken to be the list of module names that should be +imported when ‘from package import *’ is encountered. It is up to the +package author to keep this list up-to-date when a new version of the +package is released. Package authors may also decide not to support it, +if they don’t see a use for importing * from their package. For +example, the file ‘sound/effects/__init__.py’ could contain the +following code: + + __all__ = ["echo", "surround", "reverse"] + +This would mean that ‘from sound.effects import *’ would import the +three named submodules of the ‘sound’ package. + +If ‘__all__’ is not defined, the statement ‘from sound.effects import *’ +does `not' import all submodules from the package ‘sound.effects’ into +the current namespace; it only ensures that the package ‘sound.effects’ +has been imported (possibly running any initialization code in +‘__init__.py’) and then imports whatever names are defined in the +package. This includes any names defined (and submodules explicitly +loaded) by ‘__init__.py’. It also includes any submodules of the +package that were explicitly loaded by previous *note import: c1d. +statements. Consider this code: + + import sound.effects.echo + import sound.effects.surround + from sound.effects import * + +In this example, the ‘echo’ and ‘surround’ modules are imported in the +current namespace because they are defined in the ‘sound.effects’ +package when the ‘from...import’ statement is executed. (This also +works when ‘__all__’ is defined.) + +Although certain modules are designed to export only names that follow +certain patterns when you use ‘import *’, it is still considered bad +practice in production code. + +Remember, there is nothing wrong with using ‘from package import +specific_submodule’! In fact, this is the recommended notation unless +the importing module needs to use submodules with the same name from +different packages. + + +File: python.info, Node: Intra-package References, Next: Packages in Multiple Directories, Prev: Importing * From a Package, Up: Packages + +2.6.4.2 Intra-package References +................................ + +When packages are structured into subpackages (as with the ‘sound’ +package in the example), you can use absolute imports to refer to +submodules of siblings packages. For example, if the module +‘sound.filters.vocoder’ needs to use the ‘echo’ module in the +‘sound.effects’ package, it can use ‘from sound.effects import echo’. + +You can also write relative imports, with the ‘from module import name’ +form of import statement. These imports use leading dots to indicate +the current and parent packages involved in the relative import. From +the ‘surround’ module for example, you might use: + + from . import echo + from .. import formats + from ..filters import equalizer + +Note that relative imports are based on the name of the current module. +Since the name of the main module is always ‘"__main__"’, modules +intended for use as the main module of a Python application must always +use absolute imports. + + +File: python.info, Node: Packages in Multiple Directories, Prev: Intra-package References, Up: Packages + +2.6.4.3 Packages in Multiple Directories +........................................ + +Packages support one more special attribute, *note __path__: f23. This +is initialized to be a list containing the name of the directory holding +the package’s ‘__init__.py’ before the code in that file is executed. +This variable can be modified; doing so affects future searches for +modules and subpackages contained in the package. + +While this feature is not often needed, it can be used to extend the set +of modules found in a package. + + +File: python.info, Node: Input and Output, Next: Errors and Exceptions, Prev: Modules, Up: The Python Tutorial + +2.7 Input and Output +==================== + +There are several ways to present the output of a program; data can be +printed in a human-readable form, or written to a file for future use. +This chapter will discuss some of the possibilities. + +* Menu: + +* Fancier Output Formatting:: +* Reading and Writing Files:: + + +File: python.info, Node: Fancier Output Formatting, Next: Reading and Writing Files, Up: Input and Output + +2.7.1 Fancier Output Formatting +------------------------------- + +So far we’ve encountered two ways of writing values: `expression +statements' and the *note print(): 886. function. (A third way is using +the ‘write()’ method of file objects; the standard output file can be +referenced as ‘sys.stdout’. See the Library Reference for more +information on this.) + +Often you’ll want more control over the formatting of your output than +simply printing space-separated values. There are several ways to +format output. + + * To use *note formatted string literals: f29, begin a string with + ‘f’ or ‘F’ before the opening quotation mark or triple quotation + mark. Inside this string, you can write a Python expression + between ‘{’ and ‘}’ characters that can refer to variables or + literal values. + + >>> year = 2016 + >>> event = 'Referendum' + >>> f'Results of the {year} {event}' + 'Results of the 2016 Referendum' + + * The *note str.format(): 4da. method of strings requires more manual + effort. You’ll still use ‘{’ and ‘}’ to mark where a variable will + be substituted and can provide detailed formatting directives, but + you’ll also need to provide the information to be formatted. + + >>> yes_votes = 42_572_654 + >>> no_votes = 43_132_495 + >>> percentage = yes_votes / (yes_votes + no_votes) + >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) + ' 42572654 YES votes 49.67%' + + * Finally, you can do all the string handling yourself by using + string slicing and concatenation operations to create any layout + you can imagine. The string type has some methods that perform + useful operations for padding strings to a given column width. + +When you don’t need fancy output but just want a quick display of some +variables for debugging purposes, you can convert any value to a string +with the *note repr(): 7cc. or *note str(): 330. functions. + +The *note str(): 330. function is meant to return representations of +values which are fairly human-readable, while *note repr(): 7cc. is +meant to generate representations which can be read by the interpreter +(or will force a *note SyntaxError: 458. if there is no equivalent +syntax). For objects which don’t have a particular representation for +human consumption, *note str(): 330. will return the same value as *note +repr(): 7cc. Many values, such as numbers or structures like lists and +dictionaries, have the same representation using either function. +Strings, in particular, have two distinct representations. + +Some examples: + + >>> s = 'Hello, world.' + >>> str(s) + 'Hello, world.' + >>> repr(s) + "'Hello, world.'" + >>> str(1/7) + '0.14285714285714285' + >>> x = 10 * 3.25 + >>> y = 200 * 200 + >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...' + >>> print(s) + The value of x is 32.5, and y is 40000... + >>> # The repr() of a string adds string quotes and backslashes: + ... hello = 'hello, world\n' + >>> hellos = repr(hello) + >>> print(hellos) + 'hello, world\n' + >>> # The argument to repr() may be any Python object: + ... repr((x, y, ('spam', 'eggs'))) + "(32.5, 40000, ('spam', 'eggs'))" + +The *note string: f6. module contains a *note Template: 3eb. class that +offers yet another way to substitute values into strings, using +placeholders like ‘$x’ and replacing them with values from a dictionary, +but offers much less control of the formatting. + +* Menu: + +* Formatted String Literals:: +* The String format() Method: The String format Method. +* Manual String Formatting:: +* Old string formatting:: + + +File: python.info, Node: Formatted String Literals, Next: The String format Method, Up: Fancier Output Formatting + +2.7.1.1 Formatted String Literals +................................. + +*note Formatted string literals: 15c. (also called f-strings for short) +let you include the value of Python expressions inside a string by +prefixing the string with ‘f’ or ‘F’ and writing expressions as +‘{expression}’. + +An optional format specifier can follow the expression. This allows +greater control over how the value is formatted. The following example +rounds pi to three places after the decimal: + + >>> import math + >>> print(f'The value of pi is approximately {math.pi:.3f}.') + The value of pi is approximately 3.142. + +Passing an integer after the ‘':'’ will cause that field to be a minimum +number of characters wide. This is useful for making columns line up. + + >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678} + >>> for name, phone in table.items(): + ... print(f'{name:10} ==> {phone:10d}') + ... + Sjoerd ==> 4127 + Jack ==> 4098 + Dcab ==> 7678 + +Other modifiers can be used to convert the value before it is formatted. +‘'!a'’ applies *note ascii(): d4c, ‘'!s'’ applies *note str(): 330, and +‘'!r'’ applies *note repr(): 7cc.: + + >>> animals = 'eels' + >>> print(f'My hovercraft is full of {animals}.') + My hovercraft is full of eels. + >>> print(f'My hovercraft is full of {animals!r}.') + My hovercraft is full of 'eels'. + +For a reference on these format specifications, see the reference guide +for the *note Format Specification Mini-Language: 4de. + + +File: python.info, Node: The String format Method, Next: Manual String Formatting, Prev: Formatted String Literals, Up: Fancier Output Formatting + +2.7.1.2 The String format() Method +.................................. + +Basic usage of the *note str.format(): 4da. method looks like this: + + >>> print('We are the {} who say "{}!"'.format('knights', 'Ni')) + We are the knights who say "Ni!" + +The brackets and characters within them (called format fields) are +replaced with the objects passed into the *note str.format(): 4da. +method. A number in the brackets can be used to refer to the position +of the object passed into the *note str.format(): 4da. method. + + >>> print('{0} and {1}'.format('spam', 'eggs')) + spam and eggs + >>> print('{1} and {0}'.format('spam', 'eggs')) + eggs and spam + +If keyword arguments are used in the *note str.format(): 4da. method, +their values are referred to by using the name of the argument. + + >>> print('This {food} is {adjective}.'.format( + ... food='spam', adjective='absolutely horrible')) + This spam is absolutely horrible. + +Positional and keyword arguments can be arbitrarily combined: + + >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred', + other='Georg')) + The story of Bill, Manfred, and Georg. + +If you have a really long format string that you don’t want to split up, +it would be nice if you could reference the variables to be formatted by +name instead of by position. This can be done by simply passing the +dict and using square brackets ‘'[]'’ to access the keys. + + >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} + >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; ' + ... 'Dcab: {0[Dcab]:d}'.format(table)) + Jack: 4098; Sjoerd: 4127; Dcab: 8637678 + +This could also be done by passing the table as keyword arguments with +the ‘**’ notation. + + >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} + >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table)) + Jack: 4098; Sjoerd: 4127; Dcab: 8637678 + +This is particularly useful in combination with the built-in function +*note vars(): f2d, which returns a dictionary containing all local +variables. + +As an example, the following lines produce a tidily-aligned set of +columns giving integers and their squares and cubes: + + >>> for x in range(1, 11): + ... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x)) + ... + 1 1 1 + 2 4 8 + 3 9 27 + 4 16 64 + 5 25 125 + 6 36 216 + 7 49 343 + 8 64 512 + 9 81 729 + 10 100 1000 + +For a complete overview of string formatting with *note str.format(): +4da, see *note Format String Syntax: d1a. + + +File: python.info, Node: Manual String Formatting, Next: Old string formatting, Prev: The String format Method, Up: Fancier Output Formatting + +2.7.1.3 Manual String Formatting +................................ + +Here’s the same table of squares and cubes, formatted manually: + + >>> for x in range(1, 11): + ... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ') + ... # Note use of 'end' on previous line + ... print(repr(x*x*x).rjust(4)) + ... + 1 1 1 + 2 4 8 + 3 9 27 + 4 16 64 + 5 25 125 + 6 36 216 + 7 49 343 + 8 64 512 + 9 81 729 + 10 100 1000 + +(Note that the one space between each column was added by the way *note +print(): 886. works: it always adds spaces between its arguments.) + +The *note str.rjust(): f2f. method of string objects right-justifies a +string in a field of a given width by padding it with spaces on the +left. There are similar methods *note str.ljust(): f30. and *note +str.center(): f31. These methods do not write anything, they just +return a new string. If the input string is too long, they don’t +truncate it, but return it unchanged; this will mess up your column +lay-out but that’s usually better than the alternative, which would be +lying about a value. (If you really want truncation you can always add +a slice operation, as in ‘x.ljust(n)[:n]’.) + +There is another method, *note str.zfill(): f32, which pads a numeric +string on the left with zeros. It understands about plus and minus +signs: + + >>> '12'.zfill(5) + '00012' + >>> '-3.14'.zfill(7) + '-003.14' + >>> '3.14159265359'.zfill(5) + '3.14159265359' + + +File: python.info, Node: Old string formatting, Prev: Manual String Formatting, Up: Fancier Output Formatting + +2.7.1.4 Old string formatting +............................. + +The % operator (modulo) can also be used for string formatting. Given +‘'string' % values’, instances of ‘%’ in ‘string’ are replaced with zero +or more elements of ‘values’. This operation is commonly known as +string interpolation. For example: + + >>> import math + >>> print('The value of pi is approximately %5.3f.' % math.pi) + The value of pi is approximately 3.142. + +More information can be found in the *note printf-style String +Formatting: eb9. section. + + +File: python.info, Node: Reading and Writing Files, Prev: Fancier Output Formatting, Up: Input and Output + +2.7.2 Reading and Writing Files +------------------------------- + +*note open(): 4f0. returns a *note file object: b42, and is most +commonly used with two arguments: ‘open(filename, mode)’. + + >>> f = open('workfile', 'w') + +The first argument is a string containing the filename. The second +argument is another string containing a few characters describing the +way in which the file will be used. `mode' can be ‘'r'’ when the file +will only be read, ‘'w'’ for only writing (an existing file with the +same name will be erased), and ‘'a'’ opens the file for appending; any +data written to the file is automatically added to the end. ‘'r+'’ +opens the file for both reading and writing. The `mode' argument is +optional; ‘'r'’ will be assumed if it’s omitted. + +Normally, files are opened in `text mode', that means, you read and +write strings from and to the file, which are encoded in a specific +encoding. If encoding is not specified, the default is platform +dependent (see *note open(): 4f0.). ‘'b'’ appended to the mode opens +the file in `binary mode': now the data is read and written in the form +of bytes objects. This mode should be used for all files that don’t +contain text. + +In text mode, the default when reading is to convert platform-specific +line endings (‘\n’ on Unix, ‘\r\n’ on Windows) to just ‘\n’. When +writing in text mode, the default is to convert occurrences of ‘\n’ back +to platform-specific line endings. This behind-the-scenes modification +to file data is fine for text files, but will corrupt binary data like +that in ‘JPEG’ or ‘EXE’ files. Be very careful to use binary mode when +reading and writing such files. + +It is good practice to use the *note with: 6e9. keyword when dealing +with file objects. The advantage is that the file is properly closed +after its suite finishes, even if an exception is raised at some point. +Using ‘with’ is also much shorter than writing equivalent *note try: +d72.-*note finally: 182. blocks: + + >>> with open('workfile') as f: + ... read_data = f.read() + + >>> # We can check that the file has been automatically closed. + >>> f.closed + True + +If you’re not using the *note with: 6e9. keyword, then you should call +‘f.close()’ to close the file and immediately free up any system +resources used by it. + + Warning: Calling ‘f.write()’ without using the ‘with’ keyword or + calling ‘f.close()’ `might' result in the arguments of ‘f.write()’ + not being completely written to the disk, even if the program exits + successfully. + +After a file object is closed, either by a *note with: 6e9. statement or +by calling ‘f.close()’, attempts to use the file object will +automatically fail. + + >>> f.close() + >>> f.read() + Traceback (most recent call last): + File "", line 1, in + ValueError: I/O operation on closed file. + +* Menu: + +* Methods of File Objects:: +* Saving structured data with json:: + + +File: python.info, Node: Methods of File Objects, Next: Saving structured data with json, Up: Reading and Writing Files + +2.7.2.1 Methods of File Objects +............................... + +The rest of the examples in this section will assume that a file object +called ‘f’ has already been created. + +To read a file’s contents, call ‘f.read(size)’, which reads some +quantity of data and returns it as a string (in text mode) or bytes +object (in binary mode). `size' is an optional numeric argument. When +`size' is omitted or negative, the entire contents of the file will be +read and returned; it’s your problem if the file is twice as large as +your machine’s memory. Otherwise, at most `size' characters (in text +mode) or `size' bytes (in binary mode) are read and returned. If the +end of the file has been reached, ‘f.read()’ will return an empty string +(‘''’). + + >>> f.read() + 'This is the entire file.\n' + >>> f.read() + '' + +‘f.readline()’ reads a single line from the file; a newline character +(‘\n’) is left at the end of the string, and is only omitted on the last +line of the file if the file doesn’t end in a newline. This makes the +return value unambiguous; if ‘f.readline()’ returns an empty string, the +end of the file has been reached, while a blank line is represented by +‘'\n'’, a string containing only a single newline. + + >>> f.readline() + 'This is the first line of the file.\n' + >>> f.readline() + 'Second line of the file\n' + >>> f.readline() + '' + +For reading lines from a file, you can loop over the file object. This +is memory efficient, fast, and leads to simple code: + + >>> for line in f: + ... print(line, end='') + ... + This is the first line of the file. + Second line of the file + +If you want to read all the lines of a file in a list you can also use +‘list(f)’ or ‘f.readlines()’. + +‘f.write(string)’ writes the contents of `string' to the file, returning +the number of characters written. + + >>> f.write('This is a test\n') + 15 + +Other types of objects need to be converted – either to a string (in +text mode) or a bytes object (in binary mode) – before writing them: + + >>> value = ('the answer', 42) + >>> s = str(value) # convert the tuple to string + >>> f.write(s) + 18 + +‘f.tell()’ returns an integer giving the file object’s current position +in the file represented as number of bytes from the beginning of the +file when in binary mode and an opaque number when in text mode. + +To change the file object’s position, use ‘f.seek(offset, whence)’. The +position is computed from adding `offset' to a reference point; the +reference point is selected by the `whence' argument. A `whence' value +of 0 measures from the beginning of the file, 1 uses the current file +position, and 2 uses the end of the file as the reference point. +`whence' can be omitted and defaults to 0, using the beginning of the +file as the reference point. + + >>> f = open('workfile', 'rb+') + >>> f.write(b'0123456789abcdef') + 16 + >>> f.seek(5) # Go to the 6th byte in the file + 5 + >>> f.read(1) + b'5' + >>> f.seek(-3, 2) # Go to the 3rd byte before the end + 13 + >>> f.read(1) + b'd' + +In text files (those opened without a ‘b’ in the mode string), only +seeks relative to the beginning of the file are allowed (the exception +being seeking to the very file end with ‘seek(0, 2)’) and the only valid +`offset' values are those returned from the ‘f.tell()’, or zero. Any +other `offset' value produces undefined behaviour. + +File objects have some additional methods, such as ‘isatty()’ and +‘truncate()’ which are less frequently used; consult the Library +Reference for a complete guide to file objects. + + +File: python.info, Node: Saving structured data with json, Prev: Methods of File Objects, Up: Reading and Writing Files + +2.7.2.2 Saving structured data with ‘json’ +.......................................... + +Strings can easily be written to and read from a file. Numbers take a +bit more effort, since the ‘read()’ method only returns strings, which +will have to be passed to a function like *note int(): 184, which takes +a string like ‘'123'’ and returns its numeric value 123. When you want +to save more complex data types like nested lists and dictionaries, +parsing and serializing by hand becomes complicated. + +Rather than having users constantly writing and debugging code to save +complicated data types to files, Python allows you to use the popular +data interchange format called JSON (JavaScript Object Notation)(1). +The standard module called *note json: a4. can take Python data +hierarchies, and convert them to string representations; this process is +called `serializing'. Reconstructing the data from the string +representation is called `deserializing'. Between serializing and +deserializing, the string representing the object may have been stored +in a file or data, or sent over a network connection to some distant +machine. + + Note: The JSON format is commonly used by modern applications to + allow for data exchange. Many programmers are already familiar + with it, which makes it a good choice for interoperability. + +If you have an object ‘x’, you can view its JSON string representation +with a simple line of code: + + >>> import json + >>> json.dumps([1, 'simple', 'list']) + '[1, "simple", "list"]' + +Another variant of the *note dumps(): 605. function, called *note +dump(): 604, simply serializes the object to a *note text file: f3a. So +if ‘f’ is a *note text file: f3a. object opened for writing, we can do +this: + + json.dump(x, f) + +To decode the object again, if ‘f’ is a *note text file: f3a. object +which has been opened for reading: + + x = json.load(f) + +This simple serialization technique can handle lists and dictionaries, +but serializing arbitrary class instances in JSON requires a bit of +extra effort. The reference for the *note json: a4. module contains an +explanation of this. + +See also +........ + +*note pickle: ca. - the pickle module + +Contrary to *note JSON: f39, `pickle' is a protocol which allows the +serialization of arbitrarily complex Python objects. As such, it is +specific to Python and cannot be used to communicate with applications +written in other languages. It is also insecure by default: +deserializing pickle data coming from an untrusted source can execute +arbitrary code, if the data was crafted by a skilled attacker. + + ---------- Footnotes ---------- + + (1) http://json.org + + +File: python.info, Node: Errors and Exceptions, Next: Classes, Prev: Input and Output, Up: The Python Tutorial + +2.8 Errors and Exceptions +========================= + +Until now error messages haven’t been more than mentioned, but if you +have tried out the examples you have probably seen some. There are (at +least) two distinguishable kinds of errors: `syntax errors' and +`exceptions'. + +* Menu: + +* Syntax Errors:: +* Exceptions:: +* Handling Exceptions:: +* Raising Exceptions:: +* User-defined Exceptions:: +* Defining Clean-up Actions:: +* Predefined Clean-up Actions:: + + +File: python.info, Node: Syntax Errors, Next: Exceptions, Up: Errors and Exceptions + +2.8.1 Syntax Errors +------------------- + +Syntax errors, also known as parsing errors, are perhaps the most common +kind of complaint you get while you are still learning Python: + + >>> while True print('Hello world') + File "", line 1 + while True print('Hello world') + ^ + SyntaxError: invalid syntax + +The parser repeats the offending line and displays a little ‘arrow’ +pointing at the earliest point in the line where the error was detected. +The error is caused by (or at least detected at) the token `preceding' +the arrow: in the example, the error is detected at the function *note +print(): 886, since a colon (‘':'’) is missing before it. File name and +line number are printed so you know where to look in case the input came +from a script. + + +File: python.info, Node: Exceptions, Next: Handling Exceptions, Prev: Syntax Errors, Up: Errors and Exceptions + +2.8.2 Exceptions +---------------- + +Even if a statement or expression is syntactically correct, it may cause +an error when an attempt is made to execute it. Errors detected during +execution are called `exceptions' and are not unconditionally fatal: you +will soon learn how to handle them in Python programs. Most exceptions +are not handled by programs, however, and result in error messages as +shown here: + + >>> 10 * (1/0) + Traceback (most recent call last): + File "", line 1, in + ZeroDivisionError: division by zero + >>> 4 + spam*3 + Traceback (most recent call last): + File "", line 1, in + NameError: name 'spam' is not defined + >>> '2' + 2 + Traceback (most recent call last): + File "", line 1, in + TypeError: Can't convert 'int' object to str implicitly + +The last line of the error message indicates what happened. Exceptions +come in different types, and the type is printed as part of the message: +the types in the example are *note ZeroDivisionError: f42, *note +NameError: d7b. and *note TypeError: 192. The string printed as the +exception type is the name of the built-in exception that occurred. +This is true for all built-in exceptions, but need not be true for +user-defined exceptions (although it is a useful convention). Standard +exception names are built-in identifiers (not reserved keywords). + +The rest of the line provides detail based on the type of exception and +what caused it. + +The preceding part of the error message shows the context where the +exception happened, in the form of a stack traceback. In general it +contains a stack traceback listing source lines; however, it will not +display lines read from standard input. + +*note Built-in Exceptions: f43. lists the built-in exceptions and their +meanings. + + +File: python.info, Node: Handling Exceptions, Next: Raising Exceptions, Prev: Exceptions, Up: Errors and Exceptions + +2.8.3 Handling Exceptions +------------------------- + +It is possible to write programs that handle selected exceptions. Look +at the following example, which asks the user for input until a valid +integer has been entered, but allows the user to interrupt the program +(using ‘Control-C’ or whatever the operating system supports); note that +a user-generated interruption is signalled by raising the *note +KeyboardInterrupt: 197. exception. + + >>> while True: + ... try: + ... x = int(input("Please enter a number: ")) + ... break + ... except ValueError: + ... print("Oops! That was no valid number. Try again...") + ... + +The *note try: d72. statement works as follows. + + * First, the `try clause' (the statement(s) between the *note try: + d72. and *note except: b3e. keywords) is executed. + + * If no exception occurs, the `except clause' is skipped and + execution of the *note try: d72. statement is finished. + + * If an exception occurs during execution of the try clause, the rest + of the clause is skipped. Then if its type matches the exception + named after the *note except: b3e. keyword, the except clause is + executed, and then execution continues after the *note try: d72. + statement. + + * If an exception occurs which does not match the exception named in + the except clause, it is passed on to outer *note try: d72. + statements; if no handler is found, it is an `unhandled exception' + and execution stops with a message as shown above. + +A *note try: d72. statement may have more than one except clause, to +specify handlers for different exceptions. At most one handler will be +executed. Handlers only handle exceptions that occur in the +corresponding try clause, not in other handlers of the same ‘try’ +statement. An except clause may name multiple exceptions as a +parenthesized tuple, for example: + + ... except (RuntimeError, TypeError, NameError): + ... pass + +A class in an *note except: b3e. clause is compatible with an exception +if it is the same class or a base class thereof (but not the other way +around — an except clause listing a derived class is not compatible with +a base class). For example, the following code will print B, C, D in +that order: + + class B(Exception): + pass + + class C(B): + pass + + class D(C): + pass + + for cls in [B, C, D]: + try: + raise cls() + except D: + print("D") + except C: + print("C") + except B: + print("B") + +Note that if the except clauses were reversed (with ‘except B’ first), +it would have printed B, B, B — the first matching except clause is +triggered. + +The last except clause may omit the exception name(s), to serve as a +wildcard. Use this with extreme caution, since it is easy to mask a +real programming error in this way! It can also be used to print an +error message and then re-raise the exception (allowing a caller to +handle the exception as well): + + import sys + + try: + f = open('myfile.txt') + s = f.readline() + i = int(s.strip()) + except OSError as err: + print("OS error: {0}".format(err)) + except ValueError: + print("Could not convert data to an integer.") + except: + print("Unexpected error:", sys.exc_info()[0]) + raise + +The *note try: d72. … *note except: b3e. statement has an optional `else +clause', which, when present, must follow all except clauses. It is +useful for code that must be executed if the try clause does not raise +an exception. For example: + + for arg in sys.argv[1:]: + try: + f = open(arg, 'r') + except OSError: + print('cannot open', arg) + else: + print(arg, 'has', len(f.readlines()), 'lines') + f.close() + +The use of the ‘else’ clause is better than adding additional code to +the *note try: d72. clause because it avoids accidentally catching an +exception that wasn’t raised by the code being protected by the ‘try’ … +‘except’ statement. + +When an exception occurs, it may have an associated value, also known as +the exception’s `argument'. The presence and type of the argument +depend on the exception type. + +The except clause may specify a variable after the exception name. The +variable is bound to an exception instance with the arguments stored in +‘instance.args’. For convenience, the exception instance defines *note +__str__(): e37. so the arguments can be printed directly without having +to reference ‘.args’. One may also instantiate an exception first +before raising it and add any attributes to it as desired. + + >>> try: + ... raise Exception('spam', 'eggs') + ... except Exception as inst: + ... print(type(inst)) # the exception instance + ... print(inst.args) # arguments stored in .args + ... print(inst) # __str__ allows args to be printed directly, + ... # but may be overridden in exception subclasses + ... x, y = inst.args # unpack args + ... print('x =', x) + ... print('y =', y) + ... + + ('spam', 'eggs') + ('spam', 'eggs') + x = spam + y = eggs + +If an exception has arguments, they are printed as the last part +(‘detail’) of the message for unhandled exceptions. + +Exception handlers don’t just handle exceptions if they occur +immediately in the try clause, but also if they occur inside functions +that are called (even indirectly) in the try clause. For example: + + >>> def this_fails(): + ... x = 1/0 + ... + >>> try: + ... this_fails() + ... except ZeroDivisionError as err: + ... print('Handling run-time error:', err) + ... + Handling run-time error: division by zero + + +File: python.info, Node: Raising Exceptions, Next: User-defined Exceptions, Prev: Handling Exceptions, Up: Errors and Exceptions + +2.8.4 Raising Exceptions +------------------------ + +The *note raise: c42. statement allows the programmer to force a +specified exception to occur. For example: + + >>> raise NameError('HiThere') + Traceback (most recent call last): + File "", line 1, in + NameError: HiThere + +The sole argument to *note raise: c42. indicates the exception to be +raised. This must be either an exception instance or an exception class +(a class that derives from *note Exception: 1a9.). If an exception +class is passed, it will be implicitly instantiated by calling its +constructor with no arguments: + + raise ValueError # shorthand for 'raise ValueError()' + +If you need to determine whether an exception was raised but don’t +intend to handle it, a simpler form of the *note raise: c42. statement +allows you to re-raise the exception: + + >>> try: + ... raise NameError('HiThere') + ... except NameError: + ... print('An exception flew by!') + ... raise + ... + An exception flew by! + Traceback (most recent call last): + File "", line 2, in + NameError: HiThere + + +File: python.info, Node: User-defined Exceptions, Next: Defining Clean-up Actions, Prev: Raising Exceptions, Up: Errors and Exceptions + +2.8.5 User-defined Exceptions +----------------------------- + +Programs may name their own exceptions by creating a new exception class +(see *note Classes: ed9. for more about Python classes). Exceptions +should typically be derived from the *note Exception: 1a9. class, either +directly or indirectly. + +Exception classes can be defined which do anything any other class can +do, but are usually kept simple, often only offering a number of +attributes that allow information about the error to be extracted by +handlers for the exception. When creating a module that can raise +several distinct errors, a common practice is to create a base class for +exceptions defined by that module, and subclass that to create specific +exception classes for different error conditions: + + class Error(Exception): + """Base class for exceptions in this module.""" + pass + + class InputError(Error): + """Exception raised for errors in the input. + + Attributes: + expression -- input expression in which the error occurred + message -- explanation of the error + """ + + def __init__(self, expression, message): + self.expression = expression + self.message = message + + class TransitionError(Error): + """Raised when an operation attempts a state transition that's not + allowed. + + Attributes: + previous -- state at beginning of transition + next -- attempted new state + message -- explanation of why the specific transition is not allowed + """ + + def __init__(self, previous, next, message): + self.previous = previous + self.next = next + self.message = message + +Most exceptions are defined with names that end in “Error”, similar to +the naming of the standard exceptions. + +Many standard modules define their own exceptions to report errors that +may occur in functions they define. More information on classes is +presented in chapter *note Classes: ed9. + + +File: python.info, Node: Defining Clean-up Actions, Next: Predefined Clean-up Actions, Prev: User-defined Exceptions, Up: Errors and Exceptions + +2.8.6 Defining Clean-up Actions +------------------------------- + +The *note try: d72. statement has another optional clause which is +intended to define clean-up actions that must be executed under all +circumstances. For example: + + >>> try: + ... raise KeyboardInterrupt + ... finally: + ... print('Goodbye, world!') + ... + Goodbye, world! + Traceback (most recent call last): + File "", line 2, in + KeyboardInterrupt + +If a *note finally: 182. clause is present, the ‘finally’ clause will +execute as the last task before the *note try: d72. statement completes. +The ‘finally’ clause runs whether or not the ‘try’ statement produces an +exception. The following points discuss more complex cases when an +exception occurs: + + * If an exception occurs during execution of the ‘try’ clause, the + exception may be handled by an *note except: b3e. clause. If the + exception is not handled by an ‘except’ clause, the exception is + re-raised after the ‘finally’ clause has been executed. + + * An exception could occur during execution of an ‘except’ or ‘else’ + clause. Again, the exception is re-raised after the ‘finally’ + clause has been executed. + + * If the ‘try’ statement reaches a *note break: 2db, *note continue: + 181. or *note return: 190. statement, the ‘finally’ clause will + execute just prior to the ‘break’, ‘continue’ or ‘return’ + statement’s execution. + + * If a ‘finally’ clause includes a ‘return’ statement, the returned + value will be the one from the ‘finally’ clause’s ‘return’ + statement, not the value from the ‘try’ clause’s ‘return’ + statement. + +For example: + + >>> def bool_return(): + ... try: + ... return True + ... finally: + ... return False + ... + >>> bool_return() + False + +A more complicated example: + + >>> def divide(x, y): + ... try: + ... result = x / y + ... except ZeroDivisionError: + ... print("division by zero!") + ... else: + ... print("result is", result) + ... finally: + ... print("executing finally clause") + ... + >>> divide(2, 1) + result is 2.0 + executing finally clause + >>> divide(2, 0) + division by zero! + executing finally clause + >>> divide("2", "1") + executing finally clause + Traceback (most recent call last): + File "", line 1, in + File "", line 3, in divide + TypeError: unsupported operand type(s) for /: 'str' and 'str' + +As you can see, the *note finally: 182. clause is executed in any event. +The *note TypeError: 192. raised by dividing two strings is not handled +by the *note except: b3e. clause and therefore re-raised after the +‘finally’ clause has been executed. + +In real world applications, the *note finally: 182. clause is useful for +releasing external resources (such as files or network connections), +regardless of whether the use of the resource was successful. + + +File: python.info, Node: Predefined Clean-up Actions, Prev: Defining Clean-up Actions, Up: Errors and Exceptions + +2.8.7 Predefined Clean-up Actions +--------------------------------- + +Some objects define standard clean-up actions to be undertaken when the +object is no longer needed, regardless of whether or not the operation +using the object succeeded or failed. Look at the following example, +which tries to open a file and print its contents to the screen. + + for line in open("myfile.txt"): + print(line, end="") + +The problem with this code is that it leaves the file open for an +indeterminate amount of time after this part of the code has finished +executing. This is not an issue in simple scripts, but can be a problem +for larger applications. The *note with: 6e9. statement allows objects +like files to be used in a way that ensures they are always cleaned up +promptly and correctly. + + with open("myfile.txt") as f: + for line in f: + print(line, end="") + +After the statement is executed, the file `f' is always closed, even if +a problem was encountered while processing the lines. Objects which, +like files, provide predefined clean-up actions will indicate this in +their documentation. + + +File: python.info, Node: Classes, Next: Brief Tour of the Standard Library, Prev: Errors and Exceptions, Up: The Python Tutorial + +2.9 Classes +=========== + +Classes provide a means of bundling data and functionality together. +Creating a new class creates a new `type' of object, allowing new +`instances' of that type to be made. Each class instance can have +attributes attached to it for maintaining its state. Class instances +can also have methods (defined by its class) for modifying its state. + +Compared with other programming languages, Python’s class mechanism adds +classes with a minimum of new syntax and semantics. It is a mixture of +the class mechanisms found in C++ and Modula-3. Python classes provide +all the standard features of Object Oriented Programming: the class +inheritance mechanism allows multiple base classes, a derived class can +override any methods of its base class or classes, and a method can call +the method of a base class with the same name. Objects can contain +arbitrary amounts and kinds of data. As is true for modules, classes +partake of the dynamic nature of Python: they are created at runtime, +and can be modified further after creation. + +In C++ terminology, normally class members (including the data members) +are `public' (except see below *note Private Variables: f4f.), and all +member functions are `virtual'. As in Modula-3, there are no shorthands +for referencing the object’s members from its methods: the method +function is declared with an explicit first argument representing the +object, which is provided implicitly by the call. As in Smalltalk, +classes themselves are objects. This provides semantics for importing +and renaming. Unlike C++ and Modula-3, built-in types can be used as +base classes for extension by the user. Also, like in C++, most +built-in operators with special syntax (arithmetic operators, +subscripting etc.) can be redefined for class instances. + +(Lacking universally accepted terminology to talk about classes, I will +make occasional use of Smalltalk and C++ terms. I would use Modula-3 +terms, since its object-oriented semantics are closer to those of Python +than C++, but I expect that few readers have heard of it.) + +* Menu: + +* A Word About Names and Objects:: +* Python Scopes and Namespaces:: +* A First Look at Classes:: +* Random Remarks:: +* Inheritance:: +* Private Variables:: +* Odds and Ends:: +* Iterators:: +* Generators:: +* Generator Expressions:: + + +File: python.info, Node: A Word About Names and Objects, Next: Python Scopes and Namespaces, Up: Classes + +2.9.1 A Word About Names and Objects +------------------------------------ + +Objects have individuality, and multiple names (in multiple scopes) can +be bound to the same object. This is known as aliasing in other +languages. This is usually not appreciated on a first glance at Python, +and can be safely ignored when dealing with immutable basic types +(numbers, strings, tuples). However, aliasing has a possibly surprising +effect on the semantics of Python code involving mutable objects such as +lists, dictionaries, and most other types. This is usually used to the +benefit of the program, since aliases behave like pointers in some +respects. For example, passing an object is cheap since only a pointer +is passed by the implementation; and if a function modifies an object +passed as an argument, the caller will see the change — this eliminates +the need for two different argument passing mechanisms as in Pascal. + + +File: python.info, Node: Python Scopes and Namespaces, Next: A First Look at Classes, Prev: A Word About Names and Objects, Up: Classes + +2.9.2 Python Scopes and Namespaces +---------------------------------- + +Before introducing classes, I first have to tell you something about +Python’s scope rules. Class definitions play some neat tricks with +namespaces, and you need to know how scopes and namespaces work to fully +understand what’s going on. Incidentally, knowledge about this subject +is useful for any advanced Python programmer. + +Let’s begin with some definitions. + +A `namespace' is a mapping from names to objects. Most namespaces are +currently implemented as Python dictionaries, but that’s normally not +noticeable in any way (except for performance), and it may change in the +future. Examples of namespaces are: the set of built-in names +(containing functions such as *note abs(): f54, and built-in exception +names); the global names in a module; and the local names in a function +invocation. In a sense the set of attributes of an object also form a +namespace. The important thing to know about namespaces is that there +is absolutely no relation between names in different namespaces; for +instance, two different modules may both define a function ‘maximize’ +without confusion — users of the modules must prefix it with the module +name. + +By the way, I use the word `attribute' for any name following a dot — +for example, in the expression ‘z.real’, ‘real’ is an attribute of the +object ‘z’. Strictly speaking, references to names in modules are +attribute references: in the expression ‘modname.funcname’, ‘modname’ is +a module object and ‘funcname’ is an attribute of it. In this case +there happens to be a straightforward mapping between the module’s +attributes and the global names defined in the module: they share the +same namespace! (1) + +Attributes may be read-only or writable. In the latter case, assignment +to attributes is possible. Module attributes are writable: you can +write ‘modname.the_answer = 42’. Writable attributes may also be +deleted with the *note del: f02. statement. For example, ‘del +modname.the_answer’ will remove the attribute ‘the_answer’ from the +object named by ‘modname’. + +Namespaces are created at different moments and have different +lifetimes. The namespace containing the built-in names is created when +the Python interpreter starts up, and is never deleted. The global +namespace for a module is created when the module definition is read in; +normally, module namespaces also last until the interpreter quits. The +statements executed by the top-level invocation of the interpreter, +either read from a script file or interactively, are considered part of +a module called *note __main__: 1, so they have their own global +namespace. (The built-in names actually also live in a module; this is +called *note builtins: 13.) + +The local namespace for a function is created when the function is +called, and deleted when the function returns or raises an exception +that is not handled within the function. (Actually, forgetting would be +a better way to describe what actually happens.) Of course, recursive +invocations each have their own local namespace. + +A `scope' is a textual region of a Python program where a namespace is +directly accessible. “Directly accessible” here means that an +unqualified reference to a name attempts to find the name in the +namespace. + +Although scopes are determined statically, they are used dynamically. +At any time during execution, there are 3 or 4 nested scopes whose +namespaces are directly accessible: + + * the innermost scope, which is searched first, contains the local + names + + * the scopes of any enclosing functions, which are searched starting + with the nearest enclosing scope, contains non-local, but also + non-global names + + * the next-to-last scope contains the current module’s global names + + * the outermost scope (searched last) is the namespace containing + built-in names + +If a name is declared global, then all references and assignments go +directly to the middle scope containing the module’s global names. To +rebind variables found outside of the innermost scope, the *note +nonlocal: c3f. statement can be used; if not declared nonlocal, those +variables are read-only (an attempt to write to such a variable will +simply create a `new' local variable in the innermost scope, leaving the +identically named outer variable unchanged). + +Usually, the local scope references the local names of the (textually) +current function. Outside functions, the local scope references the +same namespace as the global scope: the module’s namespace. Class +definitions place yet another namespace in the local scope. + +It is important to realize that scopes are determined textually: the +global scope of a function defined in a module is that module’s +namespace, no matter from where or by what alias the function is called. +On the other hand, the actual search for names is done dynamically, at +run time — however, the language definition is evolving towards static +name resolution, at “compile” time, so don’t rely on dynamic name +resolution! (In fact, local variables are already determined +statically.) + +A special quirk of Python is that – if no *note global: ed8. or *note +nonlocal: c3f. statement is in effect – assignments to names always go +into the innermost scope. Assignments do not copy data — they just bind +names to objects. The same is true for deletions: the statement ‘del x’ +removes the binding of ‘x’ from the namespace referenced by the local +scope. In fact, all operations that introduce new names use the local +scope: in particular, *note import: c1d. statements and function +definitions bind the module or function name in the local scope. + +The *note global: ed8. statement can be used to indicate that particular +variables live in the global scope and should be rebound there; the +*note nonlocal: c3f. statement indicates that particular variables live +in an enclosing scope and should be rebound there. + +* Menu: + +* Scopes and Namespaces Example:: + + ---------- Footnotes ---------- + + (1) (1) Except for one thing. Module objects have a secret read-only +attribute called *note __dict__: 4fc. which returns the dictionary used +to implement the module’s namespace; the name *note __dict__: 4fc. is an +attribute but not a global name. Obviously, using this violates the +abstraction of namespace implementation, and should be restricted to +things like post-mortem debuggers. + + +File: python.info, Node: Scopes and Namespaces Example, Up: Python Scopes and Namespaces + +2.9.2.1 Scopes and Namespaces Example +..................................... + +This is an example demonstrating how to reference the different scopes +and namespaces, and how *note global: ed8. and *note nonlocal: c3f. +affect variable binding: + + def scope_test(): + def do_local(): + spam = "local spam" + + def do_nonlocal(): + nonlocal spam + spam = "nonlocal spam" + + def do_global(): + global spam + spam = "global spam" + + spam = "test spam" + do_local() + print("After local assignment:", spam) + do_nonlocal() + print("After nonlocal assignment:", spam) + do_global() + print("After global assignment:", spam) + + scope_test() + print("In global scope:", spam) + +The output of the example code is: + + After local assignment: test spam + After nonlocal assignment: nonlocal spam + After global assignment: nonlocal spam + In global scope: global spam + +Note how the `local' assignment (which is default) didn’t change +`scope_test'’s binding of `spam'. The *note nonlocal: c3f. assignment +changed `scope_test'’s binding of `spam', and the *note global: ed8. +assignment changed the module-level binding. + +You can also see that there was no previous binding for `spam' before +the *note global: ed8. assignment. + + +File: python.info, Node: A First Look at Classes, Next: Random Remarks, Prev: Python Scopes and Namespaces, Up: Classes + +2.9.3 A First Look at Classes +----------------------------- + +Classes introduce a little bit of new syntax, three new object types, +and some new semantics. + +* Menu: + +* Class Definition Syntax:: +* Class Objects:: +* Instance Objects:: +* Method Objects:: +* Class and Instance Variables:: + + +File: python.info, Node: Class Definition Syntax, Next: Class Objects, Up: A First Look at Classes + +2.9.3.1 Class Definition Syntax +............................... + +The simplest form of class definition looks like this: + + class ClassName: + + . + . + . + + +Class definitions, like function definitions (*note def: dbe. +statements) must be executed before they have any effect. (You could +conceivably place a class definition in a branch of an *note if: de7. +statement, or inside a function.) + +In practice, the statements inside a class definition will usually be +function definitions, but other statements are allowed, and sometimes +useful — we’ll come back to this later. The function definitions inside +a class normally have a peculiar form of argument list, dictated by the +calling conventions for methods — again, this is explained later. + +When a class definition is entered, a new namespace is created, and used +as the local scope — thus, all assignments to local variables go into +this new namespace. In particular, function definitions bind the name +of the new function here. + +When a class definition is left normally (via the end), a `class object' +is created. This is basically a wrapper around the contents of the +namespace created by the class definition; we’ll learn more about class +objects in the next section. The original local scope (the one in +effect just before the class definition was entered) is reinstated, and +the class object is bound here to the class name given in the class +definition header (‘ClassName’ in the example). + + +File: python.info, Node: Class Objects, Next: Instance Objects, Prev: Class Definition Syntax, Up: A First Look at Classes + +2.9.3.2 Class Objects +..................... + +Class objects support two kinds of operations: attribute references and +instantiation. + +`Attribute references' use the standard syntax used for all attribute +references in Python: ‘obj.name’. Valid attribute names are all the +names that were in the class’s namespace when the class object was +created. So, if the class definition looked like this: + + class MyClass: + """A simple example class""" + i = 12345 + + def f(self): + return 'hello world' + +then ‘MyClass.i’ and ‘MyClass.f’ are valid attribute references, +returning an integer and a function object, respectively. Class +attributes can also be assigned to, so you can change the value of +‘MyClass.i’ by assignment. ‘__doc__’ is also a valid attribute, +returning the docstring belonging to the class: ‘"A simple example +class"’. + +Class `instantiation' uses function notation. Just pretend that the +class object is a parameterless function that returns a new instance of +the class. For example (assuming the above class): + + x = MyClass() + +creates a new `instance' of the class and assigns this object to the +local variable ‘x’. + +The instantiation operation (“calling” a class object) creates an empty +object. Many classes like to create objects with instances customized +to a specific initial state. Therefore a class may define a special +method named *note __init__(): d5e, like this: + + def __init__(self): + self.data = [] + +When a class defines an *note __init__(): d5e. method, class +instantiation automatically invokes *note __init__(): d5e. for the +newly-created class instance. So in this example, a new, initialized +instance can be obtained by: + + x = MyClass() + +Of course, the *note __init__(): d5e. method may have arguments for +greater flexibility. In that case, arguments given to the class +instantiation operator are passed on to *note __init__(): d5e. For +example, + + >>> class Complex: + ... def __init__(self, realpart, imagpart): + ... self.r = realpart + ... self.i = imagpart + ... + >>> x = Complex(3.0, -4.5) + >>> x.r, x.i + (3.0, -4.5) + + +File: python.info, Node: Instance Objects, Next: Method Objects, Prev: Class Objects, Up: A First Look at Classes + +2.9.3.3 Instance Objects +........................ + +Now what can we do with instance objects? The only operations +understood by instance objects are attribute references. There are two +kinds of valid attribute names: data attributes and methods. + +`data attributes' correspond to “instance variables” in Smalltalk, and +to “data members” in C++. Data attributes need not be declared; like +local variables, they spring into existence when they are first assigned +to. For example, if ‘x’ is the instance of ‘MyClass’ created above, the +following piece of code will print the value ‘16’, without leaving a +trace: + + x.counter = 1 + while x.counter < 10: + x.counter = x.counter * 2 + print(x.counter) + del x.counter + +The other kind of instance attribute reference is a `method'. A method +is a function that “belongs to” an object. (In Python, the term method +is not unique to class instances: other object types can have methods as +well. For example, list objects have methods called append, insert, +remove, sort, and so on. However, in the following discussion, we’ll +use the term method exclusively to mean methods of class instance +objects, unless explicitly stated otherwise.) + +Valid method names of an instance object depend on its class. By +definition, all attributes of a class that are function objects define +corresponding methods of its instances. So in our example, ‘x.f’ is a +valid method reference, since ‘MyClass.f’ is a function, but ‘x.i’ is +not, since ‘MyClass.i’ is not. But ‘x.f’ is not the same thing as +‘MyClass.f’ — it is a `method object', not a function object. + + +File: python.info, Node: Method Objects, Next: Class and Instance Variables, Prev: Instance Objects, Up: A First Look at Classes + +2.9.3.4 Method Objects +...................... + +Usually, a method is called right after it is bound: + + x.f() + +In the ‘MyClass’ example, this will return the string ‘'hello world'’. +However, it is not necessary to call a method right away: ‘x.f’ is a +method object, and can be stored away and called at a later time. For +example: + + xf = x.f + while True: + print(xf()) + +will continue to print ‘hello world’ until the end of time. + +What exactly happens when a method is called? You may have noticed that +‘x.f()’ was called without an argument above, even though the function +definition for ‘f()’ specified an argument. What happened to the +argument? Surely Python raises an exception when a function that +requires an argument is called without any — even if the argument isn’t +actually used… + +Actually, you may have guessed the answer: the special thing about +methods is that the instance object is passed as the first argument of +the function. In our example, the call ‘x.f()’ is exactly equivalent to +‘MyClass.f(x)’. In general, calling a method with a list of `n' +arguments is equivalent to calling the corresponding function with an +argument list that is created by inserting the method’s instance object +before the first argument. + +If you still don’t understand how methods work, a look at the +implementation can perhaps clarify matters. When a non-data attribute +of an instance is referenced, the instance’s class is searched. If the +name denotes a valid class attribute that is a function object, a method +object is created by packing (pointers to) the instance object and the +function object just found together in an abstract object: this is the +method object. When the method object is called with an argument list, +a new argument list is constructed from the instance object and the +argument list, and the function object is called with this new argument +list. + + +File: python.info, Node: Class and Instance Variables, Prev: Method Objects, Up: A First Look at Classes + +2.9.3.5 Class and Instance Variables +.................................... + +Generally speaking, instance variables are for data unique to each +instance and class variables are for attributes and methods shared by +all instances of the class: + + class Dog: + + kind = 'canine' # class variable shared by all instances + + def __init__(self, name): + self.name = name # instance variable unique to each instance + + >>> d = Dog('Fido') + >>> e = Dog('Buddy') + >>> d.kind # shared by all dogs + 'canine' + >>> e.kind # shared by all dogs + 'canine' + >>> d.name # unique to d + 'Fido' + >>> e.name # unique to e + 'Buddy' + +As discussed in *note A Word About Names and Objects: f51, shared data +can have possibly surprising effects with involving *note mutable: ebe. +objects such as lists and dictionaries. For example, the `tricks' list +in the following code should not be used as a class variable because +just a single list would be shared by all `Dog' instances: + + class Dog: + + tricks = [] # mistaken use of a class variable + + def __init__(self, name): + self.name = name + + def add_trick(self, trick): + self.tricks.append(trick) + + >>> d = Dog('Fido') + >>> e = Dog('Buddy') + >>> d.add_trick('roll over') + >>> e.add_trick('play dead') + >>> d.tricks # unexpectedly shared by all dogs + ['roll over', 'play dead'] + +Correct design of the class should use an instance variable instead: + + class Dog: + + def __init__(self, name): + self.name = name + self.tricks = [] # creates a new empty list for each dog + + def add_trick(self, trick): + self.tricks.append(trick) + + >>> d = Dog('Fido') + >>> e = Dog('Buddy') + >>> d.add_trick('roll over') + >>> e.add_trick('play dead') + >>> d.tricks + ['roll over'] + >>> e.tricks + ['play dead'] + + +File: python.info, Node: Random Remarks, Next: Inheritance, Prev: A First Look at Classes, Up: Classes + +2.9.4 Random Remarks +-------------------- + +If the same attribute name occurs in both an instance and in a class, +then attribute lookup prioritizes the instance: + + >>> class Warehouse: + purpose = 'storage' + region = 'west' + + >>> w1 = Warehouse() + >>> print(w1.purpose, w1.region) + storage west + >>> w2 = Warehouse() + >>> w2.region = 'east' + >>> print(w2.purpose, w2.region) + storage east + +Data attributes may be referenced by methods as well as by ordinary +users (“clients”) of an object. In other words, classes are not usable +to implement pure abstract data types. In fact, nothing in Python makes +it possible to enforce data hiding — it is all based upon convention. +(On the other hand, the Python implementation, written in C, can +completely hide implementation details and control access to an object +if necessary; this can be used by extensions to Python written in C.) + +Clients should use data attributes with care — clients may mess up +invariants maintained by the methods by stamping on their data +attributes. Note that clients may add data attributes of their own to +an instance object without affecting the validity of the methods, as +long as name conflicts are avoided — again, a naming convention can save +a lot of headaches here. + +There is no shorthand for referencing data attributes (or other +methods!) from within methods. I find that this actually increases the +readability of methods: there is no chance of confusing local variables +and instance variables when glancing through a method. + +Often, the first argument of a method is called ‘self’. This is nothing +more than a convention: the name ‘self’ has absolutely no special +meaning to Python. Note, however, that by not following the convention +your code may be less readable to other Python programmers, and it is +also conceivable that a `class browser' program might be written that +relies upon such a convention. + +Any function object that is a class attribute defines a method for +instances of that class. It is not necessary that the function +definition is textually enclosed in the class definition: assigning a +function object to a local variable in the class is also ok. For +example: + + # Function defined outside the class + def f1(self, x, y): + return min(x, x+y) + + class C: + f = f1 + + def g(self): + return 'hello world' + + h = g + +Now ‘f’, ‘g’ and ‘h’ are all attributes of class ‘C’ that refer to +function objects, and consequently they are all methods of instances of +‘C’ — ‘h’ being exactly equivalent to ‘g’. Note that this practice +usually only serves to confuse the reader of a program. + +Methods may call other methods by using method attributes of the ‘self’ +argument: + + class Bag: + def __init__(self): + self.data = [] + + def add(self, x): + self.data.append(x) + + def addtwice(self, x): + self.add(x) + self.add(x) + +Methods may reference global names in the same way as ordinary +functions. The global scope associated with a method is the module +containing its definition. (A class is never used as a global scope.) +While one rarely encounters a good reason for using global data in a +method, there are many legitimate uses of the global scope: for one +thing, functions and modules imported into the global scope can be used +by methods, as well as functions and classes defined in it. Usually, +the class containing the method is itself defined in this global scope, +and in the next section we’ll find some good reasons why a method would +want to reference its own class. + +Each value is an object, and therefore has a `class' (also called its +`type'). It is stored as ‘object.__class__’. + + +File: python.info, Node: Inheritance, Next: Private Variables, Prev: Random Remarks, Up: Classes + +2.9.5 Inheritance +----------------- + +Of course, a language feature would not be worthy of the name “class” +without supporting inheritance. The syntax for a derived class +definition looks like this: + + class DerivedClassName(BaseClassName): + + . + . + . + + +The name ‘BaseClassName’ must be defined in a scope containing the +derived class definition. In place of a base class name, other +arbitrary expressions are also allowed. This can be useful, for +example, when the base class is defined in another module: + + class DerivedClassName(modname.BaseClassName): + +Execution of a derived class definition proceeds the same as for a base +class. When the class object is constructed, the base class is +remembered. This is used for resolving attribute references: if a +requested attribute is not found in the class, the search proceeds to +look in the base class. This rule is applied recursively if the base +class itself is derived from some other class. + +There’s nothing special about instantiation of derived classes: +‘DerivedClassName()’ creates a new instance of the class. Method +references are resolved as follows: the corresponding class attribute is +searched, descending down the chain of base classes if necessary, and +the method reference is valid if this yields a function object. + +Derived classes may override methods of their base classes. Because +methods have no special privileges when calling other methods of the +same object, a method of a base class that calls another method defined +in the same base class may end up calling a method of a derived class +that overrides it. (For C++ programmers: all methods in Python are +effectively ‘virtual’.) + +An overriding method in a derived class may in fact want to extend +rather than simply replace the base class method of the same name. +There is a simple way to call the base class method directly: just call +‘BaseClassName.methodname(self, arguments)’. This is occasionally +useful to clients as well. (Note that this only works if the base class +is accessible as ‘BaseClassName’ in the global scope.) + +Python has two built-in functions that work with inheritance: + + * Use *note isinstance(): 44f. to check an instance’s type: + ‘isinstance(obj, int)’ will be ‘True’ only if ‘obj.__class__’ is + *note int: 184. or some class derived from *note int: 184. + + * Use *note issubclass(): 450. to check class inheritance: + ‘issubclass(bool, int)’ is ‘True’ since *note bool: 183. is a + subclass of *note int: 184. However, ‘issubclass(float, int)’ is + ‘False’ since *note float: 187. is not a subclass of *note int: + 184. + +* Menu: + +* Multiple Inheritance:: + + +File: python.info, Node: Multiple Inheritance, Up: Inheritance + +2.9.5.1 Multiple Inheritance +............................ + +Python supports a form of multiple inheritance as well. A class +definition with multiple base classes looks like this: + + class DerivedClassName(Base1, Base2, Base3): + + . + . + . + + +For most purposes, in the simplest cases, you can think of the search +for attributes inherited from a parent class as depth-first, +left-to-right, not searching twice in the same class where there is an +overlap in the hierarchy. Thus, if an attribute is not found in +‘DerivedClassName’, it is searched for in ‘Base1’, then (recursively) in +the base classes of ‘Base1’, and if it was not found there, it was +searched for in ‘Base2’, and so on. + +In fact, it is slightly more complex than that; the method resolution +order changes dynamically to support cooperative calls to *note super(): +4e2. This approach is known in some other multiple-inheritance +languages as call-next-method and is more powerful than the super call +found in single-inheritance languages. + +Dynamic ordering is necessary because all cases of multiple inheritance +exhibit one or more diamond relationships (where at least one of the +parent classes can be accessed through multiple paths from the +bottommost class). For example, all classes inherit from *note object: +2b0, so any case of multiple inheritance provides more than one path to +reach *note object: 2b0. To keep the base classes from being accessed +more than once, the dynamic algorithm linearizes the search order in a +way that preserves the left-to-right ordering specified in each class, +that calls each parent only once, and that is monotonic (meaning that a +class can be subclassed without affecting the precedence order of its +parents). Taken together, these properties make it possible to design +reliable and extensible classes with multiple inheritance. For more +detail, see ‘https://www.python.org/download/releases/2.3/mro/’. + + +File: python.info, Node: Private Variables, Next: Odds and Ends, Prev: Inheritance, Up: Classes + +2.9.6 Private Variables +----------------------- + +“Private” instance variables that cannot be accessed except from inside +an object don’t exist in Python. However, there is a convention that is +followed by most Python code: a name prefixed with an underscore (e.g. +‘_spam’) should be treated as a non-public part of the API (whether it +is a function, a method or a data member). It should be considered an +implementation detail and subject to change without notice. + +Since there is a valid use-case for class-private members (namely to +avoid name clashes of names with names defined by subclasses), there is +limited support for such a mechanism, called `name mangling'. Any +identifier of the form ‘__spam’ (at least two leading underscores, at +most one trailing underscore) is textually replaced with +‘_classname__spam’, where ‘classname’ is the current class name with +leading underscore(s) stripped. This mangling is done without regard to +the syntactic position of the identifier, as long as it occurs within +the definition of a class. + +Name mangling is helpful for letting subclasses override methods without +breaking intraclass method calls. For example: + + class Mapping: + def __init__(self, iterable): + self.items_list = [] + self.__update(iterable) + + def update(self, iterable): + for item in iterable: + self.items_list.append(item) + + __update = update # private copy of original update() method + + class MappingSubclass(Mapping): + + def update(self, keys, values): + # provides new signature for update() + # but does not break __init__() + for item in zip(keys, values): + self.items_list.append(item) + +The above example would work even if ‘MappingSubclass’ were to introduce +a ‘__update’ identifier since it is replaced with ‘_Mapping__update’ in +the ‘Mapping’ class and ‘_MappingSubclass__update’ in the +‘MappingSubclass’ class respectively. + +Note that the mangling rules are designed mostly to avoid accidents; it +still is possible to access or modify a variable that is considered +private. This can even be useful in special circumstances, such as in +the debugger. + +Notice that code passed to ‘exec()’ or ‘eval()’ does not consider the +classname of the invoking class to be the current class; this is similar +to the effect of the ‘global’ statement, the effect of which is likewise +restricted to code that is byte-compiled together. The same restriction +applies to ‘getattr()’, ‘setattr()’ and ‘delattr()’, as well as when +referencing ‘__dict__’ directly. + + +File: python.info, Node: Odds and Ends, Next: Iterators, Prev: Private Variables, Up: Classes + +2.9.7 Odds and Ends +------------------- + +Sometimes it is useful to have a data type similar to the Pascal +“record” or C “struct”, bundling together a few named data items. An +empty class definition will do nicely: + + class Employee: + pass + + john = Employee() # Create an empty employee record + + # Fill the fields of the record + john.name = 'John Doe' + john.dept = 'computer lab' + john.salary = 1000 + +A piece of Python code that expects a particular abstract data type can +often be passed a class that emulates the methods of that data type +instead. For instance, if you have a function that formats some data +from a file object, you can define a class with methods ‘read()’ and +‘readline()’ that get the data from a string buffer instead, and pass it +as an argument. + +Instance method objects have attributes, too: ‘m.__self__’ is the +instance object with the method ‘m()’, and ‘m.__func__’ is the function +object corresponding to the method. + + +File: python.info, Node: Iterators, Next: Generators, Prev: Odds and Ends, Up: Classes + +2.9.8 Iterators +--------------- + +By now you have probably noticed that most container objects can be +looped over using a *note for: c30. statement: + + for element in [1, 2, 3]: + print(element) + for element in (1, 2, 3): + print(element) + for key in {'one':1, 'two':2}: + print(key) + for char in "123": + print(char) + for line in open("myfile.txt"): + print(line, end='') + +This style of access is clear, concise, and convenient. The use of +iterators pervades and unifies Python. Behind the scenes, the *note +for: c30. statement calls *note iter(): d27. on the container object. +The function returns an iterator object that defines the method *note +__next__(): c64. which accesses elements in the container one at a time. +When there are no more elements, *note __next__(): c64. raises a *note +StopIteration: 486. exception which tells the ‘for’ loop to terminate. +You can call the *note __next__(): c64. method using the *note next(): +682. built-in function; this example shows how it all works: + + >>> s = 'abc' + >>> it = iter(s) + >>> it + + >>> next(it) + 'a' + >>> next(it) + 'b' + >>> next(it) + 'c' + >>> next(it) + Traceback (most recent call last): + File "", line 1, in + next(it) + StopIteration + +Having seen the mechanics behind the iterator protocol, it is easy to +add iterator behavior to your classes. Define an *note __iter__(): 50f. +method which returns an object with a *note __next__(): c64. method. If +the class defines ‘__next__()’, then *note __iter__(): 50f. can just +return ‘self’: + + class Reverse: + """Iterator for looping over a sequence backwards.""" + def __init__(self, data): + self.data = data + self.index = len(data) + + def __iter__(self): + return self + + def __next__(self): + if self.index == 0: + raise StopIteration + self.index = self.index - 1 + return self.data[self.index] + + >>> rev = Reverse('spam') + >>> iter(rev) + <__main__.Reverse object at 0x00A1DB50> + >>> for char in rev: + ... print(char) + ... + m + a + p + s + + +File: python.info, Node: Generators, Next: Generator Expressions, Prev: Iterators, Up: Classes + +2.9.9 Generators +---------------- + +*note Generators: 9a2. are a simple and powerful tool for creating +iterators. They are written like regular functions but use the *note +yield: 18f. statement whenever they want to return data. Each time +*note next(): 682. is called on it, the generator resumes where it left +off (it remembers all the data values and which statement was last +executed). An example shows that generators can be trivially easy to +create: + + def reverse(data): + for index in range(len(data)-1, -1, -1): + yield data[index] + + >>> for char in reverse('golf'): + ... print(char) + ... + f + l + o + g + +Anything that can be done with generators can also be done with +class-based iterators as described in the previous section. What makes +generators so compact is that the *note __iter__(): 50f. and *note +__next__(): f6f. methods are created automatically. + +Another key feature is that the local variables and execution state are +automatically saved between calls. This made the function easier to +write and much more clear than an approach using instance variables like +‘self.index’ and ‘self.data’. + +In addition to automatic method creation and saving program state, when +generators terminate, they automatically raise *note StopIteration: 486. +In combination, these features make it easy to create iterators with no +more effort than writing a regular function. + + +File: python.info, Node: Generator Expressions, Prev: Generators, Up: Classes + +2.9.10 Generator Expressions +---------------------------- + +Some simple generators can be coded succinctly as expressions using a +syntax similar to list comprehensions but with parentheses instead of +square brackets. These expressions are designed for situations where +the generator is used right away by an enclosing function. Generator +expressions are more compact but less versatile than full generator +definitions and tend to be more memory friendly than equivalent list +comprehensions. + +Examples: + + >>> sum(i*i for i in range(10)) # sum of squares + 285 + + >>> xvec = [10, 20, 30] + >>> yvec = [7, 5, 3] + >>> sum(x*y for x,y in zip(xvec, yvec)) # dot product + 260 + + >>> unique_words = set(word for line in page for word in line.split()) + + >>> valedictorian = max((student.gpa, student.name) for student in graduates) + + >>> data = 'golf' + >>> list(data[i] for i in range(len(data)-1, -1, -1)) + ['f', 'l', 'o', 'g'] + + +File: python.info, Node: Brief Tour of the Standard Library, Next: Brief Tour of the Standard Library — Part II, Prev: Classes, Up: The Python Tutorial + +2.10 Brief Tour of the Standard Library +======================================= + +* Menu: + +* Operating System Interface:: +* File Wildcards:: +* Command Line Arguments:: +* Error Output Redirection and Program Termination:: +* String Pattern Matching:: +* Mathematics:: +* Internet Access:: +* Dates and Times:: +* Data Compression:: +* Performance Measurement:: +* Quality Control:: +* Batteries Included:: + + +File: python.info, Node: Operating System Interface, Next: File Wildcards, Up: Brief Tour of the Standard Library + +2.10.1 Operating System Interface +--------------------------------- + +The *note os: c4. module provides dozens of functions for interacting +with the operating system: + + >>> import os + >>> os.getcwd() # Return the current working directory + 'C:\\Python38' + >>> os.chdir('/server/accesslogs') # Change current working directory + >>> os.system('mkdir today') # Run the command mkdir in the system shell + 0 + +Be sure to use the ‘import os’ style instead of ‘from os import *’. +This will keep *note os.open(): 665. from shadowing the built-in *note +open(): 4f0. function which operates much differently. + +The built-in *note dir(): d33. and *note help(): 572. functions are +useful as interactive aids for working with large modules like *note os: +c4.: + + >>> import os + >>> dir(os) + + >>> help(os) + + +For daily file and directory management tasks, the *note shutil: e9. +module provides a higher level interface that is easier to use: + + >>> import shutil + >>> shutil.copyfile('data.db', 'archive.db') + 'archive.db' + >>> shutil.move('/build/executables', 'installdir') + 'installdir' + + +File: python.info, Node: File Wildcards, Next: Command Line Arguments, Prev: Operating System Interface, Up: Brief Tour of the Standard Library + +2.10.2 File Wildcards +--------------------- + +The *note glob: 8a. module provides a function for making file lists +from directory wildcard searches: + + >>> import glob + >>> glob.glob('*.py') + ['primes.py', 'random.py', 'quote.py'] + + +File: python.info, Node: Command Line Arguments, Next: Error Output Redirection and Program Termination, Prev: File Wildcards, Up: Brief Tour of the Standard Library + +2.10.3 Command Line Arguments +----------------------------- + +Common utility scripts often need to process command line arguments. +These arguments are stored in the *note sys: fd. module’s `argv' +attribute as a list. For instance the following output results from +running ‘python demo.py one two three’ at the command line: + + >>> import sys + >>> print(sys.argv) + ['demo.py', 'one', 'two', 'three'] + +The *note argparse: 6. module provides a more sophisticated mechanism to +process command line arguments. The following script extracts one or +more filenames and an optional number of lines to be displayed: + + import argparse + + parser = argparse.ArgumentParser(prog = 'top', + description = 'Show top lines from each file') + parser.add_argument('filenames', nargs='+') + parser.add_argument('-l', '--lines', type=int, default=10) + args = parser.parse_args() + print(args) + +When run at the command line with ‘python top.py --lines=5 alpha.txt +beta.txt’, the script sets ‘args.lines’ to ‘5’ and ‘args.filenames’ to +‘['alpha.txt', 'beta.txt']’. + + +File: python.info, Node: Error Output Redirection and Program Termination, Next: String Pattern Matching, Prev: Command Line Arguments, Up: Brief Tour of the Standard Library + +2.10.4 Error Output Redirection and Program Termination +------------------------------------------------------- + +The *note sys: fd. module also has attributes for `stdin', `stdout', and +`stderr'. The latter is useful for emitting warnings and error messages +to make them visible even when `stdout' has been redirected: + + >>> sys.stderr.write('Warning, log file not found starting a new one\n') + Warning, log file not found starting a new one + +The most direct way to terminate a script is to use ‘sys.exit()’. + + +File: python.info, Node: String Pattern Matching, Next: Mathematics, Prev: Error Output Redirection and Program Termination, Up: Brief Tour of the Standard Library + +2.10.5 String Pattern Matching +------------------------------ + +The *note re: dd. module provides regular expression tools for advanced +string processing. For complex matching and manipulation, regular +expressions offer succinct, optimized solutions: + + >>> import re + >>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest') + ['foot', 'fell', 'fastest'] + >>> re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat') + 'cat in the hat' + +When only simple capabilities are needed, string methods are preferred +because they are easier to read and debug: + + >>> 'tea for too'.replace('too', 'two') + 'tea for two' + + +File: python.info, Node: Mathematics, Next: Internet Access, Prev: String Pattern Matching, Up: Brief Tour of the Standard Library + +2.10.6 Mathematics +------------------ + +The *note math: b1. module gives access to the underlying C library +functions for floating point math: + + >>> import math + >>> math.cos(math.pi / 4) + 0.70710678118654757 + >>> math.log(1024, 2) + 10.0 + +The *note random: dc. module provides tools for making random +selections: + + >>> import random + >>> random.choice(['apple', 'pear', 'banana']) + 'apple' + >>> random.sample(range(100), 10) # sampling without replacement + [30, 83, 16, 4, 8, 81, 41, 50, 18, 33] + >>> random.random() # random float + 0.17970987693706186 + >>> random.randrange(6) # random integer chosen from range(6) + 4 + +The *note statistics: f5. module calculates basic statistical properties +(the mean, median, variance, etc.) of numeric data: + + >>> import statistics + >>> data = [2.75, 1.75, 1.25, 0.25, 0.5, 1.25, 3.5] + >>> statistics.mean(data) + 1.6071428571428572 + >>> statistics.median(data) + 1.25 + >>> statistics.variance(data) + 1.3720238095238095 + +The SciPy project <‘https://scipy.org’> has many other modules for +numerical computations. + + +File: python.info, Node: Internet Access, Next: Dates and Times, Prev: Mathematics, Up: Brief Tour of the Standard Library + +2.10.7 Internet Access +---------------------- + +There are a number of modules for accessing the internet and processing +internet protocols. Two of the simplest are *note urllib.request: 120. +for retrieving data from URLs and *note smtplib: ed. for sending mail: + + >>> from urllib.request import urlopen + >>> with urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') as response: + ... for line in response: + ... line = line.decode('utf-8') # Decoding the binary data to text. + ... if 'EST' in line or 'EDT' in line: # look for Eastern Time + ... print(line) + +
Nov. 25, 09:43:32 PM EST + + >>> import smtplib + >>> server = smtplib.SMTP('localhost') + >>> server.sendmail('soothsayer@example.org', 'jcaesar@example.org', + ... """To: jcaesar@example.org + ... From: soothsayer@example.org + ... + ... Beware the Ides of March. + ... """) + >>> server.quit() + +(Note that the second example needs a mailserver running on localhost.) + + +File: python.info, Node: Dates and Times, Next: Data Compression, Prev: Internet Access, Up: Brief Tour of the Standard Library + +2.10.8 Dates and Times +---------------------- + +The *note datetime: 31. module supplies classes for manipulating dates +and times in both simple and complex ways. While date and time +arithmetic is supported, the focus of the implementation is on efficient +member extraction for output formatting and manipulation. The module +also supports objects that are timezone aware. + + >>> # dates are easily constructed and formatted + >>> from datetime import date + >>> now = date.today() + >>> now + datetime.date(2003, 12, 2) + >>> now.strftime("%m-%d-%y. %d %b %Y is a %A on the %d day of %B.") + '12-02-03. 02 Dec 2003 is a Tuesday on the 02 day of December.' + + >>> # dates support calendar arithmetic + >>> birthday = date(1964, 7, 31) + >>> age = now - birthday + >>> age.days + 14368 + + +File: python.info, Node: Data Compression, Next: Performance Measurement, Prev: Dates and Times, Up: Brief Tour of the Standard Library + +2.10.9 Data Compression +----------------------- + +Common data archiving and compression formats are directly supported by +modules including: *note zlib: 144, *note gzip: 8c, *note bz2: 14, *note +lzma: ad, *note zipfile: 142. and *note tarfile: 101. + + >>> import zlib + >>> s = b'witch which has which witches wrist watch' + >>> len(s) + 41 + >>> t = zlib.compress(s) + >>> len(t) + 37 + >>> zlib.decompress(t) + b'witch which has which witches wrist watch' + >>> zlib.crc32(s) + 226805979 + + +File: python.info, Node: Performance Measurement, Next: Quality Control, Prev: Data Compression, Up: Brief Tour of the Standard Library + +2.10.10 Performance Measurement +------------------------------- + +Some Python users develop a deep interest in knowing the relative +performance of different approaches to the same problem. Python +provides a measurement tool that answers those questions immediately. + +For example, it may be tempting to use the tuple packing and unpacking +feature instead of the traditional approach to swapping arguments. The +*note timeit: 10b. module quickly demonstrates a modest performance +advantage: + + >>> from timeit import Timer + >>> Timer('t=a; a=b; b=t', 'a=1; b=2').timeit() + 0.57535828626024577 + >>> Timer('a,b = b,a', 'a=1; b=2').timeit() + 0.54962537085770791 + +In contrast to *note timeit: 10b.’s fine level of granularity, the *note +profile: d3. and *note pstats: d4. modules provide tools for identifying +time critical sections in larger blocks of code. + + +File: python.info, Node: Quality Control, Next: Batteries Included, Prev: Performance Measurement, Up: Brief Tour of the Standard Library + +2.10.11 Quality Control +----------------------- + +One approach for developing high quality software is to write tests for +each function as it is developed and to run those tests frequently +during the development process. + +The *note doctest: 67. module provides a tool for scanning a module and +validating tests embedded in a program’s docstrings. Test construction +is as simple as cutting-and-pasting a typical call along with its +results into the docstring. This improves the documentation by +providing the user with an example and it allows the doctest module to +make sure the code remains true to the documentation: + + def average(values): + """Computes the arithmetic mean of a list of numbers. + + >>> print(average([20, 30, 70])) + 40.0 + """ + return sum(values) / len(values) + + import doctest + doctest.testmod() # automatically validate the embedded tests + +The *note unittest: 11b. module is not as effortless as the *note +doctest: 67. module, but it allows a more comprehensive set of tests to +be maintained in a separate file: + + import unittest + + class TestStatisticalFunctions(unittest.TestCase): + + def test_average(self): + self.assertEqual(average([20, 30, 70]), 40.0) + self.assertEqual(round(average([1, 5, 7]), 1), 4.3) + with self.assertRaises(ZeroDivisionError): + average([]) + with self.assertRaises(TypeError): + average(20, 30, 70) + + unittest.main() # Calling from the command line invokes all tests + + +File: python.info, Node: Batteries Included, Prev: Quality Control, Up: Brief Tour of the Standard Library + +2.10.12 Batteries Included +-------------------------- + +Python has a “batteries included” philosophy. This is best seen through +the sophisticated and robust capabilities of its larger packages. For +example: + + * The *note xmlrpc.client: 13f. and *note xmlrpc.server: 140. modules + make implementing remote procedure calls into an almost trivial + task. Despite the modules names, no direct knowledge or handling + of XML is needed. + + * The *note email: 69. package is a library for managing email + messages, including MIME and other RFC 2822(1)-based message + documents. Unlike *note smtplib: ed. and *note poplib: d0. which + actually send and receive messages, the email package has a + complete toolset for building or decoding complex message + structures (including attachments) and for implementing internet + encoding and header protocols. + + * The *note json: a4. package provides robust support for parsing + this popular data interchange format. The *note csv: 2a. module + supports direct reading and writing of files in Comma-Separated + Value format, commonly supported by databases and spreadsheets. + XML processing is supported by the *note xml.etree.ElementTree: + 137, *note xml.dom: 134. and *note xml.sax: 13b. packages. + Together, these modules and packages greatly simplify data + interchange between Python applications and other tools. + + * The *note sqlite3: f2. module is a wrapper for the SQLite database + library, providing a persistent database that can be updated and + accessed using slightly nonstandard SQL syntax. + + * Internationalization is supported by a number of modules including + *note gettext: 89, *note locale: a9, and the *note codecs: 1c. + package. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2822.html + + +File: python.info, Node: Brief Tour of the Standard Library — Part II, Next: Virtual Environments and Packages, Prev: Brief Tour of the Standard Library, Up: The Python Tutorial + +2.11 Brief Tour of the Standard Library — Part II +================================================= + +This second tour covers more advanced modules that support professional +programming needs. These modules rarely occur in small scripts. + +* Menu: + +* Output Formatting:: +* Templating:: +* Working with Binary Data Record Layouts:: +* Multi-threading: Multi-threading<2>. +* Logging:: +* Weak References:: +* Tools for Working with Lists:: +* Decimal Floating Point Arithmetic:: + + +File: python.info, Node: Output Formatting, Next: Templating, Up: Brief Tour of the Standard Library — Part II + +2.11.1 Output Formatting +------------------------ + +The *note reprlib: df. module provides a version of *note repr(): 7cc. +customized for abbreviated displays of large or deeply nested +containers: + + >>> import reprlib + >>> reprlib.repr(set('supercalifragilisticexpialidocious')) + "{'a', 'c', 'd', 'e', 'f', 'g', ...}" + +The *note pprint: d2. module offers more sophisticated control over +printing both built-in and user defined objects in a way that is +readable by the interpreter. When the result is longer than one line, +the “pretty printer” adds line breaks and indentation to more clearly +reveal data structure: + + >>> import pprint + >>> t = [[[['black', 'cyan'], 'white', ['green', 'red']], [['magenta', + ... 'yellow'], 'blue']]] + ... + >>> pprint.pprint(t, width=30) + [[[['black', 'cyan'], + 'white', + ['green', 'red']], + [['magenta', 'yellow'], + 'blue']]] + +The *note textwrap: 108. module formats paragraphs of text to fit a +given screen width: + + >>> import textwrap + >>> doc = """The wrap() method is just like fill() except that it returns + ... a list of strings instead of one big string with newlines to separate + ... the wrapped lines.""" + ... + >>> print(textwrap.fill(doc, width=40)) + The wrap() method is just like fill() + except that it returns a list of strings + instead of one big string with newlines + to separate the wrapped lines. + +The *note locale: a9. module accesses a database of culture specific +data formats. The grouping attribute of locale’s format function +provides a direct way of formatting numbers with group separators: + + >>> import locale + >>> locale.setlocale(locale.LC_ALL, 'English_United States.1252') + 'English_United States.1252' + >>> conv = locale.localeconv() # get a mapping of conventions + >>> x = 1234567.8 + >>> locale.format("%d", x, grouping=True) + '1,234,567' + >>> locale.format_string("%s%.*f", (conv['currency_symbol'], + ... conv['frac_digits'], x), grouping=True) + '$1,234,567.80' + + +File: python.info, Node: Templating, Next: Working with Binary Data Record Layouts, Prev: Output Formatting, Up: Brief Tour of the Standard Library — Part II + +2.11.2 Templating +----------------- + +The *note string: f6. module includes a versatile *note Template: 3eb. +class with a simplified syntax suitable for editing by end-users. This +allows users to customize their applications without having to alter the +application. + +The format uses placeholder names formed by ‘$’ with valid Python +identifiers (alphanumeric characters and underscores). Surrounding the +placeholder with braces allows it to be followed by more alphanumeric +letters with no intervening spaces. Writing ‘$$’ creates a single +escaped ‘$’: + + >>> from string import Template + >>> t = Template('${village}folk send $$10 to $cause.') + >>> t.substitute(village='Nottingham', cause='the ditch fund') + 'Nottinghamfolk send $10 to the ditch fund.' + +The *note substitute(): f94. method raises a *note KeyError: 2c7. when a +placeholder is not supplied in a dictionary or a keyword argument. For +mail-merge style applications, user supplied data may be incomplete and +the *note safe_substitute(): f95. method may be more appropriate — it +will leave placeholders unchanged if data is missing: + + >>> t = Template('Return the $item to $owner.') + >>> d = dict(item='unladen swallow') + >>> t.substitute(d) + Traceback (most recent call last): + ... + KeyError: 'owner' + >>> t.safe_substitute(d) + 'Return the unladen swallow to $owner.' + +Template subclasses can specify a custom delimiter. For example, a +batch renaming utility for a photo browser may elect to use percent +signs for placeholders such as the current date, image sequence number, +or file format: + + >>> import time, os.path + >>> photofiles = ['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg'] + >>> class BatchRename(Template): + ... delimiter = '%' + >>> fmt = input('Enter rename style (%d-date %n-seqnum %f-format): ') + Enter rename style (%d-date %n-seqnum %f-format): Ashley_%n%f + + >>> t = BatchRename(fmt) + >>> date = time.strftime('%d%b%y') + >>> for i, filename in enumerate(photofiles): + ... base, ext = os.path.splitext(filename) + ... newname = t.substitute(d=date, n=i, f=ext) + ... print('{0} --> {1}'.format(filename, newname)) + + img_1074.jpg --> Ashley_0.jpg + img_1076.jpg --> Ashley_1.jpg + img_1077.jpg --> Ashley_2.jpg + +Another application for templating is separating program logic from the +details of multiple output formats. This makes it possible to +substitute custom templates for XML files, plain text reports, and HTML +web reports. + + +File: python.info, Node: Working with Binary Data Record Layouts, Next: Multi-threading<2>, Prev: Templating, Up: Brief Tour of the Standard Library — Part II + +2.11.3 Working with Binary Data Record Layouts +---------------------------------------------- + +The *note struct: f8. module provides *note pack(): c0b. and *note +unpack(): f98. functions for working with variable length binary record +formats. The following example shows how to loop through header +information in a ZIP file without using the *note zipfile: 142. module. +Pack codes ‘"H"’ and ‘"I"’ represent two and four byte unsigned numbers +respectively. The ‘"<"’ indicates that they are standard size and in +little-endian byte order: + + import struct + + with open('myfile.zip', 'rb') as f: + data = f.read() + + start = 0 + for i in range(3): # show the first 3 file headers + start += 14 + fields = struct.unpack(', Next: Logging, Prev: Working with Binary Data Record Layouts, Up: Brief Tour of the Standard Library — Part II + +2.11.4 Multi-threading +---------------------- + +Threading is a technique for decoupling tasks which are not sequentially +dependent. Threads can be used to improve the responsiveness of +applications that accept user input while other tasks run in the +background. A related use case is running I/O in parallel with +computations in another thread. + +The following code shows how the high level *note threading: 109. module +can run tasks in background while the main program continues to run: + + import threading, zipfile + + class AsyncZip(threading.Thread): + def __init__(self, infile, outfile): + threading.Thread.__init__(self) + self.infile = infile + self.outfile = outfile + + def run(self): + f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED) + f.write(self.infile) + f.close() + print('Finished background zip of:', self.infile) + + background = AsyncZip('mydata.txt', 'myarchive.zip') + background.start() + print('The main program continues to run in foreground.') + + background.join() # Wait for the background task to finish + print('Main program waited until background was done.') + +The principal challenge of multi-threaded applications is coordinating +threads that share data or other resources. To that end, the threading +module provides a number of synchronization primitives including locks, +events, condition variables, and semaphores. + +While those tools are powerful, minor design errors can result in +problems that are difficult to reproduce. So, the preferred approach to +task coordination is to concentrate all access to a resource in a single +thread and then use the *note queue: da. module to feed that thread with +requests from other threads. Applications using *note Queue: d18. +objects for inter-thread communication and coordination are easier to +design, more readable, and more reliable. + + +File: python.info, Node: Logging, Next: Weak References, Prev: Multi-threading<2>, Up: Brief Tour of the Standard Library — Part II + +2.11.5 Logging +-------------- + +The *note logging: aa. module offers a full featured and flexible +logging system. At its simplest, log messages are sent to a file or to +‘sys.stderr’: + + import logging + logging.debug('Debugging information') + logging.info('Informational message') + logging.warning('Warning:config file %s not found', 'server.conf') + logging.error('Error occurred') + logging.critical('Critical error -- shutting down') + +This produces the following output: + + WARNING:root:Warning:config file server.conf not found + ERROR:root:Error occurred + CRITICAL:root:Critical error -- shutting down + +By default, informational and debugging messages are suppressed and the +output is sent to standard error. Other output options include routing +messages through email, datagrams, sockets, or to an HTTP Server. New +filters can select different routing based on message priority: ‘DEBUG’, +‘INFO’, ‘WARNING’, ‘ERROR’, and ‘CRITICAL’. + +The logging system can be configured directly from Python or can be +loaded from a user editable configuration file for customized logging +without altering the application. + + +File: python.info, Node: Weak References, Next: Tools for Working with Lists, Prev: Logging, Up: Brief Tour of the Standard Library — Part II + +2.11.6 Weak References +---------------------- + +Python does automatic memory management (reference counting for most +objects and *note garbage collection: f9f. to eliminate cycles). The +memory is freed shortly after the last reference to it has been +eliminated. + +This approach works fine for most applications but occasionally there is +a need to track objects only as long as they are being used by something +else. Unfortunately, just tracking them creates a reference that makes +them permanent. The *note weakref: 128. module provides tools for +tracking objects without creating a reference. When the object is no +longer needed, it is automatically removed from a weakref table and a +callback is triggered for weakref objects. Typical applications include +caching objects that are expensive to create: + + >>> import weakref, gc + >>> class A: + ... def __init__(self, value): + ... self.value = value + ... def __repr__(self): + ... return str(self.value) + ... + >>> a = A(10) # create a reference + >>> d = weakref.WeakValueDictionary() + >>> d['primary'] = a # does not create a reference + >>> d['primary'] # fetch the object if it is still alive + 10 + >>> del a # remove the one reference + >>> gc.collect() # run garbage collection right away + 0 + >>> d['primary'] # entry was automatically removed + Traceback (most recent call last): + File "", line 1, in + d['primary'] # entry was automatically removed + File "C:/python38/lib/weakref.py", line 46, in __getitem__ + o = self.data[key]() + KeyError: 'primary' + + +File: python.info, Node: Tools for Working with Lists, Next: Decimal Floating Point Arithmetic, Prev: Weak References, Up: Brief Tour of the Standard Library — Part II + +2.11.7 Tools for Working with Lists +----------------------------------- + +Many data structure needs can be met with the built-in list type. +However, sometimes there is a need for alternative implementations with +different performance trade-offs. + +The *note array: 7. module provides an *note array(): 451. object that +is like a list that stores only homogeneous data and stores it more +compactly. The following example shows an array of numbers stored as +two byte unsigned binary numbers (typecode ‘"H"’) rather than the usual +16 bytes per entry for regular lists of Python int objects: + + >>> from array import array + >>> a = array('H', [4000, 10, 700, 22222]) + >>> sum(a) + 26932 + >>> a[1:3] + array('H', [10, 700]) + +The *note collections: 1e. module provides a *note deque(): 531. object +that is like a list with faster appends and pops from the left side but +slower lookups in the middle. These objects are well suited for +implementing queues and breadth first tree searches: + + >>> from collections import deque + >>> d = deque(["task1", "task2", "task3"]) + >>> d.append("task4") + >>> print("Handling", d.popleft()) + Handling task1 + + unsearched = deque([starting_node]) + def breadth_first_search(unsearched): + node = unsearched.popleft() + for m in gen_moves(node): + if is_goal(m): + return m + unsearched.append(m) + +In addition to alternative list implementations, the library also offers +other tools such as the *note bisect: 12. module with functions for +manipulating sorted lists: + + >>> import bisect + >>> scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500, 'python')] + >>> bisect.insort(scores, (300, 'ruby')) + >>> scores + [(100, 'perl'), (200, 'tcl'), (300, 'ruby'), (400, 'lua'), (500, 'python')] + +The *note heapq: 8e. module provides functions for implementing heaps +based on regular lists. The lowest valued entry is always kept at +position zero. This is useful for applications which repeatedly access +the smallest element but do not want to run a full list sort: + + >>> from heapq import heapify, heappop, heappush + >>> data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0] + >>> heapify(data) # rearrange the list into heap order + >>> heappush(data, -5) # add a new entry + >>> [heappop(data) for i in range(3)] # fetch the three smallest entries + [-5, 0, 1] + + +File: python.info, Node: Decimal Floating Point Arithmetic, Prev: Tools for Working with Lists, Up: Brief Tour of the Standard Library — Part II + +2.11.8 Decimal Floating Point Arithmetic +---------------------------------------- + +The *note decimal: 36. module offers a *note Decimal: 188. datatype for +decimal floating point arithmetic. Compared to the built-in *note +float: 187. implementation of binary floating point, the class is +especially helpful for + + * financial applications and other uses which require exact decimal + representation, + + * control over precision, + + * control over rounding to meet legal or regulatory requirements, + + * tracking of significant decimal places, or + + * applications where the user expects the results to match + calculations done by hand. + +For example, calculating a 5% tax on a 70 cent phone charge gives +different results in decimal floating point and binary floating point. +The difference becomes significant if the results are rounded to the +nearest cent: + + >>> from decimal import * + >>> round(Decimal('0.70') * Decimal('1.05'), 2) + Decimal('0.74') + >>> round(.70 * 1.05, 2) + 0.73 + +The *note Decimal: 188. result keeps a trailing zero, automatically +inferring four place significance from multiplicands with two place +significance. Decimal reproduces mathematics as done by hand and avoids +issues that can arise when binary floating point cannot exactly +represent decimal quantities. + +Exact representation enables the *note Decimal: 188. class to perform +modulo calculations and equality tests that are unsuitable for binary +floating point: + + >>> Decimal('1.00') % Decimal('.10') + Decimal('0.00') + >>> 1.00 % 0.10 + 0.09999999999999995 + + >>> sum([Decimal('0.1')]*10) == Decimal('1.0') + True + >>> sum([0.1]*10) == 1.0 + False + +The *note decimal: 36. module provides arithmetic with as much precision +as needed: + + >>> getcontext().prec = 36 + >>> Decimal(1) / Decimal(7) + Decimal('0.142857142857142857142857142857142857') + + +File: python.info, Node: Virtual Environments and Packages, Next: What Now?, Prev: Brief Tour of the Standard Library — Part II, Up: The Python Tutorial + +2.12 Virtual Environments and Packages +====================================== + +* Menu: + +* Introduction: Introduction<4>. +* Creating Virtual Environments:: +* Managing Packages with pip:: + + +File: python.info, Node: Introduction<4>, Next: Creating Virtual Environments, Up: Virtual Environments and Packages + +2.12.1 Introduction +------------------- + +Python applications will often use packages and modules that don’t come +as part of the standard library. Applications will sometimes need a +specific version of a library, because the application may require that +a particular bug has been fixed or the application may be written using +an obsolete version of the library’s interface. + +This means it may not be possible for one Python installation to meet +the requirements of every application. If application A needs version +1.0 of a particular module but application B needs version 2.0, then the +requirements are in conflict and installing either version 1.0 or 2.0 +will leave one application unable to run. + +The solution for this problem is to create a *note virtual environment: +fa8, a self-contained directory tree that contains a Python installation +for a particular version of Python, plus a number of additional +packages. + +Different applications can then use different virtual environments. To +resolve the earlier example of conflicting requirements, application A +can have its own virtual environment with version 1.0 installed while +application B has another virtual environment with version 2.0. If +application B requires a library be upgraded to version 3.0, this will +not affect application A’s environment. + + +File: python.info, Node: Creating Virtual Environments, Next: Managing Packages with pip, Prev: Introduction<4>, Up: Virtual Environments and Packages + +2.12.2 Creating Virtual Environments +------------------------------------ + +The module used to create and manage virtual environments is called +*note venv: 125. *note venv: 125. will usually install the most recent +version of Python that you have available. If you have multiple +versions of Python on your system, you can select a specific Python +version by running ‘python3’ or whichever version you want. + +To create a virtual environment, decide upon a directory where you want +to place it, and run the *note venv: 125. module as a script with the +directory path: + + python3 -m venv tutorial-env + +This will create the ‘tutorial-env’ directory if it doesn’t exist, and +also create directories inside it containing a copy of the Python +interpreter, the standard library, and various supporting files. + +A common directory location for a virtual environment is ‘.venv’. This +name keeps the directory typically hidden in your shell and thus out of +the way while giving it a name that explains why the directory exists. +It also prevents clashing with ‘.env’ environment variable definition +files that some tooling supports. + +Once you’ve created a virtual environment, you may activate it. + +On Windows, run: + + tutorial-env\Scripts\activate.bat + +On Unix or MacOS, run: + + source tutorial-env/bin/activate + +(This script is written for the bash shell. If you use the ‘csh’ or +‘fish’ shells, there are alternate ‘activate.csh’ and ‘activate.fish’ +scripts you should use instead.) + +Activating the virtual environment will change your shell’s prompt to +show what virtual environment you’re using, and modify the environment +so that running ‘python’ will get you that particular version and +installation of Python. For example: + + $ source ~/envs/tutorial-env/bin/activate + (tutorial-env) $ python + Python 3.5.1 (default, May 6 2016, 10:59:36) + ... + >>> import sys + >>> sys.path + ['', '/usr/local/lib/python35.zip', ..., + '~/envs/tutorial-env/lib/python3.5/site-packages'] + >>> + + +File: python.info, Node: Managing Packages with pip, Prev: Creating Virtual Environments, Up: Virtual Environments and Packages + +2.12.3 Managing Packages with pip +--------------------------------- + +You can install, upgrade, and remove packages using a program called +‘pip’. By default ‘pip’ will install packages from the Python Package +Index, <‘https://pypi.org’>. You can browse the Python Package Index by +going to it in your web browser, or you can use ‘pip’’s limited search +feature: + + (tutorial-env) $ pip search astronomy + skyfield - Elegant astronomy for Python + gary - Galactic astronomy and gravitational dynamics. + novas - The United States Naval Observatory NOVAS astronomy library + astroobs - Provides astronomy ephemeris to plan telescope observations + PyAstronomy - A collection of astronomy related tools for Python. + ... + +‘pip’ has a number of subcommands: “search”, “install”, “uninstall”, +“freeze”, etc. (Consult the *note Installing Python Modules: 7f5. guide +for complete documentation for ‘pip’.) + +You can install the latest version of a package by specifying a +package’s name: + + (tutorial-env) $ pip install novas + Collecting novas + Downloading novas-3.1.1.3.tar.gz (136kB) + Installing collected packages: novas + Running setup.py install for novas + Successfully installed novas-3.1.1.3 + +You can also install a specific version of a package by giving the +package name followed by ‘==’ and the version number: + + (tutorial-env) $ pip install requests==2.6.0 + Collecting requests==2.6.0 + Using cached requests-2.6.0-py2.py3-none-any.whl + Installing collected packages: requests + Successfully installed requests-2.6.0 + +If you re-run this command, ‘pip’ will notice that the requested version +is already installed and do nothing. You can supply a different version +number to get that version, or you can run ‘pip install --upgrade’ to +upgrade the package to the latest version: + + (tutorial-env) $ pip install --upgrade requests + Collecting requests + Installing collected packages: requests + Found existing installation: requests 2.6.0 + Uninstalling requests-2.6.0: + Successfully uninstalled requests-2.6.0 + Successfully installed requests-2.7.0 + +‘pip uninstall’ followed by one or more package names will remove the +packages from the virtual environment. + +‘pip show’ will display information about a particular package: + + (tutorial-env) $ pip show requests + --- + Metadata-Version: 2.0 + Name: requests + Version: 2.7.0 + Summary: Python HTTP for Humans. + Home-page: http://python-requests.org + Author: Kenneth Reitz + Author-email: me@kennethreitz.com + License: Apache 2.0 + Location: /Users/akuchling/envs/tutorial-env/lib/python3.4/site-packages + Requires: + +‘pip list’ will display all of the packages installed in the virtual +environment: + + (tutorial-env) $ pip list + novas (3.1.1.3) + numpy (1.9.2) + pip (7.0.3) + requests (2.7.0) + setuptools (16.0) + +‘pip freeze’ will produce a similar list of the installed packages, but +the output uses the format that ‘pip install’ expects. A common +convention is to put this list in a ‘requirements.txt’ file: + + (tutorial-env) $ pip freeze > requirements.txt + (tutorial-env) $ cat requirements.txt + novas==3.1.1.3 + numpy==1.9.2 + requests==2.7.0 + +The ‘requirements.txt’ can then be committed to version control and +shipped as part of an application. Users can then install all the +necessary packages with ‘install -r’: + + (tutorial-env) $ pip install -r requirements.txt + Collecting novas==3.1.1.3 (from -r requirements.txt (line 1)) + ... + Collecting numpy==1.9.2 (from -r requirements.txt (line 2)) + ... + Collecting requests==2.7.0 (from -r requirements.txt (line 3)) + ... + Installing collected packages: novas, numpy, requests + Running setup.py install for novas + Successfully installed novas-3.1.1.3 numpy-1.9.2 requests-2.7.0 + +‘pip’ has many more options. Consult the *note Installing Python +Modules: 7f5. guide for complete documentation for ‘pip’. When you’ve +written a package and want to make it available on the Python Package +Index, consult the *note Distributing Python Modules: 7f6. guide. + + +File: python.info, Node: What Now?, Next: Interactive Input Editing and History Substitution, Prev: Virtual Environments and Packages, Up: The Python Tutorial + +2.13 What Now? +============== + +Reading this tutorial has probably reinforced your interest in using +Python — you should be eager to apply Python to solving your real-world +problems. Where should you go to learn more? + +This tutorial is part of Python’s documentation set. Some other +documents in the set are: + + * *note The Python Standard Library: e8e.: + + You should browse through this manual, which gives complete (though + terse) reference material about types, functions, and the modules + in the standard library. The standard Python distribution includes + a `lot' of additional code. There are modules to read Unix + mailboxes, retrieve documents via HTTP, generate random numbers, + parse command-line options, write CGI programs, compress data, and + many other tasks. Skimming through the Library Reference will give + you an idea of what’s available. + + * *note Installing Python Modules: 7f5. explains how to install + additional modules written by other Python users. + + * *note The Python Language Reference: e8f.: A detailed explanation + of Python’s syntax and semantics. It’s heavy reading, but is + useful as a complete guide to the language itself. + +More Python resources: + + * ‘https://www.python.org’: The major Python Web site. It contains + code, documentation, and pointers to Python-related pages around + the Web. This Web site is mirrored in various places around the + world, such as Europe, Japan, and Australia; a mirror may be faster + than the main site, depending on your geographical location. + + * ‘https://docs.python.org’: Fast access to Python’s documentation. + + * ‘https://pypi.org’: The Python Package Index, previously also + nicknamed the Cheese Shop (1), is an index of user-created Python + modules that are available for download. Once you begin releasing + code, you can register it here so that others can find it. + + * ‘https://code.activestate.com/recipes/langs/python/’: The Python + Cookbook is a sizable collection of code examples, larger modules, + and useful scripts. Particularly notable contributions are + collected in a book also titled Python Cookbook (O’Reilly & + Associates, ISBN 0-596-00797-3.) + + * ‘http://www.pyvideo.org’ collects links to Python-related videos + from conferences and user-group meetings. + + * ‘https://scipy.org’: The Scientific Python project includes modules + for fast array computations and manipulations plus a host of + packages for such things as linear algebra, Fourier transforms, + non-linear solvers, random number distributions, statistical + analysis and the like. + +For Python-related questions and problem reports, you can post to the +newsgroup ‘comp.lang.python’, or send them to the mailing list at +. The newsgroup and mailing list are gatewayed, +so messages posted to one will automatically be forwarded to the other. +There are hundreds of postings a day, asking (and answering) questions, +suggesting new features, and announcing new modules. Mailing list +archives are available at ‘https://mail.python.org/pipermail/’. + +Before posting, be sure to check the list of *note Frequently Asked +Questions: fae. (also called the FAQ). The FAQ answers many of the +questions that come up again and again, and may already contain the +solution for your problem. + + ---------- Footnotes ---------- + + (1) (1) “Cheese Shop” is a Monty Python’s sketch: a customer enters a +cheese shop, but whatever cheese he asks for, the clerk says it’s +missing. + + +File: python.info, Node: Interactive Input Editing and History Substitution, Next: Floating Point Arithmetic Issues and Limitations, Prev: What Now?, Up: The Python Tutorial + +2.14 Interactive Input Editing and History Substitution +======================================================= + +Some versions of the Python interpreter support editing of the current +input line and history substitution, similar to facilities found in the +Korn shell and the GNU Bash shell. This is implemented using the GNU +Readline(1) library, which supports various styles of editing. This +library has its own documentation which we won’t duplicate here. + +* Menu: + +* Tab Completion and History Editing:: +* Alternatives to the Interactive Interpreter:: + + ---------- Footnotes ---------- + + (1) https://tiswww.case.edu/php/chet/readline/rltop.html + + +File: python.info, Node: Tab Completion and History Editing, Next: Alternatives to the Interactive Interpreter, Up: Interactive Input Editing and History Substitution + +2.14.1 Tab Completion and History Editing +----------------------------------------- + +Completion of variable and module names is *note automatically enabled: +8e6. at interpreter startup so that the ‘Tab’ key invokes the completion +function; it looks at Python statement names, the current local +variables, and the available module names. For dotted expressions such +as ‘string.a’, it will evaluate the expression up to the final ‘'.'’ and +then suggest completions from the attributes of the resulting object. +Note that this may execute application-defined code if an object with a +*note __getattr__(): 31a. method is part of the expression. The default +configuration also saves your history into a file named +‘.python_history’ in your user directory. The history will be available +again during the next interactive interpreter session. + + +File: python.info, Node: Alternatives to the Interactive Interpreter, Prev: Tab Completion and History Editing, Up: Interactive Input Editing and History Substitution + +2.14.2 Alternatives to the Interactive Interpreter +-------------------------------------------------- + +This facility is an enormous step forward compared to earlier versions +of the interpreter; however, some wishes are left: It would be nice if +the proper indentation were suggested on continuation lines (the parser +knows if an indent token is required next). The completion mechanism +might use the interpreter’s symbol table. A command to check (or even +suggest) matching parentheses, quotes, etc., would also be useful. + +One alternative enhanced interactive interpreter that has been around +for quite some time is IPython(1), which features tab completion, object +exploration and advanced history management. It can also be thoroughly +customized and embedded into other applications. Another similar +enhanced interactive environment is bpython(2). + + ---------- Footnotes ---------- + + (1) https://ipython.org/ + + (2) https://www.bpython-interpreter.org/ + + +File: python.info, Node: Floating Point Arithmetic Issues and Limitations, Next: Appendix, Prev: Interactive Input Editing and History Substitution, Up: The Python Tutorial + +2.15 Floating Point Arithmetic: Issues and Limitations +====================================================== + +Floating-point numbers are represented in computer hardware as base 2 +(binary) fractions. For example, the decimal fraction + + 0.125 + +has value 1/10 + 2/100 + 5/1000, and in the same way the binary fraction + + 0.001 + +has value 0/2 + 0/4 + 1/8. These two fractions have identical values, +the only real difference being that the first is written in base 10 +fractional notation, and the second in base 2. + +Unfortunately, most decimal fractions cannot be represented exactly as +binary fractions. A consequence is that, in general, the decimal +floating-point numbers you enter are only approximated by the binary +floating-point numbers actually stored in the machine. + +The problem is easier to understand at first in base 10. Consider the +fraction 1/3. You can approximate that as a base 10 fraction: + + 0.3 + +or, better, + + 0.33 + +or, better, + + 0.333 + +and so on. No matter how many digits you’re willing to write down, the +result will never be exactly 1/3, but will be an increasingly better +approximation of 1/3. + +In the same way, no matter how many base 2 digits you’re willing to use, +the decimal value 0.1 cannot be represented exactly as a base 2 +fraction. In base 2, 1/10 is the infinitely repeating fraction + + 0.0001100110011001100110011001100110011001100110011... + +Stop at any finite number of bits, and you get an approximation. On +most machines today, floats are approximated using a binary fraction +with the numerator using the first 53 bits starting with the most +significant bit and with the denominator as a power of two. In the case +of 1/10, the binary fraction is ‘3602879701896397 / 2 ** 55’ which is +close to but not exactly equal to the true value of 1/10. + +Many users are not aware of the approximation because of the way values +are displayed. Python only prints a decimal approximation to the true +decimal value of the binary approximation stored by the machine. On +most machines, if Python were to print the true decimal value of the +binary approximation stored for 0.1, it would have to display + + >>> 0.1 + 0.1000000000000000055511151231257827021181583404541015625 + +That is more digits than most people find useful, so Python keeps the +number of digits manageable by displaying a rounded value instead + + >>> 1 / 10 + 0.1 + +Just remember, even though the printed result looks like the exact value +of 1/10, the actual stored value is the nearest representable binary +fraction. + +Interestingly, there are many different decimal numbers that share the +same nearest approximate binary fraction. For example, the numbers +‘0.1’ and ‘0.10000000000000001’ and +‘0.1000000000000000055511151231257827021181583404541015625’ are all +approximated by ‘3602879701896397 / 2 ** 55’. Since all of these +decimal values share the same approximation, any one of them could be +displayed while still preserving the invariant ‘eval(repr(x)) == x’. + +Historically, the Python prompt and built-in *note repr(): 7cc. function +would choose the one with 17 significant digits, ‘0.10000000000000001’. +Starting with Python 3.1, Python (on most systems) is now able to choose +the shortest of these and simply display ‘0.1’. + +Note that this is in the very nature of binary floating-point: this is +not a bug in Python, and it is not a bug in your code either. You’ll +see the same kind of thing in all languages that support your hardware’s +floating-point arithmetic (although some languages may not `display' the +difference by default, or in all output modes). + +For more pleasant output, you may wish to use string formatting to +produce a limited number of significant digits: + + >>> format(math.pi, '.12g') # give 12 significant digits + '3.14159265359' + + >>> format(math.pi, '.2f') # give 2 digits after the point + '3.14' + + >>> repr(math.pi) + '3.141592653589793' + +It’s important to realize that this is, in a real sense, an illusion: +you’re simply rounding the `display' of the true machine value. + +One illusion may beget another. For example, since 0.1 is not exactly +1/10, summing three values of 0.1 may not yield exactly 0.3, either: + + >>> .1 + .1 + .1 == .3 + False + +Also, since the 0.1 cannot get any closer to the exact value of 1/10 and +0.3 cannot get any closer to the exact value of 3/10, then pre-rounding +with *note round(): c6d. function cannot help: + + >>> round(.1, 1) + round(.1, 1) + round(.1, 1) == round(.3, 1) + False + +Though the numbers cannot be made closer to their intended exact values, +the *note round(): c6d. function can be useful for post-rounding so that +results with inexact values become comparable to one another: + + >>> round(.1 + .1 + .1, 10) == round(.3, 10) + True + +Binary floating-point arithmetic holds many surprises like this. The +problem with “0.1” is explained in precise detail below, in the +“Representation Error” section. See The Perils of Floating Point(1) for +a more complete account of other common surprises. + +As that says near the end, “there are no easy answers.” Still, don’t be +unduly wary of floating-point! The errors in Python float operations +are inherited from the floating-point hardware, and on most machines are +on the order of no more than 1 part in 2**53 per operation. That’s more +than adequate for most tasks, but you do need to keep in mind that it’s +not decimal arithmetic and that every float operation can suffer a new +rounding error. + +While pathological cases do exist, for most casual use of floating-point +arithmetic you’ll see the result you expect in the end if you simply +round the display of your final results to the number of decimal digits +you expect. *note str(): 330. usually suffices, and for finer control +see the *note str.format(): 4da. method’s format specifiers in *note +Format String Syntax: d1a. + +For use cases which require exact decimal representation, try using the +*note decimal: 36. module which implements decimal arithmetic suitable +for accounting applications and high-precision applications. + +Another form of exact arithmetic is supported by the *note fractions: +83. module which implements arithmetic based on rational numbers (so the +numbers like 1/3 can be represented exactly). + +If you are a heavy user of floating point operations you should take a +look at the Numerical Python package and many other packages for +mathematical and statistical operations supplied by the SciPy project. +See <‘https://scipy.org’>. + +Python provides tools that may help on those rare occasions when you +really `do' want to know the exact value of a float. The *note +float.as_integer_ratio(): fb9. method expresses the value of a float as +a fraction: + + >>> x = 3.14159 + >>> x.as_integer_ratio() + (3537115888337719, 1125899906842624) + +Since the ratio is exact, it can be used to losslessly recreate the +original value: + + >>> x == 3537115888337719 / 1125899906842624 + True + +The *note float.hex(): fba. method expresses a float in hexadecimal +(base 16), again giving the exact value stored by your computer: + + >>> x.hex() + '0x1.921f9f01b866ep+1' + +This precise hexadecimal representation can be used to reconstruct the +float value exactly: + + >>> x == float.fromhex('0x1.921f9f01b866ep+1') + True + +Since the representation is exact, it is useful for reliably porting +values across different versions of Python (platform independence) and +exchanging data with other languages that support the same format (such +as Java and C99). + +Another helpful tool is the *note math.fsum(): d3e. function which helps +mitigate loss-of-precision during summation. It tracks “lost digits” as +values are added onto a running total. That can make a difference in +overall accuracy so that the errors do not accumulate to the point where +they affect the final total: + + >>> sum([0.1] * 10) == 1.0 + False + >>> math.fsum([0.1] * 10) == 1.0 + True + +* Menu: + +* Representation Error:: + + ---------- Footnotes ---------- + + (1) http://www.lahey.com/float.htm + + +File: python.info, Node: Representation Error, Up: Floating Point Arithmetic Issues and Limitations + +2.15.1 Representation Error +--------------------------- + +This section explains the “0.1” example in detail, and shows how you can +perform an exact analysis of cases like this yourself. Basic +familiarity with binary floating-point representation is assumed. + +`Representation error' refers to the fact that some (most, actually) +decimal fractions cannot be represented exactly as binary (base 2) +fractions. This is the chief reason why Python (or Perl, C, C++, Java, +Fortran, and many others) often won’t display the exact decimal number +you expect. + +Why is that? 1/10 is not exactly representable as a binary fraction. +Almost all machines today (November 2000) use IEEE-754 floating point +arithmetic, and almost all platforms map Python floats to IEEE-754 +“double precision”. 754 doubles contain 53 bits of precision, so on +input the computer strives to convert 0.1 to the closest fraction it can +of the form `J'/2**`N' where `J' is an integer containing exactly 53 +bits. Rewriting + + 1 / 10 ~= J / (2**N) + +as + + J ~= 2**N / 10 + +and recalling that `J' has exactly 53 bits (is ‘>= 2**52’ but ‘< +2**53’), the best value for `N' is 56: + + >>> 2**52 <= 2**56 // 10 < 2**53 + True + +That is, 56 is the only value for `N' that leaves `J' with exactly 53 +bits. The best possible value for `J' is then that quotient rounded: + + >>> q, r = divmod(2**56, 10) + >>> r + 6 + +Since the remainder is more than half of 10, the best approximation is +obtained by rounding up: + + >>> q+1 + 7205759403792794 + +Therefore the best possible approximation to 1/10 in 754 double +precision is: + + 7205759403792794 / 2 ** 56 + +Dividing both the numerator and denominator by two reduces the fraction +to: + + 3602879701896397 / 2 ** 55 + +Note that since we rounded up, this is actually a little bit larger than +1/10; if we had not rounded up, the quotient would have been a little +bit smaller than 1/10. But in no case can it be `exactly' 1/10! + +So the computer never “sees” 1/10: what it sees is the exact fraction +given above, the best 754 double approximation it can get: + + >>> 0.1 * 2 ** 55 + 3602879701896397.0 + +If we multiply that fraction by 10**55, we can see the value out to 55 +decimal digits: + + >>> 3602879701896397 * 10 ** 55 // 2 ** 55 + 1000000000000000055511151231257827021181583404541015625 + +meaning that the exact number stored in the computer is equal to the +decimal value 0.1000000000000000055511151231257827021181583404541015625. +Instead of displaying the full decimal value, many languages (including +older versions of Python), round the result to 17 significant digits: + + >>> format(0.1, '.17f') + '0.10000000000000001' + +The *note fractions: 83. and *note decimal: 36. modules make these +calculations easy: + + >>> from decimal import Decimal + >>> from fractions import Fraction + + >>> Fraction.from_float(0.1) + Fraction(3602879701896397, 36028797018963968) + + >>> (0.1).as_integer_ratio() + (3602879701896397, 36028797018963968) + + >>> Decimal.from_float(0.1) + Decimal('0.1000000000000000055511151231257827021181583404541015625') + + >>> format(Decimal.from_float(0.1), '.17') + '0.10000000000000001' + + +File: python.info, Node: Appendix, Prev: Floating Point Arithmetic Issues and Limitations, Up: The Python Tutorial + +2.16 Appendix +============= + +* Menu: + +* Interactive Mode: Interactive Mode<2>. + + +File: python.info, Node: Interactive Mode<2>, Up: Appendix + +2.16.1 Interactive Mode +----------------------- + +* Menu: + +* Error Handling:: +* Executable Python Scripts:: +* The Interactive Startup File:: +* The Customization Modules:: + + +File: python.info, Node: Error Handling, Next: Executable Python Scripts, Up: Interactive Mode<2> + +2.16.1.1 Error Handling +....................... + +When an error occurs, the interpreter prints an error message and a +stack trace. In interactive mode, it then returns to the primary +prompt; when input came from a file, it exits with a nonzero exit status +after printing the stack trace. (Exceptions handled by an *note except: +b3e. clause in a *note try: d72. statement are not errors in this +context.) Some errors are unconditionally fatal and cause an exit with +a nonzero exit; this applies to internal inconsistencies and some cases +of running out of memory. All error messages are written to the +standard error stream; normal output from executed commands is written +to standard output. + +Typing the interrupt character (usually ‘Control-C’ or ‘Delete’) to the +primary or secondary prompt cancels the input and returns to the primary +prompt. (1) Typing an interrupt while a command is executing raises the +*note KeyboardInterrupt: 197. exception, which may be handled by a *note +try: d72. statement. + + ---------- Footnotes ---------- + + (1) (1) A problem with the GNU Readline package may prevent this. + + +File: python.info, Node: Executable Python Scripts, Next: The Interactive Startup File, Prev: Error Handling, Up: Interactive Mode<2> + +2.16.1.2 Executable Python Scripts +.................................. + +On BSD’ish Unix systems, Python scripts can be made directly executable, +like shell scripts, by putting the line + + #!/usr/bin/env python3.5 + +(assuming that the interpreter is on the user’s ‘PATH’) at the beginning +of the script and giving the file an executable mode. The ‘#!’ must be +the first two characters of the file. On some platforms, this first +line must end with a Unix-style line ending (‘'\n'’), not a Windows +(‘'\r\n'’) line ending. Note that the hash, or pound, character, ‘'#'’, +is used to start a comment in Python. + +The script can be given an executable mode, or permission, using the +‘chmod’ command. + + $ chmod +x myscript.py + +On Windows systems, there is no notion of an “executable mode”. The +Python installer automatically associates ‘.py’ files with ‘python.exe’ +so that a double-click on a Python file will run it as a script. The +extension can also be ‘.pyw’, in that case, the console window that +normally appears is suppressed. + + +File: python.info, Node: The Interactive Startup File, Next: The Customization Modules, Prev: Executable Python Scripts, Up: Interactive Mode<2> + +2.16.1.3 The Interactive Startup File +..................................... + +When you use Python interactively, it is frequently handy to have some +standard commands executed every time the interpreter is started. You +can do this by setting an environment variable named *note +PYTHONSTARTUP: 8e5. to the name of a file containing your start-up +commands. This is similar to the ‘.profile’ feature of the Unix shells. + +This file is only read in interactive sessions, not when Python reads +commands from a script, and not when ‘/dev/tty’ is given as the explicit +source of commands (which otherwise behaves like an interactive +session). It is executed in the same namespace where interactive +commands are executed, so that objects that it defines or imports can be +used without qualification in the interactive session. You can also +change the prompts ‘sys.ps1’ and ‘sys.ps2’ in this file. + +If you want to read an additional start-up file from the current +directory, you can program this in the global start-up file using code +like ‘if os.path.isfile('.pythonrc.py'): +exec(open('.pythonrc.py').read())’. If you want to use the startup file +in a script, you must do this explicitly in the script: + + import os + filename = os.environ.get('PYTHONSTARTUP') + if filename and os.path.isfile(filename): + with open(filename) as fobj: + startup_file = fobj.read() + exec(startup_file) + + +File: python.info, Node: The Customization Modules, Prev: The Interactive Startup File, Up: Interactive Mode<2> + +2.16.1.4 The Customization Modules +.................................. + +Python provides two hooks to let you customize it: ‘sitecustomize’ and +‘usercustomize’. To see how it works, you need first to find the +location of your user site-packages directory. Start Python and run +this code: + + >>> import site + >>> site.getusersitepackages() + '/home/user/.local/lib/python3.5/site-packages' + +Now you can create a file named ‘usercustomize.py’ in that directory and +put anything you want in it. It will affect every invocation of Python, +unless it is started with the *note -s: d15. option to disable the +automatic import. + +‘sitecustomize’ works in the same way, but is typically created by an +administrator of the computer in the global site-packages directory, and +is imported before ‘usercustomize’. See the documentation of the *note +site: eb. module for more details. + + +File: python.info, Node: Python Setup and Usage, Next: The Python Language Reference, Prev: The Python Tutorial, Up: Top + +3 Python Setup and Usage +************************ + +This part of the documentation is devoted to general information on the +setup of the Python environment on different platforms, the invocation +of the interpreter and things that make working with Python easier. + +* Menu: + +* Command line and environment:: +* Using Python on Unix platforms:: +* Using Python on Windows:: +* Using Python on a Macintosh:: +* Editors and IDEs:: + + +File: python.info, Node: Command line and environment, Next: Using Python on Unix platforms, Up: Python Setup and Usage + +3.1 Command line and environment +================================ + +The CPython interpreter scans the command line and the environment for +various settings. + +`CPython implementation detail:' Other implementations’ command line +schemes may differ. See *note Alternate Implementations: fcd. for +further resources. + +* Menu: + +* Command line:: +* Environment variables:: + + +File: python.info, Node: Command line, Next: Environment variables, Up: Command line and environment + +3.1.1 Command line +------------------ + +When invoking Python, you may specify any of these options: + + python [-bBdEhiIOqsSuvVWx?] [-c command | -m module-name | script | - ] [args] + +The most common use case is, of course, a simple invocation of a script: + + python myscript.py + +* Menu: + +* Interface options:: +* Generic options:: +* Miscellaneous options:: +* Options you shouldn’t use:: + + +File: python.info, Node: Interface options, Next: Generic options, Up: Command line + +3.1.1.1 Interface options +......................... + +The interpreter interface resembles that of the UNIX shell, but provides +some additional methods of invocation: + + * When called with standard input connected to a tty device, it + prompts for commands and executes them until an EOF (an end-of-file + character, you can produce that with ‘Ctrl-D’ on UNIX or ‘Ctrl-Z, + Enter’ on Windows) is read. + + * When called with a file name argument or with a file as standard + input, it reads and executes a script from that file. + + * When called with a directory name argument, it reads and executes + an appropriately named script from that directory. + + * When called with ‘-c command’, it executes the Python statement(s) + given as `command'. Here `command' may contain multiple statements + separated by newlines. Leading whitespace is significant in Python + statements! + + * When called with ‘-m module-name’, the given module is located on + the Python module path and executed as a script. + +In non-interactive mode, the entire input is parsed before it is +executed. + +An interface option terminates the list of options consumed by the +interpreter, all consecutive arguments will end up in *note sys.argv: +bfb. – note that the first element, subscript zero (‘sys.argv[0]’), is a +string reflecting the program’s source. + + -- Program Option: -c + + Execute the Python code in `command'. `command' can be one or more + statements separated by newlines, with significant leading + whitespace as in normal module code. + + If this option is given, the first element of *note sys.argv: bfb. + will be ‘"-c"’ and the current directory will be added to the start + of *note sys.path: 488. (allowing modules in that directory to be + imported as top level modules). + + Raises an *note auditing event: fd1. ‘cpython.run_command’ with + argument ‘command’. + + -- Program Option: -m + + Search *note sys.path: 488. for the named module and execute its + contents as the *note __main__: 1. module. + + Since the argument is a `module' name, you must not give a file + extension (‘.py’). The module name should be a valid absolute + Python module name, but the implementation may not always enforce + this (e.g. it may allow you to use a name that includes a hyphen). + + Package names (including namespace packages) are also permitted. + When a package name is supplied instead of a normal module, the + interpreter will execute ‘.__main__’ as the main module. This + behaviour is deliberately similar to the handling of directories + and zipfiles that are passed to the interpreter as the script + argument. + + Note: This option cannot be used with built-in modules and + extension modules written in C, since they do not have Python + module files. However, it can still be used for precompiled + modules, even if the original source file is not available. + + If this option is given, the first element of *note sys.argv: bfb. + will be the full path to the module file (while the module file is + being located, the first element will be set to ‘"-m"’). As with + the *note -c: e9e. option, the current directory will be added to + the start of *note sys.path: 488. + + *note -I: fd2. option can be used to run the script in isolated + mode where *note sys.path: 488. contains neither the current + directory nor the user’s site-packages directory. All ‘PYTHON*’ + environment variables are ignored, too. + + Many standard library modules contain code that is invoked on their + execution as a script. An example is the *note timeit: 10b. + module: + + python -m timeit -s 'setup here' 'benchmarked code here' + python -m timeit -h # for details + + Raises an *note auditing event: fd1. ‘cpython.run_module’ with + argument ‘module-name’. + + See also + ........ + + *note runpy.run_module(): fd3. + + Equivalent functionality directly available to Python code + + PEP 338(1) – Executing modules as scripts + + Changed in version 3.1: Supply the package name to run a ‘__main__’ + submodule. + + Changed in version 3.4: namespace packages are also supported + + -- Describe: - + + Read commands from standard input (*note sys.stdin: 30d.). If + standard input is a terminal, *note -i: e1f. is implied. + + If this option is given, the first element of *note sys.argv: bfb. + will be ‘"-"’ and the current directory will be added to the start + of *note sys.path: 488. + + Raises an *note auditing event: fd1. ‘cpython.run_stdin’ with no + arguments. + + -- Describe: ’ and + ‘’). + + -- Method: HTMLParser.handle_entityref (name) + + This method is called to process a named character reference of the + form ‘&name;’ (e.g. ‘>’), where `name' is a general entity + reference (e.g. ‘'gt'’). This method is never called if + `convert_charrefs' is ‘True’. + + -- Method: HTMLParser.handle_charref (name) + + This method is called to process decimal and hexadecimal numeric + character references of the form ‘&#NNN;’ and ‘&#xNNN;’. For + example, the decimal equivalent for ‘>’ is ‘>’, whereas the + hexadecimal is ‘>’; in this case the method will receive + ‘'62'’ or ‘'x3E'’. This method is never called if + `convert_charrefs' is ‘True’. + + -- Method: HTMLParser.handle_comment (data) + + This method is called when a comment is encountered (e.g. + ‘’). + + For example, the comment ‘’ will cause this method + to be called with the argument ‘' comment '’. + + The content of Internet Explorer conditional comments (condcoms) + will also be sent to this method, so, for ‘’, this method will receive + ‘'[if IE 9]>IE9-specific content’). + + The `decl' parameter will be the entire contents of the declaration + inside the ‘’ markup (e.g. ‘'DOCTYPE html'’). + + -- Method: HTMLParser.handle_pi (data) + + Method called when a processing instruction is encountered. The + `data' parameter will contain the entire processing instruction. + For example, for the processing instruction ‘’, + this method would be called as ‘handle_pi("proc color='red'")’. It + is intended to be overridden by a derived class; the base class + implementation does nothing. + + Note: The *note HTMLParser: 7bf. class uses the SGML syntactic + rules for processing instructions. An XHTML processing + instruction using the trailing ‘'?'’ will cause the ‘'?'’ to + be included in `data'. + + -- Method: HTMLParser.unknown_decl (data) + + This method is called when an unrecognized declaration is read by + the parser. + + The `data' parameter will be the entire contents of the declaration + inside the ‘’ markup. It is sometimes useful to be + overridden by a derived class. The base class implementation does + nothing. + + +File: python.info, Node: Examples<19>, Prev: HTMLParser Methods, Up: html parser — Simple HTML and XHTML parser + +5.20.2.3 Examples +................. + +The following class implements a parser that will be used to illustrate +more examples: + + from html.parser import HTMLParser + from html.entities import name2codepoint + + class MyHTMLParser(HTMLParser): + def handle_starttag(self, tag, attrs): + print("Start tag:", tag) + for attr in attrs: + print(" attr:", attr) + + def handle_endtag(self, tag): + print("End tag :", tag) + + def handle_data(self, data): + print("Data :", data) + + def handle_comment(self, data): + print("Comment :", data) + + def handle_entityref(self, name): + c = chr(name2codepoint[name]) + print("Named ent:", c) + + def handle_charref(self, name): + if name.startswith('x'): + c = chr(int(name[1:], 16)) + else: + c = chr(int(name)) + print("Num ent :", c) + + def handle_decl(self, data): + print("Decl :", data) + + parser = MyHTMLParser() + +Parsing a doctype: + + >>> parser.feed('') + Decl : DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd" + +Parsing an element with a few attributes and a title: + + >>> parser.feed('The Python logo') + Start tag: img + attr: ('src', 'python-logo.png') + attr: ('alt', 'The Python logo') + >>> + >>> parser.feed('

Python

') + Start tag: h1 + Data : Python + End tag : h1 + +The content of ‘script’ and ‘style’ elements is returned as is, without +further parsing: + + >>> parser.feed('') + Start tag: style + attr: ('type', 'text/css') + Data : #python { color: green } + End tag : style + + >>> parser.feed('') + Start tag: script + attr: ('type', 'text/javascript') + Data : alert("hello!"); + End tag : script + +Parsing comments: + + >>> parser.feed('' + ... '') + Comment : a comment + Comment : [if IE 9]>IE-specific content'’): + + >>> parser.feed('>>>') + Named ent: > + Num ent : > + Num ent : > + +Feeding incomplete chunks to *note feed(): 26dc. works, but *note +handle_data(): 26e4. might be called more than once (unless +`convert_charrefs' is set to ‘True’): + + >>> for chunk in ['buff', 'ered ', 'text']: + ... parser.feed(chunk) + ... + Start tag: span + Data : buff + Data : ered + Data : text + End tag : span + +Parsing invalid HTML (e.g. unquoted attributes) also works: + + >>> parser.feed('

tag soup

') + Start tag: p + Start tag: a + attr: ('class', 'link') + attr: ('href', '#main') + Data : tag soup + End tag : p + End tag : a + + +File: python.info, Node: html entities — Definitions of HTML general entities, Next: XML Processing Modules, Prev: html parser — Simple HTML and XHTML parser, Up: Structured Markup Processing Tools + +5.20.3 ‘html.entities’ — Definitions of HTML general entities +------------------------------------------------------------- + +`Source code:' Lib/html/entities.py(1) + +__________________________________________________________________ + +This module defines four dictionaries, *note html5: a15, *note +name2codepoint: 26ef, *note codepoint2name: 26f0, and *note entitydefs: +26f1. + + -- Data: html.entities.html5 + + A dictionary that maps HTML5 named character references (2) to the + equivalent Unicode character(s), e.g. ‘html5['gt;'] == '>'’. Note + that the trailing semicolon is included in the name (e.g. + ‘'gt;'’), however some of the names are accepted by the standard + even without the semicolon: in this case the name is present with + and without the ‘';'’. See also *note html.unescape(): 85a. + + New in version 3.3. + + -- Data: html.entities.entitydefs + + A dictionary mapping XHTML 1.0 entity definitions to their + replacement text in ISO Latin-1. + + -- Data: html.entities.name2codepoint + + A dictionary that maps HTML entity names to the Unicode code + points. + + -- Data: html.entities.codepoint2name + + A dictionary that maps Unicode code points to HTML entity names. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/html/entities.py + + (2) (1) See +‘https://www.w3.org/TR/html5/syntax.html#named-character-references’ + + +File: python.info, Node: XML Processing Modules, Next: xml etree ElementTree — The ElementTree XML API, Prev: html entities — Definitions of HTML general entities, Up: Structured Markup Processing Tools + +5.20.4 XML Processing Modules +----------------------------- + +`Source code:' Lib/xml/(1) + +__________________________________________________________________ + +Python’s interfaces for processing XML are grouped in the ‘xml’ package. + + Warning: The XML modules are not secure against erroneous or + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see the *note XML vulnerabilities: 26f5. and + *note The defusedxml Package: 26f6. sections. + +It is important to note that modules in the *note xml: 133. package +require that there be at least one SAX-compliant XML parser available. +The Expat parser is included with Python, so the *note +xml.parsers.expat: 138. module will always be available. + +The documentation for the *note xml.dom: 134. and *note xml.sax: 13b. +packages are the definition of the Python bindings for the DOM and SAX +interfaces. + +The XML handling submodules are: + + * *note xml.etree.ElementTree: 137.: the ElementTree API, a simple + and lightweight XML processor + + * *note xml.dom: 134.: the DOM API definition + + * *note xml.dom.minidom: 135.: a minimal DOM implementation + + * *note xml.dom.pulldom: 136.: support for building partial DOM trees + + * *note xml.sax: 13b.: SAX2 base classes and convenience functions + + * *note xml.parsers.expat: 138.: the Expat parser binding + +* Menu: + +* XML vulnerabilities:: +* The defusedxml Package:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/xml/ + + +File: python.info, Node: XML vulnerabilities, Next: The defusedxml Package, Up: XML Processing Modules + +5.20.4.1 XML vulnerabilities +............................ + +The XML processing modules are not secure against maliciously +constructed data. An attacker can abuse XML features to carry out +denial of service attacks, access local files, generate network +connections to other machines, or circumvent firewalls. + +The following table gives an overview of the known attacks and whether +the various modules are vulnerable to them. + +kind sax etree minidom pulldom xmlrpc + +------------------------------------------------------------------------------------------------------------------------------ + +billion laughs `Vulnerable' `Vulnerable' `Vulnerable' `Vulnerable' `Vulnerable' + + +quadratic blowup `Vulnerable' `Vulnerable' `Vulnerable' `Vulnerable' `Vulnerable' + + +external entity expansion Safe (4) Safe (1) Safe (2) Safe (4) Safe (3) + + +DTD(1) retrieval Safe (4) Safe Safe Safe (4) Safe + + +decompression bomb Safe Safe Safe Safe `Vulnerable' + + + 1. *note xml.etree.ElementTree: 137. doesn’t expand external entities + and raises a ‘ParserError’ when an entity occurs. + + 2. *note xml.dom.minidom: 135. doesn’t expand external entities and + simply returns the unexpanded entity verbatim. + + 3. ‘xmlrpclib’ doesn’t expand external entities and omits them. + + 4. Since Python 3.7.1, external general entities are no longer + processed by default. + +billion laughs / exponential entity expansion + + The Billion Laughs(2) attack – also known as exponential entity + expansion – uses multiple levels of nested entities. Each entity + refers to another entity several times, and the final entity + definition contains a small string. The exponential expansion + results in several gigabytes of text and consumes lots of memory + and CPU time. + +quadratic blowup entity expansion + + A quadratic blowup attack is similar to a Billion Laughs(3) attack; + it abuses entity expansion, too. Instead of nested entities it + repeats one large entity with a couple of thousand chars over and + over again. The attack isn’t as efficient as the exponential case + but it avoids triggering parser countermeasures that forbid + deeply-nested entities. + +external entity expansion + + Entity declarations can contain more than just text for + replacement. They can also point to external resources or local + files. The XML parser accesses the resource and embeds the content + into the XML document. + +DTD(4) retrieval + + Some XML libraries like Python’s *note xml.dom.pulldom: 136. + retrieve document type definitions from remote or local locations. + The feature has similar implications as the external entity + expansion issue. + +decompression bomb + + Decompression bombs (aka ZIP bomb(5)) apply to all XML libraries + that can parse compressed XML streams such as gzipped HTTP streams + or LZMA-compressed files. For an attacker it can reduce the amount + of transmitted data by three magnitudes or more. + +The documentation for defusedxml(6) on PyPI has further information +about all known attack vectors with examples and references. + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Document_type_definition + + (2) https://en.wikipedia.org/wiki/Billion_laughs + + (3) https://en.wikipedia.org/wiki/Billion_laughs + + (4) https://en.wikipedia.org/wiki/Document_type_definition + + (5) https://en.wikipedia.org/wiki/Zip_bomb + + (6) https://pypi.org/project/defusedxml/ + + +File: python.info, Node: The defusedxml Package, Prev: XML vulnerabilities, Up: XML Processing Modules + +5.20.4.2 The ‘defusedxml’ Package +................................. + +defusedxml(1) is a pure Python package with modified subclasses of all +stdlib XML parsers that prevent any potentially malicious operation. +Use of this package is recommended for any server code that parses +untrusted XML data. The package also ships with example exploits and +extended documentation on more XML exploits such as XPath injection. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/defusedxml/ + + +File: python.info, Node: xml etree ElementTree — The ElementTree XML API, Next: xml dom — The Document Object Model API, Prev: XML Processing Modules, Up: Structured Markup Processing Tools + +5.20.5 ‘xml.etree.ElementTree’ — The ElementTree XML API +-------------------------------------------------------- + +`Source code:' Lib/xml/etree/ElementTree.py(1) + +__________________________________________________________________ + +The *note xml.etree.ElementTree: 137. module implements a simple and +efficient API for parsing and creating XML data. + +Changed in version 3.3: This module will use a fast implementation +whenever available. The ‘xml.etree.cElementTree’ module is deprecated. + + Warning: The *note xml.etree.ElementTree: 137. module is not secure + against maliciously constructed data. If you need to parse + untrusted or unauthenticated data see *note XML vulnerabilities: + 26f5. + +* Menu: + +* Tutorial:: +* XPath support:: +* Reference: Reference<2>. +* XInclude support:: +* Reference: Reference<3>. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/xml/etree/ElementTree.py + + +File: python.info, Node: Tutorial, Next: XPath support, Up: xml etree ElementTree — The ElementTree XML API + +5.20.5.1 Tutorial +................. + +This is a short tutorial for using *note xml.etree.ElementTree: 137. +(‘ET’ in short). The goal is to demonstrate some of the building blocks +and basic concepts of the module. + +* Menu: + +* XML tree and elements:: +* Parsing XML:: +* Pull API for non-blocking parsing:: +* Finding interesting elements:: +* Modifying an XML File:: +* Building XML documents:: +* Parsing XML with Namespaces:: +* Additional resources:: + + +File: python.info, Node: XML tree and elements, Next: Parsing XML, Up: Tutorial + +5.20.5.2 XML tree and elements +.............................. + +XML is an inherently hierarchical data format, and the most natural way +to represent it is with a tree. ‘ET’ has two classes for this purpose - +*note ElementTree: 917. represents the whole XML document as a tree, and +*note Element: ac4. represents a single node in this tree. Interactions +with the whole document (reading and writing to/from files) are usually +done on the *note ElementTree: 917. level. Interactions with a single +XML element and its sub-elements are done on the *note Element: ac4. +level. + + +File: python.info, Node: Parsing XML, Next: Pull API for non-blocking parsing, Prev: XML tree and elements, Up: Tutorial + +5.20.5.3 Parsing XML +.................... + +We’ll be using the following XML document as the sample data for this +section: + + + + + 1 + 2008 + 141100 + + + + + 4 + 2011 + 59900 + + + + 68 + 2011 + 13600 + + + + + +We can import this data by reading from a file: + + import xml.etree.ElementTree as ET + tree = ET.parse('country_data.xml') + root = tree.getroot() + +Or directly from a string: + + root = ET.fromstring(country_data_as_string) + +*note fromstring(): 2700. parses XML from a string directly into an +*note Element: ac4, which is the root element of the parsed tree. Other +parsing functions may create an *note ElementTree: 917. Check the +documentation to be sure. + +As an *note Element: ac4, ‘root’ has a tag and a dictionary of +attributes: + + >>> root.tag + 'data' + >>> root.attrib + {} + +It also has children nodes over which we can iterate: + + >>> for child in root: + ... print(child.tag, child.attrib) + ... + country {'name': 'Liechtenstein'} + country {'name': 'Singapore'} + country {'name': 'Panama'} + +Children are nested, and we can access specific child nodes by index: + + >>> root[0][1].text + '2008' + + Note: Not all elements of the XML input will end up as elements of + the parsed tree. Currently, this module skips over any XML + comments, processing instructions, and document type declarations + in the input. Nevertheless, trees built using this module’s API + rather than parsing from XML text can have comments and processing + instructions in them; they will be included when generating XML + output. A document type declaration may be accessed by passing a + custom *note TreeBuilder: 253. instance to the *note XMLParser: + 252. constructor. + + +File: python.info, Node: Pull API for non-blocking parsing, Next: Finding interesting elements, Prev: Parsing XML, Up: Tutorial + +5.20.5.4 Pull API for non-blocking parsing +.......................................... + +Most parsing functions provided by this module require the whole +document to be read at once before returning any result. It is possible +to use an *note XMLParser: 252. and feed data into it incrementally, but +it is a push API that calls methods on a callback target, which is too +low-level and inconvenient for most needs. Sometimes what the user +really wants is to be able to parse XML incrementally, without blocking +operations, while enjoying the convenience of fully constructed *note +Element: ac4. objects. + +The most powerful tool for doing this is *note XMLPullParser: 913. It +does not require a blocking read to obtain the XML data, and is instead +fed with data incrementally with *note XMLPullParser.feed(): 2702. +calls. To get the parsed XML elements, call *note +XMLPullParser.read_events(): 2703. Here is an example: + + >>> parser = ET.XMLPullParser(['start', 'end']) + >>> parser.feed('sometext') + >>> list(parser.read_events()) + [('start', )] + >>> parser.feed(' more text') + >>> for event, elem in parser.read_events(): + ... print(event) + ... print(elem.tag, 'text=', elem.text) + ... + end + +The obvious use case is applications that operate in a non-blocking +fashion where the XML data is being received from a socket or read +incrementally from some storage device. In such cases, blocking reads +are unacceptable. + +Because it’s so flexible, *note XMLPullParser: 913. can be inconvenient +to use for simpler use-cases. If you don’t mind your application +blocking on reading XML data but would still like to have incremental +parsing capabilities, take a look at *note iterparse(): 948. It can be +useful when you’re reading a large XML document and don’t want to hold +it wholly in memory. + + +File: python.info, Node: Finding interesting elements, Next: Modifying an XML File, Prev: Pull API for non-blocking parsing, Up: Tutorial + +5.20.5.5 Finding interesting elements +..................................... + +*note Element: ac4. has some useful methods that help iterate +recursively over all the sub-tree below it (its children, their +children, and so on). For example, *note Element.iter(): ce7.: + + >>> for neighbor in root.iter('neighbor'): + ... print(neighbor.attrib) + ... + {'name': 'Austria', 'direction': 'E'} + {'name': 'Switzerland', 'direction': 'W'} + {'name': 'Malaysia', 'direction': 'N'} + {'name': 'Costa Rica', 'direction': 'W'} + {'name': 'Colombia', 'direction': 'E'} + +*note Element.findall(): 2705. finds only elements with a tag which are +direct children of the current element. *note Element.find(): 2706. +finds the `first' child with a particular tag, and *note Element.text: +2707. accesses the element’s text content. *note Element.get(): 2708. +accesses the element’s attributes: + + >>> for country in root.findall('country'): + ... rank = country.find('rank').text + ... name = country.get('name') + ... print(name, rank) + ... + Liechtenstein 1 + Singapore 4 + Panama 68 + +More sophisticated specification of which elements to look for is +possible by using *note XPath: 41a. + + +File: python.info, Node: Modifying an XML File, Next: Building XML documents, Prev: Finding interesting elements, Up: Tutorial + +5.20.5.6 Modifying an XML File +.............................. + +*note ElementTree: 917. provides a simple way to build XML documents and +write them to files. The *note ElementTree.write(): 918. method serves +this purpose. + +Once created, an *note Element: ac4. object may be manipulated by +directly changing its fields (such as *note Element.text: 2707.), adding +and modifying attributes (*note Element.set(): 270a. method), as well as +adding new children (for example with *note Element.append(): 270b.). + +Let’s say we want to add one to each country’s rank, and add an +‘updated’ attribute to the rank element: + + >>> for rank in root.iter('rank'): + ... new_rank = int(rank.text) + 1 + ... rank.text = str(new_rank) + ... rank.set('updated', 'yes') + ... + >>> tree.write('output.xml') + +Our XML now looks like this: + + + + + 2 + 2008 + 141100 + + + + + 5 + 2011 + 59900 + + + + 69 + 2011 + 13600 + + + + + +We can remove elements using *note Element.remove(): 270c. Let’s say we +want to remove all countries with a rank higher than 50: + + >>> for country in root.findall('country'): + ... # using root.findall() to avoid removal during traversal + ... rank = int(country.find('rank').text) + ... if rank > 50: + ... root.remove(country) + ... + >>> tree.write('output.xml') + +Note that concurrent modification while iterating can lead to problems, +just like when iterating and modifying Python lists or dicts. +Therefore, the example first collects all matching elements with +‘root.findall()’, and only then iterates over the list of matches. + +Our XML now looks like this: + + + + + 2 + 2008 + 141100 + + + + + 5 + 2011 + 59900 + + + + + +File: python.info, Node: Building XML documents, Next: Parsing XML with Namespaces, Prev: Modifying an XML File, Up: Tutorial + +5.20.5.7 Building XML documents +............................... + +The *note SubElement(): 270e. function also provides a convenient way to +create new sub-elements for a given element: + + >>> a = ET.Element('a') + >>> b = ET.SubElement(a, 'b') + >>> c = ET.SubElement(a, 'c') + >>> d = ET.SubElement(c, 'd') + >>> ET.dump(a) + + + +File: python.info, Node: Parsing XML with Namespaces, Next: Additional resources, Prev: Building XML documents, Up: Tutorial + +5.20.5.8 Parsing XML with Namespaces +.................................... + +If the XML input has namespaces(1), tags and attributes with prefixes in +the form ‘prefix:sometag’ get expanded to ‘{uri}sometag’ where the +`prefix' is replaced by the full `URI'. Also, if there is a default +namespace(2), that full URI gets prepended to all of the non-prefixed +tags. + +Here is an XML example that incorporates two namespaces, one with the +prefix “fictional” and the other serving as the default namespace: + + + + + John Cleese + Lancelot + Archie Leach + + + Eric Idle + Sir Robin + Gunther + Commander Clement + + + +One way to search and explore this XML example is to manually add the +URI to every tag or attribute in the xpath of a *note find(): 2706. or +*note findall(): 2705.: + + root = fromstring(xml_text) + for actor in root.findall('{http://people.example.com}actor'): + name = actor.find('{http://people.example.com}name') + print(name.text) + for char in actor.findall('{http://characters.example.com}character'): + print(' |-->', char.text) + +A better way to search the namespaced XML example is to create a +dictionary with your own prefixes and use those in the search functions: + + ns = {'real_person': 'http://people.example.com', + 'role': 'http://characters.example.com'} + + for actor in root.findall('real_person:actor', ns): + name = actor.find('real_person:name', ns) + print(name.text) + for char in actor.findall('role:character', ns): + print(' |-->', char.text) + +These two approaches both output: + + John Cleese + |--> Lancelot + |--> Archie Leach + Eric Idle + |--> Sir Robin + |--> Gunther + |--> Commander Clement + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/XML_namespace + + (2) https://www.w3.org/TR/xml-names/#defaulting + + +File: python.info, Node: Additional resources, Prev: Parsing XML with Namespaces, Up: Tutorial + +5.20.5.9 Additional resources +............................. + +See ‘http://effbot.org/zone/element-index.htm’ for tutorials and links +to other docs. + + +File: python.info, Node: XPath support, Next: Reference<2>, Prev: Tutorial, Up: xml etree ElementTree — The ElementTree XML API + +5.20.5.10 XPath support +....................... + +This module provides limited support for XPath expressions(1) for +locating elements in a tree. The goal is to support a small subset of +the abbreviated syntax; a full XPath engine is outside the scope of the +module. + +* Menu: + +* Example: Example<10>. +* Supported XPath syntax:: + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/xpath + + +File: python.info, Node: Example<10>, Next: Supported XPath syntax, Up: XPath support + +5.20.5.11 Example +................. + +Here’s an example that demonstrates some of the XPath capabilities of +the module. We’ll be using the ‘countrydata’ XML document from the +*note Parsing XML: 26fe. section: + + import xml.etree.ElementTree as ET + + root = ET.fromstring(countrydata) + + # Top-level elements + root.findall(".") + + # All 'neighbor' grand-children of 'country' children of the top-level + # elements + root.findall("./country/neighbor") + + # Nodes with name='Singapore' that have a 'year' child + root.findall(".//year/..[@name='Singapore']") + + # 'year' nodes that are children of nodes with name='Singapore' + root.findall(".//*[@name='Singapore']/year") + + # All 'neighbor' nodes that are the second child of their parent + root.findall(".//neighbor[2]") + +For XML with namespaces, use the usual qualified ‘{namespace}tag’ +notation: + + # All dublin-core "title" tags in the document + root.findall(".//{http://purl.org/dc/elements/1.1/}title") + + +File: python.info, Node: Supported XPath syntax, Prev: Example<10>, Up: XPath support + +5.20.5.12 Supported XPath syntax +................................ + +Syntax Meaning + +--------------------------------------------------------------------------------------- + +‘tag’ Selects all child elements with the given tag. For + example, ‘spam’ selects all child elements named ‘spam’, + and ‘spam/egg’ selects all grandchildren named ‘egg’ in + all children named ‘spam’. ‘{namespace}*’ selects all + tags in the given namespace, ‘{*}spam’ selects tags + named ‘spam’ in any (or no) namespace, and ‘{}*’ only + selects tags that are not in a namespace. + + Changed in version 3.8: Support for star-wildcards was + added. + + +‘*’ Selects all child elements, including comments and + processing instructions. For example, ‘*/egg’ selects + all grandchildren named ‘egg’. + + +‘.’ Selects the current node. This is mostly useful at the + beginning of the path, to indicate that it’s a relative + path. + + +‘//’ Selects all subelements, on all levels beneath the + current element. For example, ‘.//egg’ selects all + ‘egg’ elements in the entire tree. + + +‘..’ Selects the parent element. Returns ‘None’ if the path + attempts to reach the ancestors of the start element + (the element ‘find’ was called on). + + +‘[@attrib]’ Selects all elements that have the given attribute. + + +‘[@attrib='value']’ Selects all elements for which the given attribute has + the given value. The value cannot contain quotes. + + +‘[tag]’ Selects all elements that have a child named ‘tag’. + Only immediate children are supported. + + +‘[.='text']’ Selects all elements whose complete text content, + including descendants, equals the given ‘text’. + + New in version 3.7. + + +‘[tag='text']’ Selects all elements that have a child named ‘tag’ whose + complete text content, including descendants, equals the + given ‘text’. + + +‘[position]’ Selects all elements that are located at the given + position. The position can be either an integer (1 is + the first position), the expression ‘last()’ (for the + last position), or a position relative to the last + position (e.g. ‘last()-1’). + + +Predicates (expressions within square brackets) must be preceded by a +tag name, an asterisk, or another predicate. ‘position’ predicates must +be preceded by a tag name. + + +File: python.info, Node: Reference<2>, Next: XInclude support, Prev: XPath support, Up: xml etree ElementTree — The ElementTree XML API + +5.20.5.13 Reference +................... + +* Menu: + +* Functions: Functions<6>. + + +File: python.info, Node: Functions<6>, Up: Reference<2> + +5.20.5.14 Functions +................... + + -- Function: xml.etree.ElementTree.canonicalize (xml_data=None, *, + out=None, from_file=None, **options) + + C14N 2.0(1) transformation function. + + Canonicalization is a way to normalise XML output in a way that + allows byte-by-byte comparisons and digital signatures. It reduced + the freedom that XML serializers have and instead generates a more + constrained XML representation. The main restrictions regard the + placement of namespace declarations, the ordering of attributes, + and ignorable whitespace. + + This function takes an XML data string (`xml_data') or a file path + or file-like object (`from_file') as input, converts it to the + canonical form, and writes it out using the `out' file(-like) + object, if provided, or returns it as a text string if not. The + output file receives text, not bytes. It should therefore be + opened in text mode with ‘utf-8’ encoding. + + Typical uses: + + xml_data = "..." + print(canonicalize(xml_data)) + + with open("c14n_output.xml", mode='w', encoding='utf-8') as out_file: + canonicalize(xml_data, out=out_file) + + with open("c14n_output.xml", mode='w', encoding='utf-8') as out_file: + canonicalize(from_file="inputfile.xml", out=out_file) + + The configuration `options' are as follows: + + - `with_comments': set to true to include comments (default: + false) + + - + `strip_text': set to true to strip whitespace before and after text content + + (default: false) + + - + `rewrite_prefixes': set to true to replace namespace prefixes by “n{number}” + + (default: false) + + - + `qname_aware_tags': a set of qname aware tag names in which prefixes + + should be replaced in text content (default: empty) + + - + `qname_aware_attrs': a set of qname aware attribute names in which prefixes + + should be replaced in text content (default: empty) + + - `exclude_attrs': a set of attribute names that should not be + serialised + + - `exclude_tags': a set of tag names that should not be + serialised + + In the option list above, “a set” refers to any collection or + iterable of strings, no ordering is expected. + + New in version 3.8. + + -- Function: xml.etree.ElementTree.Comment (text=None) + + Comment element factory. This factory function creates a special + element that will be serialized as an XML comment by the standard + serializer. The comment string can be either a bytestring or a + Unicode string. `text' is a string containing the comment string. + Returns an element instance representing a comment. + + Note that *note XMLParser: 252. skips over comments in the input + instead of creating comment objects for them. An *note + ElementTree: 917. will only contain comment nodes if they have been + inserted into to the tree using one of the *note Element: ac4. + methods. + + -- Function: xml.etree.ElementTree.dump (elem) + + Writes an element tree or element structure to sys.stdout. This + function should be used for debugging only. + + The exact output format is implementation dependent. In this + version, it’s written as an ordinary XML file. + + `elem' is an element tree or an individual element. + + Changed in version 3.8: The *note dump(): 2719. function now + preserves the attribute order specified by the user. + + -- Function: xml.etree.ElementTree.fromstring (text, parser=None) + + Parses an XML section from a string constant. Same as *note XML(): + 271a. `text' is a string containing XML data. `parser' is an + optional parser instance. If not given, the standard *note + XMLParser: 252. parser is used. Returns an *note Element: ac4. + instance. + + -- Function: xml.etree.ElementTree.fromstringlist (sequence, + parser=None) + + Parses an XML document from a sequence of string fragments. + `sequence' is a list or other sequence containing XML data + fragments. `parser' is an optional parser instance. If not given, + the standard *note XMLParser: 252. parser is used. Returns an + *note Element: ac4. instance. + + New in version 3.2. + + -- Function: xml.etree.ElementTree.iselement (element) + + Check if an object appears to be a valid element object. `element' + is an element instance. Return ‘True’ if this is an element + object. + + -- Function: xml.etree.ElementTree.iterparse (source, events=None, + parser=None) + + Parses an XML section into an element tree incrementally, and + reports what’s going on to the user. `source' is a filename or + *note file object: b42. containing XML data. `events' is a + sequence of events to report back. The supported events are the + strings ‘"start"’, ‘"end"’, ‘"comment"’, ‘"pi"’, ‘"start-ns"’ and + ‘"end-ns"’ (the “ns” events are used to get detailed namespace + information). If `events' is omitted, only ‘"end"’ events are + reported. `parser' is an optional parser instance. If not given, + the standard *note XMLParser: 252. parser is used. `parser' must + be a subclass of *note XMLParser: 252. and can only use the default + *note TreeBuilder: 253. as a target. Returns an *note iterator: + 112e. providing ‘(event, elem)’ pairs. + + Note that while *note iterparse(): 948. builds the tree + incrementally, it issues blocking reads on `source' (or the file it + names). As such, it’s unsuitable for applications where blocking + reads can’t be made. For fully non-blocking parsing, see *note + XMLPullParser: 913. + + Note: *note iterparse(): 948. only guarantees that it has seen + the “>” character of a starting tag when it emits a “start” + event, so the attributes are defined, but the contents of the + text and tail attributes are undefined at that point. The + same applies to the element children; they may or may not be + present. + + If you need a fully populated element, look for “end” events + instead. + + Deprecated since version 3.4: The `parser' argument. + + Changed in version 3.8: The ‘comment’ and ‘pi’ events were added. + + -- Function: xml.etree.ElementTree.parse (source, parser=None) + + Parses an XML section into an element tree. `source' is a filename + or file object containing XML data. `parser' is an optional parser + instance. If not given, the standard *note XMLParser: 252. parser + is used. Returns an *note ElementTree: 917. instance. + + -- Function: xml.etree.ElementTree.ProcessingInstruction (target, + text=None) + + PI element factory. This factory function creates a special + element that will be serialized as an XML processing instruction. + `target' is a string containing the PI target. `text' is a string + containing the PI contents, if given. Returns an element instance, + representing a processing instruction. + + Note that *note XMLParser: 252. skips over processing instructions + in the input instead of creating comment objects for them. An + *note ElementTree: 917. will only contain processing instruction + nodes if they have been inserted into to the tree using one of the + *note Element: ac4. methods. + + -- Function: xml.etree.ElementTree.register_namespace (prefix, uri) + + Registers a namespace prefix. The registry is global, and any + existing mapping for either the given prefix or the namespace URI + will be removed. `prefix' is a namespace prefix. `uri' is a + namespace uri. Tags and attributes in this namespace will be + serialized with the given prefix, if at all possible. + + New in version 3.2. + + -- Function: xml.etree.ElementTree.SubElement (parent, tag, attrib={}, + **extra) + + Subelement factory. This function creates an element instance, and + appends it to an existing element. + + The element name, attribute names, and attribute values can be + either bytestrings or Unicode strings. `parent' is the parent + element. `tag' is the subelement name. `attrib' is an optional + dictionary, containing element attributes. `extra' contains + additional attributes, given as keyword arguments. Returns an + element instance. + + -- Function: xml.etree.ElementTree.tostring (element, + encoding="us-ascii", method="xml", *, xml_declaration=None, + default_namespace=None, short_empty_elements=True) + + Generates a string representation of an XML element, including all + subelements. `element' is an *note Element: ac4. instance. + `encoding' (2) is the output encoding (default is US-ASCII). Use + ‘encoding="unicode"’ to generate a Unicode string (otherwise, a + bytestring is generated). `method' is either ‘"xml"’, ‘"html"’ or + ‘"text"’ (default is ‘"xml"’). `xml_declaration', + `default_namespace' and `short_empty_elements' has the same meaning + as in *note ElementTree.write(): 918. Returns an (optionally) + encoded string containing the XML data. + + New in version 3.4: The `short_empty_elements' parameter. + + New in version 3.8: The `xml_declaration' and `default_namespace' + parameters. + + Changed in version 3.8: The *note tostring(): 915. function now + preserves the attribute order specified by the user. + + -- Function: xml.etree.ElementTree.tostringlist (element, + encoding="us-ascii", method="xml", *, xml_declaration=None, + default_namespace=None, short_empty_elements=True) + + Generates a string representation of an XML element, including all + subelements. `element' is an *note Element: ac4. instance. + `encoding' (3) is the output encoding (default is US-ASCII). Use + ‘encoding="unicode"’ to generate a Unicode string (otherwise, a + bytestring is generated). `method' is either ‘"xml"’, ‘"html"’ or + ‘"text"’ (default is ‘"xml"’). `xml_declaration', + `default_namespace' and `short_empty_elements' has the same meaning + as in *note ElementTree.write(): 918. Returns a list of + (optionally) encoded strings containing the XML data. It does not + guarantee any specific sequence, except that + ‘b"".join(tostringlist(element)) == tostring(element)’. + + New in version 3.2. + + New in version 3.4: The `short_empty_elements' parameter. + + New in version 3.8: The `xml_declaration' and `default_namespace' + parameters. + + Changed in version 3.8: The *note tostringlist(): 916. function now + preserves the attribute order specified by the user. + + -- Function: xml.etree.ElementTree.XML (text, parser=None) + + Parses an XML section from a string constant. This function can be + used to embed “XML literals” in Python code. `text' is a string + containing XML data. `parser' is an optional parser instance. If + not given, the standard *note XMLParser: 252. parser is used. + Returns an *note Element: ac4. instance. + + -- Function: xml.etree.ElementTree.XMLID (text, parser=None) + + Parses an XML section from a string constant, and also returns a + dictionary which maps from element id:s to elements. `text' is a + string containing XML data. `parser' is an optional parser + instance. If not given, the standard *note XMLParser: 252. parser + is used. Returns a tuple containing an *note Element: ac4. + instance and a dictionary. + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/xml-c14n2/ + + (2) (1) The encoding string included in XML output should conform to +the appropriate standards. For example, “UTF-8” is valid, but “UTF8” is +not. See +‘https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl’ and +‘https://www.iana.org/assignments/character-sets/character-sets.xhtml’. + + (3) (1) The encoding string included in XML output should conform to +the appropriate standards. For example, “UTF-8” is valid, but “UTF8” is +not. See +‘https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl’ and +‘https://www.iana.org/assignments/character-sets/character-sets.xhtml’. + + +File: python.info, Node: XInclude support, Next: Reference<3>, Prev: Reference<2>, Up: xml etree ElementTree — The ElementTree XML API + +5.20.5.15 XInclude support +.......................... + +This module provides limited support for XInclude directives(1), via the +‘xml.etree.ElementInclude’ helper module. This module can be used to +insert subtrees and text strings into element trees, based on +information in the tree. + +* Menu: + +* Example: Example<11>. + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/xinclude/ + + +File: python.info, Node: Example<11>, Up: XInclude support + +5.20.5.16 Example +................. + +Here’s an example that demonstrates use of the XInclude module. To +include an XML document in the current document, use the +‘{http://www.w3.org/2001/XInclude}include’ element and set the `parse' +attribute to ‘"xml"’, and use the `href' attribute to specify the +document to include. + + + + + + +By default, the `href' attribute is treated as a file name. You can use +custom loaders to override this behaviour. Also note that the standard +helper does not support XPointer syntax. + +To process this file, load it as usual, and pass the root element to the +*note xml.etree.ElementTree: 137. module: + + from xml.etree import ElementTree, ElementInclude + + tree = ElementTree.parse("document.xml") + root = tree.getroot() + + ElementInclude.include(root) + +The ElementInclude module replaces the +‘{http://www.w3.org/2001/XInclude}include’ element with the root element +from the `source.xml' document. The result might look something like +this: + + + This is a paragraph. + + +If the `parse' attribute is omitted, it defaults to “xml”. The href +attribute is required. + +To include a text document, use the +‘{http://www.w3.org/2001/XInclude}include’ element, and set the `parse' +attribute to “text”: + + + + Copyright (c) . + + +The result might look something like: + + + Copyright (c) 2003. + + + +File: python.info, Node: Reference<3>, Prev: XInclude support, Up: xml etree ElementTree — The ElementTree XML API + +5.20.5.17 Reference +................... + +* Menu: + +* Functions: Functions<7>. +* Element Objects:: +* ElementTree Objects:: +* QName Objects:: +* TreeBuilder Objects:: +* XMLParser Objects:: +* XMLPullParser Objects:: +* Exceptions: Exceptions<15>. + + +File: python.info, Node: Functions<7>, Next: Element Objects, Up: Reference<3> + +5.20.5.18 Functions +................... + + -- Function: xml.etree.ElementInclude.default_loader (href, parse, + encoding=None) + + Default loader. This default loader reads an included resource + from disk. `href' is a URL. `parse' is for parse mode either “xml” + or “text”. `encoding' is an optional text encoding. If not given, + encoding is ‘utf-8’. Returns the expanded resource. If the parse + mode is ‘"xml"’, this is an ElementTree instance. If the parse + mode is “text”, this is a Unicode string. If the loader fails, it + can return None or raise an exception. + + -- Function: xml.etree.ElementInclude.include (elem, loader=None) + + This function expands XInclude directives. `elem' is the root + element. `loader' is an optional resource loader. If omitted, it + defaults to *note default_loader(): 2725. If given, it should be a + callable that implements the same interface as *note + default_loader(): 2725. Returns the expanded resource. If the + parse mode is ‘"xml"’, this is an ElementTree instance. If the + parse mode is “text”, this is a Unicode string. If the loader + fails, it can return None or raise an exception. + + +File: python.info, Node: Element Objects, Next: ElementTree Objects, Prev: Functions<7>, Up: Reference<3> + +5.20.5.19 Element Objects +......................... + + -- Class: xml.etree.ElementTree.Element (tag, attrib={}, **extra) + + Element class. This class defines the Element interface, and + provides a reference implementation of this interface. + + The element name, attribute names, and attribute values can be + either bytestrings or Unicode strings. `tag' is the element name. + `attrib' is an optional dictionary, containing element attributes. + `extra' contains additional attributes, given as keyword arguments. + + -- Attribute: tag + + A string identifying what kind of data this element represents + (the element type, in other words). + + -- Attribute: text + -- Attribute: tail + + These attributes can be used to hold additional data + associated with the element. Their values are usually strings + but may be any application-specific object. If the element is + created from an XML file, the `text' attribute holds either + the text between the element’s start tag and its first child + or end tag, or ‘None’, and the `tail' attribute holds either + the text between the element’s end tag and the next tag, or + ‘None’. For the XML data + + 1234 + + the `a' element has ‘None’ for both `text' and `tail' + attributes, the `b' element has `text' ‘"1"’ and `tail' ‘"4"’, + the `c' element has `text' ‘"2"’ and `tail' ‘None’, and the + `d' element has `text' ‘None’ and `tail' ‘"3"’. + + To collect the inner text of an element, see *note itertext(): + b53, for example ‘"".join(element.itertext())’. + + Applications may store arbitrary objects in these attributes. + + -- Attribute: attrib + + A dictionary containing the element’s attributes. Note that + while the `attrib' value is always a real mutable Python + dictionary, an ElementTree implementation may choose to use + another internal representation, and create the dictionary + only if someone asks for it. To take advantage of such + implementations, use the dictionary methods below whenever + possible. + + The following dictionary-like methods work on the element + attributes. + + -- Method: clear () + + Resets an element. This function removes all subelements, + clears all attributes, and sets the text and tail attributes + to ‘None’. + + -- Method: get (key, default=None) + + Gets the element attribute named `key'. + + Returns the attribute value, or `default' if the attribute was + not found. + + -- Method: items () + + Returns the element attributes as a sequence of (name, value) + pairs. The attributes are returned in an arbitrary order. + + -- Method: keys () + + Returns the elements attribute names as a list. The names are + returned in an arbitrary order. + + -- Method: set (key, value) + + Set the attribute `key' on the element to `value'. + + The following methods work on the element’s children (subelements). + + -- Method: append (subelement) + + Adds the element `subelement' to the end of this element’s + internal list of subelements. Raises *note TypeError: 192. if + `subelement' is not an *note Element: ac4. + + -- Method: extend (subelements) + + Appends `subelements' from a sequence object with zero or more + elements. Raises *note TypeError: 192. if a subelement is not + an *note Element: ac4. + + New in version 3.2. + + -- Method: find (match, namespaces=None) + + Finds the first subelement matching `match'. `match' may be a + tag name or a *note path: 41a. Returns an element instance or + ‘None’. `namespaces' is an optional mapping from namespace + prefix to full name. Pass ‘''’ as prefix to move all + unprefixed tag names in the expression into the given + namespace. + + -- Method: findall (match, namespaces=None) + + Finds all matching subelements, by tag name or *note path: + 41a. Returns a list containing all matching elements in + document order. `namespaces' is an optional mapping from + namespace prefix to full name. Pass ‘''’ as prefix to move + all unprefixed tag names in the expression into the given + namespace. + + -- Method: findtext (match, default=None, namespaces=None) + + Finds text for the first subelement matching `match'. `match' + may be a tag name or a *note path: 41a. Returns the text + content of the first matching element, or `default' if no + element was found. Note that if the matching element has no + text content an empty string is returned. `namespaces' is an + optional mapping from namespace prefix to full name. Pass + ‘''’ as prefix to move all unprefixed tag names in the + expression into the given namespace. + + -- Method: getchildren () + + Deprecated since version 3.2, will be removed in version 3.9: + Use ‘list(elem)’ or iteration. + + -- Method: getiterator (tag=None) + + Deprecated since version 3.2, will be removed in version 3.9: + Use method *note Element.iter(): ce7. instead. + + -- Method: insert (index, subelement) + + Inserts `subelement' at the given position in this element. + Raises *note TypeError: 192. if `subelement' is not an *note + Element: ac4. + + -- Method: iter (tag=None) + + Creates a tree *note iterator: 112e. with the current element + as the root. The iterator iterates over this element and all + elements below it, in document (depth first) order. If `tag' + is not ‘None’ or ‘'*'’, only elements whose tag equals `tag' + are returned from the iterator. If the tree structure is + modified during iteration, the result is undefined. + + New in version 3.2. + + -- Method: iterfind (match, namespaces=None) + + Finds all matching subelements, by tag name or *note path: + 41a. Returns an iterable yielding all matching elements in + document order. `namespaces' is an optional mapping from + namespace prefix to full name. + + New in version 3.2. + + -- Method: itertext () + + Creates a text iterator. The iterator loops over this element + and all subelements, in document order, and returns all inner + text. + + New in version 3.2. + + -- Method: makeelement (tag, attrib) + + Creates a new element object of the same type as this element. + Do not call this method, use the *note SubElement(): 270e. + factory function instead. + + -- Method: remove (subelement) + + Removes `subelement' from the element. Unlike the find* + methods this method compares elements based on the instance + identity, not on tag value or contents. + + *note Element: ac4. objects also support the following sequence + type methods for working with subelements: *note __delitem__(): + c63, *note __getitem__(): 27c, *note __setitem__(): c62, *note + __len__(): dcb. + + Caution: Elements with no subelements will test as ‘False’. This + behavior will change in future versions. Use specific ‘len(elem)’ + or ‘elem is None’ test instead. + + element = root.find('foo') + + if not element: # careful! + print("element not found, or element has no subelements") + + if element is None: + print("element not found") + + Prior to Python 3.8, the serialisation order of the XML attributes + of elements was artificially made predictable by sorting the + attributes by their name. Based on the now guaranteed ordering of + dicts, this arbitrary reordering was removed in Python 3.8 to + preserve the order in which attributes were originally parsed or + created by user code. + + In general, user code should try not to depend on a specific + ordering of attributes, given that the XML Information Set(1) + explicitly excludes the attribute order from conveying information. + Code should be prepared to deal with any ordering on input. In + cases where deterministic XML output is required, e.g. for + cryptographic signing or test data sets, canonical serialisation is + available with the *note canonicalize(): 2717. function. + + In cases where canonical output is not applicable but a specific + attribute order is still desirable on output, code should aim for + creating the attributes directly in the desired order, to avoid + perceptual mismatches for readers of the code. In cases where this + is difficult to achieve, a recipe like the following can be applied + prior to serialisation to enforce an order independently from the + Element creation: + + def reorder_attributes(root): + for el in root.iter(): + attrib = el.attrib + if len(attrib) > 1: + # adjust attribute order, e.g. by sorting + attribs = sorted(attrib.items()) + attrib.clear() + attrib.update(attribs) + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/xml-infoset/ + + +File: python.info, Node: ElementTree Objects, Next: QName Objects, Prev: Element Objects, Up: Reference<3> + +5.20.5.20 ElementTree Objects +............................. + + -- Class: xml.etree.ElementTree.ElementTree (element=None, file=None) + + ElementTree wrapper class. This class represents an entire element + hierarchy, and adds some extra support for serialization to and + from standard XML. + + `element' is the root element. The tree is initialized with the + contents of the XML `file' if given. + + -- Method: _setroot (element) + + Replaces the root element for this tree. This discards the + current contents of the tree, and replaces it with the given + element. Use with care. `element' is an element instance. + + -- Method: find (match, namespaces=None) + + Same as *note Element.find(): 2706, starting at the root of + the tree. + + -- Method: findall (match, namespaces=None) + + Same as *note Element.findall(): 2705, starting at the root of + the tree. + + -- Method: findtext (match, default=None, namespaces=None) + + Same as *note Element.findtext(): 272f, starting at the root + of the tree. + + -- Method: getiterator (tag=None) + + Deprecated since version 3.2, will be removed in version 3.9: + Use method *note ElementTree.iter(): 273b. instead. + + -- Method: getroot () + + Returns the root element for this tree. + + -- Method: iter (tag=None) + + Creates and returns a tree iterator for the root element. The + iterator loops over all elements in this tree, in section + order. `tag' is the tag to look for (default is to return all + elements). + + -- Method: iterfind (match, namespaces=None) + + Same as *note Element.iterfind(): b52, starting at the root of + the tree. + + New in version 3.2. + + -- Method: parse (source, parser=None) + + Loads an external XML section into this element tree. + `source' is a file name or *note file object: b42. `parser' + is an optional parser instance. If not given, the standard + *note XMLParser: 252. parser is used. Returns the section + root element. + + -- Method: write (file, encoding="us-ascii", xml_declaration=None, + default_namespace=None, method="xml", *, + short_empty_elements=True) + + Writes the element tree to a file, as XML. `file' is a file + name, or a *note file object: b42. opened for writing. + `encoding' (1) is the output encoding (default is US-ASCII). + `xml_declaration' controls if an XML declaration should be + added to the file. Use ‘False’ for never, ‘True’ for always, + ‘None’ for only if not US-ASCII or UTF-8 or Unicode (default + is ‘None’). `default_namespace' sets the default XML + namespace (for “xmlns”). `method' is either ‘"xml"’, ‘"html"’ + or ‘"text"’ (default is ‘"xml"’). The keyword-only + `short_empty_elements' parameter controls the formatting of + elements that contain no content. If ‘True’ (the default), + they are emitted as a single self-closed tag, otherwise they + are emitted as a pair of start/end tags. + + The output is either a string (*note str: 330.) or binary + (*note bytes: 331.). This is controlled by the `encoding' + argument. If `encoding' is ‘"unicode"’, the output is a + string; otherwise, it’s binary. Note that this may conflict + with the type of `file' if it’s an open *note file object: + b42.; make sure you do not try to write a string to a binary + stream and vice versa. + + New in version 3.4: The `short_empty_elements' parameter. + + Changed in version 3.8: The *note write(): 918. method now + preserves the attribute order specified by the user. + +This is the XML file that is going to be manipulated: + + + + Example page + + +

Moved to example.org + or example.com.

+ + + +Example of changing the attribute “target” of every link in first +paragraph: + + >>> from xml.etree.ElementTree import ElementTree + >>> tree = ElementTree() + >>> tree.parse("index.xhtml") + + >>> p = tree.find("body/p") # Finds first occurrence of tag p in body + >>> p + + >>> links = list(p.iter("a")) # Returns list of all links + >>> links + [, ] + >>> for i in links: # Iterates through all found links + ... i.attrib["target"] = "blank" + >>> tree.write("output.xhtml") + + ---------- Footnotes ---------- + + (1) (1) The encoding string included in XML output should conform to +the appropriate standards. For example, “UTF-8” is valid, but “UTF8” is +not. See +‘https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl’ and +‘https://www.iana.org/assignments/character-sets/character-sets.xhtml’. + + +File: python.info, Node: QName Objects, Next: TreeBuilder Objects, Prev: ElementTree Objects, Up: Reference<3> + +5.20.5.21 QName Objects +....................... + + -- Class: xml.etree.ElementTree.QName (text_or_uri, tag=None) + + QName wrapper. This can be used to wrap a QName attribute value, + in order to get proper namespace handling on output. `text_or_uri' + is a string containing the QName value, in the form {uri}local, or, + if the tag argument is given, the URI part of a QName. If `tag' is + given, the first argument is interpreted as a URI, and this + argument is interpreted as a local name. *note QName: 2741. + instances are opaque. + + +File: python.info, Node: TreeBuilder Objects, Next: XMLParser Objects, Prev: QName Objects, Up: Reference<3> + +5.20.5.22 TreeBuilder Objects +............................. + + -- Class: xml.etree.ElementTree.TreeBuilder (element_factory=None, *, + comment_factory=None, pi_factory=None, insert_comments=False, + insert_pis=False) + + Generic element structure builder. This builder converts a + sequence of start, data, end, comment and pi method calls to a + well-formed element structure. You can use this class to build an + element structure using a custom XML parser, or a parser for some + other XML-like format. + + `element_factory', when given, must be a callable accepting two + positional arguments: a tag and a dict of attributes. It is + expected to return a new element instance. + + The `comment_factory' and `pi_factory' functions, when given, + should behave like the *note Comment(): 2718. and *note + ProcessingInstruction(): 271d. functions to create comments and + processing instructions. When not given, the default factories + will be used. When `insert_comments' and/or `insert_pis' is true, + comments/pis will be inserted into the tree if they appear within + the root element (but not outside of it). + + -- Method: close () + + Flushes the builder buffers, and returns the toplevel document + element. Returns an *note Element: ac4. instance. + + -- Method: data (data) + + Adds text to the current element. `data' is a string. This + should be either a bytestring, or a Unicode string. + + -- Method: end (tag) + + Closes the current element. `tag' is the element name. + Returns the closed element. + + -- Method: start (tag, attrs) + + Opens a new element. `tag' is the element name. `attrs' is a + dictionary containing element attributes. Returns the opened + element. + + -- Method: comment (text) + + Creates a comment with the given `text'. If ‘insert_comments’ + is true, this will also add it to the tree. + + New in version 3.8. + + -- Method: pi (target, text) + + Creates a comment with the given `target' name and `text'. If + ‘insert_pis’ is true, this will also add it to the tree. + + New in version 3.8. + + In addition, a custom *note TreeBuilder: 253. object can provide + the following methods: + + -- Method: doctype (name, pubid, system) + + Handles a doctype declaration. `name' is the doctype name. + `pubid' is the public identifier. `system' is the system + identifier. This method does not exist on the default *note + TreeBuilder: 253. class. + + New in version 3.2. + + -- Method: start_ns (prefix, uri) + + Is called whenever the parser encounters a new namespace + declaration, before the ‘start()’ callback for the opening + element that defines it. `prefix' is ‘''’ for the default + namespace and the declared namespace prefix name otherwise. + `uri' is the namespace URI. + + New in version 3.8. + + -- Method: end_ns (prefix) + + Is called after the ‘end()’ callback of an element that + declared a namespace prefix mapping, with the name of the + `prefix' that went out of scope. + + New in version 3.8. + + -- Class: xml.etree.ElementTree.C14NWriterTarget (write, *, + with_comments=False, strip_text=False, rewrite_prefixes=False, + qname_aware_tags=None, qname_aware_attrs=None, + exclude_attrs=None, exclude_tags=None) + + A C14N 2.0(1) writer. Arguments are the same as for the *note + canonicalize(): 2717. function. This class does not build a tree + but translates the callback events directly into a serialised form + using the `write' function. + + New in version 3.8. + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/xml-c14n2/ + + +File: python.info, Node: XMLParser Objects, Next: XMLPullParser Objects, Prev: TreeBuilder Objects, Up: Reference<3> + +5.20.5.23 XMLParser Objects +........................... + + -- Class: xml.etree.ElementTree.XMLParser (*, target=None, + encoding=None) + + This class is the low-level building block of the module. It uses + *note xml.parsers.expat: 138. for efficient, event-based parsing of + XML. It can be fed XML data incrementally with the *note feed(): + 274e. method, and parsing events are translated to a push API - by + invoking callbacks on the `target' object. If `target' is omitted, + the standard *note TreeBuilder: 253. is used. If `encoding' (1) is + given, the value overrides the encoding specified in the XML file. + + Changed in version 3.8: Parameters are now *note keyword-only: 2ac. + The `html' argument no longer supported. + + -- Method: close () + + Finishes feeding data to the parser. Returns the result of + calling the ‘close()’ method of the `target' passed during + construction; by default, this is the toplevel document + element. + + -- Method: feed (data) + + Feeds data to the parser. `data' is encoded data. + + *note XMLParser.feed(): 274e. calls `target'’s ‘start(tag, + attrs_dict)’ method for each opening tag, its ‘end(tag)’ method for + each closing tag, and data is processed by method ‘data(data)’. + For further supported callback methods, see the *note TreeBuilder: + 253. class. *note XMLParser.close(): 274f. calls `target'’s method + ‘close()’. *note XMLParser: 252. can be used not only for building + a tree structure. This is an example of counting the maximum depth + of an XML file: + + >>> from xml.etree.ElementTree import XMLParser + >>> class MaxDepth: # The target object of the parser + ... maxDepth = 0 + ... depth = 0 + ... def start(self, tag, attrib): # Called for each opening tag. + ... self.depth += 1 + ... if self.depth > self.maxDepth: + ... self.maxDepth = self.depth + ... def end(self, tag): # Called for each closing tag. + ... self.depth -= 1 + ... def data(self, data): + ... pass # We do not need to do anything with data. + ... def close(self): # Called when all data has been parsed. + ... return self.maxDepth + ... + >>> target = MaxDepth() + >>> parser = XMLParser(target=target) + >>> exampleXml = """ + ... + ... + ... + ... + ... + ... + ... + ... + ... + ... """ + >>> parser.feed(exampleXml) + >>> parser.close() + 4 + + ---------- Footnotes ---------- + + (1) (1) The encoding string included in XML output should conform to +the appropriate standards. For example, “UTF-8” is valid, but “UTF8” is +not. See +‘https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl’ and +‘https://www.iana.org/assignments/character-sets/character-sets.xhtml’. + + +File: python.info, Node: XMLPullParser Objects, Next: Exceptions<15>, Prev: XMLParser Objects, Up: Reference<3> + +5.20.5.24 XMLPullParser Objects +............................... + + -- Class: xml.etree.ElementTree.XMLPullParser (events=None) + + A pull parser suitable for non-blocking applications. Its + input-side API is similar to that of *note XMLParser: 252, but + instead of pushing calls to a callback target, *note XMLPullParser: + 913. collects an internal list of parsing events and lets the user + read from it. `events' is a sequence of events to report back. + The supported events are the strings ‘"start"’, ‘"end"’, + ‘"comment"’, ‘"pi"’, ‘"start-ns"’ and ‘"end-ns"’ (the “ns” events + are used to get detailed namespace information). If `events' is + omitted, only ‘"end"’ events are reported. + + -- Method: feed (data) + + Feed the given bytes data to the parser. + + -- Method: close () + + Signal the parser that the data stream is terminated. Unlike + *note XMLParser.close(): 274f, this method always returns + *note None: 157. Any events not yet retrieved when the parser + is closed can still be read with *note read_events(): 2703. + + -- Method: read_events () + + Return an iterator over the events which have been encountered + in the data fed to the parser. The iterator yields ‘(event, + elem)’ pairs, where `event' is a string representing the type + of event (e.g. ‘"end"’) and `elem' is the encountered *note + Element: ac4. object, or other context value as follows. + + * ‘start’, ‘end’: the current Element. + + * ‘comment’, ‘pi’: the current comment / processing + instruction + + * ‘start-ns’: a tuple ‘(prefix, uri)’ naming the declared + namespace mapping. + + * ‘end-ns’: *note None: 157. (this may change in a future + version) + + Events provided in a previous call to *note read_events(): + 2703. will not be yielded again. Events are consumed from the + internal queue only when they are retrieved from the iterator, + so multiple readers iterating in parallel over iterators + obtained from *note read_events(): 2703. will have + unpredictable results. + + Note: *note XMLPullParser: 913. only guarantees that it has + seen the “>” character of a starting tag when it emits a + “start” event, so the attributes are defined, but the contents + of the text and tail attributes are undefined at that point. + The same applies to the element children; they may or may not + be present. + + If you need a fully populated element, look for “end” events + instead. + + New in version 3.4. + + Changed in version 3.8: The ‘comment’ and ‘pi’ events were added. + + +File: python.info, Node: Exceptions<15>, Prev: XMLPullParser Objects, Up: Reference<3> + +5.20.5.25 Exceptions +.................... + + -- Class: xml.etree.ElementTree.ParseError + + XML parse error, raised by the various parsing methods in this + module when parsing fails. The string representation of an + instance of this exception will contain a user-friendly error + message. In addition, it will have the following attributes + available: + + -- Attribute: code + + A numeric error code from the expat parser. See the + documentation of *note xml.parsers.expat: 138. for the list of + error codes and their meanings. + + -- Attribute: position + + A tuple of `line', `column' numbers, specifying where the + error occurred. + + +File: python.info, Node: xml dom — The Document Object Model API, Next: xml dom minidom — Minimal DOM implementation, Prev: xml etree ElementTree — The ElementTree XML API, Up: Structured Markup Processing Tools + +5.20.6 ‘xml.dom’ — The Document Object Model API +------------------------------------------------ + +`Source code:' Lib/xml/dom/__init__.py(1) + +__________________________________________________________________ + +The Document Object Model, or “DOM,” is a cross-language API from the +World Wide Web Consortium (W3C) for accessing and modifying XML +documents. A DOM implementation presents an XML document as a tree +structure, or allows client code to build such a structure from scratch. +It then gives access to the structure through a set of objects which +provided well-known interfaces. + +The DOM is extremely useful for random-access applications. SAX only +allows you a view of one bit of the document at a time. If you are +looking at one SAX element, you have no access to another. If you are +looking at a text node, you have no access to a containing element. +When you write a SAX application, you need to keep track of your +program’s position in the document somewhere in your own code. SAX does +not do it for you. Also, if you need to look ahead in the XML document, +you are just out of luck. + +Some applications are simply impossible in an event driven model with no +access to a tree. Of course you could build some sort of tree yourself +in SAX events, but the DOM allows you to avoid writing that code. The +DOM is a standard tree representation for XML data. + +The Document Object Model is being defined by the W3C in stages, or +“levels” in their terminology. The Python mapping of the API is +substantially based on the DOM Level 2 recommendation. + +DOM applications typically start by parsing some XML into a DOM. How +this is accomplished is not covered at all by DOM Level 1, and Level 2 +provides only limited improvements: There is a ‘DOMImplementation’ +object class which provides access to ‘Document’ creation methods, but +no way to access an XML reader/parser/Document builder in an +implementation-independent way. There is also no well-defined way to +access these methods without an existing ‘Document’ object. In Python, +each DOM implementation will provide a function *note +getDOMImplementation(): 2758. DOM Level 3 adds a Load/Store +specification, which defines an interface to the reader, but this is not +yet available in the Python standard library. + +Once you have a DOM document object, you can access the parts of your +XML document through its properties and methods. These properties are +defined in the DOM specification; this portion of the reference manual +describes the interpretation of the specification in Python. + +The specification provided by the W3C defines the DOM API for Java, +ECMAScript, and OMG IDL. The Python mapping defined here is based in +large part on the IDL version of the specification, but strict +compliance is not required (though implementations are free to support +the strict mapping from IDL). See section *note Conformance: 2759. for a +detailed discussion of mapping requirements. + +See also +........ + +Document Object Model (DOM) Level 2 Specification(2) + + The W3C recommendation upon which the Python DOM API is based. + +Document Object Model (DOM) Level 1 Specification(3) + + The W3C recommendation for the DOM supported by *note + xml.dom.minidom: 135. + +Python Language Mapping Specification(4) + + This specifies the mapping from OMG IDL to Python. + +* Menu: + +* Module Contents: Module Contents<4>. +* Objects in the DOM:: +* Conformance:: + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/xml/dom/__init__.py + + (2) https://www.w3.org/TR/2000/REC-DOM-Level-2-Core-20001113/ + + (3) https://www.w3.org/TR/REC-DOM-Level-1/ + + (4) http://www.omg.org/cgi-bin/doc?formal/02-11-05.pdf + + +File: python.info, Node: Module Contents<4>, Next: Objects in the DOM, Up: xml dom — The Document Object Model API + +5.20.6.1 Module Contents +........................ + +The *note xml.dom: 134. contains the following functions: + + -- Function: xml.dom.registerDOMImplementation (name, factory) + + Register the `factory' function with the name `name'. The factory + function should return an object which implements the + ‘DOMImplementation’ interface. The factory function can return the + same object every time, or a new one for each call, as appropriate + for the specific implementation (e.g. if that implementation + supports some customization). + + -- Function: xml.dom.getDOMImplementation (name=None, features=()) + + Return a suitable DOM implementation. The `name' is either + well-known, the module name of a DOM implementation, or ‘None’. If + it is not ‘None’, imports the corresponding module and returns a + ‘DOMImplementation’ object if the import succeeds. If no name is + given, and if the environment variable ‘PYTHON_DOM’ is set, this + variable is used to find the implementation. + + If name is not given, this examines the available implementations + to find one with the required feature set. If no implementation + can be found, raise an *note ImportError: 334. The features list + must be a sequence of ‘(feature, version)’ pairs which are passed + to the ‘hasFeature()’ method on available ‘DOMImplementation’ + objects. + +Some convenience constants are also provided: + + -- Data: xml.dom.EMPTY_NAMESPACE + + The value used to indicate that no namespace is associated with a + node in the DOM. This is typically found as the ‘namespaceURI’ of a + node, or used as the `namespaceURI' parameter to a + namespaces-specific method. + + -- Data: xml.dom.XML_NAMESPACE + + The namespace URI associated with the reserved prefix ‘xml’, as + defined by Namespaces in XML(1) (section 4). + + -- Data: xml.dom.XMLNS_NAMESPACE + + The namespace URI for namespace declarations, as defined by + Document Object Model (DOM) Level 2 Core Specification(2) (section + 1.1.8). + + -- Data: xml.dom.XHTML_NAMESPACE + + The URI of the XHTML namespace as defined by XHTML 1.0: The + Extensible HyperText Markup Language(3) (section 3.1.1). + +In addition, *note xml.dom: 134. contains a base ‘Node’ class and the +DOM exception classes. The ‘Node’ class provided by this module does +not implement any of the methods or attributes defined by the DOM +specification; concrete DOM implementations must provide those. The +‘Node’ class provided as part of this module does provide the constants +used for the ‘nodeType’ attribute on concrete ‘Node’ objects; they are +located within the class rather than at the module level to conform with +the DOM specifications. + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/REC-xml-names/ + + (2) https://www.w3.org/TR/DOM-Level-2-Core/core.html + + (3) https://www.w3.org/TR/xhtml1/ + + +File: python.info, Node: Objects in the DOM, Next: Conformance, Prev: Module Contents<4>, Up: xml dom — The Document Object Model API + +5.20.6.2 Objects in the DOM +........................... + +The definitive documentation for the DOM is the DOM specification from +the W3C. + +Note that DOM attributes may also be manipulated as nodes instead of as +simple strings. It is fairly rare that you must do this, however, so +this usage is not yet documented. + +Interface Section Purpose + +------------------------------------------------------------------------------------------------------------------- + +‘DOMImplementation’ *note DOMImplementation Objects: 2762. Interface to the underlying + implementation. + + +‘Node’ *note Node Objects: 2763. Base interface for most objects in + a document. + + +‘NodeList’ *note NodeList Objects: 2764. Interface for a sequence of nodes. + + +‘DocumentType’ *note DocumentType Objects: 2765. Information about the declarations + needed to process a document. + + +‘Document’ *note Document Objects: 2766. Object which represents an entire + document. + + +‘Element’ *note Element Objects: 2767. Element nodes in the document + hierarchy. + + +‘Attr’ *note Attr Objects: 2768. Attribute value nodes on element + nodes. + + +‘Comment’ *note Comment Objects: 2769. Representation of comments in the + source document. + + +‘Text’ *note Text and CDATASection Objects: 276a.Nodes containing textual content + from the document. + + +‘ProcessingInstruction’ *note ProcessingInstruction Objects: 276b.Processing instruction + representation. + + +An additional section describes the exceptions defined for working with +the DOM in Python. + +* Menu: + +* DOMImplementation Objects:: +* Node Objects:: +* NodeList Objects:: +* DocumentType Objects:: +* Document Objects:: +* Element Objects: Element Objects<2>. +* Attr Objects:: +* NamedNodeMap Objects:: +* Comment Objects:: +* Text and CDATASection Objects:: +* ProcessingInstruction Objects:: +* Exceptions: Exceptions<16>. + + +File: python.info, Node: DOMImplementation Objects, Next: Node Objects, Up: Objects in the DOM + +5.20.6.3 DOMImplementation Objects +.................................. + +The ‘DOMImplementation’ interface provides a way for applications to +determine the availability of particular features in the DOM they are +using. DOM Level 2 added the ability to create new ‘Document’ and +‘DocumentType’ objects using the ‘DOMImplementation’ as well. + + -- Method: DOMImplementation.hasFeature (feature, version) + + Return ‘True’ if the feature identified by the pair of strings + `feature' and `version' is implemented. + + -- Method: DOMImplementation.createDocument (namespaceUri, + qualifiedName, doctype) + + Return a new ‘Document’ object (the root of the DOM), with a child + ‘Element’ object having the given `namespaceUri' and + `qualifiedName'. The `doctype' must be a ‘DocumentType’ object + created by *note createDocumentType(): 276f, or ‘None’. In the + Python DOM API, the first two arguments can also be ‘None’ in order + to indicate that no ‘Element’ child is to be created. + + -- Method: DOMImplementation.createDocumentType (qualifiedName, + publicId, systemId) + + Return a new ‘DocumentType’ object that encapsulates the given + `qualifiedName', `publicId', and `systemId' strings, representing + the information contained in an XML document type declaration. + + +File: python.info, Node: Node Objects, Next: NodeList Objects, Prev: DOMImplementation Objects, Up: Objects in the DOM + +5.20.6.4 Node Objects +..................... + +All of the components of an XML document are subclasses of ‘Node’. + + -- Attribute: Node.nodeType + + An integer representing the node type. Symbolic constants for the + types are on the ‘Node’ object: ‘ELEMENT_NODE’, ‘ATTRIBUTE_NODE’, + ‘TEXT_NODE’, ‘CDATA_SECTION_NODE’, ‘ENTITY_NODE’, + ‘PROCESSING_INSTRUCTION_NODE’, ‘COMMENT_NODE’, ‘DOCUMENT_NODE’, + ‘DOCUMENT_TYPE_NODE’, ‘NOTATION_NODE’. This is a read-only + attribute. + + -- Attribute: Node.parentNode + + The parent of the current node, or ‘None’ for the document node. + The value is always a ‘Node’ object or ‘None’. For ‘Element’ + nodes, this will be the parent element, except for the root + element, in which case it will be the ‘Document’ object. For + ‘Attr’ nodes, this is always ‘None’. This is a read-only + attribute. + + -- Attribute: Node.attributes + + A ‘NamedNodeMap’ of attribute objects. Only elements have actual + values for this; others provide ‘None’ for this attribute. This is + a read-only attribute. + + -- Attribute: Node.previousSibling + + The node that immediately precedes this one with the same parent. + For instance the element with an end-tag that comes just before the + `self' element’s start-tag. Of course, XML documents are made up + of more than just elements so the previous sibling could be text, a + comment, or something else. If this node is the first child of the + parent, this attribute will be ‘None’. This is a read-only + attribute. + + -- Attribute: Node.nextSibling + + The node that immediately follows this one with the same parent. + See also *note previousSibling: 2774. If this is the last child of + the parent, this attribute will be ‘None’. This is a read-only + attribute. + + -- Attribute: Node.childNodes + + A list of nodes contained within this node. This is a read-only + attribute. + + -- Attribute: Node.firstChild + + The first child of the node, if there are any, or ‘None’. This is + a read-only attribute. + + -- Attribute: Node.lastChild + + The last child of the node, if there are any, or ‘None’. This is a + read-only attribute. + + -- Attribute: Node.localName + + The part of the ‘tagName’ following the colon if there is one, else + the entire ‘tagName’. The value is a string. + + -- Attribute: Node.prefix + + The part of the ‘tagName’ preceding the colon if there is one, else + the empty string. The value is a string, or ‘None’. + + -- Attribute: Node.namespaceURI + + The namespace associated with the element name. This will be a + string or ‘None’. This is a read-only attribute. + + -- Attribute: Node.nodeName + + This has a different meaning for each node type; see the DOM + specification for details. You can always get the information you + would get here from another property such as the ‘tagName’ property + for elements or the ‘name’ property for attributes. For all node + types, the value of this attribute will be either a string or + ‘None’. This is a read-only attribute. + + -- Attribute: Node.nodeValue + + This has a different meaning for each node type; see the DOM + specification for details. The situation is similar to that with + *note nodeName: 277c. The value is a string or ‘None’. + + -- Method: Node.hasAttributes () + + Return ‘True’ if the node has any attributes. + + -- Method: Node.hasChildNodes () + + Return ‘True’ if the node has any child nodes. + + -- Method: Node.isSameNode (other) + + Return ‘True’ if `other' refers to the same node as this node. + This is especially useful for DOM implementations which use any + sort of proxy architecture (because more than one object can refer + to the same node). + + Note: This is based on a proposed DOM Level 3 API which is + still in the “working draft” stage, but this particular + interface appears uncontroversial. Changes from the W3C will + not necessarily affect this method in the Python DOM interface + (though any new W3C API for this would also be supported). + + -- Method: Node.appendChild (newChild) + + Add a new child node to this node at the end of the list of + children, returning `newChild'. If the node was already in the + tree, it is removed first. + + -- Method: Node.insertBefore (newChild, refChild) + + Insert a new child node before an existing child. It must be the + case that `refChild' is a child of this node; if not, *note + ValueError: 1fb. is raised. `newChild' is returned. If `refChild' + is ‘None’, it inserts `newChild' at the end of the children’s list. + + -- Method: Node.removeChild (oldChild) + + Remove a child node. `oldChild' must be a child of this node; if + not, *note ValueError: 1fb. is raised. `oldChild' is returned on + success. If `oldChild' will not be used further, its ‘unlink()’ + method should be called. + + -- Method: Node.replaceChild (newChild, oldChild) + + Replace an existing node with a new node. It must be the case that + `oldChild' is a child of this node; if not, *note ValueError: 1fb. + is raised. + + -- Method: Node.normalize () + + Join adjacent text nodes so that all stretches of text are stored + as single ‘Text’ instances. This simplifies processing text from a + DOM tree for many applications. + + -- Method: Node.cloneNode (deep) + + Clone this node. Setting `deep' means to clone all child nodes as + well. This returns the clone. + + +File: python.info, Node: NodeList Objects, Next: DocumentType Objects, Prev: Node Objects, Up: Objects in the DOM + +5.20.6.5 NodeList Objects +......................... + +A ‘NodeList’ represents a sequence of nodes. These objects are used in +two ways in the DOM Core recommendation: an ‘Element’ object provides +one as its list of child nodes, and the ‘getElementsByTagName()’ and +‘getElementsByTagNameNS()’ methods of ‘Node’ return objects with this +interface to represent query results. + +The DOM Level 2 recommendation defines one method and one attribute for +these objects: + + -- Method: NodeList.item (i) + + Return the `i'’th item from the sequence, if there is one, or + ‘None’. The index `i' is not allowed to be less than zero or + greater than or equal to the length of the sequence. + + -- Attribute: NodeList.length + + The number of nodes in the sequence. + +In addition, the Python DOM interface requires that some additional +support is provided to allow ‘NodeList’ objects to be used as Python +sequences. All ‘NodeList’ implementations must include support for +*note __len__(): dcb. and *note __getitem__(): 27c.; this allows +iteration over the ‘NodeList’ in *note for: c30. statements and proper +support for the *note len(): 150. built-in function. + +If a DOM implementation supports modification of the document, the +‘NodeList’ implementation must also support the *note __setitem__(): +c62. and *note __delitem__(): c63. methods. + + +File: python.info, Node: DocumentType Objects, Next: Document Objects, Prev: NodeList Objects, Up: Objects in the DOM + +5.20.6.6 DocumentType Objects +............................. + +Information about the notations and entities declared by a document +(including the external subset if the parser uses it and can provide the +information) is available from a ‘DocumentType’ object. The +‘DocumentType’ for a document is available from the ‘Document’ object’s +‘doctype’ attribute; if there is no ‘DOCTYPE’ declaration for the +document, the document’s ‘doctype’ attribute will be set to ‘None’ +instead of an instance of this interface. + +‘DocumentType’ is a specialization of ‘Node’, and adds the following +attributes: + + -- Attribute: DocumentType.publicId + + The public identifier for the external subset of the document type + definition. This will be a string or ‘None’. + + -- Attribute: DocumentType.systemId + + The system identifier for the external subset of the document type + definition. This will be a URI as a string, or ‘None’. + + -- Attribute: DocumentType.internalSubset + + A string giving the complete internal subset from the document. + This does not include the brackets which enclose the subset. If + the document has no internal subset, this should be ‘None’. + + -- Attribute: DocumentType.name + + The name of the root element as given in the ‘DOCTYPE’ declaration, + if present. + + -- Attribute: DocumentType.entities + + This is a ‘NamedNodeMap’ giving the definitions of external + entities. For entity names defined more than once, only the first + definition is provided (others are ignored as required by the XML + recommendation). This may be ‘None’ if the information is not + provided by the parser, or if no entities are defined. + + -- Attribute: DocumentType.notations + + This is a ‘NamedNodeMap’ giving the definitions of notations. For + notation names defined more than once, only the first definition is + provided (others are ignored as required by the XML + recommendation). This may be ‘None’ if the information is not + provided by the parser, or if no notations are defined. + + +File: python.info, Node: Document Objects, Next: Element Objects<2>, Prev: DocumentType Objects, Up: Objects in the DOM + +5.20.6.7 Document Objects +......................... + +A ‘Document’ represents an entire XML document, including its +constituent elements, attributes, processing instructions, comments etc. +Remember that it inherits properties from ‘Node’. + + -- Attribute: Document.documentElement + + The one and only root element of the document. + + -- Method: Document.createElement (tagName) + + Create and return a new element node. The element is not inserted + into the document when it is created. You need to explicitly + insert it with one of the other methods such as ‘insertBefore()’ or + ‘appendChild()’. + + -- Method: Document.createElementNS (namespaceURI, tagName) + + Create and return a new element with a namespace. The `tagName' + may have a prefix. The element is not inserted into the document + when it is created. You need to explicitly insert it with one of + the other methods such as ‘insertBefore()’ or ‘appendChild()’. + + -- Method: Document.createTextNode (data) + + Create and return a text node containing the data passed as a + parameter. As with the other creation methods, this one does not + insert the node into the tree. + + -- Method: Document.createComment (data) + + Create and return a comment node containing the data passed as a + parameter. As with the other creation methods, this one does not + insert the node into the tree. + + -- Method: Document.createProcessingInstruction (target, data) + + Create and return a processing instruction node containing the + `target' and `data' passed as parameters. As with the other + creation methods, this one does not insert the node into the tree. + + -- Method: Document.createAttribute (name) + + Create and return an attribute node. This method does not + associate the attribute node with any particular element. You must + use ‘setAttributeNode()’ on the appropriate ‘Element’ object to use + the newly created attribute instance. + + -- Method: Document.createAttributeNS (namespaceURI, qualifiedName) + + Create and return an attribute node with a namespace. The + `tagName' may have a prefix. This method does not associate the + attribute node with any particular element. You must use + ‘setAttributeNode()’ on the appropriate ‘Element’ object to use the + newly created attribute instance. + + -- Method: Document.getElementsByTagName (tagName) + + Search for all descendants (direct children, children’s children, + etc.) with a particular element type name. + + -- Method: Document.getElementsByTagNameNS (namespaceURI, localName) + + Search for all descendants (direct children, children’s children, + etc.) with a particular namespace URI and localname. The + localname is the part of the namespace after the prefix. + + +File: python.info, Node: Element Objects<2>, Next: Attr Objects, Prev: Document Objects, Up: Objects in the DOM + +5.20.6.8 Element Objects +........................ + +‘Element’ is a subclass of ‘Node’, so inherits all the attributes of +that class. + + -- Attribute: Element.tagName + + The element type name. In a namespace-using document it may have + colons in it. The value is a string. + + -- Method: Element.getElementsByTagName (tagName) + + Same as equivalent method in the ‘Document’ class. + + -- Method: Element.getElementsByTagNameNS (namespaceURI, localName) + + Same as equivalent method in the ‘Document’ class. + + -- Method: Element.hasAttribute (name) + + Return ‘True’ if the element has an attribute named by `name'. + + -- Method: Element.hasAttributeNS (namespaceURI, localName) + + Return ‘True’ if the element has an attribute named by + `namespaceURI' and `localName'. + + -- Method: Element.getAttribute (name) + + Return the value of the attribute named by `name' as a string. If + no such attribute exists, an empty string is returned, as if the + attribute had no value. + + -- Method: Element.getAttributeNode (attrname) + + Return the ‘Attr’ node for the attribute named by `attrname'. + + -- Method: Element.getAttributeNS (namespaceURI, localName) + + Return the value of the attribute named by `namespaceURI' and + `localName' as a string. If no such attribute exists, an empty + string is returned, as if the attribute had no value. + + -- Method: Element.getAttributeNodeNS (namespaceURI, localName) + + Return an attribute value as a node, given a `namespaceURI' and + `localName'. + + -- Method: Element.removeAttribute (name) + + Remove an attribute by name. If there is no matching attribute, a + *note NotFoundErr: 27a7. is raised. + + -- Method: Element.removeAttributeNode (oldAttr) + + Remove and return `oldAttr' from the attribute list, if present. + If `oldAttr' is not present, *note NotFoundErr: 27a7. is raised. + + -- Method: Element.removeAttributeNS (namespaceURI, localName) + + Remove an attribute by name. Note that it uses a localName, not a + qname. No exception is raised if there is no matching attribute. + + -- Method: Element.setAttribute (name, value) + + Set an attribute value from a string. + + -- Method: Element.setAttributeNode (newAttr) + + Add a new attribute node to the element, replacing an existing + attribute if necessary if the ‘name’ attribute matches. If a + replacement occurs, the old attribute node will be returned. If + `newAttr' is already in use, *note InuseAttributeErr: 27ac. will be + raised. + + -- Method: Element.setAttributeNodeNS (newAttr) + + Add a new attribute node to the element, replacing an existing + attribute if necessary if the ‘namespaceURI’ and ‘localName’ + attributes match. If a replacement occurs, the old attribute node + will be returned. If `newAttr' is already in use, *note + InuseAttributeErr: 27ac. will be raised. + + -- Method: Element.setAttributeNS (namespaceURI, qname, value) + + Set an attribute value from a string, given a `namespaceURI' and a + `qname'. Note that a qname is the whole attribute name. This is + different than above. + + +File: python.info, Node: Attr Objects, Next: NamedNodeMap Objects, Prev: Element Objects<2>, Up: Objects in the DOM + +5.20.6.9 Attr Objects +..................... + +‘Attr’ inherits from ‘Node’, so inherits all its attributes. + + -- Attribute: Attr.name + + The attribute name. In a namespace-using document it may include a + colon. + + -- Attribute: Attr.localName + + The part of the name following the colon if there is one, else the + entire name. This is a read-only attribute. + + -- Attribute: Attr.prefix + + The part of the name preceding the colon if there is one, else the + empty string. + + -- Attribute: Attr.value + + The text value of the attribute. This is a synonym for the + ‘nodeValue’ attribute. + + +File: python.info, Node: NamedNodeMap Objects, Next: Comment Objects, Prev: Attr Objects, Up: Objects in the DOM + +5.20.6.10 NamedNodeMap Objects +.............................. + +‘NamedNodeMap’ does `not' inherit from ‘Node’. + + -- Attribute: NamedNodeMap.length + + The length of the attribute list. + + -- Method: NamedNodeMap.item (index) + + Return an attribute with a particular index. The order you get the + attributes in is arbitrary but will be consistent for the life of a + DOM. Each item is an attribute node. Get its value with the + ‘value’ attribute. + +There are also experimental methods that give this class more mapping +behavior. You can use them or you can use the standardized +‘getAttribute*()’ family of methods on the ‘Element’ objects. + + +File: python.info, Node: Comment Objects, Next: Text and CDATASection Objects, Prev: NamedNodeMap Objects, Up: Objects in the DOM + +5.20.6.11 Comment Objects +......................... + +‘Comment’ represents a comment in the XML document. It is a subclass of +‘Node’, but cannot have child nodes. + + -- Attribute: Comment.data + + The content of the comment as a string. The attribute contains all + characters between the leading ‘’, but + does not include them. + + +File: python.info, Node: Text and CDATASection Objects, Next: ProcessingInstruction Objects, Prev: Comment Objects, Up: Objects in the DOM + +5.20.6.12 Text and CDATASection Objects +....................................... + +The ‘Text’ interface represents text in the XML document. If the parser +and DOM implementation support the DOM’s XML extension, portions of the +text enclosed in CDATA marked sections are stored in ‘CDATASection’ +objects. These two interfaces are identical, but provide different +values for the ‘nodeType’ attribute. + +These interfaces extend the ‘Node’ interface. They cannot have child +nodes. + + -- Attribute: Text.data + + The content of the text node as a string. + + Note: The use of a ‘CDATASection’ node does not indicate that the + node represents a complete CDATA marked section, only that the + content of the node was part of a CDATA section. A single CDATA + section may be represented by more than one node in the document + tree. There is no way to determine whether two adjacent + ‘CDATASection’ nodes represent different CDATA marked sections. + + +File: python.info, Node: ProcessingInstruction Objects, Next: Exceptions<16>, Prev: Text and CDATASection Objects, Up: Objects in the DOM + +5.20.6.13 ProcessingInstruction Objects +....................................... + +Represents a processing instruction in the XML document; this inherits +from the ‘Node’ interface and cannot have child nodes. + + -- Attribute: ProcessingInstruction.target + + The content of the processing instruction up to the first + whitespace character. This is a read-only attribute. + + -- Attribute: ProcessingInstruction.data + + The content of the processing instruction following the first + whitespace character. + + +File: python.info, Node: Exceptions<16>, Prev: ProcessingInstruction Objects, Up: Objects in the DOM + +5.20.6.14 Exceptions +.................... + +The DOM Level 2 recommendation defines a single exception, *note +DOMException: 27c1, and a number of constants that allow applications to +determine what sort of error occurred. *note DOMException: 27c1. +instances carry a *note code: 1b. attribute that provides the +appropriate value for the specific exception. + +The Python DOM interface provides the constants, but also expands the +set of exceptions so that a specific exception exists for each of the +exception codes defined by the DOM. The implementations must raise the +appropriate specific exception, each of which carries the appropriate +value for the *note code: 1b. attribute. + + -- Exception: xml.dom.DOMException + + Base exception class used for all specific DOM exceptions. This + exception class cannot be directly instantiated. + + -- Exception: xml.dom.DomstringSizeErr + + Raised when a specified range of text does not fit into a string. + This is not known to be used in the Python DOM implementations, but + may be received from DOM implementations not written in Python. + + -- Exception: xml.dom.HierarchyRequestErr + + Raised when an attempt is made to insert a node where the node type + is not allowed. + + -- Exception: xml.dom.IndexSizeErr + + Raised when an index or size parameter to a method is negative or + exceeds the allowed values. + + -- Exception: xml.dom.InuseAttributeErr + + Raised when an attempt is made to insert an ‘Attr’ node that is + already present elsewhere in the document. + + -- Exception: xml.dom.InvalidAccessErr + + Raised if a parameter or an operation is not supported on the + underlying object. + + -- Exception: xml.dom.InvalidCharacterErr + + This exception is raised when a string parameter contains a + character that is not permitted in the context it’s being used in + by the XML 1.0 recommendation. For example, attempting to create + an ‘Element’ node with a space in the element type name will cause + this error to be raised. + + -- Exception: xml.dom.InvalidModificationErr + + Raised when an attempt is made to modify the type of a node. + + -- Exception: xml.dom.InvalidStateErr + + Raised when an attempt is made to use an object that is not defined + or is no longer usable. + + -- Exception: xml.dom.NamespaceErr + + If an attempt is made to change any object in a way that is not + permitted with regard to the Namespaces in XML(1) recommendation, + this exception is raised. + + -- Exception: xml.dom.NotFoundErr + + Exception when a node does not exist in the referenced context. + For example, ‘NamedNodeMap.removeNamedItem()’ will raise this if + the node passed in does not exist in the map. + + -- Exception: xml.dom.NotSupportedErr + + Raised when the implementation does not support the requested type + of object or operation. + + -- Exception: xml.dom.NoDataAllowedErr + + This is raised if data is specified for a node which does not + support data. + + -- Exception: xml.dom.NoModificationAllowedErr + + Raised on attempts to modify an object where modifications are not + allowed (such as for read-only nodes). + + -- Exception: xml.dom.SyntaxErr + + Raised when an invalid or illegal string is specified. + + -- Exception: xml.dom.WrongDocumentErr + + Raised when a node is inserted in a different document than it + currently belongs to, and the implementation does not support + migrating the node from one document to the other. + +The exception codes defined in the DOM recommendation map to the +exceptions described above according to this table: + +Constant Exception + +--------------------------------------------------------------------------------- + +‘DOMSTRING_SIZE_ERR’ *note DomstringSizeErr: 27c2. + + +‘HIERARCHY_REQUEST_ERR’ *note HierarchyRequestErr: 27c3. + + +‘INDEX_SIZE_ERR’ *note IndexSizeErr: 27c4. + + +‘INUSE_ATTRIBUTE_ERR’ *note InuseAttributeErr: 27ac. + + +‘INVALID_ACCESS_ERR’ *note InvalidAccessErr: 27c5. + + +‘INVALID_CHARACTER_ERR’ *note InvalidCharacterErr: 27c6. + + +‘INVALID_MODIFICATION_ERR’ *note InvalidModificationErr: 27c7. + + +‘INVALID_STATE_ERR’ *note InvalidStateErr: 27c8. + + +‘NAMESPACE_ERR’ *note NamespaceErr: 27c9. + + +‘NOT_FOUND_ERR’ *note NotFoundErr: 27a7. + + +‘NOT_SUPPORTED_ERR’ *note NotSupportedErr: 27ca. + + +‘NO_DATA_ALLOWED_ERR’ *note NoDataAllowedErr: 27cb. + + +‘NO_MODIFICATION_ALLOWED_ERR’ *note NoModificationAllowedErr: 27cc. + + +‘SYNTAX_ERR’ *note SyntaxErr: 27cd. + + +‘WRONG_DOCUMENT_ERR’ *note WrongDocumentErr: 27ce. + + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/REC-xml-names/ + + +File: python.info, Node: Conformance, Prev: Objects in the DOM, Up: xml dom — The Document Object Model API + +5.20.6.15 Conformance +..................... + +This section describes the conformance requirements and relationships +between the Python DOM API, the W3C DOM recommendations, and the OMG IDL +mapping for Python. + +* Menu: + +* Type Mapping:: +* Accessor Methods:: + + +File: python.info, Node: Type Mapping, Next: Accessor Methods, Up: Conformance + +5.20.6.16 Type Mapping +...................... + +The IDL types used in the DOM specification are mapped to Python types +according to the following table. + +IDL Type Python Type + +----------------------------------------------------------------------- + +‘boolean’ ‘bool’ or ‘int’ + + +‘int’ ‘int’ + + +‘long int’ ‘int’ + + +‘unsigned int’ ‘int’ + + +‘DOMString’ ‘str’ or ‘bytes’ + + +‘null’ ‘None’ + + + +File: python.info, Node: Accessor Methods, Prev: Type Mapping, Up: Conformance + +5.20.6.17 Accessor Methods +.......................... + +The mapping from OMG IDL to Python defines accessor functions for IDL +‘attribute’ declarations in much the way the Java mapping does. Mapping +the IDL declarations + + readonly attribute string someValue; + attribute string anotherValue; + +yields three accessor functions: a “get” method for ‘someValue’ +(‘_get_someValue()’), and “get” and “set” methods for ‘anotherValue’ +(‘_get_anotherValue()’ and ‘_set_anotherValue()’). The mapping, in +particular, does not require that the IDL attributes are accessible as +normal Python attributes: ‘object.someValue’ is `not' required to work, +and may raise an *note AttributeError: 39b. + +The Python DOM API, however, `does' require that normal attribute access +work. This means that the typical surrogates generated by Python IDL +compilers are not likely to work, and wrapper objects may be needed on +the client if the DOM objects are accessed via CORBA. While this does +require some additional consideration for CORBA DOM clients, the +implementers with experience using DOM over CORBA from Python do not +consider this a problem. Attributes that are declared ‘readonly’ may +not restrict write access in all DOM implementations. + +In the Python DOM API, accessor functions are not required. If +provided, they should take the form defined by the Python IDL mapping, +but these methods are considered unnecessary since the attributes are +accessible directly from Python. “Set” accessors should never be +provided for ‘readonly’ attributes. + +The IDL definitions do not fully embody the requirements of the W3C DOM +API, such as the notion of certain objects, such as the return value of +‘getElementsByTagName()’, being “live”. The Python DOM API does not +require implementations to enforce such requirements. + + +File: python.info, Node: xml dom minidom — Minimal DOM implementation, Next: xml dom pulldom — Support for building partial DOM trees, Prev: xml dom — The Document Object Model API, Up: Structured Markup Processing Tools + +5.20.7 ‘xml.dom.minidom’ — Minimal DOM implementation +----------------------------------------------------- + +`Source code:' Lib/xml/dom/minidom.py(1) + +__________________________________________________________________ + +*note xml.dom.minidom: 135. is a minimal implementation of the Document +Object Model interface, with an API similar to that in other languages. +It is intended to be simpler than the full DOM and also significantly +smaller. Users who are not already proficient with the DOM should +consider using the *note xml.etree.ElementTree: 137. module for their +XML processing instead. + + Warning: The *note xml.dom.minidom: 135. module is not secure + against maliciously constructed data. If you need to parse + untrusted or unauthenticated data see *note XML vulnerabilities: + 26f5. + +DOM applications typically start by parsing some XML into a DOM. With +*note xml.dom.minidom: 135, this is done through the parse functions: + + from xml.dom.minidom import parse, parseString + + dom1 = parse('c:\\temp\\mydata.xml') # parse an XML file by name + + datasource = open('c:\\temp\\mydata.xml') + dom2 = parse(datasource) # parse an open file + + dom3 = parseString('Some data some more data') + +The *note parse(): 27d6. function can take either a filename or an open +file object. + + -- Function: xml.dom.minidom.parse (filename_or_file, parser=None, + bufsize=None) + + Return a ‘Document’ from the given input. `filename_or_file' may + be either a file name, or a file-like object. `parser', if given, + must be a SAX2 parser object. This function will change the + document handler of the parser and activate namespace support; + other parser configuration (like setting an entity resolver) must + have been done in advance. + +If you have XML in a string, you can use the *note parseString(): 27d7. +function instead: + + -- Function: xml.dom.minidom.parseString (string, parser=None) + + Return a ‘Document’ that represents the `string'. This method + creates an *note io.StringIO: 82d. object for the string and passes + that on to *note parse(): 27d6. + +Both functions return a ‘Document’ object representing the content of +the document. + +What the *note parse(): 27d6. and *note parseString(): 27d7. functions +do is connect an XML parser with a “DOM builder” that can accept parse +events from any SAX parser and convert them into a DOM tree. The name +of the functions are perhaps misleading, but are easy to grasp when +learning the interfaces. The parsing of the document will be completed +before these functions return; it’s simply that these functions do not +provide a parser implementation themselves. + +You can also create a ‘Document’ by calling a method on a “DOM +Implementation” object. You can get this object either by calling the +‘getDOMImplementation()’ function in the *note xml.dom: 134. package or +the *note xml.dom.minidom: 135. module. Once you have a ‘Document’, you +can add child nodes to it to populate the DOM: + + from xml.dom.minidom import getDOMImplementation + + impl = getDOMImplementation() + + newdoc = impl.createDocument(None, "some_tag", None) + top_element = newdoc.documentElement + text = newdoc.createTextNode('Some textual content.') + top_element.appendChild(text) + +Once you have a DOM document object, you can access the parts of your +XML document through its properties and methods. These properties are +defined in the DOM specification. The main property of the document +object is the ‘documentElement’ property. It gives you the main element +in the XML document: the one that holds all others. Here is an example +program: + + dom3 = parseString("Some data") + assert dom3.documentElement.tagName == "myxml" + +When you are finished with a DOM tree, you may optionally call the +‘unlink()’ method to encourage early cleanup of the now-unneeded +objects. ‘unlink()’ is an *note xml.dom.minidom: 135.-specific +extension to the DOM API that renders the node and its descendants are +essentially useless. Otherwise, Python’s garbage collector will +eventually take care of the objects in the tree. + +See also +........ + +Document Object Model (DOM) Level 1 Specification(2) + + The W3C recommendation for the DOM supported by *note + xml.dom.minidom: 135. + +* Menu: + +* DOM Objects:: +* DOM Example:: +* minidom and the DOM standard:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/xml/dom/minidom.py + + (2) https://www.w3.org/TR/REC-DOM-Level-1/ + + +File: python.info, Node: DOM Objects, Next: DOM Example, Up: xml dom minidom — Minimal DOM implementation + +5.20.7.1 DOM Objects +.................... + +The definition of the DOM API for Python is given as part of the *note +xml.dom: 134. module documentation. This section lists the differences +between the API and *note xml.dom.minidom: 135. + + -- Method: Node.unlink () + + Break internal references within the DOM so that it will be garbage + collected on versions of Python without cyclic GC. Even when cyclic + GC is available, using this can make large amounts of memory + available sooner, so calling this on DOM objects as soon as they + are no longer needed is good practice. This only needs to be + called on the ‘Document’ object, but may be called on child nodes + to discard children of that node. + + You can avoid calling this method explicitly by using the *note + with: 6e9. statement. The following code will automatically unlink + `dom' when the ‘with’ block is exited: + + with xml.dom.minidom.parse(datasource) as dom: + ... # Work with dom. + + -- Method: Node.writexml (writer, indent="", addindent="", newl="") + + Write XML to the writer object. The writer receives texts but not + bytes as input, it should have a ‘write()’ method which matches + that of the file object interface. The `indent' parameter is the + indentation of the current node. The `addindent' parameter is the + incremental indentation to use for subnodes of the current one. + The `newl' parameter specifies the string to use to terminate + newlines. + + For the ‘Document’ node, an additional keyword argument `encoding' + can be used to specify the encoding field of the XML header. + + Changed in version 3.8: The *note writexml(): 27db. method now + preserves the attribute order specified by the user. + + -- Method: Node.toxml (encoding=None) + + Return a string or byte string containing the XML represented by + the DOM node. + + With an explicit `encoding' (1) argument, the result is a byte + string in the specified encoding. With no `encoding' argument, the + result is a Unicode string, and the XML declaration in the + resulting string does not specify an encoding. Encoding this + string in an encoding other than UTF-8 is likely incorrect, since + UTF-8 is the default encoding of XML. + + Changed in version 3.8: The *note toxml(): 27dc. method now + preserves the attribute order specified by the user. + + -- Method: Node.toprettyxml (indent="\t", newl="\n", encoding=None) + + Return a pretty-printed version of the document. `indent' + specifies the indentation string and defaults to a tabulator; + `newl' specifies the string emitted at the end of each line and + defaults to ‘\n’. + + The `encoding' argument behaves like the corresponding argument of + *note toxml(): 27dc. + + Changed in version 3.8: The *note toprettyxml(): 27dd. method now + preserves the attribute order specified by the user. + + ---------- Footnotes ---------- + + (1) (1) The encoding name included in the XML output should conform +to the appropriate standards. For example, “UTF-8” is valid, but “UTF8” +is not valid in an XML document’s declaration, even though Python +accepts it as an encoding name. See +‘https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl’ and +‘https://www.iana.org/assignments/character-sets/character-sets.xhtml’. + + +File: python.info, Node: DOM Example, Next: minidom and the DOM standard, Prev: DOM Objects, Up: xml dom minidom — Minimal DOM implementation + +5.20.7.2 DOM Example +.................... + +This example program is a fairly realistic example of a simple program. +In this particular case, we do not take much advantage of the +flexibility of the DOM. + + import xml.dom.minidom + + document = """\ + + Demo slideshow + Slide title + This is a demo + Of a program for processing slides + + + Another demo slide + It is important + To have more than + one slide + + + """ + + dom = xml.dom.minidom.parseString(document) + + def getText(nodelist): + rc = [] + for node in nodelist: + if node.nodeType == node.TEXT_NODE: + rc.append(node.data) + return ''.join(rc) + + def handleSlideshow(slideshow): + print("") + handleSlideshowTitle(slideshow.getElementsByTagName("title")[0]) + slides = slideshow.getElementsByTagName("slide") + handleToc(slides) + handleSlides(slides) + print("") + + def handleSlides(slides): + for slide in slides: + handleSlide(slide) + + def handleSlide(slide): + handleSlideTitle(slide.getElementsByTagName("title")[0]) + handlePoints(slide.getElementsByTagName("point")) + + def handleSlideshowTitle(title): + print("%s" % getText(title.childNodes)) + + def handleSlideTitle(title): + print("

%s

" % getText(title.childNodes)) + + def handlePoints(points): + print("
    ") + for point in points: + handlePoint(point) + print("
") + + def handlePoint(point): + print("
  • %s
    • " % getText(point.childNodes)) + + def handleToc(slides): + for slide in slides: + title = slide.getElementsByTagName("title")[0] + print("

    %s

    " % getText(title.childNodes)) + + handleSlideshow(dom) + + +File: python.info, Node: minidom and the DOM standard, Prev: DOM Example, Up: xml dom minidom — Minimal DOM implementation + +5.20.7.3 minidom and the DOM standard +..................................... + +The *note xml.dom.minidom: 135. module is essentially a DOM +1.0-compatible DOM with some DOM 2 features (primarily namespace +features). + +Usage of the DOM interface in Python is straight-forward. The following +mapping rules apply: + + * Interfaces are accessed through instance objects. Applications + should not instantiate the classes themselves; they should use the + creator functions available on the ‘Document’ object. Derived + interfaces support all operations (and attributes) from the base + interfaces, plus any new operations. + + * Operations are used as methods. Since the DOM uses only *note in: + 7a3. parameters, the arguments are passed in normal order (from + left to right). There are no optional arguments. ‘void’ + operations return ‘None’. + + * IDL attributes map to instance attributes. For compatibility with + the OMG IDL language mapping for Python, an attribute ‘foo’ can + also be accessed through accessor methods ‘_get_foo()’ and + ‘_set_foo()’. ‘readonly’ attributes must not be changed; this is + not enforced at runtime. + + * The types ‘short int’, ‘unsigned int’, ‘unsigned long long’, and + ‘boolean’ all map to Python integer objects. + + * The type ‘DOMString’ maps to Python strings. *note + xml.dom.minidom: 135. supports either bytes or strings, but will + normally produce strings. Values of type ‘DOMString’ may also be + ‘None’ where allowed to have the IDL ‘null’ value by the DOM + specification from the W3C. + + * ‘const’ declarations map to variables in their respective scope + (e.g. ‘xml.dom.minidom.Node.PROCESSING_INSTRUCTION_NODE’); they + must not be changed. + + * ‘DOMException’ is currently not supported in *note xml.dom.minidom: + 135. Instead, *note xml.dom.minidom: 135. uses standard Python + exceptions such as *note TypeError: 192. and *note AttributeError: + 39b. + + * ‘NodeList’ objects are implemented using Python’s built-in list + type. These objects provide the interface defined in the DOM + specification, but with earlier versions of Python they do not + support the official API. They are, however, much more “Pythonic” + than the interface defined in the W3C recommendations. + +The following interfaces have no implementation in *note +xml.dom.minidom: 135.: + + * ‘DOMTimeStamp’ + + * ‘EntityReference’ + +Most of these reflect information in the XML document that is not of +general utility to most DOM users. + + +File: python.info, Node: xml dom pulldom — Support for building partial DOM trees, Next: xml sax — Support for SAX2 parsers, Prev: xml dom minidom — Minimal DOM implementation, Up: Structured Markup Processing Tools + +5.20.8 ‘xml.dom.pulldom’ — Support for building partial DOM trees +----------------------------------------------------------------- + +`Source code:' Lib/xml/dom/pulldom.py(1) + +__________________________________________________________________ + +The *note xml.dom.pulldom: 136. module provides a “pull parser” which +can also be asked to produce DOM-accessible fragments of the document +where necessary. The basic concept involves pulling “events” from a +stream of incoming XML and processing them. In contrast to SAX which +also employs an event-driven processing model together with callbacks, +the user of a pull parser is responsible for explicitly pulling events +from the stream, looping over those events until either processing is +finished or an error condition occurs. + + Warning: The *note xml.dom.pulldom: 136. module is not secure + against maliciously constructed data. If you need to parse + untrusted or unauthenticated data see *note XML vulnerabilities: + 26f5. + +Changed in version 3.7.1: The SAX parser no longer processes general +external entities by default to increase security by default. To enable +processing of external entities, pass a custom parser instance in: + + from xml.dom.pulldom import parse + from xml.sax import make_parser + from xml.sax.handler import feature_external_ges + + parser = make_parser() + parser.setFeature(feature_external_ges, True) + parse(filename, parser=parser) + +Example: + + from xml.dom import pulldom + + doc = pulldom.parse('sales_items.xml') + for event, node in doc: + if event == pulldom.START_ELEMENT and node.tagName == 'item': + if int(node.getAttribute('price')) > 50: + doc.expandNode(node) + print(node.toxml()) + +‘event’ is a constant and can be one of: + + * ‘START_ELEMENT’ + + * ‘END_ELEMENT’ + + * ‘COMMENT’ + + * ‘START_DOCUMENT’ + + * ‘END_DOCUMENT’ + + * ‘CHARACTERS’ + + * ‘PROCESSING_INSTRUCTION’ + + * ‘IGNORABLE_WHITESPACE’ + +‘node’ is an object of type ‘xml.dom.minidom.Document’, +‘xml.dom.minidom.Element’ or ‘xml.dom.minidom.Text’. + +Since the document is treated as a “flat” stream of events, the document +“tree” is implicitly traversed and the desired elements are found +regardless of their depth in the tree. In other words, one does not +need to consider hierarchical issues such as recursive searching of the +document nodes, although if the context of elements were important, one +would either need to maintain some context-related state (i.e. +remembering where one is in the document at any given point) or to make +use of the *note DOMEventStream.expandNode(): 27e4. method and switch to +DOM-related processing. + + -- Class: xml.dom.pulldom.PullDom (documentFactory=None) + + Subclass of *note xml.sax.handler.ContentHandler: 27e6. + + -- Class: xml.dom.pulldom.SAX2DOM (documentFactory=None) + + Subclass of *note xml.sax.handler.ContentHandler: 27e6. + + -- Function: xml.dom.pulldom.parse (stream_or_string, parser=None, + bufsize=None) + + Return a *note DOMEventStream: 27d. from the given input. + `stream_or_string' may be either a file name, or a file-like + object. `parser', if given, must be an *note XMLReader: 27e9. + object. This function will change the document handler of the + parser and activate namespace support; other parser configuration + (like setting an entity resolver) must have been done in advance. + +If you have XML in a string, you can use the *note parseString(): 27ea. +function instead: + + -- Function: xml.dom.pulldom.parseString (string, parser=None) + + Return a *note DOMEventStream: 27d. that represents the (Unicode) + `string'. + + -- Data: xml.dom.pulldom.default_bufsize + + Default value for the `bufsize' parameter to *note parse(): 27e8. + + The value of this variable can be changed before calling *note + parse(): 27e8. and the new value will take effect. + +* Menu: + +* DOMEventStream Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/xml/dom/pulldom.py + + +File: python.info, Node: DOMEventStream Objects, Up: xml dom pulldom — Support for building partial DOM trees + +5.20.8.1 DOMEventStream Objects +............................... + + -- Class: xml.dom.pulldom.DOMEventStream (stream, parser, bufsize) + + Deprecated since version 3.8: Support for *note sequence protocol: + 27c. is deprecated. + + -- Method: getEvent () + + Return a tuple containing `event' and the current `node' as + ‘xml.dom.minidom.Document’ if event equals ‘START_DOCUMENT’, + ‘xml.dom.minidom.Element’ if event equals ‘START_ELEMENT’ or + ‘END_ELEMENT’ or ‘xml.dom.minidom.Text’ if event equals + ‘CHARACTERS’. The current node does not contain information + about its children, unless *note expandNode(): 27e4. is + called. + + -- Method: expandNode (node) + + Expands all children of `node' into `node'. Example: + + from xml.dom import pulldom + + xml = 'Foo

    Some text

    and more

    ' + doc = pulldom.parseString(xml) + for event, node in doc: + if event == pulldom.START_ELEMENT and node.tagName == 'p': + # Following statement only prints '

    ' + print(node.toxml()) + doc.expandNode(node) + # Following statement prints node with all its children '

    Some text

    and more

    ' + print(node.toxml()) + + -- Method: reset () + + +File: python.info, Node: xml sax — Support for SAX2 parsers, Next: xml sax handler — Base classes for SAX handlers, Prev: xml dom pulldom — Support for building partial DOM trees, Up: Structured Markup Processing Tools + +5.20.9 ‘xml.sax’ — Support for SAX2 parsers +------------------------------------------- + +`Source code:' Lib/xml/sax/__init__.py(1) + +__________________________________________________________________ + +The *note xml.sax: 13b. package provides a number of modules which +implement the Simple API for XML (SAX) interface for Python. The +package itself provides the SAX exceptions and the convenience functions +which will be most used by users of the SAX API. + + Warning: The *note xml.sax: 13b. module is not secure against + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see *note XML vulnerabilities: 26f5. + +Changed in version 3.7.1: The SAX parser no longer processes general +external entities by default to increase security. Before, the parser +created network connections to fetch remote files or loaded local files +from the file system for DTD and entities. The feature can be enabled +again with method *note setFeature(): 27f2. on the parser object and +argument *note feature_external_ges: 27f3. + +The convenience functions are: + + -- Function: xml.sax.make_parser (parser_list=[]) + + Create and return a SAX *note XMLReader: 27e9. object. The first + parser found will be used. If `parser_list' is provided, it must + be an iterable of strings which name modules that have a function + named ‘create_parser()’. Modules listed in `parser_list' will be + used before modules in the default list of parsers. + + Changed in version 3.8: The `parser_list' argument can be any + iterable, not just a list. + + -- Function: xml.sax.parse (filename_or_stream, handler, + error_handler=handler.ErrorHandler()) + + Create a SAX parser and use it to parse a document. The document, + passed in as `filename_or_stream', can be a filename or a file + object. The `handler' parameter needs to be a SAX *note + ContentHandler: 27e6. instance. If `error_handler' is given, it + must be a SAX *note ErrorHandler: 27f6. instance; if omitted, *note + SAXParseException: 27f7. will be raised on all errors. There is no + return value; all work must be done by the `handler' passed in. + + -- Function: xml.sax.parseString (string, handler, + error_handler=handler.ErrorHandler()) + + Similar to *note parse(): 27f5, but parses from a buffer `string' + received as a parameter. `string' must be a *note str: 330. + instance or a *note bytes-like object: 5e8. + + Changed in version 3.5: Added support of *note str: 330. instances. + +A typical SAX application uses three kinds of objects: readers, handlers +and input sources. “Reader” in this context is another term for parser, +i.e. some piece of code that reads the bytes or characters from the +input source, and produces a sequence of events. The events then get +distributed to the handler objects, i.e. the reader invokes a method on +the handler. A SAX application must therefore obtain a reader object, +create or open the input sources, create the handlers, and connect these +objects all together. As the final step of preparation, the reader is +called to parse the input. During parsing, methods on the handler +objects are called based on structural and syntactic events from the +input data. + +For these objects, only the interfaces are relevant; they are normally +not instantiated by the application itself. Since Python does not have +an explicit notion of interface, they are formally introduced as +classes, but applications may use implementations which do not inherit +from the provided classes. The *note InputSource: 792, *note Locator: +27f8, ‘Attributes’, ‘AttributesNS’, and *note XMLReader: 27e9. +interfaces are defined in the module *note xml.sax.xmlreader: 13e. The +handler interfaces are defined in *note xml.sax.handler: 13c. For +convenience, *note InputSource: 792. (which is often instantiated +directly) and the handler classes are also available from *note xml.sax: +13b. These interfaces are described below. + +In addition to these classes, *note xml.sax: 13b. provides the following +exception classes. + + -- Exception: xml.sax.SAXException (msg, exception=None) + + Encapsulate an XML error or warning. This class can contain basic + error or warning information from either the XML parser or the + application: it can be subclassed to provide additional + functionality or to add localization. Note that although the + handlers defined in the *note ErrorHandler: 27f6. interface receive + instances of this exception, it is not required to actually raise + the exception — it is also useful as a container for information. + + When instantiated, `msg' should be a human-readable description of + the error. The optional `exception' parameter, if given, should be + ‘None’ or an exception that was caught by the parsing code and is + being passed along as information. + + This is the base class for the other SAX exception classes. + + -- Exception: xml.sax.SAXParseException (msg, exception, locator) + + Subclass of *note SAXException: 27f9. raised on parse errors. + Instances of this class are passed to the methods of the SAX *note + ErrorHandler: 27f6. interface to provide information about the + parse error. This class supports the SAX *note Locator: 27f8. + interface as well as the *note SAXException: 27f9. interface. + + -- Exception: xml.sax.SAXNotRecognizedException (msg, exception=None) + + Subclass of *note SAXException: 27f9. raised when a SAX *note + XMLReader: 27e9. is confronted with an unrecognized feature or + property. SAX applications and extensions may use this class for + similar purposes. + + -- Exception: xml.sax.SAXNotSupportedException (msg, exception=None) + + Subclass of *note SAXException: 27f9. raised when a SAX *note + XMLReader: 27e9. is asked to enable a feature that is not + supported, or to set a property to a value that the implementation + does not support. SAX applications and extensions may use this + class for similar purposes. + +See also +........ + +SAX: The Simple API for XML(2) + + This site is the focal point for the definition of the SAX API. It + provides a Java implementation and online documentation. Links to + implementations and historical information are also available. + +Module *note xml.sax.handler: 13c. + + Definitions of the interfaces for application-provided objects. + +Module *note xml.sax.saxutils: 13d. + + Convenience functions for use in SAX applications. + +Module *note xml.sax.xmlreader: 13e. + + Definitions of the interfaces for parser-provided objects. + +* Menu: + +* SAXException Objects:: + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/xml/sax/__init__.py + + (2) http://www.saxproject.org/ + + +File: python.info, Node: SAXException Objects, Up: xml sax — Support for SAX2 parsers + +5.20.9.1 SAXException Objects +............................. + +The *note SAXException: 27f9. exception class supports the following +methods: + + -- Method: SAXException.getMessage () + + Return a human-readable message describing the error condition. + + -- Method: SAXException.getException () + + Return an encapsulated exception object, or ‘None’. + + +File: python.info, Node: xml sax handler — Base classes for SAX handlers, Next: xml sax saxutils — SAX Utilities, Prev: xml sax — Support for SAX2 parsers, Up: Structured Markup Processing Tools + +5.20.10 ‘xml.sax.handler’ — Base classes for SAX handlers +--------------------------------------------------------- + +`Source code:' Lib/xml/sax/handler.py(1) + +__________________________________________________________________ + +The SAX API defines four kinds of handlers: content handlers, DTD +handlers, error handlers, and entity resolvers. Applications normally +only need to implement those interfaces whose events they are interested +in; they can implement the interfaces in a single object or in multiple +objects. Handler implementations should inherit from the base classes +provided in the module *note xml.sax.handler: 13c, so that all methods +get default implementations. + + -- Class: xml.sax.handler.ContentHandler + + This is the main callback interface in SAX, and the one most + important to applications. The order of events in this interface + mirrors the order of the information in the document. + + -- Class: xml.sax.handler.DTDHandler + + Handle DTD events. + + This interface specifies only those DTD events required for basic + parsing (unparsed entities and attributes). + + -- Class: xml.sax.handler.EntityResolver + + Basic interface for resolving entities. If you create an object + implementing this interface, then register the object with your + Parser, the parser will call the method in your object to resolve + all external entities. + + -- Class: xml.sax.handler.ErrorHandler + + Interface used by the parser to present error and warning messages + to the application. The methods of this object control whether + errors are immediately converted to exceptions or are handled in + some other way. + +In addition to these classes, *note xml.sax.handler: 13c. provides +symbolic constants for the feature and property names. + + -- Data: xml.sax.handler.feature_namespaces + + value: ‘"http://xml.org/sax/features/namespaces"’ + true: Perform Namespace processing. + false: Optionally do not perform Namespace processing (implies namespace-prefixes; default). + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.feature_namespace_prefixes + + value: ‘"http://xml.org/sax/features/namespace-prefixes"’ + true: Report the original prefixed names and attributes used for Namespace declarations. + false: Do not report attributes used for Namespace declarations, and optionally do not report original prefixed names (default). + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.feature_string_interning + + value: ‘"http://xml.org/sax/features/string-interning"’ + true: All element names, prefixes, attribute names, Namespace URIs, and local names are interned using the built-in intern function. + false: Names are not necessarily interned, although they may be (default). + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.feature_validation + + value: ‘"http://xml.org/sax/features/validation"’ + true: Report all validation errors (implies external-general-entities and external-parameter-entities). + false: Do not report validation errors. + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.feature_external_ges + + value: ‘"http://xml.org/sax/features/external-general-entities"’ + true: Include all external general (text) entities. + false: Do not include external general entities. + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.feature_external_pes + + value: ‘"http://xml.org/sax/features/external-parameter-entities"’ + true: Include all external parameter entities, including the external DTD subset. + false: Do not include any external parameter entities, even the external DTD subset. + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.all_features + + List of all features. + + -- Data: xml.sax.handler.property_lexical_handler + + value: ‘"http://xml.org/sax/properties/lexical-handler"’ + data type: xml.sax.sax2lib.LexicalHandler (not supported in Python 2) + description: An optional extension handler for lexical events like comments. + access: read/write + + -- Data: xml.sax.handler.property_declaration_handler + + value: ‘"http://xml.org/sax/properties/declaration-handler"’ + data type: xml.sax.sax2lib.DeclHandler (not supported in Python 2) + description: An optional extension handler for DTD-related events other than notations and unparsed entities. + access: read/write + + -- Data: xml.sax.handler.property_dom_node + + value: ‘"http://xml.org/sax/properties/dom-node"’ + data type: org.w3c.dom.Node (not supported in Python 2) + description: When parsing, the current DOM node being visited if this is a DOM iterator; when not parsing, the root DOM node for iteration. + access: (parsing) read-only; (not parsing) read/write + + -- Data: xml.sax.handler.property_xml_string + + value: ‘"http://xml.org/sax/properties/xml-string"’ + data type: String + description: The literal string of characters that was the source for the current event. + access: read-only + + -- Data: xml.sax.handler.all_properties + + List of all known property names. + +* Menu: + +* ContentHandler Objects:: +* DTDHandler Objects:: +* EntityResolver Objects:: +* ErrorHandler Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/xml/sax/handler.py + + +File: python.info, Node: ContentHandler Objects, Next: DTDHandler Objects, Up: xml sax handler — Base classes for SAX handlers + +5.20.10.1 ContentHandler Objects +................................ + +Users are expected to subclass *note ContentHandler: 27e6. to support +their application. The following methods are called by the parser on +the appropriate events in the input document: + + -- Method: ContentHandler.setDocumentLocator (locator) + + Called by the parser to give the application a locator for locating + the origin of document events. + + SAX parsers are strongly encouraged (though not absolutely + required) to supply a locator: if it does so, it must supply the + locator to the application by invoking this method before invoking + any of the other methods in the DocumentHandler interface. + + The locator allows the application to determine the end position of + any document-related event, even if the parser is not reporting an + error. Typically, the application will use this information for + reporting its own errors (such as character content that does not + match an application’s business rules). The information returned + by the locator is probably not sufficient for use with a search + engine. + + Note that the locator will return correct information only during + the invocation of the events in this interface. The application + should not attempt to use it at any other time. + + -- Method: ContentHandler.startDocument () + + Receive notification of the beginning of a document. + + The SAX parser will invoke this method only once, before any other + methods in this interface or in DTDHandler (except for *note + setDocumentLocator(): 2811.). + + -- Method: ContentHandler.endDocument () + + Receive notification of the end of a document. + + The SAX parser will invoke this method only once, and it will be + the last method invoked during the parse. The parser shall not + invoke this method until it has either abandoned parsing (because + of an unrecoverable error) or reached the end of input. + + -- Method: ContentHandler.startPrefixMapping (prefix, uri) + + Begin the scope of a prefix-URI Namespace mapping. + + The information from this event is not necessary for normal + Namespace processing: the SAX XML reader will automatically replace + prefixes for element and attribute names when the + ‘feature_namespaces’ feature is enabled (the default). + + There are cases, however, when applications need to use prefixes in + character data or in attribute values, where they cannot safely be + expanded automatically; the *note startPrefixMapping(): 2814. and + *note endPrefixMapping(): 2815. events supply the information to + the application to expand prefixes in those contexts itself, if + necessary. + + Note that *note startPrefixMapping(): 2814. and *note + endPrefixMapping(): 2815. events are not guaranteed to be properly + nested relative to each-other: all *note startPrefixMapping(): + 2814. events will occur before the corresponding *note + startElement(): 2816. event, and all *note endPrefixMapping(): + 2815. events will occur after the corresponding *note endElement(): + 2817. event, but their order is not guaranteed. + + -- Method: ContentHandler.endPrefixMapping (prefix) + + End the scope of a prefix-URI mapping. + + See *note startPrefixMapping(): 2814. for details. This event will + always occur after the corresponding *note endElement(): 2817. + event, but the order of *note endPrefixMapping(): 2815. events is + not otherwise guaranteed. + + -- Method: ContentHandler.startElement (name, attrs) + + Signals the start of an element in non-namespace mode. + + The `name' parameter contains the raw XML 1.0 name of the element + type as a string and the `attrs' parameter holds an object of the + ‘Attributes’ interface (see *note The Attributes Interface: 2818.) + containing the attributes of the element. The object passed as + `attrs' may be re-used by the parser; holding on to a reference to + it is not a reliable way to keep a copy of the attributes. To keep + a copy of the attributes, use the *note copy(): 26. method of the + `attrs' object. + + -- Method: ContentHandler.endElement (name) + + Signals the end of an element in non-namespace mode. + + The `name' parameter contains the name of the element type, just as + with the *note startElement(): 2816. event. + + -- Method: ContentHandler.startElementNS (name, qname, attrs) + + Signals the start of an element in namespace mode. + + The `name' parameter contains the name of the element type as a + ‘(uri, localname)’ tuple, the `qname' parameter contains the raw + XML 1.0 name used in the source document, and the `attrs' parameter + holds an instance of the ‘AttributesNS’ interface (see *note The + AttributesNS Interface: 281a.) containing the attributes of the + element. If no namespace is associated with the element, the `uri' + component of `name' will be ‘None’. The object passed as `attrs' + may be re-used by the parser; holding on to a reference to it is + not a reliable way to keep a copy of the attributes. To keep a + copy of the attributes, use the *note copy(): 26. method of the + `attrs' object. + + Parsers may set the `qname' parameter to ‘None’, unless the + ‘feature_namespace_prefixes’ feature is activated. + + -- Method: ContentHandler.endElementNS (name, qname) + + Signals the end of an element in namespace mode. + + The `name' parameter contains the name of the element type, just as + with the *note startElementNS(): 2819. method, likewise the `qname' + parameter. + + -- Method: ContentHandler.characters (content) + + Receive notification of character data. + + The Parser will call this method to report each chunk of character + data. SAX parsers may return all contiguous character data in a + single chunk, or they may split it into several chunks; however, + all of the characters in any single event must come from the same + external entity so that the Locator provides useful information. + + `content' may be a string or bytes instance; the ‘expat’ reader + module always produces strings. + + Note: The earlier SAX 1 interface provided by the Python XML + Special Interest Group used a more Java-like interface for + this method. Since most parsers used from Python did not take + advantage of the older interface, the simpler signature was + chosen to replace it. To convert old code to the new + interface, use `content' instead of slicing content with the + old `offset' and `length' parameters. + + -- Method: ContentHandler.ignorableWhitespace (whitespace) + + Receive notification of ignorable whitespace in element content. + + Validating Parsers must use this method to report each chunk of + ignorable whitespace (see the W3C XML 1.0 recommendation, section + 2.10): non-validating parsers may also use this method if they are + capable of parsing and using content models. + + SAX parsers may return all contiguous whitespace in a single chunk, + or they may split it into several chunks; however, all of the + characters in any single event must come from the same external + entity, so that the Locator provides useful information. + + -- Method: ContentHandler.processingInstruction (target, data) + + Receive notification of a processing instruction. + + The Parser will invoke this method once for each processing + instruction found: note that processing instructions may occur + before or after the main document element. + + A SAX parser should never report an XML declaration (XML 1.0, + section 2.8) or a text declaration (XML 1.0, section 4.3.1) using + this method. + + -- Method: ContentHandler.skippedEntity (name) + + Receive notification of a skipped entity. + + The Parser will invoke this method once for each entity skipped. + Non-validating processors may skip entities if they have not seen + the declarations (because, for example, the entity was declared in + an external DTD subset). All processors may skip external + entities, depending on the values of the ‘feature_external_ges’ and + the ‘feature_external_pes’ properties. + + +File: python.info, Node: DTDHandler Objects, Next: EntityResolver Objects, Prev: ContentHandler Objects, Up: xml sax handler — Base classes for SAX handlers + +5.20.10.2 DTDHandler Objects +............................ + +*note DTDHandler: 2802. instances provide the following methods: + + -- Method: DTDHandler.notationDecl (name, publicId, systemId) + + Handle a notation declaration event. + + -- Method: DTDHandler.unparsedEntityDecl (name, publicId, systemId, + ndata) + + Handle an unparsed entity declaration event. + + +File: python.info, Node: EntityResolver Objects, Next: ErrorHandler Objects, Prev: DTDHandler Objects, Up: xml sax handler — Base classes for SAX handlers + +5.20.10.3 EntityResolver Objects +................................ + + -- Method: EntityResolver.resolveEntity (publicId, systemId) + + Resolve the system identifier of an entity and return either the + system identifier to read from as a string, or an InputSource to + read from. The default implementation returns `systemId'. + + +File: python.info, Node: ErrorHandler Objects, Prev: EntityResolver Objects, Up: xml sax handler — Base classes for SAX handlers + +5.20.10.4 ErrorHandler Objects +.............................. + +Objects with this interface are used to receive error and warning +information from the *note XMLReader: 27e9. If you create an object +that implements this interface, then register the object with your *note +XMLReader: 27e9, the parser will call the methods in your object to +report all warnings and errors. There are three levels of errors +available: warnings, (possibly) recoverable errors, and unrecoverable +errors. All methods take a ‘SAXParseException’ as the only parameter. +Errors and warnings may be converted to an exception by raising the +passed-in exception object. + + -- Method: ErrorHandler.error (exception) + + Called when the parser encounters a recoverable error. If this + method does not raise an exception, parsing may continue, but + further document information should not be expected by the + application. Allowing the parser to continue may allow additional + errors to be discovered in the input document. + + -- Method: ErrorHandler.fatalError (exception) + + Called when the parser encounters an error it cannot recover from; + parsing is expected to terminate when this method returns. + + -- Method: ErrorHandler.warning (exception) + + Called when the parser presents minor warning information to the + application. Parsing is expected to continue when this method + returns, and document information will continue to be passed to the + application. Raising an exception in this method will cause + parsing to end. + + +File: python.info, Node: xml sax saxutils — SAX Utilities, Next: xml sax xmlreader — Interface for XML parsers, Prev: xml sax handler — Base classes for SAX handlers, Up: Structured Markup Processing Tools + +5.20.11 ‘xml.sax.saxutils’ — SAX Utilities +------------------------------------------ + +`Source code:' Lib/xml/sax/saxutils.py(1) + +__________________________________________________________________ + +The module *note xml.sax.saxutils: 13d. contains a number of classes and +functions that are commonly useful when creating SAX applications, +either in direct use, or as base classes. + + -- Function: xml.sax.saxutils.escape (data, entities={}) + + Escape ‘'&'’, ‘'<'’, and ‘'>'’ in a string of data. + + You can escape other strings of data by passing a dictionary as the + optional `entities' parameter. The keys and values must all be + strings; each key will be replaced with its corresponding value. + The characters ‘'&'’, ‘'<'’ and ‘'>'’ are always escaped, even if + `entities' is provided. + + -- Function: xml.sax.saxutils.unescape (data, entities={}) + + Unescape ‘'&'’, ‘'<'’, and ‘'>'’ in a string of data. + + You can unescape other strings of data by passing a dictionary as + the optional `entities' parameter. The keys and values must all be + strings; each key will be replaced with its corresponding value. + ‘'&'’, ‘'<'’, and ‘'>'’ are always unescaped, even if + `entities' is provided. + + -- Function: xml.sax.saxutils.quoteattr (data, entities={}) + + Similar to *note escape(): 282e, but also prepares `data' to be + used as an attribute value. The return value is a quoted version + of `data' with any additional required replacements. *note + quoteattr(): 2830. will select a quote character based on the + content of `data', attempting to avoid encoding any quote + characters in the string. If both single- and double-quote + characters are already in `data', the double-quote characters will + be encoded and `data' will be wrapped in double-quotes. The + resulting string can be used directly as an attribute value: + + >>> print("" % quoteattr("ab ' cd \" ef")) + + + This function is useful when generating attribute values for HTML + or any SGML using the reference concrete syntax. + + -- Class: xml.sax.saxutils.XMLGenerator (out=None, + encoding='iso-8859-1', short_empty_elements=False) + + This class implements the *note ContentHandler: 27e6. interface by + writing SAX events back into an XML document. In other words, + using an *note XMLGenerator: 2831. as the content handler will + reproduce the original document being parsed. `out' should be a + file-like object which will default to `sys.stdout'. `encoding' is + the encoding of the output stream which defaults to ‘'iso-8859-1'’. + `short_empty_elements' controls the formatting of elements that + contain no content: if ‘False’ (the default) they are emitted as a + pair of start/end tags, if set to ‘True’ they are emitted as a + single self-closed tag. + + New in version 3.2: The `short_empty_elements' parameter. + + -- Class: xml.sax.saxutils.XMLFilterBase (base) + + This class is designed to sit between an *note XMLReader: 27e9. and + the client application’s event handlers. By default, it does + nothing but pass requests up to the reader and events on to the + handlers unmodified, but subclasses can override specific methods + to modify the event stream or the configuration requests as they + pass through. + + -- Function: xml.sax.saxutils.prepare_input_source (source, base='') + + This function takes an input source and an optional base URL and + returns a fully resolved *note InputSource: 792. object ready for + reading. The input source can be given as a string, a file-like + object, or an *note InputSource: 792. object; parsers will use this + function to implement the polymorphic `source' argument to their + ‘parse()’ method. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/xml/sax/saxutils.py + + +File: python.info, Node: xml sax xmlreader — Interface for XML parsers, Next: xml parsers expat — Fast XML parsing using Expat, Prev: xml sax saxutils — SAX Utilities, Up: Structured Markup Processing Tools + +5.20.12 ‘xml.sax.xmlreader’ — Interface for XML parsers +------------------------------------------------------- + +`Source code:' Lib/xml/sax/xmlreader.py(1) + +__________________________________________________________________ + +SAX parsers implement the *note XMLReader: 27e9. interface. They are +implemented in a Python module, which must provide a function +‘create_parser()’. This function is invoked by *note +xml.sax.make_parser(): 27f4. with no arguments to create a new parser +object. + + -- Class: xml.sax.xmlreader.XMLReader + + Base class which can be inherited by SAX parsers. + + -- Class: xml.sax.xmlreader.IncrementalParser + + In some cases, it is desirable not to parse an input source at + once, but to feed chunks of the document as they get available. + Note that the reader will normally not read the entire file, but + read it in chunks as well; still ‘parse()’ won’t return until the + entire document is processed. So these interfaces should be used + if the blocking behaviour of ‘parse()’ is not desirable. + + When the parser is instantiated it is ready to begin accepting data + from the feed method immediately. After parsing has been finished + with a call to close the reset method must be called to make the + parser ready to accept new data, either from feed or using the + parse method. + + Note that these methods must `not' be called during parsing, that + is, after parse has been called and before it returns. + + By default, the class also implements the parse method of the + XMLReader interface using the feed, close and reset methods of the + IncrementalParser interface as a convenience to SAX 2.0 driver + writers. + + -- Class: xml.sax.xmlreader.Locator + + Interface for associating a SAX event with a document location. A + locator object will return valid results only during calls to + DocumentHandler methods; at any other time, the results are + unpredictable. If information is not available, methods may return + ‘None’. + + -- Class: xml.sax.xmlreader.InputSource (system_id=None) + + Encapsulation of the information needed by the *note XMLReader: + 27e9. to read entities. + + This class may include information about the public identifier, + system identifier, byte stream (possibly with character encoding + information) and/or the character stream of an entity. + + Applications will create objects of this class for use in the *note + XMLReader.parse(): 2837. method and for returning from + EntityResolver.resolveEntity. + + An *note InputSource: 792. belongs to the application, the *note + XMLReader: 27e9. is not allowed to modify *note InputSource: 792. + objects passed to it from the application, although it may make + copies and modify those. + + -- Class: xml.sax.xmlreader.AttributesImpl (attrs) + + This is an implementation of the ‘Attributes’ interface (see + section *note The Attributes Interface: 2818.). This is a + dictionary-like object which represents the element attributes in a + ‘startElement()’ call. In addition to the most useful dictionary + operations, it supports a number of other methods as described by + the interface. Objects of this class should be instantiated by + readers; `attrs' must be a dictionary-like object containing a + mapping from attribute names to attribute values. + + -- Class: xml.sax.xmlreader.AttributesNSImpl (attrs, qnames) + + Namespace-aware variant of *note AttributesImpl: 2838, which will + be passed to ‘startElementNS()’. It is derived from *note + AttributesImpl: 2838, but understands attribute names as two-tuples + of `namespaceURI' and `localname'. In addition, it provides a + number of methods expecting qualified names as they appear in the + original document. This class implements the ‘AttributesNS’ + interface (see section *note The AttributesNS Interface: 281a.). + +* Menu: + +* XMLReader Objects:: +* IncrementalParser Objects:: +* Locator Objects:: +* InputSource Objects:: +* The Attributes Interface:: +* The AttributesNS Interface:: + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/xml/sax/xmlreader.py + + +File: python.info, Node: XMLReader Objects, Next: IncrementalParser Objects, Up: xml sax xmlreader — Interface for XML parsers + +5.20.12.1 XMLReader Objects +........................... + +The *note XMLReader: 27e9. interface supports the following methods: + + -- Method: XMLReader.parse (source) + + Process an input source, producing SAX events. The `source' object + can be a system identifier (a string identifying the input source – + typically a file name or a URL), a *note pathlib.Path: 201. or + *note path-like: 36a. object, or an *note InputSource: 792. object. + When *note parse(): 2837. returns, the input is completely + processed, and the parser object can be discarded or reset. + + Changed in version 3.5: Added support of character streams. + + Changed in version 3.8: Added support of path-like objects. + + -- Method: XMLReader.getContentHandler () + + Return the current *note ContentHandler: 27e6. + + -- Method: XMLReader.setContentHandler (handler) + + Set the current *note ContentHandler: 27e6. If no *note + ContentHandler: 27e6. is set, content events will be discarded. + + -- Method: XMLReader.getDTDHandler () + + Return the current *note DTDHandler: 2802. + + -- Method: XMLReader.setDTDHandler (handler) + + Set the current *note DTDHandler: 2802. If no *note DTDHandler: + 2802. is set, DTD events will be discarded. + + -- Method: XMLReader.getEntityResolver () + + Return the current *note EntityResolver: 2803. + + -- Method: XMLReader.setEntityResolver (handler) + + Set the current *note EntityResolver: 2803. If no *note + EntityResolver: 2803. is set, attempts to resolve an external + entity will result in opening the system identifier for the entity, + and fail if it is not available. + + -- Method: XMLReader.getErrorHandler () + + Return the current *note ErrorHandler: 27f6. + + -- Method: XMLReader.setErrorHandler (handler) + + Set the current error handler. If no *note ErrorHandler: 27f6. is + set, errors will be raised as exceptions, and warnings will be + printed. + + -- Method: XMLReader.setLocale (locale) + + Allow an application to set the locale for errors and warnings. + + SAX parsers are not required to provide localization for errors and + warnings; if they cannot support the requested locale, however, + they must raise a SAX exception. Applications may request a locale + change in the middle of a parse. + + -- Method: XMLReader.getFeature (featurename) + + Return the current setting for feature `featurename'. If the + feature is not recognized, ‘SAXNotRecognizedException’ is raised. + The well-known featurenames are listed in the module *note + xml.sax.handler: 13c. + + -- Method: XMLReader.setFeature (featurename, value) + + Set the `featurename' to `value'. If the feature is not + recognized, ‘SAXNotRecognizedException’ is raised. If the feature + or its setting is not supported by the parser, + `SAXNotSupportedException' is raised. + + -- Method: XMLReader.getProperty (propertyname) + + Return the current setting for property `propertyname'. If the + property is not recognized, a ‘SAXNotRecognizedException’ is + raised. The well-known propertynames are listed in the module + *note xml.sax.handler: 13c. + + -- Method: XMLReader.setProperty (propertyname, value) + + Set the `propertyname' to `value'. If the property is not + recognized, ‘SAXNotRecognizedException’ is raised. If the property + or its setting is not supported by the parser, + `SAXNotSupportedException' is raised. + + +File: python.info, Node: IncrementalParser Objects, Next: Locator Objects, Prev: XMLReader Objects, Up: xml sax xmlreader — Interface for XML parsers + +5.20.12.2 IncrementalParser Objects +................................... + +Instances of *note IncrementalParser: 2836. offer the following +additional methods: + + -- Method: IncrementalParser.feed (data) + + Process a chunk of `data'. + + -- Method: IncrementalParser.close () + + Assume the end of the document. That will check well-formedness + conditions that can be checked only at the end, invoke handlers, + and may clean up resources allocated during parsing. + + -- Method: IncrementalParser.reset () + + This method is called after close has been called to reset the + parser so that it is ready to parse new documents. The results of + calling parse or feed after close without calling reset are + undefined. + + +File: python.info, Node: Locator Objects, Next: InputSource Objects, Prev: IncrementalParser Objects, Up: xml sax xmlreader — Interface for XML parsers + +5.20.12.3 Locator Objects +......................... + +Instances of *note Locator: 27f8. provide these methods: + + -- Method: Locator.getColumnNumber () + + Return the column number where the current event begins. + + -- Method: Locator.getLineNumber () + + Return the line number where the current event begins. + + -- Method: Locator.getPublicId () + + Return the public identifier for the current event. + + -- Method: Locator.getSystemId () + + Return the system identifier for the current event. + + +File: python.info, Node: InputSource Objects, Next: The Attributes Interface, Prev: Locator Objects, Up: xml sax xmlreader — Interface for XML parsers + +5.20.12.4 InputSource Objects +............................. + + -- Method: InputSource.setPublicId (id) + + Sets the public identifier of this *note InputSource: 792. + + -- Method: InputSource.getPublicId () + + Returns the public identifier of this *note InputSource: 792. + + -- Method: InputSource.setSystemId (id) + + Sets the system identifier of this *note InputSource: 792. + + -- Method: InputSource.getSystemId () + + Returns the system identifier of this *note InputSource: 792. + + -- Method: InputSource.setEncoding (encoding) + + Sets the character encoding of this *note InputSource: 792. + + The encoding must be a string acceptable for an XML encoding + declaration (see section 4.3.3 of the XML recommendation). + + The encoding attribute of the *note InputSource: 792. is ignored if + the *note InputSource: 792. also contains a character stream. + + -- Method: InputSource.getEncoding () + + Get the character encoding of this InputSource. + + -- Method: InputSource.setByteStream (bytefile) + + Set the byte stream (a *note binary file: 1966.) for this input + source. + + The SAX parser will ignore this if there is also a character stream + specified, but it will use a byte stream in preference to opening a + URI connection itself. + + If the application knows the character encoding of the byte stream, + it should set it with the setEncoding method. + + -- Method: InputSource.getByteStream () + + Get the byte stream for this input source. + + The getEncoding method will return the character encoding for this + byte stream, or ‘None’ if unknown. + + -- Method: InputSource.setCharacterStream (charfile) + + Set the character stream (a *note text file: f3a.) for this input + source. + + If there is a character stream specified, the SAX parser will + ignore any byte stream and will not attempt to open a URI + connection to the system identifier. + + -- Method: InputSource.getCharacterStream () + + Get the character stream for this input source. + + +File: python.info, Node: The Attributes Interface, Next: The AttributesNS Interface, Prev: InputSource Objects, Up: xml sax xmlreader — Interface for XML parsers + +5.20.12.5 The ‘Attributes’ Interface +.................................... + +‘Attributes’ objects implement a portion of the *note mapping protocol: +b39, including the methods ‘copy()’, ‘get()’, *note __contains__(): d28, +‘items()’, ‘keys()’, and ‘values()’. The following methods are also +provided: + + -- Method: Attributes.getLength () + + Return the number of attributes. + + -- Method: Attributes.getNames () + + Return the names of the attributes. + + -- Method: Attributes.getType (name) + + Returns the type of the attribute `name', which is normally + ‘'CDATA'’. + + -- Method: Attributes.getValue (name) + + Return the value of attribute `name'. + + +File: python.info, Node: The AttributesNS Interface, Prev: The Attributes Interface, Up: xml sax xmlreader — Interface for XML parsers + +5.20.12.6 The ‘AttributesNS’ Interface +...................................... + +This interface is a subtype of the ‘Attributes’ interface (see section +*note The Attributes Interface: 2818.). All methods supported by that +interface are also available on ‘AttributesNS’ objects. + +The following methods are also available: + + -- Method: AttributesNS.getValueByQName (name) + + Return the value for a qualified name. + + -- Method: AttributesNS.getNameByQName (name) + + Return the ‘(namespace, localname)’ pair for a qualified `name'. + + -- Method: AttributesNS.getQNameByName (name) + + Return the qualified name for a ‘(namespace, localname)’ pair. + + -- Method: AttributesNS.getQNames () + + Return the qualified names of all attributes. + + +File: python.info, Node: xml parsers expat — Fast XML parsing using Expat, Prev: xml sax xmlreader — Interface for XML parsers, Up: Structured Markup Processing Tools + +5.20.13 ‘xml.parsers.expat’ — Fast XML parsing using Expat +---------------------------------------------------------- + +__________________________________________________________________ + + Warning: The ‘pyexpat’ module is not secure against maliciously + constructed data. If you need to parse untrusted or + unauthenticated data see *note XML vulnerabilities: 26f5. + +The *note xml.parsers.expat: 138. module is a Python interface to the +Expat non-validating XML parser. The module provides a single extension +type, ‘xmlparser’, that represents the current state of an XML parser. +After an ‘xmlparser’ object has been created, various attributes of the +object can be set to handler functions. When an XML document is then +fed to the parser, the handler functions are called for the character +data and markup in the XML document. + +This module uses the ‘pyexpat’ module to provide access to the Expat +parser. Direct use of the ‘pyexpat’ module is deprecated. + +This module provides one exception and one type object: + + -- Exception: xml.parsers.expat.ExpatError + + The exception raised when Expat reports an error. See section + *note ExpatError Exceptions: 286b. for more information on + interpreting Expat errors. + + -- Exception: xml.parsers.expat.error + + Alias for *note ExpatError: c0d. + + -- Data: xml.parsers.expat.XMLParserType + + The type of the return values from the *note ParserCreate(): 286e. + function. + +The *note xml.parsers.expat: 138. module contains two functions: + + -- Function: xml.parsers.expat.ErrorString (errno) + + Returns an explanatory string for a given error number `errno'. + + -- Function: xml.parsers.expat.ParserCreate (encoding=None, + namespace_separator=None) + + Creates and returns a new ‘xmlparser’ object. `encoding', if + specified, must be a string naming the encoding used by the XML + data. Expat doesn’t support as many encodings as Python does, and + its repertoire of encodings can’t be extended; it supports UTF-8, + UTF-16, ISO-8859-1 (Latin1), and ASCII. If `encoding' (1) is given + it will override the implicit or explicit encoding of the document. + + Expat can optionally do XML namespace processing for you, enabled + by providing a value for `namespace_separator'. The value must be + a one-character string; a *note ValueError: 1fb. will be raised if + the string has an illegal length (‘None’ is considered the same as + omission). When namespace processing is enabled, element type + names and attribute names that belong to a namespace will be + expanded. The element name passed to the element handlers + ‘StartElementHandler’ and ‘EndElementHandler’ will be the + concatenation of the namespace URI, the namespace separator + character, and the local part of the name. If the namespace + separator is a zero byte (‘chr(0)’) then the namespace URI and the + local part will be concatenated without any separator. + + For example, if `namespace_separator' is set to a space character + (‘' '’) and the following document is parsed: + + + + + + + + ‘StartElementHandler’ will receive the following strings for each + element: + + http://default-namespace.org/ root + http://www.python.org/ns/ elem1 + elem2 + + Due to limitations in the ‘Expat’ library used by ‘pyexpat’, the + ‘xmlparser’ instance returned can only be used to parse a single + XML document. Call ‘ParserCreate’ for each document to provide + unique parser instances. + +See also +........ + +The Expat XML Parser(2) + + Home page of the Expat project. + +* Menu: + +* XMLParser Objects: XMLParser Objects<2>. +* ExpatError Exceptions:: +* Example: Example<12>. +* Content Model Descriptions:: +* Expat error constants:: + + ---------- Footnotes ---------- + + (1) (1) The encoding string included in XML output should conform to +the appropriate standards. For example, “UTF-8” is valid, but “UTF8” is +not. See +‘https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl’ and +‘https://www.iana.org/assignments/character-sets/character-sets.xhtml’. + + (2) http://www.libexpat.org/ + + +File: python.info, Node: XMLParser Objects<2>, Next: ExpatError Exceptions, Up: xml parsers expat — Fast XML parsing using Expat + +5.20.13.1 XMLParser Objects +........................... + +‘xmlparser’ objects have the following methods: + + -- Method: xmlparser.Parse (data[, isfinal]) + + Parses the contents of the string `data', calling the appropriate + handler functions to process the parsed data. `isfinal' must be + true on the final call to this method; it allows the parsing of a + single file in fragments, not the submission of multiple files. + `data' can be the empty string at any time. + + -- Method: xmlparser.ParseFile (file) + + Parse XML data reading from the object `file'. `file' only needs + to provide the ‘read(nbytes)’ method, returning the empty string + when there’s no more data. + + -- Method: xmlparser.SetBase (base) + + Sets the base to be used for resolving relative URIs in system + identifiers in declarations. Resolving relative identifiers is + left to the application: this value will be passed through as the + `base' argument to the *note ExternalEntityRefHandler(): 2875, + *note NotationDeclHandler(): 2876, and *note + UnparsedEntityDeclHandler(): 2877. functions. + + -- Method: xmlparser.GetBase () + + Returns a string containing the base set by a previous call to + *note SetBase(): 2874, or ‘None’ if *note SetBase(): 2874. hasn’t + been called. + + -- Method: xmlparser.GetInputContext () + + Returns the input data that generated the current event as a + string. The data is in the encoding of the entity which contains + the text. When called while an event handler is not active, the + return value is ‘None’. + + -- Method: xmlparser.ExternalEntityParserCreate (context[, encoding]) + + Create a “child” parser which can be used to parse an external + parsed entity referred to by content parsed by the parent parser. + The `context' parameter should be the string passed to the *note + ExternalEntityRefHandler(): 2875. handler function, described + below. The child parser is created with the *note + ordered_attributes: 287b. and *note specified_attributes: 287c. set + to the values of this parser. + + -- Method: xmlparser.SetParamEntityParsing (flag) + + Control parsing of parameter entities (including the external DTD + subset). Possible `flag' values are + ‘XML_PARAM_ENTITY_PARSING_NEVER’, + ‘XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE’ and + ‘XML_PARAM_ENTITY_PARSING_ALWAYS’. Return true if setting the flag + was successful. + + -- Method: xmlparser.UseForeignDTD ([flag]) + + Calling this with a true value for `flag' (the default) will cause + Expat to call the *note ExternalEntityRefHandler: 2875. with *note + None: 157. for all arguments to allow an alternate DTD to be + loaded. If the document does not contain a document type + declaration, the *note ExternalEntityRefHandler: 2875. will still + be called, but the *note StartDoctypeDeclHandler: 287f. and *note + EndDoctypeDeclHandler: 2880. will not be called. + + Passing a false value for `flag' will cancel a previous call that + passed a true value, but otherwise has no effect. + + This method can only be called before the *note Parse(): 2872. or + *note ParseFile(): 2873. methods are called; calling it after + either of those have been called causes *note ExpatError: c0d. to + be raised with the *note code: 1b. attribute set to + ‘errors.codes[errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING]’. + +‘xmlparser’ objects have the following attributes: + + -- Attribute: xmlparser.buffer_size + + The size of the buffer used when *note buffer_text: 2882. is true. + A new buffer size can be set by assigning a new integer value to + this attribute. When the size is changed, the buffer will be + flushed. + + -- Attribute: xmlparser.buffer_text + + Setting this to true causes the ‘xmlparser’ object to buffer + textual content returned by Expat to avoid multiple calls to the + *note CharacterDataHandler(): 2883. callback whenever possible. + This can improve performance substantially since Expat normally + breaks character data into chunks at every line ending. This + attribute is false by default, and may be changed at any time. + + -- Attribute: xmlparser.buffer_used + + If *note buffer_text: 2882. is enabled, the number of bytes stored + in the buffer. These bytes represent UTF-8 encoded text. This + attribute has no meaningful interpretation when *note buffer_text: + 2882. is false. + + -- Attribute: xmlparser.ordered_attributes + + Setting this attribute to a non-zero integer causes the attributes + to be reported as a list rather than a dictionary. The attributes + are presented in the order found in the document text. For each + attribute, two list entries are presented: the attribute name and + the attribute value. (Older versions of this module also used this + format.) By default, this attribute is false; it may be changed at + any time. + + -- Attribute: xmlparser.specified_attributes + + If set to a non-zero integer, the parser will report only those + attributes which were specified in the document instance and not + those which were derived from attribute declarations. Applications + which set this need to be especially careful to use what additional + information is available from the declarations as needed to comply + with the standards for the behavior of XML processors. By default, + this attribute is false; it may be changed at any time. + +The following attributes contain values relating to the most recent +error encountered by an ‘xmlparser’ object, and will only have correct +values once a call to ‘Parse()’ or ‘ParseFile()’ has raised an *note +xml.parsers.expat.ExpatError: c0d. exception. + + -- Attribute: xmlparser.ErrorByteIndex + + Byte index at which an error occurred. + + -- Attribute: xmlparser.ErrorCode + + Numeric code specifying the problem. This value can be passed to + the *note ErrorString(): 286f. function, or compared to one of the + constants defined in the ‘errors’ object. + + -- Attribute: xmlparser.ErrorColumnNumber + + Column number at which an error occurred. + + -- Attribute: xmlparser.ErrorLineNumber + + Line number at which an error occurred. + +The following attributes contain values relating to the current parse +location in an ‘xmlparser’ object. During a callback reporting a parse +event they indicate the location of the first of the sequence of +characters that generated the event. When called outside of a callback, +the position indicated will be just past the last parse event +(regardless of whether there was an associated callback). + + -- Attribute: xmlparser.CurrentByteIndex + + Current byte index in the parser input. + + -- Attribute: xmlparser.CurrentColumnNumber + + Current column number in the parser input. + + -- Attribute: xmlparser.CurrentLineNumber + + Current line number in the parser input. + +Here is the list of handlers that can be set. To set a handler on an +‘xmlparser’ object `o', use ‘o.handlername = func’. `handlername' must +be taken from the following list, and `func' must be a callable object +accepting the correct number of arguments. The arguments are all +strings, unless otherwise stated. + + -- Method: xmlparser.XmlDeclHandler (version, encoding, standalone) + + Called when the XML declaration is parsed. The XML declaration is + the (optional) declaration of the applicable version of the XML + recommendation, the encoding of the document text, and an optional + “standalone” declaration. `version' and `encoding' will be + strings, and `standalone' will be ‘1’ if the document is declared + standalone, ‘0’ if it is declared not to be standalone, or ‘-1’ if + the standalone clause was omitted. This is only available with + Expat version 1.95.0 or newer. + + -- Method: xmlparser.StartDoctypeDeclHandler (doctypeName, systemId, + publicId, has_internal_subset) + + Called when Expat begins parsing the document type declaration + (‘'’. + + -- Method: xmlparser.StartCdataSectionHandler () + + Called at the start of a CDATA section. This and *note + EndCdataSectionHandler: 2893. are needed to be able to identify the + syntactical start and end for CDATA sections. + + -- Method: xmlparser.EndCdataSectionHandler () + + Called at the end of a CDATA section. + + -- Method: xmlparser.DefaultHandler (data) + + Called for any characters in the XML document for which no + applicable handler has been specified. This means characters that + are part of a construct which could be reported, but for which no + handler has been supplied. + + -- Method: xmlparser.DefaultHandlerExpand (data) + + This is the same as the *note DefaultHandler(): 2898, but doesn’t + inhibit expansion of internal entities. The entity reference will + not be passed to the default handler. + + -- Method: xmlparser.NotStandaloneHandler () + + Called if the XML document hasn’t been declared as being a + standalone document. This happens when there is an external subset + or a reference to a parameter entity, but the XML declaration does + not set standalone to ‘yes’ in an XML declaration. If this handler + returns ‘0’, then the parser will raise an + ‘XML_ERROR_NOT_STANDALONE’ error. If this handler is not set, no + exception is raised by the parser for this condition. + + -- Method: xmlparser.ExternalEntityRefHandler (context, base, systemId, + publicId) + + Called for references to external entities. `base' is the current + base, as set by a previous call to *note SetBase(): 2874. The + public and system identifiers, `systemId' and `publicId', are + strings if given; if the public identifier is not given, `publicId' + will be ‘None’. The `context' value is opaque and should only be + used as described below. + + For external entities to be parsed, this handler must be + implemented. It is responsible for creating the sub-parser using + ‘ExternalEntityParserCreate(context)’, initializing it with the + appropriate callbacks, and parsing the entity. This handler should + return an integer; if it returns ‘0’, the parser will raise an + ‘XML_ERROR_EXTERNAL_ENTITY_HANDLING’ error, otherwise parsing will + continue. + + If this handler is not provided, external entities are reported by + the *note DefaultHandler: 2898. callback, if provided. + + +File: python.info, Node: ExpatError Exceptions, Next: Example<12>, Prev: XMLParser Objects<2>, Up: xml parsers expat — Fast XML parsing using Expat + +5.20.13.2 ExpatError Exceptions +............................... + +*note ExpatError: c0d. exceptions have a number of interesting +attributes: + + -- Attribute: ExpatError.code + + Expat’s internal error number for the specific error. The *note + errors.messages: 289d. dictionary maps these error numbers to + Expat’s error messages. For example: + + from xml.parsers.expat import ParserCreate, ExpatError, errors + + p = ParserCreate() + try: + p.Parse(some_xml_document) + except ExpatError as err: + print("Error:", errors.messages[err.code]) + + The *note errors: 139. module also provides error message constants + and a dictionary *note codes: 289e. mapping these messages back to + the error codes, see below. + + -- Attribute: ExpatError.lineno + + Line number on which the error was detected. The first line is + numbered ‘1’. + + -- Attribute: ExpatError.offset + + Character offset into the line where the error occurred. The first + column is numbered ‘0’. + + +File: python.info, Node: Example<12>, Next: Content Model Descriptions, Prev: ExpatError Exceptions, Up: xml parsers expat — Fast XML parsing using Expat + +5.20.13.3 Example +................. + +The following program defines three handlers that just print out their +arguments. + + import xml.parsers.expat + + # 3 handler functions + def start_element(name, attrs): + print('Start element:', name, attrs) + def end_element(name): + print('End element:', name) + def char_data(data): + print('Character data:', repr(data)) + + p = xml.parsers.expat.ParserCreate() + + p.StartElementHandler = start_element + p.EndElementHandler = end_element + p.CharacterDataHandler = char_data + + p.Parse(""" + Text goes here + More text + """, 1) + +The output from this program is: + + Start element: parent {'id': 'top'} + Start element: child1 {'name': 'paul'} + Character data: 'Text goes here' + End element: child1 + Character data: '\n' + Start element: child2 {'name': 'fred'} + Character data: 'More text' + End element: child2 + Character data: '\n' + End element: parent + + +File: python.info, Node: Content Model Descriptions, Next: Expat error constants, Prev: Example<12>, Up: xml parsers expat — Fast XML parsing using Expat + +5.20.13.4 Content Model Descriptions +.................................... + +Content models are described using nested tuples. Each tuple contains +four values: the type, the quantifier, the name, and a tuple of +children. Children are simply additional content model descriptions. + +The values of the first two fields are constants defined in the *note +xml.parsers.expat.model: 13a. module. These constants can be collected +in two groups: the model type group and the quantifier group. + +The constants in the model type group are: + + -- Data: xml.parsers.expat.model.XML_CTYPE_ANY + + The element named by the model name was declared to have a content + model of ‘ANY’. + + -- Data: xml.parsers.expat.model.XML_CTYPE_CHOICE + + The named element allows a choice from a number of options; this is + used for content models such as ‘(A | B | C)’. + + -- Data: xml.parsers.expat.model.XML_CTYPE_EMPTY + + Elements which are declared to be ‘EMPTY’ have this model type. + + -- Data: xml.parsers.expat.model.XML_CTYPE_MIXED + + -- Data: xml.parsers.expat.model.XML_CTYPE_NAME + + -- Data: xml.parsers.expat.model.XML_CTYPE_SEQ + + Models which represent a series of models which follow one after + the other are indicated with this model type. This is used for + models such as ‘(A, B, C)’. + +The constants in the quantifier group are: + + -- Data: xml.parsers.expat.model.XML_CQUANT_NONE + + No modifier is given, so it can appear exactly once, as for ‘A’. + + -- Data: xml.parsers.expat.model.XML_CQUANT_OPT + + The model is optional: it can appear once or not at all, as for + ‘A?’. + + -- Data: xml.parsers.expat.model.XML_CQUANT_PLUS + + The model must occur one or more times (like ‘A+’). + + -- Data: xml.parsers.expat.model.XML_CQUANT_REP + + The model must occur zero or more times, as for ‘A*’. + + +File: python.info, Node: Expat error constants, Prev: Content Model Descriptions, Up: xml parsers expat — Fast XML parsing using Expat + +5.20.13.5 Expat error constants +............................... + +The following constants are provided in the *note +xml.parsers.expat.errors: 139. module. These constants are useful in +interpreting some of the attributes of the ‘ExpatError’ exception +objects raised when an error has occurred. Since for backwards +compatibility reasons, the constants’ value is the error `message' and +not the numeric error `code', you do this by comparing its *note code: +1b. attribute with ‘errors.codes[errors.XML_ERROR_`CONSTANT_NAME']’. + +The ‘errors’ module has the following attributes: + + -- Data: xml.parsers.expat.errors.codes + + A dictionary mapping string descriptions to their error codes. + + New in version 3.2. + + -- Data: xml.parsers.expat.errors.messages + + A dictionary mapping numeric error codes to their string + descriptions. + + New in version 3.2. + + -- Data: xml.parsers.expat.errors.XML_ERROR_ASYNC_ENTITY + + -- Data: + xml.parsers.expat.errors.XML_ERROR_ATTRIBUTE_EXTERNAL_ENTITY_REF + + An entity reference in an attribute value referred to an external + entity instead of an internal entity. + + -- Data: xml.parsers.expat.errors.XML_ERROR_BAD_CHAR_REF + + A character reference referred to a character which is illegal in + XML (for example, character ‘0’, or ‘‘�’’). + + -- Data: xml.parsers.expat.errors.XML_ERROR_BINARY_ENTITY_REF + + An entity reference referred to an entity which was declared with a + notation, so cannot be parsed. + + -- Data: xml.parsers.expat.errors.XML_ERROR_DUPLICATE_ATTRIBUTE + + An attribute was used more than once in a start tag. + + -- Data: xml.parsers.expat.errors.XML_ERROR_INCORRECT_ENCODING + + -- Data: xml.parsers.expat.errors.XML_ERROR_INVALID_TOKEN + + Raised when an input byte could not properly be assigned to a + character; for example, a NUL byte (value ‘0’) in a UTF-8 input + stream. + + -- Data: xml.parsers.expat.errors.XML_ERROR_JUNK_AFTER_DOC_ELEMENT + + Something other than whitespace occurred after the document + element. + + -- Data: xml.parsers.expat.errors.XML_ERROR_MISPLACED_XML_PI + + An XML declaration was found somewhere other than the start of the + input data. + + -- Data: xml.parsers.expat.errors.XML_ERROR_NO_ELEMENTS + + The document contains no elements (XML requires all documents to + contain exactly one top-level element).. + + -- Data: xml.parsers.expat.errors.XML_ERROR_NO_MEMORY + + Expat was not able to allocate memory internally. + + -- Data: xml.parsers.expat.errors.XML_ERROR_PARAM_ENTITY_REF + + A parameter entity reference was found where it was not allowed. + + -- Data: xml.parsers.expat.errors.XML_ERROR_PARTIAL_CHAR + + An incomplete character was found in the input. + + -- Data: xml.parsers.expat.errors.XML_ERROR_RECURSIVE_ENTITY_REF + + An entity reference contained another reference to the same entity; + possibly via a different name, and possibly indirectly. + + -- Data: xml.parsers.expat.errors.XML_ERROR_SYNTAX + + Some unspecified syntax error was encountered. + + -- Data: xml.parsers.expat.errors.XML_ERROR_TAG_MISMATCH + + An end tag did not match the innermost open start tag. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNCLOSED_TOKEN + + Some token (such as a start tag) was not closed before the end of + the stream or the next token was encountered. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNDEFINED_ENTITY + + A reference was made to an entity which was not defined. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNKNOWN_ENCODING + + The document encoding is not supported by Expat. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNCLOSED_CDATA_SECTION + + A CDATA marked section was not closed. + + -- Data: xml.parsers.expat.errors.XML_ERROR_EXTERNAL_ENTITY_HANDLING + + -- Data: xml.parsers.expat.errors.XML_ERROR_NOT_STANDALONE + + The parser determined that the document was not “standalone” though + it declared itself to be in the XML declaration, and the + ‘NotStandaloneHandler’ was set and returned ‘0’. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNEXPECTED_STATE + + -- Data: xml.parsers.expat.errors.XML_ERROR_ENTITY_DECLARED_IN_PE + + -- Data: xml.parsers.expat.errors.XML_ERROR_FEATURE_REQUIRES_XML_DTD + + An operation was requested that requires DTD support to be compiled + in, but Expat was configured without DTD support. This should + never be reported by a standard build of the *note + xml.parsers.expat: 138. module. + + -- Data: + xml.parsers.expat.errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING + + A behavioral change was requested after parsing started that can + only be changed before parsing has started. This is (currently) + only raised by ‘UseForeignDTD()’. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNBOUND_PREFIX + + An undeclared prefix was found when namespace processing was + enabled. + + -- Data: xml.parsers.expat.errors.XML_ERROR_UNDECLARING_PREFIX + + The document attempted to remove the namespace declaration + associated with a prefix. + + -- Data: xml.parsers.expat.errors.XML_ERROR_INCOMPLETE_PE + + A parameter entity contained incomplete markup. + + -- Data: xml.parsers.expat.errors.XML_ERROR_XML_DECL + + The document contained no document element at all. + + -- Data: xml.parsers.expat.errors.XML_ERROR_TEXT_DECL + + There was an error parsing a text declaration in an external + entity. + + -- Data: xml.parsers.expat.errors.XML_ERROR_PUBLICID + + Characters were found in the public id that are not allowed. + + -- Data: xml.parsers.expat.errors.XML_ERROR_SUSPENDED + + The requested operation was made on a suspended parser, but isn’t + allowed. This includes attempts to provide additional input or to + stop the parser. + + -- Data: xml.parsers.expat.errors.XML_ERROR_NOT_SUSPENDED + + An attempt to resume the parser was made when the parser had not + been suspended. + + -- Data: xml.parsers.expat.errors.XML_ERROR_ABORTED + + This should not be reported to Python applications. + + -- Data: xml.parsers.expat.errors.XML_ERROR_FINISHED + + The requested operation was made on a parser which was finished + parsing input, but isn’t allowed. This includes attempts to + provide additional input or to stop the parser. + + -- Data: xml.parsers.expat.errors.XML_ERROR_SUSPEND_PE + + +File: python.info, Node: Internet Protocols and Support, Next: Multimedia Services, Prev: Structured Markup Processing Tools, Up: The Python Standard Library + +5.21 Internet Protocols and Support +=================================== + +The modules described in this chapter implement Internet protocols and +support for related technology. They are all implemented in Python. +Most of these modules require the presence of the system-dependent +module *note socket: ef, which is currently supported on most popular +platforms. Here is an overview: + +* Menu: + +* webbrowser — Convenient Web-browser controller:: +* cgi — Common Gateway Interface support:: +* cgitb — Traceback manager for CGI scripts:: +* wsgiref — WSGI Utilities and Reference Implementation:: +* urllib — URL handling modules:: +* urllib.request — Extensible library for opening URLs: urllib request — Extensible library for opening URLs. +* urllib.response — Response classes used by urllib: urllib response — Response classes used by urllib. +* urllib.parse — Parse URLs into components: urllib parse — Parse URLs into components. +* urllib.error — Exception classes raised by urllib.request: urllib error — Exception classes raised by urllib request. +* urllib.robotparser — Parser for robots.txt: urllib robotparser — Parser for robots txt. +* http — HTTP modules:: +* http.client — HTTP protocol client: http client — HTTP protocol client. +* ftplib — FTP protocol client:: +* poplib — POP3 protocol client:: +* imaplib — IMAP4 protocol client:: +* nntplib — NNTP protocol client:: +* smtplib — SMTP protocol client:: +* smtpd — SMTP Server:: +* telnetlib — Telnet client:: +* uuid — UUID objects according to RFC 4122:: +* socketserver — A framework for network servers:: +* http.server — HTTP servers: http server — HTTP servers. +* http.cookies — HTTP state management: http cookies — HTTP state management. +* http.cookiejar — Cookie handling for HTTP clients: http cookiejar — Cookie handling for HTTP clients. +* xmlrpc — XMLRPC server and client modules:: +* xmlrpc.client — XML-RPC client access: xmlrpc client — XML-RPC client access. +* xmlrpc.server — Basic XML-RPC servers: xmlrpc server — Basic XML-RPC servers. +* ipaddress — IPv4/IPv6 manipulation library:: + + +File: python.info, Node: webbrowser — Convenient Web-browser controller, Next: cgi — Common Gateway Interface support, Up: Internet Protocols and Support + +5.21.1 ‘webbrowser’ — Convenient Web-browser controller +------------------------------------------------------- + +`Source code:' Lib/webbrowser.py(1) + +__________________________________________________________________ + +The *note webbrowser: 129. module provides a high-level interface to +allow displaying Web-based documents to users. Under most +circumstances, simply calling the *note open(): 28d1. function from this +module will do the right thing. + +Under Unix, graphical browsers are preferred under X11, but text-mode +browsers will be used if graphical browsers are not available or an X11 +display isn’t available. If text-mode browsers are used, the calling +process will block until the user exits the browser. + +If the environment variable ‘BROWSER’ exists, it is interpreted as the +*note os.pathsep: ff0.-separated list of browsers to try ahead of the +platform defaults. When the value of a list part contains the string +‘%s’, then it is interpreted as a literal browser command line to be +used with the argument URL substituted for ‘%s’; if the part does not +contain ‘%s’, it is simply interpreted as the name of the browser to +launch. (2) + +For non-Unix platforms, or when a remote browser is available on Unix, +the controlling process will not wait for the user to finish with the +browser, but allow the remote browser to maintain its own windows on the +display. If remote browsers are not available on Unix, the controlling +process will launch a new browser and wait. + +The script ‘webbrowser’ can be used as a command-line interface for the +module. It accepts a URL as the argument. It accepts the following +optional parameters: ‘-n’ opens the URL in a new browser window, if +possible; ‘-t’ opens the URL in a new browser page (“tab”). The options +are, naturally, mutually exclusive. Usage example: + + python -m webbrowser -t "http://www.python.org" + +The following exception is defined: + + -- Exception: webbrowser.Error + + Exception raised when a browser control error occurs. + +The following functions are defined: + + -- Function: webbrowser.open (url, new=0, autoraise=True) + + Display `url' using the default browser. If `new' is 0, the `url' + is opened in the same browser window if possible. If `new' is 1, a + new browser window is opened if possible. If `new' is 2, a new + browser page (“tab”) is opened if possible. If `autoraise' is + ‘True’, the window is raised if possible (note that under many + window managers this will occur regardless of the setting of this + variable). + + Note that on some platforms, trying to open a filename using this + function, may work and start the operating system’s associated + program. However, this is neither supported nor portable. + + Raises an *note auditing event: fd1. ‘webbrowser.open’ with + argument ‘url’. + + -- Function: webbrowser.open_new (url) + + Open `url' in a new window of the default browser, if possible, + otherwise, open `url' in the only browser window. + + -- Function: webbrowser.open_new_tab (url) + + Open `url' in a new page (“tab”) of the default browser, if + possible, otherwise equivalent to *note open_new(): 28d3. + + -- Function: webbrowser.get (using=None) + + Return a controller object for the browser type `using'. If + `using' is ‘None’, return a controller for a default browser + appropriate to the caller’s environment. + + -- Function: webbrowser.register (name, constructor, instance=None, *, + preferred=False) + + Register the browser type `name'. Once a browser type is + registered, the *note get(): 28d5. function can return a controller + for that browser type. If `instance' is not provided, or is + ‘None’, `constructor' will be called without parameters to create + an instance when needed. If `instance' is provided, `constructor' + will never be called, and may be ‘None’. + + Setting `preferred' to ‘True’ makes this browser a preferred result + for a *note get(): 28d5. call with no argument. Otherwise, this + entry point is only useful if you plan to either set the ‘BROWSER’ + variable or call *note get(): 28d5. with a nonempty argument + matching the name of a handler you declare. + + Changed in version 3.7: `preferred' keyword-only parameter was + added. + +A number of browser types are predefined. This table gives the type +names that may be passed to the *note get(): 28d5. function and the +corresponding instantiations for the controller classes, all defined in +this module. + +Type Name Class Name Notes + +--------------------------------------------------------------------------------------- + +‘'mozilla'’ ‘Mozilla('mozilla')’ + + +‘'firefox'’ ‘Mozilla('mozilla')’ + + +‘'netscape'’ ‘Mozilla('netscape')’ + + +‘'galeon'’ ‘Galeon('galeon')’ + + +‘'epiphany'’ ‘Galeon('epiphany')’ + + +‘'skipstone'’ ‘BackgroundBrowser('skipstone')’ + + +‘'kfmclient'’ ‘Konqueror()’ (1) + + +‘'konqueror'’ ‘Konqueror()’ (1) + + +‘'kfm'’ ‘Konqueror()’ (1) + + +‘'mosaic'’ ‘BackgroundBrowser('mosaic')’ + + +‘'opera'’ ‘Opera()’ + + +‘'grail'’ ‘Grail()’ + + +‘'links'’ ‘GenericBrowser('links')’ + + +‘'elinks'’ ‘Elinks('elinks')’ + + +‘'lynx'’ ‘GenericBrowser('lynx')’ + + +‘'w3m'’ ‘GenericBrowser('w3m')’ + + +‘'windows-default'’ ‘WindowsDefault’ (2) + + +‘'macosx'’ ‘MacOSX('default')’ (3) + + +‘'safari'’ ‘MacOSX('safari')’ (3) + + +‘'google-chrome'’ ‘Chrome('google-chrome')’ + + +‘'chrome'’ ‘Chrome('chrome')’ + + +‘'chromium'’ ‘Chromium('chromium')’ + + +‘'chromium-browser'’ ‘Chromium('chromium-browser')’ + + +Notes: + + 1. “Konqueror” is the file manager for the KDE desktop environment for + Unix, and only makes sense to use if KDE is running. Some way of + reliably detecting KDE would be nice; the ‘KDEDIR’ variable is not + sufficient. Note also that the name “kfm” is used even when using + the ‘konqueror’ command with KDE 2 — the implementation selects the + best strategy for running Konqueror. + + 2. Only on Windows platforms. + + 3. Only on Mac OS X platform. + +New in version 3.3: Support for Chrome/Chromium has been added. + +Here are some simple examples: + + url = 'http://docs.python.org/' + + # Open URL in a new tab, if a browser window is already open. + webbrowser.open_new_tab(url) + + # Open URL in new window, raising the window if possible. + webbrowser.open_new(url) + +* Menu: + +* Browser Controller Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/webbrowser.py + + (2) (1) Executables named here without a full path will be searched +in the directories given in the ‘PATH’ environment variable. + + +File: python.info, Node: Browser Controller Objects, Up: webbrowser — Convenient Web-browser controller + +5.21.1.1 Browser Controller Objects +................................... + +Browser controllers provide these methods which parallel three of the +module-level convenience functions: + + -- Method: controller.open (url, new=0, autoraise=True) + + Display `url' using the browser handled by this controller. If + `new' is 1, a new browser window is opened if possible. If `new' + is 2, a new browser page (“tab”) is opened if possible. + + -- Method: controller.open_new (url) + + Open `url' in a new window of the browser handled by this + controller, if possible, otherwise, open `url' in the only browser + window. Alias *note open_new(): 28d3. + + -- Method: controller.open_new_tab (url) + + Open `url' in a new page (“tab”) of the browser handled by this + controller, if possible, otherwise equivalent to *note open_new(): + 28d3. + + +File: python.info, Node: cgi — Common Gateway Interface support, Next: cgitb — Traceback manager for CGI scripts, Prev: webbrowser — Convenient Web-browser controller, Up: Internet Protocols and Support + +5.21.2 ‘cgi’ — Common Gateway Interface support +----------------------------------------------- + +`Source code:' Lib/cgi.py(1) + +__________________________________________________________________ + +Support module for Common Gateway Interface (CGI) scripts. + +This module defines a number of utilities for use by CGI scripts written +in Python. + +* Menu: + +* Introduction: Introduction<10>. +* Using the cgi module:: +* Higher Level Interface:: +* Functions: Functions<8>. +* Caring about security:: +* Installing your CGI script on a Unix system:: +* Testing your CGI script:: +* Debugging CGI scripts:: +* Common problems and solutions:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/cgi.py + + +File: python.info, Node: Introduction<10>, Next: Using the cgi module, Up: cgi — Common Gateway Interface support + +5.21.2.1 Introduction +..................... + +A CGI script is invoked by an HTTP server, usually to process user input +submitted through an HTML ‘
    ’ or ‘’ element. + +Most often, CGI scripts live in the server’s special ‘cgi-bin’ +directory. The HTTP server places all sorts of information about the +request (such as the client’s hostname, the requested URL, the query +string, and lots of other goodies) in the script’s shell environment, +executes the script, and sends the script’s output back to the client. + +The script’s input is connected to the client too, and sometimes the +form data is read this way; at other times the form data is passed via +the “query string” part of the URL. This module is intended to take care +of the different cases and provide a simpler interface to the Python +script. It also provides a number of utilities that help in debugging +scripts, and the latest addition is support for file uploads from a form +(if your browser supports it). + +The output of a CGI script should consist of two sections, separated by +a blank line. The first section contains a number of headers, telling +the client what kind of data is following. Python code to generate a +minimal header section looks like this: + + print("Content-Type: text/html") # HTML is following + print() # blank line, end of headers + +The second section is usually HTML, which allows the client software to +display nicely formatted text with header, in-line images, etc. Here’s +Python code that prints a simple piece of HTML: + + print("CGI script output") + print("

    This is my first CGI script

    ") + print("Hello, world!") + + +File: python.info, Node: Using the cgi module, Next: Higher Level Interface, Prev: Introduction<10>, Up: cgi — Common Gateway Interface support + +5.21.2.2 Using the cgi module +............................. + +Begin by writing ‘import cgi’. + +When you write a new script, consider adding these lines: + + import cgitb + cgitb.enable() + +This activates a special exception handler that will display detailed +reports in the Web browser if any errors occur. If you’d rather not +show the guts of your program to users of your script, you can have the +reports saved to files instead, with code like this: + + import cgitb + cgitb.enable(display=0, logdir="/path/to/logdir") + +It’s very helpful to use this feature during script development. The +reports produced by *note cgitb: 17. provide information that can save +you a lot of time in tracking down bugs. You can always remove the +‘cgitb’ line later when you have tested your script and are confident +that it works correctly. + +To get at submitted form data, use the ‘FieldStorage’ class. If the +form contains non-ASCII characters, use the `encoding' keyword parameter +set to the value of the encoding defined for the document. It is +usually contained in the META tag in the HEAD section of the HTML +document or by the ‘Content-Type’ header). This reads the form contents +from the standard input or the environment (depending on the value of +various environment variables set according to the CGI standard). Since +it may consume standard input, it should be instantiated only once. + +The ‘FieldStorage’ instance can be indexed like a Python dictionary. It +allows membership testing with the *note in: 7a3. operator, and also +supports the standard dictionary method *note keys(): c2a. and the +built-in function *note len(): 150. Form fields containing empty +strings are ignored and do not appear in the dictionary; to keep such +values, provide a true value for the optional `keep_blank_values' +keyword parameter when creating the ‘FieldStorage’ instance. + +For instance, the following code (which assumes that the ‘Content-Type’ +header and blank line have already been printed) checks that the fields +‘name’ and ‘addr’ are both set to a non-empty string: + + form = cgi.FieldStorage() + if "name" not in form or "addr" not in form: + print("

    Error

    ") + print("Please fill in the name and addr fields.") + return + print("

    name:", form["name"].value) + print("

    addr:", form["addr"].value) + ...further form processing here... + +Here the fields, accessed through ‘form[key]’, are themselves instances +of ‘FieldStorage’ (or ‘MiniFieldStorage’, depending on the form +encoding). The ‘value’ attribute of the instance yields the string +value of the field. The ‘getvalue()’ method returns this string value +directly; it also accepts an optional second argument as a default to +return if the requested key is not present. + +If the submitted form data contains more than one field with the same +name, the object retrieved by ‘form[key]’ is not a ‘FieldStorage’ or +‘MiniFieldStorage’ instance but a list of such instances. Similarly, in +this situation, ‘form.getvalue(key)’ would return a list of strings. If +you expect this possibility (when your HTML form contains multiple +fields with the same name), use the *note getlist(): 28e2. method, which +always returns a list of values (so that you do not need to special-case +the single item case). For example, this code concatenates any number +of username fields, separated by commas: + + value = form.getlist("username") + usernames = ",".join(value) + +If a field represents an uploaded file, accessing the value via the +‘value’ attribute or the ‘getvalue()’ method reads the entire file in +memory as bytes. This may not be what you want. You can test for an +uploaded file by testing either the ‘filename’ attribute or the ‘file’ +attribute. You can then read the data from the ‘file’ attribute before +it is automatically closed as part of the garbage collection of the +‘FieldStorage’ instance (the *note read(): 702. and *note readline(): +13ae. methods will return bytes): + + fileitem = form["userfile"] + if fileitem.file: + # It's an uploaded file; count lines + linecount = 0 + while True: + line = fileitem.file.readline() + if not line: break + linecount = linecount + 1 + +‘FieldStorage’ objects also support being used in a *note with: 6e9. +statement, which will automatically close them when done. + +If an error is encountered when obtaining the contents of an uploaded +file (for example, when the user interrupts the form submission by +clicking on a Back or Cancel button) the ‘done’ attribute of the object +for the field will be set to the value -1. + +The file upload draft standard entertains the possibility of uploading +multiple files from one field (using a recursive ‘multipart/*’ +encoding). When this occurs, the item will be a dictionary-like +‘FieldStorage’ item. This can be determined by testing its ‘type’ +attribute, which should be ‘multipart/form-data’ (or perhaps another +MIME type matching ‘multipart/*’). In this case, it can be iterated +over recursively just like the top-level form object. + +When a form is submitted in the “old” format (as the query string or as +a single data part of type ‘application/x-www-form-urlencoded’), the +items will actually be instances of the class ‘MiniFieldStorage’. In +this case, the ‘list’, ‘file’, and ‘filename’ attributes are always +‘None’. + +A form submitted via POST that also has a query string will contain both +‘FieldStorage’ and ‘MiniFieldStorage’ items. + +Changed in version 3.4: The ‘file’ attribute is automatically closed +upon the garbage collection of the creating ‘FieldStorage’ instance. + +Changed in version 3.5: Added support for the context management +protocol to the ‘FieldStorage’ class. + + +File: python.info, Node: Higher Level Interface, Next: Functions<8>, Prev: Using the cgi module, Up: cgi — Common Gateway Interface support + +5.21.2.3 Higher Level Interface +............................... + +The previous section explains how to read CGI form data using the +‘FieldStorage’ class. This section describes a higher level interface +which was added to this class to allow one to do it in a more readable +and intuitive way. The interface doesn’t make the techniques described +in previous sections obsolete — they are still useful to process file +uploads efficiently, for example. + +The interface consists of two simple methods. Using the methods you can +process form data in a generic way, without the need to worry whether +only one or more values were posted under one name. + +In the previous section, you learned to write following code anytime you +expected a user to post more than one value under one name: + + item = form.getvalue("item") + if isinstance(item, list): + # The user is requesting more than one item. + else: + # The user is requesting only one item. + +This situation is common for example when a form contains a group of +multiple checkboxes with the same name: + + + + +In most situations, however, there’s only one form control with a +particular name in a form and then you expect and need only one value +associated with this name. So you write a script containing for example +this code: + + user = form.getvalue("user").upper() + +The problem with the code is that you should never expect that a client +will provide valid input to your scripts. For example, if a curious +user appends another ‘user=foo’ pair to the query string, then the +script would crash, because in this situation the ‘getvalue("user")’ +method call returns a list instead of a string. Calling the *note +upper(): 1309. method on a list is not valid (since lists do not have a +method of this name) and results in an *note AttributeError: 39b. +exception. + +Therefore, the appropriate way to read form data values was to always +use the code which checks whether the obtained value is a single value +or a list of values. That’s annoying and leads to less readable +scripts. + +A more convenient approach is to use the methods *note getfirst(): 28e4. +and *note getlist(): 28e2. provided by this higher level interface. + + -- Method: FieldStorage.getfirst (name, default=None) + + This method always returns only one value associated with form + field `name'. The method returns only the first value in case that + more values were posted under such name. Please note that the + order in which the values are received may vary from browser to + browser and should not be counted on. (1) If no such form field or + value exists then the method returns the value specified by the + optional parameter `default'. This parameter defaults to ‘None’ if + not specified. + + -- Method: FieldStorage.getlist (name) + + This method always returns a list of values associated with form + field `name'. The method returns an empty list if no such form + field or value exists for `name'. It returns a list consisting of + one item if only one such value exists. + +Using these methods you can write nice compact code: + + import cgi + form = cgi.FieldStorage() + user = form.getfirst("user", "").upper() # This way it's safe. + for item in form.getlist("item"): + do_something(item) + + ---------- Footnotes ---------- + + (1) (1) Note that some recent versions of the HTML specification do +state what order the field values should be supplied in, but knowing +whether a request was received from a conforming browser, or even from a +browser at all, is tedious and error-prone. + + +File: python.info, Node: Functions<8>, Next: Caring about security, Prev: Higher Level Interface, Up: cgi — Common Gateway Interface support + +5.21.2.4 Functions +.................. + +These are useful if you want more control, or if you want to employ some +of the algorithms implemented in this module in other circumstances. + + -- Function: cgi.parse (fp=None, environ=os.environ, + keep_blank_values=False, strict_parsing=False, separator="&") + + Parse a query in the environment or from a file (the file defaults + to ‘sys.stdin’). The `keep_blank_values', `strict_parsing' and + `separator' parameters are passed to *note urllib.parse.parse_qs(): + 2eb. unchanged. + + Changed in version 3.8.8: Added the `separator' parameter. + + -- Function: cgi.parse_multipart (fp, pdict, encoding="utf-8", + errors="replace", separator="&") + + Parse input of type ‘multipart/form-data’ (for file uploads). + Arguments are `fp' for the input file, `pdict' for a dictionary + containing other parameters in the ‘Content-Type’ header, and + `encoding', the request encoding. + + Returns a dictionary just like *note urllib.parse.parse_qs(): 2eb.: + keys are the field names, each value is a list of values for that + field. For non-file fields, the value is a list of strings. + + This is easy to use but not much good if you are expecting + megabytes to be uploaded — in that case, use the ‘FieldStorage’ + class instead which is much more flexible. + + Changed in version 3.7: Added the `encoding' and `errors' + parameters. For non-file fields, the value is now a list of + strings, not bytes. + + Changed in version 3.8.8: Added the `separator' parameter. + + -- Function: cgi.parse_header (string) + + Parse a MIME header (such as ‘Content-Type’) into a main value and + a dictionary of parameters. + + -- Function: cgi.test () + + Robust test CGI script, usable as main program. Writes minimal + HTTP headers and formats all information provided to the script in + HTML form. + + -- Function: cgi.print_environ () + + Format the shell environment in HTML. + + -- Function: cgi.print_form (form) + + Format a form in HTML. + + -- Function: cgi.print_directory () + + Format the current directory in HTML. + + -- Function: cgi.print_environ_usage () + + Print a list of useful (used by CGI) environment variables in HTML. + + +File: python.info, Node: Caring about security, Next: Installing your CGI script on a Unix system, Prev: Functions<8>, Up: cgi — Common Gateway Interface support + +5.21.2.5 Caring about security +.............................. + +There’s one important rule: if you invoke an external program (via the +*note os.system(): dc1. or *note os.popen(): 2a9. functions. or others +with similar functionality), make very sure you don’t pass arbitrary +strings received from the client to the shell. This is a well-known +security hole whereby clever hackers anywhere on the Web can exploit a +gullible CGI script to invoke arbitrary shell commands. Even parts of +the URL or field names cannot be trusted, since the request doesn’t have +to come from your form! + +To be on the safe side, if you must pass a string gotten from a form to +a shell command, you should make sure the string contains only +alphanumeric characters, dashes, underscores, and periods. + + +File: python.info, Node: Installing your CGI script on a Unix system, Next: Testing your CGI script, Prev: Caring about security, Up: cgi — Common Gateway Interface support + +5.21.2.6 Installing your CGI script on a Unix system +.................................................... + +Read the documentation for your HTTP server and check with your local +system administrator to find the directory where CGI scripts should be +installed; usually this is in a directory ‘cgi-bin’ in the server tree. + +Make sure that your script is readable and executable by “others”; the +Unix file mode should be ‘0o755’ octal (use ‘chmod 0755 filename’). +Make sure that the first line of the script contains ‘#!’ starting in +column 1 followed by the pathname of the Python interpreter, for +instance: + + #!/usr/local/bin/python + +Make sure the Python interpreter exists and is executable by “others”. + +Make sure that any files your script needs to read or write are readable +or writable, respectively, by “others” — their mode should be ‘0o644’ +for readable and ‘0o666’ for writable. This is because, for security +reasons, the HTTP server executes your script as user “nobody”, without +any special privileges. It can only read (write, execute) files that +everybody can read (write, execute). The current directory at execution +time is also different (it is usually the server’s cgi-bin directory) +and the set of environment variables is also different from what you get +when you log in. In particular, don’t count on the shell’s search path +for executables ( ‘PATH’) or the Python module search path ( *note +PYTHONPATH: 953.) to be set to anything interesting. + +If you need to load modules from a directory which is not on Python’s +default module search path, you can change the path in your script, +before importing other modules. For example: + + import sys + sys.path.insert(0, "/usr/home/joe/lib/python") + sys.path.insert(0, "/usr/local/lib/python") + +(This way, the directory inserted last will be searched first!) + +Instructions for non-Unix systems will vary; check your HTTP server’s +documentation (it will usually have a section on CGI scripts). + + +File: python.info, Node: Testing your CGI script, Next: Debugging CGI scripts, Prev: Installing your CGI script on a Unix system, Up: cgi — Common Gateway Interface support + +5.21.2.7 Testing your CGI script +................................ + +Unfortunately, a CGI script will generally not run when you try it from +the command line, and a script that works perfectly from the command +line may fail mysteriously when run from the server. There’s one reason +why you should still test your script from the command line: if it +contains a syntax error, the Python interpreter won’t execute it at all, +and the HTTP server will most likely send a cryptic error to the client. + +Assuming your script has no syntax errors, yet it does not work, you +have no choice but to read the next section. + + +File: python.info, Node: Debugging CGI scripts, Next: Common problems and solutions, Prev: Testing your CGI script, Up: cgi — Common Gateway Interface support + +5.21.2.8 Debugging CGI scripts +.............................. + +First of all, check for trivial installation errors — reading the +section above on installing your CGI script carefully can save you a lot +of time. If you wonder whether you have understood the installation +procedure correctly, try installing a copy of this module file +(‘cgi.py’) as a CGI script. When invoked as a script, the file will +dump its environment and the contents of the form in HTML form. Give it +the right mode etc, and send it a request. If it’s installed in the +standard ‘cgi-bin’ directory, it should be possible to send it a request +by entering a URL into your browser of the form: + + http://yourhostname/cgi-bin/cgi.py?name=Joe+Blow&addr=At+Home + +If this gives an error of type 404, the server cannot find the script – +perhaps you need to install it in a different directory. If it gives +another error, there’s an installation problem that you should fix +before trying to go any further. If you get a nicely formatted listing +of the environment and form content (in this example, the fields should +be listed as “addr” with value “At Home” and “name” with value “Joe +Blow”), the ‘cgi.py’ script has been installed correctly. If you follow +the same procedure for your own script, you should now be able to debug +it. + +The next step could be to call the *note cgi: 16. module’s *note test(): +105. function from your script: replace its main code with the single +statement + + cgi.test() + +This should produce the same results as those gotten from installing the +‘cgi.py’ file itself. + +When an ordinary Python script raises an unhandled exception (for +whatever reason: of a typo in a module name, a file that can’t be +opened, etc.), the Python interpreter prints a nice traceback and exits. +While the Python interpreter will still do this when your CGI script +raises an exception, most likely the traceback will end up in one of the +HTTP server’s log files, or be discarded altogether. + +Fortunately, once you have managed to get your script to execute `some' +code, you can easily send tracebacks to the Web browser using the *note +cgitb: 17. module. If you haven’t done so already, just add the lines: + + import cgitb + cgitb.enable() + +to the top of your script. Then try running it again; when a problem +occurs, you should see a detailed report that will likely make apparent +the cause of the crash. + +If you suspect that there may be a problem in importing the *note cgitb: +17. module, you can use an even more robust approach (which only uses +built-in modules): + + import sys + sys.stderr = sys.stdout + print("Content-Type: text/plain") + print() + ...your code here... + +This relies on the Python interpreter to print the traceback. The +content type of the output is set to plain text, which disables all HTML +processing. If your script works, the raw HTML will be displayed by +your client. If it raises an exception, most likely after the first two +lines have been printed, a traceback will be displayed. Because no HTML +interpretation is going on, the traceback will be readable. + + +File: python.info, Node: Common problems and solutions, Prev: Debugging CGI scripts, Up: cgi — Common Gateway Interface support + +5.21.2.9 Common problems and solutions +...................................... + + * Most HTTP servers buffer the output from CGI scripts until the + script is completed. This means that it is not possible to display + a progress report on the client’s display while the script is + running. + + * Check the installation instructions above. + + * Check the HTTP server’s log files. (‘tail -f logfile’ in a + separate window may be useful!) + + * Always check a script for syntax errors first, by doing something + like ‘python script.py’. + + * If your script does not have any syntax errors, try adding ‘import + cgitb; cgitb.enable()’ to the top of the script. + + * When invoking external programs, make sure they can be found. + Usually, this means using absolute path names — ‘PATH’ is usually + not set to a very useful value in a CGI script. + + * When reading or writing external files, make sure they can be read + or written by the userid under which your CGI script will be + running: this is typically the userid under which the web server is + running, or some explicitly specified userid for a web server’s + ‘suexec’ feature. + + * Don’t try to give a CGI script a set-uid mode. This doesn’t work + on most systems, and is a security liability as well. + + +File: python.info, Node: cgitb — Traceback manager for CGI scripts, Next: wsgiref — WSGI Utilities and Reference Implementation, Prev: cgi — Common Gateway Interface support, Up: Internet Protocols and Support + +5.21.3 ‘cgitb’ — Traceback manager for CGI scripts +-------------------------------------------------- + +`Source code:' Lib/cgitb.py(1) + +__________________________________________________________________ + +The *note cgitb: 17. module provides a special exception handler for +Python scripts. (Its name is a bit misleading. It was originally +designed to display extensive traceback information in HTML for CGI +scripts. It was later generalized to also display this information in +plain text.) After this module is activated, if an uncaught exception +occurs, a detailed, formatted report will be displayed. The report +includes a traceback showing excerpts of the source code for each level, +as well as the values of the arguments and local variables to currently +running functions, to help you debug the problem. Optionally, you can +save this information to a file instead of sending it to the browser. + +To enable this feature, simply add this to the top of your CGI script: + + import cgitb + cgitb.enable() + +The options to the *note enable(): 28f5. function control whether the +report is displayed in the browser and whether the report is logged to a +file for later analysis. + + -- Function: cgitb.enable (display=1, logdir=None, context=5, + format="html") + + This function causes the *note cgitb: 17. module to take over the + interpreter’s default handling for exceptions by setting the value + of *note sys.excepthook: e63. + + The optional argument `display' defaults to ‘1’ and can be set to + ‘0’ to suppress sending the traceback to the browser. If the + argument `logdir' is present, the traceback reports are written to + files. The value of `logdir' should be a directory where these + files will be placed. The optional argument `context' is the + number of lines of context to display around the current line of + source code in the traceback; this defaults to ‘5’. If the + optional argument `format' is ‘"html"’, the output is formatted as + HTML. Any other value forces plain text output. The default value + is ‘"html"’. + + -- Function: cgitb.text (info, context=5) + + This function handles the exception described by `info' (a 3-tuple + containing the result of *note sys.exc_info(): c5f.), formatting + its traceback as text and returning the result as a string. The + optional argument `context' is the number of lines of context to + display around the current line of source code in the traceback; + this defaults to ‘5’. + + -- Function: cgitb.html (info, context=5) + + This function handles the exception described by `info' (a 3-tuple + containing the result of *note sys.exc_info(): c5f.), formatting + its traceback as HTML and returning the result as a string. The + optional argument `context' is the number of lines of context to + display around the current line of source code in the traceback; + this defaults to ‘5’. + + -- Function: cgitb.handler (info=None) + + This function handles an exception using the default settings (that + is, show a report in the browser, but don’t log to a file). This + can be used when you’ve caught an exception and want to report it + using *note cgitb: 17. The optional `info' argument should be a + 3-tuple containing an exception type, exception value, and + traceback object, exactly like the tuple returned by *note + sys.exc_info(): c5f. If the `info' argument is not supplied, the + current exception is obtained from *note sys.exc_info(): c5f. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/cgitb.py + + +File: python.info, Node: wsgiref — WSGI Utilities and Reference Implementation, Next: urllib — URL handling modules, Prev: cgitb — Traceback manager for CGI scripts, Up: Internet Protocols and Support + +5.21.4 ‘wsgiref’ — WSGI Utilities and Reference Implementation +-------------------------------------------------------------- + +__________________________________________________________________ + +The Web Server Gateway Interface (WSGI) is a standard interface between +web server software and web applications written in Python. Having a +standard interface makes it easy to use an application that supports +WSGI with a number of different web servers. + +Only authors of web servers and programming frameworks need to know +every detail and corner case of the WSGI design. You don’t need to +understand every detail of WSGI just to install a WSGI application or to +write a web application using an existing framework. + +*note wsgiref: 12c. is a reference implementation of the WSGI +specification that can be used to add WSGI support to a web server or +framework. It provides utilities for manipulating WSGI environment +variables and response headers, base classes for implementing WSGI +servers, a demo HTTP server that serves WSGI applications, and a +validation tool that checks WSGI servers and applications for +conformance to the WSGI specification ( PEP 3333(1)). + +See wsgi.readthedocs.io(2) for more information about WSGI, and links to +tutorials and other resources. + +* Menu: + +* wsgiref.util – WSGI environment utilities: wsgiref util – WSGI environment utilities. +* wsgiref.headers – WSGI response header tools: wsgiref headers – WSGI response header tools. +* wsgiref.simple_server – a simple WSGI HTTP server: wsgiref simple_server – a simple WSGI HTTP server. +* wsgiref.validate — WSGI conformance checker: wsgiref validate — WSGI conformance checker. +* wsgiref.handlers – server/gateway base classes: wsgiref handlers – server/gateway base classes. +* Examples: Examples<20>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3333 + + (2) https://wsgi.readthedocs.io/ + + +File: python.info, Node: wsgiref util – WSGI environment utilities, Next: wsgiref headers – WSGI response header tools, Up: wsgiref — WSGI Utilities and Reference Implementation + +5.21.4.1 ‘wsgiref.util’ – WSGI environment utilities +.................................................... + +This module provides a variety of utility functions for working with +WSGI environments. A WSGI environment is a dictionary containing HTTP +request variables as described in PEP 3333(1). All of the functions +taking an `environ' parameter expect a WSGI-compliant dictionary to be +supplied; please see PEP 3333(2) for a detailed specification. + + -- Function: wsgiref.util.guess_scheme (environ) + + Return a guess for whether ‘wsgi.url_scheme’ should be “http” or + “https”, by checking for a ‘HTTPS’ environment variable in the + `environ' dictionary. The return value is a string. + + This function is useful when creating a gateway that wraps CGI or a + CGI-like protocol such as FastCGI. Typically, servers providing + such protocols will include a ‘HTTPS’ variable with a value of “1”, + “yes”, or “on” when a request is received via SSL. So, this + function returns “https” if such a value is found, and “http” + otherwise. + + -- Function: wsgiref.util.request_uri (environ, include_query=True) + + Return the full request URI, optionally including the query string, + using the algorithm found in the “URL Reconstruction” section of + PEP 3333(3). If `include_query' is false, the query string is not + included in the resulting URI. + + -- Function: wsgiref.util.application_uri (environ) + + Similar to *note request_uri(): 28fd, except that the ‘PATH_INFO’ + and ‘QUERY_STRING’ variables are ignored. The result is the base + URI of the application object addressed by the request. + + -- Function: wsgiref.util.shift_path_info (environ) + + Shift a single name from ‘PATH_INFO’ to ‘SCRIPT_NAME’ and return + the name. The `environ' dictionary is `modified' in-place; use a + copy if you need to keep the original ‘PATH_INFO’ or ‘SCRIPT_NAME’ + intact. + + If there are no remaining path segments in ‘PATH_INFO’, ‘None’ is + returned. + + Typically, this routine is used to process each portion of a + request URI path, for example to treat the path as a series of + dictionary keys. This routine modifies the passed-in environment + to make it suitable for invoking another WSGI application that is + located at the target URI. For example, if there is a WSGI + application at ‘/foo’, and the request URI path is ‘/foo/bar/baz’, + and the WSGI application at ‘/foo’ calls *note shift_path_info(): + 28ff, it will receive the string “bar”, and the environment will be + updated to be suitable for passing to a WSGI application at + ‘/foo/bar’. That is, ‘SCRIPT_NAME’ will change from ‘/foo’ to + ‘/foo/bar’, and ‘PATH_INFO’ will change from ‘/bar/baz’ to ‘/baz’. + + When ‘PATH_INFO’ is just a “/”, this routine returns an empty + string and appends a trailing slash to ‘SCRIPT_NAME’, even though + empty path segments are normally ignored, and ‘SCRIPT_NAME’ doesn’t + normally end in a slash. This is intentional behavior, to ensure + that an application can tell the difference between URIs ending in + ‘/x’ from ones ending in ‘/x/’ when using this routine to do object + traversal. + + -- Function: wsgiref.util.setup_testing_defaults (environ) + + Update `environ' with trivial defaults for testing purposes. + + This routine adds various parameters required for WSGI, including + ‘HTTP_HOST’, ‘SERVER_NAME’, ‘SERVER_PORT’, ‘REQUEST_METHOD’, + ‘SCRIPT_NAME’, ‘PATH_INFO’, and all of the PEP 3333(4)-defined + ‘wsgi.*’ variables. It only supplies default values, and does not + replace any existing settings for these variables. + + This routine is intended to make it easier for unit tests of WSGI + servers and applications to set up dummy environments. It should + NOT be used by actual WSGI servers or applications, since the data + is fake! + + Example usage: + + from wsgiref.util import setup_testing_defaults + from wsgiref.simple_server import make_server + + # A relatively simple WSGI application. It's going to print out the + # environment dictionary after being updated by setup_testing_defaults + def simple_app(environ, start_response): + setup_testing_defaults(environ) + + status = '200 OK' + headers = [('Content-type', 'text/plain; charset=utf-8')] + + start_response(status, headers) + + ret = [("%s: %s\n" % (key, value)).encode("utf-8") + for key, value in environ.items()] + return ret + + with make_server('', 8000, simple_app) as httpd: + print("Serving on port 8000...") + httpd.serve_forever() + +In addition to the environment functions above, the *note wsgiref.util: +130. module also provides these miscellaneous utilities: + + -- Function: wsgiref.util.is_hop_by_hop (header_name) + + Return ‘True’ if ‘header_name’ is an HTTP/1.1 “Hop-by-Hop” header, + as defined by RFC 2616(5). + + -- Class: wsgiref.util.FileWrapper (filelike, blksize=8192) + + A wrapper to convert a file-like object to an *note iterator: 112e. + The resulting objects support both *note __getitem__(): 27c. and + *note __iter__(): 50f. iteration styles, for compatibility with + Python 2.1 and Jython. As the object is iterated over, the + optional `blksize' parameter will be repeatedly passed to the + `filelike' object’s ‘read()’ method to obtain bytestrings to yield. + When ‘read()’ returns an empty bytestring, iteration is ended and + is not resumable. + + If `filelike' has a ‘close()’ method, the returned object will also + have a ‘close()’ method, and it will invoke the `filelike' object’s + ‘close()’ method when called. + + Example usage: + + from io import StringIO + from wsgiref.util import FileWrapper + + # We're using a StringIO-buffer for as the file-like object + filelike = StringIO("This is an example file-like object"*10) + wrapper = FileWrapper(filelike, blksize=5) + + for chunk in wrapper: + print(chunk) + + Deprecated since version 3.8: Support for *note sequence protocol: + 27c. is deprecated. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3333 + + (2) https://www.python.org/dev/peps/pep-3333 + + (3) https://www.python.org/dev/peps/pep-3333 + + (4) https://www.python.org/dev/peps/pep-3333 + + (5) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: wsgiref headers – WSGI response header tools, Next: wsgiref simple_server – a simple WSGI HTTP server, Prev: wsgiref util – WSGI environment utilities, Up: wsgiref — WSGI Utilities and Reference Implementation + +5.21.4.2 ‘wsgiref.headers’ – WSGI response header tools +....................................................... + +This module provides a single class, *note Headers: 78f, for convenient +manipulation of WSGI response headers using a mapping-like interface. + + -- Class: wsgiref.headers.Headers ([headers]) + + Create a mapping-like object wrapping `headers', which must be a + list of header name/value tuples as described in PEP 3333(1). The + default value of `headers' is an empty list. + + *note Headers: 78f. objects support typical mapping operations + including *note __getitem__(): 27c, ‘get()’, *note __setitem__(): + c62, ‘setdefault()’, *note __delitem__(): c63. and *note + __contains__(): d28. For each of these methods, the key is the + header name (treated case-insensitively), and the value is the + first value associated with that header name. Setting a header + deletes any existing values for that header, then adds a new value + at the end of the wrapped header list. Headers’ existing order is + generally maintained, with new headers added to the end of the + wrapped list. + + Unlike a dictionary, *note Headers: 78f. objects do not raise an + error when you try to get or delete a key that isn’t in the wrapped + header list. Getting a nonexistent header just returns ‘None’, and + deleting a nonexistent header does nothing. + + *note Headers: 78f. objects also support ‘keys()’, ‘values()’, and + ‘items()’ methods. The lists returned by ‘keys()’ and ‘items()’ + can include the same key more than once if there is a multi-valued + header. The ‘len()’ of a *note Headers: 78f. object is the same as + the length of its ‘items()’, which is the same as the length of the + wrapped header list. In fact, the ‘items()’ method just returns a + copy of the wrapped header list. + + Calling ‘bytes()’ on a *note Headers: 78f. object returns a + formatted bytestring suitable for transmission as HTTP response + headers. Each header is placed on a line with its value, separated + by a colon and a space. Each line is terminated by a carriage + return and line feed, and the bytestring is terminated with a blank + line. + + In addition to their mapping interface and formatting features, + *note Headers: 78f. objects also have the following methods for + querying and adding multi-valued headers, and for adding headers + with MIME parameters: + + -- Method: get_all (name) + + Return a list of all the values for the named header. + + The returned list will be sorted in the order they appeared in + the original header list or were added to this instance, and + may contain duplicates. Any fields deleted and re-inserted + are always appended to the header list. If no fields exist + with the given name, returns an empty list. + + -- Method: add_header (name, value, **_params) + + Add a (possibly multi-valued) header, with optional MIME + parameters specified via keyword arguments. + + `name' is the header field to add. Keyword arguments can be + used to set MIME parameters for the header field. Each + parameter must be a string or ‘None’. Underscores in + parameter names are converted to dashes, since dashes are + illegal in Python identifiers, but many MIME parameter names + include dashes. If the parameter value is a string, it is + added to the header value parameters in the form + ‘name="value"’. If it is ‘None’, only the parameter name is + added. (This is used for MIME parameters without a value.) + Example usage: + + h.add_header('content-disposition', 'attachment', filename='bud.gif') + + The above will add a header that looks like this: + + Content-Disposition: attachment; filename="bud.gif" + + Changed in version 3.5: `headers' parameter is optional. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3333 + + +File: python.info, Node: wsgiref simple_server – a simple WSGI HTTP server, Next: wsgiref validate — WSGI conformance checker, Prev: wsgiref headers – WSGI response header tools, Up: wsgiref — WSGI Utilities and Reference Implementation + +5.21.4.3 ‘wsgiref.simple_server’ – a simple WSGI HTTP server +............................................................ + +This module implements a simple HTTP server (based on *note http.server: +97.) that serves WSGI applications. Each server instance serves a +single WSGI application on a given host and port. If you want to serve +multiple applications on a single host and port, you should create a +WSGI application that parses ‘PATH_INFO’ to select which application to +invoke for each request. (E.g., using the ‘shift_path_info()’ function +from *note wsgiref.util: 130.) + + -- Function: wsgiref.simple_server.make_server (host, port, app, + server_class=WSGIServer, handler_class=WSGIRequestHandler) + + Create a new WSGI server listening on `host' and `port', accepting + connections for `app'. The return value is an instance of the + supplied `server_class', and will process requests using the + specified `handler_class'. `app' must be a WSGI application + object, as defined by PEP 3333(1). + + Example usage: + + from wsgiref.simple_server import make_server, demo_app + + with make_server('', 8000, demo_app) as httpd: + print("Serving HTTP on port 8000...") + + # Respond to requests until process is killed + httpd.serve_forever() + + # Alternative: serve one request, then exit + httpd.handle_request() + + -- Function: wsgiref.simple_server.demo_app (environ, start_response) + + This function is a small but complete WSGI application that returns + a text page containing the message “Hello world!” and a list of the + key/value pairs provided in the `environ' parameter. It’s useful + for verifying that a WSGI server (such as *note + wsgiref.simple_server: 12f.) is able to run a simple WSGI + application correctly. + + -- Class: wsgiref.simple_server.WSGIServer (server_address, + RequestHandlerClass) + + Create a *note WSGIServer: 2908. instance. `server_address' should + be a ‘(host,port)’ tuple, and `RequestHandlerClass' should be the + subclass of *note http.server.BaseHTTPRequestHandler: a0e. that + will be used to process requests. + + You do not normally need to call this constructor, as the *note + make_server(): 2906. function can handle all the details for you. + + *note WSGIServer: 2908. is a subclass of *note + http.server.HTTPServer: 2909, so all of its methods (such as + ‘serve_forever()’ and ‘handle_request()’) are available. *note + WSGIServer: 2908. also provides these WSGI-specific methods: + + -- Method: set_app (application) + + Sets the callable `application' as the WSGI application that + will receive requests. + + -- Method: get_app () + + Returns the currently-set application callable. + + Normally, however, you do not need to use these additional methods, + as *note set_app(): 290a. is normally called by *note + make_server(): 2906, and the *note get_app(): 290b. exists mainly + for the benefit of request handler instances. + + -- Class: wsgiref.simple_server.WSGIRequestHandler (request, + client_address, server) + + Create an HTTP handler for the given `request' (i.e. a socket), + `client_address' (a ‘(host,port)’ tuple), and `server' (*note + WSGIServer: 2908. instance). + + You do not need to create instances of this class directly; they + are automatically created as needed by *note WSGIServer: 2908. + objects. You can, however, subclass this class and supply it as a + `handler_class' to the *note make_server(): 2906. function. Some + possibly relevant methods for overriding in subclasses: + + -- Method: get_environ () + + Returns a dictionary containing the WSGI environment for a + request. The default implementation copies the contents of + the *note WSGIServer: 2908. object’s ‘base_environ’ dictionary + attribute and then adds various headers derived from the HTTP + request. Each call to this method should return a new + dictionary containing all of the relevant CGI environment + variables as specified in PEP 3333(2). + + -- Method: get_stderr () + + Return the object that should be used as the ‘wsgi.errors’ + stream. The default implementation just returns ‘sys.stderr’. + + -- Method: handle () + + Process the HTTP request. The default implementation creates + a handler instance using a *note wsgiref.handlers: 12d. class + to implement the actual WSGI application interface. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3333 + + (2) https://www.python.org/dev/peps/pep-3333 + + +File: python.info, Node: wsgiref validate — WSGI conformance checker, Next: wsgiref handlers – server/gateway base classes, Prev: wsgiref simple_server – a simple WSGI HTTP server, Up: wsgiref — WSGI Utilities and Reference Implementation + +5.21.4.4 ‘wsgiref.validate’ — WSGI conformance checker +...................................................... + +When creating new WSGI application objects, frameworks, servers, or +middleware, it can be useful to validate the new code’s conformance +using *note wsgiref.validate: 131. This module provides a function that +creates WSGI application objects that validate communications between a +WSGI server or gateway and a WSGI application object, to check both +sides for protocol conformance. + +Note that this utility does not guarantee complete PEP 3333(1) +compliance; an absence of errors from this module does not necessarily +mean that errors do not exist. However, if this module does produce an +error, then it is virtually certain that either the server or +application is not 100% compliant. + +This module is based on the ‘paste.lint’ module from Ian Bicking’s +“Python Paste” library. + + -- Function: wsgiref.validate.validator (application) + + Wrap `application' and return a new WSGI application object. The + returned application will forward all requests to the original + `application', and will check that both the `application' and the + server invoking it are conforming to the WSGI specification and to + RFC 2616(2). + + Any detected nonconformance results in an *note AssertionError: + 1223. being raised; note, however, that how these errors are + handled is server-dependent. For example, *note + wsgiref.simple_server: 12f. and other servers based on *note + wsgiref.handlers: 12d. (that don’t override the error handling + methods to do something else) will simply output a message that an + error has occurred, and dump the traceback to ‘sys.stderr’ or some + other error stream. + + This wrapper may also generate output using the *note warnings: + 126. module to indicate behaviors that are questionable but which + may not actually be prohibited by PEP 3333(3). Unless they are + suppressed using Python command-line options or the *note warnings: + 126. API, any such warnings will be written to ‘sys.stderr’ (`not' + ‘wsgi.errors’, unless they happen to be the same object). + + Example usage: + + from wsgiref.validate import validator + from wsgiref.simple_server import make_server + + # Our callable object which is intentionally not compliant to the + # standard, so the validator is going to break + def simple_app(environ, start_response): + status = '200 OK' # HTTP Status + headers = [('Content-type', 'text/plain')] # HTTP Headers + start_response(status, headers) + + # This is going to break because we need to return a list, and + # the validator is going to inform us + return b"Hello World" + + # This is the application wrapped in a validator + validator_app = validator(simple_app) + + with make_server('', 8000, validator_app) as httpd: + print("Listening on port 8000....") + httpd.serve_forever() + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3333 + + (2) https://tools.ietf.org/html/rfc2616.html + + (3) https://www.python.org/dev/peps/pep-3333 + + +File: python.info, Node: wsgiref handlers – server/gateway base classes, Next: Examples<20>, Prev: wsgiref validate — WSGI conformance checker, Up: wsgiref — WSGI Utilities and Reference Implementation + +5.21.4.5 ‘wsgiref.handlers’ – server/gateway base classes +......................................................... + +This module provides base handler classes for implementing WSGI servers +and gateways. These base classes handle most of the work of +communicating with a WSGI application, as long as they are given a +CGI-like environment, along with input, output, and error streams. + + -- Class: wsgiref.handlers.CGIHandler + + CGI-based invocation via ‘sys.stdin’, ‘sys.stdout’, ‘sys.stderr’ + and ‘os.environ’. This is useful when you have a WSGI application + and want to run it as a CGI script. Simply invoke + ‘CGIHandler().run(app)’, where ‘app’ is the WSGI application object + you wish to invoke. + + This class is a subclass of *note BaseCGIHandler: 2914. that sets + ‘wsgi.run_once’ to true, ‘wsgi.multithread’ to false, and + ‘wsgi.multiprocess’ to true, and always uses *note sys: fd. and + *note os: c4. to obtain the necessary CGI streams and environment. + + -- Class: wsgiref.handlers.IISCGIHandler + + A specialized alternative to *note CGIHandler: 2913, for use when + deploying on Microsoft’s IIS web server, without having set the + config allowPathInfo option (IIS>=7) or metabase + allowPathInfoForScriptMappings (IIS<7). + + By default, IIS gives a ‘PATH_INFO’ that duplicates the + ‘SCRIPT_NAME’ at the front, causing problems for WSGI applications + that wish to implement routing. This handler strips any such + duplicated path. + + IIS can be configured to pass the correct ‘PATH_INFO’, but this + causes another bug where ‘PATH_TRANSLATED’ is wrong. Luckily this + variable is rarely used and is not guaranteed by WSGI. On IIS<7, + though, the setting can only be made on a vhost level, affecting + all other script mappings, many of which break when exposed to the + ‘PATH_TRANSLATED’ bug. For this reason IIS<7 is almost never + deployed with the fix (Even IIS7 rarely uses it because there is + still no UI for it.). + + There is no way for CGI code to tell whether the option was set, so + a separate handler class is provided. It is used in the same way + as *note CGIHandler: 2913, i.e., by calling + ‘IISCGIHandler().run(app)’, where ‘app’ is the WSGI application + object you wish to invoke. + + New in version 3.2. + + -- Class: wsgiref.handlers.BaseCGIHandler (stdin, stdout, stderr, + environ, multithread=True, multiprocess=False) + + Similar to *note CGIHandler: 2913, but instead of using the *note + sys: fd. and *note os: c4. modules, the CGI environment and I/O + streams are specified explicitly. The `multithread' and + `multiprocess' values are used to set the ‘wsgi.multithread’ and + ‘wsgi.multiprocess’ flags for any applications run by the handler + instance. + + This class is a subclass of *note SimpleHandler: 2916. intended for + use with software other than HTTP “origin servers”. If you are + writing a gateway protocol implementation (such as CGI, FastCGI, + SCGI, etc.) that uses a ‘Status:’ header to send an HTTP status, + you probably want to subclass this instead of *note SimpleHandler: + 2916. + + -- Class: wsgiref.handlers.SimpleHandler (stdin, stdout, stderr, + environ, multithread=True, multiprocess=False) + + Similar to *note BaseCGIHandler: 2914, but designed for use with + HTTP origin servers. If you are writing an HTTP server + implementation, you will probably want to subclass this instead of + *note BaseCGIHandler: 2914. + + This class is a subclass of *note BaseHandler: 2917. It overrides + the *note __init__(): d5e, ‘get_stdin()’, ‘get_stderr()’, + ‘add_cgi_vars()’, ‘_write()’, and ‘_flush()’ methods to support + explicitly setting the environment and streams via the constructor. + The supplied environment and streams are stored in the ‘stdin’, + ‘stdout’, ‘stderr’, and ‘environ’ attributes. + + The *note write(): 589. method of `stdout' should write each chunk + in full, like *note io.BufferedIOBase: 588. + + -- Class: wsgiref.handlers.BaseHandler + + This is an abstract base class for running WSGI applications. Each + instance will handle a single HTTP request, although in principle + you could create a subclass that was reusable for multiple + requests. + + *note BaseHandler: 2917. instances have only one method intended + for external use: + + -- Method: run (app) + + Run the specified WSGI application, `app'. + + All of the other *note BaseHandler: 2917. methods are invoked by + this method in the process of running the application, and thus + exist primarily to allow customizing the process. + + The following methods MUST be overridden in a subclass: + + -- Method: _write (data) + + Buffer the bytes `data' for transmission to the client. It’s + okay if this method actually transmits the data; *note + BaseHandler: 2917. just separates write and flush operations + for greater efficiency when the underlying system actually has + such a distinction. + + -- Method: _flush () + + Force buffered data to be transmitted to the client. It’s + okay if this method is a no-op (i.e., if *note _write(): 2919. + actually sends the data). + + -- Method: get_stdin () + + Return an input stream object suitable for use as the + ‘wsgi.input’ of the request currently being processed. + + -- Method: get_stderr () + + Return an output stream object suitable for use as the + ‘wsgi.errors’ of the request currently being processed. + + -- Method: add_cgi_vars () + + Insert CGI variables for the current request into the + ‘environ’ attribute. + + Here are some other methods and attributes you may wish to + override. This list is only a summary, however, and does not + include every method that can be overridden. You should consult + the docstrings and source code for additional information before + attempting to create a customized *note BaseHandler: 2917. + subclass. + + Attributes and methods for customizing the WSGI environment: + + -- Attribute: wsgi_multithread + + The value to be used for the ‘wsgi.multithread’ environment + variable. It defaults to true in *note BaseHandler: 2917, but + may have a different default (or be set by the constructor) in + the other subclasses. + + -- Attribute: wsgi_multiprocess + + The value to be used for the ‘wsgi.multiprocess’ environment + variable. It defaults to true in *note BaseHandler: 2917, but + may have a different default (or be set by the constructor) in + the other subclasses. + + -- Attribute: wsgi_run_once + + The value to be used for the ‘wsgi.run_once’ environment + variable. It defaults to false in *note BaseHandler: 2917, + but *note CGIHandler: 2913. sets it to true by default. + + -- Attribute: os_environ + + The default environment variables to be included in every + request’s WSGI environment. By default, this is a copy of + ‘os.environ’ at the time that *note wsgiref.handlers: 12d. was + imported, but subclasses can either create their own at the + class or instance level. Note that the dictionary should be + considered read-only, since the default value is shared + between multiple classes and instances. + + -- Attribute: server_software + + If the *note origin_server: 2923. attribute is set, this + attribute’s value is used to set the default ‘SERVER_SOFTWARE’ + WSGI environment variable, and also to set a default ‘Server:’ + header in HTTP responses. It is ignored for handlers (such as + *note BaseCGIHandler: 2914. and *note CGIHandler: 2913.) that + are not HTTP origin servers. + + Changed in version 3.3: The term “Python” is replaced with + implementation specific term like “CPython”, “Jython” etc. + + -- Method: get_scheme () + + Return the URL scheme being used for the current request. The + default implementation uses the ‘guess_scheme()’ function from + *note wsgiref.util: 130. to guess whether the scheme should be + “http” or “https”, based on the current request’s ‘environ’ + variables. + + -- Method: setup_environ () + + Set the ‘environ’ attribute to a fully-populated WSGI + environment. The default implementation uses all of the above + methods and attributes, plus the *note get_stdin(): 291b, + *note get_stderr(): 291c, and *note add_cgi_vars(): 291d. + methods and the *note wsgi_file_wrapper: 2926. attribute. It + also inserts a ‘SERVER_SOFTWARE’ key if not present, as long + as the *note origin_server: 2923. attribute is a true value + and the *note server_software: 2922. attribute is set. + + Methods and attributes for customizing exception handling: + + -- Method: log_exception (exc_info) + + Log the `exc_info' tuple in the server log. `exc_info' is a + ‘(type, value, traceback)’ tuple. The default implementation + simply writes the traceback to the request’s ‘wsgi.errors’ + stream and flushes it. Subclasses can override this method to + change the format or retarget the output, mail the traceback + to an administrator, or whatever other action may be deemed + suitable. + + -- Attribute: traceback_limit + + The maximum number of frames to include in tracebacks output + by the default *note log_exception(): 2927. method. If + ‘None’, all frames are included. + + -- Method: error_output (environ, start_response) + + This method is a WSGI application to generate an error page + for the user. It is only invoked if an error occurs before + headers are sent to the client. + + This method can access the current error information using + ‘sys.exc_info()’, and should pass that information to + `start_response' when calling it (as described in the “Error + Handling” section of PEP 3333(1)). + + The default implementation just uses the *note error_status: + 292a, *note error_headers: 292b, and *note error_body: 292c. + attributes to generate an output page. Subclasses can + override this to produce more dynamic error output. + + Note, however, that it’s not recommended from a security + perspective to spit out diagnostics to any old user; ideally, + you should have to do something special to enable diagnostic + output, which is why the default implementation doesn’t + include any. + + -- Attribute: error_status + + The HTTP status used for error responses. This should be a + status string as defined in PEP 3333(2); it defaults to a 500 + code and message. + + -- Attribute: error_headers + + The HTTP headers used for error responses. This should be a + list of WSGI response headers (‘(name, value)’ tuples), as + described in PEP 3333(3). The default list just sets the + content type to ‘text/plain’. + + -- Attribute: error_body + + The error response body. This should be an HTTP response body + bytestring. It defaults to the plain text, “A server error + occurred. Please contact the administrator.” + + Methods and attributes for PEP 3333(4)’s “Optional + Platform-Specific File Handling” feature: + + -- Attribute: wsgi_file_wrapper + + A ‘wsgi.file_wrapper’ factory, or ‘None’. The default value + of this attribute is the *note wsgiref.util.FileWrapper: 27e. + class. + + -- Method: sendfile () + + Override to implement platform-specific file transmission. + This method is called only if the application’s return value + is an instance of the class specified by the *note + wsgi_file_wrapper: 2926. attribute. It should return a true + value if it was able to successfully transmit the file, so + that the default transmission code will not be executed. The + default implementation of this method just returns a false + value. + + Miscellaneous methods and attributes: + + -- Attribute: origin_server + + This attribute should be set to a true value if the handler’s + *note _write(): 2919. and *note _flush(): 291a. are being used + to communicate directly to the client, rather than via a + CGI-like gateway protocol that wants the HTTP status in a + special ‘Status:’ header. + + This attribute’s default value is true in *note BaseHandler: + 2917, but false in *note BaseCGIHandler: 2914. and *note + CGIHandler: 2913. + + -- Attribute: http_version + + If *note origin_server: 2923. is true, this string attribute + is used to set the HTTP version of the response set to the + client. It defaults to ‘"1.0"’. + + -- Function: wsgiref.handlers.read_environ () + + Transcode CGI variables from ‘os.environ’ to PEP 3333(5) “bytes in + unicode” strings, returning a new dictionary. This function is + used by *note CGIHandler: 2913. and *note IISCGIHandler: 2915. in + place of directly using ‘os.environ’, which is not necessarily + WSGI-compliant on all platforms and web servers using Python 3 – + specifically, ones where the OS’s actual environment is Unicode + (i.e. Windows), or ones where the environment is bytes, but the + system encoding used by Python to decode it is anything other than + ISO-8859-1 (e.g. Unix systems using UTF-8). + + If you are implementing a CGI-based handler of your own, you + probably want to use this routine instead of just copying values + out of ‘os.environ’ directly. + + New in version 3.2. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3333 + + (2) https://www.python.org/dev/peps/pep-3333 + + (3) https://www.python.org/dev/peps/pep-3333 + + (4) https://www.python.org/dev/peps/pep-3333 + + (5) https://www.python.org/dev/peps/pep-3333 + + +File: python.info, Node: Examples<20>, Prev: wsgiref handlers – server/gateway base classes, Up: wsgiref — WSGI Utilities and Reference Implementation + +5.21.4.6 Examples +................. + +This is a working “Hello World” WSGI application: + + from wsgiref.simple_server import make_server + + # Every WSGI application must have an application object - a callable + # object that accepts two arguments. For that purpose, we're going to + # use a function (note that you're not limited to a function, you can + # use a class for example). The first argument passed to the function + # is a dictionary containing CGI-style environment variables and the + # second variable is the callable object. + def hello_world_app(environ, start_response): + status = '200 OK' # HTTP Status + headers = [('Content-type', 'text/plain; charset=utf-8')] # HTTP Headers + start_response(status, headers) + + # The returned object is going to be printed + return [b"Hello World"] + + with make_server('', 8000, hello_world_app) as httpd: + print("Serving on port 8000...") + + # Serve until process is killed + httpd.serve_forever() + +Example of a WSGI application serving the current directory, accept +optional directory and port number (default: 8000) on the command line: + + #!/usr/bin/env python3 + ''' + Small wsgiref based web server. Takes a path to serve from and an + optional port number (defaults to 8000), then tries to serve files. + Mime types are guessed from the file names, 404 errors are raised + if the file is not found. Used for the make serve target in Doc. + ''' + import sys + import os + import mimetypes + from wsgiref import simple_server, util + + def app(environ, respond): + + fn = os.path.join(path, environ['PATH_INFO'][1:]) + if '.' not in fn.split(os.path.sep)[-1]: + fn = os.path.join(fn, 'index.html') + type = mimetypes.guess_type(fn)[0] + + if os.path.exists(fn): + respond('200 OK', [('Content-Type', type)]) + return util.FileWrapper(open(fn, "rb")) + else: + respond('404 Not Found', [('Content-Type', 'text/plain')]) + return [b'not found'] + + if __name__ == '__main__': + path = sys.argv[1] if len(sys.argv) > 1 else os.getcwd() + port = int(sys.argv[2]) if len(sys.argv) > 2 else 8000 + httpd = simple_server.make_server('', port, app) + print("Serving {} on port {}, control-C to stop".format(path, port)) + try: + httpd.serve_forever() + except KeyboardInterrupt: + print("Shutting down.") + httpd.server_close() + + +File: python.info, Node: urllib — URL handling modules, Next: urllib request — Extensible library for opening URLs, Prev: wsgiref — WSGI Utilities and Reference Implementation, Up: Internet Protocols and Support + +5.21.5 ‘urllib’ — URL handling modules +-------------------------------------- + +`Source code:' Lib/urllib/(1) + +__________________________________________________________________ + +‘urllib’ is a package that collects several modules for working with +URLs: + + * *note urllib.request: 120. for opening and reading URLs + + * *note urllib.error: 11e. containing the exceptions raised by *note + urllib.request: 120. + + * *note urllib.parse: 11f. for parsing URLs + + * *note urllib.robotparser: 122. for parsing ‘robots.txt’ files + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/urllib/ + + +File: python.info, Node: urllib request — Extensible library for opening URLs, Next: urllib response — Response classes used by urllib, Prev: urllib — URL handling modules, Up: Internet Protocols and Support + +5.21.6 ‘urllib.request’ — Extensible library for opening URLs +------------------------------------------------------------- + +`Source code:' Lib/urllib/request.py(1) + +__________________________________________________________________ + +The *note urllib.request: 120. module defines functions and classes +which help in opening URLs (mostly HTTP) in a complex world — basic and +digest authentication, redirections, cookies and more. + +See also +........ + +The Requests package(2) is recommended for a higher-level HTTP client +interface. + +The *note urllib.request: 120. module defines the following functions: + + -- Function: urllib.request.urlopen (url, data=None[, timeout], *, + cafile=None, capath=None, cadefault=False, context=None) + + Open the URL `url', which can be either a string or a *note + Request: 8f6. object. + + `data' must be an object specifying additional data to be sent to + the server, or ‘None’ if no such data is needed. See *note + Request: 8f6. for details. + + urllib.request module uses HTTP/1.1 and includes ‘Connection:close’ + header in its HTTP requests. + + The optional `timeout' parameter specifies a timeout in seconds for + blocking operations like the connection attempt (if not specified, + the global default timeout setting will be used). This actually + only works for HTTP, HTTPS and FTP connections. + + If `context' is specified, it must be a *note ssl.SSLContext: 3e4. + instance describing the various SSL options. See *note + HTTPSConnection: 393. for more details. + + The optional `cafile' and `capath' parameters specify a set of + trusted CA certificates for HTTPS requests. `cafile' should point + to a single file containing a bundle of CA certificates, whereas + `capath' should point to a directory of hashed certificate files. + More information can be found in *note + ssl.SSLContext.load_verify_locations(): 7eb. + + The `cadefault' parameter is ignored. + + This function always returns an object which can work as a *note + context manager: 568. and has methods such as + + * ‘geturl()’ — return the URL of the resource retrieved, + commonly used to determine if a redirect was followed + + * ‘info()’ — return the meta-information of the page, such as + headers, in the form of an *note email.message_from_string(): + 2516. instance (see Quick Reference to HTTP Headers(3)) + + * ‘getcode()’ – return the HTTP status code of the response. + + For HTTP and HTTPS URLs, this function returns a *note + http.client.HTTPResponse: a11. object slightly modified. In + addition to the three new methods above, the msg attribute contains + the same information as the *note reason: 2934. attribute — the + reason phrase returned by server — instead of the response headers + as it is specified in the documentation for *note HTTPResponse: + a11. + + For FTP, file, and data URLs and requests explicitly handled by + legacy *note URLopener: 2935. and *note FancyURLopener: 2936. + classes, this function returns a ‘urllib.response.addinfourl’ + object. + + Raises *note URLError: 2937. on protocol errors. + + Note that ‘None’ may be returned if no handler handles the request + (though the default installed global *note OpenerDirector: 2938. + uses *note UnknownHandler: 2939. to ensure this never happens). + + In addition, if proxy settings are detected (for example, when a + ‘*_proxy’ environment variable like ‘http_proxy’ is set), *note + ProxyHandler: 293a. is default installed and makes sure the + requests are handled through the proxy. + + The legacy ‘urllib.urlopen’ function from Python 2.6 and earlier + has been discontinued; *note urllib.request.urlopen(): 78c. + corresponds to the old ‘urllib2.urlopen’. Proxy handling, which + was done by passing a dictionary parameter to ‘urllib.urlopen’, can + be obtained by using *note ProxyHandler: 293a. objects. + + The default opener raises an *note auditing event: fd1. + ‘urllib.Request’ with arguments ‘fullurl’, ‘data’, ‘headers’, + ‘method’ taken from the request object. + + Changed in version 3.2: `cafile' and `capath' were added. + + Changed in version 3.2: HTTPS virtual hosts are now supported if + possible (that is, if *note ssl.HAS_SNI: 23d2. is true). + + New in version 3.2: `data' can be an iterable object. + + Changed in version 3.3: `cadefault' was added. + + Changed in version 3.4.3: `context' was added. + + Deprecated since version 3.6: `cafile', `capath' and `cadefault' + are deprecated in favor of `context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + + -- Function: urllib.request.install_opener (opener) + + Install an *note OpenerDirector: 2938. instance as the default + global opener. Installing an opener is only necessary if you want + urlopen to use that opener; otherwise, simply call *note + OpenerDirector.open(): 8fa. instead of *note urlopen(): 78c. The + code does not check for a real *note OpenerDirector: 2938, and any + class with the appropriate interface will work. + + -- Function: urllib.request.build_opener ([handler, ...]) + + Return an *note OpenerDirector: 2938. instance, which chains the + handlers in the order given. `handler's can be either instances of + *note BaseHandler: 293d, or subclasses of *note BaseHandler: 293d. + (in which case it must be possible to call the constructor without + any parameters). Instances of the following classes will be in + front of the `handler's, unless the `handler's contain them, + instances of them or subclasses of them: *note ProxyHandler: 293a. + (if proxy settings are detected), *note UnknownHandler: 2939, *note + HTTPHandler: 293e, *note HTTPDefaultErrorHandler: 293f, *note + HTTPRedirectHandler: 2940, *note FTPHandler: 2941, *note + FileHandler: 2942, *note HTTPErrorProcessor: 2943. + + If the Python installation has SSL support (i.e., if the *note ssl: + f3. module can be imported), *note HTTPSHandler: ba8. will also be + added. + + A *note BaseHandler: 293d. subclass may also change its + ‘handler_order’ attribute to modify its position in the handlers + list. + + -- Function: urllib.request.pathname2url (path) + + Convert the pathname `path' from the local syntax for a path to the + form used in the path component of a URL. This does not produce a + complete URL. The return value will already be quoted using the + *note quote(): 40d. function. + + -- Function: urllib.request.url2pathname (path) + + Convert the path component `path' from a percent-encoded URL to the + local syntax for a path. This does not accept a complete URL. This + function uses *note unquote(): 2946. to decode `path'. + + -- Function: urllib.request.getproxies () + + This helper function returns a dictionary of scheme to proxy server + URL mappings. It scans the environment for variables named + ‘_proxy’, in a case insensitive approach, for all operating + systems first, and when it cannot find it, looks for proxy + information from Mac OSX System Configuration for Mac OS X and + Windows Systems Registry for Windows. If both lowercase and + uppercase environment variables exist (and disagree), lowercase is + preferred. + + Note: If the environment variable ‘REQUEST_METHOD’ is set, + which usually indicates your script is running in a CGI + environment, the environment variable ‘HTTP_PROXY’ (uppercase + ‘_PROXY’) will be ignored. This is because that variable can + be injected by a client using the “Proxy:” HTTP header. If + you need to use an HTTP proxy in a CGI environment, either use + ‘ProxyHandler’ explicitly, or make sure the variable name is + in lowercase (or at least the ‘_proxy’ suffix). + +The following classes are provided: + + -- Class: urllib.request.Request (url, data=None, headers={}, + origin_req_host=None, unverifiable=False, method=None) + + This class is an abstraction of a URL request. + + `url' should be a string containing a valid URL. + + `data' must be an object specifying additional data to send to the + server, or ‘None’ if no such data is needed. Currently HTTP + requests are the only ones that use `data'. The supported object + types include bytes, file-like objects, and iterables of bytes-like + objects. If no ‘Content-Length’ nor ‘Transfer-Encoding’ header + field has been provided, *note HTTPHandler: 293e. will set these + headers according to the type of `data'. ‘Content-Length’ will be + used to send bytes objects, while ‘Transfer-Encoding: chunked’ as + specified in RFC 7230(4), Section 3.3.1 will be used to send files + and other iterables. + + For an HTTP POST request method, `data' should be a buffer in the + standard ‘application/x-www-form-urlencoded’ format. The *note + urllib.parse.urlencode(): 78b. function takes a mapping or sequence + of 2-tuples and returns an ASCII string in this format. It should + be encoded to bytes before being used as the `data' parameter. + + `headers' should be a dictionary, and will be treated as if *note + add_header(): 2948. was called with each key and value as + arguments. This is often used to “spoof” the ‘User-Agent’ header + value, which is used by a browser to identify itself – some HTTP + servers only allow requests coming from common browsers as opposed + to scripts. For example, Mozilla Firefox may identify itself as + ‘"Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 + Firefox/2.0.0.11"’, while *note urllib: 11d.’s default user agent + string is ‘"Python-urllib/2.6"’ (on Python 2.6). + + An appropriate ‘Content-Type’ header should be included if the + `data' argument is present. If this header has not been provided + and `data' is not None, ‘Content-Type: + application/x-www-form-urlencoded’ will be added as a default. + + The next two arguments are only of interest for correct handling of + third-party HTTP cookies: + + `origin_req_host' should be the request-host of the origin + transaction, as defined by RFC 2965(5). It defaults to + ‘http.cookiejar.request_host(self)’. This is the host name or IP + address of the original request that was initiated by the user. + For example, if the request is for an image in an HTML document, + this should be the request-host of the request for the page + containing the image. + + `unverifiable' should indicate whether the request is unverifiable, + as defined by RFC 2965(6). It defaults to ‘False’. An + unverifiable request is one whose URL the user did not have the + option to approve. For example, if the request is for an image in + an HTML document, and the user had no option to approve the + automatic fetching of the image, this should be true. + + `method' should be a string that indicates the HTTP request method + that will be used (e.g. ‘'HEAD'’). If provided, its value is + stored in the *note method: 8f7. attribute and is used by *note + get_method(): ac1. The default is ‘'GET'’ if `data' is ‘None’ or + ‘'POST'’ otherwise. Subclasses may indicate a different default + method by setting the *note method: 8f7. attribute in the class + itself. + + Note: The request will not work as expected if the data object + is unable to deliver its content more than once (e.g. a file + or an iterable that can produce the content only once) and the + request is retried for HTTP redirects or authentication. The + `data' is sent to the HTTP server right away after the + headers. There is no support for a 100-continue expectation + in the library. + + Changed in version 3.3: *note Request.method: 8f7. argument is + added to the Request class. + + Changed in version 3.4: Default *note Request.method: 8f7. may be + indicated at the class level. + + Changed in version 3.6: Do not raise an error if the + ‘Content-Length’ has not been provided and `data' is neither ‘None’ + nor a bytes object. Fall back to use chunked transfer encoding + instead. + + -- Class: urllib.request.OpenerDirector + + The *note OpenerDirector: 2938. class opens URLs via *note + BaseHandler: 293d.s chained together. It manages the chaining of + handlers, and recovery from errors. + + -- Class: urllib.request.BaseHandler + + This is the base class for all registered handlers — and handles + only the simple mechanics of registration. + + -- Class: urllib.request.HTTPDefaultErrorHandler + + A class which defines a default handler for HTTP error responses; + all responses are turned into *note HTTPError: 8fc. exceptions. + + -- Class: urllib.request.HTTPRedirectHandler + + A class to handle redirections. + + -- Class: urllib.request.HTTPCookieProcessor (cookiejar=None) + + A class to handle HTTP Cookies. + + -- Class: urllib.request.ProxyHandler (proxies=None) + + Cause requests to go through a proxy. If `proxies' is given, it + must be a dictionary mapping protocol names to URLs of proxies. + The default is to read the list of proxies from the environment + variables ‘_proxy’. If no proxy environment variables + are set, then in a Windows environment proxy settings are obtained + from the registry’s Internet Settings section, and in a Mac OS X + environment proxy information is retrieved from the OS X System + Configuration Framework. + + To disable autodetected proxy pass an empty dictionary. + + The ‘no_proxy’ environment variable can be used to specify hosts + which shouldn’t be reached via proxy; if set, it should be a + comma-separated list of hostname suffixes, optionally with ‘:port’ + appended, for example ‘cern.ch,ncsa.uiuc.edu,some.host:8080’. + + Note: ‘HTTP_PROXY’ will be ignored if a variable + ‘REQUEST_METHOD’ is set; see the documentation on *note + getproxies(): 2947. + + -- Class: urllib.request.HTTPPasswordMgr + + Keep a database of ‘(realm, uri) -> (user, password)’ mappings. + + -- Class: urllib.request.HTTPPasswordMgrWithDefaultRealm + + Keep a database of ‘(realm, uri) -> (user, password)’ mappings. A + realm of ‘None’ is considered a catch-all realm, which is searched + if no other realm fits. + + -- Class: urllib.request.HTTPPasswordMgrWithPriorAuth + + A variant of *note HTTPPasswordMgrWithDefaultRealm: 294b. that also + has a database of ‘uri -> is_authenticated’ mappings. Can be used + by a BasicAuth handler to determine when to send authentication + credentials immediately instead of waiting for a ‘401’ response + first. + + New in version 3.5. + + -- Class: urllib.request.AbstractBasicAuthHandler (password_mgr=None) + + This is a mixin class that helps with HTTP authentication, both to + the remote host and to a proxy. `password_mgr', if given, should + be something that is compatible with *note HTTPPasswordMgr: 294a.; + refer to section *note HTTPPasswordMgr Objects: 294d. for + information on the interface that must be supported. If + `passwd_mgr' also provides ‘is_authenticated’ and + ‘update_authenticated’ methods (see *note + HTTPPasswordMgrWithPriorAuth Objects: 294e.), then the handler will + use the ‘is_authenticated’ result for a given URI to determine + whether or not to send authentication credentials with the request. + If ‘is_authenticated’ returns ‘True’ for the URI, credentials are + sent. If ‘is_authenticated’ is ‘False’, credentials are not sent, + and then if a ‘401’ response is received the request is re-sent + with the authentication credentials. If authentication succeeds, + ‘update_authenticated’ is called to set ‘is_authenticated’ ‘True’ + for the URI, so that subsequent requests to the URI or any of its + super-URIs will automatically include the authentication + credentials. + + New in version 3.5: Added ‘is_authenticated’ support. + + -- Class: urllib.request.HTTPBasicAuthHandler (password_mgr=None) + + Handle authentication with the remote host. `password_mgr', if + given, should be something that is compatible with *note + HTTPPasswordMgr: 294a.; refer to section *note HTTPPasswordMgr + Objects: 294d. for information on the interface that must be + supported. HTTPBasicAuthHandler will raise a *note ValueError: + 1fb. when presented with a wrong Authentication scheme. + + -- Class: urllib.request.ProxyBasicAuthHandler (password_mgr=None) + + Handle authentication with the proxy. `password_mgr', if given, + should be something that is compatible with *note HTTPPasswordMgr: + 294a.; refer to section *note HTTPPasswordMgr Objects: 294d. for + information on the interface that must be supported. + + -- Class: urllib.request.AbstractDigestAuthHandler (password_mgr=None) + + This is a mixin class that helps with HTTP authentication, both to + the remote host and to a proxy. `password_mgr', if given, should + be something that is compatible with *note HTTPPasswordMgr: 294a.; + refer to section *note HTTPPasswordMgr Objects: 294d. for + information on the interface that must be supported. + + -- Class: urllib.request.HTTPDigestAuthHandler (password_mgr=None) + + Handle authentication with the remote host. `password_mgr', if + given, should be something that is compatible with *note + HTTPPasswordMgr: 294a.; refer to section *note HTTPPasswordMgr + Objects: 294d. for information on the interface that must be + supported. When both Digest Authentication Handler and Basic + Authentication Handler are both added, Digest Authentication is + always tried first. If the Digest Authentication returns a 40x + response again, it is sent to Basic Authentication handler to + Handle. This Handler method will raise a *note ValueError: 1fb. + when presented with an authentication scheme other than Digest or + Basic. + + Changed in version 3.3: Raise *note ValueError: 1fb. on unsupported + Authentication Scheme. + + -- Class: urllib.request.ProxyDigestAuthHandler (password_mgr=None) + + Handle authentication with the proxy. `password_mgr', if given, + should be something that is compatible with *note HTTPPasswordMgr: + 294a.; refer to section *note HTTPPasswordMgr Objects: 294d. for + information on the interface that must be supported. + + -- Class: urllib.request.HTTPHandler + + A class to handle opening of HTTP URLs. + + -- Class: urllib.request.HTTPSHandler (debuglevel=0, context=None, + check_hostname=None) + + A class to handle opening of HTTPS URLs. `context' and + `check_hostname' have the same meaning as in *note + http.client.HTTPSConnection: 393. + + Changed in version 3.2: `context' and `check_hostname' were added. + + -- Class: urllib.request.FileHandler + + Open local files. + + -- Class: urllib.request.DataHandler + + Open data URLs. + + New in version 3.4. + + -- Class: urllib.request.FTPHandler + + Open FTP URLs. + + -- Class: urllib.request.CacheFTPHandler + + Open FTP URLs, keeping a cache of open FTP connections to minimize + delays. + + -- Class: urllib.request.UnknownHandler + + A catch-all class to handle unknown URLs. + + -- Class: urllib.request.HTTPErrorProcessor + + Process HTTP error responses. + +* Menu: + +* Request Objects:: +* OpenerDirector Objects:: +* BaseHandler Objects:: +* HTTPRedirectHandler Objects:: +* HTTPCookieProcessor Objects:: +* ProxyHandler Objects:: +* HTTPPasswordMgr Objects:: +* HTTPPasswordMgrWithPriorAuth Objects:: +* AbstractBasicAuthHandler Objects:: +* HTTPBasicAuthHandler Objects:: +* ProxyBasicAuthHandler Objects:: +* AbstractDigestAuthHandler Objects:: +* HTTPDigestAuthHandler Objects:: +* ProxyDigestAuthHandler Objects:: +* HTTPHandler Objects:: +* HTTPSHandler Objects:: +* FileHandler Objects:: +* DataHandler Objects:: +* FTPHandler Objects:: +* CacheFTPHandler Objects:: +* UnknownHandler Objects:: +* HTTPErrorProcessor Objects:: +* Examples: Examples<21>. +* Legacy interface:: +* urllib.request Restrictions: urllib request Restrictions. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/urllib/request.py + + (2) https://requests.readthedocs.io/en/master/ + + (3) http://jkorpela.fi/http.html + + (4) https://tools.ietf.org/html/rfc7230.html + + (5) https://tools.ietf.org/html/rfc2965.html + + (6) https://tools.ietf.org/html/rfc2965.html + + +File: python.info, Node: Request Objects, Next: OpenerDirector Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.1 Request Objects +........................ + +The following methods describe *note Request: 8f6.’s public interface, +and so all may be overridden in subclasses. It also defines several +public attributes that can be used by clients to inspect the parsed +request. + + -- Attribute: Request.full_url + + The original URL passed to the constructor. + + Changed in version 3.4. + + Request.full_url is a property with setter, getter and a deleter. + Getting *note full_url: 8f8. returns the original request URL with + the fragment, if it was present. + + -- Attribute: Request.type + + The URI scheme. + + -- Attribute: Request.host + + The URI authority, typically a host, but may also contain a port + separated by a colon. + + -- Attribute: Request.origin_req_host + + The original host for the request, without port. + + -- Attribute: Request.selector + + The URI path. If the *note Request: 8f6. uses a proxy, then + selector will be the full URL that is passed to the proxy. + + -- Attribute: Request.data + + The entity body for the request, or ‘None’ if not specified. + + Changed in version 3.4: Changing value of *note Request.data: 8f9. + now deletes “Content-Length” header if it was previously set or + calculated. + + -- Attribute: Request.unverifiable + + boolean, indicates whether the request is unverifiable as defined + by RFC 2965(1). + + -- Attribute: Request.method + + The HTTP request method to use. By default its value is *note + None: 157, which means that *note get_method(): ac1. will do its + normal computation of the method to be used. Its value can be set + (thus overriding the default computation in *note get_method(): + ac1.) either by providing a default value by setting it at the + class level in a *note Request: 8f6. subclass, or by passing a + value in to the *note Request: 8f6. constructor via the `method' + argument. + + New in version 3.3. + + Changed in version 3.4: A default value can now be set in + subclasses; previously it could only be set via the constructor + argument. + + -- Method: Request.get_method () + + Return a string indicating the HTTP request method. If *note + Request.method: 8f7. is not ‘None’, return its value, otherwise + return ‘'GET'’ if *note Request.data: 8f9. is ‘None’, or ‘'POST'’ + if it’s not. This is only meaningful for HTTP requests. + + Changed in version 3.3: get_method now looks at the value of *note + Request.method: 8f7. + + -- Method: Request.add_header (key, val) + + Add another header to the request. Headers are currently ignored + by all handlers except HTTP handlers, where they are added to the + list of headers sent to the server. Note that there cannot be more + than one header with the same name, and later calls will overwrite + previous calls in case the `key' collides. Currently, this is no + loss of HTTP functionality, since all headers which have meaning + when used more than once have a (header-specific) way of gaining + the same functionality using only one header. + + -- Method: Request.add_unredirected_header (key, header) + + Add a header that will not be added to a redirected request. + + -- Method: Request.has_header (header) + + Return whether the instance has the named header (checks both + regular and unredirected). + + -- Method: Request.remove_header (header) + + Remove named header from the request instance (both from regular + and unredirected headers). + + New in version 3.4. + + -- Method: Request.get_full_url () + + Return the URL given in the constructor. + + Changed in version 3.4. + + Returns *note Request.full_url: 8f8. + + -- Method: Request.set_proxy (host, type) + + Prepare the request by connecting to a proxy server. The `host' + and `type' will replace those of the instance, and the instance’s + selector will be the original URL given in the constructor. + + -- Method: Request.get_header (header_name, default=None) + + Return the value of the given header. If the header is not + present, return the default value. + + -- Method: Request.header_items () + + Return a list of tuples (header_name, header_value) of the Request + headers. + +Changed in version 3.4: The request methods add_data, has_data, +get_data, get_type, get_host, get_selector, get_origin_req_host and +is_unverifiable that were deprecated since 3.3 have been removed. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2965.html + + +File: python.info, Node: OpenerDirector Objects, Next: BaseHandler Objects, Prev: Request Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.2 OpenerDirector Objects +............................... + +*note OpenerDirector: 2938. instances have the following methods: + + -- Method: OpenerDirector.add_handler (handler) + + `handler' should be an instance of *note BaseHandler: 293d. The + following methods are searched, and added to the possible chains + (note that HTTP errors are a special case). Note that, in the + following, `protocol' should be replaced with the actual protocol + to handle, for example ‘http_response()’ would be the HTTP protocol + response handler. Also `type' should be replaced with the actual + HTTP code, for example ‘http_error_404()’ would handle HTTP 404 + errors. + + * ‘_open()’ — signal that the handler knows how to + open `protocol' URLs. + + See *note BaseHandler._open(): 2965. for more + information. + + * ‘http_error_()’ — signal that the handler knows how to + handle HTTP errors with HTTP error code `type'. + + See *note BaseHandler.http_error_(): 2966. for more + information. + + * ‘_error()’ — signal that the handler knows how to + handle errors from (non-‘http’) `protocol'. + + * ‘_request()’ — signal that the handler knows how to + pre-process `protocol' requests. + + See *note BaseHandler._request(): 2967. for more + information. + + * ‘_response()’ — signal that the handler knows how to + post-process `protocol' responses. + + See *note BaseHandler._response(): 2968. for more + information. + + -- Method: OpenerDirector.open (url, data=None[, timeout]) + + Open the given `url' (which can be a request object or a string), + optionally passing the given `data'. Arguments, return values and + exceptions raised are the same as those of *note urlopen(): 78c. + (which simply calls the *note open(): 4f0. method on the currently + installed global *note OpenerDirector: 2938.). The optional + `timeout' parameter specifies a timeout in seconds for blocking + operations like the connection attempt (if not specified, the + global default timeout setting will be used). The timeout feature + actually works only for HTTP, HTTPS and FTP connections). + + -- Method: OpenerDirector.error (proto, *args) + + Handle an error of the given protocol. This will call the + registered error handlers for the given protocol with the given + arguments (which are protocol specific). The HTTP protocol is a + special case which uses the HTTP response code to determine the + specific error handler; refer to the ‘http_error_()’ methods + of the handler classes. + + Return values and exceptions raised are the same as those of *note + urlopen(): 78c. + +OpenerDirector objects open URLs in three stages: + +The order in which these methods are called within each stage is +determined by sorting the handler instances. + + 1. Every handler with a method named like ‘_request()’ has + that method called to pre-process the request. + + 2. Handlers with a method named like ‘_open()’ are called to + handle the request. This stage ends when a handler either returns + a non-*note None: 157. value (ie. a response), or raises an + exception (usually *note URLError: 2937.). Exceptions are allowed + to propagate. + + In fact, the above algorithm is first tried for methods named + ‘default_open()’. If all such methods return *note None: 157, the + algorithm is repeated for methods named like ‘_open()’. + If all such methods return *note None: 157, the algorithm is + repeated for methods named ‘unknown_open()’. + + Note that the implementation of these methods may involve calls of + the parent *note OpenerDirector: 2938. instance’s *note open(): + 8fa. and *note error(): 2969. methods. + + 3. Every handler with a method named like ‘_response()’ has + that method called to post-process the response. + + +File: python.info, Node: BaseHandler Objects, Next: HTTPRedirectHandler Objects, Prev: OpenerDirector Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.3 BaseHandler Objects +............................ + +*note BaseHandler: 293d. objects provide a couple of methods that are +directly useful, and others that are meant to be used by derived +classes. These are intended for direct use: + + -- Method: BaseHandler.add_parent (director) + + Add a director as parent. + + -- Method: BaseHandler.close () + + Remove any parents. + +The following attribute and methods should only be used by classes +derived from *note BaseHandler: 293d. + + Note: The convention has been adopted that subclasses defining + ‘_request()’ or ‘_response()’ methods are named + ‘*Processor’; all others are named ‘*Handler’. + + -- Attribute: BaseHandler.parent + + A valid *note OpenerDirector: 2938, which can be used to open using + a different protocol, or handle errors. + + -- Method: BaseHandler.default_open (req) + + This method is `not' defined in *note BaseHandler: 293d, but + subclasses should define it if they want to catch all URLs. + + This method, if implemented, will be called by the parent *note + OpenerDirector: 2938. It should return a file-like object as + described in the return value of the *note open(): 4f0. of *note + OpenerDirector: 2938, or ‘None’. It should raise *note URLError: + 2937, unless a truly exceptional thing happens (for example, *note + MemoryError: 13af. should not be mapped to ‘URLError’). + + This method will be called before any protocol-specific open + method. + + -- Method: BaseHandler._open(req) + + This method is `not' defined in *note BaseHandler: 293d, but + subclasses should define it if they want to handle URLs with the + given protocol. + + This method, if defined, will be called by the parent *note + OpenerDirector: 2938. Return values should be the same as for + ‘default_open()’. + + -- Method: BaseHandler.unknown_open (req) + + This method is `not' defined in *note BaseHandler: 293d, but + subclasses should define it if they want to catch all URLs with no + specific registered handler to open it. + + This method, if implemented, will be called by the *note parent: + 296e. *note OpenerDirector: 2938. Return values should be the same + as for *note default_open(): 296f. + + -- Method: BaseHandler.http_error_default (req, fp, code, msg, hdrs) + + This method is `not' defined in *note BaseHandler: 293d, but + subclasses should override it if they intend to provide a catch-all + for otherwise unhandled HTTP errors. It will be called + automatically by the *note OpenerDirector: 2938. getting the error, + and should not normally be called in other circumstances. + + `req' will be a *note Request: 8f6. object, `fp' will be a + file-like object with the HTTP error body, `code' will be the + three-digit code of the error, `msg' will be the user-visible + explanation of the code and `hdrs' will be a mapping object with + the headers of the error. + + Return values and exceptions raised should be the same as those of + *note urlopen(): 78c. + + -- Method: BaseHandler.http_error_(req, fp, code, msg, hdrs) + + `nnn' should be a three-digit HTTP error code. This method is also + not defined in *note BaseHandler: 293d, but will be called, if it + exists, on an instance of a subclass, when an HTTP error with code + `nnn' occurs. + + Subclasses should override this method to handle specific HTTP + errors. + + Arguments, return values and exceptions raised should be the same + as for ‘http_error_default()’. + + -- Method: BaseHandler._request(req) + + This method is `not' defined in *note BaseHandler: 293d, but + subclasses should define it if they want to pre-process requests of + the given protocol. + + This method, if defined, will be called by the parent *note + OpenerDirector: 2938. `req' will be a *note Request: 8f6. object. + The return value should be a *note Request: 8f6. object. + + -- Method: BaseHandler._response(req, response) + + This method is `not' defined in *note BaseHandler: 293d, but + subclasses should define it if they want to post-process responses + of the given protocol. + + This method, if defined, will be called by the parent *note + OpenerDirector: 2938. `req' will be a *note Request: 8f6. object. + `response' will be an object implementing the same interface as the + return value of *note urlopen(): 78c. The return value should + implement the same interface as the return value of *note + urlopen(): 78c. + + +File: python.info, Node: HTTPRedirectHandler Objects, Next: HTTPCookieProcessor Objects, Prev: BaseHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.4 HTTPRedirectHandler Objects +.................................... + + Note: Some HTTP redirections require action from this module’s + client code. If this is the case, *note HTTPError: 8fc. is raised. + See RFC 2616(1) for details of the precise meanings of the various + redirection codes. + + An ‘HTTPError’ exception raised as a security consideration if the + HTTPRedirectHandler is presented with a redirected URL which is not + an HTTP, HTTPS or FTP URL. + + -- Method: HTTPRedirectHandler.redirect_request (req, fp, code, msg, + hdrs, newurl) + + Return a *note Request: 8f6. or ‘None’ in response to a redirect. + This is called by the default implementations of the + ‘http_error_30*()’ methods when a redirection is received from the + server. If a redirection should take place, return a new *note + Request: 8f6. to allow ‘http_error_30*()’ to perform the redirect + to `newurl'. Otherwise, raise *note HTTPError: 8fc. if no other + handler should try to handle this URL, or return ‘None’ if you + can’t but another handler might. + + Note: The default implementation of this method does not + strictly follow RFC 2616(2), which says that 301 and 302 + responses to ‘POST’ requests must not be automatically + redirected without confirmation by the user. In reality, + browsers do allow automatic redirection of these responses, + changing the POST to a ‘GET’, and the default implementation + reproduces this behavior. + + -- Method: HTTPRedirectHandler.http_error_301 (req, fp, code, msg, + hdrs) + + Redirect to the ‘Location:’ or ‘URI:’ URL. This method is called by + the parent *note OpenerDirector: 2938. when getting an HTTP ‘moved + permanently’ response. + + -- Method: HTTPRedirectHandler.http_error_302 (req, fp, code, msg, + hdrs) + + The same as *note http_error_301(): 2975, but called for the + ‘found’ response. + + -- Method: HTTPRedirectHandler.http_error_303 (req, fp, code, msg, + hdrs) + + The same as *note http_error_301(): 2975, but called for the ‘see + other’ response. + + -- Method: HTTPRedirectHandler.http_error_307 (req, fp, code, msg, + hdrs) + + The same as *note http_error_301(): 2975, but called for the + ‘temporary redirect’ response. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2616.html + + (2) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: HTTPCookieProcessor Objects, Next: ProxyHandler Objects, Prev: HTTPRedirectHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.5 HTTPCookieProcessor Objects +.................................... + +*note HTTPCookieProcessor: 2949. instances have one attribute: + + -- Attribute: HTTPCookieProcessor.cookiejar + + The *note http.cookiejar.CookieJar: 297c. in which cookies are + stored. + + +File: python.info, Node: ProxyHandler Objects, Next: HTTPPasswordMgr Objects, Prev: HTTPCookieProcessor Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.6 ProxyHandler Objects +............................. + + -- Method: ProxyHandler._open(request) + + The *note ProxyHandler: 293a. will have a method + ‘_open()’ for every `protocol' which has a proxy in the + `proxies' dictionary given in the constructor. The method will + modify requests to go through the proxy, by calling + ‘request.set_proxy()’, and call the next handler in the chain to + actually execute the protocol. + + +File: python.info, Node: HTTPPasswordMgr Objects, Next: HTTPPasswordMgrWithPriorAuth Objects, Prev: ProxyHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.7 HTTPPasswordMgr Objects +................................ + +These methods are available on *note HTTPPasswordMgr: 294a. and *note +HTTPPasswordMgrWithDefaultRealm: 294b. objects. + + -- Method: HTTPPasswordMgr.add_password (realm, uri, user, passwd) + + `uri' can be either a single URI, or a sequence of URIs. `realm', + `user' and `passwd' must be strings. This causes ‘(user, passwd)’ + to be used as authentication tokens when authentication for `realm' + and a super-URI of any of the given URIs is given. + + -- Method: HTTPPasswordMgr.find_user_password (realm, authuri) + + Get user/password for given realm and URI, if any. This method + will return ‘(None, None)’ if there is no matching user/password. + + For *note HTTPPasswordMgrWithDefaultRealm: 294b. objects, the realm + ‘None’ will be searched if the given `realm' has no matching + user/password. + + +File: python.info, Node: HTTPPasswordMgrWithPriorAuth Objects, Next: AbstractBasicAuthHandler Objects, Prev: HTTPPasswordMgr Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.8 HTTPPasswordMgrWithPriorAuth Objects +............................................. + +This password manager extends *note HTTPPasswordMgrWithDefaultRealm: +294b. to support tracking URIs for which authentication credentials +should always be sent. + + -- Method: HTTPPasswordMgrWithPriorAuth.add_password (realm, uri, user, + passwd, is_authenticated=False) + + `realm', `uri', `user', `passwd' are as for *note + HTTPPasswordMgr.add_password(): 2980. `is_authenticated' sets the + initial value of the ‘is_authenticated’ flag for the given URI or + list of URIs. If `is_authenticated' is specified as ‘True’, + `realm' is ignored. + + -- Method: HTTPPasswordMgrWithPriorAuth.find_user_password (realm, + authuri) + + Same as for *note HTTPPasswordMgrWithDefaultRealm: 294b. objects + + -- Method: HTTPPasswordMgrWithPriorAuth.update_authenticated (self, + uri, is_authenticated=False) + + Update the ‘is_authenticated’ flag for the given `uri' or list of + URIs. + + -- Method: HTTPPasswordMgrWithPriorAuth.is_authenticated (self, + authuri) + + Returns the current state of the ‘is_authenticated’ flag for the + given URI. + + +File: python.info, Node: AbstractBasicAuthHandler Objects, Next: HTTPBasicAuthHandler Objects, Prev: HTTPPasswordMgrWithPriorAuth Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.9 AbstractBasicAuthHandler Objects +......................................... + + -- Method: AbstractBasicAuthHandler.http_error_auth_reqed (authreq, + host, req, headers) + + Handle an authentication request by getting a user/password pair, + and re-trying the request. `authreq' should be the name of the + header where the information about the realm is included in the + request, `host' specifies the URL and path to authenticate for, + `req' should be the (failed) *note Request: 8f6. object, and + `headers' should be the error headers. + + `host' is either an authority (e.g. ‘"python.org"’) or a URL + containing an authority component (e.g. ‘"http://python.org/"’). + In either case, the authority must not contain a userinfo component + (so, ‘"python.org"’ and ‘"python.org:80"’ are fine, + ‘"joe:password@python.org"’ is not). + + +File: python.info, Node: HTTPBasicAuthHandler Objects, Next: ProxyBasicAuthHandler Objects, Prev: AbstractBasicAuthHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.10 HTTPBasicAuthHandler Objects +...................................... + + -- Method: HTTPBasicAuthHandler.http_error_401 (req, fp, code, msg, + hdrs) + + Retry the request with authentication information, if available. + + +File: python.info, Node: ProxyBasicAuthHandler Objects, Next: AbstractDigestAuthHandler Objects, Prev: HTTPBasicAuthHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.11 ProxyBasicAuthHandler Objects +....................................... + + -- Method: ProxyBasicAuthHandler.http_error_407 (req, fp, code, msg, + hdrs) + + Retry the request with authentication information, if available. + + +File: python.info, Node: AbstractDigestAuthHandler Objects, Next: HTTPDigestAuthHandler Objects, Prev: ProxyBasicAuthHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.12 AbstractDigestAuthHandler Objects +........................................... + + -- Method: AbstractDigestAuthHandler.http_error_auth_reqed (authreq, + host, req, headers) + + `authreq' should be the name of the header where the information + about the realm is included in the request, `host' should be the + host to authenticate to, `req' should be the (failed) *note + Request: 8f6. object, and `headers' should be the error headers. + + +File: python.info, Node: HTTPDigestAuthHandler Objects, Next: ProxyDigestAuthHandler Objects, Prev: AbstractDigestAuthHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.13 HTTPDigestAuthHandler Objects +....................................... + + -- Method: HTTPDigestAuthHandler.http_error_401 (req, fp, code, msg, + hdrs) + + Retry the request with authentication information, if available. + + +File: python.info, Node: ProxyDigestAuthHandler Objects, Next: HTTPHandler Objects, Prev: HTTPDigestAuthHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.14 ProxyDigestAuthHandler Objects +........................................ + + -- Method: ProxyDigestAuthHandler.http_error_407 (req, fp, code, msg, + hdrs) + + Retry the request with authentication information, if available. + + +File: python.info, Node: HTTPHandler Objects, Next: HTTPSHandler Objects, Prev: ProxyDigestAuthHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.15 HTTPHandler Objects +............................. + + -- Method: HTTPHandler.http_open (req) + + Send an HTTP request, which can be either GET or POST, depending on + ‘req.has_data()’. + + +File: python.info, Node: HTTPSHandler Objects, Next: FileHandler Objects, Prev: HTTPHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.16 HTTPSHandler Objects +.............................. + + -- Method: HTTPSHandler.https_open (req) + + Send an HTTPS request, which can be either GET or POST, depending + on ‘req.has_data()’. + + +File: python.info, Node: FileHandler Objects, Next: DataHandler Objects, Prev: HTTPSHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.17 FileHandler Objects +............................. + + -- Method: FileHandler.file_open (req) + + Open the file locally, if there is no host name, or the host name + is ‘'localhost'’. + + Changed in version 3.2: This method is applicable only for local + hostnames. When a remote hostname is given, an *note URLError: + 2937. is raised. + + +File: python.info, Node: DataHandler Objects, Next: FTPHandler Objects, Prev: FileHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.18 DataHandler Objects +............................. + + -- Method: DataHandler.data_open (req) + + Read a data URL. This kind of URL contains the content encoded in + the URL itself. The data URL syntax is specified in RFC 2397(1). + This implementation ignores white spaces in base64 encoded data + URLs so the URL may be wrapped in whatever source file it comes + from. But even though some browsers don’t mind about a missing + padding at the end of a base64 encoded data URL, this + implementation will raise an *note ValueError: 1fb. in that case. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2397.html + + +File: python.info, Node: FTPHandler Objects, Next: CacheFTPHandler Objects, Prev: DataHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.19 FTPHandler Objects +............................ + + -- Method: FTPHandler.ftp_open (req) + + Open the FTP file indicated by `req'. The login is always done + with empty username and password. + + +File: python.info, Node: CacheFTPHandler Objects, Next: UnknownHandler Objects, Prev: FTPHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.20 CacheFTPHandler Objects +................................. + +*note CacheFTPHandler: 2954. objects are *note FTPHandler: 2941. objects +with the following additional methods: + + -- Method: CacheFTPHandler.setTimeout (t) + + Set timeout of connections to `t' seconds. + + -- Method: CacheFTPHandler.setMaxConns (m) + + Set maximum number of cached connections to `m'. + + +File: python.info, Node: UnknownHandler Objects, Next: HTTPErrorProcessor Objects, Prev: CacheFTPHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.21 UnknownHandler Objects +................................ + + -- Method: UnknownHandler.unknown_open () + + Raise a *note URLError: 2937. exception. + + +File: python.info, Node: HTTPErrorProcessor Objects, Next: Examples<21>, Prev: UnknownHandler Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.22 HTTPErrorProcessor Objects +.................................... + + -- Method: HTTPErrorProcessor.http_response (request, response) + + Process HTTP error responses. + + For 200 error codes, the response object is returned immediately. + + For non-200 error codes, this simply passes the job on to the + ‘http_error_()’ handler methods, via *note + OpenerDirector.error(): 2969. Eventually, *note + HTTPDefaultErrorHandler: 293f. will raise an *note HTTPError: 8fc. + if no other handler handles the error. + + -- Method: HTTPErrorProcessor.https_response (request, response) + + Process HTTPS error responses. + + The behavior is same as *note http_response(): 29b1. + + +File: python.info, Node: Examples<21>, Next: Legacy interface, Prev: HTTPErrorProcessor Objects, Up: urllib request — Extensible library for opening URLs + +5.21.6.23 Examples +.................. + +In addition to the examples below, more examples are given in *note +HOWTO Fetch Internet Resources Using The urllib Package: 29b5. + +This example gets the python.org main page and displays the first 300 +bytes of it. + + >>> import urllib.request + >>> with urllib.request.urlopen('http://www.python.org/') as f: + ... print(f.read(300)) + ... + b'\n\n\n\n\n\n + \n + Python Programming ' + +Note that urlopen returns a bytes object. This is because there is no +way for urlopen to automatically determine the encoding of the byte +stream it receives from the HTTP server. In general, a program will +decode the returned bytes object to string once it determines or guesses +the appropriate encoding. + +The following W3C document, +‘https://www.w3.org/International/O-charset’, lists the various ways in +which an (X)HTML or an XML document could have specified its encoding +information. + +As the python.org website uses `utf-8' encoding as specified in its meta +tag, we will use the same for decoding the bytes object. + + >>> with urllib.request.urlopen('http://www.python.org/') as f: + ... print(f.read(100).decode('utf-8')) + ... + <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtm + +It is also possible to achieve the same result without using the *note +context manager: 568. approach. + + >>> import urllib.request + >>> f = urllib.request.urlopen('http://www.python.org/') + >>> print(f.read(100).decode('utf-8')) + <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtm + +In the following example, we are sending a data-stream to the stdin of a +CGI and reading the data it returns to us. Note that this example will +only work when the Python installation supports SSL. + + >>> import urllib.request + >>> req = urllib.request.Request(url='https://localhost/cgi-bin/test.cgi', + ... data=b'This data is passed to stdin of the CGI') + >>> with urllib.request.urlopen(req) as f: + ... print(f.read().decode('utf-8')) + ... + Got Data: "This data is passed to stdin of the CGI" + +The code for the sample CGI used in the above example is: + + #!/usr/bin/env python + import sys + data = sys.stdin.read() + print('Content-type: text/plain\n\nGot Data: "%s"' % data) + +Here is an example of doing a ‘PUT’ request using *note Request: 8f6.: + + import urllib.request + DATA = b'some data' + req = urllib.request.Request(url='http://localhost:8080', data=DATA,method='PUT') + with urllib.request.urlopen(req) as f: + pass + print(f.status) + print(f.reason) + +Use of Basic HTTP Authentication: + + import urllib.request + # Create an OpenerDirector with support for Basic HTTP Authentication... + auth_handler = urllib.request.HTTPBasicAuthHandler() + auth_handler.add_password(realm='PDQ Application', + uri='https://mahler:8092/site-updates.py', + user='klem', + passwd='kadidd!ehopper') + opener = urllib.request.build_opener(auth_handler) + # ...and install it globally so it can be used with urlopen. + urllib.request.install_opener(opener) + urllib.request.urlopen('http://www.example.com/login.html') + +*note build_opener(): 293c. provides many handlers by default, including +a *note ProxyHandler: 293a. By default, *note ProxyHandler: 293a. uses +the environment variables named ‘<scheme>_proxy’, where ‘<scheme>’ is +the URL scheme involved. For example, the ‘http_proxy’ environment +variable is read to obtain the HTTP proxy’s URL. + +This example replaces the default *note ProxyHandler: 293a. with one +that uses programmatically-supplied proxy URLs, and adds proxy +authorization support with *note ProxyBasicAuthHandler: 2950. + + proxy_handler = urllib.request.ProxyHandler({'http': 'http://www.example.com:3128/'}) + proxy_auth_handler = urllib.request.ProxyBasicAuthHandler() + proxy_auth_handler.add_password('realm', 'host', 'username', 'password') + + opener = urllib.request.build_opener(proxy_handler, proxy_auth_handler) + # This time, rather than install the OpenerDirector, we use it directly: + opener.open('http://www.example.com/login.html') + +Adding HTTP headers: + +Use the `headers' argument to the *note Request: 8f6. constructor, or: + + import urllib.request + req = urllib.request.Request('http://www.example.com/') + req.add_header('Referer', 'http://www.python.org/') + # Customize the default User-Agent header value: + req.add_header('User-Agent', 'urllib-example/0.1 (Contact: . . .)') + r = urllib.request.urlopen(req) + +*note OpenerDirector: 2938. automatically adds a ‘User-Agent’ header to +every *note Request: 8f6. To change this: + + import urllib.request + opener = urllib.request.build_opener() + opener.addheaders = [('User-agent', 'Mozilla/5.0')] + opener.open('http://www.example.com/') + +Also, remember that a few standard headers (‘Content-Length’, +‘Content-Type’ and ‘Host’) are added when the *note Request: 8f6. is +passed to *note urlopen(): 78c. (or *note OpenerDirector.open(): 8fa.). +Here is an example session that uses the ‘GET’ method to retrieve a URL +containing parameters: + + >>> import urllib.request + >>> import urllib.parse + >>> params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) + >>> url = "http://www.musi-cal.com/cgi-bin/query?%s" % params + >>> with urllib.request.urlopen(url) as f: + ... print(f.read().decode('utf-8')) + ... + +The following example uses the ‘POST’ method instead. Note that params +output from urlencode is encoded to bytes before it is sent to urlopen +as data: + + >>> import urllib.request + >>> import urllib.parse + >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) + >>> data = data.encode('ascii') + >>> with urllib.request.urlopen("http://requestb.in/xrbl82xr", data) as f: + ... print(f.read().decode('utf-8')) + ... + +The following example uses an explicitly specified HTTP proxy, +overriding environment settings: + + >>> import urllib.request + >>> proxies = {'http': 'http://proxy.example.com:8080/'} + >>> opener = urllib.request.FancyURLopener(proxies) + >>> with opener.open("http://www.python.org") as f: + ... f.read().decode('utf-8') + ... + +The following example uses no proxies at all, overriding environment +settings: + + >>> import urllib.request + >>> opener = urllib.request.FancyURLopener({}) + >>> with opener.open("http://www.python.org/") as f: + ... f.read().decode('utf-8') + ... + + +File: python.info, Node: Legacy interface, Next: urllib request Restrictions, Prev: Examples<21>, Up: urllib request — Extensible library for opening URLs + +5.21.6.24 Legacy interface +.......................... + +The following functions and classes are ported from the Python 2 module +‘urllib’ (as opposed to ‘urllib2’). They might become deprecated at +some point in the future. + + -- Function: urllib.request.urlretrieve (url, filename=None, + reporthook=None, data=None) + + Copy a network object denoted by a URL to a local file. If the URL + points to a local file, the object will not be copied unless + filename is supplied. Return a tuple ‘(filename, headers)’ where + `filename' is the local file name under which the object can be + found, and `headers' is whatever the ‘info()’ method of the object + returned by *note urlopen(): 78c. returned (for a remote object). + Exceptions are the same as for *note urlopen(): 78c. + + The second argument, if present, specifies the file location to + copy to (if absent, the location will be a tempfile with a + generated name). The third argument, if present, is a callable + that will be called once on establishment of the network connection + and once after each block read thereafter. The callable will be + passed three arguments; a count of blocks transferred so far, a + block size in bytes, and the total size of the file. The third + argument may be ‘-1’ on older FTP servers which do not return a + file size in response to a retrieval request. + + The following example illustrates the most common usage scenario: + + >>> import urllib.request + >>> local_filename, headers = urllib.request.urlretrieve('http://python.org/') + >>> html = open(local_filename) + >>> html.close() + + If the `url' uses the ‘http:’ scheme identifier, the optional + `data' argument may be given to specify a ‘POST’ request (normally + the request type is ‘GET’). The `data' argument must be a bytes + object in standard ‘application/x-www-form-urlencoded’ format; see + the *note urllib.parse.urlencode(): 78b. function. + + *note urlretrieve(): 29b8. will raise ‘ContentTooShortError’ when + it detects that the amount of data available was less than the + expected amount (which is the size reported by a `Content-Length' + header). This can occur, for example, when the download is + interrupted. + + The `Content-Length' is treated as a lower bound: if there’s more + data to read, urlretrieve reads more data, but if less data is + available, it raises the exception. + + You can still retrieve the downloaded data in this case, it is + stored in the ‘content’ attribute of the exception instance. + + If no `Content-Length' header was supplied, urlretrieve can not + check the size of the data it has downloaded, and just returns it. + In this case you just have to assume that the download was + successful. + + -- Function: urllib.request.urlcleanup () + + Cleans up temporary files that may have been left behind by + previous calls to *note urlretrieve(): 29b8. + + -- Class: urllib.request.URLopener (proxies=None, **x509) + + Deprecated since version 3.3. + + Base class for opening and reading URLs. Unless you need to + support opening objects using schemes other than ‘http:’, ‘ftp:’, + or ‘file:’, you probably want to use *note FancyURLopener: 2936. + + By default, the *note URLopener: 2935. class sends a ‘User-Agent’ + header of ‘urllib/VVV’, where `VVV' is the *note urllib: 11d. + version number. Applications can define their own ‘User-Agent’ + header by subclassing *note URLopener: 2935. or *note + FancyURLopener: 2936. and setting the class attribute *note + version: 29ba. to an appropriate string value in the subclass + definition. + + The optional `proxies' parameter should be a dictionary mapping + scheme names to proxy URLs, where an empty dictionary turns proxies + off completely. Its default value is ‘None’, in which case + environmental proxy settings will be used if present, as discussed + in the definition of *note urlopen(): 78c, above. + + Additional keyword parameters, collected in `x509', may be used for + authentication of the client when using the ‘https:’ scheme. The + keywords `key_file' and `cert_file' are supported to provide an SSL + key and certificate; both are needed to support client + authentication. + + *note URLopener: 2935. objects will raise an *note OSError: 1d3. + exception if the server returns an error code. + + -- Method: open (fullurl, data=None) + + Open `fullurl' using the appropriate protocol. This method + sets up cache and proxy information, then calls the + appropriate open method with its input arguments. If the + scheme is not recognized, *note open_unknown(): 29bc. is + called. The `data' argument has the same meaning as the + `data' argument of *note urlopen(): 78c. + + This method always quotes `fullurl' using *note quote(): 40d. + + -- Method: open_unknown (fullurl, data=None) + + Overridable interface to open unknown URL types. + + -- Method: retrieve (url, filename=None, reporthook=None, + data=None) + + Retrieves the contents of `url' and places it in `filename'. + The return value is a tuple consisting of a local filename and + either an *note email.message.Message: 541. object containing + the response headers (for remote URLs) or ‘None’ (for local + URLs). The caller must then open and read the contents of + `filename'. If `filename' is not given and the URL refers to + a local file, the input filename is returned. If the URL is + non-local and `filename' is not given, the filename is the + output of *note tempfile.mktemp(): 1932. with a suffix that + matches the suffix of the last path component of the input + URL. If `reporthook' is given, it must be a function accepting + three numeric parameters: A chunk number, the maximum size + chunks are read in and the total size of the download (-1 if + unknown). It will be called once at the start and after each + chunk of data is read from the network. `reporthook' is + ignored for local URLs. + + If the `url' uses the ‘http:’ scheme identifier, the optional + `data' argument may be given to specify a ‘POST’ request + (normally the request type is ‘GET’). The `data' argument + must in standard ‘application/x-www-form-urlencoded’ format; + see the *note urllib.parse.urlencode(): 78b. function. + + -- Attribute: version + + Variable that specifies the user agent of the opener object. + To get *note urllib: 11d. to tell servers that it is a + particular user agent, set this in a subclass as a class + variable or in the constructor before calling the base + constructor. + + -- Class: urllib.request.FancyURLopener (...) + + Deprecated since version 3.3. + + *note FancyURLopener: 2936. subclasses *note URLopener: 2935. + providing default handling for the following HTTP response codes: + 301, 302, 303, 307 and 401. For the 30x response codes listed + above, the ‘Location’ header is used to fetch the actual URL. For + 401 response codes (authentication required), basic HTTP + authentication is performed. For the 30x response codes, recursion + is bounded by the value of the `maxtries' attribute, which defaults + to 10. + + For all other response codes, the method ‘http_error_default()’ is + called which you can override in subclasses to handle the error + appropriately. + + Note: According to the letter of RFC 2616(1), 301 and 302 + responses to POST requests must not be automatically + redirected without confirmation by the user. In reality, + browsers do allow automatic redirection of these responses, + changing the POST to a GET, and *note urllib: 11d. reproduces + this behaviour. + + The parameters to the constructor are the same as those for *note + URLopener: 2935. + + Note: When performing basic authentication, a *note + FancyURLopener: 2936. instance calls its *note + prompt_user_passwd(): 29be. method. The default + implementation asks the users for the required information on + the controlling terminal. A subclass may override this method + to support more appropriate behavior if needed. + + The *note FancyURLopener: 2936. class offers one additional method + that should be overloaded to provide the appropriate behavior: + + -- Method: prompt_user_passwd (host, realm) + + Return information needed to authenticate the user at the + given host in the specified security realm. The return value + should be a tuple, ‘(user, password)’, which can be used for + basic authentication. + + The implementation prompts for this information on the + terminal; an application should override this method to use an + appropriate interaction model in the local environment. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: urllib request Restrictions, Prev: Legacy interface, Up: urllib request — Extensible library for opening URLs + +5.21.6.25 ‘urllib.request’ Restrictions +....................................... + + * Currently, only the following protocols are supported: HTTP + (versions 0.9 and 1.0), FTP, local files, and data URLs. + + Changed in version 3.4: Added support for data URLs. + + * The caching feature of *note urlretrieve(): 29b8. has been disabled + until someone finds the time to hack proper processing of + Expiration time headers. + + * There should be a function to query whether a particular URL is in + the cache. + + * For backward compatibility, if a URL appears to point to a local + file but the file can’t be opened, the URL is re-interpreted using + the FTP protocol. This can sometimes cause confusing error + messages. + + * The *note urlopen(): 78c. and *note urlretrieve(): 29b8. functions + can cause arbitrarily long delays while waiting for a network + connection to be set up. This means that it is difficult to build + an interactive Web client using these functions without using + threads. + + * The data returned by *note urlopen(): 78c. or *note urlretrieve(): + 29b8. is the raw data returned by the server. This may be binary + data (such as an image), plain text or (for example) HTML. The HTTP + protocol provides type information in the reply header, which can + be inspected by looking at the ‘Content-Type’ header. If the + returned data is HTML, you can use the module *note html.parser: + 92. to parse it. + + * The code handling the FTP protocol cannot differentiate between a + file and a directory. This can lead to unexpected behavior when + attempting to read a URL that points to a file that is not + accessible. If the URL ends in a ‘/’, it is assumed to refer to a + directory and will be handled accordingly. But if an attempt to + read a file leads to a 550 error (meaning the URL cannot be found + or is not accessible, often for permission reasons), then the path + is treated as a directory in order to handle the case when a + directory is specified by a URL but the trailing ‘/’ has been left + off. This can cause misleading results when you try to fetch a + file whose read permissions make it inaccessible; the FTP code will + try to read it, fail with a 550 error, and then perform a directory + listing for the unreadable file. If fine-grained control is + needed, consider using the *note ftplib: 84. module, subclassing + *note FancyURLopener: 2936, or changing `_urlopener' to meet your + needs. + + +File: python.info, Node: urllib response — Response classes used by urllib, Next: urllib parse — Parse URLs into components, Prev: urllib request — Extensible library for opening URLs, Up: Internet Protocols and Support + +5.21.7 ‘urllib.response’ — Response classes used by urllib +---------------------------------------------------------- + +The *note urllib.response: 121. module defines functions and classes +which define a minimal file like interface, including ‘read()’ and +‘readline()’. The typical response object is an addinfourl instance, +which defines an ‘info()’ method and that returns headers and a +‘geturl()’ method that returns the url. Functions defined by this +module are used internally by the *note urllib.request: 120. module. + + +File: python.info, Node: urllib parse — Parse URLs into components, Next: urllib error — Exception classes raised by urllib request, Prev: urllib response — Response classes used by urllib, Up: Internet Protocols and Support + +5.21.8 ‘urllib.parse’ — Parse URLs into components +-------------------------------------------------- + +`Source code:' Lib/urllib/parse.py(1) + +__________________________________________________________________ + +This module defines a standard interface to break Uniform Resource +Locator (URL) strings up in components (addressing scheme, network +location, path etc.), to combine the components back into a URL string, +and to convert a “relative URL” to an absolute URL given a “base URL.” + +The module has been designed to match the Internet RFC on Relative +Uniform Resource Locators. It supports the following URL schemes: +‘file’, ‘ftp’, ‘gopher’, ‘hdl’, ‘http’, ‘https’, ‘imap’, ‘mailto’, +‘mms’, ‘news’, ‘nntp’, ‘prospero’, ‘rsync’, ‘rtsp’, ‘rtspu’, ‘sftp’, +‘shttp’, ‘sip’, ‘sips’, ‘snews’, ‘svn’, ‘svn+ssh’, ‘telnet’, ‘wais’, +‘ws’, ‘wss’. + +The *note urllib.parse: 11f. module defines functions that fall into two +broad categories: URL parsing and URL quoting. These are covered in +detail in the following sections. + +* Menu: + +* URL Parsing:: +* Parsing ASCII Encoded Bytes:: +* Structured Parse Results:: +* URL Quoting:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/urllib/parse.py + + +File: python.info, Node: URL Parsing, Next: Parsing ASCII Encoded Bytes, Up: urllib parse — Parse URLs into components + +5.21.8.1 URL Parsing +.................... + +The URL parsing functions focus on splitting a URL string into its +components, or on combining URL components into a URL string. + + -- Function: urllib.parse.urlparse (urlstring, scheme='', + allow_fragments=True) + + Parse a URL into six components, returning a 6-item *note named + tuple: aa3. This corresponds to the general structure of a URL: + ‘scheme://netloc/path;parameters?query#fragment’. Each tuple item + is a string, possibly empty. The components are not broken up in + smaller parts (for example, the network location is a single + string), and % escapes are not expanded. The delimiters as shown + above are not part of the result, except for a leading slash in the + `path' component, which is retained if present. For example: + + >>> from urllib.parse import urlparse + >>> o = urlparse('http://www.cwi.nl:80/%7Eguido/Python.html') + >>> o + ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', + params='', query='', fragment='') + >>> o.scheme + 'http' + >>> o.port + 80 + >>> o.geturl() + 'http://www.cwi.nl:80/%7Eguido/Python.html' + + Following the syntax specifications in RFC 1808(1), urlparse + recognizes a netloc only if it is properly introduced by ‘//’. + Otherwise the input is presumed to be a relative URL and thus to + start with a path component. + + >>> from urllib.parse import urlparse + >>> urlparse('//www.cwi.nl:80/%7Eguido/Python.html') + ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', + params='', query='', fragment='') + >>> urlparse('www.cwi.nl/%7Eguido/Python.html') + ParseResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html', + params='', query='', fragment='') + >>> urlparse('help/Python.html') + ParseResult(scheme='', netloc='', path='help/Python.html', params='', + query='', fragment='') + + The `scheme' argument gives the default addressing scheme, to be + used only if the URL does not specify one. It should be the same + type (text or bytes) as `urlstring', except that the default value + ‘''’ is always allowed, and is automatically converted to ‘b''’ if + appropriate. + + If the `allow_fragments' argument is false, fragment identifiers + are not recognized. Instead, they are parsed as part of the path, + parameters or query component, and ‘fragment’ is set to the empty + string in the return value. + + The return value is a *note named tuple: aa3, which means that its + items can be accessed by index or as named attributes, which are: + + Attribute Index Value Value if not present + + --------------------------------------------------------------------------------------------- + + ‘scheme’ 0 URL scheme specifier `scheme' parameter + + + ‘netloc’ 1 Network location part empty string + + + ‘path’ 2 Hierarchical path empty string + + + ‘params’ 3 Parameters for last path empty string + element + + + ‘query’ 4 Query component empty string + + + ‘fragment’ 5 Fragment identifier empty string + + + ‘username’ User name *note None: 157. + + + ‘password’ Password *note None: 157. + + + ‘hostname’ Host name (lower case) *note None: 157. + + + ‘port’ Port number as integer, if *note None: 157. + present + + + Reading the ‘port’ attribute will raise a *note ValueError: 1fb. if + an invalid port is specified in the URL. See section *note + Structured Parse Results: 29c4. for more information on the result + object. + + Unmatched square brackets in the ‘netloc’ attribute will raise a + *note ValueError: 1fb. + + Characters in the ‘netloc’ attribute that decompose under NFKC + normalization (as used by the IDNA encoding) into any of ‘/’, ‘?’, + ‘#’, ‘@’, or ‘:’ will raise a *note ValueError: 1fb. If the URL is + decomposed before parsing, no error will be raised. + + As is the case with all named tuples, the subclass has a few + additional methods and attributes that are particularly useful. + One such method is ‘_replace()’. The ‘_replace()’ method will + return a new ParseResult object replacing specified fields with new + values. + + >>> from urllib.parse import urlparse + >>> u = urlparse('//www.cwi.nl:80/%7Eguido/Python.html') + >>> u + ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', + params='', query='', fragment='') + >>> u._replace(scheme='http') + ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', + params='', query='', fragment='') + + Changed in version 3.2: Added IPv6 URL parsing capabilities. + + Changed in version 3.3: The fragment is now parsed for all URL + schemes (unless `allow_fragment' is false), in accordance with RFC + 3986(2). Previously, a whitelist of schemes that support fragments + existed. + + Changed in version 3.6: Out-of-range port numbers now raise *note + ValueError: 1fb, instead of returning *note None: 157. + + Changed in version 3.8: Characters that affect netloc parsing under + NFKC normalization will now raise *note ValueError: 1fb. + + -- Function: urllib.parse.parse_qs (qs, keep_blank_values=False, + strict_parsing=False, encoding='utf-8', errors='replace', + max_num_fields=None, separator='&') + + Parse a query string given as a string argument (data of type + ‘application/x-www-form-urlencoded’). Data are returned as a + dictionary. The dictionary keys are the unique query variable + names and the values are lists of values for each name. + + The optional argument `keep_blank_values' is a flag indicating + whether blank values in percent-encoded queries should be treated + as blank strings. A true value indicates that blanks should be + retained as blank strings. The default false value indicates that + blank values are to be ignored and treated as if they were not + included. + + The optional argument `strict_parsing' is a flag indicating what to + do with parsing errors. If false (the default), errors are + silently ignored. If true, errors raise a *note ValueError: 1fb. + exception. + + The optional `encoding' and `errors' parameters specify how to + decode percent-encoded sequences into Unicode characters, as + accepted by the *note bytes.decode(): c38. method. + + The optional argument `max_num_fields' is the maximum number of + fields to read. If set, then throws a *note ValueError: 1fb. if + there are more than `max_num_fields' fields read. + + The optional argument `separator' is the symbol to use for + separating the query arguments. It defaults to ‘&’. + + Use the *note urllib.parse.urlencode(): 78b. function (with the + ‘doseq’ parameter set to ‘True’) to convert such dictionaries into + query strings. + + Changed in version 3.2: Add `encoding' and `errors' parameters. + + Changed in version 3.8: Added `max_num_fields' parameter. + + Changed in version 3.8.8: Added `separator' parameter with the + default value of ‘&’. Python versions earlier than Python 3.8.8 + allowed using both ‘;’ and ‘&’ as query parameter separator. This + has been changed to allow only a single separator key, with ‘&’ as + the default separator. + + -- Function: urllib.parse.parse_qsl (qs, keep_blank_values=False, + strict_parsing=False, encoding='utf-8', errors='replace', + max_num_fields=None, separator='&') + + Parse a query string given as a string argument (data of type + ‘application/x-www-form-urlencoded’). Data are returned as a list + of name, value pairs. + + The optional argument `keep_blank_values' is a flag indicating + whether blank values in percent-encoded queries should be treated + as blank strings. A true value indicates that blanks should be + retained as blank strings. The default false value indicates that + blank values are to be ignored and treated as if they were not + included. + + The optional argument `strict_parsing' is a flag indicating what to + do with parsing errors. If false (the default), errors are + silently ignored. If true, errors raise a *note ValueError: 1fb. + exception. + + The optional `encoding' and `errors' parameters specify how to + decode percent-encoded sequences into Unicode characters, as + accepted by the *note bytes.decode(): c38. method. + + The optional argument `max_num_fields' is the maximum number of + fields to read. If set, then throws a *note ValueError: 1fb. if + there are more than `max_num_fields' fields read. + + The optional argument `separator' is the symbol to use for + separating the query arguments. It defaults to ‘&’. + + Use the *note urllib.parse.urlencode(): 78b. function to convert + such lists of pairs into query strings. + + Changed in version 3.2: Add `encoding' and `errors' parameters. + + Changed in version 3.8: Added `max_num_fields' parameter. + + Changed in version 3.8.8: Added `separator' parameter with the + default value of ‘&’. Python versions earlier than Python 3.8.8 + allowed using both ‘;’ and ‘&’ as query parameter separator. This + has been changed to allow only a single separator key, with ‘&’ as + the default separator. + + -- Function: urllib.parse.urlunparse (parts) + + Construct a URL from a tuple as returned by ‘urlparse()’. The + `parts' argument can be any six-item iterable. This may result in + a slightly different, but equivalent URL, if the URL that was + parsed originally had unnecessary delimiters (for example, a ‘?’ + with an empty query; the RFC states that these are equivalent). + + -- Function: urllib.parse.urlsplit (urlstring, scheme='', + allow_fragments=True) + + This is similar to *note urlparse(): 5fa, but does not split the + params from the URL. This should generally be used instead of *note + urlparse(): 5fa. if the more recent URL syntax allowing parameters + to be applied to each segment of the `path' portion of the URL (see + RFC 2396(3)) is wanted. A separate function is needed to separate + the path segments and parameters. This function returns a 5-item + *note named tuple: aa3.: + + (addressing scheme, network location, path, query, fragment identifier). + + The return value is a *note named tuple: aa3, its items can be + accessed by index or as named attributes: + + Attribute Index Value Value if not present + + -------------------------------------------------------------------------------------------- + + ‘scheme’ 0 URL scheme specifier `scheme' parameter + + + ‘netloc’ 1 Network location part empty string + + + ‘path’ 2 Hierarchical path empty string + + + ‘query’ 3 Query component empty string + + + ‘fragment’ 4 Fragment identifier empty string + + + ‘username’ User name *note None: 157. + + + ‘password’ Password *note None: 157. + + + ‘hostname’ Host name (lower case) *note None: 157. + + + ‘port’ Port number as integer, if *note None: 157. + present + + + Reading the ‘port’ attribute will raise a *note ValueError: 1fb. if + an invalid port is specified in the URL. See section *note + Structured Parse Results: 29c4. for more information on the result + object. + + Unmatched square brackets in the ‘netloc’ attribute will raise a + *note ValueError: 1fb. + + Characters in the ‘netloc’ attribute that decompose under NFKC + normalization (as used by the IDNA encoding) into any of ‘/’, ‘?’, + ‘#’, ‘@’, or ‘:’ will raise a *note ValueError: 1fb. If the URL is + decomposed before parsing, no error will be raised. + + Changed in version 3.6: Out-of-range port numbers now raise *note + ValueError: 1fb, instead of returning *note None: 157. + + Changed in version 3.8: Characters that affect netloc parsing under + NFKC normalization will now raise *note ValueError: 1fb. + + -- Function: urllib.parse.urlunsplit (parts) + + Combine the elements of a tuple as returned by *note urlsplit(): + 5f9. into a complete URL as a string. The `parts' argument can be + any five-item iterable. This may result in a slightly different, + but equivalent URL, if the URL that was parsed originally had + unnecessary delimiters (for example, a ? with an empty query; the + RFC states that these are equivalent). + + -- Function: urllib.parse.urljoin (base, url, allow_fragments=True) + + Construct a full (“absolute”) URL by combining a “base URL” + (`base') with another URL (`url'). Informally, this uses + components of the base URL, in particular the addressing scheme, + the network location and (part of) the path, to provide missing + components in the relative URL. For example: + + >>> from urllib.parse import urljoin + >>> urljoin('http://www.cwi.nl/%7Eguido/Python.html', 'FAQ.html') + 'http://www.cwi.nl/%7Eguido/FAQ.html' + + The `allow_fragments' argument has the same meaning and default as + for *note urlparse(): 5fa. + + Note: If `url' is an absolute URL (that is, starting with ‘//’ + or ‘scheme://’), the `url'’s host name and/or scheme will be + present in the result. For example: + + >>> urljoin('http://www.cwi.nl/%7Eguido/Python.html', + ... '//www.python.org/%7Eguido') + 'http://www.python.org/%7Eguido' + + If you do not want that behavior, preprocess the `url' with *note + urlsplit(): 5f9. and *note urlunsplit(): 29c6, removing possible + `scheme' and `netloc' parts. + + Changed in version 3.5: Behaviour updated to match the semantics + defined in RFC 3986(4). + + -- Function: urllib.parse.urldefrag (url) + + If `url' contains a fragment identifier, return a modified version + of `url' with no fragment identifier, and the fragment identifier + as a separate string. If there is no fragment identifier in `url', + return `url' unmodified and an empty string. + + The return value is a *note named tuple: aa3, its items can be + accessed by index or as named attributes: + + Attribute Index Value Value if not present + + -------------------------------------------------------------------------------------------- + + ‘url’ 0 URL with no fragment empty string + + + ‘fragment’ 1 Fragment identifier empty string + + + See section *note Structured Parse Results: 29c4. for more + information on the result object. + + Changed in version 3.2: Result is a structured object rather than a + simple 2-tuple. + + -- Function: urllib.parse.unwrap (url) + + Extract the url from a wrapped URL (that is, a string formatted as + ‘<URL:scheme://host/path>’, ‘<scheme://host/path>’, + ‘URL:scheme://host/path’ or ‘scheme://host/path’). If `url' is not + a wrapped URL, it is returned without changes. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc1808.html + + (2) https://tools.ietf.org/html/rfc3986.html + + (3) https://tools.ietf.org/html/rfc2396.html + + (4) https://tools.ietf.org/html/rfc3986.html + + +File: python.info, Node: Parsing ASCII Encoded Bytes, Next: Structured Parse Results, Prev: URL Parsing, Up: urllib parse — Parse URLs into components + +5.21.8.2 Parsing ASCII Encoded Bytes +.................................... + +The URL parsing functions were originally designed to operate on +character strings only. In practice, it is useful to be able to +manipulate properly quoted and encoded URLs as sequences of ASCII bytes. +Accordingly, the URL parsing functions in this module all operate on +*note bytes: 331. and *note bytearray: 332. objects in addition to *note +str: 330. objects. + +If *note str: 330. data is passed in, the result will also contain only +*note str: 330. data. If *note bytes: 331. or *note bytearray: 332. +data is passed in, the result will contain only *note bytes: 331. data. + +Attempting to mix *note str: 330. data with *note bytes: 331. or *note +bytearray: 332. in a single function call will result in a *note +TypeError: 192. being raised, while attempting to pass in non-ASCII byte +values will trigger *note UnicodeDecodeError: 1fd. + +To support easier conversion of result objects between *note str: 330. +and *note bytes: 331, all return values from URL parsing functions +provide either an ‘encode()’ method (when the result contains *note str: +330. data) or a ‘decode()’ method (when the result contains *note bytes: +331. data). The signatures of these methods match those of the +corresponding *note str: 330. and *note bytes: 331. methods (except that +the default encoding is ‘'ascii'’ rather than ‘'utf-8'’). Each produces +a value of a corresponding type that contains either *note bytes: 331. +data (for ‘encode()’ methods) or *note str: 330. data (for ‘decode()’ +methods). + +Applications that need to operate on potentially improperly quoted URLs +that may contain non-ASCII data will need to do their own decoding from +bytes to characters before invoking the URL parsing methods. + +The behaviour described in this section applies only to the URL parsing +functions. The URL quoting functions use their own rules when producing +or consuming byte sequences as detailed in the documentation of the +individual URL quoting functions. + +Changed in version 3.2: URL parsing functions now accept ASCII encoded +byte sequences + + +File: python.info, Node: Structured Parse Results, Next: URL Quoting, Prev: Parsing ASCII Encoded Bytes, Up: urllib parse — Parse URLs into components + +5.21.8.3 Structured Parse Results +................................. + +The result objects from the *note urlparse(): 5fa, *note urlsplit(): +5f9. and *note urldefrag(): bde. functions are subclasses of the *note +tuple: 47e. type. These subclasses add the attributes listed in the +documentation for those functions, the encoding and decoding support +described in the previous section, as well as an additional method: + + -- Method: urllib.parse.SplitResult.geturl () + + Return the re-combined version of the original URL as a string. + This may differ from the original URL in that the scheme may be + normalized to lower case and empty components may be dropped. + Specifically, empty parameters, queries, and fragment identifiers + will be removed. + + For *note urldefrag(): bde. results, only empty fragment + identifiers will be removed. For *note urlsplit(): 5f9. and *note + urlparse(): 5fa. results, all noted changes will be made to the URL + returned by this method. + + The result of this method remains unchanged if passed back through + the original parsing function: + + >>> from urllib.parse import urlsplit + >>> url = 'HTTP://www.Python.org/doc/#' + >>> r1 = urlsplit(url) + >>> r1.geturl() + 'http://www.Python.org/doc/' + >>> r2 = urlsplit(r1.geturl()) + >>> r2.geturl() + 'http://www.Python.org/doc/' + +The following classes provide the implementations of the structured +parse results when operating on *note str: 330. objects: + + -- Class: urllib.parse.DefragResult (url, fragment) + + Concrete class for *note urldefrag(): bde. results containing *note + str: 330. data. The ‘encode()’ method returns a *note + DefragResultBytes: 29cc. instance. + + New in version 3.2. + + -- Class: urllib.parse.ParseResult (scheme, netloc, path, params, + query, fragment) + + Concrete class for *note urlparse(): 5fa. results containing *note + str: 330. data. The ‘encode()’ method returns a *note + ParseResultBytes: 29ce. instance. + + -- Class: urllib.parse.SplitResult (scheme, netloc, path, query, + fragment) + + Concrete class for *note urlsplit(): 5f9. results containing *note + str: 330. data. The ‘encode()’ method returns a *note + SplitResultBytes: 29d0. instance. + +The following classes provide the implementations of the parse results +when operating on *note bytes: 331. or *note bytearray: 332. objects: + + -- Class: urllib.parse.DefragResultBytes (url, fragment) + + Concrete class for *note urldefrag(): bde. results containing *note + bytes: 331. data. The ‘decode()’ method returns a *note + DefragResult: 29cb. instance. + + New in version 3.2. + + -- Class: urllib.parse.ParseResultBytes (scheme, netloc, path, params, + query, fragment) + + Concrete class for *note urlparse(): 5fa. results containing *note + bytes: 331. data. The ‘decode()’ method returns a *note + ParseResult: 29cd. instance. + + New in version 3.2. + + -- Class: urllib.parse.SplitResultBytes (scheme, netloc, path, query, + fragment) + + Concrete class for *note urlsplit(): 5f9. results containing *note + bytes: 331. data. The ‘decode()’ method returns a *note + SplitResult: 29cf. instance. + + New in version 3.2. + + +File: python.info, Node: URL Quoting, Prev: Structured Parse Results, Up: urllib parse — Parse URLs into components + +5.21.8.4 URL Quoting +.................... + +The URL quoting functions focus on taking program data and making it +safe for use as URL components by quoting special characters and +appropriately encoding non-ASCII text. They also support reversing +these operations to recreate the original data from the contents of a +URL component if that task isn’t already covered by the URL parsing +functions above. + + -- Function: urllib.parse.quote (string, safe='/', encoding=None, + errors=None) + + Replace special characters in `string' using the ‘%xx’ escape. + Letters, digits, and the characters ‘'_.-~'’ are never quoted. By + default, this function is intended for quoting the path section of + URL. The optional `safe' parameter specifies additional ASCII + characters that should not be quoted — its default value is ‘'/'’. + + `string' may be either a *note str: 330. or a *note bytes: 331. + + Changed in version 3.7: Moved from RFC 2396(1) to RFC 3986(2) for + quoting URL strings. “~” is now included in the set of unreserved + characters. + + The optional `encoding' and `errors' parameters specify how to deal + with non-ASCII characters, as accepted by the *note str.encode(): + c37. method. `encoding' defaults to ‘'utf-8'’. `errors' defaults + to ‘'strict'’, meaning unsupported characters raise a *note + UnicodeEncodeError: 1fc. `encoding' and `errors' must not be + supplied if `string' is a *note bytes: 331, or a *note TypeError: + 192. is raised. + + Note that ‘quote(string, safe, encoding, errors)’ is equivalent to + ‘quote_from_bytes(string.encode(encoding, errors), safe)’. + + Example: ‘quote('/El Niño/')’ yields ‘'/El%20Ni%C3%B1o/'’. + + -- Function: urllib.parse.quote_plus (string, safe='', encoding=None, + errors=None) + + Like *note quote(): 40d, but also replace spaces by plus signs, as + required for quoting HTML form values when building up a query + string to go into a URL. Plus signs in the original string are + escaped unless they are included in `safe'. It also does not have + `safe' default to ‘'/'’. + + Example: ‘quote_plus('/El Niño/')’ yields ‘'%2FEl+Ni%C3%B1o%2F'’. + + -- Function: urllib.parse.quote_from_bytes (bytes, safe='/') + + Like *note quote(): 40d, but accepts a *note bytes: 331. object + rather than a *note str: 330, and does not perform string-to-bytes + encoding. + + Example: ‘quote_from_bytes(b'a&\xef')’ yields ‘'a%26%EF'’. + + -- Function: urllib.parse.unquote (string, encoding='utf-8', + errors='replace') + + Replace ‘%xx’ escapes by their single-character equivalent. The + optional `encoding' and `errors' parameters specify how to decode + percent-encoded sequences into Unicode characters, as accepted by + the *note bytes.decode(): c38. method. + + `string' must be a *note str: 330. + + `encoding' defaults to ‘'utf-8'’. `errors' defaults to + ‘'replace'’, meaning invalid sequences are replaced by a + placeholder character. + + Example: ‘unquote('/El%20Ni%C3%B1o/')’ yields ‘'/El Niño/'’. + + -- Function: urllib.parse.unquote_plus (string, encoding='utf-8', + errors='replace') + + Like *note unquote(): 2946, but also replace plus signs by spaces, + as required for unquoting HTML form values. + + `string' must be a *note str: 330. + + Example: ‘unquote_plus('/El+Ni%C3%B1o/')’ yields ‘'/El Niño/'’. + + -- Function: urllib.parse.unquote_to_bytes (string) + + Replace ‘%xx’ escapes by their single-octet equivalent, and return + a *note bytes: 331. object. + + `string' may be either a *note str: 330. or a *note bytes: 331. + + If it is a *note str: 330, unescaped non-ASCII characters in + `string' are encoded into UTF-8 bytes. + + Example: ‘unquote_to_bytes('a%26%EF')’ yields ‘b'a&\xef'’. + + -- Function: urllib.parse.urlencode (query, doseq=False, safe='', + encoding=None, errors=None, quote_via=quote_plus) + + Convert a mapping object or a sequence of two-element tuples, which + may contain *note str: 330. or *note bytes: 331. objects, to a + percent-encoded ASCII text string. If the resultant string is to + be used as a `data' for POST operation with the *note urlopen(): + 78c. function, then it should be encoded to bytes, otherwise it + would result in a *note TypeError: 192. + + The resulting string is a series of ‘key=value’ pairs separated by + ‘'&'’ characters, where both `key' and `value' are quoted using the + `quote_via' function. By default, *note quote_plus(): bdf. is used + to quote the values, which means spaces are quoted as a ‘'+'’ + character and ‘/’ characters are encoded as ‘%2F’, which follows + the standard for GET requests + (‘application/x-www-form-urlencoded’). An alternate function that + can be passed as `quote_via' is *note quote(): 40d, which will + encode spaces as ‘%20’ and not encode ‘/’ characters. For maximum + control of what is quoted, use ‘quote’ and specify a value for + `safe'. + + When a sequence of two-element tuples is used as the `query' + argument, the first element of each tuple is a key and the second + is a value. The value element in itself can be a sequence and in + that case, if the optional parameter `doseq' is evaluates to + ‘True’, individual ‘key=value’ pairs separated by ‘'&'’ are + generated for each element of the value sequence for the key. The + order of parameters in the encoded string will match the order of + parameter tuples in the sequence. + + The `safe', `encoding', and `errors' parameters are passed down to + `quote_via' (the `encoding' and `errors' parameters are only passed + when a query element is a *note str: 330.). + + To reverse this encoding process, *note parse_qs(): 2eb. and *note + parse_qsl(): 2ec. are provided in this module to parse query + strings into Python data structures. + + Refer to *note urllib examples: 29b6. to find out how urlencode + method can be used for generating query string for a URL or data + for POST. + + Changed in version 3.2: Query parameter supports bytes and string + objects. + + New in version 3.5: `quote_via' parameter. + +See also +........ + +RFC 3986(3) - Uniform Resource Identifiers + + This is the current standard (STD66). Any changes to urllib.parse + module should conform to this. Certain deviations could be + observed, which are mostly for backward compatibility purposes and + for certain de-facto parsing requirements as commonly observed in + major browsers. + +RFC 2732(4) - Format for Literal IPv6 Addresses in URL’s. + + This specifies the parsing requirements of IPv6 URLs. + +RFC 2396(5) - Uniform Resource Identifiers (URI): Generic Syntax + + Document describing the generic syntactic requirements for both + Uniform Resource Names (URNs) and Uniform Resource Locators (URLs). + +RFC 2368(6) - The mailto URL scheme. + + Parsing requirements for mailto URL schemes. + +RFC 1808(7) - Relative Uniform Resource Locators + + This Request For Comments includes the rules for joining an + absolute and a relative URL, including a fair number of “Abnormal + Examples” which govern the treatment of border cases. + +RFC 1738(8) - Uniform Resource Locators (URL) + + This specifies the formal syntax and semantics of absolute URLs. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2396.html + + (2) https://tools.ietf.org/html/rfc3986.html + + (3) https://tools.ietf.org/html/rfc3986.html + + (4) https://tools.ietf.org/html/rfc2732.html + + (5) https://tools.ietf.org/html/rfc2396.html + + (6) https://tools.ietf.org/html/rfc2368.html + + (7) https://tools.ietf.org/html/rfc1808.html + + (8) https://tools.ietf.org/html/rfc1738.html + + +File: python.info, Node: urllib error — Exception classes raised by urllib request, Next: urllib robotparser — Parser for robots txt, Prev: urllib parse — Parse URLs into components, Up: Internet Protocols and Support + +5.21.9 ‘urllib.error’ — Exception classes raised by urllib.request +------------------------------------------------------------------ + +`Source code:' Lib/urllib/error.py(1) + +__________________________________________________________________ + +The *note urllib.error: 11e. module defines the exception classes for +exceptions raised by *note urllib.request: 120. The base exception +class is *note URLError: 2937. + +The following exceptions are raised by *note urllib.error: 11e. as +appropriate: + + -- Exception: urllib.error.URLError + + The handlers raise this exception (or derived exceptions) when they + run into a problem. It is a subclass of *note OSError: 1d3. + + -- Attribute: reason + + The reason for this error. It can be a message string or + another exception instance. + + Changed in version 3.3: *note URLError: 2937. has been made a + subclass of *note OSError: 1d3. instead of *note IOError: 992. + + -- Exception: urllib.error.HTTPError + + Though being an exception (a subclass of *note URLError: 2937.), an + *note HTTPError: 8fc. can also function as a non-exceptional + file-like return value (the same thing that *note urlopen(): 78c. + returns). This is useful when handling exotic HTTP errors, such as + requests for authentication. + + -- Attribute: code + + An HTTP status code as defined in RFC 2616(2). This numeric + value corresponds to a value found in the dictionary of codes + as found in *note + http.server.BaseHTTPRequestHandler.responses: 29d9. + + -- Attribute: reason + + This is usually a string explaining the reason for this error. + + -- Attribute: headers + + The HTTP response headers for the HTTP request that caused the + *note HTTPError: 8fc. + + New in version 3.4. + + -- Exception: urllib.error.ContentTooShortError (msg, content) + + This exception is raised when the *note urlretrieve(): 29b8. + function detects that the amount of the downloaded data is less + than the expected amount (given by the `Content-Length' header). + The ‘content’ attribute stores the downloaded (and supposedly + truncated) data. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/urllib/error.py + + (2) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: urllib robotparser — Parser for robots txt, Next: http — HTTP modules, Prev: urllib error — Exception classes raised by urllib request, Up: Internet Protocols and Support + +5.21.10 ‘urllib.robotparser’ — Parser for robots.txt +---------------------------------------------------- + +`Source code:' Lib/urllib/robotparser.py(1) + +__________________________________________________________________ + +This module provides a single class, *note RobotFileParser: 5ad, which +answers questions about whether or not a particular user agent can fetch +a URL on the Web site that published the ‘robots.txt’ file. For more +details on the structure of ‘robots.txt’ files, see +‘http://www.robotstxt.org/orig.html’. + + -- Class: urllib.robotparser.RobotFileParser (url='') + + This class provides methods to read, parse and answer questions + about the ‘robots.txt’ file at `url'. + + -- Method: set_url (url) + + Sets the URL referring to a ‘robots.txt’ file. + + -- Method: read () + + Reads the ‘robots.txt’ URL and feeds it to the parser. + + -- Method: parse (lines) + + Parses the lines argument. + + -- Method: can_fetch (useragent, url) + + Returns ‘True’ if the `useragent' is allowed to fetch the + `url' according to the rules contained in the parsed + ‘robots.txt’ file. + + -- Method: mtime () + + Returns the time the ‘robots.txt’ file was last fetched. This + is useful for long-running web spiders that need to check for + new ‘robots.txt’ files periodically. + + -- Method: modified () + + Sets the time the ‘robots.txt’ file was last fetched to the + current time. + + -- Method: crawl_delay (useragent) + + Returns the value of the ‘Crawl-delay’ parameter from + ‘robots.txt’ for the `useragent' in question. If there is no + such parameter or it doesn’t apply to the `useragent' + specified or the ‘robots.txt’ entry for this parameter has + invalid syntax, return ‘None’. + + New in version 3.6. + + -- Method: request_rate (useragent) + + Returns the contents of the ‘Request-rate’ parameter from + ‘robots.txt’ as a *note named tuple: aa3. + ‘RequestRate(requests, seconds)’. If there is no such + parameter or it doesn’t apply to the `useragent' specified or + the ‘robots.txt’ entry for this parameter has invalid syntax, + return ‘None’. + + New in version 3.6. + + -- Method: site_maps () + + Returns the contents of the ‘Sitemap’ parameter from + ‘robots.txt’ in the form of a *note list(): 262. If there is + no such parameter or the ‘robots.txt’ entry for this parameter + has invalid syntax, return ‘None’. + + New in version 3.8. + +The following example demonstrates basic use of the *note +RobotFileParser: 5ad. class: + + >>> import urllib.robotparser + >>> rp = urllib.robotparser.RobotFileParser() + >>> rp.set_url("http://www.musi-cal.com/robots.txt") + >>> rp.read() + >>> rrate = rp.request_rate("*") + >>> rrate.requests + 3 + >>> rrate.seconds + 20 + >>> rp.crawl_delay("*") + 6 + >>> rp.can_fetch("*", "http://www.musi-cal.com/cgi-bin/search?city=San+Francisco") + False + >>> rp.can_fetch("*", "http://www.musi-cal.com/") + True + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/urllib/robotparser.py + + +File: python.info, Node: http — HTTP modules, Next: http client — HTTP protocol client, Prev: urllib robotparser — Parser for robots txt, Up: Internet Protocols and Support + +5.21.11 ‘http’ — HTTP modules +----------------------------- + +`Source code:' Lib/http/__init__.py(1) + +__________________________________________________________________ + +*note http: 93. is a package that collects several modules for working +with the HyperText Transfer Protocol: + + * *note http.client: 94. is a low-level HTTP protocol client; for + high-level URL opening use *note urllib.request: 120. + + * *note http.server: 97. contains basic HTTP server classes based on + *note socketserver: f0. + + * *note http.cookies: 96. has utilities for implementing state + management with cookies + + * *note http.cookiejar: 95. provides persistence of cookies + +*note http: 93. is also a module that defines a number of HTTP status +codes and associated messages through the *note http.HTTPStatus: 6e2. +enum: + + -- Class: http.HTTPStatus + + New in version 3.5. + + A subclass of *note enum.IntEnum: 58d. that defines a set of HTTP + status codes, reason phrases and long descriptions written in + English. + + Usage: + + >>> from http import HTTPStatus + >>> HTTPStatus.OK + <HTTPStatus.OK: 200> + >>> HTTPStatus.OK == 200 + True + >>> HTTPStatus.OK.value + 200 + >>> HTTPStatus.OK.phrase + 'OK' + >>> HTTPStatus.OK.description + 'Request fulfilled, document follows' + >>> list(HTTPStatus) + [<HTTPStatus.CONTINUE: 100>, <HTTPStatus.SWITCHING_PROTOCOLS: 101>, ...] + +* Menu: + +* HTTP status codes:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/http/__init__.py + + +File: python.info, Node: HTTP status codes, Up: http — HTTP modules + +5.21.11.1 HTTP status codes +........................... + +Supported, IANA-registered(1) status codes available in *note +http.HTTPStatus: 6e2. are: + +Code Enum Name Details + +---------------------------------------------------------------------------------------------------------------------------------------- + +‘100’ ‘CONTINUE’ HTTP/1.1 RFC 7231(2), Section 6.2.1 + + +‘101’ ‘SWITCHING_PROTOCOLS’ HTTP/1.1 RFC 7231(3), Section 6.2.2 + + +‘102’ ‘PROCESSING’ WebDAV RFC 2518(4), Section 10.1 + + +‘200’ ‘OK’ HTTP/1.1 RFC 7231(5), Section 6.3.1 + + +‘201’ ‘CREATED’ HTTP/1.1 RFC 7231(6), Section 6.3.2 + + +‘202’ ‘ACCEPTED’ HTTP/1.1 RFC 7231(7), Section 6.3.3 + + +‘203’ ‘NON_AUTHORITATIVE_INFORMATION’ HTTP/1.1 RFC 7231(8), Section 6.3.4 + + +‘204’ ‘NO_CONTENT’ HTTP/1.1 RFC 7231(9), Section 6.3.5 + + +‘205’ ‘RESET_CONTENT’ HTTP/1.1 RFC 7231(10), Section 6.3.6 + + +‘206’ ‘PARTIAL_CONTENT’ HTTP/1.1 RFC 7233(11), Section 4.1 + + +‘207’ ‘MULTI_STATUS’ WebDAV RFC 4918(12), Section 11.1 + + +‘208’ ‘ALREADY_REPORTED’ WebDAV Binding Extensions RFC 5842(13), Section 7.1 (Experimental) + + +‘226’ ‘IM_USED’ Delta Encoding in HTTP RFC 3229(14), Section 10.4.1 + + +‘300’ ‘MULTIPLE_CHOICES’ HTTP/1.1 RFC 7231(15), Section 6.4.1 + + +‘301’ ‘MOVED_PERMANENTLY’ HTTP/1.1 RFC 7231(16), Section 6.4.2 + + +‘302’ ‘FOUND’ HTTP/1.1 RFC 7231(17), Section 6.4.3 + + +‘303’ ‘SEE_OTHER’ HTTP/1.1 RFC 7231(18), Section 6.4.4 + + +‘304’ ‘NOT_MODIFIED’ HTTP/1.1 RFC 7232(19), Section 4.1 + + +‘305’ ‘USE_PROXY’ HTTP/1.1 RFC 7231(20), Section 6.4.5 + + +‘307’ ‘TEMPORARY_REDIRECT’ HTTP/1.1 RFC 7231(21), Section 6.4.7 + + +‘308’ ‘PERMANENT_REDIRECT’ Permanent Redirect RFC 7238(22), Section 3 (Experimental) + + +‘400’ ‘BAD_REQUEST’ HTTP/1.1 RFC 7231(23), Section 6.5.1 + + +‘401’ ‘UNAUTHORIZED’ HTTP/1.1 Authentication RFC 7235(24), Section 3.1 + + +‘402’ ‘PAYMENT_REQUIRED’ HTTP/1.1 RFC 7231(25), Section 6.5.2 + + +‘403’ ‘FORBIDDEN’ HTTP/1.1 RFC 7231(26), Section 6.5.3 + + +‘404’ ‘NOT_FOUND’ HTTP/1.1 RFC 7231(27), Section 6.5.4 + + +‘405’ ‘METHOD_NOT_ALLOWED’ HTTP/1.1 RFC 7231(28), Section 6.5.5 + + +‘406’ ‘NOT_ACCEPTABLE’ HTTP/1.1 RFC 7231(29), Section 6.5.6 + + +‘407’ ‘PROXY_AUTHENTICATION_REQUIRED’ HTTP/1.1 Authentication RFC 7235(30), Section 3.2 + + +‘408’ ‘REQUEST_TIMEOUT’ HTTP/1.1 RFC 7231(31), Section 6.5.7 + + +‘409’ ‘CONFLICT’ HTTP/1.1 RFC 7231(32), Section 6.5.8 + + +‘410’ ‘GONE’ HTTP/1.1 RFC 7231(33), Section 6.5.9 + + +‘411’ ‘LENGTH_REQUIRED’ HTTP/1.1 RFC 7231(34), Section 6.5.10 + + +‘412’ ‘PRECONDITION_FAILED’ HTTP/1.1 RFC 7232(35), Section 4.2 + + +‘413’ ‘REQUEST_ENTITY_TOO_LARGE’ HTTP/1.1 RFC 7231(36), Section 6.5.11 + + +‘414’ ‘REQUEST_URI_TOO_LONG’ HTTP/1.1 RFC 7231(37), Section 6.5.12 + + +‘415’ ‘UNSUPPORTED_MEDIA_TYPE’ HTTP/1.1 RFC 7231(38), Section 6.5.13 + + +‘416’ ‘REQUESTED_RANGE_NOT_SATISFIABLE’ HTTP/1.1 Range Requests RFC 7233(39), Section 4.4 + + +‘417’ ‘EXPECTATION_FAILED’ HTTP/1.1 RFC 7231(40), Section 6.5.14 + + +‘421’ ‘MISDIRECTED_REQUEST’ HTTP/2 RFC 7540(41), Section 9.1.2 + + +‘422’ ‘UNPROCESSABLE_ENTITY’ WebDAV RFC 4918(42), Section 11.2 + + +‘423’ ‘LOCKED’ WebDAV RFC 4918(43), Section 11.3 + + +‘424’ ‘FAILED_DEPENDENCY’ WebDAV RFC 4918(44), Section 11.4 + + +‘426’ ‘UPGRADE_REQUIRED’ HTTP/1.1 RFC 7231(45), Section 6.5.15 + + +‘428’ ‘PRECONDITION_REQUIRED’ Additional HTTP Status Codes RFC 6585(46) + + +‘429’ ‘TOO_MANY_REQUESTS’ Additional HTTP Status Codes RFC 6585(47) + + +‘431’ ‘REQUEST_HEADER_FIELDS_TOO_LARGE’ Additional HTTP Status Codes RFC 6585(48) + + +‘451’ ‘UNAVAILABLE_FOR_LEGAL_REASONS’ An HTTP Status Code to Report Legal Obstacles RFC 7725(49) + + +‘500’ ‘INTERNAL_SERVER_ERROR’ HTTP/1.1 RFC 7231(50), Section 6.6.1 + + +‘501’ ‘NOT_IMPLEMENTED’ HTTP/1.1 RFC 7231(51), Section 6.6.2 + + +‘502’ ‘BAD_GATEWAY’ HTTP/1.1 RFC 7231(52), Section 6.6.3 + + +‘503’ ‘SERVICE_UNAVAILABLE’ HTTP/1.1 RFC 7231(53), Section 6.6.4 + + +‘504’ ‘GATEWAY_TIMEOUT’ HTTP/1.1 RFC 7231(54), Section 6.6.5 + + +‘505’ ‘HTTP_VERSION_NOT_SUPPORTED’ HTTP/1.1 RFC 7231(55), Section 6.6.6 + + +‘506’ ‘VARIANT_ALSO_NEGOTIATES’ Transparent Content Negotiation in HTTP RFC 2295(56), Section 8.1 (Experimental) + + +‘507’ ‘INSUFFICIENT_STORAGE’ WebDAV RFC 4918(57), Section 11.5 + + +‘508’ ‘LOOP_DETECTED’ WebDAV Binding Extensions RFC 5842(58), Section 7.2 (Experimental) + + +‘510’ ‘NOT_EXTENDED’ An HTTP Extension Framework RFC 2774(59), Section 7 (Experimental) + + +‘511’ ‘NETWORK_AUTHENTICATION_REQUIRED’ Additional HTTP Status Codes RFC 6585(60), Section 6 + + +In order to preserve backwards compatibility, enum values are also +present in the *note http.client: 94. module in the form of constants. +The enum name is equal to the constant name (i.e. ‘http.HTTPStatus.OK’ +is also available as ‘http.client.OK’). + +Changed in version 3.7: Added ‘421 MISDIRECTED_REQUEST’ status code. + +New in version 3.8: Added ‘451 UNAVAILABLE_FOR_LEGAL_REASONS’ status +code. + + ---------- Footnotes ---------- + + (1) +https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml + + (2) https://tools.ietf.org/html/rfc7231.html + + (3) https://tools.ietf.org/html/rfc7231.html + + (4) https://tools.ietf.org/html/rfc2518.html + + (5) https://tools.ietf.org/html/rfc7231.html + + (6) https://tools.ietf.org/html/rfc7231.html + + (7) https://tools.ietf.org/html/rfc7231.html + + (8) https://tools.ietf.org/html/rfc7231.html + + (9) https://tools.ietf.org/html/rfc7231.html + + (10) https://tools.ietf.org/html/rfc7231.html + + (11) https://tools.ietf.org/html/rfc7233.html + + (12) https://tools.ietf.org/html/rfc4918.html + + (13) https://tools.ietf.org/html/rfc5842.html + + (14) https://tools.ietf.org/html/rfc3229.html + + (15) https://tools.ietf.org/html/rfc7231.html + + (16) https://tools.ietf.org/html/rfc7231.html + + (17) https://tools.ietf.org/html/rfc7231.html + + (18) https://tools.ietf.org/html/rfc7231.html + + (19) https://tools.ietf.org/html/rfc7232.html + + (20) https://tools.ietf.org/html/rfc7231.html + + (21) https://tools.ietf.org/html/rfc7231.html + + (22) https://tools.ietf.org/html/rfc7238.html + + (23) https://tools.ietf.org/html/rfc7231.html + + (24) https://tools.ietf.org/html/rfc7235.html + + (25) https://tools.ietf.org/html/rfc7231.html + + (26) https://tools.ietf.org/html/rfc7231.html + + (27) https://tools.ietf.org/html/rfc7231.html + + (28) https://tools.ietf.org/html/rfc7231.html + + (29) https://tools.ietf.org/html/rfc7231.html + + (30) https://tools.ietf.org/html/rfc7235.html + + (31) https://tools.ietf.org/html/rfc7231.html + + (32) https://tools.ietf.org/html/rfc7231.html + + (33) https://tools.ietf.org/html/rfc7231.html + + (34) https://tools.ietf.org/html/rfc7231.html + + (35) https://tools.ietf.org/html/rfc7232.html + + (36) https://tools.ietf.org/html/rfc7231.html + + (37) https://tools.ietf.org/html/rfc7231.html + + (38) https://tools.ietf.org/html/rfc7231.html + + (39) https://tools.ietf.org/html/rfc7233.html + + (40) https://tools.ietf.org/html/rfc7231.html + + (41) https://tools.ietf.org/html/rfc7540.html + + (42) https://tools.ietf.org/html/rfc4918.html + + (43) https://tools.ietf.org/html/rfc4918.html + + (44) https://tools.ietf.org/html/rfc4918.html + + (45) https://tools.ietf.org/html/rfc7231.html + + (46) https://tools.ietf.org/html/rfc6585.html + + (47) https://tools.ietf.org/html/rfc6585.html + + (48) https://tools.ietf.org/html/rfc6585.html + + (49) https://tools.ietf.org/html/rfc7725.html + + (50) https://tools.ietf.org/html/rfc7231.html + + (51) https://tools.ietf.org/html/rfc7231.html + + (52) https://tools.ietf.org/html/rfc7231.html + + (53) https://tools.ietf.org/html/rfc7231.html + + (54) https://tools.ietf.org/html/rfc7231.html + + (55) https://tools.ietf.org/html/rfc7231.html + + (56) https://tools.ietf.org/html/rfc2295.html + + (57) https://tools.ietf.org/html/rfc4918.html + + (58) https://tools.ietf.org/html/rfc5842.html + + (59) https://tools.ietf.org/html/rfc2774.html + + (60) https://tools.ietf.org/html/rfc6585.html + + +File: python.info, Node: http client — HTTP protocol client, Next: ftplib — FTP protocol client, Prev: http — HTTP modules, Up: Internet Protocols and Support + +5.21.12 ‘http.client’ — HTTP protocol client +-------------------------------------------- + +`Source code:' Lib/http/client.py(1) + +__________________________________________________________________ + +This module defines classes which implement the client side of the HTTP +and HTTPS protocols. It is normally not used directly — the module +*note urllib.request: 120. uses it to handle URLs that use HTTP and +HTTPS. + +See also +........ + +The Requests package(2) is recommended for a higher-level HTTP client +interface. + + Note: HTTPS support is only available if Python was compiled with + SSL support (through the *note ssl: f3. module). + +The module provides the following classes: + + -- Class: http.client.HTTPConnection (host, port=None[, timeout], + source_address=None, blocksize=8192) + + An *note HTTPConnection: 392. instance represents one transaction + with an HTTP server. It should be instantiated passing it a host + and optional port number. If no port number is passed, the port is + extracted from the host string if it has the form ‘host:port’, else + the default HTTP port (80) is used. If the optional `timeout' + parameter is given, blocking operations (like connection attempts) + will timeout after that many seconds (if it is not given, the + global default timeout setting is used). The optional + `source_address' parameter may be a tuple of a (host, port) to use + as the source address the HTTP connection is made from. The + optional `blocksize' parameter sets the buffer size in bytes for + sending a file-like message body. + + For example, the following calls all create instances that connect + to the server at the same host and port: + + >>> h1 = http.client.HTTPConnection('www.python.org') + >>> h2 = http.client.HTTPConnection('www.python.org:80') + >>> h3 = http.client.HTTPConnection('www.python.org', 80) + >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10) + + Changed in version 3.2: `source_address' was added. + + Changed in version 3.4: The `strict' parameter was removed. HTTP + 0.9-style “Simple Responses” are not longer supported. + + Changed in version 3.7: `blocksize' parameter was added. + + -- Class: http.client.HTTPSConnection (host, port=None, key_file=None, + cert_file=None[, timeout], source_address=None, *, + context=None, check_hostname=None, blocksize=8192) + + A subclass of *note HTTPConnection: 392. that uses SSL for + communication with secure servers. Default port is ‘443’. If + `context' is specified, it must be a *note ssl.SSLContext: 3e4. + instance describing the various SSL options. + + Please read *note Security considerations: 22a2. for more + information on best practices. + + Changed in version 3.2: `source_address', `context' and + `check_hostname' were added. + + Changed in version 3.2: This class now supports HTTPS virtual hosts + if possible (that is, if *note ssl.HAS_SNI: 23d2. is true). + + Changed in version 3.4: The `strict' parameter was removed. HTTP + 0.9-style “Simple Responses” are no longer supported. + + Changed in version 3.4.3: This class now performs all the necessary + certificate and hostname checks by default. To revert to the + previous, unverified, behavior ‘ssl._create_unverified_context()’ + can be passed to the `context' parameter. + + Changed in version 3.8: This class now enables TLS 1.3 *note + ssl.SSLContext.post_handshake_auth: 224. for the default `context' + or when `cert_file' is passed with a custom `context'. + + Deprecated since version 3.6: `key_file' and `cert_file' are + deprecated in favor of `context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + + The `check_hostname' parameter is also deprecated; the *note + ssl.SSLContext.check_hostname: 23bd. attribute of `context' should + be used instead. + + -- Class: http.client.HTTPResponse (sock, debuglevel=0, method=None, + url=None) + + Class whose instances are returned upon successful connection. Not + instantiated directly by user. + + Changed in version 3.4: The `strict' parameter was removed. HTTP + 0.9 style “Simple Responses” are no longer supported. + +This module provides the following function: + + -- Function: http.client.parse_headers (fp) + + Parse the headers from a file pointer `fp' representing a HTTP + request/response. The file has to be a ‘BufferedIOBase’ reader + (i.e. not text) and must provide a valid RFC 2822(3) style header. + + This function returns an instance of ‘http.client.HTTPMessage’ that + holds the header fields, but no payload (the same as *note + HTTPResponse.msg: 29ee. and *note + http.server.BaseHTTPRequestHandler.headers: 29ef.). After + returning, the file pointer `fp' is ready to read the HTTP body. + + Note: *note parse_headers(): 29ed. does not parse the + start-line of a HTTP message; it only parses the ‘Name: value’ + lines. The file has to be ready to read these field lines, so + the first line should already be consumed before calling the + function. + +The following exceptions are raised as appropriate: + + -- Exception: http.client.HTTPException + + The base class of the other exceptions in this module. It is a + subclass of *note Exception: 1a9. + + -- Exception: http.client.NotConnected + + A subclass of *note HTTPException: 29f0. + + -- Exception: http.client.InvalidURL + + A subclass of *note HTTPException: 29f0, raised if a port is given + and is either non-numeric or empty. + + -- Exception: http.client.UnknownProtocol + + A subclass of *note HTTPException: 29f0. + + -- Exception: http.client.UnknownTransferEncoding + + A subclass of *note HTTPException: 29f0. + + -- Exception: http.client.UnimplementedFileMode + + A subclass of *note HTTPException: 29f0. + + -- Exception: http.client.IncompleteRead + + A subclass of *note HTTPException: 29f0. + + -- Exception: http.client.ImproperConnectionState + + A subclass of *note HTTPException: 29f0. + + -- Exception: http.client.CannotSendRequest + + A subclass of *note ImproperConnectionState: 29f7. + + -- Exception: http.client.CannotSendHeader + + A subclass of *note ImproperConnectionState: 29f7. + + -- Exception: http.client.ResponseNotReady + + A subclass of *note ImproperConnectionState: 29f7. + + -- Exception: http.client.BadStatusLine + + A subclass of *note HTTPException: 29f0. Raised if a server + responds with a HTTP status code that we don’t understand. + + -- Exception: http.client.LineTooLong + + A subclass of *note HTTPException: 29f0. Raised if an excessively + long line is received in the HTTP protocol from the server. + + -- Exception: http.client.RemoteDisconnected + + A subclass of *note ConnectionResetError: 9a0. and *note + BadStatusLine: 29fb. Raised by *note HTTPConnection.getresponse(): + 6e4. when the attempt to read the response results in no data read + from the connection, indicating that the remote end has closed the + connection. + + New in version 3.5: Previously, *note BadStatusLine: 29fb.‘('')’ + was raised. + +The constants defined in this module are: + + -- Data: http.client.HTTP_PORT + + The default port for the HTTP protocol (always ‘80’). + + -- Data: http.client.HTTPS_PORT + + The default port for the HTTPS protocol (always ‘443’). + + -- Data: http.client.responses + + This dictionary maps the HTTP 1.1 status codes to the W3C names. + + Example: ‘http.client.responses[http.client.NOT_FOUND]’ is ‘'Not + Found'’. + +See *note HTTP status codes: 29e9. for a list of HTTP status codes that +are available in this module as constants. + +* Menu: + +* HTTPConnection Objects:: +* HTTPResponse Objects:: +* Examples: Examples<22>. +* HTTPMessage Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/http/client.py + + (2) https://requests.readthedocs.io/en/master/ + + (3) https://tools.ietf.org/html/rfc2822.html + + +File: python.info, Node: HTTPConnection Objects, Next: HTTPResponse Objects, Up: http client — HTTP protocol client + +5.21.12.1 HTTPConnection Objects +................................ + +*note HTTPConnection: 392. instances have the following methods: + + -- Method: HTTPConnection.request (method, url, body=None, headers={}, + *, encode_chunked=False) + + This will send a request to the server using the HTTP request + method `method' and the selector `url'. + + If `body' is specified, the specified data is sent after the + headers are finished. It may be a *note str: 330, a *note + bytes-like object: 5e8, an open *note file object: b42, or an + iterable of *note bytes: 331. If `body' is a string, it is encoded + as ISO-8859-1, the default for HTTP. If it is a bytes-like object, + the bytes are sent as is. If it is a *note file object: b42, the + contents of the file is sent; this file object should support at + least the ‘read()’ method. If the file object is an instance of + *note io.TextIOBase: c39, the data returned by the ‘read()’ method + will be encoded as ISO-8859-1, otherwise the data returned by + ‘read()’ is sent as is. If `body' is an iterable, the elements of + the iterable are sent as is until the iterable is exhausted. + + The `headers' argument should be a mapping of extra HTTP headers to + send with the request. + + If `headers' contains neither Content-Length nor Transfer-Encoding, + but there is a request body, one of those header fields will be + added automatically. If `body' is ‘None’, the Content-Length + header is set to ‘0’ for methods that expect a body (‘PUT’, ‘POST’, + and ‘PATCH’). If `body' is a string or a bytes-like object that is + not also a *note file: b42, the Content-Length header is set to its + length. Any other type of `body' (files and iterables in general) + will be chunk-encoded, and the Transfer-Encoding header will + automatically be set instead of Content-Length. + + The `encode_chunked' argument is only relevant if Transfer-Encoding + is specified in `headers'. If `encode_chunked' is ‘False’, the + HTTPConnection object assumes that all encoding is handled by the + calling code. If it is ‘True’, the body will be chunk-encoded. + + Note: Chunked transfer encoding has been added to the HTTP + protocol version 1.1. Unless the HTTP server is known to + handle HTTP 1.1, the caller must either specify the + Content-Length, or must pass a *note str: 330. or bytes-like + object that is not also a file as the body representation. + + New in version 3.2: `body' can now be an iterable. + + Changed in version 3.6: If neither Content-Length nor + Transfer-Encoding are set in `headers', file and iterable `body' + objects are now chunk-encoded. The `encode_chunked' argument was + added. No attempt is made to determine the Content-Length for file + objects. + + -- Method: HTTPConnection.getresponse () + + Should be called after a request is sent to get the response from + the server. Returns an *note HTTPResponse: a11. instance. + + Note: Note that you must have read the whole response before + you can send a new request to the server. + + Changed in version 3.5: If a *note ConnectionError: 6e6. or + subclass is raised, the *note HTTPConnection: 392. object will be + ready to reconnect when a new request is sent. + + -- Method: HTTPConnection.set_debuglevel (level) + + Set the debugging level. The default debug level is ‘0’, meaning + no debugging output is printed. Any value greater than ‘0’ will + cause all currently defined debug output to be printed to stdout. + The ‘debuglevel’ is passed to any new *note HTTPResponse: a11. + objects that are created. + + New in version 3.1. + + -- Method: HTTPConnection.set_tunnel (host, port=None, headers=None) + + Set the host and the port for HTTP Connect Tunnelling. This allows + running the connection through a proxy server. + + The host and port arguments specify the endpoint of the tunneled + connection (i.e. the address included in the CONNECT request, + `not' the address of the proxy server). + + The headers argument should be a mapping of extra HTTP headers to + send with the CONNECT request. + + For example, to tunnel through a HTTPS proxy server running locally + on port 8080, we would pass the address of the proxy to the *note + HTTPSConnection: 393. constructor, and the address of the host that + we eventually want to reach to the *note set_tunnel(): bad. method: + + >>> import http.client + >>> conn = http.client.HTTPSConnection("localhost", 8080) + >>> conn.set_tunnel("www.python.org") + >>> conn.request("HEAD","/index.html") + + New in version 3.2. + + -- Method: HTTPConnection.connect () + + Connect to the server specified when the object was created. By + default, this is called automatically when making a request if the + client does not already have a connection. + + -- Method: HTTPConnection.close () + + Close the connection to the server. + + -- Attribute: HTTPConnection.blocksize + + Buffer size in bytes for sending a file-like message body. + + New in version 3.7. + +As an alternative to using the ‘request()’ method described above, you +can also send your request step by step, by using the four functions +below. + + -- Method: HTTPConnection.putrequest (method, url, skip_host=False, + skip_accept_encoding=False) + + This should be the first call after the connection to the server + has been made. It sends a line to the server consisting of the + `method' string, the `url' string, and the HTTP version + (‘HTTP/1.1’). To disable automatic sending of ‘Host:’ or + ‘Accept-Encoding:’ headers (for example to accept additional + content encodings), specify `skip_host' or `skip_accept_encoding' + with non-False values. + + -- Method: HTTPConnection.putheader (header, argument[, ...]) + + Send an RFC 822(1)-style header to the server. It sends a line to + the server consisting of the header, a colon and a space, and the + first argument. If more arguments are given, continuation lines + are sent, each consisting of a tab and an argument. + + -- Method: HTTPConnection.endheaders (message_body=None, *, + encode_chunked=False) + + Send a blank line to the server, signalling the end of the headers. + The optional `message_body' argument can be used to pass a message + body associated with the request. + + If `encode_chunked' is ‘True’, the result of each iteration of + `message_body' will be chunk-encoded as specified in RFC 7230(2), + Section 3.3.1. How the data is encoded is dependent on the type of + `message_body'. If `message_body' implements the *note buffer + interface: 1291. the encoding will result in a single chunk. If + `message_body' is a *note collections.abc.Iterable: 1601, each + iteration of `message_body' will result in a chunk. If + `message_body' is a *note file object: b42, each call to ‘.read()’ + will result in a chunk. The method automatically signals the end + of the chunk-encoded data immediately after `message_body'. + + Note: Due to the chunked encoding specification, empty chunks + yielded by an iterator body will be ignored by the + chunk-encoder. This is to avoid premature termination of the + read of the request by the target server due to malformed + encoding. + + New in version 3.6: Chunked encoding support. The `encode_chunked' + parameter was added. + + -- Method: HTTPConnection.send (data) + + Send data to the server. This should be used directly only after + the *note endheaders(): 550. method has been called and before + *note getresponse(): 6e4. is called. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc822.html + + (2) https://tools.ietf.org/html/rfc7230.html + + +File: python.info, Node: HTTPResponse Objects, Next: Examples<22>, Prev: HTTPConnection Objects, Up: http client — HTTP protocol client + +5.21.12.2 HTTPResponse Objects +.............................. + +An *note HTTPResponse: a11. instance wraps the HTTP response from the +server. It provides access to the request headers and the entity body. +The response is an iterable object and can be used in a with statement. + +Changed in version 3.5: The *note io.BufferedIOBase: 588. interface is +now implemented and all of its reader operations are supported. + + -- Method: HTTPResponse.read ([amt]) + + Reads and returns the response body, or up to the next `amt' bytes. + + -- Method: HTTPResponse.readinto (b) + + Reads up to the next len(b) bytes of the response body into the + buffer `b'. Returns the number of bytes read. + + New in version 3.3. + + -- Method: HTTPResponse.getheader (name, default=None) + + Return the value of the header `name', or `default' if there is no + header matching `name'. If there is more than one header with the + name `name', return all of the values joined by ‘, ‘. If ‘default’ + is any iterable other than a single string, its elements are + similarly returned joined by commas. + + -- Method: HTTPResponse.getheaders () + + Return a list of (header, value) tuples. + + -- Method: HTTPResponse.fileno () + + Return the ‘fileno’ of the underlying socket. + + -- Attribute: HTTPResponse.msg + + A ‘http.client.HTTPMessage’ instance containing the response + headers. ‘http.client.HTTPMessage’ is a subclass of *note + email.message.Message: 541. + + -- Attribute: HTTPResponse.version + + HTTP protocol version used by server. 10 for HTTP/1.0, 11 for + HTTP/1.1. + + -- Attribute: HTTPResponse.status + + Status code returned by server. + + -- Attribute: HTTPResponse.reason + + Reason phrase returned by server. + + -- Attribute: HTTPResponse.debuglevel + + A debugging hook. If *note debuglevel: 2a11. is greater than zero, + messages will be printed to stdout as the response is read and + parsed. + + -- Attribute: HTTPResponse.closed + + Is ‘True’ if the stream is closed. + + +File: python.info, Node: Examples<22>, Next: HTTPMessage Objects, Prev: HTTPResponse Objects, Up: http client — HTTP protocol client + +5.21.12.3 Examples +.................. + +Here is an example session that uses the ‘GET’ method: + + >>> import http.client + >>> conn = http.client.HTTPSConnection("www.python.org") + >>> conn.request("GET", "/") + >>> r1 = conn.getresponse() + >>> print(r1.status, r1.reason) + 200 OK + >>> data1 = r1.read() # This will return entire content. + >>> # The following example demonstrates reading data in chunks. + >>> conn.request("GET", "/") + >>> r1 = conn.getresponse() + >>> while chunk := r1.read(200): + ... print(repr(chunk)) + b'<!doctype html>\n<!--[if"... + ... + >>> # Example of an invalid request + >>> conn = http.client.HTTPSConnection("docs.python.org") + >>> conn.request("GET", "/parrot.spam") + >>> r2 = conn.getresponse() + >>> print(r2.status, r2.reason) + 404 Not Found + >>> data2 = r2.read() + >>> conn.close() + +Here is an example session that uses the ‘HEAD’ method. Note that the +‘HEAD’ method never returns any data. + + >>> import http.client + >>> conn = http.client.HTTPSConnection("www.python.org") + >>> conn.request("HEAD", "/") + >>> res = conn.getresponse() + >>> print(res.status, res.reason) + 200 OK + >>> data = res.read() + >>> print(len(data)) + 0 + >>> data == b'' + True + +Here is an example session that shows how to ‘POST’ requests: + + >>> import http.client, urllib.parse + >>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'}) + >>> headers = {"Content-type": "application/x-www-form-urlencoded", + ... "Accept": "text/plain"} + >>> conn = http.client.HTTPConnection("bugs.python.org") + >>> conn.request("POST", "", params, headers) + >>> response = conn.getresponse() + >>> print(response.status, response.reason) + 302 Found + >>> data = response.read() + >>> data + b'Redirecting to <a href="http://bugs.python.org/issue12524">http://bugs.python.org/issue12524</a>' + >>> conn.close() + +Client side ‘HTTP PUT’ requests are very similar to ‘POST’ requests. +The difference lies only the server side where HTTP server will allow +resources to be created via ‘PUT’ request. It should be noted that +custom HTTP methods are also handled in *note urllib.request.Request: +8f6. by setting the appropriate method attribute. Here is an example +session that shows how to send a ‘PUT’ request using http.client: + + >>> # This creates an HTTP message + >>> # with the content of BODY as the enclosed representation + >>> # for the resource http://localhost:8080/file + ... + >>> import http.client + >>> BODY = "***filecontents***" + >>> conn = http.client.HTTPConnection("localhost", 8080) + >>> conn.request("PUT", "/file", BODY) + >>> response = conn.getresponse() + >>> print(response.status, response.reason) + 200, OK + + +File: python.info, Node: HTTPMessage Objects, Prev: Examples<22>, Up: http client — HTTP protocol client + +5.21.12.4 HTTPMessage Objects +............................. + +An ‘http.client.HTTPMessage’ instance holds the headers from an HTTP +response. It is implemented using the *note email.message.Message: 541. +class. + + +File: python.info, Node: ftplib — FTP protocol client, Next: poplib — POP3 protocol client, Prev: http client — HTTP protocol client, Up: Internet Protocols and Support + +5.21.13 ‘ftplib’ — FTP protocol client +-------------------------------------- + +`Source code:' Lib/ftplib.py(1) + +__________________________________________________________________ + +This module defines the class *note FTP: 2f0. and a few related items. +The *note FTP: 2f0. class implements the client side of the FTP +protocol. You can use this to write Python programs that perform a +variety of automated FTP jobs, such as mirroring other FTP servers. It +is also used by the module *note urllib.request: 120. to handle URLs +that use FTP. For more information on FTP (File Transfer Protocol), see +Internet RFC 959(2). + +Here’s a sample session using the *note ftplib: 84. module: + + >>> from ftplib import FTP + >>> ftp = FTP('ftp.debian.org') # connect to host, default port + >>> ftp.login() # user anonymous, passwd anonymous@ + '230 Login successful.' + >>> ftp.cwd('debian') # change into "debian" directory + >>> ftp.retrlines('LIST') # list directory contents + -rw-rw-r-- 1 1176 1176 1063 Jun 15 10:18 README + ... + drwxr-sr-x 5 1176 1176 4096 Dec 19 2000 pool + drwxr-sr-x 4 1176 1176 4096 Nov 17 2008 project + drwxr-xr-x 3 1176 1176 4096 Oct 10 2012 tools + '226 Directory send OK.' + >>> with open('README', 'wb') as fp: + >>> ftp.retrbinary('RETR README', fp.write) + '226 Transfer complete.' + >>> ftp.quit() + +The module defines the following items: + + -- Class: ftplib.FTP (host='', user='', passwd='', acct='', + timeout=None, source_address=None) + + Return a new instance of the *note FTP: 2f0. class. When `host' is + given, the method call ‘connect(host)’ is made. When `user' is + given, additionally the method call ‘login(user, passwd, acct)’ is + made (where `passwd' and `acct' default to the empty string when + not given). The optional `timeout' parameter specifies a timeout + in seconds for blocking operations like the connection attempt (if + is not specified, the global default timeout setting will be used). + `source_address' is a 2-tuple ‘(host, port)’ for the socket to bind + to as its source address before connecting. + + The *note FTP: 2f0. class supports the *note with: 6e9. statement, + e.g.: + + >>> from ftplib import FTP + >>> with FTP("ftp1.at.proftpd.org") as ftp: + ... ftp.login() + ... ftp.dir() + ... + '230 Anonymous login ok, restrictions apply.' + dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 . + dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 .. + dr-xr-xr-x 5 ftp ftp 4096 May 6 10:43 CentOS + dr-xr-xr-x 3 ftp ftp 18 Jul 10 2008 Fedora + >>> + + Changed in version 3.2: Support for the *note with: 6e9. statement + was added. + + Changed in version 3.3: `source_address' parameter was added. + + -- Class: ftplib.FTP_TLS (host='', user='', passwd='', acct='', + keyfile=None, certfile=None, context=None, timeout=None, + source_address=None) + + A *note FTP: 2f0. subclass which adds TLS support to FTP as + described in RFC 4217(3). Connect as usual to port 21 implicitly + securing the FTP control connection before authenticating. + Securing the data connection requires the user to explicitly ask + for it by calling the *note prot_p(): 2a18. method. `context' is a + *note ssl.SSLContext: 3e4. object which allows bundling SSL + configuration options, certificates and private keys into a single + (potentially long-lived) structure. Please read *note Security + considerations: 22a2. for best practices. + + `keyfile' and `certfile' are a legacy alternative to `context' – + they can point to PEM-formatted private key and certificate chain + files (respectively) for the SSL connection. + + New in version 3.2. + + Changed in version 3.3: `source_address' parameter was added. + + Changed in version 3.4: The class now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + Deprecated since version 3.6: `keyfile' and `certfile' are + deprecated in favor of `context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + + Here’s a sample session using the *note FTP_TLS: a03. class: + + >>> ftps = FTP_TLS('ftp.pureftpd.org') + >>> ftps.login() + '230 Anonymous user logged in' + >>> ftps.prot_p() + '200 Data protection level set to "private"' + >>> ftps.nlst() + ['6jack', 'OpenBSD', 'antilink', 'blogbench', 'bsdcam', 'clockspeed', 'djbdns-jedi', 'docs', 'eaccelerator-jedi', 'favicon.ico', 'francotone', 'fugu', 'ignore', 'libpuzzle', 'metalog', 'minidentd', 'misc', 'mysql-udf-global-user-variables', 'php-jenkins-hash', 'php-skein-hash', 'php-webdav', 'phpaudit', 'phpbench', 'pincaster', 'ping', 'posto', 'pub', 'public', 'public_keys', 'pure-ftpd', 'qscan', 'qtc', 'sharedance', 'skycache', 'sound', 'tmp', 'ucarp'] + + -- Exception: ftplib.error_reply + + Exception raised when an unexpected reply is received from the + server. + + -- Exception: ftplib.error_temp + + Exception raised when an error code signifying a temporary error + (response codes in the range 400–499) is received. + + -- Exception: ftplib.error_perm + + Exception raised when an error code signifying a permanent error + (response codes in the range 500–599) is received. + + -- Exception: ftplib.error_proto + + Exception raised when a reply is received from the server that does + not fit the response specifications of the File Transfer Protocol, + i.e. begin with a digit in the range 1–5. + + -- Data: ftplib.all_errors + + The set of all exceptions (as a tuple) that methods of *note FTP: + 2f0. instances may raise as a result of problems with the FTP + connection (as opposed to programming errors made by the caller). + This set includes the four exceptions listed above as well as *note + OSError: 1d3. and *note EOFError: c6c. + +See also +........ + +Module *note netrc: be. + + Parser for the ‘.netrc’ file format. The file ‘.netrc’ is + typically used by FTP clients to load user authentication + information before prompting the user. + +* Menu: + +* FTP Objects:: +* FTP_TLS Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/ftplib.py + + (2) https://tools.ietf.org/html/rfc959.html + + (3) https://tools.ietf.org/html/rfc4217.html + + +File: python.info, Node: FTP Objects, Next: FTP_TLS Objects, Up: ftplib — FTP protocol client + +5.21.13.1 FTP Objects +..................... + +Several methods are available in two flavors: one for handling text +files and another for binary files. These are named for the command +which is used followed by ‘lines’ for the text version or ‘binary’ for +the binary version. + +*note FTP: 2f0. instances have the following methods: + + -- Method: FTP.set_debuglevel (level) + + Set the instance’s debugging level. This controls the amount of + debugging output printed. The default, ‘0’, produces no debugging + output. A value of ‘1’ produces a moderate amount of debugging + output, generally a single line per request. A value of ‘2’ or + higher produces the maximum amount of debugging output, logging + each line sent and received on the control connection. + + -- Method: FTP.connect (host='', port=0, timeout=None, + source_address=None) + + Connect to the given host and port. The default port number is + ‘21’, as specified by the FTP protocol specification. It is rarely + needed to specify a different port number. This function should be + called only once for each instance; it should not be called at all + if a host was given when the instance was created. All other + methods can only be used after a connection has been made. The + optional `timeout' parameter specifies a timeout in seconds for the + connection attempt. If no `timeout' is passed, the global default + timeout setting will be used. `source_address' is a 2-tuple + ‘(host, port)’ for the socket to bind to as its source address + before connecting. + + Raises an *note auditing event: fd1. ‘ftplib.connect’ with + arguments ‘self’, ‘host’, ‘port’. + + Changed in version 3.3: `source_address' parameter was added. + + -- Method: FTP.getwelcome () + + Return the welcome message sent by the server in reply to the + initial connection. (This message sometimes contains disclaimers + or help information that may be relevant to the user.) + + -- Method: FTP.login (user='anonymous', passwd='', acct='') + + Log in as the given `user'. The `passwd' and `acct' parameters are + optional and default to the empty string. If no `user' is + specified, it defaults to ‘'anonymous'’. If `user' is + ‘'anonymous'’, the default `passwd' is ‘'anonymous@'’. This + function should be called only once for each instance, after a + connection has been established; it should not be called at all if + a host and user were given when the instance was created. Most FTP + commands are only allowed after the client has logged in. The + `acct' parameter supplies “accounting information”; few systems + implement this. + + -- Method: FTP.abort () + + Abort a file transfer that is in progress. Using this does not + always work, but it’s worth a try. + + -- Method: FTP.sendcmd (cmd) + + Send a simple command string to the server and return the response + string. + + Raises an *note auditing event: fd1. ‘ftplib.sendcmd’ with + arguments ‘self’, ‘cmd’. + + -- Method: FTP.voidcmd (cmd) + + Send a simple command string to the server and handle the response. + Return nothing if a response code corresponding to success (codes + in the range 200–299) is received. Raise *note error_reply: 2a19. + otherwise. + + Raises an *note auditing event: fd1. ‘ftplib.sendcmd’ with + arguments ‘self’, ‘cmd’. + + -- Method: FTP.retrbinary (cmd, callback, blocksize=8192, rest=None) + + Retrieve a file in binary transfer mode. `cmd' should be an + appropriate ‘RETR’ command: ‘'RETR filename'’. The `callback' + function is called for each block of data received, with a single + bytes argument giving the data block. The optional `blocksize' + argument specifies the maximum chunk size to read on the low-level + socket object created to do the actual transfer (which will also be + the largest size of the data blocks passed to `callback'). A + reasonable default is chosen. `rest' means the same thing as in + the *note transfercmd(): 2a28. method. + + -- Method: FTP.retrlines (cmd, callback=None) + + Retrieve a file or directory listing in ASCII transfer mode. `cmd' + should be an appropriate ‘RETR’ command (see *note retrbinary(): + 2a27.) or a command such as ‘LIST’ or ‘NLST’ (usually just the + string ‘'LIST'’). ‘LIST’ retrieves a list of files and information + about those files. ‘NLST’ retrieves a list of file names. The + `callback' function is called for each line with a string argument + containing the line with the trailing CRLF stripped. The default + `callback' prints the line to ‘sys.stdout’. + + -- Method: FTP.set_pasv (val) + + Enable “passive” mode if `val' is true, otherwise disable passive + mode. Passive mode is on by default. + + -- Method: FTP.storbinary (cmd, fp, blocksize=8192, callback=None, + rest=None) + + Store a file in binary transfer mode. `cmd' should be an + appropriate ‘STOR’ command: ‘"STOR filename"’. `fp' is a *note + file object: b42. (opened in binary mode) which is read until EOF + using its ‘read()’ method in blocks of size `blocksize' to provide + the data to be stored. The `blocksize' argument defaults to 8192. + `callback' is an optional single parameter callable that is called + on each block of data after it is sent. `rest' means the same + thing as in the *note transfercmd(): 2a28. method. + + Changed in version 3.2: `rest' parameter added. + + -- Method: FTP.storlines (cmd, fp, callback=None) + + Store a file in ASCII transfer mode. `cmd' should be an + appropriate ‘STOR’ command (see *note storbinary(): c9f.). Lines + are read until EOF from the *note file object: b42. `fp' (opened in + binary mode) using its *note readline(): 13ae. method to provide + the data to be stored. `callback' is an optional single parameter + callable that is called on each line after it is sent. + + -- Method: FTP.transfercmd (cmd, rest=None) + + Initiate a transfer over the data connection. If the transfer is + active, send an ‘EPRT’ or ‘PORT’ command and the transfer command + specified by `cmd', and accept the connection. If the server is + passive, send an ‘EPSV’ or ‘PASV’ command, connect to it, and start + the transfer command. Either way, return the socket for the + connection. + + If optional `rest' is given, a ‘REST’ command is sent to the + server, passing `rest' as an argument. `rest' is usually a byte + offset into the requested file, telling the server to restart + sending the file’s bytes at the requested offset, skipping over the + initial bytes. Note however that RFC 959(1) requires only that + `rest' be a string containing characters in the printable range + from ASCII code 33 to ASCII code 126. The *note transfercmd(): + 2a28. method, therefore, converts `rest' to a string, but no check + is performed on the string’s contents. If the server does not + recognize the ‘REST’ command, an *note error_reply: 2a19. exception + will be raised. If this happens, simply call *note transfercmd(): + 2a28. without a `rest' argument. + + -- Method: FTP.ntransfercmd (cmd, rest=None) + + Like *note transfercmd(): 2a28, but returns a tuple of the data + connection and the expected size of the data. If the expected size + could not be computed, ‘None’ will be returned as the expected + size. `cmd' and `rest' means the same thing as in *note + transfercmd(): 2a28. + + -- Method: FTP.mlsd (path="", facts=[]) + + List a directory in a standardized format by using ‘MLSD’ command ( + RFC 3659(2)). If `path' is omitted the current directory is + assumed. `facts' is a list of strings representing the type of + information desired (e.g. ‘["type", "size", "perm"]’). Return a + generator object yielding a tuple of two elements for every file + found in path. First element is the file name, the second one is a + dictionary containing facts about the file name. Content of this + dictionary might be limited by the `facts' argument but server is + not guaranteed to return all requested facts. + + New in version 3.3. + + -- Method: FTP.nlst (argument[, ...]) + + Return a list of file names as returned by the ‘NLST’ command. The + optional `argument' is a directory to list (default is the current + server directory). Multiple arguments can be used to pass + non-standard options to the ‘NLST’ command. + + Note: If your server supports the command, *note mlsd(): a05. + offers a better API. + + -- Method: FTP.dir (argument[, ...]) + + Produce a directory listing as returned by the ‘LIST’ command, + printing it to standard output. The optional `argument' is a + directory to list (default is the current server directory). + Multiple arguments can be used to pass non-standard options to the + ‘LIST’ command. If the last argument is a function, it is used as + a `callback' function as for *note retrlines(): 2a29.; the default + prints to ‘sys.stdout’. This method returns ‘None’. + + Note: If your server supports the command, *note mlsd(): a05. + offers a better API. + + -- Method: FTP.rename (fromname, toname) + + Rename file `fromname' on the server to `toname'. + + -- Method: FTP.delete (filename) + + Remove the file named `filename' from the server. If successful, + returns the text of the response, otherwise raises *note + error_perm: 2a1b. on permission errors or *note error_reply: 2a19. + on other errors. + + -- Method: FTP.cwd (pathname) + + Set the current directory on the server. + + -- Method: FTP.mkd (pathname) + + Create a new directory on the server. + + -- Method: FTP.pwd () + + Return the pathname of the current directory on the server. + + -- Method: FTP.rmd (dirname) + + Remove the directory named `dirname' on the server. + + -- Method: FTP.size (filename) + + Request the size of the file named `filename' on the server. On + success, the size of the file is returned as an integer, otherwise + ‘None’ is returned. Note that the ‘SIZE’ command is not + standardized, but is supported by many common server + implementations. + + -- Method: FTP.quit () + + Send a ‘QUIT’ command to the server and close the connection. This + is the “polite” way to close a connection, but it may raise an + exception if the server responds with an error to the ‘QUIT’ + command. This implies a call to the *note close(): 2a35. method + which renders the *note FTP: 2f0. instance useless for subsequent + calls (see below). + + -- Method: FTP.close () + + Close the connection unilaterally. This should not be applied to + an already closed connection such as after a successful call to + *note quit(): 2a34. After this call the *note FTP: 2f0. instance + should not be used any more (after a call to *note close(): 2a35. + or *note quit(): 2a34. you cannot reopen the connection by issuing + another *note login(): 2a23. method). + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc959.html + + (2) https://tools.ietf.org/html/rfc3659.html + + +File: python.info, Node: FTP_TLS Objects, Prev: FTP Objects, Up: ftplib — FTP protocol client + +5.21.13.2 FTP_TLS Objects +......................... + +*note FTP_TLS: a03. class inherits from *note FTP: 2f0, defining these +additional objects: + + -- Attribute: FTP_TLS.ssl_version + + The SSL version to use (defaults to *note ssl.PROTOCOL_SSLv23: + 23c2.). + + -- Method: FTP_TLS.auth () + + Set up a secure control connection by using TLS or SSL, depending + on what is specified in the *note ssl_version: 2a37. attribute. + + Changed in version 3.4: The method now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + -- Method: FTP_TLS.ccc () + + Revert control channel back to plaintext. This can be useful to + take advantage of firewalls that know how to handle NAT with + non-secure FTP without opening fixed ports. + + New in version 3.3. + + -- Method: FTP_TLS.prot_p () + + Set up secure data connection. + + -- Method: FTP_TLS.prot_c () + + Set up clear text data connection. + + +File: python.info, Node: poplib — POP3 protocol client, Next: imaplib — IMAP4 protocol client, Prev: ftplib — FTP protocol client, Up: Internet Protocols and Support + +5.21.14 ‘poplib’ — POP3 protocol client +--------------------------------------- + +`Source code:' Lib/poplib.py(1) + +__________________________________________________________________ + +This module defines a class, *note POP3: 2a3c, which encapsulates a +connection to a POP3 server and implements the protocol as defined in +RFC 1939(2). The *note POP3: 2a3c. class supports both the minimal and +optional command sets from RFC 1939(3). The *note POP3: 2a3c. class +also supports the ‘STLS’ command introduced in RFC 2595(4) to enable +encrypted communication on an already established connection. + +Additionally, this module provides a class *note POP3_SSL: bc1, which +provides support for connecting to POP3 servers that use SSL as an +underlying protocol layer. + +Note that POP3, though widely supported, is obsolescent. The +implementation quality of POP3 servers varies widely, and too many are +quite poor. If your mailserver supports IMAP, you would be better off +using the *note imaplib.IMAP4: 60b. class, as IMAP servers tend to be +better implemented. + +The *note poplib: d0. module provides two classes: + + -- Class: poplib.POP3 (host, port=POP3_PORT[, timeout]) + + This class implements the actual POP3 protocol. The connection is + created when the instance is initialized. If `port' is omitted, + the standard POP3 port (110) is used. The optional `timeout' + parameter specifies a timeout in seconds for the connection attempt + (if not specified, the global default timeout setting will be + used). + + Raises an *note auditing event: fd1. ‘poplib.connect’ with + arguments ‘self’, ‘host’, ‘port’. + + All commands will raise an *note auditing event: fd1. + ‘poplib.putline’ with arguments ‘self’ and ‘line’, where ‘line’ is + the bytes about to be sent to the remote host. + + -- Class: poplib.POP3_SSL (host, port=POP3_SSL_PORT, keyfile=None, + certfile=None, timeout=None, context=None) + + This is a subclass of *note POP3: 2a3c. that connects to the server + over an SSL encrypted socket. If `port' is not specified, 995, the + standard POP3-over-SSL port is used. `timeout' works as in the + *note POP3: 2a3c. constructor. `context' is an optional *note + ssl.SSLContext: 3e4. object which allows bundling SSL configuration + options, certificates and private keys into a single (potentially + long-lived) structure. Please read *note Security considerations: + 22a2. for best practices. + + `keyfile' and `certfile' are a legacy alternative to `context' - + they can point to PEM-formatted private key and certificate chain + files, respectively, for the SSL connection. + + Raises an *note auditing event: fd1. ‘poplib.connect’ with + arguments ‘self’, ‘host’, ‘port’. + + All commands will raise an *note auditing event: fd1. + ‘poplib.putline’ with arguments ‘self’ and ‘line’, where ‘line’ is + the bytes about to be sent to the remote host. + + Changed in version 3.2: `context' parameter added. + + Changed in version 3.4: The class now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + Deprecated since version 3.6: `keyfile' and `certfile' are + deprecated in favor of `context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + +One exception is defined as an attribute of the *note poplib: d0. +module: + + -- Exception: poplib.error_proto + + Exception raised on any errors from this module (errors from *note + socket: ef. module are not caught). The reason for the exception + is passed to the constructor as a string. + +See also +........ + +Module *note imaplib: 98. + + The standard Python IMAP module. + +Frequently Asked Questions About Fetchmail(5) + + The FAQ for the ‘fetchmail’ POP/IMAP client collects information on + POP3 server variations and RFC noncompliance that may be useful if + you need to write an application based on the POP protocol. + +* Menu: + +* POP3 Objects:: +* POP3 Example:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/poplib.py + + (2) https://tools.ietf.org/html/rfc1939.html + + (3) https://tools.ietf.org/html/rfc1939.html + + (4) https://tools.ietf.org/html/rfc2595.html + + (5) http://www.catb.org/~esr/fetchmail/fetchmail-FAQ.html + + +File: python.info, Node: POP3 Objects, Next: POP3 Example, Up: poplib — POP3 protocol client + +5.21.14.1 POP3 Objects +...................... + +All POP3 commands are represented by methods of the same name, in +lower-case; most return the response text sent by the server. + +An *note POP3: 2a3c. instance has the following methods: + + -- Method: POP3.set_debuglevel (level) + + Set the instance’s debugging level. This controls the amount of + debugging output printed. The default, ‘0’, produces no debugging + output. A value of ‘1’ produces a moderate amount of debugging + output, generally a single line per request. A value of ‘2’ or + higher produces the maximum amount of debugging output, logging + each line sent and received on the control connection. + + -- Method: POP3.getwelcome () + + Returns the greeting string sent by the POP3 server. + + -- Method: POP3.capa () + + Query the server’s capabilities as specified in RFC 2449(1). + Returns a dictionary in the form ‘{'name': ['param'...]}’. + + New in version 3.4. + + -- Method: POP3.user (username) + + Send user command, response should indicate that a password is + required. + + -- Method: POP3.pass_ (password) + + Send password, response includes message count and mailbox size. + Note: the mailbox on the server is locked until ‘quit()’ is called. + + -- Method: POP3.apop (user, secret) + + Use the more secure APOP authentication to log into the POP3 + server. + + -- Method: POP3.rpop (user) + + Use RPOP authentication (similar to UNIX r-commands) to log into + POP3 server. + + -- Method: POP3.stat () + + Get mailbox status. The result is a tuple of 2 integers: ‘(message + count, mailbox size)’. + + -- Method: POP3.list ([which]) + + Request message list, result is in the form ‘(response, ['mesg_num + octets', ...], octets)’. If `which' is set, it is the message to + list. + + -- Method: POP3.retr (which) + + Retrieve whole message number `which', and set its seen flag. + Result is in form ‘(response, ['line', ...], octets)’. + + -- Method: POP3.dele (which) + + Flag message number `which' for deletion. On most servers + deletions are not actually performed until QUIT (the major + exception is Eudora QPOP, which deliberately violates the RFCs by + doing pending deletes on any disconnect). + + -- Method: POP3.rset () + + Remove any deletion marks for the mailbox. + + -- Method: POP3.noop () + + Do nothing. Might be used as a keep-alive. + + -- Method: POP3.quit () + + Signoff: commit changes, unlock mailbox, drop connection. + + -- Method: POP3.top (which, howmuch) + + Retrieves the message header plus `howmuch' lines of the message + after the header of message number `which'. Result is in form + ‘(response, ['line', ...], octets)’. + + The POP3 TOP command this method uses, unlike the RETR command, + doesn’t set the message’s seen flag; unfortunately, TOP is poorly + specified in the RFCs and is frequently broken in off-brand + servers. Test this method by hand against the POP3 servers you + will use before trusting it. + + -- Method: POP3.uidl (which=None) + + Return message digest (unique id) list. If `which' is specified, + result contains the unique id for that message in the form + ‘'response mesgnum uid’, otherwise result is list ‘(response, + ['mesgnum uid', ...], octets)’. + + -- Method: POP3.utf8 () + + Try to switch to UTF-8 mode. Returns the server response if + successful, raises *note error_proto: b17. if not. Specified in + RFC 6856(2). + + New in version 3.5. + + -- Method: POP3.stls (context=None) + + Start a TLS session on the active connection as specified in RFC + 2595(3). This is only allowed before user authentication + + `context' parameter is a *note ssl.SSLContext: 3e4. object which + allows bundling SSL configuration options, certificates and private + keys into a single (potentially long-lived) structure. Please read + *note Security considerations: 22a2. for best practices. + + This method supports hostname checking via *note + ssl.SSLContext.check_hostname: 23bd. and `Server Name Indication' + (see *note ssl.HAS_SNI: 23d2.). + + New in version 3.4. + +Instances of *note POP3_SSL: bc1. have no additional methods. The +interface of this subclass is identical to its parent. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2449.html + + (2) https://tools.ietf.org/html/rfc6856.html + + (3) https://tools.ietf.org/html/rfc2595.html + + +File: python.info, Node: POP3 Example, Prev: POP3 Objects, Up: poplib — POP3 protocol client + +5.21.14.2 POP3 Example +...................... + +Here is a minimal example (without error checking) that opens a mailbox +and retrieves and prints all messages: + + import getpass, poplib + + M = poplib.POP3('localhost') + M.user(getpass.getuser()) + M.pass_(getpass.getpass()) + numMessages = len(M.list()[1]) + for i in range(numMessages): + for j in M.retr(i+1)[1]: + print(j) + +At the end of the module, there is a test section that contains a more +extensive example of usage. + + +File: python.info, Node: imaplib — IMAP4 protocol client, Next: nntplib — NNTP protocol client, Prev: poplib — POP3 protocol client, Up: Internet Protocols and Support + +5.21.15 ‘imaplib’ — IMAP4 protocol client +----------------------------------------- + +`Source code:' Lib/imaplib.py(1) + +__________________________________________________________________ + +This module defines three classes, *note IMAP4: 60b, *note IMAP4_SSL: +a17. and *note IMAP4_stream: 2a51, which encapsulate a connection to an +IMAP4 server and implement a large subset of the IMAP4rev1 client +protocol as defined in RFC 2060(2). It is backward compatible with +IMAP4 ( RFC 1730(3)) servers, but note that the ‘STATUS’ command is not +supported in IMAP4. + +Three classes are provided by the *note imaplib: 98. module, *note +IMAP4: 60b. is the base class: + + -- Class: imaplib.IMAP4 (host='', port=IMAP4_PORT) + + This class implements the actual IMAP4 protocol. The connection is + created and protocol version (IMAP4 or IMAP4rev1) is determined + when the instance is initialized. If `host' is not specified, ‘''’ + (the local host) is used. If `port' is omitted, the standard IMAP4 + port (143) is used. + + The *note IMAP4: 60b. class supports the *note with: 6e9. + statement. When used like this, the IMAP4 ‘LOGOUT’ command is + issued automatically when the ‘with’ statement exits. E.g.: + + >>> from imaplib import IMAP4 + >>> with IMAP4("domain.org") as M: + ... M.noop() + ... + ('OK', [b'Nothing Accomplished. d25if65hy903weo.87']) + + Changed in version 3.5: Support for the *note with: 6e9. statement + was added. + +Three exceptions are defined as attributes of the *note IMAP4: 60b. +class: + + -- Exception: IMAP4.error + + Exception raised on any errors. The reason for the exception is + passed to the constructor as a string. + + -- Exception: IMAP4.abort + + IMAP4 server errors cause this exception to be raised. This is a + sub-class of *note IMAP4.error: 2a52. Note that closing the + instance and instantiating a new one will usually allow recovery + from this exception. + + -- Exception: IMAP4.readonly + + This exception is raised when a writable mailbox has its status + changed by the server. This is a sub-class of *note IMAP4.error: + 2a52. Some other client now has write permission, and the mailbox + will need to be re-opened to re-obtain write permission. + +There’s also a subclass for secure connections: + + -- Class: imaplib.IMAP4_SSL (host='', port=IMAP4_SSL_PORT, + keyfile=None, certfile=None, ssl_context=None) + + This is a subclass derived from *note IMAP4: 60b. that connects + over an SSL encrypted socket (to use this class you need a socket + module that was compiled with SSL support). If `host' is not + specified, ‘''’ (the local host) is used. If `port' is omitted, + the standard IMAP4-over-SSL port (993) is used. `ssl_context' is a + *note ssl.SSLContext: 3e4. object which allows bundling SSL + configuration options, certificates and private keys into a single + (potentially long-lived) structure. Please read *note Security + considerations: 22a2. for best practices. + + `keyfile' and `certfile' are a legacy alternative to `ssl_context' + - they can point to PEM-formatted private key and certificate chain + files for the SSL connection. Note that the `keyfile'/`certfile' + parameters are mutually exclusive with `ssl_context', a *note + ValueError: 1fb. is raised if `keyfile'/`certfile' is provided + along with `ssl_context'. + + Changed in version 3.3: `ssl_context' parameter added. + + Changed in version 3.4: The class now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + Deprecated since version 3.6: `keyfile' and `certfile' are + deprecated in favor of `ssl_context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + +The second subclass allows for connections created by a child process: + + -- Class: imaplib.IMAP4_stream (command) + + This is a subclass derived from *note IMAP4: 60b. that connects to + the ‘stdin/stdout’ file descriptors created by passing `command' to + ‘subprocess.Popen()’. + +The following utility functions are defined: + + -- Function: imaplib.Internaldate2tuple (datestr) + + Parse an IMAP4 ‘INTERNALDATE’ string and return corresponding local + time. The return value is a *note time.struct_time: 599. tuple or + ‘None’ if the string has wrong format. + + -- Function: imaplib.Int2AP (num) + + Converts an integer into a bytes representation using characters + from the set [‘A’ .. ‘P’]. + + -- Function: imaplib.ParseFlags (flagstr) + + Converts an IMAP4 ‘FLAGS’ response to a tuple of individual flags. + + -- Function: imaplib.Time2Internaldate (date_time) + + Convert `date_time' to an IMAP4 ‘INTERNALDATE’ representation. The + return value is a string in the form: ‘"DD-Mmm-YYYY HH:MM:SS + +HHMM"’ (including double-quotes). The `date_time' argument can be + a number (int or float) representing seconds since epoch (as + returned by *note time.time(): 31d.), a 9-tuple representing local + time an instance of *note time.struct_time: 599. (as returned by + *note time.localtime(): 155c.), an aware instance of *note + datetime.datetime: 194, or a double-quoted string. In the last + case, it is assumed to already be in the correct format. + +Note that IMAP4 message numbers change as the mailbox changes; in +particular, after an ‘EXPUNGE’ command performs deletions the remaining +messages are renumbered. So it is highly advisable to use UIDs instead, +with the UID command. + +At the end of the module, there is a test section that contains a more +extensive example of usage. + +See also +........ + +Documents describing the protocol, sources for servers implementing it, +by the University of Washington’s IMAP Information Center can all be +found at (`Source Code') ‘https://github.com/uw-imap/imap’ (`Not +Maintained'). + +* Menu: + +* IMAP4 Objects:: +* IMAP4 Example:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/imaplib.py + + (2) https://tools.ietf.org/html/rfc2060.html + + (3) https://tools.ietf.org/html/rfc1730.html + + +File: python.info, Node: IMAP4 Objects, Next: IMAP4 Example, Up: imaplib — IMAP4 protocol client + +5.21.15.1 IMAP4 Objects +....................... + +All IMAP4rev1 commands are represented by methods of the same name, +either upper-case or lower-case. + +All arguments to commands are converted to strings, except for +‘AUTHENTICATE’, and the last argument to ‘APPEND’ which is passed as an +IMAP4 literal. If necessary (the string contains IMAP4 +protocol-sensitive characters and isn’t enclosed with either parentheses +or double quotes) each string is quoted. However, the `password' +argument to the ‘LOGIN’ command is always quoted. If you want to avoid +having an argument string quoted (eg: the `flags' argument to ‘STORE’) +then enclose the string in parentheses (eg: ‘r'(\Deleted)'’). + +Each command returns a tuple: ‘(type, [data, ...])’ where `type' is +usually ‘'OK'’ or ‘'NO'’, and `data' is either the text from the command +response, or mandated results from the command. Each `data' is either a +‘bytes’, or a tuple. If a tuple, then the first part is the header of +the response, and the second part contains the data (ie: ‘literal’ +value). + +The `message_set' options to commands below is a string specifying one +or more messages to be acted upon. It may be a simple message number +(‘'1'’), a range of message numbers (‘'2:4'’), or a group of +non-contiguous ranges separated by commas (‘'1:3,6:9'’). A range can +contain an asterisk to indicate an infinite upper bound (‘'3:*'’). + +An *note IMAP4: 60b. instance has the following methods: + + -- Method: IMAP4.append (mailbox, flags, date_time, message) + + Append `message' to named mailbox. + + -- Method: IMAP4.authenticate (mechanism, authobject) + + Authenticate command — requires response processing. + + `mechanism' specifies which authentication mechanism is to be used + - it should appear in the instance variable ‘capabilities’ in the + form ‘AUTH=mechanism’. + + `authobject' must be a callable object: + + data = authobject(response) + + It will be called to process server continuation responses; the + `response' argument it is passed will be ‘bytes’. It should return + ‘bytes’ `data' that will be base64 encoded and sent to the server. + It should return ‘None’ if the client abort response ‘*’ should be + sent instead. + + Changed in version 3.5: string usernames and passwords are now + encoded to ‘utf-8’ instead of being limited to ASCII. + + -- Method: IMAP4.check () + + Checkpoint mailbox on server. + + -- Method: IMAP4.close () + + Close currently selected mailbox. Deleted messages are removed + from writable mailbox. This is the recommended command before + ‘LOGOUT’. + + -- Method: IMAP4.copy (message_set, new_mailbox) + + Copy `message_set' messages onto end of `new_mailbox'. + + -- Method: IMAP4.create (mailbox) + + Create new mailbox named `mailbox'. + + -- Method: IMAP4.delete (mailbox) + + Delete old mailbox named `mailbox'. + + -- Method: IMAP4.deleteacl (mailbox, who) + + Delete the ACLs (remove any rights) set for who on mailbox. + + -- Method: IMAP4.enable (capability) + + Enable `capability' (see RFC 5161(1)). Most capabilities do not + need to be enabled. Currently only the ‘UTF8=ACCEPT’ capability is + supported (see RFC 6855(2)). + + New in version 3.5: The *note enable(): 6ea. method itself, and RFC + 6855(3) support. + + -- Method: IMAP4.expunge () + + Permanently remove deleted items from selected mailbox. Generates + an ‘EXPUNGE’ response for each deleted message. Returned data + contains a list of ‘EXPUNGE’ message numbers in order received. + + -- Method: IMAP4.fetch (message_set, message_parts) + + Fetch (parts of) messages. `message_parts' should be a string of + message part names enclosed within parentheses, eg: ‘"(UID + BODY[TEXT])"’. Returned data are tuples of message part envelope + and data. + + -- Method: IMAP4.getacl (mailbox) + + Get the ‘ACL’s for `mailbox'. The method is non-standard, but is + supported by the ‘Cyrus’ server. + + -- Method: IMAP4.getannotation (mailbox, entry, attribute) + + Retrieve the specified ‘ANNOTATION’s for `mailbox'. The method is + non-standard, but is supported by the ‘Cyrus’ server. + + -- Method: IMAP4.getquota (root) + + Get the ‘quota’ `root'’s resource usage and limits. This method is + part of the IMAP4 QUOTA extension defined in rfc2087. + + -- Method: IMAP4.getquotaroot (mailbox) + + Get the list of ‘quota’ ‘roots’ for the named `mailbox'. This + method is part of the IMAP4 QUOTA extension defined in rfc2087. + + -- Method: IMAP4.list ([directory[, pattern]]) + + List mailbox names in `directory' matching `pattern'. `directory' + defaults to the top-level mail folder, and `pattern' defaults to + match anything. Returned data contains a list of ‘LIST’ responses. + + -- Method: IMAP4.login (user, password) + + Identify the client using a plaintext password. The `password' + will be quoted. + + -- Method: IMAP4.login_cram_md5 (user, password) + + Force use of ‘CRAM-MD5’ authentication when identifying the client + to protect the password. Will only work if the server ‘CAPABILITY’ + response includes the phrase ‘AUTH=CRAM-MD5’. + + -- Method: IMAP4.logout () + + Shutdown connection to server. Returns server ‘BYE’ response. + + Changed in version 3.8: The method no longer ignores silently + arbitrary exceptions. + + -- Method: IMAP4.lsub (directory='""', pattern='*') + + List subscribed mailbox names in directory matching pattern. + `directory' defaults to the top level directory and `pattern' + defaults to match any mailbox. Returned data are tuples of message + part envelope and data. + + -- Method: IMAP4.myrights (mailbox) + + Show my ACLs for a mailbox (i.e. the rights that I have on + mailbox). + + -- Method: IMAP4.namespace () + + Returns IMAP namespaces as defined in RFC 2342(4). + + -- Method: IMAP4.noop () + + Send ‘NOOP’ to server. + + -- Method: IMAP4.open (host, port) + + Opens socket to `port' at `host'. This method is implicitly called + by the *note IMAP4: 60b. constructor. The connection objects + established by this method will be used in the *note IMAP4.read(): + 2a72, *note IMAP4.readline(): 2a73, *note IMAP4.send(): 2a74, and + *note IMAP4.shutdown(): 2a75. methods. You may override this + method. + + Raises an *note auditing event: fd1. ‘imaplib.open’ with arguments + ‘self’, ‘host’, ‘port’. + + -- Method: IMAP4.partial (message_num, message_part, start, length) + + Fetch truncated part of a message. Returned data is a tuple of + message part envelope and data. + + -- Method: IMAP4.proxyauth (user) + + Assume authentication as `user'. Allows an authorised + administrator to proxy into any user’s mailbox. + + -- Method: IMAP4.read (size) + + Reads `size' bytes from the remote server. You may override this + method. + + -- Method: IMAP4.readline () + + Reads one line from the remote server. You may override this + method. + + -- Method: IMAP4.recent () + + Prompt server for an update. Returned data is ‘None’ if no new + messages, else value of ‘RECENT’ response. + + -- Method: IMAP4.rename (oldmailbox, newmailbox) + + Rename mailbox named `oldmailbox' to `newmailbox'. + + -- Method: IMAP4.response (code) + + Return data for response `code' if received, or ‘None’. Returns + the given code, instead of the usual type. + + -- Method: IMAP4.search (charset, criterion[, ...]) + + Search mailbox for matching messages. `charset' may be ‘None’, in + which case no ‘CHARSET’ will be specified in the request to the + server. The IMAP protocol requires that at least one criterion be + specified; an exception will be raised when the server returns an + error. `charset' must be ‘None’ if the ‘UTF8=ACCEPT’ capability + was enabled using the *note enable(): 6ea. command. + + Example: + + # M is a connected IMAP4 instance... + typ, msgnums = M.search(None, 'FROM', '"LDJ"') + + # or: + typ, msgnums = M.search(None, '(FROM "LDJ")') + + -- Method: IMAP4.select (mailbox='INBOX', readonly=False) + + Select a mailbox. Returned data is the count of messages in + `mailbox' (‘EXISTS’ response). The default `mailbox' is ‘'INBOX'’. + If the `readonly' flag is set, modifications to the mailbox are not + allowed. + + -- Method: IMAP4.send (data) + + Sends ‘data’ to the remote server. You may override this method. + + Raises an *note auditing event: fd1. ‘imaplib.send’ with arguments + ‘self’, ‘data’. + + -- Method: IMAP4.setacl (mailbox, who, what) + + Set an ‘ACL’ for `mailbox'. The method is non-standard, but is + supported by the ‘Cyrus’ server. + + -- Method: IMAP4.setannotation (mailbox, entry, attribute[, ...]) + + Set ‘ANNOTATION’s for `mailbox'. The method is non-standard, but + is supported by the ‘Cyrus’ server. + + -- Method: IMAP4.setquota (root, limits) + + Set the ‘quota’ `root'’s resource `limits'. This method is part of + the IMAP4 QUOTA extension defined in rfc2087. + + -- Method: IMAP4.shutdown () + + Close connection established in ‘open’. This method is implicitly + called by *note IMAP4.logout(): 2a6c. You may override this + method. + + -- Method: IMAP4.socket () + + Returns socket instance used to connect to server. + + -- Method: IMAP4.sort (sort_criteria, charset, search_criterion[, ...]) + + The ‘sort’ command is a variant of ‘search’ with sorting semantics + for the results. Returned data contains a space separated list of + matching message numbers. + + Sort has two arguments before the `search_criterion' argument(s); a + parenthesized list of `sort_criteria', and the searching `charset'. + Note that unlike ‘search’, the searching `charset' argument is + mandatory. There is also a ‘uid sort’ command which corresponds to + ‘sort’ the way that ‘uid search’ corresponds to ‘search’. The + ‘sort’ command first searches the mailbox for messages that match + the given searching criteria using the charset argument for the + interpretation of strings in the searching criteria. It then + returns the numbers of matching messages. + + This is an ‘IMAP4rev1’ extension command. + + -- Method: IMAP4.starttls (ssl_context=None) + + Send a ‘STARTTLS’ command. The `ssl_context' argument is optional + and should be a *note ssl.SSLContext: 3e4. object. This will + enable encryption on the IMAP connection. Please read *note + Security considerations: 22a2. for best practices. + + New in version 3.2. + + Changed in version 3.4: The method now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + -- Method: IMAP4.status (mailbox, names) + + Request named status conditions for `mailbox'. + + -- Method: IMAP4.store (message_set, command, flag_list) + + Alters flag dispositions for messages in mailbox. `command' is + specified by section 6.4.6 of RFC 2060(5) as being one of “FLAGS”, + “+FLAGS”, or “-FLAGS”, optionally with a suffix of “.SILENT”. + + For example, to set the delete flag on all messages: + + typ, data = M.search(None, 'ALL') + for num in data[0].split(): + M.store(num, '+FLAGS', '\\Deleted') + M.expunge() + + Note: Creating flags containing ‘]’ (for example: “[test]”) + violates RFC 3501(6) (the IMAP protocol). However, imaplib + has historically allowed creation of such tags, and popular + IMAP servers, such as Gmail, accept and produce such flags. + There are non-Python programs which also create such tags. + Although it is an RFC violation and IMAP clients and servers + are supposed to be strict, imaplib nonetheless continues to + allow such tags to be created for backward compatibility + reasons, and as of Python 3.6, handles them if they are sent + from the server, since this improves real-world compatibility. + + -- Method: IMAP4.subscribe (mailbox) + + Subscribe to new mailbox. + + -- Method: IMAP4.thread (threading_algorithm, charset, + search_criterion[, ...]) + + The ‘thread’ command is a variant of ‘search’ with threading + semantics for the results. Returned data contains a space + separated list of thread members. + + Thread members consist of zero or more messages numbers, delimited + by spaces, indicating successive parent and child. + + Thread has two arguments before the `search_criterion' argument(s); + a `threading_algorithm', and the searching `charset'. Note that + unlike ‘search’, the searching `charset' argument is mandatory. + There is also a ‘uid thread’ command which corresponds to ‘thread’ + the way that ‘uid search’ corresponds to ‘search’. The ‘thread’ + command first searches the mailbox for messages that match the + given searching criteria using the charset argument for the + interpretation of strings in the searching criteria. It then + returns the matching messages threaded according to the specified + threading algorithm. + + This is an ‘IMAP4rev1’ extension command. + + -- Method: IMAP4.uid (command, arg[, ...]) + + Execute command args with messages identified by UID, rather than + message number. Returns response appropriate to command. At least + one argument must be supplied; if none are provided, the server + will return an error and an exception will be raised. + + -- Method: IMAP4.unsubscribe (mailbox) + + Unsubscribe from old mailbox. + + -- Method: IMAP4.xatom (name[, ...]) + + Allow simple extension commands notified by server in ‘CAPABILITY’ + response. + +The following attributes are defined on instances of *note IMAP4: 60b.: + + -- Attribute: IMAP4.PROTOCOL_VERSION + + The most recent supported protocol in the ‘CAPABILITY’ response + from the server. + + -- Attribute: IMAP4.debug + + Integer value to control debugging output. The initialize value is + taken from the module variable ‘Debug’. Values greater than three + trace each command. + + -- Attribute: IMAP4.utf8_enabled + + Boolean value that is normally ‘False’, but is set to ‘True’ if an + *note enable(): 6ea. command is successfully issued for the + ‘UTF8=ACCEPT’ capability. + + New in version 3.5. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc5161.html + + (2) https://tools.ietf.org/html/rfc6855.html + + (3) https://tools.ietf.org/html/rfc6855.html + + (4) https://tools.ietf.org/html/rfc2342.html + + (5) https://tools.ietf.org/html/rfc2060.html + + (6) https://tools.ietf.org/html/rfc3501.html + + +File: python.info, Node: IMAP4 Example, Prev: IMAP4 Objects, Up: imaplib — IMAP4 protocol client + +5.21.15.2 IMAP4 Example +....................... + +Here is a minimal example (without error checking) that opens a mailbox +and retrieves and prints all messages: + + import getpass, imaplib + + M = imaplib.IMAP4() + M.login(getpass.getuser(), getpass.getpass()) + M.select() + typ, data = M.search(None, 'ALL') + for num in data[0].split(): + typ, data = M.fetch(num, '(RFC822)') + print('Message %s\n%s\n' % (num, data[0][1])) + M.close() + M.logout() + + +File: python.info, Node: nntplib — NNTP protocol client, Next: smtplib — SMTP protocol client, Prev: imaplib — IMAP4 protocol client, Up: Internet Protocols and Support + +5.21.16 ‘nntplib’ — NNTP protocol client +---------------------------------------- + +`Source code:' Lib/nntplib.py(1) + +__________________________________________________________________ + +This module defines the class *note NNTP: a2c. which implements the +client side of the Network News Transfer Protocol. It can be used to +implement a news reader or poster, or automated news processors. It is +compatible with RFC 3977(2) as well as the older RFC 977(3) and RFC +2980(4). + +Here are two small examples of how it can be used. To list some +statistics about a newsgroup and print the subjects of the last 10 +articles: + + >>> s = nntplib.NNTP('news.gmane.io') + >>> resp, count, first, last, name = s.group('gmane.comp.python.committers') + >>> print('Group', name, 'has', count, 'articles, range', first, 'to', last) + Group gmane.comp.python.committers has 1096 articles, range 1 to 1096 + >>> resp, overviews = s.over((last - 9, last)) + >>> for id, over in overviews: + ... print(id, nntplib.decode_header(over['subject'])) + ... + 1087 Re: Commit privileges for Łukasz Langa + 1088 Re: 3.2 alpha 2 freeze + 1089 Re: 3.2 alpha 2 freeze + 1090 Re: Commit privileges for Łukasz Langa + 1091 Re: Commit privileges for Łukasz Langa + 1092 Updated ssh key + 1093 Re: Updated ssh key + 1094 Re: Updated ssh key + 1095 Hello fellow committers! + 1096 Re: Hello fellow committers! + >>> s.quit() + '205 Bye!' + +To post an article from a binary file (this assumes that the article has +valid headers, and that you have right to post on the particular +newsgroup): + + >>> s = nntplib.NNTP('news.gmane.io') + >>> f = open('article.txt', 'rb') + >>> s.post(f) + '240 Article posted successfully.' + >>> s.quit() + '205 Bye!' + +The module itself defines the following classes: + + -- Class: nntplib.NNTP (host, port=119, user=None, password=None, + readermode=None, usenetrc=False[, timeout]) + + Return a new *note NNTP: a2c. object, representing a connection to + the NNTP server running on host `host', listening at port `port'. + An optional `timeout' can be specified for the socket connection. + If the optional `user' and `password' are provided, or if suitable + credentials are present in ‘/.netrc’ and the optional flag + `usenetrc' is true, the ‘AUTHINFO USER’ and ‘AUTHINFO PASS’ + commands are used to identify and authenticate the user to the + server. If the optional flag `readermode' is true, then a ‘mode + reader’ command is sent before authentication is performed. Reader + mode is sometimes necessary if you are connecting to an NNTP server + on the local machine and intend to call reader-specific commands, + such as ‘group’. If you get unexpected *note NNTPPermanentError: + 2a8f.s, you might need to set `readermode'. The *note NNTP: a2c. + class supports the *note with: 6e9. statement to unconditionally + consume *note OSError: 1d3. exceptions and to close the NNTP + connection when done, e.g.: + + >>> from nntplib import NNTP + >>> with NNTP('news.gmane.io') as n: + ... n.group('gmane.comp.python.committers') + ... + ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') + >>> + + Raises an *note auditing event: fd1. ‘nntplib.connect’ with + arguments ‘self’, ‘host’, ‘port’. + + All commands will raise an *note auditing event: fd1. + ‘nntplib.putline’ with arguments ‘self’ and ‘line’, where ‘line’ is + the bytes about to be sent to the remote host. + + Changed in version 3.2: `usenetrc' is now ‘False’ by default. + + Changed in version 3.3: Support for the *note with: 6e9. statement + was added. + + -- Class: nntplib.NNTP_SSL (host, port=563, user=None, password=None, + ssl_context=None, readermode=None, usenetrc=False[, timeout]) + + Return a new *note NNTP_SSL: ba5. object, representing an encrypted + connection to the NNTP server running on host `host', listening at + port `port'. *note NNTP_SSL: ba5. objects have the same methods as + *note NNTP: a2c. objects. If `port' is omitted, port 563 (NNTPS) + is used. `ssl_context' is also optional, and is a *note + SSLContext: 3e4. object. Please read *note Security + considerations: 22a2. for best practices. All other parameters + behave the same as for *note NNTP: a2c. + + Note that SSL-on-563 is discouraged per RFC 4642(5), in favor of + STARTTLS as described below. However, some servers only support + the former. + + Raises an *note auditing event: fd1. ‘nntplib.connect’ with + arguments ‘self’, ‘host’, ‘port’. + + All commands will raise an *note auditing event: fd1. + ‘nntplib.putline’ with arguments ‘self’ and ‘line’, where ‘line’ is + the bytes about to be sent to the remote host. + + New in version 3.2. + + Changed in version 3.4: The class now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + -- Exception: nntplib.NNTPError + + Derived from the standard exception *note Exception: 1a9, this is + the base class for all exceptions raised by the *note nntplib: c0. + module. Instances of this class have the following attribute: + + -- Attribute: response + + The response of the server if available, as a *note str: 330. + object. + + -- Exception: nntplib.NNTPReplyError + + Exception raised when an unexpected reply is received from the + server. + + -- Exception: nntplib.NNTPTemporaryError + + Exception raised when a response code in the range 400–499 is + received. + + -- Exception: nntplib.NNTPPermanentError + + Exception raised when a response code in the range 500–599 is + received. + + -- Exception: nntplib.NNTPProtocolError + + Exception raised when a reply is received from the server that does + not begin with a digit in the range 1–5. + + -- Exception: nntplib.NNTPDataError + + Exception raised when there is some error in the response data. + +* Menu: + +* NNTP Objects:: +* Utility functions: Utility functions<2>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/nntplib.py + + (2) https://tools.ietf.org/html/rfc3977.html + + (3) https://tools.ietf.org/html/rfc977.html + + (4) https://tools.ietf.org/html/rfc2980.html + + (5) https://tools.ietf.org/html/rfc4642.html + + +File: python.info, Node: NNTP Objects, Next: Utility functions<2>, Up: nntplib — NNTP protocol client + +5.21.16.1 NNTP Objects +...................... + +When connected, *note NNTP: a2c. and *note NNTP_SSL: ba5. objects +support the following methods and attributes. + +* Menu: + +* Attributes:: +* Methods: Methods<3>. + + +File: python.info, Node: Attributes, Next: Methods<3>, Up: NNTP Objects + +5.21.16.2 Attributes +.................... + + -- Attribute: NNTP.nntp_version + + An integer representing the version of the NNTP protocol supported + by the server. In practice, this should be ‘2’ for servers + advertising RFC 3977(1) compliance and ‘1’ for others. + + New in version 3.2. + + -- Attribute: NNTP.nntp_implementation + + A string describing the software name and version of the NNTP + server, or *note None: 157. if not advertised by the server. + + New in version 3.2. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc3977.html + + +File: python.info, Node: Methods<3>, Prev: Attributes, Up: NNTP Objects + +5.21.16.3 Methods +................. + +The `response' that is returned as the first item in the return tuple of +almost all methods is the server’s response: a string beginning with a +three-digit code. If the server’s response indicates an error, the +method raises one of the above exceptions. + +Many of the following methods take an optional keyword-only argument +`file'. When the `file' argument is supplied, it must be either a *note +file object: b42. opened for binary writing, or the name of an on-disk +file to be written to. The method will then write any data returned by +the server (except for the response line and the terminating dot) to the +file; any list of lines, tuples or objects that the method normally +returns will be empty. + +Changed in version 3.2: Many of the following methods have been reworked +and fixed, which makes them incompatible with their 3.1 counterparts. + + -- Method: NNTP.quit () + + Send a ‘QUIT’ command and close the connection. Once this method + has been called, no other methods of the NNTP object should be + called. + + -- Method: NNTP.getwelcome () + + Return the welcome message sent by the server in reply to the + initial connection. (This message sometimes contains disclaimers + or help information that may be relevant to the user.) + + -- Method: NNTP.getcapabilities () + + Return the RFC 3977(1) capabilities advertised by the server, as a + *note dict: 1b8. instance mapping capability names to (possibly + empty) lists of values. On legacy servers which don’t understand + the ‘CAPABILITIES’ command, an empty dictionary is returned + instead. + + >>> s = NNTP('news.gmane.io') + >>> 'POST' in s.getcapabilities() + True + + New in version 3.2. + + -- Method: NNTP.login (user=None, password=None, usenetrc=True) + + Send ‘AUTHINFO’ commands with the user name and password. If + `user' and `password' are ‘None’ and `usenetrc' is true, + credentials from ‘~/.netrc’ will be used if possible. + + Unless intentionally delayed, login is normally performed during + the *note NNTP: a2c. object initialization and separately calling + this function is unnecessary. To force authentication to be + delayed, you must not set `user' or `password' when creating the + object, and must set `usenetrc' to False. + + New in version 3.2. + + -- Method: NNTP.starttls (context=None) + + Send a ‘STARTTLS’ command. This will enable encryption on the NNTP + connection. The `context' argument is optional and should be a + *note ssl.SSLContext: 3e4. object. Please read *note Security + considerations: 22a2. for best practices. + + Note that this may not be done after authentication information has + been transmitted, and authentication occurs by default if possible + during a *note NNTP: a2c. object initialization. See *note + NNTP.login(): 2a9f. for information on suppressing this behavior. + + New in version 3.2. + + Changed in version 3.4: The method now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + -- Method: NNTP.newgroups (date, *, file=None) + + Send a ‘NEWGROUPS’ command. The `date' argument should be a *note + datetime.date: 193. or *note datetime.datetime: 194. object. + Return a pair ‘(response, groups)’ where `groups' is a list + representing the groups that are new since the given `date'. If + `file' is supplied, though, then `groups' will be empty. + + >>> from datetime import date, timedelta + >>> resp, groups = s.newgroups(date.today() - timedelta(days=3)) + >>> len(groups) + 85 + >>> groups[0] + GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m') + + -- Method: NNTP.newnews (group, date, *, file=None) + + Send a ‘NEWNEWS’ command. Here, `group' is a group name or ‘'*'’, + and `date' has the same meaning as for *note newgroups(): 2aa0. + Return a pair ‘(response, articles)’ where `articles' is a list of + message ids. + + This command is frequently disabled by NNTP server administrators. + + -- Method: NNTP.list (group_pattern=None, *, file=None) + + Send a ‘LIST’ or ‘LIST ACTIVE’ command. Return a pair ‘(response, + list)’ where `list' is a list of tuples representing all the groups + available from this NNTP server, optionally matching the pattern + string `group_pattern'. Each tuple has the form ‘(group, last, + first, flag)’, where `group' is a group name, `last' and `first' + are the last and first article numbers, and `flag' usually takes + one of these values: + + * ‘y’: Local postings and articles from peers are allowed. + + * ‘m’: The group is moderated and all postings must be approved. + + * ‘n’: No local postings are allowed, only articles from peers. + + * ‘j’: Articles from peers are filed in the junk group instead. + + * ‘x’: No local postings, and articles from peers are ignored. + + * ‘=foo.bar’: Articles are filed in the ‘foo.bar’ group instead. + + If `flag' has another value, then the status of the newsgroup + should be considered unknown. + + This command can return very large results, especially if + `group_pattern' is not specified. It is best to cache the results + offline unless you really need to refresh them. + + Changed in version 3.2: `group_pattern' was added. + + -- Method: NNTP.descriptions (grouppattern) + + Send a ‘LIST NEWSGROUPS’ command, where `grouppattern' is a wildmat + string as specified in RFC 3977(2) (it’s essentially the same as + DOS or UNIX shell wildcard strings). Return a pair ‘(response, + descriptions)’, where `descriptions' is a dictionary mapping group + names to textual descriptions. + + >>> resp, descs = s.descriptions('gmane.comp.python.*') + >>> len(descs) + 295 + >>> descs.popitem() + ('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)') + + -- Method: NNTP.description (group) + + Get a description for a single group `group'. If more than one + group matches (if ‘group’ is a real wildmat string), return the + first match. If no group matches, return an empty string. + + This elides the response code from the server. If the response + code is needed, use *note descriptions(): 2aa3. + + -- Method: NNTP.group (name) + + Send a ‘GROUP’ command, where `name' is the group name. The group + is selected as the current group, if it exists. Return a tuple + ‘(response, count, first, last, name)’ where `count' is the + (estimated) number of articles in the group, `first' is the first + article number in the group, `last' is the last article number in + the group, and `name' is the group name. + + -- Method: NNTP.over (message_spec, *, file=None) + + Send an ‘OVER’ command, or an ‘XOVER’ command on legacy servers. + `message_spec' can be either a string representing a message id, or + a ‘(first, last)’ tuple of numbers indicating a range of articles + in the current group, or a ‘(first, None)’ tuple indicating a range + of articles starting from `first' to the last article in the + current group, or *note None: 157. to select the current article in + the current group. + + Return a pair ‘(response, overviews)’. `overviews' is a list of + ‘(article_number, overview)’ tuples, one for each article selected + by `message_spec'. Each `overview' is a dictionary with the same + number of items, but this number depends on the server. These + items are either message headers (the key is then the lower-cased + header name) or metadata items (the key is then the metadata name + prepended with ‘":"’). The following items are guaranteed to be + present by the NNTP specification: + + * the ‘subject’, ‘from’, ‘date’, ‘message-id’ and ‘references’ + headers + + * the ‘:bytes’ metadata: the number of bytes in the entire raw + article (including headers and body) + + * the ‘:lines’ metadata: the number of lines in the article body + + The value of each item is either a string, or *note None: 157. if + not present. + + It is advisable to use the *note decode_header(): 2aa7. function on + header values when they may contain non-ASCII characters: + + >>> _, _, first, last, _ = s.group('gmane.comp.python.devel') + >>> resp, overviews = s.over((last, last)) + >>> art_num, over = overviews[0] + >>> art_num + 117216 + >>> list(over.keys()) + ['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject'] + >>> over['from'] + '=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= <martin@v.loewis.de>' + >>> nntplib.decode_header(over['from']) + '"Martin v. Löwis" <martin@v.loewis.de>' + + New in version 3.2. + + -- Method: NNTP.help (*, file=None) + + Send a ‘HELP’ command. Return a pair ‘(response, list)’ where + `list' is a list of help strings. + + -- Method: NNTP.stat (message_spec=None) + + Send a ‘STAT’ command, where `message_spec' is either a message id + (enclosed in ‘'<'’ and ‘'>'’) or an article number in the current + group. If `message_spec' is omitted or *note None: 157, the + current article in the current group is considered. Return a + triple ‘(response, number, id)’ where `number' is the article + number and `id' is the message id. + + >>> _, _, first, last, _ = s.group('gmane.comp.python.devel') + >>> resp, number, message_id = s.stat(first) + >>> number, message_id + (9099, '<20030112190404.GE29873@epoch.metaslash.com>') + + -- Method: NNTP.next () + + Send a ‘NEXT’ command. Return as for *note stat(): 2aa9. + + -- Method: NNTP.last () + + Send a ‘LAST’ command. Return as for *note stat(): 2aa9. + + -- Method: NNTP.article (message_spec=None, *, file=None) + + Send an ‘ARTICLE’ command, where `message_spec' has the same + meaning as for *note stat(): 2aa9. Return a tuple ‘(response, + info)’ where `info' is a *note namedtuple: 1b7. with three + attributes `number', `message_id' and `lines' (in that order). + `number' is the article number in the group (or 0 if the + information is not available), `message_id' the message id as a + string, and `lines' a list of lines (without terminating newlines) + comprising the raw message including headers and body. + + >>> resp, info = s.article('<20030112190404.GE29873@epoch.metaslash.com>') + >>> info.number + 0 + >>> info.message_id + '<20030112190404.GE29873@epoch.metaslash.com>' + >>> len(info.lines) + 65 + >>> info.lines[0] + b'Path: main.gmane.org!not-for-mail' + >>> info.lines[1] + b'From: Neal Norwitz <neal@metaslash.com>' + >>> info.lines[-3:] + [b'There is a patch for 2.3 as well as 2.2.', b'', b'Neal'] + + -- Method: NNTP.head (message_spec=None, *, file=None) + + Same as *note article(): 2aac, but sends a ‘HEAD’ command. The + `lines' returned (or written to `file') will only contain the + message headers, not the body. + + -- Method: NNTP.body (message_spec=None, *, file=None) + + Same as *note article(): 2aac, but sends a ‘BODY’ command. The + `lines' returned (or written to `file') will only contain the + message body, not the headers. + + -- Method: NNTP.post (data) + + Post an article using the ‘POST’ command. The `data' argument is + either a *note file object: b42. opened for binary reading, or any + iterable of bytes objects (representing raw lines of the article to + be posted). It should represent a well-formed news article, + including the required headers. The *note post(): 2aaf. method + automatically escapes lines beginning with ‘.’ and appends the + termination line. + + If the method succeeds, the server’s response is returned. If the + server refuses posting, a *note NNTPReplyError: 2a92. is raised. + + -- Method: NNTP.ihave (message_id, data) + + Send an ‘IHAVE’ command. `message_id' is the id of the message to + send to the server (enclosed in ‘'<'’ and ‘'>'’). The `data' + parameter and the return value are the same as for *note post(): + 2aaf. + + -- Method: NNTP.date () + + Return a pair ‘(response, date)’. `date' is a *note datetime: 194. + object containing the current date and time of the server. + + -- Method: NNTP.slave () + + Send a ‘SLAVE’ command. Return the server’s `response'. + + -- Method: NNTP.set_debuglevel (level) + + Set the instance’s debugging level. This controls the amount of + debugging output printed. The default, ‘0’, produces no debugging + output. A value of ‘1’ produces a moderate amount of debugging + output, generally a single line per request or response. A value + of ‘2’ or higher produces the maximum amount of debugging output, + logging each line sent and received on the connection (including + message text). + +The following are optional NNTP extensions defined in RFC 2980(3). Some +of them have been superseded by newer commands in RFC 3977(4). + + -- Method: NNTP.xhdr (hdr, str, *, file=None) + + Send an ‘XHDR’ command. The `hdr' argument is a header keyword, + e.g. ‘'subject'’. The `str' argument should have the form + ‘'first-last'’ where `first' and `last' are the first and last + article numbers to search. Return a pair ‘(response, list)’, where + `list' is a list of pairs ‘(id, text)’, where `id' is an article + number (as a string) and `text' is the text of the requested header + for that article. If the `file' parameter is supplied, then the + output of the ‘XHDR’ command is stored in a file. If `file' is a + string, then the method will open a file with that name, write to + it then close it. If `file' is a *note file object: b42, then it + will start calling ‘write()’ on it to store the lines of the + command output. If `file' is supplied, then the returned `list' is + an empty list. + + -- Method: NNTP.xover (start, end, *, file=None) + + Send an ‘XOVER’ command. `start' and `end' are article numbers + delimiting the range of articles to select. The return value is + the same of for *note over(): 2aa6. It is recommended to use *note + over(): 2aa6. instead, since it will automatically use the newer + ‘OVER’ command if available. + + -- Method: NNTP.xpath (id) + + Return a pair ‘(resp, path)’, where `path' is the directory path to + the article with message ID `id'. Most of the time, this extension + is not enabled by NNTP server administrators. + + Deprecated since version 3.3: The XPATH extension is not actively + used. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc3977.html + + (2) https://tools.ietf.org/html/rfc3977.html + + (3) https://tools.ietf.org/html/rfc2980.html + + (4) https://tools.ietf.org/html/rfc3977.html + + +File: python.info, Node: Utility functions<2>, Prev: NNTP Objects, Up: nntplib — NNTP protocol client + +5.21.16.4 Utility functions +........................... + +The module also defines the following utility function: + + -- Function: nntplib.decode_header (header_str) + + Decode a header value, un-escaping any escaped non-ASCII + characters. `header_str' must be a *note str: 330. object. The + unescaped value is returned. Using this function is recommended to + display some headers in a human readable form: + + >>> decode_header("Some subject") + 'Some subject' + >>> decode_header("=?ISO-8859-15?Q?D=E9buter_en_Python?=") + 'Débuter en Python' + >>> decode_header("Re: =?UTF-8?B?cHJvYmzDqG1lIGRlIG1hdHJpY2U=?=") + 'Re: problème de matrice' + + +File: python.info, Node: smtplib — SMTP protocol client, Next: smtpd — SMTP Server, Prev: nntplib — NNTP protocol client, Up: Internet Protocols and Support + +5.21.17 ‘smtplib’ — SMTP protocol client +---------------------------------------- + +`Source code:' Lib/smtplib.py(1) + +__________________________________________________________________ + +The *note smtplib: ed. module defines an SMTP client session object that +can be used to send mail to any Internet machine with an SMTP or ESMTP +listener daemon. For details of SMTP and ESMTP operation, consult RFC +821(2) (Simple Mail Transfer Protocol) and RFC 1869(3) (SMTP Service +Extensions). + + -- Class: smtplib.SMTP (host='', port=0, local_hostname=None[, + timeout], source_address=None) + + An *note SMTP: a81. instance encapsulates an SMTP connection. It + has methods that support a full repertoire of SMTP and ESMTP + operations. If the optional host and port parameters are given, + the SMTP *note connect(): 2aba. method is called with those + parameters during initialization. If specified, `local_hostname' + is used as the FQDN of the local host in the HELO/EHLO command. + Otherwise, the local hostname is found using *note + socket.getfqdn(): 2381. If the *note connect(): 2aba. call returns + anything other than a success code, an *note SMTPConnectError: + 2abb. is raised. The optional `timeout' parameter specifies a + timeout in seconds for blocking operations like the connection + attempt (if not specified, the global default timeout setting will + be used). If the timeout expires, *note socket.timeout: c0e. is + raised. The optional source_address parameter allows binding to + some specific source address in a machine with multiple network + interfaces, and/or to some specific source TCP port. It takes a + 2-tuple (host, port), for the socket to bind to as its source + address before connecting. If omitted (or if host or port are ‘''’ + and/or 0 respectively) the OS default behavior will be used. + + For normal use, you should only require the initialization/connect, + *note sendmail(): 744, and *note SMTP.quit(): 2abc. methods. An + example is included below. + + The *note SMTP: a81. class supports the *note with: 6e9. statement. + When used like this, the SMTP ‘QUIT’ command is issued + automatically when the ‘with’ statement exits. E.g.: + + >>> from smtplib import SMTP + >>> with SMTP("domain.org") as smtp: + ... smtp.noop() + ... + (250, b'Ok') + >>> + + All commands will raise an *note auditing event: fd1. + ‘smtplib.SMTP.send’ with arguments ‘self’ and ‘data’, where ‘data’ + is the bytes about to be sent to the remote host. + + Changed in version 3.3: Support for the *note with: 6e9. statement + was added. + + Changed in version 3.3: source_address argument was added. + + New in version 3.5: The SMTPUTF8 extension ( RFC 6531(4)) is now + supported. + + -- Class: smtplib.SMTP_SSL (host='', port=0, local_hostname=None, + keyfile=None, certfile=None[, timeout], context=None, + source_address=None) + + An *note SMTP_SSL: a82. instance behaves exactly the same as + instances of *note SMTP: a81. *note SMTP_SSL: a82. should be used + for situations where SSL is required from the beginning of the + connection and using ‘starttls()’ is not appropriate. If `host' is + not specified, the local host is used. If `port' is zero, the + standard SMTP-over-SSL port (465) is used. The optional arguments + `local_hostname', `timeout' and `source_address' have the same + meaning as they do in the *note SMTP: a81. class. `context', also + optional, can contain a *note SSLContext: 3e4. and allows + configuring various aspects of the secure connection. Please read + *note Security considerations: 22a2. for best practices. + + `keyfile' and `certfile' are a legacy alternative to `context', and + can point to a PEM formatted private key and certificate chain file + for the SSL connection. + + Changed in version 3.3: `context' was added. + + Changed in version 3.3: source_address argument was added. + + Changed in version 3.4: The class now supports hostname check with + *note ssl.SSLContext.check_hostname: 23bd. and `Server Name + Indication' (see *note ssl.HAS_SNI: 23d2.). + + Deprecated since version 3.6: `keyfile' and `certfile' are + deprecated in favor of `context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + + -- Class: smtplib.LMTP (host='', port=LMTP_PORT, local_hostname=None, + source_address=None) + + The LMTP protocol, which is very similar to ESMTP, is heavily based + on the standard SMTP client. It’s common to use Unix sockets for + LMTP, so our ‘connect()’ method must support that as well as a + regular host:port server. The optional arguments local_hostname + and source_address have the same meaning as they do in the *note + SMTP: a81. class. To specify a Unix socket, you must use an + absolute path for `host', starting with a ‘/’. + + Authentication is supported, using the regular SMTP mechanism. + When using a Unix socket, LMTP generally don’t support or require + any authentication, but your mileage might vary. + +A nice selection of exceptions is defined as well: + + -- Exception: smtplib.SMTPException + + Subclass of *note OSError: 1d3. that is the base exception class + for all the other exceptions provided by this module. + + Changed in version 3.4: SMTPException became subclass of *note + OSError: 1d3. + + -- Exception: smtplib.SMTPServerDisconnected + + This exception is raised when the server unexpectedly disconnects, + or when an attempt is made to use the *note SMTP: a81. instance + before connecting it to a server. + + -- Exception: smtplib.SMTPResponseException + + Base class for all exceptions that include an SMTP error code. + These exceptions are generated in some instances when the SMTP + server returns an error code. The error code is stored in the + ‘smtp_code’ attribute of the error, and the ‘smtp_error’ attribute + is set to the error message. + + -- Exception: smtplib.SMTPSenderRefused + + Sender address refused. In addition to the attributes set by on + all *note SMTPResponseException: 2abe. exceptions, this sets + ‘sender’ to the string that the SMTP server refused. + + -- Exception: smtplib.SMTPRecipientsRefused + + All recipient addresses refused. The errors for each recipient are + accessible through the attribute ‘recipients’, which is a + dictionary of exactly the same sort as *note SMTP.sendmail(): 744. + returns. + + -- Exception: smtplib.SMTPDataError + + The SMTP server refused to accept the message data. + + -- Exception: smtplib.SMTPConnectError + + Error occurred during establishment of a connection with the + server. + + -- Exception: smtplib.SMTPHeloError + + The server refused our ‘HELO’ message. + + -- Exception: smtplib.SMTPNotSupportedError + + The command or option attempted is not supported by the server. + + New in version 3.5. + + -- Exception: smtplib.SMTPAuthenticationError + + SMTP authentication went wrong. Most probably the server didn’t + accept the username/password combination provided. + +See also +........ + +RFC 821(5) - Simple Mail Transfer Protocol + + Protocol definition for SMTP. This document covers the model, + operating procedure, and protocol details for SMTP. + +RFC 1869(6) - SMTP Service Extensions + + Definition of the ESMTP extensions for SMTP. This describes a + framework for extending SMTP with new commands, supporting dynamic + discovery of the commands provided by the server, and defines a few + additional commands. + +* Menu: + +* SMTP Objects:: +* SMTP Example:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/smtplib.py + + (2) https://tools.ietf.org/html/rfc821.html + + (3) https://tools.ietf.org/html/rfc1869.html + + (4) https://tools.ietf.org/html/rfc6531.html + + (5) https://tools.ietf.org/html/rfc821.html + + (6) https://tools.ietf.org/html/rfc1869.html + + +File: python.info, Node: SMTP Objects, Next: SMTP Example, Up: smtplib — SMTP protocol client + +5.21.17.1 SMTP Objects +...................... + +An *note SMTP: a81. instance has the following methods: + + -- Method: SMTP.set_debuglevel (level) + + Set the debug output level. A value of 1 or ‘True’ for `level' + results in debug messages for connection and for all messages sent + to and received from the server. A value of 2 for `level' results + in these messages being timestamped. + + Changed in version 3.5: Added debuglevel 2. + + -- Method: SMTP.docmd (cmd, args='') + + Send a command `cmd' to the server. The optional argument `args' + is simply concatenated to the command, separated by a space. + + This returns a 2-tuple composed of a numeric response code and the + actual response line (multiline responses are joined into one long + line.) + + In normal operation it should not be necessary to call this method + explicitly. It is used to implement other methods and may be + useful for testing private extensions. + + If the connection to the server is lost while waiting for the + reply, *note SMTPServerDisconnected: 2abd. will be raised. + + -- Method: SMTP.connect (host='localhost', port=0) + + Connect to a host on a given port. The defaults are to connect to + the local host at the standard SMTP port (25). If the hostname + ends with a colon (‘':'’) followed by a number, that suffix will be + stripped off and the number interpreted as the port number to use. + This method is automatically invoked by the constructor if a host + is specified during instantiation. Returns a 2-tuple of the + response code and message sent by the server in its connection + response. + + Raises an *note auditing event: fd1. ‘smtplib.connect’ with + arguments ‘self’, ‘host’, ‘port’. + + -- Method: SMTP.helo (name='') + + Identify yourself to the SMTP server using ‘HELO’. The hostname + argument defaults to the fully qualified domain name of the local + host. The message returned by the server is stored as the + ‘helo_resp’ attribute of the object. + + In normal operation it should not be necessary to call this method + explicitly. It will be implicitly called by the *note sendmail(): + 744. when necessary. + + -- Method: SMTP.ehlo (name='') + + Identify yourself to an ESMTP server using ‘EHLO’. The hostname + argument defaults to the fully qualified domain name of the local + host. Examine the response for ESMTP option and store them for use + by *note has_extn(): 2aca. Also sets several informational + attributes: the message returned by the server is stored as the + ‘ehlo_resp’ attribute, ‘does_esmtp’ is set to true or false + depending on whether the server supports ESMTP, and + ‘esmtp_features’ will be a dictionary containing the names of the + SMTP service extensions this server supports, and their parameters + (if any). + + Unless you wish to use *note has_extn(): 2aca. before sending mail, + it should not be necessary to call this method explicitly. It will + be implicitly called by *note sendmail(): 744. when necessary. + + -- Method: SMTP.ehlo_or_helo_if_needed () + + This method calls *note ehlo(): 2ac9. and/or *note helo(): 2ac8. if + there has been no previous ‘EHLO’ or ‘HELO’ command this session. + It tries ESMTP ‘EHLO’ first. + + *note SMTPHeloError: 2ac2. + + The server didn’t reply properly to the ‘HELO’ greeting. + + -- Method: SMTP.has_extn (name) + + Return *note True: 499. if `name' is in the set of SMTP service + extensions returned by the server, *note False: 388. otherwise. + Case is ignored. + + -- Method: SMTP.verify (address) + + Check the validity of an address on this server using SMTP ‘VRFY’. + Returns a tuple consisting of code 250 and a full RFC 822(1) + address (including human name) if the user address is valid. + Otherwise returns an SMTP error code of 400 or greater and an error + string. + + Note: Many sites disable SMTP ‘VRFY’ in order to foil + spammers. + + -- Method: SMTP.login (user, password, *, initial_response_ok=True) + + Log in on an SMTP server that requires authentication. The + arguments are the username and the password to authenticate with. + If there has been no previous ‘EHLO’ or ‘HELO’ command this + session, this method tries ESMTP ‘EHLO’ first. This method will + return normally if the authentication was successful, or may raise + the following exceptions: + + *note SMTPHeloError: 2ac2. + + The server didn’t reply properly to the ‘HELO’ greeting. + + *note SMTPAuthenticationError: 2ac4. + + The server didn’t accept the username/password combination. + + *note SMTPNotSupportedError: 2ac3. + + The ‘AUTH’ command is not supported by the server. + + *note SMTPException: 8b7. + + No suitable authentication method was found. + + Each of the authentication methods supported by *note smtplib: ed. + are tried in turn if they are advertised as supported by the + server. See *note auth(): 742. for a list of supported + authentication methods. `initial_response_ok' is passed through to + *note auth(): 742. + + Optional keyword argument `initial_response_ok' specifies whether, + for authentication methods that support it, an “initial response” + as specified in RFC 4954(2) can be sent along with the ‘AUTH’ + command, rather than requiring a challenge/response. + + Changed in version 3.5: *note SMTPNotSupportedError: 2ac3. may be + raised, and the `initial_response_ok' parameter was added. + + -- Method: SMTP.auth (mechanism, authobject, *, + initial_response_ok=True) + + Issue an ‘SMTP’ ‘AUTH’ command for the specified authentication + `mechanism', and handle the challenge response via `authobject'. + + `mechanism' specifies which authentication mechanism is to be used + as argument to the ‘AUTH’ command; the valid values are those + listed in the ‘auth’ element of ‘esmtp_features’. + + `authobject' must be a callable object taking an optional single + argument: + + data = authobject(challenge=None) + + If optional keyword argument `initial_response_ok' is true, + ‘authobject()’ will be called first with no argument. It can + return the RFC 4954(3) “initial response” ASCII ‘str’ which will be + encoded and sent with the ‘AUTH’ command as below. If the + ‘authobject()’ does not support an initial response (e.g. because + it requires a challenge), it should return ‘None’ when called with + ‘challenge=None’. If `initial_response_ok' is false, then + ‘authobject()’ will not be called first with ‘None’. + + If the initial response check returns ‘None’, or if + `initial_response_ok' is false, ‘authobject()’ will be called to + process the server’s challenge response; the `challenge' argument + it is passed will be a ‘bytes’. It should return ASCII ‘str’ + `data' that will be base64 encoded and sent to the server. + + The ‘SMTP’ class provides ‘authobjects’ for the ‘CRAM-MD5’, + ‘PLAIN’, and ‘LOGIN’ mechanisms; they are named + ‘SMTP.auth_cram_md5’, ‘SMTP.auth_plain’, and ‘SMTP.auth_login’ + respectively. They all require that the ‘user’ and ‘password’ + properties of the ‘SMTP’ instance are set to appropriate values. + + User code does not normally need to call ‘auth’ directly, but can + instead call the *note login(): 2acd. method, which will try each + of the above mechanisms in turn, in the order listed. ‘auth’ is + exposed to facilitate the implementation of authentication methods + not (or not yet) supported directly by *note smtplib: ed. + + New in version 3.5. + + -- Method: SMTP.starttls (keyfile=None, certfile=None, context=None) + + Put the SMTP connection in TLS (Transport Layer Security) mode. + All SMTP commands that follow will be encrypted. You should then + call *note ehlo(): 2ac9. again. + + If `keyfile' and `certfile' are provided, they are used to create + an *note ssl.SSLContext: 3e4. + + Optional `context' parameter is an *note ssl.SSLContext: 3e4. + object; This is an alternative to using a keyfile and a certfile + and if specified both `keyfile' and `certfile' should be ‘None’. + + If there has been no previous ‘EHLO’ or ‘HELO’ command this + session, this method tries ESMTP ‘EHLO’ first. + + Deprecated since version 3.6: `keyfile' and `certfile' are + deprecated in favor of `context'. Please use *note + ssl.SSLContext.load_cert_chain(): a91. instead, or let *note + ssl.create_default_context(): 8c1. select the system’s trusted CA + certificates for you. + + *note SMTPHeloError: 2ac2. + + The server didn’t reply properly to the ‘HELO’ greeting. + + *note SMTPNotSupportedError: 2ac3. + + The server does not support the STARTTLS extension. + + *note RuntimeError: 2ba. + + SSL/TLS support is not available to your Python interpreter. + + Changed in version 3.3: `context' was added. + + Changed in version 3.4: The method now supports hostname check with + ‘SSLContext.check_hostname’ and `Server Name Indicator' (see *note + HAS_SNI: 23d2.). + + Changed in version 3.5: The error raised for lack of STARTTLS + support is now the *note SMTPNotSupportedError: 2ac3. subclass + instead of the base *note SMTPException: 8b7. + + -- Method: SMTP.sendmail (from_addr, to_addrs, msg, mail_options=(), + rcpt_options=()) + + Send mail. The required arguments are an RFC 822(4) from-address + string, a list of RFC 822(5) to-address strings (a bare string will + be treated as a list with 1 address), and a message string. The + caller may pass a list of ESMTP options (such as ‘8bitmime’) to be + used in ‘MAIL FROM’ commands as `mail_options'. ESMTP options + (such as ‘DSN’ commands) that should be used with all ‘RCPT’ + commands can be passed as `rcpt_options'. (If you need to use + different ESMTP options to different recipients you have to use the + low-level methods such as ‘mail()’, ‘rcpt()’ and ‘data()’ to send + the message.) + + Note: The `from_addr' and `to_addrs' parameters are used to + construct the message envelope used by the transport agents. + ‘sendmail’ does not modify the message headers in any way. + + `msg' may be a string containing characters in the ASCII range, or + a byte string. A string is encoded to bytes using the ascii codec, + and lone ‘\r’ and ‘\n’ characters are converted to ‘\r\n’ + characters. A byte string is not modified. + + If there has been no previous ‘EHLO’ or ‘HELO’ command this + session, this method tries ESMTP ‘EHLO’ first. If the server does + ESMTP, message size and each of the specified options will be + passed to it (if the option is in the feature set the server + advertises). If ‘EHLO’ fails, ‘HELO’ will be tried and ESMTP + options suppressed. + + This method will return normally if the mail is accepted for at + least one recipient. Otherwise it will raise an exception. That + is, if this method does not raise an exception, then someone should + get your mail. If this method does not raise an exception, it + returns a dictionary, with one entry for each recipient that was + refused. Each entry contains a tuple of the SMTP error code and + the accompanying error message sent by the server. + + If ‘SMTPUTF8’ is included in `mail_options', and the server + supports it, `from_addr' and `to_addrs' may contain non-ASCII + characters. + + This method may raise the following exceptions: + + *note SMTPRecipientsRefused: 2ac0. + + All recipients were refused. Nobody got the mail. The + ‘recipients’ attribute of the exception object is a dictionary + with information about the refused recipients (like the one + returned when at least one recipient was accepted). + + *note SMTPHeloError: 2ac2. + + The server didn’t reply properly to the ‘HELO’ greeting. + + *note SMTPSenderRefused: 2abf. + + The server didn’t accept the `from_addr'. + + *note SMTPDataError: 2ac1. + + The server replied with an unexpected error code (other than a + refusal of a recipient). + + *note SMTPNotSupportedError: 2ac3. + + ‘SMTPUTF8’ was given in the `mail_options' but is not + supported by the server. + + Unless otherwise noted, the connection will be open even after an + exception is raised. + + Changed in version 3.2: `msg' may be a byte string. + + Changed in version 3.5: ‘SMTPUTF8’ support added, and *note + SMTPNotSupportedError: 2ac3. may be raised if ‘SMTPUTF8’ is + specified but the server does not support it. + + -- Method: SMTP.send_message (msg, from_addr=None, to_addrs=None, + mail_options=(), rcpt_options=()) + + This is a convenience method for calling *note sendmail(): 744. + with the message represented by an *note email.message.Message: + 541. object. The arguments have the same meaning as for *note + sendmail(): 744, except that `msg' is a ‘Message’ object. + + If `from_addr' is ‘None’ or `to_addrs' is ‘None’, ‘send_message’ + fills those arguments with addresses extracted from the headers of + `msg' as specified in RFC 5322(6): `from_addr' is set to the + ‘Sender’ field if it is present, and otherwise to the ‘From’ field. + `to_addrs' combines the values (if any) of the ‘To’, ‘Cc’, and + ‘Bcc’ fields from `msg'. If exactly one set of ‘Resent-*’ headers + appear in the message, the regular headers are ignored and the + ‘Resent-*’ headers are used instead. If the message contains more + than one set of ‘Resent-*’ headers, a *note ValueError: 1fb. is + raised, since there is no way to unambiguously detect the most + recent set of ‘Resent-’ headers. + + ‘send_message’ serializes `msg' using *note BytesGenerator: b4d. + with ‘\r\n’ as the `linesep', and calls *note sendmail(): 744. to + transmit the resulting message. Regardless of the values of + `from_addr' and `to_addrs', ‘send_message’ does not transmit any + ‘Bcc’ or ‘Resent-Bcc’ headers that may appear in `msg'. If any of + the addresses in `from_addr' and `to_addrs' contain non-ASCII + characters and the server does not advertise ‘SMTPUTF8’ support, an + ‘SMTPNotSupported’ error is raised. Otherwise the ‘Message’ is + serialized with a clone of its *note policy: 75. with the *note + utf8: 6d2. attribute set to ‘True’, and ‘SMTPUTF8’ and + ‘BODY=8BITMIME’ are added to `mail_options'. + + New in version 3.2. + + New in version 3.5: Support for internationalized addresses + (‘SMTPUTF8’). + + -- Method: SMTP.quit () + + Terminate the SMTP session and close the connection. Return the + result of the SMTP ‘QUIT’ command. + +Low-level methods corresponding to the standard SMTP/ESMTP commands +‘HELP’, ‘RSET’, ‘NOOP’, ‘MAIL’, ‘RCPT’, and ‘DATA’ are also supported. +Normally these do not need to be called directly, so they are not +documented here. For details, consult the module code. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc822.html + + (2) https://tools.ietf.org/html/rfc4954.html + + (3) https://tools.ietf.org/html/rfc4954.html + + (4) https://tools.ietf.org/html/rfc822.html + + (5) https://tools.ietf.org/html/rfc822.html + + (6) https://tools.ietf.org/html/rfc5322.html + + +File: python.info, Node: SMTP Example, Prev: SMTP Objects, Up: smtplib — SMTP protocol client + +5.21.17.2 SMTP Example +...................... + +This example prompts the user for addresses needed in the message +envelope (‘To’ and ‘From’ addresses), and the message to be delivered. +Note that the headers to be included with the message must be included +in the message as entered; this example doesn’t do any processing of the +RFC 822(1) headers. In particular, the ‘To’ and ‘From’ addresses must +be included in the message headers explicitly. + + import smtplib + + def prompt(prompt): + return input(prompt).strip() + + fromaddr = prompt("From: ") + toaddrs = prompt("To: ").split() + print("Enter message, end with ^D (Unix) or ^Z (Windows):") + + # Add the From: and To: headers at the start! + msg = ("From: %s\r\nTo: %s\r\n\r\n" + % (fromaddr, ", ".join(toaddrs))) + while True: + try: + line = input() + except EOFError: + break + if not line: + break + msg = msg + line + + print("Message length is", len(msg)) + + server = smtplib.SMTP('localhost') + server.set_debuglevel(1) + server.sendmail(fromaddr, toaddrs, msg) + server.quit() + + Note: In general, you will want to use the *note email: 69. + package’s features to construct an email message, which you can + then send via *note send_message(): 745.; see *note email; + Examples: 845. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc822.html + + +File: python.info, Node: smtpd — SMTP Server, Next: telnetlib — Telnet client, Prev: smtplib — SMTP protocol client, Up: Internet Protocols and Support + +5.21.18 ‘smtpd’ — SMTP Server +----------------------------- + +`Source code:' Lib/smtpd.py(1) + +__________________________________________________________________ + +This module offers several classes to implement SMTP (email) servers. + +See also +........ + +The aiosmtpd(2) package is a recommended replacement for this module. +It is based on *note asyncio: a. and provides a more straightforward +API. *note smtpd: ec. should be considered deprecated. + +Several server implementations are present; one is a generic do-nothing +implementation, which can be overridden, while the other two offer +specific mail-sending strategies. + +Additionally the SMTPChannel may be extended to implement very specific +interaction behaviour with SMTP clients. + +The code supports RFC 5321(3), plus the RFC 1870(4) SIZE and RFC 6531(5) +SMTPUTF8 extensions. + +* Menu: + +* SMTPServer Objects:: +* DebuggingServer Objects:: +* PureProxy Objects:: +* MailmanProxy Objects:: +* SMTPChannel Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/smtpd.py + + (2) http://aiosmtpd.readthedocs.io/ + + (3) https://tools.ietf.org/html/rfc5321.html + + (4) https://tools.ietf.org/html/rfc1870.html + + (5) https://tools.ietf.org/html/rfc6531.html + + +File: python.info, Node: SMTPServer Objects, Next: DebuggingServer Objects, Up: smtpd — SMTP Server + +5.21.18.1 SMTPServer Objects +............................ + + -- Class: smtpd.SMTPServer (localaddr, remoteaddr, + data_size_limit=33554432, map=None, enable_SMTPUTF8=False, + decode_data=False) + + Create a new *note SMTPServer: 602. object, which binds to local + address `localaddr'. It will treat `remoteaddr' as an upstream + SMTP relayer. Both `localaddr' and `remoteaddr' should be a *note + (host, port): 235a. tuple. The object inherits from *note + asyncore.dispatcher: bc3, and so will insert itself into *note + asyncore: b.’s event loop on instantiation. + + `data_size_limit' specifies the maximum number of bytes that will + be accepted in a ‘DATA’ command. A value of ‘None’ or ‘0’ means no + limit. + + `map' is the socket map to use for connections (an initially empty + dictionary is a suitable value). If not specified the *note + asyncore: b. global socket map is used. + + `enable_SMTPUTF8' determines whether the ‘SMTPUTF8’ extension (as + defined in RFC 6531(1)) should be enabled. The default is ‘False’. + When ‘True’, ‘SMTPUTF8’ is accepted as a parameter to the ‘MAIL’ + command and when present is passed to *note process_message(): 603. + in the ‘kwargs['mail_options']’ list. `decode_data' and + `enable_SMTPUTF8' cannot be set to ‘True’ at the same time. + + `decode_data' specifies whether the data portion of the SMTP + transaction should be decoded using UTF-8. When `decode_data' is + ‘False’ (the default), the server advertises the ‘8BITMIME’ + extension ( RFC 6152(2)), accepts the ‘BODY=8BITMIME’ parameter to + the ‘MAIL’ command, and when present passes it to *note + process_message(): 603. in the ‘kwargs['mail_options']’ list. + `decode_data' and `enable_SMTPUTF8' cannot be set to ‘True’ at the + same time. + + -- Method: process_message (peer, mailfrom, rcpttos, data, + **kwargs) + + Raise a *note NotImplementedError: 60e. exception. Override + this in subclasses to do something useful with this message. + Whatever was passed in the constructor as `remoteaddr' will be + available as the ‘_remoteaddr’ attribute. `peer' is the + remote host’s address, `mailfrom' is the envelope originator, + `rcpttos' are the envelope recipients and `data' is a string + containing the contents of the e-mail (which should be in RFC + 5321(3) format). + + If the `decode_data' constructor keyword is set to ‘True’, the + `data' argument will be a unicode string. If it is set to + ‘False’, it will be a bytes object. + + `kwargs' is a dictionary containing additional information. + It is empty if ‘decode_data=True’ was given as an init + argument, otherwise it contains the following keys: + + `mail_options': + + a list of all received parameters to the ‘MAIL’ + command (the elements are uppercase strings; + example: ‘['BODY=8BITMIME', 'SMTPUTF8']’). + + `rcpt_options': + + same as `mail_options' but for the ‘RCPT’ command. + Currently no ‘RCPT TO’ options are supported, so for + now this will always be an empty list. + + Implementations of ‘process_message’ should use the ‘**kwargs’ + signature to accept arbitrary keyword arguments, since future + feature enhancements may add keys to the kwargs dictionary. + + Return ‘None’ to request a normal ‘250 Ok’ response; otherwise + return the desired response string in RFC 5321(4) format. + + -- Attribute: channel_class + + Override this in subclasses to use a custom *note SMTPChannel: + 601. for managing SMTP clients. + + New in version 3.4: The `map' constructor argument. + + Changed in version 3.5: `localaddr' and `remoteaddr' may now + contain IPv6 addresses. + + New in version 3.5: The `decode_data' and `enable_SMTPUTF8' + constructor parameters, and the `kwargs' parameter to *note + process_message(): 603. when `decode_data' is ‘False’. + + Changed in version 3.6: `decode_data' is now ‘False’ by default. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc6531.html + + (2) https://tools.ietf.org/html/rfc6152.html + + (3) https://tools.ietf.org/html/rfc5321.html + + (4) https://tools.ietf.org/html/rfc5321.html + + +File: python.info, Node: DebuggingServer Objects, Next: PureProxy Objects, Prev: SMTPServer Objects, Up: smtpd — SMTP Server + +5.21.18.2 DebuggingServer Objects +................................. + + -- Class: smtpd.DebuggingServer (localaddr, remoteaddr) + + Create a new debugging server. Arguments are as per *note + SMTPServer: 602. Messages will be discarded, and printed on + stdout. + + +File: python.info, Node: PureProxy Objects, Next: MailmanProxy Objects, Prev: DebuggingServer Objects, Up: smtpd — SMTP Server + +5.21.18.3 PureProxy Objects +........................... + + -- Class: smtpd.PureProxy (localaddr, remoteaddr) + + Create a new pure proxy server. Arguments are as per *note + SMTPServer: 602. Everything will be relayed to `remoteaddr'. Note + that running this has a good chance to make you into an open relay, + so please be careful. + + +File: python.info, Node: MailmanProxy Objects, Next: SMTPChannel Objects, Prev: PureProxy Objects, Up: smtpd — SMTP Server + +5.21.18.4 MailmanProxy Objects +.............................. + + -- Class: smtpd.MailmanProxy (localaddr, remoteaddr) + + Create a new pure proxy server. Arguments are as per *note + SMTPServer: 602. Everything will be relayed to `remoteaddr', + unless local mailman configurations knows about an address, in + which case it will be handled via mailman. Note that running this + has a good chance to make you into an open relay, so please be + careful. + + +File: python.info, Node: SMTPChannel Objects, Prev: MailmanProxy Objects, Up: smtpd — SMTP Server + +5.21.18.5 SMTPChannel Objects +............................. + + -- Class: smtpd.SMTPChannel (server, conn, addr, + data_size_limit=33554432, map=None, enable_SMTPUTF8=False, + decode_data=False) + + Create a new *note SMTPChannel: 601. object which manages the + communication between the server and a single SMTP client. + + `conn' and `addr' are as per the instance variables described + below. + + `data_size_limit' specifies the maximum number of bytes that will + be accepted in a ‘DATA’ command. A value of ‘None’ or ‘0’ means no + limit. + + `enable_SMTPUTF8' determines whether the ‘SMTPUTF8’ extension (as + defined in RFC 6531(1)) should be enabled. The default is ‘False’. + `decode_data' and `enable_SMTPUTF8' cannot be set to ‘True’ at the + same time. + + A dictionary can be specified in `map' to avoid using a global + socket map. + + `decode_data' specifies whether the data portion of the SMTP + transaction should be decoded using UTF-8. The default is ‘False’. + `decode_data' and `enable_SMTPUTF8' cannot be set to ‘True’ at the + same time. + + To use a custom SMTPChannel implementation you need to override the + *note SMTPServer.channel_class: 2ad3. of your *note SMTPServer: + 602. + + Changed in version 3.5: The `decode_data' and `enable_SMTPUTF8' + parameters were added. + + Changed in version 3.6: `decode_data' is now ‘False’ by default. + + The *note SMTPChannel: 601. has the following instance variables: + + -- Attribute: smtp_server + + Holds the *note SMTPServer: 602. that spawned this channel. + + -- Attribute: conn + + Holds the socket object connecting to the client. + + -- Attribute: addr + + Holds the address of the client, the second value returned by + *note socket.accept: 675. + + -- Attribute: received_lines + + Holds a list of the line strings (decoded using UTF-8) + received from the client. The lines have their ‘"\r\n"’ line + ending translated to ‘"\n"’. + + -- Attribute: smtp_state + + Holds the current state of the channel. This will be either + ‘COMMAND’ initially and then ‘DATA’ after the client sends a + “DATA” line. + + -- Attribute: seen_greeting + + Holds a string containing the greeting sent by the client in + its “HELO”. + + -- Attribute: mailfrom + + Holds a string containing the address identified in the “MAIL + FROM:” line from the client. + + -- Attribute: rcpttos + + Holds a list of strings containing the addresses identified in + the “RCPT TO:” lines from the client. + + -- Attribute: received_data + + Holds a string containing all of the data sent by the client + during the DATA state, up to but not including the terminating + ‘"\r\n.\r\n"’. + + -- Attribute: fqdn + + Holds the fully-qualified domain name of the server as + returned by *note socket.getfqdn(): 2381. + + -- Attribute: peer + + Holds the name of the client peer as returned by + ‘conn.getpeername()’ where ‘conn’ is *note conn: 2adc. + + The *note SMTPChannel: 601. operates by invoking methods named + ‘smtp_<command>’ upon reception of a command line from the client. + Built into the base *note SMTPChannel: 601. class are methods for + handling the following commands (and responding to them + appropriately): + + Command Action taken + + ------------------------------------------------------------------------------------- + + HELO Accepts the greeting from the client and stores it in + *note seen_greeting: 2ae0. Sets server to base command mode. + + + EHLO Accepts the greeting from the client and stores it in + *note seen_greeting: 2ae0. Sets server to extended command mode. + + + NOOP Takes no action. + + + QUIT Closes the connection cleanly. + + + MAIL Accepts the “MAIL FROM:” syntax and stores the supplied address as + *note mailfrom: 2ae1. In extended command mode, accepts the RFC + 1870(2) SIZE attribute and responds appropriately based on the value + of `data_size_limit'. + + + RCPT Accepts the “RCPT TO:” syntax and stores the supplied addresses in + the *note rcpttos: 2ae2. list. + + + RSET Resets the *note mailfrom: 2ae1, *note rcpttos: 2ae2, and + *note received_data: 2ae3, but not the greeting. + + + DATA Sets the internal state to ‘DATA’ and stores remaining lines from the + client in *note received_data: 2ae3. until the terminator + ‘"\r\n.\r\n"’ is received. + + + HELP Returns minimal information on command syntax + + + VRFY Returns code 252 (the server doesn’t know if the address is valid) + + + EXPN Reports that the command is not implemented. + + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc6531.html + + (2) https://tools.ietf.org/html/rfc1870.html + + +File: python.info, Node: telnetlib — Telnet client, Next: uuid — UUID objects according to RFC 4122, Prev: smtpd — SMTP Server, Up: Internet Protocols and Support + +5.21.19 ‘telnetlib’ — Telnet client +----------------------------------- + +`Source code:' Lib/telnetlib.py(1) + +__________________________________________________________________ + +The *note telnetlib: 102. module provides a *note Telnet: 597. class +that implements the Telnet protocol. See RFC 854(2) for details about +the protocol. In addition, it provides symbolic constants for the +protocol characters (see below), and for the telnet options. The +symbolic names of the telnet options follow the definitions in +‘arpa/telnet.h’, with the leading ‘TELOPT_’ removed. For symbolic names +of options which are traditionally not included in ‘arpa/telnet.h’, see +the module source itself. + +The symbolic constants for the telnet commands are: IAC, DONT, DO, WONT, +WILL, SE (Subnegotiation End), NOP (No Operation), DM (Data Mark), BRK +(Break), IP (Interrupt process), AO (Abort output), AYT (Are You There), +EC (Erase Character), EL (Erase Line), GA (Go Ahead), SB (Subnegotiation +Begin). + + -- Class: telnetlib.Telnet (host=None, port=0[, timeout]) + + *note Telnet: 597. represents a connection to a Telnet server. The + instance is initially not connected by default; the *note open(): + 2ae8. method must be used to establish a connection. + Alternatively, the host name and optional port number can be passed + to the constructor too, in which case the connection to the server + will be established before the constructor returns. The optional + `timeout' parameter specifies a timeout in seconds for blocking + operations like the connection attempt (if not specified, the + global default timeout setting will be used). + + Do not reopen an already connected instance. + + This class has many ‘read_*()’ methods. Note that some of them + raise *note EOFError: c6c. when the end of the connection is read, + because they can return an empty string for other reasons. See the + individual descriptions below. + + A *note Telnet: 597. object is a context manager and can be used in + a *note with: 6e9. statement. When the ‘with’ block ends, the + *note close(): 2ae9. method is called: + + >>> from telnetlib import Telnet + >>> with Telnet('localhost', 23) as tn: + ... tn.interact() + ... + + Changed in version 3.6: Context manager support added + +See also +........ + +RFC 854(3) - Telnet Protocol Specification + + Definition of the Telnet protocol. + +* Menu: + +* Telnet Objects:: +* Telnet Example:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/telnetlib.py + + (2) https://tools.ietf.org/html/rfc854.html + + (3) https://tools.ietf.org/html/rfc854.html + + +File: python.info, Node: Telnet Objects, Next: Telnet Example, Up: telnetlib — Telnet client + +5.21.19.1 Telnet Objects +........................ + +*note Telnet: 597. instances have the following methods: + + -- Method: Telnet.read_until (expected, timeout=None) + + Read until a given byte string, `expected', is encountered or until + `timeout' seconds have passed. + + When no match is found, return whatever is available instead, + possibly empty bytes. Raise *note EOFError: c6c. if the connection + is closed and no cooked data is available. + + -- Method: Telnet.read_all () + + Read all data until EOF as bytes; block until connection closed. + + -- Method: Telnet.read_some () + + Read at least one byte of cooked data unless EOF is hit. Return + ‘b''’ if EOF is hit. Block if no data is immediately available. + + -- Method: Telnet.read_very_eager () + + Read everything that can be without blocking in I/O (eager). + + Raise *note EOFError: c6c. if connection closed and no cooked data + available. Return ‘b''’ if no cooked data available otherwise. Do + not block unless in the midst of an IAC sequence. + + -- Method: Telnet.read_eager () + + Read readily available data. + + Raise *note EOFError: c6c. if connection closed and no cooked data + available. Return ‘b''’ if no cooked data available otherwise. Do + not block unless in the midst of an IAC sequence. + + -- Method: Telnet.read_lazy () + + Process and return data already in the queues (lazy). + + Raise *note EOFError: c6c. if connection closed and no data + available. Return ‘b''’ if no cooked data available otherwise. Do + not block unless in the midst of an IAC sequence. + + -- Method: Telnet.read_very_lazy () + + Return any data available in the cooked queue (very lazy). + + Raise *note EOFError: c6c. if connection closed and no data + available. Return ‘b''’ if no cooked data available otherwise. + This method never blocks. + + -- Method: Telnet.read_sb_data () + + Return the data collected between a SB/SE pair (suboption + begin/end). The callback should access these data when it was + invoked with a ‘SE’ command. This method never blocks. + + -- Method: Telnet.open (host, port=0[, timeout]) + + Connect to a host. The optional second argument is the port + number, which defaults to the standard Telnet port (23). The + optional `timeout' parameter specifies a timeout in seconds for + blocking operations like the connection attempt (if not specified, + the global default timeout setting will be used). + + Do not try to reopen an already connected instance. + + Raises an *note auditing event: fd1. ‘telnetlib.Telnet.open’ with + arguments ‘self’, ‘host’, ‘port’. + + -- Method: Telnet.msg (msg, *args) + + Print a debug message when the debug level is ‘>’ 0. If extra + arguments are present, they are substituted in the message using + the standard string formatting operator. + + -- Method: Telnet.set_debuglevel (debuglevel) + + Set the debug level. The higher the value of `debuglevel', the + more debug output you get (on ‘sys.stdout’). + + -- Method: Telnet.close () + + Close the connection. + + -- Method: Telnet.get_socket () + + Return the socket object used internally. + + -- Method: Telnet.fileno () + + Return the file descriptor of the socket object used internally. + + -- Method: Telnet.write (buffer) + + Write a byte string to the socket, doubling any IAC characters. + This can block if the connection is blocked. May raise *note + OSError: 1d3. if the connection is closed. + + Raises an *note auditing event: fd1. ‘telnetlib.Telnet.write’ with + arguments ‘self’, ‘buffer’. + + Changed in version 3.3: This method used to raise *note + socket.error: 995, which is now an alias of *note OSError: 1d3. + + -- Method: Telnet.interact () + + Interaction function, emulates a very dumb Telnet client. + + -- Method: Telnet.mt_interact () + + Multithreaded version of *note interact(): 2af9. + + -- Method: Telnet.expect (list, timeout=None) + + Read until one from a list of a regular expressions matches. + + The first argument is a list of regular expressions, either + compiled (*note regex objects: 89c.) or uncompiled (byte strings). + The optional second argument is a timeout, in seconds; the default + is to block indefinitely. + + Return a tuple of three items: the index in the list of the first + regular expression that matches; the match object returned; and the + bytes read up till and including the match. + + If end of file is found and no bytes were read, raise *note + EOFError: c6c. Otherwise, when nothing matches, return ‘(-1, None, + data)’ where `data' is the bytes received so far (may be empty + bytes if a timeout happened). + + If a regular expression ends with a greedy match (such as ‘.*’) or + if more than one expression can match the same input, the results + are non-deterministic, and may depend on the I/O timing. + + -- Method: Telnet.set_option_negotiation_callback (callback) + + Each time a telnet option is read on the input flow, this + `callback' (if set) is called with the following parameters: + callback(telnet socket, command (DO/DONT/WILL/WONT), option). No + other action is done afterwards by telnetlib. + + +File: python.info, Node: Telnet Example, Prev: Telnet Objects, Up: telnetlib — Telnet client + +5.21.19.2 Telnet Example +........................ + +A simple example illustrating typical use: + + import getpass + import telnetlib + + HOST = "localhost" + user = input("Enter your remote account: ") + password = getpass.getpass() + + tn = telnetlib.Telnet(HOST) + + tn.read_until(b"login: ") + tn.write(user.encode('ascii') + b"\n") + if password: + tn.read_until(b"Password: ") + tn.write(password.encode('ascii') + b"\n") + + tn.write(b"ls\n") + tn.write(b"exit\n") + + print(tn.read_all().decode('ascii')) + + +File: python.info, Node: uuid — UUID objects according to RFC 4122, Next: socketserver — A framework for network servers, Prev: telnetlib — Telnet client, Up: Internet Protocols and Support + +5.21.20 ‘uuid’ — UUID objects according to `RFC 4122' +----------------------------------------------------- + +`Source code:' Lib/uuid.py(1) + +__________________________________________________________________ + +This module provides immutable *note UUID: 260. objects (the *note UUID: +260. class) and the functions *note uuid1(): 413, *note uuid3(): 2b01, +*note uuid4(): 2b02, *note uuid5(): 2b03. for generating version 1, 3, +4, and 5 UUIDs as specified in RFC 4122(2). + +If all you want is a unique ID, you should probably call *note uuid1(): +413. or *note uuid4(): 2b02. Note that *note uuid1(): 413. may +compromise privacy since it creates a UUID containing the computer’s +network address. *note uuid4(): 2b02. creates a random UUID. + +Depending on support from the underlying platform, *note uuid1(): 413. +may or may not return a “safe” UUID. A safe UUID is one which is +generated using synchronization methods that ensure no two processes can +obtain the same UUID. All instances of *note UUID: 260. have an +‘is_safe’ attribute which relays any information about the UUID’s +safety, using this enumeration: + + -- Class: uuid.SafeUUID + + New in version 3.7. + + -- Attribute: safe + + The UUID was generated by the platform in a + multiprocessing-safe way. + + -- Attribute: unsafe + + The UUID was not generated in a multiprocessing-safe way. + + -- Attribute: unknown + + The platform does not provide information on whether the UUID + was generated safely or not. + + -- Class: uuid.UUID (hex=None, bytes=None, bytes_le=None, fields=None, + int=None, version=None, *, is_safe=SafeUUID.unknown) + + Create a UUID from either a string of 32 hexadecimal digits, a + string of 16 bytes in big-endian order as the `bytes' argument, a + string of 16 bytes in little-endian order as the `bytes_le' + argument, a tuple of six integers (32-bit `time_low', 16-bit + `time_mid', 16-bit `time_hi_version', 8-bit `clock_seq_hi_variant', + 8-bit `clock_seq_low', 48-bit `node') as the `fields' argument, or + a single 128-bit integer as the `int' argument. When a string of + hex digits is given, curly braces, hyphens, and a URN prefix are + all optional. For example, these expressions all yield the same + UUID: + + UUID('{12345678-1234-5678-1234-567812345678}') + UUID('12345678123456781234567812345678') + UUID('urn:uuid:12345678-1234-5678-1234-567812345678') + UUID(bytes=b'\x12\x34\x56\x78'*4) + UUID(bytes_le=b'\x78\x56\x34\x12\x34\x12\x78\x56' + + b'\x12\x34\x56\x78\x12\x34\x56\x78') + UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) + UUID(int=0x12345678123456781234567812345678) + + Exactly one of `hex', `bytes', `bytes_le', `fields', or `int' must + be given. The `version' argument is optional; if given, the + resulting UUID will have its variant and version number set + according to RFC 4122(3), overriding bits in the given `hex', + `bytes', `bytes_le', `fields', or `int'. + + Comparison of UUID objects are made by way of comparing their *note + UUID.int: 2b08. attributes. Comparison with a non-UUID object + raises a *note TypeError: 192. + + ‘str(uuid)’ returns a string in the form + ‘12345678-1234-5678-1234-567812345678’ where the 32 hexadecimal + digits represent the UUID. + +*note UUID: 260. instances have these read-only attributes: + + -- Attribute: UUID.bytes + + The UUID as a 16-byte string (containing the six integer fields in + big-endian byte order). + + -- Attribute: UUID.bytes_le + + The UUID as a 16-byte string (with `time_low', `time_mid', and + `time_hi_version' in little-endian byte order). + + -- Attribute: UUID.fields + + A tuple of the six integer fields of the UUID, which are also + available as six individual attributes and two derived attributes: + + Field Meaning + + ----------------------------------------------------------------------- + + ‘time_low’ the first 32 bits of the UUID + + + ‘time_mid’ the next 16 bits of the UUID + + + ‘time_hi_version’ the next 16 bits of the UUID + + + ‘clock_seq_hi_variant’ the next 8 bits of the UUID + + + ‘clock_seq_low’ the next 8 bits of the UUID + + + ‘node’ the last 48 bits of the UUID + + + *note time: 10a. the 60-bit timestamp + + + ‘clock_seq’ the 14-bit sequence number + + + -- Attribute: UUID.hex + + The UUID as a 32-character hexadecimal string. + + -- Attribute: UUID.int + + The UUID as a 128-bit integer. + + -- Attribute: UUID.urn + + The UUID as a URN as specified in RFC 4122(4). + + -- Attribute: UUID.variant + + The UUID variant, which determines the internal layout of the UUID. + This will be one of the constants *note RESERVED_NCS: 2b0f, *note + RFC_4122: 2b10, *note RESERVED_MICROSOFT: 2b11, or *note + RESERVED_FUTURE: 2b12. + + -- Attribute: UUID.version + + The UUID version number (1 through 5, meaningful only when the + variant is *note RFC_4122: 2b10.). + + -- Attribute: UUID.is_safe + + An enumeration of *note SafeUUID: 2b04. which indicates whether the + platform generated the UUID in a multiprocessing-safe way. + + New in version 3.7. + +The *note uuid: 124. module defines the following functions: + + -- Function: uuid.getnode () + + Get the hardware address as a 48-bit positive integer. The first + time this runs, it may launch a separate program, which could be + quite slow. If all attempts to obtain the hardware address fail, + we choose a random 48-bit number with the multicast bit (least + significant bit of the first octet) set to 1 as recommended in RFC + 4122(5). “Hardware address” means the MAC address of a network + interface. On a machine with multiple network interfaces, + universally administered MAC addresses (i.e. where the second + least significant bit of the first octet is `unset') will be + preferred over locally administered MAC addresses, but with no + other ordering guarantees. + + Changed in version 3.7: Universally administered MAC addresses are + preferred over locally administered MAC addresses, since the former + are guaranteed to be globally unique, while the latter are not. + + -- Function: uuid.uuid1 (node=None, clock_seq=None) + + Generate a UUID from a host ID, sequence number, and the current + time. If `node' is not given, *note getnode(): 412. is used to + obtain the hardware address. If `clock_seq' is given, it is used + as the sequence number; otherwise a random 14-bit sequence number + is chosen. + + -- Function: uuid.uuid3 (namespace, name) + + Generate a UUID based on the MD5 hash of a namespace identifier + (which is a UUID) and a name (which is a string). + + -- Function: uuid.uuid4 () + + Generate a random UUID. + + -- Function: uuid.uuid5 (namespace, name) + + Generate a UUID based on the SHA-1 hash of a namespace identifier + (which is a UUID) and a name (which is a string). + +The *note uuid: 124. module defines the following namespace identifiers +for use with *note uuid3(): 2b01. or *note uuid5(): 2b03. + + -- Data: uuid.NAMESPACE_DNS + + When this namespace is specified, the `name' string is a + fully-qualified domain name. + + -- Data: uuid.NAMESPACE_URL + + When this namespace is specified, the `name' string is a URL. + + -- Data: uuid.NAMESPACE_OID + + When this namespace is specified, the `name' string is an ISO OID. + + -- Data: uuid.NAMESPACE_X500 + + When this namespace is specified, the `name' string is an X.500 DN + in DER or a text output format. + +The *note uuid: 124. module defines the following constants for the +possible values of the ‘variant’ attribute: + + -- Data: uuid.RESERVED_NCS + + Reserved for NCS compatibility. + + -- Data: uuid.RFC_4122 + + Specifies the UUID layout given in RFC 4122(6). + + -- Data: uuid.RESERVED_MICROSOFT + + Reserved for Microsoft compatibility. + + -- Data: uuid.RESERVED_FUTURE + + Reserved for future definition. + +See also +........ + +RFC 4122(7) - A Universally Unique IDentifier (UUID) URN Namespace + + This specification defines a Uniform Resource Name namespace for + UUIDs, the internal format of UUIDs, and methods of generating + UUIDs. + +* Menu: + +* Example: Example<13>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/uuid.py + + (2) https://tools.ietf.org/html/rfc4122.html + + (3) https://tools.ietf.org/html/rfc4122.html + + (4) https://tools.ietf.org/html/rfc4122.html + + (5) https://tools.ietf.org/html/rfc4122.html + + (6) https://tools.ietf.org/html/rfc4122.html + + (7) https://tools.ietf.org/html/rfc4122.html + + +File: python.info, Node: Example<13>, Up: uuid — UUID objects according to RFC 4122 + +5.21.20.1 Example +................. + +Here are some examples of typical usage of the *note uuid: 124. module: + + >>> import uuid + + >>> # make a UUID based on the host ID and current time + >>> uuid.uuid1() + UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + + >>> # make a UUID using an MD5 hash of a namespace UUID and a name + >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') + UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + + >>> # make a random UUID + >>> uuid.uuid4() + UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + + >>> # make a UUID using a SHA-1 hash of a namespace UUID and a name + >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') + UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + + >>> # make a UUID from a string of hex digits (braces and hyphens ignored) + >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + + >>> # convert a UUID to a string of hex digits in standard form + >>> str(x) + '00010203-0405-0607-0809-0a0b0c0d0e0f' + + >>> # get the raw 16 bytes of the UUID + >>> x.bytes + b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + + >>> # make a UUID from a 16-byte string + >>> uuid.UUID(bytes=x.bytes) + UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') + + +File: python.info, Node: socketserver — A framework for network servers, Next: http server — HTTP servers, Prev: uuid — UUID objects according to RFC 4122, Up: Internet Protocols and Support + +5.21.21 ‘socketserver’ — A framework for network servers +-------------------------------------------------------- + +`Source code:' Lib/socketserver.py(1) + +__________________________________________________________________ + +The *note socketserver: f0. module simplifies the task of writing +network servers. + +There are four basic concrete server classes: + + -- Class: socketserver.TCPServer (server_address, RequestHandlerClass, + bind_and_activate=True) + + This uses the Internet TCP protocol, which provides for continuous + streams of data between the client and server. If + `bind_and_activate' is true, the constructor automatically attempts + to invoke *note server_bind(): 2b1d. and *note server_activate(): + 2b1e. The other parameters are passed to the *note BaseServer: + a89. base class. + + -- Class: socketserver.UDPServer (server_address, RequestHandlerClass, + bind_and_activate=True) + + This uses datagrams, which are discrete packets of information that + may arrive out of order or be lost while in transit. The + parameters are the same as for *note TCPServer: 2b1c. + + -- Class: socketserver.UnixStreamServer (server_address, + RequestHandlerClass, bind_and_activate=True) + -- Class: socketserver.UnixDatagramServer (server_address, + RequestHandlerClass, bind_and_activate=True) + + These more infrequently used classes are similar to the TCP and UDP + classes, but use Unix domain sockets; they’re not available on + non-Unix platforms. The parameters are the same as for *note + TCPServer: 2b1c. + +These four classes process requests `synchronously'; each request must +be completed before the next request can be started. This isn’t +suitable if each request takes a long time to complete, because it +requires a lot of computation, or because it returns a lot of data which +the client is slow to process. The solution is to create a separate +process or thread to handle each request; the *note ForkingMixIn: 3d5. +and *note ThreadingMixIn: 3d6. mix-in classes can be used to support +asynchronous behaviour. + +Creating a server requires several steps. First, you must create a +request handler class by subclassing the *note BaseRequestHandler: 2b22. +class and overriding its *note handle(): 2b23. method; this method will +process incoming requests. Second, you must instantiate one of the +server classes, passing it the server’s address and the request handler +class. It is recommended to use the server in a *note with: 6e9. +statement. Then call the *note handle_request(): 2b24. or *note +serve_forever(): a8b. method of the server object to process one or many +requests. Finally, call *note server_close(): 2b25. to close the socket +(unless you used a ‘with’ statement). + +When inheriting from *note ThreadingMixIn: 3d6. for threaded connection +behavior, you should explicitly declare how you want your threads to +behave on an abrupt shutdown. The *note ThreadingMixIn: 3d6. class +defines an attribute `daemon_threads', which indicates whether or not +the server should wait for thread termination. You should set the flag +explicitly if you would like threads to behave autonomously; the default +is *note False: 388, meaning that Python will not exit until all threads +created by *note ThreadingMixIn: 3d6. have exited. + +Server classes have the same external methods and attributes, no matter +what network protocol they use. + +* Menu: + +* Server Creation Notes:: +* Server Objects: Server Objects<2>. +* Request Handler Objects:: +* Examples: Examples<23>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/socketserver.py + + +File: python.info, Node: Server Creation Notes, Next: Server Objects<2>, Up: socketserver — A framework for network servers + +5.21.21.1 Server Creation Notes +............................... + +There are five classes in an inheritance diagram, four of which +represent synchronous servers of four types: + + +------------+ + | BaseServer | + +------------+ + | + v + +-----------+ +------------------+ + | TCPServer |------->| UnixStreamServer | + +-----------+ +------------------+ + | + v + +-----------+ +--------------------+ + | UDPServer |------->| UnixDatagramServer | + +-----------+ +--------------------+ + +Note that *note UnixDatagramServer: 2b21. derives from *note UDPServer: +2b1f, not from *note UnixStreamServer: 2b20. — the only difference +between an IP and a Unix stream server is the address family, which is +simply repeated in both Unix server classes. + + -- Class: socketserver.ForkingMixIn + -- Class: socketserver.ThreadingMixIn + + Forking and threading versions of each type of server can be + created using these mix-in classes. For instance, *note + ThreadingUDPServer: 2b27. is created as follows: + + class ThreadingUDPServer(ThreadingMixIn, UDPServer): + pass + + The mix-in class comes first, since it overrides a method defined + in *note UDPServer: 2b1f. Setting the various attributes also + changes the behavior of the underlying server mechanism. + + *note ForkingMixIn: 3d5. and the Forking classes mentioned below + are only available on POSIX platforms that support *note fork(): + 961. + + ‘socketserver.ForkingMixIn.server_close()’ waits until all child + processes complete, except if + ‘socketserver.ForkingMixIn.block_on_close’ attribute is false. + + ‘socketserver.ThreadingMixIn.server_close()’ waits until all + non-daemon threads complete, except if + ‘socketserver.ThreadingMixIn.block_on_close’ attribute is false. + Use daemonic threads by setting ‘ThreadingMixIn.daemon_threads’ to + ‘True’ to not wait until threads complete. + + Changed in version 3.7: ‘socketserver.ForkingMixIn.server_close()’ + and ‘socketserver.ThreadingMixIn.server_close()’ now waits until + all child processes and non-daemonic threads complete. Add a new + ‘socketserver.ForkingMixIn.block_on_close’ class attribute to + opt-in for the pre-3.7 behaviour. + + -- Class: socketserver.ForkingTCPServer + -- Class: socketserver.ForkingUDPServer + -- Class: socketserver.ThreadingTCPServer + -- Class: socketserver.ThreadingUDPServer + + These classes are pre-defined using the mix-in classes. + +To implement a service, you must derive a class from *note +BaseRequestHandler: 2b22. and redefine its *note handle(): 2b23. method. +You can then run various versions of the service by combining one of the +server classes with your request handler class. The request handler +class must be different for datagram or stream services. This can be +hidden by using the handler subclasses *note StreamRequestHandler: 587. +or *note DatagramRequestHandler: 2b2b. + +Of course, you still have to use your head! For instance, it makes no +sense to use a forking server if the service contains state in memory +that can be modified by different requests, since the modifications in +the child process would never reach the initial state kept in the parent +process and passed to each child. In this case, you can use a threading +server, but you will probably have to use locks to protect the integrity +of the shared data. + +On the other hand, if you are building an HTTP server where all data is +stored externally (for instance, in the file system), a synchronous +class will essentially render the service “deaf” while one request is +being handled – which may be for a very long time if a client is slow to +receive all the data it has requested. Here a threading or forking +server is appropriate. + +In some cases, it may be appropriate to process part of a request +synchronously, but to finish processing in a forked child depending on +the request data. This can be implemented by using a synchronous server +and doing an explicit fork in the request handler class *note handle(): +2b23. method. + +Another approach to handling multiple simultaneous requests in an +environment that supports neither threads nor *note fork(): 961. (or +where these are too expensive or inappropriate for the service) is to +maintain an explicit table of partially finished requests and to use +*note selectors: e6. to decide which request to work on next (or whether +to handle a new incoming request). This is particularly important for +stream services where each client can potentially be connected for a +long time (if threads or subprocesses cannot be used). See *note +asyncore: b. for another way to manage this. + + +File: python.info, Node: Server Objects<2>, Next: Request Handler Objects, Prev: Server Creation Notes, Up: socketserver — A framework for network servers + +5.21.21.2 Server Objects +........................ + + -- Class: socketserver.BaseServer (server_address, RequestHandlerClass) + + This is the superclass of all Server objects in the module. It + defines the interface, given below, but does not implement most of + the methods, which is done in subclasses. The two parameters are + stored in the respective *note server_address: 2b2d. and *note + RequestHandlerClass: 2b2e. attributes. + + -- Method: fileno () + + Return an integer file descriptor for the socket on which the + server is listening. This function is most commonly passed to + *note selectors: e6, to allow monitoring multiple servers in + the same process. + + -- Method: handle_request () + + Process a single request. This function calls the following + methods in order: *note get_request(): 2b30, *note + verify_request(): 2b31, and *note process_request(): 2b32. If + the user-provided *note handle(): 2b23. method of the handler + class raises an exception, the server’s *note handle_error(): + 5fd. method will be called. If no request is received within + *note timeout: 2b33. seconds, *note handle_timeout(): 2b34. + will be called and *note handle_request(): 2b24. will return. + + -- Method: serve_forever (poll_interval=0.5) + + Handle requests until an explicit *note shutdown(): 2b35. + request. Poll for shutdown every `poll_interval' seconds. + Ignores the *note timeout: 2b33. attribute. It also calls + *note service_actions(): a8a, which may be used by a subclass + or mixin to provide actions specific to a given service. For + example, the *note ForkingMixIn: 3d5. class uses *note + service_actions(): a8a. to clean up zombie child processes. + + Changed in version 3.3: Added ‘service_actions’ call to the + ‘serve_forever’ method. + + -- Method: service_actions () + + This is called in the *note serve_forever(): a8b. loop. This + method can be overridden by subclasses or mixin classes to + perform actions specific to a given service, such as cleanup + actions. + + New in version 3.3. + + -- Method: shutdown () + + Tell the *note serve_forever(): a8b. loop to stop and wait + until it does. *note shutdown(): 2b35. must be called while + *note serve_forever(): a8b. is running in a different thread + otherwise it will deadlock. + + -- Method: server_close () + + Clean up the server. May be overridden. + + -- Attribute: address_family + + The family of protocols to which the server’s socket belongs. + Common examples are *note socket.AF_INET: 229d. and *note + socket.AF_UNIX: 22a3. + + -- Attribute: RequestHandlerClass + + The user-provided request handler class; an instance of this + class is created for each request. + + -- Attribute: server_address + + The address on which the server is listening. The format of + addresses varies depending on the protocol family; see the + documentation for the *note socket: ef. module for details. + For Internet protocols, this is a tuple containing a string + giving the address, and an integer port number: ‘('127.0.0.1', + 80)’, for example. + + -- Attribute: socket + + The socket object on which the server will listen for incoming + requests. + + The server classes support the following class variables: + + -- Attribute: allow_reuse_address + + Whether the server will allow the reuse of an address. This + defaults to *note False: 388, and can be set in subclasses to + change the policy. + + -- Attribute: request_queue_size + + The size of the request queue. If it takes a long time to + process a single request, any requests that arrive while the + server is busy are placed into a queue, up to *note + request_queue_size: 2b39. requests. Once the queue is full, + further requests from clients will get a “Connection denied” + error. The default value is usually 5, but this can be + overridden by subclasses. + + -- Attribute: socket_type + + The type of socket used by the server; *note + socket.SOCK_STREAM: c8a. and *note socket.SOCK_DGRAM: c89. are + two common values. + + -- Attribute: timeout + + Timeout duration, measured in seconds, or *note None: 157. if + no timeout is desired. If *note handle_request(): 2b24. + receives no incoming requests within the timeout period, the + *note handle_timeout(): 2b34. method is called. + + There are various server methods that can be overridden by + subclasses of base server classes like *note TCPServer: 2b1c.; + these methods aren’t useful to external users of the server object. + + -- Method: finish_request (request, client_address) + + Actually processes the request by instantiating *note + RequestHandlerClass: 2b2e. and calling its *note handle(): + 2b23. method. + + -- Method: get_request () + + Must accept a request from the socket, and return a 2-tuple + containing the `new' socket object to be used to communicate + with the client, and the client’s address. + + -- Method: handle_error (request, client_address) + + This function is called if the *note handle(): 2b23. method of + a *note RequestHandlerClass: 2b2e. instance raises an + exception. The default action is to print the traceback to + standard error and continue handling further requests. + + Changed in version 3.6: Now only called for exceptions derived + from the *note Exception: 1a9. class. + + -- Method: handle_timeout () + + This function is called when the *note timeout: 2b33. + attribute has been set to a value other than *note None: 157. + and the timeout period has passed with no requests being + received. The default action for forking servers is to + collect the status of any child processes that have exited, + while in threading servers this method does nothing. + + -- Method: process_request (request, client_address) + + Calls *note finish_request(): 2b3b. to create an instance of + the *note RequestHandlerClass: 2b2e. If desired, this + function can create a new process or thread to handle the + request; the *note ForkingMixIn: 3d5. and *note + ThreadingMixIn: 3d6. classes do this. + + -- Method: server_activate () + + Called by the server’s constructor to activate the server. + The default behavior for a TCP server just invokes *note + listen(): 74b. on the server’s socket. May be overridden. + + -- Method: server_bind () + + Called by the server’s constructor to bind the socket to the + desired address. May be overridden. + + -- Method: verify_request (request, client_address) + + Must return a Boolean value; if the value is *note True: 499, + the request will be processed, and if it’s *note False: 388, + the request will be denied. This function can be overridden + to implement access controls for a server. The default + implementation always returns *note True: 499. + + Changed in version 3.6: Support for the *note context manager: 568. + protocol was added. Exiting the context manager is equivalent to + calling *note server_close(): 2b25. + + +File: python.info, Node: Request Handler Objects, Next: Examples<23>, Prev: Server Objects<2>, Up: socketserver — A framework for network servers + +5.21.21.3 Request Handler Objects +................................. + + -- Class: socketserver.BaseRequestHandler + + This is the superclass of all request handler objects. It defines + the interface, given below. A concrete request handler subclass + must define a new *note handle(): 2b23. method, and can override + any of the other methods. A new instance of the subclass is + created for each request. + + -- Method: setup () + + Called before the *note handle(): 2b23. method to perform any + initialization actions required. The default implementation + does nothing. + + -- Method: handle () + + This function must do all the work required to service a + request. The default implementation does nothing. Several + instance attributes are available to it; the request is + available as ‘self.request’; the client address as + ‘self.client_address’; and the server instance as + ‘self.server’, in case it needs access to per-server + information. + + The type of ‘self.request’ is different for datagram or stream + services. For stream services, ‘self.request’ is a socket + object; for datagram services, ‘self.request’ is a pair of + string and socket. + + -- Method: finish () + + Called after the *note handle(): 2b23. method to perform any + clean-up actions required. The default implementation does + nothing. If *note setup(): 2b3d. raises an exception, this + function will not be called. + + -- Class: socketserver.StreamRequestHandler + -- Class: socketserver.DatagramRequestHandler + + These *note BaseRequestHandler: 2b22. subclasses override the *note + setup(): 2b3d. and *note finish(): 2b3e. methods, and provide + ‘self.rfile’ and ‘self.wfile’ attributes. The ‘self.rfile’ and + ‘self.wfile’ attributes can be read or written, respectively, to + get the request data or return data to the client. + + The ‘rfile’ attributes of both classes support the *note + io.BufferedIOBase: 588. readable interface, and + ‘DatagramRequestHandler.wfile’ supports the *note + io.BufferedIOBase: 588. writable interface. + + Changed in version 3.6: ‘StreamRequestHandler.wfile’ also supports + the *note io.BufferedIOBase: 588. writable interface. + + +File: python.info, Node: Examples<23>, Prev: Request Handler Objects, Up: socketserver — A framework for network servers + +5.21.21.4 Examples +.................. + +* Menu: + +* socketserver.TCPServer Example: socketserver TCPServer Example. +* socketserver.UDPServer Example: socketserver UDPServer Example. +* Asynchronous Mixins:: + + +File: python.info, Node: socketserver TCPServer Example, Next: socketserver UDPServer Example, Up: Examples<23> + +5.21.21.5 ‘socketserver.TCPServer’ Example +.......................................... + +This is the server side: + + import socketserver + + class MyTCPHandler(socketserver.BaseRequestHandler): + """ + The request handler class for our server. + + It is instantiated once per connection to the server, and must + override the handle() method to implement communication to the + client. + """ + + def handle(self): + # self.request is the TCP socket connected to the client + self.data = self.request.recv(1024).strip() + print("{} wrote:".format(self.client_address[0])) + print(self.data) + # just send back the same data, but upper-cased + self.request.sendall(self.data.upper()) + + if __name__ == "__main__": + HOST, PORT = "localhost", 9999 + + # Create the server, binding to localhost on port 9999 + with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server: + # Activate the server; this will keep running until you + # interrupt the program with Ctrl-C + server.serve_forever() + +An alternative request handler class that makes use of streams +(file-like objects that simplify communication by providing the standard +file interface): + + class MyTCPHandler(socketserver.StreamRequestHandler): + + def handle(self): + # self.rfile is a file-like object created by the handler; + # we can now use e.g. readline() instead of raw recv() calls + self.data = self.rfile.readline().strip() + print("{} wrote:".format(self.client_address[0])) + print(self.data) + # Likewise, self.wfile is a file-like object used to write back + # to the client + self.wfile.write(self.data.upper()) + +The difference is that the ‘readline()’ call in the second handler will +call ‘recv()’ multiple times until it encounters a newline character, +while the single ‘recv()’ call in the first handler will just return +what has been sent from the client in one ‘sendall()’ call. + +This is the client side: + + import socket + import sys + + HOST, PORT = "localhost", 9999 + data = " ".join(sys.argv[1:]) + + # Create a socket (SOCK_STREAM means a TCP socket) + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + # Connect to server and send data + sock.connect((HOST, PORT)) + sock.sendall(bytes(data + "\n", "utf-8")) + + # Receive data from the server and shut down + received = str(sock.recv(1024), "utf-8") + + print("Sent: {}".format(data)) + print("Received: {}".format(received)) + +The output of the example should look something like this: + +Server: + + $ python TCPServer.py + 127.0.0.1 wrote: + b'hello world with TCP' + 127.0.0.1 wrote: + b'python is nice' + +Client: + + $ python TCPClient.py hello world with TCP + Sent: hello world with TCP + Received: HELLO WORLD WITH TCP + $ python TCPClient.py python is nice + Sent: python is nice + Received: PYTHON IS NICE + + +File: python.info, Node: socketserver UDPServer Example, Next: Asynchronous Mixins, Prev: socketserver TCPServer Example, Up: Examples<23> + +5.21.21.6 ‘socketserver.UDPServer’ Example +.......................................... + +This is the server side: + + import socketserver + + class MyUDPHandler(socketserver.BaseRequestHandler): + """ + This class works similar to the TCP handler class, except that + self.request consists of a pair of data and client socket, and since + there is no connection the client address must be given explicitly + when sending data back via sendto(). + """ + + def handle(self): + data = self.request[0].strip() + socket = self.request[1] + print("{} wrote:".format(self.client_address[0])) + print(data) + socket.sendto(data.upper(), self.client_address) + + if __name__ == "__main__": + HOST, PORT = "localhost", 9999 + with socketserver.UDPServer((HOST, PORT), MyUDPHandler) as server: + server.serve_forever() + +This is the client side: + + import socket + import sys + + HOST, PORT = "localhost", 9999 + data = " ".join(sys.argv[1:]) + + # SOCK_DGRAM is the socket type to use for UDP sockets + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + + # As you can see, there is no connect() call; UDP has no connections. + # Instead, data is directly sent to the recipient via sendto(). + sock.sendto(bytes(data + "\n", "utf-8"), (HOST, PORT)) + received = str(sock.recv(1024), "utf-8") + + print("Sent: {}".format(data)) + print("Received: {}".format(received)) + +The output of the example should look exactly like for the TCP server +example. + + +File: python.info, Node: Asynchronous Mixins, Prev: socketserver UDPServer Example, Up: Examples<23> + +5.21.21.7 Asynchronous Mixins +............................. + +To build asynchronous handlers, use the *note ThreadingMixIn: 3d6. and +*note ForkingMixIn: 3d5. classes. + +An example for the *note ThreadingMixIn: 3d6. class: + + import socket + import threading + import socketserver + + class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler): + + def handle(self): + data = str(self.request.recv(1024), 'ascii') + cur_thread = threading.current_thread() + response = bytes("{}: {}".format(cur_thread.name, data), 'ascii') + self.request.sendall(response) + + class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer): + pass + + def client(ip, port, message): + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.connect((ip, port)) + sock.sendall(bytes(message, 'ascii')) + response = str(sock.recv(1024), 'ascii') + print("Received: {}".format(response)) + + if __name__ == "__main__": + # Port 0 means to select an arbitrary unused port + HOST, PORT = "localhost", 0 + + server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler) + with server: + ip, port = server.server_address + + # Start a thread with the server -- that thread will then start one + # more thread for each request + server_thread = threading.Thread(target=server.serve_forever) + # Exit the server thread when the main thread terminates + server_thread.daemon = True + server_thread.start() + print("Server loop running in thread:", server_thread.name) + + client(ip, port, "Hello World 1") + client(ip, port, "Hello World 2") + client(ip, port, "Hello World 3") + + server.shutdown() + +The output of the example should look something like this: + + $ python ThreadedTCPServer.py + Server loop running in thread: Thread-1 + Received: Thread-2: Hello World 1 + Received: Thread-3: Hello World 2 + Received: Thread-4: Hello World 3 + +The *note ForkingMixIn: 3d5. class is used in the same way, except that +the server will spawn a new process for each request. Available only on +POSIX platforms that support *note fork(): 961. + + +File: python.info, Node: http server — HTTP servers, Next: http cookies — HTTP state management, Prev: socketserver — A framework for network servers, Up: Internet Protocols and Support + +5.21.22 ‘http.server’ — HTTP servers +------------------------------------ + +`Source code:' Lib/http/server.py(1) + +__________________________________________________________________ + +This module defines classes for implementing HTTP servers (Web servers). + + Warning: *note http.server: 97. is not recommended for production. + It only implements basic security checks. + +One class, *note HTTPServer: 2909, is a *note socketserver.TCPServer: +2b1c. subclass. It creates and listens at the HTTP socket, dispatching +the requests to a handler. Code to create and run the server looks like +this: + + def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler): + server_address = ('', 8000) + httpd = server_class(server_address, handler_class) + httpd.serve_forever() + + -- Class: http.server.HTTPServer (server_address, RequestHandlerClass) + + This class builds on the *note TCPServer: 2b1c. class by storing + the server address as instance variables named ‘server_name’ and + ‘server_port’. The server is accessible by the handler, typically + through the handler’s ‘server’ instance variable. + + -- Class: http.server.ThreadingHTTPServer (server_address, + RequestHandlerClass) + + This class is identical to HTTPServer but uses threads to handle + requests by using the *note ThreadingMixIn: 3d6. This is useful to + handle web browsers pre-opening sockets, on which *note HTTPServer: + 2909. would wait indefinitely. + + New in version 3.7. + +The *note HTTPServer: 2909. and *note ThreadingHTTPServer: 396. must be +given a `RequestHandlerClass' on instantiation, of which this module +provides three different variants: + + -- Class: http.server.BaseHTTPRequestHandler (request, client_address, + server) + + This class is used to handle the HTTP requests that arrive at the + server. By itself, it cannot respond to any actual HTTP requests; + it must be subclassed to handle each request method (e.g. GET or + POST). *note BaseHTTPRequestHandler: a0e. provides a number of + class and instance variables, and methods for use by subclasses. + + The handler will parse the request and the headers, then call a + method specific to the request type. The method name is + constructed from the request. For example, for the request method + ‘SPAM’, the ‘do_SPAM()’ method will be called with no arguments. + All of the relevant information is stored in instance variables of + the handler. Subclasses should not need to override or extend the + *note __init__(): d5e. method. + + *note BaseHTTPRequestHandler: a0e. has the following instance + variables: + + -- Attribute: client_address + + Contains a tuple of the form ‘(host, port)’ referring to the + client’s address. + + -- Attribute: server + + Contains the server instance. + + -- Attribute: close_connection + + Boolean that should be set before *note handle_one_request(): + 2b48. returns, indicating if another request may be expected, + or if the connection should be shut down. + + -- Attribute: requestline + + Contains the string representation of the HTTP request line. + The terminating CRLF is stripped. This attribute should be + set by *note handle_one_request(): 2b48. If no valid request + line was processed, it should be set to the empty string. + + -- Attribute: command + + Contains the command (request type). For example, ‘'GET'’. + + -- Attribute: path + + Contains the request path. + + -- Attribute: request_version + + Contains the version string from the request. For example, + ‘'HTTP/1.0'’. + + -- Attribute: headers + + Holds an instance of the class specified by the *note + MessageClass: 2b4d. class variable. This instance parses and + manages the headers in the HTTP request. The *note + parse_headers(): 29ed. function from *note http.client: 94. is + used to parse the headers and it requires that the HTTP + request provide a valid RFC 2822(2) style header. + + -- Attribute: rfile + + An *note io.BufferedIOBase: 588. input stream, ready to read + from the start of the optional input data. + + -- Attribute: wfile + + Contains the output stream for writing a response back to the + client. Proper adherence to the HTTP protocol must be used + when writing to this stream in order to achieve successful + interoperation with HTTP clients. + + Changed in version 3.6: This is an *note io.BufferedIOBase: + 588. stream. + + *note BaseHTTPRequestHandler: a0e. has the following attributes: + + -- Attribute: server_version + + Specifies the server software version. You may want to + override this. The format is multiple whitespace-separated + strings, where each string is of the form name[/version]. For + example, ‘'BaseHTTP/0.2'’. + + -- Attribute: sys_version + + Contains the Python system version, in a form usable by the + *note version_string: 2b52. method and the *note + server_version: 2b50. class variable. For example, + ‘'Python/1.4'’. + + -- Attribute: error_message_format + + Specifies a format string that should be used by *note + send_error(): 85c. method for building an error response to + the client. The string is filled by default with variables + from *note responses: 29d9. based on the status code that + passed to *note send_error(): 85c. + + -- Attribute: error_content_type + + Specifies the Content-Type HTTP header of error responses sent + to the client. The default value is ‘'text/html'’. + + -- Attribute: protocol_version + + This specifies the HTTP protocol version used in responses. + If set to ‘'HTTP/1.1'’, the server will permit HTTP persistent + connections; however, your server `must' then include an + accurate ‘Content-Length’ header (using *note send_header(): + 2b56.) in all of its responses to clients. For backwards + compatibility, the setting defaults to ‘'HTTP/1.0'’. + + -- Attribute: MessageClass + + Specifies an *note email.message.Message: 541.-like class to + parse HTTP headers. Typically, this is not overridden, and it + defaults to ‘http.client.HTTPMessage’. + + -- Attribute: responses + + This attribute contains a mapping of error code integers to + two-element tuples containing a short and long message. For + example, ‘{code: (shortmessage, longmessage)}’. The + `shortmessage' is usually used as the `message' key in an + error response, and `longmessage' as the `explain' key. It is + used by *note send_response_only(): 2b57. and *note + send_error(): 85c. methods. + + A *note BaseHTTPRequestHandler: a0e. instance has the following + methods: + + -- Method: handle () + + Calls *note handle_one_request(): 2b48. once (or, if + persistent connections are enabled, multiple times) to handle + incoming HTTP requests. You should never need to override it; + instead, implement appropriate ‘do_*()’ methods. + + -- Method: handle_one_request () + + This method will parse and dispatch the request to the + appropriate ‘do_*()’ method. You should never need to + override it. + + -- Method: handle_expect_100 () + + When a HTTP/1.1 compliant server receives an ‘Expect: + 100-continue’ request header it responds back with a ‘100 + Continue’ followed by ‘200 OK’ headers. This method can be + overridden to raise an error if the server does not want the + client to continue. For e.g. server can chose to send ‘417 + Expectation Failed’ as a response header and ‘return False’. + + New in version 3.2. + + -- Method: send_error (code, message=None, explain=None) + + Sends and logs a complete error reply to the client. The + numeric `code' specifies the HTTP error code, with `message' + as an optional, short, human readable description of the + error. The `explain' argument can be used to provide more + detailed information about the error; it will be formatted + using the *note error_message_format: 2b53. attribute and + emitted, after a complete set of headers, as the response + body. The *note responses: 29d9. attribute holds the default + values for `message' and `explain' that will be used if no + value is provided; for unknown codes the default value for + both is the string ‘???’. The body will be empty if the + method is HEAD or the response code is one of the following: + ‘1xx’, ‘204 No Content’, ‘205 Reset Content’, ‘304 Not + Modified’. + + Changed in version 3.4: The error response includes a + Content-Length header. Added the `explain' argument. + + -- Method: send_response (code, message=None) + + Adds a response header to the headers buffer and logs the + accepted request. The HTTP response line is written to the + internal buffer, followed by `Server' and `Date' headers. The + values for these two headers are picked up from the *note + version_string(): 2b52. and *note date_time_string(): 2b5b. + methods, respectively. If the server does not intend to send + any other headers using the *note send_header(): 2b56. method, + then *note send_response(): 2b5a. should be followed by an + *note end_headers(): a0f. call. + + Changed in version 3.3: Headers are stored to an internal + buffer and *note end_headers(): a0f. needs to be called + explicitly. + + -- Method: send_header (keyword, value) + + Adds the HTTP header to an internal buffer which will be + written to the output stream when either *note end_headers(): + a0f. or *note flush_headers(): a10. is invoked. `keyword' + should specify the header keyword, with `value' specifying its + value. Note that, after the send_header calls are done, *note + end_headers(): a0f. MUST BE called in order to complete the + operation. + + Changed in version 3.2: Headers are stored in an internal + buffer. + + -- Method: send_response_only (code, message=None) + + Sends the response header only, used for the purposes when + ‘100 Continue’ response is sent by the server to the client. + The headers not buffered and sent directly the output + stream.If the `message' is not specified, the HTTP message + corresponding the response `code' is sent. + + New in version 3.2. + + -- Method: end_headers () + + Adds a blank line (indicating the end of the HTTP headers in + the response) to the headers buffer and calls *note + flush_headers(): a10. + + Changed in version 3.2: The buffered headers are written to + the output stream. + + -- Method: flush_headers () + + Finally send the headers to the output stream and flush the + internal headers buffer. + + New in version 3.3. + + -- Method: log_request (code='-', size='-') + + Logs an accepted (successful) request. `code' should specify + the numeric HTTP code associated with the response. If a size + of the response is available, then it should be passed as the + `size' parameter. + + -- Method: log_error (...) + + Logs an error when a request cannot be fulfilled. By default, + it passes the message to *note log_message(): 2b5e, so it + takes the same arguments (`format' and additional values). + + -- Method: log_message (format, ...) + + Logs an arbitrary message to ‘sys.stderr’. This is typically + overridden to create custom error logging mechanisms. The + `format' argument is a standard printf-style format string, + where the additional arguments to *note log_message(): 2b5e. + are applied as inputs to the formatting. The client ip + address and current date and time are prefixed to every + message logged. + + -- Method: version_string () + + Returns the server software’s version string. This is a + combination of the *note server_version: 2b50. and *note + sys_version: 2b51. attributes. + + -- Method: date_time_string (timestamp=None) + + Returns the date and time given by `timestamp' (which must be + ‘None’ or in the format returned by *note time.time(): 31d.), + formatted for a message header. If `timestamp' is omitted, it + uses the current date and time. + + The result looks like ‘'Sun, 06 Nov 1994 08:49:37 GMT'’. + + -- Method: log_date_time_string () + + Returns the current date and time, formatted for logging. + + -- Method: address_string () + + Returns the client address. + + Changed in version 3.3: Previously, a name lookup was + performed. To avoid name resolution delays, it now always + returns the IP address. + + -- Class: http.server.SimpleHTTPRequestHandler (request, + client_address, server, directory=None) + + This class serves files from the current directory and below, + directly mapping the directory structure to HTTP requests. + + A lot of the work, such as parsing the request, is done by the base + class *note BaseHTTPRequestHandler: a0e. This class implements the + *note do_GET(): 2b61. and *note do_HEAD(): 2b62. functions. + + The following are defined as class-level attributes of *note + SimpleHTTPRequestHandler: 395.: + + -- Attribute: server_version + + This will be ‘"SimpleHTTP/" + __version__’, where + ‘__version__’ is defined at the module level. + + -- Attribute: extensions_map + + A dictionary mapping suffixes into MIME types. The default is + signified by an empty string, and is considered to be + ‘application/octet-stream’. The mapping is used + case-insensitively, and so should contain only lower-cased + keys. + + -- Attribute: directory + + If not specified, the directory to serve is the current + working directory. + + The *note SimpleHTTPRequestHandler: 395. class defines the + following methods: + + -- Method: do_HEAD () + + This method serves the ‘'HEAD'’ request type: it sends the + headers it would send for the equivalent ‘GET’ request. See + the *note do_GET(): 2b61. method for a more complete + explanation of the possible headers. + + -- Method: do_GET () + + The request is mapped to a local file by interpreting the + request as a path relative to the current working directory. + + If the request was mapped to a directory, the directory is + checked for a file named ‘index.html’ or ‘index.htm’ (in that + order). If found, the file’s contents are returned; otherwise + a directory listing is generated by calling the + ‘list_directory()’ method. This method uses *note + os.listdir(): a41. to scan the directory, and returns a ‘404’ + error response if the *note listdir(): a41. fails. + + If the request was mapped to a file, it is opened. Any *note + OSError: 1d3. exception in opening the requested file is + mapped to a ‘404’, ‘'File not found'’ error. If there was a + ‘'If-Modified-Since'’ header in the request, and the file was + not modified after this time, a ‘304’, ‘'Not Modified'’ + response is sent. Otherwise, the content type is guessed by + calling the ‘guess_type()’ method, which in turn uses the + `extensions_map' variable, and the file contents are returned. + + A ‘'Content-type:'’ header with the guessed content type is + output, followed by a ‘'Content-Length:'’ header with the + file’s size and a ‘'Last-Modified:'’ header with the file’s + modification time. + + Then follows a blank line signifying the end of the headers, + and then the contents of the file are output. If the file’s + MIME type starts with ‘text/’ the file is opened in text mode; + otherwise binary mode is used. + + For example usage, see the implementation of the *note test(): + 105. function invocation in the *note http.server: 97. module. + + Changed in version 3.7: Support of the ‘'If-Modified-Since'’ + header. + +The *note SimpleHTTPRequestHandler: 395. class can be used in the +following manner in order to create a very basic webserver serving files +relative to the current directory: + + import http.server + import socketserver + + PORT = 8000 + + Handler = http.server.SimpleHTTPRequestHandler + + with socketserver.TCPServer(("", PORT), Handler) as httpd: + print("serving at port", PORT) + httpd.serve_forever() +*note http.server: 97. can also be invoked directly using the *note -m: +337. switch of the interpreter with a ‘port number’ argument. Similar +to the previous example, this serves files relative to the current +directory: + + python -m http.server 8000 + +By default, server binds itself to all interfaces. The option +‘-b/--bind’ specifies a specific address to which it should bind. Both +IPv4 and IPv6 addresses are supported. For example, the following +command causes the server to bind to localhost only: + + python -m http.server 8000 --bind 127.0.0.1 + +New in version 3.4: ‘--bind’ argument was introduced. + +New in version 3.8: ‘--bind’ argument enhanced to support IPv6 + +By default, server uses the current directory. The option +‘-d/--directory’ specifies a directory to which it should serve the +files. For example, the following command uses a specific directory: + + python -m http.server --directory /tmp/ + +New in version 3.7: ‘--directory’ specify alternate directory + + -- Class: http.server.CGIHTTPRequestHandler (request, client_address, + server) + + This class is used to serve either files or output of CGI scripts + from the current directory and below. Note that mapping HTTP + hierarchic structure to local directory structure is exactly as in + *note SimpleHTTPRequestHandler: 395. + + Note: CGI scripts run by the *note CGIHTTPRequestHandler: + 2b66. class cannot execute redirects (HTTP code 302), because + code 200 (script output follows) is sent prior to execution of + the CGI script. This pre-empts the status code. + + The class will however, run the CGI script, instead of serving it + as a file, if it guesses it to be a CGI script. Only + directory-based CGI are used — the other common server + configuration is to treat special extensions as denoting CGI + scripts. + + The ‘do_GET()’ and ‘do_HEAD()’ functions are modified to run CGI + scripts and serve the output, instead of serving files, if the + request leads to somewhere below the ‘cgi_directories’ path. + + The *note CGIHTTPRequestHandler: 2b66. defines the following data + member: + + -- Attribute: cgi_directories + + This defaults to ‘['/cgi-bin', '/htbin']’ and describes + directories to treat as containing CGI scripts. + + The *note CGIHTTPRequestHandler: 2b66. defines the following + method: + + -- Method: do_POST () + + This method serves the ‘'POST'’ request type, only allowed for + CGI scripts. Error 501, “Can only POST to CGI scripts”, is + output when trying to POST to a non-CGI url. + + Note that CGI scripts will be run with UID of user nobody, for + security reasons. Problems with the CGI script will be translated + to error 403. + +*note CGIHTTPRequestHandler: 2b66. can be enabled in the command line by +passing the ‘--cgi’ option: + + python -m http.server --cgi 8000 + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/http/server.py + + (2) https://tools.ietf.org/html/rfc2822.html + + +File: python.info, Node: http cookies — HTTP state management, Next: http cookiejar — Cookie handling for HTTP clients, Prev: http server — HTTP servers, Up: Internet Protocols and Support + +5.21.23 ‘http.cookies’ — HTTP state management +---------------------------------------------- + +`Source code:' Lib/http/cookies.py(1) + +__________________________________________________________________ + +The *note http.cookies: 96. module defines classes for abstracting the +concept of cookies, an HTTP state management mechanism. It supports +both simple string-only cookies, and provides an abstraction for having +any serializable data-type as cookie value. + +The module formerly strictly applied the parsing rules described in the +RFC 2109(2) and RFC 2068(3) specifications. It has since been +discovered that MSIE 3.0x doesn’t follow the character rules outlined in +those specs and also many current day browsers and servers have relaxed +parsing rules when comes to Cookie handling. As a result, the parsing +rules used are a bit less strict. + +The character set, *note string.ascii_letters: c5c, *note string.digits: +13cd. and ‘!#$%&'*+-.^_`|~:’ denote the set of valid characters allowed +by this module in Cookie name (as *note key: 48d.). + +Changed in version 3.3: Allowed ‘:’ as a valid Cookie name character. + + Note: On encountering an invalid cookie, *note CookieError: 2b6b. + is raised, so if your cookie data comes from a browser you should + always prepare for invalid data and catch *note CookieError: 2b6b. + on parsing. + + -- Exception: http.cookies.CookieError + + Exception failing because of RFC 2109(4) invalidity: incorrect + attributes, incorrect ‘Set-Cookie’ header, etc. + + -- Class: http.cookies.BaseCookie ([input]) + + This class is a dictionary-like object whose keys are strings and + whose values are *note Morsel: 490. instances. Note that upon + setting a key to a value, the value is first converted to a *note + Morsel: 490. containing the key and the value. + + If `input' is given, it is passed to the *note load(): 2b6d. + method. + + -- Class: http.cookies.SimpleCookie ([input]) + + This class derives from *note BaseCookie: 2b6c. and overrides + ‘value_decode()’ and ‘value_encode()’. SimpleCookie supports + strings as cookie values. When setting the value, SimpleCookie + calls the builtin *note str(): 330. to convert the value to a + string. Values received from HTTP are kept as strings. + +See also +........ + +Module *note http.cookiejar: 95. + + HTTP cookie handling for web `clients'. The *note http.cookiejar: + 95. and *note http.cookies: 96. modules do not depend on each + other. + +RFC 2109(5) - HTTP State Management Mechanism + + This is the state management specification implemented by this + module. + +* Menu: + +* Cookie Objects:: +* Morsel Objects:: +* Example: Example<14>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/http/cookies.py + + (2) https://tools.ietf.org/html/rfc2109.html + + (3) https://tools.ietf.org/html/rfc2068.html + + (4) https://tools.ietf.org/html/rfc2109.html + + (5) https://tools.ietf.org/html/rfc2109.html + + +File: python.info, Node: Cookie Objects, Next: Morsel Objects, Up: http cookies — HTTP state management + +5.21.23.1 Cookie Objects +........................ + + -- Method: BaseCookie.value_decode (val) + + Return a tuple ‘(real_value, coded_value)’ from a string + representation. ‘real_value’ can be any type. This method does no + decoding in *note BaseCookie: 2b6c. — it exists so it can be + overridden. + + -- Method: BaseCookie.value_encode (val) + + Return a tuple ‘(real_value, coded_value)’. `val' can be any type, + but ‘coded_value’ will always be converted to a string. This + method does no encoding in *note BaseCookie: 2b6c. — it exists so + it can be overridden. + + In general, it should be the case that *note value_encode(): 2b72. + and *note value_decode(): 2b71. are inverses on the range of + `value_decode'. + + -- Method: BaseCookie.output (attrs=None, header='Set-Cookie:', + sep='\r\n') + + Return a string representation suitable to be sent as HTTP headers. + `attrs' and `header' are sent to each *note Morsel: 490.’s *note + output(): 2b73. method. `sep' is used to join the headers + together, and is by default the combination ‘'\r\n'’ (CRLF). + + -- Method: BaseCookie.js_output (attrs=None) + + Return an embeddable JavaScript snippet, which, if run on a browser + which supports JavaScript, will act the same as if the HTTP headers + was sent. + + The meaning for `attrs' is the same as in *note output(): 2b73. + + -- Method: BaseCookie.load (rawdata) + + If `rawdata' is a string, parse it as an ‘HTTP_COOKIE’ and add the + values found there as *note Morsel: 490.s. If it is a dictionary, + it is equivalent to: + + for k, v in rawdata.items(): + cookie[k] = v + + +File: python.info, Node: Morsel Objects, Next: Example<14>, Prev: Cookie Objects, Up: http cookies — HTTP state management + +5.21.23.2 Morsel Objects +........................ + + -- Class: http.cookies.Morsel + + Abstract a key/value pair, which has some RFC 2109(1) attributes. + + Morsels are dictionary-like objects, whose set of keys is constant + — the valid RFC 2109(2) attributes, which are + + * ‘expires’ + + * ‘path’ + + * ‘comment’ + + * ‘domain’ + + * ‘max-age’ + + * ‘secure’ + + * ‘version’ + + * ‘httponly’ + + * ‘samesite’ + + The attribute ‘httponly’ specifies that the cookie is only + transferred in HTTP requests, and is not accessible through + JavaScript. This is intended to mitigate some forms of cross-site + scripting. + + The attribute ‘samesite’ specifies that the browser is not allowed + to send the cookie along with cross-site requests. This helps to + mitigate CSRF attacks. Valid values for this attribute are + “Strict” and “Lax”. + + The keys are case-insensitive and their default value is ‘''’. + + Changed in version 3.5: ‘__eq__()’ now takes *note key: 48d. and + *note value: 48e. into account. + + Changed in version 3.7: Attributes *note key: 48d, *note value: + 48e. and *note coded_value: 48f. are read-only. Use *note set(): + 491. for setting them. + + Changed in version 3.8: Added support for the ‘samesite’ attribute. + + -- Attribute: Morsel.value + + The value of the cookie. + + -- Attribute: Morsel.coded_value + + The encoded value of the cookie — this is what should be sent. + + -- Attribute: Morsel.key + + The name of the cookie. + + -- Method: Morsel.set (key, value, coded_value) + + Set the `key', `value' and `coded_value' attributes. + + -- Method: Morsel.isReservedKey (K) + + Whether `K' is a member of the set of keys of a *note Morsel: 490. + + -- Method: Morsel.output (attrs=None, header='Set-Cookie:') + + Return a string representation of the Morsel, suitable to be sent + as an HTTP header. By default, all the attributes are included, + unless `attrs' is given, in which case it should be a list of + attributes to use. `header' is by default ‘"Set-Cookie:"’. + + -- Method: Morsel.js_output (attrs=None) + + Return an embeddable JavaScript snippet, which, if run on a browser + which supports JavaScript, will act the same as if the HTTP header + was sent. + + The meaning for `attrs' is the same as in *note output(): 2b78. + + -- Method: Morsel.OutputString (attrs=None) + + Return a string representing the Morsel, without any surrounding + HTTP or JavaScript. + + The meaning for `attrs' is the same as in *note output(): 2b78. + + -- Method: Morsel.update (values) + + Update the values in the Morsel dictionary with the values in the + dictionary `values'. Raise an error if any of the keys in the + `values' dict is not a valid RFC 2109(3) attribute. + + Changed in version 3.5: an error is raised for invalid keys. + + -- Method: Morsel.copy (value) + + Return a shallow copy of the Morsel object. + + Changed in version 3.5: return a Morsel object instead of a dict. + + -- Method: Morsel.setdefault (key, value=None) + + Raise an error if key is not a valid RFC 2109(4) attribute, + otherwise behave the same as *note dict.setdefault(): 9bf. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2109.html + + (2) https://tools.ietf.org/html/rfc2109.html + + (3) https://tools.ietf.org/html/rfc2109.html + + (4) https://tools.ietf.org/html/rfc2109.html + + +File: python.info, Node: Example<14>, Prev: Morsel Objects, Up: http cookies — HTTP state management + +5.21.23.3 Example +................. + +The following example demonstrates how to use the *note http.cookies: +96. module. + + >>> from http import cookies + >>> C = cookies.SimpleCookie() + >>> C["fig"] = "newton" + >>> C["sugar"] = "wafer" + >>> print(C) # generate HTTP headers + Set-Cookie: fig=newton + Set-Cookie: sugar=wafer + >>> print(C.output()) # same thing + Set-Cookie: fig=newton + Set-Cookie: sugar=wafer + >>> C = cookies.SimpleCookie() + >>> C["rocky"] = "road" + >>> C["rocky"]["path"] = "/cookie" + >>> print(C.output(header="Cookie:")) + Cookie: rocky=road; Path=/cookie + >>> print(C.output(attrs=[], header="Cookie:")) + Cookie: rocky=road + >>> C = cookies.SimpleCookie() + >>> C.load("chips=ahoy; vienna=finger") # load from a string (HTTP header) + >>> print(C) + Set-Cookie: chips=ahoy + Set-Cookie: vienna=finger + >>> C = cookies.SimpleCookie() + >>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";') + >>> print(C) + Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=\012;" + >>> C = cookies.SimpleCookie() + >>> C["oreo"] = "doublestuff" + >>> C["oreo"]["path"] = "/" + >>> print(C) + Set-Cookie: oreo=doublestuff; Path=/ + >>> C = cookies.SimpleCookie() + >>> C["twix"] = "none for you" + >>> C["twix"].value + 'none for you' + >>> C = cookies.SimpleCookie() + >>> C["number"] = 7 # equivalent to C["number"] = str(7) + >>> C["string"] = "seven" + >>> C["number"].value + '7' + >>> C["string"].value + 'seven' + >>> print(C) + Set-Cookie: number=7 + Set-Cookie: string=seven + + +File: python.info, Node: http cookiejar — Cookie handling for HTTP clients, Next: xmlrpc — XMLRPC server and client modules, Prev: http cookies — HTTP state management, Up: Internet Protocols and Support + +5.21.24 ‘http.cookiejar’ — Cookie handling for HTTP clients +----------------------------------------------------------- + +`Source code:' Lib/http/cookiejar.py(1) + +__________________________________________________________________ + +The *note http.cookiejar: 95. module defines classes for automatic +handling of HTTP cookies. It is useful for accessing web sites that +require small pieces of data – `cookies' – to be set on the client +machine by an HTTP response from a web server, and then returned to the +server in later HTTP requests. + +Both the regular Netscape cookie protocol and the protocol defined by +RFC 2965(2) are handled. RFC 2965 handling is switched off by default. +RFC 2109(3) cookies are parsed as Netscape cookies and subsequently +treated either as Netscape or RFC 2965 cookies according to the ‘policy’ +in effect. Note that the great majority of cookies on the Internet are +Netscape cookies. *note http.cookiejar: 95. attempts to follow the +de-facto Netscape cookie protocol (which differs substantially from that +set out in the original Netscape specification), including taking note +of the ‘max-age’ and ‘port’ cookie-attributes introduced with RFC 2965. + + Note: The various named parameters found in ‘Set-Cookie’ and + ‘Set-Cookie2’ headers (eg. ‘domain’ and ‘expires’) are + conventionally referred to as `attributes'. To distinguish them + from Python attributes, the documentation for this module uses the + term `cookie-attribute' instead. + +The module defines the following exception: + + -- Exception: http.cookiejar.LoadError + + Instances of *note FileCookieJar: 2b81. raise this exception on + failure to load cookies from a file. *note LoadError: 2b80. is a + subclass of *note OSError: 1d3. + + Changed in version 3.3: LoadError was made a subclass of *note + OSError: 1d3. instead of *note IOError: 992. + +The following classes are provided: + + -- Class: http.cookiejar.CookieJar (policy=None) + + `policy' is an object implementing the *note CookiePolicy: 2b82. + interface. + + The *note CookieJar: 297c. class stores HTTP cookies. It extracts + cookies from HTTP requests, and returns them in HTTP responses. + *note CookieJar: 297c. instances automatically expire contained + cookies when necessary. Subclasses are also responsible for + storing and retrieving cookies from a file or database. + + -- Class: http.cookiejar.FileCookieJar (filename, delayload=None, + policy=None) + + `policy' is an object implementing the *note CookiePolicy: 2b82. + interface. For the other arguments, see the documentation for the + corresponding attributes. + + A *note CookieJar: 297c. which can load cookies from, and perhaps + save cookies to, a file on disk. Cookies are `NOT' loaded from the + named file until either the *note load(): 2b83. or *note revert(): + 2b84. method is called. Subclasses of this class are documented in + section *note FileCookieJar subclasses and co-operation with web + browsers: 2b85. + + Changed in version 3.8: The filename parameter supports a *note + path-like object: 36a. + + -- Class: http.cookiejar.CookiePolicy + + This class is responsible for deciding whether each cookie should + be accepted from / returned to the server. + + -- Class: http.cookiejar.DefaultCookiePolicy (blocked_domains=None, + allowed_domains=None, netscape=True, rfc2965=False, + rfc2109_as_netscape=None, hide_cookie2=False, + strict_domain=False, strict_rfc2965_unverifiable=True, + strict_ns_unverifiable=False, + strict_ns_domain=DefaultCookiePolicy.DomainLiberal, + strict_ns_set_initial_dollar=False, strict_ns_set_path=False, + secure_protocols=("https", "wss")) + + Constructor arguments should be passed as keyword arguments only. + `blocked_domains' is a sequence of domain names that we never + accept cookies from, nor return cookies to. `allowed_domains' if + not *note None: 157, this is a sequence of the only domains for + which we accept and return cookies. `secure_protocols' is a + sequence of protocols for which secure cookies can be added to. By + default `https' and `wss' (secure websocket) are considered secure + protocols. For all other arguments, see the documentation for + *note CookiePolicy: 2b82. and *note DefaultCookiePolicy: 2b86. + objects. + + *note DefaultCookiePolicy: 2b86. implements the standard accept / + reject rules for Netscape and RFC 2965(4) cookies. By default, RFC + 2109(5) cookies (ie. cookies received in a ‘Set-Cookie’ header + with a version cookie-attribute of 1) are treated according to the + RFC 2965 rules. However, if RFC 2965 handling is turned off or + *note rfc2109_as_netscape: 2b87. is ‘True’, RFC 2109 cookies are + ‘downgraded’ by the *note CookieJar: 297c. instance to Netscape + cookies, by setting the ‘version’ attribute of the *note Cookie: + 2b88. instance to 0. *note DefaultCookiePolicy: 2b86. also + provides some parameters to allow some fine-tuning of policy. + + -- Class: http.cookiejar.Cookie + + This class represents Netscape, RFC 2109(6) and RFC 2965(7) + cookies. It is not expected that users of *note http.cookiejar: + 95. construct their own *note Cookie: 2b88. instances. Instead, if + necessary, call ‘make_cookies()’ on a *note CookieJar: 297c. + instance. + +See also +........ + +Module *note urllib.request: 120. + + URL opening with automatic cookie handling. + +Module *note http.cookies: 96. + + HTTP cookie classes, principally useful for server-side code. The + *note http.cookiejar: 95. and *note http.cookies: 96. modules do + not depend on each other. + +‘https://curl.haxx.se/rfc/cookie_spec.html’ + + The specification of the original Netscape cookie protocol. Though + this is still the dominant protocol, the ‘Netscape cookie protocol’ + implemented by all the major browsers (and *note http.cookiejar: + 95.) only bears a passing resemblance to the one sketched out in + ‘cookie_spec.html’. + +RFC 2109(8) - HTTP State Management Mechanism + + Obsoleted by RFC 2965(9). Uses ‘Set-Cookie’ with version=1. + +RFC 2965(10) - HTTP State Management Mechanism + + The Netscape protocol with the bugs fixed. Uses ‘Set-Cookie2’ in + place of ‘Set-Cookie’. Not widely used. + +‘http://kristol.org/cookie/errata.html’ + + Unfinished errata to RFC 2965(11). + +RFC 2964(12) - Use of HTTP State Management + +* Menu: + +* CookieJar and FileCookieJar Objects:: +* FileCookieJar subclasses and co-operation with web browsers:: +* CookiePolicy Objects:: +* DefaultCookiePolicy Objects:: +* Cookie Objects: Cookie Objects<2>. +* Examples: Examples<24>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/http/cookiejar.py + + (2) https://tools.ietf.org/html/rfc2965.html + + (3) https://tools.ietf.org/html/rfc2109.html + + (4) https://tools.ietf.org/html/rfc2965.html + + (5) https://tools.ietf.org/html/rfc2109.html + + (6) https://tools.ietf.org/html/rfc2109.html + + (7) https://tools.ietf.org/html/rfc2965.html + + (8) https://tools.ietf.org/html/rfc2109.html + + (9) https://tools.ietf.org/html/rfc2965.html + + (10) https://tools.ietf.org/html/rfc2965.html + + (11) https://tools.ietf.org/html/rfc2965.html + + (12) https://tools.ietf.org/html/rfc2964.html + + +File: python.info, Node: CookieJar and FileCookieJar Objects, Next: FileCookieJar subclasses and co-operation with web browsers, Up: http cookiejar — Cookie handling for HTTP clients + +5.21.24.1 CookieJar and FileCookieJar Objects +............................................. + +*note CookieJar: 297c. objects support the *note iterator: 112e. +protocol for iterating over contained *note Cookie: 2b88. objects. + +*note CookieJar: 297c. has the following methods: + + -- Method: CookieJar.add_cookie_header (request) + + Add correct ‘Cookie’ header to `request'. + + If policy allows (ie. the ‘rfc2965’ and ‘hide_cookie2’ attributes + of the *note CookieJar: 297c.’s *note CookiePolicy: 2b82. instance + are true and false respectively), the ‘Cookie2’ header is also + added when appropriate. + + The `request' object (usually a *note urllib.request.Request: 8f6. + instance) must support the methods ‘get_full_url()’, ‘get_host()’, + ‘get_type()’, ‘unverifiable()’, ‘has_header()’, ‘get_header()’, + ‘header_items()’, ‘add_unredirected_header()’ and ‘origin_req_host’ + attribute as documented by *note urllib.request: 120. + + Changed in version 3.3: `request' object needs ‘origin_req_host’ + attribute. Dependency on a deprecated method + ‘get_origin_req_host()’ has been removed. + + -- Method: CookieJar.extract_cookies (response, request) + + Extract cookies from HTTP `response' and store them in the *note + CookieJar: 297c, where allowed by policy. + + The *note CookieJar: 297c. will look for allowable ‘Set-Cookie’ and + ‘Set-Cookie2’ headers in the `response' argument, and store cookies + as appropriate (subject to the *note CookiePolicy.set_ok(): 2b8d. + method’s approval). + + The `response' object (usually the result of a call to *note + urllib.request.urlopen(): 78c, or similar) should support an + ‘info()’ method, which returns an *note email.message.Message: 541. + instance. + + The `request' object (usually a *note urllib.request.Request: 8f6. + instance) must support the methods ‘get_full_url()’, ‘get_host()’, + ‘unverifiable()’, and ‘origin_req_host’ attribute, as documented by + *note urllib.request: 120. The request is used to set default + values for cookie-attributes as well as for checking that the + cookie is allowed to be set. + + Changed in version 3.3: `request' object needs ‘origin_req_host’ + attribute. Dependency on a deprecated method + ‘get_origin_req_host()’ has been removed. + + -- Method: CookieJar.set_policy (policy) + + Set the *note CookiePolicy: 2b82. instance to be used. + + -- Method: CookieJar.make_cookies (response, request) + + Return sequence of *note Cookie: 2b88. objects extracted from + `response' object. + + See the documentation for *note extract_cookies(): 2b8c. for the + interfaces required of the `response' and `request' arguments. + + -- Method: CookieJar.set_cookie_if_ok (cookie, request) + + Set a *note Cookie: 2b88. if policy says it’s OK to do so. + + -- Method: CookieJar.set_cookie (cookie) + + Set a *note Cookie: 2b88, without checking with policy to see + whether or not it should be set. + + -- Method: CookieJar.clear ([domain[, path[, name]]]) + + Clear some cookies. + + If invoked without arguments, clear all cookies. If given a single + argument, only cookies belonging to that `domain' will be removed. + If given two arguments, cookies belonging to the specified `domain' + and URL `path' are removed. If given three arguments, then the + cookie with the specified `domain', `path' and `name' is removed. + + Raises *note KeyError: 2c7. if no matching cookie exists. + + -- Method: CookieJar.clear_session_cookies () + + Discard all session cookies. + + Discards all contained cookies that have a true ‘discard’ attribute + (usually because they had either no ‘max-age’ or ‘expires’ + cookie-attribute, or an explicit ‘discard’ cookie-attribute). For + interactive browsers, the end of a session usually corresponds to + closing the browser window. + + Note that the ‘save()’ method won’t save session cookies anyway, + unless you ask otherwise by passing a true `ignore_discard' + argument. + +*note FileCookieJar: 2b81. implements the following additional methods: + + -- Method: FileCookieJar.save (filename=None, ignore_discard=False, + ignore_expires=False) + + Save cookies to a file. + + This base class raises *note NotImplementedError: 60e. Subclasses + may leave this method unimplemented. + + `filename' is the name of file in which to save cookies. If + `filename' is not specified, ‘self.filename’ is used (whose default + is the value passed to the constructor, if any); if ‘self.filename’ + is *note None: 157, *note ValueError: 1fb. is raised. + + `ignore_discard': save even cookies set to be discarded. + `ignore_expires': save even cookies that have expired + + The file is overwritten if it already exists, thus wiping all the + cookies it contains. Saved cookies can be restored later using the + *note load(): 2b83. or *note revert(): 2b84. methods. + + -- Method: FileCookieJar.load (filename=None, ignore_discard=False, + ignore_expires=False) + + Load cookies from a file. + + Old cookies are kept unless overwritten by newly loaded ones. + + Arguments are as for *note save(): 2b94. + + The named file must be in the format understood by the class, or + *note LoadError: 2b80. will be raised. Also, *note OSError: 1d3. + may be raised, for example if the file does not exist. + + Changed in version 3.3: *note IOError: 992. used to be raised, it + is now an alias of *note OSError: 1d3. + + -- Method: FileCookieJar.revert (filename=None, ignore_discard=False, + ignore_expires=False) + + Clear all cookies and reload cookies from a saved file. + + *note revert(): 2b84. can raise the same exceptions as *note + load(): 2b83. If there is a failure, the object’s state will not + be altered. + +*note FileCookieJar: 2b81. instances have the following public +attributes: + + -- Attribute: FileCookieJar.filename + + Filename of default file in which to keep cookies. This attribute + may be assigned to. + + -- Attribute: FileCookieJar.delayload + + If true, load cookies lazily from disk. This attribute should not + be assigned to. This is only a hint, since this only affects + performance, not behaviour (unless the cookies on disk are + changing). A *note CookieJar: 297c. object may ignore it. None of + the *note FileCookieJar: 2b81. classes included in the standard + library lazily loads cookies. + + +File: python.info, Node: FileCookieJar subclasses and co-operation with web browsers, Next: CookiePolicy Objects, Prev: CookieJar and FileCookieJar Objects, Up: http cookiejar — Cookie handling for HTTP clients + +5.21.24.2 FileCookieJar subclasses and co-operation with web browsers +..................................................................... + +The following *note CookieJar: 297c. subclasses are provided for reading +and writing. + + -- Class: http.cookiejar.MozillaCookieJar (filename, delayload=None, + policy=None) + + A *note FileCookieJar: 2b81. that can load from and save cookies to + disk in the Mozilla ‘cookies.txt’ file format (which is also used + by the Lynx and Netscape browsers). + + Note: This loses information about RFC 2965(1) cookies, and + also about newer or non-standard cookie-attributes such as + ‘port’. + + Warning: Back up your cookies before saving if you have + cookies whose loss / corruption would be inconvenient (there + are some subtleties which may lead to slight changes in the + file over a load / save round-trip). + + Also note that cookies saved while Mozilla is running will get + clobbered by Mozilla. + + -- Class: http.cookiejar.LWPCookieJar (filename, delayload=None, + policy=None) + + A *note FileCookieJar: 2b81. that can load from and save cookies to + disk in format compatible with the libwww-perl library’s + ‘Set-Cookie3’ file format. This is convenient if you want to store + cookies in a human-readable file. + + Changed in version 3.8: The filename parameter supports a *note + path-like object: 36a. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2965.html + + +File: python.info, Node: CookiePolicy Objects, Next: DefaultCookiePolicy Objects, Prev: FileCookieJar subclasses and co-operation with web browsers, Up: http cookiejar — Cookie handling for HTTP clients + +5.21.24.3 CookiePolicy Objects +.............................. + +Objects implementing the *note CookiePolicy: 2b82. interface have the +following methods: + + -- Method: CookiePolicy.set_ok (cookie, request) + + Return boolean value indicating whether cookie should be accepted + from server. + + `cookie' is a *note Cookie: 2b88. instance. `request' is an object + implementing the interface defined by the documentation for *note + CookieJar.extract_cookies(): 2b8c. + + -- Method: CookiePolicy.return_ok (cookie, request) + + Return boolean value indicating whether cookie should be returned + to server. + + `cookie' is a *note Cookie: 2b88. instance. `request' is an object + implementing the interface defined by the documentation for *note + CookieJar.add_cookie_header(): 2b8b. + + -- Method: CookiePolicy.domain_return_ok (domain, request) + + Return ‘False’ if cookies should not be returned, given cookie + domain. + + This method is an optimization. It removes the need for checking + every cookie with a particular domain (which might involve reading + many files). Returning true from *note domain_return_ok(): 2b9d. + and *note path_return_ok(): 2b9e. leaves all the work to *note + return_ok(): 2b9c. + + If *note domain_return_ok(): 2b9d. returns true for the cookie + domain, *note path_return_ok(): 2b9e. is called for the cookie + path. Otherwise, *note path_return_ok(): 2b9e. and *note + return_ok(): 2b9c. are never called for that cookie domain. If + *note path_return_ok(): 2b9e. returns true, *note return_ok(): + 2b9c. is called with the *note Cookie: 2b88. object itself for a + full check. Otherwise, *note return_ok(): 2b9c. is never called + for that cookie path. + + Note that *note domain_return_ok(): 2b9d. is called for every + `cookie' domain, not just for the `request' domain. For example, + the function might be called with both ‘".example.com"’ and + ‘"www.example.com"’ if the request domain is ‘"www.example.com"’. + The same goes for *note path_return_ok(): 2b9e. + + The `request' argument is as documented for *note return_ok(): + 2b9c. + + -- Method: CookiePolicy.path_return_ok (path, request) + + Return ‘False’ if cookies should not be returned, given cookie + path. + + See the documentation for *note domain_return_ok(): 2b9d. + +In addition to implementing the methods above, implementations of the +*note CookiePolicy: 2b82. interface must also supply the following +attributes, indicating which protocols should be used, and how. All of +these attributes may be assigned to. + + -- Attribute: CookiePolicy.netscape + + Implement Netscape protocol. + + -- Attribute: CookiePolicy.rfc2965 + + Implement RFC 2965(1) protocol. + + -- Attribute: CookiePolicy.hide_cookie2 + + Don’t add ‘Cookie2’ header to requests (the presence of this header + indicates to the server that we understand RFC 2965(2) cookies). + +The most useful way to define a *note CookiePolicy: 2b82. class is by +subclassing from *note DefaultCookiePolicy: 2b86. and overriding some or +all of the methods above. *note CookiePolicy: 2b82. itself may be used +as a ‘null policy’ to allow setting and receiving any and all cookies +(this is unlikely to be useful). + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2965.html + + (2) https://tools.ietf.org/html/rfc2965.html + + +File: python.info, Node: DefaultCookiePolicy Objects, Next: Cookie Objects<2>, Prev: CookiePolicy Objects, Up: http cookiejar — Cookie handling for HTTP clients + +5.21.24.4 DefaultCookiePolicy Objects +..................................... + +Implements the standard rules for accepting and returning cookies. + +Both RFC 2965(1) and Netscape cookies are covered. RFC 2965 handling is +switched off by default. + +The easiest way to provide your own policy is to override this class and +call its methods in your overridden implementations before adding your +own additional checks: + + import http.cookiejar + class MyCookiePolicy(http.cookiejar.DefaultCookiePolicy): + def set_ok(self, cookie, request): + if not http.cookiejar.DefaultCookiePolicy.set_ok(self, cookie, request): + return False + if i_dont_want_to_store_this_cookie(cookie): + return False + return True + +In addition to the features required to implement the *note +CookiePolicy: 2b82. interface, this class allows you to block and allow +domains from setting and receiving cookies. There are also some +strictness switches that allow you to tighten up the rather loose +Netscape protocol rules a little bit (at the cost of blocking some +benign cookies). + +A domain blacklist and whitelist is provided (both off by default). +Only domains not in the blacklist and present in the whitelist (if the +whitelist is active) participate in cookie setting and returning. Use +the `blocked_domains' constructor argument, and ‘blocked_domains()’ and +‘set_blocked_domains()’ methods (and the corresponding argument and +methods for `allowed_domains'). If you set a whitelist, you can turn it +off again by setting it to *note None: 157. + +Domains in block or allow lists that do not start with a dot must equal +the cookie domain to be matched. For example, ‘"example.com"’ matches a +blacklist entry of ‘"example.com"’, but ‘"www.example.com"’ does not. +Domains that do start with a dot are matched by more specific domains +too. For example, both ‘"www.example.com"’ and +‘"www.coyote.example.com"’ match ‘".example.com"’ (but ‘"example.com"’ +itself does not). IP addresses are an exception, and must match +exactly. For example, if blocked_domains contains ‘"192.168.1.2"’ and +‘".168.1.2"’, 192.168.1.2 is blocked, but 193.168.1.2 is not. + +*note DefaultCookiePolicy: 2b86. implements the following additional +methods: + + -- Method: DefaultCookiePolicy.blocked_domains () + + Return the sequence of blocked domains (as a tuple). + + -- Method: DefaultCookiePolicy.set_blocked_domains (blocked_domains) + + Set the sequence of blocked domains. + + -- Method: DefaultCookiePolicy.is_blocked (domain) + + Return whether `domain' is on the blacklist for setting or + receiving cookies. + + -- Method: DefaultCookiePolicy.allowed_domains () + + Return *note None: 157, or the sequence of allowed domains (as a + tuple). + + -- Method: DefaultCookiePolicy.set_allowed_domains (allowed_domains) + + Set the sequence of allowed domains, or *note None: 157. + + -- Method: DefaultCookiePolicy.is_not_allowed (domain) + + Return whether `domain' is not on the whitelist for setting or + receiving cookies. + +*note DefaultCookiePolicy: 2b86. instances have the following +attributes, which are all initialised from the constructor arguments of +the same name, and which may all be assigned to. + + -- Attribute: DefaultCookiePolicy.rfc2109_as_netscape + + If true, request that the *note CookieJar: 297c. instance downgrade + RFC 2109(2) cookies (ie. cookies received in a ‘Set-Cookie’ header + with a version cookie-attribute of 1) to Netscape cookies by + setting the version attribute of the *note Cookie: 2b88. instance + to 0. The default value is *note None: 157, in which case RFC 2109 + cookies are downgraded if and only if RFC 2965(3) handling is + turned off. Therefore, RFC 2109 cookies are downgraded by default. + +General strictness switches: + + -- Attribute: DefaultCookiePolicy.strict_domain + + Don’t allow sites to set two-component domains with country-code + top-level domains like ‘.co.uk’, ‘.gov.uk’, ‘.co.nz’.etc. This is + far from perfect and isn’t guaranteed to work! + +RFC 2965(4) protocol strictness switches: + + -- Attribute: DefaultCookiePolicy.strict_rfc2965_unverifiable + + Follow RFC 2965(5) rules on unverifiable transactions (usually, an + unverifiable transaction is one resulting from a redirect or a + request for an image hosted on another site). If this is false, + cookies are `never' blocked on the basis of verifiability + +Netscape protocol strictness switches: + + -- Attribute: DefaultCookiePolicy.strict_ns_unverifiable + + Apply RFC 2965(6) rules on unverifiable transactions even to + Netscape cookies. + + -- Attribute: DefaultCookiePolicy.strict_ns_domain + + Flags indicating how strict to be with domain-matching rules for + Netscape cookies. See below for acceptable values. + + -- Attribute: DefaultCookiePolicy.strict_ns_set_initial_dollar + + Ignore cookies in Set-Cookie: headers that have names starting with + ‘'$'’. + + -- Attribute: DefaultCookiePolicy.strict_ns_set_path + + Don’t allow setting cookies whose path doesn’t path-match request + URI. + +‘strict_ns_domain’ is a collection of flags. Its value is constructed +by or-ing together (for example, +‘DomainStrictNoDots|DomainStrictNonDomain’ means both flags are set). + + -- Attribute: DefaultCookiePolicy.DomainStrictNoDots + + When setting cookies, the ‘host prefix’ must not contain a dot (eg. + ‘www.foo.bar.com’ can’t set a cookie for ‘.bar.com’, because + ‘www.foo’ contains a dot). + + -- Attribute: DefaultCookiePolicy.DomainStrictNonDomain + + Cookies that did not explicitly specify a ‘domain’ cookie-attribute + can only be returned to a domain equal to the domain that set the + cookie (eg. ‘spam.example.com’ won’t be returned cookies from + ‘example.com’ that had no ‘domain’ cookie-attribute). + + -- Attribute: DefaultCookiePolicy.DomainRFC2965Match + + When setting cookies, require a full RFC 2965(7) domain-match. + +The following attributes are provided for convenience, and are the most +useful combinations of the above flags: + + -- Attribute: DefaultCookiePolicy.DomainLiberal + + Equivalent to 0 (ie. all of the above Netscape domain strictness + flags switched off). + + -- Attribute: DefaultCookiePolicy.DomainStrict + + Equivalent to ‘DomainStrictNoDots|DomainStrictNonDomain’. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2965.html + + (2) https://tools.ietf.org/html/rfc2109.html + + (3) https://tools.ietf.org/html/rfc2965.html + + (4) https://tools.ietf.org/html/rfc2965.html + + (5) https://tools.ietf.org/html/rfc2965.html + + (6) https://tools.ietf.org/html/rfc2965.html + + (7) https://tools.ietf.org/html/rfc2965.html + + +File: python.info, Node: Cookie Objects<2>, Next: Examples<24>, Prev: DefaultCookiePolicy Objects, Up: http cookiejar — Cookie handling for HTTP clients + +5.21.24.5 Cookie Objects +........................ + +*note Cookie: 2b88. instances have Python attributes roughly +corresponding to the standard cookie-attributes specified in the various +cookie standards. The correspondence is not one-to-one, because there +are complicated rules for assigning default values, because the +‘max-age’ and ‘expires’ cookie-attributes contain equivalent +information, and because RFC 2109(1) cookies may be ‘downgraded’ by +*note http.cookiejar: 95. from version 1 to version 0 (Netscape) +cookies. + +Assignment to these attributes should not be necessary other than in +rare circumstances in a *note CookiePolicy: 2b82. method. The class +does not enforce internal consistency, so you should know what you’re +doing if you do that. + + -- Attribute: Cookie.version + + Integer or *note None: 157. Netscape cookies have *note version: + 2bb6. 0. RFC 2965(2) and RFC 2109(3) cookies have a ‘version’ + cookie-attribute of 1. However, note that *note http.cookiejar: + 95. may ‘downgrade’ RFC 2109 cookies to Netscape cookies, in which + case *note version: 2bb6. is 0. + + -- Attribute: Cookie.name + + Cookie name (a string). + + -- Attribute: Cookie.value + + Cookie value (a string), or *note None: 157. + + -- Attribute: Cookie.port + + String representing a port or a set of ports (eg. ‘80’, or + ‘80,8080’), or *note None: 157. + + -- Attribute: Cookie.path + + Cookie path (a string, eg. ‘'/acme/rocket_launchers'’). + + -- Attribute: Cookie.secure + + ‘True’ if cookie should only be returned over a secure connection. + + -- Attribute: Cookie.expires + + Integer expiry date in seconds since epoch, or *note None: 157. + See also the *note is_expired(): 2bbd. method. + + -- Attribute: Cookie.discard + + ‘True’ if this is a session cookie. + + -- Attribute: Cookie.comment + + String comment from the server explaining the function of this + cookie, or *note None: 157. + + -- Attribute: Cookie.comment_url + + URL linking to a comment from the server explaining the function of + this cookie, or *note None: 157. + + -- Attribute: Cookie.rfc2109 + + ‘True’ if this cookie was received as an RFC 2109(4) cookie (ie. + the cookie arrived in a ‘Set-Cookie’ header, and the value of the + Version cookie-attribute in that header was 1). This attribute is + provided because *note http.cookiejar: 95. may ‘downgrade’ RFC 2109 + cookies to Netscape cookies, in which case *note version: 2bb6. is + 0. + + -- Attribute: Cookie.port_specified + + ‘True’ if a port or set of ports was explicitly specified by the + server (in the ‘Set-Cookie’ / ‘Set-Cookie2’ header). + + -- Attribute: Cookie.domain_specified + + ‘True’ if a domain was explicitly specified by the server. + + -- Attribute: Cookie.domain_initial_dot + + ‘True’ if the domain explicitly specified by the server began with + a dot (‘'.'’). + +Cookies may have additional non-standard cookie-attributes. These may +be accessed using the following methods: + + -- Method: Cookie.has_nonstandard_attr (name) + + Return ‘True’ if cookie has the named cookie-attribute. + + -- Method: Cookie.get_nonstandard_attr (name, default=None) + + If cookie has the named cookie-attribute, return its value. + Otherwise, return `default'. + + -- Method: Cookie.set_nonstandard_attr (name, value) + + Set the value of the named cookie-attribute. + +The *note Cookie: 2b88. class also defines the following method: + + -- Method: Cookie.is_expired (now=None) + + ‘True’ if cookie has passed the time at which the server requested + it should expire. If `now' is given (in seconds since the epoch), + return whether the cookie has expired at the specified time. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2109.html + + (2) https://tools.ietf.org/html/rfc2965.html + + (3) https://tools.ietf.org/html/rfc2109.html + + (4) https://tools.ietf.org/html/rfc2109.html + + +File: python.info, Node: Examples<24>, Prev: Cookie Objects<2>, Up: http cookiejar — Cookie handling for HTTP clients + +5.21.24.6 Examples +.................. + +The first example shows the most common usage of *note http.cookiejar: +95.: + + import http.cookiejar, urllib.request + cj = http.cookiejar.CookieJar() + opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) + r = opener.open("http://example.com/") + +This example illustrates how to open a URL using your Netscape, Mozilla, +or Lynx cookies (assumes Unix/Netscape convention for location of the +cookies file): + + import os, http.cookiejar, urllib.request + cj = http.cookiejar.MozillaCookieJar() + cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt")) + opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) + r = opener.open("http://example.com/") + +The next example illustrates the use of *note DefaultCookiePolicy: 2b86. +Turn on RFC 2965(1) cookies, be more strict about domains when setting +and returning Netscape cookies, and block some domains from setting +cookies or having them returned: + + import urllib.request + from http.cookiejar import CookieJar, DefaultCookiePolicy + policy = DefaultCookiePolicy( + rfc2965=True, strict_ns_domain=Policy.DomainStrict, + blocked_domains=["ads.net", ".ads.net"]) + cj = CookieJar(policy) + opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) + r = opener.open("http://example.com/") + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2965.html + + +File: python.info, Node: xmlrpc — XMLRPC server and client modules, Next: xmlrpc client — XML-RPC client access, Prev: http cookiejar — Cookie handling for HTTP clients, Up: Internet Protocols and Support + +5.21.25 ‘xmlrpc’ — XMLRPC server and client modules +--------------------------------------------------- + +XML-RPC is a Remote Procedure Call method that uses XML passed via HTTP +as a transport. With it, a client can call methods with parameters on a +remote server (the server is named by a URI) and get back structured +data. + +‘xmlrpc’ is a package that collects server and client modules +implementing XML-RPC. The modules are: + + * *note xmlrpc.client: 13f. + + * *note xmlrpc.server: 140. + + +File: python.info, Node: xmlrpc client — XML-RPC client access, Next: xmlrpc server — Basic XML-RPC servers, Prev: xmlrpc — XMLRPC server and client modules, Up: Internet Protocols and Support + +5.21.26 ‘xmlrpc.client’ — XML-RPC client access +----------------------------------------------- + +`Source code:' Lib/xmlrpc/client.py(1) + +__________________________________________________________________ + +XML-RPC is a Remote Procedure Call method that uses XML passed via +HTTP(S) as a transport. With it, a client can call methods with +parameters on a remote server (the server is named by a URI) and get +back structured data. This module supports writing XML-RPC client code; +it handles all the details of translating between conformable Python +objects and XML on the wire. + + Warning: The *note xmlrpc.client: 13f. module is not secure against + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see *note XML vulnerabilities: 26f5. + +Changed in version 3.5: For HTTPS URIs, *note xmlrpc.client: 13f. now +performs all the necessary certificate and hostname checks by default. + + -- Class: xmlrpc.client.ServerProxy (uri, transport=None, + encoding=None, verbose=False, allow_none=False, + use_datetime=False, use_builtin_types=False, *, headers=(), + context=None) + + A *note ServerProxy: 255. instance is an object that manages + communication with a remote XML-RPC server. The required first + argument is a URI (Uniform Resource Indicator), and will normally + be the URL of the server. The optional second argument is a + transport factory instance; by default it is an internal + ‘SafeTransport’ instance for https: URLs and an internal HTTP + ‘Transport’ instance otherwise. The optional third argument is an + encoding, by default UTF-8. The optional fourth argument is a + debugging flag. + + The following parameters govern the use of the returned proxy + instance. If `allow_none' is true, the Python constant ‘None’ will + be translated into XML; the default behaviour is for ‘None’ to + raise a *note TypeError: 192. This is a commonly-used extension to + the XML-RPC specification, but isn’t supported by all clients and + servers; see http://ontosys.com/xml-rpc/extensions.php(2) for a + description. The `use_builtin_types' flag can be used to cause + date/time values to be presented as *note datetime.datetime: 194. + objects and binary data to be presented as *note bytes: 331. + objects; this flag is false by default. *note datetime.datetime: + 194, *note bytes: 331. and *note bytearray: 332. objects may be + passed to calls. The `headers' parameter is an optional sequence + of HTTP headers to send with each request, expressed as a sequence + of 2-tuples representing the header name and value. (e.g. + ‘[(‘Header-Name’, ‘value’)]’). The obsolete `use_datetime' flag is + similar to `use_builtin_types' but it applies only to date/time + values. + +Changed in version 3.3: The `use_builtin_types' flag was added. + +Changed in version 3.8: The `headers' parameter was added. + +Both the HTTP and HTTPS transports support the URL syntax extension for +HTTP Basic Authentication: ‘http://user:pass@host:port/path’. The +‘user:pass’ portion will be base64-encoded as an HTTP ‘Authorization’ +header, and sent to the remote server as part of the connection process +when invoking an XML-RPC method. You only need to use this if the +remote server requires a Basic Authentication user and password. If an +HTTPS URL is provided, `context' may be *note ssl.SSLContext: 3e4. and +configures the SSL settings of the underlying HTTPS connection. + +The returned instance is a proxy object with methods that can be used to +invoke corresponding RPC calls on the remote server. If the remote +server supports the introspection API, the proxy can also be used to +query the remote server for the methods it supports (service discovery) +and fetch other server-associated metadata. + +Types that are conformable (e.g. that can be marshalled through XML), +include the following (and except where noted, they are unmarshalled as +the same Python type): + +XML-RPC type Python type + +--------------------------------------------------------------------------------------- + +‘boolean’ *note bool: 183. + + +‘int’, ‘i1’, ‘i2’, ‘i4’, *note int: 184. in range from -2147483648 to 2147483647. +‘i8’ or ‘biginteger’ Values get the ‘<int>’ tag. + + +‘double’ or ‘float’ *note float: 187. Values get the ‘<double>’ tag. + + +‘string’ *note str: 330. + + +‘array’ *note list: 262. or *note tuple: 47e. containing + conformable elements. Arrays are returned as + *note lists: 262. + + +‘struct’ *note dict: 1b8. Keys must be strings, values may be any + conformable type. Objects of user-defined classes can be + passed in; only their *note __dict__: 4fc. attribute is + transmitted. + + +‘dateTime.iso8601’ *note DateTime: 2bcd. or *note datetime.datetime: 194. + Returned type depends on values of `use_builtin_types' + and `use_datetime' flags. + + +‘base64’ *note Binary: 2bce, *note bytes: 331. or + *note bytearray: 332. Returned type depends on the value + of the `use_builtin_types' flag. + + +‘nil’ The ‘None’ constant. Passing is allowed only if + `allow_none' is true. + + +‘bigdecimal’ *note decimal.Decimal: 188. Returned type only. + + +This is the full set of data types supported by XML-RPC. Method calls +may also raise a special *note Fault: 2bcf. instance, used to signal +XML-RPC server errors, or *note ProtocolError: 2bd0. used to signal an +error in the HTTP/HTTPS transport layer. Both *note Fault: 2bcf. and +*note ProtocolError: 2bd0. derive from a base class called ‘Error’. +Note that the xmlrpc client module currently does not marshal instances +of subclasses of built-in types. + +When passing strings, characters special to XML such as ‘<’, ‘>’, and +‘&’ will be automatically escaped. However, it’s the caller’s +responsibility to ensure that the string is free of characters that +aren’t allowed in XML, such as the control characters with ASCII values +between 0 and 31 (except, of course, tab, newline and carriage return); +failing to do this will result in an XML-RPC request that isn’t +well-formed XML. If you have to pass arbitrary bytes via XML-RPC, use +*note bytes: 331. or *note bytearray: 332. classes or the *note Binary: +2bce. wrapper class described below. + +‘Server’ is retained as an alias for *note ServerProxy: 255. for +backwards compatibility. New code should use *note ServerProxy: 255. + +Changed in version 3.5: Added the `context' argument. + +Changed in version 3.6: Added support of type tags with prefixes (e.g. +‘ex:nil’). Added support of unmarshalling additional types used by +Apache XML-RPC implementation for numerics: ‘i1’, ‘i2’, ‘i8’, +‘biginteger’, ‘float’ and ‘bigdecimal’. See +‘http://ws.apache.org/xmlrpc/types.html’ for a description. + +See also +........ + +XML-RPC HOWTO(3) + + A good description of XML-RPC operation and client software in + several languages. Contains pretty much everything an XML-RPC + client developer needs to know. + +XML-RPC Introspection(4) + + Describes the XML-RPC protocol extension for introspection. + +XML-RPC Specification(5) + + The official specification. + +Unofficial XML-RPC Errata(6) + + Fredrik Lundh’s “unofficial errata, intended to clarify certain + details in the XML-RPC specification, as well as hint at ‘best + practices’ to use when designing your own XML-RPC implementations.” + +* Menu: + +* ServerProxy Objects:: +* DateTime Objects:: +* Binary Objects:: +* Fault Objects:: +* ProtocolError Objects:: +* MultiCall Objects:: +* Convenience Functions:: +* Example of Client Usage:: +* Example of Client and Server Usage:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/xmlrpc/client.py + + (2) +https://web.archive.org/web/20130120074804/http://ontosys.com/xml-rpc/extensions.php + + (3) http://www.tldp.org/HOWTO/XML-RPC-HOWTO/index.html + + (4) http://xmlrpc-c.sourceforge.net/introspection.html + + (5) http://xmlrpc.scripting.com/spec.html + + (6) http://effbot.org/zone/xmlrpc-errata.htm + + +File: python.info, Node: ServerProxy Objects, Next: DateTime Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.1 ServerProxy Objects +............................. + +A *note ServerProxy: 255. instance has a method corresponding to each +remote procedure call accepted by the XML-RPC server. Calling the +method performs an RPC, dispatched by both name and argument signature +(e.g. the same method name can be overloaded with multiple argument +signatures). The RPC finishes by returning a value, which may be either +returned data in a conformant type or a *note Fault: 2bcf. or *note +ProtocolError: 2bd0. object indicating an error. + +Servers that support the XML introspection API support some common +methods grouped under the reserved ‘system’ attribute: + + -- Method: ServerProxy.system.listMethods () + + This method returns a list of strings, one for each (non-system) + method supported by the XML-RPC server. + + -- Method: ServerProxy.system.methodSignature (name) + + This method takes one parameter, the name of a method implemented + by the XML-RPC server. It returns an array of possible signatures + for this method. A signature is an array of types. The first of + these types is the return type of the method, the rest are + parameters. + + Because multiple signatures (ie. overloading) is permitted, this + method returns a list of signatures rather than a singleton. + + Signatures themselves are restricted to the top level parameters + expected by a method. For instance if a method expects one array + of structs as a parameter, and it returns a string, its signature + is simply “string, array”. If it expects three integers and + returns a string, its signature is “string, int, int, int”. + + If no signature is defined for the method, a non-array value is + returned. In Python this means that the type of the returned value + will be something other than list. + + -- Method: ServerProxy.system.methodHelp (name) + + This method takes one parameter, the name of a method implemented + by the XML-RPC server. It returns a documentation string + describing the use of that method. If no such string is available, + an empty string is returned. The documentation string may contain + HTML markup. + +Changed in version 3.5: Instances of *note ServerProxy: 255. support the +*note context manager: 568. protocol for closing the underlying +transport. + +A working example follows. The server code: + + from xmlrpc.server import SimpleXMLRPCServer + + def is_even(n): + return n % 2 == 0 + + server = SimpleXMLRPCServer(("localhost", 8000)) + print("Listening on port 8000...") + server.register_function(is_even, "is_even") + server.serve_forever() + +The client code for the preceding server: + + import xmlrpc.client + + with xmlrpc.client.ServerProxy("http://localhost:8000/") as proxy: + print("3 is even: %s" % str(proxy.is_even(3))) + print("100 is even: %s" % str(proxy.is_even(100))) + + +File: python.info, Node: DateTime Objects, Next: Binary Objects, Prev: ServerProxy Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.2 DateTime Objects +.......................... + + -- Class: xmlrpc.client.DateTime + + This class may be initialized with seconds since the epoch, a time + tuple, an ISO 8601 time/date string, or a *note datetime.datetime: + 194. instance. It has the following methods, supported mainly for + internal use by the marshalling/unmarshalling code: + + -- Method: decode (string) + + Accept a string as the instance’s new time value. + + -- Method: encode (out) + + Write the XML-RPC encoding of this *note DateTime: 2bcd. item + to the `out' stream object. + + It also supports certain of Python’s built-in operators through + rich comparison and *note __repr__(): 33e. methods. + +A working example follows. The server code: + + import datetime + from xmlrpc.server import SimpleXMLRPCServer + import xmlrpc.client + + def today(): + today = datetime.datetime.today() + return xmlrpc.client.DateTime(today) + + server = SimpleXMLRPCServer(("localhost", 8000)) + print("Listening on port 8000...") + server.register_function(today, "today") + server.serve_forever() + +The client code for the preceding server: + + import xmlrpc.client + import datetime + + proxy = xmlrpc.client.ServerProxy("http://localhost:8000/") + + today = proxy.today() + # convert the ISO8601 string to a datetime object + converted = datetime.datetime.strptime(today.value, "%Y%m%dT%H:%M:%S") + print("Today: %s" % converted.strftime("%d.%m.%Y, %H:%M")) + + +File: python.info, Node: Binary Objects, Next: Fault Objects, Prev: DateTime Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.3 Binary Objects +........................ + + -- Class: xmlrpc.client.Binary + + This class may be initialized from bytes data (which may include + NULs). The primary access to the content of a *note Binary: 2bce. + object is provided by an attribute: + + -- Attribute: data + + The binary data encapsulated by the *note Binary: 2bce. + instance. The data is provided as a *note bytes: 331. object. + + *note Binary: 2bce. objects have the following methods, supported + mainly for internal use by the marshalling/unmarshalling code: + + -- Method: decode (bytes) + + Accept a base64 *note bytes: 331. object and decode it as the + instance’s new data. + + -- Method: encode (out) + + Write the XML-RPC base 64 encoding of this binary item to the + `out' stream object. + + The encoded data will have newlines every 76 characters as per + RFC 2045 section 6.8(1), which was the de facto standard + base64 specification when the XML-RPC spec was written. + + It also supports certain of Python’s built-in operators through + *note __eq__(): 33f. and *note __ne__(): e56. methods. + +Example usage of the binary objects. We’re going to transfer an image +over XMLRPC: + + from xmlrpc.server import SimpleXMLRPCServer + import xmlrpc.client + + def python_logo(): + with open("python_logo.jpg", "rb") as handle: + return xmlrpc.client.Binary(handle.read()) + + server = SimpleXMLRPCServer(("localhost", 8000)) + print("Listening on port 8000...") + server.register_function(python_logo, 'python_logo') + + server.serve_forever() + +The client gets the image and saves it to a file: + + import xmlrpc.client + + proxy = xmlrpc.client.ServerProxy("http://localhost:8000/") + with open("fetched_python_logo.jpg", "wb") as handle: + handle.write(proxy.python_logo().data) + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2045.html#section-6.8 + + +File: python.info, Node: Fault Objects, Next: ProtocolError Objects, Prev: Binary Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.4 Fault Objects +....................... + + -- Class: xmlrpc.client.Fault + + A *note Fault: 2bcf. object encapsulates the content of an XML-RPC + fault tag. Fault objects have the following attributes: + + -- Attribute: faultCode + + A string indicating the fault type. + + -- Attribute: faultString + + A string containing a diagnostic message associated with the + fault. + +In the following example we’re going to intentionally cause a *note +Fault: 2bcf. by returning a complex type object. The server code: + + from xmlrpc.server import SimpleXMLRPCServer + + # A marshalling error is going to occur because we're returning a + # complex number + def add(x, y): + return x+y+0j + + server = SimpleXMLRPCServer(("localhost", 8000)) + print("Listening on port 8000...") + server.register_function(add, 'add') + + server.serve_forever() + +The client code for the preceding server: + + import xmlrpc.client + + proxy = xmlrpc.client.ServerProxy("http://localhost:8000/") + try: + proxy.add(2, 5) + except xmlrpc.client.Fault as err: + print("A fault occurred") + print("Fault code: %d" % err.faultCode) + print("Fault string: %s" % err.faultString) + + +File: python.info, Node: ProtocolError Objects, Next: MultiCall Objects, Prev: Fault Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.5 ProtocolError Objects +............................... + + -- Class: xmlrpc.client.ProtocolError + + A *note ProtocolError: 2bd0. object describes a protocol error in + the underlying transport layer (such as a 404 ‘not found’ error if + the server named by the URI does not exist). It has the following + attributes: + + -- Attribute: url + + The URI or URL that triggered the error. + + -- Attribute: errcode + + The error code. + + -- Attribute: errmsg + + The error message or diagnostic string. + + -- Attribute: headers + + A dict containing the headers of the HTTP/HTTPS request that + triggered the error. + +In the following example we’re going to intentionally cause a *note +ProtocolError: 2bd0. by providing an invalid URI: + + import xmlrpc.client + + # create a ServerProxy with a URI that doesn't respond to XMLRPC requests + proxy = xmlrpc.client.ServerProxy("http://google.com/") + + try: + proxy.some_method() + except xmlrpc.client.ProtocolError as err: + print("A protocol error occurred") + print("URL: %s" % err.url) + print("HTTP/HTTPS headers: %s" % err.headers) + print("Error code: %d" % err.errcode) + print("Error message: %s" % err.errmsg) + + +File: python.info, Node: MultiCall Objects, Next: Convenience Functions, Prev: ProtocolError Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.6 MultiCall Objects +........................... + +The *note MultiCall: 2bea. object provides a way to encapsulate multiple +calls to a remote server into a single request (1). + + -- Class: xmlrpc.client.MultiCall (server) + + Create an object used to boxcar method calls. `server' is the + eventual target of the call. Calls can be made to the result + object, but they will immediately return ‘None’, and only store the + call name and parameters in the *note MultiCall: 2bea. object. + Calling the object itself causes all stored calls to be transmitted + as a single ‘system.multicall’ request. The result of this call is + a *note generator: 9a2.; iterating over this generator yields the + individual results. + +A usage example of this class follows. The server code: + + from xmlrpc.server import SimpleXMLRPCServer + + def add(x, y): + return x + y + + def subtract(x, y): + return x - y + + def multiply(x, y): + return x * y + + def divide(x, y): + return x // y + + # A simple server with simple arithmetic functions + server = SimpleXMLRPCServer(("localhost", 8000)) + print("Listening on port 8000...") + server.register_multicall_functions() + server.register_function(add, 'add') + server.register_function(subtract, 'subtract') + server.register_function(multiply, 'multiply') + server.register_function(divide, 'divide') + server.serve_forever() + +The client code for the preceding server: + + import xmlrpc.client + + proxy = xmlrpc.client.ServerProxy("http://localhost:8000/") + multicall = xmlrpc.client.MultiCall(proxy) + multicall.add(7, 3) + multicall.subtract(7, 3) + multicall.multiply(7, 3) + multicall.divide(7, 3) + result = multicall() + + print("7+3=%d, 7-3=%d, 7*3=%d, 7//3=%d" % tuple(result)) + + ---------- Footnotes ---------- + + (1) (1) This approach has been first presented in a discussion on +xmlrpc.com +(https://web.archive.org/web/20060624230303/http://www.xmlrpc.com/discuss/msgReader$1208?mode=topic). + + +File: python.info, Node: Convenience Functions, Next: Example of Client Usage, Prev: MultiCall Objects, Up: xmlrpc client — XML-RPC client access + +5.21.26.7 Convenience Functions +............................... + + -- Function: xmlrpc.client.dumps (params, methodname=None, + methodresponse=None, encoding=None, allow_none=False) + + Convert `params' into an XML-RPC request. or into a response if + `methodresponse' is true. `params' can be either a tuple of + arguments or an instance of the *note Fault: 2bcf. exception class. + If `methodresponse' is true, only a single value can be returned, + meaning that `params' must be of length 1. `encoding', if + supplied, is the encoding to use in the generated XML; the default + is UTF-8. Python’s *note None: 157. value cannot be used in + standard XML-RPC; to allow using it via an extension, provide a + true value for `allow_none'. + + -- Function: xmlrpc.client.loads (data, use_datetime=False, + use_builtin_types=False) + + Convert an XML-RPC request or response into Python objects, a + ‘(params, methodname)’. `params' is a tuple of argument; + `methodname' is a string, or ‘None’ if no method name is present in + the packet. If the XML-RPC packet represents a fault condition, + this function will raise a *note Fault: 2bcf. exception. The + `use_builtin_types' flag can be used to cause date/time values to + be presented as *note datetime.datetime: 194. objects and binary + data to be presented as *note bytes: 331. objects; this flag is + false by default. + + The obsolete `use_datetime' flag is similar to `use_builtin_types' + but it applies only to date/time values. + + Changed in version 3.3: The `use_builtin_types' flag was added. + + +File: python.info, Node: Example of Client Usage, Next: Example of Client and Server Usage, Prev: Convenience Functions, Up: xmlrpc client — XML-RPC client access + +5.21.26.8 Example of Client Usage +................................. + + # simple test program (from the XML-RPC specification) + from xmlrpc.client import ServerProxy, Error + + # server = ServerProxy("http://localhost:8000") # local server + with ServerProxy("http://betty.userland.com") as proxy: + + print(proxy) + + try: + print(proxy.examples.getStateName(41)) + except Error as v: + print("ERROR", v) + +To access an XML-RPC server through a HTTP proxy, you need to define a +custom transport. The following example shows how: + + import http.client + import xmlrpc.client + + class ProxiedTransport(xmlrpc.client.Transport): + + def set_proxy(self, host, port=None, headers=None): + self.proxy = host, port + self.proxy_headers = headers + + def make_connection(self, host): + connection = http.client.HTTPConnection(*self.proxy) + connection.set_tunnel(host, headers=self.proxy_headers) + self._connection = host, connection + return connection + + transport = ProxiedTransport() + transport.set_proxy('proxy-server', 8080) + server = xmlrpc.client.ServerProxy('http://betty.userland.com', transport=transport) + print(server.examples.getStateName(41)) + + +File: python.info, Node: Example of Client and Server Usage, Prev: Example of Client Usage, Up: xmlrpc client — XML-RPC client access + +5.21.26.9 Example of Client and Server Usage +............................................ + +See *note SimpleXMLRPCServer Example: 2bf1. + + +File: python.info, Node: xmlrpc server — Basic XML-RPC servers, Next: ipaddress — IPv4/IPv6 manipulation library, Prev: xmlrpc client — XML-RPC client access, Up: Internet Protocols and Support + +5.21.27 ‘xmlrpc.server’ — Basic XML-RPC servers +----------------------------------------------- + +`Source code:' Lib/xmlrpc/server.py(1) + +__________________________________________________________________ + +The *note xmlrpc.server: 140. module provides a basic server framework +for XML-RPC servers written in Python. Servers can either be free +standing, using *note SimpleXMLRPCServer: 2bf4, or embedded in a CGI +environment, using *note CGIXMLRPCRequestHandler: 2bf5. + + Warning: The *note xmlrpc.server: 140. module is not secure against + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see *note XML vulnerabilities: 26f5. + + -- Class: xmlrpc.server.SimpleXMLRPCServer (addr, + requestHandler=SimpleXMLRPCRequestHandler, logRequests=True, + allow_none=False, encoding=None, bind_and_activate=True, + use_builtin_types=False) + + Create a new server instance. This class provides methods for + registration of functions that can be called by the XML-RPC + protocol. The `requestHandler' parameter should be a factory for + request handler instances; it defaults to *note + SimpleXMLRPCRequestHandler: 2bf6. The `addr' and `requestHandler' + parameters are passed to the *note socketserver.TCPServer: 2b1c. + constructor. If `logRequests' is true (the default), requests will + be logged; setting this parameter to false will turn off logging. + The `allow_none' and `encoding' parameters are passed on to *note + xmlrpc.client: 13f. and control the XML-RPC responses that will be + returned from the server. The `bind_and_activate' parameter + controls whether ‘server_bind()’ and ‘server_activate()’ are called + immediately by the constructor; it defaults to true. Setting it to + false allows code to manipulate the `allow_reuse_address' class + variable before the address is bound. The `use_builtin_types' + parameter is passed to the *note loads(): 2bed. function and + controls which types are processed when date/times values or binary + data are received; it defaults to false. + + Changed in version 3.3: The `use_builtin_types' flag was added. + + -- Class: xmlrpc.server.CGIXMLRPCRequestHandler (allow_none=False, + encoding=None, use_builtin_types=False) + + Create a new instance to handle XML-RPC requests in a CGI + environment. The `allow_none' and `encoding' parameters are passed + on to *note xmlrpc.client: 13f. and control the XML-RPC responses + that will be returned from the server. The `use_builtin_types' + parameter is passed to the *note loads(): 2bed. function and + controls which types are processed when date/times values or binary + data are received; it defaults to false. + + Changed in version 3.3: The `use_builtin_types' flag was added. + + -- Class: xmlrpc.server.SimpleXMLRPCRequestHandler + + Create a new request handler instance. This request handler + supports ‘POST’ requests and modifies logging so that the + `logRequests' parameter to the *note SimpleXMLRPCServer: 2bf4. + constructor parameter is honored. + +* Menu: + +* SimpleXMLRPCServer Objects:: +* CGIXMLRPCRequestHandler:: +* Documenting XMLRPC server:: +* DocXMLRPCServer Objects:: +* DocCGIXMLRPCRequestHandler:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/xmlrpc/server.py + + +File: python.info, Node: SimpleXMLRPCServer Objects, Next: CGIXMLRPCRequestHandler, Up: xmlrpc server — Basic XML-RPC servers + +5.21.27.1 SimpleXMLRPCServer Objects +.................................... + +The *note SimpleXMLRPCServer: 2bf4. class is based on *note +socketserver.TCPServer: 2b1c. and provides a means of creating simple, +stand alone XML-RPC servers. + + -- Method: SimpleXMLRPCServer.register_function (function=None, + name=None) + + Register a function that can respond to XML-RPC requests. If + `name' is given, it will be the method name associated with + `function', otherwise ‘function.__name__’ will be used. `name' is + a string, and may contain characters not legal in Python + identifiers, including the period character. + + This method can also be used as a decorator. When used as a + decorator, `name' can only be given as a keyword argument to + register `function' under `name'. If no `name' is given, + ‘function.__name__’ will be used. + + Changed in version 3.7: *note register_function(): 2bf9. can be + used as a decorator. + + -- Method: SimpleXMLRPCServer.register_instance (instance, + allow_dotted_names=False) + + Register an object which is used to expose method names which have + not been registered using *note register_function(): 2bf9. If + `instance' contains a ‘_dispatch()’ method, it is called with the + requested method name and the parameters from the request. Its API + is ‘def _dispatch(self, method, params)’ (note that `params' does + not represent a variable argument list). If it calls an underlying + function to perform its task, that function is called as + ‘func(*params)’, expanding the parameter list. The return value + from ‘_dispatch()’ is returned to the client as the result. If + `instance' does not have a ‘_dispatch()’ method, it is searched for + an attribute matching the name of the requested method. + + If the optional `allow_dotted_names' argument is true and the + instance does not have a ‘_dispatch()’ method, then if the + requested method name contains periods, each component of the + method name is searched for individually, with the effect that a + simple hierarchical search is performed. The value found from this + search is then called with the parameters from the request, and the + return value is passed back to the client. + + Warning: Enabling the `allow_dotted_names' option allows + intruders to access your module’s global variables and may + allow intruders to execute arbitrary code on your machine. + Only use this option on a secure, closed network. + + -- Method: SimpleXMLRPCServer.register_introspection_functions () + + Registers the XML-RPC introspection functions ‘system.listMethods’, + ‘system.methodHelp’ and ‘system.methodSignature’. + + -- Method: SimpleXMLRPCServer.register_multicall_functions () + + Registers the XML-RPC multicall function system.multicall. + + -- Attribute: SimpleXMLRPCRequestHandler.rpc_paths + + An attribute value that must be a tuple listing valid path portions + of the URL for receiving XML-RPC requests. Requests posted to + other paths will result in a 404 “no such page” HTTP error. If + this tuple is empty, all paths will be considered valid. The + default value is ‘('/', '/RPC2')’. + +* Menu: + +* SimpleXMLRPCServer Example:: + + +File: python.info, Node: SimpleXMLRPCServer Example, Up: SimpleXMLRPCServer Objects + +5.21.27.2 SimpleXMLRPCServer Example +.................................... + +Server code: + + from xmlrpc.server import SimpleXMLRPCServer + from xmlrpc.server import SimpleXMLRPCRequestHandler + + # Restrict to a particular path. + class RequestHandler(SimpleXMLRPCRequestHandler): + rpc_paths = ('/RPC2',) + + # Create server + with SimpleXMLRPCServer(('localhost', 8000), + requestHandler=RequestHandler) as server: + server.register_introspection_functions() + + # Register pow() function; this will use the value of + # pow.__name__ as the name, which is just 'pow'. + server.register_function(pow) + + # Register a function under a different name + def adder_function(x, y): + return x + y + server.register_function(adder_function, 'add') + + # Register an instance; all the methods of the instance are + # published as XML-RPC methods (in this case, just 'mul'). + class MyFuncs: + def mul(self, x, y): + return x * y + + server.register_instance(MyFuncs()) + + # Run the server's main loop + server.serve_forever() + +The following client code will call the methods made available by the +preceding server: + + import xmlrpc.client + + s = xmlrpc.client.ServerProxy('http://localhost:8000') + print(s.pow(2,3)) # Returns 2**3 = 8 + print(s.add(2,3)) # Returns 5 + print(s.mul(5,2)) # Returns 5*2 = 10 + + # Print list of available methods + print(s.system.listMethods()) + +‘register_function()’ can also be used as a decorator. The previous +server example can register functions in a decorator way: + + from xmlrpc.server import SimpleXMLRPCServer + from xmlrpc.server import SimpleXMLRPCRequestHandler + + class RequestHandler(SimpleXMLRPCRequestHandler): + rpc_paths = ('/RPC2',) + + with SimpleXMLRPCServer(('localhost', 8000), + requestHandler=RequestHandler) as server: + server.register_introspection_functions() + + # Register pow() function; this will use the value of + # pow.__name__ as the name, which is just 'pow'. + server.register_function(pow) + + # Register a function under a different name, using + # register_function as a decorator. *name* can only be given + # as a keyword argument. + @server.register_function(name='add') + def adder_function(x, y): + return x + y + + # Register a function under function.__name__. + @server.register_function + def mul(x, y): + return x * y + + server.serve_forever() + +The following example included in the ‘Lib/xmlrpc/server.py’ module +shows a server allowing dotted names and registering a multicall +function. + + Warning: Enabling the `allow_dotted_names' option allows intruders + to access your module’s global variables and may allow intruders to + execute arbitrary code on your machine. Only use this example only + within a secure, closed network. + + import datetime + + class ExampleService: + def getData(self): + return '42' + + class currentTime: + @staticmethod + def getCurrentTime(): + return datetime.datetime.now() + + with SimpleXMLRPCServer(("localhost", 8000)) as server: + server.register_function(pow) + server.register_function(lambda x,y: x+y, 'add') + server.register_instance(ExampleService(), allow_dotted_names=True) + server.register_multicall_functions() + print('Serving XML-RPC on localhost port 8000') + try: + server.serve_forever() + except KeyboardInterrupt: + print("\nKeyboard interrupt received, exiting.") + sys.exit(0) + +This ExampleService demo can be invoked from the command line: + + python -m xmlrpc.server + +The client that interacts with the above server is included in +‘Lib/xmlrpc/client.py’: + + server = ServerProxy("http://localhost:8000") + + try: + print(server.currentTime.getCurrentTime()) + except Error as v: + print("ERROR", v) + + multi = MultiCall(server) + multi.getData() + multi.pow(2,9) + multi.add(1,2) + try: + for response in multi(): + print(response) + except Error as v: + print("ERROR", v) + +This client which interacts with the demo XMLRPC server can be invoked +as: + + python -m xmlrpc.client + + +File: python.info, Node: CGIXMLRPCRequestHandler, Next: Documenting XMLRPC server, Prev: SimpleXMLRPCServer Objects, Up: xmlrpc server — Basic XML-RPC servers + +5.21.27.3 CGIXMLRPCRequestHandler +................................. + +The *note CGIXMLRPCRequestHandler: 2bf5. class can be used to handle +XML-RPC requests sent to Python CGI scripts. + + -- Method: CGIXMLRPCRequestHandler.register_function (function=None, + name=None) + + Register a function that can respond to XML-RPC requests. If + `name' is given, it will be the method name associated with + `function', otherwise ‘function.__name__’ will be used. `name' is + a string, and may contain characters not legal in Python + identifiers, including the period character. + + This method can also be used as a decorator. When used as a + decorator, `name' can only be given as a keyword argument to + register `function' under `name'. If no `name' is given, + ‘function.__name__’ will be used. + + Changed in version 3.7: *note register_function(): 2c00. can be + used as a decorator. + + -- Method: CGIXMLRPCRequestHandler.register_instance (instance) + + Register an object which is used to expose method names which have + not been registered using *note register_function(): 2c00. If + instance contains a ‘_dispatch()’ method, it is called with the + requested method name and the parameters from the request; the + return value is returned to the client as the result. If instance + does not have a ‘_dispatch()’ method, it is searched for an + attribute matching the name of the requested method; if the + requested method name contains periods, each component of the + method name is searched for individually, with the effect that a + simple hierarchical search is performed. The value found from this + search is then called with the parameters from the request, and the + return value is passed back to the client. + + -- Method: CGIXMLRPCRequestHandler.register_introspection_functions () + + Register the XML-RPC introspection functions ‘system.listMethods’, + ‘system.methodHelp’ and ‘system.methodSignature’. + + -- Method: CGIXMLRPCRequestHandler.register_multicall_functions () + + Register the XML-RPC multicall function ‘system.multicall’. + + -- Method: CGIXMLRPCRequestHandler.handle_request (request_text=None) + + Handle an XML-RPC request. If `request_text' is given, it should + be the POST data provided by the HTTP server, otherwise the + contents of stdin will be used. + +Example: + + class MyFuncs: + def mul(self, x, y): + return x * y + + + handler = CGIXMLRPCRequestHandler() + handler.register_function(pow) + handler.register_function(lambda x,y: x+y, 'add') + handler.register_introspection_functions() + handler.register_instance(MyFuncs()) + handler.handle_request() + + +File: python.info, Node: Documenting XMLRPC server, Next: DocXMLRPCServer Objects, Prev: CGIXMLRPCRequestHandler, Up: xmlrpc server — Basic XML-RPC servers + +5.21.27.4 Documenting XMLRPC server +................................... + +These classes extend the above classes to serve HTML documentation in +response to HTTP GET requests. Servers can either be free standing, +using *note DocXMLRPCServer: 2c06, or embedded in a CGI environment, +using *note DocCGIXMLRPCRequestHandler: 2c07. + + -- Class: xmlrpc.server.DocXMLRPCServer (addr, + requestHandler=DocXMLRPCRequestHandler, logRequests=True, + allow_none=False, encoding=None, bind_and_activate=True, + use_builtin_types=True) + + Create a new server instance. All parameters have the same meaning + as for *note SimpleXMLRPCServer: 2bf4.; `requestHandler' defaults + to *note DocXMLRPCRequestHandler: 2c08. + + Changed in version 3.3: The `use_builtin_types' flag was added. + + -- Class: xmlrpc.server.DocCGIXMLRPCRequestHandler + + Create a new instance to handle XML-RPC requests in a CGI + environment. + + -- Class: xmlrpc.server.DocXMLRPCRequestHandler + + Create a new request handler instance. This request handler + supports XML-RPC POST requests, documentation GET requests, and + modifies logging so that the `logRequests' parameter to the *note + DocXMLRPCServer: 2c06. constructor parameter is honored. + + +File: python.info, Node: DocXMLRPCServer Objects, Next: DocCGIXMLRPCRequestHandler, Prev: Documenting XMLRPC server, Up: xmlrpc server — Basic XML-RPC servers + +5.21.27.5 DocXMLRPCServer Objects +................................. + +The *note DocXMLRPCServer: 2c06. class is derived from *note +SimpleXMLRPCServer: 2bf4. and provides a means of creating +self-documenting, stand alone XML-RPC servers. HTTP POST requests are +handled as XML-RPC method calls. HTTP GET requests are handled by +generating pydoc-style HTML documentation. This allows a server to +provide its own web-based documentation. + + -- Method: DocXMLRPCServer.set_server_title (server_title) + + Set the title used in the generated HTML documentation. This title + will be used inside the HTML “title” element. + + -- Method: DocXMLRPCServer.set_server_name (server_name) + + Set the name used in the generated HTML documentation. This name + will appear at the top of the generated documentation inside a “h1” + element. + + -- Method: DocXMLRPCServer.set_server_documentation + (server_documentation) + + Set the description used in the generated HTML documentation. This + description will appear as a paragraph, below the server name, in + the documentation. + + +File: python.info, Node: DocCGIXMLRPCRequestHandler, Prev: DocXMLRPCServer Objects, Up: xmlrpc server — Basic XML-RPC servers + +5.21.27.6 DocCGIXMLRPCRequestHandler +.................................... + +The *note DocCGIXMLRPCRequestHandler: 2c07. class is derived from *note +CGIXMLRPCRequestHandler: 2bf5. and provides a means of creating +self-documenting, XML-RPC CGI scripts. HTTP POST requests are handled +as XML-RPC method calls. HTTP GET requests are handled by generating +pydoc-style HTML documentation. This allows a server to provide its own +web-based documentation. + + -- Method: DocCGIXMLRPCRequestHandler.set_server_title (server_title) + + Set the title used in the generated HTML documentation. This title + will be used inside the HTML “title” element. + + -- Method: DocCGIXMLRPCRequestHandler.set_server_name (server_name) + + Set the name used in the generated HTML documentation. This name + will appear at the top of the generated documentation inside a “h1” + element. + + -- Method: DocCGIXMLRPCRequestHandler.set_server_documentation + (server_documentation) + + Set the description used in the generated HTML documentation. This + description will appear as a paragraph, below the server name, in + the documentation. + + +File: python.info, Node: ipaddress — IPv4/IPv6 manipulation library, Prev: xmlrpc server — Basic XML-RPC servers, Up: Internet Protocols and Support + +5.21.28 ‘ipaddress’ — IPv4/IPv6 manipulation library +---------------------------------------------------- + +`Source code:' Lib/ipaddress.py(1) + +__________________________________________________________________ + +*note ipaddress: a2. provides the capabilities to create, manipulate and +operate on IPv4 and IPv6 addresses and networks. + +The functions and classes in this module make it straightforward to +handle various tasks related to IP addresses, including checking whether +or not two hosts are on the same subnet, iterating over all hosts in a +particular subnet, checking whether or not a string represents a valid +IP address or network definition, and so on. + +This is the full module API reference—for an overview and introduction, +see *note An introduction to the ipaddress module: 2c14. + +New in version 3.3. + +* Menu: + +* Convenience factory functions:: +* IP Addresses:: +* IP Network definitions:: +* Interface objects:: +* Other Module Level Functions:: +* Custom Exceptions:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/ipaddress.py + + +File: python.info, Node: Convenience factory functions, Next: IP Addresses, Up: ipaddress — IPv4/IPv6 manipulation library + +5.21.28.1 Convenience factory functions +....................................... + +The *note ipaddress: a2. module provides factory functions to +conveniently create IP addresses, networks and interfaces: + + -- Function: ipaddress.ip_address (address) + + Return an *note IPv4Address: 2c17. or *note IPv6Address: 2c18. + object depending on the IP address passed as argument. Either IPv4 + or IPv6 addresses may be supplied; integers less than 2**32 will be + considered to be IPv4 by default. A *note ValueError: 1fb. is + raised if `address' does not represent a valid IPv4 or IPv6 + address. + + >>> ipaddress.ip_address('192.168.0.1') + IPv4Address('192.168.0.1') + >>> ipaddress.ip_address('2001:db8::') + IPv6Address('2001:db8::') + + -- Function: ipaddress.ip_network (address, strict=True) + + Return an *note IPv4Network: 3a0. or *note IPv6Network: 39f. object + depending on the IP address passed as argument. `address' is a + string or integer representing the IP network. Either IPv4 or IPv6 + networks may be supplied; integers less than 2**32 will be + considered to be IPv4 by default. `strict' is passed to *note + IPv4Network: 3a0. or *note IPv6Network: 39f. constructor. A *note + ValueError: 1fb. is raised if `address' does not represent a valid + IPv4 or IPv6 address, or if the network has host bits set. + + >>> ipaddress.ip_network('192.168.0.0/28') + IPv4Network('192.168.0.0/28') + + -- Function: ipaddress.ip_interface (address) + + Return an *note IPv4Interface: 2c1b. or *note IPv6Interface: 2c1c. + object depending on the IP address passed as argument. `address' + is a string or integer representing the IP address. Either IPv4 or + IPv6 addresses may be supplied; integers less than 2**32 will be + considered to be IPv4 by default. A *note ValueError: 1fb. is + raised if `address' does not represent a valid IPv4 or IPv6 + address. + +One downside of these convenience functions is that the need to handle +both IPv4 and IPv6 formats means that error messages provide minimal +information on the precise error, as the functions don’t know whether +the IPv4 or IPv6 format was intended. More detailed error reporting can +be obtained by calling the appropriate version specific class +constructors directly. + + +File: python.info, Node: IP Addresses, Next: IP Network definitions, Prev: Convenience factory functions, Up: ipaddress — IPv4/IPv6 manipulation library + +5.21.28.2 IP Addresses +...................... + +* Menu: + +* Address objects:: +* Conversion to Strings and Integers:: +* Operators: Operators<3>. + + +File: python.info, Node: Address objects, Next: Conversion to Strings and Integers, Up: IP Addresses + +5.21.28.3 Address objects +......................... + +The *note IPv4Address: 2c17. and *note IPv6Address: 2c18. objects share +a lot of common attributes. Some attributes that are only meaningful +for IPv6 addresses are also implemented by *note IPv4Address: 2c17. +objects, in order to make it easier to write code that handles both IP +versions correctly. Address objects are *note hashable: 10c8, so they +can be used as keys in dictionaries. + + -- Class: ipaddress.IPv4Address (address) + + Construct an IPv4 address. An *note AddressValueError: 2c1f. is + raised if `address' is not a valid IPv4 address. + + The following constitutes a valid IPv4 address: + + 1. A string in decimal-dot notation, consisting of four decimal + integers in the inclusive range 0–255, separated by dots (e.g. + ‘192.168.0.1’). Each integer represents an octet (byte) in + the address. Leading zeroes are tolerated only for values + less than 8 (as there is no ambiguity between the decimal and + octal interpretations of such strings). + + 2. An integer that fits into 32 bits. + + 3. An integer packed into a *note bytes: 331. object of length 4 + (most significant octet first). + + >>> ipaddress.IPv4Address('192.168.0.1') + IPv4Address('192.168.0.1') + >>> ipaddress.IPv4Address(3232235521) + IPv4Address('192.168.0.1') + >>> ipaddress.IPv4Address(b'\xC0\xA8\x00\x01') + IPv4Address('192.168.0.1') + + -- Attribute: version + + The appropriate version number: ‘4’ for IPv4, ‘6’ for IPv6. + + -- Attribute: max_prefixlen + + The total number of bits in the address representation for + this version: ‘32’ for IPv4, ‘128’ for IPv6. + + The prefix defines the number of leading bits in an address + that are compared to determine whether or not an address is + part of a network. + + -- Attribute: compressed + + -- Attribute: exploded + + The string representation in dotted decimal notation. Leading + zeroes are never included in the representation. + + As IPv4 does not define a shorthand notation for addresses + with octets set to zero, these two attributes are always the + same as ‘str(addr)’ for IPv4 addresses. Exposing these + attributes makes it easier to write display code that can + handle both IPv4 and IPv6 addresses. + + -- Attribute: packed + + The binary representation of this address - a *note bytes: + 331. object of the appropriate length (most significant octet + first). This is 4 bytes for IPv4 and 16 bytes for IPv6. + + -- Attribute: reverse_pointer + + The name of the reverse DNS PTR record for the IP address, + e.g.: + + >>> ipaddress.ip_address("127.0.0.1").reverse_pointer + '1.0.0.127.in-addr.arpa' + >>> ipaddress.ip_address("2001:db8::1").reverse_pointer + '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa' + + This is the name that could be used for performing a PTR + lookup, not the resolved hostname itself. + + New in version 3.5. + + -- Attribute: is_multicast + + ‘True’ if the address is reserved for multicast use. See RFC + 3171(1) (for IPv4) or RFC 2373(2) (for IPv6). + + -- Attribute: is_private + + ‘True’ if the address is allocated for private networks. See + iana-ipv4-special-registry(3) (for IPv4) or + iana-ipv6-special-registry(4) (for IPv6). + + -- Attribute: is_global + + ‘True’ if the address is allocated for public networks. See + iana-ipv4-special-registry(5) (for IPv4) or + iana-ipv6-special-registry(6) (for IPv6). + + New in version 3.4. + + -- Attribute: is_unspecified + + ‘True’ if the address is unspecified. See RFC 5735(7) (for + IPv4) or RFC 2373(8) (for IPv6). + + -- Attribute: is_reserved + + ‘True’ if the address is otherwise IETF reserved. + + -- Attribute: is_loopback + + ‘True’ if this is a loopback address. See RFC 3330(9) (for + IPv4) or RFC 2373(10) (for IPv6). + + -- Attribute: is_link_local + + ‘True’ if the address is reserved for link-local usage. See + RFC 3927(11). + + -- Class: ipaddress.IPv6Address (address) + + Construct an IPv6 address. An *note AddressValueError: 2c1f. is + raised if `address' is not a valid IPv6 address. + + The following constitutes a valid IPv6 address: + + 1. A string consisting of eight groups of four hexadecimal + digits, each group representing 16 bits. The groups are + separated by colons. This describes an `exploded' (longhand) + notation. The string can also be `compressed' (shorthand + notation) by various means. See RFC 4291(12) for details. + For example, ‘"0000:0000:0000:0000:0000:0abc:0007:0def"’ can + be compressed to ‘"::abc:7:def"’. + + 2. An integer that fits into 128 bits. + + 3. An integer packed into a *note bytes: 331. object of length + 16, big-endian. + + >>> ipaddress.IPv6Address('2001:db8::1000') + IPv6Address('2001:db8::1000') + + -- Attribute: compressed + + The short form of the address representation, with leading zeroes + in groups omitted and the longest sequence of groups consisting + entirely of zeroes collapsed to a single empty group. + + This is also the value returned by ‘str(addr)’ for IPv6 addresses. + + -- Attribute: exploded + + The long form of the address representation, with all leading + zeroes and groups consisting entirely of zeroes included. + + For the following attributes, see the corresponding documentation + of the *note IPv4Address: 2c17. class: + + -- Attribute: packed + + -- Attribute: reverse_pointer + + -- Attribute: version + + -- Attribute: max_prefixlen + + -- Attribute: is_multicast + + -- Attribute: is_private + + -- Attribute: is_global + + -- Attribute: is_unspecified + + -- Attribute: is_reserved + + -- Attribute: is_loopback + + -- Attribute: is_link_local + + New in version 3.4: is_global + + -- Attribute: is_site_local + + ‘True’ if the address is reserved for site-local usage. Note + that the site-local address space has been deprecated by RFC + 3879(13). Use *note is_private: 2c27. to test if this address + is in the space of unique local addresses as defined by RFC + 4193(14). + + -- Attribute: ipv4_mapped + + For addresses that appear to be IPv4 mapped addresses + (starting with ‘::FFFF/96’), this property will report the + embedded IPv4 address. For any other address, this property + will be ‘None’. + + -- Attribute: sixtofour + + For addresses that appear to be 6to4 addresses (starting with + ‘2002::/16’) as defined by RFC 3056(15), this property will + report the embedded IPv4 address. For any other address, this + property will be ‘None’. + + -- Attribute: teredo + + For addresses that appear to be Teredo addresses (starting + with ‘2001::/32’) as defined by RFC 4380(16), this property + will report the embedded ‘(server, client)’ IP address pair. + For any other address, this property will be ‘None’. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc3171.html + + (2) https://tools.ietf.org/html/rfc2373.html + + (3) +https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml + + (4) +https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml + + (5) +https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml + + (6) +https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml + + (7) https://tools.ietf.org/html/rfc5735.html + + (8) https://tools.ietf.org/html/rfc2373.html + + (9) https://tools.ietf.org/html/rfc3330.html + + (10) https://tools.ietf.org/html/rfc2373.html + + (11) https://tools.ietf.org/html/rfc3927.html + + (12) https://tools.ietf.org/html/rfc4291.html + + (13) https://tools.ietf.org/html/rfc3879.html + + (14) https://tools.ietf.org/html/rfc4193.html + + (15) https://tools.ietf.org/html/rfc3056.html + + (16) https://tools.ietf.org/html/rfc4380.html + + +File: python.info, Node: Conversion to Strings and Integers, Next: Operators<3>, Prev: Address objects, Up: IP Addresses + +5.21.28.4 Conversion to Strings and Integers +............................................ + +To interoperate with networking interfaces such as the socket module, +addresses must be converted to strings or integers. This is handled +using the *note str(): 330. and *note int(): 184. builtin functions: + + >>> str(ipaddress.IPv4Address('192.168.0.1')) + '192.168.0.1' + >>> int(ipaddress.IPv4Address('192.168.0.1')) + 3232235521 + >>> str(ipaddress.IPv6Address('::1')) + '::1' + >>> int(ipaddress.IPv6Address('::1')) + 1 + + +File: python.info, Node: Operators<3>, Prev: Conversion to Strings and Integers, Up: IP Addresses + +5.21.28.5 Operators +................... + +Address objects support some operators. Unless stated otherwise, +operators can only be applied between compatible objects (i.e. IPv4 +with IPv4, IPv6 with IPv6). + +* Menu: + +* Comparison operators:: +* Arithmetic operators:: + + +File: python.info, Node: Comparison operators, Next: Arithmetic operators, Up: Operators<3> + +5.21.28.6 Comparison operators +.............................. + +Address objects can be compared with the usual set of comparison +operators. Some examples: + + >>> IPv4Address('127.0.0.2') > IPv4Address('127.0.0.1') + True + >>> IPv4Address('127.0.0.2') == IPv4Address('127.0.0.1') + False + >>> IPv4Address('127.0.0.2') != IPv4Address('127.0.0.1') + True + + +File: python.info, Node: Arithmetic operators, Prev: Comparison operators, Up: Operators<3> + +5.21.28.7 Arithmetic operators +.............................. + +Integers can be added to or subtracted from address objects. Some +examples: + + >>> IPv4Address('127.0.0.2') + 3 + IPv4Address('127.0.0.5') + >>> IPv4Address('127.0.0.2') - 3 + IPv4Address('126.255.255.255') + >>> IPv4Address('255.255.255.255') + 1 + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + ipaddress.AddressValueError: 4294967296 (>= 2**32) is not permitted as an IPv4 address + + +File: python.info, Node: IP Network definitions, Next: Interface objects, Prev: IP Addresses, Up: ipaddress — IPv4/IPv6 manipulation library + +5.21.28.8 IP Network definitions +................................ + +The *note IPv4Network: 3a0. and *note IPv6Network: 39f. objects provide +a mechanism for defining and inspecting IP network definitions. A +network definition consists of a `mask' and a `network address', and as +such defines a range of IP addresses that equal the network address when +masked (binary AND) with the mask. For example, a network definition +with the mask ‘255.255.255.0’ and the network address ‘192.168.1.0’ +consists of IP addresses in the inclusive range ‘192.168.1.0’ to +‘192.168.1.255’. + +* Menu: + +* Prefix, net mask and host mask: Prefix net mask and host mask. +* Network objects:: +* Operators: Operators<4>. + + +File: python.info, Node: Prefix net mask and host mask, Next: Network objects, Up: IP Network definitions + +5.21.28.9 Prefix, net mask and host mask +........................................ + +There are several equivalent ways to specify IP network masks. A +`prefix' ‘/<nbits>’ is a notation that denotes how many high-order bits +are set in the network mask. A `net mask' is an IP address with some +number of high-order bits set. Thus the prefix ‘/24’ is equivalent to +the net mask ‘255.255.255.0’ in IPv4, or ‘ffff:ff00::’ in IPv6. In +addition, a `host mask' is the logical inverse of a `net mask', and is +sometimes used (for example in Cisco access control lists) to denote a +network mask. The host mask equivalent to ‘/24’ in IPv4 is ‘0.0.0.255’. + + +File: python.info, Node: Network objects, Next: Operators<4>, Prev: Prefix net mask and host mask, Up: IP Network definitions + +5.21.28.10 Network objects +.......................... + +All attributes implemented by address objects are implemented by network +objects as well. In addition, network objects implement additional +attributes. All of these are common between *note IPv4Network: 3a0. and +*note IPv6Network: 39f, so to avoid duplication they are only documented +for *note IPv4Network: 3a0. Network objects are *note hashable: 10c8, +so they can be used as keys in dictionaries. + + -- Class: ipaddress.IPv4Network (address, strict=True) + + Construct an IPv4 network definition. `address' can be one of the + following: + + 1. A string consisting of an IP address and an optional mask, + separated by a slash (‘/’). The IP address is the network + address, and the mask can be either a single number, which + means it’s a `prefix', or a string representation of an IPv4 + address. If it’s the latter, the mask is interpreted as a + `net mask' if it starts with a non-zero field, or as a `host + mask' if it starts with a zero field, with the single + exception of an all-zero mask which is treated as a `net + mask'. If no mask is provided, it’s considered to be ‘/32’. + + For example, the following `address' specifications are + equivalent: ‘192.168.1.0/24’, ‘192.168.1.0/255.255.255.0’ and + ‘192.168.1.0/0.0.0.255’. + + 2. An integer that fits into 32 bits. This is equivalent to a + single-address network, with the network address being + `address' and the mask being ‘/32’. + + 3. An integer packed into a *note bytes: 331. object of length 4, + big-endian. The interpretation is similar to an integer + `address'. + + 4. A two-tuple of an address description and a netmask, where the + address description is either a string, a 32-bits integer, a + 4-bytes packed integer, or an existing IPv4Address object; and + the netmask is either an integer representing the prefix + length (e.g. ‘24’) or a string representing the prefix mask + (e.g. ‘255.255.255.0’). + + An *note AddressValueError: 2c1f. is raised if `address' is not a + valid IPv4 address. A *note NetmaskValueError: 2c44. is raised if + the mask is not valid for an IPv4 address. + + If `strict' is ‘True’ and host bits are set in the supplied + address, then *note ValueError: 1fb. is raised. Otherwise, the + host bits are masked out to determine the appropriate network + address. + + Unless stated otherwise, all network methods accepting other + network/address objects will raise *note TypeError: 192. if the + argument’s IP version is incompatible to ‘self’. + + Changed in version 3.5: Added the two-tuple form for the `address' + constructor parameter. + + -- Attribute: version + + -- Attribute: max_prefixlen + + Refer to the corresponding attribute documentation in *note + IPv4Address: 2c17. + + -- Attribute: is_multicast + + -- Attribute: is_private + + -- Attribute: is_unspecified + + -- Attribute: is_reserved + + -- Attribute: is_loopback + + -- Attribute: is_link_local + + These attributes are true for the network as a whole if they + are true for both the network address and the broadcast + address. + + -- Attribute: network_address + + The network address for the network. The network address and + the prefix length together uniquely define a network. + + -- Attribute: broadcast_address + + The broadcast address for the network. Packets sent to the + broadcast address should be received by every host on the + network. + + -- Attribute: hostmask + + The host mask, as an *note IPv4Address: 2c17. object. + + -- Attribute: netmask + + The net mask, as an *note IPv4Address: 2c17. object. + + -- Attribute: with_prefixlen + + -- Attribute: compressed + + -- Attribute: exploded + + A string representation of the network, with the mask in + prefix notation. + + ‘with_prefixlen’ and ‘compressed’ are always the same as + ‘str(network)’. ‘exploded’ uses the exploded form the network + address. + + -- Attribute: with_netmask + + A string representation of the network, with the mask in net + mask notation. + + -- Attribute: with_hostmask + + A string representation of the network, with the mask in host + mask notation. + + -- Attribute: num_addresses + + The total number of addresses in the network. + + -- Attribute: prefixlen + + Length of the network prefix, in bits. + + -- Method: hosts () + + Returns an iterator over the usable hosts in the network. The + usable hosts are all the IP addresses that belong to the + network, except the network address itself and the network + broadcast address. For networks with a mask length of 31, the + network address and network broadcast address are also + included in the result. + + >>> list(ip_network('192.0.2.0/29').hosts()) + [IPv4Address('192.0.2.1'), IPv4Address('192.0.2.2'), + IPv4Address('192.0.2.3'), IPv4Address('192.0.2.4'), + IPv4Address('192.0.2.5'), IPv4Address('192.0.2.6')] + >>> list(ip_network('192.0.2.0/31').hosts()) + [IPv4Address('192.0.2.0'), IPv4Address('192.0.2.1')] + + -- Method: overlaps (other) + + ‘True’ if this network is partly or wholly contained in + `other' or `other' is wholly contained in this network. + + -- Method: address_exclude (network) + + Computes the network definitions resulting from removing the + given `network' from this one. Returns an iterator of network + objects. Raises *note ValueError: 1fb. if `network' is not + completely contained in this network. + + >>> n1 = ip_network('192.0.2.0/28') + >>> n2 = ip_network('192.0.2.1/32') + >>> list(n1.address_exclude(n2)) + [IPv4Network('192.0.2.8/29'), IPv4Network('192.0.2.4/30'), + IPv4Network('192.0.2.2/31'), IPv4Network('192.0.2.0/32')] + + -- Method: subnets (prefixlen_diff=1, new_prefix=None) + + The subnets that join to make the current network definition, + depending on the argument values. `prefixlen_diff' is the + amount our prefix length should be increased by. `new_prefix' + is the desired new prefix of the subnets; it must be larger + than our prefix. One and only one of `prefixlen_diff' and + `new_prefix' must be set. Returns an iterator of network + objects. + + >>> list(ip_network('192.0.2.0/24').subnets()) + [IPv4Network('192.0.2.0/25'), IPv4Network('192.0.2.128/25')] + >>> list(ip_network('192.0.2.0/24').subnets(prefixlen_diff=2)) + [IPv4Network('192.0.2.0/26'), IPv4Network('192.0.2.64/26'), + IPv4Network('192.0.2.128/26'), IPv4Network('192.0.2.192/26')] + >>> list(ip_network('192.0.2.0/24').subnets(new_prefix=26)) + [IPv4Network('192.0.2.0/26'), IPv4Network('192.0.2.64/26'), + IPv4Network('192.0.2.128/26'), IPv4Network('192.0.2.192/26')] + >>> list(ip_network('192.0.2.0/24').subnets(new_prefix=23)) + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + raise ValueError('new prefix must be longer') + ValueError: new prefix must be longer + >>> list(ip_network('192.0.2.0/24').subnets(new_prefix=25)) + [IPv4Network('192.0.2.0/25'), IPv4Network('192.0.2.128/25')] + + -- Method: supernet (prefixlen_diff=1, new_prefix=None) + + The supernet containing this network definition, depending on + the argument values. `prefixlen_diff' is the amount our + prefix length should be decreased by. `new_prefix' is the + desired new prefix of the supernet; it must be smaller than + our prefix. One and only one of `prefixlen_diff' and + `new_prefix' must be set. Returns a single network object. + + >>> ip_network('192.0.2.0/24').supernet() + IPv4Network('192.0.2.0/23') + >>> ip_network('192.0.2.0/24').supernet(prefixlen_diff=2) + IPv4Network('192.0.0.0/22') + >>> ip_network('192.0.2.0/24').supernet(new_prefix=20) + IPv4Network('192.0.0.0/20') + + -- Method: subnet_of (other) + + Return ‘True’ if this network is a subnet of `other'. + + >>> a = ip_network('192.168.1.0/24') + >>> b = ip_network('192.168.1.128/30') + >>> b.subnet_of(a) + True + + New in version 3.7. + + -- Method: supernet_of (other) + + Return ‘True’ if this network is a supernet of `other'. + + >>> a = ip_network('192.168.1.0/24') + >>> b = ip_network('192.168.1.128/30') + >>> a.supernet_of(b) + True + + New in version 3.7. + + -- Method: compare_networks (other) + + Compare this network to `other'. In this comparison only the + network addresses are considered; host bits aren’t. Returns + either ‘-1’, ‘0’ or ‘1’. + + >>> ip_network('192.0.2.1/32').compare_networks(ip_network('192.0.2.2/32')) + -1 + >>> ip_network('192.0.2.1/32').compare_networks(ip_network('192.0.2.0/32')) + 1 + >>> ip_network('192.0.2.1/32').compare_networks(ip_network('192.0.2.1/32')) + 0 + + Deprecated since version 3.7: It uses the same ordering and + comparison algorithm as “<”, “==”, and “>” + + -- Class: ipaddress.IPv6Network (address, strict=True) + + Construct an IPv6 network definition. `address' can be one of the + following: + + 1. A string consisting of an IP address and an optional prefix + length, separated by a slash (‘/’). The IP address is the + network address, and the prefix length must be a single + number, the `prefix'. If no prefix length is provided, it’s + considered to be ‘/128’. + + Note that currently expanded netmasks are not supported. That + means ‘2001:db00::0/24’ is a valid argument while + ‘2001:db00::0/ffff:ff00::’ not. + + 2. An integer that fits into 128 bits. This is equivalent to a + single-address network, with the network address being + `address' and the mask being ‘/128’. + + 3. An integer packed into a *note bytes: 331. object of length + 16, big-endian. The interpretation is similar to an integer + `address'. + + 4. A two-tuple of an address description and a netmask, where the + address description is either a string, a 128-bits integer, a + 16-bytes packed integer, or an existing IPv6Address object; + and the netmask is an integer representing the prefix length. + + An *note AddressValueError: 2c1f. is raised if `address' is not a + valid IPv6 address. A *note NetmaskValueError: 2c44. is raised if + the mask is not valid for an IPv6 address. + + If `strict' is ‘True’ and host bits are set in the supplied + address, then *note ValueError: 1fb. is raised. Otherwise, the + host bits are masked out to determine the appropriate network + address. + + Changed in version 3.5: Added the two-tuple form for the `address' + constructor parameter. + + -- Attribute: version + + -- Attribute: max_prefixlen + + -- Attribute: is_multicast + + -- Attribute: is_private + + -- Attribute: is_unspecified + + -- Attribute: is_reserved + + -- Attribute: is_loopback + + -- Attribute: is_link_local + + -- Attribute: network_address + + -- Attribute: broadcast_address + + -- Attribute: hostmask + + -- Attribute: netmask + + -- Attribute: with_prefixlen + + -- Attribute: compressed + + -- Attribute: exploded + + -- Attribute: with_netmask + + -- Attribute: with_hostmask + + -- Attribute: num_addresses + + -- Attribute: prefixlen + + -- Method: hosts () + + Returns an iterator over the usable hosts in the network. The + usable hosts are all the IP addresses that belong to the + network, except the Subnet-Router anycast address. For + networks with a mask length of 127, the Subnet-Router anycast + address is also included in the result. + + -- Method: overlaps (other) + + -- Method: address_exclude (network) + + -- Method: subnets (prefixlen_diff=1, new_prefix=None) + + -- Method: supernet (prefixlen_diff=1, new_prefix=None) + + -- Method: subnet_of (other) + + -- Method: supernet_of (other) + + -- Method: compare_networks (other) + + Refer to the corresponding attribute documentation in *note + IPv4Network: 3a0. + + -- Attribute: is_site_local + + These attribute is true for the network as a whole if it is + true for both the network address and the broadcast address. + + +File: python.info, Node: Operators<4>, Prev: Network objects, Up: IP Network definitions + +5.21.28.11 Operators +.................... + +Network objects support some operators. Unless stated otherwise, +operators can only be applied between compatible objects (i.e. IPv4 +with IPv4, IPv6 with IPv6). + +* Menu: + +* Logical operators:: +* Iteration: Iteration<2>. +* Networks as containers of addresses:: + + +File: python.info, Node: Logical operators, Next: Iteration<2>, Up: Operators<4> + +5.21.28.12 Logical operators +............................ + +Network objects can be compared with the usual set of logical operators. +Network objects are ordered first by network address, then by net mask. + + +File: python.info, Node: Iteration<2>, Next: Networks as containers of addresses, Prev: Logical operators, Up: Operators<4> + +5.21.28.13 Iteration +.................... + +Network objects can be iterated to list all the addresses belonging to +the network. For iteration, `all' hosts are returned, including +unusable hosts (for usable hosts, use the *note hosts(): 2c58. method). +An example: + + >>> for addr in IPv4Network('192.0.2.0/28'): + ... addr + ... + IPv4Address('192.0.2.0') + IPv4Address('192.0.2.1') + IPv4Address('192.0.2.2') + IPv4Address('192.0.2.3') + IPv4Address('192.0.2.4') + IPv4Address('192.0.2.5') + IPv4Address('192.0.2.6') + IPv4Address('192.0.2.7') + IPv4Address('192.0.2.8') + IPv4Address('192.0.2.9') + IPv4Address('192.0.2.10') + IPv4Address('192.0.2.11') + IPv4Address('192.0.2.12') + IPv4Address('192.0.2.13') + IPv4Address('192.0.2.14') + IPv4Address('192.0.2.15') + + +File: python.info, Node: Networks as containers of addresses, Prev: Iteration<2>, Up: Operators<4> + +5.21.28.14 Networks as containers of addresses +.............................................. + +Network objects can act as containers of addresses. Some examples: + + >>> IPv4Network('192.0.2.0/28')[0] + IPv4Address('192.0.2.0') + >>> IPv4Network('192.0.2.0/28')[15] + IPv4Address('192.0.2.15') + >>> IPv4Address('192.0.2.6') in IPv4Network('192.0.2.0/28') + True + >>> IPv4Address('192.0.3.6') in IPv4Network('192.0.2.0/28') + False + + +File: python.info, Node: Interface objects, Next: Other Module Level Functions, Prev: IP Network definitions, Up: ipaddress — IPv4/IPv6 manipulation library + +5.21.28.15 Interface objects +............................ + +Interface objects are *note hashable: 10c8, so they can be used as keys +in dictionaries. + + -- Class: ipaddress.IPv4Interface (address) + + Construct an IPv4 interface. The meaning of `address' is as in the + constructor of *note IPv4Network: 3a0, except that arbitrary host + addresses are always accepted. + + *note IPv4Interface: 2c1b. is a subclass of *note IPv4Address: + 2c17, so it inherits all the attributes from that class. In + addition, the following attributes are available: + + -- Attribute: ip + + The address (*note IPv4Address: 2c17.) without network + information. + + >>> interface = IPv4Interface('192.0.2.5/24') + >>> interface.ip + IPv4Address('192.0.2.5') + + -- Attribute: network + + The network (*note IPv4Network: 3a0.) this interface belongs + to. + + >>> interface = IPv4Interface('192.0.2.5/24') + >>> interface.network + IPv4Network('192.0.2.0/24') + + -- Attribute: with_prefixlen + + A string representation of the interface with the mask in + prefix notation. + + >>> interface = IPv4Interface('192.0.2.5/24') + >>> interface.with_prefixlen + '192.0.2.5/24' + + -- Attribute: with_netmask + + A string representation of the interface with the network as a + net mask. + + >>> interface = IPv4Interface('192.0.2.5/24') + >>> interface.with_netmask + '192.0.2.5/255.255.255.0' + + -- Attribute: with_hostmask + + A string representation of the interface with the network as a + host mask. + + >>> interface = IPv4Interface('192.0.2.5/24') + >>> interface.with_hostmask + '192.0.2.5/0.0.0.255' + + -- Class: ipaddress.IPv6Interface (address) + + Construct an IPv6 interface. The meaning of `address' is as in the + constructor of *note IPv6Network: 39f, except that arbitrary host + addresses are always accepted. + + *note IPv6Interface: 2c1c. is a subclass of *note IPv6Address: + 2c18, so it inherits all the attributes from that class. In + addition, the following attributes are available: + + -- Attribute: ip + + -- Attribute: network + + -- Attribute: with_prefixlen + + -- Attribute: with_netmask + + -- Attribute: with_hostmask + + Refer to the corresponding attribute documentation in *note + IPv4Interface: 2c1b. + +* Menu: + +* Operators: Operators<5>. + + +File: python.info, Node: Operators<5>, Up: Interface objects + +5.21.28.16 Operators +.................... + +Interface objects support some operators. Unless stated otherwise, +operators can only be applied between compatible objects (i.e. IPv4 +with IPv4, IPv6 with IPv6). + +* Menu: + +* Logical operators: Logical operators<2>. + + +File: python.info, Node: Logical operators<2>, Up: Operators<5> + +5.21.28.17 Logical operators +............................ + +Interface objects can be compared with the usual set of logical +operators. + +For equality comparison (‘==’ and ‘!=’), both the IP address and network +must be the same for the objects to be equal. An interface will not +compare equal to any address or network object. + +For ordering (‘<’, ‘>’, etc) the rules are different. Interface and +address objects with the same IP version can be compared, and the +address objects will always sort before the interface objects. Two +interface objects are first compared by their networks and, if those are +the same, then by their IP addresses. + + +File: python.info, Node: Other Module Level Functions, Next: Custom Exceptions, Prev: Interface objects, Up: ipaddress — IPv4/IPv6 manipulation library + +5.21.28.18 Other Module Level Functions +....................................... + +The module also provides the following module level functions: + + -- Function: ipaddress.v4_int_to_packed (address) + + Represent an address as 4 packed bytes in network (big-endian) + order. `address' is an integer representation of an IPv4 IP + address. A *note ValueError: 1fb. is raised if the integer is + negative or too large to be an IPv4 IP address. + + >>> ipaddress.ip_address(3221225985) + IPv4Address('192.0.2.1') + >>> ipaddress.v4_int_to_packed(3221225985) + b'\xc0\x00\x02\x01' + + -- Function: ipaddress.v6_int_to_packed (address) + + Represent an address as 16 packed bytes in network (big-endian) + order. `address' is an integer representation of an IPv6 IP + address. A *note ValueError: 1fb. is raised if the integer is + negative or too large to be an IPv6 IP address. + + -- Function: ipaddress.summarize_address_range (first, last) + + Return an iterator of the summarized network range given the first + and last IP addresses. `first' is the first *note IPv4Address: + 2c17. or *note IPv6Address: 2c18. in the range and `last' is the + last *note IPv4Address: 2c17. or *note IPv6Address: 2c18. in the + range. A *note TypeError: 192. is raised if `first' or `last' are + not IP addresses or are not of the same version. A *note + ValueError: 1fb. is raised if `last' is not greater than `first' or + if `first' address version is not 4 or 6. + + >>> [ipaddr for ipaddr in ipaddress.summarize_address_range( + ... ipaddress.IPv4Address('192.0.2.0'), + ... ipaddress.IPv4Address('192.0.2.130'))] + [IPv4Network('192.0.2.0/25'), IPv4Network('192.0.2.128/31'), IPv4Network('192.0.2.130/32')] + + -- Function: ipaddress.collapse_addresses (addresses) + + Return an iterator of the collapsed *note IPv4Network: 3a0. or + *note IPv6Network: 39f. objects. `addresses' is an iterator of + *note IPv4Network: 3a0. or *note IPv6Network: 39f. objects. A + *note TypeError: 192. is raised if `addresses' contains mixed + version objects. + + >>> [ipaddr for ipaddr in + ... ipaddress.collapse_addresses([ipaddress.IPv4Network('192.0.2.0/25'), + ... ipaddress.IPv4Network('192.0.2.128/25')])] + [IPv4Network('192.0.2.0/24')] + + -- Function: ipaddress.get_mixed_type_key (obj) + + Return a key suitable for sorting between networks and addresses. + Address and Network objects are not sortable by default; they’re + fundamentally different, so the expression: + + IPv4Address('192.0.2.0') <= IPv4Network('192.0.2.0/24') + + doesn’t make sense. There are some times however, where you may + wish to have *note ipaddress: a2. sort these anyway. If you need + to do this, you can use this function as the `key' argument to + *note sorted(): 444. + + `obj' is either a network or address object. + + +File: python.info, Node: Custom Exceptions, Prev: Other Module Level Functions, Up: ipaddress — IPv4/IPv6 manipulation library + +5.21.28.19 Custom Exceptions +............................ + +To support more specific error reporting from class constructors, the +module defines the following exceptions: + + -- Exception: ipaddress.AddressValueError (ValueError) + + Any value error related to the address. + + -- Exception: ipaddress.NetmaskValueError (ValueError) + + Any value error related to the net mask. + + +File: python.info, Node: Multimedia Services, Next: Internationalization, Prev: Internet Protocols and Support, Up: The Python Standard Library + +5.22 Multimedia Services +======================== + +The modules described in this chapter implement various algorithms or +interfaces that are mainly useful for multimedia applications. They are +available at the discretion of the installation. Here’s an overview: + +* Menu: + +* audioop — Manipulate raw audio data:: +* aifc — Read and write AIFF and AIFC files:: +* sunau — Read and write Sun AU files:: +* wave — Read and write WAV files:: +* chunk — Read IFF chunked data:: +* colorsys — Conversions between color systems:: +* imghdr — Determine the type of an image:: +* sndhdr — Determine type of sound file:: +* ossaudiodev — Access to OSS-compatible audio devices:: + + +File: python.info, Node: audioop — Manipulate raw audio data, Next: aifc — Read and write AIFF and AIFC files, Up: Multimedia Services + +5.22.1 ‘audioop’ — Manipulate raw audio data +-------------------------------------------- + +__________________________________________________________________ + +The *note audioop: d. module contains some useful operations on sound +fragments. It operates on sound fragments consisting of signed integer +samples 8, 16, 24 or 32 bits wide, stored in *note bytes-like objects: +5e8. All scalar items are integers, unless specified otherwise. + +Changed in version 3.4: Support for 24-bit samples was added. All +functions now accept any *note bytes-like object: 5e8. String input now +results in an immediate error. + +This module provides support for a-LAW, u-LAW and Intel/DVI ADPCM +encodings. + +A few of the more complicated operations only take 16-bit samples, +otherwise the sample size (in bytes) is always a parameter of the +operation. + +The module defines the following variables and functions: + + -- Exception: audioop.error + + This exception is raised on all errors, such as unknown number of + bytes per sample, etc. + + -- Function: audioop.add (fragment1, fragment2, width) + + Return a fragment which is the addition of the two samples passed + as parameters. `width' is the sample width in bytes, either ‘1’, + ‘2’, ‘3’ or ‘4’. Both fragments should have the same length. + Samples are truncated in case of overflow. + + -- Function: audioop.adpcm2lin (adpcmfragment, width, state) + + Decode an Intel/DVI ADPCM coded fragment to a linear fragment. See + the description of *note lin2adpcm(): 2c98. for details on ADPCM + coding. Return a tuple ‘(sample, newstate)’ where the sample has + the width specified in `width'. + + -- Function: audioop.alaw2lin (fragment, width) + + Convert sound fragments in a-LAW encoding to linearly encoded sound + fragments. a-LAW encoding always uses 8 bits samples, so `width' + refers only to the sample width of the output fragment here. + + -- Function: audioop.avg (fragment, width) + + Return the average over all samples in the fragment. + + -- Function: audioop.avgpp (fragment, width) + + Return the average peak-peak value over all samples in the + fragment. No filtering is done, so the usefulness of this routine + is questionable. + + -- Function: audioop.bias (fragment, width, bias) + + Return a fragment that is the original fragment with a bias added + to each sample. Samples wrap around in case of overflow. + + -- Function: audioop.byteswap (fragment, width) + + “Byteswap” all samples in a fragment and returns the modified + fragment. Converts big-endian samples to little-endian and vice + versa. + + New in version 3.4. + + -- Function: audioop.cross (fragment, width) + + Return the number of zero crossings in the fragment passed as an + argument. + + -- Function: audioop.findfactor (fragment, reference) + + Return a factor `F' such that ‘rms(add(fragment, mul(reference, + -F)))’ is minimal, i.e., return the factor with which you should + multiply `reference' to make it match as well as possible to + `fragment'. The fragments should both contain 2-byte samples. + + The time taken by this routine is proportional to ‘len(fragment)’. + + -- Function: audioop.findfit (fragment, reference) + + Try to match `reference' as well as possible to a portion of + `fragment' (which should be the longer fragment). This is + (conceptually) done by taking slices out of `fragment', using *note + findfactor(): 2c9e. to compute the best match, and minimizing the + result. The fragments should both contain 2-byte samples. Return + a tuple ‘(offset, factor)’ where `offset' is the (integer) offset + into `fragment' where the optimal match started and `factor' is the + (floating-point) factor as per *note findfactor(): 2c9e. + + -- Function: audioop.findmax (fragment, length) + + Search `fragment' for a slice of length `length' samples (not + bytes!) with maximum energy, i.e., return `i' for which + ‘rms(fragment[i*2:(i+length)*2])’ is maximal. The fragments should + both contain 2-byte samples. + + The routine takes time proportional to ‘len(fragment)’. + + -- Function: audioop.getsample (fragment, width, index) + + Return the value of sample `index' from the fragment. + + -- Function: audioop.lin2adpcm (fragment, width, state) + + Convert samples to 4 bit Intel/DVI ADPCM encoding. ADPCM coding is + an adaptive coding scheme, whereby each 4 bit number is the + difference between one sample and the next, divided by a (varying) + step. The Intel/DVI ADPCM algorithm has been selected for use by + the IMA, so it may well become a standard. + + `state' is a tuple containing the state of the coder. The coder + returns a tuple ‘(adpcmfrag, newstate)’, and the `newstate' should + be passed to the next call of *note lin2adpcm(): 2c98. In the + initial call, ‘None’ can be passed as the state. `adpcmfrag' is + the ADPCM coded fragment packed 2 4-bit values per byte. + + -- Function: audioop.lin2alaw (fragment, width) + + Convert samples in the audio fragment to a-LAW encoding and return + this as a bytes object. a-LAW is an audio encoding format whereby + you get a dynamic range of about 13 bits using only 8 bit samples. + It is used by the Sun audio hardware, among others. + + -- Function: audioop.lin2lin (fragment, width, newwidth) + + Convert samples between 1-, 2-, 3- and 4-byte formats. + + Note: In some audio formats, such as .WAV files, 16, 24 and 32 + bit samples are signed, but 8 bit samples are unsigned. So + when converting to 8 bit wide samples for these formats, you + need to also add 128 to the result: + + new_frames = audioop.lin2lin(frames, old_width, 1) + new_frames = audioop.bias(new_frames, 1, 128) + + The same, in reverse, has to be applied when converting from 8 + to 16, 24 or 32 bit width samples. + + -- Function: audioop.lin2ulaw (fragment, width) + + Convert samples in the audio fragment to u-LAW encoding and return + this as a bytes object. u-LAW is an audio encoding format whereby + you get a dynamic range of about 14 bits using only 8 bit samples. + It is used by the Sun audio hardware, among others. + + -- Function: audioop.max (fragment, width) + + Return the maximum of the `absolute value' of all samples in a + fragment. + + -- Function: audioop.maxpp (fragment, width) + + Return the maximum peak-peak value in the sound fragment. + + -- Function: audioop.minmax (fragment, width) + + Return a tuple consisting of the minimum and maximum values of all + samples in the sound fragment. + + -- Function: audioop.mul (fragment, width, factor) + + Return a fragment that has all samples in the original fragment + multiplied by the floating-point value `factor'. Samples are + truncated in case of overflow. + + -- Function: audioop.ratecv (fragment, width, nchannels, inrate, + outrate, state[, weightA[, weightB]]) + + Convert the frame rate of the input fragment. + + `state' is a tuple containing the state of the converter. The + converter returns a tuple ‘(newfragment, newstate)’, and `newstate' + should be passed to the next call of *note ratecv(): 2ca9. The + initial call should pass ‘None’ as the state. + + The `weightA' and `weightB' arguments are parameters for a simple + digital filter and default to ‘1’ and ‘0’ respectively. + + -- Function: audioop.reverse (fragment, width) + + Reverse the samples in a fragment and returns the modified + fragment. + + -- Function: audioop.rms (fragment, width) + + Return the root-mean-square of the fragment, i.e. + ‘sqrt(sum(S_i^2)/n)’. + + This is a measure of the power in an audio signal. + + -- Function: audioop.tomono (fragment, width, lfactor, rfactor) + + Convert a stereo fragment to a mono fragment. The left channel is + multiplied by `lfactor' and the right channel by `rfactor' before + adding the two channels to give a mono signal. + + -- Function: audioop.tostereo (fragment, width, lfactor, rfactor) + + Generate a stereo fragment from a mono fragment. Each pair of + samples in the stereo fragment are computed from the mono sample, + whereby left channel samples are multiplied by `lfactor' and right + channel samples by `rfactor'. + + -- Function: audioop.ulaw2lin (fragment, width) + + Convert sound fragments in u-LAW encoding to linearly encoded sound + fragments. u-LAW encoding always uses 8 bits samples, so `width' + refers only to the sample width of the output fragment here. + +Note that operations such as *note mul(): 2ca8. or *note max(): 2ca5. +make no distinction between mono and stereo fragments, i.e. all samples +are treated equal. If this is a problem the stereo fragment should be +split into two mono fragments first and recombined later. Here is an +example of how to do that: + + def mul_stereo(sample, width, lfactor, rfactor): + lsample = audioop.tomono(sample, width, 1, 0) + rsample = audioop.tomono(sample, width, 0, 1) + lsample = audioop.mul(lsample, width, lfactor) + rsample = audioop.mul(rsample, width, rfactor) + lsample = audioop.tostereo(lsample, width, 1, 0) + rsample = audioop.tostereo(rsample, width, 0, 1) + return audioop.add(lsample, rsample, width) + +If you use the ADPCM coder to build network packets and you want your +protocol to be stateless (i.e. to be able to tolerate packet loss) you +should not only transmit the data but also the state. Note that you +should send the `initial' state (the one you passed to *note +lin2adpcm(): 2c98.) along to the decoder, not the final state (as +returned by the coder). If you want to use *note struct.Struct: 14b5. +to store the state in binary you can code the first element (the +predicted value) in 16 bits and the second (the delta index) in 8. + +The ADPCM coders have never been tried against other ADPCM coders, only +against themselves. It could well be that I misinterpreted the +standards in which case they will not be interoperable with the +respective standards. + +The ‘find*()’ routines might look a bit funny at first sight. They are +primarily meant to do echo cancellation. A reasonably fast way to do +this is to pick the most energetic piece of the output sample, locate +that in the input sample and subtract the whole output sample from the +input sample: + + def echocancel(outputdata, inputdata): + pos = audioop.findmax(outputdata, 800) # one tenth second + out_test = outputdata[pos*2:] + in_test = inputdata[pos*2:] + ipos, factor = audioop.findfit(in_test, out_test) + # Optional (for better cancellation): + # factor = audioop.findfactor(in_test[ipos*2:ipos*2+len(out_test)], + # out_test) + prefill = '\0'*(pos+ipos)*2 + postfill = '\0'*(len(inputdata)-len(prefill)-len(outputdata)) + outputdata = prefill + audioop.mul(outputdata, 2, -factor) + postfill + return audioop.add(inputdata, outputdata, 2) + + +File: python.info, Node: aifc — Read and write AIFF and AIFC files, Next: sunau — Read and write Sun AU files, Prev: audioop — Manipulate raw audio data, Up: Multimedia Services + +5.22.2 ‘aifc’ — Read and write AIFF and AIFC files +-------------------------------------------------- + +`Source code:' Lib/aifc.py(1) + +__________________________________________________________________ + +This module provides support for reading and writing AIFF and AIFF-C +files. AIFF is Audio Interchange File Format, a format for storing +digital audio samples in a file. AIFF-C is a newer version of the +format that includes the ability to compress the audio data. + +Audio files have a number of parameters that describe the audio data. +The sampling rate or frame rate is the number of times per second the +sound is sampled. The number of channels indicate if the audio is mono, +stereo, or quadro. Each frame consists of one sample per channel. The +sample size is the size in bytes of each sample. Thus a frame consists +of ‘nchannels * samplesize’ bytes, and a second’s worth of audio +consists of ‘nchannels * samplesize * framerate’ bytes. + +For example, CD quality audio has a sample size of two bytes (16 bits), +uses two channels (stereo) and has a frame rate of 44,100 frames/second. +This gives a frame size of 4 bytes (2*2), and a second’s worth occupies +2*2*44100 bytes (176,400 bytes). + +Module *note aifc: 5. defines the following function: + + -- Function: aifc.open (file, mode=None) + + Open an AIFF or AIFF-C file and return an object instance with + methods that are described below. The argument `file' is either a + string naming a file or a *note file object: b42. `mode' must be + ‘'r'’ or ‘'rb'’ when the file must be opened for reading, or ‘'w'’ + or ‘'wb'’ when the file must be opened for writing. If omitted, + ‘file.mode’ is used if it exists, otherwise ‘'rb'’ is used. When + used for writing, the file object should be seekable, unless you + know ahead of time how many samples you are going to write in total + and use ‘writeframesraw()’ and ‘setnframes()’. The *note open(): + 45b. function may be used in a *note with: 6e9. statement. When + the ‘with’ block completes, the *note close(): 81c. method is + called. + + Changed in version 3.4: Support for the *note with: 6e9. statement + was added. + +Objects returned by *note open(): 45b. when a file is opened for reading +have the following methods: + + -- Method: aifc.getnchannels () + + Return the number of audio channels (1 for mono, 2 for stereo). + + -- Method: aifc.getsampwidth () + + Return the size in bytes of individual samples. + + -- Method: aifc.getframerate () + + Return the sampling rate (number of audio frames per second). + + -- Method: aifc.getnframes () + + Return the number of audio frames in the file. + + -- Method: aifc.getcomptype () + + Return a bytes array of length 4 describing the type of compression + used in the audio file. For AIFF files, the returned value is + ‘b'NONE'’. + + -- Method: aifc.getcompname () + + Return a bytes array convertible to a human-readable description of + the type of compression used in the audio file. For AIFF files, + the returned value is ‘b'not compressed'’. + + -- Method: aifc.getparams () + + Returns a *note namedtuple(): 1b7. ‘(nchannels, sampwidth, + framerate, nframes, comptype, compname)’, equivalent to output of + the ‘get*()’ methods. + + -- Method: aifc.getmarkers () + + Return a list of markers in the audio file. A marker consists of a + tuple of three elements. The first is the mark ID (an integer), + the second is the mark position in frames from the beginning of the + data (an integer), the third is the name of the mark (a string). + + -- Method: aifc.getmark (id) + + Return the tuple as described in *note getmarkers(): 2cb7. for the + mark with the given `id'. + + -- Method: aifc.readframes (nframes) + + Read and return the next `nframes' frames from the audio file. The + returned data is a string containing for each frame the + uncompressed samples of all channels. + + -- Method: aifc.rewind () + + Rewind the read pointer. The next *note readframes(): 2cb9. will + start from the beginning. + + -- Method: aifc.setpos (pos) + + Seek to the specified frame number. + + -- Method: aifc.tell () + + Return the current frame number. + + -- Method: aifc.close () + + Close the AIFF file. After calling this method, the object can no + longer be used. + +Objects returned by *note open(): 45b. when a file is opened for writing +have all the above methods, except for ‘readframes()’ and ‘setpos()’. +In addition the following methods exist. The ‘get*()’ methods can only +be called after the corresponding ‘set*()’ methods have been called. +Before the first ‘writeframes()’ or ‘writeframesraw()’, all parameters +except for the number of frames must be filled in. + + -- Method: aifc.aiff () + + Create an AIFF file. The default is that an AIFF-C file is + created, unless the name of the file ends in ‘'.aiff'’ in which + case the default is an AIFF file. + + -- Method: aifc.aifc () + + Create an AIFF-C file. The default is that an AIFF-C file is + created, unless the name of the file ends in ‘'.aiff'’ in which + case the default is an AIFF file. + + -- Method: aifc.setnchannels (nchannels) + + Specify the number of channels in the audio file. + + -- Method: aifc.setsampwidth (width) + + Specify the size in bytes of audio samples. + + -- Method: aifc.setframerate (rate) + + Specify the sampling frequency in frames per second. + + -- Method: aifc.setnframes (nframes) + + Specify the number of frames that are to be written to the audio + file. If this parameter is not set, or not set correctly, the file + needs to support seeking. + + -- Method: aifc.setcomptype (type, name) + + Specify the compression type. If not specified, the audio data + will not be compressed. In AIFF files, compression is not + possible. The name parameter should be a human-readable + description of the compression type as a bytes array, the type + parameter should be a bytes array of length 4. Currently the + following compression types are supported: ‘b'NONE'’, ‘b'ULAW'’, + ‘b'ALAW'’, ‘b'G722'’. + + -- Method: aifc.setparams (nchannels, sampwidth, framerate, comptype, + compname) + + Set all the above parameters at once. The argument is a tuple + consisting of the various parameters. This means that it is + possible to use the result of a *note getparams(): 81b. call as + argument to *note setparams(): 2cc4. + + -- Method: aifc.setmark (id, pos, name) + + Add a mark with the given id (larger than 0), and the given name at + the given position. This method can be called at any time before + *note close(): 81c. + + -- Method: aifc.tell () + + Return the current write position in the output file. Useful in + combination with *note setmark(): 2cc5. + + -- Method: aifc.writeframes (data) + + Write data to the output file. This method can only be called + after the audio file parameters have been set. + + Changed in version 3.4: Any *note bytes-like object: 5e8. is now + accepted. + + -- Method: aifc.writeframesraw (data) + + Like *note writeframes(): 81e, except that the header of the audio + file is not updated. + + Changed in version 3.4: Any *note bytes-like object: 5e8. is now + accepted. + + -- Method: aifc.close () + + Close the AIFF file. The header of the file is updated to reflect + the actual size of the audio data. After calling this method, the + object can no longer be used. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/aifc.py + + +File: python.info, Node: sunau — Read and write Sun AU files, Next: wave — Read and write WAV files, Prev: aifc — Read and write AIFF and AIFC files, Up: Multimedia Services + +5.22.3 ‘sunau’ — Read and write Sun AU files +-------------------------------------------- + +`Source code:' Lib/sunau.py(1) + +__________________________________________________________________ + +The *note sunau: fa. module provides a convenient interface to the Sun +AU sound format. Note that this module is interface-compatible with the +modules *note aifc: 5. and *note wave: 127. + +An audio file consists of a header followed by the data. The fields of +the header are: + +Field Contents + +------------------------------------------------------------------------ + +magic word The four bytes ‘.snd’. + + +header size Size of the header, including info, in bytes. + + +data size Physical size of the data, in bytes. + + +encoding Indicates how the audio samples are encoded. + + +sample rate The sampling rate. + + +# of channels The number of channels in the samples. + + +info ASCII string giving a description of the audio + file (padded with null bytes). + + +Apart from the info field, all header fields are 4 bytes in size. They +are all 32-bit unsigned integers encoded in big-endian byte order. + +The *note sunau: fa. module defines the following functions: + + -- Function: sunau.open (file, mode) + + If `file' is a string, open the file by that name, otherwise treat + it as a seekable file-like object. `mode' can be any of + + ‘'r'’ + + Read only mode. + + ‘'w'’ + + Write only mode. + + Note that it does not allow read/write files. + + A `mode' of ‘'r'’ returns an ‘AU_read’ object, while a `mode' of + ‘'w'’ or ‘'wb'’ returns an ‘AU_write’ object. + + -- Function: sunau.openfp (file, mode) + + A synonym for *note open(): 472, maintained for backwards + compatibility. + + Deprecated since version 3.7, will be removed in version 3.9. + +The *note sunau: fa. module defines the following exception: + + -- Exception: sunau.Error + + An error raised when something is impossible because of Sun AU + specs or implementation deficiency. + +The *note sunau: fa. module defines the following data items: + + -- Data: sunau.AUDIO_FILE_MAGIC + + An integer every valid Sun AU file begins with, stored in + big-endian form. This is the string ‘.snd’ interpreted as an + integer. + + -- Data: sunau.AUDIO_FILE_ENCODING_MULAW_8 + -- Data: sunau.AUDIO_FILE_ENCODING_LINEAR_8 + -- Data: sunau.AUDIO_FILE_ENCODING_LINEAR_16 + -- Data: sunau.AUDIO_FILE_ENCODING_LINEAR_24 + -- Data: sunau.AUDIO_FILE_ENCODING_LINEAR_32 + -- Data: sunau.AUDIO_FILE_ENCODING_ALAW_8 + + Values of the encoding field from the AU header which are supported + by this module. + + -- Data: sunau.AUDIO_FILE_ENCODING_FLOAT + -- Data: sunau.AUDIO_FILE_ENCODING_DOUBLE + -- Data: sunau.AUDIO_FILE_ENCODING_ADPCM_G721 + -- Data: sunau.AUDIO_FILE_ENCODING_ADPCM_G722 + -- Data: sunau.AUDIO_FILE_ENCODING_ADPCM_G723_3 + -- Data: sunau.AUDIO_FILE_ENCODING_ADPCM_G723_5 + + Additional known values of the encoding field from the AU header, + but which are not supported by this module. + +* Menu: + +* AU_read Objects:: +* AU_write Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/sunau.py + + +File: python.info, Node: AU_read Objects, Next: AU_write Objects, Up: sunau — Read and write Sun AU files + +5.22.3.1 AU_read Objects +........................ + +AU_read objects, as returned by *note open(): 472. above, have the +following methods: + + -- Method: AU_read.close () + + Close the stream, and make the instance unusable. (This is called + automatically on deletion.) + + -- Method: AU_read.getnchannels () + + Returns number of audio channels (1 for mono, 2 for stereo). + + -- Method: AU_read.getsampwidth () + + Returns sample width in bytes. + + -- Method: AU_read.getframerate () + + Returns sampling frequency. + + -- Method: AU_read.getnframes () + + Returns number of audio frames. + + -- Method: AU_read.getcomptype () + + Returns compression type. Supported compression types are + ‘'ULAW'’, ‘'ALAW'’ and ‘'NONE'’. + + -- Method: AU_read.getcompname () + + Human-readable version of *note getcomptype(): 2cdd. The supported + types have the respective names ‘'CCITT G.711 u-law'’, ‘'CCITT + G.711 A-law'’ and ‘'not compressed'’. + + -- Method: AU_read.getparams () + + Returns a *note namedtuple(): 1b7. ‘(nchannels, sampwidth, + framerate, nframes, comptype, compname)’, equivalent to output of + the ‘get*()’ methods. + + -- Method: AU_read.readframes (n) + + Reads and returns at most `n' frames of audio, as a *note bytes: + 331. object. The data will be returned in linear format. If the + original data is in u-LAW format, it will be converted. + + -- Method: AU_read.rewind () + + Rewind the file pointer to the beginning of the audio stream. + +The following two methods define a term “position” which is compatible +between them, and is otherwise implementation dependent. + + -- Method: AU_read.setpos (pos) + + Set the file pointer to the specified position. Only values + returned from *note tell(): 2ce3. should be used for `pos'. + + -- Method: AU_read.tell () + + Return current file pointer position. Note that the returned value + has nothing to do with the actual position in the file. + +The following two functions are defined for compatibility with the *note +aifc: 5, and don’t do anything interesting. + + -- Method: AU_read.getmarkers () + + Returns ‘None’. + + -- Method: AU_read.getmark (id) + + Raise an error. + + +File: python.info, Node: AU_write Objects, Prev: AU_read Objects, Up: sunau — Read and write Sun AU files + +5.22.3.2 AU_write Objects +......................... + +AU_write objects, as returned by *note open(): 472. above, have the +following methods: + + -- Method: AU_write.setnchannels (n) + + Set the number of channels. + + -- Method: AU_write.setsampwidth (n) + + Set the sample width (in bytes.) + + Changed in version 3.4: Added support for 24-bit samples. + + -- Method: AU_write.setframerate (n) + + Set the frame rate. + + -- Method: AU_write.setnframes (n) + + Set the number of frames. This can be later changed, when and if + more frames are written. + + -- Method: AU_write.setcomptype (type, name) + + Set the compression type and description. Only ‘'NONE'’ and + ‘'ULAW'’ are supported on output. + + -- Method: AU_write.setparams (tuple) + + The `tuple' should be ‘(nchannels, sampwidth, framerate, nframes, + comptype, compname)’, with values valid for the ‘set*()’ methods. + Set all parameters. + + -- Method: AU_write.tell () + + Return current position in the file, with the same disclaimer for + the *note AU_read.tell(): 2ce3. and *note AU_read.setpos(): 2ce2. + methods. + + -- Method: AU_write.writeframesraw (data) + + Write audio frames, without correcting `nframes'. + + Changed in version 3.4: Any *note bytes-like object: 5e8. is now + accepted. + + -- Method: AU_write.writeframes (data) + + Write audio frames and make sure `nframes' is correct. + + Changed in version 3.4: Any *note bytes-like object: 5e8. is now + accepted. + + -- Method: AU_write.close () + + Make sure `nframes' is correct, and close the file. + + This method is called upon deletion. + +Note that it is invalid to set any parameters after calling +‘writeframes()’ or ‘writeframesraw()’. + + +File: python.info, Node: wave — Read and write WAV files, Next: chunk — Read IFF chunked data, Prev: sunau — Read and write Sun AU files, Up: Multimedia Services + +5.22.4 ‘wave’ — Read and write WAV files +---------------------------------------- + +`Source code:' Lib/wave.py(1) + +__________________________________________________________________ + +The *note wave: 127. module provides a convenient interface to the WAV +sound format. It does not support compression/decompression, but it +does support mono/stereo. + +The *note wave: 127. module defines the following function and +exception: + + -- Function: wave.open (file, mode=None) + + If `file' is a string, open the file by that name, otherwise treat + it as a file-like object. `mode' can be: + + ‘'rb'’ + + Read only mode. + + ‘'wb'’ + + Write only mode. + + Note that it does not allow read/write WAV files. + + A `mode' of ‘'rb'’ returns a ‘Wave_read’ object, while a `mode' of + ‘'wb'’ returns a ‘Wave_write’ object. If `mode' is omitted and a + file-like object is passed as `file', ‘file.mode’ is used as the + default value for `mode'. + + If you pass in a file-like object, the wave object will not close + it when its ‘close()’ method is called; it is the caller’s + responsibility to close the file object. + + The *note open(): 476. function may be used in a *note with: 6e9. + statement. When the ‘with’ block completes, the *note + Wave_read.close(): 2cf1. or *note Wave_write.close(): 2cf2. method + is called. + + Changed in version 3.4: Added support for unseekable files. + + -- Function: wave.openfp (file, mode) + + A synonym for *note open(): 476, maintained for backwards + compatibility. + + Deprecated since version 3.7, will be removed in version 3.9. + + -- Exception: wave.Error + + An error raised when something is impossible because it violates + the WAV specification or hits an implementation deficiency. + +* Menu: + +* Wave_read Objects:: +* Wave_write Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/wave.py + + +File: python.info, Node: Wave_read Objects, Next: Wave_write Objects, Up: wave — Read and write WAV files + +5.22.4.1 Wave_read Objects +.......................... + +Wave_read objects, as returned by *note open(): 476, have the following +methods: + + -- Method: Wave_read.close () + + Close the stream if it was opened by *note wave: 127, and make the + instance unusable. This is called automatically on object + collection. + + -- Method: Wave_read.getnchannels () + + Returns number of audio channels (‘1’ for mono, ‘2’ for stereo). + + -- Method: Wave_read.getsampwidth () + + Returns sample width in bytes. + + -- Method: Wave_read.getframerate () + + Returns sampling frequency. + + -- Method: Wave_read.getnframes () + + Returns number of audio frames. + + -- Method: Wave_read.getcomptype () + + Returns compression type (‘'NONE'’ is the only supported type). + + -- Method: Wave_read.getcompname () + + Human-readable version of *note getcomptype(): 2cfa. Usually ‘'not + compressed'’ parallels ‘'NONE'’. + + -- Method: Wave_read.getparams () + + Returns a *note namedtuple(): 1b7. ‘(nchannels, sampwidth, + framerate, nframes, comptype, compname)’, equivalent to output of + the ‘get*()’ methods. + + -- Method: Wave_read.readframes (n) + + Reads and returns at most `n' frames of audio, as a *note bytes: + 331. object. + + -- Method: Wave_read.rewind () + + Rewind the file pointer to the beginning of the audio stream. + +The following two methods are defined for compatibility with the *note +aifc: 5. module, and don’t do anything interesting. + + -- Method: Wave_read.getmarkers () + + Returns ‘None’. + + -- Method: Wave_read.getmark (id) + + Raise an error. + +The following two methods define a term “position” which is compatible +between them, and is otherwise implementation dependent. + + -- Method: Wave_read.setpos (pos) + + Set the file pointer to the specified position. + + -- Method: Wave_read.tell () + + Return current file pointer position. + + +File: python.info, Node: Wave_write Objects, Prev: Wave_read Objects, Up: wave — Read and write WAV files + +5.22.4.2 Wave_write Objects +........................... + +For seekable output streams, the ‘wave’ header will automatically be +updated to reflect the number of frames actually written. For +unseekable streams, the `nframes' value must be accurate when the first +frame data is written. An accurate `nframes' value can be achieved +either by calling *note setnframes(): 2d04. or *note setparams(): 2d05. +with the number of frames that will be written before *note close(): +2cf2. is called and then using *note writeframesraw(): 90c. to write the +frame data, or by calling *note writeframes(): 90d. with all of the +frame data to be written. In the latter case *note writeframes(): 90d. +will calculate the number of frames in the data and set `nframes' +accordingly before writing the frame data. + +Wave_write objects, as returned by *note open(): 476, have the following +methods: + +Changed in version 3.4: Added support for unseekable files. + + -- Method: Wave_write.close () + + Make sure `nframes' is correct, and close the file if it was opened + by *note wave: 127. This method is called upon object collection. + It will raise an exception if the output stream is not seekable and + `nframes' does not match the number of frames actually written. + + -- Method: Wave_write.setnchannels (n) + + Set the number of channels. + + -- Method: Wave_write.setsampwidth (n) + + Set the sample width to `n' bytes. + + -- Method: Wave_write.setframerate (n) + + Set the frame rate to `n'. + + Changed in version 3.2: A non-integral input to this method is + rounded to the nearest integer. + + -- Method: Wave_write.setnframes (n) + + Set the number of frames to `n'. This will be changed later if the + number of frames actually written is different (this update attempt + will raise an error if the output stream is not seekable). + + -- Method: Wave_write.setcomptype (type, name) + + Set the compression type and description. At the moment, only + compression type ‘NONE’ is supported, meaning no compression. + + -- Method: Wave_write.setparams (tuple) + + The `tuple' should be ‘(nchannels, sampwidth, framerate, nframes, + comptype, compname)’, with values valid for the ‘set*()’ methods. + Sets all parameters. + + -- Method: Wave_write.tell () + + Return current position in the file, with the same disclaimer for + the *note Wave_read.tell(): 2d02. and *note Wave_read.setpos(): + 2d01. methods. + + -- Method: Wave_write.writeframesraw (data) + + Write audio frames, without correcting `nframes'. + + Changed in version 3.4: Any *note bytes-like object: 5e8. is now + accepted. + + -- Method: Wave_write.writeframes (data) + + Write audio frames and make sure `nframes' is correct. It will + raise an error if the output stream is not seekable and the total + number of frames that have been written after `data' has been + written does not match the previously set value for `nframes'. + + Changed in version 3.4: Any *note bytes-like object: 5e8. is now + accepted. + +Note that it is invalid to set any parameters after calling +‘writeframes()’ or ‘writeframesraw()’, and any attempt to do so will +raise *note wave.Error: 2cf3. + + +File: python.info, Node: chunk — Read IFF chunked data, Next: colorsys — Conversions between color systems, Prev: wave — Read and write WAV files, Up: Multimedia Services + +5.22.5 ‘chunk’ — Read IFF chunked data +-------------------------------------- + +`Source code:' Lib/chunk.py(1) + +__________________________________________________________________ + +This module provides an interface for reading files that use EA IFF 85 +chunks. (2) This format is used in at least the Audio Interchange File +Format (AIFF/AIFF-C) and the Real Media File Format (RMFF). The WAVE +audio file format is closely related and can also be read using this +module. + +A chunk has the following structure: + +Offset Length Contents + +--------------------------------------------------------------- + +0 4 Chunk ID + + +4 4 Size of chunk in big-endian byte + order, not including the header + + +8 `n' Data bytes, where `n' is the size + given in the preceding field + + +8 + `n' 0 or 1 Pad byte needed if `n' is odd and + chunk alignment is used + + +The ID is a 4-byte string which identifies the type of chunk. + +The size field (a 32-bit value, encoded using big-endian byte order) +gives the size of the chunk data, not including the 8-byte header. + +Usually an IFF-type file consists of one or more chunks. The proposed +usage of the *note Chunk: 2d0d. class defined here is to instantiate an +instance at the start of each chunk and read from the instance until it +reaches the end, after which a new instance can be instantiated. At the +end of the file, creating a new instance will fail with an *note +EOFError: c6c. exception. + + -- Class: chunk.Chunk (file, align=True, bigendian=True, + inclheader=False) + + Class which represents a chunk. The `file' argument is expected to + be a file-like object. An instance of this class is specifically + allowed. The only method that is needed is ‘read()’. If the + methods *note seek(): 1a62. and *note tell(): 1a63. are present and + don’t raise an exception, they are also used. If these methods are + present and raise an exception, they are expected to not have + altered the object. If the optional argument `align' is true, + chunks are assumed to be aligned on 2-byte boundaries. If `align' + is false, no alignment is assumed. The default value is true. If + the optional argument `bigendian' is false, the chunk size is + assumed to be in little-endian order. This is needed for WAVE + audio files. The default value is true. If the optional argument + `inclheader' is true, the size given in the chunk header includes + the size of the header. The default value is false. + + A *note Chunk: 2d0d. object supports the following methods: + + -- Method: getname () + + Returns the name (ID) of the chunk. This is the first 4 bytes + of the chunk. + + -- Method: getsize () + + Returns the size of the chunk. + + -- Method: close () + + Close and skip to the end of the chunk. This does not close + the underlying file. + + The remaining methods will raise *note OSError: 1d3. if called + after the *note close(): 2d10. method has been called. Before + Python 3.3, they used to raise *note IOError: 992, now an alias of + *note OSError: 1d3. + + -- Method: isatty () + + Returns ‘False’. + + -- Method: seek (pos, whence=0) + + Set the chunk’s current position. The `whence' argument is + optional and defaults to ‘0’ (absolute file positioning); + other values are ‘1’ (seek relative to the current position) + and ‘2’ (seek relative to the file’s end). There is no return + value. If the underlying file does not allow seek, only + forward seeks are allowed. + + -- Method: tell () + + Return the current position into the chunk. + + -- Method: read (size=-1) + + Read at most `size' bytes from the chunk (less if the read + hits the end of the chunk before obtaining `size' bytes). If + the `size' argument is negative or omitted, read all data + until the end of the chunk. An empty bytes object is returned + when the end of the chunk is encountered immediately. + + -- Method: skip () + + Skip to the end of the chunk. All further calls to *note + read(): 2d14. for the chunk will return ‘b''’. If you are not + interested in the contents of the chunk, this method should be + called so that the file points to the start of the next chunk. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/chunk.py + + (2) (1) “EA IFF 85” Standard for Interchange Format Files, Jerry +Morrison, Electronic Arts, January 1985. + + +File: python.info, Node: colorsys — Conversions between color systems, Next: imghdr — Determine the type of an image, Prev: chunk — Read IFF chunked data, Up: Multimedia Services + +5.22.6 ‘colorsys’ — Conversions between color systems +----------------------------------------------------- + +`Source code:' Lib/colorsys.py(1) + +__________________________________________________________________ + +The *note colorsys: 20. module defines bidirectional conversions of +color values between colors expressed in the RGB (Red Green Blue) color +space used in computer monitors and three other coordinate systems: YIQ, +HLS (Hue Lightness Saturation) and HSV (Hue Saturation Value). +Coordinates in all of these color spaces are floating point values. In +the YIQ space, the Y coordinate is between 0 and 1, but the I and Q +coordinates can be positive or negative. In all other spaces, the +coordinates are all between 0 and 1. + +See also +........ + +More information about color spaces can be found at +‘http://poynton.ca/ColorFAQ.html’ and +‘https://www.cambridgeincolour.com/tutorials/color-spaces.htm’. + +The *note colorsys: 20. module defines the following functions: + + -- Function: colorsys.rgb_to_yiq (r, g, b) + + Convert the color from RGB coordinates to YIQ coordinates. + + -- Function: colorsys.yiq_to_rgb (y, i, q) + + Convert the color from YIQ coordinates to RGB coordinates. + + -- Function: colorsys.rgb_to_hls (r, g, b) + + Convert the color from RGB coordinates to HLS coordinates. + + -- Function: colorsys.hls_to_rgb (h, l, s) + + Convert the color from HLS coordinates to RGB coordinates. + + -- Function: colorsys.rgb_to_hsv (r, g, b) + + Convert the color from RGB coordinates to HSV coordinates. + + -- Function: colorsys.hsv_to_rgb (h, s, v) + + Convert the color from HSV coordinates to RGB coordinates. + +Example: + + >>> import colorsys + >>> colorsys.rgb_to_hsv(0.2, 0.4, 0.4) + (0.5, 0.5, 0.4) + >>> colorsys.hsv_to_rgb(0.5, 0.5, 0.4) + (0.2, 0.4, 0.4) + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/colorsys.py + + +File: python.info, Node: imghdr — Determine the type of an image, Next: sndhdr — Determine type of sound file, Prev: colorsys — Conversions between color systems, Up: Multimedia Services + +5.22.7 ‘imghdr’ — Determine the type of an image +------------------------------------------------ + +`Source code:' Lib/imghdr.py(1) + +__________________________________________________________________ + +The *note imghdr: 99. module determines the type of image contained in a +file or byte stream. + +The *note imghdr: 99. module defines the following function: + + -- Function: imghdr.what (filename, h=None) + + Tests the image data contained in the file named by `filename', and + returns a string describing the image type. If optional `h' is + provided, the `filename' is ignored and `h' is assumed to contain + the byte stream to test. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + +The following image types are recognized, as listed below with the +return value from *note what(): 6ed.: + +Value Image format + +--------------------------------------------------------- + +‘'rgb'’ SGI ImgLib Files + + +‘'gif'’ GIF 87a and 89a Files + + +‘'pbm'’ Portable Bitmap Files + + +‘'pgm'’ Portable Graymap Files + + +‘'ppm'’ Portable Pixmap Files + + +‘'tiff'’ TIFF Files + + +‘'rast'’ Sun Raster Files + + +‘'xbm'’ X Bitmap Files + + +‘'jpeg'’ JPEG data in JFIF or Exif formats + + +‘'bmp'’ BMP files + + +‘'png'’ Portable Network Graphics + + +‘'webp'’ WebP files + + +‘'exr'’ OpenEXR Files + + +New in version 3.5: The `exr' and `webp' formats were added. + +You can extend the list of file types *note imghdr: 99. can recognize by +appending to this variable: + + -- Data: imghdr.tests + + A list of functions performing the individual tests. Each function + takes two arguments: the byte-stream and an open file-like object. + When *note what(): 6ed. is called with a byte-stream, the file-like + object will be ‘None’. + + The test function should return a string describing the image type + if the test succeeded, or ‘None’ if it failed. + +Example: + + >>> import imghdr + >>> imghdr.what('bass.gif') + 'gif' + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/imghdr.py + + +File: python.info, Node: sndhdr — Determine type of sound file, Next: ossaudiodev — Access to OSS-compatible audio devices, Prev: imghdr — Determine the type of an image, Up: Multimedia Services + +5.22.8 ‘sndhdr’ — Determine type of sound file +---------------------------------------------- + +`Source code:' Lib/sndhdr.py(1) + +__________________________________________________________________ + +The *note sndhdr: ee. provides utility functions which attempt to +determine the type of sound data which is in a file. When these +functions are able to determine what type of sound data is stored in a +file, they return a *note namedtuple(): 1b7, containing five attributes: +(‘filetype’, ‘framerate’, ‘nchannels’, ‘nframes’, ‘sampwidth’). The +value for `type' indicates the data type and will be one of the strings +‘'aifc'’, ‘'aiff'’, ‘'au'’, ‘'hcom'’, ‘'sndr'’, ‘'sndt'’, ‘'voc'’, +‘'wav'’, ‘'8svx'’, ‘'sb'’, ‘'ub'’, or ‘'ul'’. The `sampling_rate' will +be either the actual value or ‘0’ if unknown or difficult to decode. +Similarly, `channels' will be either the number of channels or ‘0’ if it +cannot be determined or if the value is difficult to decode. The value +for `frames' will be either the number of frames or ‘-1’. The last item +in the tuple, `bits_per_sample', will either be the sample size in bits +or ‘'A'’ for A-LAW or ‘'U'’ for u-LAW. + + -- Function: sndhdr.what (filename) + + Determines the type of sound data stored in the file `filename' + using *note whathdr(): 748. If it succeeds, returns a namedtuple + as described above, otherwise ‘None’ is returned. + + Changed in version 3.5: Result changed from a tuple to a + namedtuple. + + -- Function: sndhdr.whathdr (filename) + + Determines the type of sound data stored in a file based on the + file header. The name of the file is given by `filename'. This + function returns a namedtuple as described above on success, or + ‘None’. + + Changed in version 3.5: Result changed from a tuple to a + namedtuple. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/sndhdr.py + + +File: python.info, Node: ossaudiodev — Access to OSS-compatible audio devices, Prev: sndhdr — Determine type of sound file, Up: Multimedia Services + +5.22.9 ‘ossaudiodev’ — Access to OSS-compatible audio devices +------------------------------------------------------------- + +__________________________________________________________________ + +This module allows you to access the OSS (Open Sound System) audio +interface. OSS is available for a wide range of open-source and +commercial Unices, and is the standard audio interface for Linux and +recent versions of FreeBSD. + +Changed in version 3.3: Operations in this module now raise *note +OSError: 1d3. where *note IOError: 992. was raised. + +See also +........ + +Open Sound System Programmer’s Guide(1) + + the official documentation for the OSS C API + +The module defines a large number of constants supplied by the OSS +device driver; see ‘<sys/soundcard.h>’ on either Linux or FreeBSD for a +listing. + +*note ossaudiodev: c6. defines the following variables and functions: + + -- Exception: ossaudiodev.OSSAudioError + + This exception is raised on certain errors. The argument is a + string describing what went wrong. + + (If *note ossaudiodev: c6. receives an error from a system call + such as ‘open()’, ‘write()’, or ‘ioctl()’, it raises *note OSError: + 1d3. Errors detected directly by *note ossaudiodev: c6. result in + *note OSSAudioError: 2d25.) + + (For backwards compatibility, the exception class is also available + as ‘ossaudiodev.error’.) + + -- Function: ossaudiodev.open (mode) + + -- Function: ossaudiodev.open (device, mode) + + Open an audio device and return an OSS audio device object. This + object supports many file-like methods, such as ‘read()’, + ‘write()’, and ‘fileno()’ (although there are subtle differences + between conventional Unix read/write semantics and those of OSS + audio devices). It also supports a number of audio-specific + methods; see below for the complete list of methods. + + `device' is the audio device filename to use. If it is not + specified, this module first looks in the environment variable + ‘AUDIODEV’ for a device to use. If not found, it falls back to + ‘/dev/dsp’. + + `mode' is one of ‘'r'’ for read-only (record) access, ‘'w'’ for + write-only (playback) access and ‘'rw'’ for both. Since many sound + cards only allow one process to have the recorder or player open at + a time, it is a good idea to open the device only for the activity + needed. Further, some sound cards are half-duplex: they can be + opened for reading or writing, but not both at once. + + Note the unusual calling syntax: the `first' argument is optional, + and the second is required. This is a historical artifact for + compatibility with the older ‘linuxaudiodev’ module which *note + ossaudiodev: c6. supersedes. + + -- Function: ossaudiodev.openmixer ([device]) + + Open a mixer device and return an OSS mixer device object. + `device' is the mixer device filename to use. If it is not + specified, this module first looks in the environment variable + ‘MIXERDEV’ for a device to use. If not found, it falls back to + ‘/dev/mixer’. + +* Menu: + +* Audio Device Objects:: +* Mixer Device Objects:: + + ---------- Footnotes ---------- + + (1) http://www.opensound.com/pguide/oss.pdf + + +File: python.info, Node: Audio Device Objects, Next: Mixer Device Objects, Up: ossaudiodev — Access to OSS-compatible audio devices + +5.22.9.1 Audio Device Objects +............................. + +Before you can write to or read from an audio device, you must call +three methods in the correct order: + + 1. ‘setfmt()’ to set the output format + + 2. ‘channels()’ to set the number of channels + + 3. ‘speed()’ to set the sample rate + +Alternately, you can use the ‘setparameters()’ method to set all three +audio parameters at once. This is more convenient, but may not be as +flexible in all cases. + +The audio device objects returned by *note open(): 2d26. define the +following methods and (read-only) attributes: + + -- Method: oss_audio_device.close () + + Explicitly close the audio device. When you are done writing to or + reading from an audio device, you should explicitly close it. A + closed device cannot be used again. + + -- Method: oss_audio_device.fileno () + + Return the file descriptor associated with the device. + + -- Method: oss_audio_device.read (size) + + Read `size' bytes from the audio input and return them as a Python + string. Unlike most Unix device drivers, OSS audio devices in + blocking mode (the default) will block *note read(): 2d2c. until + the entire requested amount of data is available. + + -- Method: oss_audio_device.write (data) + + Write a *note bytes-like object: 5e8. `data' to the audio device + and return the number of bytes written. If the audio device is in + blocking mode (the default), the entire data is always written + (again, this is different from usual Unix device semantics). If + the device is in non-blocking mode, some data may not be + written—see *note writeall(): 2d2e. + + Changed in version 3.5: Writable *note bytes-like object: 5e8. is + now accepted. + + -- Method: oss_audio_device.writeall (data) + + Write a *note bytes-like object: 5e8. `data' to the audio device: + waits until the audio device is able to accept data, writes as much + data as it will accept, and repeats until `data' has been + completely written. If the device is in blocking mode (the + default), this has the same effect as *note write(): 2d2d.; *note + writeall(): 2d2e. is only useful in non-blocking mode. Has no + return value, since the amount of data written is always equal to + the amount of data supplied. + + Changed in version 3.5: Writable *note bytes-like object: 5e8. is + now accepted. + +Changed in version 3.2: Audio device objects also support the context +management protocol, i.e. they can be used in a *note with: 6e9. +statement. + +The following methods each map to exactly one ‘ioctl()’ system call. +The correspondence is obvious: for example, ‘setfmt()’ corresponds to +the ‘SNDCTL_DSP_SETFMT’ ioctl, and ‘sync()’ to ‘SNDCTL_DSP_SYNC’ (this +can be useful when consulting the OSS documentation). If the underlying +‘ioctl()’ fails, they all raise *note OSError: 1d3. + + -- Method: oss_audio_device.nonblock () + + Put the device into non-blocking mode. Once in non-blocking mode, + there is no way to return it to blocking mode. + + -- Method: oss_audio_device.getfmts () + + Return a bitmask of the audio output formats supported by the + soundcard. Some of the formats supported by OSS are: + + Format Description + + -------------------------------------------------------------------------------- + + ‘AFMT_MU_LAW’ a logarithmic encoding (used by Sun ‘.au’ files + and ‘/dev/audio’) + + + ‘AFMT_A_LAW’ a logarithmic encoding + + + ‘AFMT_IMA_ADPCM’ a 4:1 compressed format defined by the + Interactive Multimedia Association + + + ‘AFMT_U8’ Unsigned, 8-bit audio + + + ‘AFMT_S16_LE’ Signed, 16-bit audio, little-endian byte order + (as used by Intel processors) + + + ‘AFMT_S16_BE’ Signed, 16-bit audio, big-endian byte order (as + used by 68k, PowerPC, Sparc) + + + ‘AFMT_S8’ Signed, 8 bit audio + + + ‘AFMT_U16_LE’ Unsigned, 16-bit little-endian audio + + + ‘AFMT_U16_BE’ Unsigned, 16-bit big-endian audio + + + Consult the OSS documentation for a full list of audio formats, and + note that most devices support only a subset of these formats. + Some older devices only support ‘AFMT_U8’; the most common format + used today is ‘AFMT_S16_LE’. + + -- Method: oss_audio_device.setfmt (format) + + Try to set the current audio format to `format'—see *note + getfmts(): 2d30. for a list. Returns the audio format that the + device was set to, which may not be the requested format. May also + be used to return the current audio format—do this by passing an + “audio format” of ‘AFMT_QUERY’. + + -- Method: oss_audio_device.channels (nchannels) + + Set the number of output channels to `nchannels'. A value of 1 + indicates monophonic sound, 2 stereophonic. Some devices may have + more than 2 channels, and some high-end devices may not support + mono. Returns the number of channels the device was set to. + + -- Method: oss_audio_device.speed (samplerate) + + Try to set the audio sampling rate to `samplerate' samples per + second. Returns the rate actually set. Most sound devices don’t + support arbitrary sampling rates. Common rates are: + + Rate Description + + ------------------------------------------------------------ + + 8000 default rate for ‘/dev/audio’ + + + 11025 speech recording + + + 22050 + + 44100 CD quality audio (at 16 bits/sample and 2 + channels) + + + 96000 DVD quality audio (at 24 bits/sample) + + + -- Method: oss_audio_device.sync () + + Wait until the sound device has played every byte in its buffer. + (This happens implicitly when the device is closed.) The OSS + documentation recommends closing and re-opening the device rather + than using *note sync(): 2d34. + + -- Method: oss_audio_device.reset () + + Immediately stop playing or recording and return the device to a + state where it can accept commands. The OSS documentation + recommends closing and re-opening the device after calling *note + reset(): 2d35. + + -- Method: oss_audio_device.post () + + Tell the driver that there is likely to be a pause in the output, + making it possible for the device to handle the pause more + intelligently. You might use this after playing a spot sound + effect, before waiting for user input, or before doing disk I/O. + +The following convenience methods combine several ioctls, or one ioctl +and some simple calculations. + + -- Method: oss_audio_device.setparameters (format, nchannels, + samplerate[, strict=False]) + + Set the key audio sampling parameters—sample format, number of + channels, and sampling rate—in one method call. `format', + `nchannels', and `samplerate' should be as specified in the *note + setfmt(): 2d31, *note channels(): 2d32, and *note speed(): 2d33. + methods. If `strict' is true, *note setparameters(): 2d37. checks + to see if each parameter was actually set to the requested value, + and raises *note OSSAudioError: 2d25. if not. Returns a tuple + (`format', `nchannels', `samplerate') indicating the parameter + values that were actually set by the device driver (i.e., the same + as the return values of *note setfmt(): 2d31, *note channels(): + 2d32, and *note speed(): 2d33.). + + For example, + + (fmt, channels, rate) = dsp.setparameters(fmt, channels, rate) + + is equivalent to + + fmt = dsp.setfmt(fmt) + channels = dsp.channels(channels) + rate = dsp.rate(rate) + + -- Method: oss_audio_device.bufsize () + + Returns the size of the hardware buffer, in samples. + + -- Method: oss_audio_device.obufcount () + + Returns the number of samples that are in the hardware buffer yet + to be played. + + -- Method: oss_audio_device.obuffree () + + Returns the number of samples that could be queued into the + hardware buffer to be played without blocking. + +Audio device objects also support several read-only attributes: + + -- Attribute: oss_audio_device.closed + + Boolean indicating whether the device has been closed. + + -- Attribute: oss_audio_device.name + + String containing the name of the device file. + + -- Attribute: oss_audio_device.mode + + The I/O mode for the file, either ‘"r"’, ‘"rw"’, or ‘"w"’. + + +File: python.info, Node: Mixer Device Objects, Prev: Audio Device Objects, Up: ossaudiodev — Access to OSS-compatible audio devices + +5.22.9.2 Mixer Device Objects +............................. + +The mixer object provides two file-like methods: + + -- Method: oss_mixer_device.close () + + This method closes the open mixer device file. Any further + attempts to use the mixer after this file is closed will raise an + *note OSError: 1d3. + + -- Method: oss_mixer_device.fileno () + + Returns the file handle number of the open mixer device file. + +Changed in version 3.2: Mixer objects also support the context +management protocol. + +The remaining methods are specific to audio mixing: + + -- Method: oss_mixer_device.controls () + + This method returns a bitmask specifying the available mixer + controls (“Control” being a specific mixable “channel”, such as + ‘SOUND_MIXER_PCM’ or ‘SOUND_MIXER_SYNTH’). This bitmask indicates + a subset of all available mixer controls—the ‘SOUND_MIXER_*’ + constants defined at module level. To determine if, for example, + the current mixer object supports a PCM mixer, use the following + Python code: + + mixer=ossaudiodev.openmixer() + if mixer.controls() & (1 << ossaudiodev.SOUND_MIXER_PCM): + # PCM is supported + ... code ... + + For most purposes, the ‘SOUND_MIXER_VOLUME’ (master volume) and + ‘SOUND_MIXER_PCM’ controls should suffice—but code that uses the + mixer should be flexible when it comes to choosing mixer controls. + On the Gravis Ultrasound, for example, ‘SOUND_MIXER_VOLUME’ does + not exist. + + -- Method: oss_mixer_device.stereocontrols () + + Returns a bitmask indicating stereo mixer controls. If a bit is + set, the corresponding control is stereo; if it is unset, the + control is either monophonic or not supported by the mixer (use in + combination with *note controls(): 2d42. to determine which). + + See the code example for the *note controls(): 2d42. function for + an example of getting data from a bitmask. + + -- Method: oss_mixer_device.reccontrols () + + Returns a bitmask specifying the mixer controls that may be used to + record. See the code example for *note controls(): 2d42. for an + example of reading from a bitmask. + + -- Method: oss_mixer_device.get (control) + + Returns the volume of a given mixer control. The returned volume + is a 2-tuple ‘(left_volume,right_volume)’. Volumes are specified + as numbers from 0 (silent) to 100 (full volume). If the control is + monophonic, a 2-tuple is still returned, but both volumes are the + same. + + Raises *note OSSAudioError: 2d25. if an invalid control is + specified, or *note OSError: 1d3. if an unsupported control is + specified. + + -- Method: oss_mixer_device.set (control, (left, right)) + + Sets the volume for a given mixer control to ‘(left,right)’. + ‘left’ and ‘right’ must be ints and between 0 (silent) and 100 + (full volume). On success, the new volume is returned as a + 2-tuple. Note that this may not be exactly the same as the volume + specified, because of the limited resolution of some soundcard’s + mixers. + + Raises *note OSSAudioError: 2d25. if an invalid mixer control was + specified, or if the specified volumes were out-of-range. + + -- Method: oss_mixer_device.get_recsrc () + + This method returns a bitmask indicating which control(s) are + currently being used as a recording source. + + -- Method: oss_mixer_device.set_recsrc (bitmask) + + Call this function to specify a recording source. Returns a + bitmask indicating the new recording source (or sources) if + successful; raises *note OSError: 1d3. if an invalid source was + specified. To set the current recording source to the microphone + input: + + mixer.setrecsrc (1 << ossaudiodev.SOUND_MIXER_MIC) + + +File: python.info, Node: Internationalization, Next: Program Frameworks, Prev: Multimedia Services, Up: The Python Standard Library + +5.23 Internationalization +========================= + +The modules described in this chapter help you write software that is +independent of language and locale by providing mechanisms for selecting +a language to be used in program messages or by tailoring output to +match local conventions. + +The list of modules described in this chapter is: + +* Menu: + +* gettext — Multilingual internationalization services:: +* locale — Internationalization services:: + + +File: python.info, Node: gettext — Multilingual internationalization services, Next: locale — Internationalization services, Up: Internationalization + +5.23.1 ‘gettext’ — Multilingual internationalization services +------------------------------------------------------------- + +`Source code:' Lib/gettext.py(1) + +__________________________________________________________________ + +The *note gettext: 89. module provides internationalization (I18N) and +localization (L10N) services for your Python modules and applications. +It supports both the GNU ‘gettext’ message catalog API and a higher +level, class-based API that may be more appropriate for Python files. +The interface described below allows you to write your module and +application messages in one natural language, and provide a catalog of +translated messages for running under different natural languages. + +Some hints on localizing your Python modules and applications are also +given. + +* Menu: + +* GNU gettext API:: +* Class-based API:: +* Internationalizing your programs and modules:: +* Acknowledgements: Acknowledgements<9>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/gettext.py + + +File: python.info, Node: GNU gettext API, Next: Class-based API, Up: gettext — Multilingual internationalization services + +5.23.1.1 GNU ‘gettext’ API +.......................... + +The *note gettext: 89. module defines the following API, which is very +similar to the GNU ‘gettext’ API. If you use this API you will affect +the translation of your entire application globally. Often this is what +you want if your application is monolingual, with the choice of language +dependent on the locale of your user. If you are localizing a Python +module, or if your application needs to switch languages on the fly, you +probably want to use the class-based API instead. + + -- Function: gettext.bindtextdomain (domain, localedir=None) + + Bind the `domain' to the locale directory `localedir'. More + concretely, *note gettext: 89. will look for binary ‘.mo’ files for + the given domain using the path (on Unix): + ‘`localedir'/`language'/LC_MESSAGES/`domain'.mo’, where `language' + is searched for in the environment variables ‘LANGUAGE’, ‘LC_ALL’, + ‘LC_MESSAGES’, and ‘LANG’ respectively. + + If `localedir' is omitted or ‘None’, then the current binding for + `domain' is returned. (1) + + -- Function: gettext.bind_textdomain_codeset (domain, codeset=None) + + Bind the `domain' to `codeset', changing the encoding of byte + strings returned by the *note lgettext(): 293, *note ldgettext(): + 294, *note lngettext(): 295. and *note ldngettext(): 296. + functions. If `codeset' is omitted, then the current binding is + returned. + + Deprecated since version 3.8, will be removed in version 3.10. + + -- Function: gettext.textdomain (domain=None) + + Change or query the current global domain. If `domain' is ‘None’, + then the current global domain is returned, otherwise the global + domain is set to `domain', which is returned. + + -- Function: gettext.gettext (message) + + Return the localized translation of `message', based on the current + global domain, language, and locale directory. This function is + usually aliased as ‘_()’ in the local namespace (see examples + below). + + -- Function: gettext.dgettext (domain, message) + + Like *note gettext(): dcd, but look the message up in the specified + `domain'. + + -- Function: gettext.ngettext (singular, plural, n) + + Like *note gettext(): dcd, but consider plural forms. If a + translation is found, apply the plural formula to `n', and return + the resulting message (some languages have more than two plural + forms). If no translation is found, return `singular' if `n' is 1; + return `plural' otherwise. + + The Plural formula is taken from the catalog header. It is a C or + Python expression that has a free variable `n'; the expression + evaluates to the index of the plural in the catalog. See the GNU + gettext documentation(2) for the precise syntax to be used in ‘.po’ + files and the formulas for a variety of languages. + + -- Function: gettext.dngettext (domain, singular, plural, n) + + Like *note ngettext(): 2d53, but look the message up in the + specified `domain'. + + -- Function: gettext.pgettext (context, message) + + -- Function: gettext.dpgettext (domain, context, message) + + -- Function: gettext.npgettext (context, singular, plural, n) + + -- Function: gettext.dnpgettext (domain, context, singular, plural, n) + + Similar to the corresponding functions without the ‘p’ in the + prefix (that is, *note gettext(): 89, *note dgettext(): 2d52, *note + ngettext(): 2d53, *note dngettext(): 2d54.), but the translation is + restricted to the given message `context'. + + New in version 3.8. + + -- Function: gettext.lgettext (message) + + -- Function: gettext.ldgettext (domain, message) + + -- Function: gettext.lngettext (singular, plural, n) + + -- Function: gettext.ldngettext (domain, singular, plural, n) + + Equivalent to the corresponding functions without the ‘l’ prefix + (*note gettext(): dcd, *note dgettext(): 2d52, *note ngettext(): + 2d53. and *note dngettext(): 2d54.), but the translation is + returned as a byte string encoded in the preferred system encoding + if no other encoding was explicitly set with *note + bind_textdomain_codeset(): 297. + + Warning: These functions should be avoided in Python 3, + because they return encoded bytes. It’s much better to use + alternatives which return Unicode strings instead, since most + Python applications will want to manipulate human readable + text as strings instead of bytes. Further, it’s possible that + you may get unexpected Unicode-related exceptions if there are + encoding problems with the translated strings. + + Deprecated since version 3.8, will be removed in version 3.10. + +Note that GNU ‘gettext’ also defines a ‘dcgettext()’ method, but this +was deemed not useful and so it is currently unimplemented. + +Here’s an example of typical usage for this API: + + import gettext + gettext.bindtextdomain('myapplication', '/path/to/my/language/directory') + gettext.textdomain('myapplication') + _ = gettext.gettext + # ... + print(_('This is a translatable string.')) + + ---------- Footnotes ---------- + + (1) (1) The default locale directory is system dependent; for +example, on RedHat Linux it is ‘/usr/share/locale’, but on Solaris it is +‘/usr/lib/locale’. The *note gettext: 89. module does not try to +support these system dependent defaults; instead its default is +‘`sys.base_prefix'/share/locale’ (see *note sys.base_prefix: 2d50.). +For this reason, it is always best to call *note bindtextdomain(): 2d4f. +with an explicit absolute path at the start of your application. + + (2) https://www.gnu.org/software/gettext/manual/gettext.html + + +File: python.info, Node: Class-based API, Next: Internationalizing your programs and modules, Prev: GNU gettext API, Up: gettext — Multilingual internationalization services + +5.23.1.2 Class-based API +........................ + +The class-based API of the *note gettext: 89. module gives you more +flexibility and greater convenience than the GNU ‘gettext’ API. It is +the recommended way of localizing your Python applications and modules. +‘gettext’ defines a *note GNUTranslations: 2d59. class which implements +the parsing of GNU ‘.mo’ format files, and has methods for returning +strings. Instances of this class can also install themselves in the +built-in namespace as the function ‘_()’. + + -- Function: gettext.find (domain, localedir=None, languages=None, + all=False) + + This function implements the standard ‘.mo’ file search algorithm. + It takes a `domain', identical to what *note textdomain(): 2d51. + takes. Optional `localedir' is as in *note bindtextdomain(): 2d4f. + Optional `languages' is a list of strings, where each string is a + language code. + + If `localedir' is not given, then the default system locale + directory is used. (1) If `languages' is not given, then the + following environment variables are searched: ‘LANGUAGE’, ‘LC_ALL’, + ‘LC_MESSAGES’, and ‘LANG’. The first one returning a non-empty + value is used for the `languages' variable. The environment + variables should contain a colon separated list of languages, which + will be split on the colon to produce the expected list of language + code strings. + + *note find(): 2d5a. then expands and normalizes the languages, and + then iterates through them, searching for an existing file built of + these components: + + ‘`localedir'/`language'/LC_MESSAGES/`domain'.mo’ + + The first such file name that exists is returned by *note find(): + 2d5a. If no such file is found, then ‘None’ is returned. If `all' + is given, it returns a list of all file names, in the order in + which they appear in the languages list or the environment + variables. + + -- Function: gettext.translation (domain, localedir=None, + languages=None, class_=None, fallback=False, codeset=None) + + Return a ‘*Translations’ instance based on the `domain', + `localedir', and `languages', which are first passed to *note + find(): 2d5a. to get a list of the associated ‘.mo’ file paths. + Instances with identical ‘.mo’ file names are cached. The actual + class instantiated is `class_' if provided, otherwise *note + GNUTranslations: 2d59. The class’s constructor must take a single + *note file object: b42. argument. If provided, `codeset' will + change the charset used to encode translated strings in the *note + lgettext(): 2d5b. and *note lngettext(): 2d5c. methods. + + If multiple files are found, later files are used as fallbacks for + earlier ones. To allow setting the fallback, *note copy.copy(): + 3cb. is used to clone each translation object from the cache; the + actual instance data is still shared with the cache. + + If no ‘.mo’ file is found, this function raises *note OSError: 1d3. + if `fallback' is false (which is the default), and returns a *note + NullTranslations: 2d5d. instance if `fallback' is true. + + Changed in version 3.3: *note IOError: 992. used to be raised + instead of *note OSError: 1d3. + + Deprecated since version 3.8, will be removed in version 3.10: The + `codeset' parameter. + + -- Function: gettext.install (domain, localedir=None, codeset=None, + names=None) + + This installs the function ‘_()’ in Python’s builtins namespace, + based on `domain', `localedir', and `codeset' which are passed to + the function *note translation(): 29a. + + For the `names' parameter, please see the description of the + translation object’s *note install(): 2d5e. method. + + As seen below, you usually mark the strings in your application + that are candidates for translation, by wrapping them in a call to + the ‘_()’ function, like this: + + print(_('This string will be translated.')) + + For convenience, you want the ‘_()’ function to be installed in + Python’s builtins namespace, so it is easily accessible in all + modules of your application. + + Deprecated since version 3.8, will be removed in version 3.10: The + `codeset' parameter. + +* Menu: + +* The NullTranslations class:: +* The GNUTranslations class:: +* Solaris message catalog support:: +* The Catalog constructor:: + + ---------- Footnotes ---------- + + (1) (2) See the footnote for *note bindtextdomain(): 2d4f. above. + + +File: python.info, Node: The NullTranslations class, Next: The GNUTranslations class, Up: Class-based API + +5.23.1.3 The ‘NullTranslations’ class +..................................... + +Translation classes are what actually implement the translation of +original source file message strings to translated message strings. The +base class used by all translation classes is *note NullTranslations: +2d5d.; this provides the basic interface you can use to write your own +specialized translation classes. Here are the methods of +‘NullTranslations’: + + -- Class: gettext.NullTranslations (fp=None) + + Takes an optional *note file object: b42. `fp', which is ignored by + the base class. Initializes “protected” instance variables `_info' + and `_charset' which are set by derived classes, as well as + `_fallback', which is set through *note add_fallback(): 2d60. It + then calls ‘self._parse(fp)’ if `fp' is not ‘None’. + + -- Method: _parse (fp) + + No-op in the base class, this method takes file object `fp', + and reads the data from the file, initializing its message + catalog. If you have an unsupported message catalog file + format, you should override this method to parse your format. + + -- Method: add_fallback (fallback) + + Add `fallback' as the fallback object for the current + translation object. A translation object should consult the + fallback if it cannot provide a translation for a given + message. + + -- Method: gettext (message) + + If a fallback has been set, forward ‘gettext()’ to the + fallback. Otherwise, return `message'. Overridden in derived + classes. + + -- Method: ngettext (singular, plural, n) + + If a fallback has been set, forward ‘ngettext()’ to the + fallback. Otherwise, return `singular' if `n' is 1; return + `plural' otherwise. Overridden in derived classes. + + -- Method: pgettext (context, message) + + If a fallback has been set, forward *note pgettext(): 1cf. to + the fallback. Otherwise, return the translated message. + Overridden in derived classes. + + New in version 3.8. + + -- Method: npgettext (context, singular, plural, n) + + If a fallback has been set, forward *note npgettext(): 2d56. + to the fallback. Otherwise, return the translated message. + Overridden in derived classes. + + New in version 3.8. + + -- Method: lgettext (message) + + -- Method: lngettext (singular, plural, n) + + Equivalent to *note gettext(): 2d62. and *note ngettext(): + 2d63, but the translation is returned as a byte string encoded + in the preferred system encoding if no encoding was explicitly + set with *note set_output_charset(): 299. Overridden in + derived classes. + + Warning: These methods should be avoided in Python 3. + See the warning for the *note lgettext(): 293. function. + + Deprecated since version 3.8, will be removed in version 3.10. + + -- Method: info () + + Return the “protected” ‘_info’ variable, a dictionary + containing the metadata found in the message catalog file. + + -- Method: charset () + + Return the encoding of the message catalog file. + + -- Method: output_charset () + + Return the encoding used to return translated messages in + *note lgettext(): 2d5b. and *note lngettext(): 2d5c. + + Deprecated since version 3.8, will be removed in version 3.10. + + -- Method: set_output_charset (charset) + + Change the encoding used to return translated messages. + + Deprecated since version 3.8, will be removed in version 3.10. + + -- Method: install (names=None) + + This method installs *note gettext(): 2d62. into the built-in + namespace, binding it to ‘_’. + + If the `names' parameter is given, it must be a sequence + containing the names of functions you want to install in the + builtins namespace in addition to ‘_()’. Supported names are + ‘'gettext'’, ‘'ngettext'’, ‘'pgettext'’, ‘'npgettext'’, + ‘'lgettext'’, and ‘'lngettext'’. + + Note that this is only one way, albeit the most convenient + way, to make the ‘_()’ function available to your application. + Because it affects the entire application globally, and + specifically the built-in namespace, localized modules should + never install ‘_()’. Instead, they should use this code to + make ‘_()’ available to their module: + + import gettext + t = gettext.translation('mymodule', ...) + _ = t.gettext + + This puts ‘_()’ only in the module’s global namespace and so + only affects calls within this module. + + Changed in version 3.8: Added ‘'pgettext'’ and ‘'npgettext'’. + + +File: python.info, Node: The GNUTranslations class, Next: Solaris message catalog support, Prev: The NullTranslations class, Up: Class-based API + +5.23.1.4 The ‘GNUTranslations’ class +.................................... + +The *note gettext: 89. module provides one additional class derived from +*note NullTranslations: 2d5d.: *note GNUTranslations: 2d59. This class +overrides ‘_parse()’ to enable reading GNU ‘gettext’ format ‘.mo’ files +in both big-endian and little-endian format. + +*note GNUTranslations: 2d59. parses optional metadata out of the +translation catalog. It is convention with GNU ‘gettext’ to include +metadata as the translation for the empty string. This metadata is in +RFC 822(1)-style ‘key: value’ pairs, and should contain the +‘Project-Id-Version’ key. If the key ‘Content-Type’ is found, then the +‘charset’ property is used to initialize the “protected” ‘_charset’ +instance variable, defaulting to ‘None’ if not found. If the charset +encoding is specified, then all message ids and message strings read +from the catalog are converted to Unicode using this encoding, else +ASCII is assumed. + +Since message ids are read as Unicode strings too, all ‘*gettext()’ +methods will assume message ids as Unicode strings, not byte strings. + +The entire set of key/value pairs are placed into a dictionary and set +as the “protected” ‘_info’ instance variable. + +If the ‘.mo’ file’s magic number is invalid, the major version number is +unexpected, or if other problems occur while reading the file, +instantiating a *note GNUTranslations: 2d59. class can raise *note +OSError: 1d3. + + -- Class: gettext.GNUTranslations + + The following methods are overridden from the base class + implementation: + + -- Method: gettext (message) + + Look up the `message' id in the catalog and return the + corresponding message string, as a Unicode string. If there + is no entry in the catalog for the `message' id, and a + fallback has been set, the look up is forwarded to the + fallback’s *note gettext(): 2d62. method. Otherwise, the + `message' id is returned. + + -- Method: ngettext (singular, plural, n) + + Do a plural-forms lookup of a message id. `singular' is used + as the message id for purposes of lookup in the catalog, while + `n' is used to determine which plural form to use. The + returned message string is a Unicode string. + + If the message id is not found in the catalog, and a fallback + is specified, the request is forwarded to the fallback’s *note + ngettext(): 2d63. method. Otherwise, when `n' is 1 `singular' + is returned, and `plural' is returned in all other cases. + + Here is an example: + + n = len(os.listdir('.')) + cat = GNUTranslations(somefile) + message = cat.ngettext( + 'There is %(num)d file in this directory', + 'There are %(num)d files in this directory', + n) % {'num': n} + + -- Method: pgettext (context, message) + + Look up the `context' and `message' id in the catalog and + return the corresponding message string, as a Unicode string. + If there is no entry in the catalog for the `message' id and + `context', and a fallback has been set, the look up is + forwarded to the fallback’s *note pgettext(): 1cf. method. + Otherwise, the `message' id is returned. + + New in version 3.8. + + -- Method: npgettext (context, singular, plural, n) + + Do a plural-forms lookup of a message id. `singular' is used + as the message id for purposes of lookup in the catalog, while + `n' is used to determine which plural form to use. + + If the message id for `context' is not found in the catalog, + and a fallback is specified, the request is forwarded to the + fallback’s *note npgettext(): 2d56. method. Otherwise, when + `n' is 1 `singular' is returned, and `plural' is returned in + all other cases. + + New in version 3.8. + + -- Method: lgettext (message) + + -- Method: lngettext (singular, plural, n) + + Equivalent to *note gettext(): 2d69. and *note ngettext(): + 2d6a, but the translation is returned as a byte string encoded + in the preferred system encoding if no encoding was explicitly + set with *note set_output_charset(): 299. + + Warning: These methods should be avoided in Python 3. + See the warning for the *note lgettext(): 293. function. + + Deprecated since version 3.8, will be removed in version 3.10. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc822.html + + +File: python.info, Node: Solaris message catalog support, Next: The Catalog constructor, Prev: The GNUTranslations class, Up: Class-based API + +5.23.1.5 Solaris message catalog support +........................................ + +The Solaris operating system defines its own binary ‘.mo’ file format, +but since no documentation can be found on this format, it is not +supported at this time. + + +File: python.info, Node: The Catalog constructor, Prev: Solaris message catalog support, Up: Class-based API + +5.23.1.6 The Catalog constructor +................................ + +GNOME uses a version of the *note gettext: 89. module by James +Henstridge, but this version has a slightly different API. Its +documented usage was: + + import gettext + cat = gettext.Catalog(domain, localedir) + _ = cat.gettext + print(_('hello world')) + +For compatibility with this older module, the function ‘Catalog()’ is an +alias for the *note translation(): 29a. function described above. + +One difference between this module and Henstridge’s: his catalog objects +supported access through a mapping API, but this appears to be unused +and so is not currently supported. + + +File: python.info, Node: Internationalizing your programs and modules, Next: Acknowledgements<9>, Prev: Class-based API, Up: gettext — Multilingual internationalization services + +5.23.1.7 Internationalizing your programs and modules +..................................................... + +Internationalization (I18N) refers to the operation by which a program +is made aware of multiple languages. Localization (L10N) refers to the +adaptation of your program, once internationalized, to the local +language and cultural habits. In order to provide multilingual messages +for your Python programs, you need to take the following steps: + + 1. prepare your program or module by specially marking translatable + strings + + 2. run a suite of tools over your marked files to generate raw + messages catalogs + + 3. create language-specific translations of the message catalogs + + 4. use the *note gettext: 89. module so that message strings are + properly translated + +In order to prepare your code for I18N, you need to look at all the +strings in your files. Any string that needs to be translated should be +marked by wrapping it in ‘_('...')’ — that is, a call to the function +‘_()’. For example: + + filename = 'mylog.txt' + message = _('writing a log message') + with open(filename, 'w') as fp: + fp.write(message) + +In this example, the string ‘'writing a log message'’ is marked as a +candidate for translation, while the strings ‘'mylog.txt'’ and ‘'w'’ are +not. + +There are a few tools to extract the strings meant for translation. The +original GNU ‘gettext’ only supported C or C++ source code but its +extended version ‘xgettext’ scans code written in a number of languages, +including Python, to find strings marked as translatable. Babel(1) is a +Python internationalization library that includes a ‘pybabel’ script to +extract and compile message catalogs. François Pinard’s program called +‘xpot’ does a similar job and is available as part of his po-utils +package(2). + +(Python also includes pure-Python versions of these programs, called +‘pygettext.py’ and ‘msgfmt.py’; some Python distributions will install +them for you. ‘pygettext.py’ is similar to ‘xgettext’, but only +understands Python source code and cannot handle other programming +languages such as C or C++. ‘pygettext.py’ supports a command-line +interface similar to ‘xgettext’; for details on its use, run +‘pygettext.py --help’. ‘msgfmt.py’ is binary compatible with GNU +‘msgfmt’. With these two programs, you may not need the GNU ‘gettext’ +package to internationalize your Python applications.) + +‘xgettext’, ‘pygettext’, and similar tools generate ‘.po’ files that are +message catalogs. They are structured human-readable files that contain +every marked string in the source code, along with a placeholder for the +translated versions of these strings. + +Copies of these ‘.po’ files are then handed over to the individual human +translators who write translations for every supported natural language. +They send back the completed language-specific versions as a +‘<language-name>.po’ file that’s compiled into a machine-readable ‘.mo’ +binary catalog file using the ‘msgfmt’ program. The ‘.mo’ files are +used by the *note gettext: 89. module for the actual translation +processing at run-time. + +How you use the *note gettext: 89. module in your code depends on +whether you are internationalizing a single module or your entire +application. The next two sections will discuss each case. + +* Menu: + +* Localizing your module:: +* Localizing your application:: +* Changing languages on the fly:: +* Deferred translations:: + + ---------- Footnotes ---------- + + (1) http://babel.pocoo.org/ + + (2) https://github.com/pinard/po-utils + + +File: python.info, Node: Localizing your module, Next: Localizing your application, Up: Internationalizing your programs and modules + +5.23.1.8 Localizing your module +............................... + +If you are localizing your module, you must take care not to make global +changes, e.g. to the built-in namespace. You should not use the GNU +‘gettext’ API but instead the class-based API. + +Let’s say your module is called “spam” and the module’s various natural +language translation ‘.mo’ files reside in ‘/usr/share/locale’ in GNU +‘gettext’ format. Here’s what you would put at the top of your module: + + import gettext + t = gettext.translation('spam', '/usr/share/locale') + _ = t.gettext + + +File: python.info, Node: Localizing your application, Next: Changing languages on the fly, Prev: Localizing your module, Up: Internationalizing your programs and modules + +5.23.1.9 Localizing your application +.................................... + +If you are localizing your application, you can install the ‘_()’ +function globally into the built-in namespace, usually in the main +driver file of your application. This will let all your +application-specific files just use ‘_('...')’ without having to +explicitly install it in each file. + +In the simple case then, you need only add the following bit of code to +the main driver file of your application: + + import gettext + gettext.install('myapplication') + +If you need to set the locale directory, you can pass it into the *note +install(): 29b. function: + + import gettext + gettext.install('myapplication', '/usr/share/locale') + + +File: python.info, Node: Changing languages on the fly, Next: Deferred translations, Prev: Localizing your application, Up: Internationalizing your programs and modules + +5.23.1.10 Changing languages on the fly +....................................... + +If your program needs to support many languages at the same time, you +may want to create multiple translation instances and then switch +between them explicitly, like so: + + import gettext + + lang1 = gettext.translation('myapplication', languages=['en']) + lang2 = gettext.translation('myapplication', languages=['fr']) + lang3 = gettext.translation('myapplication', languages=['de']) + + # start by using language1 + lang1.install() + + # ... time goes by, user selects language 2 + lang2.install() + + # ... more time goes by, user selects language 3 + lang3.install() + + +File: python.info, Node: Deferred translations, Prev: Changing languages on the fly, Up: Internationalizing your programs and modules + +5.23.1.11 Deferred translations +............................... + +In most coding situations, strings are translated where they are coded. +Occasionally however, you need to mark strings for translation, but +defer actual translation until later. A classic example is: + + animals = ['mollusk', + 'albatross', + 'rat', + 'penguin', + 'python', ] + # ... + for a in animals: + print(a) + +Here, you want to mark the strings in the ‘animals’ list as being +translatable, but you don’t actually want to translate them until they +are printed. + +Here is one way you can handle this situation: + + def _(message): return message + + animals = [_('mollusk'), + _('albatross'), + _('rat'), + _('penguin'), + _('python'), ] + + del _ + + # ... + for a in animals: + print(_(a)) + +This works because the dummy definition of ‘_()’ simply returns the +string unchanged. And this dummy definition will temporarily override +any definition of ‘_()’ in the built-in namespace (until the *note del: +f02. command). Take care, though if you have a previous definition of +‘_()’ in the local namespace. + +Note that the second use of ‘_()’ will not identify “a” as being +translatable to the ‘gettext’ program, because the parameter is not a +string literal. + +Another way to handle this is with the following example: + + def N_(message): return message + + animals = [N_('mollusk'), + N_('albatross'), + N_('rat'), + N_('penguin'), + N_('python'), ] + + # ... + for a in animals: + print(_(a)) + +In this case, you are marking translatable strings with the function +‘N_()’, which won’t conflict with any definition of ‘_()’. However, you +will need to teach your message extraction program to look for +translatable strings marked with ‘N_()’. ‘xgettext’, ‘pygettext’, +‘pybabel extract’, and ‘xpot’ all support this through the use of the +‘-k’ command-line switch. The choice of ‘N_()’ here is totally +arbitrary; it could have just as easily been +‘MarkThisStringForTranslation()’. + + +File: python.info, Node: Acknowledgements<9>, Prev: Internationalizing your programs and modules, Up: gettext — Multilingual internationalization services + +5.23.1.12 Acknowledgements +.......................... + +The following people contributed code, feedback, design suggestions, +previous implementations, and valuable experience to the creation of +this module: + + * Peter Funk + + * James Henstridge + + * Juan David Ibáñez Palomar + + * Marc-André Lemburg + + * Martin von Löwis + + * François Pinard + + * Barry Warsaw + + * Gustavo Niemeyer + + +File: python.info, Node: locale — Internationalization services, Prev: gettext — Multilingual internationalization services, Up: Internationalization + +5.23.2 ‘locale’ — Internationalization services +----------------------------------------------- + +`Source code:' Lib/locale.py(1) + +__________________________________________________________________ + +The *note locale: a9. module opens access to the POSIX locale database +and functionality. The POSIX locale mechanism allows programmers to +deal with certain cultural issues in an application, without requiring +the programmer to know all the specifics of each country where the +software is executed. + +The *note locale: a9. module is implemented on top of the ‘_locale’ +module, which in turn uses an ANSI C locale implementation if available. + +The *note locale: a9. module defines the following exception and +functions: + + -- Exception: locale.Error + + Exception raised when the locale passed to *note setlocale(): 1ce5. + is not recognized. + + -- Function: locale.setlocale (category, locale=None) + + If `locale' is given and not ‘None’, *note setlocale(): 1ce5. + modifies the locale setting for the `category'. The available + categories are listed in the data description below. `locale' may + be a string, or an iterable of two strings (language code and + encoding). If it’s an iterable, it’s converted to a locale name + using the locale aliasing engine. An empty string specifies the + user’s default settings. If the modification of the locale fails, + the exception *note Error: 2d79. is raised. If successful, the new + locale setting is returned. + + If `locale' is omitted or ‘None’, the current setting for + `category' is returned. + + *note setlocale(): 1ce5. is not thread-safe on most systems. + Applications typically start with a call of + + import locale + locale.setlocale(locale.LC_ALL, '') + + This sets the locale for all categories to the user’s default + setting (typically specified in the ‘LANG’ environment variable). + If the locale is not changed thereafter, using multithreading + should not cause problems. + + -- Function: locale.localeconv () + + Returns the database of the local conventions as a dictionary. + This dictionary has the following strings as keys: + + Category Key Meaning + + ---------------------------------------------------------------------------------------------------------- + + *note LC_NUMERIC: 2d7a. ‘'decimal_point'’ Decimal point character. + + + ‘'grouping'’ Sequence of numbers specifying + which relative positions the + ‘'thousands_sep'’ is expected. If + the sequence is terminated with + *note CHAR_MAX: 2d7b, no further + grouping is performed. If the + sequence terminates with a ‘0’, + the last group size is repeatedly + used. + + + ‘'thousands_sep'’ Character used between groups. + + + *note LC_MONETARY: 2d7c. ‘'int_curr_symbol'’ International currency symbol. + + + ‘'currency_symbol'’ Local currency symbol. + + + ‘'p_cs_precedes/n_cs_precedes'’ Whether the currency symbol + precedes the value (for positive + resp. negative values). + + + ‘'p_sep_by_space/n_sep_by_space'’ Whether the currency symbol is + separated from the value by a + space (for positive resp. + negative values). + + + ‘'mon_decimal_point'’ Decimal point used for monetary + values. + + + ‘'frac_digits'’ Number of fractional digits used + in local formatting of monetary + values. + + + ‘'int_frac_digits'’ Number of fractional digits used + in international formatting of + monetary values. + + + ‘'mon_thousands_sep'’ Group separator used for monetary + values. + + + ‘'mon_grouping'’ Equivalent to ‘'grouping'’, used + for monetary values. + + + ‘'positive_sign'’ Symbol used to annotate a positive + monetary value. + + + ‘'negative_sign'’ Symbol used to annotate a negative + monetary value. + + + ‘'p_sign_posn/n_sign_posn'’ The position of the sign (for + positive resp. negative values), + see below. + + + All numeric values can be set to *note CHAR_MAX: 2d7b. to indicate + that there is no value specified in this locale. + + The possible values for ‘'p_sign_posn'’ and ‘'n_sign_posn'’ are + given below. + + Value Explanation + + ----------------------------------------------------------------- + + ‘0’ Currency and value are surrounded by + parentheses. + + + ‘1’ The sign should precede the value and + currency symbol. + + + ‘2’ The sign should follow the value and + currency symbol. + + + ‘3’ The sign should immediately precede the + value. + + + ‘4’ The sign should immediately follow the + value. + + + ‘CHAR_MAX’ Nothing is specified in this locale. + + + The function sets temporarily the ‘LC_CTYPE’ locale to the + ‘LC_NUMERIC’ locale or the ‘LC_MONETARY’ locale if locales are + different and numeric or monetary strings are non-ASCII. This + temporary change affects other threads. + + Changed in version 3.7: The function now sets temporarily the + ‘LC_CTYPE’ locale to the ‘LC_NUMERIC’ locale in some cases. + + -- Function: locale.nl_langinfo (option) + + Return some locale-specific information as a string. This function + is not available on all systems, and the set of possible options + might also vary across platforms. The possible argument values are + numbers, for which symbolic constants are available in the locale + module. + + The *note nl_langinfo(): 2d7d. function accepts one of the + following keys. Most descriptions are taken from the corresponding + description in the GNU C library. + + -- Data: locale.CODESET + + Get a string with the name of the character encoding used in + the selected locale. + + -- Data: locale.D_T_FMT + + Get a string that can be used as a format string for *note + time.strftime(): b65. to represent date and time in a + locale-specific way. + + -- Data: locale.D_FMT + + Get a string that can be used as a format string for *note + time.strftime(): b65. to represent a date in a locale-specific + way. + + -- Data: locale.T_FMT + + Get a string that can be used as a format string for *note + time.strftime(): b65. to represent a time in a locale-specific + way. + + -- Data: locale.T_FMT_AMPM + + Get a format string for *note time.strftime(): b65. to + represent time in the am/pm format. + + -- Data: DAY_1 ... DAY_7 + + Get the name of the n-th day of the week. + + Note: This follows the US convention of ‘DAY_1’ being + Sunday, not the international convention (ISO 8601) that + Monday is the first day of the week. + + -- Data: ABDAY_1 ... ABDAY_7 + + Get the abbreviated name of the n-th day of the week. + + -- Data: MON_1 ... MON_12 + + Get the name of the n-th month. + + -- Data: ABMON_1 ... ABMON_12 + + Get the abbreviated name of the n-th month. + + -- Data: locale.RADIXCHAR + + Get the radix character (decimal dot, decimal comma, etc.). + + -- Data: locale.THOUSEP + + Get the separator character for thousands (groups of three + digits). + + -- Data: locale.YESEXPR + + Get a regular expression that can be used with the regex + function to recognize a positive response to a yes/no + question. + + Note: The expression is in the syntax suitable for the + ‘regex()’ function from the C library, which might differ + from the syntax used in *note re: dd. + + -- Data: locale.NOEXPR + + Get a regular expression that can be used with the regex(3) + function to recognize a negative response to a yes/no + question. + + -- Data: locale.CRNCYSTR + + Get the currency symbol, preceded by “-” if the symbol should + appear before the value, “+” if the symbol should appear after + the value, or “.” if the symbol should replace the radix + character. + + -- Data: locale.ERA + + Get a string that represents the era used in the current + locale. + + Most locales do not define this value. An example of a locale + which does define this value is the Japanese one. In Japan, + the traditional representation of dates includes the name of + the era corresponding to the then-emperor’s reign. + + Normally it should not be necessary to use this value + directly. Specifying the ‘E’ modifier in their format strings + causes the *note time.strftime(): b65. function to use this + information. The format of the returned string is not + specified, and therefore you should not assume knowledge of it + on different systems. + + -- Data: locale.ERA_D_T_FMT + + Get a format string for *note time.strftime(): b65. to + represent date and time in a locale-specific era-based way. + + -- Data: locale.ERA_D_FMT + + Get a format string for *note time.strftime(): b65. to + represent a date in a locale-specific era-based way. + + -- Data: locale.ERA_T_FMT + + Get a format string for *note time.strftime(): b65. to + represent a time in a locale-specific era-based way. + + -- Data: locale.ALT_DIGITS + + Get a representation of up to 100 values used to represent the + values 0 to 99. + + -- Function: locale.getdefaultlocale ([envvars]) + + Tries to determine the default locale settings and returns them as + a tuple of the form ‘(language code, encoding)’. + + According to POSIX, a program which has not called + ‘setlocale(LC_ALL, '')’ runs using the portable ‘'C'’ locale. + Calling ‘setlocale(LC_ALL, '')’ lets it use the default locale as + defined by the ‘LANG’ variable. Since we do not want to interfere + with the current locale setting we thus emulate the behavior in the + way described above. + + To maintain compatibility with other platforms, not only the ‘LANG’ + variable is tested, but a list of variables given as envvars + parameter. The first found to be defined will be used. `envvars' + defaults to the search path used in GNU gettext; it must always + contain the variable name ‘'LANG'’. The GNU gettext search path + contains ‘'LC_ALL'’, ‘'LC_CTYPE'’, ‘'LANG'’ and ‘'LANGUAGE'’, in + that order. + + Except for the code ‘'C'’, the language code corresponds to RFC + 1766(2). `language code' and `encoding' may be ‘None’ if their + values cannot be determined. + + -- Function: locale.getlocale (category=LC_CTYPE) + + Returns the current setting for the given locale category as + sequence containing `language code', `encoding'. `category' may be + one of the ‘LC_*’ values except *note LC_ALL: 2d8d. It defaults to + *note LC_CTYPE: 2d8e. + + Except for the code ‘'C'’, the language code corresponds to RFC + 1766(3). `language code' and `encoding' may be ‘None’ if their + values cannot be determined. + + -- Function: locale.getpreferredencoding (do_setlocale=True) + + Return the encoding used for text data, according to user + preferences. User preferences are expressed differently on + different systems, and might not be available programmatically on + some systems, so this function only returns a guess. + + On some systems, it is necessary to invoke *note setlocale(): 1ce5. + to obtain the user preferences, so this function is not + thread-safe. If invoking setlocale is not necessary or desired, + `do_setlocale' should be set to ‘False’. + + On Android or in the UTF-8 mode (*note -X: 155. ‘utf8’ option), + always return ‘'UTF-8'’, the locale and the `do_setlocale' argument + are ignored. + + Changed in version 3.7: The function now always returns ‘UTF-8’ on + Android or if the UTF-8 mode is enabled. + + -- Function: locale.normalize (localename) + + Returns a normalized locale code for the given locale name. The + returned locale code is formatted for use with *note setlocale(): + 1ce5. If normalization fails, the original name is returned + unchanged. + + If the given encoding is not known, the function defaults to the + default encoding for the locale code just like *note setlocale(): + 1ce5. + + -- Function: locale.resetlocale (category=LC_ALL) + + Sets the locale for `category' to the default setting. + + The default setting is determined by calling *note + getdefaultlocale(): ffc. `category' defaults to *note LC_ALL: + 2d8d. + + -- Function: locale.strcoll (string1, string2) + + Compares two strings according to the current *note LC_COLLATE: + 2d92. setting. As any other compare function, returns a negative, + or a positive value, or ‘0’, depending on whether `string1' + collates before or after `string2' or is equal to it. + + -- Function: locale.strxfrm (string) + + Transforms a string to one that can be used in locale-aware + comparisons. For example, ‘strxfrm(s1) < strxfrm(s2)’ is + equivalent to ‘strcoll(s1, s2) < 0’. This function can be used + when the same string is compared repeatedly, e.g. when collating a + sequence of strings. + + -- Function: locale.format_string (format, val, grouping=False, + monetary=False) + + Formats a number `val' according to the current *note LC_NUMERIC: + 2d7a. setting. The format follows the conventions of the ‘%’ + operator. For floating point values, the decimal point is modified + if appropriate. If `grouping' is true, also takes the grouping + into account. + + If `monetary' is true, the conversion uses monetary thousands + separator and grouping strings. + + Processes formatting specifiers as in ‘format % val’, but takes the + current locale settings into account. + + Changed in version 3.7: The `monetary' keyword parameter was added. + + -- Function: locale.format (format, val, grouping=False, + monetary=False) + + Please note that this function works like *note format_string(): + 3a4. but will only work for exactly one ‘%char’ specifier. For + example, ‘'%f'’ and ‘'%.0f'’ are both valid specifiers, but ‘'%f + KiB'’ is not. + + For whole format strings, use *note format_string(): 3a4. + + Deprecated since version 3.7: Use *note format_string(): 3a4. + instead. + + -- Function: locale.currency (val, symbol=True, grouping=False, + international=False) + + Formats a number `val' according to the current *note LC_MONETARY: + 2d7c. settings. + + The returned string includes the currency symbol if `symbol' is + true, which is the default. If `grouping' is true (which is not + the default), grouping is done with the value. If `international' + is true (which is not the default), the international currency + symbol is used. + + Note that this function will not work with the ‘C’ locale, so you + have to set a locale via *note setlocale(): 1ce5. first. + + -- Function: locale.str (float) + + Formats a floating point number using the same format as the + built-in function ‘str(float)’, but takes the decimal point into + account. + + -- Function: locale.delocalize (string) + + Converts a string into a normalized number string, following the + *note LC_NUMERIC: 2d7a. settings. + + New in version 3.5. + + -- Function: locale.atof (string) + + Converts a string to a floating point number, following the *note + LC_NUMERIC: 2d7a. settings. + + -- Function: locale.atoi (string) + + Converts a string to an integer, following the *note LC_NUMERIC: + 2d7a. conventions. + + -- Data: locale.LC_CTYPE + + Locale category for the character type functions. Depending on the + settings of this category, the functions of module *note string: + f6. dealing with case change their behaviour. + + -- Data: locale.LC_COLLATE + + Locale category for sorting strings. The functions *note + strcoll(): 2d91. and *note strxfrm(): 2d93. of the *note locale: + a9. module are affected. + + -- Data: locale.LC_TIME + + Locale category for the formatting of time. The function *note + time.strftime(): b65. follows these conventions. + + -- Data: locale.LC_MONETARY + + Locale category for formatting of monetary values. The available + options are available from the *note localeconv(): 48a. function. + + -- Data: locale.LC_MESSAGES + + Locale category for message display. Python currently does not + support application specific locale-aware messages. Messages + displayed by the operating system, like those returned by *note + os.strerror(): 1bd4. might be affected by this category. + + -- Data: locale.LC_NUMERIC + + Locale category for formatting numbers. The functions *note + format(): 468, *note atoi(): 2d97, *note atof(): 2d96. and *note + str(): 2d95. of the *note locale: a9. module are affected by that + category. All other numeric formatting operations are not + affected. + + -- Data: locale.LC_ALL + + Combination of all locale settings. If this flag is used when the + locale is changed, setting the locale for all categories is + attempted. If that fails for any category, no category is changed + at all. When the locale is retrieved using this flag, a string + indicating the setting for all categories is returned. This string + can be later used to restore the settings. + + -- Data: locale.CHAR_MAX + + This is a symbolic constant used for different values returned by + *note localeconv(): 48a. + +Example: + + >>> import locale + >>> loc = locale.getlocale() # get current locale + # use German locale; name might vary with platform + >>> locale.setlocale(locale.LC_ALL, 'de_DE') + >>> locale.strcoll('f\xe4n', 'foo') # compare a string containing an umlaut + >>> locale.setlocale(locale.LC_ALL, '') # use user's preferred locale + >>> locale.setlocale(locale.LC_ALL, 'C') # use default (C) locale + >>> locale.setlocale(locale.LC_ALL, loc) # restore saved locale + +* Menu: + +* Background, details, hints, tips and caveats: Background details hints tips and caveats. +* For extension writers and programs that embed Python:: +* Access to message catalogs:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/locale.py + + (2) https://tools.ietf.org/html/rfc1766.html + + (3) https://tools.ietf.org/html/rfc1766.html + + +File: python.info, Node: Background details hints tips and caveats, Next: For extension writers and programs that embed Python, Up: locale — Internationalization services + +5.23.2.1 Background, details, hints, tips and caveats +..................................................... + +The C standard defines the locale as a program-wide property that may be +relatively expensive to change. On top of that, some implementation are +broken in such a way that frequent locale changes may cause core dumps. +This makes the locale somewhat painful to use correctly. + +Initially, when a program is started, the locale is the ‘C’ locale, no +matter what the user’s preferred locale is. There is one exception: the +*note LC_CTYPE: 2d8e. category is changed at startup to set the current +locale encoding to the user’s preferred locale encoding. The program +must explicitly say that it wants the user’s preferred locale settings +for other categories by calling ‘setlocale(LC_ALL, '')’. + +It is generally a bad idea to call *note setlocale(): 1ce5. in some +library routine, since as a side effect it affects the entire program. +Saving and restoring it is almost as bad: it is expensive and affects +other threads that happen to run before the settings have been restored. + +If, when coding a module for general use, you need a locale independent +version of an operation that is affected by the locale (such as certain +formats used with *note time.strftime(): b65.), you will have to find a +way to do it without using the standard library routine. Even better is +convincing yourself that using locale settings is okay. Only as a last +resort should you document that your module is not compatible with +non-‘C’ locale settings. + +The only way to perform numeric operations according to the locale is to +use the special functions defined by this module: *note atof(): 2d96, +*note atoi(): 2d97, *note format(): 468, *note str(): 2d95. + +There is no way to perform case conversions and character +classifications according to the locale. For (Unicode) text strings +these are done according to the character value only, while for byte +strings, the conversions and classifications are done according to the +ASCII value of the byte, and bytes whose high bit is set (i.e., +non-ASCII bytes) are never converted or considered part of a character +class such as letter or whitespace. + + +File: python.info, Node: For extension writers and programs that embed Python, Next: Access to message catalogs, Prev: Background details hints tips and caveats, Up: locale — Internationalization services + +5.23.2.2 For extension writers and programs that embed Python +............................................................. + +Extension modules should never call *note setlocale(): 1ce5, except to +find out what the current locale is. But since the return value can +only be used portably to restore it, that is not very useful (except +perhaps to find out whether or not the locale is ‘C’). + +When Python code uses the *note locale: a9. module to change the locale, +this also affects the embedding application. If the embedding +application doesn’t want this to happen, it should remove the ‘_locale’ +extension module (which does all the work) from the table of built-in +modules in the ‘config.c’ file, and make sure that the ‘_locale’ module +is not accessible as a shared library. + + +File: python.info, Node: Access to message catalogs, Prev: For extension writers and programs that embed Python, Up: locale — Internationalization services + +5.23.2.3 Access to message catalogs +................................... + + -- Function: locale.gettext (msg) + + -- Function: locale.dgettext (domain, msg) + + -- Function: locale.dcgettext (domain, msg, category) + + -- Function: locale.textdomain (domain) + + -- Function: locale.bindtextdomain (domain, dir) + +The locale module exposes the C library’s gettext interface on systems +that provide this interface. It consists of the functions ‘gettext()’, +‘dgettext()’, ‘dcgettext()’, ‘textdomain()’, ‘bindtextdomain()’, and +‘bind_textdomain_codeset()’. These are similar to the same functions in +the *note gettext: 89. module, but use the C library’s binary format for +message catalogs, and the C library’s search algorithms for locating +message catalogs. + +Python applications should normally find no need to invoke these +functions, and should use *note gettext: 89. instead. A known exception +to this rule are applications that link with additional C libraries +which internally invoke ‘gettext()’ or ‘dcgettext()’. For these +applications, it may be necessary to bind the text domain, so that the +libraries can properly locate their message catalogs. + + +File: python.info, Node: Program Frameworks, Next: Graphical User Interfaces with Tk, Prev: Internationalization, Up: The Python Standard Library + +5.24 Program Frameworks +======================= + +The modules described in this chapter are frameworks that will largely +dictate the structure of your program. Currently the modules described +here are all oriented toward writing command-line interfaces. + +The full list of modules described in this chapter is: + +* Menu: + +* turtle — Turtle graphics:: +* cmd — Support for line-oriented command interpreters:: +* shlex — Simple lexical analysis:: + + +File: python.info, Node: turtle — Turtle graphics, Next: cmd — Support for line-oriented command interpreters, Up: Program Frameworks + +5.24.1 ‘turtle’ — Turtle graphics +--------------------------------- + +`Source code:' Lib/turtle.py(1) + +__________________________________________________________________ + +* Menu: + +* Introduction: Introduction<11>. +* Overview of available Turtle and Screen methods:: +* Methods of RawTurtle/Turtle and corresponding functions:: +* Methods of TurtleScreen/Screen and corresponding functions:: +* Public classes:: +* Help and configuration:: +* turtledemo — Demo scripts:: +* Changes since Python 2.6: Changes since Python 2 6. +* Changes since Python 3.0: Changes since Python 3 0. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/turtle.py + + +File: python.info, Node: Introduction<11>, Next: Overview of available Turtle and Screen methods, Up: turtle — Turtle graphics + +5.24.1.1 Introduction +..................... + +Turtle graphics is a popular way for introducing programming to kids. +It was part of the original Logo programming language developed by Wally +Feurzeig, Seymour Papert and Cynthia Solomon in 1967. + +Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an +‘import turtle’, give it the command ‘turtle.forward(15)’, and it moves +(on-screen!) 15 pixels in the direction it is facing, drawing a line as +it moves. Give it the command ‘turtle.right(25)’, and it rotates +in-place 25 degrees clockwise. + +Turtle star +........... + +Turtle can draw intricate shapes using programs that repeat simple +moves. + +[python-figures/turtle-star] + from turtle import * + color('red', 'yellow') + begin_fill() + while True: + forward(200) + left(170) + if abs(pos()) < 1: + break + end_fill() + done() + +By combining together these and similar commands, intricate shapes and +pictures can easily be drawn. + +The *note turtle: 116. module is an extended reimplementation of the +same-named module from the Python standard distribution up to version +Python 2.5. + +It tries to keep the merits of the old turtle module and to be (nearly) +100% compatible with it. This means in the first place to enable the +learning programmer to use all the commands, classes and methods +interactively when using the module from within IDLE run with the ‘-n’ +switch. + +The turtle module provides turtle graphics primitives, in both +object-oriented and procedure-oriented ways. Because it uses *note +tkinter: 10c. for the underlying graphics, it needs a version of Python +installed with Tk support. + +The object-oriented interface uses essentially two+two classes: + + 1. The *note TurtleScreen: 2daa. class defines graphics windows as a + playground for the drawing turtles. Its constructor needs a + ‘tkinter.Canvas’ or a *note ScrolledCanvas: 2dab. as argument. It + should be used when *note turtle: 116. is used as part of some + application. + + The function *note Screen(): 2dac. returns a singleton object of a + *note TurtleScreen: 2daa. subclass. This function should be used + when *note turtle: 116. is used as a standalone tool for doing + graphics. As a singleton object, inheriting from its class is not + possible. + + All methods of TurtleScreen/Screen also exist as functions, i.e. + as part of the procedure-oriented interface. + + 2. *note RawTurtle: 2dad. (alias: *note RawPen: 2dae.) defines Turtle + objects which draw on a *note TurtleScreen: 2daa. Its constructor + needs a Canvas, ScrolledCanvas or TurtleScreen as argument, so the + RawTurtle objects know where to draw. + + Derived from RawTurtle is the subclass *note Turtle: 2daf. (alias: + ‘Pen’), which draws on “the” *note Screen: 2dac. instance which is + automatically created, if not already present. + + All methods of RawTurtle/Turtle also exist as functions, i.e. part + of the procedure-oriented interface. + +The procedural interface provides functions which are derived from the +methods of the classes *note Screen: 2dac. and *note Turtle: 2daf. They +have the same names as the corresponding methods. A screen object is +automatically created whenever a function derived from a Screen method +is called. An (unnamed) turtle object is automatically created whenever +any of the functions derived from a Turtle method is called. + +To use multiple turtles on a screen one has to use the object-oriented +interface. + + Note: In the following documentation the argument list for + functions is given. Methods, of course, have the additional first + argument `self' which is omitted here. + + +File: python.info, Node: Overview of available Turtle and Screen methods, Next: Methods of RawTurtle/Turtle and corresponding functions, Prev: Introduction<11>, Up: turtle — Turtle graphics + +5.24.1.2 Overview of available Turtle and Screen methods +........................................................ + +* Menu: + +* Turtle methods:: +* Methods of TurtleScreen/Screen:: + + +File: python.info, Node: Turtle methods, Next: Methods of TurtleScreen/Screen, Up: Overview of available Turtle and Screen methods + +5.24.1.3 Turtle methods +....................... + +Turtle motion + + Move and draw + + *note forward(): 2db2. | *note fd(): 2db3. + *note backward(): 2db4. | *note bk(): 2db5. | *note back(): 2db6. + *note right(): 2db7. | *note rt(): 2db8. + *note left(): 2db9. | *note lt(): 2dba. + *note goto(): 2dbb. | *note setpos(): 2dbc. | *note setposition(): 2dbd. + *note setx(): 2dbe. + *note sety(): 2dbf. + *note setheading(): 2dc0. | *note seth(): 2dc1. + *note home(): 2dc2. + *note circle(): 2dc3. + *note dot(): 2dc4. + *note stamp(): 2dc5. + *note clearstamp(): 2dc6. + *note clearstamps(): 2dc7. + *note undo(): 2dc8. + *note speed(): 2dc9. + + Tell Turtle’s state + + *note position(): 2dca. | *note pos(): 2dcb. + *note towards(): 2dcc. + *note xcor(): 2dcd. + *note ycor(): 2dce. + *note heading(): 2dcf. + *note distance(): 2dd0. + + Setting and measurement + + *note degrees(): 2dd1. + *note radians(): 2dd2. + +Pen control + + Drawing state + + *note pendown(): 2dd3. | *note pd(): 2dd4. | *note down(): 2dd5. + *note penup(): 2dd6. | *note pu(): 2dd7. | *note up(): 2dd8. + *note pensize(): 2dd9. | *note width(): 2dda. + *note pen(): 2ddb. + *note isdown(): 2ddc. + + Color control + + *note color(): 2ddd. + *note pencolor(): 2dde. + *note fillcolor(): 2ddf. + + Filling + + *note filling(): 2de0. + *note begin_fill(): 2de1. + *note end_fill(): 2de2. + + More drawing control + + *note reset(): 2de3. + *note clear(): 2de4. + *note write(): 2de5. + +Turtle state + + Visibility + + *note showturtle(): 2de6. | *note st(): 2de7. + *note hideturtle(): 2de8. | *note ht(): 2de9. + *note isvisible(): 2dea. + + Appearance + + *note shape(): 2deb. + *note resizemode(): 2dec. + *note shapesize(): 2ded. | *note turtlesize(): 2dee. + *note shearfactor(): 2def. + *note settiltangle(): 2df0. + *note tiltangle(): 2df1. + *note tilt(): 2df2. + *note shapetransform(): 2df3. + *note get_shapepoly(): 2df4. + +Using events + + *note onclick(): 2df5. + *note onrelease(): 2df6. + *note ondrag(): 2df7. + +Special Turtle methods + + *note begin_poly(): 2df8. + *note end_poly(): 2df9. + *note get_poly(): 2dfa. + *note clone(): 2dfb. + *note getturtle(): 2dfc. | *note getpen(): 2dfd. + *note getscreen(): 2dfe. + *note setundobuffer(): 2dff. + *note undobufferentries(): 2e00. + + +File: python.info, Node: Methods of TurtleScreen/Screen, Prev: Turtle methods, Up: Overview of available Turtle and Screen methods + +5.24.1.4 Methods of TurtleScreen/Screen +....................................... + +Window control + + *note bgcolor(): 2e02. + *note bgpic(): 2e03. + *note clear(): 2de4. | *note clearscreen(): 2e04. + *note reset(): 2de3. | *note resetscreen(): 2e05. + *note screensize(): 2e06. + *note setworldcoordinates(): 2e07. + +Animation control + + *note delay(): 2e08. + *note tracer(): 2e09. + *note update(): 2e0a. + +Using screen events + + *note listen(): 2e0b. + *note onkey(): 2e0c. | *note onkeyrelease(): 2e0d. + *note onkeypress(): 2e0e. + *note onclick(): 2df5. | *note onscreenclick(): 2e0f. + *note ontimer(): 2e10. + *note mainloop(): 2e11. | *note done(): 2e12. + +Settings and special methods + + *note mode(): 2e13. + *note colormode(): 2e14. + *note getcanvas(): 2e15. + *note getshapes(): 2e16. + *note register_shape(): 2e17. | *note addshape(): 2e18. + *note turtles(): 2e19. + *note window_height(): 2e1a. + *note window_width(): 2e1b. + +Input methods + + *note textinput(): 2e1c. + *note numinput(): 2e1d. + +Methods specific to Screen + + *note bye(): 2e1e. + *note exitonclick(): 2e1f. + *note setup(): 2e20. + *note title(): 2e21. + + +File: python.info, Node: Methods of RawTurtle/Turtle and corresponding functions, Next: Methods of TurtleScreen/Screen and corresponding functions, Prev: Overview of available Turtle and Screen methods, Up: turtle — Turtle graphics + +5.24.1.5 Methods of RawTurtle/Turtle and corresponding functions +................................................................ + +Most of the examples in this section refer to a Turtle instance called +‘turtle’. + +* Menu: + +* Turtle motion:: +* Tell Turtle’s state:: +* Settings for measurement:: +* Pen control:: +* Turtle state:: +* Using events:: +* Special Turtle methods:: +* Compound shapes:: + + +File: python.info, Node: Turtle motion, Next: Tell Turtle’s state, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.6 Turtle motion +...................... + + -- Function: turtle.forward (distance) + -- Function: turtle.fd (distance) + + + Parameters: ‘distance’ – a number (integer or float) + + Move the turtle forward by the specified `distance', in the + direction the turtle is headed. + + >>> turtle.position() + (0.00,0.00) + >>> turtle.forward(25) + >>> turtle.position() + (25.00,0.00) + >>> turtle.forward(-75) + >>> turtle.position() + (-50.00,0.00) + + -- Function: turtle.back (distance) + -- Function: turtle.bk (distance) + -- Function: turtle.backward (distance) + + + Parameters: ‘distance’ – a number + + Move the turtle backward by `distance', opposite to the direction + the turtle is headed. Do not change the turtle’s heading. + + >>> turtle.position() + (0.00,0.00) + >>> turtle.backward(30) + >>> turtle.position() + (-30.00,0.00) + + -- Function: turtle.right (angle) + -- Function: turtle.rt (angle) + + + Parameters: ‘angle’ – a number (integer or float) + + Turn turtle right by `angle' units. (Units are by default degrees, + but can be set via the *note degrees(): 2dd1. and *note radians(): + 2dd2. functions.) Angle orientation depends on the turtle mode, + see *note mode(): 2e13. + + >>> turtle.heading() + 22.0 + >>> turtle.right(45) + >>> turtle.heading() + 337.0 + + -- Function: turtle.left (angle) + -- Function: turtle.lt (angle) + + + Parameters: ‘angle’ – a number (integer or float) + + Turn turtle left by `angle' units. (Units are by default degrees, + but can be set via the *note degrees(): 2dd1. and *note radians(): + 2dd2. functions.) Angle orientation depends on the turtle mode, + see *note mode(): 2e13. + + >>> turtle.heading() + 22.0 + >>> turtle.left(45) + >>> turtle.heading() + 67.0 + + -- Function: turtle.goto (x, y=None) + -- Function: turtle.setpos (x, y=None) + -- Function: turtle.setposition (x, y=None) + + + Parameters: + + * ‘x’ – a number or a pair/vector of numbers + + * ‘y’ – a number or ‘None’ + + If `y' is ‘None’, `x' must be a pair of coordinates or a *note + Vec2D: 2e24. (e.g. as returned by *note pos(): 2dcb.). + + Move turtle to an absolute position. If the pen is down, draw + line. Do not change the turtle’s orientation. + + >>> tp = turtle.pos() + >>> tp + (0.00,0.00) + >>> turtle.setpos(60,30) + >>> turtle.pos() + (60.00,30.00) + >>> turtle.setpos((20,80)) + >>> turtle.pos() + (20.00,80.00) + >>> turtle.setpos(tp) + >>> turtle.pos() + (0.00,0.00) + + -- Function: turtle.setx (x) + + + Parameters: ‘x’ – a number (integer or float) + + Set the turtle’s first coordinate to `x', leave second coordinate + unchanged. + + >>> turtle.position() + (0.00,240.00) + >>> turtle.setx(10) + >>> turtle.position() + (10.00,240.00) + + -- Function: turtle.sety (y) + + + Parameters: ‘y’ – a number (integer or float) + + Set the turtle’s second coordinate to `y', leave first coordinate + unchanged. + + >>> turtle.position() + (0.00,40.00) + >>> turtle.sety(-10) + >>> turtle.position() + (0.00,-10.00) + + -- Function: turtle.setheading (to_angle) + -- Function: turtle.seth (to_angle) + + + Parameters: ‘to_angle’ – a number (integer or float) + + Set the orientation of the turtle to `to_angle'. Here are some + common directions in degrees: + + standard mode logo mode + + ------------------------------------------------- + + 0 - east 0 - north + + + 90 - north 90 - east + + + 180 - west 180 - south + + + 270 - south 270 - west + + + >>> turtle.setheading(90) + >>> turtle.heading() + 90.0 + + -- Function: turtle.home () + + Move turtle to the origin – coordinates (0,0) – and set its heading + to its start-orientation (which depends on the mode, see *note + mode(): 2e13.). + + >>> turtle.heading() + 90.0 + >>> turtle.position() + (0.00,-10.00) + >>> turtle.home() + >>> turtle.position() + (0.00,0.00) + >>> turtle.heading() + 0.0 + + -- Function: turtle.circle (radius, extent=None, steps=None) + + + Parameters: + + * ‘radius’ – a number + + * ‘extent’ – a number (or ‘None’) + + * ‘steps’ – an integer (or ‘None’) + + Draw a circle with given `radius'. The center is `radius' units + left of the turtle; `extent' – an angle – determines which part of + the circle is drawn. If `extent' is not given, draw the entire + circle. If `extent' is not a full circle, one endpoint of the arc + is the current pen position. Draw the arc in counterclockwise + direction if `radius' is positive, otherwise in clockwise + direction. Finally the direction of the turtle is changed by the + amount of `extent'. + + As the circle is approximated by an inscribed regular polygon, + `steps' determines the number of steps to use. If not given, it + will be calculated automatically. May be used to draw regular + polygons. + + >>> turtle.home() + >>> turtle.position() + (0.00,0.00) + >>> turtle.heading() + 0.0 + >>> turtle.circle(50) + >>> turtle.position() + (-0.00,0.00) + >>> turtle.heading() + 0.0 + >>> turtle.circle(120, 180) # draw a semicircle + >>> turtle.position() + (0.00,240.00) + >>> turtle.heading() + 180.0 + + -- Function: turtle.dot (size=None, *color) + + + Parameters: + + * ‘size’ – an integer >= 1 (if given) + + * ‘color’ – a colorstring or a numeric color tuple + + Draw a circular dot with diameter `size', using `color'. If `size' + is not given, the maximum of pensize+4 and 2*pensize is used. + + >>> turtle.home() + >>> turtle.dot() + >>> turtle.fd(50); turtle.dot(20, "blue"); turtle.fd(50) + >>> turtle.position() + (100.00,-0.00) + >>> turtle.heading() + 0.0 + + -- Function: turtle.stamp () + + Stamp a copy of the turtle shape onto the canvas at the current + turtle position. Return a stamp_id for that stamp, which can be + used to delete it by calling ‘clearstamp(stamp_id)’. + + >>> turtle.color("blue") + >>> turtle.stamp() + 11 + >>> turtle.fd(50) + + -- Function: turtle.clearstamp (stampid) + + + Parameters: ‘stampid’ – an integer, must be return value of + previous *note stamp(): 2dc5. call + + Delete stamp with given `stampid'. + + >>> turtle.position() + (150.00,-0.00) + >>> turtle.color("blue") + >>> astamp = turtle.stamp() + >>> turtle.fd(50) + >>> turtle.position() + (200.00,-0.00) + >>> turtle.clearstamp(astamp) + >>> turtle.position() + (200.00,-0.00) + + -- Function: turtle.clearstamps (n=None) + + + Parameters: ‘n’ – an integer (or ‘None’) + + Delete all or first/last `n' of turtle’s stamps. If `n' is ‘None’, + delete all stamps, if `n' > 0 delete first `n' stamps, else if `n' + < 0 delete last `n' stamps. + + >>> for i in range(8): + ... turtle.stamp(); turtle.fd(30) + 13 + 14 + 15 + 16 + 17 + 18 + 19 + 20 + >>> turtle.clearstamps(2) + >>> turtle.clearstamps(-2) + >>> turtle.clearstamps() + + -- Function: turtle.undo () + + Undo (repeatedly) the last turtle action(s). Number of available + undo actions is determined by the size of the undobuffer. + + >>> for i in range(4): + ... turtle.fd(50); turtle.lt(80) + ... + >>> for i in range(8): + ... turtle.undo() + + -- Function: turtle.speed (speed=None) + + + Parameters: ‘speed’ – an integer in the range 0..10 or a + speedstring (see below) + + Set the turtle’s speed to an integer value in the range 0..10. If + no argument is given, return current speed. + + If input is a number greater than 10 or smaller than 0.5, speed is + set to 0. Speedstrings are mapped to speedvalues as follows: + + * “fastest”: 0 + + * “fast”: 10 + + * “normal”: 6 + + * “slow”: 3 + + * “slowest”: 1 + + Speeds from 1 to 10 enforce increasingly faster animation of line + drawing and turtle turning. + + Attention: `speed' = 0 means that `no' animation takes place. + forward/back makes turtle jump and likewise left/right make the + turtle turn instantly. + + >>> turtle.speed() + 3 + >>> turtle.speed('normal') + >>> turtle.speed() + 6 + >>> turtle.speed(9) + >>> turtle.speed() + 9 + + +File: python.info, Node: Tell Turtle’s state, Next: Settings for measurement, Prev: Turtle motion, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.7 Tell Turtle’s state +............................ + + -- Function: turtle.position () + -- Function: turtle.pos () + + Return the turtle’s current location (x,y) (as a *note Vec2D: 2e24. + vector). + + >>> turtle.pos() + (440.00,-0.00) + + -- Function: turtle.towards (x, y=None) + + + Parameters: + + * ‘x’ – a number or a pair/vector of numbers or a turtle + instance + + * ‘y’ – a number if `x' is a number, else ‘None’ + + Return the angle between the line from turtle position to position + specified by (x,y), the vector or the other turtle. This depends + on the turtle’s start orientation which depends on the mode - + “standard”/”world” or “logo”. + + >>> turtle.goto(10, 10) + >>> turtle.towards(0,0) + 225.0 + + -- Function: turtle.xcor () + + Return the turtle’s x coordinate. + + >>> turtle.home() + >>> turtle.left(50) + >>> turtle.forward(100) + >>> turtle.pos() + (64.28,76.60) + >>> print(round(turtle.xcor(), 5)) + 64.27876 + + -- Function: turtle.ycor () + + Return the turtle’s y coordinate. + + >>> turtle.home() + >>> turtle.left(60) + >>> turtle.forward(100) + >>> print(turtle.pos()) + (50.00,86.60) + >>> print(round(turtle.ycor(), 5)) + 86.60254 + + -- Function: turtle.heading () + + Return the turtle’s current heading (value depends on the turtle + mode, see *note mode(): 2e13.). + + >>> turtle.home() + >>> turtle.left(67) + >>> turtle.heading() + 67.0 + + -- Function: turtle.distance (x, y=None) + + + Parameters: + + * ‘x’ – a number or a pair/vector of numbers or a turtle + instance + + * ‘y’ – a number if `x' is a number, else ‘None’ + + Return the distance from the turtle to (x,y), the given vector, or + the given other turtle, in turtle step units. + + >>> turtle.home() + >>> turtle.distance(30,40) + 50.0 + >>> turtle.distance((30,40)) + 50.0 + >>> joe = Turtle() + >>> joe.forward(77) + >>> turtle.distance(joe) + 77.0 + + +File: python.info, Node: Settings for measurement, Next: Pen control, Prev: Tell Turtle’s state, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.8 Settings for measurement +................................. + + -- Function: turtle.degrees (fullcircle=360.0) + + + Parameters: ‘fullcircle’ – a number + + Set angle measurement units, i.e. set number of “degrees” for a + full circle. Default value is 360 degrees. + + >>> turtle.home() + >>> turtle.left(90) + >>> turtle.heading() + 90.0 + + Change angle measurement unit to grad (also known as gon, + grade, or gradian and equals 1/100-th of the right angle.) + >>> turtle.degrees(400.0) + >>> turtle.heading() + 100.0 + >>> turtle.degrees(360) + >>> turtle.heading() + 90.0 + + -- Function: turtle.radians () + + Set the angle measurement units to radians. Equivalent to + ‘degrees(2*math.pi)’. + + >>> turtle.home() + >>> turtle.left(90) + >>> turtle.heading() + 90.0 + >>> turtle.radians() + >>> turtle.heading() + 1.5707963267948966 + + +File: python.info, Node: Pen control, Next: Turtle state, Prev: Settings for measurement, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.9 Pen control +.................... + +* Menu: + +* Drawing state:: +* Color control:: +* Filling:: +* More drawing control:: + + +File: python.info, Node: Drawing state, Next: Color control, Up: Pen control + +5.24.1.10 Drawing state +....................... + + -- Function: turtle.pendown () + -- Function: turtle.pd () + -- Function: turtle.down () + + Pull the pen down – drawing when moving. + + -- Function: turtle.penup () + -- Function: turtle.pu () + -- Function: turtle.up () + + Pull the pen up – no drawing when moving. + + -- Function: turtle.pensize (width=None) + -- Function: turtle.width (width=None) + + + Parameters: ‘width’ – a positive number + + Set the line thickness to `width' or return it. If resizemode is + set to “auto” and turtleshape is a polygon, that polygon is drawn + with the same line thickness. If no argument is given, the current + pensize is returned. + + >>> turtle.pensize() + 1 + >>> turtle.pensize(10) # from here on lines of width 10 are drawn + + -- Function: turtle.pen (pen=None, **pendict) + + + Parameters: + + * ‘pen’ – a dictionary with some or all of the below listed keys + + * ‘pendict’ – one or more keyword-arguments with the below + listed keys as keywords + + Return or set the pen’s attributes in a “pen-dictionary” with the + following key/value pairs: + + * “shown”: True/False + + * “pendown”: True/False + + * “pencolor”: color-string or color-tuple + + * “fillcolor”: color-string or color-tuple + + * “pensize”: positive number + + * “speed”: number in range 0..10 + + * “resizemode”: “auto” or “user” or “noresize” + + * “stretchfactor”: (positive number, positive number) + + * “outline”: positive number + + * “tilt”: number + + This dictionary can be used as argument for a subsequent call to + *note pen(): 2ddb. to restore the former pen-state. Moreover one + or more of these attributes can be provided as keyword-arguments. + This can be used to set several pen attributes in one statement. + + >>> turtle.pen(fillcolor="black", pencolor="red", pensize=10) + >>> sorted(turtle.pen().items()) + [('fillcolor', 'black'), ('outline', 1), ('pencolor', 'red'), + ('pendown', True), ('pensize', 10), ('resizemode', 'noresize'), + ('shearfactor', 0.0), ('shown', True), ('speed', 9), + ('stretchfactor', (1.0, 1.0)), ('tilt', 0.0)] + >>> penstate=turtle.pen() + >>> turtle.color("yellow", "") + >>> turtle.penup() + >>> sorted(turtle.pen().items())[:3] + [('fillcolor', ''), ('outline', 1), ('pencolor', 'yellow')] + >>> turtle.pen(penstate, fillcolor="green") + >>> sorted(turtle.pen().items())[:3] + [('fillcolor', 'green'), ('outline', 1), ('pencolor', 'red')] + + -- Function: turtle.isdown () + + Return ‘True’ if pen is down, ‘False’ if it’s up. + + >>> turtle.penup() + >>> turtle.isdown() + False + >>> turtle.pendown() + >>> turtle.isdown() + True + + +File: python.info, Node: Color control, Next: Filling, Prev: Drawing state, Up: Pen control + +5.24.1.11 Color control +....................... + + -- Function: turtle.pencolor (*args) + + Return or set the pencolor. + + Four input formats are allowed: + + ‘pencolor()’ + + Return the current pencolor as color specification string or + as a tuple (see example). May be used as input to another + color/pencolor/fillcolor call. + + ‘pencolor(colorstring)’ + + Set pencolor to `colorstring', which is a Tk color + specification string, such as ‘"red"’, ‘"yellow"’, or + ‘"#33cc8c"’. + + ‘pencolor((r, g, b))’ + + Set pencolor to the RGB color represented by the tuple of `r', + `g', and `b'. Each of `r', `g', and `b' must be in the range + 0..colormode, where colormode is either 1.0 or 255 (see *note + colormode(): 2e14.). + + ‘pencolor(r, g, b)’ + + Set pencolor to the RGB color represented by `r', `g', and + `b'. Each of `r', `g', and `b' must be in the range + 0..colormode. + + If turtleshape is a polygon, the outline of that polygon is drawn + with the newly set pencolor. + + >>> colormode() + 1.0 + >>> turtle.pencolor() + 'red' + >>> turtle.pencolor("brown") + >>> turtle.pencolor() + 'brown' + >>> tup = (0.2, 0.8, 0.55) + >>> turtle.pencolor(tup) + >>> turtle.pencolor() + (0.2, 0.8, 0.5490196078431373) + >>> colormode(255) + >>> turtle.pencolor() + (51.0, 204.0, 140.0) + >>> turtle.pencolor('#32c18f') + >>> turtle.pencolor() + (50.0, 193.0, 143.0) + + -- Function: turtle.fillcolor (*args) + + Return or set the fillcolor. + + Four input formats are allowed: + + ‘fillcolor()’ + + Return the current fillcolor as color specification string, + possibly in tuple format (see example). May be used as input + to another color/pencolor/fillcolor call. + + ‘fillcolor(colorstring)’ + + Set fillcolor to `colorstring', which is a Tk color + specification string, such as ‘"red"’, ‘"yellow"’, or + ‘"#33cc8c"’. + + ‘fillcolor((r, g, b))’ + + Set fillcolor to the RGB color represented by the tuple of + `r', `g', and `b'. Each of `r', `g', and `b' must be in the + range 0..colormode, where colormode is either 1.0 or 255 (see + *note colormode(): 2e14.). + + ‘fillcolor(r, g, b)’ + + Set fillcolor to the RGB color represented by `r', `g', and + `b'. Each of `r', `g', and `b' must be in the range + 0..colormode. + + If turtleshape is a polygon, the interior of that polygon is drawn + with the newly set fillcolor. + + >>> turtle.fillcolor("violet") + >>> turtle.fillcolor() + 'violet' + >>> turtle.pencolor() + (50.0, 193.0, 143.0) + >>> turtle.fillcolor((50, 193, 143)) # Integers, not floats + >>> turtle.fillcolor() + (50.0, 193.0, 143.0) + >>> turtle.fillcolor('#ffffff') + >>> turtle.fillcolor() + (255.0, 255.0, 255.0) + + -- Function: turtle.color (*args) + + Return or set pencolor and fillcolor. + + Several input formats are allowed. They use 0 to 3 arguments as + follows: + + ‘color()’ + + Return the current pencolor and the current fillcolor as a + pair of color specification strings or tuples as returned by + *note pencolor(): 2dde. and *note fillcolor(): 2ddf. + + ‘color(colorstring)’, ‘color((r,g,b))’, ‘color(r,g,b)’ + + Inputs as in *note pencolor(): 2dde, set both, fillcolor and + pencolor, to the given value. + + ‘color(colorstring1, colorstring2)’, ‘color((r1,g1,b1), (r2,g2,b2))’ + + Equivalent to ‘pencolor(colorstring1)’ and + ‘fillcolor(colorstring2)’ and analogously if the other input + format is used. + + If turtleshape is a polygon, outline and interior of that polygon + is drawn with the newly set colors. + + >>> turtle.color("red", "green") + >>> turtle.color() + ('red', 'green') + >>> color("#285078", "#a0c8f0") + >>> color() + ((40.0, 80.0, 120.0), (160.0, 200.0, 240.0)) + +See also: Screen method *note colormode(): 2e14. + + +File: python.info, Node: Filling, Next: More drawing control, Prev: Color control, Up: Pen control + +5.24.1.12 Filling +................. + + -- Function: turtle.filling () + + Return fillstate (‘True’ if filling, ‘False’ else). + + >>> turtle.begin_fill() + >>> if turtle.filling(): + ... turtle.pensize(5) + ... else: + ... turtle.pensize(3) + + -- Function: turtle.begin_fill () + + To be called just before drawing a shape to be filled. + + -- Function: turtle.end_fill () + + Fill the shape drawn after the last call to *note begin_fill(): + 2de1. + + Whether or not overlap regions for self-intersecting polygons or + multiple shapes are filled depends on the operating system + graphics, type of overlap, and number of overlaps. For example, + the Turtle star above may be either all yellow or have some white + regions. + + >>> turtle.color("black", "red") + >>> turtle.begin_fill() + >>> turtle.circle(80) + >>> turtle.end_fill() + + +File: python.info, Node: More drawing control, Prev: Filling, Up: Pen control + +5.24.1.13 More drawing control +.............................. + + -- Function: turtle.reset () + + Delete the turtle’s drawings from the screen, re-center the turtle + and set variables to the default values. + + >>> turtle.goto(0,-22) + >>> turtle.left(100) + >>> turtle.position() + (0.00,-22.00) + >>> turtle.heading() + 100.0 + >>> turtle.reset() + >>> turtle.position() + (0.00,0.00) + >>> turtle.heading() + 0.0 + + -- Function: turtle.clear () + + Delete the turtle’s drawings from the screen. Do not move turtle. + State and position of the turtle as well as drawings of other + turtles are not affected. + + -- Function: turtle.write (arg, move=False, align="left", + font=("Arial", 8, "normal")) + + + Parameters: + + * ‘arg’ – object to be written to the TurtleScreen + + * ‘move’ – True/False + + * ‘align’ – one of the strings “left”, “center” or right” + + * ‘font’ – a triple (fontname, fontsize, fonttype) + + Write text - the string representation of `arg' - at the current + turtle position according to `align' (“left”, “center” or “right”) + and with the given font. If `move' is true, the pen is moved to + the bottom-right corner of the text. By default, `move' is + ‘False’. + + >>> turtle.write("Home = ", True, align="center") + >>> turtle.write((0,0), True) + + +File: python.info, Node: Turtle state, Next: Using events, Prev: Pen control, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.14 Turtle state +...................... + +* Menu: + +* Visibility:: +* Appearance:: + + +File: python.info, Node: Visibility, Next: Appearance, Up: Turtle state + +5.24.1.15 Visibility +.................... + + -- Function: turtle.hideturtle () + -- Function: turtle.ht () + + Make the turtle invisible. It’s a good idea to do this while + you’re in the middle of doing some complex drawing, because hiding + the turtle speeds up the drawing observably. + + >>> turtle.hideturtle() + + -- Function: turtle.showturtle () + -- Function: turtle.st () + + Make the turtle visible. + + >>> turtle.showturtle() + + -- Function: turtle.isvisible () + + Return ‘True’ if the Turtle is shown, ‘False’ if it’s hidden. + + >>> turtle.hideturtle() + >>> turtle.isvisible() + False + >>> turtle.showturtle() + >>> turtle.isvisible() + True + + +File: python.info, Node: Appearance, Prev: Visibility, Up: Turtle state + +5.24.1.16 Appearance +.................... + + -- Function: turtle.shape (name=None) + + + Parameters: ‘name’ – a string which is a valid shapename + + Set turtle shape to shape with given `name' or, if name is not + given, return name of current shape. Shape with `name' must exist + in the TurtleScreen’s shape dictionary. Initially there are the + following polygon shapes: “arrow”, “turtle”, “circle”, “square”, + “triangle”, “classic”. To learn about how to deal with shapes see + Screen method *note register_shape(): 2e17. + + >>> turtle.shape() + 'classic' + >>> turtle.shape("turtle") + >>> turtle.shape() + 'turtle' + + -- Function: turtle.resizemode (rmode=None) + + + Parameters: ‘rmode’ – one of the strings “auto”, “user”, “noresize” + + Set resizemode to one of the values: “auto”, “user”, “noresize”. + If `rmode' is not given, return current resizemode. Different + resizemodes have the following effects: + + - “auto”: adapts the appearance of the turtle corresponding to + the value of pensize. + + - “user”: adapts the appearance of the turtle according to the + values of stretchfactor and outlinewidth (outline), which are + set by *note shapesize(): 2ded. + + - “noresize”: no adaption of the turtle’s appearance takes + place. + + ‘resizemode("user")’ is called by *note shapesize(): 2ded. when + used with arguments. + + >>> turtle.resizemode() + 'noresize' + >>> turtle.resizemode("auto") + >>> turtle.resizemode() + 'auto' + + -- Function: turtle.shapesize (stretch_wid=None, stretch_len=None, + outline=None) + -- Function: turtle.turtlesize (stretch_wid=None, stretch_len=None, + outline=None) + + + Parameters: + + * ‘stretch_wid’ – positive number + + * ‘stretch_len’ – positive number + + * ‘outline’ – positive number + + Return or set the pen’s attributes x/y-stretchfactors and/or + outline. Set resizemode to “user”. If and only if resizemode is + set to “user”, the turtle will be displayed stretched according to + its stretchfactors: `stretch_wid' is stretchfactor perpendicular to + its orientation, `stretch_len' is stretchfactor in direction of its + orientation, `outline' determines the width of the shapes’s + outline. + + >>> turtle.shapesize() + (1.0, 1.0, 1) + >>> turtle.resizemode("user") + >>> turtle.shapesize(5, 5, 12) + >>> turtle.shapesize() + (5, 5, 12) + >>> turtle.shapesize(outline=8) + >>> turtle.shapesize() + (5, 5, 8) + + -- Function: turtle.shearfactor (shear=None) + + + Parameters: ‘shear’ – number (optional) + + Set or return the current shearfactor. Shear the turtleshape + according to the given shearfactor shear, which is the tangent of + the shear angle. Do `not' change the turtle’s heading (direction + of movement). If shear is not given: return the current + shearfactor, i. e. the tangent of the shear angle, by which lines + parallel to the heading of the turtle are sheared. + + >>> turtle.shape("circle") + >>> turtle.shapesize(5,2) + >>> turtle.shearfactor(0.5) + >>> turtle.shearfactor() + 0.5 + + -- Function: turtle.tilt (angle) + + + Parameters: ‘angle’ – a number + + Rotate the turtleshape by `angle' from its current tilt-angle, but + do `not' change the turtle’s heading (direction of movement). + + >>> turtle.reset() + >>> turtle.shape("circle") + >>> turtle.shapesize(5,2) + >>> turtle.tilt(30) + >>> turtle.fd(50) + >>> turtle.tilt(30) + >>> turtle.fd(50) + + -- Function: turtle.settiltangle (angle) + + + Parameters: ‘angle’ – a number + + Rotate the turtleshape to point in the direction specified by + `angle', regardless of its current tilt-angle. `Do not' change the + turtle’s heading (direction of movement). + + >>> turtle.reset() + >>> turtle.shape("circle") + >>> turtle.shapesize(5,2) + >>> turtle.settiltangle(45) + >>> turtle.fd(50) + >>> turtle.settiltangle(-45) + >>> turtle.fd(50) + + Deprecated since version 3.1. + + -- Function: turtle.tiltangle (angle=None) + + + Parameters: ‘angle’ – a number (optional) + + Set or return the current tilt-angle. If angle is given, rotate + the turtleshape to point in the direction specified by angle, + regardless of its current tilt-angle. Do `not' change the turtle’s + heading (direction of movement). If angle is not given: return the + current tilt-angle, i. e. the angle between the orientation of + the turtleshape and the heading of the turtle (its direction of + movement). + + >>> turtle.reset() + >>> turtle.shape("circle") + >>> turtle.shapesize(5,2) + >>> turtle.tilt(45) + >>> turtle.tiltangle() + 45.0 + + -- Function: turtle.shapetransform (t11=None, t12=None, t21=None, + t22=None) + + + Parameters: + + * ‘t11’ – a number (optional) + + * ‘t12’ – a number (optional) + + * ‘t21’ – a number (optional) + + * ‘t12’ – a number (optional) + + Set or return the current transformation matrix of the turtle + shape. + + If none of the matrix elements are given, return the transformation + matrix as a tuple of 4 elements. Otherwise set the given elements + and transform the turtleshape according to the matrix consisting of + first row t11, t12 and second row t21, t22. The determinant t11 * + t22 - t12 * t21 must not be zero, otherwise an error is raised. + Modify stretchfactor, shearfactor and tiltangle according to the + given matrix. + + >>> turtle = Turtle() + >>> turtle.shape("square") + >>> turtle.shapesize(4,2) + >>> turtle.shearfactor(-0.5) + >>> turtle.shapetransform() + (4.0, -1.0, -0.0, 2.0) + + -- Function: turtle.get_shapepoly () + + Return the current shape polygon as tuple of coordinate pairs. + This can be used to define a new shape or components of a compound + shape. + + >>> turtle.shape("square") + >>> turtle.shapetransform(4, -1, 0, 2) + >>> turtle.get_shapepoly() + ((50, -20), (30, 20), (-50, 20), (-30, -20)) + + +File: python.info, Node: Using events, Next: Special Turtle methods, Prev: Turtle state, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.17 Using events +...................... + + -- Function: turtle.onclick (fun, btn=1, add=None) + + + Parameters: + + * ‘fun’ – a function with two arguments which will be called + with the coordinates of the clicked point on the canvas + + * ‘btn’ – number of the mouse-button, defaults to 1 (left mouse + button) + + * ‘add’ – ‘True’ or ‘False’ – if ‘True’, a new binding will be + added, otherwise it will replace a former binding + + Bind `fun' to mouse-click events on this turtle. If `fun' is + ‘None’, existing bindings are removed. Example for the anonymous + turtle, i.e. the procedural way: + + >>> def turn(x, y): + ... left(180) + ... + >>> onclick(turn) # Now clicking into the turtle will turn it. + >>> onclick(None) # event-binding will be removed + + -- Function: turtle.onrelease (fun, btn=1, add=None) + + + Parameters: + + * ‘fun’ – a function with two arguments which will be called + with the coordinates of the clicked point on the canvas + + * ‘btn’ – number of the mouse-button, defaults to 1 (left mouse + button) + + * ‘add’ – ‘True’ or ‘False’ – if ‘True’, a new binding will be + added, otherwise it will replace a former binding + + Bind `fun' to mouse-button-release events on this turtle. If `fun' + is ‘None’, existing bindings are removed. + + >>> class MyTurtle(Turtle): + ... def glow(self,x,y): + ... self.fillcolor("red") + ... def unglow(self,x,y): + ... self.fillcolor("") + ... + >>> turtle = MyTurtle() + >>> turtle.onclick(turtle.glow) # clicking on turtle turns fillcolor red, + >>> turtle.onrelease(turtle.unglow) # releasing turns it to transparent. + + -- Function: turtle.ondrag (fun, btn=1, add=None) + + + Parameters: + + * ‘fun’ – a function with two arguments which will be called + with the coordinates of the clicked point on the canvas + + * ‘btn’ – number of the mouse-button, defaults to 1 (left mouse + button) + + * ‘add’ – ‘True’ or ‘False’ – if ‘True’, a new binding will be + added, otherwise it will replace a former binding + + Bind `fun' to mouse-move events on this turtle. If `fun' is + ‘None’, existing bindings are removed. + + Remark: Every sequence of mouse-move-events on a turtle is preceded + by a mouse-click event on that turtle. + + >>> turtle.ondrag(turtle.goto) + + Subsequently, clicking and dragging the Turtle will move it across + the screen thereby producing handdrawings (if pen is down). + + +File: python.info, Node: Special Turtle methods, Next: Compound shapes, Prev: Using events, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.18 Special Turtle methods +................................ + + -- Function: turtle.begin_poly () + + Start recording the vertices of a polygon. Current turtle position + is first vertex of polygon. + + -- Function: turtle.end_poly () + + Stop recording the vertices of a polygon. Current turtle position + is last vertex of polygon. This will be connected with the first + vertex. + + -- Function: turtle.get_poly () + + Return the last recorded polygon. + + >>> turtle.home() + >>> turtle.begin_poly() + >>> turtle.fd(100) + >>> turtle.left(20) + >>> turtle.fd(30) + >>> turtle.left(60) + >>> turtle.fd(50) + >>> turtle.end_poly() + >>> p = turtle.get_poly() + >>> register_shape("myFavouriteShape", p) + + -- Function: turtle.clone () + + Create and return a clone of the turtle with same position, heading + and turtle properties. + + >>> mick = Turtle() + >>> joe = mick.clone() + + -- Function: turtle.getturtle () + -- Function: turtle.getpen () + + Return the Turtle object itself. Only reasonable use: as a + function to return the “anonymous turtle”: + + >>> pet = getturtle() + >>> pet.fd(50) + >>> pet + <turtle.Turtle object at 0x...> + + -- Function: turtle.getscreen () + + Return the *note TurtleScreen: 2daa. object the turtle is drawing + on. TurtleScreen methods can then be called for that object. + + >>> ts = turtle.getscreen() + >>> ts + <turtle._Screen object at 0x...> + >>> ts.bgcolor("pink") + + -- Function: turtle.setundobuffer (size) + + + Parameters: ‘size’ – an integer or ‘None’ + + Set or disable undobuffer. If `size' is an integer, an empty + undobuffer of given size is installed. `size' gives the maximum + number of turtle actions that can be undone by the *note undo(): + 2dc8. method/function. If `size' is ‘None’, the undobuffer is + disabled. + + >>> turtle.setundobuffer(42) + + -- Function: turtle.undobufferentries () + + Return number of entries in the undobuffer. + + >>> while undobufferentries(): + ... undo() + + +File: python.info, Node: Compound shapes, Prev: Special Turtle methods, Up: Methods of RawTurtle/Turtle and corresponding functions + +5.24.1.19 Compound shapes +......................... + +To use compound turtle shapes, which consist of several polygons of +different color, you must use the helper class *note Shape: 2e33. +explicitly as described below: + + 1. Create an empty Shape object of type “compound”. + + 2. Add as many components to this object as desired, using the + ‘addcomponent()’ method. + + For example: + + >>> s = Shape("compound") + >>> poly1 = ((0,0),(10,-5),(0,10),(-10,-5)) + >>> s.addcomponent(poly1, "red", "blue") + >>> poly2 = ((0,0),(10,-5),(-10,-5)) + >>> s.addcomponent(poly2, "blue", "red") + + 3. Now add the Shape to the Screen’s shapelist and use it: + + >>> register_shape("myshape", s) + >>> shape("myshape") + + Note: The *note Shape: 2e33. class is used internally by the *note + register_shape(): 2e17. method in different ways. The application + programmer has to deal with the Shape class `only' when using + compound shapes like shown above! + + +File: python.info, Node: Methods of TurtleScreen/Screen and corresponding functions, Next: Public classes, Prev: Methods of RawTurtle/Turtle and corresponding functions, Up: turtle — Turtle graphics + +5.24.1.20 Methods of TurtleScreen/Screen and corresponding functions +.................................................................... + +Most of the examples in this section refer to a TurtleScreen instance +called ‘screen’. + +* Menu: + +* Window control:: +* Animation control:: +* Using screen events:: +* Input methods:: +* Settings and special methods:: +* Methods specific to Screen, not inherited from TurtleScreen: Methods specific to Screen not inherited from TurtleScreen. + + +File: python.info, Node: Window control, Next: Animation control, Up: Methods of TurtleScreen/Screen and corresponding functions + +5.24.1.21 Window control +........................ + + -- Function: turtle.bgcolor (*args) + + + Parameters: ‘args’ – a color string or three numbers in the range + 0..colormode or a 3-tuple of such numbers + + Set or return background color of the TurtleScreen. + + >>> screen.bgcolor("orange") + >>> screen.bgcolor() + 'orange' + >>> screen.bgcolor("#800080") + >>> screen.bgcolor() + (128.0, 0.0, 128.0) + + -- Function: turtle.bgpic (picname=None) + + + Parameters: ‘picname’ – a string, name of a gif-file or ‘"nopic"’, + or ‘None’ + + Set background image or return name of current backgroundimage. If + `picname' is a filename, set the corresponding image as background. + If `picname' is ‘"nopic"’, delete background image, if present. If + `picname' is ‘None’, return the filename of the current + backgroundimage. + + >>> screen.bgpic() + 'nopic' + >>> screen.bgpic("landscape.gif") + >>> screen.bgpic() + "landscape.gif" + + -- Function: turtle.clear () + -- Function: turtle.clearscreen () + + Delete all drawings and all turtles from the TurtleScreen. Reset + the now empty TurtleScreen to its initial state: white background, + no background image, no event bindings and tracing on. + + Note: This TurtleScreen method is available as a global + function only under the name ‘clearscreen’. The global + function ‘clear’ is a different one derived from the Turtle + method ‘clear’. + + -- Function: turtle.reset () + -- Function: turtle.resetscreen () + + Reset all Turtles on the Screen to their initial state. + + Note: This TurtleScreen method is available as a global + function only under the name ‘resetscreen’. The global + function ‘reset’ is another one derived from the Turtle method + ‘reset’. + + -- Function: turtle.screensize (canvwidth=None, canvheight=None, + bg=None) + + + Parameters: + + * ‘canvwidth’ – positive integer, new width of canvas in pixels + + * ‘canvheight’ – positive integer, new height of canvas in + pixels + + * ‘bg’ – colorstring or color-tuple, new background color + + If no arguments are given, return current (canvaswidth, + canvasheight). Else resize the canvas the turtles are drawing on. + Do not alter the drawing window. To observe hidden parts of the + canvas, use the scrollbars. With this method, one can make visible + those parts of a drawing which were outside the canvas before. + + >>> screen.screensize() + (400, 300) + >>> screen.screensize(2000,1500) + >>> screen.screensize() + (2000, 1500) + + e.g. to search for an erroneously escaped turtle ;-) + + -- Function: turtle.setworldcoordinates (llx, lly, urx, ury) + + + Parameters: + + * ‘llx’ – a number, x-coordinate of lower left corner of canvas + + * ‘lly’ – a number, y-coordinate of lower left corner of canvas + + * ‘urx’ – a number, x-coordinate of upper right corner of canvas + + * ‘ury’ – a number, y-coordinate of upper right corner of canvas + + Set up user-defined coordinate system and switch to mode “world” if + necessary. This performs a ‘screen.reset()’. If mode “world” is + already active, all drawings are redrawn according to the new + coordinates. + + `ATTENTION': in user-defined coordinate systems angles may appear + distorted. + + >>> screen.reset() + >>> screen.setworldcoordinates(-50,-7.5,50,7.5) + >>> for _ in range(72): + ... left(10) + ... + >>> for _ in range(8): + ... left(45); fd(2) # a regular octagon + + +File: python.info, Node: Animation control, Next: Using screen events, Prev: Window control, Up: Methods of TurtleScreen/Screen and corresponding functions + +5.24.1.22 Animation control +........................... + + -- Function: turtle.delay (delay=None) + + + Parameters: ‘delay’ – positive integer + + Set or return the drawing `delay' in milliseconds. (This is + approximately the time interval between two consecutive canvas + updates.) The longer the drawing delay, the slower the animation. + + Optional argument: + + >>> screen.delay() + 10 + >>> screen.delay(5) + >>> screen.delay() + 5 + + -- Function: turtle.tracer (n=None, delay=None) + + + Parameters: + + * ‘n’ – nonnegative integer + + * ‘delay’ – nonnegative integer + + Turn turtle animation on/off and set delay for update drawings. If + `n' is given, only each n-th regular screen update is really + performed. (Can be used to accelerate the drawing of complex + graphics.) When called without arguments, returns the currently + stored value of n. Second argument sets delay value (see *note + delay(): 2e08.). + + >>> screen.tracer(8, 25) + >>> dist = 2 + >>> for i in range(200): + ... fd(dist) + ... rt(90) + ... dist += 2 + + -- Function: turtle.update () + + Perform a TurtleScreen update. To be used when tracer is turned + off. + +See also the RawTurtle/Turtle method *note speed(): 2dc9. + + +File: python.info, Node: Using screen events, Next: Input methods, Prev: Animation control, Up: Methods of TurtleScreen/Screen and corresponding functions + +5.24.1.23 Using screen events +............................. + + -- Function: turtle.listen (xdummy=None, ydummy=None) + + Set focus on TurtleScreen (in order to collect key-events). Dummy + arguments are provided in order to be able to pass *note listen(): + 2e0b. to the onclick method. + + -- Function: turtle.onkey (fun, key) + -- Function: turtle.onkeyrelease (fun, key) + + + Parameters: + + * ‘fun’ – a function with no arguments or ‘None’ + + * ‘key’ – a string: key (e.g. “a”) or key-symbol (e.g. + “space”) + + Bind `fun' to key-release event of key. If `fun' is ‘None’, event + bindings are removed. Remark: in order to be able to register + key-events, TurtleScreen must have the focus. (See method *note + listen(): 2e0b.) + + >>> def f(): + ... fd(50) + ... lt(60) + ... + >>> screen.onkey(f, "Up") + >>> screen.listen() + + -- Function: turtle.onkeypress (fun, key=None) + + + Parameters: + + * ‘fun’ – a function with no arguments or ‘None’ + + * ‘key’ – a string: key (e.g. “a”) or key-symbol (e.g. + “space”) + + Bind `fun' to key-press event of key if key is given, or to any + key-press-event if no key is given. Remark: in order to be able to + register key-events, TurtleScreen must have focus. (See method + *note listen(): 2e0b.) + + >>> def f(): + ... fd(50) + ... + >>> screen.onkey(f, "Up") + >>> screen.listen() + + -- Function: turtle.onclick (fun, btn=1, add=None) + -- Function: turtle.onscreenclick (fun, btn=1, add=None) + + + Parameters: + + * ‘fun’ – a function with two arguments which will be called + with the coordinates of the clicked point on the canvas + + * ‘btn’ – number of the mouse-button, defaults to 1 (left mouse + button) + + * ‘add’ – ‘True’ or ‘False’ – if ‘True’, a new binding will be + added, otherwise it will replace a former binding + + Bind `fun' to mouse-click events on this screen. If `fun' is + ‘None’, existing bindings are removed. + + Example for a TurtleScreen instance named ‘screen’ and a Turtle + instance named ‘turtle’: + + >>> screen.onclick(turtle.goto) # Subsequently clicking into the TurtleScreen will + >>> # make the turtle move to the clicked point. + >>> screen.onclick(None) # remove event binding again + + Note: This TurtleScreen method is available as a global + function only under the name ‘onscreenclick’. The global + function ‘onclick’ is another one derived from the Turtle + method ‘onclick’. + + -- Function: turtle.ontimer (fun, t=0) + + + Parameters: + + * ‘fun’ – a function with no arguments + + * ‘t’ – a number >= 0 + + Install a timer that calls `fun' after `t' milliseconds. + + >>> running = True + >>> def f(): + ... if running: + ... fd(50) + ... lt(60) + ... screen.ontimer(f, 250) + >>> f() ### makes the turtle march around + >>> running = False + + -- Function: turtle.mainloop () + -- Function: turtle.done () + + Starts event loop - calling Tkinter’s mainloop function. Must be + the last statement in a turtle graphics program. Must `not' be + used if a script is run from within IDLE in -n mode (No subprocess) + - for interactive use of turtle graphics. + + >>> screen.mainloop() + + +File: python.info, Node: Input methods, Next: Settings and special methods, Prev: Using screen events, Up: Methods of TurtleScreen/Screen and corresponding functions + +5.24.1.24 Input methods +....................... + + -- Function: turtle.textinput (title, prompt) + + + Parameters: + + * ‘title’ – string + + * ‘prompt’ – string + + Pop up a dialog window for input of a string. Parameter title is + the title of the dialog window, prompt is a text mostly describing + what information to input. Return the string input. If the dialog + is canceled, return ‘None’. + + >>> screen.textinput("NIM", "Name of first player:") + + -- Function: turtle.numinput (title, prompt, default=None, minval=None, + maxval=None) + + + Parameters: + + * ‘title’ – string + + * ‘prompt’ – string + + * ‘default’ – number (optional) + + * ‘minval’ – number (optional) + + * ‘maxval’ – number (optional) + + Pop up a dialog window for input of a number. title is the title + of the dialog window, prompt is a text mostly describing what + numerical information to input. default: default value, minval: + minimum value for input, maxval: maximum value for input The number + input must be in the range minval .. maxval if these are given. + If not, a hint is issued and the dialog remains open for + correction. Return the number input. If the dialog is canceled, + return ‘None’. + + >>> screen.numinput("Poker", "Your stakes:", 1000, minval=10, maxval=10000) + + +File: python.info, Node: Settings and special methods, Next: Methods specific to Screen not inherited from TurtleScreen, Prev: Input methods, Up: Methods of TurtleScreen/Screen and corresponding functions + +5.24.1.25 Settings and special methods +...................................... + + -- Function: turtle.mode (mode=None) + + + Parameters: ‘mode’ – one of the strings “standard”, “logo” or + “world” + + Set turtle mode (“standard”, “logo” or “world”) and perform reset. + If mode is not given, current mode is returned. + + Mode “standard” is compatible with old *note turtle: 116. Mode + “logo” is compatible with most Logo turtle graphics. Mode “world” + uses user-defined “world coordinates”. `Attention': in this mode + angles appear distorted if ‘x/y’ unit-ratio doesn’t equal 1. + + Mode Initial turtle heading positive angles + + ----------------------------------------------------------------------- + + “standard” to the right (east) counterclockwise + + + “logo” upward (north) clockwise + + + >>> mode("logo") # resets turtle heading to north + >>> mode() + 'logo' + + -- Function: turtle.colormode (cmode=None) + + + Parameters: ‘cmode’ – one of the values 1.0 or 255 + + Return the colormode or set it to 1.0 or 255. Subsequently `r', + `g', `b' values of color triples have to be in the range + 0..`cmode'. + + >>> screen.colormode(1) + >>> turtle.pencolor(240, 160, 80) + Traceback (most recent call last): + ... + TurtleGraphicsError: bad color sequence: (240, 160, 80) + >>> screen.colormode() + 1.0 + >>> screen.colormode(255) + >>> screen.colormode() + 255 + >>> turtle.pencolor(240,160,80) + + -- Function: turtle.getcanvas () + + Return the Canvas of this TurtleScreen. Useful for insiders who + know what to do with a Tkinter Canvas. + + >>> cv = screen.getcanvas() + >>> cv + <turtle.ScrolledCanvas object ...> + + -- Function: turtle.getshapes () + + Return a list of names of all currently available turtle shapes. + + >>> screen.getshapes() + ['arrow', 'blank', 'circle', ..., 'turtle'] + + -- Function: turtle.register_shape (name, shape=None) + -- Function: turtle.addshape (name, shape=None) + + There are three different ways to call this function: + + 1. `name' is the name of a gif-file and `shape' is ‘None’: + Install the corresponding image shape. + + >>> screen.register_shape("turtle.gif") + + Note: Image shapes `do not' rotate when turning the + turtle, so they do not display the heading of the turtle! + + 2. `name' is an arbitrary string and `shape' is a tuple of pairs + of coordinates: Install the corresponding polygon shape. + + >>> screen.register_shape("triangle", ((5,-3), (0,5), (-5,-3))) + + 3. `name' is an arbitrary string and shape is a (compound) *note + Shape: 2e33. object: Install the corresponding compound shape. + + Add a turtle shape to TurtleScreen’s shapelist. Only thusly + registered shapes can be used by issuing the command + ‘shape(shapename)’. + + -- Function: turtle.turtles () + + Return the list of turtles on the screen. + + >>> for turtle in screen.turtles(): + ... turtle.color("red") + + -- Function: turtle.window_height () + + Return the height of the turtle window. + + >>> screen.window_height() + 480 + + -- Function: turtle.window_width () + + Return the width of the turtle window. + + >>> screen.window_width() + 640 + + +File: python.info, Node: Methods specific to Screen not inherited from TurtleScreen, Prev: Settings and special methods, Up: Methods of TurtleScreen/Screen and corresponding functions + +5.24.1.26 Methods specific to Screen, not inherited from TurtleScreen +..................................................................... + + -- Function: turtle.bye () + + Shut the turtlegraphics window. + + -- Function: turtle.exitonclick () + + Bind ‘bye()’ method to mouse clicks on the Screen. + + If the value “using_IDLE” in the configuration dictionary is + ‘False’ (default value), also enter mainloop. Remark: If IDLE with + the ‘-n’ switch (no subprocess) is used, this value should be set + to ‘True’ in ‘turtle.cfg’. In this case IDLE’s own mainloop is + active also for the client script. + + -- Function: turtle.setup (width=_CFG["width"], height=_CFG["height"], + startx=_CFG["leftright"], starty=_CFG["topbottom"]) + + Set the size and position of the main window. Default values of + arguments are stored in the configuration dictionary and can be + changed via a ‘turtle.cfg’ file. + + + Parameters: + + * ‘width’ – if an integer, a size in pixels, if a float, a + fraction of the screen; default is 50% of screen + + * ‘height’ – if an integer, the height in pixels, if a float, a + fraction of the screen; default is 75% of screen + + * ‘startx’ – if positive, starting position in pixels from the + left edge of the screen, if negative from the right edge, if + ‘None’, center window horizontally + + * ‘starty’ – if positive, starting position in pixels from the + top edge of the screen, if negative from the bottom edge, if + ‘None’, center window vertically + + >>> screen.setup (width=200, height=200, startx=0, starty=0) + >>> # sets window to 200x200 pixels, in upper left of screen + >>> screen.setup(width=.75, height=0.5, startx=None, starty=None) + >>> # sets window to 75% of screen by 50% of screen and centers + + -- Function: turtle.title (titlestring) + + + Parameters: ‘titlestring’ – a string that is shown in the titlebar + of the turtle graphics window + + Set title of turtle window to `titlestring'. + + >>> screen.title("Welcome to the turtle zoo!") + + +File: python.info, Node: Public classes, Next: Help and configuration, Prev: Methods of TurtleScreen/Screen and corresponding functions, Up: turtle — Turtle graphics + +5.24.1.27 Public classes +........................ + + -- Class: turtle.RawTurtle (canvas) + -- Class: turtle.RawPen (canvas) + + + Parameters: ‘canvas’ – a ‘tkinter.Canvas’, a *note ScrolledCanvas: + 2dab. or a *note TurtleScreen: 2daa. + + Create a turtle. The turtle has all methods described above as + “methods of Turtle/RawTurtle”. + + -- Class: turtle.Turtle + + Subclass of RawTurtle, has the same interface but draws on a + default *note Screen: 2dac. object created automatically when + needed for the first time. + + -- Class: turtle.TurtleScreen (cv) + + + Parameters: ‘cv’ – a ‘tkinter.Canvas’ + + Provides screen oriented methods like ‘setbg()’ etc. that are + described above. + + -- Class: turtle.Screen + + Subclass of TurtleScreen, with *note four methods added: 2e3b. + + -- Class: turtle.ScrolledCanvas (master) + + + Parameters: ‘master’ – some Tkinter widget to contain the + ScrolledCanvas, i.e. a Tkinter-canvas with scrollbars added + + Used by class Screen, which thus automatically provides a + ScrolledCanvas as playground for the turtles. + + -- Class: turtle.Shape (type_, data) + + + Parameters: ‘type_’ – one of the strings “polygon”, “image”, + “compound” + + Data structure modeling shapes. The pair ‘(type_, data)’ must + follow this specification: + + `type_' `data' + + ------------------------------------------------------------------------------- + + “polygon” a polygon-tuple, i.e. a tuple of pairs of coordinates + + + “image” an image (in this form only used internally!) + + + “compound” ‘None’ (a compound shape has to be constructed using the + *note addcomponent(): 2e3d. method) + + + -- Method: addcomponent (poly, fill, outline=None) + + + Parameters: + + * ‘poly’ – a polygon, i.e. a tuple of pairs of numbers + + * ‘fill’ – a color the `poly' will be filled with + + * ‘outline’ – a color for the poly’s outline (if given) + + Example: + + >>> poly = ((0,0),(10,-5),(0,10),(-10,-5)) + >>> s = Shape("compound") + >>> s.addcomponent(poly, "red", "blue") + >>> # ... add more components and then use register_shape() + + See *note Compound shapes: 2e32. + + -- Class: turtle.Vec2D (x, y) + + A two-dimensional vector class, used as a helper class for + implementing turtle graphics. May be useful for turtle graphics + programs too. Derived from tuple, so a vector is a tuple! + + Provides (for `a', `b' vectors, `k' number): + + * ‘a + b’ vector addition + + * ‘a - b’ vector subtraction + + * ‘a * b’ inner product + + * ‘k * a’ and ‘a * k’ multiplication with scalar + + * ‘abs(a)’ absolute value of a + + * ‘a.rotate(angle)’ rotation + + +File: python.info, Node: Help and configuration, Next: turtledemo — Demo scripts, Prev: Public classes, Up: turtle — Turtle graphics + +5.24.1.28 Help and configuration +................................ + +* Menu: + +* How to use help:: +* Translation of docstrings into different languages:: +* How to configure Screen and Turtles:: + + +File: python.info, Node: How to use help, Next: Translation of docstrings into different languages, Up: Help and configuration + +5.24.1.29 How to use help +......................... + +The public methods of the Screen and Turtle classes are documented +extensively via docstrings. So these can be used as online-help via the +Python help facilities: + + - When using IDLE, tooltips show the signatures and first lines of + the docstrings of typed in function-/method calls. + + - Calling *note help(): 572. on methods or functions displays the + docstrings: + + >>> help(Screen.bgcolor) + Help on method bgcolor in module turtle: + + bgcolor(self, *args) unbound turtle.Screen method + Set or return backgroundcolor of the TurtleScreen. + + Arguments (if given): a color string or three numbers + in the range 0..colormode or a 3-tuple of such numbers. + + + >>> screen.bgcolor("orange") + >>> screen.bgcolor() + "orange" + >>> screen.bgcolor(0.5,0,0.5) + >>> screen.bgcolor() + "#800080" + + >>> help(Turtle.penup) + Help on method penup in module turtle: + + penup(self) unbound turtle.Turtle method + Pull the pen up -- no drawing when moving. + + Aliases: penup | pu | up + + No argument + + >>> turtle.penup() + + - The docstrings of the functions which are derived from methods have + a modified form: + + >>> help(bgcolor) + Help on function bgcolor in module turtle: + + bgcolor(*args) + Set or return backgroundcolor of the TurtleScreen. + + Arguments (if given): a color string or three numbers + in the range 0..colormode or a 3-tuple of such numbers. + + Example:: + + >>> bgcolor("orange") + >>> bgcolor() + "orange" + >>> bgcolor(0.5,0,0.5) + >>> bgcolor() + "#800080" + + >>> help(penup) + Help on function penup in module turtle: + + penup() + Pull the pen up -- no drawing when moving. + + Aliases: penup | pu | up + + No argument + + Example: + >>> penup() + +These modified docstrings are created automatically together with the +function definitions that are derived from the methods at import time. + + +File: python.info, Node: Translation of docstrings into different languages, Next: How to configure Screen and Turtles, Prev: How to use help, Up: Help and configuration + +5.24.1.30 Translation of docstrings into different languages +............................................................ + +There is a utility to create a dictionary the keys of which are the +method names and the values of which are the docstrings of the public +methods of the classes Screen and Turtle. + + -- Function: turtle.write_docstringdict + (filename="turtle_docstringdict") + + + Parameters: ‘filename’ – a string, used as filename + + Create and write docstring-dictionary to a Python script with the + given filename. This function has to be called explicitly (it is + not used by the turtle graphics classes). The docstring dictionary + will be written to the Python script ‘`filename'.py’. It is + intended to serve as a template for translation of the docstrings + into different languages. + +If you (or your students) want to use *note turtle: 116. with online +help in your native language, you have to translate the docstrings and +save the resulting file as e.g. ‘turtle_docstringdict_german.py’. + +If you have an appropriate entry in your ‘turtle.cfg’ file this +dictionary will be read in at import time and will replace the original +English docstrings. + +At the time of this writing there are docstring dictionaries in German +and in Italian. (Requests please to <glingl@aon.at>.) + + +File: python.info, Node: How to configure Screen and Turtles, Prev: Translation of docstrings into different languages, Up: Help and configuration + +5.24.1.31 How to configure Screen and Turtles +............................................. + +The built-in default configuration mimics the appearance and behaviour +of the old turtle module in order to retain best possible compatibility +with it. + +If you want to use a different configuration which better reflects the +features of this module or which better fits to your needs, e.g. for +use in a classroom, you can prepare a configuration file ‘turtle.cfg’ +which will be read at import time and modify the configuration according +to its settings. + +The built in configuration would correspond to the following turtle.cfg: + + width = 0.5 + height = 0.75 + leftright = None + topbottom = None + canvwidth = 400 + canvheight = 300 + mode = standard + colormode = 1.0 + delay = 10 + undobuffersize = 1000 + shape = classic + pencolor = black + fillcolor = black + resizemode = noresize + visible = True + language = english + exampleturtle = turtle + examplescreen = screen + title = Python Turtle Graphics + using_IDLE = False + +Short explanation of selected entries: + + - The first four lines correspond to the arguments of the + ‘Screen.setup()’ method. + + - Line 5 and 6 correspond to the arguments of the method + ‘Screen.screensize()’. + + - `shape' can be any of the built-in shapes, e.g: arrow, turtle, etc. + For more info try ‘help(shape)’. + + - If you want to use no fillcolor (i.e. make the turtle + transparent), you have to write ‘fillcolor = ""’ (but all nonempty + strings must not have quotes in the cfg-file). + + - If you want to reflect the turtle its state, you have to use + ‘resizemode = auto’. + + - If you set e.g. ‘language = italian’ the docstringdict + ‘turtle_docstringdict_italian.py’ will be loaded at import time (if + present on the import path, e.g. in the same directory as *note + turtle: 116. + + - The entries `exampleturtle' and `examplescreen' define the names of + these objects as they occur in the docstrings. The transformation + of method-docstrings to function-docstrings will delete these names + from the docstrings. + + - `using_IDLE': Set this to ‘True’ if you regularly work with IDLE + and its -n switch (“no subprocess”). This will prevent *note + exitonclick(): 2e1f. to enter the mainloop. + +There can be a ‘turtle.cfg’ file in the directory where *note turtle: +116. is stored and an additional one in the current working directory. +The latter will override the settings of the first one. + +The ‘Lib/turtledemo’ directory contains a ‘turtle.cfg’ file. You can +study it as an example and see its effects when running the demos +(preferably not from within the demo-viewer). + + +File: python.info, Node: turtledemo — Demo scripts, Next: Changes since Python 2 6, Prev: Help and configuration, Up: turtle — Turtle graphics + +5.24.1.32 ‘turtledemo’ — Demo scripts +..................................... + +The *note turtledemo: 117. package includes a set of demo scripts. +These scripts can be run and viewed using the supplied demo viewer as +follows: + + python -m turtledemo + +Alternatively, you can run the demo scripts individually. For example, + + python -m turtledemo.bytedesign + +The *note turtledemo: 117. package directory contains: + + - A demo viewer ‘__main__.py’ which can be used to view the + sourcecode of the scripts and run them at the same time. + + - Multiple scripts demonstrating different features of the *note + turtle: 116. module. Examples can be accessed via the Examples + menu. They can also be run standalone. + + - A ‘turtle.cfg’ file which serves as an example of how to write and + use such files. + +The demo scripts are: + +Name Description Features + +------------------------------------------------------------------------------------ + +bytedesign complex classical turtle ‘tracer()’, delay, + graphics pattern ‘update()’ + + +chaos graphs Verhulst dynamics, shows world coordinates + that computer’s computations can + generate results sometimes + against the common sense + expectations + + +clock analog clock showing time of turtles as clock’s hands, + your computer ontimer + + +colormixer experiment with r, g, b ‘ondrag()’ + + +forest 3 breadth-first trees randomization + + +fractalcurves Hilbert & Koch curves recursion + + +lindenmayer ethnomathematics (indian kolams) L-System + + +minimal_hanoi Towers of Hanoi Rectangular Turtles as + Hanoi discs (shape, + shapesize) + + +nim play the classical nim game with turtles as nimsticks, + three heaps of sticks against event driven (mouse, + the computer. keyboard) + + +paint super minimalistic drawing ‘onclick()’ + program + + +peace elementary turtle: appearance and + animation + + +penrose aperiodic tiling with kites and ‘stamp()’ + darts + + +planet_and_moon simulation of gravitational compound shapes, ‘Vec2D’ + system + + +round_dance dancing turtles rotating compound shapes, clone + pairwise in opposite direction shapesize, tilt, + get_shapepoly, update + + +sorting_animate visual demonstration of simple alignment, + different sorting methods randomization + + +tree a (graphical) breadth first tree ‘clone()’ + (using generators) + + +two_canvases simple design turtles on two canvases + + +wikipedia a pattern from the wikipedia ‘clone()’, ‘undo()’ + article on turtle graphics + + +yinyang another elementary example ‘circle()’ + + +Have fun! + + +File: python.info, Node: Changes since Python 2 6, Next: Changes since Python 3 0, Prev: turtledemo — Demo scripts, Up: turtle — Turtle graphics + +5.24.1.33 Changes since Python 2.6 +.................................. + + - The methods ‘Turtle.tracer()’, ‘Turtle.window_width()’ and + ‘Turtle.window_height()’ have been eliminated. Methods with these + names and functionality are now available only as methods of + ‘Screen’. The functions derived from these remain available. (In + fact already in Python 2.6 these methods were merely duplications + of the corresponding ‘TurtleScreen’/‘Screen’-methods.) + + - The method ‘Turtle.fill()’ has been eliminated. The behaviour of + ‘begin_fill()’ and ‘end_fill()’ have changed slightly: now every + filling-process must be completed with an ‘end_fill()’ call. + + - A method ‘Turtle.filling()’ has been added. It returns a boolean + value: ‘True’ if a filling process is under way, ‘False’ otherwise. + This behaviour corresponds to a ‘fill()’ call without arguments in + Python 2.6. + + +File: python.info, Node: Changes since Python 3 0, Prev: Changes since Python 2 6, Up: turtle — Turtle graphics + +5.24.1.34 Changes since Python 3.0 +.................................. + + - The methods ‘Turtle.shearfactor()’, ‘Turtle.shapetransform()’ and + ‘Turtle.get_shapepoly()’ have been added. Thus the full range of + regular linear transforms is now available for transforming turtle + shapes. ‘Turtle.tiltangle()’ has been enhanced in functionality: + it now can be used to get or set the tiltangle. + ‘Turtle.settiltangle()’ has been deprecated. + + - The method ‘Screen.onkeypress()’ has been added as a complement to + ‘Screen.onkey()’ which in fact binds actions to the keyrelease + event. Accordingly the latter has got an alias: + ‘Screen.onkeyrelease()’. + + - The method ‘Screen.mainloop()’ has been added. So when working + only with Screen and Turtle objects one must not additionally + import ‘mainloop()’ anymore. + + - Two input methods has been added ‘Screen.textinput()’ and + ‘Screen.numinput()’. These popup input dialogs and return strings + and numbers respectively. + + - Two example scripts ‘tdemo_nim.py’ and ‘tdemo_round_dance.py’ have + been added to the ‘Lib/turtledemo’ directory. + + +File: python.info, Node: cmd — Support for line-oriented command interpreters, Next: shlex — Simple lexical analysis, Prev: turtle — Turtle graphics, Up: Program Frameworks + +5.24.2 ‘cmd’ — Support for line-oriented command interpreters +------------------------------------------------------------- + +`Source code:' Lib/cmd.py(1) + +__________________________________________________________________ + +The *note Cmd: 2e48. class provides a simple framework for writing +line-oriented command interpreters. These are often useful for test +harnesses, administrative tools, and prototypes that will later be +wrapped in a more sophisticated interface. + + -- Class: cmd.Cmd (completekey='tab', stdin=None, stdout=None) + + A *note Cmd: 2e48. instance or subclass instance is a line-oriented + interpreter framework. There is no good reason to instantiate + *note Cmd: 2e48. itself; rather, it’s useful as a superclass of an + interpreter class you define yourself in order to inherit *note + Cmd: 2e48.’s methods and encapsulate action methods. + + The optional argument `completekey' is the *note readline: de. name + of a completion key; it defaults to ‘Tab’. If `completekey' is not + *note None: 157. and *note readline: de. is available, command + completion is done automatically. + + The optional arguments `stdin' and `stdout' specify the input and + output file objects that the Cmd instance or subclass instance will + use for input and output. If not specified, they will default to + *note sys.stdin: 30d. and *note sys.stdout: 30e. + + If you want a given `stdin' to be used, make sure to set the + instance’s *note use_rawinput: 2e49. attribute to ‘False’, + otherwise `stdin' will be ignored. + +* Menu: + +* Cmd Objects:: +* Cmd Example:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/cmd.py + + +File: python.info, Node: Cmd Objects, Next: Cmd Example, Up: cmd — Support for line-oriented command interpreters + +5.24.2.1 Cmd Objects +.................... + +A *note Cmd: 2e48. instance has the following methods: + + -- Method: Cmd.cmdloop (intro=None) + + Repeatedly issue a prompt, accept input, parse an initial prefix + off the received input, and dispatch to action methods, passing + them the remainder of the line as argument. + + The optional argument is a banner or intro string to be issued + before the first prompt (this overrides the *note intro: 2e4d. + class attribute). + + If the *note readline: de. module is loaded, input will + automatically inherit ‘bash’-like history-list editing (e.g. + ‘Control-P’ scrolls back to the last command, ‘Control-N’ forward + to the next one, ‘Control-F’ moves the cursor to the right + non-destructively, ‘Control-B’ moves the cursor to the left + non-destructively, etc.). + + An end-of-file on input is passed back as the string ‘'EOF'’. + + An interpreter instance will recognize a command name ‘foo’ if and + only if it has a method ‘do_foo()’. As a special case, a line + beginning with the character ‘'?'’ is dispatched to the method + ‘do_help()’. As another special case, a line beginning with the + character ‘'!'’ is dispatched to the method ‘do_shell()’ (if such a + method is defined). + + This method will return when the *note postcmd(): 2e4e. method + returns a true value. The `stop' argument to *note postcmd(): + 2e4e. is the return value from the command’s corresponding ‘do_*()’ + method. + + If completion is enabled, completing commands will be done + automatically, and completing of commands args is done by calling + ‘complete_foo()’ with arguments `text', `line', `begidx', and + `endidx'. `text' is the string prefix we are attempting to match: + all returned matches must begin with it. `line' is the current + input line with leading whitespace removed, `begidx' and `endidx' + are the beginning and ending indexes of the prefix text, which + could be used to provide different completion depending upon which + position the argument is in. + + All subclasses of *note Cmd: 2e48. inherit a predefined + ‘do_help()’. This method, called with an argument ‘'bar'’, invokes + the corresponding method ‘help_bar()’, and if that is not present, + prints the docstring of ‘do_bar()’, if available. With no + argument, ‘do_help()’ lists all available help topics (that is, all + commands with corresponding ‘help_*()’ methods or commands that + have docstrings), and also lists any undocumented commands. + + -- Method: Cmd.onecmd (str) + + Interpret the argument as though it had been typed in response to + the prompt. This may be overridden, but should not normally need + to be; see the *note precmd(): 2e50. and *note postcmd(): 2e4e. + methods for useful execution hooks. The return value is a flag + indicating whether interpretation of commands by the interpreter + should stop. If there is a ‘do_*()’ method for the command `str', + the return value of that method is returned, otherwise the return + value from the *note default(): 2e51. method is returned. + + -- Method: Cmd.emptyline () + + Method called when an empty line is entered in response to the + prompt. If this method is not overridden, it repeats the last + nonempty command entered. + + -- Method: Cmd.default (line) + + Method called on an input line when the command prefix is not + recognized. If this method is not overridden, it prints an error + message and returns. + + -- Method: Cmd.completedefault (text, line, begidx, endidx) + + Method called to complete an input line when no command-specific + ‘complete_*()’ method is available. By default, it returns an + empty list. + + -- Method: Cmd.precmd (line) + + Hook method executed just before the command line `line' is + interpreted, but after the input prompt is generated and issued. + This method is a stub in *note Cmd: 2e48.; it exists to be + overridden by subclasses. The return value is used as the command + which will be executed by the *note onecmd(): 2e4f. method; the + *note precmd(): 2e50. implementation may re-write the command or + simply return `line' unchanged. + + -- Method: Cmd.postcmd (stop, line) + + Hook method executed just after a command dispatch is finished. + This method is a stub in *note Cmd: 2e48.; it exists to be + overridden by subclasses. `line' is the command line which was + executed, and `stop' is a flag which indicates whether execution + will be terminated after the call to *note postcmd(): 2e4e.; this + will be the return value of the *note onecmd(): 2e4f. method. The + return value of this method will be used as the new value for the + internal flag which corresponds to `stop'; returning false will + cause interpretation to continue. + + -- Method: Cmd.preloop () + + Hook method executed once when *note cmdloop(): 2e4c. is called. + This method is a stub in *note Cmd: 2e48.; it exists to be + overridden by subclasses. + + -- Method: Cmd.postloop () + + Hook method executed once when *note cmdloop(): 2e4c. is about to + return. This method is a stub in *note Cmd: 2e48.; it exists to be + overridden by subclasses. + +Instances of *note Cmd: 2e48. subclasses have some public instance +variables: + + -- Attribute: Cmd.prompt + + The prompt issued to solicit input. + + -- Attribute: Cmd.identchars + + The string of characters accepted for the command prefix. + + -- Attribute: Cmd.lastcmd + + The last nonempty command prefix seen. + + -- Attribute: Cmd.cmdqueue + + A list of queued input lines. The cmdqueue list is checked in + *note cmdloop(): 2e4c. when new input is needed; if it is nonempty, + its elements will be processed in order, as if entered at the + prompt. + + -- Attribute: Cmd.intro + + A string to issue as an intro or banner. May be overridden by + giving the *note cmdloop(): 2e4c. method an argument. + + -- Attribute: Cmd.doc_header + + The header to issue if the help output has a section for documented + commands. + + -- Attribute: Cmd.misc_header + + The header to issue if the help output has a section for + miscellaneous help topics (that is, there are ‘help_*()’ methods + without corresponding ‘do_*()’ methods). + + -- Attribute: Cmd.undoc_header + + The header to issue if the help output has a section for + undocumented commands (that is, there are ‘do_*()’ methods without + corresponding ‘help_*()’ methods). + + -- Attribute: Cmd.ruler + + The character used to draw separator lines under the help-message + headers. If empty, no ruler line is drawn. It defaults to ‘'='’. + + -- Attribute: Cmd.use_rawinput + + A flag, defaulting to true. If true, *note cmdloop(): 2e4c. uses + *note input(): c6b. to display a prompt and read the next command; + if false, ‘sys.stdout.write()’ and ‘sys.stdin.readline()’ are used. + (This means that by importing *note readline: de, on systems that + support it, the interpreter will automatically support ‘Emacs’-like + line editing and command-history keystrokes.) + + +File: python.info, Node: Cmd Example, Prev: Cmd Objects, Up: cmd — Support for line-oriented command interpreters + +5.24.2.2 Cmd Example +.................... + +The *note cmd: 1a. module is mainly useful for building custom shells +that let a user work with a program interactively. + +This section presents a simple example of how to build a shell around a +few of the commands in the *note turtle: 116. module. + +Basic turtle commands such as *note forward(): 2db2. are added to a +*note Cmd: 2e48. subclass with method named ‘do_forward()’. The +argument is converted to a number and dispatched to the turtle module. +The docstring is used in the help utility provided by the shell. + +The example also includes a basic record and playback facility +implemented with the *note precmd(): 2e50. method which is responsible +for converting the input to lowercase and writing the commands to a +file. The ‘do_playback()’ method reads the file and adds the recorded +commands to the ‘cmdqueue’ for immediate playback: + + import cmd, sys + from turtle import * + + class TurtleShell(cmd.Cmd): + intro = 'Welcome to the turtle shell. Type help or ? to list commands.\n' + prompt = '(turtle) ' + file = None + + # ----- basic turtle commands ----- + def do_forward(self, arg): + 'Move the turtle forward by the specified distance: FORWARD 10' + forward(*parse(arg)) + def do_right(self, arg): + 'Turn turtle right by given number of degrees: RIGHT 20' + right(*parse(arg)) + def do_left(self, arg): + 'Turn turtle left by given number of degrees: LEFT 90' + left(*parse(arg)) + def do_goto(self, arg): + 'Move turtle to an absolute position with changing orientation. GOTO 100 200' + goto(*parse(arg)) + def do_home(self, arg): + 'Return turtle to the home position: HOME' + home() + def do_circle(self, arg): + 'Draw circle with given radius an options extent and steps: CIRCLE 50' + circle(*parse(arg)) + def do_position(self, arg): + 'Print the current turtle position: POSITION' + print('Current position is %d %d\n' % position()) + def do_heading(self, arg): + 'Print the current turtle heading in degrees: HEADING' + print('Current heading is %d\n' % (heading(),)) + def do_color(self, arg): + 'Set the color: COLOR BLUE' + color(arg.lower()) + def do_undo(self, arg): + 'Undo (repeatedly) the last turtle action(s): UNDO' + def do_reset(self, arg): + 'Clear the screen and return turtle to center: RESET' + reset() + def do_bye(self, arg): + 'Stop recording, close the turtle window, and exit: BYE' + print('Thank you for using Turtle') + self.close() + bye() + return True + + # ----- record and playback ----- + def do_record(self, arg): + 'Save future commands to filename: RECORD rose.cmd' + self.file = open(arg, 'w') + def do_playback(self, arg): + 'Playback commands from a file: PLAYBACK rose.cmd' + self.close() + with open(arg) as f: + self.cmdqueue.extend(f.read().splitlines()) + def precmd(self, line): + line = line.lower() + if self.file and 'playback' not in line: + print(line, file=self.file) + return line + def close(self): + if self.file: + self.file.close() + self.file = None + + def parse(arg): + 'Convert a series of zero or more numbers to an argument tuple' + return tuple(map(int, arg.split())) + + if __name__ == '__main__': + TurtleShell().cmdloop() + +Here is a sample session with the turtle shell showing the help +functions, using blank lines to repeat commands, and the simple record +and playback facility: + + Welcome to the turtle shell. Type help or ? to list commands. + + (turtle) ? + + Documented commands (type help <topic>): + ======================================== + bye color goto home playback record right + circle forward heading left position reset undo + + (turtle) help forward + Move the turtle forward by the specified distance: FORWARD 10 + (turtle) record spiral.cmd + (turtle) position + Current position is 0 0 + + (turtle) heading + Current heading is 0 + + (turtle) reset + (turtle) circle 20 + (turtle) right 30 + (turtle) circle 40 + (turtle) right 30 + (turtle) circle 60 + (turtle) right 30 + (turtle) circle 80 + (turtle) right 30 + (turtle) circle 100 + (turtle) right 30 + (turtle) circle 120 + (turtle) right 30 + (turtle) circle 120 + (turtle) heading + Current heading is 180 + + (turtle) forward 100 + (turtle) + (turtle) right 90 + (turtle) forward 100 + (turtle) + (turtle) right 90 + (turtle) forward 400 + (turtle) right 90 + (turtle) forward 500 + (turtle) right 90 + (turtle) forward 400 + (turtle) right 90 + (turtle) forward 300 + (turtle) playback spiral.cmd + Current position is 0 0 + + Current heading is 0 + + Current heading is 180 + + (turtle) bye + Thank you for using Turtle + + +File: python.info, Node: shlex — Simple lexical analysis, Prev: cmd — Support for line-oriented command interpreters, Up: Program Frameworks + +5.24.3 ‘shlex’ — Simple lexical analysis +---------------------------------------- + +`Source code:' Lib/shlex.py(1) + +__________________________________________________________________ + +The *note shlex: 57a. class makes it easy to write lexical analyzers for +simple syntaxes resembling that of the Unix shell. This will often be +useful for writing minilanguages, (for example, in run control files for +Python applications) or for parsing quoted strings. + +The *note shlex: e8. module defines the following functions: + + -- Function: shlex.split (s, comments=False, posix=True) + + Split the string `s' using shell-like syntax. If `comments' is + *note False: 388. (the default), the parsing of comments in the + given string will be disabled (setting the *note commenters: 2e62. + attribute of the *note shlex: 57a. instance to the empty string). + This function operates in POSIX mode by default, but uses non-POSIX + mode if the `posix' argument is false. + + Note: Since the *note split(): 218. function instantiates a + *note shlex: 57a. instance, passing ‘None’ for `s' will read + the string to split from standard input. + + -- Function: shlex.join (split_command) + + Concatenate the tokens of the list `split_command' and return a + string. This function is the inverse of *note split(): 218. + + >>> from shlex import join + >>> print(join(['echo', '-n', 'Multiple words'])) + echo -n 'Multiple words' + + The returned value is shell-escaped to protect against injection + vulnerabilities (see *note quote(): a73.). + + New in version 3.8. + + -- Function: shlex.quote (s) + + Return a shell-escaped version of the string `s'. The returned + value is a string that can safely be used as one token in a shell + command line, for cases where you cannot use a list. + + This idiom would be unsafe: + + >>> filename = 'somefile; rm -rf ~' + >>> command = 'ls -l {}'.format(filename) + >>> print(command) # executed by a shell: boom! + ls -l somefile; rm -rf ~ + + *note quote(): a73. lets you plug the security hole: + + >>> from shlex import quote + >>> command = 'ls -l {}'.format(quote(filename)) + >>> print(command) + ls -l 'somefile; rm -rf ~' + >>> remote_command = 'ssh home {}'.format(quote(command)) + >>> print(remote_command) + ssh home 'ls -l '"'"'somefile; rm -rf ~'"'"'' + + The quoting is compatible with UNIX shells and with *note split(): + 218.: + + >>> from shlex import split + >>> remote_command = split(remote_command) + >>> remote_command + ['ssh', 'home', "ls -l 'somefile; rm -rf ~'"] + >>> command = split(remote_command[-1]) + >>> command + ['ls', '-l', 'somefile; rm -rf ~'] + + New in version 3.3. + +The *note shlex: e8. module defines the following class: + + -- Class: shlex.shlex (instream=None, infile=None, posix=False, + punctuation_chars=False) + + A *note shlex: 57a. instance or subclass instance is a lexical + analyzer object. The initialization argument, if present, + specifies where to read characters from. It must be a + file-/stream-like object with *note read(): 1ce1. and *note + readline(): 18b9. methods, or a string. If no argument is given, + input will be taken from ‘sys.stdin’. The second optional argument + is a filename string, which sets the initial value of the *note + infile: 2e63. attribute. If the `instream' argument is omitted or + equal to ‘sys.stdin’, this second argument defaults to “stdin”. + The `posix' argument defines the operational mode: when `posix' is + not true (default), the *note shlex: 57a. instance will operate in + compatibility mode. When operating in POSIX mode, *note shlex: + 57a. will try to be as close as possible to the POSIX shell parsing + rules. The `punctuation_chars' argument provides a way to make the + behaviour even closer to how real shells parse. This can take a + number of values: the default value, ‘False’, preserves the + behaviour seen under Python 3.5 and earlier. If set to ‘True’, + then parsing of the characters ‘();<>|&’ is changed: any run of + these characters (considered punctuation characters) is returned as + a single token. If set to a non-empty string of characters, those + characters will be used as the punctuation characters. Any + characters in the *note wordchars: 2e64. attribute that appear in + `punctuation_chars' will be removed from *note wordchars: 2e64. + See *note Improved Compatibility with Shells: 57b. for more + information. `punctuation_chars' can be set only upon *note shlex: + 57a. instance creation and can’t be modified later. + + Changed in version 3.6: The `punctuation_chars' parameter was + added. + +See also +........ + +Module *note configparser: 23. + + Parser for configuration files similar to the Windows ‘.ini’ files. + +* Menu: + +* shlex Objects:: +* Parsing Rules:: +* Improved Compatibility with Shells:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/shlex.py + + +File: python.info, Node: shlex Objects, Next: Parsing Rules, Up: shlex — Simple lexical analysis + +5.24.3.1 shlex Objects +...................... + +A *note shlex: 57a. instance has the following methods: + + -- Method: shlex.get_token () + + Return a token. If tokens have been stacked using *note + push_token(): 2e68, pop a token off the stack. Otherwise, read one + from the input stream. If reading encounters an immediate + end-of-file, *note eof: 2e69. is returned (the empty string (‘''’) + in non-POSIX mode, and ‘None’ in POSIX mode). + + -- Method: shlex.push_token (str) + + Push the argument onto the token stack. + + -- Method: shlex.read_token () + + Read a raw token. Ignore the pushback stack, and do not interpret + source requests. (This is not ordinarily a useful entry point, and + is documented here only for the sake of completeness.) + + -- Method: shlex.sourcehook (filename) + + When *note shlex: 57a. detects a source request (see *note source: + 2e6c. below) this method is given the following token as argument, + and expected to return a tuple consisting of a filename and an open + file-like object. + + Normally, this method first strips any quotes off the argument. If + the result is an absolute pathname, or there was no previous source + request in effect, or the previous source was a stream (such as + ‘sys.stdin’), the result is left alone. Otherwise, if the result + is a relative pathname, the directory part of the name of the file + immediately before it on the source inclusion stack is prepended + (this behavior is like the way the C preprocessor handles ‘#include + "file.h"’). + + The result of the manipulations is treated as a filename, and + returned as the first component of the tuple, with *note open(): + 4f0. called on it to yield the second component. (Note: this is + the reverse of the order of arguments in instance initialization!) + + This hook is exposed so that you can use it to implement directory + search paths, addition of file extensions, and other namespace + hacks. There is no corresponding ‘close’ hook, but a shlex + instance will call the *note close(): 1bdd. method of the sourced + input stream when it returns EOF. + + For more explicit control of source stacking, use the *note + push_source(): 2e6d. and *note pop_source(): 2e6e. methods. + + -- Method: shlex.push_source (newstream, newfile=None) + + Push an input source stream onto the input stack. If the filename + argument is specified it will later be available for use in error + messages. This is the same method used internally by the *note + sourcehook(): 2e6b. method. + + -- Method: shlex.pop_source () + + Pop the last-pushed input source from the input stack. This is the + same method used internally when the lexer reaches EOF on a stacked + input stream. + + -- Method: shlex.error_leader (infile=None, lineno=None) + + This method generates an error message leader in the format of a + Unix C compiler error label; the format is ‘'"%s", line %d: '’, + where the ‘%s’ is replaced with the name of the current source file + and the ‘%d’ with the current input line number (the optional + arguments can be used to override these). + + This convenience is provided to encourage *note shlex: e8. users to + generate error messages in the standard, parseable format + understood by Emacs and other Unix tools. + +Instances of *note shlex: 57a. subclasses have some public instance +variables which either control lexical analysis or can be used for +debugging: + + -- Attribute: shlex.commenters + + The string of characters that are recognized as comment beginners. + All characters from the comment beginner to end of line are + ignored. Includes just ‘'#'’ by default. + + -- Attribute: shlex.wordchars + + The string of characters that will accumulate into multi-character + tokens. By default, includes all ASCII alphanumerics and + underscore. In POSIX mode, the accented characters in the Latin-1 + set are also included. If *note punctuation_chars: 2e70. is not + empty, the characters ‘~-./*?=’, which can appear in filename + specifications and command line parameters, will also be included + in this attribute, and any characters which appear in + ‘punctuation_chars’ will be removed from ‘wordchars’ if they are + present there. If *note whitespace_split: 2e71. is set to ‘True’, + this will have no effect. + + -- Attribute: shlex.whitespace + + Characters that will be considered whitespace and skipped. + Whitespace bounds tokens. By default, includes space, tab, + linefeed and carriage-return. + + -- Attribute: shlex.escape + + Characters that will be considered as escape. This will be only + used in POSIX mode, and includes just ‘'\'’ by default. + + -- Attribute: shlex.quotes + + Characters that will be considered string quotes. The token + accumulates until the same quote is encountered again (thus, + different quote types protect each other as in the shell.) By + default, includes ASCII single and double quotes. + + -- Attribute: shlex.escapedquotes + + Characters in *note quotes: 2e74. that will interpret escape + characters defined in *note escape: 2e73. This is only used in + POSIX mode, and includes just ‘'"'’ by default. + + -- Attribute: shlex.whitespace_split + + If ‘True’, tokens will only be split in whitespaces. This is + useful, for example, for parsing command lines with *note shlex: + 57a, getting tokens in a similar way to shell arguments. When used + in combination with *note punctuation_chars: 2e70, tokens will be + split on whitespace in addition to those characters. + + Changed in version 3.8: The *note punctuation_chars: 2e70. + attribute was made compatible with the *note whitespace_split: + 2e71. attribute. + + -- Attribute: shlex.infile + + The name of the current input file, as initially set at class + instantiation time or stacked by later source requests. It may be + useful to examine this when constructing error messages. + + -- Attribute: shlex.instream + + The input stream from which this *note shlex: 57a. instance is + reading characters. + + -- Attribute: shlex.source + + This attribute is ‘None’ by default. If you assign a string to it, + that string will be recognized as a lexical-level inclusion request + similar to the ‘source’ keyword in various shells. That is, the + immediately following token will be opened as a filename and input + will be taken from that stream until EOF, at which point the *note + close(): 1bdd. method of that stream will be called and the input + source will again become the original input stream. Source + requests may be stacked any number of levels deep. + + -- Attribute: shlex.debug + + If this attribute is numeric and ‘1’ or more, a *note shlex: 57a. + instance will print verbose progress output on its behavior. If + you need to use this, you can read the module source code to learn + the details. + + -- Attribute: shlex.lineno + + Source line number (count of newlines seen so far plus one). + + -- Attribute: shlex.token + + The token buffer. It may be useful to examine this when catching + exceptions. + + -- Attribute: shlex.eof + + Token used to determine end of file. This will be set to the empty + string (‘''’), in non-POSIX mode, and to ‘None’ in POSIX mode. + + -- Attribute: shlex.punctuation_chars + + A read-only property. Characters that will be considered + punctuation. Runs of punctuation characters will be returned as a + single token. However, note that no semantic validity checking + will be performed: for example, ‘>>>’ could be returned as a token, + even though it may not be recognised as such by shells. + + New in version 3.6. + + +File: python.info, Node: Parsing Rules, Next: Improved Compatibility with Shells, Prev: shlex Objects, Up: shlex — Simple lexical analysis + +5.24.3.2 Parsing Rules +...................... + +When operating in non-POSIX mode, *note shlex: 57a. will try to obey to +the following rules. + + * Quote characters are not recognized within words (‘Do"Not"Separate’ + is parsed as the single word ‘Do"Not"Separate’); + + * Escape characters are not recognized; + + * Enclosing characters in quotes preserve the literal value of all + characters within the quotes; + + * Closing quotes separate words (‘"Do"Separate’ is parsed as ‘"Do"’ + and ‘Separate’); + + * If *note whitespace_split: 2e71. is ‘False’, any character not + declared to be a word character, whitespace, or a quote will be + returned as a single-character token. If it is ‘True’, *note + shlex: 57a. will only split words in whitespaces; + + * EOF is signaled with an empty string (‘''’); + + * It’s not possible to parse empty strings, even if quoted. + +When operating in POSIX mode, *note shlex: 57a. will try to obey to the +following parsing rules. + + * Quotes are stripped out, and do not separate words + (‘"Do"Not"Separate"’ is parsed as the single word ‘DoNotSeparate’); + + * Non-quoted escape characters (e.g. ‘'\'’) preserve the literal + value of the next character that follows; + + * Enclosing characters in quotes which are not part of *note + escapedquotes: 2e75. (e.g. ‘"'"’) preserve the literal value of + all characters within the quotes; + + * Enclosing characters in quotes which are part of *note + escapedquotes: 2e75. (e.g. ‘'"'’) preserves the literal value of + all characters within the quotes, with the exception of the + characters mentioned in *note escape: 2e73. The escape characters + retain its special meaning only when followed by the quote in use, + or the escape character itself. Otherwise the escape character + will be considered a normal character. + + * EOF is signaled with a *note None: 157. value; + + * Quoted empty strings (‘''’) are allowed. + + +File: python.info, Node: Improved Compatibility with Shells, Prev: Parsing Rules, Up: shlex — Simple lexical analysis + +5.24.3.3 Improved Compatibility with Shells +........................................... + +New in version 3.6. + +The *note shlex: e8. class provides compatibility with the parsing +performed by common Unix shells like ‘bash’, ‘dash’, and ‘sh’. To take +advantage of this compatibility, specify the ‘punctuation_chars’ +argument in the constructor. This defaults to ‘False’, which preserves +pre-3.6 behaviour. However, if it is set to ‘True’, then parsing of the +characters ‘();<>|&’ is changed: any run of these characters is returned +as a single token. While this is short of a full parser for shells +(which would be out of scope for the standard library, given the +multiplicity of shells out there), it does allow you to perform +processing of command lines more easily than you could otherwise. To +illustrate, you can see the difference in the following snippet: + + >>> import shlex + >>> text = "a && b; c && d || e; f >'abc'; (def \"ghi\")" + >>> s = shlex.shlex(text, posix=True) + >>> s.whitespace_split = True + >>> list(s) + ['a', '&&', 'b;', 'c', '&&', 'd', '||', 'e;', 'f', '>abc;', '(def', 'ghi)'] + >>> s = shlex.shlex(text, posix=True, punctuation_chars=True) + >>> s.whitespace_split = True + >>> list(s) + ['a', '&&', 'b', ';', 'c', '&&', 'd', '||', 'e', ';', 'f', '>', 'abc', ';', + '(', 'def', 'ghi', ')'] + +Of course, tokens will be returned which are not valid for shells, and +you’ll need to implement your own error checks on the returned tokens. + +Instead of passing ‘True’ as the value for the punctuation_chars +parameter, you can pass a string with specific characters, which will be +used to determine which characters constitute punctuation. For example: + + >>> import shlex + >>> s = shlex.shlex("a && b || c", punctuation_chars="|") + >>> list(s) + ['a', '&', '&', 'b', '||', 'c'] + + Note: When ‘punctuation_chars’ is specified, the *note wordchars: + 2e64. attribute is augmented with the characters ‘~-./*?=’. That + is because these characters can appear in file names (including + wildcards) and command-line arguments (e.g. ‘--color=auto’). + Hence: + + >>> import shlex + >>> s = shlex.shlex('~/a && b-c --color=auto || d *.py?', + ... punctuation_chars=True) + >>> list(s) + ['~/a', '&&', 'b-c', '--color=auto', '||', 'd', '*.py?'] + + However, to match the shell as closely as possible, it is + recommended to always use ‘posix’ and *note whitespace_split: 2e71. + when using *note punctuation_chars: 2e70, which will negate *note + wordchars: 2e64. entirely. + +For best effect, ‘punctuation_chars’ should be set in conjunction with +‘posix=True’. (Note that ‘posix=False’ is the default for *note shlex: +57a.) + + +File: python.info, Node: Graphical User Interfaces with Tk, Next: Development Tools, Prev: Program Frameworks, Up: The Python Standard Library + +5.25 Graphical User Interfaces with Tk +====================================== + +Tk/Tcl has long been an integral part of Python. It provides a robust +and platform independent windowing toolkit, that is available to Python +programmers using the *note tkinter: 10c. package, and its extension, +the *note tkinter.tix: 10e. and the *note tkinter.ttk: 10f. modules. + +The *note tkinter: 10c. package is a thin object-oriented layer on top +of Tcl/Tk. To use *note tkinter: 10c, you don’t need to write Tcl code, +but you will need to consult the Tk documentation, and occasionally the +Tcl documentation. *note tkinter: 10c. is a set of wrappers that +implement the Tk widgets as Python classes. In addition, the internal +module ‘_tkinter’ provides a threadsafe mechanism which allows Python +and Tcl to interact. + +*note tkinter: 10c.’s chief virtues are that it is fast, and that it +usually comes bundled with Python. Although its standard documentation +is weak, good material is available, which includes: references, +tutorials, a book and others. *note tkinter: 10c. is also famous for +having an outdated look and feel, which has been vastly improved in Tk +8.5. Nevertheless, there are many other GUI libraries that you could be +interested in. For more information about alternatives, see the *note +Other Graphical User Interface Packages: 2e80. section. + +* Menu: + +* tkinter — Python interface to Tcl/Tk:: +* tkinter.ttk — Tk themed widgets: tkinter ttk — Tk themed widgets. +* tkinter.tix — Extension widgets for Tk: tkinter tix — Extension widgets for Tk. +* tkinter.scrolledtext — Scrolled Text Widget: tkinter scrolledtext — Scrolled Text Widget. +* IDLE: IDLE<3>. +* Other Graphical User Interface Packages:: + + +File: python.info, Node: tkinter — Python interface to Tcl/Tk, Next: tkinter ttk — Tk themed widgets, Up: Graphical User Interfaces with Tk + +5.25.1 ‘tkinter’ — Python interface to Tcl/Tk +--------------------------------------------- + +`Source code:' Lib/tkinter/__init__.py(1) + +__________________________________________________________________ + +The *note tkinter: 10c. package (“Tk interface”) is the standard Python +interface to the Tk GUI toolkit. Both Tk and *note tkinter: 10c. are +available on most Unix platforms, as well as on Windows systems. (Tk +itself is not part of Python; it is maintained at ActiveState.) + +Running ‘python -m tkinter’ from the command line should open a window +demonstrating a simple Tk interface, letting you know that *note +tkinter: 10c. is properly installed on your system, and also showing +what version of Tcl/Tk is installed, so you can read the Tcl/Tk +documentation specific to that version. + +See also +........ + +Tkinter documentation: + +Python Tkinter Resources(2) + + The Python Tkinter Topic Guide provides a great deal of information + on using Tk from Python and links to other sources of information + on Tk. + +TKDocs(3) + + Extensive tutorial plus friendlier widget pages for some of the + widgets. + +Tkinter 8.5 reference: a GUI for Python(4) + + On-line reference material. + +Tkinter docs from effbot(5) + + Online reference for tkinter supported by effbot.org. + +Programming Python(6) + + Book by Mark Lutz, has excellent coverage of Tkinter. + +Modern Tkinter for Busy Python Developers(7) + + Book by Mark Roseman about building attractive and modern graphical + user interfaces with Python and Tkinter. + +Python and Tkinter Programming(8) + + Book by John Grayson (ISBN 1-884777-81-3). + +Tcl/Tk documentation: + +Tk commands(9) + + Most commands are available as *note tkinter: 10c. or *note + tkinter.ttk: 10f. classes. Change ‘8.6’ to match the version of + your Tcl/Tk installation. + +Tcl/Tk recent man pages(10) + + Recent Tcl/Tk manuals on www.tcl.tk. + +ActiveState Tcl Home Page(11) + + The Tk/Tcl development is largely taking place at ActiveState. + +Tcl and the Tk Toolkit(12) + + Book by John Ousterhout, the inventor of Tcl. + +Practical Programming in Tcl and Tk(13) + + Brent Welch’s encyclopedic book. + +* Menu: + +* Tkinter Modules:: +* Tkinter Life Preserver:: +* A (Very) Quick Look at Tcl/Tk: A Very Quick Look at Tcl/Tk. +* Mapping Basic Tk into Tkinter:: +* How Tk and Tkinter are Related:: +* Handy Reference:: +* File Handlers:: + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/tkinter/__init__.py + + (2) https://wiki.python.org/moin/TkInter + + (3) http://www.tkdocs.com/ + + (4) https://www.tkdocs.com/shipman/ + + (5) http://effbot.org/tkinterbook/ + + (6) http://learning-python.com/about-pp4e.html + + (7) +https://www.amazon.com/Modern-Tkinter-Python-Developers-ebook/dp/B0071QDNLO/ + + (8) https://www.manning.com/books/python-and-tkinter-programming + + (9) https://www.tcl.tk/man/tcl8.6/TkCmd/contents.htm + + (10) https://www.tcl.tk/doc/ + + (11) https://tcl.tk + + (12) https://www.amazon.com/exec/obidos/ASIN/020163337X + + (13) http://www.beedub.com/book/ + + +File: python.info, Node: Tkinter Modules, Next: Tkinter Life Preserver, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.1 Tkinter Modules +........................ + +Most of the time, *note tkinter: 10c. is all you really need, but a +number of additional modules are available as well. The Tk interface is +located in a binary module named ‘_tkinter’. This module contains the +low-level interface to Tk, and should never be used directly by +application programmers. It is usually a shared library (or DLL), but +might in some cases be statically linked with the Python interpreter. + +In addition to the Tk interface module, *note tkinter: 10c. includes a +number of Python modules, ‘tkinter.constants’ being one of the most +important. Importing *note tkinter: 10c. will automatically import +‘tkinter.constants’, so, usually, to use Tkinter all you need is a +simple import statement: + + import tkinter + +Or, more often: + + from tkinter import * + + -- Class: tkinter.Tk (screenName=None, baseName=None, className='Tk', + useTk=1) + + The *note Tk: 2e84. class is instantiated without arguments. This + creates a toplevel widget of Tk which usually is the main window of + an application. Each instance has its own associated Tcl + interpreter. + + -- Function: tkinter.Tcl (screenName=None, baseName=None, + className='Tk', useTk=0) + + The *note Tcl(): 2e85. function is a factory function which creates + an object much like that created by the *note Tk: 2e84. class, + except that it does not initialize the Tk subsystem. This is most + often useful when driving the Tcl interpreter in an environment + where one doesn’t want to create extraneous toplevel windows, or + where one cannot (such as Unix/Linux systems without an X server). + An object created by the *note Tcl(): 2e85. object can have a + Toplevel window created (and the Tk subsystem initialized) by + calling its ‘loadtk()’ method. + +Other modules that provide Tk support include: + +*note tkinter.scrolledtext: 10d. + + Text widget with a vertical scroll bar built in. + +‘tkinter.colorchooser’ + + Dialog to let the user choose a color. + +‘tkinter.commondialog’ + + Base class for the dialogs defined in the other modules listed + here. + +‘tkinter.filedialog’ + + Common dialogs to allow the user to specify a file to open or save. + +‘tkinter.font’ + + Utilities to help work with fonts. + +‘tkinter.messagebox’ + + Access to standard Tk dialog boxes. + +‘tkinter.simpledialog’ + + Basic dialogs and convenience functions. + +‘tkinter.dnd’ + + Drag-and-drop support for *note tkinter: 10c. This is experimental + and should become deprecated when it is replaced with the Tk DND. + +*note turtle: 116. + + Turtle graphics in a Tk window. + + +File: python.info, Node: Tkinter Life Preserver, Next: A Very Quick Look at Tcl/Tk, Prev: Tkinter Modules, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.2 Tkinter Life Preserver +............................... + +This section is not designed to be an exhaustive tutorial on either Tk +or Tkinter. Rather, it is intended as a stop gap, providing some +introductory orientation on the system. + +Credits: + + * Tk was written by John Ousterhout while at Berkeley. + + * Tkinter was written by Steen Lumholt and Guido van Rossum. + + * This Life Preserver was written by Matt Conway at the University of + Virginia. + + * The HTML rendering, and some liberal editing, was produced from a + FrameMaker version by Ken Manheimer. + + * Fredrik Lundh elaborated and revised the class interface + descriptions, to get them current with Tk 4.2. + + * Mike Clarkson converted the documentation to LaTeX, and compiled + the User Interface chapter of the reference manual. + +* Menu: + +* How To Use This Section:: +* A Simple Hello World Program:: + + +File: python.info, Node: How To Use This Section, Next: A Simple Hello World Program, Up: Tkinter Life Preserver + +5.25.1.3 How To Use This Section +................................ + +This section is designed in two parts: the first half (roughly) covers +background material, while the second half can be taken to the keyboard +as a handy reference. + +When trying to answer questions of the form “how do I do blah”, it is +often best to find out how to do “blah” in straight Tk, and then convert +this back into the corresponding *note tkinter: 10c. call. Python +programmers can often guess at the correct Python command by looking at +the Tk documentation. This means that in order to use Tkinter, you will +have to know a little bit about Tk. This document can’t fulfill that +role, so the best we can do is point you to the best documentation that +exists. Here are some hints: + + * The authors strongly suggest getting a copy of the Tk man pages. + Specifically, the man pages in the ‘manN’ directory are most + useful. The ‘man3’ man pages describe the C interface to the Tk + library and thus are not especially helpful for script writers. + + * Addison-Wesley publishes a book called Tcl and the Tk Toolkit by + John Ousterhout (ISBN 0-201-63337-X) which is a good introduction + to Tcl and Tk for the novice. The book is not exhaustive, and for + many details it defers to the man pages. + + * ‘tkinter/__init__.py’ is a last resort for most, but can be a good + place to go when nothing else makes sense. + + +File: python.info, Node: A Simple Hello World Program, Prev: How To Use This Section, Up: Tkinter Life Preserver + +5.25.1.4 A Simple Hello World Program +..................................... + + import tkinter as tk + + class Application(tk.Frame): + def __init__(self, master=None): + super().__init__(master) + self.master = master + self.pack() + self.create_widgets() + + def create_widgets(self): + self.hi_there = tk.Button(self) + self.hi_there["text"] = "Hello World\n(click me)" + self.hi_there["command"] = self.say_hi + self.hi_there.pack(side="top") + + self.quit = tk.Button(self, text="QUIT", fg="red", + command=self.master.destroy) + self.quit.pack(side="bottom") + + def say_hi(self): + print("hi there, everyone!") + + root = tk.Tk() + app = Application(master=root) + app.mainloop() + + +File: python.info, Node: A Very Quick Look at Tcl/Tk, Next: Mapping Basic Tk into Tkinter, Prev: Tkinter Life Preserver, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.5 A (Very) Quick Look at Tcl/Tk +...................................... + +The class hierarchy looks complicated, but in actual practice, +application programmers almost always refer to the classes at the very +bottom of the hierarchy. + +Notes: + + * These classes are provided for the purposes of organizing certain + functions under one namespace. They aren’t meant to be + instantiated independently. + + * The *note Tk: 2e84. class is meant to be instantiated only once in + an application. Application programmers need not instantiate one + explicitly, the system creates one whenever any of the other + classes are instantiated. + + * The ‘Widget’ class is not meant to be instantiated, it is meant + only for subclassing to make “real” widgets (in C++, this is called + an ‘abstract class’). + +To make use of this reference material, there will be times when you +will need to know how to read short passages of Tk and how to identify +the various parts of a Tk command. (See section *note Mapping Basic Tk +into Tkinter: 2e8a. for the *note tkinter: 10c. equivalents of what’s +below.) + +Tk scripts are Tcl programs. Like all Tcl programs, Tk scripts are just +lists of tokens separated by spaces. A Tk widget is just its `class', +the `options' that help configure it, and the `actions' that make it do +useful things. + +To make a widget in Tk, the command is always of the form: + + classCommand newPathname options + +`classCommand' + + denotes which kind of widget to make (a button, a label, a menu…) + +`newPathname' + + is the new name for this widget. All names in Tk must be unique. + To help enforce this, widgets in Tk are named with `pathnames', + just like files in a file system. The top level widget, the + `root', is called ‘.’ (period) and children are delimited by more + periods. For example, ‘.myApp.controlPanel.okButton’ might be the + name of a widget. + +`options' + + configure the widget’s appearance and in some cases, its behavior. + The options come in the form of a list of flags and values. Flags + are preceded by a ‘-‘, like Unix shell command flags, and values + are put in quotes if they are more than one word. + +For example: + + button .fred -fg red -text "hi there" + ^ ^ \______________________/ + | | | + class new options + command widget (-opt val -opt val ...) + +Once created, the pathname to the widget becomes a new command. This +new `widget command' is the programmer’s handle for getting the new +widget to perform some `action'. In C, you’d express this as +someAction(fred, someOptions), in C++, you would express this as +fred.someAction(someOptions), and in Tk, you say: + + .fred someAction someOptions + +Note that the object name, ‘.fred’, starts with a dot. + +As you’d expect, the legal values for `someAction' will depend on the +widget’s class: ‘.fred disable’ works if fred is a button (fred gets +greyed out), but does not work if fred is a label (disabling of labels +is not supported in Tk). + +The legal values of `someOptions' is action dependent. Some actions, +like ‘disable’, require no arguments, others, like a text-entry box’s +‘delete’ command, would need arguments to specify what range of text to +delete. + + +File: python.info, Node: Mapping Basic Tk into Tkinter, Next: How Tk and Tkinter are Related, Prev: A Very Quick Look at Tcl/Tk, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.6 Mapping Basic Tk into Tkinter +...................................... + +Class commands in Tk correspond to class constructors in Tkinter. + + button .fred =====> fred = Button() + +The master of an object is implicit in the new name given to it at +creation time. In Tkinter, masters are specified explicitly. + + button .panel.fred =====> fred = Button(panel) + +The configuration options in Tk are given in lists of hyphened tags +followed by values. In Tkinter, options are specified as +keyword-arguments in the instance constructor, and keyword-args for +configure calls or as instance indices, in dictionary style, for +established instances. See section *note Setting Options: 2e8c. on +setting options. + + button .fred -fg red =====> fred = Button(panel, fg="red") + .fred configure -fg red =====> fred["fg"] = red + OR ==> fred.config(fg="red") + +In Tk, to perform an action on a widget, use the widget name as a +command, and follow it with an action name, possibly with arguments +(options). In Tkinter, you call methods on the class instance to invoke +actions on the widget. The actions (methods) that a given widget can +perform are listed in ‘tkinter/__init__.py’. + + .fred invoke =====> fred.invoke() + +To give a widget to the packer (geometry manager), you call pack with +optional arguments. In Tkinter, the Pack class holds all this +functionality, and the various forms of the pack command are implemented +as methods. All widgets in *note tkinter: 10c. are subclassed from the +Packer, and so inherit all the packing methods. See the *note +tkinter.tix: 10e. module documentation for additional information on the +Form geometry manager. + + pack .fred -side left =====> fred.pack(side="left") + + +File: python.info, Node: How Tk and Tkinter are Related, Next: Handy Reference, Prev: Mapping Basic Tk into Tkinter, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.7 How Tk and Tkinter are Related +....................................... + +From the top down: + +Your App Here (Python) + + A Python application makes a *note tkinter: 10c. call. + +tkinter (Python Package) + + This call (say, for example, creating a button widget), is + implemented in the *note tkinter: 10c. package, which is written in + Python. This Python function will parse the commands and the + arguments and convert them into a form that makes them look as if + they had come from a Tk script instead of a Python script. + +_tkinter (C) + + These commands and their arguments will be passed to a C function + in the ‘_tkinter’ - note the underscore - extension module. + +Tk Widgets (C and Tcl) + + This C function is able to make calls into other C modules, + including the C functions that make up the Tk library. Tk is + implemented in C and some Tcl. The Tcl part of the Tk widgets is + used to bind certain default behaviors to widgets, and is executed + once at the point where the Python *note tkinter: 10c. package is + imported. (The user never sees this stage). + +Tk (C) + + The Tk part of the Tk Widgets implement the final mapping to … + +Xlib (C) + + the Xlib library to draw graphics on the screen. + + +File: python.info, Node: Handy Reference, Next: File Handlers, Prev: How Tk and Tkinter are Related, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.8 Handy Reference +........................ + +* Menu: + +* Setting Options:: +* The Packer:: +* Packer Options:: +* Coupling Widget Variables:: +* The Window Manager:: +* Tk Option Data Types:: +* Bindings and Events:: +* The index Parameter:: +* Images:: + + +File: python.info, Node: Setting Options, Next: The Packer, Up: Handy Reference + +5.25.1.9 Setting Options +........................ + +Options control things like the color and border width of a widget. +Options can be set in three ways: + +At object creation time, using keyword arguments + + fred = Button(self, fg="red", bg="blue") + +After object creation, treating the option name like a dictionary index + + fred["fg"] = "red" + fred["bg"] = "blue" + +Use the config() method to update multiple attrs subsequent to object creation + + fred.config(fg="red", bg="blue") + +For a complete explanation of a given option and its behavior, see the +Tk man pages for the widget in question. + +Note that the man pages list “STANDARD OPTIONS” and “WIDGET SPECIFIC +OPTIONS” for each widget. The former is a list of options that are +common to many widgets, the latter are the options that are +idiosyncratic to that particular widget. The Standard Options are +documented on the ‘options(3)’ man page. + +No distinction between standard and widget-specific options is made in +this document. Some options don’t apply to some kinds of widgets. +Whether a given widget responds to a particular option depends on the +class of the widget; buttons have a ‘command’ option, labels do not. + +The options supported by a given widget are listed in that widget’s man +page, or can be queried at runtime by calling the ‘config()’ method +without arguments, or by calling the ‘keys()’ method on that widget. +The return value of these calls is a dictionary whose key is the name of +the option as a string (for example, ‘'relief'’) and whose values are +5-tuples. + +Some options, like ‘bg’ are synonyms for common options with long names +(‘bg’ is shorthand for “background”). Passing the ‘config()’ method the +name of a shorthand option will return a 2-tuple, not 5-tuple. The +2-tuple passed back will contain the name of the synonym and the “real” +option (such as ‘('bg', 'background')’). + +Index Meaning Example + +--------------------------------------------------------------------- + +0 option name ‘'relief'’ + + +1 option name for database lookup ‘'relief'’ + + +2 option class for database lookup ‘'Relief'’ + + +3 default value ‘'raised'’ + + +4 current value ‘'groove'’ + + +Example: + + >>> print(fred.config()) + {'relief': ('relief', 'relief', 'Relief', 'raised', 'groove')} + +Of course, the dictionary printed will include all the options available +and their values. This is meant only as an example. + + +File: python.info, Node: The Packer, Next: Packer Options, Prev: Setting Options, Up: Handy Reference + +5.25.1.10 The Packer +.................... + +The packer is one of Tk’s geometry-management mechanisms. Geometry +managers are used to specify the relative positioning of widgets within +their container - their mutual `master'. In contrast to the more +cumbersome `placer' (which is used less commonly, and we do not cover +here), the packer takes qualitative relationship specification - +`above', `to the left of', `filling', etc - and works everything out to +determine the exact placement coordinates for you. + +The size of any `master' widget is determined by the size of the “slave +widgets” inside. The packer is used to control where slave widgets +appear inside the master into which they are packed. You can pack +widgets into frames, and frames into other frames, in order to achieve +the kind of layout you desire. Additionally, the arrangement is +dynamically adjusted to accommodate incremental changes to the +configuration, once it is packed. + +Note that widgets do not appear until they have had their geometry +specified with a geometry manager. It’s a common early mistake to leave +out the geometry specification, and then be surprised when the widget is +created but nothing appears. A widget will appear only after it has +had, for example, the packer’s ‘pack()’ method applied to it. + +The pack() method can be called with keyword-option/value pairs that +control where the widget is to appear within its container, and how it +is to behave when the main application window is resized. Here are some +examples: + + fred.pack() # defaults to side = "top" + fred.pack(side="left") + fred.pack(expand=1) + + +File: python.info, Node: Packer Options, Next: Coupling Widget Variables, Prev: The Packer, Up: Handy Reference + +5.25.1.11 Packer Options +........................ + +For more extensive information on the packer and the options that it can +take, see the man pages and page 183 of John Ousterhout’s book. + +anchor + + Anchor type. Denotes where the packer is to place each slave in + its parcel. + +expand + + Boolean, ‘0’ or ‘1’. + +fill + + Legal values: ‘'x'’, ‘'y'’, ‘'both'’, ‘'none'’. + +ipadx and ipady + + A distance - designating internal padding on each side of the slave + widget. + +padx and pady + + A distance - designating external padding on each side of the slave + widget. + +side + + Legal values are: ‘'left'’, ‘'right'’, ‘'top'’, ‘'bottom'’. + + +File: python.info, Node: Coupling Widget Variables, Next: The Window Manager, Prev: Packer Options, Up: Handy Reference + +5.25.1.12 Coupling Widget Variables +................................... + +The current-value setting of some widgets (like text entry widgets) can +be connected directly to application variables by using special options. +These options are ‘variable’, ‘textvariable’, ‘onvalue’, ‘offvalue’, and +‘value’. This connection works both ways: if the variable changes for +any reason, the widget it’s connected to will be updated to reflect the +new value. + +Unfortunately, in the current implementation of *note tkinter: 10c. it +is not possible to hand over an arbitrary Python variable to a widget +through a ‘variable’ or ‘textvariable’ option. The only kinds of +variables for which this works are variables that are subclassed from a +class called Variable, defined in *note tkinter: 10c. + +There are many useful subclasses of Variable already defined: +‘StringVar’, ‘IntVar’, ‘DoubleVar’, and ‘BooleanVar’. To read the +current value of such a variable, call the ‘get()’ method on it, and to +change its value you call the ‘set()’ method. If you follow this +protocol, the widget will always track the value of the variable, with +no further intervention on your part. + +For example: + + import tkinter as tk + + class App(tk.Frame): + def __init__(self, master): + super().__init__(master) + self.pack() + + self.entrythingy = tk.Entry() + self.entrythingy.pack() + + # Create the application variable. + self.contents = tk.StringVar() + # Set it to some value. + self.contents.set("this is a variable") + # Tell the entry widget to watch this variable. + self.entrythingy["textvariable"] = self.contents + + # Define a callback for when the user hits return. + # It prints the current value of the variable. + self.entrythingy.bind('<Key-Return>', + self.print_contents) + + def print_contents(self, event): + print("Hi. The current entry content is:", + self.contents.get()) + + root = tk.Tk() + myapp = App(root) + myapp.mainloop() + + +File: python.info, Node: The Window Manager, Next: Tk Option Data Types, Prev: Coupling Widget Variables, Up: Handy Reference + +5.25.1.13 The Window Manager +............................ + +In Tk, there is a utility command, ‘wm’, for interacting with the window +manager. Options to the ‘wm’ command allow you to control things like +titles, placement, icon bitmaps, and the like. In *note tkinter: 10c, +these commands have been implemented as methods on the ‘Wm’ class. +Toplevel widgets are subclassed from the ‘Wm’ class, and so can call the +‘Wm’ methods directly. + +To get at the toplevel window that contains a given widget, you can +often just refer to the widget’s master. Of course if the widget has +been packed inside of a frame, the master won’t represent a toplevel +window. To get at the toplevel window that contains an arbitrary +widget, you can call the ‘_root()’ method. This method begins with an +underscore to denote the fact that this function is part of the +implementation, and not an interface to Tk functionality. + +Here are some examples of typical usage: + + import tkinter as tk + + class App(tk.Frame): + def __init__(self, master=None): + super().__init__(master) + self.pack() + + # create the application + myapp = App() + + # + # here are method calls to the window manager class + # + myapp.master.title("My Do-Nothing Application") + myapp.master.maxsize(1000, 400) + + # start the program + myapp.mainloop() + + +File: python.info, Node: Tk Option Data Types, Next: Bindings and Events, Prev: The Window Manager, Up: Handy Reference + +5.25.1.14 Tk Option Data Types +.............................. + +anchor + + Legal values are points of the compass: ‘"n"’, ‘"ne"’, ‘"e"’, + ‘"se"’, ‘"s"’, ‘"sw"’, ‘"w"’, ‘"nw"’, and also ‘"center"’. + +bitmap + + There are eight built-in, named bitmaps: ‘'error'’, ‘'gray25'’, + ‘'gray50'’, ‘'hourglass'’, ‘'info'’, ‘'questhead'’, ‘'question'’, + ‘'warning'’. To specify an X bitmap filename, give the full path + to the file, preceded with an ‘@’, as in + ‘"@/usr/contrib/bitmap/gumby.bit"’. + +boolean + + You can pass integers 0 or 1 or the strings ‘"yes"’ or ‘"no"’. + +callback + + This is any Python function that takes no arguments. For example: + + def print_it(): + print("hi there") + fred["command"] = print_it + +color + + Colors can be given as the names of X colors in the rgb.txt file, + or as strings representing RGB values in 4 bit: ‘"#RGB"’, 8 bit: + ‘"#RRGGBB"’, 12 bit” ‘"#RRRGGGBBB"’, or 16 bit ‘"#RRRRGGGGBBBB"’ + ranges, where R,G,B here represent any legal hex digit. See page + 160 of Ousterhout’s book for details. + +cursor + + The standard X cursor names from ‘cursorfont.h’ can be used, + without the ‘XC_’ prefix. For example to get a hand cursor + (‘XC_hand2’), use the string ‘"hand2"’. You can also specify a + bitmap and mask file of your own. See page 179 of Ousterhout’s + book. + +distance + + Screen distances can be specified in either pixels or absolute + distances. Pixels are given as numbers and absolute distances as + strings, with the trailing character denoting units: ‘c’ for + centimetres, ‘i’ for inches, ‘m’ for millimetres, ‘p’ for printer’s + points. For example, 3.5 inches is expressed as ‘"3.5i"’. + +font + + Tk uses a list font name format, such as ‘{courier 10 bold}’. Font + sizes with positive numbers are measured in points; sizes with + negative numbers are measured in pixels. + +geometry + + This is a string of the form ‘widthxheight’, where width and height + are measured in pixels for most widgets (in characters for widgets + displaying text). For example: ‘fred["geometry"] = "200x100"’. + +justify + + Legal values are the strings: ‘"left"’, ‘"center"’, ‘"right"’, and + ‘"fill"’. + +region + + This is a string with four space-delimited elements, each of which + is a legal distance (see above). For example: ‘"2 3 4 5"’ and ‘"3i + 2i 4.5i 2i"’ and ‘"3c 2c 4c 10.43c"’ are all legal regions. + +relief + + Determines what the border style of a widget will be. Legal values + are: ‘"raised"’, ‘"sunken"’, ‘"flat"’, ‘"groove"’, and ‘"ridge"’. + +scrollcommand + + This is almost always the ‘set()’ method of some scrollbar widget, + but can be any widget method that takes a single argument. + +wrap: + + Must be one of: ‘"none"’, ‘"char"’, or ‘"word"’. + + +File: python.info, Node: Bindings and Events, Next: The index Parameter, Prev: Tk Option Data Types, Up: Handy Reference + +5.25.1.15 Bindings and Events +............................. + +The bind method from the widget command allows you to watch for certain +events and to have a callback function trigger when that event type +occurs. The form of the bind method is: + + def bind(self, sequence, func, add=''): + +where: + +sequence + + is a string that denotes the target kind of event. (See the bind + man page and page 201 of John Ousterhout’s book for details). + +func + + is a Python function, taking one argument, to be invoked when the + event occurs. An Event instance will be passed as the argument. + (Functions deployed this way are commonly known as `callbacks'.) + +add + + is optional, either ‘''’ or ‘'+'’. Passing an empty string denotes + that this binding is to replace any other bindings that this event + is associated with. Passing a ‘'+'’ means that this function is to + be added to the list of functions bound to this event type. + +For example: + + def turn_red(self, event): + event.widget["activeforeground"] = "red" + + self.button.bind("<Enter>", self.turn_red) + +Notice how the widget field of the event is being accessed in the +‘turn_red()’ callback. This field contains the widget that caught the X +event. The following table lists the other event fields you can access, +and how they are denoted in Tk, which can be useful when referring to +the Tk man pages. + +Tk Tkinter Event Field Tk Tkinter Event Field + +---------------------------------------------------------------------- + +%f focus %A char + + +%h height %E send_event + + +%k keycode %K keysym + + +%s state %N keysym_num + + +%t time %T type + + +%w width %W widget + + +%x x %X x_root + + +%y y %Y y_root + + + +File: python.info, Node: The index Parameter, Next: Images, Prev: Bindings and Events, Up: Handy Reference + +5.25.1.16 The index Parameter +............................. + +A number of widgets require “index” parameters to be passed. These are +used to point at a specific place in a Text widget, or to particular +characters in an Entry widget, or to particular menu items in a Menu +widget. + +Entry widget indexes (index, view index, etc.) + + Entry widgets have options that refer to character positions in the + text being displayed. You can use these *note tkinter: 10c. + functions to access these special points in text widgets: + +Text widget indexes + + The index notation for Text widgets is very rich and is best + described in the Tk man pages. + +Menu indexes (menu.invoke(), menu.entryconfig(), etc.) + + Some options and methods for menus manipulate specific menu + entries. Anytime a menu index is needed for an option or a + parameter, you may pass in: + + * an integer which refers to the numeric position of the entry + in the widget, counted from the top, starting with 0; + + * the string ‘"active"’, which refers to the menu position that + is currently under the cursor; + + * the string ‘"last"’ which refers to the last menu item; + + * An integer preceded by ‘@’, as in ‘@6’, where the integer is + interpreted as a y pixel coordinate in the menu’s coordinate + system; + + * the string ‘"none"’, which indicates no menu entry at all, + most often used with menu.activate() to deactivate all + entries, and finally, + + * a text string that is pattern matched against the label of the + menu entry, as scanned from the top of the menu to the bottom. + Note that this index type is considered after all the others, + which means that matches for menu items labelled ‘last’, + ‘active’, or ‘none’ may be interpreted as the above literals, + instead. + + +File: python.info, Node: Images, Prev: The index Parameter, Up: Handy Reference + +5.25.1.17 Images +................ + +Images of different formats can be created through the corresponding +subclass of ‘tkinter.Image’: + + * ‘BitmapImage’ for images in XBM format. + + * ‘PhotoImage’ for images in PGM, PPM, GIF and PNG formats. The + latter is supported starting with Tk 8.6. + +Either type of image is created through either the ‘file’ or the ‘data’ +option (other options are available as well). + +The image object can then be used wherever an ‘image’ option is +supported by some widget (e.g. labels, buttons, menus). In these +cases, Tk will not keep a reference to the image. When the last Python +reference to the image object is deleted, the image data is deleted as +well, and Tk will display an empty box wherever the image was used. + +See also +........ + +The Pillow(1) package adds support for formats such as BMP, JPEG, TIFF, +and WebP, among others. + + ---------- Footnotes ---------- + + (1) http://python-pillow.org/ + + +File: python.info, Node: File Handlers, Prev: Handy Reference, Up: tkinter — Python interface to Tcl/Tk + +5.25.1.18 File Handlers +....................... + +Tk allows you to register and unregister a callback function which will +be called from the Tk mainloop when I/O is possible on a file +descriptor. Only one handler may be registered per file descriptor. +Example code: + + import tkinter + widget = tkinter.Tk() + mask = tkinter.READABLE | tkinter.WRITABLE + widget.tk.createfilehandler(file, mask, callback) + ... + widget.tk.deletefilehandler(file) + +This feature is not available on Windows. + +Since you don’t know how many bytes are available for reading, you may +not want to use the *note BufferedIOBase: 588. or *note TextIOBase: c39. +*note read(): 1a22. or *note readline(): 13ae. methods, since these will +insist on reading a predefined number of bytes. For sockets, the *note +recv(): 677. or *note recvfrom(): 678. methods will work fine; for other +files, use raw reads or ‘os.read(file.fileno(), maxbytecount)’. + + -- Method: Widget.tk.createfilehandler (file, mask, func) + + Registers the file handler callback function `func'. The `file' + argument may either be an object with a *note fileno(): 1bdb. + method (such as a file or socket object), or an integer file + descriptor. The `mask' argument is an ORed combination of any of + the three constants below. The callback is called as follows: + + callback(file, mask) + + -- Method: Widget.tk.deletefilehandler (file) + + Unregisters a file handler. + + -- Data: tkinter.READABLE + -- Data: tkinter.WRITABLE + -- Data: tkinter.EXCEPTION + + Constants used in the `mask' arguments. + + +File: python.info, Node: tkinter ttk — Tk themed widgets, Next: tkinter tix — Extension widgets for Tk, Prev: tkinter — Python interface to Tcl/Tk, Up: Graphical User Interfaces with Tk + +5.25.2 ‘tkinter.ttk’ — Tk themed widgets +---------------------------------------- + +`Source code:' Lib/tkinter/ttk.py(1) + +__________________________________________________________________ + +The *note tkinter.ttk: 10f. module provides access to the Tk themed +widget set, introduced in Tk 8.5. If Python has not been compiled +against Tk 8.5, this module can still be accessed if `Tile' has been +installed. The former method using Tk 8.5 provides additional benefits +including anti-aliased font rendering under X11 and window transparency +(requiring a composition window manager on X11). + +The basic idea for *note tkinter.ttk: 10f. is to separate, to the extent +possible, the code implementing a widget’s behavior from the code +implementing its appearance. + +See also +........ + +Tk Widget Styling Support(2) + + A document introducing theming support for Tk + +* Menu: + +* Using Ttk:: +* Ttk Widgets:: +* Widget:: +* Combobox:: +* Spinbox:: +* Notebook:: +* Progressbar:: +* Separator:: +* Sizegrip:: +* Treeview:: +* Ttk Styling:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/tkinter/ttk.py + + (2) https://core.tcl.tk/tips/doc/trunk/tip/48.md + + +File: python.info, Node: Using Ttk, Next: Ttk Widgets, Up: tkinter ttk — Tk themed widgets + +5.25.2.1 Using Ttk +.................. + +To start using Ttk, import its module: + + from tkinter import ttk + +To override the basic Tk widgets, the import should follow the Tk +import: + + from tkinter import * + from tkinter.ttk import * + +That code causes several *note tkinter.ttk: 10f. widgets (‘Button’, +‘Checkbutton’, ‘Entry’, ‘Frame’, ‘Label’, ‘LabelFrame’, ‘Menubutton’, +‘PanedWindow’, ‘Radiobutton’, ‘Scale’ and ‘Scrollbar’) to automatically +replace the Tk widgets. + +This has the direct benefit of using the new widgets which gives a +better look and feel across platforms; however, the replacement widgets +are not completely compatible. The main difference is that widget +options such as “fg”, “bg” and others related to widget styling are no +longer present in Ttk widgets. Instead, use the ‘ttk.Style’ class for +improved styling effects. + +See also +........ + +Converting existing applications to use Tile widgets(1) + + A monograph (using Tcl terminology) about differences typically + encountered when moving applications to use the new widgets. + + ---------- Footnotes ---------- + + (1) http://tktable.sourceforge.net/tile/doc/converting.txt + + +File: python.info, Node: Ttk Widgets, Next: Widget, Prev: Using Ttk, Up: tkinter ttk — Tk themed widgets + +5.25.2.2 Ttk Widgets +.................... + +Ttk comes with 18 widgets, twelve of which already existed in tkinter: +‘Button’, ‘Checkbutton’, ‘Entry’, ‘Frame’, ‘Label’, ‘LabelFrame’, +‘Menubutton’, ‘PanedWindow’, ‘Radiobutton’, ‘Scale’, ‘Scrollbar’, and +*note Spinbox: 3fd. The other six are new: *note Combobox: 2ea3, *note +Notebook: 2ea4, *note Progressbar: 2ea5, ‘Separator’, ‘Sizegrip’ and +*note Treeview: 2bd. And all them are subclasses of *note Widget: 2ea6. + +Using the Ttk widgets gives the application an improved look and feel. +As discussed above, there are differences in how the styling is coded. + +Tk code: + + l1 = tkinter.Label(text="Test", fg="black", bg="white") + l2 = tkinter.Label(text="Test", fg="black", bg="white") + +Ttk code: + + style = ttk.Style() + style.configure("BW.TLabel", foreground="black", background="white") + + l1 = ttk.Label(text="Test", style="BW.TLabel") + l2 = ttk.Label(text="Test", style="BW.TLabel") + +For more information about *note TtkStyling: 2ea7, see the *note Style: +2ea8. class documentation. + + +File: python.info, Node: Widget, Next: Combobox, Prev: Ttk Widgets, Up: tkinter ttk — Tk themed widgets + +5.25.2.3 Widget +............... + +‘ttk.Widget’ defines standard options and methods supported by Tk themed +widgets and is not supposed to be directly instantiated. + +* Menu: + +* Standard Options:: +* Scrollable Widget Options:: +* Label Options:: +* Compatibility Options:: +* Widget States:: +* ttk.Widget: ttk Widget. + + +File: python.info, Node: Standard Options, Next: Scrollable Widget Options, Up: Widget + +5.25.2.4 Standard Options +......................... + +All the ‘ttk’ Widgets accepts the following options: + + Option Description + + ----------------------------------------------------------------------------------- + + class Specifies the window class. The class is used when querying the + option database for the window’s other options, to determine the + default bindtags for the window, and to select the widget’s + default layout and style. This option is read-only, and may + only be specified when the window is created. + + + cursor Specifies the mouse cursor to be used for the widget. If set to + the empty string (the default), the cursor is inherited for the + parent widget. + + + takefocus Determines whether the window accepts the focus during keyboard + traversal. 0, 1 or an empty string is returned. If 0 is + returned, it means that the window should be skipped entirely + during keyboard traversal. If 1, it means that the window + should receive the input focus as long as it is viewable. And + an empty string means that the traversal scripts make the + decision about whether or not to focus on the window. + + + style May be used to specify a custom widget style. + + + +File: python.info, Node: Scrollable Widget Options, Next: Label Options, Prev: Standard Options, Up: Widget + +5.25.2.5 Scrollable Widget Options +.................................. + +The following options are supported by widgets that are controlled by a +scrollbar. + + Option Description + + ----------------------------------------------------------------------------------- + + xscrollcommand Used to communicate with horizontal scrollbars. + + When the view in the widget’s window change, the widget + will generate a Tcl command based on the scrollcommand. + + Usually this option consists of the method + ‘Scrollbar.set()’ of some scrollbar. This will cause the + scrollbar to be updated whenever the view in the window + changes. + + + yscrollcommand Used to communicate with vertical scrollbars. For some + more information, see above. + + + +File: python.info, Node: Label Options, Next: Compatibility Options, Prev: Scrollable Widget Options, Up: Widget + +5.25.2.6 Label Options +...................... + +The following options are supported by labels, buttons and other +button-like widgets. + + Option Description + + ----------------------------------------------------------------------------------- + + text Specifies a text string to be displayed inside the widget. + + + textvariable Specifies a name whose value will be used in place of the + text option resource. + + + underline If set, specifies the index (0-based) of a character to + underline in the text string. The underline character is + used for mnemonic activation. + + + image Specifies an image to display. This is a list of 1 or more + elements. The first element is the default image name. The + rest of the list if a sequence of statespec/value pairs as + defined by *note Style.map(): 2ead, specifying different + images to use when the widget is in a particular state or a + combination of states. All images in the list should have + the same size. + + + compound Specifies how to display the image relative to the text, in + the case both text and images options are present. Valid + values are: + + * text: display text only + + * image: display image only + + * top, bottom, left, right: display image above, below, + left of, or right of the text, respectively. + + * none: the default. display the image if present, + otherwise the text. + + + width If greater than zero, specifies how much space, in character + widths, to allocate for the text label, if less than zero, + specifies a minimum width. If zero or unspecified, the + natural width of the text label is used. + + + +File: python.info, Node: Compatibility Options, Next: Widget States, Prev: Label Options, Up: Widget + +5.25.2.7 Compatibility Options +.............................. + + Option Description + + ---------------------------------------------------------------------------------- + + state May be set to “normal” or “disabled” to control the “disabled” + state bit. This is a write-only option: setting it changes the + widget state, but the *note Widget.state(): 2eaf. method does not + affect this option. + + + +File: python.info, Node: Widget States, Next: ttk Widget, Prev: Compatibility Options, Up: Widget + +5.25.2.8 Widget States +...................... + +The widget state is a bitmap of independent state flags. + + Flag Description + + ----------------------------------------------------------------------------------- + + active The mouse cursor is over the widget and pressing a mouse button + will cause some action to occur + + + disabled Widget is disabled under program control + + + focus Widget has keyboard focus + + + pressed Widget is being pressed + + + selected “On”, “true”, or “current” for things like Checkbuttons and + radiobuttons + + + background Windows and Mac have a notion of an “active” or foreground + window. The `background' state is set for widgets in a + background window, and cleared for those in the foreground + window + + + readonly Widget should not allow user modification + + + alternate A widget-specific alternate display format + + + invalid The widget’s value is invalid + + +A state specification is a sequence of state names, optionally prefixed +with an exclamation point indicating that the bit is off. + + +File: python.info, Node: ttk Widget, Prev: Widget States, Up: Widget + +5.25.2.9 ttk.Widget +................... + +Besides the methods described below, the ‘ttk.Widget’ supports the +methods ‘tkinter.Widget.cget()’ and ‘tkinter.Widget.configure()’. + + -- Class: tkinter.ttk.Widget + + -- Method: identify (x, y) + + Returns the name of the element at position `x' `y', or the + empty string if the point does not lie within any element. + + `x' and `y' are pixel coordinates relative to the widget. + + -- Method: instate (statespec, callback=None, *args, **kw) + + Test the widget’s state. If a callback is not specified, + returns ‘True’ if the widget state matches `statespec' and + ‘False’ otherwise. If callback is specified then it is called + with args if widget state matches `statespec'. + + -- Method: state (statespec=None) + + Modify or inquire widget state. If `statespec' is specified, + sets the widget state according to it and return a new + `statespec' indicating which flags were changed. If + `statespec' is not specified, returns the currently-enabled + state flags. + + `statespec' will usually be a list or a tuple. + + +File: python.info, Node: Combobox, Next: Spinbox, Prev: Widget, Up: tkinter ttk — Tk themed widgets + +5.25.2.10 Combobox +.................. + +The ‘ttk.Combobox’ widget combines a text field with a pop-down list of +values. This widget is a subclass of ‘Entry’. + +Besides the methods inherited from *note Widget: 2ea6.: ‘Widget.cget()’, +‘Widget.configure()’, *note Widget.identify(): 2eb2, *note +Widget.instate(): 2eb3. and *note Widget.state(): 2eaf, and the +following inherited from ‘Entry’: ‘Entry.bbox()’, ‘Entry.delete()’, +‘Entry.icursor()’, ‘Entry.index()’, ‘Entry.insert()’, +‘Entry.selection()’, ‘Entry.xview()’, it has some other methods, +described at ‘ttk.Combobox’. + +* Menu: + +* Options:: +* Virtual events:: +* ttk.Combobox: ttk Combobox. + + +File: python.info, Node: Options, Next: Virtual events, Up: Combobox + +5.25.2.11 Options +................. + +This widget accepts the following specific options: + + Option Description + + ----------------------------------------------------------------------------------- + + exportselection Boolean value. If set, the widget selection is linked to + the Window Manager selection (which can be returned by + invoking Misc.selection_get, for example). + + + justify Specifies how the text is aligned within the widget. One + of “left”, “center”, or “right”. + + + height Specifies the height of the pop-down listbox, in rows. + + + postcommand A script (possibly registered with Misc.register) that is + called immediately before displaying the values. It may + specify which values to display. + + + state One of “normal”, “readonly”, or “disabled”. In the + “readonly” state, the value may not be edited directly, + and the user can only selection of the values from the + dropdown list. In the “normal” state, the text field is + directly editable. In the “disabled” state, no + interaction is possible. + + + textvariable Specifies a name whose value is linked to the widget + value. Whenever the value associated with that name + changes, the widget value is updated, and vice versa. See + ‘tkinter.StringVar’. + + + values Specifies the list of values to display in the drop-down + listbox. + + + width Specifies an integer value indicating the desired width of + the entry window, in average-size characters of the + widget’s font. + + + +File: python.info, Node: Virtual events, Next: ttk Combobox, Prev: Options, Up: Combobox + +5.25.2.12 Virtual events +........................ + +The combobox widgets generates a `<<ComboboxSelected>>' virtual event +when the user selects an element from the list of values. + + +File: python.info, Node: ttk Combobox, Prev: Virtual events, Up: Combobox + +5.25.2.13 ttk.Combobox +...................... + + -- Class: tkinter.ttk.Combobox + + -- Method: current (newindex=None) + + If `newindex' is specified, sets the combobox value to the + element position `newindex'. Otherwise, returns the index of + the current value or -1 if the current value is not in the + values list. + + -- Method: get () + + Returns the current value of the combobox. + + -- Method: set (value) + + Sets the value of the combobox to `value'. + + +File: python.info, Node: Spinbox, Next: Notebook, Prev: Combobox, Up: tkinter ttk — Tk themed widgets + +5.25.2.14 Spinbox +................. + +The ‘ttk.Spinbox’ widget is a ‘ttk.Entry’ enhanced with increment and +decrement arrows. It can be used for numbers or lists of string values. +This widget is a subclass of ‘Entry’. + +Besides the methods inherited from *note Widget: 2ea6.: ‘Widget.cget()’, +‘Widget.configure()’, *note Widget.identify(): 2eb2, *note +Widget.instate(): 2eb3. and *note Widget.state(): 2eaf, and the +following inherited from ‘Entry’: ‘Entry.bbox()’, ‘Entry.delete()’, +‘Entry.icursor()’, ‘Entry.index()’, ‘Entry.insert()’, ‘Entry.xview()’, +it has some other methods, described at ‘ttk.Spinbox’. + +* Menu: + +* Options: Options<2>. +* Virtual events: Virtual events<2>. +* ttk.Spinbox: ttk Spinbox. + + +File: python.info, Node: Options<2>, Next: Virtual events<2>, Up: Spinbox + +5.25.2.15 Options +................. + +This widget accepts the following specific options: + +Option Description + +-------------------------------------------------------------------------------------- + +from Float value. If set, this is the minimum value to which + the decrement button will decrement. Must be spelled as + ‘from_’ when used as an argument, since ‘from’ is a + Python keyword. + + +to Float value. If set, this is the maximum value to which + the increment button will increment. + + +increment Float value. Specifies the amount which the + increment/decrement buttons change the value. Defaults + to 1.0. + + +values Sequence of string or float values. If specified, the + increment/decrement buttons will cycle through the items + in this sequence rather than incrementing or + decrementing numbers. + + +wrap Boolean value. If ‘True’, increment and decrement + buttons will cycle from the ‘to’ value to the ‘from’ + value or the ‘from’ value to the ‘to’ value, + respectively. + + +format String value. This specifies the format of numbers set + by the increment/decrement buttons. It must be in the + form “%W.Pf”, where W is the padded width of the value, + P is the precision, and ‘%’ and ‘f’ are literal. + + +command Python callable. Will be called with no arguments + whenever either of the increment or decrement buttons + are pressed. + + + +File: python.info, Node: Virtual events<2>, Next: ttk Spinbox, Prev: Options<2>, Up: Spinbox + +5.25.2.16 Virtual events +........................ + +The spinbox widget generates an `<<Increment>>' virtual event when the +user presses <Up>, and a `<<Decrement>>' virtual event when the user +presses <Down>. + + +File: python.info, Node: ttk Spinbox, Prev: Virtual events<2>, Up: Spinbox + +5.25.2.17 ttk.Spinbox +..................... + + -- Class: tkinter.ttk.Spinbox + + -- Method: get () + + Returns the current value of the spinbox. + + -- Method: set (value) + + Sets the value of the spinbox to `value'. + + +File: python.info, Node: Notebook, Next: Progressbar, Prev: Spinbox, Up: tkinter ttk — Tk themed widgets + +5.25.2.18 Notebook +.................. + +Ttk Notebook widget manages a collection of windows and displays a +single one at a time. Each child window is associated with a tab, which +the user may select to change the currently-displayed window. + +* Menu: + +* Options: Options<3>. +* Tab Options:: +* Tab Identifiers:: +* Virtual Events:: +* ttk.Notebook: ttk Notebook. + + +File: python.info, Node: Options<3>, Next: Tab Options, Up: Notebook + +5.25.2.19 Options +................. + +This widget accepts the following specific options: + + Option Description + + ----------------------------------------------------------------------------------- + + height If present and greater than zero, specifies the desired height of + the pane area (not including internal padding or tabs). + Otherwise, the maximum height of all panes is used. + + + padding Specifies the amount of extra space to add around the outside of + the notebook. The padding is a list up to four length + specifications left top right bottom. If fewer than four elements + are specified, bottom defaults to top, right defaults to left, and + top defaults to left. + + + width If present and greater than zero, specified the desired width of + the pane area (not including internal padding). Otherwise, the + maximum width of all panes is used. + + + +File: python.info, Node: Tab Options, Next: Tab Identifiers, Prev: Options<3>, Up: Notebook + +5.25.2.20 Tab Options +..................... + +There are also specific options for tabs: + + Option Description + + ----------------------------------------------------------------------------------- + + state Either “normal”, “disabled” or “hidden”. If “disabled”, then + the tab is not selectable. If “hidden”, then the tab is not + shown. + + + sticky Specifies how the child window is positioned within the pane + area. Value is a string containing zero or more of the + characters “n”, “s”, “e” or “w”. Each letter refers to a side + (north, south, east or west) that the child window will stick + to, as per the ‘grid()’ geometry manager. + + + padding Specifies the amount of extra space to add between the notebook + and this pane. Syntax is the same as for the option padding + used by this widget. + + + text Specifies a text to be displayed in the tab. + + + image Specifies an image to display in the tab. See the option image + described in *note Widget: 2ea6. + + + compound Specifies how to display the image relative to the text, in the + case both options text and image are present. See + *note Label Options: 2eac. for legal values. + + + underline Specifies the index (0-based) of a character to underline in the + text string. The underlined character is used for mnemonic + activation if *note Notebook.enable_traversal(): 2ec4. is + called. + + + +File: python.info, Node: Tab Identifiers, Next: Virtual Events, Prev: Tab Options, Up: Notebook + +5.25.2.21 Tab Identifiers +......................... + +The tab_id present in several methods of ‘ttk.Notebook’ may take any of +the following forms: + + * An integer between zero and the number of tabs + + * The name of a child window + + * A positional specification of the form “@x,y”, which identifies the + tab + + * The literal string “current”, which identifies the + currently-selected tab + + * The literal string “end”, which returns the number of tabs (only + valid for *note Notebook.index(): 2ec6.) + + +File: python.info, Node: Virtual Events, Next: ttk Notebook, Prev: Tab Identifiers, Up: Notebook + +5.25.2.22 Virtual Events +........................ + +This widget generates a `<<NotebookTabChanged>>' virtual event after a +new tab is selected. + + +File: python.info, Node: ttk Notebook, Prev: Virtual Events, Up: Notebook + +5.25.2.23 ttk.Notebook +...................... + + -- Class: tkinter.ttk.Notebook + + -- Method: add (child, **kw) + + Adds a new tab to the notebook. + + If window is currently managed by the notebook but hidden, it + is restored to its previous position. + + See *note Tab Options: 2ec3. for the list of available + options. + + -- Method: forget (tab_id) + + Removes the tab specified by `tab_id', unmaps and unmanages + the associated window. + + -- Method: hide (tab_id) + + Hides the tab specified by `tab_id'. + + The tab will not be displayed, but the associated window + remains managed by the notebook and its configuration + remembered. Hidden tabs may be restored with the *note add(): + 2ec9. command. + + -- Method: identify (x, y) + + Returns the name of the tab element at position `x', `y', or + the empty string if none. + + -- Method: index (tab_id) + + Returns the numeric index of the tab specified by `tab_id', or + the total number of tabs if `tab_id' is the string “end”. + + -- Method: insert (pos, child, **kw) + + Inserts a pane at the specified position. + + `pos' is either the string “end”, an integer index, or the + name of a managed child. If `child' is already managed by the + notebook, moves it to the specified position. + + See *note Tab Options: 2ec3. for the list of available + options. + + -- Method: select (tab_id=None) + + Selects the specified `tab_id'. + + The associated child window will be displayed, and the + previously-selected window (if different) is unmapped. If + `tab_id' is omitted, returns the widget name of the currently + selected pane. + + -- Method: tab (tab_id, option=None, **kw) + + Query or modify the options of the specific `tab_id'. + + If `kw' is not given, returns a dictionary of the tab option + values. If `option' is specified, returns the value of that + `option'. Otherwise, sets the options to the corresponding + values. + + -- Method: tabs () + + Returns a list of windows managed by the notebook. + + -- Method: enable_traversal () + + Enable keyboard traversal for a toplevel window containing + this notebook. + + This will extend the bindings for the toplevel window + containing the notebook as follows: + + * ‘Control-Tab’: selects the tab following the currently + selected one. + + * ‘Shift-Control-Tab’: selects the tab preceding the + currently selected one. + + * ‘Alt-K’: where `K' is the mnemonic (underlined) character + of any tab, will select that tab. + + Multiple notebooks in a single toplevel may be enabled for + traversal, including nested notebooks. However, notebook + traversal only works properly if all panes have the notebook + they are in as master. + + +File: python.info, Node: Progressbar, Next: Separator, Prev: Notebook, Up: tkinter ttk — Tk themed widgets + +5.25.2.24 Progressbar +..................... + +The ‘ttk.Progressbar’ widget shows the status of a long-running +operation. It can operate in two modes: 1) the determinate mode which +shows the amount completed relative to the total amount of work to be +done and 2) the indeterminate mode which provides an animated display to +let the user know that work is progressing. + +* Menu: + +* Options: Options<4>. +* ttk.Progressbar: ttk Progressbar. + + +File: python.info, Node: Options<4>, Next: ttk Progressbar, Up: Progressbar + +5.25.2.25 Options +................. + +This widget accepts the following specific options: + + Option Description + + ----------------------------------------------------------------------------------- + + orient One of “horizontal” or “vertical”. Specifies the orientation of + the progress bar. + + + length Specifies the length of the long axis of the progress bar (width + if horizontal, height if vertical). + + + mode One of “determinate” or “indeterminate”. + + + maximum A number specifying the maximum value. Defaults to 100. + + + value The current value of the progress bar. In “determinate” mode, + this represents the amount of work completed. In “indeterminate” + mode, it is interpreted as modulo `maximum'; that is, the + progress bar completes one “cycle” when its value increases by + `maximum'. + + + variable A name which is linked to the option value. If specified, the + value of the progress bar is automatically set to the value of + this name whenever the latter is modified. + + + phase Read-only option. The widget periodically increments the value + of this option whenever its value is greater than 0 and, in + determinate mode, less than maximum. This option may be used by + the current theme to provide additional animation effects. + + + +File: python.info, Node: ttk Progressbar, Prev: Options<4>, Up: Progressbar + +5.25.2.26 ttk.Progressbar +......................... + + -- Class: tkinter.ttk.Progressbar + + -- Method: start (interval=None) + + Begin autoincrement mode: schedules a recurring timer event + that calls *note Progressbar.step(): 2ed5. every `interval' + milliseconds. If omitted, `interval' defaults to 50 + milliseconds. + + -- Method: step (amount=None) + + Increments the progress bar’s value by `amount'. + + `amount' defaults to 1.0 if omitted. + + -- Method: stop () + + Stop autoincrement mode: cancels any recurring timer event + initiated by *note Progressbar.start(): 2ed4. for this + progress bar. + + +File: python.info, Node: Separator, Next: Sizegrip, Prev: Progressbar, Up: tkinter ttk — Tk themed widgets + +5.25.2.27 Separator +................... + +The ‘ttk.Separator’ widget displays a horizontal or vertical separator +bar. + +It has no other methods besides the ones inherited from ‘ttk.Widget’. + +* Menu: + +* Options: Options<5>. + + +File: python.info, Node: Options<5>, Up: Separator + +5.25.2.28 Options +................. + +This widget accepts the following specific option: + + Option Description + + ---------------------------------------------------------------------------------- + + orient One of “horizontal” or “vertical”. Specifies the orientation of + the separator. + + + +File: python.info, Node: Sizegrip, Next: Treeview, Prev: Separator, Up: tkinter ttk — Tk themed widgets + +5.25.2.29 Sizegrip +.................. + +The ‘ttk.Sizegrip’ widget (also known as a grow box) allows the user to +resize the containing toplevel window by pressing and dragging the grip. + +This widget has neither specific options nor specific methods, besides +the ones inherited from ‘ttk.Widget’. + +* Menu: + +* Platform-specific notes:: +* Bugs:: + + +File: python.info, Node: Platform-specific notes, Next: Bugs, Up: Sizegrip + +5.25.2.30 Platform-specific notes +................................. + + * On MacOS X, toplevel windows automatically include a built-in size + grip by default. Adding a ‘Sizegrip’ is harmless, since the + built-in grip will just mask the widget. + + +File: python.info, Node: Bugs, Prev: Platform-specific notes, Up: Sizegrip + +5.25.2.31 Bugs +.............. + + * If the containing toplevel’s position was specified relative to the + right or bottom of the screen (e.g. ….), the ‘Sizegrip’ widget + will not resize the window. + + * This widget supports only “southeast” resizing. + + +File: python.info, Node: Treeview, Next: Ttk Styling, Prev: Sizegrip, Up: tkinter ttk — Tk themed widgets + +5.25.2.32 Treeview +.................. + +The ‘ttk.Treeview’ widget displays a hierarchical collection of items. +Each item has a textual label, an optional image, and an optional list +of data values. The data values are displayed in successive columns +after the tree label. + +The order in which data values are displayed may be controlled by +setting the widget option ‘displaycolumns’. The tree widget can also +display column headings. Columns may be accessed by number or symbolic +names listed in the widget option columns. See *note Column +Identifiers: 2edd. + +Each item is identified by a unique name. The widget will generate item +IDs if they are not supplied by the caller. There is a distinguished +root item, named ‘{}’. The root item itself is not displayed; its +children appear at the top level of the hierarchy. + +Each item also has a list of tags, which can be used to associate event +bindings with individual items and control the appearance of the item. + +The Treeview widget supports horizontal and vertical scrolling, +according to the options described in *note Scrollable Widget Options: +2eab. and the methods *note Treeview.xview(): 2ede. and *note +Treeview.yview(): 2edf. + +* Menu: + +* Options: Options<6>. +* Item Options:: +* Tag Options:: +* Column Identifiers:: +* Virtual Events: Virtual Events<2>. +* ttk.Treeview: ttk Treeview. + + +File: python.info, Node: Options<6>, Next: Item Options, Up: Treeview + +5.25.2.33 Options +................. + +This widget accepts the following specific options: + + Option Description + + ---------------------------------------------------------------------------------- + + columns A list of column identifiers, specifying the number of + columns and their names. + + + displaycolumns A list of column identifiers (either symbolic or integer + indices) specifying which data columns are displayed and + the order in which they appear, or the string “#all”. + + + height Specifies the number of rows which should be visible. + Note: the requested width is determined from the sum of + the column widths. + + + padding Specifies the internal padding for the widget. The + padding is a list of up to four length specifications. + + + selectmode Controls how the built-in class bindings manage the + selection. One of “extended”, “browse” or “none”. If set + to “extended” (the default), multiple items may be + selected. If “browse”, only a single item will be + selected at a time. If “none”, the selection will not be + changed. + + Note that the application code and tag bindings can set + the selection however they wish, regardless of the value + of this option. + + + show A list containing zero or more of the following values, + specifying which elements of the tree to display. + + * tree: display tree labels in column #0. + + * headings: display the heading row. + + The default is “tree headings”, i.e., show all elements. + + `Note': Column #0 always refers to the tree column, even + if show=”tree” is not specified. + + + +File: python.info, Node: Item Options, Next: Tag Options, Prev: Options<6>, Up: Treeview + +5.25.2.34 Item Options +...................... + +The following item options may be specified for items in the insert and +item widget commands. + + Option Description + + --------------------------------------------------------------------------------- + + text The textual label to display for the item. + + + image A Tk Image, displayed to the left of the label. + + + values The list of values associated with the item. + + Each item should have the same number of values as the widget + option columns. If there are fewer values than columns, the + remaining values are assumed empty. If there are more values + than columns, the extra values are ignored. + + + open ‘True’/‘False’ value indicating whether the item’s children + should be displayed or hidden. + + + tags A list of tags associated with this item. + + + +File: python.info, Node: Tag Options, Next: Column Identifiers, Prev: Item Options, Up: Treeview + +5.25.2.35 Tag Options +..................... + +The following options may be specified on tags: + + Option Description + + --------------------------------------------------------------------------------- + + foreground Specifies the text foreground color. + + + background Specifies the cell or item background color. + + + font Specifies the font to use when drawing text. + + + image Specifies the item image, in case the item’s image option is + empty. + + + +File: python.info, Node: Column Identifiers, Next: Virtual Events<2>, Prev: Tag Options, Up: Treeview + +5.25.2.36 Column Identifiers +............................ + +Column identifiers take any of the following forms: + + * A symbolic name from the list of columns option. + + * An integer n, specifying the nth data column. + + * A string of the form #n, where n is an integer, specifying the nth + display column. + +Notes: + + * Item’s option values may be displayed in a different order than the + order in which they are stored. + + * Column #0 always refers to the tree column, even if show=”tree” is + not specified. + +A data column number is an index into an item’s option values list; a +display column number is the column number in the tree where the values +are displayed. Tree labels are displayed in column #0. If option +displaycolumns is not set, then data column n is displayed in column +#n+1. Again, `column #0 always refers to the tree column'. + + +File: python.info, Node: Virtual Events<2>, Next: ttk Treeview, Prev: Column Identifiers, Up: Treeview + +5.25.2.37 Virtual Events +........................ + +The Treeview widget generates the following virtual events. + + Event Description + + -------------------------------------------------------------------------------- + + <<TreeviewSelect>> Generated whenever the selection changes. + + + <<TreeviewOpen>> Generated just before settings the focus item to + open=True. + + + <<TreeviewClose>> Generated just after setting the focus item to + open=False. + + +The *note Treeview.focus(): 2ee4. and *note Treeview.selection(): 2bc. +methods can be used to determine the affected item or items. + + +File: python.info, Node: ttk Treeview, Prev: Virtual Events<2>, Up: Treeview + +5.25.2.38 ttk.Treeview +...................... + + -- Class: tkinter.ttk.Treeview + + -- Method: bbox (item, column=None) + + Returns the bounding box (relative to the treeview widget’s + window) of the specified `item' in the form (x, y, width, + height). + + If `column' is specified, returns the bounding box of that + cell. If the `item' is not visible (i.e., if it is a + descendant of a closed item or is scrolled offscreen), returns + an empty string. + + -- Method: get_children (item=None) + + Returns the list of children belonging to `item'. + + If `item' is not specified, returns root children. + + -- Method: set_children (item, *newchildren) + + Replaces `item'’s child with `newchildren'. + + Children present in `item' that are not present in + `newchildren' are detached from the tree. No items in + `newchildren' may be an ancestor of `item'. Note that not + specifying `newchildren' results in detaching `item'’s + children. + + -- Method: column (column, option=None, **kw) + + Query or modify the options for the specified `column'. + + If `kw' is not given, returns a dict of the column option + values. If `option' is specified then the value for that + `option' is returned. Otherwise, sets the options to the + corresponding values. + + The valid options/values are: + + * + id + + Returns the column name. This is a read-only + option. + + * + anchor: One of the standard Tk anchor values. + + Specifies how the text in this column should be + aligned with respect to the cell. + + * + minwidth: width + + The minimum width of the column in pixels. The + treeview widget will not make the column any smaller + than specified by this option when the widget is + resized or the user drags a column. + + * + stretch: ‘True’/‘False’ + + Specifies whether the column’s width should be + adjusted when the widget is resized. + + * + width: width + + The width of the column in pixels. + + To configure the tree column, call this with column = “#0” + + -- Method: delete (*items) + + Delete all specified `items' and all their descendants. + + The root item may not be deleted. + + -- Method: detach (*items) + + Unlinks all of the specified `items' from the tree. + + The items and all of their descendants are still present, and + may be reinserted at another point in the tree, but will not + be displayed. + + The root item may not be detached. + + -- Method: exists (item) + + Returns ‘True’ if the specified `item' is present in the tree. + + -- Method: focus (item=None) + + If `item' is specified, sets the focus item to `item'. + Otherwise, returns the current focus item, or ‘’ if there is + none. + + -- Method: heading (column, option=None, **kw) + + Query or modify the heading options for the specified + `column'. + + If `kw' is not given, returns a dict of the heading option + values. If `option' is specified then the value for that + `option' is returned. Otherwise, sets the options to the + corresponding values. + + The valid options/values are: + + * + text: text + + The text to display in the column heading. + + * + image: imageName + + Specifies an image to display to the right of the + column heading. + + * + anchor: anchor + + Specifies how the heading text should be aligned. + One of the standard Tk anchor values. + + * + command: callback + + A callback to be invoked when the heading label is + pressed. + + To configure the tree column heading, call this with column = + “#0”. + + -- Method: identify (component, x, y) + + Returns a description of the specified `component' under the + point given by `x' and `y', or the empty string if no such + `component' is present at that position. + + -- Method: identify_row (y) + + Returns the item ID of the item at position `y'. + + -- Method: identify_column (x) + + Returns the data column identifier of the cell at position + `x'. + + The tree column has ID #0. + + -- Method: identify_region (x, y) + + Returns one of: + + region meaning + + ----------------------------------------------------------- + + heading Tree heading area. + + + separator Space between two columns headings. + + + tree The tree area. + + + cell A data cell. + + + Availability: Tk 8.6. + + -- Method: identify_element (x, y) + + Returns the element at position `x', `y'. + + Availability: Tk 8.6. + + -- Method: index (item) + + Returns the integer index of `item' within its parent’s list + of children. + + -- Method: insert (parent, index, iid=None, **kw) + + Creates a new item and returns the item identifier of the + newly created item. + + `parent' is the item ID of the parent item, or the empty + string to create a new top-level item. `index' is an integer, + or the value “end”, specifying where in the list of parent’s + children to insert the new item. If `index' is less than or + equal to zero, the new node is inserted at the beginning; if + `index' is greater than or equal to the current number of + children, it is inserted at the end. If `iid' is specified, + it is used as the item identifier; `iid' must not already + exist in the tree. Otherwise, a new unique identifier is + generated. + + See *note Item Options: 2ee1. for the list of available + points. + + -- Method: item (item, option=None, **kw) + + Query or modify the options for the specified `item'. + + If no options are given, a dict with options/values for the + item is returned. If `option' is specified then the value for + that option is returned. Otherwise, sets the options to the + corresponding values as given by `kw'. + + -- Method: move (item, parent, index) + + Moves `item' to position `index' in `parent'’s list of + children. + + It is illegal to move an item under one of its descendants. + If `index' is less than or equal to zero, `item' is moved to + the beginning; if greater than or equal to the number of + children, it is moved to the end. If `item' was detached it + is reattached. + + -- Method: next (item) + + Returns the identifier of `item'’s next sibling, or ‘’ if + `item' is the last child of its parent. + + -- Method: parent (item) + + Returns the ID of the parent of `item', or ‘’ if `item' is at + the top level of the hierarchy. + + -- Method: prev (item) + + Returns the identifier of `item'’s previous sibling, or ‘’ if + `item' is the first child of its parent. + + -- Method: reattach (item, parent, index) + + An alias for *note Treeview.move(): 2ef6. + + -- Method: see (item) + + Ensure that `item' is visible. + + Sets all of `item'’s ancestors open option to ‘True’, and + scrolls the widget if necessary so that `item' is within the + visible portion of the tree. + + -- Method: selection () + + Returns a tuple of selected items. + + Changed in version 3.8: ‘selection()’ no longer takes + arguments. For changing the selection state use the following + selection methods. + + -- Method: selection_set (*items) + + `items' becomes the new selection. + + Changed in version 3.6: `items' can be passed as separate + arguments, not just as a single tuple. + + -- Method: selection_add (*items) + + Add `items' to the selection. + + Changed in version 3.6: `items' can be passed as separate + arguments, not just as a single tuple. + + -- Method: selection_remove (*items) + + Remove `items' from the selection. + + Changed in version 3.6: `items' can be passed as separate + arguments, not just as a single tuple. + + -- Method: selection_toggle (*items) + + Toggle the selection state of each item in `items'. + + Changed in version 3.6: `items' can be passed as separate + arguments, not just as a single tuple. + + -- Method: set (item, column=None, value=None) + + With one argument, returns a dictionary of column/value pairs + for the specified `item'. With two arguments, returns the + current value of the specified `column'. With three + arguments, sets the value of given `column' in given `item' to + the specified `value'. + + -- Method: tag_bind (tagname, sequence=None, callback=None) + + Bind a callback for the given event `sequence' to the tag + `tagname'. When an event is delivered to an item, the + callbacks for each of the item’s tags option are called. + + -- Method: tag_configure (tagname, option=None, **kw) + + Query or modify the options for the specified `tagname'. + + If `kw' is not given, returns a dict of the option settings + for `tagname'. If `option' is specified, returns the value + for that `option' for the specified `tagname'. Otherwise, + sets the options to the corresponding values for the given + `tagname'. + + -- Method: tag_has (tagname, item=None) + + If `item' is specified, returns 1 or 0 depending on whether + the specified `item' has the given `tagname'. Otherwise, + returns a list of all items that have the specified tag. + + Availability: Tk 8.6 + + -- Method: xview (*args) + + Query or modify horizontal position of the treeview. + + -- Method: yview (*args) + + Query or modify vertical position of the treeview. + + +File: python.info, Node: Ttk Styling, Prev: Treeview, Up: tkinter ttk — Tk themed widgets + +5.25.2.39 Ttk Styling +..................... + +Each widget in ‘ttk’ is assigned a style, which specifies the set of +elements making up the widget and how they are arranged, along with +dynamic and default settings for element options. By default the style +name is the same as the widget’s class name, but it may be overridden by +the widget’s style option. If you don’t know the class name of a +widget, use the method ‘Misc.winfo_class()’ (somewidget.winfo_class()). + +See also +........ + +Tcl’2004 conference presentation(1) + + This document explains how the theme engine works + + -- Class: tkinter.ttk.Style + + This class is used to manipulate the style database. + + -- Method: configure (style, query_opt=None, **kw) + + Query or set the default value of the specified option(s) in + `style'. + + Each key in `kw' is an option and each value is a string + identifying the value for that option. + + For example, to change every default button to be a flat + button with some padding and a different background color: + + from tkinter import ttk + import tkinter + + root = tkinter.Tk() + + ttk.Style().configure("TButton", padding=6, relief="flat", + background="#ccc") + + btn = ttk.Button(text="Sample") + btn.pack() + + root.mainloop() + + -- Method: map (style, query_opt=None, **kw) + + Query or sets dynamic values of the specified option(s) in + `style'. + + Each key in `kw' is an option and each value should be a list + or a tuple (usually) containing statespecs grouped in tuples, + lists, or some other preference. A statespec is a compound of + one or more states and then a value. + + An example may make it more understandable: + + import tkinter + from tkinter import ttk + + root = tkinter.Tk() + + style = ttk.Style() + style.map("C.TButton", + foreground=[('pressed', 'red'), ('active', 'blue')], + background=[('pressed', '!disabled', 'black'), ('active', 'white')] + ) + + colored_btn = ttk.Button(text="Test", style="C.TButton").pack() + + root.mainloop() + + Note that the order of the (states, value) sequences for an + option does matter, if the order is changed to ‘[('active', + 'blue'), ('pressed', 'red')]’ in the foreground option, for + example, the result would be a blue foreground when the widget + were in active or pressed states. + + -- Method: lookup (style, option, state=None, default=None) + + Returns the value specified for `option' in `style'. + + If `state' is specified, it is expected to be a sequence of + one or more states. If the `default' argument is set, it is + used as a fallback value in case no specification for option + is found. + + To check what font a Button uses by default: + + from tkinter import ttk + + print(ttk.Style().lookup("TButton", "font")) + + -- Method: layout (style, layoutspec=None) + + Define the widget layout for given `style'. If `layoutspec' + is omitted, return the layout specification for given style. + + `layoutspec', if specified, is expected to be a list or some + other sequence type (excluding strings), where each item + should be a tuple and the first item is the layout name and + the second item should have the format described in *note + Layouts: 2f07. + + To understand the format, see the following example (it is not + intended to do anything useful): + + from tkinter import ttk + import tkinter + + root = tkinter.Tk() + + style = ttk.Style() + style.layout("TMenubutton", [ + ("Menubutton.background", None), + ("Menubutton.button", {"children": + [("Menubutton.focus", {"children": + [("Menubutton.padding", {"children": + [("Menubutton.label", {"side": "left", "expand": 1})] + })] + })] + }), + ]) + + mbtn = ttk.Menubutton(text='Text') + mbtn.pack() + root.mainloop() + + -- Method: element_create (elementname, etype, *args, **kw) + + Create a new element in the current theme, of the given + `etype' which is expected to be either “image”, “from” or + “vsapi”. The latter is only available in Tk 8.6a for Windows + XP and Vista and is not described here. + + If “image” is used, `args' should contain the default image + name followed by statespec/value pairs (this is the + imagespec), and `kw' may have the following options: + + * + border=padding + + padding is a list of up to four integers, + specifying the left, top, right, and bottom + borders, respectively. + + * + height=height + + Specifies a minimum height for the element. If + less than zero, the base image’s height is used + as a default. + + * + padding=padding + + Specifies the element’s interior padding. + Defaults to border’s value if not specified. + + * + sticky=spec + + Specifies how the image is placed within the + final parcel. spec contains zero or more + characters “n”, “s”, “w”, or “e”. + + * + width=width + + Specifies a minimum width for the element. If + less than zero, the base image’s width is used + as a default. + + If “from” is used as the value of `etype', *note + element_create(): 2f08. will clone an existing element. + `args' is expected to contain a themename, from which the + element will be cloned, and optionally an element to clone + from. If this element to clone from is not specified, an + empty element will be used. `kw' is discarded. + + -- Method: element_names () + + Returns the list of elements defined in the current theme. + + -- Method: element_options (elementname) + + Returns the list of `elementname'’s options. + + -- Method: theme_create (themename, parent=None, settings=None) + + Create a new theme. + + It is an error if `themename' already exists. If `parent' is + specified, the new theme will inherit styles, elements and + layouts from the parent theme. If `settings' are present they + are expected to have the same syntax used for *note + theme_settings(): 2f0c. + + -- Method: theme_settings (themename, settings) + + Temporarily sets the current theme to `themename', apply + specified `settings' and then restore the previous theme. + + Each key in `settings' is a style and each value may contain + the keys ‘configure’, ‘map’, ‘layout’ and ‘element create’ and + they are expected to have the same format as specified by the + methods *note Style.configure(): 2f04, *note Style.map(): + 2ead, *note Style.layout(): 2f06. and *note + Style.element_create(): 2f08. respectively. + + As an example, let’s change the Combobox for the default theme + a bit: + + from tkinter import ttk + import tkinter + + root = tkinter.Tk() + + style = ttk.Style() + style.theme_settings("default", { + "TCombobox": { + "configure": {"padding": 5}, + "map": { + "background": [("active", "green2"), + ("!disabled", "green4")], + "fieldbackground": [("!disabled", "green3")], + "foreground": [("focus", "OliveDrab1"), + ("!disabled", "OliveDrab2")] + } + } + }) + + combo = ttk.Combobox().pack() + + root.mainloop() + + -- Method: theme_names () + + Returns a list of all known themes. + + -- Method: theme_use (themename=None) + + If `themename' is not given, returns the theme in use. + Otherwise, sets the current theme to `themename', refreshes + all widgets and emits a <<ThemeChanged>> event. + +* Menu: + +* Layouts:: + + ---------- Footnotes ---------- + + (1) http://tktable.sourceforge.net/tile/tile-tcl2004.pdf + + +File: python.info, Node: Layouts, Up: Ttk Styling + +5.25.2.40 Layouts +................. + +A layout can be just ‘None’, if it takes no options, or a dict of +options specifying how to arrange the element. The layout mechanism +uses a simplified version of the pack geometry manager: given an initial +cavity, each element is allocated a parcel. Valid options/values are: + + * + side: whichside + + Specifies which side of the cavity to place the element; + one of top, right, bottom or left. If omitted, the + element occupies the entire cavity. + + * + sticky: nswe + + Specifies where the element is placed inside its + allocated parcel. + + * + unit: 0 or 1 + + If set to 1, causes the element and all of its + descendants to be treated as a single element for the + purposes of *note Widget.identify(): 2eb2. et al. It’s + used for things like scrollbar thumbs with grips. + + * + children: [sublayout… ] + + Specifies a list of elements to place inside the element. + Each element is a tuple (or other sequence type) where + the first item is the layout name, and the other is a + *note Layout: 2f07. + + +File: python.info, Node: tkinter tix — Extension widgets for Tk, Next: tkinter scrolledtext — Scrolled Text Widget, Prev: tkinter ttk — Tk themed widgets, Up: Graphical User Interfaces with Tk + +5.25.3 ‘tkinter.tix’ — Extension widgets for Tk +----------------------------------------------- + +`Source code:' Lib/tkinter/tix.py(1) + +Deprecated since version 3.6: This Tk extension is unmaintained and +should not be used in new code. Use *note tkinter.ttk: 10f. instead. + +__________________________________________________________________ + +The *note tkinter.tix: 10e. (Tk Interface Extension) module provides an +additional rich set of widgets. Although the standard Tk library has +many useful widgets, they are far from complete. The *note tkinter.tix: +10e. library provides most of the commonly needed widgets that are +missing from standard Tk: *note HList: 2f13, *note ComboBox: 2f14, *note +Control: 2f15. (a.k.a. SpinBox) and an assortment of scrollable +widgets. *note tkinter.tix: 10e. also includes many more widgets that +are generally useful in a wide range of applications: *note NoteBook: +2f16, *note FileEntry: 2f17, *note PanedWindow: 2f18, etc; there are +more than 40 of them. + +With all these new widgets, you can introduce new interaction techniques +into applications, creating more useful and more intuitive user +interfaces. You can design your application by choosing the most +appropriate widgets to match the special needs of your application and +users. + +See also +........ + +Tix Homepage(2) + + The home page for ‘Tix’. This includes links to additional + documentation and downloads. + +Tix Man Pages(3) + + On-line version of the man pages and reference material. + +Tix Programming Guide(4) + + On-line version of the programmer’s reference material. + +Tix Development Applications(5) + + Tix applications for development of Tix and Tkinter programs. Tide + applications work under Tk or Tkinter, and include ‘TixInspect’, an + inspector to remotely modify and debug Tix/Tk/Tkinter applications. + +* Menu: + +* Using Tix:: +* Tix Widgets:: +* Tix Commands:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/tkinter/tix.py + + (2) http://tix.sourceforge.net/ + + (3) http://tix.sourceforge.net/dist/current/man/ + + (4) +http://tix.sourceforge.net/dist/current/docs/tix-book/tix.book.html + + (5) http://tix.sourceforge.net/Tixapps/src/Tide.html + + +File: python.info, Node: Using Tix, Next: Tix Widgets, Up: tkinter tix — Extension widgets for Tk + +5.25.3.1 Using Tix +.................. + + -- Class: tkinter.tix.Tk (screenName=None, baseName=None, + className='Tix') + + Toplevel widget of Tix which represents mostly the main window of + an application. It has an associated Tcl interpreter. + + Classes in the *note tkinter.tix: 10e. module subclasses the + classes in the *note tkinter: 10c. The former imports the latter, + so to use *note tkinter.tix: 10e. with Tkinter, all you need to do + is to import one module. In general, you can just import *note + tkinter.tix: 10e, and replace the toplevel call to *note + tkinter.Tk: 2e84. with ‘tix.Tk’: + + from tkinter import tix + from tkinter.constants import * + root = tix.Tk() + +To use *note tkinter.tix: 10e, you must have the Tix widgets installed, +usually alongside your installation of the Tk widgets. To test your +installation, try the following: + + from tkinter import tix + root = tix.Tk() + root.tk.eval('package require Tix') + + +File: python.info, Node: Tix Widgets, Next: Tix Commands, Prev: Using Tix, Up: tkinter tix — Extension widgets for Tk + +5.25.3.2 Tix Widgets +.................... + +Tix(1) introduces over 40 widget classes to the *note tkinter: 10c. +repertoire. + +* Menu: + +* Basic Widgets:: +* File Selectors:: +* Hierarchical ListBox:: +* Tabular ListBox:: +* Manager Widgets:: +* Image Types:: +* Miscellaneous Widgets:: +* Form Geometry Manager:: + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/TixIntro.htm + + +File: python.info, Node: Basic Widgets, Next: File Selectors, Up: Tix Widgets + +5.25.3.3 Basic Widgets +...................... + + -- Class: tkinter.tix.Balloon + + A Balloon(1) that pops up over a widget to provide help. When the + user moves the cursor inside a widget to which a Balloon widget has + been bound, a small pop-up window with a descriptive message will + be shown on the screen. + + -- Class: tkinter.tix.ButtonBox + + The ButtonBox(2) widget creates a box of buttons, such as is + commonly used for ‘Ok Cancel’. + + -- Class: tkinter.tix.ComboBox + + The ComboBox(3) widget is similar to the combo box control in MS + Windows. The user can select a choice by either typing in the + entry subwidget or selecting from the listbox subwidget. + + -- Class: tkinter.tix.Control + + The Control(4) widget is also known as the ‘SpinBox’ widget. The + user can adjust the value by pressing the two arrow buttons or by + entering the value directly into the entry. The new value will be + checked against the user-defined upper and lower limits. + + -- Class: tkinter.tix.LabelEntry + + The LabelEntry(5) widget packages an entry widget and a label into + one mega widget. It can be used to simplify the creation of + “entry-form” type of interface. + + -- Class: tkinter.tix.LabelFrame + + The LabelFrame(6) widget packages a frame widget and a label into + one mega widget. To create widgets inside a LabelFrame widget, one + creates the new widgets relative to the ‘frame’ subwidget and + manage them inside the ‘frame’ subwidget. + + -- Class: tkinter.tix.Meter + + The Meter(7) widget can be used to show the progress of a + background job which may take a long time to execute. + + -- Class: tkinter.tix.OptionMenu + + The OptionMenu(8) creates a menu button of options. + + -- Class: tkinter.tix.PopupMenu + + The PopupMenu(9) widget can be used as a replacement of the + ‘tk_popup’ command. The advantage of the ‘Tix’ *note PopupMenu: + 2f23. widget is it requires less application code to manipulate. + + -- Class: tkinter.tix.Select + + The Select(10) widget is a container of button subwidgets. It can + be used to provide radio-box or check-box style of selection + options for the user. + + -- Class: tkinter.tix.StdButtonBox + + The StdButtonBox(11) widget is a group of standard buttons for + Motif-like dialog boxes. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixBalloon.htm + + (2) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixButtonBox.htm + + (3) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixComboBox.htm + + (4) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixControl.htm + + (5) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixLabelEntry.htm + + (6) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixLabelFrame.htm + + (7) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixMeter.htm + + (8) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixOptionMenu.htm + + (9) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixPopupMenu.htm + + (10) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixSelect.htm + + (11) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixStdButtonBox.htm + + +File: python.info, Node: File Selectors, Next: Hierarchical ListBox, Prev: Basic Widgets, Up: Tix Widgets + +5.25.3.4 File Selectors +....................... + + -- Class: tkinter.tix.DirList + + The DirList(1) widget displays a list view of a directory, its + previous directories and its sub-directories. The user can choose + one of the directories displayed in the list or change to another + directory. + + -- Class: tkinter.tix.DirTree + + The DirTree(2) widget displays a tree view of a directory, its + previous directories and its sub-directories. The user can choose + one of the directories displayed in the list or change to another + directory. + + -- Class: tkinter.tix.DirSelectDialog + + The DirSelectDialog(3) widget presents the directories in the file + system in a dialog window. The user can use this dialog window to + navigate through the file system to select the desired directory. + + -- Class: tkinter.tix.DirSelectBox + + The *note DirSelectBox: 2f2a. is similar to the standard Motif(TM) + directory-selection box. It is generally used for the user to + choose a directory. DirSelectBox stores the directories mostly + recently selected into a ComboBox widget so that they can be + quickly selected again. + + -- Class: tkinter.tix.ExFileSelectBox + + The ExFileSelectBox(4) widget is usually embedded in a + tixExFileSelectDialog widget. It provides a convenient method for + the user to select files. The style of the *note ExFileSelectBox: + 2f2b. widget is very similar to the standard file dialog on MS + Windows 3.1. + + -- Class: tkinter.tix.FileSelectBox + + The FileSelectBox(5) is similar to the standard Motif(TM) + file-selection box. It is generally used for the user to choose a + file. FileSelectBox stores the files mostly recently selected into + a *note ComboBox: 2f14. widget so that they can be quickly selected + again. + + -- Class: tkinter.tix.FileEntry + + The FileEntry(6) widget can be used to input a filename. The user + can type in the filename manually. Alternatively, the user can + press the button widget that sits next to the entry, which will + bring up a file selection dialog. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixDirList.htm + + (2) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixDirTree.htm + + (3) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixDirSelectDialog.htm + + (4) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixExFileSelectBox.htm + + (5) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixFileSelectBox.htm + + (6) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixFileEntry.htm + + +File: python.info, Node: Hierarchical ListBox, Next: Tabular ListBox, Prev: File Selectors, Up: Tix Widgets + +5.25.3.5 Hierarchical ListBox +............................. + + -- Class: tkinter.tix.HList + + The HList(1) widget can be used to display any data that have a + hierarchical structure, for example, file system directory trees. + The list entries are indented and connected by branch lines + according to their places in the hierarchy. + + -- Class: tkinter.tix.CheckList + + The CheckList(2) widget displays a list of items to be selected by + the user. CheckList acts similarly to the Tk checkbutton or + radiobutton widgets, except it is capable of handling many more + items than checkbuttons or radiobuttons. + + -- Class: tkinter.tix.Tree + + The Tree(3) widget can be used to display hierarchical data in a + tree form. The user can adjust the view of the tree by opening or + closing parts of the tree. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixHList.htm + + (2) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixCheckList.htm + + (3) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixTree.htm + + +File: python.info, Node: Tabular ListBox, Next: Manager Widgets, Prev: Hierarchical ListBox, Up: Tix Widgets + +5.25.3.6 Tabular ListBox +........................ + + -- Class: tkinter.tix.TList + + The TList(1) widget can be used to display data in a tabular + format. The list entries of a *note TList: 2f31. widget are + similar to the entries in the Tk listbox widget. The main + differences are (1) the *note TList: 2f31. widget can display the + list entries in a two dimensional format and (2) you can use + graphical images as well as multiple colors and fonts for the list + entries. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixTList.htm + + +File: python.info, Node: Manager Widgets, Next: Image Types, Prev: Tabular ListBox, Up: Tix Widgets + +5.25.3.7 Manager Widgets +........................ + + -- Class: tkinter.tix.PanedWindow + + The PanedWindow(1) widget allows the user to interactively + manipulate the sizes of several panes. The panes can be arranged + either vertically or horizontally. The user changes the sizes of + the panes by dragging the resize handle between two panes. + + -- Class: tkinter.tix.ListNoteBook + + The ListNoteBook(2) widget is very similar to the ‘TixNoteBook’ + widget: it can be used to display many windows in a limited space + using a notebook metaphor. The notebook is divided into a stack of + pages (windows). At one time only one of these pages can be shown. + The user can navigate through these pages by choosing the name of + the desired page in the ‘hlist’ subwidget. + + -- Class: tkinter.tix.NoteBook + + The NoteBook(3) widget can be used to display many windows in a + limited space using a notebook metaphor. The notebook is divided + into a stack of pages. At one time only one of these pages can be + shown. The user can navigate through these pages by choosing the + visual “tabs” at the top of the NoteBook widget. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixPanedWindow.htm + + (2) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixListNoteBook.htm + + (3) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixNoteBook.htm + + +File: python.info, Node: Image Types, Next: Miscellaneous Widgets, Prev: Manager Widgets, Up: Tix Widgets + +5.25.3.8 Image Types +.................... + +The *note tkinter.tix: 10e. module adds: + + * pixmap(1) capabilities to all *note tkinter.tix: 10e. and *note + tkinter: 10c. widgets to create color images from XPM files. + + * Compound(2) image types can be used to create images that consists + of multiple horizontal lines; each line is composed of a series of + items (texts, bitmaps, images or spaces) arranged from left to + right. For example, a compound image can be used to display a + bitmap and a text string simultaneously in a Tk ‘Button’ widget. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/pixmap.htm + + (2) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/compound.htm + + +File: python.info, Node: Miscellaneous Widgets, Next: Form Geometry Manager, Prev: Image Types, Up: Tix Widgets + +5.25.3.9 Miscellaneous Widgets +.............................. + + -- Class: tkinter.tix.InputOnly + + The InputOnly(1) widgets are to accept inputs from the user, which + can be done with the ‘bind’ command (Unix only). + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixInputOnly.htm + + +File: python.info, Node: Form Geometry Manager, Prev: Miscellaneous Widgets, Up: Tix Widgets + +5.25.3.10 Form Geometry Manager +............................... + +In addition, *note tkinter.tix: 10e. augments *note tkinter: 10c. by +providing: + + -- Class: tkinter.tix.Form + + The Form(1) geometry manager based on attachment rules for all Tk + widgets. + + ---------- Footnotes ---------- + + (1) +http://tix.sourceforge.net/dist/current/man/html/TixCmd/tixForm.htm + + +File: python.info, Node: Tix Commands, Prev: Tix Widgets, Up: tkinter tix — Extension widgets for Tk + +5.25.3.11 Tix Commands +...................... + + -- Class: tkinter.tix.tixCommand + + The tix commands(1) provide access to miscellaneous elements of + ‘Tix’’s internal state and the ‘Tix’ application context. Most of + the information manipulated by these methods pertains to the + application as a whole, or to a screen or display, rather than to a + particular window. + + To view the current settings, the common usage is: + + from tkinter import tix + root = tix.Tk() + print(root.tix_configure()) + + -- Method: tixCommand.tix_configure (cnf=None, **kw) + + Query or modify the configuration options of the Tix application + context. If no option is specified, returns a dictionary all of + the available options. If option is specified with no value, then + the method returns a list describing the one named option (this + list will be identical to the corresponding sublist of the value + returned if no option is specified). If one or more option-value + pairs are specified, then the method modifies the given option(s) + to have the given value(s); in this case the method returns an + empty string. Option may be any of the configuration options. + + -- Method: tixCommand.tix_cget (option) + + Returns the current value of the configuration option given by + `option'. Option may be any of the configuration options. + + -- Method: tixCommand.tix_getbitmap (name) + + Locates a bitmap file of the name ‘name.xpm’ or ‘name’ in one of + the bitmap directories (see the *note tix_addbitmapdir(): 2f3e. + method). By using *note tix_getbitmap(): 2f3d, you can avoid hard + coding the pathnames of the bitmap files in your application. When + successful, it returns the complete pathname of the bitmap file, + prefixed with the character ‘@’. The returned value can be used to + configure the ‘bitmap’ option of the Tk and Tix widgets. + + -- Method: tixCommand.tix_addbitmapdir (directory) + + Tix maintains a list of directories under which the *note + tix_getimage(): 2f3f. and *note tix_getbitmap(): 2f3d. methods will + search for image files. The standard bitmap directory is + ‘$TIX_LIBRARY/bitmaps’. The *note tix_addbitmapdir(): 2f3e. method + adds `directory' into this list. By using this method, the image + files of an applications can also be located using the *note + tix_getimage(): 2f3f. or *note tix_getbitmap(): 2f3d. method. + + -- Method: tixCommand.tix_filedialog ([dlgclass]) + + Returns the file selection dialog that may be shared among + different calls from this application. This method will create a + file selection dialog widget when it is called the first time. + This dialog will be returned by all subsequent calls to *note + tix_filedialog(): 2f40. An optional dlgclass parameter can be + passed as a string to specified what type of file selection dialog + widget is desired. Possible options are ‘tix’, ‘FileSelectDialog’ + or ‘tixExFileSelectDialog’. + + -- Method: tixCommand.tix_getimage (self, name) + + Locates an image file of the name ‘name.xpm’, ‘name.xbm’ or + ‘name.ppm’ in one of the bitmap directories (see the *note + tix_addbitmapdir(): 2f3e. method above). If more than one file + with the same name (but different extensions) exist, then the image + type is chosen according to the depth of the X display: xbm images + are chosen on monochrome displays and color images are chosen on + color displays. By using *note tix_getimage(): 2f3f, you can avoid + hard coding the pathnames of the image files in your application. + When successful, this method returns the name of the newly created + image, which can be used to configure the ‘image’ option of the Tk + and Tix widgets. + + -- Method: tixCommand.tix_option_get (name) + + Gets the options maintained by the Tix scheme mechanism. + + -- Method: tixCommand.tix_resetoptions (newScheme, newFontSet[, + newScmPrio]) + + Resets the scheme and fontset of the Tix application to `newScheme' + and `newFontSet', respectively. This affects only those widgets + created after this call. Therefore, it is best to call the + resetoptions method before the creation of any widgets in a Tix + application. + + The optional parameter `newScmPrio' can be given to reset the + priority level of the Tk options set by the Tix schemes. + + Because of the way Tk handles the X option database, after Tix has + been has imported and inited, it is not possible to reset the color + schemes and font sets using the ‘tix_config()’ method. Instead, + the *note tix_resetoptions(): 2f42. method must be used. + + ---------- Footnotes ---------- + + (1) http://tix.sourceforge.net/dist/current/man/html/TixCmd/tix.htm + + +File: python.info, Node: tkinter scrolledtext — Scrolled Text Widget, Next: IDLE<3>, Prev: tkinter tix — Extension widgets for Tk, Up: Graphical User Interfaces with Tk + +5.25.4 ‘tkinter.scrolledtext’ — Scrolled Text Widget +---------------------------------------------------- + +`Source code:' Lib/tkinter/scrolledtext.py(1) + +__________________________________________________________________ + +The *note tkinter.scrolledtext: 10d. module provides a class of the same +name which implements a basic text widget which has a vertical scroll +bar configured to do the “right thing.” Using the ‘ScrolledText’ class +is a lot easier than setting up a text widget and scroll bar directly. +The constructor is the same as that of the ‘tkinter.Text’ class. + +The text widget and scrollbar are packed together in a ‘Frame’, and the +methods of the ‘Grid’ and ‘Pack’ geometry managers are acquired from the +‘Frame’ object. This allows the ‘ScrolledText’ widget to be used +directly to achieve most normal geometry management behavior. + +Should more specific control be necessary, the following attributes are +available: + + -- Attribute: ScrolledText.frame + + The frame which surrounds the text and scroll bar widgets. + + -- Attribute: ScrolledText.vbar + + The scroll bar widget. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/tkinter/scrolledtext.py + + +File: python.info, Node: IDLE<3>, Next: Other Graphical User Interface Packages, Prev: tkinter scrolledtext — Scrolled Text Widget, Up: Graphical User Interfaces with Tk + +5.25.5 IDLE +----------- + +`Source code:' Lib/idlelib/(1) + +__________________________________________________________________ + +IDLE is Python’s Integrated Development and Learning Environment. + +IDLE has the following features: + + * coded in 100% pure Python, using the *note tkinter: 10c. GUI + toolkit + + * cross-platform: works mostly the same on Windows, Unix, and macOS + + * Python shell window (interactive interpreter) with colorizing of + code input, output, and error messages + + * multi-window text editor with multiple undo, Python colorizing, + smart indent, call tips, auto completion, and other features + + * search within any window, replace within editor windows, and search + through multiple files (grep) + + * debugger with persistent breakpoints, stepping, and viewing of + global and local namespaces + + * configuration, browsers, and other dialogs + +* Menu: + +* Menus:: +* Editing and navigation:: +* Startup and code execution:: +* Help and preferences:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/idlelib/ + + +File: python.info, Node: Menus, Next: Editing and navigation, Up: IDLE<3> + +5.25.5.1 Menus +.............. + +IDLE has two main window types, the Shell window and the Editor window. +It is possible to have multiple editor windows simultaneously. On +Windows and Linux, each has its own top menu. Each menu documented +below indicates which window type it is associated with. + +Output windows, such as used for Edit => Find in Files, are a subtype of +editor window. They currently have the same top menu but a different +default title and context menu. + +On macOS, there is one application menu. It dynamically changes +according to the window currently selected. It has an IDLE menu, and +some entries described below are moved around to conform to Apple +guidelines. + +* Menu: + +* File menu (Shell and Editor): File menu Shell and Editor. +* Edit menu (Shell and Editor): Edit menu Shell and Editor. +* Format menu (Editor window only): Format menu Editor window only. +* Run menu (Editor window only): Run menu Editor window only. +* Shell menu (Shell window only): Shell menu Shell window only. +* Debug menu (Shell window only): Debug menu Shell window only. +* Options menu (Shell and Editor): Options menu Shell and Editor. +* Window menu (Shell and Editor): Window menu Shell and Editor. +* Help menu (Shell and Editor): Help menu Shell and Editor. +* Context Menus:: + + +File: python.info, Node: File menu Shell and Editor, Next: Edit menu Shell and Editor, Up: Menus + +5.25.5.2 File menu (Shell and Editor) +..................................... + +New File + + Create a new file editing window. + +Open… + + Open an existing file with an Open dialog. + +Recent Files + + Open a list of recent files. Click one to open it. + +Open Module… + + Open an existing module (searches sys.path). + +Class Browser + + Show functions, classes, and methods in the current Editor file in + a tree structure. In the shell, open a module first. + +Path Browser + + Show sys.path directories, modules, functions, classes and methods + in a tree structure. + +Save + + Save the current window to the associated file, if there is one. + Windows that have been changed since being opened or last saved + have a * before and after the window title. If there is no + associated file, do Save As instead. + +Save As… + + Save the current window with a Save As dialog. The file saved + becomes the new associated file for the window. + +Save Copy As… + + Save the current window to different file without changing the + associated file. + +Print Window + + Print the current window to the default printer. + +Close + + Close the current window (ask to save if unsaved). + +Exit + + Close all windows and quit IDLE (ask to save unsaved windows). + + +File: python.info, Node: Edit menu Shell and Editor, Next: Format menu Editor window only, Prev: File menu Shell and Editor, Up: Menus + +5.25.5.3 Edit menu (Shell and Editor) +..................................... + +Undo + + Undo the last change to the current window. A maximum of 1000 + changes may be undone. + +Redo + + Redo the last undone change to the current window. + +Cut + + Copy selection into the system-wide clipboard; then delete the + selection. + +Copy + + Copy selection into the system-wide clipboard. + +Paste + + Insert contents of the system-wide clipboard into the current + window. + +The clipboard functions are also available in context menus. + +Select All + + Select the entire contents of the current window. + +Find… + + Open a search dialog with many options + +Find Again + + Repeat the last search, if there is one. + +Find Selection + + Search for the currently selected string, if there is one. + +Find in Files… + + Open a file search dialog. Put results in a new output window. + +Replace… + + Open a search-and-replace dialog. + +Go to Line + + Move the cursor to the beginning of the line requested and make + that line visible. A request past the end of the file goes to the + end. Clear any selection and update the line and column status. + +Show Completions + + Open a scrollable list allowing selection of existing names. See + *note Completions: 2f4c. in the Editing and navigation section + below. + +Expand Word + + Expand a prefix you have typed to match a full word in the same + window; repeat to get a different expansion. + +Show call tip + + After an unclosed parenthesis for a function, open a small window + with function parameter hints. See *note Calltips: 2f4d. in the + Editing and navigation section below. + +Show surrounding parens + + Highlight the surrounding parenthesis. + + +File: python.info, Node: Format menu Editor window only, Next: Run menu Editor window only, Prev: Edit menu Shell and Editor, Up: Menus + +5.25.5.4 Format menu (Editor window only) +......................................... + +Indent Region + + Shift selected lines right by the indent width (default 4 spaces). + +Dedent Region + + Shift selected lines left by the indent width (default 4 spaces). + +Comment Out Region + + Insert ## in front of selected lines. + +Uncomment Region + + Remove leading # or ## from selected lines. + +Tabify Region + + Turn `leading' stretches of spaces into tabs. (Note: We recommend + using 4 space blocks to indent Python code.) + +Untabify Region + + Turn `all' tabs into the correct number of spaces. + +Toggle Tabs + + Open a dialog to switch between indenting with spaces and tabs. + +New Indent Width + + Open a dialog to change indent width. The accepted default by the + Python community is 4 spaces. + +Format Paragraph + + Reformat the current blank-line-delimited paragraph in comment + block or multiline string or selected line in a string. All lines + in the paragraph will be formatted to less than N columns, where N + defaults to 72. + +Strip trailing whitespace + + Remove trailing space and other whitespace characters after the + last non-whitespace character of a line by applying str.rstrip to + each line, including lines within multiline strings. Except for + Shell windows, remove extra newlines at the end of the file. + + +File: python.info, Node: Run menu Editor window only, Next: Shell menu Shell window only, Prev: Format menu Editor window only, Up: Menus + +5.25.5.5 Run menu (Editor window only) +...................................... + +Run Module + + Do *note Check Module: 2f52. If no error, restart the shell to + clean the environment, then execute the module. Output is + displayed in the Shell window. Note that output requires use of + ‘print’ or ‘write’. When execution is complete, the Shell retains + focus and displays a prompt. At this point, one may interactively + explore the result of execution. This is similar to executing a + file with ‘python -i file’ at a command line. + +Run… Customized + + Same as *note Run Module: 2f51, but run the module with customized + settings. `Command Line Arguments' extend *note sys.argv: bfb. as + if passed on a command line. The module can be run in the Shell + without restarting. + +Check Module + + Check the syntax of the module currently open in the Editor window. + If the module has not been saved IDLE will either prompt the user + to save or autosave, as selected in the General tab of the Idle + Settings dialog. If there is a syntax error, the approximate + location is indicated in the Editor window. + +Python Shell + + Open or wake up the Python Shell window. + + +File: python.info, Node: Shell menu Shell window only, Next: Debug menu Shell window only, Prev: Run menu Editor window only, Up: Menus + +5.25.5.6 Shell menu (Shell window only) +....................................... + +View Last Restart + + Scroll the shell window to the last Shell restart. + +Restart Shell + + Restart the shell to clean the environment and reset display and + exception handling. + +Previous History + + Cycle through earlier commands in history which match the current + entry. + +Next History + + Cycle through later commands in history which match the current + entry. + +Interrupt Execution + + Stop a running program. + + +File: python.info, Node: Debug menu Shell window only, Next: Options menu Shell and Editor, Prev: Shell menu Shell window only, Up: Menus + +5.25.5.7 Debug menu (Shell window only) +....................................... + +Go to File/Line + + Look on the current line. with the cursor, and the line above for + a filename and line number. If found, open the file if not already + open, and show the line. Use this to view source lines referenced + in an exception traceback and lines found by Find in Files. Also + available in the context menu of the Shell window and Output + windows. + +Debugger (toggle) + + When activated, code entered in the Shell or run from an Editor + will run under the debugger. In the Editor, breakpoints can be set + with the context menu. This feature is still incomplete and + somewhat experimental. + +Stack Viewer + + Show the stack traceback of the last exception in a tree widget, + with access to locals and globals. + +Auto-open Stack Viewer + + Toggle automatically opening the stack viewer on an unhandled + exception. + + +File: python.info, Node: Options menu Shell and Editor, Next: Window menu Shell and Editor, Prev: Debug menu Shell window only, Up: Menus + +5.25.5.8 Options menu (Shell and Editor) +........................................ + +Configure IDLE + + Open a configuration dialog and change preferences for the + following: fonts, indentation, keybindings, text color themes, + startup windows and size, additional help sources, and extensions. + On macOS, open the configuration dialog by selecting Preferences in + the application menu. For more details, see *note Setting + preferences: 2f58. under Help and preferences. + +Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window. + +Show/Hide Code Context (Editor Window only) + + Open a pane at the top of the edit window which shows the block + context of the code which has scrolled above the top of the window. + See *note Code Context: 2f59. in the Editing and Navigation section + below. + +Show/Hide Line Numbers (Editor Window only) + + Open a column to the left of the edit window which shows the number + of each line of text. The default is off, which may be changed in + the preferences (see *note Setting preferences: 2f58.). + +Zoom/Restore Height + + Toggles the window between normal size and maximum height. The + initial size defaults to 40 lines by 80 chars unless changed on the + General tab of the Configure IDLE dialog. The maximum height for a + screen is determined by momentarily maximizing a window the first + time one is zoomed on the screen. Changing screen settings may + invalidate the saved height. This toggle has no effect when a + window is maximized. + + +File: python.info, Node: Window menu Shell and Editor, Next: Help menu Shell and Editor, Prev: Options menu Shell and Editor, Up: Menus + +5.25.5.9 Window menu (Shell and Editor) +....................................... + +Lists the names of all open windows; select one to bring it to the +foreground (deiconifying it if necessary). + + +File: python.info, Node: Help menu Shell and Editor, Next: Context Menus, Prev: Window menu Shell and Editor, Up: Menus + +5.25.5.10 Help menu (Shell and Editor) +...................................... + +About IDLE + + Display version, copyright, license, credits, and more. + +IDLE Help + + Display this IDLE document, detailing the menu options, basic + editing and navigation, and other tips. + +Python Docs + + Access local Python documentation, if installed, or start a web + browser and open docs.python.org showing the latest Python + documentation. + +Turtle Demo + + Run the turtledemo module with example Python code and turtle + drawings. + +Additional help sources may be added here with the Configure IDLE dialog +under the General tab. See the *note Help sources: 2f5c. subsection +below for more on Help menu choices. + + +File: python.info, Node: Context Menus, Prev: Help menu Shell and Editor, Up: Menus + +5.25.5.11 Context Menus +....................... + +Open a context menu by right-clicking in a window (Control-click on +macOS). Context menus have the standard clipboard functions also on the +Edit menu. + +Cut + + Copy selection into the system-wide clipboard; then delete the + selection. + +Copy + + Copy selection into the system-wide clipboard. + +Paste + + Insert contents of the system-wide clipboard into the current + window. + +Editor windows also have breakpoint functions. Lines with a breakpoint +set are specially marked. Breakpoints only have an effect when running +under the debugger. Breakpoints for a file are saved in the user’s +‘.idlerc’ directory. + +Set Breakpoint + + Set a breakpoint on the current line. + +Clear Breakpoint + + Clear the breakpoint on that line. + +Shell and Output windows also have the following. + +Go to file/line + + Same as in Debug menu. + +The Shell window also has an output squeezing facility explained in the +`Python Shell window' subsection below. + +Squeeze + + If the cursor is over an output line, squeeze all the output + between the code above and the prompt below down to a ‘Squeezed + text’ label. + + +File: python.info, Node: Editing and navigation, Next: Startup and code execution, Prev: Menus, Up: IDLE<3> + +5.25.5.12 Editing and navigation +................................ + +* Menu: + +* Editor windows:: +* Key bindings:: +* Automatic indentation:: +* Completions:: +* Calltips:: +* Code Context:: +* Python Shell window:: +* Text colors:: + + +File: python.info, Node: Editor windows, Next: Key bindings, Up: Editing and navigation + +5.25.5.13 Editor windows +........................ + +IDLE may open editor windows when it starts, depending on settings and +how you start IDLE. Thereafter, use the File menu. There can be only +one open editor window for a given file. + +The title bar contains the name of the file, the full path, and the +version of Python and IDLE running the window. The status bar contains +the line number (‘Ln’) and column number (‘Col’). Line numbers start +with 1; column numbers with 0. + +IDLE assumes that files with a known .py* extension contain Python code +and that other files do not. Run Python code with the Run menu. + + +File: python.info, Node: Key bindings, Next: Automatic indentation, Prev: Editor windows, Up: Editing and navigation + +5.25.5.14 Key bindings +...................... + +In this section, ‘C’ refers to the ‘Control’ key on Windows and Unix and +the ‘Command’ key on macOS. + + * ‘Backspace’ deletes to the left; ‘Del’ deletes to the right + + * ‘C-Backspace’ delete word left; ‘C-Del’ delete word to the right + + * Arrow keys and ‘Page Up’/‘Page Down’ to move around + + * ‘C-LeftArrow’ and ‘C-RightArrow’ moves by words + + * ‘Home’/‘End’ go to begin/end of line + + * ‘C-Home’/‘C-End’ go to begin/end of file + + * Some useful Emacs bindings are inherited from Tcl/Tk: + + * ‘C-a’ beginning of line + + * ‘C-e’ end of line + + * ‘C-k’ kill line (but doesn’t put it in clipboard) + + * ‘C-l’ center window around the insertion point + + * ‘C-b’ go backward one character without deleting (usually + you can also use the cursor key for this) + + * ‘C-f’ go forward one character without deleting (usually + you can also use the cursor key for this) + + * ‘C-p’ go up one line (usually you can also use the cursor + key for this) + + * ‘C-d’ delete next character + +Standard keybindings (like ‘C-c’ to copy and ‘C-v’ to paste) may work. +Keybindings are selected in the Configure IDLE dialog. + + +File: python.info, Node: Automatic indentation, Next: Completions, Prev: Key bindings, Up: Editing and navigation + +5.25.5.15 Automatic indentation +............................... + +After a block-opening statement, the next line is indented by 4 spaces +(in the Python Shell window by one tab). After certain keywords (break, +return etc.) the next line is dedented. In leading indentation, +‘Backspace’ deletes up to 4 spaces if they are there. ‘Tab’ inserts +spaces (in the Python Shell window one tab), number depends on Indent +width. Currently, tabs are restricted to four spaces due to Tcl/Tk +limitations. + +See also the indent/dedent region commands on the *note Format menu: +2f4e. + + +File: python.info, Node: Completions, Next: Calltips, Prev: Automatic indentation, Up: Editing and navigation + +5.25.5.16 Completions +..................... + +Completions are supplied, when requested and available, for module +names, attributes of classes or functions, or filenames. Each request +method displays a completion box with existing names. (See tab +completions below for an exception.) For any box, change the name being +completed and the item highlighted in the box by typing and deleting +characters; by hitting ‘Up’, ‘Down’, ‘PageUp’, ‘PageDown’, ‘Home’, and +‘End’ keys; and by a single click within the box. Close the box with +‘Escape’, ‘Enter’, and double ‘Tab’ keys or clicks outside the box. A +double click within the box selects and closes. + +One way to open a box is to type a key character and wait for a +predefined interval. This defaults to 2 seconds; customize it in the +settings dialog. (To prevent auto popups, set the delay to a large +number of milliseconds, such as 100000000.) For imported module names +or class or function attributes, type ‘.’. For filenames in the root +directory, type *note os.sep: 1936. or *note os.altsep: 1937. +immediately after an opening quote. (On Windows, one can specify a +drive first.) Move into subdirectories by typing a directory name and a +separator. + +Instead of waiting, or after a box is closed, open a completion box +immediately with Show Completions on the Edit menu. The default hot key +is ‘C-space’. If one types a prefix for the desired name before opening +the box, the first match or near miss is made visible. The result is +the same as if one enters a prefix after the box is displayed. Show +Completions after a quote completes filenames in the current directory +instead of a root directory. + +Hitting ‘Tab’ after a prefix usually has the same effect as Show +Completions. (With no prefix, it indents.) However, if there is only +one match to the prefix, that match is immediately added to the editor +text without opening a box. + +Invoking ‘Show Completions’, or hitting ‘Tab’ after a prefix, outside of +a string and without a preceding ‘.’ opens a box with keywords, builtin +names, and available module-level names. + +When editing code in an editor (as oppose to Shell), increase the +available module-level names by running your code and not restarting the +Shell thereafter. This is especially useful after adding imports at the +top of a file. This also increases possible attribute completions. + +Completion boxes intially exclude names beginning with ‘_’ or, for +modules, not included in ‘__all__’. The hidden names can be accessed by +typing ‘_’ after ‘.’, either before or after the box is opened. + + +File: python.info, Node: Calltips, Next: Code Context, Prev: Completions, Up: Editing and navigation + +5.25.5.17 Calltips +.................. + +A calltip is shown automatically when one types ‘(’ after the name of an +`accessible' function. A function name expression may include dots and +subscripts. A calltip remains until it is clicked, the cursor is moved +out of the argument area, or ‘)’ is typed. Whenever the cursor is in +the argument part of a definition, select Edit and “Show Call Tip” on +the menu or enter its shortcut to display a calltip. + +The calltip consists of the function’s signature and docstring up to the +latter’s first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A ‘/’ or ‘*’ in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change. + +In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart. + +For example, restart the Shell and enter ‘itertools.count(’. A calltip +appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter ‘turtle.write(’ and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter ‘import turtle’. Thereafter, ‘turtle.write(’ will +display a calltip. + +In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file. + + +File: python.info, Node: Code Context, Next: Python Shell window, Prev: Calltips, Up: Editing and navigation + +5.25.5.18 Code Context +...................... + +Within an editor window containing Python code, code context can be +toggled in order to show or hide a pane at the top of the window. When +shown, this pane freezes the opening lines for block code, such as those +beginning with ‘class’, ‘def’, or ‘if’ keywords, that would have +otherwise scrolled out of view. The size of the pane will be expanded +and contracted as needed to show the all current levels of context, up +to the maximum number of lines defined in the Configure IDLE dialog +(which defaults to 15). If there are no current context lines and the +feature is toggled on, a single blank line will display. Clicking on a +line in the context pane will move that line to the top of the editor. + +The text and background colors for the context pane can be configured +under the Highlights tab in the Configure IDLE dialog. + + +File: python.info, Node: Python Shell window, Next: Text colors, Prev: Code Context, Up: Editing and navigation + +5.25.5.19 Python Shell window +............................. + +With IDLE’s Shell, one enters, edits, and recalls complete statements. +Most consoles and terminals only work with a single physical line at a +time. + +When one pastes code into Shell, it is not compiled and possibly +executed until one hits ‘Return’. One may edit pasted code first. If +one pastes more that one statement into Shell, the result will be a +*note SyntaxError: 458. when multiple statements are compiled as if they +were one. + +The editing features described in previous subsections work when +entering code interactively. IDLE’s Shell window also responds to the +following keys. + + * ‘C-c’ interrupts executing command + + * ‘C-d’ sends end-of-file; closes window if typed at a ‘>>>’ prompt + + * ‘Alt-/’ (Expand word) is also useful to reduce typing + + Command history + + * ‘Alt-p’ retrieves previous command matching what you have + typed. On macOS use ‘C-p’. + + * ‘Alt-n’ retrieves next. On macOS use ‘C-n’. + + * ‘Return’ while on any previous command retrieves that command + + +File: python.info, Node: Text colors, Prev: Python Shell window, Up: Editing and navigation + +5.25.5.20 Text colors +..................... + +Idle defaults to black on white text, but colors text with special +meanings. For the shell, these are shell output, shell error, user +output, and user error. For Python code, at the shell prompt or in an +editor, these are keywords, builtin class and function names, names +following ‘class’ and ‘def’, strings, and comments. For any text +window, these are the cursor (when present), found text (when possible), +and selected text. + +Text coloring is done in the background, so uncolorized text is +occasionally visible. To change the color scheme, use the Configure +IDLE dialog Highlighting tab. The marking of debugger breakpoint lines +in the editor and text in popups and dialogs is not user-configurable. + + +File: python.info, Node: Startup and code execution, Next: Help and preferences, Prev: Editing and navigation, Up: IDLE<3> + +5.25.5.21 Startup and code execution +.................................... + +Upon startup with the ‘-s’ option, IDLE will execute the file referenced +by the environment variables ‘IDLESTARTUP’ or *note PYTHONSTARTUP: 8e5. +IDLE first checks for ‘IDLESTARTUP’; if ‘IDLESTARTUP’ is present the +file referenced is run. If ‘IDLESTARTUP’ is not present, IDLE checks +for ‘PYTHONSTARTUP’. Files referenced by these environment variables +are convenient places to store functions that are used frequently from +the IDLE shell, or for executing import statements to import common +modules. + +In addition, ‘Tk’ also loads a startup file if it is present. Note that +the Tk file is loaded unconditionally. This additional file is +‘.Idle.py’ and is looked for in the user’s home directory. Statements +in this file will be executed in the Tk namespace, so this file is not +useful for importing functions to be used from IDLE’s Python shell. + +* Menu: + +* Command line usage:: +* Startup failure:: +* Running user code:: +* User output in Shell:: +* Developing tkinter applications:: +* Running without a subprocess:: + + +File: python.info, Node: Command line usage, Next: Startup failure, Up: Startup and code execution + +5.25.5.22 Command line usage +............................ + + idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ... + + -c command run command in the shell window + -d enable debugger and open shell window + -e open editor window + -h print help message with legal combinations and exit + -i open shell window + -r file run file in shell window + -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window + -t title set title of shell window + - run stdin in shell (- must be last option before args) + +If there are arguments: + + * If ‘-’, ‘-c’, or ‘r’ is used, all arguments are placed in + ‘sys.argv[1:...]’ and ‘sys.argv[0]’ is set to ‘''’, ‘'-c'’, or + ‘'-r'’. No editor window is opened, even if that is the default + set in the Options dialog. + + * Otherwise, arguments are files opened for editing and ‘sys.argv’ + reflects the arguments passed to IDLE itself. + + +File: python.info, Node: Startup failure, Next: Running user code, Prev: Command line usage, Up: Startup and code execution + +5.25.5.23 Startup failure +......................... + +IDLE uses a socket to communicate between the IDLE GUI process and the +user code execution process. A connection must be established whenever +the Shell starts or restarts. (The latter is indicated by a divider +line that says ‘RESTART’). If the user process fails to connect to the +GUI process, it usually displays a ‘Tk’ error box with a ‘cannot +connect’ message that directs the user here. It then exits. + +One specific connection failure on Unix systems results from +misconfigured masquerading rules somewhere in a system’s network setup. +When IDLE is started from a terminal, one will see a message starting +with ‘** Invalid host:’. The valid value is ‘127.0.0.1 +(idlelib.rpc.LOCALHOST)’. One can diagnose with ‘tcpconnect -irv +127.0.0.1 6543’ in one terminal window and ‘tcplisten <same args>’ in +another. + +A common cause of failure is a user-written file with the same name as a +standard library module, such as `random.py' and `tkinter.py'. When +such a file is located in the same directory as a file that is about to +be run, IDLE cannot import the stdlib file. The current fix is to +rename the user file. + +Though less common than in the past, an antivirus or firewall program +may stop the connection. If the program cannot be taught to allow the +connection, then it must be turned off for IDLE to work. It is safe to +allow this internal connection because no data is visible on external +ports. A similar problem is a network mis-configuration that blocks +connections. + +Python installation issues occasionally stop IDLE: multiple versions can +clash, or a single installation might need admin access. If one undo +the clash, or cannot or does not want to run as admin, it might be +easiest to completely remove Python and start over. + +A zombie pythonw.exe process could be a problem. On Windows, use Task +Manager to check for one and stop it if there is. Sometimes a restart +initiated by a program crash or Keyboard Interrupt (control-C) may fail +to connect. Dismissing the error box or using Restart Shell on the +Shell menu may fix a temporary problem. + +When IDLE first starts, it attempts to read user configuration files in +‘~/.idlerc/’ (~ is one’s home directory). If there is a problem, an +error message should be displayed. Leaving aside random disk glitches, +this can be prevented by never editing the files by hand. Instead, use +the configuration dialog, under Options. Once there is an error in a +user configuration file, the best solution may be to delete it and start +over with the settings dialog. + +If IDLE quits with no message, and it was not started from a console, +try starting it from a console or terminal (‘python -m idlelib’) and see +if this results in an error message. + +On Unix-based systems with tcl/tk older than ‘8.6.11’ (see ‘About IDLE’) +certain characters of certain fonts can cause a tk failure with a +message to the terminal. This can happen either if one starts IDLE to +edit a file with such a character or later when entering such a +character. If one cannot upgrade tcl/tk, then re-configure IDLE to use +a font that works better. + + +File: python.info, Node: Running user code, Next: User output in Shell, Prev: Startup failure, Up: Startup and code execution + +5.25.5.24 Running user code +........................... + +With rare exceptions, the result of executing Python code with IDLE is +intended to be the same as executing the same code by the default +method, directly with Python in a text-mode system console or terminal +window. However, the different interface and operation occasionally +affect visible results. For instance, ‘sys.modules’ starts with more +entries, and ‘threading.activeCount()’ returns 2 instead of 1. + +By default, IDLE runs user code in a separate OS process rather than in +the user interface process that runs the shell and editor. In the +execution process, it replaces ‘sys.stdin’, ‘sys.stdout’, and +‘sys.stderr’ with objects that get input from and send output to the +Shell window. The original values stored in ‘sys.__stdin__’, +‘sys.__stdout__’, and ‘sys.__stderr__’ are not touched, but may be +‘None’. + +Sending print output from one process to a text widget in another is +slower than printing to a system terminal in the same process. This has +the most effect when printing multiple arguments, as the string for each +argument, each separator, the newline are sent separately. For +development, this is usually not a problem, but if one wants to print +faster in IDLE, format and join together everything one wants displayed +together and then print a single string. Both format strings and *note +str.join(): 12dd. can help combine fields and lines. + +IDLE’s standard stream replacements are not inherited by subprocesses +created in the execution process, whether directly by user code or by +modules such as multiprocessing. If such subprocess use ‘input’ from +sys.stdin or ‘print’ or ‘write’ to sys.stdout or sys.stderr, IDLE should +be started in a command line window. The secondary subprocess will then +be attached to that window for input and output. + +If ‘sys’ is reset by user code, such as with ‘importlib.reload(sys)’, +IDLE’s changes are lost and input from the keyboard and output to the +screen will not work correctly. + +When Shell has the focus, it controls the keyboard and screen. This is +normally transparent, but functions that directly access the keyboard +and screen will not work. These include system-specific functions that +determine whether a key has been pressed and if so, which. + +The IDLE code running in the execution process adds frames to the call +stack that would not be there otherwise. IDLE wraps +‘sys.getrecursionlimit’ and ‘sys.setrecursionlimit’ to reduce the effect +of the additional stack frames. + +When user code raises SystemExit either directly or by calling sys.exit, +IDLE returns to a Shell prompt instead of exiting. + + +File: python.info, Node: User output in Shell, Next: Developing tkinter applications, Prev: Running user code, Up: Startup and code execution + +5.25.5.25 User output in Shell +.............................. + +When a program outputs text, the result is determined by the +corresponding output device. When IDLE executes user code, ‘sys.stdout’ +and ‘sys.stderr’ are connected to the display area of IDLE’s Shell. +Some of its features are inherited from the underlying Tk Text widget. +Others are programmed additions. Where it matters, Shell is designed +for development rather than production runs. + +For instance, Shell never throws away output. A program that sends +unlimited output to Shell will eventually fill memory, resulting in a +memory error. In contrast, some system text windows only keep the last +n lines of output. A Windows console, for instance, keeps a +user-settable 1 to 9999 lines, with 300 the default. + +A Tk Text widget, and hence IDLE’s Shell, displays characters +(codepoints) in the BMP (Basic Multilingual Plane) subset of Unicode. +Which characters are displayed with a proper glyph and which with a +replacement box depends on the operating system and installed fonts. +Tab characters cause the following text to begin after the next tab +stop. (They occur every 8 ‘characters’). Newline characters cause +following text to appear on a new line. Other control characters are +ignored or displayed as a space, box, or something else, depending on +the operating system and font. (Moving the text cursor through such +output with arrow keys may exhibit some surprising spacing behavior.) + + >>> s = 'a\tb\a<\x02><\r>\bc\nd' # Enter 22 chars. + >>> len(s) + 14 + >>> s # Display repr(s) + 'a\tb\x07<\x02><\r>\x08c\nd' + >>> print(s, end='') # Display s as is. + # Result varies by OS and font. Try it. + +The ‘repr’ function is used for interactive echo of expression values. +It returns an altered version of the input string in which control +codes, some BMP codepoints, and all non-BMP codepoints are replaced with +escape codes. As demonstrated above, it allows one to identify the +characters in a string, regardless of how they are displayed. + +Normal and error output are generally kept separate (on separate lines) +from code input and each other. They each get different highlight +colors. + +For SyntaxError tracebacks, the normal ‘^’ marking where the error was +detected is replaced by coloring the text with an error highlight. When +code run from a file causes other exceptions, one may right click on a +traceback line to jump to the corresponding line in an IDLE editor. The +file will be opened if necessary. + +Shell has a special facility for squeezing output lines down to a +‘Squeezed text’ label. This is done automatically for output over N +lines (N = 50 by default). N can be changed in the PyShell section of +the General page of the Settings dialog. Output with fewer lines can be +squeezed by right clicking on the output. This can be useful lines long +enough to slow down scrolling. + +Squeezed output is expanded in place by double-clicking the label. It +can also be sent to the clipboard or a separate view window by +right-clicking the label. + + +File: python.info, Node: Developing tkinter applications, Next: Running without a subprocess, Prev: User output in Shell, Up: Startup and code execution + +5.25.5.26 Developing tkinter applications +......................................... + +IDLE is intentionally different from standard Python in order to +facilitate development of tkinter programs. Enter ‘import tkinter as +tk; root = tk.Tk()’ in standard Python and nothing appears. Enter the +same in IDLE and a tk window appears. In standard Python, one must also +enter ‘root.update()’ to see the window. IDLE does the equivalent in +the background, about 20 times a second, which is about every 50 +milliseconds. Next enter ‘b = tk.Button(root, text='button'); +b.pack()’. Again, nothing visibly changes in standard Python until one +enters ‘root.update()’. + +Most tkinter programs run ‘root.mainloop()’, which usually does not +return until the tk app is destroyed. If the program is run with +‘python -i’ or from an IDLE editor, a ‘>>>’ shell prompt does not appear +until ‘mainloop()’ returns, at which time there is nothing left to +interact with. + +When running a tkinter program from an IDLE editor, one can comment out +the mainloop call. One then gets a shell prompt immediately and can +interact with the live application. One just has to remember to +re-enable the mainloop call when running in standard Python. + + +File: python.info, Node: Running without a subprocess, Prev: Developing tkinter applications, Up: Startup and code execution + +5.25.5.27 Running without a subprocess +...................................... + +By default, IDLE executes user code in a separate subprocess via a +socket, which uses the internal loopback interface. This connection is +not externally visible and no data is sent to or received from the +Internet. If firewall software complains anyway, you can ignore it. + +If the attempt to make the socket connection fails, Idle will notify +you. Such failures are sometimes transient, but if persistent, the +problem may be either a firewall blocking the connection or +misconfiguration of a particular system. Until the problem is fixed, +one can run Idle with the -n command line switch. + +If IDLE is started with the -n command line switch it will run in a +single process and will not create the subprocess which runs the RPC +Python execution server. This can be useful if Python cannot create the +subprocess or the RPC socket interface on your platform. However, in +this mode user code is not isolated from IDLE itself. Also, the +environment is not restarted when Run/Run Module (F5) is selected. If +your code has been modified, you must reload() the affected modules and +re-import any specific items (e.g. from foo import baz) if the changes +are to take effect. For these reasons, it is preferable to run IDLE +with the default subprocess if at all possible. + +Deprecated since version 3.4. + + +File: python.info, Node: Help and preferences, Prev: Startup and code execution, Up: IDLE<3> + +5.25.5.28 Help and preferences +.............................. + +* Menu: + +* Help sources:: +* Setting preferences:: +* IDLE on macOS:: +* Extensions:: + + +File: python.info, Node: Help sources, Next: Setting preferences, Up: Help and preferences + +5.25.5.29 Help sources +...................... + +Help menu entry “IDLE Help” displays a formatted html version of the +IDLE chapter of the Library Reference. The result, in a read-only +tkinter text window, is close to what one sees in a web browser. +Navigate through the text with a mousewheel, the scrollbar, or up and +down arrow keys held down. Or click the TOC (Table of Contents) button +and select a section header in the opened box. + +Help menu entry “Python Docs” opens the extensive sources of help, +including tutorials, available at ‘docs.python.org/x.y’, where ‘x.y’ is +the currently running Python version. If your system has an off-line +copy of the docs (this may be an installation option), that will be +opened instead. + +Selected URLs can be added or removed from the help menu at any time +using the General tab of the Configure IDLE dialog. + + +File: python.info, Node: Setting preferences, Next: IDLE on macOS, Prev: Help sources, Up: Help and preferences + +5.25.5.30 Setting preferences +............................. + +The font preferences, highlighting, keys, and general preferences can be +changed via Configure IDLE on the Option menu. Non-default user +settings are saved in a ‘.idlerc’ directory in the user’s home +directory. Problems caused by bad user configuration files are solved +by editing or deleting one or more of the files in ‘.idlerc’. + +On the Font tab, see the text sample for the effect of font face and +size on multiple characters in multiple languages. Edit the sample to +add other characters of personal interest. Use the sample to select +monospaced fonts. If particular characters have problems in Shell or an +editor, add them to the top of the sample and try changing first size +and then font. + +On the Highlights and Keys tab, select a built-in or custom color theme +and key set. To use a newer built-in color theme or key set with older +IDLEs, save it as a new custom theme or key set and it well be +accessible to older IDLEs. + + +File: python.info, Node: IDLE on macOS, Next: Extensions, Prev: Setting preferences, Up: Help and preferences + +5.25.5.31 IDLE on macOS +....................... + +Under System Preferences: Dock, one can set “Prefer tabs when opening +documents” to “Always”. This setting is not compatible with the +tk/tkinter GUI framework used by IDLE, and it breaks a few IDLE +features. + + +File: python.info, Node: Extensions, Prev: IDLE on macOS, Up: Help and preferences + +5.25.5.32 Extensions +.................... + +IDLE contains an extension facility. Preferences for extensions can be +changed with the Extensions tab of the preferences dialog. See the +beginning of config-extensions.def in the idlelib directory for further +information. The only current default extension is zzdummy, an example +also used for testing. + + +File: python.info, Node: Other Graphical User Interface Packages, Prev: IDLE<3>, Up: Graphical User Interfaces with Tk + +5.25.6 Other Graphical User Interface Packages +---------------------------------------------- + +Major cross-platform (Windows, Mac OS X, Unix-like) GUI toolkits are +available for Python: + +See also +........ + +PyGObject(1) + + PyGObject provides introspection bindings for C libraries using + GObject(2). One of these libraries is the GTK+ 3(3) widget set. + GTK+ comes with many more widgets than Tkinter provides. An online + Python GTK+ 3 Tutorial(4) is available. + +PyGTK(5) + + PyGTK provides bindings for an older version of the library, GTK+ + 2. It provides an object oriented interface that is slightly + higher level than the C one. There are also bindings to GNOME(6). + An online tutorial(7) is available. + +PyQt(8) + + PyQt is a ‘sip’-wrapped binding to the Qt toolkit. Qt is an + extensive C++ GUI application development framework that is + available for Unix, Windows and Mac OS X. ‘sip’ is a tool for + generating bindings for C++ libraries as Python classes, and is + specifically designed for Python. + +PySide2(9) + + Also known as the Qt for Python project, PySide2 is a newer binding + to the Qt toolkit. It is provided by The Qt Company and aims to + provide a complete port of PySide to Qt 5. Compared to PyQt, its + licensing scheme is friendlier to non-open source applications. + +wxPython(10) + + wxPython is a cross-platform GUI toolkit for Python that is built + around the popular wxWidgets(11) (formerly wxWindows) C++ toolkit. + It provides a native look and feel for applications on Windows, Mac + OS X, and Unix systems by using each platform’s native widgets + where ever possible, (GTK+ on Unix-like systems). In addition to + an extensive set of widgets, wxPython provides classes for online + documentation and context sensitive help, printing, HTML viewing, + low-level device context drawing, drag and drop, system clipboard + access, an XML-based resource format and more, including an ever + growing library of user-contributed modules. + +PyGTK, PyQt, PySide2, and wxPython, all have a modern look and feel and +more widgets than Tkinter. In addition, there are many other GUI +toolkits for Python, both cross-platform, and platform-specific. See +the GUI Programming(12) page in the Python Wiki for a much more complete +list, and also for links to documents where the different GUI toolkits +are compared. + + ---------- Footnotes ---------- + + (1) https://wiki.gnome.org/Projects/PyGObject + + (2) https://developer.gnome.org/gobject/stable/ + + (3) https://www.gtk.org/ + + (4) https://python-gtk-3-tutorial.readthedocs.io/ + + (5) http://www.pygtk.org/ + + (6) https://www.gnome.org/ + + (7) http://www.pygtk.org/pygtk2tutorial/index.html + + (8) https://riverbankcomputing.com/software/pyqt/intro + + (9) https://doc.qt.io/qtforpython/ + + (10) https://www.wxpython.org + + (11) https://www.wxwidgets.org/ + + (12) https://wiki.python.org/moin/GuiProgramming + + +File: python.info, Node: Development Tools, Next: Debugging and Profiling, Prev: Graphical User Interfaces with Tk, Up: The Python Standard Library + +5.26 Development Tools +====================== + +The modules described in this chapter help you write software. For +example, the *note pydoc: d9. module takes a module and generates +documentation based on the module’s contents. The *note doctest: 67. +and *note unittest: 11b. modules contains frameworks for writing unit +tests that automatically exercise code and verify that the expected +output is produced. ‘2to3’ can translate Python 2.x source code into +valid Python 3.x code. + +The list of modules described in this chapter is: + +* Menu: + +* typing — Support for type hints:: +* pydoc — Documentation generator and online help system:: +* doctest — Test interactive Python examples:: +* unittest — Unit testing framework:: +* unittest.mock — mock object library: unittest mock — mock object library. +* unittest.mock — getting started: unittest mock — getting started. +* 2to3 - Automated Python 2 to 3 code translation:: +* test — Regression tests package for Python:: +* test.support — Utilities for the Python test suite: test support — Utilities for the Python test suite. +* test.support.script_helper — Utilities for the Python execution tests: test support script_helper — Utilities for the Python execution tests. + + +File: python.info, Node: typing — Support for type hints, Next: pydoc — Documentation generator and online help system, Up: Development Tools + +5.26.1 ‘typing’ — Support for type hints +---------------------------------------- + +New in version 3.5. + +`Source code:' Lib/typing.py(1) + + Note: The Python runtime does not enforce function and variable + type annotations. They can be used by third party tools such as + type checkers, IDEs, linters, etc. + +__________________________________________________________________ + +This module provides runtime support for type hints as specified by PEP +484(2), PEP 526(3), PEP 544(4), PEP 586(5), PEP 589(6), and PEP 591(7). +The most fundamental support consists of the types *note Any: 652, *note +Union: 2f7b, *note Tuple: 2f7c, *note Callable: 2f7d, *note TypeVar: +2f7e, and *note Generic: 2f7f. For full specification please see PEP +484(8). For a simplified introduction to type hints see PEP 483(9). + +The function below takes and returns a string and is annotated as +follows: + + def greeting(name: str) -> str: + return 'Hello ' + name + +In the function ‘greeting’, the argument ‘name’ is expected to be of +type *note str: 330. and the return type *note str: 330. Subtypes are +accepted as arguments. + +* Menu: + +* Type aliases:: +* NewType:: +* Callable:: +* Generics:: +* User-defined generic types:: +* The Any type:: +* Nominal vs structural subtyping:: +* Classes, functions, and decorators: Classes functions and decorators. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/typing.py + + (2) https://www.python.org/dev/peps/pep-0484 + + (3) https://www.python.org/dev/peps/pep-0526 + + (4) https://www.python.org/dev/peps/pep-0544 + + (5) https://www.python.org/dev/peps/pep-0586 + + (6) https://www.python.org/dev/peps/pep-0589 + + (7) https://www.python.org/dev/peps/pep-0591 + + (8) https://www.python.org/dev/peps/pep-0484 + + (9) https://www.python.org/dev/peps/pep-0483 + + +File: python.info, Node: Type aliases, Next: NewType, Up: typing — Support for type hints + +5.26.1.1 Type aliases +..................... + +A type alias is defined by assigning the type to the alias. In this +example, ‘Vector’ and ‘List[float]’ will be treated as interchangeable +synonyms: + + from typing import List + Vector = List[float] + + def scale(scalar: float, vector: Vector) -> Vector: + return [scalar * num for num in vector] + + # typechecks; a list of floats qualifies as a Vector. + new_vector = scale(2.0, [1.0, -4.2, 5.4]) + +Type aliases are useful for simplifying complex type signatures. For +example: + + from typing import Dict, Tuple, Sequence + + ConnectionOptions = Dict[str, str] + Address = Tuple[str, int] + Server = Tuple[Address, ConnectionOptions] + + def broadcast_message(message: str, servers: Sequence[Server]) -> None: + ... + + # The static type checker will treat the previous type signature as + # being exactly equivalent to this one. + def broadcast_message( + message: str, + servers: Sequence[Tuple[Tuple[str, int], Dict[str, str]]]) -> None: + ... + +Note that ‘None’ as a type hint is a special case and is replaced by +‘type(None)’. + + +File: python.info, Node: NewType, Next: Callable, Prev: Type aliases, Up: typing — Support for type hints + +5.26.1.2 NewType +................ + +Use the *note NewType(): 5a5. helper function to create distinct types: + + from typing import NewType + + UserId = NewType('UserId', int) + some_id = UserId(524313) + +The static type checker will treat the new type as if it were a subclass +of the original type. This is useful in helping catch logical errors: + + def get_user_name(user_id: UserId) -> str: + ... + + # typechecks + user_a = get_user_name(UserId(42351)) + + # does not typecheck; an int is not a UserId + user_b = get_user_name(-1) + +You may still perform all ‘int’ operations on a variable of type +‘UserId’, but the result will always be of type ‘int’. This lets you +pass in a ‘UserId’ wherever an ‘int’ might be expected, but will prevent +you from accidentally creating a ‘UserId’ in an invalid way: + + # 'output' is of type 'int', not 'UserId' + output = UserId(23413) + UserId(54341) + +Note that these checks are enforced only by the static type checker. At +runtime, the statement ‘Derived = NewType('Derived', Base)’ will make +‘Derived’ a function that immediately returns whatever parameter you +pass it. That means the expression ‘Derived(some_value)’ does not +create a new class or introduce any overhead beyond that of a regular +function call. + +More precisely, the expression ‘some_value is Derived(some_value)’ is +always true at runtime. + +This also means that it is not possible to create a subtype of ‘Derived’ +since it is an identity function at runtime, not an actual type: + + from typing import NewType + + UserId = NewType('UserId', int) + + # Fails at runtime and does not typecheck + class AdminUserId(UserId): pass + +However, it is possible to create a *note NewType(): 5a5. based on a +‘derived’ ‘NewType’: + + from typing import NewType + + UserId = NewType('UserId', int) + + ProUserId = NewType('ProUserId', UserId) + +and typechecking for ‘ProUserId’ will work as expected. + +See PEP 484(1) for more details. + + Note: Recall that the use of a type alias declares two types to be + `equivalent' to one another. Doing ‘Alias = Original’ will make + the static type checker treat ‘Alias’ as being `exactly equivalent' + to ‘Original’ in all cases. This is useful when you want to + simplify complex type signatures. + + In contrast, ‘NewType’ declares one type to be a `subtype' of + another. Doing ‘Derived = NewType('Derived', Original)’ will make + the static type checker treat ‘Derived’ as a `subclass' of + ‘Original’, which means a value of type ‘Original’ cannot be used + in places where a value of type ‘Derived’ is expected. This is + useful when you want to prevent logic errors with minimal runtime + cost. + +New in version 3.5.2. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + +File: python.info, Node: Callable, Next: Generics, Prev: NewType, Up: typing — Support for type hints + +5.26.1.3 Callable +................. + +Frameworks expecting callback functions of specific signatures might be +type hinted using ‘Callable[[Arg1Type, Arg2Type], ReturnType]’. + +For example: + + from typing import Callable + + def feeder(get_next_item: Callable[[], str]) -> None: + # Body + + def async_query(on_success: Callable[[int], None], + on_error: Callable[[int, Exception], None]) -> None: + # Body + +It is possible to declare the return type of a callable without +specifying the call signature by substituting a literal ellipsis for the +list of arguments in the type hint: ‘Callable[..., ReturnType]’. + + +File: python.info, Node: Generics, Next: User-defined generic types, Prev: Callable, Up: typing — Support for type hints + +5.26.1.4 Generics +................. + +Since type information about objects kept in containers cannot be +statically inferred in a generic way, abstract base classes have been +extended to support subscription to denote expected types for container +elements. + + from typing import Mapping, Sequence + + def notify_by_email(employees: Sequence[Employee], + overrides: Mapping[str, str]) -> None: ... + +Generics can be parameterized by using a new factory available in typing +called *note TypeVar: 2f7e. + + from typing import Sequence, TypeVar + + T = TypeVar('T') # Declare type variable + + def first(l: Sequence[T]) -> T: # Generic function + return l[0] + + +File: python.info, Node: User-defined generic types, Next: The Any type, Prev: Generics, Up: typing — Support for type hints + +5.26.1.5 User-defined generic types +................................... + +A user-defined class can be defined as a generic class. + + from typing import TypeVar, Generic + from logging import Logger + + T = TypeVar('T') + + class LoggedVar(Generic[T]): + def __init__(self, value: T, name: str, logger: Logger) -> None: + self.name = name + self.logger = logger + self.value = value + + def set(self, new: T) -> None: + self.log('Set ' + repr(self.value)) + self.value = new + + def get(self) -> T: + self.log('Get ' + repr(self.value)) + return self.value + + def log(self, message: str) -> None: + self.logger.info('%s: %s', self.name, message) + +‘Generic[T]’ as a base class defines that the class ‘LoggedVar’ takes a +single type parameter ‘T’ . This also makes ‘T’ valid as a type within +the class body. + +The *note Generic: 2f7f. base class defines *note __class_getitem__(): +327. so that ‘LoggedVar[t]’ is valid as a type: + + from typing import Iterable + + def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None: + for var in vars: + var.set(0) + +A generic type can have any number of type variables, and type variables +may be constrained: + + from typing import TypeVar, Generic + ... + + T = TypeVar('T') + S = TypeVar('S', int, str) + + class StrangePair(Generic[T, S]): + ... + +Each type variable argument to *note Generic: 2f7f. must be distinct. +This is thus invalid: + + from typing import TypeVar, Generic + ... + + T = TypeVar('T') + + class Pair(Generic[T, T]): # INVALID + ... + +You can use multiple inheritance with *note Generic: 2f7f.: + + from typing import TypeVar, Generic, Sized + + T = TypeVar('T') + + class LinkedList(Sized, Generic[T]): + ... + +When inheriting from generic classes, some type variables could be +fixed: + + from typing import TypeVar, Mapping + + T = TypeVar('T') + + class MyDict(Mapping[str, T]): + ... + +In this case ‘MyDict’ has a single parameter, ‘T’. + +Using a generic class without specifying type parameters assumes *note +Any: 652. for each position. In the following example, ‘MyIterable’ is +not generic but implicitly inherits from ‘Iterable[Any]’: + + from typing import Iterable + + class MyIterable(Iterable): # Same as Iterable[Any] + +User defined generic type aliases are also supported. Examples: + + from typing import TypeVar, Iterable, Tuple, Union + S = TypeVar('S') + Response = Union[Iterable[S], int] + + # Return type here is same as Union[Iterable[str], int] + def response(query: str) -> Response[str]: + ... + + T = TypeVar('T', int, float, complex) + Vec = Iterable[Tuple[T, T]] + + def inproduct(v: Vec[T]) -> T: # Same as Iterable[Tuple[T, T]] + return sum(x*y for x, y in v) + +Changed in version 3.7: *note Generic: 2f7f. no longer has a custom +metaclass. + +A user-defined generic class can have ABCs as base classes without a +metaclass conflict. Generic metaclasses are not supported. The outcome +of parameterizing generics is cached, and most types in the typing +module are hashable and comparable for equality. + + +File: python.info, Node: The Any type, Next: Nominal vs structural subtyping, Prev: User-defined generic types, Up: typing — Support for type hints + +5.26.1.6 The ‘Any’ type +....................... + +A special kind of type is *note Any: 652. A static type checker will +treat every type as being compatible with *note Any: 652. and *note Any: +652. as being compatible with every type. + +This means that it is possible to perform any operation or method call +on a value of type *note Any: 652. and assign it to any variable: + + from typing import Any + + a = None # type: Any + a = [] # OK + a = 2 # OK + + s = '' # type: str + s = a # OK + + def foo(item: Any) -> int: + # Typechecks; 'item' could be any type, + # and that type might have a 'bar' method + item.bar() + ... + +Notice that no typechecking is performed when assigning a value of type +*note Any: 652. to a more precise type. For example, the static type +checker did not report an error when assigning ‘a’ to ‘s’ even though +‘s’ was declared to be of type *note str: 330. and receives an *note +int: 184. value at runtime! + +Furthermore, all functions without a return type or parameter types will +implicitly default to using *note Any: 652.: + + def legacy_parser(text): + ... + return data + + # A static type checker will treat the above + # as having the same signature as: + def legacy_parser(text: Any) -> Any: + ... + return data + +This behavior allows *note Any: 652. to be used as an `escape hatch' +when you need to mix dynamically and statically typed code. + +Contrast the behavior of *note Any: 652. with the behavior of *note +object: 2b0. Similar to *note Any: 652, every type is a subtype of +*note object: 2b0. However, unlike *note Any: 652, the reverse is not +true: *note object: 2b0. is `not' a subtype of every other type. + +That means when the type of a value is *note object: 2b0, a type checker +will reject almost all operations on it, and assigning it to a variable +(or using it as a return value) of a more specialized type is a type +error. For example: + + def hash_a(item: object) -> int: + # Fails; an object does not have a 'magic' method. + item.magic() + ... + + def hash_b(item: Any) -> int: + # Typechecks + item.magic() + ... + + # Typechecks, since ints and strs are subclasses of object + hash_a(42) + hash_a("foo") + + # Typechecks, since Any is compatible with all types + hash_b(42) + hash_b("foo") + +Use *note object: 2b0. to indicate that a value could be any type in a +typesafe manner. Use *note Any: 652. to indicate that a value is +dynamically typed. + + +File: python.info, Node: Nominal vs structural subtyping, Next: Classes functions and decorators, Prev: The Any type, Up: typing — Support for type hints + +5.26.1.7 Nominal vs structural subtyping +........................................ + +Initially PEP 484(1) defined Python static type system as using `nominal +subtyping'. This means that a class ‘A’ is allowed where a class ‘B’ is +expected if and only if ‘A’ is a subclass of ‘B’. + +This requirement previously also applied to abstract base classes, such +as *note Iterable: 1601. The problem with this approach is that a class +had to be explicitly marked to support them, which is unpythonic and +unlike what one would normally do in idiomatic dynamically typed Python +code. For example, this conforms to PEP 484(2): + + from typing import Sized, Iterable, Iterator + + class Bucket(Sized, Iterable[int]): + ... + def __len__(self) -> int: ... + def __iter__(self) -> Iterator[int]: ... + +PEP 544(3) allows to solve this problem by allowing users to write the +above code without explicit base classes in the class definition, +allowing ‘Bucket’ to be implicitly considered a subtype of both ‘Sized’ +and ‘Iterable[int]’ by static type checkers. This is known as +`structural subtyping' (or static duck-typing): + + from typing import Iterator, Iterable + + class Bucket: # Note: no base classes + ... + def __len__(self) -> int: ... + def __iter__(self) -> Iterator[int]: ... + + def collect(items: Iterable[int]) -> int: ... + result = collect(Bucket()) # Passes type check + +Moreover, by subclassing a special class *note Protocol: 23f, a user can +define new custom protocols to fully enjoy structural subtyping (see +examples below). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + (2) https://www.python.org/dev/peps/pep-0484 + + (3) https://www.python.org/dev/peps/pep-0544 + + +File: python.info, Node: Classes functions and decorators, Prev: Nominal vs structural subtyping, Up: typing — Support for type hints + +5.26.1.8 Classes, functions, and decorators +........................................... + +The module defines the following classes, functions and decorators: + + -- Class: typing.TypeVar + + Type variable. + + Usage: + + T = TypeVar('T') # Can be anything + A = TypeVar('A', str, bytes) # Must be str or bytes + + Type variables exist primarily for the benefit of static type + checkers. They serve as the parameters for generic types as well + as for generic function definitions. See class Generic for more + information on generic types. Generic functions work as follows: + + def repeat(x: T, n: int) -> Sequence[T]: + """Return a list containing n references to x.""" + return [x]*n + + def longest(x: A, y: A) -> A: + """Return the longest of two strings.""" + return x if len(x) >= len(y) else y + + The latter example’s signature is essentially the overloading of + ‘(str, str) -> str’ and ‘(bytes, bytes) -> bytes’. Also note that + if the arguments are instances of some subclass of *note str: 330, + the return type is still plain *note str: 330. + + At runtime, ‘isinstance(x, T)’ will raise *note TypeError: 192. In + general, *note isinstance(): 44f. and *note issubclass(): 450. + should not be used with types. + + Type variables may be marked covariant or contravariant by passing + ‘covariant=True’ or ‘contravariant=True’. See PEP 484(1) for more + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ‘bound=<type>’. + This means that an actual type substituted (explicitly or + implicitly) for the type variable must be a subclass of the + boundary type, see PEP 484(2). + + -- Class: typing.Generic + + Abstract base class for generic types. + + A generic type is typically declared by inheriting from an + instantiation of this class with one or more type variables. For + example, a generic mapping type might be defined as: + + class Mapping(Generic[KT, VT]): + def __getitem__(self, key: KT) -> VT: + ... + # Etc. + + This class can then be used as follows: + + X = TypeVar('X') + Y = TypeVar('Y') + + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: + try: + return mapping[key] + except KeyError: + return default + + -- Class: typing.Protocol (Generic) + + Base class for protocol classes. Protocol classes are defined like + this: + + class Proto(Protocol): + def meth(self) -> int: + ... + + Such classes are primarily used with static type checkers that + recognize structural subtyping (static duck-typing), for example: + + class C: + def meth(self) -> int: + return 0 + + def func(x: Proto) -> int: + return x.meth() + + func(C()) # Passes static type check + + See PEP 544(3) for details. Protocol classes decorated with *note + runtime_checkable(): 240. (described later) act as simple-minded + runtime protocols that check only the presence of given attributes, + ignoring their type signatures. + + Protocol classes can be generic, for example: + + class GenProto(Protocol[T]): + def meth(self) -> T: + ... + + New in version 3.8. + + -- Class: typing.Type (Generic[CT_co]) + + A variable annotated with ‘C’ may accept a value of type ‘C’. In + contrast, a variable annotated with ‘Type[C]’ may accept values + that are classes themselves – specifically, it will accept the + `class object' of ‘C’. For example: + + a = 3 # Has type 'int' + b = int # Has type 'Type[int]' + c = type(a) # Also has type 'Type[int]' + + Note that ‘Type[C]’ is covariant: + + class User: ... + class BasicUser(User): ... + class ProUser(User): ... + class TeamUser(User): ... + + # Accepts User, BasicUser, ProUser, TeamUser, ... + def make_new_user(user_class: Type[User]) -> User: + # ... + return user_class() + + The fact that ‘Type[C]’ is covariant implies that all subclasses of + ‘C’ should implement the same constructor signature and class + method signatures as ‘C’. The type checker should flag violations + of this, but should also allow constructor calls in subclasses that + match the constructor calls in the indicated base class. How the + type checker is required to handle this particular case may change + in future revisions of PEP 484(4). + + The only legal parameters for *note Type: 2f8a. are classes, *note + Any: 652, *note type variables: 2f84, and unions of any of these + types. For example: + + def new_non_team_user(user_class: Type[Union[BasicUser, ProUser]]): ... + + ‘Type[Any]’ is equivalent to ‘Type’ which in turn is equivalent to + ‘type’, which is the root of Python’s metaclass hierarchy. + + New in version 3.5.2. + + -- Class: typing.Iterable (Generic[T_co]) + + A generic version of *note collections.abc.Iterable: 1601. + + -- Class: typing.Iterator (Iterable[T_co]) + + A generic version of *note collections.abc.Iterator: 1602. + + -- Class: typing.Reversible (Iterable[T_co]) + + A generic version of *note collections.abc.Reversible: 52e. + + -- Class: typing.SupportsInt + + An ABC with one abstract method ‘__int__’. + + -- Class: typing.SupportsFloat + + An ABC with one abstract method ‘__float__’. + + -- Class: typing.SupportsComplex + + An ABC with one abstract method ‘__complex__’. + + -- Class: typing.SupportsBytes + + An ABC with one abstract method ‘__bytes__’. + + -- Class: typing.SupportsIndex + + An ABC with one abstract method ‘__index__’. + + New in version 3.8. + + -- Class: typing.SupportsAbs + + An ABC with one abstract method ‘__abs__’ that is covariant in its + return type. + + -- Class: typing.SupportsRound + + An ABC with one abstract method ‘__round__’ that is covariant in + its return type. + + -- Class: typing.Container (Generic[T_co]) + + A generic version of *note collections.abc.Container: 15ff. + + -- Class: typing.Hashable + + An alias to *note collections.abc.Hashable: 1600. + + -- Class: typing.Sized + + An alias to *note collections.abc.Sized: 1603. + + -- Class: typing.Collection (Sized, Iterable[T_co], Container[T_co]) + + A generic version of *note collections.abc.Collection: 52d. + + New in version 3.6.0. + + -- Class: typing.AbstractSet (Sized, Collection[T_co]) + + A generic version of *note collections.abc.Set: 1385. + + -- Class: typing.MutableSet (AbstractSet[T]) + + A generic version of *note collections.abc.MutableSet: 1606. + + -- Class: typing.Mapping (Sized, Collection[KT], Generic[VT_co]) + + A generic version of *note collections.abc.Mapping: 15f3. This + type can be used as follows: + + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] + + -- Class: typing.MutableMapping (Mapping[KT, VT]) + + A generic version of *note collections.abc.MutableMapping: 9ef. + + -- Class: typing.Sequence (Reversible[T_co], Collection[T_co]) + + A generic version of *note collections.abc.Sequence: 12db. + + -- Class: typing.MutableSequence (Sequence[T]) + + A generic version of *note collections.abc.MutableSequence: 6ac. + + -- Class: typing.ByteString (Sequence[int]) + + A generic version of *note collections.abc.ByteString: 1605. + + This type represents the types *note bytes: 331, *note bytearray: + 332, and *note memoryview: 25c. of byte sequences. + + As a shorthand for this type, *note bytes: 331. can be used to + annotate arguments of any of the types mentioned above. + + -- Class: typing.Deque (deque, MutableSequence[T]) + + A generic version of *note collections.deque: 531. + + New in version 3.5.4. + + New in version 3.6.1. + + -- Class: typing.List (list, MutableSequence[T]) + + Generic version of *note list: 262. Useful for annotating return + types. To annotate arguments it is preferred to use an abstract + collection type such as *note Sequence: 2f9a. or *note Iterable: + 2f8b. + + This type may be used as follows: + + T = TypeVar('T', int, float) + + def vec2(x: T, y: T) -> List[T]: + return [x, y] + + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] + + -- Class: typing.Set (set, MutableSet[T]) + + A generic version of *note builtins.set: b6f. Useful for + annotating return types. To annotate arguments it is preferred to + use an abstract collection type such as *note AbstractSet: 2f96. + + -- Class: typing.FrozenSet (frozenset, AbstractSet[T_co]) + + A generic version of *note builtins.frozenset: bee. + + -- Class: typing.MappingView (Sized, Iterable[T_co]) + + A generic version of *note collections.abc.MappingView: 1607. + + -- Class: typing.KeysView (MappingView[KT_co], AbstractSet[KT_co]) + + A generic version of *note collections.abc.KeysView: 1609. + + -- Class: typing.ItemsView (MappingView, Generic[KT_co, VT_co]) + + A generic version of *note collections.abc.ItemsView: 1608. + + -- Class: typing.ValuesView (MappingView[VT_co]) + + A generic version of *note collections.abc.ValuesView: 160a. + + -- Class: typing.Awaitable (Generic[T_co]) + + A generic version of *note collections.abc.Awaitable: 6b6. + + New in version 3.5.2. + + -- Class: typing.Coroutine (Awaitable[V_co], Generic[T_co, T_contra, + V_co]) + + A generic version of *note collections.abc.Coroutine: 6b7. The + variance and order of type variables correspond to those of *note + Generator: 2fa7, for example: + + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int + + New in version 3.5.3. + + -- Class: typing.AsyncIterable (Generic[T_co]) + + A generic version of *note collections.abc.AsyncIterable: 6b9. + + New in version 3.5.2. + + -- Class: typing.AsyncIterator (AsyncIterable[T_co]) + + A generic version of *note collections.abc.AsyncIterator: 6b8. + + New in version 3.5.2. + + -- Class: typing.ContextManager (Generic[T_co]) + + A generic version of *note contextlib.AbstractContextManager: 534. + + New in version 3.5.4. + + New in version 3.6.0. + + -- Class: typing.AsyncContextManager (Generic[T_co]) + + A generic version of *note contextlib.AbstractAsyncContextManager: + 378. + + New in version 3.5.4. + + New in version 3.6.2. + + -- Class: typing.Dict (dict, MutableMapping[KT, VT]) + + A generic version of *note dict: 1b8. Useful for annotating return + types. To annotate arguments it is preferred to use an abstract + collection type such as *note Mapping: 2f98. + + This type can be used as follows: + + def count_words(text: str) -> Dict[str, int]: + ... + + -- Class: typing.DefaultDict (collections.defaultdict, + MutableMapping[KT, VT]) + + A generic version of *note collections.defaultdict: b3a. + + New in version 3.5.2. + + -- Class: typing.OrderedDict (collections.OrderedDict, + MutableMapping[KT, VT]) + + A generic version of *note collections.OrderedDict: 1b9. + + New in version 3.7.2. + + -- Class: typing.Counter (collections.Counter, Dict[T, int]) + + A generic version of *note collections.Counter: 9da. + + New in version 3.5.4. + + New in version 3.6.1. + + -- Class: typing.ChainMap (collections.ChainMap, MutableMapping[KT, + VT]) + + A generic version of *note collections.ChainMap: 4a9. + + New in version 3.5.4. + + New in version 3.6.1. + + -- Class: typing.Generator (Iterator[T_co], Generic[T_co, T_contra, + V_co]) + + A generator can be annotated by the generic type + ‘Generator[YieldType, SendType, ReturnType]’. For example: + + def echo_round() -> Generator[int, float, str]: + sent = yield 0 + while sent >= 0: + sent = yield round(sent) + return 'Done' + + Note that unlike many other generics in the typing module, the + ‘SendType’ of *note Generator: 2fa7. behaves contravariantly, not + covariantly or invariantly. + + If your generator will only yield values, set the ‘SendType’ and + ‘ReturnType’ to ‘None’: + + def infinite_stream(start: int) -> Generator[int, None, None]: + while True: + yield start + start += 1 + + Alternatively, annotate your generator as having a return type of + either ‘Iterable[YieldType]’ or ‘Iterator[YieldType]’: + + def infinite_stream(start: int) -> Iterator[int]: + while True: + yield start + start += 1 + + -- Class: typing.AsyncGenerator (AsyncIterator[T_co], Generic[T_co, + T_contra]) + + An async generator can be annotated by the generic type + ‘AsyncGenerator[YieldType, SendType]’. For example: + + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded + + Unlike normal generators, async generators cannot return a value, + so there is no ‘ReturnType’ type parameter. As with *note + Generator: 2fa7, the ‘SendType’ behaves contravariantly. + + If your generator will only yield values, set the ‘SendType’ to + ‘None’: + + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) + + Alternatively, annotate your generator as having a return type of + either ‘AsyncIterable[YieldType]’ or ‘AsyncIterator[YieldType]’: + + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) + + New in version 3.6.1. + + -- Class: typing.Text + + ‘Text’ is an alias for ‘str’. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ‘Text’ is an alias + for ‘unicode’. + + Use ‘Text’ to indicate that a value must contain a unicode string + in a manner that is compatible with both Python 2 and Python 3: + + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' + + New in version 3.5.2. + + -- Class: typing.IO + -- Class: typing.TextIO + -- Class: typing.BinaryIO + + Generic type ‘IO[AnyStr]’ and its subclasses ‘TextIO(IO[str])’ and + ‘BinaryIO(IO[bytes])’ represent the types of I/O streams such as + returned by *note open(): 4f0. + + -- Class: typing.Pattern + -- Class: typing.Match + + These type aliases correspond to the return types from *note + re.compile(): 44a. and *note re.match(): bb3. These types (and the + corresponding functions) are generic in ‘AnyStr’ and can be made + specific by writing ‘Pattern[str]’, ‘Pattern[bytes]’, ‘Match[str]’, + or ‘Match[bytes]’. + + -- Class: typing.NamedTuple + + Typed version of *note collections.namedtuple(): 1b7. + + Usage: + + class Employee(NamedTuple): + name: str + id: int + + This is equivalent to: + + Employee = collections.namedtuple('Employee', ['name', 'id']) + + To give a field a default value, you can assign to it in the class + body: + + class Employee(NamedTuple): + name: str + id: int = 3 + + employee = Employee('Guido') + assert employee.id == 3 + + Fields with a default value must come after any fields without a + default. + + The resulting class has an extra attribute ‘__annotations__’ giving + a dict that maps the field names to the field types. (The field + names are in the ‘_fields’ attribute and the default values are in + the ‘_field_defaults’ attribute both of which are part of the + namedtuple API.) + + ‘NamedTuple’ subclasses can also have docstrings and methods: + + class Employee(NamedTuple): + """Represents an employee.""" + name: str + id: int = 3 + + def __repr__(self) -> str: + return f'<Employee {self.name}, id={self.id}>' + + Backward-compatible usage: + + Employee = NamedTuple('Employee', [('name', str), ('id', int)]) + + Changed in version 3.6: Added support for PEP 526(5) variable + annotation syntax. + + Changed in version 3.6.1: Added support for default values, + methods, and docstrings. + + Deprecated since version 3.8, will be removed in version 3.9: + Deprecated the ‘_field_types’ attribute in favor of the more + standard ‘__annotations__’ attribute which has the same + information. + + Changed in version 3.8: The ‘_field_types’ and ‘__annotations__’ + attributes are now regular dictionaries instead of instances of + ‘OrderedDict’. + + -- Class: typing.TypedDict (dict) + + A simple typed namespace. At runtime it is equivalent to a plain + *note dict: 1b8. + + ‘TypedDict’ creates a dictionary type that expects all of its + instances to have a certain set of keys, where each key is + associated with a value of a consistent type. This expectation is + not checked at runtime but is only enforced by type checkers. + Usage: + + class Point2D(TypedDict): + x: int + y: int + label: str + + a: Point2D = {'x': 1, 'y': 2, 'label': 'good'} # OK + b: Point2D = {'z': 3, 'label': 'bad'} # Fails type check + + assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first') + + The type info for introspection can be accessed via + ‘Point2D.__annotations__’ and ‘Point2D.__total__’. To allow using + this feature with older versions of Python that do not support PEP + 526(6), ‘TypedDict’ supports two additional equivalent syntactic + forms: + + Point2D = TypedDict('Point2D', x=int, y=int, label=str) + Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) + + By default, all keys must be present in a TypedDict. It is + possible to override this by specifying totality. Usage: + + class point2D(TypedDict, total=False): + x: int + y: int + + This means that a point2D TypedDict can have any of the keys + omitted. A type checker is only expected to support a literal + False or True as the value of the total argument. True is the + default, and makes all items defined in the class body be required. + + See PEP 589(7) for more examples and detailed rules of using + ‘TypedDict’. + + New in version 3.8. + + -- Class: typing.ForwardRef + + A class used for internal typing representation of string forward + references. For example, ‘List["SomeClass"]’ is implicitly + transformed into ‘List[ForwardRef("SomeClass")]’. This class + should not be instantiated by a user, but may be used by + introspection tools. + + New in version 3.7.4. + + -- Function: typing.NewType (name, tp) + + A helper function to indicate a distinct type to a typechecker, see + *note NewType: 2f81. At runtime it returns a function that returns + its argument. Usage: + + UserId = NewType('UserId', int) + first_user = UserId(1) + + New in version 3.5.2. + + -- Function: typing.cast (typ, val) + + Cast a value to a type. + + This returns the value unchanged. To the type checker this signals + that the return value has the designated type, but at runtime we + intentionally don’t check anything (we want this to be as fast as + possible). + + -- Function: typing.get_type_hints (obj[, globals[, locals]]) + + Return a dictionary containing type hints for a function, method, + module or class object. + + This is often the same as ‘obj.__annotations__’. In addition, + forward references encoded as string literals are handled by + evaluating them in ‘globals’ and ‘locals’ namespaces. If + necessary, ‘Optional[t]’ is added for function and method + annotations if a default value equal to ‘None’ is set. For a class + ‘C’, return a dictionary constructed by merging all the + ‘__annotations__’ along ‘C.__mro__’ in reverse order. + + -- Function: typing.get_origin (tp) + + -- Function: typing.get_args (tp) + + Provide basic introspection for generic types and special typing + forms. + + For a typing object of the form ‘X[Y, Z, ...]’ these functions + return ‘X’ and ‘(Y, Z, ...)’. If ‘X’ is a generic alias for a + builtin or *note collections: 1e. class, it gets normalized to the + original class. For unsupported objects return ‘None’ and ‘()’ + correspondingly. Examples: + + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) + + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) + + New in version 3.8. + + -- Function: @typing.overload + + The ‘@overload’ decorator allows describing functions and methods + that support multiple different combinations of argument types. A + series of ‘@overload’-decorated definitions must be followed by + exactly one non-‘@overload’-decorated definition (for the same + function/method). The ‘@overload’-decorated definitions are for + the benefit of the type checker only, since they will be + overwritten by the non-‘@overload’-decorated definition, while the + latter is used at runtime but should be ignored by a type checker. + At runtime, calling a ‘@overload’-decorated function directly will + raise *note NotImplementedError: 60e. An example of overload that + gives a more precise type than can be expressed using a union or a + type variable: + + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + <actual implementation> + + See PEP 484(8) for details and comparison with other typing + semantics. + + -- Function: @typing.final + + A decorator to indicate to type checkers that the decorated method + cannot be overridden, and the decorated class cannot be subclassed. + For example: + + class Base: + @final + def done(self) -> None: + ... + class Sub(Base): + def done(self) -> None: # Error reported by type checker + ... + + @final + class Leaf: + ... + class Other(Leaf): # Error reported by type checker + ... + + There is no runtime checking of these properties. See PEP 591(9) + for more details. + + New in version 3.8. + + -- Function: @typing.no_type_check + + Decorator to indicate that annotations are not type hints. + + This works as class or function *note decorator: 283. With a + class, it applies recursively to all methods defined in that class + (but not to methods defined in its superclasses or subclasses). + + This mutates the function(s) in place. + + -- Function: @typing.no_type_check_decorator + + Decorator to give another decorator the *note no_type_check(): + 2fba. effect. + + This wraps the decorator with something that wraps the decorated + function in *note no_type_check(): 2fba. + + -- Function: @typing.type_check_only + + Decorator to mark a class or function to be unavailable at runtime. + + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if an + implementation returns an instance of a private class: + + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... + + def fetch_response() -> Response: ... + + Note that returning instances of private classes is not + recommended. It is usually preferable to make such classes public. + + -- Function: @typing.runtime_checkable + + Mark a protocol class as a runtime protocol. + + Such a protocol can be used with *note isinstance(): 44f. and *note + issubclass(): 450. This raises *note TypeError: 192. when applied + to a non-protocol class. This allows a simple-minded structural + check, very similar to “one trick ponies” in *note collections.abc: + 1f. such as *note Iterable: 1601. For example: + + @runtime_checkable + class Closable(Protocol): + def close(self): ... + + assert isinstance(open('/some/file'), Closable) + + `Warning:' this will check only the presence of the required + methods, not their type signatures! + + New in version 3.8. + + -- Data: typing.Any + + Special type indicating an unconstrained type. + + * Every type is compatible with *note Any: 652. + + * *note Any: 652. is compatible with every type. + + -- Data: typing.NoReturn + + Special type indicating that a function never returns. For + example: + + from typing import NoReturn + + def stop() -> NoReturn: + raise RuntimeError('no way') + + New in version 3.5.4. + + New in version 3.6.2. + + -- Data: typing.Union + + Union type; ‘Union[X, Y]’ means either X or Y. + + To define a union, use e.g. ‘Union[int, str]’. Details: + + * The arguments must be types and there must be at least one. + + * Unions of unions are flattened, e.g.: + + Union[Union[int, str], float] == Union[int, str, float] + + * Unions of a single argument vanish, e.g.: + + Union[int] == int # The constructor actually returns int + + * Redundant arguments are skipped, e.g.: + + Union[int, str, int] == Union[int, str] + + * When comparing unions, the argument order is ignored, e.g.: + + Union[int, str] == Union[str, int] + + * You cannot subclass or instantiate a union. + + * You cannot write ‘Union[X][Y]’. + + * You can use ‘Optional[X]’ as a shorthand for ‘Union[X, None]’. + + Changed in version 3.7: Don’t remove explicit subclasses from + unions at runtime. + + -- Data: typing.Optional + + Optional type. + + ‘Optional[X]’ is equivalent to ‘Union[X, None]’. + + Note that this is not the same concept as an optional argument, + which is one that has a default. An optional argument with a + default does not require the ‘Optional’ qualifier on its type + annotation just because it is optional. For example: + + def foo(arg: int = 0) -> None: + ... + + On the other hand, if an explicit value of ‘None’ is allowed, the + use of ‘Optional’ is appropriate, whether the argument is optional + or not. For example: + + def foo(arg: Optional[int] = None) -> None: + ... + + -- Data: typing.Tuple + + Tuple type; ‘Tuple[X, Y]’ is the type of a tuple of two items with + the first item of type X and the second of type Y. The type of the + empty tuple can be written as ‘Tuple[()]’. + + Example: ‘Tuple[T1, T2]’ is a tuple of two elements corresponding + to type variables T1 and T2. ‘Tuple[int, float, str]’ is a tuple + of an int, a float and a string. + + To specify a variable-length tuple of homogeneous type, use literal + ellipsis, e.g. ‘Tuple[int, ...]’. A plain *note Tuple: 2f7c. is + equivalent to ‘Tuple[Any, ...]’, and in turn to *note tuple: 47e. + + -- Data: typing.Callable + + Callable type; ‘Callable[[int], str]’ is a function of (int) -> + str. + + The subscription syntax must always be used with exactly two + values: the argument list and the return type. The argument list + must be a list of types or an ellipsis; the return type must be a + single type. + + There is no syntax to indicate optional or keyword arguments; such + function types are rarely used as callback types. ‘Callable[..., + ReturnType]’ (literal ellipsis) can be used to type hint a callable + taking any number of arguments and returning ‘ReturnType’. A plain + *note Callable: 2f7d. is equivalent to ‘Callable[..., Any]’, and in + turn to *note collections.abc.Callable: 1604. + + -- Data: typing.Literal + + A type that can be used to indicate to type checkers that the + corresponding variable or function parameter has a value equivalent + to the provided literal (or one of several literals). For example: + + def validate_simple(data: Any) -> Literal[True]: # always returns True + ... + + MODE = Literal['r', 'rb', 'w', 'wb'] + def open_helper(file: str, mode: MODE) -> str: + ... + + open_helper('/some/path', 'r') # Passes type check + open_helper('/other/path', 'typo') # Error in type checker + + ‘Literal[...]’ cannot be subclassed. At runtime, an arbitrary + value is allowed as type argument to ‘Literal[...]’, but type + checkers may impose restrictions. See PEP 586(10) for more details + about literal types. + + New in version 3.8. + + -- Data: typing.ClassVar + + Special type construct to mark class variables. + + As introduced in PEP 526(11), a variable annotation wrapped in + ClassVar indicates that a given attribute is intended to be used as + a class variable and should not be set on instances of that class. + Usage: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + *note ClassVar: 5a3. accepts only types and cannot be further + subscribed. + + *note ClassVar: 5a3. is not a class itself, and should not be used + with *note isinstance(): 44f. or *note issubclass(): 450. *note + ClassVar: 5a3. does not change Python runtime behavior, but it can + be used by third-party type checkers. For example, a type checker + might flag the following code as an error: + + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK + + New in version 3.5.3. + + -- Data: typing.Final + + A special typing construct to indicate to type checkers that a name + cannot be re-assigned or overridden in a subclass. For example: + + MAX_SIZE: Final = 9000 + MAX_SIZE += 1 # Error reported by type checker + + class Connection: + TIMEOUT: Final[int] = 10 + + class FastConnector(Connection): + TIMEOUT = 1 # Error reported by type checker + + There is no runtime checking of these properties. See PEP 591(12) + for more details. + + New in version 3.8. + + -- Data: typing.AnyStr + + ‘AnyStr’ is a type variable defined as ‘AnyStr = TypeVar('AnyStr', + str, bytes)’. + + It is meant to be used for functions that may accept any kind of + string without allowing different kinds of strings to mix. For + example: + + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b + + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + + -- Data: typing.TYPE_CHECKING + + A special constant that is assumed to be ‘True’ by 3rd party static + type checkers. It is ‘False’ at runtime. Usage: + + if TYPE_CHECKING: + import expensive_mod + + def fun(arg: 'expensive_mod.SomeType') -> None: + local_var: expensive_mod.AnotherType = other_fun() + + Note that the first type annotation must be enclosed in quotes, + making it a “forward reference”, to hide the ‘expensive_mod’ + reference from the interpreter runtime. Type annotations for local + variables are not evaluated, so the second annotation does not need + to be enclosed in quotes. + + New in version 3.5.2. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + (2) https://www.python.org/dev/peps/pep-0484 + + (3) https://www.python.org/dev/peps/pep-0544 + + (4) https://www.python.org/dev/peps/pep-0484 + + (5) https://www.python.org/dev/peps/pep-0526 + + (6) https://www.python.org/dev/peps/pep-0526 + + (7) https://www.python.org/dev/peps/pep-0589 + + (8) https://www.python.org/dev/peps/pep-0484 + + (9) https://www.python.org/dev/peps/pep-0591 + + (10) https://www.python.org/dev/peps/pep-0586 + + (11) https://www.python.org/dev/peps/pep-0526 + + (12) https://www.python.org/dev/peps/pep-0591 + + +File: python.info, Node: pydoc — Documentation generator and online help system, Next: doctest — Test interactive Python examples, Prev: typing — Support for type hints, Up: Development Tools + +5.26.2 ‘pydoc’ — Documentation generator and online help system +--------------------------------------------------------------- + +`Source code:' Lib/pydoc.py(1) + +__________________________________________________________________ + +The *note pydoc: d9. module automatically generates documentation from +Python modules. The documentation can be presented as pages of text on +the console, served to a Web browser, or saved to HTML files. + +For modules, classes, functions and methods, the displayed documentation +is derived from the docstring (i.e. the ‘__doc__’ attribute) of the +object, and recursively of its documentable members. If there is no +docstring, *note pydoc: d9. tries to obtain a description from the block +of comment lines just above the definition of the class, function or +method in the source file, or at the top of the module (see *note +inspect.getcomments(): 2fc2.). + +The built-in function *note help(): 572. invokes the online help system +in the interactive interpreter, which uses *note pydoc: d9. to generate +its documentation as text on the console. The same text documentation +can also be viewed from outside the Python interpreter by running +‘pydoc’ as a script at the operating system’s command prompt. For +example, running + + pydoc sys + +at a shell prompt will display documentation on the *note sys: fd. +module, in a style similar to the manual pages shown by the Unix ‘man’ +command. The argument to ‘pydoc’ can be the name of a function, module, +or package, or a dotted reference to a class, method, or function within +a module or module in a package. If the argument to ‘pydoc’ looks like +a path (that is, it contains the path separator for your operating +system, such as a slash in Unix), and refers to an existing Python +source file, then documentation is produced for that file. + + Note: In order to find objects and their documentation, *note + pydoc: d9. imports the module(s) to be documented. Therefore, any + code on module level will be executed on that occasion. Use an ‘if + __name__ == '__main__':’ guard to only execute code when a file is + invoked as a script and not just imported. + +When printing output to the console, ‘pydoc’ attempts to paginate the +output for easier reading. If the ‘PAGER’ environment variable is set, +‘pydoc’ will use its value as a pagination program. + +Specifying a ‘-w’ flag before the argument will cause HTML documentation +to be written out to a file in the current directory, instead of +displaying text on the console. + +Specifying a ‘-k’ flag before the argument will search the synopsis +lines of all available modules for the keyword given as the argument, +again in a manner similar to the Unix ‘man’ command. The synopsis line +of a module is the first line of its documentation string. + +You can also use ‘pydoc’ to start an HTTP server on the local machine +that will serve documentation to visiting Web browsers. ‘pydoc -p 1234’ +will start a HTTP server on port 1234, allowing you to browse the +documentation at ‘http://localhost:1234/’ in your preferred Web browser. +Specifying ‘0’ as the port number will select an arbitrary unused port. + +‘pydoc -n <hostname>’ will start the server listening at the given +hostname. By default the hostname is ‘localhost’ but if you want the +server to be reached from other machines, you may want to change the +host name that the server responds to. During development this is +especially useful if you want to run pydoc from within a container. + +‘pydoc -b’ will start the server and additionally open a web browser to +a module index page. Each served page has a navigation bar at the top +where you can `Get' help on an individual item, `Search' all modules +with a keyword in their synopsis line, and go to the `Module index', +`Topics' and `Keywords' pages. + +When ‘pydoc’ generates documentation, it uses the current environment +and path to locate modules. Thus, invoking ‘pydoc spam’ documents +precisely the version of the module you would get if you started the +Python interpreter and typed ‘import spam’. + +Module docs for core modules are assumed to reside in +‘https://docs.python.org/X.Y/library/’ where ‘X’ and ‘Y’ are the major +and minor version numbers of the Python interpreter. This can be +overridden by setting the ‘PYTHONDOCS’ environment variable to a +different URL or to a local directory containing the Library Reference +Manual pages. + +Changed in version 3.2: Added the ‘-b’ option. + +Changed in version 3.3: The ‘-g’ command line option was removed. + +Changed in version 3.4: *note pydoc: d9. now uses *note +inspect.signature(): 55b. rather than *note inspect.getfullargspec(): +55d. to extract signature information from callables. + +Changed in version 3.7: Added the ‘-n’ option. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pydoc.py + + +File: python.info, Node: doctest — Test interactive Python examples, Next: unittest — Unit testing framework, Prev: pydoc — Documentation generator and online help system, Up: Development Tools + +5.26.3 ‘doctest’ — Test interactive Python examples +--------------------------------------------------- + +`Source code:' Lib/doctest.py(1) + +__________________________________________________________________ + +The *note doctest: 67. module searches for pieces of text that look like +interactive Python sessions, and then executes those sessions to verify +that they work exactly as shown. There are several common ways to use +doctest: + + * To check that a module’s docstrings are up-to-date by verifying + that all interactive examples still work as documented. + + * To perform regression testing by verifying that interactive + examples from a test file or a test object work as expected. + + * To write tutorial documentation for a package, liberally + illustrated with input-output examples. Depending on whether the + examples or the expository text are emphasized, this has the flavor + of “literate testing” or “executable documentation”. + +Here’s a complete but small example module: + + """ + This is the "example" module. + + The example module supplies one function, factorial(). For example, + + >>> factorial(5) + 120 + """ + + def factorial(n): + """Return the factorial of n, an exact integer >= 0. + + >>> [factorial(n) for n in range(6)] + [1, 1, 2, 6, 24, 120] + >>> factorial(30) + 265252859812191058636308480000000 + >>> factorial(-1) + Traceback (most recent call last): + ... + ValueError: n must be >= 0 + + Factorials of floats are OK, but the float must be an exact integer: + >>> factorial(30.1) + Traceback (most recent call last): + ... + ValueError: n must be exact integer + >>> factorial(30.0) + 265252859812191058636308480000000 + + It must also not be ridiculously large: + >>> factorial(1e100) + Traceback (most recent call last): + ... + OverflowError: n too large + """ + + import math + if not n >= 0: + raise ValueError("n must be >= 0") + if math.floor(n) != n: + raise ValueError("n must be exact integer") + if n+1 == n: # catch a value like 1e300 + raise OverflowError("n too large") + result = 1 + factor = 2 + while factor <= n: + result *= factor + factor += 1 + return result + + + if __name__ == "__main__": + import doctest + doctest.testmod() + +If you run ‘example.py’ directly from the command line, *note doctest: +67. works its magic: + + $ python example.py + $ + +There’s no output! That’s normal, and it means all the examples worked. +Pass ‘-v’ to the script, and *note doctest: 67. prints a detailed log of +what it’s trying, and prints a summary at the end: + + $ python example.py -v + Trying: + factorial(5) + Expecting: + 120 + ok + Trying: + [factorial(n) for n in range(6)] + Expecting: + [1, 1, 2, 6, 24, 120] + ok + +And so on, eventually ending with: + + Trying: + factorial(1e100) + Expecting: + Traceback (most recent call last): + ... + OverflowError: n too large + ok + 2 items passed all tests: + 1 tests in __main__ + 8 tests in __main__.factorial + 9 tests in 2 items. + 9 passed and 0 failed. + Test passed. + $ + +That’s all you need to know to start making productive use of *note +doctest: 67.! Jump in. The following sections provide full details. +Note that there are many examples of doctests in the standard Python +test suite and libraries. Especially useful examples can be found in +the standard test file ‘Lib/test/test_doctest.py’. + +* Menu: + +* Simple Usage; Checking Examples in Docstrings: Simple Usage Checking Examples in Docstrings. +* Simple Usage; Checking Examples in a Text File: Simple Usage Checking Examples in a Text File. +* How It Works:: +* Basic API:: +* Unittest API:: +* Advanced API:: +* Debugging:: +* Soapbox:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/doctest.py + + +File: python.info, Node: Simple Usage Checking Examples in Docstrings, Next: Simple Usage Checking Examples in a Text File, Up: doctest — Test interactive Python examples + +5.26.3.1 Simple Usage: Checking Examples in Docstrings +...................................................... + +The simplest way to start using doctest (but not necessarily the way +you’ll continue to do it) is to end each module ‘M’ with: + + if __name__ == "__main__": + import doctest + doctest.testmod() + +*note doctest: 67. then examines docstrings in module ‘M’. + +Running the module as a script causes the examples in the docstrings to +get executed and verified: + + python M.py + +This won’t display anything unless an example fails, in which case the +failing example(s) and the cause(s) of the failure(s) are printed to +stdout, and the final line of output is ‘***Test Failed*** N failures.’, +where `N' is the number of examples that failed. + +Run it with the ‘-v’ switch instead: + + python M.py -v + +and a detailed report of all examples tried is printed to standard +output, along with assorted summaries at the end. + +You can force verbose mode by passing ‘verbose=True’ to *note testmod(): +dd0, or prohibit it by passing ‘verbose=False’. In either of those +cases, ‘sys.argv’ is not examined by *note testmod(): dd0. (so passing +‘-v’ or not has no effect). + +There is also a command line shortcut for running *note testmod(): dd0. +You can instruct the Python interpreter to run the doctest module +directly from the standard library and pass the module name(s) on the +command line: + + python -m doctest -v example.py + +This will import ‘example.py’ as a standalone module and run *note +testmod(): dd0. on it. Note that this may not work correctly if the +file is part of a package and imports other submodules from that +package. + +For more information on *note testmod(): dd0, see section *note Basic +API: 2fc7. + + +File: python.info, Node: Simple Usage Checking Examples in a Text File, Next: How It Works, Prev: Simple Usage Checking Examples in Docstrings, Up: doctest — Test interactive Python examples + +5.26.3.2 Simple Usage: Checking Examples in a Text File +....................................................... + +Another simple application of doctest is testing interactive examples in +a text file. This can be done with the *note testfile(): 2fca. +function: + + import doctest + doctest.testfile("example.txt") + +That short script executes and verifies any interactive Python examples +contained in the file ‘example.txt’. The file content is treated as if +it were a single giant docstring; the file doesn’t need to contain a +Python program! For example, perhaps ‘example.txt’ contains this: + + The ``example`` module + ====================== + + Using ``factorial`` + ------------------- + + This is an example text file in reStructuredText format. First import + ``factorial`` from the ``example`` module: + + >>> from example import factorial + + Now use it: + + >>> factorial(6) + 120 + +Running ‘doctest.testfile("example.txt")’ then finds the error in this +documentation: + + File "./example.txt", line 14, in example.txt + Failed example: + factorial(6) + Expected: + 120 + Got: + 720 + +As with *note testmod(): dd0, *note testfile(): 2fca. won’t display +anything unless an example fails. If an example does fail, then the +failing example(s) and the cause(s) of the failure(s) are printed to +stdout, using the same format as *note testmod(): dd0. + +By default, *note testfile(): 2fca. looks for files in the calling +module’s directory. See section *note Basic API: 2fc7. for a +description of the optional arguments that can be used to tell it to +look for files in other locations. + +Like *note testmod(): dd0, *note testfile(): 2fca.’s verbosity can be +set with the ‘-v’ command-line switch or with the optional keyword +argument `verbose'. + +There is also a command line shortcut for running *note testfile(): +2fca. You can instruct the Python interpreter to run the doctest module +directly from the standard library and pass the file name(s) on the +command line: + + python -m doctest -v example.txt + +Because the file name does not end with ‘.py’, *note doctest: 67. infers +that it must be run with *note testfile(): 2fca, not *note testmod(): +dd0. + +For more information on *note testfile(): 2fca, see section *note Basic +API: 2fc7. + + +File: python.info, Node: How It Works, Next: Basic API, Prev: Simple Usage Checking Examples in a Text File, Up: doctest — Test interactive Python examples + +5.26.3.3 How It Works +..................... + +This section examines in detail how doctest works: which docstrings it +looks at, how it finds interactive examples, what execution context it +uses, how it handles exceptions, and how option flags can be used to +control its behavior. This is the information that you need to know to +write doctest examples; for information about actually running doctest +on these examples, see the following sections. + +* Menu: + +* Which Docstrings Are Examined?:: +* How are Docstring Examples Recognized?:: +* What’s the Execution Context?:: +* What About Exceptions?:: +* Option Flags:: +* Directives:: +* Warnings: Warnings<2>. + + +File: python.info, Node: Which Docstrings Are Examined?, Next: How are Docstring Examples Recognized?, Up: How It Works + +5.26.3.4 Which Docstrings Are Examined? +....................................... + +The module docstring, and all function, class and method docstrings are +searched. Objects imported into the module are not searched. + +In addition, if ‘M.__test__’ exists and “is true”, it must be a dict, +and each entry maps a (string) name to a function object, class object, +or string. Function and class object docstrings found from ‘M.__test__’ +are searched, and strings are treated as if they were docstrings. In +output, a key ‘K’ in ‘M.__test__’ appears with name + + <name of M>.__test__.K + +Any classes found are recursively searched similarly, to test docstrings +in their contained methods and nested classes. + +`CPython implementation detail:' Prior to version 3.4, extension modules +written in C were not fully searched by doctest. + + +File: python.info, Node: How are Docstring Examples Recognized?, Next: What’s the Execution Context?, Prev: Which Docstrings Are Examined?, Up: How It Works + +5.26.3.5 How are Docstring Examples Recognized? +............................................... + +In most cases a copy-and-paste of an interactive console session works +fine, but doctest isn’t trying to do an exact emulation of any specific +Python shell. + + >>> # comments are ignored + >>> x = 12 + >>> x + 12 + >>> if x == 13: + ... print("yes") + ... else: + ... print("no") + ... print("NO") + ... print("NO!!!") + ... + no + NO + NO!!! + >>> + +Any expected output must immediately follow the final ‘'>>> '’ or ‘'... +'’ line containing the code, and the expected output (if any) extends to +the next ‘'>>> '’ or all-whitespace line. + +The fine print: + + * Expected output cannot contain an all-whitespace line, since such a + line is taken to signal the end of expected output. If expected + output does contain a blank line, put ‘<BLANKLINE>’ in your doctest + example each place a blank line is expected. + + * All hard tab characters are expanded to spaces, using 8-column tab + stops. Tabs in output generated by the tested code are not + modified. Because any hard tabs in the sample output `are' + expanded, this means that if the code output includes hard tabs, + the only way the doctest can pass is if the *note + NORMALIZE_WHITESPACE: 2fd1. option or *note directive: 2fd2. is in + effect. Alternatively, the test can be rewritten to capture the + output and compare it to an expected value as part of the test. + This handling of tabs in the source was arrived at through trial + and error, and has proven to be the least error prone way of + handling them. It is possible to use a different algorithm for + handling tabs by writing a custom *note DocTestParser: 2fd3. class. + + * Output to stdout is captured, but not output to stderr (exception + tracebacks are captured via a different means). + + * If you continue a line via backslashing in an interactive session, + or for any other reason use a backslash, you should use a raw + docstring, which will preserve your backslashes exactly as you type + them: + + >>> def f(x): + ... r'''Backslashes in a raw docstring: m\n''' + >>> print(f.__doc__) + Backslashes in a raw docstring: m\n + + Otherwise, the backslash will be interpreted as part of the string. + For example, the ‘\n’ above would be interpreted as a newline + character. Alternatively, you can double each backslash in the + doctest version (and not use a raw string): + + >>> def f(x): + ... '''Backslashes in a raw docstring: m\\n''' + >>> print(f.__doc__) + Backslashes in a raw docstring: m\n + + * The starting column doesn’t matter: + + >>> assert "Easy!" + >>> import math + >>> math.floor(1.9) + 1 + + and as many leading whitespace characters are stripped from the + expected output as appeared in the initial ‘'>>> '’ line that + started the example. + + +File: python.info, Node: What’s the Execution Context?, Next: What About Exceptions?, Prev: How are Docstring Examples Recognized?, Up: How It Works + +5.26.3.6 What’s the Execution Context? +...................................... + +By default, each time *note doctest: 67. finds a docstring to test, it +uses a `shallow copy' of ‘M’’s globals, so that running tests doesn’t +change the module’s real globals, and so that one test in ‘M’ can’t +leave behind crumbs that accidentally allow another test to work. This +means examples can freely use any names defined at top-level in ‘M’, and +names defined earlier in the docstring being run. Examples cannot see +names defined in other docstrings. + +You can force use of your own dict as the execution context by passing +‘globs=your_dict’ to *note testmod(): dd0. or *note testfile(): 2fca. +instead. + + +File: python.info, Node: What About Exceptions?, Next: Option Flags, Prev: What’s the Execution Context?, Up: How It Works + +5.26.3.7 What About Exceptions? +............................... + +No problem, provided that the traceback is the only output produced by +the example: just paste in the traceback. (1) Since tracebacks contain +details that are likely to change rapidly (for example, exact file paths +and line numbers), this is one case where doctest works hard to be +flexible in what it accepts. + +Simple example: + + >>> [1, 2, 3].remove(42) + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + ValueError: list.remove(x): x not in list + +That doctest succeeds if *note ValueError: 1fb. is raised, with the +‘list.remove(x): x not in list’ detail as shown. + +The expected output for an exception must start with a traceback header, +which may be either of the following two lines, indented the same as the +first line of the example: + + Traceback (most recent call last): + Traceback (innermost last): + +The traceback header is followed by an optional traceback stack, whose +contents are ignored by doctest. The traceback stack is typically +omitted, or copied verbatim from an interactive session. + +The traceback stack is followed by the most interesting part: the +line(s) containing the exception type and detail. This is usually the +last line of a traceback, but can extend across multiple lines if the +exception has a multi-line detail: + + >>> raise ValueError('multi\n line\ndetail') + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + ValueError: multi + line + detail + +The last three lines (starting with *note ValueError: 1fb.) are compared +against the exception’s type and detail, and the rest are ignored. + +Best practice is to omit the traceback stack, unless it adds significant +documentation value to the example. So the last example is probably +better as: + + >>> raise ValueError('multi\n line\ndetail') + Traceback (most recent call last): + ... + ValueError: multi + line + detail + +Note that tracebacks are treated very specially. In particular, in the +rewritten example, the use of ‘...’ is independent of doctest’s *note +ELLIPSIS: dd1. option. The ellipsis in that example could be left out, +or could just as well be three (or three hundred) commas or digits, or +an indented transcript of a Monty Python skit. + +Some details you should read once, but won’t need to remember: + + * Doctest can’t guess whether your expected output came from an + exception traceback or from ordinary printing. So, e.g., an + example that expects ‘ValueError: 42 is prime’ will pass whether + *note ValueError: 1fb. is actually raised or if the example merely + prints that traceback text. In practice, ordinary output rarely + begins with a traceback header line, so this doesn’t create real + problems. + + * Each line of the traceback stack (if present) must be indented + further than the first line of the example, `or' start with a + non-alphanumeric character. The first line following the traceback + header indented the same and starting with an alphanumeric is taken + to be the start of the exception detail. Of course this does the + right thing for genuine tracebacks. + + * When the *note IGNORE_EXCEPTION_DETAIL: 2fd8. doctest option is + specified, everything following the leftmost colon and any module + information in the exception name is ignored. + + * The interactive shell omits the traceback header line for some + *note SyntaxError: 458.s. But doctest uses the traceback header + line to distinguish exceptions from non-exceptions. So in the rare + case where you need to test a *note SyntaxError: 458. that omits + the traceback header, you will need to manually add the traceback + header line to your test example. + + * For some *note SyntaxError: 458.s, Python displays the character + position of the syntax error, using a ‘^’ marker: + + >>> 1 1 + File "<stdin>", line 1 + 1 1 + ^ + SyntaxError: invalid syntax + + Since the lines showing the position of the error come before the + exception type and detail, they are not checked by doctest. For + example, the following test would pass, even though it puts the ‘^’ + marker in the wrong location: + + >>> 1 1 + Traceback (most recent call last): + File "<stdin>", line 1 + 1 1 + ^ + SyntaxError: invalid syntax + + ---------- Footnotes ---------- + + (1) (1) Examples containing both expected output and an exception are +not supported. Trying to guess where one ends and the other begins is +too error-prone, and that also makes for a confusing test. + + +File: python.info, Node: Option Flags, Next: Directives, Prev: What About Exceptions?, Up: How It Works + +5.26.3.8 Option Flags +..................... + +A number of option flags control various aspects of doctest’s behavior. +Symbolic names for the flags are supplied as module constants, which can +be *note bitwise ORed: 11e4. together and passed to various functions. +The names can also be used in *note doctest directives: 2fd2, and may be +passed to the doctest command line interface via the ‘-o’ option. + +New in version 3.4: The ‘-o’ command line option. + +The first group of options define test semantics, controlling aspects of +how doctest decides whether actual output matches an example’s expected +output: + + -- Data: doctest.DONT_ACCEPT_TRUE_FOR_1 + + By default, if an expected output block contains just ‘1’, an + actual output block containing just ‘1’ or just ‘True’ is + considered to be a match, and similarly for ‘0’ versus ‘False’. + When *note DONT_ACCEPT_TRUE_FOR_1: 2fdb. is specified, neither + substitution is allowed. The default behavior caters to that + Python changed the return type of many functions from integer to + boolean; doctests expecting “little integer” output still work in + these cases. This option will probably go away, but not for + several years. + + -- Data: doctest.DONT_ACCEPT_BLANKLINE + + By default, if an expected output block contains a line containing + only the string ‘<BLANKLINE>’, then that line will match a blank + line in the actual output. Because a genuinely blank line delimits + the expected output, this is the only way to communicate that a + blank line is expected. When *note DONT_ACCEPT_BLANKLINE: 2fdc. is + specified, this substitution is not allowed. + + -- Data: doctest.NORMALIZE_WHITESPACE + + When specified, all sequences of whitespace (blanks and newlines) + are treated as equal. Any sequence of whitespace within the + expected output will match any sequence of whitespace within the + actual output. By default, whitespace must match exactly. *note + NORMALIZE_WHITESPACE: 2fd1. is especially useful when a line of + expected output is very long, and you want to wrap it across + multiple lines in your source. + + -- Data: doctest.ELLIPSIS + + When specified, an ellipsis marker (‘...’) in the expected output + can match any substring in the actual output. This includes + substrings that span line boundaries, and empty substrings, so it’s + best to keep usage of this simple. Complicated uses can lead to + the same kinds of “oops, it matched too much!” surprises that ‘.*’ + is prone to in regular expressions. + + -- Data: doctest.IGNORE_EXCEPTION_DETAIL + + When specified, an example that expects an exception passes if an + exception of the expected type is raised, even if the exception + detail does not match. For example, an example expecting + ‘ValueError: 42’ will pass if the actual exception raised is + ‘ValueError: 3*14’, but will fail, e.g., if *note TypeError: 192. + is raised. + + It will also ignore the module name used in Python 3 doctest + reports. Hence both of these variations will work with the flag + specified, regardless of whether the test is run under Python 2.7 + or Python 3.2 (or later versions): + + >>> raise CustomError('message') + Traceback (most recent call last): + CustomError: message + + >>> raise CustomError('message') + Traceback (most recent call last): + my_module.CustomError: message + + Note that *note ELLIPSIS: dd1. can also be used to ignore the + details of the exception message, but such a test may still fail + based on whether or not the module details are printed as part of + the exception name. Using *note IGNORE_EXCEPTION_DETAIL: 2fd8. and + the details from Python 2.3 is also the only clear way to write a + doctest that doesn’t care about the exception detail yet continues + to pass under Python 2.3 or earlier (those releases do not support + *note doctest directives: 2fd2. and ignore them as irrelevant + comments). For example: + + >>> (1, 2)[3] = 'moo' + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + TypeError: object doesn't support item assignment + + passes under Python 2.3 and later Python versions with the flag + specified, even though the detail changed in Python 2.4 to say + “does not” instead of “doesn’t”. + + Changed in version 3.2: *note IGNORE_EXCEPTION_DETAIL: 2fd8. now + also ignores any information relating to the module containing the + exception under test. + + -- Data: doctest.SKIP + + When specified, do not run the example at all. This can be useful + in contexts where doctest examples serve as both documentation and + test cases, and an example should be included for documentation + purposes, but should not be checked. E.g., the example’s output + might be random; or the example might depend on resources which + would be unavailable to the test driver. + + The SKIP flag can also be used for temporarily “commenting out” + examples. + + -- Data: doctest.COMPARISON_FLAGS + + A bitmask or’ing together all the comparison flags above. + +The second group of options controls how test failures are reported: + + -- Data: doctest.REPORT_UDIFF + + When specified, failures that involve multi-line expected and + actual outputs are displayed using a unified diff. + + -- Data: doctest.REPORT_CDIFF + + When specified, failures that involve multi-line expected and + actual outputs will be displayed using a context diff. + + -- Data: doctest.REPORT_NDIFF + + When specified, differences are computed by ‘difflib.Differ’, using + the same algorithm as the popular ‘ndiff.py’ utility. This is the + only method that marks differences within lines as well as across + lines. For example, if a line of expected output contains digit + ‘1’ where actual output contains letter ‘l’, a line is inserted + with a caret marking the mismatching column positions. + + -- Data: doctest.REPORT_ONLY_FIRST_FAILURE + + When specified, display the first failing example in each doctest, + but suppress output for all remaining examples. This will prevent + doctest from reporting correct examples that break because of + earlier failures; but it might also hide incorrect examples that + fail independently of the first failure. When *note + REPORT_ONLY_FIRST_FAILURE: 2fdf. is specified, the remaining + examples are still run, and still count towards the total number of + failures reported; only the output is suppressed. + + -- Data: doctest.FAIL_FAST + + When specified, exit after the first failing example and don’t + attempt to run the remaining examples. Thus, the number of + failures reported will be at most 1. This flag may be useful + during debugging, since examples after the first failure won’t even + produce debugging output. + + The doctest command line accepts the option ‘-f’ as a shorthand for + ‘-o FAIL_FAST’. + + New in version 3.4. + + -- Data: doctest.REPORTING_FLAGS + + A bitmask or’ing together all the reporting flags above. + +There is also a way to register new option flag names, though this isn’t +useful unless you intend to extend *note doctest: 67. internals via +subclassing: + + -- Function: doctest.register_optionflag (name) + + Create a new option flag with a given name, and return the new + flag’s integer value. *note register_optionflag(): 2fe1. can be + used when subclassing *note OutputChecker: 2fe2. or *note + DocTestRunner: 2fe3. to create new options that are supported by + your subclasses. *note register_optionflag(): 2fe1. should always + be called using the following idiom: + + MY_FLAG = register_optionflag('MY_FLAG') + + +File: python.info, Node: Directives, Next: Warnings<2>, Prev: Option Flags, Up: How It Works + +5.26.3.9 Directives +................... + +Doctest directives may be used to modify the *note option flags: 83c. +for an individual example. Doctest directives are special Python +comments following an example’s source code: + + directive ::= "#" "doctest:" directive_options + directive_options ::= directive_option ("," directive_option)\* + directive_option ::= on_or_off directive_option_name + on_or_off ::= "+" \| "-" + directive_option_name ::= "DONT_ACCEPT_BLANKLINE" \| "NORMALIZE_WHITESPACE" \| ... + +Whitespace is not allowed between the ‘+’ or ‘-’ and the directive +option name. The directive option name can be any of the option flag +names explained above. + +An example’s doctest directives modify doctest’s behavior for that +single example. Use ‘+’ to enable the named behavior, or ‘-’ to disable +it. + +For example, this test passes: + + >>> print(list(range(20))) + [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, + 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] + +Without the directive it would fail, both because the actual output +doesn’t have two blanks before the single-digit list elements, and +because the actual output is on a single line. This test also passes, +and also requires a directive to do so: + + >>> print(list(range(20))) + [0, 1, ..., 18, 19] + +Multiple directives can be used on a single physical line, separated by +commas: + + >>> print(list(range(20))) + [0, 1, ..., 18, 19] + +If multiple directive comments are used for a single example, then they +are combined: + + >>> print(list(range(20))) + ... + [0, 1, ..., 18, 19] + +As the previous example shows, you can add ‘...’ lines to your example +containing only directives. This can be useful when an example is too +long for a directive to comfortably fit on the same line: + + >>> print(list(range(5)) + list(range(10, 20)) + list(range(30, 40))) + ... + [0, ..., 4, 10, ..., 19, 30, ..., 39] + +Note that since all options are disabled by default, and directives +apply only to the example they appear in, enabling options (via ‘+’ in a +directive) is usually the only meaningful choice. However, option flags +can also be passed to functions that run doctests, establishing +different defaults. In such cases, disabling an option via ‘-’ in a +directive can be useful. + + +File: python.info, Node: Warnings<2>, Prev: Directives, Up: How It Works + +5.26.3.10 Warnings +.................. + +*note doctest: 67. is serious about requiring exact matches in expected +output. If even a single character doesn’t match, the test fails. This +will probably surprise you a few times, as you learn exactly what Python +does and doesn’t guarantee about output. For example, when printing a +set, Python doesn’t guarantee that the element is printed in any +particular order, so a test like + + >>> foo() + {"Hermione", "Harry"} + +is vulnerable! One workaround is to do + + >>> foo() == {"Hermione", "Harry"} + True + +instead. Another is to do + + >>> d = sorted(foo()) + >>> d + ['Harry', 'Hermione'] + + Note: Before Python 3.6, when printing a dict, Python did not + guarantee that the key-value pairs was printed in any particular + order. + +There are others, but you get the idea. + +Another bad idea is to print things that embed an object address, like + + >>> id(1.0) # certain to fail some of the time + 7948648 + >>> class C: pass + >>> C() # the default repr() for instances embeds an address + <__main__.C instance at 0x00AC18F0> + +The *note ELLIPSIS: dd1. directive gives a nice approach for the last +example: + + >>> C() + <__main__.C instance at 0x...> + +Floating-point numbers are also subject to small output variations +across platforms, because Python defers to the platform C library for +float formatting, and C libraries vary widely in quality here. + + >>> 1./7 # risky + 0.14285714285714285 + >>> print(1./7) # safer + 0.142857142857 + >>> print(round(1./7, 6)) # much safer + 0.142857 + +Numbers of the form ‘I/2.**J’ are safe across all platforms, and I often +contrive doctest examples to produce numbers of that form: + + >>> 3./4 # utterly safe + 0.75 + +Simple fractions are also easier for people to understand, and that +makes for better documentation. + + +File: python.info, Node: Basic API, Next: Unittest API, Prev: How It Works, Up: doctest — Test interactive Python examples + +5.26.3.11 Basic API +................... + +The functions *note testmod(): dd0. and *note testfile(): 2fca. provide +a simple interface to doctest that should be sufficient for most basic +uses. For a less formal introduction to these two functions, see +sections *note Simple Usage; Checking Examples in Docstrings: 2fc5. and +*note Simple Usage; Checking Examples in a Text File: 2fc8. + + -- Function: doctest.testfile (filename, module_relative=True, + name=None, package=None, globs=None, verbose=None, + report=True, optionflags=0, extraglobs=None, + raise_on_error=False, parser=DocTestParser(), encoding=None) + + All arguments except `filename' are optional, and should be + specified in keyword form. + + Test examples in the file named `filename'. Return + ‘(failure_count, test_count)’. + + Optional argument `module_relative' specifies how the filename + should be interpreted: + + * If `module_relative' is ‘True’ (the default), then `filename' + specifies an OS-independent module-relative path. By default, + this path is relative to the calling module’s directory; but + if the `package' argument is specified, then it is relative to + that package. To ensure OS-independence, `filename' should + use ‘/’ characters to separate path segments, and may not be + an absolute path (i.e., it may not begin with ‘/’). + + * If `module_relative' is ‘False’, then `filename' specifies an + OS-specific path. The path may be absolute or relative; + relative paths are resolved with respect to the current + working directory. + + Optional argument `name' gives the name of the test; by default, or + if ‘None’, ‘os.path.basename(filename)’ is used. + + Optional argument `package' is a Python package or the name of a + Python package whose directory should be used as the base directory + for a module-relative filename. If no package is specified, then + the calling module’s directory is used as the base directory for + module-relative filenames. It is an error to specify `package' if + `module_relative' is ‘False’. + + Optional argument `globs' gives a dict to be used as the globals + when executing examples. A new shallow copy of this dict is + created for the doctest, so its examples start with a clean slate. + By default, or if ‘None’, a new empty dict is used. + + Optional argument `extraglobs' gives a dict merged into the globals + used to execute examples. This works like *note dict.update(): + dc9.: if `globs' and `extraglobs' have a common key, the associated + value in `extraglobs' appears in the combined dict. By default, or + if ‘None’, no extra globals are used. This is an advanced feature + that allows parameterization of doctests. For example, a doctest + can be written for a base class, using a generic name for the + class, then reused to test any number of subclasses by passing an + `extraglobs' dict mapping the generic name to the subclass to be + tested. + + Optional argument `verbose' prints lots of stuff if true, and + prints only failures if false; by default, or if ‘None’, it’s true + if and only if ‘'-v'’ is in ‘sys.argv’. + + Optional argument `report' prints a summary at the end when true, + else prints nothing at the end. In verbose mode, the summary is + detailed, else the summary is very brief (in fact, empty if all + tests passed). + + Optional argument `optionflags' (default value 0) takes the *note + bitwise OR: 11e4. of option flags. See section *note Option Flags: + 83c. + + Optional argument `raise_on_error' defaults to false. If true, an + exception is raised upon the first failure or unexpected exception + in an example. This allows failures to be post-mortem debugged. + Default behavior is to continue running examples. + + Optional argument `parser' specifies a *note DocTestParser: 2fd3. + (or subclass) that should be used to extract tests from the files. + It defaults to a normal parser (i.e., ‘DocTestParser()’). + + Optional argument `encoding' specifies an encoding that should be + used to convert the file to unicode. + + -- Function: doctest.testmod (m=None, name=None, globs=None, + verbose=None, report=True, optionflags=0, extraglobs=None, + raise_on_error=False, exclude_empty=False) + + All arguments are optional, and all except for `m' should be + specified in keyword form. + + Test examples in docstrings in functions and classes reachable from + module `m' (or module *note __main__: 1. if `m' is not supplied or + is ‘None’), starting with ‘m.__doc__’. + + Also test examples reachable from dict ‘m.__test__’, if it exists + and is not ‘None’. ‘m.__test__’ maps names (strings) to functions, + classes and strings; function and class docstrings are searched for + examples; strings are searched directly, as if they were + docstrings. + + Only docstrings attached to objects belonging to module `m' are + searched. + + Return ‘(failure_count, test_count)’. + + Optional argument `name' gives the name of the module; by default, + or if ‘None’, ‘m.__name__’ is used. + + Optional argument `exclude_empty' defaults to false. If true, + objects for which no doctests are found are excluded from + consideration. The default is a backward compatibility hack, so + that code still using ‘doctest.master.summarize()’ in conjunction + with *note testmod(): dd0. continues to get output for objects with + no tests. The `exclude_empty' argument to the newer *note + DocTestFinder: 2fed. constructor defaults to true. + + Optional arguments `extraglobs', `verbose', `report', + `optionflags', `raise_on_error', and `globs' are the same as for + function *note testfile(): 2fca. above, except that `globs' + defaults to ‘m.__dict__’. + + -- Function: doctest.run_docstring_examples (f, globs, verbose=False, + name="NoName", compileflags=None, optionflags=0) + + Test examples associated with object `f'; for example, `f' may be a + string, a module, a function, or a class object. + + A shallow copy of dictionary argument `globs' is used for the + execution context. + + Optional argument `name' is used in failure messages, and defaults + to ‘"NoName"’. + + If optional argument `verbose' is true, output is generated even if + there are no failures. By default, output is generated only in + case of an example failure. + + Optional argument `compileflags' gives the set of flags that should + be used by the Python compiler when running the examples. By + default, or if ‘None’, flags are deduced corresponding to the set + of future features found in `globs'. + + Optional argument `optionflags' works as for function *note + testfile(): 2fca. above. + + +File: python.info, Node: Unittest API, Next: Advanced API, Prev: Basic API, Up: doctest — Test interactive Python examples + +5.26.3.12 Unittest API +...................... + +As your collection of doctest’ed modules grows, you’ll want a way to run +all their doctests systematically. *note doctest: 67. provides two +functions that can be used to create *note unittest: 11b. test suites +from modules and text files containing doctests. To integrate with +*note unittest: 11b. test discovery, include a ‘load_tests()’ function +in your test module: + + import unittest + import doctest + import my_module_with_doctests + + def load_tests(loader, tests, ignore): + tests.addTests(doctest.DocTestSuite(my_module_with_doctests)) + return tests + +There are two main functions for creating *note unittest.TestSuite: 6ce. +instances from text files and modules with doctests: + + -- Function: doctest.DocFileSuite (*paths, module_relative=True, + package=None, setUp=None, tearDown=None, globs=None, + optionflags=0, parser=DocTestParser(), encoding=None) + + Convert doctest tests from one or more text files to a *note + unittest.TestSuite: 6ce. + + The returned *note unittest.TestSuite: 6ce. is to be run by the + unittest framework and runs the interactive examples in each file. + If an example in any file fails, then the synthesized unit test + fails, and a ‘failureException’ exception is raised showing the + name of the file containing the test and a (sometimes approximate) + line number. + + Pass one or more paths (as strings) to text files to be examined. + + Options may be provided as keyword arguments: + + Optional argument `module_relative' specifies how the filenames in + `paths' should be interpreted: + + * If `module_relative' is ‘True’ (the default), then each + filename in `paths' specifies an OS-independent + module-relative path. By default, this path is relative to + the calling module’s directory; but if the `package' argument + is specified, then it is relative to that package. To ensure + OS-independence, each filename should use ‘/’ characters to + separate path segments, and may not be an absolute path (i.e., + it may not begin with ‘/’). + + * If `module_relative' is ‘False’, then each filename in `paths' + specifies an OS-specific path. The path may be absolute or + relative; relative paths are resolved with respect to the + current working directory. + + Optional argument `package' is a Python package or the name of a + Python package whose directory should be used as the base directory + for module-relative filenames in `paths'. If no package is + specified, then the calling module’s directory is used as the base + directory for module-relative filenames. It is an error to specify + `package' if `module_relative' is ‘False’. + + Optional argument `setUp' specifies a set-up function for the test + suite. This is called before running the tests in each file. The + `setUp' function will be passed a *note DocTest: 2ff2. object. The + setUp function can access the test globals as the `globs' attribute + of the test passed. + + Optional argument `tearDown' specifies a tear-down function for the + test suite. This is called after running the tests in each file. + The `tearDown' function will be passed a *note DocTest: 2ff2. + object. The setUp function can access the test globals as the + `globs' attribute of the test passed. + + Optional argument `globs' is a dictionary containing the initial + global variables for the tests. A new copy of this dictionary is + created for each test. By default, `globs' is a new empty + dictionary. + + Optional argument `optionflags' specifies the default doctest + options for the tests, created by or-ing together individual option + flags. See section *note Option Flags: 83c. See function *note + set_unittest_reportflags(): 2ff3. below for a better way to set + reporting options. + + Optional argument `parser' specifies a *note DocTestParser: 2fd3. + (or subclass) that should be used to extract tests from the files. + It defaults to a normal parser (i.e., ‘DocTestParser()’). + + Optional argument `encoding' specifies an encoding that should be + used to convert the file to unicode. + + The global ‘__file__’ is added to the globals provided to doctests + loaded from a text file using *note DocFileSuite(): 2ff1. + + -- Function: doctest.DocTestSuite (module=None, globs=None, + extraglobs=None, test_finder=None, setUp=None, tearDown=None, + checker=None) + + Convert doctest tests for a module to a *note unittest.TestSuite: + 6ce. + + The returned *note unittest.TestSuite: 6ce. is to be run by the + unittest framework and runs each doctest in the module. If any of + the doctests fail, then the synthesized unit test fails, and a + ‘failureException’ exception is raised showing the name of the file + containing the test and a (sometimes approximate) line number. + + Optional argument `module' provides the module to be tested. It + can be a module object or a (possibly dotted) module name. If not + specified, the module calling this function is used. + + Optional argument `globs' is a dictionary containing the initial + global variables for the tests. A new copy of this dictionary is + created for each test. By default, `globs' is a new empty + dictionary. + + Optional argument `extraglobs' specifies an extra set of global + variables, which is merged into `globs'. By default, no extra + globals are used. + + Optional argument `test_finder' is the *note DocTestFinder: 2fed. + object (or a drop-in replacement) that is used to extract doctests + from the module. + + Optional arguments `setUp', `tearDown', and `optionflags' are the + same as for function *note DocFileSuite(): 2ff1. above. + + This function uses the same search technique as *note testmod(): + dd0. + + Changed in version 3.5: *note DocTestSuite(): 6cd. returns an empty + *note unittest.TestSuite: 6ce. if `module' contains no docstrings + instead of raising *note ValueError: 1fb. + +Under the covers, *note DocTestSuite(): 6cd. creates a *note +unittest.TestSuite: 6ce. out of ‘doctest.DocTestCase’ instances, and +‘DocTestCase’ is a subclass of *note unittest.TestCase: 8ff. +‘DocTestCase’ isn’t documented here (it’s an internal detail), but +studying its code can answer questions about the exact details of *note +unittest: 11b. integration. + +Similarly, *note DocFileSuite(): 2ff1. creates a *note +unittest.TestSuite: 6ce. out of ‘doctest.DocFileCase’ instances, and +‘DocFileCase’ is a subclass of ‘DocTestCase’. + +So both ways of creating a *note unittest.TestSuite: 6ce. run instances +of ‘DocTestCase’. This is important for a subtle reason: when you run +*note doctest: 67. functions yourself, you can control the *note +doctest: 67. options in use directly, by passing option flags to *note +doctest: 67. functions. However, if you’re writing a *note unittest: +11b. framework, *note unittest: 11b. ultimately controls when and how +tests get run. The framework author typically wants to control *note +doctest: 67. reporting options (perhaps, e.g., specified by command line +options), but there’s no way to pass options through *note unittest: +11b. to *note doctest: 67. test runners. + +For this reason, *note doctest: 67. also supports a notion of *note +doctest: 67. reporting flags specific to *note unittest: 11b. support, +via this function: + + -- Function: doctest.set_unittest_reportflags (flags) + + Set the *note doctest: 67. reporting flags to use. + + Argument `flags' takes the *note bitwise OR: 11e4. of option flags. + See section *note Option Flags: 83c. Only “reporting flags” can be + used. + + This is a module-global setting, and affects all future doctests + run by module *note unittest: 11b.: the ‘runTest()’ method of + ‘DocTestCase’ looks at the option flags specified for the test case + when the ‘DocTestCase’ instance was constructed. If no reporting + flags were specified (which is the typical and expected case), + *note doctest: 67.’s *note unittest: 11b. reporting flags are *note + bitwise ORed: 11e4. into the option flags, and the option flags so + augmented are passed to the *note DocTestRunner: 2fe3. instance + created to run the doctest. If any reporting flags were specified + when the ‘DocTestCase’ instance was constructed, *note doctest: + 67.’s *note unittest: 11b. reporting flags are ignored. + + The value of the *note unittest: 11b. reporting flags in effect + before the function was called is returned by the function. + + +File: python.info, Node: Advanced API, Next: Debugging, Prev: Unittest API, Up: doctest — Test interactive Python examples + +5.26.3.13 Advanced API +...................... + +The basic API is a simple wrapper that’s intended to make doctest easy +to use. It is fairly flexible, and should meet most users’ needs; +however, if you require more fine-grained control over testing, or wish +to extend doctest’s capabilities, then you should use the advanced API. + +The advanced API revolves around two container classes, which are used +to store the interactive examples extracted from doctest cases: + + * *note Example: 2ff6.: A single Python *note statement: 1847, paired + with its expected output. + + * *note DocTest: 2ff2.: A collection of *note Example: 2ff6.s, + typically extracted from a single docstring or text file. + +Additional processing classes are defined to find, parse, and run, and +check doctest examples: + + * *note DocTestFinder: 2fed.: Finds all docstrings in a given module, + and uses a *note DocTestParser: 2fd3. to create a *note DocTest: + 2ff2. from every docstring that contains interactive examples. + + * *note DocTestParser: 2fd3.: Creates a *note DocTest: 2ff2. object + from a string (such as an object’s docstring). + + * *note DocTestRunner: 2fe3.: Executes the examples in a *note + DocTest: 2ff2, and uses an *note OutputChecker: 2fe2. to verify + their output. + + * *note OutputChecker: 2fe2.: Compares the actual output from a + doctest example with the expected output, and decides whether they + match. + +The relationships among these processing classes are summarized in the +following diagram: + + list of: + +------+ +---------+ + |module| --DocTestFinder-> | DocTest | --DocTestRunner-> results + +------+ | ^ +---------+ | ^ (printed) + | | | Example | | | + v | | ... | v | + DocTestParser | Example | OutputChecker + +---------+ + +* Menu: + +* DocTest Objects:: +* Example Objects:: +* DocTestFinder objects:: +* DocTestParser objects:: +* DocTestRunner objects:: +* OutputChecker objects:: + + +File: python.info, Node: DocTest Objects, Next: Example Objects, Up: Advanced API + +5.26.3.14 DocTest Objects +......................... + + -- Class: doctest.DocTest (examples, globs, name, filename, lineno, + docstring) + + A collection of doctest examples that should be run in a single + namespace. The constructor arguments are used to initialize the + attributes of the same names. + + *note DocTest: 2ff2. defines the following attributes. They are + initialized by the constructor, and should not be modified + directly. + + -- Attribute: examples + + A list of *note Example: 2ff6. objects encoding the individual + interactive Python examples that should be run by this test. + + -- Attribute: globs + + The namespace (aka globals) that the examples should be run + in. This is a dictionary mapping names to values. Any + changes to the namespace made by the examples (such as binding + new variables) will be reflected in *note globs: 2ffa. after + the test is run. + + -- Attribute: name + + A string name identifying the *note DocTest: 2ff2. Typically, + this is the name of the object or file that the test was + extracted from. + + -- Attribute: filename + + The name of the file that this *note DocTest: 2ff2. was + extracted from; or ‘None’ if the filename is unknown, or if + the *note DocTest: 2ff2. was not extracted from a file. + + -- Attribute: lineno + + The line number within *note filename: 2ffc. where this *note + DocTest: 2ff2. begins, or ‘None’ if the line number is + unavailable. This line number is zero-based with respect to + the beginning of the file. + + -- Attribute: docstring + + The string that the test was extracted from, or ‘None’ if the + string is unavailable, or if the test was not extracted from a + string. + + +File: python.info, Node: Example Objects, Next: DocTestFinder objects, Prev: DocTest Objects, Up: Advanced API + +5.26.3.15 Example Objects +......................... + + -- Class: doctest.Example (source, want, exc_msg=None, lineno=0, + indent=0, options=None) + + A single interactive example, consisting of a Python statement and + its expected output. The constructor arguments are used to + initialize the attributes of the same names. + + *note Example: 2ff6. defines the following attributes. They are + initialized by the constructor, and should not be modified + directly. + + -- Attribute: source + + A string containing the example’s source code. This source + code consists of a single Python statement, and always ends + with a newline; the constructor adds a newline when necessary. + + -- Attribute: want + + The expected output from running the example’s source code + (either from stdout, or a traceback in case of exception). + *note want: 3002. ends with a newline unless no output is + expected, in which case it’s an empty string. The constructor + adds a newline when necessary. + + -- Attribute: exc_msg + + The exception message generated by the example, if the example + is expected to generate an exception; or ‘None’ if it is not + expected to generate an exception. This exception message is + compared against the return value of *note + traceback.format_exception_only(): 3004. *note exc_msg: 3003. + ends with a newline unless it’s ‘None’. The constructor adds + a newline if needed. + + -- Attribute: lineno + + The line number within the string containing this example + where the example begins. This line number is zero-based with + respect to the beginning of the containing string. + + -- Attribute: indent + + The example’s indentation in the containing string, i.e., the + number of space characters that precede the example’s first + prompt. + + -- Attribute: options + + A dictionary mapping from option flags to ‘True’ or ‘False’, + which is used to override default options for this example. + Any option flags not contained in this dictionary are left at + their default value (as specified by the *note DocTestRunner: + 2fe3.’s ‘optionflags’). By default, no options are set. + + +File: python.info, Node: DocTestFinder objects, Next: DocTestParser objects, Prev: Example Objects, Up: Advanced API + +5.26.3.16 DocTestFinder objects +............................... + + -- Class: doctest.DocTestFinder (verbose=False, parser=DocTestParser(), + recurse=True, exclude_empty=True) + + A processing class used to extract the *note DocTest: 2ff2.s that + are relevant to a given object, from its docstring and the + docstrings of its contained objects. *note DocTest: 2ff2.s can be + extracted from modules, classes, functions, methods, staticmethods, + classmethods, and properties. + + The optional argument `verbose' can be used to display the objects + searched by the finder. It defaults to ‘False’ (no output). + + The optional argument `parser' specifies the *note DocTestParser: + 2fd3. object (or a drop-in replacement) that is used to extract + doctests from docstrings. + + If the optional argument `recurse' is false, then *note + DocTestFinder.find(): 300a. will only examine the given object, and + not any contained objects. + + If the optional argument `exclude_empty' is false, then *note + DocTestFinder.find(): 300a. will include tests for objects with + empty docstrings. + + *note DocTestFinder: 2fed. defines the following method: + + -- Method: find (obj[, name][, module][, globs][, extraglobs]) + + Return a list of the *note DocTest: 2ff2.s that are defined by + `obj'’s docstring, or by any of its contained objects’ + docstrings. + + The optional argument `name' specifies the object’s name; this + name will be used to construct names for the returned *note + DocTest: 2ff2.s. If `name' is not specified, then + ‘obj.__name__’ is used. + + The optional parameter `module' is the module that contains + the given object. If the module is not specified or is + ‘None’, then the test finder will attempt to automatically + determine the correct module. The object’s module is used: + + * As a default namespace, if `globs' is not specified. + + * To prevent the DocTestFinder from extracting DocTests + from objects that are imported from other modules. + (Contained objects with modules other than `module' are + ignored.) + + * To find the name of the file containing the object. + + * To help find the line number of the object within its + file. + + If `module' is ‘False’, no attempt to find the module will be + made. This is obscure, of use mostly in testing doctest + itself: if `module' is ‘False’, or is ‘None’ but cannot be + found automatically, then all objects are considered to belong + to the (non-existent) module, so all contained objects will + (recursively) be searched for doctests. + + The globals for each *note DocTest: 2ff2. is formed by + combining `globs' and `extraglobs' (bindings in `extraglobs' + override bindings in `globs'). A new shallow copy of the + globals dictionary is created for each *note DocTest: 2ff2. + If `globs' is not specified, then it defaults to the module’s + `__dict__', if specified, or ‘{}’ otherwise. If `extraglobs' + is not specified, then it defaults to ‘{}’. + + +File: python.info, Node: DocTestParser objects, Next: DocTestRunner objects, Prev: DocTestFinder objects, Up: Advanced API + +5.26.3.17 DocTestParser objects +............................... + + -- Class: doctest.DocTestParser + + A processing class used to extract interactive examples from a + string, and use them to create a *note DocTest: 2ff2. object. + + *note DocTestParser: 2fd3. defines the following methods: + + -- Method: get_doctest (string, globs, name, filename, lineno) + + Extract all doctest examples from the given string, and + collect them into a *note DocTest: 2ff2. object. + + `globs', `name', `filename', and `lineno' are attributes for + the new *note DocTest: 2ff2. object. See the documentation + for *note DocTest: 2ff2. for more information. + + -- Method: get_examples (string, name='<string>') + + Extract all doctest examples from the given string, and return + them as a list of *note Example: 2ff6. objects. Line numbers + are 0-based. The optional argument `name' is a name + identifying this string, and is only used for error messages. + + -- Method: parse (string, name='<string>') + + Divide the given string into examples and intervening text, + and return them as a list of alternating *note Example: 2ff6.s + and strings. Line numbers for the *note Example: 2ff6.s are + 0-based. The optional argument `name' is a name identifying + this string, and is only used for error messages. + + +File: python.info, Node: DocTestRunner objects, Next: OutputChecker objects, Prev: DocTestParser objects, Up: Advanced API + +5.26.3.18 DocTestRunner objects +............................... + + -- Class: doctest.DocTestRunner (checker=None, verbose=None, + optionflags=0) + + A processing class used to execute and verify the interactive + examples in a *note DocTest: 2ff2. + + The comparison between expected outputs and actual outputs is done + by an *note OutputChecker: 2fe2. This comparison may be customized + with a number of option flags; see section *note Option Flags: 83c. + for more information. If the option flags are insufficient, then + the comparison may also be customized by passing a subclass of + *note OutputChecker: 2fe2. to the constructor. + + The test runner’s display output can be controlled in two ways. + First, an output function can be passed to ‘TestRunner.run()’; this + function will be called with strings that should be displayed. It + defaults to ‘sys.stdout.write’. If capturing the output is not + sufficient, then the display output can be also customized by + subclassing DocTestRunner, and overriding the methods *note + report_start(): 3012, *note report_success(): 3013, *note + report_unexpected_exception(): 3014, and *note report_failure(): + 3015. + + The optional keyword argument `checker' specifies the *note + OutputChecker: 2fe2. object (or drop-in replacement) that should be + used to compare the expected outputs to the actual outputs of + doctest examples. + + The optional keyword argument `verbose' controls the *note + DocTestRunner: 2fe3.’s verbosity. If `verbose' is ‘True’, then + information is printed about each example, as it is run. If + `verbose' is ‘False’, then only failures are printed. If `verbose' + is unspecified, or ‘None’, then verbose output is used iff the + command-line switch ‘-v’ is used. + + The optional keyword argument `optionflags' can be used to control + how the test runner compares expected output to actual output, and + how it displays failures. For more information, see section *note + Option Flags: 83c. + + *note DocTestParser: 2fd3. defines the following methods: + + -- Method: report_start (out, test, example) + + Report that the test runner is about to process the given + example. This method is provided to allow subclasses of *note + DocTestRunner: 2fe3. to customize their output; it should not + be called directly. + + `example' is the example about to be processed. `test' is the + test `containing example'. `out' is the output function that + was passed to *note DocTestRunner.run(): 3016. + + -- Method: report_success (out, test, example, got) + + Report that the given example ran successfully. This method + is provided to allow subclasses of *note DocTestRunner: 2fe3. + to customize their output; it should not be called directly. + + `example' is the example about to be processed. `got' is the + actual output from the example. `test' is the test containing + `example'. `out' is the output function that was passed to + *note DocTestRunner.run(): 3016. + + -- Method: report_failure (out, test, example, got) + + Report that the given example failed. This method is provided + to allow subclasses of *note DocTestRunner: 2fe3. to customize + their output; it should not be called directly. + + `example' is the example about to be processed. `got' is the + actual output from the example. `test' is the test containing + `example'. `out' is the output function that was passed to + *note DocTestRunner.run(): 3016. + + -- Method: report_unexpected_exception (out, test, example, + exc_info) + + Report that the given example raised an unexpected exception. + This method is provided to allow subclasses of *note + DocTestRunner: 2fe3. to customize their output; it should not + be called directly. + + `example' is the example about to be processed. `exc_info' is + a tuple containing information about the unexpected exception + (as returned by *note sys.exc_info(): c5f.). `test' is the + test containing `example'. `out' is the output function that + was passed to *note DocTestRunner.run(): 3016. + + -- Method: run (test, compileflags=None, out=None, + clear_globs=True) + + Run the examples in `test' (a *note DocTest: 2ff2. object), + and display the results using the writer function `out'. + + The examples are run in the namespace ‘test.globs’. If + `clear_globs' is true (the default), then this namespace will + be cleared after the test runs, to help with garbage + collection. If you would like to examine the namespace after + the test completes, then use `clear_globs=False'. + + `compileflags' gives the set of flags that should be used by + the Python compiler when running the examples. If not + specified, then it will default to the set of future-import + flags that apply to `globs'. + + The output of each example is checked using the *note + DocTestRunner: 2fe3.’s output checker, and the results are + formatted by the ‘DocTestRunner.report_*()’ methods. + + -- Method: summarize (verbose=None) + + Print a summary of all the test cases that have been run by + this DocTestRunner, and return a *note named tuple: aa3. + ‘TestResults(failed, attempted)’. + + The optional `verbose' argument controls how detailed the + summary is. If the verbosity is not specified, then the *note + DocTestRunner: 2fe3.’s verbosity is used. + + +File: python.info, Node: OutputChecker objects, Prev: DocTestRunner objects, Up: Advanced API + +5.26.3.19 OutputChecker objects +............................... + + -- Class: doctest.OutputChecker + + A class used to check the whether the actual output from a doctest + example matches the expected output. *note OutputChecker: 2fe2. + defines two methods: *note check_output(): 301a, which compares a + given pair of outputs, and returns ‘True’ if they match; and *note + output_difference(): 301b, which returns a string describing the + differences between two outputs. + + *note OutputChecker: 2fe2. defines the following methods: + + -- Method: check_output (want, got, optionflags) + + Return ‘True’ iff the actual output from an example (`got') + matches the expected output (`want'). These strings are + always considered to match if they are identical; but + depending on what option flags the test runner is using, + several non-exact match types are also possible. See section + *note Option Flags: 83c. for more information about option + flags. + + -- Method: output_difference (example, got, optionflags) + + Return a string describing the differences between the + expected output for a given example (`example') and the actual + output (`got'). `optionflags' is the set of option flags used + to compare `want' and `got'. + + +File: python.info, Node: Debugging, Next: Soapbox, Prev: Advanced API, Up: doctest — Test interactive Python examples + +5.26.3.20 Debugging +................... + +Doctest provides several mechanisms for debugging doctest examples: + + * Several functions convert doctests to executable Python programs, + which can be run under the Python debugger, *note pdb: c9. + + * The *note DebugRunner: 301e. class is a subclass of *note + DocTestRunner: 2fe3. that raises an exception for the first failing + example, containing information about that example. This + information can be used to perform post-mortem debugging on the + example. + + * The *note unittest: 11b. cases generated by *note DocTestSuite(): + 6cd. support the *note debug(): 301f. method defined by *note + unittest.TestCase: 8ff. + + * You can add a call to *note pdb.set_trace(): 3c2. in a doctest + example, and you’ll drop into the Python debugger when that line is + executed. Then you can inspect current values of variables, and so + on. For example, suppose ‘a.py’ contains just this module + docstring: + + """ + >>> def f(x): + ... g(x*2) + >>> def g(x): + ... print(x+3) + ... import pdb; pdb.set_trace() + >>> f(3) + 9 + """ + + Then an interactive Python session may look like this: + + >>> import a, doctest + >>> doctest.testmod(a) + --Return-- + > <doctest a[1]>(3)g()->None + -> import pdb; pdb.set_trace() + (Pdb) list + 1 def g(x): + 2 print(x+3) + 3 -> import pdb; pdb.set_trace() + [EOF] + (Pdb) p x + 6 + (Pdb) step + --Return-- + > <doctest a[0]>(2)f()->None + -> g(x*2) + (Pdb) list + 1 def f(x): + 2 -> g(x*2) + [EOF] + (Pdb) p x + 3 + (Pdb) step + --Return-- + > <doctest a[2]>(1)?()->None + -> f(3) + (Pdb) cont + (0, 3) + >>> + +Functions that convert doctests to Python code, and possibly run the +synthesized code under the debugger: + + -- Function: doctest.script_from_examples (s) + + Convert text with examples to a script. + + Argument `s' is a string containing doctest examples. The string + is converted to a Python script, where doctest examples in `s' are + converted to regular code, and everything else is converted to + Python comments. The generated script is returned as a string. + For example, + + import doctest + print(doctest.script_from_examples(r""" + Set x and y to 1 and 2. + >>> x, y = 1, 2 + + Print their sum: + >>> print(x+y) + 3 + """)) + + displays: + + # Set x and y to 1 and 2. + x, y = 1, 2 + # + # Print their sum: + print(x+y) + # Expected: + ## 3 + + This function is used internally by other functions (see below), + but can also be useful when you want to transform an interactive + Python session into a Python script. + + -- Function: doctest.testsource (module, name) + + Convert the doctest for an object to a script. + + Argument `module' is a module object, or dotted name of a module, + containing the object whose doctests are of interest. Argument + `name' is the name (within the module) of the object with the + doctests of interest. The result is a string, containing the + object’s docstring converted to a Python script, as described for + *note script_from_examples(): 3020. above. For example, if module + ‘a.py’ contains a top-level function ‘f()’, then + + import a, doctest + print(doctest.testsource(a, "a.f")) + + prints a script version of function ‘f()’’s docstring, with + doctests converted to code, and the rest placed in comments. + + -- Function: doctest.debug (module, name, pm=False) + + Debug the doctests for an object. + + The `module' and `name' arguments are the same as for function + *note testsource(): 3021. above. The synthesized Python script for + the named object’s docstring is written to a temporary file, and + then that file is run under the control of the Python debugger, + *note pdb: c9. + + A shallow copy of ‘module.__dict__’ is used for both local and + global execution context. + + Optional argument `pm' controls whether post-mortem debugging is + used. If `pm' has a true value, the script file is run directly, + and the debugger gets involved only if the script terminates via + raising an unhandled exception. If it does, then post-mortem + debugging is invoked, via *note pdb.post_mortem(): d45, passing the + traceback object from the unhandled exception. If `pm' is not + specified, or is false, the script is run under the debugger from + the start, via passing an appropriate *note exec(): c44. call to + *note pdb.run(): 3022. + + -- Function: doctest.debug_src (src, pm=False, globs=None) + + Debug the doctests in a string. + + This is like function *note debug(): 301f. above, except that a + string containing doctest examples is specified directly, via the + `src' argument. + + Optional argument `pm' has the same meaning as in function *note + debug(): 301f. above. + + Optional argument `globs' gives a dictionary to use as both local + and global execution context. If not specified, or ‘None’, an + empty dictionary is used. If specified, a shallow copy of the + dictionary is used. + +The *note DebugRunner: 301e. class, and the special exceptions it may +raise, are of most interest to testing framework authors, and will only +be sketched here. See the source code, and especially *note +DebugRunner: 301e.’s docstring (which is a doctest!) for more details: + + -- Class: doctest.DebugRunner (checker=None, verbose=None, + optionflags=0) + + A subclass of *note DocTestRunner: 2fe3. that raises an exception + as soon as a failure is encountered. If an unexpected exception + occurs, an *note UnexpectedException: 3024. exception is raised, + containing the test, the example, and the original exception. If + the output doesn’t match, then a *note DocTestFailure: 3025. + exception is raised, containing the test, the example, and the + actual output. + + For information about the constructor parameters and methods, see + the documentation for *note DocTestRunner: 2fe3. in section *note + Advanced API: 2ff5. + +There are two exceptions that may be raised by *note DebugRunner: 301e. +instances: + + -- Exception: doctest.DocTestFailure (test, example, got) + + An exception raised by *note DocTestRunner: 2fe3. to signal that a + doctest example’s actual output did not match its expected output. + The constructor arguments are used to initialize the attributes of + the same names. + +*note DocTestFailure: 3025. defines the following attributes: + + -- Attribute: DocTestFailure.test + + The *note DocTest: 2ff2. object that was being run when the example + failed. + + -- Attribute: DocTestFailure.example + + The *note Example: 2ff6. that failed. + + -- Attribute: DocTestFailure.got + + The example’s actual output. + + -- Exception: doctest.UnexpectedException (test, example, exc_info) + + An exception raised by *note DocTestRunner: 2fe3. to signal that a + doctest example raised an unexpected exception. The constructor + arguments are used to initialize the attributes of the same names. + +*note UnexpectedException: 3024. defines the following attributes: + + -- Attribute: UnexpectedException.test + + The *note DocTest: 2ff2. object that was being run when the example + failed. + + -- Attribute: UnexpectedException.example + + The *note Example: 2ff6. that failed. + + -- Attribute: UnexpectedException.exc_info + + A tuple containing information about the unexpected exception, as + returned by *note sys.exc_info(): c5f. + + +File: python.info, Node: Soapbox, Prev: Debugging, Up: doctest — Test interactive Python examples + +5.26.3.21 Soapbox +................. + +As mentioned in the introduction, *note doctest: 67. has grown to have +three primary uses: + + 1. Checking examples in docstrings. + + 2. Regression testing. + + 3. Executable documentation / literate testing. + +These uses have different requirements, and it is important to +distinguish them. In particular, filling your docstrings with obscure +test cases makes for bad documentation. + +When writing a docstring, choose docstring examples with care. There’s +an art to this that needs to be learned—it may not be natural at first. +Examples should add genuine value to the documentation. A good example +can often be worth many words. If done with care, the examples will be +invaluable for your users, and will pay back the time it takes to +collect them many times over as the years go by and things change. I’m +still amazed at how often one of my *note doctest: 67. examples stops +working after a “harmless” change. + +Doctest also makes an excellent tool for regression testing, especially +if you don’t skimp on explanatory text. By interleaving prose and +examples, it becomes much easier to keep track of what’s actually being +tested, and why. When a test fails, good prose can make it much easier +to figure out what the problem is, and how it should be fixed. It’s +true that you could write extensive comments in code-based testing, but +few programmers do. Many have found that using doctest approaches +instead leads to much clearer tests. Perhaps this is simply because +doctest makes writing prose a little easier than writing code, while +writing comments in code is a little harder. I think it goes deeper +than just that: the natural attitude when writing a doctest-based test +is that you want to explain the fine points of your software, and +illustrate them with examples. This in turn naturally leads to test +files that start with the simplest features, and logically progress to +complications and edge cases. A coherent narrative is the result, +instead of a collection of isolated functions that test isolated bits of +functionality seemingly at random. It’s a different attitude, and +produces different results, blurring the distinction between testing and +explaining. + +Regression testing is best confined to dedicated objects or files. +There are several options for organizing tests: + + * Write text files containing test cases as interactive examples, and + test the files using *note testfile(): 2fca. or *note + DocFileSuite(): 2ff1. This is recommended, although is easiest to + do for new projects, designed from the start to use doctest. + + * Define functions named ‘_regrtest_topic’ that consist of single + docstrings, containing test cases for the named topics. These + functions can be included in the same file as the module, or + separated out into a separate test file. + + * Define a ‘__test__’ dictionary mapping from regression test topics + to docstrings containing test cases. + +When you have placed your tests in a module, the module can itself be +the test runner. When a test fails, you can arrange for your test +runner to re-run only the failing doctest while you debug the problem. +Here is a minimal example of such a test runner: + + if __name__ == '__main__': + import doctest + flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST + if len(sys.argv) > 1: + name = sys.argv[1] + if name in globals(): + obj = globals()[name] + else: + obj = __test__[name] + doctest.run_docstring_examples(obj, globals(), name=name, + optionflags=flags) + else: + fail, total = doctest.testmod(optionflags=flags) + print("{} failures out of {} tests".format(fail, total)) + + +File: python.info, Node: unittest — Unit testing framework, Next: unittest mock — mock object library, Prev: doctest — Test interactive Python examples, Up: Development Tools + +5.26.4 ‘unittest’ — Unit testing framework +------------------------------------------ + +`Source code:' Lib/unittest/__init__.py(1) + +__________________________________________________________________ + +(If you are already familiar with the basic concepts of testing, you +might want to skip to *note the list of assert methods: 3030.) + +The *note unittest: 11b. unit testing framework was originally inspired +by JUnit and has a similar flavor as major unit testing frameworks in +other languages. It supports test automation, sharing of setup and +shutdown code for tests, aggregation of tests into collections, and +independence of the tests from the reporting framework. + +To achieve this, *note unittest: 11b. supports some important concepts +in an object-oriented way: + +test fixture + + A `test fixture' represents the preparation needed to perform one + or more tests, and any associated cleanup actions. This may + involve, for example, creating temporary or proxy databases, + directories, or starting a server process. + +test case + + A `test case' is the individual unit of testing. It checks for a + specific response to a particular set of inputs. *note unittest: + 11b. provides a base class, *note TestCase: 8ff, which may be used + to create new test cases. + +test suite + + A `test suite' is a collection of test cases, test suites, or both. + It is used to aggregate tests that should be executed together. + +test runner + + A `test runner' is a component which orchestrates the execution of + tests and provides the outcome to the user. The runner may use a + graphical interface, a textual interface, or return a special value + to indicate the results of executing the tests. + +See also +........ + +Module *note doctest: 67. + + Another test-support module with a very different flavor. + +Simple Smalltalk Testing: With Patterns(2) + + Kent Beck’s original paper on testing frameworks using the pattern + shared by *note unittest: 11b. + +pytest(3) + + Third-party unittest framework with a lighter-weight syntax for + writing tests. For example, ‘assert func(10) == 42’. + +The Python Testing Tools Taxonomy(4) + + An extensive list of Python testing tools including functional + testing frameworks and mock object libraries. + +Testing in Python Mailing List(5) + + A special-interest-group for discussion of testing, and testing + tools, in Python. + +The script ‘Tools/unittestgui/unittestgui.py’ in the Python source +distribution is a GUI tool for test discovery and execution. This is +intended largely for ease of use for those new to unit testing. For +production environments it is recommended that tests be driven by a +continuous integration system such as Buildbot(6), Jenkins(7) or +Travis-CI(8), or AppVeyor(9). + +* Menu: + +* Basic example:: +* Command-Line Interface: Command-Line Interface<3>. +* Test Discovery:: +* Organizing test code:: +* Re-using old test code:: +* Skipping tests and expected failures:: +* Distinguishing test iterations using subtests:: +* Classes and functions:: +* Class and Module Fixtures:: +* Signal Handling:: + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/unittest/__init__.py + + (2) +https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm + + (3) https://docs.pytest.org/ + + (4) https://wiki.python.org/moin/PythonTestingToolsTaxonomy + + (5) http://lists.idyll.org/listinfo/testing-in-python + + (6) https://buildbot.net/ + + (7) https://jenkins.io/ + + (8) https://travis-ci.com + + (9) https://www.appveyor.com/ + + +File: python.info, Node: Basic example, Next: Command-Line Interface<3>, Up: unittest — Unit testing framework + +5.26.4.1 Basic example +...................... + +The *note unittest: 11b. module provides a rich set of tools for +constructing and running tests. This section demonstrates that a small +subset of the tools suffice to meet the needs of most users. + +Here is a short script to test three string methods: + + import unittest + + class TestStringMethods(unittest.TestCase): + + def test_upper(self): + self.assertEqual('foo'.upper(), 'FOO') + + def test_isupper(self): + self.assertTrue('FOO'.isupper()) + self.assertFalse('Foo'.isupper()) + + def test_split(self): + s = 'hello world' + self.assertEqual(s.split(), ['hello', 'world']) + # check that s.split fails when the separator is not a string + with self.assertRaises(TypeError): + s.split(2) + + if __name__ == '__main__': + unittest.main() + +A testcase is created by subclassing *note unittest.TestCase: 8ff. The +three individual tests are defined with methods whose names start with +the letters ‘test’. This naming convention informs the test runner +about which methods represent tests. + +The crux of each test is a call to *note assertEqual(): bb5. to check +for an expected result; *note assertTrue(): bb4. or *note assertFalse(): +cc7. to verify a condition; or *note assertRaises(): aba. to verify that +a specific exception gets raised. These methods are used instead of the +*note assert: e06. statement so the test runner can accumulate all test +results and produce a report. + +The *note setUp(): ccb. and *note tearDown(): ccc. methods allow you to +define instructions that will be executed before and after each test +method. They are covered in more detail in the section *note Organizing +test code: 3033. + +The final block shows a simple way to run the tests. *note +unittest.main(): 902. provides a command-line interface to the test +script. When run from the command line, the above script produces an +output that looks like this: + + ... + ---------------------------------------------------------------------- + Ran 3 tests in 0.000s + + OK + +Passing the ‘-v’ option to your test script will instruct *note +unittest.main(): 902. to enable a higher level of verbosity, and produce +the following output: + + test_isupper (__main__.TestStringMethods) ... ok + test_split (__main__.TestStringMethods) ... ok + test_upper (__main__.TestStringMethods) ... ok + + ---------------------------------------------------------------------- + Ran 3 tests in 0.001s + + OK + +The above examples show the most commonly used *note unittest: 11b. +features which are sufficient to meet many everyday testing needs. The +remainder of the documentation explores the full feature set from first +principles. + + +File: python.info, Node: Command-Line Interface<3>, Next: Test Discovery, Prev: Basic example, Up: unittest — Unit testing framework + +5.26.4.2 Command-Line Interface +............................... + +The unittest module can be used from the command line to run tests from +modules, classes or even individual test methods: + + python -m unittest test_module1 test_module2 + python -m unittest test_module.TestClass + python -m unittest test_module.TestClass.test_method + +You can pass in a list with any combination of module names, and fully +qualified class or method names. + +Test modules can be specified by file path as well: + + python -m unittest tests/test_something.py + +This allows you to use the shell filename completion to specify the test +module. The file specified must still be importable as a module. The +path is converted to a module name by removing the ‘.py’ and converting +path separators into ‘.’. If you want to execute a test file that isn’t +importable as a module you should execute the file directly instead. + +You can run tests with more detail (higher verbosity) by passing in the +-v flag: + + python -m unittest -v test_module + +When executed without arguments *note Test Discovery: 3036. is started: + + python -m unittest + +For a list of all the command-line options: + + python -m unittest -h + +Changed in version 3.2: In earlier versions it was only possible to run +individual test methods and not modules or classes. + +* Menu: + +* Command-line options: Command-line options<3>. + + +File: python.info, Node: Command-line options<3>, Up: Command-Line Interface<3> + +5.26.4.3 Command-line options +............................. + +‘unittest’ supports these command-line options: + + -- Program Option: -b, --buffer + + The standard output and standard error streams are buffered during + the test run. Output during a passing test is discarded. Output + is echoed normally on test fail or error and is added to the + failure messages. + + -- Program Option: -c, --catch + + ‘Control-C’ during the test run waits for the current test to end + and then reports all the results so far. A second ‘Control-C’ + raises the normal *note KeyboardInterrupt: 197. exception. + + See *note Signal Handling: 3038. for the functions that provide + this functionality. + + -- Program Option: -f, --failfast + + Stop the test run on the first error or failure. + + -- Program Option: -k + + Only run test methods and classes that match the pattern or + substring. This option may be used multiple times, in which case + all test cases that match of the given patterns are included. + + Patterns that contain a wildcard character (‘*’) are matched + against the test name using *note fnmatch.fnmatchcase(): 193b.; + otherwise simple case-sensitive substring matching is used. + + Patterns are matched against the fully qualified test method name + as imported by the test loader. + + For example, ‘-k foo’ matches ‘foo_tests.SomeTest.test_something’, + ‘bar_tests.SomeTest.test_foo’, but not + ‘bar_tests.FooTest.test_something’. + + -- Program Option: --locals + + Show local variables in tracebacks. + +New in version 3.2: The command-line options ‘-b’, ‘-c’ and ‘-f’ were +added. + +New in version 3.5: The command-line option ‘--locals’. + +New in version 3.7: The command-line option ‘-k’. + +The command line can also be used for test discovery, for running all of +the tests in a project or just a subset. + + +File: python.info, Node: Test Discovery, Next: Organizing test code, Prev: Command-Line Interface<3>, Up: unittest — Unit testing framework + +5.26.4.4 Test Discovery +....................... + +New in version 3.2. + +Unittest supports simple test discovery. In order to be compatible with +test discovery, all of the test files must be *note modules: f11. or +*note packages: f1e. (including *note namespace packages: 1158.) +importable from the top-level directory of the project (this means that +their filenames must be valid *note identifiers: 1071.). + +Test discovery is implemented in *note TestLoader.discover(): 904, but +can also be used from the command line. The basic command-line usage +is: + + cd project_directory + python -m unittest discover + + Note: As a shortcut, ‘python -m unittest’ is the equivalent of + ‘python -m unittest discover’. If you want to pass arguments to + test discovery the ‘discover’ sub-command must be used explicitly. + +The ‘discover’ sub-command has the following options: + + -- Program Option: -v, --verbose + + Verbose output + + -- Program Option: -s, --start-directory directory + + Directory to start discovery (‘.’ default) + + -- Program Option: -p, --pattern pattern + + Pattern to match test files (‘test*.py’ default) + + -- Program Option: -t, --top-level-directory directory + + Top level directory of project (defaults to start directory) + +The *note -s: 303d, *note -p: 303e, and *note -t: 303f. options can be +passed in as positional arguments in that order. The following two +command lines are equivalent: + + python -m unittest discover -s project_directory -p "*_test.py" + python -m unittest discover project_directory "*_test.py" + +As well as being a path it is possible to pass a package name, for +example ‘myproject.subpackage.test’, as the start directory. The +package name you supply will then be imported and its location on the +filesystem will be used as the start directory. + + Caution: Test discovery loads tests by importing them. Once test + discovery has found all the test files from the start directory you + specify it turns the paths into package names to import. For + example ‘foo/bar/baz.py’ will be imported as ‘foo.bar.baz’. + + If you have a package installed globally and attempt test discovery + on a different copy of the package then the import `could' happen + from the wrong place. If this happens test discovery will warn you + and exit. + + If you supply the start directory as a package name rather than a + path to a directory then discover assumes that whichever location + it imports from is the location you intended, so you will not get + the warning. + +Test modules and packages can customize test loading and discovery by +through the *note load_tests protocol: 3040. + +Changed in version 3.4: Test discovery supports *note namespace +packages: 1158. for start directory. Note that you need to the top +level directory too. (e.g. ‘python -m unittest discover -s +root/namespace -t root’). + + +File: python.info, Node: Organizing test code, Next: Re-using old test code, Prev: Test Discovery, Up: unittest — Unit testing framework + +5.26.4.5 Organizing test code +............................. + +The basic building blocks of unit testing are `test cases' — single +scenarios that must be set up and checked for correctness. In *note +unittest: 11b, test cases are represented by *note unittest.TestCase: +8ff. instances. To make your own test cases you must write subclasses +of *note TestCase: 8ff. or use *note FunctionTestCase: 3042. + +The testing code of a *note TestCase: 8ff. instance should be entirely +self contained, such that it can be run either in isolation or in +arbitrary combination with any number of other test cases. + +The simplest *note TestCase: 8ff. subclass will simply implement a test +method (i.e. a method whose name starts with ‘test’) in order to +perform specific testing code: + + import unittest + + class DefaultWidgetSizeTestCase(unittest.TestCase): + def test_default_widget_size(self): + widget = Widget('The widget') + self.assertEqual(widget.size(), (50, 50)) + +Note that in order to test something, we use one of the ‘assert*()’ +methods provided by the *note TestCase: 8ff. base class. If the test +fails, an exception will be raised with an explanatory message, and +*note unittest: 11b. will identify the test case as a `failure'. Any +other exceptions will be treated as `errors'. + +Tests can be numerous, and their set-up can be repetitive. Luckily, we +can factor out set-up code by implementing a method called *note +setUp(): ccb, which the testing framework will automatically call for +every single test we run: + + import unittest + + class WidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + def test_default_widget_size(self): + self.assertEqual(self.widget.size(), (50,50), + 'incorrect default size') + + def test_widget_resize(self): + self.widget.resize(100,150) + self.assertEqual(self.widget.size(), (100,150), + 'wrong size after resize') + + Note: The order in which the various tests will be run is + determined by sorting the test method names with respect to the + built-in ordering for strings. + +If the *note setUp(): ccb. method raises an exception while the test is +running, the framework will consider the test to have suffered an error, +and the test method will not be executed. + +Similarly, we can provide a *note tearDown(): ccc. method that tidies up +after the test method has been run: + + import unittest + + class WidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + def tearDown(self): + self.widget.dispose() + +If *note setUp(): ccb. succeeded, *note tearDown(): ccc. will be run +whether the test method succeeded or not. + +Such a working environment for the testing code is called a `test +fixture'. A new TestCase instance is created as a unique test fixture +used to execute each individual test method. Thus *note setUp(): ccb, +*note tearDown(): ccc, and ‘__init__()’ will be called once per test. + +It is recommended that you use TestCase implementations to group tests +together according to the features they test. *note unittest: 11b. +provides a mechanism for this: the `test suite', represented by *note +unittest: 11b.’s *note TestSuite: 6ce. class. In most cases, calling +*note unittest.main(): 902. will do the right thing and collect all the +module’s test cases for you and execute them. + +However, should you want to customize the building of your test suite, +you can do it yourself: + + def suite(): + suite = unittest.TestSuite() + suite.addTest(WidgetTestCase('test_default_widget_size')) + suite.addTest(WidgetTestCase('test_widget_resize')) + return suite + + if __name__ == '__main__': + runner = unittest.TextTestRunner() + runner.run(suite()) + +You can place the definitions of test cases and test suites in the same +modules as the code they are to test (such as ‘widget.py’), but there +are several advantages to placing the test code in a separate module, +such as ‘test_widget.py’: + + * The test module can be run standalone from the command line. + + * The test code can more easily be separated from shipped code. + + * There is less temptation to change test code to fit the code it + tests without a good reason. + + * Test code should be modified much less frequently than the code it + tests. + + * Tested code can be refactored more easily. + + * Tests for modules written in C must be in separate modules anyway, + so why not be consistent? + + * If the testing strategy changes, there is no need to change the + source code. + + +File: python.info, Node: Re-using old test code, Next: Skipping tests and expected failures, Prev: Organizing test code, Up: unittest — Unit testing framework + +5.26.4.6 Re-using old test code +............................... + +Some users will find that they have existing test code that they would +like to run from *note unittest: 11b, without converting every old test +function to a *note TestCase: 8ff. subclass. + +For this reason, *note unittest: 11b. provides a *note FunctionTestCase: +3042. class. This subclass of *note TestCase: 8ff. can be used to wrap +an existing test function. Set-up and tear-down functions can also be +provided. + +Given the following test function: + + def testSomething(): + something = makeSomething() + assert something.name is not None + # ... + +one can create an equivalent test case instance as follows, with +optional set-up and tear-down methods: + + testcase = unittest.FunctionTestCase(testSomething, + setUp=makeSomethingDB, + tearDown=deleteSomethingDB) + + Note: Even though *note FunctionTestCase: 3042. can be used to + quickly convert an existing test base over to a *note unittest: + 11b.-based system, this approach is not recommended. Taking the + time to set up proper *note TestCase: 8ff. subclasses will make + future test refactorings infinitely easier. + +In some cases, the existing tests may have been written using the *note +doctest: 67. module. If so, *note doctest: 67. provides a +‘DocTestSuite’ class that can automatically build *note +unittest.TestSuite: 6ce. instances from the existing *note doctest: +67.-based tests. + + +File: python.info, Node: Skipping tests and expected failures, Next: Distinguishing test iterations using subtests, Prev: Re-using old test code, Up: unittest — Unit testing framework + +5.26.4.7 Skipping tests and expected failures +............................................. + +New in version 3.1. + +Unittest supports skipping individual test methods and even whole +classes of tests. In addition, it supports marking a test as an +“expected failure,” a test that is broken and will fail, but shouldn’t +be counted as a failure on a *note TestResult: abf. + +Skipping a test is simply a matter of using the *note skip(): 3047. +*note decorator: 283. or one of its conditional variants, calling *note +TestCase.skipTest(): 3048. within a *note setUp(): ccb. or test method, +or raising *note SkipTest: 903. directly. + +Basic skipping looks like this: + + class MyTestCase(unittest.TestCase): + + @unittest.skip("demonstrating skipping") + def test_nothing(self): + self.fail("shouldn't happen") + + @unittest.skipIf(mylib.__version__ < (1, 3), + "not supported in this library version") + def test_format(self): + # Tests that work for only a certain version of the library. + pass + + @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") + def test_windows_support(self): + # windows specific testing code + pass + + def test_maybe_skipped(self): + if not external_resource_available(): + self.skipTest("external resource not available") + # test code that depends on the external resource + pass + +This is the output of running the example above in verbose mode: + + test_format (__main__.MyTestCase) ... skipped 'not supported in this library version' + test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping' + test_maybe_skipped (__main__.MyTestCase) ... skipped 'external resource not available' + test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows' + + ---------------------------------------------------------------------- + Ran 4 tests in 0.005s + + OK (skipped=4) + +Classes can be skipped just like methods: + + @unittest.skip("showing class skipping") + class MySkippedTestCase(unittest.TestCase): + def test_not_run(self): + pass + +*note TestCase.setUp(): ccb. can also skip the test. This is useful +when a resource that needs to be set up is not available. + +Expected failures use the *note expectedFailure(): 3049. decorator. + + class ExpectedFailureTestCase(unittest.TestCase): + @unittest.expectedFailure + def test_fail(self): + self.assertEqual(1, 0, "broken") + +It’s easy to roll your own skipping decorators by making a decorator +that calls *note skip(): 3047. on the test when it wants it to be +skipped. This decorator skips the test unless the passed object has a +certain attribute: + + def skipUnlessHasattr(obj, attr): + if hasattr(obj, attr): + return lambda func: func + return unittest.skip("{!r} doesn't have {!r}".format(obj, attr)) + +The following decorators and exception implement test skipping and +expected failures: + + -- Function: @unittest.skip (reason) + + Unconditionally skip the decorated test. `reason' should describe + why the test is being skipped. + + -- Function: @unittest.skipIf (condition, reason) + + Skip the decorated test if `condition' is true. + + -- Function: @unittest.skipUnless (condition, reason) + + Skip the decorated test unless `condition' is true. + + -- Function: @unittest.expectedFailure + + Mark the test as an expected failure or error. If the test fails + or errors it will be considered a success. If the test passes, it + will be considered a failure. + + -- Exception: unittest.SkipTest (reason) + + This exception is raised to skip a test. + + Usually you can use *note TestCase.skipTest(): 3048. or one of the + skipping decorators instead of raising this directly. + +Skipped tests will not have *note setUp(): ccb. or *note tearDown(): +ccc. run around them. Skipped classes will not have *note setUpClass(): +24c. or *note tearDownClass(): cc9. run. Skipped modules will not have +‘setUpModule()’ or ‘tearDownModule()’ run. + + +File: python.info, Node: Distinguishing test iterations using subtests, Next: Classes and functions, Prev: Skipping tests and expected failures, Up: unittest — Unit testing framework + +5.26.4.8 Distinguishing test iterations using subtests +...................................................... + +New in version 3.4. + +When there are very small differences among your tests, for instance +some parameters, unittest allows you to distinguish them inside the body +of a test method using the *note subTest(): 900. context manager. + +For example, the following test: + + class NumbersTest(unittest.TestCase): + + def test_even(self): + """ + Test that numbers between 0 and 5 are all even. + """ + for i in range(0, 6): + with self.subTest(i=i): + self.assertEqual(i % 2, 0) + +will produce the following output: + + ====================================================================== + FAIL: test_even (__main__.NumbersTest) (i=1) + ---------------------------------------------------------------------- + Traceback (most recent call last): + File "subtests.py", line 32, in test_even + self.assertEqual(i % 2, 0) + AssertionError: 1 != 0 + + ====================================================================== + FAIL: test_even (__main__.NumbersTest) (i=3) + ---------------------------------------------------------------------- + Traceback (most recent call last): + File "subtests.py", line 32, in test_even + self.assertEqual(i % 2, 0) + AssertionError: 1 != 0 + + ====================================================================== + FAIL: test_even (__main__.NumbersTest) (i=5) + ---------------------------------------------------------------------- + Traceback (most recent call last): + File "subtests.py", line 32, in test_even + self.assertEqual(i % 2, 0) + AssertionError: 1 != 0 + +Without using a subtest, execution would stop after the first failure, +and the error would be less easy to diagnose because the value of ‘i’ +wouldn’t be displayed: + + ====================================================================== + FAIL: test_even (__main__.NumbersTest) + ---------------------------------------------------------------------- + Traceback (most recent call last): + File "subtests.py", line 32, in test_even + self.assertEqual(i % 2, 0) + AssertionError: 1 != 0 + + +File: python.info, Node: Classes and functions, Next: Class and Module Fixtures, Prev: Distinguishing test iterations using subtests, Up: unittest — Unit testing framework + +5.26.4.9 Classes and functions +.............................. + +This section describes in depth the API of *note unittest: 11b. + +* Menu: + +* Test cases:: +* Grouping tests:: +* Loading and running tests:: + + +File: python.info, Node: Test cases, Next: Grouping tests, Up: Classes and functions + +5.26.4.10 Test cases +.................... + + -- Class: unittest.TestCase (methodName='runTest') + + Instances of the *note TestCase: 8ff. class represent the logical + test units in the *note unittest: 11b. universe. This class is + intended to be used as a base class, with specific tests being + implemented by concrete subclasses. This class implements the + interface needed by the test runner to allow it to drive the tests, + and methods that the test code can use to check for and report + various kinds of failure. + + Each instance of *note TestCase: 8ff. will run a single base + method: the method named `methodName'. In most uses of *note + TestCase: 8ff, you will neither change the `methodName' nor + reimplement the default ‘runTest()’ method. + + Changed in version 3.2: *note TestCase: 8ff. can be instantiated + successfully without providing a `methodName'. This makes it + easier to experiment with *note TestCase: 8ff. from the interactive + interpreter. + + *note TestCase: 8ff. instances provide three groups of methods: one + group used to run the test, another used by the test implementation + to check conditions and report failures, and some inquiry methods + allowing information about the test itself to be gathered. + + Methods in the first group (running the test) are: + + -- Method: setUp () + + Method called to prepare the test fixture. This is called + immediately before calling the test method; other than *note + AssertionError: 1223. or *note SkipTest: 903, any exception + raised by this method will be considered an error rather than + a test failure. The default implementation does nothing. + + -- Method: tearDown () + + Method called immediately after the test method has been + called and the result recorded. This is called even if the + test method raised an exception, so the implementation in + subclasses may need to be particularly careful about checking + internal state. Any exception, other than *note + AssertionError: 1223. or *note SkipTest: 903, raised by this + method will be considered an additional error rather than a + test failure (thus increasing the total number of reported + errors). This method will only be called if the *note + setUp(): ccb. succeeds, regardless of the outcome of the test + method. The default implementation does nothing. + + -- Method: setUpClass () + + A class method called before tests in an individual class are + run. ‘setUpClass’ is called with the class as the only + argument and must be decorated as a *note classmethod(): 1d8.: + + @classmethod + def setUpClass(cls): + ... + + See *note Class and Module Fixtures: 3051. for more details. + + New in version 3.2. + + -- Method: tearDownClass () + + A class method called after tests in an individual class have + run. ‘tearDownClass’ is called with the class as the only + argument and must be decorated as a *note classmethod(): 1d8.: + + @classmethod + def tearDownClass(cls): + ... + + See *note Class and Module Fixtures: 3051. for more details. + + New in version 3.2. + + -- Method: run (result=None) + + Run the test, collecting the result into the *note TestResult: + abf. object passed as `result'. If `result' is omitted or + ‘None’, a temporary result object is created (by calling the + *note defaultTestResult(): 3052. method) and used. The result + object is returned to *note run(): abe.’s caller. + + The same effect may be had by simply calling the *note + TestCase: 8ff. instance. + + Changed in version 3.3: Previous versions of ‘run’ did not + return the result. Neither did calling an instance. + + -- Method: skipTest (reason) + + Calling this during a test method or *note setUp(): ccb. skips + the current test. See *note Skipping tests and expected + failures: 3046. for more information. + + New in version 3.1. + + -- Method: subTest (msg=None, **params) + + Return a context manager which executes the enclosed code + block as a subtest. `msg' and `params' are optional, + arbitrary values which are displayed whenever a subtest fails, + allowing you to identify them clearly. + + A test case can contain any number of subtest declarations, + and they can be arbitrarily nested. + + See *note Distinguishing test iterations using subtests: 901. + for more information. + + New in version 3.4. + + -- Method: debug () + + Run the test without collecting the result. This allows + exceptions raised by the test to be propagated to the caller, + and can be used to support running tests under a debugger. + The *note TestCase: 8ff. class provides several assert methods to + check for and report failures. The following table lists the most + commonly used methods (see the tables below for more assert + methods): + + Method Checks that New in + + ---------------------------------------------------------------------------------------------------- + + *note assertEqual(a, b): bb5. ‘a == b’ + + + *note assertNotEqual(a, b): bb6. ‘a != b’ + + + *note assertTrue(x): bb4. ‘bool(x) is True’ + + + *note assertFalse(x): cc7. ‘bool(x) is False’ + + + *note assertIs(a, b): ccf. ‘a is b’ 3.1 + + + *note assertIsNot(a, b): cd0. ‘a is not b’ 3.1 + + + *note assertIsNone(x): ccd. ‘x is None’ 3.1 + + + *note assertIsNotNone(x): cce. ‘x is not None’ 3.1 + + + *note assertIn(a, b): cd8. ‘a in b’ 3.1 + + + *note assertNotIn(a, b): cd9. ‘a not in b’ 3.1 + + + *note assertIsInstance(a, b): cd1. ‘isinstance(a, b)’ 3.2 + + + *note assertNotIsInstance(a, b): cd2. ‘not isinstance(a, b)’ 3.2 + + + All the assert methods accept a `msg' argument that, if specified, + is used as the error message on failure (see also *note + longMessage: cc8.). Note that the `msg' keyword argument can be + passed to *note assertRaises(): aba, *note assertRaisesRegex(): + abb, *note assertWarns(): abc, *note assertWarnsRegex(): abd. only + when they are used as a context manager. + + -- Method: assertEqual (first, second, msg=None) + + Test that `first' and `second' are equal. If the values do + not compare equal, the test will fail. + + In addition, if `first' and `second' are the exact same type + and one of list, tuple, dict, set, frozenset or str or any + type that a subclass registers with *note + addTypeEqualityFunc(): ce1. the type-specific equality + function will be called in order to generate a more useful + default error message (see also the *note list of + type-specific methods: 3054.). + + Changed in version 3.1: Added the automatic calling of + type-specific equality function. + + Changed in version 3.2: *note assertMultiLineEqual(): cd7. + added as the default type equality function for comparing + strings. + + -- Method: assertNotEqual (first, second, msg=None) + + Test that `first' and `second' are not equal. If the values + do compare equal, the test will fail. + + -- Method: assertTrue (expr, msg=None) + -- Method: assertFalse (expr, msg=None) + + Test that `expr' is true (or false). + + Note that this is equivalent to ‘bool(expr) is True’ and not + to ‘expr is True’ (use ‘assertIs(expr, True)’ for the latter). + This method should also be avoided when more specific methods + are available (e.g. ‘assertEqual(a, b)’ instead of + ‘assertTrue(a == b)’), because they provide a better error + message in case of failure. + + -- Method: assertIs (first, second, msg=None) + -- Method: assertIsNot (first, second, msg=None) + + Test that `first' and `second' are (or are not) the same + object. + + New in version 3.1. + + -- Method: assertIsNone (expr, msg=None) + -- Method: assertIsNotNone (expr, msg=None) + + Test that `expr' is (or is not) ‘None’. + + New in version 3.1. + + -- Method: assertIn (member, container, msg=None) + -- Method: assertNotIn (member, container, msg=None) + + Test that `member' is (or is not) in `container'. + + New in version 3.1. + + -- Method: assertIsInstance (obj, cls, msg=None) + -- Method: assertNotIsInstance (obj, cls, msg=None) + + Test that `obj' is (or is not) an instance of `cls' (which can + be a class or a tuple of classes, as supported by *note + isinstance(): 44f.). To check for the exact type, use *note + assertIs(type(obj), cls): ccf. + + New in version 3.2. + + It is also possible to check the production of exceptions, + warnings, and log messages using the following methods: + + Method Checks that New in + + -------------------------------------------------------------------------------------------------------------------------- + + *note assertRaises(exc, fun, *args, **kwds): aba. ‘fun(*args, **kwds)’ raises `exc' + + + *note assertRaisesRegex(exc, r, fun, *args, **kwds): abb. ‘fun(*args, **kwds)’ raises `exc' and 3.1 + the message matches regex `r' + + + *note assertWarns(warn, fun, *args, **kwds): abc. ‘fun(*args, **kwds)’ raises `warn' 3.2 + + + *note assertWarnsRegex(warn, r, fun, *args, **kwds): abd. ‘fun(*args, **kwds)’ raises `warn' and 3.2 + the message matches regex `r' + + + *note assertLogs(logger, level): 905. The ‘with’ block logs on `logger' with 3.4 + minimum `level' + + + -- Method: assertRaises (exception, callable, *args, **kwds) + + -- Method: assertRaises (exception, *, msg=None) + + Test that an exception is raised when `callable' is called + with any positional or keyword arguments that are also passed + to *note assertRaises(): aba. The test passes if `exception' + is raised, is an error if another exception is raised, or + fails if no exception is raised. To catch any of a group of + exceptions, a tuple containing the exception classes may be + passed as `exception'. + + If only the `exception' and possibly the `msg' arguments are + given, return a context manager so that the code under test + can be written inline rather than as a function: + + with self.assertRaises(SomeException): + do_something() + + When used as a context manager, *note assertRaises(): aba. + accepts the additional keyword argument `msg'. + + The context manager will store the caught exception object in + its ‘exception’ attribute. This can be useful if the + intention is to perform additional checks on the exception + raised: + + with self.assertRaises(SomeException) as cm: + do_something() + + the_exception = cm.exception + self.assertEqual(the_exception.error_code, 3) + + Changed in version 3.1: Added the ability to use *note + assertRaises(): aba. as a context manager. + + Changed in version 3.2: Added the ‘exception’ attribute. + + Changed in version 3.3: Added the `msg' keyword argument when + used as a context manager. + + -- Method: assertRaisesRegex (exception, regex, callable, *args, + **kwds) + + -- Method: assertRaisesRegex (exception, regex, *, msg=None) + + Like *note assertRaises(): aba. but also tests that `regex' + matches on the string representation of the raised exception. + `regex' may be a regular expression object or a string + containing a regular expression suitable for use by *note + re.search(): bb2. Examples: + + self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", + int, 'XYZ') + + or: + + with self.assertRaisesRegex(ValueError, 'literal'): + int('XYZ') + + New in version 3.1: Added under the name ‘assertRaisesRegexp’. + + Changed in version 3.2: Renamed to *note assertRaisesRegex(): + abb. + + Changed in version 3.3: Added the `msg' keyword argument when + used as a context manager. + + -- Method: assertWarns (warning, callable, *args, **kwds) + + -- Method: assertWarns (warning, *, msg=None) + + Test that a warning is triggered when `callable' is called + with any positional or keyword arguments that are also passed + to *note assertWarns(): abc. The test passes if `warning' is + triggered and fails if it isn’t. Any exception is an error. + To catch any of a group of warnings, a tuple containing the + warning classes may be passed as `warnings'. + + If only the `warning' and possibly the `msg' arguments are + given, return a context manager so that the code under test + can be written inline rather than as a function: + + with self.assertWarns(SomeWarning): + do_something() + + When used as a context manager, *note assertWarns(): abc. + accepts the additional keyword argument `msg'. + + The context manager will store the caught warning object in + its ‘warning’ attribute, and the source line which triggered + the warnings in the ‘filename’ and ‘lineno’ attributes. This + can be useful if the intention is to perform additional checks + on the warning caught: + + with self.assertWarns(SomeWarning) as cm: + do_something() + + self.assertIn('myfile.py', cm.filename) + self.assertEqual(320, cm.lineno) + + This method works regardless of the warning filters in place + when it is called. + + New in version 3.2. + + Changed in version 3.3: Added the `msg' keyword argument when + used as a context manager. + + -- Method: assertWarnsRegex (warning, regex, callable, *args, + **kwds) + + -- Method: assertWarnsRegex (warning, regex, *, msg=None) + + Like *note assertWarns(): abc. but also tests that `regex' + matches on the message of the triggered warning. `regex' may + be a regular expression object or a string containing a + regular expression suitable for use by *note re.search(): bb2. + Example: + + self.assertWarnsRegex(DeprecationWarning, + r'legacy_function\(\) is deprecated', + legacy_function, 'XYZ') + + or: + + with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'): + frobnicate('/etc/passwd') + + New in version 3.2. + + Changed in version 3.3: Added the `msg' keyword argument when + used as a context manager. + + -- Method: assertLogs (logger=None, level=None) + + A context manager to test that at least one message is logged + on the `logger' or one of its children, with at least the + given `level'. + + If given, `logger' should be a *note logging.Logger: 3a7. + object or a *note str: 330. giving the name of a logger. The + default is the root logger, which will catch all messages that + were not blocked by a non-propagating descendent logger. + + If given, `level' should be either a numeric logging level or + its string equivalent (for example either ‘"ERROR"’ or + ‘logging.ERROR’). The default is ‘logging.INFO’. + + The test passes if at least one message emitted inside the + ‘with’ block matches the `logger' and `level' conditions, + otherwise it fails. + + The object returned by the context manager is a recording + helper which keeps tracks of the matching log messages. It + has two attributes: + + -- Attribute: records + + A list of *note logging.LogRecord: 906. objects of the + matching log messages. + + -- Attribute: output + + A list of *note str: 330. objects with the formatted + output of matching messages. + + Example: + + with self.assertLogs('foo', level='INFO') as cm: + logging.getLogger('foo').info('first message') + logging.getLogger('foo.bar').error('second message') + self.assertEqual(cm.output, ['INFO:foo:first message', + 'ERROR:foo.bar:second message']) + + New in version 3.4. + + There are also other methods used to perform more specific checks, + such as: + + Method Checks that New in + + ---------------------------------------------------------------------------------------------------- + + *note assertAlmostEqual(a, b): bb7. ‘round(a-b, 7) == 0’ + + + *note assertNotAlmostEqual(a, b): bb8. ‘round(a-b, 7) != 0’ + + + *note assertGreater(a, b): cd3. ‘a > b’ 3.1 + + + *note assertGreaterEqual(a, b): cd4. ‘a >= b’ 3.1 + + + *note assertLess(a, b): cd5. ‘a < b’ 3.1 + + + *note assertLessEqual(a, b): cd6. ‘a <= b’ 3.1 + + + *note assertRegex(s, r): bb1. ‘r.search(s)’ 3.1 + + + *note assertNotRegex(s, r): 3057. ‘not r.search(s)’ 3.2 + + + *note assertCountEqual(a, b): baf. `a' and `b' have the same elements 3.2 + in the same number, regardless of + their order. + + + -- Method: assertAlmostEqual (first, second, places=7, msg=None, + delta=None) + -- Method: assertNotAlmostEqual (first, second, places=7, + msg=None, delta=None) + + Test that `first' and `second' are approximately (or not + approximately) equal by computing the difference, rounding to + the given number of decimal `places' (default 7), and + comparing to zero. Note that these methods round the values + to the given number of `decimal places' (i.e. like the *note + round(): c6d. function) and not `significant digits'. + + If `delta' is supplied instead of `places' then the difference + between `first' and `second' must be less or equal to (or + greater than) `delta'. + + Supplying both `delta' and `places' raises a *note TypeError: + 192. + + Changed in version 3.2: *note assertAlmostEqual(): bb7. + automatically considers almost equal objects that compare + equal. *note assertNotAlmostEqual(): bb8. automatically fails + if the objects compare equal. Added the `delta' keyword + argument. + + -- Method: assertGreater (first, second, msg=None) + -- Method: assertGreaterEqual (first, second, msg=None) + -- Method: assertLess (first, second, msg=None) + -- Method: assertLessEqual (first, second, msg=None) + + Test that `first' is respectively >, >=, < or <= than `second' + depending on the method name. If not, the test will fail: + + >>> self.assertGreaterEqual(3, 4) + AssertionError: "3" unexpectedly not greater than or equal to "4" + + New in version 3.1. + + -- Method: assertRegex (text, regex, msg=None) + -- Method: assertNotRegex (text, regex, msg=None) + + Test that a `regex' search matches (or does not match) `text'. + In case of failure, the error message will include the pattern + and the `text' (or the pattern and the part of `text' that + unexpectedly matched). `regex' may be a regular expression + object or a string containing a regular expression suitable + for use by *note re.search(): bb2. + + New in version 3.1: Added under the name + ‘assertRegexpMatches’. + + Changed in version 3.2: The method ‘assertRegexpMatches()’ has + been renamed to *note assertRegex(): bb1. + + New in version 3.2: *note assertNotRegex(): 3057. + + New in version 3.5: The name ‘assertNotRegexpMatches’ is a + deprecated alias for *note assertNotRegex(): 3057. + + -- Method: assertCountEqual (first, second, msg=None) + + Test that sequence `first' contains the same elements as + `second', regardless of their order. When they don’t, an + error message listing the differences between the sequences + will be generated. + + Duplicate elements are `not' ignored when comparing `first' + and `second'. It verifies whether each element has the same + count in both sequences. Equivalent to: + ‘assertEqual(Counter(list(first)), Counter(list(second)))’ but + works with sequences of unhashable objects as well. + + New in version 3.2. + + The *note assertEqual(): bb5. method dispatches the equality check + for objects of the same type to different type-specific methods. + These methods are already implemented for most of the built-in + types, but it’s also possible to register new methods using *note + addTypeEqualityFunc(): ce1.: + + -- Method: addTypeEqualityFunc (typeobj, function) + + Registers a type-specific method called by *note + assertEqual(): bb5. to check if two objects of exactly the + same `typeobj' (not subclasses) compare equal. `function' + must take two positional arguments and a third msg=None + keyword argument just as *note assertEqual(): bb5. does. It + must raise *note self.failureException(msg): 3058. when + inequality between the first two parameters is detected – + possibly providing useful information and explaining the + inequalities in details in the error message. + + New in version 3.1. + + The list of type-specific methods automatically used by *note + assertEqual(): bb5. are summarized in the following table. Note + that it’s usually not necessary to invoke these methods directly. + + Method Used to compare New in + + --------------------------------------------------------------------------------------------------- + + *note assertMultiLineEqual(a, b): cd7. strings 3.1 + + + *note assertSequenceEqual(a, b): cdd. sequences 3.1 + + + *note assertListEqual(a, b): cdb. lists 3.1 + + + *note assertTupleEqual(a, b): cdc. tuples 3.1 + + + *note assertSetEqual(a, b): cda. sets or frozensets 3.1 + + + *note assertDictEqual(a, b): cde. dicts 3.1 + + + -- Method: assertMultiLineEqual (first, second, msg=None) + + Test that the multiline string `first' is equal to the string + `second'. When not equal a diff of the two strings + highlighting the differences will be included in the error + message. This method is used by default when comparing + strings with *note assertEqual(): bb5. + + New in version 3.1. + + -- Method: assertSequenceEqual (first, second, msg=None, + seq_type=None) + + Tests that two sequences are equal. If a `seq_type' is + supplied, both `first' and `second' must be instances of + `seq_type' or a failure will be raised. If the sequences are + different an error message is constructed that shows the + difference between the two. + + This method is not called directly by *note assertEqual(): + bb5, but it’s used to implement *note assertListEqual(): cdb. + and *note assertTupleEqual(): cdc. + + New in version 3.1. + + -- Method: assertListEqual (first, second, msg=None) + -- Method: assertTupleEqual (first, second, msg=None) + + Tests that two lists or tuples are equal. If not, an error + message is constructed that shows only the differences between + the two. An error is also raised if either of the parameters + are of the wrong type. These methods are used by default when + comparing lists or tuples with *note assertEqual(): bb5. + + New in version 3.1. + + -- Method: assertSetEqual (first, second, msg=None) + + Tests that two sets are equal. If not, an error message is + constructed that lists the differences between the sets. This + method is used by default when comparing sets or frozensets + with *note assertEqual(): bb5. + + Fails if either of `first' or `second' does not have a + ‘set.difference()’ method. + + New in version 3.1. + + -- Method: assertDictEqual (first, second, msg=None) + + Test that two dictionaries are equal. If not, an error + message is constructed that shows the differences in the + dictionaries. This method will be used by default to compare + dictionaries in calls to *note assertEqual(): bb5. + + New in version 3.1. + + Finally the *note TestCase: 8ff. provides the following methods and + attributes: + + -- Method: fail (msg=None) + + Signals a test failure unconditionally, with `msg' or ‘None’ + for the error message. + + -- Attribute: failureException + + This class attribute gives the exception raised by the test + method. If a test framework needs to use a specialized + exception, possibly to carry additional information, it must + subclass this exception in order to “play fair” with the + framework. The initial value of this attribute is *note + AssertionError: 1223. + + -- Attribute: longMessage + + This class attribute determines what happens when a custom + failure message is passed as the msg argument to an assertXYY + call that fails. ‘True’ is the default value. In this case, + the custom message is appended to the end of the standard + failure message. When set to ‘False’, the custom message + replaces the standard message. + + The class setting can be overridden in individual test methods + by assigning an instance attribute, self.longMessage, to + ‘True’ or ‘False’ before calling the assert methods. + + The class setting gets reset before each test call. + + New in version 3.1. + + -- Attribute: maxDiff + + This attribute controls the maximum length of diffs output by + assert methods that report diffs on failure. It defaults to + 80*8 characters. Assert methods affected by this attribute + are *note assertSequenceEqual(): cdd. (including all the + sequence comparison methods that delegate to it), *note + assertDictEqual(): cde. and *note assertMultiLineEqual(): cd7. + + Setting ‘maxDiff’ to ‘None’ means that there is no maximum + length of diffs. + + New in version 3.2. + + Testing frameworks can use the following methods to collect + information on the test: + + -- Method: countTestCases () + + Return the number of tests represented by this test object. + For *note TestCase: 8ff. instances, this will always be ‘1’. + + -- Method: defaultTestResult () + + Return an instance of the test result class that should be + used for this test case class (if no other result instance is + provided to the *note run(): abe. method). + + For *note TestCase: 8ff. instances, this will always be an + instance of *note TestResult: abf.; subclasses of *note + TestCase: 8ff. should override this as necessary. + + -- Method: id () + + Return a string identifying the specific test case. This is + usually the full name of the test method, including the module + and class name. + + -- Method: shortDescription () + + Returns a description of the test, or ‘None’ if no description + has been provided. The default implementation of this method + returns the first line of the test method’s docstring, if + available, or ‘None’. + + Changed in version 3.1: In 3.1 this was changed to add the + test name to the short description even in the presence of a + docstring. This caused compatibility issues with unittest + extensions and adding the test name was moved to the *note + TextTestResult: 305e. in Python 3.2. + + -- Method: addCleanup (function, *args, **kwargs) + + Add a function to be called after *note tearDown(): ccc. to + cleanup resources used during the test. Functions will be + called in reverse order to the order they are added (LIFO). + They are called with any arguments and keyword arguments + passed into *note addCleanup(): 2a2. when they are added. + + If *note setUp(): ccb. fails, meaning that *note tearDown(): + ccc. is not called, then any cleanup functions added will + still be called. + + New in version 3.1. + + -- Method: doCleanups () + + This method is called unconditionally after *note tearDown(): + ccc, or after *note setUp(): ccb. if *note setUp(): ccb. + raises an exception. + + It is responsible for calling all the cleanup functions added + by *note addCleanup(): 2a2. If you need cleanup functions to + be called `prior' to *note tearDown(): ccc. then you can call + *note doCleanups(): cca. yourself. + + *note doCleanups(): cca. pops methods off the stack of cleanup + functions one at a time, so it can be called at any time. + + New in version 3.1. + + -- Method: classmethod addClassCleanup (function, /, *args, + **kwargs) + + Add a function to be called after *note tearDownClass(): cc9. + to cleanup resources used during the test class. Functions + will be called in reverse order to the order they are added + (LIFO). They are called with any arguments and keyword + arguments passed into *note addClassCleanup(): 24b. when they + are added. + + If *note setUpClass(): 24c. fails, meaning that *note + tearDownClass(): cc9. is not called, then any cleanup + functions added will still be called. + + New in version 3.8. + + -- Method: classmethod doClassCleanups () + + This method is called unconditionally after *note + tearDownClass(): cc9, or after *note setUpClass(): 24c. if + *note setUpClass(): 24c. raises an exception. + + It is responsible for calling all the cleanup functions added + by *note addClassCleanup(): 24b. If you need cleanup + functions to be called `prior' to *note tearDownClass(): cc9. + then you can call *note doClassCleanups(): 305f. yourself. + + *note doClassCleanups(): 305f. pops methods off the stack of + cleanup functions one at a time, so it can be called at any + time. + + New in version 3.8. + + -- Class: unittest.IsolatedAsyncioTestCase (methodName='runTest') + + This class provides an API similar to *note TestCase: 8ff. and also + accepts coroutines as test functions. + + New in version 3.8. + + -- Method: coroutine asyncSetUp () + + Method called to prepare the test fixture. This is called + after ‘setUp()’. This is called immediately before calling + the test method; other than *note AssertionError: 1223. or + *note SkipTest: 903, any exception raised by this method will + be considered an error rather than a test failure. The + default implementation does nothing. + + -- Method: coroutine asyncTearDown () + + Method called immediately after the test method has been + called and the result recorded. This is called before + ‘tearDown()’. This is called even if the test method raised + an exception, so the implementation in subclasses may need to + be particularly careful about checking internal state. Any + exception, other than *note AssertionError: 1223. or *note + SkipTest: 903, raised by this method will be considered an + additional error rather than a test failure (thus increasing + the total number of reported errors). This method will only + be called if the *note asyncSetUp(): 3060. succeeds, + regardless of the outcome of the test method. The default + implementation does nothing. + + -- Method: addAsyncCleanup (function, /, *args, **kwargs) + + This method accepts a coroutine that can be used as a cleanup + function. + + -- Method: run (result=None) + + Sets up a new event loop to run the test, collecting the + result into the *note TestResult: abf. object passed as + `result'. If `result' is omitted or ‘None’, a temporary + result object is created (by calling the ‘defaultTestResult()’ + method) and used. The result object is returned to *note + run(): 3063.’s caller. At the end of the test all the tasks + in the event loop are cancelled. + + An example illustrating the order: + + from unittest import IsolatedAsyncioTestCase + + events = [] + + + class Test(IsolatedAsyncioTestCase): + + + def setUp(self): + events.append("setUp") + + async def asyncSetUp(self): + self._async_connection = await AsyncConnection() + events.append("asyncSetUp") + + async def test_response(self): + events.append("test_response") + response = await self._async_connection.get("https://example.com") + self.assertEqual(response.status_code, 200) + self.addAsyncCleanup(self.on_cleanup) + + def tearDown(self): + events.append("tearDown") + + async def asyncTearDown(self): + await self._async_connection.close() + events.append("asyncTearDown") + + async def on_cleanup(self): + events.append("cleanup") + + if __name__ == "__main__": + unittest.main() + + After running the test, ‘events’ would contain ‘["setUp", + "asyncSetUp", "test_response", "asyncTearDown", "tearDown", + "cleanup"]’. + + -- Class: unittest.FunctionTestCase (testFunc, setUp=None, + tearDown=None, description=None) + + This class implements the portion of the *note TestCase: 8ff. + interface which allows the test runner to drive the test, but does + not provide the methods which test code can use to check and report + errors. This is used to create test cases using legacy test code, + allowing it to be integrated into a *note unittest: 11b.-based test + framework. + +* Menu: + +* Deprecated aliases:: + + +File: python.info, Node: Deprecated aliases, Up: Test cases + +5.26.4.11 Deprecated aliases +............................ + +For historical reasons, some of the *note TestCase: 8ff. methods had one +or more aliases that are now deprecated. The following table lists the +correct names along with their deprecated aliases: + + Method Name Deprecated alias Deprecated alias + + ------------------------------------------------------------------------------------------ + + *note assertEqual(): bb5. failUnlessEqual assertEquals + + + *note assertNotEqual(): bb6. failIfEqual assertNotEquals + + + *note assertTrue(): bb4. failUnless assert_ + + + *note assertFalse(): cc7. failIf + + + *note assertRaises(): aba. failUnlessRaises + + + *note assertAlmostEqual(): bb7. failUnlessAlmostEqual assertAlmostEquals + + + *note assertNotAlmostEqual(): bb8. failIfAlmostEqual assertNotAlmostEquals + + + *note assertRegex(): bb1. assertRegexpMatches + + + *note assertNotRegex(): 3057. assertNotRegexpMatches + + + *note assertRaisesRegex(): abb. assertRaisesRegexp + + + Deprecated since version 3.1: The fail* aliases listed in the + second column have been deprecated. + + Deprecated since version 3.2: The assert* aliases listed in the + third column have been deprecated. + + Deprecated since version 3.2: ‘assertRegexpMatches’ and + ‘assertRaisesRegexp’ have been renamed to *note assertRegex(): bb1. + and *note assertRaisesRegex(): abb. + + Deprecated since version 3.5: The ‘assertNotRegexpMatches’ name is + deprecated in favor of *note assertNotRegex(): 3057. + + +File: python.info, Node: Grouping tests, Next: Loading and running tests, Prev: Test cases, Up: Classes and functions + +5.26.4.12 Grouping tests +........................ + + -- Class: unittest.TestSuite (tests=()) + + This class represents an aggregation of individual test cases and + test suites. The class presents the interface needed by the test + runner to allow it to be run as any other test case. Running a + *note TestSuite: 6ce. instance is the same as iterating over the + suite, running each test individually. + + If `tests' is given, it must be an iterable of individual test + cases or other test suites that will be used to build the suite + initially. Additional methods are provided to add test cases and + suites to the collection later on. + + *note TestSuite: 6ce. objects behave much like *note TestCase: 8ff. + objects, except they do not actually implement a test. Instead, + they are used to aggregate tests into groups of tests that should + be run together. Some additional methods are available to add + tests to *note TestSuite: 6ce. instances: + + -- Method: addTest (test) + + Add a *note TestCase: 8ff. or *note TestSuite: 6ce. to the + suite. + + -- Method: addTests (tests) + + Add all the tests from an iterable of *note TestCase: 8ff. and + *note TestSuite: 6ce. instances to this test suite. + + This is equivalent to iterating over `tests', calling *note + addTest(): 3067. for each element. + + *note TestSuite: 6ce. shares the following methods with *note + TestCase: 8ff.: + + -- Method: run (result) + + Run the tests associated with this suite, collecting the + result into the test result object passed as `result'. Note + that unlike *note TestCase.run(): abe, *note TestSuite.run(): + 3069. requires the result object to be passed in. + + -- Method: debug () + + Run the tests associated with this suite without collecting + the result. This allows exceptions raised by the test to be + propagated to the caller and can be used to support running + tests under a debugger. + + -- Method: countTestCases () + + Return the number of tests represented by this test object, + including all individual tests and sub-suites. + + -- Method: __iter__ () + + Tests grouped by a *note TestSuite: 6ce. are always accessed + by iteration. Subclasses can lazily provide tests by + overriding *note __iter__(): 962. Note that this method may + be called several times on a single suite (for example when + counting tests or comparing for equality) so the tests + returned by repeated iterations before *note TestSuite.run(): + 3069. must be the same for each call iteration. After *note + TestSuite.run(): 3069, callers should not rely on the tests + returned by this method unless the caller uses a subclass that + overrides ‘TestSuite._removeTestAtIndex()’ to preserve test + references. + + Changed in version 3.2: In earlier versions the *note + TestSuite: 6ce. accessed tests directly rather than through + iteration, so overriding *note __iter__(): 962. wasn’t + sufficient for providing tests. + + Changed in version 3.4: In earlier versions the *note + TestSuite: 6ce. held references to each *note TestCase: 8ff. + after *note TestSuite.run(): 3069. Subclasses can restore + that behavior by overriding ‘TestSuite._removeTestAtIndex()’. + + In the typical usage of a *note TestSuite: 6ce. object, the *note + run(): 3069. method is invoked by a ‘TestRunner’ rather than by the + end-user test harness. + + +File: python.info, Node: Loading and running tests, Prev: Grouping tests, Up: Classes and functions + +5.26.4.13 Loading and running tests +................................... + + -- Class: unittest.TestLoader + + The *note TestLoader: 782. class is used to create test suites from + classes and modules. Normally, there is no need to create an + instance of this class; the *note unittest: 11b. module provides an + instance that can be shared as *note unittest.defaultTestLoader: + 306d. Using a subclass or instance, however, allows customization + of some configurable properties. + + *note TestLoader: 782. objects have the following attributes: + + -- Attribute: errors + + A list of the non-fatal errors encountered while loading + tests. Not reset by the loader at any point. Fatal errors + are signalled by the relevant a method raising an exception to + the caller. Non-fatal errors are also indicated by a + synthetic test that will raise the original error when run. + + New in version 3.5. + + *note TestLoader: 782. objects have the following methods: + + -- Method: loadTestsFromTestCase (testCaseClass) + + Return a suite of all test cases contained in the *note + TestCase: 8ff.-derived ‘testCaseClass’. + + A test case instance is created for each method named by *note + getTestCaseNames(): 306f. By default these are the method + names beginning with ‘test’. If *note getTestCaseNames(): + 306f. returns no methods, but the ‘runTest()’ method is + implemented, a single test case is created for that method + instead. + + -- Method: loadTestsFromModule (module, pattern=None) + + Return a suite of all test cases contained in the given + module. This method searches `module' for classes derived + from *note TestCase: 8ff. and creates an instance of the class + for each test method defined for the class. + + Note: While using a hierarchy of *note TestCase: + 8ff.-derived classes can be convenient in sharing + fixtures and helper functions, defining test methods on + base classes that are not intended to be instantiated + directly does not play well with this method. Doing so, + however, can be useful when the fixtures are different + and defined in subclasses. + + If a module provides a ‘load_tests’ function it will be called + to load the tests. This allows modules to customize test + loading. This is the *note load_tests protocol: 3040. The + `pattern' argument is passed as the third argument to + ‘load_tests’. + + Changed in version 3.2: Support for ‘load_tests’ added. + + Changed in version 3.5: The undocumented and unofficial + `use_load_tests' default argument is deprecated and ignored, + although it is still accepted for backward compatibility. The + method also now accepts a keyword-only argument `pattern' + which is passed to ‘load_tests’ as the third argument. + + -- Method: loadTestsFromName (name, module=None) + + Return a suite of all test cases given a string specifier. + + The specifier `name' is a “dotted name” that may resolve + either to a module, a test case class, a test method within a + test case class, a *note TestSuite: 6ce. instance, or a + callable object which returns a *note TestCase: 8ff. or *note + TestSuite: 6ce. instance. These checks are applied in the + order listed here; that is, a method on a possible test case + class will be picked up as “a test method within a test case + class”, rather than “a callable object”. + + For example, if you have a module ‘SampleTests’ containing a + *note TestCase: 8ff.-derived class ‘SampleTestCase’ with three + test methods (‘test_one()’, ‘test_two()’, and ‘test_three()’), + the specifier ‘'SampleTests.SampleTestCase'’ would cause this + method to return a suite which will run all three test + methods. Using the specifier + ‘'SampleTests.SampleTestCase.test_two'’ would cause it to + return a test suite which will run only the ‘test_two()’ test + method. The specifier can refer to modules and packages which + have not been imported; they will be imported as a + side-effect. + + The method optionally resolves `name' relative to the given + `module'. + + Changed in version 3.5: If an *note ImportError: 334. or *note + AttributeError: 39b. occurs while traversing `name' then a + synthetic test that raises that error when run will be + returned. These errors are included in the errors accumulated + by self.errors. + + -- Method: loadTestsFromNames (names, module=None) + + Similar to *note loadTestsFromName(): cdf, but takes a + sequence of names rather than a single name. The return value + is a test suite which supports all the tests defined for each + name. + + -- Method: getTestCaseNames (testCaseClass) + + Return a sorted sequence of method names found within + `testCaseClass'; this should be a subclass of *note TestCase: + 8ff. + + -- Method: discover (start_dir, pattern='test*.py', + top_level_dir=None) + + Find all the test modules by recursing into subdirectories + from the specified start directory, and return a TestSuite + object containing them. Only test files that match `pattern' + will be loaded. (Using shell style pattern matching.) Only + module names that are importable (i.e. are valid Python + identifiers) will be loaded. + + All test modules must be importable from the top level of the + project. If the start directory is not the top level + directory then the top level directory must be specified + separately. + + If importing a module fails, for example due to a syntax + error, then this will be recorded as a single error and + discovery will continue. If the import failure is due to + *note SkipTest: 903. being raised, it will be recorded as a + skip instead of an error. + + If a package (a directory containing a file named + ‘__init__.py’) is found, the package will be checked for a + ‘load_tests’ function. If this exists then it will be called + ‘package.load_tests(loader, tests, pattern)’. Test discovery + takes care to ensure that a package is only checked for tests + once during an invocation, even if the load_tests function + itself calls ‘loader.discover’. + + If ‘load_tests’ exists then discovery does `not' recurse into + the package, ‘load_tests’ is responsible for loading all tests + in the package. + + The pattern is deliberately not stored as a loader attribute + so that packages can continue discovery themselves. + `top_level_dir' is stored so ‘load_tests’ does not need to + pass this argument in to ‘loader.discover()’. + + `start_dir' can be a dotted module name as well as a + directory. + + New in version 3.2. + + Changed in version 3.4: Modules that raise *note SkipTest: + 903. on import are recorded as skips, not errors. + + Changed in version 3.4: `start_dir' can be a *note namespace + packages: 1158. + + Changed in version 3.4: Paths are sorted before being imported + so that execution order is the same even if the underlying + file system’s ordering is not dependent on file name. + + Changed in version 3.5: Found packages are now checked for + ‘load_tests’ regardless of whether their path matches + `pattern', because it is impossible for a package name to + match the default pattern. + + The following attributes of a *note TestLoader: 782. can be + configured either by subclassing or assignment on an instance: + + -- Attribute: testMethodPrefix + + String giving the prefix of method names which will be + interpreted as test methods. The default value is ‘'test'’. + + This affects *note getTestCaseNames(): 306f. and all the + ‘loadTestsFrom*()’ methods. + + -- Attribute: sortTestMethodsUsing + + Function to be used to compare method names when sorting them + in *note getTestCaseNames(): 306f. and all the + ‘loadTestsFrom*()’ methods. + + -- Attribute: suiteClass + + Callable object that constructs a test suite from a list of + tests. No methods on the resulting object are needed. The + default value is the *note TestSuite: 6ce. class. + + This affects all the ‘loadTestsFrom*()’ methods. + + -- Attribute: testNamePatterns + + List of Unix shell-style wildcard test name patterns that test + methods have to match to be included in test suites (see ‘-v’ + option). + + If this attribute is not ‘None’ (the default), all test + methods to be included in test suites must match one of the + patterns in this list. Note that matches are always performed + using *note fnmatch.fnmatchcase(): 193b, so unlike patterns + passed to the ‘-v’ option, simple substring patterns will have + to be converted using ‘*’ wildcards. + + This affects all the ‘loadTestsFrom*()’ methods. + + New in version 3.7. + + -- Class: unittest.TestResult + + This class is used to compile information about which tests have + succeeded and which have failed. + + A *note TestResult: abf. object stores the results of a set of + tests. The *note TestCase: 8ff. and *note TestSuite: 6ce. classes + ensure that results are properly recorded; test authors do not need + to worry about recording the outcome of tests. + + Testing frameworks built on top of *note unittest: 11b. may want + access to the *note TestResult: abf. object generated by running a + set of tests for reporting purposes; a *note TestResult: abf. + instance is returned by the ‘TestRunner.run()’ method for this + purpose. + + *note TestResult: abf. instances have the following attributes that + will be of interest when inspecting the results of running a set of + tests: + + -- Attribute: errors + + A list containing 2-tuples of *note TestCase: 8ff. instances + and strings holding formatted tracebacks. Each tuple + represents a test which raised an unexpected exception. + + -- Attribute: failures + + A list containing 2-tuples of *note TestCase: 8ff. instances + and strings holding formatted tracebacks. Each tuple + represents a test where a failure was explicitly signalled + using the ‘TestCase.assert*()’ methods. + + -- Attribute: skipped + + A list containing 2-tuples of *note TestCase: 8ff. instances + and strings holding the reason for skipping the test. + + New in version 3.1. + + -- Attribute: expectedFailures + + A list containing 2-tuples of *note TestCase: 8ff. instances + and strings holding formatted tracebacks. Each tuple + represents an expected failure or error of the test case. + + -- Attribute: unexpectedSuccesses + + A list containing *note TestCase: 8ff. instances that were + marked as expected failures, but succeeded. + + -- Attribute: shouldStop + + Set to ‘True’ when the execution of tests should stop by *note + stop(): 307a. + + -- Attribute: testsRun + + The total number of tests run so far. + + -- Attribute: buffer + + If set to true, ‘sys.stdout’ and ‘sys.stderr’ will be buffered + in between *note startTest(): 307d. and *note stopTest(): + 307e. being called. Collected output will only be echoed onto + the real ‘sys.stdout’ and ‘sys.stderr’ if the test fails or + errors. Any output is also attached to the failure / error + message. + + New in version 3.2. + + -- Attribute: failfast + + If set to true *note stop(): 307a. will be called on the first + failure or error, halting the test run. + + New in version 3.2. + + -- Attribute: tb_locals + + If set to true then local variables will be shown in + tracebacks. + + New in version 3.5. + + -- Method: wasSuccessful () + + Return ‘True’ if all tests run so far have passed, otherwise + returns ‘False’. + + Changed in version 3.4: Returns ‘False’ if there were any + *note unexpectedSuccesses: 3078. from tests marked with the + *note expectedFailure(): 3049. decorator. + + -- Method: stop () + + This method can be called to signal that the set of tests + being run should be aborted by setting the *note shouldStop: + 3079. attribute to ‘True’. ‘TestRunner’ objects should + respect this flag and return without running any additional + tests. + + For example, this feature is used by the *note TextTestRunner: + 3082. class to stop the test framework when the user signals + an interrupt from the keyboard. Interactive tools which + provide ‘TestRunner’ implementations can use this in a similar + manner. + + The following methods of the *note TestResult: abf. class are used + to maintain the internal data structures, and may be extended in + subclasses to support additional reporting requirements. This is + particularly useful in building tools which support interactive + reporting while tests are being run. + + -- Method: startTest (test) + + Called when the test case `test' is about to be run. + + -- Method: stopTest (test) + + Called after the test case `test' has been executed, + regardless of the outcome. + + -- Method: startTestRun () + + Called once before any tests are executed. + + New in version 3.1. + + -- Method: stopTestRun () + + Called once after all tests are executed. + + New in version 3.1. + + -- Method: addError (test, err) + + Called when the test case `test' raises an unexpected + exception. `err' is a tuple of the form returned by *note + sys.exc_info(): c5f.: ‘(type, value, traceback)’. + + The default implementation appends a tuple ‘(test, + formatted_err)’ to the instance’s *note errors: 3074. + attribute, where `formatted_err' is a formatted traceback + derived from `err'. + + -- Method: addFailure (test, err) + + Called when the test case `test' signals a failure. `err' is + a tuple of the form returned by *note sys.exc_info(): c5f.: + ‘(type, value, traceback)’. + + The default implementation appends a tuple ‘(test, + formatted_err)’ to the instance’s *note failures: 3075. + attribute, where `formatted_err' is a formatted traceback + derived from `err'. + + -- Method: addSuccess (test) + + Called when the test case `test' succeeds. + + The default implementation does nothing. + + -- Method: addSkip (test, reason) + + Called when the test case `test' is skipped. `reason' is the + reason the test gave for skipping. + + The default implementation appends a tuple ‘(test, reason)’ to + the instance’s *note skipped: 3076. attribute. + + -- Method: addExpectedFailure (test, err) + + Called when the test case `test' fails or errors, but was + marked with the *note expectedFailure(): 3049. decorator. + + The default implementation appends a tuple ‘(test, + formatted_err)’ to the instance’s *note expectedFailures: + 3077. attribute, where `formatted_err' is a formatted + traceback derived from `err'. + + -- Method: addUnexpectedSuccess (test) + + Called when the test case `test' was marked with the *note + expectedFailure(): 3049. decorator, but succeeded. + + The default implementation appends the test to the instance’s + *note unexpectedSuccesses: 3078. attribute. + + -- Method: addSubTest (test, subtest, outcome) + + Called when a subtest finishes. `test' is the test case + corresponding to the test method. `subtest' is a custom *note + TestCase: 8ff. instance describing the subtest. + + If `outcome' is *note None: 157, the subtest succeeded. + Otherwise, it failed with an exception where `outcome' is a + tuple of the form returned by *note sys.exc_info(): c5f.: + ‘(type, value, traceback)’. + + The default implementation does nothing when the outcome is a + success, and records subtest failures as normal failures. + + New in version 3.4. + + -- Class: unittest.TextTestResult (stream, descriptions, verbosity) + + A concrete implementation of *note TestResult: abf. used by the + *note TextTestRunner: 3082. + + New in version 3.2: This class was previously named + ‘_TextTestResult’. The old name still exists as an alias but is + deprecated. + + -- Data: unittest.defaultTestLoader + + Instance of the *note TestLoader: 782. class intended to be shared. + If no customization of the *note TestLoader: 782. is needed, this + instance can be used instead of repeatedly creating new instances. + + -- Class: unittest.TextTestRunner (stream=None, descriptions=True, + verbosity=1, failfast=False, buffer=False, resultclass=None, + warnings=None, *, tb_locals=False) + + A basic test runner implementation that outputs results to a + stream. If `stream' is ‘None’, the default, *note sys.stderr: 30f. + is used as the output stream. This class has a few configurable + parameters, but is essentially very simple. Graphical applications + which run test suites should provide alternate implementations. + Such implementations should accept ‘**kwargs’ as the interface to + construct runners changes when features are added to unittest. + + By default this runner shows *note DeprecationWarning: 278, *note + PendingDeprecationWarning: 279, *note ResourceWarning: 4d0. and + *note ImportWarning: 5d8. even if they are *note ignored by + default: 308a. Deprecation warnings caused by *note deprecated + unittest methods: bb9. are also special-cased and, when the warning + filters are ‘'default'’ or ‘'always'’, they will appear only once + per-module, in order to avoid too many warning messages. This + behavior can be overridden using Python’s ‘-Wd’ or ‘-Wa’ options + (see *note Warning control: fe6.) and leaving `warnings' to ‘None’. + + Changed in version 3.2: Added the ‘warnings’ argument. + + Changed in version 3.2: The default stream is set to *note + sys.stderr: 30f. at instantiation time rather than import time. + + Changed in version 3.5: Added the tb_locals parameter. + + -- Method: _makeResult () + + This method returns the instance of ‘TestResult’ used by *note + run(): 308c. It is not intended to be called directly, but + can be overridden in subclasses to provide a custom + ‘TestResult’. + + ‘_makeResult()’ instantiates the class or callable passed in + the ‘TextTestRunner’ constructor as the ‘resultclass’ + argument. It defaults to *note TextTestResult: 305e. if no + ‘resultclass’ is provided. The result class is instantiated + with the following arguments: + + stream, descriptions, verbosity + + -- Method: run (test) + + This method is the main public interface to the + ‘TextTestRunner’. This method takes a *note TestSuite: 6ce. + or *note TestCase: 8ff. instance. A *note TestResult: abf. is + created by calling *note _makeResult(): 308b. and the test(s) + are run and the results printed to stdout. + + -- Function: unittest.main (module='__main__', defaultTest=None, + argv=None, testRunner=None, + testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, + failfast=None, catchbreak=None, buffer=None, warnings=None) + + A command-line program that loads a set of tests from `module' and + runs them; this is primarily for making test modules conveniently + executable. The simplest use for this function is to include the + following line at the end of a test script: + + if __name__ == '__main__': + unittest.main() + + You can run tests with more detailed information by passing in the + verbosity argument: + + if __name__ == '__main__': + unittest.main(verbosity=2) + + The `defaultTest' argument is either the name of a single test or + an iterable of test names to run if no test names are specified via + `argv'. If not specified or ‘None’ and no test names are provided + via `argv', all tests found in `module' are run. + + The `argv' argument can be a list of options passed to the program, + with the first element being the program name. If not specified or + ‘None’, the values of *note sys.argv: bfb. are used. + + The `testRunner' argument can either be a test runner class or an + already created instance of it. By default ‘main’ calls *note + sys.exit(): ce2. with an exit code indicating success or failure of + the tests run. + + The `testLoader' argument has to be a *note TestLoader: 782. + instance, and defaults to *note defaultTestLoader: 306d. + + ‘main’ supports being used from the interactive interpreter by + passing in the argument ‘exit=False’. This displays the result on + standard output without calling *note sys.exit(): ce2.: + + >>> from unittest import main + >>> main(module='test_module', exit=False) + + The `failfast', `catchbreak' and `buffer' parameters have the same + effect as the same-name *note command-line options: 3037. + + The `warnings' argument specifies the *note warning filter: fe7. + that should be used while running the tests. If it’s not + specified, it will remain ‘None’ if a ‘-W’ option is passed to + ‘python’ (see *note Warning control: fe6.), otherwise it will be + set to ‘'default'’. + + Calling ‘main’ actually returns an instance of the ‘TestProgram’ + class. This stores the result of the tests run as the ‘result’ + attribute. + + Changed in version 3.1: The `exit' parameter was added. + + Changed in version 3.2: The `verbosity', `failfast', `catchbreak', + `buffer' and `warnings' parameters were added. + + Changed in version 3.4: The `defaultTest' parameter was changed to + also accept an iterable of test names. + +* Menu: + +* load_tests Protocol:: + + +File: python.info, Node: load_tests Protocol, Up: Loading and running tests + +5.26.4.14 load_tests Protocol +............................. + +New in version 3.2. + +Modules or packages can customize how tests are loaded from them during +normal test runs or test discovery by implementing a function called +‘load_tests’. + +If a test module defines ‘load_tests’ it will be called by *note +TestLoader.loadTestsFromModule(): 780. with the following arguments: + + load_tests(loader, standard_tests, pattern) + +where `pattern' is passed straight through from ‘loadTestsFromModule’. +It defaults to ‘None’. + +It should return a *note TestSuite: 6ce. + +`loader' is the instance of *note TestLoader: 782. doing the loading. +`standard_tests' are the tests that would be loaded by default from the +module. It is common for test modules to only want to add or remove +tests from the standard set of tests. The third argument is used when +loading packages as part of test discovery. + +A typical ‘load_tests’ function that loads tests from a specific set of +*note TestCase: 8ff. classes may look like: + + test_cases = (TestCase1, TestCase2, TestCase3) + + def load_tests(loader, tests, pattern): + suite = TestSuite() + for test_class in test_cases: + tests = loader.loadTestsFromTestCase(test_class) + suite.addTests(tests) + return suite + +If discovery is started in a directory containing a package, either from +the command line or by calling *note TestLoader.discover(): 904, then +the package ‘__init__.py’ will be checked for ‘load_tests’. If that +function does not exist, discovery will recurse into the package as +though it were just another directory. Otherwise, discovery of the +package’s tests will be left up to ‘load_tests’ which is called with the +following arguments: + + load_tests(loader, standard_tests, pattern) + +This should return a *note TestSuite: 6ce. representing all the tests +from the package. (‘standard_tests’ will only contain tests collected +from ‘__init__.py’.) + +Because the pattern is passed into ‘load_tests’ the package is free to +continue (and potentially modify) test discovery. A ‘do nothing’ +‘load_tests’ function for a test package would look like: + + def load_tests(loader, standard_tests, pattern): + # top level directory cached on loader instance + this_dir = os.path.dirname(__file__) + package_tests = loader.discover(start_dir=this_dir, pattern=pattern) + standard_tests.addTests(package_tests) + return standard_tests + +Changed in version 3.5: Discovery no longer checks package names for +matching `pattern' due to the impossibility of package names matching +the default pattern. + + +File: python.info, Node: Class and Module Fixtures, Next: Signal Handling, Prev: Classes and functions, Up: unittest — Unit testing framework + +5.26.4.15 Class and Module Fixtures +................................... + +Class and module level fixtures are implemented in *note TestSuite: 6ce. +When the test suite encounters a test from a new class then +‘tearDownClass()’ from the previous class (if there is one) is called, +followed by ‘setUpClass()’ from the new class. + +Similarly if a test is from a different module from the previous test +then ‘tearDownModule’ from the previous module is run, followed by +‘setUpModule’ from the new module. + +After all the tests have run the final ‘tearDownClass’ and +‘tearDownModule’ are run. + +Note that shared fixtures do not play well with [potential] features +like test parallelization and they break test isolation. They should be +used with care. + +The default ordering of tests created by the unittest test loaders is to +group all tests from the same modules and classes together. This will +lead to ‘setUpClass’ / ‘setUpModule’ (etc) being called exactly once per +class and module. If you randomize the order, so that tests from +different modules and classes are adjacent to each other, then these +shared fixture functions may be called multiple times in a single test +run. + +Shared fixtures are not intended to work with suites with non-standard +ordering. A ‘BaseTestSuite’ still exists for frameworks that don’t want +to support shared fixtures. + +If there are any exceptions raised during one of the shared fixture +functions the test is reported as an error. Because there is no +corresponding test instance an ‘_ErrorHolder’ object (that has the same +interface as a *note TestCase: 8ff.) is created to represent the error. +If you are just using the standard unittest test runner then this detail +doesn’t matter, but if you are a framework author it may be relevant. + +* Menu: + +* setUpClass and tearDownClass:: +* setUpModule and tearDownModule:: + + +File: python.info, Node: setUpClass and tearDownClass, Next: setUpModule and tearDownModule, Up: Class and Module Fixtures + +5.26.4.16 setUpClass and tearDownClass +...................................... + +These must be implemented as class methods: + + import unittest + + class Test(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls._connection = createExpensiveConnectionObject() + + @classmethod + def tearDownClass(cls): + cls._connection.destroy() + +If you want the ‘setUpClass’ and ‘tearDownClass’ on base classes called +then you must call up to them yourself. The implementations in *note +TestCase: 8ff. are empty. + +If an exception is raised during a ‘setUpClass’ then the tests in the +class are not run and the ‘tearDownClass’ is not run. Skipped classes +will not have ‘setUpClass’ or ‘tearDownClass’ run. If the exception is +a *note SkipTest: 903. exception then the class will be reported as +having been skipped instead of as an error. + + +File: python.info, Node: setUpModule and tearDownModule, Prev: setUpClass and tearDownClass, Up: Class and Module Fixtures + +5.26.4.17 setUpModule and tearDownModule +........................................ + +These should be implemented as functions: + + def setUpModule(): + createConnection() + + def tearDownModule(): + closeConnection() + +If an exception is raised in a ‘setUpModule’ then none of the tests in +the module will be run and the ‘tearDownModule’ will not be run. If the +exception is a *note SkipTest: 903. exception then the module will be +reported as having been skipped instead of as an error. + +To add cleanup code that must be run even in the case of an exception, +use ‘addModuleCleanup’: + + -- Function: unittest.addModuleCleanup (function, /, *args, **kwargs) + + Add a function to be called after ‘tearDownModule()’ to cleanup + resources used during the test class. Functions will be called in + reverse order to the order they are added (LIFO). They are called + with any arguments and keyword arguments passed into *note + addModuleCleanup(): 24a. when they are added. + + If ‘setUpModule()’ fails, meaning that ‘tearDownModule()’ is not + called, then any cleanup functions added will still be called. + + New in version 3.8. + + -- Function: unittest.doModuleCleanups () + + This function is called unconditionally after ‘tearDownModule()’, + or after ‘setUpModule()’ if ‘setUpModule()’ raises an exception. + + It is responsible for calling all the cleanup functions added by + ‘addCleanupModule()’. If you need cleanup functions to be called + `prior' to ‘tearDownModule()’ then you can call *note + doModuleCleanups(): 308f. yourself. + + *note doModuleCleanups(): 308f. pops methods off the stack of + cleanup functions one at a time, so it can be called at any time. + + New in version 3.8. + + +File: python.info, Node: Signal Handling, Prev: Class and Module Fixtures, Up: unittest — Unit testing framework + +5.26.4.18 Signal Handling +......................... + +New in version 3.2. + +The *note -c/–catch: cc4. command-line option to unittest, along with +the ‘catchbreak’ parameter to *note unittest.main(): 902, provide more +friendly handling of control-C during a test run. With catch break +behavior enabled control-C will allow the currently running test to +complete, and the test run will then end and report all the results so +far. A second control-c will raise a *note KeyboardInterrupt: 197. in +the usual way. + +The control-c handling signal handler attempts to remain compatible with +code or tests that install their own *note signal.SIGINT: 21c3. handler. +If the ‘unittest’ handler is called but `isn’t' the installed *note +signal.SIGINT: 21c3. handler, i.e. it has been replaced by the system +under test and delegated to, then it calls the default handler. This +will normally be the expected behavior by code that replaces an +installed handler and delegates to it. For individual tests that need +‘unittest’ control-c handling disabled the *note removeHandler(): cc5. +decorator can be used. + +There are a few utility functions for framework authors to enable +control-c handling functionality within test frameworks. + + -- Function: unittest.installHandler () + + Install the control-c handler. When a *note signal.SIGINT: 21c3. + is received (usually in response to the user pressing control-c) + all registered results have *note stop(): 307a. called. + + -- Function: unittest.registerResult (result) + + Register a *note TestResult: abf. object for control-c handling. + Registering a result stores a weak reference to it, so it doesn’t + prevent the result from being garbage collected. + + Registering a *note TestResult: abf. object has no side-effects if + control-c handling is not enabled, so test frameworks can + unconditionally register all results they create independently of + whether or not handling is enabled. + + -- Function: unittest.removeResult (result) + + Remove a registered result. Once a result has been removed then + *note stop(): 307a. will no longer be called on that result object + in response to a control-c. + + -- Function: unittest.removeHandler (function=None) + + When called without arguments this function removes the control-c + handler if it has been installed. This function can also be used + as a test decorator to temporarily remove the handler while the + test is being executed: + + @unittest.removeHandler + def test_signal_handling(self): + ... + + +File: python.info, Node: unittest mock — mock object library, Next: unittest mock — getting started, Prev: unittest — Unit testing framework, Up: Development Tools + +5.26.5 ‘unittest.mock’ — mock object library +-------------------------------------------- + +New in version 3.3. + +`Source code:' Lib/unittest/mock.py(1) + +__________________________________________________________________ + +*note unittest.mock: 11c. is a library for testing in Python. It allows +you to replace parts of your system under test with mock objects and +make assertions about how they have been used. + +*note unittest.mock: 11c. provides a core *note Mock: 249. class +removing the need to create a host of stubs throughout your test suite. +After performing an action, you can make assertions about which methods +/ attributes were used and arguments they were called with. You can +also specify return values and set needed attributes in the normal way. + +Additionally, mock provides a *note patch(): 788. decorator that handles +patching module and class level attributes within the scope of a test, +along with *note sentinel: 40a. for creating unique objects. See the +*note quick guide: 3095. for some examples of how to use *note Mock: +249, *note MagicMock: 785. and *note patch(): 788. + +Mock is very easy to use and is designed for use with *note unittest: +11b. Mock is based on the ‘action -> assertion’ pattern instead of +‘record -> replay’ used by many mocking frameworks. + +There is a backport of *note unittest.mock: 11c. for earlier versions of +Python, available as mock on PyPI(2). + +* Menu: + +* Quick Guide:: +* The Mock Class:: +* The patchers:: +* MagicMock and magic method support:: +* Helpers:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/unittest/mock.py + + (2) https://pypi.org/project/mock + + +File: python.info, Node: Quick Guide, Next: The Mock Class, Up: unittest mock — mock object library + +5.26.5.1 Quick Guide +.................... + +*note Mock: 249. and *note MagicMock: 785. objects create all attributes +and methods as you access them and store details of how they have been +used. You can configure them, to specify return values or limit what +attributes are available, and then make assertions about how they have +been used: + + >>> from unittest.mock import MagicMock + >>> thing = ProductionClass() + >>> thing.method = MagicMock(return_value=3) + >>> thing.method(3, 4, 5, key='value') + 3 + >>> thing.method.assert_called_with(3, 4, 5, key='value') + +‘side_effect’ allows you to perform side effects, including raising an +exception when a mock is called: + + >>> mock = Mock(side_effect=KeyError('foo')) + >>> mock() + Traceback (most recent call last): + ... + KeyError: 'foo' + + >>> values = {'a': 1, 'b': 2, 'c': 3} + >>> def side_effect(arg): + ... return values[arg] + ... + >>> mock.side_effect = side_effect + >>> mock('a'), mock('b'), mock('c') + (1, 2, 3) + >>> mock.side_effect = [5, 4, 3, 2, 1] + >>> mock(), mock(), mock() + (5, 4, 3) + +Mock has many other ways you can configure it and control its behaviour. +For example the `spec' argument configures the mock to take its +specification from another object. Attempting to access attributes or +methods on the mock that don’t exist on the spec will fail with an *note +AttributeError: 39b. + +The *note patch(): 788. decorator / context manager makes it easy to +mock classes or objects in a module under test. The object you specify +will be replaced with a mock (or other object) during the test and +restored when the test ends: + + >>> from unittest.mock import patch + >>> @patch('module.ClassName2') + ... @patch('module.ClassName1') + ... def test(MockClass1, MockClass2): + ... module.ClassName1() + ... module.ClassName2() + ... assert MockClass1 is module.ClassName1 + ... assert MockClass2 is module.ClassName2 + ... assert MockClass1.called + ... assert MockClass2.called + ... + >>> test() + + Note: When you nest patch decorators the mocks are passed in to the + decorated function in the same order they applied (the normal + `Python' order that decorators are applied). This means from the + bottom up, so in the example above the mock for ‘module.ClassName1’ + is passed in first. + + With *note patch(): 788. it matters that you patch objects in the + namespace where they are looked up. This is normally + straightforward, but for a quick guide read *note where to patch: + 3096. + +As well as a decorator *note patch(): 788. can be used as a context +manager in a with statement: + + >>> with patch.object(ProductionClass, 'method', return_value=None) as mock_method: + ... thing = ProductionClass() + ... thing.method(1, 2, 3) + ... + >>> mock_method.assert_called_once_with(1, 2, 3) + +There is also *note patch.dict(): 3097. for setting values in a +dictionary just during a scope and restoring the dictionary to its +original state when the test ends: + + >>> foo = {'key': 'value'} + >>> original = foo.copy() + >>> with patch.dict(foo, {'newkey': 'newvalue'}, clear=True): + ... assert foo == {'newkey': 'newvalue'} + ... + >>> assert foo == original + +Mock supports the mocking of Python *note magic methods: 3098. The +easiest way of using magic methods is with the *note MagicMock: 785. +class. It allows you to do things like: + + >>> mock = MagicMock() + >>> mock.__str__.return_value = 'foobarbaz' + >>> str(mock) + 'foobarbaz' + >>> mock.__str__.assert_called_with() + +Mock allows you to assign functions (or other Mock instances) to magic +methods and they will be called appropriately. The *note MagicMock: +785. class is just a Mock variant that has all of the magic methods +pre-created for you (well, all the useful ones anyway). + +The following is an example of using magic methods with the ordinary +Mock class: + + >>> mock = Mock() + >>> mock.__str__ = Mock(return_value='wheeeeee') + >>> str(mock) + 'wheeeeee' + +For ensuring that the mock objects in your tests have the same api as +the objects they are replacing, you can use *note auto-speccing: 3099. +Auto-speccing can be done through the `autospec' argument to patch, or +the *note create_autospec(): 309a. function. Auto-speccing creates mock +objects that have the same attributes and methods as the objects they +are replacing, and any functions and methods (including constructors) +have the same call signature as the real object. + +This ensures that your mocks will fail in the same way as your +production code if they are used incorrectly: + + >>> from unittest.mock import create_autospec + >>> def function(a, b, c): + ... pass + ... + >>> mock_function = create_autospec(function, return_value='fishy') + >>> mock_function(1, 2, 3) + 'fishy' + >>> mock_function.assert_called_once_with(1, 2, 3) + >>> mock_function('wrong arguments') + Traceback (most recent call last): + ... + TypeError: <lambda>() takes exactly 3 arguments (1 given) + +*note create_autospec(): 309a. can also be used on classes, where it +copies the signature of the ‘__init__’ method, and on callable objects +where it copies the signature of the ‘__call__’ method. + + +File: python.info, Node: The Mock Class, Next: The patchers, Prev: Quick Guide, Up: unittest mock — mock object library + +5.26.5.2 The Mock Class +....................... + +*note Mock: 249. is a flexible mock object intended to replace the use +of stubs and test doubles throughout your code. Mocks are callable and +create attributes as new mocks when you access them (1). Accessing the +same attribute will always return the same mock. Mocks record how you +use them, allowing you to make assertions about what your code has done +to them. + +*note MagicMock: 785. is a subclass of *note Mock: 249. with all the +magic methods pre-created and ready to use. There are also non-callable +variants, useful when you are mocking out objects that aren’t callable: +*note NonCallableMock: 309c. and *note NonCallableMagicMock: 309d. + +The *note patch(): 788. decorators makes it easy to temporarily replace +classes in a particular module with a *note Mock: 249. object. By +default *note patch(): 788. will create a *note MagicMock: 785. for you. +You can specify an alternative class of *note Mock: 249. using the +`new_callable' argument to *note patch(): 788. + + -- Class: unittest.mock.Mock (spec=None, side_effect=None, + return_value=DEFAULT, wraps=None, name=None, spec_set=None, + unsafe=False, **kwargs) + + Create a new *note Mock: 249. object. *note Mock: 249. takes + several optional arguments that specify the behaviour of the Mock + object: + + * `spec': This can be either a list of strings or an existing + object (a class or instance) that acts as the specification + for the mock object. If you pass in an object then a list of + strings is formed by calling dir on the object (excluding + unsupported magic attributes and methods). Accessing any + attribute not in this list will raise an *note AttributeError: + 39b. + + If `spec' is an object (rather than a list of strings) then + *note __class__: e0a. returns the class of the spec object. + This allows mocks to pass *note isinstance(): 44f. tests. + + * `spec_set': A stricter variant of `spec'. If used, attempting + to `set' or get an attribute on the mock that isn’t on the + object passed as `spec_set' will raise an *note + AttributeError: 39b. + + * `side_effect': A function to be called whenever the Mock is + called. See the *note side_effect: 309e. attribute. Useful + for raising exceptions or dynamically changing return values. + The function is called with the same arguments as the mock, + and unless it returns *note DEFAULT: 309f, the return value of + this function is used as the return value. + + Alternatively `side_effect' can be an exception class or + instance. In this case the exception will be raised when the + mock is called. + + If `side_effect' is an iterable then each call to the mock + will return the next value from the iterable. + + A `side_effect' can be cleared by setting it to ‘None’. + + * `return_value': The value returned when the mock is called. + By default this is a new Mock (created on first access). See + the *note return_value: 30a0. attribute. + + * `unsafe': By default if any attribute starts with `assert' or + `assret' will raise an *note AttributeError: 39b. Passing + ‘unsafe=True’ will allow access to these attributes. + + New in version 3.5. + + * `wraps': Item for the mock object to wrap. If `wraps' is not + ‘None’ then calling the Mock will pass the call through to the + wrapped object (returning the real result). Attribute access + on the mock will return a Mock object that wraps the + corresponding attribute of the wrapped object (so attempting + to access an attribute that doesn’t exist will raise an *note + AttributeError: 39b.). + + If the mock has an explicit `return_value' set then calls are + not passed to the wrapped object and the `return_value' is + returned instead. + + * `name': If the mock has a name then it will be used in the + repr of the mock. This can be useful for debugging. The name + is propagated to child mocks. + + Mocks can also be called with arbitrary keyword arguments. These + will be used to set attributes on the mock after it is created. + See the *note configure_mock(): 30a1. method for details. + + -- Method: assert_called () + + Assert that the mock was called at least once. + + >>> mock = Mock() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called() + + New in version 3.6. + + -- Method: assert_called_once () + + Assert that the mock was called exactly once. + + >>> mock = Mock() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called_once() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called_once() + Traceback (most recent call last): + ... + AssertionError: Expected 'method' to have been called once. Called 2 times. + + New in version 3.6. + + -- Method: assert_called_with (*args, **kwargs) + + This method is a convenient way of asserting that the last + call has been made in a particular way: + + >>> mock = Mock() + >>> mock.method(1, 2, 3, test='wow') + <Mock name='mock.method()' id='...'> + >>> mock.method.assert_called_with(1, 2, 3, test='wow') + + -- Method: assert_called_once_with (*args, **kwargs) + + Assert that the mock was called exactly once and that that + call was with the specified arguments. + + >>> mock = Mock(return_value=None) + >>> mock('foo', bar='baz') + >>> mock.assert_called_once_with('foo', bar='baz') + >>> mock('other', bar='values') + >>> mock.assert_called_once_with('other', bar='values') + Traceback (most recent call last): + ... + AssertionError: Expected 'mock' to be called once. Called 2 times. + + -- Method: assert_any_call (*args, **kwargs) + + assert the mock has been called with the specified arguments. + + The assert passes if the mock has `ever' been called, unlike + *note assert_called_with(): 30a2. and *note + assert_called_once_with(): 30a3. that only pass if the call is + the most recent one, and in the case of *note + assert_called_once_with(): 30a3. it must also be the only + call. + + >>> mock = Mock(return_value=None) + >>> mock(1, 2, arg='thing') + >>> mock('some', 'thing', 'else') + >>> mock.assert_any_call(1, 2, arg='thing') + + -- Method: assert_has_calls (calls, any_order=False) + + assert the mock has been called with the specified calls. The + *note mock_calls: 30a6. list is checked for the calls. + + If `any_order' is false then the calls must be sequential. + There can be extra calls before or after the specified calls. + + If `any_order' is true then the calls can be in any order, but + they must all appear in *note mock_calls: 30a6. + + >>> mock = Mock(return_value=None) + >>> mock(1) + >>> mock(2) + >>> mock(3) + >>> mock(4) + >>> calls = [call(2), call(3)] + >>> mock.assert_has_calls(calls) + >>> calls = [call(4), call(2), call(3)] + >>> mock.assert_has_calls(calls, any_order=True) + + -- Method: assert_not_called () + + Assert the mock was never called. + + >>> m = Mock() + >>> m.hello.assert_not_called() + >>> obj = m.hello() + >>> m.hello.assert_not_called() + Traceback (most recent call last): + ... + AssertionError: Expected 'hello' to not have been called. Called 1 times. + + New in version 3.5. + + -- Method: reset_mock (*, return_value=False, side_effect=False) + + The reset_mock method resets all the call attributes on a mock + object: + + >>> mock = Mock(return_value=None) + >>> mock('hello') + >>> mock.called + True + >>> mock.reset_mock() + >>> mock.called + False + + Changed in version 3.6: Added two keyword only argument to the + reset_mock function. + + This can be useful where you want to make a series of + assertions that reuse the same object. Note that *note + reset_mock(): 5aa. `doesn’t' clear the return value, *note + side_effect: 309e. or any child attributes you have set using + normal assignment by default. In case you want to reset + `return_value' or *note side_effect: 309e, then pass the + corresponding parameter as ‘True’. Child mocks and the return + value mock (if any) are reset as well. + + Note: `return_value', and *note side_effect: 309e. are + keyword only argument. + + -- Method: mock_add_spec (spec, spec_set=False) + + Add a spec to a mock. `spec' can either be an object or a + list of strings. Only attributes on the `spec' can be fetched + as attributes from the mock. + + If `spec_set' is true then only attributes on the spec can be + set. + + -- Method: attach_mock (mock, attribute) + + Attach a mock as an attribute of this one, replacing its name + and parent. Calls to the attached mock will be recorded in + the *note method_calls: 30a9. and *note mock_calls: 30a6. + attributes of this one. + + -- Method: configure_mock (**kwargs) + + Set attributes on the mock through keyword arguments. + + Attributes plus return values and side effects can be set on + child mocks using standard dot notation and unpacking a + dictionary in the method call: + + >>> mock = Mock() + >>> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} + >>> mock.configure_mock(**attrs) + >>> mock.method() + 3 + >>> mock.other() + Traceback (most recent call last): + ... + KeyError + + The same thing can be achieved in the constructor call to + mocks: + + >>> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} + >>> mock = Mock(some_attribute='eggs', **attrs) + >>> mock.some_attribute + 'eggs' + >>> mock.method() + 3 + >>> mock.other() + Traceback (most recent call last): + ... + KeyError + + *note configure_mock(): 30a1. exists to make it easier to do + configuration after the mock has been created. + + -- Method: __dir__ () + + *note Mock: 249. objects limit the results of ‘dir(some_mock)’ + to useful results. For mocks with a `spec' this includes all + the permitted attributes for the mock. + + See *note FILTER_DIR: 30ab. for what this filtering does, and + how to switch it off. + + -- Method: _get_child_mock (**kw) + + Create the child mocks for attributes and return value. By + default child mocks will be the same type as the parent. + Subclasses of Mock may want to override this to customize the + way child mocks are made. + + For non-callable mocks the callable variant will be used + (rather than any custom subclass). + + -- Attribute: called + + A boolean representing whether or not the mock object has been + called: + + >>> mock = Mock(return_value=None) + >>> mock.called + False + >>> mock() + >>> mock.called + True + + -- Attribute: call_count + + An integer telling you how many times the mock object has been + called: + + >>> mock = Mock(return_value=None) + >>> mock.call_count + 0 + >>> mock() + >>> mock() + >>> mock.call_count + 2 + + -- Attribute: return_value + + Set this to configure the value returned by calling the mock: + + >>> mock = Mock() + >>> mock.return_value = 'fish' + >>> mock() + 'fish' + + The default return value is a mock object and you can + configure it in the normal way: + + >>> mock = Mock() + >>> mock.return_value.attribute = sentinel.Attribute + >>> mock.return_value() + <Mock name='mock()()' id='...'> + >>> mock.return_value.assert_called_with() + + *note return_value: 30a0. can also be set in the constructor: + + >>> mock = Mock(return_value=3) + >>> mock.return_value + 3 + >>> mock() + 3 + + -- Attribute: side_effect + + This can either be a function to be called when the mock is + called, an iterable or an exception (class or instance) to be + raised. + + If you pass in a function it will be called with same + arguments as the mock and unless the function returns the + *note DEFAULT: 309f. singleton the call to the mock will then + return whatever the function returns. If the function returns + *note DEFAULT: 309f. then the mock will return its normal + value (from the *note return_value: 30a0.). + + If you pass in an iterable, it is used to retrieve an iterator + which must yield a value on every call. This value can either + be an exception instance to be raised, or a value to be + returned from the call to the mock (*note DEFAULT: 309f. + handling is identical to the function case). + + An example of a mock that raises an exception (to test + exception handling of an API): + + >>> mock = Mock() + >>> mock.side_effect = Exception('Boom!') + >>> mock() + Traceback (most recent call last): + ... + Exception: Boom! + + Using *note side_effect: 309e. to return a sequence of values: + + >>> mock = Mock() + >>> mock.side_effect = [3, 2, 1] + >>> mock(), mock(), mock() + (3, 2, 1) + + Using a callable: + + >>> mock = Mock(return_value=3) + >>> def side_effect(*args, **kwargs): + ... return DEFAULT + ... + >>> mock.side_effect = side_effect + >>> mock() + 3 + + *note side_effect: 309e. can be set in the constructor. + Here’s an example that adds one to the value the mock is + called with and returns it: + + >>> side_effect = lambda value: value + 1 + >>> mock = Mock(side_effect=side_effect) + >>> mock(3) + 4 + >>> mock(-8) + -7 + + Setting *note side_effect: 309e. to ‘None’ clears it: + + >>> m = Mock(side_effect=KeyError, return_value=3) + >>> m() + Traceback (most recent call last): + ... + KeyError + >>> m.side_effect = None + >>> m() + 3 + + -- Attribute: call_args + + This is either ‘None’ (if the mock hasn’t been called), or the + arguments that the mock was last called with. This will be in + the form of a tuple: the first member, which can also be + accessed through the ‘args’ property, is any ordered arguments + the mock was called with (or an empty tuple) and the second + member, which can also be accessed through the ‘kwargs’ + property, is any keyword arguments (or an empty dictionary). + + >>> mock = Mock(return_value=None) + >>> print(mock.call_args) + None + >>> mock() + >>> mock.call_args + call() + >>> mock.call_args == () + True + >>> mock(3, 4) + >>> mock.call_args + call(3, 4) + >>> mock.call_args == ((3, 4),) + True + >>> mock.call_args.args + (3, 4) + >>> mock.call_args.kwargs + {} + >>> mock(3, 4, 5, key='fish', next='w00t!') + >>> mock.call_args + call(3, 4, 5, key='fish', next='w00t!') + >>> mock.call_args.args + (3, 4, 5) + >>> mock.call_args.kwargs + {'key': 'fish', 'next': 'w00t!'} + + *note call_args: 30af, along with members of the lists *note + call_args_list: 30b0, *note method_calls: 30a9. and *note + mock_calls: 30a6. are *note call: 30b1. objects. These are + tuples, so they can be unpacked to get at the individual + arguments and make more complex assertions. See *note calls + as tuples: 30b2. + + Changed in version 3.8: Added ‘args’ and ‘kwargs’ properties. + + -- Attribute: call_args_list + + This is a list of all the calls made to the mock object in + sequence (so the length of the list is the number of times it + has been called). Before any calls have been made it is an + empty list. The *note call: 30b1. object can be used for + conveniently constructing lists of calls to compare with *note + call_args_list: 30b0. + + >>> mock = Mock(return_value=None) + >>> mock() + >>> mock(3, 4) + >>> mock(key='fish', next='w00t!') + >>> mock.call_args_list + [call(), call(3, 4), call(key='fish', next='w00t!')] + >>> expected = [(), ((3, 4),), ({'key': 'fish', 'next': 'w00t!'},)] + >>> mock.call_args_list == expected + True + + Members of *note call_args_list: 30b0. are *note call: 30b1. + objects. These can be unpacked as tuples to get at the + individual arguments. See *note calls as tuples: 30b2. + + -- Attribute: method_calls + + As well as tracking calls to themselves, mocks also track + calls to methods and attributes, and `their' methods and + attributes: + + >>> mock = Mock() + >>> mock.method() + <Mock name='mock.method()' id='...'> + >>> mock.property.method.attribute() + <Mock name='mock.property.method.attribute()' id='...'> + >>> mock.method_calls + [call.method(), call.property.method.attribute()] + + Members of *note method_calls: 30a9. are *note call: 30b1. + objects. These can be unpacked as tuples to get at the + individual arguments. See *note calls as tuples: 30b2. + + -- Attribute: mock_calls + + *note mock_calls: 30a6. records `all' calls to the mock + object, its methods, magic methods `and' return value mocks. + + >>> mock = MagicMock() + >>> result = mock(1, 2, 3) + >>> mock.first(a=3) + <MagicMock name='mock.first()' id='...'> + >>> mock.second() + <MagicMock name='mock.second()' id='...'> + >>> int(mock) + 1 + >>> result(1) + <MagicMock name='mock()()' id='...'> + >>> expected = [call(1, 2, 3), call.first(a=3), call.second(), + ... call.__int__(), call()(1)] + >>> mock.mock_calls == expected + True + + Members of *note mock_calls: 30a6. are *note call: 30b1. + objects. These can be unpacked as tuples to get at the + individual arguments. See *note calls as tuples: 30b2. + + Note: The way *note mock_calls: 30a6. are recorded means + that where nested calls are made, the parameters of + ancestor calls are not recorded and so will always + compare equal: + + >>> mock = MagicMock() + >>> mock.top(a=3).bottom() + <MagicMock name='mock.top().bottom()' id='...'> + >>> mock.mock_calls + [call.top(a=3), call.top().bottom()] + >>> mock.mock_calls[-1] == call.top(a=-1).bottom() + True + + -- Attribute: __class__ + + Normally the *note __class__: 30b3. attribute of an object + will return its type. For a mock object with a ‘spec’, + ‘__class__’ returns the spec class instead. This allows mock + objects to pass *note isinstance(): 44f. tests for the object + they are replacing / masquerading as: + + >>> mock = Mock(spec=3) + >>> isinstance(mock, int) + True + + *note __class__: 30b3. is assignable to, this allows a mock to + pass an *note isinstance(): 44f. check without forcing you to + use a spec: + + >>> mock = Mock() + >>> mock.__class__ = dict + >>> isinstance(mock, dict) + True + + -- Class: unittest.mock.NonCallableMock (spec=None, wraps=None, + name=None, spec_set=None, **kwargs) + + A non-callable version of *note Mock: 249. The constructor + parameters have the same meaning of *note Mock: 249, with the + exception of `return_value' and `side_effect' which have no meaning + on a non-callable mock. + +Mock objects that use a class or an instance as a ‘spec’ or ‘spec_set’ +are able to pass *note isinstance(): 44f. tests: + + >>> mock = Mock(spec=SomeClass) + >>> isinstance(mock, SomeClass) + True + >>> mock = Mock(spec_set=SomeClass()) + >>> isinstance(mock, SomeClass) + True + +The *note Mock: 249. classes have support for mocking magic methods. +See *note magic methods: 3098. for the full details. + +The mock classes and the *note patch(): 788. decorators all take +arbitrary keyword arguments for configuration. For the *note patch(): +788. decorators the keywords are passed to the constructor of the mock +being created. The keyword arguments are for configuring attributes of +the mock: + + >>> m = MagicMock(attribute=3, other='fish') + >>> m.attribute + 3 + >>> m.other + 'fish' + +The return value and side effect of child mocks can be set in the same +way, using dotted notation. As you can’t use dotted names directly in a +call you have to create a dictionary and unpack it using ‘**’: + + >>> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} + >>> mock = Mock(some_attribute='eggs', **attrs) + >>> mock.some_attribute + 'eggs' + >>> mock.method() + 3 + >>> mock.other() + Traceback (most recent call last): + ... + KeyError + +A callable mock which was created with a `spec' (or a `spec_set') will +introspect the specification object’s signature when matching calls to +the mock. Therefore, it can match the actual call’s arguments +regardless of whether they were passed positionally or by name: + + >>> def f(a, b, c): pass + ... + >>> mock = Mock(spec=f) + >>> mock(1, 2, c=3) + <Mock name='mock()' id='140161580456576'> + >>> mock.assert_called_with(1, 2, 3) + >>> mock.assert_called_with(a=1, b=2, c=3) + +This applies to *note assert_called_with(): 30a2, *note +assert_called_once_with(): 30a3, *note assert_has_calls(): 30a5. and +*note assert_any_call(): 30a4. When *note Autospeccing: 3099, it will +also apply to method calls on the mock object. + + Changed in version 3.4: Added signature introspection on specced + and autospecced mock objects. + + -- Class: unittest.mock.PropertyMock (*args, **kwargs) + + A mock intended to be used as a property, or other descriptor, on a + class. *note PropertyMock: 30b4. provides *note __get__(): 10dd. + and *note __set__(): 10e1. methods so you can specify a return + value when it is fetched. + + Fetching a *note PropertyMock: 30b4. instance from an object calls + the mock, with no args. Setting it calls the mock with the value + being set. + + >>> class Foo: + ... @property + ... def foo(self): + ... return 'something' + ... @foo.setter + ... def foo(self, value): + ... pass + ... + >>> with patch('__main__.Foo.foo', new_callable=PropertyMock) as mock_foo: + ... mock_foo.return_value = 'mockity-mock' + ... this_foo = Foo() + ... print(this_foo.foo) + ... this_foo.foo = 6 + ... + mockity-mock + >>> mock_foo.mock_calls + [call(), call(6)] + +Because of the way mock attributes are stored you can’t directly attach +a *note PropertyMock: 30b4. to a mock object. Instead you can attach it +to the mock type object: + + >>> m = MagicMock() + >>> p = PropertyMock(return_value=3) + >>> type(m).foo = p + >>> m.foo + 3 + >>> p.assert_called_once_with() + + -- Class: unittest.mock.AsyncMock (spec=None, side_effect=None, + return_value=DEFAULT, wraps=None, name=None, spec_set=None, + unsafe=False, **kwargs) + + An asynchronous version of *note Mock: 249. The *note AsyncMock: + 248. object will behave so the object is recognized as an async + function, and the result of a call is an awaitable. + + >>> mock = AsyncMock() + >>> asyncio.iscoroutinefunction(mock) + True + >>> inspect.isawaitable(mock()) + True + + The result of ‘mock()’ is an async function which will have the + outcome of ‘side_effect’ or ‘return_value’ after it has been + awaited: + + - if ‘side_effect’ is a function, the async function will return + the result of that function, + + - if ‘side_effect’ is an exception, the async function will + raise the exception, + + - if ‘side_effect’ is an iterable, the async function will + return the next value of the iterable, however, if the + sequence of result is exhausted, ‘StopAsyncIteration’ is + raised immediately, + + - if ‘side_effect’ is not defined, the async function will + return the value defined by ‘return_value’, hence, by default, + the async function returns a new *note AsyncMock: 248. object. + + Setting the `spec' of a *note Mock: 249. or *note MagicMock: 785. + to an async function will result in a coroutine object being + returned after calling. + + >>> async def async_func(): pass + ... + >>> mock = MagicMock(async_func) + >>> mock + <MagicMock spec='function' id='...'> + >>> mock() + <coroutine object AsyncMockMixin._mock_call at ...> + + Setting the `spec' of a *note Mock: 249, *note MagicMock: 785, or + *note AsyncMock: 248. to a class with asynchronous and synchronous + functions will automatically detect the synchronous functions and + set them as *note MagicMock: 785. (if the parent mock is *note + AsyncMock: 248. or *note MagicMock: 785.) or *note Mock: 249. (if + the parent mock is *note Mock: 249.). All asynchronous functions + will be *note AsyncMock: 248. + + >>> class ExampleClass: + ... def sync_foo(): + ... pass + ... async def async_foo(): + ... pass + ... + >>> a_mock = AsyncMock(ExampleClass) + >>> a_mock.sync_foo + <MagicMock name='mock.sync_foo' id='...'> + >>> a_mock.async_foo + <AsyncMock name='mock.async_foo' id='...'> + >>> mock = Mock(ExampleClass) + >>> mock.sync_foo + <Mock name='mock.sync_foo' id='...'> + >>> mock.async_foo + <AsyncMock name='mock.async_foo' id='...'> + + New in version 3.8. + + -- Method: assert_awaited () + + Assert that the mock was awaited at least once. Note that + this is separate from the object having been called, the + ‘await’ keyword must be used: + + >>> mock = AsyncMock() + >>> async def main(coroutine_mock): + ... await coroutine_mock + ... + >>> coroutine_mock = mock() + >>> mock.called + True + >>> mock.assert_awaited() + Traceback (most recent call last): + ... + AssertionError: Expected mock to have been awaited. + >>> asyncio.run(main(coroutine_mock)) + >>> mock.assert_awaited() + + -- Method: assert_awaited_once () + + Assert that the mock was awaited exactly once. + + >>> mock = AsyncMock() + >>> async def main(): + ... await mock() + ... + >>> asyncio.run(main()) + >>> mock.assert_awaited_once() + >>> asyncio.run(main()) + >>> mock.method.assert_awaited_once() + Traceback (most recent call last): + ... + AssertionError: Expected mock to have been awaited once. Awaited 2 times. + + -- Method: assert_awaited_with (*args, **kwargs) + + Assert that the last await was with the specified arguments. + + >>> mock = AsyncMock() + >>> async def main(*args, **kwargs): + ... await mock(*args, **kwargs) + ... + >>> asyncio.run(main('foo', bar='bar')) + >>> mock.assert_awaited_with('foo', bar='bar') + >>> mock.assert_awaited_with('other') + Traceback (most recent call last): + ... + AssertionError: expected call not found. + Expected: mock('other') + Actual: mock('foo', bar='bar') + + -- Method: assert_awaited_once_with (*args, **kwargs) + + Assert that the mock was awaited exactly once and with the + specified arguments. + + >>> mock = AsyncMock() + >>> async def main(*args, **kwargs): + ... await mock(*args, **kwargs) + ... + >>> asyncio.run(main('foo', bar='bar')) + >>> mock.assert_awaited_once_with('foo', bar='bar') + >>> asyncio.run(main('foo', bar='bar')) + >>> mock.assert_awaited_once_with('foo', bar='bar') + Traceback (most recent call last): + ... + AssertionError: Expected mock to have been awaited once. Awaited 2 times. + + -- Method: assert_any_await (*args, **kwargs) + + Assert the mock has ever been awaited with the specified + arguments. + + >>> mock = AsyncMock() + >>> async def main(*args, **kwargs): + ... await mock(*args, **kwargs) + ... + >>> asyncio.run(main('foo', bar='bar')) + >>> asyncio.run(main('hello')) + >>> mock.assert_any_await('foo', bar='bar') + >>> mock.assert_any_await('other') + Traceback (most recent call last): + ... + AssertionError: mock('other') await not found + + -- Method: assert_has_awaits (calls, any_order=False) + + Assert the mock has been awaited with the specified calls. + The *note await_args_list: 30bb. list is checked for the + awaits. + + If `any_order' is false then the awaits must be sequential. + There can be extra calls before or after the specified awaits. + + If `any_order' is true then the awaits can be in any order, + but they must all appear in *note await_args_list: 30bb. + + >>> mock = AsyncMock() + >>> async def main(*args, **kwargs): + ... await mock(*args, **kwargs) + ... + >>> calls = [call("foo"), call("bar")] + >>> mock.assert_has_awaits(calls) + Traceback (most recent call last): + ... + AssertionError: Awaits not found. + Expected: [call('foo'), call('bar')] + Actual: [] + >>> asyncio.run(main('foo')) + >>> asyncio.run(main('bar')) + >>> mock.assert_has_awaits(calls) + + -- Method: assert_not_awaited () + + Assert that the mock was never awaited. + + >>> mock = AsyncMock() + >>> mock.assert_not_awaited() + + -- Method: reset_mock (*args, **kwargs) + + See *note Mock.reset_mock(): 5aa. Also sets *note + await_count: 30be. to 0, *note await_args: 30bf. to None, and + clears the *note await_args_list: 30bb. + + -- Attribute: await_count + + An integer keeping track of how many times the mock object has + been awaited. + + >>> mock = AsyncMock() + >>> async def main(): + ... await mock() + ... + >>> asyncio.run(main()) + >>> mock.await_count + 1 + >>> asyncio.run(main()) + >>> mock.await_count + 2 + + -- Attribute: await_args + + This is either ‘None’ (if the mock hasn’t been awaited), or + the arguments that the mock was last awaited with. Functions + the same as *note Mock.call_args: 30af. + + >>> mock = AsyncMock() + >>> async def main(*args): + ... await mock(*args) + ... + >>> mock.await_args + >>> asyncio.run(main('foo')) + >>> mock.await_args + call('foo') + >>> asyncio.run(main('bar')) + >>> mock.await_args + call('bar') + + -- Attribute: await_args_list + + This is a list of all the awaits made to the mock object in + sequence (so the length of the list is the number of times it + has been awaited). Before any awaits have been made it is an + empty list. + + >>> mock = AsyncMock() + >>> async def main(*args): + ... await mock(*args) + ... + >>> mock.await_args_list + [] + >>> asyncio.run(main('foo')) + >>> mock.await_args_list + [call('foo')] + >>> asyncio.run(main('bar')) + >>> mock.await_args_list + [call('foo'), call('bar')] + +* Menu: + +* Calling:: +* Deleting Attributes:: +* Mock names and the name attribute:: +* Attaching Mocks as Attributes:: + + ---------- Footnotes ---------- + + (1) (1) The only exceptions are magic methods and attributes (those +that have leading and trailing double underscores). Mock doesn’t create +these but instead raises an *note AttributeError: 39b. This is because +the interpreter will often implicitly request these methods, and gets +`very' confused to get a new Mock object when it expects a magic method. +If you need magic method support see *note magic methods: 3098. + + +File: python.info, Node: Calling, Next: Deleting Attributes, Up: The Mock Class + +5.26.5.3 Calling +................ + +Mock objects are callable. The call will return the value set as the +*note return_value: 30a0. attribute. The default return value is a new +Mock object; it is created the first time the return value is accessed +(either explicitly or by calling the Mock) - but it is stored and the +same one returned each time. + +Calls made to the object will be recorded in the attributes like *note +call_args: 30af. and *note call_args_list: 30b0. + +If *note side_effect: 309e. is set then it will be called after the call +has been recorded, so if ‘side_effect’ raises an exception the call is +still recorded. + +The simplest way to make a mock raise an exception when called is to +make *note side_effect: 309e. an exception class or instance: + + >>> m = MagicMock(side_effect=IndexError) + >>> m(1, 2, 3) + Traceback (most recent call last): + ... + IndexError + >>> m.mock_calls + [call(1, 2, 3)] + >>> m.side_effect = KeyError('Bang!') + >>> m('two', 'three', 'four') + Traceback (most recent call last): + ... + KeyError: 'Bang!' + >>> m.mock_calls + [call(1, 2, 3), call('two', 'three', 'four')] + +If ‘side_effect’ is a function then whatever that function returns is +what calls to the mock return. The ‘side_effect’ function is called +with the same arguments as the mock. This allows you to vary the return +value of the call dynamically, based on the input: + + >>> def side_effect(value): + ... return value + 1 + ... + >>> m = MagicMock(side_effect=side_effect) + >>> m(1) + 2 + >>> m(2) + 3 + >>> m.mock_calls + [call(1), call(2)] + +If you want the mock to still return the default return value (a new +mock), or any set return value, then there are two ways of doing this. +Either return ‘mock.return_value’ from inside ‘side_effect’, or return +*note DEFAULT: 309f.: + + >>> m = MagicMock() + >>> def side_effect(*args, **kwargs): + ... return m.return_value + ... + >>> m.side_effect = side_effect + >>> m.return_value = 3 + >>> m() + 3 + >>> def side_effect(*args, **kwargs): + ... return DEFAULT + ... + >>> m.side_effect = side_effect + >>> m() + 3 + +To remove a ‘side_effect’, and return to the default behaviour, set the +‘side_effect’ to ‘None’: + + >>> m = MagicMock(return_value=6) + >>> def side_effect(*args, **kwargs): + ... return 3 + ... + >>> m.side_effect = side_effect + >>> m() + 3 + >>> m.side_effect = None + >>> m() + 6 + +The ‘side_effect’ can also be any iterable object. Repeated calls to +the mock will return values from the iterable (until the iterable is +exhausted and a *note StopIteration: 486. is raised): + + >>> m = MagicMock(side_effect=[1, 2, 3]) + >>> m() + 1 + >>> m() + 2 + >>> m() + 3 + >>> m() + Traceback (most recent call last): + ... + StopIteration + +If any members of the iterable are exceptions they will be raised +instead of returned: + + >>> iterable = (33, ValueError, 66) + >>> m = MagicMock(side_effect=iterable) + >>> m() + 33 + >>> m() + Traceback (most recent call last): + ... + ValueError + >>> m() + 66 + + +File: python.info, Node: Deleting Attributes, Next: Mock names and the name attribute, Prev: Calling, Up: The Mock Class + +5.26.5.4 Deleting Attributes +............................ + +Mock objects create attributes on demand. This allows them to pretend +to be objects of any type. + +You may want a mock object to return ‘False’ to a *note hasattr(): 447. +call, or raise an *note AttributeError: 39b. when an attribute is +fetched. You can do this by providing an object as a ‘spec’ for a mock, +but that isn’t always convenient. + +You “block” attributes by deleting them. Once deleted, accessing an +attribute will raise an *note AttributeError: 39b. + + >>> mock = MagicMock() + >>> hasattr(mock, 'm') + True + >>> del mock.m + >>> hasattr(mock, 'm') + False + >>> del mock.f + >>> mock.f + Traceback (most recent call last): + ... + AttributeError: f + + +File: python.info, Node: Mock names and the name attribute, Next: Attaching Mocks as Attributes, Prev: Deleting Attributes, Up: The Mock Class + +5.26.5.5 Mock names and the name attribute +.......................................... + +Since “name” is an argument to the *note Mock: 249. constructor, if you +want your mock object to have a “name” attribute you can’t just pass it +in at creation time. There are two alternatives. One option is to use +*note configure_mock(): 30a1.: + + >>> mock = MagicMock() + >>> mock.configure_mock(name='my_name') + >>> mock.name + 'my_name' + +A simpler option is to simply set the “name” attribute after mock +creation: + + >>> mock = MagicMock() + >>> mock.name = "foo" + + +File: python.info, Node: Attaching Mocks as Attributes, Prev: Mock names and the name attribute, Up: The Mock Class + +5.26.5.6 Attaching Mocks as Attributes +...................................... + +When you attach a mock as an attribute of another mock (or as the return +value) it becomes a “child” of that mock. Calls to the child are +recorded in the *note method_calls: 30a9. and *note mock_calls: 30a6. +attributes of the parent. This is useful for configuring child mocks +and then attaching them to the parent, or for attaching mocks to a +parent that records all calls to the children and allows you to make +assertions about the order of calls between mocks: + + >>> parent = MagicMock() + >>> child1 = MagicMock(return_value=None) + >>> child2 = MagicMock(return_value=None) + >>> parent.child1 = child1 + >>> parent.child2 = child2 + >>> child1(1) + >>> child2(2) + >>> parent.mock_calls + [call.child1(1), call.child2(2)] + +The exception to this is if the mock has a name. This allows you to +prevent the “parenting” if for some reason you don’t want it to happen. + + >>> mock = MagicMock() + >>> not_a_child = MagicMock(name='not-a-child') + >>> mock.attribute = not_a_child + >>> mock.attribute() + <MagicMock name='not-a-child()' id='...'> + >>> mock.mock_calls + [] + +Mocks created for you by *note patch(): 788. are automatically given +names. To attach mocks that have names to a parent you use the *note +attach_mock(): 30a8. method: + + >>> thing1 = object() + >>> thing2 = object() + >>> parent = MagicMock() + >>> with patch('__main__.thing1', return_value=None) as child1: + ... with patch('__main__.thing2', return_value=None) as child2: + ... parent.attach_mock(child1, 'child1') + ... parent.attach_mock(child2, 'child2') + ... child1('one') + ... child2('two') + ... + >>> parent.mock_calls + [call.child1('one'), call.child2('two')] + + +File: python.info, Node: The patchers, Next: MagicMock and magic method support, Prev: The Mock Class, Up: unittest mock — mock object library + +5.26.5.7 The patchers +..................... + +The patch decorators are used for patching objects only within the scope +of the function they decorate. They automatically handle the unpatching +for you, even if exceptions are raised. All of these functions can also +be used in with statements or as class decorators. + +* Menu: + +* patch:: +* patch.object: patch object. +* patch.dict: patch dict. +* patch.multiple: patch multiple. +* patch methods; start and stop: patch methods start and stop. +* patch builtins:: +* TEST_PREFIX:: +* Nesting Patch Decorators:: +* Where to patch:: +* Patching Descriptors and Proxy Objects:: + + +File: python.info, Node: patch, Next: patch object, Up: The patchers + +5.26.5.8 patch +.............. + + Note: *note patch(): 788. is straightforward to use. The key is to + do the patching in the right namespace. See the section *note + where to patch: 30c7. + + -- Function: unittest.mock.patch (target, new=DEFAULT, spec=None, + create=False, spec_set=None, autospec=None, new_callable=None, + **kwargs) + + *note patch(): 788. acts as a function decorator, class decorator + or a context manager. Inside the body of the function or with + statement, the `target' is patched with a `new' object. When the + function/with statement exits the patch is undone. + + If `new' is omitted, then the target is replaced with an *note + AsyncMock: 248. if the patched object is an async function or a + *note MagicMock: 785. otherwise. If *note patch(): 788. is used as + a decorator and `new' is omitted, the created mock is passed in as + an extra argument to the decorated function. If *note patch(): + 788. is used as a context manager the created mock is returned by + the context manager. + + `target' should be a string in the form + ‘'package.module.ClassName'’. The `target' is imported and the + specified object replaced with the `new' object, so the `target' + must be importable from the environment you are calling *note + patch(): 788. from. The target is imported when the decorated + function is executed, not at decoration time. + + The `spec' and `spec_set' keyword arguments are passed to the *note + MagicMock: 785. if patch is creating one for you. + + In addition you can pass ‘spec=True’ or ‘spec_set=True’, which + causes patch to pass in the object being mocked as the + spec/spec_set object. + + `new_callable' allows you to specify a different class, or callable + object, that will be called to create the `new' object. By default + *note AsyncMock: 248. is used for async functions and *note + MagicMock: 785. for the rest. + + A more powerful form of `spec' is `autospec'. If you set + ‘autospec=True’ then the mock will be created with a spec from the + object being replaced. All attributes of the mock will also have + the spec of the corresponding attribute of the object being + replaced. Methods and functions being mocked will have their + arguments checked and will raise a *note TypeError: 192. if they + are called with the wrong signature. For mocks replacing a class, + their return value (the ‘instance’) will have the same spec as the + class. See the *note create_autospec(): 309a. function and *note + Autospeccing: 3099. + + Instead of ‘autospec=True’ you can pass ‘autospec=some_object’ to + use an arbitrary object as the spec instead of the one being + replaced. + + By default *note patch(): 788. will fail to replace attributes that + don’t exist. If you pass in ‘create=True’, and the attribute + doesn’t exist, patch will create the attribute for you when the + patched function is called, and delete it again after the patched + function has exited. This is useful for writing tests against + attributes that your production code creates at runtime. It is off + by default because it can be dangerous. With it switched on you + can write passing tests against APIs that don’t actually exist! + + Note: + Changed in version 3.5: If you are patching builtins in a + module then you don’t need to pass ‘create=True’, it will be + added by default. + + Patch can be used as a ‘TestCase’ class decorator. It works by + decorating each test method in the class. This reduces the + boilerplate code when your test methods share a common patchings + set. *note patch(): 788. finds tests by looking for method names + that start with ‘patch.TEST_PREFIX’. By default this is ‘'test'’, + which matches the way *note unittest: 11b. finds tests. You can + specify an alternative prefix by setting ‘patch.TEST_PREFIX’. + + Patch can be used as a context manager, with the with statement. + Here the patching applies to the indented block after the with + statement. If you use “as” then the patched object will be bound + to the name after the “as”; very useful if *note patch(): 788. is + creating a mock object for you. + + *note patch(): 788. takes arbitrary keyword arguments. These will + be passed to the *note Mock: 249. (or `new_callable') on + construction. + + ‘patch.dict(...)’, ‘patch.multiple(...)’ and ‘patch.object(...)’ + are available for alternate use-cases. + +*note patch(): 788. as function decorator, creating the mock for you and +passing it into the decorated function: + + >>> @patch('__main__.SomeClass') + ... def function(normal_argument, mock_class): + ... print(mock_class is SomeClass) + ... + >>> function(None) + True + +Patching a class replaces the class with a *note MagicMock: 785. +`instance'. If the class is instantiated in the code under test then it +will be the *note return_value: 30a0. of the mock that will be used. + +If the class is instantiated multiple times you could use *note +side_effect: 309e. to return a new mock each time. Alternatively you +can set the `return_value' to be anything you want. + +To configure return values on methods of `instances' on the patched +class you must do this on the ‘return_value’. For example: + + >>> class Class: + ... def method(self): + ... pass + ... + >>> with patch('__main__.Class') as MockClass: + ... instance = MockClass.return_value + ... instance.method.return_value = 'foo' + ... assert Class() is instance + ... assert Class().method() == 'foo' + ... + +If you use `spec' or `spec_set' and *note patch(): 788. is replacing a +`class', then the return value of the created mock will have the same +spec. + + >>> Original = Class + >>> patcher = patch('__main__.Class', spec=True) + >>> MockClass = patcher.start() + >>> instance = MockClass() + >>> assert isinstance(instance, Original) + >>> patcher.stop() + +The `new_callable' argument is useful where you want to use an +alternative class to the default *note MagicMock: 785. for the created +mock. For example, if you wanted a *note NonCallableMock: 309c. to be +used: + + >>> thing = object() + >>> with patch('__main__.thing', new_callable=NonCallableMock) as mock_thing: + ... assert thing is mock_thing + ... thing() + ... + Traceback (most recent call last): + ... + TypeError: 'NonCallableMock' object is not callable + +Another use case might be to replace an object with an *note +io.StringIO: 82d. instance: + + >>> from io import StringIO + >>> def foo(): + ... print('Something') + ... + >>> @patch('sys.stdout', new_callable=StringIO) + ... def test(mock_stdout): + ... foo() + ... assert mock_stdout.getvalue() == 'Something\n' + ... + >>> test() + +When *note patch(): 788. is creating a mock for you, it is common that +the first thing you need to do is to configure the mock. Some of that +configuration can be done in the call to patch. Any arbitrary keywords +you pass into the call will be used to set attributes on the created +mock: + + >>> patcher = patch('__main__.thing', first='one', second='two') + >>> mock_thing = patcher.start() + >>> mock_thing.first + 'one' + >>> mock_thing.second + 'two' + +As well as attributes on the created mock attributes, like the *note +return_value: 30a0. and *note side_effect: 309e, of child mocks can also +be configured. These aren’t syntactically valid to pass in directly as +keyword arguments, but a dictionary with these as keys can still be +expanded into a *note patch(): 788. call using ‘**’: + + >>> config = {'method.return_value': 3, 'other.side_effect': KeyError} + >>> patcher = patch('__main__.thing', **config) + >>> mock_thing = patcher.start() + >>> mock_thing.method() + 3 + >>> mock_thing.other() + Traceback (most recent call last): + ... + KeyError + +By default, attempting to patch a function in a module (or a method or +an attribute in a class) that does not exist will fail with *note +AttributeError: 39b.: + + >>> @patch('sys.non_existing_attribute', 42) + ... def test(): + ... assert sys.non_existing_attribute == 42 + ... + >>> test() + Traceback (most recent call last): + ... + AttributeError: <module 'sys' (built-in)> does not have the attribute 'non_existing' + +but adding ‘create=True’ in the call to *note patch(): 788. will make +the previous example work as expected: + + >>> @patch('sys.non_existing_attribute', 42, create=True) + ... def test(mock_stdout): + ... assert sys.non_existing_attribute == 42 + ... + >>> test() + +Changed in version 3.8: *note patch(): 788. now returns an *note +AsyncMock: 248. if the target is an async function. + + +File: python.info, Node: patch object, Next: patch dict, Prev: patch, Up: The patchers + +5.26.5.9 patch.object +..................... + + -- Function: patch.object (target, attribute, new=DEFAULT, spec=None, + create=False, spec_set=None, autospec=None, new_callable=None, + **kwargs) + + patch the named member (`attribute') on an object (`target') with a + mock object. + + *note patch.object(): 30c9. can be used as a decorator, class + decorator or a context manager. Arguments `new', `spec', `create', + `spec_set', `autospec' and `new_callable' have the same meaning as + for *note patch(): 788. Like *note patch(): 788, *note + patch.object(): 30c9. takes arbitrary keyword arguments for + configuring the mock object it creates. + + When used as a class decorator *note patch.object(): 30c9. honours + ‘patch.TEST_PREFIX’ for choosing which methods to wrap. + +You can either call *note patch.object(): 30c9. with three arguments or +two arguments. The three argument form takes the object to be patched, +the attribute name and the object to replace the attribute with. + +When calling with the two argument form you omit the replacement object, +and a mock is created for you and passed in as an extra argument to the +decorated function: + + >>> @patch.object(SomeClass, 'class_method') + ... def test(mock_method): + ... SomeClass.class_method(3) + ... mock_method.assert_called_with(3) + ... + >>> test() + +`spec', `create' and the other arguments to *note patch.object(): 30c9. +have the same meaning as they do for *note patch(): 788. + + +File: python.info, Node: patch dict, Next: patch multiple, Prev: patch object, Up: The patchers + +5.26.5.10 patch.dict +.................... + + -- Function: patch.dict (in_dict, values=(), clear=False, **kwargs) + + Patch a dictionary, or dictionary like object, and restore the + dictionary to its original state after the test. + + `in_dict' can be a dictionary or a mapping like container. If it + is a mapping then it must at least support getting, setting and + deleting items plus iterating over keys. + + `in_dict' can also be a string specifying the name of the + dictionary, which will then be fetched by importing it. + + `values' can be a dictionary of values to set in the dictionary. + `values' can also be an iterable of ‘(key, value)’ pairs. + + If `clear' is true then the dictionary will be cleared before the + new values are set. + + *note patch.dict(): 3097. can also be called with arbitrary keyword + arguments to set values in the dictionary. + + Changed in version 3.8: *note patch.dict(): 3097. now returns the + patched dictionary when used as a context manager. + +*note patch.dict(): 3097. can be used as a context manager, decorator or +class decorator: + + >>> foo = {} + >>> @patch.dict(foo, {'newkey': 'newvalue'}) + ... def test(): + ... assert foo == {'newkey': 'newvalue'} + >>> test() + >>> assert foo == {} + +When used as a class decorator *note patch.dict(): 3097. honours +‘patch.TEST_PREFIX’ (default to ‘'test'’) for choosing which methods to +wrap: + + >>> import os + >>> import unittest + >>> from unittest.mock import patch + >>> @patch.dict('os.environ', {'newkey': 'newvalue'}) + ... class TestSample(unittest.TestCase): + ... def test_sample(self): + ... self.assertEqual(os.environ['newkey'], 'newvalue') + +If you want to use a different prefix for your test, you can inform the +patchers of the different prefix by setting ‘patch.TEST_PREFIX’. For +more details about how to change the value of see *note TEST_PREFIX: +30cb. + +*note patch.dict(): 3097. can be used to add members to a dictionary, or +simply let a test change a dictionary, and ensure the dictionary is +restored when the test ends. + + >>> foo = {} + >>> with patch.dict(foo, {'newkey': 'newvalue'}) as patched_foo: + ... assert foo == {'newkey': 'newvalue'} + ... assert patched_foo == {'newkey': 'newvalue'} + ... # You can add, update or delete keys of foo (or patched_foo, it's the same dict) + ... patched_foo['spam'] = 'eggs' + ... + >>> assert foo == {} + >>> assert patched_foo == {} + + >>> import os + >>> with patch.dict('os.environ', {'newkey': 'newvalue'}): + ... print(os.environ['newkey']) + ... + newvalue + >>> assert 'newkey' not in os.environ + +Keywords can be used in the *note patch.dict(): 3097. call to set values +in the dictionary: + + >>> mymodule = MagicMock() + >>> mymodule.function.return_value = 'fish' + >>> with patch.dict('sys.modules', mymodule=mymodule): + ... import mymodule + ... mymodule.function('some', 'args') + ... + 'fish' + +*note patch.dict(): 3097. can be used with dictionary like objects that +aren’t actually dictionaries. At the very minimum they must support +item getting, setting, deleting and either iteration or membership test. +This corresponds to the magic methods *note __getitem__(): 27c, *note +__setitem__(): c62, *note __delitem__(): c63. and either *note +__iter__(): 50f. or *note __contains__(): d28. + + >>> class Container: + ... def __init__(self): + ... self.values = {} + ... def __getitem__(self, name): + ... return self.values[name] + ... def __setitem__(self, name, value): + ... self.values[name] = value + ... def __delitem__(self, name): + ... del self.values[name] + ... def __iter__(self): + ... return iter(self.values) + ... + >>> thing = Container() + >>> thing['one'] = 1 + >>> with patch.dict(thing, one=2, two=3): + ... assert thing['one'] == 2 + ... assert thing['two'] == 3 + ... + >>> assert thing['one'] == 1 + >>> assert list(thing) == ['one'] + + +File: python.info, Node: patch multiple, Next: patch methods start and stop, Prev: patch dict, Up: The patchers + +5.26.5.11 patch.multiple +........................ + + -- Function: patch.multiple (target, spec=None, create=False, + spec_set=None, autospec=None, new_callable=None, **kwargs) + + Perform multiple patches in a single call. It takes the object to + be patched (either as an object or a string to fetch the object by + importing) and keyword arguments for the patches: + + with patch.multiple(settings, FIRST_PATCH='one', SECOND_PATCH='two'): + ... + + Use *note DEFAULT: 309f. as the value if you want *note + patch.multiple(): 30cd. to create mocks for you. In this case the + created mocks are passed into a decorated function by keyword, and + a dictionary is returned when *note patch.multiple(): 30cd. is used + as a context manager. + + *note patch.multiple(): 30cd. can be used as a decorator, class + decorator or a context manager. The arguments `spec', `spec_set', + `create', `autospec' and `new_callable' have the same meaning as + for *note patch(): 788. These arguments will be applied to `all' + patches done by *note patch.multiple(): 30cd. + + When used as a class decorator *note patch.multiple(): 30cd. + honours ‘patch.TEST_PREFIX’ for choosing which methods to wrap. + +If you want *note patch.multiple(): 30cd. to create mocks for you, then +you can use *note DEFAULT: 309f. as the value. If you use *note +patch.multiple(): 30cd. as a decorator then the created mocks are passed +into the decorated function by keyword. + + >>> thing = object() + >>> other = object() + + >>> @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT) + ... def test_function(thing, other): + ... assert isinstance(thing, MagicMock) + ... assert isinstance(other, MagicMock) + ... + >>> test_function() + +*note patch.multiple(): 30cd. can be nested with other ‘patch’ +decorators, but put arguments passed by keyword `after' any of the +standard arguments created by *note patch(): 788.: + + >>> @patch('sys.exit') + ... @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT) + ... def test_function(mock_exit, other, thing): + ... assert 'other' in repr(other) + ... assert 'thing' in repr(thing) + ... assert 'exit' in repr(mock_exit) + ... + >>> test_function() + +If *note patch.multiple(): 30cd. is used as a context manager, the value +returned by the context manager is a dictionary where created mocks are +keyed by name: + + >>> with patch.multiple('__main__', thing=DEFAULT, other=DEFAULT) as values: + ... assert 'other' in repr(values['other']) + ... assert 'thing' in repr(values['thing']) + ... assert values['thing'] is thing + ... assert values['other'] is other + ... + + +File: python.info, Node: patch methods start and stop, Next: patch builtins, Prev: patch multiple, Up: The patchers + +5.26.5.12 patch methods: start and stop +....................................... + +All the patchers have ‘start()’ and ‘stop()’ methods. These make it +simpler to do patching in ‘setUp’ methods or where you want to do +multiple patches without nesting decorators or with statements. + +To use them call *note patch(): 788, *note patch.object(): 30c9. or +*note patch.dict(): 3097. as normal and keep a reference to the returned +‘patcher’ object. You can then call ‘start()’ to put the patch in place +and ‘stop()’ to undo it. + +If you are using *note patch(): 788. to create a mock for you then it +will be returned by the call to ‘patcher.start’. + + >>> patcher = patch('package.module.ClassName') + >>> from package import module + >>> original = module.ClassName + >>> new_mock = patcher.start() + >>> assert module.ClassName is not original + >>> assert module.ClassName is new_mock + >>> patcher.stop() + >>> assert module.ClassName is original + >>> assert module.ClassName is not new_mock + +A typical use case for this might be for doing multiple patches in the +‘setUp’ method of a ‘TestCase’: + + >>> class MyTest(unittest.TestCase): + ... def setUp(self): + ... self.patcher1 = patch('package.module.Class1') + ... self.patcher2 = patch('package.module.Class2') + ... self.MockClass1 = self.patcher1.start() + ... self.MockClass2 = self.patcher2.start() + ... + ... def tearDown(self): + ... self.patcher1.stop() + ... self.patcher2.stop() + ... + ... def test_something(self): + ... assert package.module.Class1 is self.MockClass1 + ... assert package.module.Class2 is self.MockClass2 + ... + >>> MyTest('test_something').run() + + Caution: If you use this technique you must ensure that the + patching is “undone” by calling ‘stop’. This can be fiddlier than + you might think, because if an exception is raised in the ‘setUp’ + then ‘tearDown’ is not called. *note + unittest.TestCase.addCleanup(): 2a2. makes this easier: + + >>> class MyTest(unittest.TestCase): + ... def setUp(self): + ... patcher = patch('package.module.Class') + ... self.MockClass = patcher.start() + ... self.addCleanup(patcher.stop) + ... + ... def test_something(self): + ... assert package.module.Class is self.MockClass + ... + + As an added bonus you no longer need to keep a reference to the + ‘patcher’ object. + +It is also possible to stop all patches which have been started by using +*note patch.stopall(): 30d0. + + -- Function: patch.stopall () + + Stop all active patches. Only stops patches started with ‘start’. + + +File: python.info, Node: patch builtins, Next: TEST_PREFIX, Prev: patch methods start and stop, Up: The patchers + +5.26.5.13 patch builtins +........................ + +You can patch any builtins within a module. The following example +patches builtin *note ord(): 10c6.: + + >>> @patch('__main__.ord') + ... def test(mock_ord): + ... mock_ord.return_value = 101 + ... print(ord('c')) + ... + >>> test() + 101 + + +File: python.info, Node: TEST_PREFIX, Next: Nesting Patch Decorators, Prev: patch builtins, Up: The patchers + +5.26.5.14 TEST_PREFIX +..................... + +All of the patchers can be used as class decorators. When used in this +way they wrap every test method on the class. The patchers recognise +methods that start with ‘'test'’ as being test methods. This is the +same way that the *note unittest.TestLoader: 782. finds test methods by +default. + +It is possible that you want to use a different prefix for your tests. +You can inform the patchers of the different prefix by setting +‘patch.TEST_PREFIX’: + + >>> patch.TEST_PREFIX = 'foo' + >>> value = 3 + >>> + >>> @patch('__main__.value', 'not three') + ... class Thing: + ... def foo_one(self): + ... print(value) + ... def foo_two(self): + ... print(value) + ... + >>> + >>> Thing().foo_one() + not three + >>> Thing().foo_two() + not three + >>> value + 3 + + +File: python.info, Node: Nesting Patch Decorators, Next: Where to patch, Prev: TEST_PREFIX, Up: The patchers + +5.26.5.15 Nesting Patch Decorators +.................................. + +If you want to perform multiple patches then you can simply stack up the +decorators. + +You can stack up multiple patch decorators using this pattern: + + >>> @patch.object(SomeClass, 'class_method') + ... @patch.object(SomeClass, 'static_method') + ... def test(mock1, mock2): + ... assert SomeClass.static_method is mock1 + ... assert SomeClass.class_method is mock2 + ... SomeClass.static_method('foo') + ... SomeClass.class_method('bar') + ... return mock1, mock2 + ... + >>> mock1, mock2 = test() + >>> mock1.assert_called_once_with('foo') + >>> mock2.assert_called_once_with('bar') + +Note that the decorators are applied from the bottom upwards. This is +the standard way that Python applies decorators. The order of the +created mocks passed into your test function matches this order. + + +File: python.info, Node: Where to patch, Next: Patching Descriptors and Proxy Objects, Prev: Nesting Patch Decorators, Up: The patchers + +5.26.5.16 Where to patch +........................ + +*note patch(): 788. works by (temporarily) changing the object that a +`name' points to with another one. There can be many names pointing to +any individual object, so for patching to work you must ensure that you +patch the name used by the system under test. + +The basic principle is that you patch where an object is `looked up', +which is not necessarily the same place as where it is defined. A +couple of examples will help to clarify this. + +Imagine we have a project that we want to test with the following +structure: + + a.py + -> Defines SomeClass + + b.py + -> from a import SomeClass + -> some_function instantiates SomeClass + +Now we want to test ‘some_function’ but we want to mock out ‘SomeClass’ +using *note patch(): 788. The problem is that when we import module b, +which we will have to do then it imports ‘SomeClass’ from module a. If +we use *note patch(): 788. to mock out ‘a.SomeClass’ then it will have +no effect on our test; module b already has a reference to the `real' +‘SomeClass’ and it looks like our patching had no effect. + +The key is to patch out ‘SomeClass’ where it is used (or where it is +looked up). In this case ‘some_function’ will actually look up +‘SomeClass’ in module b, where we have imported it. The patching should +look like: + + @patch('b.SomeClass') + +However, consider the alternative scenario where instead of ‘from a +import SomeClass’ module b does ‘import a’ and ‘some_function’ uses +‘a.SomeClass’. Both of these import forms are common. In this case the +class we want to patch is being looked up in the module and so we have +to patch ‘a.SomeClass’ instead: + + @patch('a.SomeClass') + + +File: python.info, Node: Patching Descriptors and Proxy Objects, Prev: Where to patch, Up: The patchers + +5.26.5.17 Patching Descriptors and Proxy Objects +................................................ + +Both *note patch: 30c6. and *note patch.object: 30c8. correctly patch +and restore descriptors: class methods, static methods and properties. +You should patch these on the `class' rather than an instance. They +also work with `some' objects that proxy attribute access, like the +django settings object(1). + + ---------- Footnotes ---------- + + (1) +http://www.voidspace.org.uk/python/weblog/arch_d7_2010_12_04.shtml#e1198 + + +File: python.info, Node: MagicMock and magic method support, Next: Helpers, Prev: The patchers, Up: unittest mock — mock object library + +5.26.5.18 MagicMock and magic method support +............................................ + +* Menu: + +* Mocking Magic Methods:: +* Magic Mock:: + + +File: python.info, Node: Mocking Magic Methods, Next: Magic Mock, Up: MagicMock and magic method support + +5.26.5.19 Mocking Magic Methods +............................... + +*note Mock: 249. supports mocking the Python protocol methods, also +known as “magic methods”. This allows mock objects to replace +containers or other objects that implement Python protocols. + +Because magic methods are looked up differently from normal methods (1), +this support has been specially implemented. This means that only +specific magic methods are supported. The supported list includes +`almost' all of them. If there are any missing that you need please let +us know. + +You mock magic methods by setting the method you are interested in to a +function or a mock instance. If you are using a function then it `must' +take ‘self’ as the first argument (2). + + >>> def __str__(self): + ... return 'fooble' + ... + >>> mock = Mock() + >>> mock.__str__ = __str__ + >>> str(mock) + 'fooble' + + >>> mock = Mock() + >>> mock.__str__ = Mock() + >>> mock.__str__.return_value = 'fooble' + >>> str(mock) + 'fooble' + + >>> mock = Mock() + >>> mock.__iter__ = Mock(return_value=iter([])) + >>> list(mock) + [] + +One use case for this is for mocking objects used as context managers in +a *note with: 6e9. statement: + + >>> mock = Mock() + >>> mock.__enter__ = Mock(return_value='foo') + >>> mock.__exit__ = Mock(return_value=False) + >>> with mock as m: + ... assert m == 'foo' + ... + >>> mock.__enter__.assert_called_with() + >>> mock.__exit__.assert_called_with(None, None, None) + +Calls to magic methods do not appear in *note method_calls: 30a9, but +they are recorded in *note mock_calls: 30a6. + + Note: If you use the `spec' keyword argument to create a mock then + attempting to set a magic method that isn’t in the spec will raise + an *note AttributeError: 39b. + +The full list of supported magic methods is: + + * ‘__hash__’, ‘__sizeof__’, ‘__repr__’ and ‘__str__’ + + * ‘__dir__’, ‘__format__’ and ‘__subclasses__’ + + * ‘__round__’, ‘__floor__’, ‘__trunc__’ and ‘__ceil__’ + + * Comparisons: ‘__lt__’, ‘__gt__’, ‘__le__’, ‘__ge__’, ‘__eq__’ and + ‘__ne__’ + + * Container methods: ‘__getitem__’, ‘__setitem__’, ‘__delitem__’, + ‘__contains__’, ‘__len__’, ‘__iter__’, ‘__reversed__’ and + ‘__missing__’ + + * Context manager: ‘__enter__’, ‘__exit__’, ‘__aenter__’ and + ‘__aexit__’ + + * Unary numeric methods: ‘__neg__’, ‘__pos__’ and ‘__invert__’ + + * The numeric methods (including right hand and in-place variants): + ‘__add__’, ‘__sub__’, ‘__mul__’, ‘__matmul__’, ‘__div__’, + ‘__truediv__’, ‘__floordiv__’, ‘__mod__’, ‘__divmod__’, + ‘__lshift__’, ‘__rshift__’, ‘__and__’, ‘__xor__’, ‘__or__’, and + ‘__pow__’ + + * Numeric conversion methods: ‘__complex__’, ‘__int__’, ‘__float__’ + and ‘__index__’ + + * Descriptor methods: ‘__get__’, ‘__set__’ and ‘__delete__’ + + * Pickling: ‘__reduce__’, ‘__reduce_ex__’, ‘__getinitargs__’, + ‘__getnewargs__’, ‘__getstate__’ and ‘__setstate__’ + + * File system path representation: ‘__fspath__’ + + * Asynchronous iteration methods: ‘__aiter__’ and ‘__anext__’ + +Changed in version 3.8: Added support for *note +os.PathLike.__fspath__(): 4ec. + +Changed in version 3.8: Added support for ‘__aenter__’, ‘__aexit__’, +‘__aiter__’ and ‘__anext__’. + +The following methods exist but are `not' supported as they are either +in use by mock, can’t be set dynamically, or can cause problems: + + * ‘__getattr__’, ‘__setattr__’, ‘__init__’ and ‘__new__’ + + * ‘__prepare__’, ‘__instancecheck__’, ‘__subclasscheck__’, ‘__del__’ + + ---------- Footnotes ---------- + + (1) (2) Magic methods `should' be looked up on the class rather than +the instance. Different versions of Python are inconsistent about +applying this rule. The supported protocol methods should work with all +supported versions of Python. + + (2) (3) The function is basically hooked up to the class, but each +‘Mock’ instance is kept isolated from the others. + + +File: python.info, Node: Magic Mock, Prev: Mocking Magic Methods, Up: MagicMock and magic method support + +5.26.5.20 Magic Mock +.................... + +There are two ‘MagicMock’ variants: *note MagicMock: 785. and *note +NonCallableMagicMock: 309d. + + -- Class: unittest.mock.MagicMock (*args, **kw) + + ‘MagicMock’ is a subclass of *note Mock: 249. with default + implementations of most of the magic methods. You can use + ‘MagicMock’ without having to configure the magic methods yourself. + + The constructor parameters have the same meaning as for *note Mock: + 249. + + If you use the `spec' or `spec_set' arguments then `only' magic + methods that exist in the spec will be created. + + -- Class: unittest.mock.NonCallableMagicMock (*args, **kw) + + A non-callable version of *note MagicMock: 785. + + The constructor parameters have the same meaning as for *note + MagicMock: 785, with the exception of `return_value' and + `side_effect' which have no meaning on a non-callable mock. + +The magic methods are setup with *note MagicMock: 785. objects, so you +can configure them and use them in the usual way: + + >>> mock = MagicMock() + >>> mock[3] = 'fish' + >>> mock.__setitem__.assert_called_with(3, 'fish') + >>> mock.__getitem__.return_value = 'result' + >>> mock[2] + 'result' + +By default many of the protocol methods are required to return objects +of a specific type. These methods are preconfigured with a default +return value, so that they can be used without you having to do anything +if you aren’t interested in the return value. You can still `set' the +return value manually if you want to change the default. + +Methods and their defaults: + + * ‘__lt__’: ‘NotImplemented’ + + * ‘__gt__’: ‘NotImplemented’ + + * ‘__le__’: ‘NotImplemented’ + + * ‘__ge__’: ‘NotImplemented’ + + * ‘__int__’: ‘1’ + + * ‘__contains__’: ‘False’ + + * ‘__len__’: ‘0’ + + * ‘__iter__’: ‘iter([])’ + + * ‘__exit__’: ‘False’ + + * ‘__aexit__’: ‘False’ + + * ‘__complex__’: ‘1j’ + + * ‘__float__’: ‘1.0’ + + * ‘__bool__’: ‘True’ + + * ‘__index__’: ‘1’ + + * ‘__hash__’: default hash for the mock + + * ‘__str__’: default str for the mock + + * ‘__sizeof__’: default sizeof for the mock + +For example: + + >>> mock = MagicMock() + >>> int(mock) + 1 + >>> len(mock) + 0 + >>> list(mock) + [] + >>> object() in mock + False + +The two equality methods, *note __eq__(): 33f. and *note __ne__(): e56, +are special. They do the default equality comparison on identity, using +the *note side_effect: 309e. attribute, unless you change their return +value to return something else: + + >>> MagicMock() == 3 + False + >>> MagicMock() != 3 + True + >>> mock = MagicMock() + >>> mock.__eq__.return_value = True + >>> mock == 3 + True + +The return value of ‘MagicMock.__iter__()’ can be any iterable object +and isn’t required to be an iterator: + + >>> mock = MagicMock() + >>> mock.__iter__.return_value = ['a', 'b', 'c'] + >>> list(mock) + ['a', 'b', 'c'] + >>> list(mock) + ['a', 'b', 'c'] + +If the return value `is' an iterator, then iterating over it once will +consume it and subsequent iterations will result in an empty list: + + >>> mock.__iter__.return_value = iter(['a', 'b', 'c']) + >>> list(mock) + ['a', 'b', 'c'] + >>> list(mock) + [] + +‘MagicMock’ has all of the supported magic methods configured except for +some of the obscure and obsolete ones. You can still set these up if +you want. + +Magic methods that are supported but not setup by default in ‘MagicMock’ +are: + + * ‘__subclasses__’ + + * ‘__dir__’ + + * ‘__format__’ + + * ‘__get__’, ‘__set__’ and ‘__delete__’ + + * ‘__reversed__’ and ‘__missing__’ + + * ‘__reduce__’, ‘__reduce_ex__’, ‘__getinitargs__’, ‘__getnewargs__’, + ‘__getstate__’ and ‘__setstate__’ + + * ‘__getformat__’ and ‘__setformat__’ + + +File: python.info, Node: Helpers, Prev: MagicMock and magic method support, Up: unittest mock — mock object library + +5.26.5.21 Helpers +................. + +* Menu: + +* sentinel:: +* DEFAULT:: +* call:: +* create_autospec:: +* ANY:: +* FILTER_DIR:: +* mock_open:: +* Autospeccing:: +* Sealing mocks:: + + +File: python.info, Node: sentinel, Next: DEFAULT, Up: Helpers + +5.26.5.22 sentinel +.................. + + -- Data: unittest.mock.sentinel + + The ‘sentinel’ object provides a convenient way of providing unique + objects for your tests. + + Attributes are created on demand when you access them by name. + Accessing the same attribute will always return the same object. + The objects returned have a sensible repr so that test failure + messages are readable. + + Changed in version 3.7: The ‘sentinel’ attributes now preserve + their identity when they are *note copied: 26. or *note pickled: + ca. + +Sometimes when testing you need to test that a specific object is passed +as an argument to another method, or returned. It can be common to +create named sentinel objects to test this. *note sentinel: 40a. +provides a convenient way of creating and testing the identity of +objects like this. + +In this example we monkey patch ‘method’ to return +‘sentinel.some_object’: + + >>> real = ProductionClass() + >>> real.method = Mock(name="method") + >>> real.method.return_value = sentinel.some_object + >>> result = real.method() + >>> assert result is sentinel.some_object + >>> sentinel.some_object + sentinel.some_object + + +File: python.info, Node: DEFAULT, Next: call, Prev: sentinel, Up: Helpers + +5.26.5.23 DEFAULT +................. + + -- Data: unittest.mock.DEFAULT + + The *note DEFAULT: 309f. object is a pre-created sentinel (actually + ‘sentinel.DEFAULT’). It can be used by *note side_effect: 309e. + functions to indicate that the normal return value should be used. + + +File: python.info, Node: call, Next: create_autospec, Prev: DEFAULT, Up: Helpers + +5.26.5.24 call +.............. + + -- Function: unittest.mock.call (*args, **kwargs) + + *note call(): 30b1. is a helper object for making simpler + assertions, for comparing with *note call_args: 30af, *note + call_args_list: 30b0, *note mock_calls: 30a6. and *note + method_calls: 30a9. *note call(): 30b1. can also be used with + *note assert_has_calls(): 30a5. + + >>> m = MagicMock(return_value=None) + >>> m(1, 2, a='foo', b='bar') + >>> m() + >>> m.call_args_list == [call(1, 2, a='foo', b='bar'), call()] + True + + -- Method: call.call_list () + + For a call object that represents multiple calls, *note + call_list(): 30dd. returns a list of all the intermediate calls as + well as the final call. + +‘call_list’ is particularly useful for making assertions on “chained +calls”. A chained call is multiple calls on a single line of code. +This results in multiple entries in *note mock_calls: 30a6. on a mock. +Manually constructing the sequence of calls can be tedious. + +*note call_list(): 30dd. can construct the sequence of calls from the +same chained call: + + >>> m = MagicMock() + >>> m(1).method(arg='foo').other('bar')(2.0) + <MagicMock name='mock().method().other()()' id='...'> + >>> kall = call(1).method(arg='foo').other('bar')(2.0) + >>> kall.call_list() + [call(1), + call().method(arg='foo'), + call().method().other('bar'), + call().method().other()(2.0)] + >>> m.mock_calls == kall.call_list() + True +A ‘call’ object is either a tuple of (positional args, keyword args) or +(name, positional args, keyword args) depending on how it was +constructed. When you construct them yourself this isn’t particularly +interesting, but the ‘call’ objects that are in the *note +Mock.call_args: 30af, *note Mock.call_args_list: 30b0. and *note +Mock.mock_calls: 30a6. attributes can be introspected to get at the +individual arguments they contain. + +The ‘call’ objects in *note Mock.call_args: 30af. and *note +Mock.call_args_list: 30b0. are two-tuples of (positional args, keyword +args) whereas the ‘call’ objects in *note Mock.mock_calls: 30a6, along +with ones you construct yourself, are three-tuples of (name, positional +args, keyword args). + +You can use their “tupleness” to pull out the individual arguments for +more complex introspection and assertions. The positional arguments are +a tuple (an empty tuple if there are no positional arguments) and the +keyword arguments are a dictionary: + + >>> m = MagicMock(return_value=None) + >>> m(1, 2, 3, arg='one', arg2='two') + >>> kall = m.call_args + >>> kall.args + (1, 2, 3) + >>> kall.kwargs + {'arg': 'one', 'arg2': 'two'} + >>> kall.args is kall[0] + True + >>> kall.kwargs is kall[1] + True + + >>> m = MagicMock() + >>> m.foo(4, 5, 6, arg='two', arg2='three') + <MagicMock name='mock.foo()' id='...'> + >>> kall = m.mock_calls[0] + >>> name, args, kwargs = kall + >>> name + 'foo' + >>> args + (4, 5, 6) + >>> kwargs + {'arg': 'two', 'arg2': 'three'} + >>> name is m.mock_calls[0][0] + True + + +File: python.info, Node: create_autospec, Next: ANY, Prev: call, Up: Helpers + +5.26.5.25 create_autospec +......................... + + -- Function: unittest.mock.create_autospec (spec, spec_set=False, + instance=False, **kwargs) + + Create a mock object using another object as a spec. Attributes on + the mock will use the corresponding attribute on the `spec' object + as their spec. + + Functions or methods being mocked will have their arguments checked + to ensure that they are called with the correct signature. + + If `spec_set' is ‘True’ then attempting to set attributes that + don’t exist on the spec object will raise an *note AttributeError: + 39b. + + If a class is used as a spec then the return value of the mock (the + instance of the class) will have the same spec. You can use a + class as the spec for an instance object by passing + ‘instance=True’. The returned mock will only be callable if + instances of the mock are callable. + + *note create_autospec(): 309a. also takes arbitrary keyword + arguments that are passed to the constructor of the created mock. + +See *note Autospeccing: 3099. for examples of how to use auto-speccing +with *note create_autospec(): 309a. and the `autospec' argument to *note +patch(): 788. + +Changed in version 3.8: *note create_autospec(): 309a. now returns an +*note AsyncMock: 248. if the target is an async function. + + +File: python.info, Node: ANY, Next: FILTER_DIR, Prev: create_autospec, Up: Helpers + +5.26.5.26 ANY +............. + + -- Data: unittest.mock.ANY + +Sometimes you may need to make assertions about `some' of the arguments +in a call to mock, but either not care about some of the arguments or +want to pull them individually out of *note call_args: 30af. and make +more complex assertions on them. + +To ignore certain arguments you can pass in objects that compare equal +to `everything'. Calls to *note assert_called_with(): 30a2. and *note +assert_called_once_with(): 30a3. will then succeed no matter what was +passed in. + + >>> mock = Mock(return_value=None) + >>> mock('foo', bar=object()) + >>> mock.assert_called_once_with('foo', bar=ANY) + +*note ANY: 30e0. can also be used in comparisons with call lists like +*note mock_calls: 30a6.: + + >>> m = MagicMock(return_value=None) + >>> m(1) + >>> m(1, 2) + >>> m(object()) + >>> m.mock_calls == [call(1), call(1, 2), ANY] + True + + +File: python.info, Node: FILTER_DIR, Next: mock_open, Prev: ANY, Up: Helpers + +5.26.5.27 FILTER_DIR +.................... + + -- Data: unittest.mock.FILTER_DIR + +*note FILTER_DIR: 30ab. is a module level variable that controls the way +mock objects respond to *note dir(): d33. (only for Python 2.6 or more +recent). The default is ‘True’, which uses the filtering described +below, to only show useful members. If you dislike this filtering, or +need to switch it off for diagnostic purposes, then set ‘mock.FILTER_DIR += False’. + +With filtering on, ‘dir(some_mock)’ shows only useful attributes and +will include any dynamically created attributes that wouldn’t normally +be shown. If the mock was created with a `spec' (or `autospec' of +course) then all the attributes from the original are shown, even if +they haven’t been accessed yet: + + >>> dir(Mock()) + ['assert_any_call', + 'assert_called', + 'assert_called_once', + 'assert_called_once_with', + 'assert_called_with', + 'assert_has_calls', + 'assert_not_called', + 'attach_mock', + ... + >>> from urllib import request + >>> dir(Mock(spec=request)) + ['AbstractBasicAuthHandler', + 'AbstractDigestAuthHandler', + 'AbstractHTTPHandler', + 'BaseHandler', + ... + +Many of the not-very-useful (private to *note Mock: 249. rather than the +thing being mocked) underscore and double underscore prefixed attributes +have been filtered from the result of calling *note dir(): d33. on a +*note Mock: 249. If you dislike this behaviour you can switch it off by +setting the module level switch *note FILTER_DIR: 30ab.: + + >>> from unittest import mock + >>> mock.FILTER_DIR = False + >>> dir(mock.Mock()) + ['_NonCallableMock__get_return_value', + '_NonCallableMock__get_side_effect', + '_NonCallableMock__return_value_doc', + '_NonCallableMock__set_return_value', + '_NonCallableMock__set_side_effect', + '__call__', + '__class__', + ... + +Alternatively you can just use ‘vars(my_mock)’ (instance members) and +‘dir(type(my_mock))’ (type members) to bypass the filtering irrespective +of ‘mock.FILTER_DIR’. + + +File: python.info, Node: mock_open, Next: Autospeccing, Prev: FILTER_DIR, Up: Helpers + +5.26.5.28 mock_open +................... + + -- Function: unittest.mock.mock_open (mock=None, read_data=None) + + A helper function to create a mock to replace the use of *note + open(): 4f0. It works for *note open(): 4f0. called directly or + used as a context manager. + + The `mock' argument is the mock object to configure. If ‘None’ + (the default) then a *note MagicMock: 785. will be created for you, + with the API limited to methods or attributes available on standard + file handles. + + `read_data' is a string for the ‘read()’, *note readline(): 13ae, + and *note readlines(): 1438. methods of the file handle to return. + Calls to those methods will take data from `read_data' until it is + depleted. The mock of these methods is pretty simplistic: every + time the `mock' is called, the `read_data' is rewound to the start. + If you need more control over the data that you are feeding to the + tested code you will need to customize this mock for yourself. + When that is insufficient, one of the in-memory filesystem packages + on PyPI(1) can offer a realistic filesystem for testing. + + Changed in version 3.4: Added *note readline(): 13ae. and *note + readlines(): 1438. support. The mock of ‘read()’ changed to + consume `read_data' rather than returning it on each call. + + Changed in version 3.5: `read_data' is now reset on each call to + the `mock'. + + Changed in version 3.8: Added *note __iter__(): 50f. to + implementation so that iteration (such as in for loops) correctly + consumes `read_data'. + +Using *note open(): 4f0. as a context manager is a great way to ensure +your file handles are closed properly and is becoming common: + + with open('/some/path', 'w') as f: + f.write('something') + +The issue is that even if you mock out the call to *note open(): 4f0. it +is the `returned object' that is used as a context manager (and has +*note __enter__(): c95. and *note __exit__(): c96. called). + +Mocking context managers with a *note MagicMock: 785. is common enough +and fiddly enough that a helper function is useful. + + >>> m = mock_open() + >>> with patch('__main__.open', m): + ... with open('foo', 'w') as h: + ... h.write('some stuff') + ... + >>> m.mock_calls + [call('foo', 'w'), + call().__enter__(), + call().write('some stuff'), + call().__exit__(None, None, None)] + >>> m.assert_called_once_with('foo', 'w') + >>> handle = m() + >>> handle.write.assert_called_once_with('some stuff') + +And for reading files: + + >>> with patch('__main__.open', mock_open(read_data='bibble')) as m: + ... with open('foo') as h: + ... result = h.read() + ... + >>> m.assert_called_once_with('foo') + >>> assert result == 'bibble' + + ---------- Footnotes ---------- + + (1) https://pypi.org + + +File: python.info, Node: Autospeccing, Next: Sealing mocks, Prev: mock_open, Up: Helpers + +5.26.5.29 Autospeccing +...................... + +Autospeccing is based on the existing ‘spec’ feature of mock. It limits +the api of mocks to the api of an original object (the spec), but it is +recursive (implemented lazily) so that attributes of mocks only have the +same api as the attributes of the spec. In addition mocked functions / +methods have the same call signature as the original so they raise a +*note TypeError: 192. if they are called incorrectly. + +Before I explain how auto-speccing works, here’s why it is needed. + +*note Mock: 249. is a very powerful and flexible object, but it suffers +from two flaws when used to mock out objects from a system under test. +One of these flaws is specific to the *note Mock: 249. api and the other +is a more general problem with using mock objects. + +First the problem specific to *note Mock: 249. *note Mock: 249. has two +assert methods that are extremely handy: *note assert_called_with(): +30a2. and *note assert_called_once_with(): 30a3. + + >>> mock = Mock(name='Thing', return_value=None) + >>> mock(1, 2, 3) + >>> mock.assert_called_once_with(1, 2, 3) + >>> mock(1, 2, 3) + >>> mock.assert_called_once_with(1, 2, 3) + Traceback (most recent call last): + ... + AssertionError: Expected 'mock' to be called once. Called 2 times. + +Because mocks auto-create attributes on demand, and allow you to call +them with arbitrary arguments, if you misspell one of these assert +methods then your assertion is gone: + + >>> mock = Mock(name='Thing', return_value=None) + >>> mock(1, 2, 3) + >>> mock.assret_called_once_with(4, 5, 6) + +Your tests can pass silently and incorrectly because of the typo. + +The second issue is more general to mocking. If you refactor some of +your code, rename members and so on, any tests for code that is still +using the `old api' but uses mocks instead of the real objects will +still pass. This means your tests can all pass even though your code is +broken. + +Note that this is another reason why you need integration tests as well +as unit tests. Testing everything in isolation is all fine and dandy, +but if you don’t test how your units are “wired together” there is still +lots of room for bugs that tests might have caught. + +‘mock’ already provides a feature to help with this, called speccing. +If you use a class or instance as the ‘spec’ for a mock then you can +only access attributes on the mock that exist on the real class: + + >>> from urllib import request + >>> mock = Mock(spec=request.Request) + >>> mock.assret_called_with + Traceback (most recent call last): + ... + AttributeError: Mock object has no attribute 'assret_called_with' + +The spec only applies to the mock itself, so we still have the same +issue with any methods on the mock: + + >>> mock.has_data() + <mock.Mock object at 0x...> + >>> mock.has_data.assret_called_with() + +Auto-speccing solves this problem. You can either pass ‘autospec=True’ +to *note patch(): 788. / *note patch.object(): 30c9. or use the *note +create_autospec(): 309a. function to create a mock with a spec. If you +use the ‘autospec=True’ argument to *note patch(): 788. then the object +that is being replaced will be used as the spec object. Because the +speccing is done “lazily” (the spec is created as attributes on the mock +are accessed) you can use it with very complex or deeply nested objects +(like modules that import modules that import modules) without a big +performance hit. + +Here’s an example of it in use: + + >>> from urllib import request + >>> patcher = patch('__main__.request', autospec=True) + >>> mock_request = patcher.start() + >>> request is mock_request + True + >>> mock_request.Request + <MagicMock name='request.Request' spec='Request' id='...'> + +You can see that ‘request.Request’ has a spec. ‘request.Request’ takes +two arguments in the constructor (one of which is `self'). Here’s what +happens if we try to call it incorrectly: + + >>> req = request.Request() + Traceback (most recent call last): + ... + TypeError: <lambda>() takes at least 2 arguments (1 given) + +The spec also applies to instantiated classes (i.e. the return value of +specced mocks): + + >>> req = request.Request('foo') + >>> req + <NonCallableMagicMock name='request.Request()' spec='Request' id='...'> + +‘Request’ objects are not callable, so the return value of instantiating +our mocked out ‘request.Request’ is a non-callable mock. With the spec +in place any typos in our asserts will raise the correct error: + + >>> req.add_header('spam', 'eggs') + <MagicMock name='request.Request().add_header()' id='...'> + >>> req.add_header.assret_called_with + Traceback (most recent call last): + ... + AttributeError: Mock object has no attribute 'assret_called_with' + >>> req.add_header.assert_called_with('spam', 'eggs') + +In many cases you will just be able to add ‘autospec=True’ to your +existing *note patch(): 788. calls and then be protected against bugs +due to typos and api changes. + +As well as using `autospec' through *note patch(): 788. there is a *note +create_autospec(): 309a. for creating autospecced mocks directly: + + >>> from urllib import request + >>> mock_request = create_autospec(request) + >>> mock_request.Request('foo', 'bar') + <NonCallableMagicMock name='mock.Request()' spec='Request' id='...'> + +This isn’t without caveats and limitations however, which is why it is +not the default behaviour. In order to know what attributes are +available on the spec object, autospec has to introspect (access +attributes) the spec. As you traverse attributes on the mock a +corresponding traversal of the original object is happening under the +hood. If any of your specced objects have properties or descriptors +that can trigger code execution then you may not be able to use +autospec. On the other hand it is much better to design your objects so +that introspection is safe (1). + +A more serious problem is that it is common for instance attributes to +be created in the *note __init__(): d5e. method and not to exist on the +class at all. `autospec' can’t know about any dynamically created +attributes and restricts the api to visible attributes. + + >>> class Something: + ... def __init__(self): + ... self.a = 33 + ... + >>> with patch('__main__.Something', autospec=True): + ... thing = Something() + ... thing.a + ... + Traceback (most recent call last): + ... + AttributeError: Mock object has no attribute 'a' + +There are a few different ways of resolving this problem. The easiest, +but not necessarily the least annoying, way is to simply set the +required attributes on the mock after creation. Just because `autospec' +doesn’t allow you to fetch attributes that don’t exist on the spec it +doesn’t prevent you setting them: + + >>> with patch('__main__.Something', autospec=True): + ... thing = Something() + ... thing.a = 33 + ... + +There is a more aggressive version of both `spec' and `autospec' that +`does' prevent you setting non-existent attributes. This is useful if +you want to ensure your code only `sets' valid attributes too, but +obviously it prevents this particular scenario: + + >>> with patch('__main__.Something', autospec=True, spec_set=True): + ... thing = Something() + ... thing.a = 33 + ... + Traceback (most recent call last): + ... + AttributeError: Mock object has no attribute 'a' + +Probably the best way of solving the problem is to add class attributes +as default values for instance members initialised in *note __init__(): +d5e. Note that if you are only setting default attributes in *note +__init__(): d5e. then providing them via class attributes (shared +between instances of course) is faster too. e.g. + + class Something: + a = 33 + +This brings up another issue. It is relatively common to provide a +default value of ‘None’ for members that will later be an object of a +different type. ‘None’ would be useless as a spec because it wouldn’t +let you access `any' attributes or methods on it. As ‘None’ is `never' +going to be useful as a spec, and probably indicates a member that will +normally of some other type, autospec doesn’t use a spec for members +that are set to ‘None’. These will just be ordinary mocks (well - +MagicMocks): + + >>> class Something: + ... member = None + ... + >>> mock = create_autospec(Something) + >>> mock.member.foo.bar.baz() + <MagicMock name='mock.member.foo.bar.baz()' id='...'> + +If modifying your production classes to add defaults isn’t to your +liking then there are more options. One of these is simply to use an +instance as the spec rather than the class. The other is to create a +subclass of the production class and add the defaults to the subclass +without affecting the production class. Both of these require you to +use an alternative object as the spec. Thankfully *note patch(): 788. +supports this - you can simply pass the alternative object as the +`autospec' argument: + + >>> class Something: + ... def __init__(self): + ... self.a = 33 + ... + >>> class SomethingForTest(Something): + ... a = 33 + ... + >>> p = patch('__main__.Something', autospec=SomethingForTest) + >>> mock = p.start() + >>> mock.a + <NonCallableMagicMock name='Something.a' spec='int' id='...'> + + ---------- Footnotes ---------- + + (1) (4) This only applies to classes or already instantiated objects. +Calling a mocked class to create a mock instance `does not' create a +real instance. It is only attribute lookups - along with calls to *note +dir(): d33. - that are done. + + +File: python.info, Node: Sealing mocks, Prev: Autospeccing, Up: Helpers + +5.26.5.30 Sealing mocks +....................... + + -- Function: unittest.mock.seal (mock) + + Seal will disable the automatic creation of mocks when accessing an + attribute of the mock being sealed or any of its attributes that + are already mocks recursively. + + If a mock instance with a name or a spec is assigned to an + attribute it won’t be considered in the sealing chain. This allows + one to prevent seal from fixing part of the mock object. + + >>> mock = Mock() + >>> mock.submock.attribute1 = 2 + >>> mock.not_submock = mock.Mock(name="sample_name") + >>> seal(mock) + >>> mock.new_attribute # This will raise AttributeError. + >>> mock.submock.attribute2 # This will raise AttributeError. + >>> mock.not_submock.attribute2 # This won't raise. + + New in version 3.7. + + +File: python.info, Node: unittest mock — getting started, Next: 2to3 - Automated Python 2 to 3 code translation, Prev: unittest mock — mock object library, Up: Development Tools + +5.26.6 ‘unittest.mock’ — getting started +---------------------------------------- + +New in version 3.3. + +* Menu: + +* Using Mock:: +* Patch Decorators:: +* Further Examples:: + + +File: python.info, Node: Using Mock, Next: Patch Decorators, Up: unittest mock — getting started + +5.26.6.1 Using Mock +................... + +* Menu: + +* Mock Patching Methods:: +* Mock for Method Calls on an Object:: +* Mocking Classes:: +* Naming your mocks:: +* Tracking all Calls:: +* Setting Return Values and Attributes:: +* Raising exceptions with mocks:: +* Side effect functions and iterables:: +* Mocking asynchronous iterators:: +* Mocking asynchronous context manager:: +* Creating a Mock from an Existing Object:: + + +File: python.info, Node: Mock Patching Methods, Next: Mock for Method Calls on an Object, Up: Using Mock + +5.26.6.2 Mock Patching Methods +.............................. + +Common uses for *note Mock: 249. objects include: + + * Patching methods + + * Recording method calls on objects + +You might want to replace a method on an object to check that it is +called with the correct arguments by another part of the system: + + >>> real = SomeClass() + >>> real.method = MagicMock(name='method') + >>> real.method(3, 4, 5, key='value') + <MagicMock name='method()' id='...'> + +Once our mock has been used (‘real.method’ in this example) it has +methods and attributes that allow you to make assertions about how it +has been used. + + Note: In most of these examples the *note Mock: 249. and *note + MagicMock: 785. classes are interchangeable. As the ‘MagicMock’ is + the more capable class it makes a sensible one to use by default. + +Once the mock has been called its *note called: 30ad. attribute is set +to ‘True’. More importantly we can use the *note assert_called_with(): +30a2. or *note assert_called_once_with(): 30a3. method to check that it +was called with the correct arguments. + +This example tests that calling ‘ProductionClass().method’ results in a +call to the ‘something’ method: + + >>> class ProductionClass: + ... def method(self): + ... self.something(1, 2, 3) + ... def something(self, a, b, c): + ... pass + ... + >>> real = ProductionClass() + >>> real.something = MagicMock() + >>> real.method() + >>> real.something.assert_called_once_with(1, 2, 3) + + +File: python.info, Node: Mock for Method Calls on an Object, Next: Mocking Classes, Prev: Mock Patching Methods, Up: Using Mock + +5.26.6.3 Mock for Method Calls on an Object +........................................... + +In the last example we patched a method directly on an object to check +that it was called correctly. Another common use case is to pass an +object into a method (or some part of the system under test) and then +check that it is used in the correct way. + +The simple ‘ProductionClass’ below has a ‘closer’ method. If it is +called with an object then it calls ‘close’ on it. + + >>> class ProductionClass: + ... def closer(self, something): + ... something.close() + ... + +So to test it we need to pass in an object with a ‘close’ method and +check that it was called correctly. + + >>> real = ProductionClass() + >>> mock = Mock() + >>> real.closer(mock) + >>> mock.close.assert_called_with() + +We don’t have to do any work to provide the ‘close’ method on our mock. +Accessing close creates it. So, if ‘close’ hasn’t already been called +then accessing it in the test will create it, but *note +assert_called_with(): 30a2. will raise a failure exception. + + +File: python.info, Node: Mocking Classes, Next: Naming your mocks, Prev: Mock for Method Calls on an Object, Up: Using Mock + +5.26.6.4 Mocking Classes +........................ + +A common use case is to mock out classes instantiated by your code under +test. When you patch a class, then that class is replaced with a mock. +Instances are created by `calling the class'. This means you access the +“mock instance” by looking at the return value of the mocked class. + +In the example below we have a function ‘some_function’ that +instantiates ‘Foo’ and calls a method on it. The call to *note patch(): +788. replaces the class ‘Foo’ with a mock. The ‘Foo’ instance is the +result of calling the mock, so it is configured by modifying the mock +*note return_value: 30a0. + + >>> def some_function(): + ... instance = module.Foo() + ... return instance.method() + ... + >>> with patch('module.Foo') as mock: + ... instance = mock.return_value + ... instance.method.return_value = 'the result' + ... result = some_function() + ... assert result == 'the result' + + +File: python.info, Node: Naming your mocks, Next: Tracking all Calls, Prev: Mocking Classes, Up: Using Mock + +5.26.6.5 Naming your mocks +.......................... + +It can be useful to give your mocks a name. The name is shown in the +repr of the mock and can be helpful when the mock appears in test +failure messages. The name is also propagated to attributes or methods +of the mock: + + >>> mock = MagicMock(name='foo') + >>> mock + <MagicMock name='foo' id='...'> + >>> mock.method + <MagicMock name='foo.method' id='...'> + + +File: python.info, Node: Tracking all Calls, Next: Setting Return Values and Attributes, Prev: Naming your mocks, Up: Using Mock + +5.26.6.6 Tracking all Calls +........................... + +Often you want to track more than a single call to a method. The *note +mock_calls: 30a6. attribute records all calls to child attributes of the +mock - and also to their children. + + >>> mock = MagicMock() + >>> mock.method() + <MagicMock name='mock.method()' id='...'> + >>> mock.attribute.method(10, x=53) + <MagicMock name='mock.attribute.method()' id='...'> + >>> mock.mock_calls + [call.method(), call.attribute.method(10, x=53)] + +If you make an assertion about ‘mock_calls’ and any unexpected methods +have been called, then the assertion will fail. This is useful because +as well as asserting that the calls you expected have been made, you are +also checking that they were made in the right order and with no +additional calls: + +You use the *note call: 30b1. object to construct lists for comparing +with ‘mock_calls’: + + >>> expected = [call.method(), call.attribute.method(10, x=53)] + >>> mock.mock_calls == expected + True + +However, parameters to calls that return mocks are not recorded, which +means it is not possible to track nested calls where the parameters used +to create ancestors are important: + + >>> m = Mock() + >>> m.factory(important=True).deliver() + <Mock name='mock.factory().deliver()' id='...'> + >>> m.mock_calls[-1] == call.factory(important=False).deliver() + True + + +File: python.info, Node: Setting Return Values and Attributes, Next: Raising exceptions with mocks, Prev: Tracking all Calls, Up: Using Mock + +5.26.6.7 Setting Return Values and Attributes +............................................. + +Setting the return values on a mock object is trivially easy: + + >>> mock = Mock() + >>> mock.return_value = 3 + >>> mock() + 3 + +Of course you can do the same for methods on the mock: + + >>> mock = Mock() + >>> mock.method.return_value = 3 + >>> mock.method() + 3 + +The return value can also be set in the constructor: + + >>> mock = Mock(return_value=3) + >>> mock() + 3 + +If you need an attribute setting on your mock, just do it: + + >>> mock = Mock() + >>> mock.x = 3 + >>> mock.x + 3 + +Sometimes you want to mock up a more complex situation, like for example +‘mock.connection.cursor().execute("SELECT 1")’. If we wanted this call +to return a list, then we have to configure the result of the nested +call. + +We can use *note call: 30b1. to construct the set of calls in a “chained +call” like this for easy assertion afterwards: + + >>> mock = Mock() + >>> cursor = mock.connection.cursor.return_value + >>> cursor.execute.return_value = ['foo'] + >>> mock.connection.cursor().execute("SELECT 1") + ['foo'] + >>> expected = call.connection.cursor().execute("SELECT 1").call_list() + >>> mock.mock_calls + [call.connection.cursor(), call.connection.cursor().execute('SELECT 1')] + >>> mock.mock_calls == expected + True + +It is the call to ‘.call_list()’ that turns our call object into a list +of calls representing the chained calls. + + +File: python.info, Node: Raising exceptions with mocks, Next: Side effect functions and iterables, Prev: Setting Return Values and Attributes, Up: Using Mock + +5.26.6.8 Raising exceptions with mocks +...................................... + +A useful attribute is *note side_effect: 309e. If you set this to an +exception class or instance then the exception will be raised when the +mock is called. + + >>> mock = Mock(side_effect=Exception('Boom!')) + >>> mock() + Traceback (most recent call last): + ... + Exception: Boom! + + +File: python.info, Node: Side effect functions and iterables, Next: Mocking asynchronous iterators, Prev: Raising exceptions with mocks, Up: Using Mock + +5.26.6.9 Side effect functions and iterables +............................................ + +‘side_effect’ can also be set to a function or an iterable. The use +case for ‘side_effect’ as an iterable is where your mock is going to be +called several times, and you want each call to return a different +value. When you set ‘side_effect’ to an iterable every call to the mock +returns the next value from the iterable: + + >>> mock = MagicMock(side_effect=[4, 5, 6]) + >>> mock() + 4 + >>> mock() + 5 + >>> mock() + 6 + +For more advanced use cases, like dynamically varying the return values +depending on what the mock is called with, ‘side_effect’ can be a +function. The function will be called with the same arguments as the +mock. Whatever the function returns is what the call returns: + + >>> vals = {(1, 2): 1, (2, 3): 2} + >>> def side_effect(*args): + ... return vals[args] + ... + >>> mock = MagicMock(side_effect=side_effect) + >>> mock(1, 2) + 1 + >>> mock(2, 3) + 2 + + +File: python.info, Node: Mocking asynchronous iterators, Next: Mocking asynchronous context manager, Prev: Side effect functions and iterables, Up: Using Mock + +5.26.6.10 Mocking asynchronous iterators +........................................ + +Since Python 3.8, ‘AsyncMock’ and ‘MagicMock’ have support to mock *note +Asynchronous Iterators: 646. through ‘__aiter__’. The *note +return_value: 30a0. attribute of ‘__aiter__’ can be used to set the +return values to be used for iteration. + + >>> mock = MagicMock() # AsyncMock also works here + >>> mock.__aiter__.return_value = [1, 2, 3] + >>> async def main(): + ... return [i async for i in mock] + ... + >>> asyncio.run(main()) + [1, 2, 3] + + +File: python.info, Node: Mocking asynchronous context manager, Next: Creating a Mock from an Existing Object, Prev: Mocking asynchronous iterators, Up: Using Mock + +5.26.6.11 Mocking asynchronous context manager +.............................................. + +Since Python 3.8, ‘AsyncMock’ and ‘MagicMock’ have support to mock *note +Asynchronous Context Managers: 1139. through ‘__aenter__’ and +‘__aexit__’. By default, ‘__aenter__’ and ‘__aexit__’ are ‘AsyncMock’ +instances that return an async function. + + >>> class AsyncContextManager: + ... async def __aenter__(self): + ... return self + ... async def __aexit__(self, exc_type, exc, tb): + ... pass + ... + >>> mock_instance = MagicMock(AsyncContextManager()) # AsyncMock also works here + >>> async def main(): + ... async with mock_instance as result: + ... pass + ... + >>> asyncio.run(main()) + >>> mock_instance.__aenter__.assert_awaited_once() + >>> mock_instance.__aexit__.assert_awaited_once() + + +File: python.info, Node: Creating a Mock from an Existing Object, Prev: Mocking asynchronous context manager, Up: Using Mock + +5.26.6.12 Creating a Mock from an Existing Object +................................................. + +One problem with over use of mocking is that it couples your tests to +the implementation of your mocks rather than your real code. Suppose +you have a class that implements ‘some_method’. In a test for another +class, you provide a mock of this object that `also' provides +‘some_method’. If later you refactor the first class, so that it no +longer has ‘some_method’ - then your tests will continue to pass even +though your code is now broken! + +*note Mock: 249. allows you to provide an object as a specification for +the mock, using the `spec' keyword argument. Accessing methods / +attributes on the mock that don’t exist on your specification object +will immediately raise an attribute error. If you change the +implementation of your specification, then tests that use that class +will start failing immediately without you having to instantiate the +class in those tests. + + >>> mock = Mock(spec=SomeClass) + >>> mock.old_method() + Traceback (most recent call last): + ... + AttributeError: object has no attribute 'old_method' + +Using a specification also enables a smarter matching of calls made to +the mock, regardless of whether some parameters were passed as +positional or named arguments: + + >>> def f(a, b, c): pass + ... + >>> mock = Mock(spec=f) + >>> mock(1, 2, 3) + <Mock name='mock()' id='140161580456576'> + >>> mock.assert_called_with(a=1, b=2, c=3) + +If you want this smarter matching to also work with method calls on the +mock, you can use *note auto-speccing: 3099. + +If you want a stronger form of specification that prevents the setting +of arbitrary attributes as well as the getting of them then you can use +`spec_set' instead of `spec'. + + +File: python.info, Node: Patch Decorators, Next: Further Examples, Prev: Using Mock, Up: unittest mock — getting started + +5.26.6.13 Patch Decorators +.......................... + + Note: With *note patch(): 788. it matters that you patch objects in + the namespace where they are looked up. This is normally + straightforward, but for a quick guide read *note where to patch: + 3096. + +A common need in tests is to patch a class attribute or a module +attribute, for example patching a builtin or patching a class in a +module to test that it is instantiated. Modules and classes are +effectively global, so patching on them has to be undone after the test +or the patch will persist into other tests and cause hard to diagnose +problems. + +mock provides three convenient decorators for this: *note patch(): 788, +*note patch.object(): 30c9. and *note patch.dict(): 3097. ‘patch’ takes +a single string, of the form ‘package.module.Class.attribute’ to specify +the attribute you are patching. It also optionally takes a value that +you want the attribute (or class or whatever) to be replaced with. +‘patch.object’ takes an object and the name of the attribute you would +like patched, plus optionally the value to patch it with. + +‘patch.object’: + + >>> original = SomeClass.attribute + >>> @patch.object(SomeClass, 'attribute', sentinel.attribute) + ... def test(): + ... assert SomeClass.attribute == sentinel.attribute + ... + >>> test() + >>> assert SomeClass.attribute == original + + >>> @patch('package.module.attribute', sentinel.attribute) + ... def test(): + ... from package.module import attribute + ... assert attribute is sentinel.attribute + ... + >>> test() + +If you are patching a module (including *note builtins: 13.) then use +*note patch(): 788. instead of *note patch.object(): 30c9.: + + >>> mock = MagicMock(return_value=sentinel.file_handle) + >>> with patch('builtins.open', mock): + ... handle = open('filename', 'r') + ... + >>> mock.assert_called_with('filename', 'r') + >>> assert handle == sentinel.file_handle, "incorrect file handle returned" + +The module name can be ‘dotted’, in the form ‘package.module’ if needed: + + >>> @patch('package.module.ClassName.attribute', sentinel.attribute) + ... def test(): + ... from package.module import ClassName + ... assert ClassName.attribute == sentinel.attribute + ... + >>> test() + +A nice pattern is to actually decorate test methods themselves: + + >>> class MyTest(unittest.TestCase): + ... @patch.object(SomeClass, 'attribute', sentinel.attribute) + ... def test_something(self): + ... self.assertEqual(SomeClass.attribute, sentinel.attribute) + ... + >>> original = SomeClass.attribute + >>> MyTest('test_something').test_something() + >>> assert SomeClass.attribute == original + +If you want to patch with a Mock, you can use *note patch(): 788. with +only one argument (or *note patch.object(): 30c9. with two arguments). +The mock will be created for you and passed into the test function / +method: + + >>> class MyTest(unittest.TestCase): + ... @patch.object(SomeClass, 'static_method') + ... def test_something(self, mock_method): + ... SomeClass.static_method() + ... mock_method.assert_called_with() + ... + >>> MyTest('test_something').test_something() + +You can stack up multiple patch decorators using this pattern: + + >>> class MyTest(unittest.TestCase): + ... @patch('package.module.ClassName1') + ... @patch('package.module.ClassName2') + ... def test_something(self, MockClass2, MockClass1): + ... self.assertIs(package.module.ClassName1, MockClass1) + ... self.assertIs(package.module.ClassName2, MockClass2) + ... + >>> MyTest('test_something').test_something() + +When you nest patch decorators the mocks are passed in to the decorated +function in the same order they applied (the normal `Python' order that +decorators are applied). This means from the bottom up, so in the +example above the mock for ‘test_module.ClassName2’ is passed in first. + +There is also *note patch.dict(): 3097. for setting values in a +dictionary just during a scope and restoring the dictionary to its +original state when the test ends: + + >>> foo = {'key': 'value'} + >>> original = foo.copy() + >>> with patch.dict(foo, {'newkey': 'newvalue'}, clear=True): + ... assert foo == {'newkey': 'newvalue'} + ... + >>> assert foo == original + +‘patch’, ‘patch.object’ and ‘patch.dict’ can all be used as context +managers. + +Where you use *note patch(): 788. to create a mock for you, you can get +a reference to the mock using the “as” form of the with statement: + + >>> class ProductionClass: + ... def method(self): + ... pass + ... + >>> with patch.object(ProductionClass, 'method') as mock_method: + ... mock_method.return_value = None + ... real = ProductionClass() + ... real.method(1, 2, 3) + ... + >>> mock_method.assert_called_with(1, 2, 3) + +As an alternative ‘patch’, ‘patch.object’ and ‘patch.dict’ can be used +as class decorators. When used in this way it is the same as applying +the decorator individually to every method whose name starts with +“test”. + + +File: python.info, Node: Further Examples, Prev: Patch Decorators, Up: unittest mock — getting started + +5.26.6.14 Further Examples +.......................... + +Here are some more examples for some slightly more advanced scenarios. + +* Menu: + +* Mocking chained calls:: +* Partial mocking:: +* Mocking a Generator Method:: +* Applying the same patch to every test method:: +* Mocking Unbound Methods:: +* Checking multiple calls with mock:: +* Coping with mutable arguments:: +* Nesting Patches:: +* Mocking a dictionary with MagicMock:: +* Mock subclasses and their attributes:: +* Mocking imports with patch.dict: Mocking imports with patch dict. +* Tracking order of calls and less verbose call assertions:: +* More complex argument matching:: + + +File: python.info, Node: Mocking chained calls, Next: Partial mocking, Up: Further Examples + +5.26.6.15 Mocking chained calls +............................... + +Mocking chained calls is actually straightforward with mock once you +understand the *note return_value: 30a0. attribute. When a mock is +called for the first time, or you fetch its ‘return_value’ before it has +been called, a new *note Mock: 249. is created. + +This means that you can see how the object returned from a call to a +mocked object has been used by interrogating the ‘return_value’ mock: + + >>> mock = Mock() + >>> mock().foo(a=2, b=3) + <Mock name='mock().foo()' id='...'> + >>> mock.return_value.foo.assert_called_with(a=2, b=3) + +From here it is a simple step to configure and then make assertions +about chained calls. Of course another alternative is writing your code +in a more testable way in the first place… + +So, suppose we have some code that looks a little bit like this: + + >>> class Something: + ... def __init__(self): + ... self.backend = BackendProvider() + ... def method(self): + ... response = self.backend.get_endpoint('foobar').create_call('spam', 'eggs').start_call() + ... # more code + +Assuming that ‘BackendProvider’ is already well tested, how do we test +‘method()’? Specifically, we want to test that the code section ‘# more +code’ uses the response object in the correct way. + +As this chain of calls is made from an instance attribute we can monkey +patch the ‘backend’ attribute on a ‘Something’ instance. In this +particular case we are only interested in the return value from the +final call to ‘start_call’ so we don’t have much configuration to do. +Let’s assume the object it returns is ‘file-like’, so we’ll ensure that +our response object uses the builtin *note open(): 4f0. as its ‘spec’. + +To do this we create a mock instance as our mock backend and create a +mock response object for it. To set the response as the return value +for that final ‘start_call’ we could do this: + + mock_backend.get_endpoint.return_value.create_call.return_value.start_call.return_value = mock_response + +We can do that in a slightly nicer way using the *note configure_mock(): +30a1. method to directly set the return value for us: + + >>> something = Something() + >>> mock_response = Mock(spec=open) + >>> mock_backend = Mock() + >>> config = {'get_endpoint.return_value.create_call.return_value.start_call.return_value': mock_response} + >>> mock_backend.configure_mock(**config) + +With these we monkey patch the “mock backend” in place and can make the +real call: + + >>> something.backend = mock_backend + >>> something.method() + +Using *note mock_calls: 30a6. we can check the chained call with a +single assert. A chained call is several calls in one line of code, so +there will be several entries in ‘mock_calls’. We can use *note +call.call_list(): 30dd. to create this list of calls for us: + + >>> chained = call.get_endpoint('foobar').create_call('spam', 'eggs').start_call() + >>> call_list = chained.call_list() + >>> assert mock_backend.mock_calls == call_list + + +File: python.info, Node: Partial mocking, Next: Mocking a Generator Method, Prev: Mocking chained calls, Up: Further Examples + +5.26.6.16 Partial mocking +......................... + +In some tests I wanted to mock out a call to *note +datetime.date.today(): 1551. to return a known date, but I didn’t want +to prevent the code under test from creating new date objects. +Unfortunately *note datetime.date: 193. is written in C, and so I +couldn’t just monkey-patch out the static ‘date.today()’ method. + +I found a simple way of doing this that involved effectively wrapping +the date class with a mock, but passing through calls to the constructor +to the real class (and returning real instances). + +The *note patch decorator: 788. is used here to mock out the ‘date’ +class in the module under test. The ‘side_effect’ attribute on the mock +date class is then set to a lambda function that returns a real date. +When the mock date class is called a real date will be constructed and +returned by ‘side_effect’. + + >>> from datetime import date + >>> with patch('mymodule.date') as mock_date: + ... mock_date.today.return_value = date(2010, 10, 8) + ... mock_date.side_effect = lambda *args, **kw: date(*args, **kw) + ... + ... assert mymodule.date.today() == date(2010, 10, 8) + ... assert mymodule.date(2009, 6, 8) == date(2009, 6, 8) + +Note that we don’t patch *note datetime.date: 193. globally, we patch +‘date’ in the module that `uses' it. See *note where to patch: 3096. + +When ‘date.today()’ is called a known date is returned, but calls to the +‘date(...)’ constructor still return normal dates. Without this you can +find yourself having to calculate an expected result using exactly the +same algorithm as the code under test, which is a classic testing +anti-pattern. + +Calls to the date constructor are recorded in the ‘mock_date’ attributes +(‘call_count’ and friends) which may also be useful for your tests. + +An alternative way of dealing with mocking dates, or other builtin +classes, is discussed in this blog entry(1). + + ---------- Footnotes ---------- + + (1) +https://williambert.online/2011/07/how-to-unit-testing-in-django-with-mocking-and-patching/ + + +File: python.info, Node: Mocking a Generator Method, Next: Applying the same patch to every test method, Prev: Partial mocking, Up: Further Examples + +5.26.6.17 Mocking a Generator Method +.................................... + +A Python generator is a function or method that uses the *note yield: +18f. statement to return a series of values when iterated over (1). + +A generator method / function is called to return the generator object. +It is the generator object that is then iterated over. The protocol +method for iteration is *note __iter__(): 12d4, so we can mock this +using a *note MagicMock: 785. + +Here’s an example class with an “iter” method implemented as a +generator: + + >>> class Foo: + ... def iter(self): + ... for i in [1, 2, 3]: + ... yield i + ... + >>> foo = Foo() + >>> list(foo.iter()) + [1, 2, 3] + +How would we mock this class, and in particular its “iter” method? + +To configure the values returned from the iteration (implicit in the +call to *note list: 262.), we need to configure the object returned by +the call to ‘foo.iter()’. + + >>> mock_foo = MagicMock() + >>> mock_foo.iter.return_value = iter([1, 2, 3]) + >>> list(mock_foo.iter()) + [1, 2, 3] + + ---------- Footnotes ---------- + + (1) (1) There are also generator expressions and more advanced uses +(http://www.dabeaz.com/coroutines/index.html) of generators, but we +aren’t concerned about them here. A very good introduction to +generators and how powerful they are is: Generator Tricks for Systems +Programmers (http://www.dabeaz.com/generators/). + + +File: python.info, Node: Applying the same patch to every test method, Next: Mocking Unbound Methods, Prev: Mocking a Generator Method, Up: Further Examples + +5.26.6.18 Applying the same patch to every test method +...................................................... + +If you want several patches in place for multiple test methods the +obvious way is to apply the patch decorators to every method. This can +feel like unnecessary repetition. For Python 2.6 or more recent you can +use *note patch(): 788. (in all its various forms) as a class decorator. +This applies the patches to all test methods on the class. A test +method is identified by methods whose names start with ‘test’: + + >>> @patch('mymodule.SomeClass') + ... class MyTest(unittest.TestCase): + ... + ... def test_one(self, MockSomeClass): + ... self.assertIs(mymodule.SomeClass, MockSomeClass) + ... + ... def test_two(self, MockSomeClass): + ... self.assertIs(mymodule.SomeClass, MockSomeClass) + ... + ... def not_a_test(self): + ... return 'something' + ... + >>> MyTest('test_one').test_one() + >>> MyTest('test_two').test_two() + >>> MyTest('test_two').not_a_test() + 'something' + +An alternative way of managing patches is to use the *note patch +methods; start and stop: 30cf. These allow you to move the patching +into your ‘setUp’ and ‘tearDown’ methods. + + >>> class MyTest(unittest.TestCase): + ... def setUp(self): + ... self.patcher = patch('mymodule.foo') + ... self.mock_foo = self.patcher.start() + ... + ... def test_foo(self): + ... self.assertIs(mymodule.foo, self.mock_foo) + ... + ... def tearDown(self): + ... self.patcher.stop() + ... + >>> MyTest('test_foo').run() + +If you use this technique you must ensure that the patching is “undone” +by calling ‘stop’. This can be fiddlier than you might think, because +if an exception is raised in the setUp then tearDown is not called. +*note unittest.TestCase.addCleanup(): 2a2. makes this easier: + + >>> class MyTest(unittest.TestCase): + ... def setUp(self): + ... patcher = patch('mymodule.foo') + ... self.addCleanup(patcher.stop) + ... self.mock_foo = patcher.start() + ... + ... def test_foo(self): + ... self.assertIs(mymodule.foo, self.mock_foo) + ... + >>> MyTest('test_foo').run() + + +File: python.info, Node: Mocking Unbound Methods, Next: Checking multiple calls with mock, Prev: Applying the same patch to every test method, Up: Further Examples + +5.26.6.19 Mocking Unbound Methods +................................. + +Whilst writing tests today I needed to patch an `unbound method' +(patching the method on the class rather than on the instance). I +needed self to be passed in as the first argument because I want to make +asserts about which objects were calling this particular method. The +issue is that you can’t patch with a mock for this, because if you +replace an unbound method with a mock it doesn’t become a bound method +when fetched from the instance, and so it doesn’t get self passed in. +The workaround is to patch the unbound method with a real function +instead. The *note patch(): 788. decorator makes it so simple to patch +out methods with a mock that having to create a real function becomes a +nuisance. + +If you pass ‘autospec=True’ to patch then it does the patching with a +`real' function object. This function object has the same signature as +the one it is replacing, but delegates to a mock under the hood. You +still get your mock auto-created in exactly the same way as before. +What it means though, is that if you use it to patch out an unbound +method on a class the mocked function will be turned into a bound method +if it is fetched from an instance. It will have ‘self’ passed in as the +first argument, which is exactly what I wanted: + + >>> class Foo: + ... def foo(self): + ... pass + ... + >>> with patch.object(Foo, 'foo', autospec=True) as mock_foo: + ... mock_foo.return_value = 'foo' + ... foo = Foo() + ... foo.foo() + ... + 'foo' + >>> mock_foo.assert_called_once_with(foo) + +If we don’t use ‘autospec=True’ then the unbound method is patched out +with a Mock instance instead, and isn’t called with ‘self’. + + +File: python.info, Node: Checking multiple calls with mock, Next: Coping with mutable arguments, Prev: Mocking Unbound Methods, Up: Further Examples + +5.26.6.20 Checking multiple calls with mock +........................................... + +mock has a nice API for making assertions about how your mock objects +are used. + + >>> mock = Mock() + >>> mock.foo_bar.return_value = None + >>> mock.foo_bar('baz', spam='eggs') + >>> mock.foo_bar.assert_called_with('baz', spam='eggs') + +If your mock is only being called once you can use the +‘assert_called_once_with()’ method that also asserts that the +‘call_count’ is one. + + >>> mock.foo_bar.assert_called_once_with('baz', spam='eggs') + >>> mock.foo_bar() + >>> mock.foo_bar.assert_called_once_with('baz', spam='eggs') + Traceback (most recent call last): + ... + AssertionError: Expected to be called once. Called 2 times. + +Both ‘assert_called_with’ and ‘assert_called_once_with’ make assertions +about the `most recent' call. If your mock is going to be called +several times, and you want to make assertions about `all' those calls +you can use *note call_args_list: 30b0.: + + >>> mock = Mock(return_value=None) + >>> mock(1, 2, 3) + >>> mock(4, 5, 6) + >>> mock() + >>> mock.call_args_list + [call(1, 2, 3), call(4, 5, 6), call()] + +The *note call: 30b1. helper makes it easy to make assertions about +these calls. You can build up a list of expected calls and compare it +to ‘call_args_list’. This looks remarkably similar to the repr of the +‘call_args_list’: + + >>> expected = [call(1, 2, 3), call(4, 5, 6), call()] + >>> mock.call_args_list == expected + True + + +File: python.info, Node: Coping with mutable arguments, Next: Nesting Patches, Prev: Checking multiple calls with mock, Up: Further Examples + +5.26.6.21 Coping with mutable arguments +....................................... + +Another situation is rare, but can bite you, is when your mock is called +with mutable arguments. ‘call_args’ and ‘call_args_list’ store +`references' to the arguments. If the arguments are mutated by the code +under test then you can no longer make assertions about what the values +were when the mock was called. + +Here’s some example code that shows the problem. Imagine the following +functions defined in ‘mymodule’: + + def frob(val): + pass + + def grob(val): + "First frob and then clear val" + frob(val) + val.clear() + +When we try to test that ‘grob’ calls ‘frob’ with the correct argument +look what happens: + + >>> with patch('mymodule.frob') as mock_frob: + ... val = {6} + ... mymodule.grob(val) + ... + >>> val + set() + >>> mock_frob.assert_called_with({6}) + Traceback (most recent call last): + ... + AssertionError: Expected: (({6},), {}) + Called with: ((set(),), {}) + +One possibility would be for mock to copy the arguments you pass in. +This could then cause problems if you do assertions that rely on object +identity for equality. + +Here’s one solution that uses the ‘side_effect’ functionality. If you +provide a ‘side_effect’ function for a mock then ‘side_effect’ will be +called with the same args as the mock. This gives us an opportunity to +copy the arguments and store them for later assertions. In this example +I’m using `another' mock to store the arguments so that I can use the +mock methods for doing the assertion. Again a helper function sets this +up for me. + + >>> from copy import deepcopy + >>> from unittest.mock import Mock, patch, DEFAULT + >>> def copy_call_args(mock): + ... new_mock = Mock() + ... def side_effect(*args, **kwargs): + ... args = deepcopy(args) + ... kwargs = deepcopy(kwargs) + ... new_mock(*args, **kwargs) + ... return DEFAULT + ... mock.side_effect = side_effect + ... return new_mock + ... + >>> with patch('mymodule.frob') as mock_frob: + ... new_mock = copy_call_args(mock_frob) + ... val = {6} + ... mymodule.grob(val) + ... + >>> new_mock.assert_called_with({6}) + >>> new_mock.call_args + call({6}) + +‘copy_call_args’ is called with the mock that will be called. It +returns a new mock that we do the assertion on. The ‘side_effect’ +function makes a copy of the args and calls our ‘new_mock’ with the +copy. + + Note: If your mock is only going to be used once there is an easier + way of checking arguments at the point they are called. You can + simply do the checking inside a ‘side_effect’ function. + + >>> def side_effect(arg): + ... assert arg == {6} + ... + >>> mock = Mock(side_effect=side_effect) + >>> mock({6}) + >>> mock(set()) + Traceback (most recent call last): + ... + AssertionError + +An alternative approach is to create a subclass of *note Mock: 249. or +*note MagicMock: 785. that copies (using *note copy.deepcopy(): 3cc.) +the arguments. Here’s an example implementation: + + >>> from copy import deepcopy + >>> class CopyingMock(MagicMock): + ... def __call__(self, /, *args, **kwargs): + ... args = deepcopy(args) + ... kwargs = deepcopy(kwargs) + ... return super(CopyingMock, self).__call__(*args, **kwargs) + ... + >>> c = CopyingMock(return_value=None) + >>> arg = set() + >>> c(arg) + >>> arg.add(1) + >>> c.assert_called_with(set()) + >>> c.assert_called_with(arg) + Traceback (most recent call last): + ... + AssertionError: Expected call: mock({1}) + Actual call: mock(set()) + >>> c.foo + <CopyingMock name='mock.foo' id='...'> + +When you subclass ‘Mock’ or ‘MagicMock’ all dynamically created +attributes, and the ‘return_value’ will use your subclass automatically. +That means all children of a ‘CopyingMock’ will also have the type +‘CopyingMock’. + + +File: python.info, Node: Nesting Patches, Next: Mocking a dictionary with MagicMock, Prev: Coping with mutable arguments, Up: Further Examples + +5.26.6.22 Nesting Patches +......................... + +Using patch as a context manager is nice, but if you do multiple patches +you can end up with nested with statements indenting further and further +to the right: + + >>> class MyTest(unittest.TestCase): + ... + ... def test_foo(self): + ... with patch('mymodule.Foo') as mock_foo: + ... with patch('mymodule.Bar') as mock_bar: + ... with patch('mymodule.Spam') as mock_spam: + ... assert mymodule.Foo is mock_foo + ... assert mymodule.Bar is mock_bar + ... assert mymodule.Spam is mock_spam + ... + >>> original = mymodule.Foo + >>> MyTest('test_foo').test_foo() + >>> assert mymodule.Foo is original + +With unittest ‘cleanup’ functions and the *note patch methods; start and +stop: 30cf. we can achieve the same effect without the nested +indentation. A simple helper method, ‘create_patch’, puts the patch in +place and returns the created mock for us: + + >>> class MyTest(unittest.TestCase): + ... + ... def create_patch(self, name): + ... patcher = patch(name) + ... thing = patcher.start() + ... self.addCleanup(patcher.stop) + ... return thing + ... + ... def test_foo(self): + ... mock_foo = self.create_patch('mymodule.Foo') + ... mock_bar = self.create_patch('mymodule.Bar') + ... mock_spam = self.create_patch('mymodule.Spam') + ... + ... assert mymodule.Foo is mock_foo + ... assert mymodule.Bar is mock_bar + ... assert mymodule.Spam is mock_spam + ... + >>> original = mymodule.Foo + >>> MyTest('test_foo').run() + >>> assert mymodule.Foo is original + + +File: python.info, Node: Mocking a dictionary with MagicMock, Next: Mock subclasses and their attributes, Prev: Nesting Patches, Up: Further Examples + +5.26.6.23 Mocking a dictionary with MagicMock +............................................. + +You may want to mock a dictionary, or other container object, recording +all access to it whilst having it still behave like a dictionary. + +We can do this with *note MagicMock: 785, which will behave like a +dictionary, and using *note side_effect: 309e. to delegate dictionary +access to a real underlying dictionary that is under our control. + +When the *note __getitem__(): 27c. and *note __setitem__(): c62. methods +of our ‘MagicMock’ are called (normal dictionary access) then +‘side_effect’ is called with the key (and in the case of ‘__setitem__’ +the value too). We can also control what is returned. + +After the ‘MagicMock’ has been used we can use attributes like *note +call_args_list: 30b0. to assert about how the dictionary was used: + + >>> my_dict = {'a': 1, 'b': 2, 'c': 3} + >>> def getitem(name): + ... return my_dict[name] + ... + >>> def setitem(name, val): + ... my_dict[name] = val + ... + >>> mock = MagicMock() + >>> mock.__getitem__.side_effect = getitem + >>> mock.__setitem__.side_effect = setitem + + Note: An alternative to using ‘MagicMock’ is to use ‘Mock’ and + `only' provide the magic methods you specifically want: + + >>> mock = Mock() + >>> mock.__getitem__ = Mock(side_effect=getitem) + >>> mock.__setitem__ = Mock(side_effect=setitem) + + A `third' option is to use ‘MagicMock’ but passing in ‘dict’ as the + `spec' (or `spec_set') argument so that the ‘MagicMock’ created + only has dictionary magic methods available: + + >>> mock = MagicMock(spec_set=dict) + >>> mock.__getitem__.side_effect = getitem + >>> mock.__setitem__.side_effect = setitem + +With these side effect functions in place, the ‘mock’ will behave like a +normal dictionary but recording the access. It even raises a *note +KeyError: 2c7. if you try to access a key that doesn’t exist. + + >>> mock['a'] + 1 + >>> mock['c'] + 3 + >>> mock['d'] + Traceback (most recent call last): + ... + KeyError: 'd' + >>> mock['b'] = 'fish' + >>> mock['d'] = 'eggs' + >>> mock['b'] + 'fish' + >>> mock['d'] + 'eggs' + +After it has been used you can make assertions about the access using +the normal mock methods and attributes: + + >>> mock.__getitem__.call_args_list + [call('a'), call('c'), call('d'), call('b'), call('d')] + >>> mock.__setitem__.call_args_list + [call('b', 'fish'), call('d', 'eggs')] + >>> my_dict + {'a': 1, 'b': 'fish', 'c': 3, 'd': 'eggs'} + + +File: python.info, Node: Mock subclasses and their attributes, Next: Mocking imports with patch dict, Prev: Mocking a dictionary with MagicMock, Up: Further Examples + +5.26.6.24 Mock subclasses and their attributes +.............................................. + +There are various reasons why you might want to subclass *note Mock: +249. One reason might be to add helper methods. Here’s a silly +example: + + >>> class MyMock(MagicMock): + ... def has_been_called(self): + ... return self.called + ... + >>> mymock = MyMock(return_value=None) + >>> mymock + <MyMock id='...'> + >>> mymock.has_been_called() + False + >>> mymock() + >>> mymock.has_been_called() + True + +The standard behaviour for ‘Mock’ instances is that attributes and the +return value mocks are of the same type as the mock they are accessed +on. This ensures that ‘Mock’ attributes are ‘Mocks’ and ‘MagicMock’ +attributes are ‘MagicMocks’ (1). So if you’re subclassing to add helper +methods then they’ll also be available on the attributes and return +value mock of instances of your subclass. + + >>> mymock.foo + <MyMock name='mock.foo' id='...'> + >>> mymock.foo.has_been_called() + False + >>> mymock.foo() + <MyMock name='mock.foo()' id='...'> + >>> mymock.foo.has_been_called() + True + +Sometimes this is inconvenient. For example, one user(2) is subclassing +mock to created a Twisted adaptor(3). Having this applied to attributes +too actually causes errors. + +‘Mock’ (in all its flavours) uses a method called ‘_get_child_mock’ to +create these “sub-mocks” for attributes and return values. You can +prevent your subclass being used for attributes by overriding this +method. The signature is that it takes arbitrary keyword arguments +(‘**kwargs’) which are then passed onto the mock constructor: + + >>> class Subclass(MagicMock): + ... def _get_child_mock(self, /, **kwargs): + ... return MagicMock(**kwargs) + ... + >>> mymock = Subclass() + >>> mymock.foo + <MagicMock name='mock.foo' id='...'> + >>> assert isinstance(mymock, Subclass) + >>> assert not isinstance(mymock.foo, Subclass) + >>> assert not isinstance(mymock(), Subclass) + + ---------- Footnotes ---------- + + (1) (2) An exception to this rule are the non-callable mocks. +Attributes use the callable variant because otherwise non-callable mocks +couldn’t have callable methods. + + (2) https://code.google.com/archive/p/mock/issues/105 + + (3) +https://twistedmatrix.com/documents/11.0.0/api/twisted.python.components.html + + +File: python.info, Node: Mocking imports with patch dict, Next: Tracking order of calls and less verbose call assertions, Prev: Mock subclasses and their attributes, Up: Further Examples + +5.26.6.25 Mocking imports with patch.dict +......................................... + +One situation where mocking can be hard is where you have a local import +inside a function. These are harder to mock because they aren’t using +an object from the module namespace that we can patch out. + +Generally local imports are to be avoided. They are sometimes done to +prevent circular dependencies, for which there is `usually' a much +better way to solve the problem (refactor the code) or to prevent “up +front costs” by delaying the import. This can also be solved in better +ways than an unconditional local import (store the module as a class or +module attribute and only do the import on first use). + +That aside there is a way to use ‘mock’ to affect the results of an +import. Importing fetches an `object' from the *note sys.modules: 1152. +dictionary. Note that it fetches an `object', which need not be a +module. Importing a module for the first time results in a module +object being put in ‘sys.modules’, so usually when you import something +you get a module back. This need not be the case however. + +This means you can use *note patch.dict(): 3097. to `temporarily' put a +mock in place in *note sys.modules: 1152. Any imports whilst this patch +is active will fetch the mock. When the patch is complete (the +decorated function exits, the with statement body is complete or +‘patcher.stop()’ is called) then whatever was there previously will be +restored safely. + +Here’s an example that mocks out the ‘fooble’ module. + + >>> import sys + >>> mock = Mock() + >>> with patch.dict('sys.modules', {'fooble': mock}): + ... import fooble + ... fooble.blob() + ... + <Mock name='mock.blob()' id='...'> + >>> assert 'fooble' not in sys.modules + >>> mock.blob.assert_called_once_with() + +As you can see the ‘import fooble’ succeeds, but on exit there is no +‘fooble’ left in *note sys.modules: 1152. + +This also works for the ‘from module import name’ form: + + >>> mock = Mock() + >>> with patch.dict('sys.modules', {'fooble': mock}): + ... from fooble import blob + ... blob.blip() + ... + <Mock name='mock.blob.blip()' id='...'> + >>> mock.blob.blip.assert_called_once_with() + +With slightly more work you can also mock package imports: + + >>> mock = Mock() + >>> modules = {'package': mock, 'package.module': mock.module} + >>> with patch.dict('sys.modules', modules): + ... from package.module import fooble + ... fooble() + ... + <Mock name='mock.module.fooble()' id='...'> + >>> mock.module.fooble.assert_called_once_with() + + +File: python.info, Node: Tracking order of calls and less verbose call assertions, Next: More complex argument matching, Prev: Mocking imports with patch dict, Up: Further Examples + +5.26.6.26 Tracking order of calls and less verbose call assertions +.................................................................. + +The *note Mock: 249. class allows you to track the `order' of method +calls on your mock objects through the *note method_calls: 30a9. +attribute. This doesn’t allow you to track the order of calls between +separate mock objects, however we can use *note mock_calls: 30a6. to +achieve the same effect. + +Because mocks track calls to child mocks in ‘mock_calls’, and accessing +an arbitrary attribute of a mock creates a child mock, we can create our +separate mocks from a parent one. Calls to those child mock will then +all be recorded, in order, in the ‘mock_calls’ of the parent: + + >>> manager = Mock() + >>> mock_foo = manager.foo + >>> mock_bar = manager.bar + + >>> mock_foo.something() + <Mock name='mock.foo.something()' id='...'> + >>> mock_bar.other.thing() + <Mock name='mock.bar.other.thing()' id='...'> + + >>> manager.mock_calls + [call.foo.something(), call.bar.other.thing()] + +We can then assert about the calls, including the order, by comparing +with the ‘mock_calls’ attribute on the manager mock: + + >>> expected_calls = [call.foo.something(), call.bar.other.thing()] + >>> manager.mock_calls == expected_calls + True + +If ‘patch’ is creating, and putting in place, your mocks then you can +attach them to a manager mock using the *note attach_mock(): 30a8. +method. After attaching calls will be recorded in ‘mock_calls’ of the +manager. + + >>> manager = MagicMock() + >>> with patch('mymodule.Class1') as MockClass1: + ... with patch('mymodule.Class2') as MockClass2: + ... manager.attach_mock(MockClass1, 'MockClass1') + ... manager.attach_mock(MockClass2, 'MockClass2') + ... MockClass1().foo() + ... MockClass2().bar() + <MagicMock name='mock.MockClass1().foo()' id='...'> + <MagicMock name='mock.MockClass2().bar()' id='...'> + >>> manager.mock_calls + [call.MockClass1(), + call.MockClass1().foo(), + call.MockClass2(), + call.MockClass2().bar()] + +If many calls have been made, but you’re only interested in a particular +sequence of them then an alternative is to use the *note +assert_has_calls(): 30a5. method. This takes a list of calls +(constructed with the *note call: 30b1. object). If that sequence of +calls are in *note mock_calls: 30a6. then the assert succeeds. + + >>> m = MagicMock() + >>> m().foo().bar().baz() + <MagicMock name='mock().foo().bar().baz()' id='...'> + >>> m.one().two().three() + <MagicMock name='mock.one().two().three()' id='...'> + >>> calls = call.one().two().three().call_list() + >>> m.assert_has_calls(calls) + +Even though the chained call ‘m.one().two().three()’ aren’t the only +calls that have been made to the mock, the assert still succeeds. + +Sometimes a mock may have several calls made to it, and you are only +interested in asserting about `some' of those calls. You may not even +care about the order. In this case you can pass ‘any_order=True’ to +‘assert_has_calls’: + + >>> m = MagicMock() + >>> m(1), m.two(2, 3), m.seven(7), m.fifty('50') + (...) + >>> calls = [call.fifty('50'), call(1), call.seven(7)] + >>> m.assert_has_calls(calls, any_order=True) + + +File: python.info, Node: More complex argument matching, Prev: Tracking order of calls and less verbose call assertions, Up: Further Examples + +5.26.6.27 More complex argument matching +........................................ + +Using the same basic concept as *note ANY: 30e0. we can implement +matchers to do more complex assertions on objects used as arguments to +mocks. + +Suppose we expect some object to be passed to a mock that by default +compares equal based on object identity (which is the Python default for +user defined classes). To use *note assert_called_with(): 30a2. we +would need to pass in the exact same object. If we are only interested +in some of the attributes of this object then we can create a matcher +that will check these attributes for us. + +You can see in this example how a ‘standard’ call to +‘assert_called_with’ isn’t sufficient: + + >>> class Foo: + ... def __init__(self, a, b): + ... self.a, self.b = a, b + ... + >>> mock = Mock(return_value=None) + >>> mock(Foo(1, 2)) + >>> mock.assert_called_with(Foo(1, 2)) + Traceback (most recent call last): + ... + AssertionError: Expected: call(<__main__.Foo object at 0x...>) + Actual call: call(<__main__.Foo object at 0x...>) + +A comparison function for our ‘Foo’ class might look something like +this: + + >>> def compare(self, other): + ... if not type(self) == type(other): + ... return False + ... if self.a != other.a: + ... return False + ... if self.b != other.b: + ... return False + ... return True + ... + +And a matcher object that can use comparison functions like this for its +equality operation would look something like this: + + >>> class Matcher: + ... def __init__(self, compare, some_obj): + ... self.compare = compare + ... self.some_obj = some_obj + ... def __eq__(self, other): + ... return self.compare(self.some_obj, other) + ... + +Putting all this together: + + >>> match_foo = Matcher(compare, Foo(1, 2)) + >>> mock.assert_called_with(match_foo) + +The ‘Matcher’ is instantiated with our compare function and the ‘Foo’ +object we want to compare against. In ‘assert_called_with’ the +‘Matcher’ equality method will be called, which compares the object the +mock was called with against the one we created our matcher with. If +they match then ‘assert_called_with’ passes, and if they don’t an *note +AssertionError: 1223. is raised: + + >>> match_wrong = Matcher(compare, Foo(3, 4)) + >>> mock.assert_called_with(match_wrong) + Traceback (most recent call last): + ... + AssertionError: Expected: ((<Matcher object at 0x...>,), {}) + Called with: ((<Foo object at 0x...>,), {}) + +With a bit of tweaking you could have the comparison function raise the +*note AssertionError: 1223. directly and provide a more useful failure +message. + +As of version 1.5, the Python testing library PyHamcrest(1) provides +similar functionality, that may be useful here, in the form of its +equality matcher (hamcrest.library.integration.match_equality(2)). + + ---------- Footnotes ---------- + + (1) https://pyhamcrest.readthedocs.io/ + + (2) +https://pyhamcrest.readthedocs.io/en/release-1.8/integration/#module-hamcrest.library.integration.match_equality + + +File: python.info, Node: 2to3 - Automated Python 2 to 3 code translation, Next: test — Regression tests package for Python, Prev: unittest mock — getting started, Up: Development Tools + +5.26.7 2to3 - Automated Python 2 to 3 code translation +------------------------------------------------------ + +2to3 is a Python program that reads Python 2.x source code and applies a +series of `fixers' to transform it into valid Python 3.x code. The +standard library contains a rich set of fixers that will handle almost +all code. 2to3 supporting library *note lib2to3: a7. is, however, a +flexible and generic library, so it is possible to write your own fixers +for 2to3. *note lib2to3: a7. could also be adapted to custom +applications in which Python code needs to be edited automatically. + +* Menu: + +* Using 2to3:: +* Fixers:: +* lib2to3 - 2to3’s library:: + + +File: python.info, Node: Using 2to3, Next: Fixers, Up: 2to3 - Automated Python 2 to 3 code translation + +5.26.7.1 Using 2to3 +................... + +2to3 will usually be installed with the Python interpreter as a script. +It is also located in the ‘Tools/scripts’ directory of the Python root. + +2to3’s basic arguments are a list of files or directories to transform. +The directories are recursively traversed for Python sources. + +Here is a sample Python 2.x source file, ‘example.py’: + + def greet(name): + print "Hello, {0}!".format(name) + print "What's your name?" + name = raw_input() + greet(name) + +It can be converted to Python 3.x code via 2to3 on the command line: + + $ 2to3 example.py + +A diff against the original source file is printed. 2to3 can also write +the needed modifications right back to the source file. (A backup of +the original file is made unless ‘-n’ is also given.) Writing the +changes back is enabled with the ‘-w’ flag: + + $ 2to3 -w example.py + +After transformation, ‘example.py’ looks like this: + + def greet(name): + print("Hello, {0}!".format(name)) + print("What's your name?") + name = input() + greet(name) + +Comments and exact indentation are preserved throughout the translation +process. + +By default, 2to3 runs a set of *note predefined fixers: 3109. The ‘-l’ +flag lists all available fixers. An explicit set of fixers to run can +be given with ‘-f’. Likewise the ‘-x’ explicitly disables a fixer. The +following example runs only the ‘imports’ and ‘has_key’ fixers: + + $ 2to3 -f imports -f has_key example.py + +This command runs every fixer except the ‘apply’ fixer: + + $ 2to3 -x apply example.py + +Some fixers are `explicit', meaning they aren’t run by default and must +be listed on the command line to be run. Here, in addition to the +default fixers, the ‘idioms’ fixer is run: + + $ 2to3 -f all -f idioms example.py + +Notice how passing ‘all’ enables all default fixers. + +Sometimes 2to3 will find a place in your source code that needs to be +changed, but 2to3 cannot fix automatically. In this case, 2to3 will +print a warning beneath the diff for a file. You should address the +warning in order to have compliant 3.x code. + +2to3 can also refactor doctests. To enable this mode, use the ‘-d’ +flag. Note that `only' doctests will be refactored. This also doesn’t +require the module to be valid Python. For example, doctest like +examples in a reST document could also be refactored with this option. + +The ‘-v’ option enables output of more information on the translation +process. + +Since some print statements can be parsed as function calls or +statements, 2to3 cannot always read files containing the print function. +When 2to3 detects the presence of the ‘from __future__ import +print_function’ compiler directive, it modifies its internal grammar to +interpret *note print(): 886. as a function. This change can also be +enabled manually with the ‘-p’ flag. Use ‘-p’ to run fixers on code +that already has had its print statements converted. + +The ‘-o’ or ‘--output-dir’ option allows specification of an alternate +directory for processed output files to be written to. The ‘-n’ flag is +required when using this as backup files do not make sense when not +overwriting the input files. + +New in version 3.2.3: The ‘-o’ option was added. + +The ‘-W’ or ‘--write-unchanged-files’ flag tells 2to3 to always write +output files even if no changes were required to the file. This is most +useful with ‘-o’ so that an entire Python source tree is copied with +translation from one directory to another. This option implies the ‘-w’ +flag as it would not make sense otherwise. + +New in version 3.2.3: The ‘-W’ flag was added. + +The ‘--add-suffix’ option specifies a string to append to all output +filenames. The ‘-n’ flag is required when specifying this as backups +are not necessary when writing to different filenames. Example: + + $ 2to3 -n -W --add-suffix=3 example.py + +Will cause a converted file named ‘example.py3’ to be written. + +New in version 3.2.3: The ‘--add-suffix’ option was added. + +To translate an entire project from one directory tree to another use: + + $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode + + +File: python.info, Node: Fixers, Next: lib2to3 - 2to3’s library, Prev: Using 2to3, Up: 2to3 - Automated Python 2 to 3 code translation + +5.26.7.2 Fixers +............... + +Each step of transforming code is encapsulated in a fixer. The command +‘2to3 -l’ lists them. As *note documented above: 3107, each can be +turned on and off individually. They are described here in more detail. + + -- 2to3fixer: apply + + Removes usage of ‘apply()’. For example ‘apply(function, *args, + **kwargs)’ is converted to ‘function(*args, **kwargs)’. + + -- 2to3fixer: asserts + + Replaces deprecated *note unittest: 11b. method names with the + correct ones. + + From To + + ------------------------------------------------------------------------------------ + + ‘failUnlessEqual(a, b)’ *note assertEqual(a, b): bb5. + + + ‘assertEquals(a, b)’ *note assertEqual(a, b): bb5. + + + ‘failIfEqual(a, b)’ *note assertNotEqual(a, b): bb6. + + + ‘assertNotEquals(a, b)’ *note assertNotEqual(a, b): bb6. + + + ‘failUnless(a)’ *note assertTrue(a): bb4. + + + ‘assert_(a)’ *note assertTrue(a): bb4. + + + ‘failIf(a)’ *note assertFalse(a): cc7. + + + ‘failUnlessRaises(exc, cal)’ *note assertRaises(exc, cal): aba. + + + ‘failUnlessAlmostEqual(a, b)’ *note assertAlmostEqual(a, b): bb7. + + + ‘assertAlmostEquals(a, b)’ *note assertAlmostEqual(a, b): bb7. + + + ‘failIfAlmostEqual(a, b)’ *note assertNotAlmostEqual(a, b): bb8. + + + ‘assertNotAlmostEquals(a, b)’ *note assertNotAlmostEqual(a, b): bb8. + + + -- 2to3fixer: basestring + + Converts ‘basestring’ to *note str: 330. + + -- 2to3fixer: buffer + + Converts ‘buffer’ to *note memoryview: 25c. This fixer is optional + because the *note memoryview: 25c. API is similar but not exactly + the same as that of ‘buffer’. + + -- 2to3fixer: dict + + Fixes dictionary iteration methods. ‘dict.iteritems()’ is + converted to *note dict.items(): c2b, ‘dict.iterkeys()’ to *note + dict.keys(): c2a, and ‘dict.itervalues()’ to *note dict.values(): + c2c. Similarly, ‘dict.viewitems()’, ‘dict.viewkeys()’ and + ‘dict.viewvalues()’ are converted respectively to *note + dict.items(): c2b, *note dict.keys(): c2a. and *note dict.values(): + c2c. It also wraps existing usages of *note dict.items(): c2b, + *note dict.keys(): c2a, and *note dict.values(): c2c. in a call to + *note list: 262. + + -- 2to3fixer: except + + Converts ‘except X, T’ to ‘except X as T’. + + -- 2to3fixer: exec + + Converts the ‘exec’ statement to the *note exec(): c44. function. + + -- 2to3fixer: execfile + + Removes usage of ‘execfile()’. The argument to ‘execfile()’ is + wrapped in calls to *note open(): 4f0, *note compile(): 1b4, and + *note exec(): c44. + + -- 2to3fixer: exitfunc + + Changes assignment of ‘sys.exitfunc’ to use of the *note atexit: c. + module. + + -- 2to3fixer: filter + + Wraps *note filter(): c2e. usage in a *note list: 262. call. + + -- 2to3fixer: funcattrs + + Fixes function attributes that have been renamed. For example, + ‘my_function.func_closure’ is converted to + ‘my_function.__closure__’. + + -- 2to3fixer: future + + Removes ‘from __future__ import new_feature’ statements. + + -- 2to3fixer: getcwdu + + Renames ‘os.getcwdu()’ to *note os.getcwd(): 188d. + + -- 2to3fixer: has_key + + Changes ‘dict.has_key(key)’ to ‘key in dict’. + + -- 2to3fixer: idioms + + This optional fixer performs several transformations that make + Python code more idiomatic. Type comparisons like ‘type(x) is + SomeClass’ and ‘type(x) == SomeClass’ are converted to + ‘isinstance(x, SomeClass)’. ‘while 1’ becomes ‘while True’. This + fixer also tries to make use of *note sorted(): 444. in appropriate + places. For example, this block + + L = list(some_iterable) + L.sort() + + is changed to + + L = sorted(some_iterable) + + -- 2to3fixer: import + + Detects sibling imports and converts them to relative imports. + + -- 2to3fixer: imports + + Handles module renames in the standard library. + + -- 2to3fixer: imports2 + + Handles other modules renames in the standard library. It is + separate from the *note imports: 311b. fixer only because of + technical limitations. + + -- 2to3fixer: input + + Converts ‘input(prompt)’ to ‘eval(input(prompt))’. + + -- 2to3fixer: intern + + Converts ‘intern()’ to *note sys.intern(): c6e. + + -- 2to3fixer: isinstance + + Fixes duplicate types in the second argument of *note isinstance(): + 44f. For example, ‘isinstance(x, (int, int))’ is converted to + ‘isinstance(x, int)’ and ‘isinstance(x, (int, float, int))’ is + converted to ‘isinstance(x, (int, float))’. + + -- 2to3fixer: itertools_imports + + Removes imports of ‘itertools.ifilter()’, ‘itertools.izip()’, and + ‘itertools.imap()’. Imports of ‘itertools.ifilterfalse()’ are also + changed to *note itertools.filterfalse(): 1296. + + -- 2to3fixer: itertools + + Changes usage of ‘itertools.ifilter()’, ‘itertools.izip()’, and + ‘itertools.imap()’ to their built-in equivalents. + ‘itertools.ifilterfalse()’ is changed to *note + itertools.filterfalse(): 1296. + + -- 2to3fixer: long + + Renames ‘long’ to *note int: 184. + + -- 2to3fixer: map + + Wraps *note map(): c2d. in a *note list: 262. call. It also + changes ‘map(None, x)’ to ‘list(x)’. Using ‘from future_builtins + import map’ disables this fixer. + + -- 2to3fixer: metaclass + + Converts the old metaclass syntax (‘__metaclass__ = Meta’ in the + class body) to the new (‘class X(metaclass=Meta)’). + + -- 2to3fixer: methodattrs + + Fixes old method attribute names. For example, ‘meth.im_func’ is + converted to ‘meth.__func__’. + + -- 2to3fixer: ne + + Converts the old not-equal syntax, ‘<>’, to ‘!=’. + + -- 2to3fixer: next + + Converts the use of iterator’s ‘next()’ methods to the *note + next(): 682. function. It also renames *note next(): 682. methods + to *note __next__(): c64. + + -- 2to3fixer: nonzero + + Renames ‘__nonzero__()’ to *note __bool__(): c68. + + -- 2to3fixer: numliterals + + Converts octal literals into the new syntax. + + -- 2to3fixer: operator + + Converts calls to various functions in the *note operator: c2. + module to other, but equivalent, function calls. When needed, the + appropriate ‘import’ statements are added, e.g. ‘import + collections.abc’. The following mapping are made: + + From To + + ----------------------------------------------------------------------------------------- + + ‘operator.isCallable(obj)’ ‘callable(obj)’ + + + ‘operator.sequenceIncludes(obj)’ ‘operator.contains(obj)’ + + + ‘operator.isSequenceType(obj)’ ‘isinstance(obj, collections.abc.Sequence)’ + + + ‘operator.isMappingType(obj)’ ‘isinstance(obj, collections.abc.Mapping)’ + + + ‘operator.isNumberType(obj)’ ‘isinstance(obj, numbers.Number)’ + + + ‘operator.repeat(obj, n)’ ‘operator.mul(obj, n)’ + + + ‘operator.irepeat(obj, n)’ ‘operator.imul(obj, n)’ + + + -- 2to3fixer: paren + + Add extra parenthesis where they are required in list + comprehensions. For example, ‘[x for x in 1, 2]’ becomes ‘[x for x + in (1, 2)]’. + + -- 2to3fixer: print + + Converts the ‘print’ statement to the *note print(): 886. function. + + -- 2to3fixer: raise + + Converts ‘raise E, V’ to ‘raise E(V)’, and ‘raise E, V, T’ to + ‘raise E(V).with_traceback(T)’. If ‘E’ is a tuple, the translation + will be incorrect because substituting tuples for exceptions has + been removed in 3.0. + + -- 2to3fixer: raw_input + + Converts ‘raw_input()’ to *note input(): c6b. + + -- 2to3fixer: reduce + + Handles the move of ‘reduce()’ to *note functools.reduce(): c6f. + + -- 2to3fixer: reload + + Converts ‘reload()’ to *note importlib.reload(): 399. + + -- 2to3fixer: renames + + Changes ‘sys.maxint’ to *note sys.maxsize: b43. + + -- 2to3fixer: repr + + Replaces backtick repr with the *note repr(): 7cc. function. + + -- 2to3fixer: set_literal + + Replaces use of the *note set: b6f. constructor with set literals. + This fixer is optional. + + -- 2to3fixer: standarderror + + Renames ‘StandardError’ to *note Exception: 1a9. + + -- 2to3fixer: sys_exc + + Changes the deprecated ‘sys.exc_value’, ‘sys.exc_type’, + ‘sys.exc_traceback’ to use *note sys.exc_info(): c5f. + + -- 2to3fixer: throw + + Fixes the API change in generator’s ‘throw()’ method. + + -- 2to3fixer: tuple_params + + Removes implicit tuple parameter unpacking. This fixer inserts + temporary variables. + + -- 2to3fixer: types + + Fixes code broken from the removal of some members in the *note + types: 118. module. + + -- 2to3fixer: unicode + + Renames ‘unicode’ to *note str: 330. + + -- 2to3fixer: urllib + + Handles the rename of *note urllib: 11d. and ‘urllib2’ to the *note + urllib: 11d. package. + + -- 2to3fixer: ws_comma + + Removes excess whitespace from comma separated items. This fixer + is optional. + + -- 2to3fixer: xrange + + Renames ‘xrange()’ to *note range(): 9be. and wraps existing *note + range(): 9be. calls with *note list: 262. + + -- 2to3fixer: xreadlines + + Changes ‘for x in file.xreadlines()’ to ‘for x in file’. + + -- 2to3fixer: zip + + Wraps *note zip(): c32. usage in a *note list: 262. call. This is + disabled when ‘from future_builtins import zip’ appears. + + +File: python.info, Node: lib2to3 - 2to3’s library, Prev: Fixers, Up: 2to3 - Automated Python 2 to 3 code translation + +5.26.7.3 ‘lib2to3’ - 2to3’s library +................................... + +`Source code:' Lib/lib2to3/(1) + +__________________________________________________________________ + + Note: The *note lib2to3: a7. API should be considered unstable and + may change drastically in the future. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/lib2to3/ + + +File: python.info, Node: test — Regression tests package for Python, Next: test support — Utilities for the Python test suite, Prev: 2to3 - Automated Python 2 to 3 code translation, Up: Development Tools + +5.26.8 ‘test’ — Regression tests package for Python +--------------------------------------------------- + + Note: The *note test: 105. package is meant for internal use by + Python only. It is documented for the benefit of the core + developers of Python. Any use of this package outside of Python’s + standard library is discouraged as code mentioned here can change + or be removed without notice between releases of Python. + +__________________________________________________________________ + +The *note test: 105. package contains all regression tests for Python as +well as the modules *note test.support: 106. and ‘test.regrtest’. *note +test.support: 106. is used to enhance your tests while ‘test.regrtest’ +drives the testing suite. + +Each module in the *note test: 105. package whose name starts with +‘test_’ is a testing suite for a specific module or feature. All new +tests should be written using the *note unittest: 11b. or *note doctest: +67. module. Some older tests are written using a “traditional” testing +style that compares output printed to ‘sys.stdout’; this style of test +is considered deprecated. + +See also +........ + +Module *note unittest: 11b. + + Writing PyUnit regression tests. + +Module *note doctest: 67. + + Tests embedded in documentation strings. + +* Menu: + +* Writing Unit Tests for the test package:: +* Running tests using the command-line interface:: + + +File: python.info, Node: Writing Unit Tests for the test package, Next: Running tests using the command-line interface, Up: test — Regression tests package for Python + +5.26.8.1 Writing Unit Tests for the ‘test’ package +.................................................. + +It is preferred that tests that use the *note unittest: 11b. module +follow a few guidelines. One is to name the test module by starting it +with ‘test_’ and end it with the name of the module being tested. The +test methods in the test module should start with ‘test_’ and end with a +description of what the method is testing. This is needed so that the +methods are recognized by the test driver as test methods. Also, no +documentation string for the method should be included. A comment (such +as ‘# Tests function returns only True or False’) should be used to +provide documentation for test methods. This is done because +documentation strings get printed out if they exist and thus what test +is being run is not stated. + +A basic boilerplate is often used: + + import unittest + from test import support + + class MyTestCase1(unittest.TestCase): + + # Only use setUp() and tearDown() if necessary + + def setUp(self): + ... code to execute in preparation for tests ... + + def tearDown(self): + ... code to execute to clean up after tests ... + + def test_feature_one(self): + # Test feature one. + ... testing code ... + + def test_feature_two(self): + # Test feature two. + ... testing code ... + + ... more test methods ... + + class MyTestCase2(unittest.TestCase): + ... same structure as MyTestCase1 ... + + ... more test classes ... + + if __name__ == '__main__': + unittest.main() + +This code pattern allows the testing suite to be run by ‘test.regrtest’, +on its own as a script that supports the *note unittest: 11b. CLI, or +via the ‘python -m unittest’ CLI. + +The goal for regression testing is to try to break code. This leads to +a few guidelines to be followed: + + * The testing suite should exercise all classes, functions, and + constants. This includes not just the external API that is to be + presented to the outside world but also “private” code. + + * Whitebox testing (examining the code being tested when the tests + are being written) is preferred. Blackbox testing (testing only + the published user interface) is not complete enough to make sure + all boundary and edge cases are tested. + + * Make sure all possible values are tested including invalid ones. + This makes sure that not only all valid values are acceptable but + also that improper values are handled correctly. + + * Exhaust as many code paths as possible. Test where branching + occurs and thus tailor input to make sure as many different paths + through the code are taken. + + * Add an explicit test for any bugs discovered for the tested code. + This will make sure that the error does not crop up again if the + code is changed in the future. + + * Make sure to clean up after your tests (such as close and remove + all temporary files). + + * If a test is dependent on a specific condition of the operating + system then verify the condition already exists before attempting + the test. + + * Import as few modules as possible and do it as soon as possible. + This minimizes external dependencies of tests and also minimizes + possible anomalous behavior from side-effects of importing a + module. + + * Try to maximize code reuse. On occasion, tests will vary by + something as small as what type of input is used. Minimize code + duplication by subclassing a basic test class with a class that + specifies the input: + + class TestFuncAcceptsSequencesMixin: + + func = mySuperWhammyFunction + + def test_func(self): + self.func(self.arg) + + class AcceptLists(TestFuncAcceptsSequencesMixin, unittest.TestCase): + arg = [1, 2, 3] + + class AcceptStrings(TestFuncAcceptsSequencesMixin, unittest.TestCase): + arg = 'abc' + + class AcceptTuples(TestFuncAcceptsSequencesMixin, unittest.TestCase): + arg = (1, 2, 3) + + When using this pattern, remember that all classes that inherit + from *note unittest.TestCase: 8ff. are run as tests. The ‘Mixin’ + class in the example above does not have any data and so can’t be + run by itself, thus it does not inherit from *note + unittest.TestCase: 8ff. + +See also +........ + +Test Driven Development + + A book by Kent Beck on writing tests before code. + + +File: python.info, Node: Running tests using the command-line interface, Prev: Writing Unit Tests for the test package, Up: test — Regression tests package for Python + +5.26.8.2 Running tests using the command-line interface +....................................................... + +The *note test: 105. package can be run as a script to drive Python’s +regression test suite, thanks to the *note -m: 337. option: ‘python -m +test’. Under the hood, it uses ‘test.regrtest’; the call ‘python -m +test.regrtest’ used in previous Python versions still works. Running +the script by itself automatically starts running all regression tests +in the *note test: 105. package. It does this by finding all modules in +the package whose name starts with ‘test_’, importing them, and +executing the function ‘test_main()’ if present or loading the tests via +unittest.TestLoader.loadTestsFromModule if ‘test_main’ does not exist. +The names of tests to execute may also be passed to the script. +Specifying a single regression test (‘python -m test test_spam’) will +minimize output and only print whether the test passed or failed. + +Running *note test: 105. directly allows what resources are available +for tests to use to be set. You do this by using the ‘-u’ command-line +option. Specifying ‘all’ as the value for the ‘-u’ option enables all +possible resources: ‘python -m test -uall’. If all but one resource is +desired (a more common case), a comma-separated list of resources that +are not desired may be listed after ‘all’. The command ‘python -m test +-uall,-audio,-largefile’ will run *note test: 105. with all resources +except the ‘audio’ and ‘largefile’ resources. For a list of all +resources and more command-line options, run ‘python -m test -h’. + +Some other ways to execute the regression tests depend on what platform +the tests are being executed on. On Unix, you can run ‘make test’ at +the top-level directory where Python was built. On Windows, executing +‘rt.bat’ from your ‘PCbuild’ directory will run all regression tests. + + +File: python.info, Node: test support — Utilities for the Python test suite, Next: test support script_helper — Utilities for the Python execution tests, Prev: test — Regression tests package for Python, Up: Development Tools + +5.26.9 ‘test.support’ — Utilities for the Python test suite +----------------------------------------------------------- + +The *note test.support: 106. module provides support for Python’s +regression test suite. + + Note: *note test.support: 106. is not a public module. It is + documented here to help Python developers write tests. The API of + this module is subject to change without backwards compatibility + concerns between releases. + +This module defines the following exceptions: + + -- Exception: test.support.TestFailed + + Exception to be raised when a test fails. This is deprecated in + favor of *note unittest: 11b.-based tests and *note + unittest.TestCase: 8ff.’s assertion methods. + + -- Exception: test.support.ResourceDenied + + Subclass of *note unittest.SkipTest: 903. Raised when a resource + (such as a network connection) is not available. Raised by the + *note requires(): 3148. function. + +The *note test.support: 106. module defines the following constants: + + -- Data: test.support.verbose + + ‘True’ when verbose output is enabled. Should be checked when more + detailed information is desired about a running test. `verbose' is + set by ‘test.regrtest’. + + -- Data: test.support.is_jython + + ‘True’ if the running interpreter is Jython. + + -- Data: test.support.is_android + + ‘True’ if the system is Android. + + -- Data: test.support.unix_shell + + Path for shell if not on Windows; otherwise ‘None’. + + -- Data: test.support.FS_NONASCII + + A non-ASCII character encodable by *note os.fsencode(): 4ef. + + -- Data: test.support.TESTFN + + Set to a name that is safe to use as the name of a temporary file. + Any temporary file that is created should be closed and unlinked + (removed). + + -- Data: test.support.TESTFN_UNICODE + + Set to a non-ASCII name for a temporary file. + + -- Data: test.support.TESTFN_ENCODING + + Set to *note sys.getfilesystemencoding(): 4f6. + + -- Data: test.support.TESTFN_UNENCODABLE + + Set to a filename (str type) that should not be able to be encoded + by file system encoding in strict mode. It may be ‘None’ if it’s + not possible to generate such a filename. + + -- Data: test.support.TESTFN_UNDECODABLE + + Set to a filename (bytes type) that should not be able to be + decoded by file system encoding in strict mode. It may be ‘None’ + if it’s not possible to generate such a filename. + + -- Data: test.support.TESTFN_NONASCII + + Set to a filename containing the *note FS_NONASCII: 314d. + character. + + -- Data: test.support.IPV6_ENABLED + + Set to ‘True’ if IPV6 is enabled on this host, ‘False’ otherwise. + + -- Data: test.support.SAVEDCWD + + Set to *note os.getcwd(): 188d. + + -- Data: test.support.PGO + + Set when tests can be skipped when they are not useful for PGO. + + -- Data: test.support.PIPE_MAX_SIZE + + A constant that is likely larger than the underlying OS pipe buffer + size, to make writes blocking. + + -- Data: test.support.SOCK_MAX_SIZE + + A constant that is likely larger than the underlying OS socket + buffer size, to make writes blocking. + + -- Data: test.support.TEST_SUPPORT_DIR + + Set to the top level directory that contains *note test.support: + 106. + + -- Data: test.support.TEST_HOME_DIR + + Set to the top level directory for the test package. + + -- Data: test.support.TEST_DATA_DIR + + Set to the ‘data’ directory within the test package. + + -- Data: test.support.MAX_Py_ssize_t + + Set to *note sys.maxsize: b43. for big memory tests. + + -- Data: test.support.max_memuse + + Set by *note set_memlimit(): 315e. as the memory limit for big + memory tests. Limited by *note MAX_Py_ssize_t: 315c. + + -- Data: test.support.real_max_memuse + + Set by *note set_memlimit(): 315e. as the memory limit for big + memory tests. Not limited by *note MAX_Py_ssize_t: 315c. + + -- Data: test.support.MISSING_C_DOCSTRINGS + + Return ‘True’ if running on CPython, not on Windows, and + configuration not set with ‘WITH_DOC_STRINGS’. + + -- Data: test.support.HAVE_DOCSTRINGS + + Check for presence of docstrings. + + -- Data: test.support.TEST_HTTP_URL + + Define the URL of a dedicated HTTP server for the network tests. + + -- Data: test.support.ALWAYS_EQ + + Object that is equal to anything. Used to test mixed type + comparison. + + -- Data: test.support.LARGEST + + Object that is greater than anything (except itself). Used to test + mixed type comparison. + + -- Data: test.support.SMALLEST + + Object that is less than anything (except itself). Used to test + mixed type comparison. + +The *note test.support: 106. module defines the following functions: + + -- Function: test.support.forget (module_name) + + Remove the module named `module_name' from ‘sys.modules’ and delete + any byte-compiled files of the module. + + -- Function: test.support.unload (name) + + Delete `name' from ‘sys.modules’. + + -- Function: test.support.unlink (filename) + + Call *note os.unlink(): a3c. on `filename'. On Windows platforms, + this is wrapped with a wait loop that checks for the existence fo + the file. + + -- Function: test.support.rmdir (filename) + + Call *note os.rmdir(): a3a. on `filename'. On Windows platforms, + this is wrapped with a wait loop that checks for the existence of + the file. + + -- Function: test.support.rmtree (path) + + Call *note shutil.rmtree(): 21c. on `path' or call *note + os.lstat(): 1f2. and *note os.rmdir(): a3a. to remove a path and + its contents. On Windows platforms, this is wrapped with a wait + loop that checks for the existence of the files. + + -- Function: test.support.make_legacy_pyc (source) + + Move a PEP 3147(1)/ PEP 488(2) pyc file to its legacy pyc location + and return the file system path to the legacy pyc file. The + `source' value is the file system path to the source file. It does + not need to exist, however the PEP 3147/488 pyc file must exist. + + -- Function: test.support.is_resource_enabled (resource) + + Return ‘True’ if `resource' is enabled and available. The list of + available resources is only set when ‘test.regrtest’ is executing + the tests. + + -- Function: test.support.python_is_optimized () + + Return ‘True’ if Python was not built with ‘-O0’ or ‘-Og’. + + -- Function: test.support.with_pymalloc () + + Return ‘_testcapi.WITH_PYMALLOC’. + + -- Function: test.support.requires (resource, msg=None) + + Raise *note ResourceDenied: 3147. if `resource' is not available. + `msg' is the argument to *note ResourceDenied: 3147. if it is + raised. Always returns ‘True’ if called by a function whose + ‘__name__’ is ‘'__main__'’. Used when tests are executed by + ‘test.regrtest’. + + -- Function: test.support.system_must_validate_cert (f) + + Raise *note unittest.SkipTest: 903. on TLS certification validation + failures. + + -- Function: test.support.sortdict (dict) + + Return a repr of `dict' with keys sorted. + + -- Function: test.support.findfile (filename, subdir=None) + + Return the path to the file named `filename'. If no match is found + `filename' is returned. This does not equal a failure since it + could be the path to the file. + + Setting `subdir' indicates a relative path to use to find the file + rather than looking directly in the path directories. + + -- Function: test.support.create_empty_file (filename) + + Create an empty file with `filename'. If it already exists, + truncate it. + + -- Function: test.support.fd_count () + + Count the number of open file descriptors. + + -- Function: test.support.match_test (test) + + Match `test' to patterns set in *note set_match_tests(): 3175. + + -- Function: test.support.set_match_tests (patterns) + + Define match test with regular expression `patterns'. + + -- Function: test.support.run_unittest (*classes) + + Execute *note unittest.TestCase: 8ff. subclasses passed to the + function. The function scans the classes for methods starting with + the prefix ‘test_’ and executes the tests individually. + + It is also legal to pass strings as parameters; these should be + keys in ‘sys.modules’. Each associated module will be scanned by + ‘unittest.TestLoader.loadTestsFromModule()’. This is usually seen + in the following ‘test_main()’ function: + + def test_main(): + support.run_unittest(__name__) + + This will run all tests defined in the named module. + + -- Function: test.support.run_doctest (module, verbosity=None, + optionflags=0) + + Run *note doctest.testmod(): dd0. on the given `module'. Return + ‘(failure_count, test_count)’. + + If `verbosity' is ‘None’, *note doctest.testmod(): dd0. is run with + verbosity set to *note verbose: 3149. Otherwise, it is run with + verbosity set to ‘None’. `optionflags' is passed as ‘optionflags’ + to *note doctest.testmod(): dd0. + + -- Function: test.support.setswitchinterval (interval) + + Set the *note sys.setswitchinterval(): beb. to the given + `interval'. Defines a minimum interval for Android systems to + prevent the system from hanging. + + -- Function: test.support.check_impl_detail (**guards) + + Use this check to guard CPython’s implementation-specific tests or + to run them only on the implementations guarded by the arguments: + + check_impl_detail() # Only on CPython (default). + check_impl_detail(jython=True) # Only on Jython. + check_impl_detail(cpython=False) # Everywhere except CPython. + + -- Function: test.support.check_warnings (*filters, quiet=True) + + A convenience wrapper for *note warnings.catch_warnings(): 317b. + that makes it easier to test that a warning was correctly raised. + It is approximately equivalent to calling + ‘warnings.catch_warnings(record=True)’ with *note + warnings.simplefilter(): 317c. set to ‘always’ and with the option + to automatically validate the results that are recorded. + + ‘check_warnings’ accepts 2-tuples of the form ‘("message regexp", + WarningCategory)’ as positional arguments. If one or more + `filters' are provided, or if the optional keyword argument `quiet' + is ‘False’, it checks to make sure the warnings are as expected: + each specified filter must match at least one of the warnings + raised by the enclosed code or the test fails, and if any warnings + are raised that do not match any of the specified filters the test + fails. To disable the first of these checks, set `quiet' to + ‘True’. + + If no arguments are specified, it defaults to: + + check_warnings(("", Warning), quiet=True) + + In this case all warnings are caught and no errors are raised. + + On entry to the context manager, a ‘WarningRecorder’ instance is + returned. The underlying warnings list from *note + catch_warnings(): 317b. is available via the recorder object’s + *note warnings: 126. attribute. As a convenience, the attributes + of the object representing the most recent warning can also be + accessed directly through the recorder object (see example below). + If no warning has been raised, then any of the attributes that + would otherwise be expected on an object representing a warning + will return ‘None’. + + The recorder object also has a ‘reset()’ method, which clears the + warnings list. + + The context manager is designed to be used like this: + + with check_warnings(("assertion is always true", SyntaxWarning), + ("", UserWarning)): + exec('assert(False, "Hey!")') + warnings.warn(UserWarning("Hide me!")) + + In this case if either warning was not raised, or some other + warning was raised, *note check_warnings(): 317a. would raise an + error. + + When a test needs to look more deeply into the warnings, rather + than just checking whether or not they occurred, code like this can + be used: + + with check_warnings(quiet=True) as w: + warnings.warn("foo") + assert str(w.args[0]) == "foo" + warnings.warn("bar") + assert str(w.args[0]) == "bar" + assert str(w.warnings[0].args[0]) == "foo" + assert str(w.warnings[1].args[0]) == "bar" + w.reset() + assert len(w.warnings) == 0 + + Here all warnings will be caught, and the test code tests the + captured warnings directly. + + Changed in version 3.2: New optional arguments `filters' and + `quiet'. + + -- Function: test.support.check_no_resource_warning (testcase) + + Context manager to check that no *note ResourceWarning: 4d0. was + raised. You must remove the object which may emit *note + ResourceWarning: 4d0. before the end of the context manager. + + -- Function: test.support.set_memlimit (limit) + + Set the values for *note max_memuse: 315d. and *note + real_max_memuse: 315f. for big memory tests. + + -- Function: test.support.record_original_stdout (stdout) + + Store the value from `stdout'. It is meant to hold the stdout at + the time the regrtest began. + + -- Function: test.support.get_original_stdout () + + Return the original stdout set by *note record_original_stdout(): + 317e. or ‘sys.stdout’ if it’s not set. + + -- Function: test.support.strip_python_strerr (stderr) + + Strip the `stderr' of a Python process from potential debug output + emitted by the interpreter. This will typically be run on the + result of *note subprocess.Popen.communicate(): 2142. + + -- Function: test.support.args_from_interpreter_flags () + + Return a list of command line arguments reproducing the current + settings in ‘sys.flags’ and ‘sys.warnoptions’. + + -- Function: test.support.optim_args_from_interpreter_flags () + + Return a list of command line arguments reproducing the current + optimization settings in ‘sys.flags’. + + -- Function: test.support.captured_stdin () + -- Function: test.support.captured_stdout () + -- Function: test.support.captured_stderr () + + A context managers that temporarily replaces the named stream with + *note io.StringIO: 82d. object. + + Example use with output streams: + + with captured_stdout() as stdout, captured_stderr() as stderr: + print("hello") + print("error", file=sys.stderr) + assert stdout.getvalue() == "hello\n" + assert stderr.getvalue() == "error\n" + + Example use with input stream: + + with captured_stdin() as stdin: + stdin.write('hello\n') + stdin.seek(0) + # call test code that consumes from sys.stdin + captured = input() + self.assertEqual(captured, "hello") + + -- Function: test.support.temp_dir (path=None, quiet=False) + + A context manager that creates a temporary directory at `path' and + yields the directory. + + If `path' is ‘None’, the temporary directory is created using *note + tempfile.mkdtemp(): 1927. If `quiet' is ‘False’, the context + manager raises an exception on error. Otherwise, if `path' is + specified and cannot be created, only a warning is issued. + + -- Function: test.support.change_cwd (path, quiet=False) + + A context manager that temporarily changes the current working + directory to `path' and yields the directory. + + If `quiet' is ‘False’, the context manager raises an exception on + error. Otherwise, it issues only a warning and keeps the current + working directory the same. + + -- Function: test.support.temp_cwd (name='tempcwd', quiet=False) + + A context manager that temporarily creates a new directory and + changes the current working directory (CWD). + + The context manager creates a temporary directory in the current + directory with name `name' before temporarily changing the current + working directory. If `name' is ‘None’, the temporary directory is + created using *note tempfile.mkdtemp(): 1927. + + If `quiet' is ‘False’ and it is not possible to create or change + the CWD, an error is raised. Otherwise, only a warning is raised + and the original CWD is used. + + -- Function: test.support.temp_umask (umask) + + A context manager that temporarily sets the process umask. + + -- Function: test.support.transient_internet (resource_name, *, + timeout=30.0, errnos=()) + + A context manager that raises *note ResourceDenied: 3147. when + various issues with the internet connection manifest themselves as + exceptions. + + -- Function: test.support.disable_faulthandler () + + A context manager that replaces ‘sys.stderr’ with ‘sys.__stderr__’. + + -- Function: test.support.gc_collect () + + Force as many objects as possible to be collected. This is needed + because timely deallocation is not guaranteed by the garbage + collector. This means that ‘__del__’ methods may be called later + than expected and weakrefs may remain alive for longer than + expected. + + -- Function: test.support.disable_gc () + + A context manager that disables the garbage collector upon entry + and reenables it upon exit. + + -- Function: test.support.swap_attr (obj, attr, new_val) + + Context manager to swap out an attribute with a new object. + + Usage: + + with swap_attr(obj, "attr", 5): + ... + + This will set ‘obj.attr’ to 5 for the duration of the ‘with’ block, + restoring the old value at the end of the block. If ‘attr’ doesn’t + exist on ‘obj’, it will be created and then deleted at the end of + the block. + + The old value (or ‘None’ if it doesn’t exist) will be assigned to + the target of the “as” clause, if there is one. + + -- Function: test.support.swap_item (obj, attr, new_val) + + Context manager to swap out an item with a new object. + + Usage: + + with swap_item(obj, "item", 5): + ... + + This will set ‘obj["item"]’ to 5 for the duration of the ‘with’ + block, restoring the old value at the end of the block. If ‘item’ + doesn’t exist on ‘obj’, it will be created and then deleted at the + end of the block. + + The old value (or ‘None’ if it doesn’t exist) will be assigned to + the target of the “as” clause, if there is one. + + -- Function: test.support.wait_threads_exit (timeout=60.0) + + Context manager to wait until all threads created in the ‘with’ + statement exit. + + -- Function: test.support.start_threads (threads, unlock=None) + + Context manager to start `threads'. It attempts to join the + threads upon exit. + + -- Function: test.support.calcobjsize (fmt) + + Return *note struct.calcsize(): 14b8. for ‘nP{fmt}0n’ or, if + ‘gettotalrefcount’ exists, ‘2PnP{fmt}0P’. + + -- Function: test.support.calcvobjsize (fmt) + + Return *note struct.calcsize(): 14b8. for ‘nPn{fmt}0n’ or, if + ‘gettotalrefcount’ exists, ‘2PnPn{fmt}0P’. + + -- Function: test.support.checksizeof (test, o, size) + + For testcase `test', assert that the ‘sys.getsizeof’ for `o' plus + the GC header size equals `size'. + + -- Function: test.support.can_symlink () + + Return ‘True’ if the OS supports symbolic links, ‘False’ otherwise. + + -- Function: test.support.can_xattr () + + Return ‘True’ if the OS supports xattr, ‘False’ otherwise. + + -- Function: @test.support.skip_unless_symlink + + A decorator for running tests that require support for symbolic + links. + + -- Function: @test.support.skip_unless_xattr + + A decorator for running tests that require support for xattr. + + -- Function: @test.support.skip_unless_bind_unix_socket + + A decorator for running tests that require a functional bind() for + Unix sockets. + + -- Function: @test.support.anticipate_failure (condition) + + A decorator to conditionally mark tests with *note + unittest.expectedFailure(): 3049. Any use of this decorator should + have an associated comment identifying the relevant tracker issue. + + -- Function: @test.support.run_with_locale (catstr, *locales) + + A decorator for running a function in a different locale, correctly + resetting it after it has finished. `catstr' is the locale + category as a string (for example ‘"LC_ALL"’). The `locales' + passed will be tried sequentially, and the first valid locale will + be used. + + -- Function: @test.support.run_with_tz (tz) + + A decorator for running a function in a specific timezone, + correctly resetting it after it has finished. + + -- Function: @test.support.requires_freebsd_version (*min_version) + + Decorator for the minimum version when running test on FreeBSD. If + the FreeBSD version is less than the minimum, raise *note + unittest.SkipTest: 903. + + -- Function: @test.support.requires_linux_version (*min_version) + + Decorator for the minimum version when running test on Linux. If + the Linux version is less than the minimum, raise *note + unittest.SkipTest: 903. + + -- Function: @test.support.requires_mac_version (*min_version) + + Decorator for the minimum version when running test on Mac OS X. If + the MAC OS X version is less than the minimum, raise *note + unittest.SkipTest: 903. + + -- Function: @test.support.requires_IEEE_754 + + Decorator for skipping tests on non-IEEE 754 platforms. + + -- Function: @test.support.requires_zlib + + Decorator for skipping tests if *note zlib: 144. doesn’t exist. + + -- Function: @test.support.requires_gzip + + Decorator for skipping tests if *note gzip: 8c. doesn’t exist. + + -- Function: @test.support.requires_bz2 + + Decorator for skipping tests if *note bz2: 14. doesn’t exist. + + -- Function: @test.support.requires_lzma + + Decorator for skipping tests if *note lzma: ad. doesn’t exist. + + -- Function: @test.support.requires_resource (resource) + + Decorator for skipping tests if `resource' is not available. + + -- Function: @test.support.requires_docstrings + + Decorator for only running the test if *note HAVE_DOCSTRINGS: 3161. + + -- Function: @test.support.cpython_only (test) + + Decorator for tests only applicable to CPython. + + -- Function: @test.support.impl_detail (msg=None, **guards) + + Decorator for invoking *note check_impl_detail(): 3179. on + `guards'. If that returns ‘False’, then uses `msg' as the reason + for skipping the test. + + -- Function: @test.support.no_tracing (func) + + Decorator to temporarily turn off tracing for the duration of the + test. + + -- Function: @test.support.refcount_test (test) + + Decorator for tests which involve reference counting. The + decorator does not run the test if it is not run by CPython. Any + trace function is unset for the duration of the test to prevent + unexpected refcounts caused by the trace function. + + -- Function: @test.support.reap_threads (func) + + Decorator to ensure the threads are cleaned up even if the test + fails. + + -- Function: @test.support.bigmemtest (size, memuse, dry_run=True) + + Decorator for bigmem tests. + + `size' is a requested size for the test (in arbitrary, + test-interpreted units.) `memuse' is the number of bytes per unit + for the test, or a good estimate of it. For example, a test that + needs two byte buffers, of 4 GiB each, could be decorated with + ‘@bigmemtest(size=_4G, memuse=2)’. + + The `size' argument is normally passed to the decorated test method + as an extra argument. If `dry_run' is ‘True’, the value passed to + the test method may be less than the requested value. If `dry_run' + is ‘False’, it means the test doesn’t support dummy runs when ‘-M’ + is not specified. + + -- Function: @test.support.bigaddrspacetest (f) + + Decorator for tests that fill the address space. `f' is the + function to wrap. + + -- Function: test.support.make_bad_fd () + + Create an invalid file descriptor by opening and closing a + temporary file, and returning its descriptor. + + -- Function: test.support.check_syntax_error (testcase, statement, + errtext='', *, lineno=None, offset=None) + + Test for syntax errors in `statement' by attempting to compile + `statement'. `testcase' is the *note unittest: 11b. instance for + the test. `errtext' is the regular expression which should match + the string representation of the raised *note SyntaxError: 458. If + `lineno' is not ‘None’, compares to the line of the exception. If + `offset' is not ‘None’, compares to the offset of the exception. + + -- Function: test.support.check_syntax_warning (testcase, statement, + errtext='', *, lineno=1, offset=None) + + Test for syntax warning in `statement' by attempting to compile + `statement'. Test also that the *note SyntaxWarning: 191. is + emitted only once, and that it will be converted to a *note + SyntaxError: 458. when turned into error. `testcase' is the *note + unittest: 11b. instance for the test. `errtext' is the regular + expression which should match the string representation of the + emitted *note SyntaxWarning: 191. and raised *note SyntaxError: + 458. If `lineno' is not ‘None’, compares to the line of the + warning and exception. If `offset' is not ‘None’, compares to the + offset of the exception. + + New in version 3.8. + + -- Function: test.support.open_urlresource (url, *args, **kw) + + Open `url'. If open fails, raises *note TestFailed: 3146. + + -- Function: test.support.import_module (name, deprecated=False, *, + required_on()) + + This function imports and returns the named module. Unlike a + normal import, this function raises *note unittest.SkipTest: 903. + if the module cannot be imported. + + Module and package deprecation messages are suppressed during this + import if `deprecated' is ‘True’. If a module is required on a + platform but optional for others, set `required_on' to an iterable + of platform prefixes which will be compared against *note + sys.platform: 2b1. + + New in version 3.1. + + -- Function: test.support.import_fresh_module (name, fresh=(), + blocked=(), deprecated=False) + + This function imports and returns a fresh copy of the named Python + module by removing the named module from ‘sys.modules’ before doing + the import. Note that unlike ‘reload()’, the original module is + not affected by this operation. + + `fresh' is an iterable of additional module names that are also + removed from the ‘sys.modules’ cache before doing the import. + + `blocked' is an iterable of module names that are replaced with + ‘None’ in the module cache during the import to ensure that + attempts to import them raise *note ImportError: 334. + + The named module and any modules named in the `fresh' and `blocked' + parameters are saved before starting the import and then reinserted + into ‘sys.modules’ when the fresh import is complete. + + Module and package deprecation messages are suppressed during this + import if `deprecated' is ‘True’. + + This function will raise *note ImportError: 334. if the named + module cannot be imported. + + Example use: + + # Get copies of the warnings module for testing without affecting the + # version being used by the rest of the test suite. One copy uses the + # C implementation, the other is forced to use the pure Python fallback + # implementation + py_warnings = import_fresh_module('warnings', blocked=['_warnings']) + c_warnings = import_fresh_module('warnings', fresh=['_warnings']) + + New in version 3.1. + + -- Function: test.support.modules_setup () + + Return a copy of *note sys.modules: 1152. + + -- Function: test.support.modules_cleanup (oldmodules) + + Remove modules except for `oldmodules' and ‘encodings’ in order to + preserve internal cache. + + -- Function: test.support.threading_setup () + + Return current thread count and copy of dangling threads. + + -- Function: test.support.threading_cleanup (*original_values) + + Cleanup up threads not specified in `original_values'. Designed to + emit a warning if a test leaves running threads in the background. + + -- Function: test.support.join_thread (thread, timeout=30.0) + + Join a `thread' within `timeout'. Raise an *note AssertionError: + 1223. if thread is still alive after `timeout' seconds. + + -- Function: test.support.reap_children () + + Use this at the end of ‘test_main’ whenever sub-processes are + started. This will help ensure that no extra children (zombies) + stick around to hog resources and create problems when looking for + refleaks. + + -- Function: test.support.get_attribute (obj, name) + + Get an attribute, raising *note unittest.SkipTest: 903. if *note + AttributeError: 39b. is raised. + + -- Function: test.support.bind_port (sock, host=HOST) + + Bind the socket to a free port and return the port number. Relies + on ephemeral ports in order to ensure we are using an unbound port. + This is important as many tests may be running simultaneously, + especially in a buildbot environment. This method raises an + exception if the ‘sock.family’ is *note AF_INET: 229d. and + ‘sock.type’ is *note SOCK_STREAM: c8a, and the socket has + ‘SO_REUSEADDR’ or ‘SO_REUSEPORT’ set on it. Tests should never set + these socket options for TCP/IP sockets. The only case for setting + these options is testing multicasting via multiple UDP sockets. + + Additionally, if the ‘SO_EXCLUSIVEADDRUSE’ socket option is + available (i.e. on Windows), it will be set on the socket. This + will prevent anyone else from binding to our host/port for the + duration of the test. + + -- Function: test.support.bind_unix_socket (sock, addr) + + Bind a unix socket, raising *note unittest.SkipTest: 903. if *note + PermissionError: 5ff. is raised. + + -- Function: test.support.catch_threading_exception () + + Context manager catching *note threading.Thread: 235. exception + using *note threading.excepthook(): 231. + + Attributes set when an exception is catched: + + * ‘exc_type’ + + * ‘exc_value’ + + * ‘exc_traceback’ + + * ‘thread’ + + See *note threading.excepthook(): 231. documentation. + + These attributes are deleted at the context manager exit. + + Usage: + + with support.catch_threading_exception() as cm: + # code spawning a thread which raises an exception + ... + + # check the thread exception, use cm attributes: + # exc_type, exc_value, exc_traceback, thread + ... + + # exc_type, exc_value, exc_traceback, thread attributes of cm no longer + # exists at this point + # (to avoid reference cycles) + + New in version 3.8. + + -- Function: test.support.catch_unraisable_exception () + + Context manager catching unraisable exception using *note + sys.unraisablehook(): 22d. + + Storing the exception value (‘cm.unraisable.exc_value’) creates a + reference cycle. The reference cycle is broken explicitly when the + context manager exits. + + Storing the object (‘cm.unraisable.object’) can resurrect it if it + is set to an object which is being finalized. Exiting the context + manager clears the stored object. + + Usage: + + with support.catch_unraisable_exception() as cm: + # code creating an "unraisable exception" + ... + + # check the unraisable exception: use cm.unraisable + ... + + # cm.unraisable attribute no longer exists at this point + # (to break a reference cycle) + + New in version 3.8. + + -- Function: test.support.find_unused_port (family=socket.AF_INET, + socktype=socket.SOCK_STREAM) + + Returns an unused port that should be suitable for binding. This + is achieved by creating a temporary socket with the same family and + type as the ‘sock’ parameter (default is *note AF_INET: 229d, *note + SOCK_STREAM: c8a.), and binding it to the specified host address + (defaults to ‘0.0.0.0’) with the port set to 0, eliciting an unused + ephemeral port from the OS. The temporary socket is then closed and + deleted, and the ephemeral port is returned. + + Either this method or *note bind_port(): 31bb. should be used for + any tests where a server socket needs to be bound to a particular + port for the duration of the test. Which one to use depends on + whether the calling code is creating a Python socket, or if an + unused port needs to be provided in a constructor or passed to an + external program (i.e. the ‘-accept’ argument to openssl’s + s_server mode). Always prefer *note bind_port(): 31bb. over *note + find_unused_port(): 31bf. where possible. Using a hard coded port + is discouraged since it can make multiple instances of the test + impossible to run simultaneously, which is a problem for buildbots. + + -- Function: test.support.load_package_tests (pkg_dir, loader, + standard_tests, pattern) + + Generic implementation of the *note unittest: 11b. ‘load_tests’ + protocol for use in test packages. `pkg_dir' is the root directory + of the package; `loader', `standard_tests', and `pattern' are the + arguments expected by ‘load_tests’. In simple cases, the test + package’s ‘__init__.py’ can be the following: + + import os + from test.support import load_package_tests + + def load_tests(*args): + return load_package_tests(os.path.dirname(__file__), *args) + + -- Function: test.support.fs_is_case_insensitive (directory) + + Return ‘True’ if the file system for `directory' is + case-insensitive. + + -- Function: test.support.detect_api_mismatch (ref_api, other_api, *, + ignore=()) + + Returns the set of attributes, functions or methods of `ref_api' + not found on `other_api', except for a defined list of items to be + ignored in this check specified in `ignore'. + + By default this skips private attributes beginning with ‘_’ but + includes all magic methods, i.e. those starting and ending in + ‘__’. + + New in version 3.5. + + -- Function: test.support.patch (test_instance, object_to_patch, + attr_name, new_value) + + Override `object_to_patch.attr_name' with `new_value'. Also add + cleanup procedure to `test_instance' to restore `object_to_patch' + for `attr_name'. The `attr_name' should be a valid attribute for + `object_to_patch'. + + -- Function: test.support.run_in_subinterp (code) + + Run `code' in subinterpreter. Raise *note unittest.SkipTest: 903. + if *note tracemalloc: 114. is enabled. + + -- Function: test.support.check_free_after_iterating (test, iter, cls, + args=()) + + Assert that `iter' is deallocated after iterating. + + -- Function: test.support.missing_compiler_executable (cmd_names=[]) + + Check for the existence of the compiler executables whose names are + listed in `cmd_names' or all the compiler executables when + `cmd_names' is empty and return the first missing executable or + ‘None’ when none is found missing. + + -- Function: test.support.check__all__ (test_case, module, + name_of_module=None, extra=(), blacklist=()) + + Assert that the ‘__all__’ variable of `module' contains all public + names. + + The module’s public names (its API) are detected automatically + based on whether they match the public name convention and were + defined in `module'. + + The `name_of_module' argument can specify (as a string or tuple + thereof) what module(s) an API could be defined in order to be + detected as a public API. One case for this is when `module' + imports part of its public API from other modules, possibly a C + backend (like ‘csv’ and its ‘_csv’). + + The `extra' argument can be a set of names that wouldn’t otherwise + be automatically detected as “public”, like objects without a + proper ‘__module__’ attribute. If provided, it will be added to + the automatically detected ones. + + The `blacklist' argument can be a set of names that must not be + treated as part of the public API even though their names indicate + otherwise. + + Example use: + + import bar + import foo + import unittest + from test import support + + class MiscTestCase(unittest.TestCase): + def test__all__(self): + support.check__all__(self, foo) + + class OtherTestCase(unittest.TestCase): + def test__all__(self): + extra = {'BAR_CONST', 'FOO_CONST'} + blacklist = {'baz'} # Undocumented name. + # bar imports part of its API from _bar. + support.check__all__(self, bar, ('bar', '_bar'), + extra=extra, blacklist=blacklist) + + New in version 3.6. + +The *note test.support: 106. module defines the following classes: + + -- Class: test.support.TransientResource (exc, **kwargs) + + Instances are a context manager that raises *note ResourceDenied: + 3147. if the specified exception type is raised. Any keyword + arguments are treated as attribute/value pairs to be compared + against any exception raised within the *note with: 6e9. statement. + Only if all pairs match properly against attributes on the + exception is *note ResourceDenied: 3147. raised. + + -- Class: test.support.EnvironmentVarGuard + + Class used to temporarily set or unset environment variables. + Instances can be used as a context manager and have a complete + dictionary interface for querying/modifying the underlying + ‘os.environ’. After exit from the context manager all changes to + environment variables done through this instance will be rolled + back. + + Changed in version 3.1: Added dictionary interface. + + -- Method: EnvironmentVarGuard.set (envvar, value) + + Temporarily set the environment variable ‘envvar’ to the value of + ‘value’. + + -- Method: EnvironmentVarGuard.unset (envvar) + + Temporarily unset the environment variable ‘envvar’. + + -- Class: test.support.SuppressCrashReport + + A context manager used to try to prevent crash dialog popups on + tests that are expected to crash a subprocess. + + On Windows, it disables Windows Error Reporting dialogs using + SetErrorMode(3). + + On UNIX, *note resource.setrlimit(): 31cd. is used to set *note + resource.RLIMIT_CORE: 31ce.’s soft limit to 0 to prevent coredump + file creation. + + On both platforms, the old value is restored by *note __exit__(): + c96. + + -- Class: test.support.CleanImport (*module_names) + + A context manager to force import to return a new module reference. + This is useful for testing module-level behaviors, such as the + emission of a DeprecationWarning on import. Example usage: + + with CleanImport('foo'): + importlib.import_module('foo') # New reference. + + -- Class: test.support.DirsOnSysPath (*paths) + + A context manager to temporarily add directories to sys.path. + + This makes a copy of *note sys.path: 488, appends any directories + given as positional arguments, then reverts *note sys.path: 488. to + the copied settings when the context ends. + + Note that `all' *note sys.path: 488. modifications in the body of + the context manager, including replacement of the object, will be + reverted at the end of the block. + + -- Class: test.support.SaveSignals + + Class to save and restore signal handlers registered by the Python + signal handler. + + -- Class: test.support.Matcher + + -- Method: matches (self, d, **kwargs) + + Try to match a single dict with the supplied arguments. + + -- Method: match_value (self, k, dv, v) + + Try to match a single stored value (`dv') with a supplied + value (`v'). + + -- Class: test.support.WarningsRecorder + + Class used to record warnings for unit tests. See documentation of + *note check_warnings(): 317a. above for more details. + + -- Class: test.support.BasicTestRunner + + -- Method: run (test) + + Run `test' and return the result. + + -- Class: test.support.TestHandler (logging.handlers.BufferingHandler) + + Class for logging support. + + -- Class: test.support.FakePath (path) + + Simple *note path-like object: 36a. It implements the + ‘__fspath__()’ method which just returns the `path' argument. If + `path' is an exception, it will be raised in ‘__fspath__()’. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + (2) https://www.python.org/dev/peps/pep-0488 + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/ms680621.aspx + + +File: python.info, Node: test support script_helper — Utilities for the Python execution tests, Prev: test support — Utilities for the Python test suite, Up: Development Tools + +5.26.10 ‘test.support.script_helper’ — Utilities for the Python execution tests +------------------------------------------------------------------------------- + +The *note test.support.script_helper: 107. module provides support for +Python’s script execution tests. + + -- Function: + test.support.script_helper.interpreter_requires_environment () + + Return ‘True’ if ‘sys.executable interpreter’ requires environment + variables in order to be able to run at all. + + This is designed to be used with ‘@unittest.skipIf()’ to annotate + tests that need to use an ‘assert_python*()’ function to launch an + isolated mode (‘-I’) or no environment mode (‘-E’) sub-interpreter + process. + + A normal build & test does not run into this situation but it can + happen when trying to run the standard library test suite from an + interpreter that doesn’t have an obvious home with Python’s current + home finding logic. + + Setting *note PYTHONHOME: 4d6. is one way to get most of the + testsuite to run in that situation. *note PYTHONPATH: 953. or + ‘PYTHONUSERSITE’ are other common environment variables that might + impact whether or not the interpreter can start. + + -- Function: test.support.script_helper.run_python_until_end (*args, + **env_vars) + + Set up the environment based on `env_vars' for running the + interpreter in a subprocess. The values can include ‘__isolated’, + ‘__cleanenv’, ‘__cwd’, and ‘TERM’. + + -- Function: test.support.script_helper.assert_python_ok (*args, + **env_vars) + + Assert that running the interpreter with `args' and optional + environment variables `env_vars' succeeds (‘rc == 0’) and return a + ‘(return code, stdout, stderr)’ tuple. + + If the ‘__cleanenv’ keyword is set, `env_vars' is used as a fresh + environment. + + Python is started in isolated mode (command line option ‘-I’), + except if the ‘__isolated’ keyword is set to ‘False’. + + -- Function: test.support.script_helper.assert_python_failure (*args, + **env_vars) + + Assert that running the interpreter with `args' and optional + environment variables `env_vars' fails (‘rc != 0’) and return a + ‘(return code, stdout, stderr)’ tuple. + + See *note assert_python_ok(): 31dd. for more options. + + -- Function: test.support.script_helper.spawn_python (*args, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kw) + + Run a Python subprocess with the given arguments. + + `kw' is extra keyword args to pass to *note subprocess.Popen(): + 2b9. Returns a *note subprocess.Popen: 2b9. object. + + -- Function: test.support.script_helper.kill_python (p) + + Run the given *note subprocess.Popen: 2b9. process until completion + and return stdout. + + -- Function: test.support.script_helper.make_script (script_dir, + script_basename, source, omit_suffix=False) + + Create script containing `source' in path `script_dir' and + `script_basename'. If `omit_suffix' is ‘False’, append ‘.py’ to + the name. Return the full script path. + + -- Function: test.support.script_helper.make_zip_script (zip_dir, + zip_basename, script_name, name_in_zip=None) + + Create zip file at `zip_dir' and `zip_basename' with extension + ‘zip’ which contains the files in `script_name'. `name_in_zip' is + the archive name. Return a tuple containing ‘(full path, full path + of archive name)’. + + -- Function: test.support.script_helper.make_pkg (pkg_dir, + init_source='') + + Create a directory named `pkg_dir' containing an ‘__init__’ file + with `init_source' as its contents. + + -- Function: test.support.script_helper.make_zip_pkg (zip_dir, + zip_basename, pkg_name, script_basename, source, depth=1, + compiled=False) + + Create a zip package directory with a path of `zip_dir' and + `zip_basename' containing an empty ‘__init__’ file and a file + `script_basename' containing the `source'. If `compiled' is + ‘True’, both source files will be compiled and added to the zip + package. Return a tuple of the full zip path and the archive name + for the zip file. + +See also the Python development mode: the *note -X: 155. ‘dev’ option +and *note PYTHONDEVMODE: 32c. environment variable. + + +File: python.info, Node: Debugging and Profiling, Next: Software Packaging and Distribution, Prev: Development Tools, Up: The Python Standard Library + +5.27 Debugging and Profiling +============================ + +These libraries help you with Python development: the debugger enables +you to step through code, analyze stack frames and set breakpoints etc., +and the profilers run code and give you a detailed breakdown of +execution times, allowing you to identify bottlenecks in your programs. +Auditing events provide visibility into runtime behaviors that would +otherwise require intrusive debugging or patching. + +* Menu: + +* Audit events table:: +* bdb — Debugger framework:: +* faulthandler — Dump the Python traceback:: +* pdb — The Python Debugger:: +* The Python Profilers:: +* timeit — Measure execution time of small code snippets:: +* trace — Trace or track Python statement execution:: +* tracemalloc — Trace memory allocations:: + + +File: python.info, Node: Audit events table, Next: bdb — Debugger framework, Up: Debugging and Profiling + +5.27.1 Audit events table +------------------------- + +This table contains all events raised by *note sys.audit(): 31ea. or +*note PySys_Audit(): 31eb. calls throughout the CPython runtime and the +standard library. These calls were added in 3.8.0 or later. + +See *note sys.addaudithook(): 31ec. and *note PySys_AddAuditHook(): +31ed. for information on handling these events. + +`CPython implementation detail:' This table is generated from the +CPython documentation, and may not represent events raised by other +implementations. See your runtime specific documentation for actual +events raised. + +Audit event Arguments References + +------------------------------------------------------------------------------------------------------------------- + +array.__new__ ‘typecode’, ‘initializer’ *note [1]: 451. + + +builtins.breakpoint ‘breakpointhook’ *note [1]: 2f9. + + +builtins.id ‘id’ *note [1]: d86. + + +builtins.input ‘prompt’ *note [1]: c6b. + + +builtins.input/result ‘result’ *note [1]: c6b. + + +code.__new__ ‘code’, ‘filename’, ‘name’, ‘argcount’, *note [1]: 198. + ‘posonlyargcount’, ‘kwonlyargcount’, ‘nlocals’, + ‘stacksize’, ‘flags’ + + +compile ‘source’, ‘filename’ *note [1]: 1b4. + + +cpython.PyInterpreterState_Clear *note [1]: 31ee. + + +cpython.PyInterpreterState_New *note [1]: 31ef. + + +cpython._PySys_ClearAuditHooks *note [1]: 5c9. + + +cpython.run_command ‘command’ *note [1]: e9e. + + +cpython.run_file ‘filename’ *note [1]: 31f0. + + +cpython.run_interactivehook ‘hook’ *note [1]: 8e4. + + +cpython.run_module ‘module-name’ *note [1]: 337. + + +cpython.run_startup ‘filename’ *note [1]: 8e5. + + +cpython.run_stdin *note [1]: 31f1. + + +ctypes.addressof ‘obj’ *note [1]: 2004. + + +ctypes.call_function ‘func_pointer’, ‘arguments’ *note [1]: 1ff9. + + +ctypes.cdata ‘address’ *note [1]: 2017. + + +ctypes.cdata/buffer ‘pointer’, ‘size’, ‘offset’ *note [1]: 2015.*note [2]: 2016. + + +ctypes.create_string_buffer ‘init’, ‘size’ *note [1]: 1fb4. + + +ctypes.create_unicode_buffer ‘init’, ‘size’ *note [1]: 1fb5. + + +ctypes.dlopen ‘name’ *note [1]: 1ff6. + + +ctypes.dlsym ‘library’, ‘name’ *note [1]: 1ff6. + + +ctypes.dlsym/handle ‘handle’, ‘name’ *note [1]: 1ff6. + + +ctypes.get_errno *note [1]: 1ff0. + + +ctypes.get_last_error *note [1]: 1ff2. + + +ctypes.seh_exception ‘code’ *note [1]: 1ff9. + + +ctypes.set_errno ‘errno’ *note [1]: 1ff1. + + +ctypes.set_last_error ‘error’ *note [1]: 1ff3. + + +ctypes.string_at ‘address’, ‘size’ *note [1]: 200d. + + +ctypes.wstring_at ‘address’, ‘size’ *note [1]: 200f. + + +ensurepip.bootstrap ‘root’ *note [1]: 31f2. + + +exec ‘code_object’ *note [1]: b90.*note [2]: c44. + + +fcntl.fcntl ‘fd’, ‘cmd’, ‘arg’ *note [1]: 2393. + + +fcntl.flock ‘fd’, ‘operation’ *note [1]: 31f3. + + +fcntl.ioctl ‘fd’, ‘request’, ‘arg’ *note [1]: dde. + + +fcntl.lockf ‘fd’, ‘cmd’, ‘len’, ‘start’, ‘whence’ *note [1]: d52. + + +ftplib.connect ‘self’, ‘host’, ‘port’ *note [1]: 2a21. + + +ftplib.sendcmd ‘self’, ‘cmd’ *note [1]: 2a25.*note [2]: 2a26. + + +function.__new__ ‘code’ *note [1]: 1658. + + +gc.get_objects ‘generation’ *note [1]: 1cd. + + +gc.get_referents ‘objs’ *note [1]: 31f4. + + +gc.get_referrers ‘objs’ *note [1]: 31f5. + + +glob.glob ‘pathname’, ‘recursive’ *note [1]: 5c6.*note [2]: 5c7. + + +imaplib.open ‘self’, ‘host’, ‘port’ *note [1]: 2a71. + + +imaplib.send ‘self’, ‘data’ *note [1]: 2a74. + + +import ‘module’, ‘filename’, ‘sys.path’, ‘sys.meta_path’, *note [1]: c1d. + ‘sys.path_hooks’ + + +mmap.__new__ ‘fileno’, ‘length’, ‘access’, ‘offset’ *note [1]: 1ec. + + +msvcrt.get_osfhandle ‘fd’ *note [1]: 31f6. + + +msvcrt.locking ‘fd’, ‘mode’, ‘nbytes’ *note [1]: 31f7. + + +msvcrt.open_osfhandle ‘handle’, ‘flags’ *note [1]: 31f8. + + +nntplib.connect ‘self’, ‘host’, ‘port’ *note [1]: a2c.*note [2]: ba5. + + +nntplib.putline ‘self’, ‘line’ *note [1]: a2c.*note [2]: ba5. + + +object.__delattr__ ‘obj’, ‘name’ *note [1]: 10d1. + + +object.__getattr__ ‘obj’, ‘name’ *note [1]: 449. + + +object.__setattr__ ‘obj’, ‘name’, ‘value’ *note [1]: e2c. + + +open ‘file’, ‘mode’, ‘flags’ *note [1]: 4f0.*note [2]: 65a.*note [3]: 665. + + +os.add_dll_directory ‘path’ *note [1]: 1c2. + + +os.chdir ‘path’ *note [1]: a3f.*note [2]: 65b. + + +os.chflags ‘path’, ‘flags’ *note [1]: a32.*note [2]: 1c22. + + +os.chmod ‘path’, ‘mode’, ‘dir_fd’ *note [1]: a33.*note [2]: 65c.*note [3]: 1c23. + + +os.chown ‘path’, ‘uid’, ‘gid’, ‘dir_fd’ *note [1]: a34.*note [2]: 65d.*note [3]: 1c24. + + +os.exec ‘path’, ‘args’, ‘env’ *note [1]: 1c60. + + +os.fork *note [1]: 961. + + +os.forkpty *note [1]: 1c78. + + +os.getxattr ‘path’, ‘attribute’ *note [1]: a4b. + + +os.kill ‘pid’, ‘sig’ *note [1]: cf3. + + +os.killpg ‘pgid’, ‘sig’ *note [1]: 1c7b. + + +os.link ‘src’, ‘dst’, ‘src_dir_fd’, ‘dst_dir_fd’ *note [1]: a35. + + +os.listdir ‘path’ *note [1]: a41. + + +os.listxattr ‘path’ *note [1]: a4c. + + +os.lockf ‘fd’, ‘cmd’, ‘len’ *note [1]: a5a. + + +os.mkdir ‘path’, ‘mode’, ‘dir_fd’ *note [1]: 3bd.*note [2]: a36. + + +os.posix_spawn ‘path’, ‘argv’, ‘env’ *note [1]: 257.*note [2]: 1c7d. + + +os.putenv ‘key’, ‘value’ *note [1]: 1bb6. + + +os.remove ‘path’, ‘dir_fd’ *note [1]: a37.*note [2]: 1c2b.*note [3]: a3c. + + +os.removexattr ‘path’, ‘attribute’ *note [1]: a4d. + + +os.rename ‘src’, ‘dst’, ‘src_dir_fd’, ‘dst_dir_fd’ *note [1]: a38.*note [2]: 1c2c.*note [3]: a39. + + +os.rmdir ‘path’, ‘dir_fd’ *note [1]: a3a. + + +os.scandir ‘path’ *note [1]: 25f. + + +os.setxattr ‘path’, ‘attribute’, ‘value’, ‘flags’ *note [1]: a4e. + + +os.spawn ‘mode’, ‘path’, ‘args’, ‘env’ *note [1]: 1c1a. + + +os.startfile ‘path’, ‘operation’ *note [1]: 1c8e. + + +os.symlink ‘src’, ‘dst’, ‘dir_fd’ *note [1]: a3b. + + +os.system ‘command’ *note [1]: dc1. + + +os.truncate ‘fd’, ‘length’ *note [1]: 662.*note [2]: 724. + + +os.unsetenv ‘key’ *note [1]: d43. + + +os.utime ‘path’, ‘times’, ‘ns’, ‘dir_fd’ *note [1]: a3d. + + +pdb.Pdb *note [1]: 56c. + + +pickle.find_class ‘module’, ‘name’ *note [1]: 1983. + + +poplib.connect ‘self’, ‘host’, ‘port’ *note [1]: 2a3c.*note [2]: bc1. + + +poplib.putline ‘self’, ‘line’ *note [1]: 2a3c.*note [2]: bc1. + + +pty.spawn ‘argv’ *note [1]: 898. + + +resource.prlimit ‘pid’, ‘resource’, ‘limits’ *note [1]: 89f. + + +resource.setrlimit ‘resource’, ‘limits’ *note [1]: 31cd. + + +setopencodehook *note [1]: 1cc1. + + +shutil.chown ‘path’, ‘user’, ‘group’ *note [1]: a76. + + +shutil.copyfile ‘src’, ‘dst’ *note [1]: 259.*note [2]: 25a.*note [3]: 258. + + +shutil.copymode ‘src’, ‘dst’ *note [1]: 259.*note [2]: 1947. + + +shutil.copystat ‘src’, ‘dst’ *note [1]: 25a.*note [2]: a77. + + +shutil.copytree ‘src’, ‘dst’ *note [1]: 21a. + + +shutil.make_archive ‘base_name’, ‘format’, ‘root_dir’, ‘base_dir’ *note [1]: 21b. + + +shutil.move ‘src’, ‘dst’ *note [1]: 25b. + + +shutil.rmtree ‘path’ *note [1]: 21c. + + +shutil.unpack_archive ‘filename’, ‘extract_dir’, ‘format’ *note [1]: b97. + + +signal.pthread_kill ‘thread_id’, ‘signalnum’ *note [1]: a7a. + + +smtplib.connect ‘self’, ‘host’, ‘port’ *note [1]: 2aba. + + +smtplib.send ‘self’, ‘data’ *note [1]: a81. + + +socket.__new__ ‘self’, ‘family’, ‘type’, ‘protocol’ *note [1]: 674. + + +socket.bind ‘self’, ‘address’ *note [1]: 238d. + + +socket.connect ‘self’, ‘address’ *note [1]: 676.*note [2]: 238f. + + +socket.getaddrinfo ‘host’, ‘port’, ‘family’, ‘type’, ‘protocol’ *note [1]: 22b1. + + +socket.gethostbyaddr ‘ip_address’ *note [1]: 2364. + + +socket.gethostbyname ‘hostname’ *note [1]: 2382.*note [2]: 2363. + + +socket.gethostname *note [1]: 1bd6. + + +socket.getnameinfo ‘sockaddr’ *note [1]: 22b2. + + +socket.getservbyname ‘servicename’, ‘protocolname’ *note [1]: 2384. + + +socket.getservbyport ‘port’, ‘protocolname’ *note [1]: 2385. + + +socket.sendmsg ‘self’, ‘address’ *note [1]: 67c. + + +socket.sendto ‘self’, ‘address’ *note [1]: 67d. + + +socket.sethostname ‘name’ *note [1]: a87. + + +sqlite3.connect ‘database’ *note [1]: 3da. + + +subprocess.Popen ‘executable’, ‘args’, ‘cwd’, ‘env’ *note [1]: 2b9. + + +sys._current_frames *note [1]: d9b. + + +sys._getframe *note [1]: e64. + + +sys.addaudithook *note [1]: 31ed.*note [2]: 31ec. + + +sys.excepthook ‘hook’, ‘type’, ‘value’, ‘traceback’ *note [1]: e63. + + +sys.set_asyncgen_hooks_finalizer *note [1]: 11b2. + + +sys.set_asyncgen_hooks_firstiter *note [1]: 11b2. + + +sys.setprofile *note [1]: e3d. + + +sys.settrace *note [1]: e3e. + + +sys.unraisablehook ‘hook’, ‘unraisable’ *note [1]: 22d. + + +syslog.closelog *note [1]: 31f9. + + +syslog.openlog ‘ident’, ‘logoption’, ‘facility’ *note [1]: 31fa. + + +syslog.setlogmask ‘maskpri’ *note [1]: 31fb. + + +syslog.syslog ‘priority’, ‘message’ *note [1]: 31fc. + + +telnetlib.Telnet.open ‘self’, ‘host’, ‘port’ *note [1]: 2ae8. + + +telnetlib.Telnet.write ‘self’, ‘buffer’ *note [1]: 2af8. + + +tempfile.mkdtemp ‘fullpath’ *note [1]: bc8.*note [2]: 1927. + + +tempfile.mkstemp ‘fullpath’ *note [1]: d49.*note [2]: 1925.*note [3]: 1926. + + +urllib.Request ‘fullurl’, ‘data’, ‘headers’, ‘method’ *note [1]: 78c. + + +webbrowser.open ‘url’ *note [1]: 28d1. + + +winreg.ConnectRegistry ‘computer_name’, ‘key’ *note [1]: 31fd. + + +winreg.CreateKey ‘key’, ‘sub_key’, ‘access’ *note [1]: 31fe.*note [2]: 31ff. + + +winreg.DeleteKey ‘key’, ‘sub_key’, ‘access’ *note [1]: 3200.*note [2]: 3201. + + +winreg.DeleteValue ‘key’, ‘value’ *note [1]: 3202. + + +winreg.DisableReflectionKey ‘key’ *note [1]: 3203. + + +winreg.EnableReflectionKey ‘key’ *note [1]: 3204. + + +winreg.EnumKey ‘key’, ‘index’ *note [1]: 3205. + + +winreg.EnumValue ‘key’, ‘index’ *note [1]: 3206. + + +winreg.ExpandEnvironmentStrings ‘str’ *note [1]: 3207. + + +winreg.LoadKey ‘key’, ‘sub_key’, ‘file_name’ *note [1]: 3208. + + +winreg.OpenKey ‘key’, ‘sub_key’, ‘access’ *note [1]: 3209. + + +winreg.OpenKey/result ‘key’ *note [1]: 31fe.*note [2]: 31ff.*note [3]: 3209. + + +winreg.PyHKEY.Detach ‘key’ *note [1]: 320a. + + +winreg.QueryInfoKey ‘key’ *note [1]: 320b. + + +winreg.QueryReflectionKey ‘key’ *note [1]: 320c. + + +winreg.QueryValue ‘key’, ‘sub_key’, ‘value_name’ *note [1]: 320d.*note [2]: 320e. + + +winreg.SaveKey ‘key’, ‘file_name’ *note [1]: 320f. + + +winreg.SetValue ‘key’, ‘sub_key’, ‘type’, ‘value’ *note [1]: 3210.*note [2]: 3211. + + +The following events are raised internally and do not correspond to any +public API of CPython: + +Audit event Arguments + +------------------------------------------------------------------------------- + +_winapi.CreateFile ‘file_name’, ‘desired_access’, ‘share_mode’, + ‘creation_disposition’, + ‘flags_and_attributes’ + + +_winapi.CreateJunction ‘src_path’, ‘dst_path’ + + +_winapi.CreateNamedPipe ‘name’, ‘open_mode’, ‘pipe_mode’ + + +_winapi.CreatePipe + +_winapi.CreateProcess ‘application_name’, ‘command_line’, + ‘current_directory’ + + +_winapi.OpenProcess ‘process_id’, ‘desired_access’ + + +_winapi.TerminateProcess ‘handle’, ‘exit_code’ + + +ctypes.PyObj_FromPtr ‘obj’ + + + +File: python.info, Node: bdb — Debugger framework, Next: faulthandler — Dump the Python traceback, Prev: Audit events table, Up: Debugging and Profiling + +5.27.2 ‘bdb’ — Debugger framework +--------------------------------- + +`Source code:' Lib/bdb.py(1) + +__________________________________________________________________ + +The *note bdb: f. module handles basic debugger functions, like setting +breakpoints or managing execution via the debugger. + +The following exception is defined: + + -- Exception: bdb.BdbQuit + + Exception raised by the *note Bdb: c98. class for quitting the + debugger. + +The *note bdb: f. module also defines two classes: + + -- Class: bdb.Breakpoint (self, file, line, temporary=0, cond=None, + funcname=None) + + This class implements temporary breakpoints, ignore counts, + disabling and (re-)enabling, and conditionals. + + Breakpoints are indexed by number through a list called + ‘bpbynumber’ and by ‘(file, line)’ pairs through ‘bplist’. The + former points to a single instance of class *note Breakpoint: 3215. + The latter points to a list of such instances since there may be + more than one breakpoint per line. + + When creating a breakpoint, its associated filename should be in + canonical form. If a `funcname' is defined, a breakpoint hit will + be counted when the first line of that function is executed. A + conditional breakpoint always counts a hit. + + *note Breakpoint: 3215. instances have the following methods: + + -- Method: deleteMe () + + Delete the breakpoint from the list associated to a file/line. + If it is the last breakpoint in that position, it also deletes + the entry for the file/line. + + -- Method: enable () + + Mark the breakpoint as enabled. + + -- Method: disable () + + Mark the breakpoint as disabled. + + -- Method: bpformat () + + Return a string with all the information about the breakpoint, + nicely formatted: + + * The breakpoint number. + + * If it is temporary or not. + + * Its file,line position. + + * The condition that causes a break. + + * If it must be ignored the next N times. + + * The breakpoint hit count. + + New in version 3.2. + + -- Method: bpprint (out=None) + + Print the output of *note bpformat(): 3219. to the file `out', + or if it is ‘None’, to standard output. + + -- Class: bdb.Bdb (skip=None) + + The *note Bdb: c98. class acts as a generic Python debugger base + class. + + This class takes care of the details of the trace facility; a + derived class should implement user interaction. The standard + debugger class (*note pdb.Pdb: 56c.) is an example. + + The `skip' argument, if given, must be an iterable of glob-style + module name patterns. The debugger will not step into frames that + originate in a module that matches one of these patterns. Whether + a frame is considered to originate in a certain module is + determined by the ‘__name__’ in the frame globals. + + New in version 3.1: The `skip' argument. + + The following methods of *note Bdb: c98. normally don’t need to be + overridden. + + -- Method: canonic (filename) + + Auxiliary method for getting a filename in a canonical form, + that is, as a case-normalized (on case-insensitive + filesystems) absolute path, stripped of surrounding angle + brackets. + + -- Method: reset () + + Set the ‘botframe’, ‘stopframe’, ‘returnframe’ and ‘quitting’ + attributes with values ready to start debugging. + + -- Method: trace_dispatch (frame, event, arg) + + This function is installed as the trace function of debugged + frames. Its return value is the new trace function (in most + cases, that is, itself). + + The default implementation decides how to dispatch a frame, + depending on the type of event (passed as a string) that is + about to be executed. `event' can be one of the following: + + * ‘"line"’: A new line of code is going to be executed. + + * ‘"call"’: A function is about to be called, or another + code block entered. + + * ‘"return"’: A function or other code block is about to + return. + + * ‘"exception"’: An exception has occurred. + + * ‘"c_call"’: A C function is about to be called. + + * ‘"c_return"’: A C function has returned. + + * ‘"c_exception"’: A C function has raised an exception. + + For the Python events, specialized functions (see below) are + called. For the C events, no action is taken. + + The `arg' parameter depends on the previous event. + + See the documentation for *note sys.settrace(): e3e. for more + information on the trace function. For more information on + code and frame objects, refer to *note The standard type + hierarchy: 10c0. + + -- Method: dispatch_line (frame) + + If the debugger should stop on the current line, invoke the + *note user_line(): 321f. method (which should be overridden in + subclasses). Raise a *note BdbQuit: 3214. exception if the + ‘Bdb.quitting’ flag is set (which can be set from *note + user_line(): 321f.). Return a reference to the *note + trace_dispatch(): 321d. method for further tracing in that + scope. + + -- Method: dispatch_call (frame, arg) + + If the debugger should stop on this function call, invoke the + *note user_call(): 3221. method (which should be overridden in + subclasses). Raise a *note BdbQuit: 3214. exception if the + ‘Bdb.quitting’ flag is set (which can be set from *note + user_call(): 3221.). Return a reference to the *note + trace_dispatch(): 321d. method for further tracing in that + scope. + + -- Method: dispatch_return (frame, arg) + + If the debugger should stop on this function return, invoke + the *note user_return(): 3223. method (which should be + overridden in subclasses). Raise a *note BdbQuit: 3214. + exception if the ‘Bdb.quitting’ flag is set (which can be set + from *note user_return(): 3223.). Return a reference to the + *note trace_dispatch(): 321d. method for further tracing in + that scope. + + -- Method: dispatch_exception (frame, arg) + + If the debugger should stop at this exception, invokes the + *note user_exception(): 3225. method (which should be + overridden in subclasses). Raise a *note BdbQuit: 3214. + exception if the ‘Bdb.quitting’ flag is set (which can be set + from *note user_exception(): 3225.). Return a reference to + the *note trace_dispatch(): 321d. method for further tracing + in that scope. + + Normally derived classes don’t override the following methods, but + they may if they want to redefine the definition of stopping and + breakpoints. + + -- Method: stop_here (frame) + + This method checks if the `frame' is somewhere below + ‘botframe’ in the call stack. ‘botframe’ is the frame in + which debugging started. + + -- Method: break_here (frame) + + This method checks if there is a breakpoint in the filename + and line belonging to `frame' or, at least, in the current + function. If the breakpoint is a temporary one, this method + deletes it. + + -- Method: break_anywhere (frame) + + This method checks if there is a breakpoint in the filename of + the current frame. + + Derived classes should override these methods to gain control over + debugger operation. + + -- Method: user_call (frame, argument_list) + + This method is called from *note dispatch_call(): 3220. when + there is the possibility that a break might be necessary + anywhere inside the called function. + + -- Method: user_line (frame) + + This method is called from *note dispatch_line(): 321e. when + either *note stop_here(): 3226. or *note break_here(): 3227. + yields ‘True’. + + -- Method: user_return (frame, return_value) + + This method is called from *note dispatch_return(): 3222. when + *note stop_here(): 3226. yields ‘True’. + + -- Method: user_exception (frame, exc_info) + + This method is called from *note dispatch_exception(): 3224. + when *note stop_here(): 3226. yields ‘True’. + + -- Method: do_clear (arg) + + Handle how a breakpoint must be removed when it is a temporary + one. + + This method must be implemented by derived classes. + + Derived classes and clients can call the following methods to + affect the stepping state. + + -- Method: set_step () + + Stop after one line of code. + + -- Method: set_next (frame) + + Stop on the next line in or below the given frame. + + -- Method: set_return (frame) + + Stop when returning from the given frame. + + -- Method: set_until (frame) + + Stop when the line with the line no greater than the current + one is reached or when returning from current frame. + + -- Method: set_trace ([frame]) + + Start debugging from `frame'. If `frame' is not specified, + debugging starts from caller’s frame. + + -- Method: set_continue () + + Stop only at breakpoints or when finished. If there are no + breakpoints, set the system trace function to ‘None’. + + -- Method: set_quit () + + Set the ‘quitting’ attribute to ‘True’. This raises *note + BdbQuit: 3214. in the next call to one of the ‘dispatch_*()’ + methods. + + Derived classes and clients can call the following methods to + manipulate breakpoints. These methods return a string containing + an error message if something went wrong, or ‘None’ if all is well. + + -- Method: set_break (filename, lineno, temporary=0, cond, + funcname) + + Set a new breakpoint. If the `lineno' line doesn’t exist for + the `filename' passed as argument, return an error message. + The `filename' should be in canonical form, as described in + the *note canonic(): 321b. method. + + -- Method: clear_break (filename, lineno) + + Delete the breakpoints in `filename' and `lineno'. If none + were set, an error message is returned. + + -- Method: clear_bpbynumber (arg) + + Delete the breakpoint which has the index `arg' in the + ‘Breakpoint.bpbynumber’. If `arg' is not numeric or out of + range, return an error message. + + -- Method: clear_all_file_breaks (filename) + + Delete all breakpoints in `filename'. If none were set, an + error message is returned. + + -- Method: clear_all_breaks () + + Delete all existing breakpoints. + + -- Method: get_bpbynumber (arg) + + Return a breakpoint specified by the given number. If `arg' + is a string, it will be converted to a number. If `arg' is a + non-numeric string, if the given breakpoint never existed or + has been deleted, a *note ValueError: 1fb. is raised. + + New in version 3.2. + + -- Method: get_break (filename, lineno) + + Check if there is a breakpoint for `lineno' of `filename'. + + -- Method: get_breaks (filename, lineno) + + Return all breakpoints for `lineno' in `filename', or an empty + list if none are set. + + -- Method: get_file_breaks (filename) + + Return all breakpoints in `filename', or an empty list if none + are set. + + -- Method: get_all_breaks () + + Return all breakpoints that are set. + + Derived classes and clients can call the following methods to get a + data structure representing a stack trace. + + -- Method: get_stack (f, t) + + Get a list of records for a frame and all higher (calling) and + lower frames, and the size of the higher part. + + -- Method: format_stack_entry (frame_lineno, lprefix=': ') + + Return a string with information about a stack entry, + identified by a ‘(frame, lineno)’ tuple: + + * The canonical form of the filename which contains the + frame. + + * The function name, or ‘"<lambda>"’. + + * The input arguments. + + * The return value. + + * The line of code (if it exists). + + The following two methods can be called by clients to use a + debugger to debug a *note statement: 1847, given as a string. + + -- Method: run (cmd, globals=None, locals=None) + + Debug a statement executed via the *note exec(): c44. + function. `globals' defaults to ‘__main__.__dict__’, `locals' + defaults to `globals'. + + -- Method: runeval (expr, globals=None, locals=None) + + Debug an expression executed via the *note eval(): b90. + function. `globals' and `locals' have the same meaning as in + *note run(): 323d. + + -- Method: runctx (cmd, globals, locals) + + For backwards compatibility. Calls the *note run(): 323d. + method. + + -- Method: runcall (func, *args, **kwds) + + Debug a single function call, and return its result. + +Finally, the module defines the following functions: + + -- Function: bdb.checkfuncname (b, frame) + + Check whether we should break here, depending on the way the + breakpoint `b' was set. + + If it was set via line number, it checks if ‘b.line’ is the same as + the one in the frame also passed as argument. If the breakpoint + was set via function name, we have to check we are in the right + frame (the right function) and if we are in its first executable + line. + + -- Function: bdb.effective (file, line, frame) + + Determine if there is an effective (active) breakpoint at this line + of code. Return a tuple of the breakpoint and a boolean that + indicates if it is ok to delete a temporary breakpoint. Return + ‘(None, None)’ if there is no matching breakpoint. + + -- Function: bdb.set_trace () + + Start debugging with a *note Bdb: c98. instance from caller’s + frame. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/bdb.py + + +File: python.info, Node: faulthandler — Dump the Python traceback, Next: pdb — The Python Debugger, Prev: bdb — Debugger framework, Up: Debugging and Profiling + +5.27.3 ‘faulthandler’ — Dump the Python traceback +------------------------------------------------- + +New in version 3.3. + +__________________________________________________________________ + +This module contains functions to dump Python tracebacks explicitly, on +a fault, after a timeout, or on a user signal. Call *note +faulthandler.enable(): 548. to install fault handlers for the ‘SIGSEGV’, +‘SIGFPE’, ‘SIGABRT’, ‘SIGBUS’, and ‘SIGILL’ signals. You can also +enable them at startup by setting the *note PYTHONFAULTHANDLER: 9c8. +environment variable or by using the *note -X: 155. ‘faulthandler’ +command line option. + +The fault handler is compatible with system fault handlers like Apport +or the Windows fault handler. The module uses an alternative stack for +signal handlers if the ‘sigaltstack()’ function is available. This +allows it to dump the traceback even on a stack overflow. + +The fault handler is called on catastrophic cases and therefore can only +use signal-safe functions (e.g. it cannot allocate memory on the heap). +Because of this limitation traceback dumping is minimal compared to +normal Python tracebacks: + + * Only ASCII is supported. The ‘backslashreplace’ error handler is + used on encoding. + + * Each string is limited to 500 characters. + + * Only the filename, the function name and the line number are + displayed. (no source code) + + * It is limited to 100 frames and 100 threads. + + * The order is reversed: the most recent call is shown first. + +By default, the Python traceback is written to *note sys.stderr: 30f. +To see tracebacks, applications must be run in the terminal. A log file +can alternatively be passed to *note faulthandler.enable(): 548. + +The module is implemented in C, so tracebacks can be dumped on a crash +or when Python is deadlocked. + +* Menu: + +* Dumping the traceback:: +* Fault handler state:: +* Dumping the tracebacks after a timeout:: +* Dumping the traceback on a user signal:: +* Issue with file descriptors:: +* Example: Example<15>. + + +File: python.info, Node: Dumping the traceback, Next: Fault handler state, Up: faulthandler — Dump the Python traceback + +5.27.3.1 Dumping the traceback +.............................. + + -- Function: faulthandler.dump_traceback (file=sys.stderr, + all_threads=True) + + Dump the tracebacks of all threads into `file'. If `all_threads' + is ‘False’, dump only the current thread. + + Changed in version 3.5: Added support for passing file descriptor + to this function. + + +File: python.info, Node: Fault handler state, Next: Dumping the tracebacks after a timeout, Prev: Dumping the traceback, Up: faulthandler — Dump the Python traceback + +5.27.3.2 Fault handler state +............................ + + -- Function: faulthandler.enable (file=sys.stderr, all_threads=True) + + Enable the fault handler: install handlers for the ‘SIGSEGV’, + ‘SIGFPE’, ‘SIGABRT’, ‘SIGBUS’ and ‘SIGILL’ signals to dump the + Python traceback. If `all_threads' is ‘True’, produce tracebacks + for every running thread. Otherwise, dump only the current thread. + + The `file' must be kept open until the fault handler is disabled: + see *note issue with file descriptors: 3247. + + Changed in version 3.5: Added support for passing file descriptor + to this function. + + Changed in version 3.6: On Windows, a handler for Windows exception + is also installed. + + -- Function: faulthandler.disable () + + Disable the fault handler: uninstall the signal handlers installed + by *note enable(): 548. + + -- Function: faulthandler.is_enabled () + + Check if the fault handler is enabled. + + +File: python.info, Node: Dumping the tracebacks after a timeout, Next: Dumping the traceback on a user signal, Prev: Fault handler state, Up: faulthandler — Dump the Python traceback + +5.27.3.3 Dumping the tracebacks after a timeout +............................................... + + -- Function: faulthandler.dump_traceback_later (timeout, repeat=False, + file=sys.stderr, exit=False) + + Dump the tracebacks of all threads, after a timeout of `timeout' + seconds, or every `timeout' seconds if `repeat' is ‘True’. If + `exit' is ‘True’, call ‘_exit()’ with status=1 after dumping the + tracebacks. (Note ‘_exit()’ exits the process immediately, which + means it doesn’t do any cleanup like flushing file buffers.) If + the function is called twice, the new call replaces previous + parameters and resets the timeout. The timer has a sub-second + resolution. + + The `file' must be kept open until the traceback is dumped or *note + cancel_dump_traceback_later(): 324b. is called: see *note issue + with file descriptors: 3247. + + This function is implemented using a watchdog thread. + + Changed in version 3.7: This function is now always available. + + Changed in version 3.5: Added support for passing file descriptor + to this function. + + -- Function: faulthandler.cancel_dump_traceback_later () + + Cancel the last call to *note dump_traceback_later(): 6d9. + + +File: python.info, Node: Dumping the traceback on a user signal, Next: Issue with file descriptors, Prev: Dumping the tracebacks after a timeout, Up: faulthandler — Dump the Python traceback + +5.27.3.4 Dumping the traceback on a user signal +............................................... + + -- Function: faulthandler.register (signum, file=sys.stderr, + all_threads=True, chain=False) + + Register a user signal: install a handler for the `signum' signal + to dump the traceback of all threads, or of the current thread if + `all_threads' is ‘False’, into `file'. Call the previous handler + if chain is ‘True’. + + The `file' must be kept open until the signal is unregistered by + *note unregister(): 324d.: see *note issue with file descriptors: + 3247. + + Not available on Windows. + + Changed in version 3.5: Added support for passing file descriptor + to this function. + + -- Function: faulthandler.unregister (signum) + + Unregister a user signal: uninstall the handler of the `signum' + signal installed by *note register(): 6d7. Return ‘True’ if the + signal was registered, ‘False’ otherwise. + + Not available on Windows. + + +File: python.info, Node: Issue with file descriptors, Next: Example<15>, Prev: Dumping the traceback on a user signal, Up: faulthandler — Dump the Python traceback + +5.27.3.5 Issue with file descriptors +.................................... + +*note enable(): 548, *note dump_traceback_later(): 6d9. and *note +register(): 6d7. keep the file descriptor of their `file' argument. If +the file is closed and its file descriptor is reused by a new file, or +if *note os.dup2(): 3be. is used to replace the file descriptor, the +traceback will be written into a different file. Call these functions +again each time that the file is replaced. + + +File: python.info, Node: Example<15>, Prev: Issue with file descriptors, Up: faulthandler — Dump the Python traceback + +5.27.3.6 Example +................ + +Example of a segmentation fault on Linux with and without enabling the +fault handler: + + $ python3 -c "import ctypes; ctypes.string_at(0)" + Segmentation fault + + $ python3 -q -X faulthandler + >>> import ctypes + >>> ctypes.string_at(0) + Fatal Python error: Segmentation fault + + Current thread 0x00007fb899f39700 (most recent call first): + File "/home/python/cpython/Lib/ctypes/__init__.py", line 486 in string_at + File "<stdin>", line 1 in <module> + Segmentation fault + + +File: python.info, Node: pdb — The Python Debugger, Next: The Python Profilers, Prev: faulthandler — Dump the Python traceback, Up: Debugging and Profiling + +5.27.4 ‘pdb’ — The Python Debugger +---------------------------------- + +`Source code:' Lib/pdb.py(1) + +__________________________________________________________________ + +The module *note pdb: c9. defines an interactive source code debugger +for Python programs. It supports setting (conditional) breakpoints and +single stepping at the source line level, inspection of stack frames, +source code listing, and evaluation of arbitrary Python code in the +context of any stack frame. It also supports post-mortem debugging and +can be called under program control. + +The debugger is extensible – it is actually defined as the class *note +Pdb: 56c. This is currently undocumented but easily understood by +reading the source. The extension interface uses the modules *note bdb: +f. and *note cmd: 1a. + +The debugger’s prompt is ‘(Pdb)’. Typical usage to run a program under +control of the debugger is: + + >>> import pdb + >>> import mymodule + >>> pdb.run('mymodule.test()') + > <string>(0)?() + (Pdb) continue + > <string>(1)?() + (Pdb) continue + NameError: 'spam' + > <string>(1)?() + (Pdb) + +Changed in version 3.3: Tab-completion via the *note readline: de. +module is available for commands and command arguments, e.g. the +current global and local names are offered as arguments of the ‘p’ +command. + +‘pdb.py’ can also be invoked as a script to debug other scripts. For +example: + + python3 -m pdb myscript.py + +When invoked as a script, pdb will automatically enter post-mortem +debugging if the program being debugged exits abnormally. After +post-mortem debugging (or after normal exit of the program), pdb will +restart the program. Automatic restarting preserves pdb’s state (such +as breakpoints) and in most cases is more useful than quitting the +debugger upon program’s exit. + +New in version 3.2: ‘pdb.py’ now accepts a ‘-c’ option that executes +commands as if given in a ‘.pdbrc’ file, see *note Debugger Commands: +3253. + +New in version 3.7: ‘pdb.py’ now accepts a ‘-m’ option that execute +modules similar to the way ‘python3 -m’ does. As with a script, the +debugger will pause execution just before the first line of the module. + +The typical usage to break into the debugger from a running program is +to insert + + import pdb; pdb.set_trace() + +at the location you want to break into the debugger. You can then step +through the code following this statement, and continue running without +the debugger using the *note continue: 3254. command. + +New in version 3.7: The built-in *note breakpoint(): 2f9, when called +with defaults, can be used instead of ‘import pdb; pdb.set_trace()’. + +The typical usage to inspect a crashed program is: + + >>> import pdb + >>> import mymodule + >>> mymodule.test() + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + File "./mymodule.py", line 4, in test + test2() + File "./mymodule.py", line 3, in test2 + print(spam) + NameError: spam + >>> pdb.pm() + > ./mymodule.py(3)test2() + -> print(spam) + (Pdb) + +The module defines the following functions; each enters the debugger in +a slightly different way: + + -- Function: pdb.run (statement, globals=None, locals=None) + + Execute the `statement' (given as a string or a code object) under + debugger control. The debugger prompt appears before any code is + executed; you can set breakpoints and type *note continue: 3254, or + you can step through the statement using *note step: 3255. or *note + next: 3256. (all these commands are explained below). The optional + `globals' and `locals' arguments specify the environment in which + the code is executed; by default the dictionary of the module *note + __main__: 1. is used. (See the explanation of the built-in *note + exec(): c44. or *note eval(): b90. functions.) + + -- Function: pdb.runeval (expression, globals=None, locals=None) + + Evaluate the `expression' (given as a string or a code object) + under debugger control. When *note runeval(): 3257. returns, it + returns the value of the expression. Otherwise this function is + similar to *note run(): 3022. + + -- Function: pdb.runcall (function, *args, **kwds) + + Call the `function' (a function or method object, not a string) + with the given arguments. When *note runcall(): 3258. returns, it + returns whatever the function call returned. The debugger prompt + appears as soon as the function is entered. + + -- Function: pdb.set_trace (*, header=None) + + Enter the debugger at the calling stack frame. This is useful to + hard-code a breakpoint at a given point in a program, even if the + code is not otherwise being debugged (e.g. when an assertion + fails). If given, `header' is printed to the console just before + debugging begins. + + Changed in version 3.7: The keyword-only argument `header'. + + -- Function: pdb.post_mortem (traceback=None) + + Enter post-mortem debugging of the given `traceback' object. If no + `traceback' is given, it uses the one of the exception that is + currently being handled (an exception must be being handled if the + default is to be used). + + -- Function: pdb.pm () + + Enter post-mortem debugging of the traceback found in *note + sys.last_traceback: 325a. + +The ‘run*’ functions and *note set_trace(): 3c2. are aliases for +instantiating the *note Pdb: 56c. class and calling the method of the +same name. If you want to access further features, you have to do this +yourself: + + -- Class: pdb.Pdb (completekey='tab', stdin=None, stdout=None, + skip=None, nosigint=False, readrc=True) + + *note Pdb: 56c. is the debugger class. + + The `completekey', `stdin' and `stdout' arguments are passed to the + underlying *note cmd.Cmd: 2e48. class; see the description there. + + The `skip' argument, if given, must be an iterable of glob-style + module name patterns. The debugger will not step into frames that + originate in a module that matches one of these patterns. (2) + + By default, Pdb sets a handler for the SIGINT signal (which is sent + when the user presses ‘Ctrl-C’ on the console) when you give a + ‘continue’ command. This allows you to break into the debugger + again by pressing ‘Ctrl-C’. If you want Pdb not to touch the + SIGINT handler, set `nosigint' to true. + + The `readrc' argument defaults to true and controls whether Pdb + will load .pdbrc files from the filesystem. + + Example call to enable tracing with `skip': + + import pdb; pdb.Pdb(skip=['django.*']).set_trace() + + Raises an *note auditing event: fd1. ‘pdb.Pdb’ with no arguments. + + New in version 3.1: The `skip' argument. + + New in version 3.2: The `nosigint' argument. Previously, a SIGINT + handler was never set by Pdb. + + Changed in version 3.6: The `readrc' argument. + + -- Method: run (statement, globals=None, locals=None) + -- Method: runeval (expression, globals=None, locals=None) + -- Method: runcall (function, *args, **kwds) + -- Method: set_trace () + + See the documentation for the functions explained above. + +* Menu: + +* Debugger Commands:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pdb.py + + (2) (1) Whether a frame is considered to originate in a certain +module is determined by the ‘__name__’ in the frame globals. + + +File: python.info, Node: Debugger Commands, Up: pdb — The Python Debugger + +5.27.4.1 Debugger Commands +.......................... + +The commands recognized by the debugger are listed below. Most commands +can be abbreviated to one or two letters as indicated; e.g. ‘h(elp)’ +means that either ‘h’ or ‘help’ can be used to enter the help command +(but not ‘he’ or ‘hel’, nor ‘H’ or ‘Help’ or ‘HELP’). Arguments to +commands must be separated by whitespace (spaces or tabs). Optional +arguments are enclosed in square brackets (‘[]’) in the command syntax; +the square brackets must not be typed. Alternatives in the command +syntax are separated by a vertical bar (‘|’). + +Entering a blank line repeats the last command entered. Exception: if +the last command was a *note list: 3260. command, the next 11 lines are +listed. + +Commands that the debugger doesn’t recognize are assumed to be Python +statements and are executed in the context of the program being +debugged. Python statements can also be prefixed with an exclamation +point (‘!’). This is a powerful way to inspect the program being +debugged; it is even possible to change a variable or call a function. +When an exception occurs in such a statement, the exception name is +printed but the debugger’s state is not changed. + +The debugger supports *note aliases: 3261. Aliases can have parameters +which allows one a certain level of adaptability to the context under +examination. + +Multiple commands may be entered on a single line, separated by ‘;;’. +(A single ‘;’ is not used as it is the separator for multiple commands +in a line that is passed to the Python parser.) No intelligence is +applied to separating the commands; the input is split at the first ‘;;’ +pair, even if it is in the middle of a quoted string. + +If a file ‘.pdbrc’ exists in the user’s home directory or in the current +directory, it is read in and executed as if it had been typed at the +debugger prompt. This is particularly useful for aliases. If both +files exist, the one in the home directory is read first and aliases +defined there can be overridden by the local file. + +Changed in version 3.2: ‘.pdbrc’ can now contain commands that continue +debugging, such as *note continue: 3254. or *note next: 3256. +Previously, these commands had no effect. + + -- Pdbcommand: h(elp) [command] + + Without argument, print the list of available commands. With a + `command' as argument, print help about that command. ‘help pdb’ + displays the full documentation (the docstring of the *note pdb: + c9. module). Since the `command' argument must be an identifier, + ‘help exec’ must be entered to get help on the ‘!’ command. + + -- Pdbcommand: w(here) + + Print a stack trace, with the most recent frame at the bottom. An + arrow indicates the current frame, which determines the context of + most commands. + + -- Pdbcommand: d(own) [count] + + Move the current frame `count' (default one) levels down in the + stack trace (to a newer frame). + + -- Pdbcommand: u(p) [count] + + Move the current frame `count' (default one) levels up in the stack + trace (to an older frame). + + -- Pdbcommand: b(reak) [([filename:]lineno | function) [, condition]] + + With a `lineno' argument, set a break there in the current file. + With a `function' argument, set a break at the first executable + statement within that function. The line number may be prefixed + with a filename and a colon, to specify a breakpoint in another + file (probably one that hasn’t been loaded yet). The file is + searched on *note sys.path: 488. Note that each breakpoint is + assigned a number to which all the other breakpoint commands refer. + + If a second argument is present, it is an expression which must + evaluate to true before the breakpoint is honored. + + Without argument, list all breaks, including for each breakpoint, + the number of times that breakpoint has been hit, the current + ignore count, and the associated condition if any. + + -- Pdbcommand: tbreak [([filename:]lineno | function) [, condition]] + + Temporary breakpoint, which is removed automatically when it is + first hit. The arguments are the same as for *note break: 3266. + + -- Pdbcommand: cl(ear) [filename:lineno | bpnumber [bpnumber ...]] + + With a `filename:lineno' argument, clear all the breakpoints at + this line. With a space separated list of breakpoint numbers, + clear those breakpoints. Without argument, clear all breaks (but + first ask confirmation). + + -- Pdbcommand: disable [bpnumber [bpnumber ...]] + + Disable the breakpoints given as a space separated list of + breakpoint numbers. Disabling a breakpoint means it cannot cause + the program to stop execution, but unlike clearing a breakpoint, it + remains in the list of breakpoints and can be (re-)enabled. + + -- Pdbcommand: enable [bpnumber [bpnumber ...]] + + Enable the breakpoints specified. + + -- Pdbcommand: ignore bpnumber [count] + + Set the ignore count for the given breakpoint number. If count is + omitted, the ignore count is set to 0. A breakpoint becomes active + when the ignore count is zero. When non-zero, the count is + decremented each time the breakpoint is reached and the breakpoint + is not disabled and any associated condition evaluates to true. + + -- Pdbcommand: condition bpnumber [condition] + + Set a new `condition' for the breakpoint, an expression which must + evaluate to true before the breakpoint is honored. If `condition' + is absent, any existing condition is removed; i.e., the breakpoint + is made unconditional. + + -- Pdbcommand: commands [bpnumber] + + Specify a list of commands for breakpoint number `bpnumber'. The + commands themselves appear on the following lines. Type a line + containing just ‘end’ to terminate the commands. An example: + + (Pdb) commands 1 + (com) p some_variable + (com) end + (Pdb) + + To remove all commands from a breakpoint, type ‘commands’ and + follow it immediately with ‘end’; that is, give no commands. + + With no `bpnumber' argument, ‘commands’ refers to the last + breakpoint set. + + You can use breakpoint commands to start your program up again. + Simply use the *note continue: 3254. command, or *note step: 3255, + or any other command that resumes execution. + + Specifying any command resuming execution (currently *note + continue: 3254, *note step: 3255, *note next: 3256, *note return: + 326e, *note jump: 326f, *note quit: 3270. and their abbreviations) + terminates the command list (as if that command was immediately + followed by end). This is because any time you resume execution + (even with a simple next or step), you may encounter another + breakpoint—which could have its own command list, leading to + ambiguities about which list to execute. + + If you use the ‘silent’ command in the command list, the usual + message about stopping at a breakpoint is not printed. This may be + desirable for breakpoints that are to print a specific message and + then continue. If none of the other commands print anything, you + see no sign that the breakpoint was reached. + + -- Pdbcommand: s(tep) + + Execute the current line, stop at the first possible occasion + (either in a function that is called or on the next line in the + current function). + + -- Pdbcommand: n(ext) + + Continue execution until the next line in the current function is + reached or it returns. (The difference between *note next: 3256. + and *note step: 3255. is that *note step: 3255. stops inside a + called function, while *note next: 3256. executes called functions + at (nearly) full speed, only stopping at the next line in the + current function.) + + -- Pdbcommand: unt(il) [lineno] + + Without argument, continue execution until the line with a number + greater than the current one is reached. + + With a line number, continue execution until a line with a number + greater or equal to that is reached. In both cases, also stop when + the current frame returns. + + Changed in version 3.2: Allow giving an explicit line number. + + -- Pdbcommand: r(eturn) + + Continue execution until the current function returns. + + -- Pdbcommand: c(ont(inue)) + + Continue execution, only stop when a breakpoint is encountered. + + -- Pdbcommand: j(ump) lineno + + Set the next line that will be executed. Only available in the + bottom-most frame. This lets you jump back and execute code again, + or jump forward to skip code that you don’t want to run. + + It should be noted that not all jumps are allowed – for instance it + is not possible to jump into the middle of a *note for: c30. loop + or out of a *note finally: 182. clause. + + -- Pdbcommand: l(ist) [first[, last]] + + List source code for the current file. Without arguments, list 11 + lines around the current line or continue the previous listing. + With ‘.’ as argument, list 11 lines around the current line. With + one argument, list 11 lines around at that line. With two + arguments, list the given range; if the second argument is less + than the first, it is interpreted as a count. + + The current line in the current frame is indicated by ‘->’. If an + exception is being debugged, the line where the exception was + originally raised or propagated is indicated by ‘>>’, if it differs + from the current line. + + New in version 3.2: The ‘>>’ marker. + + -- Pdbcommand: ll | longlist + + List all source code for the current function or frame. + Interesting lines are marked as for *note list: 3260. + + New in version 3.2. + + -- Pdbcommand: a(rgs) + + Print the argument list of the current function. + + -- Pdbcommand: p expression + + Evaluate the `expression' in the current context and print its + value. + + Note: ‘print()’ can also be used, but is not a debugger + command — this executes the Python *note print(): 886. + function. + + -- Pdbcommand: pp expression + + Like the *note p: 887. command, except the value of the expression + is pretty-printed using the *note pprint: d2. module. + + -- Pdbcommand: whatis expression + + Print the type of the `expression'. + + -- Pdbcommand: source expression + + Try to get source code for the given object and display it. + + New in version 3.2. + + -- Pdbcommand: display [expression] + + Display the value of the expression if it changed, each time + execution stops in the current frame. + + Without expression, list all display expressions for the current + frame. + + New in version 3.2. + + -- Pdbcommand: undisplay [expression] + + Do not display the expression any more in the current frame. + Without expression, clear all display expressions for the current + frame. + + New in version 3.2. + + -- Pdbcommand: interact + + Start an interactive interpreter (using the *note code: 1b. module) + whose global namespace contains all the (global and local) names + found in the current scope. + + New in version 3.2. + + -- Pdbcommand: alias [name [command]] + + Create an alias called `name' that executes `command'. The command + must `not' be enclosed in quotes. Replaceable parameters can be + indicated by ‘%1’, ‘%2’, and so on, while ‘%*’ is replaced by all + the parameters. If no command is given, the current alias for + `name' is shown. If no arguments are given, all aliases are + listed. + + Aliases may be nested and can contain anything that can be legally + typed at the pdb prompt. Note that internal pdb commands `can' be + overridden by aliases. Such a command is then hidden until the + alias is removed. Aliasing is recursively applied to the first + word of the command line; all other words in the line are left + alone. + + As an example, here are two useful aliases (especially when placed + in the ‘.pdbrc’ file): + + # Print instance variables (usage "pi classInst") + alias pi for k in %1.__dict__.keys(): print("%1.",k,"=",%1.__dict__[k]) + # Print instance variables in self + alias ps pi self + + -- Pdbcommand: unalias name + + Delete the specified alias. + + -- Pdbcommand: ! statement + + Execute the (one-line) `statement' in the context of the current + stack frame. The exclamation point can be omitted unless the first + word of the statement resembles a debugger command. To set a + global variable, you can prefix the assignment command with a *note + global: ed8. statement on the same line, e.g.: + + (Pdb) global list_options; list_options = ['-l'] + (Pdb) + + -- Pdbcommand: run [args ...] + -- Pdbcommand: restart [args ...] + + Restart the debugged Python program. If an argument is supplied, + it is split with *note shlex: e8. and the result is used as the new + *note sys.argv: bfb. History, breakpoints, actions and debugger + options are preserved. *note restart: 327e. is an alias for *note + run: 327d. + + -- Pdbcommand: q(uit) + + Quit from the debugger. The program being executed is aborted. + + -- Pdbcommand: debug code + + Enter a recursive debugger that steps through the code argument + (which is an arbitrary expression or statement to be executed in + the current environment). + + -- Pdbcommand: retval + + Print the return value for the last return of a function. + + +File: python.info, Node: The Python Profilers, Next: timeit — Measure execution time of small code snippets, Prev: pdb — The Python Debugger, Up: Debugging and Profiling + +5.27.5 The Python Profilers +--------------------------- + +`Source code:' Lib/profile.py(1) and Lib/pstats.py(2) + +__________________________________________________________________ + +* Menu: + +* Introduction to the profilers:: +* Instant User’s Manual:: +* profile and cProfile Module Reference:: +* The Stats Class:: +* What Is Deterministic Profiling?:: +* Limitations:: +* Calibration:: +* Using a custom timer:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/profile.py + + (2) https://github.com/python/cpython/tree/3.8/Lib/pstats.py + + +File: python.info, Node: Introduction to the profilers, Next: Instant User’s Manual, Up: The Python Profilers + +5.27.5.1 Introduction to the profilers +...................................... + +*note cProfile: 28. and *note profile: d3. provide `deterministic +profiling' of Python programs. A `profile' is a set of statistics that +describes how often and for how long various parts of the program +executed. These statistics can be formatted into reports via the *note +pstats: d4. module. + +The Python standard library provides two different implementations of +the same profiling interface: + + 1. *note cProfile: 28. is recommended for most users; it’s a C + extension with reasonable overhead that makes it suitable for + profiling long-running programs. Based on ‘lsprof’, contributed by + Brett Rosen and Ted Czotter. + + 2. *note profile: d3, a pure Python module whose interface is imitated + by *note cProfile: 28, but which adds significant overhead to + profiled programs. If you’re trying to extend the profiler in some + way, the task might be easier with this module. Originally + designed and written by Jim Roskind. + + Note: The profiler modules are designed to provide an execution + profile for a given program, not for benchmarking purposes (for + that, there is *note timeit: 10b. for reasonably accurate results). + This particularly applies to benchmarking Python code against C + code: the profilers introduce overhead for Python code, but not for + C-level functions, and so the C code would seem faster than any + Python one. + + +File: python.info, Node: Instant User’s Manual, Next: profile and cProfile Module Reference, Prev: Introduction to the profilers, Up: The Python Profilers + +5.27.5.2 Instant User’s Manual +.............................. + +This section is provided for users that “don’t want to read the manual.” +It provides a very brief overview, and allows a user to rapidly perform +profiling on an existing application. + +To profile a function that takes a single argument, you can do: + + import cProfile + import re + cProfile.run('re.compile("foo|bar")') + +(Use *note profile: d3. instead of *note cProfile: 28. if the latter is +not available on your system.) + +The above action would run *note re.compile(): 44a. and print profile +results like the following: + + 197 function calls (192 primitive calls) in 0.002 seconds + + Ordered by: standard name + + ncalls tottime percall cumtime percall filename:lineno(function) + 1 0.000 0.000 0.001 0.001 <string>:1(<module>) + 1 0.000 0.000 0.001 0.001 re.py:212(compile) + 1 0.000 0.000 0.001 0.001 re.py:268(_compile) + 1 0.000 0.000 0.000 0.000 sre_compile.py:172(_compile_charset) + 1 0.000 0.000 0.000 0.000 sre_compile.py:201(_optimize_charset) + 4 0.000 0.000 0.000 0.000 sre_compile.py:25(_identityfunction) + 3/1 0.000 0.000 0.000 0.000 sre_compile.py:33(_compile) + +The first line indicates that 197 calls were monitored. Of those calls, +192 were `primitive', meaning that the call was not induced via +recursion. The next line: ‘Ordered by: standard name’, indicates that +the text string in the far right column was used to sort the output. +The column headings include: + +ncalls + + for the number of calls. + +tottime + + for the total time spent in the given function (and excluding time + made in calls to sub-functions) + +percall + + is the quotient of ‘tottime’ divided by ‘ncalls’ + +cumtime + + is the cumulative time spent in this and all subfunctions (from + invocation till exit). This figure is accurate `even' for + recursive functions. + +percall + + is the quotient of ‘cumtime’ divided by primitive calls + +filename:lineno(function) + + provides the respective data of each function + +When there are two numbers in the first column (for example ‘3/1’), it +means that the function recursed. The second value is the number of +primitive calls and the former is the total number of calls. Note that +when the function does not recurse, these two values are the same, and +only the single figure is printed. + +Instead of printing the output at the end of the profile run, you can +save the results to a file by specifying a filename to the ‘run()’ +function: + + import cProfile + import re + cProfile.run('re.compile("foo|bar")', 'restats') + +The *note pstats.Stats: 3288. class reads profile results from a file +and formats them in various ways. + +The files *note cProfile: 28. and *note profile: d3. can also be invoked +as a script to profile another script. For example: + + python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py) + +‘-o’ writes the profile results to a file instead of to stdout + +‘-s’ specifies one of the *note sort_stats(): 3289. sort values to sort +the output by. This only applies when ‘-o’ is not supplied. + +‘-m’ specifies that a module is being profiled instead of a script. + + New in version 3.7: Added the ‘-m’ option to *note cProfile: 28. + + New in version 3.8: Added the ‘-m’ option to *note profile: d3. + +The *note pstats: d4. module’s *note Stats: 3288. class has a variety of +methods for manipulating and printing the data saved into a profile +results file: + + import pstats + from pstats import SortKey + p = pstats.Stats('restats') + p.strip_dirs().sort_stats(-1).print_stats() + +The *note strip_dirs(): 328a. method removed the extraneous path from +all the module names. The *note sort_stats(): 3289. method sorted all +the entries according to the standard module/line/name string that is +printed. The *note print_stats(): 328b. method printed out all the +statistics. You might try the following sort calls: + + p.sort_stats(SortKey.NAME) + p.print_stats() + +The first call will actually sort the list by function name, and the +second call will print out the statistics. The following are some +interesting calls to experiment with: + + p.sort_stats(SortKey.CUMULATIVE).print_stats(10) + +This sorts the profile by cumulative time in a function, and then only +prints the ten most significant lines. If you want to understand what +algorithms are taking time, the above line is what you would use. + +If you were looking to see what functions were looping a lot, and taking +a lot of time, you would do: + + p.sort_stats(SortKey.TIME).print_stats(10) + +to sort according to time spent within each function, and then print the +statistics for the top ten functions. + +You might also try: + + p.sort_stats(SortKey.FILENAME).print_stats('__init__') + +This will sort all the statistics by file name, and then print out +statistics for only the class init methods (since they are spelled with +‘__init__’ in them). As one final example, you could try: + + p.sort_stats(SortKey.TIME, SortKey.CUMULATIVE).print_stats(.5, 'init') + +This line sorts statistics with a primary key of time, and a secondary +key of cumulative time, and then prints out some of the statistics. To +be specific, the list is first culled down to 50% (re: ‘.5’) of its +original size, then only lines containing ‘init’ are maintained, and +that sub-sub-list is printed. + +If you wondered what functions called the above functions, you could now +(‘p’ is still sorted according to the last criteria) do: + + p.print_callers(.5, 'init') + +and you would get a list of callers for each of the listed functions. + +If you want more functionality, you’re going to have to read the manual, +or guess what the following functions do: + + p.print_callees() + p.add('restats') + +Invoked as a script, the *note pstats: d4. module is a statistics +browser for reading and examining profile dumps. It has a simple +line-oriented interface (implemented using *note cmd: 1a.) and +interactive help. + + +File: python.info, Node: profile and cProfile Module Reference, Next: The Stats Class, Prev: Instant User’s Manual, Up: The Python Profilers + +5.27.5.3 ‘profile’ and ‘cProfile’ Module Reference +.................................................. + +Both the *note profile: d3. and *note cProfile: 28. modules provide the +following functions: + + -- Function: profile.run (command, filename=None, sort=-1) + + This function takes a single argument that can be passed to the + *note exec(): c44. function, and an optional file name. In all + cases this routine executes: + + exec(command, __main__.__dict__, __main__.__dict__) + + and gathers profiling statistics from the execution. If no file + name is present, then this function automatically creates a *note + Stats: 3288. instance and prints a simple profiling report. If the + sort value is specified, it is passed to this *note Stats: 3288. + instance to control how the results are sorted. + + -- Function: profile.runctx (command, globals, locals, filename=None, + sort=-1) + + This function is similar to *note run(): 328d, with added arguments + to supply the globals and locals dictionaries for the `command' + string. This routine executes: + + exec(command, globals, locals) + + and gathers profiling statistics as in the *note run(): 328d. + function above. + + -- Class: profile.Profile (timer=None, timeunit=0.0, subcalls=True, + builtins=True) + + This class is normally only used if more precise control over + profiling is needed than what the ‘cProfile.run()’ function + provides. + + A custom timer can be supplied for measuring how long code takes to + run via the `timer' argument. This must be a function that returns + a single number representing the current time. If the number is an + integer, the `timeunit' specifies a multiplier that specifies the + duration of each unit of time. For example, if the timer returns + times measured in thousands of seconds, the time unit would be + ‘.001’. + + Directly using the *note Profile: 1bb. class allows formatting + profile results without writing the profile data to a file: + + import cProfile, pstats, io + from pstats import SortKey + pr = cProfile.Profile() + pr.enable() + # ... do something ... + pr.disable() + s = io.StringIO() + sortby = SortKey.CUMULATIVE + ps = pstats.Stats(pr, stream=s).sort_stats(sortby) + ps.print_stats() + print(s.getvalue()) + + The *note Profile: 1bb. class can also be used as a context manager + (supported only in *note cProfile: 28. module. see *note Context + Manager Types: 112a.): + + import cProfile + + with cProfile.Profile() as pr: + # ... do something ... + + pr.print_stats() + + Changed in version 3.8: Added context manager support. + + -- Method: enable () + + Start collecting profiling data. Only in *note cProfile: 28. + + -- Method: disable () + + Stop collecting profiling data. Only in *note cProfile: 28. + + -- Method: create_stats () + + Stop collecting profiling data and record the results + internally as the current profile. + + -- Method: print_stats (sort=-1) + + Create a *note Stats: 3288. object based on the current + profile and print the results to stdout. + + -- Method: dump_stats (filename) + + Write the results of the current profile to `filename'. + + -- Method: run (cmd) + + Profile the cmd via *note exec(): c44. + + -- Method: runctx (cmd, globals, locals) + + Profile the cmd via *note exec(): c44. with the specified + global and local environment. + + -- Method: runcall (func, *args, **kwargs) + + Profile ‘func(*args, **kwargs)’ + +Note that profiling will only work if the called command/function +actually returns. If the interpreter is terminated (e.g. via a *note +sys.exit(): ce2. call during the called command/function execution) no +profiling results will be printed. + + +File: python.info, Node: The Stats Class, Next: What Is Deterministic Profiling?, Prev: profile and cProfile Module Reference, Up: The Python Profilers + +5.27.5.4 The ‘Stats’ Class +.......................... + +Analysis of the profiler data is done using the *note Stats: 3288. +class. + + -- Class: pstats.Stats (*filenames or profile, stream=sys.stdout) + + This class constructor creates an instance of a “statistics object” + from a `filename' (or list of filenames) or from a ‘Profile’ + instance. Output will be printed to the stream specified by + `stream'. + + The file selected by the above constructor must have been created + by the corresponding version of *note profile: d3. or *note + cProfile: 28. To be specific, there is `no' file compatibility + guaranteed with future versions of this profiler, and there is no + compatibility with files produced by other profilers, or the same + profiler run on a different operating system. If several files are + provided, all the statistics for identical functions will be + coalesced, so that an overall view of several processes can be + considered in a single report. If additional files need to be + combined with data in an existing *note Stats: 3288. object, the + *note add(): 3298. method can be used. + + Instead of reading the profile data from a file, a + ‘cProfile.Profile’ or *note profile.Profile: 1bb. object can be + used as the profile data source. + + *note Stats: 3288. objects have the following methods: + + -- Method: strip_dirs () + + This method for the *note Stats: 3288. class removes all + leading path information from file names. It is very useful + in reducing the size of the printout to fit within (close to) + 80 columns. This method modifies the object, and the stripped + information is lost. After performing a strip operation, the + object is considered to have its entries in a “random” order, + as it was just after object initialization and loading. If + *note strip_dirs(): 328a. causes two function names to be + indistinguishable (they are on the same line of the same + filename, and have the same function name), then the + statistics for these two entries are accumulated into a single + entry. + + -- Method: add (*filenames) + + This method of the *note Stats: 3288. class accumulates + additional profiling information into the current profiling + object. Its arguments should refer to filenames created by + the corresponding version of *note profile.run(): 328d. or + ‘cProfile.run()’. Statistics for identically named (re: file, + line, name) functions are automatically accumulated into + single function statistics. + + -- Method: dump_stats (filename) + + Save the data loaded into the *note Stats: 3288. object to a + file named `filename'. The file is created if it does not + exist, and is overwritten if it already exists. This is + equivalent to the method of the same name on the *note + profile.Profile: 1bb. and ‘cProfile.Profile’ classes. + + -- Method: sort_stats (*keys) + + This method modifies the *note Stats: 3288. object by sorting + it according to the supplied criteria. The argument can be + either a string or a SortKey enum identifying the basis of a + sort (example: ‘'time'’, ‘'name'’, ‘SortKey.TIME’ or + ‘SortKey.NAME’). The SortKey enums argument have advantage + over the string argument in that it is more robust and less + error prone. + + When more than one key is provided, then additional keys are + used as secondary criteria when there is equality in all keys + selected before them. For example, ‘sort_stats(SortKey.NAME, + SortKey.FILE)’ will sort all the entries according to their + function name, and resolve all ties (identical function names) + by sorting by file name. + + For the string argument, abbreviations can be used for any key + names, as long as the abbreviation is unambiguous. + + The following are the valid string and SortKey: + + Valid String Arg Valid enum Arg Meaning + + ---------------------------------------------------------------------------- + + ‘'calls'’ SortKey.CALLS call count + + + ‘'cumulative'’ SortKey.CUMULATIVE cumulative time + + + ‘'cumtime'’ N/A cumulative time + + + ‘'file'’ N/A file name + + + ‘'filename'’ SortKey.FILENAME file name + + + ‘'module'’ N/A file name + + + ‘'ncalls'’ N/A call count + + + ‘'pcalls'’ SortKey.PCALLS primitive call count + + + ‘'line'’ SortKey.LINE line number + + + ‘'name'’ SortKey.NAME function name + + + ‘'nfl'’ SortKey.NFL name/file/line + + + ‘'stdname'’ SortKey.STDNAME standard name + + + ‘'time'’ SortKey.TIME internal time + + + ‘'tottime'’ N/A internal time + + + Note that all sorts on statistics are in descending order + (placing most time consuming items first), where as name, + file, and line number searches are in ascending order + (alphabetical). The subtle distinction between ‘SortKey.NFL’ + and ‘SortKey.STDNAME’ is that the standard name is a sort of + the name as printed, which means that the embedded line + numbers get compared in an odd way. For example, lines 3, 20, + and 40 would (if the file names were the same) appear in the + string order 20, 3 and 40. In contrast, ‘SortKey.NFL’ does a + numeric compare of the line numbers. In fact, + ‘sort_stats(SortKey.NFL)’ is the same as + ‘sort_stats(SortKey.NAME, SortKey.FILENAME, SortKey.LINE)’. + + For backward-compatibility reasons, the numeric arguments + ‘-1’, ‘0’, ‘1’, and ‘2’ are permitted. They are interpreted + as ‘'stdname'’, ‘'calls'’, ‘'time'’, and ‘'cumulative'’ + respectively. If this old style format (numeric) is used, + only one sort key (the numeric key) will be used, and + additional arguments will be silently ignored. + + New in version 3.7: Added the SortKey enum. + + -- Method: reverse_order () + + This method for the *note Stats: 3288. class reverses the + ordering of the basic list within the object. Note that by + default ascending vs descending order is properly selected + based on the sort key of choice. + + -- Method: print_stats (*restrictions) + + This method for the *note Stats: 3288. class prints out a + report as described in the *note profile.run(): 328d. + definition. + + The order of the printing is based on the last *note + sort_stats(): 3289. operation done on the object (subject to + caveats in *note add(): 3298. and *note strip_dirs(): 328a.). + + The arguments provided (if any) can be used to limit the list + down to the significant entries. Initially, the list is taken + to be the complete set of profiled functions. Each + restriction is either an integer (to select a count of lines), + or a decimal fraction between 0.0 and 1.0 inclusive (to select + a percentage of lines), or a string that will interpreted as a + regular expression (to pattern match the standard name that is + printed). If several restrictions are provided, then they are + applied sequentially. For example: + + print_stats(.1, 'foo:') + + would first limit the printing to first 10% of list, and then + only print functions that were part of filename ‘.*foo:’. In + contrast, the command: + + print_stats('foo:', .1) + + would limit the list to all functions having file names + ‘.*foo:’, and then proceed to only print the first 10% of + them. + + -- Method: print_callers (*restrictions) + + This method for the *note Stats: 3288. class prints a list of + all functions that called each function in the profiled + database. The ordering is identical to that provided by *note + print_stats(): 328b, and the definition of the restricting + argument is also identical. Each caller is reported on its + own line. The format differs slightly depending on the + profiler that produced the stats: + + * With *note profile: d3, a number is shown in parentheses + after each caller to show how many times this specific + call was made. For convenience, a second + non-parenthesized number repeats the cumulative time + spent in the function at the right. + + * With *note cProfile: 28, each caller is preceded by three + numbers: the number of times this specific call was made, + and the total and cumulative times spent in the current + function while it was invoked by this specific caller. + + -- Method: print_callees (*restrictions) + + This method for the *note Stats: 3288. class prints a list of + all function that were called by the indicated function. + Aside from this reversal of direction of calls (re: called vs + was called by), the arguments and ordering are identical to + the *note print_callers(): 329b. method. + + +File: python.info, Node: What Is Deterministic Profiling?, Next: Limitations, Prev: The Stats Class, Up: The Python Profilers + +5.27.5.5 What Is Deterministic Profiling? +......................................... + +`Deterministic profiling' is meant to reflect the fact that all +`function call', `function return', and `exception' events are +monitored, and precise timings are made for the intervals between these +events (during which time the user’s code is executing). In contrast, +`statistical profiling' (which is not done by this module) randomly +samples the effective instruction pointer, and deduces where time is +being spent. The latter technique traditionally involves less overhead +(as the code does not need to be instrumented), but provides only +relative indications of where time is being spent. + +In Python, since there is an interpreter active during execution, the +presence of instrumented code is not required in order to do +deterministic profiling. Python automatically provides a `hook' +(optional callback) for each event. In addition, the interpreted nature +of Python tends to add so much overhead to execution, that deterministic +profiling tends to only add small processing overhead in typical +applications. The result is that deterministic profiling is not that +expensive, yet provides extensive run time statistics about the +execution of a Python program. + +Call count statistics can be used to identify bugs in code (surprising +counts), and to identify possible inline-expansion points (high call +counts). Internal time statistics can be used to identify “hot loops” +that should be carefully optimized. Cumulative time statistics should +be used to identify high level errors in the selection of algorithms. +Note that the unusual handling of cumulative times in this profiler +allows statistics for recursive implementations of algorithms to be +directly compared to iterative implementations. + + +File: python.info, Node: Limitations, Next: Calibration, Prev: What Is Deterministic Profiling?, Up: The Python Profilers + +5.27.5.6 Limitations +.................... + +One limitation has to do with accuracy of timing information. There is +a fundamental problem with deterministic profilers involving accuracy. +The most obvious restriction is that the underlying “clock” is only +ticking at a rate (typically) of about .001 seconds. Hence no +measurements will be more accurate than the underlying clock. If enough +measurements are taken, then the “error” will tend to average out. +Unfortunately, removing this first error induces a second source of +error. + +The second problem is that it “takes a while” from when an event is +dispatched until the profiler’s call to get the time actually `gets' the +state of the clock. Similarly, there is a certain lag when exiting the +profiler event handler from the time that the clock’s value was obtained +(and then squirreled away), until the user’s code is once again +executing. As a result, functions that are called many times, or call +many functions, will typically accumulate this error. The error that +accumulates in this fashion is typically less than the accuracy of the +clock (less than one clock tick), but it `can' accumulate and become +very significant. + +The problem is more important with *note profile: d3. than with the +lower-overhead *note cProfile: 28. For this reason, *note profile: d3. +provides a means of calibrating itself for a given platform so that this +error can be probabilistically (on the average) removed. After the +profiler is calibrated, it will be more accurate (in a least square +sense), but it will sometimes produce negative numbers (when call counts +are exceptionally low, and the gods of probability work against you :-). +) Do `not' be alarmed by negative numbers in the profile. They should +`only' appear if you have calibrated your profiler, and the results are +actually better than without calibration. + + +File: python.info, Node: Calibration, Next: Using a custom timer, Prev: Limitations, Up: The Python Profilers + +5.27.5.7 Calibration +.................... + +The profiler of the *note profile: d3. module subtracts a constant from +each event handling time to compensate for the overhead of calling the +time function, and socking away the results. By default, the constant +is 0. The following procedure can be used to obtain a better constant +for a given platform (see *note Limitations: 32a0.). + + import profile + pr = profile.Profile() + for i in range(5): + print(pr.calibrate(10000)) + +The method executes the number of Python calls given by the argument, +directly and again under the profiler, measuring the time for both. It +then computes the hidden overhead per profiler event, and returns that +as a float. For example, on a 1.8Ghz Intel Core i5 running Mac OS X, +and using Python’s time.process_time() as the timer, the magical number +is about 4.04e-6. + +The object of this exercise is to get a fairly consistent result. If +your computer is `very' fast, or your timer function has poor +resolution, you might have to pass 100000, or even 1000000, to get +consistent results. + +When you have a consistent answer, there are three ways you can use it: + + import profile + + # 1. Apply computed bias to all Profile instances created hereafter. + profile.Profile.bias = your_computed_bias + + # 2. Apply computed bias to a specific Profile instance. + pr = profile.Profile() + pr.bias = your_computed_bias + + # 3. Specify computed bias in instance constructor. + pr = profile.Profile(bias=your_computed_bias) + +If you have a choice, you are better off choosing a smaller constant, +and then your results will “less often” show up as negative in profile +statistics. + + +File: python.info, Node: Using a custom timer, Prev: Calibration, Up: The Python Profilers + +5.27.5.8 Using a custom timer +............................. + +If you want to change how current time is determined (for example, to +force use of wall-clock time or elapsed process time), pass the timing +function you want to the ‘Profile’ class constructor: + + pr = profile.Profile(your_time_func) + +The resulting profiler will then call ‘your_time_func’. Depending on +whether you are using *note profile.Profile: 1bb. or ‘cProfile.Profile’, +‘your_time_func’’s return value will be interpreted differently: + +*note profile.Profile: 1bb. + + ‘your_time_func’ should return a single number, or a list of + numbers whose sum is the current time (like what *note os.times(): + a5c. returns). If the function returns a single time number, or + the list of returned numbers has length 2, then you will get an + especially fast version of the dispatch routine. + + Be warned that you should calibrate the profiler class for the + timer function that you choose (see *note Calibration: 32a2.). For + most machines, a timer that returns a lone integer value will + provide the best results in terms of low overhead during profiling. + (*note os.times(): a5c. is `pretty' bad, as it returns a tuple of + floating point values). If you want to substitute a better timer + in the cleanest fashion, derive a class and hardwire a replacement + dispatch method that best handles your timer call, along with the + appropriate calibration constant. + +‘cProfile.Profile’ + + ‘your_time_func’ should return a single number. If it returns + integers, you can also invoke the class constructor with a second + argument specifying the real duration of one unit of time. For + example, if ‘your_integer_time_func’ returns times measured in + thousands of seconds, you would construct the ‘Profile’ instance as + follows: + + pr = cProfile.Profile(your_integer_time_func, 0.001) + + As the ‘cProfile.Profile’ class cannot be calibrated, custom timer + functions should be used with care and should be as fast as + possible. For the best results with a custom timer, it might be + necessary to hard-code it in the C source of the internal ‘_lsprof’ + module. + +Python 3.3 adds several new functions in *note time: 10a. that can be +used to make precise measurements of process or wall-clock time. For +example, see *note time.perf_counter(): 2aa. + + +File: python.info, Node: timeit — Measure execution time of small code snippets, Next: trace — Trace or track Python statement execution, Prev: The Python Profilers, Up: Debugging and Profiling + +5.27.6 ‘timeit’ — Measure execution time of small code snippets +--------------------------------------------------------------- + +`Source code:' Lib/timeit.py(1) + +__________________________________________________________________ + +This module provides a simple way to time small bits of Python code. It +has both a *note Command-Line Interface: 32a7. as well as a *note +callable: 32a8. one. It avoids a number of common traps for measuring +execution times. See also Tim Peters’ introduction to the “Algorithms” +chapter in the `Python Cookbook', published by O’Reilly. + +* Menu: + +* Basic Examples: Basic Examples<2>. +* Python Interface:: +* Command-Line Interface: Command-Line Interface<4>. +* Examples: Examples<25>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/timeit.py + + +File: python.info, Node: Basic Examples<2>, Next: Python Interface, Up: timeit — Measure execution time of small code snippets + +5.27.6.1 Basic Examples +....................... + +The following example shows how the *note Command-Line Interface: 32a7. +can be used to compare three different expressions: + + $ python3 -m timeit '"-".join(str(n) for n in range(100))' + 10000 loops, best of 5: 30.2 usec per loop + $ python3 -m timeit '"-".join([str(n) for n in range(100)])' + 10000 loops, best of 5: 27.5 usec per loop + $ python3 -m timeit '"-".join(map(str, range(100)))' + 10000 loops, best of 5: 23.2 usec per loop + +This can be achieved from the *note Python Interface: 32a8. with: + + >>> import timeit + >>> timeit.timeit('"-".join(str(n) for n in range(100))', number=10000) + 0.3018611848820001 + >>> timeit.timeit('"-".join([str(n) for n in range(100)])', number=10000) + 0.2727368790656328 + >>> timeit.timeit('"-".join(map(str, range(100)))', number=10000) + 0.23702679807320237 + +A callable can also be passed from the *note Python Interface: 32a8.: + + >>> timeit.timeit(lambda: "-".join(map(str, range(100))), number=10000) + 0.19665591977536678 + +Note however that *note timeit(): 771. will automatically determine the +number of repetitions only when the command-line interface is used. In +the *note Examples: 32aa. section you can find more advanced examples. + + +File: python.info, Node: Python Interface, Next: Command-Line Interface<4>, Prev: Basic Examples<2>, Up: timeit — Measure execution time of small code snippets + +5.27.6.2 Python Interface +......................... + +The module defines three convenience functions and a public class: + + -- Function: timeit.timeit (stmt='pass', setup='pass', timer=<default + timer>, number=1000000, globals=None) + + Create a *note Timer: 32ac. instance with the given statement, + `setup' code and `timer' function and run its *note timeit(): 59c. + method with `number' executions. The optional `globals' argument + specifies a namespace in which to execute the code. + + Changed in version 3.5: The optional `globals' parameter was added. + + -- Function: timeit.repeat (stmt='pass', setup='pass', timer=<default + timer>, repeat=5, number=1000000, globals=None) + + Create a *note Timer: 32ac. instance with the given statement, + `setup' code and `timer' function and run its *note repeat(): 32ae. + method with the given `repeat' count and `number' executions. The + optional `globals' argument specifies a namespace in which to + execute the code. + + Changed in version 3.5: The optional `globals' parameter was added. + + Changed in version 3.7: Default value of `repeat' changed from 3 to + 5. + + -- Function: timeit.default_timer () + + The default timer, which is always *note time.perf_counter(): 2aa. + + Changed in version 3.3: *note time.perf_counter(): 2aa. is now the + default timer. + + -- Class: timeit.Timer (stmt='pass', setup='pass', timer=<timer + function>, globals=None) + + Class for timing execution speed of small code snippets. + + The constructor takes a statement to be timed, an additional + statement used for setup, and a timer function. Both statements + default to ‘'pass'’; the timer function is platform-dependent (see + the module doc string). `stmt' and `setup' may also contain + multiple statements separated by ‘;’ or newlines, as long as they + don’t contain multi-line string literals. The statement will by + default be executed within timeit’s namespace; this behavior can be + controlled by passing a namespace to `globals'. + + To measure the execution time of the first statement, use the *note + timeit(): 59c. method. The *note repeat(): 32ae. and *note + autorange(): 59b. methods are convenience methods to call *note + timeit(): 59c. multiple times. + + The execution time of `setup' is excluded from the overall timed + execution run. + + The `stmt' and `setup' parameters can also take objects that are + callable without arguments. This will embed calls to them in a + timer function that will then be executed by *note timeit(): 59c. + Note that the timing overhead is a little larger in this case + because of the extra function calls. + + Changed in version 3.5: The optional `globals' parameter was added. + + -- Method: timeit (number=1000000) + + Time `number' executions of the main statement. This executes + the setup statement once, and then returns the time it takes + to execute the main statement a number of times, measured in + seconds as a float. The argument is the number of times + through the loop, defaulting to one million. The main + statement, the setup statement and the timer function to be + used are passed to the constructor. + + Note: By default, *note timeit(): 59c. temporarily turns + off *note garbage collection: f9f. during the timing. + The advantage of this approach is that it makes + independent timings more comparable. The disadvantage is + that GC may be an important component of the performance + of the function being measured. If so, GC can be + re-enabled as the first statement in the `setup' string. + For example: + + timeit.Timer('for i in range(10): oct(i)', 'gc.enable()').timeit() + + -- Method: autorange (callback=None) + + Automatically determine how many times to call *note timeit(): + 59c. + + This is a convenience function that calls *note timeit(): 59c. + repeatedly so that the total time >= 0.2 second, returning the + eventual (number of loops, time taken for that number of + loops). It calls *note timeit(): 59c. with increasing numbers + from the sequence 1, 2, 5, 10, 20, 50, … until the time taken + is at least 0.2 second. + + If `callback' is given and is not ‘None’, it will be called + after each trial with two arguments: ‘callback(number, + time_taken)’. + + New in version 3.6. + + -- Method: repeat (repeat=5, number=1000000) + + Call *note timeit(): 59c. a few times. + + This is a convenience function that calls the *note timeit(): + 59c. repeatedly, returning a list of results. The first + argument specifies how many times to call *note timeit(): 59c. + The second argument specifies the `number' argument for *note + timeit(): 59c. + + Note: It’s tempting to calculate mean and standard + deviation from the result vector and report these. + However, this is not very useful. In a typical case, the + lowest value gives a lower bound for how fast your + machine can run the given code snippet; higher values in + the result vector are typically not caused by variability + in Python’s speed, but by other processes interfering + with your timing accuracy. So the *note min(): 809. of + the result is probably the only number you should be + interested in. After that, you should look at the entire + vector and apply common sense rather than statistics. + + Changed in version 3.7: Default value of `repeat' changed from + 3 to 5. + + -- Method: print_exc (file=None) + + Helper to print a traceback from the timed code. + + Typical use: + + t = Timer(...) # outside the try/except + try: + t.timeit(...) # or t.repeat(...) + except Exception: + t.print_exc() + + The advantage over the standard traceback is that source lines + in the compiled template will be displayed. The optional + `file' argument directs where the traceback is sent; it + defaults to *note sys.stderr: 30f. + + +File: python.info, Node: Command-Line Interface<4>, Next: Examples<25>, Prev: Python Interface, Up: timeit — Measure execution time of small code snippets + +5.27.6.3 Command-Line Interface +............................... + +When called as a program from the command line, the following form is +used: + + python -m timeit [-n N] [-r N] [-u U] [-s S] [-h] [statement ...] + +Where the following options are understood: + + -- Program Option: -n N, --number=N + + how many times to execute ‘statement’ + + -- Program Option: -r N, --repeat=N + + how many times to repeat the timer (default 5) + + -- Program Option: -s S, --setup=S + + statement to be executed once initially (default ‘pass’) + + -- Program Option: -p, --process + + measure process time, not wallclock time, using *note + time.process_time(): 2ab. instead of *note time.perf_counter(): + 2aa, which is the default + + New in version 3.3. + + -- Program Option: -u, --unit=U + + specify a time unit for timer output; can select nsec, usec, msec, + or sec + + New in version 3.5. + + -- Program Option: -v, --verbose + + print raw timing results; repeat for more digits precision + + -- Program Option: -h, --help + + print a short usage message and exit + +A multi-line statement may be given by specifying each line as a +separate statement argument; indented lines are possible by enclosing an +argument in quotes and using leading spaces. Multiple *note -s: 32b4. +options are treated similarly. + +If *note -n: 32b2. is not given, a suitable number of loops is +calculated by trying increasing numbers from the sequence 1, 2, 5, 10, +20, 50, … until the total time is at least 0.2 seconds. + +*note default_timer(): 32af. measurements can be affected by other +programs running on the same machine, so the best thing to do when +accurate timing is necessary is to repeat the timing a few times and use +the best time. The *note -r: 32b3. option is good for this; the default +of 5 repetitions is probably enough in most cases. You can use *note +time.process_time(): 2ab. to measure CPU time. + + Note: There is a certain baseline overhead associated with + executing a pass statement. The code here doesn’t try to hide it, + but you should be aware of it. The baseline overhead can be + measured by invoking the program without arguments, and it might + differ between Python versions. + + +File: python.info, Node: Examples<25>, Prev: Command-Line Interface<4>, Up: timeit — Measure execution time of small code snippets + +5.27.6.4 Examples +................. + +It is possible to provide a setup statement that is executed only once +at the beginning: + + $ python -m timeit -s 'text = "sample string"; char = "g"' 'char in text' + 5000000 loops, best of 5: 0.0877 usec per loop + $ python -m timeit -s 'text = "sample string"; char = "g"' 'text.find(char)' + 1000000 loops, best of 5: 0.342 usec per loop + + >>> import timeit + >>> timeit.timeit('char in text', setup='text = "sample string"; char = "g"') + 0.41440500499993504 + >>> timeit.timeit('text.find(char)', setup='text = "sample string"; char = "g"') + 1.7246671520006203 + +The same can be done using the *note Timer: 32ac. class and its methods: + + >>> import timeit + >>> t = timeit.Timer('char in text', setup='text = "sample string"; char = "g"') + >>> t.timeit() + 0.3955516149999312 + >>> t.repeat() + [0.40183617287970225, 0.37027556854118704, 0.38344867356679524, 0.3712595970846668, 0.37866875250654886] + +The following examples show how to time expressions that contain +multiple lines. Here we compare the cost of using *note hasattr(): 447. +vs. *note try: d72./*note except: b3e. to test for missing and present +object attributes: + + $ python -m timeit 'try:' ' str.__bool__' 'except AttributeError:' ' pass' + 20000 loops, best of 5: 15.7 usec per loop + $ python -m timeit 'if hasattr(str, "__bool__"): pass' + 50000 loops, best of 5: 4.26 usec per loop + + $ python -m timeit 'try:' ' int.__bool__' 'except AttributeError:' ' pass' + 200000 loops, best of 5: 1.43 usec per loop + $ python -m timeit 'if hasattr(int, "__bool__"): pass' + 100000 loops, best of 5: 2.23 usec per loop + + >>> import timeit + >>> # attribute is missing + >>> s = """\ + ... try: + ... str.__bool__ + ... except AttributeError: + ... pass + ... """ + >>> timeit.timeit(stmt=s, number=100000) + 0.9138244460009446 + >>> s = "if hasattr(str, '__bool__'): pass" + >>> timeit.timeit(stmt=s, number=100000) + 0.5829014980008651 + >>> + >>> # attribute is present + >>> s = """\ + ... try: + ... int.__bool__ + ... except AttributeError: + ... pass + ... """ + >>> timeit.timeit(stmt=s, number=100000) + 0.04215312199994514 + >>> s = "if hasattr(int, '__bool__'): pass" + >>> timeit.timeit(stmt=s, number=100000) + 0.08588060699912603 + +To give the *note timeit: 10b. module access to functions you define, +you can pass a `setup' parameter which contains an import statement: + + def test(): + """Stupid test function""" + L = [i for i in range(100)] + + if __name__ == '__main__': + import timeit + print(timeit.timeit("test()", setup="from __main__ import test")) + +Another option is to pass *note globals(): 128c. to the `globals' +parameter, which will cause the code to be executed within your current +global namespace. This can be more convenient than individually +specifying imports: + + def f(x): + return x**2 + def g(x): + return x**4 + def h(x): + return x**8 + + import timeit + print(timeit.timeit('[func(42) for func in (f,g,h)]', globals=globals())) + + +File: python.info, Node: trace — Trace or track Python statement execution, Next: tracemalloc — Trace memory allocations, Prev: timeit — Measure execution time of small code snippets, Up: Debugging and Profiling + +5.27.7 ‘trace’ — Trace or track Python statement execution +---------------------------------------------------------- + +`Source code:' Lib/trace.py(1) + +__________________________________________________________________ + +The *note trace: 112. module allows you to trace program execution, +generate annotated statement coverage listings, print caller/callee +relationships and list functions executed during a program run. It can +be used in another program or from the command line. + +See also +........ + +Coverage.py(2) + + A popular third-party coverage tool that provides HTML output along + with advanced features such as branch coverage. + +* Menu: + +* Command-Line Usage:: +* Programmatic Interface:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/trace.py + + (2) https://coverage.readthedocs.io/ + + +File: python.info, Node: Command-Line Usage, Next: Programmatic Interface, Up: trace — Trace or track Python statement execution + +5.27.7.1 Command-Line Usage +........................... + +The *note trace: 112. module can be invoked from the command line. It +can be as simple as + + python -m trace --count -C . somefile.py ... + +The above will execute ‘somefile.py’ and generate annotated listings of +all Python modules imported during the execution into the current +directory. + + -- Program Option: --help + + Display usage and exit. + + -- Program Option: --version + + Display the version of the module and exit. + +New in version 3.8: Added ‘--module’ option that allows to run an +executable module. + +* Menu: + +* Main options:: +* Modifiers:: +* Filters:: + + +File: python.info, Node: Main options, Next: Modifiers, Up: Command-Line Usage + +5.27.7.2 Main options +..................... + +At least one of the following options must be specified when invoking +*note trace: 112. The *note –listfuncs: 32c1. option is mutually +exclusive with the *note –trace: 32c2. and *note –count: 32c3. options. +When *note –listfuncs: 32c1. is provided, neither *note –count: 32c3. +nor *note –trace: 32c2. are accepted, and vice versa. + + -- Program Option: -c, --count + + Produce a set of annotated listing files upon program completion + that shows how many times each statement was executed. See also + *note –coverdir: 32c4, *note –file: 32c5. and *note –no-report: + 32c6. below. + + -- Program Option: -t, --trace + + Display lines as they are executed. + + -- Program Option: -l, --listfuncs + + Display the functions executed by running the program. + + -- Program Option: -r, --report + + Produce an annotated list from an earlier program run that used the + *note –count: 32c3. and *note –file: 32c5. option. This does not + execute any code. + + -- Program Option: -T, --trackcalls + + Display the calling relationships exposed by running the program. + + +File: python.info, Node: Modifiers, Next: Filters, Prev: Main options, Up: Command-Line Usage + +5.27.7.3 Modifiers +.................. + + -- Program Option: -f, --file=<file> + + Name of a file to accumulate counts over several tracing runs. + Should be used with the *note –count: 32c3. option. + + -- Program Option: -C, --coverdir=<dir> + + Directory where the report files go. The coverage report for + ‘package.module’ is written to file + ‘`dir'/`package'/`module'.cover’. + + -- Program Option: -m, --missing + + When generating annotated listings, mark lines which were not + executed with ‘>>>>>>’. + + -- Program Option: -s, --summary + + When using *note –count: 32c3. or *note –report: 32c7, write a + brief summary to stdout for each file processed. + + -- Program Option: -R, --no-report + + Do not generate annotated listings. This is useful if you intend + to make several runs with *note –count: 32c3, and then produce a + single set of annotated listings at the end. + + -- Program Option: -g, --timing + + Prefix each line with the time since the program started. Only + used while tracing. + + +File: python.info, Node: Filters, Prev: Modifiers, Up: Command-Line Usage + +5.27.7.4 Filters +................ + +These options may be repeated multiple times. + + -- Program Option: --ignore-module=<mod> + + Ignore each of the given module names and its submodules (if it is + a package). The argument can be a list of names separated by a + comma. + + -- Program Option: --ignore-dir=<dir> + + Ignore all modules and packages in the named directory and + subdirectories. The argument can be a list of directories + separated by *note os.pathsep: ff0. + + +File: python.info, Node: Programmatic Interface, Prev: Command-Line Usage, Up: trace — Trace or track Python statement execution + +5.27.7.5 Programmatic Interface +............................... + + -- Class: trace.Trace (count=1, trace=1, countfuncs=0, countcallers=0, + ignoremods=(), ignoredirs=(), infile=None, outfile=None, + timing=False) + + Create an object to trace execution of a single statement or + expression. All parameters are optional. `count' enables counting + of line numbers. `trace' enables line execution tracing. + `countfuncs' enables listing of the functions called during the + run. `countcallers' enables call relationship tracking. + `ignoremods' is a list of modules or packages to ignore. + `ignoredirs' is a list of directories whose modules or packages + should be ignored. `infile' is the name of the file from which to + read stored count information. `outfile' is the name of the file + in which to write updated count information. `timing' enables a + timestamp relative to when tracing was started to be displayed. + + -- Method: run (cmd) + + Execute the command and gather statistics from the execution + with the current tracing parameters. `cmd' must be a string + or code object, suitable for passing into *note exec(): c44. + + -- Method: runctx (cmd, globals=None, locals=None) + + Execute the command and gather statistics from the execution + with the current tracing parameters, in the defined global and + local environments. If not defined, `globals' and `locals' + default to empty dictionaries. + + -- Method: runfunc (func, *args, **kwds) + + Call `func' with the given arguments under control of the + *note Trace: 32d2. object with the current tracing parameters. + + -- Method: results () + + Return a *note CoverageResults: 32d6. object that contains the + cumulative results of all previous calls to ‘run’, ‘runctx’ + and ‘runfunc’ for the given *note Trace: 32d2. instance. Does + not reset the accumulated trace results. + + -- Class: trace.CoverageResults + + A container for coverage results, created by *note Trace.results(): + 32d5. Should not be created directly by the user. + + -- Method: update (other) + + Merge in data from another *note CoverageResults: 32d6. + object. + + -- Method: write_results (show_missing=True, summary=False, + coverdir=None) + + Write coverage results. Set `show_missing' to show lines that + had no hits. Set `summary' to include in the output the + coverage summary per module. `coverdir' specifies the + directory into which the coverage result files will be output. + If ‘None’, the results for each source file are placed in its + directory. + +A simple example demonstrating the use of the programmatic interface: + + import sys + import trace + + # create a Trace object, telling it what to ignore, and whether to + # do tracing or line-counting or both. + tracer = trace.Trace( + ignoredirs=[sys.prefix, sys.exec_prefix], + trace=0, + count=1) + + # run the new command using the given tracer + tracer.run('main()') + + # make a report, placing output in the current directory + r = tracer.results() + r.write_results(show_missing=True, coverdir=".") + + +File: python.info, Node: tracemalloc — Trace memory allocations, Prev: trace — Trace or track Python statement execution, Up: Debugging and Profiling + +5.27.8 ‘tracemalloc’ — Trace memory allocations +----------------------------------------------- + +New in version 3.4. + +`Source code:' Lib/tracemalloc.py(1) + +__________________________________________________________________ + +The tracemalloc module is a debug tool to trace memory blocks allocated +by Python. It provides the following information: + + * Traceback where an object was allocated + + * Statistics on allocated memory blocks per filename and per line + number: total size, number and average size of allocated memory + blocks + + * Compute the differences between two snapshots to detect memory + leaks + +To trace most memory blocks allocated by Python, the module should be +started as early as possible by setting the *note PYTHONTRACEMALLOC: +ff6. environment variable to ‘1’, or by using *note -X: 155. +‘tracemalloc’ command line option. The *note tracemalloc.start(): fea. +function can be called at runtime to start tracing Python memory +allocations. + +By default, a trace of an allocated memory block only stores the most +recent frame (1 frame). To store 25 frames at startup: set the *note +PYTHONTRACEMALLOC: ff6. environment variable to ‘25’, or use the *note +-X: 155. ‘tracemalloc=25’ command line option. + +* Menu: + +* Examples: Examples<26>. +* API:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/tracemalloc.py + + +File: python.info, Node: Examples<26>, Next: API, Up: tracemalloc — Trace memory allocations + +5.27.8.1 Examples +................. + +* Menu: + +* Display the top 10:: +* Compute differences:: +* Get the traceback of a memory block:: +* Pretty top:: + + +File: python.info, Node: Display the top 10, Next: Compute differences, Up: Examples<26> + +5.27.8.2 Display the top 10 +........................... + +Display the 10 files allocating the most memory: + + import tracemalloc + + tracemalloc.start() + + # ... run your application ... + + snapshot = tracemalloc.take_snapshot() + top_stats = snapshot.statistics('lineno') + + print("[ Top 10 ]") + for stat in top_stats[:10]: + print(stat) + +Example of output of the Python test suite: + + [ Top 10 ] + <frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B + <frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B + /usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B + /usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B + /usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B + /usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B + <frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B + <frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B + <string>:5: size=49.7 KiB, count=148, average=344 B + /usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB + +We can see that Python loaded ‘4855 KiB’ data (bytecode and constants) +from modules and that the *note collections: 1e. module allocated ‘244 +KiB’ to build *note namedtuple: 1b7. types. + +See *note Snapshot.statistics(): 32dd. for more options. + + +File: python.info, Node: Compute differences, Next: Get the traceback of a memory block, Prev: Display the top 10, Up: Examples<26> + +5.27.8.3 Compute differences +............................ + +Take two snapshots and display the differences: + + import tracemalloc + tracemalloc.start() + # ... start your application ... + + snapshot1 = tracemalloc.take_snapshot() + # ... call the function leaking memory ... + snapshot2 = tracemalloc.take_snapshot() + + top_stats = snapshot2.compare_to(snapshot1, 'lineno') + + print("[ Top 10 differences ]") + for stat in top_stats[:10]: + print(stat) + +Example of output before/after running some tests of the Python test +suite: + + [ Top 10 differences ] + <frozen importlib._bootstrap>:716: size=8173 KiB (+4428 KiB), count=71332 (+39369), average=117 B + /usr/lib/python3.4/linecache.py:127: size=940 KiB (+940 KiB), count=8106 (+8106), average=119 B + /usr/lib/python3.4/unittest/case.py:571: size=298 KiB (+298 KiB), count=589 (+589), average=519 B + <frozen importlib._bootstrap>:284: size=1005 KiB (+166 KiB), count=7423 (+1526), average=139 B + /usr/lib/python3.4/mimetypes.py:217: size=112 KiB (+112 KiB), count=1334 (+1334), average=86 B + /usr/lib/python3.4/http/server.py:848: size=96.0 KiB (+96.0 KiB), count=1 (+1), average=96.0 KiB + /usr/lib/python3.4/inspect.py:1465: size=83.5 KiB (+83.5 KiB), count=109 (+109), average=784 B + /usr/lib/python3.4/unittest/mock.py:491: size=77.7 KiB (+77.7 KiB), count=143 (+143), average=557 B + /usr/lib/python3.4/urllib/parse.py:476: size=71.8 KiB (+71.8 KiB), count=969 (+969), average=76 B + /usr/lib/python3.4/contextlib.py:38: size=67.2 KiB (+67.2 KiB), count=126 (+126), average=546 B + +We can see that Python has loaded ‘8173 KiB’ of module data (bytecode +and constants), and that this is ‘4428 KiB’ more than had been loaded +before the tests, when the previous snapshot was taken. Similarly, the +*note linecache: a8. module has cached ‘940 KiB’ of Python source code +to format tracebacks, all of it since the previous snapshot. + +If the system has little free memory, snapshots can be written on disk +using the *note Snapshot.dump(): 32df. method to analyze the snapshot +offline. Then use the *note Snapshot.load(): 32e0. method reload the +snapshot. + + +File: python.info, Node: Get the traceback of a memory block, Next: Pretty top, Prev: Compute differences, Up: Examples<26> + +5.27.8.4 Get the traceback of a memory block +............................................ + +Code to display the traceback of the biggest memory block: + + import tracemalloc + + # Store 25 frames + tracemalloc.start(25) + + # ... run your application ... + + snapshot = tracemalloc.take_snapshot() + top_stats = snapshot.statistics('traceback') + + # pick the biggest memory block + stat = top_stats[0] + print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024)) + for line in stat.traceback.format(): + print(line) + +Example of output of the Python test suite (traceback limited to 25 +frames): + + 903 memory blocks: 870.1 KiB + File "<frozen importlib._bootstrap>", line 716 + File "<frozen importlib._bootstrap>", line 1036 + File "<frozen importlib._bootstrap>", line 934 + File "<frozen importlib._bootstrap>", line 1068 + File "<frozen importlib._bootstrap>", line 619 + File "<frozen importlib._bootstrap>", line 1581 + File "<frozen importlib._bootstrap>", line 1614 + File "/usr/lib/python3.4/doctest.py", line 101 + import pdb + File "<frozen importlib._bootstrap>", line 284 + File "<frozen importlib._bootstrap>", line 938 + File "<frozen importlib._bootstrap>", line 1068 + File "<frozen importlib._bootstrap>", line 619 + File "<frozen importlib._bootstrap>", line 1581 + File "<frozen importlib._bootstrap>", line 1614 + File "/usr/lib/python3.4/test/support/__init__.py", line 1728 + import doctest + File "/usr/lib/python3.4/test/test_pickletools.py", line 21 + support.run_doctest(pickletools) + File "/usr/lib/python3.4/test/regrtest.py", line 1276 + test_runner() + File "/usr/lib/python3.4/test/regrtest.py", line 976 + display_failure=not verbose) + File "/usr/lib/python3.4/test/regrtest.py", line 761 + match_tests=ns.match_tests) + File "/usr/lib/python3.4/test/regrtest.py", line 1563 + main() + File "/usr/lib/python3.4/test/__main__.py", line 3 + regrtest.main_in_temp_cwd() + File "/usr/lib/python3.4/runpy.py", line 73 + exec(code, run_globals) + File "/usr/lib/python3.4/runpy.py", line 160 + "__main__", fname, loader, pkg_name) + +We can see that the most memory was allocated in the *note importlib: +9b. module to load data (bytecode and constants) from modules: ‘870.1 +KiB’. The traceback is where the *note importlib: 9b. loaded data most +recently: on the ‘import pdb’ line of the *note doctest: 67. module. +The traceback may change if a new module is loaded. + + +File: python.info, Node: Pretty top, Prev: Get the traceback of a memory block, Up: Examples<26> + +5.27.8.5 Pretty top +................... + +Code to display the 10 lines allocating the most memory with a pretty +output, ignoring ‘<frozen importlib._bootstrap>’ and ‘<unknown>’ files: + + import linecache + import os + import tracemalloc + + def display_top(snapshot, key_type='lineno', limit=10): + snapshot = snapshot.filter_traces(( + tracemalloc.Filter(False, "<frozen importlib._bootstrap>"), + tracemalloc.Filter(False, "<unknown>"), + )) + top_stats = snapshot.statistics(key_type) + + print("Top %s lines" % limit) + for index, stat in enumerate(top_stats[:limit], 1): + frame = stat.traceback[0] + print("#%s: %s:%s: %.1f KiB" + % (index, frame.filename, frame.lineno, stat.size / 1024)) + line = linecache.getline(frame.filename, frame.lineno).strip() + if line: + print(' %s' % line) + + other = top_stats[limit:] + if other: + size = sum(stat.size for stat in other) + print("%s other: %.1f KiB" % (len(other), size / 1024)) + total = sum(stat.size for stat in top_stats) + print("Total allocated size: %.1f KiB" % (total / 1024)) + + tracemalloc.start() + + # ... run your application ... + + snapshot = tracemalloc.take_snapshot() + display_top(snapshot) + +Example of output of the Python test suite: + + Top 10 lines + #1: Lib/base64.py:414: 419.8 KiB + _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars] + #2: Lib/base64.py:306: 419.8 KiB + _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars] + #3: collections/__init__.py:368: 293.6 KiB + exec(class_definition, namespace) + #4: Lib/abc.py:133: 115.2 KiB + cls = super().__new__(mcls, name, bases, namespace) + #5: unittest/case.py:574: 103.1 KiB + testMethod() + #6: Lib/linecache.py:127: 95.4 KiB + lines = fp.readlines() + #7: urllib/parse.py:476: 71.8 KiB + for a in _hexdig for b in _hexdig} + #8: <string>:5: 62.0 KiB + #9: Lib/_weakrefset.py:37: 60.0 KiB + self.data = set() + #10: Lib/base64.py:142: 59.8 KiB + _b32tab2 = [a + b for a in _b32tab for b in _b32tab] + 6220 other: 3602.8 KiB + Total allocated size: 5303.1 KiB + +See *note Snapshot.statistics(): 32dd. for more options. + + +File: python.info, Node: API, Prev: Examples<26>, Up: tracemalloc — Trace memory allocations + +5.27.8.6 API +............ + +* Menu: + +* Functions: Functions<9>. +* DomainFilter:: +* Filter:: +* Frame:: +* Snapshot:: +* Statistic:: +* StatisticDiff:: +* Trace:: +* Traceback:: + + +File: python.info, Node: Functions<9>, Next: DomainFilter, Up: API + +5.27.8.7 Functions +.................. + + -- Function: tracemalloc.clear_traces () + + Clear traces of memory blocks allocated by Python. + + See also *note stop(): 32e6. + + -- Function: tracemalloc.get_object_traceback (obj) + + Get the traceback where the Python object `obj' was allocated. + Return a *note Traceback: 3ff. instance, or ‘None’ if the *note + tracemalloc: 114. module is not tracing memory allocations or did + not trace the allocation of the object. + + See also *note gc.get_referrers(): 31f5. and *note sys.getsizeof(): + 32e8. functions. + + -- Function: tracemalloc.get_traceback_limit () + + Get the maximum number of frames stored in the traceback of a + trace. + + The *note tracemalloc: 114. module must be tracing memory + allocations to get the limit, otherwise an exception is raised. + + The limit is set by the *note start(): fea. function. + + -- Function: tracemalloc.get_traced_memory () + + Get the current size and peak size of memory blocks traced by the + *note tracemalloc: 114. module as a tuple: ‘(current: int, peak: + int)’. + + -- Function: tracemalloc.get_tracemalloc_memory () + + Get the memory usage in bytes of the *note tracemalloc: 114. module + used to store traces of memory blocks. Return an *note int: 184. + + -- Function: tracemalloc.is_tracing () + + ‘True’ if the *note tracemalloc: 114. module is tracing Python + memory allocations, ‘False’ otherwise. + + See also *note start(): fea. and *note stop(): 32e6. functions. + + -- Function: tracemalloc.start (nframe: int=1) + + Start tracing Python memory allocations: install hooks on Python + memory allocators. Collected tracebacks of traces will be limited + to `nframe' frames. By default, a trace of a memory block only + stores the most recent frame: the limit is ‘1’. `nframe' must be + greater or equal to ‘1’. + + Storing more than ‘1’ frame is only useful to compute statistics + grouped by ‘'traceback'’ or to compute cumulative statistics: see + the *note Snapshot.compare_to(): 32ed. and *note + Snapshot.statistics(): 32dd. methods. + + Storing more frames increases the memory and CPU overhead of the + *note tracemalloc: 114. module. Use the *note + get_tracemalloc_memory(): 32eb. function to measure how much memory + is used by the *note tracemalloc: 114. module. + + The *note PYTHONTRACEMALLOC: ff6. environment variable + (‘PYTHONTRACEMALLOC=NFRAME’) and the *note -X: 155. + ‘tracemalloc=NFRAME’ command line option can be used to start + tracing at startup. + + See also *note stop(): 32e6, *note is_tracing(): 32ec. and *note + get_traceback_limit(): 32e9. functions. + + -- Function: tracemalloc.stop () + + Stop tracing Python memory allocations: uninstall hooks on Python + memory allocators. Also clears all previously collected traces of + memory blocks allocated by Python. + + Call *note take_snapshot(): 32ee. function to take a snapshot of + traces before clearing them. + + See also *note start(): fea, *note is_tracing(): 32ec. and *note + clear_traces(): 32e5. functions. + + -- Function: tracemalloc.take_snapshot () + + Take a snapshot of traces of memory blocks allocated by Python. + Return a new *note Snapshot: 32ef. instance. + + The snapshot does not include memory blocks allocated before the + *note tracemalloc: 114. module started to trace memory allocations. + + Tracebacks of traces are limited to *note get_traceback_limit(): + 32e9. frames. Use the `nframe' parameter of the *note start(): + fea. function to store more frames. + + The *note tracemalloc: 114. module must be tracing memory + allocations to take a snapshot, see the *note start(): fea. + function. + + See also the *note get_object_traceback(): 32e7. function. + + +File: python.info, Node: DomainFilter, Next: Filter, Prev: Functions<9>, Up: API + +5.27.8.8 DomainFilter +..................... + + -- Class: tracemalloc.DomainFilter (inclusive: bool, domain: int) + + Filter traces of memory blocks by their address space (domain). + + New in version 3.6. + + -- Attribute: inclusive + + If `inclusive' is ‘True’ (include), match memory blocks + allocated in the address space *note domain: 32f2. + + If `inclusive' is ‘False’ (exclude), match memory blocks not + allocated in the address space *note domain: 32f2. + + -- Attribute: domain + + Address space of a memory block (‘int’). Read-only property. + + +File: python.info, Node: Filter, Next: Frame, Prev: DomainFilter, Up: API + +5.27.8.9 Filter +............... + + -- Class: tracemalloc.Filter (inclusive: bool, filename_pattern: str, + lineno: int=None, all_frames: bool=False, domain: int=None) + + Filter on traces of memory blocks. + + See the *note fnmatch.fnmatch(): 1935. function for the syntax of + `filename_pattern'. The ‘'.pyc'’ file extension is replaced with + ‘'.py'’. + + Examples: + + * ‘Filter(True, subprocess.__file__)’ only includes traces of + the *note subprocess: f9. module + + * ‘Filter(False, tracemalloc.__file__)’ excludes traces of the + *note tracemalloc: 114. module + + * ‘Filter(False, "<unknown>")’ excludes empty tracebacks + + Changed in version 3.5: The ‘'.pyo'’ file extension is no longer + replaced with ‘'.py'’. + + Changed in version 3.6: Added the *note domain: 32f5. attribute. + + -- Attribute: domain + + Address space of a memory block (‘int’ or ‘None’). + + tracemalloc uses the domain ‘0’ to trace memory allocations + made by Python. C extensions can use other domains to trace + other resources. + + -- Attribute: inclusive + + If `inclusive' is ‘True’ (include), only match memory blocks + allocated in a file with a name matching *note + filename_pattern: 32f7. at line number *note lineno: 32f8. + + If `inclusive' is ‘False’ (exclude), ignore memory blocks + allocated in a file with a name matching *note + filename_pattern: 32f7. at line number *note lineno: 32f8. + + -- Attribute: lineno + + Line number (‘int’) of the filter. If `lineno' is ‘None’, the + filter matches any line number. + + -- Attribute: filename_pattern + + Filename pattern of the filter (‘str’). Read-only property. + + -- Attribute: all_frames + + If `all_frames' is ‘True’, all frames of the traceback are + checked. If `all_frames' is ‘False’, only the most recent + frame is checked. + + This attribute has no effect if the traceback limit is ‘1’. + See the *note get_traceback_limit(): 32e9. function and *note + Snapshot.traceback_limit: 32fa. attribute. + + +File: python.info, Node: Frame, Next: Snapshot, Prev: Filter, Up: API + +5.27.8.10 Frame +............... + + -- Class: tracemalloc.Frame + + Frame of a traceback. + + The *note Traceback: 3ff. class is a sequence of *note Frame: 32fc. + instances. + + -- Attribute: filename + + Filename (‘str’). + + -- Attribute: lineno + + Line number (‘int’). + + +File: python.info, Node: Snapshot, Next: Statistic, Prev: Frame, Up: API + +5.27.8.11 Snapshot +.................. + + -- Class: tracemalloc.Snapshot + + Snapshot of traces of memory blocks allocated by Python. + + The *note take_snapshot(): 32ee. function creates a snapshot + instance. + + -- Method: compare_to (old_snapshot: Snapshot, key_type: str, + cumulative: bool=False) + + Compute the differences with an old snapshot. Get statistics + as a sorted list of *note StatisticDiff: 3300. instances + grouped by `key_type'. + + See the *note Snapshot.statistics(): 32dd. method for + `key_type' and `cumulative' parameters. + + The result is sorted from the biggest to the smallest by: + absolute value of *note StatisticDiff.size_diff: 3301, *note + StatisticDiff.size: 3302, absolute value of *note + StatisticDiff.count_diff: 3303, *note Statistic.count: 3304. + and then by *note StatisticDiff.traceback: 3305. + + -- Method: dump (filename) + + Write the snapshot into a file. + + Use *note load(): 32e0. to reload the snapshot. + + -- Method: filter_traces (filters) + + Create a new *note Snapshot: 32ef. instance with a filtered + *note traces: 3307. sequence, `filters' is a list of *note + DomainFilter: 5a0. and *note Filter: 32f4. instances. If + `filters' is an empty list, return a new *note Snapshot: 32ef. + instance with a copy of the traces. + + All inclusive filters are applied at once, a trace is ignored + if no inclusive filters match it. A trace is ignored if at + least one exclusive filter matches it. + + Changed in version 3.6: *note DomainFilter: 5a0. instances are + now also accepted in `filters'. + + -- Method: classmethod load (filename) + + Load a snapshot from a file. + + See also *note dump(): 32df. + + -- Method: statistics (key_type: str, cumulative: bool=False) + + Get statistics as a sorted list of *note Statistic: 3308. + instances grouped by `key_type': + + key_type description + + ------------------------------------------------------- + + ‘'filename'’ filename + + + ‘'lineno'’ filename and line number + + + ‘'traceback'’ traceback + + + If `cumulative' is ‘True’, cumulate size and count of memory + blocks of all frames of the traceback of a trace, not only the + most recent frame. The cumulative mode can only be used with + `key_type' equals to ‘'filename'’ and ‘'lineno'’. + + The result is sorted from the biggest to the smallest by: + *note Statistic.size: 3309, *note Statistic.count: 3304. and + then by *note Statistic.traceback: 330a. + + -- Attribute: traceback_limit + + Maximum number of frames stored in the traceback of *note + traces: 3307.: result of the *note get_traceback_limit(): + 32e9. when the snapshot was taken. + + -- Attribute: traces + + Traces of all memory blocks allocated by Python: sequence of + *note Trace: 330b. instances. + + The sequence has an undefined order. Use the *note + Snapshot.statistics(): 32dd. method to get a sorted list of + statistics. + + +File: python.info, Node: Statistic, Next: StatisticDiff, Prev: Snapshot, Up: API + +5.27.8.12 Statistic +................... + + -- Class: tracemalloc.Statistic + + Statistic on memory allocations. + + *note Snapshot.statistics(): 32dd. returns a list of *note + Statistic: 3308. instances. + + See also the *note StatisticDiff: 3300. class. + + -- Attribute: count + + Number of memory blocks (‘int’). + + -- Attribute: size + + Total size of memory blocks in bytes (‘int’). + + -- Attribute: traceback + + Traceback where the memory block was allocated, *note + Traceback: 3ff. instance. + + +File: python.info, Node: StatisticDiff, Next: Trace, Prev: Statistic, Up: API + +5.27.8.13 StatisticDiff +....................... + + -- Class: tracemalloc.StatisticDiff + + Statistic difference on memory allocations between an old and a new + *note Snapshot: 32ef. instance. + + *note Snapshot.compare_to(): 32ed. returns a list of *note + StatisticDiff: 3300. instances. See also the *note Statistic: + 3308. class. + + -- Attribute: count + + Number of memory blocks in the new snapshot (‘int’): ‘0’ if + the memory blocks have been released in the new snapshot. + + -- Attribute: count_diff + + Difference of number of memory blocks between the old and the + new snapshots (‘int’): ‘0’ if the memory blocks have been + allocated in the new snapshot. + + -- Attribute: size + + Total size of memory blocks in bytes in the new snapshot + (‘int’): ‘0’ if the memory blocks have been released in the + new snapshot. + + -- Attribute: size_diff + + Difference of total size of memory blocks in bytes between the + old and the new snapshots (‘int’): ‘0’ if the memory blocks + have been allocated in the new snapshot. + + -- Attribute: traceback + + Traceback where the memory blocks were allocated, *note + Traceback: 3ff. instance. + + +File: python.info, Node: Trace, Next: Traceback, Prev: StatisticDiff, Up: API + +5.27.8.14 Trace +............... + + -- Class: tracemalloc.Trace + + Trace of a memory block. + + The *note Snapshot.traces: 3307. attribute is a sequence of *note + Trace: 330b. instances. + + Changed in version 3.6: Added the *note domain: 3310. attribute. + + -- Attribute: domain + + Address space of a memory block (‘int’). Read-only property. + + tracemalloc uses the domain ‘0’ to trace memory allocations + made by Python. C extensions can use other domains to trace + other resources. + + -- Attribute: size + + Size of the memory block in bytes (‘int’). + + -- Attribute: traceback + + Traceback where the memory block was allocated, *note + Traceback: 3ff. instance. + + +File: python.info, Node: Traceback, Prev: Trace, Up: API + +5.27.8.15 Traceback +................... + + -- Class: tracemalloc.Traceback + + Sequence of *note Frame: 32fc. instances sorted from the oldest + frame to the most recent frame. + + A traceback contains at least ‘1’ frame. If the ‘tracemalloc’ + module failed to get a frame, the filename ‘"<unknown>"’ at line + number ‘0’ is used. + + When a snapshot is taken, tracebacks of traces are limited to *note + get_traceback_limit(): 32e9. frames. See the *note + take_snapshot(): 32ee. function. + + The *note Trace.traceback: 3312. attribute is an instance of *note + Traceback: 3ff. instance. + + Changed in version 3.7: Frames are now sorted from the oldest to + the most recent, instead of most recent to oldest. + + -- Method: format (limit=None, most_recent_first=False) + + Format the traceback as a list of lines with newlines. Use + the *note linecache: a8. module to retrieve lines from the + source code. If `limit' is set, format the `limit' most + recent frames if `limit' is positive. Otherwise, format the + ‘abs(limit)’ oldest frames. If `most_recent_first' is ‘True’, + the order of the formatted frames is reversed, returning the + most recent frame first instead of last. + + Similar to the *note traceback.format_tb(): 3314. function, + except that *note format(): 400. does not include newlines. + + Example: + + print("Traceback (most recent call first):") + for line in traceback: + print(line) + + Output: + + Traceback (most recent call first): + File "test.py", line 9 + obj = Object() + File "test.py", line 12 + tb = tracemalloc.get_object_traceback(f()) + + +File: python.info, Node: Software Packaging and Distribution, Next: Python Runtime Services, Prev: Debugging and Profiling, Up: The Python Standard Library + +5.28 Software Packaging and Distribution +======================================== + +These libraries help you with publishing and installing Python software. +While these modules are designed to work in conjunction with the Python +Package Index(1), they can also be used with a local index server, or +without any index server at all. + +* Menu: + +* distutils — Building and installing Python modules:: +* ensurepip — Bootstrapping the pip installer:: +* venv — Creation of virtual environments:: +* zipapp — Manage executable Python zip archives:: + + ---------- Footnotes ---------- + + (1) https://pypi.org + + +File: python.info, Node: distutils — Building and installing Python modules, Next: ensurepip — Bootstrapping the pip installer, Up: Software Packaging and Distribution + +5.28.1 ‘distutils’ — Building and installing Python modules +----------------------------------------------------------- + +__________________________________________________________________ + +The *note distutils: 39. package provides support for building and +installing additional modules into a Python installation. The new +modules may be either 100%-pure Python, or may be extension modules +written in C, or may be collections of Python packages which include +modules coded in both Python and C. + +Most Python users will `not' want to use this module directly, but +instead use the cross-version tools maintained by the Python Packaging +Authority. In particular, setuptools(1) is an enhanced alternative to +*note distutils: 39. that provides: + + * support for declaring project dependencies + + * additional mechanisms for configuring which files to include in + source releases (including plugins for integration with version + control systems) + + * the ability to declare project “entry points”, which can be used as + the basis for application plugin systems + + * the ability to automatically generate Windows command line + executables at installation time rather than needing to prebuild + them + + * consistent behaviour across all supported Python versions + +The recommended pip(2) installer runs all ‘setup.py’ scripts with +‘setuptools’, even if the script itself only imports ‘distutils’. Refer +to the Python Packaging User Guide(3) for more information. + +For the benefits of packaging tool authors and users seeking a deeper +understanding of the details of the current packaging and distribution +system, the legacy *note distutils: 39. based user documentation and API +reference remain available: + + * *note Installing Python Modules (Legacy version): 7f7. + + * *note Distributing Python Modules (Legacy version): 7f8. + + ---------- Footnotes ---------- + + (1) https://setuptools.readthedocs.io/en/latest/ + + (2) https://pip.pypa.io/ + + (3) https://packaging.python.org + + +File: python.info, Node: ensurepip — Bootstrapping the pip installer, Next: venv — Creation of virtual environments, Prev: distutils — Building and installing Python modules, Up: Software Packaging and Distribution + +5.28.2 ‘ensurepip’ — Bootstrapping the ‘pip’ installer +------------------------------------------------------ + +New in version 3.4. + +__________________________________________________________________ + +The *note ensurepip: 7a. package provides support for bootstrapping the +‘pip’ installer into an existing Python installation or virtual +environment. This bootstrapping approach reflects the fact that ‘pip’ +is an independent project with its own release cycle, and the latest +available stable version is bundled with maintenance and feature +releases of the CPython reference interpreter. + +In most cases, end users of Python shouldn’t need to invoke this module +directly (as ‘pip’ should be bootstrapped by default), but it may be +needed if installing ‘pip’ was skipped when installing Python (or when +creating a virtual environment) or after explicitly uninstalling ‘pip’. + + Note: This module `does not' access the internet. All of the + components needed to bootstrap ‘pip’ are included as internal parts + of the package. + +See also +........ + +*note Installing Python Modules: 7f5. + + The end user guide for installing Python packages + +PEP 453(1): Explicit bootstrapping of pip in Python installations + + The original rationale and specification for this module. + +* Menu: + +* Command line interface:: +* Module API:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0453 + + +File: python.info, Node: Command line interface, Next: Module API, Up: ensurepip — Bootstrapping the pip installer + +5.28.2.1 Command line interface +............................... + +The command line interface is invoked using the interpreter’s ‘-m’ +switch. + +The simplest possible invocation is: + + python -m ensurepip + +This invocation will install ‘pip’ if it is not already installed, but +otherwise does nothing. To ensure the installed version of ‘pip’ is at +least as recent as the one bundled with ‘ensurepip’, pass the +‘--upgrade’ option: + + python -m ensurepip --upgrade + +By default, ‘pip’ is installed into the current virtual environment (if +one is active) or into the system site packages (if there is no active +virtual environment). The installation location can be controlled +through two additional command line options: + + * ‘--root <dir>’: Installs ‘pip’ relative to the given root directory + rather than the root of the currently active virtual environment + (if any) or the default root for the current Python installation. + + * ‘--user’: Installs ‘pip’ into the user site packages directory + rather than globally for the current Python installation (this + option is not permitted inside an active virtual environment). + +By default, the scripts ‘pipX’ and ‘pipX.Y’ will be installed (where X.Y +stands for the version of Python used to invoke ‘ensurepip’). The +scripts installed can be controlled through two additional command line +options: + + * ‘--altinstall’: if an alternate installation is requested, the + ‘pipX’ script will `not' be installed. + + * ‘--default-pip’: if a “default pip” installation is requested, the + ‘pip’ script will be installed in addition to the two regular + scripts. + +Providing both of the script selection options will trigger an +exception. + + +File: python.info, Node: Module API, Prev: Command line interface, Up: ensurepip — Bootstrapping the pip installer + +5.28.2.2 Module API +................... + +*note ensurepip: 7a. exposes two functions for programmatic use: + + -- Function: ensurepip.version () + + Returns a string specifying the bundled version of pip that will be + installed when bootstrapping an environment. + + -- Function: ensurepip.bootstrap (root=None, upgrade=False, user=False, + altinstall=False, default_pip=False, verbosity=0) + + Bootstraps ‘pip’ into the current or designated environment. + + `root' specifies an alternative root directory to install relative + to. If `root' is ‘None’, then installation uses the default + install location for the current environment. + + `upgrade' indicates whether or not to upgrade an existing + installation of an earlier version of ‘pip’ to the bundled version. + + `user' indicates whether to use the user scheme rather than + installing globally. + + By default, the scripts ‘pipX’ and ‘pipX.Y’ will be installed + (where X.Y stands for the current version of Python). + + If `altinstall' is set, then ‘pipX’ will `not' be installed. + + If `default_pip' is set, then ‘pip’ will be installed in addition + to the two regular scripts. + + Setting both `altinstall' and `default_pip' will trigger *note + ValueError: 1fb. + + `verbosity' controls the level of output to *note sys.stdout: 30e. + from the bootstrapping operation. + + Raises an *note auditing event: fd1. ‘ensurepip.bootstrap’ with + argument ‘root’. + + Note: The bootstrapping process has side effects on both + ‘sys.path’ and ‘os.environ’. Invoking the command line + interface in a subprocess instead allows these side effects to + be avoided. + + Note: The bootstrapping process may install additional modules + required by ‘pip’, but other software should not assume those + dependencies will always be present by default (as the + dependencies may be removed in a future version of ‘pip’). + + +File: python.info, Node: venv — Creation of virtual environments, Next: zipapp — Manage executable Python zip archives, Prev: ensurepip — Bootstrapping the pip installer, Up: Software Packaging and Distribution + +5.28.3 ‘venv’ — Creation of virtual environments +------------------------------------------------ + +New in version 3.3. + +`Source code:' Lib/venv/(1) + +__________________________________________________________________ + +The *note venv: 125. module provides support for creating lightweight +“virtual environments” with their own site directories, optionally +isolated from system site directories. Each virtual environment has its +own Python binary (which matches the version of the binary that was used +to create this environment) and can have its own independent set of +installed Python packages in its site directories. + +See PEP 405(2) for more information about Python virtual environments. + +See also +........ + +Python Packaging User Guide: Creating and using virtual environments(3) + +* Menu: + +* Creating virtual environments:: +* API: API<2>. +* An example of extending EnvBuilder:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/venv/ + + (2) https://www.python.org/dev/peps/pep-0405 + + (3) +https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment + + +File: python.info, Node: Creating virtual environments, Next: API<2>, Up: venv — Creation of virtual environments + +5.28.3.1 Creating virtual environments +...................................... + +Creation of *note virtual environments: 3321. is done by executing the +command ‘venv’: + + python3 -m venv /path/to/new/virtual/environment + +Running this command creates the target directory (creating any parent +directories that don’t exist already) and places a ‘pyvenv.cfg’ file in +it with a ‘home’ key pointing to the Python installation from which the +command was run (a common name for the target directory is ‘.venv’). It +also creates a ‘bin’ (or ‘Scripts’ on Windows) subdirectory containing a +copy/symlink of the Python binary/binaries (as appropriate for the +platform or arguments used at environment creation time). It also +creates an (initially empty) ‘lib/pythonX.Y/site-packages’ subdirectory +(on Windows, this is ‘Lib\site-packages’). If an existing directory is +specified, it will be re-used. + +Deprecated since version 3.6: ‘pyvenv’ was the recommended tool for +creating virtual environments for Python 3.3 and 3.4, and is deprecated +in Python 3.6(1). + +Changed in version 3.5: The use of ‘venv’ is now recommended for +creating virtual environments. + +On Windows, invoke the ‘venv’ command as follows: + + c:\>c:\Python35\python -m venv c:\path\to\myenv + +Alternatively, if you configured the ‘PATH’ and ‘PATHEXT’ variables for +your *note Python installation: 2d9.: + + c:\>python -m venv c:\path\to\myenv + +The command, if run with ‘-h’, will show the available options: + + usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear] + [--upgrade] [--without-pip] [--prompt PROMPT] + ENV_DIR [ENV_DIR ...] + + Creates virtual Python environments in one or more target directories. + + positional arguments: + ENV_DIR A directory to create the environment in. + + optional arguments: + -h, --help show this help message and exit + --system-site-packages + Give the virtual environment access to the system + site-packages dir. + --symlinks Try to use symlinks rather than copies, when symlinks + are not the default for the platform. + --copies Try to use copies rather than symlinks, even when + symlinks are the default for the platform. + --clear Delete the contents of the environment directory if it + already exists, before environment creation. + --upgrade Upgrade the environment directory to use this version + of Python, assuming Python has been upgraded in-place. + --without-pip Skips installing or upgrading pip in the virtual + environment (pip is bootstrapped by default) + --prompt PROMPT Provides an alternative prompt prefix for this + environment. + + Once an environment has been created, you may wish to activate it, e.g. by + sourcing an activate script in its bin directory. + +Changed in version 3.4: Installs pip by default, added the +‘--without-pip’ and ‘--copies’ options + +Changed in version 3.4: In earlier versions, if the target directory +already existed, an error was raised, unless the ‘--clear’ or +‘--upgrade’ option was provided. + + Note: While symlinks are supported on Windows, they are not + recommended. Of particular note is that double-clicking + ‘python.exe’ in File Explorer will resolve the symlink eagerly and + ignore the virtual environment. + + Note: On Microsoft Windows, it may be required to enable the + ‘Activate.ps1’ script by setting the execution policy for the user. + You can do this by issuing the following PowerShell command: + + PS C:> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope + CurrentUser + + See About Execution Policies(2) for more information. + +The created ‘pyvenv.cfg’ file also includes the +‘include-system-site-packages’ key, set to ‘true’ if ‘venv’ is run with +the ‘--system-site-packages’ option, ‘false’ otherwise. + +Unless the ‘--without-pip’ option is given, *note ensurepip: 7a. will be +invoked to bootstrap ‘pip’ into the virtual environment. + +Multiple paths can be given to ‘venv’, in which case an identical +virtual environment will be created, according to the given options, at +each provided path. + +Once a virtual environment has been created, it can be “activated” using +a script in the virtual environment’s binary directory. The invocation +of the script is platform-specific (‘<venv>’ must be replaced by the +path of the directory containing the virtual environment): + +Platform Shell Command to activate virtual environment + +-------------------------------------------------------------------------------------- + +POSIX bash/zsh $ source <venv>/bin/activate + + + fish $ . <venv>/bin/activate.fish + + + csh/tcsh $ source <venv>/bin/activate.csh + + + PowerShell Core $ <venv>/bin/Activate.ps1 + + +Windows cmd.exe C:\> <venv>\Scripts\activate.bat + + + PowerShell PS C:\> <venv>\Scripts\Activate.ps1 + + +When a virtual environment is active, the ‘VIRTUAL_ENV’ environment +variable is set to the path of the virtual environment. This can be +used to check if one is running inside a virtual environment. + +You don’t specifically `need' to activate an environment; activation +just prepends the virtual environment’s binary directory to your path, +so that “python” invokes the virtual environment’s Python interpreter +and you can run installed scripts without having to use their full path. +However, all scripts installed in a virtual environment should be +runnable without activating it, and run with the virtual environment’s +Python automatically. + +You can deactivate a virtual environment by typing “deactivate” in your +shell. The exact mechanism is platform-specific and is an internal +implementation detail (typically a script or shell function will be +used). + +New in version 3.4: ‘fish’ and ‘csh’ activation scripts. + +New in version 3.8: PowerShell activation scripts installed under POSIX +for PowerShell Core support. + + Note: A virtual environment is a Python environment such that the + Python interpreter, libraries and scripts installed into it are + isolated from those installed in other virtual environments, and + (by default) any libraries installed in a “system” Python, i.e., + one which is installed as part of your operating system. + + A virtual environment is a directory tree which contains Python + executable files and other files which indicate that it is a + virtual environment. + + Common installation tools such as setuptools(3) and pip(4) work as + expected with virtual environments. In other words, when a virtual + environment is active, they install Python packages into the + virtual environment without needing to be told to do so explicitly. + + When a virtual environment is active (i.e., the virtual + environment’s Python interpreter is running), the attributes *note + sys.prefix: 3322. and *note sys.exec_prefix: 3323. point to the + base directory of the virtual environment, whereas *note + sys.base_prefix: 2d50. and *note sys.base_exec_prefix: 3324. point + to the non-virtual environment Python installation which was used + to create the virtual environment. If a virtual environment is not + active, then *note sys.prefix: 3322. is the same as *note + sys.base_prefix: 2d50. and *note sys.exec_prefix: 3323. is the same + as *note sys.base_exec_prefix: 3324. (they all point to a + non-virtual environment Python installation). + + When a virtual environment is active, any options that change the + installation path will be ignored from all *note distutils: 39. + configuration files to prevent projects being inadvertently + installed outside of the virtual environment. + + When working in a command shell, users can make a virtual + environment active by running an ‘activate’ script in the virtual + environment’s executables directory (the precise filename and + command to use the file is shell-dependent), which prepends the + virtual environment’s directory for executables to the ‘PATH’ + environment variable for the running shell. There should be no + need in other circumstances to activate a virtual environment; + scripts installed into virtual environments have a “shebang” line + which points to the virtual environment’s Python interpreter. This + means that the script will run with that interpreter regardless of + the value of ‘PATH’. On Windows, “shebang” line processing is + supported if you have the Python Launcher for Windows installed + (this was added to Python in 3.3 - see PEP 397(5) for more + details). Thus, double-clicking an installed script in a Windows + Explorer window should run the script with the correct interpreter + without there needing to be any reference to its virtual + environment in ‘PATH’. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/dev/whatsnew/3.6.html#deprecated-features + + (2) https://go.microsoft.com/fwlink/?LinkID=135170 + + (3) https://pypi.org/project/setuptools/ + + (4) https://pypi.org/project/pip/ + + (5) https://www.python.org/dev/peps/pep-0397 + + +File: python.info, Node: API<2>, Next: An example of extending EnvBuilder, Prev: Creating virtual environments, Up: venv — Creation of virtual environments + +5.28.3.2 API +............ + +The high-level method described above makes use of a simple API which +provides mechanisms for third-party virtual environment creators to +customize environment creation according to their needs, the *note +EnvBuilder: 908. class. + + -- Class: venv.EnvBuilder (system_site_packages=False, clear=False, + symlinks=False, upgrade=False, with_pip=False, prompt=None) + + The *note EnvBuilder: 908. class accepts the following keyword + arguments on instantiation: + + * ‘system_site_packages’ – a Boolean value indicating that the + system Python site-packages should be available to the + environment (defaults to ‘False’). + + * ‘clear’ – a Boolean value which, if true, will delete the + contents of any existing target directory, before creating the + environment. + + * ‘symlinks’ – a Boolean value indicating whether to attempt to + symlink the Python binary rather than copying. + + * ‘upgrade’ – a Boolean value which, if true, will upgrade an + existing environment with the running Python - for use when + that Python has been upgraded in-place (defaults to ‘False’). + + * ‘with_pip’ – a Boolean value which, if true, ensures pip is + installed in the virtual environment. This uses *note + ensurepip: 7a. with the ‘--default-pip’ option. + + * ‘prompt’ – a String to be used after virtual environment is + activated (defaults to ‘None’ which means directory name of + the environment would be used). + + Changed in version 3.4: Added the ‘with_pip’ parameter + + New in version 3.6: Added the ‘prompt’ parameter + + Creators of third-party virtual environment tools will be free to + use the provided *note EnvBuilder: 908. class as a base class. + + The returned env-builder is an object which has a method, ‘create’: + + -- Method: create (env_dir) + + Create a virtual environment by specifying the target + directory (absolute or relative to the current directory) + which is to contain the virtual environment. The ‘create’ + method will either create the environment in the specified + directory, or raise an appropriate exception. + + The ‘create’ method of the *note EnvBuilder: 908. class + illustrates the hooks available for subclass customization: + + def create(self, env_dir): + """ + Create a virtualized Python environment in a directory. + env_dir is the target directory to create an environment in. + """ + env_dir = os.path.abspath(env_dir) + context = self.ensure_directories(env_dir) + self.create_configuration(context) + self.setup_python(context) + self.setup_scripts(context) + self.post_setup(context) + + Each of the methods *note ensure_directories(): 3327, *note + create_configuration(): 3328, *note setup_python(): 3329, + *note setup_scripts(): 332a. and *note post_setup(): 332b. can + be overridden. + + -- Method: ensure_directories (env_dir) + + Creates the environment directory and all necessary + directories, and returns a context object. This is just a + holder for attributes (such as paths), for use by the other + methods. The directories are allowed to exist already, as + long as either ‘clear’ or ‘upgrade’ were specified to allow + operating on an existing environment directory. + + -- Method: create_configuration (context) + + Creates the ‘pyvenv.cfg’ configuration file in the + environment. + + -- Method: setup_python (context) + + Creates a copy or symlink to the Python executable in the + environment. On POSIX systems, if a specific executable + ‘python3.x’ was used, symlinks to ‘python’ and ‘python3’ will + be created pointing to that executable, unless files with + those names already exist. + + -- Method: setup_scripts (context) + + Installs activation scripts appropriate to the platform into + the virtual environment. + + -- Method: post_setup (context) + + A placeholder method which can be overridden in third party + implementations to pre-install packages in the virtual + environment or perform other post-creation steps. + + Changed in version 3.7.2: Windows now uses redirector scripts for + ‘python[w].exe’ instead of copying the actual binaries. In 3.7.2 + only *note setup_python(): 3329. does nothing unless running from a + build in the source tree. + + Changed in version 3.7.3: Windows copies the redirector scripts as + part of *note setup_python(): 3329. instead of *note + setup_scripts(): 332a. This was not the case in 3.7.2. When using + symlinks, the original executables will be linked. + + In addition, *note EnvBuilder: 908. provides this utility method + that can be called from *note setup_scripts(): 332a. or *note + post_setup(): 332b. in subclasses to assist in installing custom + scripts into the virtual environment. + + -- Method: install_scripts (context, path) + + `path' is the path to a directory that should contain + subdirectories “common”, “posix”, “nt”, each containing + scripts destined for the bin directory in the environment. + The contents of “common” and the directory corresponding to + *note os.name: 1bb0. are copied after some text replacement of + placeholders: + + * ‘__VENV_DIR__’ is replaced with the absolute path of the + environment directory. + + * ‘__VENV_NAME__’ is replaced with the environment name + (final path segment of environment directory). + + * ‘__VENV_PROMPT__’ is replaced with the prompt (the + environment name surrounded by parentheses and with a + following space) + + * ‘__VENV_BIN_NAME__’ is replaced with the name of the bin + directory (either ‘bin’ or ‘Scripts’). + + * ‘__VENV_PYTHON__’ is replaced with the absolute path of + the environment’s executable. + + The directories are allowed to exist (for when an existing + environment is being upgraded). + +There is also a module-level convenience function: + + -- Function: venv.create (env_dir, system_site_packages=False, + clear=False, symlinks=False, with_pip=False, prompt=None) + + Create an *note EnvBuilder: 908. with the given keyword arguments, + and call its *note create(): 3326. method with the `env_dir' + argument. + + New in version 3.3. + + Changed in version 3.4: Added the ‘with_pip’ parameter + + Changed in version 3.6: Added the ‘prompt’ parameter + + +File: python.info, Node: An example of extending EnvBuilder, Prev: API<2>, Up: venv — Creation of virtual environments + +5.28.3.3 An example of extending ‘EnvBuilder’ +............................................. + +The following script shows how to extend *note EnvBuilder: 908. by +implementing a subclass which installs setuptools and pip into a created +virtual environment: + + import os + import os.path + from subprocess import Popen, PIPE + import sys + from threading import Thread + from urllib.parse import urlparse + from urllib.request import urlretrieve + import venv + + class ExtendedEnvBuilder(venv.EnvBuilder): + """ + This builder installs setuptools and pip so that you can pip or + easy_install other packages into the created virtual environment. + + :param nodist: If true, setuptools and pip are not installed into the + created virtual environment. + :param nopip: If true, pip is not installed into the created + virtual environment. + :param progress: If setuptools or pip are installed, the progress of the + installation can be monitored by passing a progress + callable. If specified, it is called with two + arguments: a string indicating some progress, and a + context indicating where the string is coming from. + The context argument can have one of three values: + 'main', indicating that it is called from virtualize() + itself, and 'stdout' and 'stderr', which are obtained + by reading lines from the output streams of a subprocess + which is used to install the app. + + If a callable is not specified, default progress + information is output to sys.stderr. + """ + + def __init__(self, *args, **kwargs): + self.nodist = kwargs.pop('nodist', False) + self.nopip = kwargs.pop('nopip', False) + self.progress = kwargs.pop('progress', None) + self.verbose = kwargs.pop('verbose', False) + super().__init__(*args, **kwargs) + + def post_setup(self, context): + """ + Set up any packages which need to be pre-installed into the + virtual environment being created. + + :param context: The information for the virtual environment + creation request being processed. + """ + os.environ['VIRTUAL_ENV'] = context.env_dir + if not self.nodist: + self.install_setuptools(context) + # Can't install pip without setuptools + if not self.nopip and not self.nodist: + self.install_pip(context) + + def reader(self, stream, context): + """ + Read lines from a subprocess' output stream and either pass to a progress + callable (if specified) or write progress information to sys.stderr. + """ + progress = self.progress + while True: + s = stream.readline() + if not s: + break + if progress is not None: + progress(s, context) + else: + if not self.verbose: + sys.stderr.write('.') + else: + sys.stderr.write(s.decode('utf-8')) + sys.stderr.flush() + stream.close() + + def install_script(self, context, name, url): + _, _, path, _, _, _ = urlparse(url) + fn = os.path.split(path)[-1] + binpath = context.bin_path + distpath = os.path.join(binpath, fn) + # Download script into the virtual environment's binaries folder + urlretrieve(url, distpath) + progress = self.progress + if self.verbose: + term = '\n' + else: + term = '' + if progress is not None: + progress('Installing %s ...%s' % (name, term), 'main') + else: + sys.stderr.write('Installing %s ...%s' % (name, term)) + sys.stderr.flush() + # Install in the virtual environment + args = [context.env_exe, fn] + p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath) + t1 = Thread(target=self.reader, args=(p.stdout, 'stdout')) + t1.start() + t2 = Thread(target=self.reader, args=(p.stderr, 'stderr')) + t2.start() + p.wait() + t1.join() + t2.join() + if progress is not None: + progress('done.', 'main') + else: + sys.stderr.write('done.\n') + # Clean up - no longer needed + os.unlink(distpath) + + def install_setuptools(self, context): + """ + Install setuptools in the virtual environment. + + :param context: The information for the virtual environment + creation request being processed. + """ + url = 'https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py' + self.install_script(context, 'setuptools', url) + # clear up the setuptools archive which gets downloaded + pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz') + files = filter(pred, os.listdir(context.bin_path)) + for f in files: + f = os.path.join(context.bin_path, f) + os.unlink(f) + + def install_pip(self, context): + """ + Install pip in the virtual environment. + + :param context: The information for the virtual environment + creation request being processed. + """ + url = 'https://bootstrap.pypa.io/get-pip.py' + self.install_script(context, 'pip', url) + + def main(args=None): + compatible = True + if sys.version_info < (3, 3): + compatible = False + elif not hasattr(sys, 'base_prefix'): + compatible = False + if not compatible: + raise ValueError('This script is only for use with ' + 'Python 3.3 or later') + else: + import argparse + + parser = argparse.ArgumentParser(prog=__name__, + description='Creates virtual Python ' + 'environments in one or ' + 'more target ' + 'directories.') + parser.add_argument('dirs', metavar='ENV_DIR', nargs='+', + help='A directory in which to create the + 'virtual environment.') + parser.add_argument('--no-setuptools', default=False, + action='store_true', dest='nodist', + help="Don't install setuptools or pip in the " + "virtual environment.") + parser.add_argument('--no-pip', default=False, + action='store_true', dest='nopip', + help="Don't install pip in the virtual " + "environment.") + parser.add_argument('--system-site-packages', default=False, + action='store_true', dest='system_site', + help='Give the virtual environment access to the ' + 'system site-packages dir.') + if os.name == 'nt': + use_symlinks = False + else: + use_symlinks = True + parser.add_argument('--symlinks', default=use_symlinks, + action='store_true', dest='symlinks', + help='Try to use symlinks rather than copies, ' + 'when symlinks are not the default for ' + 'the platform.') + parser.add_argument('--clear', default=False, action='store_true', + dest='clear', help='Delete the contents of the ' + 'virtual environment ' + 'directory if it already ' + 'exists, before virtual ' + 'environment creation.') + parser.add_argument('--upgrade', default=False, action='store_true', + dest='upgrade', help='Upgrade the virtual ' + 'environment directory to ' + 'use this version of ' + 'Python, assuming Python ' + 'has been upgraded ' + 'in-place.') + parser.add_argument('--verbose', default=False, action='store_true', + dest='verbose', help='Display the output ' + 'from the scripts which ' + 'install setuptools and pip.') + options = parser.parse_args(args) + if options.upgrade and options.clear: + raise ValueError('you cannot supply --upgrade and --clear together.') + builder = ExtendedEnvBuilder(system_site_packages=options.system_site, + clear=options.clear, + symlinks=options.symlinks, + upgrade=options.upgrade, + nodist=options.nodist, + nopip=options.nopip, + verbose=options.verbose) + for d in options.dirs: + builder.create(d) + + if __name__ == '__main__': + rc = 1 + try: + main() + rc = 0 + except Exception as e: + print('Error: %s' % e, file=sys.stderr) + sys.exit(rc) + +This script is also available for download online(1). + + ---------- Footnotes ---------- + + (1) https://gist.github.com/vsajip/4673395 + + +File: python.info, Node: zipapp — Manage executable Python zip archives, Prev: venv — Creation of virtual environments, Up: Software Packaging and Distribution + +5.28.4 ‘zipapp’ — Manage executable Python zip archives +------------------------------------------------------- + +New in version 3.5. + +`Source code:' Lib/zipapp.py(1) + +__________________________________________________________________ + +This module provides tools to manage the creation of zip files +containing Python code, which can be *note executed directly by the +Python interpreter: fd0. The module provides both a *note Command-Line +Interface: 3331. and a *note Python API: 3332. + +* Menu: + +* Basic Example:: +* Command-Line Interface: Command-Line Interface<5>. +* Python API:: +* Examples: Examples<27>. +* Specifying the Interpreter:: +* Creating Standalone Applications with zipapp:: +* The Python Zip Application Archive Format:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/zipapp.py + + +File: python.info, Node: Basic Example, Next: Command-Line Interface<5>, Up: zipapp — Manage executable Python zip archives + +5.28.4.1 Basic Example +...................... + +The following example shows how the *note Command-Line Interface: 3331. +can be used to create an executable archive from a directory containing +Python code. When run, the archive will execute the ‘main’ function +from the module ‘myapp’ in the archive. + + $ python -m zipapp myapp -m "myapp:main" + $ python myapp.pyz + <output from myapp> + + +File: python.info, Node: Command-Line Interface<5>, Next: Python API, Prev: Basic Example, Up: zipapp — Manage executable Python zip archives + +5.28.4.2 Command-Line Interface +............................... + +When called as a program from the command line, the following form is +used: + + $ python -m zipapp source [options] + +If `source' is a directory, this will create an archive from the +contents of `source'. If `source' is a file, it should be an archive, +and it will be copied to the target archive (or the contents of its +shebang line will be displayed if the –info option is specified). + +The following options are understood: + + -- Program Option: -o <output>, --output=<output> + + Write the output to a file named `output'. If this option is not + specified, the output filename will be the same as the input + `source', with the extension ‘.pyz’ added. If an explicit filename + is given, it is used as is (so a ‘.pyz’ extension should be + included if required). + + An output filename must be specified if the `source' is an archive + (and in that case, `output' must not be the same as `source'). + + -- Program Option: -p <interpreter>, --python=<interpreter> + + Add a ‘#!’ line to the archive specifying `interpreter' as the + command to run. Also, on POSIX, make the archive executable. The + default is to write no ‘#!’ line, and not make the file executable. + + -- Program Option: -m <mainfn>, --main=<mainfn> + + Write a ‘__main__.py’ file to the archive that executes `mainfn'. + The `mainfn' argument should have the form “pkg.mod:fn”, where + “pkg.mod” is a package/module in the archive, and “fn” is a + callable in the given module. The ‘__main__.py’ file will execute + that callable. + + *note –main: 3337. cannot be specified when copying an archive. + + -- Program Option: -c, --compress + + Compress files with the deflate method, reducing the size of the + output file. By default, files are stored uncompressed in the + archive. + + *note –compress: 3338. has no effect when copying an archive. + + New in version 3.7. + + -- Program Option: --info + + Display the interpreter embedded in the archive, for diagnostic + purposes. In this case, any other options are ignored and SOURCE + must be an archive, not a directory. + + -- Program Option: -h, --help + + Print a short usage message and exit. + + +File: python.info, Node: Python API, Next: Examples<27>, Prev: Command-Line Interface<5>, Up: zipapp — Manage executable Python zip archives + +5.28.4.3 Python API +................... + +The module defines two convenience functions: + + -- Function: zipapp.create_archive (source, target=None, + interpreter=None, main=None, filter=None, compressed=False) + + Create an application archive from `source'. The source can be any + of the following: + + * The name of a directory, or a *note path-like object: 36a. + referring to a directory, in which case a new application + archive will be created from the content of that directory. + + * The name of an existing application archive file, or a *note + path-like object: 36a. referring to such a file, in which case + the file is copied to the target (modifying it to reflect the + value given for the `interpreter' argument). The file name + should include the ‘.pyz’ extension, if required. + + * A file object open for reading in bytes mode. The content of + the file should be an application archive, and the file object + is assumed to be positioned at the start of the archive. + + The `target' argument determines where the resulting archive will + be written: + + * If it is the name of a file, or a *note path-like object: 36a, + the archive will be written to that file. + + * If it is an open file object, the archive will be written to + that file object, which must be open for writing in bytes + mode. + + * If the target is omitted (or ‘None’), the source must be a + directory and the target will be a file with the same name as + the source, with a ‘.pyz’ extension added. + + The `interpreter' argument specifies the name of the Python + interpreter with which the archive will be executed. It is written + as a “shebang” line at the start of the archive. On POSIX, this + will be interpreted by the OS, and on Windows it will be handled by + the Python launcher. Omitting the `interpreter' results in no + shebang line being written. If an interpreter is specified, and + the target is a filename, the executable bit of the target file + will be set. + + The `main' argument specifies the name of a callable which will be + used as the main program for the archive. It can only be specified + if the source is a directory, and the source does not already + contain a ‘__main__.py’ file. The `main' argument should take the + form “pkg.module:callable” and the archive will be run by importing + “pkg.module” and executing the given callable with no arguments. + It is an error to omit `main' if the source is a directory and does + not contain a ‘__main__.py’ file, as otherwise the resulting + archive would not be executable. + + The optional `filter' argument specifies a callback function that + is passed a Path object representing the path to the file being + added (relative to the source directory). It should return ‘True’ + if the file is to be added. + + The optional `compressed' argument determines whether files are + compressed. If set to ‘True’, files in the archive are compressed + with the deflate method; otherwise, files are stored uncompressed. + This argument has no effect when copying an existing archive. + + If a file object is specified for `source' or `target', it is the + caller’s responsibility to close it after calling create_archive. + + When copying an existing archive, file objects supplied only need + ‘read’ and ‘readline’, or ‘write’ methods. When creating an + archive from a directory, if the target is a file object it will be + passed to the ‘zipfile.ZipFile’ class, and must supply the methods + needed by that class. + + New in version 3.7: Added the `filter' and `compressed' arguments. + + -- Function: zipapp.get_interpreter (archive) + + Return the interpreter specified in the ‘#!’ line at the start of + the archive. If there is no ‘#!’ line, return *note None: 157. + The `archive' argument can be a filename or a file-like object open + for reading in bytes mode. It is assumed to be at the start of the + archive. + + +File: python.info, Node: Examples<27>, Next: Specifying the Interpreter, Prev: Python API, Up: zipapp — Manage executable Python zip archives + +5.28.4.4 Examples +................. + +Pack up a directory into an archive, and run it. + + $ python -m zipapp myapp + $ python myapp.pyz + <output from myapp> + +The same can be done using the *note create_archive(): 41d. function: + + >>> import zipapp + >>> zipapp.create_archive('myapp', 'myapp.pyz') + +To make the application directly executable on POSIX, specify an +interpreter to use. + + $ python -m zipapp myapp -p "/usr/bin/env python" + $ ./myapp.pyz + <output from myapp> + +To replace the shebang line on an existing archive, create a modified +archive using the *note create_archive(): 41d. function: + + >>> import zipapp + >>> zipapp.create_archive('old_archive.pyz', 'new_archive.pyz', '/usr/bin/python3') + +To update the file in place, do the replacement in memory using a +‘BytesIO’ object, and then overwrite the source afterwards. Note that +there is a risk when overwriting a file in place that an error will +result in the loss of the original file. This code does not protect +against such errors, but production code should do so. Also, this +method will only work if the archive fits in memory: + + >>> import zipapp + >>> import io + >>> temp = io.BytesIO() + >>> zipapp.create_archive('myapp.pyz', temp, '/usr/bin/python2') + >>> with open('myapp.pyz', 'wb') as f: + >>> f.write(temp.getvalue()) + + +File: python.info, Node: Specifying the Interpreter, Next: Creating Standalone Applications with zipapp, Prev: Examples<27>, Up: zipapp — Manage executable Python zip archives + +5.28.4.5 Specifying the Interpreter +................................... + +Note that if you specify an interpreter and then distribute your +application archive, you need to ensure that the interpreter used is +portable. The Python launcher for Windows supports most common forms of +POSIX ‘#!’ line, but there are other issues to consider: + + * If you use “/usr/bin/env python” (or other forms of the “python” + command, such as “/usr/bin/python”), you need to consider that your + users may have either Python 2 or Python 3 as their default, and + write your code to work under both versions. + + * If you use an explicit version, for example “/usr/bin/env python3” + your application will not work for users who do not have that + version. (This may be what you want if you have not made your code + Python 2 compatible). + + * There is no way to say “python X.Y or later”, so be careful of + using an exact version like “/usr/bin/env python3.4” as you will + need to change your shebang line for users of Python 3.5, for + example. + +Typically, you should use an “/usr/bin/env python2” or “/usr/bin/env +python3”, depending on whether your code is written for Python 2 or 3. + + +File: python.info, Node: Creating Standalone Applications with zipapp, Next: The Python Zip Application Archive Format, Prev: Specifying the Interpreter, Up: zipapp — Manage executable Python zip archives + +5.28.4.6 Creating Standalone Applications with zipapp +..................................................... + +Using the *note zipapp: 141. module, it is possible to create +self-contained Python programs, which can be distributed to end users +who only need to have a suitable version of Python installed on their +system. The key to doing this is to bundle all of the application’s +dependencies into the archive, along with the application code. + +The steps to create a standalone archive are as follows: + + 1. Create your application in a directory as normal, so you have a + ‘myapp’ directory containing a ‘__main__.py’ file, and any + supporting application code. + + 2. Install all of your application’s dependencies into the ‘myapp’ + directory, using pip: + + $ python -m pip install -r requirements.txt --target myapp + + (this assumes you have your project requirements in a + ‘requirements.txt’ file - if not, you can just list the + dependencies manually on the pip command line). + + 3. Optionally, delete the ‘.dist-info’ directories created by pip in + the ‘myapp’ directory. These hold metadata for pip to manage the + packages, and as you won’t be making any further use of pip they + aren’t required - although it won’t do any harm if you leave them. + + 4. Package the application using: + + $ python -m zipapp -p "interpreter" myapp + +This will produce a standalone executable, which can be run on any +machine with the appropriate interpreter available. See *note +Specifying the Interpreter: 3340. for details. It can be shipped to +users as a single file. + +On Unix, the ‘myapp.pyz’ file is executable as it stands. You can +rename the file to remove the ‘.pyz’ extension if you prefer a “plain” +command name. On Windows, the ‘myapp.pyz[w]’ file is executable by +virtue of the fact that the Python interpreter registers the ‘.pyz’ and +‘.pyzw’ file extensions when installed. + +* Menu: + +* Making a Windows executable:: +* Caveats:: + + +File: python.info, Node: Making a Windows executable, Next: Caveats, Up: Creating Standalone Applications with zipapp + +5.28.4.7 Making a Windows executable +.................................... + +On Windows, registration of the ‘.pyz’ extension is optional, and +furthermore, there are certain places that don’t recognise registered +extensions “transparently” (the simplest example is that +‘subprocess.run(['myapp'])’ won’t find your application - you need to +explicitly specify the extension). + +On Windows, therefore, it is often preferable to create an executable +from the zipapp. This is relatively easy, although it does require a C +compiler. The basic approach relies on the fact that zipfiles can have +arbitrary data prepended, and Windows exe files can have arbitrary data +appended. So by creating a suitable launcher and tacking the ‘.pyz’ +file onto the end of it, you end up with a single-file executable that +runs your application. + +A suitable launcher can be as simple as the following: + + #define Py_LIMITED_API 1 + #include "Python.h" + + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + + #ifdef WINDOWS + int WINAPI wWinMain( + HINSTANCE hInstance, /* handle to current instance */ + HINSTANCE hPrevInstance, /* handle to previous instance */ + LPWSTR lpCmdLine, /* pointer to command line */ + int nCmdShow /* show state of window */ + ) + #else + int wmain() + #endif + { + wchar_t **myargv = _alloca((__argc + 1) * sizeof(wchar_t*)); + myargv[0] = __wargv[0]; + memcpy(myargv + 1, __wargv, __argc * sizeof(wchar_t *)); + return Py_Main(__argc+1, myargv); + } + +If you define the ‘WINDOWS’ preprocessor symbol, this will generate a +GUI executable, and without it, a console executable. + +To compile the executable, you can either just use the standard MSVC +command line tools, or you can take advantage of the fact that distutils +knows how to compile Python source: + + >>> from distutils.ccompiler import new_compiler + >>> import distutils.sysconfig + >>> import sys + >>> import os + >>> from pathlib import Path + + >>> def compile(src): + >>> src = Path(src) + >>> cc = new_compiler() + >>> exe = src.stem + >>> cc.add_include_dir(distutils.sysconfig.get_python_inc()) + >>> cc.add_library_dir(os.path.join(sys.base_exec_prefix, 'libs')) + >>> # First the CLI executable + >>> objs = cc.compile([str(src)]) + >>> cc.link_executable(objs, exe) + >>> # Now the GUI executable + >>> cc.define_macro('WINDOWS') + >>> objs = cc.compile([str(src)]) + >>> cc.link_executable(objs, exe + 'w') + + >>> if __name__ == "__main__": + >>> compile("zastub.c") + +The resulting launcher uses the “Limited ABI”, so it will run unchanged +with any version of Python 3.x. All it needs is for Python +(‘python3.dll’) to be on the user’s ‘PATH’. + +For a fully standalone distribution, you can distribute the launcher +with your application appended, bundled with the Python “embedded” +distribution. This will run on any PC with the appropriate architecture +(32 bit or 64 bit). + + +File: python.info, Node: Caveats, Prev: Making a Windows executable, Up: Creating Standalone Applications with zipapp + +5.28.4.8 Caveats +................ + +There are some limitations to the process of bundling your application +into a single file. In most, if not all, cases they can be addressed +without needing major changes to your application. + + 1. If your application depends on a package that includes a C + extension, that package cannot be run from a zip file (this is an + OS limitation, as executable code must be present in the filesystem + for the OS loader to load it). In this case, you can exclude that + dependency from the zipfile, and either require your users to have + it installed, or ship it alongside your zipfile and add code to + your ‘__main__.py’ to include the directory containing the unzipped + module in ‘sys.path’. In this case, you will need to make sure to + ship appropriate binaries for your target architecture(s) (and + potentially pick the correct version to add to ‘sys.path’ at + runtime, based on the user’s machine). + + 2. If you are shipping a Windows executable as described above, you + either need to ensure that your users have ‘python3.dll’ on their + PATH (which is not the default behaviour of the installer) or you + should bundle your application with the embedded distribution. + + 3. The suggested launcher above uses the Python embedding API. This + means that in your application, ‘sys.executable’ will be your + application, and `not' a conventional Python interpreter. Your + code and its dependencies need to be prepared for this possibility. + For example, if your application uses the *note multiprocessing: + b7. module, it will need to call *note + multiprocessing.set_executable(): 20b7. to let the module know + where to find the standard Python interpreter. + + +File: python.info, Node: The Python Zip Application Archive Format, Prev: Creating Standalone Applications with zipapp, Up: zipapp — Manage executable Python zip archives + +5.28.4.9 The Python Zip Application Archive Format +.................................................. + +Python has been able to execute zip files which contain a ‘__main__.py’ +file since version 2.6. In order to be executed by Python, an +application archive simply has to be a standard zip file containing a +‘__main__.py’ file which will be run as the entry point for the +application. As usual for any Python script, the parent of the script +(in this case the zip file) will be placed on *note sys.path: 488. and +thus further modules can be imported from the zip file. + +The zip file format allows arbitrary data to be prepended to a zip file. +The zip application format uses this ability to prepend a standard POSIX +“shebang” line to the file (‘#!/path/to/interpreter’). + +Formally, the Python zip application format is therefore: + + 1. An optional shebang line, containing the characters ‘b'#!'’ + followed by an interpreter name, and then a newline (‘b'\n'’) + character. The interpreter name can be anything acceptable to the + OS “shebang” processing, or the Python launcher on Windows. The + interpreter should be encoded in UTF-8 on Windows, and in *note + sys.getfilesystemencoding(): 4f6. on POSIX. + + 2. Standard zipfile data, as generated by the *note zipfile: 142. + module. The zipfile content `must' include a file called + ‘__main__.py’ (which must be in the “root” of the zipfile - i.e., + it cannot be in a subdirectory). The zipfile data can be + compressed or uncompressed. + +If an application archive has a shebang line, it may have the executable +bit set on POSIX systems, to allow it to be executed directly. + +There is no requirement that the tools in this module are used to create +application archives - the module is a convenience, but archives in the +above format created by any means are acceptable to Python. + + +File: python.info, Node: Python Runtime Services, Next: Custom Python Interpreters, Prev: Software Packaging and Distribution, Up: The Python Standard Library + +5.29 Python Runtime Services +============================ + +The modules described in this chapter provide a wide range of services +related to the Python interpreter and its interaction with its +environment. Here’s an overview: + +* Menu: + +* sys — System-specific parameters and functions:: +* sysconfig — Provide access to Python’s configuration information:: +* builtins — Built-in objects:: +* __main__ — Top-level script environment:: +* warnings — Warning control:: +* dataclasses — Data Classes:: +* contextlib — Utilities for with-statement contexts:: +* abc — Abstract Base Classes:: +* atexit — Exit handlers:: +* traceback — Print or retrieve a stack traceback:: +* __future__ — Future statement definitions:: +* gc — Garbage Collector interface:: +* inspect — Inspect live objects:: +* site — Site-specific configuration hook:: + + +File: python.info, Node: sys — System-specific parameters and functions, Next: sysconfig — Provide access to Python’s configuration information, Up: Python Runtime Services + +5.29.1 ‘sys’ — System-specific parameters and functions +------------------------------------------------------- + +__________________________________________________________________ + +This module provides access to some variables used or maintained by the +interpreter and to functions that interact strongly with the +interpreter. It is always available. + + -- Data: sys.abiflags + + On POSIX systems where Python was built with the standard + ‘configure’ script, this contains the ABI flags as specified by PEP + 3149(1). + + Changed in version 3.8: Default flags became an empty string (‘m’ + flag for pymalloc has been removed). + + New in version 3.2. + + -- Function: sys.addaudithook (hook) + + Append the callable `hook' to the list of active auditing hooks for + the current interpreter. + + When an auditing event is raised through the *note sys.audit(): + 31ea. function, each hook will be called in the order it was added + with the event name and the tuple of arguments. Native hooks added + by *note PySys_AddAuditHook(): 31ed. are called first, followed by + hooks added in the current interpreter. Hooks can then log the + event, raise an exception to abort the operation, or terminate the + process entirely. + + Calling *note sys.addaudithook(): 31ec. will itself raise an + auditing event named ‘sys.addaudithook’ with no arguments. If any + existing hooks raise an exception derived from *note RuntimeError: + 2ba, the new hook will not be added and the exception suppressed. + As a result, callers cannot assume that their hook has been added + unless they control all existing hooks. + + See the *note audit events table: 31e7. for all events raised by + CPython, and PEP 578(2) for the original design discussion. + + New in version 3.8. + + Changed in version 3.8.1: Exceptions derived from *note Exception: + 1a9. but not *note RuntimeError: 2ba. are no longer suppressed. + + `CPython implementation detail:' When tracing is enabled (see *note + settrace(): e3e.), Python hooks are only traced if the callable has + a ‘__cantrace__’ member that is set to a true value. Otherwise, + trace functions will skip the hook. + + -- Data: sys.argv + + The list of command line arguments passed to a Python script. + ‘argv[0]’ is the script name (it is operating system dependent + whether this is a full pathname or not). If the command was + executed using the *note -c: e9e. command line option to the + interpreter, ‘argv[0]’ is set to the string ‘'-c'’. If no script + name was passed to the Python interpreter, ‘argv[0]’ is the empty + string. + + To loop over the standard input, or the list of files given on the + command line, see the *note fileinput: 80. module. + + Note: On Unix, command line arguments are passed by bytes from + OS. Python decodes them with filesystem encoding and + “surrogateescape” error handler. When you need original + bytes, you can get it by ‘[os.fsencode(arg) for arg in + sys.argv]’. + -- Function: sys.audit (event, *args) + + Raise an auditing event and trigger any active auditing hooks. + `event' is a string identifying the event, and `args' may contain + optional arguments with more information about the event. The + number and types of arguments for a given event are considered a + public and stable API and should not be modified between releases. + + For example, one auditing event is named ‘os.chdir’. This event + has one argument called `path' that will contain the requested new + working directory. + + *note sys.audit(): 31ea. will call the existing auditing hooks, + passing the event name and arguments, and will re-raise the first + exception from any hook. In general, if an exception is raised, it + should not be handled and the process should be terminated as + quickly as possible. This allows hook implementations to decide + how to respond to particular events: they can merely log the event + or abort the operation by raising an exception. + + Hooks are added using the *note sys.addaudithook(): 31ec. or *note + PySys_AddAuditHook(): 31ed. functions. + + The native equivalent of this function is *note PySys_Audit(): + 31eb. Using the native function is preferred when possible. + + See the *note audit events table: 31e7. for all events raised by + CPython. + + New in version 3.8. + + -- Data: sys.base_exec_prefix + + Set during Python startup, before ‘site.py’ is run, to the same + value as *note exec_prefix: 3323. If not running in a *note + virtual environment: 3321, the values will stay the same; if + ‘site.py’ finds that a virtual environment is in use, the values of + *note prefix: 3322. and *note exec_prefix: 3323. will be changed to + point to the virtual environment, whereas *note base_prefix: 2d50. + and *note base_exec_prefix: 3324. will remain pointing to the base + Python installation (the one which the virtual environment was + created from). + + New in version 3.3. + + -- Data: sys.base_prefix + + Set during Python startup, before ‘site.py’ is run, to the same + value as *note prefix: 3322. If not running in a *note virtual + environment: 3321, the values will stay the same; if ‘site.py’ + finds that a virtual environment is in use, the values of *note + prefix: 3322. and *note exec_prefix: 3323. will be changed to point + to the virtual environment, whereas *note base_prefix: 2d50. and + *note base_exec_prefix: 3324. will remain pointing to the base + Python installation (the one which the virtual environment was + created from). + + New in version 3.3. + + -- Data: sys.byteorder + + An indicator of the native byte order. This will have the value + ‘'big'’ on big-endian (most-significant byte first) platforms, and + ‘'little'’ on little-endian (least-significant byte first) + platforms. + + -- Data: sys.builtin_module_names + + A tuple of strings giving the names of all modules that are + compiled into this Python interpreter. (This information is not + available in any other way — ‘modules.keys()’ only lists the + imported modules.) + + -- Function: sys.call_tracing (func, args) + + Call ‘func(*args)’, while tracing is enabled. The tracing state is + saved, and restored afterwards. This is intended to be called from + a debugger from a checkpoint, to recursively debug some other code. + + -- Data: sys.copyright + + A string containing the copyright pertaining to the Python + interpreter. + + -- Function: sys._clear_type_cache () + + Clear the internal type cache. The type cache is used to speed up + attribute and method lookups. Use the function `only' to drop + unnecessary references during reference leak debugging. + + This function should be used for internal and specialized purposes + only. + + -- Function: sys._current_frames () + + Return a dictionary mapping each thread’s identifier to the topmost + stack frame currently active in that thread at the time the + function is called. Note that functions in the *note traceback: + 113. module can build the call stack given such a frame. + + This is most useful for debugging deadlock: this function does not + require the deadlocked threads’ cooperation, and such threads’ call + stacks are frozen for as long as they remain deadlocked. The frame + returned for a non-deadlocked thread may bear no relationship to + that thread’s current activity by the time calling code examines + the frame. + + This function should be used for internal and specialized purposes + only. + + Raises an *note auditing event: fd1. ‘sys._current_frames’ with no + arguments. + + -- Function: sys.breakpointhook () + + This hook function is called by built-in *note breakpoint(): 2f9. + By default, it drops you into the *note pdb: c9. debugger, but it + can be set to any other function so that you can choose which + debugger gets used. + + The signature of this function is dependent on what it calls. For + example, the default binding (e.g. ‘pdb.set_trace()’) expects no + arguments, but you might bind it to a function that expects + additional arguments (positional and/or keyword). The built-in + ‘breakpoint()’ function passes its ‘*args’ and ‘**kws’ straight + through. Whatever ‘breakpointhooks()’ returns is returned from + ‘breakpoint()’. + + The default implementation first consults the environment variable + *note PYTHONBREAKPOINT: 314. If that is set to ‘"0"’ then this + function returns immediately; i.e. it is a no-op. If the + environment variable is not set, or is set to the empty string, + ‘pdb.set_trace()’ is called. Otherwise this variable should name a + function to run, using Python’s dotted-import nomenclature, e.g. + ‘package.subpackage.module.function’. In this case, + ‘package.subpackage.module’ would be imported and the resulting + module must have a callable named ‘function()’. This is run, + passing in ‘*args’ and ‘**kws’, and whatever ‘function()’ returns, + ‘sys.breakpointhook()’ returns to the built-in *note breakpoint(): + 2f9. function. + + Note that if anything goes wrong while importing the callable named + by *note PYTHONBREAKPOINT: 314, a *note RuntimeWarning: 2c0. is + reported and the breakpoint is ignored. + + Also note that if ‘sys.breakpointhook()’ is overridden + programmatically, *note PYTHONBREAKPOINT: 314. is `not' consulted. + + New in version 3.7. + + -- Function: sys._debugmallocstats () + + Print low-level information to stderr about the state of CPython’s + memory allocator. + + If Python is configured –with-pydebug, it also performs some + expensive internal consistency checks. + + New in version 3.3. + + `CPython implementation detail:' This function is specific to + CPython. The exact output format is not defined here, and may + change. + + -- Data: sys.dllhandle + + Integer specifying the handle of the Python DLL. + + *note Availability: ffb.: Windows. + + -- Function: sys.displayhook (value) + + If `value' is not ‘None’, this function prints ‘repr(value)’ to + ‘sys.stdout’, and saves `value' in ‘builtins._’. If ‘repr(value)’ + is not encodable to ‘sys.stdout.encoding’ with ‘sys.stdout.errors’ + error handler (which is probably ‘'strict'’), encode it to + ‘sys.stdout.encoding’ with ‘'backslashreplace'’ error handler. + + ‘sys.displayhook’ is called on the result of evaluating an *note + expression: 3350. entered in an interactive Python session. The + display of these values can be customized by assigning another + one-argument function to ‘sys.displayhook’. + + Pseudo-code: + + def displayhook(value): + if value is None: + return + # Set '_' to None to avoid recursion + builtins._ = None + text = repr(value) + try: + sys.stdout.write(text) + except UnicodeEncodeError: + bytes = text.encode(sys.stdout.encoding, 'backslashreplace') + if hasattr(sys.stdout, 'buffer'): + sys.stdout.buffer.write(bytes) + else: + text = bytes.decode(sys.stdout.encoding, 'strict') + sys.stdout.write(text) + sys.stdout.write("\n") + builtins._ = value + + Changed in version 3.2: Use ‘'backslashreplace'’ error handler on + *note UnicodeEncodeError: 1fc. + + -- Data: sys.dont_write_bytecode + + If this is true, Python won’t try to write ‘.pyc’ files on the + import of source modules. This value is initially set to ‘True’ or + ‘False’ depending on the *note -B: d38. command line option and the + *note PYTHONDONTWRITEBYTECODE: d39. environment variable, but you + can set it yourself to control bytecode file generation. + + -- Data: sys.pycache_prefix + + If this is set (not ‘None’), Python will write bytecode-cache + ‘.pyc’ files to (and read them from) a parallel directory tree + rooted at this directory, rather than from ‘__pycache__’ + directories in the source code tree. Any ‘__pycache__’ directories + in the source code tree will be ignored and new ‘.pyc’ files + written within the pycache prefix. Thus if you use *note + compileall: 21. as a pre-build step, you must ensure you run it + with the same pycache prefix (if any) that you will use at runtime. + + A relative path is interpreted relative to the current working + directory. + + This value is initially set based on the value of the *note -X: + 155. ‘pycache_prefix=PATH’ command-line option or the *note + PYTHONPYCACHEPREFIX: 154. environment variable (command-line takes + precedence). If neither are set, it is ‘None’. + + New in version 3.8. + + -- Function: sys.excepthook (type, value, traceback) + + This function prints out a given traceback and exception to + ‘sys.stderr’. + + When an exception is raised and uncaught, the interpreter calls + ‘sys.excepthook’ with three arguments, the exception class, + exception instance, and a traceback object. In an interactive + session this happens just before control is returned to the prompt; + in a Python program this happens just before the program exits. + The handling of such top-level exceptions can be customized by + assigning another three-argument function to ‘sys.excepthook’. + + Raise an auditing event ‘sys.excepthook’ with arguments ‘hook’, + ‘type’, ‘value’, ‘traceback’ when an uncaught exception occurs. If + no hook has been set, ‘hook’ may be ‘None’. If any hook raises an + exception derived from *note RuntimeError: 2ba. the call to the + hook will be suppressed. Otherwise, the audit hook exception will + be reported as unraisable and ‘sys.excepthook’ will be called. + + See also + ........ + + The *note sys.unraisablehook(): 22d. function handles unraisable + exceptions and the *note threading.excepthook(): 231. function + handles exception raised by *note threading.Thread.run(): 232. + + -- Data: sys.__breakpointhook__ + -- Data: sys.__displayhook__ + -- Data: sys.__excepthook__ + -- Data: sys.__unraisablehook__ + + These objects contain the original values of ‘breakpointhook’, + ‘displayhook’, ‘excepthook’, and ‘unraisablehook’ at the start of + the program. They are saved so that ‘breakpointhook’, + ‘displayhook’ and ‘excepthook’, ‘unraisablehook’ can be restored in + case they happen to get replaced with broken or alternative + objects. + + New in version 3.7: __breakpointhook__ + + New in version 3.8: __unraisablehook__ + + -- Function: sys.exc_info () + + This function returns a tuple of three values that give information + about the exception that is currently being handled. The + information returned is specific both to the current thread and to + the current stack frame. If the current stack frame is not + handling an exception, the information is taken from the calling + stack frame, or its caller, and so on until a stack frame is found + that is handling an exception. Here, “handling an exception” is + defined as “executing an except clause.” For any stack frame, only + information about the exception being currently handled is + accessible. + + If no exception is being handled anywhere on the stack, a tuple + containing three ‘None’ values is returned. Otherwise, the values + returned are ‘(type, value, traceback)’. Their meaning is: `type' + gets the type of the exception being handled (a subclass of *note + BaseException: 1a8.); `value' gets the exception instance (an + instance of the exception type); `traceback' gets a *note traceback + object: 336. which encapsulates the call stack at the point where + the exception originally occurred. + + -- Data: sys.exec_prefix + + A string giving the site-specific directory prefix where the + platform-dependent Python files are installed; by default, this is + also ‘'/usr/local'’. This can be set at build time with the + ‘--exec-prefix’ argument to the ‘configure’ script. Specifically, + all configuration files (e.g. the ‘pyconfig.h’ header file) are + installed in the directory ‘`exec_prefix'/lib/python`X.Y'/config’, + and shared library modules are installed in + ‘`exec_prefix'/lib/python`X.Y'/lib-dynload’, where `X.Y' is the + version number of Python, for example ‘3.2’. + + Note: If a *note virtual environment: 3321. is in effect, this + value will be changed in ‘site.py’ to point to the virtual + environment. The value for the Python installation will still + be available, via *note base_exec_prefix: 3324. + + -- Data: sys.executable + + A string giving the absolute path of the executable binary for the + Python interpreter, on systems where this makes sense. If Python + is unable to retrieve the real path to its executable, *note + sys.executable: 274. will be an empty string or ‘None’. + + -- Function: sys.exit ([arg]) + + Exit from Python. This is implemented by raising the *note + SystemExit: 5fc. exception, so cleanup actions specified by finally + clauses of *note try: d72. statements are honored, and it is + possible to intercept the exit attempt at an outer level. + + The optional argument `arg' can be an integer giving the exit + status (defaulting to zero), or another type of object. If it is + an integer, zero is considered “successful termination” and any + nonzero value is considered “abnormal termination” by shells and + the like. Most systems require it to be in the range 0–127, and + produce undefined results otherwise. Some systems have a + convention for assigning specific meanings to specific exit codes, + but these are generally underdeveloped; Unix programs generally use + 2 for command line syntax errors and 1 for all other kind of + errors. If another type of object is passed, ‘None’ is equivalent + to passing zero, and any other object is printed to *note stderr: + 30f. and results in an exit code of 1. In particular, + ‘sys.exit("some error message")’ is a quick way to exit a program + when an error occurs. + + Since *note exit(): 12b9. ultimately “only” raises an exception, it + will only exit the process when called from the main thread, and + the exception is not intercepted. + + Changed in version 3.6: If an error occurs in the cleanup after the + Python interpreter has caught *note SystemExit: 5fc. (such as an + error flushing buffered data in the standard streams), the exit + status is changed to 120. + + -- Data: sys.flags + + The *note named tuple: aa3. `flags' exposes the status of command + line flags. The attributes are read only. + + attribute flag + + -------------------------------------------------------------------- + + ‘debug’ *note -d: fda. + + + *note inspect: a0. *note -i: e1f. + + + ‘interactive’ *note -i: e1f. + + + ‘isolated’ *note -I: fd2. + + + ‘optimize’ *note -O: 68b. or + *note -OO: 68c. + + + *note dont_write_bytecode: 3351. *note -B: d38. + + + ‘no_user_site’ *note -s: d15. + + + ‘no_site’ *note -S: b24. + + + ‘ignore_environment’ *note -E: fdc. + + + ‘verbose’ *note -v: d88. + + + ‘bytes_warning’ *note -b: 415. + + + ‘quiet’ *note -q: fdf. + + + ‘hash_randomization’ *note -R: fe0. + + + ‘dev_mode’ *note -X: 155. ‘dev’ + + + ‘utf8_mode’ *note -X: 155. ‘utf8’ + + + Changed in version 3.2: Added ‘quiet’ attribute for the new *note + -q: fdf. flag. + + New in version 3.2.3: The ‘hash_randomization’ attribute. + + Changed in version 3.3: Removed obsolete ‘division_warning’ + attribute. + + Changed in version 3.4: Added ‘isolated’ attribute for *note -I: + fd2. ‘isolated’ flag. + + Changed in version 3.7: Added ‘dev_mode’ attribute for the new + *note -X: 155. ‘dev’ flag and ‘utf8_mode’ attribute for the new + *note -X: 155. ‘utf8’ flag. + + -- Data: sys.float_info + + A *note named tuple: aa3. holding information about the float type. + It contains low level information about the precision and internal + representation. The values correspond to the various + floating-point constants defined in the standard header file + ‘float.h’ for the ‘C’ programming language; see section 5.2.4.2.2 + of the 1999 ISO/IEC C standard *note [C99]: 3356, ‘Characteristics + of floating types’, for details. + + attribute float.h macro explanation + + ------------------------------------------------------------------------------------------------------ + + ‘epsilon’ DBL_EPSILON difference between 1.0 and the least value greater + than 1.0 that is representable as a float + + + ‘dig’ DBL_DIG maximum number of decimal digits that can be + faithfully represented in a float; see below + + + ‘mant_dig’ DBL_MANT_DIG float precision: the number of base-‘radix’ digits + in the significand of a float + + + *note max: 80a. DBL_MAX maximum representable positive finite float + + + ‘max_exp’ DBL_MAX_EXP maximum integer `e' such that ‘radix**(e-1)’ is a + representable finite float + + + ‘max_10_exp’ DBL_MAX_10_EXP maximum integer `e' such that ‘10**e’ is in the + range of representable finite floats + + + *note min: 809. DBL_MIN minimum representable positive `normalized' float + + + ‘min_exp’ DBL_MIN_EXP minimum integer `e' such that ‘radix**(e-1)’ is a + normalized float + + + ‘min_10_exp’ DBL_MIN_10_EXP minimum integer `e' such that ‘10**e’ is a + normalized float + + + ‘radix’ FLT_RADIX radix of exponent representation + + + ‘rounds’ FLT_ROUNDS integer constant representing the rounding mode used + for arithmetic operations. This reflects the value + of the system FLT_ROUNDS macro at interpreter + startup time. See section 5.2.4.2.2 of the C99 + standard for an explanation of the possible values + and their meanings. + + + The attribute ‘sys.float_info.dig’ needs further explanation. If + ‘s’ is any string representing a decimal number with at most + ‘sys.float_info.dig’ significant digits, then converting ‘s’ to a + float and back again will recover a string representing the same + decimal value: + + >>> import sys + >>> sys.float_info.dig + 15 + >>> s = '3.14159265358979' # decimal string with 15 significant digits + >>> format(float(s), '.15g') # convert to float and back -> same value + '3.14159265358979' + + But for strings with more than ‘sys.float_info.dig’ significant + digits, this isn’t always true: + + >>> s = '9876543211234567' # 16 significant digits is too many! + >>> format(float(s), '.16g') # conversion changes value + '9876543211234568' + + -- Data: sys.float_repr_style + + A string indicating how the *note repr(): 7cc. function behaves for + floats. If the string has value ‘'short'’ then for a finite float + ‘x’, ‘repr(x)’ aims to produce a short string with the property + that ‘float(repr(x)) == x’. This is the usual behaviour in Python + 3.1 and later. Otherwise, ‘float_repr_style’ has value ‘'legacy'’ + and ‘repr(x)’ behaves in the same way as it did in versions of + Python prior to 3.1. + + New in version 3.1. + + -- Function: sys.getallocatedblocks () + + Return the number of memory blocks currently allocated by the + interpreter, regardless of their size. This function is mainly + useful for tracking and debugging memory leaks. Because of the + interpreter’s internal caches, the result can vary from call to + call; you may have to call *note _clear_type_cache(): 334d. and + *note gc.collect(): 22e. to get more predictable results. + + If a Python build or implementation cannot reasonably compute this + information, *note getallocatedblocks(): 8e2. is allowed to return + 0 instead. + + New in version 3.4. + + -- Function: sys.getandroidapilevel () + + Return the build time API version of Android as an integer. + + *note Availability: ffb.: Android. + + New in version 3.7. + + -- Function: sys.getcheckinterval () + + Return the interpreter’s “check interval”; see *note + setcheckinterval(): 3357. + + Deprecated since version 3.2: Use *note getswitchinterval(): 3358. + instead. + + -- Function: sys.getdefaultencoding () + + Return the name of the current default string encoding used by the + Unicode implementation. + + -- Function: sys.getdlopenflags () + + Return the current value of the flags that are used for ‘dlopen()’ + calls. Symbolic names for the flag values can be found in the + *note os: c4. module (‘RTLD_xxx’ constants, e.g. *note + os.RTLD_LAZY: a5f.). + + *note Availability: ffb.: Unix. + + -- Function: sys.getfilesystemencoding () + + Return the name of the encoding used to convert between Unicode + filenames and bytes filenames. For best compatibility, str should + be used for filenames in all cases, although representing filenames + as bytes is also supported. Functions accepting or returning + filenames should support either str or bytes and internally convert + to the system’s preferred representation. + + This encoding is always ASCII-compatible. + + *note os.fsencode(): 4ef. and *note os.fsdecode(): 4ee. should be + used to ensure that the correct encoding and errors mode are used. + + * In the UTF-8 mode, the encoding is ‘utf-8’ on any platform. + + * On macOS, the encoding is ‘'utf-8'’. + + * On Unix, the encoding is the locale encoding. + + * On Windows, the encoding may be ‘'utf-8'’ or ‘'mbcs'’, + depending on user configuration. + + * On Android, the encoding is ‘'utf-8'’. + + * On VxWorks, the encoding is ‘'utf-8'’. + + Changed in version 3.2: *note getfilesystemencoding(): 4f6. result + cannot be ‘None’ anymore. + + Changed in version 3.6: Windows is no longer guaranteed to return + ‘'mbcs'’. See PEP 529(3) and *note + _enablelegacywindowsfsencoding(): 4f8. for more information. + + Changed in version 3.7: Return ‘utf-8’ in the UTF-8 mode. + + -- Function: sys.getfilesystemencodeerrors () + + Return the name of the error mode used to convert between Unicode + filenames and bytes filenames. The encoding name is returned from + *note getfilesystemencoding(): 4f6. + + *note os.fsencode(): 4ef. and *note os.fsdecode(): 4ee. should be + used to ensure that the correct encoding and errors mode are used. + + New in version 3.6. + + -- Function: sys.getrefcount (object) + + Return the reference count of the `object'. The count returned is + generally one higher than you might expect, because it includes the + (temporary) reference as an argument to *note getrefcount(): 335a. + + -- Function: sys.getrecursionlimit () + + Return the current value of the recursion limit, the maximum depth + of the Python interpreter stack. This limit prevents infinite + recursion from causing an overflow of the C stack and crashing + Python. It can be set by *note setrecursionlimit(): e7b. + + -- Function: sys.getsizeof (object[, default]) + + Return the size of an object in bytes. The object can be any type + of object. All built-in objects will return correct results, but + this does not have to hold true for third-party extensions as it is + implementation specific. + + Only the memory consumption directly attributed to the object is + accounted for, not the memory consumption of objects it refers to. + + If given, `default' will be returned if the object does not provide + means to retrieve the size. Otherwise a *note TypeError: 192. will + be raised. + + *note getsizeof(): 32e8. calls the object’s ‘__sizeof__’ method and + adds an additional garbage collector overhead if the object is + managed by the garbage collector. + + See recursive sizeof recipe(4) for an example of using *note + getsizeof(): 32e8. recursively to find the size of containers and + all their contents. + + -- Function: sys.getswitchinterval () + + Return the interpreter’s “thread switch interval”; see *note + setswitchinterval(): beb. + + New in version 3.2. + + -- Function: sys._getframe ([depth]) + + Return a frame object from the call stack. If optional integer + `depth' is given, return the frame object that many calls below the + top of the stack. If that is deeper than the call stack, *note + ValueError: 1fb. is raised. The default for `depth' is zero, + returning the frame at the top of the call stack. + + Raises an *note auditing event: fd1. ‘sys._getframe’ with no + arguments. + + `CPython implementation detail:' This function should be used for + internal and specialized purposes only. It is not guaranteed to + exist in all implementations of Python. + + -- Function: sys.getprofile () + + Get the profiler function as set by *note setprofile(): e3d. + + -- Function: sys.gettrace () + + Get the trace function as set by *note settrace(): e3e. + + `CPython implementation detail:' The *note gettrace(): d48. + function is intended only for implementing debuggers, profilers, + coverage tools and the like. Its behavior is part of the + implementation platform, rather than part of the language + definition, and thus may not be available in all Python + implementations. + + -- Function: sys.getwindowsversion () + + Return a named tuple describing the Windows version currently + running. The named elements are `major', `minor', `build', + `platform', `service_pack', `service_pack_minor', + `service_pack_major', `suite_mask', `product_type' and + `platform_version'. `service_pack' contains a string, + `platform_version' a 3-tuple and all other values are integers. + The components can also be accessed by name, so + ‘sys.getwindowsversion()[0]’ is equivalent to + ‘sys.getwindowsversion().major’. For compatibility with prior + versions, only the first 5 elements are retrievable by indexing. + + `platform' will be ‘2 (VER_PLATFORM_WIN32_NT)’. + + `product_type' may be one of the following values: + + Constant Meaning + + ---------------------------------------------------------------------------------- + + ‘1 (VER_NT_WORKSTATION)’ The system is a workstation. + + + ‘2 (VER_NT_DOMAIN_CONTROLLER)’ The system is a domain controller. + + + ‘3 (VER_NT_SERVER)’ The system is a server, but not a + domain controller. + + + This function wraps the Win32 ‘GetVersionEx()’ function; see the + Microsoft documentation on ‘OSVERSIONINFOEX()’ for more information + about these fields. + + `platform_version' returns the accurate major version, minor + version and build number of the current operating system, rather + than the version that is being emulated for the process. It is + intended for use in logging rather than for feature detection. + + *note Availability: ffb.: Windows. + + Changed in version 3.2: Changed to a named tuple and added + `service_pack_minor', `service_pack_major', `suite_mask', and + `product_type'. + + Changed in version 3.6: Added `platform_version' + + -- Function: sys.get_asyncgen_hooks () + + Returns an `asyncgen_hooks' object, which is similar to a *note + namedtuple: 1b7. of the form ‘(firstiter, finalizer)’, where + `firstiter' and `finalizer' are expected to be either ‘None’ or + functions which take an *note asynchronous generator iterator: + 335c. as an argument, and are used to schedule finalization of an + asynchronous generator by an event loop. + + New in version 3.6: See PEP 525(5) for more details. + + Note: This function has been added on a provisional basis (see + PEP 411(6) for details.) + + -- Function: sys.get_coroutine_origin_tracking_depth () + + Get the current coroutine origin tracking depth, as set by *note + set_coroutine_origin_tracking_depth(): 3f3. + + New in version 3.7. + + Note: This function has been added on a provisional basis (see + PEP 411(7) for details.) Use it only for debugging purposes. + + -- Data: sys.hash_info + + A *note named tuple: aa3. giving parameters of the numeric hash + implementation. For more details about hashing of numeric types, + see *note Hashing of numeric types: 12d2. + + attribute explanation + + --------------------------------------------------------------------------------- + + ‘width’ width in bits used for hash values + + + ‘modulus’ prime modulus P used for numeric hash scheme + + + ‘inf’ hash value returned for a positive infinity + + + ‘nan’ hash value returned for a nan + + + ‘imag’ multiplier used for the imaginary part of a complex + number + + + ‘algorithm’ name of the algorithm for hashing of str, bytes, and + memoryview + + + ‘hash_bits’ internal output size of the hash algorithm + + + ‘seed_bits’ size of the seed key of the hash algorithm + + + New in version 3.2. + + Changed in version 3.4: Added `algorithm', `hash_bits' and + `seed_bits' + + -- Data: sys.hexversion + + The version number encoded as a single integer. This is guaranteed + to increase with each version, including proper support for + non-production releases. For example, to test that the Python + interpreter is at least version 1.5.2, use: + + if sys.hexversion >= 0x010502F0: + # use some advanced feature + ... + else: + # use an alternative implementation or warn the user + ... + + This is called ‘hexversion’ since it only really looks meaningful + when viewed as the result of passing it to the built-in *note + hex(): c66. function. The *note named tuple: aa3. *note + sys.version_info: b1a. may be used for a more human-friendly + encoding of the same information. + + More details of ‘hexversion’ can be found at *note API and ABI + Versioning: 335e. + + -- Data: sys.implementation + + An object containing information about the implementation of the + currently running Python interpreter. The following attributes are + required to exist in all Python implementations. + + `name' is the implementation’s identifier, e.g. ‘'cpython'’. The + actual string is defined by the Python implementation, but it is + guaranteed to be lower case. + + `version' is a named tuple, in the same format as *note + sys.version_info: b1a. It represents the version of the Python + `implementation'. This has a distinct meaning from the specific + version of the Python `language' to which the currently running + interpreter conforms, which ‘sys.version_info’ represents. For + example, for PyPy 1.8 ‘sys.implementation.version’ might be + ‘sys.version_info(1, 8, 0, 'final', 0)’, whereas ‘sys.version_info’ + would be ‘sys.version_info(2, 7, 2, 'final', 0)’. For CPython they + are the same value, since it is the reference implementation. + + `hexversion' is the implementation version in hexadecimal format, + like *note sys.hexversion: 335d. + + `cache_tag' is the tag used by the import machinery in the + filenames of cached modules. By convention, it would be a + composite of the implementation’s name and version, like + ‘'cpython-33'’. However, a Python implementation may use some + other value if appropriate. If ‘cache_tag’ is set to ‘None’, it + indicates that module caching should be disabled. + + *note sys.implementation: 9aa. may contain additional attributes + specific to the Python implementation. These non-standard + attributes must start with an underscore, and are not described + here. Regardless of its contents, *note sys.implementation: 9aa. + will not change during a run of the interpreter, nor between + implementation versions. (It may change between Python language + versions, however.) See PEP 421(8) for more information. + + New in version 3.3. + + Note: The addition of new required attributes must go through + the normal PEP process. See PEP 421(9) for more information. + + -- Data: sys.int_info + + A *note named tuple: aa3. that holds information about Python’s + internal representation of integers. The attributes are read only. + + Attribute Explanation + + --------------------------------------------------------------------------------- + + ‘bits_per_digit’ number of bits held in each digit. Python + integers are stored internally in base + ‘2**int_info.bits_per_digit’ + + + ‘sizeof_digit’ size in bytes of the C type used to represent a + digit + + + New in version 3.1. + + -- Data: sys.__interactivehook__ + + When this attribute exists, its value is automatically called (with + no arguments) when the interpreter is launched in *note interactive + mode: 8e3. This is done after the *note PYTHONSTARTUP: 8e5. file + is read, so that you can set this hook there. The *note site: eb. + module *note sets this: 8e6. + + Raises an *note auditing event: fd1. ‘cpython.run_interactivehook’ + with the hook object as the argument when the hook is called on + startup. + + New in version 3.4. + + -- Function: sys.intern (string) + + Enter `string' in the table of “interned” strings and return the + interned string – which is `string' itself or a copy. Interning + strings is useful to gain a little performance on dictionary lookup + – if the keys in a dictionary are interned, and the lookup key is + interned, the key comparisons (after hashing) can be done by a + pointer compare instead of a string compare. Normally, the names + used in Python programs are automatically interned, and the + dictionaries used to hold module, class or instance attributes have + interned keys. + + Interned strings are not immortal; you must keep a reference to the + return value of *note intern(): c6e. around to benefit from it. + + -- Function: sys.is_finalizing () + + Return *note True: 499. if the Python interpreter is *note shutting + down: 763, *note False: 388. otherwise. + + New in version 3.5. + + -- Data: sys.last_type + -- Data: sys.last_value + -- Data: sys.last_traceback + + These three variables are not always defined; they are set when an + exception is not handled and the interpreter prints an error + message and a stack traceback. Their intended use is to allow an + interactive user to import a debugger module and engage in + post-mortem debugging without having to re-execute the command that + caused the error. (Typical use is ‘import pdb; pdb.pm()’ to enter + the post-mortem debugger; see *note pdb: c9. module for more + information.) + + The meaning of the variables is the same as that of the return + values from *note exc_info(): c5f. above. + + -- Data: sys.maxsize + + An integer giving the maximum value a variable of type ‘Py_ssize_t’ + can take. It’s usually ‘2**31 - 1’ on a 32-bit platform and ‘2**63 + - 1’ on a 64-bit platform. + + -- Data: sys.maxunicode + + An integer giving the value of the largest Unicode code point, i.e. + ‘1114111’ (‘0x10FFFF’ in hexadecimal). + + Changed in version 3.3: Before PEP 393(10), ‘sys.maxunicode’ used + to be either ‘0xFFFF’ or ‘0x10FFFF’, depending on the configuration + option that specified whether Unicode characters were stored as + UCS-2 or UCS-4. + + -- Data: sys.meta_path + + A list of *note meta path finder: 9b1. objects that have their + *note find_spec(): 463. methods called to see if one of the objects + can find the module to be imported. The *note find_spec(): 463. + method is called with at least the absolute name of the module + being imported. If the module to be imported is contained in a + package, then the parent package’s *note __path__: f23. attribute + is passed in as a second argument. The method returns a *note + module spec: 3360, or ‘None’ if the module cannot be found. + + See also + ........ + + *note importlib.abc.MetaPathFinder: 9b3. + + The abstract base class defining the interface of finder + objects on *note meta_path: 5e6. + + *note importlib.machinery.ModuleSpec: 116b. + + The concrete class which *note find_spec(): 463. should return + instances of. + + Changed in version 3.4: *note Module specs: 3360. were introduced + in Python 3.4, by PEP 451(11). Earlier versions of Python looked + for a method called *note find_module(): 462. This is still called + as a fallback if a *note meta_path: 5e6. entry doesn’t have a *note + find_spec(): 463. method. + + -- Data: sys.modules + + This is a dictionary that maps module names to modules which have + already been loaded. This can be manipulated to force reloading of + modules and other tricks. However, replacing the dictionary will + not necessarily work as expected and deleting essential items from + the dictionary may cause Python to fail. + + -- Data: sys.path + + A list of strings that specifies the search path for modules. + Initialized from the environment variable *note PYTHONPATH: 953, + plus an installation-dependent default. + + As initialized upon program startup, the first item of this list, + ‘path[0]’, is the directory containing the script that was used to + invoke the Python interpreter. If the script directory is not + available (e.g. if the interpreter is invoked interactively or if + the script is read from standard input), ‘path[0]’ is the empty + string, which directs Python to search modules in the current + directory first. Notice that the script directory is inserted + `before' the entries inserted as a result of *note PYTHONPATH: 953. + + A program is free to modify this list for its own purposes. Only + strings and bytes should be added to *note sys.path: 488.; all + other data types are ignored during import. + + See also + ........ + + Module *note site: eb. This describes how to use .pth files to + extend *note sys.path: 488. + + -- Data: sys.path_hooks + + A list of callables that take a path argument to try to create a + *note finder: 115f. for the path. If a finder can be created, it + is to be returned by the callable, else raise *note ImportError: + 334. + + Originally specified in PEP 302(12). + + -- Data: sys.path_importer_cache + + A dictionary acting as a cache for *note finder: 115f. objects. + The keys are paths that have been passed to *note sys.path_hooks: + 95c. and the values are the finders that are found. If a path is a + valid file system path but no finder is found on *note + sys.path_hooks: 95c. then ‘None’ is stored. + + Originally specified in PEP 302(13). + + Changed in version 3.3: ‘None’ is stored instead of *note + imp.NullImporter: 9bb. when no finder is found. + + -- Data: sys.platform + + This string contains a platform identifier that can be used to + append platform-specific components to *note sys.path: 488, for + instance. + + For Unix systems, except on Linux and AIX, this is the lowercased + OS name as returned by ‘uname -s’ with the first part of the + version as returned by ‘uname -r’ appended, e.g. ‘'sunos5'’ or + ‘'freebsd8'’, `at the time when Python was built'. Unless you want + to test for a specific system version, it is therefore recommended + to use the following idiom: + + if sys.platform.startswith('freebsd'): + # FreeBSD-specific code here... + elif sys.platform.startswith('linux'): + # Linux-specific code here... + elif sys.platform.startswith('aix'): + # AIX-specific code here... + + For other systems, the values are: + + System ‘platform’ value + + ----------------------------------------------------- + + AIX ‘'aix'’ + + + Linux ‘'linux'’ + + + Windows ‘'win32'’ + + + Windows/Cygwin ‘'cygwin'’ + + + macOS ‘'darwin'’ + + + Changed in version 3.3: On Linux, *note sys.platform: 2b1. doesn’t + contain the major version anymore. It is always ‘'linux'’, instead + of ‘'linux2'’ or ‘'linux3'’. Since older Python versions include + the version number, it is recommended to always use the + ‘startswith’ idiom presented above. + + Changed in version 3.8: On AIX, *note sys.platform: 2b1. doesn’t + contain the major version anymore. It is always ‘'aix'’, instead + of ‘'aix5'’ or ‘'aix7'’. Since older Python versions include the + version number, it is recommended to always use the ‘startswith’ + idiom presented above. + + See also + ........ + + *note os.name: 1bb0. has a coarser granularity. *note os.uname(): + a5d. gives system-dependent version information. + + The *note platform: ce. module provides detailed checks for the + system’s identity. + + -- Data: sys.prefix + + A string giving the site-specific directory prefix where the + platform independent Python files are installed; by default, this + is the string ‘'/usr/local'’. This can be set at build time with + the ‘--prefix’ argument to the ‘configure’ script. The main + collection of Python library modules is installed in the directory + ‘`prefix'/lib/python`X.Y'’ while the platform independent header + files (all except ‘pyconfig.h’) are stored in + ‘`prefix'/include/python`X.Y'’, where `X.Y' is the version number + of Python, for example ‘3.2’. + + Note: If a *note virtual environment: 3321. is in effect, this + value will be changed in ‘site.py’ to point to the virtual + environment. The value for the Python installation will still + be available, via *note base_prefix: 2d50. + + -- Data: sys.ps1 + -- Data: sys.ps2 + + Strings specifying the primary and secondary prompt of the + interpreter. These are only defined if the interpreter is in + interactive mode. Their initial values in this case are ‘'>>> '’ + and ‘'... '’. If a non-string object is assigned to either + variable, its *note str(): 330. is re-evaluated each time the + interpreter prepares to read a new interactive command; this can be + used to implement a dynamic prompt. + + -- Function: sys.setcheckinterval (interval) + + Set the interpreter’s “check interval”. This integer value + determines how often the interpreter checks for periodic things + such as thread switches and signal handlers. The default is ‘100’, + meaning the check is performed every 100 Python virtual + instructions. Setting it to a larger value may increase + performance for programs using threads. Setting it to a value ‘<=’ + 0 checks every virtual instruction, maximizing responsiveness as + well as overhead. + + Deprecated since version 3.2: This function doesn’t have an effect + anymore, as the internal logic for thread switching and + asynchronous tasks has been rewritten. Use *note + setswitchinterval(): beb. instead. + + -- Function: sys.setdlopenflags (n) + + Set the flags used by the interpreter for ‘dlopen()’ calls, such as + when the interpreter loads extension modules. Among other things, + this will enable a lazy resolving of symbols when importing a + module, if called as ‘sys.setdlopenflags(0)’. To share symbols + across extension modules, call as + ‘sys.setdlopenflags(os.RTLD_GLOBAL)’. Symbolic names for the flag + values can be found in the *note os: c4. module (‘RTLD_xxx’ + constants, e.g. *note os.RTLD_LAZY: a5f.). + + *note Availability: ffb.: Unix. + + -- Function: sys.setprofile (profilefunc) + + Set the system’s profile function, which allows you to implement a + Python source code profiler in Python. See chapter *note The + Python Profilers: 3282. for more information on the Python + profiler. The system’s profile function is called similarly to the + system’s trace function (see *note settrace(): e3e.), but it is + called with different events, for example it isn’t called for each + executed line of code (only on call and return, but the return + event is reported even when an exception has been set). The + function is thread-specific, but there is no way for the profiler + to know about context switches between threads, so it does not make + sense to use this in the presence of multiple threads. Also, its + return value is not used, so it can simply return ‘None’. Error in + the profile function will cause itself unset. + + Profile functions should have three arguments: `frame', `event', + and `arg'. `frame' is the current stack frame. `event' is a + string: ‘'call'’, ‘'return'’, ‘'c_call'’, ‘'c_return'’, or + ‘'c_exception'’. `arg' depends on the event type. + + Raises an *note auditing event: fd1. ‘sys.setprofile’ with no + arguments. + + The events have the following meaning: + + ‘'call'’ + + A function is called (or some other code block entered). The + profile function is called; `arg' is ‘None’. + + ‘'return'’ + + A function (or other code block) is about to return. The + profile function is called; `arg' is the value that will be + returned, or ‘None’ if the event is caused by an exception + being raised. + + ‘'c_call'’ + + A C function is about to be called. This may be an extension + function or a built-in. `arg' is the C function object. + + ‘'c_return'’ + + A C function has returned. `arg' is the C function object. + + ‘'c_exception'’ + + A C function has raised an exception. `arg' is the C function + object. + + -- Function: sys.setrecursionlimit (limit) + + Set the maximum depth of the Python interpreter stack to `limit'. + This limit prevents infinite recursion from causing an overflow of + the C stack and crashing Python. + + The highest possible limit is platform-dependent. A user may need + to set the limit higher when they have a program that requires deep + recursion and a platform that supports a higher limit. This should + be done with care, because a too-high limit can lead to a crash. + + If the new limit is too low at the current recursion depth, a *note + RecursionError: 634. exception is raised. + + Changed in version 3.5.1: A *note RecursionError: 634. exception is + now raised if the new limit is too low at the current recursion + depth. + + -- Function: sys.setswitchinterval (interval) + + Set the interpreter’s thread switch interval (in seconds). This + floating-point value determines the ideal duration of the + “timeslices” allocated to concurrently running Python threads. + Please note that the actual value can be higher, especially if + long-running internal functions or methods are used. Also, which + thread becomes scheduled at the end of the interval is the + operating system’s decision. The interpreter doesn’t have its own + scheduler. + + New in version 3.2. + + -- Function: sys.settrace (tracefunc) + + Set the system’s trace function, which allows you to implement a + Python source code debugger in Python. The function is + thread-specific; for a debugger to support multiple threads, it + must register a trace function using *note settrace(): e3e. for + each thread being debugged or use *note threading.settrace(): 203d. + + Trace functions should have three arguments: `frame', `event', and + `arg'. `frame' is the current stack frame. `event' is a string: + ‘'call'’, ‘'line'’, ‘'return'’, ‘'exception'’ or ‘'opcode'’. `arg' + depends on the event type. + + The trace function is invoked (with `event' set to ‘'call'’) + whenever a new local scope is entered; it should return a reference + to a local trace function to be used for the new scope, or ‘None’ + if the scope shouldn’t be traced. + + The local trace function should return a reference to itself (or to + another function for further tracing in that scope), or ‘None’ to + turn off tracing in that scope. + + If there is any error occurred in the trace function, it will be + unset, just like ‘settrace(None)’ is called. + + The events have the following meaning: + + ‘'call'’ + + A function is called (or some other code block entered). The + global trace function is called; `arg' is ‘None’; the return + value specifies the local trace function. + + ‘'line'’ + + The interpreter is about to execute a new line of code or + re-execute the condition of a loop. The local trace function + is called; `arg' is ‘None’; the return value specifies the new + local trace function. See ‘Objects/lnotab_notes.txt’ for a + detailed explanation of how this works. Per-line events may + be disabled for a frame by setting ‘f_trace_lines’ to *note + False: 388. on that frame. + + ‘'return'’ + + A function (or other code block) is about to return. The + local trace function is called; `arg' is the value that will + be returned, or ‘None’ if the event is caused by an exception + being raised. The trace function’s return value is ignored. + + ‘'exception'’ + + An exception has occurred. The local trace function is + called; `arg' is a tuple ‘(exception, value, traceback)’; the + return value specifies the new local trace function. + + ‘'opcode'’ + + The interpreter is about to execute a new opcode (see *note + dis: 38. for opcode details). The local trace function is + called; `arg' is ‘None’; the return value specifies the new + local trace function. Per-opcode events are not emitted by + default: they must be explicitly requested by setting + ‘f_trace_opcodes’ to *note True: 499. on the frame. + + Note that as an exception is propagated down the chain of callers, + an ‘'exception'’ event is generated at each level. + + For more fine-grained usage, it’s possible to set a trace function + by assigning ‘frame.f_trace = tracefunc’ explicitly, rather than + relying on it being set indirectly via the return value from an + already installed trace function. This is also required for + activating the trace function on the current frame, which *note + settrace(): e3e. doesn’t do. Note that in order for this to work, + a global tracing function must have been installed with *note + settrace(): e3e. in order to enable the runtime tracing machinery, + but it doesn’t need to be the same tracing function (e.g. it could + be a low overhead tracing function that simply returns ‘None’ to + disable itself immediately on each frame). + + For more information on code and frame objects, refer to *note The + standard type hierarchy: 10c0. + + Raises an *note auditing event: fd1. ‘sys.settrace’ with no + arguments. + + `CPython implementation detail:' The *note settrace(): e3e. + function is intended only for implementing debuggers, profilers, + coverage tools and the like. Its behavior is part of the + implementation platform, rather than part of the language + definition, and thus may not be available in all Python + implementations. + + Changed in version 3.7: ‘'opcode'’ event type added; + ‘f_trace_lines’ and ‘f_trace_opcodes’ attributes added to frames + + -- Function: sys.set_asyncgen_hooks (firstiter, finalizer) + + Accepts two optional keyword arguments which are callables that + accept an *note asynchronous generator iterator: 335c. as an + argument. The `firstiter' callable will be called when an + asynchronous generator is iterated for the first time. The + `finalizer' will be called when an asynchronous generator is about + to be garbage collected. + + Raises an *note auditing event: fd1. + ‘sys.set_asyncgen_hooks_firstiter’ with no arguments. + + Raises an *note auditing event: fd1. + ‘sys.set_asyncgen_hooks_finalizer’ with no arguments. + + Two auditing events are raised because the underlying API consists + of two calls, each of which must raise its own event. + + New in version 3.6: See PEP 525(14) for more details, and for a + reference example of a `finalizer' method see the implementation of + ‘asyncio.Loop.shutdown_asyncgens’ in Lib/asyncio/base_events.py(15) + + Note: This function has been added on a provisional basis (see + PEP 411(16) for details.) + + -- Function: sys.set_coroutine_origin_tracking_depth (depth) + + Allows enabling or disabling coroutine origin tracking. When + enabled, the ‘cr_origin’ attribute on coroutine objects will + contain a tuple of (filename, line number, function name) tuples + describing the traceback where the coroutine object was created, + with the most recent call first. When disabled, ‘cr_origin’ will + be None. + + To enable, pass a `depth' value greater than zero; this sets the + number of frames whose information will be captured. To disable, + pass set `depth' to zero. + + This setting is thread-specific. + + New in version 3.7. + + Note: This function has been added on a provisional basis (see + PEP 411(17) for details.) Use it only for debugging purposes. + + -- Function: sys._enablelegacywindowsfsencoding () + + Changes the default filesystem encoding and errors mode to ‘mbcs’ + and ‘replace’ respectively, for consistency with versions of Python + prior to 3.6. + + This is equivalent to defining the *note + PYTHONLEGACYWINDOWSFSENCODING: 4f7. environment variable before + launching Python. + + *note Availability: ffb.: Windows. + + New in version 3.6: See PEP 529(18) for more details. + + -- Data: sys.stdin + -- Data: sys.stdout + -- Data: sys.stderr + + *note File objects: b42. used by the interpreter for standard + input, output and errors: + + * ‘stdin’ is used for all interactive input (including calls to + *note input(): c6b.); + + * ‘stdout’ is used for the output of *note print(): 886. and + *note expression: 3350. statements and for the prompts of + *note input(): c6b.; + + * The interpreter’s own prompts and its error messages go to + ‘stderr’. + + These streams are regular *note text files: f3a. like those + returned by the *note open(): 4f0. function. Their parameters are + chosen as follows: + + * The character encoding is platform-dependent. Non-Windows + platforms use the locale encoding (see *note + locale.getpreferredencoding(): 3a5.). + + On Windows, UTF-8 is used for the console device. + Non-character devices such as disk files and pipes use the + system locale encoding (i.e. the ANSI codepage). Non-console + character devices such as NUL (i.e. where ‘isatty()’ returns + ‘True’) use the value of the console input and output + codepages at startup, respectively for stdin and + stdout/stderr. This defaults to the system locale encoding if + the process is not initially attached to a console. + + The special behaviour of the console can be overridden by + setting the environment variable PYTHONLEGACYWINDOWSSTDIO + before starting Python. In that case, the console codepages + are used as for any other character device. + + Under all platforms, you can override the character encoding + by setting the *note PYTHONIOENCODING: 92f. environment + variable before starting Python or by using the new *note -X: + 155. ‘utf8’ command line option and *note PYTHONUTF8: 311. + environment variable. However, for the Windows console, this + only applies when *note PYTHONLEGACYWINDOWSSTDIO: 4fa. is also + set. + + * When interactive, ‘stdout’ and ‘stderr’ streams are + line-buffered. Otherwise, they are block-buffered like + regular text files. You can override this value with the + *note -u: fe3. command-line option. + + Note: To write or read binary data from/to the standard + streams, use the underlying binary *note buffer: c3a. object. + For example, to write bytes to *note stdout: 30e, use + ‘sys.stdout.buffer.write(b'abc')’. + + However, if you are writing a library (and do not control in + which context its code will be executed), be aware that the + standard streams may be replaced with file-like objects like + *note io.StringIO: 82d. which do not support the ‘buffer’ + attribute. + + -- Data: sys.__stdin__ + -- Data: sys.__stdout__ + -- Data: sys.__stderr__ + + These objects contain the original values of ‘stdin’, ‘stderr’ and + ‘stdout’ at the start of the program. They are used during + finalization, and could be useful to print to the actual standard + stream no matter if the ‘sys.std*’ object has been redirected. + + It can also be used to restore the actual files to known working + file objects in case they have been overwritten with a broken + object. However, the preferred way to do this is to explicitly + save the previous stream before replacing it, and restore the saved + object. + + Note: Under some conditions ‘stdin’, ‘stdout’ and ‘stderr’ as + well as the original values ‘__stdin__’, ‘__stdout__’ and + ‘__stderr__’ can be ‘None’. It is usually the case for + Windows GUI apps that aren’t connected to a console and Python + apps started with ‘pythonw’. + + -- Data: sys.thread_info + + A *note named tuple: aa3. holding information about the thread + implementation. + + Attribute Explanation + + ------------------------------------------------------------------------------------- + + ‘name’ Name of the thread implementation: + + * ‘'nt'’: Windows threads + + * ‘'pthread'’: POSIX threads + + * ‘'solaris'’: Solaris threads + + + ‘lock’ Name of the lock implementation: + + * ‘'semaphore'’: a lock uses a semaphore + + * ‘'mutex+cond'’: a lock uses a mutex and a + condition variable + + * ‘None’ if this information is unknown + + + *note version: 5d3. Name and version of the thread library. It is a string, or + ‘None’ if this information is unknown. + + + New in version 3.3. + + -- Data: sys.tracebacklimit + + When this variable is set to an integer value, it determines the + maximum number of levels of traceback information printed when an + unhandled exception occurs. The default is ‘1000’. When set to + ‘0’ or less, all traceback information is suppressed and only the + exception type and value are printed. + + -- Function: sys.unraisablehook (unraisable, /) + + Handle an unraisable exception. + + Called when an exception has occurred but there is no way for + Python to handle it. For example, when a destructor raises an + exception or during garbage collection (*note gc.collect(): 22e.). + + The `unraisable' argument has the following attributes: + + * `exc_type': Exception type. + + * `exc_value': Exception value, can be ‘None’. + + * `exc_traceback': Exception traceback, can be ‘None’. + + * `err_msg': Error message, can be ‘None’. + + * `object': Object causing the exception, can be ‘None’. + + The default hook formats `err_msg' and `object' as: ‘f'{err_msg}: + {object!r}'’; use “Exception ignored in” error message if `err_msg' + is ‘None’. + + *note sys.unraisablehook(): 22d. can be overridden to control how + unraisable exceptions are handled. + + Storing `exc_value' using a custom hook can create a reference + cycle. It should be cleared explicitly to break the reference + cycle when the exception is no longer needed. + + Storing `object' using a custom hook can resurrect it if it is set + to an object which is being finalized. Avoid storing `object' + after the custom hook completes to avoid resurrecting objects. + + See also *note excepthook(): e63. which handles uncaught + exceptions. + + Raise an auditing event ‘sys.unraisablehook’ with arguments ‘hook’, + ‘unraisable’ when an exception that cannot be handled occurs. The + ‘unraisable’ object is the same as what will be passed to the hook. + If no hook has been set, ‘hook’ may be ‘None’. + + New in version 3.8. + + -- Data: sys.version + + A string containing the version number of the Python interpreter + plus additional information on the build number and compiler used. + This string is displayed when the interactive interpreter is + started. Do not extract version information out of it, rather, use + *note version_info: b1a. and the functions provided by the *note + platform: ce. module. + + -- Data: sys.api_version + + The C API version for this interpreter. Programmers may find this + useful when debugging version conflicts between Python and + extension modules. + + -- Data: sys.version_info + + A tuple containing the five components of the version number: + `major', `minor', `micro', `releaselevel', and `serial'. All + values except `releaselevel' are integers; the release level is + ‘'alpha'’, ‘'beta'’, ‘'candidate'’, or ‘'final'’. The + ‘version_info’ value corresponding to the Python version 2.0 is + ‘(2, 0, 0, 'final', 0)’. The components can also be accessed by + name, so ‘sys.version_info[0]’ is equivalent to + ‘sys.version_info.major’ and so on. + + Changed in version 3.1: Added named component attributes. + + -- Data: sys.warnoptions + + This is an implementation detail of the warnings framework; do not + modify this value. Refer to the *note warnings: 126. module for + more information on the warnings framework. + + -- Data: sys.winver + + The version number used to form registry keys on Windows platforms. + This is stored as string resource 1000 in the Python DLL. The value + is normally the first three characters of *note version: 5d3. It + is provided in the *note sys: fd. module for informational + purposes; modifying this value has no effect on the registry keys + used by Python. + + *note Availability: ffb.: Windows. + + -- Data: sys._xoptions + + A dictionary of the various implementation-specific flags passed + through the *note -X: 155. command-line option. Option names are + either mapped to their values, if given explicitly, or to *note + True: 499. Example: + + $ ./python -Xa=b -Xc + Python 3.2a3+ (py3k, Oct 16 2010, 20:14:50) + [GCC 4.4.3] on linux2 + Type "help", "copyright", "credits" or "license" for more information. + >>> import sys + >>> sys._xoptions + {'a': 'b', 'c': True} + + `CPython implementation detail:' This is a CPython-specific way of + accessing options passed through *note -X: 155. Other + implementations may export them through other means, or not at all. + + New in version 3.2. + +Citations +......... + +(C99) ISO/IEC 9899:1999. “Programming languages – C.” A public draft of +this standard is available at +‘http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf’. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3149 + + (2) https://www.python.org/dev/peps/pep-0578 + + (3) https://www.python.org/dev/peps/pep-0529 + + (4) https://code.activestate.com/recipes/577504 + + (5) https://www.python.org/dev/peps/pep-0525 + + (6) https://www.python.org/dev/peps/pep-0411 + + (7) https://www.python.org/dev/peps/pep-0411 + + (8) https://www.python.org/dev/peps/pep-0421 + + (9) https://www.python.org/dev/peps/pep-0421 + + (10) https://www.python.org/dev/peps/pep-0393 + + (11) https://www.python.org/dev/peps/pep-0451 + + (12) https://www.python.org/dev/peps/pep-0302 + + (13) https://www.python.org/dev/peps/pep-0302 + + (14) https://www.python.org/dev/peps/pep-0525 + + (15) +https://github.com/python/cpython/tree/3.8/Lib/asyncio/base_events.py + + (16) https://www.python.org/dev/peps/pep-0411 + + (17) https://www.python.org/dev/peps/pep-0411 + + (18) https://www.python.org/dev/peps/pep-0529 + + +File: python.info, Node: sysconfig — Provide access to Python’s configuration information, Next: builtins — Built-in objects, Prev: sys — System-specific parameters and functions, Up: Python Runtime Services + +5.29.2 ‘sysconfig’ — Provide access to Python’s configuration information +------------------------------------------------------------------------- + +New in version 3.2. + +`Source code:' Lib/sysconfig.py(1) + +__________________________________________________________________ + +The *note sysconfig: fe. module provides access to Python’s +configuration information like the list of installation paths and the +configuration variables relevant for the current platform. + +* Menu: + +* Configuration variables:: +* Installation paths:: +* Other functions: Other functions<3>. +* Using sysconfig as a script:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/sysconfig.py + + +File: python.info, Node: Configuration variables, Next: Installation paths, Up: sysconfig — Provide access to Python’s configuration information + +5.29.2.1 Configuration variables +................................ + +A Python distribution contains a ‘Makefile’ and a ‘pyconfig.h’ header +file that are necessary to build both the Python binary itself and +third-party C extensions compiled using *note distutils: 39. + +*note sysconfig: fe. puts all variables found in these files in a +dictionary that can be accessed using *note get_config_vars(): 965. or +*note get_config_var(): 964. + +Notice that on Windows, it’s a much smaller set. + + -- Function: sysconfig.get_config_vars (*args) + + With no arguments, return a dictionary of all configuration + variables relevant for the current platform. + + With arguments, return a list of values that result from looking up + each argument in the configuration variable dictionary. + + For each argument, if the value is not found, return ‘None’. + + -- Function: sysconfig.get_config_var (name) + + Return the value of a single variable `name'. Equivalent to + ‘get_config_vars().get(name)’. + + If `name' is not found, return ‘None’. + +Example of usage: + + >>> import sysconfig + >>> sysconfig.get_config_var('Py_ENABLE_SHARED') + 0 + >>> sysconfig.get_config_var('LIBDIR') + '/usr/local/lib' + >>> sysconfig.get_config_vars('AR', 'CXX') + ['ar', 'g++'] + + +File: python.info, Node: Installation paths, Next: Other functions<3>, Prev: Configuration variables, Up: sysconfig — Provide access to Python’s configuration information + +5.29.2.2 Installation paths +........................... + +Python uses an installation scheme that differs depending on the +platform and on the installation options. These schemes are stored in +*note sysconfig: fe. under unique identifiers based on the value +returned by *note os.name: 1bb0. + +Every new component that is installed using *note distutils: 39. or a +Distutils-based system will follow the same scheme to copy its file in +the right places. + +Python currently supports seven schemes: + + - `posix_prefix': scheme for POSIX platforms like Linux or Mac OS X. + This is the default scheme used when Python or a component is + installed. + + - `posix_home': scheme for POSIX platforms used when a `home' option + is used upon installation. This scheme is used when a component is + installed through Distutils with a specific home prefix. + + - `posix_user': scheme for POSIX platforms used when a component is + installed through Distutils and the `user' option is used. This + scheme defines paths located under the user home directory. + + - `nt': scheme for NT platforms like Windows. + + - `nt_user': scheme for NT platforms, when the `user' option is used. + +Each scheme is itself composed of a series of paths and each path has a +unique identifier. Python currently uses eight paths: + + - `stdlib': directory containing the standard Python library files + that are not platform-specific. + + - `platstdlib': directory containing the standard Python library + files that are platform-specific. + + - `platlib': directory for site-specific, platform-specific files. + + - `purelib': directory for site-specific, non-platform-specific + files. + + - `include': directory for non-platform-specific header files. + + - `platinclude': directory for platform-specific header files. + + - `scripts': directory for script files. + + - `data': directory for data files. + +*note sysconfig: fe. provides some functions to determine these paths. + + -- Function: sysconfig.get_scheme_names () + + Return a tuple containing all schemes currently supported in *note + sysconfig: fe. + + -- Function: sysconfig.get_path_names () + + Return a tuple containing all path names currently supported in + *note sysconfig: fe. + + -- Function: sysconfig.get_path (name[, scheme[, vars[, expand]]]) + + Return an installation path corresponding to the path `name', from + the install scheme named `scheme'. + + `name' has to be a value from the list returned by *note + get_path_names(): 336b. + + *note sysconfig: fe. stores installation paths corresponding to + each path name, for each platform, with variables to be expanded. + For instance the `stdlib' path for the `nt' scheme is: + ‘{base}/Lib’. + + *note get_path(): cbe. will use the variables returned by *note + get_config_vars(): 965. to expand the path. All variables have + default values for each platform so one may call this function and + get the default value. + + If `scheme' is provided, it must be a value from the list returned + by *note get_scheme_names(): 336a. Otherwise, the default scheme + for the current platform is used. + + If `vars' is provided, it must be a dictionary of variables that + will update the dictionary return by *note get_config_vars(): 965. + + If `expand' is set to ‘False’, the path will not be expanded using + the variables. + + If `name' is not found, return ‘None’. + + -- Function: sysconfig.get_paths ([scheme[, vars[, expand]]]) + + Return a dictionary containing all installation paths corresponding + to an installation scheme. See *note get_path(): cbe. for more + information. + + If `scheme' is not provided, will use the default scheme for the + current platform. + + If `vars' is provided, it must be a dictionary of variables that + will update the dictionary used to expand the paths. + + If `expand' is set to false, the paths will not be expanded. + + If `scheme' is not an existing scheme, *note get_paths(): bd9. will + raise a *note KeyError: 2c7. + + +File: python.info, Node: Other functions<3>, Next: Using sysconfig as a script, Prev: Installation paths, Up: sysconfig — Provide access to Python’s configuration information + +5.29.2.3 Other functions +........................ + + -- Function: sysconfig.get_python_version () + + Return the ‘MAJOR.MINOR’ Python version number as a string. + Similar to ‘'%d.%d' % sys.version_info[:2]’. + + -- Function: sysconfig.get_platform () + + Return a string that identifies the current platform. + + This is used mainly to distinguish platform-specific build + directories and platform-specific built distributions. Typically + includes the OS name and version and the architecture (as supplied + by ‘os.uname()’), although the exact information included depends + on the OS; e.g., on Linux, the kernel version isn’t particularly + important. + + Examples of returned values: + + - linux-i586 + + - linux-alpha (?) + + - solaris-2.6-sun4u + + Windows will return one of: + + - win-amd64 (64bit Windows on AMD64, aka x86_64, Intel64, and + EM64T) + + - win32 (all others - specifically, sys.platform is returned) + + Mac OS X can return: + + - macosx-10.6-ppc + + - macosx-10.4-ppc64 + + - macosx-10.3-i386 + + - macosx-10.4-fat + + For other non-POSIX platforms, currently just returns *note + sys.platform: 2b1. + + -- Function: sysconfig.is_python_build () + + Return ‘True’ if the running Python interpreter was built from + source and is being run from its built location, and not from a + location resulting from e.g. running ‘make install’ or installing + via a binary installer. + + -- Function: sysconfig.parse_config_h (fp[, vars]) + + Parse a ‘config.h’-style file. + + `fp' is a file-like object pointing to the ‘config.h’-like file. + + A dictionary containing name/value pairs is returned. If an + optional dictionary is passed in as the second argument, it is used + instead of a new dictionary, and updated with the values read in + the file. + + -- Function: sysconfig.get_config_h_filename () + + Return the path of ‘pyconfig.h’. + + -- Function: sysconfig.get_makefile_filename () + + Return the path of ‘Makefile’. + + +File: python.info, Node: Using sysconfig as a script, Prev: Other functions<3>, Up: sysconfig — Provide access to Python’s configuration information + +5.29.2.4 Using ‘sysconfig’ as a script +...................................... + +You can use *note sysconfig: fe. as a script with Python’s `-m' option: + + $ python -m sysconfig + Platform: "macosx-10.4-i386" + Python version: "3.2" + Current installation scheme: "posix_prefix" + + Paths: + data = "/usr/local" + include = "/Users/tarek/Dev/svn.python.org/py3k/Include" + platinclude = "." + platlib = "/usr/local/lib/python3.2/site-packages" + platstdlib = "/usr/local/lib/python3.2" + purelib = "/usr/local/lib/python3.2/site-packages" + scripts = "/usr/local/bin" + stdlib = "/usr/local/lib/python3.2" + + Variables: + AC_APPLE_UNIVERSAL_BUILD = "0" + AIX_GENUINE_CPLUSPLUS = "0" + AR = "ar" + ARFLAGS = "rc" + ... + +This call will print in the standard output the information returned by +*note get_platform(): bd7, *note get_python_version(): bd8, *note +get_path(): cbe. and *note get_config_vars(): 965. + + +File: python.info, Node: builtins — Built-in objects, Next: __main__ — Top-level script environment, Prev: sysconfig — Provide access to Python’s configuration information, Up: Python Runtime Services + +5.29.3 ‘builtins’ — Built-in objects +------------------------------------ + +__________________________________________________________________ + +This module provides direct access to all ‘built-in’ identifiers of +Python; for example, ‘builtins.open’ is the full name for the built-in +function *note open(): 4f0. See *note Built-in Functions: bf3. and +*note Built-in Constants: 12b5. for documentation. + +This module is not normally accessed explicitly by most applications, +but can be useful in modules that provide objects with the same name as +a built-in value, but in which the built-in of that name is also needed. +For example, in a module that wants to implement an *note open(): 4f0. +function that wraps the built-in *note open(): 4f0, this module can be +used directly: + + import builtins + + def open(path): + f = builtins.open(path, 'r') + return UpperCaser(f) + + class UpperCaser: + '''Wrapper around a file that converts output to upper-case.''' + + def __init__(self, f): + self._f = f + + def read(self, count=-1): + return self._f.read(count).upper() + + # ... + +As an implementation detail, most modules have the name ‘__builtins__’ +made available as part of their globals. The value of ‘__builtins__’ is +normally either this module or the value of this module’s *note +__dict__: 4fc. attribute. Since this is an implementation detail, it +may not be used by alternate implementations of Python. + + +File: python.info, Node: __main__ — Top-level script environment, Next: warnings — Warning control, Prev: builtins — Built-in objects, Up: Python Runtime Services + +5.29.4 ‘__main__’ — Top-level script environment +------------------------------------------------ + +__________________________________________________________________ + +‘'__main__'’ is the name of the scope in which top-level code executes. +A module’s __name__ is set equal to ‘'__main__'’ when read from standard +input, a script, or from an interactive prompt. + +A module can discover whether or not it is running in the main scope by +checking its own ‘__name__’, which allows a common idiom for +conditionally executing code in a module when it is run as a script or +with ‘python -m’ but not when it is imported: + + if __name__ == "__main__": + # execute only if run as a script + main() + +For a package, the same effect can be achieved by including a +‘__main__.py’ module, the contents of which will be executed when the +module is run with ‘-m’. + + +File: python.info, Node: warnings — Warning control, Next: dataclasses — Data Classes, Prev: __main__ — Top-level script environment, Up: Python Runtime Services + +5.29.5 ‘warnings’ — Warning control +----------------------------------- + +`Source code:' Lib/warnings.py(1) + +__________________________________________________________________ + +Warning messages are typically issued in situations where it is useful +to alert the user of some condition in a program, where that condition +(normally) doesn’t warrant raising an exception and terminating the +program. For example, one might want to issue a warning when a program +uses an obsolete module. + +Python programmers issue warnings by calling the *note warn(): e58. +function defined in this module. (C programmers use *note +PyErr_WarnEx(): db1.; see *note Exception Handling: 3377. for details). + +Warning messages are normally written to *note sys.stderr: 30f, but +their disposition can be changed flexibly, from ignoring all warnings to +turning them into exceptions. The disposition of warnings can vary +based on the *note warning category: 13c1, the text of the warning +message, and the source location where it is issued. Repetitions of a +particular warning for the same source location are typically +suppressed. + +There are two stages in warning control: first, each time a warning is +issued, a determination is made whether a message should be issued or +not; next, if a message is to be issued, it is formatted and printed +using a user-settable hook. + +The determination whether to issue a warning message is controlled by +the *note warning filter: fe7, which is a sequence of matching rules and +actions. Rules can be added to the filter by calling *note +filterwarnings(): e07. and reset to its default state by calling *note +resetwarnings(): 3378. + +The printing of warning messages is done by calling *note showwarning(): +3379, which may be overridden; the default implementation of this +function formats the message by calling *note formatwarning(): 1dad, +which is also available for use by custom implementations. + +See also +........ + +*note logging.captureWarnings(): 1dac. allows you to handle all warnings +with the standard logging infrastructure. + +* Menu: + +* Warning Categories:: +* The Warnings Filter:: +* Temporarily Suppressing Warnings:: +* Testing Warnings:: +* Updating Code For New Versions of Dependencies:: +* Available Functions:: +* Available Context Managers:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/warnings.py + + +File: python.info, Node: Warning Categories, Next: The Warnings Filter, Up: warnings — Warning control + +5.29.5.1 Warning Categories +........................... + +There are a number of built-in exceptions that represent warning +categories. This categorization is useful to be able to filter out +groups of warnings. + +While these are technically *note built-in exceptions: 13bf, they are +documented here, because conceptually they belong to the warnings +mechanism. + +User code can define additional warning categories by subclassing one of +the standard warning categories. A warning category must always be a +subclass of the *note Warning: 13c2. class. + +The following warnings category classes are currently defined: + +Class Description + +------------------------------------------------------------------------------------------- + +*note Warning: 13c2. This is the base class of all warning category + classes. It is a subclass of + *note Exception: 1a9. + + +*note UserWarning: 13c3. The default category for *note warn(): e58. + + +*note DeprecationWarning: 278. Base category for warnings about deprecated + features when those warnings are intended for + other Python developers (ignored by default, + unless triggered by code in ‘__main__’). + + +*note SyntaxWarning: 191. Base category for warnings about dubious + syntactic features. + + +*note RuntimeWarning: 2c0. Base category for warnings about dubious runtime + features. + + +*note FutureWarning: 325. Base category for warnings about deprecated + features when those warnings are intended for end + users of applications that are written in Python. + + +*note PendingDeprecationWarning: 279. Base category for warnings about features that + will be deprecated in the future (ignored by + default). + + +*note ImportWarning: 5d8. Base category for warnings triggered during the + process of importing a module (ignored by + default). + + +*note UnicodeWarning: d87. Base category for warnings related to Unicode. + + +*note BytesWarning: 4b5. Base category for warnings related to + *note bytes: 331. and *note bytearray: 332. + + +*note ResourceWarning: 4d0. Base category for warnings related to resource + usage. + + +Changed in version 3.7: Previously *note DeprecationWarning: 278. and +*note FutureWarning: 325. were distinguished based on whether a feature +was being removed entirely or changing its behaviour. They are now +distinguished based on their intended audience and the way they’re +handled by the default warnings filters. + + +File: python.info, Node: The Warnings Filter, Next: Temporarily Suppressing Warnings, Prev: Warning Categories, Up: warnings — Warning control + +5.29.5.2 The Warnings Filter +............................ + +The warnings filter controls whether warnings are ignored, displayed, or +turned into errors (raising an exception). + +Conceptually, the warnings filter maintains an ordered list of filter +specifications; any specific warning is matched against each filter +specification in the list in turn until a match is found; the filter +determines the disposition of the match. Each entry is a tuple of the +form (`action', `message', `category', `module', `lineno'), where: + + * `action' is one of the following strings: + + Value Disposition + + ----------------------------------------------------------------------- + + ‘"default"’ print the first occurrence of matching warnings + for each location (module + line number) where + the warning is issued + + + ‘"error"’ turn matching warnings into exceptions + + + ‘"ignore"’ never print matching warnings + + + ‘"always"’ always print matching warnings + + + ‘"module"’ print the first occurrence of matching warnings + for each module where the warning is issued + (regardless of line number) + + + ‘"once"’ print only the first occurrence of matching + warnings, regardless of location + + + * `message' is a string containing a regular expression that the + start of the warning message must match. The expression is + compiled to always be case-insensitive. + + * `category' is a class (a subclass of *note Warning: 13c2.) of which + the warning category must be a subclass in order to match. + + * `module' is a string containing a regular expression that the + module name must match. The expression is compiled to be + case-sensitive. + + * `lineno' is an integer that the line number where the warning + occurred must match, or ‘0’ to match all line numbers. + +Since the *note Warning: 13c2. class is derived from the built-in *note +Exception: 1a9. class, to turn a warning into an error we simply raise +‘category(message)’. + +If a warning is reported and doesn’t match any registered filter then +the “default” action is applied (hence its name). + +* Menu: + +* Describing Warning Filters:: +* Default Warning Filter:: +* Overriding the default filter:: + + +File: python.info, Node: Describing Warning Filters, Next: Default Warning Filter, Up: The Warnings Filter + +5.29.5.3 Describing Warning Filters +................................... + +The warnings filter is initialized by *note -W: 417. options passed to +the Python interpreter command line and the *note PYTHONWARNINGS: 418. +environment variable. The interpreter saves the arguments for all +supplied entries without interpretation in *note sys.warnoptions: 416.; +the *note warnings: 126. module parses these when it is first imported +(invalid options are ignored, after printing a message to *note +sys.stderr: 30f.). + +Individual warnings filters are specified as a sequence of fields +separated by colons: + + action:message:category:module:line + +The meaning of each of these fields is as described in *note The +Warnings Filter: fe7. When listing multiple filters on a single line +(as for *note PYTHONWARNINGS: 418.), the individual filters are +separated by commas and the filters listed later take precedence over +those listed before them (as they’re applied left-to-right, and the most +recently applied filters take precedence over earlier ones). + +Commonly used warning filters apply to either all warnings, warnings in +a particular category, or warnings raised by particular modules or +packages. Some examples: + + default # Show all warnings (even those ignored by default) + ignore # Ignore all warnings + error # Convert all warnings to errors + error::ResourceWarning # Treat ResourceWarning messages as errors + default::DeprecationWarning # Show DeprecationWarning messages + ignore,default:::mymodule # Only report warnings triggered by "mymodule" + error:::mymodule[.*] # Convert warnings to errors in "mymodule" + # and any subpackages of "mymodule" + + +File: python.info, Node: Default Warning Filter, Next: Overriding the default filter, Prev: Describing Warning Filters, Up: The Warnings Filter + +5.29.5.4 Default Warning Filter +............................... + +By default, Python installs several warning filters, which can be +overridden by the *note -W: 417. command-line option, the *note +PYTHONWARNINGS: 418. environment variable and calls to *note +filterwarnings(): e07. + +In regular release builds, the default warning filter has the following +entries (in order of precedence): + + default::DeprecationWarning:__main__ + ignore::DeprecationWarning + ignore::PendingDeprecationWarning + ignore::ImportWarning + ignore::ResourceWarning + +In debug builds, the list of default warning filters is empty. + +Changed in version 3.2: *note DeprecationWarning: 278. is now ignored by +default in addition to *note PendingDeprecationWarning: 279. + +Changed in version 3.7: *note DeprecationWarning: 278. is once again +shown by default when triggered directly by code in ‘__main__’. + +Changed in version 3.7: *note BytesWarning: 4b5. no longer appears in +the default filter list and is instead configured via *note +sys.warnoptions: 416. when *note -b: 415. is specified twice. + + +File: python.info, Node: Overriding the default filter, Prev: Default Warning Filter, Up: The Warnings Filter + +5.29.5.5 Overriding the default filter +...................................... + +Developers of applications written in Python may wish to hide `all' +Python level warnings from their users by default, and only display them +when running tests or otherwise working on the application. The *note +sys.warnoptions: 416. attribute used to pass filter configurations to +the interpreter can be used as a marker to indicate whether or not +warnings should be disabled: + + import sys + + if not sys.warnoptions: + import warnings + warnings.simplefilter("ignore") + +Developers of test runners for Python code are advised to instead ensure +that `all' warnings are displayed by default for the code under test, +using code like: + + import sys + + if not sys.warnoptions: + import os, warnings + warnings.simplefilter("default") # Change the filter in this process + os.environ["PYTHONWARNINGS"] = "default" # Also affect subprocesses + +Finally, developers of interactive shells that run user code in a +namespace other than ‘__main__’ are advised to ensure that *note +DeprecationWarning: 278. messages are made visible by default, using +code like the following (where ‘user_ns’ is the module used to execute +code entered interactively): + + import warnings + warnings.filterwarnings("default", category=DeprecationWarning, + module=user_ns.get("__name__")) + + +File: python.info, Node: Temporarily Suppressing Warnings, Next: Testing Warnings, Prev: The Warnings Filter, Up: warnings — Warning control + +5.29.5.6 Temporarily Suppressing Warnings +......................................... + +If you are using code that you know will raise a warning, such as a +deprecated function, but do not want to see the warning (even when +warnings have been explicitly configured via the command line), then it +is possible to suppress the warning using the *note catch_warnings: +317b. context manager: + + import warnings + + def fxn(): + warnings.warn("deprecated", DeprecationWarning) + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + fxn() + +While within the context manager all warnings will simply be ignored. +This allows you to use known-deprecated code without having to see the +warning while not suppressing the warning for other code that might not +be aware of its use of deprecated code. Note: this can only be +guaranteed in a single-threaded application. If two or more threads use +the *note catch_warnings: 317b. context manager at the same time, the +behavior is undefined. + + +File: python.info, Node: Testing Warnings, Next: Updating Code For New Versions of Dependencies, Prev: Temporarily Suppressing Warnings, Up: warnings — Warning control + +5.29.5.7 Testing Warnings +......................... + +To test warnings raised by code, use the *note catch_warnings: 317b. +context manager. With it you can temporarily mutate the warnings filter +to facilitate your testing. For instance, do the following to capture +all raised warnings to check: + + import warnings + + def fxn(): + warnings.warn("deprecated", DeprecationWarning) + + with warnings.catch_warnings(record=True) as w: + # Cause all warnings to always be triggered. + warnings.simplefilter("always") + # Trigger a warning. + fxn() + # Verify some things + assert len(w) == 1 + assert issubclass(w[-1].category, DeprecationWarning) + assert "deprecated" in str(w[-1].message) + +One can also cause all warnings to be exceptions by using ‘error’ +instead of ‘always’. One thing to be aware of is that if a warning has +already been raised because of a ‘once’/‘default’ rule, then no matter +what filters are set the warning will not be seen again unless the +warnings registry related to the warning has been cleared. + +Once the context manager exits, the warnings filter is restored to its +state when the context was entered. This prevents tests from changing +the warnings filter in unexpected ways between tests and leading to +indeterminate test results. The *note showwarning(): 3379. function in +the module is also restored to its original value. Note: this can only +be guaranteed in a single-threaded application. If two or more threads +use the *note catch_warnings: 317b. context manager at the same time, +the behavior is undefined. + +When testing multiple operations that raise the same kind of warning, it +is important to test them in a manner that confirms each operation is +raising a new warning (e.g. set warnings to be raised as exceptions and +check the operations raise exceptions, check that the length of the +warning list continues to increase after each operation, or else delete +the previous entries from the warnings list before each new operation). + + +File: python.info, Node: Updating Code For New Versions of Dependencies, Next: Available Functions, Prev: Testing Warnings, Up: warnings — Warning control + +5.29.5.8 Updating Code For New Versions of Dependencies +....................................................... + +Warning categories that are primarily of interest to Python developers +(rather than end users of applications written in Python) are ignored by +default. + +Notably, this “ignored by default” list includes *note +DeprecationWarning: 278. (for every module except ‘__main__’), which +means developers should make sure to test their code with typically +ignored warnings made visible in order to receive timely notifications +of future breaking API changes (whether in the standard library or third +party packages). + +In the ideal case, the code will have a suitable test suite, and the +test runner will take care of implicitly enabling all warnings when +running tests (the test runner provided by the *note unittest: 11b. +module does this). + +In less ideal cases, applications can be checked for use of deprecated +interfaces by passing *note -Wd: 417. to the Python interpreter (this is +shorthand for ‘-W default’) or setting ‘PYTHONWARNINGS=default’ in the +environment. This enables default handling for all warnings, including +those that are ignored by default. To change what action is taken for +encountered warnings you can change what argument is passed to *note -W: +417. (e.g. ‘-W error’). See the *note -W: 417. flag for more details +on what is possible. + + +File: python.info, Node: Available Functions, Next: Available Context Managers, Prev: Updating Code For New Versions of Dependencies, Up: warnings — Warning control + +5.29.5.9 Available Functions +............................ + + -- Function: warnings.warn (message, category=None, stacklevel=1, + source=None) + + Issue a warning, or maybe ignore it or raise an exception. The + `category' argument, if given, must be a *note warning category + class: 13c1.; it defaults to *note UserWarning: 13c3. + Alternatively, `message' can be a *note Warning: 13c2. instance, in + which case `category' will be ignored and ‘message.__class__’ will + be used. In this case, the message text will be ‘str(message)’. + This function raises an exception if the particular warning issued + is changed into an error by the *note warnings filter: fe7. The + `stacklevel' argument can be used by wrapper functions written in + Python, like this: + + def deprecation(message): + warnings.warn(message, DeprecationWarning, stacklevel=2) + + This makes the warning refer to ‘deprecation()’’s caller, rather + than to the source of ‘deprecation()’ itself (since the latter + would defeat the purpose of the warning message). + + `source', if supplied, is the destroyed object which emitted a + *note ResourceWarning: 4d0. + + Changed in version 3.6: Added `source' parameter. + + -- Function: warnings.warn_explicit (message, category, filename, + lineno, module=None, registry=None, module_globals=None, + source=None) + + This is a low-level interface to the functionality of *note warn(): + e58, passing in explicitly the message, category, filename and line + number, and optionally the module name and the registry (which + should be the ‘__warningregistry__’ dictionary of the module). The + module name defaults to the filename with ‘.py’ stripped; if no + registry is passed, the warning is never suppressed. `message' + must be a string and `category' a subclass of *note Warning: 13c2. + or `message' may be a *note Warning: 13c2. instance, in which case + `category' will be ignored. + + `module_globals', if supplied, should be the global namespace in + use by the code for which the warning is issued. (This argument is + used to support displaying source for modules found in zipfiles or + other non-filesystem import sources). + + `source', if supplied, is the destroyed object which emitted a + *note ResourceWarning: 4d0. + + Changed in version 3.6: Add the `source' parameter. + + -- Function: warnings.showwarning (message, category, filename, lineno, + file=None, line=None) + + Write a warning to a file. The default implementation calls + ‘formatwarning(message, category, filename, lineno, line)’ and + writes the resulting string to `file', which defaults to *note + sys.stderr: 30f. You may replace this function with any callable + by assigning to ‘warnings.showwarning’. `line' is a line of source + code to be included in the warning message; if `line' is not + supplied, *note showwarning(): 3379. will try to read the line + specified by `filename' and `lineno'. + + -- Function: warnings.formatwarning (message, category, filename, + lineno, line=None) + + Format a warning the standard way. This returns a string which may + contain embedded newlines and ends in a newline. `line' is a line + of source code to be included in the warning message; if `line' is + not supplied, *note formatwarning(): 1dad. will try to read the + line specified by `filename' and `lineno'. + + -- Function: warnings.filterwarnings (action, message='', + category=Warning, module='', lineno=0, append=False) + + Insert an entry into the list of *note warnings filter + specifications: fe7. The entry is inserted at the front by + default; if `append' is true, it is inserted at the end. This + checks the types of the arguments, compiles the `message' and + `module' regular expressions, and inserts them as a tuple in the + list of warnings filters. Entries closer to the front of the list + override entries later in the list, if both match a particular + warning. Omitted arguments default to a value that matches + everything. + + -- Function: warnings.simplefilter (action, category=Warning, lineno=0, + append=False) + + Insert a simple entry into the list of *note warnings filter + specifications: fe7. The meaning of the function parameters is as + for *note filterwarnings(): e07, but regular expressions are not + needed as the filter inserted always matches any message in any + module as long as the category and line number match. + + -- Function: warnings.resetwarnings () + + Reset the warnings filter. This discards the effect of all + previous calls to *note filterwarnings(): e07, including that of + the *note -W: 417. command line options and calls to *note + simplefilter(): 317c. + + +File: python.info, Node: Available Context Managers, Prev: Available Functions, Up: warnings — Warning control + +5.29.5.10 Available Context Managers +.................................... + + -- Class: warnings.catch_warnings (*, record=False, module=None) + + A context manager that copies and, upon exit, restores the warnings + filter and the *note showwarning(): 3379. function. If the + `record' argument is *note False: 388. (the default) the context + manager returns *note None: 157. on entry. If `record' is *note + True: 499, a list is returned that is progressively populated with + objects as seen by a custom *note showwarning(): 3379. function + (which also suppresses output to ‘sys.stdout’). Each object in the + list has attributes with the same names as the arguments to *note + showwarning(): 3379. + + The `module' argument takes a module that will be used instead of + the module returned when you import *note warnings: 126. whose + filter will be protected. This argument exists primarily for + testing the *note warnings: 126. module itself. + + Note: The *note catch_warnings: 317b. manager works by + replacing and then later restoring the module’s *note + showwarning(): 3379. function and internal list of filter + specifications. This means the context manager is modifying + global state and therefore is not thread-safe. + + +File: python.info, Node: dataclasses — Data Classes, Next: contextlib — Utilities for with-statement contexts, Prev: warnings — Warning control, Up: Python Runtime Services + +5.29.6 ‘dataclasses’ — Data Classes +----------------------------------- + +`Source code:' Lib/dataclasses.py(1) + +__________________________________________________________________ + +This module provides a decorator and functions for automatically adding +generated *note special method: 338b.s such as *note __init__(): d5e. +and *note __repr__(): 33e. to user-defined classes. It was originally +described in PEP 557(2). + +The member variables to use in these generated methods are defined using +PEP 526(3) type annotations. For example this code: + + from dataclasses import dataclass + + @dataclass + class InventoryItem: + """Class for keeping track of an item in inventory.""" + name: str + unit_price: float + quantity_on_hand: int = 0 + + def total_cost(self) -> float: + return self.unit_price * self.quantity_on_hand + +Will add, among other things, a *note __init__(): d5e. that looks like: + + def __init__(self, name: str, unit_price: float, quantity_on_hand: int=0): + self.name = name + self.unit_price = unit_price + self.quantity_on_hand = quantity_on_hand + +Note that this method is automatically added to the class: it is not +directly specified in the ‘InventoryItem’ definition shown above. + +New in version 3.7. + +* Menu: + +* Module-level decorators, classes, and functions: Module-level decorators classes and functions. +* Post-init processing:: +* Class variables:: +* Init-only variables:: +* Frozen instances:: +* Inheritance: Inheritance<2>. +* Default factory functions:: +* Mutable default values:: +* Exceptions: Exceptions<17>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/dataclasses.py + + (2) https://www.python.org/dev/peps/pep-0557 + + (3) https://www.python.org/dev/peps/pep-0526 + + +File: python.info, Node: Module-level decorators classes and functions, Next: Post-init processing, Up: dataclasses — Data Classes + +5.29.6.1 Module-level decorators, classes, and functions +........................................................ + + -- Function: @dataclasses.dataclass (*, init=True, repr=True, eq=True, + order=False, unsafe_hash=False, frozen=False) + + This function is a *note decorator: 283. that is used to add + generated *note special method: 338b.s to classes, as described + below. + + The *note dataclass(): 33d. decorator examines the class to find + ‘field’s. A ‘field’ is defined as class variable that has a *note + type annotation: 61f. With two exceptions described below, nothing + in *note dataclass(): 33d. examines the type specified in the + variable annotation. + + The order of the fields in all of the generated methods is the + order in which they appear in the class definition. + + The *note dataclass(): 33d. decorator will add various “dunder” + methods to the class, described below. If any of the added methods + already exist on the class, the behavior depends on the parameter, + as documented below. The decorator returns the same class that is + called on; no new class is created. + + If *note dataclass(): 33d. is used just as a simple decorator with + no parameters, it acts as if it has the default values documented + in this signature. That is, these three uses of *note dataclass(): + 33d. are equivalent: + + @dataclass + class C: + ... + + @dataclass() + class C: + ... + + @dataclass(init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False) + class C: + ... + + The parameters to *note dataclass(): 33d. are: + + - ‘init’: If true (the default), a *note __init__(): d5e. method + will be generated. + + If the class already defines *note __init__(): d5e, this + parameter is ignored. + + - ‘repr’: If true (the default), a *note __repr__(): 33e. method + will be generated. The generated repr string will have the + class name and the name and repr of each field, in the order + they are defined in the class. Fields that are marked as + being excluded from the repr are not included. For example: + ‘InventoryItem(name='widget', unit_price=3.0, + quantity_on_hand=10)’. + + If the class already defines *note __repr__(): 33e, this + parameter is ignored. + + - ‘eq’: If true (the default), an *note __eq__(): 33f. method + will be generated. This method compares the class as if it + were a tuple of its fields, in order. Both instances in the + comparison must be of the identical type. + + If the class already defines *note __eq__(): 33f, this + parameter is ignored. + + - ‘order’: If true (the default is ‘False’), *note __lt__(): + c34, *note __le__(): ca0, *note __gt__(): ca1, and *note + __ge__(): ca2. methods will be generated. These compare the + class as if it were a tuple of its fields, in order. Both + instances in the comparison must be of the identical type. If + ‘order’ is true and ‘eq’ is false, a *note ValueError: 1fb. is + raised. + + If the class already defines any of *note __lt__(): c34, *note + __le__(): ca0, *note __gt__(): ca1, or *note __ge__(): ca2, + then *note TypeError: 192. is raised. + + - ‘unsafe_hash’: If ‘False’ (the default), a *note __hash__(): + 340. method is generated according to how ‘eq’ and ‘frozen’ + are set. + + *note __hash__(): 340. is used by built-in *note hash(): 9c4, + and when objects are added to hashed collections such as + dictionaries and sets. Having a *note __hash__(): 340. + implies that instances of the class are immutable. Mutability + is a complicated property that depends on the programmer’s + intent, the existence and behavior of *note __eq__(): 33f, and + the values of the ‘eq’ and ‘frozen’ flags in the *note + dataclass(): 33d. decorator. + + By default, *note dataclass(): 33d. will not implicitly add a + *note __hash__(): 340. method unless it is safe to do so. + Neither will it add or change an existing explicitly defined + *note __hash__(): 340. method. Setting the class attribute + ‘__hash__ = None’ has a specific meaning to Python, as + described in the *note __hash__(): 340. documentation. + + If *note __hash__(): 340. is not explicitly defined, or if it + is set to ‘None’, then *note dataclass(): 33d. `may' add an + implicit *note __hash__(): 340. method. Although not + recommended, you can force *note dataclass(): 33d. to create a + *note __hash__(): 340. method with ‘unsafe_hash=True’. This + might be the case if your class is logically immutable but can + nonetheless be mutated. This is a specialized use case and + should be considered carefully. + + Here are the rules governing implicit creation of a *note + __hash__(): 340. method. Note that you cannot both have an + explicit *note __hash__(): 340. method in your dataclass and + set ‘unsafe_hash=True’; this will result in a *note TypeError: + 192. + + If ‘eq’ and ‘frozen’ are both true, by default *note + dataclass(): 33d. will generate a *note __hash__(): 340. + method for you. If ‘eq’ is true and ‘frozen’ is false, *note + __hash__(): 340. will be set to ‘None’, marking it unhashable + (which it is, since it is mutable). If ‘eq’ is false, *note + __hash__(): 340. will be left untouched meaning the *note + __hash__(): 340. method of the superclass will be used (if the + superclass is *note object: 2b0, this means it will fall back + to id-based hashing). + + - ‘frozen’: If true (the default is ‘False’), assigning to + fields will generate an exception. This emulates read-only + frozen instances. If *note __setattr__(): e2c. or *note + __delattr__(): 10d1. is defined in the class, then *note + TypeError: 192. is raised. See the discussion below. + + ‘field’s may optionally specify a default value, using normal + Python syntax: + + @dataclass + class C: + a: int # 'a' has no default value + b: int = 0 # assign a default value for 'b' + + In this example, both ‘a’ and ‘b’ will be included in the added + *note __init__(): d5e. method, which will be defined as: + + def __init__(self, a: int, b: int = 0): + + *note TypeError: 192. will be raised if a field without a default + value follows a field with a default value. This is true either + when this occurs in a single class, or as a result of class + inheritance. + + -- Function: dataclasses.field (*, default=MISSING, + default_factory=MISSING, repr=True, hash=None, init=True, + compare=True, metadata=None) + + For common and simple use cases, no other functionality is + required. There are, however, some dataclass features that require + additional per-field information. To satisfy this need for + additional information, you can replace the default field value + with a call to the provided *note field(): 338d. function. For + example: + + @dataclass + class C: + mylist: List[int] = field(default_factory=list) + + c = C() + c.mylist += [1, 2, 3] + + As shown above, the ‘MISSING’ value is a sentinel object used to + detect if the ‘default’ and ‘default_factory’ parameters are + provided. This sentinel is used because ‘None’ is a valid value + for ‘default’. No code should directly use the ‘MISSING’ value. + + The parameters to *note field(): 338d. are: + + - ‘default’: If provided, this will be the default value for + this field. This is needed because the *note field(): 338d. + call itself replaces the normal position of the default value. + + - ‘default_factory’: If provided, it must be a zero-argument + callable that will be called when a default value is needed + for this field. Among other purposes, this can be used to + specify fields with mutable default values, as discussed + below. It is an error to specify both ‘default’ and + ‘default_factory’. + + - ‘init’: If true (the default), this field is included as a + parameter to the generated *note __init__(): d5e. method. + + - ‘repr’: If true (the default), this field is included in the + string returned by the generated *note __repr__(): 33e. + method. + + - ‘compare’: If true (the default), this field is included in + the generated equality and comparison methods (*note __eq__(): + 33f, *note __gt__(): ca1, et al.). + + - ‘hash’: This can be a bool or ‘None’. If true, this field is + included in the generated *note __hash__(): 340. method. If + ‘None’ (the default), use the value of ‘compare’: this would + normally be the expected behavior. A field should be + considered in the hash if it’s used for comparisons. Setting + this value to anything other than ‘None’ is discouraged. + + One possible reason to set ‘hash=False’ but ‘compare=True’ + would be if a field is expensive to compute a hash value for, + that field is needed for equality testing, and there are other + fields that contribute to the type’s hash value. Even if a + field is excluded from the hash, it will still be used for + comparisons. + + - ‘metadata’: This can be a mapping or None. None is treated as + an empty dict. This value is wrapped in *note + MappingProxyType(): ab6. to make it read-only, and exposed on + the *note Field: 338e. object. It is not used at all by Data + Classes, and is provided as a third-party extension mechanism. + Multiple third-parties can each have their own key, to use as + a namespace in the metadata. + + If the default value of a field is specified by a call to *note + field(): 338d, then the class attribute for this field will be + replaced by the specified ‘default’ value. If no ‘default’ is + provided, then the class attribute will be deleted. The intent is + that after the *note dataclass(): 33d. decorator runs, the class + attributes will all contain the default values for the fields, just + as if the default value itself were specified. For example, after: + + @dataclass + class C: + x: int + y: int = field(repr=False) + z: int = field(repr=False, default=10) + t: int = 20 + + The class attribute ‘C.z’ will be ‘10’, the class attribute ‘C.t’ + will be ‘20’, and the class attributes ‘C.x’ and ‘C.y’ will not be + set. + + -- Class: dataclasses.Field + + *note Field: 338e. objects describe each defined field. These + objects are created internally, and are returned by the *note + fields(): 338f. module-level method (see below). Users should + never instantiate a *note Field: 338e. object directly. Its + documented attributes are: + + - ‘name’: The name of the field. + + - ‘type’: The type of the field. + + - ‘default’, ‘default_factory’, ‘init’, ‘repr’, ‘hash’, + ‘compare’, and ‘metadata’ have the identical meaning and + values as they do in the *note field(): 338d. + declaration. + + Other attributes may exist, but they are private and must not be + inspected or relied on. + + -- Function: dataclasses.fields (class_or_instance) + + Returns a tuple of *note Field: 338e. objects that define the + fields for this dataclass. Accepts either a dataclass, or an + instance of a dataclass. Raises *note TypeError: 192. if not + passed a dataclass or instance of one. Does not return + pseudo-fields which are ‘ClassVar’ or ‘InitVar’. + + -- Function: dataclasses.asdict (instance, *, dict_factory=dict) + + Converts the dataclass ‘instance’ to a dict (by using the factory + function ‘dict_factory’). Each dataclass is converted to a dict of + its fields, as ‘name: value’ pairs. dataclasses, dicts, lists, and + tuples are recursed into. For example: + + @dataclass + class Point: + x: int + y: int + + @dataclass + class C: + mylist: List[Point] + + p = Point(10, 20) + assert asdict(p) == {'x': 10, 'y': 20} + + c = C([Point(0, 0), Point(10, 4)]) + assert asdict(c) == {'mylist': [{'x': 0, 'y': 0}, {'x': 10, 'y': 4}]} + + Raises *note TypeError: 192. if ‘instance’ is not a dataclass + instance. + + -- Function: dataclasses.astuple (instance, *, tuple_factory=tuple) + + Converts the dataclass ‘instance’ to a tuple (by using the factory + function ‘tuple_factory’). Each dataclass is converted to a tuple + of its field values. dataclasses, dicts, lists, and tuples are + recursed into. + + Continuing from the previous example: + + assert astuple(p) == (10, 20) + assert astuple(c) == ([(0, 0), (10, 4)],) + + Raises *note TypeError: 192. if ‘instance’ is not a dataclass + instance. + + -- Function: dataclasses.make_dataclass (cls_name, fields, *, bases=(), + namespace=None, init=True, repr=True, eq=True, order=False, + unsafe_hash=False, frozen=False) + + Creates a new dataclass with name ‘cls_name’, fields as defined in + ‘fields’, base classes as given in ‘bases’, and initialized with a + namespace as given in ‘namespace’. ‘fields’ is an iterable whose + elements are each either ‘name’, ‘(name, type)’, or ‘(name, type, + Field)’. If just ‘name’ is supplied, ‘typing.Any’ is used for + ‘type’. The values of ‘init’, ‘repr’, ‘eq’, ‘order’, + ‘unsafe_hash’, and ‘frozen’ have the same meaning as they do in + *note dataclass(): 33d. + + This function is not strictly required, because any Python + mechanism for creating a new class with ‘__annotations__’ can then + apply the *note dataclass(): 33d. function to convert that class to + a dataclass. This function is provided as a convenience. For + example: + + C = make_dataclass('C', + [('x', int), + 'y', + ('z', int, field(default=5))], + namespace={'add_one': lambda self: self.x + 1}) + + Is equivalent to: + + @dataclass + class C: + x: int + y: 'typing.Any' + z: int = 5 + + def add_one(self): + return self.x + 1 + + -- Function: dataclasses.replace (instance, **changes) + + Creates a new object of the same type of ‘instance’, replacing + fields with values from ‘changes’. If ‘instance’ is not a Data + Class, raises *note TypeError: 192. If values in ‘changes’ do not + specify fields, raises *note TypeError: 192. + + The newly returned object is created by calling the *note + __init__(): d5e. method of the dataclass. This ensures that + ‘__post_init__()’, if present, is also called. + + Init-only variables without default values, if any exist, must be + specified on the call to *note replace(): 3393. so that they can be + passed to *note __init__(): d5e. and ‘__post_init__()’. + + It is an error for ‘changes’ to contain any fields that are defined + as having ‘init=False’. A *note ValueError: 1fb. will be raised in + this case. + + Be forewarned about how ‘init=False’ fields work during a call to + *note replace(): 3393. They are not copied from the source object, + but rather are initialized in ‘__post_init__()’, if they’re + initialized at all. It is expected that ‘init=False’ fields will + be rarely and judiciously used. If they are used, it might be wise + to have alternate class constructors, or perhaps a custom + ‘replace()’ (or similarly named) method which handles instance + copying. + + -- Function: dataclasses.is_dataclass (class_or_instance) + + Return ‘True’ if its parameter is a dataclass or an instance of + one, otherwise return ‘False’. + + If you need to know if a class is an instance of a dataclass (and + not a dataclass itself), then add a further check for ‘not + isinstance(obj, type)’: + + def is_dataclass_instance(obj): + return is_dataclass(obj) and not isinstance(obj, type) + + +File: python.info, Node: Post-init processing, Next: Class variables, Prev: Module-level decorators classes and functions, Up: dataclasses — Data Classes + +5.29.6.2 Post-init processing +............................. + +The generated *note __init__(): d5e. code will call a method named +‘__post_init__()’, if ‘__post_init__()’ is defined on the class. It +will normally be called as ‘self.__post_init__()’. However, if any +‘InitVar’ fields are defined, they will also be passed to +‘__post_init__()’ in the order they were defined in the class. If no +*note __init__(): d5e. method is generated, then ‘__post_init__()’ will +not automatically be called. + +Among other uses, this allows for initializing field values that depend +on one or more other fields. For example: + + @dataclass + class C: + a: float + b: float + c: float = field(init=False) + + def __post_init__(self): + self.c = self.a + self.b + +See the section below on init-only variables for ways to pass parameters +to ‘__post_init__()’. Also see the warning about how *note replace(): +3393. handles ‘init=False’ fields. + + +File: python.info, Node: Class variables, Next: Init-only variables, Prev: Post-init processing, Up: dataclasses — Data Classes + +5.29.6.3 Class variables +........................ + +One of two places where *note dataclass(): 33d. actually inspects the +type of a field is to determine if a field is a class variable as +defined in PEP 526(1). It does this by checking if the type of the +field is ‘typing.ClassVar’. If a field is a ‘ClassVar’, it is excluded +from consideration as a field and is ignored by the dataclass +mechanisms. Such ‘ClassVar’ pseudo-fields are not returned by the +module-level *note fields(): 338f. function. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0526 + + +File: python.info, Node: Init-only variables, Next: Frozen instances, Prev: Class variables, Up: dataclasses — Data Classes + +5.29.6.4 Init-only variables +............................ + +The other place where *note dataclass(): 33d. inspects a type annotation +is to determine if a field is an init-only variable. It does this by +seeing if the type of a field is of type ‘dataclasses.InitVar’. If a +field is an ‘InitVar’, it is considered a pseudo-field called an +init-only field. As it is not a true field, it is not returned by the +module-level *note fields(): 338f. function. Init-only fields are added +as parameters to the generated *note __init__(): d5e. method, and are +passed to the optional ‘__post_init__()’ method. They are not otherwise +used by dataclasses. + +For example, suppose a field will be initialized from a database, if a +value is not provided when creating the class: + + @dataclass + class C: + i: int + j: int = None + database: InitVar[DatabaseType] = None + + def __post_init__(self, database): + if self.j is None and database is not None: + self.j = database.lookup('j') + + c = C(10, database=my_database) + +In this case, *note fields(): 338f. will return *note Field: 338e. +objects for ‘i’ and ‘j’, but not for ‘database’. + + +File: python.info, Node: Frozen instances, Next: Inheritance<2>, Prev: Init-only variables, Up: dataclasses — Data Classes + +5.29.6.5 Frozen instances +......................... + +It is not possible to create truly immutable Python objects. However, +by passing ‘frozen=True’ to the *note dataclass(): 33d. decorator you +can emulate immutability. In that case, dataclasses will add *note +__setattr__(): e2c. and *note __delattr__(): 10d1. methods to the class. +These methods will raise a *note FrozenInstanceError: 3399. when +invoked. + +There is a tiny performance penalty when using ‘frozen=True’: *note +__init__(): d5e. cannot use simple assignment to initialize fields, and +must use *note object.__setattr__(): e2c. + + +File: python.info, Node: Inheritance<2>, Next: Default factory functions, Prev: Frozen instances, Up: dataclasses — Data Classes + +5.29.6.6 Inheritance +.................... + +When the dataclass is being created by the *note dataclass(): 33d. +decorator, it looks through all of the class’s base classes in reverse +MRO (that is, starting at *note object: 2b0.) and, for each dataclass +that it finds, adds the fields from that base class to an ordered +mapping of fields. After all of the base class fields are added, it +adds its own fields to the ordered mapping. All of the generated +methods will use this combined, calculated ordered mapping of fields. +Because the fields are in insertion order, derived classes override base +classes. An example: + + @dataclass + class Base: + x: Any = 15.0 + y: int = 0 + + @dataclass + class C(Base): + z: int = 10 + x: int = 15 + +The final list of fields is, in order, ‘x’, ‘y’, ‘z’. The final type of +‘x’ is ‘int’, as specified in class ‘C’. + +The generated *note __init__(): d5e. method for ‘C’ will look like: + + def __init__(self, x: int = 15, y: int = 0, z: int = 10): + + +File: python.info, Node: Default factory functions, Next: Mutable default values, Prev: Inheritance<2>, Up: dataclasses — Data Classes + +5.29.6.7 Default factory functions +.................................. + + If a *note field(): 338d. specifies a ‘default_factory’, it is + called with zero arguments when a default value for the field is + needed. For example, to create a new instance of a list, use: + + mylist: list = field(default_factory=list) + + If a field is excluded from *note __init__(): d5e. (using + ‘init=False’) and the field also specifies ‘default_factory’, then + the default factory function will always be called from the + generated *note __init__(): d5e. function. This happens because + there is no other way to give the field an initial value. + + +File: python.info, Node: Mutable default values, Next: Exceptions<17>, Prev: Default factory functions, Up: dataclasses — Data Classes + +5.29.6.8 Mutable default values +............................... + + Python stores default member variable values in class attributes. + Consider this example, not using dataclasses: + + class C: + x = [] + def add(self, element): + self.x.append(element) + + o1 = C() + o2 = C() + o1.add(1) + o2.add(2) + assert o1.x == [1, 2] + assert o1.x is o2.x + + Note that the two instances of class ‘C’ share the same class + variable ‘x’, as expected. + + Using dataclasses, `if' this code was valid: + + @dataclass + class D: + x: List = [] + def add(self, element): + self.x += element + + it would generate code similar to: + + class D: + x = [] + def __init__(self, x=x): + self.x = x + def add(self, element): + self.x += element + + assert D().x is D().x + + This has the same issue as the original example using class ‘C’. + That is, two instances of class ‘D’ that do not specify a value for + ‘x’ when creating a class instance will share the same copy of ‘x’. + Because dataclasses just use normal Python class creation they also + share this behavior. There is no general way for Data Classes to + detect this condition. Instead, dataclasses will raise a *note + TypeError: 192. if it detects a default parameter of type ‘list’, + ‘dict’, or ‘set’. This is a partial solution, but it does protect + against many common errors. + + Using default factory functions is a way to create new instances of + mutable types as default values for fields: + + @dataclass + class D: + x: list = field(default_factory=list) + + assert D().x is not D().x + + +File: python.info, Node: Exceptions<17>, Prev: Mutable default values, Up: dataclasses — Data Classes + +5.29.6.9 Exceptions +................... + + -- Exception: dataclasses.FrozenInstanceError + + Raised when an implicitly defined *note __setattr__(): e2c. or + *note __delattr__(): 10d1. is called on a dataclass which was + defined with ‘frozen=True’. + + +File: python.info, Node: contextlib — Utilities for with-statement contexts, Next: abc — Abstract Base Classes, Prev: dataclasses — Data Classes, Up: Python Runtime Services + +5.29.7 ‘contextlib’ — Utilities for ‘with’-statement contexts +------------------------------------------------------------- + +`Source code:' Lib/contextlib.py(1) + +__________________________________________________________________ + +This module provides utilities for common tasks involving the *note +with: 6e9. statement. For more information see also *note Context +Manager Types: 112a. and *note With Statement Context Managers: 1128. + +* Menu: + +* Utilities:: +* Examples and Recipes: Examples and Recipes<2>. +* Single use, reusable and reentrant context managers: Single use reusable and reentrant context managers. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/contextlib.py + + +File: python.info, Node: Utilities, Next: Examples and Recipes<2>, Up: contextlib — Utilities for with-statement contexts + +5.29.7.1 Utilities +.................. + +Functions and classes provided: + + -- Class: contextlib.AbstractContextManager + + An *note abstract base class: b33. for classes that implement *note + object.__enter__(): c95. and *note object.__exit__(): c96. A + default implementation for *note object.__enter__(): c95. is + provided which returns ‘self’ while *note object.__exit__(): c96. + is an abstract method which by default returns ‘None’. See also + the definition of *note Context Manager Types: 112a. + + New in version 3.6. + + -- Class: contextlib.AbstractAsyncContextManager + + An *note abstract base class: b33. for classes that implement *note + object.__aenter__(): 113b. and *note object.__aexit__(): 113c. A + default implementation for *note object.__aenter__(): 113b. is + provided which returns ‘self’ while *note object.__aexit__(): 113c. + is an abstract method which by default returns ‘None’. See also + the definition of *note Asynchronous Context Managers: 1139. + + New in version 3.7. + + -- Function: @contextlib.contextmanager + + This function is a *note decorator: 283. that can be used to define + a factory function for *note with: 6e9. statement context managers, + without needing to create a class or separate *note __enter__(): + c95. and *note __exit__(): c96. methods. + + While many objects natively support use in with statements, + sometimes a resource needs to be managed that isn’t a context + manager in its own right, and doesn’t implement a ‘close()’ method + for use with ‘contextlib.closing’ + + An abstract example would be the following to ensure correct + resource management: + + from contextlib import contextmanager + + @contextmanager + def managed_resource(*args, **kwds): + # Code to acquire resource, e.g.: + resource = acquire_resource(*args, **kwds) + try: + yield resource + finally: + # Code to release resource, e.g.: + release_resource(resource) + + >>> with managed_resource(timeout=3600) as resource: + ... # Resource is released at the end of this block, + ... # even if code in the block raises an exception + + The function being decorated must return a *note generator: + 9a2.-iterator when called. This iterator must yield exactly one + value, which will be bound to the targets in the *note with: 6e9. + statement’s ‘as’ clause, if any. + + At the point where the generator yields, the block nested in the + *note with: 6e9. statement is executed. The generator is then + resumed after the block is exited. If an unhandled exception + occurs in the block, it is reraised inside the generator at the + point where the yield occurred. Thus, you can use a *note try: + d72.…*note except: b3e.…*note finally: 182. statement to trap the + error (if any), or ensure that some cleanup takes place. If an + exception is trapped merely in order to log it or to perform some + action (rather than to suppress it entirely), the generator must + reraise that exception. Otherwise the generator context manager + will indicate to the ‘with’ statement that the exception has been + handled, and execution will resume with the statement immediately + following the ‘with’ statement. + + *note contextmanager(): b7e. uses *note ContextDecorator: b7d. so + the context managers it creates can be used as decorators as well + as in *note with: 6e9. statements. When used as a decorator, a new + generator instance is implicitly created on each function call + (this allows the otherwise “one-shot” context managers created by + *note contextmanager(): b7e. to meet the requirement that context + managers support multiple invocations in order to be used as + decorators). + + Changed in version 3.2: Use of *note ContextDecorator: b7d. + + -- Function: @contextlib.asynccontextmanager + + Similar to *note contextmanager(): b7e, but creates an *note + asynchronous context manager: 1139. + + This function is a *note decorator: 283. that can be used to define + a factory function for *note async with: 643. statement + asynchronous context managers, without needing to create a class or + separate *note __aenter__(): 113b. and *note __aexit__(): 113c. + methods. It must be applied to an *note asynchronous generator: + 11a9. function. + + A simple example: + + from contextlib import asynccontextmanager + + @asynccontextmanager + async def get_connection(): + conn = await acquire_db_connection() + try: + yield conn + finally: + await release_db_connection(conn) + + async def get_all_users(): + async with get_connection() as conn: + return conn.query('SELECT ...') + + New in version 3.7. + + -- Function: contextlib.closing (thing) + + Return a context manager that closes `thing' upon completion of the + block. This is basically equivalent to: + + from contextlib import contextmanager + + @contextmanager + def closing(thing): + try: + yield thing + finally: + thing.close() + + And lets you write code like this: + + from contextlib import closing + from urllib.request import urlopen + + with closing(urlopen('http://www.python.org')) as page: + for line in page: + print(line) + + without needing to explicitly close ‘page’. Even if an error + occurs, ‘page.close()’ will be called when the *note with: 6e9. + block is exited. + -- Function: contextlib.nullcontext (enter_result=None) + + Return a context manager that returns `enter_result' from + ‘__enter__’, but otherwise does nothing. It is intended to be used + as a stand-in for an optional context manager, for example: + + def myfunction(arg, ignore_exceptions=False): + if ignore_exceptions: + # Use suppress to ignore all exceptions. + cm = contextlib.suppress(Exception) + else: + # Do not ignore any exceptions, cm has no effect. + cm = contextlib.nullcontext() + with cm: + # Do something + + An example using `enter_result': + + def process_file(file_or_path): + if isinstance(file_or_path, str): + # If string, open file + cm = open(file_or_path) + else: + # Caller is responsible for closing file + cm = nullcontext(file_or_path) + + with cm as file: + # Perform processing on the file + + New in version 3.7. + + -- Function: contextlib.suppress (*exceptions) + + Return a context manager that suppresses any of the specified + exceptions if they occur in the body of a with statement and then + resumes execution with the first statement following the end of the + with statement. + + As with any other mechanism that completely suppresses exceptions, + this context manager should be used only to cover very specific + errors where silently continuing with program execution is known to + be the right thing to do. + + For example: + + from contextlib import suppress + + with suppress(FileNotFoundError): + os.remove('somefile.tmp') + + with suppress(FileNotFoundError): + os.remove('someotherfile.tmp') + + This code is equivalent to: + + try: + os.remove('somefile.tmp') + except FileNotFoundError: + pass + + try: + os.remove('someotherfile.tmp') + except FileNotFoundError: + pass + + This context manager is *note reentrant: 33a3. + + New in version 3.4. + + -- Function: contextlib.redirect_stdout (new_target) + + Context manager for temporarily redirecting *note sys.stdout: 30e. + to another file or file-like object. + + This tool adds flexibility to existing functions or classes whose + output is hardwired to stdout. + + For example, the output of *note help(): 572. normally is sent to + `sys.stdout'. You can capture that output in a string by + redirecting the output to an *note io.StringIO: 82d. object: + + f = io.StringIO() + with redirect_stdout(f): + help(pow) + s = f.getvalue() + + To send the output of *note help(): 572. to a file on disk, + redirect the output to a regular file: + + with open('help.txt', 'w') as f: + with redirect_stdout(f): + help(pow) + + To send the output of *note help(): 572. to `sys.stderr': + + with redirect_stdout(sys.stderr): + help(pow) + + Note that the global side effect on *note sys.stdout: 30e. means + that this context manager is not suitable for use in library code + and most threaded applications. It also has no effect on the + output of subprocesses. However, it is still a useful approach for + many utility scripts. + + This context manager is *note reentrant: 33a3. + + New in version 3.4. + + -- Function: contextlib.redirect_stderr (new_target) + + Similar to *note redirect_stdout(): 6c2. but redirecting *note + sys.stderr: 30f. to another file or file-like object. + + This context manager is *note reentrant: 33a3. + + New in version 3.5. + + -- Class: contextlib.ContextDecorator + + A base class that enables a context manager to also be used as a + decorator. + + Context managers inheriting from ‘ContextDecorator’ have to + implement ‘__enter__’ and ‘__exit__’ as normal. ‘__exit__’ retains + its optional exception handling even when used as a decorator. + + ‘ContextDecorator’ is used by *note contextmanager(): b7e, so you + get this functionality automatically. + + Example of ‘ContextDecorator’: + + from contextlib import ContextDecorator + + class mycontext(ContextDecorator): + def __enter__(self): + print('Starting') + return self + + def __exit__(self, *exc): + print('Finishing') + return False + + >>> @mycontext() + ... def function(): + ... print('The bit in the middle') + ... + >>> function() + Starting + The bit in the middle + Finishing + + >>> with mycontext(): + ... print('The bit in the middle') + ... + Starting + The bit in the middle + Finishing + + This change is just syntactic sugar for any construct of the + following form: + + def f(): + with cm(): + # Do stuff + + ‘ContextDecorator’ lets you instead write: + + @cm() + def f(): + # Do stuff + + It makes it clear that the ‘cm’ applies to the whole function, + rather than just a piece of it (and saving an indentation level is + nice, too). + + Existing context managers that already have a base class can be + extended by using ‘ContextDecorator’ as a mixin class: + + from contextlib import ContextDecorator + + class mycontext(ContextBaseClass, ContextDecorator): + def __enter__(self): + return self + + def __exit__(self, *exc): + return False + + Note: As the decorated function must be able to be called + multiple times, the underlying context manager must support + use in multiple *note with: 6e9. statements. If this is not + the case, then the original construct with the explicit ‘with’ + statement inside the function should be used. + + New in version 3.2. + + -- Class: contextlib.ExitStack + + A context manager that is designed to make it easy to + programmatically combine other context managers and cleanup + functions, especially those that are optional or otherwise driven + by input data. + + For example, a set of files may easily be handled in a single with + statement as follows: + + with ExitStack() as stack: + files = [stack.enter_context(open(fname)) for fname in filenames] + # All opened files will automatically be closed at the end of + # the with statement, even if attempts to open files later + # in the list raise an exception + + Each instance maintains a stack of registered callbacks that are + called in reverse order when the instance is closed (either + explicitly or implicitly at the end of a *note with: 6e9. + statement). Note that callbacks are `not' invoked implicitly when + the context stack instance is garbage collected. + + This stack model is used so that context managers that acquire + their resources in their ‘__init__’ method (such as file objects) + can be handled correctly. + + Since registered callbacks are invoked in the reverse order of + registration, this ends up behaving as if multiple nested *note + with: 6e9. statements had been used with the registered set of + callbacks. This even extends to exception handling - if an inner + callback suppresses or replaces an exception, then outer callbacks + will be passed arguments based on that updated state. + + This is a relatively low level API that takes care of the details + of correctly unwinding the stack of exit callbacks. It provides a + suitable foundation for higher level context managers that + manipulate the exit stack in application specific ways. + + New in version 3.3. + + -- Method: enter_context (cm) + + Enters a new context manager and adds its *note __exit__(): + c96. method to the callback stack. The return value is the + result of the context manager’s own *note __enter__(): c95. + method. + + These context managers may suppress exceptions just as they + normally would if used directly as part of a *note with: 6e9. + statement. + + -- Method: push (exit) + + Adds a context manager’s *note __exit__(): c96. method to the + callback stack. + + As ‘__enter__’ is `not' invoked, this method can be used to + cover part of an *note __enter__(): c95. implementation with a + context manager’s own *note __exit__(): c96. method. + + If passed an object that is not a context manager, this method + assumes it is a callback with the same signature as a context + manager’s *note __exit__(): c96. method and adds it directly + to the callback stack. + + By returning true values, these callbacks can suppress + exceptions the same way context manager *note __exit__(): c96. + methods can. + + The passed in object is returned from the function, allowing + this method to be used as a function decorator. + + -- Method: callback (callback, *args, **kwds) + + Accepts an arbitrary callback function and arguments and adds + it to the callback stack. + + Unlike the other methods, callbacks added this way cannot + suppress exceptions (as they are never passed the exception + details). + + The passed in callback is returned from the function, allowing + this method to be used as a function decorator. + + -- Method: pop_all () + + Transfers the callback stack to a fresh *note ExitStack: 376. + instance and returns it. No callbacks are invoked by this + operation - instead, they will now be invoked when the new + stack is closed (either explicitly or implicitly at the end of + a *note with: 6e9. statement). + + For example, a group of files can be opened as an “all or + nothing” operation as follows: + + with ExitStack() as stack: + files = [stack.enter_context(open(fname)) for fname in filenames] + # Hold onto the close method, but don't call it yet. + close_files = stack.pop_all().close + # If opening any file fails, all previously opened files will be + # closed automatically. If all files are opened successfully, + # they will remain open even after the with statement ends. + # close_files() can then be invoked explicitly to close them all. + + -- Method: close () + + Immediately unwinds the callback stack, invoking callbacks in + the reverse order of registration. For any context managers + and exit callbacks registered, the arguments passed in will + indicate that no exception occurred. + + -- Class: contextlib.AsyncExitStack + + An *note asynchronous context manager: 1139, similar to *note + ExitStack: 376, that supports combining both synchronous and + asynchronous context managers, as well as having coroutines for + cleanup logic. + + The ‘close()’ method is not implemented, *note aclose(): 33a8. must + be used instead. + + -- Method: enter_async_context (cm) + + Similar to ‘enter_context()’ but expects an asynchronous + context manager. + + -- Method: push_async_exit (exit) + + Similar to ‘push()’ but expects either an asynchronous context + manager or a coroutine function. + + -- Method: push_async_callback (callback, *args, **kwds) + + Similar to ‘callback()’ but expects a coroutine function. + + -- Method: aclose () + + Similar to ‘close()’ but properly handles awaitables. + + Continuing the example for *note asynccontextmanager(): 377.: + + async with AsyncExitStack() as stack: + connections = [await stack.enter_async_context(get_connection()) + for i in range(5)] + # All opened connections will automatically be released at the end of + # the async with statement, even if attempts to open a connection + # later in the list raise an exception. + + New in version 3.7. + + +File: python.info, Node: Examples and Recipes<2>, Next: Single use reusable and reentrant context managers, Prev: Utilities, Up: contextlib — Utilities for with-statement contexts + +5.29.7.2 Examples and Recipes +............................. + +This section describes some examples and recipes for making effective +use of the tools provided by *note contextlib: 24. + +* Menu: + +* Supporting a variable number of context managers:: +* Catching exceptions from __enter__ methods:: +* Cleaning up in an __enter__ implementation:: +* Replacing any use of try-finally and flag variables:: +* Using a context manager as a function decorator:: + + +File: python.info, Node: Supporting a variable number of context managers, Next: Catching exceptions from __enter__ methods, Up: Examples and Recipes<2> + +5.29.7.3 Supporting a variable number of context managers +......................................................... + +The primary use case for *note ExitStack: 376. is the one given in the +class documentation: supporting a variable number of context managers +and other cleanup operations in a single *note with: 6e9. statement. +The variability may come from the number of context managers needed +being driven by user input (such as opening a user specified collection +of files), or from some of the context managers being optional: + + with ExitStack() as stack: + for resource in resources: + stack.enter_context(resource) + if need_special_resource(): + special = acquire_special_resource() + stack.callback(release_special_resource, special) + # Perform operations that use the acquired resources + +As shown, *note ExitStack: 376. also makes it quite easy to use *note +with: 6e9. statements to manage arbitrary resources that don’t natively +support the context management protocol. + + +File: python.info, Node: Catching exceptions from __enter__ methods, Next: Cleaning up in an __enter__ implementation, Prev: Supporting a variable number of context managers, Up: Examples and Recipes<2> + +5.29.7.4 Catching exceptions from ‘__enter__’ methods +..................................................... + +It is occasionally desirable to catch exceptions from an ‘__enter__’ +method implementation, `without' inadvertently catching exceptions from +the *note with: 6e9. statement body or the context manager’s ‘__exit__’ +method. By using *note ExitStack: 376. the steps in the context +management protocol can be separated slightly in order to allow this: + + stack = ExitStack() + try: + x = stack.enter_context(cm) + except Exception: + # handle __enter__ exception + else: + with stack: + # Handle normal case + +Actually needing to do this is likely to indicate that the underlying +API should be providing a direct resource management interface for use +with *note try: d72./*note except: b3e./*note finally: 182. statements, +but not all APIs are well designed in that regard. When a context +manager is the only resource management API provided, then *note +ExitStack: 376. can make it easier to handle various situations that +can’t be handled directly in a *note with: 6e9. statement. + + +File: python.info, Node: Cleaning up in an __enter__ implementation, Next: Replacing any use of try-finally and flag variables, Prev: Catching exceptions from __enter__ methods, Up: Examples and Recipes<2> + +5.29.7.5 Cleaning up in an ‘__enter__’ implementation +..................................................... + +As noted in the documentation of *note ExitStack.push(): 33a5, this +method can be useful in cleaning up an already allocated resource if +later steps in the *note __enter__(): c95. implementation fail. + +Here’s an example of doing this for a context manager that accepts +resource acquisition and release functions, along with an optional +validation function, and maps them to the context management protocol: + + from contextlib import contextmanager, AbstractContextManager, ExitStack + + class ResourceManager(AbstractContextManager): + + def __init__(self, acquire_resource, release_resource, check_resource_ok=None): + self.acquire_resource = acquire_resource + self.release_resource = release_resource + if check_resource_ok is None: + def check_resource_ok(resource): + return True + self.check_resource_ok = check_resource_ok + + @contextmanager + def _cleanup_on_error(self): + with ExitStack() as stack: + stack.push(self) + yield + # The validation check passed and didn't raise an exception + # Accordingly, we want to keep the resource, and pass it + # back to our caller + stack.pop_all() + + def __enter__(self): + resource = self.acquire_resource() + with self._cleanup_on_error(): + if not self.check_resource_ok(resource): + msg = "Failed validation for {!r}" + raise RuntimeError(msg.format(resource)) + return resource + + def __exit__(self, *exc_details): + # We don't need to duplicate any of our resource release logic + self.release_resource() + + +File: python.info, Node: Replacing any use of try-finally and flag variables, Next: Using a context manager as a function decorator, Prev: Cleaning up in an __enter__ implementation, Up: Examples and Recipes<2> + +5.29.7.6 Replacing any use of ‘try-finally’ and flag variables +.............................................................. + +A pattern you will sometimes see is a ‘try-finally’ statement with a +flag variable to indicate whether or not the body of the ‘finally’ +clause should be executed. In its simplest form (that can’t already be +handled just by using an ‘except’ clause instead), it looks something +like this: + + cleanup_needed = True + try: + result = perform_operation() + if result: + cleanup_needed = False + finally: + if cleanup_needed: + cleanup_resources() + +As with any ‘try’ statement based code, this can cause problems for +development and review, because the setup code and the cleanup code can +end up being separated by arbitrarily long sections of code. + +*note ExitStack: 376. makes it possible to instead register a callback +for execution at the end of a ‘with’ statement, and then later decide to +skip executing that callback: + + from contextlib import ExitStack + + with ExitStack() as stack: + stack.callback(cleanup_resources) + result = perform_operation() + if result: + stack.pop_all() + +This allows the intended cleanup up behaviour to be made explicit up +front, rather than requiring a separate flag variable. + +If a particular application uses this pattern a lot, it can be +simplified even further by means of a small helper class: + + from contextlib import ExitStack + + class Callback(ExitStack): + def __init__(self, callback, /, *args, **kwds): + super(Callback, self).__init__() + self.callback(callback, *args, **kwds) + + def cancel(self): + self.pop_all() + + with Callback(cleanup_resources) as cb: + result = perform_operation() + if result: + cb.cancel() + +If the resource cleanup isn’t already neatly bundled into a standalone +function, then it is still possible to use the decorator form of *note +ExitStack.callback(): 2a5. to declare the resource cleanup in advance: + + from contextlib import ExitStack + + with ExitStack() as stack: + @stack.callback + def cleanup_resources(): + ... + result = perform_operation() + if result: + stack.pop_all() + +Due to the way the decorator protocol works, a callback function +declared this way cannot take any parameters. Instead, any resources to +be released must be accessed as closure variables. + + +File: python.info, Node: Using a context manager as a function decorator, Prev: Replacing any use of try-finally and flag variables, Up: Examples and Recipes<2> + +5.29.7.7 Using a context manager as a function decorator +........................................................ + +*note ContextDecorator: b7d. makes it possible to use a context manager +in both an ordinary ‘with’ statement and also as a function decorator. + +For example, it is sometimes useful to wrap functions or groups of +statements with a logger that can track the time of entry and time of +exit. Rather than writing both a function decorator and a context +manager for the task, inheriting from *note ContextDecorator: b7d. +provides both capabilities in a single definition: + + from contextlib import ContextDecorator + import logging + + logging.basicConfig(level=logging.INFO) + + class track_entry_and_exit(ContextDecorator): + def __init__(self, name): + self.name = name + + def __enter__(self): + logging.info('Entering: %s', self.name) + + def __exit__(self, exc_type, exc, exc_tb): + logging.info('Exiting: %s', self.name) + +Instances of this class can be used as both a context manager: + + with track_entry_and_exit('widget loader'): + print('Some time consuming activity goes here') + load_widget() + +And also as a function decorator: + + @track_entry_and_exit('widget loader') + def activity(): + print('Some time consuming activity goes here') + load_widget() + +Note that there is one additional limitation when using context managers +as function decorators: there’s no way to access the return value of +*note __enter__(): c95. If that value is needed, then it is still +necessary to use an explicit ‘with’ statement. + +See also +........ + +PEP 343(1) - The “with” statement + + The specification, background, and examples for the Python *note + with: 6e9. statement. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0343 + + +File: python.info, Node: Single use reusable and reentrant context managers, Prev: Examples and Recipes<2>, Up: contextlib — Utilities for with-statement contexts + +5.29.7.8 Single use, reusable and reentrant context managers +............................................................ + +Most context managers are written in a way that means they can only be +used effectively in a *note with: 6e9. statement once. These single use +context managers must be created afresh each time they’re used - +attempting to use them a second time will trigger an exception or +otherwise not work correctly. + +This common limitation means that it is generally advisable to create +context managers directly in the header of the *note with: 6e9. +statement where they are used (as shown in all of the usage examples +above). + +Files are an example of effectively single use context managers, since +the first *note with: 6e9. statement will close the file, preventing any +further IO operations using that file object. + +Context managers created using *note contextmanager(): b7e. are also +single use context managers, and will complain about the underlying +generator failing to yield if an attempt is made to use them a second +time: + + >>> from contextlib import contextmanager + >>> @contextmanager + ... def singleuse(): + ... print("Before") + ... yield + ... print("After") + ... + >>> cm = singleuse() + >>> with cm: + ... pass + ... + Before + After + >>> with cm: + ... pass + ... + Traceback (most recent call last): + ... + RuntimeError: generator didn't yield + +* Menu: + +* Reentrant context managers:: +* Reusable context managers:: + + +File: python.info, Node: Reentrant context managers, Next: Reusable context managers, Up: Single use reusable and reentrant context managers + +5.29.7.9 Reentrant context managers +................................... + +More sophisticated context managers may be “reentrant”. These context +managers can not only be used in multiple *note with: 6e9. statements, +but may also be used `inside' a ‘with’ statement that is already using +the same context manager. + +*note threading.RLock: bef. is an example of a reentrant context +manager, as are *note suppress(): 82c. and *note redirect_stdout(): 6c2. +Here’s a very simple example of reentrant use: + + >>> from contextlib import redirect_stdout + >>> from io import StringIO + >>> stream = StringIO() + >>> write_to_stream = redirect_stdout(stream) + >>> with write_to_stream: + ... print("This is written to the stream rather than stdout") + ... with write_to_stream: + ... print("This is also written to the stream") + ... + >>> print("This is written directly to stdout") + This is written directly to stdout + >>> print(stream.getvalue()) + This is written to the stream rather than stdout + This is also written to the stream + +Real world examples of reentrancy are more likely to involve multiple +functions calling each other and hence be far more complicated than this +example. + +Note also that being reentrant is `not' the same thing as being thread +safe. *note redirect_stdout(): 6c2, for example, is definitely not +thread safe, as it makes a global modification to the system state by +binding *note sys.stdout: 30e. to a different stream. + + +File: python.info, Node: Reusable context managers, Prev: Reentrant context managers, Up: Single use reusable and reentrant context managers + +5.29.7.10 Reusable context managers +................................... + +Distinct from both single use and reentrant context managers are +“reusable” context managers (or, to be completely explicit, “reusable, +but not reentrant” context managers, since reentrant context managers +are also reusable). These context managers support being used multiple +times, but will fail (or otherwise not work correctly) if the specific +context manager instance has already been used in a containing with +statement. + +*note threading.Lock: 2050. is an example of a reusable, but not +reentrant, context manager (for a reentrant lock, it is necessary to use +*note threading.RLock: bef. instead). + +Another example of a reusable, but not reentrant, context manager is +*note ExitStack: 376, as it invokes `all' currently registered callbacks +when leaving any with statement, regardless of where those callbacks +were added: + + >>> from contextlib import ExitStack + >>> stack = ExitStack() + >>> with stack: + ... stack.callback(print, "Callback: from first context") + ... print("Leaving first context") + ... + Leaving first context + Callback: from first context + >>> with stack: + ... stack.callback(print, "Callback: from second context") + ... print("Leaving second context") + ... + Leaving second context + Callback: from second context + >>> with stack: + ... stack.callback(print, "Callback: from outer context") + ... with stack: + ... stack.callback(print, "Callback: from inner context") + ... print("Leaving inner context") + ... print("Leaving outer context") + ... + Leaving inner context + Callback: from inner context + Callback: from outer context + Leaving outer context + +As the output from the example shows, reusing a single stack object +across multiple with statements works correctly, but attempting to nest +them will cause the stack to be cleared at the end of the innermost with +statement, which is unlikely to be desirable behaviour. + +Using separate *note ExitStack: 376. instances instead of reusing a +single instance avoids that problem: + + >>> from contextlib import ExitStack + >>> with ExitStack() as outer_stack: + ... outer_stack.callback(print, "Callback: from outer context") + ... with ExitStack() as inner_stack: + ... inner_stack.callback(print, "Callback: from inner context") + ... print("Leaving inner context") + ... print("Leaving outer context") + ... + Leaving inner context + Callback: from inner context + Leaving outer context + Callback: from outer context + + +File: python.info, Node: abc — Abstract Base Classes, Next: atexit — Exit handlers, Prev: contextlib — Utilities for with-statement contexts, Up: Python Runtime Services + +5.29.8 ‘abc’ — Abstract Base Classes +------------------------------------ + +`Source code:' Lib/abc.py(1) + +__________________________________________________________________ + +This module provides the infrastructure for defining *note abstract base +classes: b33. (ABCs) in Python, as outlined in PEP 3119(2); see the PEP +for why this was added to Python. (See also PEP 3141(3) and the *note +numbers: c1. module regarding a type hierarchy for numbers based on +ABCs.) + +The *note collections: 1e. module has some concrete classes that derive +from ABCs; these can, of course, be further derived. In addition, the +*note collections.abc: 1f. submodule has some ABCs that can be used to +test whether a class or instance provides a particular interface, for +example, if it is hashable or if it is a mapping. + +This module provides the metaclass *note ABCMeta: 819. for defining ABCs +and a helper class *note ABC: 818. to alternatively define ABCs through +inheritance: + + -- Class: abc.ABC + + A helper class that has *note ABCMeta: 819. as its metaclass. With + this class, an abstract base class can be created by simply + deriving from *note ABC: 818. avoiding sometimes confusing + metaclass usage, for example: + + from abc import ABC + + class MyABC(ABC): + pass + + Note that the type of *note ABC: 818. is still *note ABCMeta: 819, + therefore inheriting from *note ABC: 818. requires the usual + precautions regarding metaclass usage, as multiple inheritance may + lead to metaclass conflicts. One may also define an abstract base + class by passing the metaclass keyword and using *note ABCMeta: + 819. directly, for example: + + from abc import ABCMeta + + class MyABC(metaclass=ABCMeta): + pass + + New in version 3.4. + + -- Class: abc.ABCMeta + + Metaclass for defining Abstract Base Classes (ABCs). + + Use this metaclass to create an ABC. An ABC can be subclassed + directly, and then acts as a mix-in class. You can also register + unrelated concrete classes (even built-in classes) and unrelated + ABCs as “virtual subclasses” – these and their descendants will be + considered subclasses of the registering ABC by the built-in *note + issubclass(): 450. function, but the registering ABC won’t show up + in their MRO (Method Resolution Order) nor will method + implementations defined by the registering ABC be callable (not + even via *note super(): 4e2.). (4) + + Classes created with a metaclass of *note ABCMeta: 819. have the + following method: + + -- Method: register (subclass) + + Register `subclass' as a “virtual subclass” of this ABC. For + example: + + from abc import ABC + + class MyABC(ABC): + pass + + MyABC.register(tuple) + + assert issubclass(tuple, MyABC) + assert isinstance((), MyABC) + + Changed in version 3.3: Returns the registered subclass, to + allow usage as a class decorator. + + Changed in version 3.4: To detect calls to *note register(): + 9d1, you can use the *note get_cache_token(): 817. function. + + You can also override this method in an abstract base class: + + -- Method: __subclasshook__ (subclass) + + (Must be defined as a class method.) + + Check whether `subclass' is considered a subclass of this ABC. + This means that you can customize the behavior of ‘issubclass’ + further without the need to call *note register(): 9d1. on + every class you want to consider a subclass of the ABC. (This + class method is called from the ‘__subclasscheck__()’ method + of the ABC.) + + This method should return ‘True’, ‘False’ or ‘NotImplemented’. + If it returns ‘True’, the `subclass' is considered a subclass + of this ABC. If it returns ‘False’, the `subclass' is not + considered a subclass of this ABC, even if it would normally + be one. If it returns ‘NotImplemented’, the subclass check is + continued with the usual mechanism. + + For a demonstration of these concepts, look at this example ABC + definition: + + class Foo: + def __getitem__(self, index): + ... + def __len__(self): + ... + def get_iterator(self): + return iter(self) + + class MyIterable(ABC): + + @abstractmethod + def __iter__(self): + while False: + yield None + + def get_iterator(self): + return self.__iter__() + + @classmethod + def __subclasshook__(cls, C): + if cls is MyIterable: + if any("__iter__" in B.__dict__ for B in C.__mro__): + return True + return NotImplemented + + MyIterable.register(Foo) + + The ABC ‘MyIterable’ defines the standard iterable method, *note + __iter__(): 12d5, as an abstract method. The implementation given + here can still be called from subclasses. The ‘get_iterator()’ + method is also part of the ‘MyIterable’ abstract base class, but it + does not have to be overridden in non-abstract derived classes. + + The *note __subclasshook__(): 33b7. class method defined here says + that any class that has an *note __iter__(): 12d5. method in its + *note __dict__: 4fc. (or in that of one of its base classes, + accessed via the *note __mro__: 12af. list) is considered a + ‘MyIterable’ too. + + Finally, the last line makes ‘Foo’ a virtual subclass of + ‘MyIterable’, even though it does not define an *note __iter__(): + 12d5. method (it uses the old-style iterable protocol, defined in + terms of *note __len__(): dcb. and *note __getitem__(): 27c.). + Note that this will not make ‘get_iterator’ available as a method + of ‘Foo’, so it is provided separately. + +The *note abc: 4. module also provides the following decorator: + + -- Function: @abc.abstractmethod + + A decorator indicating abstract methods. + + Using this decorator requires that the class’s metaclass is *note + ABCMeta: 819. or is derived from it. A class that has a metaclass + derived from *note ABCMeta: 819. cannot be instantiated unless all + of its abstract methods and properties are overridden. The + abstract methods can be called using any of the normal ‘super’ call + mechanisms. *note abstractmethod(): 9ce. may be used to declare + abstract methods for properties and descriptors. + + Dynamically adding abstract methods to a class, or attempting to + modify the abstraction status of a method or class once it is + created, are not supported. The *note abstractmethod(): 9ce. only + affects subclasses derived using regular inheritance; “virtual + subclasses” registered with the ABC’s ‘register()’ method are not + affected. + + When *note abstractmethod(): 9ce. is applied in combination with + other method descriptors, it should be applied as the innermost + decorator, as shown in the following usage examples: + + class C(ABC): + @abstractmethod + def my_abstract_method(self, ...): + ... + @classmethod + @abstractmethod + def my_abstract_classmethod(cls, ...): + ... + @staticmethod + @abstractmethod + def my_abstract_staticmethod(...): + ... + + @property + @abstractmethod + def my_abstract_property(self): + ... + @my_abstract_property.setter + @abstractmethod + def my_abstract_property(self, val): + ... + + @abstractmethod + def _get_x(self): + ... + @abstractmethod + def _set_x(self, val): + ... + x = property(_get_x, _set_x) + + In order to correctly interoperate with the abstract base class + machinery, the descriptor must identify itself as abstract using + ‘__isabstractmethod__’. In general, this attribute should be + ‘True’ if any of the methods used to compose the descriptor are + abstract. For example, Python’s built-in *note property: 1d7. does + the equivalent of: + + class Descriptor: + ... + @property + def __isabstractmethod__(self): + return any(getattr(f, '__isabstractmethod__', False) for + f in (self._fget, self._fset, self._fdel)) + + Note: Unlike Java abstract methods, these abstract methods may + have an implementation. This implementation can be called via + the *note super(): 4e2. mechanism from the class that + overrides it. This could be useful as an end-point for a + super-call in a framework that uses cooperative + multiple-inheritance. + +The *note abc: 4. module also supports the following legacy decorators: + + -- Function: @abc.abstractclassmethod + + New in version 3.2. + + Deprecated since version 3.3: It is now possible to use *note + classmethod: 1d8. with *note abstractmethod(): 9ce, making this + decorator redundant. + + A subclass of the built-in *note classmethod(): 1d8, indicating an + abstract classmethod. Otherwise it is similar to *note + abstractmethod(): 9ce. + + This special case is deprecated, as the *note classmethod(): 1d8. + decorator is now correctly identified as abstract when applied to + an abstract method: + + class C(ABC): + @classmethod + @abstractmethod + def my_abstract_classmethod(cls, ...): + ... + + -- Function: @abc.abstractstaticmethod + + New in version 3.2. + + Deprecated since version 3.3: It is now possible to use *note + staticmethod: 1d9. with *note abstractmethod(): 9ce, making this + decorator redundant. + + A subclass of the built-in *note staticmethod(): 1d9, indicating an + abstract staticmethod. Otherwise it is similar to *note + abstractmethod(): 9ce. + + This special case is deprecated, as the *note staticmethod(): 1d9. + decorator is now correctly identified as abstract when applied to + an abstract method: + + class C(ABC): + @staticmethod + @abstractmethod + def my_abstract_staticmethod(...): + ... + + -- Function: @abc.abstractproperty + + Deprecated since version 3.3: It is now possible to use *note + property: 1d7, ‘property.getter()’, ‘property.setter()’ and + ‘property.deleter()’ with *note abstractmethod(): 9ce, making this + decorator redundant. + + A subclass of the built-in *note property(): 1d7, indicating an + abstract property. + + This special case is deprecated, as the *note property(): 1d7. + decorator is now correctly identified as abstract when applied to + an abstract method: + + class C(ABC): + @property + @abstractmethod + def my_abstract_property(self): + ... + + The above example defines a read-only property; you can also define + a read-write abstract property by appropriately marking one or more + of the underlying methods as abstract: + + class C(ABC): + @property + def x(self): + ... + + @x.setter + @abstractmethod + def x(self, val): + ... + + If only some components are abstract, only those components need to + be updated to create a concrete property in a subclass: + + class D(C): + @C.x.setter + def x(self, val): + ... + +The *note abc: 4. module also provides the following functions: + + -- Function: abc.get_cache_token () + + Returns the current abstract base class cache token. + + The token is an opaque object (that supports equality testing) + identifying the current version of the abstract base class cache + for virtual subclasses. The token changes with every call to *note + ABCMeta.register(): 9d1. on any ABC. + + New in version 3.4. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/abc.py + + (2) https://www.python.org/dev/peps/pep-3119 + + (3) https://www.python.org/dev/peps/pep-3141 + + (4) (1) C++ programmers should note that Python’s virtual base class +concept is not the same as C++’s. + + +File: python.info, Node: atexit — Exit handlers, Next: traceback — Print or retrieve a stack traceback, Prev: abc — Abstract Base Classes, Up: Python Runtime Services + +5.29.9 ‘atexit’ — Exit handlers +------------------------------- + +__________________________________________________________________ + +The *note atexit: c. module defines functions to register and unregister +cleanup functions. Functions thus registered are automatically executed +upon normal interpreter termination. *note atexit: c. runs these +functions in the `reverse' order in which they were registered; if you +register ‘A’, ‘B’, and ‘C’, at interpreter termination time they will be +run in the order ‘C’, ‘B’, ‘A’. + +`Note:' The functions registered via this module are not called when the +program is killed by a signal not handled by Python, when a Python fatal +internal error is detected, or when *note os._exit(): 13b6. is called. + +Changed in version 3.7: When used with C-API subinterpreters, registered +functions are local to the interpreter they were registered in. + + -- Function: atexit.register (func, *args, **kwargs) + + Register `func' as a function to be executed at termination. Any + optional arguments that are to be passed to `func' must be passed + as arguments to *note register(): e85. It is possible to register + the same function and arguments more than once. + + At normal program termination (for instance, if *note sys.exit(): + ce2. is called or the main module’s execution completes), all + functions registered are called in last in, first out order. The + assumption is that lower level modules will normally be imported + before higher level modules and thus must be cleaned up later. + + If an exception is raised during execution of the exit handlers, a + traceback is printed (unless *note SystemExit: 5fc. is raised) and + the exception information is saved. After all exit handlers have + had a chance to run the last exception to be raised is re-raised. + + This function returns `func', which makes it possible to use it as + a decorator. + + -- Function: atexit.unregister (func) + + Remove `func' from the list of functions to be run at interpreter + shutdown. After calling *note unregister(): 33ba, `func' is + guaranteed not to be called when the interpreter shuts down, even + if it was registered more than once. *note unregister(): 33ba. + silently does nothing if `func' was not previously registered. + +See also +........ + +Module *note readline: de. + + Useful example of *note atexit: c. to read and write *note + readline: de. history files. + +* Menu: + +* atexit Example:: + + +File: python.info, Node: atexit Example, Up: atexit — Exit handlers + +5.29.9.1 ‘atexit’ Example +......................... + +The following simple example demonstrates how a module can initialize a +counter from a file when it is imported and save the counter’s updated +value automatically when the program terminates without relying on the +application making an explicit call into this module at termination. + + try: + with open("counterfile") as infile: + _count = int(infile.read()) + except FileNotFoundError: + _count = 0 + + def incrcounter(n): + global _count + _count = _count + n + + def savecounter(): + with open("counterfile", "w") as outfile: + outfile.write("%d" % _count) + + import atexit + atexit.register(savecounter) + +Positional and keyword arguments may also be passed to *note register(): +e85. to be passed along to the registered function when it is called: + + def goodbye(name, adjective): + print('Goodbye, %s, it was %s to meet you.' % (name, adjective)) + + import atexit + atexit.register(goodbye, 'Donny', 'nice') + + # or: + atexit.register(goodbye, adjective='nice', name='Donny') + +Usage as a *note decorator: 283.: + + import atexit + + @atexit.register + def goodbye(): + print("You are now leaving the Python sector.") + +This only works with functions that can be called without arguments. + + +File: python.info, Node: traceback — Print or retrieve a stack traceback, Next: __future__ — Future statement definitions, Prev: atexit — Exit handlers, Up: Python Runtime Services + +5.29.10 ‘traceback’ — Print or retrieve a stack traceback +--------------------------------------------------------- + +`Source code:' Lib/traceback.py(1) + +__________________________________________________________________ + +This module provides a standard interface to extract, format and print +stack traces of Python programs. It exactly mimics the behavior of the +Python interpreter when it prints a stack trace. This is useful when +you want to print stack traces under program control, such as in a +“wrapper” around the interpreter. + +The module uses traceback objects — this is the object type that is +stored in the *note sys.last_traceback: 325a. variable and returned as +the third item from *note sys.exc_info(): c5f. + +The module defines the following functions: + + -- Function: traceback.print_tb (tb, limit=None, file=None) + + Print up to `limit' stack trace entries from traceback object `tb' + (starting from the caller’s frame) if `limit' is positive. + Otherwise, print the last ‘abs(limit)’ entries. If `limit' is + omitted or ‘None’, all entries are printed. If `file' is omitted + or ‘None’, the output goes to ‘sys.stderr’; otherwise it should be + an open file or file-like object to receive the output. + + Changed in version 3.5: Added negative `limit' support. + + -- Function: traceback.print_exception (etype, value, tb, limit=None, + file=None, chain=True) + + Print exception information and stack trace entries from traceback + object `tb' to `file'. This differs from *note print_tb(): 779. in + the following ways: + + * if `tb' is not ‘None’, it prints a header ‘Traceback (most + recent call last):’ + + * it prints the exception `etype' and `value' after the stack + trace + + * if `type(value)' is *note SyntaxError: 458. and `value' has + the appropriate format, it prints the line where the syntax + error occurred with a caret indicating the approximate + position of the error. + + The optional `limit' argument has the same meaning as for *note + print_tb(): 779. If `chain' is true (the default), then chained + exceptions (the ‘__cause__’ or ‘__context__’ attributes of the + exception) will be printed as well, like the interpreter itself + does when printing an unhandled exception. + + Changed in version 3.5: The `etype' argument is ignored and + inferred from the type of `value'. + + -- Function: traceback.print_exc (limit=None, file=None, chain=True) + + This is a shorthand for ‘print_exception(*sys.exc_info(), limit, + file, chain)’. + + -- Function: traceback.print_last (limit=None, file=None, chain=True) + + This is a shorthand for ‘print_exception(sys.last_type, + sys.last_value, sys.last_traceback, limit, file, chain)’. In + general it will work only after an exception has reached an + interactive prompt (see *note sys.last_type: c5a.). + + -- Function: traceback.print_stack (f=None, limit=None, file=None) + + Print up to `limit' stack trace entries (starting from the + invocation point) if `limit' is positive. Otherwise, print the + last ‘abs(limit)’ entries. If `limit' is omitted or ‘None’, all + entries are printed. The optional `f' argument can be used to + specify an alternate stack frame to start. The optional `file' + argument has the same meaning as for *note print_tb(): 779. + + Changed in version 3.5: Added negative `limit' support. + + -- Function: traceback.extract_tb (tb, limit=None) + + Return a *note StackSummary: 777. object representing a list of + “pre-processed” stack trace entries extracted from the traceback + object `tb'. It is useful for alternate formatting of stack + traces. The optional `limit' argument has the same meaning as for + *note print_tb(): 779. A “pre-processed” stack trace entry is a + *note FrameSummary: 778. object containing attributes ‘filename’, + ‘lineno’, ‘name’, and ‘line’ representing the information that is + usually printed for a stack trace. The ‘line’ is a string with + leading and trailing whitespace stripped; if the source is not + available it is ‘None’. + + -- Function: traceback.extract_stack (f=None, limit=None) + + Extract the raw traceback from the current stack frame. The return + value has the same format as for *note extract_tb(): 33c1. The + optional `f' and `limit' arguments have the same meaning as for + *note print_stack(): 77a. + + -- Function: traceback.format_list (extracted_list) + + Given a list of tuples or *note FrameSummary: 778. objects as + returned by *note extract_tb(): 33c1. or *note extract_stack(): + 33c2, return a list of strings ready for printing. Each string in + the resulting list corresponds to the item with the same index in + the argument list. Each string ends in a newline; the strings may + contain internal newlines as well, for those items whose source + text line is not ‘None’. + + -- Function: traceback.format_exception_only (etype, value) + + Format the exception part of a traceback. The arguments are the + exception type and value such as given by ‘sys.last_type’ and + ‘sys.last_value’. The return value is a list of strings, each + ending in a newline. Normally, the list contains a single string; + however, for *note SyntaxError: 458. exceptions, it contains + several lines that (when printed) display detailed information + about where the syntax error occurred. The message indicating + which exception occurred is the always last string in the list. + + -- Function: traceback.format_exception (etype, value, tb, limit=None, + chain=True) + + Format a stack trace and the exception information. The arguments + have the same meaning as the corresponding arguments to *note + print_exception(): 1d8b. The return value is a list of strings, + each ending in a newline and some containing internal newlines. + When these lines are concatenated and printed, exactly the same + text is printed as does *note print_exception(): 1d8b. + + Changed in version 3.5: The `etype' argument is ignored and + inferred from the type of `value'. + + -- Function: traceback.format_exc (limit=None, chain=True) + + This is like ‘print_exc(limit)’ but returns a string instead of + printing to a file. + + -- Function: traceback.format_tb (tb, limit=None) + + A shorthand for ‘format_list(extract_tb(tb, limit))’. + + -- Function: traceback.format_stack (f=None, limit=None) + + A shorthand for ‘format_list(extract_stack(f, limit))’. + + -- Function: traceback.clear_frames (tb) + + Clears the local variables of all the stack frames in a traceback + `tb' by calling the ‘clear()’ method of each frame object. + + New in version 3.4. + + -- Function: traceback.walk_stack (f) + + Walk a stack following ‘f.f_back’ from the given frame, yielding + the frame and line number for each frame. If `f' is ‘None’, the + current stack is used. This helper is used with *note + StackSummary.extract(): 33c7. + + New in version 3.5. + + -- Function: traceback.walk_tb (tb) + + Walk a traceback following ‘tb_next’ yielding the frame and line + number for each frame. This helper is used with *note + StackSummary.extract(): 33c7. + + New in version 3.5. + +The module also defines the following classes: + +* Menu: + +* TracebackException Objects:: +* StackSummary Objects:: +* FrameSummary Objects:: +* Traceback Examples:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/traceback.py + + +File: python.info, Node: TracebackException Objects, Next: StackSummary Objects, Up: traceback — Print or retrieve a stack traceback + +5.29.10.1 ‘TracebackException’ Objects +...................................... + +New in version 3.5. + +*note TracebackException: 776. objects are created from actual +exceptions to capture data for later printing in a lightweight fashion. + + -- Class: traceback.TracebackException (exc_type, exc_value, + exc_traceback, *, limit=None, lookup_lines=True, + capture_locals=False) + + Capture an exception for later rendering. `limit', `lookup_lines' + and `capture_locals' are as for the *note StackSummary: 777. class. + + Note that when locals are captured, they are also shown in the + traceback. + + -- Attribute: __cause__ + + A *note TracebackException: 776. of the original ‘__cause__’. + + -- Attribute: __context__ + + A *note TracebackException: 776. of the original + ‘__context__’. + + -- Attribute: __suppress_context__ + + The ‘__suppress_context__’ value from the original exception. + + -- Attribute: stack + + A *note StackSummary: 777. representing the traceback. + + -- Attribute: exc_type + + The class of the original traceback. + + -- Attribute: filename + + For syntax errors - the file name where the error occurred. + + -- Attribute: lineno + + For syntax errors - the line number where the error occurred. + + -- Attribute: text + + For syntax errors - the text where the error occurred. + + -- Attribute: offset + + For syntax errors - the offset into the text where the error + occurred. + + -- Attribute: msg + + For syntax errors - the compiler error message. + + -- Method: classmethod from_exception (exc, *, limit=None, + lookup_lines=True, capture_locals=False) + + Capture an exception for later rendering. `limit', + `lookup_lines' and `capture_locals' are as for the *note + StackSummary: 777. class. + + Note that when locals are captured, they are also shown in the + traceback. + + -- Method: format (*, chain=True) + + Format the exception. + + If `chain' is not ‘True’, ‘__cause__’ and ‘__context__’ will + not be formatted. + + The return value is a generator of strings, each ending in a + newline and some containing internal newlines. *note + print_exception(): 1d8b. is a wrapper around this method which + just prints the lines to a file. + + The message indicating which exception occurred is always the + last string in the output. + + -- Method: format_exception_only () + + Format the exception part of the traceback. + + The return value is a generator of strings, each ending in a + newline. + + Normally, the generator emits a single string; however, for + *note SyntaxError: 458. exceptions, it emits several lines + that (when printed) display detailed information about where + the syntax error occurred. + + The message indicating which exception occurred is always the + last string in the output. + + +File: python.info, Node: StackSummary Objects, Next: FrameSummary Objects, Prev: TracebackException Objects, Up: traceback — Print or retrieve a stack traceback + +5.29.10.2 ‘StackSummary’ Objects +................................ + +New in version 3.5. + +*note StackSummary: 777. objects represent a call stack ready for +formatting. + + -- Class: traceback.StackSummary + + -- Method: classmethod extract (frame_gen, *, limit=None, + lookup_lines=True, capture_locals=False) + + Construct a *note StackSummary: 777. object from a frame + generator (such as is returned by *note walk_stack(): 774. or + *note walk_tb(): 775.). + + If `limit' is supplied, only this many frames are taken from + `frame_gen'. If `lookup_lines' is ‘False’, the returned *note + FrameSummary: 778. objects will not have read their lines in + yet, making the cost of creating the *note StackSummary: 777. + cheaper (which may be valuable if it may not actually get + formatted). If `capture_locals' is ‘True’ the local variables + in each *note FrameSummary: 778. are captured as object + representations. + + -- Method: classmethod from_list (a_list) + + Construct a *note StackSummary: 777. object from a supplied + list of *note FrameSummary: 778. objects or old-style list of + tuples. Each tuple should be a 4-tuple with filename, lineno, + name, line as the elements. + + -- Method: format () + + Returns a list of strings ready for printing. Each string in + the resulting list corresponds to a single frame from the + stack. Each string ends in a newline; the strings may contain + internal newlines as well, for those items with source text + lines. + + For long sequences of the same frame and line, the first few + repetitions are shown, followed by a summary line stating the + exact number of further repetitions. + + Changed in version 3.6: Long sequences of repeated frames are + now abbreviated. + + +File: python.info, Node: FrameSummary Objects, Next: Traceback Examples, Prev: StackSummary Objects, Up: traceback — Print or retrieve a stack traceback + +5.29.10.3 ‘FrameSummary’ Objects +................................ + +New in version 3.5. + +*note FrameSummary: 778. objects represent a single frame in a +traceback. + + -- Class: traceback.FrameSummary (filename, lineno, name, + lookup_line=True, locals=None, line=None) + + Represent a single frame in the traceback or stack that is being + formatted or printed. It may optionally have a stringified version + of the frames locals included in it. If `lookup_line' is ‘False’, + the source code is not looked up until the *note FrameSummary: 778. + has the ‘line’ attribute accessed (which also happens when casting + it to a tuple). ‘line’ may be directly provided, and will prevent + line lookups happening at all. `locals' is an optional local + variable dictionary, and if supplied the variable representations + are stored in the summary for later display. + + +File: python.info, Node: Traceback Examples, Prev: FrameSummary Objects, Up: traceback — Print or retrieve a stack traceback + +5.29.10.4 Traceback Examples +............................ + +This simple example implements a basic read-eval-print loop, similar to +(but less useful than) the standard Python interactive interpreter loop. +For a more complete implementation of the interpreter loop, refer to the +*note code: 1b. module. + + import sys, traceback + + def run_user_code(envdir): + source = input(">>> ") + try: + exec(source, envdir) + except Exception: + print("Exception in user code:") + print("-"*60) + traceback.print_exc(file=sys.stdout) + print("-"*60) + + envdir = {} + while True: + run_user_code(envdir) + +The following example demonstrates the different ways to print and +format the exception and traceback: + + import sys, traceback + + def lumberjack(): + bright_side_of_death() + + def bright_side_of_death(): + return tuple()[0] + + try: + lumberjack() + except IndexError: + exc_type, exc_value, exc_traceback = sys.exc_info() + print("*** print_tb:") + traceback.print_tb(exc_traceback, limit=1, file=sys.stdout) + print("*** print_exception:") + # exc_type below is ignored on 3.5 and later + traceback.print_exception(exc_type, exc_value, exc_traceback, + limit=2, file=sys.stdout) + print("*** print_exc:") + traceback.print_exc(limit=2, file=sys.stdout) + print("*** format_exc, first and last line:") + formatted_lines = traceback.format_exc().splitlines() + print(formatted_lines[0]) + print(formatted_lines[-1]) + print("*** format_exception:") + # exc_type below is ignored on 3.5 and later + print(repr(traceback.format_exception(exc_type, exc_value, + exc_traceback))) + print("*** extract_tb:") + print(repr(traceback.extract_tb(exc_traceback))) + print("*** format_tb:") + print(repr(traceback.format_tb(exc_traceback))) + print("*** tb_lineno:", exc_traceback.tb_lineno) + +The output for the example would look similar to this: + + *** print_tb: + File "<doctest...>", line 10, in <module> + lumberjack() + *** print_exception: + Traceback (most recent call last): + File "<doctest...>", line 10, in <module> + lumberjack() + File "<doctest...>", line 4, in lumberjack + bright_side_of_death() + IndexError: tuple index out of range + *** print_exc: + Traceback (most recent call last): + File "<doctest...>", line 10, in <module> + lumberjack() + File "<doctest...>", line 4, in lumberjack + bright_side_of_death() + IndexError: tuple index out of range + *** format_exc, first and last line: + Traceback (most recent call last): + IndexError: tuple index out of range + *** format_exception: + ['Traceback (most recent call last):\n', + ' File "<doctest...>", line 10, in <module>\n lumberjack()\n', + ' File "<doctest...>", line 4, in lumberjack\n bright_side_of_death()\n', + ' File "<doctest...>", line 7, in bright_side_of_death\n return tuple()[0]\n', + 'IndexError: tuple index out of range\n'] + *** extract_tb: + [<FrameSummary file <doctest...>, line 10 in <module>>, + <FrameSummary file <doctest...>, line 4 in lumberjack>, + <FrameSummary file <doctest...>, line 7 in bright_side_of_death>] + *** format_tb: + [' File "<doctest...>", line 10, in <module>\n lumberjack()\n', + ' File "<doctest...>", line 4, in lumberjack\n bright_side_of_death()\n', + ' File "<doctest...>", line 7, in bright_side_of_death\n return tuple()[0]\n'] + *** tb_lineno: 10 + +The following example shows the different ways to print and format the +stack: + + >>> import traceback + >>> def another_function(): + ... lumberstack() + ... + >>> def lumberstack(): + ... traceback.print_stack() + ... print(repr(traceback.extract_stack())) + ... print(repr(traceback.format_stack())) + ... + >>> another_function() + File "<doctest>", line 10, in <module> + another_function() + File "<doctest>", line 3, in another_function + lumberstack() + File "<doctest>", line 6, in lumberstack + traceback.print_stack() + [('<doctest>', 10, '<module>', 'another_function()'), + ('<doctest>', 3, 'another_function', 'lumberstack()'), + ('<doctest>', 7, 'lumberstack', 'print(repr(traceback.extract_stack()))')] + [' File "<doctest>", line 10, in <module>\n another_function()\n', + ' File "<doctest>", line 3, in another_function\n lumberstack()\n', + ' File "<doctest>", line 8, in lumberstack\n print(repr(traceback.format_stack()))\n'] + +This last example demonstrates the final few formatting functions: + + >>> import traceback + >>> traceback.format_list([('spam.py', 3, '<module>', 'spam.eggs()'), + ... ('eggs.py', 42, 'eggs', 'return "bacon"')]) + [' File "spam.py", line 3, in <module>\n spam.eggs()\n', + ' File "eggs.py", line 42, in eggs\n return "bacon"\n'] + >>> an_error = IndexError('tuple index out of range') + >>> traceback.format_exception_only(type(an_error), an_error) + ['IndexError: tuple index out of range\n'] + + +File: python.info, Node: __future__ — Future statement definitions, Next: gc — Garbage Collector interface, Prev: traceback — Print or retrieve a stack traceback, Up: Python Runtime Services + +5.29.11 ‘__future__’ — Future statement definitions +--------------------------------------------------- + +`Source code:' Lib/__future__.py(1) + +__________________________________________________________________ + +*note __future__: 0. is a real module, and serves three purposes: + + * To avoid confusing existing tools that analyze import statements + and expect to find the modules they’re importing. + + * To ensure that *note future statements: 1236. run under releases + prior to 2.1 at least yield runtime exceptions (the import of *note + __future__: 0. will fail, because there was no module of that name + prior to 2.1). + + * To document when incompatible changes were introduced, and when + they will be — or were — made mandatory. This is a form of + executable documentation, and can be inspected programmatically via + importing *note __future__: 0. and examining its contents. + +Each statement in ‘__future__.py’ is of the form: + + FeatureName = _Feature(OptionalRelease, MandatoryRelease, + CompilerFlag) + +where, normally, `OptionalRelease' is less than `MandatoryRelease', and +both are 5-tuples of the same form as *note sys.version_info: b1a.: + + (PY_MAJOR_VERSION, # the 2 in 2.1.0a3; an int + PY_MINOR_VERSION, # the 1; an int + PY_MICRO_VERSION, # the 0; an int + PY_RELEASE_LEVEL, # "alpha", "beta", "candidate" or "final"; string + PY_RELEASE_SERIAL # the 3; an int + ) + +`OptionalRelease' records the first release in which the feature was +accepted. + +In the case of a `MandatoryRelease' that has not yet occurred, +`MandatoryRelease' predicts the release in which the feature will become +part of the language. + +Else `MandatoryRelease' records when the feature became part of the +language; in releases at or after that, modules no longer need a future +statement to use the feature in question, but may continue to use such +imports. + +`MandatoryRelease' may also be ‘None’, meaning that a planned feature +got dropped. + +Instances of class ‘_Feature’ have two corresponding methods, +‘getOptionalRelease()’ and ‘getMandatoryRelease()’. + +`CompilerFlag' is the (bitfield) flag that should be passed in the +fourth argument to the built-in function *note compile(): 1b4. to enable +the feature in dynamically compiled code. This flag is stored in the +‘compiler_flag’ attribute on ‘_Feature’ instances. + +No feature description will ever be deleted from *note __future__: 0. +Since its introduction in Python 2.1 the following features have found +their way into the language using this mechanism: + +feature optional in mandatory in effect + +-------------------------------------------------------------------------------------------------------------- + +nested_scopes 2.1.0b1 2.2 PEP 227(2): `Statically Nested Scopes' + + +generators 2.2.0a1 2.3 PEP 255(3): `Simple Generators' + + +division 2.2.0a2 3.0 PEP 238(4): `Changing the Division Operator' + + +absolute_import 2.5.0a1 3.0 PEP 328(5): `Imports: Multi-Line and + Absolute/Relative' + + +with_statement 2.5.0a1 2.6 PEP 343(6): `The “with” Statement' + + +print_function 2.6.0a2 3.0 PEP 3105(7): `Make print a function' + + +unicode_literals 2.6.0a2 3.0 PEP 3112(8): `Bytes literals in Python 3000' + + +generator_stop 3.5.0b1 3.7 PEP 479(9): `StopIteration handling inside + generators' + + +annotations 3.7.0b1 3.10 PEP 563(10): `Postponed evaluation of + annotations' + + +See also +........ + +*note Future statements: 1236. + + How the compiler treats future imports. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/__future__.py + + (2) https://www.python.org/dev/peps/pep-0227 + + (3) https://www.python.org/dev/peps/pep-0255 + + (4) https://www.python.org/dev/peps/pep-0238 + + (5) https://www.python.org/dev/peps/pep-0328 + + (6) https://www.python.org/dev/peps/pep-0343 + + (7) https://www.python.org/dev/peps/pep-3105 + + (8) https://www.python.org/dev/peps/pep-3112 + + (9) https://www.python.org/dev/peps/pep-0479 + + (10) https://www.python.org/dev/peps/pep-0563 + + +File: python.info, Node: gc — Garbage Collector interface, Next: inspect — Inspect live objects, Prev: __future__ — Future statement definitions, Up: Python Runtime Services + +5.29.12 ‘gc’ — Garbage Collector interface +------------------------------------------ + +__________________________________________________________________ + +This module provides an interface to the optional garbage collector. It +provides the ability to disable the collector, tune the collection +frequency, and set debugging options. It also provides access to +unreachable objects that the collector found but cannot free. Since the +collector supplements the reference counting already used in Python, you +can disable the collector if you are sure your program does not create +reference cycles. Automatic collection can be disabled by calling +‘gc.disable()’. To debug a leaking program call +‘gc.set_debug(gc.DEBUG_LEAK)’. Notice that this includes +‘gc.DEBUG_SAVEALL’, causing garbage-collected objects to be saved in +gc.garbage for inspection. + +The *note gc: 86. module provides the following functions: + + -- Function: gc.enable () + + Enable automatic garbage collection. + + -- Function: gc.disable () + + Disable automatic garbage collection. + + -- Function: gc.isenabled () + + Return ‘True’ if automatic collection is enabled. + + -- Function: gc.collect (generation=2) + + With no arguments, run a full collection. The optional argument + `generation' may be an integer specifying which generation to + collect (from 0 to 2). A *note ValueError: 1fb. is raised if the + generation number is invalid. The number of unreachable objects + found is returned. + + The free lists maintained for a number of built-in types are + cleared whenever a full collection or collection of the highest + generation (2) is run. Not all items in some free lists may be + freed due to the particular implementation, in particular *note + float: 187. + + -- Function: gc.set_debug (flags) + + Set the garbage collection debugging flags. Debugging information + will be written to ‘sys.stderr’. See below for a list of debugging + flags which can be combined using bit operations to control + debugging. + + -- Function: gc.get_debug () + + Return the debugging flags currently set. + + -- Function: gc.get_objects (generation=None) + + Returns a list of all objects tracked by the collector, excluding + the list returned. If `generation' is not None, return only the + objects tracked by the collector that are in that generation. + + Changed in version 3.8: New `generation' parameter. + + Raises an *note auditing event: fd1. ‘gc.get_objects’ with argument + ‘generation’. + + -- Function: gc.get_stats () + + Return a list of three per-generation dictionaries containing + collection statistics since interpreter start. The number of keys + may change in the future, but currently each dictionary will + contain the following items: + + * ‘collections’ is the number of times this generation was + collected; + + * ‘collected’ is the total number of objects collected inside + this generation; + + * ‘uncollectable’ is the total number of objects which were + found to be uncollectable (and were therefore moved to the + *note garbage: b40. list) inside this generation. + + New in version 3.4. + + -- Function: gc.set_threshold (threshold0[, threshold1[, threshold2]]) + + Set the garbage collection thresholds (the collection frequency). + Setting `threshold0' to zero disables collection. + + The GC classifies objects into three generations depending on how + many collection sweeps they have survived. New objects are placed + in the youngest generation (generation ‘0’). If an object survives + a collection it is moved into the next older generation. Since + generation ‘2’ is the oldest generation, objects in that generation + remain there after a collection. In order to decide when to run, + the collector keeps track of the number object allocations and + deallocations since the last collection. When the number of + allocations minus the number of deallocations exceeds `threshold0', + collection starts. Initially only generation ‘0’ is examined. If + generation ‘0’ has been examined more than `threshold1' times since + generation ‘1’ has been examined, then generation ‘1’ is examined + as well. With the third generation, things are a bit more + complicated, see Collecting the oldest generation(1) for more + information. + + -- Function: gc.get_count () + + Return the current collection counts as a tuple of ‘(count0, + count1, count2)’. + + -- Function: gc.get_threshold () + + Return the current collection thresholds as a tuple of + ‘(threshold0, threshold1, threshold2)’. + + -- Function: gc.get_referrers (*objs) + + Return the list of objects that directly refer to any of objs. + This function will only locate those containers which support + garbage collection; extension types which do refer to other objects + but do not support garbage collection will not be found. + + Note that objects which have already been dereferenced, but which + live in cycles and have not yet been collected by the garbage + collector can be listed among the resulting referrers. To get only + currently live objects, call *note collect(): 22e. before calling + *note get_referrers(): 31f5. + + Warning: Care must be taken when using objects returned by + *note get_referrers(): 31f5. because some of them could still + be under construction and hence in a temporarily invalid + state. Avoid using *note get_referrers(): 31f5. for any + purpose other than debugging. + + Raises an *note auditing event: fd1. ‘gc.get_referrers’ with + argument ‘objs’. + + -- Function: gc.get_referents (*objs) + + Return a list of objects directly referred to by any of the + arguments. The referents returned are those objects visited by the + arguments’ C-level *note tp_traverse: 33e8. methods (if any), and + may not be all objects actually directly reachable. *note + tp_traverse: 33e8. methods are supported only by objects that + support garbage collection, and are only required to visit objects + that may be involved in a cycle. So, for example, if an integer is + directly reachable from an argument, that integer object may or may + not appear in the result list. + + Raises an *note auditing event: fd1. ‘gc.get_referents’ with + argument ‘objs’. + + -- Function: gc.is_tracked (obj) + + Returns ‘True’ if the object is currently tracked by the garbage + collector, ‘False’ otherwise. As a general rule, instances of + atomic types aren’t tracked and instances of non-atomic types + (containers, user-defined objects…) are. However, some + type-specific optimizations can be present in order to suppress the + garbage collector footprint of simple instances (e.g. dicts + containing only atomic keys and values): + + >>> gc.is_tracked(0) + False + >>> gc.is_tracked("a") + False + >>> gc.is_tracked([]) + True + >>> gc.is_tracked({}) + False + >>> gc.is_tracked({"a": 1}) + False + >>> gc.is_tracked({"a": []}) + True + + New in version 3.1. + + -- Function: gc.freeze () + + Freeze all the objects tracked by gc - move them to a permanent + generation and ignore all the future collections. This can be used + before a POSIX fork() call to make the gc copy-on-write friendly or + to speed up collection. Also collection before a POSIX fork() call + may free pages for future allocation which can cause copy-on-write + too so it’s advised to disable gc in parent process and freeze + before fork and enable gc in child process. + + New in version 3.7. + + -- Function: gc.unfreeze () + + Unfreeze the objects in the permanent generation, put them back + into the oldest generation. + + New in version 3.7. + + -- Function: gc.get_freeze_count () + + Return the number of objects in the permanent generation. + + New in version 3.7. + +The following variables are provided for read-only access (you can +mutate the values but should not rebind them): + + -- Data: gc.garbage + + A list of objects which the collector found to be unreachable but + could not be freed (uncollectable objects). Starting with Python + 3.4, this list should be empty most of the time, except when using + instances of C extension types with a non-‘NULL’ ‘tp_del’ slot. + + If *note DEBUG_SAVEALL: 33e9. is set, then all unreachable objects + will be added to this list rather than freed. + + Changed in version 3.2: If this list is non-empty at *note + interpreter shutdown: 763, a *note ResourceWarning: 4d0. is + emitted, which is silent by default. If *note DEBUG_UNCOLLECTABLE: + b41. is set, in addition all uncollectable objects are printed. + + Changed in version 3.4: Following PEP 442(2), objects with a *note + __del__(): 91f. method don’t end up in *note gc.garbage: b40. + anymore. + + -- Data: gc.callbacks + + A list of callbacks that will be invoked by the garbage collector + before and after collection. The callbacks will be called with two + arguments, `phase' and `info'. + + `phase' can be one of two values: + + “start”: The garbage collection is about to start. + + “stop”: The garbage collection has finished. + + `info' is a dict providing more information for the callback. The + following keys are currently defined: + + “generation”: The oldest generation being collected. + + “collected”: When `phase' is “stop”, the number of objects + successfully collected. + + “uncollectable”: When `phase' is “stop”, the number of objects + that could not be collected and were put in *note garbage: + b40. + + Applications can add their own callbacks to this list. The primary + use cases are: + + Gathering statistics about garbage collection, such as how + often various generations are collected, and how long the + collection takes. + + Allowing applications to identify and clear their own + uncollectable types when they appear in *note garbage: b40. + + New in version 3.3. + +The following constants are provided for use with *note set_debug(): +33e3.: + + -- Data: gc.DEBUG_STATS + + Print statistics during collection. This information can be useful + when tuning the collection frequency. + + -- Data: gc.DEBUG_COLLECTABLE + + Print information on collectable objects found. + + -- Data: gc.DEBUG_UNCOLLECTABLE + + Print information of uncollectable objects found (objects which are + not reachable but cannot be freed by the collector). These objects + will be added to the ‘garbage’ list. + + Changed in version 3.2: Also print the contents of the *note + garbage: b40. list at *note interpreter shutdown: 763, if it isn’t + empty. + + -- Data: gc.DEBUG_SAVEALL + + When set, all unreachable objects found will be appended to + `garbage' rather than being freed. This can be useful for + debugging a leaking program. + + -- Data: gc.DEBUG_LEAK + + The debugging flags necessary for the collector to print + information about a leaking program (equal to ‘DEBUG_COLLECTABLE | + DEBUG_UNCOLLECTABLE | DEBUG_SAVEALL’). + + ---------- Footnotes ---------- + + (1) +https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation + + (2) https://www.python.org/dev/peps/pep-0442 + + +File: python.info, Node: inspect — Inspect live objects, Next: site — Site-specific configuration hook, Prev: gc — Garbage Collector interface, Up: Python Runtime Services + +5.29.13 ‘inspect’ — Inspect live objects +---------------------------------------- + +`Source code:' Lib/inspect.py(1) + +__________________________________________________________________ + +The *note inspect: a0. module provides several useful functions to help +get information about live objects such as modules, classes, methods, +functions, tracebacks, frame objects, and code objects. For example, it +can help you examine the contents of a class, retrieve the source code +of a method, extract and format the argument list for a function, or get +all the information you need to display a detailed traceback. + +There are four main kinds of services provided by this module: type +checking, getting source code, inspecting classes and functions, and +examining the interpreter stack. + +* Menu: + +* Types and members:: +* Retrieving source code:: +* Introspecting callables with the Signature object:: +* Classes and functions: Classes and functions<2>. +* The interpreter stack:: +* Fetching attributes statically:: +* Current State of Generators and Coroutines:: +* Code Objects Bit Flags:: +* Command Line Interface: Command Line Interface<3>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/inspect.py + + +File: python.info, Node: Types and members, Next: Retrieving source code, Up: inspect — Inspect live objects + +5.29.13.1 Types and members +........................... + +The *note getmembers(): 33f1. function retrieves the members of an +object such as a class or module. The functions whose names begin with +“is” are mainly provided as convenient choices for the second argument +to *note getmembers(): 33f1. They also help you determine when you can +expect to find the following special attributes: + +Type Attribute Description + +------------------------------------------------------------------------ + +module __doc__ documentation string + + + __file__ filename (missing for + built-in modules) + + +class __doc__ documentation string + + + __name__ name with which this class + was defined + + + __qualname__ qualified name + + + __module__ name of module in which this + class was defined + + +method __doc__ documentation string + + + __name__ name with which this method + was defined + + + __qualname__ qualified name + + + __func__ function object containing + implementation of method + + + __self__ instance to which this method + is bound, or ‘None’ + + + __module__ name of module in which this + method was defined + + +function __doc__ documentation string + + + __name__ name with which this function + was defined + + + __qualname__ qualified name + + + __code__ code object containing + compiled function + *note bytecode: 614. + + + __defaults__ tuple of any default values + for positional or keyword + parameters + + + __kwdefaults__ mapping of any default values + for keyword-only parameters + + + __globals__ global namespace in which + this function was defined + + + __annotations__ mapping of parameters names + to annotations; ‘"return"’ + key is reserved for return + annotations. + + + __module__ name of module in which this + function was defined + + +traceback tb_frame frame object at this level + + + tb_lasti index of last attempted + instruction in bytecode + + + tb_lineno current line number in Python + source code + + + tb_next next inner traceback object + (called by this level) + + +frame f_back next outer frame object (this + frame’s caller) + + + f_builtins builtins namespace seen by + this frame + + + f_code code object being executed in + this frame + + + f_globals global namespace seen by this + frame + + + f_lasti index of last attempted + instruction in bytecode + + + f_lineno current line number in Python + source code + + + f_locals local namespace seen by this + frame + + + f_trace tracing function for this + frame, or ‘None’ + + +code co_argcount number of arguments (not + including keyword only + arguments, * or ** args) + + + co_code string of raw compiled + bytecode + + + co_cellvars tuple of names of cell + variables (referenced by + containing scopes) + + + co_consts tuple of constants used in + the bytecode + + + co_filename name of file in which this + code object was created + + + co_firstlineno number of first line in + Python source code + + + co_flags bitmap of ‘CO_*’ flags, read + more *note here: 33f2. + + + co_lnotab encoded mapping of line + numbers to bytecode indices + + + co_freevars tuple of names of free + variables (referenced via a + function’s closure) + + + co_posonlyargcount number of positional only + arguments + + + co_kwonlyargcount number of keyword only + arguments (not including ** + arg) + + + co_name name with which this code + object was defined + + + co_names tuple of names of local + variables + + + co_nlocals number of local variables + + + co_stacksize virtual machine stack space + required + + + co_varnames tuple of names of arguments + and local variables + + +generator __name__ name + + + __qualname__ qualified name + + + gi_frame frame + + + gi_running is the generator running? + + + gi_code code + + + gi_yieldfrom object being iterated by + ‘yield from’, or ‘None’ + + +coroutine __name__ name + + + __qualname__ qualified name + + + cr_await object being awaited on, or + ‘None’ + + + cr_frame frame + + + cr_running is the coroutine running? + + + cr_code code + + + cr_origin where coroutine was created, + or ‘None’. See + *note sys.set_coroutine_origin_tracking_depth(): 3f3. + + +builtin __doc__ documentation string + + + __name__ original name of this + function or method + + + __qualname__ qualified name + + + __self__ instance to which a method is + bound, or ‘None’ + + +Changed in version 3.5: Add ‘__qualname__’ and ‘gi_yieldfrom’ attributes +to generators. + +The ‘__name__’ attribute of generators is now set from the function +name, instead of the code name, and it can now be modified. + +Changed in version 3.7: Add ‘cr_origin’ attribute to coroutines. + + -- Function: inspect.getmembers (object[, predicate]) + + Return all the members of an object in a list of ‘(name, value)’ + pairs sorted by name. If the optional `predicate' argument—which + will be called with the ‘value’ object of each member—is supplied, + only members for which the predicate returns a true value are + included. + + Note: *note getmembers(): 33f1. will only return class + attributes defined in the metaclass when the argument is a + class and those attributes have been listed in the metaclass’ + custom *note __dir__(): 31b. + + -- Function: inspect.getmodulename (path) + + Return the name of the module named by the file `path', without + including the names of enclosing packages. The file extension is + checked against all of the entries in *note + importlib.machinery.all_suffixes(): 33f3. If it matches, the final + path component is returned with the extension removed. Otherwise, + ‘None’ is returned. + + Note that this function `only' returns a meaningful name for actual + Python modules - paths that potentially refer to Python packages + will still return ‘None’. + + Changed in version 3.3: The function is based directly on *note + importlib: 9b. + + -- Function: inspect.ismodule (object) + + Return ‘True’ if the object is a module. + + -- Function: inspect.isclass (object) + + Return ‘True’ if the object is a class, whether built-in or created + in Python code. + + -- Function: inspect.ismethod (object) + + Return ‘True’ if the object is a bound method written in Python. + + -- Function: inspect.isfunction (object) + + Return ‘True’ if the object is a Python function, which includes + functions created by a *note lambda: 33f8. expression. + + -- Function: inspect.isgeneratorfunction (object) + + Return ‘True’ if the object is a Python generator function. + + Changed in version 3.8: Functions wrapped in *note + functools.partial(): 7c8. now return ‘True’ if the wrapped function + is a Python generator function. + + -- Function: inspect.isgenerator (object) + + Return ‘True’ if the object is a generator. + + -- Function: inspect.iscoroutinefunction (object) + + Return ‘True’ if the object is a *note coroutine function: 63f. (a + function defined with an *note async def: 284. syntax). + + New in version 3.5. + + Changed in version 3.8: Functions wrapped in *note + functools.partial(): 7c8. now return ‘True’ if the wrapped function + is a *note coroutine function: 63f. + + -- Function: inspect.iscoroutine (object) + + Return ‘True’ if the object is a *note coroutine: 1a6. created by + an *note async def: 284. function. + + New in version 3.5. + + -- Function: inspect.isawaitable (object) + + Return ‘True’ if the object can be used in *note await: 1a3. + expression. + + Can also be used to distinguish generator-based coroutines from + regular generators: + + def gen(): + yield + @types.coroutine + def gen_coro(): + yield + + assert not isawaitable(gen()) + assert isawaitable(gen_coro()) + + New in version 3.5. + + -- Function: inspect.isasyncgenfunction (object) + + Return ‘True’ if the object is an *note asynchronous generator: + 11a9. function, for example: + + >>> async def agen(): + ... yield 1 + ... + >>> inspect.isasyncgenfunction(agen) + True + + New in version 3.6. + + Changed in version 3.8: Functions wrapped in *note + functools.partial(): 7c8. now return ‘True’ if the wrapped function + is a *note asynchronous generator: 11a9. function. + + -- Function: inspect.isasyncgen (object) + + Return ‘True’ if the object is an *note asynchronous generator + iterator: 335c. created by an *note asynchronous generator: 11a9. + function. + + New in version 3.6. + + -- Function: inspect.istraceback (object) + + Return ‘True’ if the object is a traceback. + + -- Function: inspect.isframe (object) + + Return ‘True’ if the object is a frame. + + -- Function: inspect.iscode (object) + + Return ‘True’ if the object is a code. + + -- Function: inspect.isbuiltin (object) + + Return ‘True’ if the object is a built-in function or a bound + built-in method. + + -- Function: inspect.isroutine (object) + + Return ‘True’ if the object is a user-defined or built-in function + or method. + + -- Function: inspect.isabstract (object) + + Return ‘True’ if the object is an abstract base class. + + -- Function: inspect.ismethoddescriptor (object) + + Return ‘True’ if the object is a method descriptor, but not if + *note ismethod(): 33f6, *note isclass(): 33f5, *note isfunction(): + 33f7. or *note isbuiltin(): 3400. are true. + + This, for example, is true of ‘int.__add__’. An object passing + this test has a *note __get__(): 10dd. method but not a *note + __set__(): 10e1. method, but beyond that the set of attributes + varies. A *note __name__: c67. attribute is usually sensible, and + ‘__doc__’ often is. + + Methods implemented via descriptors that also pass one of the other + tests return ‘False’ from the *note ismethoddescriptor(): 3403. + test, simply because the other tests promise more – you can, e.g., + count on having the ‘__func__’ attribute (etc) when an object + passes *note ismethod(): 33f6. + + -- Function: inspect.isdatadescriptor (object) + + Return ‘True’ if the object is a data descriptor. + + Data descriptors have a *note __set__: 10e1. or a *note __delete__: + 10e2. method. Examples are properties (defined in Python), + getsets, and members. The latter two are defined in C and there + are more specific tests available for those types, which is robust + across Python implementations. Typically, data descriptors will + also have *note __name__: c67. and ‘__doc__’ attributes + (properties, getsets, and members have both of these attributes), + but this is not guaranteed. + + -- Function: inspect.isgetsetdescriptor (object) + + Return ‘True’ if the object is a getset descriptor. + + `CPython implementation detail:' getsets are attributes defined in + extension modules via *note PyGetSetDef: 427. structures. For + Python implementations without such types, this method will always + return ‘False’. + + -- Function: inspect.ismemberdescriptor (object) + + Return ‘True’ if the object is a member descriptor. + + `CPython implementation detail:' Member descriptors are attributes + defined in extension modules via *note PyMemberDef: 426. + structures. For Python implementations without such types, this + method will always return ‘False’. + + +File: python.info, Node: Retrieving source code, Next: Introspecting callables with the Signature object, Prev: Types and members, Up: inspect — Inspect live objects + +5.29.13.2 Retrieving source code +................................ + + -- Function: inspect.getdoc (object) + + Get the documentation string for an object, cleaned up with *note + cleandoc(): 3409. If the documentation string for an object is not + provided and the object is a class, a method, a property or a + descriptor, retrieve the documentation string from the inheritance + hierarchy. + + Changed in version 3.5: Documentation strings are now inherited if + not overridden. + + -- Function: inspect.getcomments (object) + + Return in a single string any lines of comments immediately + preceding the object’s source code (for a class, function, or + method), or at the top of the Python source file (if the object is + a module). If the object’s source code is unavailable, return + ‘None’. This could happen if the object has been defined in C or + the interactive shell. + + -- Function: inspect.getfile (object) + + Return the name of the (text or binary) file in which an object was + defined. This will fail with a *note TypeError: 192. if the object + is a built-in module, class, or function. + + -- Function: inspect.getmodule (object) + + Try to guess which module an object was defined in. + + -- Function: inspect.getsourcefile (object) + + Return the name of the Python source file in which an object was + defined. This will fail with a *note TypeError: 192. if the object + is a built-in module, class, or function. + + -- Function: inspect.getsourcelines (object) + + Return a list of source lines and starting line number for an + object. The argument may be a module, class, method, function, + traceback, frame, or code object. The source code is returned as a + list of the lines corresponding to the object and the line number + indicates where in the original source file the first line of code + was found. An *note OSError: 1d3. is raised if the source code + cannot be retrieved. + + Changed in version 3.3: *note OSError: 1d3. is raised instead of + *note IOError: 992, now an alias of the former. + + -- Function: inspect.getsource (object) + + Return the text of the source code for an object. The argument may + be a module, class, method, function, traceback, frame, or code + object. The source code is returned as a single string. An *note + OSError: 1d3. is raised if the source code cannot be retrieved. + + Changed in version 3.3: *note OSError: 1d3. is raised instead of + *note IOError: 992, now an alias of the former. + + -- Function: inspect.cleandoc (doc) + + Clean up indentation from docstrings that are indented to line up + with blocks of code. + + All leading whitespace is removed from the first line. Any leading + whitespace that can be uniformly removed from the second line + onwards is removed. Empty lines at the beginning and end are + subsequently removed. Also, all tabs are expanded to spaces. + + +File: python.info, Node: Introspecting callables with the Signature object, Next: Classes and functions<2>, Prev: Retrieving source code, Up: inspect — Inspect live objects + +5.29.13.3 Introspecting callables with the Signature object +........................................................... + +New in version 3.3. + +The Signature object represents the call signature of a callable object +and its return annotation. To retrieve a Signature object, use the +*note signature(): 55b. function. + + -- Function: inspect.signature (callable, *, follow_wrapped=True) + + Return a *note Signature: 6f3. object for the given ‘callable’: + + >>> from inspect import signature + >>> def foo(a, *, b:int, **kwargs): + ... pass + + >>> sig = signature(foo) + + >>> str(sig) + '(a, *, b:int, **kwargs)' + + >>> str(sig.parameters['b']) + 'b:int' + + >>> sig.parameters['b'].annotation + <class 'int'> + + Accepts a wide range of Python callables, from plain functions and + classes to *note functools.partial(): 7c8. objects. + + Raises *note ValueError: 1fb. if no signature can be provided, and + *note TypeError: 192. if that type of object is not supported. + + A slash(/) in the signature of a function denotes that the + parameters prior to it are positional-only. For more info, see + *note the FAQ entry on positional-only parameters: 129d. + + New in version 3.5: ‘follow_wrapped’ parameter. Pass ‘False’ to + get a signature of ‘callable’ specifically (‘callable.__wrapped__’ + will not be used to unwrap decorated callables.) + + Note: Some callables may not be introspectable in certain + implementations of Python. For example, in CPython, some + built-in functions defined in C provide no metadata about + their arguments. + + -- Class: inspect.Signature (parameters=None, *, + return_annotation=Signature.empty) + + A Signature object represents the call signature of a function and + its return annotation. For each parameter accepted by the function + it stores a *note Parameter: 6f4. object in its *note parameters: + 3411. collection. + + The optional `parameters' argument is a sequence of *note + Parameter: 6f4. objects, which is validated to check that there are + no parameters with duplicate names, and that the parameters are in + the right order, i.e. positional-only first, then + positional-or-keyword, and that parameters with defaults follow + parameters without defaults. + + The optional `return_annotation' argument, can be an arbitrary + Python object, is the “return” annotation of the callable. + + Signature objects are `immutable'. Use *note Signature.replace(): + 3412. to make a modified copy. + + Changed in version 3.5: Signature objects are picklable and + hashable. + + -- Attribute: empty + + A special class-level marker to specify absence of a return + annotation. + + -- Attribute: parameters + + An ordered mapping of parameters’ names to the corresponding + *note Parameter: 6f4. objects. Parameters appear in strict + definition order, including keyword-only parameters. + + Changed in version 3.7: Python only explicitly guaranteed that + it preserved the declaration order of keyword-only parameters + as of version 3.7, although in practice this order had always + been preserved in Python 3. + + -- Attribute: return_annotation + + The “return” annotation for the callable. If the callable has + no “return” annotation, this attribute is set to *note + Signature.empty: 3413. + + -- Method: bind (*args, **kwargs) + + Create a mapping from positional and keyword arguments to + parameters. Returns *note BoundArguments: 9a8. if ‘*args’ and + ‘**kwargs’ match the signature, or raises a *note TypeError: + 192. + + -- Method: bind_partial (*args, **kwargs) + + Works the same way as *note Signature.bind(): 3415, but allows + the omission of some required arguments (mimics *note + functools.partial(): 7c8. behavior.) Returns *note + BoundArguments: 9a8, or raises a *note TypeError: 192. if the + passed arguments do not match the signature. + + -- Method: replace (*[, parameters][, return_annotation]) + + Create a new Signature instance based on the instance replace + was invoked on. It is possible to pass different ‘parameters’ + and/or ‘return_annotation’ to override the corresponding + properties of the base signature. To remove return_annotation + from the copied Signature, pass in *note Signature.empty: + 3413. + + >>> def test(a, b): + ... pass + >>> sig = signature(test) + >>> new_sig = sig.replace(return_annotation="new return anno") + >>> str(new_sig) + "(a, b) -> 'new return anno'" + + -- Method: classmethod from_callable (obj, *, follow_wrapped=True) + + Return a *note Signature: 6f3. (or its subclass) object for a + given callable ‘obj’. Pass ‘follow_wrapped=False’ to get a + signature of ‘obj’ without unwrapping its ‘__wrapped__’ chain. + + This method simplifies subclassing of *note Signature: 6f3.: + + class MySignature(Signature): + pass + sig = MySignature.from_callable(min) + assert isinstance(sig, MySignature) + + New in version 3.5. + + -- Class: inspect.Parameter (name, kind, *, default=Parameter.empty, + annotation=Parameter.empty) + + Parameter objects are `immutable'. Instead of modifying a + Parameter object, you can use *note Parameter.replace(): 3417. to + create a modified copy. + + Changed in version 3.5: Parameter objects are picklable and + hashable. + + -- Attribute: empty + + A special class-level marker to specify absence of default + values and annotations. + + -- Attribute: name + + The name of the parameter as a string. The name must be a + valid Python identifier. + + `CPython implementation detail:' CPython generates implicit + parameter names of the form ‘.0’ on the code objects used to + implement comprehensions and generator expressions. + + Changed in version 3.6: These parameter names are exposed by + this module as names like ‘implicit0’. + + -- Attribute: default + + The default value for the parameter. If the parameter has no + default value, this attribute is set to *note Parameter.empty: + 3418. + + -- Attribute: annotation + + The annotation for the parameter. If the parameter has no + annotation, this attribute is set to *note Parameter.empty: + 3418. + + -- Attribute: kind + + Describes how argument values are bound to the parameter. + Possible values (accessible via *note Parameter: 6f4, like + ‘Parameter.KEYWORD_ONLY’): + + Name Meaning + + -------------------------------------------------------------------------------- + + `POSITIONAL_ONLY' Value must be supplied as a positional argument. + Positional only parameters are those which + appear before a ‘/’ entry (if present) in a + Python function definition. + + + `POSITIONAL_OR_KEYWORD' Value may be supplied as either a keyword or + positional argument (this is the standard + binding behaviour for functions implemented in + Python.) + + + `VAR_POSITIONAL' A tuple of positional arguments that aren’t + bound to any other parameter. This corresponds + to a ‘*args’ parameter in a Python function + definition. + + + `KEYWORD_ONLY' Value must be supplied as a keyword argument. + Keyword only parameters are those which appear + after a ‘*’ or ‘*args’ entry in a Python + function definition. + + + `VAR_KEYWORD' A dict of keyword arguments that aren’t bound to + any other parameter. This corresponds to a + ‘**kwargs’ parameter in a Python function + definition. + + + Example: print all keyword-only arguments without default + values: + + >>> def foo(a, b, *, c, d=10): + ... pass + + >>> sig = signature(foo) + >>> for param in sig.parameters.values(): + ... if (param.kind == param.KEYWORD_ONLY and + ... param.default is param.empty): + ... print('Parameter:', param) + Parameter: c + + -- Attribute: kind.description + + Describes a enum value of Parameter.kind. + + New in version 3.8. + + Example: print all descriptions of arguments: + + >>> def foo(a, b, *, c, d=10): + ... pass + + >>> sig = signature(foo) + >>> for param in sig.parameters.values(): + ... print(param.kind.description) + positional or keyword + positional or keyword + keyword-only + keyword-only + + -- Method: replace (*[, name][, kind][, default][, annotation]) + + Create a new Parameter instance based on the instance + replaced was invoked on. To override a *note Parameter: + 6f4. attribute, pass the corresponding argument. To + remove a default value or/and an annotation from a + Parameter, pass *note Parameter.empty: 3418. + + >>> from inspect import Parameter + >>> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42) + >>> str(param) + 'foo=42' + + >>> str(param.replace()) # Will create a shallow copy of 'param' + 'foo=42' + + >>> str(param.replace(default=Parameter.empty, annotation='spam')) + "foo:'spam'" + + Changed in version 3.4: In Python 3.3 Parameter objects were + allowed to have ‘name’ set to ‘None’ if their ‘kind’ was set + to ‘POSITIONAL_ONLY’. This is no longer permitted. + + -- Class: inspect.BoundArguments + + Result of a *note Signature.bind(): 3415. or *note + Signature.bind_partial(): 3416. call. Holds the mapping of + arguments to the function’s parameters. + + -- Attribute: arguments + + An ordered, mutable mapping (*note collections.OrderedDict: + 1b9.) of parameters’ names to arguments’ values. Contains + only explicitly bound arguments. Changes in *note arguments: + 341e. will reflect in *note args: 341f. and *note kwargs: + 3420. + + Should be used in conjunction with *note Signature.parameters: + 3411. for any argument processing purposes. + + Note: Arguments for which *note Signature.bind(): 3415. + or *note Signature.bind_partial(): 3416. relied on a + default value are skipped. However, if needed, use *note + BoundArguments.apply_defaults(): 6f5. to add them. + + -- Attribute: args + + A tuple of positional arguments values. Dynamically computed + from the *note arguments: 341e. attribute. + + -- Attribute: kwargs + + A dict of keyword arguments values. Dynamically computed from + the *note arguments: 341e. attribute. + + -- Attribute: signature + + A reference to the parent *note Signature: 6f3. object. + + -- Method: apply_defaults () + + Set default values for missing arguments. + + For variable-positional arguments (‘*args’) the default is an + empty tuple. + + For variable-keyword arguments (‘**kwargs’) the default is an + empty dict. + + >>> def foo(a, b='ham', *args): pass + >>> ba = inspect.signature(foo).bind('spam') + >>> ba.apply_defaults() + >>> ba.arguments + OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())]) + + New in version 3.5. + + The *note args: 341f. and *note kwargs: 3420. properties can be + used to invoke functions: + + def test(a, *, b): + ... + + sig = signature(test) + ba = sig.bind(10, b=20) + test(*ba.args, **ba.kwargs) + +See also +........ + +PEP 362(1) - Function Signature Object. + + The detailed specification, implementation details and examples. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0362 + + +File: python.info, Node: Classes and functions<2>, Next: The interpreter stack, Prev: Introspecting callables with the Signature object, Up: inspect — Inspect live objects + +5.29.13.4 Classes and functions +............................... + + -- Function: inspect.getclasstree (classes, unique=False) + + Arrange the given list of classes into a hierarchy of nested lists. + Where a nested list appears, it contains classes derived from the + class whose entry immediately precedes the list. Each entry is a + 2-tuple containing a class and a tuple of its base classes. If the + `unique' argument is true, exactly one entry appears in the + returned structure for each class in the given list. Otherwise, + classes using multiple inheritance and their descendants will + appear multiple times. + + -- Function: inspect.getargspec (func) + + Get the names and default values of a Python function’s parameters. + A *note named tuple: aa3. ‘ArgSpec(args, varargs, keywords, + defaults)’ is returned. `args' is a list of the parameter names. + `varargs' and `keywords' are the names of the ‘*’ and ‘**’ + parameters or ‘None’. `defaults' is a tuple of default argument + values or ‘None’ if there are no default arguments; if this tuple + has `n' elements, they correspond to the last `n' elements listed + in `args'. + + Deprecated since version 3.0: Use *note getfullargspec(): 55d. for + an updated API that is usually a drop-in replacement, but also + correctly handles function annotations and keyword-only parameters. + + Alternatively, use *note signature(): 55b. and *note Signature + Object: 340f, which provide a more structured introspection API for + callables. + + -- Function: inspect.getfullargspec (func) + + Get the names and default values of a Python function’s parameters. + A *note named tuple: aa3. is returned: + + ‘FullArgSpec(args, varargs, varkw, defaults, kwonlyargs, + kwonlydefaults, annotations)’ + + `args' is a list of the positional parameter names. `varargs' is + the name of the ‘*’ parameter or ‘None’ if arbitrary positional + arguments are not accepted. `varkw' is the name of the ‘**’ + parameter or ‘None’ if arbitrary keyword arguments are not + accepted. `defaults' is an `n'-tuple of default argument values + corresponding to the last `n' positional parameters, or ‘None’ if + there are no such defaults defined. `kwonlyargs' is a list of + keyword-only parameter names in declaration order. + `kwonlydefaults' is a dictionary mapping parameter names from + `kwonlyargs' to the default values used if no argument is supplied. + `annotations' is a dictionary mapping parameter names to + annotations. The special key ‘"return"’ is used to report the + function return value annotation (if any). + + Note that *note signature(): 55b. and *note Signature Object: 340f. + provide the recommended API for callable introspection, and support + additional behaviours (like positional-only arguments) that are + sometimes encountered in extension module APIs. This function is + retained primarily for use in code that needs to maintain + compatibility with the Python 2 ‘inspect’ module API. + + Changed in version 3.4: This function is now based on *note + signature(): 55b, but still ignores ‘__wrapped__’ attributes and + includes the already bound first parameter in the signature output + for bound methods. + + Changed in version 3.6: This method was previously documented as + deprecated in favour of *note signature(): 55b. in Python 3.5, but + that decision has been reversed in order to restore a clearly + supported standard interface for single-source Python 2/3 code + migrating away from the legacy *note getargspec(): 55c. API. + + Changed in version 3.7: Python only explicitly guaranteed that it + preserved the declaration order of keyword-only parameters as of + version 3.7, although in practice this order had always been + preserved in Python 3. + + -- Function: inspect.getargvalues (frame) + + Get information about arguments passed into a particular frame. A + *note named tuple: aa3. ‘ArgInfo(args, varargs, keywords, locals)’ + is returned. `args' is a list of the argument names. `varargs' + and `keywords' are the names of the ‘*’ and ‘**’ arguments or + ‘None’. `locals' is the locals dictionary of the given frame. + + Note: This function was inadvertently marked as deprecated in + Python 3.5. + + -- Function: inspect.formatargspec (args[, varargs, varkw, defaults, + kwonlyargs, kwonlydefaults, annotations[, formatarg, + formatvarargs, formatvarkw, formatvalue, formatreturns, + formatannotations]]) + + Format a pretty argument spec from the values returned by *note + getfullargspec(): 55d. + + The first seven arguments are (‘args’, ‘varargs’, ‘varkw’, + ‘defaults’, ‘kwonlyargs’, ‘kwonlydefaults’, ‘annotations’). + + The other six arguments are functions that are called to turn + argument names, ‘*’ argument name, ‘**’ argument name, default + values, return annotation and individual annotations into strings, + respectively. + + For example: + + >>> from inspect import formatargspec, getfullargspec + >>> def f(a: int, b: float): + ... pass + ... + >>> formatargspec(*getfullargspec(f)) + '(a: int, b: float)' + + Deprecated since version 3.5: Use *note signature(): 55b. and *note + Signature Object: 340f, which provide a better introspecting API + for callables. + + -- Function: inspect.formatargvalues (args[, varargs, varkw, locals, + formatarg, formatvarargs, formatvarkw, formatvalue]) + + Format a pretty argument spec from the four values returned by + *note getargvalues(): 7b8. The format* arguments are the + corresponding optional formatting functions that are called to turn + names and values into strings. + + Note: This function was inadvertently marked as deprecated in + Python 3.5. + + -- Function: inspect.getmro (cls) + + Return a tuple of class cls’s base classes, including cls, in + method resolution order. No class appears more than once in this + tuple. Note that the method resolution order depends on cls’s + type. Unless a very peculiar user-defined metatype is in use, cls + will be the first element of the tuple. + + -- Function: inspect.getcallargs (func, /, *args, **kwds) + + Bind the `args' and `kwds' to the argument names of the Python + function or method `func', as if it was called with them. For + bound methods, bind also the first argument (typically named + ‘self’) to the associated instance. A dict is returned, mapping + the argument names (including the names of the ‘*’ and ‘**’ + arguments, if any) to their values from `args' and `kwds'. In case + of invoking `func' incorrectly, i.e. whenever ‘func(*args, + **kwds)’ would raise an exception because of incompatible + signature, an exception of the same type and the same or similar + message is raised. For example: + + >>> from inspect import getcallargs + >>> def f(a, b=1, *pos, **named): + ... pass + >>> getcallargs(f, 1, 2, 3) == {'a': 1, 'named': {}, 'b': 2, 'pos': (3,)} + True + >>> getcallargs(f, a=2, x=4) == {'a': 2, 'named': {'x': 4}, 'b': 1, 'pos': ()} + True + >>> getcallargs(f) + Traceback (most recent call last): + ... + TypeError: f() missing 1 required positional argument: 'a' + + New in version 3.2. + + Deprecated since version 3.5: Use *note Signature.bind(): 3415. and + *note Signature.bind_partial(): 3416. instead. + + -- Function: inspect.getclosurevars (func) + + Get the mapping of external name references in a Python function or + method `func' to their current values. A *note named tuple: aa3. + ‘ClosureVars(nonlocals, globals, builtins, unbound)’ is returned. + `nonlocals' maps referenced names to lexical closure variables, + `globals' to the function’s module globals and `builtins' to the + builtins visible from the function body. `unbound' is the set of + names referenced in the function that could not be resolved at all + given the current module globals and builtins. + + *note TypeError: 192. is raised if `func' is not a Python function + or method. + + New in version 3.3. + + -- Function: inspect.unwrap (func, *, stop=None) + + Get the object wrapped by `func'. It follows the chain of + ‘__wrapped__’ attributes returning the last object in the chain. + + `stop' is an optional callback accepting an object in the wrapper + chain as its sole argument that allows the unwrapping to be + terminated early if the callback returns a true value. If the + callback never returns a true value, the last object in the chain + is returned as usual. For example, *note signature(): 55b. uses + this to stop unwrapping if any object in the chain has a + ‘__signature__’ attribute defined. + + *note ValueError: 1fb. is raised if a cycle is encountered. + + New in version 3.4. + + +File: python.info, Node: The interpreter stack, Next: Fetching attributes statically, Prev: Classes and functions<2>, Up: inspect — Inspect live objects + +5.29.13.5 The interpreter stack +............................... + +When the following functions return “frame records,” each record is a +*note named tuple: aa3. ‘FrameInfo(frame, filename, lineno, function, +code_context, index)’. The tuple contains the frame object, the +filename, the line number of the current line, the function name, a list +of lines of context from the source code, and the index of the current +line within that list. + +Changed in version 3.5: Return a named tuple instead of a tuple. + + Note: Keeping references to frame objects, as found in the first + element of the frame records these functions return, can cause your + program to create reference cycles. Once a reference cycle has + been created, the lifespan of all objects which can be accessed + from the objects which form the cycle can become much longer even + if Python’s optional cycle detector is enabled. If such cycles + must be created, it is important to ensure they are explicitly + broken to avoid the delayed destruction of objects and increased + memory consumption which occurs. + + Though the cycle detector will catch these, destruction of the + frames (and local variables) can be made deterministic by removing + the cycle in a *note finally: 182. clause. This is also important + if the cycle detector was disabled when Python was compiled or + using *note gc.disable(): 33e1. For example: + + def handle_stackframe_without_leak(): + frame = inspect.currentframe() + try: + # do something with the frame + finally: + del frame + + If you want to keep the frame around (for example to print a + traceback later), you can also break reference cycles by using the + *note frame.clear(): 80b. method. + +The optional `context' argument supported by most of these functions +specifies the number of lines of context to return, which are centered +around the current line. + + -- Function: inspect.getframeinfo (frame, context=1) + + Get information about a frame or traceback object. A *note named + tuple: aa3. ‘Traceback(filename, lineno, function, code_context, + index)’ is returned. + + -- Function: inspect.getouterframes (frame, context=1) + + Get a list of frame records for a frame and all outer frames. + These frames represent the calls that lead to the creation of + `frame'. The first entry in the returned list represents `frame'; + the last entry represents the outermost call on `frame'’s stack. + + Changed in version 3.5: A list of *note named tuples: aa3. + ‘FrameInfo(frame, filename, lineno, function, code_context, index)’ + is returned. + + -- Function: inspect.getinnerframes (traceback, context=1) + + Get a list of frame records for a traceback’s frame and all inner + frames. These frames represent calls made as a consequence of + `frame'. The first entry in the list represents `traceback'; the + last entry represents where the exception was raised. + + Changed in version 3.5: A list of *note named tuples: aa3. + ‘FrameInfo(frame, filename, lineno, function, code_context, index)’ + is returned. + + -- Function: inspect.currentframe () + + Return the frame object for the caller’s stack frame. + + `CPython implementation detail:' This function relies on Python + stack frame support in the interpreter, which isn’t guaranteed to + exist in all implementations of Python. If running in an + implementation without Python stack frame support this function + returns ‘None’. + + -- Function: inspect.stack (context=1) + + Return a list of frame records for the caller’s stack. The first + entry in the returned list represents the caller; the last entry + represents the outermost call on the stack. + + Changed in version 3.5: A list of *note named tuples: aa3. + ‘FrameInfo(frame, filename, lineno, function, code_context, index)’ + is returned. + + -- Function: inspect.trace (context=1) + + Return a list of frame records for the stack between the current + frame and the frame in which an exception currently being handled + was raised in. The first entry in the list represents the caller; + the last entry represents where the exception was raised. + + Changed in version 3.5: A list of *note named tuples: aa3. + ‘FrameInfo(frame, filename, lineno, function, code_context, index)’ + is returned. + + +File: python.info, Node: Fetching attributes statically, Next: Current State of Generators and Coroutines, Prev: The interpreter stack, Up: inspect — Inspect live objects + +5.29.13.6 Fetching attributes statically +........................................ + +Both *note getattr(): 448. and *note hasattr(): 447. can trigger code +execution when fetching or checking for the existence of attributes. +Descriptors, like properties, will be invoked and *note __getattr__(): +31a. and *note __getattribute__(): 449. may be called. + +For cases where you want passive introspection, like documentation +tools, this can be inconvenient. *note getattr_static(): bcb. has the +same signature as *note getattr(): 448. but avoids executing code when +it fetches attributes. + + -- Function: inspect.getattr_static (obj, attr, default=None) + + Retrieve attributes without triggering dynamic lookup via the + descriptor protocol, *note __getattr__(): 31a. or *note + __getattribute__(): 449. + + Note: this function may not be able to retrieve all attributes that + getattr can fetch (like dynamically created attributes) and may + find attributes that getattr can’t (like descriptors that raise + AttributeError). It can also return descriptors objects instead of + instance members. + + If the instance *note __dict__: 4fc. is shadowed by another member + (for example a property) then this function will be unable to find + instance members. + + New in version 3.2. + +*note getattr_static(): bcb. does not resolve descriptors, for example +slot descriptors or getset descriptors on objects implemented in C. The +descriptor object is returned instead of the underlying attribute. + +You can handle these with code like the following. Note that for +arbitrary getset descriptors invoking these may trigger code execution: + + # example code for resolving the builtin descriptor types + class _foo: + __slots__ = ['foo'] + + slot_descriptor = type(_foo.foo) + getset_descriptor = type(type(open(__file__)).name) + wrapper_descriptor = type(str.__dict__['__add__']) + descriptor_types = (slot_descriptor, getset_descriptor, wrapper_descriptor) + + result = getattr_static(some_object, 'foo') + if type(result) in descriptor_types: + try: + result = result.__get__() + except AttributeError: + # descriptors can raise AttributeError to + # indicate there is no underlying value + # in which case the descriptor itself will + # have to do + pass + + +File: python.info, Node: Current State of Generators and Coroutines, Next: Code Objects Bit Flags, Prev: Fetching attributes statically, Up: inspect — Inspect live objects + +5.29.13.7 Current State of Generators and Coroutines +.................................................... + +When implementing coroutine schedulers and for other advanced uses of +generators, it is useful to determine whether a generator is currently +executing, is waiting to start or resume or execution, or has already +terminated. *note getgeneratorstate(): bca. allows the current state of +a generator to be determined easily. + + -- Function: inspect.getgeneratorstate (generator) + + Get current state of a generator-iterator. + + Possible states are: + + * GEN_CREATED: Waiting to start execution. + + * GEN_RUNNING: Currently being executed by the interpreter. + + * GEN_SUSPENDED: Currently suspended at a yield expression. + + * GEN_CLOSED: Execution has completed. + + New in version 3.2. + + -- Function: inspect.getcoroutinestate (coroutine) + + Get current state of a coroutine object. The function is intended + to be used with coroutine objects created by *note async def: 284. + functions, but will accept any coroutine-like object that has + ‘cr_running’ and ‘cr_frame’ attributes. + + Possible states are: + + * CORO_CREATED: Waiting to start execution. + + * CORO_RUNNING: Currently being executed by the + interpreter. + + * CORO_SUSPENDED: Currently suspended at an await + expression. + + * CORO_CLOSED: Execution has completed. + + New in version 3.5. + +The current internal state of the generator can also be queried. This +is mostly useful for testing purposes, to ensure that internal state is +being updated as expected: + + -- Function: inspect.getgeneratorlocals (generator) + + Get the mapping of live local variables in `generator' to their + current values. A dictionary is returned that maps from variable + names to values. This is the equivalent of calling *note locals(): + 455. in the body of the generator, and all the same caveats apply. + + If `generator' is a *note generator: 9a2. with no currently + associated frame, then an empty dictionary is returned. *note + TypeError: 192. is raised if `generator' is not a Python generator + object. + + `CPython implementation detail:' This function relies on the + generator exposing a Python stack frame for introspection, which + isn’t guaranteed to be the case in all implementations of Python. + In such cases, this function will always return an empty + dictionary. + + New in version 3.3. + + -- Function: inspect.getcoroutinelocals (coroutine) + + This function is analogous to *note getgeneratorlocals(): a1a, but + works for coroutine objects created by *note async def: 284. + functions. + + New in version 3.5. + + +File: python.info, Node: Code Objects Bit Flags, Next: Command Line Interface<3>, Prev: Current State of Generators and Coroutines, Up: inspect — Inspect live objects + +5.29.13.8 Code Objects Bit Flags +................................ + +Python code objects have a ‘co_flags’ attribute, which is a bitmap of +the following flags: + + -- Data: inspect.CO_OPTIMIZED + + The code object is optimized, using fast locals. + + -- Data: inspect.CO_NEWLOCALS + + If set, a new dict will be created for the frame’s ‘f_locals’ when + the code object is executed. + + -- Data: inspect.CO_VARARGS + + The code object has a variable positional parameter (‘*args’-like). + + -- Data: inspect.CO_VARKEYWORDS + + The code object has a variable keyword parameter (‘**kwargs’-like). + + -- Data: inspect.CO_NESTED + + The flag is set when the code object is a nested function. + + -- Data: inspect.CO_GENERATOR + + The flag is set when the code object is a generator function, i.e. + a generator object is returned when the code object is executed. + + -- Data: inspect.CO_NOFREE + + The flag is set if there are no free or cell variables. + + -- Data: inspect.CO_COROUTINE + + The flag is set when the code object is a coroutine function. When + the code object is executed it returns a coroutine object. See PEP + 492(1) for more details. + + New in version 3.5. + + -- Data: inspect.CO_ITERABLE_COROUTINE + + The flag is used to transform generators into generator-based + coroutines. Generator objects with this flag can be used in + ‘await’ expression, and can ‘yield from’ coroutine objects. See + PEP 492(2) for more details. + + New in version 3.5. + + -- Data: inspect.CO_ASYNC_GENERATOR + + The flag is set when the code object is an asynchronous generator + function. When the code object is executed it returns an + asynchronous generator object. See PEP 525(3) for more details. + + New in version 3.6. + + Note: The flags are specific to CPython, and may not be defined in + other Python implementations. Furthermore, the flags are an + implementation detail, and can be removed or deprecated in future + Python releases. It’s recommended to use public APIs from the + *note inspect: a0. module for any introspection needs. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0492 + + (2) https://www.python.org/dev/peps/pep-0492 + + (3) https://www.python.org/dev/peps/pep-0525 + + +File: python.info, Node: Command Line Interface<3>, Prev: Code Objects Bit Flags, Up: inspect — Inspect live objects + +5.29.13.9 Command Line Interface +................................ + +The *note inspect: a0. module also provides a basic introspection +capability from the command line. + +By default, accepts the name of a module and prints the source of that +module. A class or function within the module can be printed instead by +appended a colon and the qualified name of the target object. + + -- Program Option: --details + + Print information about the specified object rather than the source + code + + +File: python.info, Node: site — Site-specific configuration hook, Prev: inspect — Inspect live objects, Up: Python Runtime Services + +5.29.14 ‘site’ — Site-specific configuration hook +------------------------------------------------- + +`Source code:' Lib/site.py(1) + +__________________________________________________________________ + +`This module is automatically imported during initialization.' The +automatic import can be suppressed using the interpreter’s *note -S: +b24. option. + +Importing this module will append site-specific paths to the module +search path and add a few builtins, unless *note -S: b24. was used. In +that case, this module can be safely imported with no automatic +modifications to the module search path or additions to the builtins. +To explicitly trigger the usual site-specific additions, call the *note +site.main(): fe2. function. + +Changed in version 3.3: Importing the module used to trigger paths +manipulation even when using *note -S: b24. + +It starts by constructing up to four directories from a head and a tail +part. For the head part, it uses ‘sys.prefix’ and ‘sys.exec_prefix’; +empty heads are skipped. For the tail part, it uses the empty string +and then ‘lib/site-packages’ (on Windows) or +‘lib/python`X.Y'/site-packages’ (on Unix and Macintosh). For each of +the distinct head-tail combinations, it sees if it refers to an existing +directory, and if so, adds it to ‘sys.path’ and also inspects the newly +added path for configuration files. + +Changed in version 3.5: Support for the “site-python” directory has been +removed. + +If a file named “pyvenv.cfg” exists one directory above sys.executable, +sys.prefix and sys.exec_prefix are set to that directory and it is also +checked for site-packages (sys.base_prefix and sys.base_exec_prefix will +always be the “real” prefixes of the Python installation). If +“pyvenv.cfg” (a bootstrap configuration file) contains the key +“include-system-site-packages” set to anything other than “true” +(case-insensitive), the system-level prefixes will not be searched for +site-packages; otherwise they will. + +A path configuration file is a file whose name has the form ‘`name'.pth’ +and exists in one of the four directories mentioned above; its contents +are additional items (one per line) to be added to ‘sys.path’. +Non-existing items are never added to ‘sys.path’, and no check is made +that the item refers to a directory rather than a file. No item is +added to ‘sys.path’ more than once. Blank lines and lines beginning +with ‘#’ are skipped. Lines starting with ‘import’ (followed by space +or tab) are executed. + + Note: An executable line in a ‘.pth’ file is run at every Python + startup, regardless of whether a particular module is actually + going to be used. Its impact should thus be kept to a minimum. + The primary intended purpose of executable lines is to make the + corresponding module(s) importable (load 3rd-party import hooks, + adjust ‘PATH’ etc). Any other initialization is supposed to be + done upon a module’s actual import, if and when it happens. + Limiting a code chunk to a single line is a deliberate measure to + discourage putting anything more complex here. + +For example, suppose ‘sys.prefix’ and ‘sys.exec_prefix’ are set to +‘/usr/local’. The Python X.Y library is then installed in +‘/usr/local/lib/python`X.Y'’. Suppose this has a subdirectory +‘/usr/local/lib/python`X.Y'/site-packages’ with three subsubdirectories, +‘foo’, ‘bar’ and ‘spam’, and two path configuration files, ‘foo.pth’ and +‘bar.pth’. Assume ‘foo.pth’ contains the following: + + # foo package configuration + + foo + bar + bletch + +and ‘bar.pth’ contains: + + # bar package configuration + + bar + +Then the following version-specific directories are added to ‘sys.path’, +in this order: + + /usr/local/lib/pythonX.Y/site-packages/bar + /usr/local/lib/pythonX.Y/site-packages/foo + +Note that ‘bletch’ is omitted because it doesn’t exist; the ‘bar’ +directory precedes the ‘foo’ directory because ‘bar.pth’ comes +alphabetically before ‘foo.pth’; and ‘spam’ is omitted because it is not +mentioned in either path configuration file. + +After these path manipulations, an attempt is made to import a module +named ‘sitecustomize’, which can perform arbitrary site-specific +customizations. It is typically created by a system administrator in +the site-packages directory. If this import fails with an *note +ImportError: 334. or its subclass exception, and the exception’s ‘name’ +attribute equals to ‘'sitecustomize'’, it is silently ignored. If +Python is started without output streams available, as with +‘pythonw.exe’ on Windows (which is used by default to start IDLE), +attempted output from ‘sitecustomize’ is ignored. Any other exception +causes a silent and perhaps mysterious failure of the process. + +After this, an attempt is made to import a module named ‘usercustomize’, +which can perform arbitrary user-specific customizations, if *note +ENABLE_USER_SITE: 343b. is true. This file is intended to be created in +the user site-packages directory (see below), which is part of +‘sys.path’ unless disabled by *note -s: d15. If this import fails with +an *note ImportError: 334. or its subclass exception, and the +exception’s ‘name’ attribute equals to ‘'usercustomize'’, it is silently +ignored. + +Note that for some non-Unix systems, ‘sys.prefix’ and ‘sys.exec_prefix’ +are empty, and the path manipulations are skipped; however the import of +‘sitecustomize’ and ‘usercustomize’ is still attempted. + +* Menu: + +* Readline configuration:: +* Module contents: Module contents<3>. +* Command Line Interface: Command Line Interface<4>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/site.py + + +File: python.info, Node: Readline configuration, Next: Module contents<3>, Up: site — Site-specific configuration hook + +5.29.14.1 Readline configuration +................................ + +On systems that support *note readline: de, this module will also import +and configure the *note rlcompleter: e1. module, if Python is started in +*note interactive mode: 8e3. and without the *note -S: b24. option. The +default behavior is enable tab-completion and to use ‘~/.python_history’ +as the history save file. To disable it, delete (or override) the *note +sys.__interactivehook__: 8e4. attribute in your ‘sitecustomize’ or +‘usercustomize’ module or your *note PYTHONSTARTUP: 8e5. file. + +Changed in version 3.4: Activation of rlcompleter and history was made +automatic. + + +File: python.info, Node: Module contents<3>, Next: Command Line Interface<4>, Prev: Readline configuration, Up: site — Site-specific configuration hook + +5.29.14.2 Module contents +......................... + + -- Data: site.PREFIXES + + A list of prefixes for site-packages directories. + + -- Data: site.ENABLE_USER_SITE + + Flag showing the status of the user site-packages directory. + ‘True’ means that it is enabled and was added to ‘sys.path’. + ‘False’ means that it was disabled by user request (with *note -s: + d15. or *note PYTHONNOUSERSITE: d16.). ‘None’ means it was + disabled for security reasons (mismatch between user or group id + and effective id) or by an administrator. + + -- Data: site.USER_SITE + + Path to the user site-packages for the running Python. Can be + ‘None’ if *note getusersitepackages(): bd5. hasn’t been called yet. + Default value is ‘~/.local/lib/python`X.Y'/site-packages’ for UNIX + and non-framework Mac OS X builds, + ‘~/Library/Python/`X.Y'/lib/python/site-packages’ for Mac framework + builds, and ‘`%APPDATA%'\Python\Python`XY'\site-packages’ on + Windows. This directory is a site directory, which means that + ‘.pth’ files in it will be processed. + + -- Data: site.USER_BASE + + Path to the base directory for the user site-packages. Can be + ‘None’ if *note getuserbase(): bd4. hasn’t been called yet. + Default value is ‘~/.local’ for UNIX and Mac OS X non-framework + builds, ‘~/Library/Python/`X.Y'’ for Mac framework builds, and + ‘`%APPDATA%'\Python’ for Windows. This value is used by Distutils + to compute the installation directories for scripts, data files, + Python modules, etc. for the *note user installation scheme: ff4. + See also *note PYTHONUSERBASE: d14. + + -- Function: site.main () + + Adds all the standard site-specific directories to the module + search path. This function is called automatically when this + module is imported, unless the Python interpreter was started with + the *note -S: b24. flag. + + Changed in version 3.3: This function used to be called + unconditionally. + + -- Function: site.addsitedir (sitedir, known_paths=None) + + Add a directory to sys.path and process its ‘.pth’ files. + Typically used in ‘sitecustomize’ or ‘usercustomize’ (see above). + + -- Function: site.getsitepackages () + + Return a list containing all global site-packages directories. + + New in version 3.2. + + -- Function: site.getuserbase () + + Return the path of the user base directory, *note USER_BASE: ff3. + If it is not initialized yet, this function will also set it, + respecting *note PYTHONUSERBASE: d14. + + New in version 3.2. + + -- Function: site.getusersitepackages () + + Return the path of the user-specific site-packages directory, *note + USER_SITE: fe1. If it is not initialized yet, this function will + also set it, respecting *note USER_BASE: ff3. To determine if the + user-specific site-packages was added to ‘sys.path’ *note + ENABLE_USER_SITE: 343b. should be used. + + New in version 3.2. + + +File: python.info, Node: Command Line Interface<4>, Prev: Module contents<3>, Up: site — Site-specific configuration hook + +5.29.14.3 Command Line Interface +................................ + +The *note site: eb. module also provides a way to get the user +directories from the command line: + + $ python3 -m site --user-site + /home/user/.local/lib/python3.3/site-packages + +If it is called without arguments, it will print the contents of *note +sys.path: 488. on the standard output, followed by the value of *note +USER_BASE: ff3. and whether the directory exists, then the same thing +for *note USER_SITE: fe1, and finally the value of *note +ENABLE_USER_SITE: 343b. + + -- Program Option: --user-base + + Print the path to the user base directory. + + -- Program Option: --user-site + + Print the path to the user site-packages directory. + +If both options are given, user base and user site will be printed +(always in this order), separated by *note os.pathsep: ff0. + +If any option is given, the script will exit with one of these values: +‘0’ if the user site-packages directory is enabled, ‘1’ if it was +disabled by the user, ‘2’ if it is disabled for security reasons or by +an administrator, and a value greater than 2 if there is an error. + +See also +........ + +PEP 370(1) – Per user site-packages directory + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0370 + + +File: python.info, Node: Custom Python Interpreters, Next: Importing Modules, Prev: Python Runtime Services, Up: The Python Standard Library + +5.30 Custom Python Interpreters +=============================== + +The modules described in this chapter allow writing interfaces similar +to Python’s interactive interpreter. If you want a Python interpreter +that supports some special feature in addition to the Python language, +you should look at the *note code: 1b. module. (The *note codeop: 1d. +module is lower-level, used to support compiling a possibly-incomplete +chunk of Python code.) + +The full list of modules described in this chapter is: + +* Menu: + +* code — Interpreter base classes:: +* codeop — Compile Python code:: + + +File: python.info, Node: code — Interpreter base classes, Next: codeop — Compile Python code, Up: Custom Python Interpreters + +5.30.1 ‘code’ — Interpreter base classes +---------------------------------------- + +`Source code:' Lib/code.py(1) + +__________________________________________________________________ + +The ‘code’ module provides facilities to implement read-eval-print loops +in Python. Two classes and convenience functions are included which can +be used to build applications which provide an interactive interpreter +prompt. + + -- Class: code.InteractiveInterpreter (locals=None) + + This class deals with parsing and interpreter state (the user’s + namespace); it does not deal with input buffering or prompting or + input file naming (the filename is always passed in explicitly). + The optional `locals' argument specifies the dictionary in which + code will be executed; it defaults to a newly created dictionary + with key ‘'__name__'’ set to ‘'__console__'’ and key ‘'__doc__'’ + set to ‘None’. + + -- Class: code.InteractiveConsole (locals=None, filename="<console>") + + Closely emulate the behavior of the interactive Python interpreter. + This class builds on *note InteractiveInterpreter: 3449. and adds + prompting using the familiar ‘sys.ps1’ and ‘sys.ps2’, and input + buffering. + + -- Function: code.interact (banner=None, readfunc=None, local=None, + exitmsg=None) + + Convenience function to run a read-eval-print loop. This creates a + new instance of *note InteractiveConsole: 14a9. and sets `readfunc' + to be used as the *note InteractiveConsole.raw_input(): 344b. + method, if provided. If `local' is provided, it is passed to the + *note InteractiveConsole: 14a9. constructor for use as the default + namespace for the interpreter loop. The *note interact(): 344a. + method of the instance is then run with `banner' and `exitmsg' + passed as the banner and exit message to use, if provided. The + console object is discarded after use. + + Changed in version 3.6: Added `exitmsg' parameter. + + -- Function: code.compile_command (source, filename="<input>", + symbol="single") + + This function is useful for programs that want to emulate Python’s + interpreter main loop (a.k.a. the read-eval-print loop). The + tricky part is to determine when the user has entered an incomplete + command that can be completed by entering more text (as opposed to + a complete command or a syntax error). This function `almost' + always makes the same decision as the real interpreter main loop. + + `source' is the source string; `filename' is the optional filename + from which source was read, defaulting to ‘'<input>'’; and `symbol' + is the optional grammar start symbol, which should be ‘'single'’ + (the default), ‘'eval'’ or ‘'exec'’. + + Returns a code object (the same as ‘compile(source, filename, + symbol)’) if the command is complete and valid; ‘None’ if the + command is incomplete; raises *note SyntaxError: 458. if the + command is complete and contains a syntax error, or raises *note + OverflowError: 960. or *note ValueError: 1fb. if the command + contains an invalid literal. + +* Menu: + +* Interactive Interpreter Objects:: +* Interactive Console Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/code.py + + +File: python.info, Node: Interactive Interpreter Objects, Next: Interactive Console Objects, Up: code — Interpreter base classes + +5.30.1.1 Interactive Interpreter Objects +........................................ + + -- Method: InteractiveInterpreter.runsource (source, + filename="<input>", symbol="single") + + Compile and run some source in the interpreter. Arguments are the + same as for *note compile_command(): 344c.; the default for + `filename' is ‘'<input>'’, and for `symbol' is ‘'single'’. One of + several things can happen: + + * The input is incorrect; *note compile_command(): 344c. raised + an exception (*note SyntaxError: 458. or *note OverflowError: + 960.). A syntax traceback will be printed by calling the + *note showsyntaxerror(): 3450. method. *note runsource(): + 344f. returns ‘False’. + + * The input is incomplete, and more input is required; *note + compile_command(): 344c. returned ‘None’. *note runsource(): + 344f. returns ‘True’. + + * The input is complete; *note compile_command(): 344c. returned + a code object. The code is executed by calling the *note + runcode(): 3451. (which also handles run-time exceptions, + except for *note SystemExit: 5fc.). *note runsource(): 344f. + returns ‘False’. + + The return value can be used to decide whether to use ‘sys.ps1’ or + ‘sys.ps2’ to prompt the next line. + + -- Method: InteractiveInterpreter.runcode (code) + + Execute a code object. When an exception occurs, *note + showtraceback(): 6a7. is called to display a traceback. All + exceptions are caught except *note SystemExit: 5fc, which is + allowed to propagate. + + A note about *note KeyboardInterrupt: 197.: this exception may + occur elsewhere in this code, and may not always be caught. The + caller should be prepared to deal with it. + + -- Method: InteractiveInterpreter.showsyntaxerror (filename=None) + + Display the syntax error that just occurred. This does not display + a stack trace because there isn’t one for syntax errors. If + `filename' is given, it is stuffed into the exception instead of + the default filename provided by Python’s parser, because it always + uses ‘'<string>'’ when reading from a string. The output is + written by the *note write(): 3452. method. + + -- Method: InteractiveInterpreter.showtraceback () + + Display the exception that just occurred. We remove the first + stack item because it is within the interpreter object + implementation. The output is written by the *note write(): 3452. + method. + + Changed in version 3.5: The full chained traceback is displayed + instead of just the primary traceback. + + -- Method: InteractiveInterpreter.write (data) + + Write a string to the standard error stream (‘sys.stderr’). + Derived classes should override this to provide the appropriate + output handling as needed. + + +File: python.info, Node: Interactive Console Objects, Prev: Interactive Interpreter Objects, Up: code — Interpreter base classes + +5.30.1.2 Interactive Console Objects +.................................... + +The *note InteractiveConsole: 14a9. class is a subclass of *note +InteractiveInterpreter: 3449, and so offers all the methods of the +interpreter objects as well as the following additions. + + -- Method: InteractiveConsole.interact (banner=None, exitmsg=None) + + Closely emulate the interactive Python console. The optional + `banner' argument specify the banner to print before the first + interaction; by default it prints a banner similar to the one + printed by the standard Python interpreter, followed by the class + name of the console object in parentheses (so as not to confuse + this with the real interpreter – since it’s so close!). + + The optional `exitmsg' argument specifies an exit message printed + when exiting. Pass the empty string to suppress the exit message. + If `exitmsg' is not given or ‘None’, a default message is printed. + + Changed in version 3.4: To suppress printing any banner, pass an + empty string. + + Changed in version 3.6: Print an exit message when exiting. + + -- Method: InteractiveConsole.push (line) + + Push a line of source text to the interpreter. The line should not + have a trailing newline; it may have internal newlines. The line + is appended to a buffer and the interpreter’s ‘runsource()’ method + is called with the concatenated contents of the buffer as source. + If this indicates that the command was executed or invalid, the + buffer is reset; otherwise, the command is incomplete, and the + buffer is left as it was after the line was appended. The return + value is ‘True’ if more input is required, ‘False’ if the line was + dealt with in some way (this is the same as ‘runsource()’). + + -- Method: InteractiveConsole.resetbuffer () + + Remove any unhandled source text from the input buffer. + + -- Method: InteractiveConsole.raw_input (prompt="") + + Write a prompt and read a line. The returned line does not include + the trailing newline. When the user enters the EOF key sequence, + *note EOFError: c6c. is raised. The base implementation reads from + ‘sys.stdin’; a subclass may replace this with a different + implementation. + + +File: python.info, Node: codeop — Compile Python code, Prev: code — Interpreter base classes, Up: Custom Python Interpreters + +5.30.2 ‘codeop’ — Compile Python code +------------------------------------- + +`Source code:' Lib/codeop.py(1) + +__________________________________________________________________ + +The *note codeop: 1d. module provides utilities upon which the Python +read-eval-print loop can be emulated, as is done in the *note code: 1b. +module. As a result, you probably don’t want to use the module +directly; if you want to include such a loop in your program you +probably want to use the *note code: 1b. module instead. + +There are two parts to this job: + + 1. Being able to tell if a line of input completes a Python statement: + in short, telling whether to print ‘‘>>>’’ or ‘‘...’’ next. + + 2. Remembering which future statements the user has entered, so + subsequent input can be compiled with these in effect. + +The *note codeop: 1d. module provides a way of doing each of these +things, and a way of doing them both. + +To do just the former: + + -- Function: codeop.compile_command (source, filename="<input>", + symbol="single") + + Tries to compile `source', which should be a string of Python code + and return a code object if `source' is valid Python code. In that + case, the filename attribute of the code object will be `filename', + which defaults to ‘'<input>'’. Returns ‘None’ if `source' is `not' + valid Python code, but is a prefix of valid Python code. + + If there is a problem with `source', an exception will be raised. + *note SyntaxError: 458. is raised if there is invalid Python + syntax, and *note OverflowError: 960. or *note ValueError: 1fb. if + there is an invalid literal. + + The `symbol' argument determines whether `source' is compiled as a + statement (‘'single'’, the default), as a sequence of statements + (‘'exec'’) or as an *note expression: 3350. (‘'eval'’). Any other + value will cause *note ValueError: 1fb. to be raised. + + Note: It is possible (but not likely) that the parser stops + parsing with a successful outcome before reaching the end of + the source; in this case, trailing symbols may be ignored + instead of causing an error. For example, a backslash + followed by two newlines may be followed by arbitrary garbage. + This will be fixed once the API for the parser is better. + + -- Class: codeop.Compile + + Instances of this class have *note __call__(): 10ce. methods + identical in signature to the built-in function *note compile(): + 1b4, but with the difference that if the instance compiles program + text containing a *note __future__: 0. statement, the instance + ‘remembers’ and compiles all subsequent program texts with the + statement in force. + + -- Class: codeop.CommandCompiler + + Instances of this class have *note __call__(): 10ce. methods + identical in signature to *note compile_command(): 345a.; the + difference is that if the instance compiles program text containing + a ‘__future__’ statement, the instance ‘remembers’ and compiles all + subsequent program texts with the statement in force. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/codeop.py + + +File: python.info, Node: Importing Modules, Next: Python Language Services, Prev: Custom Python Interpreters, Up: The Python Standard Library + +5.31 Importing Modules +====================== + +The modules described in this chapter provide new ways to import other +Python modules and hooks for customizing the import process. + +The full list of modules described in this chapter is: + +* Menu: + +* zipimport — Import modules from Zip archives:: +* pkgutil — Package extension utility:: +* modulefinder — Find modules used by a script:: +* runpy — Locating and executing Python modules:: +* importlib — The implementation of import:: +* Using importlib.metadata: Using importlib metadata. + + +File: python.info, Node: zipimport — Import modules from Zip archives, Next: pkgutil — Package extension utility, Up: Importing Modules + +5.31.1 ‘zipimport’ — Import modules from Zip archives +----------------------------------------------------- + +`Source code:' Lib/zipimport.py(1) + +__________________________________________________________________ + +This module adds the ability to import Python modules (‘*.py’, ‘*.pyc’) +and packages from ZIP-format archives. It is usually not needed to use +the *note zipimport: 143. module explicitly; it is automatically used by +the built-in *note import: c1d. mechanism for *note sys.path: 488. items +that are paths to ZIP archives. + +Typically, *note sys.path: 488. is a list of directory names as strings. +This module also allows an item of *note sys.path: 488. to be a string +naming a ZIP file archive. The ZIP archive can contain a subdirectory +structure to support package imports, and a path within the archive can +be specified to only import from a subdirectory. For example, the path +‘example.zip/lib/’ would only import from the ‘lib/’ subdirectory within +the archive. + +Any files may be present in the ZIP archive, but only files ‘.py’ and +‘.pyc’ are available for import. ZIP import of dynamic modules (‘.pyd’, +‘.so’) is disallowed. Note that if an archive only contains ‘.py’ +files, Python will not attempt to modify the archive by adding the +corresponding ‘.pyc’ file, meaning that if a ZIP archive doesn’t contain +‘.pyc’ files, importing may be rather slow. + +Changed in version 3.8: Previously, ZIP archives with an archive comment +were not supported. + +See also +........ + +PKZIP Application Note(2) + + Documentation on the ZIP file format by Phil Katz, the creator of + the format and algorithms used. + +PEP 273(3) - Import Modules from Zip Archives + + Written by James C. Ahlstrom, who also provided an implementation. + Python 2.3 follows the specification in PEP 273(4), but uses an + implementation written by Just van Rossum that uses the import + hooks described in PEP 302(5). + +PEP 302(6) - New Import Hooks + + The PEP to add the import hooks that help this module work. + +This module defines an exception: + + -- Exception: zipimport.ZipImportError + + Exception raised by zipimporter objects. It’s a subclass of *note + ImportError: 334, so it can be caught as *note ImportError: 334, + too. + +* Menu: + +* zipimporter Objects:: +* Examples: Examples<28>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/zipimport.py + + (2) https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT + + (3) https://www.python.org/dev/peps/pep-0273 + + (4) https://www.python.org/dev/peps/pep-0273 + + (5) https://www.python.org/dev/peps/pep-0302 + + (6) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: zipimporter Objects, Next: Examples<28>, Up: zipimport — Import modules from Zip archives + +5.31.1.1 zipimporter Objects +............................ + +*note zipimporter: 3465. is the class for importing ZIP files. + + -- Class: zipimport.zipimporter (archivepath) + + Create a new zipimporter instance. `archivepath' must be a path to + a ZIP file, or to a specific path within a ZIP file. For example, + an `archivepath' of ‘foo/bar.zip/lib’ will look for modules in the + ‘lib’ directory inside the ZIP file ‘foo/bar.zip’ (provided that it + exists). + + *note ZipImportError: 3462. is raised if `archivepath' doesn’t + point to a valid ZIP archive. + + -- Method: find_module (fullname[, path]) + + Search for a module specified by `fullname'. `fullname' must + be the fully qualified (dotted) module name. It returns the + zipimporter instance itself if the module was found, or *note + None: 157. if it wasn’t. The optional `path' argument is + ignored—it’s there for compatibility with the importer + protocol. + + -- Method: get_code (fullname) + + Return the code object for the specified module. Raise *note + ZipImportError: 3462. if the module couldn’t be found. + + -- Method: get_data (pathname) + + Return the data associated with `pathname'. Raise *note + OSError: 1d3. if the file wasn’t found. + + Changed in version 3.3: *note IOError: 992. used to be raised + instead of *note OSError: 1d3. + + -- Method: get_filename (fullname) + + Return the value ‘__file__’ would be set to if the specified + module was imported. Raise *note ZipImportError: 3462. if the + module couldn’t be found. + + New in version 3.1. + + -- Method: get_source (fullname) + + Return the source code for the specified module. Raise *note + ZipImportError: 3462. if the module couldn’t be found, return + *note None: 157. if the archive does contain the module, but + has no source for it. + + -- Method: is_package (fullname) + + Return ‘True’ if the module specified by `fullname' is a + package. Raise *note ZipImportError: 3462. if the module + couldn’t be found. + + -- Method: load_module (fullname) + + Load the module specified by `fullname'. `fullname' must be + the fully qualified (dotted) module name. It returns the + imported module, or raises *note ZipImportError: 3462. if it + wasn’t found. + + -- Attribute: archive + + The file name of the importer’s associated ZIP file, without a + possible subpath. + + -- Attribute: prefix + + The subpath within the ZIP file where modules are searched. + This is the empty string for zipimporter objects which point + to the root of the ZIP file. + + The *note archive: 346d. and *note prefix: 346e. attributes, when + combined with a slash, equal the original `archivepath' argument + given to the *note zipimporter: 3465. constructor. + + +File: python.info, Node: Examples<28>, Prev: zipimporter Objects, Up: zipimport — Import modules from Zip archives + +5.31.1.2 Examples +................. + +Here is an example that imports a module from a ZIP archive - note that +the *note zipimport: 143. module is not explicitly used. + + $ unzip -l example.zip + Archive: example.zip + Length Date Time Name + -------- ---- ---- ---- + 8467 11-26-02 22:30 jwzthreading.py + -------- ------- + 8467 1 file + $ ./python + Python 2.3 (#1, Aug 1 2003, 19:54:32) + >>> import sys + >>> sys.path.insert(0, 'example.zip') # Add .zip file to front of path + >>> import jwzthreading + >>> jwzthreading.__file__ + 'example.zip/jwzthreading.py' + + +File: python.info, Node: pkgutil — Package extension utility, Next: modulefinder — Find modules used by a script, Prev: zipimport — Import modules from Zip archives, Up: Importing Modules + +5.31.2 ‘pkgutil’ — Package extension utility +-------------------------------------------- + +`Source code:' Lib/pkgutil.py(1) + +__________________________________________________________________ + +This module provides utilities for the import system, in particular +package support. + + -- Class: pkgutil.ModuleInfo (module_finder, name, ispkg) + + A namedtuple that holds a brief summary of a module’s info. + + New in version 3.6. + + -- Function: pkgutil.extend_path (path, name) + + Extend the search path for the modules which comprise a package. + Intended use is to place the following code in a package’s + ‘__init__.py’: + + from pkgutil import extend_path + __path__ = extend_path(__path__, __name__) + + This will add to the package’s ‘__path__’ all subdirectories of + directories on ‘sys.path’ named after the package. This is useful + if one wants to distribute different parts of a single logical + package as multiple directories. + + It also looks for ‘*.pkg’ files beginning where ‘*’ matches the + `name' argument. This feature is similar to ‘*.pth’ files (see the + *note site: eb. module for more information), except that it + doesn’t special-case lines starting with ‘import’. A ‘*.pkg’ file + is trusted at face value: apart from checking for duplicates, all + entries found in a ‘*.pkg’ file are added to the path, regardless + of whether they exist on the filesystem. (This is a feature.) + + If the input path is not a list (as is the case for frozen + packages) it is returned unchanged. The input path is not + modified; an extended copy is returned. Items are only appended to + the copy at the end. + + It is assumed that *note sys.path: 488. is a sequence. Items of + *note sys.path: 488. that are not strings referring to existing + directories are ignored. Unicode items on *note sys.path: 488. + that cause errors when used as filenames may cause this function to + raise an exception (in line with *note os.path.isdir(): 1f8. + behavior). + + -- Class: pkgutil.ImpImporter (dirname=None) + + PEP 302(2) Finder that wraps Python’s “classic” import algorithm. + + If `dirname' is a string, a PEP 302(3) finder is created that + searches that directory. If `dirname' is ‘None’, a PEP 302(4) + finder is created that searches the current *note sys.path: 488, + plus any modules that are frozen or built-in. + + Note that *note ImpImporter: 3474. does not currently support being + used by placement on *note sys.meta_path: 5e6. + + Deprecated since version 3.3: This emulation is no longer needed, + as the standard import mechanism is now fully PEP 302(5) compliant + and available in *note importlib: 9b. + + -- Class: pkgutil.ImpLoader (fullname, file, filename, etc) + + *note Loader: 1160. that wraps Python’s “classic” import algorithm. + + Deprecated since version 3.3: This emulation is no longer needed, + as the standard import mechanism is now fully PEP 302(6) compliant + and available in *note importlib: 9b. + + -- Function: pkgutil.find_loader (fullname) + + Retrieve a module *note loader: 1160. for the given `fullname'. + + This is a backwards compatibility wrapper around *note + importlib.util.find_spec(): 938. that converts most failures to + *note ImportError: 334. and only returns the loader rather than the + full ‘ModuleSpec’. + + Changed in version 3.3: Updated to be based directly on *note + importlib: 9b. rather than relying on the package internal PEP + 302(7) import emulation. + + Changed in version 3.4: Updated to be based on PEP 451(8) + + -- Function: pkgutil.get_importer (path_item) + + Retrieve a *note finder: 115f. for the given `path_item'. + + The returned finder is cached in *note sys.path_importer_cache: + 49d. if it was newly created by a path hook. + + The cache (or part of it) can be cleared manually if a rescan of + *note sys.path_hooks: 95c. is necessary. + + Changed in version 3.3: Updated to be based directly on *note + importlib: 9b. rather than relying on the package internal PEP + 302(9) import emulation. + + -- Function: pkgutil.get_loader (module_or_name) + + Get a *note loader: 1160. object for `module_or_name'. + + If the module or package is accessible via the normal import + mechanism, a wrapper around the relevant part of that machinery is + returned. Returns ‘None’ if the module cannot be found or + imported. If the named module is not already imported, its + containing package (if any) is imported, in order to establish the + package ‘__path__’. + + Changed in version 3.3: Updated to be based directly on *note + importlib: 9b. rather than relying on the package internal PEP + 302(10) import emulation. + + Changed in version 3.4: Updated to be based on PEP 451(11) + + -- Function: pkgutil.iter_importers (fullname='') + + Yield *note finder: 115f. objects for the given module name. + + If fullname contains a ‘.’, the finders will be for the package + containing fullname, otherwise they will be all registered top + level finders (i.e. those on both sys.meta_path and + sys.path_hooks). + + If the named module is in a package, that package is imported as a + side effect of invoking this function. + + If no module name is specified, all top level finders are produced. + + Changed in version 3.3: Updated to be based directly on *note + importlib: 9b. rather than relying on the package internal PEP + 302(12) import emulation. + + -- Function: pkgutil.iter_modules (path=None, prefix='') + + Yields *note ModuleInfo: 60d. for all submodules on `path', or, if + `path' is ‘None’, all top-level modules on ‘sys.path’. + + `path' should be either ‘None’ or a list of paths to look for + modules in. + + `prefix' is a string to output on the front of every module name on + output. + + Note: Only works for a *note finder: 115f. which defines an + ‘iter_modules()’ method. This interface is non-standard, so + the module also provides implementations for *note + importlib.machinery.FileFinder: 9b6. and *note + zipimport.zipimporter: 3465. + + Changed in version 3.3: Updated to be based directly on *note + importlib: 9b. rather than relying on the package internal PEP + 302(13) import emulation. + + -- Function: pkgutil.walk_packages (path=None, prefix='', onerror=None) + + Yields *note ModuleInfo: 60d. for all modules recursively on + `path', or, if `path' is ‘None’, all accessible modules. + + `path' should be either ‘None’ or a list of paths to look for + modules in. + + `prefix' is a string to output on the front of every module name on + output. + + Note that this function must import all `packages' (`not' all + modules!) on the given `path', in order to access the ‘__path__’ + attribute to find submodules. + + `onerror' is a function which gets called with one argument (the + name of the package which was being imported) if any exception + occurs while trying to import a package. If no `onerror' function + is supplied, *note ImportError: 334.s are caught and ignored, while + all other exceptions are propagated, terminating the search. + + Examples: + + # list all modules python can access + walk_packages() + + # list all submodules of ctypes + walk_packages(ctypes.__path__, ctypes.__name__ + '.') + + Note: Only works for a *note finder: 115f. which defines an + ‘iter_modules()’ method. This interface is non-standard, so + the module also provides implementations for *note + importlib.machinery.FileFinder: 9b6. and *note + zipimport.zipimporter: 3465. + + Changed in version 3.3: Updated to be based directly on *note + importlib: 9b. rather than relying on the package internal PEP + 302(14) import emulation. + + -- Function: pkgutil.get_data (package, resource) + + Get a resource from a package. + + This is a wrapper for the *note loader: 1160. *note get_data: 347a. + API. The `package' argument should be the name of a package, in + standard module format (‘foo.bar’). The `resource' argument should + be in the form of a relative filename, using ‘/’ as the path + separator. The parent directory name ‘..’ is not allowed, and nor + is a rooted name (starting with a ‘/’). + + The function returns a binary string that is the contents of the + specified resource. + + For packages located in the filesystem, which have already been + imported, this is the rough equivalent of: + + d = os.path.dirname(sys.modules[package].__file__) + data = open(os.path.join(d, resource), 'rb').read() + + If the package cannot be located or loaded, or it uses a *note + loader: 1160. which does not support *note get_data: 347a, then + ‘None’ is returned. In particular, the *note loader: 1160. for + *note namespace packages: 1158. does not support *note get_data: + 347a. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pkgutil.py + + (2) https://www.python.org/dev/peps/pep-0302 + + (3) https://www.python.org/dev/peps/pep-0302 + + (4) https://www.python.org/dev/peps/pep-0302 + + (5) https://www.python.org/dev/peps/pep-0302 + + (6) https://www.python.org/dev/peps/pep-0302 + + (7) https://www.python.org/dev/peps/pep-0302 + + (8) https://www.python.org/dev/peps/pep-0451 + + (9) https://www.python.org/dev/peps/pep-0302 + + (10) https://www.python.org/dev/peps/pep-0302 + + (11) https://www.python.org/dev/peps/pep-0451 + + (12) https://www.python.org/dev/peps/pep-0302 + + (13) https://www.python.org/dev/peps/pep-0302 + + (14) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: modulefinder — Find modules used by a script, Next: runpy — Locating and executing Python modules, Prev: pkgutil — Package extension utility, Up: Importing Modules + +5.31.3 ‘modulefinder’ — Find modules used by a script +----------------------------------------------------- + +`Source code:' Lib/modulefinder.py(1) + +__________________________________________________________________ + +This module provides a *note ModuleFinder: 347d. class that can be used +to determine the set of modules imported by a script. ‘modulefinder.py’ +can also be run as a script, giving the filename of a Python script as +its argument, after which a report of the imported modules will be +printed. + + -- Function: modulefinder.AddPackagePath (pkg_name, path) + + Record that the package named `pkg_name' can be found in the + specified `path'. + + -- Function: modulefinder.ReplacePackage (oldname, newname) + + Allows specifying that the module named `oldname' is in fact the + package named `newname'. + + -- Class: modulefinder.ModuleFinder (path=None, debug=0, excludes=[], + replace_paths=[]) + + This class provides *note run_script(): 3480. and *note report(): + 3481. methods to determine the set of modules imported by a script. + `path' can be a list of directories to search for modules; if not + specified, ‘sys.path’ is used. `debug' sets the debugging level; + higher values make the class print debugging messages about what + it’s doing. `excludes' is a list of module names to exclude from + the analysis. `replace_paths' is a list of ‘(oldpath, newpath)’ + tuples that will be replaced in module paths. + + -- Method: report () + + Print a report to standard output that lists the modules + imported by the script and their paths, as well as modules + that are missing or seem to be missing. + + -- Method: run_script (pathname) + + Analyze the contents of the `pathname' file, which must + contain Python code. + + -- Attribute: modules + + A dictionary mapping module names to modules. See *note + Example usage of ModuleFinder: 3483. + +* Menu: + +* Example usage of ModuleFinder:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/modulefinder.py + + +File: python.info, Node: Example usage of ModuleFinder, Up: modulefinder — Find modules used by a script + +5.31.3.1 Example usage of ‘ModuleFinder’ +........................................ + +The script that is going to get analyzed later on (bacon.py): + + import re, itertools + + try: + import baconhameggs + except ImportError: + pass + + try: + import guido.python.ham + except ImportError: + pass + +The script that will output the report of bacon.py: + + from modulefinder import ModuleFinder + + finder = ModuleFinder() + finder.run_script('bacon.py') + + print('Loaded modules:') + for name, mod in finder.modules.items(): + print('%s: ' % name, end='') + print(','.join(list(mod.globalnames.keys())[:3])) + + print('-'*50) + print('Modules not imported:') + print('\n'.join(finder.badmodules.keys())) + +Sample output (may vary depending on the architecture): + + Loaded modules: + _types: + copyreg: _inverted_registry,_slotnames,__all__ + sre_compile: isstring,_sre,_optimize_unicode + _sre: + sre_constants: REPEAT_ONE,makedict,AT_END_LINE + sys: + re: __module__,finditer,_expand + itertools: + __main__: re,itertools,baconhameggs + sre_parse: _PATTERNENDERS,SRE_FLAG_UNICODE + array: + types: __module__,IntType,TypeType + --------------------------------------------------- + Modules not imported: + guido.python.ham + baconhameggs + + +File: python.info, Node: runpy — Locating and executing Python modules, Next: importlib — The implementation of import, Prev: modulefinder — Find modules used by a script, Up: Importing Modules + +5.31.4 ‘runpy’ — Locating and executing Python modules +------------------------------------------------------ + +`Source code:' Lib/runpy.py(1) + +__________________________________________________________________ + +The *note runpy: e2. module is used to locate and run Python modules +without importing them first. Its main use is to implement the *note +-m: 337. command line switch that allows scripts to be located using the +Python module namespace rather than the filesystem. + +Note that this is `not' a sandbox module - all code is executed in the +current process, and any side effects (such as cached imports of other +modules) will remain in place after the functions have returned. + +Furthermore, any functions and classes defined by the executed code are +not guaranteed to work correctly after a *note runpy: e2. function has +returned. If that limitation is not acceptable for a given use case, +*note importlib: 9b. is likely to be a more suitable choice than this +module. + +The *note runpy: e2. module provides two functions: + + -- Function: runpy.run_module (mod_name, init_globals=None, + run_name=None, alter_sys=False) + + Execute the code of the specified module and return the resulting + module globals dictionary. The module’s code is first located + using the standard import mechanism (refer to PEP 302(2) for + details) and then executed in a fresh module namespace. + + The `mod_name' argument should be an absolute module name. If the + module name refers to a package rather than a normal module, then + that package is imported and the ‘__main__’ submodule within that + package is then executed and the resulting module globals + dictionary returned. + + The optional dictionary argument `init_globals' may be used to + pre-populate the module’s globals dictionary before the code is + executed. The supplied dictionary will not be modified. If any of + the special global variables below are defined in the supplied + dictionary, those definitions are overridden by *note run_module(): + fd3. + + The special global variables ‘__name__’, ‘__spec__’, ‘__file__’, + ‘__cached__’, ‘__loader__’ and ‘__package__’ are set in the globals + dictionary before the module code is executed (Note that this is a + minimal set of variables - other variables may be set implicitly as + an interpreter implementation detail). + + ‘__name__’ is set to `run_name' if this optional argument is not + *note None: 157, to ‘mod_name + '.__main__'’ if the named module is + a package and to the `mod_name' argument otherwise. + + ‘__spec__’ will be set appropriately for the `actually' imported + module (that is, ‘__spec__.name’ will always be `mod_name' or + ‘mod_name + '.__main__’, never `run_name'). + + ‘__file__’, ‘__cached__’, ‘__loader__’ and ‘__package__’ are *note + set as normal: 1167. based on the module spec. + + If the argument `alter_sys' is supplied and evaluates to *note + True: 499, then ‘sys.argv[0]’ is updated with the value of + ‘__file__’ and ‘sys.modules[__name__]’ is updated with a temporary + module object for the module being executed. Both ‘sys.argv[0]’ + and ‘sys.modules[__name__]’ are restored to their original values + before the function returns. + + Note that this manipulation of *note sys: fd. is not thread-safe. + Other threads may see the partially initialised module, as well as + the altered list of arguments. It is recommended that the *note + sys: fd. module be left alone when invoking this function from + threaded code. + + See also + ........ + + The *note -m: 337. option offering equivalent functionality from + the command line. + + Changed in version 3.1: Added ability to execute packages by + looking for a ‘__main__’ submodule. + + Changed in version 3.2: Added ‘__cached__’ global variable (see PEP + 3147(3)). + + Changed in version 3.4: Updated to take advantage of the module + spec feature added by PEP 451(4). This allows ‘__cached__’ to be + set correctly for modules run this way, as well as ensuring the + real module name is always accessible as ‘__spec__.name’. + + -- Function: runpy.run_path (file_path, init_globals=None, + run_name=None) + + Execute the code at the named filesystem location and return the + resulting module globals dictionary. As with a script name + supplied to the CPython command line, the supplied path may refer + to a Python source file, a compiled bytecode file or a valid + sys.path entry containing a ‘__main__’ module (e.g. a zipfile + containing a top-level ‘__main__.py’ file). + + For a simple script, the specified code is simply executed in a + fresh module namespace. For a valid sys.path entry (typically a + zipfile or directory), the entry is first added to the beginning of + ‘sys.path’. The function then looks for and executes a *note + __main__: 1. module using the updated path. Note that there is no + special protection against invoking an existing *note __main__: 1. + entry located elsewhere on ‘sys.path’ if there is no such module at + the specified location. + + The optional dictionary argument `init_globals' may be used to + pre-populate the module’s globals dictionary before the code is + executed. The supplied dictionary will not be modified. If any of + the special global variables below are defined in the supplied + dictionary, those definitions are overridden by *note run_path(): + cb1. + + The special global variables ‘__name__’, ‘__spec__’, ‘__file__’, + ‘__cached__’, ‘__loader__’ and ‘__package__’ are set in the globals + dictionary before the module code is executed (Note that this is a + minimal set of variables - other variables may be set implicitly as + an interpreter implementation detail). + + ‘__name__’ is set to `run_name' if this optional argument is not + *note None: 157. and to ‘'<run_path>'’ otherwise. + + If the supplied path directly references a script file (whether as + source or as precompiled byte code), then ‘__file__’ will be set to + the supplied path, and ‘__spec__’, ‘__cached__’, ‘__loader__’ and + ‘__package__’ will all be set to *note None: 157. + + If the supplied path is a reference to a valid sys.path entry, then + ‘__spec__’ will be set appropriately for the imported ‘__main__’ + module (that is, ‘__spec__.name’ will always be ‘__main__’). + ‘__file__’, ‘__cached__’, ‘__loader__’ and ‘__package__’ will be + *note set as normal: 1167. based on the module spec. + + A number of alterations are also made to the *note sys: fd. module. + Firstly, ‘sys.path’ may be altered as described above. + ‘sys.argv[0]’ is updated with the value of ‘file_path’ and + ‘sys.modules[__name__]’ is updated with a temporary module object + for the module being executed. All modifications to items in *note + sys: fd. are reverted before the function returns. + + Note that, unlike *note run_module(): fd3, the alterations made to + *note sys: fd. are not optional in this function as these + adjustments are essential to allowing the execution of sys.path + entries. As the thread-safety limitations still apply, use of this + function in threaded code should be either serialised with the + import lock or delegated to a separate process. + + See also + ........ + + *note Interface options: fd0. for equivalent functionality on the + command line (‘python path/to/script’). + + New in version 3.2. + + Changed in version 3.4: Updated to take advantage of the module + spec feature added by PEP 451(5). This allows ‘__cached__’ to be + set correctly in the case where ‘__main__’ is imported from a valid + sys.path entry rather than being executed directly. + +See also +........ + +PEP 338(6) – Executing modules as scripts + + PEP written and implemented by Nick Coghlan. + +PEP 366(7) – Main module explicit relative imports + + PEP written and implemented by Nick Coghlan. + +PEP 451(8) – A ModuleSpec Type for the Import System + + PEP written and implemented by Eric Snow + +*note Command line and environment: e9f. - CPython command line details + +The *note importlib.import_module(): b13. function + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/runpy.py + + (2) https://www.python.org/dev/peps/pep-0302 + + (3) https://www.python.org/dev/peps/pep-3147 + + (4) https://www.python.org/dev/peps/pep-0451 + + (5) https://www.python.org/dev/peps/pep-0451 + + (6) https://www.python.org/dev/peps/pep-0338 + + (7) https://www.python.org/dev/peps/pep-0366 + + (8) https://www.python.org/dev/peps/pep-0451 + + +File: python.info, Node: importlib — The implementation of import, Next: Using importlib metadata, Prev: runpy — Locating and executing Python modules, Up: Importing Modules + +5.31.5 ‘importlib’ — The implementation of ‘import’ +--------------------------------------------------- + +New in version 3.1. + +`Source code:' Lib/importlib/__init__.py(1) + +__________________________________________________________________ + +* Menu: + +* Introduction: Introduction<12>. +* Functions: Functions<10>. +* importlib.abc – Abstract base classes related to import: importlib abc – Abstract base classes related to import. +* importlib.resources – Resources: importlib resources – Resources. +* importlib.machinery – Importers and path hooks: importlib machinery – Importers and path hooks. +* importlib.util – Utility code for importers: importlib util – Utility code for importers. +* Examples: Examples<29>. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/importlib/__init__.py + + +File: python.info, Node: Introduction<12>, Next: Functions<10>, Up: importlib — The implementation of import + +5.31.5.1 Introduction +..................... + +The purpose of the *note importlib: 9b. package is two-fold. One is to +provide the implementation of the *note import: c1d. statement (and +thus, by extension, the *note __import__(): 610. function) in Python +source code. This provides an implementation of ‘import’ which is +portable to any Python interpreter. This also provides an +implementation which is easier to comprehend than one implemented in a +programming language other than Python. + +Two, the components to implement *note import: c1d. are exposed in this +package, making it easier for users to create their own custom objects +(known generically as an *note importer: 1161.) to participate in the +import process. + +See also +........ + +*note The import statement: c1d. + + The language reference for the *note import: c1d. statement. + +Packages specification(1) + + Original specification of packages. Some semantics have changed + since the writing of this document (e.g. redirecting based on + ‘None’ in *note sys.modules: 1152.). + +The *note __import__(): 9ae. function + + The *note import: c1d. statement is syntactic sugar for this + function. + +PEP 235(2) + + Import on Case-Insensitive Platforms + +PEP 263(3) + + Defining Python Source Code Encodings + +PEP 302(4) + + New Import Hooks + +PEP 328(5) + + Imports: Multi-Line and Absolute/Relative + +PEP 366(6) + + Main module explicit relative imports + +PEP 420(7) + + Implicit namespace packages + +PEP 451(8) + + A ModuleSpec Type for the Import System + +PEP 488(9) + + Elimination of PYO files + +PEP 489(10) + + Multi-phase extension module initialization + +PEP 552(11) + + Deterministic pycs + +PEP 3120(12) + + Using UTF-8 as the Default Source Encoding + +PEP 3147(13) + + PYC Repository Directories + + ---------- Footnotes ---------- + + (1) https://www.python.org/doc/essays/packages/ + + (2) https://www.python.org/dev/peps/pep-0235 + + (3) https://www.python.org/dev/peps/pep-0263 + + (4) https://www.python.org/dev/peps/pep-0302 + + (5) https://www.python.org/dev/peps/pep-0328 + + (6) https://www.python.org/dev/peps/pep-0366 + + (7) https://www.python.org/dev/peps/pep-0420 + + (8) https://www.python.org/dev/peps/pep-0451 + + (9) https://www.python.org/dev/peps/pep-0488 + + (10) https://www.python.org/dev/peps/pep-0489 + + (11) https://www.python.org/dev/peps/pep-0552 + + (12) https://www.python.org/dev/peps/pep-3120 + + (13) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: Functions<10>, Next: importlib abc – Abstract base classes related to import, Prev: Introduction<12>, Up: importlib — The implementation of import + +5.31.5.2 Functions +.................. + + -- Function: importlib.__import__ (name, globals=None, locals=None, + fromlist=(), level=0) + + An implementation of the built-in *note __import__(): 610. + function. + + Note: Programmatic importing of modules should use *note + import_module(): b13. instead of this function. + + -- Function: importlib.import_module (name, package=None) + + Import a module. The `name' argument specifies what module to + import in absolute or relative terms (e.g. either ‘pkg.mod’ or + ‘..mod’). If the name is specified in relative terms, then the + `package' argument must be set to the name of the package which is + to act as the anchor for resolving the package name (e.g. + ‘import_module('..mod', 'pkg.subpkg')’ will import ‘pkg.mod’). + + The *note import_module(): b13. function acts as a simplifying + wrapper around *note importlib.__import__(): 9ae. This means all + semantics of the function are derived from *note + importlib.__import__(): 9ae. The most important difference between + these two functions is that *note import_module(): b13. returns the + specified package or module (e.g. ‘pkg.mod’), while *note + __import__(): 610. returns the top-level package or module (e.g. + ‘pkg’). + + If you are dynamically importing a module that was created since + the interpreter began execution (e.g., created a Python source + file), you may need to call *note invalidate_caches(): 49c. in + order for the new module to be noticed by the import system. + + Changed in version 3.3: Parent packages are automatically imported. + + -- Function: importlib.find_loader (name, path=None) + + Find the loader for a module, optionally within the specified + `path'. If the module is in *note sys.modules: 1152, then + ‘sys.modules[name].__loader__’ is returned (unless the loader would + be ‘None’ or is not set, in which case *note ValueError: 1fb. is + raised). Otherwise a search using *note sys.meta_path: 5e6. is + done. ‘None’ is returned if no loader is found. + + A dotted name does not have its parents implicitly imported as that + requires loading them and that may not be desired. To properly + import a submodule you will need to import all parent packages of + the submodule and use the correct argument to `path'. + + New in version 3.3. + + Changed in version 3.4: If ‘__loader__’ is not set, raise *note + ValueError: 1fb, just like when the attribute is set to ‘None’. + + Deprecated since version 3.4: Use *note importlib.util.find_spec(): + 938. instead. + + -- Function: importlib.invalidate_caches () + + Invalidate the internal caches of finders stored at *note + sys.meta_path: 5e6. If a finder implements ‘invalidate_caches()’ + then it will be called to perform the invalidation. This function + should be called if any modules are created/installed while your + program is running to guarantee all finders will notice the new + module’s existence. + + New in version 3.3. + + -- Function: importlib.reload (module) + + Reload a previously imported `module'. The argument must be a + module object, so it must have been successfully imported before. + This is useful if you have edited the module source file using an + external editor and want to try out the new version without leaving + the Python interpreter. The return value is the module object + (which can be different if re-importing causes a different object + to be placed in *note sys.modules: 1152.). + + When *note reload(): 399. is executed: + + * Python module’s code is recompiled and the module-level code + re-executed, defining a new set of objects which are bound to + names in the module’s dictionary by reusing the *note loader: + 1160. which originally loaded the module. The ‘init’ function + of extension modules is not called a second time. + + * As with all other objects in Python the old objects are only + reclaimed after their reference counts drop to zero. + + * The names in the module namespace are updated to point to any + new or changed objects. + + * Other references to the old objects (such as names external to + the module) are not rebound to refer to the new objects and + must be updated in each namespace where they occur if that is + desired. + + There are a number of other caveats: + + When a module is reloaded, its dictionary (containing the module’s + global variables) is retained. Redefinitions of names will + override the old definitions, so this is generally not a problem. + If the new version of a module does not define a name that was + defined by the old version, the old definition remains. This + feature can be used to the module’s advantage if it maintains a + global table or cache of objects — with a *note try: d72. statement + it can test for the table’s presence and skip its initialization if + desired: + + try: + cache + except NameError: + cache = {} + + It is generally not very useful to reload built-in or dynamically + loaded modules. Reloading *note sys: fd, *note __main__: 1, *note + builtins: 13. and other key modules is not recommended. In many + cases extension modules are not designed to be initialized more + than once, and may fail in arbitrary ways when reloaded. + + If a module imports objects from another module using *note from: + c45. … *note import: c1d. …, calling *note reload(): 399. for the + other module does not redefine the objects imported from it — one + way around this is to re-execute the ‘from’ statement, another is + to use ‘import’ and qualified names (`module.name') instead. + + If a module instantiates instances of a class, reloading the module + that defines the class does not affect the method definitions of + the instances — they continue to use the old class definition. The + same is true for derived classes. + + New in version 3.4. + + Changed in version 3.7: *note ModuleNotFoundError: 39a. is raised + when the module being reloaded lacks a ‘ModuleSpec’. + + +File: python.info, Node: importlib abc – Abstract base classes related to import, Next: importlib resources – Resources, Prev: Functions<10>, Up: importlib — The implementation of import + +5.31.5.3 ‘importlib.abc’ – Abstract base classes related to import +.................................................................. + +`Source code:' Lib/importlib/abc.py(1) + +__________________________________________________________________ + +The *note importlib.abc: 9c. module contains all of the core abstract +base classes used by *note import: c1d. Some subclasses of the core +abstract base classes are also provided to help in implementing the core +ABCs. + +ABC hierarchy: + + object + +-- Finder (deprecated) + | +-- MetaPathFinder + | +-- PathEntryFinder + +-- Loader + +-- ResourceLoader --------+ + +-- InspectLoader | + +-- ExecutionLoader --+ + +-- FileLoader + +-- SourceLoader + + -- Class: importlib.abc.Finder + + An abstract base class representing a *note finder: 115f. + + Deprecated since version 3.3: Use *note MetaPathFinder: 9b3. or + *note PathEntryFinder: 9b4. instead. + + -- Method: abstractmethod find_module (fullname, path=None) + + An abstract method for finding a *note loader: 1160. for the + specified module. Originally specified in PEP 302(2), this + method was meant for use in *note sys.meta_path: 5e6. and in + the path-based import subsystem. + + Changed in version 3.4: Returns ‘None’ when called instead of + raising *note NotImplementedError: 60e. + + -- Class: importlib.abc.MetaPathFinder + + An abstract base class representing a *note meta path finder: 9b1. + For compatibility, this is a subclass of *note Finder: 9b5. + + New in version 3.3. + + -- Method: find_spec (fullname, path, target=None) + + An abstract method for finding a *note spec: 3360. for the + specified module. If this is a top-level import, `path' will + be ‘None’. Otherwise, this is a search for a subpackage or + module and `path' will be the value of *note __path__: f23. + from the parent package. If a spec cannot be found, ‘None’ is + returned. When passed in, ‘target’ is a module object that + the finder may use to make a more educated guess about what + spec to return. *note importlib.util.spec_from_loader(): + 348d. may be useful for implementing concrete + ‘MetaPathFinders’. + + New in version 3.4. + + -- Method: find_module (fullname, path) + + A legacy method for finding a *note loader: 1160. for the + specified module. If this is a top-level import, `path' will + be ‘None’. Otherwise, this is a search for a subpackage or + module and `path' will be the value of *note __path__: f23. + from the parent package. If a loader cannot be found, ‘None’ + is returned. + + If *note find_spec(): 463. is defined, backwards-compatible + functionality is provided. + + Changed in version 3.4: Returns ‘None’ when called instead of + raising *note NotImplementedError: 60e. Can use *note + find_spec(): 463. to provide functionality. + + Deprecated since version 3.4: Use *note find_spec(): 463. + instead. + + -- Method: invalidate_caches () + + An optional method which, when called, should invalidate any + internal cache used by the finder. Used by *note + importlib.invalidate_caches(): 49c. when invalidating the + caches of all finders on *note sys.meta_path: 5e6. + + Changed in version 3.4: Returns ‘None’ when called instead of + ‘NotImplemented’. + + -- Class: importlib.abc.PathEntryFinder + + An abstract base class representing a *note path entry finder: 9b2. + Though it bears some similarities to *note MetaPathFinder: 9b3, + ‘PathEntryFinder’ is meant for use only within the path-based + import subsystem provided by ‘PathFinder’. This ABC is a subclass + of *note Finder: 9b5. for compatibility reasons only. + + New in version 3.3. + + -- Method: find_spec (fullname, target=None) + + An abstract method for finding a *note spec: 3360. for the + specified module. The finder will search for the module only + within the *note path entry: 1175. to which it is assigned. + If a spec cannot be found, ‘None’ is returned. When passed + in, ‘target’ is a module object that the finder may use to + make a more educated guess about what spec to return. *note + importlib.util.spec_from_loader(): 348d. may be useful for + implementing concrete ‘PathEntryFinders’. + + New in version 3.4. + + -- Method: find_loader (fullname) + + A legacy method for finding a *note loader: 1160. for the + specified module. Returns a 2-tuple of ‘(loader, portion)’ + where ‘portion’ is a sequence of file system locations + contributing to part of a namespace package. The loader may + be ‘None’ while specifying ‘portion’ to signify the + contribution of the file system locations to a namespace + package. An empty list can be used for ‘portion’ to signify + the loader is not part of a namespace package. If ‘loader’ is + ‘None’ and ‘portion’ is the empty list then no loader or + location for a namespace package were found (i.e. failure to + find anything for the module). + + If *note find_spec(): 465. is defined then + backwards-compatible functionality is provided. + + Changed in version 3.4: Returns ‘(None, [])’ instead of + raising *note NotImplementedError: 60e. Uses *note + find_spec(): 465. when available to provide functionality. + + Deprecated since version 3.4: Use *note find_spec(): 465. + instead. + + -- Method: find_module (fullname) + + A concrete implementation of *note Finder.find_module(): 348c. + which is equivalent to ‘self.find_loader(fullname)[0]’. + + Deprecated since version 3.4: Use *note find_spec(): 465. + instead. + + -- Method: invalidate_caches () + + An optional method which, when called, should invalidate any + internal cache used by the finder. Used by + ‘PathFinder.invalidate_caches()’ when invalidating the caches + of all cached finders. + + -- Class: importlib.abc.Loader + + An abstract base class for a *note loader: 1160. See PEP 302(3) + for the exact definition for a loader. + + Loaders that wish to support resource reading should implement a + ‘get_resource_reader(fullname)’ method as specified by *note + importlib.abc.ResourceReader: 342. + + Changed in version 3.7: Introduced the optional + ‘get_resource_reader()’ method. + + -- Method: create_module (spec) + + A method that returns the module object to use when importing + a module. This method may return ‘None’, indicating that + default module creation semantics should take place. + + New in version 3.4. + + Changed in version 3.5: Starting in Python 3.6, this method + will not be optional when *note exec_module(): 5e4. is + defined. + + -- Method: exec_module (module) + + An abstract method that executes the module in its own + namespace when a module is imported or reloaded. The module + should already be initialized when ‘exec_module()’ is called. + When this method exists, *note create_module(): 554. must be + defined. + + New in version 3.4. + + Changed in version 3.6: *note create_module(): 554. must also + be defined. + + -- Method: load_module (fullname) + + A legacy method for loading a module. If the module cannot be + loaded, *note ImportError: 334. is raised, otherwise the + loaded module is returned. + + If the requested module already exists in *note sys.modules: + 1152, that module should be used and reloaded. Otherwise the + loader should create a new module and insert it into *note + sys.modules: 1152. before any loading begins, to prevent + recursion from the import. If the loader inserted a module + and the load fails, it must be removed by the loader from + *note sys.modules: 1152.; modules already in *note + sys.modules: 1152. before the loader began execution should be + left alone (see *note importlib.util.module_for_loader(): + 942.). + + The loader should set several attributes on the module. (Note + that some of these attributes can change when a module is + reloaded): + + - + *note __name__: d12. + + The name of the module. + + - + *note __file__: 10d0. + + The path to where the module data is stored (not set + for built-in modules). + + - + *note __cached__: b32. + + The path to where a compiled version of the module + is/should be stored (not set when the attribute + would be inappropriate). + + - + *note __path__: f23. + + A list of strings specifying the search path within + a package. This attribute is not set on modules. + + - + *note __package__: 955. + + The fully-qualified name of the package under which + the module was loaded as a submodule (or the empty + string for top-level modules). For packages, it is + the same as *note __name__: d12. The *note + importlib.util.module_for_loader(): 942. decorator + can handle the details for *note __package__: 955. + + - + *note __loader__: 956. + + The loader used to load the module. The *note + importlib.util.module_for_loader(): 942. decorator + can handle the details for *note __package__: 955. + + When *note exec_module(): 5e4. is available then + backwards-compatible functionality is provided. + + Changed in version 3.4: Raise *note ImportError: 334. when + called instead of *note NotImplementedError: 60e. + Functionality provided when *note exec_module(): 5e4. is + available. + + Deprecated since version 3.4: The recommended API for loading + a module is *note exec_module(): 5e4. (and *note + create_module(): 554.). Loaders should implement it instead + of load_module(). The import machinery takes care of all the + other responsibilities of load_module() when exec_module() is + implemented. + + -- Method: module_repr (module) + + A legacy method which when implemented calculates and returns + the given module’s repr, as a string. The module type’s + default repr() will use the result of this method as + appropriate. + + New in version 3.3. + + Changed in version 3.4: Made optional instead of an + abstractmethod. + + Deprecated since version 3.4: The import machinery now takes + care of this automatically. + + -- Class: importlib.abc.ResourceReader + + An *note abstract base class: b33. to provide the ability to read + `resources'. + + From the perspective of this ABC, a `resource' is a binary artifact + that is shipped within a package. Typically this is something like + a data file that lives next to the ‘__init__.py’ file of the + package. The purpose of this class is to help abstract out the + accessing of such data files so that it does not matter if the + package and its data file(s) are stored in a e.g. zip file versus + on the file system. + + For any of methods of this class, a `resource' argument is expected + to be a *note path-like object: 36a. which represents conceptually + just a file name. This means that no subdirectory paths should be + included in the `resource' argument. This is because the location + of the package the reader is for, acts as the “directory”. Hence + the metaphor for directories and file names is packages and + resources, respectively. This is also why instances of this class + are expected to directly correlate to a specific package (instead + of potentially representing multiple packages or a module). + + Loaders that wish to support resource reading are expected to + provide a method called ‘get_resource_reader(fullname)’ which + returns an object implementing this ABC’s interface. If the module + specified by fullname is not a package, this method should return + *note None: 157. An object compatible with this ABC should only be + returned when the specified module is a package. + + New in version 3.7. + + -- Method: abstractmethod open_resource (resource) + + Returns an opened, *note file-like object: 1928. for binary + reading of the `resource'. + + If the resource cannot be found, *note FileNotFoundError: 7c0. + is raised. + + -- Method: abstractmethod resource_path (resource) + + Returns the file system path to the `resource'. + + If the resource does not concretely exist on the file system, + raise *note FileNotFoundError: 7c0. + + -- Method: abstractmethod is_resource (name) + + Returns ‘True’ if the named `name' is considered a resource. + *note FileNotFoundError: 7c0. is raised if `name' does not + exist. + + -- Method: abstractmethod contents () + + Returns an *note iterable: bac. of strings over the contents + of the package. Do note that it is not required that all + names returned by the iterator be actual resources, e.g. it + is acceptable to return names for which *note is_resource(): + 3492. would be false. + + Allowing non-resource names to be returned is to allow for + situations where how a package and its resources are stored + are known a priori and the non-resource names would be useful. + For instance, returning subdirectory names is allowed so that + when it is known that the package and resources are stored on + the file system then those subdirectory names can be used + directly. + + The abstract method returns an iterable of no items. + + -- Class: importlib.abc.ResourceLoader + + An abstract base class for a *note loader: 1160. which implements + the optional PEP 302(4) protocol for loading arbitrary resources + from the storage back-end. + + Deprecated since version 3.7: This ABC is deprecated in favour of + supporting resource loading through *note + importlib.abc.ResourceReader: 342. + + -- Method: abstractmethod get_data (path) + + An abstract method to return the bytes for the data located at + `path'. Loaders that have a file-like storage back-end that + allows storing arbitrary data can implement this abstract + method to give direct access to the data stored. *note + OSError: 1d3. is to be raised if the `path' cannot be found. + The `path' is expected to be constructed using a module’s + *note __file__: 10d0. attribute or an item from a package’s + *note __path__: f23. + + Changed in version 3.4: Raises *note OSError: 1d3. instead of + *note NotImplementedError: 60e. + + -- Class: importlib.abc.InspectLoader + + An abstract base class for a *note loader: 1160. which implements + the optional PEP 302(5) protocol for loaders that inspect modules. + + -- Method: get_code (fullname) + + Return the code object for a module, or ‘None’ if the module + does not have a code object (as would be the case, for + example, for a built-in module). Raise an *note ImportError: + 334. if loader cannot find the requested module. + + Note: While the method has a default implementation, it + is suggested that it be overridden if possible for + performance. + + Changed in version 3.4: No longer abstract and a concrete + implementation is provided. + + -- Method: abstractmethod get_source (fullname) + + An abstract method to return the source of a module. It is + returned as a text string using *note universal newlines: 5f4, + translating all recognized line separators into ‘'\n'’ + characters. Returns ‘None’ if no source is available (e.g. a + built-in module). Raises *note ImportError: 334. if the + loader cannot find the module specified. + + Changed in version 3.4: Raises *note ImportError: 334. instead + of *note NotImplementedError: 60e. + + -- Method: is_package (fullname) + + An abstract method to return a true value if the module is a + package, a false value otherwise. *note ImportError: 334. is + raised if the *note loader: 1160. cannot find the module. + + Changed in version 3.4: Raises *note ImportError: 334. instead + of *note NotImplementedError: 60e. + + -- Method: static source_to_code (data, path='<string>') + + Create a code object from Python source. + + The `data' argument can be whatever the *note compile(): 1b4. + function supports (i.e. string or bytes). The `path' + argument should be the “path” to where the source code + originated from, which can be an abstract concept (e.g. + location in a zip file). + + With the subsequent code object one can execute it in a module + by running ‘exec(code, module.__dict__)’. + + New in version 3.4. + + Changed in version 3.5: Made the method static. + + -- Method: exec_module (module) + + Implementation of *note Loader.exec_module(): 5e4. + + New in version 3.4. + + -- Method: load_module (fullname) + + Implementation of *note Loader.load_module(): 5e3. + + Deprecated since version 3.4: use *note exec_module(): 93f. + instead. + + -- Class: importlib.abc.ExecutionLoader + + An abstract base class which inherits from *note InspectLoader: + 860. that, when implemented, helps a module to be executed as a + script. The ABC represents an optional PEP 302(6) protocol. + + -- Method: abstractmethod get_filename (fullname) + + An abstract method that is to return the value of *note + __file__: 10d0. for the specified module. If no path is + available, *note ImportError: 334. is raised. + + If source code is available, then the method should return the + path to the source file, regardless of whether a bytecode was + used to load the module. + + Changed in version 3.4: Raises *note ImportError: 334. instead + of *note NotImplementedError: 60e. + + -- Class: importlib.abc.FileLoader (fullname, path) + + An abstract base class which inherits from *note ResourceLoader: + 466. and *note ExecutionLoader: 3495, providing concrete + implementations of *note ResourceLoader.get_data(): 347a. and *note + ExecutionLoader.get_filename(): 3496. + + The `fullname' argument is a fully resolved name of the module the + loader is to handle. The `path' argument is the path to the file + for the module. + + New in version 3.3. + + -- Attribute: name + + The name of the module the loader can handle. + + -- Attribute: path + + Path to the file of the module. + + -- Method: load_module (fullname) + + Calls super’s ‘load_module()’. + + Deprecated since version 3.4: Use *note Loader.exec_module(): + 5e4. instead. + + -- Method: abstractmethod get_filename (fullname) + + Returns *note path: 3498. + + -- Method: abstractmethod get_data (path) + + Reads `path' as a binary file and returns the bytes from it. + + -- Class: importlib.abc.SourceLoader + + An abstract base class for implementing source (and optionally + bytecode) file loading. The class inherits from both *note + ResourceLoader: 466. and *note ExecutionLoader: 3495, requiring the + implementation of: + + * *note ResourceLoader.get_data(): 347a. + + * + *note ExecutionLoader.get_filename(): 3496. + + Should only return the path to the source file; + sourceless loading is not supported. + + The abstract methods defined by this class are to add optional + bytecode file support. Not implementing these optional methods (or + causing them to raise *note NotImplementedError: 60e.) causes the + loader to only work with source code. Implementing the methods + allows the loader to work with source `and' bytecode files; it does + not allow for `sourceless' loading where only bytecode is provided. + Bytecode files are an optimization to speed up loading by removing + the parsing step of Python’s compiler, and so no bytecode-specific + API is exposed. + + -- Method: path_stats (path) + + Optional abstract method which returns a *note dict: 1b8. + containing metadata about the specified path. Supported + dictionary keys are: + + - ‘'mtime'’ (mandatory): an integer or floating-point + number representing the modification time of the source + code; + + - ‘'size'’ (optional): the size in bytes of the source + code. + + Any other keys in the dictionary are ignored, to allow for + future extensions. If the path cannot be handled, *note + OSError: 1d3. is raised. + + New in version 3.3. + + Changed in version 3.4: Raise *note OSError: 1d3. instead of + *note NotImplementedError: 60e. + + -- Method: path_mtime (path) + + Optional abstract method which returns the modification time + for the specified path. + + Deprecated since version 3.3: This method is deprecated in + favour of *note path_stats(): aec. You don’t have to + implement it, but it is still available for compatibility + purposes. Raise *note OSError: 1d3. if the path cannot be + handled. + + Changed in version 3.4: Raise *note OSError: 1d3. instead of + *note NotImplementedError: 60e. + + -- Method: set_data (path, data) + + Optional abstract method which writes the specified bytes to a + file path. Any intermediate directories which do not exist + are to be created automatically. + + When writing to the path fails because the path is read-only + (*note errno.EACCES: 1f28./*note PermissionError: 5ff.), do + not propagate the exception. + + Changed in version 3.4: No longer raises *note + NotImplementedError: 60e. when called. + + -- Method: get_code (fullname) + + Concrete implementation of *note InspectLoader.get_code(): + 861. + + -- Method: exec_module (module) + + Concrete implementation of *note Loader.exec_module(): 5e4. + + New in version 3.4. + + -- Method: load_module (fullname) + + Concrete implementation of *note Loader.load_module(): 5e3. + + Deprecated since version 3.4: Use *note exec_module(): 940. + instead. + + -- Method: get_source (fullname) + + Concrete implementation of *note InspectLoader.get_source(): + 865. + + -- Method: is_package (fullname) + + Concrete implementation of *note InspectLoader.is_package(): + 3494. A module is determined to be a package if its file path + (as provided by *note ExecutionLoader.get_filename(): 3496.) + is a file named ‘__init__’ when the file extension is removed + `and' the module name itself does not end in ‘__init__’. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/importlib/abc.py + + (2) https://www.python.org/dev/peps/pep-0302 + + (3) https://www.python.org/dev/peps/pep-0302 + + (4) https://www.python.org/dev/peps/pep-0302 + + (5) https://www.python.org/dev/peps/pep-0302 + + (6) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: importlib resources – Resources, Next: importlib machinery – Importers and path hooks, Prev: importlib abc – Abstract base classes related to import, Up: importlib — The implementation of import + +5.31.5.4 ‘importlib.resources’ – Resources +.......................................... + +`Source code:' Lib/importlib/resources.py(1) + +__________________________________________________________________ + +New in version 3.7. + +This module leverages Python’s import system to provide access to +`resources' within `packages'. If you can import a package, you can +access resources within that package. Resources can be opened or read, +in either binary or text mode. + +Resources are roughly akin to files inside directories, though it’s +important to keep in mind that this is just a metaphor. Resources and +packages `do not' have to exist as physical files and directories on the +file system. + + Note: This module provides functionality similar to + pkg_resources(2) Basic Resource Access(3) without the performance + overhead of that package. This makes reading resources included in + packages easier, with more stable and consistent semantics. + + The standalone backport of this module provides more information on + using importlib.resources(4) and migrating from pkg_resources to + importlib.resources(5). + +Loaders that wish to support resource reading should implement a +‘get_resource_reader(fullname)’ method as specified by *note +importlib.abc.ResourceReader: 342. + +The following types are defined. + + -- Data: importlib.resources.Package + + The ‘Package’ type is defined as ‘Union[str, ModuleType]’. This + means that where the function describes accepting a ‘Package’, you + can pass in either a string or a module. Module objects must have + a resolvable ‘__spec__.submodule_search_locations’ that is not + ‘None’. + + -- Data: importlib.resources.Resource + + This type describes the resource names passed into the various + functions in this package. This is defined as ‘Union[str, + os.PathLike]’. + +The following functions are available. + + -- Function: importlib.resources.open_binary (package, resource) + + Open for binary reading the `resource' within `package'. + + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. `resource' is the name of the resource to + open within `package'; it may not contain path separators and it + may not have sub-resources (i.e. it cannot be a directory). This + function returns a ‘typing.BinaryIO’ instance, a binary I/O stream + open for reading. + + -- Function: importlib.resources.open_text (package, resource, + encoding='utf-8', errors='strict') + + Open for text reading the `resource' within `package'. By default, + the resource is opened for reading as UTF-8. + + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. `resource' is the name of the resource to + open within `package'; it may not contain path separators and it + may not have sub-resources (i.e. it cannot be a directory). + `encoding' and `errors' have the same meaning as with built-in + *note open(): 4f0. + + This function returns a ‘typing.TextIO’ instance, a text I/O stream + open for reading. + + -- Function: importlib.resources.read_binary (package, resource) + + Read and return the contents of the `resource' within `package' as + ‘bytes’. + + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. `resource' is the name of the resource to + open within `package'; it may not contain path separators and it + may not have sub-resources (i.e. it cannot be a directory). This + function returns the contents of the resource as *note bytes: 331. + + -- Function: importlib.resources.read_text (package, resource, + encoding='utf-8', errors='strict') + + Read and return the contents of `resource' within `package' as a + ‘str’. By default, the contents are read as strict UTF-8. + + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. `resource' is the name of the resource to + open within `package'; it may not contain path separators and it + may not have sub-resources (i.e. it cannot be a directory). + `encoding' and `errors' have the same meaning as with built-in + *note open(): 4f0. This function returns the contents of the + resource as *note str: 330. + + -- Function: importlib.resources.path (package, resource) + + Return the path to the `resource' as an actual file system path. + This function returns a context manager for use in a *note with: + 6e9. statement. The context manager provides a *note pathlib.Path: + 201. object. + + Exiting the context manager cleans up any temporary file created + when the resource needs to be extracted from e.g. a zip file. + + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. `resource' is the name of the resource to + open within `package'; it may not contain path separators and it + may not have sub-resources (i.e. it cannot be a directory). + + -- Function: importlib.resources.is_resource (package, name) + + Return ‘True’ if there is a resource named `name' in the package, + otherwise ‘False’. Remember that directories are `not' resources! + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. + + -- Function: importlib.resources.contents (package) + + Return an iterable over the named items within the package. The + iterable returns *note str: 330. resources (e.g. files) and + non-resources (e.g. directories). The iterable does not recurse + into subdirectories. + + `package' is either a name or a module object which conforms to the + ‘Package’ requirements. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/importlib/resources.py + + (2) https://setuptools.readthedocs.io/en/latest/pkg_resources.html + + (3) +http://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access + + (4) http://importlib-resources.readthedocs.io/en/latest/using.html + + (5) +http://importlib-resources.readthedocs.io/en/latest/migration.html + + +File: python.info, Node: importlib machinery – Importers and path hooks, Next: importlib util – Utility code for importers, Prev: importlib resources – Resources, Up: importlib — The implementation of import + +5.31.5.5 ‘importlib.machinery’ – Importers and path hooks +......................................................... + +`Source code:' Lib/importlib/machinery.py(1) + +__________________________________________________________________ + +This module contains the various objects that help *note import: c1d. +find and load modules. + + -- Attribute: importlib.machinery.SOURCE_SUFFIXES + + A list of strings representing the recognized file suffixes for + source modules. + + New in version 3.3. + + -- Attribute: importlib.machinery.DEBUG_BYTECODE_SUFFIXES + + A list of strings representing the file suffixes for non-optimized + bytecode modules. + + New in version 3.3. + + Deprecated since version 3.5: Use *note BYTECODE_SUFFIXES: 34ac. + instead. + + -- Attribute: importlib.machinery.OPTIMIZED_BYTECODE_SUFFIXES + + A list of strings representing the file suffixes for optimized + bytecode modules. + + New in version 3.3. + + Deprecated since version 3.5: Use *note BYTECODE_SUFFIXES: 34ac. + instead. + + -- Attribute: importlib.machinery.BYTECODE_SUFFIXES + + A list of strings representing the recognized file suffixes for + bytecode modules (including the leading dot). + + New in version 3.3. + + Changed in version 3.5: The value is no longer dependent on + ‘__debug__’. + + -- Attribute: importlib.machinery.EXTENSION_SUFFIXES + + A list of strings representing the recognized file suffixes for + extension modules. + + New in version 3.3. + + -- Function: importlib.machinery.all_suffixes () + + Returns a combined list of strings representing all file suffixes + for modules recognized by the standard import machinery. This is a + helper for code which simply needs to know if a filesystem path + potentially refers to a module without needing any details on the + kind of module (for example, *note inspect.getmodulename(): 5f2.). + + New in version 3.3. + + -- Class: importlib.machinery.BuiltinImporter + + An *note importer: 1161. for built-in modules. All known built-in + modules are listed in *note sys.builtin_module_names: 334a. This + class implements the *note importlib.abc.MetaPathFinder: 9b3. and + *note importlib.abc.InspectLoader: 860. ABCs. + + Only class methods are defined by this class to alleviate the need + for instantiation. + + Changed in version 3.5: As part of PEP 489(2), the builtin importer + now implements ‘Loader.create_module()’ and ‘Loader.exec_module()’ + + -- Class: importlib.machinery.FrozenImporter + + An *note importer: 1161. for frozen modules. This class implements + the *note importlib.abc.MetaPathFinder: 9b3. and *note + importlib.abc.InspectLoader: 860. ABCs. + + Only class methods are defined by this class to alleviate the need + for instantiation. + + Changed in version 3.4: Gained ‘create_module()’ and + ‘exec_module()’ methods. + + -- Class: importlib.machinery.WindowsRegistryFinder + + *note Finder: 115f. for modules declared in the Windows registry. + This class implements the *note importlib.abc.MetaPathFinder: 9b3. + ABC. + + Only class methods are defined by this class to alleviate the need + for instantiation. + + New in version 3.3. + + Deprecated since version 3.6: Use *note site: eb. configuration + instead. Future versions of Python may not enable this finder by + default. + + -- Class: importlib.machinery.PathFinder + + A *note Finder: 115f. for *note sys.path: 488. and package + ‘__path__’ attributes. This class implements the *note + importlib.abc.MetaPathFinder: 9b3. ABC. + + Only class methods are defined by this class to alleviate the need + for instantiation. + + -- Method: classmethod find_spec (fullname, path=None, + target=None) + + Class method that attempts to find a *note spec: 3360. for the + module specified by `fullname' on *note sys.path: 488. or, if + defined, on `path'. For each path entry that is searched, + *note sys.path_importer_cache: 49d. is checked. If a + non-false object is found then it is used as the *note path + entry finder: 9b2. to look for the module being searched for. + If no entry is found in *note sys.path_importer_cache: 49d, + then *note sys.path_hooks: 95c. is searched for a finder for + the path entry and, if found, is stored in *note + sys.path_importer_cache: 49d. along with being queried about + the module. If no finder is ever found then ‘None’ is both + stored in the cache and returned. + + New in version 3.4. + + Changed in version 3.5: If the current working directory – + represented by an empty string – is no longer valid then + ‘None’ is returned but no value is cached in *note + sys.path_importer_cache: 49d. + + -- Method: classmethod find_module (fullname, path=None) + + A legacy wrapper around *note find_spec(): 93a. + + Deprecated since version 3.4: Use *note find_spec(): 93a. + instead. + + -- Method: classmethod invalidate_caches () + + Calls *note importlib.abc.PathEntryFinder.invalidate_caches(): + 348f. on all finders stored in *note sys.path_importer_cache: + 49d. that define the method. Otherwise entries in *note + sys.path_importer_cache: 49d. set to ‘None’ are deleted. + + Changed in version 3.7: Entries of ‘None’ in *note + sys.path_importer_cache: 49d. are deleted. + + Changed in version 3.4: Calls objects in *note sys.path_hooks: 95c. + with the current working directory for ‘''’ (i.e. the empty + string). + + -- Class: importlib.machinery.FileFinder (path, *loader_details) + + A concrete implementation of *note importlib.abc.PathEntryFinder: + 9b4. which caches results from the file system. + + The `path' argument is the directory for which the finder is in + charge of searching. + + The `loader_details' argument is a variable number of 2-item tuples + each containing a loader and a sequence of file suffixes the loader + recognizes. The loaders are expected to be callables which accept + two arguments of the module’s name and the path to the file found. + + The finder will cache the directory contents as necessary, making + stat calls for each module search to verify the cache is not + outdated. Because cache staleness relies upon the granularity of + the operating system’s state information of the file system, there + is a potential race condition of searching for a module, creating a + new file, and then searching for the module the new file + represents. If the operations happen fast enough to fit within the + granularity of stat calls, then the module search will fail. To + prevent this from happening, when you create a module dynamically, + make sure to call *note importlib.invalidate_caches(): 49c. + + New in version 3.3. + + -- Attribute: path + + The path the finder will search in. + + -- Method: find_spec (fullname, target=None) + + Attempt to find the spec to handle `fullname' within *note + path: 34af. + + New in version 3.4. + + -- Method: find_loader (fullname) + + Attempt to find the loader to handle `fullname' within *note + path: 34af. + + -- Method: invalidate_caches () + + Clear out the internal cache. + + -- Method: classmethod path_hook (*loader_details) + + A class method which returns a closure for use on *note + sys.path_hooks: 95c. An instance of *note FileFinder: 9b6. is + returned by the closure using the path argument given to the + closure directly and `loader_details' indirectly. + + If the argument to the closure is not an existing directory, + *note ImportError: 334. is raised. + + -- Class: importlib.machinery.SourceFileLoader (fullname, path) + + A concrete implementation of *note importlib.abc.SourceLoader: + 349b. by subclassing *note importlib.abc.FileLoader: 9b7. and + providing some concrete implementations of other methods. + + New in version 3.3. + + -- Attribute: name + + The name of the module that this loader will handle. + + -- Attribute: path + + The path to the source file. + + -- Method: is_package (fullname) + + Return ‘True’ if *note path: 34b4. appears to be for a + package. + + -- Method: path_stats (path) + + Concrete implementation of *note + importlib.abc.SourceLoader.path_stats(): aec. + + -- Method: set_data (path, data) + + Concrete implementation of *note + importlib.abc.SourceLoader.set_data(): 349c. + + -- Method: load_module (name=None) + + Concrete implementation of *note + importlib.abc.Loader.load_module(): 5e3. where specifying the + name of the module to load is optional. + + Deprecated since version 3.6: Use *note + importlib.abc.Loader.exec_module(): 5e4. instead. + + -- Class: importlib.machinery.SourcelessFileLoader (fullname, path) + + A concrete implementation of *note importlib.abc.FileLoader: 9b7. + which can import bytecode files (i.e. no source code files exist). + + Please note that direct use of bytecode files (and thus not source + code files) inhibits your modules from being usable by all Python + implementations or new versions of Python which change the bytecode + format. + + New in version 3.3. + + -- Attribute: name + + The name of the module the loader will handle. + + -- Attribute: path + + The path to the bytecode file. + + -- Method: is_package (fullname) + + Determines if the module is a package based on *note path: + 34b9. + + -- Method: get_code (fullname) + + Returns the code object for *note name: 34b8. created from + *note path: 34b9. + + -- Method: get_source (fullname) + + Returns ‘None’ as bytecode files have no source when this + loader is used. + + -- Method: load_module (name=None) + + Concrete implementation of *note + importlib.abc.Loader.load_module(): 5e3. where specifying the name + of the module to load is optional. + + Deprecated since version 3.6: Use *note + importlib.abc.Loader.exec_module(): 5e4. instead. + + -- Class: importlib.machinery.ExtensionFileLoader (fullname, path) + + A concrete implementation of *note importlib.abc.ExecutionLoader: + 3495. for extension modules. + + The `fullname' argument specifies the name of the module the loader + is to support. The `path' argument is the path to the extension + module’s file. + + New in version 3.3. + + -- Attribute: name + + Name of the module the loader supports. + + -- Attribute: path + + Path to the extension module. + + -- Method: create_module (spec) + + Creates the module object from the given specification in + accordance with PEP 489(3). + + New in version 3.5. + + -- Method: exec_module (module) + + Initializes the given module object in accordance with PEP + 489(4). + + New in version 3.5. + + -- Method: is_package (fullname) + + Returns ‘True’ if the file path points to a package’s + ‘__init__’ module based on *note EXTENSION_SUFFIXES: 34ae. + + -- Method: get_code (fullname) + + Returns ‘None’ as extension modules lack a code object. + + -- Method: get_source (fullname) + + Returns ‘None’ as extension modules do not have source code. + + -- Method: get_filename (fullname) + + Returns *note path: 34be. + + New in version 3.4. + + -- Class: importlib.machinery.ModuleSpec (name, loader, *, origin=None, + loader_state=None, is_package=None) + + A specification for a module’s import-system-related state. This + is typically exposed as the module’s ‘__spec__’ attribute. In the + descriptions below, the names in parentheses give the corresponding + attribute available directly on the module object. E.g. + ‘module.__spec__.origin == module.__file__’. Note however that + while the `values' are usually equivalent, they can differ since + there is no synchronization between the two objects. Thus it is + possible to update the module’s ‘__path__’ at runtime, and this + will not be automatically reflected in + ‘__spec__.submodule_search_locations’. + + New in version 3.4. + + -- Attribute: name + + (‘__name__’) + + A string for the fully-qualified name of the module. + + -- Attribute: loader + + (‘__loader__’) + + The *note Loader: 1160. that should be used when loading the + module. *note Finders: 115f. should always set this. + + -- Attribute: origin + + (‘__file__’) + + Name of the place from which the module is loaded, e.g. “builtin” + for built-in modules and the filename for modules loaded from + source. Normally “origin” should be set, but it may be ‘None’ (the + default) which indicates it is unspecified (e.g. for namespace + packages). + + -- Attribute: submodule_search_locations + + (‘__path__’) + + List of strings for where to find submodules, if a package (‘None’ + otherwise). + + -- Attribute: loader_state + + Container of extra module-specific data for use during loading (or + ‘None’). + + -- Attribute: cached + + (‘__cached__’) + + String for where the compiled module should be stored (or ‘None’). + + -- Attribute: parent + + (‘__package__’) + + (Read-only) The fully-qualified name of the package under which the + module should be loaded as a submodule (or the empty string for + top-level modules). For packages, it is the same as *note + __name__: d12. + + -- Attribute: has_location + + Boolean indicating whether or not the module’s “origin” attribute + refers to a loadable location. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/importlib/machinery.py + + (2) https://www.python.org/dev/peps/pep-0489 + + (3) https://www.python.org/dev/peps/pep-0489 + + (4) https://www.python.org/dev/peps/pep-0489 + + +File: python.info, Node: importlib util – Utility code for importers, Next: Examples<29>, Prev: importlib machinery – Importers and path hooks, Up: importlib — The implementation of import + +5.31.5.6 ‘importlib.util’ – Utility code for importers +...................................................... + +`Source code:' Lib/importlib/util.py(1) + +__________________________________________________________________ + +This module contains the various objects that help in the construction +of an *note importer: 1161. + + -- Attribute: importlib.util.MAGIC_NUMBER + + The bytes which represent the bytecode version number. If you need + help with loading/writing bytecode then consider *note + importlib.abc.SourceLoader: 349b. + + New in version 3.4. + + -- Function: importlib.util.cache_from_source (path, + debug_override=None, *, optimization=None) + + Return the PEP 3147(2)/ PEP 488(3) path to the byte-compiled file + associated with the source `path'. For example, if `path' is + ‘/foo/bar/baz.py’ the return value would be + ‘/foo/bar/__pycache__/baz.cpython-32.pyc’ for Python 3.2. The + ‘cpython-32’ string comes from the current magic tag (see + ‘get_tag()’; if ‘sys.implementation.cache_tag’ is not defined then + *note NotImplementedError: 60e. will be raised). + + The `optimization' parameter is used to specify the optimization + level of the bytecode file. An empty string represents no + optimization, so ‘/foo/bar/baz.py’ with an `optimization' of ‘''’ + will result in a bytecode path of + ‘/foo/bar/__pycache__/baz.cpython-32.pyc’. ‘None’ causes the + interpreter’s optimization level to be used. Any other value’s + string representation is used, so ‘/foo/bar/baz.py’ with an + `optimization' of ‘2’ will lead to the bytecode path of + ‘/foo/bar/__pycache__/baz.cpython-32.opt-2.pyc’. The string + representation of `optimization' can only be alphanumeric, else + *note ValueError: 1fb. is raised. + + The `debug_override' parameter is deprecated and can be used to + override the system’s value for ‘__debug__’. A ‘True’ value is the + equivalent of setting `optimization' to the empty string. A + ‘False’ value is the same as setting `optimization' to ‘1’. If + both `debug_override' an `optimization' are not ‘None’ then *note + TypeError: 192. is raised. + + New in version 3.4. + + Changed in version 3.5: The `optimization' parameter was added and + the `debug_override' parameter was deprecated. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + + -- Function: importlib.util.source_from_cache (path) + + Given the `path' to a PEP 3147(4) file name, return the associated + source code file path. For example, if `path' is + ‘/foo/bar/__pycache__/baz.cpython-32.pyc’ the returned path would + be ‘/foo/bar/baz.py’. `path' need not exist, however if it does + not conform to PEP 3147(5) or PEP 488(6) format, a *note + ValueError: 1fb. is raised. If ‘sys.implementation.cache_tag’ is + not defined, *note NotImplementedError: 60e. is raised. + + New in version 3.4. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + + -- Function: importlib.util.decode_source (source_bytes) + + Decode the given bytes representing source code and return it as a + string with universal newlines (as required by *note + importlib.abc.InspectLoader.get_source(): 865.). + + New in version 3.4. + + -- Function: importlib.util.resolve_name (name, package) + + Resolve a relative module name to an absolute one. + + If `name' has no leading dots, then `name' is simply returned. + This allows for usage such as ‘importlib.util.resolve_name('sys', + __spec__.parent)’ without doing a check to see if the `package' + argument is needed. + + *note ValueError: 1fb. is raised if `name' is a relative module + name but package is a false value (e.g. ‘None’ or the empty + string). *note ValueError: 1fb. is also raised a relative name + would escape its containing package (e.g. requesting ‘..bacon’ + from within the ‘spam’ package). + + New in version 3.3. + + -- Function: importlib.util.find_spec (name, package=None) + + Find the *note spec: 3360. for a module, optionally relative to the + specified `package' name. If the module is in *note sys.modules: + 1152, then ‘sys.modules[name].__spec__’ is returned (unless the + spec would be ‘None’ or is not set, in which case *note ValueError: + 1fb. is raised). Otherwise a search using *note sys.meta_path: + 5e6. is done. ‘None’ is returned if no spec is found. + + If `name' is for a submodule (contains a dot), the parent module is + automatically imported. + + `name' and `package' work the same as for ‘import_module()’. + + New in version 3.4. + + Changed in version 3.7: Raises *note ModuleNotFoundError: 39a. + instead of *note AttributeError: 39b. if `package' is in fact not a + package (i.e. lacks a *note __path__: f23. attribute). + + -- Function: importlib.util.module_from_spec (spec) + + Create a new module based on `spec' and *note + spec.loader.create_module: 554. + + If *note spec.loader.create_module: 554. does not return ‘None’, + then any pre-existing attributes will not be reset. Also, no *note + AttributeError: 39b. will be raised if triggered while accessing + `spec' or setting an attribute on the module. + + This function is preferred over using *note types.ModuleType: 6f1. + to create a new module as `spec' is used to set as many + import-controlled attributes on the module as possible. + + New in version 3.5. + + -- Function: @importlib.util.module_for_loader + + A *note decorator: 283. for *note + importlib.abc.Loader.load_module(): 5e3. to handle selecting the + proper module object to load with. The decorated method is + expected to have a call signature taking two positional arguments + (e.g. ‘load_module(self, module)’) for which the second argument + will be the module `object' to be used by the loader. Note that + the decorator will not work on static methods because of the + assumption of two arguments. + + The decorated method will take in the `name' of the module to be + loaded as expected for a *note loader: 1160. If the module is not + found in *note sys.modules: 1152. then a new one is constructed. + Regardless of where the module came from, *note __loader__: 956. + set to `self' and *note __package__: 955. is set based on what + *note importlib.abc.InspectLoader.is_package(): 3494. returns (if + available). These attributes are set unconditionally to support + reloading. + + If an exception is raised by the decorated method and a module was + added to *note sys.modules: 1152, then the module will be removed + to prevent a partially initialized module from being in left in + *note sys.modules: 1152. If the module was already in *note + sys.modules: 1152. then it is left alone. + + Changed in version 3.3: *note __loader__: 956. and *note + __package__: 955. are automatically set (when possible). + + Changed in version 3.4: Set *note __name__: d12, *note __loader__: + 956. *note __package__: 955. unconditionally to support reloading. + + Deprecated since version 3.4: The import machinery now directly + performs all the functionality provided by this function. + + -- Function: @importlib.util.set_loader + + A *note decorator: 283. for *note + importlib.abc.Loader.load_module(): 5e3. to set the *note + __loader__: 956. attribute on the returned module. If the + attribute is already set the decorator does nothing. It is assumed + that the first positional argument to the wrapped method (i.e. + ‘self’) is what *note __loader__: 956. should be set to. + + Changed in version 3.4: Set ‘__loader__’ if set to ‘None’, as if + the attribute does not exist. + + Deprecated since version 3.4: The import machinery takes care of + this automatically. + + -- Function: @importlib.util.set_package + + A *note decorator: 283. for *note + importlib.abc.Loader.load_module(): 5e3. to set the *note + __package__: 955. attribute on the returned module. If *note + __package__: 955. is set and has a value other than ‘None’ it will + not be changed. + + Deprecated since version 3.4: The import machinery takes care of + this automatically. + + -- Function: importlib.util.spec_from_loader (name, loader, *, + origin=None, is_package=None) + + A factory function for creating a ‘ModuleSpec’ instance based on a + loader. The parameters have the same meaning as they do for + ModuleSpec. The function uses available *note loader: 1160. APIs, + such as ‘InspectLoader.is_package()’, to fill in any missing + information on the spec. + + New in version 3.4. + + -- Function: importlib.util.spec_from_file_location (name, location, *, + loader=None, submodule_search_locations=None) + + A factory function for creating a ‘ModuleSpec’ instance based on + the path to a file. Missing information will be filled in on the + spec by making use of loader APIs and by the implication that the + module will be file-based. + + New in version 3.4. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + + -- Function: importlib.util.source_hash (source_bytes) + + Return the hash of `source_bytes' as bytes. A hash-based ‘.pyc’ + file embeds the *note source_hash(): 34cb. of the corresponding + source file’s contents in its header. + + New in version 3.7. + + -- Class: importlib.util.LazyLoader (loader) + + A class which postpones the execution of the loader of a module + until the module has an attribute accessed. + + This class `only' works with loaders that define *note + exec_module(): 5e4. as control over what module type is used for + the module is required. For those same reasons, the loader’s *note + create_module(): 554. method must return ‘None’ or a type for which + its ‘__class__’ attribute can be mutated along with not using *note + slots: 34cc. Finally, modules which substitute the object placed + into *note sys.modules: 1152. will not work as there is no way to + properly replace the module references throughout the interpreter + safely; *note ValueError: 1fb. is raised if such a substitution is + detected. + + Note: For projects where startup time is critical, this class + allows for potentially minimizing the cost of loading a module + if it is never used. For projects where startup time is not + essential then use of this class is `heavily' discouraged due + to error messages created during loading being postponed and + thus occurring out of context. + + New in version 3.5. + + Changed in version 3.6: Began calling *note create_module(): 554, + removing the compatibility warning for *note + importlib.machinery.BuiltinImporter: 555. and *note + importlib.machinery.ExtensionFileLoader: 556. + + -- Method: classmethod factory (loader) + + A static method which returns a callable that creates a lazy + loader. This is meant to be used in situations where the + loader is passed by class instead of by instance. + + suffixes = importlib.machinery.SOURCE_SUFFIXES + loader = importlib.machinery.SourceFileLoader + lazy_loader = importlib.util.LazyLoader.factory(loader) + finder = importlib.machinery.FileFinder(path, (lazy_loader, suffixes)) + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/importlib/util.py + + (2) https://www.python.org/dev/peps/pep-3147 + + (3) https://www.python.org/dev/peps/pep-0488 + + (4) https://www.python.org/dev/peps/pep-3147 + + (5) https://www.python.org/dev/peps/pep-3147 + + (6) https://www.python.org/dev/peps/pep-0488 + + +File: python.info, Node: Examples<29>, Prev: importlib util – Utility code for importers, Up: importlib — The implementation of import + +5.31.5.7 Examples +................. + +* Menu: + +* Importing programmatically:: +* Checking if a module can be imported:: +* Importing a source file directly:: +* Setting up an importer:: +* Approximating importlib.import_module(): Approximating importlib import_module. + + +File: python.info, Node: Importing programmatically, Next: Checking if a module can be imported, Up: Examples<29> + +5.31.5.8 Importing programmatically +................................... + +To programmatically import a module, use *note +importlib.import_module(): b13. + + import importlib + + itertools = importlib.import_module('itertools') + + +File: python.info, Node: Checking if a module can be imported, Next: Importing a source file directly, Prev: Importing programmatically, Up: Examples<29> + +5.31.5.9 Checking if a module can be imported +............................................. + +If you need to find out if a module can be imported without actually +doing the import, then you should use *note importlib.util.find_spec(): +938. + + import importlib.util + import sys + + # For illustrative purposes. + name = 'itertools' + + if name in sys.modules: + print(f"{name!r} already in sys.modules") + elif (spec := importlib.util.find_spec(name)) is not None: + # If you chose to perform the actual import ... + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + print(f"{name!r} has been imported") + else: + print(f"can't find the {name!r} module") + + +File: python.info, Node: Importing a source file directly, Next: Setting up an importer, Prev: Checking if a module can be imported, Up: Examples<29> + +5.31.5.10 Importing a source file directly +.......................................... + +To import a Python source file directly, use the following recipe +(Python 3.5 and newer only): + + import importlib.util + import sys + + # For illustrative purposes. + import tokenize + file_path = tokenize.__file__ + module_name = tokenize.__name__ + + spec = importlib.util.spec_from_file_location(module_name, file_path) + module = importlib.util.module_from_spec(spec) + sys.modules[module_name] = module + spec.loader.exec_module(module) + + +File: python.info, Node: Setting up an importer, Next: Approximating importlib import_module, Prev: Importing a source file directly, Up: Examples<29> + +5.31.5.11 Setting up an importer +................................ + +For deep customizations of import, you typically want to implement an +*note importer: 1161. This means managing both the *note finder: 115f. +and *note loader: 1160. side of things. For finders there are two +flavours to choose from depending on your needs: a *note meta path +finder: 9b1. or a *note path entry finder: 9b2. The former is what you +would put on *note sys.meta_path: 5e6. while the latter is what you +create using a *note path entry hook: 1177. on *note sys.path_hooks: +95c. which works with *note sys.path: 488. entries to potentially create +a finder. This example will show you how to register your own importers +so that import will use them (for creating an importer for yourself, +read the documentation for the appropriate classes defined within this +package): + + import importlib.machinery + import sys + + # For illustrative purposes only. + SpamMetaPathFinder = importlib.machinery.PathFinder + SpamPathEntryFinder = importlib.machinery.FileFinder + loader_details = (importlib.machinery.SourceFileLoader, + importlib.machinery.SOURCE_SUFFIXES) + + # Setting up a meta path finder. + # Make sure to put the finder in the proper location in the list in terms of + # priority. + sys.meta_path.append(SpamMetaPathFinder) + + # Setting up a path entry finder. + # Make sure to put the path hook in the proper location in the list in terms + # of priority. + sys.path_hooks.append(SpamPathEntryFinder.path_hook(loader_details)) + + +File: python.info, Node: Approximating importlib import_module, Prev: Setting up an importer, Up: Examples<29> + +5.31.5.12 Approximating ‘importlib.import_module()’ +................................................... + +Import itself is implemented in Python code, making it possible to +expose most of the import machinery through importlib. The following +helps illustrate the various APIs that importlib exposes by providing an +approximate implementation of *note importlib.import_module(): b13. +(Python 3.4 and newer for the importlib usage, Python 3.6 and newer for +other parts of the code). + + import importlib.util + import sys + + def import_module(name, package=None): + """An approximate implementation of import.""" + absolute_name = importlib.util.resolve_name(name, package) + try: + return sys.modules[absolute_name] + except KeyError: + pass + + path = None + if '.' in absolute_name: + parent_name, _, child_name = absolute_name.rpartition('.') + parent_module = import_module(parent_name) + path = parent_module.__spec__.submodule_search_locations + for finder in sys.meta_path: + spec = finder.find_spec(absolute_name, path) + if spec is not None: + break + else: + msg = f'No module named {absolute_name!r}' + raise ModuleNotFoundError(msg, name=absolute_name) + module = importlib.util.module_from_spec(spec) + sys.modules[absolute_name] = module + spec.loader.exec_module(module) + if path is not None: + setattr(parent_module, child_name, module) + return module + + +File: python.info, Node: Using importlib metadata, Prev: importlib — The implementation of import, Up: Importing Modules + +5.31.6 Using ‘importlib.metadata’ +--------------------------------- + + Note: This functionality is provisional and may deviate from the + usual version semantics of the standard library. + +‘importlib.metadata’ is a library that provides for access to installed +package metadata. Built in part on Python’s import system, this library +intends to replace similar functionality in the entry point API(1) and +metadata API(2) of ‘pkg_resources’. Along with *note +importlib.resources: 9e. in Python 3.7 and newer (backported as +importlib_resources(3) for older versions of Python), this can eliminate +the need to use the older and less efficient ‘pkg_resources’ package. + +By “installed package” we generally mean a third-party package installed +into Python’s ‘site-packages’ directory via tools such as pip(4). +Specifically, it means a package with either a discoverable ‘dist-info’ +or ‘egg-info’ directory, and metadata defined by PEP 566(5) or its older +specifications. By default, package metadata can live on the file +system or in zip archives on *note sys.path: 488. Through an extension +mechanism, the metadata can live almost anywhere. + +* Menu: + +* Overview: Overview<2>. +* Functional API: Functional API<2>. +* Distributions:: +* Extending the search algorithm:: + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points + + (2) +https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api + + (3) https://importlib-resources.readthedocs.io/en/latest/index.html + + (4) https://pypi.org/project/pip/ + + (5) https://www.python.org/dev/peps/pep-0566 + + +File: python.info, Node: Overview<2>, Next: Functional API<2>, Up: Using importlib metadata + +5.31.6.1 Overview +................. + +Let’s say you wanted to get the version string for a package you’ve +installed using ‘pip’. We start by creating a virtual environment and +installing something into it: + + $ python3 -m venv example + $ source example/bin/activate + (example) $ pip install wheel + +You can get the version string for ‘wheel’ by running the following: + + (example) $ python + >>> from importlib.metadata import version + >>> version('wheel') + '0.32.3' + +You can also get the set of entry points keyed by group, such as +‘console_scripts’, ‘distutils.commands’ and others. Each group contains +a sequence of *note EntryPoint: 34d9. objects. + +You can get the *note metadata for a distribution: 34da.: + + >>> list(metadata('wheel')) + ['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'Maintainer', 'Maintainer-email', 'License', 'Project-URL', 'Project-URL', 'Project-URL', 'Keywords', 'Platform', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Requires-Python', 'Provides-Extra', 'Requires-Dist', 'Requires-Dist'] + +You can also get a *note distribution’s version number: 34db, list its +*note constituent files: 34dc, and get a list of the distribution’s +*note Distribution requirements: 34dd. + + +File: python.info, Node: Functional API<2>, Next: Distributions, Prev: Overview<2>, Up: Using importlib metadata + +5.31.6.2 Functional API +....................... + +This package provides the following functionality via its public API. + +* Menu: + +* Entry points:: +* Distribution metadata:: +* Distribution versions:: +* Distribution files:: +* Distribution requirements:: + + +File: python.info, Node: Entry points, Next: Distribution metadata, Up: Functional API<2> + +5.31.6.3 Entry points +..................... + +The ‘entry_points()’ function returns a dictionary of all entry points, +keyed by group. Entry points are represented by ‘EntryPoint’ instances; +each ‘EntryPoint’ has a ‘.name’, ‘.group’, and ‘.value’ attributes and a +‘.load()’ method to resolve the value. + + >>> eps = entry_points() + >>> list(eps) + ['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation'] + >>> scripts = eps['console_scripts'] + >>> wheel = [ep for ep in scripts if ep.name == 'wheel'][0] + >>> wheel + EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts') + >>> main = wheel.load() + >>> main + <function main at 0x103528488> + +The ‘group’ and ‘name’ are arbitrary values defined by the package +author and usually a client will wish to resolve all entry points for a +particular group. Read the setuptools docs(1) for more information on +entrypoints, their definition, and usage. + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins + + +File: python.info, Node: Distribution metadata, Next: Distribution versions, Prev: Entry points, Up: Functional API<2> + +5.31.6.4 Distribution metadata +.............................. + +Every distribution includes some metadata, which you can extract using +the ‘metadata()’ function: + + >>> wheel_metadata = metadata('wheel') + +The keys of the returned data structure (1) name the metadata keywords, +and their values are returned unparsed from the distribution metadata: + + >>> wheel_metadata['Requires-Python'] + '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*' + + ---------- Footnotes ---------- + + (1) (1) Technically, the returned distribution metadata object is an +*note email.message.EmailMessage: 542. instance, but this is an +implementation detail, and not part of the stable API. You should only +use dictionary-like methods and syntax to access the metadata contents. + + +File: python.info, Node: Distribution versions, Next: Distribution files, Prev: Distribution metadata, Up: Functional API<2> + +5.31.6.5 Distribution versions +.............................. + +The ‘version()’ function is the quickest way to get a distribution’s +version number, as a string: + + >>> version('wheel') + '0.32.3' + + +File: python.info, Node: Distribution files, Next: Distribution requirements, Prev: Distribution versions, Up: Functional API<2> + +5.31.6.6 Distribution files +........................... + +You can also get the full set of files contained within a distribution. +The ‘files()’ function takes a distribution package name and returns all +of the files installed by this distribution. Each file object returned +is a ‘PackagePath’, a *note pathlib.Path: 201. derived object with +additional ‘dist’, ‘size’, and ‘hash’ properties as indicated by the +metadata. For example: + + >>> util = [p for p in files('wheel') if 'util.py' in str(p)][0] + >>> util + PackagePath('wheel/util.py') + >>> util.size + 859 + >>> util.dist + <importlib.metadata._hooks.PathDistribution object at 0x101e0cef0> + >>> util.hash + <FileHash mode: sha256 value: bYkw5oMccfazVCoYQwKkkemoVyMAFoR34mmKBx8R1NI> + +Once you have the file, you can also read its contents: + + >>> print(util.read_text()) + import base64 + import sys + ... + def as_bytes(s): + if isinstance(s, text_type): + return s.encode('utf-8') + return s + +In the case where the metadata file listing files (RECORD or +SOURCES.txt) is missing, ‘files()’ will return ‘None’. The caller may +wish to wrap calls to ‘files()’ in always_iterable(1) or otherwise guard +against this condition if the target distribution is not known to have +the metadata present. + + ---------- Footnotes ---------- + + (1) +https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_iterable + + +File: python.info, Node: Distribution requirements, Prev: Distribution files, Up: Functional API<2> + +5.31.6.7 Distribution requirements +.................................. + +To get the full set of requirements for a distribution, use the +‘requires()’ function: + + >>> requires('wheel') + ["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"] + + +File: python.info, Node: Distributions, Next: Extending the search algorithm, Prev: Functional API<2>, Up: Using importlib metadata + +5.31.6.8 Distributions +...................... + +While the above API is the most common and convenient usage, you can get +all of that information from the ‘Distribution’ class. A ‘Distribution’ +is an abstract object that represents the metadata for a Python package. +You can get the ‘Distribution’ instance: + + >>> from importlib.metadata import distribution + >>> dist = distribution('wheel') + +Thus, an alternative way to get the version number is through the +‘Distribution’ instance: + + >>> dist.version + '0.32.3' + +There are all kinds of additional metadata available on the +‘Distribution’ instance: + + >>> dist.metadata['Requires-Python'] + '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*' + >>> dist.metadata['License'] + 'MIT' + +The full set of available metadata is not described here. See PEP +566(1) for additional details. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0566 + + +File: python.info, Node: Extending the search algorithm, Prev: Distributions, Up: Using importlib metadata + +5.31.6.9 Extending the search algorithm +....................................... + +Because package metadata is not available through *note sys.path: 488. +searches, or package loaders directly, the metadata for a package is +found through import system *note finders: 115d. To find a distribution +package’s metadata, ‘importlib.metadata’ queries the list of *note meta +path finders: 9b1. on *note sys.meta_path: 5e6. + +The default ‘PathFinder’ for Python includes a hook that calls into +‘importlib.metadata.MetadataPathFinder’ for finding distributions loaded +from typical file-system-based paths. + +The abstract class *note importlib.abc.MetaPathFinder: 9b3. defines the +interface expected of finders by Python’s import system. +‘importlib.metadata’ extends this protocol by looking for an optional +‘find_distributions’ callable on the finders from *note sys.meta_path: +5e6. and presents this extended interface as the ‘DistributionFinder’ +abstract base class, which defines this abstract method: + + @abc.abstractmethod + def find_distributions(context=DistributionFinder.Context()): + """Return an iterable of all Distribution instances capable of + loading the metadata for packages for the indicated ``context``. + """ + +The ‘DistributionFinder.Context’ object provides ‘.path’ and ‘.name’ +properties indicating the path to search and names to match and may +supply other relevant context. + +What this means in practice is that to support finding distribution +package metadata in locations other than the file system, subclass +‘Distribution’ and implement the abstract methods. Then from a custom +finder, return instances of this derived ‘Distribution’ in the +‘find_distributions()’ method. + + +File: python.info, Node: Python Language Services, Next: Miscellaneous Services, Prev: Importing Modules, Up: The Python Standard Library + +5.32 Python Language Services +============================= + +Python provides a number of modules to assist in working with the Python +language. These modules support tokenizing, parsing, syntax analysis, +bytecode disassembly, and various other facilities. + +These modules include: + +* Menu: + +* parser — Access Python parse trees:: +* ast — Abstract Syntax Trees:: +* symtable — Access to the compiler’s symbol tables:: +* symbol — Constants used with Python parse trees:: +* token — Constants used with Python parse trees:: +* keyword — Testing for Python keywords:: +* tokenize — Tokenizer for Python source:: +* tabnanny — Detection of ambiguous indentation:: +* pyclbr — Python module browser support:: +* py_compile — Compile Python source files:: +* compileall — Byte-compile Python libraries:: +* dis — Disassembler for Python bytecode:: +* pickletools — Tools for pickle developers:: + + +File: python.info, Node: parser — Access Python parse trees, Next: ast — Abstract Syntax Trees, Up: Python Language Services + +5.32.1 ‘parser’ — Access Python parse trees +------------------------------------------- + +__________________________________________________________________ + +The *note parser: c7. module provides an interface to Python’s internal +parser and byte-code compiler. The primary purpose for this interface +is to allow Python code to edit the parse tree of a Python expression +and create executable code from this. This is better than trying to +parse and modify an arbitrary Python code fragment as a string because +parsing is performed in a manner identical to the code forming the +application. It is also faster. + + Note: From Python 2.5 onward, it’s much more convenient to cut in + at the Abstract Syntax Tree (AST) generation and compilation stage, + using the *note ast: 8. module. + +There are a few things to note about this module which are important to +making use of the data structures created. This is not a tutorial on +editing the parse trees for Python code, but some examples of using the +*note parser: c7. module are presented. + +Most importantly, a good understanding of the Python grammar processed +by the internal parser is required. For full information on the +language syntax, refer to *note The Python Language Reference: e8f. The +parser itself is created from a grammar specification defined in the +file ‘Grammar/Grammar’ in the standard Python distribution. The parse +trees stored in the ST objects created by this module are the actual +output from the internal parser when created by the *note expr(): 34eb. +or *note suite(): 34ec. functions, described below. The ST objects +created by *note sequence2st(): 34ed. faithfully simulate those +structures. Be aware that the values of the sequences which are +considered “correct” will vary from one version of Python to another as +the formal grammar for the language is revised. However, transporting +code from one Python version to another as source text will always allow +correct parse trees to be created in the target version, with the only +restriction being that migrating to an older version of the interpreter +will not support more recent language constructs. The parse trees are +not typically compatible from one version to another, though source code +has usually been forward-compatible within a major release series. + +Each element of the sequences returned by *note st2list(): 34ee. or +*note st2tuple(): 34ef. has a simple form. Sequences representing +non-terminal elements in the grammar always have a length greater than +one. The first element is an integer which identifies a production in +the grammar. These integers are given symbolic names in the C header +file ‘Include/graminit.h’ and the Python module *note symbol: fb. Each +additional element of the sequence represents a component of the +production as recognized in the input string: these are always sequences +which have the same form as the parent. An important aspect of this +structure which should be noted is that keywords used to identify the +parent node type, such as the keyword *note if: de7. in an ‘if_stmt’, +are included in the node tree without any special treatment. For +example, the ‘if’ keyword is represented by the tuple ‘(1, 'if')’, where +‘1’ is the numeric value associated with all ‘NAME’ tokens, including +variable and function names defined by the user. In an alternate form +returned when line number information is requested, the same token might +be represented as ‘(1, 'if', 12)’, where the ‘12’ represents the line +number at which the terminal symbol was found. + +Terminal elements are represented in much the same way, but without any +child elements and the addition of the source text which was identified. +The example of the *note if: de7. keyword above is representative. The +various types of terminal symbols are defined in the C header file +‘Include/token.h’ and the Python module *note token: 110. + +The ST objects are not required to support the functionality of this +module, but are provided for three purposes: to allow an application to +amortize the cost of processing complex parse trees, to provide a parse +tree representation which conserves memory space when compared to the +Python list or tuple representation, and to ease the creation of +additional modules in C which manipulate parse trees. A simple +“wrapper” class may be created in Python to hide the use of ST objects. + +The *note parser: c7. module defines functions for a few distinct +purposes. The most important purposes are to create ST objects and to +convert ST objects to other representations such as parse trees and +compiled code objects, but there are also functions which serve to query +the type of parse tree represented by an ST object. + +See also +........ + +Module *note symbol: fb. + + Useful constants representing internal nodes of the parse tree. + +Module *note token: 110. + + Useful constants representing leaf nodes of the parse tree and + functions for testing node values. + +* Menu: + +* Creating ST Objects:: +* Converting ST Objects:: +* Queries on ST Objects:: +* Exceptions and Error Handling:: +* ST Objects:: +* Example; Emulation of compile(): Example Emulation of compile. + + +File: python.info, Node: Creating ST Objects, Next: Converting ST Objects, Up: parser — Access Python parse trees + +5.32.1.1 Creating ST Objects +............................ + +ST objects may be created from source code or from a parse tree. When +creating an ST object from source, different functions are used to +create the ‘'eval'’ and ‘'exec'’ forms. + + -- Function: parser.expr (source) + + The *note expr(): 34eb. function parses the parameter `source' as + if it were an input to ‘compile(source, 'file.py', 'eval')’. If + the parse succeeds, an ST object is created to hold the internal + parse tree representation, otherwise an appropriate exception is + raised. + + -- Function: parser.suite (source) + + The *note suite(): 34ec. function parses the parameter `source' as + if it were an input to ‘compile(source, 'file.py', 'exec')’. If + the parse succeeds, an ST object is created to hold the internal + parse tree representation, otherwise an appropriate exception is + raised. + + -- Function: parser.sequence2st (sequence) + + This function accepts a parse tree represented as a sequence and + builds an internal representation if possible. If it can validate + that the tree conforms to the Python grammar and all nodes are + valid node types in the host version of Python, an ST object is + created from the internal representation and returned to the + called. If there is a problem creating the internal + representation, or if the tree cannot be validated, a *note + ParserError: 34f2. exception is raised. An ST object created this + way should not be assumed to compile correctly; normal exceptions + raised by compilation may still be initiated when the ST object is + passed to *note compilest(): 34f3. This may indicate problems not + related to syntax (such as a *note MemoryError: 13af. exception), + but may also be due to constructs such as the result of parsing + ‘del f(0)’, which escapes the Python parser but is checked by the + bytecode compiler. + + Sequences representing terminal tokens may be represented as either + two-element lists of the form ‘(1, 'name')’ or as three-element + lists of the form ‘(1, 'name', 56)’. If the third element is + present, it is assumed to be a valid line number. The line number + may be specified for any subset of the terminal symbols in the + input tree. + + -- Function: parser.tuple2st (sequence) + + This is the same function as *note sequence2st(): 34ed. This entry + point is maintained for backward compatibility. + + +File: python.info, Node: Converting ST Objects, Next: Queries on ST Objects, Prev: Creating ST Objects, Up: parser — Access Python parse trees + +5.32.1.2 Converting ST Objects +.............................. + +ST objects, regardless of the input used to create them, may be +converted to parse trees represented as list- or tuple- trees, or may be +compiled into executable code objects. Parse trees may be extracted +with or without line numbering information. + + -- Function: parser.st2list (st, line_info=False, col_info=False) + + This function accepts an ST object from the caller in `st' and + returns a Python list representing the equivalent parse tree. The + resulting list representation can be used for inspection or the + creation of a new parse tree in list form. This function does not + fail so long as memory is available to build the list + representation. If the parse tree will only be used for + inspection, *note st2tuple(): 34ef. should be used instead to + reduce memory consumption and fragmentation. When the list + representation is required, this function is significantly faster + than retrieving a tuple representation and converting that to + nested lists. + + If `line_info' is true, line number information will be included + for all terminal tokens as a third element of the list representing + the token. Note that the line number provided specifies the line + on which the token `ends'. This information is omitted if the flag + is false or omitted. + + -- Function: parser.st2tuple (st, line_info=False, col_info=False) + + This function accepts an ST object from the caller in `st' and + returns a Python tuple representing the equivalent parse tree. + Other than returning a tuple instead of a list, this function is + identical to *note st2list(): 34ee. + + If `line_info' is true, line number information will be included + for all terminal tokens as a third element of the list representing + the token. This information is omitted if the flag is false or + omitted. + + -- Function: parser.compilest (st, filename='<syntax-tree>') + + The Python byte compiler can be invoked on an ST object to produce + code objects which can be used as part of a call to the built-in + *note exec(): c44. or *note eval(): b90. functions. This function + provides the interface to the compiler, passing the internal parse + tree from `st' to the parser, using the source file name specified + by the `filename' parameter. The default value supplied for + `filename' indicates that the source was an ST object. + + Compiling an ST object may result in exceptions related to + compilation; an example would be a *note SyntaxError: 458. caused + by the parse tree for ‘del f(0)’: this statement is considered + legal within the formal grammar for Python but is not a legal + language construct. The *note SyntaxError: 458. raised for this + condition is actually generated by the Python byte-compiler + normally, which is why it can be raised at this point by the *note + parser: c7. module. Most causes of compilation failure can be + diagnosed programmatically by inspection of the parse tree. + + +File: python.info, Node: Queries on ST Objects, Next: Exceptions and Error Handling, Prev: Converting ST Objects, Up: parser — Access Python parse trees + +5.32.1.3 Queries on ST Objects +.............................. + +Two functions are provided which allow an application to determine if an +ST was created as an expression or a suite. Neither of these functions +can be used to determine if an ST was created from source code via *note +expr(): 34eb. or *note suite(): 34ec. or from a parse tree via *note +sequence2st(): 34ed. + + -- Function: parser.isexpr (st) + + When `st' represents an ‘'eval'’ form, this function returns + ‘True’, otherwise it returns ‘False’. This is useful, since code + objects normally cannot be queried for this information using + existing built-in functions. Note that the code objects created by + *note compilest(): 34f3. cannot be queried like this either, and + are identical to those created by the built-in *note compile(): + 1b4. function. + + -- Function: parser.issuite (st) + + This function mirrors *note isexpr(): 34f9. in that it reports + whether an ST object represents an ‘'exec'’ form, commonly known as + a “suite.” It is not safe to assume that this function is + equivalent to ‘not isexpr(st)’, as additional syntactic fragments + may be supported in the future. + + +File: python.info, Node: Exceptions and Error Handling, Next: ST Objects, Prev: Queries on ST Objects, Up: parser — Access Python parse trees + +5.32.1.4 Exceptions and Error Handling +...................................... + +The parser module defines a single exception, but may also pass other +built-in exceptions from other portions of the Python runtime +environment. See each function for information about the exceptions it +can raise. + + -- Exception: parser.ParserError + + Exception raised when a failure occurs within the parser module. + This is generally produced for validation failures rather than the + built-in *note SyntaxError: 458. raised during normal parsing. The + exception argument is either a string describing the reason of the + failure or a tuple containing a sequence causing the failure from a + parse tree passed to *note sequence2st(): 34ed. and an explanatory + string. Calls to *note sequence2st(): 34ed. need to be able to + handle either type of exception, while calls to other functions in + the module will only need to be aware of the simple string values. + +Note that the functions *note compilest(): 34f3, *note expr(): 34eb, and +*note suite(): 34ec. may raise exceptions which are normally raised by +the parsing and compilation process. These include the built in +exceptions *note MemoryError: 13af, *note OverflowError: 960, *note +SyntaxError: 458, and *note SystemError: 5fb. In these cases, these +exceptions carry all the meaning normally associated with them. Refer +to the descriptions of each function for detailed information. + + +File: python.info, Node: ST Objects, Next: Example Emulation of compile, Prev: Exceptions and Error Handling, Up: parser — Access Python parse trees + +5.32.1.5 ST Objects +................... + +Ordered and equality comparisons are supported between ST objects. +Pickling of ST objects (using the *note pickle: ca. module) is also +supported. + + -- Data: parser.STType + + The type of the objects returned by *note expr(): 34eb, *note + suite(): 34ec. and *note sequence2st(): 34ed. + +ST objects have the following methods: + + -- Method: ST.compile (filename='<syntax-tree>') + + Same as ‘compilest(st, filename)’. + + -- Method: ST.isexpr () + + Same as ‘isexpr(st)’. + + -- Method: ST.issuite () + + Same as ‘issuite(st)’. + + -- Method: ST.tolist (line_info=False, col_info=False) + + Same as ‘st2list(st, line_info, col_info)’. + + -- Method: ST.totuple (line_info=False, col_info=False) + + Same as ‘st2tuple(st, line_info, col_info)’. + + +File: python.info, Node: Example Emulation of compile, Prev: ST Objects, Up: parser — Access Python parse trees + +5.32.1.6 Example: Emulation of ‘compile()’ +.......................................... + +While many useful operations may take place between parsing and bytecode +generation, the simplest operation is to do nothing. For this purpose, +using the *note parser: c7. module to produce an intermediate data +structure is equivalent to the code + + >>> code = compile('a + 5', 'file.py', 'eval') + >>> a = 5 + >>> eval(code) + 10 + +The equivalent operation using the *note parser: c7. module is somewhat +longer, and allows the intermediate internal parse tree to be retained +as an ST object: + + >>> import parser + >>> st = parser.expr('a + 5') + >>> code = st.compile('file.py') + >>> a = 5 + >>> eval(code) + 10 + +An application which needs both ST and code objects can package this +code into readily available functions: + + import parser + + def load_suite(source_string): + st = parser.suite(source_string) + return st, st.compile() + + def load_expression(source_string): + st = parser.expr(source_string) + return st, st.compile() + + +File: python.info, Node: ast — Abstract Syntax Trees, Next: symtable — Access to the compiler’s symbol tables, Prev: parser — Access Python parse trees, Up: Python Language Services + +5.32.2 ‘ast’ — Abstract Syntax Trees +------------------------------------ + +`Source code:' Lib/ast.py(1) + +__________________________________________________________________ + +The *note ast: 8. module helps Python applications to process trees of +the Python abstract syntax grammar. The abstract syntax itself might +change with each Python release; this module helps to find out +programmatically what the current grammar looks like. + +An abstract syntax tree can be generated by passing ‘ast.PyCF_ONLY_AST’ +as a flag to the *note compile(): 1b4. built-in function, or using the +*note parse(): 1a1. helper provided in this module. The result will be +a tree of objects whose classes all inherit from *note ast.AST: 3508. +An abstract syntax tree can be compiled into a Python code object using +the built-in *note compile(): 1b4. function. + +* Menu: + +* Node classes:: +* Abstract Grammar:: +* ast Helpers:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/ast.py + + +File: python.info, Node: Node classes, Next: Abstract Grammar, Up: ast — Abstract Syntax Trees + +5.32.2.1 Node classes +..................... + + -- Class: ast.AST + + This is the base of all AST node classes. The actual node classes + are derived from the ‘Parser/Python.asdl’ file, which is reproduced + *note below: 350a. They are defined in the ‘_ast’ C module and + re-exported in *note ast: 8. + + There is one class defined for each left-hand side symbol in the + abstract grammar (for example, ‘ast.stmt’ or ‘ast.expr’). In + addition, there is one class defined for each constructor on the + right-hand side; these classes inherit from the classes for the + left-hand side trees. For example, ‘ast.BinOp’ inherits from + ‘ast.expr’. For production rules with alternatives (aka “sums”), + the left-hand side class is abstract: only instances of specific + constructor nodes are ever created. + + -- Attribute: _fields + + Each concrete class has an attribute *note _fields: 350b. + which gives the names of all child nodes. + + Each instance of a concrete class has one attribute for each + child node, of the type as defined in the grammar. For + example, ‘ast.BinOp’ instances have an attribute ‘left’ of + type ‘ast.expr’. + + If these attributes are marked as optional in the grammar + (using a question mark), the value might be ‘None’. If the + attributes can have zero-or-more values (marked with an + asterisk), the values are represented as Python lists. All + possible attributes must be present and have valid values when + compiling an AST with *note compile(): 1b4. + + -- Attribute: lineno + -- Attribute: col_offset + -- Attribute: end_lineno + -- Attribute: end_col_offset + + Instances of ‘ast.expr’ and ‘ast.stmt’ subclasses have *note + lineno: 350c, *note col_offset: 350d, *note lineno: 350c, and + *note col_offset: 350d. attributes. The *note lineno: 350c. + and *note end_lineno: 350e. are the first and last line + numbers of source text span (1-indexed so the first line is + line 1) and the *note col_offset: 350d. and *note + end_col_offset: 350f. are the corresponding UTF-8 byte offsets + of the first and last tokens that generated the node. The + UTF-8 offset is recorded because the parser uses UTF-8 + internally. + + Note that the end positions are not required by the compiler + and are therefore optional. The end offset is `after' the + last symbol, for example one can get the source segment of a + one-line expression node using ‘source_line[node.col_offset : + node.end_col_offset]’. + + The constructor of a class ‘ast.T’ parses its arguments as follows: + + * If there are positional arguments, there must be as many as + there are items in ‘T._fields’; they will be assigned as + attributes of these names. + + * If there are keyword arguments, they will set the attributes + of the same names to the given values. + + For example, to create and populate an ‘ast.UnaryOp’ node, you + could use + + node = ast.UnaryOp() + node.op = ast.USub() + node.operand = ast.Constant() + node.operand.value = 5 + node.operand.lineno = 0 + node.operand.col_offset = 0 + node.lineno = 0 + node.col_offset = 0 + + or the more compact + + node = ast.UnaryOp(ast.USub(), ast.Constant(5, lineno=0, col_offset=0), + lineno=0, col_offset=0) + +Changed in version 3.8: Class ‘ast.Constant’ is now used for all +constants. + +Deprecated since version 3.8: Old classes ‘ast.Num’, ‘ast.Str’, +‘ast.Bytes’, ‘ast.NameConstant’ and ‘ast.Ellipsis’ are still available, +but they will be removed in future Python releases. In the meanwhile, +instantiating them will return an instance of a different class. + + +File: python.info, Node: Abstract Grammar, Next: ast Helpers, Prev: Node classes, Up: ast — Abstract Syntax Trees + +5.32.2.2 Abstract Grammar +......................... + +The abstract grammar is currently defined as follows: + + -- ASDL's 5 builtin types are: + -- identifier, int, string, object, constant + + module Python + { + mod = Module(stmt* body, type_ignore *type_ignores) + | Interactive(stmt* body) + | Expression(expr body) + | FunctionType(expr* argtypes, expr returns) + + -- not really an actual node but useful in Jython's typesystem. + | Suite(stmt* body) + + stmt = FunctionDef(identifier name, arguments args, + stmt* body, expr* decorator_list, expr? returns, + string? type_comment) + | AsyncFunctionDef(identifier name, arguments args, + stmt* body, expr* decorator_list, expr? returns, + string? type_comment) + + | ClassDef(identifier name, + expr* bases, + keyword* keywords, + stmt* body, + expr* decorator_list) + | Return(expr? value) + + | Delete(expr* targets) + | Assign(expr* targets, expr value, string? type_comment) + | AugAssign(expr target, operator op, expr value) + -- 'simple' indicates that we annotate simple name without parens + | AnnAssign(expr target, expr annotation, expr? value, int simple) + + -- use 'orelse' because else is a keyword in target languages + | For(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment) + | AsyncFor(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment) + | While(expr test, stmt* body, stmt* orelse) + | If(expr test, stmt* body, stmt* orelse) + | With(withitem* items, stmt* body, string? type_comment) + | AsyncWith(withitem* items, stmt* body, string? type_comment) + + | Raise(expr? exc, expr? cause) + | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) + | Assert(expr test, expr? msg) + + | Import(alias* names) + | ImportFrom(identifier? module, alias* names, int? level) + + | Global(identifier* names) + | Nonlocal(identifier* names) + | Expr(expr value) + | Pass | Break | Continue + + -- XXX Jython will be different + -- col_offset is the byte offset in the utf8 string the parser uses + attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) + + -- BoolOp() can use left & right? + expr = BoolOp(boolop op, expr* values) + | NamedExpr(expr target, expr value) + | BinOp(expr left, operator op, expr right) + | UnaryOp(unaryop op, expr operand) + | Lambda(arguments args, expr body) + | IfExp(expr test, expr body, expr orelse) + | Dict(expr* keys, expr* values) + | Set(expr* elts) + | ListComp(expr elt, comprehension* generators) + | SetComp(expr elt, comprehension* generators) + | DictComp(expr key, expr value, comprehension* generators) + | GeneratorExp(expr elt, comprehension* generators) + -- the grammar constrains where yield expressions can occur + | Await(expr value) + | Yield(expr? value) + | YieldFrom(expr value) + -- need sequences for compare to distinguish between + -- x < 4 < 3 and (x < 4) < 3 + | Compare(expr left, cmpop* ops, expr* comparators) + | Call(expr func, expr* args, keyword* keywords) + | FormattedValue(expr value, int? conversion, expr? format_spec) + | JoinedStr(expr* values) + | Constant(constant value, string? kind) + + -- the following expression can appear in assignment context + | Attribute(expr value, identifier attr, expr_context ctx) + | Subscript(expr value, slice slice, expr_context ctx) + | Starred(expr value, expr_context ctx) + | Name(identifier id, expr_context ctx) + | List(expr* elts, expr_context ctx) + | Tuple(expr* elts, expr_context ctx) + + -- col_offset is the byte offset in the utf8 string the parser uses + attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) + + expr_context = Load | Store | Del | AugLoad | AugStore | Param + + slice = Slice(expr? lower, expr? upper, expr? step) + | ExtSlice(slice* dims) + | Index(expr value) + + boolop = And | Or + + operator = Add | Sub | Mult | MatMult | Div | Mod | Pow | LShift + | RShift | BitOr | BitXor | BitAnd | FloorDiv + + unaryop = Invert | Not | UAdd | USub + + cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn + + comprehension = (expr target, expr iter, expr* ifs, int is_async) + + excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) + attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) + + arguments = (arg* posonlyargs, arg* args, arg? vararg, arg* kwonlyargs, + expr* kw_defaults, arg? kwarg, expr* defaults) + + arg = (identifier arg, expr? annotation, string? type_comment) + attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) + + -- keyword arguments supplied to call (NULL identifier for **kwargs) + keyword = (identifier? arg, expr value) + + -- import name with optional 'as' alias. + alias = (identifier name, identifier? asname) + + withitem = (expr context_expr, expr? optional_vars) + + type_ignore = TypeIgnore(int lineno, string tag) + } + + +File: python.info, Node: ast Helpers, Prev: Abstract Grammar, Up: ast — Abstract Syntax Trees + +5.32.2.3 ‘ast’ Helpers +...................... + +Apart from the node classes, the *note ast: 8. module defines these +utility functions and classes for traversing abstract syntax trees: + + -- Function: ast.parse (source, filename='<unknown>', mode='exec', *, + type_comments=False, feature_version=None) + + Parse the source into an AST node. Equivalent to ‘compile(source, + filename, mode, ast.PyCF_ONLY_AST)’. + + If ‘type_comments=True’ is given, the parser is modified to check + and return type comments as specified by PEP 484(1) and PEP 526(2). + This is equivalent to adding ‘ast.PyCF_TYPE_COMMENTS’ to the flags + passed to *note compile(): 1b4. This will report syntax errors for + misplaced type comments. Without this flag, type comments will be + ignored, and the ‘type_comment’ field on selected AST nodes will + always be ‘None’. In addition, the locations of ‘# type: ignore’ + comments will be returned as the ‘type_ignores’ attribute of + ‘Module’ (otherwise it is always an empty list). + + In addition, if ‘mode’ is ‘'func_type'’, the input syntax is + modified to correspond to PEP 484(3) “signature type comments”, + e.g. ‘(str, int) -> List[str]’. + + Also, setting ‘feature_version’ to a tuple ‘(major, minor)’ will + attempt to parse using that Python version’s grammar. Currently + ‘major’ must equal to ‘3’. For example, setting + ‘feature_version=(3, 4)’ will allow the use of ‘async’ and ‘await’ + as variable names. The lowest supported version is ‘(3, 4)’; the + highest is ‘sys.version_info[0:2]’. + + Warning: It is possible to crash the Python interpreter with a + sufficiently large/complex string due to stack depth + limitations in Python’s AST compiler. + + Changed in version 3.8: Added ‘type_comments’, ‘mode='func_type'’ + and ‘feature_version’. + + -- Function: ast.literal_eval (node_or_string) + + Safely evaluate an expression node or a string containing a Python + literal or container display. The string or node provided may only + consist of the following Python literal structures: strings, bytes, + numbers, tuples, lists, dicts, sets, booleans, and ‘None’. + + This can be used for safely evaluating strings containing Python + values from untrusted sources without the need to parse the values + oneself. It is not capable of evaluating arbitrarily complex + expressions, for example involving operators or indexing. + + Warning: It is possible to crash the Python interpreter with a + sufficiently large/complex string due to stack depth + limitations in Python’s AST compiler. + + Changed in version 3.2: Now allows bytes and set literals. + + -- Function: ast.get_docstring (node, clean=True) + + Return the docstring of the given `node' (which must be a + ‘FunctionDef’, ‘AsyncFunctionDef’, ‘ClassDef’, or ‘Module’ node), + or ‘None’ if it has no docstring. If `clean' is true, clean up the + docstring’s indentation with *note inspect.cleandoc(): 3409. + + Changed in version 3.5: ‘AsyncFunctionDef’ is now supported. + + -- Function: ast.get_source_segment (source, node, *, padded=False) + + Get source code segment of the `source' that generated `node'. If + some location information (‘lineno’, ‘end_lineno’, ‘col_offset’, or + ‘end_col_offset’) is missing, return ‘None’. + + If `padded' is ‘True’, the first line of a multi-line statement + will be padded with spaces to match its original position. + + New in version 3.8. + + -- Function: ast.fix_missing_locations (node) + + When you compile a node tree with *note compile(): 1b4, the + compiler expects ‘lineno’ and ‘col_offset’ attributes for every + node that supports them. This is rather tedious to fill in for + generated nodes, so this helper adds these attributes recursively + where not already set, by setting them to the values of the parent + node. It works recursively starting at `node'. + + -- Function: ast.increment_lineno (node, n=1) + + Increment the line number and end line number of each node in the + tree starting at `node' by `n'. This is useful to “move code” to a + different location in a file. + + -- Function: ast.copy_location (new_node, old_node) + + Copy source location (‘lineno’, ‘col_offset’, ‘end_lineno’, and + ‘end_col_offset’) from `old_node' to `new_node' if possible, and + return `new_node'. + + -- Function: ast.iter_fields (node) + + Yield a tuple of ‘(fieldname, value)’ for each field in + ‘node._fields’ that is present on `node'. + + -- Function: ast.iter_child_nodes (node) + + Yield all direct child nodes of `node', that is, all fields that + are nodes and all items of fields that are lists of nodes. + + -- Function: ast.walk (node) + + Recursively yield all descendant nodes in the tree starting at + `node' (including `node' itself), in no specified order. This is + useful if you only want to modify nodes in place and don’t care + about the context. + + -- Class: ast.NodeVisitor + + A node visitor base class that walks the abstract syntax tree and + calls a visitor function for every node found. This function may + return a value which is forwarded by the *note visit(): 3519. + method. + + This class is meant to be subclassed, with the subclass adding + visitor methods. + + -- Method: visit (node) + + Visit a node. The default implementation calls the method + called ‘self.visit_`classname'’ where `classname' is the name + of the node class, or *note generic_visit(): 351a. if that + method doesn’t exist. + + -- Method: generic_visit (node) + + This visitor calls *note visit(): 3519. on all children of the + node. + + Note that child nodes of nodes that have a custom visitor + method won’t be visited unless the visitor calls *note + generic_visit(): 351a. or visits them itself. + + Don’t use the *note NodeVisitor: 281. if you want to apply changes + to nodes during traversal. For this a special visitor exists + (*note NodeTransformer: 351b.) that allows modifications. + + Deprecated since version 3.8: Methods ‘visit_Num()’, ‘visit_Str()’, + ‘visit_Bytes()’, ‘visit_NameConstant()’ and ‘visit_Ellipsis()’ are + deprecated now and will not be called in future Python versions. + Add the ‘visit_Constant()’ method to handle all constant nodes. + + -- Class: ast.NodeTransformer + + A *note NodeVisitor: 281. subclass that walks the abstract syntax + tree and allows modification of nodes. + + The *note NodeTransformer: 351b. will walk the AST and use the + return value of the visitor methods to replace or remove the old + node. If the return value of the visitor method is ‘None’, the + node will be removed from its location, otherwise it is replaced + with the return value. The return value may be the original node + in which case no replacement takes place. + + Here is an example transformer that rewrites all occurrences of + name lookups (‘foo’) to ‘data['foo']’: + + class RewriteName(NodeTransformer): + + def visit_Name(self, node): + return Subscript( + value=Name(id='data', ctx=Load()), + slice=Index(value=Constant(value=node.id)), + ctx=node.ctx + ) + + Keep in mind that if the node you’re operating on has child nodes + you must either transform the child nodes yourself or call the + ‘generic_visit()’ method for the node first. + + For nodes that were part of a collection of statements (that + applies to all statement nodes), the visitor may also return a list + of nodes rather than just a single node. + + If *note NodeTransformer: 351b. introduces new nodes (that weren’t + part of original tree) without giving them location information + (such as ‘lineno’), *note fix_missing_locations(): 3513. should be + called with the new sub-tree to recalculate the location + information: + + tree = ast.parse('foo', mode='eval') + new_tree = fix_missing_locations(RewriteName().visit(tree)) + + Usually you use the transformer like this: + + node = YourTransformer().visit(node) + + -- Function: ast.dump (node, annotate_fields=True, + include_attributes=False) + + Return a formatted dump of the tree in `node'. This is mainly + useful for debugging purposes. If `annotate_fields' is true (by + default), the returned string will show the names and the values + for fields. If `annotate_fields' is false, the result string will + be more compact by omitting unambiguous field names. Attributes + such as line numbers and column offsets are not dumped by default. + If this is wanted, `include_attributes' can be set to true. + +See also +........ + +Green Tree Snakes(4), an external documentation resource, has good +details on working with Python ASTs. + +ASTTokens(5) annotates Python ASTs with the positions of tokens and text +in the source code that generated them. This is helpful for tools that +make source code transformations. + +leoAst.py(6) unifies the token-based and parse-tree-based views of +python programs by inserting two-way links between tokens and ast nodes. + +LibCST(7) parses code as a Concrete Syntax Tree that looks like an ast +tree and keeps all formatting details. It’s useful for building +automated refactoring (codemod) applications and linters. + +Parso(8) is a Python parser that supports error recovery and round-trip +parsing for different Python versions (in multiple Python versions). +Parso is also able to list multiple syntax errors in your python file. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + (2) https://www.python.org/dev/peps/pep-0526 + + (3) https://www.python.org/dev/peps/pep-0484 + + (4) https://greentreesnakes.readthedocs.io/ + + (5) https://asttokens.readthedocs.io/en/latest/user-guide.html + + (6) http://leoeditor.com/appendices.html#leoast-py + + (7) https://libcst.readthedocs.io/ + + (8) https://parso.readthedocs.io + + +File: python.info, Node: symtable — Access to the compiler’s symbol tables, Next: symbol — Constants used with Python parse trees, Prev: ast — Abstract Syntax Trees, Up: Python Language Services + +5.32.3 ‘symtable’ — Access to the compiler’s symbol tables +---------------------------------------------------------- + +`Source code:' Lib/symtable.py(1) + +__________________________________________________________________ + +Symbol tables are generated by the compiler from AST just before +bytecode is generated. The symbol table is responsible for calculating +the scope of every identifier in the code. *note symtable: fc. provides +an interface to examine these tables. + +* Menu: + +* Generating Symbol Tables:: +* Examining Symbol Tables:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/symtable.py + + +File: python.info, Node: Generating Symbol Tables, Next: Examining Symbol Tables, Up: symtable — Access to the compiler’s symbol tables + +5.32.3.1 Generating Symbol Tables +................................. + + -- Function: symtable.symtable (code, filename, compile_type) + + Return the toplevel *note SymbolTable: 3521. for the Python source + `code'. `filename' is the name of the file containing the code. + `compile_type' is like the `mode' argument to *note compile(): 1b4. + + +File: python.info, Node: Examining Symbol Tables, Prev: Generating Symbol Tables, Up: symtable — Access to the compiler’s symbol tables + +5.32.3.2 Examining Symbol Tables +................................ + + -- Class: symtable.SymbolTable + + A namespace table for a block. The constructor is not public. + + -- Method: get_type () + + Return the type of the symbol table. Possible values are + ‘'class'’, ‘'module'’, and ‘'function'’. + + -- Method: get_id () + + Return the table’s identifier. + + -- Method: get_name () + + Return the table’s name. This is the name of the class if the + table is for a class, the name of the function if the table is + for a function, or ‘'top'’ if the table is global (*note + get_type(): 3523. returns ‘'module'’). + + -- Method: get_lineno () + + Return the number of the first line in the block this table + represents. + + -- Method: is_optimized () + + Return ‘True’ if the locals in this table can be optimized. + + -- Method: is_nested () + + Return ‘True’ if the block is a nested class or function. + + -- Method: has_children () + + Return ‘True’ if the block has nested namespaces within it. + These can be obtained with *note get_children(): 352a. + + -- Method: has_exec () + + Return ‘True’ if the block uses ‘exec’. + + -- Method: get_identifiers () + + Return a list of names of symbols in this table. + + -- Method: lookup (name) + + Lookup `name' in the table and return a *note Symbol: 352e. + instance. + + -- Method: get_symbols () + + Return a list of *note Symbol: 352e. instances for names in + the table. + + -- Method: get_children () + + Return a list of the nested symbol tables. + + -- Class: symtable.Function + + A namespace for a function or method. This class inherits *note + SymbolTable: 3521. + + -- Method: get_parameters () + + Return a tuple containing names of parameters to this + function. + + -- Method: get_locals () + + Return a tuple containing names of locals in this function. + + -- Method: get_globals () + + Return a tuple containing names of globals in this function. + + -- Method: get_nonlocals () + + Return a tuple containing names of nonlocals in this function. + + -- Method: get_frees () + + Return a tuple containing names of free variables in this + function. + + -- Class: symtable.Class + + A namespace of a class. This class inherits *note SymbolTable: + 3521. + + -- Method: get_methods () + + Return a tuple containing the names of methods declared in the + class. + + -- Class: symtable.Symbol + + An entry in a *note SymbolTable: 3521. corresponding to an + identifier in the source. The constructor is not public. + + -- Method: get_name () + + Return the symbol’s name. + + -- Method: is_referenced () + + Return ‘True’ if the symbol is used in its block. + + -- Method: is_imported () + + Return ‘True’ if the symbol is created from an import + statement. + + -- Method: is_parameter () + + Return ‘True’ if the symbol is a parameter. + + -- Method: is_global () + + Return ‘True’ if the symbol is global. + + -- Method: is_nonlocal () + + Return ‘True’ if the symbol is nonlocal. + + -- Method: is_declared_global () + + Return ‘True’ if the symbol is declared global with a global + statement. + + -- Method: is_local () + + Return ‘True’ if the symbol is local to its block. + + -- Method: is_annotated () + + Return ‘True’ if the symbol is annotated. + + New in version 3.6. + + -- Method: is_free () + + Return ‘True’ if the symbol is referenced in its block, but + not assigned to. + + -- Method: is_assigned () + + Return ‘True’ if the symbol is assigned to in its block. + + -- Method: is_namespace () + + Return ‘True’ if name binding introduces new namespace. + + If the name is used as the target of a function or class + statement, this will be true. + + For example: + + >>> table = symtable.symtable("def some_func(): pass", "string", "exec") + >>> table.lookup("some_func").is_namespace() + True + + Note that a single name can be bound to multiple objects. If + the result is ‘True’, the name may also be bound to other + objects, like an int or list, that does not introduce a new + namespace. + + -- Method: get_namespaces () + + Return a list of namespaces bound to this name. + + -- Method: get_namespace () + + Return the namespace bound to this name. If more than one + namespace is bound, *note ValueError: 1fb. is raised. + + +File: python.info, Node: symbol — Constants used with Python parse trees, Next: token — Constants used with Python parse trees, Prev: symtable — Access to the compiler’s symbol tables, Up: Python Language Services + +5.32.4 ‘symbol’ — Constants used with Python parse trees +-------------------------------------------------------- + +`Source code:' Lib/symbol.py(1) + +__________________________________________________________________ + +This module provides constants which represent the numeric values of +internal nodes of the parse tree. Unlike most Python constants, these +use lower-case names. Refer to the file ‘Grammar/Grammar’ in the Python +distribution for the definitions of the names in the context of the +language grammar. The specific numeric values which the names map to +may change between Python versions. + +This module also provides one additional data object: + + -- Data: symbol.sym_name + + Dictionary mapping the numeric values of the constants defined in + this module back to name strings, allowing more human-readable + representation of parse trees to be generated. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/symbol.py + + +File: python.info, Node: token — Constants used with Python parse trees, Next: keyword — Testing for Python keywords, Prev: symbol — Constants used with Python parse trees, Up: Python Language Services + +5.32.5 ‘token’ — Constants used with Python parse trees +------------------------------------------------------- + +`Source code:' Lib/token.py(1) + +__________________________________________________________________ + +This module provides constants which represent the numeric values of +leaf nodes of the parse tree (terminal tokens). Refer to the file +‘Grammar/Grammar’ in the Python distribution for the definitions of the +names in the context of the language grammar. The specific numeric +values which the names map to may change between Python versions. + +The module also provides a mapping from numeric codes to names and some +functions. The functions mirror definitions in the Python C header +files. + + -- Data: token.tok_name + + Dictionary mapping the numeric values of the constants defined in + this module back to name strings, allowing more human-readable + representation of parse trees to be generated. + + -- Function: token.ISTERMINAL (x) + + Return ‘True’ for terminal token values. + + -- Function: token.ISNONTERMINAL (x) + + Return ‘True’ for non-terminal token values. + + -- Function: token.ISEOF (x) + + Return ‘True’ if `x' is the marker indicating the end of input. + +The token constants are: + + -- Data: token.ENDMARKER + + -- Data: token.NAME + + -- Data: token.NUMBER + + -- Data: token.STRING + + -- Data: token.NEWLINE + + -- Data: token.INDENT + + -- Data: token.DEDENT + + -- Data: token.LPAR + + Token value for ‘"("’. + + -- Data: token.RPAR + + Token value for ‘")"’. + + -- Data: token.LSQB + + Token value for ‘"["’. + + -- Data: token.RSQB + + Token value for ‘"]"’. + + -- Data: token.COLON + + Token value for ‘":"’. + + -- Data: token.COMMA + + Token value for ‘","’. + + -- Data: token.SEMI + + Token value for ‘";"’. + + -- Data: token.PLUS + + Token value for ‘"+"’. + + -- Data: token.MINUS + + Token value for ‘"-"’. + + -- Data: token.STAR + + Token value for ‘"*"’. + + -- Data: token.SLASH + + Token value for ‘"/"’. + + -- Data: token.VBAR + + Token value for ‘"|"’. + + -- Data: token.AMPER + + Token value for ‘"&"’. + + -- Data: token.LESS + + Token value for ‘"<"’. + + -- Data: token.GREATER + + Token value for ‘">"’. + + -- Data: token.EQUAL + + Token value for ‘"="’. + + -- Data: token.DOT + + Token value for ‘"."’. + + -- Data: token.PERCENT + + Token value for ‘"%"’. + + -- Data: token.LBRACE + + Token value for ‘"{"’. + + -- Data: token.RBRACE + + Token value for ‘"}"’. + + -- Data: token.EQEQUAL + + Token value for ‘"=="’. + + -- Data: token.NOTEQUAL + + Token value for ‘"!="’. + + -- Data: token.LESSEQUAL + + Token value for ‘"<="’. + + -- Data: token.GREATEREQUAL + + Token value for ‘">="’. + + -- Data: token.TILDE + + Token value for ‘"~"’. + + -- Data: token.CIRCUMFLEX + + Token value for ‘"^"’. + + -- Data: token.LEFTSHIFT + + Token value for ‘"<<"’. + + -- Data: token.RIGHTSHIFT + + Token value for ‘">>"’. + + -- Data: token.DOUBLESTAR + + Token value for ‘"**"’. + + -- Data: token.PLUSEQUAL + + Token value for ‘"+="’. + + -- Data: token.MINEQUAL + + Token value for ‘"-="’. + + -- Data: token.STAREQUAL + + Token value for ‘"*="’. + + -- Data: token.SLASHEQUAL + + Token value for ‘"/="’. + + -- Data: token.PERCENTEQUAL + + Token value for ‘"%="’. + + -- Data: token.AMPEREQUAL + + Token value for ‘"&="’. + + -- Data: token.VBAREQUAL + + Token value for ‘"|="’. + + -- Data: token.CIRCUMFLEXEQUAL + + Token value for ‘"^="’. + + -- Data: token.LEFTSHIFTEQUAL + + Token value for ‘"<<="’. + + -- Data: token.RIGHTSHIFTEQUAL + + Token value for ‘">>="’. + + -- Data: token.DOUBLESTAREQUAL + + Token value for ‘"**="’. + + -- Data: token.DOUBLESLASH + + Token value for ‘"//"’. + + -- Data: token.DOUBLESLASHEQUAL + + Token value for ‘"//="’. + + -- Data: token.AT + + Token value for ‘"@"’. + + -- Data: token.ATEQUAL + + Token value for ‘"@="’. + + -- Data: token.RARROW + + Token value for ‘"->"’. + + -- Data: token.ELLIPSIS + + Token value for ‘"..."’. + + -- Data: token.COLONEQUAL + + Token value for ‘":="’. + + -- Data: token.OP + + -- Data: token.AWAIT + + -- Data: token.ASYNC + + -- Data: token.TYPE_IGNORE + + -- Data: token.TYPE_COMMENT + + -- Data: token.ERRORTOKEN + + -- Data: token.N_TOKENS + + -- Data: token.NT_OFFSET + +The following token type values aren’t used by the C tokenizer but are +needed for the *note tokenize: 111. module. + + -- Data: token.COMMENT + + Token value used to indicate a comment. + + -- Data: token.NL + + Token value used to indicate a non-terminating newline. The *note + NEWLINE: 3552. token indicates the end of a logical line of Python + code; ‘NL’ tokens are generated when a logical line of code is + continued over multiple physical lines. + + -- Data: token.ENCODING + + Token value that indicates the encoding used to decode the source + bytes into text. The first token returned by *note + tokenize.tokenize(): c5b. will always be an ‘ENCODING’ token. + + -- Data: token.TYPE_COMMENT + + Token value indicating that a type comment was recognized. Such + tokens are only produced when *note ast.parse(): 1a1. is invoked + with ‘type_comments=True’. + +Changed in version 3.5: Added *note AWAIT: 3585. and *note ASYNC: 3586. +tokens. + +Changed in version 3.7: Added *note COMMENT: 358c, *note NL: 358d. and +*note ENCODING: 358e. tokens. + +Changed in version 3.7: Removed *note AWAIT: 3585. and *note ASYNC: +3586. tokens. “async” and “await” are now tokenized as *note NAME: +354f. tokens. + +Changed in version 3.8: Added *note TYPE_COMMENT: 3588, *note +TYPE_IGNORE: 3587, *note COLONEQUAL: 3583. Added *note AWAIT: 3585. and +*note ASYNC: 3586. tokens back (they’re needed to support parsing older +Python versions for *note ast.parse(): 1a1. with ‘feature_version’ set +to 6 or lower). + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/token.py + + +File: python.info, Node: keyword — Testing for Python keywords, Next: tokenize — Tokenizer for Python source, Prev: token — Constants used with Python parse trees, Up: Python Language Services + +5.32.6 ‘keyword’ — Testing for Python keywords +---------------------------------------------- + +`Source code:' Lib/keyword.py(1) + +__________________________________________________________________ + +This module allows a Python program to determine if a string is a *note +keyword: 1079. + + -- Function: keyword.iskeyword (s) + + Return ‘True’ if `s' is a Python *note keyword: 1079. + + -- Data: keyword.kwlist + + Sequence containing all the *note keywords: 1079. defined for the + interpreter. If any keywords are defined to only be active when + particular *note __future__: 0. statements are in effect, these + will be included as well. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/keyword.py + + +File: python.info, Node: tokenize — Tokenizer for Python source, Next: tabnanny — Detection of ambiguous indentation, Prev: keyword — Testing for Python keywords, Up: Python Language Services + +5.32.7 ‘tokenize’ — Tokenizer for Python source +----------------------------------------------- + +`Source code:' Lib/tokenize.py(1) + +__________________________________________________________________ + +The *note tokenize: 111. module provides a lexical scanner for Python +source code, implemented in Python. The scanner in this module returns +comments as tokens as well, making it useful for implementing +“pretty-printers”, including colorizers for on-screen displays. + +To simplify token stream handling, all *note operator: 10b6. and *note +delimiter: 10b7. tokens and *note Ellipsis: 12b6. are returned using the +generic *note OP: 3584. token type. The exact type can be determined by +checking the ‘exact_type’ property on the *note named tuple: aa3. +returned from *note tokenize.tokenize(): c5b. + +* Menu: + +* Tokenizing Input:: +* Command-Line Usage: Command-Line Usage<2>. +* Examples: Examples<30>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/tokenize.py + + +File: python.info, Node: Tokenizing Input, Next: Command-Line Usage<2>, Up: tokenize — Tokenizer for Python source + +5.32.7.1 Tokenizing Input +......................... + +The primary entry point is a *note generator: 9a2.: + + -- Function: tokenize.tokenize (readline) + + The *note tokenize(): c5b. generator requires one argument, + `readline', which must be a callable object which provides the same + interface as the *note io.IOBase.readline(): 13ae. method of file + objects. Each call to the function should return one line of input + as bytes. + + The generator produces 5-tuples with these members: the token type; + the token string; a 2-tuple ‘(srow, scol)’ of ints specifying the + row and column where the token begins in the source; a 2-tuple + ‘(erow, ecol)’ of ints specifying the row and column where the + token ends in the source; and the line on which the token was + found. The line passed (the last tuple item) is the `physical' + line. The 5 tuple is returned as a *note named tuple: aa3. with + the field names: ‘type string start end line’. + + The returned *note named tuple: aa3. has an additional property + named ‘exact_type’ that contains the exact operator type for *note + OP: 3584. tokens. For all other token types ‘exact_type’ equals + the named tuple ‘type’ field. + + Changed in version 3.1: Added support for named tuples. + + Changed in version 3.3: Added support for ‘exact_type’. + + *note tokenize(): c5b. determines the source encoding of the file + by looking for a UTF-8 BOM or encoding cookie, according to PEP + 263(1). + + -- Function: tokenize.generate_tokens (readline) + + Tokenize a source reading unicode strings instead of bytes. + + Like *note tokenize(): c5b, the `readline' argument is a callable + returning a single line of input. However, *note + generate_tokens(): 3595. expects `readline' to return a str object + rather than bytes. + + The result is an iterator yielding named tuples, exactly like *note + tokenize(): c5b. It does not yield an *note ENCODING: 358e. token. + +All constants from the *note token: 110. module are also exported from +*note tokenize: 111. + +Another function is provided to reverse the tokenization process. This +is useful for creating tools that tokenize a script, modify the token +stream, and write back the modified script. + + -- Function: tokenize.untokenize (iterable) + + Converts tokens back into Python source code. The `iterable' must + return sequences with at least two elements, the token type and the + token string. Any additional sequence elements are ignored. + + The reconstructed script is returned as a single string. The + result is guaranteed to tokenize back to match the input so that + the conversion is lossless and round-trips are assured. The + guarantee applies only to the token type and token string as the + spacing between tokens (column positions) may change. + + It returns bytes, encoded using the *note ENCODING: 358e. token, + which is the first token sequence output by *note tokenize(): c5b. + If there is no encoding token in the input, it returns a str + instead. + +*note tokenize(): c5b. needs to detect the encoding of source files it +tokenizes. The function it uses to do this is available: + + -- Function: tokenize.detect_encoding (readline) + + The *note detect_encoding(): 1940. function is used to detect the + encoding that should be used to decode a Python source file. It + requires one argument, readline, in the same way as the *note + tokenize(): c5b. generator. + + It will call readline a maximum of twice, and return the encoding + used (as a string) and a list of any lines (not decoded from bytes) + it has read in. + + It detects the encoding from the presence of a UTF-8 BOM or an + encoding cookie as specified in PEP 263(2). If both a BOM and a + cookie are present, but disagree, a *note SyntaxError: 458. will be + raised. Note that if the BOM is found, ‘'utf-8-sig'’ will be + returned as an encoding. + + If no encoding is specified, then the default of ‘'utf-8'’ will be + returned. + + Use *note open(): 193f. to open Python source files: it uses *note + detect_encoding(): 1940. to detect the file encoding. + + -- Function: tokenize.open (filename) + + Open a file in read only mode using the encoding detected by *note + detect_encoding(): 1940. + + New in version 3.2. + + -- Exception: tokenize.TokenError + + Raised when either a docstring or expression that may be split over + several lines is not completed anywhere in the file, for example: + + """Beginning of + docstring + + or: + + [1, + 2, + 3 + +Note that unclosed single-quoted strings do not cause an error to be +raised. They are tokenized as *note ERRORTOKEN: 3589, followed by the +tokenization of their contents. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0263 + + (2) https://www.python.org/dev/peps/pep-0263 + + +File: python.info, Node: Command-Line Usage<2>, Next: Examples<30>, Prev: Tokenizing Input, Up: tokenize — Tokenizer for Python source + +5.32.7.2 Command-Line Usage +........................... + +New in version 3.3. + +The *note tokenize: 111. module can be executed as a script from the +command line. It is as simple as: + + python -m tokenize [-e] [filename.py] + +The following options are accepted: + + -- Program Option: -h, --help + + show this help message and exit + + -- Program Option: -e, --exact + + display token names using the exact type + +If ‘filename.py’ is specified its contents are tokenized to stdout. +Otherwise, tokenization is performed on stdin. + + +File: python.info, Node: Examples<30>, Prev: Command-Line Usage<2>, Up: tokenize — Tokenizer for Python source + +5.32.7.3 Examples +................. + +Example of a script rewriter that transforms float literals into Decimal +objects: + + from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP + from io import BytesIO + + def decistmt(s): + """Substitute Decimals for floats in a string of statements. + + >>> from decimal import Decimal + >>> s = 'print(+21.3e-5*-.1234/81.7)' + >>> decistmt(s) + "print (+Decimal ('21.3e-5')*-Decimal ('.1234')/Decimal ('81.7'))" + + The format of the exponent is inherited from the platform C library. + Known cases are "e-007" (Windows) and "e-07" (not Windows). Since + we're only showing 12 digits, and the 13th isn't close to 5, the + rest of the output should be platform-independent. + + >>> exec(s) #doctest: +ELLIPSIS + -3.21716034272e-0...7 + + Output from calculations with Decimal should be identical across all + platforms. + + >>> exec(decistmt(s)) + -3.217160342717258261933904529E-7 + """ + result = [] + g = tokenize(BytesIO(s.encode('utf-8')).readline) # tokenize the string + for toknum, tokval, _, _, _ in g: + if toknum == NUMBER and '.' in tokval: # replace NUMBER tokens + result.extend([ + (NAME, 'Decimal'), + (OP, '('), + (STRING, repr(tokval)), + (OP, ')') + ]) + else: + result.append((toknum, tokval)) + return untokenize(result).decode('utf-8') + +Example of tokenizing from the command line. The script: + + def say_hello(): + print("Hello, World!") + + say_hello() + +will be tokenized to the following output where the first column is the +range of the line/column coordinates where the token is found, the +second column is the name of the token, and the final column is the +value of the token (if any) + + $ python -m tokenize hello.py + 0,0-0,0: ENCODING 'utf-8' + 1,0-1,3: NAME 'def' + 1,4-1,13: NAME 'say_hello' + 1,13-1,14: OP '(' + 1,14-1,15: OP ')' + 1,15-1,16: OP ':' + 1,16-1,17: NEWLINE '\n' + 2,0-2,4: INDENT ' ' + 2,4-2,9: NAME 'print' + 2,9-2,10: OP '(' + 2,10-2,25: STRING '"Hello, World!"' + 2,25-2,26: OP ')' + 2,26-2,27: NEWLINE '\n' + 3,0-3,1: NL '\n' + 4,0-4,0: DEDENT '' + 4,0-4,9: NAME 'say_hello' + 4,9-4,10: OP '(' + 4,10-4,11: OP ')' + 4,11-4,12: NEWLINE '\n' + 5,0-5,0: ENDMARKER '' + +The exact token type names can be displayed using the *note -e: 359b. +option: + + $ python -m tokenize -e hello.py + 0,0-0,0: ENCODING 'utf-8' + 1,0-1,3: NAME 'def' + 1,4-1,13: NAME 'say_hello' + 1,13-1,14: LPAR '(' + 1,14-1,15: RPAR ')' + 1,15-1,16: COLON ':' + 1,16-1,17: NEWLINE '\n' + 2,0-2,4: INDENT ' ' + 2,4-2,9: NAME 'print' + 2,9-2,10: LPAR '(' + 2,10-2,25: STRING '"Hello, World!"' + 2,25-2,26: RPAR ')' + 2,26-2,27: NEWLINE '\n' + 3,0-3,1: NL '\n' + 4,0-4,0: DEDENT '' + 4,0-4,9: NAME 'say_hello' + 4,9-4,10: LPAR '(' + 4,10-4,11: RPAR ')' + 4,11-4,12: NEWLINE '\n' + 5,0-5,0: ENDMARKER '' + +Example of tokenizing a file programmatically, reading unicode strings +instead of bytes with *note generate_tokens(): 3595.: + + import tokenize + + with tokenize.open('hello.py') as f: + tokens = tokenize.generate_tokens(f.readline) + for token in tokens: + print(token) + +Or reading bytes directly with *note tokenize(): c5b.: + + import tokenize + + with open('hello.py', 'rb') as f: + tokens = tokenize.tokenize(f.readline) + for token in tokens: + print(token) + + +File: python.info, Node: tabnanny — Detection of ambiguous indentation, Next: pyclbr — Python module browser support, Prev: tokenize — Tokenizer for Python source, Up: Python Language Services + +5.32.8 ‘tabnanny’ — Detection of ambiguous indentation +------------------------------------------------------ + +`Source code:' Lib/tabnanny.py(1) + +__________________________________________________________________ + +For the time being this module is intended to be called as a script. +However it is possible to import it into an IDE and use the function +*note check(): 359f. described below. + + Note: The API provided by this module is likely to change in future + releases; such changes may not be backward compatible. + + -- Function: tabnanny.check (file_or_dir) + + If `file_or_dir' is a directory and not a symbolic link, then + recursively descend the directory tree named by `file_or_dir', + checking all ‘.py’ files along the way. If `file_or_dir' is an + ordinary Python source file, it is checked for whitespace related + problems. The diagnostic messages are written to standard output + using the *note print(): 886. function. + + -- Data: tabnanny.verbose + + Flag indicating whether to print verbose messages. This is + incremented by the ‘-v’ option if called as a script. + + -- Data: tabnanny.filename_only + + Flag indicating whether to print only the filenames of files + containing whitespace related problems. This is set to true by the + ‘-q’ option if called as a script. + + -- Exception: tabnanny.NannyNag + + Raised by *note process_tokens(): 35a3. if detecting an ambiguous + indent. Captured and handled in *note check(): 359f. + + -- Function: tabnanny.process_tokens (tokens) + + This function is used by *note check(): 359f. to process tokens + generated by the *note tokenize: 111. module. + +See also +........ + +Module *note tokenize: 111. + + Lexical scanner for Python source code. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/tabnanny.py + + +File: python.info, Node: pyclbr — Python module browser support, Next: py_compile — Compile Python source files, Prev: tabnanny — Detection of ambiguous indentation, Up: Python Language Services + +5.32.9 ‘pyclbr’ — Python module browser support +----------------------------------------------- + +`Source code:' Lib/pyclbr.py(1) + +__________________________________________________________________ + +The *note pyclbr: d8. module provides limited information about the +functions, classes, and methods defined in a Python-coded module. The +information is sufficient to implement a module browser. The +information is extracted from the Python source code rather than by +importing the module, so this module is safe to use with untrusted code. +This restriction makes it impossible to use this module with modules not +implemented in Python, including all standard and optional extension +modules. + + -- Function: pyclbr.readmodule (module, path=None) + + Return a dictionary mapping module-level class names to class + descriptors. If possible, descriptors for imported base classes + are included. Parameter `module' is a string with the name of the + module to read; it may be the name of a module within a package. + If given, `path' is a sequence of directory paths prepended to + ‘sys.path’, which is used to locate the module source code. + + This function is the original interface and is only kept for back + compatibility. It returns a filtered version of the following. + + -- Function: pyclbr.readmodule_ex (module, path=None) + + Return a dictionary-based tree containing a function or class + descriptors for each function and class defined in the module with + a ‘def’ or ‘class’ statement. The returned dictionary maps + module-level function and class names to their descriptors. Nested + objects are entered into the children dictionary of their parent. + As with readmodule, `module' names the module to be read and `path' + is prepended to sys.path. If the module being read is a package, + the returned dictionary has a key ‘'__path__'’ whose value is a + list containing the package search path. + +New in version 3.7: Descriptors for nested definitions. They are +accessed through the new children attribute. Each has a new parent +attribute. + +The descriptors returned by these functions are instances of Function +and Class classes. Users are not expected to create instances of these +classes. + +* Menu: + +* Function Objects:: +* Class Objects: Class Objects<2>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pyclbr.py + + +File: python.info, Node: Function Objects, Next: Class Objects<2>, Up: pyclbr — Python module browser support + +5.32.9.1 Function Objects +......................... + +Class ‘Function’ instances describe functions defined by def statements. +They have the following attributes: + + -- Attribute: Function.file + + Name of the file in which the function is defined. + + -- Attribute: Function.module + + The name of the module defining the function described. + + -- Attribute: Function.name + + The name of the function. + + -- Attribute: Function.lineno + + The line number in the file where the definition starts. + + -- Attribute: Function.parent + + For top-level functions, None. For nested functions, the parent. + + New in version 3.7. + + -- Attribute: Function.children + + A dictionary mapping names to descriptors for nested functions and + classes. + + New in version 3.7. + + +File: python.info, Node: Class Objects<2>, Prev: Function Objects, Up: pyclbr — Python module browser support + +5.32.9.2 Class Objects +...................... + +Class ‘Class’ instances describe classes defined by class statements. +They have the same attributes as Functions and two more. + + -- Attribute: Class.file + + Name of the file in which the class is defined. + + -- Attribute: Class.module + + The name of the module defining the class described. + + -- Attribute: Class.name + + The name of the class. + + -- Attribute: Class.lineno + + The line number in the file where the definition starts. + + -- Attribute: Class.parent + + For top-level classes, None. For nested classes, the parent. + + New in version 3.7. + + -- Attribute: Class.children + + A dictionary mapping names to descriptors for nested functions and + classes. + + New in version 3.7. + + -- Attribute: Class.super + + A list of ‘Class’ objects which describe the immediate base classes + of the class being described. Classes which are named as + superclasses but which are not discoverable by *note + readmodule_ex(): 35a7. are listed as a string with the class name + instead of as ‘Class’ objects. + + -- Attribute: Class.methods + + A dictionary mapping method names to line numbers. This can be + derived from the newer children dictionary, but remains for + back-compatibility. + + +File: python.info, Node: py_compile — Compile Python source files, Next: compileall — Byte-compile Python libraries, Prev: pyclbr — Python module browser support, Up: Python Language Services + +5.32.10 ‘py_compile’ — Compile Python source files +-------------------------------------------------- + +`Source code:' Lib/py_compile.py(1) + +__________________________________________________________________ + +The *note py_compile: d7. module provides a function to generate a +byte-code file from a source file, and another function used when the +module source file is invoked as a script. + +Though not often needed, this function can be useful when installing +modules for shared use, especially if some of the users may not have +permission to write the byte-code cache files in the directory +containing the source code. + + -- Exception: py_compile.PyCompileError + + Exception raised when an error occurs while attempting to compile + the file. + + -- Function: py_compile.compile (file, cfile=None, dfile=None, + doraise=False, optimize=-1, + invalidation_mode=PycInvalidationMode.TIMESTAMP, quiet=0) + + Compile a source file to byte-code and write out the byte-code + cache file. The source code is loaded from the file named `file'. + The byte-code is written to `cfile', which defaults to the PEP + 3147(2)/ PEP 488(3) path, ending in ‘.pyc’. For example, if `file' + is ‘/foo/bar/baz.py’ `cfile' will default to + ‘/foo/bar/__pycache__/baz.cpython-32.pyc’ for Python 3.2. If + `dfile' is specified, it is used as the name of the source file in + error messages instead of `file'. If `doraise' is true, a *note + PyCompileError: 35bc. is raised when an error is encountered while + compiling `file'. If `doraise' is false (the default), an error + string is written to ‘sys.stderr’, but no exception is raised. + This function returns the path to byte-compiled file, i.e. + whatever `cfile' value was used. + + The `doraise' and `quiet' arguments determine how errors are + handled while compiling file. If `quiet' is 0 or 1, and `doraise' + is false, the default behaviour is enabled: an error string is + written to ‘sys.stderr’, and the function returns ‘None’ instead of + a path. If `doraise' is true, a *note PyCompileError: 35bc. is + raised instead. However if `quiet' is 2, no message is written, + and `doraise' has no effect. + + If the path that `cfile' becomes (either explicitly specified or + computed) is a symlink or non-regular file, *note FileExistsError: + 958. will be raised. This is to act as a warning that import will + turn those paths into regular files if it is allowed to write + byte-compiled files to those paths. This is a side-effect of + import using file renaming to place the final byte-compiled file + into place to prevent concurrent file writing issues. + + `optimize' controls the optimization level and is passed to the + built-in *note compile(): 1b4. function. The default of ‘-1’ + selects the optimization level of the current interpreter. + + `invalidation_mode' should be a member of the *note + PycInvalidationMode: 35bd. enum and controls how the generated + bytecode cache is invalidated at runtime. The default is *note + PycInvalidationMode.CHECKED_HASH: 35be. if the ‘SOURCE_DATE_EPOCH’ + environment variable is set, otherwise the default is *note + PycInvalidationMode.TIMESTAMP: 35bf. + + Changed in version 3.2: Changed default value of `cfile' to be PEP + 3147(4)-compliant. Previous default was `file' + ‘'c'’ (‘'o'’ if + optimization was enabled). Also added the `optimize' parameter. + + Changed in version 3.4: Changed code to use *note importlib: 9b. + for the byte-code cache file writing. This means file + creation/writing semantics now match what *note importlib: 9b. + does, e.g. permissions, write-and-move semantics, etc. Also added + the caveat that *note FileExistsError: 958. is raised if `cfile' is + a symlink or non-regular file. + + Changed in version 3.7: The `invalidation_mode' parameter was added + as specified in PEP 552(5). If the ‘SOURCE_DATE_EPOCH’ environment + variable is set, `invalidation_mode' will be forced to *note + PycInvalidationMode.CHECKED_HASH: 35be. + + Changed in version 3.7.2: The ‘SOURCE_DATE_EPOCH’ environment + variable no longer overrides the value of the `invalidation_mode' + argument, and determines its default value instead. + + Changed in version 3.8: The `quiet' parameter was added. + + -- Class: py_compile.PycInvalidationMode + + A enumeration of possible methods the interpreter can use to + determine whether a bytecode file is up to date with a source file. + The ‘.pyc’ file indicates the desired invalidation mode in its + header. See *note Cached bytecode invalidation: 329. for more + information on how Python invalidates ‘.pyc’ files at runtime. + + New in version 3.7. + + -- Attribute: TIMESTAMP + + The ‘.pyc’ file includes the timestamp and size of the source + file, which Python will compare against the metadata of the + source file at runtime to determine if the ‘.pyc’ file needs + to be regenerated. + + -- Attribute: CHECKED_HASH + + The ‘.pyc’ file includes a hash of the source file content, + which Python will compare against the source at runtime to + determine if the ‘.pyc’ file needs to be regenerated. + + -- Attribute: UNCHECKED_HASH + + Like *note CHECKED_HASH: 35be, the ‘.pyc’ file includes a hash + of the source file content. However, Python will at runtime + assume the ‘.pyc’ file is up to date and not validate the + ‘.pyc’ against the source file at all. + + This option is useful when the ‘.pycs’ are kept up to date by + some system external to Python like a build system. + + -- Function: py_compile.main (args=None) + + Compile several source files. The files named in `args' (or on the + command line, if `args' is ‘None’) are compiled and the resulting + byte-code is cached in the normal manner. This function does not + search a directory structure to locate source files; it only + compiles files named explicitly. If ‘'-'’ is the only parameter in + args, the list of files is taken from standard input. + + Changed in version 3.2: Added support for ‘'-'’. + +When this module is run as a script, the *note main(): 35c1. is used to +compile all the files named on the command line. The exit status is +nonzero if one of the files could not be compiled. + +See also +........ + +Module *note compileall: 21. + + Utilities to compile all Python source files in a directory tree. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/py_compile.py + + (2) https://www.python.org/dev/peps/pep-3147 + + (3) https://www.python.org/dev/peps/pep-0488 + + (4) https://www.python.org/dev/peps/pep-3147 + + (5) https://www.python.org/dev/peps/pep-0552 + + +File: python.info, Node: compileall — Byte-compile Python libraries, Next: dis — Disassembler for Python bytecode, Prev: py_compile — Compile Python source files, Up: Python Language Services + +5.32.11 ‘compileall’ — Byte-compile Python libraries +---------------------------------------------------- + +`Source code:' Lib/compileall.py(1) + +__________________________________________________________________ + +This module provides some utility functions to support installing Python +libraries. These functions compile Python source files in a directory +tree. This module can be used to create the cached byte-code files at +library installation time, which makes them available for use even by +users who don’t have write permission to the library directories. + +* Menu: + +* Command-line use:: +* Public functions:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/compileall.py + + +File: python.info, Node: Command-line use, Next: Public functions, Up: compileall — Byte-compile Python libraries + +5.32.11.1 Command-line use +.......................... + +This module can work as a script (using ‘python -m compileall’) to +compile Python sources. + + -- Program Option: directory ... + -- Program Option: file ... + + Positional arguments are files to compile or directories that + contain source files, traversed recursively. If no argument is + given, behave as if the command line was ‘-l <directories from + sys.path>’. + + -- Program Option: -l + + Do not recurse into subdirectories, only compile source code files + directly contained in the named or implied directories. + + -- Program Option: -f + + Force rebuild even if timestamps are up-to-date. + + -- Program Option: -q + + Do not print the list of files compiled. If passed once, error + messages will still be printed. If passed twice (‘-qq’), all + output is suppressed. + + -- Program Option: -d destdir + + Directory prepended to the path to each file being compiled. This + will appear in compilation time tracebacks, and is also compiled in + to the byte-code file, where it will be used in tracebacks and + other messages in cases where the source file does not exist at the + time the byte-code file is executed. + + -- Program Option: -x regex + + regex is used to search the full path to each file considered for + compilation, and if the regex produces a match, the file is + skipped. + + -- Program Option: -i list + + Read the file ‘list’ and add each line that it contains to the list + of files and directories to compile. If ‘list’ is ‘-’, read lines + from ‘stdin’. + + -- Program Option: -b + + Write the byte-code files to their legacy locations and names, + which may overwrite byte-code files created by another version of + Python. The default is to write files to their PEP 3147(1) + locations and names, which allows byte-code files from multiple + versions of Python to coexist. + + -- Program Option: -r + + Control the maximum recursion level for subdirectories. If this is + given, then ‘-l’ option will not be taken into account. ‘python -m + compileall <directory> -r 0’ is equivalent to ‘python -m compileall + <directory> -l’. + + -- Program Option: -j N + + Use `N' workers to compile the files within the given directory. + If ‘0’ is used, then the result of *note os.cpu_count(): 87f. will + be used. + + -- Program Option: --invalidation-mode + [timestamp|checked-hash|unchecked-hash] + + Control how the generated byte-code files are invalidated at + runtime. The ‘timestamp’ value, means that ‘.pyc’ files with the + source timestamp and size embedded will be generated. The + ‘checked-hash’ and ‘unchecked-hash’ values cause hash-based pycs to + be generated. Hash-based pycs embed a hash of the source file + contents rather than a timestamp. See *note Cached bytecode + invalidation: 329. for more information on how Python validates + bytecode cache files at runtime. The default is ‘timestamp’ if the + ‘SOURCE_DATE_EPOCH’ environment variable is not set, and + ‘checked-hash’ if the ‘SOURCE_DATE_EPOCH’ environment variable is + set. + +Changed in version 3.2: Added the ‘-i’, ‘-b’ and ‘-h’ options. + +Changed in version 3.5: Added the ‘-j’, ‘-r’, and ‘-qq’ options. ‘-q’ +option was changed to a multilevel value. ‘-b’ will always produce a +byte-code file ending in ‘.pyc’, never ‘.pyo’. + +Changed in version 3.7: Added the ‘--invalidation-mode’ option. + +There is no command-line option to control the optimization level used +by the *note compile(): 1b4. function, because the Python interpreter +itself already provides the option: ‘python -O -m compileall’. + +Similarly, the *note compile(): 1b4. function respects the *note +sys.pycache_prefix: 156. setting. The generated bytecode cache will +only be useful if *note compile(): 1b4. is run with the same *note +sys.pycache_prefix: 156. (if any) that will be used at runtime. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: Public functions, Prev: Command-line use, Up: compileall — Byte-compile Python libraries + +5.32.11.2 Public functions +.......................... + + -- Function: compileall.compile_dir (dir, maxlevels=10, ddir=None, + force=False, rx=None, quiet=0, legacy=False, optimize=-1, + workers=1, invalidation_mode=None) + + Recursively descend the directory tree named by `dir', compiling + all ‘.py’ files along the way. Return a true value if all the + files compiled successfully, and a false value otherwise. + + The `maxlevels' parameter is used to limit the depth of the + recursion; it defaults to ‘10’. + + If `ddir' is given, it is prepended to the path to each file being + compiled for use in compilation time tracebacks, and is also + compiled in to the byte-code file, where it will be used in + tracebacks and other messages in cases where the source file does + not exist at the time the byte-code file is executed. + + If `force' is true, modules are re-compiled even if the timestamps + are up to date. + + If `rx' is given, its search method is called on the complete path + to each file considered for compilation, and if it returns a true + value, the file is skipped. + + If `quiet' is ‘False’ or ‘0’ (the default), the filenames and other + information are printed to standard out. Set to ‘1’, only errors + are printed. Set to ‘2’, all output is suppressed. + + If `legacy' is true, byte-code files are written to their legacy + locations and names, which may overwrite byte-code files created by + another version of Python. The default is to write files to their + PEP 3147(1) locations and names, which allows byte-code files from + multiple versions of Python to coexist. + + `optimize' specifies the optimization level for the compiler. It + is passed to the built-in *note compile(): 1b4. function. + + The argument `workers' specifies how many workers are used to + compile files in parallel. The default is to not use multiple + workers. If the platform can’t use multiple workers and `workers' + argument is given, then sequential compilation will be used as a + fallback. If `workers' is 0, the number of cores in the system is + used. If `workers' is lower than ‘0’, a *note ValueError: 1fb. + will be raised. + + `invalidation_mode' should be a member of the *note + py_compile.PycInvalidationMode: 35bd. enum and controls how the + generated pycs are invalidated at runtime. + + Changed in version 3.2: Added the `legacy' and `optimize' + parameter. + + Changed in version 3.5: Added the `workers' parameter. + + Changed in version 3.5: `quiet' parameter was changed to a + multilevel value. + + Changed in version 3.5: The `legacy' parameter only writes out + ‘.pyc’ files, not ‘.pyo’ files no matter what the value of + `optimize' is. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + + Changed in version 3.7: The `invalidation_mode' parameter was + added. + + Changed in version 3.7.2: The `invalidation_mode' parameter’s + default value is updated to None. + + Changed in version 3.8: Setting `workers' to 0 now chooses the + optimal number of cores. + + -- Function: compileall.compile_file (fullname, ddir=None, force=False, + rx=None, quiet=0, legacy=False, optimize=-1, + invalidation_mode=None) + + Compile the file with path `fullname'. Return a true value if the + file compiled successfully, and a false value otherwise. + + If `ddir' is given, it is prepended to the path to the file being + compiled for use in compilation time tracebacks, and is also + compiled in to the byte-code file, where it will be used in + tracebacks and other messages in cases where the source file does + not exist at the time the byte-code file is executed. + + If `rx' is given, its search method is passed the full path name to + the file being compiled, and if it returns a true value, the file + is not compiled and ‘True’ is returned. + + If `quiet' is ‘False’ or ‘0’ (the default), the filenames and other + information are printed to standard out. Set to ‘1’, only errors + are printed. Set to ‘2’, all output is suppressed. + + If `legacy' is true, byte-code files are written to their legacy + locations and names, which may overwrite byte-code files created by + another version of Python. The default is to write files to their + PEP 3147(2) locations and names, which allows byte-code files from + multiple versions of Python to coexist. + + `optimize' specifies the optimization level for the compiler. It + is passed to the built-in *note compile(): 1b4. function. + + `invalidation_mode' should be a member of the *note + py_compile.PycInvalidationMode: 35bd. enum and controls how the + generated pycs are invalidated at runtime. + + New in version 3.2. + + Changed in version 3.5: `quiet' parameter was changed to a + multilevel value. + + Changed in version 3.5: The `legacy' parameter only writes out + ‘.pyc’ files, not ‘.pyo’ files no matter what the value of + `optimize' is. + + Changed in version 3.7: The `invalidation_mode' parameter was + added. + + Changed in version 3.7.2: The `invalidation_mode' parameter’s + default value is updated to None. + + -- Function: compileall.compile_path (skip_curdir=True, maxlevels=0, + force=False, quiet=0, legacy=False, optimize=-1, + invalidation_mode=None) + + Byte-compile all the ‘.py’ files found along ‘sys.path’. Return a + true value if all the files compiled successfully, and a false + value otherwise. + + If `skip_curdir' is true (the default), the current directory is + not included in the search. All other parameters are passed to the + *note compile_dir(): 372. function. Note that unlike the other + compile functions, ‘maxlevels’ defaults to ‘0’. + + Changed in version 3.2: Added the `legacy' and `optimize' + parameter. + + Changed in version 3.5: `quiet' parameter was changed to a + multilevel value. + + Changed in version 3.5: The `legacy' parameter only writes out + ‘.pyc’ files, not ‘.pyo’ files no matter what the value of + `optimize' is. + + Changed in version 3.7: The `invalidation_mode' parameter was + added. + + Changed in version 3.7.2: The `invalidation_mode' parameter’s + default value is updated to None. + +To force a recompile of all the ‘.py’ files in the ‘Lib/’ subdirectory +and all its subdirectories: + + import compileall + + compileall.compile_dir('Lib/', force=True) + + # Perform same compilation, excluding files in .svn directories. + import re + compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) + + # pathlib.Path objects can also be used. + import pathlib + compileall.compile_dir(pathlib.Path('Lib/'), force=True) + +See also +........ + +Module *note py_compile: d7. + + Byte-compile a single source file. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + (2) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: dis — Disassembler for Python bytecode, Next: pickletools — Tools for pickle developers, Prev: compileall — Byte-compile Python libraries, Up: Python Language Services + +5.32.12 ‘dis’ — Disassembler for Python bytecode +------------------------------------------------ + +`Source code:' Lib/dis.py(1) + +__________________________________________________________________ + +The *note dis: 38. module supports the analysis of CPython *note +bytecode: 614. by disassembling it. The CPython bytecode which this +module takes as an input is defined in the file ‘Include/opcode.h’ and +used by the compiler and the interpreter. + +`CPython implementation detail:' Bytecode is an implementation detail of +the CPython interpreter. No guarantees are made that bytecode will not +be added, removed, or changed between versions of Python. Use of this +module should not be considered to work across Python VMs or Python +releases. + +Changed in version 3.6: Use 2 bytes for each instruction. Previously +the number of bytes varied by instruction. + +Example: Given the function ‘myfunc()’: + + def myfunc(alist): + return len(alist) + +the following command can be used to display the disassembly of +‘myfunc()’: + + >>> dis.dis(myfunc) + 2 0 LOAD_GLOBAL 0 (len) + 2 LOAD_FAST 0 (alist) + 4 CALL_FUNCTION 1 + 6 RETURN_VALUE + +(The “2” is a line number). + +* Menu: + +* Bytecode analysis:: +* Analysis functions:: +* Python Bytecode Instructions:: +* Opcode collections:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/dis.py + + +File: python.info, Node: Bytecode analysis, Next: Analysis functions, Up: dis — Disassembler for Python bytecode + +5.32.12.1 Bytecode analysis +........................... + +New in version 3.4. + +The bytecode analysis API allows pieces of Python code to be wrapped in +a *note Bytecode: 837. object that provides easy access to details of +the compiled code. + + -- Class: dis.Bytecode (x, *, first_line=None, current_offset=None) + + Analyse the bytecode corresponding to a function, generator, + asynchronous generator, coroutine, method, string of source code, + or a code object (as returned by *note compile(): 1b4.). + + This is a convenience wrapper around many of the functions listed + below, most notably *note get_instructions(): 836, as iterating + over a *note Bytecode: 837. instance yields the bytecode operations + as *note Instruction: 835. instances. + + If `first_line' is not ‘None’, it indicates the line number that + should be reported for the first source line in the disassembled + code. Otherwise, the source line information (if any) is taken + directly from the disassembled code object. + + If `current_offset' is not ‘None’, it refers to an instruction + offset in the disassembled code. Setting this means *note dis(): + 838. will display a “current instruction” marker against the + specified opcode. + + -- Method: classmethod from_traceback (tb) + + Construct a *note Bytecode: 837. instance from the given + traceback, setting `current_offset' to the instruction + responsible for the exception. + + -- Data: codeobj + + The compiled code object. + + -- Data: first_line + + The first source line of the code object (if available) + + -- Method: dis () + + Return a formatted view of the bytecode operations (the same + as printed by *note dis.dis(): 384, but returned as a + multi-line string). + + -- Method: info () + + Return a formatted multi-line string with detailed information + about the code object, like *note code_info(): bce. + + Changed in version 3.7: This can now handle coroutine and + asynchronous generator objects. + +Example: + + >>> bytecode = dis.Bytecode(myfunc) + >>> for instr in bytecode: + ... print(instr.opname) + ... + LOAD_GLOBAL + LOAD_FAST + CALL_FUNCTION + RETURN_VALUE + + +File: python.info, Node: Analysis functions, Next: Python Bytecode Instructions, Prev: Bytecode analysis, Up: dis — Disassembler for Python bytecode + +5.32.12.2 Analysis functions +............................ + +The *note dis: 38. module also defines the following analysis functions +that convert the input directly to the desired output. They can be +useful if only a single operation is being performed, so the +intermediate analysis object isn’t useful: + + -- Function: dis.code_info (x) + + Return a formatted multi-line string with detailed code object + information for the supplied function, generator, asynchronous + generator, coroutine, method, source code string or code object. + + Note that the exact contents of code info strings are highly + implementation dependent and they may change arbitrarily across + Python VMs or Python releases. + + New in version 3.2. + + Changed in version 3.7: This can now handle coroutine and + asynchronous generator objects. + + -- Function: dis.show_code (x, *, file=None) + + Print detailed code object information for the supplied function, + method, source code string or code object to `file' (or + ‘sys.stdout’ if `file' is not specified). + + This is a convenient shorthand for ‘print(code_info(x), + file=file)’, intended for interactive exploration at the + interpreter prompt. + + New in version 3.2. + + Changed in version 3.4: Added `file' parameter. + + -- Function: dis.dis (x=None, *, file=None, depth=None) + + Disassemble the `x' object. `x' can denote either a module, a + class, a method, a function, a generator, an asynchronous + generator, a coroutine, a code object, a string of source code or a + byte sequence of raw bytecode. For a module, it disassembles all + functions. For a class, it disassembles all methods (including + class and static methods). For a code object or sequence of raw + bytecode, it prints one line per bytecode instruction. It also + recursively disassembles nested code objects (the code of + comprehensions, generator expressions and nested functions, and the + code used for building nested classes). Strings are first compiled + to code objects with the *note compile(): 1b4. built-in function + before being disassembled. If no object is provided, this function + disassembles the last traceback. + + The disassembly is written as text to the supplied `file' argument + if provided and to ‘sys.stdout’ otherwise. + + The maximal depth of recursion is limited by `depth' unless it is + ‘None’. ‘depth=0’ means no recursion. + + Changed in version 3.4: Added `file' parameter. + + Changed in version 3.7: Implemented recursive disassembling and + added `depth' parameter. + + Changed in version 3.7: This can now handle coroutine and + asynchronous generator objects. + + -- Function: dis.distb (tb=None, *, file=None) + + Disassemble the top-of-stack function of a traceback, using the + last traceback if none was passed. The instruction causing the + exception is indicated. + + The disassembly is written as text to the supplied `file' argument + if provided and to ‘sys.stdout’ otherwise. + + Changed in version 3.4: Added `file' parameter. + + -- Function: dis.disassemble (code, lasti=-1, *, file=None) + -- Function: dis.disco (code, lasti=-1, *, file=None) + + Disassemble a code object, indicating the last instruction if + `lasti' was provided. The output is divided in the following + columns: + + 1. the line number, for the first instruction of each line + + 2. the current instruction, indicated as ‘-->’, + + 3. a labelled instruction, indicated with ‘>>’, + + 4. the address of the instruction, + + 5. the operation code name, + + 6. operation parameters, and + + 7. interpretation of the parameters in parentheses. + + The parameter interpretation recognizes local and global variable + names, constant values, branch targets, and compare operators. + + The disassembly is written as text to the supplied `file' argument + if provided and to ‘sys.stdout’ otherwise. + + Changed in version 3.4: Added `file' parameter. + + -- Function: dis.get_instructions (x, *, first_line=None) + + Return an iterator over the instructions in the supplied function, + method, source code string or code object. + + The iterator generates a series of *note Instruction: 835. named + tuples giving the details of each operation in the supplied code. + + If `first_line' is not ‘None’, it indicates the line number that + should be reported for the first source line in the disassembled + code. Otherwise, the source line information (if any) is taken + directly from the disassembled code object. + + New in version 3.4. + + -- Function: dis.findlinestarts (code) + + This generator function uses the ‘co_firstlineno’ and ‘co_lnotab’ + attributes of the code object `code' to find the offsets which are + starts of lines in the source code. They are generated as + ‘(offset, lineno)’ pairs. See Objects/lnotab_notes.txt(1) for the + ‘co_lnotab’ format and how to decode it. + + Changed in version 3.6: Line numbers can be decreasing. Before, + they were always increasing. + + -- Function: dis.findlabels (code) + + Detect all offsets in the raw compiled bytecode string `code' which + are jump targets, and return a list of these offsets. + + -- Function: dis.stack_effect (opcode, oparg=None, *, jump=None) + + Compute the stack effect of `opcode' with argument `oparg'. + + If the code has a jump target and `jump' is ‘True’, *note + stack_effect(): 83a. will return the stack effect of jumping. If + `jump' is ‘False’, it will return the stack effect of not jumping. + And if `jump' is ‘None’ (default), it will return the maximal stack + effect of both cases. + + New in version 3.4. + + Changed in version 3.8: Added `jump' parameter. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Objects/lnotab_notes.txt + + +File: python.info, Node: Python Bytecode Instructions, Next: Opcode collections, Prev: Analysis functions, Up: dis — Disassembler for Python bytecode + +5.32.12.3 Python Bytecode Instructions +...................................... + +The *note get_instructions(): 836. function and *note Bytecode: 837. +class provide details of bytecode instructions as *note Instruction: +835. instances: + + -- Class: dis.Instruction + + Details for a bytecode operation + + -- Data: opcode + + numeric code for operation, corresponding to the opcode values + listed below and the bytecode values in the *note Opcode + collections: 35df. + + -- Data: opname + + human readable name for operation + + -- Data: arg + + numeric argument to operation (if any), otherwise ‘None’ + + -- Data: argval + + resolved arg value (if known), otherwise same as arg + + -- Data: argrepr + + human readable description of operation argument + + -- Data: offset + + start index of operation within bytecode sequence + + -- Data: starts_line + + line started by this opcode (if any), otherwise ‘None’ + + -- Data: is_jump_target + + ‘True’ if other code jumps to here, otherwise ‘False’ + + New in version 3.4. + +The Python compiler currently generates the following bytecode +instructions. + +`General instructions' + + -- Opcode: NOP + + Do nothing code. Used as a placeholder by the bytecode optimizer. + + -- Opcode: POP_TOP + + Removes the top-of-stack (TOS) item. + + -- Opcode: ROT_TWO + + Swaps the two top-most stack items. + + -- Opcode: ROT_THREE + + Lifts second and third stack item one position up, moves top down + to position three. + + -- Opcode: ROT_FOUR + + Lifts second, third and fourth stack items one position up, moves + top down to position four. + + New in version 3.8. + + -- Opcode: DUP_TOP + + Duplicates the reference on top of the stack. + + New in version 3.2. + + -- Opcode: DUP_TOP_TWO + + Duplicates the two references on top of the stack, leaving them in + the same order. + + New in version 3.2. + +`Unary operations' + +Unary operations take the top of the stack, apply the operation, and +push the result back on the stack. + + -- Opcode: UNARY_POSITIVE + + Implements ‘TOS = +TOS’. + + -- Opcode: UNARY_NEGATIVE + + Implements ‘TOS = -TOS’. + + -- Opcode: UNARY_NOT + + Implements ‘TOS = not TOS’. + + -- Opcode: UNARY_INVERT + + Implements ‘TOS = ~TOS’. + + -- Opcode: GET_ITER + + Implements ‘TOS = iter(TOS)’. + + -- Opcode: GET_YIELD_FROM_ITER + + If ‘TOS’ is a *note generator iterator: 457. or *note coroutine: + 1a6. object it is left as is. Otherwise, implements ‘TOS = + iter(TOS)’. + + New in version 3.5. + +`Binary operations' + +Binary operations remove the top of the stack (TOS) and the second +top-most stack item (TOS1) from the stack. They perform the operation, +and put the result back on the stack. + + -- Opcode: BINARY_POWER + + Implements ‘TOS = TOS1 ** TOS’. + + -- Opcode: BINARY_MULTIPLY + + Implements ‘TOS = TOS1 * TOS’. + + -- Opcode: BINARY_MATRIX_MULTIPLY + + Implements ‘TOS = TOS1 @ TOS’. + + New in version 3.5. + + -- Opcode: BINARY_FLOOR_DIVIDE + + Implements ‘TOS = TOS1 // TOS’. + + -- Opcode: BINARY_TRUE_DIVIDE + + Implements ‘TOS = TOS1 / TOS’. + + -- Opcode: BINARY_MODULO + + Implements ‘TOS = TOS1 % TOS’. + + -- Opcode: BINARY_ADD + + Implements ‘TOS = TOS1 + TOS’. + + -- Opcode: BINARY_SUBTRACT + + Implements ‘TOS = TOS1 - TOS’. + + -- Opcode: BINARY_SUBSCR + + Implements ‘TOS = TOS1[TOS]’. + + -- Opcode: BINARY_LSHIFT + + Implements ‘TOS = TOS1 << TOS’. + + -- Opcode: BINARY_RSHIFT + + Implements ‘TOS = TOS1 >> TOS’. + + -- Opcode: BINARY_AND + + Implements ‘TOS = TOS1 & TOS’. + + -- Opcode: BINARY_XOR + + Implements ‘TOS = TOS1 ^ TOS’. + + -- Opcode: BINARY_OR + + Implements ‘TOS = TOS1 | TOS’. + +`In-place operations' + +In-place operations are like binary operations, in that they remove TOS +and TOS1, and push the result back on the stack, but the operation is +done in-place when TOS1 supports it, and the resulting TOS may be (but +does not have to be) the original TOS1. + + -- Opcode: INPLACE_POWER + + Implements in-place ‘TOS = TOS1 ** TOS’. + + -- Opcode: INPLACE_MULTIPLY + + Implements in-place ‘TOS = TOS1 * TOS’. + + -- Opcode: INPLACE_MATRIX_MULTIPLY + + Implements in-place ‘TOS = TOS1 @ TOS’. + + New in version 3.5. + + -- Opcode: INPLACE_FLOOR_DIVIDE + + Implements in-place ‘TOS = TOS1 // TOS’. + + -- Opcode: INPLACE_TRUE_DIVIDE + + Implements in-place ‘TOS = TOS1 / TOS’. + + -- Opcode: INPLACE_MODULO + + Implements in-place ‘TOS = TOS1 % TOS’. + + -- Opcode: INPLACE_ADD + + Implements in-place ‘TOS = TOS1 + TOS’. + + -- Opcode: INPLACE_SUBTRACT + + Implements in-place ‘TOS = TOS1 - TOS’. + + -- Opcode: INPLACE_LSHIFT + + Implements in-place ‘TOS = TOS1 << TOS’. + + -- Opcode: INPLACE_RSHIFT + + Implements in-place ‘TOS = TOS1 >> TOS’. + + -- Opcode: INPLACE_AND + + Implements in-place ‘TOS = TOS1 & TOS’. + + -- Opcode: INPLACE_XOR + + Implements in-place ‘TOS = TOS1 ^ TOS’. + + -- Opcode: INPLACE_OR + + Implements in-place ‘TOS = TOS1 | TOS’. + + -- Opcode: STORE_SUBSCR + + Implements ‘TOS1[TOS] = TOS2’. + + -- Opcode: DELETE_SUBSCR + + Implements ‘del TOS1[TOS]’. + +`Coroutine opcodes' + + -- Opcode: GET_AWAITABLE + + Implements ‘TOS = get_awaitable(TOS)’, where ‘get_awaitable(o)’ + returns ‘o’ if ‘o’ is a coroutine object or a generator object with + the CO_ITERABLE_COROUTINE flag, or resolves ‘o.__await__’. + + New in version 3.5. + + -- Opcode: GET_AITER + + Implements ‘TOS = TOS.__aiter__()’. + + New in version 3.5. + + Changed in version 3.7: Returning awaitable objects from + ‘__aiter__’ is no longer supported. + + -- Opcode: GET_ANEXT + + Implements ‘PUSH(get_awaitable(TOS.__anext__()))’. See + ‘GET_AWAITABLE’ for details about ‘get_awaitable’ + + New in version 3.5. + + -- Opcode: END_ASYNC_FOR + + Terminates an *note async for: 2e3. loop. Handles an exception + raised when awaiting a next item. If TOS is *note + StopAsyncIteration: 10cd. pop 7 values from the stack and restore + the exception state using the second three of them. Otherwise + re-raise the exception using the three values from the stack. An + exception handler block is removed from the block stack. + + New in version 3.8. + + -- Opcode: BEFORE_ASYNC_WITH + + Resolves ‘__aenter__’ and ‘__aexit__’ from the object on top of the + stack. Pushes ‘__aexit__’ and result of ‘__aenter__()’ to the + stack. + + New in version 3.5. + + -- Opcode: SETUP_ASYNC_WITH + + Creates a new frame object. + + New in version 3.5. + +`Miscellaneous opcodes' + + -- Opcode: PRINT_EXPR + + Implements the expression statement for the interactive mode. TOS + is removed from the stack and printed. In non-interactive mode, an + expression statement is terminated with *note POP_TOP: 35e8. + + -- Opcode: SET_ADD (i) + + Calls ‘set.add(TOS1[-i], TOS)’. Used to implement set + comprehensions. + + -- Opcode: LIST_APPEND (i) + + Calls ‘list.append(TOS1[-i], TOS)’. Used to implement list + comprehensions. + + -- Opcode: MAP_ADD (i) + + Calls ‘dict.__setitem__(TOS1[-i], TOS1, TOS)’. Used to implement + dict comprehensions. + + New in version 3.1. + + Changed in version 3.8: Map value is TOS and map key is TOS1. + Before, those were reversed. + +For all of the *note SET_ADD: 3616, *note LIST_APPEND: 3617. and *note +MAP_ADD: 2e4. instructions, while the added value or key/value pair is +popped off, the container object remains on the stack so that it is +available for further iterations of the loop. + + -- Opcode: RETURN_VALUE + + Returns with TOS to the caller of the function. + + -- Opcode: YIELD_VALUE + + Pops TOS and yields it from a *note generator: 9a2. + + -- Opcode: YIELD_FROM + + Pops TOS and delegates to it as a subiterator from a *note + generator: 9a2. + + New in version 3.3. + + -- Opcode: SETUP_ANNOTATIONS + + Checks whether ‘__annotations__’ is defined in ‘locals()’, if not + it is set up to an empty ‘dict’. This opcode is only emitted if a + class or module body contains *note variable annotations: 61f. + statically. + + New in version 3.6. + + -- Opcode: IMPORT_STAR + + Loads all symbols not starting with ‘'_'’ directly from the module + TOS to the local namespace. The module is popped after loading all + names. This opcode implements ‘from module import *’. + + -- Opcode: POP_BLOCK + + Removes one block from the block stack. Per frame, there is a + stack of blocks, denoting *note try: d72. statements, and such. + + -- Opcode: POP_EXCEPT + + Removes one block from the block stack. The popped block must be + an exception handler block, as implicitly created when entering an + except handler. In addition to popping extraneous values from the + frame stack, the last three popped values are used to restore the + exception state. + + -- Opcode: POP_FINALLY (preserve_tos) + + Cleans up the value stack and the block stack. If `preserve_tos' + is not ‘0’ TOS first is popped from the stack and pushed on the + stack after performing other stack operations: + + * If TOS is ‘NULL’ or an integer (pushed by *note BEGIN_FINALLY: + 2dd. or *note CALL_FINALLY: 2de.) it is popped from the stack. + + * If TOS is an exception type (pushed when an exception has been + raised) 6 values are popped from the stack, the last three + popped values are used to restore the exception state. An + exception handler block is removed from the block stack. + + It is similar to *note END_FINALLY: 2e0, but doesn’t change the + bytecode counter nor raise an exception. Used for implementing + *note break: 2db, *note continue: 181. and *note return: 190. in + the *note finally: 182. block. + + New in version 3.8. + + -- Opcode: BEGIN_FINALLY + + Pushes ‘NULL’ onto the stack for using it in *note END_FINALLY: + 2e0, *note POP_FINALLY: 2df, *note WITH_CLEANUP_START: 2e1. and + *note WITH_CLEANUP_FINISH: 361e. Starts the *note finally: 182. + block. + + New in version 3.8. + + -- Opcode: END_FINALLY + + Terminates a *note finally: 182. clause. The interpreter recalls + whether the exception has to be re-raised or execution has to be + continued depending on the value of TOS. + + * If TOS is ‘NULL’ (pushed by *note BEGIN_FINALLY: 2dd.) + continue from the next instruction. TOS is popped. + + * If TOS is an integer (pushed by *note CALL_FINALLY: 2de.), + sets the bytecode counter to TOS. TOS is popped. + + * If TOS is an exception type (pushed when an exception has been + raised) 6 values are popped from the stack, the first three + popped values are used to re-raise the exception and the last + three popped values are used to restore the exception state. + An exception handler block is removed from the block stack. + + -- Opcode: LOAD_BUILD_CLASS + + Pushes ‘builtins.__build_class__()’ onto the stack. It is later + called by *note CALL_FUNCTION: 619. to construct a class. + + -- Opcode: SETUP_WITH (delta) + + This opcode performs several operations before a with block starts. + First, it loads *note __exit__(): c96. from the context manager and + pushes it onto the stack for later use by *note WITH_CLEANUP_START: + 2e1. Then, *note __enter__(): c95. is called, and a finally block + pointing to `delta' is pushed. Finally, the result of calling the + ‘__enter__()’ method is pushed onto the stack. The next opcode + will either ignore it (*note POP_TOP: 35e8.), or store it in (a) + variable(s) (*note STORE_FAST: 3621, *note STORE_NAME: 3622, or + *note UNPACK_SEQUENCE: 3623.). + + New in version 3.2. + + -- Opcode: WITH_CLEANUP_START + + Starts cleaning up the stack when a *note with: 6e9. statement + block exits. + + At the top of the stack are either ‘NULL’ (pushed by *note + BEGIN_FINALLY: 2dd.) or 6 values pushed if an exception has been + raised in the with block. Below is the context manager’s *note + __exit__(): c96. or *note __aexit__(): 113c. bound method. + + If TOS is ‘NULL’, calls ‘SECOND(None, None, None)’, removes the + function from the stack, leaving TOS, and pushes ‘None’ to the + stack. Otherwise calls ‘SEVENTH(TOP, SECOND, THIRD)’, shifts the + bottom 3 values of the stack down, replaces the empty spot with + ‘NULL’ and pushes TOS. Finally pushes the result of the call. + + -- Opcode: WITH_CLEANUP_FINISH + + Finishes cleaning up the stack when a *note with: 6e9. statement + block exits. + + TOS is result of ‘__exit__()’ or ‘__aexit__()’ function call pushed + by *note WITH_CLEANUP_START: 2e1. SECOND is ‘None’ or an exception + type (pushed when an exception has been raised). + + Pops two values from the stack. If SECOND is not None and TOS is + true unwinds the EXCEPT_HANDLER block which was created when the + exception was caught and pushes ‘NULL’ to the stack. + +All of the following opcodes use their arguments. + + -- Opcode: STORE_NAME (namei) + + Implements ‘name = TOS’. `namei' is the index of `name' in the + attribute ‘co_names’ of the code object. The compiler tries to use + *note STORE_FAST: 3621. or *note STORE_GLOBAL: 3624. if possible. + + -- Opcode: DELETE_NAME (namei) + + Implements ‘del name’, where `namei' is the index into ‘co_names’ + attribute of the code object. + + -- Opcode: UNPACK_SEQUENCE (count) + + Unpacks TOS into `count' individual values, which are put onto the + stack right-to-left. + + -- Opcode: UNPACK_EX (counts) + + Implements assignment with a starred target: Unpacks an iterable in + TOS into individual values, where the total number of values can be + smaller than the number of items in the iterable: one of the new + values will be a list of all leftover items. + + The low byte of `counts' is the number of values before the list + value, the high byte of `counts' the number of values after it. + The resulting values are put onto the stack right-to-left. + + -- Opcode: STORE_ATTR (namei) + + Implements ‘TOS.name = TOS1’, where `namei' is the index of name in + ‘co_names’. + + -- Opcode: DELETE_ATTR (namei) + + Implements ‘del TOS.name’, using `namei' as index into ‘co_names’. + + -- Opcode: STORE_GLOBAL (namei) + + Works as *note STORE_NAME: 3622, but stores the name as a global. + + -- Opcode: DELETE_GLOBAL (namei) + + Works as *note DELETE_NAME: 3625, but deletes a global name. + + -- Opcode: LOAD_CONST (consti) + + Pushes ‘co_consts[consti]’ onto the stack. + + -- Opcode: LOAD_NAME (namei) + + Pushes the value associated with ‘co_names[namei]’ onto the stack. + + -- Opcode: BUILD_TUPLE (count) + + Creates a tuple consuming `count' items from the stack, and pushes + the resulting tuple onto the stack. + + -- Opcode: BUILD_LIST (count) + + Works as *note BUILD_TUPLE: 362c, but creates a list. + + -- Opcode: BUILD_SET (count) + + Works as *note BUILD_TUPLE: 362c, but creates a set. + + -- Opcode: BUILD_MAP (count) + + Pushes a new dictionary object onto the stack. Pops ‘2 * count’ + items so that the dictionary holds `count' entries: ‘{..., TOS3: + TOS2, TOS1: TOS}’. + + Changed in version 3.5: The dictionary is created from stack items + instead of creating an empty dictionary pre-sized to hold `count' + items. + + -- Opcode: BUILD_CONST_KEY_MAP (count) + + The version of *note BUILD_MAP: 362f. specialized for constant + keys. Pops the top element on the stack which contains a tuple of + keys, then starting from ‘TOS1’, pops `count' values to form values + in the built dictionary. + + New in version 3.6. + + -- Opcode: BUILD_STRING (count) + + Concatenates `count' strings from the stack and pushes the + resulting string onto the stack. + + New in version 3.6. + + -- Opcode: BUILD_TUPLE_UNPACK (count) + + Pops `count' iterables from the stack, joins them in a single + tuple, and pushes the result. Implements iterable unpacking in + tuple displays ‘(*x, *y, *z)’. + + New in version 3.5. + + -- Opcode: BUILD_TUPLE_UNPACK_WITH_CALL (count) + + This is similar to *note BUILD_TUPLE_UNPACK: 3630, but is used for + ‘f(*x, *y, *z)’ call syntax. The stack item at position ‘count + + 1’ should be the corresponding callable ‘f’. + + New in version 3.6. + + -- Opcode: BUILD_LIST_UNPACK (count) + + This is similar to *note BUILD_TUPLE_UNPACK: 3630, but pushes a + list instead of tuple. Implements iterable unpacking in list + displays ‘[*x, *y, *z]’. + + New in version 3.5. + + -- Opcode: BUILD_SET_UNPACK (count) + + This is similar to *note BUILD_TUPLE_UNPACK: 3630, but pushes a set + instead of tuple. Implements iterable unpacking in set displays + ‘{*x, *y, *z}’. + + New in version 3.5. + + -- Opcode: BUILD_MAP_UNPACK (count) + + Pops `count' mappings from the stack, merges them into a single + dictionary, and pushes the result. Implements dictionary unpacking + in dictionary displays ‘{**x, **y, **z}’. + + New in version 3.5. + + -- Opcode: BUILD_MAP_UNPACK_WITH_CALL (count) + + This is similar to *note BUILD_MAP_UNPACK: 3633, but is used for + ‘f(**x, **y, **z)’ call syntax. The stack item at position ‘count + + 2’ should be the corresponding callable ‘f’. + + New in version 3.5. + + Changed in version 3.6: The position of the callable is determined + by adding 2 to the opcode argument instead of encoding it in the + second byte of the argument. + + -- Opcode: LOAD_ATTR (namei) + + Replaces TOS with ‘getattr(TOS, co_names[namei])’. + + -- Opcode: COMPARE_OP (opname) + + Performs a Boolean operation. The operation name can be found in + ‘cmp_op[opname]’. + + -- Opcode: IMPORT_NAME (namei) + + Imports the module ‘co_names[namei]’. TOS and TOS1 are popped and + provide the `fromlist' and `level' arguments of *note __import__(): + 610. The module object is pushed onto the stack. The current + namespace is not affected: for a proper import statement, a + subsequent *note STORE_FAST: 3621. instruction modifies the + namespace. + + -- Opcode: IMPORT_FROM (namei) + + Loads the attribute ‘co_names[namei]’ from the module found in TOS. + The resulting object is pushed onto the stack, to be subsequently + stored by a *note STORE_FAST: 3621. instruction. + + -- Opcode: JUMP_FORWARD (delta) + + Increments bytecode counter by `delta'. + + -- Opcode: POP_JUMP_IF_TRUE (target) + + If TOS is true, sets the bytecode counter to `target'. TOS is + popped. + + New in version 3.1. + + -- Opcode: POP_JUMP_IF_FALSE (target) + + If TOS is false, sets the bytecode counter to `target'. TOS is + popped. + + New in version 3.1. + + -- Opcode: JUMP_IF_TRUE_OR_POP (target) + + If TOS is true, sets the bytecode counter to `target' and leaves + TOS on the stack. Otherwise (TOS is false), TOS is popped. + + New in version 3.1. + + -- Opcode: JUMP_IF_FALSE_OR_POP (target) + + If TOS is false, sets the bytecode counter to `target' and leaves + TOS on the stack. Otherwise (TOS is true), TOS is popped. + + New in version 3.1. + + -- Opcode: JUMP_ABSOLUTE (target) + + Set bytecode counter to `target'. + + -- Opcode: FOR_ITER (delta) + + TOS is an *note iterator: 112e. Call its *note __next__(): c64. + method. If this yields a new value, push it on the stack (leaving + the iterator below it). If the iterator indicates it is exhausted + TOS is popped, and the byte code counter is incremented by `delta'. + + -- Opcode: LOAD_GLOBAL (namei) + + Loads the global named ‘co_names[namei]’ onto the stack. + + -- Opcode: SETUP_FINALLY (delta) + + Pushes a try block from a try-finally or try-except clause onto the + block stack. `delta' points to the finally block or the first + except block. + + -- Opcode: CALL_FINALLY (delta) + + Pushes the address of the next instruction onto the stack and + increments bytecode counter by `delta'. Used for calling the + finally block as a “subroutine”. + + New in version 3.8. + + -- Opcode: LOAD_FAST (var_num) + + Pushes a reference to the local ‘co_varnames[var_num]’ onto the + stack. + + -- Opcode: STORE_FAST (var_num) + + Stores TOS into the local ‘co_varnames[var_num]’. + + -- Opcode: DELETE_FAST (var_num) + + Deletes local ‘co_varnames[var_num]’. + + -- Opcode: LOAD_CLOSURE (i) + + Pushes a reference to the cell contained in slot `i' of the cell + and free variable storage. The name of the variable is + ‘co_cellvars[i]’ if `i' is less than the length of `co_cellvars'. + Otherwise it is ‘co_freevars[i - len(co_cellvars)]’. + + -- Opcode: LOAD_DEREF (i) + + Loads the cell contained in slot `i' of the cell and free variable + storage. Pushes a reference to the object the cell contains on the + stack. + + -- Opcode: LOAD_CLASSDEREF (i) + + Much like *note LOAD_DEREF: 3644. but first checks the locals + dictionary before consulting the cell. This is used for loading + free variables in class bodies. + + New in version 3.4. + + -- Opcode: STORE_DEREF (i) + + Stores TOS into the cell contained in slot `i' of the cell and free + variable storage. + + -- Opcode: DELETE_DEREF (i) + + Empties the cell contained in slot `i' of the cell and free + variable storage. Used by the *note del: f02. statement. + + New in version 3.2. + + -- Opcode: RAISE_VARARGS (argc) + + Raises an exception using one of the 3 forms of the ‘raise’ + statement, depending on the value of `argc': + + * 0: ‘raise’ (re-raise previous exception) + + * 1: ‘raise TOS’ (raise exception instance or type at ‘TOS’) + + * 2: ‘raise TOS1 from TOS’ (raise exception instance or type at + ‘TOS1’ with ‘__cause__’ set to ‘TOS’) + + -- Opcode: CALL_FUNCTION (argc) + + Calls a callable object with positional arguments. `argc' + indicates the number of positional arguments. The top of the stack + contains positional arguments, with the right-most argument on top. + Below the arguments is a callable object to call. ‘CALL_FUNCTION’ + pops all arguments and the callable object off the stack, calls the + callable object with those arguments, and pushes the return value + returned by the callable object. + + Changed in version 3.6: This opcode is used only for calls with + positional arguments. + + -- Opcode: CALL_FUNCTION_KW (argc) + + Calls a callable object with positional (if any) and keyword + arguments. `argc' indicates the total number of positional and + keyword arguments. The top element on the stack contains a tuple + of keyword argument names. Below that are keyword arguments in the + order corresponding to the tuple. Below that are positional + arguments, with the right-most parameter on top. Below the + arguments is a callable object to call. ‘CALL_FUNCTION_KW’ pops + all arguments and the callable object off the stack, calls the + callable object with those arguments, and pushes the return value + returned by the callable object. + + Changed in version 3.6: Keyword arguments are packed in a tuple + instead of a dictionary, `argc' indicates the total number of + arguments. + + -- Opcode: CALL_FUNCTION_EX (flags) + + Calls a callable object with variable set of positional and keyword + arguments. If the lowest bit of `flags' is set, the top of the + stack contains a mapping object containing additional keyword + arguments. Below that is an iterable object containing positional + arguments and a callable object to call. *note + BUILD_MAP_UNPACK_WITH_CALL: 61b. and *note + BUILD_TUPLE_UNPACK_WITH_CALL: 61d. can be used for merging multiple + mapping objects and iterables containing arguments. Before the + callable is called, the mapping object and iterable object are each + “unpacked” and their contents passed in as keyword and positional + arguments respectively. ‘CALL_FUNCTION_EX’ pops all arguments and + the callable object off the stack, calls the callable object with + those arguments, and pushes the return value returned by the + callable object. + + New in version 3.6. + + -- Opcode: LOAD_METHOD (namei) + + Loads a method named ‘co_names[namei]’ from the TOS object. TOS is + popped. This bytecode distinguishes two cases: if TOS has a method + with the correct name, the bytecode pushes the unbound method and + TOS. TOS will be used as the first argument (‘self’) by *note + CALL_METHOD: 4ae. when calling the unbound method. Otherwise, + ‘NULL’ and the object return by the attribute lookup are pushed. + + New in version 3.7. + + -- Opcode: CALL_METHOD (argc) + + Calls a method. `argc' is the number of positional arguments. + Keyword arguments are not supported. This opcode is designed to be + used with *note LOAD_METHOD: 4ad. Positional arguments are on top + of the stack. Below them, the two items described in *note + LOAD_METHOD: 4ad. are on the stack (either ‘self’ and an unbound + method object or ‘NULL’ and an arbitrary callable). All of them + are popped and the return value is pushed. + + New in version 3.7. + + -- Opcode: MAKE_FUNCTION (flags) + + Pushes a new function object on the stack. From bottom to top, the + consumed stack must consist of values if the argument carries a + specified flag value + + * ‘0x01’ a tuple of default values for positional-only and + positional-or-keyword parameters in positional order + + * ‘0x02’ a dictionary of keyword-only parameters’ default values + + * ‘0x04’ an annotation dictionary + + * ‘0x08’ a tuple containing cells for free variables, making a + closure + + * the code associated with the function (at TOS1) + + * the *note qualified name: 10ca. of the function (at TOS) + + -- Opcode: BUILD_SLICE (argc) + + Pushes a slice object on the stack. `argc' must be 2 or 3. If it + is 2, ‘slice(TOS1, TOS)’ is pushed; if it is 3, ‘slice(TOS2, TOS1, + TOS)’ is pushed. See the *note slice(): e04. built-in function for + more information. + + -- Opcode: EXTENDED_ARG (ext) + + Prefixes any opcode which has an argument too big to fit into the + default one byte. `ext' holds an additional byte which act as + higher bits in the argument. For each opcode, at most three + prefixal ‘EXTENDED_ARG’ are allowed, forming an argument from + two-byte to four-byte. + + -- Opcode: FORMAT_VALUE (flags) + + Used for implementing formatted literal strings (f-strings). Pops + an optional `fmt_spec' from the stack, then a required `value'. + `flags' is interpreted as follows: + + * ‘(flags & 0x03) == 0x00’: `value' is formatted as-is. + + * ‘(flags & 0x03) == 0x01’: call *note str(): 330. on `value' + before formatting it. + + * ‘(flags & 0x03) == 0x02’: call *note repr(): 7cc. on `value' + before formatting it. + + * ‘(flags & 0x03) == 0x03’: call *note ascii(): d4c. on `value' + before formatting it. + + * ‘(flags & 0x04) == 0x04’: pop `fmt_spec' from the stack and + use it, else use an empty `fmt_spec'. + + Formatting is performed using ‘PyObject_Format()’. The result is + pushed on the stack. + + New in version 3.6. + + -- Opcode: HAVE_ARGUMENT + + This is not really an opcode. It identifies the dividing line + between opcodes which don’t use their argument and those that do + (‘< HAVE_ARGUMENT’ and ‘>= HAVE_ARGUMENT’, respectively). + + Changed in version 3.6: Now every instruction has an argument, but + opcodes ‘< HAVE_ARGUMENT’ ignore it. Before, only opcodes ‘>= + HAVE_ARGUMENT’ had an argument. + + +File: python.info, Node: Opcode collections, Prev: Python Bytecode Instructions, Up: dis — Disassembler for Python bytecode + +5.32.12.4 Opcode collections +............................ + +These collections are provided for automatic introspection of bytecode +instructions: + + -- Data: dis.opname + + Sequence of operation names, indexable using the bytecode. + + -- Data: dis.opmap + + Dictionary mapping operation names to bytecodes. + + -- Data: dis.cmp_op + + Sequence of all compare operation names. + + -- Data: dis.hasconst + + Sequence of bytecodes that access a constant. + + -- Data: dis.hasfree + + Sequence of bytecodes that access a free variable (note that ‘free’ + in this context refers to names in the current scope that are + referenced by inner scopes or names in outer scopes that are + referenced from this scope. It does `not' include references to + global or builtin scopes). + + -- Data: dis.hasname + + Sequence of bytecodes that access an attribute by name. + + -- Data: dis.hasjrel + + Sequence of bytecodes that have a relative jump target. + + -- Data: dis.hasjabs + + Sequence of bytecodes that have an absolute jump target. + + -- Data: dis.haslocal + + Sequence of bytecodes that access a local variable. + + -- Data: dis.hascompare + + Sequence of bytecodes of Boolean operations. + + +File: python.info, Node: pickletools — Tools for pickle developers, Prev: dis — Disassembler for Python bytecode, Up: Python Language Services + +5.32.13 ‘pickletools’ — Tools for pickle developers +--------------------------------------------------- + +`Source code:' Lib/pickletools.py(1) + +__________________________________________________________________ + +This module contains various constants relating to the intimate details +of the *note pickle: ca. module, some lengthy comments about the +implementation, and a few useful functions for analyzing pickled data. +The contents of this module are useful for Python core developers who +are working on the *note pickle: ca.; ordinary users of the *note +pickle: ca. module probably won’t find the *note pickletools: cb. module +relevant. + +* Menu: + +* Command line usage: Command line usage<2>. +* Programmatic Interface: Programmatic Interface<2>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pickletools.py + + +File: python.info, Node: Command line usage<2>, Next: Programmatic Interface<2>, Up: pickletools — Tools for pickle developers + +5.32.13.1 Command line usage +............................ + +New in version 3.2. + +When invoked from the command line, ‘python -m pickletools’ will +disassemble the contents of one or more pickle files. Note that if you +want to see the Python object stored in the pickle rather than the +details of pickle format, you may want to use ‘-m pickle’ instead. +However, when the pickle file that you want to examine comes from an +untrusted source, ‘-m pickletools’ is a safer option because it does not +execute pickle bytecode. + +For example, with a tuple ‘(1, 2)’ pickled in file ‘x.pickle’: + + $ python -m pickle x.pickle + (1, 2) + + $ python -m pickletools x.pickle + 0: \x80 PROTO 3 + 2: K BININT1 1 + 4: K BININT1 2 + 6: \x86 TUPLE2 + 7: q BINPUT 0 + 9: . STOP + highest protocol among opcodes = 2 + +* Menu: + +* Command line options: Command line options<3>. + + +File: python.info, Node: Command line options<3>, Up: Command line usage<2> + +5.32.13.2 Command line options +.............................. + + -- Program Option: -a, --annotate + + Annotate each line with a short opcode description. + + -- Program Option: -o, --output=<file> + + Name of a file where the output should be written. + + -- Program Option: -l, --indentlevel=<num> + + The number of blanks by which to indent a new MARK level. + + -- Program Option: -m, --memo + + When multiple objects are disassembled, preserve memo between + disassemblies. + + -- Program Option: -p, --preamble=<preamble> + + When more than one pickle file are specified, print given preamble + before each disassembly. + + +File: python.info, Node: Programmatic Interface<2>, Prev: Command line usage<2>, Up: pickletools — Tools for pickle developers + +5.32.13.3 Programmatic Interface +................................ + + -- Function: pickletools.dis (pickle, out=None, memo=None, + indentlevel=4, annotate=0) + + Outputs a symbolic disassembly of the pickle to the file-like + object `out', defaulting to ‘sys.stdout’. `pickle' can be a string + or a file-like object. `memo' can be a Python dictionary that will + be used as the pickle’s memo; it can be used to perform + disassemblies across multiple pickles created by the same pickler. + Successive levels, indicated by ‘MARK’ opcodes in the stream, are + indented by `indentlevel' spaces. If a nonzero value is given to + `annotate', each opcode in the output is annotated with a short + description. The value of `annotate' is used as a hint for the + column where annotation should start. + + New in version 3.2: The `annotate' argument. + + -- Function: pickletools.genops (pickle) + + Provides an *note iterator: 112e. over all of the opcodes in a + pickle, returning a sequence of ‘(opcode, arg, pos)’ triples. + `opcode' is an instance of an ‘OpcodeInfo’ class; `arg' is the + decoded value, as a Python object, of the opcode’s argument; `pos' + is the position at which this opcode is located. `pickle' can be a + string or a file-like object. + + -- Function: pickletools.optimize (picklestring) + + Returns a new equivalent pickle string after eliminating unused + ‘PUT’ opcodes. The optimized pickle is shorter, takes less + transmission time, requires less storage space, and unpickles more + efficiently. + + +File: python.info, Node: Miscellaneous Services, Next: MS Windows Specific Services, Prev: Python Language Services, Up: The Python Standard Library + +5.33 Miscellaneous Services +=========================== + +The modules described in this chapter provide miscellaneous services +that are available in all Python versions. Here’s an overview: + +* Menu: + +* formatter — Generic output formatting:: + + +File: python.info, Node: formatter — Generic output formatting, Up: Miscellaneous Services + +5.33.1 ‘formatter’ — Generic output formatting +---------------------------------------------- + +Deprecated since version 3.4: Due to lack of usage, the formatter module +has been deprecated. + +__________________________________________________________________ + +This module supports two interface definitions, each with multiple +implementations: The `formatter' interface, and the `writer' interface +which is required by the formatter interface. + +Formatter objects transform an abstract flow of formatting events into +specific output events on writer objects. Formatters manage several +stack structures to allow various properties of a writer object to be +changed and restored; writers need not be able to handle relative +changes nor any sort of “change back” operation. Specific writer +properties which may be controlled via formatter objects are horizontal +alignment, font, and left margin indentations. A mechanism is provided +which supports providing arbitrary, non-exclusive style settings to a +writer as well. Additional interfaces facilitate formatting events +which are not reversible, such as paragraph separation. + +Writer objects encapsulate device interfaces. Abstract devices, such as +file formats, are supported as well as physical devices. The provided +implementations all work with abstract devices. The interface makes +available mechanisms for setting the properties which formatter objects +manage and inserting data into the output. + +* Menu: + +* The Formatter Interface:: +* Formatter Implementations:: +* The Writer Interface:: +* Writer Implementations:: + + +File: python.info, Node: The Formatter Interface, Next: Formatter Implementations, Up: formatter — Generic output formatting + +5.33.1.1 The Formatter Interface +................................ + +Interfaces to create formatters are dependent on the specific formatter +class being instantiated. The interfaces described below are the +required interfaces which all formatters must support once initialized. + +One data element is defined at the module level: + + -- Data: formatter.AS_IS + + Value which can be used in the font specification passed to the + ‘push_font()’ method described below, or as the new value to any + other ‘push_property()’ method. Pushing the ‘AS_IS’ value allows + the corresponding ‘pop_property()’ method to be called without + having to track whether the property was changed. + +The following attributes are defined for formatter instance objects: + + -- Attribute: formatter.writer + + The writer instance with which the formatter interacts. + + -- Method: formatter.end_paragraph (blanklines) + + Close any open paragraphs and insert at least `blanklines' before + the next paragraph. + + -- Method: formatter.add_line_break () + + Add a hard line break if one does not already exist. This does not + break the logical paragraph. + + -- Method: formatter.add_hor_rule (*args, **kw) + + Insert a horizontal rule in the output. A hard break is inserted + if there is data in the current paragraph, but the logical + paragraph is not broken. The arguments and keywords are passed on + to the writer’s ‘send_line_break()’ method. + + -- Method: formatter.add_flowing_data (data) + + Provide data which should be formatted with collapsed whitespace. + Whitespace from preceding and successive calls to *note + add_flowing_data(): 366d. is considered as well when the whitespace + collapse is performed. The data which is passed to this method is + expected to be word-wrapped by the output device. Note that any + word-wrapping still must be performed by the writer object due to + the need to rely on device and font information. + + -- Method: formatter.add_literal_data (data) + + Provide data which should be passed to the writer unchanged. + Whitespace, including newline and tab characters, are considered + legal in the value of `data'. + + -- Method: formatter.add_label_data (format, counter) + + Insert a label which should be placed to the left of the current + left margin. This should be used for constructing bulleted or + numbered lists. If the `format' value is a string, it is + interpreted as a format specification for `counter', which should + be an integer. The result of this formatting becomes the value of + the label; if `format' is not a string it is used as the label + value directly. The label value is passed as the only argument to + the writer’s ‘send_label_data()’ method. Interpretation of + non-string label values is dependent on the associated writer. + + Format specifications are strings which, in combination with a + counter value, are used to compute label values. Each character in + the format string is copied to the label value, with some + characters recognized to indicate a transform on the counter value. + Specifically, the character ‘'1'’ represents the counter value + formatter as an Arabic number, the characters ‘'A'’ and ‘'a'’ + represent alphabetic representations of the counter value in upper + and lower case, respectively, and ‘'I'’ and ‘'i'’ represent the + counter value in Roman numerals, in upper and lower case. Note + that the alphabetic and roman transforms require that the counter + value be greater than zero. + + -- Method: formatter.flush_softspace () + + Send any pending whitespace buffered from a previous call to *note + add_flowing_data(): 366d. to the associated writer object. This + should be called before any direct manipulation of the writer + object. + + -- Method: formatter.push_alignment (align) + + Push a new alignment setting onto the alignment stack. This may be + *note AS_IS: 3668. if no change is desired. If the alignment value + is changed from the previous setting, the writer’s + ‘new_alignment()’ method is called with the `align' value. + + -- Method: formatter.pop_alignment () + + Restore the previous alignment. + + -- Method: formatter.push_font ((size, italic, bold, teletype)) + + Change some or all font properties of the writer object. + Properties which are not set to *note AS_IS: 3668. are set to the + values passed in while others are maintained at their current + settings. The writer’s ‘new_font()’ method is called with the + fully resolved font specification. + + -- Method: formatter.pop_font () + + Restore the previous font. + + -- Method: formatter.push_margin (margin) + + Increase the number of left margin indentations by one, associating + the logical tag `margin' with the new indentation. The initial + margin level is ‘0’. Changed values of the logical tag must be + true values; false values other than *note AS_IS: 3668. are not + sufficient to change the margin. + + -- Method: formatter.pop_margin () + + Restore the previous margin. + + -- Method: formatter.push_style (*styles) + + Push any number of arbitrary style specifications. All styles are + pushed onto the styles stack in order. A tuple representing the + entire stack, including *note AS_IS: 3668. values, is passed to the + writer’s ‘new_styles()’ method. + + -- Method: formatter.pop_style (n=1) + + Pop the last `n' style specifications passed to *note push_style(): + 3677. A tuple representing the revised stack, including *note + AS_IS: 3668. values, is passed to the writer’s ‘new_styles()’ + method. + + -- Method: formatter.set_spacing (spacing) + + Set the spacing style for the writer. + + -- Method: formatter.assert_line_data (flag=1) + + Inform the formatter that data has been added to the current + paragraph out-of-band. This should be used when the writer has + been manipulated directly. The optional `flag' argument can be set + to false if the writer manipulations produced a hard line break at + the end of the output. + + +File: python.info, Node: Formatter Implementations, Next: The Writer Interface, Prev: The Formatter Interface, Up: formatter — Generic output formatting + +5.33.1.2 Formatter Implementations +.................................. + +Two implementations of formatter objects are provided by this module. +Most applications may use one of these classes without modification or +subclassing. + + -- Class: formatter.NullFormatter (writer=None) + + A formatter which does nothing. If `writer' is omitted, a *note + NullWriter: 367e. instance is created. No methods of the writer + are called by *note NullFormatter: 367d. instances. + Implementations should inherit from this class if implementing a + writer interface but don’t need to inherit any implementation. + + -- Class: formatter.AbstractFormatter (writer) + + The standard formatter. This implementation has demonstrated wide + applicability to many writers, and may be used directly in most + circumstances. It has been used to implement a full-featured World + Wide Web browser. + + +File: python.info, Node: The Writer Interface, Next: Writer Implementations, Prev: Formatter Implementations, Up: formatter — Generic output formatting + +5.33.1.3 The Writer Interface +............................. + +Interfaces to create writers are dependent on the specific writer class +being instantiated. The interfaces described below are the required +interfaces which all writers must support once initialized. Note that +while most applications can use the *note AbstractFormatter: 367f. class +as a formatter, the writer must typically be provided by the +application. + + -- Method: writer.flush () + + Flush any buffered output or device control events. + + -- Method: writer.new_alignment (align) + + Set the alignment style. The `align' value can be any object, but + by convention is a string or ‘None’, where ‘None’ indicates that + the writer’s “preferred” alignment should be used. Conventional + `align' values are ‘'left'’, ‘'center'’, ‘'right'’, and + ‘'justify'’. + + -- Method: writer.new_font (font) + + Set the font style. The value of `font' will be ‘None’, indicating + that the device’s default font should be used, or a tuple of the + form ‘(size, italic, bold, teletype)’. Size will be a string + indicating the size of font that should be used; specific strings + and their interpretation must be defined by the application. The + `italic', `bold', and `teletype' values are Boolean values + specifying which of those font attributes should be used. + + -- Method: writer.new_margin (margin, level) + + Set the margin level to the integer `level' and the logical tag to + `margin'. Interpretation of the logical tag is at the writer’s + discretion; the only restriction on the value of the logical tag is + that it not be a false value for non-zero values of `level'. + + -- Method: writer.new_spacing (spacing) + + Set the spacing style to `spacing'. + + -- Method: writer.new_styles (styles) + + Set additional styles. The `styles' value is a tuple of arbitrary + values; the value *note AS_IS: 3668. should be ignored. The + `styles' tuple may be interpreted either as a set or as a stack + depending on the requirements of the application and writer + implementation. + + -- Method: writer.send_line_break () + + Break the current line. + + -- Method: writer.send_paragraph (blankline) + + Produce a paragraph separation of at least `blankline' blank lines, + or the equivalent. The `blankline' value will be an integer. Note + that the implementation will receive a call to *note + send_line_break(): 3688. before this call if a line break is + needed; this method should not include ending the last line of the + paragraph. It is only responsible for vertical spacing between + paragraphs. + + -- Method: writer.send_hor_rule (*args, **kw) + + Display a horizontal rule on the output device. The arguments to + this method are entirely application- and writer-specific, and + should be interpreted with care. The method implementation may + assume that a line break has already been issued via *note + send_line_break(): 3688. + + -- Method: writer.send_flowing_data (data) + + Output character data which may be word-wrapped and re-flowed as + needed. Within any sequence of calls to this method, the writer + may assume that spans of multiple whitespace characters have been + collapsed to single space characters. + + -- Method: writer.send_literal_data (data) + + Output character data which has already been formatted for display. + Generally, this should be interpreted to mean that line breaks + indicated by newline characters should be preserved and no new line + breaks should be introduced. The data may contain embedded newline + and tab characters, unlike data provided to the + ‘send_formatted_data()’ interface. + + -- Method: writer.send_label_data (data) + + Set `data' to the left of the current left margin, if possible. + The value of `data' is not restricted; treatment of non-string + values is entirely application- and writer-dependent. This method + will only be called at the beginning of a line. + + +File: python.info, Node: Writer Implementations, Prev: The Writer Interface, Up: formatter — Generic output formatting + +5.33.1.4 Writer Implementations +............................... + +Three implementations of the writer object interface are provided as +examples by this module. Most applications will need to derive new +writer classes from the *note NullWriter: 367e. class. + + -- Class: formatter.NullWriter + + A writer which only provides the interface definition; no actions + are taken on any methods. This should be the base class for all + writers which do not need to inherit any implementation methods. + + -- Class: formatter.AbstractWriter + + A writer which can be used in debugging formatters, but not much + else. Each method simply announces itself by printing its name and + arguments on standard output. + + -- Class: formatter.DumbWriter (file=None, maxcol=72) + + Simple writer class which writes output on the *note file object: + b42. passed in as `file' or, if `file' is omitted, on standard + output. The output is simply word-wrapped to the number of columns + specified by `maxcol'. This class is suitable for reflowing a + sequence of paragraphs. + + +File: python.info, Node: MS Windows Specific Services, Next: Unix Specific Services, Prev: Miscellaneous Services, Up: The Python Standard Library + +5.34 MS Windows Specific Services +================================= + +This chapter describes modules that are only available on MS Windows +platforms. + +* Menu: + +* msilib — Read and write Microsoft Installer files:: +* msvcrt — Useful routines from the MS VC++ runtime:: +* winreg — Windows registry access:: +* winsound — Sound-playing interface for Windows:: + + +File: python.info, Node: msilib — Read and write Microsoft Installer files, Next: msvcrt — Useful routines from the MS VC++ runtime, Up: MS Windows Specific Services + +5.34.1 ‘msilib’ — Read and write Microsoft Installer files +---------------------------------------------------------- + +`Source code:' Lib/msilib/__init__.py(1) + +__________________________________________________________________ + +The *note msilib: b5. supports the creation of Microsoft Installer +(‘.msi’) files. Because these files often contain an embedded “cabinet” +file (‘.cab’), it also exposes an API to create CAB files. Support for +reading ‘.cab’ files is currently not implemented; read support for the +‘.msi’ database is possible. + +This package aims to provide complete access to all tables in an ‘.msi’ +file, therefore, it is a fairly low-level API. Two primary applications +of this package are the *note distutils: 39. command ‘bdist_msi’, and +the creation of Python installer package itself (although that currently +uses a different version of ‘msilib’). + +The package contents can be roughly split into four parts: low-level CAB +routines, low-level MSI routines, higher-level MSI routines, and +standard table structures. + + -- Function: msilib.FCICreate (cabname, files) + + Create a new CAB file named `cabname'. `files' must be a list of + tuples, each containing the name of the file on disk, and the name + of the file inside the CAB file. + + The files are added to the CAB file in the order they appear in the + list. All files are added into a single CAB file, using the MSZIP + compression algorithm. + + Callbacks to Python for the various steps of MSI creation are + currently not exposed. + + -- Function: msilib.UuidCreate () + + Return the string representation of a new unique identifier. This + wraps the Windows API functions ‘UuidCreate()’ and + ‘UuidToString()’. + + -- Function: msilib.OpenDatabase (path, persist) + + Return a new database object by calling MsiOpenDatabase. `path' is + the file name of the MSI file; `persist' can be one of the + constants ‘MSIDBOPEN_CREATEDIRECT’, ‘MSIDBOPEN_CREATE’, + ‘MSIDBOPEN_DIRECT’, ‘MSIDBOPEN_READONLY’, or ‘MSIDBOPEN_TRANSACT’, + and may include the flag ‘MSIDBOPEN_PATCHFILE’. See the Microsoft + documentation for the meaning of these flags; depending on the + flags, an existing database is opened, or a new one created. + + -- Function: msilib.CreateRecord (count) + + Return a new record object by calling ‘MSICreateRecord()’. `count' + is the number of fields of the record. + + -- Function: msilib.init_database (name, schema, ProductName, + ProductCode, ProductVersion, Manufacturer) + + Create and return a new database `name', initialize it with + `schema', and set the properties `ProductName', `ProductCode', + `ProductVersion', and `Manufacturer'. + + `schema' must be a module object containing ‘tables’ and + ‘_Validation_records’ attributes; typically, *note msilib.schema: + 369b. should be used. + + The database will contain just the schema and the validation + records when this function returns. + + -- Function: msilib.add_data (database, table, records) + + Add all `records' to the table named `table' in `database'. + + The `table' argument must be one of the predefined tables in the + MSI schema, e.g. ‘'Feature'’, ‘'File'’, ‘'Component'’, ‘'Dialog'’, + ‘'Control'’, etc. + + `records' should be a list of tuples, each one containing all + fields of a record according to the schema of the table. For + optional fields, ‘None’ can be passed. + + Field values can be ints, strings, or instances of the Binary + class. + + -- Class: msilib.Binary (filename) + + Represents entries in the Binary table; inserting such an object + using *note add_data(): 369c. reads the file named `filename' into + the table. + + -- Function: msilib.add_tables (database, module) + + Add all table content from `module' to `database'. `module' must + contain an attribute `tables' listing all tables for which content + should be added, and one attribute per table that has the actual + content. + + This is typically used to install the sequence tables. + + -- Function: msilib.add_stream (database, name, path) + + Add the file `path' into the ‘_Stream’ table of `database', with + the stream name `name'. + + -- Function: msilib.gen_uuid () + + Return a new UUID, in the format that MSI typically requires (i.e. + in curly braces, and with all hexdigits in upper-case). + +See also +........ + +FCICreate(2) UuidCreate(3) UuidToString(4) + +* Menu: + +* Database Objects:: +* View Objects:: +* Summary Information Objects:: +* Record Objects:: +* Errors:: +* CAB Objects:: +* Directory Objects:: +* Features: Features<3>. +* GUI classes:: +* Precomputed tables:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/msilib/__init__.py + + (2) https://msdn.microsoft.com/en-us/library/bb432265.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa379205.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa379352.aspx + + +File: python.info, Node: Database Objects, Next: View Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.1 Database Objects +......................... + + -- Method: Database.OpenView (sql) + + Return a view object, by calling ‘MSIDatabaseOpenView()’. `sql' is + the SQL statement to execute. + + -- Method: Database.Commit () + + Commit the changes pending in the current transaction, by calling + ‘MSIDatabaseCommit()’. + + -- Method: Database.GetSummaryInformation (count) + + Return a new summary information object, by calling + ‘MsiGetSummaryInformation()’. `count' is the maximum number of + updated values. + + -- Method: Database.Close () + + Close the database object, through ‘MsiCloseHandle()’. + + New in version 3.7. + +See also +........ + +MSIDatabaseOpenView(1) MSIDatabaseCommit(2) MSIGetSummaryInformation(3) +MsiCloseHandle(4) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370082.aspx + + (2) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370075.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370301.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370067.aspx + + +File: python.info, Node: View Objects, Next: Summary Information Objects, Prev: Database Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.2 View Objects +..................... + + -- Method: View.Execute (params) + + Execute the SQL query of the view, through ‘MSIViewExecute()’. If + `params' is not ‘None’, it is a record describing actual values of + the parameter tokens in the query. + + -- Method: View.GetColumnInfo (kind) + + Return a record describing the columns of the view, through calling + ‘MsiViewGetColumnInfo()’. `kind' can be either ‘MSICOLINFO_NAMES’ + or ‘MSICOLINFO_TYPES’. + + -- Method: View.Fetch () + + Return a result record of the query, through calling + ‘MsiViewFetch()’. + + -- Method: View.Modify (kind, data) + + Modify the view, by calling ‘MsiViewModify()’. `kind' can be one + of ‘MSIMODIFY_SEEK’, ‘MSIMODIFY_REFRESH’, ‘MSIMODIFY_INSERT’, + ‘MSIMODIFY_UPDATE’, ‘MSIMODIFY_ASSIGN’, ‘MSIMODIFY_REPLACE’, + ‘MSIMODIFY_MERGE’, ‘MSIMODIFY_DELETE’, + ‘MSIMODIFY_INSERT_TEMPORARY’, ‘MSIMODIFY_VALIDATE’, + ‘MSIMODIFY_VALIDATE_NEW’, ‘MSIMODIFY_VALIDATE_FIELD’, or + ‘MSIMODIFY_VALIDATE_DELETE’. + + `data' must be a record describing the new data. + + -- Method: View.Close () + + Close the view, through ‘MsiViewClose()’. + +See also +........ + +MsiViewExecute(1) MSIViewGetColumnInfo(2) MsiViewFetch(3) +MsiViewModify(4) MsiViewClose(5) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370513.aspx + + (2) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370516.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370514.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370519.aspx + + (5) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370510.aspx + + +File: python.info, Node: Summary Information Objects, Next: Record Objects, Prev: View Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.3 Summary Information Objects +.................................... + + -- Method: SummaryInformation.GetProperty (field) + + Return a property of the summary, through + ‘MsiSummaryInfoGetProperty()’. `field' is the name of the + property, and can be one of the constants ‘PID_CODEPAGE’, + ‘PID_TITLE’, ‘PID_SUBJECT’, ‘PID_AUTHOR’, ‘PID_KEYWORDS’, + ‘PID_COMMENTS’, ‘PID_TEMPLATE’, ‘PID_LASTAUTHOR’, ‘PID_REVNUMBER’, + ‘PID_LASTPRINTED’, ‘PID_CREATE_DTM’, ‘PID_LASTSAVE_DTM’, + ‘PID_PAGECOUNT’, ‘PID_WORDCOUNT’, ‘PID_CHARCOUNT’, ‘PID_APPNAME’, + or ‘PID_SECURITY’. + + -- Method: SummaryInformation.GetPropertyCount () + + Return the number of summary properties, through + ‘MsiSummaryInfoGetPropertyCount()’. + + -- Method: SummaryInformation.SetProperty (field, value) + + Set a property through ‘MsiSummaryInfoSetProperty()’. `field' can + have the same values as in *note GetProperty(): 36af, `value' is + the new value of the property. Possible value types are integer + and string. + + -- Method: SummaryInformation.Persist () + + Write the modified properties to the summary information stream, + using ‘MsiSummaryInfoPersist()’. + +See also +........ + +MsiSummaryInfoGetProperty(1) MsiSummaryInfoGetPropertyCount(2) +MsiSummaryInfoSetProperty(3) MsiSummaryInfoPersist(4) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370409.aspx + + (2) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370488.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370491.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370490.aspx + + +File: python.info, Node: Record Objects, Next: Errors, Prev: Summary Information Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.4 Record Objects +....................... + + -- Method: Record.GetFieldCount () + + Return the number of fields of the record, through + ‘MsiRecordGetFieldCount()’. + + -- Method: Record.GetInteger (field) + + Return the value of `field' as an integer where possible. `field' + must be an integer. + + -- Method: Record.GetString (field) + + Return the value of `field' as a string where possible. `field' + must be an integer. + + -- Method: Record.SetString (field, value) + + Set `field' to `value' through ‘MsiRecordSetString()’. `field' + must be an integer; `value' a string. + + -- Method: Record.SetStream (field, value) + + Set `field' to the contents of the file named `value', through + ‘MsiRecordSetStream()’. `field' must be an integer; `value' a + string. + + -- Method: Record.SetInteger (field, value) + + Set `field' to `value' through ‘MsiRecordSetInteger()’. Both + `field' and `value' must be an integer. + + -- Method: Record.ClearData () + + Set all fields of the record to 0, through ‘MsiRecordClearData()’. + +See also +........ + +MsiRecordGetFieldCount(1) MsiRecordSetString(2) MsiRecordSetStream(3) +MsiRecordSetInteger(4) MsiRecordClearData(5) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370366.aspx + + (2) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370373.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370372.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370371.aspx + + (5) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370364.aspx + + +File: python.info, Node: Errors, Next: CAB Objects, Prev: Record Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.5 Errors +............... + +All wrappers around MSI functions raise ‘MSIError’; the string inside +the exception will contain more detail. + + +File: python.info, Node: CAB Objects, Next: Directory Objects, Prev: Errors, Up: msilib — Read and write Microsoft Installer files + +5.34.1.6 CAB Objects +.................... + + -- Class: msilib.CAB (name) + + The class *note CAB: 36c0. represents a CAB file. During MSI + construction, files will be added simultaneously to the ‘Files’ + table, and to a CAB file. Then, when all files have been added, + the CAB file can be written, then added to the MSI file. + + `name' is the name of the CAB file in the MSI file. + + -- Method: append (full, file, logical) + + Add the file with the pathname `full' to the CAB file, under + the name `logical'. If there is already a file named + `logical', a new file name is created. + + Return the index of the file in the CAB file, and the new name + of the file inside the CAB file. + + -- Method: commit (database) + + Generate a CAB file, add it as a stream to the MSI file, put + it into the ‘Media’ table, and remove the generated file from + the disk. + + +File: python.info, Node: Directory Objects, Next: Features<3>, Prev: CAB Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.7 Directory Objects +.......................... + + -- Class: msilib.Directory (database, cab, basedir, physical, logical, + default[, componentflags]) + + Create a new directory in the Directory table. There is a current + component at each point in time for the directory, which is either + explicitly created through *note start_component(): 36c6, or + implicitly when files are added for the first time. Files are + added into the current component, and into the cab file. To create + a directory, a base directory object needs to be specified (can be + ‘None’), the path to the physical directory, and a logical + directory name. `default' specifies the DefaultDir slot in the + directory table. `componentflags' specifies the default flags that + new components get. + + -- Method: start_component (component=None, feature=None, + flags=None, keyfile=None, uuid=None) + + Add an entry to the Component table, and make this component + the current component for this directory. If no component + name is given, the directory name is used. If no `feature' is + given, the current feature is used. If no `flags' are given, + the directory’s default flags are used. If no `keyfile' is + given, the KeyPath is left null in the Component table. + + -- Method: add_file (file, src=None, version=None, language=None) + + Add a file to the current component of the directory, starting + a new one if there is no current component. By default, the + file name in the source and the file table will be identical. + If the `src' file is specified, it is interpreted relative to + the current directory. Optionally, a `version' and a + `language' can be specified for the entry in the File table. + + -- Method: glob (pattern, exclude=None) + + Add a list of files to the current component as specified in + the glob pattern. Individual files can be excluded in the + `exclude' list. + + -- Method: remove_pyc () + + Remove ‘.pyc’ files on uninstall. + +See also +........ + +Directory Table(1) File Table(2) Component Table(3) FeatureComponents +Table(4) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368295.aspx + + (2) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368596.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368007.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368579.aspx + + +File: python.info, Node: Features<3>, Next: GUI classes, Prev: Directory Objects, Up: msilib — Read and write Microsoft Installer files + +5.34.1.8 Features +................. + + -- Class: msilib.Feature (db, id, title, desc, display, level=1, + parent=None, directory=None, attributes=0) + + Add a new record to the ‘Feature’ table, using the values `id', + `parent.id', `title', `desc', `display', `level', `directory', and + `attributes'. The resulting feature object can be passed to the + ‘start_component()’ method of *note Directory: 36c5. + + -- Method: set_current () + + Make this feature the current feature of *note msilib: b5. + New components are automatically added to the default feature, + unless a feature is explicitly specified. + +See also +........ + +Feature Table(1) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368585.aspx + + +File: python.info, Node: GUI classes, Next: Precomputed tables, Prev: Features<3>, Up: msilib — Read and write Microsoft Installer files + +5.34.1.9 GUI classes +.................... + +*note msilib: b5. provides several classes that wrap the GUI tables in +an MSI database. However, no standard user interface is provided; use +*note bdist_msi: 41. to create MSI files with a user-interface for +installing Python packages. + + -- Class: msilib.Control (dlg, name) + + Base class of the dialog controls. `dlg' is the dialog object the + control belongs to, and `name' is the control’s name. + + -- Method: event (event, argument, condition=1, ordering=None) + + Make an entry into the ‘ControlEvent’ table for this control. + + -- Method: mapping (event, attribute) + + Make an entry into the ‘EventMapping’ table for this control. + + -- Method: condition (action, condition) + + Make an entry into the ‘ControlCondition’ table for this + control. + + -- Class: msilib.RadioButtonGroup (dlg, name, property) + + Create a radio button control named `name'. `property' is the + installer property that gets set when a radio button is selected. + + -- Method: add (name, x, y, width, height, text, value=None) + + Add a radio button named `name' to the group, at the + coordinates `x', `y', `width', `height', and with the label + `text'. If `value' is ‘None’, it defaults to `name'. + + -- Class: msilib.Dialog (db, name, x, y, w, h, attr, title, first, + default, cancel) + + Return a new *note Dialog: 36d6. object. An entry in the ‘Dialog’ + table is made, with the specified coordinates, dialog attributes, + title, name of the first, default, and cancel controls. + + -- Method: control (name, type, x, y, width, height, attributes, + property, text, control_next, help) + + Return a new *note Control: 36d0. object. An entry in the + ‘Control’ table is made with the specified parameters. + + This is a generic method; for specific types, specialized + methods are provided. + + -- Method: text (name, x, y, width, height, attributes, text) + + Add and return a ‘Text’ control. + + -- Method: bitmap (name, x, y, width, height, text) + + Add and return a ‘Bitmap’ control. + + -- Method: line (name, x, y, width, height) + + Add and return a ‘Line’ control. + + -- Method: pushbutton (name, x, y, width, height, attributes, + text, next_control) + + Add and return a ‘PushButton’ control. + + -- Method: radiogroup (name, x, y, width, height, attributes, + property, text, next_control) + + Add and return a ‘RadioButtonGroup’ control. + + -- Method: checkbox (name, x, y, width, height, attributes, + property, text, next_control) + + Add and return a ‘CheckBox’ control. + +See also +........ + +Dialog Table(1) Control Table(2) Control Types(3) ControlCondition +Table(4) ControlEvent Table(5) EventMapping Table(6) RadioButton +Table(7) + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368286.aspx + + (2) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368044.aspx + + (3) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368039.aspx + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368035.aspx + + (5) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368037.aspx + + (6) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa368559.aspx + + (7) +https://msdn.microsoft.com/en-us/library/windows/desktop/aa370962.aspx + + +File: python.info, Node: Precomputed tables, Prev: GUI classes, Up: msilib — Read and write Microsoft Installer files + +5.34.1.10 Precomputed tables +............................ + +*note msilib: b5. provides a few subpackages that contain only schema +and table definitions. Currently, these definitions are based on MSI +version 2.0. + + -- Data: msilib.schema + + This is the standard MSI schema for MSI 2.0, with the `tables' + variable providing a list of table definitions, and + `_Validation_records' providing the data for MSI validation. + + -- Data: msilib.sequence + + This module contains table contents for the standard sequence + tables: `AdminExecuteSequence', `AdminUISequence', + `AdvtExecuteSequence', `InstallExecuteSequence', and + `InstallUISequence'. + + -- Data: msilib.text + + This module contains definitions for the UIText and ActionText + tables, for the standard installer actions. + + +File: python.info, Node: msvcrt — Useful routines from the MS VC++ runtime, Next: winreg — Windows registry access, Prev: msilib — Read and write Microsoft Installer files, Up: MS Windows Specific Services + +5.34.2 ‘msvcrt’ — Useful routines from the MS VC++ runtime +---------------------------------------------------------- + +__________________________________________________________________ + +These functions provide access to some useful capabilities on Windows +platforms. Some higher-level modules use these functions to build the +Windows implementations of their services. For example, the *note +getpass: 88. module uses this in the implementation of the *note +getpass(): 88. function. + +Further documentation on these functions can be found in the Platform +API documentation. + +The module implements both the normal and wide char variants of the +console I/O api. The normal API deals only with ASCII characters and is +of limited use for internationalized applications. The wide char API +should be used where ever possible. + +Changed in version 3.3: Operations in this module now raise *note +OSError: 1d3. where *note IOError: 992. was raised. + +* Menu: + +* File Operations:: +* Console I/O:: +* Other Functions:: + + +File: python.info, Node: File Operations, Next: Console I/O, Up: msvcrt — Useful routines from the MS VC++ runtime + +5.34.2.1 File Operations +........................ + + -- Function: msvcrt.locking (fd, mode, nbytes) + + Lock part of a file based on file descriptor `fd' from the C + runtime. Raises *note OSError: 1d3. on failure. The locked region + of the file extends from the current file position for `nbytes' + bytes, and may continue beyond the end of the file. `mode' must be + one of the ‘LK_*’ constants listed below. Multiple regions in a + file may be locked at the same time, but may not overlap. Adjacent + regions are not merged; they must be unlocked individually. + + Raises an *note auditing event: fd1. ‘msvcrt.locking’ with + arguments ‘fd’, ‘mode’, ‘nbytes’. + + -- Data: msvcrt.LK_LOCK + -- Data: msvcrt.LK_RLCK + + Locks the specified bytes. If the bytes cannot be locked, the + program immediately tries again after 1 second. If, after 10 + attempts, the bytes cannot be locked, *note OSError: 1d3. is + raised. + + -- Data: msvcrt.LK_NBLCK + -- Data: msvcrt.LK_NBRLCK + + Locks the specified bytes. If the bytes cannot be locked, *note + OSError: 1d3. is raised. + + -- Data: msvcrt.LK_UNLCK + + Unlocks the specified bytes, which must have been previously + locked. + + -- Function: msvcrt.setmode (fd, flags) + + Set the line-end translation mode for the file descriptor `fd'. To + set it to text mode, `flags' should be *note os.O_TEXT: 1bfc.; for + binary, it should be *note os.O_BINARY: 1bed. + + -- Function: msvcrt.open_osfhandle (handle, flags) + + Create a C runtime file descriptor from the file handle `handle'. + The `flags' parameter should be a bitwise OR of *note os.O_APPEND: + 1bef, *note os.O_RDONLY: 1beb, and *note os.O_TEXT: 1bfc. The + returned file descriptor may be used as a parameter to *note + os.fdopen(): 10d2. to create a file object. + + Raises an *note auditing event: fd1. ‘msvcrt.open_osfhandle’ with + arguments ‘handle’, ‘flags’. + + -- Function: msvcrt.get_osfhandle (fd) + + Return the file handle for the file descriptor `fd'. Raises *note + OSError: 1d3. if `fd' is not recognized. + + Raises an *note auditing event: fd1. ‘msvcrt.get_osfhandle’ with + argument ‘fd’. + + +File: python.info, Node: Console I/O, Next: Other Functions, Prev: File Operations, Up: msvcrt — Useful routines from the MS VC++ runtime + +5.34.2.2 Console I/O +.................... + + -- Function: msvcrt.kbhit () + + Return ‘True’ if a keypress is waiting to be read. + + -- Function: msvcrt.getch () + + Read a keypress and return the resulting character as a byte + string. Nothing is echoed to the console. This call will block if + a keypress is not already available, but will not wait for ‘Enter’ + to be pressed. If the pressed key was a special function key, this + will return ‘'\000'’ or ‘'\xe0'’; the next call will return the + keycode. The ‘Control-C’ keypress cannot be read with this + function. + + -- Function: msvcrt.getwch () + + Wide char variant of *note getch(): 36ef, returning a Unicode + value. + + -- Function: msvcrt.getche () + + Similar to *note getch(): 36ef, but the keypress will be echoed if + it represents a printable character. + + -- Function: msvcrt.getwche () + + Wide char variant of *note getche(): 36f1, returning a Unicode + value. + + -- Function: msvcrt.putch (char) + + Print the byte string `char' to the console without buffering. + + -- Function: msvcrt.putwch (unicode_char) + + Wide char variant of *note putch(): 36f3, accepting a Unicode + value. + + -- Function: msvcrt.ungetch (char) + + Cause the byte string `char' to be “pushed back” into the console + buffer; it will be the next character read by *note getch(): 36ef. + or *note getche(): 36f1. + + -- Function: msvcrt.ungetwch (unicode_char) + + Wide char variant of *note ungetch(): 36f5, accepting a Unicode + value. + + +File: python.info, Node: Other Functions, Prev: Console I/O, Up: msvcrt — Useful routines from the MS VC++ runtime + +5.34.2.3 Other Functions +........................ + + -- Function: msvcrt.heapmin () + + Force the ‘malloc()’ heap to clean itself up and return unused + blocks to the operating system. On failure, this raises *note + OSError: 1d3. + + +File: python.info, Node: winreg — Windows registry access, Next: winsound — Sound-playing interface for Windows, Prev: msvcrt — Useful routines from the MS VC++ runtime, Up: MS Windows Specific Services + +5.34.3 ‘winreg’ — Windows registry access +----------------------------------------- + +__________________________________________________________________ + +These functions expose the Windows registry API to Python. Instead of +using an integer as the registry handle, a *note handle object: 36fc. is +used to ensure that the handles are closed correctly, even if the +programmer neglects to explicitly close them. + +Changed in version 3.3: Several functions in this module used to raise a +*note WindowsError: 994, which is now an alias of *note OSError: 1d3. + +* Menu: + +* Functions: Functions<11>. +* Constants: Constants<10>. +* Registry Handle Objects:: + + +File: python.info, Node: Functions<11>, Next: Constants<10>, Up: winreg — Windows registry access + +5.34.3.1 Functions +.................. + +This module offers the following functions: + + -- Function: winreg.CloseKey (hkey) + + Closes a previously opened registry key. The `hkey' argument + specifies a previously opened key. + + Note: If `hkey' is not closed using this method (or via *note + hkey.Close(): 3701.), it is closed when the `hkey' object is + destroyed by Python. + + -- Function: winreg.ConnectRegistry (computer_name, key) + + Establishes a connection to a predefined registry handle on another + computer, and returns a *note handle object: 36fc. + + `computer_name' is the name of the remote computer, of the form + ‘r"\\computername"’. If ‘None’, the local computer is used. + + `key' is the predefined handle to connect to. + + The return value is the handle of the opened key. If the function + fails, an *note OSError: 1d3. exception is raised. + + Raises an *note auditing event: fd1. ‘winreg.ConnectRegistry’ with + arguments ‘computer_name’, ‘key’. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.CreateKey (key, sub_key) + + Creates or opens the specified key, returning a *note handle + object: 36fc. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that names the key this method opens or + creates. + + If `key' is one of the predefined keys, `sub_key' may be ‘None’. + In that case, the handle returned is the same key handle passed in + to the function. + + If the key already exists, this function opens the existing key. + + The return value is the handle of the opened key. If the function + fails, an *note OSError: 1d3. exception is raised. + + Raises an *note auditing event: fd1. ‘winreg.CreateKey’ with + arguments ‘key’, ‘sub_key’, ‘access’. + + Raises an *note auditing event: fd1. ‘winreg.OpenKey/result’ with + argument ‘key’. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.CreateKeyEx (key, sub_key, reserved=0, + access=KEY_WRITE) + + Creates or opens the specified key, returning a *note handle + object: 36fc. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that names the key this method opens or + creates. + + `reserved' is a reserved integer, and must be zero. The default is + zero. + + `access' is an integer that specifies an access mask that describes + the desired security access for the key. Default is *note + KEY_WRITE: 3703. See *note Access Rights: 3704. for other allowed + values. + + If `key' is one of the predefined keys, `sub_key' may be ‘None’. + In that case, the handle returned is the same key handle passed in + to the function. + + If the key already exists, this function opens the existing key. + + The return value is the handle of the opened key. If the function + fails, an *note OSError: 1d3. exception is raised. + + Raises an *note auditing event: fd1. ‘winreg.CreateKey’ with + arguments ‘key’, ‘sub_key’, ‘access’. + + Raises an *note auditing event: fd1. ‘winreg.OpenKey/result’ with + argument ‘key’. + + New in version 3.2. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.DeleteKey (key, sub_key) + + Deletes the specified key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that must be a subkey of the key identified + by the `key' parameter. This value must not be ‘None’, and the key + may not have subkeys. + + `This method can not delete keys with subkeys.' + + If the method succeeds, the entire key, including all of its + values, is removed. If the method fails, an *note OSError: 1d3. + exception is raised. + + Raises an *note auditing event: fd1. ‘winreg.DeleteKey’ with + arguments ‘key’, ‘sub_key’, ‘access’. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.DeleteKeyEx (key, sub_key, access=KEY_WOW64_64KEY, + reserved=0) + + Deletes the specified key. + + Note: The *note DeleteKeyEx(): 3201. function is implemented + with the RegDeleteKeyEx Windows API function, which is + specific to 64-bit versions of Windows. See the + RegDeleteKeyEx documentation(1). + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that must be a subkey of the key identified + by the `key' parameter. This value must not be ‘None’, and the key + may not have subkeys. + + `reserved' is a reserved integer, and must be zero. The default is + zero. + + `access' is an integer that specifies an access mask that describes + the desired security access for the key. Default is *note + KEY_WOW64_64KEY: 3705. See *note Access Rights: 3704. for other + allowed values. + + `This method can not delete keys with subkeys.' + + If the method succeeds, the entire key, including all of its + values, is removed. If the method fails, an *note OSError: 1d3. + exception is raised. + + On unsupported Windows versions, *note NotImplementedError: 60e. is + raised. + + Raises an *note auditing event: fd1. ‘winreg.DeleteKey’ with + arguments ‘key’, ‘sub_key’, ‘access’. + + New in version 3.2. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.DeleteValue (key, value) + + Removes a named value from a registry key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `value' is a string that identifies the value to remove. + + Raises an *note auditing event: fd1. ‘winreg.DeleteValue’ with + arguments ‘key’, ‘value’. + + -- Function: winreg.EnumKey (key, index) + + Enumerates subkeys of an open registry key, returning a string. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `index' is an integer that identifies the index of the key to + retrieve. + + The function retrieves the name of one subkey each time it is + called. It is typically called repeatedly until an *note OSError: + 1d3. exception is raised, indicating, no more values are available. + + Raises an *note auditing event: fd1. ‘winreg.EnumKey’ with + arguments ‘key’, ‘index’. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.EnumValue (key, index) + + Enumerates values of an open registry key, returning a tuple. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `index' is an integer that identifies the index of the value to + retrieve. + + The function retrieves the name of one subkey each time it is + called. It is typically called repeatedly, until an *note OSError: + 1d3. exception is raised, indicating no more values. + + The result is a tuple of 3 items: + + Index Meaning + + ------------------------------------------------------------- + + ‘0’ A string that identifies the value name + + + ‘1’ An object that holds the value data, and whose + type depends on the underlying registry type + + + ‘2’ An integer that identifies the type of the + value data (see table in docs for + *note SetValueEx(): 3211.) + + + Raises an *note auditing event: fd1. ‘winreg.EnumValue’ with + arguments ‘key’, ‘index’. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.ExpandEnvironmentStrings (str) + + Expands environment variable placeholders ‘%NAME%’ in strings like + *note REG_EXPAND_SZ: 3706.: + + >>> ExpandEnvironmentStrings('%windir%') + 'C:\\Windows' + + Raises an *note auditing event: fd1. + ‘winreg.ExpandEnvironmentStrings’ with argument ‘str’. + + -- Function: winreg.FlushKey (key) + + Writes all the attributes of a key to the registry. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + It is not necessary to call *note FlushKey(): 3707. to change a + key. Registry changes are flushed to disk by the registry using + its lazy flusher. Registry changes are also flushed to disk at + system shutdown. Unlike *note CloseKey(): 3700, the *note + FlushKey(): 3707. method returns only when all the data has been + written to the registry. An application should only call *note + FlushKey(): 3707. if it requires absolute certainty that registry + changes are on disk. + + Note: If you don’t know whether a *note FlushKey(): 3707. call + is required, it probably isn’t. + + -- Function: winreg.LoadKey (key, sub_key, file_name) + + Creates a subkey under the specified key and stores registration + information from a specified file into that subkey. + + `key' is a handle returned by *note ConnectRegistry(): 31fd. or one + of the constants *note HKEY_USERS: 3708. or *note + HKEY_LOCAL_MACHINE: 3709. + + `sub_key' is a string that identifies the subkey to load. + + `file_name' is the name of the file to load registry data from. + This file must have been created with the *note SaveKey(): 320f. + function. Under the file allocation table (FAT) file system, the + filename may not have an extension. + + A call to *note LoadKey(): 3208. fails if the calling process does + not have the ‘SE_RESTORE_PRIVILEGE’ privilege. Note that + privileges are different from permissions – see the RegLoadKey + documentation(2) for more details. + + If `key' is a handle returned by *note ConnectRegistry(): 31fd, + then the path specified in `file_name' is relative to the remote + computer. + + Raises an *note auditing event: fd1. ‘winreg.LoadKey’ with + arguments ‘key’, ‘sub_key’, ‘file_name’. + + -- Function: winreg.OpenKey (key, sub_key, reserved=0, access=KEY_READ) + -- Function: winreg.OpenKeyEx (key, sub_key, reserved=0, + access=KEY_READ) + + Opens the specified key, returning a *note handle object: 36fc. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that identifies the sub_key to open. + + `reserved' is a reserved integer, and must be zero. The default is + zero. + + `access' is an integer that specifies an access mask that describes + the desired security access for the key. Default is *note + KEY_READ: 370b. See *note Access Rights: 3704. for other allowed + values. + + The result is a new handle to the specified key. + + If the function fails, *note OSError: 1d3. is raised. + + Raises an *note auditing event: fd1. ‘winreg.OpenKey’ with + arguments ‘key’, ‘sub_key’, ‘access’. + + Raises an *note auditing event: fd1. ‘winreg.OpenKey/result’ with + argument ‘key’. + + Changed in version 3.2: Allow the use of named arguments. + + Changed in version 3.3: See *note above: 36fd. + + -- Function: winreg.QueryInfoKey (key) + + Returns information about a key, as a tuple. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + The result is a tuple of 3 items: + + Index Meaning + + -------------------------------------------------------------- + + ‘0’ An integer giving the number of sub keys this + key has. + + + ‘1’ An integer giving the number of values this key + has. + + + ‘2’ An integer giving when the key was last + modified (if available) as 100’s of nanoseconds + since Jan 1, 1601. + + + Raises an *note auditing event: fd1. ‘winreg.QueryInfoKey’ with + argument ‘key’. + + -- Function: winreg.QueryValue (key, sub_key) + + Retrieves the unnamed value for a key, as a string. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that holds the name of the subkey with which + the value is associated. If this parameter is ‘None’ or empty, the + function retrieves the value set by the *note SetValue(): 3210. + method for the key identified by `key'. + + Values in the registry have name, type, and data components. This + method retrieves the data for a key’s first value that has a ‘NULL’ + name. But the underlying API call doesn’t return the type, so + always use *note QueryValueEx(): 320e. if possible. + + Raises an *note auditing event: fd1. ‘winreg.QueryValue’ with + arguments ‘key’, ‘sub_key’, ‘value_name’. + + -- Function: winreg.QueryValueEx (key, value_name) + + Retrieves the type and data for a specified value name associated + with an open registry key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `value_name' is a string indicating the value to query. + + The result is a tuple of 2 items: + + Index Meaning + + ---------------------------------------------------------- + + ‘0’ The value of the registry item. + + + ‘1’ An integer giving the registry type for + this value (see table in docs for + *note SetValueEx(): 3211.) + + + Raises an *note auditing event: fd1. ‘winreg.QueryValue’ with + arguments ‘key’, ‘sub_key’, ‘value_name’. + + -- Function: winreg.SaveKey (key, file_name) + + Saves the specified key, and all its subkeys to the specified file. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `file_name' is the name of the file to save registry data to. This + file cannot already exist. If this filename includes an extension, + it cannot be used on file allocation table (FAT) file systems by + the *note LoadKey(): 3208. method. + + If `key' represents a key on a remote computer, the path described + by `file_name' is relative to the remote computer. The caller of + this method must possess the ‘SeBackupPrivilege’ security + privilege. Note that privileges are different than permissions – + see the Conflicts Between User Rights and Permissions + documentation(3) for more details. + + This function passes ‘NULL’ for `security_attributes' to the API. + + Raises an *note auditing event: fd1. ‘winreg.SaveKey’ with + arguments ‘key’, ‘file_name’. + + -- Function: winreg.SetValue (key, sub_key, type, value) + + Associates a value with a specified key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `sub_key' is a string that names the subkey with which the value is + associated. + + `type' is an integer that specifies the type of the data. + Currently this must be *note REG_SZ: 370c, meaning only strings are + supported. Use the *note SetValueEx(): 3211. function for support + for other data types. + + `value' is a string that specifies the new value. + + If the key specified by the `sub_key' parameter does not exist, the + SetValue function creates it. + + Value lengths are limited by available memory. Long values (more + than 2048 bytes) should be stored as files with the filenames + stored in the configuration registry. This helps the registry + perform efficiently. + + The key identified by the `key' parameter must have been opened + with *note KEY_SET_VALUE: 370d. access. + + Raises an *note auditing event: fd1. ‘winreg.SetValue’ with + arguments ‘key’, ‘sub_key’, ‘type’, ‘value’. + + -- Function: winreg.SetValueEx (key, value_name, reserved, type, value) + + Stores data in the value field of an open registry key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + `value_name' is a string that names the subkey with which the value + is associated. + + `reserved' can be anything – zero is always passed to the API. + + `type' is an integer that specifies the type of the data. See + *note Value Types: 370e. for the available types. + + `value' is a string that specifies the new value. + + This method can also set additional value and type information for + the specified key. The key identified by the key parameter must + have been opened with *note KEY_SET_VALUE: 370d. access. + + To open the key, use the *note CreateKey(): 31fe. or *note + OpenKey(): 3209. methods. + + Value lengths are limited by available memory. Long values (more + than 2048 bytes) should be stored as files with the filenames + stored in the configuration registry. This helps the registry + perform efficiently. + + Raises an *note auditing event: fd1. ‘winreg.SetValue’ with + arguments ‘key’, ‘sub_key’, ‘type’, ‘value’. + + -- Function: winreg.DisableReflectionKey (key) + + Disables registry reflection for 32-bit processes running on a + 64-bit operating system. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + Will generally raise *note NotImplementedError: 60e. if executed on + a 32-bit operating system. + + If the key is not on the reflection list, the function succeeds but + has no effect. Disabling reflection for a key does not affect + reflection of any subkeys. + + Raises an *note auditing event: fd1. ‘winreg.DisableReflectionKey’ + with argument ‘key’. + + -- Function: winreg.EnableReflectionKey (key) + + Restores registry reflection for the specified disabled key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + Will generally raise *note NotImplementedError: 60e. if executed on + a 32-bit operating system. + + Restoring reflection for a key does not affect reflection of any + subkeys. + + Raises an *note auditing event: fd1. ‘winreg.EnableReflectionKey’ + with argument ‘key’. + + -- Function: winreg.QueryReflectionKey (key) + + Determines the reflection state for the specified key. + + `key' is an already open key, or one of the predefined *note HKEY_* + constants: 3702. + + Returns ‘True’ if reflection is disabled. + + Will generally raise *note NotImplementedError: 60e. if executed on + a 32-bit operating system. + + Raises an *note auditing event: fd1. ‘winreg.QueryReflectionKey’ + with argument ‘key’. + + ---------- Footnotes ---------- + + (1) https://msdn.microsoft.com/en-us/library/ms724847%28VS.85%29.aspx + + (2) +https://msdn.microsoft.com/en-us/library/ms724889%28v=VS.85%29.aspx + + (3) +https://msdn.microsoft.com/en-us/library/ms724878%28v=VS.85%29.aspx + + +File: python.info, Node: Constants<10>, Next: Registry Handle Objects, Prev: Functions<11>, Up: winreg — Windows registry access + +5.34.3.2 Constants +.................. + +The following constants are defined for use in many ‘_winreg’ functions. + +* Menu: + +* HKEY_* Constants:: +* Access Rights:: +* Value Types:: + + +File: python.info, Node: HKEY_* Constants, Next: Access Rights, Up: Constants<10> + +5.34.3.3 HKEY_* Constants +......................... + + -- Data: winreg.HKEY_CLASSES_ROOT + + Registry entries subordinate to this key define types (or classes) + of documents and the properties associated with those types. Shell + and COM applications use the information stored under this key. + + -- Data: winreg.HKEY_CURRENT_USER + + Registry entries subordinate to this key define the preferences of + the current user. These preferences include the settings of + environment variables, data about program groups, colors, printers, + network connections, and application preferences. + + -- Data: winreg.HKEY_LOCAL_MACHINE + + Registry entries subordinate to this key define the physical state + of the computer, including data about the bus type, system memory, + and installed hardware and software. + + -- Data: winreg.HKEY_USERS + + Registry entries subordinate to this key define the default user + configuration for new users on the local computer and the user + configuration for the current user. + + -- Data: winreg.HKEY_PERFORMANCE_DATA + + Registry entries subordinate to this key allow you to access + performance data. The data is not actually stored in the registry; + the registry functions cause the system to collect the data from + its source. + + -- Data: winreg.HKEY_CURRENT_CONFIG + + Contains information about the current hardware profile of the + local computer system. + + -- Data: winreg.HKEY_DYN_DATA + + This key is not used in versions of Windows after 98. + + +File: python.info, Node: Access Rights, Next: Value Types, Prev: HKEY_* Constants, Up: Constants<10> + +5.34.3.4 Access Rights +...................... + +For more information, see Registry Key Security and Access(1). + + -- Data: winreg.KEY_ALL_ACCESS + + Combines the STANDARD_RIGHTS_REQUIRED, *note KEY_QUERY_VALUE: 3719, + *note KEY_SET_VALUE: 370d, *note KEY_CREATE_SUB_KEY: 371a, *note + KEY_ENUMERATE_SUB_KEYS: 371b, *note KEY_NOTIFY: 371c, and *note + KEY_CREATE_LINK: 371d. access rights. + + -- Data: winreg.KEY_WRITE + + Combines the STANDARD_RIGHTS_WRITE, *note KEY_SET_VALUE: 370d, and + *note KEY_CREATE_SUB_KEY: 371a. access rights. + + -- Data: winreg.KEY_READ + + Combines the STANDARD_RIGHTS_READ, *note KEY_QUERY_VALUE: 3719, + *note KEY_ENUMERATE_SUB_KEYS: 371b, and *note KEY_NOTIFY: 371c. + values. + + -- Data: winreg.KEY_EXECUTE + + Equivalent to *note KEY_READ: 370b. + + -- Data: winreg.KEY_QUERY_VALUE + + Required to query the values of a registry key. + + -- Data: winreg.KEY_SET_VALUE + + Required to create, delete, or set a registry value. + + -- Data: winreg.KEY_CREATE_SUB_KEY + + Required to create a subkey of a registry key. + + -- Data: winreg.KEY_ENUMERATE_SUB_KEYS + + Required to enumerate the subkeys of a registry key. + + -- Data: winreg.KEY_NOTIFY + + Required to request change notifications for a registry key or for + subkeys of a registry key. + + -- Data: winreg.KEY_CREATE_LINK + + Reserved for system use. + +* Menu: + +* 64-bit Specific:: + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/ms724878%28v=VS.85%29.aspx + + +File: python.info, Node: 64-bit Specific, Up: Access Rights + +5.34.3.5 64-bit Specific +........................ + +For more information, see Accessing an Alternate Registry View(1). + + -- Data: winreg.KEY_WOW64_64KEY + + Indicates that an application on 64-bit Windows should operate on + the 64-bit registry view. + + -- Data: winreg.KEY_WOW64_32KEY + + Indicates that an application on 64-bit Windows should operate on + the 32-bit registry view. + + ---------- Footnotes ---------- + + (1) https://msdn.microsoft.com/en-us/library/aa384129(v=VS.85).aspx + + +File: python.info, Node: Value Types, Prev: Access Rights, Up: Constants<10> + +5.34.3.6 Value Types +.................... + +For more information, see Registry Value Types(1). + + -- Data: winreg.REG_BINARY + + Binary data in any form. + + -- Data: winreg.REG_DWORD + + 32-bit number. + + -- Data: winreg.REG_DWORD_LITTLE_ENDIAN + + A 32-bit number in little-endian format. Equivalent to *note + REG_DWORD: 3724. + + -- Data: winreg.REG_DWORD_BIG_ENDIAN + + A 32-bit number in big-endian format. + + -- Data: winreg.REG_EXPAND_SZ + + Null-terminated string containing references to environment + variables (‘%PATH%’). + + -- Data: winreg.REG_LINK + + A Unicode symbolic link. + + -- Data: winreg.REG_MULTI_SZ + + A sequence of null-terminated strings, terminated by two null + characters. (Python handles this termination automatically.) + + -- Data: winreg.REG_NONE + + No defined value type. + + -- Data: winreg.REG_QWORD + + A 64-bit number. + + New in version 3.6. + + -- Data: winreg.REG_QWORD_LITTLE_ENDIAN + + A 64-bit number in little-endian format. Equivalent to *note + REG_QWORD: 5b2. + + New in version 3.6. + + -- Data: winreg.REG_RESOURCE_LIST + + A device-driver resource list. + + -- Data: winreg.REG_FULL_RESOURCE_DESCRIPTOR + + A hardware setting. + + -- Data: winreg.REG_RESOURCE_REQUIREMENTS_LIST + + A hardware resource list. + + -- Data: winreg.REG_SZ + + A null-terminated string. + + ---------- Footnotes ---------- + + (1) +https://msdn.microsoft.com/en-us/library/ms724884%28v=VS.85%29.aspx + + +File: python.info, Node: Registry Handle Objects, Prev: Constants<10>, Up: winreg — Windows registry access + +5.34.3.7 Registry Handle Objects +................................ + +This object wraps a Windows HKEY object, automatically closing it when +the object is destroyed. To guarantee cleanup, you can call either the +*note Close(): 3701. method on the object, or the *note CloseKey(): +3700. function. + +All registry functions in this module return one of these objects. + +All registry functions in this module which accept a handle object also +accept an integer, however, use of the handle object is encouraged. + +Handle objects provide semantics for *note __bool__(): c68. – thus + + if handle: + print("Yes") + +will print ‘Yes’ if the handle is currently valid (has not been closed +or detached). + +The object also support comparison semantics, so handle objects will +compare true if they both reference the same underlying Windows handle +value. + +Handle objects can be converted to an integer (e.g., using the built-in +*note int(): 184. function), in which case the underlying Windows handle +value is returned. You can also use the *note Detach(): 320a. method to +return the integer handle, and also disconnect the Windows handle from +the handle object. + + -- Method: PyHKEY.Close () + + Closes the underlying Windows handle. + + If the handle is already closed, no error is raised. + + -- Method: PyHKEY.Detach () + + Detaches the Windows handle from the handle object. + + The result is an integer that holds the value of the handle before + it is detached. If the handle is already detached or closed, this + will return zero. + + After calling this function, the handle is effectively invalidated, + but the handle is not closed. You would call this function when + you need the underlying Win32 handle to exist beyond the lifetime + of the handle object. + + Raises an *note auditing event: fd1. ‘winreg.PyHKEY.Detach’ with + argument ‘key’. + + -- Method: PyHKEY.__enter__ () + -- Method: PyHKEY.__exit__ (*exc_info) + + The HKEY object implements *note __enter__(): c95. and *note + __exit__(): c96. and thus supports the context protocol for the + *note with: 6e9. statement: + + with OpenKey(HKEY_LOCAL_MACHINE, "foo") as key: + ... # work with key + + will automatically close `key' when control leaves the *note with: + 6e9. block. + + +File: python.info, Node: winsound — Sound-playing interface for Windows, Prev: winreg — Windows registry access, Up: MS Windows Specific Services + +5.34.4 ‘winsound’ — Sound-playing interface for Windows +------------------------------------------------------- + +__________________________________________________________________ + +The *note winsound: 12b. module provides access to the basic +sound-playing machinery provided by Windows platforms. It includes +functions and several constants. + + -- Function: winsound.Beep (frequency, duration) + + Beep the PC’s speaker. The `frequency' parameter specifies + frequency, in hertz, of the sound, and must be in the range 37 + through 32,767. The `duration' parameter specifies the number of + milliseconds the sound should last. If the system is not able to + beep the speaker, *note RuntimeError: 2ba. is raised. + + -- Function: winsound.PlaySound (sound, flags) + + Call the underlying ‘PlaySound()’ function from the Platform API. + The `sound' parameter may be a filename, a system sound alias, + audio data as a *note bytes-like object: 5e8, or ‘None’. Its + interpretation depends on the value of `flags', which can be a + bitwise ORed combination of the constants described below. If the + `sound' parameter is ‘None’, any currently playing waveform sound + is stopped. If the system indicates an error, *note RuntimeError: + 2ba. is raised. + + -- Function: winsound.MessageBeep (type=MB_OK) + + Call the underlying ‘MessageBeep()’ function from the Platform API. + This plays a sound as specified in the registry. The `type' + argument specifies which sound to play; possible values are ‘-1’, + ‘MB_ICONASTERISK’, ‘MB_ICONEXCLAMATION’, ‘MB_ICONHAND’, + ‘MB_ICONQUESTION’, and ‘MB_OK’, all described below. The value + ‘-1’ produces a “simple beep”; this is the final fallback if a + sound cannot be played otherwise. If the system indicates an + error, *note RuntimeError: 2ba. is raised. + + -- Data: winsound.SND_FILENAME + + The `sound' parameter is the name of a WAV file. Do not use with + *note SND_ALIAS: 3734. + + -- Data: winsound.SND_ALIAS + + The `sound' parameter is a sound association name from the + registry. If the registry contains no such name, play the system + default sound unless *note SND_NODEFAULT: 3735. is also specified. + If no default sound is registered, raise *note RuntimeError: 2ba. + Do not use with *note SND_FILENAME: 3733. + + All Win32 systems support at least the following; most systems + support many more: + + *note PlaySound(): 5b6. Corresponding Control Panel Sound name + `name' + + ---------------------------------------------------------------------------- + + ‘'SystemAsterisk'’ Asterisk + + + ‘'SystemExclamation'’ Exclamation + + + ‘'SystemExit'’ Exit Windows + + + ‘'SystemHand'’ Critical Stop + + + ‘'SystemQuestion'’ Question + + + For example: + + import winsound + # Play Windows exit sound. + winsound.PlaySound("SystemExit", winsound.SND_ALIAS) + + # Probably play Windows default sound, if any is registered (because + # "*" probably isn't the registered name of any sound). + winsound.PlaySound("*", winsound.SND_ALIAS) + + -- Data: winsound.SND_LOOP + + Play the sound repeatedly. The *note SND_ASYNC: 3737. flag must + also be used to avoid blocking. Cannot be used with *note + SND_MEMORY: 3738. + + -- Data: winsound.SND_MEMORY + + The `sound' parameter to *note PlaySound(): 5b6. is a memory image + of a WAV file, as a *note bytes-like object: 5e8. + + Note: This module does not support playing from a memory image + asynchronously, so a combination of this flag and *note + SND_ASYNC: 3737. will raise *note RuntimeError: 2ba. + + -- Data: winsound.SND_PURGE + + Stop playing all instances of the specified sound. + + Note: This flag is not supported on modern Windows platforms. + + -- Data: winsound.SND_ASYNC + + Return immediately, allowing sounds to play asynchronously. + + -- Data: winsound.SND_NODEFAULT + + If the specified sound cannot be found, do not play the system + default sound. + + -- Data: winsound.SND_NOSTOP + + Do not interrupt sounds currently playing. + + -- Data: winsound.SND_NOWAIT + + Return immediately if the sound driver is busy. + + Note: This flag is not supported on modern Windows platforms. + + -- Data: winsound.MB_ICONASTERISK + + Play the ‘SystemDefault’ sound. + + -- Data: winsound.MB_ICONEXCLAMATION + + Play the ‘SystemExclamation’ sound. + + -- Data: winsound.MB_ICONHAND + + Play the ‘SystemHand’ sound. + + -- Data: winsound.MB_ICONQUESTION + + Play the ‘SystemQuestion’ sound. + + -- Data: winsound.MB_OK + + Play the ‘SystemDefault’ sound. + + +File: python.info, Node: Unix Specific Services, Next: Superseded Modules, Prev: MS Windows Specific Services, Up: The Python Standard Library + +5.35 Unix Specific Services +=========================== + +The modules described in this chapter provide interfaces to features +that are unique to the Unix operating system, or in some cases to some +or many variants of it. Here’s an overview: + +* Menu: + +* posix — The most common POSIX system calls:: +* pwd — The password database:: +* spwd — The shadow password database:: +* grp — The group database:: +* crypt — Function to check Unix passwords:: +* termios — POSIX style tty control:: +* tty — Terminal control functions:: +* pty — Pseudo-terminal utilities:: +* fcntl — The fcntl and ioctl system calls:: +* pipes — Interface to shell pipelines:: +* resource — Resource usage information:: +* nis — Interface to Sun’s NIS (Yellow Pages): nis — Interface to Sun’s NIS Yellow Pages. +* syslog — Unix syslog library routines:: + + +File: python.info, Node: posix — The most common POSIX system calls, Next: pwd — The password database, Up: Unix Specific Services + +5.35.1 ‘posix’ — The most common POSIX system calls +--------------------------------------------------- + +__________________________________________________________________ + +This module provides access to operating system functionality that is +standardized by the C Standard and the POSIX standard (a thinly +disguised Unix interface). + +`Do not import this module directly.' Instead, import the module *note +os: c4, which provides a `portable' version of this interface. On Unix, +the *note os: c4. module provides a superset of the *note posix: d1. +interface. On non-Unix operating systems the *note posix: d1. module is +not available, but a subset is always available through the *note os: +c4. interface. Once *note os: c4. is imported, there is `no' +performance penalty in using it instead of *note posix: d1. In +addition, *note os: c4. provides some additional functionality, such as +automatically calling *note putenv(): 1bb6. when an entry in +‘os.environ’ is changed. + +Errors are reported as exceptions; the usual exceptions are given for +type errors, while errors reported by the system calls raise *note +OSError: 1d3. + +* Menu: + +* Large File Support:: +* Notable Module Contents:: + + +File: python.info, Node: Large File Support, Next: Notable Module Contents, Up: posix — The most common POSIX system calls + +5.35.1.1 Large File Support +........................... + +Several operating systems (including AIX, HP-UX, Irix and Solaris) +provide support for files that are larger than 2 GiB from a C +programming model where ‘int’ and ‘long’ are 32-bit values. This is +typically accomplished by defining the relevant size and offset types as +64-bit values. Such files are sometimes referred to as `large files'. + +Large file support is enabled in Python when the size of an ‘off_t’ is +larger than a ‘long’ and the ‘long long’ is at least as large as an +‘off_t’. It may be necessary to configure and compile Python with +certain compiler flags to enable this mode. For example, it is enabled +by default with recent versions of Irix, but with Solaris 2.6 and 2.7 +you need to do something like: + + CFLAGS="`getconf LFS_CFLAGS`" OPT="-g -O2 $CFLAGS" \ + ./configure + +On large-file-capable Linux systems, this might work: + + CFLAGS='-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64' OPT="-g -O2 $CFLAGS" \ + ./configure + + +File: python.info, Node: Notable Module Contents, Prev: Large File Support, Up: posix — The most common POSIX system calls + +5.35.1.2 Notable Module Contents +................................ + +In addition to many functions described in the *note os: c4. module +documentation, *note posix: d1. defines the following data item: + + -- Data: posix.environ + + A dictionary representing the string environment at the time the + interpreter was started. Keys and values are bytes on Unix and str + on Windows. For example, ‘environ[b'HOME']’ (‘environ['HOME']’ on + Windows) is the pathname of your home directory, equivalent to + ‘getenv("HOME")’ in C. + + Modifying this dictionary does not affect the string environment + passed on by *note execv(): 1bc9, *note popen(): 2a9. or *note + system(): dc1.; if you need to change the environment, pass + ‘environ’ to *note execve(): a40. or add variable assignments and + export statements to the command string for *note system(): dc1. or + *note popen(): 2a9. + + Changed in version 3.2: On Unix, keys and values are bytes. + + Note: The *note os: c4. module provides an alternate + implementation of ‘environ’ which updates the environment on + modification. Note also that updating *note os.environ: b37. + will render this dictionary obsolete. Use of the *note os: + c4. module version of this is recommended over direct access + to the *note posix: d1. module. + + +File: python.info, Node: pwd — The password database, Next: spwd — The shadow password database, Prev: posix — The most common POSIX system calls, Up: Unix Specific Services + +5.35.2 ‘pwd’ — The password database +------------------------------------ + +__________________________________________________________________ + +This module provides access to the Unix user account and password +database. It is available on all Unix versions. + +Password database entries are reported as a tuple-like object, whose +attributes correspond to the members of the ‘passwd’ structure +(Attribute field below, see ‘<pwd.h>’): + +Index Attribute Meaning + +------------------------------------------------------------------ + +0 ‘pw_name’ Login name + + +1 ‘pw_passwd’ Optional encrypted password + + +2 ‘pw_uid’ Numerical user ID + + +3 ‘pw_gid’ Numerical group ID + + +4 ‘pw_gecos’ User name or comment field + + +5 ‘pw_dir’ User home directory + + +6 ‘pw_shell’ User command interpreter + + +The uid and gid items are integers, all others are strings. *note +KeyError: 2c7. is raised if the entry asked for cannot be found. + + Note: + In traditional Unix the field ‘pw_passwd’ usually contains a + password encrypted with a DES derived algorithm (see module *note + crypt: 29.). However most modern unices use a so-called `shadow + password' system. On those unices the `pw_passwd' field only + contains an asterisk (‘'*'’) or the letter ‘'x'’ where the + encrypted password is stored in a file ‘/etc/shadow’ which is not + world readable. Whether the `pw_passwd' field contains anything + useful is system-dependent. If available, the *note spwd: f1. + module should be used where access to the encrypted password is + required. + +It defines the following items: + + -- Function: pwd.getpwuid (uid) + + Return the password database entry for the given numeric user ID. + + -- Function: pwd.getpwnam (name) + + Return the password database entry for the given user name. + + -- Function: pwd.getpwall () + + Return a list of all available password database entries, in + arbitrary order. + +See also +........ + +Module *note grp: 8b. + + An interface to the group database, similar to this. + +Module *note spwd: f1. + + An interface to the shadow password database, similar to this. + + +File: python.info, Node: spwd — The shadow password database, Next: grp — The group database, Prev: pwd — The password database, Up: Unix Specific Services + +5.35.3 ‘spwd’ — The shadow password database +-------------------------------------------- + +__________________________________________________________________ + +This module provides access to the Unix shadow password database. It is +available on various Unix versions. + +You must have enough privileges to access the shadow password database +(this usually means you have to be root). + +Shadow password database entries are reported as a tuple-like object, +whose attributes correspond to the members of the ‘spwd’ structure +(Attribute field below, see ‘<shadow.h>’): + +Index Attribute Meaning + +---------------------------------------------------------------------- + +0 ‘sp_namp’ Login name + + +1 ‘sp_pwdp’ Encrypted password + + +2 ‘sp_lstchg’ Date of last change + + +3 ‘sp_min’ Minimal number of days between + changes + + +4 ‘sp_max’ Maximum number of days between + changes + + +5 ‘sp_warn’ Number of days before password + expires to warn user about it + + +6 ‘sp_inact’ Number of days after password + expires until account is disabled + + +7 ‘sp_expire’ Number of days since 1970-01-01 + when account expires + + +8 ‘sp_flag’ Reserved + + +The sp_namp and sp_pwdp items are strings, all others are integers. +*note KeyError: 2c7. is raised if the entry asked for cannot be found. + +The following functions are defined: + + -- Function: spwd.getspnam (name) + + Return the shadow password database entry for the given user name. + + Changed in version 3.6: Raises a *note PermissionError: 5ff. + instead of *note KeyError: 2c7. if the user doesn’t have + privileges. + + -- Function: spwd.getspall () + + Return a list of all available shadow password database entries, in + arbitrary order. + +See also +........ + +Module *note grp: 8b. + + An interface to the group database, similar to this. + +Module *note pwd: d6. + + An interface to the normal password database, similar to this. + + +File: python.info, Node: grp — The group database, Next: crypt — Function to check Unix passwords, Prev: spwd — The shadow password database, Up: Unix Specific Services + +5.35.4 ‘grp’ — The group database +--------------------------------- + +__________________________________________________________________ + +This module provides access to the Unix group database. It is available +on all Unix versions. + +Group database entries are reported as a tuple-like object, whose +attributes correspond to the members of the ‘group’ structure (Attribute +field below, see ‘<pwd.h>’): + +Index Attribute Meaning + +------------------------------------------------------------------ + +0 gr_name the name of the group + + +1 gr_passwd the (encrypted) group password; + often empty + + +2 gr_gid the numerical group ID + + +3 gr_mem all the group member’s user names + + +The gid is an integer, name and password are strings, and the member +list is a list of strings. (Note that most users are not explicitly +listed as members of the group they are in according to the password +database. Check both databases to get complete membership information. +Also note that a ‘gr_name’ that starts with a ‘+’ or ‘-’ is likely to be +a YP/NIS reference and may not be accessible via *note getgrnam(): 3755. +or *note getgrgid(): 5df.) + +It defines the following items: + + -- Function: grp.getgrgid (gid) + + Return the group database entry for the given numeric group ID. + *note KeyError: 2c7. is raised if the entry asked for cannot be + found. + + Deprecated since version 3.6: Since Python 3.6 the support of + non-integer arguments like floats or strings in *note getgrgid(): + 5df. is deprecated. + + -- Function: grp.getgrnam (name) + + Return the group database entry for the given group name. *note + KeyError: 2c7. is raised if the entry asked for cannot be found. + + -- Function: grp.getgrall () + + Return a list of all available group entries, in arbitrary order. + +See also +........ + +Module *note pwd: d6. + + An interface to the user database, similar to this. + +Module *note spwd: f1. + + An interface to the shadow password database, similar to this. + + +File: python.info, Node: crypt — Function to check Unix passwords, Next: termios — POSIX style tty control, Prev: grp — The group database, Up: Unix Specific Services + +5.35.5 ‘crypt’ — Function to check Unix passwords +------------------------------------------------- + +`Source code:' Lib/crypt.py(1) + +__________________________________________________________________ + +This module implements an interface to the ‘crypt(3)’ routine, which is +a one-way hash function based upon a modified DES algorithm; see the +Unix man page for further details. Possible uses include storing hashed +passwords so you can check passwords without storing the actual +password, or attempting to crack Unix passwords with a dictionary. + +Notice that the behavior of this module depends on the actual +implementation of the ‘crypt(3)’ routine in the running system. +Therefore, any extensions available on the current implementation will +also be available on this module. + +*note Availability: ffb.: Unix. Not available on VxWorks. + +* Menu: + +* Hashing Methods:: +* Module Attributes:: +* Module Functions: Module Functions<2>. +* Examples: Examples<31>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/crypt.py + + +File: python.info, Node: Hashing Methods, Next: Module Attributes, Up: crypt — Function to check Unix passwords + +5.35.5.1 Hashing Methods +........................ + +New in version 3.3. + +The *note crypt: 29. module defines the list of hashing methods (not all +methods are available on all platforms): + + -- Data: crypt.METHOD_SHA512 + + A Modular Crypt Format method with 16 character salt and 86 + character hash based on the SHA-512 hash function. This is the + strongest method. + + -- Data: crypt.METHOD_SHA256 + + Another Modular Crypt Format method with 16 character salt and 43 + character hash based on the SHA-256 hash function. + + -- Data: crypt.METHOD_BLOWFISH + + Another Modular Crypt Format method with 22 character salt and 31 + character hash based on the Blowfish cipher. + + New in version 3.7. + + -- Data: crypt.METHOD_MD5 + + Another Modular Crypt Format method with 8 character salt and 22 + character hash based on the MD5 hash function. + + -- Data: crypt.METHOD_CRYPT + + The traditional method with a 2 character salt and 13 characters of + hash. This is the weakest method. + + +File: python.info, Node: Module Attributes, Next: Module Functions<2>, Prev: Hashing Methods, Up: crypt — Function to check Unix passwords + +5.35.5.2 Module Attributes +.......................... + +New in version 3.3. + + -- Attribute: crypt.methods + + A list of available password hashing algorithms, as + ‘crypt.METHOD_*’ objects. This list is sorted from strongest to + weakest. + + +File: python.info, Node: Module Functions<2>, Next: Examples<31>, Prev: Module Attributes, Up: crypt — Function to check Unix passwords + +5.35.5.3 Module Functions +......................... + +The *note crypt: 29. module defines the following functions: + + -- Function: crypt.crypt (word, salt=None) + + `word' will usually be a user’s password as typed at a prompt or in + a graphical interface. The optional `salt' is either a string as + returned from *note mksalt(): 37c, one of the ‘crypt.METHOD_*’ + values (though not all may be available on all platforms), or a + full encrypted password including salt, as returned by this + function. If `salt' is not provided, the strongest method will be + used (as returned by *note methods(): 375f.). + + Checking a password is usually done by passing the plain-text + password as `word' and the full results of a previous *note + crypt(): 29. call, which should be the same as the results of this + call. + + `salt' (either a random 2 or 16 character string, possibly prefixed + with ‘$digit$’ to indicate the method) which will be used to + perturb the encryption algorithm. The characters in `salt' must be + in the set ‘[./a-zA-Z0-9]’, with the exception of Modular Crypt + Format which prefixes a ‘$digit$’. + + Returns the hashed password as a string, which will be composed of + characters from the same alphabet as the salt. + + Since a few ‘crypt(3)’ extensions allow different values, with + different sizes in the `salt', it is recommended to use the full + crypted password as salt when checking for a password. + + Changed in version 3.3: Accept ‘crypt.METHOD_*’ values in addition + to strings for `salt'. + + -- Function: crypt.mksalt (method=None, *, rounds=None) + + Return a randomly generated salt of the specified method. If no + `method' is given, the strongest method available as returned by + *note methods(): 375f. is used. + + The return value is a string suitable for passing as the `salt' + argument to *note crypt(): 29. + + `rounds' specifies the number of rounds for ‘METHOD_SHA256’, + ‘METHOD_SHA512’ and ‘METHOD_BLOWFISH’. For ‘METHOD_SHA256’ and + ‘METHOD_SHA512’ it must be an integer between ‘1000’ and + ‘999_999_999’, the default is ‘5000’. For ‘METHOD_BLOWFISH’ it + must be a power of two between ‘16’ (2^4) and ‘2_147_483_648’ + (2^31), the default is ‘4096’ (2^12). + + New in version 3.3. + + Changed in version 3.7: Added the `rounds' parameter. + + +File: python.info, Node: Examples<31>, Prev: Module Functions<2>, Up: crypt — Function to check Unix passwords + +5.35.5.4 Examples +................. + +A simple example illustrating typical use (a constant-time comparison +operation is needed to limit exposure to timing attacks. *note +hmac.compare_digest(): a0c. is suitable for this purpose): + + import pwd + import crypt + import getpass + from hmac import compare_digest as compare_hash + + def login(): + username = input('Python login: ') + cryptedpasswd = pwd.getpwnam(username)[1] + if cryptedpasswd: + if cryptedpasswd == 'x' or cryptedpasswd == '*': + raise ValueError('no support for shadow passwords') + cleartext = getpass.getpass() + return compare_hash(crypt.crypt(cleartext, cryptedpasswd), cryptedpasswd) + else: + return True + +To generate a hash of a password using the strongest available method +and check it against the original: + + import crypt + from hmac import compare_digest as compare_hash + + hashed = crypt.crypt(plaintext) + if not compare_hash(hashed, crypt.crypt(plaintext, hashed)): + raise ValueError("hashed version doesn't validate against original") + + +File: python.info, Node: termios — POSIX style tty control, Next: tty — Terminal control functions, Prev: crypt — Function to check Unix passwords, Up: Unix Specific Services + +5.35.6 ‘termios’ — POSIX style tty control +------------------------------------------ + +__________________________________________________________________ + +This module provides an interface to the POSIX calls for tty I/O +control. For a complete description of these calls, see ‘termios(3)’ +Unix manual page. It is only available for those Unix versions that +support POSIX `termios' style tty I/O control configured during +installation. + +All functions in this module take a file descriptor `fd' as their first +argument. This can be an integer file descriptor, such as returned by +‘sys.stdin.fileno()’, or a *note file object: b42, such as ‘sys.stdin’ +itself. + +This module also defines all the constants needed to work with the +functions provided here; these have the same name as their counterparts +in C. Please refer to your system documentation for more information on +using these terminal control interfaces. + +The module defines the following functions: + + -- Function: termios.tcgetattr (fd) + + Return a list containing the tty attributes for file descriptor + `fd', as follows: ‘[iflag, oflag, cflag, lflag, ispeed, ospeed, + cc]’ where `cc' is a list of the tty special characters (each a + string of length 1, except the items with indices ‘VMIN’ and + ‘VTIME’, which are integers when these fields are defined). The + interpretation of the flags and the speeds as well as the indexing + in the `cc' array must be done using the symbolic constants defined + in the *note termios: 104. module. + + -- Function: termios.tcsetattr (fd, when, attributes) + + Set the tty attributes for file descriptor `fd' from the + `attributes', which is a list like the one returned by *note + tcgetattr(): 3765. The `when' argument determines when the + attributes are changed: ‘TCSANOW’ to change immediately, + ‘TCSADRAIN’ to change after transmitting all queued output, or + ‘TCSAFLUSH’ to change after transmitting all queued output and + discarding all queued input. + + -- Function: termios.tcsendbreak (fd, duration) + + Send a break on file descriptor `fd'. A zero `duration' sends a + break for 0.25–0.5 seconds; a nonzero `duration' has a system + dependent meaning. + + -- Function: termios.tcdrain (fd) + + Wait until all output written to file descriptor `fd' has been + transmitted. + + -- Function: termios.tcflush (fd, queue) + + Discard queued data on file descriptor `fd'. The `queue' selector + specifies which queue: ‘TCIFLUSH’ for the input queue, ‘TCOFLUSH’ + for the output queue, or ‘TCIOFLUSH’ for both queues. + + -- Function: termios.tcflow (fd, action) + + Suspend or resume input or output on file descriptor `fd'. The + `action' argument can be ‘TCOOFF’ to suspend output, ‘TCOON’ to + restart output, ‘TCIOFF’ to suspend input, or ‘TCION’ to restart + input. + +See also +........ + +Module *note tty: 115. + + Convenience functions for common terminal control operations. + +* Menu: + +* Example: Example<16>. + + +File: python.info, Node: Example<16>, Up: termios — POSIX style tty control + +5.35.6.1 Example +................ + +Here’s a function that prompts for a password with echoing turned off. +Note the technique using a separate *note tcgetattr(): 3765. call and a +*note try: d72. … *note finally: 182. statement to ensure that the old +tty attributes are restored exactly no matter what happens: + + def getpass(prompt="Password: "): + import termios, sys + fd = sys.stdin.fileno() + old = termios.tcgetattr(fd) + new = termios.tcgetattr(fd) + new[3] = new[3] & ~termios.ECHO # lflags + try: + termios.tcsetattr(fd, termios.TCSADRAIN, new) + passwd = input(prompt) + finally: + termios.tcsetattr(fd, termios.TCSADRAIN, old) + return passwd + + +File: python.info, Node: tty — Terminal control functions, Next: pty — Pseudo-terminal utilities, Prev: termios — POSIX style tty control, Up: Unix Specific Services + +5.35.7 ‘tty’ — Terminal control functions +----------------------------------------- + +`Source code:' Lib/tty.py(1) + +__________________________________________________________________ + +The *note tty: 115. module defines functions for putting the tty into +cbreak and raw modes. + +Because it requires the *note termios: 104. module, it will work only on +Unix. + +The *note tty: 115. module defines the following functions: + + -- Function: tty.setraw (fd, when=termios.TCSAFLUSH) + + Change the mode of the file descriptor `fd' to raw. If `when' is + omitted, it defaults to ‘termios.TCSAFLUSH’, and is passed to *note + termios.tcsetattr(): 3766. + + -- Function: tty.setcbreak (fd, when=termios.TCSAFLUSH) + + Change the mode of file descriptor `fd' to cbreak. If `when' is + omitted, it defaults to ‘termios.TCSAFLUSH’, and is passed to *note + termios.tcsetattr(): 3766. + +See also +........ + +Module *note termios: 104. + + Low-level terminal control interface. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/tty.py + + +File: python.info, Node: pty — Pseudo-terminal utilities, Next: fcntl — The fcntl and ioctl system calls, Prev: tty — Terminal control functions, Up: Unix Specific Services + +5.35.8 ‘pty’ — Pseudo-terminal utilities +---------------------------------------- + +`Source code:' Lib/pty.py(1) + +__________________________________________________________________ + +The *note pty: d5. module defines operations for handling the +pseudo-terminal concept: starting another process and being able to +write to and read from its controlling terminal programmatically. + +Because pseudo-terminal handling is highly platform dependent, there is +code to do it only for Linux. (The Linux code is supposed to work on +other platforms, but hasn’t been tested yet.) + +The *note pty: d5. module defines the following functions: + + -- Function: pty.fork () + + Fork. Connect the child’s controlling terminal to a + pseudo-terminal. Return value is ‘(pid, fd)’. Note that the child + gets `pid' 0, and the `fd' is `invalid'. The parent’s return value + is the `pid' of the child, and `fd' is a file descriptor connected + to the child’s controlling terminal (and also to the child’s + standard input and output). + + -- Function: pty.openpty () + + Open a new pseudo-terminal pair, using *note os.openpty(): 1c02. if + possible, or emulation code for generic Unix systems. Return a + pair of file descriptors ‘(master, slave)’, for the master and the + slave end, respectively. + + -- Function: pty.spawn (argv[, master_read[, stdin_read]]) + + Spawn a process, and connect its controlling terminal with the + current process’s standard io. This is often used to baffle + programs which insist on reading from the controlling terminal. It + is expected that the process spawned behind the pty will eventually + terminate, and when it does `spawn' will return. + + The functions `master_read' and `stdin_read' are passed a file + descriptor which they should read from, and they should always + return a byte string. In order to force spawn to return before the + child process exits an *note OSError: 1d3. should be thrown. + + The default implementation for both functions will read and return + up to 1024 bytes each time the function is called. The + `master_read' callback is passed the pseudoterminal’s master file + descriptor to read output from the child process, and `stdin_read' + is passed file descriptor 0, to read from the parent process’s + standard input. + + Returning an empty byte string from either callback is interpreted + as an end-of-file (EOF) condition, and that callback will not be + called after that. If `stdin_read' signals EOF the controlling + terminal can no longer communicate with the parent process OR the + child process. Unless the child process will quit without any + input, `spawn' will then loop forever. If `master_read' signals + EOF the same behavior results (on linux at least). + + If both callbacks signal EOF then `spawn' will probably never + return, unless `select' throws an error on your platform when + passed three empty lists. This is a bug, documented in issue + 26228(2). + + Raises an *note auditing event: fd1. ‘pty.spawn’ with argument + ‘argv’. + + Changed in version 3.4: *note spawn(): 898. now returns the status + value from *note os.waitpid(): 66d. on the child process. + +* Menu: + +* Example: Example<17>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pty.py + + (2) https://bugs.python.org/issue26228 + + +File: python.info, Node: Example<17>, Up: pty — Pseudo-terminal utilities + +5.35.8.1 Example +................ + +The following program acts like the Unix command ‘script(1)’, using a +pseudo-terminal to record all input and output of a terminal session in +a “typescript”. + + import argparse + import os + import pty + import sys + import time + + parser = argparse.ArgumentParser() + parser.add_argument('-a', dest='append', action='store_true') + parser.add_argument('-p', dest='use_python', action='store_true') + parser.add_argument('filename', nargs='?', default='typescript') + options = parser.parse_args() + + shell = sys.executable if options.use_python else os.environ.get('SHELL', 'sh') + filename = options.filename + mode = 'ab' if options.append else 'wb' + + with open(filename, mode) as script: + def read(fd): + data = os.read(fd, 1024) + script.write(data) + return data + + print('Script started, file is', filename) + script.write(('Script started on %s\n' % time.asctime()).encode()) + + pty.spawn(shell, read) + + script.write(('Script done on %s\n' % time.asctime()).encode()) + print('Script done, file is', filename) + + +File: python.info, Node: fcntl — The fcntl and ioctl system calls, Next: pipes — Interface to shell pipelines, Prev: pty — Pseudo-terminal utilities, Up: Unix Specific Services + +5.35.9 ‘fcntl’ — The ‘fcntl’ and ‘ioctl’ system calls +----------------------------------------------------- + +__________________________________________________________________ + +This module performs file control and I/O control on file descriptors. +It is an interface to the ‘fcntl()’ and ‘ioctl()’ Unix routines. For a +complete description of these calls, see ‘fcntl(2)’ and ‘ioctl(2)’ Unix +manual pages. + +All functions in this module take a file descriptor `fd' as their first +argument. This can be an integer file descriptor, such as returned by +‘sys.stdin.fileno()’, or an *note io.IOBase: 1db. object, such as +‘sys.stdin’ itself, which provides a *note fileno(): 1bdb. that returns +a genuine file descriptor. + +Changed in version 3.3: Operations in this module used to raise an *note +IOError: 992. where they now raise an *note OSError: 1d3. + +Changed in version 3.8: The fcntl module now contains ‘F_ADD_SEALS’, +‘F_GET_SEALS’, and ‘F_SEAL_*’ constants for sealing of *note +os.memfd_create(): 1f0. file descriptors. + +The module defines the following functions: + + -- Function: fcntl.fcntl (fd, cmd, arg=0) + + Perform the operation `cmd' on file descriptor `fd' (file objects + providing a *note fileno(): 1bdb. method are accepted as well). + The values used for `cmd' are operating system dependent, and are + available as constants in the *note fcntl: 7e. module, using the + same names as used in the relevant C header files. The argument + `arg' can either be an integer value, or a *note bytes: 331. + object. With an integer value, the return value of this function + is the integer return value of the C ‘fcntl()’ call. When the + argument is bytes it represents a binary structure, e.g. created + by *note struct.pack(): c0b. The binary data is copied to a buffer + whose address is passed to the C ‘fcntl()’ call. The return value + after a successful call is the contents of the buffer, converted to + a *note bytes: 331. object. The length of the returned object will + be the same as the length of the `arg' argument. This is limited + to 1024 bytes. If the information returned in the buffer by the + operating system is larger than 1024 bytes, this is most likely to + result in a segmentation violation or a more subtle data + corruption. + + If the ‘fcntl()’ fails, an *note OSError: 1d3. is raised. + + Raises an *note auditing event: fd1. ‘fcntl.fcntl’ with arguments + ‘fd’, ‘cmd’, ‘arg’. + + -- Function: fcntl.ioctl (fd, request, arg=0, mutate_flag=True) + + This function is identical to the *note fcntl(): 2393. function, + except that the argument handling is even more complicated. + + The `request' parameter is limited to values that can fit in + 32-bits. Additional constants of interest for use as the `request' + argument can be found in the *note termios: 104. module, under the + same names as used in the relevant C header files. + + The parameter `arg' can be one of an integer, an object supporting + the read-only buffer interface (like *note bytes: 331.) or an + object supporting the read-write buffer interface (like *note + bytearray: 332.). + + In all but the last case, behaviour is as for the *note fcntl(): + 2393. function. + + If a mutable buffer is passed, then the behaviour is determined by + the value of the `mutate_flag' parameter. + + If it is false, the buffer’s mutability is ignored and behaviour is + as for a read-only buffer, except that the 1024 byte limit + mentioned above is avoided – so long as the buffer you pass is at + least as long as what the operating system wants to put there, + things should work. + + If `mutate_flag' is true (the default), then the buffer is (in + effect) passed to the underlying *note ioctl(): dde. system call, + the latter’s return code is passed back to the calling Python, and + the buffer’s new contents reflect the action of the *note ioctl(): + dde. This is a slight simplification, because if the supplied + buffer is less than 1024 bytes long it is first copied into a + static buffer 1024 bytes long which is then passed to *note + ioctl(): dde. and copied back into the supplied buffer. + + If the ‘ioctl()’ fails, an *note OSError: 1d3. exception is raised. + + An example: + + >>> import array, fcntl, struct, termios, os + >>> os.getpgrp() + 13341 + >>> struct.unpack('h', fcntl.ioctl(0, termios.TIOCGPGRP, " "))[0] + 13341 + >>> buf = array.array('h', [0]) + >>> fcntl.ioctl(0, termios.TIOCGPGRP, buf, 1) + 0 + >>> buf + array('h', [13341]) + + Raises an *note auditing event: fd1. ‘fcntl.ioctl’ with arguments + ‘fd’, ‘request’, ‘arg’. + + -- Function: fcntl.flock (fd, operation) + + Perform the lock operation `operation' on file descriptor `fd' + (file objects providing a *note fileno(): 1bdb. method are accepted + as well). See the Unix manual ‘flock(2)’ for details. (On some + systems, this function is emulated using ‘fcntl()’.) + + If the ‘flock()’ fails, an *note OSError: 1d3. exception is raised. + + Raises an *note auditing event: fd1. ‘fcntl.flock’ with arguments + ‘fd’, ‘operation’. + + -- Function: fcntl.lockf (fd, cmd, len=0, start=0, whence=0) + + This is essentially a wrapper around the *note fcntl(): 2393. + locking calls. `fd' is the file descriptor (file objects providing + a *note fileno(): 1bdb. method are accepted as well) of the file to + lock or unlock, and `cmd' is one of the following values: + + * ‘LOCK_UN’ – unlock + + * ‘LOCK_SH’ – acquire a shared lock + + * ‘LOCK_EX’ – acquire an exclusive lock + + When `cmd' is ‘LOCK_SH’ or ‘LOCK_EX’, it can also be bitwise ORed + with ‘LOCK_NB’ to avoid blocking on lock acquisition. If ‘LOCK_NB’ + is used and the lock cannot be acquired, an *note OSError: 1d3. + will be raised and the exception will have an `errno' attribute set + to ‘EACCES’ or ‘EAGAIN’ (depending on the operating system; for + portability, check for both values). On at least some systems, + ‘LOCK_EX’ can only be used if the file descriptor refers to a file + opened for writing. + + `len' is the number of bytes to lock, `start' is the byte offset at + which the lock starts, relative to `whence', and `whence' is as + with *note io.IOBase.seek(): 1a62, specifically: + + * ‘0’ – relative to the start of the file (*note os.SEEK_SET: + d94.) + + * ‘1’ – relative to the current buffer position (*note + os.SEEK_CUR: d95.) + + * ‘2’ – relative to the end of the file (*note os.SEEK_END: + d96.) + + The default for `start' is 0, which means to start at the beginning + of the file. The default for `len' is 0 which means to lock to the + end of the file. The default for `whence' is also 0. + + Raises an *note auditing event: fd1. ‘fcntl.lockf’ with arguments + ‘fd’, ‘cmd’, ‘len’, ‘start’, ‘whence’. + +Examples (all on a SVR4 compliant system): + + import struct, fcntl, os + + f = open(...) + rv = fcntl.fcntl(f, fcntl.F_SETFL, os.O_NDELAY) + + lockdata = struct.pack('hhllhh', fcntl.F_WRLCK, 0, 0, 0, 0, 0) + rv = fcntl.fcntl(f, fcntl.F_SETLKW, lockdata) + +Note that in the first example the return value variable `rv' will hold +an integer value; in the second example it will hold a *note bytes: 331. +object. The structure lay-out for the `lockdata' variable is system +dependent — therefore using the *note flock(): 31f3. call may be better. + +See also +........ + +Module *note os: c4. + + If the locking flags *note O_SHLOCK: d97. and *note O_EXLOCK: d98. + are present in the *note os: c4. module (on BSD only), the *note + os.open(): 665. function provides an alternative to the *note + lockf(): d52. and *note flock(): 31f3. functions. + + +File: python.info, Node: pipes — Interface to shell pipelines, Next: resource — Resource usage information, Prev: fcntl — The fcntl and ioctl system calls, Up: Unix Specific Services + +5.35.10 ‘pipes’ — Interface to shell pipelines +---------------------------------------------- + +`Source code:' Lib/pipes.py(1) + +__________________________________________________________________ + +The *note pipes: cc. module defines a class to abstract the concept of a +`pipeline' — a sequence of converters from one file to another. + +Because the module uses ‘/bin/sh’ command lines, a POSIX or compatible +shell for *note os.system(): dc1. and *note os.popen(): 2a9. is +required. + +The *note pipes: cc. module defines the following class: + + -- Class: pipes.Template + + An abstraction of a pipeline. + +Example: + + >>> import pipes + >>> t = pipes.Template() + >>> t.append('tr a-z A-Z', '--') + >>> f = t.open('pipefile', 'w') + >>> f.write('hello world') + >>> f.close() + >>> open('pipefile').read() + 'HELLO WORLD' + +* Menu: + +* Template Objects:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/pipes.py + + +File: python.info, Node: Template Objects, Up: pipes — Interface to shell pipelines + +5.35.10.1 Template Objects +.......................... + +Template objects following methods: + + -- Method: Template.reset () + + Restore a pipeline template to its initial state. + + -- Method: Template.clone () + + Return a new, equivalent, pipeline template. + + -- Method: Template.debug (flag) + + If `flag' is true, turn debugging on. Otherwise, turn debugging + off. When debugging is on, commands to be executed are printed, + and the shell is given ‘set -x’ command to be more verbose. + + -- Method: Template.append (cmd, kind) + + Append a new action at the end. The `cmd' variable must be a valid + bourne shell command. The `kind' variable consists of two letters. + + The first letter can be either of ‘'-'’ (which means the command + reads its standard input), ‘'f'’ (which means the commands reads a + given file on the command line) or ‘'.'’ (which means the commands + reads no input, and hence must be first.) + + Similarly, the second letter can be either of ‘'-'’ (which means + the command writes to standard output), ‘'f'’ (which means the + command writes a file on the command line) or ‘'.'’ (which means + the command does not write anything, and hence must be last.) + + -- Method: Template.prepend (cmd, kind) + + Add a new action at the beginning. See *note append(): 3780. for + explanations of the arguments. + + -- Method: Template.open (file, mode) + + Return a file-like object, open to `file', but read from or written + to by the pipeline. Note that only one of ‘'r'’, ‘'w'’ may be + given. + + -- Method: Template.copy (infile, outfile) + + Copy `infile' to `outfile' through the pipe. + + +File: python.info, Node: resource — Resource usage information, Next: nis — Interface to Sun’s NIS Yellow Pages, Prev: pipes — Interface to shell pipelines, Up: Unix Specific Services + +5.35.11 ‘resource’ — Resource usage information +----------------------------------------------- + +__________________________________________________________________ + +This module provides basic mechanisms for measuring and controlling +system resources utilized by a program. + +Symbolic constants are used to specify particular system resources and +to request usage information about either the current process or its +children. + +An *note OSError: 1d3. is raised on syscall failure. + + -- Exception: resource.error + + A deprecated alias of *note OSError: 1d3. + + Changed in version 3.3: Following PEP 3151(1), this class was made + an alias of *note OSError: 1d3. + +* Menu: + +* Resource Limits:: +* Resource Usage:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3151 + + +File: python.info, Node: Resource Limits, Next: Resource Usage, Up: resource — Resource usage information + +5.35.11.1 Resource Limits +......................... + +Resources usage can be limited using the *note setrlimit(): 31cd. +function described below. Each resource is controlled by a pair of +limits: a soft limit and a hard limit. The soft limit is the current +limit, and may be lowered or raised by a process over time. The soft +limit can never exceed the hard limit. The hard limit can be lowered to +any value greater than the soft limit, but not raised. (Only processes +with the effective UID of the super-user can raise a hard limit.) + +The specific resources that can be limited are system dependent. They +are described in the ‘getrlimit(2)’ man page. The resources listed +below are supported when the underlying operating system supports them; +resources which cannot be checked or controlled by the operating system +are not defined in this module for those platforms. + + -- Data: resource.RLIM_INFINITY + + Constant used to represent the limit for an unlimited resource. + + -- Function: resource.getrlimit (resource) + + Returns a tuple ‘(soft, hard)’ with the current soft and hard + limits of `resource'. Raises *note ValueError: 1fb. if an invalid + resource is specified, or *note error: 3786. if the underlying + system call fails unexpectedly. + + -- Function: resource.setrlimit (resource, limits) + + Sets new limits of consumption of `resource'. The `limits' + argument must be a tuple ‘(soft, hard)’ of two integers describing + the new limits. A value of *note RLIM_INFINITY: 3788. can be used + to request a limit that is unlimited. + + Raises *note ValueError: 1fb. if an invalid resource is specified, + if the new soft limit exceeds the hard limit, or if a process tries + to raise its hard limit. Specifying a limit of *note + RLIM_INFINITY: 3788. when the hard or system limit for that + resource is not unlimited will result in a *note ValueError: 1fb. + A process with the effective UID of super-user can request any + valid limit value, including unlimited, but *note ValueError: 1fb. + will still be raised if the requested limit exceeds the system + imposed limit. + + ‘setrlimit’ may also raise *note error: 3786. if the underlying + system call fails. + + VxWorks only supports setting *note RLIMIT_NOFILE: 378a. + + Raises an *note auditing event: fd1. ‘resource.setrlimit’ with + arguments ‘resource’, ‘limits’. + + -- Function: resource.prlimit (pid, resource[, limits]) + + Combines *note setrlimit(): 31cd. and *note getrlimit(): 3789. in + one function and supports to get and set the resources limits of an + arbitrary process. If `pid' is 0, then the call applies to the + current process. `resource' and `limits' have the same meaning as + in *note setrlimit(): 31cd, except that `limits' is optional. + + When `limits' is not given the function returns the `resource' + limit of the process `pid'. When `limits' is given the `resource' + limit of the process is set and the former resource limit is + returned. + + Raises *note ProcessLookupError: 99b. when `pid' can’t be found and + *note PermissionError: 5ff. when the user doesn’t have + ‘CAP_SYS_RESOURCE’ for the process. + + Raises an *note auditing event: fd1. ‘resource.prlimit’ with + arguments ‘pid’, ‘resource’, ‘limits’. + + *note Availability: ffb.: Linux 2.6.36 or later with glibc 2.13 or + later. + + New in version 3.4. + +These symbols define resources whose consumption can be controlled using +the *note setrlimit(): 31cd. and *note getrlimit(): 3789. functions +described below. The values of these symbols are exactly the constants +used by C programs. + +The Unix man page for ‘getrlimit(2)’ lists the available resources. +Note that not all systems use the same symbol or same value to denote +the same resource. This module does not attempt to mask platform +differences — symbols not defined for a platform will not be available +from this module on that platform. + + -- Data: resource.RLIMIT_CORE + + The maximum size (in bytes) of a core file that the current process + can create. This may result in the creation of a partial core file + if a larger core would be required to contain the entire process + image. + + -- Data: resource.RLIMIT_CPU + + The maximum amount of processor time (in seconds) that a process + can use. If this limit is exceeded, a ‘SIGXCPU’ signal is sent to + the process. (See the *note signal: ea. module documentation for + information about how to catch this signal and do something useful, + e.g. flush open files to disk.) + + -- Data: resource.RLIMIT_FSIZE + + The maximum size of a file which the process may create. + + -- Data: resource.RLIMIT_DATA + + The maximum size (in bytes) of the process’s heap. + + -- Data: resource.RLIMIT_STACK + + The maximum size (in bytes) of the call stack for the current + process. This only affects the stack of the main thread in a + multi-threaded process. + + -- Data: resource.RLIMIT_RSS + + The maximum resident set size that should be made available to the + process. + + -- Data: resource.RLIMIT_NPROC + + The maximum number of processes the current process may create. + + -- Data: resource.RLIMIT_NOFILE + + The maximum number of open file descriptors for the current + process. + + -- Data: resource.RLIMIT_OFILE + + The BSD name for *note RLIMIT_NOFILE: 378a. + + -- Data: resource.RLIMIT_MEMLOCK + + The maximum address space which may be locked in memory. + + -- Data: resource.RLIMIT_VMEM + + The largest area of mapped memory which the process may occupy. + + -- Data: resource.RLIMIT_AS + + The maximum area (in bytes) of address space which may be taken by + the process. + + -- Data: resource.RLIMIT_MSGQUEUE + + The number of bytes that can be allocated for POSIX message queues. + + *note Availability: ffb.: Linux 2.6.8 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_NICE + + The ceiling for the process’s nice level (calculated as 20 - + rlim_cur). + + *note Availability: ffb.: Linux 2.6.12 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_RTPRIO + + The ceiling of the real-time priority. + + *note Availability: ffb.: Linux 2.6.12 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_RTTIME + + The time limit (in microseconds) on CPU time that a process can + spend under real-time scheduling without making a blocking syscall. + + *note Availability: ffb.: Linux 2.6.25 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_SIGPENDING + + The number of signals which the process may queue. + + *note Availability: ffb.: Linux 2.6.8 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_SBSIZE + + The maximum size (in bytes) of socket buffer usage for this user. + This limits the amount of network memory, and hence the amount of + mbufs, that this user may hold at any time. + + *note Availability: ffb.: FreeBSD 9 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_SWAP + + The maximum size (in bytes) of the swap space that may be reserved + or used by all of this user id’s processes. This limit is enforced + only if bit 1 of the vm.overcommit sysctl is set. Please see + tuning(7)(1) for a complete description of this sysctl. + + *note Availability: ffb.: FreeBSD 9 or later. + + New in version 3.4. + + -- Data: resource.RLIMIT_NPTS + + The maximum number of pseudo-terminals created by this user id. + + *note Availability: ffb.: FreeBSD 9 or later. + + New in version 3.4. + + ---------- Footnotes ---------- + + (1) https://www.freebsd.org/cgi/man.cgi?query=tuning&sektion=7 + + +File: python.info, Node: Resource Usage, Prev: Resource Limits, Up: resource — Resource usage information + +5.35.11.2 Resource Usage +........................ + +These functions are used to retrieve resource usage information: + + -- Function: resource.getrusage (who) + + This function returns an object that describes the resources + consumed by either the current process or its children, as + specified by the `who' parameter. The `who' parameter should be + specified using one of the ‘RUSAGE_*’ constants described below. + + A simple example: + + from resource import * + import time + + # a non CPU-bound task + time.sleep(3) + print(getrusage(RUSAGE_SELF)) + + # a CPU-bound task + for i in range(10 ** 8): + _ = 1 + 1 + print(getrusage(RUSAGE_SELF)) + + The fields of the return value each describe how a particular + system resource has been used, e.g. amount of time spent running + is user mode or number of times the process was swapped out of main + memory. Some values are dependent on the clock tick internal, e.g. + the amount of memory the process is using. + + For backward compatibility, the return value is also accessible as + a tuple of 16 elements. + + The fields ‘ru_utime’ and ‘ru_stime’ of the return value are + floating point values representing the amount of time spent + executing in user mode and the amount of time spent executing in + system mode, respectively. The remaining values are integers. + Consult the ‘getrusage(2)’ man page for detailed information about + these values. A brief summary is presented here: + + Index Field Resource + + ----------------------------------------------------------------------------------- + + ‘0’ ‘ru_utime’ time in user mode (float seconds) + + + ‘1’ ‘ru_stime’ time in system mode (float seconds) + + + ‘2’ ‘ru_maxrss’ maximum resident set size + + + ‘3’ ‘ru_ixrss’ shared memory size + + + ‘4’ ‘ru_idrss’ unshared memory size + + + ‘5’ ‘ru_isrss’ unshared stack size + + + ‘6’ ‘ru_minflt’ page faults not requiring I/O + + + ‘7’ ‘ru_majflt’ page faults requiring I/O + + + ‘8’ ‘ru_nswap’ number of swap outs + + + ‘9’ ‘ru_inblock’ block input operations + + + ‘10’ ‘ru_oublock’ block output operations + + + ‘11’ ‘ru_msgsnd’ messages sent + + + ‘12’ ‘ru_msgrcv’ messages received + + + ‘13’ ‘ru_nsignals’ signals received + + + ‘14’ ‘ru_nvcsw’ voluntary context switches + + + ‘15’ ‘ru_nivcsw’ involuntary context switches + + + This function will raise a *note ValueError: 1fb. if an invalid + `who' parameter is specified. It may also raise *note error: 3786. + exception in unusual circumstances. + + -- Function: resource.getpagesize () + + Returns the number of bytes in a system page. (This need not be + the same as the hardware page size.) + +The following ‘RUSAGE_*’ symbols are passed to the *note getrusage(): +d99. function to specify which processes information should be provided +for. + + -- Data: resource.RUSAGE_SELF + + Pass to *note getrusage(): d99. to request resources consumed by + the calling process, which is the sum of resources used by all + threads in the process. + + -- Data: resource.RUSAGE_CHILDREN + + Pass to *note getrusage(): d99. to request resources consumed by + child processes of the calling process which have been terminated + and waited for. + + -- Data: resource.RUSAGE_BOTH + + Pass to *note getrusage(): d99. to request resources consumed by + both the current process and child processes. May not be available + on all systems. + + -- Data: resource.RUSAGE_THREAD + + Pass to *note getrusage(): d99. to request resources consumed by + the current thread. May not be available on all systems. + + New in version 3.2. + + +File: python.info, Node: nis — Interface to Sun’s NIS Yellow Pages, Next: syslog — Unix syslog library routines, Prev: resource — Resource usage information, Up: Unix Specific Services + +5.35.12 ‘nis’ — Interface to Sun’s NIS (Yellow Pages) +----------------------------------------------------- + +__________________________________________________________________ + +The *note nis: bf. module gives a thin wrapper around the NIS library, +useful for central administration of several hosts. + +Because NIS exists only on Unix systems, this module is only available +for Unix. + +The *note nis: bf. module defines the following functions: + + -- Function: nis.match (key, mapname, domain=default_domain) + + Return the match for `key' in map `mapname', or raise an error + (*note nis.error: 379d.) if there is none. Both should be strings, + `key' is 8-bit clean. Return value is an arbitrary array of bytes + (may contain ‘NULL’ and other joys). + + Note that `mapname' is first checked if it is an alias to another + name. + + The `domain' argument allows overriding the NIS domain used for the + lookup. If unspecified, lookup is in the default NIS domain. + + -- Function: nis.cat (mapname, domain=default_domain) + + Return a dictionary mapping `key' to `value' such that ‘match(key, + mapname)==value’. Note that both keys and values of the dictionary + are arbitrary arrays of bytes. + + Note that `mapname' is first checked if it is an alias to another + name. + + The `domain' argument allows overriding the NIS domain used for the + lookup. If unspecified, lookup is in the default NIS domain. + + -- Function: nis.maps (domain=default_domain) + + Return a list of all valid maps. + + The `domain' argument allows overriding the NIS domain used for the + lookup. If unspecified, lookup is in the default NIS domain. + + -- Function: nis.get_default_domain () + + Return the system default NIS domain. + +The *note nis: bf. module defines the following exception: + + -- Exception: nis.error + + An error raised when a NIS function returns an error code. + + +File: python.info, Node: syslog — Unix syslog library routines, Prev: nis — Interface to Sun’s NIS Yellow Pages, Up: Unix Specific Services + +5.35.13 ‘syslog’ — Unix syslog library routines +----------------------------------------------- + +__________________________________________________________________ + +This module provides an interface to the Unix ‘syslog’ library routines. +Refer to the Unix manual pages for a detailed description of the +‘syslog’ facility. + +This module wraps the system ‘syslog’ family of routines. A pure Python +library that can speak to a syslog server is available in the *note +logging.handlers: ac. module as ‘SysLogHandler’. + +The module defines the following functions: + + -- Function: syslog.syslog (message) + + -- Function: syslog.syslog (priority, message) + + Send the string `message' to the system logger. A trailing newline + is added if necessary. Each message is tagged with a priority + composed of a `facility' and a `level'. The optional `priority' + argument, which defaults to ‘LOG_INFO’, determines the message + priority. If the facility is not encoded in `priority' using + logical-or (‘LOG_INFO | LOG_USER’), the value given in the *note + openlog(): 31fa. call is used. + + If *note openlog(): 31fa. has not been called prior to the call to + *note syslog(): ff, ‘openlog()’ will be called with no arguments. + + Raises an *note auditing event: fd1. ‘syslog.syslog’ with arguments + ‘priority’, ‘message’. + + -- Function: syslog.openlog ([ident[, logoption[, facility]]]) + + Logging options of subsequent *note syslog(): ff. calls can be set + by calling *note openlog(): 31fa. *note syslog(): ff. will call + *note openlog(): 31fa. with no arguments if the log is not + currently open. + + The optional `ident' keyword argument is a string which is + prepended to every message, and defaults to ‘sys.argv[0]’ with + leading path components stripped. The optional `logoption' keyword + argument (default is 0) is a bit field – see below for possible + values to combine. The optional `facility' keyword argument + (default is ‘LOG_USER’) sets the default facility for messages + which do not have a facility explicitly encoded. + + Raises an *note auditing event: fd1. ‘syslog.openlog’ with + arguments ‘ident’, ‘logoption’, ‘facility’. + + Changed in version 3.2: In previous versions, keyword arguments + were not allowed, and `ident' was required. The default for + `ident' was dependent on the system libraries, and often was + ‘python’ instead of the name of the Python program file. + + -- Function: syslog.closelog () + + Reset the syslog module values and call the system library + ‘closelog()’. + + This causes the module to behave as it does when initially + imported. For example, *note openlog(): 31fa. will be called on + the first *note syslog(): ff. call (if *note openlog(): 31fa. + hasn’t already been called), and `ident' and other *note openlog(): + 31fa. parameters are reset to defaults. + + Raises an *note auditing event: fd1. ‘syslog.closelog’ with no + arguments. + + -- Function: syslog.setlogmask (maskpri) + + Set the priority mask to `maskpri' and return the previous mask + value. Calls to *note syslog(): ff. with a priority level not set + in `maskpri' are ignored. The default is to log all priorities. + The function ‘LOG_MASK(pri)’ calculates the mask for the individual + priority `pri'. The function ‘LOG_UPTO(pri)’ calculates the mask + for all priorities up to and including `pri'. + + Raises an *note auditing event: fd1. ‘syslog.setlogmask’ with + argument ‘maskpri’. + +The module defines the following constants: + +Priority levels (high to low): + + ‘LOG_EMERG’, ‘LOG_ALERT’, ‘LOG_CRIT’, ‘LOG_ERR’, ‘LOG_WARNING’, + ‘LOG_NOTICE’, ‘LOG_INFO’, ‘LOG_DEBUG’. + +Facilities: + + ‘LOG_KERN’, ‘LOG_USER’, ‘LOG_MAIL’, ‘LOG_DAEMON’, ‘LOG_AUTH’, + ‘LOG_LPR’, ‘LOG_NEWS’, ‘LOG_UUCP’, ‘LOG_CRON’, ‘LOG_SYSLOG’, + ‘LOG_LOCAL0’ to ‘LOG_LOCAL7’, and, if defined in ‘<syslog.h>’, + ‘LOG_AUTHPRIV’. + +Log options: + + ‘LOG_PID’, ‘LOG_CONS’, ‘LOG_NDELAY’, and, if defined in + ‘<syslog.h>’, ‘LOG_ODELAY’, ‘LOG_NOWAIT’, and ‘LOG_PERROR’. + +* Menu: + +* Examples: Examples<32>. + + +File: python.info, Node: Examples<32>, Up: syslog — Unix syslog library routines + +5.35.13.1 Examples +.................. + +* Menu: + +* Simple example:: + + +File: python.info, Node: Simple example, Up: Examples<32> + +5.35.13.2 Simple example +........................ + +A simple set of examples: + + import syslog + + syslog.syslog('Processing started') + if error: + syslog.syslog(syslog.LOG_ERR, 'Processing started') + +An example of setting some log options, these would include the process +ID in logged messages, and write the messages to the destination +facility used for mail logging: + + syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL) + syslog.syslog('E-mail processing initiated...') + + +File: python.info, Node: Superseded Modules, Next: Undocumented Modules, Prev: Unix Specific Services, Up: The Python Standard Library + +5.36 Superseded Modules +======================= + +The modules described in this chapter are deprecated and only kept for +backwards compatibility. They have been superseded by other modules. + +* Menu: + +* optparse — Parser for command line options:: +* imp — Access the import internals:: + + +File: python.info, Node: optparse — Parser for command line options, Next: imp — Access the import internals, Up: Superseded Modules + +5.36.1 ‘optparse’ — Parser for command line options +--------------------------------------------------- + +`Source code:' Lib/optparse.py(1) + +Deprecated since version 3.2: The *note optparse: c3. module is +deprecated and will not be developed further; development will continue +with the *note argparse: 6. module. + +__________________________________________________________________ + +*note optparse: c3. is a more convenient, flexible, and powerful library +for parsing command-line options than the old *note getopt: 87. module. +*note optparse: c3. uses a more declarative style of command-line +parsing: you create an instance of *note OptionParser: 37a9, populate it +with options, and parse the command line. *note optparse: c3. allows +users to specify options in the conventional GNU/POSIX syntax, and +additionally generates usage and help messages for you. + +Here’s an example of using *note optparse: c3. in a simple script: + + from optparse import OptionParser + ... + parser = OptionParser() + parser.add_option("-f", "--file", dest="filename", + help="write report to FILE", metavar="FILE") + parser.add_option("-q", "--quiet", + action="store_false", dest="verbose", default=True, + help="don't print status messages to stdout") + + (options, args) = parser.parse_args() + +With these few lines of code, users of your script can now do the “usual +thing” on the command-line, for example: + + <yourscript> --file=outfile -q + +As it parses the command line, *note optparse: c3. sets attributes of +the ‘options’ object returned by ‘parse_args()’ based on user-supplied +command-line values. When ‘parse_args()’ returns from parsing this +command line, ‘options.filename’ will be ‘"outfile"’ and +‘options.verbose’ will be ‘False’. *note optparse: c3. supports both +long and short options, allows short options to be merged together, and +allows options to be associated with their arguments in a variety of +ways. Thus, the following command lines are all equivalent to the above +example: + + <yourscript> -f outfile --quiet + <yourscript> --quiet --file outfile + <yourscript> -q -foutfile + <yourscript> -qfoutfile + +Additionally, users can run one of the following + + <yourscript> -h + <yourscript> --help + +and *note optparse: c3. will print out a brief summary of your script’s +options: + + Usage: <yourscript> [options] + + Options: + -h, --help show this help message and exit + -f FILE, --file=FILE write report to FILE + -q, --quiet don't print status messages to stdout + +where the value of `yourscript' is determined at runtime (normally from +‘sys.argv[0]’). + +* Menu: + +* Background:: +* Tutorial: Tutorial<2>. +* Reference Guide:: +* Option Callbacks:: +* Extending optparse:: + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/optparse.py + + +File: python.info, Node: Background, Next: Tutorial<2>, Up: optparse — Parser for command line options + +5.36.1.1 Background +................... + +*note optparse: c3. was explicitly designed to encourage the creation of +programs with straightforward, conventional command-line interfaces. To +that end, it supports only the most common command-line syntax and +semantics conventionally used under Unix. If you are unfamiliar with +these conventions, read this section to acquaint yourself with them. + +* Menu: + +* Terminology:: +* What are options for?:: +* What are positional arguments for?:: + + +File: python.info, Node: Terminology, Next: What are options for?, Up: Background + +5.36.1.2 Terminology +.................... + +argument + + a string entered on the command-line, and passed by the shell to + ‘execl()’ or ‘execv()’. In Python, arguments are elements of + ‘sys.argv[1:]’ (‘sys.argv[0]’ is the name of the program being + executed). Unix shells also use the term “word”. + + It is occasionally desirable to substitute an argument list other + than ‘sys.argv[1:]’, so you should read “argument” as “an element + of ‘sys.argv[1:]’, or of some other list provided as a substitute + for ‘sys.argv[1:]’”. + +option + + an argument used to supply extra information to guide or customize + the execution of a program. There are many different syntaxes for + options; the traditional Unix syntax is a hyphen (“-”) followed by + a single letter, e.g. ‘-x’ or ‘-F’. Also, traditional Unix syntax + allows multiple options to be merged into a single argument, e.g. + ‘-x -F’ is equivalent to ‘-xF’. The GNU project introduced ‘--’ + followed by a series of hyphen-separated words, e.g. ‘--file’ or + ‘--dry-run’. These are the only two option syntaxes provided by + *note optparse: c3. + + Some other option syntaxes that the world has seen include: + + * a hyphen followed by a few letters, e.g. ‘-pf’ (this is `not' + the same as multiple options merged into a single argument) + + * a hyphen followed by a whole word, e.g. ‘-file’ (this is + technically equivalent to the previous syntax, but they aren’t + usually seen in the same program) + + * a plus sign followed by a single letter, or a few letters, or + a word, e.g. ‘+f’, ‘+rgb’ + + * a slash followed by a letter, or a few letters, or a word, + e.g. ‘/f’, ‘/file’ + + These option syntaxes are not supported by *note optparse: c3, and + they never will be. This is deliberate: the first three are + non-standard on any environment, and the last only makes sense if + you’re exclusively targeting VMS, MS-DOS, and/or Windows. + +option argument + + an argument that follows an option, is closely associated with that + option, and is consumed from the argument list when that option is. + With *note optparse: c3, option arguments may either be in a + separate argument from their option: + + -f foo + --file foo + + or included in the same argument: + + -ffoo + --file=foo + + Typically, a given option either takes an argument or it doesn’t. + Lots of people want an “optional option arguments” feature, meaning + that some options will take an argument if they see it, and won’t + if they don’t. This is somewhat controversial, because it makes + parsing ambiguous: if ‘-a’ takes an optional argument and ‘-b’ is + another option entirely, how do we interpret ‘-ab’? Because of + this ambiguity, *note optparse: c3. does not support this feature. + +positional argument + + something leftover in the argument list after options have been + parsed, i.e. after options and their arguments have been parsed + and removed from the argument list. + +required option + + an option that must be supplied on the command-line; note that the + phrase “required option” is self-contradictory in English. *note + optparse: c3. doesn’t prevent you from implementing required + options, but doesn’t give you much help at it either. + +For example, consider this hypothetical command-line: + + prog -v --report report.txt foo bar + +‘-v’ and ‘--report’ are both options. Assuming that ‘--report’ takes +one argument, ‘report.txt’ is an option argument. ‘foo’ and ‘bar’ are +positional arguments. + + +File: python.info, Node: What are options for?, Next: What are positional arguments for?, Prev: Terminology, Up: Background + +5.36.1.3 What are options for? +.............................. + +Options are used to provide extra information to tune or customize the +execution of a program. In case it wasn’t clear, options are usually +`optional'. A program should be able to run just fine with no options +whatsoever. (Pick a random program from the Unix or GNU toolsets. Can +it run without any options at all and still make sense? The main +exceptions are ‘find’, ‘tar’, and ‘dd’—all of which are mutant oddballs +that have been rightly criticized for their non-standard syntax and +confusing interfaces.) + +Lots of people want their programs to have “required options”. Think +about it. If it’s required, then it’s `not optional'! If there is a +piece of information that your program absolutely requires in order to +run successfully, that’s what positional arguments are for. + +As an example of good command-line interface design, consider the humble +‘cp’ utility, for copying files. It doesn’t make much sense to try to +copy files without supplying a destination and at least one source. +Hence, ‘cp’ fails if you run it with no arguments. However, it has a +flexible, useful syntax that does not require any options at all: + + cp SOURCE DEST + cp SOURCE ... DEST-DIR + +You can get pretty far with just that. Most ‘cp’ implementations +provide a bunch of options to tweak exactly how the files are copied: +you can preserve mode and modification time, avoid following symlinks, +ask before clobbering existing files, etc. But none of this distracts +from the core mission of ‘cp’, which is to copy either one file to +another, or several files to another directory. + + +File: python.info, Node: What are positional arguments for?, Prev: What are options for?, Up: Background + +5.36.1.4 What are positional arguments for? +........................................... + +Positional arguments are for those pieces of information that your +program absolutely, positively requires to run. + +A good user interface should have as few absolute requirements as +possible. If your program requires 17 distinct pieces of information in +order to run successfully, it doesn’t much matter `how' you get that +information from the user—most people will give up and walk away before +they successfully run the program. This applies whether the user +interface is a command-line, a configuration file, or a GUI: if you make +that many demands on your users, most of them will simply give up. + +In short, try to minimize the amount of information that users are +absolutely required to supply—use sensible defaults whenever possible. +Of course, you also want to make your programs reasonably flexible. +That’s what options are for. Again, it doesn’t matter if they are +entries in a config file, widgets in the “Preferences” dialog of a GUI, +or command-line options—the more options you implement, the more +flexible your program is, and the more complicated its implementation +becomes. Too much flexibility has drawbacks as well, of course; too +many options can overwhelm users and make your code much harder to +maintain. + + +File: python.info, Node: Tutorial<2>, Next: Reference Guide, Prev: Background, Up: optparse — Parser for command line options + +5.36.1.5 Tutorial +................. + +While *note optparse: c3. is quite flexible and powerful, it’s also +straightforward to use in most cases. This section covers the code +patterns that are common to any *note optparse: c3.-based program. + +First, you need to import the OptionParser class; then, early in the +main program, create an OptionParser instance: + + from optparse import OptionParser + ... + parser = OptionParser() + +Then you can start defining options. The basic syntax is: + + parser.add_option(opt_str, ..., + attr=value, ...) + +Each option has one or more option strings, such as ‘-f’ or ‘--file’, +and several option attributes that tell *note optparse: c3. what to +expect and what to do when it encounters that option on the command +line. + +Typically, each option will have one short option string and one long +option string, e.g.: + + parser.add_option("-f", "--file", ...) + +You’re free to define as many short option strings and as many long +option strings as you like (including zero), as long as there is at +least one option string overall. + +The option strings passed to *note OptionParser.add_option(): 1d4f. are +effectively labels for the option defined by that call. For brevity, we +will frequently refer to `encountering an option' on the command line; +in reality, *note optparse: c3. encounters `option strings' and looks up +options from them. + +Once all of your options are defined, instruct *note optparse: c3. to +parse your program’s command line: + + (options, args) = parser.parse_args() + +(If you like, you can pass a custom argument list to ‘parse_args()’, but +that’s rarely necessary: by default it uses ‘sys.argv[1:]’.) + +‘parse_args()’ returns two values: + + * ‘options’, an object containing values for all of your options—e.g. + if ‘--file’ takes a single string argument, then ‘options.file’ + will be the filename supplied by the user, or ‘None’ if the user + did not supply that option + + * ‘args’, the list of positional arguments leftover after parsing + options + +This tutorial section only covers the four most important option +attributes: *note action: 37b4, *note type: 37b5, *note dest: 37b6. +(destination), and *note help: 37b7. Of these, *note action: 37b4. is +the most fundamental. + +* Menu: + +* Understanding option actions:: +* The store action:: +* Handling boolean (flag) options: Handling boolean flag options. +* Other actions:: +* Default values:: +* Generating help:: +* Printing a version string:: +* How optparse handles errors:: +* Putting it all together:: + + +File: python.info, Node: Understanding option actions, Next: The store action, Up: Tutorial<2> + +5.36.1.6 Understanding option actions +..................................... + +Actions tell *note optparse: c3. what to do when it encounters an option +on the command line. There is a fixed set of actions hard-coded into +*note optparse: c3.; adding new actions is an advanced topic covered in +section *note Extending optparse: 37ba. Most actions tell *note +optparse: c3. to store a value in some variable—for example, take a +string from the command line and store it in an attribute of ‘options’. + +If you don’t specify an option action, *note optparse: c3. defaults to +‘store’. + + +File: python.info, Node: The store action, Next: Handling boolean flag options, Prev: Understanding option actions, Up: Tutorial<2> + +5.36.1.7 The store action +......................... + +The most common option action is ‘store’, which tells *note optparse: +c3. to take the next argument (or the remainder of the current +argument), ensure that it is of the correct type, and store it to your +chosen destination. + +For example: + + parser.add_option("-f", "--file", + action="store", type="string", dest="filename") + +Now let’s make up a fake command line and ask *note optparse: c3. to +parse it: + + args = ["-f", "foo.txt"] + (options, args) = parser.parse_args(args) + +When *note optparse: c3. sees the option string ‘-f’, it consumes the +next argument, ‘foo.txt’, and stores it in ‘options.filename’. So, +after this call to ‘parse_args()’, ‘options.filename’ is ‘"foo.txt"’. + +Some other option types supported by *note optparse: c3. are ‘int’ and +‘float’. Here’s an option that expects an integer argument: + + parser.add_option("-n", type="int", dest="num") + +Note that this option has no long option string, which is perfectly +acceptable. Also, there’s no explicit action, since the default is +‘store’. + +Let’s parse another fake command-line. This time, we’ll jam the option +argument right up against the option: since ‘-n42’ (one argument) is +equivalent to ‘-n 42’ (two arguments), the code + + (options, args) = parser.parse_args(["-n42"]) + print(options.num) + +will print ‘42’. + +If you don’t specify a type, *note optparse: c3. assumes ‘string’. +Combined with the fact that the default action is ‘store’, that means +our first example can be a lot shorter: + + parser.add_option("-f", "--file", dest="filename") + +If you don’t supply a destination, *note optparse: c3. figures out a +sensible default from the option strings: if the first long option +string is ‘--foo-bar’, then the default destination is ‘foo_bar’. If +there are no long option strings, *note optparse: c3. looks at the first +short option string: the default destination for ‘-f’ is ‘f’. + +*note optparse: c3. also includes the built-in ‘complex’ type. Adding +types is covered in section *note Extending optparse: 37ba. + + +File: python.info, Node: Handling boolean flag options, Next: Other actions, Prev: The store action, Up: Tutorial<2> + +5.36.1.8 Handling boolean (flag) options +........................................ + +Flag options—set a variable to true or false when a particular option is +seen—are quite common. *note optparse: c3. supports them with two +separate actions, ‘store_true’ and ‘store_false’. For example, you +might have a ‘verbose’ flag that is turned on with ‘-v’ and off with +‘-q’: + + parser.add_option("-v", action="store_true", dest="verbose") + parser.add_option("-q", action="store_false", dest="verbose") + +Here we have two different options with the same destination, which is +perfectly OK. (It just means you have to be a bit careful when setting +default values—see below.) + +When *note optparse: c3. encounters ‘-v’ on the command line, it sets +‘options.verbose’ to ‘True’; when it encounters ‘-q’, ‘options.verbose’ +is set to ‘False’. + + +File: python.info, Node: Other actions, Next: Default values, Prev: Handling boolean flag options, Up: Tutorial<2> + +5.36.1.9 Other actions +...................... + +Some other actions supported by *note optparse: c3. are: + +‘"store_const"’ + + store a constant value + +‘"append"’ + + append this option’s argument to a list + +‘"count"’ + + increment a counter by one + +‘"callback"’ + + call a specified function + +These are covered in section *note Reference Guide: 37c1, and section +*note Option Callbacks: 37c2. + + +File: python.info, Node: Default values, Next: Generating help, Prev: Other actions, Up: Tutorial<2> + +5.36.1.10 Default values +........................ + +All of the above examples involve setting some variable (the +“destination”) when certain command-line options are seen. What happens +if those options are never seen? Since we didn’t supply any defaults, +they are all set to ‘None’. This is usually fine, but sometimes you +want more control. *note optparse: c3. lets you supply a default value +for each destination, which is assigned before the command line is +parsed. + +First, consider the verbose/quiet example. If we want *note optparse: +c3. to set ‘verbose’ to ‘True’ unless ‘-q’ is seen, then we can do this: + + parser.add_option("-v", action="store_true", dest="verbose", default=True) + parser.add_option("-q", action="store_false", dest="verbose") + +Since default values apply to the `destination' rather than to any +particular option, and these two options happen to have the same +destination, this is exactly equivalent: + + parser.add_option("-v", action="store_true", dest="verbose") + parser.add_option("-q", action="store_false", dest="verbose", default=True) + +Consider this: + + parser.add_option("-v", action="store_true", dest="verbose", default=False) + parser.add_option("-q", action="store_false", dest="verbose", default=True) + +Again, the default value for ‘verbose’ will be ‘True’: the last default +value supplied for any particular destination is the one that counts. + +A clearer way to specify default values is the ‘set_defaults()’ method +of OptionParser, which you can call at any time before calling +‘parse_args()’: + + parser.set_defaults(verbose=True) + parser.add_option(...) + (options, args) = parser.parse_args() + +As before, the last value specified for a given option destination is +the one that counts. For clarity, try to use one method or the other of +setting default values, not both. + + +File: python.info, Node: Generating help, Next: Printing a version string, Prev: Default values, Up: Tutorial<2> + +5.36.1.11 Generating help +......................... + +*note optparse: c3.’s ability to generate help and usage text +automatically is useful for creating user-friendly command-line +interfaces. All you have to do is supply a *note help: 37b7. value for +each option, and optionally a short usage message for your whole +program. Here’s an OptionParser populated with user-friendly +(documented) options: + + usage = "usage: %prog [options] arg1 arg2" + parser = OptionParser(usage=usage) + parser.add_option("-v", "--verbose", + action="store_true", dest="verbose", default=True, + help="make lots of noise [default]") + parser.add_option("-q", "--quiet", + action="store_false", dest="verbose", + help="be vewwy quiet (I'm hunting wabbits)") + parser.add_option("-f", "--filename", + metavar="FILE", help="write output to FILE") + parser.add_option("-m", "--mode", + default="intermediate", + help="interaction mode: novice, intermediate, " + "or expert [default: %default]") + +If *note optparse: c3. encounters either ‘-h’ or ‘--help’ on the +command-line, or if you just call ‘parser.print_help()’, it prints the +following to standard output: + + Usage: <yourscript> [options] arg1 arg2 + + Options: + -h, --help show this help message and exit + -v, --verbose make lots of noise [default] + -q, --quiet be vewwy quiet (I'm hunting wabbits) + -f FILE, --filename=FILE + write output to FILE + -m MODE, --mode=MODE interaction mode: novice, intermediate, or + expert [default: intermediate] + +(If the help output is triggered by a help option, *note optparse: c3. +exits after printing the help text.) + +There’s a lot going on here to help *note optparse: c3. generate the +best possible help message: + + * the script defines its own usage message: + + usage = "usage: %prog [options] arg1 arg2" + + *note optparse: c3. expands ‘%prog’ in the usage string to the name + of the current program, i.e. ‘os.path.basename(sys.argv[0])’. The + expanded string is then printed before the detailed option help. + + If you don’t supply a usage string, *note optparse: c3. uses a + bland but sensible default: ‘"Usage: %prog [options]"’, which is + fine if your script doesn’t take any positional arguments. + + * every option defines a help string, and doesn’t worry about + line-wrapping—*note optparse: c3. takes care of wrapping lines and + making the help output look good. + + * options that take a value indicate this fact in their + automatically-generated help message, e.g. for the “mode” option: + + -m MODE, --mode=MODE + + Here, “MODE” is called the meta-variable: it stands for the + argument that the user is expected to supply to ‘-m’/‘--mode’. By + default, *note optparse: c3. converts the destination variable name + to uppercase and uses that for the meta-variable. Sometimes, + that’s not what you want—for example, the ‘--filename’ option + explicitly sets ‘metavar="FILE"’, resulting in this + automatically-generated option description: + + -f FILE, --filename=FILE + + This is important for more than just saving space, though: the + manually written help text uses the meta-variable ‘FILE’ to clue + the user in that there’s a connection between the semi-formal + syntax ‘-f FILE’ and the informal semantic description “write + output to FILE”. This is a simple but effective way to make your + help text a lot clearer and more useful for end users. + + * options that have a default value can include ‘%default’ in the + help string—*note optparse: c3. will replace it with *note str(): + 330. of the option’s default value. If an option has no default + value (or the default value is ‘None’), ‘%default’ expands to + ‘none’. + +* Menu: + +* Grouping Options:: + + +File: python.info, Node: Grouping Options, Up: Generating help + +5.36.1.12 Grouping Options +.......................... + +When dealing with many options, it is convenient to group these options +for better help output. An *note OptionParser: 37a9. can contain +several option groups, each of which can contain several options. + +An option group is obtained using the class *note OptionGroup: 37c8.: + + -- Class: optparse.OptionGroup (parser, title, description=None) + + where + + * parser is the *note OptionParser: 37a9. instance the group + will be inserted in to + + * title is the group title + + * description, optional, is a long description of the group + +*note OptionGroup: 37c8. inherits from ‘OptionContainer’ (like *note +OptionParser: 37a9.) and so the ‘add_option()’ method can be used to add +an option to the group. + +Once all the options are declared, using the *note OptionParser: 37a9. +method ‘add_option_group()’ the group is added to the previously defined +parser. + +Continuing with the parser defined in the previous section, adding an +*note OptionGroup: 37c8. to a parser is easy: + + group = OptionGroup(parser, "Dangerous Options", + "Caution: use these options at your own risk. " + "It is believed that some of them bite.") + group.add_option("-g", action="store_true", help="Group option.") + parser.add_option_group(group) + +This would result in the following help output: + + Usage: <yourscript> [options] arg1 arg2 + + Options: + -h, --help show this help message and exit + -v, --verbose make lots of noise [default] + -q, --quiet be vewwy quiet (I'm hunting wabbits) + -f FILE, --filename=FILE + write output to FILE + -m MODE, --mode=MODE interaction mode: novice, intermediate, or + expert [default: intermediate] + + Dangerous Options: + Caution: use these options at your own risk. It is believed that some + of them bite. + + -g Group option. + +A bit more complete example might involve using more than one group: +still extending the previous example: + + group = OptionGroup(parser, "Dangerous Options", + "Caution: use these options at your own risk. " + "It is believed that some of them bite.") + group.add_option("-g", action="store_true", help="Group option.") + parser.add_option_group(group) + + group = OptionGroup(parser, "Debug Options") + group.add_option("-d", "--debug", action="store_true", + help="Print debug information") + group.add_option("-s", "--sql", action="store_true", + help="Print all SQL statements executed") + group.add_option("-e", action="store_true", help="Print every action done") + parser.add_option_group(group) + +that results in the following output: + + Usage: <yourscript> [options] arg1 arg2 + + Options: + -h, --help show this help message and exit + -v, --verbose make lots of noise [default] + -q, --quiet be vewwy quiet (I'm hunting wabbits) + -f FILE, --filename=FILE + write output to FILE + -m MODE, --mode=MODE interaction mode: novice, intermediate, or expert + [default: intermediate] + + Dangerous Options: + Caution: use these options at your own risk. It is believed that some + of them bite. + + -g Group option. + + Debug Options: + -d, --debug Print debug information + -s, --sql Print all SQL statements executed + -e Print every action done + +Another interesting method, in particular when working programmatically +with option groups is: + + -- Method: OptionParser.get_option_group (opt_str) + + Return the *note OptionGroup: 37c8. to which the short or long + option string `opt_str' (e.g. ‘'-o'’ or ‘'--option'’) belongs. If + there’s no such *note OptionGroup: 37c8, return ‘None’. + + +File: python.info, Node: Printing a version string, Next: How optparse handles errors, Prev: Generating help, Up: Tutorial<2> + +5.36.1.13 Printing a version string +................................... + +Similar to the brief usage string, *note optparse: c3. can also print a +version string for your program. You have to supply the string as the +‘version’ argument to OptionParser: + + parser = OptionParser(usage="%prog [-f] [-q]", version="%prog 1.0") + +‘%prog’ is expanded just like it is in ‘usage’. Apart from that, +‘version’ can contain anything you like. When you supply it, *note +optparse: c3. automatically adds a ‘--version’ option to your parser. +If it encounters this option on the command line, it expands your +‘version’ string (by replacing ‘%prog’), prints it to stdout, and exits. + +For example, if your script is called ‘/usr/bin/foo’: + + $ /usr/bin/foo --version + foo 1.0 + +The following two methods can be used to print and get the ‘version’ +string: + + -- Method: OptionParser.print_version (file=None) + + Print the version message for the current program (‘self.version’) + to `file' (default stdout). As with *note print_usage(): 37cd, any + occurrence of ‘%prog’ in ‘self.version’ is replaced with the name + of the current program. Does nothing if ‘self.version’ is empty or + undefined. + + -- Method: OptionParser.get_version () + + Same as *note print_version(): 37cc. but returns the version string + instead of printing it. + + +File: python.info, Node: How optparse handles errors, Next: Putting it all together, Prev: Printing a version string, Up: Tutorial<2> + +5.36.1.14 How ‘optparse’ handles errors +....................................... + +There are two broad classes of errors that *note optparse: c3. has to +worry about: programmer errors and user errors. Programmer errors are +usually erroneous calls to *note OptionParser.add_option(): 1d4f, e.g. +invalid option strings, unknown option attributes, missing option +attributes, etc. These are dealt with in the usual way: raise an +exception (either ‘optparse.OptionError’ or *note TypeError: 192.) and +let the program crash. + +Handling user errors is much more important, since they are guaranteed +to happen no matter how stable your code is. *note optparse: c3. can +automatically detect some user errors, such as bad option arguments +(passing ‘-n 4x’ where ‘-n’ takes an integer argument), missing +arguments (‘-n’ at the end of the command line, where ‘-n’ takes an +argument of any type). Also, you can call ‘OptionParser.error()’ to +signal an application-defined error condition: + + (options, args) = parser.parse_args() + ... + if options.a and options.b: + parser.error("options -a and -b are mutually exclusive") + +In either case, *note optparse: c3. handles the error the same way: it +prints the program’s usage message and an error message to standard +error and exits with error status 2. + +Consider the first example above, where the user passes ‘4x’ to an +option that takes an integer: + + $ /usr/bin/foo -n 4x + Usage: foo [options] + + foo: error: option -n: invalid integer value: '4x' + +Or, where the user fails to pass a value at all: + + $ /usr/bin/foo -n + Usage: foo [options] + + foo: error: -n option requires an argument + +*note optparse: c3.-generated error messages take care always to mention +the option involved in the error; be sure to do the same when calling +‘OptionParser.error()’ from your application code. + +If *note optparse: c3.’s default error-handling behaviour does not suit +your needs, you’ll need to subclass OptionParser and override its +‘exit()’ and/or ‘error()’ methods. + + +File: python.info, Node: Putting it all together, Prev: How optparse handles errors, Up: Tutorial<2> + +5.36.1.15 Putting it all together +................................. + +Here’s what *note optparse: c3.-based scripts usually look like: + + from optparse import OptionParser + ... + def main(): + usage = "usage: %prog [options] arg" + parser = OptionParser(usage) + parser.add_option("-f", "--file", dest="filename", + help="read data from FILENAME") + parser.add_option("-v", "--verbose", + action="store_true", dest="verbose") + parser.add_option("-q", "--quiet", + action="store_false", dest="verbose") + ... + (options, args) = parser.parse_args() + if len(args) != 1: + parser.error("incorrect number of arguments") + if options.verbose: + print("reading %s..." % options.filename) + ... + + if __name__ == "__main__": + main() + + +File: python.info, Node: Reference Guide, Next: Option Callbacks, Prev: Tutorial<2>, Up: optparse — Parser for command line options + +5.36.1.16 Reference Guide +......................... + +* Menu: + +* Creating the parser:: +* Populating the parser:: +* Defining options:: +* Option attributes:: +* Standard option actions:: +* Standard option types:: +* Parsing arguments: Parsing arguments<2>. +* Querying and manipulating your option parser:: +* Conflicts between options:: +* Cleanup: Cleanup<2>. +* Other methods:: + + +File: python.info, Node: Creating the parser, Next: Populating the parser, Up: Reference Guide + +5.36.1.17 Creating the parser +............................. + +The first step in using *note optparse: c3. is to create an OptionParser +instance. + + -- Class: optparse.OptionParser (...) + + The OptionParser constructor has no required arguments, but a + number of optional keyword arguments. You should always pass them + as keyword arguments, i.e. do not rely on the order in which the + arguments are declared. + + ‘usage’ (default: ‘"%prog [options]"’) + + The usage summary to print when your program is run + incorrectly or with a help option. When *note optparse: c3. + prints the usage string, it expands ‘%prog’ to + ‘os.path.basename(sys.argv[0])’ (or to ‘prog’ if you passed + that keyword argument). To suppress a usage message, pass the + special value ‘optparse.SUPPRESS_USAGE’. + + ‘option_list’ (default: ‘[]’) + + A list of Option objects to populate the parser with. The + options in ‘option_list’ are added after any options in + ‘standard_option_list’ (a class attribute that may be set by + OptionParser subclasses), but before any version or help + options. Deprecated; use *note add_option(): 1d4f. after + creating the parser instead. + + ‘option_class’ (default: optparse.Option) + + Class to use when adding options to the parser in *note + add_option(): 1d4f. + + ‘version’ (default: ‘None’) + + A version string to print when the user supplies a version + option. If you supply a true value for ‘version’, *note + optparse: c3. automatically adds a version option with the + single option string ‘--version’. The substring ‘%prog’ is + expanded the same as for ‘usage’. + + ‘conflict_handler’ (default: ‘"error"’) + + Specifies what to do when options with conflicting option + strings are added to the parser; see section *note Conflicts + between options: 37d6. + + ‘description’ (default: ‘None’) + + A paragraph of text giving a brief overview of your program. + *note optparse: c3. reformats this paragraph to fit the + current terminal width and prints it when the user requests + help (after ‘usage’, but before the list of options). + + ‘formatter’ (default: a new ‘IndentedHelpFormatter’) + + An instance of optparse.HelpFormatter that will be used for + printing help text. *note optparse: c3. provides two concrete + classes for this purpose: IndentedHelpFormatter and + TitledHelpFormatter. + + ‘add_help_option’ (default: ‘True’) + + If true, *note optparse: c3. will add a help option (with + option strings ‘-h’ and ‘--help’) to the parser. + + ‘prog’ + + The string to use when expanding ‘%prog’ in ‘usage’ and + ‘version’ instead of ‘os.path.basename(sys.argv[0])’. + + ‘epilog’ (default: ‘None’) + + A paragraph of help text to print after the option help. + + +File: python.info, Node: Populating the parser, Next: Defining options, Prev: Creating the parser, Up: Reference Guide + +5.36.1.18 Populating the parser +............................... + +There are several ways to populate the parser with options. The +preferred way is by using *note OptionParser.add_option(): 1d4f, as +shown in section *note Tutorial: 37b2. ‘add_option()’ can be called in +one of two ways: + + * pass it an Option instance (as returned by ‘make_option()’) + + * pass it any combination of positional and keyword arguments that + are acceptable to ‘make_option()’ (i.e., to the Option + constructor), and it will create the Option instance for you + +The other alternative is to pass a list of pre-constructed Option +instances to the OptionParser constructor, as in: + + option_list = [ + make_option("-f", "--filename", + action="store", type="string", dest="filename"), + make_option("-q", "--quiet", + action="store_false", dest="verbose"), + ] + parser = OptionParser(option_list=option_list) + +(‘make_option()’ is a factory function for creating Option instances; +currently it is an alias for the Option constructor. A future version +of *note optparse: c3. may split Option into several classes, and +‘make_option()’ will pick the right class to instantiate. Do not +instantiate Option directly.) + + +File: python.info, Node: Defining options, Next: Option attributes, Prev: Populating the parser, Up: Reference Guide + +5.36.1.19 Defining options +.......................... + +Each Option instance represents a set of synonymous command-line option +strings, e.g. ‘-f’ and ‘--file’. You can specify any number of short +or long option strings, but you must specify at least one overall option +string. + +The canonical way to create an ‘Option’ instance is with the +‘add_option()’ method of *note OptionParser: 37a9. + + -- Method: OptionParser.add_option (option) + + -- Method: OptionParser.add_option (*opt_str, attr=value, ...) + + To define an option with only a short option string: + + parser.add_option("-f", attr=value, ...) + + And to define an option with only a long option string: + + parser.add_option("--foo", attr=value, ...) + + The keyword arguments define attributes of the new Option object. + The most important option attribute is *note action: 37b4, and it + largely determines which other attributes are relevant or required. + If you pass irrelevant option attributes, or fail to pass required + ones, *note optparse: c3. raises an ‘OptionError’ exception + explaining your mistake. + + An option’s `action' determines what *note optparse: c3. does when + it encounters this option on the command-line. The standard option + actions hard-coded into *note optparse: c3. are: + + ‘"store"’ + + store this option’s argument (default) + + ‘"store_const"’ + + store a constant value + + ‘"store_true"’ + + store ‘True’ + + ‘"store_false"’ + + store ‘False’ + + ‘"append"’ + + append this option’s argument to a list + + ‘"append_const"’ + + append a constant value to a list + + ‘"count"’ + + increment a counter by one + + ‘"callback"’ + + call a specified function + + ‘"help"’ + + print a usage message including all options and the + documentation for them + + (If you don’t supply an action, the default is ‘"store"’. For this + action, you may also supply *note type: 37b5. and *note dest: 37b6. + option attributes; see *note Standard option actions: 37db.) + +As you can see, most actions involve storing or updating a value +somewhere. *note optparse: c3. always creates a special object for +this, conventionally called ‘options’ (it happens to be an instance of +‘optparse.Values’). Option arguments (and various other values) are +stored as attributes of this object, according to the *note dest: 37b6. +(destination) option attribute. + +For example, when you call + + parser.parse_args() + +one of the first things *note optparse: c3. does is create the ‘options’ +object: + + options = Values() + +If one of the options in this parser is defined with + + parser.add_option("-f", "--file", action="store", type="string", dest="filename") + +and the command-line being parsed includes any of the following: + + -ffoo + -f foo + --file=foo + --file foo + +then *note optparse: c3, on seeing this option, will do the equivalent +of + + options.filename = "foo" + +The *note type: 37b5. and *note dest: 37b6. option attributes are almost +as important as *note action: 37b4, but *note action: 37b4. is the only +one that makes sense for `all' options. + + +File: python.info, Node: Option attributes, Next: Standard option actions, Prev: Defining options, Up: Reference Guide + +5.36.1.20 Option attributes +........................... + +The following option attributes may be passed as keyword arguments to +*note OptionParser.add_option(): 1d4f. If you pass an option attribute +that is not relevant to a particular option, or fail to pass a required +option attribute, *note optparse: c3. raises ‘OptionError’. + + -- Attribute: Option.action + + (default: ‘"store"’) + + Determines *note optparse: c3.’s behaviour when this option is seen + on the command line; the available options are documented *note + here: 37db. + + -- Attribute: Option.type + + (default: ‘"string"’) + + The argument type expected by this option (e.g., ‘"string"’ or + ‘"int"’); the available option types are documented *note here: + 37de. + + -- Attribute: Option.dest + + (default: derived from option strings) + + If the option’s action implies writing or modifying a value + somewhere, this tells *note optparse: c3. where to write it: *note + dest: 37b6. names an attribute of the ‘options’ object that *note + optparse: c3. builds as it parses the command line. + + -- Attribute: Option.default + + The value to use for this option’s destination if the option is not + seen on the command line. See also *note + OptionParser.set_defaults(): 37e0. + + -- Attribute: Option.nargs + + (default: 1) + + How many arguments of type *note type: 37b5. should be consumed + when this option is seen. If > 1, *note optparse: c3. will store a + tuple of values to *note dest: 37b6. + + -- Attribute: Option.const + + For actions that store a constant value, the constant value to + store. + + -- Attribute: Option.choices + + For options of type ‘"choice"’, the list of strings the user may + choose from. + + -- Attribute: Option.callback + + For options with action ‘"callback"’, the callable to call when + this option is seen. See section *note Option Callbacks: 37c2. for + detail on the arguments passed to the callable. + + -- Attribute: Option.callback_args + -- Attribute: Option.callback_kwargs + + Additional positional and keyword arguments to pass to ‘callback’ + after the four standard callback arguments. + + -- Attribute: Option.help + + Help text to print for this option when listing all available + options after the user supplies a *note help: 37b7. option (such as + ‘--help’). If no help text is supplied, the option will be listed + without help text. To hide this option, use the special value + ‘optparse.SUPPRESS_HELP’. + + -- Attribute: Option.metavar + + (default: derived from option strings) + + Stand-in for the option argument(s) to use when printing help text. + See section *note Tutorial: 37b2. for an example. + + +File: python.info, Node: Standard option actions, Next: Standard option types, Prev: Option attributes, Up: Reference Guide + +5.36.1.21 Standard option actions +................................. + +The various option actions all have slightly different requirements and +effects. Most actions have several relevant option attributes which you +may specify to guide *note optparse: c3.’s behaviour; a few have +required attributes, which you must specify for any option using that +action. + + * ‘"store"’ [relevant: *note type: 37b5, *note dest: 37b6, *note + nargs: 37e1, *note choices: 37e3.] + + The option must be followed by an argument, which is converted to a + value according to *note type: 37b5. and stored in *note dest: + 37b6. If *note nargs: 37e1. > 1, multiple arguments will be + consumed from the command line; all will be converted according to + *note type: 37b5. and stored to *note dest: 37b6. as a tuple. See + the *note Standard option types: 37de. section. + + If *note choices: 37e3. is supplied (a list or tuple of strings), + the type defaults to ‘"choice"’. + + If *note type: 37b5. is not supplied, it defaults to ‘"string"’. + + If *note dest: 37b6. is not supplied, *note optparse: c3. derives a + destination from the first long option string (e.g., ‘--foo-bar’ + implies ‘foo_bar’). If there are no long option strings, *note + optparse: c3. derives a destination from the first short option + string (e.g., ‘-f’ implies ‘f’). + + Example: + + parser.add_option("-f") + parser.add_option("-p", type="float", nargs=3, dest="point") + + As it parses the command line + + -f foo.txt -p 1 -3.5 4 -fbar.txt + + *note optparse: c3. will set + + options.f = "foo.txt" + options.point = (1.0, -3.5, 4.0) + options.f = "bar.txt" + + * ‘"store_const"’ [required: *note const: 37e2.; relevant: *note + dest: 37b6.] + + The value *note const: 37e2. is stored in *note dest: 37b6. + + Example: + + parser.add_option("-q", "--quiet", + action="store_const", const=0, dest="verbose") + parser.add_option("-v", "--verbose", + action="store_const", const=1, dest="verbose") + parser.add_option("--noisy", + action="store_const", const=2, dest="verbose") + + If ‘--noisy’ is seen, *note optparse: c3. will set + + options.verbose = 2 + + * ‘"store_true"’ [relevant: *note dest: 37b6.] + + A special case of ‘"store_const"’ that stores ‘True’ to *note dest: + 37b6. + + * ‘"store_false"’ [relevant: *note dest: 37b6.] + + Like ‘"store_true"’, but stores ‘False’. + + Example: + + parser.add_option("--clobber", action="store_true", dest="clobber") + parser.add_option("--no-clobber", action="store_false", dest="clobber") + + * ‘"append"’ [relevant: *note type: 37b5, *note dest: 37b6, *note + nargs: 37e1, *note choices: 37e3.] + + The option must be followed by an argument, which is appended to + the list in *note dest: 37b6. If no default value for *note dest: + 37b6. is supplied, an empty list is automatically created when + *note optparse: c3. first encounters this option on the + command-line. If *note nargs: 37e1. > 1, multiple arguments are + consumed, and a tuple of length *note nargs: 37e1. is appended to + *note dest: 37b6. + + The defaults for *note type: 37b5. and *note dest: 37b6. are the + same as for the ‘"store"’ action. + + Example: + + parser.add_option("-t", "--tracks", action="append", type="int") + + If ‘-t3’ is seen on the command-line, *note optparse: c3. does the + equivalent of: + + options.tracks = [] + options.tracks.append(int("3")) + + If, a little later on, ‘--tracks=4’ is seen, it does: + + options.tracks.append(int("4")) + + The ‘append’ action calls the ‘append’ method on the current value + of the option. This means that any default value specified must + have an ‘append’ method. It also means that if the default value + is non-empty, the default elements will be present in the parsed + value for the option, with any values from the command line + appended after those default values: + + >>> parser.add_option("--files", action="append", default=['~/.mypkg/defaults']) + >>> opts, args = parser.parse_args(['--files', 'overrides.mypkg']) + >>> opts.files + ['~/.mypkg/defaults', 'overrides.mypkg'] + + * ‘"append_const"’ [required: *note const: 37e2.; relevant: *note + dest: 37b6.] + + Like ‘"store_const"’, but the value *note const: 37e2. is appended + to *note dest: 37b6.; as with ‘"append"’, *note dest: 37b6. + defaults to ‘None’, and an empty list is automatically created the + first time the option is encountered. + + * ‘"count"’ [relevant: *note dest: 37b6.] + + Increment the integer stored at *note dest: 37b6. If no default + value is supplied, *note dest: 37b6. is set to zero before being + incremented the first time. + + Example: + + parser.add_option("-v", action="count", dest="verbosity") + + The first time ‘-v’ is seen on the command line, *note optparse: + c3. does the equivalent of: + + options.verbosity = 0 + options.verbosity += 1 + + Every subsequent occurrence of ‘-v’ results in + + options.verbosity += 1 + + * ‘"callback"’ [required: *note callback: 37e4.; relevant: *note + type: 37b5, *note nargs: 37e1, *note callback_args: 37e5, *note + callback_kwargs: 37e6.] + + Call the function specified by *note callback: 37e4, which is + called as + + func(option, opt_str, value, parser, *args, **kwargs) + + See section *note Option Callbacks: 37c2. for more detail. + + * ‘"help"’ + + Prints a complete help message for all the options in the current + option parser. The help message is constructed from the ‘usage’ + string passed to OptionParser’s constructor and the *note help: + 37b7. string passed to every option. + + If no *note help: 37b7. string is supplied for an option, it will + still be listed in the help message. To omit an option entirely, + use the special value ‘optparse.SUPPRESS_HELP’. + + *note optparse: c3. automatically adds a *note help: 37b7. option + to all OptionParsers, so you do not normally need to create one. + + Example: + + from optparse import OptionParser, SUPPRESS_HELP + + # usually, a help option is added automatically, but that can + # be suppressed using the add_help_option argument + parser = OptionParser(add_help_option=False) + + parser.add_option("-h", "--help", action="help") + parser.add_option("-v", action="store_true", dest="verbose", + help="Be moderately verbose") + parser.add_option("--file", dest="filename", + help="Input file to read data from") + parser.add_option("--secret", help=SUPPRESS_HELP) + + If *note optparse: c3. sees either ‘-h’ or ‘--help’ on the command + line, it will print something like the following help message to + stdout (assuming ‘sys.argv[0]’ is ‘"foo.py"’): + + Usage: foo.py [options] + + Options: + -h, --help Show this help message and exit + -v Be moderately verbose + --file=FILENAME Input file to read data from + + After printing the help message, *note optparse: c3. terminates + your process with ‘sys.exit(0)’. + + * ‘"version"’ + + Prints the version number supplied to the OptionParser to stdout + and exits. The version number is actually formatted and printed by + the ‘print_version()’ method of OptionParser. Generally only + relevant if the ‘version’ argument is supplied to the OptionParser + constructor. As with *note help: 37b7. options, you will rarely + create ‘version’ options, since *note optparse: c3. automatically + adds them when needed. + + +File: python.info, Node: Standard option types, Next: Parsing arguments<2>, Prev: Standard option actions, Up: Reference Guide + +5.36.1.22 Standard option types +............................... + +*note optparse: c3. has five built-in option types: ‘"string"’, ‘"int"’, +‘"choice"’, ‘"float"’ and ‘"complex"’. If you need to add new option +types, see section *note Extending optparse: 37ba. + +Arguments to string options are not checked or converted in any way: the +text on the command line is stored in the destination (or passed to the +callback) as-is. + +Integer arguments (type ‘"int"’) are parsed as follows: + + * if the number starts with ‘0x’, it is parsed as a hexadecimal + number + + * if the number starts with ‘0’, it is parsed as an octal number + + * if the number starts with ‘0b’, it is parsed as a binary number + + * otherwise, the number is parsed as a decimal number + +The conversion is done by calling *note int(): 184. with the appropriate +base (2, 8, 10, or 16). If this fails, so will *note optparse: c3, +although with a more useful error message. + +‘"float"’ and ‘"complex"’ option arguments are converted directly with +*note float(): 187. and *note complex(): 189, with similar +error-handling. + +‘"choice"’ options are a subtype of ‘"string"’ options. The *note +choices: 37e3. option attribute (a sequence of strings) defines the set +of allowed option arguments. ‘optparse.check_choice()’ compares +user-supplied option arguments against this master list and raises +‘OptionValueError’ if an invalid string is given. + + +File: python.info, Node: Parsing arguments<2>, Next: Querying and manipulating your option parser, Prev: Standard option types, Up: Reference Guide + +5.36.1.23 Parsing arguments +........................... + +The whole point of creating and populating an OptionParser is to call +its ‘parse_args()’ method: + + (options, args) = parser.parse_args(args=None, values=None) + +where the input parameters are + +‘args’ + + the list of arguments to process (default: ‘sys.argv[1:]’) + +‘values’ + + an ‘optparse.Values’ object to store option arguments in (default: + a new instance of ‘Values’) – if you give an existing object, the + option defaults will not be initialized on it + +and the return values are + +‘options’ + + the same object that was passed in as ‘values’, or the + optparse.Values instance created by *note optparse: c3. + +‘args’ + + the leftover positional arguments after all options have been + processed + +The most common usage is to supply neither keyword argument. If you +supply ‘values’, it will be modified with repeated *note setattr(): +1285. calls (roughly one for every option argument stored to an option +destination) and returned by ‘parse_args()’. + +If ‘parse_args()’ encounters any errors in the argument list, it calls +the OptionParser’s ‘error()’ method with an appropriate end-user error +message. This ultimately terminates your process with an exit status of +2 (the traditional Unix exit status for command-line errors). + + +File: python.info, Node: Querying and manipulating your option parser, Next: Conflicts between options, Prev: Parsing arguments<2>, Up: Reference Guide + +5.36.1.24 Querying and manipulating your option parser +...................................................... + +The default behavior of the option parser can be customized slightly, +and you can also poke around your option parser and see what’s there. +OptionParser provides several methods to help you out: + + -- Method: OptionParser.disable_interspersed_args () + + Set parsing to stop on the first non-option. For example, if ‘-a’ + and ‘-b’ are both simple options that take no arguments, *note + optparse: c3. normally accepts this syntax: + + prog -a arg1 -b arg2 + + and treats it as equivalent to + + prog -a -b arg1 arg2 + + To disable this feature, call *note disable_interspersed_args(): + 1d50. This restores traditional Unix syntax, where option parsing + stops with the first non-option argument. + + Use this if you have a command processor which runs another command + which has options of its own and you want to make sure these + options don’t get confused. For example, each command might have a + different set of options. + + -- Method: OptionParser.enable_interspersed_args () + + Set parsing to not stop on the first non-option, allowing + interspersing switches with command arguments. This is the default + behavior. + + -- Method: OptionParser.get_option (opt_str) + + Returns the Option instance with the option string `opt_str', or + ‘None’ if no options have that option string. + + -- Method: OptionParser.has_option (opt_str) + + Return ‘True’ if the OptionParser has an option with option string + `opt_str' (e.g., ‘-q’ or ‘--verbose’). + + -- Method: OptionParser.remove_option (opt_str) + + If the *note OptionParser: 37a9. has an option corresponding to + `opt_str', that option is removed. If that option provided any + other option strings, all of those option strings become invalid. + If `opt_str' does not occur in any option belonging to this *note + OptionParser: 37a9, raises *note ValueError: 1fb. + + +File: python.info, Node: Conflicts between options, Next: Cleanup<2>, Prev: Querying and manipulating your option parser, Up: Reference Guide + +5.36.1.25 Conflicts between options +................................... + +If you’re not careful, it’s easy to define options with conflicting +option strings: + + parser.add_option("-n", "--dry-run", ...) + ... + parser.add_option("-n", "--noisy", ...) + +(This is particularly true if you’ve defined your own OptionParser +subclass with some standard options.) + +Every time you add an option, *note optparse: c3. checks for conflicts +with existing options. If it finds any, it invokes the current +conflict-handling mechanism. You can set the conflict-handling +mechanism either in the constructor: + + parser = OptionParser(..., conflict_handler=handler) + +or with a separate call: + + parser.set_conflict_handler(handler) + +The available conflict handlers are: + + ‘"error"’ (default) + + assume option conflicts are a programming error and raise + ‘OptionConflictError’ + + ‘"resolve"’ + + resolve option conflicts intelligently (see below) + +As an example, let’s define an *note OptionParser: 37a9. that resolves +conflicts intelligently and add conflicting options to it: + + parser = OptionParser(conflict_handler="resolve") + parser.add_option("-n", "--dry-run", ..., help="do no harm") + parser.add_option("-n", "--noisy", ..., help="be noisy") + +At this point, *note optparse: c3. detects that a previously-added +option is already using the ‘-n’ option string. Since +‘conflict_handler’ is ‘"resolve"’, it resolves the situation by removing +‘-n’ from the earlier option’s list of option strings. Now ‘--dry-run’ +is the only way for the user to activate that option. If the user asks +for help, the help message will reflect that: + + Options: + --dry-run do no harm + ... + -n, --noisy be noisy + +It’s possible to whittle away the option strings for a previously-added +option until there are none left, and the user has no way of invoking +that option from the command-line. In that case, *note optparse: c3. +removes that option completely, so it doesn’t show up in help text or +anywhere else. Carrying on with our existing OptionParser: + + parser.add_option("--dry-run", ..., help="new dry-run option") + +At this point, the original ‘-n’/‘--dry-run’ option is no longer +accessible, so *note optparse: c3. removes it, leaving this help text: + + Options: + ... + -n, --noisy be noisy + --dry-run new dry-run option + + +File: python.info, Node: Cleanup<2>, Next: Other methods, Prev: Conflicts between options, Up: Reference Guide + +5.36.1.26 Cleanup +................. + +OptionParser instances have several cyclic references. This should not +be a problem for Python’s garbage collector, but you may wish to break +the cyclic references explicitly by calling ‘destroy()’ on your +OptionParser once you are done with it. This is particularly useful in +long-running applications where large object graphs are reachable from +your OptionParser. + + +File: python.info, Node: Other methods, Prev: Cleanup<2>, Up: Reference Guide + +5.36.1.27 Other methods +....................... + +OptionParser supports several other public methods: + + -- Method: OptionParser.set_usage (usage) + + Set the usage string according to the rules described above for the + ‘usage’ constructor keyword argument. Passing ‘None’ sets the + default usage string; use ‘optparse.SUPPRESS_USAGE’ to suppress a + usage message. + + -- Method: OptionParser.print_usage (file=None) + + Print the usage message for the current program (‘self.usage’) to + `file' (default stdout). Any occurrence of the string ‘%prog’ in + ‘self.usage’ is replaced with the name of the current program. + Does nothing if ‘self.usage’ is empty or not defined. + + -- Method: OptionParser.get_usage () + + Same as *note print_usage(): 37cd. but returns the usage string + instead of printing it. + + -- Method: OptionParser.set_defaults (dest=value, ...) + + Set default values for several option destinations at once. Using + *note set_defaults(): 37e0. is the preferred way to set default + values for options, since multiple options can share the same + destination. For example, if several “mode” options all set the + same destination, any one of them can set the default, and the last + one wins: + + parser.add_option("--advanced", action="store_const", + dest="mode", const="advanced", + default="novice") # overridden below + parser.add_option("--novice", action="store_const", + dest="mode", const="novice", + default="advanced") # overrides above setting + + To avoid this confusion, use *note set_defaults(): 37e0.: + + parser.set_defaults(mode="advanced") + parser.add_option("--advanced", action="store_const", + dest="mode", const="advanced") + parser.add_option("--novice", action="store_const", + dest="mode", const="novice") + + +File: python.info, Node: Option Callbacks, Next: Extending optparse, Prev: Reference Guide, Up: optparse — Parser for command line options + +5.36.1.28 Option Callbacks +.......................... + +When *note optparse: c3.’s built-in actions and types aren’t quite +enough for your needs, you have two choices: extend *note optparse: c3. +or define a callback option. Extending *note optparse: c3. is more +general, but overkill for a lot of simple cases. Quite often a simple +callback is all you need. + +There are two steps to defining a callback option: + + * define the option itself using the ‘"callback"’ action + + * write the callback; this is a function (or method) that takes at + least four arguments, as described below + +* Menu: + +* Defining a callback option:: +* How callbacks are called:: +* Raising errors in a callback:: +* Callback example 1; trivial callback: Callback example 1 trivial callback. +* Callback example 2; check option order: Callback example 2 check option order. +* Callback example 3; check option order (generalized): Callback example 3 check option order generalized. +* Callback example 4; check arbitrary condition: Callback example 4 check arbitrary condition. +* Callback example 5; fixed arguments: Callback example 5 fixed arguments. +* Callback example 6; variable arguments: Callback example 6 variable arguments. + + +File: python.info, Node: Defining a callback option, Next: How callbacks are called, Up: Option Callbacks + +5.36.1.29 Defining a callback option +.................................... + +As always, the easiest way to define a callback option is by using the +*note OptionParser.add_option(): 1d4f. method. Apart from *note action: +37b4, the only option attribute you must specify is ‘callback’, the +function to call: + + parser.add_option("-c", action="callback", callback=my_callback) + +‘callback’ is a function (or other callable object), so you must have +already defined ‘my_callback()’ when you create this callback option. +In this simple case, *note optparse: c3. doesn’t even know if ‘-c’ takes +any arguments, which usually means that the option takes no +arguments—the mere presence of ‘-c’ on the command-line is all it needs +to know. In some circumstances, though, you might want your callback to +consume an arbitrary number of command-line arguments. This is where +writing callbacks gets tricky; it’s covered later in this section. + +*note optparse: c3. always passes four particular arguments to your +callback, and it will only pass additional arguments if you specify them +via *note callback_args: 37e5. and *note callback_kwargs: 37e6. Thus, +the minimal callback function signature is: + + def my_callback(option, opt, value, parser): + +The four arguments to a callback are described below. + +There are several other option attributes that you can supply when you +define a callback option: + +*note type: 37b5. + + has its usual meaning: as with the ‘"store"’ or ‘"append"’ actions, + it instructs *note optparse: c3. to consume one argument and + convert it to *note type: 37b5. Rather than storing the converted + value(s) anywhere, though, *note optparse: c3. passes it to your + callback function. + +*note nargs: 37e1. + + also has its usual meaning: if it is supplied and > 1, *note + optparse: c3. will consume *note nargs: 37e1. arguments, each of + which must be convertible to *note type: 37b5. It then passes a + tuple of converted values to your callback. + +*note callback_args: 37e5. + + a tuple of extra positional arguments to pass to the callback + +*note callback_kwargs: 37e6. + + a dictionary of extra keyword arguments to pass to the callback + + +File: python.info, Node: How callbacks are called, Next: Raising errors in a callback, Prev: Defining a callback option, Up: Option Callbacks + +5.36.1.30 How callbacks are called +.................................. + +All callbacks are called as follows: + + func(option, opt_str, value, parser, *args, **kwargs) + +where + +‘option’ + + is the Option instance that’s calling the callback + +‘opt_str’ + + is the option string seen on the command-line that’s triggering the + callback. (If an abbreviated long option was used, ‘opt_str’ will + be the full, canonical option string—e.g. if the user puts ‘--foo’ + on the command-line as an abbreviation for ‘--foobar’, then + ‘opt_str’ will be ‘"--foobar"’.) + +‘value’ + + is the argument to this option seen on the command-line. *note + optparse: c3. will only expect an argument if *note type: 37b5. is + set; the type of ‘value’ will be the type implied by the option’s + type. If *note type: 37b5. for this option is ‘None’ (no argument + expected), then ‘value’ will be ‘None’. If *note nargs: 37e1. > 1, + ‘value’ will be a tuple of values of the appropriate type. + +‘parser’ + + is the OptionParser instance driving the whole thing, mainly useful + because you can access some other interesting data through its + instance attributes: + + ‘parser.largs’ + + the current list of leftover arguments, ie. arguments that + have been consumed but are neither options nor option + arguments. Feel free to modify ‘parser.largs’, e.g. by + adding more arguments to it. (This list will become ‘args’, + the second return value of ‘parse_args()’.) + + ‘parser.rargs’ + + the current list of remaining arguments, ie. with ‘opt_str’ + and ‘value’ (if applicable) removed, and only the arguments + following them still there. Feel free to modify + ‘parser.rargs’, e.g. by consuming more arguments. + + ‘parser.values’ + + the object where option values are by default stored (an + instance of optparse.OptionValues). This lets callbacks use + the same mechanism as the rest of *note optparse: c3. for + storing option values; you don’t need to mess around with + globals or closures. You can also access or modify the + value(s) of any options already encountered on the + command-line. + +‘args’ + + is a tuple of arbitrary positional arguments supplied via the *note + callback_args: 37e5. option attribute. + +‘kwargs’ + + is a dictionary of arbitrary keyword arguments supplied via *note + callback_kwargs: 37e6. + + +File: python.info, Node: Raising errors in a callback, Next: Callback example 1 trivial callback, Prev: How callbacks are called, Up: Option Callbacks + +5.36.1.31 Raising errors in a callback +...................................... + +The callback function should raise ‘OptionValueError’ if there are any +problems with the option or its argument(s). *note optparse: c3. +catches this and terminates the program, printing the error message you +supply to stderr. Your message should be clear, concise, accurate, and +mention the option at fault. Otherwise, the user will have a hard time +figuring out what they did wrong. + + +File: python.info, Node: Callback example 1 trivial callback, Next: Callback example 2 check option order, Prev: Raising errors in a callback, Up: Option Callbacks + +5.36.1.32 Callback example 1: trivial callback +.............................................. + +Here’s an example of a callback option that takes no arguments, and +simply records that the option was seen: + + def record_foo_seen(option, opt_str, value, parser): + parser.values.saw_foo = True + + parser.add_option("--foo", action="callback", callback=record_foo_seen) + +Of course, you could do that with the ‘"store_true"’ action. + + +File: python.info, Node: Callback example 2 check option order, Next: Callback example 3 check option order generalized, Prev: Callback example 1 trivial callback, Up: Option Callbacks + +5.36.1.33 Callback example 2: check option order +................................................ + +Here’s a slightly more interesting example: record the fact that ‘-a’ is +seen, but blow up if it comes after ‘-b’ in the command-line. + + def check_order(option, opt_str, value, parser): + if parser.values.b: + raise OptionValueError("can't use -a after -b") + parser.values.a = 1 + ... + parser.add_option("-a", action="callback", callback=check_order) + parser.add_option("-b", action="store_true", dest="b") + + +File: python.info, Node: Callback example 3 check option order generalized, Next: Callback example 4 check arbitrary condition, Prev: Callback example 2 check option order, Up: Option Callbacks + +5.36.1.34 Callback example 3: check option order (generalized) +.............................................................. + +If you want to re-use this callback for several similar options (set a +flag, but blow up if ‘-b’ has already been seen), it needs a bit of +work: the error message and the flag that it sets must be generalized. + + def check_order(option, opt_str, value, parser): + if parser.values.b: + raise OptionValueError("can't use %s after -b" % opt_str) + setattr(parser.values, option.dest, 1) + ... + parser.add_option("-a", action="callback", callback=check_order, dest='a') + parser.add_option("-b", action="store_true", dest="b") + parser.add_option("-c", action="callback", callback=check_order, dest='c') + + +File: python.info, Node: Callback example 4 check arbitrary condition, Next: Callback example 5 fixed arguments, Prev: Callback example 3 check option order generalized, Up: Option Callbacks + +5.36.1.35 Callback example 4: check arbitrary condition +....................................................... + +Of course, you could put any condition in there—you’re not limited to +checking the values of already-defined options. For example, if you +have options that should not be called when the moon is full, all you +have to do is this: + + def check_moon(option, opt_str, value, parser): + if is_moon_full(): + raise OptionValueError("%s option invalid when moon is full" + % opt_str) + setattr(parser.values, option.dest, 1) + ... + parser.add_option("--foo", + action="callback", callback=check_moon, dest="foo") + +(The definition of ‘is_moon_full()’ is left as an exercise for the +reader.) + + +File: python.info, Node: Callback example 5 fixed arguments, Next: Callback example 6 variable arguments, Prev: Callback example 4 check arbitrary condition, Up: Option Callbacks + +5.36.1.36 Callback example 5: fixed arguments +............................................. + +Things get slightly more interesting when you define callback options +that take a fixed number of arguments. Specifying that a callback +option takes arguments is similar to defining a ‘"store"’ or ‘"append"’ +option: if you define *note type: 37b5, then the option takes one +argument that must be convertible to that type; if you further define +*note nargs: 37e1, then the option takes *note nargs: 37e1. arguments. + +Here’s an example that just emulates the standard ‘"store"’ action: + + def store_value(option, opt_str, value, parser): + setattr(parser.values, option.dest, value) + ... + parser.add_option("--foo", + action="callback", callback=store_value, + type="int", nargs=3, dest="foo") + +Note that *note optparse: c3. takes care of consuming 3 arguments and +converting them to integers for you; all you have to do is store them. +(Or whatever; obviously you don’t need a callback for this example.) + + +File: python.info, Node: Callback example 6 variable arguments, Prev: Callback example 5 fixed arguments, Up: Option Callbacks + +5.36.1.37 Callback example 6: variable arguments +................................................ + +Things get hairy when you want an option to take a variable number of +arguments. For this case, you must write a callback, as *note optparse: +c3. doesn’t provide any built-in capabilities for it. And you have to +deal with certain intricacies of conventional Unix command-line parsing +that *note optparse: c3. normally handles for you. In particular, +callbacks should implement the conventional rules for bare ‘--’ and ‘-’ +arguments: + + * either ‘--’ or ‘-’ can be option arguments + + * bare ‘--’ (if not the argument to some option): halt command-line + processing and discard the ‘--’ + + * bare ‘-’ (if not the argument to some option): halt command-line + processing but keep the ‘-’ (append it to ‘parser.largs’) + +If you want an option that takes a variable number of arguments, there +are several subtle, tricky issues to worry about. The exact +implementation you choose will be based on which trade-offs you’re +willing to make for your application (which is why *note optparse: c3. +doesn’t support this sort of thing directly). + +Nevertheless, here’s a stab at a callback for an option with variable +arguments: + + def vararg_callback(option, opt_str, value, parser): + assert value is None + value = [] + + def floatable(str): + try: + float(str) + return True + except ValueError: + return False + + for arg in parser.rargs: + # stop on --foo like options + if arg[:2] == "--" and len(arg) > 2: + break + # stop on -a, but not on -3 or -3.0 + if arg[:1] == "-" and len(arg) > 1 and not floatable(arg): + break + value.append(arg) + + del parser.rargs[:len(value)] + setattr(parser.values, option.dest, value) + + ... + parser.add_option("-c", "--callback", dest="vararg_attr", + action="callback", callback=vararg_callback) + + +File: python.info, Node: Extending optparse, Prev: Option Callbacks, Up: optparse — Parser for command line options + +5.36.1.38 Extending ‘optparse’ +.............................. + +Since the two major controlling factors in how *note optparse: c3. +interprets command-line options are the action and type of each option, +the most likely direction of extension is to add new actions and new +types. + +* Menu: + +* Adding new types:: +* Adding new actions:: + + +File: python.info, Node: Adding new types, Next: Adding new actions, Up: Extending optparse + +5.36.1.39 Adding new types +.......................... + +To add new types, you need to define your own subclass of *note +optparse: c3.’s ‘Option’ class. This class has a couple of attributes +that define *note optparse: c3.’s types: *note TYPES: 380f. and *note +TYPE_CHECKER: 3810. + + -- Attribute: Option.TYPES + + A tuple of type names; in your subclass, simply define a new tuple + *note TYPES: 380f. that builds on the standard one. + + -- Attribute: Option.TYPE_CHECKER + + A dictionary mapping type names to type-checking functions. A + type-checking function has the following signature: + + def check_mytype(option, opt, value) + + where ‘option’ is an ‘Option’ instance, ‘opt’ is an option string + (e.g., ‘-f’), and ‘value’ is the string from the command line that + must be checked and converted to your desired type. + ‘check_mytype()’ should return an object of the hypothetical type + ‘mytype’. The value returned by a type-checking function will wind + up in the OptionValues instance returned by + ‘OptionParser.parse_args()’, or be passed to a callback as the + ‘value’ parameter. + + Your type-checking function should raise ‘OptionValueError’ if it + encounters any problems. ‘OptionValueError’ takes a single string + argument, which is passed as-is to *note OptionParser: 37a9.’s + ‘error()’ method, which in turn prepends the program name and the + string ‘"error:"’ and prints everything to stderr before + terminating the process. + +Here’s a silly example that demonstrates adding a ‘"complex"’ option +type to parse Python-style complex numbers on the command line. (This +is even sillier than it used to be, because *note optparse: c3. 1.3 +added built-in support for complex numbers, but never mind.) + +First, the necessary imports: + + from copy import copy + from optparse import Option, OptionValueError + +You need to define your type-checker first, since it’s referred to later +(in the *note TYPE_CHECKER: 3810. class attribute of your Option +subclass): + + def check_complex(option, opt, value): + try: + return complex(value) + except ValueError: + raise OptionValueError( + "option %s: invalid complex value: %r" % (opt, value)) + +Finally, the Option subclass: + + class MyOption (Option): + TYPES = Option.TYPES + ("complex",) + TYPE_CHECKER = copy(Option.TYPE_CHECKER) + TYPE_CHECKER["complex"] = check_complex + +(If we didn’t make a *note copy(): 26. of *note Option.TYPE_CHECKER: +3810, we would end up modifying the *note TYPE_CHECKER: 3810. attribute +of *note optparse: c3.’s Option class. This being Python, nothing stops +you from doing that except good manners and common sense.) + +That’s it! Now you can write a script that uses the new option type +just like any other *note optparse: c3.-based script, except you have to +instruct your OptionParser to use MyOption instead of Option: + + parser = OptionParser(option_class=MyOption) + parser.add_option("-c", type="complex") + +Alternately, you can build your own option list and pass it to +OptionParser; if you don’t use ‘add_option()’ in the above way, you +don’t need to tell OptionParser which option class to use: + + option_list = [MyOption("-c", action="store", type="complex", dest="c")] + parser = OptionParser(option_list=option_list) + + +File: python.info, Node: Adding new actions, Prev: Adding new types, Up: Extending optparse + +5.36.1.40 Adding new actions +............................ + +Adding new actions is a bit trickier, because you have to understand +that *note optparse: c3. has a couple of classifications for actions: + +“store” actions + + actions that result in *note optparse: c3. storing a value to an + attribute of the current OptionValues instance; these options + require a *note dest: 37b6. attribute to be supplied to the Option + constructor. + +“typed” actions + + actions that take a value from the command line and expect it to be + of a certain type; or rather, a string that can be converted to a + certain type. These options require a *note type: 37b5. attribute + to the Option constructor. + +These are overlapping sets: some default “store” actions are ‘"store"’, +‘"store_const"’, ‘"append"’, and ‘"count"’, while the default “typed” +actions are ‘"store"’, ‘"append"’, and ‘"callback"’. + +When you add an action, you need to categorize it by listing it in at +least one of the following class attributes of Option (all are lists of +strings): + + -- Attribute: Option.ACTIONS + + All actions must be listed in ACTIONS. + + -- Attribute: Option.STORE_ACTIONS + + “store” actions are additionally listed here. + + -- Attribute: Option.TYPED_ACTIONS + + “typed” actions are additionally listed here. + + -- Attribute: Option.ALWAYS_TYPED_ACTIONS + + Actions that always take a type (i.e. whose options always take a + value) are additionally listed here. The only effect of this is + that *note optparse: c3. assigns the default type, ‘"string"’, to + options with no explicit type whose action is listed in *note + ALWAYS_TYPED_ACTIONS: 3816. + +In order to actually implement your new action, you must override +Option’s ‘take_action()’ method and add a case that recognizes your +action. + +For example, let’s add an ‘"extend"’ action. This is similar to the +standard ‘"append"’ action, but instead of taking a single value from +the command-line and appending it to an existing list, ‘"extend"’ will +take multiple values in a single comma-delimited string, and extend an +existing list with them. That is, if ‘--names’ is an ‘"extend"’ option +of type ‘"string"’, the command line + + --names=foo,bar --names blah --names ding,dong + +would result in a list + + ["foo", "bar", "blah", "ding", "dong"] + +Again we define a subclass of Option: + + class MyOption(Option): + + ACTIONS = Option.ACTIONS + ("extend",) + STORE_ACTIONS = Option.STORE_ACTIONS + ("extend",) + TYPED_ACTIONS = Option.TYPED_ACTIONS + ("extend",) + ALWAYS_TYPED_ACTIONS = Option.ALWAYS_TYPED_ACTIONS + ("extend",) + + def take_action(self, action, dest, opt, value, values, parser): + if action == "extend": + lvalue = value.split(",") + values.ensure_value(dest, []).extend(lvalue) + else: + Option.take_action( + self, action, dest, opt, value, values, parser) + +Features of note: + + * ‘"extend"’ both expects a value on the command-line and stores that + value somewhere, so it goes in both *note STORE_ACTIONS: 3814. and + *note TYPED_ACTIONS: 3815. + + * to ensure that *note optparse: c3. assigns the default type of + ‘"string"’ to ‘"extend"’ actions, we put the ‘"extend"’ action in + *note ALWAYS_TYPED_ACTIONS: 3816. as well. + + * ‘MyOption.take_action()’ implements just this one new action, and + passes control back to ‘Option.take_action()’ for the standard + *note optparse: c3. actions. + + * ‘values’ is an instance of the optparse_parser.Values class, which + provides the very useful ‘ensure_value()’ method. ‘ensure_value()’ + is essentially *note getattr(): 448. with a safety valve; it is + called as + + values.ensure_value(attr, value) + + If the ‘attr’ attribute of ‘values’ doesn’t exist or is ‘None’, + then ensure_value() first sets it to ‘value’, and then returns + ‘value. This is very handy for actions like ‘"extend"’, + ‘"append"’, and ‘"count"’, all of which accumulate data in a + variable and expect that variable to be of a certain type (a list + for the first two, an integer for the latter). Using + ‘ensure_value()’ means that scripts using your action don’t have to + worry about setting a default value for the option destinations in + question; they can just leave the default as ‘None’ and + ‘ensure_value()’ will take care of getting it right when it’s + needed. + + +File: python.info, Node: imp — Access the import internals, Prev: optparse — Parser for command line options, Up: Superseded Modules + +5.36.2 ‘imp’ — Access the import internals +------------------------------------------ + +`Source code:' Lib/imp.py(1) + +Deprecated since version 3.4: The *note imp: 9a. module is deprecated in +favor of *note importlib: 9b. + +__________________________________________________________________ + +This module provides an interface to the mechanisms used to implement +the *note import: c1d. statement. It defines the following constants +and functions: + + -- Function: imp.get_magic () + + Return the magic string value used to recognize byte-compiled code + files (‘.pyc’ files). (This value may be different for each Python + version.) + + Deprecated since version 3.4: Use *note + importlib.util.MAGIC_NUMBER: 862. instead. + + -- Function: imp.get_suffixes () + + Return a list of 3-element tuples, each describing a particular + type of module. Each triple has the form ‘(suffix, mode, type)’, + where `suffix' is a string to be appended to the module name to + form the filename to search for, `mode' is the mode string to pass + to the built-in *note open(): 4f0. function to open the file (this + can be ‘'r'’ for text files or ‘'rb'’ for binary files), and `type' + is the file type, which has one of the values *note PY_SOURCE: + 381a, *note PY_COMPILED: 381b, or *note C_EXTENSION: 381c, + described below. + + Deprecated since version 3.3: Use the constants defined on *note + importlib.machinery: 9d. instead. + + -- Function: imp.find_module (name[, path]) + + Try to find the module `name'. If `path' is omitted or ‘None’, the + list of directory names given by ‘sys.path’ is searched, but first + a few special places are searched: the function tries to find a + built-in module with the given name (*note C_BUILTIN: 381e.), then + a frozen module (*note PY_FROZEN: 381f.), and on some systems some + other places are looked in as well (on Windows, it looks in the + registry which may point to a specific file). + + Otherwise, `path' must be a list of directory names; each directory + is searched for files with any of the suffixes returned by *note + get_suffixes(): 3819. above. Invalid names in the list are + silently ignored (but all list items must be strings). + + If search is successful, the return value is a 3-element tuple + ‘(file, pathname, description)’: + + `file' is an open *note file object: b42. positioned at the + beginning, `pathname' is the pathname of the file found, and + `description' is a 3-element tuple as contained in the list + returned by *note get_suffixes(): 3819. describing the kind of + module found. + + If the module is built-in or frozen then `file' and `pathname' are + both ‘None’ and the `description' tuple contains empty strings for + its suffix and mode; the module type is indicated as given in + parentheses above. If the search is unsuccessful, *note + ImportError: 334. is raised. Other exceptions indicate problems + with the arguments or environment. + + If the module is a package, `file' is ‘None’, `pathname' is the + package path and the last item in the `description' tuple is *note + PKG_DIRECTORY: 3820. + + This function does not handle hierarchical module names (names + containing dots). In order to find `P.M', that is, submodule `M' + of package `P', use *note find_module(): 381d. and *note + load_module(): 3821. to find and load package `P', and then use + *note find_module(): 381d. with the `path' argument set to + ‘P.__path__’. When `P' itself has a dotted name, apply this recipe + recursively. + + Deprecated since version 3.3: Use *note importlib.util.find_spec(): + 938. instead unless Python 3.3 compatibility is required, in which + case use *note importlib.find_loader(): 937. For example usage of + the former case, see the *note Examples: 34cf. section of the *note + importlib: 9b. documentation. + + -- Function: imp.load_module (name, file, pathname, description) + + Load a module that was previously found by *note find_module(): + 381d. (or by an otherwise conducted search yielding compatible + results). This function does more than importing the module: if + the module was already imported, it will reload the module! The + `name' argument indicates the full module name (including the + package name, if this is a submodule of a package). The `file' + argument is an open file, and `pathname' is the corresponding file + name; these can be ‘None’ and ‘''’, respectively, when the module + is a package or not being loaded from a file. The `description' + argument is a tuple, as would be returned by *note get_suffixes(): + 3819, describing what kind of module must be loaded. + + If the load is successful, the return value is the module object; + otherwise, an exception (usually *note ImportError: 334.) is + raised. + + `Important:' the caller is responsible for closing the `file' + argument, if it was not ‘None’, even when an exception is raised. + This is best done using a *note try: d72. … *note finally: 182. + statement. + + Deprecated since version 3.3: If previously used in conjunction + with *note imp.find_module(): 381d. then consider using *note + importlib.import_module(): b13, otherwise use the loader returned + by the replacement you chose for *note imp.find_module(): 381d. If + you called *note imp.load_module(): 3821. and related functions + directly with file path arguments then use a combination of *note + importlib.util.spec_from_file_location(): 559. and *note + importlib.util.module_from_spec(): 6f0. See the *note Examples: + 34cf. section of the *note importlib: 9b. documentation for details + of the various approaches. + + -- Function: imp.new_module (name) + + Return a new empty module object called `name'. This object is + `not' inserted in ‘sys.modules’. + + Deprecated since version 3.4: Use *note + importlib.util.module_from_spec(): 6f0. instead. + + -- Function: imp.reload (module) + + Reload a previously imported `module'. The argument must be a + module object, so it must have been successfully imported before. + This is useful if you have edited the module source file using an + external editor and want to try out the new version without leaving + the Python interpreter. The return value is the module object (the + same as the `module' argument). + + When ‘reload(module)’ is executed: + + * Python modules’ code is recompiled and the module-level code + reexecuted, defining a new set of objects which are bound to + names in the module’s dictionary. The ‘init’ function of + extension modules is not called a second time. + + * As with all other objects in Python the old objects are only + reclaimed after their reference counts drop to zero. + + * The names in the module namespace are updated to point to any + new or changed objects. + + * Other references to the old objects (such as names external to + the module) are not rebound to refer to the new objects and + must be updated in each namespace where they occur if that is + desired. + + There are a number of other caveats: + + When a module is reloaded, its dictionary (containing the module’s + global variables) is retained. Redefinitions of names will + override the old definitions, so this is generally not a problem. + If the new version of a module does not define a name that was + defined by the old version, the old definition remains. This + feature can be used to the module’s advantage if it maintains a + global table or cache of objects — with a *note try: d72. statement + it can test for the table’s presence and skip its initialization if + desired: + + try: + cache + except NameError: + cache = {} + + It is legal though generally not very useful to reload built-in or + dynamically loaded modules, except for *note sys: fd, *note + __main__: 1. and *note builtins: 13. In many cases, however, + extension modules are not designed to be initialized more than + once, and may fail in arbitrary ways when reloaded. + + If a module imports objects from another module using *note from: + c45. … *note import: c1d. …, calling *note reload(): c70. for the + other module does not redefine the objects imported from it — one + way around this is to re-execute the ‘from’ statement, another is + to use ‘import’ and qualified names (`module'.*name*) instead. + + If a module instantiates instances of a class, reloading the module + that defines the class does not affect the method definitions of + the instances — they continue to use the old class definition. The + same is true for derived classes. + + Changed in version 3.3: Relies on both ‘__name__’ and ‘__loader__’ + being defined on the module being reloaded instead of just + ‘__name__’. + + Deprecated since version 3.4: Use *note importlib.reload(): 399. + instead. + +The following functions are conveniences for handling PEP 3147(2) +byte-compiled file paths. + +New in version 3.2. + + -- Function: imp.cache_from_source (path, debug_override=None) + + Return the PEP 3147(3) path to the byte-compiled file associated + with the source `path'. For example, if `path' is + ‘/foo/bar/baz.py’ the return value would be + ‘/foo/bar/__pycache__/baz.cpython-32.pyc’ for Python 3.2. The + ‘cpython-32’ string comes from the current magic tag (see *note + get_tag(): 3824.; if ‘sys.implementation.cache_tag’ is not defined + then *note NotImplementedError: 60e. will be raised). By passing + in ‘True’ or ‘False’ for `debug_override' you can override the + system’s value for ‘__debug__’, leading to optimized bytecode. + + `path' need not exist. + + Changed in version 3.3: If ‘sys.implementation.cache_tag’ is + ‘None’, then *note NotImplementedError: 60e. is raised. + + Deprecated since version 3.4: Use *note + importlib.util.cache_from_source(): 557. instead. + + Changed in version 3.5: The `debug_override' parameter no longer + creates a ‘.pyo’ file. + + -- Function: imp.source_from_cache (path) + + Given the `path' to a PEP 3147(4) file name, return the associated + source code file path. For example, if `path' is + ‘/foo/bar/__pycache__/baz.cpython-32.pyc’ the returned path would + be ‘/foo/bar/baz.py’. `path' need not exist, however if it does + not conform to PEP 3147(5) format, a *note ValueError: 1fb. is + raised. If ‘sys.implementation.cache_tag’ is not defined, *note + NotImplementedError: 60e. is raised. + + Changed in version 3.3: Raise *note NotImplementedError: 60e. when + ‘sys.implementation.cache_tag’ is not defined. + + Deprecated since version 3.4: Use *note + importlib.util.source_from_cache(): 558. instead. + + -- Function: imp.get_tag () + + Return the PEP 3147(6) magic tag string matching this version of + Python’s magic number, as returned by *note get_magic(): 863. + + Deprecated since version 3.4: Use ‘sys.implementation.cache_tag’ + directly starting in Python 3.3. + +The following functions help interact with the import system’s internal +locking mechanism. Locking semantics of imports are an implementation +detail which may vary from release to release. However, Python ensures +that circular imports work without any deadlocks. + + -- Function: imp.lock_held () + + Return ‘True’ if the global import lock is currently held, else + ‘False’. On platforms without threads, always return ‘False’. + + On platforms with threads, a thread executing an import first holds + a global import lock, then sets up a per-module lock for the rest + of the import. This blocks other threads from importing the same + module until the original import completes, preventing other + threads from seeing incomplete module objects constructed by the + original thread. An exception is made for circular imports, which + by construction have to expose an incomplete module object at some + point. + + Changed in version 3.3: The locking scheme has changed to + per-module locks for the most part. A global import lock is kept + for some critical tasks, such as initializing the per-module locks. + + Deprecated since version 3.4. + + -- Function: imp.acquire_lock () + + Acquire the interpreter’s global import lock for the current + thread. This lock should be used by import hooks to ensure + thread-safety when importing modules. + + Once a thread has acquired the import lock, the same thread may + acquire it again without blocking; the thread must release it once + for each time it has acquired it. + + On platforms without threads, this function does nothing. + + Changed in version 3.3: The locking scheme has changed to + per-module locks for the most part. A global import lock is kept + for some critical tasks, such as initializing the per-module locks. + + Deprecated since version 3.4. + + -- Function: imp.release_lock () + + Release the interpreter’s global import lock. On platforms without + threads, this function does nothing. + + Changed in version 3.3: The locking scheme has changed to + per-module locks for the most part. A global import lock is kept + for some critical tasks, such as initializing the per-module locks. + + Deprecated since version 3.4. + +The following constants with integer values, defined in this module, are +used to indicate the search result of *note find_module(): 381d. + + -- Data: imp.PY_SOURCE + + The module was found as a source file. + + Deprecated since version 3.3. + + -- Data: imp.PY_COMPILED + + The module was found as a compiled code object file. + + Deprecated since version 3.3. + + -- Data: imp.C_EXTENSION + + The module was found as dynamically loadable shared library. + + Deprecated since version 3.3. + + -- Data: imp.PKG_DIRECTORY + + The module was found as a package directory. + + Deprecated since version 3.3. + + -- Data: imp.C_BUILTIN + + The module was found as a built-in module. + + Deprecated since version 3.3. + + -- Data: imp.PY_FROZEN + + The module was found as a frozen module. + + Deprecated since version 3.3. + + -- Class: imp.NullImporter (path_string) + + The *note NullImporter: 9bb. type is a PEP 302(7) import hook that + handles non-directory path strings by failing to find any modules. + Calling this type with an existing directory or empty string raises + *note ImportError: 334. Otherwise, a *note NullImporter: 9bb. + instance is returned. + + Instances have only one method: + + -- Method: find_module (fullname[, path]) + + This method always returns ‘None’, indicating that the + requested module could not be found. + + Changed in version 3.3: ‘None’ is inserted into + ‘sys.path_importer_cache’ instead of an instance of *note + NullImporter: 9bb. + + Deprecated since version 3.4: Insert ‘None’ into + ‘sys.path_importer_cache’ instead. + +* Menu: + +* Examples: Examples<33>. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Lib/imp.py + + (2) https://www.python.org/dev/peps/pep-3147 + + (3) https://www.python.org/dev/peps/pep-3147 + + (4) https://www.python.org/dev/peps/pep-3147 + + (5) https://www.python.org/dev/peps/pep-3147 + + (6) https://www.python.org/dev/peps/pep-3147 + + (7) https://www.python.org/dev/peps/pep-0302 + + +File: python.info, Node: Examples<33>, Up: imp — Access the import internals + +5.36.2.1 Examples +................. + +The following function emulates what was the standard import statement +up to Python 1.4 (no hierarchical module names). (This `implementation' +wouldn’t work in that version, since *note find_module(): 381d. has been +extended and *note load_module(): 3821. has been added in 1.4.) + + import imp + import sys + + def __import__(name, globals=None, locals=None, fromlist=None): + # Fast path: see if the module has already been imported. + try: + return sys.modules[name] + except KeyError: + pass + + # If any of the following calls raises an exception, + # there's a problem we can't handle -- let the caller handle it. + + fp, pathname, description = imp.find_module(name) + + try: + return imp.load_module(name, fp, pathname, description) + finally: + # Since we may exit via an exception, close fp explicitly. + if fp: + fp.close() + + +File: python.info, Node: Undocumented Modules, Prev: Superseded Modules, Up: The Python Standard Library + +5.37 Undocumented Modules +========================= + +Here’s a quick listing of modules that are currently undocumented, but +that should be documented. Feel free to contribute documentation for +them! (Send via email to <docs@python.org>.) + +The idea and original contents for this chapter were taken from a +posting by Fredrik Lundh; the specific contents of this chapter have +been substantially revised. + +* Menu: + +* Platform specific modules:: + + +File: python.info, Node: Platform specific modules, Up: Undocumented Modules + +5.37.1 Platform specific modules +-------------------------------- + +These modules are used to implement the *note os.path: c5. module, and +are not documented beyond this mention. There’s little need to document +these. + +‘ntpath’ + + — Implementation of *note os.path: c5. on Win32 and Win64 + platforms. + +‘posixpath’ + + — Implementation of *note os.path: c5. on POSIX. + + +File: python.info, Node: Extending and Embedding the Python Interpreter, Next: Python/C API Reference Manual, Prev: The Python Standard Library, Up: Top + +6 Extending and Embedding the Python Interpreter +************************************************ + +This document describes how to write modules in C or C++ to extend the +Python interpreter with new modules. Those modules can not only define +new functions but also new object types and their methods. The document +also describes how to embed the Python interpreter in another +application, for use as an extension language. Finally, it shows how to +compile and link extension modules so that they can be loaded +dynamically (at run time) into the interpreter, if the underlying +operating system supports this feature. + +This document assumes basic knowledge about Python. For an informal +introduction to the language, see *note The Python Tutorial: e8d. *note +The Python Language Reference: e8f. gives a more formal definition of +the language. *note The Python Standard Library: e8e. documents the +existing object types, functions and modules (both built-in and written +in Python) that give the language its wide application range. + +For a detailed description of the whole Python/C API, see the separate +*note Python/C API Reference Manual: e91. + +* Menu: + +* Recommended third party tools:: +* Creating extensions without third party tools:: +* Embedding the CPython runtime in a larger application:: + + +File: python.info, Node: Recommended third party tools, Next: Creating extensions without third party tools, Up: Extending and Embedding the Python Interpreter + +6.1 Recommended third party tools +================================= + +This guide only covers the basic tools for creating extensions provided +as part of this version of CPython. Third party tools like Cython(1), +cffi(2), SWIG(3) and Numba(4) offer both simpler and more sophisticated +approaches to creating C and C++ extensions for Python. + +See also +........ + +Python Packaging User Guide: Binary Extensions(5) + + The Python Packaging User Guide not only covers several available + tools that simplify the creation of binary extensions, but also + discusses the various reasons why creating an extension module may + be desirable in the first place. + + ---------- Footnotes ---------- + + (1) http://cython.org/ + + (2) https://cffi.readthedocs.io + + (3) http://www.swig.org + + (4) https://numba.pydata.org/ + + (5) https://packaging.python.org/guides/packaging-binary-extensions/ + + +File: python.info, Node: Creating extensions without third party tools, Next: Embedding the CPython runtime in a larger application, Prev: Recommended third party tools, Up: Extending and Embedding the Python Interpreter + +6.2 Creating extensions without third party tools +================================================= + +This section of the guide covers creating C and C++ extensions without +assistance from third party tools. It is intended primarily for +creators of those tools, rather than being a recommended way to create +your own C extensions. + +* Menu: + +* Extending Python with C or C++:: +* Defining Extension Types; Tutorial: Defining Extension Types Tutorial. +* Defining Extension Types; Assorted Topics: Defining Extension Types Assorted Topics. +* Building C and C++ Extensions:: +* Building C and C++ Extensions on Windows:: + + +File: python.info, Node: Extending Python with C or C++, Next: Defining Extension Types Tutorial, Up: Creating extensions without third party tools + +6.2.1 Extending Python with C or C++ +------------------------------------ + +It is quite easy to add new built-in modules to Python, if you know how +to program in C. Such `extension modules' can do two things that can’t +be done directly in Python: they can implement new built-in object +types, and they can call C library functions and system calls. + +To support extensions, the Python API (Application Programmers +Interface) defines a set of functions, macros and variables that provide +access to most aspects of the Python run-time system. The Python API is +incorporated in a C source file by including the header ‘"Python.h"’. + +The compilation of an extension module depends on its intended use as +well as on your system setup; details are given in later chapters. + + Note: The C extension interface is specific to CPython, and + extension modules do not work on other Python implementations. In + many cases, it is possible to avoid writing C extensions and + preserve portability to other implementations. For example, if + your use case is calling C library functions or system calls, you + should consider using the *note ctypes: 2b. module or the cffi(1) + library rather than writing custom C code. These modules let you + write Python code to interface with C code and are more portable + between implementations of Python than writing and compiling a C + extension module. + +* Menu: + +* A Simple Example:: +* Intermezzo; Errors and Exceptions: Intermezzo Errors and Exceptions. +* Back to the Example:: +* The Module’s Method Table and Initialization Function:: +* Compilation and Linkage:: +* Calling Python Functions from C:: +* Extracting Parameters in Extension Functions:: +* Keyword Parameters for Extension Functions:: +* Building Arbitrary Values:: +* Reference Counts:: +* Writing Extensions in C++:: +* Providing a C API for an Extension Module:: + + ---------- Footnotes ---------- + + (1) https://cffi.readthedocs.io/ + + +File: python.info, Node: A Simple Example, Next: Intermezzo Errors and Exceptions, Up: Extending Python with C or C++ + +6.2.1.1 A Simple Example +........................ + +Let’s create an extension module called ‘spam’ (the favorite food of +Monty Python fans…) and let’s say we want to create a Python interface +to the C library function ‘system()’ (1). This function takes a +null-terminated character string as argument and returns an integer. We +want this function to be callable from Python as follows: + + >>> import spam + >>> status = spam.system("ls -l") + +Begin by creating a file ‘spammodule.c’. (Historically, if a module is +called ‘spam’, the C file containing its implementation is called +‘spammodule.c’; if the module name is very long, like ‘spammify’, the +module name can be just ‘spammify.c’.) + +The first two lines of our file can be: + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + +which pulls in the Python API (you can add a comment describing the +purpose of the module and a copyright notice if you like). + + Note: Since Python may define some pre-processor definitions which + affect the standard headers on some systems, you `must' include + ‘Python.h’ before any standard headers are included. + + It is recommended to always define ‘PY_SSIZE_T_CLEAN’ before + including ‘Python.h’. See *note Extracting Parameters in Extension + Functions: 3839. for a description of this macro. + +All user-visible symbols defined by ‘Python.h’ have a prefix of ‘Py’ or +‘PY’, except those defined in standard header files. For convenience, +and since they are used extensively by the Python interpreter, +‘"Python.h"’ includes a few standard header files: ‘<stdio.h>’, +‘<string.h>’, ‘<errno.h>’, and ‘<stdlib.h>’. If the latter header file +does not exist on your system, it declares the functions ‘malloc()’, +‘free()’ and ‘realloc()’ directly. + +The next thing we add to our module file is the C function that will be +called when the Python expression ‘spam.system(string)’ is evaluated +(we’ll see shortly how it ends up being called): + + static PyObject * + spam_system(PyObject *self, PyObject *args) + { + const char *command; + int sts; + + if (!PyArg_ParseTuple(args, "s", &command)) + return NULL; + sts = system(command); + return PyLong_FromLong(sts); + } + +There is a straightforward translation from the argument list in Python +(for example, the single expression ‘"ls -l"’) to the arguments passed +to the C function. The C function always has two arguments, +conventionally named `self' and `args'. + +The `self' argument points to the module object for module-level +functions; for a method it would point to the object instance. + +The `args' argument will be a pointer to a Python tuple object +containing the arguments. Each item of the tuple corresponds to an +argument in the call’s argument list. The arguments are Python objects +— in order to do anything with them in our C function we have to convert +them to C values. The function *note PyArg_ParseTuple(): 26a. in the +Python API checks the argument types and converts them to C values. It +uses a template string to determine the required types of the arguments +as well as the types of the C variables into which to store the +converted values. More about this later. + +*note PyArg_ParseTuple(): 26a. returns true (nonzero) if all arguments +have the right type and its components have been stored in the variables +whose addresses are passed. It returns false (zero) if an invalid +argument list was passed. In the latter case it also raises an +appropriate exception so the calling function can return ‘NULL’ +immediately (as we saw in the example). + + ---------- Footnotes ---------- + + (1) (1) An interface for this function already exists in the standard +module *note os: c4. — it was chosen as a simple and straightforward +example. + + +File: python.info, Node: Intermezzo Errors and Exceptions, Next: Back to the Example, Prev: A Simple Example, Up: Extending Python with C or C++ + +6.2.1.2 Intermezzo: Errors and Exceptions +......................................... + +An important convention throughout the Python interpreter is the +following: when a function fails, it should set an exception condition +and return an error value (usually a ‘NULL’ pointer). Exceptions are +stored in a static global variable inside the interpreter; if this +variable is ‘NULL’ no exception has occurred. A second global variable +stores the “associated value” of the exception (the second argument to +*note raise: c42.). A third variable contains the stack traceback in +case the error originated in Python code. These three variables are the +C equivalents of the result in Python of *note sys.exc_info(): c5f. (see +the section on module *note sys: fd. in the Python Library Reference). +It is important to know about them to understand how errors are passed +around. + +The Python API defines a number of functions to set various types of +exceptions. + +The most common one is *note PyErr_SetString(): 383c. Its arguments are +an exception object and a C string. The exception object is usually a +predefined object like ‘PyExc_ZeroDivisionError’. The C string +indicates the cause of the error and is converted to a Python string +object and stored as the “associated value” of the exception. + +Another useful function is *note PyErr_SetFromErrno(): 383d, which only +takes an exception argument and constructs the associated value by +inspection of the global variable ‘errno’. The most general function is +*note PyErr_SetObject(): 383e, which takes two object arguments, the +exception and its associated value. You don’t need to *note +Py_INCREF(): 265. the objects passed to any of these functions. + +You can test non-destructively whether an exception has been set with +*note PyErr_Occurred(): 383f. This returns the current exception +object, or ‘NULL’ if no exception has occurred. You normally don’t need +to call *note PyErr_Occurred(): 383f. to see whether an error occurred +in a function call, since you should be able to tell from the return +value. + +When a function `f' that calls another function `g' detects that the +latter fails, `f' should itself return an error value (usually ‘NULL’ or +‘-1’). It should `not' call one of the ‘PyErr_*()’ functions — one has +already been called by `g'. `f'’s caller is then supposed to also +return an error indication to `its' caller, again `without' calling +‘PyErr_*()’, and so on — the most detailed cause of the error was +already reported by the function that first detected it. Once the error +reaches the Python interpreter’s main loop, this aborts the currently +executing Python code and tries to find an exception handler specified +by the Python programmer. + +(There are situations where a module can actually give a more detailed +error message by calling another ‘PyErr_*()’ function, and in such cases +it is fine to do so. As a general rule, however, this is not necessary, +and can cause information about the cause of the error to be lost: most +operations can fail for a variety of reasons.) + +To ignore an exception set by a function call that failed, the exception +condition must be cleared explicitly by calling *note PyErr_Clear(): +96b. The only time C code should call *note PyErr_Clear(): 96b. is if +it doesn’t want to pass the error on to the interpreter but wants to +handle it completely by itself (possibly by trying something else, or +pretending nothing went wrong). + +Every failing ‘malloc()’ call must be turned into an exception — the +direct caller of ‘malloc()’ (or ‘realloc()’) must call *note +PyErr_NoMemory(): 3840. and return a failure indicator itself. All the +object-creating functions (for example, *note PyLong_FromLong(): 3841.) +already do this, so this note is only relevant to those who call +‘malloc()’ directly. + +Also note that, with the important exception of *note +PyArg_ParseTuple(): 26a. and friends, functions that return an integer +status usually return a positive value or zero for success and ‘-1’ for +failure, like Unix system calls. + +Finally, be careful to clean up garbage (by making *note Py_XDECREF(): +268. or *note Py_DECREF(): 266. calls for objects you have already +created) when you return an error indicator! + +The choice of which exception to raise is entirely yours. There are +predeclared C objects corresponding to all built-in Python exceptions, +such as ‘PyExc_ZeroDivisionError’, which you can use directly. Of +course, you should choose exceptions wisely — don’t use +‘PyExc_TypeError’ to mean that a file couldn’t be opened (that should +probably be ‘PyExc_IOError’). If something’s wrong with the argument +list, the *note PyArg_ParseTuple(): 26a. function usually raises +‘PyExc_TypeError’. If you have an argument whose value must be in a +particular range or must satisfy other conditions, ‘PyExc_ValueError’ is +appropriate. + +You can also define a new exception that is unique to your module. For +this, you usually declare a static object variable at the beginning of +your file: + + static PyObject *SpamError; + +and initialize it in your module’s initialization function +(‘PyInit_spam()’) with an exception object: + + PyMODINIT_FUNC + PyInit_spam(void) + { + PyObject *m; + + m = PyModule_Create(&spammodule); + if (m == NULL) + return NULL; + + SpamError = PyErr_NewException("spam.error", NULL, NULL); + Py_XINCREF(SpamError); + if (PyModule_AddObject(m, "error", SpamError) < 0) { + Py_XDECREF(SpamError); + Py_CLEAR(SpamError); + Py_DECREF(m); + return NULL; + } + + return m; + } + +Note that the Python name for the exception object is ‘spam.error’. The +*note PyErr_NewException(): c00. function may create a class with the +base class being *note Exception: 1a9. (unless another class is passed +in instead of ‘NULL’), described in *note Built-in Exceptions: f43. + +Note also that the ‘SpamError’ variable retains a reference to the newly +created exception class; this is intentional! Since the exception could +be removed from the module by external code, an owned reference to the +class is needed to ensure that it will not be discarded, causing +‘SpamError’ to become a dangling pointer. Should it become a dangling +pointer, C code which raises the exception could cause a core dump or +other unintended side effects. + +We discuss the use of ‘PyMODINIT_FUNC’ as a function return type later +in this sample. + +The ‘spam.error’ exception can be raised in your extension module using +a call to *note PyErr_SetString(): 383c. as shown below: + + static PyObject * + spam_system(PyObject *self, PyObject *args) + { + const char *command; + int sts; + + if (!PyArg_ParseTuple(args, "s", &command)) + return NULL; + sts = system(command); + if (sts < 0) { + PyErr_SetString(SpamError, "System command failed"); + return NULL; + } + return PyLong_FromLong(sts); + } + + +File: python.info, Node: Back to the Example, Next: The Module’s Method Table and Initialization Function, Prev: Intermezzo Errors and Exceptions, Up: Extending Python with C or C++ + +6.2.1.3 Back to the Example +........................... + +Going back to our example function, you should now be able to understand +this statement: + + if (!PyArg_ParseTuple(args, "s", &command)) + return NULL; + +It returns ‘NULL’ (the error indicator for functions returning object +pointers) if an error is detected in the argument list, relying on the +exception set by *note PyArg_ParseTuple(): 26a. Otherwise the string +value of the argument has been copied to the local variable ‘command’. +This is a pointer assignment and you are not supposed to modify the +string to which it points (so in Standard C, the variable ‘command’ +should properly be declared as ‘const char *command’). + +The next statement is a call to the Unix function ‘system()’, passing it +the string we just got from *note PyArg_ParseTuple(): 26a.: + + sts = system(command); + +Our ‘spam.system()’ function must return the value of ‘sts’ as a Python +object. This is done using the function *note PyLong_FromLong(): 3841. + + return PyLong_FromLong(sts); + +In this case, it will return an integer object. (Yes, even integers are +objects on the heap in Python!) + +If you have a C function that returns no useful argument (a function +returning ‘void’), the corresponding Python function must return ‘None’. +You need this idiom to do so (which is implemented by the *note +Py_RETURN_NONE: dd6. macro): + + Py_INCREF(Py_None); + return Py_None; + +*note Py_None: 3844. is the C name for the special Python object ‘None’. +It is a genuine Python object rather than a ‘NULL’ pointer, which means +“error” in most contexts, as we have seen. + + +File: python.info, Node: The Module’s Method Table and Initialization Function, Next: Compilation and Linkage, Prev: Back to the Example, Up: Extending Python with C or C++ + +6.2.1.4 The Module’s Method Table and Initialization Function +............................................................. + +I promised to show how ‘spam_system()’ is called from Python programs. +First, we need to list its name and address in a “method table”: + + static PyMethodDef SpamMethods[] = { + ... + {"system", spam_system, METH_VARARGS, + "Execute a shell command."}, + ... + {NULL, NULL, 0, NULL} /* Sentinel */ + }; + +Note the third entry (‘METH_VARARGS’). This is a flag telling the +interpreter the calling convention to be used for the C function. It +should normally always be ‘METH_VARARGS’ or ‘METH_VARARGS | +METH_KEYWORDS’; a value of ‘0’ means that an obsolete variant of *note +PyArg_ParseTuple(): 26a. is used. + +When using only ‘METH_VARARGS’, the function should expect the +Python-level parameters to be passed in as a tuple acceptable for +parsing via *note PyArg_ParseTuple(): 26a.; more information on this +function is provided below. + +The ‘METH_KEYWORDS’ bit may be set in the third field if keyword +arguments should be passed to the function. In this case, the C +function should accept a third ‘PyObject *’ parameter which will be a +dictionary of keywords. Use *note PyArg_ParseTupleAndKeywords(): 5ca. +to parse the arguments to such a function. + +The method table must be referenced in the module definition structure: + + static struct PyModuleDef spammodule = { + PyModuleDef_HEAD_INIT, + "spam", /* name of module */ + spam_doc, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + SpamMethods + }; + +This structure, in turn, must be passed to the interpreter in the +module’s initialization function. The initialization function must be +named ‘PyInit_name()’, where `name' is the name of the module, and +should be the only non-‘static’ item defined in the module file: + + PyMODINIT_FUNC + PyInit_spam(void) + { + return PyModule_Create(&spammodule); + } + +Note that PyMODINIT_FUNC declares the function as ‘PyObject *’ return +type, declares any special linkage declarations required by the +platform, and for C++ declares the function as ‘extern "C"’. + +When the Python program imports module ‘spam’ for the first time, +‘PyInit_spam()’ is called. (See below for comments about embedding +Python.) It calls *note PyModule_Create(): 3847, which returns a module +object, and inserts built-in function objects into the newly created +module based upon the table (an array of *note PyMethodDef: e1b. +structures) found in the module definition. *note PyModule_Create(): +3847. returns a pointer to the module object that it creates. It may +abort with a fatal error for certain errors, or return ‘NULL’ if the +module could not be initialized satisfactorily. The init function must +return the module object to its caller, so that it then gets inserted +into ‘sys.modules’. + +When embedding Python, the ‘PyInit_spam()’ function is not called +automatically unless there’s an entry in the ‘PyImport_Inittab’ table. +To add the module to the initialization table, use *note +PyImport_AppendInittab(): 3848, optionally followed by an import of the +module: + + int + main(int argc, char *argv[]) + { + wchar_t *program = Py_DecodeLocale(argv[0], NULL); + if (program == NULL) { + fprintf(stderr, "Fatal error: cannot decode argv[0]\n"); + exit(1); + } + + /* Add a built-in module, before Py_Initialize */ + if (PyImport_AppendInittab("spam", PyInit_spam) == -1) { + fprintf(stderr, "Error: could not extend in-built modules table\n"); + exit(1); + } + + /* Pass argv[0] to the Python interpreter */ + Py_SetProgramName(program); + + /* Initialize the Python interpreter. Required. + If this step fails, it will be a fatal error. */ + Py_Initialize(); + + /* Optionally import the module; alternatively, + import can be deferred until the embedded script + imports it. */ + pmodule = PyImport_ImportModule("spam"); + if (!pmodule) { + PyErr_Print(); + fprintf(stderr, "Error: could not import module 'spam'\n"); + } + + ... + + PyMem_RawFree(program); + return 0; + } + + Note: Removing entries from ‘sys.modules’ or importing compiled + modules into multiple interpreters within a process (or following a + ‘fork()’ without an intervening ‘exec()’) can create problems for + some extension modules. Extension module authors should exercise + caution when initializing internal data structures. + +A more substantial example module is included in the Python source +distribution as ‘Modules/xxmodule.c’. This file may be used as a +template or simply read as an example. + + Note: Unlike our ‘spam’ example, ‘xxmodule’ uses `multi-phase + initialization' (new in Python 3.5), where a PyModuleDef structure + is returned from ‘PyInit_spam’, and creation of the module is left + to the import machinery. For details on multi-phase + initialization, see PEP 489(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0489 + + +File: python.info, Node: Compilation and Linkage, Next: Calling Python Functions from C, Prev: The Module’s Method Table and Initialization Function, Up: Extending Python with C or C++ + +6.2.1.5 Compilation and Linkage +............................... + +There are two more things to do before you can use your new extension: +compiling and linking it with the Python system. If you use dynamic +loading, the details may depend on the style of dynamic loading your +system uses; see the chapters about building extension modules (chapter +*note Building C and C++ Extensions: 384b.) and additional information +that pertains only to building on Windows (chapter *note Building C and +C++ Extensions on Windows: 1039.) for more information about this. + +If you can’t use dynamic loading, or if you want to make your module a +permanent part of the Python interpreter, you will have to change the +configuration setup and rebuild the interpreter. Luckily, this is very +simple on Unix: just place your file (‘spammodule.c’ for example) in the +‘Modules/’ directory of an unpacked source distribution, add a line to +the file ‘Modules/Setup.local’ describing your file: + + spam spammodule.o + +and rebuild the interpreter by running ‘make’ in the toplevel directory. +You can also run ‘make’ in the ‘Modules/’ subdirectory, but then you +must first rebuild ‘Makefile’ there by running ‘‘make’ Makefile’. (This +is necessary each time you change the ‘Setup’ file.) + +If your module requires additional libraries to link with, these can be +listed on the line in the configuration file as well, for instance: + + spam spammodule.o -lX11 + + +File: python.info, Node: Calling Python Functions from C, Next: Extracting Parameters in Extension Functions, Prev: Compilation and Linkage, Up: Extending Python with C or C++ + +6.2.1.6 Calling Python Functions from C +....................................... + +So far we have concentrated on making C functions callable from Python. +The reverse is also useful: calling Python functions from C. This is +especially the case for libraries that support so-called “callback” +functions. If a C interface makes use of callbacks, the equivalent +Python often needs to provide a callback mechanism to the Python +programmer; the implementation will require calling the Python callback +functions from a C callback. Other uses are also imaginable. + +Fortunately, the Python interpreter is easily called recursively, and +there is a standard interface to call a Python function. (I won’t dwell +on how to call the Python parser with a particular string as input — if +you’re interested, have a look at the implementation of the *note -c: +e9e. command line option in ‘Modules/main.c’ from the Python source +code.) + +Calling a Python function is easy. First, the Python program must +somehow pass you the Python function object. You should provide a +function (or some other interface) to do this. When this function is +called, save a pointer to the Python function object (be careful to +*note Py_INCREF(): 265. it!) in a global variable — or wherever you see +fit. For example, the following function might be part of a module +definition: + + static PyObject *my_callback = NULL; + + static PyObject * + my_set_callback(PyObject *dummy, PyObject *args) + { + PyObject *result = NULL; + PyObject *temp; + + if (PyArg_ParseTuple(args, "O:set_callback", &temp)) { + if (!PyCallable_Check(temp)) { + PyErr_SetString(PyExc_TypeError, "parameter must be callable"); + return NULL; + } + Py_XINCREF(temp); /* Add a reference to new callback */ + Py_XDECREF(my_callback); /* Dispose of previous callback */ + my_callback = temp; /* Remember new callback */ + /* Boilerplate to return "None" */ + Py_INCREF(Py_None); + result = Py_None; + } + return result; + } + +This function must be registered with the interpreter using the *note +METH_VARARGS: e48. flag; this is described in section *note The Module’s +Method Table and Initialization Function: 3845. The *note +PyArg_ParseTuple(): 26a. function and its arguments are documented in +section *note Extracting Parameters in Extension Functions: 3839. + +The macros *note Py_XINCREF(): 267. and *note Py_XDECREF(): 268. +increment/decrement the reference count of an object and are safe in the +presence of ‘NULL’ pointers (but note that `temp' will not be ‘NULL’ in +this context). More info on them in section *note Reference Counts: +384e. + +Later, when it is time to call the function, you call the C function +*note PyObject_CallObject(): 384f. This function has two arguments, +both pointers to arbitrary Python objects: the Python function, and the +argument list. The argument list must always be a tuple object, whose +length is the number of arguments. To call the Python function with no +arguments, pass in ‘NULL’, or an empty tuple; to call it with one +argument, pass a singleton tuple. *note Py_BuildValue(): 2ce. returns a +tuple when its format string consists of zero or more format codes +between parentheses. For example: + + int arg; + PyObject *arglist; + PyObject *result; + ... + arg = 123; + ... + /* Time to call the callback */ + arglist = Py_BuildValue("(i)", arg); + result = PyObject_CallObject(my_callback, arglist); + Py_DECREF(arglist); + +*note PyObject_CallObject(): 384f. returns a Python object pointer: this +is the return value of the Python function. *note +PyObject_CallObject(): 384f. is “reference-count-neutral” with respect +to its arguments. In the example a new tuple was created to serve as +the argument list, which is *note Py_DECREF(): 266.-ed immediately after +the *note PyObject_CallObject(): 384f. call. + +The return value of *note PyObject_CallObject(): 384f. is “new”: either +it is a brand new object, or it is an existing object whose reference +count has been incremented. So, unless you want to save it in a global +variable, you should somehow *note Py_DECREF(): 266. the result, even +(especially!) if you are not interested in its value. + +Before you do this, however, it is important to check that the return +value isn’t ‘NULL’. If it is, the Python function terminated by raising +an exception. If the C code that called *note PyObject_CallObject(): +384f. is called from Python, it should now return an error indication to +its Python caller, so the interpreter can print a stack trace, or the +calling Python code can handle the exception. If this is not possible +or desirable, the exception should be cleared by calling *note +PyErr_Clear(): 96b. For example: + + if (result == NULL) + return NULL; /* Pass error back */ + ...use result... + Py_DECREF(result); + +Depending on the desired interface to the Python callback function, you +may also have to provide an argument list to *note +PyObject_CallObject(): 384f. In some cases the argument list is also +provided by the Python program, through the same interface that +specified the callback function. It can then be saved and used in the +same manner as the function object. In other cases, you may have to +construct a new tuple to pass as the argument list. The simplest way to +do this is to call *note Py_BuildValue(): 2ce. For example, if you want +to pass an integral event code, you might use the following code: + + PyObject *arglist; + ... + arglist = Py_BuildValue("(l)", eventcode); + result = PyObject_CallObject(my_callback, arglist); + Py_DECREF(arglist); + if (result == NULL) + return NULL; /* Pass error back */ + /* Here maybe use the result */ + Py_DECREF(result); + +Note the placement of ‘Py_DECREF(arglist)’ immediately after the call, +before the error check! Also note that strictly speaking this code is +not complete: *note Py_BuildValue(): 2ce. may run out of memory, and +this should be checked. + +You may also call a function with keyword arguments by using *note +PyObject_Call(): 3850, which supports arguments and keyword arguments. +As in the above example, we use *note Py_BuildValue(): 2ce. to construct +the dictionary. + + PyObject *dict; + ... + dict = Py_BuildValue("{s:i}", "name", val); + result = PyObject_Call(my_callback, NULL, dict); + Py_DECREF(dict); + if (result == NULL) + return NULL; /* Pass error back */ + /* Here maybe use the result */ + Py_DECREF(result); + + +File: python.info, Node: Extracting Parameters in Extension Functions, Next: Keyword Parameters for Extension Functions, Prev: Calling Python Functions from C, Up: Extending Python with C or C++ + +6.2.1.7 Extracting Parameters in Extension Functions +.................................................... + +The *note PyArg_ParseTuple(): 26a. function is declared as follows: + + int PyArg_ParseTuple(PyObject *arg, const char *format, ...); + +The `arg' argument must be a tuple object containing an argument list +passed from Python to a C function. The `format' argument must be a +format string, whose syntax is explained in *note Parsing arguments and +building values: 2d0. in the Python/C API Reference Manual. The +remaining arguments must be addresses of variables whose type is +determined by the format string. + +Note that while *note PyArg_ParseTuple(): 26a. checks that the Python +arguments have the required types, it cannot check the validity of the +addresses of C variables passed to the call: if you make mistakes there, +your code will probably crash or at least overwrite random bits in +memory. So be careful! + +Note that any Python object references which are provided to the caller +are `borrowed' references; do not decrement their reference count! + +Some example calls: + + #define PY_SSIZE_T_CLEAN /* Make "s#" use Py_ssize_t rather than int. */ + #include <Python.h> + + int ok; + int i, j; + long k, l; + const char *s; + Py_ssize_t size; + + ok = PyArg_ParseTuple(args, ""); /* No arguments */ + /* Python call: f() */ + + ok = PyArg_ParseTuple(args, "s", &s); /* A string */ + /* Possible Python call: f('whoops!') */ + + ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */ + /* Possible Python call: f(1, 2, 'three') */ + + ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size); + /* A pair of ints and a string, whose size is also returned */ + /* Possible Python call: f((1, 2), 'three') */ + + { + const char *file; + const char *mode = "r"; + int bufsize = 0; + ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize); + /* A string, and optionally another string and an integer */ + /* Possible Python calls: + f('spam') + f('spam', 'w') + f('spam', 'wb', 100000) */ + } + + { + int left, top, right, bottom, h, v; + ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)", + &left, &top, &right, &bottom, &h, &v); + /* A rectangle and a point */ + /* Possible Python call: + f(((0, 0), (400, 300)), (10, 10)) */ + } + + { + Py_complex c; + ok = PyArg_ParseTuple(args, "D:myfunction", &c); + /* a complex, also providing a function name for errors */ + /* Possible Python call: myfunction(1+2j) */ + } + + +File: python.info, Node: Keyword Parameters for Extension Functions, Next: Building Arbitrary Values, Prev: Extracting Parameters in Extension Functions, Up: Extending Python with C or C++ + +6.2.1.8 Keyword Parameters for Extension Functions +.................................................. + +The *note PyArg_ParseTupleAndKeywords(): 5ca. function is declared as +follows: + + int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict, + const char *format, char *kwlist[], ...); + +The `arg' and `format' parameters are identical to those of the *note +PyArg_ParseTuple(): 26a. function. The `kwdict' parameter is the +dictionary of keywords received as the third parameter from the Python +runtime. The `kwlist' parameter is a ‘NULL’-terminated list of strings +which identify the parameters; the names are matched with the type +information from `format' from left to right. On success, *note +PyArg_ParseTupleAndKeywords(): 5ca. returns true, otherwise it returns +false and raises an appropriate exception. + + Note: Nested tuples cannot be parsed when using keyword arguments! + Keyword parameters passed in which are not present in the `kwlist' + will cause *note TypeError: 192. to be raised. + +Here is an example module which uses keywords, based on an example by +Geoff Philbrick (<philbrick@hks.com>): + + #define PY_SSIZE_T_CLEAN /* Make "s#" use Py_ssize_t rather than int. */ + #include <Python.h> + + static PyObject * + keywdarg_parrot(PyObject *self, PyObject *args, PyObject *keywds) + { + int voltage; + const char *state = "a stiff"; + const char *action = "voom"; + const char *type = "Norwegian Blue"; + + static char *kwlist[] = {"voltage", "state", "action", "type", NULL}; + + if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist, + &voltage, &state, &action, &type)) + return NULL; + + printf("-- This parrot wouldn't %s if you put %i Volts through it.\n", + action, voltage); + printf("-- Lovely plumage, the %s -- It's %s!\n", type, state); + + Py_RETURN_NONE; + } + + static PyMethodDef keywdarg_methods[] = { + /* The cast of the function is necessary since PyCFunction values + * only take two PyObject* parameters, and keywdarg_parrot() takes + * three. + */ + {"parrot", (PyCFunction)(void(*)(void))keywdarg_parrot, METH_VARARGS | METH_KEYWORDS, + "Print a lovely skit to standard output."}, + {NULL, NULL, 0, NULL} /* sentinel */ + }; + + static struct PyModuleDef keywdargmodule = { + PyModuleDef_HEAD_INIT, + "keywdarg", + NULL, + -1, + keywdarg_methods + }; + + PyMODINIT_FUNC + PyInit_keywdarg(void) + { + return PyModule_Create(&keywdargmodule); + } + + +File: python.info, Node: Building Arbitrary Values, Next: Reference Counts, Prev: Keyword Parameters for Extension Functions, Up: Extending Python with C or C++ + +6.2.1.9 Building Arbitrary Values +................................. + +This function is the counterpart to *note PyArg_ParseTuple(): 26a. It +is declared as follows: + + PyObject *Py_BuildValue(const char *format, ...); + +It recognizes a set of format units similar to the ones recognized by +*note PyArg_ParseTuple(): 26a, but the arguments (which are input to the +function, not output) must not be pointers, just values. It returns a +new Python object, suitable for returning from a C function called from +Python. + +One difference with *note PyArg_ParseTuple(): 26a.: while the latter +requires its first argument to be a tuple (since Python argument lists +are always represented as tuples internally), *note Py_BuildValue(): +2ce. does not always build a tuple. It builds a tuple only if its +format string contains two or more format units. If the format string +is empty, it returns ‘None’; if it contains exactly one format unit, it +returns whatever object is described by that format unit. To force it +to return a tuple of size 0 or one, parenthesize the format string. + +Examples (to the left the call, to the right the resulting Python +value): + + Py_BuildValue("") None + Py_BuildValue("i", 123) 123 + Py_BuildValue("iii", 123, 456, 789) (123, 456, 789) + Py_BuildValue("s", "hello") 'hello' + Py_BuildValue("y", "hello") b'hello' + Py_BuildValue("ss", "hello", "world") ('hello', 'world') + Py_BuildValue("s#", "hello", 4) 'hell' + Py_BuildValue("y#", "hello", 4) b'hell' + Py_BuildValue("()") () + Py_BuildValue("(i)", 123) (123,) + Py_BuildValue("(ii)", 123, 456) (123, 456) + Py_BuildValue("(i,i)", 123, 456) (123, 456) + Py_BuildValue("[i,i]", 123, 456) [123, 456] + Py_BuildValue("{s:i,s:i}", + "abc", 123, "def", 456) {'abc': 123, 'def': 456} + Py_BuildValue("((ii)(ii)) (ii)", + 1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6)) + + +File: python.info, Node: Reference Counts, Next: Writing Extensions in C++, Prev: Building Arbitrary Values, Up: Extending Python with C or C++ + +6.2.1.10 Reference Counts +......................... + +In languages like C or C++, the programmer is responsible for dynamic +allocation and deallocation of memory on the heap. In C, this is done +using the functions ‘malloc()’ and ‘free()’. In C++, the operators +‘new’ and ‘delete’ are used with essentially the same meaning and we’ll +restrict the following discussion to the C case. + +Every block of memory allocated with ‘malloc()’ should eventually be +returned to the pool of available memory by exactly one call to +‘free()’. It is important to call ‘free()’ at the right time. If a +block’s address is forgotten but ‘free()’ is not called for it, the +memory it occupies cannot be reused until the program terminates. This +is called a `memory leak'. On the other hand, if a program calls +‘free()’ for a block and then continues to use the block, it creates a +conflict with re-use of the block through another ‘malloc()’ call. This +is called `using freed memory'. It has the same bad consequences as +referencing uninitialized data — core dumps, wrong results, mysterious +crashes. + +Common causes of memory leaks are unusual paths through the code. For +instance, a function may allocate a block of memory, do some +calculation, and then free the block again. Now a change in the +requirements for the function may add a test to the calculation that +detects an error condition and can return prematurely from the function. +It’s easy to forget to free the allocated memory block when taking this +premature exit, especially when it is added later to the code. Such +leaks, once introduced, often go undetected for a long time: the error +exit is taken only in a small fraction of all calls, and most modern +machines have plenty of virtual memory, so the leak only becomes +apparent in a long-running process that uses the leaking function +frequently. Therefore, it’s important to prevent leaks from happening +by having a coding convention or strategy that minimizes this kind of +errors. + +Since Python makes heavy use of ‘malloc()’ and ‘free()’, it needs a +strategy to avoid memory leaks as well as the use of freed memory. The +chosen method is called `reference counting'. The principle is simple: +every object contains a counter, which is incremented when a reference +to the object is stored somewhere, and which is decremented when a +reference to it is deleted. When the counter reaches zero, the last +reference to the object has been deleted and the object is freed. + +An alternative strategy is called `automatic garbage collection'. +(Sometimes, reference counting is also referred to as a garbage +collection strategy, hence my use of “automatic” to distinguish the +two.) The big advantage of automatic garbage collection is that the +user doesn’t need to call ‘free()’ explicitly. (Another claimed +advantage is an improvement in speed or memory usage — this is no hard +fact however.) The disadvantage is that for C, there is no truly +portable automatic garbage collector, while reference counting can be +implemented portably (as long as the functions ‘malloc()’ and ‘free()’ +are available — which the C Standard guarantees). Maybe some day a +sufficiently portable automatic garbage collector will be available for +C. Until then, we’ll have to live with reference counts. + +While Python uses the traditional reference counting implementation, it +also offers a cycle detector that works to detect reference cycles. +This allows applications to not worry about creating direct or indirect +circular references; these are the weakness of garbage collection +implemented using only reference counting. Reference cycles consist of +objects which contain (possibly indirect) references to themselves, so +that each object in the cycle has a reference count which is non-zero. +Typical reference counting implementations are not able to reclaim the +memory belonging to any objects in a reference cycle, or referenced from +the objects in the cycle, even though there are no further references to +the cycle itself. + +The cycle detector is able to detect garbage cycles and can reclaim +them. The *note gc: 86. module exposes a way to run the detector (the +*note collect(): 22e. function), as well as configuration interfaces and +the ability to disable the detector at runtime. The cycle detector is +considered an optional component; though it is included by default, it +can be disabled at build time using the ‘--without-cycle-gc’ option to +the ‘configure’ script on Unix platforms (including Mac OS X). If the +cycle detector is disabled in this way, the *note gc: 86. module will +not be available. + +* Menu: + +* Reference Counting in Python:: +* Ownership Rules:: +* Thin Ice:: +* NULL Pointers:: + + +File: python.info, Node: Reference Counting in Python, Next: Ownership Rules, Up: Reference Counts + +6.2.1.11 Reference Counting in Python +..................................... + +There are two macros, ‘Py_INCREF(x)’ and ‘Py_DECREF(x)’, which handle +the incrementing and decrementing of the reference count. *note +Py_DECREF(): 266. also frees the object when the count reaches zero. +For flexibility, it doesn’t call ‘free()’ directly — rather, it makes a +call through a function pointer in the object’s `type object'. For this +purpose (and others), every object also contains a pointer to its type +object. + +The big question now remains: when to use ‘Py_INCREF(x)’ and +‘Py_DECREF(x)’? Let’s first introduce some terms. Nobody “owns” an +object; however, you can `own a reference' to an object. An object’s +reference count is now defined as the number of owned references to it. +The owner of a reference is responsible for calling *note Py_DECREF(): +266. when the reference is no longer needed. Ownership of a reference +can be transferred. There are three ways to dispose of an owned +reference: pass it on, store it, or call *note Py_DECREF(): 266. +Forgetting to dispose of an owned reference creates a memory leak. + +It is also possible to `borrow' (1) a reference to an object. The +borrower of a reference should not call *note Py_DECREF(): 266. The +borrower must not hold on to the object longer than the owner from which +it was borrowed. Using a borrowed reference after the owner has +disposed of it risks using freed memory and should be avoided completely +(2). + +The advantage of borrowing over owning a reference is that you don’t +need to take care of disposing of the reference on all possible paths +through the code — in other words, with a borrowed reference you don’t +run the risk of leaking when a premature exit is taken. The +disadvantage of borrowing over owning is that there are some subtle +situations where in seemingly correct code a borrowed reference can be +used after the owner from which it was borrowed has in fact disposed of +it. + +A borrowed reference can be changed into an owned reference by calling +*note Py_INCREF(): 265. This does not affect the status of the owner +from which the reference was borrowed — it creates a new owned +reference, and gives full owner responsibilities (the new owner must +dispose of the reference properly, as well as the previous owner). + + ---------- Footnotes ---------- + + (1) (2) The metaphor of “borrowing” a reference is not completely +correct: the owner still has a copy of the reference. + + (2) (3) Checking that the reference count is at least 1 `does not +work' — the reference count itself could be in freed memory and may thus +be reused for another object! + + +File: python.info, Node: Ownership Rules, Next: Thin Ice, Prev: Reference Counting in Python, Up: Reference Counts + +6.2.1.12 Ownership Rules +........................ + +Whenever an object reference is passed into or out of a function, it is +part of the function’s interface specification whether ownership is +transferred with the reference or not. + +Most functions that return a reference to an object pass on ownership +with the reference. In particular, all functions whose function it is +to create a new object, such as *note PyLong_FromLong(): 3841. and *note +Py_BuildValue(): 2ce, pass ownership to the receiver. Even if the +object is not actually new, you still receive ownership of a new +reference to that object. For instance, *note PyLong_FromLong(): 3841. +maintains a cache of popular values and can return a reference to a +cached item. + +Many functions that extract objects from other objects also transfer +ownership with the reference, for instance *note +PyObject_GetAttrString(): 385b. The picture is less clear, here, +however, since a few common routines are exceptions: *note +PyTuple_GetItem(): 385c, *note PyList_GetItem(): 385d, *note +PyDict_GetItem(): 385e, and *note PyDict_GetItemString(): 385f. all +return references that you borrow from the tuple, list or dictionary. + +The function *note PyImport_AddModule(): 3860. also returns a borrowed +reference, even though it may actually create the object it returns: +this is possible because an owned reference to the object is stored in +‘sys.modules’. + +When you pass an object reference into another function, in general, the +function borrows the reference from you — if it needs to store it, it +will use *note Py_INCREF(): 265. to become an independent owner. There +are exactly two important exceptions to this rule: *note +PyTuple_SetItem(): 3861. and *note PyList_SetItem(): 3862. These +functions take over ownership of the item passed to them — even if they +fail! (Note that *note PyDict_SetItem(): 3863. and friends don’t take +over ownership — they are “normal.”) + +When a C function is called from Python, it borrows references to its +arguments from the caller. The caller owns a reference to the object, +so the borrowed reference’s lifetime is guaranteed until the function +returns. Only when such a borrowed reference must be stored or passed +on, it must be turned into an owned reference by calling *note +Py_INCREF(): 265. + +The object reference returned from a C function that is called from +Python must be an owned reference — ownership is transferred from the +function to its caller. + + +File: python.info, Node: Thin Ice, Next: NULL Pointers, Prev: Ownership Rules, Up: Reference Counts + +6.2.1.13 Thin Ice +................. + +There are a few situations where seemingly harmless use of a borrowed +reference can lead to problems. These all have to do with implicit +invocations of the interpreter, which can cause the owner of a reference +to dispose of it. + +The first and most important case to know about is using *note +Py_DECREF(): 266. on an unrelated object while borrowing a reference to +a list item. For instance: + + void + bug(PyObject *list) + { + PyObject *item = PyList_GetItem(list, 0); + + PyList_SetItem(list, 1, PyLong_FromLong(0L)); + PyObject_Print(item, stdout, 0); /* BUG! */ + } + +This function first borrows a reference to ‘list[0]’, then replaces +‘list[1]’ with the value ‘0’, and finally prints the borrowed reference. +Looks harmless, right? But it’s not! + +Let’s follow the control flow into *note PyList_SetItem(): 3862. The +list owns references to all its items, so when item 1 is replaced, it +has to dispose of the original item 1. Now let’s suppose the original +item 1 was an instance of a user-defined class, and let’s further +suppose that the class defined a *note __del__(): 91f. method. If this +class instance has a reference count of 1, disposing of it will call its +*note __del__(): 91f. method. + +Since it is written in Python, the *note __del__(): 91f. method can +execute arbitrary Python code. Could it perhaps do something to +invalidate the reference to ‘item’ in ‘bug()’? You bet! Assuming that +the list passed into ‘bug()’ is accessible to the *note __del__(): 91f. +method, it could execute a statement to the effect of ‘del list[0]’, and +assuming this was the last reference to that object, it would free the +memory associated with it, thereby invalidating ‘item’. + +The solution, once you know the source of the problem, is easy: +temporarily increment the reference count. The correct version of the +function reads: + + void + no_bug(PyObject *list) + { + PyObject *item = PyList_GetItem(list, 0); + + Py_INCREF(item); + PyList_SetItem(list, 1, PyLong_FromLong(0L)); + PyObject_Print(item, stdout, 0); + Py_DECREF(item); + } + +This is a true story. An older version of Python contained variants of +this bug and someone spent a considerable amount of time in a C debugger +to figure out why his *note __del__(): 91f. methods would fail… + +The second case of problems with a borrowed reference is a variant +involving threads. Normally, multiple threads in the Python interpreter +can’t get in each other’s way, because there is a global lock protecting +Python’s entire object space. However, it is possible to temporarily +release this lock using the macro *note Py_BEGIN_ALLOW_THREADS: 3866, +and to re-acquire it using *note Py_END_ALLOW_THREADS: 2b5. This is +common around blocking I/O calls, to let other threads use the processor +while waiting for the I/O to complete. Obviously, the following +function has the same problem as the previous one: + + void + bug(PyObject *list) + { + PyObject *item = PyList_GetItem(list, 0); + Py_BEGIN_ALLOW_THREADS + ...some blocking I/O call... + Py_END_ALLOW_THREADS + PyObject_Print(item, stdout, 0); /* BUG! */ + } + + +File: python.info, Node: NULL Pointers, Prev: Thin Ice, Up: Reference Counts + +6.2.1.14 NULL Pointers +...................... + +In general, functions that take object references as arguments do not +expect you to pass them ‘NULL’ pointers, and will dump core (or cause +later core dumps) if you do so. Functions that return object references +generally return ‘NULL’ only to indicate that an exception occurred. +The reason for not testing for ‘NULL’ arguments is that functions often +pass the objects they receive on to other function — if each function +were to test for ‘NULL’, there would be a lot of redundant tests and the +code would run more slowly. + +It is better to test for ‘NULL’ only at the “source:” when a pointer +that may be ‘NULL’ is received, for example, from ‘malloc()’ or from a +function that may raise an exception. + +The macros *note Py_INCREF(): 265. and *note Py_DECREF(): 266. do not +check for ‘NULL’ pointers — however, their variants *note Py_XINCREF(): +267. and *note Py_XDECREF(): 268. do. + +The macros for checking for a particular object type (‘Pytype_Check()’) +don’t check for ‘NULL’ pointers — again, there is much code that calls +several of these in a row to test an object against various different +expected types, and this would generate redundant tests. There are no +variants with ‘NULL’ checking. + +The C function calling mechanism guarantees that the argument list +passed to C functions (‘args’ in the examples) is never ‘NULL’ — in fact +it guarantees that it is always a tuple (1). + +It is a severe error to ever let a ‘NULL’ pointer “escape” to the Python +user. + + ---------- Footnotes ---------- + + (1) (4) These guarantees don’t hold when you use the “old” style +calling convention — this is still found in much existing code. + + +File: python.info, Node: Writing Extensions in C++, Next: Providing a C API for an Extension Module, Prev: Reference Counts, Up: Extending Python with C or C++ + +6.2.1.15 Writing Extensions in C++ +.................................. + +It is possible to write extension modules in C++. Some restrictions +apply. If the main program (the Python interpreter) is compiled and +linked by the C compiler, global or static objects with constructors +cannot be used. This is not a problem if the main program is linked by +the C++ compiler. Functions that will be called by the Python +interpreter (in particular, module initialization functions) have to be +declared using ‘extern "C"’. It is unnecessary to enclose the Python +header files in ‘extern "C" {...}’ — they use this form already if the +symbol ‘__cplusplus’ is defined (all recent C++ compilers define this +symbol). + + +File: python.info, Node: Providing a C API for an Extension Module, Prev: Writing Extensions in C++, Up: Extending Python with C or C++ + +6.2.1.16 Providing a C API for an Extension Module +.................................................. + +Many extension modules just provide new functions and types to be used +from Python, but sometimes the code in an extension module can be useful +for other extension modules. For example, an extension module could +implement a type “collection” which works like lists without order. +Just like the standard Python list type has a C API which permits +extension modules to create and manipulate lists, this new collection +type should have a set of C functions for direct manipulation from other +extension modules. + +At first sight this seems easy: just write the functions (without +declaring them ‘static’, of course), provide an appropriate header file, +and document the C API. And in fact this would work if all extension +modules were always linked statically with the Python interpreter. When +modules are used as shared libraries, however, the symbols defined in +one module may not be visible to another module. The details of +visibility depend on the operating system; some systems use one global +namespace for the Python interpreter and all extension modules (Windows, +for example), whereas others require an explicit list of imported +symbols at module link time (AIX is one example), or offer a choice of +different strategies (most Unices). And even if symbols are globally +visible, the module whose functions one wishes to call might not have +been loaded yet! + +Portability therefore requires not to make any assumptions about symbol +visibility. This means that all symbols in extension modules should be +declared ‘static’, except for the module’s initialization function, in +order to avoid name clashes with other extension modules (as discussed +in section *note The Module’s Method Table and Initialization Function: +3845.). And it means that symbols that `should' be accessible from +other extension modules must be exported in a different way. + +Python provides a special mechanism to pass C-level information +(pointers) from one extension module to another one: Capsules. A +Capsule is a Python data type which stores a pointer (‘void *’). +Capsules can only be created and accessed via their C API, but they can +be passed around like any other Python object. In particular, they can +be assigned to a name in an extension module’s namespace. Other +extension modules can then import this module, retrieve the value of +this name, and then retrieve the pointer from the Capsule. + +There are many ways in which Capsules can be used to export the C API of +an extension module. Each function could get its own Capsule, or all C +API pointers could be stored in an array whose address is published in a +Capsule. And the various tasks of storing and retrieving the pointers +can be distributed in different ways between the module providing the +code and the client modules. + +Whichever method you choose, it’s important to name your Capsules +properly. The function *note PyCapsule_New(): 386c. takes a name +parameter (‘const char *’); you’re permitted to pass in a ‘NULL’ name, +but we strongly encourage you to specify a name. Properly named +Capsules provide a degree of runtime type-safety; there is no feasible +way to tell one unnamed Capsule from another. + +In particular, Capsules used to expose C APIs should be given a name +following this convention: + + modulename.attributename + +The convenience function *note PyCapsule_Import(): 386d. makes it easy +to load a C API provided via a Capsule, but only if the Capsule’s name +matches this convention. This behavior gives C API users a high degree +of certainty that the Capsule they load contains the correct C API. + +The following example demonstrates an approach that puts most of the +burden on the writer of the exporting module, which is appropriate for +commonly used library modules. It stores all C API pointers (just one +in the example!) in an array of ‘void’ pointers which becomes the value +of a Capsule. The header file corresponding to the module provides a +macro that takes care of importing the module and retrieving its C API +pointers; client modules only have to call this macro before accessing +the C API. + +The exporting module is a modification of the ‘spam’ module from section +*note A Simple Example: 3838. The function ‘spam.system()’ does not +call the C library function ‘system()’ directly, but a function +‘PySpam_System()’, which would of course do something more complicated +in reality (such as adding “spam” to every command). This function +‘PySpam_System()’ is also exported to other extension modules. + +The function ‘PySpam_System()’ is a plain C function, declared ‘static’ +like everything else: + + static int + PySpam_System(const char *command) + { + return system(command); + } + +The function ‘spam_system()’ is modified in a trivial way: + + static PyObject * + spam_system(PyObject *self, PyObject *args) + { + const char *command; + int sts; + + if (!PyArg_ParseTuple(args, "s", &command)) + return NULL; + sts = PySpam_System(command); + return PyLong_FromLong(sts); + } + +In the beginning of the module, right after the line + + #include <Python.h> + +two more lines must be added: + + #define SPAM_MODULE + #include "spammodule.h" + +The ‘#define’ is used to tell the header file that it is being included +in the exporting module, not a client module. Finally, the module’s +initialization function must take care of initializing the C API pointer +array: + + PyMODINIT_FUNC + PyInit_spam(void) + { + PyObject *m; + static void *PySpam_API[PySpam_API_pointers]; + PyObject *c_api_object; + + m = PyModule_Create(&spammodule); + if (m == NULL) + return NULL; + + /* Initialize the C API pointer array */ + PySpam_API[PySpam_System_NUM] = (void *)PySpam_System; + + /* Create a Capsule containing the API pointer array's address */ + c_api_object = PyCapsule_New((void *)PySpam_API, "spam._C_API", NULL); + + if (PyModule_AddObject(m, "_C_API", c_api_object) < 0) { + Py_XDECREF(c_api_object); + Py_DECREF(m); + return NULL; + } + + return m; + } + +Note that ‘PySpam_API’ is declared ‘static’; otherwise the pointer array +would disappear when ‘PyInit_spam()’ terminates! + +The bulk of the work is in the header file ‘spammodule.h’, which looks +like this: + + #ifndef Py_SPAMMODULE_H + #define Py_SPAMMODULE_H + #ifdef __cplusplus + extern "C" { + #endif + + /* Header file for spammodule */ + + /* C API functions */ + #define PySpam_System_NUM 0 + #define PySpam_System_RETURN int + #define PySpam_System_PROTO (const char *command) + + /* Total number of C API pointers */ + #define PySpam_API_pointers 1 + + + #ifdef SPAM_MODULE + /* This section is used when compiling spammodule.c */ + + static PySpam_System_RETURN PySpam_System PySpam_System_PROTO; + + #else + /* This section is used in modules that use spammodule's API */ + + static void **PySpam_API; + + #define PySpam_System \ + (*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM]) + + /* Return -1 on error, 0 on success. + * PyCapsule_Import will set an exception if there's an error. + */ + static int + import_spam(void) + { + PySpam_API = (void **)PyCapsule_Import("spam._C_API", 0); + return (PySpam_API != NULL) ? 0 : -1; + } + + #endif + + #ifdef __cplusplus + } + #endif + + #endif /* !defined(Py_SPAMMODULE_H) */ + +All that a client module must do in order to have access to the function +‘PySpam_System()’ is to call the function (or rather macro) +‘import_spam()’ in its initialization function: + + PyMODINIT_FUNC + PyInit_client(void) + { + PyObject *m; + + m = PyModule_Create(&clientmodule); + if (m == NULL) + return NULL; + if (import_spam() < 0) + return NULL; + /* additional initialization can happen here */ + return m; + } + +The main disadvantage of this approach is that the file ‘spammodule.h’ +is rather complicated. However, the basic structure is the same for +each function that is exported, so it has to be learned only once. + +Finally it should be mentioned that Capsules offer additional +functionality, which is especially useful for memory allocation and +deallocation of the pointer stored in a Capsule. The details are +described in the Python/C API Reference Manual in the section *note +Capsules: 386e. and in the implementation of Capsules (files +‘Include/pycapsule.h’ and ‘Objects/pycapsule.c’ in the Python source +code distribution). + + +File: python.info, Node: Defining Extension Types Tutorial, Next: Defining Extension Types Assorted Topics, Prev: Extending Python with C or C++, Up: Creating extensions without third party tools + +6.2.2 Defining Extension Types: Tutorial +---------------------------------------- + +Python allows the writer of a C extension module to define new types +that can be manipulated from Python code, much like the built-in *note +str: 330. and *note list: 262. types. The code for all extension types +follows a pattern, but there are some details that you need to +understand before you can get started. This document is a gentle +introduction to the topic. + +* Menu: + +* The Basics:: +* Adding data and methods to the Basic example:: +* Providing finer control over data attributes:: +* Supporting cyclic garbage collection:: +* Subclassing other types:: + + +File: python.info, Node: The Basics, Next: Adding data and methods to the Basic example, Up: Defining Extension Types Tutorial + +6.2.2.1 The Basics +.................. + +The *note CPython: 10d7. runtime sees all Python objects as variables of +type *note PyObject*: 4ba, which serves as a “base type” for all Python +objects. The *note PyObject: 4ba. structure itself only contains the +object’s *note reference count: 3874. and a pointer to the object’s +“type object”. This is where the action is; the type object determines +which (C) functions get called by the interpreter when, for instance, an +attribute gets looked up on an object, a method called, or it is +multiplied by another object. These C functions are called “type +methods”. + +So, if you want to define a new extension type, you need to create a new +type object. + +This sort of thing can only be explained by example, so here’s a +minimal, but complete, module that defines a new type named ‘Custom’ +inside a C extension module ‘custom’: + + Note: What we’re showing here is the traditional way of defining + `static' extension types. It should be adequate for most uses. + The C API also allows defining heap-allocated extension types using + the *note PyType_FromSpec(): 2d1. function, which isn’t covered in + this tutorial. + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + + typedef struct { + PyObject_HEAD + /* Type-specific fields go here. */ + } CustomObject; + + static PyTypeObject CustomType = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "custom.Custom", + .tp_doc = "Custom objects", + .tp_basicsize = sizeof(CustomObject), + .tp_itemsize = 0, + .tp_flags = Py_TPFLAGS_DEFAULT, + .tp_new = PyType_GenericNew, + }; + + static PyModuleDef custommodule = { + PyModuleDef_HEAD_INIT, + .m_name = "custom", + .m_doc = "Example module that creates an extension type.", + .m_size = -1, + }; + + PyMODINIT_FUNC + PyInit_custom(void) + { + PyObject *m; + if (PyType_Ready(&CustomType) < 0) + return NULL; + + m = PyModule_Create(&custommodule); + if (m == NULL) + return NULL; + + Py_INCREF(&CustomType); + if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { + Py_DECREF(&CustomType); + Py_DECREF(m); + return NULL; + } + + return m; + } + +Now that’s quite a bit to take in at once, but hopefully bits will seem +familiar from the previous chapter. This file defines three things: + + 1. What a ‘Custom’ `object' contains: this is the ‘CustomObject’ + struct, which is allocated once for each ‘Custom’ instance. + + 2. How the ‘Custom’ `type' behaves: this is the ‘CustomType’ struct, + which defines a set of flags and function pointers that the + interpreter inspects when specific operations are requested. + + 3. How to initialize the ‘custom’ module: this is the ‘PyInit_custom’ + function and the associated ‘custommodule’ struct. + +The first bit is: + + typedef struct { + PyObject_HEAD + } CustomObject; + +This is what a Custom object will contain. ‘PyObject_HEAD’ is mandatory +at the start of each object struct and defines a field called ‘ob_base’ +of type *note PyObject: 4ba, containing a pointer to a type object and a +reference count (these can be accessed using the macros *note Py_REFCNT: +3875. and *note Py_TYPE: 3876. respectively). The reason for the macro +is to abstract away the layout and to enable additional fields in debug +builds. + + Note: There is no semicolon above after the *note PyObject_HEAD: + c72. macro. Be wary of adding one by accident: some compilers will + complain. + +Of course, objects generally store additional data besides the standard +‘PyObject_HEAD’ boilerplate; for example, here is the definition for +standard Python floats: + + typedef struct { + PyObject_HEAD + double ob_fval; + } PyFloatObject; + +The second bit is the definition of the type object. + + static PyTypeObject CustomType = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "custom.Custom", + .tp_doc = "Custom objects", + .tp_basicsize = sizeof(CustomObject), + .tp_itemsize = 0, + .tp_flags = Py_TPFLAGS_DEFAULT, + .tp_new = PyType_GenericNew, + }; + + Note: We recommend using C99-style designated initializers as + above, to avoid listing all the *note PyTypeObject: 2d6. fields + that you don’t care about and also to avoid caring about the + fields’ declaration order. + +The actual definition of *note PyTypeObject: 2d6. in ‘object.h’ has many +more *note fields: 3877. than the definition above. The remaining +fields will be filled with zeros by the C compiler, and it’s common +practice to not specify them explicitly unless you need them. + +We’re going to pick it apart, one field at a time: + + PyVarObject_HEAD_INIT(NULL, 0) + +This line is mandatory boilerplate to initialize the ‘ob_base’ field +mentioned above. + + .tp_name = "custom.Custom", + +The name of our type. This will appear in the default textual +representation of our objects and in some error messages, for example: + + >>> "" + custom.Custom() + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + TypeError: can only concatenate str (not "custom.Custom") to str + +Note that the name is a dotted name that includes both the module name +and the name of the type within the module. The module in this case is +‘custom’ and the type is ‘Custom’, so we set the type name to +‘custom.Custom’. Using the real dotted import path is important to make +your type compatible with the *note pydoc: d9. and *note pickle: ca. +modules. + + .tp_basicsize = sizeof(CustomObject), + .tp_itemsize = 0, + +This is so that Python knows how much memory to allocate when creating +new ‘Custom’ instances. *note tp_itemsize: 3878. is only used for +variable-sized objects and should otherwise be zero. + + Note: If you want your type to be subclassable from Python, and + your type has the same *note tp_basicsize: 3879. as its base type, + you may have problems with multiple inheritance. A Python subclass + of your type will have to list your type first in its *note + __bases__: e09, or else it will not be able to call your type’s + *note __new__(): 889. method without getting an error. You can + avoid this problem by ensuring that your type has a larger value + for *note tp_basicsize: 3879. than its base type does. Most of the + time, this will be true anyway, because either your base type will + be *note object: 2b0, or else you will be adding data members to + your base type, and therefore increasing its size. + +We set the class flags to *note Py_TPFLAGS_DEFAULT: 387a. + + .tp_flags = Py_TPFLAGS_DEFAULT, + +All types should include this constant in their flags. It enables all +of the members defined until at least Python 3.3. If you need further +members, you will need to OR the corresponding flags. + +We provide a doc string for the type in *note tp_doc: 387b. + + .tp_doc = "Custom objects", + +To enable object creation, we have to provide a *note tp_new: 387c. +handler. This is the equivalent of the Python method *note __new__(): +889, but has to be specified explicitly. In this case, we can just use +the default implementation provided by the API function *note +PyType_GenericNew(): 387d. + + .tp_new = PyType_GenericNew, + +Everything else in the file should be familiar, except for some code in +‘PyInit_custom()’: + + if (PyType_Ready(&CustomType) < 0) + return; + +This initializes the ‘Custom’ type, filling in a number of members to +the appropriate default values, including ‘ob_type’ that we initially +set to ‘NULL’. + + Py_INCREF(&CustomType); + if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { + Py_DECREF(&CustomType); + Py_DECREF(m); + return NULL; + } + +This adds the type to the module dictionary. This allows us to create +‘Custom’ instances by calling the ‘Custom’ class: + + >>> import custom + >>> mycustom = custom.Custom() + +That’s it! All that remains is to build it; put the above code in a +file called ‘custom.c’ and: + + from distutils.core import setup, Extension + setup(name="custom", version="1.0", + ext_modules=[Extension("custom", ["custom.c"])]) + +in a file called ‘setup.py’; then typing + + $ python setup.py build + +at a shell should produce a file ‘custom.so’ in a subdirectory; move to +that directory and fire up Python — you should be able to ‘import +custom’ and play around with Custom objects. + +That wasn’t so hard, was it? + +Of course, the current Custom type is pretty uninteresting. It has no +data and doesn’t do anything. It can’t even be subclassed. + + Note: While this documentation showcases the standard *note + distutils: 39. module for building C extensions, it is recommended + in real-world use cases to use the newer and better-maintained + ‘setuptools’ library. Documentation on how to do this is out of + scope for this document and can be found in the Python Packaging + User’s Guide(1). + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/tutorials/distributing-packages/ + + +File: python.info, Node: Adding data and methods to the Basic example, Next: Providing finer control over data attributes, Prev: The Basics, Up: Defining Extension Types Tutorial + +6.2.2.2 Adding data and methods to the Basic example +.................................................... + +Let’s extend the basic example to add some data and methods. Let’s also +make the type usable as a base class. We’ll create a new module, +‘custom2’ that adds these capabilities: + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + #include "structmember.h" + + typedef struct { + PyObject_HEAD + PyObject *first; /* first name */ + PyObject *last; /* last name */ + int number; + } CustomObject; + + static void + Custom_dealloc(CustomObject *self) + { + Py_XDECREF(self->first); + Py_XDECREF(self->last); + Py_TYPE(self)->tp_free((PyObject *) self); + } + + static PyObject * + Custom_new(PyTypeObject *type, PyObject *args, PyObject *kwds) + { + CustomObject *self; + self = (CustomObject *) type->tp_alloc(type, 0); + if (self != NULL) { + self->first = PyUnicode_FromString(""); + if (self->first == NULL) { + Py_DECREF(self); + return NULL; + } + self->last = PyUnicode_FromString(""); + if (self->last == NULL) { + Py_DECREF(self); + return NULL; + } + self->number = 0; + } + return (PyObject *) self; + } + + static int + Custom_init(CustomObject *self, PyObject *args, PyObject *kwds) + { + static char *kwlist[] = {"first", "last", "number", NULL}; + PyObject *first = NULL, *last = NULL, *tmp; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OOi", kwlist, + &first, &last, + &self->number)) + return -1; + + if (first) { + tmp = self->first; + Py_INCREF(first); + self->first = first; + Py_XDECREF(tmp); + } + if (last) { + tmp = self->last; + Py_INCREF(last); + self->last = last; + Py_XDECREF(tmp); + } + return 0; + } + + static PyMemberDef Custom_members[] = { + {"first", T_OBJECT_EX, offsetof(CustomObject, first), 0, + "first name"}, + {"last", T_OBJECT_EX, offsetof(CustomObject, last), 0, + "last name"}, + {"number", T_INT, offsetof(CustomObject, number), 0, + "custom number"}, + {NULL} /* Sentinel */ + }; + + static PyObject * + Custom_name(CustomObject *self, PyObject *Py_UNUSED(ignored)) + { + if (self->first == NULL) { + PyErr_SetString(PyExc_AttributeError, "first"); + return NULL; + } + if (self->last == NULL) { + PyErr_SetString(PyExc_AttributeError, "last"); + return NULL; + } + return PyUnicode_FromFormat("%S %S", self->first, self->last); + } + + static PyMethodDef Custom_methods[] = { + {"name", (PyCFunction) Custom_name, METH_NOARGS, + "Return the name, combining the first and last name" + }, + {NULL} /* Sentinel */ + }; + + static PyTypeObject CustomType = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "custom2.Custom", + .tp_doc = "Custom objects", + .tp_basicsize = sizeof(CustomObject), + .tp_itemsize = 0, + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, + .tp_new = Custom_new, + .tp_init = (initproc) Custom_init, + .tp_dealloc = (destructor) Custom_dealloc, + .tp_members = Custom_members, + .tp_methods = Custom_methods, + }; + + static PyModuleDef custommodule = { + PyModuleDef_HEAD_INIT, + .m_name = "custom2", + .m_doc = "Example module that creates an extension type.", + .m_size = -1, + }; + + PyMODINIT_FUNC + PyInit_custom2(void) + { + PyObject *m; + if (PyType_Ready(&CustomType) < 0) + return NULL; + + m = PyModule_Create(&custommodule); + if (m == NULL) + return NULL; + + Py_INCREF(&CustomType); + if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { + Py_DECREF(&CustomType); + Py_DECREF(m); + return NULL; + } + + return m; + } + +This version of the module has a number of changes. + +We’ve added an extra include: + + #include <structmember.h> + +This include provides declarations that we use to handle attributes, as +described a bit later. + +The ‘Custom’ type now has three data attributes in its C struct, +`first', `last', and `number'. The `first' and `last' variables are +Python strings containing first and last names. The `number' attribute +is a C integer. + +The object structure is updated accordingly: + + typedef struct { + PyObject_HEAD + PyObject *first; /* first name */ + PyObject *last; /* last name */ + int number; + } CustomObject; + +Because we now have data to manage, we have to be more careful about +object allocation and deallocation. At a minimum, we need a +deallocation method: + + static void + Custom_dealloc(CustomObject *self) + { + Py_XDECREF(self->first); + Py_XDECREF(self->last); + Py_TYPE(self)->tp_free((PyObject *) self); + } + +which is assigned to the *note tp_dealloc: 387f. member: + + .tp_dealloc = (destructor) Custom_dealloc, + +This method first clears the reference counts of the two Python +attributes. *note Py_XDECREF(): 268. correctly handles the case where +its argument is ‘NULL’ (which might happen here if ‘tp_new’ failed +midway). It then calls the *note tp_free: 3880. member of the object’s +type (computed by ‘Py_TYPE(self)’) to free the object’s memory. Note +that the object’s type might not be ‘CustomType’, because the object may +be an instance of a subclass. + + Note: The explicit cast to ‘destructor’ above is needed because we + defined ‘Custom_dealloc’ to take a ‘CustomObject *’ argument, but + the ‘tp_dealloc’ function pointer expects to receive a ‘PyObject *’ + argument. Otherwise, the compiler will emit a warning. This is + object-oriented polymorphism, in C! + +We want to make sure that the first and last names are initialized to +empty strings, so we provide a ‘tp_new’ implementation: + + static PyObject * + Custom_new(PyTypeObject *type, PyObject *args, PyObject *kwds) + { + CustomObject *self; + self = (CustomObject *) type->tp_alloc(type, 0); + if (self != NULL) { + self->first = PyUnicode_FromString(""); + if (self->first == NULL) { + Py_DECREF(self); + return NULL; + } + self->last = PyUnicode_FromString(""); + if (self->last == NULL) { + Py_DECREF(self); + return NULL; + } + self->number = 0; + } + return (PyObject *) self; + } + +and install it in the *note tp_new: 387c. member: + + .tp_new = Custom_new, + +The ‘tp_new’ handler is responsible for creating (as opposed to +initializing) objects of the type. It is exposed in Python as the *note +__new__(): 889. method. It is not required to define a ‘tp_new’ member, +and indeed many extension types will simply reuse *note +PyType_GenericNew(): 387d. as done in the first version of the ‘Custom’ +type above. In this case, we use the ‘tp_new’ handler to initialize the +‘first’ and ‘last’ attributes to non-‘NULL’ default values. + +‘tp_new’ is passed the type being instantiated (not necessarily +‘CustomType’, if a subclass is instantiated) and any arguments passed +when the type was called, and is expected to return the instance +created. ‘tp_new’ handlers always accept positional and keyword +arguments, but they often ignore the arguments, leaving the argument +handling to initializer (a.k.a. ‘tp_init’ in C or ‘__init__’ in Python) +methods. + + Note: ‘tp_new’ shouldn’t call ‘tp_init’ explicitly, as the + interpreter will do it itself. + +The ‘tp_new’ implementation calls the *note tp_alloc: 3881. slot to +allocate memory: + + self = (CustomObject *) type->tp_alloc(type, 0); + +Since memory allocation may fail, we must check the *note tp_alloc: +3881. result against ‘NULL’ before proceeding. + + Note: We didn’t fill the *note tp_alloc: 3881. slot ourselves. + Rather *note PyType_Ready(): 3882. fills it for us by inheriting it + from our base class, which is *note object: 2b0. by default. Most + types use the default allocation strategy. + + Note: If you are creating a co-operative *note tp_new: 387c. (one + that calls a base type’s *note tp_new: 387c. or *note __new__(): + 889.), you must `not' try to determine what method to call using + method resolution order at runtime. Always statically determine + what type you are going to call, and call its *note tp_new: 387c. + directly, or via ‘type->tp_base->tp_new’. If you do not do this, + Python subclasses of your type that also inherit from other + Python-defined classes may not work correctly. (Specifically, you + may not be able to create instances of such subclasses without + getting a *note TypeError: 192.) + +We also define an initialization function which accepts arguments to +provide initial values for our instance: + + static int + Custom_init(CustomObject *self, PyObject *args, PyObject *kwds) + { + static char *kwlist[] = {"first", "last", "number", NULL}; + PyObject *first = NULL, *last = NULL, *tmp; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OOi", kwlist, + &first, &last, + &self->number)) + return -1; + + if (first) { + tmp = self->first; + Py_INCREF(first); + self->first = first; + Py_XDECREF(tmp); + } + if (last) { + tmp = self->last; + Py_INCREF(last); + self->last = last; + Py_XDECREF(tmp); + } + return 0; + } + +by filling the *note tp_init: 3883. slot. + + .tp_init = (initproc) Custom_init, + +The *note tp_init: 3883. slot is exposed in Python as the *note +__init__(): d5e. method. It is used to initialize an object after it’s +created. Initializers always accept positional and keyword arguments, +and they should return either ‘0’ on success or ‘-1’ on error. + +Unlike the ‘tp_new’ handler, there is no guarantee that ‘tp_init’ is +called at all (for example, the *note pickle: ca. module by default +doesn’t call *note __init__(): d5e. on unpickled instances). It can +also be called multiple times. Anyone can call the *note __init__(): +d5e. method on our objects. For this reason, we have to be extra +careful when assigning the new attribute values. We might be tempted, +for example to assign the ‘first’ member like this: + + if (first) { + Py_XDECREF(self->first); + Py_INCREF(first); + self->first = first; + } + +But this would be risky. Our type doesn’t restrict the type of the +‘first’ member, so it could be any kind of object. It could have a +destructor that causes code to be executed that tries to access the +‘first’ member; or that destructor could release the *note Global +interpreter Lock: bea. and let arbitrary code run in other threads that +accesses and modifies our object. + +To be paranoid and protect ourselves against this possibility, we almost +always reassign members before decrementing their reference counts. +When don’t we have to do this? + + * when we absolutely know that the reference count is greater than 1; + + * when we know that deallocation of the object (1) will neither + release the *note GIL: bea. nor cause any calls back into our + type’s code; + + * when decrementing a reference count in a *note tp_dealloc: 387f. + handler on a type which doesn’t support cyclic garbage collection + (2). + +We want to expose our instance variables as attributes. There are a +number of ways to do that. The simplest way is to define member +definitions: + + static PyMemberDef Custom_members[] = { + {"first", T_OBJECT_EX, offsetof(CustomObject, first), 0, + "first name"}, + {"last", T_OBJECT_EX, offsetof(CustomObject, last), 0, + "last name"}, + {"number", T_INT, offsetof(CustomObject, number), 0, + "custom number"}, + {NULL} /* Sentinel */ + }; + +and put the definitions in the *note tp_members: 3884. slot: + + .tp_members = Custom_members, + +Each member definition has a member name, type, offset, access flags and +documentation string. See the *note Generic Attribute Management: 3885. +section below for details. + +A disadvantage of this approach is that it doesn’t provide a way to +restrict the types of objects that can be assigned to the Python +attributes. We expect the first and last names to be strings, but any +Python objects can be assigned. Further, the attributes can be deleted, +setting the C pointers to ‘NULL’. Even though we can make sure the +members are initialized to non-‘NULL’ values, the members can be set to +‘NULL’ if the attributes are deleted. + +We define a single method, ‘Custom.name()’, that outputs the objects +name as the concatenation of the first and last names. + + static PyObject * + Custom_name(CustomObject *self, PyObject *Py_UNUSED(ignored)) + { + if (self->first == NULL) { + PyErr_SetString(PyExc_AttributeError, "first"); + return NULL; + } + if (self->last == NULL) { + PyErr_SetString(PyExc_AttributeError, "last"); + return NULL; + } + return PyUnicode_FromFormat("%S %S", self->first, self->last); + } + +The method is implemented as a C function that takes a ‘Custom’ (or +‘Custom’ subclass) instance as the first argument. Methods always take +an instance as the first argument. Methods often take positional and +keyword arguments as well, but in this case we don’t take any and don’t +need to accept a positional argument tuple or keyword argument +dictionary. This method is equivalent to the Python method: + + def name(self): + return "%s %s" % (self.first, self.last) + +Note that we have to check for the possibility that our ‘first’ and +‘last’ members are ‘NULL’. This is because they can be deleted, in +which case they are set to ‘NULL’. It would be better to prevent +deletion of these attributes and to restrict the attribute values to be +strings. We’ll see how to do that in the next section. + +Now that we’ve defined the method, we need to create an array of method +definitions: + + static PyMethodDef Custom_methods[] = { + {"name", (PyCFunction) Custom_name, METH_NOARGS, + "Return the name, combining the first and last name" + }, + {NULL} /* Sentinel */ + }; + +(note that we used the *note METH_NOARGS: e18. flag to indicate that the +method is expecting no arguments other than `self') + +and assign it to the *note tp_methods: 3886. slot: + + .tp_methods = Custom_methods, + +Finally, we’ll make our type usable as a base class for subclassing. +We’ve written our methods carefully so far so that they don’t make any +assumptions about the type of the object being created or used, so all +we need to do is to add the *note Py_TPFLAGS_BASETYPE: 3887. to our +class flag definition: + + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, + +We rename ‘PyInit_custom()’ to ‘PyInit_custom2()’, update the module +name in the *note PyModuleDef: 3888. struct, and update the full class +name in the *note PyTypeObject: 2d6. struct. + +Finally, we update our ‘setup.py’ file to build the new module: + + from distutils.core import setup, Extension + setup(name="custom", version="1.0", + ext_modules=[ + Extension("custom", ["custom.c"]), + Extension("custom2", ["custom2.c"]), + ]) + + ---------- Footnotes ---------- + + (1) (1) This is true when we know that the object is a basic type, +like a string or a float. + + (2) (2) We relied on this in the *note tp_dealloc: 387f. handler in +this example, because our type doesn’t support garbage collection. + + +File: python.info, Node: Providing finer control over data attributes, Next: Supporting cyclic garbage collection, Prev: Adding data and methods to the Basic example, Up: Defining Extension Types Tutorial + +6.2.2.3 Providing finer control over data attributes +.................................................... + +In this section, we’ll provide finer control over how the ‘first’ and +‘last’ attributes are set in the ‘Custom’ example. In the previous +version of our module, the instance variables ‘first’ and ‘last’ could +be set to non-string values or even deleted. We want to make sure that +these attributes always contain strings. + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + #include "structmember.h" + + typedef struct { + PyObject_HEAD + PyObject *first; /* first name */ + PyObject *last; /* last name */ + int number; + } CustomObject; + + static void + Custom_dealloc(CustomObject *self) + { + Py_XDECREF(self->first); + Py_XDECREF(self->last); + Py_TYPE(self)->tp_free((PyObject *) self); + } + + static PyObject * + Custom_new(PyTypeObject *type, PyObject *args, PyObject *kwds) + { + CustomObject *self; + self = (CustomObject *) type->tp_alloc(type, 0); + if (self != NULL) { + self->first = PyUnicode_FromString(""); + if (self->first == NULL) { + Py_DECREF(self); + return NULL; + } + self->last = PyUnicode_FromString(""); + if (self->last == NULL) { + Py_DECREF(self); + return NULL; + } + self->number = 0; + } + return (PyObject *) self; + } + + static int + Custom_init(CustomObject *self, PyObject *args, PyObject *kwds) + { + static char *kwlist[] = {"first", "last", "number", NULL}; + PyObject *first = NULL, *last = NULL, *tmp; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|UUi", kwlist, + &first, &last, + &self->number)) + return -1; + + if (first) { + tmp = self->first; + Py_INCREF(first); + self->first = first; + Py_DECREF(tmp); + } + if (last) { + tmp = self->last; + Py_INCREF(last); + self->last = last; + Py_DECREF(tmp); + } + return 0; + } + + static PyMemberDef Custom_members[] = { + {"number", T_INT, offsetof(CustomObject, number), 0, + "custom number"}, + {NULL} /* Sentinel */ + }; + + static PyObject * + Custom_getfirst(CustomObject *self, void *closure) + { + Py_INCREF(self->first); + return self->first; + } + + static int + Custom_setfirst(CustomObject *self, PyObject *value, void *closure) + { + PyObject *tmp; + if (value == NULL) { + PyErr_SetString(PyExc_TypeError, "Cannot delete the first attribute"); + return -1; + } + if (!PyUnicode_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "The first attribute value must be a string"); + return -1; + } + tmp = self->first; + Py_INCREF(value); + self->first = value; + Py_DECREF(tmp); + return 0; + } + + static PyObject * + Custom_getlast(CustomObject *self, void *closure) + { + Py_INCREF(self->last); + return self->last; + } + + static int + Custom_setlast(CustomObject *self, PyObject *value, void *closure) + { + PyObject *tmp; + if (value == NULL) { + PyErr_SetString(PyExc_TypeError, "Cannot delete the last attribute"); + return -1; + } + if (!PyUnicode_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "The last attribute value must be a string"); + return -1; + } + tmp = self->last; + Py_INCREF(value); + self->last = value; + Py_DECREF(tmp); + return 0; + } + + static PyGetSetDef Custom_getsetters[] = { + {"first", (getter) Custom_getfirst, (setter) Custom_setfirst, + "first name", NULL}, + {"last", (getter) Custom_getlast, (setter) Custom_setlast, + "last name", NULL}, + {NULL} /* Sentinel */ + }; + + static PyObject * + Custom_name(CustomObject *self, PyObject *Py_UNUSED(ignored)) + { + return PyUnicode_FromFormat("%S %S", self->first, self->last); + } + + static PyMethodDef Custom_methods[] = { + {"name", (PyCFunction) Custom_name, METH_NOARGS, + "Return the name, combining the first and last name" + }, + {NULL} /* Sentinel */ + }; + + static PyTypeObject CustomType = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "custom3.Custom", + .tp_doc = "Custom objects", + .tp_basicsize = sizeof(CustomObject), + .tp_itemsize = 0, + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, + .tp_new = Custom_new, + .tp_init = (initproc) Custom_init, + .tp_dealloc = (destructor) Custom_dealloc, + .tp_members = Custom_members, + .tp_methods = Custom_methods, + .tp_getset = Custom_getsetters, + }; + + static PyModuleDef custommodule = { + PyModuleDef_HEAD_INIT, + .m_name = "custom3", + .m_doc = "Example module that creates an extension type.", + .m_size = -1, + }; + + PyMODINIT_FUNC + PyInit_custom3(void) + { + PyObject *m; + if (PyType_Ready(&CustomType) < 0) + return NULL; + + m = PyModule_Create(&custommodule); + if (m == NULL) + return NULL; + + Py_INCREF(&CustomType); + if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { + Py_DECREF(&CustomType); + Py_DECREF(m); + return NULL; + } + + return m; + } + +To provide greater control, over the ‘first’ and ‘last’ attributes, +we’ll use custom getter and setter functions. Here are the functions +for getting and setting the ‘first’ attribute: + + static PyObject * + Custom_getfirst(CustomObject *self, void *closure) + { + Py_INCREF(self->first); + return self->first; + } + + static int + Custom_setfirst(CustomObject *self, PyObject *value, void *closure) + { + PyObject *tmp; + if (value == NULL) { + PyErr_SetString(PyExc_TypeError, "Cannot delete the first attribute"); + return -1; + } + if (!PyUnicode_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "The first attribute value must be a string"); + return -1; + } + tmp = self->first; + Py_INCREF(value); + self->first = value; + Py_DECREF(tmp); + return 0; + } + +The getter function is passed a ‘Custom’ object and a “closure”, which +is a void pointer. In this case, the closure is ignored. (The closure +supports an advanced usage in which definition data is passed to the +getter and setter. This could, for example, be used to allow a single +set of getter and setter functions that decide the attribute to get or +set based on data in the closure.) + +The setter function is passed the ‘Custom’ object, the new value, and +the closure. The new value may be ‘NULL’, in which case the attribute +is being deleted. In our setter, we raise an error if the attribute is +deleted or if its new value is not a string. + +We create an array of *note PyGetSetDef: 427. structures: + + static PyGetSetDef Custom_getsetters[] = { + {"first", (getter) Custom_getfirst, (setter) Custom_setfirst, + "first name", NULL}, + {"last", (getter) Custom_getlast, (setter) Custom_setlast, + "last name", NULL}, + {NULL} /* Sentinel */ + }; + +and register it in the *note tp_getset: 388a. slot: + + .tp_getset = Custom_getsetters, + +The last item in a *note PyGetSetDef: 427. structure is the “closure” +mentioned above. In this case, we aren’t using a closure, so we just +pass ‘NULL’. + +We also remove the member definitions for these attributes: + + static PyMemberDef Custom_members[] = { + {"number", T_INT, offsetof(CustomObject, number), 0, + "custom number"}, + {NULL} /* Sentinel */ + }; + +We also need to update the *note tp_init: 3883. handler to only allow +strings (1) to be passed: + + static int + Custom_init(CustomObject *self, PyObject *args, PyObject *kwds) + { + static char *kwlist[] = {"first", "last", "number", NULL}; + PyObject *first = NULL, *last = NULL, *tmp; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|UUi", kwlist, + &first, &last, + &self->number)) + return -1; + + if (first) { + tmp = self->first; + Py_INCREF(first); + self->first = first; + Py_DECREF(tmp); + } + if (last) { + tmp = self->last; + Py_INCREF(last); + self->last = last; + Py_DECREF(tmp); + } + return 0; + } + +With these changes, we can assure that the ‘first’ and ‘last’ members +are never ‘NULL’ so we can remove checks for ‘NULL’ values in almost all +cases. This means that most of the *note Py_XDECREF(): 268. calls can +be converted to *note Py_DECREF(): 266. calls. The only place we can’t +change these calls is in the ‘tp_dealloc’ implementation, where there is +the possibility that the initialization of these members failed in +‘tp_new’. + +We also rename the module initialization function and module name in the +initialization function, as we did before, and we add an extra +definition to the ‘setup.py’ file. + + ---------- Footnotes ---------- + + (1) (3) We now know that the first and last members are strings, so +perhaps we could be less careful about decrementing their reference +counts, however, we accept instances of string subclasses. Even though +deallocating normal strings won’t call back into our objects, we can’t +guarantee that deallocating an instance of a string subclass won’t call +back into our objects. + + +File: python.info, Node: Supporting cyclic garbage collection, Next: Subclassing other types, Prev: Providing finer control over data attributes, Up: Defining Extension Types Tutorial + +6.2.2.4 Supporting cyclic garbage collection +............................................ + +Python has a *note cyclic garbage collector (GC): f9f. that can identify +unneeded objects even when their reference counts are not zero. This +can happen when objects are involved in cycles. For example, consider: + + >>> l = [] + >>> l.append(l) + >>> del l + +In this example, we create a list that contains itself. When we delete +it, it still has a reference from itself. Its reference count doesn’t +drop to zero. Fortunately, Python’s cyclic garbage collector will +eventually figure out that the list is garbage and free it. + +In the second version of the ‘Custom’ example, we allowed any kind of +object to be stored in the ‘first’ or ‘last’ attributes (1). Besides, +in the second and third versions, we allowed subclassing ‘Custom’, and +subclasses may add arbitrary attributes. For any of those two reasons, +‘Custom’ objects can participate in cycles: + + >>> import custom3 + >>> class Derived(custom3.Custom): pass + ... + >>> n = Derived() + >>> n.some_attribute = n + +To allow a ‘Custom’ instance participating in a reference cycle to be +properly detected and collected by the cyclic GC, our ‘Custom’ type +needs to fill two additional slots and to enable a flag that enables +these slots: + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + #include "structmember.h" + + typedef struct { + PyObject_HEAD + PyObject *first; /* first name */ + PyObject *last; /* last name */ + int number; + } CustomObject; + + static int + Custom_traverse(CustomObject *self, visitproc visit, void *arg) + { + Py_VISIT(self->first); + Py_VISIT(self->last); + return 0; + } + + static int + Custom_clear(CustomObject *self) + { + Py_CLEAR(self->first); + Py_CLEAR(self->last); + return 0; + } + + static void + Custom_dealloc(CustomObject *self) + { + PyObject_GC_UnTrack(self); + Custom_clear(self); + Py_TYPE(self)->tp_free((PyObject *) self); + } + + static PyObject * + Custom_new(PyTypeObject *type, PyObject *args, PyObject *kwds) + { + CustomObject *self; + self = (CustomObject *) type->tp_alloc(type, 0); + if (self != NULL) { + self->first = PyUnicode_FromString(""); + if (self->first == NULL) { + Py_DECREF(self); + return NULL; + } + self->last = PyUnicode_FromString(""); + if (self->last == NULL) { + Py_DECREF(self); + return NULL; + } + self->number = 0; + } + return (PyObject *) self; + } + + static int + Custom_init(CustomObject *self, PyObject *args, PyObject *kwds) + { + static char *kwlist[] = {"first", "last", "number", NULL}; + PyObject *first = NULL, *last = NULL, *tmp; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|UUi", kwlist, + &first, &last, + &self->number)) + return -1; + + if (first) { + tmp = self->first; + Py_INCREF(first); + self->first = first; + Py_DECREF(tmp); + } + if (last) { + tmp = self->last; + Py_INCREF(last); + self->last = last; + Py_DECREF(tmp); + } + return 0; + } + + static PyMemberDef Custom_members[] = { + {"number", T_INT, offsetof(CustomObject, number), 0, + "custom number"}, + {NULL} /* Sentinel */ + }; + + static PyObject * + Custom_getfirst(CustomObject *self, void *closure) + { + Py_INCREF(self->first); + return self->first; + } + + static int + Custom_setfirst(CustomObject *self, PyObject *value, void *closure) + { + if (value == NULL) { + PyErr_SetString(PyExc_TypeError, "Cannot delete the first attribute"); + return -1; + } + if (!PyUnicode_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "The first attribute value must be a string"); + return -1; + } + Py_INCREF(value); + Py_CLEAR(self->first); + self->first = value; + return 0; + } + + static PyObject * + Custom_getlast(CustomObject *self, void *closure) + { + Py_INCREF(self->last); + return self->last; + } + + static int + Custom_setlast(CustomObject *self, PyObject *value, void *closure) + { + if (value == NULL) { + PyErr_SetString(PyExc_TypeError, "Cannot delete the last attribute"); + return -1; + } + if (!PyUnicode_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "The last attribute value must be a string"); + return -1; + } + Py_INCREF(value); + Py_CLEAR(self->last); + self->last = value; + return 0; + } + + static PyGetSetDef Custom_getsetters[] = { + {"first", (getter) Custom_getfirst, (setter) Custom_setfirst, + "first name", NULL}, + {"last", (getter) Custom_getlast, (setter) Custom_setlast, + "last name", NULL}, + {NULL} /* Sentinel */ + }; + + static PyObject * + Custom_name(CustomObject *self, PyObject *Py_UNUSED(ignored)) + { + return PyUnicode_FromFormat("%S %S", self->first, self->last); + } + + static PyMethodDef Custom_methods[] = { + {"name", (PyCFunction) Custom_name, METH_NOARGS, + "Return the name, combining the first and last name" + }, + {NULL} /* Sentinel */ + }; + + static PyTypeObject CustomType = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "custom4.Custom", + .tp_doc = "Custom objects", + .tp_basicsize = sizeof(CustomObject), + .tp_itemsize = 0, + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, + .tp_new = Custom_new, + .tp_init = (initproc) Custom_init, + .tp_dealloc = (destructor) Custom_dealloc, + .tp_traverse = (traverseproc) Custom_traverse, + .tp_clear = (inquiry) Custom_clear, + .tp_members = Custom_members, + .tp_methods = Custom_methods, + .tp_getset = Custom_getsetters, + }; + + static PyModuleDef custommodule = { + PyModuleDef_HEAD_INIT, + .m_name = "custom4", + .m_doc = "Example module that creates an extension type.", + .m_size = -1, + }; + + PyMODINIT_FUNC + PyInit_custom4(void) + { + PyObject *m; + if (PyType_Ready(&CustomType) < 0) + return NULL; + + m = PyModule_Create(&custommodule); + if (m == NULL) + return NULL; + + Py_INCREF(&CustomType); + if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { + Py_DECREF(&CustomType); + Py_DECREF(m); + return NULL; + } + + return m; + } + +First, the traversal method lets the cyclic GC know about subobjects +that could participate in cycles: + + static int + Custom_traverse(CustomObject *self, visitproc visit, void *arg) + { + int vret; + if (self->first) { + vret = visit(self->first, arg); + if (vret != 0) + return vret; + } + if (self->last) { + vret = visit(self->last, arg); + if (vret != 0) + return vret; + } + return 0; + } + +For each subobject that can participate in cycles, we need to call the +‘visit()’ function, which is passed to the traversal method. The +‘visit()’ function takes as arguments the subobject and the extra +argument `arg' passed to the traversal method. It returns an integer +value that must be returned if it is non-zero. + +Python provides a *note Py_VISIT(): 388c. macro that automates calling +visit functions. With *note Py_VISIT(): 388c, we can minimize the +amount of boilerplate in ‘Custom_traverse’: + + static int + Custom_traverse(CustomObject *self, visitproc visit, void *arg) + { + Py_VISIT(self->first); + Py_VISIT(self->last); + return 0; + } + + Note: The *note tp_traverse: 33e8. implementation must name its + arguments exactly `visit' and `arg' in order to use *note + Py_VISIT(): 388c. + +Second, we need to provide a method for clearing any subobjects that can +participate in cycles: + + static int + Custom_clear(CustomObject *self) + { + Py_CLEAR(self->first); + Py_CLEAR(self->last); + return 0; + } + +Notice the use of the *note Py_CLEAR(): 388d. macro. It is the +recommended and safe way to clear data attributes of arbitrary types +while decrementing their reference counts. If you were to call *note +Py_XDECREF(): 268. instead on the attribute before setting it to ‘NULL’, +there is a possibility that the attribute’s destructor would call back +into code that reads the attribute again (`especially' if there is a +reference cycle). + + Note: You could emulate *note Py_CLEAR(): 388d. by writing: + + PyObject *tmp; + tmp = self->first; + self->first = NULL; + Py_XDECREF(tmp); + + Nevertheless, it is much easier and less error-prone to always use + *note Py_CLEAR(): 388d. when deleting an attribute. Don’t try to + micro-optimize at the expense of robustness! + +The deallocator ‘Custom_dealloc’ may call arbitrary code when clearing +attributes. It means the circular GC can be triggered inside the +function. Since the GC assumes reference count is not zero, we need to +untrack the object from the GC by calling *note PyObject_GC_UnTrack(): +e45. before clearing members. Here is our reimplemented deallocator +using *note PyObject_GC_UnTrack(): e45. and ‘Custom_clear’: + + static void + Custom_dealloc(CustomObject *self) + { + PyObject_GC_UnTrack(self); + Custom_clear(self); + Py_TYPE(self)->tp_free((PyObject *) self); + } + +Finally, we add the *note Py_TPFLAGS_HAVE_GC: 388e. flag to the class +flags: + + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, + +That’s pretty much it. If we had written custom *note tp_alloc: 3881. +or *note tp_free: 3880. handlers, we’d need to modify them for cyclic +garbage collection. Most extensions will use the versions automatically +provided. + + ---------- Footnotes ---------- + + (1) (4) Also, even with our attributes restricted to strings +instances, the user could pass arbitrary *note str: 330. subclasses and +therefore still create reference cycles. + + +File: python.info, Node: Subclassing other types, Prev: Supporting cyclic garbage collection, Up: Defining Extension Types Tutorial + +6.2.2.5 Subclassing other types +............................... + +It is possible to create new extension types that are derived from +existing types. It is easiest to inherit from the built in types, since +an extension can easily use the *note PyTypeObject: 2d6. it needs. It +can be difficult to share these *note PyTypeObject: 2d6. structures +between extension modules. + +In this example we will create a ‘SubList’ type that inherits from the +built-in *note list: 262. type. The new type will be completely +compatible with regular lists, but will have an additional ‘increment()’ +method that increases an internal counter: + + >>> import sublist + >>> s = sublist.SubList(range(3)) + >>> s.extend(s) + >>> print(len(s)) + 6 + >>> print(s.increment()) + 1 + >>> print(s.increment()) + 2 + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + + typedef struct { + PyListObject list; + int state; + } SubListObject; + + static PyObject * + SubList_increment(SubListObject *self, PyObject *unused) + { + self->state++; + return PyLong_FromLong(self->state); + } + + static PyMethodDef SubList_methods[] = { + {"increment", (PyCFunction) SubList_increment, METH_NOARGS, + PyDoc_STR("increment state counter")}, + {NULL}, + }; + + static int + SubList_init(SubListObject *self, PyObject *args, PyObject *kwds) + { + if (PyList_Type.tp_init((PyObject *) self, args, kwds) < 0) + return -1; + self->state = 0; + return 0; + } + + static PyTypeObject SubListType = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "sublist.SubList", + .tp_doc = "SubList objects", + .tp_basicsize = sizeof(SubListObject), + .tp_itemsize = 0, + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, + .tp_init = (initproc) SubList_init, + .tp_methods = SubList_methods, + }; + + static PyModuleDef sublistmodule = { + PyModuleDef_HEAD_INIT, + .m_name = "sublist", + .m_doc = "Example module that creates an extension type.", + .m_size = -1, + }; + + PyMODINIT_FUNC + PyInit_sublist(void) + { + PyObject *m; + SubListType.tp_base = &PyList_Type; + if (PyType_Ready(&SubListType) < 0) + return NULL; + + m = PyModule_Create(&sublistmodule); + if (m == NULL) + return NULL; + + Py_INCREF(&SubListType); + if (PyModule_AddObject(m, "SubList", (PyObject *) &SubListType) < 0) { + Py_DECREF(&SubListType); + Py_DECREF(m); + return NULL; + } + + return m; + } + +As you can see, the source code closely resembles the ‘Custom’ examples +in previous sections. We will break down the main differences between +them. + + typedef struct { + PyListObject list; + int state; + } SubListObject; + +The primary difference for derived type objects is that the base type’s +object structure must be the first value. The base type will already +include the *note PyObject_HEAD(): c72. at the beginning of its +structure. + +When a Python object is a ‘SubList’ instance, its ‘PyObject *’ pointer +can be safely cast to both ‘PyListObject *’ and ‘SubListObject *’: + + static int + SubList_init(SubListObject *self, PyObject *args, PyObject *kwds) + { + if (PyList_Type.tp_init((PyObject *) self, args, kwds) < 0) + return -1; + self->state = 0; + return 0; + } + +We see above how to call through to the *note __init__: d5e. method of +the base type. + +This pattern is important when writing a type with custom *note tp_new: +387c. and *note tp_dealloc: 387f. members. The *note tp_new: 387c. +handler should not actually create the memory for the object with its +*note tp_alloc: 3881, but let the base class handle it by calling its +own *note tp_new: 387c. + +The *note PyTypeObject: 2d6. struct supports a *note tp_base: 3890. +specifying the type’s concrete base class. Due to cross-platform +compiler issues, you can’t fill that field directly with a reference to +*note PyList_Type: 3891.; it should be done later in the module +initialization function: + + PyMODINIT_FUNC + PyInit_sublist(void) + { + PyObject* m; + SubListType.tp_base = &PyList_Type; + if (PyType_Ready(&SubListType) < 0) + return NULL; + + m = PyModule_Create(&sublistmodule); + if (m == NULL) + return NULL; + + Py_INCREF(&SubListType); + if (PyModule_AddObject(m, "SubList", (PyObject *) &SubListType) < 0) { + Py_DECREF(&SubListType); + Py_DECREF(m); + return NULL; + } + + return m; + } + +Before calling *note PyType_Ready(): 3882, the type structure must have +the *note tp_base: 3890. slot filled in. When we are deriving an +existing type, it is not necessary to fill out the *note tp_alloc: 3881. +slot with *note PyType_GenericNew(): 387d. – the allocation function +from the base type will be inherited. + +After that, calling *note PyType_Ready(): 3882. and adding the type +object to the module is the same as with the basic ‘Custom’ examples. + + +File: python.info, Node: Defining Extension Types Assorted Topics, Next: Building C and C++ Extensions, Prev: Defining Extension Types Tutorial, Up: Creating extensions without third party tools + +6.2.3 Defining Extension Types: Assorted Topics +----------------------------------------------- + +This section aims to give a quick fly-by on the various type methods you +can implement and what they do. + +Here is the definition of *note PyTypeObject: 2d6, with some fields only +used in debug builds omitted: + + typedef struct _typeobject { + PyObject_VAR_HEAD + const char *tp_name; /* For printing, in format "<module>.<name>" */ + Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ + + /* Methods to implement standard operations */ + + destructor tp_dealloc; + Py_ssize_t tp_vectorcall_offset; + getattrfunc tp_getattr; + setattrfunc tp_setattr; + PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2) + or tp_reserved (Python 3) */ + reprfunc tp_repr; + + /* Method suites for standard classes */ + + PyNumberMethods *tp_as_number; + PySequenceMethods *tp_as_sequence; + PyMappingMethods *tp_as_mapping; + + /* More standard operations (here for binary compatibility) */ + + hashfunc tp_hash; + ternaryfunc tp_call; + reprfunc tp_str; + getattrofunc tp_getattro; + setattrofunc tp_setattro; + + /* Functions to access object as input/output buffer */ + PyBufferProcs *tp_as_buffer; + + /* Flags to define presence of optional/expanded features */ + unsigned long tp_flags; + + const char *tp_doc; /* Documentation string */ + + /* call function for all accessible objects */ + traverseproc tp_traverse; + + /* delete references to contained objects */ + inquiry tp_clear; + + /* rich comparisons */ + richcmpfunc tp_richcompare; + + /* weak reference enabler */ + Py_ssize_t tp_weaklistoffset; + + /* Iterators */ + getiterfunc tp_iter; + iternextfunc tp_iternext; + + /* Attribute descriptor and subclassing stuff */ + struct PyMethodDef *tp_methods; + struct PyMemberDef *tp_members; + struct PyGetSetDef *tp_getset; + struct _typeobject *tp_base; + PyObject *tp_dict; + descrgetfunc tp_descr_get; + descrsetfunc tp_descr_set; + Py_ssize_t tp_dictoffset; + initproc tp_init; + allocfunc tp_alloc; + newfunc tp_new; + freefunc tp_free; /* Low-level free-memory routine */ + inquiry tp_is_gc; /* For PyObject_IS_GC */ + PyObject *tp_bases; + PyObject *tp_mro; /* method resolution order */ + PyObject *tp_cache; + PyObject *tp_subclasses; + PyObject *tp_weaklist; + destructor tp_del; + + /* Type attribute cache version tag. Added in version 2.6 */ + unsigned int tp_version_tag; + + destructor tp_finalize; + + } PyTypeObject; + +Now that’s a `lot' of methods. Don’t worry too much though – if you +have a type you want to define, the chances are very good that you will +only implement a handful of these. + +As you probably expect by now, we’re going to go over this and give more +information about the various handlers. We won’t go in the order they +are defined in the structure, because there is a lot of historical +baggage that impacts the ordering of the fields. It’s often easiest to +find an example that includes the fields you need and then change the +values to suit your new type. + + const char *tp_name; /* For printing */ + +The name of the type – as mentioned in the previous chapter, this will +appear in various places, almost entirely for diagnostic purposes. Try +to choose something that will be helpful in such a situation! + + Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ + +These fields tell the runtime how much memory to allocate when new +objects of this type are created. Python has some built-in support for +variable length structures (think: strings, tuples) which is where the +*note tp_itemsize: 3878. field comes in. This will be dealt with later. + + const char *tp_doc; + +Here you can put a string (or its address) that you want returned when +the Python script references ‘obj.__doc__’ to retrieve the doc string. + +Now we come to the basic type methods – the ones most extension types +will implement. + +* Menu: + +* Finalization and De-allocation:: +* Object Presentation:: +* Attribute Management:: +* Object Comparison:: +* Abstract Protocol Support:: +* Weak Reference Support:: +* More Suggestions:: + + +File: python.info, Node: Finalization and De-allocation, Next: Object Presentation, Up: Defining Extension Types Assorted Topics + +6.2.3.1 Finalization and De-allocation +...................................... + + destructor tp_dealloc; + +This function is called when the reference count of the instance of your +type is reduced to zero and the Python interpreter wants to reclaim it. +If your type has memory to free or other clean-up to perform, you can +put it here. The object itself needs to be freed here as well. Here is +an example of this function: + + static void + newdatatype_dealloc(newdatatypeobject *obj) + { + free(obj->obj_UnderlyingDatatypePtr); + Py_TYPE(obj)->tp_free(obj); + } + +One important requirement of the deallocator function is that it leaves +any pending exceptions alone. This is important since deallocators are +frequently called as the interpreter unwinds the Python stack; when the +stack is unwound due to an exception (rather than normal returns), +nothing is done to protect the deallocators from seeing that an +exception has already been set. Any actions which a deallocator +performs which may cause additional Python code to be executed may +detect that an exception has been set. This can lead to misleading +errors from the interpreter. The proper way to protect against this is +to save a pending exception before performing the unsafe action, and +restoring it when done. This can be done using the *note PyErr_Fetch(): +96a. and *note PyErr_Restore(): 3897. functions: + + static void + my_dealloc(PyObject *obj) + { + MyObject *self = (MyObject *) obj; + PyObject *cbresult; + + if (self->my_callback != NULL) { + PyObject *err_type, *err_value, *err_traceback; + + /* This saves the current exception state */ + PyErr_Fetch(&err_type, &err_value, &err_traceback); + + cbresult = PyObject_CallObject(self->my_callback, NULL); + if (cbresult == NULL) + PyErr_WriteUnraisable(self->my_callback); + else + Py_DECREF(cbresult); + + /* This restores the saved exception state */ + PyErr_Restore(err_type, err_value, err_traceback); + + Py_DECREF(self->my_callback); + } + Py_TYPE(obj)->tp_free((PyObject*)self); + } + + Note: There are limitations to what you can safely do in a + deallocator function. First, if your type supports garbage + collection (using *note tp_traverse: 33e8. and/or *note tp_clear: + 3898.), some of the object’s members can have been cleared or + finalized by the time *note tp_dealloc: 387f. is called. Second, + in *note tp_dealloc: 387f, your object is in an unstable state: its + reference count is equal to zero. Any call to a non-trivial object + or API (as in the example above) might end up calling *note + tp_dealloc: 387f. again, causing a double free and a crash. + + Starting with Python 3.4, it is recommended not to put any complex + finalization code in *note tp_dealloc: 387f, and instead use the + new *note tp_finalize: 2d7. type method. + + See also + ........ + + PEP 442(1) explains the new finalization scheme. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0442 + + +File: python.info, Node: Object Presentation, Next: Attribute Management, Prev: Finalization and De-allocation, Up: Defining Extension Types Assorted Topics + +6.2.3.2 Object Presentation +........................... + +In Python, there are two ways to generate a textual representation of an +object: the *note repr(): 7cc. function, and the *note str(): 330. +function. (The *note print(): 886. function just calls *note str(): +330.) These handlers are both optional. + + reprfunc tp_repr; + reprfunc tp_str; + +The *note tp_repr: 389a. handler should return a string object +containing a representation of the instance for which it is called. +Here is a simple example: + + static PyObject * + newdatatype_repr(newdatatypeobject * obj) + { + return PyUnicode_FromFormat("Repr-ified_newdatatype{{size:%d}}", + obj->obj_UnderlyingDatatypePtr->size); + } + +If no *note tp_repr: 389a. handler is specified, the interpreter will +supply a representation that uses the type’s *note tp_name: 389b. and a +uniquely-identifying value for the object. + +The *note tp_str: 389c. handler is to *note str(): 330. what the *note +tp_repr: 389a. handler described above is to *note repr(): 7cc.; that +is, it is called when Python code calls *note str(): 330. on an instance +of your object. Its implementation is very similar to the *note +tp_repr: 389a. function, but the resulting string is intended for human +consumption. If *note tp_str: 389c. is not specified, the *note +tp_repr: 389a. handler is used instead. + +Here is a simple example: + + static PyObject * + newdatatype_str(newdatatypeobject * obj) + { + return PyUnicode_FromFormat("Stringified_newdatatype{{size:%d}}", + obj->obj_UnderlyingDatatypePtr->size); + } + + +File: python.info, Node: Attribute Management, Next: Object Comparison, Prev: Object Presentation, Up: Defining Extension Types Assorted Topics + +6.2.3.3 Attribute Management +............................ + +For every object which can support attributes, the corresponding type +must provide the functions that control how the attributes are resolved. +There needs to be a function which can retrieve attributes (if any are +defined), and another to set attributes (if setting attributes is +allowed). Removing an attribute is a special case, for which the new +value passed to the handler is ‘NULL’. + +Python supports two pairs of attribute handlers; a type that supports +attributes only needs to implement the functions for one pair. The +difference is that one pair takes the name of the attribute as a +‘char*’, while the other accepts a *note PyObject*: 4ba. Each type can +use whichever pair makes more sense for the implementation’s +convenience. + + getattrfunc tp_getattr; /* char * version */ + setattrfunc tp_setattr; + /* ... */ + getattrofunc tp_getattro; /* PyObject * version */ + setattrofunc tp_setattro; + +If accessing attributes of an object is always a simple operation (this +will be explained shortly), there are generic implementations which can +be used to provide the *note PyObject*: 4ba. version of the attribute +management functions. The actual need for type-specific attribute +handlers almost completely disappeared starting with Python 2.2, though +there are many examples which have not been updated to use some of the +new generic mechanism that is available. + +* Menu: + +* Generic Attribute Management:: +* Type-specific Attribute Management:: + + +File: python.info, Node: Generic Attribute Management, Next: Type-specific Attribute Management, Up: Attribute Management + +6.2.3.4 Generic Attribute Management +.................................... + +Most extension types only use `simple' attributes. So, what makes the +attributes simple? There are only a couple of conditions that must be +met: + + 1. The name of the attributes must be known when *note PyType_Ready(): + 3882. is called. + + 2. No special processing is needed to record that an attribute was + looked up or set, nor do actions need to be taken based on the + value. + +Note that this list does not place any restrictions on the values of the +attributes, when the values are computed, or how relevant data is +stored. + +When *note PyType_Ready(): 3882. is called, it uses three tables +referenced by the type object to create *note descriptor: 12b0.s which +are placed in the dictionary of the type object. Each descriptor +controls access to one attribute of the instance object. Each of the +tables is optional; if all three are ‘NULL’, instances of the type will +only have attributes that are inherited from their base type, and should +leave the *note tp_getattro: 389f. and *note tp_setattro: 38a0. fields +‘NULL’ as well, allowing the base type to handle attributes. + +The tables are declared as three fields of the type object: + + struct PyMethodDef *tp_methods; + struct PyMemberDef *tp_members; + struct PyGetSetDef *tp_getset; + +If *note tp_methods: 3886. is not ‘NULL’, it must refer to an array of +*note PyMethodDef: e1b. structures. Each entry in the table is an +instance of this structure: + + typedef struct PyMethodDef { + const char *ml_name; /* method name */ + PyCFunction ml_meth; /* implementation function */ + int ml_flags; /* flags */ + const char *ml_doc; /* docstring */ + } PyMethodDef; + +One entry should be defined for each method provided by the type; no +entries are needed for methods inherited from a base type. One +additional entry is needed at the end; it is a sentinel that marks the +end of the array. The ‘ml_name’ field of the sentinel must be ‘NULL’. + +The second table is used to define attributes which map directly to data +stored in the instance. A variety of primitive C types are supported, +and access may be read-only or read-write. The structures in the table +are defined as: + + typedef struct PyMemberDef { + const char *name; + int type; + int offset; + int flags; + const char *doc; + } PyMemberDef; + +For each entry in the table, a *note descriptor: 12b0. will be +constructed and added to the type which will be able to extract a value +from the instance structure. The *note type: 608. field should contain +one of the type codes defined in the ‘structmember.h’ header; the value +will be used to determine how to convert Python values to and from C +values. The ‘flags’ field is used to store flags which control how the +attribute can be accessed. + +The following flag constants are defined in ‘structmember.h’; they may +be combined using bitwise-OR. + +Constant Meaning + +----------------------------------------------------------------------------------- + +‘READONLY’ Never writable. + + +‘READ_RESTRICTED’ Not readable in restricted mode. + + +‘WRITE_RESTRICTED’ Not writable in restricted mode. + + +‘RESTRICTED’ Not readable or writable in restricted mode. + + +An interesting advantage of using the *note tp_members: 3884. table to +build descriptors that are used at runtime is that any attribute defined +this way can have an associated doc string simply by providing the text +in the table. An application can use the introspection API to retrieve +the descriptor from the class object, and get the doc string using its +‘__doc__’ attribute. + +As with the *note tp_methods: 3886. table, a sentinel entry with a +‘name’ value of ‘NULL’ is required. + + +File: python.info, Node: Type-specific Attribute Management, Prev: Generic Attribute Management, Up: Attribute Management + +6.2.3.5 Type-specific Attribute Management +.......................................... + +For simplicity, only the ‘char*’ version will be demonstrated here; the +type of the name parameter is the only difference between the ‘char*’ +and *note PyObject*: 4ba. flavors of the interface. This example +effectively does the same thing as the generic example above, but does +not use the generic support added in Python 2.2. It explains how the +handler functions are called, so that if you do need to extend their +functionality, you’ll understand what needs to be done. + +The *note tp_getattr: 38a2. handler is called when the object requires +an attribute look-up. It is called in the same situations where the +*note __getattr__(): 31a. method of a class would be called. + +Here is an example: + + static PyObject * + newdatatype_getattr(newdatatypeobject *obj, char *name) + { + if (strcmp(name, "data") == 0) + { + return PyLong_FromLong(obj->data); + } + + PyErr_Format(PyExc_AttributeError, + "'%.50s' object has no attribute '%.400s'", + tp->tp_name, name); + return NULL; + } + +The *note tp_setattr: 38a3. handler is called when the *note +__setattr__(): e2c. or *note __delattr__(): 10d1. method of a class +instance would be called. When an attribute should be deleted, the +third parameter will be ‘NULL’. Here is an example that simply raises +an exception; if this were really all you wanted, the *note tp_setattr: +38a3. handler should be set to ‘NULL’. + + static int + newdatatype_setattr(newdatatypeobject *obj, char *name, PyObject *v) + { + PyErr_Format(PyExc_RuntimeError, "Read-only attribute: %s", name); + return -1; + } + + +File: python.info, Node: Object Comparison, Next: Abstract Protocol Support, Prev: Attribute Management, Up: Defining Extension Types Assorted Topics + +6.2.3.6 Object Comparison +......................... + + richcmpfunc tp_richcompare; + +The *note tp_richcompare: 38a5. handler is called when comparisons are +needed. It is analogous to the *note rich comparison methods: 10da, +like *note __lt__(): c34, and also called by *note +PyObject_RichCompare(): 38a6. and *note PyObject_RichCompareBool(): +38a7. + +This function is called with two Python objects and the operator as +arguments, where the operator is one of ‘Py_EQ’, ‘Py_NE’, ‘Py_LE’, +‘Py_GT’, ‘Py_LT’ or ‘Py_GT’. It should compare the two objects with +respect to the specified operator and return ‘Py_True’ or ‘Py_False’ if +the comparison is successful, ‘Py_NotImplemented’ to indicate that +comparison is not implemented and the other object’s comparison method +should be tried, or ‘NULL’ if an exception was set. + +Here is a sample implementation, for a datatype that is considered equal +if the size of an internal pointer is equal: + + static PyObject * + newdatatype_richcmp(PyObject *obj1, PyObject *obj2, int op) + { + PyObject *result; + int c, size1, size2; + + /* code to make sure that both arguments are of type + newdatatype omitted */ + + size1 = obj1->obj_UnderlyingDatatypePtr->size; + size2 = obj2->obj_UnderlyingDatatypePtr->size; + + switch (op) { + case Py_LT: c = size1 < size2; break; + case Py_LE: c = size1 <= size2; break; + case Py_EQ: c = size1 == size2; break; + case Py_NE: c = size1 != size2; break; + case Py_GT: c = size1 > size2; break; + case Py_GE: c = size1 >= size2; break; + } + result = c ? Py_True : Py_False; + Py_INCREF(result); + return result; + } + + +File: python.info, Node: Abstract Protocol Support, Next: Weak Reference Support, Prev: Object Comparison, Up: Defining Extension Types Assorted Topics + +6.2.3.7 Abstract Protocol Support +................................. + +Python supports a variety of `abstract' ‘protocols;’ the specific +interfaces provided to use these interfaces are documented in *note +Abstract Objects Layer: 38a9. + +A number of these abstract interfaces were defined early in the +development of the Python implementation. In particular, the number, +mapping, and sequence protocols have been part of Python since the +beginning. Other protocols have been added over time. For protocols +which depend on several handler routines from the type implementation, +the older protocols have been defined as optional blocks of handlers +referenced by the type object. For newer protocols there are additional +slots in the main type object, with a flag bit being set to indicate +that the slots are present and should be checked by the interpreter. +(The flag bit does not indicate that the slot values are non-‘NULL’. +The flag may be set to indicate the presence of a slot, but a slot may +still be unfilled.) + + PyNumberMethods *tp_as_number; + PySequenceMethods *tp_as_sequence; + PyMappingMethods *tp_as_mapping; + +If you wish your object to be able to act like a number, a sequence, or +a mapping object, then you place the address of a structure that +implements the C type *note PyNumberMethods: d81, *note +PySequenceMethods: 38aa, or *note PyMappingMethods: 38ab, respectively. +It is up to you to fill in this structure with appropriate values. You +can find examples of the use of each of these in the ‘Objects’ directory +of the Python source distribution. + + hashfunc tp_hash; + +This function, if you choose to provide it, should return a hash number +for an instance of your data type. Here is a simple example: + + static Py_hash_t + newdatatype_hash(newdatatypeobject *obj) + { + Py_hash_t result; + result = obj->some_size + 32767 * obj->some_number; + if (result == -1) + result = -2; + return result; + } + +‘Py_hash_t’ is a signed integer type with a platform-varying width. +Returning ‘-1’ from *note tp_hash: 38ac. indicates an error, which is +why you should be careful to avoid returning it when hash computation is +successful, as seen above. + + ternaryfunc tp_call; + +This function is called when an instance of your data type is “called”, +for example, if ‘obj1’ is an instance of your data type and the Python +script contains ‘obj1('hello')’, the *note tp_call: 38ad. handler is +invoked. + +This function takes three arguments: + + 1. `self' is the instance of the data type which is the subject of the + call. If the call is ‘obj1('hello')’, then `self' is ‘obj1’. + + 2. `args' is a tuple containing the arguments to the call. You can + use *note PyArg_ParseTuple(): 26a. to extract the arguments. + + 3. `kwds' is a dictionary of keyword arguments that were passed. If + this is non-‘NULL’ and you support keyword arguments, use *note + PyArg_ParseTupleAndKeywords(): 5ca. to extract the arguments. If + you do not want to support keyword arguments and this is + non-‘NULL’, raise a *note TypeError: 192. with a message saying + that keyword arguments are not supported. + +Here is a toy ‘tp_call’ implementation: + + static PyObject * + newdatatype_call(newdatatypeobject *self, PyObject *args, PyObject *kwds) + { + PyObject *result; + const char *arg1; + const char *arg2; + const char *arg3; + + if (!PyArg_ParseTuple(args, "sss:call", &arg1, &arg2, &arg3)) { + return NULL; + } + result = PyUnicode_FromFormat( + "Returning -- value: [%d] arg1: [%s] arg2: [%s] arg3: [%s]\n", + obj->obj_UnderlyingDatatypePtr->size, + arg1, arg2, arg3); + return result; + } + + /* Iterators */ + getiterfunc tp_iter; + iternextfunc tp_iternext; + +These functions provide support for the iterator protocol. Both +handlers take exactly one parameter, the instance for which they are +being called, and return a new reference. In the case of an error, they +should set an exception and return ‘NULL’. *note tp_iter: e30. +corresponds to the Python *note __iter__(): 50f. method, while *note +tp_iternext: e31. corresponds to the Python *note __next__(): c64. +method. + +Any *note iterable: bac. object must implement the *note tp_iter: e30. +handler, which must return an *note iterator: 112e. object. Here the +same guidelines apply as for Python classes: + + * For collections (such as lists and tuples) which can support + multiple independent iterators, a new iterator should be created + and returned by each call to *note tp_iter: e30. + + * Objects which can only be iterated over once (usually due to side + effects of iteration, such as file objects) can implement *note + tp_iter: e30. by returning a new reference to themselves – and + should also therefore implement the *note tp_iternext: e31. + handler. + +Any *note iterator: 112e. object should implement both *note tp_iter: +e30. and *note tp_iternext: e31. An iterator’s *note tp_iter: e30. +handler should return a new reference to the iterator. Its *note +tp_iternext: e31. handler should return a new reference to the next +object in the iteration, if there is one. If the iteration has reached +the end, *note tp_iternext: e31. may return ‘NULL’ without setting an +exception, or it may set *note StopIteration: 486. `in addition' to +returning ‘NULL’; avoiding the exception can yield slightly better +performance. If an actual error occurs, *note tp_iternext: e31. should +always set an exception and return ‘NULL’. + + +File: python.info, Node: Weak Reference Support, Next: More Suggestions, Prev: Abstract Protocol Support, Up: Defining Extension Types Assorted Topics + +6.2.3.8 Weak Reference Support +.............................. + +One of the goals of Python’s weak reference implementation is to allow +any type to participate in the weak reference mechanism without +incurring the overhead on performance-critical objects (such as +numbers). + +See also +........ + +Documentation for the *note weakref: 128. module. + +For an object to be weakly referencable, the extension type must do two +things: + + 1. Include a *note PyObject*: 4ba. field in the C object structure + dedicated to the weak reference mechanism. The object’s + constructor should leave it ‘NULL’ (which is automatic when using + the default *note tp_alloc: 3881.). + + 2. Set the *note tp_weaklistoffset: 38af. type member to the offset of + the aforementioned field in the C object structure, so that the + interpreter knows how to access and modify that field. + +Concretely, here is how a trivial object structure would be augmented +with the required field: + + typedef struct { + PyObject_HEAD + PyObject *weakreflist; /* List of weak references */ + } TrivialObject; + +And the corresponding member in the statically-declared type object: + + static PyTypeObject TrivialType = { + PyVarObject_HEAD_INIT(NULL, 0) + /* ... other members omitted for brevity ... */ + .tp_weaklistoffset = offsetof(TrivialObject, weakreflist), + }; + +The only further addition is that ‘tp_dealloc’ needs to clear any weak +references (by calling ‘PyObject_ClearWeakRefs()’) if the field is +non-‘NULL’: + + static void + Trivial_dealloc(TrivialObject *self) + { + /* Clear weakrefs first before calling any destructors */ + if (self->weakreflist != NULL) + PyObject_ClearWeakRefs((PyObject *) self); + /* ... remainder of destruction code omitted for brevity ... */ + Py_TYPE(self)->tp_free((PyObject *) self); + } + + +File: python.info, Node: More Suggestions, Prev: Weak Reference Support, Up: Defining Extension Types Assorted Topics + +6.2.3.9 More Suggestions +........................ + +In order to learn how to implement any specific method for your new data +type, get the *note CPython: 10d7. source code. Go to the ‘Objects’ +directory, then search the C source files for ‘tp_’ plus the function +you want (for example, ‘tp_richcompare’). You will find examples of the +function you want to implement. + +When you need to verify that an object is a concrete instance of the +type you are implementing, use the *note PyObject_TypeCheck(): 38b1. +function. A sample of its use might be something like the following: + + if (!PyObject_TypeCheck(some_object, &MyType)) { + PyErr_SetString(PyExc_TypeError, "arg #1 not a mything"); + return NULL; + } + +See also +........ + +Download CPython source releases. + + ‘https://www.python.org/downloads/source/’ + +The CPython project on GitHub, where the CPython source code is developed. + + ‘https://github.com/python/cpython’ + + +File: python.info, Node: Building C and C++ Extensions, Next: Building C and C++ Extensions on Windows, Prev: Defining Extension Types Assorted Topics, Up: Creating extensions without third party tools + +6.2.4 Building C and C++ Extensions +----------------------------------- + +A C extension for CPython is a shared library (e.g. a ‘.so’ file on +Linux, ‘.pyd’ on Windows), which exports an `initialization function'. + +To be importable, the shared library must be available on *note +PYTHONPATH: 953, and must be named after the module name, with an +appropriate extension. When using distutils, the correct filename is +generated automatically. + +The initialization function has the signature: + + -- C Function: PyObject* PyInit_modulename (void) + +It returns either a fully-initialized module, or a *note PyModuleDef: +3888. instance. See *note Initializing C modules: 38b5. for details. + +For modules with ASCII-only names, the function must be named +‘PyInit_<modulename>’, with ‘<modulename>’ replaced by the name of the +module. When using *note Multi-phase initialization: 38b6, non-ASCII +module names are allowed. In this case, the initialization function +name is ‘PyInitU_<modulename>’, with ‘<modulename>’ encoded using +Python’s `punycode' encoding with hyphens replaced by underscores. In +Python: + + def initfunc_name(name): + try: + suffix = b'_' + name.encode('ascii') + except UnicodeEncodeError: + suffix = b'U_' + name.encode('punycode').replace(b'-', b'_') + return b'PyInit' + suffix + +It is possible to export multiple modules from a single shared library +by defining multiple initialization functions. However, importing them +requires using symbolic links or a custom importer, because by default +only the function corresponding to the filename is found. See the +`“Multiple modules in one library”' section in PEP 489(1) for details. + +* Menu: + +* Building C and C++ Extensions with distutils:: +* Distributing your extension modules:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0489 + + +File: python.info, Node: Building C and C++ Extensions with distutils, Next: Distributing your extension modules, Up: Building C and C++ Extensions + +6.2.4.1 Building C and C++ Extensions with distutils +.................................................... + +Extension modules can be built using distutils, which is included in +Python. Since distutils also supports creation of binary packages, +users don’t necessarily need a compiler and distutils to install the +extension. + +A distutils package contains a driver script, ‘setup.py’. This is a +plain Python file, which, in the most simple case, could look like this: + + from distutils.core import setup, Extension + + module1 = Extension('demo', + sources = ['demo.c']) + + setup (name = 'PackageName', + version = '1.0', + description = 'This is a demo package', + ext_modules = [module1]) + +With this ‘setup.py’, and a file ‘demo.c’, running + + python setup.py build + +will compile ‘demo.c’, and produce an extension module named ‘demo’ in +the ‘build’ directory. Depending on the system, the module file will +end up in a subdirectory ‘build/lib.system’, and may have a name like +‘demo.so’ or ‘demo.pyd’. + +In the ‘setup.py’, all execution is performed by calling the ‘setup’ +function. This takes a variable number of keyword arguments, of which +the example above uses only a subset. Specifically, the example +specifies meta-information to build packages, and it specifies the +contents of the package. Normally, a package will contain additional +modules, like Python source modules, documentation, subpackages, etc. +Please refer to the distutils documentation in *note Distributing Python +Modules (Legacy version): 7f8. to learn more about the features of +distutils; this section explains building extension modules only. + +It is common to pre-compute arguments to ‘setup()’, to better structure +the driver script. In the example above, the ‘ext_modules’ argument to +*note setup(): 38b8. is a list of extension modules, each of which is an +instance of the ‘Extension’. In the example, the instance defines an +extension named ‘demo’ which is build by compiling a single source file, +‘demo.c’. + +In many cases, building an extension is more complex, since additional +preprocessor defines and libraries may be needed. This is demonstrated +in the example below. + + from distutils.core import setup, Extension + + module1 = Extension('demo', + define_macros = [('MAJOR_VERSION', '1'), + ('MINOR_VERSION', '0')], + include_dirs = ['/usr/local/include'], + libraries = ['tcl83'], + library_dirs = ['/usr/local/lib'], + sources = ['demo.c']) + + setup (name = 'PackageName', + version = '1.0', + description = 'This is a demo package', + author = 'Martin v. Loewis', + author_email = 'martin@v.loewis.de', + url = 'https://docs.python.org/extending/building', + long_description = ''' + This is really just a demo package. + ''', + ext_modules = [module1]) + +In this example, *note setup(): 38b8. is called with additional +meta-information, which is recommended when distribution packages have +to be built. For the extension itself, it specifies preprocessor +defines, include directories, library directories, and libraries. +Depending on the compiler, distutils passes this information in +different ways to the compiler. For example, on Unix, this may result +in the compilation commands + + gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -DMAJOR_VERSION=1 -DMINOR_VERSION=0 -I/usr/local/include -I/usr/local/include/python2.2 -c demo.c -o build/temp.linux-i686-2.2/demo.o + + gcc -shared build/temp.linux-i686-2.2/demo.o -L/usr/local/lib -ltcl83 -o build/lib.linux-i686-2.2/demo.so + +These lines are for demonstration purposes only; distutils users should +trust that distutils gets the invocations right. + + +File: python.info, Node: Distributing your extension modules, Prev: Building C and C++ Extensions with distutils, Up: Building C and C++ Extensions + +6.2.4.2 Distributing your extension modules +........................................... + +When an extension has been successfully built, there are three ways to +use it. + +End-users will typically want to install the module, they do so by +running + + python setup.py install + +Module maintainers should produce source packages; to do so, they run + + python setup.py sdist + +In some cases, additional files need to be included in a source +distribution; this is done through a ‘MANIFEST.in’ file; see *note +Specifying the files to distribute: 38bb. for details. + +If the source distribution has been built successfully, maintainers can +also create binary distributions. Depending on the platform, one of the +following commands can be used to do so. + + python setup.py bdist_wininst + python setup.py bdist_rpm + python setup.py bdist_dumb + + +File: python.info, Node: Building C and C++ Extensions on Windows, Prev: Building C and C++ Extensions, Up: Creating extensions without third party tools + +6.2.5 Building C and C++ Extensions on Windows +---------------------------------------------- + +This chapter briefly explains how to create a Windows extension module +for Python using Microsoft Visual C++, and follows with more detailed +background information on how it works. The explanatory material is +useful for both the Windows programmer learning to build Python +extensions and the Unix programmer interested in producing software +which can be successfully built on both Unix and Windows. + +Module authors are encouraged to use the distutils approach for building +extension modules, instead of the one described in this section. You +will still need the C compiler that was used to build Python; typically +Microsoft Visual C++. + + Note: This chapter mentions a number of filenames that include an + encoded Python version number. These filenames are represented + with the version number shown as ‘XY’; in practice, ‘'X'’ will be + the major version number and ‘'Y'’ will be the minor version number + of the Python release you’re working with. For example, if you are + using Python 2.2.1, ‘XY’ will actually be ‘22’. + +* Menu: + +* A Cookbook Approach:: +* Differences Between Unix and Windows:: +* Using DLLs in Practice:: + + +File: python.info, Node: A Cookbook Approach, Next: Differences Between Unix and Windows, Up: Building C and C++ Extensions on Windows + +6.2.5.1 A Cookbook Approach +........................... + +There are two approaches to building extension modules on Windows, just +as there are on Unix: use the *note distutils: 39. package to control +the build process, or do things manually. The distutils approach works +well for most extensions; documentation on using *note distutils: 39. to +build and package extension modules is available in *note Distributing +Python Modules (Legacy version): 7f8. If you find you really need to do +things manually, it may be instructive to study the project file for the +winsound(1) standard library module. + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/PCbuild/winsound.vcxproj + + +File: python.info, Node: Differences Between Unix and Windows, Next: Using DLLs in Practice, Prev: A Cookbook Approach, Up: Building C and C++ Extensions on Windows + +6.2.5.2 Differences Between Unix and Windows +............................................ + +Unix and Windows use completely different paradigms for run-time loading +of code. Before you try to build a module that can be dynamically +loaded, be aware of how your system works. + +In Unix, a shared object (‘.so’) file contains code to be used by the +program, and also the names of functions and data that it expects to +find in the program. When the file is joined to the program, all +references to those functions and data in the file’s code are changed to +point to the actual locations in the program where the functions and +data are placed in memory. This is basically a link operation. + +In Windows, a dynamic-link library (‘.dll’) file has no dangling +references. Instead, an access to functions or data goes through a +lookup table. So the DLL code does not have to be fixed up at runtime +to refer to the program’s memory; instead, the code already uses the +DLL’s lookup table, and the lookup table is modified at runtime to point +to the functions and data. + +In Unix, there is only one type of library file (‘.a’) which contains +code from several object files (‘.o’). During the link step to create a +shared object file (‘.so’), the linker may find that it doesn’t know +where an identifier is defined. The linker will look for it in the +object files in the libraries; if it finds it, it will include all the +code from that object file. + +In Windows, there are two types of library, a static library and an +import library (both called ‘.lib’). A static library is like a Unix +‘.a’ file; it contains code to be included as necessary. An import +library is basically used only to reassure the linker that a certain +identifier is legal, and will be present in the program when the DLL is +loaded. So the linker uses the information from the import library to +build the lookup table for using identifiers that are not included in +the DLL. When an application or a DLL is linked, an import library may +be generated, which will need to be used for all future DLLs that depend +on the symbols in the application or DLL. + +Suppose you are building two dynamic-load modules, B and C, which should +share another block of code A. On Unix, you would `not' pass ‘A.a’ to +the linker for ‘B.so’ and ‘C.so’; that would cause it to be included +twice, so that B and C would each have their own copy. In Windows, +building ‘A.dll’ will also build ‘A.lib’. You `do' pass ‘A.lib’ to the +linker for B and C. ‘A.lib’ does not contain code; it just contains +information which will be used at runtime to access A’s code. + +In Windows, using an import library is sort of like using ‘import spam’; +it gives you access to spam’s names, but does not create a separate +copy. On Unix, linking with a library is more like ‘from spam import +*’; it does create a separate copy. + + +File: python.info, Node: Using DLLs in Practice, Prev: Differences Between Unix and Windows, Up: Building C and C++ Extensions on Windows + +6.2.5.3 Using DLLs in Practice +.............................. + +Windows Python is built in Microsoft Visual C++; using other compilers +may or may not work (though Borland seems to). The rest of this section +is MSVC++ specific. + +When creating DLLs in Windows, you must pass ‘pythonXY.lib’ to the +linker. To build two DLLs, spam and ni (which uses C functions found in +spam), you could use these commands: + + cl /LD /I/python/include spam.c ../libs/pythonXY.lib + cl /LD /I/python/include ni.c spam.lib ../libs/pythonXY.lib + +The first command created three files: ‘spam.obj’, ‘spam.dll’ and +‘spam.lib’. ‘Spam.dll’ does not contain any Python functions (such as +*note PyArg_ParseTuple(): 26a.), but it does know how to find the Python +code thanks to ‘pythonXY.lib’. + +The second command created ‘ni.dll’ (and ‘.obj’ and ‘.lib’), which knows +how to find the necessary functions from spam, and also from the Python +executable. + +Not every identifier is exported to the lookup table. If you want any +other modules (including Python) to be able to see your identifiers, you +have to say ‘_declspec(dllexport)’, as in ‘void _declspec(dllexport) +initspam(void)’ or ‘PyObject _declspec(dllexport) *NiGetSpamData(void)’. + +Developer Studio will throw in a lot of import libraries that you do not +really need, adding about 100K to your executable. To get rid of them, +use the Project Settings dialog, Link tab, to specify `ignore default +libraries'. Add the correct ‘msvcrtxx.lib’ to the list of libraries. + + +File: python.info, Node: Embedding the CPython runtime in a larger application, Prev: Creating extensions without third party tools, Up: Extending and Embedding the Python Interpreter + +6.3 Embedding the CPython runtime in a larger application +========================================================= + +Sometimes, rather than creating an extension that runs inside the Python +interpreter as the main application, it is desirable to instead embed +the CPython runtime inside a larger application. This section covers +some of the details involved in doing that successfully. + +* Menu: + +* Embedding Python in Another Application:: + + +File: python.info, Node: Embedding Python in Another Application, Up: Embedding the CPython runtime in a larger application + +6.3.1 Embedding Python in Another Application +--------------------------------------------- + +The previous chapters discussed how to extend Python, that is, how to +extend the functionality of Python by attaching a library of C functions +to it. It is also possible to do it the other way around: enrich your +C/C++ application by embedding Python in it. Embedding provides your +application with the ability to implement some of the functionality of +your application in Python rather than C or C++. This can be used for +many purposes; one example would be to allow users to tailor the +application to their needs by writing some scripts in Python. You can +also use it yourself if some of the functionality can be written in +Python more easily. + +Embedding Python is similar to extending it, but not quite. The +difference is that when you extend Python, the main program of the +application is still the Python interpreter, while if you embed Python, +the main program may have nothing to do with Python — instead, some +parts of the application occasionally call the Python interpreter to run +some Python code. + +So if you are embedding Python, you are providing your own main program. +One of the things this main program has to do is initialize the Python +interpreter. At the very least, you have to call the function *note +Py_Initialize(): 439. There are optional calls to pass command line +arguments to Python. Then later you can call the interpreter from any +part of the application. + +There are several different ways to call the interpreter: you can pass a +string containing Python statements to *note PyRun_SimpleString(): 38c8, +or you can pass a stdio file pointer and a file name (for identification +in error messages only) to *note PyRun_SimpleFile(): 38c9. You can also +call the lower-level operations described in the previous chapters to +construct and use Python objects. + +See also +........ + +*note Python/C API Reference Manual: e91. + + The details of Python’s C interface are given in this manual. A + great deal of necessary information can be found here. + +* Menu: + +* Very High Level Embedding:: +* Beyond Very High Level Embedding; An overview: Beyond Very High Level Embedding An overview. +* Pure Embedding:: +* Extending Embedded Python:: +* Embedding Python in C++:: +* Compiling and Linking under Unix-like systems:: + + +File: python.info, Node: Very High Level Embedding, Next: Beyond Very High Level Embedding An overview, Up: Embedding Python in Another Application + +6.3.1.1 Very High Level Embedding +................................. + +The simplest form of embedding Python is the use of the very high level +interface. This interface is intended to execute a Python script +without needing to interact with the application directly. This can for +example be used to perform some operation on a file. + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + + int + main(int argc, char *argv[]) + { + wchar_t *program = Py_DecodeLocale(argv[0], NULL); + if (program == NULL) { + fprintf(stderr, "Fatal error: cannot decode argv[0]\n"); + exit(1); + } + Py_SetProgramName(program); /* optional but recommended */ + Py_Initialize(); + PyRun_SimpleString("from time import time,ctime\n" + "print('Today is', ctime(time()))\n"); + if (Py_FinalizeEx() < 0) { + exit(120); + } + PyMem_RawFree(program); + return 0; + } + +The *note Py_SetProgramName(): 1031. function should be called before +*note Py_Initialize(): 439. to inform the interpreter about paths to +Python run-time libraries. Next, the Python interpreter is initialized +with *note Py_Initialize(): 439, followed by the execution of a +hard-coded Python script that prints the date and time. Afterwards, the +*note Py_FinalizeEx(): 5c9. call shuts the interpreter down, followed by +the end of the program. In a real program, you may want to get the +Python script from another source, perhaps a text-editor routine, a +file, or a database. Getting the Python code from a file can better be +done by using the *note PyRun_SimpleFile(): 38c9. function, which saves +you the trouble of allocating memory space and loading the file +contents. + + +File: python.info, Node: Beyond Very High Level Embedding An overview, Next: Pure Embedding, Prev: Very High Level Embedding, Up: Embedding Python in Another Application + +6.3.1.2 Beyond Very High Level Embedding: An overview +..................................................... + +The high level interface gives you the ability to execute arbitrary +pieces of Python code from your application, but exchanging data values +is quite cumbersome to say the least. If you want that, you should use +lower level calls. At the cost of having to write more C code, you can +achieve almost anything. + +It should be noted that extending Python and embedding Python is quite +the same activity, despite the different intent. Most topics discussed +in the previous chapters are still valid. To show this, consider what +the extension code from Python to C really does: + + 1. Convert data values from Python to C, + + 2. Perform a function call to a C routine using the converted values, + and + + 3. Convert the data values from the call from C to Python. + +When embedding Python, the interface code does: + + 1. Convert data values from C to Python, + + 2. Perform a function call to a Python interface routine using the + converted values, and + + 3. Convert the data values from the call from Python to C. + +As you can see, the data conversion steps are simply swapped to +accommodate the different direction of the cross-language transfer. The +only difference is the routine that you call between both data +conversions. When extending, you call a C routine, when embedding, you +call a Python routine. + +This chapter will not discuss how to convert data from Python to C and +vice versa. Also, proper use of references and dealing with errors is +assumed to be understood. Since these aspects do not differ from +extending the interpreter, you can refer to earlier chapters for the +required information. + + +File: python.info, Node: Pure Embedding, Next: Extending Embedded Python, Prev: Beyond Very High Level Embedding An overview, Up: Embedding Python in Another Application + +6.3.1.3 Pure Embedding +...................... + +The first program aims to execute a function in a Python script. Like +in the section about the very high level interface, the Python +interpreter does not directly interact with the application (but that +will change in the next section). + +The code to run a function defined in a Python script is: + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + + int + main(int argc, char *argv[]) + { + PyObject *pName, *pModule, *pFunc; + PyObject *pArgs, *pValue; + int i; + + if (argc < 3) { + fprintf(stderr,"Usage: call pythonfile funcname [args]\n"); + return 1; + } + + Py_Initialize(); + pName = PyUnicode_DecodeFSDefault(argv[1]); + /* Error checking of pName left out */ + + pModule = PyImport_Import(pName); + Py_DECREF(pName); + + if (pModule != NULL) { + pFunc = PyObject_GetAttrString(pModule, argv[2]); + /* pFunc is a new reference */ + + if (pFunc && PyCallable_Check(pFunc)) { + pArgs = PyTuple_New(argc - 3); + for (i = 0; i < argc - 3; ++i) { + pValue = PyLong_FromLong(atoi(argv[i + 3])); + if (!pValue) { + Py_DECREF(pArgs); + Py_DECREF(pModule); + fprintf(stderr, "Cannot convert argument\n"); + return 1; + } + /* pValue reference stolen here: */ + PyTuple_SetItem(pArgs, i, pValue); + } + pValue = PyObject_CallObject(pFunc, pArgs); + Py_DECREF(pArgs); + if (pValue != NULL) { + printf("Result of call: %ld\n", PyLong_AsLong(pValue)); + Py_DECREF(pValue); + } + else { + Py_DECREF(pFunc); + Py_DECREF(pModule); + PyErr_Print(); + fprintf(stderr,"Call failed\n"); + return 1; + } + } + else { + if (PyErr_Occurred()) + PyErr_Print(); + fprintf(stderr, "Cannot find function \"%s\"\n", argv[2]); + } + Py_XDECREF(pFunc); + Py_DECREF(pModule); + } + else { + PyErr_Print(); + fprintf(stderr, "Failed to load \"%s\"\n", argv[1]); + return 1; + } + if (Py_FinalizeEx() < 0) { + return 120; + } + return 0; + } + +This code loads a Python script using ‘argv[1]’, and calls the function +named in ‘argv[2]’. Its integer arguments are the other values of the +‘argv’ array. If you *note compile and link: 38d0. this program (let’s +call the finished executable ‘call’), and use it to execute a Python +script, such as: + + def multiply(a,b): + print("Will compute", a, "times", b) + c = 0 + for i in range(0, a): + c = c + b + return c + +then the result should be: + + $ call multiply multiply 3 2 + Will compute 3 times 2 + Result of call: 6 + +Although the program is quite large for its functionality, most of the +code is for data conversion between Python and C, and for error +reporting. The interesting part with respect to embedding Python starts +with + + Py_Initialize(); + pName = PyUnicode_DecodeFSDefault(argv[1]); + /* Error checking of pName left out */ + pModule = PyImport_Import(pName); + +After initializing the interpreter, the script is loaded using *note +PyImport_Import(): d5f. This routine needs a Python string as its +argument, which is constructed using the *note PyUnicode_FromString(): +38d1. data conversion routine. + + pFunc = PyObject_GetAttrString(pModule, argv[2]); + /* pFunc is a new reference */ + + if (pFunc && PyCallable_Check(pFunc)) { + ... + } + Py_XDECREF(pFunc); + +Once the script is loaded, the name we’re looking for is retrieved using +*note PyObject_GetAttrString(): 385b. If the name exists, and the +object returned is callable, you can safely assume that it is a +function. The program then proceeds by constructing a tuple of +arguments as normal. The call to the Python function is then made with: + + pValue = PyObject_CallObject(pFunc, pArgs); + +Upon return of the function, ‘pValue’ is either ‘NULL’ or it contains a +reference to the return value of the function. Be sure to release the +reference after examining the value. + + +File: python.info, Node: Extending Embedded Python, Next: Embedding Python in C++, Prev: Pure Embedding, Up: Embedding Python in Another Application + +6.3.1.4 Extending Embedded Python +................................. + +Until now, the embedded Python interpreter had no access to +functionality from the application itself. The Python API allows this +by extending the embedded interpreter. That is, the embedded +interpreter gets extended with routines provided by the application. +While it sounds complex, it is not so bad. Simply forget for a while +that the application starts the Python interpreter. Instead, consider +the application to be a set of subroutines, and write some glue code +that gives Python access to those routines, just like you would write a +normal Python extension. For example: + + static int numargs=0; + + /* Return the number of arguments of the application command line */ + static PyObject* + emb_numargs(PyObject *self, PyObject *args) + { + if(!PyArg_ParseTuple(args, ":numargs")) + return NULL; + return PyLong_FromLong(numargs); + } + + static PyMethodDef EmbMethods[] = { + {"numargs", emb_numargs, METH_VARARGS, + "Return the number of arguments received by the process."}, + {NULL, NULL, 0, NULL} + }; + + static PyModuleDef EmbModule = { + PyModuleDef_HEAD_INIT, "emb", NULL, -1, EmbMethods, + NULL, NULL, NULL, NULL + }; + + static PyObject* + PyInit_emb(void) + { + return PyModule_Create(&EmbModule); + } + +Insert the above code just above the ‘main()’ function. Also, insert +the following two statements before the call to *note Py_Initialize(): +439.: + + numargs = argc; + PyImport_AppendInittab("emb", &PyInit_emb); + +These two lines initialize the ‘numargs’ variable, and make the +‘emb.numargs()’ function accessible to the embedded Python interpreter. +With these extensions, the Python script can do things like + + import emb + print("Number of arguments", emb.numargs()) + +In a real application, the methods will expose an API of the application +to Python. + + +File: python.info, Node: Embedding Python in C++, Next: Compiling and Linking under Unix-like systems, Prev: Extending Embedded Python, Up: Embedding Python in Another Application + +6.3.1.5 Embedding Python in C++ +............................... + +It is also possible to embed Python in a C++ program; precisely how this +is done will depend on the details of the C++ system used; in general +you will need to write the main program in C++, and use the C++ compiler +to compile and link your program. There is no need to recompile Python +itself using C++. + + +File: python.info, Node: Compiling and Linking under Unix-like systems, Prev: Embedding Python in C++, Up: Embedding Python in Another Application + +6.3.1.6 Compiling and Linking under Unix-like systems +..................................................... + +It is not necessarily trivial to find the right flags to pass to your +compiler (and linker) in order to embed the Python interpreter into your +application, particularly because Python needs to load library modules +implemented as C dynamic extensions (‘.so’ files) linked against it. + +To find out the required compiler and linker flags, you can execute the +‘python`X.Y'-config’ script which is generated as part of the +installation process (a ‘python3-config’ script may also be available). +This script has several options, of which the following will be directly +useful to you: + + * ‘pythonX.Y-config --cflags’ will give you the recommended flags + when compiling: + + $ /opt/bin/python3.4-config --cflags + -I/opt/include/python3.4m -I/opt/include/python3.4m -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes + + * ‘pythonX.Y-config --ldflags’ will give you the recommended flags + when linking: + + $ /opt/bin/python3.4-config --ldflags + -L/opt/lib/python3.4/config-3.4m -lpthread -ldl -lutil -lm -lpython3.4m -Xlinker -export-dynamic + + Note: To avoid confusion between several Python installations (and + especially between the system Python and your own compiled Python), + it is recommended that you use the absolute path to + ‘python`X.Y'-config’, as in the above example. + +If this procedure doesn’t work for you (it is not guaranteed to work for +all Unix-like platforms; however, we welcome *note bug reports: 38d7.) +you will have to read your system’s documentation about dynamic linking +and/or examine Python’s ‘Makefile’ (use *note +sysconfig.get_makefile_filename(): 336f. to find its location) and +compilation options. In this case, the *note sysconfig: fe. module is a +useful tool to programmatically extract the configuration values that +you will want to combine together. For example: + + >>> import sysconfig + >>> sysconfig.get_config_var('LIBS') + '-lpthread -ldl -lutil' + >>> sysconfig.get_config_var('LINKFORSHARED') + '-Xlinker -export-dynamic' + + +File: python.info, Node: Python/C API Reference Manual, Next: Distributing Python Modules, Prev: Extending and Embedding the Python Interpreter, Up: Top + +7 Python/C API Reference Manual +******************************* + +This manual documents the API used by C and C++ programmers who want to +write extension modules or embed Python. It is a companion to *note +Extending and Embedding the Python Interpreter: e90, which describes the +general principles of extension writing but does not document the API +functions in detail. + +* Menu: + +* Introduction: Introduction<13>. +* Stable Application Binary Interface:: +* The Very High Level Layer:: +* Reference Counting:: +* Exception Handling:: +* Utilities: Utilities<2>. +* Abstract Objects Layer:: +* Concrete Objects Layer:: +* Initialization, Finalization, and Threads: Initialization Finalization and Threads. +* Python Initialization Configuration:: +* Memory Management:: +* Object Implementation Support:: +* API and ABI Versioning:: + + +File: python.info, Node: Introduction<13>, Next: Stable Application Binary Interface, Up: Python/C API Reference Manual + +7.1 Introduction +================ + +The Application Programmer’s Interface to Python gives C and C++ +programmers access to the Python interpreter at a variety of levels. +The API is equally usable from C++, but for brevity it is generally +referred to as the Python/C API. There are two fundamentally different +reasons for using the Python/C API. The first reason is to write +`extension modules' for specific purposes; these are C modules that +extend the Python interpreter. This is probably the most common use. +The second reason is to use Python as a component in a larger +application; this technique is generally referred to as `embedding' +Python in an application. + +Writing an extension module is a relatively well-understood process, +where a “cookbook” approach works well. There are several tools that +automate the process to some extent. While people have embedded Python +in other applications since its early existence, the process of +embedding Python is less straightforward than writing an extension. + +Many API functions are useful independent of whether you’re embedding or +extending Python; moreover, most applications that embed Python will +need to provide a custom extension as well, so it’s probably a good idea +to become familiar with writing an extension before attempting to embed +Python in a real application. + +* Menu: + +* Coding standards:: +* Include Files:: +* Useful macros:: +* Objects, Types and Reference Counts: Objects Types and Reference Counts. +* Exceptions: Exceptions<18>. +* Embedding Python: Embedding Python<2>. +* Debugging Builds:: + + +File: python.info, Node: Coding standards, Next: Include Files, Up: Introduction<13> + +7.1.1 Coding standards +---------------------- + +If you’re writing C code for inclusion in CPython, you `must' follow the +guidelines and standards defined in PEP 7(1). These guidelines apply +regardless of the version of Python you are contributing to. Following +these conventions is not necessary for your own third party extension +modules, unless you eventually expect to contribute them to Python. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0007 + + +File: python.info, Node: Include Files, Next: Useful macros, Prev: Coding standards, Up: Introduction<13> + +7.1.2 Include Files +------------------- + +All function, type and macro definitions needed to use the Python/C API +are included in your code by the following line: + + #define PY_SSIZE_T_CLEAN + #include <Python.h> + +This implies inclusion of the following standard headers: ‘<stdio.h>’, +‘<string.h>’, ‘<errno.h>’, ‘<limits.h>’, ‘<assert.h>’ and ‘<stdlib.h>’ +(if available). + + Note: Since Python may define some pre-processor definitions which + affect the standard headers on some systems, you `must' include + ‘Python.h’ before any standard headers are included. + + It is recommended to always define ‘PY_SSIZE_T_CLEAN’ before + including ‘Python.h’. See *note Parsing arguments and building + values: 2d0. for a description of this macro. + +All user visible names defined by Python.h (except those defined by the +included standard headers) have one of the prefixes ‘Py’ or ‘_Py’. +Names beginning with ‘_Py’ are for internal use by the Python +implementation and should not be used by extension writers. Structure +member names do not have a reserved prefix. + + Note: User code should never define names that begin with ‘Py’ or + ‘_Py’. This confuses the reader, and jeopardizes the portability + of the user code to future Python versions, which may define + additional names beginning with one of these prefixes. + +The header files are typically installed with Python. On Unix, these +are located in the directories ‘`prefix'/include/pythonversion/’ and +‘`exec_prefix'/include/pythonversion/’, where ‘prefix’ and ‘exec_prefix’ +are defined by the corresponding parameters to Python’s ‘configure’ +script and `version' is ‘'%d.%d' % sys.version_info[:2]’. On Windows, +the headers are installed in ‘`prefix'/include’, where ‘prefix’ is the +installation directory specified to the installer. + +To include the headers, place both directories (if different) on your +compiler’s search path for includes. Do `not' place the parent +directories on the search path and then use ‘#include +<pythonX.Y/Python.h>’; this will break on multi-platform builds since +the platform independent headers under ‘prefix’ include the platform +specific headers from ‘exec_prefix’. + +C++ users should note that although the API is defined entirely using C, +the header files properly declare the entry points to be ‘extern "C"’. +As a result, there is no need to do anything special to use the API from +C++. + + +File: python.info, Node: Useful macros, Next: Objects Types and Reference Counts, Prev: Include Files, Up: Introduction<13> + +7.1.3 Useful macros +------------------- + +Several useful macros are defined in the Python header files. Many are +defined closer to where they are useful (e.g. *note Py_RETURN_NONE: +dd6.). Others of a more general utility are defined here. This is not +necessarily a complete listing. + + -- C Macro: Py_UNREACHABLE () + + Use this when you have a code path that you do not expect to be + reached. For example, in the ‘default:’ clause in a ‘switch’ + statement for which all possible values are covered in ‘case’ + statements. Use this in places where you might be tempted to put + an ‘assert(0)’ or ‘abort()’ call. + + New in version 3.7. + + -- C Macro: Py_ABS (x) + + Return the absolute value of ‘x’. + + New in version 3.3. + + -- C Macro: Py_MIN (x, y) + + Return the minimum value between ‘x’ and ‘y’. + + New in version 3.3. + + -- C Macro: Py_MAX (x, y) + + Return the maximum value between ‘x’ and ‘y’. + + New in version 3.3. + + -- C Macro: Py_STRINGIFY (x) + + Convert ‘x’ to a C string. E.g. ‘Py_STRINGIFY(123)’ returns + ‘"123"’. + + New in version 3.4. + + -- C Macro: Py_MEMBER_SIZE (type, member) + + Return the size of a structure (‘type’) ‘member’ in bytes. + + New in version 3.6. + + -- C Macro: Py_CHARMASK (c) + + Argument must be a character or an integer in the range [-128, 127] + or [0, 255]. This macro returns ‘c’ cast to an ‘unsigned char’. + + -- C Macro: Py_GETENV (s) + + Like ‘getenv(s)’, but returns ‘NULL’ if *note -E: fdc. was passed + on the command line (i.e. if ‘Py_IgnoreEnvironmentFlag’ is set). + + -- C Macro: Py_UNUSED (arg) + + Use this for unused arguments in a function definition to silence + compiler warnings. Example: ‘int func(int a, int Py_UNUSED(b)) { + return a; }’. + + New in version 3.4. + + -- C Macro: Py_DEPRECATED (version) + + Use this for deprecated declarations. The macro must be placed + before the symbol name. + + Example: + + Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void); + + Changed in version 3.8: MSVC support was added. + + -- C Macro: PyDoc_STRVAR (name, str) + + Creates a variable with name ‘name’ that can be used in docstrings. + If Python is built without docstrings, the value will be empty. + + Use *note PyDoc_STRVAR: 38ea. for docstrings to support building + Python without docstrings, as specified in PEP 7(1). + + Example: + + PyDoc_STRVAR(pop_doc, "Remove and return the rightmost element."); + + static PyMethodDef deque_methods[] = { + // ... + {"pop", (PyCFunction)deque_pop, METH_NOARGS, pop_doc}, + // ... + } + + -- C Macro: PyDoc_STR (str) + + Creates a docstring for the given input string or an empty string + if docstrings are disabled. + + Use *note PyDoc_STR: 38eb. in specifying docstrings to support + building Python without docstrings, as specified in PEP 7(2). + + Example: + + static PyMethodDef pysqlite_row_methods[] = { + {"keys", (PyCFunction)pysqlite_row_keys, METH_NOARGS, + PyDoc_STR("Returns the keys of the row.")}, + {NULL, NULL} + }; + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0007 + + (2) https://www.python.org/dev/peps/pep-0007 + + +File: python.info, Node: Objects Types and Reference Counts, Next: Exceptions<18>, Prev: Useful macros, Up: Introduction<13> + +7.1.4 Objects, Types and Reference Counts +----------------------------------------- + +Most Python/C API functions have one or more arguments as well as a +return value of type *note PyObject*: 4ba. This type is a pointer to an +opaque data type representing an arbitrary Python object. Since all +Python object types are treated the same way by the Python language in +most situations (e.g., assignments, scope rules, and argument passing), +it is only fitting that they should be represented by a single C type. +Almost all Python objects live on the heap: you never declare an +automatic or static variable of type *note PyObject: 4ba, only pointer +variables of type *note PyObject*: 4ba. can be declared. The sole +exception are the type objects; since these must never be deallocated, +they are typically static *note PyTypeObject: 2d6. objects. + +All Python objects (even Python integers) have a `type' and a `reference +count'. An object’s type determines what kind of object it is (e.g., an +integer, a list, or a user-defined function; there are many more as +explained in *note The standard type hierarchy: 10c0.). For each of the +well-known types there is a macro to check whether an object is of that +type; for instance, ‘PyList_Check(a)’ is true if (and only if) the +object pointed to by `a' is a Python list. + +* Menu: + +* Reference Counts: Reference Counts<2>. +* Types:: + + +File: python.info, Node: Reference Counts<2>, Next: Types, Up: Objects Types and Reference Counts + +7.1.4.1 Reference Counts +........................ + +The reference count is important because today’s computers have a finite +(and often severely limited) memory size; it counts how many different +places there are that have a reference to an object. Such a place could +be another object, or a global (or static) C variable, or a local +variable in some C function. When an object’s reference count becomes +zero, the object is deallocated. If it contains references to other +objects, their reference count is decremented. Those other objects may +be deallocated in turn, if this decrement makes their reference count +become zero, and so on. (There’s an obvious problem with objects that +reference each other here; for now, the solution is “don’t do that.”) + +Reference counts are always manipulated explicitly. The normal way is +to use the macro *note Py_INCREF(): 265. to increment an object’s +reference count by one, and *note Py_DECREF(): 266. to decrement it by +one. The *note Py_DECREF(): 266. macro is considerably more complex +than the incref one, since it must check whether the reference count +becomes zero and then cause the object’s deallocator to be called. The +deallocator is a function pointer contained in the object’s type +structure. The type-specific deallocator takes care of decrementing the +reference counts for other objects contained in the object if this is a +compound object type, such as a list, as well as performing any +additional finalization that’s needed. There’s no chance that the +reference count can overflow; at least as many bits are used to hold the +reference count as there are distinct memory locations in virtual memory +(assuming ‘sizeof(Py_ssize_t) >= sizeof(void*)’). Thus, the reference +count increment is a simple operation. + +It is not necessary to increment an object’s reference count for every +local variable that contains a pointer to an object. In theory, the +object’s reference count goes up by one when the variable is made to +point to it and it goes down by one when the variable goes out of scope. +However, these two cancel each other out, so at the end the reference +count hasn’t changed. The only real reason to use the reference count +is to prevent the object from being deallocated as long as our variable +is pointing to it. If we know that there is at least one other +reference to the object that lives at least as long as our variable, +there is no need to increment the reference count temporarily. An +important situation where this arises is in objects that are passed as +arguments to C functions in an extension module that are called from +Python; the call mechanism guarantees to hold a reference to every +argument for the duration of the call. + +However, a common pitfall is to extract an object from a list and hold +on to it for a while without incrementing its reference count. Some +other operation might conceivably remove the object from the list, +decrementing its reference count and possibly deallocating it. The real +danger is that innocent-looking operations may invoke arbitrary Python +code which could do this; there is a code path which allows control to +flow back to the user from a *note Py_DECREF(): 266, so almost any +operation is potentially dangerous. + +A safe approach is to always use the generic operations (functions whose +name begins with ‘PyObject_’, ‘PyNumber_’, ‘PySequence_’ or +‘PyMapping_’). These operations always increment the reference count of +the object they return. This leaves the caller with the responsibility +to call *note Py_DECREF(): 266. when they are done with the result; this +soon becomes second nature. + +* Menu: + +* Reference Count Details:: + + +File: python.info, Node: Reference Count Details, Up: Reference Counts<2> + +7.1.4.2 Reference Count Details +............................... + +The reference count behavior of functions in the Python/C API is best +explained in terms of `ownership of references'. Ownership pertains to +references, never to objects (objects are not owned: they are always +shared). “Owning a reference” means being responsible for calling +Py_DECREF on it when the reference is no longer needed. Ownership can +also be transferred, meaning that the code that receives ownership of +the reference then becomes responsible for eventually decref’ing it by +calling *note Py_DECREF(): 266. or *note Py_XDECREF(): 268. when it’s no +longer needed—or passing on this responsibility (usually to its caller). +When a function passes ownership of a reference on to its caller, the +caller is said to receive a `new' reference. When no ownership is +transferred, the caller is said to `borrow' the reference. Nothing +needs to be done for a borrowed reference. + +Conversely, when a calling function passes in a reference to an object, +there are two possibilities: the function `steals' a reference to the +object, or it does not. `Stealing a reference' means that when you pass +a reference to a function, that function assumes that it now owns that +reference, and you are not responsible for it any longer. + +Few functions steal references; the two notable exceptions are *note +PyList_SetItem(): 3862. and *note PyTuple_SetItem(): 3861, which steal a +reference to the item (but not to the tuple or list into which the item +is put!). These functions were designed to steal a reference because of +a common idiom for populating a tuple or list with newly created +objects; for example, the code to create the tuple ‘(1, 2, "three")’ +could look like this (forgetting about error handling for the moment; a +better way to code this is shown below): + + PyObject *t; + + t = PyTuple_New(3); + PyTuple_SetItem(t, 0, PyLong_FromLong(1L)); + PyTuple_SetItem(t, 1, PyLong_FromLong(2L)); + PyTuple_SetItem(t, 2, PyUnicode_FromString("three")); + +Here, *note PyLong_FromLong(): 3841. returns a new reference which is +immediately stolen by *note PyTuple_SetItem(): 3861. When you want to +keep using an object although the reference to it will be stolen, use +*note Py_INCREF(): 265. to grab another reference before calling the +reference-stealing function. + +Incidentally, *note PyTuple_SetItem(): 3861. is the `only' way to set +tuple items; *note PySequence_SetItem(): 38f2. and *note +PyObject_SetItem(): 38f3. refuse to do this since tuples are an +immutable data type. You should only use *note PyTuple_SetItem(): 3861. +for tuples that you are creating yourself. + +Equivalent code for populating a list can be written using *note +PyList_New(): 38f4. and *note PyList_SetItem(): 3862. + +However, in practice, you will rarely use these ways of creating and +populating a tuple or list. There’s a generic function, *note +Py_BuildValue(): 2ce, that can create most common objects from C values, +directed by a `format string'. For example, the above two blocks of +code could be replaced by the following (which also takes care of the +error checking): + + PyObject *tuple, *list; + + tuple = Py_BuildValue("(iis)", 1, 2, "three"); + list = Py_BuildValue("[iis]", 1, 2, "three"); + +It is much more common to use *note PyObject_SetItem(): 38f3. and +friends with items whose references you are only borrowing, like +arguments that were passed in to the function you are writing. In that +case, their behaviour regarding reference counts is much saner, since +you don’t have to increment a reference count so you can give a +reference away (“have it be stolen”). For example, this function sets +all items of a list (actually, any mutable sequence) to a given item: + + int + set_all(PyObject *target, PyObject *item) + { + Py_ssize_t i, n; + + n = PyObject_Length(target); + if (n < 0) + return -1; + for (i = 0; i < n; i++) { + PyObject *index = PyLong_FromSsize_t(i); + if (!index) + return -1; + if (PyObject_SetItem(target, index, item) < 0) { + Py_DECREF(index); + return -1; + } + Py_DECREF(index); + } + return 0; + } + +The situation is slightly different for function return values. While +passing a reference to most functions does not change your ownership +responsibilities for that reference, many functions that return a +reference to an object give you ownership of the reference. The reason +is simple: in many cases, the returned object is created on the fly, and +the reference you get is the only reference to the object. Therefore, +the generic functions that return object references, like *note +PyObject_GetItem(): 38f5. and *note PySequence_GetItem(): 38f6, always +return a new reference (the caller becomes the owner of the reference). + +It is important to realize that whether you own a reference returned by +a function depends on which function you call only — `the plumage' (the +type of the object passed as an argument to the function) `doesn’t enter +into it!' Thus, if you extract an item from a list using *note +PyList_GetItem(): 385d, you don’t own the reference — but if you obtain +the same item from the same list using *note PySequence_GetItem(): 38f6. +(which happens to take exactly the same arguments), you do own a +reference to the returned object. + +Here is an example of how you could write a function that computes the +sum of the items in a list of integers; once using *note +PyList_GetItem(): 385d, and once using *note PySequence_GetItem(): 38f6. + + long + sum_list(PyObject *list) + { + Py_ssize_t i, n; + long total = 0, value; + PyObject *item; + + n = PyList_Size(list); + if (n < 0) + return -1; /* Not a list */ + for (i = 0; i < n; i++) { + item = PyList_GetItem(list, i); /* Can't fail */ + if (!PyLong_Check(item)) continue; /* Skip non-integers */ + value = PyLong_AsLong(item); + if (value == -1 && PyErr_Occurred()) + /* Integer too big to fit in a C long, bail out */ + return -1; + total += value; + } + return total; + } + + long + sum_sequence(PyObject *sequence) + { + Py_ssize_t i, n; + long total = 0, value; + PyObject *item; + n = PySequence_Length(sequence); + if (n < 0) + return -1; /* Has no length */ + for (i = 0; i < n; i++) { + item = PySequence_GetItem(sequence, i); + if (item == NULL) + return -1; /* Not a sequence, or other failure */ + if (PyLong_Check(item)) { + value = PyLong_AsLong(item); + Py_DECREF(item); + if (value == -1 && PyErr_Occurred()) + /* Integer too big to fit in a C long, bail out */ + return -1; + total += value; + } + else { + Py_DECREF(item); /* Discard reference ownership */ + } + } + return total; + } + + +File: python.info, Node: Types, Prev: Reference Counts<2>, Up: Objects Types and Reference Counts + +7.1.4.3 Types +............. + +There are few other data types that play a significant role in the +Python/C API; most are simple C types such as ‘int’, ‘long’, ‘double’ +and ‘char*’. A few structure types are used to describe static tables +used to list the functions exported by a module or the data attributes +of a new object type, and another is used to describe the value of a +complex number. These will be discussed together with the functions +that use them. + + +File: python.info, Node: Exceptions<18>, Next: Embedding Python<2>, Prev: Objects Types and Reference Counts, Up: Introduction<13> + +7.1.5 Exceptions +---------------- + +The Python programmer only needs to deal with exceptions if specific +error handling is required; unhandled exceptions are automatically +propagated to the caller, then to the caller’s caller, and so on, until +they reach the top-level interpreter, where they are reported to the +user accompanied by a stack traceback. + +For C programmers, however, error checking always has to be explicit. +All functions in the Python/C API can raise exceptions, unless an +explicit claim is made otherwise in a function’s documentation. In +general, when a function encounters an error, it sets an exception, +discards any object references that it owns, and returns an error +indicator. If not documented otherwise, this indicator is either ‘NULL’ +or ‘-1’, depending on the function’s return type. A few functions +return a Boolean true/false result, with false indicating an error. +Very few functions return no explicit error indicator or have an +ambiguous return value, and require explicit testing for errors with +*note PyErr_Occurred(): 383f. These exceptions are always explicitly +documented. + +Exception state is maintained in per-thread storage (this is equivalent +to using global storage in an unthreaded application). A thread can be +in one of two states: an exception has occurred, or not. The function +*note PyErr_Occurred(): 383f. can be used to check for this: it returns +a borrowed reference to the exception type object when an exception has +occurred, and ‘NULL’ otherwise. There are a number of functions to set +the exception state: *note PyErr_SetString(): 383c. is the most common +(though not the most general) function to set the exception state, and +*note PyErr_Clear(): 96b. clears the exception state. + +The full exception state consists of three objects (all of which can be +‘NULL’): the exception type, the corresponding exception value, and the +traceback. These have the same meanings as the Python result of +‘sys.exc_info()’; however, they are not the same: the Python objects +represent the last exception being handled by a Python *note try: d72. … +*note except: b3e. statement, while the C level exception state only +exists while an exception is being passed on between C functions until +it reaches the Python bytecode interpreter’s main loop, which takes care +of transferring it to ‘sys.exc_info()’ and friends. + +Note that starting with Python 1.5, the preferred, thread-safe way to +access the exception state from Python code is to call the function +*note sys.exc_info(): c5f, which returns the per-thread exception state +for Python code. Also, the semantics of both ways to access the +exception state have changed so that a function which catches an +exception will save and restore its thread’s exception state so as to +preserve the exception state of its caller. This prevents common bugs +in exception handling code caused by an innocent-looking function +overwriting the exception being handled; it also reduces the often +unwanted lifetime extension for objects that are referenced by the stack +frames in the traceback. + +As a general principle, a function that calls another function to +perform some task should check whether the called function raised an +exception, and if so, pass the exception state on to its caller. It +should discard any object references that it owns, and return an error +indicator, but it should `not' set another exception — that would +overwrite the exception that was just raised, and lose important +information about the exact cause of the error. + +A simple example of detecting exceptions and passing them on is shown in +the ‘sum_sequence()’ example above. It so happens that this example +doesn’t need to clean up any owned references when it detects an error. +The following example function shows some error cleanup. First, to +remind you why you like Python, we show the equivalent Python code: + + def incr_item(dict, key): + try: + item = dict[key] + except KeyError: + item = 0 + dict[key] = item + 1 + +Here is the corresponding C code, in all its glory: + + int + incr_item(PyObject *dict, PyObject *key) + { + /* Objects all initialized to NULL for Py_XDECREF */ + PyObject *item = NULL, *const_one = NULL, *incremented_item = NULL; + int rv = -1; /* Return value initialized to -1 (failure) */ + + item = PyObject_GetItem(dict, key); + if (item == NULL) { + /* Handle KeyError only: */ + if (!PyErr_ExceptionMatches(PyExc_KeyError)) + goto error; + + /* Clear the error and use zero: */ + PyErr_Clear(); + item = PyLong_FromLong(0L); + if (item == NULL) + goto error; + } + const_one = PyLong_FromLong(1L); + if (const_one == NULL) + goto error; + + incremented_item = PyNumber_Add(item, const_one); + if (incremented_item == NULL) + goto error; + + if (PyObject_SetItem(dict, key, incremented_item) < 0) + goto error; + rv = 0; /* Success */ + /* Continue with cleanup code */ + + error: + /* Cleanup code, shared by success and failure path */ + + /* Use Py_XDECREF() to ignore NULL references */ + Py_XDECREF(item); + Py_XDECREF(const_one); + Py_XDECREF(incremented_item); + + return rv; /* -1 for error, 0 for success */ + } + +This example represents an endorsed use of the ‘goto’ statement in C! It +illustrates the use of *note PyErr_ExceptionMatches(): 38fb. and *note +PyErr_Clear(): 96b. to handle specific exceptions, and the use of *note +Py_XDECREF(): 268. to dispose of owned references that may be ‘NULL’ +(note the ‘'X'’ in the name; *note Py_DECREF(): 266. would crash when +confronted with a ‘NULL’ reference). It is important that the variables +used to hold owned references are initialized to ‘NULL’ for this to +work; likewise, the proposed return value is initialized to ‘-1’ +(failure) and only set to success after the final call made is +successful. + + +File: python.info, Node: Embedding Python<2>, Next: Debugging Builds, Prev: Exceptions<18>, Up: Introduction<13> + +7.1.6 Embedding Python +---------------------- + +The one important task that only embedders (as opposed to extension +writers) of the Python interpreter have to worry about is the +initialization, and possibly the finalization, of the Python +interpreter. Most functionality of the interpreter can only be used +after the interpreter has been initialized. + +The basic initialization function is *note Py_Initialize(): 439. This +initializes the table of loaded modules, and creates the fundamental +modules *note builtins: 13, *note __main__: 1, and *note sys: fd. It +also initializes the module search path (‘sys.path’). + +*note Py_Initialize(): 439. does not set the “script argument list” +(‘sys.argv’). If this variable is needed by Python code that will be +executed later, it must be set explicitly with a call to +‘PySys_SetArgvEx(argc, argv, updatepath)’ after the call to *note +Py_Initialize(): 439. + +On most systems (in particular, on Unix and Windows, although the +details are slightly different), *note Py_Initialize(): 439. calculates +the module search path based upon its best guess for the location of the +standard Python interpreter executable, assuming that the Python library +is found in a fixed location relative to the Python interpreter +executable. In particular, it looks for a directory named +‘lib/python`X.Y'’ relative to the parent directory where the executable +named ‘python’ is found on the shell command search path (the +environment variable ‘PATH’). + +For instance, if the Python executable is found in +‘/usr/local/bin/python’, it will assume that the libraries are in +‘/usr/local/lib/python`X.Y'’. (In fact, this particular path is also +the “fallback” location, used when no executable file named ‘python’ is +found along ‘PATH’.) The user can override this behavior by setting the +environment variable *note PYTHONHOME: 4d6, or insert additional +directories in front of the standard path by setting *note PYTHONPATH: +953. + +The embedding application can steer the search by calling +‘Py_SetProgramName(file)’ `before' calling *note Py_Initialize(): 439. +Note that *note PYTHONHOME: 4d6. still overrides this and *note +PYTHONPATH: 953. is still inserted in front of the standard path. An +application that requires total control has to provide its own +implementation of *note Py_GetPath(): 38fe, *note Py_GetPrefix(): 38ff, +*note Py_GetExecPrefix(): 3900, and *note Py_GetProgramFullPath(): 275. +(all defined in ‘Modules/getpath.c’). + +Sometimes, it is desirable to “uninitialize” Python. For instance, the +application may want to start over (make another call to *note +Py_Initialize(): 439.) or the application is simply done with its use of +Python and wants to free memory allocated by Python. This can be +accomplished by calling *note Py_FinalizeEx(): 5c9. The function *note +Py_IsInitialized(): 3901. returns true if Python is currently in the +initialized state. More information about these functions is given in a +later chapter. Notice that *note Py_FinalizeEx(): 5c9. does `not' free +all memory allocated by the Python interpreter, e.g. memory allocated +by extension modules currently cannot be released. + + +File: python.info, Node: Debugging Builds, Prev: Embedding Python<2>, Up: Introduction<13> + +7.1.7 Debugging Builds +---------------------- + +Python can be built with several macros to enable extra checks of the +interpreter and extension modules. These checks tend to add a large +amount of overhead to the runtime so they are not enabled by default. + +A full list of the various types of debugging builds is in the file +‘Misc/SpecialBuilds.txt’ in the Python source distribution. Builds are +available that support tracing of reference counts, debugging the memory +allocator, or low-level profiling of the main interpreter loop. Only +the most frequently-used builds will be described in the remainder of +this section. + +Compiling the interpreter with the ‘Py_DEBUG’ macro defined produces +what is generally meant by “a debug build” of Python. ‘Py_DEBUG’ is +enabled in the Unix build by adding ‘--with-pydebug’ to the +‘./configure’ command. It is also implied by the presence of the +not-Python-specific ‘_DEBUG’ macro. When ‘Py_DEBUG’ is enabled in the +Unix build, compiler optimization is disabled. + +In addition to the reference count debugging described below, the +following extra checks are performed: + + * Extra checks are added to the object allocator. + + * Extra checks are added to the parser and compiler. + + * Downcasts from wide types to narrow types are checked for loss of + information. + + * A number of assertions are added to the dictionary and set + implementations. In addition, the set object acquires a + ‘test_c_api()’ method. + + * Sanity checks of the input arguments are added to frame creation. + + * The storage for ints is initialized with a known invalid pattern to + catch reference to uninitialized digits. + + * Low-level tracing and extra exception checking are added to the + runtime virtual machine. + + * Extra checks are added to the memory arena implementation. + + * Extra debugging is added to the thread module. + +There may be additional checks not mentioned here. + +Defining ‘Py_TRACE_REFS’ enables reference tracing. When defined, a +circular doubly linked list of active objects is maintained by adding +two extra fields to every *note PyObject: 4ba. Total allocations are +tracked as well. Upon exit, all existing references are printed. (In +interactive mode this happens after every statement run by the +interpreter.) Implied by ‘Py_DEBUG’. + +Please refer to ‘Misc/SpecialBuilds.txt’ in the Python source +distribution for more detailed information. + + +File: python.info, Node: Stable Application Binary Interface, Next: The Very High Level Layer, Prev: Introduction<13>, Up: Python/C API Reference Manual + +7.2 Stable Application Binary Interface +======================================= + +Traditionally, the C API of Python will change with every release. Most +changes will be source-compatible, typically by only adding API, rather +than changing existing API or removing API (although some interfaces do +get removed after being deprecated first). + +Unfortunately, the API compatibility does not extend to binary +compatibility (the ABI). The reason is primarily the evolution of struct +definitions, where addition of a new field, or changing the type of a +field, might not break the API, but can break the ABI. As a consequence, +extension modules need to be recompiled for every Python release +(although an exception is possible on Unix when none of the affected +interfaces are used). In addition, on Windows, extension modules link +with a specific pythonXY.dll and need to be recompiled to link with a +newer one. + +Since Python 3.2, a subset of the API has been declared to guarantee a +stable ABI. Extension modules wishing to use this API (called “limited +API”) need to define ‘Py_LIMITED_API’. A number of interpreter details +then become hidden from the extension module; in return, a module is +built that works on any 3.x version (x>=2) without recompilation. + +In some cases, the stable ABI needs to be extended with new functions. +Extension modules wishing to use these new APIs need to set +‘Py_LIMITED_API’ to the ‘PY_VERSION_HEX’ value (see *note API and ABI +Versioning: 335e.) of the minimum Python version they want to support +(e.g. ‘0x03030000’ for Python 3.3). Such modules will work on all +subsequent Python releases, but fail to load (because of missing +symbols) on the older releases. + +As of Python 3.2, the set of functions available to the limited API is +documented in PEP 384(1). In the C API documentation, API elements that +are not part of the limited API are marked as “Not part of the limited +API.” + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0384 + + +File: python.info, Node: The Very High Level Layer, Next: Reference Counting, Prev: Stable Application Binary Interface, Up: Python/C API Reference Manual + +7.3 The Very High Level Layer +============================= + +The functions in this chapter will let you execute Python source code +given in a file or a buffer, but they will not let you interact in a +more detailed way with the interpreter. + +Several of these functions accept a start symbol from the grammar as a +parameter. The available start symbols are ‘Py_eval_input’, +‘Py_file_input’, and ‘Py_single_input’. These are described following +the functions which accept them as parameters. + +Note also that several of these functions take ‘FILE*’ parameters. One +particular issue which needs to be handled carefully is that the ‘FILE’ +structure for different C libraries can be different and incompatible. +Under Windows (at least), it is possible for dynamically linked +extensions to actually use different libraries, so care should be taken +that ‘FILE*’ parameters are only passed to these functions if it is +certain that they were created by the same library that the Python +runtime is using. + + -- C Function: int Py_Main (int argc, wchar_t **argv) + + The main program for the standard interpreter. This is made + available for programs which embed Python. The `argc' and `argv' + parameters should be prepared exactly as those which are passed to + a C program’s ‘main()’ function (converted to wchar_t according to + the user’s locale). It is important to note that the argument list + may be modified (but the contents of the strings pointed to by the + argument list are not). The return value will be ‘0’ if the + interpreter exits normally (i.e., without an exception), ‘1’ if the + interpreter exits due to an exception, or ‘2’ if the parameter list + does not represent a valid Python command line. + + Note that if an otherwise unhandled *note SystemExit: 5fc. is + raised, this function will not return ‘1’, but exit the process, as + long as ‘Py_InspectFlag’ is not set. + + -- C Function: int Py_BytesMain (int argc, char **argv) + + Similar to *note Py_Main(): 4b7. but `argv' is an array of bytes + strings. + + New in version 3.8. + + -- C Function: int PyRun_AnyFile (FILE *fp, const char *filename) + + This is a simplified interface to *note PyRun_AnyFileExFlags(): + 390b. below, leaving `closeit' set to ‘0’ and `flags' set to + ‘NULL’. + + -- C Function: int PyRun_AnyFileFlags (FILE *fp, const char *filename, + PyCompilerFlags *flags) + + This is a simplified interface to *note PyRun_AnyFileExFlags(): + 390b. below, leaving the `closeit' argument set to ‘0’. + + -- C Function: int PyRun_AnyFileEx (FILE *fp, const char *filename, + int closeit) + + This is a simplified interface to *note PyRun_AnyFileExFlags(): + 390b. below, leaving the `flags' argument set to ‘NULL’. + + -- C Function: int PyRun_AnyFileExFlags (FILE *fp, const + char *filename, int closeit, PyCompilerFlags *flags) + + If `fp' refers to a file associated with an interactive device + (console or terminal input or Unix pseudo-terminal), return the + value of *note PyRun_InteractiveLoop(): 390e, otherwise return the + result of *note PyRun_SimpleFile(): 38c9. `filename' is decoded + from the filesystem encoding (*note sys.getfilesystemencoding(): + 4f6.). If `filename' is ‘NULL’, this function uses ‘"???"’ as the + filename. + + -- C Function: int PyRun_SimpleString (const char *command) + + This is a simplified interface to *note PyRun_SimpleStringFlags(): + 390f. below, leaving the *note PyCompilerFlags: 2cc.* argument set + to ‘NULL’. + + -- C Function: int PyRun_SimpleStringFlags (const char *command, + PyCompilerFlags *flags) + + Executes the Python source code from `command' in the *note + __main__: 1. module according to the `flags' argument. If *note + __main__: 1. does not already exist, it is created. Returns ‘0’ on + success or ‘-1’ if an exception was raised. If there was an error, + there is no way to get the exception information. For the meaning + of `flags', see below. + + Note that if an otherwise unhandled *note SystemExit: 5fc. is + raised, this function will not return ‘-1’, but exit the process, + as long as ‘Py_InspectFlag’ is not set. + + -- C Function: int PyRun_SimpleFile (FILE *fp, const char *filename) + + This is a simplified interface to *note PyRun_SimpleFileExFlags(): + 3910. below, leaving `closeit' set to ‘0’ and `flags' set to + ‘NULL’. + + -- C Function: int PyRun_SimpleFileEx (FILE *fp, const char *filename, + int closeit) + + This is a simplified interface to *note PyRun_SimpleFileExFlags(): + 3910. below, leaving `flags' set to ‘NULL’. + + -- C Function: int PyRun_SimpleFileExFlags (FILE *fp, const + char *filename, int closeit, PyCompilerFlags *flags) + + Similar to *note PyRun_SimpleStringFlags(): 390f, but the Python + source code is read from `fp' instead of an in-memory string. + `filename' should be the name of the file, it is decoded from the + filesystem encoding (*note sys.getfilesystemencoding(): 4f6.). If + `closeit' is true, the file is closed before + PyRun_SimpleFileExFlags returns. + + Note: On Windows, `fp' should be opened as binary mode (e.g. + ‘fopen(filename, "rb")’. Otherwise, Python may not handle + script file with LF line ending correctly. + + -- C Function: int PyRun_InteractiveOne (FILE *fp, const + char *filename) + + This is a simplified interface to *note + PyRun_InteractiveOneFlags(): 3913. below, leaving `flags' set to + ‘NULL’. + + -- C Function: int PyRun_InteractiveOneFlags (FILE *fp, const + char *filename, PyCompilerFlags *flags) + + Read and execute a single statement from a file associated with an + interactive device according to the `flags' argument. The user + will be prompted using ‘sys.ps1’ and ‘sys.ps2’. `filename' is + decoded from the filesystem encoding (*note + sys.getfilesystemencoding(): 4f6.). + + Returns ‘0’ when the input was executed successfully, ‘-1’ if there + was an exception, or an error code from the ‘errcode.h’ include + file distributed as part of Python if there was a parse error. + (Note that ‘errcode.h’ is not included by ‘Python.h’, so must be + included specifically if needed.) + + -- C Function: int PyRun_InteractiveLoop (FILE *fp, const + char *filename) + + This is a simplified interface to *note + PyRun_InteractiveLoopFlags(): 3914. below, leaving `flags' set to + ‘NULL’. + + -- C Function: int PyRun_InteractiveLoopFlags (FILE *fp, const + char *filename, PyCompilerFlags *flags) + + Read and execute statements from a file associated with an + interactive device until EOF is reached. The user will be prompted + using ‘sys.ps1’ and ‘sys.ps2’. `filename' is decoded from the + filesystem encoding (*note sys.getfilesystemencoding(): 4f6.). + Returns ‘0’ at EOF or a negative number upon failure. + + -- C Variable: int (*PyOS_InputHook) (void) + + Can be set to point to a function with the prototype ‘int + func(void)’. The function will be called when Python’s interpreter + prompt is about to become idle and wait for user input from the + terminal. The return value is ignored. Overriding this hook can + be used to integrate the interpreter’s prompt with other event + loops, as done in the ‘Modules/_tkinter.c’ in the Python source + code. + + -- C Variable: char* (*PyOS_ReadlineFunctionPointer) (FILE *, FILE *, + const char *) + + Can be set to point to a function with the prototype ‘char + *func(FILE *stdin, FILE *stdout, char *prompt)’, overriding the + default function used to read a single line of input at the + interpreter’s prompt. The function is expected to output the + string `prompt' if it’s not ‘NULL’, and then read a line of input + from the provided standard input file, returning the resulting + string. For example, The *note readline: de. module sets this hook + to provide line-editing and tab-completion features. + + The result must be a string allocated by *note PyMem_RawMalloc(): + 96d. or *note PyMem_RawRealloc(): 96e, or ‘NULL’ if an error + occurred. + + Changed in version 3.4: The result must be allocated by *note + PyMem_RawMalloc(): 96d. or *note PyMem_RawRealloc(): 96e, instead + of being allocated by *note PyMem_Malloc(): 505. or *note + PyMem_Realloc(): 96f. + + -- C Function: struct _node* PyParser_SimpleParseString (const + char *str, int start) + + This is a simplified interface to *note + PyParser_SimpleParseStringFlagsFilename(): 3917. below, leaving + `filename' set to ‘NULL’ and `flags' set to ‘0’. + + -- C Function: struct _node* PyParser_SimpleParseStringFlags (const + char *str, int start, int flags) + + This is a simplified interface to *note + PyParser_SimpleParseStringFlagsFilename(): 3917. below, leaving + `filename' set to ‘NULL’. + + -- C Function: struct _node* PyParser_SimpleParseStringFlagsFilename + (const char *str, const char *filename, int start, int flags) + + Parse Python source code from `str' using the start token `start' + according to the `flags' argument. The result can be used to + create a code object which can be evaluated efficiently. This is + useful if a code fragment must be evaluated many times. `filename' + is decoded from the filesystem encoding (*note + sys.getfilesystemencoding(): 4f6.). + + -- C Function: struct _node* PyParser_SimpleParseFile (FILE *fp, const + char *filename, int start) + + This is a simplified interface to *note + PyParser_SimpleParseFileFlags(): 391a. below, leaving `flags' set + to ‘0’. + + -- C Function: struct _node* PyParser_SimpleParseFileFlags (FILE *fp, + const char *filename, int start, int flags) + + Similar to *note PyParser_SimpleParseStringFlagsFilename(): 3917, + but the Python source code is read from `fp' instead of an + in-memory string. + + -- C Function: PyObject* PyRun_String (const char *str, int start, + PyObject *globals, PyObject *locals) + `Return value: New reference.' This is a simplified interface to + *note PyRun_StringFlags(): 391c. below, leaving `flags' set to + ‘NULL’. + + -- C Function: PyObject* PyRun_StringFlags (const char *str, int start, + PyObject *globals, PyObject *locals, PyCompilerFlags *flags) + `Return value: New reference.' Execute Python source code from + `str' in the context specified by the objects `globals' and + `locals' with the compiler flags specified by `flags'. `globals' + must be a dictionary; `locals' can be any object that implements + the mapping protocol. The parameter `start' specifies the start + token that should be used to parse the source code. + + Returns the result of executing the code as a Python object, or + ‘NULL’ if an exception was raised. + + -- C Function: PyObject* PyRun_File (FILE *fp, const char *filename, + int start, PyObject *globals, PyObject *locals) + `Return value: New reference.' This is a simplified interface to + *note PyRun_FileExFlags(): 391e. below, leaving `closeit' set to + ‘0’ and `flags' set to ‘NULL’. + + -- C Function: PyObject* PyRun_FileEx (FILE *fp, const char *filename, + int start, PyObject *globals, PyObject *locals, int closeit) + `Return value: New reference.' This is a simplified interface to + *note PyRun_FileExFlags(): 391e. below, leaving `flags' set to + ‘NULL’. + + -- C Function: PyObject* PyRun_FileFlags (FILE *fp, const + char *filename, int start, PyObject *globals, + PyObject *locals, PyCompilerFlags *flags) + `Return value: New reference.' This is a simplified interface to + *note PyRun_FileExFlags(): 391e. below, leaving `closeit' set to + ‘0’. + + -- C Function: PyObject* PyRun_FileExFlags (FILE *fp, const + char *filename, int start, PyObject *globals, + PyObject *locals, int closeit, PyCompilerFlags *flags) + `Return value: New reference.' Similar to *note + PyRun_StringFlags(): 391c, but the Python source code is read from + `fp' instead of an in-memory string. `filename' should be the name + of the file, it is decoded from the filesystem encoding (*note + sys.getfilesystemencoding(): 4f6.). If `closeit' is true, the file + is closed before *note PyRun_FileExFlags(): 391e. returns. + + -- C Function: PyObject* Py_CompileString (const char *str, const + char *filename, int start) + `Return value: New reference.' This is a simplified interface to + *note Py_CompileStringFlags(): 3922. below, leaving `flags' set to + ‘NULL’. + + -- C Function: PyObject* Py_CompileStringFlags (const char *str, const + char *filename, int start, PyCompilerFlags *flags) + `Return value: New reference.' This is a simplified interface to + *note Py_CompileStringExFlags(): 3923. below, with `optimize' set + to ‘-1’. + + -- C Function: PyObject* Py_CompileStringObject (const char *str, + PyObject *filename, int start, PyCompilerFlags *flags, + int optimize) + `Return value: New reference.' Parse and compile the Python source + code in `str', returning the resulting code object. The start + token is given by `start'; this can be used to constrain the code + which can be compiled and should be ‘Py_eval_input’, + ‘Py_file_input’, or ‘Py_single_input’. The filename specified by + `filename' is used to construct the code object and may appear in + tracebacks or *note SyntaxError: 458. exception messages. This + returns ‘NULL’ if the code cannot be parsed or compiled. + + The integer `optimize' specifies the optimization level of the + compiler; a value of ‘-1’ selects the optimization level of the + interpreter as given by *note -O: 68b. options. Explicit levels + are ‘0’ (no optimization; ‘__debug__’ is true), ‘1’ (asserts are + removed, ‘__debug__’ is false) or ‘2’ (docstrings are removed too). + + New in version 3.4. + + -- C Function: PyObject* Py_CompileStringExFlags (const char *str, + const char *filename, int start, PyCompilerFlags *flags, + int optimize) + `Return value: New reference.' Like *note + Py_CompileStringObject(): 3924, but `filename' is a byte string + decoded from the filesystem encoding (*note os.fsdecode(): 4ee.). + + New in version 3.2. + + -- C Function: PyObject* PyEval_EvalCode (PyObject *co, + PyObject *globals, PyObject *locals) + `Return value: New reference.' This is a simplified interface to + *note PyEval_EvalCodeEx(): 3926, with just the code object, and + global and local variables. The other arguments are set to ‘NULL’. + + -- C Function: PyObject* PyEval_EvalCodeEx (PyObject *co, + PyObject *globals, PyObject *locals, PyObject *const *args, + int argcount, PyObject *const *kws, int kwcount, PyObject + *const *defs, int defcount, PyObject *kwdefs, + PyObject *closure) + `Return value: New reference.' Evaluate a precompiled code object, + given a particular environment for its evaluation. This + environment consists of a dictionary of global variables, a mapping + object of local variables, arrays of arguments, keywords and + defaults, a dictionary of default values for *note keyword-only: + 2ac. arguments and a closure tuple of cells. + + -- C Type: PyFrameObject + + The C structure of the objects used to describe frame objects. The + fields of this type are subject to change at any time. + + -- C Function: PyObject* PyEval_EvalFrame (PyFrameObject *f) + `Return value: New reference.' Evaluate an execution frame. This + is a simplified interface to *note PyEval_EvalFrameEx(): 967, for + backward compatibility. + + -- C Function: PyObject* PyEval_EvalFrameEx (PyFrameObject *f, + int throwflag) + `Return value: New reference.' This is the main, unvarnished + function of Python interpretation. The code object associated with + the execution frame `f' is executed, interpreting bytecode and + executing calls as needed. The additional `throwflag' parameter + can mostly be ignored - if true, then it causes an exception to + immediately be thrown; this is used for the *note throw(): 1134. + methods of generator objects. + + Changed in version 3.4: This function now includes a debug + assertion to help ensure that it does not silently discard an + active exception. + + -- C Function: int PyEval_MergeCompilerFlags (PyCompilerFlags *cf) + + This function changes the flags of the current evaluation frame, + and returns true on success, false on failure. + + -- C Variable: int Py_eval_input + + The start symbol from the Python grammar for isolated expressions; + for use with *note Py_CompileString(): 3921. + + -- C Variable: int Py_file_input + + The start symbol from the Python grammar for sequences of + statements as read from a file or other source; for use with *note + Py_CompileString(): 3921. This is the symbol to use when compiling + arbitrarily long Python source code. + + -- C Variable: int Py_single_input + + The start symbol from the Python grammar for a single statement; + for use with *note Py_CompileString(): 3921. This is the symbol + used for the interactive interpreter loop. + + -- C Type: struct PyCompilerFlags + + This is the structure used to hold compiler flags. In cases where + code is only being compiled, it is passed as ‘int flags’, and in + cases where code is being executed, it is passed as + ‘PyCompilerFlags *flags’. In this case, ‘from __future__ import’ + can modify `flags'. + + Whenever ‘PyCompilerFlags *flags’ is ‘NULL’, ‘cf_flags’ is treated + as equal to ‘0’, and any modification due to ‘from __future__ + import’ is discarded. + + -- C Member: int cf_flags + + Compiler flags. + + -- C Member: int cf_feature_version + + `cf_feature_version' is the minor Python version. It should + be initialized to ‘PY_MINOR_VERSION’. + + The field is ignored by default, it is used if and only if + ‘PyCF_ONLY_AST’ flag is set in `cf_flags'. + + Changed in version 3.8: Added `cf_feature_version' field. + + -- C Variable: int CO_FUTURE_DIVISION + + This bit can be set in `flags' to cause division operator ‘/’ to be + interpreted as “true division” according to PEP 238(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0238 + + +File: python.info, Node: Reference Counting, Next: Exception Handling, Prev: The Very High Level Layer, Up: Python/C API Reference Manual + +7.4 Reference Counting +====================== + +The macros in this section are used for managing reference counts of +Python objects. + + -- C Function: void Py_INCREF (PyObject *o) + + Increment the reference count for object `o'. The object must not + be ‘NULL’; if you aren’t sure that it isn’t ‘NULL’, use *note + Py_XINCREF(): 267. + + -- C Function: void Py_XINCREF (PyObject *o) + + Increment the reference count for object `o'. The object may be + ‘NULL’, in which case the macro has no effect. + + -- C Function: void Py_DECREF (PyObject *o) + + Decrement the reference count for object `o'. The object must not + be ‘NULL’; if you aren’t sure that it isn’t ‘NULL’, use *note + Py_XDECREF(): 268. If the reference count reaches zero, the + object’s type’s deallocation function (which must not be ‘NULL’) is + invoked. + + Warning: The deallocation function can cause arbitrary Python + code to be invoked (e.g. when a class instance with a *note + __del__(): 91f. method is deallocated). While exceptions in + such code are not propagated, the executed code has free + access to all Python global variables. This means that any + object that is reachable from a global variable should be in a + consistent state before *note Py_DECREF(): 266. is invoked. + For example, code to delete an object from a list should copy + a reference to the deleted object in a temporary variable, + update the list data structure, and then call *note + Py_DECREF(): 266. for the temporary variable. + + -- C Function: void Py_XDECREF (PyObject *o) + + Decrement the reference count for object `o'. The object may be + ‘NULL’, in which case the macro has no effect; otherwise the effect + is the same as for *note Py_DECREF(): 266, and the same warning + applies. + + -- C Function: void Py_CLEAR (PyObject *o) + + Decrement the reference count for object `o'. The object may be + ‘NULL’, in which case the macro has no effect; otherwise the effect + is the same as for *note Py_DECREF(): 266, except that the argument + is also set to ‘NULL’. The warning for *note Py_DECREF(): 266. + does not apply with respect to the object passed because the macro + carefully uses a temporary variable and sets the argument to ‘NULL’ + before decrementing its reference count. + + It is a good idea to use this macro whenever decrementing the + reference count of an object that might be traversed during garbage + collection. + +The following functions are for runtime dynamic embedding of Python: +‘Py_IncRef(PyObject *o)’, ‘Py_DecRef(PyObject *o)’. They are simply +exported function versions of *note Py_XINCREF(): 267. and *note +Py_XDECREF(): 268, respectively. + +The following functions or macros are only for use within the +interpreter core: ‘_Py_Dealloc()’, ‘_Py_ForgetReference()’, +‘_Py_NewReference()’, as well as the global variable ‘_Py_RefTotal’. + + +File: python.info, Node: Exception Handling, Next: Utilities<2>, Prev: Reference Counting, Up: Python/C API Reference Manual + +7.5 Exception Handling +====================== + +The functions described in this chapter will let you handle and raise +Python exceptions. It is important to understand some of the basics of +Python exception handling. It works somewhat like the POSIX ‘errno’ +variable: there is a global indicator (per thread) of the last error +that occurred. Most C API functions don’t clear this on success, but +will set it to indicate the cause of the error on failure. Most C API +functions also return an error indicator, usually ‘NULL’ if they are +supposed to return a pointer, or ‘-1’ if they return an integer +(exception: the ‘PyArg_*()’ functions return ‘1’ for success and ‘0’ for +failure). + +Concretely, the error indicator consists of three object pointers: the +exception’s type, the exception’s value, and the traceback object. Any +of those pointers can be ‘NULL’ if non-set (although some combinations +are forbidden, for example you can’t have a non-‘NULL’ traceback if the +exception type is ‘NULL’). + +When a function must fail because some function it called failed, it +generally doesn’t set the error indicator; the function it called +already set it. It is responsible for either handling the error and +clearing the exception or returning after cleaning up any resources it +holds (such as object references or memory allocations); it should `not' +continue normally if it is not prepared to handle the error. If +returning due to an error, it is important to indicate to the caller +that an error has been set. If the error is not handled or carefully +propagated, additional calls into the Python/C API may not behave as +intended and may fail in mysterious ways. + + Note: The error indicator is `not' the result of *note + sys.exc_info(): c5f. The former corresponds to an exception that + is not yet caught (and is therefore still propagating), while the + latter returns an exception after it is caught (and has therefore + stopped propagating). + +* Menu: + +* Printing and clearing:: +* Raising exceptions:: +* Issuing warnings:: +* Querying the error indicator:: +* Signal Handling: Signal Handling<2>. +* Exception Classes:: +* Exception Objects:: +* Unicode Exception Objects:: +* Recursion Control:: +* Standard Exceptions:: +* Standard Warning Categories:: + + +File: python.info, Node: Printing and clearing, Next: Raising exceptions, Up: Exception Handling + +7.5.1 Printing and clearing +--------------------------- + + -- C Function: void PyErr_Clear () + + Clear the error indicator. If the error indicator is not set, + there is no effect. + + -- C Function: void PyErr_PrintEx (int set_sys_last_vars) + + Print a standard traceback to ‘sys.stderr’ and clear the error + indicator. `Unless' the error is a ‘SystemExit’, in that case no + traceback is printed and the Python process will exit with the + error code specified by the ‘SystemExit’ instance. + + Call this function `only' when the error indicator is set. + Otherwise it will cause a fatal error! + + If `set_sys_last_vars' is nonzero, the variables *note + sys.last_type: c5a, *note sys.last_value: 335f. and *note + sys.last_traceback: 325a. will be set to the type, value and + traceback of the printed exception, respectively. + + -- C Function: void PyErr_Print () + + Alias for ‘PyErr_PrintEx(1)’. + + -- C Function: void PyErr_WriteUnraisable (PyObject *obj) + + Call *note sys.unraisablehook(): 22d. using the current exception + and `obj' argument. + + This utility function prints a warning message to ‘sys.stderr’ when + an exception has been set but it is impossible for the interpreter + to actually raise the exception. It is used, for example, when an + exception occurs in an *note __del__(): 91f. method. + + The function is called with a single argument `obj' that identifies + the context in which the unraisable exception occurred. If + possible, the repr of `obj' will be printed in the warning message. + + An exception must be set when calling this function. + + +File: python.info, Node: Raising exceptions, Next: Issuing warnings, Prev: Printing and clearing, Up: Exception Handling + +7.5.2 Raising exceptions +------------------------ + +These functions help you set the current thread’s error indicator. For +convenience, some of these functions will always return a ‘NULL’ pointer +for use in a ‘return’ statement. + + -- C Function: void PyErr_SetString (PyObject *type, const + char *message) + + This is the most common way to set the error indicator. The first + argument specifies the exception type; it is normally one of the + standard exceptions, e.g. ‘PyExc_RuntimeError’. You need not + increment its reference count. The second argument is an error + message; it is decoded from ‘'utf-8’’. + + -- C Function: void PyErr_SetObject (PyObject *type, PyObject *value) + + This function is similar to *note PyErr_SetString(): 383c. but lets + you specify an arbitrary Python object for the “value” of the + exception. + + -- C Function: PyObject* PyErr_Format (PyObject *exception, const + char *format, ...) + `Return value: Always NULL.' This function sets the error indicator + and returns ‘NULL’. `exception' should be a Python exception + class. The `format' and subsequent parameters help format the + error message; they have the same meaning and values as in *note + PyUnicode_FromFormat(): 7cb. `format' is an ASCII-encoded string. + + -- C Function: PyObject* PyErr_FormatV (PyObject *exception, const + char *format, va_list vargs) + `Return value: Always NULL.' Same as *note PyErr_Format(): 7aa, but + taking a ‘va_list’ argument rather than a variable number of + arguments. + + New in version 3.5. + + -- C Function: void PyErr_SetNone (PyObject *type) + + This is a shorthand for ‘PyErr_SetObject(type, Py_None)’. + + -- C Function: int PyErr_BadArgument () + + This is a shorthand for ‘PyErr_SetString(PyExc_TypeError, + message)’, where `message' indicates that a built-in operation was + invoked with an illegal argument. It is mostly for internal use. + + -- C Function: PyObject* PyErr_NoMemory () + `Return value: Always NULL.' This is a shorthand for + ‘PyErr_SetNone(PyExc_MemoryError)’; it returns ‘NULL’ so an object + allocation function can write ‘return PyErr_NoMemory();’ when it + runs out of memory. + + -- C Function: PyObject* PyErr_SetFromErrno (PyObject *type) + `Return value: Always NULL.' + + This is a convenience function to raise an exception when a C + library function has returned an error and set the C variable + ‘errno’. It constructs a tuple object whose first item is the + integer ‘errno’ value and whose second item is the corresponding + error message (gotten from ‘strerror()’), and then calls + ‘PyErr_SetObject(type, object)’. On Unix, when the ‘errno’ value + is ‘EINTR’, indicating an interrupted system call, this calls *note + PyErr_CheckSignals(): 393b, and if that set the error indicator, + leaves it set to that. The function always returns ‘NULL’, so a + wrapper function around a system call can write ‘return + PyErr_SetFromErrno(type);’ when the system call returns an error. + + -- C Function: PyObject* PyErr_SetFromErrnoWithFilenameObject + (PyObject *type, PyObject *filenameObject) + `Return value: Always NULL.' Similar to *note PyErr_SetFromErrno(): + 383d, with the additional behavior that if `filenameObject' is not + ‘NULL’, it is passed to the constructor of `type' as a third + parameter. In the case of *note OSError: 1d3. exception, this is + used to define the ‘filename’ attribute of the exception instance. + + -- C Function: PyObject* PyErr_SetFromErrnoWithFilenameObjects + (PyObject *type, PyObject *filenameObject, + PyObject *filenameObject2) + `Return value: Always NULL.' Similar to *note + PyErr_SetFromErrnoWithFilenameObject(): 393c, but takes a second + filename object, for raising errors when a function that takes two + filenames fails. + + New in version 3.4. + + -- C Function: PyObject* PyErr_SetFromErrnoWithFilename + (PyObject *type, const char *filename) + `Return value: Always NULL.' Similar to *note + PyErr_SetFromErrnoWithFilenameObject(): 393c, but the filename is + given as a C string. `filename' is decoded from the filesystem + encoding (*note os.fsdecode(): 4ee.). + + -- C Function: PyObject* PyErr_SetFromWindowsErr (int ierr) + `Return value: Always NULL.' This is a convenience function to + raise *note WindowsError: 994. If called with `ierr' of ‘0’, the + error code returned by a call to ‘GetLastError()’ is used instead. + It calls the Win32 function ‘FormatMessage()’ to retrieve the + Windows description of error code given by `ierr' or + ‘GetLastError()’, then it constructs a tuple object whose first + item is the `ierr' value and whose second item is the corresponding + error message (gotten from ‘FormatMessage()’), and then calls + ‘PyErr_SetObject(PyExc_WindowsError, object)’. This function + always returns ‘NULL’. + + *note Availability: ffb.: Windows. + + -- C Function: PyObject* PyErr_SetExcFromWindowsErr (PyObject *type, + int ierr) + `Return value: Always NULL.' Similar to *note + PyErr_SetFromWindowsErr(): 393f, with an additional parameter + specifying the exception type to be raised. + + *note Availability: ffb.: Windows. + + -- C Function: PyObject* PyErr_SetFromWindowsErrWithFilename (int ierr, + const char *filename) + `Return value: Always NULL.' Similar to + ‘PyErr_SetFromWindowsErrWithFilenameObject()’, but the filename is + given as a C string. `filename' is decoded from the filesystem + encoding (*note os.fsdecode(): 4ee.). + + *note Availability: ffb.: Windows. + + -- C Function: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObject + (PyObject *type, int ierr, PyObject *filename) + `Return value: Always NULL.' Similar to + ‘PyErr_SetFromWindowsErrWithFilenameObject()’, with an additional + parameter specifying the exception type to be raised. + + *note Availability: ffb.: Windows. + + -- C Function: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObjects + (PyObject *type, int ierr, PyObject *filename, + PyObject *filename2) + `Return value: Always NULL.' Similar to *note + PyErr_SetExcFromWindowsErrWithFilenameObject(): 3942, but accepts a + second filename object. + + *note Availability: ffb.: Windows. + + New in version 3.4. + + -- C Function: PyObject* PyErr_SetExcFromWindowsErrWithFilename + (PyObject *type, int ierr, const char *filename) + `Return value: Always NULL.' Similar to *note + PyErr_SetFromWindowsErrWithFilename(): 3941, with an additional + parameter specifying the exception type to be raised. + + *note Availability: ffb.: Windows. + + -- C Function: PyObject* PyErr_SetImportError (PyObject *msg, + PyObject *name, PyObject *path) + `Return value: Always NULL.' This is a convenience function to + raise *note ImportError: 334. `msg' will be set as the exception’s + message string. `name' and `path', both of which can be ‘NULL’, + will be set as the *note ImportError: 334.’s respective ‘name’ and + ‘path’ attributes. + + New in version 3.3. + + -- C Function: void PyErr_SyntaxLocationObject (PyObject *filename, + int lineno, int col_offset) + + Set file, line, and offset information for the current exception. + If the current exception is not a *note SyntaxError: 458, then it + sets additional attributes, which make the exception printing + subsystem think the exception is a *note SyntaxError: 458. + + New in version 3.4. + + -- C Function: void PyErr_SyntaxLocationEx (const char *filename, + int lineno, int col_offset) + + Like *note PyErr_SyntaxLocationObject(): 3945, but `filename' is a + byte string decoded from the filesystem encoding (*note + os.fsdecode(): 4ee.). + + New in version 3.2. + + -- C Function: void PyErr_SyntaxLocation (const char *filename, + int lineno) + + Like *note PyErr_SyntaxLocationEx(): 3946, but the col_offset + parameter is omitted. + + -- C Function: void PyErr_BadInternalCall () + + This is a shorthand for ‘PyErr_SetString(PyExc_SystemError, + message)’, where `message' indicates that an internal operation + (e.g. a Python/C API function) was invoked with an illegal + argument. It is mostly for internal use. + + +File: python.info, Node: Issuing warnings, Next: Querying the error indicator, Prev: Raising exceptions, Up: Exception Handling + +7.5.3 Issuing warnings +---------------------- + +Use these functions to issue warnings from C code. They mirror similar +functions exported by the Python *note warnings: 126. module. They +normally print a warning message to `sys.stderr'; however, it is also +possible that the user has specified that warnings are to be turned into +errors, and in that case they will raise an exception. It is also +possible that the functions raise an exception because of a problem with +the warning machinery. The return value is ‘0’ if no exception is +raised, or ‘-1’ if an exception is raised. (It is not possible to +determine whether a warning message is actually printed, nor what the +reason is for the exception; this is intentional.) If an exception is +raised, the caller should do its normal exception handling (for example, +*note Py_DECREF(): 266. owned references and return an error value). + + -- C Function: int PyErr_WarnEx (PyObject *category, const + char *message, Py_ssize_t stack_level) + + Issue a warning message. The `category' argument is a warning + category (see below) or ‘NULL’; the `message' argument is a UTF-8 + encoded string. `stack_level' is a positive number giving a number + of stack frames; the warning will be issued from the currently + executing line of code in that stack frame. A `stack_level' of 1 + is the function calling *note PyErr_WarnEx(): db1, 2 is the + function above that, and so forth. + + Warning categories must be subclasses of ‘PyExc_Warning’; + ‘PyExc_Warning’ is a subclass of ‘PyExc_Exception’; the default + warning category is ‘PyExc_RuntimeWarning’. The standard Python + warning categories are available as global variables whose names + are enumerated at *note Standard Warning Categories: 394a. + + For information about warning control, see the documentation for + the *note warnings: 126. module and the *note -W: 417. option in + the command line documentation. There is no C API for warning + control. + + -- C Function: PyObject* PyErr_SetImportErrorSubclass + (PyObject *exception, PyObject *msg, PyObject *name, + PyObject *path) + `Return value: Always NULL.' Much like *note + PyErr_SetImportError(): 5f8. but this function allows for + specifying a subclass of *note ImportError: 334. to raise. + + New in version 3.6. + + -- C Function: int PyErr_WarnExplicitObject (PyObject *category, + PyObject *message, PyObject *filename, int lineno, + PyObject *module, PyObject *registry) + + Issue a warning message with explicit control over all warning + attributes. This is a straightforward wrapper around the Python + function *note warnings.warn_explicit(): 5b0, see there for more + information. The `module' and `registry' arguments may be set to + ‘NULL’ to get the default effect described there. + + New in version 3.4. + + -- C Function: int PyErr_WarnExplicit (PyObject *category, const + char *message, const char *filename, int lineno, const + char *module, PyObject *registry) + + Similar to *note PyErr_WarnExplicitObject(): 394b. except that + `message' and `module' are UTF-8 encoded strings, and `filename' is + decoded from the filesystem encoding (*note os.fsdecode(): 4ee.). + + -- C Function: int PyErr_WarnFormat (PyObject *category, + Py_ssize_t stack_level, const char *format, ...) + + Function similar to *note PyErr_WarnEx(): db1, but use *note + PyUnicode_FromFormat(): 7cb. to format the warning message. + `format' is an ASCII-encoded string. + + New in version 3.2. + + -- C Function: int PyErr_ResourceWarning (PyObject *source, + Py_ssize_t stack_level, const char *format, ...) + + Function similar to *note PyErr_WarnFormat(): 394d, but `category' + is *note ResourceWarning: 4d0. and it passes `source' to + ‘warnings.WarningMessage()’. + + New in version 3.6. + + +File: python.info, Node: Querying the error indicator, Next: Signal Handling<2>, Prev: Issuing warnings, Up: Exception Handling + +7.5.4 Querying the error indicator +---------------------------------- + + -- C Function: PyObject* PyErr_Occurred () + `Return value: Borrowed reference.' Test whether the error + indicator is set. If set, return the exception `type' (the first + argument to the last call to one of the ‘PyErr_Set*()’ functions or + to *note PyErr_Restore(): 3897.). If not set, return ‘NULL’. You + do not own a reference to the return value, so you do not need to + *note Py_DECREF(): 266. it. + + Note: Do not compare the return value to a specific exception; + use *note PyErr_ExceptionMatches(): 38fb. instead, shown + below. (The comparison could easily fail since the exception + may be an instance instead of a class, in the case of a class + exception, or it may be a subclass of the expected exception.) + + -- C Function: int PyErr_ExceptionMatches (PyObject *exc) + + Equivalent to ‘PyErr_GivenExceptionMatches(PyErr_Occurred(), exc)’. + This should only be called when an exception is actually set; a + memory access violation will occur if no exception has been raised. + + -- C Function: int PyErr_GivenExceptionMatches (PyObject *given, + PyObject *exc) + + Return true if the `given' exception matches the exception type in + `exc'. If `exc' is a class object, this also returns true when + `given' is an instance of a subclass. If `exc' is a tuple, all + exception types in the tuple (and recursively in subtuples) are + searched for a match. + + -- C Function: void PyErr_Fetch (PyObject **ptype, PyObject **pvalue, + PyObject **ptraceback) + + Retrieve the error indicator into three variables whose addresses + are passed. If the error indicator is not set, set all three + variables to ‘NULL’. If it is set, it will be cleared and you own + a reference to each object retrieved. The value and traceback + object may be ‘NULL’ even when the type object is not. + + Note: This function is normally only used by code that needs + to catch exceptions or by code that needs to save and restore + the error indicator temporarily, e.g.: + + { + PyObject *type, *value, *traceback; + PyErr_Fetch(&type, &value, &traceback); + + /* ... code that might produce other errors ... */ + + PyErr_Restore(type, value, traceback); + } + + -- C Function: void PyErr_Restore (PyObject *type, PyObject *value, + PyObject *traceback) + + Set the error indicator from the three objects. If the error + indicator is already set, it is cleared first. If the objects are + ‘NULL’, the error indicator is cleared. Do not pass a ‘NULL’ type + and non-‘NULL’ value or traceback. The exception type should be a + class. Do not pass an invalid exception type or value. (Violating + these rules will cause subtle problems later.) This call takes + away a reference to each object: you must own a reference to each + object before the call and after the call you no longer own these + references. (If you don’t understand this, don’t use this + function. I warned you.) + + Note: This function is normally only used by code that needs + to save and restore the error indicator temporarily. Use + *note PyErr_Fetch(): 96a. to save the current error indicator. + + -- C Function: void PyErr_NormalizeException (PyObject**exc, + PyObject**val, PyObject**tb) + + Under certain circumstances, the values returned by *note + PyErr_Fetch(): 96a. below can be “unnormalized”, meaning that + ‘*exc’ is a class object but ‘*val’ is not an instance of the same + class. This function can be used to instantiate the class in that + case. If the values are already normalized, nothing happens. The + delayed normalization is implemented to improve performance. + + Note: This function `does not' implicitly set the + ‘__traceback__’ attribute on the exception value. If setting + the traceback appropriately is desired, the following + additional snippet is needed: + + if (tb != NULL) { + PyException_SetTraceback(val, tb); + } + + -- C Function: void PyErr_GetExcInfo (PyObject **ptype, + PyObject **pvalue, PyObject **ptraceback) + + Retrieve the exception info, as known from ‘sys.exc_info()’. This + refers to an exception that was `already caught', not to an + exception that was freshly raised. Returns new references for the + three objects, any of which may be ‘NULL’. Does not modify the + exception info state. + + Note: This function is not normally used by code that wants to + handle exceptions. Rather, it can be used when code needs to + save and restore the exception state temporarily. Use *note + PyErr_SetExcInfo(): 3952. to restore or clear the exception + state. + + New in version 3.3. + + -- C Function: void PyErr_SetExcInfo (PyObject *type, PyObject *value, + PyObject *traceback) + + Set the exception info, as known from ‘sys.exc_info()’. This + refers to an exception that was `already caught', not to an + exception that was freshly raised. This function steals the + references of the arguments. To clear the exception state, pass + ‘NULL’ for all three arguments. For general rules about the three + arguments, see *note PyErr_Restore(): 3897. + + Note: This function is not normally used by code that wants to + handle exceptions. Rather, it can be used when code needs to + save and restore the exception state temporarily. Use *note + PyErr_GetExcInfo(): 3951. to read the exception state. + + New in version 3.3. + + +File: python.info, Node: Signal Handling<2>, Next: Exception Classes, Prev: Querying the error indicator, Up: Exception Handling + +7.5.5 Signal Handling +--------------------- + + -- C Function: int PyErr_CheckSignals () + + This function interacts with Python’s signal handling. It checks + whether a signal has been sent to the processes and if so, invokes + the corresponding signal handler. If the *note signal: ea. module + is supported, this can invoke a signal handler written in Python. + In all cases, the default effect for ‘SIGINT’ is to raise the *note + KeyboardInterrupt: 197. exception. If an exception is raised the + error indicator is set and the function returns ‘-1’; otherwise the + function returns ‘0’. The error indicator may or may not be + cleared if it was previously set. + + -- C Function: void PyErr_SetInterrupt () + + Simulate the effect of a ‘SIGINT’ signal arriving. The next time + *note PyErr_CheckSignals(): 393b. is called, the Python signal + handler for ‘SIGINT’ will be called. + + If ‘SIGINT’ isn’t handled by Python (it was set to *note + signal.SIG_DFL: 21c4. or *note signal.SIG_IGN: 21c5.), this + function does nothing. + + -- C Function: int PySignal_SetWakeupFd (int fd) + + This utility function specifies a file descriptor to which the + signal number is written as a single byte whenever a signal is + received. `fd' must be non-blocking. It returns the previous such + file descriptor. + + The value ‘-1’ disables the feature; this is the initial state. + This is equivalent to *note signal.set_wakeup_fd(): 3ce. in Python, + but without any error checking. `fd' should be a valid file + descriptor. The function should only be called from the main + thread. + + Changed in version 3.5: On Windows, the function now also supports + socket handles. + + +File: python.info, Node: Exception Classes, Next: Exception Objects, Prev: Signal Handling<2>, Up: Exception Handling + +7.5.6 Exception Classes +----------------------- + + -- C Function: PyObject* PyErr_NewException (const char *name, + PyObject *base, PyObject *dict) + `Return value: New reference.' This utility function creates and + returns a new exception class. The `name' argument must be the + name of the new exception, a C string of the form + ‘module.classname’. The `base' and `dict' arguments are normally + ‘NULL’. This creates a class object derived from *note Exception: + 1a9. (accessible in C as ‘PyExc_Exception’). + + The ‘__module__’ attribute of the new class is set to the first + part (up to the last dot) of the `name' argument, and the class + name is set to the last part (after the last dot). The `base' + argument can be used to specify alternate base classes; it can + either be only one class or a tuple of classes. The `dict' + argument can be used to specify a dictionary of class variables and + methods. + + -- C Function: PyObject* PyErr_NewExceptionWithDoc (const char *name, + const char *doc, PyObject *base, PyObject *dict) + `Return value: New reference.' Same as *note PyErr_NewException(): + c00, except that the new exception class can easily be given a + docstring: If `doc' is non-‘NULL’, it will be used as the docstring + for the exception class. + + New in version 3.2. + + +File: python.info, Node: Exception Objects, Next: Unicode Exception Objects, Prev: Exception Classes, Up: Exception Handling + +7.5.7 Exception Objects +----------------------- + + -- C Function: PyObject* PyException_GetTraceback (PyObject *ex) + `Return value: New reference.' Return the traceback associated + with the exception as a new reference, as accessible from Python + through ‘__traceback__’. If there is no traceback associated, this + returns ‘NULL’. + + -- C Function: int PyException_SetTraceback (PyObject *ex, + PyObject *tb) + + Set the traceback associated with the exception to `tb'. Use + ‘Py_None’ to clear it. + + -- C Function: PyObject* PyException_GetContext (PyObject *ex) + `Return value: New reference.' Return the context (another + exception instance during whose handling `ex' was raised) + associated with the exception as a new reference, as accessible + from Python through ‘__context__’. If there is no context + associated, this returns ‘NULL’. + + -- C Function: void PyException_SetContext (PyObject *ex, + PyObject *ctx) + + Set the context associated with the exception to `ctx'. Use ‘NULL’ + to clear it. There is no type check to make sure that `ctx' is an + exception instance. This steals a reference to `ctx'. + + -- C Function: PyObject* PyException_GetCause (PyObject *ex) + `Return value: New reference.' Return the cause (either an + exception instance, or *note None: 157, set by ‘raise ... from + ...’) associated with the exception as a new reference, as + accessible from Python through ‘__cause__’. + + -- C Function: void PyException_SetCause (PyObject *ex, + PyObject *cause) + + Set the cause associated with the exception to `cause'. Use ‘NULL’ + to clear it. There is no type check to make sure that `cause' is + either an exception instance or *note None: 157. This steals a + reference to `cause'. + + ‘__suppress_context__’ is implicitly set to ‘True’ by this + function. + + +File: python.info, Node: Unicode Exception Objects, Next: Recursion Control, Prev: Exception Objects, Up: Exception Handling + +7.5.8 Unicode Exception Objects +------------------------------- + +The following functions are used to create and modify Unicode exceptions +from C. + + -- C Function: PyObject* PyUnicodeDecodeError_Create (const + char *encoding, const char *object, Py_ssize_t length, + Py_ssize_t start, Py_ssize_t end, const char *reason) + `Return value: New reference.' Create a *note UnicodeDecodeError: + 1fd. object with the attributes `encoding', `object', `length', + `start', `end' and `reason'. `encoding' and `reason' are UTF-8 + encoded strings. + + -- C Function: PyObject* PyUnicodeEncodeError_Create (const + char *encoding, const Py_UNICODE *object, Py_ssize_t length, + Py_ssize_t start, Py_ssize_t end, const char *reason) + `Return value: New reference.' Create a *note UnicodeEncodeError: + 1fc. object with the attributes `encoding', `object', `length', + `start', `end' and `reason'. `encoding' and `reason' are UTF-8 + encoded strings. + + Deprecated since version 3.3: 3.11 + + ‘Py_UNICODE’ is deprecated since Python 3.3. Please migrate to + ‘PyObject_CallFunction(PyExc_UnicodeEncodeError, "sOnns", ...)’. + + -- C Function: PyObject* PyUnicodeTranslateError_Create (const + Py_UNICODE *object, Py_ssize_t length, Py_ssize_t start, + Py_ssize_t end, const char *reason) + `Return value: New reference.' Create a *note + UnicodeTranslateError: 13bd. object with the attributes `object', + `length', `start', `end' and `reason'. `reason' is a UTF-8 encoded + string. + + Deprecated since version 3.3: 3.11 + + ‘Py_UNICODE’ is deprecated since Python 3.3. Please migrate to + ‘PyObject_CallFunction(PyExc_UnicodeTranslateError, "Onns", ...)’. + + -- C Function: PyObject* PyUnicodeDecodeError_GetEncoding + (PyObject *exc) + -- C Function: PyObject* PyUnicodeEncodeError_GetEncoding + (PyObject *exc) + `Return value: New reference.' Return the `encoding' attribute of + the given exception object. + + -- C Function: PyObject* PyUnicodeDecodeError_GetObject (PyObject *exc) + -- C Function: PyObject* PyUnicodeEncodeError_GetObject (PyObject *exc) + -- C Function: PyObject* PyUnicodeTranslateError_GetObject + (PyObject *exc) + `Return value: New reference.' Return the `object' attribute of + the given exception object. + + -- C Function: int PyUnicodeDecodeError_GetStart (PyObject *exc, + Py_ssize_t *start) + -- C Function: int PyUnicodeEncodeError_GetStart (PyObject *exc, + Py_ssize_t *start) + -- C Function: int PyUnicodeTranslateError_GetStart (PyObject *exc, + Py_ssize_t *start) + + Get the `start' attribute of the given exception object and place + it into `*start'. `start' must not be ‘NULL’. Return ‘0’ on + success, ‘-1’ on failure. + + -- C Function: int PyUnicodeDecodeError_SetStart (PyObject *exc, + Py_ssize_t start) + -- C Function: int PyUnicodeEncodeError_SetStart (PyObject *exc, + Py_ssize_t start) + -- C Function: int PyUnicodeTranslateError_SetStart (PyObject *exc, + Py_ssize_t start) + + Set the `start' attribute of the given exception object to `start'. + Return ‘0’ on success, ‘-1’ on failure. + + -- C Function: int PyUnicodeDecodeError_GetEnd (PyObject *exc, + Py_ssize_t *end) + -- C Function: int PyUnicodeEncodeError_GetEnd (PyObject *exc, + Py_ssize_t *end) + -- C Function: int PyUnicodeTranslateError_GetEnd (PyObject *exc, + Py_ssize_t *end) + + Get the `end' attribute of the given exception object and place it + into `*end'. `end' must not be ‘NULL’. Return ‘0’ on success, + ‘-1’ on failure. + + -- C Function: int PyUnicodeDecodeError_SetEnd (PyObject *exc, + Py_ssize_t end) + -- C Function: int PyUnicodeEncodeError_SetEnd (PyObject *exc, + Py_ssize_t end) + -- C Function: int PyUnicodeTranslateError_SetEnd (PyObject *exc, + Py_ssize_t end) + + Set the `end' attribute of the given exception object to `end'. + Return ‘0’ on success, ‘-1’ on failure. + + -- C Function: PyObject* PyUnicodeDecodeError_GetReason (PyObject *exc) + -- C Function: PyObject* PyUnicodeEncodeError_GetReason (PyObject *exc) + -- C Function: PyObject* PyUnicodeTranslateError_GetReason + (PyObject *exc) + `Return value: New reference.' Return the `reason' attribute of + the given exception object. + + -- C Function: int PyUnicodeDecodeError_SetReason (PyObject *exc, const + char *reason) + -- C Function: int PyUnicodeEncodeError_SetReason (PyObject *exc, const + char *reason) + -- C Function: int PyUnicodeTranslateError_SetReason (PyObject *exc, + const char *reason) + + Set the `reason' attribute of the given exception object to + `reason'. Return ‘0’ on success, ‘-1’ on failure. + + +File: python.info, Node: Recursion Control, Next: Standard Exceptions, Prev: Unicode Exception Objects, Up: Exception Handling + +7.5.9 Recursion Control +----------------------- + +These two functions provide a way to perform safe recursive calls at the +C level, both in the core and in extension modules. They are needed if +the recursive code does not necessarily invoke Python code (which tracks +its recursion depth automatically). + + -- C Function: int Py_EnterRecursiveCall (const char *where) + + Marks a point where a recursive C-level call is about to be + performed. + + If ‘USE_STACKCHECK’ is defined, this function checks if the OS + stack overflowed using *note PyOS_CheckStack(): 397b. In this is + the case, it sets a *note MemoryError: 13af. and returns a nonzero + value. + + The function then checks if the recursion limit is reached. If + this is the case, a *note RecursionError: 634. is set and a nonzero + value is returned. Otherwise, zero is returned. + + `where' should be a string such as ‘" in instance check"’ to be + concatenated to the *note RecursionError: 634. message caused by + the recursion depth limit. + + -- C Function: void Py_LeaveRecursiveCall () + + Ends a *note Py_EnterRecursiveCall(): 397a. Must be called once + for each `successful' invocation of *note Py_EnterRecursiveCall(): + 397a. + +Properly implementing *note tp_repr: 389a. for container types requires +special recursion handling. In addition to protecting the stack, *note +tp_repr: 389a. also needs to track objects to prevent cycles. The +following two functions facilitate this functionality. Effectively, +these are the C equivalent to *note reprlib.recursive_repr(): b70. + + -- C Function: int Py_ReprEnter (PyObject *object) + + Called at the beginning of the *note tp_repr: 389a. implementation + to detect cycles. + + If the object has already been processed, the function returns a + positive integer. In that case the *note tp_repr: 389a. + implementation should return a string object indicating a cycle. + As examples, *note dict: 1b8. objects return ‘{...}’ and *note + list: 262. objects return ‘[...]’. + + The function will return a negative integer if the recursion limit + is reached. In that case the *note tp_repr: 389a. implementation + should typically return ‘NULL’. + + Otherwise, the function returns zero and the *note tp_repr: 389a. + implementation can continue normally. + + -- C Function: void Py_ReprLeave (PyObject *object) + + Ends a *note Py_ReprEnter(): 397d. Must be called once for each + invocation of *note Py_ReprEnter(): 397d. that returns zero. + + +File: python.info, Node: Standard Exceptions, Next: Standard Warning Categories, Prev: Recursion Control, Up: Exception Handling + +7.5.10 Standard Exceptions +-------------------------- + +All standard Python exceptions are available as global variables whose +names are ‘PyExc_’ followed by the Python exception name. These have +the type *note PyObject*: 4ba.; they are all class objects. For +completeness, here are all the variables: + +C Name Python Name Notes + +--------------------------------------------------------------------------------------------------- + +‘PyExc_BaseException’ *note BaseException: 1a8. (1) + + +‘PyExc_Exception’ *note Exception: 1a9. (1) + + +‘PyExc_ArithmeticError’ *note ArithmeticError: 13a9. (1) + + +‘PyExc_AssertionError’ *note AssertionError: 1223. + + +‘PyExc_AttributeError’ *note AttributeError: 39b. + + +‘PyExc_BlockingIOError’ *note BlockingIOError: 997. + + +‘PyExc_BrokenPipeError’ *note BrokenPipeError: 99d. + + +‘PyExc_BufferError’ *note BufferError: 13ab. + + +‘PyExc_ChildProcessError’ *note ChildProcessError: 998. + + +‘PyExc_ConnectionAbortedError’ *note ConnectionAbortedError: 99e. + + +‘PyExc_ConnectionError’ *note ConnectionError: 6e6. + + +‘PyExc_ConnectionRefusedError’ *note ConnectionRefusedError: 99f. + + +‘PyExc_ConnectionResetError’ *note ConnectionResetError: 9a0. + + +‘PyExc_EOFError’ *note EOFError: c6c. + + +‘PyExc_FileExistsError’ *note FileExistsError: 958. + + +‘PyExc_FileNotFoundError’ *note FileNotFoundError: 7c0. + + +‘PyExc_FloatingPointError’ *note FloatingPointError: 13aa. + + +‘PyExc_GeneratorExit’ *note GeneratorExit: d32. + + +‘PyExc_ImportError’ *note ImportError: 334. + + +‘PyExc_IndentationError’ *note IndentationError: e78. + + +‘PyExc_IndexError’ *note IndexError: e75. + + +‘PyExc_InterruptedError’ *note InterruptedError: 659. + + +‘PyExc_IsADirectoryError’ *note IsADirectoryError: 999. + + +‘PyExc_KeyError’ *note KeyError: 2c7. + + +‘PyExc_KeyboardInterrupt’ *note KeyboardInterrupt: 197. + + +‘PyExc_LookupError’ *note LookupError: 1308. (1) + + +‘PyExc_MemoryError’ *note MemoryError: 13af. + + +‘PyExc_ModuleNotFoundError’ *note ModuleNotFoundError: 39a. + + +‘PyExc_NameError’ *note NameError: d7b. + + +‘PyExc_NotADirectoryError’ *note NotADirectoryError: 99a. + + +‘PyExc_NotImplementedError’ *note NotImplementedError: 60e. + + +‘PyExc_OSError’ *note OSError: 1d3. (1) + + +‘PyExc_OverflowError’ *note OverflowError: 960. + + +‘PyExc_PermissionError’ *note PermissionError: 5ff. + + +‘PyExc_ProcessLookupError’ *note ProcessLookupError: 99b. + + +‘PyExc_RecursionError’ *note RecursionError: 634. + + +‘PyExc_ReferenceError’ *note ReferenceError: e4d. (2) + + +‘PyExc_RuntimeError’ *note RuntimeError: 2ba. + + +‘PyExc_StopAsyncIteration’ *note StopAsyncIteration: 10cd. + + +‘PyExc_StopIteration’ *note StopIteration: 486. + + +‘PyExc_SyntaxError’ *note SyntaxError: 458. + + +‘PyExc_SystemError’ *note SystemError: 5fb. + + +‘PyExc_SystemExit’ *note SystemExit: 5fc. + + +‘PyExc_TabError’ *note TabError: e77. + + +‘PyExc_TimeoutError’ *note TimeoutError: 99c. + + +‘PyExc_TypeError’ *note TypeError: 192. + + +‘PyExc_UnboundLocalError’ *note UnboundLocalError: e76. + + +‘PyExc_UnicodeDecodeError’ *note UnicodeDecodeError: 1fd. + + +‘PyExc_UnicodeEncodeError’ *note UnicodeEncodeError: 1fc. + + +‘PyExc_UnicodeError’ *note UnicodeError: c3b. + + +‘PyExc_UnicodeTranslateError’ *note UnicodeTranslateError: 13bd. + + +‘PyExc_ValueError’ *note ValueError: 1fb. + + +‘PyExc_ZeroDivisionError’ *note ZeroDivisionError: f42. + + +New in version 3.3: ‘PyExc_BlockingIOError’, ‘PyExc_BrokenPipeError’, +‘PyExc_ChildProcessError’, ‘PyExc_ConnectionError’, +‘PyExc_ConnectionAbortedError’, ‘PyExc_ConnectionRefusedError’, +‘PyExc_ConnectionResetError’, ‘PyExc_FileExistsError’, +‘PyExc_FileNotFoundError’, ‘PyExc_InterruptedError’, +‘PyExc_IsADirectoryError’, ‘PyExc_NotADirectoryError’, +‘PyExc_PermissionError’, ‘PyExc_ProcessLookupError’ and +‘PyExc_TimeoutError’ were introduced following PEP 3151(1). + +New in version 3.5: ‘PyExc_StopAsyncIteration’ and +‘PyExc_RecursionError’. + +New in version 3.6: ‘PyExc_ModuleNotFoundError’. + +These are compatibility aliases to ‘PyExc_OSError’: + +C Name Notes + +--------------------------------------------------------- + +‘PyExc_EnvironmentError’ + +‘PyExc_IOError’ + +‘PyExc_WindowsError’ (3) + + +Changed in version 3.3: These aliases used to be separate exception +types. + +Notes: + + 1. This is a base class for other standard exceptions. + + 2. Only defined on Windows; protect code that uses this by testing + that the preprocessor macro ‘MS_WINDOWS’ is defined. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3151 + + +File: python.info, Node: Standard Warning Categories, Prev: Standard Exceptions, Up: Exception Handling + +7.5.11 Standard Warning Categories +---------------------------------- + +All standard Python warning categories are available as global variables +whose names are ‘PyExc_’ followed by the Python exception name. These +have the type *note PyObject*: 4ba.; they are all class objects. For +completeness, here are all the variables: + +C Name Python Name Notes + +---------------------------------------------------------------------------------------------------- + +‘PyExc_Warning’ *note Warning: 13c2. (1) + + +‘PyExc_BytesWarning’ *note BytesWarning: 4b5. + + +‘PyExc_DeprecationWarning’ *note DeprecationWarning: 278. + + +‘PyExc_FutureWarning’ *note FutureWarning: 325. + + +‘PyExc_ImportWarning’ *note ImportWarning: 5d8. + + +‘PyExc_PendingDeprecationWarning’ *note PendingDeprecationWarning: 279. + + +‘PyExc_ResourceWarning’ *note ResourceWarning: 4d0. + + +‘PyExc_RuntimeWarning’ *note RuntimeWarning: 2c0. + + +‘PyExc_SyntaxWarning’ *note SyntaxWarning: 191. + + +‘PyExc_UnicodeWarning’ *note UnicodeWarning: d87. + + +‘PyExc_UserWarning’ *note UserWarning: 13c3. + + +New in version 3.2: ‘PyExc_ResourceWarning’. + +Notes: + + 1. This is a base class for other standard warning categories. + + +File: python.info, Node: Utilities<2>, Next: Abstract Objects Layer, Prev: Exception Handling, Up: Python/C API Reference Manual + +7.6 Utilities +============= + +The functions in this chapter perform various utility tasks, ranging +from helping C code be more portable across platforms, using Python +modules from C, and parsing function arguments and constructing Python +values from C values. + +* Menu: + +* Operating System Utilities:: +* System Functions:: +* Process Control:: +* Importing Modules: Importing Modules<2>. +* Data marshalling support:: +* Parsing arguments and building values:: +* String conversion and formatting:: +* Reflection:: +* Codec registry and support functions:: + + +File: python.info, Node: Operating System Utilities, Next: System Functions, Up: Utilities<2> + +7.6.1 Operating System Utilities +-------------------------------- + + -- C Function: PyObject* PyOS_FSPath (PyObject *path) + `Return value: New reference.' Return the file system + representation for `path'. If the object is a *note str: 330. or + *note bytes: 331. object, then its reference count is incremented. + If the object implements the *note os.PathLike: 4eb. interface, + then *note __fspath__(): 4ec. is returned as long as it is a *note + str: 330. or *note bytes: 331. object. Otherwise *note TypeError: + 192. is raised and ‘NULL’ is returned. + + New in version 3.6. + + -- C Function: int Py_FdIsInteractive (FILE *fp, const char *filename) + + Return true (nonzero) if the standard I/O file `fp' with name + `filename' is deemed interactive. This is the case for files for + which ‘isatty(fileno(fp))’ is true. If the global flag *note + Py_InteractiveFlag: 3989. is true, this function also returns true + if the `filename' pointer is ‘NULL’ or if the name is equal to one + of the strings ‘'<stdin>'’ or ‘'???'’. + + -- C Function: void PyOS_BeforeFork () + + Function to prepare some internal state before a process fork. + This should be called before calling ‘fork()’ or any similar + function that clones the current process. Only available on + systems where ‘fork()’ is defined. + + Warning: The C ‘fork()’ call should only be made from the + *note “main” thread: 398a. (of the *note “main” interpreter: + 398b.). The same is true for ‘PyOS_BeforeFork()’. + + New in version 3.7. + + -- C Function: void PyOS_AfterFork_Parent () + + Function to update some internal state after a process fork. This + should be called from the parent process after calling ‘fork()’ or + any similar function that clones the current process, regardless of + whether process cloning was successful. Only available on systems + where ‘fork()’ is defined. + + Warning: The C ‘fork()’ call should only be made from the + *note “main” thread: 398a. (of the *note “main” interpreter: + 398b.). The same is true for ‘PyOS_AfterFork_Parent()’. + + New in version 3.7. + + -- C Function: void PyOS_AfterFork_Child () + + Function to update internal interpreter state after a process fork. + This must be called from the child process after calling ‘fork()’, + or any similar function that clones the current process, if there + is any chance the process will call back into the Python + interpreter. Only available on systems where ‘fork()’ is defined. + + Warning: The C ‘fork()’ call should only be made from the + *note “main” thread: 398a. (of the *note “main” interpreter: + 398b.). The same is true for ‘PyOS_AfterFork_Child()’. + + New in version 3.7. + + See also + ........ + + *note os.register_at_fork(): 3b6. allows registering custom Python + functions to be called by *note PyOS_BeforeFork(): 432, *note + PyOS_AfterFork_Parent(): 433. and *note PyOS_AfterFork_Child(): + 2cd. + + -- C Function: void PyOS_AfterFork () + + Function to update some internal state after a process fork; this + should be called in the new process if the Python interpreter will + continue to be used. If a new executable is loaded into the new + process, this function does not need to be called. + + Deprecated since version 3.7: This function is superseded by *note + PyOS_AfterFork_Child(): 2cd. + + -- C Function: int PyOS_CheckStack () + + Return true when the interpreter runs out of stack space. This is + a reliable check, but is only available when ‘USE_STACKCHECK’ is + defined (currently on Windows using the Microsoft Visual C++ + compiler). ‘USE_STACKCHECK’ will be defined automatically; you + should never change the definition in your own code. + + -- C Function: PyOS_sighandler_t PyOS_getsig (int i) + + Return the current signal handler for signal `i'. This is a thin + wrapper around either ‘sigaction()’ or ‘signal()’. Do not call + those functions directly! ‘PyOS_sighandler_t’ is a typedef alias + for ‘void (*)(int)’. + + -- C Function: PyOS_sighandler_t PyOS_setsig (int i, + PyOS_sighandler_t h) + + Set the signal handler for signal `i' to be `h'; return the old + signal handler. This is a thin wrapper around either ‘sigaction()’ + or ‘signal()’. Do not call those functions directly! + ‘PyOS_sighandler_t’ is a typedef alias for ‘void (*)(int)’. + + -- C Function: wchar_t* Py_DecodeLocale (const char* arg, size_t *size) + + Decode a byte string from the locale encoding with the *note + surrogateescape error handler: 14f1.: undecodable bytes are decoded + as characters in range U+DC80..U+DCFF. If a byte sequence can be + decoded as a surrogate character, escape the bytes using the + surrogateescape error handler instead of decoding them. + + Encoding, highest priority to lowest priority: + + * ‘UTF-8’ on macOS, Android, and VxWorks; + + * ‘UTF-8’ on Windows if *note Py_LegacyWindowsFSEncodingFlag: + 398e. is zero; + + * ‘UTF-8’ if the Python UTF-8 mode is enabled; + + * ‘ASCII’ if the ‘LC_CTYPE’ locale is ‘"C"’, + ‘nl_langinfo(CODESET)’ returns the ‘ASCII’ encoding (or an + alias), and ‘mbstowcs()’ and ‘wcstombs()’ functions uses the + ‘ISO-8859-1’ encoding. + + * the current locale encoding. + + Return a pointer to a newly allocated wide character string, use + *note PyMem_RawFree(): 398f. to free the memory. If size is not + ‘NULL’, write the number of wide characters excluding the null + character into ‘*size’ + + Return ‘NULL’ on decoding error or memory allocation error. If + `size' is not ‘NULL’, ‘*size’ is set to ‘(size_t)-1’ on memory + error or set to ‘(size_t)-2’ on decoding error. + + Decoding errors should never happen, unless there is a bug in the C + library. + + Use the *note Py_EncodeLocale(): 43d. function to encode the + character string back to a byte string. + + See also + ........ + + The *note PyUnicode_DecodeFSDefaultAndSize(): 3990. and *note + PyUnicode_DecodeLocaleAndSize(): 43e. functions. + + New in version 3.5. + + Changed in version 3.7: The function now uses the UTF-8 encoding in + the UTF-8 mode. + + Changed in version 3.8: The function now uses the UTF-8 encoding on + Windows if *note Py_LegacyWindowsFSEncodingFlag: 398e. is zero; + + -- C Function: char* Py_EncodeLocale (const wchar_t *text, + size_t *error_pos) + + Encode a wide character string to the locale encoding with the + *note surrogateescape error handler: 14f1.: surrogate characters in + the range U+DC80..U+DCFF are converted to bytes 0x80..0xFF. + + Encoding, highest priority to lowest priority: + + * ‘UTF-8’ on macOS, Android, and VxWorks; + + * ‘UTF-8’ on Windows if *note Py_LegacyWindowsFSEncodingFlag: + 398e. is zero; + + * ‘UTF-8’ if the Python UTF-8 mode is enabled; + + * ‘ASCII’ if the ‘LC_CTYPE’ locale is ‘"C"’, + ‘nl_langinfo(CODESET)’ returns the ‘ASCII’ encoding (or an + alias), and ‘mbstowcs()’ and ‘wcstombs()’ functions uses the + ‘ISO-8859-1’ encoding. + + * the current locale encoding. + + The function uses the UTF-8 encoding in the Python UTF-8 mode. + + Return a pointer to a newly allocated byte string, use *note + PyMem_Free(): da9. to free the memory. Return ‘NULL’ on encoding + error or memory allocation error + + If error_pos is not ‘NULL’, ‘*error_pos’ is set to ‘(size_t)-1’ on + success, or set to the index of the invalid character on encoding + error. + + Use the *note Py_DecodeLocale(): 43c. function to decode the bytes + string back to a wide character string. + + See also + ........ + + The *note PyUnicode_EncodeFSDefault(): 3991. and *note + PyUnicode_EncodeLocale(): 43f. functions. + + New in version 3.5. + + Changed in version 3.7: The function now uses the UTF-8 encoding in + the UTF-8 mode. + + Changed in version 3.8: The function now uses the UTF-8 encoding on + Windows if *note Py_LegacyWindowsFSEncodingFlag: 398e. is zero; + + +File: python.info, Node: System Functions, Next: Process Control, Prev: Operating System Utilities, Up: Utilities<2> + +7.6.2 System Functions +---------------------- + +These are utility functions that make functionality from the *note sys: +fd. module accessible to C code. They all work with the current +interpreter thread’s *note sys: fd. module’s dict, which is contained in +the internal thread state structure. + + -- C Function: PyObject *PySys_GetObject (const char *name) + `Return value: Borrowed reference.' Return the object `name' from + the *note sys: fd. module or ‘NULL’ if it does not exist, without + setting an exception. + + -- C Function: int PySys_SetObject (const char *name, PyObject *v) + + Set `name' in the *note sys: fd. module to `v' unless `v' is + ‘NULL’, in which case `name' is deleted from the sys module. + Returns ‘0’ on success, ‘-1’ on error. + + -- C Function: void PySys_ResetWarnOptions () + + Reset *note sys.warnoptions: 416. to an empty list. This function + may be called prior to *note Py_Initialize(): 439. + + -- C Function: void PySys_AddWarnOption (const wchar_t *s) + + Append `s' to *note sys.warnoptions: 416. This function must be + called prior to *note Py_Initialize(): 439. in order to affect the + warnings filter list. + + -- C Function: void PySys_AddWarnOptionUnicode (PyObject *unicode) + + Append `unicode' to *note sys.warnoptions: 416. + + Note: this function is not currently usable from outside the + CPython implementation, as it must be called prior to the implicit + import of *note warnings: 126. in *note Py_Initialize(): 439. to be + effective, but can’t be called until enough of the runtime has been + initialized to permit the creation of Unicode objects. + + -- C Function: void PySys_SetPath (const wchar_t *path) + + Set *note sys.path: 488. to a list object of paths found in `path' + which should be a list of paths separated with the platform’s + search path delimiter (‘:’ on Unix, ‘;’ on Windows). + + -- C Function: void PySys_WriteStdout (const char *format, ...) + + Write the output string described by `format' to *note sys.stdout: + 30e. No exceptions are raised, even if truncation occurs (see + below). + + `format' should limit the total size of the formatted output string + to 1000 bytes or less – after 1000 bytes, the output string is + truncated. In particular, this means that no unrestricted “%s” + formats should occur; these should be limited using “%.<N>s” where + <N> is a decimal number calculated so that <N> plus the maximum + size of other formatted text does not exceed 1000 bytes. Also + watch out for “%f”, which can print hundreds of digits for very + large numbers. + + If a problem occurs, or *note sys.stdout: 30e. is unset, the + formatted message is written to the real (C level) `stdout'. + + -- C Function: void PySys_WriteStderr (const char *format, ...) + + As *note PySys_WriteStdout(): 3998, but write to *note sys.stderr: + 30f. or `stderr' instead. + + -- C Function: void PySys_FormatStdout (const char *format, ...) + + Function similar to PySys_WriteStdout() but format the message + using *note PyUnicode_FromFormatV(): 399b. and don’t truncate the + message to an arbitrary length. + + New in version 3.2. + + -- C Function: void PySys_FormatStderr (const char *format, ...) + + As *note PySys_FormatStdout(): 399a, but write to *note sys.stderr: + 30f. or `stderr' instead. + + New in version 3.2. + + -- C Function: void PySys_AddXOption (const wchar_t *s) + + Parse `s' as a set of *note -X: 155. options and add them to the + current options mapping as returned by *note PySys_GetXOptions(): + 399e. This function may be called prior to *note Py_Initialize(): + 439. + + New in version 3.2. + + -- C Function: PyObject *PySys_GetXOptions () + `Return value: Borrowed reference.' Return the current dictionary + of *note -X: 155. options, similarly to *note sys._xoptions: fec. + On error, ‘NULL’ is returned and an exception is set. + + New in version 3.2. + + -- C Function: int PySys_Audit (const char *event, const char *format, + ...) + + Raise an auditing event with any active hooks. Return zero for + success and non-zero with an exception set on failure. + + If any hooks have been added, `format' and other arguments will be + used to construct a tuple to pass. Apart from ‘N’, the same format + characters as used in *note Py_BuildValue(): 2ce. are available. + If the built value is not a tuple, it will be added into a + single-element tuple. (The ‘N’ format option consumes a reference, + but since there is no way to know whether arguments to this + function will be consumed, using it may cause reference leaks.) + + Note that ‘#’ format characters should always be treated as + ‘Py_ssize_t’, regardless of whether ‘PY_SSIZE_T_CLEAN’ was defined. + + *note sys.audit(): 31ea. performs the same function from Python + code. + + New in version 3.8. + + Changed in version 3.8.2: Require ‘Py_ssize_t’ for ‘#’ format + characters. Previously, an unavoidable deprecation warning was + raised. + + -- C Function: int PySys_AddAuditHook (Py_AuditHookFunction hook, + void *userData) + + Append the callable `hook' to the list of active auditing hooks. + Return zero for success and non-zero on failure. If the runtime + has been initialized, also set an error on failure. Hooks added + through this API are called for all interpreters created by the + runtime. + + The `userData' pointer is passed into the hook function. Since + hook functions may be called from different runtimes, this pointer + should not refer directly to Python state. + + This function is safe to call before *note Py_Initialize(): 439. + When called after runtime initialization, existing audit hooks are + notified and may silently abort the operation by raising an error + subclassed from *note Exception: 1a9. (other errors will not be + silenced). + + The hook function is of type ‘int (*)(const char *event, PyObject + *args, void *userData)’, where `args' is guaranteed to be a *note + PyTupleObject: 399f. The hook function is always called with the + GIL held by the Python interpreter that raised the event. + + See PEP 578(1) for a detailed description of auditing. Functions + in the runtime and standard library that raise events are listed in + the *note audit events table: 31e7. Details are in each function’s + documentation. + + If the interpreter is initialized, this function raises a auditing + event ‘sys.addaudithook’ with no arguments. If any existing hooks + raise an exception derived from *note Exception: 1a9, the new hook + will not be added and the exception is cleared. As a result, + callers cannot assume that their hook has been added unless they + control all existing hooks. + + New in version 3.8. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0578 + + +File: python.info, Node: Process Control, Next: Importing Modules<2>, Prev: System Functions, Up: Utilities<2> + +7.6.3 Process Control +--------------------- + + -- C Function: void Py_FatalError (const char *message) + + Print a fatal error message and kill the process. No cleanup is + performed. This function should only be invoked when a condition + is detected that would make it dangerous to continue using the + Python interpreter; e.g., when the object administration appears to + be corrupted. On Unix, the standard C library function ‘abort()’ + is called which will attempt to produce a ‘core’ file. + + -- C Function: void Py_Exit (int status) + + Exit the current process. This calls *note Py_FinalizeEx(): 5c9. + and then calls the standard C library function ‘exit(status)’. If + *note Py_FinalizeEx(): 5c9. indicates an error, the exit status is + set to 120. + + Changed in version 3.6: Errors from finalization no longer ignored. + + -- C Function: int Py_AtExit (void (*func)()) + + Register a cleanup function to be called by *note Py_FinalizeEx(): + 5c9. The cleanup function will be called with no arguments and + should return no value. At most 32 cleanup functions can be + registered. When the registration is successful, *note + Py_AtExit(): 39a3. returns ‘0’; on failure, it returns ‘-1’. The + cleanup function registered last is called first. Each cleanup + function will be called at most once. Since Python’s internal + finalization will have completed before the cleanup function, no + Python APIs should be called by `func'. + + +File: python.info, Node: Importing Modules<2>, Next: Data marshalling support, Prev: Process Control, Up: Utilities<2> + +7.6.4 Importing Modules +----------------------- + + -- C Function: PyObject* PyImport_ImportModule (const char *name) + `Return value: New reference.' + + This is a simplified interface to *note PyImport_ImportModuleEx(): + b21. below, leaving the `globals' and `locals' arguments set to + ‘NULL’ and `level' set to 0. When the `name' argument contains a + dot (when it specifies a submodule of a package), the `fromlist' + argument is set to the list ‘['*']’ so that the return value is the + named module rather than the top-level package containing it as + would otherwise be the case. (Unfortunately, this has an + additional side effect when `name' in fact specifies a subpackage + instead of a submodule: the submodules specified in the package’s + ‘__all__’ variable are loaded.) Return a new reference to the + imported module, or ‘NULL’ with an exception set on failure. A + failing import of a module doesn’t leave the module in *note + sys.modules: 1152. + + This function always uses absolute imports. + + -- C Function: PyObject* PyImport_ImportModuleNoBlock (const + char *name) + `Return value: New reference.' This function is a deprecated alias + of *note PyImport_ImportModule(): c73. + + Changed in version 3.3: This function used to fail immediately when + the import lock was held by another thread. In Python 3.3 though, + the locking scheme switched to per-module locks for most purposes, + so this function’s special behaviour isn’t needed anymore. + + -- C Function: PyObject* PyImport_ImportModuleEx (const char *name, + PyObject *globals, PyObject *locals, PyObject *fromlist) + `Return value: New reference.' + + Import a module. This is best described by referring to the + built-in Python function *note __import__(): 610. + + The return value is a new reference to the imported module or + top-level package, or ‘NULL’ with an exception set on failure. + Like for *note __import__(): 610, the return value when a submodule + of a package was requested is normally the top-level package, + unless a non-empty `fromlist' was given. + + Failing imports remove incomplete module objects, like with *note + PyImport_ImportModule(): c73. + + -- C Function: PyObject* PyImport_ImportModuleLevelObject + (PyObject *name, PyObject *globals, PyObject *locals, + PyObject *fromlist, int level) + `Return value: New reference.' Import a module. This is best + described by referring to the built-in Python function *note + __import__(): 610, as the standard *note __import__(): 610. + function calls this function directly. + + The return value is a new reference to the imported module or + top-level package, or ‘NULL’ with an exception set on failure. + Like for *note __import__(): 610, the return value when a submodule + of a package was requested is normally the top-level package, + unless a non-empty `fromlist' was given. + + New in version 3.3. + + -- C Function: PyObject* PyImport_ImportModuleLevel (const char *name, + PyObject *globals, PyObject *locals, PyObject *fromlist, + int level) + `Return value: New reference.' Similar to *note + PyImport_ImportModuleLevelObject(): 39a7, but the name is a UTF-8 + encoded string instead of a Unicode object. + + Changed in version 3.3: Negative values for `level' are no longer + accepted. + + -- C Function: PyObject* PyImport_Import (PyObject *name) + `Return value: New reference.' This is a higher-level interface + that calls the current “import hook function” (with an explicit + `level' of 0, meaning absolute import). It invokes the *note + __import__(): 610. function from the ‘__builtins__’ of the current + globals. This means that the import is done using whatever import + hooks are installed in the current environment. + + This function always uses absolute imports. + + -- C Function: PyObject* PyImport_ReloadModule (PyObject *m) + `Return value: New reference.' Reload a module. Return a new + reference to the reloaded module, or ‘NULL’ with an exception set + on failure (the module still exists in this case). + + -- C Function: PyObject* PyImport_AddModuleObject (PyObject *name) + `Return value: Borrowed reference.' Return the module object + corresponding to a module name. The `name' argument may be of the + form ‘package.module’. First check the modules dictionary if + there’s one there, and if not, create a new one and insert it in + the modules dictionary. Return ‘NULL’ with an exception set on + failure. + + Note: This function does not load or import the module; if the + module wasn’t already loaded, you will get an empty module + object. Use *note PyImport_ImportModule(): c73. or one of its + variants to import a module. Package structures implied by a + dotted name for `name' are not created if not already present. + + New in version 3.3. + + -- C Function: PyObject* PyImport_AddModule (const char *name) + `Return value: Borrowed reference.' Similar to *note + PyImport_AddModuleObject(): 39a9, but the name is a UTF-8 encoded + string instead of a Unicode object. + + -- C Function: PyObject* PyImport_ExecCodeModule (const char *name, + PyObject *co) + `Return value: New reference.' + + Given a module name (possibly of the form ‘package.module’) and a + code object read from a Python bytecode file or obtained from the + built-in function *note compile(): 1b4, load the module. Return a + new reference to the module object, or ‘NULL’ with an exception set + if an error occurred. `name' is removed from *note sys.modules: + 1152. in error cases, even if `name' was already in *note + sys.modules: 1152. on entry to *note PyImport_ExecCodeModule(): + 39aa. Leaving incompletely initialized modules in *note + sys.modules: 1152. is dangerous, as imports of such modules have no + way to know that the module object is an unknown (and probably + damaged with respect to the module author’s intents) state. + + The module’s *note __spec__: 116d. and *note __loader__: 956. will + be set, if not set already, with the appropriate values. The + spec’s loader will be set to the module’s ‘__loader__’ (if set) and + to an instance of ‘SourceFileLoader’ otherwise. + + The module’s *note __file__: 10d0. attribute will be set to the + code object’s ‘co_filename’. If applicable, *note __cached__: b32. + will also be set. + + This function will reload the module if it was already imported. + See *note PyImport_ReloadModule(): 39a8. for the intended way to + reload a module. + + If `name' points to a dotted name of the form ‘package.module’, any + package structures not already created will still not be created. + + See also *note PyImport_ExecCodeModuleEx(): 39ab. and *note + PyImport_ExecCodeModuleWithPathnames(): 39ac. + + -- C Function: PyObject* PyImport_ExecCodeModuleEx (const char *name, + PyObject *co, const char *pathname) + `Return value: New reference.' Like *note + PyImport_ExecCodeModule(): 39aa, but the *note __file__: 10d0. + attribute of the module object is set to `pathname' if it is + non-‘NULL’. + + See also *note PyImport_ExecCodeModuleWithPathnames(): 39ac. + + -- C Function: PyObject* PyImport_ExecCodeModuleObject (PyObject *name, + PyObject *co, PyObject *pathname, PyObject *cpathname) + `Return value: New reference.' Like *note + PyImport_ExecCodeModuleEx(): 39ab, but the *note __cached__: b32. + attribute of the module object is set to `cpathname' if it is + non-‘NULL’. Of the three functions, this is the preferred one to + use. + + New in version 3.3. + + -- C Function: PyObject* PyImport_ExecCodeModuleWithPathnames (const + char *name, PyObject *co, const char *pathname, const + char *cpathname) + `Return value: New reference.' Like *note + PyImport_ExecCodeModuleObject(): 39ad, but `name', `pathname' and + `cpathname' are UTF-8 encoded strings. Attempts are also made to + figure out what the value for `pathname' should be from `cpathname' + if the former is set to ‘NULL’. + + New in version 3.2. + + Changed in version 3.3: Uses *note imp.source_from_cache(): 3825. + in calculating the source path if only the bytecode path is + provided. + + -- C Function: long PyImport_GetMagicNumber () + + Return the magic number for Python bytecode files (a.k.a. ‘.pyc’ + file). The magic number should be present in the first four bytes + of the bytecode file, in little-endian byte order. Returns ‘-1’ on + error. + + Changed in version 3.3: Return value of ‘-1’ upon failure. + + -- C Function: const char * PyImport_GetMagicTag () + + Return the magic tag string for PEP 3147(1) format Python bytecode + file names. Keep in mind that the value at + ‘sys.implementation.cache_tag’ is authoritative and should be used + instead of this function. + + New in version 3.2. + + -- C Function: PyObject* PyImport_GetModuleDict () + `Return value: Borrowed reference.' Return the dictionary used for + the module administration (a.k.a. ‘sys.modules’). Note that this + is a per-interpreter variable. + + -- C Function: PyObject* PyImport_GetModule (PyObject *name) + `Return value: New reference.' Return the already imported module + with the given name. If the module has not been imported yet then + returns ‘NULL’ but does not set an error. Returns ‘NULL’ and sets + an error if the lookup failed. + + New in version 3.7. + + -- C Function: PyObject* PyImport_GetImporter (PyObject *path) + `Return value: New reference.' Return a finder object for a *note + sys.path: 488./‘pkg.__path__’ item `path', possibly by fetching it + from the *note sys.path_importer_cache: 49d. dict. If it wasn’t + yet cached, traverse *note sys.path_hooks: 95c. until a hook is + found that can handle the path item. Return ‘None’ if no hook + could; this tells our caller that the *note path based finder: + 1165. could not find a finder for this path item. Cache the result + in *note sys.path_importer_cache: 49d. Return a new reference to + the finder object. + + -- C Function: void _PyImport_Init () + + Initialize the import mechanism. For internal use only. + + -- C Function: void PyImport_Cleanup () + + Empty the module table. For internal use only. + + -- C Function: void _PyImport_Fini () + + Finalize the import mechanism. For internal use only. + + -- C Function: int PyImport_ImportFrozenModuleObject (PyObject *name) + `Return value: New reference.' Load a frozen module named `name'. + Return ‘1’ for success, ‘0’ if the module is not found, and ‘-1’ + with an exception set if the initialization failed. To access the + imported module on a successful load, use *note + PyImport_ImportModule(): c73. (Note the misnomer — this function + would reload the module if it was already imported.) + + New in version 3.3. + + Changed in version 3.4: The ‘__file__’ attribute is no longer set + on the module. + + -- C Function: int PyImport_ImportFrozenModule (const char *name) + + Similar to *note PyImport_ImportFrozenModuleObject(): 39b4, but the + name is a UTF-8 encoded string instead of a Unicode object. + + -- C Type: struct _frozen + + This is the structure type definition for frozen module + descriptors, as generated by the ‘freeze’ utility (see + ‘Tools/freeze/’ in the Python source distribution). Its + definition, found in ‘Include/import.h’, is: + + struct _frozen { + const char *name; + const unsigned char *code; + int size; + }; + + -- C Variable: const struct _frozen* PyImport_FrozenModules + + This pointer is initialized to point to an array of ‘struct + _frozen’ records, terminated by one whose members are all ‘NULL’ or + zero. When a frozen module is imported, it is searched in this + table. Third-party code could play tricks with this to provide a + dynamically created collection of frozen modules. + + -- C Function: int PyImport_AppendInittab (const char *name, PyObject* + (*initfunc)(void)) + + Add a single module to the existing table of built-in modules. + This is a convenience wrapper around *note + PyImport_ExtendInittab(): 39b7, returning ‘-1’ if the table could + not be extended. The new module can be imported by the name + `name', and uses the function `initfunc' as the initialization + function called on the first attempted import. This should be + called before *note Py_Initialize(): 439. + + -- C Type: struct _inittab + + Structure describing a single entry in the list of built-in + modules. Each of these structures gives the name and + initialization function for a module built into the interpreter. + The name is an ASCII encoded string. Programs which embed Python + may use an array of these structures in conjunction with *note + PyImport_ExtendInittab(): 39b7. to provide additional built-in + modules. The structure is defined in ‘Include/import.h’ as: + + struct _inittab { + const char *name; /* ASCII encoded string */ + PyObject* (*initfunc)(void); + }; + + -- C Function: int PyImport_ExtendInittab (struct _inittab *newtab) + + Add a collection of modules to the table of built-in modules. The + `newtab' array must end with a sentinel entry which contains ‘NULL’ + for the ‘name’ field; failure to provide the sentinel value can + result in a memory fault. Returns ‘0’ on success or ‘-1’ if + insufficient memory could be allocated to extend the internal + table. In the event of failure, no modules are added to the + internal table. This should be called before *note + Py_Initialize(): 439. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: Data marshalling support, Next: Parsing arguments and building values, Prev: Importing Modules<2>, Up: Utilities<2> + +7.6.5 Data marshalling support +------------------------------ + +These routines allow C code to work with serialized objects using the +same data format as the *note marshal: b0. module. There are functions +to write data into the serialization format, and additional functions +that can be used to read the data back. Files used to store marshalled +data must be opened in binary mode. + +Numeric values are stored with the least significant byte first. + +The module supports two versions of the data format: version 0 is the +historical version, version 1 shares interned strings in the file, and +upon unmarshalling. Version 2 uses a binary format for floating point +numbers. ‘Py_MARSHAL_VERSION’ indicates the current file format +(currently 2). + + -- C Function: void PyMarshal_WriteLongToFile (long value, FILE *file, + int version) + + Marshal a ‘long’ integer, `value', to `file'. This will only write + the least-significant 32 bits of `value'; regardless of the size of + the native ‘long’ type. `version' indicates the file format. + + -- C Function: void PyMarshal_WriteObjectToFile (PyObject *value, + FILE *file, int version) + + Marshal a Python object, `value', to `file'. `version' indicates + the file format. + + -- C Function: PyObject* PyMarshal_WriteObjectToString + (PyObject *value, int version) + `Return value: New reference.' Return a bytes object containing + the marshalled representation of `value'. `version' indicates the + file format. + +The following functions allow marshalled values to be read back in. + + -- C Function: long PyMarshal_ReadLongFromFile (FILE *file) + + Return a C ‘long’ from the data stream in a ‘FILE*’ opened for + reading. Only a 32-bit value can be read in using this function, + regardless of the native size of ‘long’. + + On error, sets the appropriate exception (*note EOFError: c6c.) and + returns ‘-1’. + + -- C Function: int PyMarshal_ReadShortFromFile (FILE *file) + + Return a C ‘short’ from the data stream in a ‘FILE*’ opened for + reading. Only a 16-bit value can be read in using this function, + regardless of the native size of ‘short’. + + On error, sets the appropriate exception (*note EOFError: c6c.) and + returns ‘-1’. + + -- C Function: PyObject* PyMarshal_ReadObjectFromFile (FILE *file) + `Return value: New reference.' Return a Python object from the + data stream in a ‘FILE*’ opened for reading. + + On error, sets the appropriate exception (*note EOFError: c6c, + *note ValueError: 1fb. or *note TypeError: 192.) and returns + ‘NULL’. + + -- C Function: PyObject* PyMarshal_ReadLastObjectFromFile (FILE *file) + `Return value: New reference.' Return a Python object from the + data stream in a ‘FILE*’ opened for reading. Unlike *note + PyMarshal_ReadObjectFromFile(): 39c1, this function assumes that no + further objects will be read from the file, allowing it to + aggressively load file data into memory so that the + de-serialization can operate from data in memory rather than + reading a byte at a time from the file. Only use these variant if + you are certain that you won’t be reading anything else from the + file. + + On error, sets the appropriate exception (*note EOFError: c6c, + *note ValueError: 1fb. or *note TypeError: 192.) and returns + ‘NULL’. + + -- C Function: PyObject* PyMarshal_ReadObjectFromString (const + char *data, Py_ssize_t len) + `Return value: New reference.' Return a Python object from the + data stream in a byte buffer containing `len' bytes pointed to by + `data'. + + On error, sets the appropriate exception (*note EOFError: c6c, + *note ValueError: 1fb. or *note TypeError: 192.) and returns + ‘NULL’. + + +File: python.info, Node: Parsing arguments and building values, Next: String conversion and formatting, Prev: Data marshalling support, Up: Utilities<2> + +7.6.6 Parsing arguments and building values +------------------------------------------- + +These functions are useful when creating your own extensions functions +and methods. Additional information and examples are available in *note +Extending and Embedding the Python Interpreter: e90. + +The first three of these functions described, *note PyArg_ParseTuple(): +26a, *note PyArg_ParseTupleAndKeywords(): 5ca, and *note PyArg_Parse(): +39c6, all use `format strings' which are used to tell the function about +the expected arguments. The format strings use the same syntax for each +of these functions. + +* Menu: + +* Parsing arguments: Parsing arguments<3>. +* Building values:: + + +File: python.info, Node: Parsing arguments<3>, Next: Building values, Up: Parsing arguments and building values + +7.6.6.1 Parsing arguments +......................... + +A format string consists of zero or more “format units.” A format unit +describes one Python object; it is usually a single character or a +parenthesized sequence of format units. With a few exceptions, a format +unit that is not a parenthesized sequence normally corresponds to a +single address argument to these functions. In the following +description, the quoted form is the format unit; the entry in (round) +parentheses is the Python object type that matches the format unit; and +the entry in [square] brackets is the type of the C variable(s) whose +address should be passed. + +* Menu: + +* Strings and buffers:: +* Numbers: Numbers<2>. +* Other objects:: +* API Functions:: + + +File: python.info, Node: Strings and buffers, Next: Numbers<2>, Up: Parsing arguments<3> + +7.6.6.2 Strings and buffers +........................... + +These formats allow accessing an object as a contiguous chunk of memory. +You don’t have to provide raw storage for the returned unicode or bytes +area. + +In general, when a format sets a pointer to a buffer, the buffer is +managed by the corresponding Python object, and the buffer shares the +lifetime of this object. You won’t have to release any memory yourself. +The only exceptions are ‘es’, ‘es#’, ‘et’ and ‘et#’. + +However, when a *note Py_buffer: b1b. structure gets filled, the +underlying buffer is locked so that the caller can subsequently use the +buffer even inside a *note Py_BEGIN_ALLOW_THREADS: 3866. block without +the risk of mutable data being resized or destroyed. As a result, `you +have to call' *note PyBuffer_Release(): d54. after you have finished +processing the data (or in any early abort case). + +Unless otherwise stated, buffers are not NUL-terminated. + +Some formats require a read-only *note bytes-like object: 5e8, and set a +pointer instead of a buffer structure. They work by checking that the +object’s *note PyBufferProcs.bf_releasebuffer: 39c9. field is ‘NULL’, +which disallows mutable objects such as *note bytearray: 332. + + Note: For all ‘#’ variants of formats (‘s#’, ‘y#’, etc.), the type + of the length argument (int or ‘Py_ssize_t’) is controlled by + defining the macro ‘PY_SSIZE_T_CLEAN’ before including ‘Python.h’. + If the macro was defined, length is a ‘Py_ssize_t’ rather than an + ‘int’. This behavior will change in a future Python version to + only support ‘Py_ssize_t’ and drop ‘int’ support. It is best to + always define ‘PY_SSIZE_T_CLEAN’. + +‘s’ (*note str: 330.) [const char *] + + Convert a Unicode object to a C pointer to a character string. A + pointer to an existing string is stored in the character pointer + variable whose address you pass. The C string is NUL-terminated. + The Python string must not contain embedded null code points; if it + does, a *note ValueError: 1fb. exception is raised. Unicode + objects are converted to C strings using ‘'utf-8'’ encoding. If + this conversion fails, a *note UnicodeError: c3b. is raised. + + Note: This format does not accept *note bytes-like objects: + 5e8. If you want to accept filesystem paths and convert them + to C character strings, it is preferable to use the ‘O&’ + format with *note PyUnicode_FSConverter(): 5ce. as + `converter'. + + Changed in version 3.5: Previously, *note TypeError: 192. was + raised when embedded null code points were encountered in the + Python string. + +‘s*’ (*note str: 330. or *note bytes-like object: 5e8.) [Py_buffer] + + This format accepts Unicode objects as well as bytes-like objects. + It fills a *note Py_buffer: b1b. structure provided by the caller. + In this case the resulting C string may contain embedded NUL bytes. + Unicode objects are converted to C strings using ‘'utf-8'’ + encoding. + +‘s#’ (*note str: 330, read-only *note bytes-like object: 5e8.) [const char *, int or ‘Py_ssize_t’] + + Like ‘s*’, except that it doesn’t accept mutable objects. The + result is stored into two C variables, the first one a pointer to a + C string, the second one its length. The string may contain + embedded null bytes. Unicode objects are converted to C strings + using ‘'utf-8'’ encoding. + +‘z’ (*note str: 330. or ‘None’) [const char *] + + Like ‘s’, but the Python object may also be ‘None’, in which case + the C pointer is set to ‘NULL’. + +‘z*’ (*note str: 330, *note bytes-like object: 5e8. or ‘None’) [Py_buffer] + + Like ‘s*’, but the Python object may also be ‘None’, in which case + the ‘buf’ member of the *note Py_buffer: b1b. structure is set to + ‘NULL’. + +‘z#’ (*note str: 330, read-only *note bytes-like object: 5e8. or ‘None’) [const char *, int or ‘Py_ssize_t’] + + Like ‘s#’, but the Python object may also be ‘None’, in which case + the C pointer is set to ‘NULL’. + +‘y’ (read-only *note bytes-like object: 5e8.) [const char *] + + This format converts a bytes-like object to a C pointer to a + character string; it does not accept Unicode objects. The bytes + buffer must not contain embedded null bytes; if it does, a *note + ValueError: 1fb. exception is raised. + + Changed in version 3.5: Previously, *note TypeError: 192. was + raised when embedded null bytes were encountered in the bytes + buffer. + +‘y*’ (*note bytes-like object: 5e8.) [Py_buffer] + + This variant on ‘s*’ doesn’t accept Unicode objects, only + bytes-like objects. `This is the recommended way to accept binary + data.' + +‘y#’ (read-only *note bytes-like object: 5e8.) [const char *, int or ‘Py_ssize_t’] + + This variant on ‘s#’ doesn’t accept Unicode objects, only + bytes-like objects. + +‘S’ (*note bytes: 331.) [PyBytesObject *] + + Requires that the Python object is a *note bytes: 331. object, + without attempting any conversion. Raises *note TypeError: 192. if + the object is not a bytes object. The C variable may also be + declared as *note PyObject*: 4ba. + +‘Y’ (*note bytearray: 332.) [PyByteArrayObject *] + + Requires that the Python object is a *note bytearray: 332. object, + without attempting any conversion. Raises *note TypeError: 192. if + the object is not a *note bytearray: 332. object. The C variable + may also be declared as *note PyObject*: 4ba. + +‘u’ (*note str: 330.) [const Py_UNICODE *] + + Convert a Python Unicode object to a C pointer to a NUL-terminated + buffer of Unicode characters. You must pass the address of a *note + Py_UNICODE: aee. pointer variable, which will be filled with the + pointer to an existing Unicode buffer. Please note that the width + of a *note Py_UNICODE: aee. character depends on compilation + options (it is either 16 or 32 bits). The Python string must not + contain embedded null code points; if it does, a *note ValueError: + 1fb. exception is raised. + + Changed in version 3.5: Previously, *note TypeError: 192. was + raised when embedded null code points were encountered in the + Python string. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsWideCharString(): 438. + +‘u#’ (*note str: 330.) [const Py_UNICODE *, int or ‘Py_ssize_t’] + + This variant on ‘u’ stores into two C variables, the first one a + pointer to a Unicode data buffer, the second one its length. This + variant allows null code points. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsWideCharString(): 438. + +‘Z’ (*note str: 330. or ‘None’) [const Py_UNICODE *] + + Like ‘u’, but the Python object may also be ‘None’, in which case + the *note Py_UNICODE: aee. pointer is set to ‘NULL’. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsWideCharString(): 438. + +‘Z#’ (*note str: 330. or ‘None’) [const Py_UNICODE *, int or ‘Py_ssize_t’] + + Like ‘u#’, but the Python object may also be ‘None’, in which case + the *note Py_UNICODE: aee. pointer is set to ‘NULL’. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsWideCharString(): 438. + +‘U’ (*note str: 330.) [PyObject *] + + Requires that the Python object is a Unicode object, without + attempting any conversion. Raises *note TypeError: 192. if the + object is not a Unicode object. The C variable may also be + declared as *note PyObject*: 4ba. + +‘w*’ (read-write *note bytes-like object: 5e8.) [Py_buffer] + + This format accepts any object which implements the read-write + buffer interface. It fills a *note Py_buffer: b1b. structure + provided by the caller. The buffer may contain embedded null + bytes. The caller have to call *note PyBuffer_Release(): d54. when + it is done with the buffer. + +‘es’ (*note str: 330.) [const char *encoding, char **buffer] + + This variant on ‘s’ is used for encoding Unicode into a character + buffer. It only works for encoded data without embedded NUL bytes. + + This format requires two arguments. The first is only used as + input, and must be a ‘const char*’ which points to the name of an + encoding as a NUL-terminated string, or ‘NULL’, in which case + ‘'utf-8'’ encoding is used. An exception is raised if the named + encoding is not known to Python. The second argument must be a + ‘char**’; the value of the pointer it references will be set to a + buffer with the contents of the argument text. The text will be + encoded in the encoding specified by the first argument. + + *note PyArg_ParseTuple(): 26a. will allocate a buffer of the needed + size, copy the encoded data into this buffer and adjust `*buffer' + to reference the newly allocated storage. The caller is + responsible for calling *note PyMem_Free(): da9. to free the + allocated buffer after use. + +‘et’ (*note str: 330, *note bytes: 331. or *note bytearray: 332.) [const char *encoding, char **buffer] + + Same as ‘es’ except that byte string objects are passed through + without recoding them. Instead, the implementation assumes that + the byte string object uses the encoding passed in as parameter. + +‘es#’ (*note str: 330.) [const char *encoding, char **buffer, int or ‘Py_ssize_t’ *buffer_length] + + This variant on ‘s#’ is used for encoding Unicode into a character + buffer. Unlike the ‘es’ format, this variant allows input data + which contains NUL characters. + + It requires three arguments. The first is only used as input, and + must be a ‘const char*’ which points to the name of an encoding as + a NUL-terminated string, or ‘NULL’, in which case ‘'utf-8'’ + encoding is used. An exception is raised if the named encoding is + not known to Python. The second argument must be a ‘char**’; the + value of the pointer it references will be set to a buffer with the + contents of the argument text. The text will be encoded in the + encoding specified by the first argument. The third argument must + be a pointer to an integer; the referenced integer will be set to + the number of bytes in the output buffer. + + There are two modes of operation: + + If `*buffer' points a ‘NULL’ pointer, the function will allocate a + buffer of the needed size, copy the encoded data into this buffer + and set `*buffer' to reference the newly allocated storage. The + caller is responsible for calling *note PyMem_Free(): da9. to free + the allocated buffer after usage. + + If `*buffer' points to a non-‘NULL’ pointer (an already allocated + buffer), *note PyArg_ParseTuple(): 26a. will use this location as + the buffer and interpret the initial value of `*buffer_length' as + the buffer size. It will then copy the encoded data into the + buffer and NUL-terminate it. If the buffer is not large enough, a + *note ValueError: 1fb. will be set. + + In both cases, `*buffer_length' is set to the length of the encoded + data without the trailing NUL byte. + +‘et#’ (*note str: 330, *note bytes: 331. or *note bytearray: 332.) [const char *encoding, char **buffer, int or ‘Py_ssize_t’ *buffer_length] + + Same as ‘es#’ except that byte string objects are passed through + without recoding them. Instead, the implementation assumes that + the byte string object uses the encoding passed in as parameter. + + +File: python.info, Node: Numbers<2>, Next: Other objects, Prev: Strings and buffers, Up: Parsing arguments<3> + +7.6.6.3 Numbers +............... + +‘b’ (*note int: 184.) [unsigned char] + + Convert a nonnegative Python integer to an unsigned tiny int, + stored in a C ‘unsigned char’. + +‘B’ (*note int: 184.) [unsigned char] + + Convert a Python integer to a tiny int without overflow checking, + stored in a C ‘unsigned char’. + +‘h’ (*note int: 184.) [short int] + + Convert a Python integer to a C ‘short int’. + +‘H’ (*note int: 184.) [unsigned short int] + + Convert a Python integer to a C ‘unsigned short int’, without + overflow checking. + +‘i’ (*note int: 184.) [int] + + Convert a Python integer to a plain C ‘int’. + +‘I’ (*note int: 184.) [unsigned int] + + Convert a Python integer to a C ‘unsigned int’, without overflow + checking. + +‘l’ (*note int: 184.) [long int] + + Convert a Python integer to a C ‘long int’. + +‘k’ (*note int: 184.) [unsigned long] + + Convert a Python integer to a C ‘unsigned long’ without overflow + checking. + +‘L’ (*note int: 184.) [long long] + + Convert a Python integer to a C ‘long long’. + +‘K’ (*note int: 184.) [unsigned long long] + + Convert a Python integer to a C ‘unsigned long long’ without + overflow checking. + +‘n’ (*note int: 184.) [Py_ssize_t] + + Convert a Python integer to a C ‘Py_ssize_t’. + +‘c’ (*note bytes: 331. or *note bytearray: 332. of length 1) [char] + + Convert a Python byte, represented as a *note bytes: 331. or *note + bytearray: 332. object of length 1, to a C ‘char’. + + Changed in version 3.3: Allow *note bytearray: 332. objects. + +‘C’ (*note str: 330. of length 1) [int] + + Convert a Python character, represented as a *note str: 330. object + of length 1, to a C ‘int’. + +‘f’ (*note float: 187.) [float] + + Convert a Python floating point number to a C ‘float’. + +‘d’ (*note float: 187.) [double] + + Convert a Python floating point number to a C ‘double’. + +‘D’ (*note complex: 189.) [Py_complex] + + Convert a Python complex number to a C *note Py_complex: 39cb. + structure. + + +File: python.info, Node: Other objects, Next: API Functions, Prev: Numbers<2>, Up: Parsing arguments<3> + +7.6.6.4 Other objects +..................... + +‘O’ (object) [PyObject *] + + Store a Python object (without any conversion) in a C object + pointer. The C program thus receives the actual object that was + passed. The object’s reference count is not increased. The + pointer stored is not ‘NULL’. + +‘O!’ (object) [`typeobject', PyObject *] + + Store a Python object in a C object pointer. This is similar to + ‘O’, but takes two C arguments: the first is the address of a + Python type object, the second is the address of the C variable (of + type *note PyObject*: 4ba.) into which the object pointer is + stored. If the Python object does not have the required type, + *note TypeError: 192. is raised. + +‘O&’ (object) [`converter', `anything'] + + Convert a Python object to a C variable through a `converter' + function. This takes two arguments: the first is a function, the + second is the address of a C variable (of arbitrary type), + converted to ‘void *’. The `converter' function in turn is called + as follows: + + status = converter(object, address); + + where `object' is the Python object to be converted and `address' + is the ‘void*’ argument that was passed to the *note + PyArg_Parse*(): 39c6. function. The returned `status' should be + ‘1’ for a successful conversion and ‘0’ if the conversion has + failed. When the conversion fails, the `converter' function should + raise an exception and leave the content of `address' unmodified. + + If the `converter' returns ‘Py_CLEANUP_SUPPORTED’, it may get + called a second time if the argument parsing eventually fails, + giving the converter a chance to release any memory that it had + already allocated. In this second call, the `object' parameter + will be ‘NULL’; `address' will have the same value as in the + original call. + + Changed in version 3.1: ‘Py_CLEANUP_SUPPORTED’ was added. + +‘p’ (*note bool: 183.) [int] + + Tests the value passed in for truth (a boolean `p'redicate) and + converts the result to its equivalent C true/false integer value. + Sets the int to ‘1’ if the expression was true and ‘0’ if it was + false. This accepts any valid Python value. See *note Truth Value + Testing: 128d. for more information about how Python tests values + for truth. + + New in version 3.3. + +‘(items)’ (*note tuple: 47e.) [`matching-items'] + + The object must be a Python sequence whose length is the number of + format units in `items'. The C arguments must correspond to the + individual format units in `items'. Format units for sequences may + be nested. + +It is possible to pass “long” integers (integers whose value exceeds the +platform’s ‘LONG_MAX’) however no proper range checking is done — the +most significant bits are silently truncated when the receiving field is +too small to receive the value (actually, the semantics are inherited +from downcasts in C — your mileage may vary). + +A few other characters have a meaning in a format string. These may not +occur inside nested parentheses. They are: + +‘|’ + + Indicates that the remaining arguments in the Python argument list + are optional. The C variables corresponding to optional arguments + should be initialized to their default value — when an optional + argument is not specified, *note PyArg_ParseTuple(): 26a. does not + touch the contents of the corresponding C variable(s). + +‘$’ + + *note PyArg_ParseTupleAndKeywords(): 5ca. only: Indicates that the + remaining arguments in the Python argument list are keyword-only. + Currently, all keyword-only arguments must also be optional + arguments, so ‘|’ must always be specified before ‘$’ in the format + string. + + New in version 3.3. + +‘:’ + + The list of format units ends here; the string after the colon is + used as the function name in error messages (the “associated value” + of the exception that *note PyArg_ParseTuple(): 26a. raises). + +‘;’ + + The list of format units ends here; the string after the semicolon + is used as the error message `instead' of the default error + message. ‘:’ and ‘;’ mutually exclude each other. + +Note that any Python object references which are provided to the caller +are `borrowed' references; do not decrement their reference count! + +Additional arguments passed to these functions must be addresses of +variables whose type is determined by the format string; these are used +to store values from the input tuple. There are a few cases, as +described in the list of format units above, where these parameters are +used as input values; they should match what is specified for the +corresponding format unit in that case. + +For the conversion to succeed, the `arg' object must match the format +and the format must be exhausted. On success, the *note PyArg_Parse*(): +39c6. functions return true, otherwise they return false and raise an +appropriate exception. When the *note PyArg_Parse*(): 39c6. functions +fail due to conversion failure in one of the format units, the variables +at the addresses corresponding to that and the following format units +are left untouched. + + +File: python.info, Node: API Functions, Prev: Other objects, Up: Parsing arguments<3> + +7.6.6.5 API Functions +..................... + + -- C Function: int PyArg_ParseTuple (PyObject *args, const + char *format, ...) + + Parse the parameters of a function that takes only positional + parameters into local variables. Returns true on success; on + failure, it returns false and raises the appropriate exception. + + -- C Function: int PyArg_VaParse (PyObject *args, const char *format, + va_list vargs) + + Identical to *note PyArg_ParseTuple(): 26a, except that it accepts + a va_list rather than a variable number of arguments. + + -- C Function: int PyArg_ParseTupleAndKeywords (PyObject *args, + PyObject *kw, const char *format, char *keywords[], ...) + + Parse the parameters of a function that takes both positional and + keyword parameters into local variables. The `keywords' argument + is a ‘NULL’-terminated array of keyword parameter names. Empty + names denote *note positional-only parameters: 2a7. Returns true + on success; on failure, it returns false and raises the appropriate + exception. + + Changed in version 3.6: Added support for *note positional-only + parameters: 2a7. + + -- C Function: int PyArg_VaParseTupleAndKeywords (PyObject *args, + PyObject *kw, const char *format, char *keywords[], + va_list vargs) + + Identical to *note PyArg_ParseTupleAndKeywords(): 5ca, except that + it accepts a va_list rather than a variable number of arguments. + + -- C Function: int PyArg_ValidateKeywordArguments (PyObject *) + + Ensure that the keys in the keywords argument dictionary are + strings. This is only needed if *note + PyArg_ParseTupleAndKeywords(): 5ca. is not used, since the latter + already does this check. + + New in version 3.2. + + -- C Function: int PyArg_Parse (PyObject *args, const char *format, + ...) + + Function used to deconstruct the argument lists of “old-style” + functions — these are functions which use the ‘METH_OLDARGS’ + parameter parsing method, which has been removed in Python 3. This + is not recommended for use in parameter parsing in new code, and + most code in the standard interpreter has been modified to no + longer use this for that purpose. It does remain a convenient way + to decompose other tuples, however, and may continue to be used for + that purpose. + + -- C Function: int PyArg_UnpackTuple (PyObject *args, const char *name, + Py_ssize_t min, Py_ssize_t max, ...) + + A simpler form of parameter retrieval which does not use a format + string to specify the types of the arguments. Functions which use + this method to retrieve their parameters should be declared as + *note METH_VARARGS: e48. in function or method tables. The tuple + containing the actual parameters should be passed as `args'; it + must actually be a tuple. The length of the tuple must be at least + `min' and no more than `max'; `min' and `max' may be equal. + Additional arguments must be passed to the function, each of which + should be a pointer to a *note PyObject*: 4ba. variable; these will + be filled in with the values from `args'; they will contain + borrowed references. The variables which correspond to optional + parameters not given by `args' will not be filled in; these should + be initialized by the caller. This function returns true on + success and false if `args' is not a tuple or contains the wrong + number of elements; an exception will be set if there was a + failure. + + This is an example of the use of this function, taken from the + sources for the ‘_weakref’ helper module for weak references: + + static PyObject * + weakref_ref(PyObject *self, PyObject *args) + { + PyObject *object; + PyObject *callback = NULL; + PyObject *result = NULL; + + if (PyArg_UnpackTuple(args, "ref", 1, 2, &object, &callback)) { + result = PyWeakref_NewRef(object, callback); + } + return result; + } + + The call to *note PyArg_UnpackTuple(): e46. in this example is + entirely equivalent to this call to *note PyArg_ParseTuple(): 26a.: + + PyArg_ParseTuple(args, "O|O:ref", &object, &callback) + + +File: python.info, Node: Building values, Prev: Parsing arguments<3>, Up: Parsing arguments and building values + +7.6.6.6 Building values +....................... + + -- C Function: PyObject* Py_BuildValue (const char *format, ...) + `Return value: New reference.' Create a new value based on a + format string similar to those accepted by the *note + PyArg_Parse*(): 39c6. family of functions and a sequence of values. + Returns the value or ‘NULL’ in the case of an error; an exception + will be raised if ‘NULL’ is returned. + + *note Py_BuildValue(): 2ce. does not always build a tuple. It + builds a tuple only if its format string contains two or more + format units. If the format string is empty, it returns ‘None’; if + it contains exactly one format unit, it returns whatever object is + described by that format unit. To force it to return a tuple of + size 0 or one, parenthesize the format string. + + When memory buffers are passed as parameters to supply data to + build objects, as for the ‘s’ and ‘s#’ formats, the required data + is copied. Buffers provided by the caller are never referenced by + the objects created by *note Py_BuildValue(): 2ce. In other words, + if your code invokes ‘malloc()’ and passes the allocated memory to + *note Py_BuildValue(): 2ce, your code is responsible for calling + ‘free()’ for that memory once *note Py_BuildValue(): 2ce. returns. + + In the following description, the quoted form is the format unit; + the entry in (round) parentheses is the Python object type that the + format unit will return; and the entry in [square] brackets is the + type of the C value(s) to be passed. + + The characters space, tab, colon and comma are ignored in format + strings (but not within format units such as ‘s#’). This can be + used to make long format strings a tad more readable. + + ‘s’ (*note str: 330. or ‘None’) [const char *] + + Convert a null-terminated C string to a Python *note str: 330. + object using ‘'utf-8'’ encoding. If the C string pointer is + ‘NULL’, ‘None’ is used. + + ‘s#’ (*note str: 330. or ‘None’) [const char *, int or ‘Py_ssize_t’] + + Convert a C string and its length to a Python *note str: 330. + object using ‘'utf-8'’ encoding. If the C string pointer is + ‘NULL’, the length is ignored and ‘None’ is returned. + + ‘y’ (*note bytes: 331.) [const char *] + + This converts a C string to a Python *note bytes: 331. object. + If the C string pointer is ‘NULL’, ‘None’ is returned. + + ‘y#’ (*note bytes: 331.) [const char *, int or ‘Py_ssize_t’] + + This converts a C string and its lengths to a Python object. + If the C string pointer is ‘NULL’, ‘None’ is returned. + + ‘z’ (*note str: 330. or ‘None’) [const char *] + + Same as ‘s’. + + ‘z#’ (*note str: 330. or ‘None’) [const char *, int or ‘Py_ssize_t’] + + Same as ‘s#’. + + ‘u’ (*note str: 330.) [const wchar_t *] + + Convert a null-terminated ‘wchar_t’ buffer of Unicode (UTF-16 + or UCS-4) data to a Python Unicode object. If the Unicode + buffer pointer is ‘NULL’, ‘None’ is returned. + + ‘u#’ (*note str: 330.) [const wchar_t *, int or ‘Py_ssize_t’] + + Convert a Unicode (UTF-16 or UCS-4) data buffer and its length + to a Python Unicode object. If the Unicode buffer pointer is + ‘NULL’, the length is ignored and ‘None’ is returned. + + ‘U’ (*note str: 330. or ‘None’) [const char *] + + Same as ‘s’. + + ‘U#’ (*note str: 330. or ‘None’) [const char *, int or ‘Py_ssize_t’] + + Same as ‘s#’. + + ‘i’ (*note int: 184.) [int] + + Convert a plain C ‘int’ to a Python integer object. + + ‘b’ (*note int: 184.) [char] + + Convert a plain C ‘char’ to a Python integer object. + + ‘h’ (*note int: 184.) [short int] + + Convert a plain C ‘short int’ to a Python integer object. + + ‘l’ (*note int: 184.) [long int] + + Convert a C ‘long int’ to a Python integer object. + + ‘B’ (*note int: 184.) [unsigned char] + + Convert a C ‘unsigned char’ to a Python integer object. + + ‘H’ (*note int: 184.) [unsigned short int] + + Convert a C ‘unsigned short int’ to a Python integer object. + + ‘I’ (*note int: 184.) [unsigned int] + + Convert a C ‘unsigned int’ to a Python integer object. + + ‘k’ (*note int: 184.) [unsigned long] + + Convert a C ‘unsigned long’ to a Python integer object. + + ‘L’ (*note int: 184.) [long long] + + Convert a C ‘long long’ to a Python integer object. + + ‘K’ (*note int: 184.) [unsigned long long] + + Convert a C ‘unsigned long long’ to a Python integer object. + + ‘n’ (*note int: 184.) [Py_ssize_t] + + Convert a C ‘Py_ssize_t’ to a Python integer. + + ‘c’ (*note bytes: 331. of length 1) [char] + + Convert a C ‘int’ representing a byte to a Python *note bytes: + 331. object of length 1. + + ‘C’ (*note str: 330. of length 1) [int] + + Convert a C ‘int’ representing a character to Python *note + str: 330. object of length 1. + + ‘d’ (*note float: 187.) [double] + + Convert a C ‘double’ to a Python floating point number. + + ‘f’ (*note float: 187.) [float] + + Convert a C ‘float’ to a Python floating point number. + + ‘D’ (*note complex: 189.) [Py_complex *] + + Convert a C *note Py_complex: 39cb. structure to a Python + complex number. + + ‘O’ (object) [PyObject *] + + Pass a Python object untouched (except for its reference + count, which is incremented by one). If the object passed in + is a ‘NULL’ pointer, it is assumed that this was caused + because the call producing the argument found an error and set + an exception. Therefore, *note Py_BuildValue(): 2ce. will + return ‘NULL’ but won’t raise an exception. If no exception + has been raised yet, *note SystemError: 5fb. is set. + + ‘S’ (object) [PyObject *] + + Same as ‘O’. + + ‘N’ (object) [PyObject *] + + Same as ‘O’, except it doesn’t increment the reference count + on the object. Useful when the object is created by a call to + an object constructor in the argument list. + + ‘O&’ (object) [`converter', `anything'] + + Convert `anything' to a Python object through a `converter' + function. The function is called with `anything' (which + should be compatible with ‘void*’) as its argument and should + return a “new” Python object, or ‘NULL’ if an error occurred. + + ‘(items)’ (*note tuple: 47e.) [`matching-items'] + + Convert a sequence of C values to a Python tuple with the same + number of items. + + ‘[items]’ (*note list: 262.) [`matching-items'] + + Convert a sequence of C values to a Python list with the same + number of items. + + ‘{items}’ (*note dict: 1b8.) [`matching-items'] + + Convert a sequence of C values to a Python dictionary. Each + pair of consecutive C values adds one item to the dictionary, + serving as key and value, respectively. + + If there is an error in the format string, the *note SystemError: + 5fb. exception is set and ‘NULL’ returned. + + -- C Function: PyObject* Py_VaBuildValue (const char *format, + va_list vargs) + `Return value: New reference.' Identical to *note Py_BuildValue(): + 2ce, except that it accepts a va_list rather than a variable number + of arguments. + + +File: python.info, Node: String conversion and formatting, Next: Reflection, Prev: Parsing arguments and building values, Up: Utilities<2> + +7.6.7 String conversion and formatting +-------------------------------------- + +Functions for number conversion and formatted string output. + + -- C Function: int PyOS_snprintf (char *str, size_t size, const + char *format, ...) + + Output not more than `size' bytes to `str' according to the format + string `format' and the extra arguments. See the Unix man page + ‘snprintf(3)’. + + -- C Function: int PyOS_vsnprintf (char *str, size_t size, const + char *format, va_list va) + + Output not more than `size' bytes to `str' according to the format + string `format' and the variable argument list `va'. Unix man page + ‘vsnprintf(3)’. + +*note PyOS_snprintf(): e49. and *note PyOS_vsnprintf(): e4a. wrap the +Standard C library functions ‘snprintf()’ and ‘vsnprintf()’. Their +purpose is to guarantee consistent behavior in corner cases, which the +Standard C functions do not. + +The wrappers ensure that ‘str[size-1]’ is always ‘'\0'’ upon return. +They never write more than `size' bytes (including the trailing ‘'\0'’) +into str. Both functions require that ‘str != NULL’, ‘size > 0’ and +‘format != NULL’. + +If the platform doesn’t have ‘vsnprintf()’ and the buffer size needed to +avoid truncation exceeds `size' by more than 512 bytes, Python aborts +with a *note Py_FatalError(): 39a2. + +The return value (`rv') for these functions should be interpreted as +follows: + + * When ‘0 <= rv < size’, the output conversion was successful and + `rv' characters were written to `str' (excluding the trailing + ‘'\0'’ byte at ‘str[rv]’). + + * When ‘rv >= size’, the output conversion was truncated and a buffer + with ‘rv + 1’ bytes would have been needed to succeed. + ‘str[size-1]’ is ‘'\0'’ in this case. + + * When ‘rv < 0’, “something bad happened.” ‘str[size-1]’ is ‘'\0'’ in + this case too, but the rest of `str' is undefined. The exact cause + of the error depends on the underlying platform. + +The following functions provide locale-independent string to number +conversions. + + -- C Function: double PyOS_string_to_double (const char *s, + char **endptr, PyObject *overflow_exception) + + Convert a string ‘s’ to a ‘double’, raising a Python exception on + failure. The set of accepted strings corresponds to the set of + strings accepted by Python’s *note float(): 187. constructor, + except that ‘s’ must not have leading or trailing whitespace. The + conversion is independent of the current locale. + + If ‘endptr’ is ‘NULL’, convert the whole string. Raise *note + ValueError: 1fb. and return ‘-1.0’ if the string is not a valid + representation of a floating-point number. + + If endptr is not ‘NULL’, convert as much of the string as possible + and set ‘*endptr’ to point to the first unconverted character. If + no initial segment of the string is the valid representation of a + floating-point number, set ‘*endptr’ to point to the beginning of + the string, raise ValueError, and return ‘-1.0’. + + If ‘s’ represents a value that is too large to store in a float + (for example, ‘"1e500"’ is such a string on many platforms) then if + ‘overflow_exception’ is ‘NULL’ return ‘Py_HUGE_VAL’ (with an + appropriate sign) and don’t set any exception. Otherwise, + ‘overflow_exception’ must point to a Python exception object; raise + that exception and return ‘-1.0’. In both cases, set ‘*endptr’ to + point to the first character after the converted value. + + If any other error occurs during the conversion (for example an + out-of-memory error), set the appropriate Python exception and + return ‘-1.0’. + + New in version 3.1. + + -- C Function: char* PyOS_double_to_string (double val, + char format_code, int precision, int flags, int *ptype) + + Convert a ‘double’ `val' to a string using supplied `format_code', + `precision', and `flags'. + + `format_code' must be one of ‘'e'’, ‘'E'’, ‘'f'’, ‘'F'’, ‘'g'’, + ‘'G'’ or ‘'r'’. For ‘'r'’, the supplied `precision' must be 0 and + is ignored. The ‘'r'’ format code specifies the standard *note + repr(): 7cc. format. + + `flags' can be zero or more of the values ‘Py_DTSF_SIGN’, + ‘Py_DTSF_ADD_DOT_0’, or ‘Py_DTSF_ALT’, or-ed together: + + * ‘Py_DTSF_SIGN’ means to always precede the returned string + with a sign character, even if `val' is non-negative. + + * ‘Py_DTSF_ADD_DOT_0’ means to ensure that the returned string + will not look like an integer. + + * ‘Py_DTSF_ALT’ means to apply “alternate” formatting rules. + See the documentation for the *note PyOS_snprintf(): e49. + ‘'#'’ specifier for details. + + If `ptype' is non-‘NULL’, then the value it points to will be set + to one of ‘Py_DTST_FINITE’, ‘Py_DTST_INFINITE’, or ‘Py_DTST_NAN’, + signifying that `val' is a finite number, an infinite number, or + not a number, respectively. + + The return value is a pointer to `buffer' with the converted string + or ‘NULL’ if the conversion failed. The caller is responsible for + freeing the returned string by calling *note PyMem_Free(): da9. + + New in version 3.1. + + -- C Function: int PyOS_stricmp (const char *s1, const char *s2) + + Case insensitive comparison of strings. The function works almost + identically to ‘strcmp()’ except that it ignores the case. + + -- C Function: int PyOS_strnicmp (const char *s1, const char *s2, + Py_ssize_t size) + + Case insensitive comparison of strings. The function works almost + identically to ‘strncmp()’ except that it ignores the case. + + +File: python.info, Node: Reflection, Next: Codec registry and support functions, Prev: String conversion and formatting, Up: Utilities<2> + +7.6.8 Reflection +---------------- + + -- C Function: PyObject* PyEval_GetBuiltins () + `Return value: Borrowed reference.' Return a dictionary of the + builtins in the current execution frame, or the interpreter of the + thread state if no frame is currently executing. + + -- C Function: PyObject* PyEval_GetLocals () + `Return value: Borrowed reference.' Return a dictionary of the + local variables in the current execution frame, or ‘NULL’ if no + frame is currently executing. + + -- C Function: PyObject* PyEval_GetGlobals () + `Return value: Borrowed reference.' Return a dictionary of the + global variables in the current execution frame, or ‘NULL’ if no + frame is currently executing. + + -- C Function: PyFrameObject* PyEval_GetFrame () + `Return value: Borrowed reference.' Return the current thread + state’s frame, which is ‘NULL’ if no frame is currently executing. + + -- C Function: int PyFrame_GetLineNumber (PyFrameObject *frame) + + Return the line number that `frame' is currently executing. + + -- C Function: const char* PyEval_GetFuncName (PyObject *func) + + Return the name of `func' if it is a function, class or instance + object, else the name of `func's type. + + -- C Function: const char* PyEval_GetFuncDesc (PyObject *func) + + Return a description string, depending on the type of `func'. + Return values include “()” for functions and methods, ” + constructor”, ” instance”, and ” object”. Concatenated with the + result of *note PyEval_GetFuncName(): 39e0, the result will be a + description of `func'. + + +File: python.info, Node: Codec registry and support functions, Prev: Reflection, Up: Utilities<2> + +7.6.9 Codec registry and support functions +------------------------------------------ + + -- C Function: int PyCodec_Register (PyObject *search_function) + + Register a new codec search function. + + As side effect, this tries to load the ‘encodings’ package, if not + yet done, to make sure that it is always first in the list of + search functions. + + -- C Function: int PyCodec_KnownEncoding (const char *encoding) + + Return ‘1’ or ‘0’ depending on whether there is a registered codec + for the given `encoding'. This function always succeeds. + + -- C Function: PyObject* PyCodec_Encode (PyObject *object, const + char *encoding, const char *errors) + `Return value: New reference.' Generic codec based encoding API. + + `object' is passed through the encoder function found for the given + `encoding' using the error handling method defined by `errors'. + `errors' may be ‘NULL’ to use the default method defined for the + codec. Raises a *note LookupError: 1308. if no encoder can be + found. + + -- C Function: PyObject* PyCodec_Decode (PyObject *object, const + char *encoding, const char *errors) + `Return value: New reference.' Generic codec based decoding API. + + `object' is passed through the decoder function found for the given + `encoding' using the error handling method defined by `errors'. + `errors' may be ‘NULL’ to use the default method defined for the + codec. Raises a *note LookupError: 1308. if no encoder can be + found. + +* Menu: + +* Codec lookup API:: +* Registry API for Unicode encoding error handlers:: + + +File: python.info, Node: Codec lookup API, Next: Registry API for Unicode encoding error handlers, Up: Codec registry and support functions + +7.6.9.1 Codec lookup API +........................ + +In the following functions, the `encoding' string is looked up converted +to all lower-case characters, which makes encodings looked up through +this mechanism effectively case-insensitive. If no codec is found, a +*note KeyError: 2c7. is set and ‘NULL’ returned. + + -- C Function: PyObject* PyCodec_Encoder (const char *encoding) + `Return value: New reference.' Get an encoder function for the + given `encoding'. + + -- C Function: PyObject* PyCodec_Decoder (const char *encoding) + `Return value: New reference.' Get a decoder function for the + given `encoding'. + + -- C Function: PyObject* PyCodec_IncrementalEncoder (const + char *encoding, const char *errors) + `Return value: New reference.' Get an *note IncrementalEncoder: + 14d4. object for the given `encoding'. + + -- C Function: PyObject* PyCodec_IncrementalDecoder (const + char *encoding, const char *errors) + `Return value: New reference.' Get an *note IncrementalDecoder: + 14d5. object for the given `encoding'. + + -- C Function: PyObject* PyCodec_StreamReader (const char *encoding, + PyObject *stream, const char *errors) + `Return value: New reference.' Get a *note StreamReader: 14d9. + factory function for the given `encoding'. + + -- C Function: PyObject* PyCodec_StreamWriter (const char *encoding, + PyObject *stream, const char *errors) + `Return value: New reference.' Get a *note StreamWriter: 14d8. + factory function for the given `encoding'. + + +File: python.info, Node: Registry API for Unicode encoding error handlers, Prev: Codec lookup API, Up: Codec registry and support functions + +7.6.9.2 Registry API for Unicode encoding error handlers +........................................................ + + -- C Function: int PyCodec_RegisterError (const char *name, + PyObject *error) + + Register the error handling callback function `error' under the + given `name'. This callback function will be called by a codec + when it encounters unencodable characters/undecodable bytes and + `name' is specified as the error parameter in the call to the + encode/decode function. + + The callback gets a single argument, an instance of *note + UnicodeEncodeError: 1fc, *note UnicodeDecodeError: 1fd. or *note + UnicodeTranslateError: 13bd. that holds information about the + problematic sequence of characters or bytes and their offset in the + original string (see *note Unicode Exception Objects: 395e. for + functions to extract this information). The callback must either + raise the given exception, or return a two-item tuple containing + the replacement for the problematic sequence, and an integer giving + the offset in the original string at which encoding/decoding should + be resumed. + + Return ‘0’ on success, ‘-1’ on error. + + -- C Function: PyObject* PyCodec_LookupError (const char *name) + `Return value: New reference.' Lookup the error handling callback + function registered under `name'. As a special case ‘NULL’ can be + passed, in which case the error handling callback for “strict” will + be returned. + + -- C Function: PyObject* PyCodec_StrictErrors (PyObject *exc) + `Return value: Always NULL.' Raise `exc' as an exception. + + -- C Function: PyObject* PyCodec_IgnoreErrors (PyObject *exc) + `Return value: New reference.' Ignore the unicode error, skipping + the faulty input. + + -- C Function: PyObject* PyCodec_ReplaceErrors (PyObject *exc) + `Return value: New reference.' Replace the unicode encode error + with ‘?’ or ‘U+FFFD’. + + -- C Function: PyObject* PyCodec_XMLCharRefReplaceErrors + (PyObject *exc) + `Return value: New reference.' Replace the unicode encode error + with XML character references. + + -- C Function: PyObject* PyCodec_BackslashReplaceErrors (PyObject *exc) + `Return value: New reference.' Replace the unicode encode error + with backslash escapes (‘\x’, ‘\u’ and ‘\U’). + + -- C Function: PyObject* PyCodec_NameReplaceErrors (PyObject *exc) + `Return value: New reference.' Replace the unicode encode error + with ‘\N{...}’ escapes. + + New in version 3.5. + + +File: python.info, Node: Abstract Objects Layer, Next: Concrete Objects Layer, Prev: Utilities<2>, Up: Python/C API Reference Manual + +7.7 Abstract Objects Layer +========================== + +The functions in this chapter interact with Python objects regardless of +their type, or with wide classes of object types (e.g. all numerical +types, or all sequence types). When used on object types for which they +do not apply, they will raise a Python exception. + +It is not possible to use these functions on objects that are not +properly initialized, such as a list object that has been created by +*note PyList_New(): 38f4, but whose items have not been set to some +non-‘NULL’ value yet. + +* Menu: + +* Object Protocol:: +* Number Protocol:: +* Sequence Protocol:: +* Mapping Protocol:: +* Iterator Protocol:: +* Buffer Protocol:: +* Old Buffer Protocol:: + + +File: python.info, Node: Object Protocol, Next: Number Protocol, Up: Abstract Objects Layer + +7.7.1 Object Protocol +--------------------- + + -- C Variable: PyObject* Py_NotImplemented + + The ‘NotImplemented’ singleton, used to signal that an operation is + not implemented for the given type combination. + + -- C Macro: Py_RETURN_NOTIMPLEMENTED + + Properly handle returning *note Py_NotImplemented: 39fc. from + within a C function (that is, increment the reference count of + NotImplemented and return it). + + -- C Function: int PyObject_Print (PyObject *o, FILE *fp, int flags) + + Print an object `o', on file `fp'. Returns ‘-1’ on error. The + flags argument is used to enable certain printing options. The + only option currently supported is ‘Py_PRINT_RAW’; if given, the + *note str(): 330. of the object is written instead of the *note + repr(): 7cc. + + -- C Function: int PyObject_HasAttr (PyObject *o, PyObject *attr_name) + + Returns ‘1’ if `o' has the attribute `attr_name', and ‘0’ + otherwise. This is equivalent to the Python expression ‘hasattr(o, + attr_name)’. This function always succeeds. + + Note that exceptions which occur while calling *note __getattr__(): + 31a. and *note __getattribute__(): 449. methods will get + suppressed. To get error reporting use *note PyObject_GetAttr(): + 3a00. instead. + + -- C Function: int PyObject_HasAttrString (PyObject *o, const + char *attr_name) + + Returns ‘1’ if `o' has the attribute `attr_name', and ‘0’ + otherwise. This is equivalent to the Python expression ‘hasattr(o, + attr_name)’. This function always succeeds. + + Note that exceptions which occur while calling *note __getattr__(): + 31a. and *note __getattribute__(): 449. methods and creating a + temporary string object will get suppressed. To get error + reporting use *note PyObject_GetAttrString(): 385b. instead. + + -- C Function: PyObject* PyObject_GetAttr (PyObject *o, + PyObject *attr_name) + `Return value: New reference.' Retrieve an attribute named + `attr_name' from object `o'. Returns the attribute value on + success, or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘o.attr_name’. + + -- C Function: PyObject* PyObject_GetAttrString (PyObject *o, const + char *attr_name) + `Return value: New reference.' Retrieve an attribute named + `attr_name' from object `o'. Returns the attribute value on + success, or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘o.attr_name’. + + -- C Function: PyObject* PyObject_GenericGetAttr (PyObject *o, + PyObject *name) + `Return value: New reference.' Generic attribute getter function + that is meant to be put into a type object’s ‘tp_getattro’ slot. + It looks for a descriptor in the dictionary of classes in the + object’s MRO as well as an attribute in the object’s *note + __dict__: 4fc. (if present). As outlined in *note Implementing + Descriptors: 4e9, data descriptors take preference over instance + attributes, while non-data descriptors don’t. Otherwise, an *note + AttributeError: 39b. is raised. + + -- C Function: int PyObject_SetAttr (PyObject *o, PyObject *attr_name, + PyObject *v) + + Set the value of the attribute named `attr_name', for object `o', + to the value `v'. Raise an exception and return ‘-1’ on failure; + return ‘0’ on success. This is the equivalent of the Python + statement ‘o.attr_name = v’. + + If `v' is ‘NULL’, the attribute is deleted, however this feature is + deprecated in favour of using *note PyObject_DelAttr(): 3a04. + + -- C Function: int PyObject_SetAttrString (PyObject *o, const + char *attr_name, PyObject *v) + + Set the value of the attribute named `attr_name', for object `o', + to the value `v'. Raise an exception and return ‘-1’ on failure; + return ‘0’ on success. This is the equivalent of the Python + statement ‘o.attr_name = v’. + + If `v' is ‘NULL’, the attribute is deleted, however this feature is + deprecated in favour of using *note PyObject_DelAttrString(): 3a06. + + -- C Function: int PyObject_GenericSetAttr (PyObject *o, + PyObject *name, PyObject *value) + + Generic attribute setter and deleter function that is meant to be + put into a type object’s *note tp_setattro: 38a0. slot. It looks + for a data descriptor in the dictionary of classes in the object’s + MRO, and if found it takes preference over setting or deleting the + attribute in the instance dictionary. Otherwise, the attribute is + set or deleted in the object’s *note __dict__: 4fc. (if present). + On success, ‘0’ is returned, otherwise an *note AttributeError: + 39b. is raised and ‘-1’ is returned. + + -- C Function: int PyObject_DelAttr (PyObject *o, PyObject *attr_name) + + Delete attribute named `attr_name', for object `o'. Returns ‘-1’ + on failure. This is the equivalent of the Python statement ‘del + o.attr_name’. + + -- C Function: int PyObject_DelAttrString (PyObject *o, const + char *attr_name) + + Delete attribute named `attr_name', for object `o'. Returns ‘-1’ + on failure. This is the equivalent of the Python statement ‘del + o.attr_name’. + + -- C Function: PyObject* PyObject_GenericGetDict (PyObject *o, + void *context) + `Return value: New reference.' A generic implementation for the + getter of a ‘__dict__’ descriptor. It creates the dictionary if + necessary. + + New in version 3.3. + + -- C Function: int PyObject_GenericSetDict (PyObject *o, + PyObject *value, void *context) + + A generic implementation for the setter of a ‘__dict__’ descriptor. + This implementation does not allow the dictionary to be deleted. + + New in version 3.3. + + -- C Function: PyObject* PyObject_RichCompare (PyObject *o1, + PyObject *o2, int opid) + `Return value: New reference.' Compare the values of `o1' and `o2' + using the operation specified by `opid', which must be one of + ‘Py_LT’, ‘Py_LE’, ‘Py_EQ’, ‘Py_NE’, ‘Py_GT’, or ‘Py_GE’, + corresponding to ‘<’, ‘<=’, ‘==’, ‘!=’, ‘>’, or ‘>=’ respectively. + This is the equivalent of the Python expression ‘o1 op o2’, where + ‘op’ is the operator corresponding to `opid'. Returns the value of + the comparison on success, or ‘NULL’ on failure. + + -- C Function: int PyObject_RichCompareBool (PyObject *o1, + PyObject *o2, int opid) + + Compare the values of `o1' and `o2' using the operation specified + by `opid', which must be one of ‘Py_LT’, ‘Py_LE’, ‘Py_EQ’, ‘Py_NE’, + ‘Py_GT’, or ‘Py_GE’, corresponding to ‘<’, ‘<=’, ‘==’, ‘!=’, ‘>’, + or ‘>=’ respectively. Returns ‘-1’ on error, ‘0’ if the result is + false, ‘1’ otherwise. This is the equivalent of the Python + expression ‘o1 op o2’, where ‘op’ is the operator corresponding to + `opid'. + + Note: If `o1' and `o2' are the same object, *note + PyObject_RichCompareBool(): 38a7. will always return ‘1’ for + ‘Py_EQ’ and ‘0’ for ‘Py_NE’. + + -- C Function: PyObject* PyObject_Repr (PyObject *o) + `Return value: New reference.' + + Compute a string representation of object `o'. Returns the string + representation on success, ‘NULL’ on failure. This is the + equivalent of the Python expression ‘repr(o)’. Called by the *note + repr(): 7cc. built-in function. + + Changed in version 3.4: This function now includes a debug + assertion to help ensure that it does not silently discard an + active exception. + + -- C Function: PyObject* PyObject_ASCII (PyObject *o) + `Return value: New reference.' + + As *note PyObject_Repr(): 968, compute a string representation of + object `o', but escape the non-ASCII characters in the string + returned by *note PyObject_Repr(): 968. with ‘\x’, ‘\u’ or ‘\U’ + escapes. This generates a string similar to that returned by *note + PyObject_Repr(): 968. in Python 2. Called by the *note ascii(): + d4c. built-in function. + + -- C Function: PyObject* PyObject_Str (PyObject *o) + `Return value: New reference.' Compute a string representation of + object `o'. Returns the string representation on success, ‘NULL’ + on failure. This is the equivalent of the Python expression + ‘str(o)’. Called by the *note str(): 330. built-in function and, + therefore, by the *note print(): 886. function. + + Changed in version 3.4: This function now includes a debug + assertion to help ensure that it does not silently discard an + active exception. + + -- C Function: PyObject* PyObject_Bytes (PyObject *o) + `Return value: New reference.' + + Compute a bytes representation of object `o'. ‘NULL’ is returned + on failure and a bytes object on success. This is equivalent to + the Python expression ‘bytes(o)’, when `o' is not an integer. + Unlike ‘bytes(o)’, a TypeError is raised when `o' is an integer + instead of a zero-initialized bytes object. + + -- C Function: int PyObject_IsSubclass (PyObject *derived, + PyObject *cls) + + Return ‘1’ if the class `derived' is identical to or derived from + the class `cls', otherwise return ‘0’. In case of an error, return + ‘-1’. + + If `cls' is a tuple, the check will be done against every entry in + `cls'. The result will be ‘1’ when at least one of the checks + returns ‘1’, otherwise it will be ‘0’. + + If `cls' has a *note __subclasscheck__(): 10f3. method, it will be + called to determine the subclass status as described in PEP + 3119(1). Otherwise, `derived' is a subclass of `cls' if it is a + direct or indirect subclass, i.e. contained in ‘cls.__mro__’. + + Normally only class objects, i.e. instances of *note type: 608. or + a derived class, are considered classes. However, objects can + override this by having a ‘__bases__’ attribute (which must be a + tuple of base classes). + + -- C Function: int PyObject_IsInstance (PyObject *inst, PyObject *cls) + + Return ‘1’ if `inst' is an instance of the class `cls' or a + subclass of `cls', or ‘0’ if not. On error, returns ‘-1’ and sets + an exception. + + If `cls' is a tuple, the check will be done against every entry in + `cls'. The result will be ‘1’ when at least one of the checks + returns ‘1’, otherwise it will be ‘0’. + + If `cls' has a *note __instancecheck__(): 10f2. method, it will be + called to determine the subclass status as described in PEP + 3119(2). Otherwise, `inst' is an instance of `cls' if its class is + a subclass of `cls'. + + An instance `inst' can override what is considered its class by + having a ‘__class__’ attribute. + + An object `cls' can override if it is considered a class, and what + its base classes are, by having a ‘__bases__’ attribute (which must + be a tuple of base classes). + + -- C Function: int PyCallable_Check (PyObject *o) + + Determine if the object `o' is callable. Return ‘1’ if the object + is callable and ‘0’ otherwise. This function always succeeds. + + -- C Function: PyObject* PyObject_Call (PyObject *callable, + PyObject *args, PyObject *kwargs) + `Return value: New reference.' Call a callable Python object + `callable', with arguments given by the tuple `args', and named + arguments given by the dictionary `kwargs'. + + `args' must not be ‘NULL’, use an empty tuple if no arguments are + needed. If no named arguments are needed, `kwargs' can be ‘NULL’. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + This is the equivalent of the Python expression: ‘callable(*args, + **kwargs)’. + + -- C Function: PyObject* PyObject_CallObject (PyObject *callable, + PyObject *args) + `Return value: New reference.' Call a callable Python object + `callable', with arguments given by the tuple `args'. If no + arguments are needed, then `args' can be ‘NULL’. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + This is the equivalent of the Python expression: ‘callable(*args)’. + + -- C Function: PyObject* PyObject_CallFunction (PyObject *callable, + const char *format, ...) + `Return value: New reference.' Call a callable Python object + `callable', with a variable number of C arguments. The C arguments + are described using a *note Py_BuildValue(): 2ce. style format + string. The format can be ‘NULL’, indicating that no arguments are + provided. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + This is the equivalent of the Python expression: ‘callable(*args)’. + + Note that if you only pass *note PyObject *: 4ba. args, *note + PyObject_CallFunctionObjArgs(): 3a0d. is a faster alternative. + + Changed in version 3.4: The type of `format' was changed from ‘char + *’. + + -- C Function: PyObject* PyObject_CallMethod (PyObject *obj, const + char *name, const char *format, ...) + `Return value: New reference.' Call the method named `name' of + object `obj' with a variable number of C arguments. The C + arguments are described by a *note Py_BuildValue(): 2ce. format + string that should produce a tuple. + + The format can be ‘NULL’, indicating that no arguments are + provided. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + This is the equivalent of the Python expression: ‘obj.name(arg1, + arg2, ...)’. + + Note that if you only pass *note PyObject *: 4ba. args, *note + PyObject_CallMethodObjArgs(): 3a0f. is a faster alternative. + + Changed in version 3.4: The types of `name' and `format' were + changed from ‘char *’. + + -- C Function: PyObject* PyObject_CallFunctionObjArgs + (PyObject *callable, ...) + `Return value: New reference.' Call a callable Python object + `callable', with a variable number of *note PyObject*: 4ba. + arguments. The arguments are provided as a variable number of + parameters followed by ‘NULL’. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + This is the equivalent of the Python expression: ‘callable(arg1, + arg2, ...)’. + + -- C Function: PyObject* PyObject_CallMethodObjArgs (PyObject *obj, + PyObject *name, ...) + `Return value: New reference.' Calls a method of the Python object + `obj', where the name of the method is given as a Python string + object in `name'. It is called with a variable number of *note + PyObject*: 4ba. arguments. The arguments are provided as a + variable number of parameters followed by ‘NULL’. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + -- C Function: PyObject* _PyObject_Vectorcall (PyObject *callable, + PyObject *const *args, size_t nargsf, PyObject *kwnames) + + Call a callable Python object `callable', using *note vectorcall: + 3a11. if possible. + + `args' is a C array with the positional arguments. + + `nargsf' is the number of positional arguments plus optionally the + flag ‘PY_VECTORCALL_ARGUMENTS_OFFSET’ (see below). To get actual + number of arguments, use *note PyVectorcall_NARGS(nargsf): 3a12. + + `kwnames' can be either ‘NULL’ (no keyword arguments) or a tuple of + keyword names. In the latter case, the values of the keyword + arguments are stored in `args' after the positional arguments. The + number of keyword arguments does not influence `nargsf'. + + `kwnames' must contain only objects of type ‘str’ (not a subclass), + and all keys must be unique. + + Return the result of the call on success, or raise an exception and + return ‘NULL’ on failure. + + This uses the vectorcall protocol if the callable supports it; + otherwise, the arguments are converted to use *note tp_call: 38ad. + + Note: This function is provisional and expected to become + public in Python 3.9, with a different name and, possibly, + changed semantics. If you use the function, plan for updating + your code for Python 3.9. + + New in version 3.8. + + -- C Macro: PY_VECTORCALL_ARGUMENTS_OFFSET + + If set in a vectorcall `nargsf' argument, the callee is allowed to + temporarily change ‘args[-1]’. In other words, `args' points to + argument 1 (not 0) in the allocated vector. The callee must + restore the value of ‘args[-1]’ before returning. + + Whenever they can do so cheaply (without additional allocation), + callers are encouraged to use ‘PY_VECTORCALL_ARGUMENTS_OFFSET’. + Doing so will allow callables such as bound methods to make their + onward calls (which include a prepended `self' argument) cheaply. + + New in version 3.8. + + -- C Function: Py_ssize_t PyVectorcall_NARGS (size_t nargsf) + + Given a vectorcall `nargsf' argument, return the actual number of + arguments. Currently equivalent to ‘nargsf & + ~PY_VECTORCALL_ARGUMENTS_OFFSET’. + + New in version 3.8. + + -- C Function: PyObject* _PyObject_FastCallDict (PyObject *callable, + PyObject *const *args, size_t nargsf, PyObject *kwdict) + + Same as *note _PyObject_Vectorcall(): 3a10. except that the keyword + arguments are passed as a dictionary in `kwdict'. This may be + ‘NULL’ if there are no keyword arguments. + + For callables supporting *note vectorcall: 3a11, the arguments are + internally converted to the vectorcall convention. Therefore, this + function adds some overhead compared to *note + _PyObject_Vectorcall(): 3a10. It should only be used if the caller + already has a dictionary ready to use. + + Note: This function is provisional and expected to become + public in Python 3.9, with a different name and, possibly, + changed semantics. If you use the function, plan for updating + your code for Python 3.9. + + New in version 3.8. + + -- C Function: Py_hash_t PyObject_Hash (PyObject *o) + + Compute and return the hash value of an object `o'. On failure, + return ‘-1’. This is the equivalent of the Python expression + ‘hash(o)’. + + Changed in version 3.2: The return type is now Py_hash_t. This is + a signed integer the same size as Py_ssize_t. + + -- C Function: Py_hash_t PyObject_HashNotImplemented (PyObject *o) + + Set a *note TypeError: 192. indicating that ‘type(o)’ is not + hashable and return ‘-1’. This function receives special treatment + when stored in a ‘tp_hash’ slot, allowing a type to explicitly + indicate to the interpreter that it is not hashable. + + -- C Function: int PyObject_IsTrue (PyObject *o) + + Returns ‘1’ if the object `o' is considered to be true, and ‘0’ + otherwise. This is equivalent to the Python expression ‘not not + o’. On failure, return ‘-1’. + + -- C Function: int PyObject_Not (PyObject *o) + + Returns ‘0’ if the object `o' is considered to be true, and ‘1’ + otherwise. This is equivalent to the Python expression ‘not o’. + On failure, return ‘-1’. + + -- C Function: PyObject* PyObject_Type (PyObject *o) + `Return value: New reference.' + + When `o' is non-‘NULL’, returns a type object corresponding to the + object type of object `o'. On failure, raises *note SystemError: + 5fb. and returns ‘NULL’. This is equivalent to the Python + expression ‘type(o)’. This function increments the reference count + of the return value. There’s really no reason to use this function + instead of the common expression ‘o->ob_type’, which returns a + pointer of type *note PyTypeObject*: 2d6, except when the + incremented reference count is needed. + + -- C Function: int PyObject_TypeCheck (PyObject *o, PyTypeObject *type) + + Return true if the object `o' is of type `type' or a subtype of + `type'. Both parameters must be non-‘NULL’. + + -- C Function: Py_ssize_t PyObject_Size (PyObject *o) + -- C Function: Py_ssize_t PyObject_Length (PyObject *o) + + Return the length of object `o'. If the object `o' provides either + the sequence and mapping protocols, the sequence length is + returned. On error, ‘-1’ is returned. This is the equivalent to + the Python expression ‘len(o)’. + + -- C Function: Py_ssize_t PyObject_LengthHint (PyObject *o, + Py_ssize_t default) + + Return an estimated length for the object `o'. First try to return + its actual length, then an estimate using *note __length_hint__(): + 80c, and finally return the default value. On error return ‘-1’. + This is the equivalent to the Python expression + ‘operator.length_hint(o, default)’. + + New in version 3.4. + + -- C Function: PyObject* PyObject_GetItem (PyObject *o, PyObject *key) + `Return value: New reference.' Return element of `o' corresponding + to the object `key' or ‘NULL’ on failure. This is the equivalent + of the Python expression ‘o[key]’. + + -- C Function: int PyObject_SetItem (PyObject *o, PyObject *key, + PyObject *v) + + Map the object `key' to the value `v'. Raise an exception and + return ‘-1’ on failure; return ‘0’ on success. This is the + equivalent of the Python statement ‘o[key] = v’. This function + `does not' steal a reference to `v'. + + -- C Function: int PyObject_DelItem (PyObject *o, PyObject *key) + + Remove the mapping for the object `key' from the object `o'. + Return ‘-1’ on failure. This is equivalent to the Python statement + ‘del o[key]’. + + -- C Function: PyObject* PyObject_Dir (PyObject *o) + `Return value: New reference.' This is equivalent to the Python + expression ‘dir(o)’, returning a (possibly empty) list of strings + appropriate for the object argument, or ‘NULL’ if there was an + error. If the argument is ‘NULL’, this is like the Python ‘dir()’, + returning the names of the current locals; in this case, if no + execution frame is active then ‘NULL’ is returned but *note + PyErr_Occurred(): 383f. will return false. + + -- C Function: PyObject* PyObject_GetIter (PyObject *o) + `Return value: New reference.' This is equivalent to the Python + expression ‘iter(o)’. It returns a new iterator for the object + argument, or the object itself if the object is already an + iterator. Raises *note TypeError: 192. and returns ‘NULL’ if the + object cannot be iterated. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3119 + + (2) https://www.python.org/dev/peps/pep-3119 + + +File: python.info, Node: Number Protocol, Next: Sequence Protocol, Prev: Object Protocol, Up: Abstract Objects Layer + +7.7.2 Number Protocol +--------------------- + + -- C Function: int PyNumber_Check (PyObject *o) + + Returns ‘1’ if the object `o' provides numeric protocols, and false + otherwise. This function always succeeds. + + Changed in version 3.8: Returns ‘1’ if `o' is an index integer. + + -- C Function: PyObject* PyNumber_Add (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the result of adding `o1' + and `o2', or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘o1 + o2’. + + -- C Function: PyObject* PyNumber_Subtract (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the result of subtracting + `o2' from `o1', or ‘NULL’ on failure. This is the equivalent of + the Python expression ‘o1 - o2’. + + -- C Function: PyObject* PyNumber_Multiply (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the result of multiplying + `o1' and `o2', or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘o1 * o2’. + + -- C Function: PyObject* PyNumber_MatrixMultiply (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of matrix + multiplication on `o1' and `o2', or ‘NULL’ on failure. This is the + equivalent of the Python expression ‘o1 @ o2’. + + New in version 3.5. + + -- C Function: PyObject* PyNumber_FloorDivide (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Return the floor of `o1' divided by + `o2', or ‘NULL’ on failure. This is equivalent to the “classic” + division of integers. + + -- C Function: PyObject* PyNumber_TrueDivide (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Return a reasonable approximation + for the mathematical value of `o1' divided by `o2', or ‘NULL’ on + failure. The return value is “approximate” because binary floating + point numbers are approximate; it is not possible to represent all + real numbers in base two. This function can return a floating + point value when passed two integers. + + -- C Function: PyObject* PyNumber_Remainder (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the remainder of dividing + `o1' by `o2', or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘o1 % o2’. + + -- C Function: PyObject* PyNumber_Divmod (PyObject *o1, PyObject *o2) + `Return value: New reference.' + + See the built-in function *note divmod(): 152. Returns ‘NULL’ on + failure. This is the equivalent of the Python expression + ‘divmod(o1, o2)’. + + -- C Function: PyObject* PyNumber_Power (PyObject *o1, PyObject *o2, + PyObject *o3) + `Return value: New reference.' + + See the built-in function *note pow(): 19a. Returns ‘NULL’ on + failure. This is the equivalent of the Python expression ‘pow(o1, + o2, o3)’, where `o3' is optional. If `o3' is to be ignored, pass + *note Py_None: 3844. in its place (passing ‘NULL’ for `o3' would + cause an illegal memory access). + + -- C Function: PyObject* PyNumber_Negative (PyObject *o) + `Return value: New reference.' Returns the negation of `o' on + success, or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘-o’. + + -- C Function: PyObject* PyNumber_Positive (PyObject *o) + `Return value: New reference.' Returns `o' on success, or ‘NULL’ + on failure. This is the equivalent of the Python expression ‘+o’. + + -- C Function: PyObject* PyNumber_Absolute (PyObject *o) + `Return value: New reference.' + + Returns the absolute value of `o', or ‘NULL’ on failure. This is + the equivalent of the Python expression ‘abs(o)’. + + -- C Function: PyObject* PyNumber_Invert (PyObject *o) + `Return value: New reference.' Returns the bitwise negation of `o' + on success, or ‘NULL’ on failure. This is the equivalent of the + Python expression ‘~o’. + + -- C Function: PyObject* PyNumber_Lshift (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the result of left shifting + `o1' by `o2' on success, or ‘NULL’ on failure. This is the + equivalent of the Python expression ‘o1 << o2’. + + -- C Function: PyObject* PyNumber_Rshift (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the result of right + shifting `o1' by `o2' on success, or ‘NULL’ on failure. This is + the equivalent of the Python expression ‘o1 >> o2’. + + -- C Function: PyObject* PyNumber_And (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the “bitwise and” of `o1' + and `o2' on success and ‘NULL’ on failure. This is the equivalent + of the Python expression ‘o1 & o2’. + + -- C Function: PyObject* PyNumber_Xor (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the “bitwise exclusive or” + of `o1' by `o2' on success, or ‘NULL’ on failure. This is the + equivalent of the Python expression ‘o1 ^ o2’. + + -- C Function: PyObject* PyNumber_Or (PyObject *o1, PyObject *o2) + `Return value: New reference.' Returns the “bitwise or” of `o1' + and `o2' on success, or ‘NULL’ on failure. This is the equivalent + of the Python expression ‘o1 | o2’. + + -- C Function: PyObject* PyNumber_InPlaceAdd (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of adding `o1' + and `o2', or ‘NULL’ on failure. The operation is done `in-place' + when `o1' supports it. This is the equivalent of the Python + statement ‘o1 += o2’. + + -- C Function: PyObject* PyNumber_InPlaceSubtract (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of subtracting + `o2' from `o1', or ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python statement ‘o1 -= o2’. + + -- C Function: PyObject* PyNumber_InPlaceMultiply (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of multiplying + `o1' and `o2', or ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python statement ‘o1 *= o2’. + + -- C Function: PyObject* PyNumber_InPlaceMatrixMultiply (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of matrix + multiplication on `o1' and `o2', or ‘NULL’ on failure. The + operation is done `in-place' when `o1' supports it. This is the + equivalent of the Python statement ‘o1 @= o2’. + + New in version 3.5. + + -- C Function: PyObject* PyNumber_InPlaceFloorDivide (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the mathematical floor of + dividing `o1' by `o2', or ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python statement ‘o1 //= o2’. + + -- C Function: PyObject* PyNumber_InPlaceTrueDivide (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Return a reasonable approximation + for the mathematical value of `o1' divided by `o2', or ‘NULL’ on + failure. The return value is “approximate” because binary floating + point numbers are approximate; it is not possible to represent all + real numbers in base two. This function can return a floating + point value when passed two integers. The operation is done + `in-place' when `o1' supports it. + + -- C Function: PyObject* PyNumber_InPlaceRemainder (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the remainder of dividing + `o1' by `o2', or ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python statement ‘o1 %= o2’. + + -- C Function: PyObject* PyNumber_InPlacePower (PyObject *o1, + PyObject *o2, PyObject *o3) + `Return value: New reference.' + + See the built-in function *note pow(): 19a. Returns ‘NULL’ on + failure. The operation is done `in-place' when `o1' supports it. + This is the equivalent of the Python statement ‘o1 **= o2’ when o3 + is *note Py_None: 3844, or an in-place variant of ‘pow(o1, o2, o3)’ + otherwise. If `o3' is to be ignored, pass *note Py_None: 3844. in + its place (passing ‘NULL’ for `o3' would cause an illegal memory + access). + + -- C Function: PyObject* PyNumber_InPlaceLshift (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of left shifting + `o1' by `o2' on success, or ‘NULL’ on failure. The operation is + done `in-place' when `o1' supports it. This is the equivalent of + the Python statement ‘o1 <<= o2’. + + -- C Function: PyObject* PyNumber_InPlaceRshift (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the result of right + shifting `o1' by `o2' on success, or ‘NULL’ on failure. The + operation is done `in-place' when `o1' supports it. This is the + equivalent of the Python statement ‘o1 >>= o2’. + + -- C Function: PyObject* PyNumber_InPlaceAnd (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the “bitwise and” of `o1' + and `o2' on success and ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python statement ‘o1 &= o2’. + + -- C Function: PyObject* PyNumber_InPlaceXor (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the “bitwise exclusive or” + of `o1' by `o2' on success, or ‘NULL’ on failure. The operation is + done `in-place' when `o1' supports it. This is the equivalent of + the Python statement ‘o1 ^= o2’. + + -- C Function: PyObject* PyNumber_InPlaceOr (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Returns the “bitwise or” of `o1' + and `o2' on success, or ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python statement ‘o1 |= o2’. + + -- C Function: PyObject* PyNumber_Long (PyObject *o) + `Return value: New reference.' + + Returns the `o' converted to an integer object on success, or + ‘NULL’ on failure. This is the equivalent of the Python expression + ‘int(o)’. + + -- C Function: PyObject* PyNumber_Float (PyObject *o) + `Return value: New reference.' + + Returns the `o' converted to a float object on success, or ‘NULL’ + on failure. This is the equivalent of the Python expression + ‘float(o)’. + + -- C Function: PyObject* PyNumber_Index (PyObject *o) + `Return value: New reference.' Returns the `o' converted to a + Python int on success or ‘NULL’ with a *note TypeError: 192. + exception raised on failure. + + -- C Function: PyObject* PyNumber_ToBase (PyObject *n, int base) + `Return value: New reference.' Returns the integer `n' converted + to base `base' as a string. The `base' argument must be one of 2, + 8, 10, or 16. For base 2, 8, or 16, the returned string is + prefixed with a base marker of ‘'0b'’, ‘'0o'’, or ‘'0x'’, + respectively. If `n' is not a Python int, it is converted with + *note PyNumber_Index(): 3a3e. first. + + -- C Function: Py_ssize_t PyNumber_AsSsize_t (PyObject *o, + PyObject *exc) + + Returns `o' converted to a Py_ssize_t value if `o' can be + interpreted as an integer. If the call fails, an exception is + raised and ‘-1’ is returned. + + If `o' can be converted to a Python int but the attempt to convert + to a Py_ssize_t value would raise an *note OverflowError: 960, then + the `exc' argument is the type of exception that will be raised + (usually *note IndexError: e75. or *note OverflowError: 960.). If + `exc' is ‘NULL’, then the exception is cleared and the value is + clipped to ‘PY_SSIZE_T_MIN’ for a negative integer or + ‘PY_SSIZE_T_MAX’ for a positive integer. + + -- C Function: int PyIndex_Check (PyObject *o) + + Returns ‘1’ if `o' is an index integer (has the nb_index slot of + the tp_as_number structure filled in), and ‘0’ otherwise. This + function always succeeds. + + +File: python.info, Node: Sequence Protocol, Next: Mapping Protocol, Prev: Number Protocol, Up: Abstract Objects Layer + +7.7.3 Sequence Protocol +----------------------- + + -- C Function: int PySequence_Check (PyObject *o) + + Return ‘1’ if the object provides sequence protocol, and ‘0’ + otherwise. Note that it returns ‘1’ for Python classes with a + *note __getitem__(): 27c. method unless they are *note dict: 1b8. + subclasses since in general case it is impossible to determine what + the type of keys it supports. This function always succeeds. + + -- C Function: Py_ssize_t PySequence_Size (PyObject *o) + -- C Function: Py_ssize_t PySequence_Length (PyObject *o) + + Returns the number of objects in sequence `o' on success, and ‘-1’ + on failure. This is equivalent to the Python expression ‘len(o)’. + + -- C Function: PyObject* PySequence_Concat (PyObject *o1, PyObject *o2) + `Return value: New reference.' Return the concatenation of `o1' + and `o2' on success, and ‘NULL’ on failure. This is the equivalent + of the Python expression ‘o1 + o2’. + + -- C Function: PyObject* PySequence_Repeat (PyObject *o, + Py_ssize_t count) + `Return value: New reference.' Return the result of repeating + sequence object `o' `count' times, or ‘NULL’ on failure. This is + the equivalent of the Python expression ‘o * count’. + + -- C Function: PyObject* PySequence_InPlaceConcat (PyObject *o1, + PyObject *o2) + `Return value: New reference.' Return the concatenation of `o1' + and `o2' on success, and ‘NULL’ on failure. The operation is done + `in-place' when `o1' supports it. This is the equivalent of the + Python expression ‘o1 += o2’. + + -- C Function: PyObject* PySequence_InPlaceRepeat (PyObject *o, + Py_ssize_t count) + `Return value: New reference.' Return the result of repeating + sequence object `o' `count' times, or ‘NULL’ on failure. The + operation is done `in-place' when `o' supports it. This is the + equivalent of the Python expression ‘o *= count’. + + -- C Function: PyObject* PySequence_GetItem (PyObject *o, Py_ssize_t i) + `Return value: New reference.' Return the `i'th element of `o', or + ‘NULL’ on failure. This is the equivalent of the Python expression + ‘o[i]’. + + -- C Function: PyObject* PySequence_GetSlice (PyObject *o, + Py_ssize_t i1, Py_ssize_t i2) + `Return value: New reference.' Return the slice of sequence object + `o' between `i1' and `i2', or ‘NULL’ on failure. This is the + equivalent of the Python expression ‘o[i1:i2]’. + + -- C Function: int PySequence_SetItem (PyObject *o, Py_ssize_t i, + PyObject *v) + + Assign object `v' to the `i'th element of `o'. Raise an exception + and return ‘-1’ on failure; return ‘0’ on success. This is the + equivalent of the Python statement ‘o[i] = v’. This function `does + not' steal a reference to `v'. + + If `v' is ‘NULL’, the element is deleted, however this feature is + deprecated in favour of using *note PySequence_DelItem(): 3a4d. + + -- C Function: int PySequence_DelItem (PyObject *o, Py_ssize_t i) + + Delete the `i'th element of object `o'. Returns ‘-1’ on failure. + This is the equivalent of the Python statement ‘del o[i]’. + + -- C Function: int PySequence_SetSlice (PyObject *o, Py_ssize_t i1, + Py_ssize_t i2, PyObject *v) + + Assign the sequence object `v' to the slice in sequence object `o' + from `i1' to `i2'. This is the equivalent of the Python statement + ‘o[i1:i2] = v’. + + -- C Function: int PySequence_DelSlice (PyObject *o, Py_ssize_t i1, + Py_ssize_t i2) + + Delete the slice in sequence object `o' from `i1' to `i2'. Returns + ‘-1’ on failure. This is the equivalent of the Python statement + ‘del o[i1:i2]’. + + -- C Function: Py_ssize_t PySequence_Count (PyObject *o, + PyObject *value) + + Return the number of occurrences of `value' in `o', that is, return + the number of keys for which ‘o[key] == value’. On failure, return + ‘-1’. This is equivalent to the Python expression + ‘o.count(value)’. + + -- C Function: int PySequence_Contains (PyObject *o, PyObject *value) + + Determine if `o' contains `value'. If an item in `o' is equal to + `value', return ‘1’, otherwise return ‘0’. On error, return ‘-1’. + This is equivalent to the Python expression ‘value in o’. + + -- C Function: Py_ssize_t PySequence_Index (PyObject *o, + PyObject *value) + + Return the first index `i' for which ‘o[i] == value’. On error, + return ‘-1’. This is equivalent to the Python expression + ‘o.index(value)’. + + -- C Function: PyObject* PySequence_List (PyObject *o) + `Return value: New reference.' Return a list object with the same + contents as the sequence or iterable `o', or ‘NULL’ on failure. + The returned list is guaranteed to be new. This is equivalent to + the Python expression ‘list(o)’. + + -- C Function: PyObject* PySequence_Tuple (PyObject *o) + `Return value: New reference.' + + Return a tuple object with the same contents as the sequence or + iterable `o', or ‘NULL’ on failure. If `o' is a tuple, a new + reference will be returned, otherwise a tuple will be constructed + with the appropriate contents. This is equivalent to the Python + expression ‘tuple(o)’. + + -- C Function: PyObject* PySequence_Fast (PyObject *o, const char *m) + `Return value: New reference.' Return the sequence or iterable `o' + as an object usable by the other ‘PySequence_Fast*’ family of + functions. If the object is not a sequence or iterable, raises + *note TypeError: 192. with `m' as the message text. Returns ‘NULL’ + on failure. + + The ‘PySequence_Fast*’ functions are thus named because they assume + `o' is a *note PyTupleObject: 399f. or a *note PyListObject: 3a56. + and access the data fields of `o' directly. + + As a CPython implementation detail, if `o' is already a sequence or + list, it will be returned. + + -- C Function: Py_ssize_t PySequence_Fast_GET_SIZE (PyObject *o) + + Returns the length of `o', assuming that `o' was returned by *note + PySequence_Fast(): 3a55. and that `o' is not ‘NULL’. The size can + also be gotten by calling *note PySequence_Size(): 3a46. on `o', + but *note PySequence_Fast_GET_SIZE(): 3a57. is faster because it + can assume `o' is a list or tuple. + + -- C Function: PyObject* PySequence_Fast_GET_ITEM (PyObject *o, + Py_ssize_t i) + `Return value: Borrowed reference.' Return the `i'th element of + `o', assuming that `o' was returned by *note PySequence_Fast(): + 3a55, `o' is not ‘NULL’, and that `i' is within bounds. + + -- C Function: PyObject** PySequence_Fast_ITEMS (PyObject *o) + + Return the underlying array of PyObject pointers. Assumes that `o' + was returned by *note PySequence_Fast(): 3a55. and `o' is not + ‘NULL’. + + Note, if a list gets resized, the reallocation may relocate the + items array. So, only use the underlying array pointer in contexts + where the sequence cannot change. + + -- C Function: PyObject* PySequence_ITEM (PyObject *o, Py_ssize_t i) + `Return value: New reference.' Return the `i'th element of `o' or + ‘NULL’ on failure. Faster form of *note PySequence_GetItem(): + 38f6. but without checking that *note PySequence_Check(): 3a45. on + `o' is true and without adjustment for negative indices. + + +File: python.info, Node: Mapping Protocol, Next: Iterator Protocol, Prev: Sequence Protocol, Up: Abstract Objects Layer + +7.7.4 Mapping Protocol +---------------------- + +See also *note PyObject_GetItem(): 38f5, *note PyObject_SetItem(): 38f3. +and *note PyObject_DelItem(): 3a1b. + + -- C Function: int PyMapping_Check (PyObject *o) + + Return ‘1’ if the object provides mapping protocol or supports + slicing, and ‘0’ otherwise. Note that it returns ‘1’ for Python + classes with a *note __getitem__(): 27c. method since in general + case it is impossible to determine what type of keys it supports. + This function always succeeds. + + -- C Function: Py_ssize_t PyMapping_Size (PyObject *o) + -- C Function: Py_ssize_t PyMapping_Length (PyObject *o) + + Returns the number of keys in object `o' on success, and ‘-1’ on + failure. This is equivalent to the Python expression ‘len(o)’. + + -- C Function: PyObject* PyMapping_GetItemString (PyObject *o, const + char *key) + `Return value: New reference.' Return element of `o' corresponding + to the string `key' or ‘NULL’ on failure. This is the equivalent + of the Python expression ‘o[key]’. See also *note + PyObject_GetItem(): 38f5. + + -- C Function: int PyMapping_SetItemString (PyObject *o, const + char *key, PyObject *v) + + Map the string `key' to the value `v' in object `o'. Returns ‘-1’ + on failure. This is the equivalent of the Python statement ‘o[key] + = v’. See also *note PyObject_SetItem(): 38f3. This function + `does not' steal a reference to `v'. + + -- C Function: int PyMapping_DelItem (PyObject *o, PyObject *key) + + Remove the mapping for the object `key' from the object `o'. + Return ‘-1’ on failure. This is equivalent to the Python statement + ‘del o[key]’. This is an alias of *note PyObject_DelItem(): 3a1b. + + -- C Function: int PyMapping_DelItemString (PyObject *o, const + char *key) + + Remove the mapping for the string `key' from the object `o'. + Return ‘-1’ on failure. This is equivalent to the Python statement + ‘del o[key]’. + + -- C Function: int PyMapping_HasKey (PyObject *o, PyObject *key) + + Return ‘1’ if the mapping object has the key `key' and ‘0’ + otherwise. This is equivalent to the Python expression ‘key in o’. + This function always succeeds. + + Note that exceptions which occur while calling the *note + __getitem__(): 27c. method will get suppressed. To get error + reporting use *note PyObject_GetItem(): 38f5. instead. + + -- C Function: int PyMapping_HasKeyString (PyObject *o, const + char *key) + + Return ‘1’ if the mapping object has the key `key' and ‘0’ + otherwise. This is equivalent to the Python expression ‘key in o’. + This function always succeeds. + + Note that exceptions which occur while calling the *note + __getitem__(): 27c. method and creating a temporary string object + will get suppressed. To get error reporting use *note + PyMapping_GetItemString(): 3a61. instead. + + -- C Function: PyObject* PyMapping_Keys (PyObject *o) + `Return value: New reference.' On success, return a list of the + keys in object `o'. On failure, return ‘NULL’. + + Changed in version 3.7: Previously, the function returned a list or + a tuple. + + -- C Function: PyObject* PyMapping_Values (PyObject *o) + `Return value: New reference.' On success, return a list of the + values in object `o'. On failure, return ‘NULL’. + + Changed in version 3.7: Previously, the function returned a list or + a tuple. + + -- C Function: PyObject* PyMapping_Items (PyObject *o) + `Return value: New reference.' On success, return a list of the + items in object `o', where each item is a tuple containing a + key-value pair. On failure, return ‘NULL’. + + Changed in version 3.7: Previously, the function returned a list or + a tuple. + + +File: python.info, Node: Iterator Protocol, Next: Buffer Protocol, Prev: Mapping Protocol, Up: Abstract Objects Layer + +7.7.5 Iterator Protocol +----------------------- + +There are two functions specifically for working with iterators. + + -- C Function: int PyIter_Check (PyObject *o) + + Return true if the object `o' supports the iterator protocol. + + -- C Function: PyObject* PyIter_Next (PyObject *o) + `Return value: New reference.' Return the next value from the + iteration `o'. The object must be an iterator (it is up to the + caller to check this). If there are no remaining values, returns + ‘NULL’ with no exception set. If an error occurs while retrieving + the item, returns ‘NULL’ and passes along the exception. + +To write a loop which iterates over an iterator, the C code should look +something like this: + + PyObject *iterator = PyObject_GetIter(obj); + PyObject *item; + + if (iterator == NULL) { + /* propagate error */ + } + + while ((item = PyIter_Next(iterator))) { + /* do something with item */ + ... + /* release reference when done */ + Py_DECREF(item); + } + + Py_DECREF(iterator); + + if (PyErr_Occurred()) { + /* propagate error */ + } + else { + /* continue doing useful work */ + } + + +File: python.info, Node: Buffer Protocol, Next: Old Buffer Protocol, Prev: Iterator Protocol, Up: Abstract Objects Layer + +7.7.6 Buffer Protocol +--------------------- + +Certain objects available in Python wrap access to an underlying memory +array or `buffer'. Such objects include the built-in *note bytes: 331. +and *note bytearray: 332, and some extension types like *note +array.array: 451. Third-party libraries may define their own types for +special purposes, such as image processing or numeric analysis. + +While each of these types have their own semantics, they share the +common characteristic of being backed by a possibly large memory buffer. +It is then desirable, in some situations, to access that buffer directly +and without intermediate copying. + +Python provides such a facility at the C level in the form of the *note +buffer protocol: 1291. This protocol has two sides: + + - on the producer side, a type can export a “buffer interface” which + allows objects of that type to expose information about their + underlying buffer. This interface is described in the section + *note Buffer Object Structures: 3a6e.; + + - on the consumer side, several means are available to obtain a + pointer to the raw underlying data of an object (for example a + method parameter). + +Simple objects such as *note bytes: 331. and *note bytearray: 332. +expose their underlying buffer in byte-oriented form. Other forms are +possible; for example, the elements exposed by an *note array.array: +451. can be multi-byte values. + +An example consumer of the buffer interface is the *note write(): 589. +method of file objects: any object that can export a series of bytes +through the buffer interface can be written to a file. While ‘write()’ +only needs read-only access to the internal contents of the object +passed to it, other methods such as *note readinto(): 1ccf. need write +access to the contents of their argument. The buffer interface allows +objects to selectively allow or reject exporting of read-write and +read-only buffers. + +There are two ways for a consumer of the buffer interface to acquire a +buffer over a target object: + + * call *note PyObject_GetBuffer(): d25. with the right parameters; + + * call *note PyArg_ParseTuple(): 26a. (or one of its siblings) with + one of the ‘y*’, ‘w*’ or ‘s*’ *note format codes: 2d0. + +In both cases, *note PyBuffer_Release(): d54. must be called when the +buffer isn’t needed anymore. Failure to do so could lead to various +issues such as resource leaks. + +* Menu: + +* Buffer structure:: +* Buffer request types:: +* Complex arrays:: +* Buffer-related functions:: + + +File: python.info, Node: Buffer structure, Next: Buffer request types, Up: Buffer Protocol + +7.7.6.1 Buffer structure +........................ + +Buffer structures (or simply “buffers”) are useful as a way to expose +the binary data from another object to the Python programmer. They can +also be used as a zero-copy slicing mechanism. Using their ability to +reference a block of memory, it is possible to expose any data to the +Python programmer quite easily. The memory could be a large, constant +array in a C extension, it could be a raw block of memory for +manipulation before passing to an operating system library, or it could +be used to pass around structured data in its native, in-memory format. + +Contrary to most data types exposed by the Python interpreter, buffers +are not *note PyObject: 4ba. pointers but rather simple C structures. +This allows them to be created and copied very simply. When a generic +wrapper around a buffer is needed, a *note memoryview: 3a71. object can +be created. + +For short instructions how to write an exporting object, see *note +Buffer Object Structures: 3a6e. For obtaining a buffer, see *note +PyObject_GetBuffer(): d25. + + -- C Type: Py_buffer + + -- C Member: void *buf + + A pointer to the start of the logical structure described by + the buffer fields. This can be any location within the + underlying physical memory block of the exporter. For + example, with negative *note strides: 3a73. the value may + point to the end of the memory block. + + For *note contiguous: 1360. arrays, the value points to the + beginning of the memory block. + + -- C Member: void *obj + + A new reference to the exporting object. The reference is + owned by the consumer and automatically decremented and set to + ‘NULL’ by *note PyBuffer_Release(): d54. The field is the + equivalent of the return value of any standard C-API function. + + As a special case, for `temporary' buffers that are wrapped by + *note PyMemoryView_FromBuffer(): 3a75. or *note + PyBuffer_FillInfo(): 3a76. this field is ‘NULL’. In general, + exporting objects MUST NOT use this scheme. + + -- C Member: Py_ssize_t len + + ‘product(shape) * itemsize’. For contiguous arrays, this is + the length of the underlying memory block. For non-contiguous + arrays, it is the length that the logical structure would have + if it were copied to a contiguous representation. + + Accessing ‘((char *)buf)[0] up to ((char *)buf)[len-1]’ is + only valid if the buffer has been obtained by a request that + guarantees contiguity. In most cases such a request will be + *note PyBUF_SIMPLE: 3a78. or *note PyBUF_WRITABLE: 3a79. + + -- C Member: int readonly + + An indicator of whether the buffer is read-only. This field + is controlled by the *note PyBUF_WRITABLE: 3a79. flag. + + -- C Member: Py_ssize_t itemsize + + Item size in bytes of a single element. Same as the value of + *note struct.calcsize(): 14b8. called on non-‘NULL’ *note + format: 3a7c. values. + + Important exception: If a consumer requests a buffer without + the *note PyBUF_FORMAT: 3a7d. flag, *note format: 3a7c. will + be set to ‘NULL’, but *note itemsize: 3a7b. still has the + value for the original format. + + If *note shape: 3a7e. is present, the equality ‘product(shape) + * itemsize == len’ still holds and the consumer can use *note + itemsize: 3a7b. to navigate the buffer. + + If *note shape: 3a7e. is ‘NULL’ as a result of a *note + PyBUF_SIMPLE: 3a78. or a *note PyBUF_WRITABLE: 3a79. request, + the consumer must disregard *note itemsize: 3a7b. and assume + ‘itemsize == 1’. + + -- C Member: const char *format + + A `NUL' terminated string in *note struct: f8. module style + syntax describing the contents of a single item. If this is + ‘NULL’, ‘"B"’ (unsigned bytes) is assumed. + + This field is controlled by the *note PyBUF_FORMAT: 3a7d. + flag. + + -- C Member: int ndim + + The number of dimensions the memory represents as an + n-dimensional array. If it is ‘0’, *note buf: 3a72. points to + a single item representing a scalar. In this case, *note + shape: 3a7e, *note strides: 3a73. and *note suboffsets: 3a80. + MUST be ‘NULL’. + + The macro ‘PyBUF_MAX_NDIM’ limits the maximum number of + dimensions to 64. Exporters MUST respect this limit, + consumers of multi-dimensional buffers SHOULD be able to + handle up to ‘PyBUF_MAX_NDIM’ dimensions. + + -- C Member: Py_ssize_t *shape + + An array of ‘Py_ssize_t’ of length *note ndim: 3a7f. + indicating the shape of the memory as an n-dimensional array. + Note that ‘shape[0] * ... * shape[ndim-1] * itemsize’ MUST be + equal to *note len: 3a77. + + Shape values are restricted to ‘shape[n] >= 0’. The case + ‘shape[n] == 0’ requires special attention. See *note complex + arrays: 3a81. for further information. + + The shape array is read-only for the consumer. + + -- C Member: Py_ssize_t *strides + + An array of ‘Py_ssize_t’ of length *note ndim: 3a7f. giving + the number of bytes to skip to get to a new element in each + dimension. + + Stride values can be any integer. For regular arrays, strides + are usually positive, but a consumer MUST be able to handle + the case ‘strides[n] <= 0’. See *note complex arrays: 3a81. + for further information. + + The strides array is read-only for the consumer. + + -- C Member: Py_ssize_t *suboffsets + + An array of ‘Py_ssize_t’ of length *note ndim: 3a7f. If + ‘suboffsets[n] >= 0’, the values stored along the nth + dimension are pointers and the suboffset value dictates how + many bytes to add to each pointer after de-referencing. A + suboffset value that is negative indicates that no + de-referencing should occur (striding in a contiguous memory + block). + + If all suboffsets are negative (i.e. no de-referencing is + needed), then this field must be ‘NULL’ (the default value). + + This type of array representation is used by the Python + Imaging Library (PIL). See *note complex arrays: 3a81. for + further information how to access elements of such an array. + + The suboffsets array is read-only for the consumer. + + -- C Member: void *internal + + This is for use internally by the exporting object. For + example, this might be re-cast as an integer by the exporter + and used to store flags about whether or not the shape, + strides, and suboffsets arrays must be freed when the buffer + is released. The consumer MUST NOT alter this value. + + +File: python.info, Node: Buffer request types, Next: Complex arrays, Prev: Buffer structure, Up: Buffer Protocol + +7.7.6.2 Buffer request types +............................ + +Buffers are usually obtained by sending a buffer request to an exporting +object via *note PyObject_GetBuffer(): d25. Since the complexity of the +logical structure of the memory can vary drastically, the consumer uses +the `flags' argument to specify the exact buffer type it can handle. + +All *note Py_buffer: b1b. fields are unambiguously defined by the +request type. + +* Menu: + +* request-independent fields:: +* readonly, format: readonly format. +* shape, strides, suboffsets: shape strides suboffsets. +* contiguity requests:: +* compound requests:: + + +File: python.info, Node: request-independent fields, Next: readonly format, Up: Buffer request types + +7.7.6.3 request-independent fields +.................................. + +The following fields are not influenced by `flags' and must always be +filled in with the correct values: *note obj: 3a74, *note buf: 3a72, +*note len: 3a77, *note itemsize: 3a7b, *note ndim: 3a7f. + + +File: python.info, Node: readonly format, Next: shape strides suboffsets, Prev: request-independent fields, Up: Buffer request types + +7.7.6.4 readonly, format +........................ + + -- C Macro: PyBUF_WRITABLE + + Controls the *note readonly: 3a7a. field. If set, the + exporter MUST provide a writable buffer or else report + failure. Otherwise, the exporter MAY provide either a + read-only or writable buffer, but the choice MUST be + consistent for all consumers. + + -- C Macro: PyBUF_FORMAT + + Controls the *note format: 3a7c. field. If set, this field + MUST be filled in correctly. Otherwise, this field MUST be + ‘NULL’. + +*note PyBUF_WRITABLE: 3a79. can be |’d to any of the flags in the next +section. Since *note PyBUF_SIMPLE: 3a78. is defined as 0, *note +PyBUF_WRITABLE: 3a79. can be used as a stand-alone flag to request a +simple writable buffer. + +*note PyBUF_FORMAT: 3a7d. can be |’d to any of the flags except *note +PyBUF_SIMPLE: 3a78. The latter already implies format ‘B’ (unsigned +bytes). + + +File: python.info, Node: shape strides suboffsets, Next: contiguity requests, Prev: readonly format, Up: Buffer request types + +7.7.6.5 shape, strides, suboffsets +.................................. + +The flags that control the logical structure of the memory are listed in +decreasing order of complexity. Note that each flag contains all bits +of the flags below it. + +Request shape strides suboffsets + +----------------------------------------------------------------------------- + + -- C Macro: PyBUF_INDIRECT yes yes if needed + + + -- C Macro: PyBUF_STRIDES yes yes NULL + + + -- C Macro: PyBUF_ND yes NULL NULL + + + -- C Macro: PyBUF_SIMPLE NULL NULL NULL + + + +File: python.info, Node: contiguity requests, Next: compound requests, Prev: shape strides suboffsets, Up: Buffer request types + +7.7.6.6 contiguity requests +........................... + +C or Fortran *note contiguity: 1360. can be explicitly requested, with +and without stride information. Without stride information, the buffer +must be C-contiguous. + +Request shape strides suboffsets contig + +------------------------------------------------------------------------------------------------ + + -- C Macro: PyBUF_C_CONTIGUOUS yes yes NULL C + + + -- C Macro: PyBUF_F_CONTIGUOUS yes yes NULL F + + + -- C Macro: PyBUF_ANY_CONTIGUOUS yes yes NULL C or F + + +*note PyBUF_ND: 3a8a. yes NULL NULL C + + + +File: python.info, Node: compound requests, Prev: contiguity requests, Up: Buffer request types + +7.7.6.7 compound requests +......................... + +All possible requests are fully defined by some combination of the flags +in the previous section. For convenience, the buffer protocol provides +frequently used combinations as single flags. + +In the following table `U' stands for undefined contiguity. The +consumer would have to call *note PyBuffer_IsContiguous(): 3a90. to +determine contiguity. + +Request shape strides suboffsets contig readonly format + +------------------------------------------------------------------------------------------------------------------------ + + -- C Macro: PyBUF_FULL yes yes if needed U 0 yes + + + -- C Macro: PyBUF_FULL_RO yes yes if needed U 1 or 0 yes + + + -- C Macro: PyBUF_RECORDS yes yes NULL U 0 yes + + + -- C Macro: PyBUF_RECORDS_RO yes yes NULL U 1 or 0 yes + + + -- C Macro: PyBUF_STRIDED yes yes NULL U 0 NULL + + + -- C Macro: PyBUF_STRIDED_RO yes yes NULL U 1 or 0 NULL + + + -- C Macro: PyBUF_CONTIG yes NULL NULL C 0 NULL + + + -- C Macro: PyBUF_CONTIG_RO yes NULL NULL C 1 or 0 NULL + + + +File: python.info, Node: Complex arrays, Next: Buffer-related functions, Prev: Buffer request types, Up: Buffer Protocol + +7.7.6.8 Complex arrays +...................... + +* Menu: + +* NumPy-style; shape and strides: NumPy-style shape and strides. +* PIL-style; shape, strides and suboffsets: PIL-style shape strides and suboffsets. + + +File: python.info, Node: NumPy-style shape and strides, Next: PIL-style shape strides and suboffsets, Up: Complex arrays + +7.7.6.9 NumPy-style: shape and strides +...................................... + +The logical structure of NumPy-style arrays is defined by *note +itemsize: 3a7b, *note ndim: 3a7f, *note shape: 3a7e. and *note strides: +3a73. + +If ‘ndim == 0’, the memory location pointed to by *note buf: 3a72. is +interpreted as a scalar of size *note itemsize: 3a7b. In that case, +both *note shape: 3a7e. and *note strides: 3a73. are ‘NULL’. + +If *note strides: 3a73. is ‘NULL’, the array is interpreted as a +standard n-dimensional C-array. Otherwise, the consumer must access an +n-dimensional array as follows: + + ptr = (char *)buf + indices[0] * strides[0] + ... + indices[n-1] * strides[n-1]; + item = *((typeof(item) *)ptr); + +As noted above, *note buf: 3a72. can point to any location within the +actual memory block. An exporter can check the validity of a buffer +with this function: + + def verify_structure(memlen, itemsize, ndim, shape, strides, offset): + """Verify that the parameters represent a valid array within + the bounds of the allocated memory: + char *mem: start of the physical memory block + memlen: length of the physical memory block + offset: (char *)buf - mem + """ + if offset % itemsize: + return False + if offset < 0 or offset+itemsize > memlen: + return False + if any(v % itemsize for v in strides): + return False + + if ndim <= 0: + return ndim == 0 and not shape and not strides + if 0 in shape: + return True + + imin = sum(strides[j]*(shape[j]-1) for j in range(ndim) + if strides[j] <= 0) + imax = sum(strides[j]*(shape[j]-1) for j in range(ndim) + if strides[j] > 0) + + return 0 <= offset+imin and offset+imax+itemsize <= memlen + + +File: python.info, Node: PIL-style shape strides and suboffsets, Prev: NumPy-style shape and strides, Up: Complex arrays + +7.7.6.10 PIL-style: shape, strides and suboffsets +................................................. + +In addition to the regular items, PIL-style arrays can contain pointers +that must be followed in order to get to the next element in a +dimension. For example, the regular three-dimensional C-array ‘char +v[2][2][3]’ can also be viewed as an array of 2 pointers to 2 +two-dimensional arrays: ‘char (*v[2])[2][3]’. In suboffsets +representation, those two pointers can be embedded at the start of *note +buf: 3a72, pointing to two ‘char x[2][3]’ arrays that can be located +anywhere in memory. + +Here is a function that returns a pointer to the element in an N-D array +pointed to by an N-dimensional index when there are both non-‘NULL’ +strides and suboffsets: + + void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides, + Py_ssize_t *suboffsets, Py_ssize_t *indices) { + char *pointer = (char*)buf; + int i; + for (i = 0; i < ndim; i++) { + pointer += strides[i] * indices[i]; + if (suboffsets[i] >=0 ) { + pointer = *((char**)pointer) + suboffsets[i]; + } + } + return (void*)pointer; + } + + +File: python.info, Node: Buffer-related functions, Prev: Complex arrays, Up: Buffer Protocol + +7.7.6.11 Buffer-related functions +................................. + + -- C Function: int PyObject_CheckBuffer (PyObject *obj) + + Return ‘1’ if `obj' supports the buffer interface otherwise ‘0’. + When ‘1’ is returned, it doesn’t guarantee that *note + PyObject_GetBuffer(): d25. will succeed. This function always + succeeds. + + -- C Function: int PyObject_GetBuffer (PyObject *exporter, + Py_buffer *view, int flags) + + Send a request to `exporter' to fill in `view' as specified by + `flags'. If the exporter cannot provide a buffer of the exact + type, it MUST raise ‘PyExc_BufferError’, set ‘view->obj’ to ‘NULL’ + and return ‘-1’. + + On success, fill in `view', set ‘view->obj’ to a new reference to + `exporter' and return 0. In the case of chained buffer providers + that redirect requests to a single object, ‘view->obj’ MAY refer to + this object instead of `exporter' (See *note Buffer Object + Structures: 3a6e.). + + Successful calls to *note PyObject_GetBuffer(): d25. must be paired + with calls to *note PyBuffer_Release(): d54, similar to ‘malloc()’ + and ‘free()’. Thus, after the consumer is done with the buffer, + *note PyBuffer_Release(): d54. must be called exactly once. + + -- C Function: void PyBuffer_Release (Py_buffer *view) + + Release the buffer `view' and decrement the reference count for + ‘view->obj’. This function MUST be called when the buffer is no + longer being used, otherwise reference leaks may occur. + + It is an error to call this function on a buffer that was not + obtained via *note PyObject_GetBuffer(): d25. + + -- C Function: Py_ssize_t PyBuffer_SizeFromFormat (const char *) + + Return the implied *note itemsize: 3a7b. from *note format: 3a7c. + This function is not yet implemented. + + -- C Function: int PyBuffer_IsContiguous (Py_buffer *view, char order) + + Return ‘1’ if the memory defined by the `view' is C-style (`order' + is ‘'C'’) or Fortran-style (`order' is ‘'F'’) *note contiguous: + 1360. or either one (`order' is ‘'A'’). Return ‘0’ otherwise. + This function always succeeds. + + -- C Function: void* PyBuffer_GetPointer (Py_buffer *view, + Py_ssize_t *indices) + + Get the memory area pointed to by the `indices' inside the given + `view'. `indices' must point to an array of ‘view->ndim’ indices. + + -- C Function: int PyBuffer_FromContiguous (Py_buffer *view, void *buf, + Py_ssize_t len, char fort) + + Copy contiguous `len' bytes from `buf' to `view'. `fort' can be + ‘'C'’ or ‘'F'’ (for C-style or Fortran-style ordering). ‘0’ is + returned on success, ‘-1’ on error. + + -- C Function: int PyBuffer_ToContiguous (void *buf, Py_buffer *src, + Py_ssize_t len, char order) + + Copy `len' bytes from `src' to its contiguous representation in + `buf'. `order' can be ‘'C'’ or ‘'F'’ or ‘'A'’ (for C-style or + Fortran-style ordering or either one). ‘0’ is returned on success, + ‘-1’ on error. + + This function fails if `len' != `src->len'. + + -- C Function: void PyBuffer_FillContiguousStrides (int ndims, + Py_ssize_t *shape, Py_ssize_t *strides, int itemsize, + char order) + + Fill the `strides' array with byte-strides of a *note contiguous: + 1360. (C-style if `order' is ‘'C'’ or Fortran-style if `order' is + ‘'F'’) array of the given shape with the given number of bytes per + element. + + -- C Function: int PyBuffer_FillInfo (Py_buffer *view, + PyObject *exporter, void *buf, Py_ssize_t len, int readonly, + int flags) + + Handle buffer requests for an exporter that wants to expose `buf' + of size `len' with writability set according to `readonly'. `buf' + is interpreted as a sequence of unsigned bytes. + + The `flags' argument indicates the request type. This function + always fills in `view' as specified by flags, unless `buf' has been + designated as read-only and *note PyBUF_WRITABLE: 3a79. is set in + `flags'. + + On success, set ‘view->obj’ to a new reference to `exporter' and + return 0. Otherwise, raise ‘PyExc_BufferError’, set ‘view->obj’ to + ‘NULL’ and return ‘-1’; + + If this function is used as part of a *note getbufferproc: 3a6e, + `exporter' MUST be set to the exporting object and `flags' must be + passed unmodified. Otherwise, `exporter' MUST be ‘NULL’. + + +File: python.info, Node: Old Buffer Protocol, Prev: Buffer Protocol, Up: Abstract Objects Layer + +7.7.7 Old Buffer Protocol +------------------------- + +Deprecated since version 3.0. + +These functions were part of the “old buffer protocol” API in Python 2. +In Python 3, this protocol doesn’t exist anymore but the functions are +still exposed to ease porting 2.x code. They act as a compatibility +wrapper around the *note new buffer protocol: 1291, but they don’t give +you control over the lifetime of the resources acquired when a buffer is +exported. + +Therefore, it is recommended that you call *note PyObject_GetBuffer(): +d25. (or the ‘y*’ or ‘w*’ *note format codes: 2d0. with the *note +PyArg_ParseTuple(): 26a. family of functions) to get a buffer view over +an object, and *note PyBuffer_Release(): d54. when the buffer view can +be released. + + -- C Function: int PyObject_AsCharBuffer (PyObject *obj, const + char **buffer, Py_ssize_t *buffer_len) + + Returns a pointer to a read-only memory location usable as + character-based input. The `obj' argument must support the + single-segment character buffer interface. On success, returns + ‘0’, sets `buffer' to the memory location and `buffer_len' to the + buffer length. Returns ‘-1’ and sets a *note TypeError: 192. on + error. + + -- C Function: int PyObject_AsReadBuffer (PyObject *obj, const + void **buffer, Py_ssize_t *buffer_len) + + Returns a pointer to a read-only memory location containing + arbitrary data. The `obj' argument must support the single-segment + readable buffer interface. On success, returns ‘0’, sets `buffer' + to the memory location and `buffer_len' to the buffer length. + Returns ‘-1’ and sets a *note TypeError: 192. on error. + + -- C Function: int PyObject_CheckReadBuffer (PyObject *o) + + Returns ‘1’ if `o' supports the single-segment readable buffer + interface. Otherwise returns ‘0’. This function always succeeds. + + Note that this function tries to get and release a buffer, and + exceptions which occur while calling corresponding functions will + get suppressed. To get error reporting use *note + PyObject_GetBuffer(): d25. instead. + + -- C Function: int PyObject_AsWriteBuffer (PyObject *obj, + void **buffer, Py_ssize_t *buffer_len) + + Returns a pointer to a writable memory location. The `obj' + argument must support the single-segment, character buffer + interface. On success, returns ‘0’, sets `buffer' to the memory + location and `buffer_len' to the buffer length. Returns ‘-1’ and + sets a *note TypeError: 192. on error. + + +File: python.info, Node: Concrete Objects Layer, Next: Initialization Finalization and Threads, Prev: Abstract Objects Layer, Up: Python/C API Reference Manual + +7.8 Concrete Objects Layer +========================== + +The functions in this chapter are specific to certain Python object +types. Passing them an object of the wrong type is not a good idea; if +you receive an object from a Python program and you are not sure that it +has the right type, you must perform a type check first; for example, to +check that an object is a dictionary, use *note PyDict_Check(): 3aab. +The chapter is structured like the “family tree” of Python object types. + + Warning: While the functions described in this chapter carefully + check the type of the objects which are passed in, many of them do + not check for ‘NULL’ being passed instead of a valid object. + Allowing ‘NULL’ to be passed in can cause memory access violations + and immediate termination of the interpreter. + +* Menu: + +* Fundamental Objects:: +* Numeric Objects:: +* Sequence Objects:: +* Container Objects:: +* Function Objects: Function Objects<2>. +* Other Objects:: + + +File: python.info, Node: Fundamental Objects, Next: Numeric Objects, Up: Concrete Objects Layer + +7.8.1 Fundamental Objects +------------------------- + +This section describes Python type objects and the singleton object +‘None’. + +* Menu: + +* Type Objects: Type Objects<2>. +* The None Object:: + + +File: python.info, Node: Type Objects<2>, Next: The None Object, Up: Fundamental Objects + +7.8.1.1 Type Objects +.................... + + -- C Type: PyTypeObject + + The C structure of the objects used to describe built-in types. + + -- C Variable: PyObject* PyType_Type + + This is the type object for type objects; it is the same object as + *note type: 608. in the Python layer. + + -- C Function: int PyType_Check (PyObject *o) + + Return true if the object `o' is a type object, including instances + of types derived from the standard type object. Return false in + all other cases. + + -- C Function: int PyType_CheckExact (PyObject *o) + + Return true if the object `o' is a type object, but not a subtype + of the standard type object. Return false in all other cases. + + -- C Function: unsigned int PyType_ClearCache () + + Clear the internal lookup cache. Return the current version tag. + + -- C Function: unsigned long PyType_GetFlags (PyTypeObject* type) + + Return the *note tp_flags: 3ab6. member of `type'. This function + is primarily meant for use with ‘Py_LIMITED_API’; the individual + flag bits are guaranteed to be stable across Python releases, but + access to *note tp_flags: 3ab6. itself is not part of the limited + API. + + New in version 3.2. + + Changed in version 3.4: The return type is now ‘unsigned long’ + rather than ‘long’. + + -- C Function: void PyType_Modified (PyTypeObject *type) + + Invalidate the internal lookup cache for the type and all of its + subtypes. This function must be called after any manual + modification of the attributes or base classes of the type. + + -- C Function: int PyType_HasFeature (PyTypeObject *o, int feature) + + Return true if the type object `o' sets the feature `feature'. + Type features are denoted by single bit flags. + + -- C Function: int PyType_IS_GC (PyTypeObject *o) + + Return true if the type object includes support for the cycle + detector; this tests the type flag *note Py_TPFLAGS_HAVE_GC: 388e. + + -- C Function: int PyType_IsSubtype (PyTypeObject *a, PyTypeObject *b) + + Return true if `a' is a subtype of `b'. + + This function only checks for actual subtypes, which means that + *note __subclasscheck__(): 10f3. is not called on `b'. Call *note + PyObject_IsSubclass(): 79e. to do the same check that *note + issubclass(): 450. would do. + + -- C Function: PyObject* PyType_GenericAlloc (PyTypeObject *type, + Py_ssize_t nitems) + `Return value: New reference.' Generic handler for the *note + tp_alloc: 3881. slot of a type object. Use Python’s default memory + allocation mechanism to allocate a new instance and initialize all + its contents to ‘NULL’. + + -- C Function: PyObject* PyType_GenericNew (PyTypeObject *type, + PyObject *args, PyObject *kwds) + `Return value: New reference.' Generic handler for the *note + tp_new: 387c. slot of a type object. Create a new instance using + the type’s *note tp_alloc: 3881. slot. + + -- C Function: int PyType_Ready (PyTypeObject *type) + + Finalize a type object. This should be called on all type objects + to finish their initialization. This function is responsible for + adding inherited slots from a type’s base class. Return ‘0’ on + success, or return ‘-1’ and sets an exception on error. + + -- C Function: void* PyType_GetSlot (PyTypeObject *type, int slot) + + Return the function pointer stored in the given slot. If the + result is ‘NULL’, this indicates that either the slot is ‘NULL’, or + that the function was called with invalid parameters. Callers will + typically cast the result pointer into the appropriate function + type. + + See ‘PyType_Slot.slot’ for possible values of the `slot' argument. + + An exception is raised if `type' is not a heap type. + + New in version 3.4. + +* Menu: + +* Creating Heap-Allocated Types:: + + +File: python.info, Node: Creating Heap-Allocated Types, Up: Type Objects<2> + +7.8.1.2 Creating Heap-Allocated Types +..................................... + +The following functions and structs are used to create *note heap types: +3abc. + + -- C Function: PyObject* PyType_FromSpecWithBases (PyType_Spec *spec, + PyObject *bases) + `Return value: New reference.' Creates and returns a heap type + object from the `spec' (*note Py_TPFLAGS_HEAPTYPE: 3abe.). + + If `bases' is a tuple, the created heap type contains all types + contained in it as base types. + + If `bases' is ‘NULL’, the `Py_tp_bases' slot is used instead. If + that also is ‘NULL’, the `Py_tp_base' slot is used instead. If + that also is ‘NULL’, the new type derives from *note object: 2b0. + + This function calls *note PyType_Ready(): 3882. on the new type. + + New in version 3.3. + + -- C Function: PyObject* PyType_FromSpec (PyType_Spec *spec) + `Return value: New reference.' Equivalent to + ‘PyType_FromSpecWithBases(spec, NULL)’. + + -- C Type: PyType_Spec + + Structure defining a type’s behavior. + + -- C Member: const char* PyType_Spec.name + + Name of the type, used to set *note PyTypeObject.tp_name: + 389b. + + -- C Member: int PyType_Spec.basicsize + + -- C Member: int PyType_Spec.itemsize + + Size of the instance in bytes, used to set *note + PyTypeObject.tp_basicsize: 3879. and *note + PyTypeObject.tp_itemsize: 3878. + + -- C Member: int PyType_Spec.flags + + Type flags, used to set *note PyTypeObject.tp_flags: 3ab6. + + If the ‘Py_TPFLAGS_HEAPTYPE’ flag is not set, *note + PyType_FromSpecWithBases(): 3abd. sets it automatically. + + -- C Member: PyType_Slot *PyType_Spec.slots + + Array of *note PyType_Slot: 3ac5. structures. Terminated by + the special slot value ‘{0, NULL}’. + + -- C Type: PyType_Slot + + Structure defining optional functionality of a type, containing a + slot ID and a value pointer. + + -- C Member: int PyType_Slot.slot + + A slot ID. + + Slot IDs are named like the field names of the structures + *note PyTypeObject: 2d6, *note PyNumberMethods: d81, *note + PySequenceMethods: 38aa, *note PyMappingMethods: 38ab. and + *note PyAsyncMethods: 3ac7. with an added ‘Py_’ prefix. For + example, use: + + * ‘Py_tp_dealloc’ to set *note PyTypeObject.tp_dealloc: + 387f. + + * ‘Py_nb_add’ to set *note PyNumberMethods.nb_add: 3ac8. + + * ‘Py_sq_length’ to set *note PySequenceMethods.sq_length: + 3ac9. + + The following fields cannot be set using *note PyType_Spec: + 3abf. and *note PyType_Slot: 3ac5.: + + * *note tp_dict: 3aca. + + * *note tp_mro: 3acb. + + * *note tp_cache: 3acc. + + * *note tp_subclasses: 3acd. + + * *note tp_weaklist: 3ace. + + * ‘tp_print’ + + * *note tp_weaklistoffset: 38af. + + * *note tp_dictoffset: 3acf. + + * *note bf_getbuffer: 3ad0. + + * *note bf_releasebuffer: 39c9. + + Setting ‘Py_tp_bases’ or ‘Py_tp_base’ may be problematic on + some platforms. To avoid issues, use the `bases' argument of + ‘PyType_FromSpecWithBases()’ instead. + + -- C Member: void *PyType_Slot.pfunc + + The desired value of the slot. In most cases, this is a + pointer to a function. + + May not be ‘NULL’. + + +File: python.info, Node: The None Object, Prev: Type Objects<2>, Up: Fundamental Objects + +7.8.1.3 The ‘None’ Object +......................... + +Note that the *note PyTypeObject: 2d6. for ‘None’ is not directly +exposed in the Python/C API. Since ‘None’ is a singleton, testing for +object identity (using ‘==’ in C) is sufficient. There is no +‘PyNone_Check()’ function for the same reason. + + -- C Variable: PyObject* Py_None + + The Python ‘None’ object, denoting lack of value. This object has + no methods. It needs to be treated just like any other object with + respect to reference counts. + + -- C Macro: Py_RETURN_NONE + + Properly handle returning *note Py_None: 3844. from within a C + function (that is, increment the reference count of ‘None’ and + return it.) + + +File: python.info, Node: Numeric Objects, Next: Sequence Objects, Prev: Fundamental Objects, Up: Concrete Objects Layer + +7.8.2 Numeric Objects +--------------------- + +* Menu: + +* Integer Objects:: +* Boolean Objects:: +* Floating Point Objects:: +* Complex Number Objects:: + + +File: python.info, Node: Integer Objects, Next: Boolean Objects, Up: Numeric Objects + +7.8.2.1 Integer Objects +....................... + +All integers are implemented as “long” integer objects of arbitrary +size. + +On error, most ‘PyLong_As*’ APIs return ‘(return type)-1’ which cannot +be distinguished from a number. Use *note PyErr_Occurred(): 383f. to +disambiguate. + + -- C Type: PyLongObject + + This subtype of *note PyObject: 4ba. represents a Python integer + object. + + -- C Variable: PyTypeObject PyLong_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + integer type. This is the same object as *note int: 184. in the + Python layer. + + -- C Function: int PyLong_Check (PyObject *p) + + Return true if its argument is a *note PyLongObject: 3ada. or a + subtype of *note PyLongObject: 3ada. + + -- C Function: int PyLong_CheckExact (PyObject *p) + + Return true if its argument is a *note PyLongObject: 3ada, but not + a subtype of *note PyLongObject: 3ada. + + -- C Function: PyObject* PyLong_FromLong (long v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from `v', or ‘NULL’ on failure. + + The current implementation keeps an array of integer objects for + all integers between ‘-5’ and ‘256’, when you create an int in that + range you actually just get back a reference to the existing + object. So it should be possible to change the value of ‘1’. I + suspect the behaviour of Python in this case is undefined. :-) + + -- C Function: PyObject* PyLong_FromUnsignedLong (unsigned long v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from a C ‘unsigned long’, or ‘NULL’ on failure. + + -- C Function: PyObject* PyLong_FromSsize_t (Py_ssize_t v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from a C ‘Py_ssize_t’, or ‘NULL’ on failure. + + -- C Function: PyObject* PyLong_FromSize_t (size_t v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from a C ‘size_t’, or ‘NULL’ on failure. + + -- C Function: PyObject* PyLong_FromLongLong (long long v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from a C ‘long long’, or ‘NULL’ on failure. + + -- C Function: PyObject* PyLong_FromUnsignedLongLong (unsigned long + long v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from a C ‘unsigned long long’, or ‘NULL’ on failure. + + -- C Function: PyObject* PyLong_FromDouble (double v) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. object from the integer part of `v', or ‘NULL’ on failure. + + -- C Function: PyObject* PyLong_FromString (const char *str, + char **pend, int base) + `Return value: New reference.' Return a new *note PyLongObject: + 3ada. based on the string value in `str', which is interpreted + according to the radix in `base'. If `pend' is non-‘NULL’, `*pend' + will point to the first character in `str' which follows the + representation of the number. If `base' is ‘0’, `str' is + interpreted using the *note Integer literals: 109f. definition; in + this case, leading zeros in a non-zero decimal number raises a + *note ValueError: 1fb. If `base' is not ‘0’, it must be between + ‘2’ and ‘36’, inclusive. Leading spaces and single underscores + after a base specifier and between digits are ignored. If there + are no digits, *note ValueError: 1fb. will be raised. + + -- C Function: PyObject* PyLong_FromUnicode (Py_UNICODE *u, + Py_ssize_t length, int base) + `Return value: New reference.' Convert a sequence of Unicode + digits to a Python integer value. + + Deprecated since version 3.3, will be removed in version 3.10: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyLong_FromUnicodeObject(): 3ae6. + + -- C Function: PyObject* PyLong_FromUnicodeObject (PyObject *u, + int base) + `Return value: New reference.' Convert a sequence of Unicode + digits in the string `u' to a Python integer value. + + New in version 3.3. + + -- C Function: PyObject* PyLong_FromVoidPtr (void *p) + `Return value: New reference.' Create a Python integer from the + pointer `p'. The pointer value can be retrieved from the resulting + value using *note PyLong_AsVoidPtr(): 3ae8. + + -- C Function: long PyLong_AsLong (PyObject *obj) + + Return a C ‘long’ representation of `obj'. If `obj' is not an + instance of *note PyLongObject: 3ada, first call its *note + __index__(): 18a. or *note __int__(): 18b. method (if present) to + convert it to a *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `obj' is out of + range for a ‘long’. + + Returns ‘-1’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + Deprecated since version 3.8: Using *note __int__(): 18b. is + deprecated. + + -- C Function: long PyLong_AsLongAndOverflow (PyObject *obj, + int *overflow) + + Return a C ‘long’ representation of `obj'. If `obj' is not an + instance of *note PyLongObject: 3ada, first call its *note + __index__(): 18a. or *note __int__(): 18b. method (if present) to + convert it to a *note PyLongObject: 3ada. + + If the value of `obj' is greater than ‘LONG_MAX’ or less than + ‘LONG_MIN’, set `*overflow' to ‘1’ or ‘-1’, respectively, and + return ‘-1’; otherwise, set `*overflow' to ‘0’. If any other + exception occurs set `*overflow' to ‘0’ and return ‘-1’ as usual. + + Returns ‘-1’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + Deprecated since version 3.8: Using *note __int__(): 18b. is + deprecated. + + -- C Function: long long PyLong_AsLongLong (PyObject *obj) + + Return a C ‘long long’ representation of `obj'. If `obj' is not an + instance of *note PyLongObject: 3ada, first call its *note + __index__(): 18a. or *note __int__(): 18b. method (if present) to + convert it to a *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `obj' is out of + range for a ‘long long’. + + Returns ‘-1’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + Deprecated since version 3.8: Using *note __int__(): 18b. is + deprecated. + + -- C Function: long long PyLong_AsLongLongAndOverflow (PyObject *obj, + int *overflow) + + Return a C ‘long long’ representation of `obj'. If `obj' is not an + instance of *note PyLongObject: 3ada, first call its *note + __index__(): 18a. or *note __int__(): 18b. method (if present) to + convert it to a *note PyLongObject: 3ada. + + If the value of `obj' is greater than ‘PY_LLONG_MAX’ or less than + ‘PY_LLONG_MIN’, set `*overflow' to ‘1’ or ‘-1’, respectively, and + return ‘-1’; otherwise, set `*overflow' to ‘0’. If any other + exception occurs set `*overflow' to ‘0’ and return ‘-1’ as usual. + + Returns ‘-1’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + New in version 3.2. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + Deprecated since version 3.8: Using *note __int__(): 18b. is + deprecated. + + -- C Function: Py_ssize_t PyLong_AsSsize_t (PyObject *pylong) + + Return a C ‘Py_ssize_t’ representation of `pylong'. `pylong' must + be an instance of *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `pylong' is out of + range for a ‘Py_ssize_t’. + + Returns ‘-1’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + -- C Function: unsigned long PyLong_AsUnsignedLong (PyObject *pylong) + + Return a C ‘unsigned long’ representation of `pylong'. `pylong' + must be an instance of *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `pylong' is out of + range for a ‘unsigned long’. + + Returns ‘(unsigned long)-1’ on error. Use *note PyErr_Occurred(): + 383f. to disambiguate. + + -- C Function: size_t PyLong_AsSize_t (PyObject *pylong) + + Return a C ‘size_t’ representation of `pylong'. `pylong' must be + an instance of *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `pylong' is out of + range for a ‘size_t’. + + Returns ‘(size_t)-1’ on error. Use *note PyErr_Occurred(): 383f. + to disambiguate. + + -- C Function: unsigned long long PyLong_AsUnsignedLongLong + (PyObject *pylong) + + Return a C ‘unsigned long long’ representation of `pylong'. + `pylong' must be an instance of *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `pylong' is out of + range for an ‘unsigned long long’. + + Returns ‘(unsigned long long)-1’ on error. Use *note + PyErr_Occurred(): 383f. to disambiguate. + + Changed in version 3.1: A negative `pylong' now raises *note + OverflowError: 960, not *note TypeError: 192. + + -- C Function: unsigned long PyLong_AsUnsignedLongMask (PyObject *obj) + + Return a C ‘unsigned long’ representation of `obj'. If `obj' is + not an instance of *note PyLongObject: 3ada, first call its *note + __index__(): 18a. or *note __int__(): 18b. method (if present) to + convert it to a *note PyLongObject: 3ada. + + If the value of `obj' is out of range for an ‘unsigned long’, + return the reduction of that value modulo ‘ULONG_MAX + 1’. + + Returns ‘(unsigned long)-1’ on error. Use *note PyErr_Occurred(): + 383f. to disambiguate. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + Deprecated since version 3.8: Using *note __int__(): 18b. is + deprecated. + + -- C Function: unsigned long long PyLong_AsUnsignedLongLongMask + (PyObject *obj) + + Return a C ‘unsigned long long’ representation of `obj'. If `obj' + is not an instance of *note PyLongObject: 3ada, first call its + *note __index__(): 18a. or *note __int__(): 18b. method (if + present) to convert it to a *note PyLongObject: 3ada. + + If the value of `obj' is out of range for an ‘unsigned long long’, + return the reduction of that value modulo ‘PY_ULLONG_MAX + 1’. + + Returns ‘(unsigned long long)-1’ on error. Use *note + PyErr_Occurred(): 383f. to disambiguate. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + Deprecated since version 3.8: Using *note __int__(): 18b. is + deprecated. + + -- C Function: double PyLong_AsDouble (PyObject *pylong) + + Return a C ‘double’ representation of `pylong'. `pylong' must be + an instance of *note PyLongObject: 3ada. + + Raise *note OverflowError: 960. if the value of `pylong' is out of + range for a ‘double’. + + Returns ‘-1.0’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + -- C Function: void* PyLong_AsVoidPtr (PyObject *pylong) + + Convert a Python integer `pylong' to a C ‘void’ pointer. If + `pylong' cannot be converted, an *note OverflowError: 960. will be + raised. This is only assured to produce a usable ‘void’ pointer + for values created with *note PyLong_FromVoidPtr(): 3ae7. + + Returns ‘NULL’ on error. Use *note PyErr_Occurred(): 383f. to + disambiguate. + + +File: python.info, Node: Boolean Objects, Next: Floating Point Objects, Prev: Integer Objects, Up: Numeric Objects + +7.8.2.2 Boolean Objects +....................... + +Booleans in Python are implemented as a subclass of integers. There are +only two booleans, ‘Py_False’ and ‘Py_True’. As such, the normal +creation and deletion functions don’t apply to booleans. The following +macros are available, however. + + -- C Function: int PyBool_Check (PyObject *o) + + Return true if `o' is of type ‘PyBool_Type’. + + -- C Variable: PyObject* Py_False + + The Python ‘False’ object. This object has no methods. It needs + to be treated just like any other object with respect to reference + counts. + + -- C Variable: PyObject* Py_True + + The Python ‘True’ object. This object has no methods. It needs to + be treated just like any other object with respect to reference + counts. + + -- C Macro: Py_RETURN_FALSE + + Return ‘Py_False’ from a function, properly incrementing its + reference count. + + -- C Macro: Py_RETURN_TRUE + + Return ‘Py_True’ from a function, properly incrementing its + reference count. + + -- C Function: PyObject* PyBool_FromLong (long v) + `Return value: New reference.' Return a new reference to ‘Py_True’ + or ‘Py_False’ depending on the truth value of `v'. + + +File: python.info, Node: Floating Point Objects, Next: Complex Number Objects, Prev: Boolean Objects, Up: Numeric Objects + +7.8.2.3 Floating Point Objects +.............................. + + -- C Type: PyFloatObject + + This subtype of *note PyObject: 4ba. represents a Python floating + point object. + + -- C Variable: PyTypeObject PyFloat_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + floating point type. This is the same object as *note float: 187. + in the Python layer. + + -- C Function: int PyFloat_Check (PyObject *p) + + Return true if its argument is a *note PyFloatObject: 3afa. or a + subtype of *note PyFloatObject: 3afa. + + -- C Function: int PyFloat_CheckExact (PyObject *p) + + Return true if its argument is a *note PyFloatObject: 3afa, but not + a subtype of *note PyFloatObject: 3afa. + + -- C Function: PyObject* PyFloat_FromString (PyObject *str) + `Return value: New reference.' Create a *note PyFloatObject: 3afa. + object based on the string value in `str', or ‘NULL’ on failure. + + -- C Function: PyObject* PyFloat_FromDouble (double v) + `Return value: New reference.' Create a *note PyFloatObject: 3afa. + object from `v', or ‘NULL’ on failure. + + -- C Function: double PyFloat_AsDouble (PyObject *pyfloat) + + Return a C ‘double’ representation of the contents of `pyfloat'. + If `pyfloat' is not a Python floating point object but has a *note + __float__(): 18c. method, this method will first be called to + convert `pyfloat' into a float. If ‘__float__()’ is not defined + then it falls back to *note __index__(): 18a. This method returns + ‘-1.0’ upon failure, so one should call *note PyErr_Occurred(): + 383f. to check for errors. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + -- C Function: double PyFloat_AS_DOUBLE (PyObject *pyfloat) + + Return a C ‘double’ representation of the contents of `pyfloat', + but without error checking. + + -- C Function: PyObject* PyFloat_GetInfo (void) + `Return value: New reference.' Return a structseq instance which + contains information about the precision, minimum and maximum + values of a float. It’s a thin wrapper around the header file + ‘float.h’. + + -- C Function: double PyFloat_GetMax () + + Return the maximum representable finite float `DBL_MAX' as C + ‘double’. + + -- C Function: double PyFloat_GetMin () + + Return the minimum normalized positive float `DBL_MIN' as C + ‘double’. + + -- C Function: int PyFloat_ClearFreeList () + + Clear the float free list. Return the number of items that could + not be freed. + + +File: python.info, Node: Complex Number Objects, Prev: Floating Point Objects, Up: Numeric Objects + +7.8.2.4 Complex Number Objects +.............................. + +Python’s complex number objects are implemented as two distinct types +when viewed from the C API: one is the Python object exposed to Python +programs, and the other is a C structure which represents the actual +complex number value. The API provides functions for working with both. + +* Menu: + +* Complex Numbers as C Structures:: +* Complex Numbers as Python Objects:: + + +File: python.info, Node: Complex Numbers as C Structures, Next: Complex Numbers as Python Objects, Up: Complex Number Objects + +7.8.2.5 Complex Numbers as C Structures +....................................... + +Note that the functions which accept these structures as parameters and +return them as results do so `by value' rather than dereferencing them +through pointers. This is consistent throughout the API. + + -- C Type: Py_complex + + The C structure which corresponds to the value portion of a Python + complex number object. Most of the functions for dealing with + complex number objects use structures of this type as input or + output values, as appropriate. It is defined as: + + typedef struct { + double real; + double imag; + } Py_complex; + + -- C Function: Py_complex _Py_c_sum (Py_complex left, Py_complex right) + + Return the sum of two complex numbers, using the C *note + Py_complex: 39cb. representation. + + -- C Function: Py_complex _Py_c_diff (Py_complex left, + Py_complex right) + + Return the difference between two complex numbers, using the C + *note Py_complex: 39cb. representation. + + -- C Function: Py_complex _Py_c_neg (Py_complex complex) + + Return the negation of the complex number `complex', using the C + *note Py_complex: 39cb. representation. + + -- C Function: Py_complex _Py_c_prod (Py_complex left, + Py_complex right) + + Return the product of two complex numbers, using the C *note + Py_complex: 39cb. representation. + + -- C Function: Py_complex _Py_c_quot (Py_complex dividend, + Py_complex divisor) + + Return the quotient of two complex numbers, using the C *note + Py_complex: 39cb. representation. + + If `divisor' is null, this method returns zero and sets ‘errno’ to + ‘EDOM’. + + -- C Function: Py_complex _Py_c_pow (Py_complex num, Py_complex exp) + + Return the exponentiation of `num' by `exp', using the C *note + Py_complex: 39cb. representation. + + If `num' is null and `exp' is not a positive real number, this + method returns zero and sets ‘errno’ to ‘EDOM’. + + +File: python.info, Node: Complex Numbers as Python Objects, Prev: Complex Numbers as C Structures, Up: Complex Number Objects + +7.8.2.6 Complex Numbers as Python Objects +......................................... + + -- C Type: PyComplexObject + + This subtype of *note PyObject: 4ba. represents a Python complex + number object. + + -- C Variable: PyTypeObject PyComplex_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + complex number type. It is the same object as *note complex: 189. + in the Python layer. + + -- C Function: int PyComplex_Check (PyObject *p) + + Return true if its argument is a *note PyComplexObject: 3b0d. or a + subtype of *note PyComplexObject: 3b0d. + + -- C Function: int PyComplex_CheckExact (PyObject *p) + + Return true if its argument is a *note PyComplexObject: 3b0d, but + not a subtype of *note PyComplexObject: 3b0d. + + -- C Function: PyObject* PyComplex_FromCComplex (Py_complex v) + `Return value: New reference.' Create a new Python complex number + object from a C *note Py_complex: 39cb. value. + + -- C Function: PyObject* PyComplex_FromDoubles (double real, + double imag) + `Return value: New reference.' Return a new *note PyComplexObject: + 3b0d. object from `real' and `imag'. + + -- C Function: double PyComplex_RealAsDouble (PyObject *op) + + Return the real part of `op' as a C ‘double’. + + -- C Function: double PyComplex_ImagAsDouble (PyObject *op) + + Return the imaginary part of `op' as a C ‘double’. + + -- C Function: Py_complex PyComplex_AsCComplex (PyObject *op) + + Return the *note Py_complex: 39cb. value of the complex number + `op'. + + If `op' is not a Python complex number object but has a *note + __complex__(): 18d. method, this method will first be called to + convert `op' to a Python complex number object. If ‘__complex__()’ + is not defined then it falls back to *note __float__(): 18c. If + ‘__float__()’ is not defined then it falls back to *note + __index__(): 18a. Upon failure, this method returns ‘-1.0’ as a + real value. + + Changed in version 3.8: Use *note __index__(): 18a. if available. + + +File: python.info, Node: Sequence Objects, Next: Container Objects, Prev: Numeric Objects, Up: Concrete Objects Layer + +7.8.3 Sequence Objects +---------------------- + +Generic operations on sequence objects were discussed in the previous +chapter; this section deals with the specific kinds of sequence objects +that are intrinsic to the Python language. + +* Menu: + +* Bytes Objects: Bytes Objects<2>. +* Byte Array Objects:: +* Unicode Objects and Codecs:: +* Tuple Objects:: +* Struct Sequence Objects:: +* List Objects:: + + +File: python.info, Node: Bytes Objects<2>, Next: Byte Array Objects, Up: Sequence Objects + +7.8.3.1 Bytes Objects +..................... + +These functions raise *note TypeError: 192. when expecting a bytes +parameter and are called with a non-bytes parameter. + + -- C Type: PyBytesObject + + This subtype of *note PyObject: 4ba. represents a Python bytes + object. + + -- C Variable: PyTypeObject PyBytes_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + bytes type; it is the same object as *note bytes: 331. in the + Python layer. + + -- C Function: int PyBytes_Check (PyObject *o) + + Return true if the object `o' is a bytes object or an instance of a + subtype of the bytes type. + + -- C Function: int PyBytes_CheckExact (PyObject *o) + + Return true if the object `o' is a bytes object, but not an + instance of a subtype of the bytes type. + + -- C Function: PyObject* PyBytes_FromString (const char *v) + `Return value: New reference.' Return a new bytes object with a + copy of the string `v' as value on success, and ‘NULL’ on failure. + The parameter `v' must not be ‘NULL’; it will not be checked. + + -- C Function: PyObject* PyBytes_FromStringAndSize (const char *v, + Py_ssize_t len) + `Return value: New reference.' Return a new bytes object with a + copy of the string `v' as value and length `len' on success, and + ‘NULL’ on failure. If `v' is ‘NULL’, the contents of the bytes + object are uninitialized. + + -- C Function: PyObject* PyBytes_FromFormat (const char *format, ...) + `Return value: New reference.' Take a C ‘printf()’-style `format' + string and a variable number of arguments, calculate the size of + the resulting Python bytes object and return a bytes object with + the values formatted into it. The variable arguments must be C + types and must correspond exactly to the format characters in the + `format' string. The following format characters are allowed: + + Format Characters Type Comment + + --------------------------------------------------------------------------------- + + ‘%%’ `n/a' The literal % character. + + + ‘%c’ int A single byte, represented as a C + int. + + + ‘%d’ int Equivalent to ‘printf("%d")’. (1) + + + ‘%u’ unsigned int Equivalent to ‘printf("%u")’. (2) + + + ‘%ld’ long Equivalent to ‘printf("%ld")’. + (3) + + + ‘%lu’ unsigned long Equivalent to ‘printf("%lu")’. + (4) + + + ‘%zd’ Py_ssize_t Equivalent to ‘printf("%zd")’. + (5) + + + ‘%zu’ size_t Equivalent to ‘printf("%zu")’. + (6) + + + ‘%i’ int Equivalent to ‘printf("%i")’. (7) + + + ‘%x’ int Equivalent to ‘printf("%x")’. (8) + + + ‘%s’ const char* A null-terminated C character + array. + + + ‘%p’ const void* The hex representation of a C + pointer. Mostly equivalent to + ‘printf("%p")’ except that it is + guaranteed to start with the + literal ‘0x’ regardless of what + the platform’s ‘printf’ yields. + + + An unrecognized format character causes all the rest of the format + string to be copied as-is to the result object, and any extra + arguments discarded. + + -- C Function: PyObject* PyBytes_FromFormatV (const char *format, + va_list vargs) + `Return value: New reference.' Identical to *note + PyBytes_FromFormat(): 3b1d. except that it takes exactly two + arguments. + + -- C Function: PyObject* PyBytes_FromObject (PyObject *o) + `Return value: New reference.' Return the bytes representation of + object `o' that implements the buffer protocol. + + -- C Function: Py_ssize_t PyBytes_Size (PyObject *o) + + Return the length of the bytes in bytes object `o'. + + -- C Function: Py_ssize_t PyBytes_GET_SIZE (PyObject *o) + + Macro form of *note PyBytes_Size(): 3b20. but without error + checking. + + -- C Function: char* PyBytes_AsString (PyObject *o) + + Return a pointer to the contents of `o'. The pointer refers to the + internal buffer of `o', which consists of ‘len(o) + 1’ bytes. The + last byte in the buffer is always null, regardless of whether there + are any other null bytes. The data must not be modified in any + way, unless the object was just created using + ‘PyBytes_FromStringAndSize(NULL, size)’. It must not be + deallocated. If `o' is not a bytes object at all, *note + PyBytes_AsString(): 3b22. returns ‘NULL’ and raises *note + TypeError: 192. + + -- C Function: char* PyBytes_AS_STRING (PyObject *string) + + Macro form of *note PyBytes_AsString(): 3b22. but without error + checking. + + -- C Function: int PyBytes_AsStringAndSize (PyObject *obj, + char **buffer, Py_ssize_t *length) + + Return the null-terminated contents of the object `obj' through the + output variables `buffer' and `length'. + + If `length' is ‘NULL’, the bytes object may not contain embedded + null bytes; if it does, the function returns ‘-1’ and a *note + ValueError: 1fb. is raised. + + The buffer refers to an internal buffer of `obj', which includes an + additional null byte at the end (not counted in `length'). The + data must not be modified in any way, unless the object was just + created using ‘PyBytes_FromStringAndSize(NULL, size)’. It must not + be deallocated. If `obj' is not a bytes object at all, *note + PyBytes_AsStringAndSize(): 3b24. returns ‘-1’ and raises *note + TypeError: 192. + + Changed in version 3.5: Previously, *note TypeError: 192. was + raised when embedded null bytes were encountered in the bytes + object. + + -- C Function: void PyBytes_Concat (PyObject **bytes, + PyObject *newpart) + + Create a new bytes object in `*bytes' containing the contents of + `newpart' appended to `bytes'; the caller will own the new + reference. The reference to the old value of `bytes' will be + stolen. If the new object cannot be created, the old reference to + `bytes' will still be discarded and the value of `*bytes' will be + set to ‘NULL’; the appropriate exception will be set. + + -- C Function: void PyBytes_ConcatAndDel (PyObject **bytes, + PyObject *newpart) + + Create a new bytes object in `*bytes' containing the contents of + `newpart' appended to `bytes'. This version decrements the + reference count of `newpart'. + + -- C Function: int _PyBytes_Resize (PyObject **bytes, + Py_ssize_t newsize) + + A way to resize a bytes object even though it is “immutable”. Only + use this to build up a brand new bytes object; don’t use this if + the bytes may already be known in other parts of the code. It is + an error to call this function if the refcount on the input bytes + object is not one. Pass the address of an existing bytes object as + an lvalue (it may be written into), and the new size desired. On + success, `*bytes' holds the resized bytes object and ‘0’ is + returned; the address in `*bytes' may differ from its input value. + If the reallocation fails, the original bytes object at `*bytes' is + deallocated, `*bytes' is set to ‘NULL’, *note MemoryError: 13af. is + set, and ‘-1’ is returned. + + ---------- Footnotes ---------- + + (1) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (2) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (3) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (4) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (5) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (6) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (7) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + (8) (1) For integer specifiers (d, u, ld, lu, zd, zu, i, x): the +0-conversion flag has effect even when a precision is given. + + +File: python.info, Node: Byte Array Objects, Next: Unicode Objects and Codecs, Prev: Bytes Objects<2>, Up: Sequence Objects + +7.8.3.2 Byte Array Objects +.......................... + + -- C Type: PyByteArrayObject + + This subtype of *note PyObject: 4ba. represents a Python bytearray + object. + + -- C Variable: PyTypeObject PyByteArray_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + bytearray type; it is the same object as *note bytearray: 332. in + the Python layer. + +* Menu: + +* Type check macros:: +* Direct API functions:: +* Macros:: + + +File: python.info, Node: Type check macros, Next: Direct API functions, Up: Byte Array Objects + +7.8.3.3 Type check macros +......................... + + -- C Function: int PyByteArray_Check (PyObject *o) + + Return true if the object `o' is a bytearray object or an instance + of a subtype of the bytearray type. + + -- C Function: int PyByteArray_CheckExact (PyObject *o) + + Return true if the object `o' is a bytearray object, but not an + instance of a subtype of the bytearray type. + + +File: python.info, Node: Direct API functions, Next: Macros, Prev: Type check macros, Up: Byte Array Objects + +7.8.3.4 Direct API functions +............................ + + -- C Function: PyObject* PyByteArray_FromObject (PyObject *o) + `Return value: New reference.' Return a new bytearray object from + any object, `o', that implements the *note buffer protocol: 1291. + + -- C Function: PyObject* PyByteArray_FromStringAndSize (const + char *string, Py_ssize_t len) + `Return value: New reference.' Create a new bytearray object from + `string' and its length, `len'. On failure, ‘NULL’ is returned. + + -- C Function: PyObject* PyByteArray_Concat (PyObject *a, PyObject *b) + `Return value: New reference.' Concat bytearrays `a' and `b' and + return a new bytearray with the result. + + -- C Function: Py_ssize_t PyByteArray_Size (PyObject *bytearray) + + Return the size of `bytearray' after checking for a ‘NULL’ pointer. + + -- C Function: char* PyByteArray_AsString (PyObject *bytearray) + + Return the contents of `bytearray' as a char array after checking + for a ‘NULL’ pointer. The returned array always has an extra null + byte appended. + + -- C Function: int PyByteArray_Resize (PyObject *bytearray, + Py_ssize_t len) + + Resize the internal buffer of `bytearray' to `len'. + + +File: python.info, Node: Macros, Prev: Direct API functions, Up: Byte Array Objects + +7.8.3.5 Macros +.............. + +These macros trade safety for speed and they don’t check pointers. + + -- C Function: char* PyByteArray_AS_STRING (PyObject *bytearray) + + Macro version of *note PyByteArray_AsString(): 3b33. + + -- C Function: Py_ssize_t PyByteArray_GET_SIZE (PyObject *bytearray) + + Macro version of *note PyByteArray_Size(): 3b32. + + +File: python.info, Node: Unicode Objects and Codecs, Next: Tuple Objects, Prev: Byte Array Objects, Up: Sequence Objects + +7.8.3.6 Unicode Objects and Codecs +.................................. + +* Menu: + +* Unicode Objects:: +* Built-in Codecs:: +* Methods and Slot Functions:: + + +File: python.info, Node: Unicode Objects, Next: Built-in Codecs, Up: Unicode Objects and Codecs + +7.8.3.7 Unicode Objects +....................... + +Since the implementation of PEP 393(1) in Python 3.3, Unicode objects +internally use a variety of representations, in order to allow handling +the complete range of Unicode characters while staying memory efficient. +There are special cases for strings where all code points are below 128, +256, or 65536; otherwise, code points must be below 1114112 (which is +the full Unicode range). + +*note Py_UNICODE*: aee. and UTF-8 representations are created on demand +and cached in the Unicode object. The *note Py_UNICODE*: aee. +representation is deprecated and inefficient. + +Due to the transition between the old APIs and the new APIs, Unicode +objects can internally be in two states depending on how they were +created: + + * “canonical” Unicode objects are all objects created by a + non-deprecated Unicode API. They use the most efficient + representation allowed by the implementation. + + * “legacy” Unicode objects have been created through one of the + deprecated APIs (typically *note PyUnicode_FromUnicode(): aef.) and + only bear the *note Py_UNICODE*: aee. representation; you will have + to call *note PyUnicode_READY(): ad6. on them before calling any + other API. + + Note: The “legacy” Unicode object will be removed in Python 3.12 + with deprecated APIs. All Unicode objects will be “canonical” + since then. See PEP 623(2) for more information. + +* Menu: + +* Unicode Type:: +* Unicode Character Properties:: +* Creating and accessing Unicode strings:: +* Deprecated Py_UNICODE APIs:: +* Locale Encoding:: +* File System Encoding:: +* wchar_t Support:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + (2) https://www.python.org/dev/peps/pep-0623 + + +File: python.info, Node: Unicode Type, Next: Unicode Character Properties, Up: Unicode Objects + +7.8.3.8 Unicode Type +.................... + +These are the basic Unicode object types used for the Unicode +implementation in Python: + + -- C Type: Py_UCS4 + -- C Type: Py_UCS2 + -- C Type: Py_UCS1 + + These types are typedefs for unsigned integer types wide enough to + contain characters of 32 bits, 16 bits and 8 bits, respectively. + When dealing with single Unicode characters, use *note Py_UCS4: + ad3. + + New in version 3.3. + + -- C Type: Py_UNICODE + + This is a typedef of ‘wchar_t’, which is a 16-bit type or 32-bit + type depending on the platform. + + Changed in version 3.3: In previous versions, this was a 16-bit + type or a 32-bit type depending on whether you selected a “narrow” + or “wide” Unicode version of Python at build time. + + -- C Type: PyASCIIObject + -- C Type: PyCompactUnicodeObject + -- C Type: PyUnicodeObject + + These subtypes of *note PyObject: 4ba. represent a Python Unicode + object. In almost all cases, they shouldn’t be used directly, + since all API functions that deal with Unicode objects take and + return *note PyObject: 4ba. pointers. + + New in version 3.3. + + -- C Variable: PyTypeObject PyUnicode_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + Unicode type. It is exposed to Python code as ‘str’. + +The following APIs are really C macros and can be used to do fast checks +and to access internal read-only data of Unicode objects: + + -- C Function: int PyUnicode_Check (PyObject *o) + + Return true if the object `o' is a Unicode object or an instance of + a Unicode subtype. + + -- C Function: int PyUnicode_CheckExact (PyObject *o) + + Return true if the object `o' is a Unicode object, but not an + instance of a subtype. + + -- C Function: int PyUnicode_READY (PyObject *o) + + Ensure the string object `o' is in the “canonical” representation. + This is required before using any of the access macros described + below. + + Returns ‘0’ on success and ‘-1’ with an exception set on failure, + which in particular happens if memory allocation fails. + + New in version 3.3. + + Deprecated since version 3.10, will be removed in version 3.12: + This API will be removed with *note PyUnicode_FromUnicode(): aef. + + -- C Function: Py_ssize_t PyUnicode_GET_LENGTH (PyObject *o) + + Return the length of the Unicode string, in code points. `o' has + to be a Unicode object in the “canonical” representation (not + checked). + + New in version 3.3. + + -- C Function: Py_UCS1* PyUnicode_1BYTE_DATA (PyObject *o) + -- C Function: Py_UCS2* PyUnicode_2BYTE_DATA (PyObject *o) + -- C Function: Py_UCS4* PyUnicode_4BYTE_DATA (PyObject *o) + + Return a pointer to the canonical representation cast to UCS1, UCS2 + or UCS4 integer types for direct character access. No checks are + performed if the canonical representation has the correct character + size; use *note PyUnicode_KIND(): ade. to select the right macro. + Make sure *note PyUnicode_READY(): ad6. has been called before + accessing this. + + New in version 3.3. + + -- C Macro: PyUnicode_WCHAR_KIND + -- C Macro: PyUnicode_1BYTE_KIND + -- C Macro: PyUnicode_2BYTE_KIND + -- C Macro: PyUnicode_4BYTE_KIND + + Return values of the *note PyUnicode_KIND(): ade. macro. + + New in version 3.3. + + Deprecated since version 3.10, will be removed in version 3.12: + ‘PyUnicode_WCHAR_KIND’ is deprecated. + + -- C Function: int PyUnicode_KIND (PyObject *o) + + Return one of the PyUnicode kind constants (see above) that + indicate how many bytes per character this Unicode object uses to + store its data. `o' has to be a Unicode object in the “canonical” + representation (not checked). + + New in version 3.3. + + -- C Function: void* PyUnicode_DATA (PyObject *o) + + Return a void pointer to the raw Unicode buffer. `o' has to be a + Unicode object in the “canonical” representation (not checked). + + New in version 3.3. + + -- C Function: void PyUnicode_WRITE (int kind, void *data, + Py_ssize_t index, Py_UCS4 value) + + Write into a canonical representation `data' (as obtained with + *note PyUnicode_DATA(): ada.). This macro does not do any sanity + checks and is intended for usage in loops. The caller should cache + the `kind' value and `data' pointer as obtained from other macro + calls. `index' is the index in the string (starts at 0) and + `value' is the new code point value which should be written to that + location. + + New in version 3.3. + + -- C Function: Py_UCS4 PyUnicode_READ (int kind, void *data, + Py_ssize_t index) + + Read a code point from a canonical representation `data' (as + obtained with *note PyUnicode_DATA(): ada.). No checks or ready + calls are performed. + + New in version 3.3. + + -- C Function: Py_UCS4 PyUnicode_READ_CHAR (PyObject *o, + Py_ssize_t index) + + Read a character from a Unicode object `o', which must be in the + “canonical” representation. This is less efficient than *note + PyUnicode_READ(): ae3. if you do multiple consecutive reads. + + New in version 3.3. + + -- C Macro: PyUnicode_MAX_CHAR_VALUE (o) + + Return the maximum code point that is suitable for creating another + string based on `o', which must be in the “canonical” + representation. This is always an approximation but more efficient + than iterating over the string. + + New in version 3.3. + + -- C Function: int PyUnicode_ClearFreeList () + + Clear the free list. Return the total number of freed items. + + -- C Function: Py_ssize_t PyUnicode_GET_SIZE (PyObject *o) + + Return the size of the deprecated *note Py_UNICODE: aee. + representation, in code units (this includes surrogate pairs as 2 + units). `o' has to be a Unicode object (not checked). + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using *note + PyUnicode_GET_LENGTH(): acc. + + -- C Function: Py_ssize_t PyUnicode_GET_DATA_SIZE (PyObject *o) + + Return the size of the deprecated *note Py_UNICODE: aee. + representation in bytes. `o' has to be a Unicode object (not + checked). + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using *note + PyUnicode_GET_LENGTH(): acc. + + -- C Function: Py_UNICODE* PyUnicode_AS_UNICODE (PyObject *o) + -- C Function: const char* PyUnicode_AS_DATA (PyObject *o) + + Return a pointer to a *note Py_UNICODE: aee. representation of the + object. The returned buffer is always terminated with an extra + null code point. It may also contain embedded null code points, + which would cause the string to be truncated when used in most C + functions. The ‘AS_DATA’ form casts the pointer to ‘const char *’. + The `o' argument has to be a Unicode object (not checked). + + Changed in version 3.3: This macro is now inefficient – because in + many cases the *note Py_UNICODE: aee. representation does not exist + and needs to be created – and can fail (return ‘NULL’ with an + exception set). Try to port the code to use the new + ‘PyUnicode_nBYTE_DATA()’ macros or use *note PyUnicode_WRITE(): + ae5. or *note PyUnicode_READ(): ae3. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using the + ‘PyUnicode_nBYTE_DATA()’ family of macros. + + +File: python.info, Node: Unicode Character Properties, Next: Creating and accessing Unicode strings, Prev: Unicode Type, Up: Unicode Objects + +7.8.3.9 Unicode Character Properties +.................................... + +Unicode provides many different character properties. The most often +needed ones are available through these macros which are mapped to C +functions depending on the Python configuration. + + -- C Function: int Py_UNICODE_ISSPACE (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a whitespace + character. + + -- C Function: int Py_UNICODE_ISLOWER (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a lowercase + character. + + -- C Function: int Py_UNICODE_ISUPPER (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is an uppercase + character. + + -- C Function: int Py_UNICODE_ISTITLE (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a titlecase + character. + + -- C Function: int Py_UNICODE_ISLINEBREAK (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a linebreak + character. + + -- C Function: int Py_UNICODE_ISDECIMAL (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a decimal character. + + -- C Function: int Py_UNICODE_ISDIGIT (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a digit character. + + -- C Function: int Py_UNICODE_ISNUMERIC (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a numeric character. + + -- C Function: int Py_UNICODE_ISALPHA (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is an alphabetic + character. + + -- C Function: int Py_UNICODE_ISALNUM (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is an alphanumeric + character. + + -- C Function: int Py_UNICODE_ISPRINTABLE (Py_UNICODE ch) + + Return ‘1’ or ‘0’ depending on whether `ch' is a printable + character. Nonprintable characters are those characters defined in + the Unicode character database as “Other” or “Separator”, excepting + the ASCII space (0x20) which is considered printable. (Note that + printable characters in this context are those which should not be + escaped when *note repr(): 7cc. is invoked on a string. It has no + bearing on the handling of strings written to *note sys.stdout: + 30e. or *note sys.stderr: 30f.) + +These APIs can be used for fast direct character conversions: + + -- C Function: Py_UNICODE Py_UNICODE_TOLOWER (Py_UNICODE ch) + + Return the character `ch' converted to lower case. + + Deprecated since version 3.3: This function uses simple case + mappings. + + -- C Function: Py_UNICODE Py_UNICODE_TOUPPER (Py_UNICODE ch) + + Return the character `ch' converted to upper case. + + Deprecated since version 3.3: This function uses simple case + mappings. + + -- C Function: Py_UNICODE Py_UNICODE_TOTITLE (Py_UNICODE ch) + + Return the character `ch' converted to title case. + + Deprecated since version 3.3: This function uses simple case + mappings. + + -- C Function: int Py_UNICODE_TODECIMAL (Py_UNICODE ch) + + Return the character `ch' converted to a decimal positive integer. + Return ‘-1’ if this is not possible. This macro does not raise + exceptions. + + -- C Function: int Py_UNICODE_TODIGIT (Py_UNICODE ch) + + Return the character `ch' converted to a single digit integer. + Return ‘-1’ if this is not possible. This macro does not raise + exceptions. + + -- C Function: double Py_UNICODE_TONUMERIC (Py_UNICODE ch) + + Return the character `ch' converted to a double. Return ‘-1.0’ if + this is not possible. This macro does not raise exceptions. + +These APIs can be used to work with surrogates: + + -- C Macro: Py_UNICODE_IS_SURROGATE (ch) + + Check if `ch' is a surrogate (‘0xD800 <= ch <= 0xDFFF’). + + -- C Macro: Py_UNICODE_IS_HIGH_SURROGATE (ch) + + Check if `ch' is a high surrogate (‘0xD800 <= ch <= 0xDBFF’). + + -- C Macro: Py_UNICODE_IS_LOW_SURROGATE (ch) + + Check if `ch' is a low surrogate (‘0xDC00 <= ch <= 0xDFFF’). + + -- C Macro: Py_UNICODE_JOIN_SURROGATES (high, low) + + Join two surrogate characters and return a single Py_UCS4 value. + `high' and `low' are respectively the leading and trailing + surrogates in a surrogate pair. + + +File: python.info, Node: Creating and accessing Unicode strings, Next: Deprecated Py_UNICODE APIs, Prev: Unicode Character Properties, Up: Unicode Objects + +7.8.3.10 Creating and accessing Unicode strings +............................................... + +To create Unicode objects and access their basic sequence properties, +use these APIs: + + -- C Function: PyObject* PyUnicode_New (Py_ssize_t size, + Py_UCS4 maxchar) + `Return value: New reference.' Create a new Unicode object. + `maxchar' should be the true maximum code point to be placed in the + string. As an approximation, it can be rounded up to the nearest + value in the sequence 127, 255, 65535, 1114111. + + This is the recommended way to allocate a new Unicode object. + Objects created using this function are not resizable. + + New in version 3.3. + + -- C Function: PyObject* PyUnicode_FromKindAndData (int kind, const + void *buffer, Py_ssize_t size) + `Return value: New reference.' Create a new Unicode object with + the given `kind' (possible values are *note PyUnicode_1BYTE_KIND: + ae0. etc., as returned by *note PyUnicode_KIND(): ade.). The + `buffer' must point to an array of `size' units of 1, 2 or 4 bytes + per character, as given by the kind. + + New in version 3.3. + + -- C Function: PyObject* PyUnicode_FromStringAndSize (const char *u, + Py_ssize_t size) + `Return value: New reference.' Create a Unicode object from the + char buffer `u'. The bytes will be interpreted as being UTF-8 + encoded. The buffer is copied into the new object. If the buffer + is not ‘NULL’, the return value might be a shared object, i.e. + modification of the data is not allowed. + + If `u' is ‘NULL’, this function behaves like *note + PyUnicode_FromUnicode(): aef. with the buffer set to ‘NULL’. This + usage is deprecated in favor of *note PyUnicode_New(): acd, and + will be removed in Python 3.12. + + -- C Function: PyObject *PyUnicode_FromString (const char *u) + `Return value: New reference.' Create a Unicode object from a + UTF-8 encoded null-terminated char buffer `u'. + + -- C Function: PyObject* PyUnicode_FromFormat (const char *format, ...) + `Return value: New reference.' Take a C ‘printf()’-style `format' + string and a variable number of arguments, calculate the size of + the resulting Python Unicode string and return a string with the + values formatted into it. The variable arguments must be C types + and must correspond exactly to the format characters in the + `format' ASCII-encoded string. The following format characters are + allowed: + + Format Characters Type Comment + + ----------------------------------------------------------------------------------------- + + ‘%%’ `n/a' The literal % character. + + + ‘%c’ int A single character, represented as a + C int. + + + ‘%d’ int Equivalent to ‘printf("%d")’. (1) + + + ‘%u’ unsigned int Equivalent to ‘printf("%u")’. (2) + + + ‘%ld’ long Equivalent to ‘printf("%ld")’. (3) + + + ‘%li’ long Equivalent to ‘printf("%li")’. (4) + + + ‘%lu’ unsigned long Equivalent to ‘printf("%lu")’. (5) + + + ‘%lld’ long long Equivalent to ‘printf("%lld")’. (6) + + + ‘%lli’ long long Equivalent to ‘printf("%lli")’. (7) + + + ‘%llu’ unsigned long long Equivalent to ‘printf("%llu")’. (8) + + + ‘%zd’ Py_ssize_t Equivalent to ‘printf("%zd")’. (9) + + + ‘%zi’ Py_ssize_t Equivalent to ‘printf("%zi")’. (10) + + + ‘%zu’ size_t Equivalent to ‘printf("%zu")’. (11) + + + ‘%i’ int Equivalent to ‘printf("%i")’. (12) + + + ‘%x’ int Equivalent to ‘printf("%x")’. (13) + + + ‘%s’ const char* A null-terminated C character array. + + + ‘%p’ const void* The hex representation of a C + pointer. Mostly equivalent to + ‘printf("%p")’ except that it is + guaranteed to start with the literal + ‘0x’ regardless of what the + platform’s ‘printf’ yields. + + + ‘%A’ PyObject* The result of calling + *note ascii(): d4c. + + + ‘%U’ PyObject* A Unicode object. + + + ‘%V’ PyObject*, const char* A Unicode object (which may be + ‘NULL’) and a null-terminated C + character array as a second + parameter (which will be used, if + the first parameter is ‘NULL’). + + + ‘%S’ PyObject* The result of calling + *note PyObject_Str(): 969. + + + ‘%R’ PyObject* The result of calling + *note PyObject_Repr(): 968. + + + An unrecognized format character causes all the rest of the format + string to be copied as-is to the result string, and any extra + arguments discarded. + + Note: The width formatter unit is number of characters rather + than bytes. The precision formatter unit is number of bytes + for ‘"%s"’ and ‘"%V"’ (if the ‘PyObject*’ argument is ‘NULL’), + and a number of characters for ‘"%A"’, ‘"%U"’, ‘"%S"’, ‘"%R"’ + and ‘"%V"’ (if the ‘PyObject*’ argument is not ‘NULL’). + + Changed in version 3.2: Support for ‘"%lld"’ and ‘"%llu"’ added. + + Changed in version 3.3: Support for ‘"%li"’, ‘"%lli"’ and ‘"%zi"’ + added. + + Changed in version 3.4: Support width and precision formatter for + ‘"%s"’, ‘"%A"’, ‘"%U"’, ‘"%V"’, ‘"%S"’, ‘"%R"’ added. + + -- C Function: PyObject* PyUnicode_FromFormatV (const char *format, + va_list vargs) + `Return value: New reference.' Identical to *note + PyUnicode_FromFormat(): 7cb. except that it takes exactly two + arguments. + + -- C Function: PyObject* PyUnicode_FromEncodedObject (PyObject *obj, + const char *encoding, const char *errors) + `Return value: New reference.' Decode an encoded object `obj' to a + Unicode object. + + *note bytes: 331, *note bytearray: 332. and other *note bytes-like + objects: 5e8. are decoded according to the given `encoding' and + using the error handling defined by `errors'. Both can be ‘NULL’ + to have the interface use the default values (see *note Built-in + Codecs: 3b5a. for details). + + All other objects, including Unicode objects, cause a *note + TypeError: 192. to be set. + + The API returns ‘NULL’ if there was an error. The caller is + responsible for decref’ing the returned objects. + + -- C Function: Py_ssize_t PyUnicode_GetLength (PyObject *unicode) + + Return the length of the Unicode object, in code points. + + New in version 3.3. + + -- C Function: Py_ssize_t PyUnicode_CopyCharacters (PyObject *to, + Py_ssize_t to_start, PyObject *from, Py_ssize_t from_start, + Py_ssize_t how_many) + + Copy characters from one Unicode object into another. This + function performs character conversion when necessary and falls + back to ‘memcpy()’ if possible. Returns ‘-1’ and sets an exception + on error, otherwise returns the number of copied characters. + + New in version 3.3. + + -- C Function: Py_ssize_t PyUnicode_Fill (PyObject *unicode, + Py_ssize_t start, Py_ssize_t length, Py_UCS4 fill_char) + + Fill a string with a character: write `fill_char' into + ‘unicode[start:start+length]’. + + Fail if `fill_char' is bigger than the string maximum character, or + if the string has more than 1 reference. + + Return the number of written character, or return ‘-1’ and raise an + exception on error. + + New in version 3.3. + + -- C Function: int PyUnicode_WriteChar (PyObject *unicode, + Py_ssize_t index, Py_UCS4 character) + + Write a character to a string. The string must have been created + through *note PyUnicode_New(): acd. Since Unicode strings are + supposed to be immutable, the string must not be shared, or have + been hashed yet. + + This function checks that `unicode' is a Unicode object, that the + index is not out of bounds, and that the object can be modified + safely (i.e. that it its reference count is one). + + New in version 3.3. + + -- C Function: Py_UCS4 PyUnicode_ReadChar (PyObject *unicode, + Py_ssize_t index) + + Read a character from a string. This function checks that + `unicode' is a Unicode object and the index is not out of bounds, + in contrast to the macro version *note PyUnicode_READ_CHAR(): ae4. + + New in version 3.3. + + -- C Function: PyObject* PyUnicode_Substring (PyObject *str, + Py_ssize_t start, Py_ssize_t end) + `Return value: New reference.' Return a substring of `str', from + character index `start' (included) to character index `end' + (excluded). Negative indices are not supported. + + New in version 3.3. + + -- C Function: Py_UCS4* PyUnicode_AsUCS4 (PyObject *u, Py_UCS4 *buffer, + Py_ssize_t buflen, int copy_null) + + Copy the string `u' into a UCS4 buffer, including a null character, + if `copy_null' is set. Returns ‘NULL’ and sets an exception on + error (in particular, a *note SystemError: 5fb. if `buflen' is + smaller than the length of `u'). `buffer' is returned on success. + + New in version 3.3. + + -- C Function: Py_UCS4* PyUnicode_AsUCS4Copy (PyObject *u) + + Copy the string `u' into a new UCS4 buffer that is allocated using + *note PyMem_Malloc(): 505. If this fails, ‘NULL’ is returned with + a *note MemoryError: 13af. set. The returned buffer always has an + extra null code point appended. + + New in version 3.3. + + ---------- Footnotes ---------- + + (1) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (2) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (3) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (4) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (5) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (6) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (7) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (8) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (9) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (10) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (11) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (12) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + (13) (1) For integer specifiers (d, u, ld, li, lu, lld, lli, llu, zd, +zi, zu, i, x): the 0-conversion flag has effect even when a precision is +given. + + +File: python.info, Node: Deprecated Py_UNICODE APIs, Next: Locale Encoding, Prev: Creating and accessing Unicode strings, Up: Unicode Objects + +7.8.3.11 Deprecated Py_UNICODE APIs +................................... + +Deprecated since version 3.3, will be removed in version 3.12. + +These API functions are deprecated with the implementation of PEP +393(1). Extension modules can continue using them, as they will not be +removed in Python 3.x, but need to be aware that their use can now cause +performance and memory hits. + + -- C Function: PyObject* PyUnicode_FromUnicode (const Py_UNICODE *u, + Py_ssize_t size) + `Return value: New reference.' Create a Unicode object from the + Py_UNICODE buffer `u' of the given size. `u' may be ‘NULL’ which + causes the contents to be undefined. It is the user’s + responsibility to fill in the needed data. The buffer is copied + into the new object. + + If the buffer is not ‘NULL’, the return value might be a shared + object. Therefore, modification of the resulting Unicode object is + only allowed when `u' is ‘NULL’. + + If the buffer is ‘NULL’, *note PyUnicode_READY(): ad6. must be + called once the string content has been filled before using any of + the access macros such as *note PyUnicode_KIND(): ade. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using *note + PyUnicode_FromKindAndData(): ad7, *note PyUnicode_FromWideChar(): + af0, or *note PyUnicode_New(): acd. + + -- C Function: Py_UNICODE* PyUnicode_AsUnicode (PyObject *unicode) + + Return a read-only pointer to the Unicode object’s internal *note + Py_UNICODE: aee. buffer, or ‘NULL’ on error. This will create the + *note Py_UNICODE*: aee. representation of the object if it is not + yet available. The buffer is always terminated with an extra null + code point. Note that the resulting *note Py_UNICODE: aee. string + may also contain embedded null code points, which would cause the + string to be truncated when used in most C functions. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using *note + PyUnicode_AsUCS4(): ad8, *note PyUnicode_AsWideChar(): 3b5c, *note + PyUnicode_ReadChar(): acf. or similar new APIs. + + Deprecated since version 3.3, will be removed in version 3.10. + + -- C Function: PyObject* PyUnicode_TransformDecimalToASCII + (Py_UNICODE *s, Py_ssize_t size) + `Return value: New reference.' Create a Unicode object by + replacing all decimal digits in *note Py_UNICODE: aee. buffer of + the given `size' by ASCII digits 0–9 according to their decimal + value. Return ‘NULL’ if an exception occurs. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note Py_UNICODE_TODECIMAL(): 3b50. + + -- C Function: Py_UNICODE* PyUnicode_AsUnicodeAndSize + (PyObject *unicode, Py_ssize_t *size) + + Like *note PyUnicode_AsUnicode(): af2, but also saves the *note + Py_UNICODE(): aee. array length (excluding the extra null + terminator) in `size'. Note that the resulting *note Py_UNICODE*: + aee. string may contain embedded null code points, which would + cause the string to be truncated when used in most C functions. + + New in version 3.3. + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using *note + PyUnicode_AsUCS4(): ad8, *note PyUnicode_AsWideChar(): 3b5c, *note + PyUnicode_ReadChar(): acf. or similar new APIs. + + -- C Function: Py_UNICODE* PyUnicode_AsUnicodeCopy (PyObject *unicode) + + Create a copy of a Unicode string ending with a null code point. + Return ‘NULL’ and raise a *note MemoryError: 13af. exception on + memory allocation failure, otherwise return a new allocated buffer + (use *note PyMem_Free(): da9. to free the buffer). Note that the + resulting *note Py_UNICODE*: aee. string may contain embedded null + code points, which would cause the string to be truncated when used + in most C functions. + + New in version 3.2. + + Please migrate to using *note PyUnicode_AsUCS4Copy(): ad9. or + similar new APIs. + + -- C Function: Py_ssize_t PyUnicode_GetSize (PyObject *unicode) + + Return the size of the deprecated *note Py_UNICODE: aee. + representation, in code units (this includes surrogate pairs as 2 + units). + + Deprecated since version 3.3, will be removed in version 3.12: Part + of the old-style Unicode API, please migrate to using *note + PyUnicode_GET_LENGTH(): acc. + + -- C Function: PyObject* PyUnicode_FromObject (PyObject *obj) + `Return value: New reference.' Copy an instance of a Unicode + subtype to a new true Unicode object if necessary. If `obj' is + already a true Unicode object (not a subtype), return the reference + with incremented refcount. + + Objects other than Unicode or its subtypes will cause a *note + TypeError: 192. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0393 + + +File: python.info, Node: Locale Encoding, Next: File System Encoding, Prev: Deprecated Py_UNICODE APIs, Up: Unicode Objects + +7.8.3.12 Locale Encoding +........................ + +The current locale encoding can be used to decode text from the +operating system. + + -- C Function: PyObject* PyUnicode_DecodeLocaleAndSize (const + char *str, Py_ssize_t len, const char *errors) + `Return value: New reference.' Decode a string from UTF-8 on + Android and VxWorks, or from the current locale encoding on other + platforms. The supported error handlers are ‘"strict"’ and + ‘"surrogateescape"’ ( PEP 383(1)). The decoder uses ‘"strict"’ + error handler if `errors' is ‘NULL’. `str' must end with a null + character but cannot contain embedded null characters. + + Use *note PyUnicode_DecodeFSDefaultAndSize(): 3990. to decode a + string from ‘Py_FileSystemDefaultEncoding’ (the locale encoding + read at Python startup). + + This function ignores the Python UTF-8 mode. + + See also + ........ + + The *note Py_DecodeLocale(): 43c. function. + + New in version 3.3. + + Changed in version 3.7: The function now also uses the current + locale encoding for the ‘surrogateescape’ error handler, except on + Android. Previously, *note Py_DecodeLocale(): 43c. was used for + the ‘surrogateescape’, and the current locale encoding was used for + ‘strict’. + + -- C Function: PyObject* PyUnicode_DecodeLocale (const char *str, const + char *errors) + `Return value: New reference.' Similar to *note + PyUnicode_DecodeLocaleAndSize(): 43e, but compute the string length + using ‘strlen()’. + + New in version 3.3. + + -- C Function: PyObject* PyUnicode_EncodeLocale (PyObject *unicode, + const char *errors) + `Return value: New reference.' Encode a Unicode object to UTF-8 on + Android and VxWorks, or to the current locale encoding on other + platforms. The supported error handlers are ‘"strict"’ and + ‘"surrogateescape"’ ( PEP 383(2)). The encoder uses ‘"strict"’ + error handler if `errors' is ‘NULL’. Return a *note bytes: 331. + object. `unicode' cannot contain embedded null characters. + + Use *note PyUnicode_EncodeFSDefault(): 3991. to encode a string to + ‘Py_FileSystemDefaultEncoding’ (the locale encoding read at Python + startup). + + This function ignores the Python UTF-8 mode. + + See also + ........ + + The *note Py_EncodeLocale(): 43d. function. + + New in version 3.3. + + Changed in version 3.7: The function now also uses the current + locale encoding for the ‘surrogateescape’ error handler, except on + Android. Previously, *note Py_EncodeLocale(): 43d. was used for + the ‘surrogateescape’, and the current locale encoding was used for + ‘strict’. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0383 + + (2) https://www.python.org/dev/peps/pep-0383 + + +File: python.info, Node: File System Encoding, Next: wchar_t Support, Prev: Locale Encoding, Up: Unicode Objects + +7.8.3.13 File System Encoding +............................. + +To encode and decode file names and other environment strings, +‘Py_FileSystemDefaultEncoding’ should be used as the encoding, and +‘Py_FileSystemDefaultEncodeErrors’ should be used as the error handler ( +PEP 383(1) and PEP 529(2)). To encode file names to *note bytes: 331. +during argument parsing, the ‘"O&"’ converter should be used, passing +*note PyUnicode_FSConverter(): 5ce. as the conversion function: + + -- C Function: int PyUnicode_FSConverter (PyObject* obj, void* result) + + ParseTuple converter: encode *note str: 330. objects – obtained + directly or through the *note os.PathLike: 4eb. interface – to + *note bytes: 331. using *note PyUnicode_EncodeFSDefault(): 3991.; + *note bytes: 331. objects are output as-is. `result' must be a + *note PyBytesObject*: d1e. which must be released when it is no + longer used. + + New in version 3.1. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + +To decode file names to *note str: 330. during argument parsing, the +‘"O&"’ converter should be used, passing *note PyUnicode_FSDecoder(): +5cf. as the conversion function: + + -- C Function: int PyUnicode_FSDecoder (PyObject* obj, void* result) + + ParseTuple converter: decode *note bytes: 331. objects – obtained + either directly or indirectly through the *note os.PathLike: 4eb. + interface – to *note str: 330. using *note + PyUnicode_DecodeFSDefaultAndSize(): 3990.; *note str: 330. objects + are output as-is. `result' must be a *note PyUnicodeObject*: 3b3c. + which must be released when it is no longer used. + + New in version 3.2. + + Changed in version 3.6: Accepts a *note path-like object: 36a. + + -- C Function: PyObject* PyUnicode_DecodeFSDefaultAndSize (const + char *s, Py_ssize_t size) + `Return value: New reference.' Decode a string using + ‘Py_FileSystemDefaultEncoding’ and the + ‘Py_FileSystemDefaultEncodeErrors’ error handler. + + If ‘Py_FileSystemDefaultEncoding’ is not set, fall back to the + locale encoding. + + ‘Py_FileSystemDefaultEncoding’ is initialized at startup from the + locale encoding and cannot be modified later. If you need to + decode a string from the current locale encoding, use *note + PyUnicode_DecodeLocaleAndSize(): 43e. + + See also + ........ + + The *note Py_DecodeLocale(): 43c. function. + + Changed in version 3.6: Use ‘Py_FileSystemDefaultEncodeErrors’ + error handler. + + -- C Function: PyObject* PyUnicode_DecodeFSDefault (const char *s) + `Return value: New reference.' Decode a null-terminated string + using ‘Py_FileSystemDefaultEncoding’ and the + ‘Py_FileSystemDefaultEncodeErrors’ error handler. + + If ‘Py_FileSystemDefaultEncoding’ is not set, fall back to the + locale encoding. + + Use *note PyUnicode_DecodeFSDefaultAndSize(): 3990. if you know the + string length. + + Changed in version 3.6: Use ‘Py_FileSystemDefaultEncodeErrors’ + error handler. + + -- C Function: PyObject* PyUnicode_EncodeFSDefault (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object to + ‘Py_FileSystemDefaultEncoding’ with the + ‘Py_FileSystemDefaultEncodeErrors’ error handler, and return *note + bytes: 331. Note that the resulting *note bytes: 331. object may + contain null bytes. + + If ‘Py_FileSystemDefaultEncoding’ is not set, fall back to the + locale encoding. + + ‘Py_FileSystemDefaultEncoding’ is initialized at startup from the + locale encoding and cannot be modified later. If you need to + encode a string to the current locale encoding, use *note + PyUnicode_EncodeLocale(): 43f. + + See also + ........ + + The *note Py_EncodeLocale(): 43d. function. + + New in version 3.2. + + Changed in version 3.6: Use ‘Py_FileSystemDefaultEncodeErrors’ + error handler. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0383 + + (2) https://www.python.org/dev/peps/pep-0529 + + +File: python.info, Node: wchar_t Support, Prev: File System Encoding, Up: Unicode Objects + +7.8.3.14 wchar_t Support +........................ + +‘wchar_t’ support for platforms which support it: + + -- C Function: PyObject* PyUnicode_FromWideChar (const wchar_t *w, + Py_ssize_t size) + `Return value: New reference.' Create a Unicode object from the + ‘wchar_t’ buffer `w' of the given `size'. Passing ‘-1’ as the + `size' indicates that the function must itself compute the length, + using wcslen. Return ‘NULL’ on failure. + + -- C Function: Py_ssize_t PyUnicode_AsWideChar (PyObject *unicode, + wchar_t *w, Py_ssize_t size) + + Copy the Unicode object contents into the ‘wchar_t’ buffer `w'. At + most `size' ‘wchar_t’ characters are copied (excluding a possibly + trailing null termination character). Return the number of + ‘wchar_t’ characters copied or ‘-1’ in case of an error. Note that + the resulting ‘wchar_t*’ string may or may not be null-terminated. + It is the responsibility of the caller to make sure that the + ‘wchar_t*’ string is null-terminated in case this is required by + the application. Also, note that the ‘wchar_t*’ string might + contain null characters, which would cause the string to be + truncated when used with most C functions. + + -- C Function: wchar_t* PyUnicode_AsWideCharString (PyObject *unicode, + Py_ssize_t *size) + + Convert the Unicode object to a wide character string. The output + string always ends with a null character. If `size' is not ‘NULL’, + write the number of wide characters (excluding the trailing null + termination character) into `*size'. Note that the resulting + ‘wchar_t’ string might contain null characters, which would cause + the string to be truncated when used with most C functions. If + `size' is ‘NULL’ and the ‘wchar_t*’ string contains null characters + a *note ValueError: 1fb. is raised. + + Returns a buffer allocated by ‘PyMem_Alloc()’ (use *note + PyMem_Free(): da9. to free it) on success. On error, returns + ‘NULL’ and `*size' is undefined. Raises a *note MemoryError: 13af. + if memory allocation is failed. + + New in version 3.2. + + Changed in version 3.7: Raises a *note ValueError: 1fb. if `size' + is ‘NULL’ and the ‘wchar_t*’ string contains null characters. + + +File: python.info, Node: Built-in Codecs, Next: Methods and Slot Functions, Prev: Unicode Objects, Up: Unicode Objects and Codecs + +7.8.3.15 Built-in Codecs +........................ + +Python provides a set of built-in codecs which are written in C for +speed. All of these codecs are directly usable via the following +functions. + +Many of the following APIs take two arguments encoding and errors, and +they have the same semantics as the ones of the built-in *note str(): +330. string object constructor. + +Setting encoding to ‘NULL’ causes the default encoding to be used which +is ASCII. The file system calls should use *note +PyUnicode_FSConverter(): 5ce. for encoding file names. This uses the +variable ‘Py_FileSystemDefaultEncoding’ internally. This variable +should be treated as read-only: on some systems, it will be a pointer to +a static string, on others, it will change at run-time (such as when the +application invokes setlocale). + +Error handling is set by errors which may also be set to ‘NULL’ meaning +to use the default handling defined for the codec. Default error +handling for all built-in codecs is “strict” (*note ValueError: 1fb. is +raised). + +The codecs all use a similar interface. Only deviation from the +following generic ones are documented for simplicity. + +* Menu: + +* Generic Codecs:: +* UTF-8 Codecs:: +* UTF-32 Codecs:: +* UTF-16 Codecs:: +* UTF-7 Codecs:: +* Unicode-Escape Codecs:: +* Raw-Unicode-Escape Codecs:: +* Latin-1 Codecs:: +* ASCII Codecs:: +* Character Map Codecs:: +* MBCS codecs for Windows:: +* Methods & Slots:: + + +File: python.info, Node: Generic Codecs, Next: UTF-8 Codecs, Up: Built-in Codecs + +7.8.3.16 Generic Codecs +....................... + +These are the generic codec APIs: + + -- C Function: PyObject* PyUnicode_Decode (const char *s, + Py_ssize_t size, const char *encoding, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the encoded string `s'. `encoding' and `errors' + have the same meaning as the parameters of the same name in the + *note str(): 330. built-in function. The codec to be used is + looked up using the Python codec registry. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_AsEncodedString (PyObject *unicode, + const char *encoding, const char *errors) + `Return value: New reference.' Encode a Unicode object and return + the result as Python bytes object. `encoding' and `errors' have + the same meaning as the parameters of the same name in the Unicode + *note encode(): c37. method. The codec to be used is looked up + using the Python codec registry. Return ‘NULL’ if an exception was + raised by the codec. + + -- C Function: PyObject* PyUnicode_Encode (const Py_UNICODE *s, + Py_ssize_t size, const char *encoding, const char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer `s' of the given `size' and return a Python bytes object. + `encoding' and `errors' have the same meaning as the parameters of + the same name in the Unicode *note encode(): c37. method. The + codec to be used is looked up using the Python codec registry. + Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: UTF-8 Codecs, Next: UTF-32 Codecs, Prev: Generic Codecs, Up: Built-in Codecs + +7.8.3.17 UTF-8 Codecs +..................... + +These are the UTF-8 codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeUTF8 (const char *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the UTF-8 encoded string `s'. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_DecodeUTF8Stateful (const char *s, + Py_ssize_t size, const char *errors, Py_ssize_t *consumed) + `Return value: New reference.' If `consumed' is ‘NULL’, behave + like *note PyUnicode_DecodeUTF8(): 3b68. If `consumed' is not + ‘NULL’, trailing incomplete UTF-8 byte sequences will not be + treated as an error. Those bytes will not be decoded and the + number of bytes that have been decoded will be stored in + `consumed'. + + -- C Function: PyObject* PyUnicode_AsUTF8String (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object using UTF-8 + and return the result as Python bytes object. Error handling is + “strict”. Return ‘NULL’ if an exception was raised by the codec. + + -- C Function: const char* PyUnicode_AsUTF8AndSize (PyObject *unicode, + Py_ssize_t *size) + + Return a pointer to the UTF-8 encoding of the Unicode object, and + store the size of the encoded representation (in bytes) in `size'. + The `size' argument can be ‘NULL’; in this case no size will be + stored. The returned buffer always has an extra null byte appended + (not included in `size'), regardless of whether there are any other + null code points. + + In the case of an error, ‘NULL’ is returned with an exception set + and no `size' is stored. + + This caches the UTF-8 representation of the string in the Unicode + object, and subsequent calls will return a pointer to the same + buffer. The caller is not responsible for deallocating the buffer. + + New in version 3.3. + + Changed in version 3.7: The return type is now ‘const char *’ + rather of ‘char *’. + + -- C Function: const char* PyUnicode_AsUTF8 (PyObject *unicode) + + As *note PyUnicode_AsUTF8AndSize(): 42a, but does not store the + size. + + New in version 3.3. + + Changed in version 3.7: The return type is now ‘const char *’ + rather of ‘char *’. + + -- C Function: PyObject* PyUnicode_EncodeUTF8 (const Py_UNICODE *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer `s' of the given `size' using UTF-8 and return a Python + bytes object. Return ‘NULL’ if an exception was raised by the + codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsUTF8String(): aff, *note + PyUnicode_AsUTF8AndSize(): 42a. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: UTF-32 Codecs, Next: UTF-16 Codecs, Prev: UTF-8 Codecs, Up: Built-in Codecs + +7.8.3.18 UTF-32 Codecs +...................... + +These are the UTF-32 codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeUTF32 (const char *s, + Py_ssize_t size, const char *errors, int *byteorder) + `Return value: New reference.' Decode `size' bytes from a UTF-32 + encoded buffer string and return the corresponding Unicode object. + `errors' (if non-‘NULL’) defines the error handling. It defaults + to “strict”. + + If `byteorder' is non-‘NULL’, the decoder starts decoding using the + given byte order: + + *byteorder == -1: little endian + *byteorder == 0: native order + *byteorder == 1: big endian + + If ‘*byteorder’ is zero, and the first four bytes of the input data + are a byte order mark (BOM), the decoder switches to this byte + order and the BOM is not copied into the resulting Unicode string. + If ‘*byteorder’ is ‘-1’ or ‘1’, any byte order mark is copied to + the output. + + After completion, `*byteorder' is set to the current byte order at + the end of input data. + + If `byteorder' is ‘NULL’, the codec starts in native order mode. + + Return ‘NULL’ if an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_DecodeUTF32Stateful (const char *s, + Py_ssize_t size, const char *errors, int *byteorder, + Py_ssize_t *consumed) + `Return value: New reference.' If `consumed' is ‘NULL’, behave + like *note PyUnicode_DecodeUTF32(): 3b6b. If `consumed' is not + ‘NULL’, *note PyUnicode_DecodeUTF32Stateful(): 3b6c. will not treat + trailing incomplete UTF-32 byte sequences (such as a number of + bytes not divisible by four) as an error. Those bytes will not be + decoded and the number of bytes that have been decoded will be + stored in `consumed'. + + -- C Function: PyObject* PyUnicode_AsUTF32String (PyObject *unicode) + `Return value: New reference.' Return a Python byte string using + the UTF-32 encoding in native byte order. The string always starts + with a BOM mark. Error handling is “strict”. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_EncodeUTF32 (const Py_UNICODE *s, + Py_ssize_t size, const char *errors, int byteorder) + `Return value: New reference.' Return a Python bytes object + holding the UTF-32 encoded value of the Unicode data in `s'. + Output is written according to the following byte order: + + byteorder == -1: little endian + byteorder == 0: native byte order (writes a BOM mark) + byteorder == 1: big endian + + If byteorder is ‘0’, the output string will always start with the + Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is + prepended. + + If ‘Py_UNICODE_WIDE’ is not defined, surrogate pairs will be output + as a single code point. + + Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsUTF32String(): 3b6d. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: UTF-16 Codecs, Next: UTF-7 Codecs, Prev: UTF-32 Codecs, Up: Built-in Codecs + +7.8.3.19 UTF-16 Codecs +...................... + +These are the UTF-16 codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeUTF16 (const char *s, + Py_ssize_t size, const char *errors, int *byteorder) + `Return value: New reference.' Decode `size' bytes from a UTF-16 + encoded buffer string and return the corresponding Unicode object. + `errors' (if non-‘NULL’) defines the error handling. It defaults + to “strict”. + + If `byteorder' is non-‘NULL’, the decoder starts decoding using the + given byte order: + + *byteorder == -1: little endian + *byteorder == 0: native order + *byteorder == 1: big endian + + If ‘*byteorder’ is zero, and the first two bytes of the input data + are a byte order mark (BOM), the decoder switches to this byte + order and the BOM is not copied into the resulting Unicode string. + If ‘*byteorder’ is ‘-1’ or ‘1’, any byte order mark is copied to + the output (where it will result in either a ‘\ufeff’ or a ‘\ufffe’ + character). + + After completion, `*byteorder' is set to the current byte order at + the end of input data. + + If `byteorder' is ‘NULL’, the codec starts in native order mode. + + Return ‘NULL’ if an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_DecodeUTF16Stateful (const char *s, + Py_ssize_t size, const char *errors, int *byteorder, + Py_ssize_t *consumed) + `Return value: New reference.' If `consumed' is ‘NULL’, behave + like *note PyUnicode_DecodeUTF16(): 3b6f. If `consumed' is not + ‘NULL’, *note PyUnicode_DecodeUTF16Stateful(): 3b70. will not treat + trailing incomplete UTF-16 byte sequences (such as an odd number of + bytes or a split surrogate pair) as an error. Those bytes will not + be decoded and the number of bytes that have been decoded will be + stored in `consumed'. + + -- C Function: PyObject* PyUnicode_AsUTF16String (PyObject *unicode) + `Return value: New reference.' Return a Python byte string using + the UTF-16 encoding in native byte order. The string always starts + with a BOM mark. Error handling is “strict”. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_EncodeUTF16 (const Py_UNICODE *s, + Py_ssize_t size, const char *errors, int byteorder) + `Return value: New reference.' Return a Python bytes object + holding the UTF-16 encoded value of the Unicode data in `s'. + Output is written according to the following byte order: + + byteorder == -1: little endian + byteorder == 0: native byte order (writes a BOM mark) + byteorder == 1: big endian + + If byteorder is ‘0’, the output string will always start with the + Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is + prepended. + + If ‘Py_UNICODE_WIDE’ is defined, a single *note Py_UNICODE: aee. + value may get represented as a surrogate pair. If it is not + defined, each *note Py_UNICODE: aee. values is interpreted as a + UCS-2 character. + + Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsUTF16String(): 3b71. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: UTF-7 Codecs, Next: Unicode-Escape Codecs, Prev: UTF-16 Codecs, Up: Built-in Codecs + +7.8.3.20 UTF-7 Codecs +..................... + +These are the UTF-7 codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeUTF7 (const char *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the UTF-7 encoded string `s'. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_DecodeUTF7Stateful (const char *s, + Py_ssize_t size, const char *errors, Py_ssize_t *consumed) + `Return value: New reference.' If `consumed' is ‘NULL’, behave + like *note PyUnicode_DecodeUTF7(): 3b73. If `consumed' is not + ‘NULL’, trailing incomplete UTF-7 base-64 sections will not be + treated as an error. Those bytes will not be decoded and the + number of bytes that have been decoded will be stored in + `consumed'. + + -- C Function: PyObject* PyUnicode_EncodeUTF7 (const Py_UNICODE *s, + Py_ssize_t size, int base64SetO, int base64WhiteSpace, const + char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given size using UTF-7 and return a Python bytes + object. Return ‘NULL’ if an exception was raised by the codec. + + If `base64SetO' is nonzero, “Set O” (punctuation that has no + otherwise special meaning) will be encoded in base-64. If + `base64WhiteSpace' is nonzero, whitespace will be encoded in + base-64. Both are set to zero for the Python “utf-7” codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: Unicode-Escape Codecs, Next: Raw-Unicode-Escape Codecs, Prev: UTF-7 Codecs, Up: Built-in Codecs + +7.8.3.21 Unicode-Escape Codecs +.............................. + +These are the “Unicode Escape” codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeUnicodeEscape (const char *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the Unicode-Escape encoded string `s'. Return + ‘NULL’ if an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_AsUnicodeEscapeString + (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object using + Unicode-Escape and return the result as a bytes object. Error + handling is “strict”. Return ‘NULL’ if an exception was raised by + the codec. + + -- C Function: PyObject* PyUnicode_EncodeUnicodeEscape (const + Py_UNICODE *s, Py_ssize_t size) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given `size' using Unicode-Escape and return a bytes + object. Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsUnicodeEscapeString(): b03. + + +File: python.info, Node: Raw-Unicode-Escape Codecs, Next: Latin-1 Codecs, Prev: Unicode-Escape Codecs, Up: Built-in Codecs + +7.8.3.22 Raw-Unicode-Escape Codecs +.................................. + +These are the “Raw Unicode Escape” codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeRawUnicodeEscape (const + char *s, Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the Raw-Unicode-Escape encoded string `s'. Return + ‘NULL’ if an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_AsRawUnicodeEscapeString + (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object using + Raw-Unicode-Escape and return the result as a bytes object. Error + handling is “strict”. Return ‘NULL’ if an exception was raised by + the codec. + + -- C Function: PyObject* PyUnicode_EncodeRawUnicodeEscape (const + Py_UNICODE *s, Py_ssize_t size) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given `size' using Raw-Unicode-Escape and return a + bytes object. Return ‘NULL’ if an exception was raised by the + codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsRawUnicodeEscapeString(): b05. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: Latin-1 Codecs, Next: ASCII Codecs, Prev: Raw-Unicode-Escape Codecs, Up: Built-in Codecs + +7.8.3.23 Latin-1 Codecs +....................... + +These are the Latin-1 codec APIs: Latin-1 corresponds to the first 256 +Unicode ordinals and only these are accepted by the codecs during +encoding. + + -- C Function: PyObject* PyUnicode_DecodeLatin1 (const char *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the Latin-1 encoded string `s'. Return ‘NULL’ if + an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_AsLatin1String (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object using + Latin-1 and return the result as Python bytes object. Error + handling is “strict”. Return ‘NULL’ if an exception was raised by + the codec. + + -- C Function: PyObject* PyUnicode_EncodeLatin1 (const Py_UNICODE *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given `size' using Latin-1 and return a Python bytes + object. Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsLatin1String(): b07. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: ASCII Codecs, Next: Character Map Codecs, Prev: Latin-1 Codecs, Up: Built-in Codecs + +7.8.3.24 ASCII Codecs +..................... + +These are the ASCII codec APIs. Only 7-bit ASCII data is accepted. All +other codes generate errors. + + -- C Function: PyObject* PyUnicode_DecodeASCII (const char *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the ASCII encoded string `s'. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_AsASCIIString (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object using ASCII + and return the result as Python bytes object. Error handling is + “strict”. Return ‘NULL’ if an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_EncodeASCII (const Py_UNICODE *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given `size' using ASCII and return a Python bytes + object. Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsASCIIString(): b09. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: Character Map Codecs, Next: MBCS codecs for Windows, Prev: ASCII Codecs, Up: Built-in Codecs + +7.8.3.25 Character Map Codecs +............................. + +This codec is special in that it can be used to implement many different +codecs (and this is in fact what was done to obtain most of the standard +codecs included in the ‘encodings’ package). The codec uses mapping to +encode and decode characters. The mapping objects provided must support +the *note __getitem__(): 27c. mapping interface; dictionaries and +sequences work well. + +These are the mapping codec APIs: + + -- C Function: PyObject* PyUnicode_DecodeCharmap (const char *data, + Py_ssize_t size, PyObject *mapping, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the encoded string `s' using the given `mapping' + object. Return ‘NULL’ if an exception was raised by the codec. + + If `mapping' is ‘NULL’, Latin-1 decoding will be applied. Else + `mapping' must map bytes ordinals (integers in the range from 0 to + 255) to Unicode strings, integers (which are then interpreted as + Unicode ordinals) or ‘None’. Unmapped data bytes – ones which + cause a *note LookupError: 1308, as well as ones which get mapped + to ‘None’, ‘0xFFFE’ or ‘'\ufffe'’, are treated as undefined + mappings and cause an error. + + -- C Function: PyObject* PyUnicode_AsCharmapString (PyObject *unicode, + PyObject *mapping) + `Return value: New reference.' Encode a Unicode object using the + given `mapping' object and return the result as a bytes object. + Error handling is “strict”. Return ‘NULL’ if an exception was + raised by the codec. + + The `mapping' object must map Unicode ordinal integers to bytes + objects, integers in the range from 0 to 255 or ‘None’. Unmapped + character ordinals (ones which cause a *note LookupError: 1308.) as + well as mapped to ‘None’ are treated as “undefined mapping” and + cause an error. + + -- C Function: PyObject* PyUnicode_EncodeCharmap (const Py_UNICODE *s, + Py_ssize_t size, PyObject *mapping, const char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given `size' using the given `mapping' object and + return the result as a bytes object. Return ‘NULL’ if an exception + was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsCharmapString(): 3b7f. or *note + PyUnicode_AsEncodedString(): 3b66. + +The following codec API is special in that maps Unicode to Unicode. + + -- C Function: PyObject* PyUnicode_Translate (PyObject *str, + PyObject *table, const char *errors) + `Return value: New reference.' Translate a string by applying a + character mapping table to it and return the resulting Unicode + object. Return ‘NULL’ if an exception was raised by the codec. + + The mapping table must map Unicode ordinal integers to Unicode + ordinal integers or ‘None’ (causing deletion of the character). + + Mapping tables need only provide the *note __getitem__(): 27c. + interface; dictionaries and sequences work well. Unmapped + character ordinals (ones which cause a *note LookupError: 1308.) + are left untouched and are copied as-is. + + `errors' has the usual meaning for codecs. It may be ‘NULL’ which + indicates to use the default error handling. + + -- C Function: PyObject* PyUnicode_TranslateCharmap (const + Py_UNICODE *s, Py_ssize_t size, PyObject *mapping, const + char *errors) + `Return value: New reference.' Translate a *note Py_UNICODE: aee. + buffer of the given `size' by applying a character `mapping' table + to it and return the resulting Unicode object. Return ‘NULL’ when + an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 3.11: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_Translate(): 3b80. or *note generic codec + based API: 5ee. + + +File: python.info, Node: MBCS codecs for Windows, Next: Methods & Slots, Prev: Character Map Codecs, Up: Built-in Codecs + +7.8.3.26 MBCS codecs for Windows +................................ + +These are the MBCS codec APIs. They are currently only available on +Windows and use the Win32 MBCS converters to implement the conversions. +Note that MBCS (or DBCS) is a class of encodings, not just one. The +target encoding is defined by the user settings on the machine running +the codec. + + -- C Function: PyObject* PyUnicode_DecodeMBCS (const char *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Create a Unicode object by decoding + `size' bytes of the MBCS encoded string `s'. Return ‘NULL’ if an + exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_DecodeMBCSStateful (const char *s, + Py_ssize_t size, const char *errors, Py_ssize_t *consumed) + `Return value: New reference.' If `consumed' is ‘NULL’, behave + like *note PyUnicode_DecodeMBCS(): 3b82. If `consumed' is not + ‘NULL’, *note PyUnicode_DecodeMBCSStateful(): 3b83. will not decode + trailing lead byte and the number of bytes that have been decoded + will be stored in `consumed'. + + -- C Function: PyObject* PyUnicode_AsMBCSString (PyObject *unicode) + `Return value: New reference.' Encode a Unicode object using MBCS + and return the result as Python bytes object. Error handling is + “strict”. Return ‘NULL’ if an exception was raised by the codec. + + -- C Function: PyObject* PyUnicode_EncodeCodePage (int code_page, + PyObject *unicode, const char *errors) + `Return value: New reference.' Encode the Unicode object using the + specified code page and return a Python bytes object. Return + ‘NULL’ if an exception was raised by the codec. Use ‘CP_ACP’ code + page to get the MBCS encoder. + + New in version 3.3. + + -- C Function: PyObject* PyUnicode_EncodeMBCS (const Py_UNICODE *s, + Py_ssize_t size, const char *errors) + `Return value: New reference.' Encode the *note Py_UNICODE: aee. + buffer of the given `size' using MBCS and return a Python bytes + object. Return ‘NULL’ if an exception was raised by the codec. + + Deprecated since version 3.3, will be removed in version 4.0: Part + of the old-style *note Py_UNICODE: aee. API; please migrate to + using *note PyUnicode_AsMBCSString(): b0d, *note + PyUnicode_EncodeCodePage(): b0e. or *note + PyUnicode_AsEncodedString(): 3b66. + + +File: python.info, Node: Methods & Slots, Prev: MBCS codecs for Windows, Up: Built-in Codecs + +7.8.3.27 Methods & Slots +........................ + + +File: python.info, Node: Methods and Slot Functions, Prev: Built-in Codecs, Up: Unicode Objects and Codecs + +7.8.3.28 Methods and Slot Functions +................................... + +The following APIs are capable of handling Unicode objects and strings +on input (we refer to them as strings in the descriptions) and return +Unicode objects or integers as appropriate. + +They all return ‘NULL’ or ‘-1’ if an exception occurs. + + -- C Function: PyObject* PyUnicode_Concat (PyObject *left, + PyObject *right) + `Return value: New reference.' Concat two strings giving a new + Unicode string. + + -- C Function: PyObject* PyUnicode_Split (PyObject *s, PyObject *sep, + Py_ssize_t maxsplit) + `Return value: New reference.' Split a string giving a list of + Unicode strings. If `sep' is ‘NULL’, splitting will be done at all + whitespace substrings. Otherwise, splits occur at the given + separator. At most `maxsplit' splits will be done. If negative, + no limit is set. Separators are not included in the resulting + list. + + -- C Function: PyObject* PyUnicode_Splitlines (PyObject *s, + int keepend) + `Return value: New reference.' Split a Unicode string at line + breaks, returning a list of Unicode strings. CRLF is considered to + be one line break. If `keepend' is ‘0’, the Line break characters + are not included in the resulting strings. + + -- C Function: PyObject* PyUnicode_Join (PyObject *separator, + PyObject *seq) + `Return value: New reference.' Join a sequence of strings using + the given `separator' and return the resulting Unicode string. + + -- C Function: Py_ssize_t PyUnicode_Tailmatch (PyObject *str, + PyObject *substr, Py_ssize_t start, Py_ssize_t end, + int direction) + + Return ‘1’ if `substr' matches ‘str[start:end]’ at the given tail + end (`direction' == ‘-1’ means to do a prefix match, `direction' == + ‘1’ a suffix match), ‘0’ otherwise. Return ‘-1’ if an error + occurred. + + -- C Function: Py_ssize_t PyUnicode_Find (PyObject *str, + PyObject *substr, Py_ssize_t start, Py_ssize_t end, + int direction) + + Return the first position of `substr' in ‘str[start:end]’ using the + given `direction' (`direction' == ‘1’ means to do a forward search, + `direction' == ‘-1’ a backward search). The return value is the + index of the first match; a value of ‘-1’ indicates that no match + was found, and ‘-2’ indicates that an error occurred and an + exception has been set. + + -- C Function: Py_ssize_t PyUnicode_FindChar (PyObject *str, + Py_UCS4 ch, Py_ssize_t start, Py_ssize_t end, int direction) + + Return the first position of the character `ch' in ‘str[start:end]’ + using the given `direction' (`direction' == ‘1’ means to do a + forward search, `direction' == ‘-1’ a backward search). The return + value is the index of the first match; a value of ‘-1’ indicates + that no match was found, and ‘-2’ indicates that an error occurred + and an exception has been set. + + New in version 3.3. + + Changed in version 3.7: `start' and `end' are now adjusted to + behave like ‘str[start:end]’. + + -- C Function: Py_ssize_t PyUnicode_Count (PyObject *str, + PyObject *substr, Py_ssize_t start, Py_ssize_t end) + + Return the number of non-overlapping occurrences of `substr' in + ‘str[start:end]’. Return ‘-1’ if an error occurred. + + -- C Function: PyObject* PyUnicode_Replace (PyObject *str, + PyObject *substr, PyObject *replstr, Py_ssize_t maxcount) + `Return value: New reference.' Replace at most `maxcount' + occurrences of `substr' in `str' with `replstr' and return the + resulting Unicode object. `maxcount' == ‘-1’ means replace all + occurrences. + + -- C Function: int PyUnicode_Compare (PyObject *left, PyObject *right) + + Compare two strings and return ‘-1’, ‘0’, ‘1’ for less than, equal, + and greater than, respectively. + + This function returns ‘-1’ upon failure, so one should call *note + PyErr_Occurred(): 383f. to check for errors. + + -- C Function: int PyUnicode_CompareWithASCIIString (PyObject *uni, + const char *string) + + Compare a Unicode object, `uni', with `string' and return ‘-1’, + ‘0’, ‘1’ for less than, equal, and greater than, respectively. It + is best to pass only ASCII-encoded strings, but the function + interprets the input string as ISO-8859-1 if it contains non-ASCII + characters. + + This function does not raise exceptions. + + -- C Function: PyObject* PyUnicode_RichCompare (PyObject *left, + PyObject *right, int op) + `Return value: New reference.' Rich compare two Unicode strings + and return one of the following: + + * ‘NULL’ in case an exception was raised + + * ‘Py_True’ or ‘Py_False’ for successful comparisons + + * ‘Py_NotImplemented’ in case the type combination is unknown + + Possible values for `op' are ‘Py_GT’, ‘Py_GE’, ‘Py_EQ’, ‘Py_NE’, + ‘Py_LT’, and ‘Py_LE’. + + -- C Function: PyObject* PyUnicode_Format (PyObject *format, + PyObject *args) + `Return value: New reference.' Return a new string object from + `format' and `args'; this is analogous to ‘format % args’. + + -- C Function: int PyUnicode_Contains (PyObject *container, + PyObject *element) + + Check whether `element' is contained in `container' and return true + or false accordingly. + + `element' has to coerce to a one element Unicode string. ‘-1’ is + returned if there was an error. + + -- C Function: void PyUnicode_InternInPlace (PyObject **string) + + Intern the argument `*string' in place. The argument must be the + address of a pointer variable pointing to a Python Unicode string + object. If there is an existing interned string that is the same + as `*string', it sets `*string' to it (decrementing the reference + count of the old string object and incrementing the reference count + of the interned string object), otherwise it leaves `*string' alone + and interns it (incrementing its reference count). (Clarification: + even though there is a lot of talk about reference counts, think of + this function as reference-count-neutral; you own the object after + the call if and only if you owned it before the call.) + + -- C Function: PyObject* PyUnicode_InternFromString (const char *v) + `Return value: New reference.' A combination of *note + PyUnicode_FromString(): 38d1. and *note PyUnicode_InternInPlace(): + 3b8f, returning either a new Unicode string object that has been + interned, or a new (“owned”) reference to an earlier interned + string object with the same value. + + +File: python.info, Node: Tuple Objects, Next: Struct Sequence Objects, Prev: Unicode Objects and Codecs, Up: Sequence Objects + +7.8.3.29 Tuple Objects +...................... + + -- C Type: PyTupleObject + + This subtype of *note PyObject: 4ba. represents a Python tuple + object. + + -- C Variable: PyTypeObject PyTuple_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + tuple type; it is the same object as *note tuple: 47e. in the + Python layer. + + -- C Function: int PyTuple_Check (PyObject *p) + + Return true if `p' is a tuple object or an instance of a subtype of + the tuple type. + + -- C Function: int PyTuple_CheckExact (PyObject *p) + + Return true if `p' is a tuple object, but not an instance of a + subtype of the tuple type. + + -- C Function: PyObject* PyTuple_New (Py_ssize_t len) + `Return value: New reference.' Return a new tuple object of size + `len', or ‘NULL’ on failure. + + -- C Function: PyObject* PyTuple_Pack (Py_ssize_t n, ...) + `Return value: New reference.' Return a new tuple object of size + `n', or ‘NULL’ on failure. The tuple values are initialized to the + subsequent `n' C arguments pointing to Python objects. + ‘PyTuple_Pack(2, a, b)’ is equivalent to ‘Py_BuildValue("(OO)", a, + b)’. + + -- C Function: Py_ssize_t PyTuple_Size (PyObject *p) + + Take a pointer to a tuple object, and return the size of that + tuple. + + -- C Function: Py_ssize_t PyTuple_GET_SIZE (PyObject *p) + + Return the size of the tuple `p', which must be non-‘NULL’ and + point to a tuple; no error checking is performed. + + -- C Function: PyObject* PyTuple_GetItem (PyObject *p, Py_ssize_t pos) + `Return value: Borrowed reference.' Return the object at position + `pos' in the tuple pointed to by `p'. If `pos' is out of bounds, + return ‘NULL’ and set an *note IndexError: e75. exception. + + -- C Function: PyObject* PyTuple_GET_ITEM (PyObject *p, Py_ssize_t pos) + `Return value: Borrowed reference.' Like *note PyTuple_GetItem(): + 385c, but does no checking of its arguments. + + -- C Function: PyObject* PyTuple_GetSlice (PyObject *p, Py_ssize_t low, + Py_ssize_t high) + `Return value: New reference.' Return the slice of the tuple + pointed to by `p' between `low' and `high', or ‘NULL’ on failure. + This is the equivalent of the Python expression ‘p[low:high]’. + Indexing from the end of the list is not supported. + + -- C Function: int PyTuple_SetItem (PyObject *p, Py_ssize_t pos, + PyObject *o) + + Insert a reference to object `o' at position `pos' of the tuple + pointed to by `p'. Return ‘0’ on success. If `pos' is out of + bounds, return ‘-1’ and set an *note IndexError: e75. exception. + + Note: This function “steals” a reference to `o' and discards a + reference to an item already in the tuple at the affected + position. + + -- C Function: void PyTuple_SET_ITEM (PyObject *p, Py_ssize_t pos, + PyObject *o) + + Like *note PyTuple_SetItem(): 3861, but does no error checking, and + should `only' be used to fill in brand new tuples. + + Note: This macro “steals” a reference to `o', and, unlike + *note PyTuple_SetItem(): 3861, does `not' discard a reference + to any item that is being replaced; any reference in the tuple + at position `pos' will be leaked. + + -- C Function: int _PyTuple_Resize (PyObject **p, Py_ssize_t newsize) + + Can be used to resize a tuple. `newsize' will be the new length of + the tuple. Because tuples are `supposed' to be immutable, this + should only be used if there is only one reference to the object. + Do `not' use this if the tuple may already be known to some other + part of the code. The tuple will always grow or shrink at the end. + Think of this as destroying the old tuple and creating a new one, + only more efficiently. Returns ‘0’ on success. Client code should + never assume that the resulting value of ‘*p’ will be the same as + before calling this function. If the object referenced by ‘*p’ is + replaced, the original ‘*p’ is destroyed. On failure, returns ‘-1’ + and sets ‘*p’ to ‘NULL’, and raises *note MemoryError: 13af. or + *note SystemError: 5fb. + + -- C Function: int PyTuple_ClearFreeList () + + Clear the free list. Return the total number of freed items. + + +File: python.info, Node: Struct Sequence Objects, Next: List Objects, Prev: Tuple Objects, Up: Sequence Objects + +7.8.3.30 Struct Sequence Objects +................................ + +Struct sequence objects are the C equivalent of *note namedtuple(): 1b7. +objects, i.e. a sequence whose items can also be accessed through +attributes. To create a struct sequence, you first have to create a +specific struct sequence type. + + -- C Function: PyTypeObject* PyStructSequence_NewType + (PyStructSequence_Desc *desc) + `Return value: New reference.' Create a new struct sequence type + from the data in `desc', described below. Instances of the + resulting type can be created with *note PyStructSequence_New(): + 3ba1. + + -- C Function: void PyStructSequence_InitType (PyTypeObject *type, + PyStructSequence_Desc *desc) + + Initializes a struct sequence type `type' from `desc' in place. + + -- C Function: int PyStructSequence_InitType2 (PyTypeObject *type, + PyStructSequence_Desc *desc) + + The same as ‘PyStructSequence_InitType’, but returns ‘0’ on success + and ‘-1’ on failure. + + New in version 3.4. + + -- C Type: PyStructSequence_Desc + + Contains the meta information of a struct sequence type to create. + + Field C Type Meaning + + ------------------------------------------------------------------------------------------------------ + + ‘name’ ‘const char *’ name of the struct sequence type + + + ‘doc’ ‘const char *’ pointer to docstring for the type or + ‘NULL’ to omit + + + ‘fields’ ‘PyStructSequence_Field *’ pointer to ‘NULL’-terminated array with + field names of the new type + + + ‘n_in_sequence’ ‘int’ number of fields visible to the Python + side (if used as tuple) + + + -- C Type: PyStructSequence_Field + + Describes a field of a struct sequence. As a struct sequence is + modeled as a tuple, all fields are typed as *note PyObject*: 4ba. + The index in the ‘fields’ array of the *note PyStructSequence_Desc: + 429. determines which field of the struct sequence is described. + + Field C Type Meaning + + ------------------------------------------------------------------------------------- + + ‘name’ ‘const char *’ name for the field or ‘NULL’ to end the + list of named fields, set to + *note PyStructSequence_UnnamedField: 3ba2. + to leave unnamed + + + ‘doc’ ‘const char *’ field docstring or ‘NULL’ to omit + + + -- C Variable: char* PyStructSequence_UnnamedField + + Special value for a field name to leave it unnamed. + + -- C Function: PyObject* PyStructSequence_New (PyTypeObject *type) + `Return value: New reference.' Creates an instance of `type', + which must have been created with *note PyStructSequence_NewType(): + 3ba0. + + -- C Function: PyObject* PyStructSequence_GetItem (PyObject *p, + Py_ssize_t pos) + `Return value: Borrowed reference.' Return the object at position + `pos' in the struct sequence pointed to by `p'. No bounds checking + is performed. + + -- C Function: PyObject* PyStructSequence_GET_ITEM (PyObject *p, + Py_ssize_t pos) + `Return value: Borrowed reference.' Macro equivalent of *note + PyStructSequence_GetItem(): 3ba3. + + -- C Function: void PyStructSequence_SetItem (PyObject *p, + Py_ssize_t pos, PyObject *o) + + Sets the field at index `pos' of the struct sequence `p' to value + `o'. Like *note PyTuple_SET_ITEM(): 3b9d, this should only be used + to fill in brand new instances. + + Note: This function “steals” a reference to `o'. + + -- C Function: void PyStructSequence_SET_ITEM (PyObject *p, + Py_ssize_t *pos, PyObject *o) + + Macro equivalent of *note PyStructSequence_SetItem(): 3ba5. + + Note: This function “steals” a reference to `o'. + + +File: python.info, Node: List Objects, Prev: Struct Sequence Objects, Up: Sequence Objects + +7.8.3.31 List Objects +..................... + + -- C Type: PyListObject + + This subtype of *note PyObject: 4ba. represents a Python list + object. + + -- C Variable: PyTypeObject PyList_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + list type. This is the same object as *note list: 262. in the + Python layer. + + -- C Function: int PyList_Check (PyObject *p) + + Return true if `p' is a list object or an instance of a subtype of + the list type. + + -- C Function: int PyList_CheckExact (PyObject *p) + + Return true if `p' is a list object, but not an instance of a + subtype of the list type. + + -- C Function: PyObject* PyList_New (Py_ssize_t len) + `Return value: New reference.' Return a new list of length `len' + on success, or ‘NULL’ on failure. + + Note: If `len' is greater than zero, the returned list + object’s items are set to ‘NULL’. Thus you cannot use + abstract API functions such as *note PySequence_SetItem(): + 38f2. or expose the object to Python code before setting all + items to a real object with *note PyList_SetItem(): 3862. + + -- C Function: Py_ssize_t PyList_Size (PyObject *list) + + Return the length of the list object in `list'; this is equivalent + to ‘len(list)’ on a list object. + + -- C Function: Py_ssize_t PyList_GET_SIZE (PyObject *list) + + Macro form of *note PyList_Size(): d7e. without error checking. + + -- C Function: PyObject* PyList_GetItem (PyObject *list, + Py_ssize_t index) + `Return value: Borrowed reference.' Return the object at position + `index' in the list pointed to by `list'. The position must be + non-negative; indexing from the end of the list is not supported. + If `index' is out of bounds (<0 or >=len(list)), return ‘NULL’ and + set an *note IndexError: e75. exception. + + -- C Function: PyObject* PyList_GET_ITEM (PyObject *list, Py_ssize_t i) + `Return value: Borrowed reference.' Macro form of *note + PyList_GetItem(): 385d. without error checking. + + -- C Function: int PyList_SetItem (PyObject *list, Py_ssize_t index, + PyObject *item) + + Set the item at index `index' in list to `item'. Return ‘0’ on + success. If `index' is out of bounds, return ‘-1’ and set an *note + IndexError: e75. exception. + + Note: This function “steals” a reference to `item' and + discards a reference to an item already in the list at the + affected position. + + -- C Function: void PyList_SET_ITEM (PyObject *list, Py_ssize_t i, + PyObject *o) + + Macro form of *note PyList_SetItem(): 3862. without error checking. + This is normally only used to fill in new lists where there is no + previous content. + + Note: This macro “steals” a reference to `item', and, unlike + *note PyList_SetItem(): 3862, does `not' discard a reference + to any item that is being replaced; any reference in `list' at + position `i' will be leaked. + + -- C Function: int PyList_Insert (PyObject *list, Py_ssize_t index, + PyObject *item) + + Insert the item `item' into list `list' in front of index `index'. + Return ‘0’ if successful; return ‘-1’ and set an exception if + unsuccessful. Analogous to ‘list.insert(index, item)’. + + -- C Function: int PyList_Append (PyObject *list, PyObject *item) + + Append the object `item' at the end of list `list'. Return ‘0’ if + successful; return ‘-1’ and set an exception if unsuccessful. + Analogous to ‘list.append(item)’. + + -- C Function: PyObject* PyList_GetSlice (PyObject *list, + Py_ssize_t low, Py_ssize_t high) + `Return value: New reference.' Return a list of the objects in + `list' containing the objects `between' `low' and `high'. Return + ‘NULL’ and set an exception if unsuccessful. Analogous to + ‘list[low:high]’. Indexing from the end of the list is not + supported. + + -- C Function: int PyList_SetSlice (PyObject *list, Py_ssize_t low, + Py_ssize_t high, PyObject *itemlist) + + Set the slice of `list' between `low' and `high' to the contents of + `itemlist'. Analogous to ‘list[low:high] = itemlist’. The + `itemlist' may be ‘NULL’, indicating the assignment of an empty + list (slice deletion). Return ‘0’ on success, ‘-1’ on failure. + Indexing from the end of the list is not supported. + + -- C Function: int PyList_Sort (PyObject *list) + + Sort the items of `list' in place. Return ‘0’ on success, ‘-1’ on + failure. This is equivalent to ‘list.sort()’. + + -- C Function: int PyList_Reverse (PyObject *list) + + Reverse the items of `list' in place. Return ‘0’ on success, ‘-1’ + on failure. This is the equivalent of ‘list.reverse()’. + + -- C Function: PyObject* PyList_AsTuple (PyObject *list) + `Return value: New reference.' + + Return a new tuple object containing the contents of `list'; + equivalent to ‘tuple(list)’. + + -- C Function: int PyList_ClearFreeList () + + Clear the free list. Return the total number of freed items. + + New in version 3.3. + + +File: python.info, Node: Container Objects, Next: Function Objects<2>, Prev: Sequence Objects, Up: Concrete Objects Layer + +7.8.4 Container Objects +----------------------- + +* Menu: + +* Dictionary Objects:: +* Set Objects:: + + +File: python.info, Node: Dictionary Objects, Next: Set Objects, Up: Container Objects + +7.8.4.1 Dictionary Objects +.......................... + + -- C Type: PyDictObject + + This subtype of *note PyObject: 4ba. represents a Python dictionary + object. + + -- C Variable: PyTypeObject PyDict_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + dictionary type. This is the same object as *note dict: 1b8. in + the Python layer. + + -- C Function: int PyDict_Check (PyObject *p) + + Return true if `p' is a dict object or an instance of a subtype of + the dict type. + + -- C Function: int PyDict_CheckExact (PyObject *p) + + Return true if `p' is a dict object, but not an instance of a + subtype of the dict type. + + -- C Function: PyObject* PyDict_New () + `Return value: New reference.' Return a new empty dictionary, or + ‘NULL’ on failure. + + -- C Function: PyObject* PyDictProxy_New (PyObject *mapping) + `Return value: New reference.' Return a *note + types.MappingProxyType: ab6. object for a mapping which enforces + read-only behavior. This is normally used to create a view to + prevent modification of the dictionary for non-dynamic class types. + + -- C Function: void PyDict_Clear (PyObject *p) + + Empty an existing dictionary of all key-value pairs. + + -- C Function: int PyDict_Contains (PyObject *p, PyObject *key) + + Determine if dictionary `p' contains `key'. If an item in `p' is + matches `key', return ‘1’, otherwise return ‘0’. On error, return + ‘-1’. This is equivalent to the Python expression ‘key in p’. + + -- C Function: PyObject* PyDict_Copy (PyObject *p) + `Return value: New reference.' Return a new dictionary that + contains the same key-value pairs as `p'. + + -- C Function: int PyDict_SetItem (PyObject *p, PyObject *key, + PyObject *val) + + Insert `val' into the dictionary `p' with a key of `key'. `key' + must be *note hashable: 10c8.; if it isn’t, *note TypeError: 192. + will be raised. Return ‘0’ on success or ‘-1’ on failure. This + function `does not' steal a reference to `val'. + + -- C Function: int PyDict_SetItemString (PyObject *p, const char *key, + PyObject *val) + + Insert `val' into the dictionary `p' using `key' as a key. `key' + should be a ‘const char*’. The key object is created using + ‘PyUnicode_FromString(key)’. Return ‘0’ on success or ‘-1’ on + failure. This function `does not' steal a reference to `val'. + + -- C Function: int PyDict_DelItem (PyObject *p, PyObject *key) + + Remove the entry in dictionary `p' with key `key'. `key' must be + hashable; if it isn’t, *note TypeError: 192. is raised. If `key' + is not in the dictionary, *note KeyError: 2c7. is raised. Return + ‘0’ on success or ‘-1’ on failure. + + -- C Function: int PyDict_DelItemString (PyObject *p, const char *key) + + Remove the entry in dictionary `p' which has a key specified by the + string `key'. If `key' is not in the dictionary, *note KeyError: + 2c7. is raised. Return ‘0’ on success or ‘-1’ on failure. + + -- C Function: PyObject* PyDict_GetItem (PyObject *p, PyObject *key) + `Return value: Borrowed reference.' Return the object from + dictionary `p' which has a key `key'. Return ‘NULL’ if the key + `key' is not present, but `without' setting an exception. + + Note that exceptions which occur while calling *note __hash__(): + 340. and *note __eq__(): 33f. methods will get suppressed. To get + error reporting use *note PyDict_GetItemWithError(): 3bc7. instead. + + -- C Function: PyObject* PyDict_GetItemWithError (PyObject *p, + PyObject *key) + `Return value: Borrowed reference.' Variant of *note + PyDict_GetItem(): 385e. that does not suppress exceptions. Return + ‘NULL’ `with' an exception set if an exception occurred. Return + ‘NULL’ `without' an exception set if the key wasn’t present. + + -- C Function: PyObject* PyDict_GetItemString (PyObject *p, const + char *key) + `Return value: Borrowed reference.' This is the same as *note + PyDict_GetItem(): 385e, but `key' is specified as a ‘const char*’, + rather than a *note PyObject*: 4ba. + + Note that exceptions which occur while calling *note __hash__(): + 340. and *note __eq__(): 33f. methods and creating a temporary + string object will get suppressed. To get error reporting use + *note PyDict_GetItemWithError(): 3bc7. instead. + + -- C Function: PyObject* PyDict_SetDefault (PyObject *p, PyObject *key, + PyObject *defaultobj) + `Return value: Borrowed reference.' This is the same as the + Python-level *note dict.setdefault(): 9bf. If present, it returns + the value corresponding to `key' from the dictionary `p'. If the + key is not in the dict, it is inserted with value `defaultobj' and + `defaultobj' is returned. This function evaluates the hash + function of `key' only once, instead of evaluating it independently + for the lookup and the insertion. + + New in version 3.4. + + -- C Function: PyObject* PyDict_Items (PyObject *p) + `Return value: New reference.' Return a *note PyListObject: 3a56. + containing all the items from the dictionary. + + -- C Function: PyObject* PyDict_Keys (PyObject *p) + `Return value: New reference.' Return a *note PyListObject: 3a56. + containing all the keys from the dictionary. + + -- C Function: PyObject* PyDict_Values (PyObject *p) + `Return value: New reference.' Return a *note PyListObject: 3a56. + containing all the values from the dictionary `p'. + + -- C Function: Py_ssize_t PyDict_Size (PyObject *p) + + Return the number of items in the dictionary. This is equivalent + to ‘len(p)’ on a dictionary. + + -- C Function: int PyDict_Next (PyObject *p, Py_ssize_t *ppos, + PyObject **pkey, PyObject **pvalue) + + Iterate over all key-value pairs in the dictionary `p'. The + ‘Py_ssize_t’ referred to by `ppos' must be initialized to ‘0’ prior + to the first call to this function to start the iteration; the + function returns true for each pair in the dictionary, and false + once all pairs have been reported. The parameters `pkey' and + `pvalue' should either point to *note PyObject*: 4ba. variables + that will be filled in with each key and value, respectively, or + may be ‘NULL’. Any references returned through them are borrowed. + `ppos' should not be altered during iteration. Its value + represents offsets within the internal dictionary structure, and + since the structure is sparse, the offsets are not consecutive. + + For example: + + PyObject *key, *value; + Py_ssize_t pos = 0; + + while (PyDict_Next(self->dict, &pos, &key, &value)) { + /* do something interesting with the values... */ + ... + } + + The dictionary `p' should not be mutated during iteration. It is + safe to modify the values of the keys as you iterate over the + dictionary, but only so long as the set of keys does not change. + For example: + + PyObject *key, *value; + Py_ssize_t pos = 0; + + while (PyDict_Next(self->dict, &pos, &key, &value)) { + long i = PyLong_AsLong(value); + if (i == -1 && PyErr_Occurred()) { + return -1; + } + PyObject *o = PyLong_FromLong(i + 1); + if (o == NULL) + return -1; + if (PyDict_SetItem(self->dict, key, o) < 0) { + Py_DECREF(o); + return -1; + } + Py_DECREF(o); + } + + -- C Function: int PyDict_Merge (PyObject *a, PyObject *b, + int override) + + Iterate over mapping object `b' adding key-value pairs to + dictionary `a'. `b' may be a dictionary, or any object supporting + *note PyMapping_Keys(): 42c. and *note PyObject_GetItem(): 38f5. + If `override' is true, existing pairs in `a' will be replaced if a + matching key is found in `b', otherwise pairs will only be added if + there is not a matching key in `a'. Return ‘0’ on success or ‘-1’ + if an exception was raised. + + -- C Function: int PyDict_Update (PyObject *a, PyObject *b) + + This is the same as ‘PyDict_Merge(a, b, 1)’ in C, and is similar to + ‘a.update(b)’ in Python except that *note PyDict_Update(): 3bcf. + doesn’t fall back to the iterating over a sequence of key value + pairs if the second argument has no “keys” attribute. Return ‘0’ + on success or ‘-1’ if an exception was raised. + + -- C Function: int PyDict_MergeFromSeq2 (PyObject *a, PyObject *seq2, + int override) + + Update or merge into dictionary `a', from the key-value pairs in + `seq2'. `seq2' must be an iterable object producing iterable + objects of length 2, viewed as key-value pairs. In case of + duplicate keys, the last wins if `override' is true, else the first + wins. Return ‘0’ on success or ‘-1’ if an exception was raised. + Equivalent Python (except for the return value): + + def PyDict_MergeFromSeq2(a, seq2, override): + for key, value in seq2: + if override or key not in a: + a[key] = value + + -- C Function: int PyDict_ClearFreeList () + + Clear the free list. Return the total number of freed items. + + New in version 3.3. + + +File: python.info, Node: Set Objects, Prev: Dictionary Objects, Up: Container Objects + +7.8.4.2 Set Objects +................... + +This section details the public API for *note set: b6f. and *note +frozenset: bee. objects. Any functionality not listed below is best +accessed using the either the abstract object protocol (including *note +PyObject_CallMethod(): 3a0e, *note PyObject_RichCompareBool(): 38a7, +*note PyObject_Hash(): 3a15, *note PyObject_Repr(): 968, *note +PyObject_IsTrue(): 3a16, *note PyObject_Print(): 39fe, and *note +PyObject_GetIter(): 3a1d.) or the abstract number protocol (including +*note PyNumber_And(): 3a2f, *note PyNumber_Subtract(): 3a22, *note +PyNumber_Or(): 3a31, *note PyNumber_Xor(): 3a30, *note +PyNumber_InPlaceAnd(): 3a3b, *note PyNumber_InPlaceSubtract(): 3a33, +*note PyNumber_InPlaceOr(): 3a3d, and *note PyNumber_InPlaceXor(): +3a3c.). + + -- C Type: PySetObject + + This subtype of *note PyObject: 4ba. is used to hold the internal + data for both *note set: b6f. and *note frozenset: bee. objects. + It is like a *note PyDictObject: 3bbc. in that it is a fixed size + for small sets (much like tuple storage) and will point to a + separate, variable sized block of memory for medium and large sized + sets (much like list storage). None of the fields of this + structure should be considered public and are subject to change. + All access should be done through the documented API rather than by + manipulating the values in the structure. + + -- C Variable: PyTypeObject PySet_Type + + This is an instance of *note PyTypeObject: 2d6. representing the + Python *note set: b6f. type. + + -- C Variable: PyTypeObject PyFrozenSet_Type + + This is an instance of *note PyTypeObject: 2d6. representing the + Python *note frozenset: bee. type. + +The following type check macros work on pointers to any Python object. +Likewise, the constructor functions work with any iterable Python +object. + + -- C Function: int PySet_Check (PyObject *p) + + Return true if `p' is a *note set: b6f. object or an instance of a + subtype. + + -- C Function: int PyFrozenSet_Check (PyObject *p) + + Return true if `p' is a *note frozenset: bee. object or an instance + of a subtype. + + -- C Function: int PyAnySet_Check (PyObject *p) + + Return true if `p' is a *note set: b6f. object, a *note frozenset: + bee. object, or an instance of a subtype. + + -- C Function: int PyAnySet_CheckExact (PyObject *p) + + Return true if `p' is a *note set: b6f. object or a *note + frozenset: bee. object but not an instance of a subtype. + + -- C Function: int PyFrozenSet_CheckExact (PyObject *p) + + Return true if `p' is a *note frozenset: bee. object but not an + instance of a subtype. + + -- C Function: PyObject* PySet_New (PyObject *iterable) + `Return value: New reference.' Return a new *note set: b6f. + containing objects returned by the `iterable'. The `iterable' may + be ‘NULL’ to create a new empty set. Return the new set on success + or ‘NULL’ on failure. Raise *note TypeError: 192. if `iterable' is + not actually iterable. The constructor is also useful for copying + a set (‘c=set(s)’). + + -- C Function: PyObject* PyFrozenSet_New (PyObject *iterable) + `Return value: New reference.' Return a new *note frozenset: bee. + containing objects returned by the `iterable'. The `iterable' may + be ‘NULL’ to create a new empty frozenset. Return the new set on + success or ‘NULL’ on failure. Raise *note TypeError: 192. if + `iterable' is not actually iterable. + +The following functions and macros are available for instances of *note +set: b6f. or *note frozenset: bee. or instances of their subtypes. + + -- C Function: Py_ssize_t PySet_Size (PyObject *anyset) + + Return the length of a *note set: b6f. or *note frozenset: bee. + object. Equivalent to ‘len(anyset)’. Raises a ‘PyExc_SystemError’ + if `anyset' is not a *note set: b6f, *note frozenset: bee, or an + instance of a subtype. + + -- C Function: Py_ssize_t PySet_GET_SIZE (PyObject *anyset) + + Macro form of *note PySet_Size(): db0. without error checking. + + -- C Function: int PySet_Contains (PyObject *anyset, PyObject *key) + + Return ‘1’ if found, ‘0’ if not found, and ‘-1’ if an error is + encountered. Unlike the Python *note __contains__(): d28. method, + this function does not automatically convert unhashable sets into + temporary frozensets. Raise a *note TypeError: 192. if the `key' + is unhashable. Raise ‘PyExc_SystemError’ if `anyset' is not a + *note set: b6f, *note frozenset: bee, or an instance of a subtype. + + -- C Function: int PySet_Add (PyObject *set, PyObject *key) + + Add `key' to a *note set: b6f. instance. Also works with *note + frozenset: bee. instances (like *note PyTuple_SetItem(): 3861. it + can be used to fill-in the values of brand new frozensets before + they are exposed to other code). Return ‘0’ on success or ‘-1’ on + failure. Raise a *note TypeError: 192. if the `key' is unhashable. + Raise a *note MemoryError: 13af. if there is no room to grow. + Raise a *note SystemError: 5fb. if `set' is not an instance of + *note set: b6f. or its subtype. + +The following functions are available for instances of *note set: b6f. +or its subtypes but not for instances of *note frozenset: bee. or its +subtypes. + + -- C Function: int PySet_Discard (PyObject *set, PyObject *key) + + Return ‘1’ if found and removed, ‘0’ if not found (no action + taken), and ‘-1’ if an error is encountered. Does not raise *note + KeyError: 2c7. for missing keys. Raise a *note TypeError: 192. if + the `key' is unhashable. Unlike the Python ‘discard()’ method, + this function does not automatically convert unhashable sets into + temporary frozensets. Raise ‘PyExc_SystemError’ if `set' is not an + instance of *note set: b6f. or its subtype. + + -- C Function: PyObject* PySet_Pop (PyObject *set) + `Return value: New reference.' Return a new reference to an + arbitrary object in the `set', and removes the object from the + `set'. Return ‘NULL’ on failure. Raise *note KeyError: 2c7. if + the set is empty. Raise a *note SystemError: 5fb. if `set' is not + an instance of *note set: b6f. or its subtype. + + -- C Function: int PySet_Clear (PyObject *set) + + Empty an existing set of all elements. + + -- C Function: int PySet_ClearFreeList () + + Clear the free list. Return the total number of freed items. + + New in version 3.3. + + +File: python.info, Node: Function Objects<2>, Next: Other Objects, Prev: Container Objects, Up: Concrete Objects Layer + +7.8.5 Function Objects +---------------------- + +* Menu: + +* Function Objects: Function Objects<3>. +* Instance Method Objects:: +* Method Objects: Method Objects<2>. +* Cell Objects:: +* Code Objects: Code Objects<2>. + + +File: python.info, Node: Function Objects<3>, Next: Instance Method Objects, Up: Function Objects<2> + +7.8.5.1 Function Objects +........................ + +There are a few functions specific to Python functions. + + -- C Type: PyFunctionObject + + The C structure used for functions. + + -- C Variable: PyTypeObject PyFunction_Type + + This is an instance of *note PyTypeObject: 2d6. and represents the + Python function type. It is exposed to Python programmers as + ‘types.FunctionType’. + + -- C Function: int PyFunction_Check (PyObject *o) + + Return true if `o' is a function object (has type *note + PyFunction_Type: 3be7.). The parameter must not be ‘NULL’. + + -- C Function: PyObject* PyFunction_New (PyObject *code, + PyObject *globals) + `Return value: New reference.' Return a new function object + associated with the code object `code'. `globals' must be a + dictionary with the global variables accessible to the function. + + The function’s docstring and name are retrieved from the code + object. `__module__' is retrieved from `globals'. The argument + defaults, annotations and closure are set to ‘NULL’. + `__qualname__' is set to the same value as the function’s name. + + -- C Function: PyObject* PyFunction_NewWithQualName (PyObject *code, + PyObject *globals, PyObject *qualname) + `Return value: New reference.' As *note PyFunction_New(): 3be9, + but also allows setting the function object’s ‘__qualname__’ + attribute. `qualname' should be a unicode object or ‘NULL’; if + ‘NULL’, the ‘__qualname__’ attribute is set to the same value as + its ‘__name__’ attribute. + + New in version 3.3. + + -- C Function: PyObject* PyFunction_GetCode (PyObject *op) + `Return value: Borrowed reference.' Return the code object + associated with the function object `op'. + + -- C Function: PyObject* PyFunction_GetGlobals (PyObject *op) + `Return value: Borrowed reference.' Return the globals dictionary + associated with the function object `op'. + + -- C Function: PyObject* PyFunction_GetModule (PyObject *op) + `Return value: Borrowed reference.' Return the `__module__' + attribute of the function object `op'. This is normally a string + containing the module name, but can be set to any other object by + Python code. + + -- C Function: PyObject* PyFunction_GetDefaults (PyObject *op) + `Return value: Borrowed reference.' Return the argument default + values of the function object `op'. This can be a tuple of + arguments or ‘NULL’. + + -- C Function: int PyFunction_SetDefaults (PyObject *op, + PyObject *defaults) + + Set the argument default values for the function object `op'. + `defaults' must be ‘Py_None’ or a tuple. + + Raises *note SystemError: 5fb. and returns ‘-1’ on failure. + + -- C Function: PyObject* PyFunction_GetClosure (PyObject *op) + `Return value: Borrowed reference.' Return the closure associated + with the function object `op'. This can be ‘NULL’ or a tuple of + cell objects. + + -- C Function: int PyFunction_SetClosure (PyObject *op, + PyObject *closure) + + Set the closure associated with the function object `op'. + `closure' must be ‘Py_None’ or a tuple of cell objects. + + Raises *note SystemError: 5fb. and returns ‘-1’ on failure. + + -- C Function: PyObject *PyFunction_GetAnnotations (PyObject *op) + `Return value: Borrowed reference.' Return the annotations of the + function object `op'. This can be a mutable dictionary or ‘NULL’. + + -- C Function: int PyFunction_SetAnnotations (PyObject *op, + PyObject *annotations) + + Set the annotations for the function object `op'. `annotations' + must be a dictionary or ‘Py_None’. + + Raises *note SystemError: 5fb. and returns ‘-1’ on failure. + + +File: python.info, Node: Instance Method Objects, Next: Method Objects<2>, Prev: Function Objects<3>, Up: Function Objects<2> + +7.8.5.2 Instance Method Objects +............................... + +An instance method is a wrapper for a *note PyCFunction: ddb. and the +new way to bind a *note PyCFunction: ddb. to a class object. It +replaces the former call ‘PyMethod_New(func, NULL, class)’. + + -- C Variable: PyTypeObject PyInstanceMethod_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + instance method type. It is not exposed to Python programs. + + -- C Function: int PyInstanceMethod_Check (PyObject *o) + + Return true if `o' is an instance method object (has type *note + PyInstanceMethod_Type: 3bf7.). The parameter must not be ‘NULL’. + + -- C Function: PyObject* PyInstanceMethod_New (PyObject *func) + `Return value: New reference.' Return a new instance method + object, with `func' being any callable object `func' is the + function that will be called when the instance method is called. + + -- C Function: PyObject* PyInstanceMethod_Function (PyObject *im) + `Return value: Borrowed reference.' Return the function object + associated with the instance method `im'. + + -- C Function: PyObject* PyInstanceMethod_GET_FUNCTION (PyObject *im) + `Return value: Borrowed reference.' Macro version of *note + PyInstanceMethod_Function(): 3bfa. which avoids error checking. + + +File: python.info, Node: Method Objects<2>, Next: Cell Objects, Prev: Instance Method Objects, Up: Function Objects<2> + +7.8.5.3 Method Objects +...................... + +Methods are bound function objects. Methods are always bound to an +instance of a user-defined class. Unbound methods (methods bound to a +class object) are no longer available. + + -- C Variable: PyTypeObject PyMethod_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + method type. This is exposed to Python programs as + ‘types.MethodType’. + + -- C Function: int PyMethod_Check (PyObject *o) + + Return true if `o' is a method object (has type *note + PyMethod_Type: 3bfe.). The parameter must not be ‘NULL’. + + -- C Function: PyObject* PyMethod_New (PyObject *func, PyObject *self) + `Return value: New reference.' Return a new method object, with + `func' being any callable object and `self' the instance the method + should be bound. `func' is the function that will be called when + the method is called. `self' must not be ‘NULL’. + + -- C Function: PyObject* PyMethod_Function (PyObject *meth) + `Return value: Borrowed reference.' Return the function object + associated with the method `meth'. + + -- C Function: PyObject* PyMethod_GET_FUNCTION (PyObject *meth) + `Return value: Borrowed reference.' Macro version of *note + PyMethod_Function(): 3c01. which avoids error checking. + + -- C Function: PyObject* PyMethod_Self (PyObject *meth) + `Return value: Borrowed reference.' Return the instance associated + with the method `meth'. + + -- C Function: PyObject* PyMethod_GET_SELF (PyObject *meth) + `Return value: Borrowed reference.' Macro version of *note + PyMethod_Self(): 3c03. which avoids error checking. + + -- C Function: int PyMethod_ClearFreeList () + + Clear the free list. Return the total number of freed items. + + +File: python.info, Node: Cell Objects, Next: Code Objects<2>, Prev: Method Objects<2>, Up: Function Objects<2> + +7.8.5.4 Cell Objects +.................... + +“Cell” objects are used to implement variables referenced by multiple +scopes. For each such variable, a cell object is created to store the +value; the local variables of each stack frame that references the value +contains a reference to the cells from outer scopes which also use that +variable. When the value is accessed, the value contained in the cell +is used instead of the cell object itself. This de-referencing of the +cell object requires support from the generated byte-code; these are not +automatically de-referenced when accessed. Cell objects are not likely +to be useful elsewhere. + + -- C Type: PyCellObject + + The C structure used for cell objects. + + -- C Variable: PyTypeObject PyCell_Type + + The type object corresponding to cell objects. + + -- C Function: int PyCell_Check (ob) + + Return true if `ob' is a cell object; `ob' must not be ‘NULL’. + + -- C Function: PyObject* PyCell_New (PyObject *ob) + `Return value: New reference.' Create and return a new cell object + containing the value `ob'. The parameter may be ‘NULL’. + + -- C Function: PyObject* PyCell_Get (PyObject *cell) + `Return value: New reference.' Return the contents of the cell + `cell'. + + -- C Function: PyObject* PyCell_GET (PyObject *cell) + `Return value: Borrowed reference.' Return the contents of the + cell `cell', but without checking that `cell' is non-‘NULL’ and a + cell object. + + -- C Function: int PyCell_Set (PyObject *cell, PyObject *value) + + Set the contents of the cell object `cell' to `value'. This + releases the reference to any current content of the cell. `value' + may be ‘NULL’. `cell' must be non-‘NULL’; if it is not a cell + object, ‘-1’ will be returned. On success, ‘0’ will be returned. + + -- C Function: void PyCell_SET (PyObject *cell, PyObject *value) + + Sets the value of the cell object `cell' to `value'. No reference + counts are adjusted, and no checks are made for safety; `cell' must + be non-‘NULL’ and must be a cell object. + + +File: python.info, Node: Code Objects<2>, Prev: Cell Objects, Up: Function Objects<2> + +7.8.5.5 Code Objects +.................... + +Code objects are a low-level detail of the CPython implementation. Each +one represents a chunk of executable code that hasn’t yet been bound +into a function. + + -- C Type: PyCodeObject + + The C structure of the objects used to describe code objects. The + fields of this type are subject to change at any time. + + -- C Variable: PyTypeObject PyCode_Type + + This is an instance of *note PyTypeObject: 2d6. representing the + Python *note code: 1b. type. + + -- C Function: int PyCode_Check (PyObject *co) + + Return true if `co' is a *note code: 1b. object. + + -- C Function: int PyCode_GetNumFree (PyCodeObject *co) + + Return the number of free variables in `co'. + + -- C Function: PyCodeObject* PyCode_New (int argcount, + int kwonlyargcount, int nlocals, int stacksize, int flags, + PyObject *code, PyObject *consts, PyObject *names, + PyObject *varnames, PyObject *freevars, PyObject *cellvars, + PyObject *filename, PyObject *name, int firstlineno, + PyObject *lnotab) + `Return value: New reference.' Return a new code object. If you + need a dummy code object to create a frame, use *note + PyCode_NewEmpty(): cea. instead. Calling *note PyCode_New(): 272. + directly can bind you to a precise Python version since the + definition of the bytecode changes often. + + -- C Function: PyCodeObject* PyCode_NewWithPosOnlyArgs (int argcount, + int posonlyargcount, int kwonlyargcount, int nlocals, + int stacksize, int flags, PyObject *code, PyObject *consts, + PyObject *names, PyObject *varnames, PyObject *freevars, + PyObject *cellvars, PyObject *filename, PyObject *name, + int firstlineno, PyObject *lnotab) + `Return value: New reference.' Similar to *note PyCode_New(): 272, + but with an extra “posonlyargcount” for positional-only arguments. + + New in version 3.8. + + -- C Function: PyCodeObject* PyCode_NewEmpty (const char *filename, + const char *funcname, int firstlineno) + `Return value: New reference.' Return a new empty code object with + the specified filename, function name, and first line number. It + is illegal to *note exec(): c44. or *note eval(): b90. the + resulting code object. + + +File: python.info, Node: Other Objects, Prev: Function Objects<2>, Up: Concrete Objects Layer + +7.8.6 Other Objects +------------------- + +* Menu: + +* File Objects:: +* Module Objects:: +* Iterator Objects:: +* Descriptor Objects:: +* Slice Objects:: +* Ellipsis Object:: +* MemoryView objects:: +* Weak Reference Objects: Weak Reference Objects<2>. +* Capsules: Capsules<2>. +* Generator Objects:: +* Coroutine Objects: Coroutine Objects<2>. +* Context Variables Objects:: +* DateTime Objects: DateTime Objects<2>. + + +File: python.info, Node: File Objects, Next: Module Objects, Up: Other Objects + +7.8.6.1 File Objects +.................... + +These APIs are a minimal emulation of the Python 2 C API for built-in +file objects, which used to rely on the buffered I/O (‘FILE*’) support +from the C standard library. In Python 3, files and streams use the new +*note io: a1. module, which defines several layers over the low-level +unbuffered I/O of the operating system. The functions described below +are convenience C wrappers over these new APIs, and meant mostly for +internal error reporting in the interpreter; third-party code is advised +to access the *note io: a1. APIs instead. + + -- C Function: PyObject* PyFile_FromFd (int fd, const char *name, const + char *mode, int buffering, const char *encoding, const + char *errors, const char *newline, int closefd) + `Return value: New reference.' Create a Python file object from + the file descriptor of an already opened file `fd'. The arguments + `name', `encoding', `errors' and `newline' can be ‘NULL’ to use the + defaults; `buffering' can be `-1' to use the default. `name' is + ignored and kept for backward compatibility. Return ‘NULL’ on + failure. For a more comprehensive description of the arguments, + please refer to the *note io.open(): 65a. function documentation. + + Warning: Since Python streams have their own buffering layer, + mixing them with OS-level file descriptors can produce various + issues (such as unexpected ordering of data). + + Changed in version 3.2: Ignore `name' attribute. + + -- C Function: int PyObject_AsFileDescriptor (PyObject *p) + + Return the file descriptor associated with `p' as an ‘int’. If the + object is an integer, its value is returned. If not, the object’s + *note fileno(): 1bdb. method is called if it exists; the method + must return an integer, which is returned as the file descriptor + value. Sets an exception and returns ‘-1’ on failure. + + -- C Function: PyObject* PyFile_GetLine (PyObject *p, int n) + `Return value: New reference.' + + Equivalent to ‘p.readline([n])’, this function reads one line from + the object `p'. `p' may be a file object or any object with a + *note readline(): 13ae. method. If `n' is ‘0’, exactly one line is + read, regardless of the length of the line. If `n' is greater than + ‘0’, no more than `n' bytes will be read from the file; a partial + line can be returned. In both cases, an empty string is returned + if the end of the file is reached immediately. If `n' is less than + ‘0’, however, one line is read regardless of length, but *note + EOFError: c6c. is raised if the end of the file is reached + immediately. + + -- C Function: int PyFile_SetOpenCodeHook + (Py_OpenCodeHookFunction handler) + + Overrides the normal behavior of *note io.open_code(): 1cc0. to + pass its parameter through the provided handler. + + The handler is a function of type ‘PyObject *(*)(PyObject *path, + void *userData)’, where `path' is guaranteed to be *note + PyUnicodeObject: 3b3c. + + The `userData' pointer is passed into the hook function. Since + hook functions may be called from different runtimes, this pointer + should not refer directly to Python state. + + As this hook is intentionally used during import, avoid importing + new modules during its execution unless they are known to be frozen + or available in ‘sys.modules’. + + Once a hook has been set, it cannot be removed or replaced, and + later calls to *note PyFile_SetOpenCodeHook(): 1cc1. will fail. On + failure, the function returns -1 and sets an exception if the + interpreter has been initialized. + + This function is safe to call before *note Py_Initialize(): 439. + + Raises an *note auditing event: fd1. ‘setopencodehook’ with no + arguments. + + New in version 3.8. + + -- C Function: int PyFile_WriteObject (PyObject *obj, PyObject *p, + int flags) + + Write object `obj' to file object `p'. The only supported flag for + `flags' is ‘Py_PRINT_RAW’; if given, the *note str(): 330. of the + object is written instead of the *note repr(): 7cc. Return ‘0’ on + success or ‘-1’ on failure; the appropriate exception will be set. + + -- C Function: int PyFile_WriteString (const char *s, PyObject *p) + + Write string `s' to file object `p'. Return ‘0’ on success or ‘-1’ + on failure; the appropriate exception will be set. + + +File: python.info, Node: Module Objects, Next: Iterator Objects, Prev: File Objects, Up: Other Objects + +7.8.6.2 Module Objects +...................... + + -- C Variable: PyTypeObject PyModule_Type + + This instance of *note PyTypeObject: 2d6. represents the Python + module type. This is exposed to Python programs as + ‘types.ModuleType’. + + -- C Function: int PyModule_Check (PyObject *p) + + Return true if `p' is a module object, or a subtype of a module + object. + + -- C Function: int PyModule_CheckExact (PyObject *p) + + Return true if `p' is a module object, but not a subtype of *note + PyModule_Type: 3c24. + + -- C Function: PyObject* PyModule_NewObject (PyObject *name) + `Return value: New reference.' + + Return a new module object with the *note __name__: d12. attribute + set to `name'. The module’s *note __name__: d12, ‘__doc__’, *note + __package__: 955, and *note __loader__: 956. attributes are filled + in (all but *note __name__: d12. are set to ‘None’); the caller is + responsible for providing a *note __file__: 10d0. attribute. + + New in version 3.3. + + Changed in version 3.4: *note __package__: 955. and *note + __loader__: 956. are set to ‘None’. + + -- C Function: PyObject* PyModule_New (const char *name) + `Return value: New reference.' Similar to *note + PyModule_NewObject(): 3c27, but the name is a UTF-8 encoded string + instead of a Unicode object. + + -- C Function: PyObject* PyModule_GetDict (PyObject *module) + `Return value: Borrowed reference.' + + Return the dictionary object that implements `module'’s namespace; + this object is the same as the *note __dict__: 4fc. attribute of + the module object. If `module' is not a module object (or a + subtype of a module object), *note SystemError: 5fb. is raised and + ‘NULL’ is returned. + + It is recommended extensions use other ‘PyModule_*()’ and + ‘PyObject_*()’ functions rather than directly manipulate a module’s + *note __dict__: 4fc. + + -- C Function: PyObject* PyModule_GetNameObject (PyObject *module) + `Return value: New reference.' + + Return `module'’s *note __name__: d12. value. If the module does + not provide one, or if it is not a string, *note SystemError: 5fb. + is raised and ‘NULL’ is returned. + + New in version 3.3. + + -- C Function: const char* PyModule_GetName (PyObject *module) + + Similar to *note PyModule_GetNameObject(): 3c2a. but return the + name encoded to ‘'utf-8'’. + + -- C Function: void* PyModule_GetState (PyObject *module) + + Return the “state” of the module, that is, a pointer to the block + of memory allocated at module creation time, or ‘NULL’. See *note + PyModuleDef.m_size: 3c2d. + + -- C Function: PyModuleDef* PyModule_GetDef (PyObject *module) + + Return a pointer to the *note PyModuleDef: 3888. struct from which + the module was created, or ‘NULL’ if the module wasn’t created from + a definition. + + -- C Function: PyObject* PyModule_GetFilenameObject (PyObject *module) + `Return value: New reference.' + + Return the name of the file from which `module' was loaded using + `module'’s *note __file__: 10d0. attribute. If this is not + defined, or if it is not a unicode string, raise *note SystemError: + 5fb. and return ‘NULL’; otherwise return a reference to a Unicode + object. + + New in version 3.2. + + -- C Function: const char* PyModule_GetFilename (PyObject *module) + + Similar to *note PyModule_GetFilenameObject(): 3c2f. but return the + filename encoded to ‘utf-8’. + + Deprecated since version 3.2: *note PyModule_GetFilename(): 3c30. + raises ‘UnicodeEncodeError’ on unencodable filenames, use *note + PyModule_GetFilenameObject(): 3c2f. instead. + +* Menu: + +* Initializing C modules:: +* Module lookup:: + + +File: python.info, Node: Initializing C modules, Next: Module lookup, Up: Module Objects + +7.8.6.3 Initializing C modules +.............................. + +Modules objects are usually created from extension modules (shared +libraries which export an initialization function), or compiled-in +modules (where the initialization function is added using *note +PyImport_AppendInittab(): 3848.). See *note Building C and C++ +Extensions: 384b. or *note Extending Embedded Python: 38d3. for details. + +The initialization function can either pass a module definition instance +to *note PyModule_Create(): 3847, and return the resulting module +object, or request “multi-phase initialization” by returning the +definition struct itself. + + -- C Type: PyModuleDef + + The module definition struct, which holds all information needed to + create a module object. There is usually only one statically + initialized variable of this type for each module. + + -- C Member: PyModuleDef_Base m_base + + Always initialize this member to ‘PyModuleDef_HEAD_INIT’. + + -- C Member: const char *m_name + + Name for the new module. + + -- C Member: const char *m_doc + + Docstring for the module; usually a docstring variable created + with *note PyDoc_STRVAR: 38ea. is used. + + -- C Member: Py_ssize_t m_size + + Module state may be kept in a per-module memory area that can + be retrieved with *note PyModule_GetState(): 3c2c, rather than + in static globals. This makes modules safe for use in + multiple sub-interpreters. + + This memory area is allocated based on `m_size' on module + creation, and freed when the module object is deallocated, + after the ‘m_free’ function has been called, if present. + + Setting ‘m_size’ to ‘-1’ means that the module does not + support sub-interpreters, because it has global state. + + Setting it to a non-negative value means that the module can + be re-initialized and specifies the additional amount of + memory it requires for its state. Non-negative ‘m_size’ is + required for multi-phase initialization. + + See PEP 3121(1) for more details. + + -- C Member: PyMethodDef* m_methods + + A pointer to a table of module-level functions, described by + *note PyMethodDef: e1b. values. Can be ‘NULL’ if no functions + are present. + + -- C Member: PyModuleDef_Slot* m_slots + + An array of slot definitions for multi-phase initialization, + terminated by a ‘{0, NULL}’ entry. When using single-phase + initialization, `m_slots' must be ‘NULL’. + + Changed in version 3.5: Prior to version 3.5, this member was + always set to ‘NULL’, and was defined as: + + -- C Member: inquiry m_reload + + -- C Member: traverseproc m_traverse + + A traversal function to call during GC traversal of the module + object, or ‘NULL’ if not needed. This function may be called + before module state is allocated (*note PyModule_GetState(): + 3c2c. may return ‘NULL’), and before the *note Py_mod_exec: + 3c39. function is executed. + + -- C Member: inquiry m_clear + + A clear function to call during GC clearing of the module + object, or ‘NULL’ if not needed. This function may be called + before module state is allocated (*note PyModule_GetState(): + 3c2c. may return ‘NULL’), and before the *note Py_mod_exec: + 3c39. function is executed. + + -- C Member: freefunc m_free + + A function to call during deallocation of the module object, + or ‘NULL’ if not needed. This function may be called before + module state is allocated (*note PyModule_GetState(): 3c2c. + may return ‘NULL’), and before the *note Py_mod_exec: 3c39. + function is executed. + +* Menu: + +* Single-phase initialization:: +* Multi-phase initialization:: +* Low-level module creation functions:: +* Support functions:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3121 + + +File: python.info, Node: Single-phase initialization, Next: Multi-phase initialization, Up: Initializing C modules + +7.8.6.4 Single-phase initialization +................................... + +The module initialization function may create and return the module +object directly. This is referred to as “single-phase initialization”, +and uses one of the following two module creation functions: + + -- C Function: PyObject* PyModule_Create (PyModuleDef *def) + `Return value: New reference.' Create a new module object, given + the definition in `def'. This behaves like *note + PyModule_Create2(): 3c3d. with `module_api_version' set to + ‘PYTHON_API_VERSION’. + + -- C Function: PyObject* PyModule_Create2 (PyModuleDef *def, + int module_api_version) + `Return value: New reference.' Create a new module object, given + the definition in `def', assuming the API version + `module_api_version'. If that version does not match the version + of the running interpreter, a *note RuntimeWarning: 2c0. is + emitted. + + Note: Most uses of this function should be using *note + PyModule_Create(): 3847. instead; only use this if you are + sure you need it. + +Before it is returned from in the initialization function, the resulting +module object is typically populated using functions like *note +PyModule_AddObject(): 3c3e. + + +File: python.info, Node: Multi-phase initialization, Next: Low-level module creation functions, Prev: Single-phase initialization, Up: Initializing C modules + +7.8.6.5 Multi-phase initialization +.................................. + +An alternate way to specify extensions is to request “multi-phase +initialization”. Extension modules created this way behave more like +Python modules: the initialization is split between the `creation +phase', when the module object is created, and the `execution phase', +when it is populated. The distinction is similar to the *note +__new__(): 889. and *note __init__(): d5e. methods of classes. + +Unlike modules created using single-phase initialization, these modules +are not singletons: if the `sys.modules' entry is removed and the module +is re-imported, a new module object is created, and the old module is +subject to normal garbage collection – as with Python modules. By +default, multiple modules created from the same definition should be +independent: changes to one should not affect the others. This means +that all state should be specific to the module object (using e.g. +using *note PyModule_GetState(): 3c2c.), or its contents (such as the +module’s *note __dict__: 4fc. or individual classes created with *note +PyType_FromSpec(): 2d1.). + +All modules created using multi-phase initialization are expected to +support *note sub-interpreters: 398b. Making sure multiple modules are +independent is typically enough to achieve this. + +To request multi-phase initialization, the initialization function +(PyInit_modulename) returns a *note PyModuleDef: 3888. instance with +non-empty *note m_slots: 3c36. Before it is returned, the ‘PyModuleDef’ +instance must be initialized with the following function: + + -- C Function: PyObject* PyModuleDef_Init (PyModuleDef *def) + `Return value: Borrowed reference.' Ensures a module definition is + a properly initialized Python object that correctly reports its + type and reference count. + + Returns `def' cast to ‘PyObject*’, or ‘NULL’ if an error occurred. + + New in version 3.5. + +The `m_slots' member of the module definition must point to an array of +‘PyModuleDef_Slot’ structures: + + -- C Type: PyModuleDef_Slot + + -- C Member: int slot + + A slot ID, chosen from the available values explained below. + + -- C Member: void* value + + Value of the slot, whose meaning depends on the slot ID. + + New in version 3.5. + +The `m_slots' array must be terminated by a slot with id 0. + +The available slot types are: + + -- C Macro: Py_mod_create + + Specifies a function that is called to create the module object + itself. The `value' pointer of this slot must point to a function + of the signature: + + -- C Function: PyObject* create_module (PyObject *spec, + PyModuleDef *def) + + The function receives a *note ModuleSpec: 116b. instance, as + defined in PEP 451(1), and the module definition. It should return + a new module object, or set an error and return ‘NULL’. + + This function should be kept minimal. In particular, it should not + call arbitrary Python code, as trying to import the same module + again may result in an infinite loop. + + Multiple ‘Py_mod_create’ slots may not be specified in one module + definition. + + If ‘Py_mod_create’ is not specified, the import machinery will + create a normal module object using *note PyModule_New(): 3c28. + The name is taken from `spec', not the definition, to allow + extension modules to dynamically adjust to their place in the + module hierarchy and be imported under different names through + symlinks, all while sharing a single module definition. + + There is no requirement for the returned object to be an instance + of *note PyModule_Type: 3c24. Any type can be used, as long as it + supports setting and getting import-related attributes. However, + only ‘PyModule_Type’ instances may be returned if the ‘PyModuleDef’ + has non-‘NULL’ ‘m_traverse’, ‘m_clear’, ‘m_free’; non-zero + ‘m_size’; or slots other than ‘Py_mod_create’. + + -- C Macro: Py_mod_exec + + Specifies a function that is called to `execute' the module. This + is equivalent to executing the code of a Python module: typically, + this function adds classes and constants to the module. The + signature of the function is: + + -- C Function: int exec_module (PyObject* module) + + If multiple ‘Py_mod_exec’ slots are specified, they are processed + in the order they appear in the `m_slots' array. + +See PEP 489(2) for more details on multi-phase initialization. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0451 + + (2) https://www.python.org/dev/peps/pep-0489 + + +File: python.info, Node: Low-level module creation functions, Next: Support functions, Prev: Multi-phase initialization, Up: Initializing C modules + +7.8.6.6 Low-level module creation functions +........................................... + +The following functions are called under the hood when using multi-phase +initialization. They can be used directly, for example when creating +module objects dynamically. Note that both ‘PyModule_FromDefAndSpec’ +and ‘PyModule_ExecDef’ must be called to fully initialize a module. + + -- C Function: PyObject * PyModule_FromDefAndSpec (PyModuleDef *def, + PyObject *spec) + `Return value: New reference.' Create a new module object, given + the definition in `module' and the ModuleSpec `spec'. This behaves + like *note PyModule_FromDefAndSpec2(): 7ac. with + `module_api_version' set to ‘PYTHON_API_VERSION’. + + New in version 3.5. + + -- C Function: PyObject * PyModule_FromDefAndSpec2 (PyModuleDef *def, + PyObject *spec, int module_api_version) + `Return value: New reference.' Create a new module object, given + the definition in `module' and the ModuleSpec `spec', assuming the + API version `module_api_version'. If that version does not match + the version of the running interpreter, a *note RuntimeWarning: + 2c0. is emitted. + + Note: Most uses of this function should be using *note + PyModule_FromDefAndSpec(): 7ab. instead; only use this if you + are sure you need it. + + New in version 3.5. + + -- C Function: int PyModule_ExecDef (PyObject *module, + PyModuleDef *def) + + Process any execution slots (*note Py_mod_exec: 3c39.) given in + `def'. + + New in version 3.5. + + -- C Function: int PyModule_SetDocString (PyObject *module, const + char *docstring) + + Set the docstring for `module' to `docstring'. This function is + called automatically when creating a module from ‘PyModuleDef’, + using either ‘PyModule_Create’ or ‘PyModule_FromDefAndSpec’. + + New in version 3.5. + + -- C Function: int PyModule_AddFunctions (PyObject *module, + PyMethodDef *functions) + + Add the functions from the ‘NULL’ terminated `functions' array to + `module'. Refer to the *note PyMethodDef: e1b. documentation for + details on individual entries (due to the lack of a shared module + namespace, module level “functions” implemented in C typically + receive the module as their first parameter, making them similar to + instance methods on Python classes). This function is called + automatically when creating a module from ‘PyModuleDef’, using + either ‘PyModule_Create’ or ‘PyModule_FromDefAndSpec’. + + New in version 3.5. + + +File: python.info, Node: Support functions, Prev: Low-level module creation functions, Up: Initializing C modules + +7.8.6.7 Support functions +......................... + +The module initialization function (if using single phase +initialization) or a function called from a module execution slot (if +using multi-phase initialization), can use the following functions to +help initialize the module state: + + -- C Function: int PyModule_AddObject (PyObject *module, const + char *name, PyObject *value) + + Add an object to `module' as `name'. This is a convenience + function which can be used from the module’s initialization + function. This steals a reference to `value' on success. Return + ‘-1’ on error, ‘0’ on success. + + Note: Unlike other functions that steal references, + ‘PyModule_AddObject()’ only decrements the reference count of + `value' `on success'. + + This means that its return value must be checked, and calling + code must *note Py_DECREF(): 266. `value' manually on error. + Example usage: + + Py_INCREF(spam); + if (PyModule_AddObject(module, "spam", spam) < 0) { + Py_DECREF(module); + Py_DECREF(spam); + return NULL; + } + + -- C Function: int PyModule_AddIntConstant (PyObject *module, const + char *name, long value) + + Add an integer constant to `module' as `name'. This convenience + function can be used from the module’s initialization function. + Return ‘-1’ on error, ‘0’ on success. + + -- C Function: int PyModule_AddStringConstant (PyObject *module, const + char *name, const char *value) + + Add a string constant to `module' as `name'. This convenience + function can be used from the module’s initialization function. + The string `value' must be ‘NULL’-terminated. Return ‘-1’ on + error, ‘0’ on success. + + -- C Function: int PyModule_AddIntMacro (PyObject *module, macro) + + Add an int constant to `module'. The name and the value are taken + from `macro'. For example ‘PyModule_AddIntMacro(module, AF_INET)’ + adds the int constant `AF_INET' with the value of `AF_INET' to + `module'. Return ‘-1’ on error, ‘0’ on success. + + -- C Function: int PyModule_AddStringMacro (PyObject *module, macro) + + Add a string constant to `module'. + + +File: python.info, Node: Module lookup, Prev: Initializing C modules, Up: Module Objects + +7.8.6.8 Module lookup +..................... + +Single-phase initialization creates singleton modules that can be looked +up in the context of the current interpreter. This allows the module +object to be retrieved later with only a reference to the module +definition. + +These functions will not work on modules created using multi-phase +initialization, since multiple such modules can be created from a single +definition. + + -- C Function: PyObject* PyState_FindModule (PyModuleDef *def) + `Return value: Borrowed reference.' Returns the module object that + was created from `def' for the current interpreter. This method + requires that the module object has been attached to the + interpreter state with *note PyState_AddModule(): 3c50. beforehand. + In case the corresponding module object is not found or has not + been attached to the interpreter state yet, it returns ‘NULL’. + + -- C Function: int PyState_AddModule (PyObject *module, + PyModuleDef *def) + + Attaches the module object passed to the function to the + interpreter state. This allows the module object to be accessible + via *note PyState_FindModule(): 3c4f. + + Only effective on modules created using single-phase + initialization. + + Python calls ‘PyState_AddModule’ automatically after importing a + module, so it is unnecessary (but harmless) to call it from module + initialization code. An explicit call is needed only if the + module’s own init code subsequently calls ‘PyState_FindModule’. + The function is mainly intended for implementing alternative import + mechanisms (either by calling it directly, or by referring to its + implementation for details of the required state updates). + + Return 0 on success or -1 on failure. + + New in version 3.3. + + -- C Function: int PyState_RemoveModule (PyModuleDef *def) + + Removes the module object created from `def' from the interpreter + state. Return 0 on success or -1 on failure. + + New in version 3.3. + + +File: python.info, Node: Iterator Objects, Next: Descriptor Objects, Prev: Module Objects, Up: Other Objects + +7.8.6.9 Iterator Objects +........................ + +Python provides two general-purpose iterator objects. The first, a +sequence iterator, works with an arbitrary sequence supporting the *note +__getitem__(): 27c. method. The second works with a callable object and +a sentinel value, calling the callable for each item in the sequence, +and ending the iteration when the sentinel value is returned. + + -- C Variable: PyTypeObject PySeqIter_Type + + Type object for iterator objects returned by *note PySeqIter_New(): + 3c56. and the one-argument form of the *note iter(): d27. built-in + function for built-in sequence types. + + -- C Function: int PySeqIter_Check (op) + + Return true if the type of `op' is *note PySeqIter_Type: 3c55. + + -- C Function: PyObject* PySeqIter_New (PyObject *seq) + `Return value: New reference.' Return an iterator that works with + a general sequence object, `seq'. The iteration ends when the + sequence raises *note IndexError: e75. for the subscripting + operation. + + -- C Variable: PyTypeObject PyCallIter_Type + + Type object for iterator objects returned by *note + PyCallIter_New(): 3c59. and the two-argument form of the *note + iter(): d27. built-in function. + + -- C Function: int PyCallIter_Check (op) + + Return true if the type of `op' is *note PyCallIter_Type: 3c58. + + -- C Function: PyObject* PyCallIter_New (PyObject *callable, + PyObject *sentinel) + `Return value: New reference.' Return a new iterator. The first + parameter, `callable', can be any Python callable object that can + be called with no parameters; each call to it should return the + next item in the iteration. When `callable' returns a value equal + to `sentinel', the iteration will be terminated. + + +File: python.info, Node: Descriptor Objects, Next: Slice Objects, Prev: Iterator Objects, Up: Other Objects + +7.8.6.10 Descriptor Objects +........................... + +“Descriptors” are objects that describe some attribute of an object. +They are found in the dictionary of type objects. + + -- C Variable: PyTypeObject PyProperty_Type + + The type object for the built-in descriptor types. + + -- C Function: PyObject* PyDescr_NewGetSet (PyTypeObject *type, struct + PyGetSetDef *getset) + `Return value: New reference.' + + -- C Function: PyObject* PyDescr_NewMember (PyTypeObject *type, struct + PyMemberDef *meth) + `Return value: New reference.' + + -- C Function: PyObject* PyDescr_NewMethod (PyTypeObject *type, struct + PyMethodDef *meth) + `Return value: New reference.' + + -- C Function: PyObject* PyDescr_NewWrapper (PyTypeObject *type, struct + wrapperbase *wrapper, void *wrapped) + `Return value: New reference.' + + -- C Function: PyObject* PyDescr_NewClassMethod (PyTypeObject *type, + PyMethodDef *method) + `Return value: New reference.' + + -- C Function: int PyDescr_IsData (PyObject *descr) + + Return true if the descriptor objects `descr' describes a data + attribute, or false if it describes a method. `descr' must be a + descriptor object; there is no error checking. + + -- C Function: PyObject* PyWrapper_New (PyObject *, PyObject *) + `Return value: New reference.' + + +File: python.info, Node: Slice Objects, Next: Ellipsis Object, Prev: Descriptor Objects, Up: Other Objects + +7.8.6.11 Slice Objects +...................... + + -- C Variable: PyTypeObject PySlice_Type + + The type object for slice objects. This is the same as *note + slice: e04. in the Python layer. + + -- C Function: int PySlice_Check (PyObject *ob) + + Return true if `ob' is a slice object; `ob' must not be ‘NULL’. + + -- C Function: PyObject* PySlice_New (PyObject *start, PyObject *stop, + PyObject *step) + `Return value: New reference.' Return a new slice object with the + given values. The `start', `stop', and `step' parameters are used + as the values of the slice object attributes of the same names. + Any of the values may be ‘NULL’, in which case the ‘None’ will be + used for the corresponding attribute. Return ‘NULL’ if the new + object could not be allocated. + + -- C Function: int PySlice_GetIndices (PyObject *slice, + Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, + Py_ssize_t *step) + + Retrieve the start, stop and step indices from the slice object + `slice', assuming a sequence of length `length'. Treats indices + greater than `length' as errors. + + Returns ‘0’ on success and ‘-1’ on error with no exception set + (unless one of the indices was not *note None: 157. and failed to + be converted to an integer, in which case ‘-1’ is returned with an + exception set). + + You probably do not want to use this function. + + Changed in version 3.2: The parameter type for the `slice' + parameter was ‘PySliceObject*’ before. + + -- C Function: int PySlice_GetIndicesEx (PyObject *slice, + Py_ssize_t length, Py_ssize_t *start, Py_ssize_t *stop, + Py_ssize_t *step, Py_ssize_t *slicelength) + + Usable replacement for *note PySlice_GetIndices(): 3c6c. Retrieve + the start, stop, and step indices from the slice object `slice' + assuming a sequence of length `length', and store the length of the + slice in `slicelength'. Out of bounds indices are clipped in a + manner consistent with the handling of normal slices. + + Returns ‘0’ on success and ‘-1’ on error with exception set. + + Note: This function is considered not safe for resizable + sequences. Its invocation should be replaced by a combination + of *note PySlice_Unpack(): 42f. and *note + PySlice_AdjustIndices(): 430. where + + if (PySlice_GetIndicesEx(slice, length, &start, &stop, &step, &slicelength) < 0) { + // return error + } + + is replaced by + + if (PySlice_Unpack(slice, &start, &stop, &step) < 0) { + // return error + } + slicelength = PySlice_AdjustIndices(length, &start, &stop, step); + + Changed in version 3.2: The parameter type for the `slice' + parameter was ‘PySliceObject*’ before. + + Changed in version 3.6.1: If ‘Py_LIMITED_API’ is not set or set to + the value between ‘0x03050400’ and ‘0x03060000’ (not including) or + ‘0x03060100’ or higher ‘PySlice_GetIndicesEx()’ is implemented as a + macro using ‘PySlice_Unpack()’ and ‘PySlice_AdjustIndices()’. + Arguments `start', `stop' and `step' are evaluated more than once. + + Deprecated since version 3.6.1: If ‘Py_LIMITED_API’ is set to the + value less than ‘0x03050400’ or between ‘0x03060000’ and + ‘0x03060100’ (not including) ‘PySlice_GetIndicesEx()’ is a + deprecated function. + + -- C Function: int PySlice_Unpack (PyObject *slice, Py_ssize_t *start, + Py_ssize_t *stop, Py_ssize_t *step) + + Extract the start, stop and step data members from a slice object + as C integers. Silently reduce values larger than ‘PY_SSIZE_T_MAX’ + to ‘PY_SSIZE_T_MAX’, silently boost the start and stop values less + than ‘PY_SSIZE_T_MIN’ to ‘PY_SSIZE_T_MIN’, and silently boost the + step values less than ‘-PY_SSIZE_T_MAX’ to ‘-PY_SSIZE_T_MAX’. + + Return ‘-1’ on error, ‘0’ on success. + + New in version 3.6.1. + + -- C Function: Py_ssize_t PySlice_AdjustIndices (Py_ssize_t length, + Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t step) + + Adjust start/end slice indices assuming a sequence of the specified + length. Out of bounds indices are clipped in a manner consistent + with the handling of normal slices. + + Return the length of the slice. Always successful. Doesn’t call + Python code. + + New in version 3.6.1. + + +File: python.info, Node: Ellipsis Object, Next: MemoryView objects, Prev: Slice Objects, Up: Other Objects + +7.8.6.12 Ellipsis Object +........................ + + -- C Variable: PyObject *Py_Ellipsis + + The Python ‘Ellipsis’ object. This object has no methods. It + needs to be treated just like any other object with respect to + reference counts. Like *note Py_None: 3844. it is a singleton + object. + + +File: python.info, Node: MemoryView objects, Next: Weak Reference Objects<2>, Prev: Ellipsis Object, Up: Other Objects + +7.8.6.13 MemoryView objects +........................... + +A *note memoryview: 25c. object exposes the C level *note buffer +interface: 1291. as a Python object which can then be passed around like +any other object. + + -- C Function: PyObject *PyMemoryView_FromObject (PyObject *obj) + `Return value: New reference.' Create a memoryview object from an + object that provides the buffer interface. If `obj' supports + writable buffer exports, the memoryview object will be read/write, + otherwise it may be either read-only or read/write at the + discretion of the exporter. + + -- C Function: PyObject *PyMemoryView_FromMemory (char *mem, + Py_ssize_t size, int flags) + `Return value: New reference.' Create a memoryview object using + `mem' as the underlying buffer. `flags' can be one of ‘PyBUF_READ’ + or ‘PyBUF_WRITE’. + + New in version 3.3. + + -- C Function: PyObject *PyMemoryView_FromBuffer (Py_buffer *view) + `Return value: New reference.' Create a memoryview object wrapping + the given buffer structure `view'. For simple byte buffers, *note + PyMemoryView_FromMemory(): ac9. is the preferred function. + + -- C Function: PyObject *PyMemoryView_GetContiguous (PyObject *obj, + int buffertype, char order) + `Return value: New reference.' Create a memoryview object to a + *note contiguous: 1360. chunk of memory (in either ‘C’ or ‘F’ortran + `order') from an object that defines the buffer interface. If + memory is contiguous, the memoryview object points to the original + memory. Otherwise, a copy is made and the memoryview points to a + new bytes object. + + -- C Function: int PyMemoryView_Check (PyObject *obj) + + Return true if the object `obj' is a memoryview object. It is not + currently allowed to create subclasses of *note memoryview: 25c. + + -- C Function: Py_buffer *PyMemoryView_GET_BUFFER (PyObject *mview) + + Return a pointer to the memoryview’s private copy of the exporter’s + buffer. `mview' `must' be a memoryview instance; this macro + doesn’t check its type, you must do it yourself or you will risk + crashes. + + -- C Function: Py_buffer *PyMemoryView_GET_BASE (PyObject *mview) + + Return either a pointer to the exporting object that the memoryview + is based on or ‘NULL’ if the memoryview has been created by one of + the functions *note PyMemoryView_FromMemory(): ac9. or *note + PyMemoryView_FromBuffer(): 3a75. `mview' `must' be a memoryview + instance. + + +File: python.info, Node: Weak Reference Objects<2>, Next: Capsules<2>, Prev: MemoryView objects, Up: Other Objects + +7.8.6.14 Weak Reference Objects +............................... + +Python supports `weak references' as first-class objects. There are two +specific object types which directly implement weak references. The +first is a simple reference object, and the second acts as a proxy for +the original object as much as it can. + + -- C Function: int PyWeakref_Check (ob) + + Return true if `ob' is either a reference or proxy object. + + -- C Function: int PyWeakref_CheckRef (ob) + + Return true if `ob' is a reference object. + + -- C Function: int PyWeakref_CheckProxy (ob) + + Return true if `ob' is a proxy object. + + -- C Function: PyObject* PyWeakref_NewRef (PyObject *ob, + PyObject *callback) + `Return value: New reference.' Return a weak reference object for + the object `ob'. This will always return a new reference, but is + not guaranteed to create a new object; an existing reference object + may be returned. The second parameter, `callback', can be a + callable object that receives notification when `ob' is garbage + collected; it should accept a single parameter, which will be the + weak reference object itself. `callback' may also be ‘None’ or + ‘NULL’. If `ob' is not a weakly-referencable object, or if + `callback' is not callable, ‘None’, or ‘NULL’, this will return + ‘NULL’ and raise *note TypeError: 192. + + -- C Function: PyObject* PyWeakref_NewProxy (PyObject *ob, + PyObject *callback) + `Return value: New reference.' Return a weak reference proxy + object for the object `ob'. This will always return a new + reference, but is not guaranteed to create a new object; an + existing proxy object may be returned. The second parameter, + `callback', can be a callable object that receives notification + when `ob' is garbage collected; it should accept a single + parameter, which will be the weak reference object itself. + `callback' may also be ‘None’ or ‘NULL’. If `ob' is not a + weakly-referencable object, or if `callback' is not callable, + ‘None’, or ‘NULL’, this will return ‘NULL’ and raise *note + TypeError: 192. + + -- C Function: PyObject* PyWeakref_GetObject (PyObject *ref) + `Return value: Borrowed reference.' Return the referenced object + from a weak reference, `ref'. If the referent is no longer live, + returns ‘Py_None’. + + Note: This function returns a `borrowed reference' to the + referenced object. This means that you should always call + *note Py_INCREF(): 265. on the object except if you know that + it cannot be destroyed while you are still using it. + + -- C Function: PyObject* PyWeakref_GET_OBJECT (PyObject *ref) + `Return value: Borrowed reference.' Similar to *note + PyWeakref_GetObject(): 3c7e, but implemented as a macro that does + no error checking. + + +File: python.info, Node: Capsules<2>, Next: Generator Objects, Prev: Weak Reference Objects<2>, Up: Other Objects + +7.8.6.15 Capsules +................. + +Refer to *note Providing a C API for an Extension Module: cf1. for more +information on using these objects. + +New in version 3.1. + + -- C Type: PyCapsule + + This subtype of *note PyObject: 4ba. represents an opaque value, + useful for C extension modules who need to pass an opaque value (as + a ‘void*’ pointer) through Python code to other C code. It is + often used to make a C function pointer defined in one module + available to other modules, so the regular import mechanism can be + used to access C APIs defined in dynamically loaded modules. + + -- C Type: PyCapsule_Destructor + + The type of a destructor callback for a capsule. Defined as: + + typedef void (*PyCapsule_Destructor)(PyObject *); + + See *note PyCapsule_New(): 386c. for the semantics of + PyCapsule_Destructor callbacks. + + -- C Function: int PyCapsule_CheckExact (PyObject *p) + + Return true if its argument is a *note PyCapsule: c07. + + -- C Function: PyObject* PyCapsule_New (void *pointer, const + char *name, PyCapsule_Destructor destructor) + `Return value: New reference.' Create a *note PyCapsule: c07. + encapsulating the `pointer'. The `pointer' argument may not be + ‘NULL’. + + On failure, set an exception and return ‘NULL’. + + The `name' string may either be ‘NULL’ or a pointer to a valid C + string. If non-‘NULL’, this string must outlive the capsule. + (Though it is permitted to free it inside the `destructor'.) + + If the `destructor' argument is not ‘NULL’, it will be called with + the capsule as its argument when it is destroyed. + + If this capsule will be stored as an attribute of a module, the + `name' should be specified as ‘modulename.attributename’. This + will enable other modules to import the capsule using *note + PyCapsule_Import(): 386d. + + -- C Function: void* PyCapsule_GetPointer (PyObject *capsule, const + char *name) + + Retrieve the `pointer' stored in the capsule. On failure, set an + exception and return ‘NULL’. + + The `name' parameter must compare exactly to the name stored in the + capsule. If the name stored in the capsule is ‘NULL’, the `name' + passed in must also be ‘NULL’. Python uses the C function + ‘strcmp()’ to compare capsule names. + + -- C Function: PyCapsule_Destructor PyCapsule_GetDestructor + (PyObject *capsule) + + Return the current destructor stored in the capsule. On failure, + set an exception and return ‘NULL’. + + It is legal for a capsule to have a ‘NULL’ destructor. This makes + a ‘NULL’ return code somewhat ambiguous; use *note + PyCapsule_IsValid(): cf0. or *note PyErr_Occurred(): 383f. to + disambiguate. + + -- C Function: void* PyCapsule_GetContext (PyObject *capsule) + + Return the current context stored in the capsule. On failure, set + an exception and return ‘NULL’. + + It is legal for a capsule to have a ‘NULL’ context. This makes a + ‘NULL’ return code somewhat ambiguous; use *note + PyCapsule_IsValid(): cf0. or *note PyErr_Occurred(): 383f. to + disambiguate. + + -- C Function: const char* PyCapsule_GetName (PyObject *capsule) + + Return the current name stored in the capsule. On failure, set an + exception and return ‘NULL’. + + It is legal for a capsule to have a ‘NULL’ name. This makes a + ‘NULL’ return code somewhat ambiguous; use *note + PyCapsule_IsValid(): cf0. or *note PyErr_Occurred(): 383f. to + disambiguate. + + -- C Function: void* PyCapsule_Import (const char *name, int no_block) + + Import a pointer to a C object from a capsule attribute in a + module. The `name' parameter should specify the full name to the + attribute, as in ‘module.attribute’. The `name' stored in the + capsule must match this string exactly. If `no_block' is true, + import the module without blocking (using *note + PyImport_ImportModuleNoBlock(): 9c1.). If `no_block' is false, + import the module conventionally (using *note + PyImport_ImportModule(): c73.). + + Return the capsule’s internal `pointer' on success. On failure, + set an exception and return ‘NULL’. + + -- C Function: int PyCapsule_IsValid (PyObject *capsule, const + char *name) + + Determines whether or not `capsule' is a valid capsule. A valid + capsule is non-‘NULL’, passes *note PyCapsule_CheckExact(): 3c83, + has a non-‘NULL’ pointer stored in it, and its internal name + matches the `name' parameter. (See *note PyCapsule_GetPointer(): + 3c84. for information on how capsule names are compared.) + + In other words, if *note PyCapsule_IsValid(): cf0. returns a true + value, calls to any of the accessors (any function starting with + ‘PyCapsule_Get()’) are guaranteed to succeed. + + Return a nonzero value if the object is valid and matches the name + passed in. Return ‘0’ otherwise. This function will not fail. + + -- C Function: int PyCapsule_SetContext (PyObject *capsule, + void *context) + + Set the context pointer inside `capsule' to `context'. + + Return ‘0’ on success. Return nonzero and set an exception on + failure. + + -- C Function: int PyCapsule_SetDestructor (PyObject *capsule, + PyCapsule_Destructor destructor) + + Set the destructor inside `capsule' to `destructor'. + + Return ‘0’ on success. Return nonzero and set an exception on + failure. + + -- C Function: int PyCapsule_SetName (PyObject *capsule, const + char *name) + + Set the name inside `capsule' to `name'. If non-‘NULL’, the name + must outlive the capsule. If the previous `name' stored in the + capsule was not ‘NULL’, no attempt is made to free it. + + Return ‘0’ on success. Return nonzero and set an exception on + failure. + + -- C Function: int PyCapsule_SetPointer (PyObject *capsule, + void *pointer) + + Set the void pointer inside `capsule' to `pointer'. The pointer + may not be ‘NULL’. + + Return ‘0’ on success. Return nonzero and set an exception on + failure. + + +File: python.info, Node: Generator Objects, Next: Coroutine Objects<2>, Prev: Capsules<2>, Up: Other Objects + +7.8.6.16 Generator Objects +.......................... + +Generator objects are what Python uses to implement generator iterators. +They are normally created by iterating over a function that yields +values, rather than explicitly calling *note PyGen_New(): 3c8f. or *note +PyGen_NewWithQualName(): 3c90. + + -- C Type: PyGenObject + + The C structure used for generator objects. + + -- C Variable: PyTypeObject PyGen_Type + + The type object corresponding to generator objects. + + -- C Function: int PyGen_Check (PyObject *ob) + + Return true if `ob' is a generator object; `ob' must not be ‘NULL’. + + -- C Function: int PyGen_CheckExact (PyObject *ob) + + Return true if `ob'’s type is *note PyGen_Type: 3c92.; `ob' must + not be ‘NULL’. + + -- C Function: PyObject* PyGen_New (PyFrameObject *frame) + `Return value: New reference.' Create and return a new generator + object based on the `frame' object. A reference to `frame' is + stolen by this function. The argument must not be ‘NULL’. + + -- C Function: PyObject* PyGen_NewWithQualName (PyFrameObject *frame, + PyObject *name, PyObject *qualname) + `Return value: New reference.' Create and return a new generator + object based on the `frame' object, with ‘__name__’ and + ‘__qualname__’ set to `name' and `qualname'. A reference to + `frame' is stolen by this function. The `frame' argument must not + be ‘NULL’. + + +File: python.info, Node: Coroutine Objects<2>, Next: Context Variables Objects, Prev: Generator Objects, Up: Other Objects + +7.8.6.17 Coroutine Objects +.......................... + +New in version 3.5. + +Coroutine objects are what functions declared with an ‘async’ keyword +return. + + -- C Type: PyCoroObject + + The C structure used for coroutine objects. + + -- C Variable: PyTypeObject PyCoro_Type + + The type object corresponding to coroutine objects. + + -- C Function: int PyCoro_CheckExact (PyObject *ob) + + Return true if `ob'’s type is *note PyCoro_Type: 3c98.; `ob' must + not be ‘NULL’. + + -- C Function: PyObject* PyCoro_New (PyFrameObject *frame, + PyObject *name, PyObject *qualname) + `Return value: New reference.' Create and return a new coroutine + object based on the `frame' object, with ‘__name__’ and + ‘__qualname__’ set to `name' and `qualname'. A reference to + `frame' is stolen by this function. The `frame' argument must not + be ‘NULL’. + + +File: python.info, Node: Context Variables Objects, Next: DateTime Objects<2>, Prev: Coroutine Objects<2>, Up: Other Objects + +7.8.6.18 Context Variables Objects +.................................. + + Note Changed in version 3.7.1:: In Python 3.7.1 the signatures of + all context variables C APIs were `changed' to use *note PyObject: + 4ba. pointers instead of *note PyContext: 3c9d, *note PyContextVar: + 3c9e, and *note PyContextToken: 3c9f, e.g.: + + // in 3.7.0: + PyContext *PyContext_New(void); + + // in 3.7.1+: + PyObject *PyContext_New(void); + + See bpo-34762(1) for more details. + +New in version 3.7. + +This section details the public C API for the *note contextvars: 25. +module. + + -- C Type: PyContext + + The C structure used to represent a *note contextvars.Context: + 21ab. object. + + -- C Type: PyContextVar + + The C structure used to represent a *note contextvars.ContextVar: + 21a9. object. + + -- C Type: PyContextToken + + The C structure used to represent a ‘contextvars.Token’ object. + + -- C Variable: PyTypeObject PyContext_Type + + The type object representing the `context' type. + + -- C Variable: PyTypeObject PyContextVar_Type + + The type object representing the `context variable' type. + + -- C Variable: PyTypeObject PyContextToken_Type + + The type object representing the `context variable token' type. + +Type-check macros: + + -- C Function: int PyContext_CheckExact (PyObject *o) + + Return true if `o' is of type *note PyContext_Type: 3ca0. `o' must + not be ‘NULL’. This function always succeeds. + + -- C Function: int PyContextVar_CheckExact (PyObject *o) + + Return true if `o' is of type *note PyContextVar_Type: 3ca1. `o' + must not be ‘NULL’. This function always succeeds. + + -- C Function: int PyContextToken_CheckExact (PyObject *o) + + Return true if `o' is of type *note PyContextToken_Type: 3ca2. `o' + must not be ‘NULL’. This function always succeeds. + +Context object management functions: + + -- C Function: PyObject *PyContext_New (void) + `Return value: New reference.' Create a new empty context object. + Returns ‘NULL’ if an error has occurred. + + -- C Function: PyObject *PyContext_Copy (PyObject *ctx) + `Return value: New reference.' Create a shallow copy of the passed + `ctx' context object. Returns ‘NULL’ if an error has occurred. + + -- C Function: PyObject *PyContext_CopyCurrent (void) + `Return value: New reference.' Create a shallow copy of the + current thread context. Returns ‘NULL’ if an error has occurred. + + -- C Function: int PyContext_Enter (PyObject *ctx) + + Set `ctx' as the current context for the current thread. Returns + ‘0’ on success, and ‘-1’ on error. + + -- C Function: int PyContext_Exit (PyObject *ctx) + + Deactivate the `ctx' context and restore the previous context as + the current context for the current thread. Returns ‘0’ on + success, and ‘-1’ on error. + + -- C Function: int PyContext_ClearFreeList () + + Clear the context variable free list. Return the total number of + freed items. This function always succeeds. + +Context variable functions: + + -- C Function: PyObject *PyContextVar_New (const char *name, + PyObject *def) + `Return value: New reference.' Create a new ‘ContextVar’ object. + The `name' parameter is used for introspection and debug purposes. + The `def' parameter may optionally specify the default value for + the context variable. If an error has occurred, this function + returns ‘NULL’. + + -- C Function: int PyContextVar_Get (PyObject *var, + PyObject *default_value, PyObject **value) + + Get the value of a context variable. Returns ‘-1’ if an error has + occurred during lookup, and ‘0’ if no error occurred, whether or + not a value was found. + + If the context variable was found, `value' will be a pointer to it. + If the context variable was `not' found, `value' will point to: + + - `default_value', if not ‘NULL’; + + - the default value of `var', if not ‘NULL’; + + - ‘NULL’ + + If the value was found, the function will create a new reference to + it. + + -- C Function: PyObject *PyContextVar_Set (PyObject *var, + PyObject *value) + `Return value: New reference.' Set the value of `var' to `value' + in the current context. Returns a pointer to a *note PyObject: + 4ba. object, or ‘NULL’ if an error has occurred. + + -- C Function: int PyContextVar_Reset (PyObject *var, PyObject *token) + + Reset the state of the `var' context variable to that it was in + before *note PyContextVar_Set(): 3cae. that returned the `token' + was called. This function returns ‘0’ on success and ‘-1’ on + error. + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/issue34762 + + +File: python.info, Node: DateTime Objects<2>, Prev: Context Variables Objects, Up: Other Objects + +7.8.6.19 DateTime Objects +......................... + +Various date and time objects are supplied by the *note datetime: 31. +module. Before using any of these functions, the header file +‘datetime.h’ must be included in your source (note that this is not +included by ‘Python.h’), and the macro ‘PyDateTime_IMPORT’ must be +invoked, usually as part of the module initialisation function. The +macro puts a pointer to a C structure into a static variable, +‘PyDateTimeAPI’, that is used by the following macros. + +Macro for access to the UTC singleton: + + -- C Variable: PyObject* PyDateTime_TimeZone_UTC + + Returns the time zone singleton representing UTC, the same object + as *note datetime.timezone.utc: 1599. + + New in version 3.7. + +Type-check macros: + + -- C Function: int PyDate_Check (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_DateType’ or a subtype + of ‘PyDateTime_DateType’. `ob' must not be ‘NULL’. + + -- C Function: int PyDate_CheckExact (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_DateType’. `ob' must + not be ‘NULL’. + + -- C Function: int PyDateTime_Check (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_DateTimeType’ or a + subtype of ‘PyDateTime_DateTimeType’. `ob' must not be ‘NULL’. + + -- C Function: int PyDateTime_CheckExact (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_DateTimeType’. `ob' + must not be ‘NULL’. + + -- C Function: int PyTime_Check (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_TimeType’ or a subtype + of ‘PyDateTime_TimeType’. `ob' must not be ‘NULL’. + + -- C Function: int PyTime_CheckExact (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_TimeType’. `ob' must + not be ‘NULL’. + + -- C Function: int PyDelta_Check (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_DeltaType’ or a subtype + of ‘PyDateTime_DeltaType’. `ob' must not be ‘NULL’. + + -- C Function: int PyDelta_CheckExact (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_DeltaType’. `ob' must + not be ‘NULL’. + + -- C Function: int PyTZInfo_Check (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_TZInfoType’ or a subtype + of ‘PyDateTime_TZInfoType’. `ob' must not be ‘NULL’. + + -- C Function: int PyTZInfo_CheckExact (PyObject *ob) + + Return true if `ob' is of type ‘PyDateTime_TZInfoType’. `ob' must + not be ‘NULL’. + +Macros to create objects: + + -- C Function: PyObject* PyDate_FromDate (int year, int month, int day) + `Return value: New reference.' Return a *note datetime.date: 193. + object with the specified year, month and day. + + -- C Function: PyObject* PyDateTime_FromDateAndTime (int year, + int month, int day, int hour, int minute, int second, + int usecond) + `Return value: New reference.' Return a *note datetime.datetime: + 194. object with the specified year, month, day, hour, minute, + second and microsecond. + + -- C Function: PyObject* PyDateTime_FromDateAndTimeAndFold (int year, + int month, int day, int hour, int minute, int second, + int usecond, int fold) + `Return value: New reference.' Return a *note datetime.datetime: + 194. object with the specified year, month, day, hour, minute, + second, microsecond and fold. + + New in version 3.6. + + -- C Function: PyObject* PyTime_FromTime (int hour, int minute, + int second, int usecond) + `Return value: New reference.' Return a *note datetime.time: 4f3. + object with the specified hour, minute, second and microsecond. + + -- C Function: PyObject* PyTime_FromTimeAndFold (int hour, int minute, + int second, int usecond, int fold) + `Return value: New reference.' Return a *note datetime.time: 4f3. + object with the specified hour, minute, second, microsecond and + fold. + + New in version 3.6. + + -- C Function: PyObject* PyDelta_FromDSU (int days, int seconds, + int useconds) + `Return value: New reference.' Return a *note datetime.timedelta: + 195. object representing the given number of days, seconds and + microseconds. Normalization is performed so that the resulting + number of microseconds and seconds lie in the ranges documented for + *note datetime.timedelta: 195. objects. + + -- C Function: PyObject* PyTimeZone_FromOffset + (PyDateTime_DeltaType* offset) + `Return value: New reference.' Return a *note datetime.timezone: + a01. object with an unnamed fixed offset represented by the + `offset' argument. + + New in version 3.7. + + -- C Function: PyObject* PyTimeZone_FromOffsetAndName + (PyDateTime_DeltaType* offset, PyUnicode* name) + `Return value: New reference.' Return a *note datetime.timezone: + a01. object with a fixed offset represented by the `offset' + argument and with tzname `name'. + + New in version 3.7. + +Macros to extract fields from date objects. The argument must be an +instance of ‘PyDateTime_Date’, including subclasses (such as +‘PyDateTime_DateTime’). The argument must not be ‘NULL’, and the type +is not checked: + + -- C Function: int PyDateTime_GET_YEAR (PyDateTime_Date *o) + + Return the year, as a positive int. + + -- C Function: int PyDateTime_GET_MONTH (PyDateTime_Date *o) + + Return the month, as an int from 1 through 12. + + -- C Function: int PyDateTime_GET_DAY (PyDateTime_Date *o) + + Return the day, as an int from 1 through 31. + +Macros to extract fields from datetime objects. The argument must be an +instance of ‘PyDateTime_DateTime’, including subclasses. The argument +must not be ‘NULL’, and the type is not checked: + + -- C Function: int PyDateTime_DATE_GET_HOUR (PyDateTime_DateTime *o) + + Return the hour, as an int from 0 through 23. + + -- C Function: int PyDateTime_DATE_GET_MINUTE (PyDateTime_DateTime *o) + + Return the minute, as an int from 0 through 59. + + -- C Function: int PyDateTime_DATE_GET_SECOND (PyDateTime_DateTime *o) + + Return the second, as an int from 0 through 59. + + -- C Function: int PyDateTime_DATE_GET_MICROSECOND + (PyDateTime_DateTime *o) + + Return the microsecond, as an int from 0 through 999999. + +Macros to extract fields from time objects. The argument must be an +instance of ‘PyDateTime_Time’, including subclasses. The argument must +not be ‘NULL’, and the type is not checked: + + -- C Function: int PyDateTime_TIME_GET_HOUR (PyDateTime_Time *o) + + Return the hour, as an int from 0 through 23. + + -- C Function: int PyDateTime_TIME_GET_MINUTE (PyDateTime_Time *o) + + Return the minute, as an int from 0 through 59. + + -- C Function: int PyDateTime_TIME_GET_SECOND (PyDateTime_Time *o) + + Return the second, as an int from 0 through 59. + + -- C Function: int PyDateTime_TIME_GET_MICROSECOND (PyDateTime_Time *o) + + Return the microsecond, as an int from 0 through 999999. + +Macros to extract fields from time delta objects. The argument must be +an instance of ‘PyDateTime_Delta’, including subclasses. The argument +must not be ‘NULL’, and the type is not checked: + + -- C Function: int PyDateTime_DELTA_GET_DAYS (PyDateTime_Delta *o) + + Return the number of days, as an int from -999999999 to 999999999. + + New in version 3.3. + + -- C Function: int PyDateTime_DELTA_GET_SECONDS (PyDateTime_Delta *o) + + Return the number of seconds, as an int from 0 through 86399. + + New in version 3.3. + + -- C Function: int PyDateTime_DELTA_GET_MICROSECONDS + (PyDateTime_Delta *o) + + Return the number of microseconds, as an int from 0 through 999999. + + New in version 3.3. + +Macros for the convenience of modules implementing the DB API: + + -- C Function: PyObject* PyDateTime_FromTimestamp (PyObject *args) + `Return value: New reference.' Create and return a new *note + datetime.datetime: 194. object given an argument tuple suitable for + passing to *note datetime.datetime.fromtimestamp(): 156a. + + -- C Function: PyObject* PyDate_FromTimestamp (PyObject *args) + `Return value: New reference.' Create and return a new *note + datetime.date: 193. object given an argument tuple suitable for + passing to *note datetime.date.fromtimestamp(): 1552. + + +File: python.info, Node: Initialization Finalization and Threads, Next: Python Initialization Configuration, Prev: Concrete Objects Layer, Up: Python/C API Reference Manual + +7.9 Initialization, Finalization, and Threads +============================================= + +See also *note Python Initialization Configuration: 17d. + +* Menu: + +* Before Python Initialization:: +* Global configuration variables:: +* Initializing and finalizing the interpreter:: +* Process-wide parameters:: +* Thread State and the Global Interpreter Lock:: +* Sub-interpreter support:: +* Asynchronous Notifications:: +* Profiling and Tracing:: +* Advanced Debugger Support:: +* Thread Local Storage Support:: + + +File: python.info, Node: Before Python Initialization, Next: Global configuration variables, Up: Initialization Finalization and Threads + +7.9.1 Before Python Initialization +---------------------------------- + +In an application embedding Python, the *note Py_Initialize(): 439. +function must be called before using any other Python/C API functions; +with the exception of a few functions and the *note global configuration +variables: 3cd7. + +The following functions can be safely called before Python is +initialized: + + * Configuration functions: + + * *note PyImport_AppendInittab(): 3848. + + * *note PyImport_ExtendInittab(): 39b7. + + * ‘PyInitFrozenExtensions()’ + + * *note PyMem_SetAllocator(): 3cd8. + + * *note PyMem_SetupDebugHooks(): 50a. + + * *note PyObject_SetArenaAllocator(): 3cd9. + + * *note Py_SetPath(): 273. + + * *note Py_SetProgramName(): 1031. + + * *note Py_SetPythonHome(): 3cda. + + * *note Py_SetStandardStreamEncoding(): 925. + + * *note PySys_AddWarnOption(): 4b3. + + * *note PySys_AddXOption(): 399d. + + * *note PySys_ResetWarnOptions(): 3996. + + * Informative functions: + + * *note Py_IsInitialized(): 3901. + + * *note PyMem_GetAllocator(): 3cdb. + + * *note PyObject_GetArenaAllocator(): 3cdc. + + * *note Py_GetBuildInfo(): d9a. + + * *note Py_GetCompiler(): 3cdd. + + * *note Py_GetCopyright(): 3cde. + + * *note Py_GetPlatform(): 3cdf. + + * *note Py_GetVersion(): 3ce0. + + * Utilities: + + * *note Py_DecodeLocale(): 43c. + + * Memory allocators: + + * *note PyMem_RawMalloc(): 96d. + + * *note PyMem_RawRealloc(): 96e. + + * *note PyMem_RawCalloc(): 7a5. + + * *note PyMem_RawFree(): 398f. + + Note: The following functions `should not be called' before *note + Py_Initialize(): 439.: *note Py_EncodeLocale(): 43d, *note + Py_GetPath(): 38fe, *note Py_GetPrefix(): 38ff, *note + Py_GetExecPrefix(): 3900, *note Py_GetProgramFullPath(): 275, *note + Py_GetPythonHome(): 3ce1, *note Py_GetProgramName(): 276. and *note + PyEval_InitThreads(): c12. + + +File: python.info, Node: Global configuration variables, Next: Initializing and finalizing the interpreter, Prev: Before Python Initialization, Up: Initialization Finalization and Threads + +7.9.2 Global configuration variables +------------------------------------ + +Python has variables for the global configuration to control different +features and options. By default, these flags are controlled by *note +command line options: fd0. + +When a flag is set by an option, the value of the flag is the number of +times that the option was set. For example, ‘-b’ sets *note +Py_BytesWarningFlag: 4b4. to 1 and ‘-bb’ sets *note Py_BytesWarningFlag: +4b4. to 2. + + -- C Variable: int Py_BytesWarningFlag + + Issue a warning when comparing *note bytes: 331. or *note + bytearray: 332. with *note str: 330. or *note bytes: 331. with + *note int: 184. Issue an error if greater or equal to ‘2’. + + Set by the *note -b: 415. option. + + -- C Variable: int Py_DebugFlag + + Turn on parser debugging output (for expert only, depending on + compilation options). + + Set by the *note -d: fda. option and the *note PYTHONDEBUG: fdb. + environment variable. + + -- C Variable: int Py_DontWriteBytecodeFlag + + If set to non-zero, Python won’t try to write ‘.pyc’ files on the + import of source modules. + + Set by the *note -B: d38. option and the *note + PYTHONDONTWRITEBYTECODE: d39. environment variable. + + -- C Variable: int Py_FrozenFlag + + Suppress error messages when calculating the module search path in + *note Py_GetPath(): 38fe. + + Private flag used by ‘_freeze_importlib’ and ‘frozenmain’ programs. + + -- C Variable: int Py_HashRandomizationFlag + + Set to ‘1’ if the *note PYTHONHASHSEED: 9c5. environment variable + is set to a non-empty string. + + If the flag is non-zero, read the *note PYTHONHASHSEED: 9c5. + environment variable to initialize the secret hash seed. + + -- C Variable: int Py_IgnoreEnvironmentFlag + + Ignore all ‘PYTHON*’ environment variables, e.g. *note PYTHONPATH: + 953. and *note PYTHONHOME: 4d6, that might be set. + + Set by the *note -E: fdc. and *note -I: fd2. options. + + -- C Variable: int Py_InspectFlag + + When a script is passed as first argument or the *note -c: e9e. + option is used, enter interactive mode after executing the script + or the command, even when *note sys.stdin: 30d. does not appear to + be a terminal. + + Set by the *note -i: e1f. option and the *note PYTHONINSPECT: e1e. + environment variable. + + -- C Variable: int Py_InteractiveFlag + + Set by the *note -i: e1f. option. + + -- C Variable: int Py_IsolatedFlag + + Run Python in isolated mode. In isolated mode *note sys.path: 488. + contains neither the script’s directory nor the user’s + site-packages directory. + + Set by the *note -I: fd2. option. + + New in version 3.4. + + -- C Variable: int Py_LegacyWindowsFSEncodingFlag + + If the flag is non-zero, use the ‘mbcs’ encoding instead of the + UTF-8 encoding for the filesystem encoding. + + Set to ‘1’ if the *note PYTHONLEGACYWINDOWSFSENCODING: 4f7. + environment variable is set to a non-empty string. + + See PEP 529(1) for more details. + + *note Availability: ffb.: Windows. + + -- C Variable: int Py_LegacyWindowsStdioFlag + + If the flag is non-zero, use *note io.FileIO: ca4. instead of + ‘WindowsConsoleIO’ for *note sys: fd. standard streams. + + Set to ‘1’ if the *note PYTHONLEGACYWINDOWSSTDIO: 4fa. environment + variable is set to a non-empty string. + + See PEP 528(2) for more details. + + *note Availability: ffb.: Windows. + + -- C Variable: int Py_NoSiteFlag + + Disable the import of the module *note site: eb. and the + site-dependent manipulations of *note sys.path: 488. that it + entails. Also disable these manipulations if *note site: eb. is + explicitly imported later (call *note site.main(): fe2. if you want + them to be triggered). + + Set by the *note -S: b24. option. + + -- C Variable: int Py_NoUserSiteDirectory + + Don’t add the *note user site-packages directory: fe1. to *note + sys.path: 488. + + Set by the *note -s: d15. and *note -I: fd2. options, and the *note + PYTHONNOUSERSITE: d16. environment variable. + + -- C Variable: int Py_OptimizeFlag + + Set by the *note -O: 68b. option and the *note PYTHONOPTIMIZE: fde. + environment variable. + + -- C Variable: int Py_QuietFlag + + Don’t display the copyright and version messages even in + interactive mode. + + Set by the *note -q: fdf. option. + + New in version 3.2. + + -- C Variable: int Py_UnbufferedStdioFlag + + Force the stdout and stderr streams to be unbuffered. + + Set by the *note -u: fe3. option and the *note PYTHONUNBUFFERED: + fe4. environment variable. + + -- C Variable: int Py_VerboseFlag + + Print a message each time a module is initialized, showing the + place (filename or built-in module) from which it is loaded. If + greater or equal to ‘2’, print a message for each file that is + checked for when searching for a module. Also provides information + on module cleanup at exit. + + Set by the *note -v: d88. option and the *note PYTHONVERBOSE: fe5. + environment variable. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0529 + + (2) https://www.python.org/dev/peps/pep-0528 + + +File: python.info, Node: Initializing and finalizing the interpreter, Next: Process-wide parameters, Prev: Global configuration variables, Up: Initialization Finalization and Threads + +7.9.3 Initializing and finalizing the interpreter +------------------------------------------------- + + -- C Function: void Py_Initialize () + + Initialize the Python interpreter. In an application embedding + Python, this should be called before using any other Python/C API + functions; see *note Before Python Initialization: 43a. for the few + exceptions. + + This initializes the table of loaded modules (‘sys.modules’), and + creates the fundamental modules *note builtins: 13, *note __main__: + 1. and *note sys: fd. It also initializes the module search path + (‘sys.path’). It does not set ‘sys.argv’; use *note + PySys_SetArgvEx(): bfa. for that. This is a no-op when called for + a second time (without calling *note Py_FinalizeEx(): 5c9. first). + There is no return value; it is a fatal error if the initialization + fails. + + Note: On Windows, changes the console mode from ‘O_TEXT’ to + ‘O_BINARY’, which will also affect non-Python uses of the + console using the C Runtime. + + -- C Function: void Py_InitializeEx (int initsigs) + + This function works like *note Py_Initialize(): 439. if `initsigs' + is ‘1’. If `initsigs' is ‘0’, it skips initialization registration + of signal handlers, which might be useful when Python is embedded. + + -- C Function: int Py_IsInitialized () + + Return true (nonzero) when the Python interpreter has been + initialized, false (zero) if not. After *note Py_FinalizeEx(): + 5c9. is called, this returns false until *note Py_Initialize(): + 439. is called again. + + -- C Function: int Py_FinalizeEx () + + Undo all initializations made by *note Py_Initialize(): 439. and + subsequent use of Python/C API functions, and destroy all + sub-interpreters (see *note Py_NewInterpreter(): 3cf1. below) that + were created and not yet destroyed since the last call to *note + Py_Initialize(): 439. Ideally, this frees all memory allocated by + the Python interpreter. This is a no-op when called for a second + time (without calling *note Py_Initialize(): 439. again first). + Normally the return value is ‘0’. If there were errors during + finalization (flushing buffered data), ‘-1’ is returned. + + This function is provided for a number of reasons. An embedding + application might want to restart Python without having to restart + the application itself. An application that has loaded the Python + interpreter from a dynamically loadable library (or DLL) might want + to free all memory allocated by Python before unloading the DLL. + During a hunt for memory leaks in an application a developer might + want to free all memory allocated by Python before exiting from the + application. + + `Bugs and caveats:' The destruction of modules and objects in + modules is done in random order; this may cause destructors (*note + __del__(): 91f. methods) to fail when they depend on other objects + (even functions) or modules. Dynamically loaded extension modules + loaded by Python are not unloaded. Small amounts of memory + allocated by the Python interpreter may not be freed (if you find a + leak, please report it). Memory tied up in circular references + between objects is not freed. Some memory allocated by extension + modules may not be freed. Some extensions may not work properly if + their initialization routine is called more than once; this can + happen if an application calls *note Py_Initialize(): 439. and + *note Py_FinalizeEx(): 5c9. more than once. + + Raises an *note auditing event: fd1. + ‘cpython._PySys_ClearAuditHooks’ with no arguments. + + New in version 3.6. + + -- C Function: void Py_Finalize () + + This is a backwards-compatible version of *note Py_FinalizeEx(): + 5c9. that disregards the return value. + + +File: python.info, Node: Process-wide parameters, Next: Thread State and the Global Interpreter Lock, Prev: Initializing and finalizing the interpreter, Up: Initialization Finalization and Threads + +7.9.4 Process-wide parameters +----------------------------- + + -- C Function: int Py_SetStandardStreamEncoding (const char *encoding, + const char *errors) + + This function should be called before *note Py_Initialize(): 439, + if it is called at all. It specifies which encoding and error + handling to use with standard IO, with the same meanings as in + *note str.encode(): c37. + + It overrides *note PYTHONIOENCODING: 92f. values, and allows + embedding code to control IO encoding when the environment variable + does not work. + + `encoding' and/or `errors' may be ‘NULL’ to use *note + PYTHONIOENCODING: 92f. and/or default values (depending on other + settings). + + Note that *note sys.stderr: 30f. always uses the “backslashreplace” + error handler, regardless of this (or any other) setting. + + If *note Py_FinalizeEx(): 5c9. is called, this function will need + to be called again in order to affect subsequent calls to *note + Py_Initialize(): 439. + + Returns ‘0’ if successful, a nonzero value on error (e.g. calling + after the interpreter has already been initialized). + + New in version 3.4. + + -- C Function: void Py_SetProgramName (const wchar_t *name) + + This function should be called before *note Py_Initialize(): 439. + is called for the first time, if it is called at all. It tells the + interpreter the value of the ‘argv[0]’ argument to the ‘main()’ + function of the program (converted to wide characters). This is + used by *note Py_GetPath(): 38fe. and some other functions below to + find the Python run-time libraries relative to the interpreter + executable. The default value is ‘'python'’. The argument should + point to a zero-terminated wide character string in static storage + whose contents will not change for the duration of the program’s + execution. No code in the Python interpreter will change the + contents of this storage. + + Use *note Py_DecodeLocale(): 43c. to decode a bytes string to get a + ‘wchar_*’ string. + + -- C Function: wchar* Py_GetProgramName () + + Return the program name set with *note Py_SetProgramName(): 1031, + or the default. The returned string points into static storage; + the caller should not modify its value. + + -- C Function: wchar_t* Py_GetPrefix () + + Return the `prefix' for installed platform-independent files. This + is derived through a number of complicated rules from the program + name set with *note Py_SetProgramName(): 1031. and some environment + variables; for example, if the program name is + ‘'/usr/local/bin/python'’, the prefix is ‘'/usr/local'’. The + returned string points into static storage; the caller should not + modify its value. This corresponds to the ‘prefix’ variable in the + top-level ‘Makefile’ and the ‘--prefix’ argument to the ‘configure’ + script at build time. The value is available to Python code as + ‘sys.prefix’. It is only useful on Unix. See also the next + function. + + -- C Function: wchar_t* Py_GetExecPrefix () + + Return the `exec-prefix' for installed platform-`dependent' files. + This is derived through a number of complicated rules from the + program name set with *note Py_SetProgramName(): 1031. and some + environment variables; for example, if the program name is + ‘'/usr/local/bin/python'’, the exec-prefix is ‘'/usr/local'’. The + returned string points into static storage; the caller should not + modify its value. This corresponds to the ‘exec_prefix’ variable + in the top-level ‘Makefile’ and the ‘--exec-prefix’ argument to the + ‘configure’ script at build time. The value is available to Python + code as ‘sys.exec_prefix’. It is only useful on Unix. + + Background: The exec-prefix differs from the prefix when platform + dependent files (such as executables and shared libraries) are + installed in a different directory tree. In a typical + installation, platform dependent files may be installed in the + ‘/usr/local/plat’ subtree while platform independent may be + installed in ‘/usr/local’. + + Generally speaking, a platform is a combination of hardware and + software families, e.g. Sparc machines running the Solaris 2.x + operating system are considered the same platform, but Intel + machines running Solaris 2.x are another platform, and Intel + machines running Linux are yet another platform. Different major + revisions of the same operating system generally also form + different platforms. Non-Unix operating systems are a different + story; the installation strategies on those systems are so + different that the prefix and exec-prefix are meaningless, and set + to the empty string. Note that compiled Python bytecode files are + platform independent (but not independent from the Python version + by which they were compiled!). + + System administrators will know how to configure the ‘mount’ or + ‘automount’ programs to share ‘/usr/local’ between platforms while + having ‘/usr/local/plat’ be a different filesystem for each + platform. + + -- C Function: wchar_t* Py_GetProgramFullPath () + + Return the full program name of the Python executable; this is + computed as a side-effect of deriving the default module search + path from the program name (set by *note Py_SetProgramName(): 1031. + above). The returned string points into static storage; the caller + should not modify its value. The value is available to Python code + as ‘sys.executable’. + + -- C Function: wchar_t* Py_GetPath () + + Return the default module search path; this is computed from the + program name (set by *note Py_SetProgramName(): 1031. above) and + some environment variables. The returned string consists of a + series of directory names separated by a platform dependent + delimiter character. The delimiter character is ‘':'’ on Unix and + Mac OS X, ‘';'’ on Windows. The returned string points into static + storage; the caller should not modify its value. The list *note + sys.path: 488. is initialized with this value on interpreter + startup; it can be (and usually is) modified later to change the + search path for loading modules. + + -- C Function: void Py_SetPath (const wchar_t *) + + Set the default module search path. If this function is called + before *note Py_Initialize(): 439, then *note Py_GetPath(): 38fe. + won’t attempt to compute a default search path but uses the one + provided instead. This is useful if Python is embedded by an + application that has full knowledge of the location of all modules. + The path components should be separated by the platform dependent + delimiter character, which is ‘':'’ on Unix and Mac OS X, ‘';'’ on + Windows. + + This also causes *note sys.executable: 274. to be set to the + program full path (see *note Py_GetProgramFullPath(): 275.) and for + *note sys.prefix: 3322. and *note sys.exec_prefix: 3323. to be + empty. It is up to the caller to modify these if required after + calling *note Py_Initialize(): 439. + + Use *note Py_DecodeLocale(): 43c. to decode a bytes string to get a + ‘wchar_*’ string. + + The path argument is copied internally, so the caller may free it + after the call completes. + + Changed in version 3.8: The program full path is now used for *note + sys.executable: 274, instead of the program name. + + -- C Function: const char* Py_GetVersion () + + Return the version of this Python interpreter. This is a string + that looks something like + + "3.0a5+ (py3k:63103M, May 12 2008, 00:53:55) \n[GCC 4.2.3]" + + The first word (up to the first space character) is the current + Python version; the first three characters are the major and minor + version separated by a period. The returned string points into + static storage; the caller should not modify its value. The value + is available to Python code as *note sys.version: 5d3. + + -- C Function: const char* Py_GetPlatform () + + Return the platform identifier for the current platform. On Unix, + this is formed from the “official” name of the operating system, + converted to lower case, followed by the major revision number; + e.g., for Solaris 2.x, which is also known as SunOS 5.x, the value + is ‘'sunos5'’. On Mac OS X, it is ‘'darwin'’. On Windows, it is + ‘'win'’. The returned string points into static storage; the + caller should not modify its value. The value is available to + Python code as ‘sys.platform’. + + -- C Function: const char* Py_GetCopyright () + + Return the official copyright string for the current Python + version, for example + + ‘'Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam'’ + + The returned string points into static storage; the caller should + not modify its value. The value is available to Python code as + ‘sys.copyright’. + + -- C Function: const char* Py_GetCompiler () + + Return an indication of the compiler used to build the current + Python version, in square brackets, for example: + + "[GCC 2.7.2.2]" + + The returned string points into static storage; the caller should + not modify its value. The value is available to Python code as + part of the variable ‘sys.version’. + + -- C Function: const char* Py_GetBuildInfo () + + Return information about the sequence number and build date and + time of the current Python interpreter instance, for example + + "#67, Aug 1 1997, 22:34:28" + + The returned string points into static storage; the caller should + not modify its value. The value is available to Python code as + part of the variable ‘sys.version’. + + -- C Function: void PySys_SetArgvEx (int argc, wchar_t **argv, + int updatepath) + + Set *note sys.argv: bfb. based on `argc' and `argv'. These + parameters are similar to those passed to the program’s ‘main()’ + function with the difference that the first entry should refer to + the script file to be executed rather than the executable hosting + the Python interpreter. If there isn’t a script that will be run, + the first entry in `argv' can be an empty string. If this function + fails to initialize *note sys.argv: bfb, a fatal condition is + signalled using *note Py_FatalError(): 39a2. + + If `updatepath' is zero, this is all the function does. If + `updatepath' is non-zero, the function also modifies *note + sys.path: 488. according to the following algorithm: + + - If the name of an existing script is passed in ‘argv[0]’, the + absolute path of the directory where the script is located is + prepended to *note sys.path: 488. + + - Otherwise (that is, if `argc' is ‘0’ or ‘argv[0]’ doesn’t + point to an existing file name), an empty string is prepended + to *note sys.path: 488, which is the same as prepending the + current working directory (‘"."’). + + Use *note Py_DecodeLocale(): 43c. to decode a bytes string to get a + ‘wchar_*’ string. + + Note: It is recommended that applications embedding the Python + interpreter for purposes other than executing a single script + pass ‘0’ as `updatepath', and update *note sys.path: 488. + themselves if desired. See CVE-2008-5983(1). + + On versions before 3.1.3, you can achieve the same effect by + manually popping the first *note sys.path: 488. element after + having called *note PySys_SetArgv(): cec, for example using: + + PyRun_SimpleString("import sys; sys.path.pop(0)\n"); + + New in version 3.1.3. + + -- C Function: void PySys_SetArgv (int argc, wchar_t **argv) + + This function works like *note PySys_SetArgvEx(): bfa. with + `updatepath' set to ‘1’ unless the ‘python’ interpreter was started + with the *note -I: fd2. + + Use *note Py_DecodeLocale(): 43c. to decode a bytes string to get a + ‘wchar_*’ string. + + Changed in version 3.4: The `updatepath' value depends on *note -I: + fd2. + + -- C Function: void Py_SetPythonHome (const wchar_t *home) + + Set the default “home” directory, that is, the location of the + standard Python libraries. See *note PYTHONHOME: 4d6. for the + meaning of the argument string. + + The argument should point to a zero-terminated character string in + static storage whose contents will not change for the duration of + the program’s execution. No code in the Python interpreter will + change the contents of this storage. + + Use *note Py_DecodeLocale(): 43c. to decode a bytes string to get a + ‘wchar_*’ string. + + -- C Function: w_char* Py_GetPythonHome () + + Return the default “home”, that is, the value set by a previous + call to *note Py_SetPythonHome(): 3cda, or the value of the *note + PYTHONHOME: 4d6. environment variable if it is set. + + ---------- Footnotes ---------- + + (1) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5983 + + +File: python.info, Node: Thread State and the Global Interpreter Lock, Next: Sub-interpreter support, Prev: Process-wide parameters, Up: Initialization Finalization and Threads + +7.9.5 Thread State and the Global Interpreter Lock +-------------------------------------------------- + +The Python interpreter is not fully thread-safe. In order to support +multi-threaded Python programs, there’s a global lock, called the *note +global interpreter lock: 506. or *note GIL: bea, that must be held by +the current thread before it can safely access Python objects. Without +the lock, even the simplest operations could cause problems in a +multi-threaded program: for example, when two threads simultaneously +increment the reference count of the same object, the reference count +could end up being incremented only once instead of twice. + +Therefore, the rule exists that only the thread that has acquired the +*note GIL: bea. may operate on Python objects or call Python/C API +functions. In order to emulate concurrency of execution, the +interpreter regularly tries to switch threads (see *note +sys.setswitchinterval(): beb.). The lock is also released around +potentially blocking I/O operations like reading or writing a file, so +that other Python threads can run in the meantime. + +The Python interpreter keeps some thread-specific bookkeeping +information inside a data structure called *note PyThreadState: 3cf5. +There’s also one global variable pointing to the current *note +PyThreadState: 3cf5.: it can be retrieved using *note +PyThreadState_Get(): 3cf6. + +* Menu: + +* Releasing the GIL from extension code:: +* Non-Python created threads:: +* Cautions about fork(): Cautions about fork. +* High-level API:: +* Low-level API:: + + +File: python.info, Node: Releasing the GIL from extension code, Next: Non-Python created threads, Up: Thread State and the Global Interpreter Lock + +7.9.5.1 Releasing the GIL from extension code +............................................. + +Most extension code manipulating the *note GIL: bea. has the following +simple structure: + + Save the thread state in a local variable. + Release the global interpreter lock. + ... Do some blocking I/O operation ... + Reacquire the global interpreter lock. + Restore the thread state from the local variable. + +This is so common that a pair of macros exists to simplify it: + + Py_BEGIN_ALLOW_THREADS + ... Do some blocking I/O operation ... + Py_END_ALLOW_THREADS + +The *note Py_BEGIN_ALLOW_THREADS: 3866. macro opens a new block and +declares a hidden local variable; the *note Py_END_ALLOW_THREADS: 2b5. +macro closes the block. + +The block above expands to the following code: + + PyThreadState *_save; + + _save = PyEval_SaveThread(); + ... Do some blocking I/O operation ... + PyEval_RestoreThread(_save); + +Here is how these functions work: the global interpreter lock is used to +protect the pointer to the current thread state. When releasing the +lock and saving the thread state, the current thread state pointer must +be retrieved before the lock is released (since another thread could +immediately acquire the lock and store its own thread state in the +global variable). Conversely, when acquiring the lock and restoring the +thread state, the lock must be acquired before storing the thread state +pointer. + + Note: Calling system I/O functions is the most common use case for + releasing the GIL, but it can also be useful before calling + long-running computations which don’t need access to Python + objects, such as compression or cryptographic functions operating + over memory buffers. For example, the standard *note zlib: 144. + and *note hashlib: 8d. modules release the GIL when compressing or + hashing data. + + +File: python.info, Node: Non-Python created threads, Next: Cautions about fork, Prev: Releasing the GIL from extension code, Up: Thread State and the Global Interpreter Lock + +7.9.5.2 Non-Python created threads +.................................. + +When threads are created using the dedicated Python APIs (such as the +*note threading: 109. module), a thread state is automatically +associated to them and the code showed above is therefore correct. +However, when threads are created from C (for example by a third-party +library with its own thread management), they don’t hold the GIL, nor is +there a thread state structure for them. + +If you need to call Python code from these threads (often this will be +part of a callback API provided by the aforementioned third-party +library), you must first register these threads with the interpreter by +creating a thread state data structure, then acquiring the GIL, and +finally storing their thread state pointer, before you can start using +the Python/C API. When you are done, you should reset the thread state +pointer, release the GIL, and finally free the thread state data +structure. + +The *note PyGILState_Ensure(): 2b6. and *note PyGILState_Release(): +3cfa. functions do all of the above automatically. The typical idiom +for calling into Python from a C thread is: + + PyGILState_STATE gstate; + gstate = PyGILState_Ensure(); + + /* Perform Python actions here. */ + result = CallSomeFunction(); + /* evaluate result or handle exception */ + + /* Release the thread. No Python API allowed beyond this point. */ + PyGILState_Release(gstate); + +Note that the ‘PyGILState_*()’ functions assume there is only one global +interpreter (created automatically by *note Py_Initialize(): 439.). +Python supports the creation of additional interpreters (using *note +Py_NewInterpreter(): 3cf1.), but mixing multiple interpreters and the +‘PyGILState_*()’ API is unsupported. + + +File: python.info, Node: Cautions about fork, Next: High-level API, Prev: Non-Python created threads, Up: Thread State and the Global Interpreter Lock + +7.9.5.3 Cautions about fork() +............................. + +Another important thing to note about threads is their behaviour in the +face of the C ‘fork()’ call. On most systems with ‘fork()’, after a +process forks only the thread that issued the fork will exist. This has +a concrete impact both on how locks must be handled and on all stored +state in CPython’s runtime. + +The fact that only the “current” thread remains means any locks held by +other threads will never be released. Python solves this for *note +os.fork(): 961. by acquiring the locks it uses internally before the +fork, and releasing them afterwards. In addition, it resets any *note +Lock Objects: 204d. in the child. When extending or embedding Python, +there is no way to inform Python of additional (non-Python) locks that +need to be acquired before or reset after a fork. OS facilities such as +‘pthread_atfork()’ would need to be used to accomplish the same thing. +Additionally, when extending or embedding Python, calling ‘fork()’ +directly rather than through *note os.fork(): 961. (and returning to or +calling into Python) may result in a deadlock by one of Python’s +internal locks being held by a thread that is defunct after the fork. +*note PyOS_AfterFork_Child(): 2cd. tries to reset the necessary locks, +but is not always able to. + +The fact that all other threads go away also means that CPython’s +runtime state there must be cleaned up properly, which *note os.fork(): +961. does. This means finalizing all other *note PyThreadState: 3cf5. +objects belonging to the current interpreter and all other *note +PyInterpreterState: 2c2. objects. Due to this and the special nature of +the *note “main” interpreter: 398b, ‘fork()’ should only be called in +that interpreter’s “main” thread, where the CPython global runtime was +originally initialized. The only exception is if ‘exec()’ will be +called immediately after. + + +File: python.info, Node: High-level API, Next: Low-level API, Prev: Cautions about fork, Up: Thread State and the Global Interpreter Lock + +7.9.5.4 High-level API +...................... + +These are the most commonly used types and functions when writing C +extension code, or when embedding the Python interpreter: + + -- C Type: PyInterpreterState + + This data structure represents the state shared by a number of + cooperating threads. Threads belonging to the same interpreter + share their module administration and a few other internal items. + There are no public members in this structure. + + Threads belonging to different interpreters initially share + nothing, except process state like available memory, open file + descriptors and such. The global interpreter lock is also shared + by all threads, regardless of to which interpreter they belong. + + -- C Type: PyThreadState + + This data structure represents the state of a single thread. The + only public data member is ‘interp’ (*note PyInterpreterState *: + 2c2.), which points to this thread’s interpreter state. + + -- C Function: void PyEval_InitThreads () + + Initialize and acquire the global interpreter lock. It should be + called in the main thread before creating a second thread or + engaging in any other thread operations such as + ‘PyEval_ReleaseThread(tstate)’. It is not needed before calling + *note PyEval_SaveThread(): c11. or *note PyEval_RestoreThread(): + 2b4. + + This is a no-op when called for a second time. + + Changed in version 3.7: This function is now called by *note + Py_Initialize(): 439, so you don’t have to call it yourself + anymore. + + Changed in version 3.2: This function cannot be called before *note + Py_Initialize(): 439. anymore. + + -- C Function: int PyEval_ThreadsInitialized () + + Returns a non-zero value if *note PyEval_InitThreads(): c12. has + been called. This function can be called without holding the GIL, + and therefore can be used to avoid calls to the locking API when + running single-threaded. + + Changed in version 3.7: The *note GIL: bea. is now initialized by + *note Py_Initialize(): 439. + + -- C Function: PyThreadState* PyEval_SaveThread () + + Release the global interpreter lock (if it has been created) and + reset the thread state to ‘NULL’, returning the previous thread + state (which is not ‘NULL’). If the lock has been created, the + current thread must have acquired it. + + -- C Function: void PyEval_RestoreThread (PyThreadState *tstate) + + Acquire the global interpreter lock (if it has been created) and + set the thread state to `tstate', which must not be ‘NULL’. If the + lock has been created, the current thread must not have acquired + it, otherwise deadlock ensues. + + Note: Calling this function from a thread when the runtime is + finalizing will terminate the thread, even if the thread was + not created by Python. You can use ‘_Py_IsFinalizing()’ or + *note sys.is_finalizing(): 762. to check if the interpreter is + in process of being finalized before calling this function to + avoid unwanted termination. + + -- C Function: PyThreadState* PyThreadState_Get () + + Return the current thread state. The global interpreter lock must + be held. When the current thread state is ‘NULL’, this issues a + fatal error (so that the caller needn’t check for ‘NULL’). + + -- C Function: PyThreadState* PyThreadState_Swap + (PyThreadState *tstate) + + Swap the current thread state with the thread state given by the + argument `tstate', which may be ‘NULL’. The global interpreter + lock must be held and is not released. + +The following functions use thread-local storage, and are not compatible +with sub-interpreters: + + -- C Function: PyGILState_STATE PyGILState_Ensure () + + Ensure that the current thread is ready to call the Python C API + regardless of the current state of Python, or of the global + interpreter lock. This may be called as many times as desired by a + thread as long as each call is matched with a call to *note + PyGILState_Release(): 3cfa. In general, other thread-related APIs + may be used between *note PyGILState_Ensure(): 2b6. and *note + PyGILState_Release(): 3cfa. calls as long as the thread state is + restored to its previous state before the Release(). For example, + normal usage of the *note Py_BEGIN_ALLOW_THREADS: 3866. and *note + Py_END_ALLOW_THREADS: 2b5. macros is acceptable. + + The return value is an opaque “handle” to the thread state when + *note PyGILState_Ensure(): 2b6. was called, and must be passed to + *note PyGILState_Release(): 3cfa. to ensure Python is left in the + same state. Even though recursive calls are allowed, these handles + `cannot' be shared - each unique call to *note PyGILState_Ensure(): + 2b6. must save the handle for its call to *note + PyGILState_Release(): 3cfa. + + When the function returns, the current thread will hold the GIL and + be able to call arbitrary Python code. Failure is a fatal error. + + Note: Calling this function from a thread when the runtime is + finalizing will terminate the thread, even if the thread was + not created by Python. You can use ‘_Py_IsFinalizing()’ or + *note sys.is_finalizing(): 762. to check if the interpreter is + in process of being finalized before calling this function to + avoid unwanted termination. + + -- C Function: void PyGILState_Release (PyGILState_STATE) + + Release any resources previously acquired. After this call, + Python’s state will be the same as it was prior to the + corresponding *note PyGILState_Ensure(): 2b6. call (but generally + this state will be unknown to the caller, hence the use of the + GILState API). + + Every call to *note PyGILState_Ensure(): 2b6. must be matched by a + call to *note PyGILState_Release(): 3cfa. on the same thread. + + -- C Function: PyThreadState* PyGILState_GetThisThreadState () + + Get the current thread state for this thread. May return ‘NULL’ if + no GILState API has been used on the current thread. Note that the + main thread always has such a thread-state, even if no + auto-thread-state call has been made on the main thread. This is + mainly a helper/diagnostic function. + + -- C Function: int PyGILState_Check () + + Return ‘1’ if the current thread is holding the GIL and ‘0’ + otherwise. This function can be called from any thread at any + time. Only if it has had its Python thread state initialized and + currently is holding the GIL will it return ‘1’. This is mainly a + helper/diagnostic function. It can be useful for example in + callback contexts or memory allocation functions when knowing that + the GIL is locked can allow the caller to perform sensitive actions + or otherwise behave differently. + + New in version 3.4. + +The following macros are normally used without a trailing semicolon; +look for example usage in the Python source distribution. + + -- C Macro: Py_BEGIN_ALLOW_THREADS + + This macro expands to ‘{ PyThreadState *_save; _save = + PyEval_SaveThread();’. Note that it contains an opening brace; it + must be matched with a following *note Py_END_ALLOW_THREADS: 2b5. + macro. See above for further discussion of this macro. + + -- C Macro: Py_END_ALLOW_THREADS + + This macro expands to ‘PyEval_RestoreThread(_save); }’. Note that + it contains a closing brace; it must be matched with an earlier + *note Py_BEGIN_ALLOW_THREADS: 3866. macro. See above for further + discussion of this macro. + + -- C Macro: Py_BLOCK_THREADS + + This macro expands to ‘PyEval_RestoreThread(_save);’: it is + equivalent to *note Py_END_ALLOW_THREADS: 2b5. without the closing + brace. + + -- C Macro: Py_UNBLOCK_THREADS + + This macro expands to ‘_save = PyEval_SaveThread();’: it is + equivalent to *note Py_BEGIN_ALLOW_THREADS: 3866. without the + opening brace and variable declaration. + + +File: python.info, Node: Low-level API, Prev: High-level API, Up: Thread State and the Global Interpreter Lock + +7.9.5.5 Low-level API +..................... + +All of the following functions must be called after *note +Py_Initialize(): 439. + +Changed in version 3.7: *note Py_Initialize(): 439. now initializes the +*note GIL: bea. + + -- C Function: PyInterpreterState* PyInterpreterState_New () + + Create a new interpreter state object. The global interpreter lock + need not be held, but may be held if it is necessary to serialize + calls to this function. + + Raises an *note auditing event: fd1. + ‘cpython.PyInterpreterState_New’ with no arguments. + + -- C Function: void PyInterpreterState_Clear + (PyInterpreterState *interp) + + Reset all information in an interpreter state object. The global + interpreter lock must be held. + + Raises an *note auditing event: fd1. + ‘cpython.PyInterpreterState_Clear’ with no arguments. + + -- C Function: void PyInterpreterState_Delete + (PyInterpreterState *interp) + + Destroy an interpreter state object. The global interpreter lock + need not be held. The interpreter state must have been reset with + a previous call to *note PyInterpreterState_Clear(): 31ee. + + -- C Function: PyThreadState* PyThreadState_New + (PyInterpreterState *interp) + + Create a new thread state object belonging to the given interpreter + object. The global interpreter lock need not be held, but may be + held if it is necessary to serialize calls to this function. + + -- C Function: void PyThreadState_Clear (PyThreadState *tstate) + + Reset all information in a thread state object. The global + interpreter lock must be held. + + -- C Function: void PyThreadState_Delete (PyThreadState *tstate) + + Destroy a thread state object. The global interpreter lock need + not be held. The thread state must have been reset with a previous + call to *note PyThreadState_Clear(): 3d05. + + -- C Function: PY_INT64_T PyInterpreterState_GetID + (PyInterpreterState *interp) + + Return the interpreter’s unique ID. If there was any error in doing + so then ‘-1’ is returned and an error is set. + + New in version 3.7. + + -- C Function: PyObject* PyInterpreterState_GetDict + (PyInterpreterState *interp) + + Return a dictionary in which interpreter-specific data may be + stored. If this function returns ‘NULL’ then no exception has been + raised and the caller should assume no interpreter-specific dict is + available. + + This is not a replacement for *note PyModule_GetState(): 3c2c, + which extensions should use to store interpreter-specific state + information. + + New in version 3.8. + + -- C Function: PyObject* PyThreadState_GetDict () + `Return value: Borrowed reference.' Return a dictionary in which + extensions can store thread-specific state information. Each + extension should use a unique key to use to store state in the + dictionary. It is okay to call this function when no current + thread state is available. If this function returns ‘NULL’, no + exception has been raised and the caller should assume no current + thread state is available. + + -- C Function: int PyThreadState_SetAsyncExc (unsigned long id, + PyObject *exc) + + Asynchronously raise an exception in a thread. The `id' argument + is the thread id of the target thread; `exc' is the exception + object to be raised. This function does not steal any references + to `exc'. To prevent naive misuse, you must write your own C + extension to call this. Must be called with the GIL held. Returns + the number of thread states modified; this is normally one, but + will be zero if the thread id isn’t found. If `exc' is ‘NULL’, the + pending exception (if any) for the thread is cleared. This raises + no exceptions. + + Changed in version 3.7: The type of the `id' parameter changed from + ‘long’ to ‘unsigned long’. + + -- C Function: void PyEval_AcquireThread (PyThreadState *tstate) + + Acquire the global interpreter lock and set the current thread + state to `tstate', which should not be ‘NULL’. The lock must have + been created earlier. If this thread already has the lock, + deadlock ensues. + + Note: Calling this function from a thread when the runtime is + finalizing will terminate the thread, even if the thread was + not created by Python. You can use ‘_Py_IsFinalizing()’ or + *note sys.is_finalizing(): 762. to check if the interpreter is + in process of being finalized before calling this function to + avoid unwanted termination. + + Changed in version 3.8: Updated to be consistent with *note + PyEval_RestoreThread(): 2b4, *note Py_END_ALLOW_THREADS(): 2b5, and + *note PyGILState_Ensure(): 2b6, and terminate the current thread if + called while the interpreter is finalizing. + + *note PyEval_RestoreThread(): 2b4. is a higher-level function which + is always available (even when threads have not been initialized). + + -- C Function: void PyEval_ReleaseThread (PyThreadState *tstate) + + Reset the current thread state to ‘NULL’ and release the global + interpreter lock. The lock must have been created earlier and must + be held by the current thread. The `tstate' argument, which must + not be ‘NULL’, is only used to check that it represents the current + thread state — if it isn’t, a fatal error is reported. + + *note PyEval_SaveThread(): c11. is a higher-level function which is + always available (even when threads have not been initialized). + + -- C Function: void PyEval_AcquireLock () + + Acquire the global interpreter lock. The lock must have been + created earlier. If this thread already has the lock, a deadlock + ensues. + + Deprecated since version 3.2: This function does not update the + current thread state. Please use *note PyEval_RestoreThread(): + 2b4. or *note PyEval_AcquireThread(): 2b3. instead. + + Note: Calling this function from a thread when the runtime is + finalizing will terminate the thread, even if the thread was + not created by Python. You can use ‘_Py_IsFinalizing()’ or + *note sys.is_finalizing(): 762. to check if the interpreter is + in process of being finalized before calling this function to + avoid unwanted termination. + + Changed in version 3.8: Updated to be consistent with *note + PyEval_RestoreThread(): 2b4, *note Py_END_ALLOW_THREADS(): 2b5, and + *note PyGILState_Ensure(): 2b6, and terminate the current thread if + called while the interpreter is finalizing. + + -- C Function: void PyEval_ReleaseLock () + + Release the global interpreter lock. The lock must have been + created earlier. + + Deprecated since version 3.2: This function does not update the + current thread state. Please use *note PyEval_SaveThread(): c11. + or *note PyEval_ReleaseThread(): 3d09. instead. + + +File: python.info, Node: Sub-interpreter support, Next: Asynchronous Notifications, Prev: Thread State and the Global Interpreter Lock, Up: Initialization Finalization and Threads + +7.9.6 Sub-interpreter support +----------------------------- + +While in most uses, you will only embed a single Python interpreter, +there are cases where you need to create several independent +interpreters in the same process and perhaps even in the same thread. +Sub-interpreters allow you to do that. + +The “main” interpreter is the first one created when the runtime +initializes. It is usually the only Python interpreter in a process. +Unlike sub-interpreters, the main interpreter has unique process-global +responsibilities like signal handling. It is also responsible for +execution during runtime initialization and is usually the active +interpreter during runtime finalization. The *note +PyInterpreterState_Main(): 3d0b. function returns a pointer to its +state. + +You can switch between sub-interpreters using the *note +PyThreadState_Swap(): 3cfd. function. You can create and destroy them +using the following functions: + + -- C Function: PyThreadState* Py_NewInterpreter () + + Create a new sub-interpreter. This is an (almost) totally separate + environment for the execution of Python code. In particular, the + new interpreter has separate, independent versions of all imported + modules, including the fundamental modules *note builtins: 13, + *note __main__: 1. and *note sys: fd. The table of loaded modules + (‘sys.modules’) and the module search path (‘sys.path’) are also + separate. The new environment has no ‘sys.argv’ variable. It has + new standard I/O stream file objects ‘sys.stdin’, ‘sys.stdout’ and + ‘sys.stderr’ (however these refer to the same underlying file + descriptors). + + The return value points to the first thread state created in the + new sub-interpreter. This thread state is made in the current + thread state. Note that no actual thread is created; see the + discussion of thread states below. If creation of the new + interpreter is unsuccessful, ‘NULL’ is returned; no exception is + set since the exception state is stored in the current thread state + and there may not be a current thread state. (Like all other + Python/C API functions, the global interpreter lock must be held + before calling this function and is still held when it returns; + however, unlike most other Python/C API functions, there needn’t be + a current thread state on entry.) + + Extension modules are shared between (sub-)interpreters as follows: + + * For modules using multi-phase initialization, e.g. *note + PyModule_FromDefAndSpec(): 7ab, a separate module object is + created and initialized for each interpreter. Only C-level + static and global variables are shared between these module + objects. + + * For modules using single-phase initialization, e.g. *note + PyModule_Create(): 3847, the first time a particular extension + is imported, it is initialized normally, and a (shallow) copy + of its module’s dictionary is squirreled away. When the same + extension is imported by another (sub-)interpreter, a new + module is initialized and filled with the contents of this + copy; the extension’s ‘init’ function is not called. Objects + in the module’s dictionary thus end up shared across + (sub-)interpreters, which might cause unwanted behavior (see + *note Bugs and caveats: 3d0c. below). + + Note that this is different from what happens when an + extension is imported after the interpreter has been + completely re-initialized by calling *note Py_FinalizeEx(): + 5c9. and *note Py_Initialize(): 439.; in that case, the + extension’s ‘initmodule’ function `is' called again. As with + multi-phase initialization, this means that only C-level + static and global variables are shared between these modules. + + -- C Function: void Py_EndInterpreter (PyThreadState *tstate) + + Destroy the (sub-)interpreter represented by the given thread + state. The given thread state must be the current thread state. + See the discussion of thread states below. When the call returns, + the current thread state is ‘NULL’. All thread states associated + with this interpreter are destroyed. (The global interpreter lock + must be held before calling this function and is still held when it + returns.) *note Py_FinalizeEx(): 5c9. will destroy all + sub-interpreters that haven’t been explicitly destroyed at that + point. + +* Menu: + +* Bugs and caveats:: + + +File: python.info, Node: Bugs and caveats, Up: Sub-interpreter support + +7.9.6.1 Bugs and caveats +........................ + +Because sub-interpreters (and the main interpreter) are part of the same +process, the insulation between them isn’t perfect — for example, using +low-level file operations like *note os.close(): 3d2. they can +(accidentally or maliciously) affect each other’s open files. Because +of the way extensions are shared between (sub-)interpreters, some +extensions may not work properly; this is especially likely when using +single-phase initialization or (static) global variables. It is +possible to insert objects created in one sub-interpreter into a +namespace of another (sub-)interpreter; this should be avoided if +possible. + +Special care should be taken to avoid sharing user-defined functions, +methods, instances or classes between sub-interpreters, since import +operations executed by such objects may affect the wrong +(sub-)interpreter’s dictionary of loaded modules. It is equally +important to avoid sharing objects from which the above are reachable. + +Also note that combining this functionality with ‘PyGILState_*()’ APIs +is delicate, because these APIs assume a bijection between Python thread +states and OS-level threads, an assumption broken by the presence of +sub-interpreters. It is highly recommended that you don’t switch +sub-interpreters between a pair of matching *note PyGILState_Ensure(): +2b6. and *note PyGILState_Release(): 3cfa. calls. Furthermore, +extensions (such as *note ctypes: 2b.) using these APIs to allow calling +of Python code from non-Python created threads will probably be broken +when using sub-interpreters. + + +File: python.info, Node: Asynchronous Notifications, Next: Profiling and Tracing, Prev: Sub-interpreter support, Up: Initialization Finalization and Threads + +7.9.7 Asynchronous Notifications +-------------------------------- + +A mechanism is provided to make asynchronous notifications to the main +interpreter thread. These notifications take the form of a function +pointer and a void pointer argument. + + -- C Function: int Py_AddPendingCall (int (*func)(void *), void *arg) + + Schedule a function to be called from the main interpreter thread. + On success, ‘0’ is returned and `func' is queued for being called + in the main thread. On failure, ‘-1’ is returned without setting + any exception. + + When successfully queued, `func' will be `eventually' called from + the main interpreter thread with the argument `arg'. It will be + called asynchronously with respect to normally running Python code, + but with both these conditions met: + + * on a *note bytecode: 614. boundary; + + * with the main thread holding the *note global interpreter + lock: 506. (`func' can therefore use the full C API). + + `func' must return ‘0’ on success, or ‘-1’ on failure with an + exception set. `func' won’t be interrupted to perform another + asynchronous notification recursively, but it can still be + interrupted to switch threads if the global interpreter lock is + released. + + This function doesn’t need a current thread state to run, and it + doesn’t need the global interpreter lock. + + Warning: This is a low-level function, only useful for very + special cases. There is no guarantee that `func' will be + called as quick as possible. If the main thread is busy + executing a system call, `func' won’t be called before the + system call returns. This function is generally `not' + suitable for calling Python code from arbitrary C threads. + Instead, use the *note PyGILState API: 3cf8. + + New in version 3.1. + + +File: python.info, Node: Profiling and Tracing, Next: Advanced Debugger Support, Prev: Asynchronous Notifications, Up: Initialization Finalization and Threads + +7.9.8 Profiling and Tracing +--------------------------- + +The Python interpreter provides some low-level support for attaching +profiling and execution tracing facilities. These are used for +profiling, debugging, and coverage analysis tools. + +This C interface allows the profiling or tracing code to avoid the +overhead of calling through Python-level callable objects, making a +direct C function call instead. The essential attributes of the +facility have not changed; the interface allows trace functions to be +installed per-thread, and the basic events reported to the trace +function are the same as had been reported to the Python-level trace +functions in previous versions. + + -- C Type: int (*Py_tracefunc) (PyObject *obj, PyFrameObject *frame, + int what, PyObject *arg) + + The type of the trace function registered using *note + PyEval_SetProfile(): e3b. and *note PyEval_SetTrace(): e3c. The + first parameter is the object passed to the registration function + as `obj', `frame' is the frame object to which the event pertains, + `what' is one of the constants ‘PyTrace_CALL’, ‘PyTrace_EXCEPTION’, + ‘PyTrace_LINE’, ‘PyTrace_RETURN’, ‘PyTrace_C_CALL’, + ‘PyTrace_C_EXCEPTION’, ‘PyTrace_C_RETURN’, or ‘PyTrace_OPCODE’, and + `arg' depends on the value of `what': + + Value of `what' Meaning of `arg' + + -------------------------------------------------------------------------------- + + ‘PyTrace_CALL’ Always *note Py_None: 3844. + + + ‘PyTrace_EXCEPTION’ Exception information as returned by + *note sys.exc_info(): c5f. + + + ‘PyTrace_LINE’ Always *note Py_None: 3844. + + + ‘PyTrace_RETURN’ Value being returned to the caller, or + ‘NULL’ if caused by an exception. + + + ‘PyTrace_C_CALL’ Function object being called. + + + ‘PyTrace_C_EXCEPTION’ Function object being called. + + + ‘PyTrace_C_RETURN’ Function object being called. + + + ‘PyTrace_OPCODE’ Always *note Py_None: 3844. + + + -- C Variable: int PyTrace_CALL + + The value of the `what' parameter to a *note Py_tracefunc: 3d11. + function when a new call to a function or method is being reported, + or a new entry into a generator. Note that the creation of the + iterator for a generator function is not reported as there is no + control transfer to the Python bytecode in the corresponding frame. + + -- C Variable: int PyTrace_EXCEPTION + + The value of the `what' parameter to a *note Py_tracefunc: 3d11. + function when an exception has been raised. The callback function + is called with this value for `what' when after any bytecode is + processed after which the exception becomes set within the frame + being executed. The effect of this is that as exception + propagation causes the Python stack to unwind, the callback is + called upon return to each frame as the exception propagates. Only + trace functions receives these events; they are not needed by the + profiler. + + -- C Variable: int PyTrace_LINE + + The value passed as the `what' parameter to a *note Py_tracefunc: + 3d11. function (but not a profiling function) when a line-number + event is being reported. It may be disabled for a frame by setting + ‘f_trace_lines’ to `0' on that frame. + + -- C Variable: int PyTrace_RETURN + + The value for the `what' parameter to *note Py_tracefunc: 3d11. + functions when a call is about to return. + + -- C Variable: int PyTrace_C_CALL + + The value for the `what' parameter to *note Py_tracefunc: 3d11. + functions when a C function is about to be called. + + -- C Variable: int PyTrace_C_EXCEPTION + + The value for the `what' parameter to *note Py_tracefunc: 3d11. + functions when a C function has raised an exception. + + -- C Variable: int PyTrace_C_RETURN + + The value for the `what' parameter to *note Py_tracefunc: 3d11. + functions when a C function has returned. + + -- C Variable: int PyTrace_OPCODE + + The value for the `what' parameter to *note Py_tracefunc: 3d11. + functions (but not profiling functions) when a new opcode is about + to be executed. This event is not emitted by default: it must be + explicitly requested by setting ‘f_trace_opcodes’ to `1' on the + frame. + + -- C Function: void PyEval_SetProfile (Py_tracefunc func, + PyObject *obj) + + Set the profiler function to `func'. The `obj' parameter is passed + to the function as its first parameter, and may be any Python + object, or ‘NULL’. If the profile function needs to maintain + state, using a different value for `obj' for each thread provides a + convenient and thread-safe place to store it. The profile function + is called for all monitored events except ‘PyTrace_LINE’ + ‘PyTrace_OPCODE’ and ‘PyTrace_EXCEPTION’. + + -- C Function: void PyEval_SetTrace (Py_tracefunc func, PyObject *obj) + + Set the tracing function to `func'. This is similar to *note + PyEval_SetProfile(): e3b, except the tracing function does receive + line-number events and per-opcode events, but does not receive any + event related to C function objects being called. Any trace + function registered using *note PyEval_SetTrace(): e3c. will not + receive ‘PyTrace_C_CALL’, ‘PyTrace_C_EXCEPTION’ or + ‘PyTrace_C_RETURN’ as a value for the `what' parameter. + + +File: python.info, Node: Advanced Debugger Support, Next: Thread Local Storage Support, Prev: Profiling and Tracing, Up: Initialization Finalization and Threads + +7.9.9 Advanced Debugger Support +------------------------------- + +These functions are only intended to be used by advanced debugging +tools. + + -- C Function: PyInterpreterState* PyInterpreterState_Head () + + Return the interpreter state object at the head of the list of all + such objects. + + -- C Function: PyInterpreterState* PyInterpreterState_Main () + + Return the main interpreter state object. + + -- C Function: PyInterpreterState* PyInterpreterState_Next + (PyInterpreterState *interp) + + Return the next interpreter state object after `interp' from the + list of all such objects. + + -- C Function: PyThreadState * PyInterpreterState_ThreadHead + (PyInterpreterState *interp) + + Return the pointer to the first *note PyThreadState: 3cf5. object + in the list of threads associated with the interpreter `interp'. + + -- C Function: PyThreadState* PyThreadState_Next + (PyThreadState *tstate) + + Return the next thread state object after `tstate' from the list of + all such objects belonging to the same *note PyInterpreterState: + 2c2. object. + + +File: python.info, Node: Thread Local Storage Support, Prev: Advanced Debugger Support, Up: Initialization Finalization and Threads + +7.9.10 Thread Local Storage Support +----------------------------------- + +The Python interpreter provides low-level support for thread-local +storage (TLS) which wraps the underlying native TLS implementation to +support the Python-level thread local storage API (*note +threading.local: 1fdc.). The CPython C level APIs are similar to those +offered by pthreads and Windows: use a thread key and functions to +associate a ‘void*’ value per thread. + +The GIL does `not' need to be held when calling these functions; they +supply their own locking. + +Note that ‘Python.h’ does not include the declaration of the TLS APIs, +you need to include ‘pythread.h’ to use thread-local storage. + + Note: None of these API functions handle memory management on + behalf of the ‘void*’ values. You need to allocate and deallocate + them yourself. If the ‘void*’ values happen to be *note PyObject*: + 4ba, these functions don’t do refcount operations on them either. + +* Menu: + +* Thread Specific Storage (TSS) API: Thread Specific Storage TSS API. +* Thread Local Storage (TLS) API: Thread Local Storage TLS API. + + +File: python.info, Node: Thread Specific Storage TSS API, Next: Thread Local Storage TLS API, Up: Thread Local Storage Support + +7.9.10.1 Thread Specific Storage (TSS) API +.......................................... + +TSS API is introduced to supersede the use of the existing TLS API +within the CPython interpreter. This API uses a new type *note +Py_tss_t: 318. instead of ‘int’ to represent thread keys. + +New in version 3.7. + +See also +........ + +“A New C-API for Thread-Local Storage in CPython” ( PEP 539(1)) + + -- C Type: Py_tss_t + + This data structure represents the state of a thread key, the + definition of which may depend on the underlying TLS + implementation, and it has an internal field representing the key’s + initialization state. There are no public members in this + structure. + + When *note Py_LIMITED_API: 3905. is not defined, static allocation + of this type by *note Py_tss_NEEDS_INIT: 3d1f. is allowed. + + -- C Macro: Py_tss_NEEDS_INIT + + This macro expands to the initializer for *note Py_tss_t: 318. + variables. Note that this macro won’t be defined with *note + Py_LIMITED_API: 3905. + +* Menu: + +* Dynamic Allocation:: +* Methods: Methods<4>. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0539 + + +File: python.info, Node: Dynamic Allocation, Next: Methods<4>, Up: Thread Specific Storage TSS API + +7.9.10.2 Dynamic Allocation +........................... + +Dynamic allocation of the *note Py_tss_t: 318, required in extension +modules built with *note Py_LIMITED_API: 3905, where static allocation +of this type is not possible due to its implementation being opaque at +build time. + + -- C Function: Py_tss_t* PyThread_tss_alloc () + + Return a value which is the same state as a value initialized with + *note Py_tss_NEEDS_INIT: 3d1f, or ‘NULL’ in the case of dynamic + allocation failure. + + -- C Function: void PyThread_tss_free (Py_tss_t *key) + + Free the given `key' allocated by *note PyThread_tss_alloc(): 3d21, + after first calling *note PyThread_tss_delete(): 3d23. to ensure + any associated thread locals have been unassigned. This is a no-op + if the `key' argument is ‘NULL’. + + Note: A freed key becomes a dangling pointer, you should reset + the key to ‘NULL’. + + +File: python.info, Node: Methods<4>, Prev: Dynamic Allocation, Up: Thread Specific Storage TSS API + +7.9.10.3 Methods +................ + +The parameter `key' of these functions must not be ‘NULL’. Moreover, +the behaviors of *note PyThread_tss_set(): 3d25. and *note +PyThread_tss_get(): 3d26. are undefined if the given *note Py_tss_t: +318. has not been initialized by *note PyThread_tss_create(): 3d27. + + -- C Function: int PyThread_tss_is_created (Py_tss_t *key) + + Return a non-zero value if the given *note Py_tss_t: 318. has been + initialized by *note PyThread_tss_create(): 3d27. + + -- C Function: int PyThread_tss_create (Py_tss_t *key) + + Return a zero value on successful initialization of a TSS key. The + behavior is undefined if the value pointed to by the `key' argument + is not initialized by *note Py_tss_NEEDS_INIT: 3d1f. This function + can be called repeatedly on the same key – calling it on an already + initialized key is a no-op and immediately returns success. + + -- C Function: void PyThread_tss_delete (Py_tss_t *key) + + Destroy a TSS key to forget the values associated with the key + across all threads, and change the key’s initialization state to + uninitialized. A destroyed key is able to be initialized again by + *note PyThread_tss_create(): 3d27. This function can be called + repeatedly on the same key – calling it on an already destroyed key + is a no-op. + + -- C Function: int PyThread_tss_set (Py_tss_t *key, void *value) + + Return a zero value to indicate successfully associating a ‘void*’ + value with a TSS key in the current thread. Each thread has a + distinct mapping of the key to a ‘void*’ value. + + -- C Function: void* PyThread_tss_get (Py_tss_t *key) + + Return the ‘void*’ value associated with a TSS key in the current + thread. This returns ‘NULL’ if no value is associated with the key + in the current thread. + + +File: python.info, Node: Thread Local Storage TLS API, Prev: Thread Specific Storage TSS API, Up: Thread Local Storage Support + +7.9.10.4 Thread Local Storage (TLS) API +....................................... + +Deprecated since version 3.7: This API is superseded by *note Thread +Specific Storage (TSS) API: 317. + + Note: This version of the API does not support platforms where the + native TLS key is defined in a way that cannot be safely cast to + ‘int’. On such platforms, *note PyThread_create_key(): 3d2a. will + return immediately with a failure status, and the other TLS + functions will all be no-ops on such platforms. + +Due to the compatibility problem noted above, this version of the API +should not be used in new code. + + -- C Function: int PyThread_create_key () + + -- C Function: void PyThread_delete_key (int key) + + -- C Function: int PyThread_set_key_value (int key, void *value) + + -- C Function: void* PyThread_get_key_value (int key) + + -- C Function: void PyThread_delete_key_value (int key) + + -- C Function: void PyThread_ReInitTLS () + + +File: python.info, Node: Python Initialization Configuration, Next: Memory Management, Prev: Initialization Finalization and Threads, Up: Python/C API Reference Manual + +7.10 Python Initialization Configuration +======================================== + +New in version 3.8. + +Structures: + + * *note PyConfig: 15f. + + * *note PyPreConfig: 160. + + * *note PyStatus: 161. + + * *note PyWideStringList: 162. + +Functions: + + * *note PyConfig_Clear(): 163. + + * *note PyConfig_InitIsolatedConfig(): 164. + + * *note PyConfig_InitPythonConfig(): 165. + + * *note PyConfig_Read(): 166. + + * *note PyConfig_SetArgv(): 167. + + * *note PyConfig_SetBytesArgv(): 168. + + * *note PyConfig_SetBytesString(): 169. + + * *note PyConfig_SetString(): 16a. + + * *note PyConfig_SetWideStringList(): 3d31. + + * *note PyPreConfig_InitIsolatedConfig(): 16b. + + * *note PyPreConfig_InitPythonConfig(): 16c. + + * *note PyStatus_Error(): 16d. + + * *note PyStatus_Exception(): 16e. + + * *note PyStatus_Exit(): 16f. + + * *note PyStatus_IsError(): 170. + + * *note PyStatus_IsExit(): 171. + + * *note PyStatus_NoMemory(): 172. + + * *note PyStatus_Ok(): 173. + + * *note PyWideStringList_Append(): 174. + + * *note PyWideStringList_Insert(): 175. + + * *note Py_ExitStatusException(): 177. + + * *note Py_InitializeFromConfig(): 178. + + * *note Py_PreInitialize(): 179. + + * *note Py_PreInitializeFromArgs(): 17a. + + * *note Py_PreInitializeFromBytesArgs(): 17b. + + * *note Py_RunMain(): 17c. + +The preconfiguration (‘PyPreConfig’ type) is stored in +‘_PyRuntime.preconfig’ and the configuration (‘PyConfig’ type) is stored +in ‘PyInterpreterState.config’. + +See also *note Initialization, Finalization, and Threads: 3cd4. + +See also +........ + +PEP 587(1) “Python Initialization Configuration”. + +* Menu: + +* PyWideStringList:: +* PyStatus:: +* PyPreConfig:: +* Preinitialization with PyPreConfig:: +* PyConfig:: +* Initialization with PyConfig:: +* Isolated Configuration:: +* Python Configuration:: +* Path Configuration:: +* Py_RunMain(): Py_RunMain. +* Multi-Phase Initialization Private Provisional API:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0587 + + +File: python.info, Node: PyWideStringList, Next: PyStatus, Up: Python Initialization Configuration + +7.10.1 PyWideStringList +----------------------- + + -- C Type: PyWideStringList + + List of ‘wchar_t*’ strings. + + If `length' is non-zero, `items' must be non-‘NULL’ and all strings + must be non-‘NULL’. + + Methods: + + -- C Function: PyStatus PyWideStringList_Append + (PyWideStringList *list, const wchar_t *item) + + Append `item' to `list'. + + Python must be preinitialized to call this function. + + -- C Function: PyStatus PyWideStringList_Insert + (PyWideStringList *list, Py_ssize_t index, const + wchar_t *item) + + Insert `item' into `list' at `index'. + + If `index' is greater than or equal to `list' length, append + `item' to `list'. + + `index' must be greater than or equal to 0. + + Python must be preinitialized to call this function. + + Structure fields: + + -- C Member: Py_ssize_t length + + List length. + + -- C Member: wchar_t** items + + List items. + + +File: python.info, Node: PyStatus, Next: PyPreConfig, Prev: PyWideStringList, Up: Python Initialization Configuration + +7.10.2 PyStatus +--------------- + + -- C Type: PyStatus + + Structure to store an initialization function status: success, + error or exit. + + For an error, it can store the C function name which created the + error. + + Structure fields: + + -- C Member: int exitcode + + Exit code. Argument passed to ‘exit()’. + + -- C Member: const char *err_msg + + Error message. + + -- C Member: const char *func + + Name of the function which created an error, can be ‘NULL’. + + Functions to create a status: + + -- C Function: PyStatus PyStatus_Ok (void) + + Success. + + -- C Function: PyStatus PyStatus_Error (const char *err_msg) + + Initialization error with a message. + + -- C Function: PyStatus PyStatus_NoMemory (void) + + Memory allocation failure (out of memory). + + -- C Function: PyStatus PyStatus_Exit (int exitcode) + + Exit Python with the specified exit code. + + Functions to handle a status: + + -- C Function: int PyStatus_Exception (PyStatus status) + + Is the status an error or an exit? If true, the exception + must be handled; by calling *note Py_ExitStatusException(): + 177. for example. + + -- C Function: int PyStatus_IsError (PyStatus status) + + Is the result an error? + + -- C Function: int PyStatus_IsExit (PyStatus status) + + Is the result an exit? + + -- C Function: void Py_ExitStatusException (PyStatus status) + + Call ‘exit(exitcode)’ if `status' is an exit. Print the error + message and exit with a non-zero exit code if `status' is an + error. Must only be called if ‘PyStatus_Exception(status)’ is + non-zero. + + Note: Internally, Python uses macros which set ‘PyStatus.func’, + whereas functions to create a status set ‘func’ to ‘NULL’. + +Example: + + PyStatus alloc(void **ptr, size_t size) + { + *ptr = PyMem_RawMalloc(size); + if (*ptr == NULL) { + return PyStatus_NoMemory(); + } + return PyStatus_Ok(); + } + + int main(int argc, char **argv) + { + void *ptr; + PyStatus status = alloc(&ptr, 16); + if (PyStatus_Exception(status)) { + Py_ExitStatusException(status); + } + PyMem_Free(ptr); + return 0; + } + + +File: python.info, Node: PyPreConfig, Next: Preinitialization with PyPreConfig, Prev: PyStatus, Up: Python Initialization Configuration + +7.10.3 PyPreConfig +------------------ + + -- C Type: PyPreConfig + + Structure used to preinitialize Python: + + * Set the Python memory allocator + + * Configure the LC_CTYPE locale + + * Set the UTF-8 mode + + Function to initialize a preconfiguration: + + -- C Function: void PyPreConfig_InitPythonConfig + (PyPreConfig *preconfig) + + Initialize the preconfiguration with *note Python + Configuration: 3d3a. + + -- C Function: void PyPreConfig_InitIsolatedConfig + (PyPreConfig *preconfig) + + Initialize the preconfiguration with *note Isolated + Configuration: 3d3b. + + Structure fields: + + -- C Member: int allocator + + Name of the memory allocator: + + * ‘PYMEM_ALLOCATOR_NOT_SET’ (‘0’): don’t change memory + allocators (use defaults) + + * ‘PYMEM_ALLOCATOR_DEFAULT’ (‘1’): default memory + allocators + + * ‘PYMEM_ALLOCATOR_DEBUG’ (‘2’): default memory allocators + with debug hooks + + * ‘PYMEM_ALLOCATOR_MALLOC’ (‘3’): force usage of ‘malloc()’ + + * ‘PYMEM_ALLOCATOR_MALLOC_DEBUG’ (‘4’): force usage of + ‘malloc()’ with debug hooks + + * ‘PYMEM_ALLOCATOR_PYMALLOC’ (‘5’): *note Python pymalloc + memory allocator: 5c1. + + * ‘PYMEM_ALLOCATOR_PYMALLOC_DEBUG’ (‘6’): *note Python + pymalloc memory allocator: 5c1. with debug hooks + + ‘PYMEM_ALLOCATOR_PYMALLOC’ and + ‘PYMEM_ALLOCATOR_PYMALLOC_DEBUG’ are not supported if Python + is configured using ‘--without-pymalloc’ + + See *note Memory Management: 3d3d. + + -- C Member: int configure_locale + + Set the LC_CTYPE locale to the user preferred locale? If + equals to 0, set ‘coerce_c_locale’ and ‘coerce_c_locale_warn’ + to 0. + + -- C Member: int coerce_c_locale + + If equals to 2, coerce the C locale; if equals to 1, read the + LC_CTYPE locale to decide if it should be coerced. + + -- C Member: int coerce_c_locale_warn + + If non-zero, emit a warning if the C locale is coerced. + + -- C Member: int dev_mode + + See *note PyConfig.dev_mode: 3d42. + + -- C Member: int isolated + + See *note PyConfig.isolated: 3d44. + + -- C Member: int legacy_windows_fs_encoding (Windows only) + + If non-zero, disable UTF-8 Mode, set the Python filesystem + encoding to ‘mbcs’, set the filesystem error handler to + ‘replace’. + + Only available on Windows. ‘#ifdef MS_WINDOWS’ macro can be + used for Windows specific code. + + -- C Member: int parse_argv + + If non-zero, *note Py_PreInitializeFromArgs(): 17a. and *note + Py_PreInitializeFromBytesArgs(): 17b. parse their ‘argv’ + argument the same way the regular Python parses command line + arguments: see *note Command Line Arguments: 92b. + + -- C Member: int use_environment + + See *note PyConfig.use_environment: 3d48. + + -- C Member: int utf8_mode + + If non-zero, enable the UTF-8 mode. + + +File: python.info, Node: Preinitialization with PyPreConfig, Next: PyConfig, Prev: PyPreConfig, Up: Python Initialization Configuration + +7.10.4 Preinitialization with PyPreConfig +----------------------------------------- + +Functions to preinitialize Python: + + -- C Function: PyStatus Py_PreInitialize (const PyPreConfig *preconfig) + + Preinitialize Python from `preconfig' preconfiguration. + + -- C Function: PyStatus Py_PreInitializeFromBytesArgs (const + PyPreConfig *preconfig, int argc, char * const *argv) + + Preinitialize Python from `preconfig' preconfiguration and command + line arguments (bytes strings). + + -- C Function: PyStatus Py_PreInitializeFromArgs (const + PyPreConfig *preconfig, int argc, wchar_t * const * argv) + + Preinitialize Python from `preconfig' preconfiguration and command + line arguments (wide strings). + +The caller is responsible to handle exceptions (error or exit) using +*note PyStatus_Exception(): 16e. and *note Py_ExitStatusException(): +177. + +For *note Python Configuration: 3d3a. (*note +PyPreConfig_InitPythonConfig(): 16c.), if Python is initialized with +command line arguments, the command line arguments must also be passed +to preinitialize Python, since they have an effect on the +pre-configuration like encodings. For example, the *note -X utf8: 155. +command line option enables the UTF-8 Mode. + +‘PyMem_SetAllocator()’ can be called after *note Py_PreInitialize(): +179. and before *note Py_InitializeFromConfig(): 178. to install a +custom memory allocator. It can be called before *note +Py_PreInitialize(): 179. if *note PyPreConfig.allocator: 3d3c. is set to +‘PYMEM_ALLOCATOR_NOT_SET’. + +Python memory allocation functions like *note PyMem_RawMalloc(): 96d. +must not be used before Python preinitialization, whereas calling +directly ‘malloc()’ and ‘free()’ is always safe. *note +Py_DecodeLocale(): 43c. must not be called before the preinitialization. + +Example using the preinitialization to enable the UTF-8 Mode: + + PyStatus status; + PyPreConfig preconfig; + PyPreConfig_InitPythonConfig(&preconfig); + + preconfig.utf8_mode = 1; + + status = Py_PreInitialize(&preconfig); + if (PyStatus_Exception(status)) { + Py_ExitStatusException(status); + } + + /* at this point, Python will speak UTF-8 */ + + Py_Initialize(); + /* ... use Python API here ... */ + Py_Finalize(); + + +File: python.info, Node: PyConfig, Next: Initialization with PyConfig, Prev: Preinitialization with PyPreConfig, Up: Python Initialization Configuration + +7.10.5 PyConfig +--------------- + + -- C Type: PyConfig + + Structure containing most parameters to configure Python. + + Structure methods: + + -- C Function: void PyConfig_InitPythonConfig (PyConfig *config) + + Initialize configuration with *note Python Configuration: + 3d3a. + + -- C Function: void PyConfig_InitIsolatedConfig (PyConfig *config) + + Initialize configuration with *note Isolated Configuration: + 3d3b. + + -- C Function: PyStatus PyConfig_SetString (PyConfig *config, + wchar_t * const *config_str, const wchar_t *str) + + Copy the wide character string `str' into ‘*config_str’. + + Preinitialize Python if needed. + + -- C Function: PyStatus PyConfig_SetBytesString (PyConfig *config, + wchar_t * const *config_str, const char *str) + + Decode `str' using ‘Py_DecodeLocale()’ and set the result into + ‘*config_str’. + + Preinitialize Python if needed. + + -- C Function: PyStatus PyConfig_SetArgv (PyConfig *config, + int argc, wchar_t * const *argv) + + Set command line arguments from wide character strings. + + Preinitialize Python if needed. + + -- C Function: PyStatus PyConfig_SetBytesArgv (PyConfig *config, + int argc, char * const *argv) + + Set command line arguments: decode bytes using *note + Py_DecodeLocale(): 43c. + + Preinitialize Python if needed. + + -- C Function: PyStatus PyConfig_SetWideStringList + (PyConfig *config, PyWideStringList *list, + Py_ssize_t length, wchar_t **items) + + Set the list of wide strings `list' to `length' and `items'. + + Preinitialize Python if needed. + + -- C Function: PyStatus PyConfig_Read (PyConfig *config) + + Read all Python configuration. + + Fields which are already initialized are left unchanged. + + Preinitialize Python if needed. + + -- C Function: void PyConfig_Clear (PyConfig *config) + + Release configuration memory. + + Most ‘PyConfig’ methods preinitialize Python if needed. In that + case, the Python preinitialization configuration in based on the + *note PyConfig: 15f. If configuration fields which are in common + with *note PyPreConfig: 160. are tuned, they must be set before + calling a *note PyConfig: 15f. method: + + * *note dev_mode: 3d42. + + * *note isolated: 3d44. + + * *note parse_argv: 3d4c. + + * *note use_environment: 3d48. + + Moreover, if *note PyConfig_SetArgv(): 167. or *note + PyConfig_SetBytesArgv(): 168. is used, this method must be called + first, before other methods, since the preinitialization + configuration depends on command line arguments (if ‘parse_argv’ is + non-zero). + + The caller of these methods is responsible to handle exceptions + (error or exit) using ‘PyStatus_Exception()’ and + ‘Py_ExitStatusException()’. + + Structure fields: + + -- C Member: PyWideStringList argv + + Command line arguments, *note sys.argv: bfb. See *note + parse_argv: 3d4c. to parse *note argv: 3d4d. the same way the + regular Python parses Python command line arguments. If *note + argv: 3d4d. is empty, an empty string is added to ensure that + *note sys.argv: bfb. always exists and is never empty. + + -- C Member: wchar_t* base_exec_prefix + + *note sys.base_exec_prefix: 3324. + + -- C Member: wchar_t* base_executable + + ‘sys._base_executable’: ‘__PYVENV_LAUNCHER__’ environment + variable value, or copy of *note PyConfig.executable: 3d50. + + -- C Member: wchar_t* base_prefix + + *note sys.base_prefix: 2d50. + + -- C Member: int buffered_stdio + + If equals to 0, enable unbuffered mode, making the stdout and + stderr streams unbuffered. + + stdin is always opened in buffered mode. + + -- C Member: int bytes_warning + + If equals to 1, issue a warning when comparing *note bytes: + 331. or *note bytearray: 332. with *note str: 330, or + comparing *note bytes: 331. with *note int: 184. If equal or + greater to 2, raise a *note BytesWarning: 4b5. exception. + + -- C Member: wchar_t* check_hash_pycs_mode + + Control the validation behavior of hash-based ‘.pyc’ files + (see PEP 552(1)): *note –check-hash-based-pycs: fd9. command + line option value. + + Valid values: ‘always’, ‘never’ and ‘default’. + + The default value is: ‘default’. + + -- C Member: int configure_c_stdio + + If non-zero, configure C standard streams (‘stdio’, ‘stdout’, + ‘stdout’). For example, set their mode to ‘O_BINARY’ on + Windows. + + -- C Member: int dev_mode + + Development mode: see *note -X dev: 155. + + -- C Member: int dump_refs + + If non-zero, dump all objects which are still alive at exit. + + Require a debug build of Python (‘Py_REF_DEBUG’ macro must be + defined). + + -- C Member: wchar_t* exec_prefix + + *note sys.exec_prefix: 3323. + + -- C Member: wchar_t* executable + + *note sys.executable: 274. + + -- C Member: int faulthandler + + If non-zero, call *note faulthandler.enable(): 548. at + startup. + + -- C Member: wchar_t* filesystem_encoding + + Filesystem encoding, *note sys.getfilesystemencoding(): 4f6. + + -- C Member: wchar_t* filesystem_errors + + Filesystem encoding errors, *note + sys.getfilesystemencodeerrors(): 594. + + -- C Member: unsigned long hash_seed + + -- C Member: int use_hash_seed + + Randomized hash function seed. + + If *note use_hash_seed: 3d5c. is zero, a seed is chosen + randomly at Pythonstartup, and *note hash_seed: 3d5b. is + ignored. + + -- C Member: wchar_t* home + + Python home directory. + + Initialized from *note PYTHONHOME: 4d6. environment variable + value by default. + + -- C Member: int import_time + + If non-zero, profile import time. + + -- C Member: int inspect + + Enter interactive mode after executing a script or a command. + + -- C Member: int install_signal_handlers + + Install signal handlers? + + -- C Member: int interactive + + Interactive mode. + + -- C Member: int isolated + + If greater than 0, enable isolated mode: + + * *note sys.path: 488. contains neither the script’s + directory (computed from ‘argv[0]’ or the current + directory) nor the user’s site-packages directory. + + * Python REPL doesn’t import *note readline: de. nor enable + default readline configuration on interactive prompts. + + * Set *note use_environment: 3d48. and *note + user_site_directory: 3d62. to 0. + + -- C Member: int legacy_windows_stdio + + If non-zero, use *note io.FileIO: ca4. instead of + ‘io.WindowsConsoleIO’ for *note sys.stdin: 30d, *note + sys.stdout: 30e. and *note sys.stderr: 30f. + + Only available on Windows. ‘#ifdef MS_WINDOWS’ macro can be + used for Windows specific code. + + -- C Member: int malloc_stats + + If non-zero, dump statistics on *note Python pymalloc memory + allocator: 5c1. at exit. + + The option is ignored if Python is built using + ‘--without-pymalloc’. + + -- C Member: wchar_t* pythonpath_env + + Module search paths as a string separated by ‘DELIM’ + (‘os.path.pathsep’). + + Initialized from *note PYTHONPATH: 953. environment variable + value by default. + + -- C Member: PyWideStringList module_search_paths + + -- C Member: int module_search_paths_set + + *note sys.path: 488. If *note module_search_paths_set: 3d67. + is equal to 0, the *note module_search_paths: 3d66. is + overridden by the function calculating the *note Path + Configuration: 3d68. + + -- C Member: int optimization_level + + Compilation optimization level: + + * 0: Peephole optimizer (and ‘__debug__’ is set to ‘True’) + + * 1: Remove assertions, set ‘__debug__’ to ‘False’ + + * 2: Strip docstrings + + -- C Member: int parse_argv + + If non-zero, parse *note argv: 3d4d. the same way the regular + Python command line arguments, and strip Python arguments from + *note argv: 3d4d.: see *note Command Line Arguments: 92b. + + -- C Member: int parser_debug + + If non-zero, turn on parser debugging output (for expert only, + depending on compilation options). + + -- C Member: int pathconfig_warnings + + If equal to 0, suppress warnings when calculating the *note + Path Configuration: 3d68. (Unix only, Windows does not log any + warning). Otherwise, warnings are written into ‘stderr’. + + -- C Member: wchar_t* prefix + + *note sys.prefix: 3322. + + -- C Member: wchar_t* program_name + + Program name. Used to initialize *note executable: 3d50, and + in early error messages. + + -- C Member: wchar_t* pycache_prefix + + *note sys.pycache_prefix: 156.: ‘.pyc’ cache prefix. + + If ‘NULL’, *note sys.pycache_prefix: 156. is set to ‘None’. + + -- C Member: int quiet + + Quiet mode. For example, don’t display the copyright and + version messages in interactive mode. + + -- C Member: wchar_t* run_command + + ‘python3 -c COMMAND’ argument. Used by *note Py_RunMain(): + 17c. + + -- C Member: wchar_t* run_filename + + ‘python3 FILENAME’ argument. Used by *note Py_RunMain(): 17c. + + -- C Member: wchar_t* run_module + + ‘python3 -m MODULE’ argument. Used by *note Py_RunMain(): + 17c. + + -- C Member: int show_alloc_count + + Show allocation counts at exit? + + Set to 1 by *note -X showalloccount: 155. command line option. + + Need a special Python build with ‘COUNT_ALLOCS’ macro defined. + + -- C Member: int show_ref_count + + Show total reference count at exit? + + Set to 1 by *note -X showrefcount: 155. command line option. + + Need a debug build of Python (‘Py_REF_DEBUG’ macro must be + defined). + + -- C Member: int site_import + + Import the *note site: eb. module at startup? + + -- C Member: int skip_source_first_line + + Skip the first line of the source? + + -- C Member: wchar_t* stdio_encoding + + -- C Member: wchar_t* stdio_errors + + Encoding and encoding errors of *note sys.stdin: 30d, *note + sys.stdout: 30e. and *note sys.stderr: 30f. + + -- C Member: int tracemalloc + + If non-zero, call *note tracemalloc.start(): fea. at startup. + + -- C Member: int use_environment + + If greater than 0, use *note environment variables: fef. + + -- C Member: int user_site_directory + + If non-zero, add user site directory to *note sys.path: 488. + + -- C Member: int verbose + + If non-zero, enable verbose mode. + + -- C Member: PyWideStringList warnoptions + + *note sys.warnoptions: 416.: options of the *note warnings: + 126. module to build warnings filters: lowest to highest + priority. + + The *note warnings: 126. module adds *note sys.warnoptions: + 416. in the reverse order: the last *note + PyConfig.warnoptions: 3d7b. item becomes the first item of + ‘warnings.filters’ which is checked first (highest priority). + + -- C Member: int write_bytecode + + If non-zero, write ‘.pyc’ files. + + *note sys.dont_write_bytecode: 3351. is initialized to the + inverted value of *note write_bytecode: 3d7c. + + -- C Member: PyWideStringList xoptions + + *note sys._xoptions: fec. + +If ‘parse_argv’ is non-zero, ‘argv’ arguments are parsed the same way +the regular Python parses command line arguments, and Python arguments +are stripped from ‘argv’: see *note Command Line Arguments: 92b. + +The ‘xoptions’ options are parsed to set other options: see *note -X: +155. option. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0552 + + +File: python.info, Node: Initialization with PyConfig, Next: Isolated Configuration, Prev: PyConfig, Up: Python Initialization Configuration + +7.10.6 Initialization with PyConfig +----------------------------------- + +Function to initialize Python: + + -- C Function: PyStatus Py_InitializeFromConfig (const + PyConfig *config) + + Initialize Python from `config' configuration. + +The caller is responsible to handle exceptions (error or exit) using +*note PyStatus_Exception(): 16e. and *note Py_ExitStatusException(): +177. + +If ‘PyImport_FrozenModules’, ‘PyImport_AppendInittab()’ or +‘PyImport_ExtendInittab()’ are used, they must be set or called after +Python preinitialization and before the Python initialization. + +Example setting the program name: + + void init_python(void) + { + PyStatus status; + + PyConfig config; + PyConfig_InitPythonConfig(&config); + + /* Set the program name. Implicitly preinitialize Python. */ + status = PyConfig_SetString(&config, &config.program_name, + L"/path/to/my_program"); + if (PyStatus_Exception(status)) { + goto fail; + } + + status = Py_InitializeFromConfig(&config); + if (PyStatus_Exception(status)) { + goto fail; + } + PyConfig_Clear(&config); + return; + + fail: + PyConfig_Clear(&config); + Py_ExitStatusException(status); + } + +More complete example modifying the default configuration, read the +configuration, and then override some parameters: + + PyStatus init_python(const char *program_name) + { + PyStatus status; + + PyConfig config; + PyConfig_InitPythonConfig(&config); + + /* Set the program name before reading the configuration + (decode byte string from the locale encoding). + + Implicitly preinitialize Python. */ + status = PyConfig_SetBytesString(&config, &config.program_name, + program_name); + if (PyStatus_Exception(status)) { + goto done; + } + + /* Read all configuration at once */ + status = PyConfig_Read(&config); + if (PyStatus_Exception(status)) { + goto done; + } + + /* Append our custom search path to sys.path */ + status = PyWideStringList_Append(&config.module_search_paths, + L"/path/to/more/modules"); + if (PyStatus_Exception(status)) { + goto done; + } + + /* Override executable computed by PyConfig_Read() */ + status = PyConfig_SetString(&config, &config.executable, + L"/path/to/my_executable"); + if (PyStatus_Exception(status)) { + goto done; + } + + status = Py_InitializeFromConfig(&config); + + done: + PyConfig_Clear(&config); + return status; + } + + +File: python.info, Node: Isolated Configuration, Next: Python Configuration, Prev: Initialization with PyConfig, Up: Python Initialization Configuration + +7.10.7 Isolated Configuration +----------------------------- + +*note PyPreConfig_InitIsolatedConfig(): 16b. and *note +PyConfig_InitIsolatedConfig(): 164. functions create a configuration to +isolate Python from the system. For example, to embed Python into an +application. + +This configuration ignores global configuration variables, environments +variables, command line arguments (*note PyConfig.argv: 3d4d. is not +parsed) and user site directory. The C standard streams (ex: ‘stdout’) +and the LC_CTYPE locale are left unchanged. Signal handlers are not +installed. + +Configuration files are still used with this configuration. Set the +*note Path Configuration: 3d68. (“output fields”) to ignore these +configuration files and avoid the function computing the default path +configuration. + + +File: python.info, Node: Python Configuration, Next: Path Configuration, Prev: Isolated Configuration, Up: Python Initialization Configuration + +7.10.8 Python Configuration +--------------------------- + +*note PyPreConfig_InitPythonConfig(): 16c. and *note +PyConfig_InitPythonConfig(): 165. functions create a configuration to +build a customized Python which behaves as the regular Python. + +Environments variables and command line arguments are used to configure +Python, whereas global configuration variables are ignored. + +This function enables C locale coercion ( PEP 538(1)) and UTF-8 Mode ( +PEP 540(2)) depending on the LC_CTYPE locale, *note PYTHONUTF8: 311. and +*note PYTHONCOERCECLOCALE: 30c. environment variables. + +Example of customized Python always running in isolated mode: + + int main(int argc, char **argv) + { + PyStatus status; + + PyConfig config; + PyConfig_InitPythonConfig(&config); + config.isolated = 1; + + /* Decode command line arguments. + Implicitly preinitialize Python (in isolated mode). */ + status = PyConfig_SetBytesArgv(&config, argc, argv); + if (PyStatus_Exception(status)) { + goto fail; + } + + status = Py_InitializeFromConfig(&config); + if (PyStatus_Exception(status)) { + goto fail; + } + PyConfig_Clear(&config); + + return Py_RunMain(); + + fail: + PyConfig_Clear(&config); + if (PyStatus_IsExit(status)) { + return status.exitcode; + } + /* Display the error message and exit the process with + non-zero exit code */ + Py_ExitStatusException(status); + } + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0538 + + (2) https://www.python.org/dev/peps/pep-0540 + + +File: python.info, Node: Path Configuration, Next: Py_RunMain, Prev: Python Configuration, Up: Python Initialization Configuration + +7.10.9 Path Configuration +------------------------- + +*note PyConfig: 15f. contains multiple fields for the path +configuration: + + * Path configuration inputs: + + * *note PyConfig.home: 3d5d. + + * *note PyConfig.pathconfig_warnings: 3d6b. + + * *note PyConfig.program_name: 3d6d. + + * *note PyConfig.pythonpath_env: 3d65. + + * current working directory: to get absolute paths + + * ‘PATH’ environment variable to get the program full path (from + *note PyConfig.program_name: 3d6d.) + + * ‘__PYVENV_LAUNCHER__’ environment variable + + * (Windows only) Application paths in the registry under + “SoftwarePythonPythonCoreX.YPythonPath” of HKEY_CURRENT_USER + and HKEY_LOCAL_MACHINE (where X.Y is the Python version). + + * Path configuration output fields: + + * *note PyConfig.base_exec_prefix: 3d4e. + + * *note PyConfig.base_executable: 3d4f. + + * *note PyConfig.base_prefix: 3d51. + + * *note PyConfig.exec_prefix: 3d57. + + * *note PyConfig.executable: 3d50. + + * *note PyConfig.module_search_paths_set: 3d67, *note + PyConfig.module_search_paths: 3d66. + + * *note PyConfig.prefix: 3d6c. + +If at least one “output field” is not set, Python calculates the path +configuration to fill unset fields. If *note module_search_paths_set: +3d67. is equal to 0, *note module_search_paths: 3d66. is overridden and +*note module_search_paths_set: 3d67. is set to 1. + +It is possible to completely ignore the function calculating the default +path configuration by setting explicitly all path configuration output +fields listed above. A string is considered as set even if it is +non-empty. ‘module_search_paths’ is considered as set if +‘module_search_paths_set’ is set to 1. In this case, path configuration +input fields are ignored as well. + +Set *note pathconfig_warnings: 3d6b. to 0 to suppress warnings when +calculating the path configuration (Unix only, Windows does not log any +warning). + +If *note base_prefix: 3d51. or *note base_exec_prefix: 3d4e. fields are +not set, they inherit their value from *note prefix: 3d6c. and *note +exec_prefix: 3d57. respectively. + +*note Py_RunMain(): 17c. and *note Py_Main(): 4b7. modify *note +sys.path: 488.: + + * If *note run_filename: 3d71. is set and is a directory which + contains a ‘__main__.py’ script, prepend *note run_filename: 3d71. + to *note sys.path: 488. + + * If *note isolated: 3d44. is zero: + + * If *note run_module: 3d72. is set, prepend the current + directory to *note sys.path: 488. Do nothing if the current + directory cannot be read. + + * If *note run_filename: 3d71. is set, prepend the directory of + the filename to *note sys.path: 488. + + * Otherwise, prepend an empty string to *note sys.path: 488. + +If *note site_import: 3d75. is non-zero, *note sys.path: 488. can be +modified by the *note site: eb. module. If *note user_site_directory: +3d62. is non-zero and the user’s site-package directory exists, the +*note site: eb. module appends the user’s site-package directory to +*note sys.path: 488. + +The following configuration files are used by the path configuration: + + * ‘pyvenv.cfg’ + + * ‘python._pth’ (Windows only) + + * ‘pybuilddir.txt’ (Unix only) + +The ‘__PYVENV_LAUNCHER__’ environment variable is used to set *note +PyConfig.base_executable: 3d4f. + + +File: python.info, Node: Py_RunMain, Next: Multi-Phase Initialization Private Provisional API, Prev: Path Configuration, Up: Python Initialization Configuration + +7.10.10 Py_RunMain() +-------------------- + + -- C Function: int Py_RunMain (void) + + Execute the command (*note PyConfig.run_command: 3d70.), the script + (*note PyConfig.run_filename: 3d71.) or the module (*note + PyConfig.run_module: 3d72.) specified on the command line or in the + configuration. + + By default and when if *note -i: e1f. option is used, run the REPL. + + Finally, finalizes Python and returns an exit status that can be + passed to the ‘exit()’ function. + +See *note Python Configuration: 3d3a. for an example of customized +Python always running in isolated mode using *note Py_RunMain(): 17c. + + +File: python.info, Node: Multi-Phase Initialization Private Provisional API, Prev: Py_RunMain, Up: Python Initialization Configuration + +7.10.11 Multi-Phase Initialization Private Provisional API +---------------------------------------------------------- + +This section is a private provisional API introducing multi-phase +initialization, the core feature of the PEP 432(1): + + * “Core” initialization phase, “bare minimum Python”: + + * Builtin types; + + * Builtin exceptions; + + * Builtin and frozen modules; + + * The *note sys: fd. module is only partially initialized (ex: + *note sys.path: 488. doesn’t exist yet). + + * “Main” initialization phase, Python is fully initialized: + + * Install and configure *note importlib: 9b.; + + * Apply the *note Path Configuration: 3d68.; + + * Install signal handlers; + + * Finish *note sys: fd. module initialization (ex: create *note + sys.stdout: 30e. and *note sys.path: 488.); + + * Enable optional features like *note faulthandler: 7d. and + *note tracemalloc: 114.; + + * Import the *note site: eb. module; + + * etc. + +Private provisional API: + + * ‘PyConfig._init_main’: if set to 0, *note + Py_InitializeFromConfig(): 178. stops at the “Core” initialization + phase. + + -- C Function: PyStatus _Py_InitializeMain (void) + + Move to the “Main” initialization phase, finish the Python + initialization. + +No module is imported during the “Core” phase and the ‘importlib’ module +is not configured: the *note Path Configuration: 3d68. is only applied +during the “Main” phase. It may allow to customize Python in Python to +override or tune the *note Path Configuration: 3d68, maybe install a +custom *note sys.meta_path: 5e6. importer or an import hook, etc. + +It may become possible to calculatin the *note Path Configuration: 3d68. +in Python, after the Core phase and before the Main phase, which is one +of the PEP 432(2) motivation. + +The “Core” phase is not properly defined: what should be and what should +not be available at this phase is not specified yet. The API is marked +as private and provisional: the API can be modified or even be removed +anytime until a proper public API is designed. + +Example running Python code between “Core” and “Main” initialization +phases: + + void init_python(void) + { + PyStatus status; + + PyConfig config; + PyConfig_InitPythonConfig(&config); + config._init_main = 0; + + /* ... customize 'config' configuration ... */ + + status = Py_InitializeFromConfig(&config); + PyConfig_Clear(&config); + if (PyStatus_Exception(status)) { + Py_ExitStatusException(status); + } + + /* Use sys.stderr because sys.stdout is only created + by _Py_InitializeMain() */ + int res = PyRun_SimpleString( + "import sys; " + "print('Run Python code before _Py_InitializeMain', " + "file=sys.stderr)"); + if (res < 0) { + exit(1); + } + + /* ... put more configuration code here ... */ + + status = _Py_InitializeMain(); + if (PyStatus_Exception(status)) { + Py_ExitStatusException(status); + } + } + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0432 + + (2) https://www.python.org/dev/peps/pep-0432 + + +File: python.info, Node: Memory Management, Next: Object Implementation Support, Prev: Python Initialization Configuration, Up: Python/C API Reference Manual + +7.11 Memory Management +====================== + +* Menu: + +* Overview: Overview<3>. +* Raw Memory Interface:: +* Memory Interface:: +* Object allocators:: +* Default Memory Allocators:: +* Customize Memory Allocators:: +* The pymalloc allocator:: +* tracemalloc C API:: +* Examples: Examples<34>. + + +File: python.info, Node: Overview<3>, Next: Raw Memory Interface, Up: Memory Management + +7.11.1 Overview +--------------- + +Memory management in Python involves a private heap containing all +Python objects and data structures. The management of this private heap +is ensured internally by the `Python memory manager'. The Python memory +manager has different components which deal with various dynamic storage +management aspects, like sharing, segmentation, preallocation or +caching. + +At the lowest level, a raw memory allocator ensures that there is enough +room in the private heap for storing all Python-related data by +interacting with the memory manager of the operating system. On top of +the raw memory allocator, several object-specific allocators operate on +the same heap and implement distinct memory management policies adapted +to the peculiarities of every object type. For example, integer objects +are managed differently within the heap than strings, tuples or +dictionaries because integers imply different storage requirements and +speed/space tradeoffs. The Python memory manager thus delegates some of +the work to the object-specific allocators, but ensures that the latter +operate within the bounds of the private heap. + +It is important to understand that the management of the Python heap is +performed by the interpreter itself and that the user has no control +over it, even if they regularly manipulate object pointers to memory +blocks inside that heap. The allocation of heap space for Python +objects and other internal buffers is performed on demand by the Python +memory manager through the Python/C API functions listed in this +document. + +To avoid memory corruption, extension writers should never try to +operate on Python objects with the functions exported by the C library: +‘malloc()’, ‘calloc()’, ‘realloc()’ and ‘free()’. This will result in +mixed calls between the C allocator and the Python memory manager with +fatal consequences, because they implement different algorithms and +operate on different heaps. However, one may safely allocate and +release memory blocks with the C library allocator for individual +purposes, as shown in the following example: + + PyObject *res; + char *buf = (char *) malloc(BUFSIZ); /* for I/O */ + + if (buf == NULL) + return PyErr_NoMemory(); + ...Do some I/O operation involving buf... + res = PyBytes_FromString(buf); + free(buf); /* malloc'ed */ + return res; + +In this example, the memory request for the I/O buffer is handled by the +C library allocator. The Python memory manager is involved only in the +allocation of the bytes object returned as a result. + +In most situations, however, it is recommended to allocate memory from +the Python heap specifically because the latter is under control of the +Python memory manager. For example, this is required when the +interpreter is extended with new object types written in C. Another +reason for using the Python heap is the desire to `inform' the Python +memory manager about the memory needs of the extension module. Even +when the requested memory is used exclusively for internal, +highly-specific purposes, delegating all memory requests to the Python +memory manager causes the interpreter to have a more accurate image of +its memory footprint as a whole. Consequently, under certain +circumstances, the Python memory manager may or may not trigger +appropriate actions, like garbage collection, memory compaction or other +preventive procedures. Note that by using the C library allocator as +shown in the previous example, the allocated memory for the I/O buffer +escapes completely the Python memory manager. + +See also +........ + +The *note PYTHONMALLOC: 503. environment variable can be used to +configure the memory allocators used by Python. + +The *note PYTHONMALLOCSTATS: ffa. environment variable can be used to +print statistics of the *note pymalloc memory allocator: 5c1. every time +a new pymalloc object arena is created, and on shutdown. + + +File: python.info, Node: Raw Memory Interface, Next: Memory Interface, Prev: Overview<3>, Up: Memory Management + +7.11.2 Raw Memory Interface +--------------------------- + +The following function sets are wrappers to the system allocator. These +functions are thread-safe, the *note GIL: 506. does not need to be held. + +The *note default raw memory allocator: ff8. uses the following +functions: ‘malloc()’, ‘calloc()’, ‘realloc()’ and ‘free()’; call +‘malloc(1)’ (or ‘calloc(1, 1)’) when requesting zero bytes. + +New in version 3.4. + + -- C Function: void* PyMem_RawMalloc (size_t n) + + Allocates `n' bytes and returns a pointer of type ‘void*’ to the + allocated memory, or ‘NULL’ if the request fails. + + Requesting zero bytes returns a distinct non-‘NULL’ pointer if + possible, as if ‘PyMem_RawMalloc(1)’ had been called instead. The + memory will not have been initialized in any way. + + -- C Function: void* PyMem_RawCalloc (size_t nelem, size_t elsize) + + Allocates `nelem' elements each whose size in bytes is `elsize' and + returns a pointer of type ‘void*’ to the allocated memory, or + ‘NULL’ if the request fails. The memory is initialized to zeros. + + Requesting zero elements or elements of size zero bytes returns a + distinct non-‘NULL’ pointer if possible, as if ‘PyMem_RawCalloc(1, + 1)’ had been called instead. + + New in version 3.5. + + -- C Function: void* PyMem_RawRealloc (void *p, size_t n) + + Resizes the memory block pointed to by `p' to `n' bytes. The + contents will be unchanged to the minimum of the old and the new + sizes. + + If `p' is ‘NULL’, the call is equivalent to ‘PyMem_RawMalloc(n)’; + else if `n' is equal to zero, the memory block is resized but is + not freed, and the returned pointer is non-‘NULL’. + + Unless `p' is ‘NULL’, it must have been returned by a previous call + to *note PyMem_RawMalloc(): 96d, *note PyMem_RawRealloc(): 96e. or + *note PyMem_RawCalloc(): 7a5. + + If the request fails, *note PyMem_RawRealloc(): 96e. returns ‘NULL’ + and `p' remains a valid pointer to the previous memory area. + + -- C Function: void PyMem_RawFree (void *p) + + Frees the memory block pointed to by `p', which must have been + returned by a previous call to *note PyMem_RawMalloc(): 96d, *note + PyMem_RawRealloc(): 96e. or *note PyMem_RawCalloc(): 7a5. + Otherwise, or if ‘PyMem_RawFree(p)’ has been called before, + undefined behavior occurs. + + If `p' is ‘NULL’, no operation is performed. + + +File: python.info, Node: Memory Interface, Next: Object allocators, Prev: Raw Memory Interface, Up: Memory Management + +7.11.3 Memory Interface +----------------------- + +The following function sets, modeled after the ANSI C standard, but +specifying behavior when requesting zero bytes, are available for +allocating and releasing memory from the Python heap. + +The *note default memory allocator: ff8. uses the *note pymalloc memory +allocator: 5c1. + + Warning: The *note GIL: 506. must be held when using these + functions. + +Changed in version 3.6: The default allocator is now pymalloc instead of +system ‘malloc()’. + + -- C Function: void* PyMem_Malloc (size_t n) + + Allocates `n' bytes and returns a pointer of type ‘void*’ to the + allocated memory, or ‘NULL’ if the request fails. + + Requesting zero bytes returns a distinct non-‘NULL’ pointer if + possible, as if ‘PyMem_Malloc(1)’ had been called instead. The + memory will not have been initialized in any way. + + -- C Function: void* PyMem_Calloc (size_t nelem, size_t elsize) + + Allocates `nelem' elements each whose size in bytes is `elsize' and + returns a pointer of type ‘void*’ to the allocated memory, or + ‘NULL’ if the request fails. The memory is initialized to zeros. + + Requesting zero elements or elements of size zero bytes returns a + distinct non-‘NULL’ pointer if possible, as if ‘PyMem_Calloc(1, 1)’ + had been called instead. + + New in version 3.5. + + -- C Function: void* PyMem_Realloc (void *p, size_t n) + + Resizes the memory block pointed to by `p' to `n' bytes. The + contents will be unchanged to the minimum of the old and the new + sizes. + + If `p' is ‘NULL’, the call is equivalent to ‘PyMem_Malloc(n)’; else + if `n' is equal to zero, the memory block is resized but is not + freed, and the returned pointer is non-‘NULL’. + + Unless `p' is ‘NULL’, it must have been returned by a previous call + to *note PyMem_Malloc(): 505, *note PyMem_Realloc(): 96f. or *note + PyMem_Calloc(): 7a6. + + If the request fails, *note PyMem_Realloc(): 96f. returns ‘NULL’ + and `p' remains a valid pointer to the previous memory area. + + -- C Function: void PyMem_Free (void *p) + + Frees the memory block pointed to by `p', which must have been + returned by a previous call to *note PyMem_Malloc(): 505, *note + PyMem_Realloc(): 96f. or *note PyMem_Calloc(): 7a6. Otherwise, or + if ‘PyMem_Free(p)’ has been called before, undefined behavior + occurs. + + If `p' is ‘NULL’, no operation is performed. + +The following type-oriented macros are provided for convenience. Note +that `TYPE' refers to any C type. + + -- C Function: TYPE* PyMem_New (TYPE, size_t n) + + Same as *note PyMem_Malloc(): 505, but allocates ‘(n * + sizeof(TYPE))’ bytes of memory. Returns a pointer cast to ‘TYPE*’. + The memory will not have been initialized in any way. + + -- C Function: TYPE* PyMem_Resize (void *p, TYPE, size_t n) + + Same as *note PyMem_Realloc(): 96f, but the memory block is resized + to ‘(n * sizeof(TYPE))’ bytes. Returns a pointer cast to ‘TYPE*’. + On return, `p' will be a pointer to the new memory area, or ‘NULL’ + in the event of failure. + + This is a C preprocessor macro; `p' is always reassigned. Save the + original value of `p' to avoid losing memory when handling errors. + + -- C Function: void PyMem_Del (void *p) + + Same as *note PyMem_Free(): da9. + +In addition, the following macro sets are provided for calling the +Python memory allocator directly, without involving the C API functions +listed above. However, note that their use does not preserve binary +compatibility across Python versions and is therefore deprecated in +extension modules. + + * ‘PyMem_MALLOC(size)’ + + * ‘PyMem_NEW(type, size)’ + + * ‘PyMem_REALLOC(ptr, size)’ + + * ‘PyMem_RESIZE(ptr, type, size)’ + + * ‘PyMem_FREE(ptr)’ + + * ‘PyMem_DEL(ptr)’ + + +File: python.info, Node: Object allocators, Next: Default Memory Allocators, Prev: Memory Interface, Up: Memory Management + +7.11.4 Object allocators +------------------------ + +The following function sets, modeled after the ANSI C standard, but +specifying behavior when requesting zero bytes, are available for +allocating and releasing memory from the Python heap. + +The *note default object allocator: ff8. uses the *note pymalloc memory +allocator: 5c1. + + Warning: The *note GIL: 506. must be held when using these + functions. + + -- C Function: void* PyObject_Malloc (size_t n) + + Allocates `n' bytes and returns a pointer of type ‘void*’ to the + allocated memory, or ‘NULL’ if the request fails. + + Requesting zero bytes returns a distinct non-‘NULL’ pointer if + possible, as if ‘PyObject_Malloc(1)’ had been called instead. The + memory will not have been initialized in any way. + + -- C Function: void* PyObject_Calloc (size_t nelem, size_t elsize) + + Allocates `nelem' elements each whose size in bytes is `elsize' and + returns a pointer of type ‘void*’ to the allocated memory, or + ‘NULL’ if the request fails. The memory is initialized to zeros. + + Requesting zero elements or elements of size zero bytes returns a + distinct non-‘NULL’ pointer if possible, as if ‘PyObject_Calloc(1, + 1)’ had been called instead. + + New in version 3.5. + + -- C Function: void* PyObject_Realloc (void *p, size_t n) + + Resizes the memory block pointed to by `p' to `n' bytes. The + contents will be unchanged to the minimum of the old and the new + sizes. + + If `p' is ‘NULL’, the call is equivalent to ‘PyObject_Malloc(n)’; + else if `n' is equal to zero, the memory block is resized but is + not freed, and the returned pointer is non-‘NULL’. + + Unless `p' is ‘NULL’, it must have been returned by a previous call + to *note PyObject_Malloc(): 508, *note PyObject_Realloc(): daa. or + *note PyObject_Calloc(): 7a7. + + If the request fails, *note PyObject_Realloc(): daa. returns ‘NULL’ + and `p' remains a valid pointer to the previous memory area. + + -- C Function: void PyObject_Free (void *p) + + Frees the memory block pointed to by `p', which must have been + returned by a previous call to *note PyObject_Malloc(): 508, *note + PyObject_Realloc(): daa. or *note PyObject_Calloc(): 7a7. + Otherwise, or if ‘PyObject_Free(p)’ has been called before, + undefined behavior occurs. + + If `p' is ‘NULL’, no operation is performed. + + +File: python.info, Node: Default Memory Allocators, Next: Customize Memory Allocators, Prev: Object allocators, Up: Memory Management + +7.11.5 Default Memory Allocators +-------------------------------- + +Default memory allocators: + +Configuration Name PyMem_RawMalloc PyMem_Malloc PyObject_Malloc + +--------------------------------------------------------------------------------------------------------------------------------------- + +Release build ‘"pymalloc"’ ‘malloc’ ‘pymalloc’ ‘pymalloc’ + + +Debug build ‘"pymalloc_debug"’ ‘malloc’ + debug ‘pymalloc’ + debug ‘pymalloc’ + debug + + +Release build, without pymalloc ‘"malloc"’ ‘malloc’ ‘malloc’ ‘malloc’ + + +Debug build, without pymalloc ‘"malloc_debug"’ ‘malloc’ + debug ‘malloc’ + debug ‘malloc’ + debug + + +Legend: + + * Name: value for *note PYTHONMALLOC: 503. environment variable + + * ‘malloc’: system allocators from the standard C library, C + functions: ‘malloc()’, ‘calloc()’, ‘realloc()’ and ‘free()’ + + * ‘pymalloc’: *note pymalloc memory allocator: 5c1. + + * “+ debug”: with debug hooks installed by *note + PyMem_SetupDebugHooks(): 50a. + + +File: python.info, Node: Customize Memory Allocators, Next: The pymalloc allocator, Prev: Default Memory Allocators, Up: Memory Management + +7.11.6 Customize Memory Allocators +---------------------------------- + +New in version 3.4. + + -- C Type: PyMemAllocatorEx + + Structure used to describe a memory block allocator. The structure + has four fields: + + Field Meaning + + ----------------------------------------------------------------------------------------------------------- + + ‘void *ctx’ user context passed as first argument + + + ‘void* malloc(void *ctx, size_t size)’ allocate a memory block + + + ‘void* calloc(void *ctx, size_t nelem, size_t elsize)’ allocate a memory block initialized with + zeros + + + ‘void* realloc(void *ctx, void *ptr, size_t new_size)’ allocate or resize a memory block + + + ‘void free(void *ctx, void *ptr)’ free a memory block + + + Changed in version 3.5: The ‘PyMemAllocator’ structure was renamed + to *note PyMemAllocatorEx: 7ca. and a new ‘calloc’ field was added. + + -- C Type: PyMemAllocatorDomain + + Enum used to identify an allocator domain. Domains: + + -- C Macro: PYMEM_DOMAIN_RAW + + Functions: + + * *note PyMem_RawMalloc(): 96d. + + * *note PyMem_RawRealloc(): 96e. + + * *note PyMem_RawCalloc(): 7a5. + + * *note PyMem_RawFree(): 398f. + + -- C Macro: PYMEM_DOMAIN_MEM + + Functions: + + * *note PyMem_Malloc(): 505, + + * *note PyMem_Realloc(): 96f. + + * *note PyMem_Calloc(): 7a6. + + * *note PyMem_Free(): da9. + + -- C Macro: PYMEM_DOMAIN_OBJ + + Functions: + + * *note PyObject_Malloc(): 508. + + * *note PyObject_Realloc(): daa. + + * *note PyObject_Calloc(): 7a7. + + * *note PyObject_Free(): 504. + + -- C Function: void PyMem_GetAllocator (PyMemAllocatorDomain domain, + PyMemAllocatorEx *allocator) + + Get the memory block allocator of the specified domain. + + -- C Function: void PyMem_SetAllocator (PyMemAllocatorDomain domain, + PyMemAllocatorEx *allocator) + + Set the memory block allocator of the specified domain. + + The new allocator must return a distinct non-‘NULL’ pointer when + requesting zero bytes. + + For the *note PYMEM_DOMAIN_RAW: ff9. domain, the allocator must be + thread-safe: the *note GIL: 506. is not held when the allocator is + called. + + If the new allocator is not a hook (does not call the previous + allocator), the *note PyMem_SetupDebugHooks(): 50a. function must + be called to reinstall the debug hooks on top on the new allocator. + + -- C Function: void PyMem_SetupDebugHooks (void) + + Setup hooks to detect bugs in the Python memory allocator + functions. + + Newly allocated memory is filled with the byte ‘0xCD’ + (‘CLEANBYTE’), freed memory is filled with the byte ‘0xDD’ + (‘DEADBYTE’). Memory blocks are surrounded by “forbidden bytes” + (‘FORBIDDENBYTE’: byte ‘0xFD’). + + Runtime checks: + + - Detect API violations, ex: *note PyObject_Free(): 504. called + on a buffer allocated by *note PyMem_Malloc(): 505. + + - Detect write before the start of the buffer (buffer underflow) + + - Detect write after the end of the buffer (buffer overflow) + + - Check that the *note GIL: 506. is held when allocator + functions of *note PYMEM_DOMAIN_OBJ: 507. (ex: *note + PyObject_Malloc(): 508.) and *note PYMEM_DOMAIN_MEM: 509. (ex: + *note PyMem_Malloc(): 505.) domains are called + + On error, the debug hooks use the *note tracemalloc: 114. module to + get the traceback where a memory block was allocated. The + traceback is only displayed if *note tracemalloc: 114. is tracing + Python memory allocations and the memory block was traced. + + These hooks are *note installed by default: ff8. if Python is + compiled in debug mode. The *note PYTHONMALLOC: 503. environment + variable can be used to install debug hooks on a Python compiled in + release mode. + + Changed in version 3.6: This function now also works on Python + compiled in release mode. On error, the debug hooks now use *note + tracemalloc: 114. to get the traceback where a memory block was + allocated. The debug hooks now also check if the GIL is held when + functions of *note PYMEM_DOMAIN_OBJ: 507. and *note + PYMEM_DOMAIN_MEM: 509. domains are called. + + Changed in version 3.8: Byte patterns ‘0xCB’ (‘CLEANBYTE’), ‘0xDB’ + (‘DEADBYTE’) and ‘0xFB’ (‘FORBIDDENBYTE’) have been replaced with + ‘0xCD’, ‘0xDD’ and ‘0xFD’ to use the same values than Windows CRT + debug ‘malloc()’ and ‘free()’. + + +File: python.info, Node: The pymalloc allocator, Next: tracemalloc C API, Prev: Customize Memory Allocators, Up: Memory Management + +7.11.7 The pymalloc allocator +----------------------------- + +Python has a `pymalloc' allocator optimized for small objects (smaller +or equal to 512 bytes) with a short lifetime. It uses memory mappings +called “arenas” with a fixed size of 256 KiB. It falls back to *note +PyMem_RawMalloc(): 96d. and *note PyMem_RawRealloc(): 96e. for +allocations larger than 512 bytes. + +`pymalloc' is the *note default allocator: ff8. of the *note +PYMEM_DOMAIN_MEM: 509. (ex: *note PyMem_Malloc(): 505.) and *note +PYMEM_DOMAIN_OBJ: 507. (ex: *note PyObject_Malloc(): 508.) domains. + +The arena allocator uses the following functions: + + * ‘VirtualAlloc()’ and ‘VirtualFree()’ on Windows, + + * ‘mmap()’ and ‘munmap()’ if available, + + * ‘malloc()’ and ‘free()’ otherwise. + +* Menu: + +* Customize pymalloc Arena Allocator:: + + +File: python.info, Node: Customize pymalloc Arena Allocator, Up: The pymalloc allocator + +7.11.7.1 Customize pymalloc Arena Allocator +........................................... + +New in version 3.4. + + -- C Type: PyObjectArenaAllocator + + Structure used to describe an arena allocator. The structure has + three fields: + + Field Meaning + + --------------------------------------------------------------------------------------------------- + + ‘void *ctx’ user context passed as first argument + + + ‘void* alloc(void *ctx, size_t size)’ allocate an arena of size bytes + + + ‘void free(void *ctx, void *ptr, size_t size)’ free an arena + + + -- C Function: void PyObject_GetArenaAllocator + (PyObjectArenaAllocator *allocator) + + Get the arena allocator. + + -- C Function: void PyObject_SetArenaAllocator + (PyObjectArenaAllocator *allocator) + + Set the arena allocator. + + +File: python.info, Node: tracemalloc C API, Next: Examples<34>, Prev: The pymalloc allocator, Up: Memory Management + +7.11.8 tracemalloc C API +------------------------ + +New in version 3.7. + + -- C Function: int PyTraceMalloc_Track (unsigned int domain, + uintptr_t ptr, size_t size) + + Track an allocated memory block in the *note tracemalloc: 114. + module. + + Return ‘0’ on success, return ‘-1’ on error (failed to allocate + memory to store the trace). Return ‘-2’ if tracemalloc is + disabled. + + If memory block is already tracked, update the existing trace. + + -- C Function: int PyTraceMalloc_Untrack (unsigned int domain, + uintptr_t ptr) + + Untrack an allocated memory block in the *note tracemalloc: 114. + module. Do nothing if the block was not tracked. + + Return ‘-2’ if tracemalloc is disabled, otherwise return ‘0’. + + +File: python.info, Node: Examples<34>, Prev: tracemalloc C API, Up: Memory Management + +7.11.9 Examples +--------------- + +Here is the example from section *note Overview: 3d87, rewritten so that +the I/O buffer is allocated from the Python heap by using the first +function set: + + PyObject *res; + char *buf = (char *) PyMem_Malloc(BUFSIZ); /* for I/O */ + + if (buf == NULL) + return PyErr_NoMemory(); + /* ...Do some I/O operation involving buf... */ + res = PyBytes_FromString(buf); + PyMem_Free(buf); /* allocated with PyMem_Malloc */ + return res; + +The same code using the type-oriented function set: + + PyObject *res; + char *buf = PyMem_New(char, BUFSIZ); /* for I/O */ + + if (buf == NULL) + return PyErr_NoMemory(); + /* ...Do some I/O operation involving buf... */ + res = PyBytes_FromString(buf); + PyMem_Del(buf); /* allocated with PyMem_New */ + return res; + +Note that in the two examples above, the buffer is always manipulated +via functions belonging to the same set. Indeed, it is required to use +the same memory API family for a given memory block, so that the risk of +mixing different allocators is reduced to a minimum. The following code +sequence contains two errors, one of which is labeled as `fatal' because +it mixes two different allocators operating on different heaps. + + char *buf1 = PyMem_New(char, BUFSIZ); + char *buf2 = (char *) malloc(BUFSIZ); + char *buf3 = (char *) PyMem_Malloc(BUFSIZ); + ... + PyMem_Del(buf3); /* Wrong -- should be PyMem_Free() */ + free(buf2); /* Right -- allocated via malloc() */ + free(buf1); /* Fatal -- should be PyMem_Del() */ + +In addition to the functions aimed at handling raw memory blocks from +the Python heap, objects in Python are allocated and released with *note +PyObject_New(): 2d2, *note PyObject_NewVar(): 2d3. and *note +PyObject_Del(): e16. + +These will be explained in the next chapter on defining and implementing +new object types in C. + + +File: python.info, Node: Object Implementation Support, Next: API and ABI Versioning, Prev: Memory Management, Up: Python/C API Reference Manual + +7.12 Object Implementation Support +================================== + +This chapter describes the functions, types, and macros used when +defining new object types. + +* Menu: + +* Allocating Objects on the Heap:: +* Common Object Structures:: +* Type Objects: Type Objects<3>. +* Number Object Structures:: +* Mapping Object Structures:: +* Sequence Object Structures:: +* Buffer Object Structures:: +* Async Object Structures:: +* Slot Type typedefs:: +* Examples: Examples<35>. +* Supporting Cyclic Garbage Collection:: + + +File: python.info, Node: Allocating Objects on the Heap, Next: Common Object Structures, Up: Object Implementation Support + +7.12.1 Allocating Objects on the Heap +------------------------------------- + + -- C Function: PyObject* _PyObject_New (PyTypeObject *type) + `Return value: New reference.' + + -- C Function: PyVarObject* _PyObject_NewVar (PyTypeObject *type, + Py_ssize_t size) + `Return value: New reference.' + + -- C Function: PyObject* PyObject_Init (PyObject *op, + PyTypeObject *type) + `Return value: Borrowed reference.' Initialize a newly-allocated + object `op' with its type and initial reference. Returns the + initialized object. If `type' indicates that the object + participates in the cyclic garbage detector, it is added to the + detector’s set of observed objects. Other fields of the object are + not affected. + + -- C Function: PyVarObject* PyObject_InitVar (PyVarObject *op, + PyTypeObject *type, Py_ssize_t size) + `Return value: Borrowed reference.' This does everything *note + PyObject_Init(): 26f. does, and also initializes the length + information for a variable-size object. + + -- C Function: TYPE* PyObject_New (TYPE, PyTypeObject *type) + `Return value: New reference.' Allocate a new Python object using + the C structure type `TYPE' and the Python type object `type'. + Fields not defined by the Python object header are not initialized; + the object’s reference count will be one. The size of the memory + allocation is determined from the *note tp_basicsize: 3879. field + of the type object. + + -- C Function: TYPE* PyObject_NewVar (TYPE, PyTypeObject *type, + Py_ssize_t size) + `Return value: New reference.' Allocate a new Python object using + the C structure type `TYPE' and the Python type object `type'. + Fields not defined by the Python object header are not initialized. + The allocated memory allows for the `TYPE' structure plus `size' + fields of the size given by the *note tp_itemsize: 3878. field of + `type'. This is useful for implementing objects like tuples, which + are able to determine their size at construction time. Embedding + the array of fields into the same allocation decreases the number + of allocations, improving the memory management efficiency. + + -- C Function: void PyObject_Del (void *op) + + Releases memory allocated to an object using *note PyObject_New(): + 2d2. or *note PyObject_NewVar(): 2d3. This is normally called from + the *note tp_dealloc: 387f. handler specified in the object’s type. + The fields of the object should not be accessed after this call as + the memory is no longer a valid Python object. + + -- C Variable: PyObject _Py_NoneStruct + + Object which is visible in Python as ‘None’. This should only be + accessed using the *note Py_None: 3844. macro, which evaluates to a + pointer to this object. + +See also +........ + +*note PyModule_Create(): 3847. + + To allocate and create extension modules. + + +File: python.info, Node: Common Object Structures, Next: Type Objects<3>, Prev: Allocating Objects on the Heap, Up: Object Implementation Support + +7.12.2 Common Object Structures +------------------------------- + +There are a large number of structures which are used in the definition +of object types for Python. This section describes these structures and +how they are used. + +All Python objects ultimately share a small number of fields at the +beginning of the object’s representation in memory. These are +represented by the *note PyObject: 4ba. and *note PyVarObject: 3da6. +types, which are defined, in turn, by the expansions of some macros also +used, whether directly or indirectly, in the definition of all other +Python objects. + + -- C Type: PyObject + + All object types are extensions of this type. This is a type which + contains the information Python needs to treat a pointer to an + object as an object. In a normal “release” build, it contains only + the object’s reference count and a pointer to the corresponding + type object. Nothing is actually declared to be a *note PyObject: + 4ba, but every pointer to a Python object can be cast to a *note + PyObject*: 4ba. Access to the members must be done by using the + macros *note Py_REFCNT: 3875. and *note Py_TYPE: 3876. + + -- C Type: PyVarObject + + This is an extension of *note PyObject: 4ba. that adds the + ‘ob_size’ field. This is only used for objects that have some + notion of `length'. This type does not often appear in the + Python/C API. Access to the members must be done by using the + macros *note Py_REFCNT: 3875, *note Py_TYPE: 3876, and *note + Py_SIZE: 3da7. + + -- C Macro: PyObject_HEAD + + This is a macro used when declaring new types which represent + objects without a varying length. The PyObject_HEAD macro expands + to: + + PyObject ob_base; + + See documentation of *note PyObject: 4ba. above. + + -- C Macro: PyObject_VAR_HEAD + + This is a macro used when declaring new types which represent + objects with a length that varies from instance to instance. The + PyObject_VAR_HEAD macro expands to: + + PyVarObject ob_base; + + See documentation of *note PyVarObject: 3da6. above. + + -- C Macro: Py_TYPE (o) + + This macro is used to access the ‘ob_type’ member of a Python + object. It expands to: + + (((PyObject*)(o))->ob_type) + + -- C Macro: Py_REFCNT (o) + + This macro is used to access the ‘ob_refcnt’ member of a Python + object. It expands to: + + (((PyObject*)(o))->ob_refcnt) + + -- C Macro: Py_SIZE (o) + + This macro is used to access the ‘ob_size’ member of a Python + object. It expands to: + + (((PyVarObject*)(o))->ob_size) + + -- C Macro: PyObject_HEAD_INIT (type) + + This is a macro which expands to initialization values for a new + *note PyObject: 4ba. type. This macro expands to: + + _PyObject_EXTRA_INIT + 1, type, + + -- C Macro: PyVarObject_HEAD_INIT (type, size) + + This is a macro which expands to initialization values for a new + *note PyVarObject: 3da6. type, including the ‘ob_size’ field. This + macro expands to: + + _PyObject_EXTRA_INIT + 1, type, size, + + -- C Type: PyCFunction + + Type of the functions used to implement most Python callables in C. + Functions of this type take two *note PyObject*: 4ba. parameters + and return one such value. If the return value is ‘NULL’, an + exception shall have been set. If not ‘NULL’, the return value is + interpreted as the return value of the function as exposed in + Python. The function must return a new reference. + + -- C Type: PyCFunctionWithKeywords + + Type of the functions used to implement Python callables in C with + signature ‘METH_VARARGS | METH_KEYWORDS’. + + -- C Type: _PyCFunctionFast + + Type of the functions used to implement Python callables in C with + signature *note METH_FASTCALL: 3dad. + + -- C Type: _PyCFunctionFastWithKeywords + + Type of the functions used to implement Python callables in C with + signature ‘METH_FASTCALL | METH_KEYWORDS’. + + -- C Type: PyMethodDef + + Structure used to describe a method of an extension type. This + structure has four fields: + + Field C Type Meaning + + ------------------------------------------------------------------------------- + + ‘ml_name’ const char * name of the method + + + ‘ml_meth’ PyCFunction pointer to the C implementation + + + ‘ml_flags’ int flag bits indicating how the call + should be constructed + + + ‘ml_doc’ const char * points to the contents of the + docstring + + +The ‘ml_meth’ is a C function pointer. The functions may be of +different types, but they always return *note PyObject*: 4ba. If the +function is not of the *note PyCFunction: ddb, the compiler will require +a cast in the method table. Even though *note PyCFunction: ddb. defines +the first parameter as *note PyObject*: 4ba, it is common that the +method implementation uses the specific C type of the `self' object. + +The ‘ml_flags’ field is a bitfield which can include the following +flags. The individual flags indicate either a calling convention or a +binding convention. + +There are four basic calling conventions for positional arguments and +two of them can be combined with ‘METH_KEYWORDS’ to support also keyword +arguments. So there are a total of 6 calling conventions: + + -- Data: METH_VARARGS + + This is the typical calling convention, where the methods have the + type *note PyCFunction: ddb. The function expects two *note + PyObject*: 4ba. values. The first one is the `self' object for + methods; for module functions, it is the module object. The second + parameter (often called `args') is a tuple object representing all + arguments. This parameter is typically processed using *note + PyArg_ParseTuple(): 26a. or *note PyArg_UnpackTuple(): e46. + + -- Data: METH_VARARGS | METH_KEYWORDS + + Methods with these flags must be of type *note + PyCFunctionWithKeywords: 3dab. The function expects three + parameters: `self', `args', `kwargs' where `kwargs' is a dictionary + of all the keyword arguments or possibly ‘NULL’ if there are no + keyword arguments. The parameters are typically processed using + *note PyArg_ParseTupleAndKeywords(): 5ca. + + -- Data: METH_FASTCALL + + Fast calling convention supporting only positional arguments. The + methods have the type *note _PyCFunctionFast: 3dac. The first + parameter is `self', the second parameter is a C array of *note + PyObject*: 4ba. values indicating the arguments and the third + parameter is the number of arguments (the length of the array). + + This is not part of the *note limited API: 3905. + + New in version 3.7. + + -- Data: METH_FASTCALL | METH_KEYWORDS + + Extension of *note METH_FASTCALL: 3dad. supporting also keyword + arguments, with methods of type *note _PyCFunctionFastWithKeywords: + 3dae. Keyword arguments are passed the same way as in the + vectorcall protocol: there is an additional fourth *note PyObject*: + 4ba. parameter which is a tuple representing the names of the + keyword arguments or possibly ‘NULL’ if there are no keywords. The + values of the keyword arguments are stored in the `args' array, + after the positional arguments. + + This is not part of the *note limited API: 3905. + + New in version 3.7. + + -- Data: METH_NOARGS + + Methods without parameters don’t need to check whether arguments + are given if they are listed with the *note METH_NOARGS: e18. flag. + They need to be of type *note PyCFunction: ddb. The first + parameter is typically named `self' and will hold a reference to + the module or object instance. In all cases the second parameter + will be ‘NULL’. + + -- Data: METH_O + + Methods with a single object argument can be listed with the *note + METH_O: e47. flag, instead of invoking *note PyArg_ParseTuple(): + 26a. with a ‘"O"’ argument. They have the type *note PyCFunction: + ddb, with the `self' parameter, and a *note PyObject*: 4ba. + parameter representing the single argument. + +These two constants are not used to indicate the calling convention but +the binding when use with methods of classes. These may not be used for +functions defined for modules. At most one of these flags may be set +for any given method. + + -- Data: METH_CLASS + + The method will be passed the type object as the first parameter + rather than an instance of the type. This is used to create `class + methods', similar to what is created when using the *note + classmethod(): 1d8. built-in function. + + -- Data: METH_STATIC + + The method will be passed ‘NULL’ as the first parameter rather than + an instance of the type. This is used to create `static methods', + similar to what is created when using the *note staticmethod(): + 1d9. built-in function. + +One other constant controls whether a method is loaded in place of +another definition with the same method name. + + -- Data: METH_COEXIST + + The method will be loaded in place of existing definitions. + Without `METH_COEXIST', the default is to skip repeated + definitions. Since slot wrappers are loaded before the method + table, the existence of a `sq_contains' slot, for example, would + generate a wrapped method named *note __contains__(): d28. and + preclude the loading of a corresponding PyCFunction with the same + name. With the flag defined, the PyCFunction will be loaded in + place of the wrapper object and will co-exist with the slot. This + is helpful because calls to PyCFunctions are optimized more than + wrapper object calls. + + -- C Type: PyMemberDef + + Structure which describes an attribute of a type which corresponds + to a C struct member. Its fields are: + + Field C Type Meaning + + ------------------------------------------------------------------------------- + + ‘name’ const char * name of the member + + + ‘type’ int the type of the member in the C + struct + + + ‘offset’ Py_ssize_t the offset in bytes that the + member is located on the type’s + object struct + + + ‘flags’ int flag bits indicating if the field + should be read-only or writable + + + ‘doc’ const char * points to the contents of the + docstring + + + ‘type’ can be one of many ‘T_’ macros corresponding to various C + types. When the member is accessed in Python, it will be converted + to the equivalent Python type. + + Macro name C type + + ------------------------------------------- + + T_SHORT short + + + T_INT int + + + T_LONG long + + + T_FLOAT float + + + T_DOUBLE double + + + T_STRING const char * + + + T_OBJECT PyObject * + + + T_OBJECT_EX PyObject * + + + T_CHAR char + + + T_BYTE char + + + T_UBYTE unsigned char + + + T_UINT unsigned int + + + T_USHORT unsigned short + + + T_ULONG unsigned long + + + T_BOOL char + + + T_LONGLONG long long + + + T_ULONGLONG unsigned long long + + + T_PYSSIZET Py_ssize_t + + + ‘T_OBJECT’ and ‘T_OBJECT_EX’ differ in that ‘T_OBJECT’ returns + ‘None’ if the member is ‘NULL’ and ‘T_OBJECT_EX’ raises an *note + AttributeError: 39b. Try to use ‘T_OBJECT_EX’ over ‘T_OBJECT’ + because ‘T_OBJECT_EX’ handles use of the *note del: f02. statement + on that attribute more correctly than ‘T_OBJECT’. + + ‘flags’ can be ‘0’ for write and read access or ‘READONLY’ for + read-only access. Using ‘T_STRING’ for *note type: 608. implies + ‘READONLY’. ‘T_STRING’ data is interpreted as UTF-8. Only + ‘T_OBJECT’ and ‘T_OBJECT_EX’ members can be deleted. (They are set + to ‘NULL’). + + -- C Type: PyGetSetDef + + Structure to define property-like access for a type. See also + description of the *note PyTypeObject.tp_getset: 388a. slot. + + Field C Type Meaning + + --------------------------------------------------------------------------------- + + name const char * attribute name + + + get getter C Function to get the attribute + + + set setter optional C function to set or delete + the attribute, if omitted the + attribute is readonly + + + doc const char * optional docstring + + + closure void * optional function pointer, providing + additional data for getter and setter + + + The ‘get’ function takes one *note PyObject*: 4ba. parameter (the + instance) and a function pointer (the associated ‘closure’): + + typedef PyObject *(*getter)(PyObject *, void *); + + It should return a new reference on success or ‘NULL’ with a set + exception on failure. + + ‘set’ functions take two *note PyObject*: 4ba. parameters (the + instance and the value to be set) and a function pointer (the + associated ‘closure’): + + typedef int (*setter)(PyObject *, PyObject *, void *); + + In case the attribute should be deleted the second parameter is + ‘NULL’. Should return ‘0’ on success or ‘-1’ with a set exception + on failure. + + +File: python.info, Node: Type Objects<3>, Next: Number Object Structures, Prev: Common Object Structures, Up: Object Implementation Support + +7.12.3 Type Objects +------------------- + +Perhaps one of the most important structures of the Python object system +is the structure that defines a new type: the *note PyTypeObject: 2d6. +structure. Type objects can be handled using any of the ‘PyObject_*()’ +or ‘PyType_*()’ functions, but do not offer much that’s interesting to +most Python applications. These objects are fundamental to how objects +behave, so they are very important to the interpreter itself and to any +extension module that implements new types. + +Type objects are fairly large compared to most of the standard types. +The reason for the size is that each type object stores a large number +of values, mostly C function pointers, each of which implements a small +part of the type’s functionality. The fields of the type object are +examined in detail in this section. The fields will be described in the +order in which they occur in the structure. + +In addition to the following quick reference, the *note Examples: 3db2. +section provides at-a-glance insight into the meaning and use of *note +PyTypeObject: 2d6. + +* Menu: + +* Quick Reference:: +* PyTypeObject Definition:: +* PyObject Slots:: +* PyVarObject Slots:: +* PyTypeObject Slots:: +* Heap Types:: + + +File: python.info, Node: Quick Reference, Next: PyTypeObject Definition, Up: Type Objects<3> + +7.12.3.1 Quick Reference +........................ + +* Menu: + +* “tp slots”:: +* sub-slots:: +* slot typedefs:: + + +File: python.info, Node: “tp slots”, Next: sub-slots, Up: Quick Reference + +7.12.3.2 “tp slots” +................... + +PyTypeObject Slot *note Type: 3db6. special Info +(1) methods/attrs (2) + +--------------------------------------------------------------------------------------------- + +O T D I + + +<R> const char * __name__ X X +*note tp_name: 389b. + +*note tp_basicsize: 3879.Py_ssize_t X X X + + +*note tp_itemsize: 3878.Py_ssize_t X X + + +*note tp_dealloc: 387f.*note destructor: 3db7. X X X + + +*note tp_vectorcall_offset: 3a11.Py_ssize_t ? + + +(*note tp_getattr: 38a2.)*note getattrfunc: 3db8.__getattribute__, G + __getattr__ + + +(*note tp_setattr: 38a3.)*note setattrfunc: 3db9.__setattr__, G + __delattr__ + + +*note tp_as_async: 3dba.*note PyAsyncMethods: 3ac7.*note sub-slots: 3dbb. % + * + + +*note tp_repr: 389a. *note reprfunc: 3dbc. __repr__ X X X + + +*note tp_as_number: 3dbd.*note PyNumberMethods: d81.*note sub-slots: 3dbb. % + * + + +*note tp_as_sequence: 3dbe.*note PySequenceMethods: 38aa.*note sub-slots: 3dbb. % + * + + +*note tp_as_mapping: 3dbf.*note PyMappingMethods: 38ab.*note sub-slots: 3dbb. % + * + + +*note tp_hash: 38ac. *note hashfunc: 3dc0. __hash__ X G + + +*note tp_call: 38ad. *note ternaryfunc: 3dc1.__call__ X X + + +*note tp_str: 389c. *note reprfunc: 3dbc. __str__ X X + + +*note tp_getattro: 389f.*note getattrofunc: 3dc2.__getattribute__, X X G + __getattr__ + + +*note tp_setattro: 38a0.*note setattrofunc: 3dc3.__setattr__, X X G + __delattr__ + + +*note tp_as_buffer: 3dc4.*note PyBufferProcs: 3dc5. % + * + + +*note tp_flags: 3ab6. unsigned long X X ? + + +*note tp_doc: 387b. const char * __doc__ X X + + +*note tp_traverse: 33e8.*note traverseproc: 3dc6. X G + + +*note tp_clear: 3898. *note inquiry: 3dc7. X G + + +*note tp_richcompare: 38a5.*note richcmpfunc: 3dc8.__lt__, __le__, X G + __eq__, __ne__, + __gt__, __ge__ + + +*note tp_weaklistoffset: 38af.Py_ssize_t X ? + + +*note tp_iter: e30. *note getiterfunc: 3dc9.__iter__ X + + +*note tp_iternext: e31.*note iternextfunc: 3dca.__next__ X + + +*note tp_methods: 3886.*note PyMethodDef: e1b. X X + [] + + +*note tp_members: 3884.*note PyMemberDef: 426. X + [] + + +*note tp_getset: 388a. *note PyGetSetDef: 427. X X + [] + + +*note tp_base: 3890. *note PyTypeObject: 2d6.__base__ X + * + + +*note tp_dict: 3aca. *note PyObject: 4ba. __dict__ ? + * + + +*note tp_descr_get: 3dcb.*note descrgetfunc: 3dcc.__get__ X + + +*note tp_descr_set: 3dcd.*note descrsetfunc: 3dce.__set__, __delete__ X + + +*note tp_dictoffset: 3acf.Py_ssize_t X ? + + +*note tp_init: 3883. *note initproc: 3dcf. __init__ X X X + + +*note tp_alloc: 3881. *note allocfunc: 3dd0. X ? ? + + +*note tp_new: 387c. *note newfunc: 3dd1. __new__ X X ? ? + + +*note tp_free: 3880. *note freefunc: 3dd2. X X ? ? + + +*note tp_is_gc: 3dd3. *note inquiry: 3dc7. X X + + +<*note tp_bases: 3dd4.>*note PyObject: 4ba. __bases__ ~ + * + + +<*note tp_mro: 3acb.> *note PyObject: 4ba. __mro__ ~ + * + + +[*note tp_cache: 3acc.]*note PyObject: 4ba. + * + + +[*note tp_subclasses: 3acd.]*note PyObject: 4ba.__subclasses__ + * + + +[*note tp_weaklist: 3ace.]*note PyObject: 4ba. + * + + +(*note tp_del: 3dd5.) *note destructor: 3db7. + + +[*note tp_version_tag: 3dd6.]unsigned int + + +*note tp_finalize: 2d7.*note destructor: 3db7.__del__ X + + +If ‘COUNT_ALLOCS’ is defined then the following (internal-only) fields +exist as well: + + * *note tp_allocs: 3dd7. + + * *note tp_frees: 3dd8. + + * *note tp_maxalloc: 3dd9. + + * *note tp_prev: 3dda. + + * *note tp_next: 3ddb. + + ---------- Footnotes ---------- + + (1) (1) A slot name in parentheses indicates it is (effectively) +deprecated. Names in angle brackets should be treated as read-only. +Names in square brackets are for internal use only. “<R>” (as a prefix) +means the field is required (must be non-‘NULL’). + + (2) (2) Columns: + +`“O”': set on ‘PyBaseObject_Type’ + +`“T”': set on *note PyType_Type: 3ab1. + +`“D”': default (if slot is set to ‘NULL’) + + X - PyType_Ready sets this value if it is NULL + ~ - PyType_Ready always sets this value (it should be NULL) + ? - PyType_Ready may set this value depending on other slots + + Also see the inheritance column ("I"). + +`“I”': inheritance + + X - type slot is inherited via PyType_Ready if defined with a NULL value + % - the slots of the sub-struct are inherited individually + G - inherited, but only in combination with other slots; see the slot's description + ? - it's complicated; see the slot's description + +Note that some slots are effectively inherited through the normal +attribute lookup chain. + + +File: python.info, Node: sub-slots, Next: slot typedefs, Prev: “tp slots”, Up: Quick Reference + +7.12.3.3 sub-slots +.................. + +Slot *note Type: 3db6. special + methods + +---------------------------------------------------------------------- + +*note am_await: 3ddd. *note unaryfunc: 3dde.__await__ + + +*note am_aiter: 3ddf. *note unaryfunc: 3dde.__aiter__ + + +*note am_anext: 3de0. *note unaryfunc: 3dde.__anext__ + + + +*note nb_add: 3ac8. *note binaryfunc: 3de1.__add__ + __radd__ + + +*note nb_inplace_add: 3de2. *note binaryfunc: 3de1.__iadd__ + + +*note nb_subtract: 3de3. *note binaryfunc: 3de1.__sub__ + __rsub__ + + +*note nb_inplace_subtract: 3de4.*note binaryfunc: 3de1.__sub__ + + +*note nb_multiply: 3de5. *note binaryfunc: 3de1.__mul__ + __rmul__ + + +*note nb_inplace_multiply: 3de6.*note binaryfunc: 3de1.__mul__ + + +*note nb_remainder: 3de7. *note binaryfunc: 3de1.__mod__ + __rmod__ + + +*note nb_inplace_remainder: 3de8.*note binaryfunc: 3de1.__mod__ + + +*note nb_divmod: 3de9. *note binaryfunc: 3de1.__divmod__ + __rdivmod__ + + +*note nb_power: 3dea. *note ternaryfunc: 3dc1.__pow__ + __rpow__ + + +*note nb_inplace_power: 3deb. *note ternaryfunc: 3dc1.__pow__ + + +*note nb_negative: 3dec. *note unaryfunc: 3dde.__neg__ + + +*note nb_positive: 3ded. *note unaryfunc: 3dde.__pos__ + + +*note nb_absolute: 3dee. *note unaryfunc: 3dde.__abs__ + + +*note nb_bool: 3def. *note inquiry: 3dc7. __bool__ + + +*note nb_invert: 3df0. *note unaryfunc: 3dde.__invert__ + + +*note nb_lshift: 3df1. *note binaryfunc: 3de1.__lshift__ + __rlshift__ + + +*note nb_inplace_lshift: 3df2. *note binaryfunc: 3de1.__lshift__ + + +*note nb_rshift: 3df3. *note binaryfunc: 3de1.__rshift__ + __rrshift__ + + +*note nb_inplace_rshift: 3df4. *note binaryfunc: 3de1.__rshift__ + + +*note nb_and: 3df5. *note binaryfunc: 3de1.__and__ + __rand__ + + +*note nb_inplace_and: 3df6. *note binaryfunc: 3de1.__and__ + + +*note nb_xor: 3df7. *note binaryfunc: 3de1.__xor__ + __rxor__ + + +*note nb_inplace_xor: 3df8. *note binaryfunc: 3de1.__xor__ + + +*note nb_or: 3df9. *note binaryfunc: 3de1.__or__ __ror__ + + +*note nb_inplace_or: 3dfa. *note binaryfunc: 3de1.__or__ + + +*note nb_int: 3dfb. *note unaryfunc: 3dde.__int__ + + +*note nb_reserved: 3dfc. void * + + +*note nb_float: 3dfd. *note unaryfunc: 3dde.__float__ + + +*note nb_floor_divide: 3dfe. *note binaryfunc: 3de1.__floordiv__ + + +*note nb_inplace_floor_divide: 3dff.*note binaryfunc: 3de1.__floordiv__ + + +*note nb_true_divide: 3e00. *note binaryfunc: 3de1.__truediv__ + + +*note nb_inplace_true_divide: 3e01.*note binaryfunc: 3de1.__truediv__ + + +*note nb_index: 3e02. *note unaryfunc: 3dde.__index__ + + +*note nb_matrix_multiply: 3e03.*note binaryfunc: 3de1.__matmul__ + __rmatmul__ + + +*note nb_inplace_matrix_multiply: 3e04.*note binaryfunc: 3de1.__matmul__ + + + +*note mp_length: 3e05. *note lenfunc: 3e06. __len__ + + +*note mp_subscript: 3e07. *note binaryfunc: 3de1.__getitem__ + + +*note mp_ass_subscript: 3e08. *note objobjargproc: 3e09.__setitem__, + __delitem__ + + + +*note sq_length: 3ac9. *note lenfunc: 3e06. __len__ + + +*note sq_concat: 3e0a. *note binaryfunc: 3de1.__add__ + + +*note sq_repeat: 3e0b. *note ssizeargfunc: 3e0c.__mul__ + + +*note sq_item: 3e0d. *note ssizeargfunc: 3e0c.__getitem__ + + +*note sq_ass_item: 3e0e. *note ssizeobjargproc: 3e0f.__setitem__ + __delitem__ + + +*note sq_contains: 3e10. *note objobjproc: 3e11.__contains__ + + +*note sq_inplace_concat: 3e12. *note binaryfunc: 3de1.__iadd__ + + +*note sq_inplace_repeat: 3e13. *note ssizeargfunc: 3e0c.__imul__ + + + +*note bf_getbuffer: 3ad0. *note getbufferproc(): 3e14. + + +*note bf_releasebuffer: 39c9. *note releasebufferproc(): 3e15. + + + +File: python.info, Node: slot typedefs, Prev: sub-slots, Up: Quick Reference + +7.12.3.4 slot typedefs +...................... + +typedef Parameter Types Return Type + +----------------------------------------------------------------------------------------------- + +*note allocfunc: 3dd0. *note PyTypeObject: 2d6. * *note PyObject: 4ba. * + Py_ssize_t + + +*note destructor: 3db7. void * void + + +*note freefunc: 3dd2. void * void + + +*note traverseproc: 3dc6. void * int + *note visitproc: 3e17. + void * + + +*note newfunc: 3dd1. *note PyObject: 4ba. * *note PyObject: 4ba. * + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note initproc: 3dcf. *note PyObject: 4ba. * int + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note reprfunc: 3dbc. *note PyObject: 4ba. * *note PyObject: 4ba. * + + +*note getattrfunc: 3db8. *note PyObject: 4ba. * *note PyObject: 4ba. * + const char * + + +*note setattrfunc: 3db9. *note PyObject: 4ba. * int + const char * + *note PyObject: 4ba. * + + +*note getattrofunc: 3dc2. *note PyObject: 4ba. * *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note setattrofunc: 3dc3. *note PyObject: 4ba. * int + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note descrgetfunc: 3dcc. *note PyObject: 4ba. * *note PyObject: 4ba. * + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note descrsetfunc: 3dce. *note PyObject: 4ba. * int + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note hashfunc: 3dc0. *note PyObject: 4ba. * Py_hash_t + + +*note richcmpfunc: 3dc8. *note PyObject: 4ba. * *note PyObject: 4ba. * + *note PyObject: 4ba. * + int + + +*note getiterfunc: 3dc9. *note PyObject: 4ba. * *note PyObject: 4ba. * + + +*note iternextfunc: 3dca. *note PyObject: 4ba. * *note PyObject: 4ba. * + + +*note lenfunc: 3e06. *note PyObject: 4ba. * Py_ssize_t + + +*note getbufferproc: 3e14. *note PyObject: 4ba. * int + *note Py_buffer: b1b. * + int + + +*note releasebufferproc: 3e15. *note PyObject: 4ba. * void + *note Py_buffer: b1b. * + + +*note inquiry: 3dc7. void * int + + +*note unaryfunc: 3dde. *note PyObject: 4ba. * *note PyObject: 4ba. * + + +*note binaryfunc: 3de1. *note PyObject: 4ba. * *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note ternaryfunc: 3dc1. *note PyObject: 4ba. * *note PyObject: 4ba. * + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +*note ssizeargfunc: 3e0c. *note PyObject: 4ba. * *note PyObject: 4ba. * + Py_ssize_t + + +*note ssizeobjargproc: 3e0f. *note PyObject: 4ba. * int + Py_ssize_t + + +*note objobjproc: 3e11. *note PyObject: 4ba. * int + *note PyObject: 4ba. * + + +*note objobjargproc: 3e09. *note PyObject: 4ba. * int + *note PyObject: 4ba. * + *note PyObject: 4ba. * + + +See *note Slot Type typedefs: 3e18. below for more detail. + + +File: python.info, Node: PyTypeObject Definition, Next: PyObject Slots, Prev: Quick Reference, Up: Type Objects<3> + +7.12.3.5 PyTypeObject Definition +................................ + +The structure definition for *note PyTypeObject: 2d6. can be found in +‘Include/object.h’. For convenience of reference, this repeats the +definition found there: + + typedef struct _typeobject { + PyObject_VAR_HEAD + const char *tp_name; /* For printing, in format "<module>.<name>" */ + Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ + + /* Methods to implement standard operations */ + + destructor tp_dealloc; + Py_ssize_t tp_vectorcall_offset; + getattrfunc tp_getattr; + setattrfunc tp_setattr; + PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2) + or tp_reserved (Python 3) */ + reprfunc tp_repr; + + /* Method suites for standard classes */ + + PyNumberMethods *tp_as_number; + PySequenceMethods *tp_as_sequence; + PyMappingMethods *tp_as_mapping; + + /* More standard operations (here for binary compatibility) */ + + hashfunc tp_hash; + ternaryfunc tp_call; + reprfunc tp_str; + getattrofunc tp_getattro; + setattrofunc tp_setattro; + + /* Functions to access object as input/output buffer */ + PyBufferProcs *tp_as_buffer; + + /* Flags to define presence of optional/expanded features */ + unsigned long tp_flags; + + const char *tp_doc; /* Documentation string */ + + /* call function for all accessible objects */ + traverseproc tp_traverse; + + /* delete references to contained objects */ + inquiry tp_clear; + + /* rich comparisons */ + richcmpfunc tp_richcompare; + + /* weak reference enabler */ + Py_ssize_t tp_weaklistoffset; + + /* Iterators */ + getiterfunc tp_iter; + iternextfunc tp_iternext; + + /* Attribute descriptor and subclassing stuff */ + struct PyMethodDef *tp_methods; + struct PyMemberDef *tp_members; + struct PyGetSetDef *tp_getset; + struct _typeobject *tp_base; + PyObject *tp_dict; + descrgetfunc tp_descr_get; + descrsetfunc tp_descr_set; + Py_ssize_t tp_dictoffset; + initproc tp_init; + allocfunc tp_alloc; + newfunc tp_new; + freefunc tp_free; /* Low-level free-memory routine */ + inquiry tp_is_gc; /* For PyObject_IS_GC */ + PyObject *tp_bases; + PyObject *tp_mro; /* method resolution order */ + PyObject *tp_cache; + PyObject *tp_subclasses; + PyObject *tp_weaklist; + destructor tp_del; + + /* Type attribute cache version tag. Added in version 2.6 */ + unsigned int tp_version_tag; + + destructor tp_finalize; + + } PyTypeObject; + + +File: python.info, Node: PyObject Slots, Next: PyVarObject Slots, Prev: PyTypeObject Definition, Up: Type Objects<3> + +7.12.3.6 PyObject Slots +....................... + +The type object structure extends the *note PyVarObject: 3da6. +structure. The ‘ob_size’ field is used for dynamic types (created by +‘type_new()’, usually called from a class statement). Note that *note +PyType_Type: 3ab1. (the metatype) initializes *note tp_itemsize: 3878, +which means that its instances (i.e. type objects) `must' have the +‘ob_size’ field. + + -- C Member: PyObject* PyObject._ob_next + -- C Member: PyObject* PyObject._ob_prev + + These fields are only present when the macro ‘Py_TRACE_REFS’ is + defined. Their initialization to ‘NULL’ is taken care of by the + ‘PyObject_HEAD_INIT’ macro. For statically allocated objects, + these fields always remain ‘NULL’. For dynamically allocated + objects, these two fields are used to link the object into a + doubly-linked list of `all' live objects on the heap. This could + be used for various debugging purposes; currently the only use is + to print the objects that are still alive at the end of a run when + the environment variable *note PYTHONDUMPREFS: 159. is set. + + `Inheritance:' + + These fields are not inherited by subtypes. + + -- C Member: Py_ssize_t PyObject.ob_refcnt + + This is the type object’s reference count, initialized to ‘1’ by + the ‘PyObject_HEAD_INIT’ macro. Note that for statically allocated + type objects, the type’s instances (objects whose ‘ob_type’ points + back to the type) do `not' count as references. But for + dynamically allocated type objects, the instances `do' count as + references. + + `Inheritance:' + + This field is not inherited by subtypes. + + -- C Member: PyTypeObject* PyObject.ob_type + + This is the type’s type, in other words its metatype. It is + initialized by the argument to the ‘PyObject_HEAD_INIT’ macro, and + its value should normally be ‘&PyType_Type’. However, for + dynamically loadable extension modules that must be usable on + Windows (at least), the compiler complains that this is not a valid + initializer. Therefore, the convention is to pass ‘NULL’ to the + ‘PyObject_HEAD_INIT’ macro and to initialize this field explicitly + at the start of the module’s initialization function, before doing + anything else. This is typically done like this: + + Foo_Type.ob_type = &PyType_Type; + + This should be done before any instances of the type are created. + *note PyType_Ready(): 3882. checks if ‘ob_type’ is ‘NULL’, and if + so, initializes it to the ‘ob_type’ field of the base class. *note + PyType_Ready(): 3882. will not change this field if it is non-zero. + + `Inheritance:' + + This field is inherited by subtypes. + + +File: python.info, Node: PyVarObject Slots, Next: PyTypeObject Slots, Prev: PyObject Slots, Up: Type Objects<3> + +7.12.3.7 PyVarObject Slots +.......................... + + -- C Member: Py_ssize_t PyVarObject.ob_size + + For statically allocated type objects, this should be initialized + to zero. For dynamically allocated type objects, this field has a + special internal meaning. + + `Inheritance:' + + This field is not inherited by subtypes. + + +File: python.info, Node: PyTypeObject Slots, Next: Heap Types, Prev: PyVarObject Slots, Up: Type Objects<3> + +7.12.3.8 PyTypeObject Slots +........................... + +Each slot has a section describing inheritance. If *note +PyType_Ready(): 3882. may set a value when the field is set to ‘NULL’ +then there will also be a “Default” section. (Note that many fields set +on ‘PyBaseObject_Type’ and *note PyType_Type: 3ab1. effectively act as +defaults.) + + -- C Member: const char* PyTypeObject.tp_name + + Pointer to a NUL-terminated string containing the name of the type. + For types that are accessible as module globals, the string should + be the full module name, followed by a dot, followed by the type + name; for built-in types, it should be just the type name. If the + module is a submodule of a package, the full package name is part + of the full module name. For example, a type named ‘T’ defined in + module ‘M’ in subpackage ‘Q’ in package ‘P’ should have the *note + tp_name: 389b. initializer ‘"P.Q.M.T"’. + + For dynamically allocated type objects, this should just be the + type name, and the module name explicitly stored in the type dict + as the value for key ‘'__module__'’. + + For statically allocated type objects, the tp_name field should + contain a dot. Everything before the last dot is made accessible + as the ‘__module__’ attribute, and everything after the last dot is + made accessible as the *note __name__: c67. attribute. + + If no dot is present, the entire *note tp_name: 389b. field is made + accessible as the *note __name__: c67. attribute, and the + ‘__module__’ attribute is undefined (unless explicitly set in the + dictionary, as explained above). This means your type will be + impossible to pickle. Additionally, it will not be listed in + module documentations created with pydoc. + + This field must not be ‘NULL’. It is the only required field in + *note PyTypeObject(): 2d6. (other than potentially *note + tp_itemsize: 3878.). + + `Inheritance:' + + This field is not inherited by subtypes. + + -- C Member: Py_ssize_t PyTypeObject.tp_basicsize + -- C Member: Py_ssize_t PyTypeObject.tp_itemsize + + These fields allow calculating the size in bytes of instances of + the type. + + There are two kinds of types: types with fixed-length instances + have a zero *note tp_itemsize: 3878. field, types with + variable-length instances have a non-zero *note tp_itemsize: 3878. + field. For a type with fixed-length instances, all instances have + the same size, given in *note tp_basicsize: 3879. + + For a type with variable-length instances, the instances must have + an ‘ob_size’ field, and the instance size is *note tp_basicsize: + 3879. plus N times *note tp_itemsize: 3878, where N is the “length” + of the object. The value of N is typically stored in the + instance’s ‘ob_size’ field. There are exceptions: for example, + ints use a negative ‘ob_size’ to indicate a negative number, and N + is ‘abs(ob_size)’ there. Also, the presence of an ‘ob_size’ field + in the instance layout doesn’t mean that the instance structure is + variable-length (for example, the structure for the list type has + fixed-length instances, yet those instances have a meaningful + ‘ob_size’ field). + + The basic size includes the fields in the instance declared by the + macro *note PyObject_HEAD: c72. or *note PyObject_VAR_HEAD: 3da8. + (whichever is used to declare the instance struct) and this in turn + includes the ‘_ob_prev’ and ‘_ob_next’ fields if they are present. + This means that the only correct way to get an initializer for the + *note tp_basicsize: 3879. is to use the ‘sizeof’ operator on the + struct used to declare the instance layout. The basic size does + not include the GC header size. + + A note about alignment: if the variable items require a particular + alignment, this should be taken care of by the value of *note + tp_basicsize: 3879. Example: suppose a type implements an array of + ‘double’. *note tp_itemsize: 3878. is ‘sizeof(double)’. It is the + programmer’s responsibility that *note tp_basicsize: 3879. is a + multiple of ‘sizeof(double)’ (assuming this is the alignment + requirement for ‘double’). + + For any type with variable-length instances, this field must not be + ‘NULL’. + + `Inheritance:' + + These fields are inherited separately by subtypes. If the base + type has a non-zero *note tp_itemsize: 3878, it is generally not + safe to set *note tp_itemsize: 3878. to a different non-zero value + in a subtype (though this depends on the implementation of the base + type). + + -- C Member: destructor PyTypeObject.tp_dealloc + + A pointer to the instance destructor function. This function must + be defined unless the type guarantees that its instances will never + be deallocated (as is the case for the singletons ‘None’ and + ‘Ellipsis’). The function signature is: + + void tp_dealloc(PyObject *self); + + The destructor function is called by the *note Py_DECREF(): 266. + and *note Py_XDECREF(): 268. macros when the new reference count is + zero. At this point, the instance is still in existence, but there + are no references to it. The destructor function should free all + references which the instance owns, free all memory buffers owned + by the instance (using the freeing function corresponding to the + allocation function used to allocate the buffer), and call the + type’s *note tp_free: 3880. function. If the type is not + subtypable (doesn’t have the *note Py_TPFLAGS_BASETYPE: 3887. flag + bit set), it is permissible to call the object deallocator directly + instead of via *note tp_free: 3880. The object deallocator should + be the one used to allocate the instance; this is normally *note + PyObject_Del(): e16. if the instance was allocated using *note + PyObject_New(): 2d2. or ‘PyObject_VarNew()’, or *note + PyObject_GC_Del(): e43. if the instance was allocated using *note + PyObject_GC_New(): 2d4. or *note PyObject_GC_NewVar(): 2d5. + + Finally, if the type is heap allocated (*note Py_TPFLAGS_HEAPTYPE: + 3abe.), the deallocator should decrement the reference count for + its type object after calling the type deallocator. In order to + avoid dangling pointers, the recommended way to achieve this is: + + static void foo_dealloc(foo_object *self) { + PyTypeObject *tp = Py_TYPE(self); + // free references and buffers here + tp->tp_free(self); + Py_DECREF(tp); + } + + `Inheritance:' + + This field is inherited by subtypes. + + -- C Member: Py_ssize_t PyTypeObject.tp_vectorcall_offset + + An optional offset to a per-instance function that implements + calling the object using the `vectorcall' protocol, a more + efficient alternative of the simpler *note tp_call: 38ad. + + This field is only used if the flag *note + _Py_TPFLAGS_HAVE_VECTORCALL: 3e22. is set. If so, this must be a + positive integer containing the offset in the instance of a *note + vectorcallfunc: 3e23. pointer. The signature is the same as for + *note _PyObject_Vectorcall(): 3a10.: + + PyObject *vectorcallfunc(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) + + The `vectorcallfunc' pointer may be zero, in which case the + instance behaves as if *note _Py_TPFLAGS_HAVE_VECTORCALL: 3e22. was + not set: calling the instance falls back to *note tp_call: 38ad. + + Any class that sets ‘_Py_TPFLAGS_HAVE_VECTORCALL’ must also set + *note tp_call: 38ad. and make sure its behaviour is consistent with + the `vectorcallfunc' function. This can be done by setting + `tp_call' to ‘PyVectorcall_Call’: + + -- C Function: PyObject *PyVectorcall_Call (PyObject *callable, + PyObject *tuple, PyObject *dict) + + Call `callable'’s `vectorcallfunc' with positional and keyword + arguments given in a tuple and dict, respectively. + + This function is intended to be used in the ‘tp_call’ slot. + It does not fall back to ‘tp_call’ and it currently does not + check the ‘_Py_TPFLAGS_HAVE_VECTORCALL’ flag. To call an + object, use one of the *note PyObject_Call: 3850. functions + instead. + + Note: It is not recommended for *note heap types: 3abc. to + implement the vectorcall protocol. When a user sets + ‘__call__’ in Python code, only ‘tp_call’ is updated, possibly + making it inconsistent with the vectorcall function. + + Note: The semantics of the ‘tp_vectorcall_offset’ slot are + provisional and expected to be finalized in Python 3.9. If + you use vectorcall, plan for updating your code for Python + 3.9. + + Changed in version 3.8: This slot was used for print formatting in + Python 2.x. In Python 3.0 to 3.7, it was reserved and named + ‘tp_print’. + + `Inheritance:' + + This field is inherited by subtypes together with *note tp_call: + 38ad.: a subtype inherits *note tp_vectorcall_offset: 3a11. from + its base type when the subtype’s *note tp_call: 38ad. is ‘NULL’. + + Note that *note heap types: 3e25. (including subclasses defined in + Python) do not inherit the *note _Py_TPFLAGS_HAVE_VECTORCALL: 3e22. + flag. + + -- C Member: getattrfunc PyTypeObject.tp_getattr + + An optional pointer to the get-attribute-string function. + + This field is deprecated. When it is defined, it should point to a + function that acts the same as the *note tp_getattro: 389f. + function, but taking a C string instead of a Python string object + to give the attribute name. + + `Inheritance:' + + Group: ‘tp_getattr’, ‘tp_getattro’ + + This field is inherited by subtypes together with *note + tp_getattro: 389f.: a subtype inherits both *note tp_getattr: 38a2. + and *note tp_getattro: 389f. from its base type when the subtype’s + *note tp_getattr: 38a2. and *note tp_getattro: 389f. are both + ‘NULL’. + + -- C Member: setattrfunc PyTypeObject.tp_setattr + + An optional pointer to the function for setting and deleting + attributes. + + This field is deprecated. When it is defined, it should point to a + function that acts the same as the *note tp_setattro: 38a0. + function, but taking a C string instead of a Python string object + to give the attribute name. + + `Inheritance:' + + Group: ‘tp_setattr’, ‘tp_setattro’ + + This field is inherited by subtypes together with *note + tp_setattro: 38a0.: a subtype inherits both *note tp_setattr: 38a3. + and *note tp_setattro: 38a0. from its base type when the subtype’s + *note tp_setattr: 38a3. and *note tp_setattro: 38a0. are both + ‘NULL’. + + -- C Member: PyAsyncMethods* PyTypeObject.tp_as_async + + Pointer to an additional structure that contains fields relevant + only to objects which implement *note awaitable: 519. and *note + asynchronous iterator: 645. protocols at the C-level. See *note + Async Object Structures: 3e26. for details. + + New in version 3.5: Formerly known as ‘tp_compare’ and + ‘tp_reserved’. + + `Inheritance:' + + The *note tp_as_async: 3dba. field is not inherited, but the + contained fields are inherited individually. + + -- C Member: reprfunc PyTypeObject.tp_repr + + An optional pointer to a function that implements the built-in + function *note repr(): 7cc. + + The signature is the same as for *note PyObject_Repr(): 968.: + + PyObject *tp_repr(PyObject *self); + + The function must return a string or a Unicode object. Ideally, + this function should return a string that, when passed to *note + eval(): b90, given a suitable environment, returns an object with + the same value. If this is not feasible, it should return a string + starting with ‘'<'’ and ending with ‘'>'’ from which both the type + and the value of the object can be deduced. + + `Inheritance:' + + This field is inherited by subtypes. + + `Default:' + + When this field is not set, a string of the form ‘<%s object at + %p>’ is returned, where ‘%s’ is replaced by the type name, and ‘%p’ + by the object’s memory address. + + -- C Member: PyNumberMethods* PyTypeObject.tp_as_number + + Pointer to an additional structure that contains fields relevant + only to objects which implement the number protocol. These fields + are documented in *note Number Object Structures: 3e27. + + `Inheritance:' + + The *note tp_as_number: 3dbd. field is not inherited, but the + contained fields are inherited individually. + + -- C Member: PySequenceMethods* PyTypeObject.tp_as_sequence + + Pointer to an additional structure that contains fields relevant + only to objects which implement the sequence protocol. These + fields are documented in *note Sequence Object Structures: 3e28. + + `Inheritance:' + + The *note tp_as_sequence: 3dbe. field is not inherited, but the + contained fields are inherited individually. + + -- C Member: PyMappingMethods* PyTypeObject.tp_as_mapping + + Pointer to an additional structure that contains fields relevant + only to objects which implement the mapping protocol. These fields + are documented in *note Mapping Object Structures: 3e29. + + `Inheritance:' + + The *note tp_as_mapping: 3dbf. field is not inherited, but the + contained fields are inherited individually. + + -- C Member: hashfunc PyTypeObject.tp_hash + + An optional pointer to a function that implements the built-in + function *note hash(): 9c4. + + The signature is the same as for *note PyObject_Hash(): 3a15.: + + Py_hash_t tp_hash(PyObject *); + + The value ‘-1’ should not be returned as a normal return value; + when an error occurs during the computation of the hash value, the + function should set an exception and return ‘-1’. + + When this field is not set (`and' ‘tp_richcompare’ is not set), an + attempt to take the hash of the object raises *note TypeError: 192. + This is the same as setting it to *note + PyObject_HashNotImplemented(): d31. + + This field can be set explicitly to *note + PyObject_HashNotImplemented(): d31. to block inheritance of the + hash method from a parent type. This is interpreted as the + equivalent of ‘__hash__ = None’ at the Python level, causing + ‘isinstance(o, collections.Hashable)’ to correctly return ‘False’. + Note that the converse is also true - setting ‘__hash__ = None’ on + a class at the Python level will result in the ‘tp_hash’ slot being + set to *note PyObject_HashNotImplemented(): d31. + + `Inheritance:' + + Group: ‘tp_hash’, ‘tp_richcompare’ + + This field is inherited by subtypes together with *note + tp_richcompare: 38a5.: a subtype inherits both of *note + tp_richcompare: 38a5. and *note tp_hash: 38ac, when the subtype’s + *note tp_richcompare: 38a5. and *note tp_hash: 38ac. are both + ‘NULL’. + + -- C Member: ternaryfunc PyTypeObject.tp_call + + An optional pointer to a function that implements calling the + object. This should be ‘NULL’ if the object is not callable. The + signature is the same as for *note PyObject_Call(): 3850.: + + PyObject *tp_call(PyObject *self, PyObject *args, PyObject *kwargs); + + `Inheritance:' + + This field is inherited by subtypes. + + -- C Member: reprfunc PyTypeObject.tp_str + + An optional pointer to a function that implements the built-in + operation *note str(): 330. (Note that *note str: 330. is a type + now, and *note str(): 330. calls the constructor for that type. + This constructor calls *note PyObject_Str(): 969. to do the actual + work, and *note PyObject_Str(): 969. will call this handler.) + + The signature is the same as for *note PyObject_Str(): 969.: + + PyObject *tp_str(PyObject *self); + + The function must return a string or a Unicode object. It should + be a “friendly” string representation of the object, as this is the + representation that will be used, among other things, by the *note + print(): 886. function. + + `Inheritance:' + + This field is inherited by subtypes. + + `Default:' + + When this field is not set, *note PyObject_Repr(): 968. is called + to return a string representation. + + -- C Member: getattrofunc PyTypeObject.tp_getattro + + An optional pointer to the get-attribute function. + + The signature is the same as for *note PyObject_GetAttr(): 3a00.: + + PyObject *tp_getattro(PyObject *self, PyObject *attr); + + It is usually convenient to set this field to *note + PyObject_GenericGetAttr(): 3a02, which implements the normal way of + looking for object attributes. + + `Inheritance:' + + Group: ‘tp_getattr’, ‘tp_getattro’ + + This field is inherited by subtypes together with *note tp_getattr: + 38a2.: a subtype inherits both *note tp_getattr: 38a2. and *note + tp_getattro: 389f. from its base type when the subtype’s *note + tp_getattr: 38a2. and *note tp_getattro: 389f. are both ‘NULL’. + + `Default:' + + ‘PyBaseObject_Type’ uses *note PyObject_GenericGetAttr(): 3a02. + + -- C Member: setattrofunc PyTypeObject.tp_setattro + + An optional pointer to the function for setting and deleting + attributes. + + The signature is the same as for *note PyObject_SetAttr(): 3a03.: + + PyObject *tp_setattro(PyObject *self, PyObject *attr, PyObject *value); + + In addition, setting `value' to ‘NULL’ to delete an attribute must + be supported. It is usually convenient to set this field to *note + PyObject_GenericSetAttr(): 3a07, which implements the normal way of + setting object attributes. + + `Inheritance:' + + Group: ‘tp_setattr’, ‘tp_setattro’ + + This field is inherited by subtypes together with *note tp_setattr: + 38a3.: a subtype inherits both *note tp_setattr: 38a3. and *note + tp_setattro: 38a0. from its base type when the subtype’s *note + tp_setattr: 38a3. and *note tp_setattro: 38a0. are both ‘NULL’. + + `Default:' + + ‘PyBaseObject_Type’ uses *note PyObject_GenericSetAttr(): 3a07. + + -- C Member: PyBufferProcs* PyTypeObject.tp_as_buffer + + Pointer to an additional structure that contains fields relevant + only to objects which implement the buffer interface. These fields + are documented in *note Buffer Object Structures: 3a6e. + + `Inheritance:' + + The *note tp_as_buffer: 3dc4. field is not inherited, but the + contained fields are inherited individually. + + -- C Member: unsigned long PyTypeObject.tp_flags + + This field is a bit mask of various flags. Some flags indicate + variant semantics for certain situations; others are used to + indicate that certain fields in the type object (or in the + extension structures referenced via *note tp_as_number: 3dbd, *note + tp_as_sequence: 3dbe, *note tp_as_mapping: 3dbf, and *note + tp_as_buffer: 3dc4.) that were historically not always present are + valid; if such a flag bit is clear, the type fields it guards must + not be accessed and must be considered to have a zero or ‘NULL’ + value instead. + + `Inheritance:' + + Inheritance of this field is complicated. Most flag bits are + inherited individually, i.e. if the base type has a flag bit set, + the subtype inherits this flag bit. The flag bits that pertain to + extension structures are strictly inherited if the extension + structure is inherited, i.e. the base type’s value of the flag bit + is copied into the subtype together with a pointer to the extension + structure. The *note Py_TPFLAGS_HAVE_GC: 388e. flag bit is + inherited together with the *note tp_traverse: 33e8. and *note + tp_clear: 3898. fields, i.e. if the *note Py_TPFLAGS_HAVE_GC: + 388e. flag bit is clear in the subtype and the *note tp_traverse: + 33e8. and *note tp_clear: 3898. fields in the subtype exist and + have ‘NULL’ values. + + `Default:' + + ‘PyBaseObject_Type’ uses ‘Py_TPFLAGS_DEFAULT | + Py_TPFLAGS_BASETYPE’. + + `Bit Masks:' + + The following bit masks are currently defined; these can be ORed + together using the ‘|’ operator to form the value of the *note + tp_flags: 3ab6. field. The macro *note PyType_HasFeature(): 3ab8. + takes a type and a flags value, `tp' and `f', and checks whether + ‘tp->tp_flags & f’ is non-zero. + + -- Data: Py_TPFLAGS_HEAPTYPE + + This bit is set when the type object itself is allocated on + the heap, for example, types created dynamically using *note + PyType_FromSpec(): 2d1. In this case, the ‘ob_type’ field of + its instances is considered a reference to the type, and the + type object is INCREF’ed when a new instance is created, and + DECREF’ed when an instance is destroyed (this does not apply + to instances of subtypes; only the type referenced by the + instance’s ob_type gets INCREF’ed or DECREF’ed). + + `Inheritance:' + + ??? + + -- Data: Py_TPFLAGS_BASETYPE + + This bit is set when the type can be used as the base type of + another type. If this bit is clear, the type cannot be + subtyped (similar to a “final” class in Java). + + `Inheritance:' + + ??? + + -- Data: Py_TPFLAGS_READY + + This bit is set when the type object has been fully + initialized by *note PyType_Ready(): 3882. + + `Inheritance:' + + ??? + + -- Data: Py_TPFLAGS_READYING + + This bit is set while *note PyType_Ready(): 3882. is in the + process of initializing the type object. + + `Inheritance:' + + ??? + + -- Data: Py_TPFLAGS_HAVE_GC + + This bit is set when the object supports garbage collection. + If this bit is set, instances must be created using *note + PyObject_GC_New(): 2d4. and destroyed using *note + PyObject_GC_Del(): e43. More information in section *note + Supporting Cyclic Garbage Collection: 3e2c. This bit also + implies that the GC-related fields *note tp_traverse: 33e8. + and *note tp_clear: 3898. are present in the type object. + + `Inheritance:' + + Group: *note Py_TPFLAGS_HAVE_GC: 388e, ‘tp_traverse’, + ‘tp_clear’ + + The *note Py_TPFLAGS_HAVE_GC: 388e. flag bit is inherited + together with the ‘tp_traverse’ and ‘tp_clear’ fields, i.e. + if the *note Py_TPFLAGS_HAVE_GC: 388e. flag bit is clear in + the subtype and the ‘tp_traverse’ and ‘tp_clear’ fields in the + subtype exist and have ‘NULL’ values. + + -- Data: Py_TPFLAGS_DEFAULT + + This is a bitmask of all the bits that pertain to the + existence of certain fields in the type object and its + extension structures. Currently, it includes the following + bits: ‘Py_TPFLAGS_HAVE_STACKLESS_EXTENSION’, + ‘Py_TPFLAGS_HAVE_VERSION_TAG’. + + `Inheritance:' + + ??? + + -- Data: Py_TPFLAGS_METHOD_DESCRIPTOR + + This bit indicates that objects behave like unbound methods. + + If this flag is set for ‘type(meth)’, then: + + - ‘meth.__get__(obj, cls)(*args, **kwds)’ (with ‘obj’ not + None) must be equivalent to ‘meth(obj, *args, **kwds)’. + + - ‘meth.__get__(None, cls)(*args, **kwds)’ must be + equivalent to ‘meth(*args, **kwds)’. + + This flag enables an optimization for typical method calls + like ‘obj.meth()’: it avoids creating a temporary “bound + method” object for ‘obj.meth’. + + New in version 3.8. + + `Inheritance:' + + This flag is never inherited by heap types. For extension + types, it is inherited whenever *note tp_descr_get: 3dcb. is + inherited. + + -- Data: Py_TPFLAGS_LONG_SUBCLASS + + -- Data: Py_TPFLAGS_LIST_SUBCLASS + + -- Data: Py_TPFLAGS_TUPLE_SUBCLASS + + -- Data: Py_TPFLAGS_BYTES_SUBCLASS + + -- Data: Py_TPFLAGS_UNICODE_SUBCLASS + + -- Data: Py_TPFLAGS_DICT_SUBCLASS + + -- Data: Py_TPFLAGS_BASE_EXC_SUBCLASS + + -- Data: Py_TPFLAGS_TYPE_SUBCLASS + + These flags are used by functions such as *note + PyLong_Check(): 3adc. to quickly determine if a type is a + subclass of a built-in type; such specific checks are faster + than a generic check, like *note PyObject_IsInstance(): 79d. + Custom types that inherit from built-ins should have their + *note tp_flags: 3ab6. set appropriately, or the code that + interacts with such types will behave differently depending on + what kind of check is used. + + -- Data: Py_TPFLAGS_HAVE_FINALIZE + + This bit is set when the *note tp_finalize: 2d7. slot is + present in the type structure. + + New in version 3.4. + + Deprecated since version 3.8: This flag isn’t necessary + anymore, as the interpreter assumes the *note tp_finalize: + 2d7. slot is always present in the type structure. + + -- Data: _Py_TPFLAGS_HAVE_VECTORCALL + + This bit is set when the class implements the vectorcall + protocol. See *note tp_vectorcall_offset: 3a11. for details. + + `Inheritance:' + + This bit is set on `static' subtypes if ‘tp_flags’ is not + overridden: a subtype inherits ‘_Py_TPFLAGS_HAVE_VECTORCALL’ + from its base type when the subtype’s *note tp_call: 38ad. is + ‘NULL’ and the subtype’s ‘Py_TPFLAGS_HEAPTYPE’ is not set. + + *note Heap types: 3e25. do not inherit + ‘_Py_TPFLAGS_HAVE_VECTORCALL’. + + Note: This flag is provisional and expected to become + public in Python 3.9, with a different name and, + possibly, changed semantics. If you use vectorcall, plan + for updating your code for Python 3.9. + + New in version 3.8. + + -- C Member: const char* PyTypeObject.tp_doc + + An optional pointer to a NUL-terminated C string giving the + docstring for this type object. This is exposed as the ‘__doc__’ + attribute on the type and instances of the type. + + `Inheritance:' + + This field is `not' inherited by subtypes. + + -- C Member: traverseproc PyTypeObject.tp_traverse + + An optional pointer to a traversal function for the garbage + collector. This is only used if the *note Py_TPFLAGS_HAVE_GC: + 388e. flag bit is set. The signature is: + + int tp_traverse(PyObject *self, visitproc visit, void *arg); + + More information about Python’s garbage collection scheme can be + found in section *note Supporting Cyclic Garbage Collection: 3e2c. + + The *note tp_traverse: 33e8. pointer is used by the garbage + collector to detect reference cycles. A typical implementation of + a *note tp_traverse: 33e8. function simply calls *note Py_VISIT(): + 388c. on each of the instance’s members that are Python objects + that the instance owns. For example, this is function + ‘local_traverse()’ from the *note _thread: 3. extension module: + + static int + local_traverse(localobject *self, visitproc visit, void *arg) + { + Py_VISIT(self->args); + Py_VISIT(self->kw); + Py_VISIT(self->dict); + return 0; + } + + Note that *note Py_VISIT(): 388c. is called only on those members + that can participate in reference cycles. Although there is also a + ‘self->key’ member, it can only be ‘NULL’ or a Python string and + therefore cannot be part of a reference cycle. + + On the other hand, even if you know a member can never be part of a + cycle, as a debugging aid you may want to visit it anyway just so + the *note gc: 86. module’s *note get_referents(): 31f4. function + will include it. + + Warning: When implementing *note tp_traverse: 33e8, only the + members that the instance `owns' (by having strong references + to them) must be visited. For instance, if an object supports + weak references via the *note tp_weaklist: 3ace. slot, the + pointer supporting the linked list (what `tp_weaklist' points + to) must `not' be visited as the instance does not directly + own the weak references to itself (the weakreference list is + there to support the weak reference machinery, but the + instance has no strong reference to the elements inside it, as + they are allowed to be removed even if the instance is still + alive). + + Note that *note Py_VISIT(): 388c. requires the `visit' and `arg' + parameters to ‘local_traverse()’ to have these specific names; + don’t name them just anything. + + `Inheritance:' + + Group: *note Py_TPFLAGS_HAVE_GC: 388e, ‘tp_traverse’, ‘tp_clear’ + + This field is inherited by subtypes together with *note tp_clear: + 3898. and the *note Py_TPFLAGS_HAVE_GC: 388e. flag bit: the flag + bit, *note tp_traverse: 33e8, and *note tp_clear: 3898. are all + inherited from the base type if they are all zero in the subtype. + + -- C Member: inquiry PyTypeObject.tp_clear + + An optional pointer to a clear function for the garbage collector. + This is only used if the *note Py_TPFLAGS_HAVE_GC: 388e. flag bit + is set. The signature is: + + int tp_clear(PyObject *); + + The *note tp_clear: 3898. member function is used to break + reference cycles in cyclic garbage detected by the garbage + collector. Taken together, all *note tp_clear: 3898. functions in + the system must combine to break all reference cycles. This is + subtle, and if in any doubt supply a *note tp_clear: 3898. + function. For example, the tuple type does not implement a *note + tp_clear: 3898. function, because it’s possible to prove that no + reference cycle can be composed entirely of tuples. Therefore the + *note tp_clear: 3898. functions of other types must be sufficient + to break any cycle containing a tuple. This isn’t immediately + obvious, and there’s rarely a good reason to avoid implementing + *note tp_clear: 3898. + + Implementations of *note tp_clear: 3898. should drop the instance’s + references to those of its members that may be Python objects, and + set its pointers to those members to ‘NULL’, as in the following + example: + + static int + local_clear(localobject *self) + { + Py_CLEAR(self->key); + Py_CLEAR(self->args); + Py_CLEAR(self->kw); + Py_CLEAR(self->dict); + return 0; + } + + The *note Py_CLEAR(): 388d. macro should be used, because clearing + references is delicate: the reference to the contained object must + not be decremented until after the pointer to the contained object + is set to ‘NULL’. This is because decrementing the reference count + may cause the contained object to become trash, triggering a chain + of reclamation activity that may include invoking arbitrary Python + code (due to finalizers, or weakref callbacks, associated with the + contained object). If it’s possible for such code to reference + `self' again, it’s important that the pointer to the contained + object be ‘NULL’ at that time, so that `self' knows the contained + object can no longer be used. The *note Py_CLEAR(): 388d. macro + performs the operations in a safe order. + + Because the goal of *note tp_clear: 3898. functions is to break + reference cycles, it’s not necessary to clear contained objects + like Python strings or Python integers, which can’t participate in + reference cycles. On the other hand, it may be convenient to clear + all contained Python objects, and write the type’s *note + tp_dealloc: 387f. function to invoke *note tp_clear: 3898. + + More information about Python’s garbage collection scheme can be + found in section *note Supporting Cyclic Garbage Collection: 3e2c. + + `Inheritance:' + + Group: *note Py_TPFLAGS_HAVE_GC: 388e, ‘tp_traverse’, ‘tp_clear’ + + This field is inherited by subtypes together with *note + tp_traverse: 33e8. and the *note Py_TPFLAGS_HAVE_GC: 388e. flag + bit: the flag bit, *note tp_traverse: 33e8, and *note tp_clear: + 3898. are all inherited from the base type if they are all zero in + the subtype. + + -- C Member: richcmpfunc PyTypeObject.tp_richcompare + + An optional pointer to the rich comparison function, whose + signature is: + + PyObject *tp_richcompare(PyObject *self, PyObject *other, int op); + + The first parameter is guaranteed to be an instance of the type + that is defined by *note PyTypeObject: 2d6. + + The function should return the result of the comparison (usually + ‘Py_True’ or ‘Py_False’). If the comparison is undefined, it must + return ‘Py_NotImplemented’, if another error occurred it must + return ‘NULL’ and set an exception condition. + + The following constants are defined to be used as the third + argument for *note tp_richcompare: 38a5. and for *note + PyObject_RichCompare(): 38a6.: + + Constant Comparison + + -------------------------------------- + + ‘Py_LT’ ‘<’ + + + ‘Py_LE’ ‘<=’ + + + ‘Py_EQ’ ‘==’ + + + ‘Py_NE’ ‘!=’ + + + ‘Py_GT’ ‘>’ + + + ‘Py_GE’ ‘>=’ + + + The following macro is defined to ease writing rich comparison + functions: + + -- C Macro: Py_RETURN_RICHCOMPARE (VAL_A, VAL_B, op) + + Return ‘Py_True’ or ‘Py_False’ from the function, depending on + the result of a comparison. VAL_A and VAL_B must be orderable + by C comparison operators (for example, they may be C ints or + floats). The third argument specifies the requested + operation, as for *note PyObject_RichCompare(): 38a6. + + The return value’s reference count is properly incremented. + + On error, sets an exception and returns ‘NULL’ from the + function. + + New in version 3.7. + + `Inheritance:' + + Group: ‘tp_hash’, ‘tp_richcompare’ + + This field is inherited by subtypes together with *note tp_hash: + 38ac.: a subtype inherits *note tp_richcompare: 38a5. and *note + tp_hash: 38ac. when the subtype’s *note tp_richcompare: 38a5. and + *note tp_hash: 38ac. are both ‘NULL’. + + `Default:' + + ‘PyBaseObject_Type’ provides a ‘tp_richcompare’ implementation, + which may be inherited. However, if only ‘tp_hash’ is defined, not + even the inherited function is used and instances of the type will + not be able to participate in any comparisons. + + -- C Member: Py_ssize_t PyTypeObject.tp_weaklistoffset + + If the instances of this type are weakly referenceable, this field + is greater than zero and contains the offset in the instance + structure of the weak reference list head (ignoring the GC header, + if present); this offset is used by ‘PyObject_ClearWeakRefs()’ and + the ‘PyWeakref_*()’ functions. The instance structure needs to + include a field of type *note PyObject*: 4ba. which is initialized + to ‘NULL’. + + Do not confuse this field with *note tp_weaklist: 3ace.; that is + the list head for weak references to the type object itself. + + `Inheritance:' + + This field is inherited by subtypes, but see the rules listed + below. A subtype may override this offset; this means that the + subtype uses a different weak reference list head than the base + type. Since the list head is always found via *note + tp_weaklistoffset: 38af, this should not be a problem. + + When a type defined by a class statement has no *note __slots__: + e2d. declaration, and none of its base types are weakly + referenceable, the type is made weakly referenceable by adding a + weak reference list head slot to the instance layout and setting + the *note tp_weaklistoffset: 38af. of that slot’s offset. + + When a type’s *note __slots__: e2d. declaration contains a slot + named ‘__weakref__’, that slot becomes the weak reference list head + for instances of the type, and the slot’s offset is stored in the + type’s *note tp_weaklistoffset: 38af. + + When a type’s *note __slots__: e2d. declaration does not contain a + slot named ‘__weakref__’, the type inherits its *note + tp_weaklistoffset: 38af. from its base type. + + -- C Member: getiterfunc PyTypeObject.tp_iter + + An optional pointer to a function that returns an iterator for the + object. Its presence normally signals that the instances of this + type are iterable (although sequences may be iterable without this + function). + + This function has the same signature as *note PyObject_GetIter(): + 3a1d.: + + PyObject *tp_iter(PyObject *self); + + `Inheritance:' + + This field is inherited by subtypes. + + -- C Member: iternextfunc PyTypeObject.tp_iternext + + An optional pointer to a function that returns the next item in an + iterator. The signature is: + + PyObject *tp_iternext(PyObject *self); + + When the iterator is exhausted, it must return ‘NULL’; a *note + StopIteration: 486. exception may or may not be set. When another + error occurs, it must return ‘NULL’ too. Its presence signals that + the instances of this type are iterators. + + Iterator types should also define the *note tp_iter: e30. function, + and that function should return the iterator instance itself (not a + new iterator instance). + + This function has the same signature as *note PyIter_Next(): 3a6b. + + `Inheritance:' + + This field is inherited by subtypes. + + -- C Member: struct PyMethodDef* PyTypeObject.tp_methods + + An optional pointer to a static ‘NULL’-terminated array of *note + PyMethodDef: e1b. structures, declaring regular methods of this + type. + + For each entry in the array, an entry is added to the type’s + dictionary (see *note tp_dict: 3aca. below) containing a method + descriptor. + + `Inheritance:' + + This field is not inherited by subtypes (methods are inherited + through a different mechanism). + + -- C Member: struct PyMemberDef* PyTypeObject.tp_members + + An optional pointer to a static ‘NULL’-terminated array of *note + PyMemberDef: 426. structures, declaring regular data members + (fields or slots) of instances of this type. + + For each entry in the array, an entry is added to the type’s + dictionary (see *note tp_dict: 3aca. below) containing a member + descriptor. + + `Inheritance:' + + This field is not inherited by subtypes (members are inherited + through a different mechanism). + + -- C Member: struct PyGetSetDef* PyTypeObject.tp_getset + + An optional pointer to a static ‘NULL’-terminated array of *note + PyGetSetDef: 427. structures, declaring computed attributes of + instances of this type. + + For each entry in the array, an entry is added to the type’s + dictionary (see *note tp_dict: 3aca. below) containing a getset + descriptor. + + `Inheritance:' + + This field is not inherited by subtypes (computed attributes are + inherited through a different mechanism). + + -- C Member: PyTypeObject* PyTypeObject.tp_base + + An optional pointer to a base type from which type properties are + inherited. At this level, only single inheritance is supported; + multiple inheritance require dynamically creating a type object by + calling the metatype. + + Note: + Slot initialization is subject to the rules of initializing + globals. C99 requires the initializers to be “address + constants”. Function designators like *note + PyType_GenericNew(): 387d, with implicit conversion to a + pointer, are valid C99 address constants. + + However, the unary ‘&’ operator applied to a non-static + variable like ‘PyBaseObject_Type()’ is not required to produce + an address constant. Compilers may support this (gcc does), + MSVC does not. Both compilers are strictly standard + conforming in this particular behavior. + + Consequently, *note tp_base: 3890. should be set in the + extension module’s init function. + + `Inheritance:' + + This field is not inherited by subtypes (obviously). + + `Default:' + + This field defaults to ‘&PyBaseObject_Type’ (which to Python + programmers is known as the type *note object: 2b0.). + + -- C Member: PyObject* PyTypeObject.tp_dict + + The type’s dictionary is stored here by *note PyType_Ready(): 3882. + + This field should normally be initialized to ‘NULL’ before + PyType_Ready is called; it may also be initialized to a dictionary + containing initial attributes for the type. Once *note + PyType_Ready(): 3882. has initialized the type, extra attributes + for the type may be added to this dictionary only if they don’t + correspond to overloaded operations (like *note __add__(): 10f9.). + + `Inheritance:' + + This field is not inherited by subtypes (though the attributes + defined in here are inherited through a different mechanism). + + `Default:' + + If this field is ‘NULL’, *note PyType_Ready(): 3882. will assign a + new dictionary to it. + + Warning: It is not safe to use *note PyDict_SetItem(): 3863. + on or otherwise modify *note tp_dict: 3aca. with the + dictionary C-API. + + -- C Member: descrgetfunc PyTypeObject.tp_descr_get + + An optional pointer to a “descriptor get” function. + + The function signature is: + + PyObject * tp_descr_get(PyObject *self, PyObject *obj, PyObject *type); + + `Inheritance:' + + This field is inherited by subtypes. + + -- C Member: descrsetfunc PyTypeObject.tp_descr_set + + An optional pointer to a function for setting and deleting a + descriptor’s value. + + The function signature is: + + int tp_descr_set(PyObject *self, PyObject *obj, PyObject *value); + + The `value' argument is set to ‘NULL’ to delete the value. + + `Inheritance:' + + This field is inherited by subtypes. + + -- C Member: Py_ssize_t PyTypeObject.tp_dictoffset + + If the instances of this type have a dictionary containing instance + variables, this field is non-zero and contains the offset in the + instances of the type of the instance variable dictionary; this + offset is used by *note PyObject_GenericGetAttr(): 3a02. + + Do not confuse this field with *note tp_dict: 3aca.; that is the + dictionary for attributes of the type object itself. + + If the value of this field is greater than zero, it specifies the + offset from the start of the instance structure. If the value is + less than zero, it specifies the offset from the `end' of the + instance structure. A negative offset is more expensive to use, + and should only be used when the instance structure contains a + variable-length part. This is used for example to add an instance + variable dictionary to subtypes of *note str: 330. or *note tuple: + 47e. Note that the *note tp_basicsize: 3879. field should account + for the dictionary added to the end in that case, even though the + dictionary is not included in the basic object layout. On a system + with a pointer size of 4 bytes, *note tp_dictoffset: 3acf. should + be set to ‘-4’ to indicate that the dictionary is at the very end + of the structure. + + The real dictionary offset in an instance can be computed from a + negative *note tp_dictoffset: 3acf. as follows: + + dictoffset = tp_basicsize + abs(ob_size)*tp_itemsize + tp_dictoffset + if dictoffset is not aligned on sizeof(void*): + round up to sizeof(void*) + + where *note tp_basicsize: 3879, *note tp_itemsize: 3878. and *note + tp_dictoffset: 3acf. are taken from the type object, and ‘ob_size’ + is taken from the instance. The absolute value is taken because + ints use the sign of ‘ob_size’ to store the sign of the number. + (There’s never a need to do this calculation yourself; it is done + for you by ‘_PyObject_GetDictPtr()’.) + + `Inheritance:' + + This field is inherited by subtypes, but see the rules listed + below. A subtype may override this offset; this means that the + subtype instances store the dictionary at a difference offset than + the base type. Since the dictionary is always found via *note + tp_dictoffset: 3acf, this should not be a problem. + + When a type defined by a class statement has no *note __slots__: + e2d. declaration, and none of its base types has an instance + variable dictionary, a dictionary slot is added to the instance + layout and the *note tp_dictoffset: 3acf. is set to that slot’s + offset. + + When a type defined by a class statement has a *note __slots__: + e2d. declaration, the type inherits its *note tp_dictoffset: 3acf. + from its base type. + + (Adding a slot named *note __dict__: 4fc. to the *note __slots__: + e2d. declaration does not have the expected effect, it just causes + confusion. Maybe this should be added as a feature just like + ‘__weakref__’ though.) + + `Default:' + + This slot has no default. For static types, if the field is ‘NULL’ + then no *note __dict__: 4fc. gets created for instances. + + -- C Member: initproc PyTypeObject.tp_init + + An optional pointer to an instance initialization function. + + This function corresponds to the *note __init__(): d5e. method of + classes. Like *note __init__(): d5e, it is possible to create an + instance without calling *note __init__(): d5e, and it is possible + to reinitialize an instance by calling its *note __init__(): d5e. + method again. + + The function signature is: + + int tp_init(PyObject *self, PyObject *args, PyObject *kwds); + + The self argument is the instance to be initialized; the `args' and + `kwds' arguments represent positional and keyword arguments of the + call to *note __init__(): d5e. + + The *note tp_init: 3883. function, if not ‘NULL’, is called when an + instance is created normally by calling its type, after the type’s + *note tp_new: 387c. function has returned an instance of the type. + If the *note tp_new: 387c. function returns an instance of some + other type that is not a subtype of the original type, no *note + tp_init: 3883. function is called; if *note tp_new: 387c. returns + an instance of a subtype of the original type, the subtype’s *note + tp_init: 3883. is called. + + Returns ‘0’ on success, ‘-1’ and sets an exception on error. + + `Inheritance:' + + This field is inherited by subtypes. + + `Default:' + + For static types this field does not have a default. + + -- C Member: allocfunc PyTypeObject.tp_alloc + + An optional pointer to an instance allocation function. + + The function signature is: + + PyObject *tp_alloc(PyTypeObject *self, Py_ssize_t nitems); + + `Inheritance:' + + This field is inherited by static subtypes, but not by dynamic + subtypes (subtypes created by a class statement). + + `Default:' + + For dynamic subtypes, this field is always set to *note + PyType_GenericAlloc(): 270, to force a standard heap allocation + strategy. + + For static subtypes, ‘PyBaseObject_Type’ uses *note + PyType_GenericAlloc(): 270. That is the recommended value for all + statically defined types. + + -- C Member: newfunc PyTypeObject.tp_new + + An optional pointer to an instance creation function. + + The function signature is: + + PyObject *tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds); + + The subtype argument is the type of the object being created; the + `args' and `kwds' arguments represent positional and keyword + arguments of the call to the type. Note that subtype doesn’t have + to equal the type whose *note tp_new: 387c. function is called; it + may be a subtype of that type (but not an unrelated type). + + The *note tp_new: 387c. function should call + ‘subtype->tp_alloc(subtype, nitems)’ to allocate space for the + object, and then do only as much further initialization as is + absolutely necessary. Initialization that can safely be ignored or + repeated should be placed in the *note tp_init: 3883. handler. A + good rule of thumb is that for immutable types, all initialization + should take place in *note tp_new: 387c, while for mutable types, + most initialization should be deferred to *note tp_init: 3883. + + `Inheritance:' + + This field is inherited by subtypes, except it is not inherited by + static types whose *note tp_base: 3890. is ‘NULL’ or + ‘&PyBaseObject_Type’. + + `Default:' + + For static types this field has no default. This means if the slot + is defined as ‘NULL’, the type cannot be called to create new + instances; presumably there is some other way to create instances, + like a factory function. + + -- C Member: freefunc PyTypeObject.tp_free + + An optional pointer to an instance deallocation function. Its + signature is: + + void tp_free(void *self); + + An initializer that is compatible with this signature is *note + PyObject_Free(): 504. + + `Inheritance:' + + This field is inherited by static subtypes, but not by dynamic + subtypes (subtypes created by a class statement) + + `Default:' + + In dynamic subtypes, this field is set to a deallocator suitable to + match *note PyType_GenericAlloc(): 270. and the value of the *note + Py_TPFLAGS_HAVE_GC: 388e. flag bit. + + For static subtypes, ‘PyBaseObject_Type’ uses PyObject_Del. + + -- C Member: inquiry PyTypeObject.tp_is_gc + + An optional pointer to a function called by the garbage collector. + + The garbage collector needs to know whether a particular object is + collectible or not. Normally, it is sufficient to look at the + object’s type’s *note tp_flags: 3ab6. field, and check the *note + Py_TPFLAGS_HAVE_GC: 388e. flag bit. But some types have a mixture + of statically and dynamically allocated instances, and the + statically allocated instances are not collectible. Such types + should define this function; it should return ‘1’ for a collectible + instance, and ‘0’ for a non-collectible instance. The signature + is: + + int tp_is_gc(PyObject *self); + + (The only example of this are types themselves. The metatype, + *note PyType_Type: 3ab1, defines this function to distinguish + between statically and dynamically allocated types.) + + `Inheritance:' + + This field is inherited by subtypes. + + `Default:' + + This slot has no default. If this field is ‘NULL’, *note + Py_TPFLAGS_HAVE_GC: 388e. is used as the functional equivalent. + + -- C Member: PyObject* PyTypeObject.tp_bases + + Tuple of base types. + + This is set for types created by a class statement. It should be + ‘NULL’ for statically defined types. + + `Inheritance:' + + This field is not inherited. + + -- C Member: PyObject* PyTypeObject.tp_mro + + Tuple containing the expanded set of base types, starting with the + type itself and ending with *note object: 2b0, in Method Resolution + Order. + + `Inheritance:' + + This field is not inherited; it is calculated fresh by *note + PyType_Ready(): 3882. + + -- C Member: PyObject* PyTypeObject.tp_cache + + Unused. Internal use only. + + `Inheritance:' + + This field is not inherited. + + -- C Member: PyObject* PyTypeObject.tp_subclasses + + List of weak references to subclasses. Internal use only. + + `Inheritance:' + + This field is not inherited. + + -- C Member: PyObject* PyTypeObject.tp_weaklist + + Weak reference list head, for weak references to this type object. + Not inherited. Internal use only. + + `Inheritance:' + + This field is not inherited. + + -- C Member: destructor PyTypeObject.tp_del + + This field is deprecated. Use *note tp_finalize: 2d7. instead. + + -- C Member: unsigned int PyTypeObject.tp_version_tag + + Used to index into the method cache. Internal use only. + + `Inheritance:' + + This field is not inherited. + + -- C Member: destructor PyTypeObject.tp_finalize + + An optional pointer to an instance finalization function. Its + signature is: + + void tp_finalize(PyObject *self); + + If *note tp_finalize: 2d7. is set, the interpreter calls it once + when finalizing an instance. It is called either from the garbage + collector (if the instance is part of an isolated reference cycle) + or just before the object is deallocated. Either way, it is + guaranteed to be called before attempting to break reference + cycles, ensuring that it finds the object in a sane state. + + *note tp_finalize: 2d7. should not mutate the current exception + status; therefore, a recommended way to write a non-trivial + finalizer is: + + static void + local_finalize(PyObject *self) + { + PyObject *error_type, *error_value, *error_traceback; + + /* Save the current exception, if any. */ + PyErr_Fetch(&error_type, &error_value, &error_traceback); + + /* ... */ + + /* Restore the saved exception. */ + PyErr_Restore(error_type, error_value, error_traceback); + } + + For this field to be taken into account (even through inheritance), + you must also set the *note Py_TPFLAGS_HAVE_FINALIZE: 2d8. flags + bit. + + `Inheritance:' + + This field is inherited by subtypes. + + New in version 3.4. + + See also + ........ + + “Safe object finalization” ( PEP 442(1)) + +The remaining fields are only defined if the feature test macro +‘COUNT_ALLOCS’ is defined, and are for internal use only. They are +documented here for completeness. None of these fields are inherited by +subtypes. + + -- C Member: Py_ssize_t PyTypeObject.tp_allocs + + Number of allocations. + + -- C Member: Py_ssize_t PyTypeObject.tp_frees + + Number of frees. + + -- C Member: Py_ssize_t PyTypeObject.tp_maxalloc + + Maximum simultaneously allocated objects. + + -- C Member: PyTypeObject* PyTypeObject.tp_prev + + Pointer to the previous type object with a non-zero *note + tp_allocs: 3dd7. field. + + -- C Member: PyTypeObject* PyTypeObject.tp_next + + Pointer to the next type object with a non-zero *note tp_allocs: + 3dd7. field. + +Also, note that, in a garbage collected Python, *note tp_dealloc: 387f. +may be called from any Python thread, not just the thread which created +the object (if the object becomes part of a refcount cycle, that cycle +might be collected by a garbage collection on any thread). This is not +a problem for Python API calls, since the thread on which tp_dealloc is +called will own the Global Interpreter Lock (GIL). However, if the +object being destroyed in turn destroys objects from some other C or C++ +library, care should be taken to ensure that destroying those objects on +the thread which called tp_dealloc will not violate any assumptions of +the library. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0442 + + +File: python.info, Node: Heap Types, Prev: PyTypeObject Slots, Up: Type Objects<3> + +7.12.3.9 Heap Types +................... + +Traditionally, types defined in C code are `static', that is, a static +*note PyTypeObject: 2d6. structure is defined directly in code and +initialized using *note PyType_Ready(): 3882. + +This results in types that are limited relative to types defined in +Python: + + * Static types are limited to one base, i.e. they cannot use + multiple inheritance. + + * Static type objects (but not necessarily their instances) are + immutable. It is not possible to add or modify the type object’s + attributes from Python. + + * Static type objects are shared across *note sub-interpreters: 398b, + so they should not include any subinterpreter-specific state. + +Also, since *note PyTypeObject: 2d6. is not part of the *note stable +ABI: 3905, any extension modules using static types must be compiled for +a specific Python minor version. + +An alternative to static types is `heap-allocated types', or `heap +types' for short, which correspond closely to classes created by +Python’s ‘class’ statement. + +This is done by filling a *note PyType_Spec: 3abf. structure and calling +*note PyType_FromSpecWithBases(): 3abd. + + +File: python.info, Node: Number Object Structures, Next: Mapping Object Structures, Prev: Type Objects<3>, Up: Object Implementation Support + +7.12.4 Number Object Structures +------------------------------- + + -- C Type: PyNumberMethods + + This structure holds pointers to the functions which an object uses + to implement the number protocol. Each function is used by the + function of similar name documented in the *note Number Protocol: + 3a1f. section. + + Here is the structure definition: + + typedef struct { + binaryfunc nb_add; + binaryfunc nb_subtract; + binaryfunc nb_multiply; + binaryfunc nb_remainder; + binaryfunc nb_divmod; + ternaryfunc nb_power; + unaryfunc nb_negative; + unaryfunc nb_positive; + unaryfunc nb_absolute; + inquiry nb_bool; + unaryfunc nb_invert; + binaryfunc nb_lshift; + binaryfunc nb_rshift; + binaryfunc nb_and; + binaryfunc nb_xor; + binaryfunc nb_or; + unaryfunc nb_int; + void *nb_reserved; + unaryfunc nb_float; + + binaryfunc nb_inplace_add; + binaryfunc nb_inplace_subtract; + binaryfunc nb_inplace_multiply; + binaryfunc nb_inplace_remainder; + ternaryfunc nb_inplace_power; + binaryfunc nb_inplace_lshift; + binaryfunc nb_inplace_rshift; + binaryfunc nb_inplace_and; + binaryfunc nb_inplace_xor; + binaryfunc nb_inplace_or; + + binaryfunc nb_floor_divide; + binaryfunc nb_true_divide; + binaryfunc nb_inplace_floor_divide; + binaryfunc nb_inplace_true_divide; + + unaryfunc nb_index; + + binaryfunc nb_matrix_multiply; + binaryfunc nb_inplace_matrix_multiply; + } PyNumberMethods; + + Note: Binary and ternary functions must check the type of all + their operands, and implement the necessary conversions (at + least one of the operands is an instance of the defined type). + If the operation is not defined for the given operands, binary + and ternary functions must return ‘Py_NotImplemented’, if + another error occurred they must return ‘NULL’ and set an + exception. + + Note: The ‘nb_reserved’ field should always be ‘NULL’. It was + previously called ‘nb_long’, and was renamed in Python 3.0.1. + + -- C Member: binaryfunc PyNumberMethods.nb_add + + -- C Member: binaryfunc PyNumberMethods.nb_subtract + + -- C Member: binaryfunc PyNumberMethods.nb_multiply + + -- C Member: binaryfunc PyNumberMethods.nb_remainder + + -- C Member: binaryfunc PyNumberMethods.nb_divmod + + -- C Member: ternaryfunc PyNumberMethods.nb_power + + -- C Member: unaryfunc PyNumberMethods.nb_negative + + -- C Member: unaryfunc PyNumberMethods.nb_positive + + -- C Member: unaryfunc PyNumberMethods.nb_absolute + + -- C Member: inquiry PyNumberMethods.nb_bool + + -- C Member: unaryfunc PyNumberMethods.nb_invert + + -- C Member: binaryfunc PyNumberMethods.nb_lshift + + -- C Member: binaryfunc PyNumberMethods.nb_rshift + + -- C Member: binaryfunc PyNumberMethods.nb_and + + -- C Member: binaryfunc PyNumberMethods.nb_xor + + -- C Member: binaryfunc PyNumberMethods.nb_or + + -- C Member: unaryfunc PyNumberMethods.nb_int + + -- C Member: void *PyNumberMethods.nb_reserved + + -- C Member: unaryfunc PyNumberMethods.nb_float + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_add + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_subtract + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_multiply + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_remainder + + -- C Member: ternaryfunc PyNumberMethods.nb_inplace_power + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_lshift + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_rshift + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_and + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_xor + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_or + + -- C Member: binaryfunc PyNumberMethods.nb_floor_divide + + -- C Member: binaryfunc PyNumberMethods.nb_true_divide + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_floor_divide + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_true_divide + + -- C Member: unaryfunc PyNumberMethods.nb_index + + -- C Member: binaryfunc PyNumberMethods.nb_matrix_multiply + + -- C Member: binaryfunc PyNumberMethods.nb_inplace_matrix_multiply + + +File: python.info, Node: Mapping Object Structures, Next: Sequence Object Structures, Prev: Number Object Structures, Up: Object Implementation Support + +7.12.5 Mapping Object Structures +-------------------------------- + + -- C Type: PyMappingMethods + + This structure holds pointers to the functions which an object uses + to implement the mapping protocol. It has three members: + + -- C Member: lenfunc PyMappingMethods.mp_length + + This function is used by *note PyMapping_Size(): 3a5f. and *note + PyObject_Size(): 3a19, and has the same signature. This slot may + be set to ‘NULL’ if the object has no defined length. + + -- C Member: binaryfunc PyMappingMethods.mp_subscript + + This function is used by *note PyObject_GetItem(): 38f5. and *note + PySequence_GetSlice(): 3a4c, and has the same signature as + ‘PyObject_GetItem()’. This slot must be filled for the *note + PyMapping_Check(): 3a5e. function to return ‘1’, it can be ‘NULL’ + otherwise. + + -- C Member: objobjargproc PyMappingMethods.mp_ass_subscript + + This function is used by *note PyObject_SetItem(): 38f3, *note + PyObject_DelItem(): 3a1b, ‘PyObject_SetSlice()’ and + ‘PyObject_DelSlice()’. It has the same signature as + ‘PyObject_SetItem()’, but `v' can also be set to ‘NULL’ to delete + an item. If this slot is ‘NULL’, the object does not support item + assignment and deletion. + + +File: python.info, Node: Sequence Object Structures, Next: Buffer Object Structures, Prev: Mapping Object Structures, Up: Object Implementation Support + +7.12.6 Sequence Object Structures +--------------------------------- + + -- C Type: PySequenceMethods + + This structure holds pointers to the functions which an object uses + to implement the sequence protocol. + + -- C Member: lenfunc PySequenceMethods.sq_length + + This function is used by *note PySequence_Size(): 3a46. and *note + PyObject_Size(): 3a19, and has the same signature. It is also used + for handling negative indices via the *note sq_item: 3e0d. and the + *note sq_ass_item: 3e0e. slots. + + -- C Member: binaryfunc PySequenceMethods.sq_concat + + This function is used by *note PySequence_Concat(): 3a48. and has + the same signature. It is also used by the ‘+’ operator, after + trying the numeric addition via the *note nb_add: 3ac8. slot. + + -- C Member: ssizeargfunc PySequenceMethods.sq_repeat + + This function is used by *note PySequence_Repeat(): 3a49. and has + the same signature. It is also used by the ‘*’ operator, after + trying numeric multiplication via the *note nb_multiply: 3de5. + slot. + + -- C Member: ssizeargfunc PySequenceMethods.sq_item + + This function is used by *note PySequence_GetItem(): 38f6. and has + the same signature. It is also used by *note PyObject_GetItem(): + 38f5, after trying the subscription via the *note mp_subscript: + 3e07. slot. This slot must be filled for the *note + PySequence_Check(): 3a45. function to return ‘1’, it can be ‘NULL’ + otherwise. + + Negative indexes are handled as follows: if the ‘sq_length’ slot is + filled, it is called and the sequence length is used to compute a + positive index which is passed to ‘sq_item’. If ‘sq_length’ is + ‘NULL’, the index is passed as is to the function. + + -- C Member: ssizeobjargproc PySequenceMethods.sq_ass_item + + This function is used by *note PySequence_SetItem(): 38f2. and has + the same signature. It is also used by *note PyObject_SetItem(): + 38f3. and *note PyObject_DelItem(): 3a1b, after trying the item + assignment and deletion via the *note mp_ass_subscript: 3e08. slot. + This slot may be left to ‘NULL’ if the object does not support item + assignment and deletion. + + -- C Member: objobjproc PySequenceMethods.sq_contains + + This function may be used by *note PySequence_Contains(): 3a51. and + has the same signature. This slot may be left to ‘NULL’, in this + case ‘PySequence_Contains()’ simply traverses the sequence until it + finds a match. + + -- C Member: binaryfunc PySequenceMethods.sq_inplace_concat + + This function is used by *note PySequence_InPlaceConcat(): 3a4a. + and has the same signature. It should modify its first operand, + and return it. This slot may be left to ‘NULL’, in this case + ‘PySequence_InPlaceConcat()’ will fall back to *note + PySequence_Concat(): 3a48. It is also used by the augmented + assignment ‘+=’, after trying numeric in-place addition via the + *note nb_inplace_add: 3de2. slot. + + -- C Member: ssizeargfunc PySequenceMethods.sq_inplace_repeat + + This function is used by *note PySequence_InPlaceRepeat(): 3a4b. + and has the same signature. It should modify its first operand, + and return it. This slot may be left to ‘NULL’, in this case + ‘PySequence_InPlaceRepeat()’ will fall back to *note + PySequence_Repeat(): 3a49. It is also used by the augmented + assignment ‘*=’, after trying numeric in-place multiplication via + the *note nb_inplace_multiply: 3de6. slot. + + +File: python.info, Node: Buffer Object Structures, Next: Async Object Structures, Prev: Sequence Object Structures, Up: Object Implementation Support + +7.12.7 Buffer Object Structures +------------------------------- + + -- C Type: PyBufferProcs + + This structure holds pointers to the functions required by the + *note Buffer protocol: 1291. The protocol defines how an exporter + object can expose its internal data to consumer objects. + + -- C Member: getbufferproc PyBufferProcs.bf_getbuffer + + The signature of this function is: + + int (PyObject *exporter, Py_buffer *view, int flags); + + Handle a request to `exporter' to fill in `view' as specified by + `flags'. Except for point (3), an implementation of this function + MUST take these steps: + + 1. Check if the request can be met. If not, raise + ‘PyExc_BufferError’, set ‘view->obj’ to ‘NULL’ and return + ‘-1’. + + 2. Fill in the requested fields. + + 3. Increment an internal counter for the number of exports. + + 4. Set ‘view->obj’ to `exporter' and increment ‘view->obj’. + + 5. Return ‘0’. + + If `exporter' is part of a chain or tree of buffer providers, two + main schemes can be used: + + * Re-export: Each member of the tree acts as the exporting + object and sets ‘view->obj’ to a new reference to itself. + + * Redirect: The buffer request is redirected to the root object + of the tree. Here, ‘view->obj’ will be a new reference to the + root object. + + The individual fields of `view' are described in section *note + Buffer structure: 3a6f, the rules how an exporter must react to + specific requests are in section *note Buffer request types: 3a83. + + All memory pointed to in the *note Py_buffer: b1b. structure + belongs to the exporter and must remain valid until there are no + consumers left. *note format: 3a7c, *note shape: 3a7e, *note + strides: 3a73, *note suboffsets: 3a80. and *note internal: 3a82. + are read-only for the consumer. + + *note PyBuffer_FillInfo(): 3a76. provides an easy way of exposing a + simple bytes buffer while dealing correctly with all request types. + + *note PyObject_GetBuffer(): d25. is the interface for the consumer + that wraps this function. + + -- C Member: releasebufferproc PyBufferProcs.bf_releasebuffer + + The signature of this function is: + + void (PyObject *exporter, Py_buffer *view); + + Handle a request to release the resources of the buffer. If no + resources need to be released, *note + PyBufferProcs.bf_releasebuffer: 39c9. may be ‘NULL’. Otherwise, a + standard implementation of this function will take these optional + steps: + + 1. Decrement an internal counter for the number of exports. + + 2. If the counter is ‘0’, free all memory associated with `view'. + + The exporter MUST use the *note internal: 3a82. field to keep track + of buffer-specific resources. This field is guaranteed to remain + constant, while a consumer MAY pass a copy of the original buffer + as the `view' argument. + + This function MUST NOT decrement ‘view->obj’, since that is done + automatically in *note PyBuffer_Release(): d54. (this scheme is + useful for breaking reference cycles). + + *note PyBuffer_Release(): d54. is the interface for the consumer + that wraps this function. + + +File: python.info, Node: Async Object Structures, Next: Slot Type typedefs, Prev: Buffer Object Structures, Up: Object Implementation Support + +7.12.8 Async Object Structures +------------------------------ + +New in version 3.5. + + -- C Type: PyAsyncMethods + + This structure holds pointers to the functions required to + implement *note awaitable: 519. and *note asynchronous iterator: + 645. objects. + + Here is the structure definition: + + typedef struct { + unaryfunc am_await; + unaryfunc am_aiter; + unaryfunc am_anext; + } PyAsyncMethods; + + -- C Member: unaryfunc PyAsyncMethods.am_await + + The signature of this function is: + + PyObject *am_await(PyObject *self); + + The returned object must be an iterator, i.e. *note + PyIter_Check(): 3a6a. must return ‘1’ for it. + + This slot may be set to ‘NULL’ if an object is not an *note + awaitable: 519. + + -- C Member: unaryfunc PyAsyncMethods.am_aiter + + The signature of this function is: + + PyObject *am_aiter(PyObject *self); + + Must return an *note awaitable: 519. object. See *note + __anext__(): 1138. for details. + + This slot may be set to ‘NULL’ if an object does not implement + asynchronous iteration protocol. + + -- C Member: unaryfunc PyAsyncMethods.am_anext + + The signature of this function is: + + PyObject *am_anext(PyObject *self); + + Must return an *note awaitable: 519. object. See *note + __anext__(): 1138. for details. This slot may be set to ‘NULL’. + + +File: python.info, Node: Slot Type typedefs, Next: Examples<35>, Prev: Async Object Structures, Up: Object Implementation Support + +7.12.9 Slot Type typedefs +------------------------- + + -- C Type: PyObject *(*allocfunc) (PyTypeObject *cls, + Py_ssize_t nitems) + + The purpose of this function is to separate memory allocation from + memory initialization. It should return a pointer to a block of + memory of adequate length for the instance, suitably aligned, and + initialized to zeros, but with ‘ob_refcnt’ set to ‘1’ and ‘ob_type’ + set to the type argument. If the type’s *note tp_itemsize: 3878. + is non-zero, the object’s ‘ob_size’ field should be initialized to + `nitems' and the length of the allocated memory block should be + ‘tp_basicsize + nitems*tp_itemsize’, rounded up to a multiple of + ‘sizeof(void*)’; otherwise, `nitems' is not used and the length of + the block should be *note tp_basicsize: 3879. + + This function should not do any other instance initialization, not + even to allocate additional memory; that should be done by *note + tp_new: 387c. + + -- C Type: void (*destructor) (PyObject *) + + -- C Type: PyObject *(*vectorcallfunc) (PyObject *callable, PyObject + *const *args, size_t nargsf, PyObject *kwnames) + + See *note tp_vectorcall_offset: 3a11. + + Arguments to ‘vectorcallfunc’ are the same as for *note + _PyObject_Vectorcall(): 3a10. + + New in version 3.8. + + -- C Type: void (*freefunc) (void *) + + See *note tp_free: 3880. + + -- C Type: PyObject *(*newfunc) (PyObject *, PyObject *, PyObject *) + + See *note tp_new: 387c. + + -- C Type: int (*initproc) (PyObject *, PyObject *, PyObject *) + + See *note tp_init: 3883. + + -- C Type: PyObject *(*reprfunc) (PyObject *) + + See *note tp_repr: 389a. + + -- C Type: PyObject *(*getattrfunc) (PyObject *self, char *attr) + + Return the value of the named attribute for the object. + + -- C Type: int (*setattrfunc) (PyObject *self, char *attr, + PyObject *value) + + Set the value of the named attribute for the object. The value + argument is set to ‘NULL’ to delete the attribute. + + -- C Type: PyObject *(*getattrofunc) (PyObject *self, PyObject *attr) + + Return the value of the named attribute for the object. + + See *note tp_getattro: 389f. + + -- C Type: int (*setattrofunc) (PyObject *self, PyObject *attr, + PyObject *value) + + Set the value of the named attribute for the object. The value + argument is set to ‘NULL’ to delete the attribute. + + See *note tp_setattro: 38a0. + + -- C Type: PyObject *(*descrgetfunc) (PyObject *, PyObject *, + PyObject *) + + See ‘tp_descrget’. + + -- C Type: int (*descrsetfunc) (PyObject *, PyObject *, PyObject *) + + See ‘tp_descrset’. + + -- C Type: Py_hash_t (*hashfunc) (PyObject *) + + See *note tp_hash: 38ac. + + -- C Type: PyObject *(*richcmpfunc) (PyObject *, PyObject *, int) + + See *note tp_richcompare: 38a5. + + -- C Type: PyObject *(*getiterfunc) (PyObject *) + + See *note tp_iter: e30. + + -- C Type: PyObject *(*iternextfunc) (PyObject *) + + See *note tp_iternext: e31. + + -- C Type: Py_ssize_t (*lenfunc) (PyObject *) + + -- C Type: int (*getbufferproc) (PyObject *, Py_buffer *, int) + + -- C Type: void (*releasebufferproc) (PyObject *, Py_buffer *) + + -- C Type: PyObject *(*unaryfunc) (PyObject *) + + -- C Type: PyObject *(*binaryfunc) (PyObject *, PyObject *) + + -- C Type: PyObject *(*ternaryfunc) (PyObject *, PyObject *, + PyObject *) + + -- C Type: PyObject *(*ssizeargfunc) (PyObject *, Py_ssize_t) + + -- C Type: int (*ssizeobjargproc) (PyObject *, Py_ssize_t) + + -- C Type: int (*objobjproc) (PyObject *, PyObject *) + + -- C Type: int (*objobjargproc) (PyObject *, PyObject *, PyObject *) + + +File: python.info, Node: Examples<35>, Next: Supporting Cyclic Garbage Collection, Prev: Slot Type typedefs, Up: Object Implementation Support + +7.12.10 Examples +---------------- + +The following are simple examples of Python type definitions. They +include common usage you may encounter. Some demonstrate tricky corner +cases. For more examples, practical info, and a tutorial, see *note +Defining Extension Types; Tutorial: 3871. and *note Defining Extension +Types; Assorted Topics: 3894. + +A basic static type: + + typedef struct { + PyObject_HEAD + const char *data; + } MyObject; + + static PyTypeObject MyObject_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "mymod.MyObject", + .tp_basicsize = sizeof(MyObject), + .tp_doc = "My objects", + .tp_new = myobj_new, + .tp_dealloc = (destructor)myobj_dealloc, + .tp_repr = (reprfunc)myobj_repr, + }; + +You may also find older code (especially in the CPython code base) with +a more verbose initializer: + + static PyTypeObject MyObject_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + "mymod.MyObject", /* tp_name */ + sizeof(MyObject), /* tp_basicsize */ + 0, /* tp_itemsize */ + (destructor)myobj_dealloc, /* tp_dealloc */ + 0, /* tp_vectorcall_offset */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_as_async */ + (reprfunc)myobj_repr, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + "My objects", /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + myobj_new, /* tp_new */ + }; + +A type that supports weakrefs, instance dicts, and hashing: + + typedef struct { + PyObject_HEAD + const char *data; + PyObject *inst_dict; + PyObject *weakreflist; + } MyObject; + + static PyTypeObject MyObject_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "mymod.MyObject", + .tp_basicsize = sizeof(MyObject), + .tp_doc = "My objects", + .tp_weaklistoffset = offsetof(MyObject, weakreflist), + .tp_dictoffset = offsetof(MyObject, inst_dict), + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, + .tp_new = myobj_new, + .tp_traverse = (traverseproc)myobj_traverse, + .tp_clear = (inquiry)myobj_clear, + .tp_alloc = PyType_GenericNew, + .tp_dealloc = (destructor)myobj_dealloc, + .tp_repr = (reprfunc)myobj_repr, + .tp_hash = (hashfunc)myobj_hash, + .tp_richcompare = PyBaseObject_Type.tp_richcompare, + }; + +A str subclass that cannot be subclassed and cannot be called to create +instances (e.g. uses a separate factory func): + + typedef struct { + PyUnicodeObject raw; + char *extra; + } MyStr; + + static PyTypeObject MyStr_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "mymod.MyStr", + .tp_basicsize = sizeof(MyStr), + .tp_base = NULL, // set to &PyUnicode_Type in module init + .tp_doc = "my custom str", + .tp_flags = Py_TPFLAGS_DEFAULT, + .tp_new = NULL, + .tp_repr = (reprfunc)myobj_repr, + }; + +The simplest static type (with fixed-length instances): + + typedef struct { + PyObject_HEAD + } MyObject; + + static PyTypeObject MyObject_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "mymod.MyObject", + }; + +The simplest static type (with variable-length instances): + + typedef struct { + PyObject_VAR_HEAD + const char *data[1]; + } MyObject; + + static PyTypeObject MyObject_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + .tp_name = "mymod.MyObject", + .tp_basicsize = sizeof(MyObject) - sizeof(char *), + .tp_itemsize = sizeof(char *), + }; + + +File: python.info, Node: Supporting Cyclic Garbage Collection, Prev: Examples<35>, Up: Object Implementation Support + +7.12.11 Supporting Cyclic Garbage Collection +-------------------------------------------- + +Python’s support for detecting and collecting garbage which involves +circular references requires support from object types which are +“containers” for other objects which may also be containers. Types +which do not store references to other objects, or which only store +references to atomic types (such as numbers or strings), do not need to +provide any explicit support for garbage collection. + +To create a container type, the *note tp_flags: 3ab6. field of the type +object must include the *note Py_TPFLAGS_HAVE_GC: 388e. and provide an +implementation of the *note tp_traverse: 33e8. handler. If instances of +the type are mutable, a *note tp_clear: 3898. implementation must also +be provided. + + -- Data: Py_TPFLAGS_HAVE_GC + + Objects with a type with this flag set must conform with the rules + documented here. For convenience these objects will be referred to + as container objects. + +Constructors for container types must conform to two rules: + + 1. The memory for the object must be allocated using *note + PyObject_GC_New(): 2d4. or *note PyObject_GC_NewVar(): 2d5. + + 2. Once all the fields which may contain references to other + containers are initialized, it must call *note PyObject_GC_Track(): + e44. + + -- C Function: TYPE* PyObject_GC_New (TYPE, PyTypeObject *type) + + Analogous to *note PyObject_New(): 2d2. but for container objects + with the *note Py_TPFLAGS_HAVE_GC: 388e. flag set. + + -- C Function: TYPE* PyObject_GC_NewVar (TYPE, PyTypeObject *type, + Py_ssize_t size) + + Analogous to *note PyObject_NewVar(): 2d3. but for container + objects with the *note Py_TPFLAGS_HAVE_GC: 388e. flag set. + + -- C Function: TYPE* PyObject_GC_Resize (TYPE, PyVarObject *op, + Py_ssize_t newsize) + + Resize an object allocated by *note PyObject_NewVar(): 2d3. + Returns the resized object or ‘NULL’ on failure. `op' must not be + tracked by the collector yet. + + -- C Function: void PyObject_GC_Track (PyObject *op) + + Adds the object `op' to the set of container objects tracked by the + collector. The collector can run at unexpected times so objects + must be valid while being tracked. This should be called once all + the fields followed by the *note tp_traverse: 33e8. handler become + valid, usually near the end of the constructor. + +Similarly, the deallocator for the object must conform to a similar pair +of rules: + + 1. Before fields which refer to other containers are invalidated, + *note PyObject_GC_UnTrack(): e45. must be called. + + 2. The object’s memory must be deallocated using *note + PyObject_GC_Del(): e43. + + -- C Function: void PyObject_GC_Del (void *op) + + Releases memory allocated to an object using *note + PyObject_GC_New(): 2d4. or *note PyObject_GC_NewVar(): 2d5. + + -- C Function: void PyObject_GC_UnTrack (void *op) + + Remove the object `op' from the set of container objects tracked by + the collector. Note that *note PyObject_GC_Track(): e44. can be + called again on this object to add it back to the set of tracked + objects. The deallocator (*note tp_dealloc: 387f. handler) should + call this for the object before any of the fields used by the *note + tp_traverse: 33e8. handler become invalid. + +Changed in version 3.8: The ‘_PyObject_GC_TRACK()’ and +‘_PyObject_GC_UNTRACK()’ macros have been removed from the public C API. + +The *note tp_traverse: 33e8. handler accepts a function parameter of +this type: + + -- C Type: int (*visitproc) (PyObject *object, void *arg) + + Type of the visitor function passed to the *note tp_traverse: 33e8. + handler. The function should be called with an object to traverse + as `object' and the third parameter to the *note tp_traverse: 33e8. + handler as `arg'. The Python core uses several visitor functions + to implement cyclic garbage detection; it’s not expected that users + will need to write their own visitor functions. + +The *note tp_traverse: 33e8. handler must have the following type: + + -- C Type: int (*traverseproc) (PyObject *self, visitproc visit, + void *arg) + + Traversal function for a container object. Implementations must + call the `visit' function for each object directly contained by + `self', with the parameters to `visit' being the contained object + and the `arg' value passed to the handler. The `visit' function + must not be called with a ‘NULL’ object argument. If `visit' + returns a non-zero value that value should be returned immediately. + +To simplify writing *note tp_traverse: 33e8. handlers, a *note +Py_VISIT(): 388c. macro is provided. In order to use this macro, the +*note tp_traverse: 33e8. implementation must name its arguments exactly +`visit' and `arg': + + -- C Function: void Py_VISIT (PyObject *o) + + If `o' is not ‘NULL’, call the `visit' callback, with arguments `o' + and `arg'. If `visit' returns a non-zero value, then return it. + Using this macro, *note tp_traverse: 33e8. handlers look like: + + static int + my_traverse(Noddy *self, visitproc visit, void *arg) + { + Py_VISIT(self->foo); + Py_VISIT(self->bar); + return 0; + } + +The *note tp_clear: 3898. handler must be of the *note inquiry: 3dc7. +type, or ‘NULL’ if the object is immutable. + + -- C Type: int (*inquiry) (PyObject *self) + + Drop references that may have created reference cycles. Immutable + objects do not have to define this method since they can never + directly create reference cycles. Note that the object must still + be valid after calling this method (don’t just call *note + Py_DECREF(): 266. on a reference). The collector will call this + method if it detects that this object is involved in a reference + cycle. + + +File: python.info, Node: API and ABI Versioning, Prev: Object Implementation Support, Up: Python/C API Reference Manual + +7.13 API and ABI Versioning +=========================== + +‘PY_VERSION_HEX’ is the Python version number encoded in a single +integer. + +For example if the ‘PY_VERSION_HEX’ is set to ‘0x030401a2’, the +underlying version information can be found by treating it as a 32 bit +number in the following manner: + + Bytes Bits (big endian order) Meaning + + ----------------------------------------------------------------------------------------------- + + ‘1’ ‘1-8’ ‘PY_MAJOR_VERSION’ (the ‘3’ in ‘3.4.1a2’) + + + ‘2’ ‘9-16’ ‘PY_MINOR_VERSION’ (the ‘4’ in ‘3.4.1a2’) + + + ‘3’ ‘17-24’ ‘PY_MICRO_VERSION’ (the ‘1’ in ‘3.4.1a2’) + + + ‘4’ ‘25-28’ ‘PY_RELEASE_LEVEL’ (‘0xA’ for alpha, ‘0xB’ for + beta, ‘0xC’ for release candidate and ‘0xF’ for + final), in this case it is alpha. + + + ‘29-32’ ‘PY_RELEASE_SERIAL’ (the ‘2’ in ‘3.4.1a2’, zero + for final releases) + + +Thus ‘3.4.1a2’ is hexversion ‘0x030401a2’. + +All the given macros are defined in Include/patchlevel.h(1). + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Include/patchlevel.h + + +File: python.info, Node: Distributing Python Modules, Next: Installing Python Modules, Prev: Python/C API Reference Manual, Up: Top + +8 Distributing Python Modules +***************************** + + +Email: <distutils-sig@python.org> + +As a popular open source development project, Python has an active +supporting community of contributors and users that also make their +software available for other Python developers to use under open source +license terms. + +This allows Python users to share and collaborate effectively, +benefiting from the solutions others have already created to common (and +sometimes even rare!) problems, as well as potentially contributing +their own solutions to the common pool. + +This guide covers the distribution part of the process. For a guide to +installing other Python projects, refer to the *note installation guide: +7f5. + + Note: For corporate and other institutional users, be aware that + many organisations have their own policies around using and + contributing to open source software. Please take such policies + into account when making use of the distribution and installation + tools provided with Python. + +* Menu: + +* Key terms:: +* Open source licensing and collaboration:: +* Installing the tools:: +* Reading the Python Packaging User Guide:: +* How do I…?:: + + +File: python.info, Node: Key terms, Next: Open source licensing and collaboration, Up: Distributing Python Modules + +8.1 Key terms +============= + + * the Python Packaging Index(1) is a public repository of open source + licensed packages made available for use by other Python users + + * the Python Packaging Authority(2) are the group of developers and + documentation authors responsible for the maintenance and evolution + of the standard packaging tools and the associated metadata and + file format standards. They maintain a variety of tools, + documentation and issue trackers on both GitHub(3) and + Bitbucket(4). + + * *note distutils: 39. is the original build and distribution system + first added to the Python standard library in 1998. While direct + use of *note distutils: 39. is being phased out, it still laid the + foundation for the current packaging and distribution + infrastructure, and it not only remains part of the standard + library, but its name lives on in other ways (such as the name of + the mailing list used to coordinate Python packaging standards + development). + + * setuptools(5) is a (largely) drop-in replacement for *note + distutils: 39. first published in 2004. Its most notable addition + over the unmodified *note distutils: 39. tools was the ability to + declare dependencies on other packages. It is currently + recommended as a more regularly updated alternative to *note + distutils: 39. that offers consistent support for more recent + packaging standards across a wide range of Python versions. + + * wheel(6) (in this context) is a project that adds the ‘bdist_wheel’ + command to *note distutils: 39./setuptools(7). This produces a + cross platform binary packaging format (called “wheels” or “wheel + files” and defined in PEP 427(8)) that allows Python libraries, + even those including binary extensions, to be installed on a system + without needing to be built locally. + + ---------- Footnotes ---------- + + (1) https://pypi.org + + (2) https://www.pypa.io/ + + (3) https://github.com/pypa + + (4) https://bitbucket.org/pypa/ + + (5) https://setuptools.readthedocs.io/en/latest/ + + (6) https://wheel.readthedocs.io/ + + (7) https://setuptools.readthedocs.io/en/latest/ + + (8) https://www.python.org/dev/peps/pep-0427 + + +File: python.info, Node: Open source licensing and collaboration, Next: Installing the tools, Prev: Key terms, Up: Distributing Python Modules + +8.2 Open source licensing and collaboration +=========================================== + +In most parts of the world, software is automatically covered by +copyright. This means that other developers require explicit permission +to copy, use, modify and redistribute the software. + +Open source licensing is a way of explicitly granting such permission in +a relatively consistent way, allowing developers to share and +collaborate efficiently by making common solutions to various problems +freely available. This leaves many developers free to spend more time +focusing on the problems that are relatively unique to their specific +situation. + +The distribution tools provided with Python are designed to make it +reasonably straightforward for developers to make their own +contributions back to that common pool of software if they choose to do +so. + +The same distribution tools can also be used to distribute software +within an organisation, regardless of whether that software is published +as open source software or not. + + +File: python.info, Node: Installing the tools, Next: Reading the Python Packaging User Guide, Prev: Open source licensing and collaboration, Up: Distributing Python Modules + +8.3 Installing the tools +======================== + +The standard library does not include build tools that support modern +Python packaging standards, as the core development team has found that +it is important to have standard tools that work consistently, even on +older versions of Python. + +The currently recommended build and distribution tools can be installed +by invoking the ‘pip’ module at the command line: + + python -m pip install setuptools wheel twine + + Note: For POSIX users (including Mac OS X and Linux users), these + instructions assume the use of a *note virtual environment: fa8. + + For Windows users, these instructions assume that the option to + adjust the system PATH environment variable was selected when + installing Python. + +The Python Packaging User Guide includes more details on the currently +recommended tools(1). + + ---------- Footnotes ---------- + + (1) +https://packaging.python.org/guides/tool-recommendations/#packaging-tool-recommendations + + +File: python.info, Node: Reading the Python Packaging User Guide, Next: How do I…?, Prev: Installing the tools, Up: Distributing Python Modules + +8.4 Reading the Python Packaging User Guide +=========================================== + +The Python Packaging User Guide covers the various key steps and +elements involved in creating and publishing a project: + + * Project structure(1) + + * Building and packaging the project(2) + + * Uploading the project to the Python Packaging Index(3) + + * The .pypirc file(4) + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/tutorials/distributing-packages/ + + (2) +https://packaging.python.org/tutorials/distributing-packages/#packaging-your-project + + (3) +https://packaging.python.org/tutorials/distributing-packages/#uploading-your-project-to-pypi + + (4) https://packaging.python.org/specifications/pypirc/ + + +File: python.info, Node: How do I…?, Prev: Reading the Python Packaging User Guide, Up: Distributing Python Modules + +8.5 How do I…? +============== + +These are quick answers or links for some common tasks. + +* Menu: + +* … choose a name for my project?:: +* … create and distribute binary extensions?:: + + +File: python.info, Node: … choose a name for my project?, Next: … create and distribute binary extensions?, Up: How do I…? + +8.5.1 … choose a name for my project? +------------------------------------- + +This isn’t an easy topic, but here are a few tips: + + * check the Python Packaging Index to see if the name is already in + use + + * check popular hosting sites like GitHub, Bitbucket, etc to see if + there is already a project with that name + + * check what comes up in a web search for the name you’re considering + + * avoid particularly common words, especially ones with multiple + meanings, as they can make it difficult for users to find your + software when searching for it + + +File: python.info, Node: … create and distribute binary extensions?, Prev: … choose a name for my project?, Up: How do I…? + +8.5.2 … create and distribute binary extensions? +------------------------------------------------ + +This is actually quite a complex topic, with a variety of alternatives +available depending on exactly what you’re aiming to achieve. See the +Python Packaging User Guide for more information and recommendations. + +See also +........ + +Python Packaging User Guide: Binary Extensions(1) + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/guides/packaging-binary-extensions/ + + +File: python.info, Node: Installing Python Modules, Next: Python HOWTOs, Prev: Distributing Python Modules, Up: Top + +9 Installing Python Modules +*************************** + + +Email: <distutils-sig@python.org> + +As a popular open source development project, Python has an active +supporting community of contributors and users that also make their +software available for other Python developers to use under open source +license terms. + +This allows Python users to share and collaborate effectively, +benefiting from the solutions others have already created to common (and +sometimes even rare!) problems, as well as potentially contributing +their own solutions to the common pool. + +This guide covers the installation part of the process. For a guide to +creating and sharing your own Python projects, refer to the *note +distribution guide: 7f6. + + Note: For corporate and other institutional users, be aware that + many organisations have their own policies around using and + contributing to open source software. Please take such policies + into account when making use of the distribution and installation + tools provided with Python. + +* Menu: + +* Key terms: Key terms<2>. +* Basic usage:: +* How do I …?:: +* Common installation issues:: + + +File: python.info, Node: Key terms<2>, Next: Basic usage, Up: Installing Python Modules + +9.1 Key terms +============= + + * ‘pip’ is the preferred installer program. Starting with Python + 3.4, it is included by default with the Python binary installers. + + * A `virtual environment' is a semi-isolated Python environment that + allows packages to be installed for use by a particular + application, rather than being installed system wide. + + * ‘venv’ is the standard tool for creating virtual environments, and + has been part of Python since Python 3.3. Starting with Python + 3.4, it defaults to installing ‘pip’ into all created virtual + environments. + + * ‘virtualenv’ is a third party alternative (and predecessor) to + ‘venv’. It allows virtual environments to be used on versions of + Python prior to 3.4, which either don’t provide ‘venv’ at all, or + aren’t able to automatically install ‘pip’ into created + environments. + + * The Python Packaging Index(1) is a public repository of open source + licensed packages made available for use by other Python users. + + * the Python Packaging Authority(2) is the group of developers and + documentation authors responsible for the maintenance and evolution + of the standard packaging tools and the associated metadata and + file format standards. They maintain a variety of tools, + documentation, and issue trackers on both GitHub(3) and + Bitbucket(4). + + * ‘distutils’ is the original build and distribution system first + added to the Python standard library in 1998. While direct use of + ‘distutils’ is being phased out, it still laid the foundation for + the current packaging and distribution infrastructure, and it not + only remains part of the standard library, but its name lives on in + other ways (such as the name of the mailing list used to coordinate + Python packaging standards development). + +Changed in version 3.5: The use of ‘venv’ is now recommended for +creating virtual environments. + +See also +........ + +Python Packaging User Guide: Creating and using virtual environments(5) + + ---------- Footnotes ---------- + + (1) https://pypi.org + + (2) https://www.pypa.io/ + + (3) https://github.com/pypa + + (4) https://bitbucket.org/pypa/ + + (5) +https://packaging.python.org/installing/#creating-virtual-environments + + +File: python.info, Node: Basic usage, Next: How do I …?, Prev: Key terms<2>, Up: Installing Python Modules + +9.2 Basic usage +=============== + +The standard packaging tools are all designed to be used from the +command line. + +The following command will install the latest version of a module and +its dependencies from the Python Packaging Index: + + python -m pip install SomePackage + + Note: For POSIX users (including Mac OS X and Linux users), the + examples in this guide assume the use of a *note virtual + environment: fa8. + + For Windows users, the examples in this guide assume that the + option to adjust the system PATH environment variable was selected + when installing Python. + +It’s also possible to specify an exact or minimum version directly on +the command line. When using comparator operators such as ‘>’, ‘<’ or +some other special character which get interpreted by shell, the package +name and the version should be enclosed within double quotes: + + python -m pip install SomePackage==1.0.4 # specific version + python -m pip install "SomePackage>=1.0.4" # minimum version + +Normally, if a suitable module is already installed, attempting to +install it again will have no effect. Upgrading existing modules must +be requested explicitly: + + python -m pip install --upgrade SomePackage + +More information and resources regarding ‘pip’ and its capabilities can +be found in the Python Packaging User Guide(1). + +Creation of virtual environments is done through the *note venv: 125. +module. Installing packages into an active virtual environment uses the +commands shown above. + +See also +........ + +Python Packaging User Guide: Installing Python Distribution Packages(2) + + ---------- Footnotes ---------- + + (1) https://packaging.python.org + + (2) https://packaging.python.org/installing/ + + +File: python.info, Node: How do I …?, Next: Common installation issues, Prev: Basic usage, Up: Installing Python Modules + +9.3 How do I …? +=============== + +These are quick answers or links for some common tasks. + +* Menu: + +* … install pip in versions of Python prior to Python 3.4?: … install pip in versions of Python prior to Python 3 4?. +* … install packages just for the current user?:: +* … install scientific Python packages?:: +* … work with multiple versions of Python installed in parallel?:: + + +File: python.info, Node: … install pip in versions of Python prior to Python 3 4?, Next: … install packages just for the current user?, Up: How do I …? + +9.3.1 … install ‘pip’ in versions of Python prior to Python 3.4? +---------------------------------------------------------------- + +Python only started bundling ‘pip’ with Python 3.4. For earlier +versions, ‘pip’ needs to be “bootstrapped” as described in the Python +Packaging User Guide. + +See also +........ + +Python Packaging User Guide: Requirements for Installing Packages(1) + + ---------- Footnotes ---------- + + (1) +https://packaging.python.org/installing/#requirements-for-installing-packages + + +File: python.info, Node: … install packages just for the current user?, Next: … install scientific Python packages?, Prev: … install pip in versions of Python prior to Python 3 4?, Up: How do I …? + +9.3.2 … install packages just for the current user? +--------------------------------------------------- + +Passing the ‘--user’ option to ‘python -m pip install’ will install a +package just for the current user, rather than for all users of the +system. + + +File: python.info, Node: … install scientific Python packages?, Next: … work with multiple versions of Python installed in parallel?, Prev: … install packages just for the current user?, Up: How do I …? + +9.3.3 … install scientific Python packages? +------------------------------------------- + +A number of scientific Python packages have complex binary dependencies, +and aren’t currently easy to install using ‘pip’ directly. At this +point in time, it will often be easier for users to install these +packages by other means(1) rather than attempting to install them with +‘pip’. + +See also +........ + +Python Packaging User Guide: Installing Scientific Packages(2) + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/science/ + + (2) https://packaging.python.org/science/ + + +File: python.info, Node: … work with multiple versions of Python installed in parallel?, Prev: … install scientific Python packages?, Up: How do I …? + +9.3.4 … work with multiple versions of Python installed in parallel? +-------------------------------------------------------------------- + +On Linux, Mac OS X, and other POSIX systems, use the versioned Python +commands in combination with the ‘-m’ switch to run the appropriate copy +of ‘pip’: + + python2 -m pip install SomePackage # default Python 2 + python2.7 -m pip install SomePackage # specifically Python 2.7 + python3 -m pip install SomePackage # default Python 3 + python3.4 -m pip install SomePackage # specifically Python 3.4 + +Appropriately versioned ‘pip’ commands may also be available. + +On Windows, use the ‘py’ Python launcher in combination with the ‘-m’ +switch: + + py -2 -m pip install SomePackage # default Python 2 + py -2.7 -m pip install SomePackage # specifically Python 2.7 + py -3 -m pip install SomePackage # default Python 3 + py -3.4 -m pip install SomePackage # specifically Python 3.4 + + +File: python.info, Node: Common installation issues, Prev: How do I …?, Up: Installing Python Modules + +9.4 Common installation issues +============================== + +* Menu: + +* Installing into the system Python on Linux:: +* Pip not installed:: +* Installing binary extensions:: + + +File: python.info, Node: Installing into the system Python on Linux, Next: Pip not installed, Up: Common installation issues + +9.4.1 Installing into the system Python on Linux +------------------------------------------------ + +On Linux systems, a Python installation will typically be included as +part of the distribution. Installing into this Python installation +requires root access to the system, and may interfere with the operation +of the system package manager and other components of the system if a +component is unexpectedly upgraded using ‘pip’. + +On such systems, it is often better to use a virtual environment or a +per-user installation when installing packages with ‘pip’. + + +File: python.info, Node: Pip not installed, Next: Installing binary extensions, Prev: Installing into the system Python on Linux, Up: Common installation issues + +9.4.2 Pip not installed +----------------------- + +It is possible that ‘pip’ does not get installed by default. One +potential fix is: + + python -m ensurepip --default-pip + +There are also additional resources for installing pip.(1) + + ---------- Footnotes ---------- + + (1) +https://packaging.python.org/tutorials/installing-packages/#install-pip-setuptools-and-wheel + + +File: python.info, Node: Installing binary extensions, Prev: Pip not installed, Up: Common installation issues + +9.4.3 Installing binary extensions +---------------------------------- + +Python has typically relied heavily on source based distribution, with +end users being expected to compile extension modules from source as +part of the installation process. + +With the introduction of support for the binary ‘wheel’ format, and the +ability to publish wheels for at least Windows and Mac OS X through the +Python Packaging Index, this problem is expected to diminish over time, +as users are more regularly able to install pre-built extensions rather +than needing to build them themselves. + +Some of the solutions for installing scientific software(1) that are not +yet available as pre-built ‘wheel’ files may also help with obtaining +other binary extensions without needing to build them locally. + +See also +........ + +Python Packaging User Guide: Binary Extensions(2) + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/science/ + + (2) https://packaging.python.org/extensions/ + + +File: python.info, Node: Python HOWTOs, Next: Python Frequently Asked Questions, Prev: Installing Python Modules, Up: Top + +10 Python HOWTOs +**************** + +Python HOWTOs are documents that cover a single, specific topic, and +attempt to cover it fairly completely. Modelled on the Linux +Documentation Project’s HOWTO collection, this collection is an effort +to foster documentation that’s more detailed than the Python Library +Reference. + +Currently, the HOWTOs are: + +* Menu: + +* Porting Python 2 Code to Python 3:: +* Porting Extension Modules to Python 3:: +* Curses Programming with Python:: +* Descriptor HowTo Guide:: +* Functional Programming HOWTO:: +* Logging HOWTO:: +* Logging Cookbook:: +* Regular Expression HOWTO:: +* Socket Programming HOWTO:: +* Sorting HOW TO:: +* Unicode HOWTO:: +* HOWTO Fetch Internet Resources Using The urllib Package:: +* Argparse Tutorial:: +* An introduction to the ipaddress module:: +* Argument Clinic How-To:: +* Instrumenting CPython with DTrace and SystemTap:: + + +File: python.info, Node: Porting Python 2 Code to Python 3, Next: Porting Extension Modules to Python 3, Up: Python HOWTOs + +10.1 Porting Python 2 Code to Python 3 +====================================== + + +author: Brett Cannon + +Abstract +........ + +With Python 3 being the future of Python while Python 2 is still in +active use, it is good to have your project available for both major +releases of Python. This guide is meant to help you figure out how best +to support both Python 2 & 3 simultaneously. + +If you are looking to port an extension module instead of pure Python +code, please see *note Porting Extension Modules to Python 3: c77. + +If you would like to read one core Python developer’s take on why Python +3 came into existence, you can read Nick Coghlan’s Python 3 Q & A(1) or +Brett Cannon’s Why Python 3 exists(2). + +For help with porting, you can email the python-porting(3) mailing list +with questions. + +* Menu: + +* The Short Explanation:: +* Details:: + + ---------- Footnotes ---------- + + (1) +https://ncoghlan-devs-python-notes.readthedocs.io/en/latest/python3/questions_and_answers.html + + (2) https://snarky.ca/why-python-3-exists + + (3) https://mail.python.org/mailman/listinfo/python-porting + + +File: python.info, Node: The Short Explanation, Next: Details, Up: Porting Python 2 Code to Python 3 + +10.1.1 The Short Explanation +---------------------------- + +To make your project be single-source Python 2/3 compatible, the basic +steps are: + + 1. Only worry about supporting Python 2.7 + + 2. Make sure you have good test coverage (coverage.py(1) can help; + ‘pip install coverage’) + + 3. Learn the differences between Python 2 & 3 + + 4. Use Futurize(2) (or Modernize(3)) to update your code (e.g. ‘pip + install future’) + + 5. Use Pylint(4) to help make sure you don’t regress on your Python 3 + support (‘pip install pylint’) + + 6. Use caniusepython3(5) to find out which of your dependencies are + blocking your use of Python 3 (‘pip install caniusepython3’) + + 7. Once your dependencies are no longer blocking you, use continuous + integration to make sure you stay compatible with Python 2 & 3 + (tox(6) can help test against multiple versions of Python; ‘pip + install tox’) + + 8. Consider using optional static type checking to make sure your type + usage works in both Python 2 & 3 (e.g. use mypy(7) to check your + typing under both Python 2 & Python 3). + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/coverage + + (2) http://python-future.org/automatic_conversion.html + + (3) https://python-modernize.readthedocs.io/ + + (4) https://pypi.org/project/pylint + + (5) https://pypi.org/project/caniusepython3 + + (6) https://pypi.org/project/tox + + (7) http://mypy-lang.org/ + + +File: python.info, Node: Details, Prev: The Short Explanation, Up: Porting Python 2 Code to Python 3 + +10.1.2 Details +-------------- + +A key point about supporting Python 2 & 3 simultaneously is that you can +start `today'! Even if your dependencies are not supporting Python 3 +yet that does not mean you can’t modernize your code `now' to support +Python 3. Most changes required to support Python 3 lead to cleaner +code using newer practices even in Python 2 code. + +Another key point is that modernizing your Python 2 code to also support +Python 3 is largely automated for you. While you might have to make +some API decisions thanks to Python 3 clarifying text data versus binary +data, the lower-level work is now mostly done for you and thus can at +least benefit from the automated changes immediately. + +Keep those key points in mind while you read on about the details of +porting your code to support Python 2 & 3 simultaneously. + +* Menu: + +* Drop support for Python 2.6 and older: Drop support for Python 2 6 and older. +* Make sure you specify the proper version support in your setup.py file: Make sure you specify the proper version support in your setup py file. +* Have good test coverage:: +* Learn the differences between Python 2 & 3:: +* Update your code:: +* Prevent compatibility regressions:: +* Check which dependencies block your transition:: +* Update your setup.py file to denote Python 3 compatibility: Update your setup py file to denote Python 3 compatibility. +* Use continuous integration to stay compatible:: +* Consider using optional static type checking:: + + +File: python.info, Node: Drop support for Python 2 6 and older, Next: Make sure you specify the proper version support in your setup py file, Up: Details + +10.1.2.1 Drop support for Python 2.6 and older +.............................................. + +While you can make Python 2.5 work with Python 3, it is `much' easier if +you only have to work with Python 2.7. If dropping Python 2.5 is not an +option then the six(1) project can help you support Python 2.5 & 3 +simultaneously (‘pip install six’). Do realize, though, that nearly all +the projects listed in this HOWTO will not be available to you. + +If you are able to skip Python 2.5 and older, then the required changes +to your code should continue to look and feel like idiomatic Python +code. At worst you will have to use a function instead of a method in +some instances or have to import a function instead of using a built-in +one, but otherwise the overall transformation should not feel foreign to +you. + +But you should aim for only supporting Python 2.7. Python 2.6 is no +longer freely supported and thus is not receiving bugfixes. This means +`you' will have to work around any issues you come across with Python +2.6. There are also some tools mentioned in this HOWTO which do not +support Python 2.6 (e.g., Pylint(2)), and this will become more +commonplace as time goes on. It will simply be easier for you if you +only support the versions of Python that you have to support. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/six + + (2) https://pypi.org/project/pylint + + +File: python.info, Node: Make sure you specify the proper version support in your setup py file, Next: Have good test coverage, Prev: Drop support for Python 2 6 and older, Up: Details + +10.1.2.2 Make sure you specify the proper version support in your ‘setup.py’ file +................................................................................. + +In your ‘setup.py’ file you should have the proper trove classifier(1) +specifying what versions of Python you support. As your project does +not support Python 3 yet you should at least have ‘Programming Language +:: Python :: 2 :: Only’ specified. Ideally you should also specify each +major/minor version of Python that you do support, e.g. ‘Programming +Language :: Python :: 2.7’. + + ---------- Footnotes ---------- + + (1) https://pypi.org/classifiers + + +File: python.info, Node: Have good test coverage, Next: Learn the differences between Python 2 & 3, Prev: Make sure you specify the proper version support in your setup py file, Up: Details + +10.1.2.3 Have good test coverage +................................ + +Once you have your code supporting the oldest version of Python 2 you +want it to, you will want to make sure your test suite has good +coverage. A good rule of thumb is that if you want to be confident +enough in your test suite that any failures that appear after having +tools rewrite your code are actual bugs in the tools and not in your +code. If you want a number to aim for, try to get over 80% coverage +(and don’t feel bad if you find it hard to get better than 90% +coverage). If you don’t already have a tool to measure test coverage +then coverage.py(1) is recommended. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/coverage + + +File: python.info, Node: Learn the differences between Python 2 & 3, Next: Update your code, Prev: Have good test coverage, Up: Details + +10.1.2.4 Learn the differences between Python 2 & 3 +................................................... + +Once you have your code well-tested you are ready to begin porting your +code to Python 3! But to fully understand how your code is going to +change and what you want to look out for while you code, you will want +to learn what changes Python 3 makes in terms of Python 2. Typically +the two best ways of doing that is reading the *note “What’s New”: 149. +doc for each release of Python 3 and the Porting to Python 3(1) book +(which is free online). There is also a handy cheat sheet(2) from the +Python-Future project. + + ---------- Footnotes ---------- + + (1) http://python3porting.com/ + + (2) http://python-future.org/compatible_idioms.html + + +File: python.info, Node: Update your code, Next: Prevent compatibility regressions, Prev: Learn the differences between Python 2 & 3, Up: Details + +10.1.2.5 Update your code +......................... + +Once you feel like you know what is different in Python 3 compared to +Python 2, it’s time to update your code! You have a choice between two +tools in porting your code automatically: Futurize(1) and Modernize(2). +Which tool you choose will depend on how much like Python 3 you want +your code to be. Futurize(3) does its best to make Python 3 idioms and +practices exist in Python 2, e.g. backporting the ‘bytes’ type from +Python 3 so that you have semantic parity between the major versions of +Python. Modernize(4), on the other hand, is more conservative and +targets a Python 2/3 subset of Python, directly relying on six(5) to +help provide compatibility. As Python 3 is the future, it might be best +to consider Futurize to begin adjusting to any new practices that Python +3 introduces which you are not accustomed to yet. + +Regardless of which tool you choose, they will update your code to run +under Python 3 while staying compatible with the version of Python 2 you +started with. Depending on how conservative you want to be, you may +want to run the tool over your test suite first and visually inspect the +diff to make sure the transformation is accurate. After you have +transformed your test suite and verified that all the tests still pass +as expected, then you can transform your application code knowing that +any tests which fail is a translation failure. + +Unfortunately the tools can’t automate everything to make your code work +under Python 3 and so there are a handful of things you will need to +update manually to get full Python 3 support (which of these steps are +necessary vary between the tools). Read the documentation for the tool +you choose to use to see what it fixes by default and what it can do +optionally to know what will (not) be fixed for you and what you may +have to fix on your own (e.g. using ‘io.open()’ over the built-in +‘open()’ function is off by default in Modernize). Luckily, though, +there are only a couple of things to watch out for which can be +considered large issues that may be hard to debug if not watched for. + +* Menu: + +* Division:: +* Text versus binary data:: +* Use feature detection instead of version detection:: + + ---------- Footnotes ---------- + + (1) http://python-future.org/automatic_conversion.html + + (2) https://python-modernize.readthedocs.io/ + + (3) http://python-future.org/automatic_conversion.html + + (4) https://python-modernize.readthedocs.io/ + + (5) https://pypi.org/project/six + + +File: python.info, Node: Division, Next: Text versus binary data, Up: Update your code + +10.1.2.6 Division +................. + +In Python 3, ‘5 / 2 == 2.5’ and not ‘2’; all division between ‘int’ +values result in a ‘float’. This change has actually been planned since +Python 2.2 which was released in 2002. Since then users have been +encouraged to add ‘from __future__ import division’ to any and all files +which use the ‘/’ and ‘//’ operators or to be running the interpreter +with the ‘-Q’ flag. If you have not been doing this then you will need +to go through your code and do two things: + + 1. Add ‘from __future__ import division’ to your files + + 2. Update any division operator as necessary to either use ‘//’ to use + floor division or continue using ‘/’ and expect a float + +The reason that ‘/’ isn’t simply translated to ‘//’ automatically is +that if an object defines a ‘__truediv__’ method but not ‘__floordiv__’ +then your code would begin to fail (e.g. a user-defined class that uses +‘/’ to signify some operation but not ‘//’ for the same thing or at +all). + + +File: python.info, Node: Text versus binary data, Next: Use feature detection instead of version detection, Prev: Division, Up: Update your code + +10.1.2.7 Text versus binary data +................................ + +In Python 2 you could use the ‘str’ type for both text and binary data. +Unfortunately this confluence of two different concepts could lead to +brittle code which sometimes worked for either kind of data, sometimes +not. It also could lead to confusing APIs if people didn’t explicitly +state that something that accepted ‘str’ accepted either text or binary +data instead of one specific type. This complicated the situation +especially for anyone supporting multiple languages as APIs wouldn’t +bother explicitly supporting ‘unicode’ when they claimed text data +support. + +To make the distinction between text and binary data clearer and more +pronounced, Python 3 did what most languages created in the age of the +internet have done and made text and binary data distinct types that +cannot blindly be mixed together (Python predates widespread access to +the internet). For any code that deals only with text or only binary +data, this separation doesn’t pose an issue. But for code that has to +deal with both, it does mean you might have to now care about when you +are using text compared to binary data, which is why this cannot be +entirely automated. + +To start, you will need to decide which APIs take text and which take +binary (it is `highly' recommended you don’t design APIs that can take +both due to the difficulty of keeping the code working; as stated +earlier it is difficult to do well). In Python 2 this means making sure +the APIs that take text can work with ‘unicode’ and those that work with +binary data work with the ‘bytes’ type from Python 3 (which is a subset +of ‘str’ in Python 2 and acts as an alias for ‘bytes’ type in Python 2). +Usually the biggest issue is realizing which methods exist on which +types in Python 2 & 3 simultaneously (for text that’s ‘unicode’ in +Python 2 and ‘str’ in Python 3, for binary that’s ‘str’/‘bytes’ in +Python 2 and ‘bytes’ in Python 3). The following table lists the +`unique' methods of each data type across Python 2 & 3 (e.g., the +‘decode()’ method is usable on the equivalent binary data type in either +Python 2 or 3, but it can’t be used by the textual data type +consistently between Python 2 and 3 because ‘str’ in Python 3 doesn’t +have the method). Do note that as of Python 3.5 the ‘__mod__’ method +was added to the bytes type. + +`Text data' `Binary data' + + + decode + + +encode + +format + +isdecimal + +isnumeric + +Making the distinction easier to handle can be accomplished by encoding +and decoding between binary data and text at the edge of your code. +This means that when you receive text in binary data, you should +immediately decode it. And if your code needs to send text as binary +data then encode it as late as possible. This allows your code to work +with only text internally and thus eliminates having to keep track of +what type of data you are working with. + +The next issue is making sure you know whether the string literals in +your code represent text or binary data. You should add a ‘b’ prefix to +any literal that presents binary data. For text you should add a ‘u’ +prefix to the text literal. (there is a *note __future__: 0. import to +force all unspecified literals to be Unicode, but usage has shown it +isn’t as effective as adding a ‘b’ or ‘u’ prefix to all literals +explicitly) + +As part of this dichotomy you also need to be careful about opening +files. Unless you have been working on Windows, there is a chance you +have not always bothered to add the ‘b’ mode when opening a binary file +(e.g., ‘rb’ for binary reading). Under Python 3, binary files and text +files are clearly distinct and mutually incompatible; see the *note io: +a1. module for details. Therefore, you `must' make a decision of +whether a file will be used for binary access (allowing binary data to +be read and/or written) or textual access (allowing text data to be read +and/or written). You should also use *note io.open(): 65a. for opening +files instead of the built-in *note open(): 4f0. function as the *note +io: a1. module is consistent from Python 2 to 3 while the built-in *note +open(): 4f0. function is not (in Python 3 it’s actually *note io.open(): +65a.). Do not bother with the outdated practice of using *note +codecs.open(): ffe. as that’s only necessary for keeping compatibility +with Python 2.5. + +The constructors of both ‘str’ and ‘bytes’ have different semantics for +the same arguments between Python 2 & 3. Passing an integer to ‘bytes’ +in Python 2 will give you the string representation of the integer: +‘bytes(3) == '3'’. But in Python 3, an integer argument to ‘bytes’ will +give you a bytes object as long as the integer specified, filled with +null bytes: ‘bytes(3) == b'\x00\x00\x00'’. A similar worry is necessary +when passing a bytes object to ‘str’. In Python 2 you just get the +bytes object back: ‘str(b'3') == b'3'’. But in Python 3 you get the +string representation of the bytes object: ‘str(b'3') == "b'3'"’. + +Finally, the indexing of binary data requires careful handling (slicing +does `not' require any special handling). In Python 2, ‘b'123'[1] == +b'2'’ while in Python 3 ‘b'123'[1] == 50’. Because binary data is +simply a collection of binary numbers, Python 3 returns the integer +value for the byte you index on. But in Python 2 because ‘bytes == +str’, indexing returns a one-item slice of bytes. The six(1) project +has a function named ‘six.indexbytes()’ which will return an integer +like in Python 3: ‘six.indexbytes(b'123', 1)’. + +To summarize: + + 1. Decide which of your APIs take text and which take binary data + + 2. Make sure that your code that works with text also works with + ‘unicode’ and code for binary data works with ‘bytes’ in Python 2 + (see the table above for what methods you cannot use for each type) + + 3. Mark all binary literals with a ‘b’ prefix, textual literals with a + ‘u’ prefix + + 4. Decode binary data to text as soon as possible, encode text as + binary data as late as possible + + 5. Open files using *note io.open(): 65a. and make sure to specify the + ‘b’ mode when appropriate + + 6. Be careful when indexing into binary data + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/six + + +File: python.info, Node: Use feature detection instead of version detection, Prev: Text versus binary data, Up: Update your code + +10.1.2.8 Use feature detection instead of version detection +........................................................... + +Inevitably you will have code that has to choose what to do based on +what version of Python is running. The best way to do this is with +feature detection of whether the version of Python you’re running under +supports what you need. If for some reason that doesn’t work then you +should make the version check be against Python 2 and not Python 3. To +help explain this, let’s look at an example. + +Let’s pretend that you need access to a feature of *note importlib: 9b. +that is available in Python’s standard library since Python 3.3 and +available for Python 2 through importlib2(1) on PyPI. You might be +tempted to write code to access e.g. the *note importlib.abc: 9c. +module by doing the following: + + import sys + + if sys.version_info[0] == 3: + from importlib import abc + else: + from importlib2 import abc + +The problem with this code is what happens when Python 4 comes out? It +would be better to treat Python 2 as the exceptional case instead of +Python 3 and assume that future Python versions will be more compatible +with Python 3 than Python 2: + + import sys + + if sys.version_info[0] > 2: + from importlib import abc + else: + from importlib2 import abc + +The best solution, though, is to do no version detection at all and +instead rely on feature detection. That avoids any potential issues of +getting the version detection wrong and helps keep you +future-compatible: + + try: + from importlib import abc + except ImportError: + from importlib2 import abc + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/importlib2 + + +File: python.info, Node: Prevent compatibility regressions, Next: Check which dependencies block your transition, Prev: Update your code, Up: Details + +10.1.2.9 Prevent compatibility regressions +.......................................... + +Once you have fully translated your code to be compatible with Python 3, +you will want to make sure your code doesn’t regress and stop working +under Python 3. This is especially true if you have a dependency which +is blocking you from actually running under Python 3 at the moment. + +To help with staying compatible, any new modules you create should have +at least the following block of code at the top of it: + + from __future__ import absolute_import + from __future__ import division + from __future__ import print_function + +You can also run Python 2 with the ‘-3’ flag to be warned about various +compatibility issues your code triggers during execution. If you turn +warnings into errors with ‘-Werror’ then you can make sure that you +don’t accidentally miss a warning. + +You can also use the Pylint(1) project and its ‘--py3k’ flag to lint +your code to receive warnings when your code begins to deviate from +Python 3 compatibility. This also prevents you from having to run +Modernize(2) or Futurize(3) over your code regularly to catch +compatibility regressions. This does require you only support Python +2.7 and Python 3.4 or newer as that is Pylint’s minimum Python version +support. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/pylint + + (2) https://python-modernize.readthedocs.io/ + + (3) http://python-future.org/automatic_conversion.html + + +File: python.info, Node: Check which dependencies block your transition, Next: Update your setup py file to denote Python 3 compatibility, Prev: Prevent compatibility regressions, Up: Details + +10.1.2.10 Check which dependencies block your transition +........................................................ + +`After' you have made your code compatible with Python 3 you should +begin to care about whether your dependencies have also been ported. +The caniusepython3(1) project was created to help you determine which +projects – directly or indirectly – are blocking you from supporting +Python 3. There is both a command-line tool as well as a web interface +at ‘https://caniusepython3.com’. + +The project also provides code which you can integrate into your test +suite so that you will have a failing test when you no longer have +dependencies blocking you from using Python 3. This allows you to avoid +having to manually check your dependencies and to be notified quickly +when you can start running on Python 3. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/caniusepython3 + + +File: python.info, Node: Update your setup py file to denote Python 3 compatibility, Next: Use continuous integration to stay compatible, Prev: Check which dependencies block your transition, Up: Details + +10.1.2.11 Update your ‘setup.py’ file to denote Python 3 compatibility +...................................................................... + +Once your code works under Python 3, you should update the classifiers +in your ‘setup.py’ to contain ‘Programming Language :: Python :: 3’ and +to not specify sole Python 2 support. This will tell anyone using your +code that you support Python 2 `and' 3. Ideally you will also want to +add classifiers for each major/minor version of Python you now support. + + +File: python.info, Node: Use continuous integration to stay compatible, Next: Consider using optional static type checking, Prev: Update your setup py file to denote Python 3 compatibility, Up: Details + +10.1.2.12 Use continuous integration to stay compatible +....................................................... + +Once you are able to fully run under Python 3 you will want to make sure +your code always works under both Python 2 & 3. Probably the best tool +for running your tests under multiple Python interpreters is tox(1). +You can then integrate tox with your continuous integration system so +that you never accidentally break Python 2 or 3 support. + +You may also want to use the ‘-bb’ flag with the Python 3 interpreter to +trigger an exception when you are comparing bytes to strings or bytes to +an int (the latter is available starting in Python 3.5). By default +type-differing comparisons simply return ‘False’, but if you made a +mistake in your separation of text/binary data handling or indexing on +bytes you wouldn’t easily find the mistake. This flag will raise an +exception when these kinds of comparisons occur, making the mistake much +easier to track down. + +And that’s mostly it! At this point your code base is compatible with +both Python 2 and 3 simultaneously. Your testing will also be set up so +that you don’t accidentally break Python 2 or 3 compatibility regardless +of which version you typically run your tests under while developing. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/tox + + +File: python.info, Node: Consider using optional static type checking, Prev: Use continuous integration to stay compatible, Up: Details + +10.1.2.13 Consider using optional static type checking +...................................................... + +Another way to help port your code is to use a static type checker like +mypy(1) or pytype(2) on your code. These tools can be used to analyze +your code as if it’s being run under Python 2, then you can run the tool +a second time as if your code is running under Python 3. By running a +static type checker twice like this you can discover if you’re e.g. +misusing binary data type in one version of Python compared to another. +If you add optional type hints to your code you can also explicitly +state whether your APIs use textual or binary data, helping to make sure +everything functions as expected in both versions of Python. + + ---------- Footnotes ---------- + + (1) http://mypy-lang.org/ + + (2) https://github.com/google/pytype + + +File: python.info, Node: Porting Extension Modules to Python 3, Next: Curses Programming with Python, Prev: Porting Python 2 Code to Python 3, Up: Python HOWTOs + +10.2 Porting Extension Modules to Python 3 +========================================== + +We recommend the following resources for porting extension modules to +Python 3: + + * The Migrating C extensions(1) chapter from `Supporting Python 3: An + in-depth guide', a book on moving from Python 2 to Python 3 in + general, guides the reader through porting an extension module. + + * The Porting guide(2) from the `py3c' project provides opinionated + suggestions with supporting code. + + * The Cython(3) and CFFI(4) libraries offer abstractions over + Python’s C API. Extensions generally need to be re-written to use + one of them, but the library then handles differences between + various Python versions and implementations. + + ---------- Footnotes ---------- + + (1) http://python3porting.com/cextensions.html + + (2) https://py3c.readthedocs.io/en/latest/guide.html + + (3) http://cython.org/ + + (4) https://cffi.readthedocs.io/en/latest/ + + +File: python.info, Node: Curses Programming with Python, Next: Descriptor HowTo Guide, Prev: Porting Extension Modules to Python 3, Up: Python HOWTOs + +10.3 Curses Programming with Python +=================================== + + +Author: A.M. Kuchling, Eric S. Raymond + + +Release: 2.04 + +Abstract +........ + +This document describes how to use the *note curses: 2c. extension +module to control text-mode displays. + +* Menu: + +* What is curses?:: +* Starting and ending a curses application:: +* Windows and Pads:: +* Displaying Text:: +* User Input:: +* For More Information:: + + +File: python.info, Node: What is curses?, Next: Starting and ending a curses application, Up: Curses Programming with Python + +10.3.1 What is curses? +---------------------- + +The curses library supplies a terminal-independent screen-painting and +keyboard-handling facility for text-based terminals; such terminals +include VT100s, the Linux console, and the simulated terminal provided +by various programs. Display terminals support various control codes to +perform common operations such as moving the cursor, scrolling the +screen, and erasing areas. Different terminals use widely differing +codes, and often have their own minor quirks. + +In a world of graphical displays, one might ask “why bother”? It’s true +that character-cell display terminals are an obsolete technology, but +there are niches in which being able to do fancy things with them are +still valuable. One niche is on small-footprint or embedded Unixes that +don’t run an X server. Another is tools such as OS installers and +kernel configurators that may have to run before any graphical support +is available. + +The curses library provides fairly basic functionality, providing the +programmer with an abstraction of a display containing multiple +non-overlapping windows of text. The contents of a window can be +changed in various ways—adding text, erasing it, changing its +appearance—and the curses library will figure out what control codes +need to be sent to the terminal to produce the right output. curses +doesn’t provide many user-interface concepts such as buttons, +checkboxes, or dialogs; if you need such features, consider a user +interface library such as Urwid(1). + +The curses library was originally written for BSD Unix; the later System +V versions of Unix from AT&T added many enhancements and new functions. +BSD curses is no longer maintained, having been replaced by ncurses, +which is an open-source implementation of the AT&T interface. If you’re +using an open-source Unix such as Linux or FreeBSD, your system almost +certainly uses ncurses. Since most current commercial Unix versions are +based on System V code, all the functions described here will probably +be available. The older versions of curses carried by some proprietary +Unixes may not support everything, though. + +The Windows version of Python doesn’t include the *note curses: 2c. +module. A ported version called UniCurses(2) is available. You could +also try the Console module(3) written by Fredrik Lundh, which doesn’t +use the same API as curses but provides cursor-addressable text output +and full support for mouse and keyboard input. + +* Menu: + +* The Python curses module:: + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/urwid/ + + (2) https://pypi.org/project/UniCurses + + (3) http://effbot.org/zone/console-index.htm + + +File: python.info, Node: The Python curses module, Up: What is curses? + +10.3.1.1 The Python curses module +................................. + +The Python module is a fairly simple wrapper over the C functions +provided by curses; if you’re already familiar with curses programming +in C, it’s really easy to transfer that knowledge to Python. The +biggest difference is that the Python interface makes things simpler by +merging different C functions such as ‘addstr()’, ‘mvaddstr()’, and +‘mvwaddstr()’ into a single *note addstr(): 1e42. method. You’ll see +this covered in more detail later. + +This HOWTO is an introduction to writing text-mode programs with curses +and Python. It doesn’t attempt to be a complete guide to the curses +API; for that, see the Python library guide’s section on ncurses, and +the C manual pages for ncurses. It will, however, give you the basic +ideas. + + +File: python.info, Node: Starting and ending a curses application, Next: Windows and Pads, Prev: What is curses?, Up: Curses Programming with Python + +10.3.2 Starting and ending a curses application +----------------------------------------------- + +Before doing anything, curses must be initialized. This is done by +calling the *note initscr(): 1e48. function, which will determine the +terminal type, send any required setup codes to the terminal, and create +various internal data structures. If successful, ‘initscr()’ returns a +window object representing the entire screen; this is usually called +‘stdscr’ after the name of the corresponding C variable. + + import curses + stdscr = curses.initscr() + +Usually curses applications turn off automatic echoing of keys to the +screen, in order to be able to read keys and only display them under +certain circumstances. This requires calling the *note noecho(): 1e67. +function. + + curses.noecho() + +Applications will also commonly need to react to keys instantly, without +requiring the Enter key to be pressed; this is called cbreak mode, as +opposed to the usual buffered input mode. + + curses.cbreak() + +Terminals usually return special keys, such as the cursor keys or +navigation keys such as Page Up and Home, as a multibyte escape +sequence. While you could write your application to expect such +sequences and process them accordingly, curses can do it for you, +returning a special value such as ‘curses.KEY_LEFT’. To get curses to +do the job, you’ll have to enable keypad mode. + + stdscr.keypad(True) + +Terminating a curses application is much easier than starting one. +You’ll need to call: + + curses.nocbreak() + stdscr.keypad(False) + curses.echo() + +to reverse the curses-friendly terminal settings. Then call the *note +endwin(): 1e45. function to restore the terminal to its original +operating mode. + + curses.endwin() + +A common problem when debugging a curses application is to get your +terminal messed up when the application dies without restoring the +terminal to its previous state. In Python this commonly happens when +your code is buggy and raises an uncaught exception. Keys are no longer +echoed to the screen when you type them, for example, which makes using +the shell difficult. + +In Python you can avoid these complications and make debugging much +easier by importing the *note curses.wrapper(): 2a1. function and using +it like this: + + from curses import wrapper + + def main(stdscr): + # Clear screen + stdscr.clear() + + # This raises ZeroDivisionError when i == 10. + for i in range(0, 11): + v = i-10 + stdscr.addstr(i, 0, '10 divided by {} is {}'.format(v, 10/v)) + + stdscr.refresh() + stdscr.getkey() + + wrapper(main) + +The *note wrapper(): 2a1. function takes a callable object and does the +initializations described above, also initializing colors if color +support is present. ‘wrapper()’ then runs your provided callable. Once +the callable returns, ‘wrapper()’ will restore the original state of the +terminal. The callable is called inside a *note try: d72.…*note except: +b3e. that catches exceptions, restores the state of the terminal, and +then re-raises the exception. Therefore your terminal won’t be left in +a funny state on exception and you’ll be able to read the exception’s +message and traceback. + + +File: python.info, Node: Windows and Pads, Next: Displaying Text, Prev: Starting and ending a curses application, Up: Curses Programming with Python + +10.3.3 Windows and Pads +----------------------- + +Windows are the basic abstraction in curses. A window object represents +a rectangular area of the screen, and supports methods to display text, +erase it, allow the user to input strings, and so forth. + +The ‘stdscr’ object returned by the *note initscr(): 1e48. function is a +window object that covers the entire screen. Many programs may need +only this single window, but you might wish to divide the screen into +smaller windows, in order to redraw or clear them separately. The *note +newwin(): 1e65. function creates a new window of a given size, returning +the new window object. + + begin_x = 20; begin_y = 7 + height = 5; width = 40 + win = curses.newwin(height, width, begin_y, begin_x) + +Note that the coordinate system used in curses is unusual. Coordinates +are always passed in the order `y,x', and the top-left corner of a +window is coordinate (0,0). This breaks the normal convention for +handling coordinates where the `x' coordinate comes first. This is an +unfortunate difference from most other computer applications, but it’s +been part of curses since it was first written, and it’s too late to +change things now. + +Your application can determine the size of the screen by using the +‘curses.LINES’ and ‘curses.COLS’ variables to obtain the `y' and `x' +sizes. Legal coordinates will then extend from ‘(0,0)’ to +‘(curses.LINES - 1, curses.COLS - 1)’. + +When you call a method to display or erase text, the effect doesn’t +immediately show up on the display. Instead you must call the *note +refresh(): 1e43. method of window objects to update the screen. + +This is because curses was originally written with slow 300-baud +terminal connections in mind; with these terminals, minimizing the time +required to redraw the screen was very important. Instead curses +accumulates changes to the screen and displays them in the most +efficient manner when you call ‘refresh()’. For example, if your +program displays some text in a window and then clears the window, +there’s no need to send the original text because they’re never visible. + +In practice, explicitly telling curses to redraw a window doesn’t really +complicate programming with curses much. Most programs go into a flurry +of activity, and then pause waiting for a keypress or some other action +on the part of the user. All you have to do is to be sure that the +screen has been redrawn before pausing to wait for user input, by first +calling ‘stdscr.refresh()’ or the ‘refresh()’ method of some other +relevant window. + +A pad is a special case of a window; it can be larger than the actual +display screen, and only a portion of the pad displayed at a time. +Creating a pad requires the pad’s height and width, while refreshing a +pad requires giving the coordinates of the on-screen area where a +subsection of the pad will be displayed. + + pad = curses.newpad(100, 100) + # These loops fill the pad with letters; addch() is + # explained in the next section + for y in range(0, 99): + for x in range(0, 99): + pad.addch(y,x, ord('a') + (x*x+y*y) % 26) + + # Displays a section of the pad in the middle of the screen. + # (0,0) : coordinate of upper-left corner of pad area to display. + # (5,5) : coordinate of upper-left corner of window area to be filled + # with pad content. + # (20, 75) : coordinate of lower-right corner of window area to be + # : filled with pad content. + pad.refresh( 0,0, 5,5, 20,75) + +The ‘refresh()’ call displays a section of the pad in the rectangle +extending from coordinate (5,5) to coordinate (20,75) on the screen; the +upper left corner of the displayed section is coordinate (0,0) on the +pad. Beyond that difference, pads are exactly like ordinary windows and +support the same methods. + +If you have multiple windows and pads on screen there is a more +efficient way to update the screen and prevent annoying screen flicker +as each part of the screen gets updated. ‘refresh()’ actually does two +things: + + 1. Calls the *note noutrefresh(): 1e41. method of each window to + update an underlying data structure representing the desired state + of the screen. + + 2. Calls the function *note doupdate(): 1e40. function to change the + physical screen to match the desired state recorded in the data + structure. + +Instead you can call ‘noutrefresh()’ on a number of windows to update +the data structure, and then call ‘doupdate()’ to update the screen. + + +File: python.info, Node: Displaying Text, Next: User Input, Prev: Windows and Pads, Up: Curses Programming with Python + +10.3.4 Displaying Text +---------------------- + +From a C programmer’s point of view, curses may sometimes look like a +twisty maze of functions, all subtly different. For example, ‘addstr()’ +displays a string at the current cursor location in the ‘stdscr’ window, +while ‘mvaddstr()’ moves to a given y,x coordinate first before +displaying the string. ‘waddstr()’ is just like ‘addstr()’, but allows +specifying a window to use instead of using ‘stdscr’ by default. +‘mvwaddstr()’ allows specifying both a window and a coordinate. + +Fortunately the Python interface hides all these details. ‘stdscr’ is a +window object like any other, and methods such as *note addstr(): 1e42. +accept multiple argument forms. Usually there are four different forms. + +Form Description + +------------------------------------------------------------------------------------------ + +`str' or `ch' Display the string `str' or character `ch' at the + current position + + +`str' or `ch', `attr' Display the string `str' or character `ch', using + attribute `attr' at the current position + + +`y', `x', `str' or `ch' Move to position `y,x' within the window, and + display `str' or `ch' + + +`y', `x', `str' or `ch', `attr' Move to position `y,x' within the window, and + display `str' or `ch', using attribute `attr' + + +Attributes allow displaying text in highlighted forms such as boldface, +underline, reverse code, or in color. They’ll be explained in more +detail in the next subsection. + +The *note addstr(): 1e42. method takes a Python string or bytestring as +the value to be displayed. The contents of bytestrings are sent to the +terminal as-is. Strings are encoded to bytes using the value of the +window’s ‘encoding’ attribute; this defaults to the default system +encoding as returned by *note locale.getpreferredencoding(): 3a5. + +The *note addch(): 1e80. methods take a character, which can be either a +string of length 1, a bytestring of length 1, or an integer. + +Constants are provided for extension characters; these constants are +integers greater than 255. For example, ‘ACS_PLMINUS’ is a +/- symbol, +and ‘ACS_ULCORNER’ is the upper left corner of a box (handy for drawing +borders). You can also use the appropriate Unicode character. + +Windows remember where the cursor was left after the last operation, so +if you leave out the `y,x' coordinates, the string or character will be +displayed wherever the last operation left off. You can also move the +cursor with the ‘move(y,x)’ method. Because some terminals always +display a flashing cursor, you may want to ensure that the cursor is +positioned in some location where it won’t be distracting; it can be +confusing to have the cursor blinking at some apparently random +location. + +If your application doesn’t need a blinking cursor at all, you can call +‘curs_set(False)’ to make it invisible. For compatibility with older +curses versions, there’s a ‘leaveok(bool)’ function that’s a synonym for +*note curs_set(): 1e3a. When `bool' is true, the curses library will +attempt to suppress the flashing cursor, and you won’t need to worry +about leaving it in odd locations. + +* Menu: + +* Attributes and Color:: + + +File: python.info, Node: Attributes and Color, Up: Displaying Text + +10.3.4.1 Attributes and Color +............................. + +Characters can be displayed in different ways. Status lines in a +text-based application are commonly shown in reverse video, or a text +viewer may need to highlight certain words. curses supports this by +allowing you to specify an attribute for each cell on the screen. + +An attribute is an integer, each bit representing a different attribute. +You can try to display text with multiple attribute bits set, but curses +doesn’t guarantee that all the possible combinations are available, or +that they’re all visually distinct. That depends on the ability of the +terminal being used, so it’s safest to stick to the most commonly +available attributes, listed here. + +Attribute Description + +---------------------------------------------------------------------- + +‘A_BLINK’ Blinking text + + +‘A_BOLD’ Extra bright or bold text + + +‘A_DIM’ Half bright text + + +‘A_REVERSE’ Reverse-video text + + +‘A_STANDOUT’ The best highlighting mode available + + +‘A_UNDERLINE’ Underlined text + + +So, to display a reverse-video status line on the top line of the +screen, you could code: + + stdscr.addstr(0, 0, "Current mode: Typing mode", + curses.A_REVERSE) + stdscr.refresh() + +The curses library also supports color on those terminals that provide +it. The most common such terminal is probably the Linux console, +followed by color xterms. + +To use color, you must call the *note start_color(): 1e73. function soon +after calling *note initscr(): 1e48, to initialize the default color set +(the *note curses.wrapper(): 2a1. function does this automatically). +Once that’s done, the *note has_colors(): 1e50. function returns TRUE if +the terminal in use can actually display color. (Note: curses uses the +American spelling ‘color’, instead of the Canadian/British spelling +‘colour’. If you’re used to the British spelling, you’ll have to resign +yourself to misspelling it for the sake of these functions.) + +The curses library maintains a finite number of color pairs, containing +a foreground (or text) color and a background color. You can get the +attribute value corresponding to a color pair with the *note +color_pair(): 1e38. function; this can be bitwise-OR’ed with other +attributes such as ‘A_REVERSE’, but again, such combinations are not +guaranteed to work on all terminals. + +An example, which displays a line of text using color pair 1: + + stdscr.addstr("Pretty text", curses.color_pair(1)) + stdscr.refresh() + +As I said before, a color pair consists of a foreground and background +color. The ‘init_pair(n, f, b)’ function changes the definition of +color pair `n', to foreground color f and background color b. Color +pair 0 is hard-wired to white on black, and cannot be changed. + +Colors are numbered, and ‘start_color()’ initializes 8 basic colors when +it activates color mode. They are: 0:black, 1:red, 2:green, 3:yellow, +4:blue, 5:magenta, 6:cyan, and 7:white. The *note curses: 2c. module +defines named constants for each of these colors: ‘curses.COLOR_BLACK’, +‘curses.COLOR_RED’, and so forth. + +Let’s put all this together. To change color 1 to red text on a white +background, you would call: + + curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE) + +When you change a color pair, any text already displayed using that +color pair will change to the new colors. You can also display new text +in this color with: + + stdscr.addstr(0,0, "RED ALERT!", curses.color_pair(1)) + +Very fancy terminals can change the definitions of the actual colors to +a given RGB value. This lets you change color 1, which is usually red, +to purple or blue or any other color you like. Unfortunately, the Linux +console doesn’t support this, so I’m unable to try it out, and can’t +provide any examples. You can check if your terminal can do this by +calling *note can_change_color(): 1e34, which returns ‘True’ if the +capability is there. If you’re lucky enough to have such a talented +terminal, consult your system’s man pages for more information. + + +File: python.info, Node: User Input, Next: For More Information, Prev: Displaying Text, Up: Curses Programming with Python + +10.3.5 User Input +----------------- + +The C curses library offers only very simple input mechanisms. Python’s +*note curses: 2c. module adds a basic text-input widget. (Other +libraries such as Urwid(1) have more extensive collections of widgets.) + +There are two methods for getting input from a window: + + * *note getch(): 1e4c. refreshes the screen and then waits for the + user to hit a key, displaying the key if *note echo(): 1e44. has + been called earlier. You can optionally specify a coordinate to + which the cursor should be moved before pausing. + + * *note getkey(): 1e99. does the same thing but converts the integer + to a string. Individual characters are returned as 1-character + strings, and special keys such as function keys return longer + strings containing a key name such as ‘KEY_UP’ or ‘^G’. + +It’s possible to not wait for the user using the *note nodelay(): 1eaf. +window method. After ‘nodelay(True)’, ‘getch()’ and ‘getkey()’ for the +window become non-blocking. To signal that no input is ready, ‘getch()’ +returns ‘curses.ERR’ (a value of -1) and ‘getkey()’ raises an exception. +There’s also a *note halfdelay(): 1e54. function, which can be used to +(in effect) set a timer on each ‘getch()’; if no input becomes available +within a specified delay (measured in tenths of a second), curses raises +an exception. + +The ‘getch()’ method returns an integer; if it’s between 0 and 255, it +represents the ASCII code of the key pressed. Values greater than 255 +are special keys such as Page Up, Home, or the cursor keys. You can +compare the value returned to constants such as ‘curses.KEY_PPAGE’, +‘curses.KEY_HOME’, or ‘curses.KEY_LEFT’. The main loop of your program +may look something like this: + + while True: + c = stdscr.getch() + if c == ord('p'): + PrintDocument() + elif c == ord('q'): + break # Exit the while loop + elif c == curses.KEY_HOME: + x = y = 0 + +The *note curses.ascii: 2d. module supplies ASCII class membership +functions that take either integer or 1-character string arguments; +these may be useful in writing more readable tests for such loops. It +also supplies conversion functions that take either integer or +1-character-string arguments and return the same type. For example, +*note curses.ascii.ctrl(): 1ee3. returns the control character +corresponding to its argument. + +There’s also a method to retrieve an entire string, *note getstr(): +1e9c. It isn’t used very often, because its functionality is quite +limited; the only editing keys available are the backspace key and the +Enter key, which terminates the string. It can optionally be limited to +a fixed number of characters. + + curses.echo() # Enable echoing of characters + + # Get a 15-character string, with the cursor on the top line + s = stdscr.getstr(0,0, 15) + +The *note curses.textpad: 2f. module supplies a text box that supports +an Emacs-like set of keybindings. Various methods of the *note Textbox: +1ec9. class support editing with input validation and gathering the edit +results either with or without trailing spaces. Here’s an example: + + import curses + from curses.textpad import Textbox, rectangle + + def main(stdscr): + stdscr.addstr(0, 0, "Enter IM message: (hit Ctrl-G to send)") + + editwin = curses.newwin(5,30, 2,1) + rectangle(stdscr, 1,0, 1+5+1, 1+30+1) + stdscr.refresh() + + box = Textbox(editwin) + + # Let the user edit until Ctrl-G is struck. + box.edit() + + # Get resulting contents + message = box.gather() + +See the library documentation on *note curses.textpad: 2f. for more +details. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/urwid/ + + +File: python.info, Node: For More Information, Prev: User Input, Up: Curses Programming with Python + +10.3.6 For More Information +--------------------------- + +This HOWTO doesn’t cover some advanced topics, such as reading the +contents of the screen or capturing mouse events from an xterm instance, +but the Python library page for the *note curses: 2c. module is now +reasonably complete. You should browse it next. + +If you’re in doubt about the detailed behavior of the curses functions, +consult the manual pages for your curses implementation, whether it’s +ncurses or a proprietary Unix vendor’s. The manual pages will document +any quirks, and provide complete lists of all the functions, attributes, +and ‘ACS_*’ characters available to you. + +Because the curses API is so large, some functions aren’t supported in +the Python interface. Often this isn’t because they’re difficult to +implement, but because no one has needed them yet. Also, Python doesn’t +yet support the menu library associated with ncurses. Patches adding +support for these would be welcome; see the Python Developer’s Guide(1) +to learn more about submitting patches to Python. + + * Writing Programs with NCURSES(2): a lengthy tutorial for C + programmers. + + * The ncurses man page(3) + + * The ncurses FAQ(4) + + * "Use curses... don’t swear"(5): video of a PyCon 2013 talk on + controlling terminals using curses or Urwid. + + * "Console Applications with Urwid"(6): video of a PyCon CA 2012 talk + demonstrating some applications written using Urwid. + + ---------- Footnotes ---------- + + (1) https://devguide.python.org/ + + (2) http://invisible-island.net/ncurses/ncurses-intro.html + + (3) https://linux.die.net/man/3/ncurses + + (4) http://invisible-island.net/ncurses/ncurses.faq.html + + (5) https://www.youtube.com/watch?v=eN1eZtjLEnU + + (6) http://www.pyvideo.org/video/1568/console-applications-with-urwid + + +File: python.info, Node: Descriptor HowTo Guide, Next: Functional Programming HOWTO, Prev: Curses Programming with Python, Up: Python HOWTOs + +10.4 Descriptor HowTo Guide +=========================== + + +Author: Raymond Hettinger + + +Contact: <python at rcn dot com> + +* Menu: + +* Abstract:: +* Definition and Introduction:: +* Descriptor Protocol:: +* Invoking Descriptors: Invoking Descriptors<2>. +* Descriptor Example:: +* Properties:: +* Functions and Methods:: +* Static Methods and Class Methods:: + + +File: python.info, Node: Abstract, Next: Definition and Introduction, Up: Descriptor HowTo Guide + +10.4.1 Abstract +--------------- + +Defines descriptors, summarizes the protocol, and shows how descriptors +are called. Examines a custom descriptor and several built-in Python +descriptors including functions, properties, static methods, and class +methods. Shows how each works by giving a pure Python equivalent and a +sample application. + +Learning about descriptors not only provides access to a larger toolset, +it creates a deeper understanding of how Python works and an +appreciation for the elegance of its design. + + +File: python.info, Node: Definition and Introduction, Next: Descriptor Protocol, Prev: Abstract, Up: Descriptor HowTo Guide + +10.4.2 Definition and Introduction +---------------------------------- + +In general, a descriptor is an object attribute with “binding behavior”, +one whose attribute access has been overridden by methods in the +descriptor protocol. Those methods are *note __get__(): 10dd, *note +__set__(): 10e1, and *note __delete__(): 10e2. If any of those methods +are defined for an object, it is said to be a descriptor. + +The default behavior for attribute access is to get, set, or delete the +attribute from an object’s dictionary. For instance, ‘a.x’ has a lookup +chain starting with ‘a.__dict__['x']’, then ‘type(a).__dict__['x']’, and +continuing through the base classes of ‘type(a)’ excluding metaclasses. +If the looked-up value is an object defining one of the descriptor +methods, then Python may override the default behavior and invoke the +descriptor method instead. Where this occurs in the precedence chain +depends on which descriptor methods were defined. + +Descriptors are a powerful, general purpose protocol. They are the +mechanism behind properties, methods, static methods, class methods, and +*note super(): 4e2. They are used throughout Python itself to implement +the new style classes introduced in version 2.2. Descriptors simplify +the underlying C-code and offer a flexible set of new tools for everyday +Python programs. + + +File: python.info, Node: Descriptor Protocol, Next: Invoking Descriptors<2>, Prev: Definition and Introduction, Up: Descriptor HowTo Guide + +10.4.3 Descriptor Protocol +-------------------------- + +‘descr.__get__(self, obj, type=None) -> value’ + +‘descr.__set__(self, obj, value) -> None’ + +‘descr.__delete__(self, obj) -> None’ + +That is all there is to it. Define any of these methods and an object +is considered a descriptor and can override default behavior upon being +looked up as an attribute. + +If an object defines *note __set__(): 10e1. or *note __delete__(): 10e2, +it is considered a data descriptor. Descriptors that only define *note +__get__(): 10dd. are called non-data descriptors (they are typically +used for methods but other uses are possible). + +Data and non-data descriptors differ in how overrides are calculated +with respect to entries in an instance’s dictionary. If an instance’s +dictionary has an entry with the same name as a data descriptor, the +data descriptor takes precedence. If an instance’s dictionary has an +entry with the same name as a non-data descriptor, the dictionary entry +takes precedence. + +To make a read-only data descriptor, define both *note __get__(): 10dd. +and *note __set__(): 10e1. with the *note __set__(): 10e1. raising an +*note AttributeError: 39b. when called. Defining the *note __set__(): +10e1. method with an exception raising placeholder is enough to make it +a data descriptor. + + +File: python.info, Node: Invoking Descriptors<2>, Next: Descriptor Example, Prev: Descriptor Protocol, Up: Descriptor HowTo Guide + +10.4.4 Invoking Descriptors +--------------------------- + +A descriptor can be called directly by its method name. For example, +‘d.__get__(obj)’. + +Alternatively, it is more common for a descriptor to be invoked +automatically upon attribute access. For example, ‘obj.d’ looks up ‘d’ +in the dictionary of ‘obj’. If ‘d’ defines the method *note __get__(): +10dd, then ‘d.__get__(obj)’ is invoked according to the precedence rules +listed below. + +The details of invocation depend on whether ‘obj’ is an object or a +class. + +For objects, the machinery is in *note object.__getattribute__(): 449. +which transforms ‘b.x’ into ‘type(b).__dict__['x'].__get__(b, type(b))’. +The implementation works through a precedence chain that gives data +descriptors priority over instance variables, instance variables +priority over non-data descriptors, and assigns lowest priority to *note +__getattr__(): 31a. if provided. The full C implementation can be found +in *note PyObject_GenericGetAttr(): 3a02. in Objects/object.c(1). + +For classes, the machinery is in ‘type.__getattribute__()’ which +transforms ‘B.x’ into ‘B.__dict__['x'].__get__(None, B)’. In pure +Python, it looks like: + + def __getattribute__(self, key): + "Emulate type_getattro() in Objects/typeobject.c" + v = object.__getattribute__(self, key) + if hasattr(v, '__get__'): + return v.__get__(None, self) + return v + +The important points to remember are: + + * descriptors are invoked by the *note __getattribute__(): 449. + method + + * overriding *note __getattribute__(): 449. prevents automatic + descriptor calls + + * *note object.__getattribute__(): 449. and ‘type.__getattribute__()’ + make different calls to *note __get__(): 10dd. + + * data descriptors always override instance dictionaries. + + * non-data descriptors may be overridden by instance dictionaries. + +The object returned by ‘super()’ also has a custom *note +__getattribute__(): 449. method for invoking descriptors. The attribute +lookup ‘super(B, obj).m’ searches ‘obj.__class__.__mro__’ for the base +class ‘A’ immediately following ‘B’ and then returns +‘A.__dict__['m'].__get__(obj, B)’. If not a descriptor, ‘m’ is returned +unchanged. If not in the dictionary, ‘m’ reverts to a search using +*note object.__getattribute__(): 449. + +The implementation details are in ‘super_getattro()’ in +Objects/typeobject.c(2). and a pure Python equivalent can be found in +Guido’s Tutorial(3). + +The details above show that the mechanism for descriptors is embedded in +the *note __getattribute__(): 449. methods for *note object: 2b0, *note +type: 608, and *note super(): 4e2. Classes inherit this machinery when +they derive from *note object: 2b0. or if they have a meta-class +providing similar functionality. Likewise, classes can turn-off +descriptor invocation by overriding *note __getattribute__(): 449. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Objects/object.c + + (2) https://github.com/python/cpython/tree/3.8/Objects/typeobject.c + + (3) +https://www.python.org/download/releases/2.2.3/descrintro/#cooperation + + +File: python.info, Node: Descriptor Example, Next: Properties, Prev: Invoking Descriptors<2>, Up: Descriptor HowTo Guide + +10.4.5 Descriptor Example +------------------------- + +The following code creates a class whose objects are data descriptors +which print a message for each get or set. Overriding *note +__getattribute__(): 449. is alternate approach that could do this for +every attribute. However, this descriptor is useful for monitoring just +a few chosen attributes: + + class RevealAccess(object): + """A data descriptor that sets and returns values + normally and prints a message logging their access. + """ + + def __init__(self, initval=None, name='var'): + self.val = initval + self.name = name + + def __get__(self, obj, objtype): + print('Retrieving', self.name) + return self.val + + def __set__(self, obj, val): + print('Updating', self.name) + self.val = val + + >>> class MyClass(object): + ... x = RevealAccess(10, 'var "x"') + ... y = 5 + ... + >>> m = MyClass() + >>> m.x + Retrieving var "x" + 10 + >>> m.x = 20 + Updating var "x" + >>> m.x + Retrieving var "x" + 20 + >>> m.y + 5 + +The protocol is simple and offers exciting possibilities. Several use +cases are so common that they have been packaged into individual +function calls. Properties, bound methods, static methods, and class +methods are all based on the descriptor protocol. + + +File: python.info, Node: Properties, Next: Functions and Methods, Prev: Descriptor Example, Up: Descriptor HowTo Guide + +10.4.6 Properties +----------------- + +Calling *note property(): 1d7. is a succinct way of building a data +descriptor that triggers function calls upon access to an attribute. +Its signature is: + + property(fget=None, fset=None, fdel=None, doc=None) -> property attribute + +The documentation shows a typical use to define a managed attribute ‘x’: + + class C(object): + def getx(self): return self.__x + def setx(self, value): self.__x = value + def delx(self): del self.__x + x = property(getx, setx, delx, "I'm the 'x' property.") + +To see how *note property(): 1d7. is implemented in terms of the +descriptor protocol, here is a pure Python equivalent: + + class Property(object): + "Emulate PyProperty_Type() in Objects/descrobject.c" + + def __init__(self, fget=None, fset=None, fdel=None, doc=None): + self.fget = fget + self.fset = fset + self.fdel = fdel + if doc is None and fget is not None: + doc = fget.__doc__ + self.__doc__ = doc + + def __get__(self, obj, objtype=None): + if obj is None: + return self + if self.fget is None: + raise AttributeError("unreadable attribute") + return self.fget(obj) + + def __set__(self, obj, value): + if self.fset is None: + raise AttributeError("can't set attribute") + self.fset(obj, value) + + def __delete__(self, obj): + if self.fdel is None: + raise AttributeError("can't delete attribute") + self.fdel(obj) + + def getter(self, fget): + return type(self)(fget, self.fset, self.fdel, self.__doc__) + + def setter(self, fset): + return type(self)(self.fget, fset, self.fdel, self.__doc__) + + def deleter(self, fdel): + return type(self)(self.fget, self.fset, fdel, self.__doc__) + +The *note property(): 1d7. builtin helps whenever a user interface has +granted attribute access and then subsequent changes require the +intervention of a method. + +For instance, a spreadsheet class may grant access to a cell value +through ‘Cell('b10').value’. Subsequent improvements to the program +require the cell to be recalculated on every access; however, the +programmer does not want to affect existing client code accessing the +attribute directly. The solution is to wrap access to the value +attribute in a property data descriptor: + + class Cell(object): + . . . + def getvalue(self): + "Recalculate the cell before returning value" + self.recalc() + return self._value + value = property(getvalue) + + +File: python.info, Node: Functions and Methods, Next: Static Methods and Class Methods, Prev: Properties, Up: Descriptor HowTo Guide + +10.4.7 Functions and Methods +---------------------------- + +Python’s object oriented features are built upon a function based +environment. Using non-data descriptors, the two are merged seamlessly. + +Class dictionaries store methods as functions. In a class definition, +methods are written using *note def: dbe. or *note lambda: c2f, the +usual tools for creating functions. Methods only differ from regular +functions in that the first argument is reserved for the object +instance. By Python convention, the instance reference is called `self' +but may be called `this' or any other variable name. + +To support method calls, functions include the *note __get__(): 10dd. +method for binding methods during attribute access. This means that all +functions are non-data descriptors which return bound methods when they +are invoked from an object. In pure Python, it works like this: + + class Function(object): + . . . + def __get__(self, obj, objtype=None): + "Simulate func_descr_get() in Objects/funcobject.c" + if obj is None: + return self + return types.MethodType(self, obj) + +Running the interpreter shows how the function descriptor works in +practice: + + >>> class D(object): + ... def f(self, x): + ... return x + ... + >>> d = D() + + # Access through the class dictionary does not invoke __get__. + # It just returns the underlying function object. + >>> D.__dict__['f'] + <function D.f at 0x00C45070> + + # Dotted access from a class calls __get__() which just returns + # the underlying function unchanged. + >>> D.f + <function D.f at 0x00C45070> + + # The function has a __qualname__ attribute to support introspection + >>> D.f.__qualname__ + 'D.f' + + # Dotted access from an instance calls __get__() which returns the + # function wrapped in a bound method object + >>> d.f + <bound method D.f of <__main__.D object at 0x00B18C90>> + + # Internally, the bound method stores the underlying function and + # the bound instance. + >>> d.f.__func__ + <function D.f at 0x1012e5ae8> + >>> d.f.__self__ + <__main__.D object at 0x1012e1f98> + + +File: python.info, Node: Static Methods and Class Methods, Prev: Functions and Methods, Up: Descriptor HowTo Guide + +10.4.8 Static Methods and Class Methods +--------------------------------------- + +Non-data descriptors provide a simple mechanism for variations on the +usual patterns of binding functions into methods. + +To recap, functions have a *note __get__(): 10dd. method so that they +can be converted to a method when accessed as attributes. The non-data +descriptor transforms an ‘obj.f(*args)’ call into ‘f(obj, *args)’. +Calling ‘klass.f(*args)’ becomes ‘f(*args)’. + +This chart summarizes the binding and its two most useful variants: + + Transformation Called from an Object Called from a Class + + ------------------------------------------------------------------------ + + function f(obj, *args) f(*args) + + + staticmethod f(*args) f(*args) + + + classmethod f(type(obj), *args) f(klass, *args) + + +Static methods return the underlying function without changes. Calling +either ‘c.f’ or ‘C.f’ is the equivalent of a direct lookup into +‘object.__getattribute__(c, "f")’ or ‘object.__getattribute__(C, "f")’. +As a result, the function becomes identically accessible from either an +object or a class. + +Good candidates for static methods are methods that do not reference the +‘self’ variable. + +For instance, a statistics package may include a container class for +experimental data. The class provides normal methods for computing the +average, mean, median, and other descriptive statistics that depend on +the data. However, there may be useful functions which are conceptually +related but do not depend on the data. For instance, ‘erf(x)’ is handy +conversion routine that comes up in statistical work but does not +directly depend on a particular dataset. It can be called either from +an object or the class: ‘s.erf(1.5) --> .9332’ or ‘Sample.erf(1.5) --> +.9332’. + +Since staticmethods return the underlying function with no changes, the +example calls are unexciting: + + >>> class E(object): + ... def f(x): + ... print(x) + ... f = staticmethod(f) + ... + >>> E.f(3) + 3 + >>> E().f(3) + 3 + +Using the non-data descriptor protocol, a pure Python version of *note +staticmethod(): 1d9. would look like this: + + class StaticMethod(object): + "Emulate PyStaticMethod_Type() in Objects/funcobject.c" + + def __init__(self, f): + self.f = f + + def __get__(self, obj, objtype=None): + return self.f + +Unlike static methods, class methods prepend the class reference to the +argument list before calling the function. This format is the same for +whether the caller is an object or a class: + + >>> class E(object): + ... def f(klass, x): + ... return klass.__name__, x + ... f = classmethod(f) + ... + >>> print(E.f(3)) + ('E', 3) + >>> print(E().f(3)) + ('E', 3) + +This behavior is useful whenever the function only needs to have a class +reference and does not care about any underlying data. One use for +classmethods is to create alternate class constructors. In Python 2.3, +the classmethod *note dict.fromkeys(): 137f. creates a new dictionary +from a list of keys. The pure Python equivalent is: + + class Dict(object): + . . . + def fromkeys(klass, iterable, value=None): + "Emulate dict_fromkeys() in Objects/dictobject.c" + d = klass() + for key in iterable: + d[key] = value + return d + fromkeys = classmethod(fromkeys) + +Now a new dictionary of unique keys can be constructed like this: + + >>> Dict.fromkeys('abracadabra') + {'a': None, 'r': None, 'b': None, 'c': None, 'd': None} + +Using the non-data descriptor protocol, a pure Python version of *note +classmethod(): 1d8. would look like this: + + class ClassMethod(object): + "Emulate PyClassMethod_Type() in Objects/funcobject.c" + + def __init__(self, f): + self.f = f + + def __get__(self, obj, klass=None): + if klass is None: + klass = type(obj) + def newfunc(*args): + return self.f(klass, *args) + return newfunc + + +File: python.info, Node: Functional Programming HOWTO, Next: Logging HOWTO, Prev: Descriptor HowTo Guide, Up: Python HOWTOs + +10.5 Functional Programming HOWTO +================================= + + +Author: A. M. Kuchling + + +Release: 0.32 + +In this document, we’ll take a tour of Python’s features suitable for +implementing programs in a functional style. After an introduction to +the concepts of functional programming, we’ll look at language features +such as *note iterator: 112e.s and *note generator: 9a2.s and relevant +library modules such as *note itertools: a3. and *note functools: 85. + +* Menu: + +* Introduction: Introduction<14>. +* Iterators: Iterators<2>. +* Generator expressions and list comprehensions:: +* Generators: Generators<2>. +* Built-in functions:: +* The itertools module:: +* The functools module:: +* Small functions and the lambda expression:: +* Revision History and Acknowledgements:: +* References: References<2>. + + +File: python.info, Node: Introduction<14>, Next: Iterators<2>, Up: Functional Programming HOWTO + +10.5.1 Introduction +------------------- + +This section explains the basic concept of functional programming; if +you’re just interested in learning about Python language features, skip +to the next section on *note Iterators: 3e89. + +Programming languages support decomposing problems in several different +ways: + + * Most programming languages are `procedural': programs are lists of + instructions that tell the computer what to do with the program’s + input. C, Pascal, and even Unix shells are procedural languages. + + * In `declarative' languages, you write a specification that + describes the problem to be solved, and the language implementation + figures out how to perform the computation efficiently. SQL is the + declarative language you’re most likely to be familiar with; a SQL + query describes the data set you want to retrieve, and the SQL + engine decides whether to scan tables or use indexes, which + subclauses should be performed first, etc. + + * `Object-oriented' programs manipulate collections of objects. + Objects have internal state and support methods that query or + modify this internal state in some way. Smalltalk and Java are + object-oriented languages. C++ and Python are languages that + support object-oriented programming, but don’t force the use of + object-oriented features. + + * `Functional' programming decomposes a problem into a set of + functions. Ideally, functions only take inputs and produce + outputs, and don’t have any internal state that affects the output + produced for a given input. Well-known functional languages + include the ML family (Standard ML, OCaml, and other variants) and + Haskell. + +The designers of some computer languages choose to emphasize one +particular approach to programming. This often makes it difficult to +write programs that use a different approach. Other languages are +multi-paradigm languages that support several different approaches. +Lisp, C++, and Python are multi-paradigm; you can write programs or +libraries that are largely procedural, object-oriented, or functional in +all of these languages. In a large program, different sections might be +written using different approaches; the GUI might be object-oriented +while the processing logic is procedural or functional, for example. + +In a functional program, input flows through a set of functions. Each +function operates on its input and produces some output. Functional +style discourages functions with side effects that modify internal state +or make other changes that aren’t visible in the function’s return +value. Functions that have no side effects at all are called `purely +functional'. Avoiding side effects means not using data structures that +get updated as a program runs; every function’s output must only depend +on its input. + +Some languages are very strict about purity and don’t even have +assignment statements such as ‘a=3’ or ‘c = a + b’, but it’s difficult +to avoid all side effects. Printing to the screen or writing to a disk +file are side effects, for example. For example, in Python a call to +the *note print(): 886. or *note time.sleep(): 680. function both return +no useful value; they’re only called for their side effects of sending +some text to the screen or pausing execution for a second. + +Python programs written in functional style usually won’t go to the +extreme of avoiding all I/O or all assignments; instead, they’ll provide +a functional-appearing interface but will use non-functional features +internally. For example, the implementation of a function will still +use assignments to local variables, but won’t modify global variables or +have other side effects. + +Functional programming can be considered the opposite of object-oriented +programming. Objects are little capsules containing some internal state +along with a collection of method calls that let you modify this state, +and programs consist of making the right set of state changes. +Functional programming wants to avoid state changes as much as possible +and works with data flowing between functions. In Python you might +combine the two approaches by writing functions that take and return +instances representing objects in your application (e-mail messages, +transactions, etc.). + +Functional design may seem like an odd constraint to work under. Why +should you avoid objects and side effects? There are theoretical and +practical advantages to the functional style: + + * Formal provability. + + * Modularity. + + * Composability. + + * Ease of debugging and testing. + +* Menu: + +* Formal provability:: +* Modularity:: +* Ease of debugging and testing:: +* Composability:: + + +File: python.info, Node: Formal provability, Next: Modularity, Up: Introduction<14> + +10.5.1.1 Formal provability +........................... + +A theoretical benefit is that it’s easier to construct a mathematical +proof that a functional program is correct. + +For a long time researchers have been interested in finding ways to +mathematically prove programs correct. This is different from testing a +program on numerous inputs and concluding that its output is usually +correct, or reading a program’s source code and concluding that the code +looks right; the goal is instead a rigorous proof that a program +produces the right result for all possible inputs. + +The technique used to prove programs correct is to write down +`invariants', properties of the input data and of the program’s +variables that are always true. For each line of code, you then show +that if invariants X and Y are true `before' the line is executed, the +slightly different invariants X’ and Y’ are true `after' the line is +executed. This continues until you reach the end of the program, at +which point the invariants should match the desired conditions on the +program’s output. + +Functional programming’s avoidance of assignments arose because +assignments are difficult to handle with this technique; assignments can +break invariants that were true before the assignment without producing +any new invariants that can be propagated onward. + +Unfortunately, proving programs correct is largely impractical and not +relevant to Python software. Even trivial programs require proofs that +are several pages long; the proof of correctness for a moderately +complicated program would be enormous, and few or none of the programs +you use daily (the Python interpreter, your XML parser, your web +browser) could be proven correct. Even if you wrote down or generated a +proof, there would then be the question of verifying the proof; maybe +there’s an error in it, and you wrongly believe you’ve proved the +program correct. + + +File: python.info, Node: Modularity, Next: Ease of debugging and testing, Prev: Formal provability, Up: Introduction<14> + +10.5.1.2 Modularity +................... + +A more practical benefit of functional programming is that it forces you +to break apart your problem into small pieces. Programs are more +modular as a result. It’s easier to specify and write a small function +that does one thing than a large function that performs a complicated +transformation. Small functions are also easier to read and to check +for errors. + + +File: python.info, Node: Ease of debugging and testing, Next: Composability, Prev: Modularity, Up: Introduction<14> + +10.5.1.3 Ease of debugging and testing +...................................... + +Testing and debugging a functional-style program is easier. + +Debugging is simplified because functions are generally small and +clearly specified. When a program doesn’t work, each function is an +interface point where you can check that the data are correct. You can +look at the intermediate inputs and outputs to quickly isolate the +function that’s responsible for a bug. + +Testing is easier because each function is a potential subject for a +unit test. Functions don’t depend on system state that needs to be +replicated before running a test; instead you only have to synthesize +the right input and then check that the output matches expectations. + + +File: python.info, Node: Composability, Prev: Ease of debugging and testing, Up: Introduction<14> + +10.5.1.4 Composability +...................... + +As you work on a functional-style program, you’ll write a number of +functions with varying inputs and outputs. Some of these functions will +be unavoidably specialized to a particular application, but others will +be useful in a wide variety of programs. For example, a function that +takes a directory path and returns all the XML files in the directory, +or a function that takes a filename and returns its contents, can be +applied to many different situations. + +Over time you’ll form a personal library of utilities. Often you’ll +assemble new programs by arranging existing functions in a new +configuration and writing a few functions specialized for the current +task. + + +File: python.info, Node: Iterators<2>, Next: Generator expressions and list comprehensions, Prev: Introduction<14>, Up: Functional Programming HOWTO + +10.5.2 Iterators +---------------- + +I’ll start by looking at a Python language feature that’s an important +foundation for writing functional-style programs: iterators. + +An iterator is an object representing a stream of data; this object +returns the data one element at a time. A Python iterator must support +a method called *note __next__(): c64. that takes no arguments and +always returns the next element of the stream. If there are no more +elements in the stream, *note __next__(): c64. must raise the *note +StopIteration: 486. exception. Iterators don’t have to be finite, +though; it’s perfectly reasonable to write an iterator that produces an +infinite stream of data. + +The built-in *note iter(): d27. function takes an arbitrary object and +tries to return an iterator that will return the object’s contents or +elements, raising *note TypeError: 192. if the object doesn’t support +iteration. Several of Python’s built-in data types support iteration, +the most common being lists and dictionaries. An object is called *note +iterable: bac. if you can get an iterator for it. + +You can experiment with the iteration interface manually: + + >>> L = [1, 2, 3] + >>> it = iter(L) + >>> it + <...iterator object at ...> + >>> it.__next__() # same as next(it) + 1 + >>> next(it) + 2 + >>> next(it) + 3 + >>> next(it) + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + StopIteration + >>> + +Python expects iterable objects in several different contexts, the most +important being the *note for: c30. statement. In the statement ‘for X +in Y’, Y must be an iterator or some object for which *note iter(): d27. +can create an iterator. These two statements are equivalent: + + for i in iter(obj): + print(i) + + for i in obj: + print(i) + +Iterators can be materialized as lists or tuples by using the *note +list(): 262. or *note tuple(): 47e. constructor functions: + + >>> L = [1, 2, 3] + >>> iterator = iter(L) + >>> t = tuple(iterator) + >>> t + (1, 2, 3) + +Sequence unpacking also supports iterators: if you know an iterator will +return N elements, you can unpack them into an N-tuple: + + >>> L = [1, 2, 3] + >>> iterator = iter(L) + >>> a, b, c = iterator + >>> a, b, c + (1, 2, 3) + +Built-in functions such as *note max(): 80a. and *note min(): 809. can +take a single iterator argument and will return the largest or smallest +element. The ‘"in"’ and ‘"not in"’ operators also support iterators: ‘X +in iterator’ is true if X is found in the stream returned by the +iterator. You’ll run into obvious problems if the iterator is infinite; +*note max(): 80a, *note min(): 809. will never return, and if the +element X never appears in the stream, the ‘"in"’ and ‘"not in"’ +operators won’t return either. + +Note that you can only go forward in an iterator; there’s no way to get +the previous element, reset the iterator, or make a copy of it. +Iterator objects can optionally provide these additional capabilities, +but the iterator protocol only specifies the *note __next__(): c64. +method. Functions may therefore consume all of the iterator’s output, +and if you need to do something different with the same stream, you’ll +have to create a new iterator. + +* Menu: + +* Data Types That Support Iterators:: + + +File: python.info, Node: Data Types That Support Iterators, Up: Iterators<2> + +10.5.2.1 Data Types That Support Iterators +.......................................... + +We’ve already seen how lists and tuples support iterators. In fact, any +Python sequence type, such as strings, will automatically support +creation of an iterator. + +Calling *note iter(): d27. on a dictionary returns an iterator that will +loop over the dictionary’s keys: + + >>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6, + ... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12} + >>> for key in m: + ... print(key, m[key]) + Jan 1 + Feb 2 + Mar 3 + Apr 4 + May 5 + Jun 6 + Jul 7 + Aug 8 + Sep 9 + Oct 10 + Nov 11 + Dec 12 + +Note that starting with Python 3.7, dictionary iteration order is +guaranteed to be the same as the insertion order. In earlier versions, +the behaviour was unspecified and could vary between implementations. + +Applying *note iter(): d27. to a dictionary always loops over the keys, +but dictionaries have methods that return other iterators. If you want +to iterate over values or key/value pairs, you can explicitly call the +*note values(): c2c. or *note items(): c2b. methods to get an +appropriate iterator. + +The *note dict(): 1b8. constructor can accept an iterator that returns a +finite stream of ‘(key, value)’ tuples: + + >>> L = [('Italy', 'Rome'), ('France', 'Paris'), ('US', 'Washington DC')] + >>> dict(iter(L)) + {'Italy': 'Rome', 'France': 'Paris', 'US': 'Washington DC'} + +Files also support iteration by calling the *note readline(): 18b9. +method until there are no more lines in the file. This means you can +read each line of a file like this: + + for line in file: + # do something for each line + ... + +Sets can take their contents from an iterable and let you iterate over +the set’s elements: + + S = {2, 3, 5, 7, 11, 13} + for i in S: + print(i) + + +File: python.info, Node: Generator expressions and list comprehensions, Next: Generators<2>, Prev: Iterators<2>, Up: Functional Programming HOWTO + +10.5.3 Generator expressions and list comprehensions +---------------------------------------------------- + +Two common operations on an iterator’s output are 1) performing some +operation for every element, 2) selecting a subset of elements that meet +some condition. For example, given a list of strings, you might want to +strip off trailing whitespace from each line or extract all the strings +containing a given substring. + +List comprehensions and generator expressions (short form: “listcomps” +and “genexps”) are a concise notation for such operations, borrowed from +the functional programming language Haskell +(‘https://www.haskell.org/’). You can strip all the whitespace from a +stream of strings with the following code: + + line_list = [' line 1\n', 'line 2 \n', ...] + + # Generator expression -- returns iterator + stripped_iter = (line.strip() for line in line_list) + + # List comprehension -- returns list + stripped_list = [line.strip() for line in line_list] + +You can select only certain elements by adding an ‘"if"’ condition: + + stripped_list = [line.strip() for line in line_list + if line != ""] + +With a list comprehension, you get back a Python list; ‘stripped_list’ +is a list containing the resulting lines, not an iterator. Generator +expressions return an iterator that computes the values as necessary, +not needing to materialize all the values at once. This means that list +comprehensions aren’t useful if you’re working with iterators that +return an infinite stream or a very large amount of data. Generator +expressions are preferable in these situations. + +Generator expressions are surrounded by parentheses (“()”) and list +comprehensions are surrounded by square brackets (“[]”). Generator +expressions have the form: + + ( expression for expr in sequence1 + if condition1 + for expr2 in sequence2 + if condition2 + for expr3 in sequence3 ... + if condition3 + for exprN in sequenceN + if conditionN ) + +Again, for a list comprehension only the outside brackets are different +(square brackets instead of parentheses). + +The elements of the generated output will be the successive values of +‘expression’. The ‘if’ clauses are all optional; if present, +‘expression’ is only evaluated and added to the result when ‘condition’ +is true. + +Generator expressions always have to be written inside parentheses, but +the parentheses signalling a function call also count. If you want to +create an iterator that will be immediately passed to a function you can +write: + + obj_total = sum(obj.count for obj in list_all_objects()) + +The ‘for...in’ clauses contain the sequences to be iterated over. The +sequences do not have to be the same length, because they are iterated +over from left to right, `not' in parallel. For each element in +‘sequence1’, ‘sequence2’ is looped over from the beginning. ‘sequence3’ +is then looped over for each resulting pair of elements from ‘sequence1’ +and ‘sequence2’. + +To put it another way, a list comprehension or generator expression is +equivalent to the following Python code: + + for expr1 in sequence1: + if not (condition1): + continue # Skip this element + for expr2 in sequence2: + if not (condition2): + continue # Skip this element + ... + for exprN in sequenceN: + if not (conditionN): + continue # Skip this element + + # Output the value of + # the expression. + +This means that when there are multiple ‘for...in’ clauses but no ‘if’ +clauses, the length of the resulting output will be equal to the product +of the lengths of all the sequences. If you have two lists of length 3, +the output list is 9 elements long: + + >>> seq1 = 'abc' + >>> seq2 = (1, 2, 3) + >>> [(x, y) for x in seq1 for y in seq2] + [('a', 1), ('a', 2), ('a', 3), + ('b', 1), ('b', 2), ('b', 3), + ('c', 1), ('c', 2), ('c', 3)] + +To avoid introducing an ambiguity into Python’s grammar, if ‘expression’ +is creating a tuple, it must be surrounded with parentheses. The first +list comprehension below is a syntax error, while the second one is +correct: + + # Syntax error + [x, y for x in seq1 for y in seq2] + # Correct + [(x, y) for x in seq1 for y in seq2] + + +File: python.info, Node: Generators<2>, Next: Built-in functions, Prev: Generator expressions and list comprehensions, Up: Functional Programming HOWTO + +10.5.4 Generators +----------------- + +Generators are a special class of functions that simplify the task of +writing iterators. Regular functions compute a value and return it, but +generators return an iterator that returns a stream of values. + +You’re doubtless familiar with how regular function calls work in Python +or C. When you call a function, it gets a private namespace where its +local variables are created. When the function reaches a ‘return’ +statement, the local variables are destroyed and the value is returned +to the caller. A later call to the same function creates a new private +namespace and a fresh set of local variables. But, what if the local +variables weren’t thrown away on exiting a function? What if you could +later resume the function where it left off? This is what generators +provide; they can be thought of as resumable functions. + +Here’s the simplest example of a generator function: + + >>> def generate_ints(N): + ... for i in range(N): + ... yield i + +Any function containing a *note yield: 18f. keyword is a generator +function; this is detected by Python’s *note bytecode: 614. compiler +which compiles the function specially as a result. + +When you call a generator function, it doesn’t return a single value; +instead it returns a generator object that supports the iterator +protocol. On executing the ‘yield’ expression, the generator outputs +the value of ‘i’, similar to a ‘return’ statement. The big difference +between ‘yield’ and a ‘return’ statement is that on reaching a ‘yield’ +the generator’s state of execution is suspended and local variables are +preserved. On the next call to the generator’s *note __next__(): f6f. +method, the function will resume executing. + +Here’s a sample usage of the ‘generate_ints()’ generator: + + >>> gen = generate_ints(3) + >>> gen + <generator object generate_ints at ...> + >>> next(gen) + 0 + >>> next(gen) + 1 + >>> next(gen) + 2 + >>> next(gen) + Traceback (most recent call last): + File "stdin", line 1, in <module> + File "stdin", line 2, in generate_ints + StopIteration + +You could equally write ‘for i in generate_ints(5)’, or ‘a, b, c = +generate_ints(3)’. + +Inside a generator function, ‘return value’ causes +‘StopIteration(value)’ to be raised from the *note __next__(): f6f. +method. Once this happens, or the bottom of the function is reached, +the procession of values ends and the generator cannot yield any further +values. + +You could achieve the effect of generators manually by writing your own +class and storing all the local variables of the generator as instance +variables. For example, returning a list of integers could be done by +setting ‘self.count’ to 0, and having the *note __next__(): c64. method +increment ‘self.count’ and return it. However, for a moderately +complicated generator, writing a corresponding class can be much +messier. + +The test suite included with Python’s library, +Lib/test/test_generators.py(1), contains a number of more interesting +examples. Here’s one generator that implements an in-order traversal of +a tree using generators recursively. + + # A recursive generator that generates Tree leaves in in-order. + def inorder(t): + if t: + for x in inorder(t.left): + yield x + + yield t.label + + for x in inorder(t.right): + yield x + +Two other examples in ‘test_generators.py’ produce solutions for the +N-Queens problem (placing N queens on an NxN chess board so that no +queen threatens another) and the Knight’s Tour (finding a route that +takes a knight to every square of an NxN chessboard without visiting any +square twice). + +* Menu: + +* Passing values into a generator:: + + ---------- Footnotes ---------- + + (1) +https://github.com/python/cpython/tree/3.8/Lib/test/test_generators.py + + +File: python.info, Node: Passing values into a generator, Up: Generators<2> + +10.5.4.1 Passing values into a generator +........................................ + +In Python 2.4 and earlier, generators only produced output. Once a +generator’s code was invoked to create an iterator, there was no way to +pass any new information into the function when its execution is +resumed. You could hack together this ability by making the generator +look at a global variable or by passing in some mutable object that +callers then modify, but these approaches are messy. + +In Python 2.5 there’s a simple way to pass values into a generator. +*note yield: 18f. became an expression, returning a value that can be +assigned to a variable or otherwise operated on: + + val = (yield i) + +I recommend that you `always' put parentheses around a ‘yield’ +expression when you’re doing something with the returned value, as in +the above example. The parentheses aren’t always necessary, but it’s +easier to always add them instead of having to remember when they’re +needed. + +( PEP 342(1) explains the exact rules, which are that a +‘yield’-expression must always be parenthesized except when it occurs at +the top-level expression on the right-hand side of an assignment. This +means you can write ‘val = yield i’ but have to use parentheses when +there’s an operation, as in ‘val = (yield i) + 12’.) + +Values are sent into a generator by calling its *note send(value): 1132. +method. This method resumes the generator’s code and the ‘yield’ +expression returns the specified value. If the regular *note +__next__(): f6f. method is called, the ‘yield’ returns ‘None’. + +Here’s a simple counter that increments by 1 and allows changing the +value of the internal counter. + + def counter(maximum): + i = 0 + while i < maximum: + val = (yield i) + # If value provided, change counter + if val is not None: + i = val + else: + i += 1 + +And here’s an example of changing the counter: + + >>> it = counter(10) + >>> next(it) + 0 + >>> next(it) + 1 + >>> it.send(8) + 8 + >>> next(it) + 9 + >>> next(it) + Traceback (most recent call last): + File "t.py", line 15, in <module> + it.next() + StopIteration + +Because ‘yield’ will often be returning ‘None’, you should always check +for this case. Don’t just use its value in expressions unless you’re +sure that the *note send(): 1132. method will be the only method used to +resume your generator function. + +In addition to *note send(): 1132, there are two other methods on +generators: + + * *note throw(type, value=None, traceback=None): 1134. is used to + raise an exception inside the generator; the exception is raised by + the ‘yield’ expression where the generator’s execution is paused. + + * *note close(): 1136. raises a *note GeneratorExit: d32. exception + inside the generator to terminate the iteration. On receiving this + exception, the generator’s code must either raise *note + GeneratorExit: d32. or *note StopIteration: 486.; catching the + exception and doing anything else is illegal and will trigger a + *note RuntimeError: 2ba. *note close(): 1136. will also be called + by Python’s garbage collector when the generator is + garbage-collected. + + If you need to run cleanup code when a *note GeneratorExit: d32. + occurs, I suggest using a ‘try: ... finally:’ suite instead of + catching *note GeneratorExit: d32. + +The cumulative effect of these changes is to turn generators from +one-way producers of information into both producers and consumers. + +Generators also become `coroutines', a more generalized form of +subroutines. Subroutines are entered at one point and exited at another +point (the top of the function, and a ‘return’ statement), but +coroutines can be entered, exited, and resumed at many different points +(the ‘yield’ statements). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0342 + + +File: python.info, Node: Built-in functions, Next: The itertools module, Prev: Generators<2>, Up: Functional Programming HOWTO + +10.5.5 Built-in functions +------------------------- + +Let’s look in more detail at built-in functions often used with +iterators. + +Two of Python’s built-in functions, *note map(): c2d. and *note +filter(): c2e. duplicate the features of generator expressions: + +*note map(f, iterA, iterB, ...): c2d. returns an iterator over the sequence + + ‘f(iterA[0], iterB[0]), f(iterA[1], iterB[1]), f(iterA[2], + iterB[2]), ...’. + + >>> def upper(s): + ... return s.upper() + + >>> list(map(upper, ['sentence', 'fragment'])) + ['SENTENCE', 'FRAGMENT'] + >>> [upper(s) for s in ['sentence', 'fragment']] + ['SENTENCE', 'FRAGMENT'] + +You can of course achieve the same effect with a list comprehension. + +*note filter(predicate, iter): c2e. returns an iterator over all the +sequence elements that meet a certain condition, and is similarly +duplicated by list comprehensions. A `predicate' is a function that +returns the truth value of some condition; for use with *note filter(): +c2e, the predicate must take a single value. + + >>> def is_even(x): + ... return (x % 2) == 0 + + >>> list(filter(is_even, range(10))) + [0, 2, 4, 6, 8] + +This can also be written as a list comprehension: + + >>> list(x for x in range(10) if is_even(x)) + [0, 2, 4, 6, 8] + +*note enumerate(iter, start=0): de3. counts off the elements in the +iterable returning 2-tuples containing the count (from `start') and each +element. + + >>> for item in enumerate(['subject', 'verb', 'object']): + ... print(item) + (0, 'subject') + (1, 'verb') + (2, 'object') + +*note enumerate(): de3. is often used when looping through a list and +recording the indexes at which certain conditions are met: + + f = open('data.txt', 'r') + for i, line in enumerate(f): + if line.strip() == '': + print('Blank line at line #%i' % i) + +*note sorted(iterable, key=None, reverse=False): 444. collects all the +elements of the iterable into a list, sorts the list, and returns the +sorted result. The `key' and `reverse' arguments are passed through to +the constructed list’s *note sort(): 445. method. + + >>> import random + >>> # Generate 8 random numbers between [0, 10000) + >>> rand_list = random.sample(range(10000), 8) + >>> rand_list + [769, 7953, 9828, 6431, 8442, 9878, 6213, 2207] + >>> sorted(rand_list) + [769, 2207, 6213, 6431, 7953, 8442, 9828, 9878] + >>> sorted(rand_list, reverse=True) + [9878, 9828, 8442, 7953, 6431, 6213, 2207, 769] + +(For a more detailed discussion of sorting, see the *note Sorting HOW +TO: 12ab.) + +The *note any(iter): d84. and *note all(iter): d85. built-ins look at +the truth values of an iterable’s contents. *note any(): d84. returns +‘True’ if any element in the iterable is a true value, and *note all(): +d85. returns ‘True’ if all of the elements are true values: + + >>> any([0, 1, 0]) + True + >>> any([0, 0, 0]) + False + >>> any([1, 1, 1]) + True + >>> all([0, 1, 0]) + False + >>> all([0, 0, 0]) + False + >>> all([1, 1, 1]) + True + +*note zip(iterA, iterB, ...): c32. takes one element from each iterable +and returns them in a tuple: + + zip(['a', 'b', 'c'], (1, 2, 3)) => + ('a', 1), ('b', 2), ('c', 3) + +It doesn’t construct an in-memory list and exhaust all the input +iterators before returning; instead tuples are constructed and returned +only if they’re requested. (The technical term for this behaviour is +lazy evaluation(1).) + +This iterator is intended to be used with iterables that are all of the +same length. If the iterables are of different lengths, the resulting +stream will be the same length as the shortest iterable. + + zip(['a', 'b'], (1, 2, 3)) => + ('a', 1), ('b', 2) + +You should avoid doing this, though, because an element may be taken +from the longer iterators and discarded. This means you can’t go on to +use the iterators further because you risk skipping a discarded element. + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Lazy_evaluation + + +File: python.info, Node: The itertools module, Next: The functools module, Prev: Built-in functions, Up: Functional Programming HOWTO + +10.5.6 The itertools module +--------------------------- + +The *note itertools: a3. module contains a number of commonly-used +iterators as well as functions for combining several iterators. This +section will introduce the module’s contents by showing small examples. + +The module’s functions fall into a few broad classes: + + * Functions that create a new iterator based on an existing iterator. + + * Functions for treating an iterator’s elements as function + arguments. + + * Functions for selecting portions of an iterator’s output. + + * A function for grouping an iterator’s output. + +* Menu: + +* Creating new iterators:: +* Calling functions on elements:: +* Selecting elements:: +* Combinatoric functions:: +* Grouping elements:: + + +File: python.info, Node: Creating new iterators, Next: Calling functions on elements, Up: The itertools module + +10.5.6.1 Creating new iterators +............................... + +*note itertools.count(start, step): c1b. returns an infinite stream of +evenly spaced values. You can optionally supply the starting number, +which defaults to 0, and the interval between numbers, which defaults to +1: + + itertools.count() => + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... + itertools.count(10) => + 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, ... + itertools.count(10, 5) => + 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, ... + +*note itertools.cycle(iter): 17ef. saves a copy of the contents of a +provided iterable and returns a new iterator that returns its elements +from first to last. The new iterator will repeat these elements +infinitely. + + itertools.cycle([1, 2, 3, 4, 5]) => + 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, ... + +*note itertools.repeat(elem, [n]): 17f0. returns the provided element +`n' times, or returns the element endlessly if `n' is not provided. + + itertools.repeat('abc') => + abc, abc, abc, abc, abc, abc, abc, abc, abc, abc, ... + itertools.repeat('abc', 5) => + abc, abc, abc, abc, abc + +*note itertools.chain(iterA, iterB, ...): 12ad. takes an arbitrary +number of iterables as input, and returns all the elements of the first +iterator, then all the elements of the second, and so on, until all of +the iterables have been exhausted. + + itertools.chain(['a', 'b', 'c'], (1, 2, 3)) => + a, b, c, 1, 2, 3 + +*note itertools.islice(iter, [start], stop, [step]): 3a2. returns a +stream that’s a slice of the iterator. With a single `stop' argument, +it will return the first `stop' elements. If you supply a starting +index, you’ll get `stop-start' elements, and if you supply a value for +`step', elements will be skipped accordingly. Unlike Python’s string +and list slicing, you can’t use negative values for `start', `stop', or +`step'. + + itertools.islice(range(10), 8) => + 0, 1, 2, 3, 4, 5, 6, 7 + itertools.islice(range(10), 2, 8) => + 2, 3, 4, 5, 6, 7 + itertools.islice(range(10), 2, 8, 2) => + 2, 4, 6 + +*note itertools.tee(iter, [n]): 17f5. replicates an iterator; it returns +`n' independent iterators that will all return the contents of the +source iterator. If you don’t supply a value for `n', the default is 2. +Replicating iterators requires saving some of the contents of the source +iterator, so this can consume significant memory if the iterator is +large and one of the new iterators is consumed more than the others. + + itertools.tee( itertools.count() ) => + iterA, iterB + + where iterA -> + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... + + and iterB -> + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... + + +File: python.info, Node: Calling functions on elements, Next: Selecting elements, Prev: Creating new iterators, Up: The itertools module + +10.5.6.2 Calling functions on elements +...................................... + +The *note operator: c2. module contains a set of functions corresponding +to Python’s operators. Some examples are *note operator.add(a, b): +1817. (adds two values), *note operator.ne(a, b): 1807. (same as ‘a != +b’), and *note operator.attrgetter(’id’): 71b. (returns a callable that +fetches the ‘.id’ attribute). + +*note itertools.starmap(func, iter): a28. assumes that the iterable will +return a stream of tuples, and calls `func' using these tuples as the +arguments: + + itertools.starmap(os.path.join, + [('/bin', 'python'), ('/usr', 'bin', 'java'), + ('/usr', 'bin', 'perl'), ('/usr', 'bin', 'ruby')]) + => + /bin/python, /usr/bin/java, /usr/bin/perl, /usr/bin/ruby + + +File: python.info, Node: Selecting elements, Next: Combinatoric functions, Prev: Calling functions on elements, Up: The itertools module + +10.5.6.3 Selecting elements +........................... + +Another group of functions chooses a subset of an iterator’s elements +based on a predicate. + +*note itertools.filterfalse(predicate, iter): 1296. is the opposite of +*note filter(): c2e, returning all elements for which the predicate +returns false: + + itertools.filterfalse(is_even, itertools.count()) => + 1, 3, 5, 7, 9, 11, 13, 15, ... + +*note itertools.takewhile(predicate, iter): 17f4. returns elements for +as long as the predicate returns true. Once the predicate returns +false, the iterator will signal the end of its results. + + def less_than_10(x): + return x < 10 + + itertools.takewhile(less_than_10, itertools.count()) => + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 + + itertools.takewhile(is_even, itertools.count()) => + 0 + +*note itertools.dropwhile(predicate, iter): 17f2. discards elements +while the predicate returns true, and then returns the rest of the +iterable’s results. + + itertools.dropwhile(less_than_10, itertools.count()) => + 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, ... + + itertools.dropwhile(is_even, itertools.count()) => + 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ... + +*note itertools.compress(data, selectors): c1a. takes two iterators and +returns only those elements of `data' for which the corresponding +element of `selectors' is true, stopping whenever either one is +exhausted: + + itertools.compress([1, 2, 3, 4, 5], [True, True, False, False, True]) => + 1, 2, 5 + + +File: python.info, Node: Combinatoric functions, Next: Grouping elements, Prev: Selecting elements, Up: The itertools module + +10.5.6.4 Combinatoric functions +............................... + +The *note itertools.combinations(iterable, r): ca6. returns an iterator +giving all possible `r'-tuple combinations of the elements contained in +`iterable'. + + itertools.combinations([1, 2, 3, 4, 5], 2) => + (1, 2), (1, 3), (1, 4), (1, 5), + (2, 3), (2, 4), (2, 5), + (3, 4), (3, 5), + (4, 5) + + itertools.combinations([1, 2, 3, 4, 5], 3) => + (1, 2, 3), (1, 2, 4), (1, 2, 5), (1, 3, 4), (1, 3, 5), (1, 4, 5), + (2, 3, 4), (2, 3, 5), (2, 4, 5), + (3, 4, 5) + +The elements within each tuple remain in the same order as `iterable' +returned them. For example, the number 1 is always before 2, 3, 4, or 5 +in the examples above. A similar function, *note +itertools.permutations(iterable, r=None): 17f6, removes this constraint +on the order, returning all possible arrangements of length `r': + + itertools.permutations([1, 2, 3, 4, 5], 2) => + (1, 2), (1, 3), (1, 4), (1, 5), + (2, 1), (2, 3), (2, 4), (2, 5), + (3, 1), (3, 2), (3, 4), (3, 5), + (4, 1), (4, 2), (4, 3), (4, 5), + (5, 1), (5, 2), (5, 3), (5, 4) + + itertools.permutations([1, 2, 3, 4, 5]) => + (1, 2, 3, 4, 5), (1, 2, 3, 5, 4), (1, 2, 4, 3, 5), + ... + (5, 4, 3, 2, 1) + +If you don’t supply a value for `r' the length of the iterable is used, +meaning that all the elements are permuted. + +Note that these functions produce all of the possible combinations by +position and don’t require that the contents of `iterable' are unique: + + itertools.permutations('aba', 3) => + ('a', 'b', 'a'), ('a', 'a', 'b'), ('b', 'a', 'a'), + ('b', 'a', 'a'), ('a', 'a', 'b'), ('a', 'b', 'a') + +The identical tuple ‘('a', 'a', 'b')’ occurs twice, but the two ‘a’ +strings came from different positions. + +The *note itertools.combinations_with_replacement(iterable, r): c19. +function relaxes a different constraint: elements can be repeated within +a single tuple. Conceptually an element is selected for the first +position of each tuple and then is replaced before the second element is +selected. + + itertools.combinations_with_replacement([1, 2, 3, 4, 5], 2) => + (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), + (2, 2), (2, 3), (2, 4), (2, 5), + (3, 3), (3, 4), (3, 5), + (4, 4), (4, 5), + (5, 5) + + +File: python.info, Node: Grouping elements, Prev: Combinatoric functions, Up: The itertools module + +10.5.6.5 Grouping elements +.......................... + +The last function I’ll discuss, *note itertools.groupby(iter, +key_func=None): 17f3, is the most complicated. ‘key_func(elem)’ is a +function that can compute a key value for each element returned by the +iterable. If you don’t supply a key function, the key is simply each +element itself. + +*note groupby(): 17f3. collects all the consecutive elements from the +underlying iterable that have the same key value, and returns a stream +of 2-tuples containing a key value and an iterator for the elements with +that key. + + city_list = [('Decatur', 'AL'), ('Huntsville', 'AL'), ('Selma', 'AL'), + ('Anchorage', 'AK'), ('Nome', 'AK'), + ('Flagstaff', 'AZ'), ('Phoenix', 'AZ'), ('Tucson', 'AZ'), + ... + ] + + def get_state(city_state): + return city_state[1] + + itertools.groupby(city_list, get_state) => + ('AL', iterator-1), + ('AK', iterator-2), + ('AZ', iterator-3), ... + + where + iterator-1 => + ('Decatur', 'AL'), ('Huntsville', 'AL'), ('Selma', 'AL') + iterator-2 => + ('Anchorage', 'AK'), ('Nome', 'AK') + iterator-3 => + ('Flagstaff', 'AZ'), ('Phoenix', 'AZ'), ('Tucson', 'AZ') + +*note groupby(): 17f3. assumes that the underlying iterable’s contents +will already be sorted based on the key. Note that the returned +iterators also use the underlying iterable, so you have to consume the +results of iterator-1 before requesting iterator-2 and its corresponding +key. + + +File: python.info, Node: The functools module, Next: Small functions and the lambda expression, Prev: The itertools module, Up: Functional Programming HOWTO + +10.5.7 The functools module +--------------------------- + +The *note functools: 85. module in Python 2.5 contains some higher-order +functions. A `higher-order function' takes one or more functions as +input and returns a new function. The most useful tool in this module +is the *note functools.partial(): 7c8. function. + +For programs written in a functional style, you’ll sometimes want to +construct variants of existing functions that have some of the +parameters filled in. Consider a Python function ‘f(a, b, c)’; you may +wish to create a new function ‘g(b, c)’ that’s equivalent to ‘f(1, b, +c)’; you’re filling in a value for one of ‘f()’’s parameters. This is +called “partial function application”. + +The constructor for *note partial(): 7c8. takes the arguments +‘(function, arg1, arg2, ..., kwarg1=value1, kwarg2=value2)’. The +resulting object is callable, so you can just call it to invoke +‘function’ with the filled-in arguments. + +Here’s a small but realistic example: + + import functools + + def log(message, subsystem): + """Write the contents of 'message' to the specified subsystem.""" + print('%s: %s' % (subsystem, message)) + ... + + server_log = functools.partial(log, subsystem='server') + server_log('Unable to open socket') + +*note functools.reduce(func, iter, [initial_value]): c6f. cumulatively +performs an operation on all the iterable’s elements and, therefore, +can’t be applied to infinite iterables. `func' must be a function that +takes two elements and returns a single value. *note +functools.reduce(): c6f. takes the first two elements A and B returned +by the iterator and calculates ‘func(A, B)’. It then requests the third +element, C, calculates ‘func(func(A, B), C)’, combines this result with +the fourth element returned, and continues until the iterable is +exhausted. If the iterable returns no values at all, a *note TypeError: +192. exception is raised. If the initial value is supplied, it’s used +as a starting point and ‘func(initial_value, A)’ is the first +calculation. + + >>> import operator, functools + >>> functools.reduce(operator.concat, ['A', 'BB', 'C']) + 'ABBC' + >>> functools.reduce(operator.concat, []) + Traceback (most recent call last): + ... + TypeError: reduce() of empty sequence with no initial value + >>> functools.reduce(operator.mul, [1, 2, 3], 1) + 6 + >>> functools.reduce(operator.mul, [], 1) + 1 + +If you use *note operator.add(): 1817. with *note functools.reduce(): +c6f, you’ll add up all the elements of the iterable. This case is so +common that there’s a special built-in called *note sum(): 1e5. to +compute it: + + >>> import functools, operator + >>> functools.reduce(operator.add, [1, 2, 3, 4], 0) + 10 + >>> sum([1, 2, 3, 4]) + 10 + >>> sum([]) + 0 + +For many uses of *note functools.reduce(): c6f, though, it can be +clearer to just write the obvious *note for: c30. loop: + + import functools + # Instead of: + product = functools.reduce(operator.mul, [1, 2, 3], 1) + + # You can write: + product = 1 + for i in [1, 2, 3]: + product *= i + +A related function is *note itertools.accumulate(iterable, +func=operator.add): 1dd. It performs the same calculation, but instead +of returning only the final result, ‘accumulate()’ returns an iterator +that also yields each partial result: + + itertools.accumulate([1, 2, 3, 4, 5]) => + 1, 3, 6, 10, 15 + + itertools.accumulate([1, 2, 3, 4, 5], operator.mul) => + 1, 2, 6, 24, 120 + +* Menu: + +* The operator module:: + + +File: python.info, Node: The operator module, Up: The functools module + +10.5.7.1 The operator module +............................ + +The *note operator: c2. module was mentioned earlier. It contains a set +of functions corresponding to Python’s operators. These functions are +often useful in functional-style code because they save you from writing +trivial functions that perform a single operation. + +Some of the functions in this module are: + + * Math operations: ‘add()’, ‘sub()’, ‘mul()’, ‘floordiv()’, ‘abs()’, + … + + * Logical operations: ‘not_()’, ‘truth()’. + + * Bitwise operations: ‘and_()’, ‘or_()’, ‘invert()’. + + * Comparisons: ‘eq()’, ‘ne()’, ‘lt()’, ‘le()’, ‘gt()’, and ‘ge()’. + + * Object identity: ‘is_()’, ‘is_not()’. + +Consult the operator module’s documentation for a complete list. + + +File: python.info, Node: Small functions and the lambda expression, Next: Revision History and Acknowledgements, Prev: The functools module, Up: Functional Programming HOWTO + +10.5.8 Small functions and the lambda expression +------------------------------------------------ + +When writing functional-style programs, you’ll often need little +functions that act as predicates or that combine elements in some way. + +If there’s a Python built-in or a module function that’s suitable, you +don’t need to define a new function at all: + + stripped_lines = [line.strip() for line in lines] + existing_files = filter(os.path.exists, file_list) + +If the function you need doesn’t exist, you need to write it. One way +to write small functions is to use the *note lambda: c2f. expression. +‘lambda’ takes a number of parameters and an expression combining these +parameters, and creates an anonymous function that returns the value of +the expression: + + adder = lambda x, y: x+y + + print_assign = lambda name, value: name + '=' + str(value) + +An alternative is to just use the ‘def’ statement and define a function +in the usual way: + + def adder(x, y): + return x + y + + def print_assign(name, value): + return name + '=' + str(value) + +Which alternative is preferable? That’s a style question; my usual +course is to avoid using ‘lambda’. + +One reason for my preference is that ‘lambda’ is quite limited in the +functions it can define. The result has to be computable as a single +expression, which means you can’t have multiway ‘if... elif... else’ +comparisons or ‘try... except’ statements. If you try to do too much in +a ‘lambda’ statement, you’ll end up with an overly complicated +expression that’s hard to read. Quick, what’s the following code doing? + + import functools + total = functools.reduce(lambda a, b: (0, a[1] + b[1]), items)[1] + +You can figure it out, but it takes time to disentangle the expression +to figure out what’s going on. Using a short nested ‘def’ statements +makes things a little bit better: + + import functools + def combine(a, b): + return 0, a[1] + b[1] + + total = functools.reduce(combine, items)[1] + +But it would be best of all if I had simply used a ‘for’ loop: + + total = 0 + for a, b in items: + total += b + +Or the *note sum(): 1e5. built-in and a generator expression: + + total = sum(b for a, b in items) + +Many uses of *note functools.reduce(): c6f. are clearer when written as +‘for’ loops. + +Fredrik Lundh once suggested the following set of rules for refactoring +uses of ‘lambda’: + + 1. Write a lambda function. + + 2. Write a comment explaining what the heck that lambda does. + + 3. Study the comment for a while, and think of a name that captures + the essence of the comment. + + 4. Convert the lambda to a def statement, using that name. + + 5. Remove the comment. + +I really like these rules, but you’re free to disagree about whether +this lambda-free style is better. + + +File: python.info, Node: Revision History and Acknowledgements, Next: References<2>, Prev: Small functions and the lambda expression, Up: Functional Programming HOWTO + +10.5.9 Revision History and Acknowledgements +-------------------------------------------- + +The author would like to thank the following people for offering +suggestions, corrections and assistance with various drafts of this +article: Ian Bicking, Nick Coghlan, Nick Efford, Raymond Hettinger, Jim +Jewett, Mike Krell, Leandro Lameiro, Jussi Salmela, Collin Winter, Blake +Winton. + +Version 0.1: posted June 30 2006. + +Version 0.11: posted July 1 2006. Typo fixes. + +Version 0.2: posted July 10 2006. Merged genexp and listcomp sections +into one. Typo fixes. + +Version 0.21: Added more references suggested on the tutor mailing list. + +Version 0.30: Adds a section on the ‘functional’ module written by +Collin Winter; adds short section on the operator module; a few other +edits. + + +File: python.info, Node: References<2>, Prev: Revision History and Acknowledgements, Up: Functional Programming HOWTO + +10.5.10 References +------------------ + +* Menu: + +* General:: +* Python-specific:: +* Python documentation:: + + +File: python.info, Node: General, Next: Python-specific, Up: References<2> + +10.5.10.1 General +................. + +`Structure and Interpretation of Computer Programs', by Harold Abelson +and Gerald Jay Sussman with Julie Sussman. Full text at +‘https://mitpress.mit.edu/sicp/’. In this classic textbook of computer +science, chapters 2 and 3 discuss the use of sequences and streams to +organize the data flow inside a program. The book uses Scheme for its +examples, but many of the design approaches described in these chapters +are applicable to functional-style Python code. + +‘http://www.defmacro.org/ramblings/fp.html’: A general introduction to +functional programming that uses Java examples and has a lengthy +historical introduction. + +‘https://en.wikipedia.org/wiki/Functional_programming’: General +Wikipedia entry describing functional programming. + +‘https://en.wikipedia.org/wiki/Coroutine’: Entry for coroutines. + +‘https://en.wikipedia.org/wiki/Currying’: Entry for the concept of +currying. + + +File: python.info, Node: Python-specific, Next: Python documentation, Prev: General, Up: References<2> + +10.5.10.2 Python-specific +......................... + +‘http://gnosis.cx/TPiP/’: The first chapter of David Mertz’s book ‘Text +Processing in Python’ discusses functional programming for text +processing, in the section titled “Utilizing Higher-Order Functions in +Text Processing”. + +Mertz also wrote a 3-part series of articles on functional programming +for IBM’s DeveloperWorks site; see part 1(1), part 2(2), and part 3(3), + + ---------- Footnotes ---------- + + (1) https://developer.ibm.com/articles/l-prog/ + + (2) https://developer.ibm.com/tutorials/l-prog2/ + + (3) https://developer.ibm.com/tutorials/l-prog3/ + + +File: python.info, Node: Python documentation, Prev: Python-specific, Up: References<2> + +10.5.10.3 Python documentation +.............................. + +Documentation for the *note itertools: a3. module. + +Documentation for the *note functools: 85. module. + +Documentation for the *note operator: c2. module. + +PEP 289(1): “Generator Expressions” + +PEP 342(2): “Coroutines via Enhanced Generators” describes the new +generator features in Python 2.5. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0289 + + (2) https://www.python.org/dev/peps/pep-0342 + + +File: python.info, Node: Logging HOWTO, Next: Logging Cookbook, Prev: Functional Programming HOWTO, Up: Python HOWTOs + +10.6 Logging HOWTO +================== + + +Author: Vinay Sajip <vinay_sajip at red-dove dot com> + +* Menu: + +* Basic Logging Tutorial:: +* Advanced Logging Tutorial:: +* Logging Levels: Logging Levels<2>. +* Useful Handlers:: +* Exceptions raised during logging:: +* Using arbitrary objects as messages:: +* Optimization:: + + +File: python.info, Node: Basic Logging Tutorial, Next: Advanced Logging Tutorial, Up: Logging HOWTO + +10.6.1 Basic Logging Tutorial +----------------------------- + +Logging is a means of tracking events that happen when some software +runs. The software’s developer adds logging calls to their code to +indicate that certain events have occurred. An event is described by a +descriptive message which can optionally contain variable data (i.e. +data that is potentially different for each occurrence of the event). +Events also have an importance which the developer ascribes to the +event; the importance can also be called the `level' or `severity'. + +* Menu: + +* When to use logging:: +* A simple example:: +* Logging to a file:: +* Logging from multiple modules:: +* Logging variable data:: +* Changing the format of displayed messages:: +* Displaying the date/time in messages:: +* Next Steps:: + + +File: python.info, Node: When to use logging, Next: A simple example, Up: Basic Logging Tutorial + +10.6.1.1 When to use logging +............................ + +Logging provides a set of convenience functions for simple logging +usage. These are *note debug(): 1d64, *note info(): 1d8f, *note +warning(): 1da1, *note error(): 1da2. and *note critical(): 1da3. To +determine when to use logging, see the table below, which states, for +each of a set of common tasks, the best tool to use for it. + +Task you want to perform The best tool for the task + +------------------------------------------------------------------------------------- + +Display console output for ordinary *note print(): 886. +usage of a command line script or +program + +Report events that occur during normal *note logging.info(): 1d8f. (or +operation of a program (e.g. for *note logging.debug(): 1d64. for very +status monitoring or fault detailed output for diagnostic purposes) +investigation) + +Issue a warning regarding a particular *note warnings.warn(): e58. in library +runtime event code if the issue is avoidable and the + client application should be modified to + eliminate the warning + + *note logging.warning(): 1da1. if there + is nothing the client application can do + about the situation, but the event + should still be noted + + +Report an error regarding a particular Raise an exception +runtime event + +Report suppression of an error without *note logging.error(): 1da2, +raising an exception (e.g. error *note logging.exception(): 1da4. or +handler in a long-running server *note logging.critical(): 1da3. as +process) appropriate for the specific error and + application domain + + +The logging functions are named after the level or severity of the +events they are used to track. The standard levels and their +applicability are described below (in increasing order of severity): + +Level When it’s used + +--------------------------------------------------------------------- + +‘DEBUG’ Detailed information, typically of interest + only when diagnosing problems. + + +‘INFO’ Confirmation that things are working as + expected. + + +‘WARNING’ An indication that something unexpected + happened, or indicative of some problem in the + near future (e.g. ‘disk space low’). The + software is still working as expected. + + +‘ERROR’ Due to a more serious problem, the software has + not been able to perform some function. + + +‘CRITICAL’ A serious error, indicating that the program + itself may be unable to continue running. + + +The default level is ‘WARNING’, which means that only events of this +level and above will be tracked, unless the logging package is +configured to do otherwise. + +Events that are tracked can be handled in different ways. The simplest +way of handling tracked events is to print them to the console. Another +common way is to write them to a disk file. + + +File: python.info, Node: A simple example, Next: Logging to a file, Prev: When to use logging, Up: Basic Logging Tutorial + +10.6.1.2 A simple example +......................... + +A very simple example is: + + import logging + logging.warning('Watch out!') # will print a message to the console + logging.info('I told you so') # will not print anything + +If you type these lines into a script and run it, you’ll see: + + WARNING:root:Watch out! + +printed out on the console. The ‘INFO’ message doesn’t appear because +the default level is ‘WARNING’. The printed message includes the +indication of the level and the description of the event provided in the +logging call, i.e. ‘Watch out!’. Don’t worry about the ‘root’ part for +now: it will be explained later. The actual output can be formatted +quite flexibly if you need that; formatting options will also be +explained later. + + +File: python.info, Node: Logging to a file, Next: Logging from multiple modules, Prev: A simple example, Up: Basic Logging Tutorial + +10.6.1.3 Logging to a file +.......................... + +A very common situation is that of recording logging events in a file, +so let’s look at that next. Be sure to try the following in a +newly-started Python interpreter, and don’t just continue from the +session described above: + + import logging + logging.basicConfig(filename='example.log',level=logging.DEBUG) + logging.debug('This message should go to the log file') + logging.info('So should this') + logging.warning('And this, too') + +And now if we open the file and look at what we have, we should find the +log messages: + + DEBUG:root:This message should go to the log file + INFO:root:So should this + WARNING:root:And this, too + +This example also shows how you can set the logging level which acts as +the threshold for tracking. In this case, because we set the threshold +to ‘DEBUG’, all of the messages were printed. + +If you want to set the logging level from a command-line option such as: + + --log=INFO + +and you have the value of the parameter passed for ‘--log’ in some +variable `loglevel', you can use: + + getattr(logging, loglevel.upper()) + +to get the value which you’ll pass to *note basicConfig(): 1e0. via the +`level' argument. You may want to error check any user input value, +perhaps as in the following example: + + # assuming loglevel is bound to the string value obtained from the + # command line argument. Convert to upper case to allow the user to + # specify --log=DEBUG or --log=debug + numeric_level = getattr(logging, loglevel.upper(), None) + if not isinstance(numeric_level, int): + raise ValueError('Invalid log level: %s' % loglevel) + logging.basicConfig(level=numeric_level, ...) + +The call to *note basicConfig(): 1e0. should come `before' any calls to +*note debug(): 1d64, *note info(): 1d8f. etc. As it’s intended as a +one-off simple configuration facility, only the first call will actually +do anything: subsequent calls are effectively no-ops. + +If you run the above script several times, the messages from successive +runs are appended to the file `example.log'. If you want each run to +start afresh, not remembering the messages from earlier runs, you can +specify the `filemode' argument, by changing the call in the above +example to: + + logging.basicConfig(filename='example.log', filemode='w', level=logging.DEBUG) + +The output will be the same as before, but the log file is no longer +appended to, so the messages from earlier runs are lost. + + +File: python.info, Node: Logging from multiple modules, Next: Logging variable data, Prev: Logging to a file, Up: Basic Logging Tutorial + +10.6.1.4 Logging from multiple modules +...................................... + +If your program consists of multiple modules, here’s an example of how +you could organize logging in it: + + # myapp.py + import logging + import mylib + + def main(): + logging.basicConfig(filename='myapp.log', level=logging.INFO) + logging.info('Started') + mylib.do_something() + logging.info('Finished') + + if __name__ == '__main__': + main() + + # mylib.py + import logging + + def do_something(): + logging.info('Doing something') + +If you run `myapp.py', you should see this in `myapp.log': + + INFO:root:Started + INFO:root:Doing something + INFO:root:Finished + +which is hopefully what you were expecting to see. You can generalize +this to multiple modules, using the pattern in `mylib.py'. Note that +for this simple usage pattern, you won’t know, by looking in the log +file, `where' in your application your messages came from, apart from +looking at the event description. If you want to track the location of +your messages, you’ll need to refer to the documentation beyond the +tutorial level – see *note Advanced Logging Tutorial: b73. + + +File: python.info, Node: Logging variable data, Next: Changing the format of displayed messages, Prev: Logging from multiple modules, Up: Basic Logging Tutorial + +10.6.1.5 Logging variable data +.............................. + +To log variable data, use a format string for the event description +message and append the variable data as arguments. For example: + + import logging + logging.warning('%s before you %s', 'Look', 'leap!') + +will display: + + WARNING:root:Look before you leap! + +As you can see, merging of variable data into the event description +message uses the old, %-style of string formatting. This is for +backwards compatibility: the logging package pre-dates newer formatting +options such as *note str.format(): 4da. and *note string.Template: 3eb. +These newer formatting options `are' supported, but exploring them is +outside the scope of this tutorial: see *note Using particular +formatting styles throughout your application: 1d87. for more +information. + + +File: python.info, Node: Changing the format of displayed messages, Next: Displaying the date/time in messages, Prev: Logging variable data, Up: Basic Logging Tutorial + +10.6.1.6 Changing the format of displayed messages +.................................................. + +To change the format which is used to display messages, you need to +specify the format you want to use: + + import logging + logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG) + logging.debug('This message should appear on the console') + logging.info('So should this') + logging.warning('And this, too') + +which would print: + + DEBUG:This message should appear on the console + INFO:So should this + WARNING:And this, too + +Notice that the ‘root’ which appeared in earlier examples has +disappeared. For a full set of things that can appear in format +strings, you can refer to the documentation for *note LogRecord +attributes: 1d85, but for simple usage, you just need the `levelname' +(severity), `message' (event description, including variable data) and +perhaps to display when the event occurred. This is described in the +next section. + + +File: python.info, Node: Displaying the date/time in messages, Next: Next Steps, Prev: Changing the format of displayed messages, Up: Basic Logging Tutorial + +10.6.1.7 Displaying the date/time in messages +............................................. + +To display the date and time of an event, you would place ‘%(asctime)s’ +in your format string: + + import logging + logging.basicConfig(format='%(asctime)s %(message)s') + logging.warning('is when this event was logged.') + +which should print something like this: + + 2010-12-12 11:41:42,612 is when this event was logged. + +The default format for date/time display (shown above) is like ISO8601 +or RFC 3339(1). If you need more control over the formatting of the +date/time, provide a `datefmt' argument to ‘basicConfig’, as in this +example: + + import logging + logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p') + logging.warning('is when this event was logged.') + +which would display something like this: + + 12/12/2010 11:46:36 AM is when this event was logged. + +The format of the `datefmt' argument is the same as supported by *note +time.strftime(): b65. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc3339.html + + +File: python.info, Node: Next Steps, Prev: Displaying the date/time in messages, Up: Basic Logging Tutorial + +10.6.1.8 Next Steps +................... + +That concludes the basic tutorial. It should be enough to get you up +and running with logging. There’s a lot more that the logging package +offers, but to get the best out of it, you’ll need to invest a little +more of your time in reading the following sections. If you’re ready +for that, grab some of your favourite beverage and carry on. + +If your logging needs are simple, then use the above examples to +incorporate logging into your own scripts, and if you run into problems +or don’t understand something, please post a question on the +comp.lang.python Usenet group (available at +‘https://groups.google.com/forum/#!forum/comp.lang.python’) and you +should receive help before too long. + +Still here? You can carry on reading the next few sections, which +provide a slightly more advanced/in-depth tutorial than the basic one +above. After that, you can take a look at the *note Logging Cookbook: +b74. + + +File: python.info, Node: Advanced Logging Tutorial, Next: Logging Levels<2>, Prev: Basic Logging Tutorial, Up: Logging HOWTO + +10.6.2 Advanced Logging Tutorial +-------------------------------- + +The logging library takes a modular approach and offers several +categories of components: loggers, handlers, filters, and formatters. + + * Loggers expose the interface that application code directly uses. + + * Handlers send the log records (created by loggers) to the + appropriate destination. + + * Filters provide a finer grained facility for determining which log + records to output. + + * Formatters specify the layout of log records in the final output. + +Log event information is passed between loggers, handlers, filters and +formatters in a *note LogRecord: 906. instance. + +Logging is performed by calling methods on instances of the *note +Logger: 3a7. class (hereafter called `loggers'). Each instance has a +name, and they are conceptually arranged in a namespace hierarchy using +dots (periods) as separators. For example, a logger named ‘scan’ is the +parent of loggers ‘scan.text’, ‘scan.html’ and ‘scan.pdf’. Logger names +can be anything you want, and indicate the area of an application in +which a logged message originates. + +A good convention to use when naming loggers is to use a module-level +logger, in each module which uses logging, named as follows: + + logger = logging.getLogger(__name__) + +This means that logger names track the package/module hierarchy, and +it’s intuitively obvious where events are logged just from the logger +name. + +The root of the hierarchy of loggers is called the root logger. That’s +the logger used by the functions *note debug(): 1d64, *note info(): +1d8f, *note warning(): 1da1, *note error(): 1da2. and *note critical(): +1da3, which just call the same-named method of the root logger. The +functions and the methods have the same signatures. The root logger’s +name is printed as ‘root’ in the logged output. + +It is, of course, possible to log messages to different destinations. +Support is included in the package for writing log messages to files, +HTTP GET/POST locations, email via SMTP, generic sockets, queues, or +OS-specific logging mechanisms such as syslog or the Windows NT event +log. Destinations are served by `handler' classes. You can create your +own log destination class if you have special requirements not met by +any of the built-in handler classes. + +By default, no destination is set for any logging messages. You can +specify a destination (such as console or file) by using *note +basicConfig(): 1e0. as in the tutorial examples. If you call the +functions *note debug(): 1d64, *note info(): 1d8f, *note warning(): +1da1, *note error(): 1da2. and *note critical(): 1da3, they will check +to see if no destination is set; and if one is not set, they will set a +destination of the console (‘sys.stderr’) and a default format for the +displayed message before delegating to the root logger to do the actual +message output. + +The default format set by *note basicConfig(): 1e0. for messages is: + + severity:logger name:message + +You can change this by passing a format string to *note basicConfig(): +1e0. with the `format' keyword argument. For all options regarding how +a format string is constructed, see *note Formatter Objects: 1d83. + +* Menu: + +* Logging Flow:: +* Loggers:: +* Handlers:: +* Formatters:: +* Configuring Logging:: +* What happens if no configuration is provided:: +* Configuring Logging for a Library:: + + +File: python.info, Node: Logging Flow, Next: Loggers, Up: Advanced Logging Tutorial + +10.6.2.1 Logging Flow +..................... + +The flow of log event information in loggers and handlers is illustrated +in the following diagram. + +[python-figures/logging_flow] + +File: python.info, Node: Loggers, Next: Handlers, Prev: Logging Flow, Up: Advanced Logging Tutorial + +10.6.2.2 Loggers +................ + +*note Logger: 3a7. objects have a threefold job. First, they expose +several methods to application code so that applications can log +messages at runtime. Second, logger objects determine which log +messages to act upon based upon severity (the default filtering +facility) or filter objects. Third, logger objects pass along relevant +log messages to all interested log handlers. + +The most widely used methods on logger objects fall into two categories: +configuration and message sending. + +These are the most common configuration methods: + + * *note Logger.setLevel(): 1d5d. specifies the lowest-severity log + message a logger will handle, where debug is the lowest built-in + severity level and critical is the highest built-in severity. For + example, if the severity level is INFO, the logger will handle only + INFO, WARNING, ERROR, and CRITICAL messages and will ignore DEBUG + messages. + + * *note Logger.addHandler(): 1d6a. and *note Logger.removeHandler(): + 1d6b. add and remove handler objects from the logger object. + Handlers are covered in more detail in *note Handlers: 3eb1. + + * *note Logger.addFilter(): 1d67. and *note Logger.removeFilter(): + 1d68. add and remove filter objects from the logger object. + Filters are covered in more detail in *note Filter Objects: 1d8c. + +You don’t need to always call these methods on every logger you create. +See the last two paragraphs in this section. + +With the logger object configured, the following methods create log +messages: + + * *note Logger.debug(): 710, *note Logger.info(): 1d63, *note + Logger.warning(): 1d65, *note Logger.error(): 1d66, and *note + Logger.critical(): 70f. all create log records with a message and a + level that corresponds to their respective method names. The + message is actually a format string, which may contain the standard + string substitution syntax of ‘%s’, ‘%d’, ‘%f’, and so on. The + rest of their arguments is a list of objects that correspond with + the substitution fields in the message. With regard to ‘**kwargs’, + the logging methods care only about a keyword of ‘exc_info’ and use + it to determine whether to log exception information. + + * *note Logger.exception(): 70e. creates a log message similar to + *note Logger.error(): 1d66. The difference is that *note + Logger.exception(): 70e. dumps a stack trace along with it. Call + this method only from an exception handler. + + * *note Logger.log(): 70d. takes a log level as an explicit argument. + This is a little more verbose for logging messages than using the + log level convenience methods listed above, but this is how to log + at custom log levels. + +*note getLogger(): 1d5b. returns a reference to a logger instance with +the specified name if it is provided, or ‘root’ if not. The names are +period-separated hierarchical structures. Multiple calls to *note +getLogger(): 1d5b. with the same name will return a reference to the +same logger object. Loggers that are further down in the hierarchical +list are children of loggers higher up in the list. For example, given +a logger with a name of ‘foo’, loggers with names of ‘foo.bar’, +‘foo.bar.baz’, and ‘foo.bam’ are all descendants of ‘foo’. + +Loggers have a concept of `effective level'. If a level is not +explicitly set on a logger, the level of its parent is used instead as +its effective level. If the parent has no explicit level set, `its' +parent is examined, and so on - all ancestors are searched until an +explicitly set level is found. The root logger always has an explicit +level set (‘WARNING’ by default). When deciding whether to process an +event, the effective level of the logger is used to determine whether +the event is passed to the logger’s handlers. + +Child loggers propagate messages up to the handlers associated with +their ancestor loggers. Because of this, it is unnecessary to define +and configure handlers for all the loggers an application uses. It is +sufficient to configure handlers for a top-level logger and create child +loggers as needed. (You can, however, turn off propagation by setting +the `propagate' attribute of a logger to ‘False’.) + + +File: python.info, Node: Handlers, Next: Formatters, Prev: Loggers, Up: Advanced Logging Tutorial + +10.6.2.3 Handlers +................. + +*note Handler: 1d62. objects are responsible for dispatching the +appropriate log messages (based on the log messages’ severity) to the +handler’s specified destination. *note Logger: 3a7. objects can add +zero or more handler objects to themselves with an *note addHandler(): +1d6a. method. As an example scenario, an application may want to send +all log messages to a log file, all log messages of error or higher to +stdout, and all messages of critical to an email address. This scenario +requires three individual handlers where each handler is responsible for +sending messages of a specific severity to a specific location. + +The standard library includes quite a few handler types (see *note +Useful Handlers: 3eb3.); the tutorials use mainly *note StreamHandler: +b75. and *note FileHandler: 1dc8. in its examples. + +There are very few methods in a handler for application developers to +concern themselves with. The only handler methods that seem relevant +for application developers who are using the built-in handler objects +(that is, not creating custom handlers) are the following configuration +methods: + + * The *note setLevel(): 1d77. method, just as in logger objects, + specifies the lowest severity that will be dispatched to the + appropriate destination. Why are there two ‘setLevel()’ methods? + The level set in the logger determines which severity of messages + it will pass to its handlers. The level set in each handler + determines which messages that handler will send on. + + * *note setFormatter(): 1d78. selects a Formatter object for this + handler to use. + + * *note addFilter(): 1d79. and *note removeFilter(): 1d7a. + respectively configure and deconfigure filter objects on handlers. + +Application code should not directly instantiate and use instances of +*note Handler: 1d62. Instead, the *note Handler: 1d62. class is a base +class that defines the interface that all handlers should have and +establishes some default behavior that child classes can use (or +override). + + +File: python.info, Node: Formatters, Next: Configuring Logging, Prev: Handlers, Up: Advanced Logging Tutorial + +10.6.2.4 Formatters +................... + +Formatter objects configure the final order, structure, and contents of +the log message. Unlike the base *note logging.Handler: 1d62. class, +application code may instantiate formatter classes, although you could +likely subclass the formatter if your application needs special +behavior. The constructor takes three optional arguments – a message +format string, a date format string and a style indicator. + + -- Method: logging.Formatter.__init__ (fmt=None, datefmt=None, + style='%') + +If there is no message format string, the default is to use the raw +message. If there is no date format string, the default date format is: + + %Y-%m-%d %H:%M:%S + +with the milliseconds tacked on at the end. The ‘style’ is one of ‘%’, +‘{’ or ‘$’. If one of these is not specified, then ‘%’ will be used. + +If the ‘style’ is ‘%’, the message format string uses ‘%(<dictionary +key>)s’ styled string substitution; the possible keys are documented in +*note LogRecord attributes: 1d85. If the style is ‘{’, the message +format string is assumed to be compatible with *note str.format(): 4da. +(using keyword arguments), while if the style is ‘$’ then the message +format string should conform to what is expected by *note +string.Template.substitute(): f94. + +Changed in version 3.2: Added the ‘style’ parameter. + +The following message format string will log the time in a +human-readable format, the severity of the message, and the contents of +the message, in that order: + + '%(asctime)s - %(levelname)s - %(message)s' + +Formatters use a user-configurable function to convert the creation time +of a record to a tuple. By default, *note time.localtime(): 155c. is +used; to change this for a particular formatter instance, set the +‘converter’ attribute of the instance to a function with the same +signature as *note time.localtime(): 155c. or *note time.gmtime(): b3f. +To change it for all formatters, for example if you want all logging +times to be shown in GMT, set the ‘converter’ attribute in the Formatter +class (to ‘time.gmtime’ for GMT display). + + +File: python.info, Node: Configuring Logging, Next: What happens if no configuration is provided, Prev: Formatters, Up: Advanced Logging Tutorial + +10.6.2.5 Configuring Logging +............................ + +Programmers can configure logging in three ways: + + 1. Creating loggers, handlers, and formatters explicitly using Python + code that calls the configuration methods listed above. + + 2. Creating a logging config file and reading it using the *note + fileConfig(): 3a9. function. + + 3. Creating a dictionary of configuration information and passing it + to the *note dictConfig(): b2b. function. + +For the reference documentation on the last two options, see *note +Configuration functions: c88. The following example configures a very +simple logger, a console handler, and a simple formatter using Python +code: + + import logging + + # create logger + logger = logging.getLogger('simple_example') + logger.setLevel(logging.DEBUG) + + # create console handler and set level to debug + ch = logging.StreamHandler() + ch.setLevel(logging.DEBUG) + + # create formatter + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + + # add formatter to ch + ch.setFormatter(formatter) + + # add ch to logger + logger.addHandler(ch) + + # 'application' code + logger.debug('debug message') + logger.info('info message') + logger.warning('warn message') + logger.error('error message') + logger.critical('critical message') + +Running this module from the command line produces the following output: + + $ python simple_logging_module.py + 2005-03-19 15:10:26,618 - simple_example - DEBUG - debug message + 2005-03-19 15:10:26,620 - simple_example - INFO - info message + 2005-03-19 15:10:26,695 - simple_example - WARNING - warn message + 2005-03-19 15:10:26,697 - simple_example - ERROR - error message + 2005-03-19 15:10:26,773 - simple_example - CRITICAL - critical message + +The following Python module creates a logger, handler, and formatter +nearly identical to those in the example listed above, with the only +difference being the names of the objects: + + import logging + import logging.config + + logging.config.fileConfig('logging.conf') + + # create logger + logger = logging.getLogger('simpleExample') + + # 'application' code + logger.debug('debug message') + logger.info('info message') + logger.warning('warn message') + logger.error('error message') + logger.critical('critical message') + +Here is the logging.conf file: + + [loggers] + keys=root,simpleExample + + [handlers] + keys=consoleHandler + + [formatters] + keys=simpleFormatter + + [logger_root] + level=DEBUG + handlers=consoleHandler + + [logger_simpleExample] + level=DEBUG + handlers=consoleHandler + qualname=simpleExample + propagate=0 + + [handler_consoleHandler] + class=StreamHandler + level=DEBUG + formatter=simpleFormatter + args=(sys.stdout,) + + [formatter_simpleFormatter] + format=%(asctime)s - %(name)s - %(levelname)s - %(message)s + datefmt= + +The output is nearly identical to that of the non-config-file-based +example: + + $ python simple_logging_config.py + 2005-03-19 15:38:55,977 - simpleExample - DEBUG - debug message + 2005-03-19 15:38:55,979 - simpleExample - INFO - info message + 2005-03-19 15:38:56,054 - simpleExample - WARNING - warn message + 2005-03-19 15:38:56,055 - simpleExample - ERROR - error message + 2005-03-19 15:38:56,130 - simpleExample - CRITICAL - critical message + +You can see that the config file approach has a few advantages over the +Python code approach, mainly separation of configuration and code and +the ability of noncoders to easily modify the logging properties. + + Warning: The *note fileConfig(): 3a9. function takes a default + parameter, ‘disable_existing_loggers’, which defaults to ‘True’ for + reasons of backward compatibility. This may or may not be what you + want, since it will cause any non-root loggers existing before the + *note fileConfig(): 3a9. call to be disabled unless they (or an + ancestor) are explicitly named in the configuration. Please refer + to the reference documentation for more information, and specify + ‘False’ for this parameter if you wish. + + The dictionary passed to *note dictConfig(): b2b. can also specify + a Boolean value with key ‘disable_existing_loggers’, which if not + specified explicitly in the dictionary also defaults to being + interpreted as ‘True’. This leads to the logger-disabling + behaviour described above, which may not be what you want - in + which case, provide the key explicitly with a value of ‘False’. + +Note that the class names referenced in config files need to be either +relative to the logging module, or absolute values which can be resolved +using normal import mechanisms. Thus, you could use either *note +WatchedFileHandler: 1dd9. (relative to the logging module) or +‘mypackage.mymodule.MyHandler’ (for a class defined in package +‘mypackage’ and module ‘mymodule’, where ‘mypackage’ is available on the +Python import path). + +In Python 3.2, a new means of configuring logging has been introduced, +using dictionaries to hold configuration information. This provides a +superset of the functionality of the config-file-based approach outlined +above, and is the recommended configuration method for new applications +and deployments. Because a Python dictionary is used to hold +configuration information, and since you can populate that dictionary +using different means, you have more options for configuration. For +example, you can use a configuration file in JSON format, or, if you +have access to YAML processing functionality, a file in YAML format, to +populate the configuration dictionary. Or, of course, you can construct +the dictionary in Python code, receive it in pickled form over a socket, +or use whatever approach makes sense for your application. + +Here’s an example of the same configuration as above, in YAML format for +the new dictionary-based approach: + + version: 1 + formatters: + simple: + format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s' + handlers: + console: + class: logging.StreamHandler + level: DEBUG + formatter: simple + stream: ext://sys.stdout + loggers: + simpleExample: + level: DEBUG + handlers: [console] + propagate: no + root: + level: DEBUG + handlers: [console] + +For more information about logging using a dictionary, see *note +Configuration functions: c88. + + +File: python.info, Node: What happens if no configuration is provided, Next: Configuring Logging for a Library, Prev: Configuring Logging, Up: Advanced Logging Tutorial + +10.6.2.6 What happens if no configuration is provided +..................................................... + +If no logging configuration is provided, it is possible to have a +situation where a logging event needs to be output, but no handlers can +be found to output the event. The behaviour of the logging package in +these circumstances is dependent on the Python version. + +For versions of Python prior to 3.2, the behaviour is as follows: + + * If `logging.raiseExceptions' is ‘False’ (production mode), the + event is silently dropped. + + * If `logging.raiseExceptions' is ‘True’ (development mode), a + message ‘No handlers could be found for logger X.Y.Z’ is printed + once. + +In Python 3.2 and later, the behaviour is as follows: + + * The event is output using a ‘handler of last resort’, stored in + ‘logging.lastResort’. This internal handler is not associated with + any logger, and acts like a *note StreamHandler: b75. which writes + the event description message to the current value of ‘sys.stderr’ + (therefore respecting any redirections which may be in effect). No + formatting is done on the message - just the bare event description + message is printed. The handler’s level is set to ‘WARNING’, so + all events at this and greater severities will be output. + +To obtain the pre-3.2 behaviour, ‘logging.lastResort’ can be set to +‘None’. + + +File: python.info, Node: Configuring Logging for a Library, Prev: What happens if no configuration is provided, Up: Advanced Logging Tutorial + +10.6.2.7 Configuring Logging for a Library +.......................................... + +When developing a library which uses logging, you should take care to +document how the library uses logging - for example, the names of +loggers used. Some consideration also needs to be given to its logging +configuration. If the using application does not use logging, and +library code makes logging calls, then (as described in the previous +section) events of severity ‘WARNING’ and greater will be printed to +‘sys.stderr’. This is regarded as the best default behaviour. + +If for some reason you `don’t' want these messages printed in the +absence of any logging configuration, you can attach a do-nothing +handler to the top-level logger for your library. This avoids the +message being printed, since a handler will always be found for the +library’s events: it just doesn’t produce any output. If the library +user configures logging for application use, presumably that +configuration will add some handlers, and if levels are suitably +configured then logging calls made in library code will send output to +those handlers, as normal. + +A do-nothing handler is included in the logging package: *note +NullHandler: c1c. (since Python 3.1). An instance of this handler could +be added to the top-level logger of the logging namespace used by the +library (`if' you want to prevent your library’s logged events being +output to ‘sys.stderr’ in the absence of logging configuration). If all +logging by a library `foo' is done using loggers with names matching +‘foo.x’, ‘foo.x.y’, etc. then the code: + + import logging + logging.getLogger('foo').addHandler(logging.NullHandler()) + +should have the desired effect. If an organisation produces a number of +libraries, then the logger name specified can be ‘orgname.foo’ rather +than just ‘foo’. + + Note: It is strongly advised that you `do not add any handlers + other than' *note NullHandler: c1c. `to your library’s loggers'. + This is because the configuration of handlers is the prerogative of + the application developer who uses your library. The application + developer knows their target audience and what handlers are most + appropriate for their application: if you add handlers ‘under the + hood’, you might well interfere with their ability to carry out + unit tests and deliver logs which suit their requirements. + + +File: python.info, Node: Logging Levels<2>, Next: Useful Handlers, Prev: Advanced Logging Tutorial, Up: Logging HOWTO + +10.6.3 Logging Levels +--------------------- + +The numeric values of logging levels are given in the following table. +These are primarily of interest if you want to define your own levels, +and need them to have specific values relative to the predefined levels. +If you define a level with the same numeric value, it overwrites the +predefined value; the predefined name is lost. + +Level Numeric value + +--------------------------------------- + +‘CRITICAL’ 50 + + +‘ERROR’ 40 + + +‘WARNING’ 30 + + +‘INFO’ 20 + + +‘DEBUG’ 10 + + +‘NOTSET’ 0 + + +Levels can also be associated with loggers, being set either by the +developer or through loading a saved logging configuration. When a +logging method is called on a logger, the logger compares its own level +with the level associated with the method call. If the logger’s level +is higher than the method call’s, no logging message is actually +generated. This is the basic mechanism controlling the verbosity of +logging output. + +Logging messages are encoded as instances of the *note LogRecord: 906. +class. When a logger decides to actually log an event, a *note +LogRecord: 906. instance is created from the logging message. + +Logging messages are subjected to a dispatch mechanism through the use +of `handlers', which are instances of subclasses of the *note Handler: +1d62. class. Handlers are responsible for ensuring that a logged +message (in the form of a *note LogRecord: 906.) ends up in a particular +location (or set of locations) which is useful for the target audience +for that message (such as end users, support desk staff, system +administrators, developers). Handlers are passed *note LogRecord: 906. +instances intended for particular destinations. Each logger can have +zero, one or more handlers associated with it (via the *note +addHandler(): 1d6a. method of *note Logger: 3a7.). In addition to any +handlers directly associated with a logger, `all handlers associated +with all ancestors of the logger' are called to dispatch the message +(unless the `propagate' flag for a logger is set to a false value, at +which point the passing to ancestor handlers stops). + +Just as for loggers, handlers can have levels associated with them. A +handler’s level acts as a filter in the same way as a logger’s level +does. If a handler decides to actually dispatch an event, the *note +emit(): 1d81. method is used to send the message to its destination. +Most user-defined subclasses of *note Handler: 1d62. will need to +override this *note emit(): 1d81. + +* Menu: + +* Custom Levels:: + + +File: python.info, Node: Custom Levels, Up: Logging Levels<2> + +10.6.3.1 Custom Levels +...................... + +Defining your own levels is possible, but should not be necessary, as +the existing levels have been chosen on the basis of practical +experience. However, if you are convinced that you need custom levels, +great care should be exercised when doing this, and it is possibly `a +very bad idea to define custom levels if you are developing a library'. +That’s because if multiple library authors all define their own custom +levels, there is a chance that the logging output from such multiple +libraries used together will be difficult for the using developer to +control and/or interpret, because a given numeric value might mean +different things for different libraries. + + +File: python.info, Node: Useful Handlers, Next: Exceptions raised during logging, Prev: Logging Levels<2>, Up: Logging HOWTO + +10.6.4 Useful Handlers +---------------------- + +In addition to the base *note Handler: 1d62. class, many useful +subclasses are provided: + + 1. *note StreamHandler: b75. instances send messages to streams + (file-like objects). + + 2. *note FileHandler: 1dc8. instances send messages to disk files. + + 3. *note BaseRotatingHandler: 1ddd. is the base class for handlers + that rotate log files at a certain point. It is not meant to be + instantiated directly. Instead, use *note RotatingFileHandler: + 1db9. or *note TimedRotatingFileHandler: 86e. + + 4. *note RotatingFileHandler: 1db9. instances send messages to disk + files, with support for maximum log file sizes and log file + rotation. + + 5. *note TimedRotatingFileHandler: 86e. instances send messages to + disk files, rotating the log file at certain timed intervals. + + 6. *note SocketHandler: 86f. instances send messages to TCP/IP + sockets. Since 3.4, Unix domain sockets are also supported. + + 7. *note DatagramHandler: 870. instances send messages to UDP sockets. + Since 3.4, Unix domain sockets are also supported. + + 8. *note SMTPHandler: 1e09. instances send messages to a designated + email address. + + 9. *note SysLogHandler: a1e. instances send messages to a Unix syslog + daemon, possibly on a remote machine. + + 10. *note NTEventLogHandler: 1e01. instances send messages to a + Windows NT/2000/XP event log. + + 11. *note MemoryHandler: 1dc2. instances send messages to a buffer in + memory, which is flushed whenever specific criteria are met. + + 12. *note HTTPHandler: 711. instances send messages to an HTTP server + using either ‘GET’ or ‘POST’ semantics. + + 13. *note WatchedFileHandler: 1dd9. instances watch the file they are + logging to. If the file changes, it is closed and reopened using + the file name. This handler is only useful on Unix-like systems; + Windows does not support the underlying mechanism used. + + 14. *note QueueHandler: 1e1c. instances send messages to a queue, such + as those implemented in the *note queue: da. or *note + multiprocessing: b7. modules. + + 15. *note NullHandler: c1c. instances do nothing with error messages. + They are used by library developers who want to use logging, but + want to avoid the ‘No handlers could be found for logger XXX’ + message which can be displayed if the library user has not + configured logging. See *note Configuring Logging for a Library: + 1dd6. for more information. + +New in version 3.1: The *note NullHandler: c1c. class. + +New in version 3.2: The *note QueueHandler: 1e1c. class. + +The *note NullHandler: c1c, *note StreamHandler: b75. and *note +FileHandler: 1dc8. classes are defined in the core logging package. The +other handlers are defined in a sub-module, *note logging.handlers: ac. +(There is also another sub-module, *note logging.config: ab, for +configuration functionality.) + +Logged messages are formatted for presentation through instances of the +*note Formatter: 1d61. class. They are initialized with a format string +suitable for use with the % operator and a dictionary. + +For formatting multiple messages in a batch, instances of +‘BufferingFormatter’ can be used. In addition to the format string +(which is applied to each message in the batch), there is provision for +header and trailer format strings. + +When filtering based on logger level and/or handler level is not enough, +instances of *note Filter: b77. can be added to both *note Logger: 3a7. +and *note Handler: 1d62. instances (through their *note addFilter(): +1d79. method). Before deciding to process a message further, both +loggers and handlers consult all their filters for permission. If any +filter returns a false value, the message is not processed further. + +The basic *note Filter: b77. functionality allows filtering by specific +logger name. If this feature is used, messages sent to the named logger +and its children are allowed through the filter, and all others dropped. + + +File: python.info, Node: Exceptions raised during logging, Next: Using arbitrary objects as messages, Prev: Useful Handlers, Up: Logging HOWTO + +10.6.5 Exceptions raised during logging +--------------------------------------- + +The logging package is designed to swallow exceptions which occur while +logging in production. This is so that errors which occur while +handling logging events - such as logging misconfiguration, network or +other similar errors - do not cause the application using logging to +terminate prematurely. + +*note SystemExit: 5fc. and *note KeyboardInterrupt: 197. exceptions are +never swallowed. Other exceptions which occur during the *note emit(): +1d81. method of a *note Handler: 1d62. subclass are passed to its *note +handleError(): 1d80. method. + +The default implementation of *note handleError(): 1d80. in *note +Handler: 1d62. checks to see if a module-level variable, +‘raiseExceptions’, is set. If set, a traceback is printed to *note +sys.stderr: 30f. If not set, the exception is swallowed. + + Note: The default value of ‘raiseExceptions’ is ‘True’. This is + because during development, you typically want to be notified of + any exceptions that occur. It’s advised that you set + ‘raiseExceptions’ to ‘False’ for production usage. + + +File: python.info, Node: Using arbitrary objects as messages, Next: Optimization, Prev: Exceptions raised during logging, Up: Logging HOWTO + +10.6.6 Using arbitrary objects as messages +------------------------------------------ + +In the preceding sections and examples, it has been assumed that the +message passed when logging the event is a string. However, this is not +the only possibility. You can pass an arbitrary object as a message, +and its *note __str__(): e37. method will be called when the logging +system needs to convert it to a string representation. In fact, if you +want to, you can avoid computing a string representation altogether - +for example, the *note SocketHandler: 86f. emits an event by pickling it +and sending it over the wire. + + +File: python.info, Node: Optimization, Prev: Using arbitrary objects as messages, Up: Logging HOWTO + +10.6.7 Optimization +------------------- + +Formatting of message arguments is deferred until it cannot be avoided. +However, computing the arguments passed to the logging method can also +be expensive, and you may want to avoid doing it if the logger will just +throw away your event. To decide what to do, you can call the *note +isEnabledFor(): 1d60. method which takes a level argument and returns +true if the event would be created by the Logger for that level of call. +You can write code like this: + + if logger.isEnabledFor(logging.DEBUG): + logger.debug('Message with %s, %s', expensive_func1(), + expensive_func2()) + +so that if the logger’s threshold is set above ‘DEBUG’, the calls to +‘expensive_func1()’ and ‘expensive_func2()’ are never made. + + Note: In some cases, *note isEnabledFor(): 1d60. can itself be more + expensive than you’d like (e.g. for deeply nested loggers where an + explicit level is only set high up in the logger hierarchy). In + such cases (or if you want to avoid calling a method in tight + loops), you can cache the result of a call to *note isEnabledFor(): + 1d60. in a local or instance variable, and use that instead of + calling the method each time. Such a cached value would only need + to be recomputed when the logging configuration changes dynamically + while the application is running (which is not all that common). + +There are other optimizations which can be made for specific +applications which need more precise control over what logging +information is collected. Here’s a list of things you can do to avoid +processing during logging which you don’t need: + +What you don’t want to collect How to avoid collecting it + +------------------------------------------------------------------------------------------------- + +Information about where calls were made from. Set ‘logging._srcfile’ to ‘None’. This + avoids calling *note sys._getframe(): e64, + which may help to speed up your code in + environments like PyPy (which can’t speed + up code that uses + *note sys._getframe(): e64.), if and when + PyPy supports Python 3.x. + + +Threading information. Set ‘logging.logThreads’ to ‘0’. + + +Process information. Set ‘logging.logProcesses’ to ‘0’. + + +Also note that the core logging module only includes the basic handlers. +If you don’t import *note logging.handlers: ac. and *note +logging.config: ab, they won’t take up any memory. + +See also +........ + +Module *note logging: aa. + + API reference for the logging module. + +Module *note logging.config: ab. + + Configuration API for the logging module. + +Module *note logging.handlers: ac. + + Useful handlers included with the logging module. + +*note A logging cookbook: b74. + + +File: python.info, Node: Logging Cookbook, Next: Regular Expression HOWTO, Prev: Logging HOWTO, Up: Python HOWTOs + +10.7 Logging Cookbook +===================== + + +Author: Vinay Sajip <vinay_sajip at red-dove dot com> + +This page contains a number of recipes related to logging, which have +been found useful in the past. + +* Menu: + +* Using logging in multiple modules:: +* Logging from multiple threads:: +* Multiple handlers and formatters:: +* Logging to multiple destinations:: +* Configuration server example:: +* Dealing with handlers that block:: +* Sending and receiving logging events across a network:: +* Adding contextual information to your logging output:: +* Logging to a single file from multiple processes:: +* Using file rotation:: +* Use of alternative formatting styles:: +* Customizing LogRecord:: +* Subclassing QueueHandler - a ZeroMQ example:: +* Subclassing QueueListener - a ZeroMQ example:: +* An example dictionary-based configuration:: +* Using a rotator and namer to customize log rotation processing:: +* A more elaborate multiprocessing example:: +* Inserting a BOM into messages sent to a SysLogHandler:: +* Implementing structured logging:: +* Customizing handlers with dictConfig(): Customizing handlers with dictConfig. +* Using particular formatting styles throughout your application:: +* Configuring filters with dictConfig(): Configuring filters with dictConfig. +* Customized exception formatting:: +* Speaking logging messages:: +* Buffering logging messages and outputting them conditionally:: +* Formatting times using UTC (GMT) via configuration: Formatting times using UTC GMT via configuration. +* Using a context manager for selective logging:: +* A CLI application starter template:: +* A Qt GUI for logging:: + + +File: python.info, Node: Using logging in multiple modules, Next: Logging from multiple threads, Up: Logging Cookbook + +10.7.1 Using logging in multiple modules +---------------------------------------- + +Multiple calls to ‘logging.getLogger('someLogger')’ return a reference +to the same logger object. This is true not only within the same +module, but also across modules as long as it is in the same Python +interpreter process. It is true for references to the same object; +additionally, application code can define and configure a parent logger +in one module and create (but not configure) a child logger in a +separate module, and all logger calls to the child will pass up to the +parent. Here is a main module: + + import logging + import auxiliary_module + + # create logger with 'spam_application' + logger = logging.getLogger('spam_application') + logger.setLevel(logging.DEBUG) + # create file handler which logs even debug messages + fh = logging.FileHandler('spam.log') + fh.setLevel(logging.DEBUG) + # create console handler with a higher log level + ch = logging.StreamHandler() + ch.setLevel(logging.ERROR) + # create formatter and add it to the handlers + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + fh.setFormatter(formatter) + ch.setFormatter(formatter) + # add the handlers to the logger + logger.addHandler(fh) + logger.addHandler(ch) + + logger.info('creating an instance of auxiliary_module.Auxiliary') + a = auxiliary_module.Auxiliary() + logger.info('created an instance of auxiliary_module.Auxiliary') + logger.info('calling auxiliary_module.Auxiliary.do_something') + a.do_something() + logger.info('finished auxiliary_module.Auxiliary.do_something') + logger.info('calling auxiliary_module.some_function()') + auxiliary_module.some_function() + logger.info('done with auxiliary_module.some_function()') + +Here is the auxiliary module: + + import logging + + # create logger + module_logger = logging.getLogger('spam_application.auxiliary') + + class Auxiliary: + def __init__(self): + self.logger = logging.getLogger('spam_application.auxiliary.Auxiliary') + self.logger.info('creating an instance of Auxiliary') + + def do_something(self): + self.logger.info('doing something') + a = 1 + 1 + self.logger.info('done doing something') + + def some_function(): + module_logger.info('received a call to "some_function"') + +The output looks like this: + + 2005-03-23 23:47:11,663 - spam_application - INFO - + creating an instance of auxiliary_module.Auxiliary + 2005-03-23 23:47:11,665 - spam_application.auxiliary.Auxiliary - INFO - + creating an instance of Auxiliary + 2005-03-23 23:47:11,665 - spam_application - INFO - + created an instance of auxiliary_module.Auxiliary + 2005-03-23 23:47:11,668 - spam_application - INFO - + calling auxiliary_module.Auxiliary.do_something + 2005-03-23 23:47:11,668 - spam_application.auxiliary.Auxiliary - INFO - + doing something + 2005-03-23 23:47:11,669 - spam_application.auxiliary.Auxiliary - INFO - + done doing something + 2005-03-23 23:47:11,670 - spam_application - INFO - + finished auxiliary_module.Auxiliary.do_something + 2005-03-23 23:47:11,671 - spam_application - INFO - + calling auxiliary_module.some_function() + 2005-03-23 23:47:11,672 - spam_application.auxiliary - INFO - + received a call to 'some_function' + 2005-03-23 23:47:11,673 - spam_application - INFO - + done with auxiliary_module.some_function() + + +File: python.info, Node: Logging from multiple threads, Next: Multiple handlers and formatters, Prev: Using logging in multiple modules, Up: Logging Cookbook + +10.7.2 Logging from multiple threads +------------------------------------ + +Logging from multiple threads requires no special effort. The following +example shows logging from the main (initial) thread and another thread: + + import logging + import threading + import time + + def worker(arg): + while not arg['stop']: + logging.debug('Hi from myfunc') + time.sleep(0.5) + + def main(): + logging.basicConfig(level=logging.DEBUG, format='%(relativeCreated)6d %(threadName)s %(message)s') + info = {'stop': False} + thread = threading.Thread(target=worker, args=(info,)) + thread.start() + while True: + try: + logging.debug('Hello from main') + time.sleep(0.75) + except KeyboardInterrupt: + info['stop'] = True + break + thread.join() + + if __name__ == '__main__': + main() + +When run, the script should print something like the following: + + 0 Thread-1 Hi from myfunc + 3 MainThread Hello from main + 505 Thread-1 Hi from myfunc + 755 MainThread Hello from main + 1007 Thread-1 Hi from myfunc + 1507 MainThread Hello from main + 1508 Thread-1 Hi from myfunc + 2010 Thread-1 Hi from myfunc + 2258 MainThread Hello from main + 2512 Thread-1 Hi from myfunc + 3009 MainThread Hello from main + 3013 Thread-1 Hi from myfunc + 3515 Thread-1 Hi from myfunc + 3761 MainThread Hello from main + 4017 Thread-1 Hi from myfunc + 4513 MainThread Hello from main + 4518 Thread-1 Hi from myfunc + +This shows the logging output interspersed as one might expect. This +approach works for more threads than shown here, of course. + + +File: python.info, Node: Multiple handlers and formatters, Next: Logging to multiple destinations, Prev: Logging from multiple threads, Up: Logging Cookbook + +10.7.3 Multiple handlers and formatters +--------------------------------------- + +Loggers are plain Python objects. The *note addHandler(): 1d6a. method +has no minimum or maximum quota for the number of handlers you may add. +Sometimes it will be beneficial for an application to log all messages +of all severities to a text file while simultaneously logging errors or +above to the console. To set this up, simply configure the appropriate +handlers. The logging calls in the application code will remain +unchanged. Here is a slight modification to the previous simple +module-based configuration example: + + import logging + + logger = logging.getLogger('simple_example') + logger.setLevel(logging.DEBUG) + # create file handler which logs even debug messages + fh = logging.FileHandler('spam.log') + fh.setLevel(logging.DEBUG) + # create console handler with a higher log level + ch = logging.StreamHandler() + ch.setLevel(logging.ERROR) + # create formatter and add it to the handlers + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + ch.setFormatter(formatter) + fh.setFormatter(formatter) + # add the handlers to logger + logger.addHandler(ch) + logger.addHandler(fh) + + # 'application' code + logger.debug('debug message') + logger.info('info message') + logger.warning('warn message') + logger.error('error message') + logger.critical('critical message') + +Notice that the ‘application’ code does not care about multiple +handlers. All that changed was the addition and configuration of a new +handler named `fh'. + +The ability to create new handlers with higher- or lower-severity +filters can be very helpful when writing and testing an application. +Instead of using many ‘print’ statements for debugging, use +‘logger.debug’: Unlike the print statements, which you will have to +delete or comment out later, the logger.debug statements can remain +intact in the source code and remain dormant until you need them again. +At that time, the only change that needs to happen is to modify the +severity level of the logger and/or handler to debug. + + +File: python.info, Node: Logging to multiple destinations, Next: Configuration server example, Prev: Multiple handlers and formatters, Up: Logging Cookbook + +10.7.4 Logging to multiple destinations +--------------------------------------- + +Let’s say you want to log to console and file with different message +formats and in differing circumstances. Say you want to log messages +with levels of DEBUG and higher to file, and those messages at level +INFO and higher to the console. Let’s also assume that the file should +contain timestamps, but the console messages should not. Here’s how you +can achieve this: + + import logging + + # set up logging to file - see previous section for more details + logging.basicConfig(level=logging.DEBUG, + format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s', + datefmt='%m-%d %H:%M', + filename='/temp/myapp.log', + filemode='w') + # define a Handler which writes INFO messages or higher to the sys.stderr + console = logging.StreamHandler() + console.setLevel(logging.INFO) + # set a format which is simpler for console use + formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s') + # tell the handler to use this format + console.setFormatter(formatter) + # add the handler to the root logger + logging.getLogger('').addHandler(console) + + # Now, we can log to the root logger, or any other logger. First the root... + logging.info('Jackdaws love my big sphinx of quartz.') + + # Now, define a couple of other loggers which might represent areas in your + # application: + + logger1 = logging.getLogger('myapp.area1') + logger2 = logging.getLogger('myapp.area2') + + logger1.debug('Quick zephyrs blow, vexing daft Jim.') + logger1.info('How quickly daft jumping zebras vex.') + logger2.warning('Jail zesty vixen who grabbed pay from quack.') + logger2.error('The five boxing wizards jump quickly.') + +When you run this, on the console you will see + + root : INFO Jackdaws love my big sphinx of quartz. + myapp.area1 : INFO How quickly daft jumping zebras vex. + myapp.area2 : WARNING Jail zesty vixen who grabbed pay from quack. + myapp.area2 : ERROR The five boxing wizards jump quickly. + +and in the file you will see something like + + 10-22 22:19 root INFO Jackdaws love my big sphinx of quartz. + 10-22 22:19 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim. + 10-22 22:19 myapp.area1 INFO How quickly daft jumping zebras vex. + 10-22 22:19 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack. + 10-22 22:19 myapp.area2 ERROR The five boxing wizards jump quickly. + +As you can see, the DEBUG message only shows up in the file. The other +messages are sent to both destinations. + +This example uses console and file handlers, but you can use any number +and combination of handlers you choose. + + +File: python.info, Node: Configuration server example, Next: Dealing with handlers that block, Prev: Logging to multiple destinations, Up: Logging Cookbook + +10.7.5 Configuration server example +----------------------------------- + +Here is an example of a module using the logging configuration server: + + import logging + import logging.config + import time + import os + + # read initial config file + logging.config.fileConfig('logging.conf') + + # create and start listener on port 9999 + t = logging.config.listen(9999) + t.start() + + logger = logging.getLogger('simpleExample') + + try: + # loop through logging calls to see the difference + # new configurations make, until Ctrl+C is pressed + while True: + logger.debug('debug message') + logger.info('info message') + logger.warning('warn message') + logger.error('error message') + logger.critical('critical message') + time.sleep(5) + except KeyboardInterrupt: + # cleanup + logging.config.stopListening() + t.join() + +And here is a script that takes a filename and sends that file to the +server, properly preceded with the binary-encoded length, as the new +logging configuration: + + #!/usr/bin/env python + import socket, sys, struct + + with open(sys.argv[1], 'rb') as f: + data_to_send = f.read() + + HOST = 'localhost' + PORT = 9999 + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + print('connecting...') + s.connect((HOST, PORT)) + print('sending config...') + s.send(struct.pack('>L', len(data_to_send))) + s.send(data_to_send) + s.close() + print('complete') + + +File: python.info, Node: Dealing with handlers that block, Next: Sending and receiving logging events across a network, Prev: Configuration server example, Up: Logging Cookbook + +10.7.6 Dealing with handlers that block +--------------------------------------- + +Sometimes you have to get your logging handlers to do their work without +blocking the thread you’re logging from. This is common in Web +applications, though of course it also occurs in other scenarios. + +A common culprit which demonstrates sluggish behaviour is the *note +SMTPHandler: 1e09.: sending emails can take a long time, for a number of +reasons outside the developer’s control (for example, a poorly +performing mail or network infrastructure). But almost any +network-based handler can block: Even a *note SocketHandler: 86f. +operation may do a DNS query under the hood which is too slow (and this +query can be deep in the socket library code, below the Python layer, +and outside your control). + +One solution is to use a two-part approach. For the first part, attach +only a *note QueueHandler: 1e1c. to those loggers which are accessed +from performance-critical threads. They simply write to their queue, +which can be sized to a large enough capacity or initialized with no +upper bound to their size. The write to the queue will typically be +accepted quickly, though you will probably need to catch the *note +queue.Full: 20a1. exception as a precaution in your code. If you are a +library developer who has performance-critical threads in their code, be +sure to document this (together with a suggestion to attach only +‘QueueHandlers’ to your loggers) for the benefit of other developers who +will use your code. + +The second part of the solution is *note QueueListener: 712, which has +been designed as the counterpart to *note QueueHandler: 1e1c. A *note +QueueListener: 712. is very simple: it’s passed a queue and some +handlers, and it fires up an internal thread which listens to its queue +for LogRecords sent from ‘QueueHandlers’ (or any other source of +‘LogRecords’, for that matter). The ‘LogRecords’ are removed from the +queue and passed to the handlers for processing. + +The advantage of having a separate *note QueueListener: 712. class is +that you can use the same instance to service multiple ‘QueueHandlers’. +This is more resource-friendly than, say, having threaded versions of +the existing handler classes, which would eat up one thread per handler +for no particular benefit. + +An example of using these two classes follows (imports omitted): + + que = queue.Queue(-1) # no limit on size + queue_handler = QueueHandler(que) + handler = logging.StreamHandler() + listener = QueueListener(que, handler) + root = logging.getLogger() + root.addHandler(queue_handler) + formatter = logging.Formatter('%(threadName)s: %(message)s') + handler.setFormatter(formatter) + listener.start() + # The log output will display the thread which generated + # the event (the main thread) rather than the internal + # thread which monitors the internal queue. This is what + # you want to happen. + root.warning('Look out!') + listener.stop() + +which, when run, will produce: + + MainThread: Look out! + +Changed in version 3.5: Prior to Python 3.5, the *note QueueListener: +712. always passed every message received from the queue to every +handler it was initialized with. (This was because it was assumed that +level filtering was all done on the other side, where the queue is +filled.) From 3.5 onwards, this behaviour can be changed by passing a +keyword argument ‘respect_handler_level=True’ to the listener’s +constructor. When this is done, the listener compares the level of each +message with the handler’s level, and only passes a message to a handler +if it’s appropriate to do so. + + +File: python.info, Node: Sending and receiving logging events across a network, Next: Adding contextual information to your logging output, Prev: Dealing with handlers that block, Up: Logging Cookbook + +10.7.7 Sending and receiving logging events across a network +------------------------------------------------------------ + +Let’s say you want to send logging events across a network, and handle +them at the receiving end. A simple way of doing this is attaching a +*note SocketHandler: 86f. instance to the root logger at the sending +end: + + import logging, logging.handlers + + rootLogger = logging.getLogger('') + rootLogger.setLevel(logging.DEBUG) + socketHandler = logging.handlers.SocketHandler('localhost', + logging.handlers.DEFAULT_TCP_LOGGING_PORT) + # don't bother with a formatter, since a socket handler sends the event as + # an unformatted pickle + rootLogger.addHandler(socketHandler) + + # Now, we can log to the root logger, or any other logger. First the root... + logging.info('Jackdaws love my big sphinx of quartz.') + + # Now, define a couple of other loggers which might represent areas in your + # application: + + logger1 = logging.getLogger('myapp.area1') + logger2 = logging.getLogger('myapp.area2') + + logger1.debug('Quick zephyrs blow, vexing daft Jim.') + logger1.info('How quickly daft jumping zebras vex.') + logger2.warning('Jail zesty vixen who grabbed pay from quack.') + logger2.error('The five boxing wizards jump quickly.') + +At the receiving end, you can set up a receiver using the *note +socketserver: f0. module. Here is a basic working example: + + import pickle + import logging + import logging.handlers + import socketserver + import struct + + + class LogRecordStreamHandler(socketserver.StreamRequestHandler): + """Handler for a streaming logging request. + + This basically logs the record using whatever logging policy is + configured locally. + """ + + def handle(self): + """ + Handle multiple requests - each expected to be a 4-byte length, + followed by the LogRecord in pickle format. Logs the record + according to whatever policy is configured locally. + """ + while True: + chunk = self.connection.recv(4) + if len(chunk) < 4: + break + slen = struct.unpack('>L', chunk)[0] + chunk = self.connection.recv(slen) + while len(chunk) < slen: + chunk = chunk + self.connection.recv(slen - len(chunk)) + obj = self.unPickle(chunk) + record = logging.makeLogRecord(obj) + self.handleLogRecord(record) + + def unPickle(self, data): + return pickle.loads(data) + + def handleLogRecord(self, record): + # if a name is specified, we use the named logger rather than the one + # implied by the record. + if self.server.logname is not None: + name = self.server.logname + else: + name = record.name + logger = logging.getLogger(name) + # N.B. EVERY record gets logged. This is because Logger.handle + # is normally called AFTER logger-level filtering. If you want + # to do filtering, do it at the client end to save wasting + # cycles and network bandwidth! + logger.handle(record) + + class LogRecordSocketReceiver(socketserver.ThreadingTCPServer): + """ + Simple TCP socket-based logging receiver suitable for testing. + """ + + allow_reuse_address = True + + def __init__(self, host='localhost', + port=logging.handlers.DEFAULT_TCP_LOGGING_PORT, + handler=LogRecordStreamHandler): + socketserver.ThreadingTCPServer.__init__(self, (host, port), handler) + self.abort = 0 + self.timeout = 1 + self.logname = None + + def serve_until_stopped(self): + import select + abort = 0 + while not abort: + rd, wr, ex = select.select([self.socket.fileno()], + [], [], + self.timeout) + if rd: + self.handle_request() + abort = self.abort + + def main(): + logging.basicConfig( + format='%(relativeCreated)5d %(name)-15s %(levelname)-8s %(message)s') + tcpserver = LogRecordSocketReceiver() + print('About to start TCP server...') + tcpserver.serve_until_stopped() + + if __name__ == '__main__': + main() + +First run the server, and then the client. On the client side, nothing +is printed on the console; on the server side, you should see something +like: + + About to start TCP server... + 59 root INFO Jackdaws love my big sphinx of quartz. + 59 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim. + 69 myapp.area1 INFO How quickly daft jumping zebras vex. + 69 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack. + 69 myapp.area2 ERROR The five boxing wizards jump quickly. + +Note that there are some security issues with pickle in some scenarios. +If these affect you, you can use an alternative serialization scheme by +overriding the ‘makePickle()’ method and implementing your alternative +there, as well as adapting the above script to use your alternative +serialization. + + +File: python.info, Node: Adding contextual information to your logging output, Next: Logging to a single file from multiple processes, Prev: Sending and receiving logging events across a network, Up: Logging Cookbook + +10.7.8 Adding contextual information to your logging output +----------------------------------------------------------- + +Sometimes you want logging output to contain contextual information in +addition to the parameters passed to the logging call. For example, in +a networked application, it may be desirable to log client-specific +information in the log (e.g. remote client’s username, or IP address). +Although you could use the `extra' parameter to achieve this, it’s not +always convenient to pass the information in this way. While it might +be tempting to create ‘Logger’ instances on a per-connection basis, this +is not a good idea because these instances are not garbage collected. +While this is not a problem in practice, when the number of ‘Logger’ +instances is dependent on the level of granularity you want to use in +logging an application, it could be hard to manage if the number of +‘Logger’ instances becomes effectively unbounded. + +* Menu: + +* Using LoggerAdapters to impart contextual information:: +* Using Filters to impart contextual information:: + + +File: python.info, Node: Using LoggerAdapters to impart contextual information, Next: Using Filters to impart contextual information, Up: Adding contextual information to your logging output + +10.7.8.1 Using LoggerAdapters to impart contextual information +.............................................................. + +An easy way in which you can pass contextual information to be output +along with logging event information is to use the ‘LoggerAdapter’ +class. This class is designed to look like a ‘Logger’, so that you can +call ‘debug()’, ‘info()’, ‘warning()’, ‘error()’, ‘exception()’, +‘critical()’ and ‘log()’. These methods have the same signatures as +their counterparts in ‘Logger’, so you can use the two types of +instances interchangeably. + +When you create an instance of ‘LoggerAdapter’, you pass it a ‘Logger’ +instance and a dict-like object which contains your contextual +information. When you call one of the logging methods on an instance of +‘LoggerAdapter’, it delegates the call to the underlying instance of +‘Logger’ passed to its constructor, and arranges to pass the contextual +information in the delegated call. Here’s a snippet from the code of +‘LoggerAdapter’: + + def debug(self, msg, /, *args, **kwargs): + """ + Delegate a debug call to the underlying logger, after adding + contextual information from this adapter instance. + """ + msg, kwargs = self.process(msg, kwargs) + self.logger.debug(msg, *args, **kwargs) + +The ‘process()’ method of ‘LoggerAdapter’ is where the contextual +information is added to the logging output. It’s passed the message and +keyword arguments of the logging call, and it passes back (potentially) +modified versions of these to use in the call to the underlying logger. +The default implementation of this method leaves the message alone, but +inserts an ‘extra’ key in the keyword argument whose value is the +dict-like object passed to the constructor. Of course, if you had +passed an ‘extra’ keyword argument in the call to the adapter, it will +be silently overwritten. + +The advantage of using ‘extra’ is that the values in the dict-like +object are merged into the ‘LogRecord’ instance’s __dict__, allowing you +to use customized strings with your ‘Formatter’ instances which know +about the keys of the dict-like object. If you need a different method, +e.g. if you want to prepend or append the contextual information to the +message string, you just need to subclass ‘LoggerAdapter’ and override +‘process()’ to do what you need. Here is a simple example: + + class CustomAdapter(logging.LoggerAdapter): + """ + This example adapter expects the passed in dict-like object to have a + 'connid' key, whose value in brackets is prepended to the log message. + """ + def process(self, msg, kwargs): + return '[%s] %s' % (self.extra['connid'], msg), kwargs + +which you can use like this: + + logger = logging.getLogger(__name__) + adapter = CustomAdapter(logger, {'connid': some_conn_id}) + +Then any events that you log to the adapter will have the value of +‘some_conn_id’ prepended to the log messages. + +* Menu: + +* Using objects other than dicts to pass contextual information:: + + +File: python.info, Node: Using objects other than dicts to pass contextual information, Up: Using LoggerAdapters to impart contextual information + +10.7.8.2 Using objects other than dicts to pass contextual information +...................................................................... + +You don’t need to pass an actual dict to a ‘LoggerAdapter’ - you could +pass an instance of a class which implements ‘__getitem__’ and +‘__iter__’ so that it looks like a dict to logging. This would be +useful if you want to generate values dynamically (whereas the values in +a dict would be constant). + + +File: python.info, Node: Using Filters to impart contextual information, Prev: Using LoggerAdapters to impart contextual information, Up: Adding contextual information to your logging output + +10.7.8.3 Using Filters to impart contextual information +....................................................... + +You can also add contextual information to log output using a +user-defined ‘Filter’. ‘Filter’ instances are allowed to modify the +‘LogRecords’ passed to them, including adding additional attributes +which can then be output using a suitable format string, or if needed a +custom ‘Formatter’. + +For example in a web application, the request being processed (or at +least, the interesting parts of it) can be stored in a threadlocal +(*note threading.local: 1fdc.) variable, and then accessed from a +‘Filter’ to add, say, information from the request - say, the remote IP +address and remote user’s username - to the ‘LogRecord’, using the +attribute names ‘ip’ and ‘user’ as in the ‘LoggerAdapter’ example above. +In that case, the same format string can be used to get similar output +to that shown above. Here’s an example script: + + import logging + from random import choice + + class ContextFilter(logging.Filter): + """ + This is a filter which injects contextual information into the log. + + Rather than use actual contextual information, we just use random + data in this demo. + """ + + USERS = ['jim', 'fred', 'sheila'] + IPS = ['123.231.231.123', '127.0.0.1', '192.168.0.1'] + + def filter(self, record): + + record.ip = choice(ContextFilter.IPS) + record.user = choice(ContextFilter.USERS) + return True + + if __name__ == '__main__': + levels = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL) + logging.basicConfig(level=logging.DEBUG, + format='%(asctime)-15s %(name)-5s %(levelname)-8s IP: %(ip)-15s User: %(user)-8s %(message)s') + a1 = logging.getLogger('a.b.c') + a2 = logging.getLogger('d.e.f') + + f = ContextFilter() + a1.addFilter(f) + a2.addFilter(f) + a1.debug('A debug message') + a1.info('An info message with %s', 'some parameters') + for x in range(10): + lvl = choice(levels) + lvlname = logging.getLevelName(lvl) + a2.log(lvl, 'A message at %s level with %d %s', lvlname, 2, 'parameters') + +which, when run, produces something like: + + 2010-09-06 22:38:15,292 a.b.c DEBUG IP: 123.231.231.123 User: fred A debug message + 2010-09-06 22:38:15,300 a.b.c INFO IP: 192.168.0.1 User: sheila An info message with some parameters + 2010-09-06 22:38:15,300 d.e.f CRITICAL IP: 127.0.0.1 User: sheila A message at CRITICAL level with 2 parameters + 2010-09-06 22:38:15,300 d.e.f ERROR IP: 127.0.0.1 User: jim A message at ERROR level with 2 parameters + 2010-09-06 22:38:15,300 d.e.f DEBUG IP: 127.0.0.1 User: sheila A message at DEBUG level with 2 parameters + 2010-09-06 22:38:15,300 d.e.f ERROR IP: 123.231.231.123 User: fred A message at ERROR level with 2 parameters + 2010-09-06 22:38:15,300 d.e.f CRITICAL IP: 192.168.0.1 User: jim A message at CRITICAL level with 2 parameters + 2010-09-06 22:38:15,300 d.e.f CRITICAL IP: 127.0.0.1 User: sheila A message at CRITICAL level with 2 parameters + 2010-09-06 22:38:15,300 d.e.f DEBUG IP: 192.168.0.1 User: jim A message at DEBUG level with 2 parameters + 2010-09-06 22:38:15,301 d.e.f ERROR IP: 127.0.0.1 User: sheila A message at ERROR level with 2 parameters + 2010-09-06 22:38:15,301 d.e.f DEBUG IP: 123.231.231.123 User: fred A message at DEBUG level with 2 parameters + 2010-09-06 22:38:15,301 d.e.f INFO IP: 123.231.231.123 User: fred A message at INFO level with 2 parameters + + +File: python.info, Node: Logging to a single file from multiple processes, Next: Using file rotation, Prev: Adding contextual information to your logging output, Up: Logging Cookbook + +10.7.9 Logging to a single file from multiple processes +------------------------------------------------------- + +Although logging is thread-safe, and logging to a single file from +multiple threads in a single process `is' supported, logging to a single +file from `multiple processes' is `not' supported, because there is no +standard way to serialize access to a single file across multiple +processes in Python. If you need to log to a single file from multiple +processes, one way of doing this is to have all the processes log to a +‘SocketHandler’, and have a separate process which implements a socket +server which reads from the socket and logs to file. (If you prefer, +you can dedicate one thread in one of the existing processes to perform +this function.) *note This section: 3ec9. documents this approach in +more detail and includes a working socket receiver which can be used as +a starting point for you to adapt in your own applications. + +You could also write your own handler which uses the *note Lock: 2082. +class from the *note multiprocessing: b7. module to serialize access to +the file from your processes. The existing ‘FileHandler’ and subclasses +do not make use of *note multiprocessing: b7. at present, though they +may do so in the future. Note that at present, the *note +multiprocessing: b7. module does not provide working lock functionality +on all platforms (see ‘https://bugs.python.org/issue3770’). + +Alternatively, you can use a ‘Queue’ and a *note QueueHandler: 1e1c. to +send all logging events to one of the processes in your multi-process +application. The following example script demonstrates how you can do +this; in the example a separate listener process listens for events sent +by other processes and logs them according to its own logging +configuration. Although the example only demonstrates one way of doing +it (for example, you may want to use a listener thread rather than a +separate listener process – the implementation would be analogous) it +does allow for completely different logging configurations for the +listener and the other processes in your application, and can be used as +the basis for code meeting your own specific requirements: + + # You'll need these imports in your own code + import logging + import logging.handlers + import multiprocessing + + # Next two import lines for this demo only + from random import choice, random + import time + + # + # Because you'll want to define the logging configurations for listener and workers, the + # listener and worker process functions take a configurer parameter which is a callable + # for configuring logging for that process. These functions are also passed the queue, + # which they use for communication. + # + # In practice, you can configure the listener however you want, but note that in this + # simple example, the listener does not apply level or filter logic to received records. + # In practice, you would probably want to do this logic in the worker processes, to avoid + # sending events which would be filtered out between processes. + # + # The size of the rotated files is made small so you can see the results easily. + def listener_configurer(): + root = logging.getLogger() + h = logging.handlers.RotatingFileHandler('mptest.log', 'a', 300, 10) + f = logging.Formatter('%(asctime)s %(processName)-10s %(name)s %(levelname)-8s %(message)s') + h.setFormatter(f) + root.addHandler(h) + + # This is the listener process top-level loop: wait for logging events + # (LogRecords)on the queue and handle them, quit when you get a None for a + # LogRecord. + def listener_process(queue, configurer): + configurer() + while True: + try: + record = queue.get() + if record is None: # We send this as a sentinel to tell the listener to quit. + break + logger = logging.getLogger(record.name) + logger.handle(record) # No level or filter logic applied - just do it! + except Exception: + import sys, traceback + print('Whoops! Problem:', file=sys.stderr) + traceback.print_exc(file=sys.stderr) + + # Arrays used for random selections in this demo + + LEVELS = [logging.DEBUG, logging.INFO, logging.WARNING, + logging.ERROR, logging.CRITICAL] + + LOGGERS = ['a.b.c', 'd.e.f'] + + MESSAGES = [ + 'Random message #1', + 'Random message #2', + 'Random message #3', + ] + + # The worker configuration is done at the start of the worker process run. + # Note that on Windows you can't rely on fork semantics, so each process + # will run the logging configuration code when it starts. + def worker_configurer(queue): + h = logging.handlers.QueueHandler(queue) # Just the one handler needed + root = logging.getLogger() + root.addHandler(h) + # send all messages, for demo; no other level or filter logic applied. + root.setLevel(logging.DEBUG) + + # This is the worker process top-level loop, which just logs ten events with + # random intervening delays before terminating. + # The print messages are just so you know it's doing something! + def worker_process(queue, configurer): + configurer(queue) + name = multiprocessing.current_process().name + print('Worker started: %s' % name) + for i in range(10): + time.sleep(random()) + logger = logging.getLogger(choice(LOGGERS)) + level = choice(LEVELS) + message = choice(MESSAGES) + logger.log(level, message) + print('Worker finished: %s' % name) + + # Here's where the demo gets orchestrated. Create the queue, create and start + # the listener, create ten workers and start them, wait for them to finish, + # then send a None to the queue to tell the listener to finish. + def main(): + queue = multiprocessing.Queue(-1) + listener = multiprocessing.Process(target=listener_process, + args=(queue, listener_configurer)) + listener.start() + workers = [] + for i in range(10): + worker = multiprocessing.Process(target=worker_process, + args=(queue, worker_configurer)) + workers.append(worker) + worker.start() + for w in workers: + w.join() + queue.put_nowait(None) + listener.join() + + if __name__ == '__main__': + main() + +A variant of the above script keeps the logging in the main process, in +a separate thread: + + import logging + import logging.config + import logging.handlers + from multiprocessing import Process, Queue + import random + import threading + import time + + def logger_thread(q): + while True: + record = q.get() + if record is None: + break + logger = logging.getLogger(record.name) + logger.handle(record) + + + def worker_process(q): + qh = logging.handlers.QueueHandler(q) + root = logging.getLogger() + root.setLevel(logging.DEBUG) + root.addHandler(qh) + levels = [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, + logging.CRITICAL] + loggers = ['foo', 'foo.bar', 'foo.bar.baz', + 'spam', 'spam.ham', 'spam.ham.eggs'] + for i in range(100): + lvl = random.choice(levels) + logger = logging.getLogger(random.choice(loggers)) + logger.log(lvl, 'Message no. %d', i) + + if __name__ == '__main__': + q = Queue() + d = { + 'version': 1, + 'formatters': { + 'detailed': { + 'class': 'logging.Formatter', + 'format': '%(asctime)s %(name)-15s %(levelname)-8s %(processName)-10s %(message)s' + } + }, + 'handlers': { + 'console': { + 'class': 'logging.StreamHandler', + 'level': 'INFO', + }, + 'file': { + 'class': 'logging.FileHandler', + 'filename': 'mplog.log', + 'mode': 'w', + 'formatter': 'detailed', + }, + 'foofile': { + 'class': 'logging.FileHandler', + 'filename': 'mplog-foo.log', + 'mode': 'w', + 'formatter': 'detailed', + }, + 'errors': { + 'class': 'logging.FileHandler', + 'filename': 'mplog-errors.log', + 'mode': 'w', + 'level': 'ERROR', + 'formatter': 'detailed', + }, + }, + 'loggers': { + 'foo': { + 'handlers': ['foofile'] + } + }, + 'root': { + 'level': 'DEBUG', + 'handlers': ['console', 'file', 'errors'] + }, + } + workers = [] + for i in range(5): + wp = Process(target=worker_process, name='worker %d' % (i + 1), args=(q,)) + workers.append(wp) + wp.start() + logging.config.dictConfig(d) + lp = threading.Thread(target=logger_thread, args=(q,)) + lp.start() + # At this point, the main process could do some useful work of its own + # Once it's done that, it can wait for the workers to terminate... + for wp in workers: + wp.join() + # And now tell the logging thread to finish up, too + q.put(None) + lp.join() + +This variant shows how you can e.g. apply configuration for particular +loggers - e.g. the ‘foo’ logger has a special handler which stores all +events in the ‘foo’ subsystem in a file ‘mplog-foo.log’. This will be +used by the logging machinery in the main process (even though the +logging events are generated in the worker processes) to direct the +messages to the appropriate destinations. + +* Menu: + +* Using concurrent.futures.ProcessPoolExecutor: Using concurrent futures ProcessPoolExecutor. + + +File: python.info, Node: Using concurrent futures ProcessPoolExecutor, Up: Logging to a single file from multiple processes + +10.7.9.1 Using concurrent.futures.ProcessPoolExecutor +..................................................... + +If you want to use *note concurrent.futures.ProcessPoolExecutor: 2a4. to +start your worker processes, you need to create the queue slightly +differently. Instead of + + queue = multiprocessing.Queue(-1) + +you should use + + queue = multiprocessing.Manager().Queue(-1) # also works with the examples above + +and you can then replace the worker creation from this: + + workers = [] + for i in range(10): + worker = multiprocessing.Process(target=worker_process, + args=(queue, worker_configurer)) + workers.append(worker) + worker.start() + for w in workers: + w.join() + +to this (remembering to first import *note concurrent.futures: 22.): + + with concurrent.futures.ProcessPoolExecutor(max_workers=10) as executor: + for i in range(10): + executor.submit(worker_process, queue, worker_configurer) + + +File: python.info, Node: Using file rotation, Next: Use of alternative formatting styles, Prev: Logging to a single file from multiple processes, Up: Logging Cookbook + +10.7.10 Using file rotation +--------------------------- + +Sometimes you want to let a log file grow to a certain size, then open a +new file and log to that. You may want to keep a certain number of +these files, and when that many files have been created, rotate the +files so that the number of files and the size of the files both remain +bounded. For this usage pattern, the logging package provides a +‘RotatingFileHandler’: + + import glob + import logging + import logging.handlers + + LOG_FILENAME = 'logging_rotatingfile_example.out' + + # Set up a specific logger with our desired output level + my_logger = logging.getLogger('MyLogger') + my_logger.setLevel(logging.DEBUG) + + # Add the log message handler to the logger + handler = logging.handlers.RotatingFileHandler( + LOG_FILENAME, maxBytes=20, backupCount=5) + + my_logger.addHandler(handler) + + # Log some messages + for i in range(20): + my_logger.debug('i = %d' % i) + + # See what files are created + logfiles = glob.glob('%s*' % LOG_FILENAME) + + for filename in logfiles: + print(filename) + +The result should be 6 separate files, each with part of the log history +for the application: + + logging_rotatingfile_example.out + logging_rotatingfile_example.out.1 + logging_rotatingfile_example.out.2 + logging_rotatingfile_example.out.3 + logging_rotatingfile_example.out.4 + logging_rotatingfile_example.out.5 + +The most current file is always ‘logging_rotatingfile_example.out’, and +each time it reaches the size limit it is renamed with the suffix ‘.1’. +Each of the existing backup files is renamed to increment the suffix +(‘.1’ becomes ‘.2’, etc.) and the ‘.6’ file is erased. + +Obviously this example sets the log length much too small as an extreme +example. You would want to set `maxBytes' to an appropriate value. + + +File: python.info, Node: Use of alternative formatting styles, Next: Customizing LogRecord, Prev: Using file rotation, Up: Logging Cookbook + +10.7.11 Use of alternative formatting styles +-------------------------------------------- + +When logging was added to the Python standard library, the only way of +formatting messages with variable content was to use the %-formatting +method. Since then, Python has gained two new formatting approaches: +*note string.Template: 3eb. (added in Python 2.4) and *note +str.format(): 4da. (added in Python 2.6). + +Logging (as of 3.2) provides improved support for these two additional +formatting styles. The ‘Formatter’ class been enhanced to take an +additional, optional keyword parameter named ‘style’. This defaults to +‘'%'’, but other possible values are ‘'{'’ and ‘'$'’, which correspond +to the other two formatting styles. Backwards compatibility is +maintained by default (as you would expect), but by explicitly +specifying a style parameter, you get the ability to specify format +strings which work with *note str.format(): 4da. or *note +string.Template: 3eb. Here’s an example console session to show the +possibilities: + + >>> import logging + >>> root = logging.getLogger() + >>> root.setLevel(logging.DEBUG) + >>> handler = logging.StreamHandler() + >>> bf = logging.Formatter('{asctime} {name} {levelname:8s} {message}', + ... style='{') + >>> handler.setFormatter(bf) + >>> root.addHandler(handler) + >>> logger = logging.getLogger('foo.bar') + >>> logger.debug('This is a DEBUG message') + 2010-10-28 15:11:55,341 foo.bar DEBUG This is a DEBUG message + >>> logger.critical('This is a CRITICAL message') + 2010-10-28 15:12:11,526 foo.bar CRITICAL This is a CRITICAL message + >>> df = logging.Formatter('$asctime $name ${levelname} $message', + ... style='$') + >>> handler.setFormatter(df) + >>> logger.debug('This is a DEBUG message') + 2010-10-28 15:13:06,924 foo.bar DEBUG This is a DEBUG message + >>> logger.critical('This is a CRITICAL message') + 2010-10-28 15:13:11,494 foo.bar CRITICAL This is a CRITICAL message + >>> + +Note that the formatting of logging messages for final output to logs is +completely independent of how an individual logging message is +constructed. That can still use %-formatting, as shown here: + + >>> logger.error('This is an%s %s %s', 'other,', 'ERROR,', 'message') + 2010-10-28 15:19:29,833 foo.bar ERROR This is another, ERROR, message + >>> + +Logging calls (‘logger.debug()’, ‘logger.info()’ etc.) only take +positional parameters for the actual logging message itself, with +keyword parameters used only for determining options for how to handle +the actual logging call (e.g. the ‘exc_info’ keyword parameter to +indicate that traceback information should be logged, or the ‘extra’ +keyword parameter to indicate additional contextual information to be +added to the log). So you cannot directly make logging calls using +*note str.format(): 4da. or *note string.Template: 3eb. syntax, because +internally the logging package uses %-formatting to merge the format +string and the variable arguments. There would be no changing this +while preserving backward compatibility, since all logging calls which +are out there in existing code will be using %-format strings. + +There is, however, a way that you can use {}- and $- formatting to +construct your individual log messages. Recall that for a message you +can use an arbitrary object as a message format string, and that the +logging package will call ‘str()’ on that object to get the actual +format string. Consider the following two classes: + + class BraceMessage: + def __init__(self, fmt, /, *args, **kwargs): + self.fmt = fmt + self.args = args + self.kwargs = kwargs + + def __str__(self): + return self.fmt.format(*self.args, **self.kwargs) + + class DollarMessage: + def __init__(self, fmt, /, **kwargs): + self.fmt = fmt + self.kwargs = kwargs + + def __str__(self): + from string import Template + return Template(self.fmt).substitute(**self.kwargs) + +Either of these can be used in place of a format string, to allow {}- or +$-formatting to be used to build the actual “message” part which appears +in the formatted log output in place of “%(message)s” or “{message}” or +“$message”. It’s a little unwieldy to use the class names whenever you +want to log something, but it’s quite palatable if you use an alias such +as __ (double underscore — not to be confused with _, the single +underscore used as a synonym/alias for *note gettext.gettext(): dcd. or +its brethren). + +The above classes are not included in Python, though they’re easy enough +to copy and paste into your own code. They can be used as follows +(assuming that they’re declared in a module called ‘wherever’): + + >>> from wherever import BraceMessage as __ + >>> print(__('Message with {0} {name}', 2, name='placeholders')) + Message with 2 placeholders + >>> class Point: pass + ... + >>> p = Point() + >>> p.x = 0.5 + >>> p.y = 0.5 + >>> print(__('Message with coordinates: ({point.x:.2f}, {point.y:.2f})', + ... point=p)) + Message with coordinates: (0.50, 0.50) + >>> from wherever import DollarMessage as __ + >>> print(__('Message with $num $what', num=2, what='placeholders')) + Message with 2 placeholders + >>> + +While the above examples use ‘print()’ to show how the formatting works, +you would of course use ‘logger.debug()’ or similar to actually log +using this approach. + +One thing to note is that you pay no significant performance penalty +with this approach: the actual formatting happens not when you make the +logging call, but when (and if) the logged message is actually about to +be output to a log by a handler. So the only slightly unusual thing +which might trip you up is that the parentheses go around the format +string and the arguments, not just the format string. That’s because +the __ notation is just syntax sugar for a constructor call to one of +the XXXMessage classes. + +If you prefer, you can use a ‘LoggerAdapter’ to achieve a similar effect +to the above, as in the following example: + + import logging + + class Message: + def __init__(self, fmt, args): + self.fmt = fmt + self.args = args + + def __str__(self): + return self.fmt.format(*self.args) + + class StyleAdapter(logging.LoggerAdapter): + def __init__(self, logger, extra=None): + super(StyleAdapter, self).__init__(logger, extra or {}) + + def log(self, level, msg, /, *args, **kwargs): + if self.isEnabledFor(level): + msg, kwargs = self.process(msg, kwargs) + self.logger._log(level, Message(msg, args), (), **kwargs) + + logger = StyleAdapter(logging.getLogger(__name__)) + + def main(): + logger.debug('Hello, {}', 'world!') + + if __name__ == '__main__': + logging.basicConfig(level=logging.DEBUG) + main() + +The above script should log the message ‘Hello, world!’ when run with +Python 3.2 or later. + + +File: python.info, Node: Customizing LogRecord, Next: Subclassing QueueHandler - a ZeroMQ example, Prev: Use of alternative formatting styles, Up: Logging Cookbook + +10.7.12 Customizing ‘LogRecord’ +------------------------------- + +Every logging event is represented by a *note LogRecord: 906. instance. +When an event is logged and not filtered out by a logger’s level, a +*note LogRecord: 906. is created, populated with information about the +event and then passed to the handlers for that logger (and its +ancestors, up to and including the logger where further propagation up +the hierarchy is disabled). Before Python 3.2, there were only two +places where this creation was done: + + * *note Logger.makeRecord(): 1d6e, which is called in the normal + process of logging an event. This invoked *note LogRecord: 906. + directly to create an instance. + + * *note makeLogRecord(): 1d93, which is called with a dictionary + containing attributes to be added to the LogRecord. This is + typically invoked when a suitable dictionary has been received over + the network (e.g. in pickle form via a *note SocketHandler: 86f, + or in JSON form via an *note HTTPHandler: 711.). + +This has usually meant that if you need to do anything special with a +*note LogRecord: 906, you’ve had to do one of the following. + + * Create your own *note Logger: 3a7. subclass, which overrides *note + Logger.makeRecord(): 1d6e, and set it using *note setLoggerClass(): + 1da0. before any loggers that you care about are instantiated. + + * Add a *note Filter: b77. to a logger or handler, which does the + necessary special manipulation you need when its *note filter(): + 1d8e. method is called. + +The first approach would be a little unwieldy in the scenario where +(say) several different libraries wanted to do different things. Each +would attempt to set its own *note Logger: 3a7. subclass, and the one +which did this last would win. + +The second approach works reasonably well for many cases, but does not +allow you to e.g. use a specialized subclass of *note LogRecord: 906. +Library developers can set a suitable filter on their loggers, but they +would have to remember to do this every time they introduced a new +logger (which they would do simply by adding new packages or modules and +doing + + logger = logging.getLogger(__name__) + +at module level). It’s probably one too many things to think about. +Developers could also add the filter to a *note NullHandler: c1c. +attached to their top-level logger, but this would not be invoked if an +application developer attached a handler to a lower-level library logger +— so output from that handler would not reflect the intentions of the +library developer. + +In Python 3.2 and later, *note LogRecord: 906. creation is done through +a factory, which you can specify. The factory is just a callable you +can set with *note setLogRecordFactory(): 1d96, and interrogate with +*note getLogRecordFactory(): 1d95. The factory is invoked with the same +signature as the *note LogRecord: 906. constructor, as *note LogRecord: +906. is the default setting for the factory. + +This approach allows a custom factory to control all aspects of +LogRecord creation. For example, you could return a subclass, or just +add some additional attributes to the record once created, using a +pattern similar to this: + + old_factory = logging.getLogRecordFactory() + + def record_factory(*args, **kwargs): + record = old_factory(*args, **kwargs) + record.custom_attribute = 0xdecafbad + return record + + logging.setLogRecordFactory(record_factory) + +This pattern allows different libraries to chain factories together, and +as long as they don’t overwrite each other’s attributes or +unintentionally overwrite the attributes provided as standard, there +should be no surprises. However, it should be borne in mind that each +link in the chain adds run-time overhead to all logging operations, and +the technique should only be used when the use of a *note Filter: b77. +does not provide the desired result. + + +File: python.info, Node: Subclassing QueueHandler - a ZeroMQ example, Next: Subclassing QueueListener - a ZeroMQ example, Prev: Customizing LogRecord, Up: Logging Cookbook + +10.7.13 Subclassing QueueHandler - a ZeroMQ example +--------------------------------------------------- + +You can use a ‘QueueHandler’ subclass to send messages to other kinds of +queues, for example a ZeroMQ ‘publish’ socket. In the example below,the +socket is created separately and passed to the handler (as its ‘queue’): + + import zmq # using pyzmq, the Python binding for ZeroMQ + import json # for serializing records portably + + ctx = zmq.Context() + sock = zmq.Socket(ctx, zmq.PUB) # or zmq.PUSH, or other suitable value + sock.bind('tcp://*:5556') # or wherever + + class ZeroMQSocketHandler(QueueHandler): + def enqueue(self, record): + self.queue.send_json(record.__dict__) + + + handler = ZeroMQSocketHandler(sock) + +Of course there are other ways of organizing this, for example passing +in the data needed by the handler to create the socket: + + class ZeroMQSocketHandler(QueueHandler): + def __init__(self, uri, socktype=zmq.PUB, ctx=None): + self.ctx = ctx or zmq.Context() + socket = zmq.Socket(self.ctx, socktype) + socket.bind(uri) + super().__init__(socket) + + def enqueue(self, record): + self.queue.send_json(record.__dict__) + + def close(self): + self.queue.close() + + +File: python.info, Node: Subclassing QueueListener - a ZeroMQ example, Next: An example dictionary-based configuration, Prev: Subclassing QueueHandler - a ZeroMQ example, Up: Logging Cookbook + +10.7.14 Subclassing QueueListener - a ZeroMQ example +---------------------------------------------------- + +You can also subclass ‘QueueListener’ to get messages from other kinds +of queues, for example a ZeroMQ ‘subscribe’ socket. Here’s an example: + + class ZeroMQSocketListener(QueueListener): + def __init__(self, uri, /, *handlers, **kwargs): + self.ctx = kwargs.get('ctx') or zmq.Context() + socket = zmq.Socket(self.ctx, zmq.SUB) + socket.setsockopt_string(zmq.SUBSCRIBE, '') # subscribe to everything + socket.connect(uri) + super().__init__(socket, *handlers, **kwargs) + + def dequeue(self): + msg = self.queue.recv_json() + return logging.makeLogRecord(msg) + +See also +........ + +Module *note logging: aa. + + API reference for the logging module. + +Module *note logging.config: ab. + + Configuration API for the logging module. + +Module *note logging.handlers: ac. + + Useful handlers included with the logging module. + +*note A basic logging tutorial: b72. + +*note A more advanced logging tutorial: b73. + + +File: python.info, Node: An example dictionary-based configuration, Next: Using a rotator and namer to customize log rotation processing, Prev: Subclassing QueueListener - a ZeroMQ example, Up: Logging Cookbook + +10.7.15 An example dictionary-based configuration +------------------------------------------------- + +Below is an example of a logging configuration dictionary - it’s taken +from the documentation on the Django project(1). This dictionary is +passed to *note dictConfig(): b2b. to put the configuration into effect: + + LOGGING = { + 'version': 1, + 'disable_existing_loggers': True, + 'formatters': { + 'verbose': { + 'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s' + }, + 'simple': { + 'format': '%(levelname)s %(message)s' + }, + }, + 'filters': { + 'special': { + '()': 'project.logging.SpecialFilter', + 'foo': 'bar', + } + }, + 'handlers': { + 'null': { + 'level':'DEBUG', + 'class':'django.utils.log.NullHandler', + }, + 'console':{ + 'level':'DEBUG', + 'class':'logging.StreamHandler', + 'formatter': 'simple' + }, + 'mail_admins': { + 'level': 'ERROR', + 'class': 'django.utils.log.AdminEmailHandler', + 'filters': ['special'] + } + }, + 'loggers': { + 'django': { + 'handlers':['null'], + 'propagate': True, + 'level':'INFO', + }, + 'django.request': { + 'handlers': ['mail_admins'], + 'level': 'ERROR', + 'propagate': False, + }, + 'myproject.custom': { + 'handlers': ['console', 'mail_admins'], + 'level': 'INFO', + 'filters': ['special'] + } + } + } + +For more information about this configuration, you can see the relevant +section(2) of the Django documentation. + + ---------- Footnotes ---------- + + (1) +https://docs.djangoproject.com/en/stable/topics/logging/#configuring-logging + + (2) +https://docs.djangoproject.com/en/stable/topics/logging/#configuring-logging + + +File: python.info, Node: Using a rotator and namer to customize log rotation processing, Next: A more elaborate multiprocessing example, Prev: An example dictionary-based configuration, Up: Logging Cookbook + +10.7.16 Using a rotator and namer to customize log rotation processing +---------------------------------------------------------------------- + +An example of how you can define a namer and rotator is given in the +following snippet, which shows zlib-based compression of the log file: + + def namer(name): + return name + ".gz" + + def rotator(source, dest): + with open(source, "rb") as sf: + data = sf.read() + compressed = zlib.compress(data, 9) + with open(dest, "wb") as df: + df.write(compressed) + os.remove(source) + + rh = logging.handlers.RotatingFileHandler(...) + rh.rotator = rotator + rh.namer = namer + +These are not “true” .gz files, as they are bare compressed data, with +no “container” such as you’d find in an actual gzip file. This snippet +is just for illustration purposes. + + +File: python.info, Node: A more elaborate multiprocessing example, Next: Inserting a BOM into messages sent to a SysLogHandler, Prev: Using a rotator and namer to customize log rotation processing, Up: Logging Cookbook + +10.7.17 A more elaborate multiprocessing example +------------------------------------------------ + +The following working example shows how logging can be used with +multiprocessing using configuration files. The configurations are +fairly simple, but serve to illustrate how more complex ones could be +implemented in a real multiprocessing scenario. + +In the example, the main process spawns a listener process and some +worker processes. Each of the main process, the listener and the +workers have three separate configurations (the workers all share the +same configuration). We can see logging in the main process, how the +workers log to a QueueHandler and how the listener implements a +QueueListener and a more complex logging configuration, and arranges to +dispatch events received via the queue to the handlers specified in the +configuration. Note that these configurations are purely illustrative, +but you should be able to adapt this example to your own scenario. + +Here’s the script - the docstrings and the comments hopefully explain +how it works: + + import logging + import logging.config + import logging.handlers + from multiprocessing import Process, Queue, Event, current_process + import os + import random + import time + + class MyHandler: + """ + A simple handler for logging events. It runs in the listener process and + dispatches events to loggers based on the name in the received record, + which then get dispatched, by the logging system, to the handlers + configured for those loggers. + """ + + def handle(self, record): + if record.name == "root": + logger = logging.getLogger() + else: + logger = logging.getLogger(record.name) + + if logger.isEnabledFor(record.levelno): + # The process name is transformed just to show that it's the listener + # doing the logging to files and console + record.processName = '%s (for %s)' % (current_process().name, record.processName) + logger.handle(record) + + def listener_process(q, stop_event, config): + """ + This could be done in the main process, but is just done in a separate + process for illustrative purposes. + + This initialises logging according to the specified configuration, + starts the listener and waits for the main process to signal completion + via the event. The listener is then stopped, and the process exits. + """ + logging.config.dictConfig(config) + listener = logging.handlers.QueueListener(q, MyHandler()) + listener.start() + if os.name == 'posix': + # On POSIX, the setup logger will have been configured in the + # parent process, but should have been disabled following the + # dictConfig call. + # On Windows, since fork isn't used, the setup logger won't + # exist in the child, so it would be created and the message + # would appear - hence the "if posix" clause. + logger = logging.getLogger('setup') + logger.critical('Should not appear, because of disabled logger ...') + stop_event.wait() + listener.stop() + + def worker_process(config): + """ + A number of these are spawned for the purpose of illustration. In + practice, they could be a heterogeneous bunch of processes rather than + ones which are identical to each other. + + This initialises logging according to the specified configuration, + and logs a hundred messages with random levels to randomly selected + loggers. + + A small sleep is added to allow other processes a chance to run. This + is not strictly needed, but it mixes the output from the different + processes a bit more than if it's left out. + """ + logging.config.dictConfig(config) + levels = [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, + logging.CRITICAL] + loggers = ['foo', 'foo.bar', 'foo.bar.baz', + 'spam', 'spam.ham', 'spam.ham.eggs'] + if os.name == 'posix': + # On POSIX, the setup logger will have been configured in the + # parent process, but should have been disabled following the + # dictConfig call. + # On Windows, since fork isn't used, the setup logger won't + # exist in the child, so it would be created and the message + # would appear - hence the "if posix" clause. + logger = logging.getLogger('setup') + logger.critical('Should not appear, because of disabled logger ...') + for i in range(100): + lvl = random.choice(levels) + logger = logging.getLogger(random.choice(loggers)) + logger.log(lvl, 'Message no. %d', i) + time.sleep(0.01) + + def main(): + q = Queue() + # The main process gets a simple configuration which prints to the console. + config_initial = { + 'version': 1, + 'handlers': { + 'console': { + 'class': 'logging.StreamHandler', + 'level': 'INFO' + } + }, + 'root': { + 'handlers': ['console'], + 'level': 'DEBUG' + } + } + # The worker process configuration is just a QueueHandler attached to the + # root logger, which allows all messages to be sent to the queue. + # We disable existing loggers to disable the "setup" logger used in the + # parent process. This is needed on POSIX because the logger will + # be there in the child following a fork(). + config_worker = { + 'version': 1, + 'disable_existing_loggers': True, + 'handlers': { + 'queue': { + 'class': 'logging.handlers.QueueHandler', + 'queue': q + } + }, + 'root': { + 'handlers': ['queue'], + 'level': 'DEBUG' + } + } + # The listener process configuration shows that the full flexibility of + # logging configuration is available to dispatch events to handlers however + # you want. + # We disable existing loggers to disable the "setup" logger used in the + # parent process. This is needed on POSIX because the logger will + # be there in the child following a fork(). + config_listener = { + 'version': 1, + 'disable_existing_loggers': True, + 'formatters': { + 'detailed': { + 'class': 'logging.Formatter', + 'format': '%(asctime)s %(name)-15s %(levelname)-8s %(processName)-10s %(message)s' + }, + 'simple': { + 'class': 'logging.Formatter', + 'format': '%(name)-15s %(levelname)-8s %(processName)-10s %(message)s' + } + }, + 'handlers': { + 'console': { + 'class': 'logging.StreamHandler', + 'formatter': 'simple', + 'level': 'INFO' + }, + 'file': { + 'class': 'logging.FileHandler', + 'filename': 'mplog.log', + 'mode': 'w', + 'formatter': 'detailed' + }, + 'foofile': { + 'class': 'logging.FileHandler', + 'filename': 'mplog-foo.log', + 'mode': 'w', + 'formatter': 'detailed' + }, + 'errors': { + 'class': 'logging.FileHandler', + 'filename': 'mplog-errors.log', + 'mode': 'w', + 'formatter': 'detailed', + 'level': 'ERROR' + } + }, + 'loggers': { + 'foo': { + 'handlers': ['foofile'] + } + }, + 'root': { + 'handlers': ['console', 'file', 'errors'], + 'level': 'DEBUG' + } + } + # Log some initial events, just to show that logging in the parent works + # normally. + logging.config.dictConfig(config_initial) + logger = logging.getLogger('setup') + logger.info('About to create workers ...') + workers = [] + for i in range(5): + wp = Process(target=worker_process, name='worker %d' % (i + 1), + args=(config_worker,)) + workers.append(wp) + wp.start() + logger.info('Started worker: %s', wp.name) + logger.info('About to create listener ...') + stop_event = Event() + lp = Process(target=listener_process, name='listener', + args=(q, stop_event, config_listener)) + lp.start() + logger.info('Started listener') + # We now hang around for the workers to finish their work. + for wp in workers: + wp.join() + # Workers all done, listening can now stop. + # Logging in the parent still works normally. + logger.info('Telling listener to stop ...') + stop_event.set() + lp.join() + logger.info('All done.') + + if __name__ == '__main__': + main() + + +File: python.info, Node: Inserting a BOM into messages sent to a SysLogHandler, Next: Implementing structured logging, Prev: A more elaborate multiprocessing example, Up: Logging Cookbook + +10.7.18 Inserting a BOM into messages sent to a SysLogHandler +------------------------------------------------------------- + +RFC 5424(1) requires that a Unicode message be sent to a syslog daemon +as a set of bytes which have the following structure: an optional +pure-ASCII component, followed by a UTF-8 Byte Order Mark (BOM), +followed by Unicode encoded using UTF-8. (See the relevant section of +the specification(2).) + +In Python 3.1, code was added to *note SysLogHandler: a1e. to insert a +BOM into the message, but unfortunately, it was implemented incorrectly, +with the BOM appearing at the beginning of the message and hence not +allowing any pure-ASCII component to appear before it. + +As this behaviour is broken, the incorrect BOM insertion code is being +removed from Python 3.2.4 and later. However, it is not being replaced, +and if you want to produce RFC 5424(3)-compliant messages which include +a BOM, an optional pure-ASCII sequence before it and arbitrary Unicode +after it, encoded using UTF-8, then you need to do the following: + + 1. Attach a *note Formatter: 1d61. instance to your *note + SysLogHandler: a1e. instance, with a format string such as: + + 'ASCII section\ufeffUnicode section' + + The Unicode code point U+FEFF, when encoded using UTF-8, will be + encoded as a UTF-8 BOM – the byte-string ‘b'\xef\xbb\xbf'’. + + 2. Replace the ASCII section with whatever placeholders you like, but + make sure that the data that appears in there after substitution is + always ASCII (that way, it will remain unchanged after UTF-8 + encoding). + + 3. Replace the Unicode section with whatever placeholders you like; if + the data which appears there after substitution contains characters + outside the ASCII range, that’s fine – it will be encoded using + UTF-8. + +The formatted message `will' be encoded using UTF-8 encoding by +‘SysLogHandler’. If you follow the above rules, you should be able to +produce RFC 5424(4)-compliant messages. If you don’t, logging may not +complain, but your messages will not be RFC 5424-compliant, and your +syslog daemon may complain. + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc5424.html + + (2) https://tools.ietf.org/html/rfc5424.html#section-6 + + (3) https://tools.ietf.org/html/rfc5424.html + + (4) https://tools.ietf.org/html/rfc5424.html + + +File: python.info, Node: Implementing structured logging, Next: Customizing handlers with dictConfig, Prev: Inserting a BOM into messages sent to a SysLogHandler, Up: Logging Cookbook + +10.7.19 Implementing structured logging +--------------------------------------- + +Although most logging messages are intended for reading by humans, and +thus not readily machine-parseable, there might be circumstances where +you want to output messages in a structured format which `is' capable of +being parsed by a program (without needing complex regular expressions +to parse the log message). This is straightforward to achieve using the +logging package. There are a number of ways in which this could be +achieved, but the following is a simple approach which uses JSON to +serialise the event in a machine-parseable manner: + + import json + import logging + + class StructuredMessage: + def __init__(self, message, /, **kwargs): + self.message = message + self.kwargs = kwargs + + def __str__(self): + return '%s >>> %s' % (self.message, json.dumps(self.kwargs)) + + _ = StructuredMessage # optional, to improve readability + + logging.basicConfig(level=logging.INFO, format='%(message)s') + logging.info(_('message 1', foo='bar', bar='baz', num=123, fnum=123.456)) + +If the above script is run, it prints: + + message 1 >>> {"fnum": 123.456, "num": 123, "bar": "baz", "foo": "bar"} + +Note that the order of items might be different according to the version +of Python used. + +If you need more specialised processing, you can use a custom JSON +encoder, as in the following complete example: + + from __future__ import unicode_literals + + import json + import logging + + # This next bit is to ensure the script runs unchanged on 2.x and 3.x + try: + unicode + except NameError: + unicode = str + + class Encoder(json.JSONEncoder): + def default(self, o): + if isinstance(o, set): + return tuple(o) + elif isinstance(o, unicode): + return o.encode('unicode_escape').decode('ascii') + return super(Encoder, self).default(o) + + class StructuredMessage: + def __init__(self, message, /, **kwargs): + self.message = message + self.kwargs = kwargs + + def __str__(self): + s = Encoder().encode(self.kwargs) + return '%s >>> %s' % (self.message, s) + + _ = StructuredMessage # optional, to improve readability + + def main(): + logging.basicConfig(level=logging.INFO, format='%(message)s') + logging.info(_('message 1', set_value={1, 2, 3}, snowman='\u2603')) + + if __name__ == '__main__': + main() + +When the above script is run, it prints: + + message 1 >>> {"snowman": "\u2603", "set_value": [1, 2, 3]} + +Note that the order of items might be different according to the version +of Python used. + + +File: python.info, Node: Customizing handlers with dictConfig, Next: Using particular formatting styles throughout your application, Prev: Implementing structured logging, Up: Logging Cookbook + +10.7.20 Customizing handlers with ‘dictConfig()’ +------------------------------------------------ + +There are times when you want to customize logging handlers in +particular ways, and if you use *note dictConfig(): b2b. you may be able +to do this without subclassing. As an example, consider that you may +want to set the ownership of a log file. On POSIX, this is easily done +using *note shutil.chown(): a76, but the file handlers in the stdlib +don’t offer built-in support. You can customize handler creation using +a plain function such as: + + def owned_file_handler(filename, mode='a', encoding=None, owner=None): + if owner: + if not os.path.exists(filename): + open(filename, 'a').close() + shutil.chown(filename, *owner) + return logging.FileHandler(filename, mode, encoding) + +You can then specify, in a logging configuration passed to *note +dictConfig(): b2b, that a logging handler be created by calling this +function: + + LOGGING = { + 'version': 1, + 'disable_existing_loggers': False, + 'formatters': { + 'default': { + 'format': '%(asctime)s %(levelname)s %(name)s %(message)s' + }, + }, + 'handlers': { + 'file':{ + # The values below are popped from this dictionary and + # used to create the handler, set the handler's level and + # its formatter. + '()': owned_file_handler, + 'level':'DEBUG', + 'formatter': 'default', + # The values below are passed to the handler creator callable + # as keyword arguments. + 'owner': ['pulse', 'pulse'], + 'filename': 'chowntest.log', + 'mode': 'w', + 'encoding': 'utf-8', + }, + }, + 'root': { + 'handlers': ['file'], + 'level': 'DEBUG', + }, + } + +In this example I am setting the ownership using the ‘pulse’ user and +group, just for the purposes of illustration. Putting it together into +a working script, ‘chowntest.py’: + + import logging, logging.config, os, shutil + + def owned_file_handler(filename, mode='a', encoding=None, owner=None): + if owner: + if not os.path.exists(filename): + open(filename, 'a').close() + shutil.chown(filename, *owner) + return logging.FileHandler(filename, mode, encoding) + + LOGGING = { + 'version': 1, + 'disable_existing_loggers': False, + 'formatters': { + 'default': { + 'format': '%(asctime)s %(levelname)s %(name)s %(message)s' + }, + }, + 'handlers': { + 'file':{ + # The values below are popped from this dictionary and + # used to create the handler, set the handler's level and + # its formatter. + '()': owned_file_handler, + 'level':'DEBUG', + 'formatter': 'default', + # The values below are passed to the handler creator callable + # as keyword arguments. + 'owner': ['pulse', 'pulse'], + 'filename': 'chowntest.log', + 'mode': 'w', + 'encoding': 'utf-8', + }, + }, + 'root': { + 'handlers': ['file'], + 'level': 'DEBUG', + }, + } + + logging.config.dictConfig(LOGGING) + logger = logging.getLogger('mylogger') + logger.debug('A debug message') + +To run this, you will probably need to run as ‘root’: + + $ sudo python3.3 chowntest.py + $ cat chowntest.log + 2013-11-05 09:34:51,128 DEBUG mylogger A debug message + $ ls -l chowntest.log + -rw-r--r-- 1 pulse pulse 55 2013-11-05 09:34 chowntest.log + +Note that this example uses Python 3.3 because that’s where *note +shutil.chown(): a76. makes an appearance. This approach should work +with any Python version that supports *note dictConfig(): b2b. - namely, +Python 2.7, 3.2 or later. With pre-3.3 versions, you would need to +implement the actual ownership change using e.g. *note os.chown(): a34. + +In practice, the handler-creating function may be in a utility module +somewhere in your project. Instead of the line in the configuration: + + '()': owned_file_handler, + +you could use e.g.: + + '()': 'ext://project.util.owned_file_handler', + +where ‘project.util’ can be replaced with the actual name of the package +where the function resides. In the above working script, using +‘'ext://__main__.owned_file_handler'’ should work. Here, the actual +callable is resolved by *note dictConfig(): b2b. from the ‘ext://’ +specification. + +This example hopefully also points the way to how you could implement +other types of file change - e.g. setting specific POSIX permission +bits - in the same way, using *note os.chmod(): a33. + +Of course, the approach could also be extended to types of handler other +than a *note FileHandler: 1dc8. - for example, one of the rotating file +handlers, or a different type of handler altogether. + + +File: python.info, Node: Using particular formatting styles throughout your application, Next: Configuring filters with dictConfig, Prev: Customizing handlers with dictConfig, Up: Logging Cookbook + +10.7.21 Using particular formatting styles throughout your application +---------------------------------------------------------------------- + +In Python 3.2, the *note Formatter: 1d61. gained a ‘style’ keyword +parameter which, while defaulting to ‘%’ for backward compatibility, +allowed the specification of ‘{’ or ‘$’ to support the formatting +approaches supported by *note str.format(): 4da. and *note +string.Template: 3eb. Note that this governs the formatting of logging +messages for final output to logs, and is completely orthogonal to how +an individual logging message is constructed. + +Logging calls (*note debug(): 710, *note info(): 1d63. etc.) only take +positional parameters for the actual logging message itself, with +keyword parameters used only for determining options for how to handle +the logging call (e.g. the ‘exc_info’ keyword parameter to indicate +that traceback information should be logged, or the ‘extra’ keyword +parameter to indicate additional contextual information to be added to +the log). So you cannot directly make logging calls using *note +str.format(): 4da. or *note string.Template: 3eb. syntax, because +internally the logging package uses %-formatting to merge the format +string and the variable arguments. There would no changing this while +preserving backward compatibility, since all logging calls which are out +there in existing code will be using %-format strings. + +There have been suggestions to associate format styles with specific +loggers, but that approach also runs into backward compatibility +problems because any existing code could be using a given logger name +and using %-formatting. + +For logging to work interoperably between any third-party libraries and +your code, decisions about formatting need to be made at the level of +the individual logging call. This opens up a couple of ways in which +alternative formatting styles can be accommodated. + +* Menu: + +* Using LogRecord factories:: +* Using custom message objects:: + + +File: python.info, Node: Using LogRecord factories, Next: Using custom message objects, Up: Using particular formatting styles throughout your application + +10.7.21.1 Using LogRecord factories +................................... + +In Python 3.2, along with the *note Formatter: 1d61. changes mentioned +above, the logging package gained the ability to allow users to set +their own *note LogRecord: 906. subclasses, using the *note +setLogRecordFactory(): 1d96. function. You can use this to set your own +subclass of *note LogRecord: 906, which does the Right Thing by +overriding the *note getMessage(): 1d94. method. The base class +implementation of this method is where the ‘msg % args’ formatting +happens, and where you can substitute your alternate formatting; +however, you should be careful to support all formatting styles and +allow %-formatting as the default, to ensure interoperability with other +code. Care should also be taken to call ‘str(self.msg)’, just as the +base implementation does. + +Refer to the reference documentation on *note setLogRecordFactory(): +1d96. and *note LogRecord: 906. for more information. + + +File: python.info, Node: Using custom message objects, Prev: Using LogRecord factories, Up: Using particular formatting styles throughout your application + +10.7.21.2 Using custom message objects +...................................... + +There is another, perhaps simpler way that you can use {}- and $- +formatting to construct your individual log messages. You may recall +(from *note Using arbitrary objects as messages: 1d98.) that when +logging you can use an arbitrary object as a message format string, and +that the logging package will call *note str(): 330. on that object to +get the actual format string. Consider the following two classes: + + class BraceMessage: + def __init__(self, fmt, /, *args, **kwargs): + self.fmt = fmt + self.args = args + self.kwargs = kwargs + + def __str__(self): + return self.fmt.format(*self.args, **self.kwargs) + + class DollarMessage: + def __init__(self, fmt, /, **kwargs): + self.fmt = fmt + self.kwargs = kwargs + + def __str__(self): + from string import Template + return Template(self.fmt).substitute(**self.kwargs) + +Either of these can be used in place of a format string, to allow {}- or +$-formatting to be used to build the actual “message” part which appears +in the formatted log output in place of “%(message)s” or “{message}” or +“$message”. If you find it a little unwieldy to use the class names +whenever you want to log something, you can make it more palatable if +you use an alias such as ‘M’ or ‘_’ for the message (or perhaps ‘__’, if +you are using ‘_’ for localization). + +Examples of this approach are given below. Firstly, formatting with +*note str.format(): 4da.: + + >>> __ = BraceMessage + >>> print(__('Message with {0} {1}', 2, 'placeholders')) + Message with 2 placeholders + >>> class Point: pass + ... + >>> p = Point() + >>> p.x = 0.5 + >>> p.y = 0.5 + >>> print(__('Message with coordinates: ({point.x:.2f}, {point.y:.2f})', point=p)) + Message with coordinates: (0.50, 0.50) + +Secondly, formatting with *note string.Template: 3eb.: + + >>> __ = DollarMessage + >>> print(__('Message with $num $what', num=2, what='placeholders')) + Message with 2 placeholders + >>> + +One thing to note is that you pay no significant performance penalty +with this approach: the actual formatting happens not when you make the +logging call, but when (and if) the logged message is actually about to +be output to a log by a handler. So the only slightly unusual thing +which might trip you up is that the parentheses go around the format +string and the arguments, not just the format string. That’s because +the __ notation is just syntax sugar for a constructor call to one of +the ‘XXXMessage’ classes shown above. + + +File: python.info, Node: Configuring filters with dictConfig, Next: Customized exception formatting, Prev: Using particular formatting styles throughout your application, Up: Logging Cookbook + +10.7.22 Configuring filters with ‘dictConfig()’ +----------------------------------------------- + +You `can' configure filters using *note dictConfig(): b2b, though it +might not be obvious at first glance how to do it (hence this recipe). +Since *note Filter: b77. is the only filter class included in the +standard library, and it is unlikely to cater to many requirements (it’s +only there as a base class), you will typically need to define your own +*note Filter: b77. subclass with an overridden *note filter(): 1d8e. +method. To do this, specify the ‘()’ key in the configuration +dictionary for the filter, specifying a callable which will be used to +create the filter (a class is the most obvious, but you can provide any +callable which returns a *note Filter: b77. instance). Here is a +complete example: + + import logging + import logging.config + import sys + + class MyFilter(logging.Filter): + def __init__(self, param=None): + self.param = param + + def filter(self, record): + if self.param is None: + allow = True + else: + allow = self.param not in record.msg + if allow: + record.msg = 'changed: ' + record.msg + return allow + + LOGGING = { + 'version': 1, + 'filters': { + 'myfilter': { + '()': MyFilter, + 'param': 'noshow', + } + }, + 'handlers': { + 'console': { + 'class': 'logging.StreamHandler', + 'filters': ['myfilter'] + } + }, + 'root': { + 'level': 'DEBUG', + 'handlers': ['console'] + }, + } + + if __name__ == '__main__': + logging.config.dictConfig(LOGGING) + logging.debug('hello') + logging.debug('hello - noshow') + +This example shows how you can pass configuration data to the callable +which constructs the instance, in the form of keyword parameters. When +run, the above script will print: + + changed: hello + +which shows that the filter is working as configured. + +A couple of extra points to note: + + * If you can’t refer to the callable directly in the configuration + (e.g. if it lives in a different module, and you can’t import it + directly where the configuration dictionary is), you can use the + form ‘ext://...’ as described in *note Access to external objects: + 1dbf. For example, you could have used the text + ‘'ext://__main__.MyFilter'’ instead of ‘MyFilter’ in the above + example. + + * As well as for filters, this technique can also be used to + configure custom handlers and formatters. See *note User-defined + objects: 1db8. for more information on how logging supports using + user-defined objects in its configuration, and see the other + cookbook recipe *note Customizing handlers with dictConfig(): 3edf. + above. + + +File: python.info, Node: Customized exception formatting, Next: Speaking logging messages, Prev: Configuring filters with dictConfig, Up: Logging Cookbook + +10.7.23 Customized exception formatting +--------------------------------------- + +There might be times when you want to do customized exception formatting +- for argument’s sake, let’s say you want exactly one line per logged +event, even when exception information is present. You can do this with +a custom formatter class, as shown in the following example: + + import logging + + class OneLineExceptionFormatter(logging.Formatter): + def formatException(self, exc_info): + """ + Format an exception so that it prints on a single line. + """ + result = super(OneLineExceptionFormatter, self).formatException(exc_info) + return repr(result) # or format into one line however you want to + + def format(self, record): + s = super(OneLineExceptionFormatter, self).format(record) + if record.exc_text: + s = s.replace('\n', '') + '|' + return s + + def configure_logging(): + fh = logging.FileHandler('output.txt', 'w') + f = OneLineExceptionFormatter('%(asctime)s|%(levelname)s|%(message)s|', + '%d/%m/%Y %H:%M:%S') + fh.setFormatter(f) + root = logging.getLogger() + root.setLevel(logging.DEBUG) + root.addHandler(fh) + + def main(): + configure_logging() + logging.info('Sample message') + try: + x = 1 / 0 + except ZeroDivisionError as e: + logging.exception('ZeroDivisionError: %s', e) + + if __name__ == '__main__': + main() + +When run, this produces a file with exactly two lines: + + 28/01/2015 07:21:23|INFO|Sample message| + 28/01/2015 07:21:23|ERROR|ZeroDivisionError: integer division or modulo by zero|'Traceback (most recent call last):\n File "logtest7.py", line 30, in main\n x = 1 / 0\nZeroDivisionError: integer division or modulo by zero'| + +While the above treatment is simplistic, it points the way to how +exception information can be formatted to your liking. The *note +traceback: 113. module may be helpful for more specialized needs. + + +File: python.info, Node: Speaking logging messages, Next: Buffering logging messages and outputting them conditionally, Prev: Customized exception formatting, Up: Logging Cookbook + +10.7.24 Speaking logging messages +--------------------------------- + +There might be situations when it is desirable to have logging messages +rendered in an audible rather than a visible format. This is easy to do +if you have text-to-speech (TTS) functionality available in your system, +even if it doesn’t have a Python binding. Most TTS systems have a +command line program you can run, and this can be invoked from a handler +using *note subprocess: f9. It’s assumed here that TTS command line +programs won’t expect to interact with users or take a long time to +complete, and that the frequency of logged messages will be not so high +as to swamp the user with messages, and that it’s acceptable to have the +messages spoken one at a time rather than concurrently, The example +implementation below waits for one message to be spoken before the next +is processed, and this might cause other handlers to be kept waiting. +Here is a short example showing the approach, which assumes that the +‘espeak’ TTS package is available: + + import logging + import subprocess + import sys + + class TTSHandler(logging.Handler): + def emit(self, record): + msg = self.format(record) + # Speak slowly in a female English voice + cmd = ['espeak', '-s150', '-ven+f3', msg] + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT) + # wait for the program to finish + p.communicate() + + def configure_logging(): + h = TTSHandler() + root = logging.getLogger() + root.addHandler(h) + # the default formatter just returns the message + root.setLevel(logging.DEBUG) + + def main(): + logging.info('Hello') + logging.debug('Goodbye') + + if __name__ == '__main__': + configure_logging() + sys.exit(main()) + +When run, this script should say “Hello” and then “Goodbye” in a female +voice. + +The above approach can, of course, be adapted to other TTS systems and +even other systems altogether which can process messages via external +programs run from a command line. + + +File: python.info, Node: Buffering logging messages and outputting them conditionally, Next: Formatting times using UTC GMT via configuration, Prev: Speaking logging messages, Up: Logging Cookbook + +10.7.25 Buffering logging messages and outputting them conditionally +-------------------------------------------------------------------- + +There might be situations where you want to log messages in a temporary +area and only output them if a certain condition occurs. For example, +you may want to start logging debug events in a function, and if the +function completes without errors, you don’t want to clutter the log +with the collected debug information, but if there is an error, you want +all the debug information to be output as well as the error. + +Here is an example which shows how you could do this using a decorator +for your functions where you want logging to behave this way. It makes +use of the *note logging.handlers.MemoryHandler: 1dc2, which allows +buffering of logged events until some condition occurs, at which point +the buffered events are ‘flushed’ - passed to another handler (the +‘target’ handler) for processing. By default, the ‘MemoryHandler’ +flushed when its buffer gets filled up or an event whose level is +greater than or equal to a specified threshold is seen. You can use +this recipe with a more specialised subclass of ‘MemoryHandler’ if you +want custom flushing behavior. + +The example script has a simple function, ‘foo’, which just cycles +through all the logging levels, writing to ‘sys.stderr’ to say what +level it’s about to log at, and then actually logging a message at that +level. You can pass a parameter to ‘foo’ which, if true, will log at +ERROR and CRITICAL levels - otherwise, it only logs at DEBUG, INFO and +WARNING levels. + +The script just arranges to decorate ‘foo’ with a decorator which will +do the conditional logging that’s required. The decorator takes a +logger as a parameter and attaches a memory handler for the duration of +the call to the decorated function. The decorator can be additionally +parameterised using a target handler, a level at which flushing should +occur, and a capacity for the buffer (number of records buffered). +These default to a *note StreamHandler: b75. which writes to +‘sys.stderr’, ‘logging.ERROR’ and ‘100’ respectively. + +Here’s the script: + + import logging + from logging.handlers import MemoryHandler + import sys + + logger = logging.getLogger(__name__) + logger.addHandler(logging.NullHandler()) + + def log_if_errors(logger, target_handler=None, flush_level=None, capacity=None): + if target_handler is None: + target_handler = logging.StreamHandler() + if flush_level is None: + flush_level = logging.ERROR + if capacity is None: + capacity = 100 + handler = MemoryHandler(capacity, flushLevel=flush_level, target=target_handler) + + def decorator(fn): + def wrapper(*args, **kwargs): + logger.addHandler(handler) + try: + return fn(*args, **kwargs) + except Exception: + logger.exception('call failed') + raise + finally: + super(MemoryHandler, handler).flush() + logger.removeHandler(handler) + return wrapper + + return decorator + + def write_line(s): + sys.stderr.write('%s\n' % s) + + def foo(fail=False): + write_line('about to log at DEBUG ...') + logger.debug('Actually logged at DEBUG') + write_line('about to log at INFO ...') + logger.info('Actually logged at INFO') + write_line('about to log at WARNING ...') + logger.warning('Actually logged at WARNING') + if fail: + write_line('about to log at ERROR ...') + logger.error('Actually logged at ERROR') + write_line('about to log at CRITICAL ...') + logger.critical('Actually logged at CRITICAL') + return fail + + decorated_foo = log_if_errors(logger)(foo) + + if __name__ == '__main__': + logger.setLevel(logging.DEBUG) + write_line('Calling undecorated foo with False') + assert not foo(False) + write_line('Calling undecorated foo with True') + assert foo(True) + write_line('Calling decorated foo with False') + assert not decorated_foo(False) + write_line('Calling decorated foo with True') + assert decorated_foo(True) + +When this script is run, the following output should be observed: + + Calling undecorated foo with False + about to log at DEBUG ... + about to log at INFO ... + about to log at WARNING ... + Calling undecorated foo with True + about to log at DEBUG ... + about to log at INFO ... + about to log at WARNING ... + about to log at ERROR ... + about to log at CRITICAL ... + Calling decorated foo with False + about to log at DEBUG ... + about to log at INFO ... + about to log at WARNING ... + Calling decorated foo with True + about to log at DEBUG ... + about to log at INFO ... + about to log at WARNING ... + about to log at ERROR ... + Actually logged at DEBUG + Actually logged at INFO + Actually logged at WARNING + Actually logged at ERROR + about to log at CRITICAL ... + Actually logged at CRITICAL + +As you can see, actual logging output only occurs when an event is +logged whose severity is ERROR or greater, but in that case, any +previous events at lower severities are also logged. + +You can of course use the conventional means of decoration: + + @log_if_errors(logger) + def foo(fail=False): + ... + + +File: python.info, Node: Formatting times using UTC GMT via configuration, Next: Using a context manager for selective logging, Prev: Buffering logging messages and outputting them conditionally, Up: Logging Cookbook + +10.7.26 Formatting times using UTC (GMT) via configuration +---------------------------------------------------------- + +Sometimes you want to format times using UTC, which can be done using a +class such as ‘UTCFormatter’, shown below: + + import logging + import time + + class UTCFormatter(logging.Formatter): + converter = time.gmtime + +and you can then use the ‘UTCFormatter’ in your code instead of *note +Formatter: 1d61. If you want to do that via configuration, you can use +the *note dictConfig(): b2b. API with an approach illustrated by the +following complete example: + + import logging + import logging.config + import time + + class UTCFormatter(logging.Formatter): + converter = time.gmtime + + LOGGING = { + 'version': 1, + 'disable_existing_loggers': False, + 'formatters': { + 'utc': { + '()': UTCFormatter, + 'format': '%(asctime)s %(message)s', + }, + 'local': { + 'format': '%(asctime)s %(message)s', + } + }, + 'handlers': { + 'console1': { + 'class': 'logging.StreamHandler', + 'formatter': 'utc', + }, + 'console2': { + 'class': 'logging.StreamHandler', + 'formatter': 'local', + }, + }, + 'root': { + 'handlers': ['console1', 'console2'], + } + } + + if __name__ == '__main__': + logging.config.dictConfig(LOGGING) + logging.warning('The local time is %s', time.asctime()) + +When this script is run, it should print something like: + + 2015-10-17 12:53:29,501 The local time is Sat Oct 17 13:53:29 2015 + 2015-10-17 13:53:29,501 The local time is Sat Oct 17 13:53:29 2015 + +showing how the time is formatted both as local time and UTC, one for +each handler. + + +File: python.info, Node: Using a context manager for selective logging, Next: A CLI application starter template, Prev: Formatting times using UTC GMT via configuration, Up: Logging Cookbook + +10.7.27 Using a context manager for selective logging +----------------------------------------------------- + +There are times when it would be useful to temporarily change the +logging configuration and revert it back after doing something. For +this, a context manager is the most obvious way of saving and restoring +the logging context. Here is a simple example of such a context +manager, which allows you to optionally change the logging level and add +a logging handler purely in the scope of the context manager: + + import logging + import sys + + class LoggingContext: + def __init__(self, logger, level=None, handler=None, close=True): + self.logger = logger + self.level = level + self.handler = handler + self.close = close + + def __enter__(self): + if self.level is not None: + self.old_level = self.logger.level + self.logger.setLevel(self.level) + if self.handler: + self.logger.addHandler(self.handler) + + def __exit__(self, et, ev, tb): + if self.level is not None: + self.logger.setLevel(self.old_level) + if self.handler: + self.logger.removeHandler(self.handler) + if self.handler and self.close: + self.handler.close() + # implicit return of None => don't swallow exceptions + +If you specify a level value, the logger’s level is set to that value in +the scope of the with block covered by the context manager. If you +specify a handler, it is added to the logger on entry to the block and +removed on exit from the block. You can also ask the manager to close +the handler for you on block exit - you could do this if you don’t need +the handler any more. + +To illustrate how it works, we can add the following block of code to +the above: + + if __name__ == '__main__': + logger = logging.getLogger('foo') + logger.addHandler(logging.StreamHandler()) + logger.setLevel(logging.INFO) + logger.info('1. This should appear just once on stderr.') + logger.debug('2. This should not appear.') + with LoggingContext(logger, level=logging.DEBUG): + logger.debug('3. This should appear once on stderr.') + logger.debug('4. This should not appear.') + h = logging.StreamHandler(sys.stdout) + with LoggingContext(logger, level=logging.DEBUG, handler=h, close=True): + logger.debug('5. This should appear twice - once on stderr and once on stdout.') + logger.info('6. This should appear just once on stderr.') + logger.debug('7. This should not appear.') + +We initially set the logger’s level to ‘INFO’, so message #1 appears and +message #2 doesn’t. We then change the level to ‘DEBUG’ temporarily in +the following ‘with’ block, and so message #3 appears. After the block +exits, the logger’s level is restored to ‘INFO’ and so message #4 +doesn’t appear. In the next ‘with’ block, we set the level to ‘DEBUG’ +again but also add a handler writing to ‘sys.stdout’. Thus, message #5 +appears twice on the console (once via ‘stderr’ and once via ‘stdout’). +After the ‘with’ statement’s completion, the status is as it was before +so message #6 appears (like message #1) whereas message #7 doesn’t (just +like message #2). + +If we run the resulting script, the result is as follows: + + $ python logctx.py + 1. This should appear just once on stderr. + 3. This should appear once on stderr. + 5. This should appear twice - once on stderr and once on stdout. + 5. This should appear twice - once on stderr and once on stdout. + 6. This should appear just once on stderr. + +If we run it again, but pipe ‘stderr’ to ‘/dev/null’, we see the +following, which is the only message written to ‘stdout’: + + $ python logctx.py 2>/dev/null + 5. This should appear twice - once on stderr and once on stdout. + +Once again, but piping ‘stdout’ to ‘/dev/null’, we get: + + $ python logctx.py >/dev/null + 1. This should appear just once on stderr. + 3. This should appear once on stderr. + 5. This should appear twice - once on stderr and once on stdout. + 6. This should appear just once on stderr. + +In this case, the message #5 printed to ‘stdout’ doesn’t appear, as +expected. + +Of course, the approach described here can be generalised, for example +to attach logging filters temporarily. Note that the above code works +in Python 2 as well as Python 3. + + +File: python.info, Node: A CLI application starter template, Next: A Qt GUI for logging, Prev: Using a context manager for selective logging, Up: Logging Cookbook + +10.7.28 A CLI application starter template +------------------------------------------ + +Here’s an example which shows how you can: + + * Use a logging level based on command-line arguments + + * Dispatch to multiple subcommands in separate files, all logging at + the same level in a consistent way + + * Make use of simple, minimal configuration + +Suppose we have a command-line application whose job is to stop, start +or restart some services. This could be organised for the purposes of +illustration as a file ‘app.py’ that is the main script for the +application, with individual commands implemented in ‘start.py’, +‘stop.py’ and ‘restart.py’. Suppose further that we want to control the +verbosity of the application via a command-line argument, defaulting to +‘logging.INFO’. Here’s one way that ‘app.py’ could be written: + + import argparse + import importlib + import logging + import os + import sys + + def main(args=None): + scriptname = os.path.basename(__file__) + parser = argparse.ArgumentParser(scriptname) + levels = ('DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL') + parser.add_argument('--log-level', default='INFO', choices=levels) + subparsers = parser.add_subparsers(dest='command', + help='Available commands:') + start_cmd = subparsers.add_parser('start', help='Start a service') + start_cmd.add_argument('name', metavar='NAME', + help='Name of service to start') + stop_cmd = subparsers.add_parser('stop', + help='Stop one or more services') + stop_cmd.add_argument('names', metavar='NAME', nargs='+', + help='Name of service to stop') + restart_cmd = subparsers.add_parser('restart', + help='Restart one or more services') + restart_cmd.add_argument('names', metavar='NAME', nargs='+', + help='Name of service to restart') + options = parser.parse_args() + # the code to dispatch commands could all be in this file. For the purposes + # of illustration only, we implement each command in a separate module. + try: + mod = importlib.import_module(options.command) + cmd = getattr(mod, 'command') + except (ImportError, AttributeError): + print('Unable to find the code for command \'%s\'' % options.command) + return 1 + # Could get fancy here and load configuration from file or dictionary + logging.basicConfig(level=options.log_level, + format='%(levelname)s %(name)s %(message)s') + cmd(options) + + if __name__ == '__main__': + sys.exit(main()) + +And the ‘start’, ‘stop’ and ‘restart’ commands can be implemented in +separate modules, like so for starting: + + # start.py + import logging + + logger = logging.getLogger(__name__) + + def command(options): + logger.debug('About to start %s', options.name) + # actually do the command processing here ... + logger.info('Started the \'%s\' service.', options.name) + +and thus for stopping: + + # stop.py + import logging + + logger = logging.getLogger(__name__) + + def command(options): + n = len(options.names) + if n == 1: + plural = '' + services = '\'%s\'' % options.names[0] + else: + plural = 's' + services = ', '.join('\'%s\'' % name for name in options.names) + i = services.rfind(', ') + services = services[:i] + ' and ' + services[i + 2:] + logger.debug('About to stop %s', services) + # actually do the command processing here ... + logger.info('Stopped the %s service%s.', services, plural) + +and similarly for restarting: + + # restart.py + import logging + + logger = logging.getLogger(__name__) + + def command(options): + n = len(options.names) + if n == 1: + plural = '' + services = '\'%s\'' % options.names[0] + else: + plural = 's' + services = ', '.join('\'%s\'' % name for name in options.names) + i = services.rfind(', ') + services = services[:i] + ' and ' + services[i + 2:] + logger.debug('About to restart %s', services) + # actually do the command processing here ... + logger.info('Restarted the %s service%s.', services, plural) + +If we run this application with the default log level, we get output +like this: + + $ python app.py start foo + INFO start Started the 'foo' service. + + $ python app.py stop foo bar + INFO stop Stopped the 'foo' and 'bar' services. + + $ python app.py restart foo bar baz + INFO restart Restarted the 'foo', 'bar' and 'baz' services. + +The first word is the logging level, and the second word is the module +or package name of the place where the event was logged. + +If we change the logging level, then we can change the information sent +to the log. For example, if we want more information: + + $ python app.py --log-level DEBUG start foo + DEBUG start About to start foo + INFO start Started the 'foo' service. + + $ python app.py --log-level DEBUG stop foo bar + DEBUG stop About to stop 'foo' and 'bar' + INFO stop Stopped the 'foo' and 'bar' services. + + $ python app.py --log-level DEBUG restart foo bar baz + DEBUG restart About to restart 'foo', 'bar' and 'baz' + INFO restart Restarted the 'foo', 'bar' and 'baz' services. + +And if we want less: + + $ python app.py --log-level WARNING start foo + $ python app.py --log-level WARNING stop foo bar + $ python app.py --log-level WARNING restart foo bar baz + +In this case, the commands don’t print anything to the console, since +nothing at ‘WARNING’ level or above is logged by them. + + +File: python.info, Node: A Qt GUI for logging, Prev: A CLI application starter template, Up: Logging Cookbook + +10.7.29 A Qt GUI for logging +---------------------------- + +A question that comes up from time to time is about how to log to a GUI +application. The Qt(1) framework is a popular cross-platform UI +framework with Python bindings using PySide2(2) or PyQt5(3) libraries. + +The following example shows how to log to a Qt GUI. This introduces a +simple ‘QtHandler’ class which takes a callable, which should be a slot +in the main thread that does GUI updates. A worker thread is also +created to show how you can log to the GUI from both the UI itself (via +a button for manual logging) as well as a worker thread doing work in +the background (here, just logging messages at random levels with random +short delays in between). + +The worker thread is implemented using Qt’s ‘QThread’ class rather than +the *note threading: 109. module, as there are circumstances where one +has to use ‘QThread’, which offers better integration with other ‘Qt’ +components. + +The code should work with recent releases of either ‘PySide2’ or +‘PyQt5’. You should be able to adapt the approach to earlier versions +of Qt. Please refer to the comments in the code snippet for more +detailed information. + + import datetime + import logging + import random + import sys + import time + + # Deal with minor differences between PySide2 and PyQt5 + try: + from PySide2 import QtCore, QtGui, QtWidgets + Signal = QtCore.Signal + Slot = QtCore.Slot + except ImportError: + from PyQt5 import QtCore, QtGui, QtWidgets + Signal = QtCore.pyqtSignal + Slot = QtCore.pyqtSlot + + + logger = logging.getLogger(__name__) + + + # + # Signals need to be contained in a QObject or subclass in order to be correctly + # initialized. + # + class Signaller(QtCore.QObject): + signal = Signal(str, logging.LogRecord) + + # + # Output to a Qt GUI is only supposed to happen on the main thread. So, this + # handler is designed to take a slot function which is set up to run in the main + # thread. In this example, the function takes a string argument which is a + # formatted log message, and the log record which generated it. The formatted + # string is just a convenience - you could format a string for output any way + # you like in the slot function itself. + # + # You specify the slot function to do whatever GUI updates you want. The handler + # doesn't know or care about specific UI elements. + # + class QtHandler(logging.Handler): + def __init__(self, slotfunc, *args, **kwargs): + super(QtHandler, self).__init__(*args, **kwargs) + self.signaller = Signaller() + self.signaller.signal.connect(slotfunc) + + def emit(self, record): + s = self.format(record) + self.signaller.signal.emit(s, record) + + # + # This example uses QThreads, which means that the threads at the Python level + # are named something like "Dummy-1". The function below gets the Qt name of the + # current thread. + # + def ctname(): + return QtCore.QThread.currentThread().objectName() + + + # + # Used to generate random levels for logging. + # + LEVELS = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, + logging.CRITICAL) + + # + # This worker class represents work that is done in a thread separate to the + # main thread. The way the thread is kicked off to do work is via a button press + # that connects to a slot in the worker. + # + # Because the default threadName value in the LogRecord isn't much use, we add + # a qThreadName which contains the QThread name as computed above, and pass that + # value in an "extra" dictionary which is used to update the LogRecord with the + # QThread name. + # + # This example worker just outputs messages sequentially, interspersed with + # random delays of the order of a few seconds. + # + class Worker(QtCore.QObject): + @Slot() + def start(self): + extra = {'qThreadName': ctname() } + logger.debug('Started work', extra=extra) + i = 1 + # Let the thread run until interrupted. This allows reasonably clean + # thread termination. + while not QtCore.QThread.currentThread().isInterruptionRequested(): + delay = 0.5 + random.random() * 2 + time.sleep(delay) + level = random.choice(LEVELS) + logger.log(level, 'Message after delay of %3.1f: %d', delay, i, extra=extra) + i += 1 + + # + # Implement a simple UI for this cookbook example. This contains: + # + # * A read-only text edit window which holds formatted log messages + # * A button to start work and log stuff in a separate thread + # * A button to log something from the main thread + # * A button to clear the log window + # + class Window(QtWidgets.QWidget): + + COLORS = { + logging.DEBUG: 'black', + logging.INFO: 'blue', + logging.WARNING: 'orange', + logging.ERROR: 'red', + logging.CRITICAL: 'purple', + } + + def __init__(self, app): + super(Window, self).__init__() + self.app = app + self.textedit = te = QtWidgets.QPlainTextEdit(self) + # Set whatever the default monospace font is for the platform + f = QtGui.QFont('nosuchfont') + f.setStyleHint(f.Monospace) + te.setFont(f) + te.setReadOnly(True) + PB = QtWidgets.QPushButton + self.work_button = PB('Start background work', self) + self.log_button = PB('Log a message at a random level', self) + self.clear_button = PB('Clear log window', self) + self.handler = h = QtHandler(self.update_status) + # Remember to use qThreadName rather than threadName in the format string. + fs = '%(asctime)s %(qThreadName)-12s %(levelname)-8s %(message)s' + formatter = logging.Formatter(fs) + h.setFormatter(formatter) + logger.addHandler(h) + # Set up to terminate the QThread when we exit + app.aboutToQuit.connect(self.force_quit) + + # Lay out all the widgets + layout = QtWidgets.QVBoxLayout(self) + layout.addWidget(te) + layout.addWidget(self.work_button) + layout.addWidget(self.log_button) + layout.addWidget(self.clear_button) + self.setFixedSize(900, 400) + + # Connect the non-worker slots and signals + self.log_button.clicked.connect(self.manual_update) + self.clear_button.clicked.connect(self.clear_display) + + # Start a new worker thread and connect the slots for the worker + self.start_thread() + self.work_button.clicked.connect(self.worker.start) + # Once started, the button should be disabled + self.work_button.clicked.connect(lambda : self.work_button.setEnabled(False)) + + def start_thread(self): + self.worker = Worker() + self.worker_thread = QtCore.QThread() + self.worker.setObjectName('Worker') + self.worker_thread.setObjectName('WorkerThread') # for qThreadName + self.worker.moveToThread(self.worker_thread) + # This will start an event loop in the worker thread + self.worker_thread.start() + + def kill_thread(self): + # Just tell the worker to stop, then tell it to quit and wait for that + # to happen + self.worker_thread.requestInterruption() + if self.worker_thread.isRunning(): + self.worker_thread.quit() + self.worker_thread.wait() + else: + print('worker has already exited.') + + def force_quit(self): + # For use when the window is closed + if self.worker_thread.isRunning(): + self.kill_thread() + + # The functions below update the UI and run in the main thread because + # that's where the slots are set up + + @Slot(str, logging.LogRecord) + def update_status(self, status, record): + color = self.COLORS.get(record.levelno, 'black') + s = '<pre><font color="%s">%s</font></pre>' % (color, status) + self.textedit.appendHtml(s) + + @Slot() + def manual_update(self): + # This function uses the formatted message passed in, but also uses + # information from the record to format the message in an appropriate + # color according to its severity (level). + level = random.choice(LEVELS) + extra = {'qThreadName': ctname() } + logger.log(level, 'Manually logged!', extra=extra) + + @Slot() + def clear_display(self): + self.textedit.clear() + + + def main(): + QtCore.QThread.currentThread().setObjectName('MainThread') + logging.getLogger().setLevel(logging.DEBUG) + app = QtWidgets.QApplication(sys.argv) + example = Window(app) + example.show() + sys.exit(app.exec_()) + + if __name__=='__main__': + main() + + ---------- Footnotes ---------- + + (1) https://www.qt.io/ + + (2) https://pypi.org/project/PySide2/ + + (3) https://pypi.org/project/PyQt5/ + + +File: python.info, Node: Regular Expression HOWTO, Next: Socket Programming HOWTO, Prev: Logging Cookbook, Up: Python HOWTOs + +10.8 Regular Expression HOWTO +============================= + + +Author: A.M. Kuchling <<amk@amk.ca>> + +Abstract +........ + +This document is an introductory tutorial to using regular expressions +in Python with the *note re: dd. module. It provides a gentler +introduction than the corresponding section in the Library Reference. + +* Menu: + +* Introduction: Introduction<15>. +* Simple Patterns:: +* Using Regular Expressions:: +* More Pattern Power:: +* Modifying Strings:: +* Common Problems:: +* Feedback:: + + +File: python.info, Node: Introduction<15>, Next: Simple Patterns, Up: Regular Expression HOWTO + +10.8.1 Introduction +------------------- + +Regular expressions (called REs, or regexes, or regex patterns) are +essentially a tiny, highly specialized programming language embedded +inside Python and made available through the *note re: dd. module. +Using this little language, you specify the rules for the set of +possible strings that you want to match; this set might contain English +sentences, or e-mail addresses, or TeX commands, or anything you like. +You can then ask questions such as “Does this string match the +pattern?”, or “Is there a match for the pattern anywhere in this +string?”. You can also use REs to modify a string or to split it apart +in various ways. + +Regular expression patterns are compiled into a series of bytecodes +which are then executed by a matching engine written in C. For advanced +use, it may be necessary to pay careful attention to how the engine will +execute a given RE, and write the RE in a certain way in order to +produce bytecode that runs faster. Optimization isn’t covered in this +document, because it requires that you have a good understanding of the +matching engine’s internals. + +The regular expression language is relatively small and restricted, so +not all possible string processing tasks can be done using regular +expressions. There are also tasks that `can' be done with regular +expressions, but the expressions turn out to be very complicated. In +these cases, you may be better off writing Python code to do the +processing; while Python code will be slower than an elaborate regular +expression, it will also probably be more understandable. + + +File: python.info, Node: Simple Patterns, Next: Using Regular Expressions, Prev: Introduction<15>, Up: Regular Expression HOWTO + +10.8.2 Simple Patterns +---------------------- + +We’ll start by learning about the simplest possible regular expressions. +Since regular expressions are used to operate on strings, we’ll begin +with the most common task: matching characters. + +For a detailed explanation of the computer science underlying regular +expressions (deterministic and non-deterministic finite automata), you +can refer to almost any textbook on writing compilers. + +* Menu: + +* Matching Characters:: +* Repeating Things:: + + +File: python.info, Node: Matching Characters, Next: Repeating Things, Up: Simple Patterns + +10.8.2.1 Matching Characters +............................ + +Most letters and characters will simply match themselves. For example, +the regular expression ‘test’ will match the string ‘test’ exactly. +(You can enable a case-insensitive mode that would let this RE match +‘Test’ or ‘TEST’ as well; more about this later.) + +There are exceptions to this rule; some characters are special +`metacharacters', and don’t match themselves. Instead, they signal that +some out-of-the-ordinary thing should be matched, or they affect other +portions of the RE by repeating them or changing their meaning. Much of +this document is devoted to discussing various metacharacters and what +they do. + +Here’s a complete list of the metacharacters; their meanings will be +discussed in the rest of this HOWTO. + + . ^ $ * + ? { } [ ] \ | ( ) + +The first metacharacters we’ll look at are ‘[’ and ‘]’. They’re used +for specifying a character class, which is a set of characters that you +wish to match. Characters can be listed individually, or a range of +characters can be indicated by giving two characters and separating them +by a ‘'-'’. For example, ‘[abc]’ will match any of the characters ‘a’, +‘b’, or ‘c’; this is the same as ‘[a-c]’, which uses a range to express +the same set of characters. If you wanted to match only lowercase +letters, your RE would be ‘[a-z]’. + +Metacharacters are not active inside classes. For example, ‘[akm$]’ +will match any of the characters ‘'a'’, ‘'k'’, ‘'m'’, or ‘'$'’; ‘'$'’ is +usually a metacharacter, but inside a character class it’s stripped of +its special nature. + +You can match the characters not listed within the class by +`complementing' the set. This is indicated by including a ‘'^'’ as the +first character of the class. For example, ‘[^5]’ will match any +character except ‘'5'’. If the caret appears elsewhere in a character +class, it does not have special meaning. For example: ‘[5^]’ will match +either a ‘'5'’ or a ‘'^'’. + +Perhaps the most important metacharacter is the backslash, ‘\’. As in +Python string literals, the backslash can be followed by various +characters to signal various special sequences. It’s also used to +escape all the metacharacters so you can still match them in patterns; +for example, if you need to match a ‘[’ or ‘\’, you can precede them +with a backslash to remove their special meaning: ‘\[’ or ‘\\’. + +Some of the special sequences beginning with ‘'\'’ represent predefined +sets of characters that are often useful, such as the set of digits, the +set of letters, or the set of anything that isn’t whitespace. + +Let’s take an example: ‘\w’ matches any alphanumeric character. If the +regex pattern is expressed in bytes, this is equivalent to the class +‘[a-zA-Z0-9_]’. If the regex pattern is a string, ‘\w’ will match all +the characters marked as letters in the Unicode database provided by the +*note unicodedata: 11a. module. You can use the more restricted +definition of ‘\w’ in a string pattern by supplying the *note re.ASCII: +3c8. flag when compiling the regular expression. + +The following list of special sequences isn’t complete. For a complete +list of sequences and expanded class definitions for Unicode string +patterns, see the last part of *note Regular Expression Syntax: 13f5. in +the Standard Library reference. In general, the Unicode versions match +any character that’s in the appropriate category in the Unicode +database. + +‘\d’ + + Matches any decimal digit; this is equivalent to the class ‘[0-9]’. + +‘\D’ + + Matches any non-digit character; this is equivalent to the class + ‘[^0-9]’. + +‘\s’ + + Matches any whitespace character; this is equivalent to the class + ‘[ \t\n\r\f\v]’. + +‘\S’ + + Matches any non-whitespace character; this is equivalent to the + class ‘[^ \t\n\r\f\v]’. + +‘\w’ + + Matches any alphanumeric character; this is equivalent to the class + ‘[a-zA-Z0-9_]’. + +‘\W’ + + Matches any non-alphanumeric character; this is equivalent to the + class ‘[^a-zA-Z0-9_]’. + +These sequences can be included inside a character class. For example, +‘[\s,.]’ is a character class that will match any whitespace character, +or ‘','’ or ‘'.'’. + +The final metacharacter in this section is ‘.’. It matches anything +except a newline character, and there’s an alternate mode (*note +re.DOTALL: 13f9.) where it will match even a newline. ‘.’ is often used +where you want to match “any character”. + + +File: python.info, Node: Repeating Things, Prev: Matching Characters, Up: Simple Patterns + +10.8.2.2 Repeating Things +......................... + +Being able to match varying sets of characters is the first thing +regular expressions can do that isn’t already possible with the methods +available on strings. However, if that was the only additional +capability of regexes, they wouldn’t be much of an advance. Another +capability is that you can specify that portions of the RE must be +repeated a certain number of times. + +The first metacharacter for repeating things that we’ll look at is ‘*’. +‘*’ doesn’t match the literal character ‘'*'’; instead, it specifies +that the previous character can be matched zero or more times, instead +of exactly once. + +For example, ‘ca*t’ will match ‘'ct'’ (0 ‘'a'’ characters), ‘'cat'’ (1 +‘'a'’), ‘'caaat'’ (3 ‘'a'’ characters), and so forth. + +Repetitions such as ‘*’ are `greedy'; when repeating a RE, the matching +engine will try to repeat it as many times as possible. If later +portions of the pattern don’t match, the matching engine will then back +up and try again with fewer repetitions. + +A step-by-step example will make this more obvious. Let’s consider the +expression ‘a[bcd]*b’. This matches the letter ‘'a'’, zero or more +letters from the class ‘[bcd]’, and finally ends with a ‘'b'’. Now +imagine matching this RE against the string ‘'abcbd'’. + +Step Matched Explanation + +----------------------------------------------------------------- + +1 ‘a’ The ‘a’ in the RE matches. + + +2 ‘abcbd’ The engine matches ‘[bcd]*’, going + as far as it can, which is to the + end of the string. + + +3 `Failure' The engine tries to match ‘b’, but + the current position is at the end + of the string, so it fails. + + +4 ‘abcb’ Back up, so that ‘[bcd]*’ matches + one less character. + + +5 `Failure' Try ‘b’ again, but the current + position is at the last character, + which is a ‘'d'’. + + +6 ‘abc’ Back up again, so that ‘[bcd]*’ is + only matching ‘bc’. + + +6 ‘abcb’ Try ‘b’ again. This time the + character at the current position + is ‘'b'’, so it succeeds. + + +The end of the RE has now been reached, and it has matched ‘'abcb'’. +This demonstrates how the matching engine goes as far as it can at +first, and if no match is found it will then progressively back up and +retry the rest of the RE again and again. It will back up until it has +tried zero matches for ‘[bcd]*’, and if that subsequently fails, the +engine will conclude that the string doesn’t match the RE at all. + +Another repeating metacharacter is ‘+’, which matches one or more times. +Pay careful attention to the difference between ‘*’ and ‘+’; ‘*’ matches +`zero' or more times, so whatever’s being repeated may not be present at +all, while ‘+’ requires at least `one' occurrence. To use a similar +example, ‘ca+t’ will match ‘'cat'’ (1 ‘'a'’), ‘'caaat'’ (3 ‘'a'’s), but +won’t match ‘'ct'’. + +There are two more repeating qualifiers. The question mark character, +‘?’, matches either once or zero times; you can think of it as marking +something as being optional. For example, ‘home-?brew’ matches either +‘'homebrew'’ or ‘'home-brew'’. + +The most complicated repeated qualifier is ‘{m,n}’, where `m' and `n' +are decimal integers. This qualifier means there must be at least `m' +repetitions, and at most `n'. For example, ‘a/{1,3}b’ will match +‘'a/b'’, ‘'a//b'’, and ‘'a///b'’. It won’t match ‘'ab'’, which has no +slashes, or ‘'a////b'’, which has four. + +You can omit either `m' or `n'; in that case, a reasonable value is +assumed for the missing value. Omitting `m' is interpreted as a lower +limit of 0, while omitting `n' results in an upper bound of infinity. + +Readers of a reductionist bent may notice that the three other +qualifiers can all be expressed using this notation. ‘{0,}’ is the same +as ‘*’, ‘{1,}’ is equivalent to ‘+’, and ‘{0,1}’ is the same as ‘?’. +It’s better to use ‘*’, ‘+’, or ‘?’ when you can, simply because they’re +shorter and easier to read. + + +File: python.info, Node: Using Regular Expressions, Next: More Pattern Power, Prev: Simple Patterns, Up: Regular Expression HOWTO + +10.8.3 Using Regular Expressions +-------------------------------- + +Now that we’ve looked at some simple regular expressions, how do we +actually use them in Python? The *note re: dd. module provides an +interface to the regular expression engine, allowing you to compile REs +into objects and then perform matches with them. + +* Menu: + +* Compiling Regular Expressions:: +* The Backslash Plague:: +* Performing Matches:: +* Module-Level Functions: Module-Level Functions<2>. +* Compilation Flags:: + + +File: python.info, Node: Compiling Regular Expressions, Next: The Backslash Plague, Up: Using Regular Expressions + +10.8.3.1 Compiling Regular Expressions +...................................... + +Regular expressions are compiled into pattern objects, which have +methods for various operations such as searching for pattern matches or +performing string substitutions. + + >>> import re + >>> p = re.compile('ab*') + >>> p + re.compile('ab*') + +*note re.compile(): 44a. also accepts an optional `flags' argument, used +to enable various special features and syntax variations. We’ll go over +the available settings later, but for now a single example will do: + + >>> p = re.compile('ab*', re.IGNORECASE) + +The RE is passed to *note re.compile(): 44a. as a string. REs are +handled as strings because regular expressions aren’t part of the core +Python language, and no special syntax was created for expressing them. +(There are applications that don’t need REs at all, so there’s no need +to bloat the language specification by including them.) Instead, the +*note re: dd. module is simply a C extension module included with +Python, just like the *note socket: ef. or *note zlib: 144. modules. + +Putting REs in strings keeps the Python language simpler, but has one +disadvantage which is the topic of the next section. + + +File: python.info, Node: The Backslash Plague, Next: Performing Matches, Prev: Compiling Regular Expressions, Up: Using Regular Expressions + +10.8.3.2 The Backslash Plague +............................. + +As stated earlier, regular expressions use the backslash character +(‘'\'’) to indicate special forms or to allow special characters to be +used without invoking their special meaning. This conflicts with +Python’s usage of the same character for the same purpose in string +literals. + +Let’s say you want to write a RE that matches the string ‘\section’, +which might be found in a LaTeX file. To figure out what to write in +the program code, start with the desired string to be matched. Next, +you must escape any backslashes and other metacharacters by preceding +them with a backslash, resulting in the string ‘\\section’. The +resulting string that must be passed to *note re.compile(): 44a. must be +‘\\section’. However, to express this as a Python string literal, both +backslashes must be escaped `again'. + +Characters Stage + +----------------------------------------------------------------------- + +‘\section’ Text string to be matched + + +‘\\section’ Escaped backslash for + *note re.compile(): 44a. + + +‘"\\\\section"’ Escaped backslashes for a string literal + + +In short, to match a literal backslash, one has to write ‘'\\\\'’ as the +RE string, because the regular expression must be ‘\\’, and each +backslash must be expressed as ‘\\’ inside a regular Python string +literal. In REs that feature backslashes repeatedly, this leads to lots +of repeated backslashes and makes the resulting strings difficult to +understand. + +The solution is to use Python’s raw string notation for regular +expressions; backslashes are not handled in any special way in a string +literal prefixed with ‘'r'’, so ‘r"\n"’ is a two-character string +containing ‘'\'’ and ‘'n'’, while ‘"\n"’ is a one-character string +containing a newline. Regular expressions will often be written in +Python code using this raw string notation. + +In addition, special escape sequences that are valid in regular +expressions, but not valid as Python string literals, now result in a +*note DeprecationWarning: 278. and will eventually become a *note +SyntaxError: 458, which means the sequences will be invalid if raw +string notation or escaping the backslashes isn’t used. + +Regular String Raw string + +----------------------------------------------- + +‘"ab*"’ ‘r"ab*"’ + + +‘"\\\\section"’ ‘r"\\section"’ + + +‘"\\w+\\s+\\1"’ ‘r"\w+\s+\1"’ + + + +File: python.info, Node: Performing Matches, Next: Module-Level Functions<2>, Prev: The Backslash Plague, Up: Using Regular Expressions + +10.8.3.3 Performing Matches +........................... + +Once you have an object representing a compiled regular expression, what +do you do with it? Pattern objects have several methods and attributes. +Only the most significant ones will be covered here; consult the *note +re: dd. docs for a complete listing. + +Method/Attribute Purpose + +--------------------------------------------------------------------------- + +‘match()’ Determine if the RE matches at the beginning of + the string. + + +‘search()’ Scan through a string, looking for any location + where this RE matches. + + +‘findall()’ Find all substrings where the RE matches, and + returns them as a list. + + +‘finditer()’ Find all substrings where the RE matches, and + returns them as an *note iterator: 112e. + + +*note match(): 1404. and *note search(): 1405. return ‘None’ if no match +can be found. If they’re successful, a *note match object: 89d. +instance is returned, containing information about the match: where it +starts and ends, the substring it matched, and more. + +You can learn about this by interactively experimenting with the *note +re: dd. module. If you have *note tkinter: 10c. available, you may also +want to look at Tools/demo/redemo.py(1), a demonstration program +included with the Python distribution. It allows you to enter REs and +strings, and displays whether the RE matches or fails. ‘redemo.py’ can +be quite useful when trying to debug a complicated RE. + +This HOWTO uses the standard Python interpreter for its examples. +First, run the Python interpreter, import the *note re: dd. module, and +compile a RE: + + >>> import re + >>> p = re.compile('[a-z]+') + >>> p + re.compile('[a-z]+') + +Now, you can try matching various strings against the RE ‘[a-z]+’. An +empty string shouldn’t match at all, since ‘+’ means ‘one or more +repetitions’. *note match(): 1404. should return ‘None’ in this case, +which will cause the interpreter to print no output. You can explicitly +print the result of ‘match()’ to make this clear. + + >>> p.match("") + >>> print(p.match("")) + None + +Now, let’s try it on a string that it should match, such as ‘tempo’. In +this case, *note match(): 1404. will return a *note match object: 89d, +so you should store the result in a variable for later use. + + >>> m = p.match('tempo') + >>> m + <re.Match object; span=(0, 5), match='tempo'> + +Now you can query the *note match object: 89d. for information about the +matching string. Match object instances also have several methods and +attributes; the most important ones are: + +Method/Attribute Purpose + +------------------------------------------------------------------------ + +‘group()’ Return the string matched by the RE + + +‘start()’ Return the starting position of the match + + +‘end()’ Return the ending position of the match + + +‘span()’ Return a tuple containing the (start, end) + positions of the match + + +Trying these methods will soon clarify their meaning: + + >>> m.group() + 'tempo' + >>> m.start(), m.end() + (0, 5) + >>> m.span() + (0, 5) + +*note group(): 1419. returns the substring that was matched by the RE. +*note start(): 141d. and *note end(): 141e. return the starting and +ending index of the match. *note span(): 141f. returns both start and +end indexes in a single tuple. Since the *note match(): 1404. method +only checks if the RE matches at the start of a string, ‘start()’ will +always be zero. However, the *note search(): 1405. method of patterns +scans through the string, so the match may not start at zero in that +case. + + >>> print(p.match('::: message')) + None + >>> m = p.search('::: message'); print(m) + <re.Match object; span=(4, 11), match='message'> + >>> m.group() + 'message' + >>> m.span() + (4, 11) + +In actual programs, the most common style is to store the *note match +object: 89d. in a variable, and then check if it was ‘None’. This +usually looks like: + + p = re.compile( ... ) + m = p.match( 'string goes here' ) + if m: + print('Match found: ', m.group()) + else: + print('No match') + +Two pattern methods return all of the matches for a pattern. *note +findall(): 140f. returns a list of matching strings: + + >>> p = re.compile(r'\d+') + >>> p.findall('12 drummers drumming, 11 pipers piping, 10 lords a-leaping') + ['12', '11', '10'] + +The ‘r’ prefix, making the literal a raw string literal, is needed in +this example because escape sequences in a normal “cooked” string +literal that are not recognized by Python, as opposed to regular +expressions, now result in a *note DeprecationWarning: 278. and will +eventually become a *note SyntaxError: 458. See *note The Backslash +Plague: 3efd. + +*note findall(): 140f. has to create the entire list before it can be +returned as the result. The *note finditer(): 1410. method returns a +sequence of *note match object: 89d. instances as an *note iterator: +112e.: + + >>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...') + >>> iterator + <callable_iterator object at 0x...> + >>> for match in iterator: + ... print(match.span()) + ... + (0, 2) + (22, 24) + (29, 31) + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Tools/demo/redemo.py + + +File: python.info, Node: Module-Level Functions<2>, Next: Compilation Flags, Prev: Performing Matches, Up: Using Regular Expressions + +10.8.3.4 Module-Level Functions +............................... + +You don’t have to create a pattern object and call its methods; the +*note re: dd. module also provides top-level functions called *note +match(): bb3, *note search(): bb2, *note findall(): 963, *note sub(): +47b, and so forth. These functions take the same arguments as the +corresponding pattern method with the RE string added as the first +argument, and still return either ‘None’ or a *note match object: 89d. +instance. + + >>> print(re.match(r'From\s+', 'Fromage amk')) + None + >>> re.match(r'From\s+', 'From amk Thu May 14 19:12:10 1998') + <re.Match object; span=(0, 5), match='From '> + +Under the hood, these functions simply create a pattern object for you +and call the appropriate method on it. They also store the compiled +object in a cache, so future calls using the same RE won’t need to parse +the pattern again and again. + +Should you use these module-level functions, or should you get the +pattern and call its methods yourself? If you’re accessing a regex +within a loop, pre-compiling it will save a few function calls. Outside +of loops, there’s not much difference thanks to the internal cache. + + +File: python.info, Node: Compilation Flags, Prev: Module-Level Functions<2>, Up: Using Regular Expressions + +10.8.3.5 Compilation Flags +.......................... + +Compilation flags let you modify some aspects of how regular expressions +work. Flags are available in the *note re: dd. module under two names, +a long name such as ‘IGNORECASE’ and a short, one-letter form such as +‘I’. (If you’re familiar with Perl’s pattern modifiers, the one-letter +forms use the same letters; the short form of *note re.VERBOSE: 1408. is +*note re.X: 1400, for example.) Multiple flags can be specified by +bitwise OR-ing them; ‘re.I | re.M’ sets both the ‘I’ and ‘M’ flags, for +example. + +Here’s a table of the available flags, followed by a more detailed +explanation of each one. + +Flag Meaning + +--------------------------------------------------------------------------------------- + +‘ASCII’, ‘A’ Makes several escapes like ‘\w’, ‘\b’, ‘\s’ + and ‘\d’ match only on ASCII characters with + the respective property. + + +‘DOTALL’, ‘S’ Make ‘.’ match any character, including + newlines. + + +‘IGNORECASE’, ‘I’ Do case-insensitive matches. + + +‘LOCALE’, ‘L’ Do a locale-aware match. + + +‘MULTILINE’, ‘M’ Multi-line matching, affecting ‘^’ and ‘$’. + + +‘VERBOSE’, ‘X’ (for ‘extended’) Enable verbose REs, which can be organized + more cleanly and understandably. + + + -- Data: I + + -- Data: IGNORECASE + + Perform case-insensitive matching; character class and literal + strings will match letters by ignoring case. For example, ‘[A-Z]’ + will match lowercase letters, too. Full Unicode matching also + works unless the ‘ASCII’ flag is used to disable non-ASCII matches. + When the Unicode patterns ‘[a-z]’ or ‘[A-Z]’ are used in + combination with the ‘IGNORECASE’ flag, they will match the 52 + ASCII letters and 4 additional non-ASCII letters: ‘İ’ (U+0130, + Latin capital letter I with dot above), ‘ı’ (U+0131, Latin small + letter dotless i), ‘ſ’ (U+017F, Latin small letter long s) and ‘K’ + (U+212A, Kelvin sign). ‘Spam’ will match ‘'Spam'’, ‘'spam'’, + ‘'spAM'’, or ‘'ſpam'’ (the latter is matched only in Unicode mode). + This lowercasing doesn’t take the current locale into account; it + will if you also set the ‘LOCALE’ flag. + + -- Data: L + + -- Data: LOCALE + + Make ‘\w’, ‘\W’, ‘\b’, ‘\B’ and case-insensitive matching dependent + on the current locale instead of the Unicode database. + + Locales are a feature of the C library intended to help in writing + programs that take account of language differences. For example, + if you’re processing encoded French text, you’d want to be able to + write ‘\w+’ to match words, but ‘\w’ only matches the character + class ‘[A-Za-z]’ in bytes patterns; it won’t match bytes + corresponding to ‘é’ or ‘ç’. If your system is configured properly + and a French locale is selected, certain C functions will tell the + program that the byte corresponding to ‘é’ should also be + considered a letter. Setting the ‘LOCALE’ flag when compiling a + regular expression will cause the resulting compiled object to use + these C functions for ‘\w’; this is slower, but also enables ‘\w+’ + to match French words as you’d expect. The use of this flag is + discouraged in Python 3 as the locale mechanism is very unreliable, + it only handles one “culture” at a time, and it only works with + 8-bit locales. Unicode matching is already enabled by default in + Python 3 for Unicode (str) patterns, and it is able to handle + different locales/languages. + + -- Data: M + + -- Data: MULTILINE + + (‘^’ and ‘$’ haven’t been explained yet; they’ll be introduced in + section *note More Metacharacters: 3f01.) + + Usually ‘^’ matches only at the beginning of the string, and ‘$’ + matches only at the end of the string and immediately before the + newline (if any) at the end of the string. When this flag is + specified, ‘^’ matches at the beginning of the string and at the + beginning of each line within the string, immediately following + each newline. Similarly, the ‘$’ metacharacter matches either at + the end of the string and at the end of each line (immediately + preceding each newline). + + -- Data: S + + -- Data: DOTALL + + Makes the ‘'.'’ special character match any character at all, + including a newline; without this flag, ‘'.'’ will match anything + `except' a newline. + + -- Data: A + + -- Data: ASCII + + Make ‘\w’, ‘\W’, ‘\b’, ‘\B’, ‘\s’ and ‘\S’ perform ASCII-only + matching instead of full Unicode matching. This is only meaningful + for Unicode patterns, and is ignored for byte patterns. + + -- Data: X + + -- Data: VERBOSE + + This flag allows you to write regular expressions that are more + readable by granting you more flexibility in how you can format + them. When this flag has been specified, whitespace within the RE + string is ignored, except when the whitespace is in a character + class or preceded by an unescaped backslash; this lets you organize + and indent the RE more clearly. This flag also lets you put + comments within a RE that will be ignored by the engine; comments + are marked by a ‘'#'’ that’s neither in a character class or + preceded by an unescaped backslash. + + For example, here’s a RE that uses *note re.VERBOSE: 1408.; see how + much easier it is to read? + + charref = re.compile(r""" + &[#] # Start of a numeric entity reference + ( + 0[0-7]+ # Octal form + | [0-9]+ # Decimal form + | x[0-9a-fA-F]+ # Hexadecimal form + ) + ; # Trailing semicolon + """, re.VERBOSE) + + Without the verbose setting, the RE would look like this: + + charref = re.compile("&#(0[0-7]+" + "|[0-9]+" + "|x[0-9a-fA-F]+);") + + In the above example, Python’s automatic concatenation of string + literals has been used to break up the RE into smaller pieces, but + it’s still more difficult to understand than the version using + *note re.VERBOSE: 1408. + + +File: python.info, Node: More Pattern Power, Next: Modifying Strings, Prev: Using Regular Expressions, Up: Regular Expression HOWTO + +10.8.4 More Pattern Power +------------------------- + +So far we’ve only covered a part of the features of regular expressions. +In this section, we’ll cover some new metacharacters, and how to use +groups to retrieve portions of the text that was matched. + +* Menu: + +* More Metacharacters:: +* Grouping:: +* Non-capturing and Named Groups:: +* Lookahead Assertions:: + + +File: python.info, Node: More Metacharacters, Next: Grouping, Up: More Pattern Power + +10.8.4.1 More Metacharacters +............................ + +There are some metacharacters that we haven’t covered yet. Most of them +will be covered in this section. + +Some of the remaining metacharacters to be discussed are `zero-width +assertions'. They don’t cause the engine to advance through the string; +instead, they consume no characters at all, and simply succeed or fail. +For example, ‘\b’ is an assertion that the current position is located +at a word boundary; the position isn’t changed by the ‘\b’ at all. This +means that zero-width assertions should never be repeated, because if +they match once at a given location, they can obviously be matched an +infinite number of times. + +‘|’ + + Alternation, or the “or” operator. If `A' and `B' are regular + expressions, ‘A|B’ will match any string that matches either `A' or + `B'. ‘|’ has very low precedence in order to make it work + reasonably when you’re alternating multi-character strings. + ‘Crow|Servo’ will match either ‘'Crow'’ or ‘'Servo'’, not ‘'Cro'’, + a ‘'w'’ or an ‘'S'’, and ‘'ervo'’. + + To match a literal ‘'|'’, use ‘\|’, or enclose it inside a + character class, as in ‘[|]’. + +‘^’ + + Matches at the beginning of lines. Unless the ‘MULTILINE’ flag has + been set, this will only match at the beginning of the string. In + ‘MULTILINE’ mode, this also matches immediately after each newline + within the string. + + For example, if you wish to match the word ‘From’ only at the + beginning of a line, the RE to use is ‘^From’. + + >>> print(re.search('^From', 'From Here to Eternity')) + <re.Match object; span=(0, 4), match='From'> + >>> print(re.search('^From', 'Reciting From Memory')) + None + + To match a literal ‘'^'’, use ‘\^’. + +‘$’ + + Matches at the end of a line, which is defined as either the end of + the string, or any location followed by a newline character. + + >>> print(re.search('}$', '{block}')) + <re.Match object; span=(6, 7), match='}'> + >>> print(re.search('}$', '{block} ')) + None + >>> print(re.search('}$', '{block}\n')) + <re.Match object; span=(6, 7), match='}'> + + To match a literal ‘'$'’, use ‘\$’ or enclose it inside a character + class, as in ‘[$]’. + +‘\A’ + + Matches only at the start of the string. When not in ‘MULTILINE’ + mode, ‘\A’ and ‘^’ are effectively the same. In ‘MULTILINE’ mode, + they’re different: ‘\A’ still matches only at the beginning of the + string, but ‘^’ may match at any location inside the string that + follows a newline character. + +‘\Z’ + + Matches only at the end of the string. + +‘\b’ + + Word boundary. This is a zero-width assertion that matches only at + the beginning or end of a word. A word is defined as a sequence of + alphanumeric characters, so the end of a word is indicated by + whitespace or a non-alphanumeric character. + + The following example matches ‘class’ only when it’s a complete + word; it won’t match when it’s contained inside another word. + + >>> p = re.compile(r'\bclass\b') + >>> print(p.search('no class at all')) + <re.Match object; span=(3, 8), match='class'> + >>> print(p.search('the declassified algorithm')) + None + >>> print(p.search('one subclass is')) + None + + There are two subtleties you should remember when using this + special sequence. First, this is the worst collision between + Python’s string literals and regular expression sequences. In + Python’s string literals, ‘\b’ is the backspace character, ASCII + value 8. If you’re not using raw strings, then Python will convert + the ‘\b’ to a backspace, and your RE won’t match as you expect it + to. The following example looks the same as our previous RE, but + omits the ‘'r'’ in front of the RE string. + + >>> p = re.compile('\bclass\b') + >>> print(p.search('no class at all')) + None + >>> print(p.search('\b' + 'class' + '\b')) + <re.Match object; span=(0, 7), match='\x08class\x08'> + + Second, inside a character class, where there’s no use for this + assertion, ‘\b’ represents the backspace character, for + compatibility with Python’s string literals. + +‘\B’ + + Another zero-width assertion, this is the opposite of ‘\b’, only + matching when the current position is not at a word boundary. + + +File: python.info, Node: Grouping, Next: Non-capturing and Named Groups, Prev: More Metacharacters, Up: More Pattern Power + +10.8.4.2 Grouping +................. + +Frequently you need to obtain more information than just whether the RE +matched or not. Regular expressions are often used to dissect strings +by writing a RE divided into several subgroups which match different +components of interest. For example, an RFC-822 header line is divided +into a header name and a value, separated by a ‘':'’, like this: + + From: author@example.com + User-Agent: Thunderbird 1.5.0.9 (X11/20061227) + MIME-Version: 1.0 + To: editor@example.com + +This can be handled by writing a regular expression which matches an +entire header line, and has one group which matches the header name, and +another group which matches the header’s value. + +Groups are marked by the ‘'('’, ‘')'’ metacharacters. ‘'('’ and ‘')'’ +have much the same meaning as they do in mathematical expressions; they +group together the expressions contained inside them, and you can repeat +the contents of a group with a repeating qualifier, such as ‘*’, ‘+’, +‘?’, or ‘{m,n}’. For example, ‘(ab)*’ will match zero or more +repetitions of ‘ab’. + + >>> p = re.compile('(ab)*') + >>> print(p.match('ababababab').span()) + (0, 10) + +Groups indicated with ‘'('’, ‘')'’ also capture the starting and ending +index of the text that they match; this can be retrieved by passing an +argument to *note group(): 1419, *note start(): 141d, *note end(): 141e, +and *note span(): 141f. Groups are numbered starting with 0. Group 0 +is always present; it’s the whole RE, so *note match object: 89d. +methods all have group 0 as their default argument. Later we’ll see how +to express groups that don’t capture the span of text that they match. + + >>> p = re.compile('(a)b') + >>> m = p.match('ab') + >>> m.group() + 'ab' + >>> m.group(0) + 'ab' + +Subgroups are numbered from left to right, from 1 upward. Groups can be +nested; to determine the number, just count the opening parenthesis +characters, going from left to right. + + >>> p = re.compile('(a(b)c)d') + >>> m = p.match('abcd') + >>> m.group(0) + 'abcd' + >>> m.group(1) + 'abc' + >>> m.group(2) + 'b' + +*note group(): 1419. can be passed multiple group numbers at a time, in +which case it will return a tuple containing the corresponding values +for those groups. + + >>> m.group(2,1,2) + ('b', 'abc', 'b') + +The *note groups(): 141b. method returns a tuple containing the strings +for all the subgroups, from 1 up to however many there are. + + >>> m.groups() + ('abc', 'b') + +Backreferences in a pattern allow you to specify that the contents of an +earlier capturing group must also be found at the current location in +the string. For example, ‘\1’ will succeed if the exact contents of +group 1 can be found at the current position, and fails otherwise. +Remember that Python’s string literals also use a backslash followed by +numbers to allow including arbitrary characters in a string, so be sure +to use a raw string when incorporating backreferences in a RE. + +For example, the following RE detects doubled words in a string. + + >>> p = re.compile(r'\b(\w+)\s+\1\b') + >>> p.search('Paris in the the spring').group() + 'the the' + +Backreferences like this aren’t often useful for just searching through +a string — there are few text formats which repeat data in this way — +but you’ll soon find out that they’re `very' useful when performing +string substitutions. + + +File: python.info, Node: Non-capturing and Named Groups, Next: Lookahead Assertions, Prev: Grouping, Up: More Pattern Power + +10.8.4.3 Non-capturing and Named Groups +....................................... + +Elaborate REs may use many groups, both to capture substrings of +interest, and to group and structure the RE itself. In complex REs, it +becomes difficult to keep track of the group numbers. There are two +features which help with this problem. Both of them use a common syntax +for regular expression extensions, so we’ll look at that first. + +Perl 5 is well known for its powerful additions to standard regular +expressions. For these new features the Perl developers couldn’t choose +new single-keystroke metacharacters or new special sequences beginning +with ‘\’ without making Perl’s regular expressions confusingly different +from standard REs. If they chose ‘&’ as a new metacharacter, for +example, old expressions would be assuming that ‘&’ was a regular +character and wouldn’t have escaped it by writing ‘\&’ or ‘[&]’. + +The solution chosen by the Perl developers was to use ‘(?...)’ as the +extension syntax. ‘?’ immediately after a parenthesis was a syntax +error because the ‘?’ would have nothing to repeat, so this didn’t +introduce any compatibility problems. The characters immediately after +the ‘?’ indicate what extension is being used, so ‘(?=foo)’ is one thing +(a positive lookahead assertion) and ‘(?:foo)’ is something else (a +non-capturing group containing the subexpression ‘foo’). + +Python supports several of Perl’s extensions and adds an extension +syntax to Perl’s extension syntax. If the first character after the +question mark is a ‘P’, you know that it’s an extension that’s specific +to Python. + +Now that we’ve looked at the general extension syntax, we can return to +the features that simplify working with groups in complex REs. + +Sometimes you’ll want to use a group to denote a part of a regular +expression, but aren’t interested in retrieving the group’s contents. +You can make this fact explicit by using a non-capturing group: +‘(?:...)’, where you can replace the ‘...’ with any other regular +expression. + + >>> m = re.match("([abc])+", "abc") + >>> m.groups() + ('c',) + >>> m = re.match("(?:[abc])+", "abc") + >>> m.groups() + () + +Except for the fact that you can’t retrieve the contents of what the +group matched, a non-capturing group behaves exactly the same as a +capturing group; you can put anything inside it, repeat it with a +repetition metacharacter such as ‘*’, and nest it within other groups +(capturing or non-capturing). ‘(?:...)’ is particularly useful when +modifying an existing pattern, since you can add new groups without +changing how all the other groups are numbered. It should be mentioned +that there’s no performance difference in searching between capturing +and non-capturing groups; neither form is any faster than the other. + +A more significant feature is named groups: instead of referring to them +by numbers, groups can be referenced by a name. + +The syntax for a named group is one of the Python-specific extensions: +‘(?P<name>...)’. `name' is, obviously, the name of the group. Named +groups behave exactly like capturing groups, and additionally associate +a name with a group. The *note match object: 89d. methods that deal +with capturing groups all accept either integers that refer to the group +by number or strings that contain the desired group’s name. Named +groups are still given numbers, so you can retrieve information about a +group in two ways: + + >>> p = re.compile(r'(?P<word>\b\w+\b)') + >>> m = p.search( '(((( Lots of punctuation )))' ) + >>> m.group('word') + 'Lots' + >>> m.group(1) + 'Lots' + +Additionally, you can retrieve named groups as a dictionary with *note +groupdict(): 141c.: + + >>> m = re.match(r'(?P<first>\w+) (?P<last>\w+)', 'Jane Doe') + >>> m.groupdict() + {'first': 'Jane', 'last': 'Doe'} + +Named groups are handy because they let you use easily-remembered names, +instead of having to remember numbers. Here’s an example RE from the +*note imaplib: 98. module: + + InternalDate = re.compile(r'INTERNALDATE "' + r'(?P<day>[ 123][0-9])-(?P<mon>[A-Z][a-z][a-z])-' + r'(?P<year>[0-9][0-9][0-9][0-9])' + r' (?P<hour>[0-9][0-9]):(?P<min>[0-9][0-9]):(?P<sec>[0-9][0-9])' + r' (?P<zonen>[-+])(?P<zoneh>[0-9][0-9])(?P<zonem>[0-9][0-9])' + r'"') + +It’s obviously much easier to retrieve ‘m.group('zonem')’, instead of +having to remember to retrieve group 9. + +The syntax for backreferences in an expression such as ‘(...)\1’ refers +to the number of the group. There’s naturally a variant that uses the +group name instead of the number. This is another Python extension: +‘(?P=name)’ indicates that the contents of the group called `name' +should again be matched at the current point. The regular expression +for finding doubled words, ‘\b(\w+)\s+\1\b’ can also be written as +‘\b(?P<word>\w+)\s+(?P=word)\b’: + + >>> p = re.compile(r'\b(?P<word>\w+)\s+(?P=word)\b') + >>> p.search('Paris in the the spring').group() + 'the the' + + +File: python.info, Node: Lookahead Assertions, Prev: Non-capturing and Named Groups, Up: More Pattern Power + +10.8.4.4 Lookahead Assertions +............................. + +Another zero-width assertion is the lookahead assertion. Lookahead +assertions are available in both positive and negative form, and look +like this: + +‘(?=...)’ + + Positive lookahead assertion. This succeeds if the contained + regular expression, represented here by ‘...’, successfully matches + at the current location, and fails otherwise. But, once the + contained expression has been tried, the matching engine doesn’t + advance at all; the rest of the pattern is tried right where the + assertion started. + +‘(?!...)’ + + Negative lookahead assertion. This is the opposite of the positive + assertion; it succeeds if the contained expression `doesn’t' match + at the current position in the string. + +To make this concrete, let’s look at a case where a lookahead is useful. +Consider a simple pattern to match a filename and split it apart into a +base name and an extension, separated by a ‘.’. For example, in +‘news.rc’, ‘news’ is the base name, and ‘rc’ is the filename’s +extension. + +The pattern to match this is quite simple: + +‘.*[.].*$’ + +Notice that the ‘.’ needs to be treated specially because it’s a +metacharacter, so it’s inside a character class to only match that +specific character. Also notice the trailing ‘$’; this is added to +ensure that all the rest of the string must be included in the +extension. This regular expression matches ‘foo.bar’ and ‘autoexec.bat’ +and ‘sendmail.cf’ and ‘printers.conf’. + +Now, consider complicating the problem a bit; what if you want to match +filenames where the extension is not ‘bat’? Some incorrect attempts: + +‘.*[.][^b].*$’ The first attempt above tries to exclude ‘bat’ by +requiring that the first character of the extension is not a ‘b’. This +is wrong, because the pattern also doesn’t match ‘foo.bar’. + +‘.*[.]([^b]..|.[^a].|..[^t])$’ + +The expression gets messier when you try to patch up the first solution +by requiring one of the following cases to match: the first character of +the extension isn’t ‘b’; the second character isn’t ‘a’; or the third +character isn’t ‘t’. This accepts ‘foo.bar’ and rejects ‘autoexec.bat’, +but it requires a three-letter extension and won’t accept a filename +with a two-letter extension such as ‘sendmail.cf’. We’ll complicate the +pattern again in an effort to fix it. + +‘.*[.]([^b].?.?|.[^a]?.?|..?[^t]?)$’ + +In the third attempt, the second and third letters are all made optional +in order to allow matching extensions shorter than three characters, +such as ‘sendmail.cf’. + +The pattern’s getting really complicated now, which makes it hard to +read and understand. Worse, if the problem changes and you want to +exclude both ‘bat’ and ‘exe’ as extensions, the pattern would get even +more complicated and confusing. + +A negative lookahead cuts through all this confusion: + +‘.*[.](?!bat$)[^.]*$’ The negative lookahead means: if the expression +‘bat’ doesn’t match at this point, try the rest of the pattern; if +‘bat$’ does match, the whole pattern will fail. The trailing ‘$’ is +required to ensure that something like ‘sample.batch’, where the +extension only starts with ‘bat’, will be allowed. The ‘[^.]*’ makes +sure that the pattern works when there are multiple dots in the +filename. + +Excluding another filename extension is now easy; simply add it as an +alternative inside the assertion. The following pattern excludes +filenames that end in either ‘bat’ or ‘exe’: + +‘.*[.](?!bat$|exe$)[^.]*$’ + + +File: python.info, Node: Modifying Strings, Next: Common Problems, Prev: More Pattern Power, Up: Regular Expression HOWTO + +10.8.5 Modifying Strings +------------------------ + +Up to this point, we’ve simply performed searches against a static +string. Regular expressions are also commonly used to modify strings in +various ways, using the following pattern methods: + +Method/Attribute Purpose + +--------------------------------------------------------------------------- + +‘split()’ Split the string into a list, splitting it + wherever the RE matches + + +‘sub()’ Find all substrings where the RE matches, and + replace them with a different string + + +‘subn()’ Does the same thing as ‘sub()’, but returns the + new string and the number of replacements + + +* Menu: + +* Splitting Strings:: +* Search and Replace:: + + +File: python.info, Node: Splitting Strings, Next: Search and Replace, Up: Modifying Strings + +10.8.5.1 Splitting Strings +.......................... + +The *note split(): 140e. method of a pattern splits a string apart +wherever the RE matches, returning a list of the pieces. It’s similar +to the *note split(): 7a1. method of strings but provides much more +generality in the delimiters that you can split by; string ‘split()’ +only supports splitting by whitespace or by a fixed string. As you’d +expect, there’s a module-level *note re.split(): 3ca. function, too. + + -- Method: .split (string[, maxsplit=0]) + + Split `string' by the matches of the regular expression. If + capturing parentheses are used in the RE, then their contents will + also be returned as part of the resulting list. If `maxsplit' is + nonzero, at most `maxsplit' splits are performed. + +You can limit the number of splits made, by passing a value for +`maxsplit'. When `maxsplit' is nonzero, at most `maxsplit' splits will +be made, and the remainder of the string is returned as the final +element of the list. In the following example, the delimiter is any +sequence of non-alphanumeric characters. + + >>> p = re.compile(r'\W+') + >>> p.split('This is a test, short and sweet, of split().') + ['This', 'is', 'a', 'test', 'short', 'and', 'sweet', 'of', 'split', ''] + >>> p.split('This is a test, short and sweet, of split().', 3) + ['This', 'is', 'a', 'test, short and sweet, of split().'] + +Sometimes you’re not only interested in what the text between delimiters +is, but also need to know what the delimiter was. If capturing +parentheses are used in the RE, then their values are also returned as +part of the list. Compare the following calls: + + >>> p = re.compile(r'\W+') + >>> p2 = re.compile(r'(\W+)') + >>> p.split('This... is a test.') + ['This', 'is', 'a', 'test', ''] + >>> p2.split('This... is a test.') + ['This', '... ', 'is', ' ', 'a', ' ', 'test', '.', ''] + +The module-level function *note re.split(): 3ca. adds the RE to be used +as the first argument, but is otherwise the same. + + >>> re.split(r'[\W]+', 'Words, words, words.') + ['Words', 'words', 'words', ''] + >>> re.split(r'([\W]+)', 'Words, words, words.') + ['Words', ', ', 'words', ', ', 'words', '.', ''] + >>> re.split(r'[\W]+', 'Words, words, words.', 1) + ['Words', 'words, words.'] + + +File: python.info, Node: Search and Replace, Prev: Splitting Strings, Up: Modifying Strings + +10.8.5.2 Search and Replace +........................... + +Another common task is to find all the matches for a pattern, and +replace them with a different string. The *note sub(): 1411. method +takes a replacement value, which can be either a string or a function, +and the string to be processed. + + -- Method: .sub (replacement, string[, count=0]) + + Returns the string obtained by replacing the leftmost + non-overlapping occurrences of the RE in `string' by the + replacement `replacement'. If the pattern isn’t found, `string' is + returned unchanged. + + The optional argument `count' is the maximum number of pattern + occurrences to be replaced; `count' must be a non-negative integer. + The default value of 0 means to replace all occurrences. + +Here’s a simple example of using the *note sub(): 1411. method. It +replaces colour names with the word ‘colour’: + + >>> p = re.compile('(blue|white|red)') + >>> p.sub('colour', 'blue socks and red shoes') + 'colour socks and colour shoes' + >>> p.sub('colour', 'blue socks and red shoes', count=1) + 'colour socks and red shoes' + +The *note subn(): 1412. method does the same work, but returns a 2-tuple +containing the new string value and the number of replacements that were +performed: + + >>> p = re.compile('(blue|white|red)') + >>> p.subn('colour', 'blue socks and red shoes') + ('colour socks and colour shoes', 2) + >>> p.subn('colour', 'no colours at all') + ('no colours at all', 0) + +Empty matches are replaced only when they’re not adjacent to a previous +empty match. + + >>> p = re.compile('x*') + >>> p.sub('-', 'abxd') + '-a-b--d-' + +If `replacement' is a string, any backslash escapes in it are processed. +That is, ‘\n’ is converted to a single newline character, ‘\r’ is +converted to a carriage return, and so forth. Unknown escapes such as +‘\&’ are left alone. Backreferences, such as ‘\6’, are replaced with +the substring matched by the corresponding group in the RE. This lets +you incorporate portions of the original text in the resulting +replacement string. + +This example matches the word ‘section’ followed by a string enclosed in +‘{’, ‘}’, and changes ‘section’ to ‘subsection’: + + >>> p = re.compile('section{ ( [^}]* ) }', re.VERBOSE) + >>> p.sub(r'subsection{\1}','section{First} section{second}') + 'subsection{First} subsection{second}' + +There’s also a syntax for referring to named groups as defined by the +‘(?P<name>...)’ syntax. ‘\g<name>’ will use the substring matched by +the group named ‘name’, and ‘\g<number>’ uses the corresponding group +number. ‘\g<2>’ is therefore equivalent to ‘\2’, but isn’t ambiguous in +a replacement string such as ‘\g<2>0’. (‘\20’ would be interpreted as a +reference to group 20, not a reference to group 2 followed by the +literal character ‘'0'’.) The following substitutions are all +equivalent, but use all three variations of the replacement string. + + >>> p = re.compile('section{ (?P<name> [^}]* ) }', re.VERBOSE) + >>> p.sub(r'subsection{\1}','section{First}') + 'subsection{First}' + >>> p.sub(r'subsection{\g<1>}','section{First}') + 'subsection{First}' + >>> p.sub(r'subsection{\g<name>}','section{First}') + 'subsection{First}' + +`replacement' can also be a function, which gives you even more control. +If `replacement' is a function, the function is called for every +non-overlapping occurrence of `pattern'. On each call, the function is +passed a *note match object: 89d. argument for the match and can use +this information to compute the desired replacement string and return +it. + +In the following example, the replacement function translates decimals +into hexadecimal: + + >>> def hexrepl(match): + ... "Return the hex string for a decimal number" + ... value = int(match.group()) + ... return hex(value) + ... + >>> p = re.compile(r'\d+') + >>> p.sub(hexrepl, 'Call 65490 for printing, 49152 for user code.') + 'Call 0xffd2 for printing, 0xc000 for user code.' + +When using the module-level *note re.sub(): 47b. function, the pattern +is passed as the first argument. The pattern may be provided as an +object or as a string; if you need to specify regular expression flags, +you must either use a pattern object as the first parameter, or use +embedded modifiers in the pattern string, e.g. ‘sub("(?i)b+", "x", +"bbbb BBBB")’ returns ‘'x x'’. + + +File: python.info, Node: Common Problems, Next: Feedback, Prev: Modifying Strings, Up: Regular Expression HOWTO + +10.8.6 Common Problems +---------------------- + +Regular expressions are a powerful tool for some applications, but in +some ways their behaviour isn’t intuitive and at times they don’t behave +the way you may expect them to. This section will point out some of the +most common pitfalls. + +* Menu: + +* Use String Methods:: +* match() versus search(): match versus search. +* Greedy versus Non-Greedy:: +* Using re.VERBOSE: Using re VERBOSE. + + +File: python.info, Node: Use String Methods, Next: match versus search, Up: Common Problems + +10.8.6.1 Use String Methods +........................... + +Sometimes using the *note re: dd. module is a mistake. If you’re +matching a fixed string, or a single character class, and you’re not +using any *note re: dd. features such as the *note IGNORECASE: 1407. +flag, then the full power of regular expressions may not be required. +Strings have several methods for performing operations with fixed +strings and they’re usually much faster, because the implementation is a +single small C loop that’s been optimized for the purpose, instead of +the large, more generalized regular expression engine. + +One example might be replacing a single fixed string with another one; +for example, you might replace ‘word’ with ‘deed’. *note re.sub(): 47b. +seems like the function to use for this, but consider the *note +replace(): 12ff. method. Note that ‘replace()’ will also replace ‘word’ +inside words, turning ‘swordfish’ into ‘sdeedfish’, but the naive RE +‘word’ would have done that, too. (To avoid performing the substitution +on parts of words, the pattern would have to be ‘\bword\b’, in order to +require that ‘word’ have a word boundary on either side. This takes the +job beyond ‘replace()’’s abilities.) + +Another common task is deleting every occurrence of a single character +from a string or replacing it with another single character. You might +do this with something like ‘re.sub('\n', ' ', S)’, but *note +translate(): 12fe. is capable of doing both tasks and will be faster +than any regular expression operation can be. + +In short, before turning to the *note re: dd. module, consider whether +your problem can be solved with a faster and simpler string method. + + +File: python.info, Node: match versus search, Next: Greedy versus Non-Greedy, Prev: Use String Methods, Up: Common Problems + +10.8.6.2 match() versus search() +................................ + +The *note match(): bb3. function only checks if the RE matches at the +beginning of the string while *note search(): bb2. will scan forward +through the string for a match. It’s important to keep this distinction +in mind. Remember, ‘match()’ will only report a successful match which +will start at 0; if the match wouldn’t start at zero, ‘match()’ will +`not' report it. + + >>> print(re.match('super', 'superstition').span()) + (0, 5) + >>> print(re.match('super', 'insuperable')) + None + +On the other hand, *note search(): bb2. will scan forward through the +string, reporting the first match it finds. + + >>> print(re.search('super', 'superstition').span()) + (0, 5) + >>> print(re.search('super', 'insuperable').span()) + (2, 7) + +Sometimes you’ll be tempted to keep using *note re.match(): bb3, and +just add ‘.*’ to the front of your RE. Resist this temptation and use +*note re.search(): bb2. instead. The regular expression compiler does +some analysis of REs in order to speed up the process of looking for a +match. One such analysis figures out what the first character of a +match must be; for example, a pattern starting with ‘Crow’ must match +starting with a ‘'C'’. The analysis lets the engine quickly scan +through the string looking for the starting character, only trying the +full match if a ‘'C'’ is found. + +Adding ‘.*’ defeats this optimization, requiring scanning to the end of +the string and then backtracking to find a match for the rest of the RE. +Use *note re.search(): bb2. instead. + + +File: python.info, Node: Greedy versus Non-Greedy, Next: Using re VERBOSE, Prev: match versus search, Up: Common Problems + +10.8.6.3 Greedy versus Non-Greedy +................................. + +When repeating a regular expression, as in ‘a*’, the resulting action is +to consume as much of the pattern as possible. This fact often bites +you when you’re trying to match a pair of balanced delimiters, such as +the angle brackets surrounding an HTML tag. The naive pattern for +matching a single HTML tag doesn’t work because of the greedy nature of +‘.*’. + + >>> s = '<html><head><title>Title' + >>> len(s) + 32 + >>> print(re.match('<.*>', s).span()) + (0, 32) + >>> print(re.match('<.*>', s).group()) + Title + +The RE matches the ‘'<'’ in ‘''’, and the ‘.*’ consumes the rest +of the string. There’s still more left in the RE, though, and the ‘>’ +can’t match at the end of the string, so the regular expression engine +has to backtrack character by character until it finds a match for the +‘>’. The final match extends from the ‘'<'’ in ‘''’ to the ‘'>'’ +in ‘''’, which isn’t what you want. + +In this case, the solution is to use the non-greedy qualifiers ‘*?’, +‘+?’, ‘??’, or ‘{m,n}?’, which match as `little' text as possible. In +the above example, the ‘'>'’ is tried immediately after the first ‘'<'’ +matches, and when it fails, the engine advances a character at a time, +retrying the ‘'>'’ at every step. This produces just the right result: + + >>> print(re.match('<.*?>', s).group()) + + +(Note that parsing HTML or XML with regular expressions is painful. +Quick-and-dirty patterns will handle common cases, but HTML and XML have +special cases that will break the obvious regular expression; by the +time you’ve written a regular expression that handles all of the +possible cases, the patterns will be `very' complicated. Use an HTML or +XML parser module for such tasks.) + + +File: python.info, Node: Using re VERBOSE, Prev: Greedy versus Non-Greedy, Up: Common Problems + +10.8.6.4 Using re.VERBOSE +......................... + +By now you’ve probably noticed that regular expressions are a very +compact notation, but they’re not terribly readable. REs of moderate +complexity can become lengthy collections of backslashes, parentheses, +and metacharacters, making them difficult to read and understand. + +For such REs, specifying the *note re.VERBOSE: 1408. flag when compiling +the regular expression can be helpful, because it allows you to format +the regular expression more clearly. + +The ‘re.VERBOSE’ flag has several effects. Whitespace in the regular +expression that `isn’t' inside a character class is ignored. This means +that an expression such as ‘dog | cat’ is equivalent to the less +readable ‘dog|cat’, but ‘[a b]’ will still match the characters ‘'a'’, +‘'b'’, or a space. In addition, you can also put comments inside a RE; +comments extend from a ‘#’ character to the next newline. When used +with triple-quoted strings, this enables REs to be formatted more +neatly: + + pat = re.compile(r""" + \s* # Skip leading whitespace + (?P

    [^:]+) # Header name + \s* : # Whitespace, and a colon + (?P.*?) # The header's value -- *? used to + # lose the following trailing whitespace + \s*$ # Trailing whitespace to end-of-line + """, re.VERBOSE) + +This is far more readable than: + + pat = re.compile(r"\s*(?P
    [^:]+)\s*:(?P.*?)\s*$") + + +File: python.info, Node: Feedback, Prev: Common Problems, Up: Regular Expression HOWTO + +10.8.7 Feedback +--------------- + +Regular expressions are a complicated topic. Did this document help you +understand them? Were there parts that were unclear, or Problems you +encountered that weren’t covered here? If so, please send suggestions +for improvements to the author. + +The most complete book on regular expressions is almost certainly +Jeffrey Friedl’s Mastering Regular Expressions, published by O’Reilly. +Unfortunately, it exclusively concentrates on Perl and Java’s flavours +of regular expressions, and doesn’t contain any Python material at all, +so it won’t be useful as a reference for programming in Python. (The +first edition covered Python’s now-removed ‘regex’ module, which won’t +help you much.) Consider checking it out from your library. + + +File: python.info, Node: Socket Programming HOWTO, Next: Sorting HOW TO, Prev: Regular Expression HOWTO, Up: Python HOWTOs + +10.9 Socket Programming HOWTO +============================= + + +Author: Gordon McMillan + +Abstract +........ + +Sockets are used nearly everywhere, but are one of the most severely +misunderstood technologies around. This is a 10,000 foot overview of +sockets. It’s not really a tutorial - you’ll still have work to do in +getting things operational. It doesn’t cover the fine points (and there +are a lot of them), but I hope it will give you enough background to +begin using them decently. + +* Menu: + +* Sockets:: +* Creating a Socket:: +* Using a Socket:: +* Disconnecting:: +* Non-blocking Sockets:: + + +File: python.info, Node: Sockets, Next: Creating a Socket, Up: Socket Programming HOWTO + +10.9.1 Sockets +-------------- + +I’m only going to talk about INET (i.e. IPv4) sockets, but they account +for at least 99% of the sockets in use. And I’ll only talk about STREAM +(i.e. TCP) sockets - unless you really know what you’re doing (in which +case this HOWTO isn’t for you!), you’ll get better behavior and +performance from a STREAM socket than anything else. I will try to +clear up the mystery of what a socket is, as well as some hints on how +to work with blocking and non-blocking sockets. But I’ll start by +talking about blocking sockets. You’ll need to know how they work +before dealing with non-blocking sockets. + +Part of the trouble with understanding these things is that “socket” can +mean a number of subtly different things, depending on context. So +first, let’s make a distinction between a “client” socket - an endpoint +of a conversation, and a “server” socket, which is more like a +switchboard operator. The client application (your browser, for +example) uses “client” sockets exclusively; the web server it’s talking +to uses both “server” sockets and “client” sockets. + +* Menu: + +* History:: + + +File: python.info, Node: History, Up: Sockets + +10.9.1.1 History +................ + +Of the various forms of IPC (Inter Process Communication), sockets are +by far the most popular. On any given platform, there are likely to be +other forms of IPC that are faster, but for cross-platform +communication, sockets are about the only game in town. + +They were invented in Berkeley as part of the BSD flavor of Unix. They +spread like wildfire with the Internet. With good reason — the +combination of sockets with INET makes talking to arbitrary machines +around the world unbelievably easy (at least compared to other schemes). + + +File: python.info, Node: Creating a Socket, Next: Using a Socket, Prev: Sockets, Up: Socket Programming HOWTO + +10.9.2 Creating a Socket +------------------------ + +Roughly speaking, when you clicked on the link that brought you to this +page, your browser did something like the following: + + # create an INET, STREAMing socket + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + # now connect to the web server on port 80 - the normal http port + s.connect(("www.python.org", 80)) + +When the ‘connect’ completes, the socket ‘s’ can be used to send in a +request for the text of the page. The same socket will read the reply, +and then be destroyed. That’s right, destroyed. Client sockets are +normally only used for one exchange (or a small set of sequential +exchanges). + +What happens in the web server is a bit more complex. First, the web +server creates a “server socket”: + + # create an INET, STREAMing socket + serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + # bind the socket to a public host, and a well-known port + serversocket.bind((socket.gethostname(), 80)) + # become a server socket + serversocket.listen(5) + +A couple things to notice: we used ‘socket.gethostname()’ so that the +socket would be visible to the outside world. If we had used +‘s.bind(('localhost', 80))’ or ‘s.bind(('127.0.0.1', 80))’ we would +still have a “server” socket, but one that was only visible within the +same machine. ‘s.bind(('', 80))’ specifies that the socket is reachable +by any address the machine happens to have. + +A second thing to note: low number ports are usually reserved for “well +known” services (HTTP, SNMP etc). If you’re playing around, use a nice +high number (4 digits). + +Finally, the argument to ‘listen’ tells the socket library that we want +it to queue up as many as 5 connect requests (the normal max) before +refusing outside connections. If the rest of the code is written +properly, that should be plenty. + +Now that we have a “server” socket, listening on port 80, we can enter +the mainloop of the web server: + + while True: + # accept connections from outside + (clientsocket, address) = serversocket.accept() + # now do something with the clientsocket + # in this case, we'll pretend this is a threaded server + ct = client_thread(clientsocket) + ct.run() + +There’s actually 3 general ways in which this loop could work - +dispatching a thread to handle ‘clientsocket’, create a new process to +handle ‘clientsocket’, or restructure this app to use non-blocking +sockets, and multiplex between our “server” socket and any active +‘clientsocket’s using ‘select’. More about that later. The important +thing to understand now is this: this is `all' a “server” socket does. +It doesn’t send any data. It doesn’t receive any data. It just +produces “client” sockets. Each ‘clientsocket’ is created in response +to some `other' “client” socket doing a ‘connect()’ to the host and port +we’re bound to. As soon as we’ve created that ‘clientsocket’, we go +back to listening for more connections. The two “clients” are free to +chat it up - they are using some dynamically allocated port which will +be recycled when the conversation ends. + +* Menu: + +* IPC:: + + +File: python.info, Node: IPC, Up: Creating a Socket + +10.9.2.1 IPC +............ + +If you need fast IPC between two processes on one machine, you should +look into pipes or shared memory. If you do decide to use AF_INET +sockets, bind the “server” socket to ‘'localhost'’. On most platforms, +this will take a shortcut around a couple of layers of network code and +be quite a bit faster. + +See also +........ + +The *note multiprocessing: b7. integrates cross-platform IPC into a +higher-level API. + + +File: python.info, Node: Using a Socket, Next: Disconnecting, Prev: Creating a Socket, Up: Socket Programming HOWTO + +10.9.3 Using a Socket +--------------------- + +The first thing to note, is that the web browser’s “client” socket and +the web server’s “client” socket are identical beasts. That is, this is +a “peer to peer” conversation. Or to put it another way, `as the +designer, you will have to decide what the rules of etiquette are for a +conversation'. Normally, the ‘connect’ing socket starts the +conversation, by sending in a request, or perhaps a signon. But that’s +a design decision - it’s not a rule of sockets. + +Now there are two sets of verbs to use for communication. You can use +‘send’ and ‘recv’, or you can transform your client socket into a +file-like beast and use ‘read’ and ‘write’. The latter is the way Java +presents its sockets. I’m not going to talk about it here, except to +warn you that you need to use ‘flush’ on sockets. These are buffered +“files”, and a common mistake is to ‘write’ something, and then ‘read’ +for a reply. Without a ‘flush’ in there, you may wait forever for the +reply, because the request may still be in your output buffer. + +Now we come to the major stumbling block of sockets - ‘send’ and ‘recv’ +operate on the network buffers. They do not necessarily handle all the +bytes you hand them (or expect from them), because their major focus is +handling the network buffers. In general, they return when the +associated network buffers have been filled (‘send’) or emptied +(‘recv’). They then tell you how many bytes they handled. It is `your' +responsibility to call them again until your message has been completely +dealt with. + +When a ‘recv’ returns 0 bytes, it means the other side has closed (or is +in the process of closing) the connection. You will not receive any +more data on this connection. Ever. You may be able to send data +successfully; I’ll talk more about this later. + +A protocol like HTTP uses a socket for only one transfer. The client +sends a request, then reads a reply. That’s it. The socket is +discarded. This means that a client can detect the end of the reply by +receiving 0 bytes. + +But if you plan to reuse your socket for further transfers, you need to +realize that `there is no' EOT (End of Transfer) `on a socket.' I +repeat: if a socket ‘send’ or ‘recv’ returns after handling 0 bytes, the +connection has been broken. If the connection has `not' been broken, +you may wait on a ‘recv’ forever, because the socket will `not' tell you +that there’s nothing more to read (for now). Now if you think about +that a bit, you’ll come to realize a fundamental truth of sockets: +`messages must either be fixed length' (yuck), `or be delimited' +(shrug), `or indicate how long they are' (much better), `or end by +shutting down the connection'. The choice is entirely yours, (but some +ways are righter than others). + +Assuming you don’t want to end the connection, the simplest solution is +a fixed length message: + + class MySocket: + """demonstration class only + - coded for clarity, not efficiency + """ + + def __init__(self, sock=None): + if sock is None: + self.sock = socket.socket( + socket.AF_INET, socket.SOCK_STREAM) + else: + self.sock = sock + + def connect(self, host, port): + self.sock.connect((host, port)) + + def mysend(self, msg): + totalsent = 0 + while totalsent < MSGLEN: + sent = self.sock.send(msg[totalsent:]) + if sent == 0: + raise RuntimeError("socket connection broken") + totalsent = totalsent + sent + + def myreceive(self): + chunks = [] + bytes_recd = 0 + while bytes_recd < MSGLEN: + chunk = self.sock.recv(min(MSGLEN - bytes_recd, 2048)) + if chunk == b'': + raise RuntimeError("socket connection broken") + chunks.append(chunk) + bytes_recd = bytes_recd + len(chunk) + return b''.join(chunks) + +The sending code here is usable for almost any messaging scheme - in +Python you send strings, and you can use ‘len()’ to determine its length +(even if it has embedded ‘\0’ characters). It’s mostly the receiving +code that gets more complex. (And in C, it’s not much worse, except you +can’t use ‘strlen’ if the message has embedded ‘\0’s.) + +The easiest enhancement is to make the first character of the message an +indicator of message type, and have the type determine the length. Now +you have two ‘recv’s - the first to get (at least) that first character +so you can look up the length, and the second in a loop to get the rest. +If you decide to go the delimited route, you’ll be receiving in some +arbitrary chunk size, (4096 or 8192 is frequently a good match for +network buffer sizes), and scanning what you’ve received for a +delimiter. + +One complication to be aware of: if your conversational protocol allows +multiple messages to be sent back to back (without some kind of reply), +and you pass ‘recv’ an arbitrary chunk size, you may end up reading the +start of a following message. You’ll need to put that aside and hold +onto it, until it’s needed. + +Prefixing the message with its length (say, as 5 numeric characters) +gets more complex, because (believe it or not), you may not get all 5 +characters in one ‘recv’. In playing around, you’ll get away with it; +but in high network loads, your code will very quickly break unless you +use two ‘recv’ loops - the first to determine the length, the second to +get the data part of the message. Nasty. This is also when you’ll +discover that ‘send’ does not always manage to get rid of everything in +one pass. And despite having read this, you will eventually get bit by +it! + +In the interests of space, building your character, (and preserving my +competitive position), these enhancements are left as an exercise for +the reader. Lets move on to cleaning up. + +* Menu: + +* Binary Data:: + + +File: python.info, Node: Binary Data, Up: Using a Socket + +10.9.3.1 Binary Data +.................... + +It is perfectly possible to send binary data over a socket. The major +problem is that not all machines use the same formats for binary data. +For example, a Motorola chip will represent a 16 bit integer with the +value 1 as the two hex bytes 00 01. Intel and DEC, however, are +byte-reversed - that same 1 is 01 00. Socket libraries have calls for +converting 16 and 32 bit integers - ‘ntohl, htonl, ntohs, htons’ where +“n” means `network' and “h” means `host', “s” means `short' and “l” +means `long'. Where network order is host order, these do nothing, but +where the machine is byte-reversed, these swap the bytes around +appropriately. + +In these days of 32 bit machines, the ascii representation of binary +data is frequently smaller than the binary representation. That’s +because a surprising amount of the time, all those longs have the value +0, or maybe 1. The string “0” would be two bytes, while binary is four. +Of course, this doesn’t fit well with fixed-length messages. Decisions, +decisions. + + +File: python.info, Node: Disconnecting, Next: Non-blocking Sockets, Prev: Using a Socket, Up: Socket Programming HOWTO + +10.9.4 Disconnecting +-------------------- + +Strictly speaking, you’re supposed to use ‘shutdown’ on a socket before +you ‘close’ it. The ‘shutdown’ is an advisory to the socket at the +other end. Depending on the argument you pass it, it can mean “I’m not +going to send anymore, but I’ll still listen”, or “I’m not listening, +good riddance!”. Most socket libraries, however, are so used to +programmers neglecting to use this piece of etiquette that normally a +‘close’ is the same as ‘shutdown(); close()’. So in most situations, an +explicit ‘shutdown’ is not needed. + +One way to use ‘shutdown’ effectively is in an HTTP-like exchange. The +client sends a request and then does a ‘shutdown(1)’. This tells the +server “This client is done sending, but can still receive.” The server +can detect “EOF” by a receive of 0 bytes. It can assume it has the +complete request. The server sends a reply. If the ‘send’ completes +successfully then, indeed, the client was still receiving. + +Python takes the automatic shutdown a step further, and says that when a +socket is garbage collected, it will automatically do a ‘close’ if it’s +needed. But relying on this is a very bad habit. If your socket just +disappears without doing a ‘close’, the socket at the other end may hang +indefinitely, thinking you’re just being slow. `Please' ‘close’ your +sockets when you’re done. + +* Menu: + +* When Sockets Die:: + + +File: python.info, Node: When Sockets Die, Up: Disconnecting + +10.9.4.1 When Sockets Die +......................... + +Probably the worst thing about using blocking sockets is what happens +when the other side comes down hard (without doing a ‘close’). Your +socket is likely to hang. TCP is a reliable protocol, and it will wait +a long, long time before giving up on a connection. If you’re using +threads, the entire thread is essentially dead. There’s not much you +can do about it. As long as you aren’t doing something dumb, like +holding a lock while doing a blocking read, the thread isn’t really +consuming much in the way of resources. Do `not' try to kill the thread +- part of the reason that threads are more efficient than processes is +that they avoid the overhead associated with the automatic recycling of +resources. In other words, if you do manage to kill the thread, your +whole process is likely to be screwed up. + + +File: python.info, Node: Non-blocking Sockets, Prev: Disconnecting, Up: Socket Programming HOWTO + +10.9.5 Non-blocking Sockets +--------------------------- + +If you’ve understood the preceding, you already know most of what you +need to know about the mechanics of using sockets. You’ll still use the +same calls, in much the same ways. It’s just that, if you do it right, +your app will be almost inside-out. + +In Python, you use ‘socket.setblocking(0)’ to make it non-blocking. In +C, it’s more complex, (for one thing, you’ll need to choose between the +BSD flavor ‘O_NONBLOCK’ and the almost indistinguishable POSIX flavor +‘O_NDELAY’, which is completely different from ‘TCP_NODELAY’), but it’s +the exact same idea. You do this after creating the socket, but before +using it. (Actually, if you’re nuts, you can switch back and forth.) + +The major mechanical difference is that ‘send’, ‘recv’, ‘connect’ and +‘accept’ can return without having done anything. You have (of course) +a number of choices. You can check return code and error codes and +generally drive yourself crazy. If you don’t believe me, try it +sometime. Your app will grow large, buggy and suck CPU. So let’s skip +the brain-dead solutions and do it right. + +Use ‘select’. + +In C, coding ‘select’ is fairly complex. In Python, it’s a piece of +cake, but it’s close enough to the C version that if you understand +‘select’ in Python, you’ll have little trouble with it in C: + + ready_to_read, ready_to_write, in_error = \ + select.select( + potential_readers, + potential_writers, + potential_errs, + timeout) + +You pass ‘select’ three lists: the first contains all sockets that you +might want to try reading; the second all the sockets you might want to +try writing to, and the last (normally left empty) those that you want +to check for errors. You should note that a socket can go into more +than one list. The ‘select’ call is blocking, but you can give it a +timeout. This is generally a sensible thing to do - give it a nice long +timeout (say a minute) unless you have good reason to do otherwise. + +In return, you will get three lists. They contain the sockets that are +actually readable, writable and in error. Each of these lists is a +subset (possibly empty) of the corresponding list you passed in. + +If a socket is in the output readable list, you can be +as-close-to-certain-as-we-ever-get-in-this-business that a ‘recv’ on +that socket will return `something'. Same idea for the writable list. +You’ll be able to send `something'. Maybe not all you want to, but +`something' is better than nothing. (Actually, any reasonably healthy +socket will return as writable - it just means outbound network buffer +space is available.) + +If you have a “server” socket, put it in the potential_readers list. If +it comes out in the readable list, your ‘accept’ will (almost certainly) +work. If you have created a new socket to ‘connect’ to someone else, +put it in the potential_writers list. If it shows up in the writable +list, you have a decent chance that it has connected. + +Actually, ‘select’ can be handy even with blocking sockets. It’s one +way of determining whether you will block - the socket returns as +readable when there’s something in the buffers. However, this still +doesn’t help with the problem of determining whether the other end is +done, or just busy with something else. + +`Portability alert': On Unix, ‘select’ works both with the sockets and +files. Don’t try this on Windows. On Windows, ‘select’ works with +sockets only. Also note that in C, many of the more advanced socket +options are done differently on Windows. In fact, on Windows I usually +use threads (which work very, very well) with my sockets. + + +File: python.info, Node: Sorting HOW TO, Next: Unicode HOWTO, Prev: Socket Programming HOWTO, Up: Python HOWTOs + +10.10 Sorting HOW TO +==================== + + +Author: Andrew Dalke and Raymond Hettinger + + +Release: 0.1 + +Python lists have a built-in *note list.sort(): 445. method that +modifies the list in-place. There is also a *note sorted(): 444. +built-in function that builds a new sorted list from an iterable. + +In this document, we explore the various techniques for sorting data +using Python. + +* Menu: + +* Sorting Basics:: +* Key Functions:: +* Operator Module Functions:: +* Ascending and Descending:: +* Sort Stability and Complex Sorts:: +* The Old Way Using Decorate-Sort-Undecorate:: +* The Old Way Using the cmp Parameter:: +* Odd and Ends:: + + +File: python.info, Node: Sorting Basics, Next: Key Functions, Up: Sorting HOW TO + +10.10.1 Sorting Basics +---------------------- + +A simple ascending sort is very easy: just call the *note sorted(): 444. +function. It returns a new sorted list: + + >>> sorted([5, 2, 3, 1, 4]) + [1, 2, 3, 4, 5] + +You can also use the *note list.sort(): 445. method. It modifies the +list in-place (and returns ‘None’ to avoid confusion). Usually it’s +less convenient than *note sorted(): 444. - but if you don’t need the +original list, it’s slightly more efficient. + + >>> a = [5, 2, 3, 1, 4] + >>> a.sort() + >>> a + [1, 2, 3, 4, 5] + +Another difference is that the *note list.sort(): 445. method is only +defined for lists. In contrast, the *note sorted(): 444. function +accepts any iterable. + + >>> sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'}) + [1, 2, 3, 4, 5] + + +File: python.info, Node: Key Functions, Next: Operator Module Functions, Prev: Sorting Basics, Up: Sorting HOW TO + +10.10.2 Key Functions +--------------------- + +Both *note list.sort(): 445. and *note sorted(): 444. have a `key' +parameter to specify a function to be called on each list element prior +to making comparisons. + +For example, here’s a case-insensitive string comparison: + + >>> sorted("This is a test string from Andrew".split(), key=str.lower) + ['a', 'Andrew', 'from', 'is', 'string', 'test', 'This'] + +The value of the `key' parameter should be a function that takes a +single argument and returns a key to use for sorting purposes. This +technique is fast because the key function is called exactly once for +each input record. + +A common pattern is to sort complex objects using some of the object’s +indices as keys. For example: + + >>> student_tuples = [ + ... ('john', 'A', 15), + ... ('jane', 'B', 12), + ... ('dave', 'B', 10), + ... ] + >>> sorted(student_tuples, key=lambda student: student[2]) # sort by age + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + +The same technique works for objects with named attributes. For +example: + + >>> class Student: + ... def __init__(self, name, grade, age): + ... self.name = name + ... self.grade = grade + ... self.age = age + ... def __repr__(self): + ... return repr((self.name, self.grade, self.age)) + + >>> student_objects = [ + ... Student('john', 'A', 15), + ... Student('jane', 'B', 12), + ... Student('dave', 'B', 10), + ... ] + >>> sorted(student_objects, key=lambda student: student.age) # sort by age + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + + +File: python.info, Node: Operator Module Functions, Next: Ascending and Descending, Prev: Key Functions, Up: Sorting HOW TO + +10.10.3 Operator Module Functions +--------------------------------- + +The key-function patterns shown above are very common, so Python +provides convenience functions to make accessor functions easier and +faster. The *note operator: c2. module has *note itemgetter(): 261, +*note attrgetter(): 71b, and a *note methodcaller(): 71c. function. + +Using those functions, the above examples become simpler and faster: + + >>> from operator import itemgetter, attrgetter + + >>> sorted(student_tuples, key=itemgetter(2)) + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + + >>> sorted(student_objects, key=attrgetter('age')) + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + +The operator module functions allow multiple levels of sorting. For +example, to sort by `grade' then by `age': + + >>> sorted(student_tuples, key=itemgetter(1,2)) + [('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)] + + >>> sorted(student_objects, key=attrgetter('grade', 'age')) + [('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)] + + +File: python.info, Node: Ascending and Descending, Next: Sort Stability and Complex Sorts, Prev: Operator Module Functions, Up: Sorting HOW TO + +10.10.4 Ascending and Descending +-------------------------------- + +Both *note list.sort(): 445. and *note sorted(): 444. accept a `reverse' +parameter with a boolean value. This is used to flag descending sorts. +For example, to get the student data in reverse `age' order: + + >>> sorted(student_tuples, key=itemgetter(2), reverse=True) + [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)] + + >>> sorted(student_objects, key=attrgetter('age'), reverse=True) + [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)] + + +File: python.info, Node: Sort Stability and Complex Sorts, Next: The Old Way Using Decorate-Sort-Undecorate, Prev: Ascending and Descending, Up: Sorting HOW TO + +10.10.5 Sort Stability and Complex Sorts +---------------------------------------- + +Sorts are guaranteed to be stable(1). That means that when multiple +records have the same key, their original order is preserved. + + >>> data = [('red', 1), ('blue', 1), ('red', 2), ('blue', 2)] + >>> sorted(data, key=itemgetter(0)) + [('blue', 1), ('blue', 2), ('red', 1), ('red', 2)] + +Notice how the two records for `blue' retain their original order so +that ‘('blue', 1)’ is guaranteed to precede ‘('blue', 2)’. + +This wonderful property lets you build complex sorts in a series of +sorting steps. For example, to sort the student data by descending +`grade' and then ascending `age', do the `age' sort first and then sort +again using `grade': + + >>> s = sorted(student_objects, key=attrgetter('age')) # sort on secondary key + >>> sorted(s, key=attrgetter('grade'), reverse=True) # now sort on primary key, descending + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + +This can be abstracted out into a wrapper function that can take a list +and tuples of field and order to sort them on multiple passes. + + >>> def multisort(xs, specs): + ... for key, reverse in reversed(specs): + ... xs.sort(key=attrgetter(key), reverse=reverse) + ... return xs + + >>> multisort(list(student_objects), (('grade', True), ('age', False))) + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + +The Timsort(2) algorithm used in Python does multiple sorts efficiently +because it can take advantage of any ordering already present in a +dataset. + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Sorting_algorithm#Stability + + (2) https://en.wikipedia.org/wiki/Timsort + + +File: python.info, Node: The Old Way Using Decorate-Sort-Undecorate, Next: The Old Way Using the cmp Parameter, Prev: Sort Stability and Complex Sorts, Up: Sorting HOW TO + +10.10.6 The Old Way Using Decorate-Sort-Undecorate +-------------------------------------------------- + +This idiom is called Decorate-Sort-Undecorate after its three steps: + + * First, the initial list is decorated with new values that control + the sort order. + + * Second, the decorated list is sorted. + + * Finally, the decorations are removed, creating a list that contains + only the initial values in the new order. + +For example, to sort the student data by `grade' using the DSU approach: + + >>> decorated = [(student.grade, i, student) for i, student in enumerate(student_objects)] + >>> decorated.sort() + >>> [student for grade, i, student in decorated] # undecorate + [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)] + +This idiom works because tuples are compared lexicographically; the +first items are compared; if they are the same then the second items are +compared, and so on. + +It is not strictly necessary in all cases to include the index `i' in +the decorated list, but including it gives two benefits: + + * The sort is stable – if two items have the same key, their order + will be preserved in the sorted list. + + * The original items do not have to be comparable because the + ordering of the decorated tuples will be determined by at most the + first two items. So for example the original list could contain + complex numbers which cannot be sorted directly. + +Another name for this idiom is Schwartzian transform(1), after Randal L. +Schwartz, who popularized it among Perl programmers. + +Now that Python sorting provides key-functions, this technique is not +often needed. + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Schwartzian_transform + + +File: python.info, Node: The Old Way Using the cmp Parameter, Next: Odd and Ends, Prev: The Old Way Using Decorate-Sort-Undecorate, Up: Sorting HOW TO + +10.10.7 The Old Way Using the `cmp' Parameter +--------------------------------------------- + +Many constructs given in this HOWTO assume Python 2.4 or later. Before +that, there was no *note sorted(): 444. builtin and *note list.sort(): +445. took no keyword arguments. Instead, all of the Py2.x versions +supported a `cmp' parameter to handle user specified comparison +functions. + +In Py3.0, the `cmp' parameter was removed entirely (as part of a larger +effort to simplify and unify the language, eliminating the conflict +between rich comparisons and the ‘__cmp__()’ magic method). + +In Py2.x, sort allowed an optional function which can be called for +doing the comparisons. That function should take two arguments to be +compared and then return a negative value for less-than, return zero if +they are equal, or return a positive value for greater-than. For +example, we can do: + + >>> def numeric_compare(x, y): + ... return x - y + >>> sorted([5, 2, 4, 1, 3], cmp=numeric_compare) + [1, 2, 3, 4, 5] + +Or you can reverse the order of comparison with: + + >>> def reverse_numeric(x, y): + ... return y - x + >>> sorted([5, 2, 4, 1, 3], cmp=reverse_numeric) + [5, 4, 3, 2, 1] + +When porting code from Python 2.x to 3.x, the situation can arise when +you have the user supplying a comparison function and you need to +convert that to a key function. The following wrapper makes that easy +to do: + + def cmp_to_key(mycmp): + 'Convert a cmp= function into a key= function' + class K: + def __init__(self, obj, *args): + self.obj = obj + def __lt__(self, other): + return mycmp(self.obj, other.obj) < 0 + def __gt__(self, other): + return mycmp(self.obj, other.obj) > 0 + def __eq__(self, other): + return mycmp(self.obj, other.obj) == 0 + def __le__(self, other): + return mycmp(self.obj, other.obj) <= 0 + def __ge__(self, other): + return mycmp(self.obj, other.obj) >= 0 + def __ne__(self, other): + return mycmp(self.obj, other.obj) != 0 + return K + +To convert to a key function, just wrap the old comparison function: + + >>> sorted([5, 2, 4, 1, 3], key=cmp_to_key(reverse_numeric)) + [5, 4, 3, 2, 1] + +In Python 3.2, the *note functools.cmp_to_key(): b56. function was added +to the *note functools: 85. module in the standard library. + + +File: python.info, Node: Odd and Ends, Prev: The Old Way Using the cmp Parameter, Up: Sorting HOW TO + +10.10.8 Odd and Ends +-------------------- + + * For locale aware sorting, use *note locale.strxfrm(): 2d93. for a + key function or *note locale.strcoll(): 2d91. for a comparison + function. + + * The `reverse' parameter still maintains sort stability (so that + records with equal keys retain the original order). Interestingly, + that effect can be simulated without the parameter by using the + builtin *note reversed(): 18e. function twice: + + >>> data = [('red', 1), ('blue', 1), ('red', 2), ('blue', 2)] + >>> standard_way = sorted(data, key=itemgetter(0), reverse=True) + >>> double_reversed = list(reversed(sorted(reversed(data), key=itemgetter(0)))) + >>> assert standard_way == double_reversed + >>> standard_way + [('red', 1), ('red', 2), ('blue', 1), ('blue', 2)] + + * The sort routines are guaranteed to use *note __lt__(): c34. when + making comparisons between two objects. So, it is easy to add a + standard sort order to a class by defining an *note __lt__(): c34. + method: + + >>> Student.__lt__ = lambda self, other: self.age < other.age + >>> sorted(student_objects) + [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] + + * Key functions need not depend directly on the objects being sorted. + A key function can also access external resources. For instance, + if the student grades are stored in a dictionary, they can be used + to sort a separate list of student names: + + >>> students = ['dave', 'john', 'jane'] + >>> newgrades = {'john': 'F', 'jane':'A', 'dave': 'C'} + >>> sorted(students, key=newgrades.__getitem__) + ['jane', 'dave', 'john'] + + +File: python.info, Node: Unicode HOWTO, Next: HOWTO Fetch Internet Resources Using The urllib Package, Prev: Sorting HOW TO, Up: Python HOWTOs + +10.11 Unicode HOWTO +=================== + + +Release: 1.12 + +This HOWTO discusses Python’s support for the Unicode specification for +representing textual data, and explains various problems that people +commonly encounter when trying to work with Unicode. + +* Menu: + +* Introduction to Unicode:: +* Python’s Unicode Support:: +* Reading and Writing Unicode Data:: +* Acknowledgements: Acknowledgements<10>. + + +File: python.info, Node: Introduction to Unicode, Next: Python’s Unicode Support, Up: Unicode HOWTO + +10.11.1 Introduction to Unicode +------------------------------- + +* Menu: + +* Definitions:: +* Encodings:: +* References: References<3>. + + +File: python.info, Node: Definitions, Next: Encodings, Up: Introduction to Unicode + +10.11.1.1 Definitions +..................... + +Today’s programs need to be able to handle a wide variety of characters. +Applications are often internationalized to display messages and output +in a variety of user-selectable languages; the same program might need +to output an error message in English, French, Japanese, Hebrew, or +Russian. Web content can be written in any of these languages and can +also include a variety of emoji symbols. Python’s string type uses the +Unicode Standard for representing characters, which lets Python programs +work with all these different possible characters. + +Unicode (‘https://www.unicode.org/’) is a specification that aims to +list every character used by human languages and give each character its +own unique code. The Unicode specifications are continually revised and +updated to add new languages and symbols. + +A `character' is the smallest possible component of a text. ‘A’, ‘B’, +‘C’, etc., are all different characters. So are ‘È’ and ‘Í’. +Characters vary depending on the language or context you’re talking +about. For example, there’s a character for “Roman Numeral One”, ‘Ⅰ’, +that’s separate from the uppercase letter ‘I’. They’ll usually look the +same, but these are two different characters that have different +meanings. + +The Unicode standard describes how characters are represented by `code +points'. A code point value is an integer in the range 0 to 0x10FFFF +(about 1.1 million values, with some 110 thousand assigned so far). In +the standard and in this document, a code point is written using the +notation ‘U+265E’ to mean the character with value ‘0x265e’ (9,822 in +decimal). + +The Unicode standard contains a lot of tables listing characters and +their corresponding code points: + + 0061 'a'; LATIN SMALL LETTER A + 0062 'b'; LATIN SMALL LETTER B + 0063 'c'; LATIN SMALL LETTER C + ... + 007B '{'; LEFT CURLY BRACKET + ... + 2167 'Ⅷ'; ROMAN NUMERAL EIGHT + 2168 'Ⅸ'; ROMAN NUMERAL NINE + ... + 265E '♞'; BLACK CHESS KNIGHT + 265F '♟'; BLACK CHESS PAWN + ... + 1F600 '😀'; GRINNING FACE + 1F609 '😉'; WINKING FACE + ... + +Strictly, these definitions imply that it’s meaningless to say ‘this is +character ‘U+265E’’. ‘U+265E’ is a code point, which represents some +particular character; in this case, it represents the character ‘BLACK +CHESS KNIGHT’, ‘♞’. In informal contexts, this distinction between code +points and characters will sometimes be forgotten. + +A character is represented on a screen or on paper by a set of graphical +elements that’s called a `glyph'. The glyph for an uppercase A, for +example, is two diagonal strokes and a horizontal stroke, though the +exact details will depend on the font being used. Most Python code +doesn’t need to worry about glyphs; figuring out the correct glyph to +display is generally the job of a GUI toolkit or a terminal’s font +renderer. + + +File: python.info, Node: Encodings, Next: References<3>, Prev: Definitions, Up: Introduction to Unicode + +10.11.1.2 Encodings +................... + +To summarize the previous section: a Unicode string is a sequence of +code points, which are numbers from 0 through ‘0x10FFFF’ (1,114,111 +decimal). This sequence of code points needs to be represented in +memory as a set of `code units', and `code units' are then mapped to +8-bit bytes. The rules for translating a Unicode string into a sequence +of bytes are called a `character encoding', or just an `encoding'. + +The first encoding you might think of is using 32-bit integers as the +code unit, and then using the CPU’s representation of 32-bit integers. +In this representation, the string “Python” might look like this: + + P y t h o n + 0x50 00 00 00 79 00 00 00 74 00 00 00 68 00 00 00 6f 00 00 00 6e 00 00 00 + 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 + +This representation is straightforward but using it presents a number of +problems. + + 1. It’s not portable; different processors order the bytes + differently. + + 2. It’s very wasteful of space. In most texts, the majority of the + code points are less than 127, or less than 255, so a lot of space + is occupied by ‘0x00’ bytes. The above string takes 24 bytes + compared to the 6 bytes needed for an ASCII representation. + Increased RAM usage doesn’t matter too much (desktop computers have + gigabytes of RAM, and strings aren’t usually that large), but + expanding our usage of disk and network bandwidth by a factor of 4 + is intolerable. + + 3. It’s not compatible with existing C functions such as ‘strlen()’, + so a new family of wide string functions would need to be used. + +Therefore this encoding isn’t used very much, and people instead choose +other encodings that are more efficient and convenient, such as UTF-8. + +UTF-8 is one of the most commonly used encodings, and Python often +defaults to using it. UTF stands for “Unicode Transformation Format”, +and the ‘8’ means that 8-bit values are used in the encoding. (There +are also UTF-16 and UTF-32 encodings, but they are less frequently used +than UTF-8.) UTF-8 uses the following rules: + + 1. If the code point is < 128, it’s represented by the corresponding + byte value. + + 2. If the code point is >= 128, it’s turned into a sequence of two, + three, or four bytes, where each byte of the sequence is between + 128 and 255. + +UTF-8 has several convenient properties: + + 1. It can handle any Unicode code point. + + 2. A Unicode string is turned into a sequence of bytes that contains + embedded zero bytes only where they represent the null character + (U+0000). This means that UTF-8 strings can be processed by C + functions such as ‘strcpy()’ and sent through protocols that can’t + handle zero bytes for anything other than end-of-string markers. + + 3. A string of ASCII text is also valid UTF-8 text. + + 4. UTF-8 is fairly compact; the majority of commonly used characters + can be represented with one or two bytes. + + 5. If bytes are corrupted or lost, it’s possible to determine the + start of the next UTF-8-encoded code point and resynchronize. It’s + also unlikely that random 8-bit data will look like valid UTF-8. + + 6. UTF-8 is a byte oriented encoding. The encoding specifies that + each character is represented by a specific sequence of one or more + bytes. This avoids the byte-ordering issues that can occur with + integer and word oriented encodings, like UTF-16 and UTF-32, where + the sequence of bytes varies depending on the hardware on which the + string was encoded. + + +File: python.info, Node: References<3>, Prev: Encodings, Up: Introduction to Unicode + +10.11.1.3 References +.................... + +The Unicode Consortium site(1) has character charts, a glossary, and PDF +versions of the Unicode specification. Be prepared for some difficult +reading. A chronology(2) of the origin and development of Unicode is +also available on the site. + +On the Computerphile Youtube channel, Tom Scott briefly discusses the +history of Unicode and UTF-8(3) (9 minutes 36 seconds). + +To help understand the standard, Jukka Korpela has written an +introductory guide(4) to reading the Unicode character tables. + +Another good introductory article(5) was written by Joel Spolsky. If +this introduction didn’t make things clear to you, you should try +reading this alternate article before continuing. + +Wikipedia entries are often helpful; see the entries for “character +encoding(6)” and UTF-8(7), for example. + + ---------- Footnotes ---------- + + (1) http://www.unicode.org + + (2) http://www.unicode.org/history/ + + (3) https://www.youtube.com/watch?v=MijmeoH9LT4 + + (4) http://jkorpela.fi/unicode/guide.html + + (5) +https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/ + + (6) https://en.wikipedia.org/wiki/Character_encoding + + (7) https://en.wikipedia.org/wiki/UTF-8 + + +File: python.info, Node: Python’s Unicode Support, Next: Reading and Writing Unicode Data, Prev: Introduction to Unicode, Up: Unicode HOWTO + +10.11.2 Python’s Unicode Support +-------------------------------- + +Now that you’ve learned the rudiments of Unicode, we can look at +Python’s Unicode features. + +* Menu: + +* The String Type:: +* Converting to Bytes:: +* Unicode Literals in Python Source Code:: +* Unicode Properties:: +* Comparing Strings:: +* Unicode Regular Expressions:: +* References: References<4>. + + +File: python.info, Node: The String Type, Next: Converting to Bytes, Up: Python’s Unicode Support + +10.11.2.1 The String Type +......................... + +Since Python 3.0, the language’s *note str: 330. type contains Unicode +characters, meaning any string created using ‘"unicode rocks!"’, +‘'unicode rocks!'’, or the triple-quoted string syntax is stored as +Unicode. + +The default encoding for Python source code is UTF-8, so you can simply +include a Unicode character in a string literal: + + try: + with open('/tmp/input.txt', 'r') as f: + ... + except OSError: + # 'File not found' error message. + print("Fichier non trouvé") + +Side note: Python 3 also supports using Unicode characters in +identifiers: + + répertoire = "/tmp/records.log" + with open(répertoire, "w") as f: + f.write("test\n") + +If you can’t enter a particular character in your editor or want to keep +the source code ASCII-only for some reason, you can also use escape +sequences in string literals. (Depending on your system, you may see +the actual capital-delta glyph instead of a u escape.) + + >>> "\N{GREEK CAPITAL LETTER DELTA}" # Using the character name + '\u0394' + >>> "\u0394" # Using a 16-bit hex value + '\u0394' + >>> "\U00000394" # Using a 32-bit hex value + '\u0394' + +In addition, one can create a string using the *note decode(): c38. +method of *note bytes: 331. This method takes an `encoding' argument, +such as ‘UTF-8’, and optionally an `errors' argument. + +The `errors' argument specifies the response when the input string can’t +be converted according to the encoding’s rules. Legal values for this +argument are ‘'strict'’ (raise a *note UnicodeDecodeError: 1fd. +exception), ‘'replace'’ (use ‘U+FFFD’, ‘REPLACEMENT CHARACTER’), +‘'ignore'’ (just leave the character out of the Unicode result), or +‘'backslashreplace'’ (inserts a ‘\xNN’ escape sequence). The following +examples show the differences: + + >>> b'\x80abc'.decode("utf-8", "strict") + Traceback (most recent call last): + ... + UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: + invalid start byte + >>> b'\x80abc'.decode("utf-8", "replace") + '\ufffdabc' + >>> b'\x80abc'.decode("utf-8", "backslashreplace") + '\\x80abc' + >>> b'\x80abc'.decode("utf-8", "ignore") + 'abc' + +Encodings are specified as strings containing the encoding’s name. +Python comes with roughly 100 different encodings; see the Python +Library Reference at *note Standard Encodings: 68f. for a list. Some +encodings have multiple names; for example, ‘'latin-1'’, ‘'iso_8859_1'’ +and ‘'8859’’ are all synonyms for the same encoding. + +One-character Unicode strings can also be created with the *note chr(): +10c7. built-in function, which takes integers and returns a Unicode +string of length 1 that contains the corresponding code point. The +reverse operation is the built-in *note ord(): 10c6. function that takes +a one-character Unicode string and returns the code point value: + + >>> chr(57344) + '\ue000' + >>> ord('\ue000') + 57344 + + +File: python.info, Node: Converting to Bytes, Next: Unicode Literals in Python Source Code, Prev: The String Type, Up: Python’s Unicode Support + +10.11.2.2 Converting to Bytes +............................. + +The opposite method of *note bytes.decode(): c38. is *note str.encode(): +c37, which returns a *note bytes: 331. representation of the Unicode +string, encoded in the requested `encoding'. + +The `errors' parameter is the same as the parameter of the *note +decode(): c38. method but supports a few more possible handlers. As +well as ‘'strict'’, ‘'ignore'’, and ‘'replace'’ (which in this case +inserts a question mark instead of the unencodable character), there is +also ‘'xmlcharrefreplace'’ (inserts an XML character reference), +‘backslashreplace’ (inserts a ‘\uNNNN’ escape sequence) and +‘namereplace’ (inserts a ‘\N{...}’ escape sequence). + +The following example shows the different results: + + >>> u = chr(40960) + 'abcd' + chr(1972) + >>> u.encode('utf-8') + b'\xea\x80\x80abcd\xde\xb4' + >>> u.encode('ascii') + Traceback (most recent call last): + ... + UnicodeEncodeError: 'ascii' codec can't encode character '\ua000' in + position 0: ordinal not in range(128) + >>> u.encode('ascii', 'ignore') + b'abcd' + >>> u.encode('ascii', 'replace') + b'?abcd?' + >>> u.encode('ascii', 'xmlcharrefreplace') + b'ꀀabcd޴' + >>> u.encode('ascii', 'backslashreplace') + b'\\ua000abcd\\u07b4' + >>> u.encode('ascii', 'namereplace') + b'\\N{YI SYLLABLE IT}abcd\\u07b4' + +The low-level routines for registering and accessing the available +encodings are found in the *note codecs: 1c. module. Implementing new +encodings also requires understanding the *note codecs: 1c. module. +However, the encoding and decoding functions returned by this module are +usually more low-level than is comfortable, and writing new encodings is +a specialized task, so the module won’t be covered in this HOWTO. + + +File: python.info, Node: Unicode Literals in Python Source Code, Next: Unicode Properties, Prev: Converting to Bytes, Up: Python’s Unicode Support + +10.11.2.3 Unicode Literals in Python Source Code +................................................ + +In Python source code, specific Unicode code points can be written using +the ‘\u’ escape sequence, which is followed by four hex digits giving +the code point. The ‘\U’ escape sequence is similar, but expects eight +hex digits, not four: + + >>> s = "a\xac\u1234\u20ac\U00008000" + ... # ^^^^ two-digit hex escape + ... # ^^^^^^ four-digit Unicode escape + ... # ^^^^^^^^^^ eight-digit Unicode escape + >>> [ord(c) for c in s] + [97, 172, 4660, 8364, 32768] + +Using escape sequences for code points greater than 127 is fine in small +doses, but becomes an annoyance if you’re using many accented +characters, as you would in a program with messages in French or some +other accent-using language. You can also assemble strings using the +*note chr(): 10c7. built-in function, but this is even more tedious. + +Ideally, you’d want to be able to write literals in your language’s +natural encoding. You could then edit Python source code with your +favorite editor which would display the accented characters naturally, +and have the right characters used at runtime. + +Python supports writing source code in UTF-8 by default, but you can use +almost any encoding if you declare the encoding being used. This is +done by including a special comment as either the first or second line +of the source file: + + #!/usr/bin/env python + # -*- coding: latin-1 -*- + + u = 'abcdé' + print(ord(u[-1])) + +The syntax is inspired by Emacs’s notation for specifying variables +local to a file. Emacs supports many different variables, but Python +only supports ‘coding’. The ‘-*-’ symbols indicate to Emacs that the +comment is special; they have no significance to Python but are a +convention. Python looks for ‘coding: name’ or ‘coding=name’ in the +comment. + +If you don’t include such a comment, the default encoding used will be +UTF-8 as already mentioned. See also PEP 263(1) for more information. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0263 + + +File: python.info, Node: Unicode Properties, Next: Comparing Strings, Prev: Unicode Literals in Python Source Code, Up: Python’s Unicode Support + +10.11.2.4 Unicode Properties +............................ + +The Unicode specification includes a database of information about code +points. For each defined code point, the information includes the +character’s name, its category, the numeric value if applicable (for +characters representing numeric concepts such as the Roman numerals, +fractions such as one-third and four-fifths, etc.). There are also +display-related properties, such as how to use the code point in +bidirectional text. + +The following program displays some information about several +characters, and prints the numeric value of one particular character: + + import unicodedata + + u = chr(233) + chr(0x0bf2) + chr(3972) + chr(6000) + chr(13231) + + for i, c in enumerate(u): + print(i, '%04x' % ord(c), unicodedata.category(c), end=" ") + print(unicodedata.name(c)) + + # Get numeric value of second character + print(unicodedata.numeric(u[1])) + +When run, this prints: + + 0 00e9 Ll LATIN SMALL LETTER E WITH ACUTE + 1 0bf2 No TAMIL NUMBER ONE THOUSAND + 2 0f84 Mn TIBETAN MARK HALANTA + 3 1770 Lo TAGBANWA LETTER SA + 4 33af So SQUARE RAD OVER S SQUARED + 1000.0 + +The category codes are abbreviations describing the nature of the +character. These are grouped into categories such as “Letter”, +“Number”, “Punctuation”, or “Symbol”, which in turn are broken up into +subcategories. To take the codes from the above output, ‘'Ll'’ means +‘Letter, lowercase’, ‘'No'’ means “Number, other”, ‘'Mn'’ is “Mark, +nonspacing”, and ‘'So'’ is “Symbol, other”. See the General Category +Values section of the Unicode Character Database documentation(1) for a +list of category codes. + + ---------- Footnotes ---------- + + (1) http://www.unicode.org/reports/tr44/#General_Category_Values + + +File: python.info, Node: Comparing Strings, Next: Unicode Regular Expressions, Prev: Unicode Properties, Up: Python’s Unicode Support + +10.11.2.5 Comparing Strings +........................... + +Unicode adds some complication to comparing strings, because the same +set of characters can be represented by different sequences of code +points. For example, a letter like ‘ê’ can be represented as a single +code point U+00EA, or as U+0065 U+0302, which is the code point for ‘e’ +followed by a code point for ‘COMBINING CIRCUMFLEX ACCENT’. These will +produce the same output when printed, but one is a string of length 1 +and the other is of length 2. + +One tool for a case-insensitive comparison is the *note casefold(): 6b0. +string method that converts a string to a case-insensitive form +following an algorithm described by the Unicode Standard. This +algorithm has special handling for characters such as the German letter +‘ß’ (code point U+00DF), which becomes the pair of lowercase letters +‘ss’. + + >>> street = 'Gürzenichstraße' + >>> street.casefold() + 'gürzenichstrasse' + +A second tool is the *note unicodedata: 11a. module’s *note normalize(): +11ed. function that converts strings to one of several normal forms, +where letters followed by a combining character are replaced with single +characters. ‘normalize()’ can be used to perform string comparisons +that won’t falsely report inequality if two strings use combining +characters differently: + + import unicodedata + + def compare_strs(s1, s2): + def NFD(s): + return unicodedata.normalize('NFD', s) + + return NFD(s1) == NFD(s2) + + single_char = 'ê' + multiple_chars = '\N{LATIN SMALL LETTER E}\N{COMBINING CIRCUMFLEX ACCENT}' + print('length of first string=', len(single_char)) + print('length of second string=', len(multiple_chars)) + print(compare_strs(single_char, multiple_chars)) + +When run, this outputs: + + $ python3 compare-strs.py + length of first string= 1 + length of second string= 2 + True + +The first argument to the *note normalize(): 11ed. function is a string +giving the desired normalization form, which can be one of ‘NFC’, +‘NFKC’, ‘NFD’, and ‘NFKD’. + +The Unicode Standard also specifies how to do caseless comparisons: + + import unicodedata + + def compare_caseless(s1, s2): + def NFD(s): + return unicodedata.normalize('NFD', s) + + return NFD(NFD(s1).casefold()) == NFD(NFD(s2).casefold()) + + # Example usage + single_char = 'ê' + multiple_chars = '\N{LATIN CAPITAL LETTER E}\N{COMBINING CIRCUMFLEX ACCENT}' + + print(compare_caseless(single_char, multiple_chars)) + +This will print ‘True’. (Why is ‘NFD()’ invoked twice? Because there +are a few characters that make ‘casefold()’ return a non-normalized +string, so the result needs to be normalized again. See section 3.13 of +the Unicode Standard for a discussion and an example.) + + +File: python.info, Node: Unicode Regular Expressions, Next: References<4>, Prev: Comparing Strings, Up: Python’s Unicode Support + +10.11.2.6 Unicode Regular Expressions +..................................... + +The regular expressions supported by the *note re: dd. module can be +provided either as bytes or strings. Some of the special character +sequences such as ‘\d’ and ‘\w’ have different meanings depending on +whether the pattern is supplied as bytes or a string. For example, ‘\d’ +will match the characters ‘[0-9]’ in bytes but in strings will match any +character that’s in the ‘'Nd'’ category. + +The string in this example has the number 57 written in both Thai and +Arabic numerals: + + import re + p = re.compile(r'\d+') + + s = "Over \u0e55\u0e57 57 flavours" + m = p.search(s) + print(repr(m.group())) + +When executed, ‘\d+’ will match the Thai numerals and print them out. +If you supply the *note re.ASCII: 3c8. flag to *note compile(): 44a, +‘\d+’ will match the substring “57” instead. + +Similarly, ‘\w’ matches a wide variety of Unicode characters but only +‘[a-zA-Z0-9_]’ in bytes or if *note re.ASCII: 3c8. is supplied, and ‘\s’ +will match either Unicode whitespace characters or ‘[ \t\n\r\f\v]’. + + +File: python.info, Node: References<4>, Prev: Unicode Regular Expressions, Up: Python’s Unicode Support + +10.11.2.7 References +.................... + +Some good alternative discussions of Python’s Unicode support are: + + * Processing Text Files in Python 3(1), by Nick Coghlan. + + * Pragmatic Unicode(2), a PyCon 2012 presentation by Ned Batchelder. + +The *note str: 330. type is described in the Python library reference at +*note Text Sequence Type — str: eb7. + +The documentation for the *note unicodedata: 11a. module. + +The documentation for the *note codecs: 1c. module. + +Marc-André Lemburg gave a presentation titled "Python and Unicode" (PDF +slides)(3) at EuroPython 2002. The slides are an excellent overview of +the design of Python 2’s Unicode features (where the Unicode string type +is called ‘unicode’ and literals start with ‘u’). + + ---------- Footnotes ---------- + + (1) +http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html + + (2) https://nedbatchelder.com/text/unipain.html + + (3) https://downloads.egenix.com/python/Unicode-EPC2002-Talk.pdf + + +File: python.info, Node: Reading and Writing Unicode Data, Next: Acknowledgements<10>, Prev: Python’s Unicode Support, Up: Unicode HOWTO + +10.11.3 Reading and Writing Unicode Data +---------------------------------------- + +Once you’ve written some code that works with Unicode data, the next +problem is input/output. How do you get Unicode strings into your +program, and how do you convert Unicode into a form suitable for storage +or transmission? + +It’s possible that you may not need to do anything depending on your +input sources and output destinations; you should check whether the +libraries used in your application support Unicode natively. XML +parsers often return Unicode data, for example. Many relational +databases also support Unicode-valued columns and can return Unicode +values from an SQL query. + +Unicode data is usually converted to a particular encoding before it +gets written to disk or sent over a socket. It’s possible to do all the +work yourself: open a file, read an 8-bit bytes object from it, and +convert the bytes with ‘bytes.decode(encoding)’. However, the manual +approach is not recommended. + +One problem is the multi-byte nature of encodings; one Unicode character +can be represented by several bytes. If you want to read the file in +arbitrary-sized chunks (say, 1024 or 4096 bytes), you need to write +error-handling code to catch the case where only part of the bytes +encoding a single Unicode character are read at the end of a chunk. One +solution would be to read the entire file into memory and then perform +the decoding, but that prevents you from working with files that are +extremely large; if you need to read a 2 GiB file, you need 2 GiB of +RAM. (More, really, since for at least a moment you’d need to have both +the encoded string and its Unicode version in memory.) + +The solution would be to use the low-level decoding interface to catch +the case of partial coding sequences. The work of implementing this has +already been done for you: the built-in *note open(): 4f0. function can +return a file-like object that assumes the file’s contents are in a +specified encoding and accepts Unicode parameters for methods such as +*note read(): 1ce1. and *note write(): 1ce4. This works through *note +open(): 4f0.’s `encoding' and `errors' parameters which are interpreted +just like those in *note str.encode(): c37. and *note bytes.decode(): +c38. + +Reading Unicode from a file is therefore simple: + + with open('unicode.txt', encoding='utf-8') as f: + for line in f: + print(repr(line)) + +It’s also possible to open files in update mode, allowing both reading +and writing: + + with open('test', encoding='utf-8', mode='w+') as f: + f.write('\u4500 blah blah blah\n') + f.seek(0) + print(repr(f.readline()[:1])) + +The Unicode character ‘U+FEFF’ is used as a byte-order mark (BOM), and +is often written as the first character of a file in order to assist +with autodetection of the file’s byte ordering. Some encodings, such as +UTF-16, expect a BOM to be present at the start of a file; when such an +encoding is used, the BOM will be automatically written as the first +character and will be silently dropped when the file is read. There are +variants of these encodings, such as ‘utf-16-le’ and ‘utf-16-be’ for +little-endian and big-endian encodings, that specify one particular byte +ordering and don’t skip the BOM. + +In some areas, it is also convention to use a “BOM” at the start of +UTF-8 encoded files; the name is misleading since UTF-8 is not +byte-order dependent. The mark simply announces that the file is +encoded in UTF-8. For reading such files, use the ‘utf-8-sig’ codec to +automatically skip the mark if present. + +* Menu: + +* Unicode filenames:: +* Tips for Writing Unicode-aware Programs:: +* References: References<5>. + + +File: python.info, Node: Unicode filenames, Next: Tips for Writing Unicode-aware Programs, Up: Reading and Writing Unicode Data + +10.11.3.1 Unicode filenames +........................... + +Most of the operating systems in common use today support filenames that +contain arbitrary Unicode characters. Usually this is implemented by +converting the Unicode string into some encoding that varies depending +on the system. Today Python is converging on using UTF-8: Python on +MacOS has used UTF-8 for several versions, and Python 3.6 switched to +using UTF-8 on Windows as well. On Unix systems, there will only be a +filesystem encoding if you’ve set the ‘LANG’ or ‘LC_CTYPE’ environment +variables; if you haven’t, the default encoding is again UTF-8. + +The *note sys.getfilesystemencoding(): 4f6. function returns the +encoding to use on your current system, in case you want to do the +encoding manually, but there’s not much reason to bother. When opening +a file for reading or writing, you can usually just provide the Unicode +string as the filename, and it will be automatically converted to the +right encoding for you: + + filename = 'filename\u4500abc' + with open(filename, 'w') as f: + f.write('blah\n') + +Functions in the *note os: c4. module such as *note os.stat(): 1f1. will +also accept Unicode filenames. + +The *note os.listdir(): a41. function returns filenames, which raises an +issue: should it return the Unicode version of filenames, or should it +return bytes containing the encoded versions? *note os.listdir(): a41. +can do both, depending on whether you provided the directory path as +bytes or a Unicode string. If you pass a Unicode string as the path, +filenames will be decoded using the filesystem’s encoding and a list of +Unicode strings will be returned, while passing a byte path will return +the filenames as bytes. For example, assuming the default filesystem +encoding is UTF-8, running the following program: + + fn = 'filename\u4500abc' + f = open(fn, 'w') + f.close() + + import os + print(os.listdir(b'.')) + print(os.listdir('.')) + +will produce the following output: + + $ python listdir-test.py + [b'filename\xe4\x94\x80abc', ...] + ['filename\u4500abc', ...] + +The first list contains UTF-8-encoded filenames, and the second list +contains the Unicode versions. + +Note that on most occasions, you should can just stick with using +Unicode with these APIs. The bytes APIs should only be used on systems +where undecodable file names can be present; that’s pretty much only +Unix systems now. + + +File: python.info, Node: Tips for Writing Unicode-aware Programs, Next: References<5>, Prev: Unicode filenames, Up: Reading and Writing Unicode Data + +10.11.3.2 Tips for Writing Unicode-aware Programs +................................................. + +This section provides some suggestions on writing software that deals +with Unicode. + +The most important tip is: + + Software should only work with Unicode strings internally, decoding + the input data as soon as possible and encoding the output only at + the end. + +If you attempt to write processing functions that accept both Unicode +and byte strings, you will find your program vulnerable to bugs wherever +you combine the two different kinds of strings. There is no automatic +encoding or decoding: if you do e.g. ‘str + bytes’, a *note TypeError: +192. will be raised. + +When using data coming from a web browser or some other untrusted +source, a common technique is to check for illegal characters in a +string before using the string in a generated command line or storing it +in a database. If you’re doing this, be careful to check the decoded +string, not the encoded bytes data; some encodings may have interesting +properties, such as not being bijective or not being fully +ASCII-compatible. This is especially true if the input data also +specifies the encoding, since the attacker can then choose a clever way +to hide malicious text in the encoded bytestream. + +* Menu: + +* Converting Between File Encodings:: +* Files in an Unknown Encoding:: + + +File: python.info, Node: Converting Between File Encodings, Next: Files in an Unknown Encoding, Up: Tips for Writing Unicode-aware Programs + +10.11.3.3 Converting Between File Encodings +........................................... + +The *note StreamRecoder: 14e3. class can transparently convert between +encodings, taking a stream that returns data in encoding #1 and behaving +like a stream returning data in encoding #2. + +For example, if you have an input file `f' that’s in Latin-1, you can +wrap it with a *note StreamRecoder: 14e3. to return bytes encoded in +UTF-8: + + new_f = codecs.StreamRecoder(f, + # en/decoder: used by read() to encode its results and + # by write() to decode its input. + codecs.getencoder('utf-8'), codecs.getdecoder('utf-8'), + + # reader/writer: used to read and write to the stream. + codecs.getreader('latin-1'), codecs.getwriter('latin-1') ) + + +File: python.info, Node: Files in an Unknown Encoding, Prev: Converting Between File Encodings, Up: Tips for Writing Unicode-aware Programs + +10.11.3.4 Files in an Unknown Encoding +...................................... + +What can you do if you need to make a change to a file, but don’t know +the file’s encoding? If you know the encoding is ASCII-compatible and +only want to examine or modify the ASCII parts, you can open the file +with the ‘surrogateescape’ error handler: + + with open(fname, 'r', encoding="ascii", errors="surrogateescape") as f: + data = f.read() + + # make changes to the string 'data' + + with open(fname + '.new', 'w', + encoding="ascii", errors="surrogateescape") as f: + f.write(data) + +The ‘surrogateescape’ error handler will decode any non-ASCII bytes as +code points in a special range running from U+DC80 to U+DCFF. These code +points will then turn back into the same bytes when the +‘surrogateescape’ error handler is used to encode the data and write it +back out. + + +File: python.info, Node: References<5>, Prev: Tips for Writing Unicode-aware Programs, Up: Reading and Writing Unicode Data + +10.11.3.5 References +.................... + +One section of Mastering Python 3 Input/Output(1), a PyCon 2010 talk by +David Beazley, discusses text processing and binary data handling. + +The PDF slides for Marc-André Lemburg’s presentation "Writing +Unicode-aware Applications in Python"(2) discuss questions of character +encodings as well as how to internationalize and localize an +application. These slides cover Python 2.x only. + +The Guts of Unicode in Python(3) is a PyCon 2013 talk by Benjamin +Peterson that discusses the internal Unicode representation in Python +3.3. + + ---------- Footnotes ---------- + + (1) http://pyvideo.org/video/289/pycon-2010–mastering-python-3-i-o + + (2) +https://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf + + (3) http://pyvideo.org/video/1768/the-guts-of-unicode-in-python + + +File: python.info, Node: Acknowledgements<10>, Prev: Reading and Writing Unicode Data, Up: Unicode HOWTO + +10.11.4 Acknowledgements +------------------------ + +The initial draft of this document was written by Andrew Kuchling. It +has since been revised further by Alexander Belopolsky, Georg Brandl, +Andrew Kuchling, and Ezio Melotti. + +Thanks to the following people who have noted errors or offered +suggestions on this article: Éric Araujo, Nicholas Bastin, Nick Coghlan, +Marius Gedminas, Kent Johnson, Ken Krugler, Marc-André Lemburg, Martin +von Löwis, Terry J. Reedy, Serhiy Storchaka, Eryk Sun, Chad Whitacre, +Graham Wideman. + + +File: python.info, Node: HOWTO Fetch Internet Resources Using The urllib Package, Next: Argparse Tutorial, Prev: Unicode HOWTO, Up: Python HOWTOs + +10.12 HOWTO Fetch Internet Resources Using The urllib Package +============================================================= + + +Author: Michael Foord(1) + + Note: There is a French translation of an earlier revision of this + HOWTO, available at urllib2 - Le Manuel manquant(2). + +* Menu: + +* Introduction: Introduction<16>. +* Fetching URLs:: +* Handling Exceptions: Handling Exceptions<2>. +* info and geturl:: +* Openers and Handlers:: +* Basic Authentication:: +* Proxies:: +* Sockets and Layers:: +* Footnotes:: + + ---------- Footnotes ---------- + + (1) http://www.voidspace.org.uk/python/index.shtml + + (2) +http://www.voidspace.org.uk/python/articles/urllib2_francais.shtml + + +File: python.info, Node: Introduction<16>, Next: Fetching URLs, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.1 Introduction +-------------------- + +Related Articles +................ + +You may also find useful the following article on fetching web resources +with Python: + + * Basic Authentication(1) + + A tutorial on `Basic Authentication', with examples in Python. + +`urllib.request' is a Python module for fetching URLs (Uniform Resource +Locators). It offers a very simple interface, in the form of the +`urlopen' function. This is capable of fetching URLs using a variety of +different protocols. It also offers a slightly more complex interface +for handling common situations - like basic authentication, cookies, +proxies and so on. These are provided by objects called handlers and +openers. + +urllib.request supports fetching URLs for many “URL schemes” (identified +by the string before the ‘":"’ in URL - for example ‘"ftp"’ is the URL +scheme of ‘"ftp://python.org/"’) using their associated network +protocols (e.g. FTP, HTTP). This tutorial focuses on the most common +case, HTTP. + +For straightforward situations `urlopen' is very easy to use. But as +soon as you encounter errors or non-trivial cases when opening HTTP +URLs, you will need some understanding of the HyperText Transfer +Protocol. The most comprehensive and authoritative reference to HTTP is +RFC 2616(2). This is a technical document and not intended to be easy +to read. This HOWTO aims to illustrate using `urllib', with enough +detail about HTTP to help you through. It is not intended to replace +the *note urllib.request: 120. docs, but is supplementary to them. + + ---------- Footnotes ---------- + + (1) http://www.voidspace.org.uk/python/articles/authentication.shtml + + (2) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: Fetching URLs, Next: Handling Exceptions<2>, Prev: Introduction<16>, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.2 Fetching URLs +--------------------- + +The simplest way to use urllib.request is as follows: + + import urllib.request + with urllib.request.urlopen('http://python.org/') as response: + html = response.read() + +If you wish to retrieve a resource via URL and store it in a temporary +location, you can do so via the *note shutil.copyfileobj(): 25d. and +*note tempfile.NamedTemporaryFile(): d49. functions: + + import shutil + import tempfile + import urllib.request + + with urllib.request.urlopen('http://python.org/') as response: + with tempfile.NamedTemporaryFile(delete=False) as tmp_file: + shutil.copyfileobj(response, tmp_file) + + with open(tmp_file.name) as html: + pass + +Many uses of urllib will be that simple (note that instead of an ‘http:’ +URL we could have used a URL starting with ‘ftp:’, ‘file:’, etc.). +However, it’s the purpose of this tutorial to explain the more +complicated cases, concentrating on HTTP. + +HTTP is based on requests and responses - the client makes requests and +servers send responses. urllib.request mirrors this with a ‘Request’ +object which represents the HTTP request you are making. In its +simplest form you create a Request object that specifies the URL you +want to fetch. Calling ‘urlopen’ with this Request object returns a +response object for the URL requested. This response is a file-like +object, which means you can for example call ‘.read()’ on the response: + + import urllib.request + + req = urllib.request.Request('http://www.voidspace.org.uk') + with urllib.request.urlopen(req) as response: + the_page = response.read() + +Note that urllib.request makes use of the same Request interface to +handle all URL schemes. For example, you can make an FTP request like +so: + + req = urllib.request.Request('ftp://example.com/') + +In the case of HTTP, there are two extra things that Request objects +allow you to do: First, you can pass data to be sent to the server. +Second, you can pass extra information (“metadata”) `about' the data or +about the request itself, to the server - this information is sent as +HTTP “headers”. Let’s look at each of these in turn. + +* Menu: + +* Data:: +* Headers:: + + +File: python.info, Node: Data, Next: Headers, Up: Fetching URLs + +10.12.2.1 Data +.............. + +Sometimes you want to send data to a URL (often the URL will refer to a +CGI (Common Gateway Interface) script or other web application). With +HTTP, this is often done using what’s known as a `POST' request. This +is often what your browser does when you submit a HTML form that you +filled in on the web. Not all POSTs have to come from forms: you can +use a POST to transmit arbitrary data to your own application. In the +common case of HTML forms, the data needs to be encoded in a standard +way, and then passed to the Request object as the ‘data’ argument. The +encoding is done using a function from the *note urllib.parse: 11f. +library. + + import urllib.parse + import urllib.request + + url = 'http://www.someserver.com/cgi-bin/register.cgi' + values = {'name' : 'Michael Foord', + 'location' : 'Northampton', + 'language' : 'Python' } + + data = urllib.parse.urlencode(values) + data = data.encode('ascii') # data should be bytes + req = urllib.request.Request(url, data) + with urllib.request.urlopen(req) as response: + the_page = response.read() + +Note that other encodings are sometimes required (e.g. for file upload +from HTML forms - see HTML Specification, Form Submission(1) for more +details). + +If you do not pass the ‘data’ argument, urllib uses a `GET' request. +One way in which GET and POST requests differ is that POST requests +often have “side-effects”: they change the state of the system in some +way (for example by placing an order with the website for a +hundredweight of tinned spam to be delivered to your door). Though the +HTTP standard makes it clear that POSTs are intended to `always' cause +side-effects, and GET requests `never' to cause side-effects, nothing +prevents a GET request from having side-effects, nor a POST requests +from having no side-effects. Data can also be passed in an HTTP GET +request by encoding it in the URL itself. + +This is done as follows: + + >>> import urllib.request + >>> import urllib.parse + >>> data = {} + >>> data['name'] = 'Somebody Here' + >>> data['location'] = 'Northampton' + >>> data['language'] = 'Python' + >>> url_values = urllib.parse.urlencode(data) + >>> print(url_values) # The order may differ from below. + name=Somebody+Here&language=Python&location=Northampton + >>> url = 'http://www.example.com/example.cgi' + >>> full_url = url + '?' + url_values + >>> data = urllib.request.urlopen(full_url) + +Notice that the full URL is created by adding a ‘?’ to the URL, followed +by the encoded values. + + ---------- Footnotes ---------- + + (1) https://www.w3.org/TR/REC-html40/interact/forms.html#h-17.13 + + +File: python.info, Node: Headers, Prev: Data, Up: Fetching URLs + +10.12.2.2 Headers +................. + +We’ll discuss here one particular HTTP header, to illustrate how to add +headers to your HTTP request. + +Some websites (1) dislike being browsed by programs, or send different +versions to different browsers (2). By default urllib identifies itself +as ‘Python-urllib/x.y’ (where ‘x’ and ‘y’ are the major and minor +version numbers of the Python release, e.g. ‘Python-urllib/2.5’), which +may confuse the site, or just plain not work. The way a browser +identifies itself is through the ‘User-Agent’ header (3). When you +create a Request object you can pass a dictionary of headers in. The +following example makes the same request as above, but identifies itself +as a version of Internet Explorer (4). + + import urllib.parse + import urllib.request + + url = 'http://www.someserver.com/cgi-bin/register.cgi' + user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' + values = {'name': 'Michael Foord', + 'location': 'Northampton', + 'language': 'Python' } + headers = {'User-Agent': user_agent} + + data = urllib.parse.urlencode(values) + data = data.encode('ascii') + req = urllib.request.Request(url, data, headers) + with urllib.request.urlopen(req) as response: + the_page = response.read() + +The response also has two useful methods. See the section on *note info +and geturl: 3f40. which comes after we have a look at what happens when +things go wrong. + + ---------- Footnotes ---------- + + (1) (1) Google for example. + + (2) (2) Browser sniffing is a very bad practice for website design - +building sites using web standards is much more sensible. Unfortunately +a lot of sites still send different versions to different browsers. + + (3) (3) The user agent for MSIE 6 is `‘Mozilla/4.0 (compatible; MSIE +6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)’' + + (4) (4) For details of more HTTP request headers, see Quick Reference +to HTTP Headers (http://jkorpela.fi/http.html). + + +File: python.info, Node: Handling Exceptions<2>, Next: info and geturl, Prev: Fetching URLs, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.3 Handling Exceptions +--------------------------- + +`urlopen' raises ‘URLError’ when it cannot handle a response (though as +usual with Python APIs, built-in exceptions such as *note ValueError: +1fb, *note TypeError: 192. etc. may also be raised). + +‘HTTPError’ is the subclass of ‘URLError’ raised in the specific case of +HTTP URLs. + +The exception classes are exported from the *note urllib.error: 11e. +module. + +* Menu: + +* URLError:: +* HTTPError:: +* Wrapping it Up:: + + +File: python.info, Node: URLError, Next: HTTPError, Up: Handling Exceptions<2> + +10.12.3.1 URLError +.................. + +Often, URLError is raised because there is no network connection (no +route to the specified server), or the specified server doesn’t exist. +In this case, the exception raised will have a ‘reason’ attribute, which +is a tuple containing an error code and a text error message. + +e.g. + + >>> req = urllib.request.Request('http://www.pretend_server.org') + >>> try: urllib.request.urlopen(req) + ... except urllib.error.URLError as e: + ... print(e.reason) + ... + (4, 'getaddrinfo failed') + + +File: python.info, Node: HTTPError, Next: Wrapping it Up, Prev: URLError, Up: Handling Exceptions<2> + +10.12.3.2 HTTPError +................... + +Every HTTP response from the server contains a numeric “status code”. +Sometimes the status code indicates that the server is unable to fulfil +the request. The default handlers will handle some of these responses +for you (for example, if the response is a “redirection” that requests +the client fetch the document from a different URL, urllib will handle +that for you). For those it can’t handle, urlopen will raise an +‘HTTPError’. Typical errors include ‘404’ (page not found), ‘403’ +(request forbidden), and ‘401’ (authentication required). + +See section 10 of RFC 2616(1) for a reference on all the HTTP error +codes. + +The ‘HTTPError’ instance raised will have an integer ‘code’ attribute, +which corresponds to the error sent by the server. + +* Menu: + +* Error Codes:: + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: Error Codes, Up: HTTPError + +10.12.3.3 Error Codes +..................... + +Because the default handlers handle redirects (codes in the 300 range), +and codes in the 100–299 range indicate success, you will usually only +see error codes in the 400–599 range. + +*note http.server.BaseHTTPRequestHandler.responses: 29d9. is a useful +dictionary of response codes in that shows all the response codes used +by RFC 2616(1). The dictionary is reproduced here for convenience + + # Table mapping response codes to messages; entries have the + # form {code: (shortmessage, longmessage)}. + responses = { + 100: ('Continue', 'Request received, please continue'), + 101: ('Switching Protocols', + 'Switching to new protocol; obey Upgrade header'), + + 200: ('OK', 'Request fulfilled, document follows'), + 201: ('Created', 'Document created, URL follows'), + 202: ('Accepted', + 'Request accepted, processing continues off-line'), + 203: ('Non-Authoritative Information', 'Request fulfilled from cache'), + 204: ('No Content', 'Request fulfilled, nothing follows'), + 205: ('Reset Content', 'Clear input form for further input.'), + 206: ('Partial Content', 'Partial content follows.'), + + 300: ('Multiple Choices', + 'Object has several resources -- see URI list'), + 301: ('Moved Permanently', 'Object moved permanently -- see URI list'), + 302: ('Found', 'Object moved temporarily -- see URI list'), + 303: ('See Other', 'Object moved -- see Method and URL list'), + 304: ('Not Modified', + 'Document has not changed since given time'), + 305: ('Use Proxy', + 'You must use proxy specified in Location to access this ' + 'resource.'), + 307: ('Temporary Redirect', + 'Object moved temporarily -- see URI list'), + + 400: ('Bad Request', + 'Bad request syntax or unsupported method'), + 401: ('Unauthorized', + 'No permission -- see authorization schemes'), + 402: ('Payment Required', + 'No payment -- see charging schemes'), + 403: ('Forbidden', + 'Request forbidden -- authorization will not help'), + 404: ('Not Found', 'Nothing matches the given URI'), + 405: ('Method Not Allowed', + 'Specified method is invalid for this server.'), + 406: ('Not Acceptable', 'URI not available in preferred format.'), + 407: ('Proxy Authentication Required', 'You must authenticate with ' + 'this proxy before proceeding.'), + 408: ('Request Timeout', 'Request timed out; try again later.'), + 409: ('Conflict', 'Request conflict.'), + 410: ('Gone', + 'URI no longer exists and has been permanently removed.'), + 411: ('Length Required', 'Client must specify Content-Length.'), + 412: ('Precondition Failed', 'Precondition in headers is false.'), + 413: ('Request Entity Too Large', 'Entity is too large.'), + 414: ('Request-URI Too Long', 'URI is too long.'), + 415: ('Unsupported Media Type', 'Entity body in unsupported format.'), + 416: ('Requested Range Not Satisfiable', + 'Cannot satisfy request range.'), + 417: ('Expectation Failed', + 'Expect condition could not be satisfied.'), + + 500: ('Internal Server Error', 'Server got itself in trouble'), + 501: ('Not Implemented', + 'Server does not support this operation'), + 502: ('Bad Gateway', 'Invalid responses from another server/proxy.'), + 503: ('Service Unavailable', + 'The server cannot process the request due to a high load'), + 504: ('Gateway Timeout', + 'The gateway server did not receive a timely response'), + 505: ('HTTP Version Not Supported', 'Cannot fulfill request.'), + } + +When an error is raised the server responds by returning an HTTP error +code `and' an error page. You can use the ‘HTTPError’ instance as a +response on the page returned. This means that as well as the code +attribute, it also has read, geturl, and info, methods as returned by +the ‘urllib.response’ module: + + >>> req = urllib.request.Request('http://www.python.org/fish.html') + >>> try: + ... urllib.request.urlopen(req) + ... except urllib.error.HTTPError as e: + ... print(e.code) + ... print(e.read()) + ... + 404 + b'\n\n\nPage Not Found\n + ... + + ---------- Footnotes ---------- + + (1) https://tools.ietf.org/html/rfc2616.html + + +File: python.info, Node: Wrapping it Up, Prev: HTTPError, Up: Handling Exceptions<2> + +10.12.3.4 Wrapping it Up +........................ + +So if you want to be prepared for ‘HTTPError’ `or' ‘URLError’ there are +two basic approaches. I prefer the second approach. + +* Menu: + +* Number 1:: +* Number 2:: + + +File: python.info, Node: Number 1, Next: Number 2, Up: Wrapping it Up + +10.12.3.5 Number 1 +.................. + + from urllib.request import Request, urlopen + from urllib.error import URLError, HTTPError + req = Request(someurl) + try: + response = urlopen(req) + except HTTPError as e: + print('The server couldn\'t fulfill the request.') + print('Error code: ', e.code) + except URLError as e: + print('We failed to reach a server.') + print('Reason: ', e.reason) + else: + # everything is fine + + Note: The ‘except HTTPError’ `must' come first, otherwise ‘except + URLError’ will `also' catch an ‘HTTPError’. + + +File: python.info, Node: Number 2, Prev: Number 1, Up: Wrapping it Up + +10.12.3.6 Number 2 +.................. + + from urllib.request import Request, urlopen + from urllib.error import URLError + req = Request(someurl) + try: + response = urlopen(req) + except URLError as e: + if hasattr(e, 'reason'): + print('We failed to reach a server.') + print('Reason: ', e.reason) + elif hasattr(e, 'code'): + print('The server couldn\'t fulfill the request.') + print('Error code: ', e.code) + else: + # everything is fine + + +File: python.info, Node: info and geturl, Next: Openers and Handlers, Prev: Handling Exceptions<2>, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.4 info and geturl +----------------------- + +The response returned by urlopen (or the ‘HTTPError’ instance) has two +useful methods ‘info()’ and ‘geturl()’ and is defined in the module +*note urllib.response: 121.. + +`geturl' - this returns the real URL of the page fetched. This is +useful because ‘urlopen’ (or the opener object used) may have followed a +redirect. The URL of the page fetched may not be the same as the URL +requested. + +`info' - this returns a dictionary-like object that describes the page +fetched, particularly the headers sent by the server. It is currently +an ‘http.client.HTTPMessage’ instance. + +Typical headers include ‘Content-length’, ‘Content-type’, and so on. +See the Quick Reference to HTTP Headers(1) for a useful listing of HTTP +headers with brief explanations of their meaning and use. + + ---------- Footnotes ---------- + + (1) http://jkorpela.fi/http.html + + +File: python.info, Node: Openers and Handlers, Next: Basic Authentication, Prev: info and geturl, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.5 Openers and Handlers +---------------------------- + +When you fetch a URL you use an opener (an instance of the perhaps +confusingly-named *note urllib.request.OpenerDirector: 2938.). Normally +we have been using the default opener - via ‘urlopen’ - but you can +create custom openers. Openers use handlers. All the “heavy lifting” +is done by the handlers. Each handler knows how to open URLs for a +particular URL scheme (http, ftp, etc.), or how to handle an aspect of +URL opening, for example HTTP redirections or HTTP cookies. + +You will want to create openers if you want to fetch URLs with specific +handlers installed, for example to get an opener that handles cookies, +or to get an opener that does not handle redirections. + +To create an opener, instantiate an ‘OpenerDirector’, and then call +‘.add_handler(some_handler_instance)’ repeatedly. + +Alternatively, you can use ‘build_opener’, which is a convenience +function for creating opener objects with a single function call. +‘build_opener’ adds several handlers by default, but provides a quick +way to add more and/or override the default handlers. + +Other sorts of handlers you might want to can handle proxies, +authentication, and other common but slightly specialised situations. + +‘install_opener’ can be used to make an ‘opener’ object the (global) +default opener. This means that calls to ‘urlopen’ will use the opener +you have installed. + +Opener objects have an ‘open’ method, which can be called directly to +fetch urls in the same way as the ‘urlopen’ function: there’s no need to +call ‘install_opener’, except as a convenience. + + +File: python.info, Node: Basic Authentication, Next: Proxies, Prev: Openers and Handlers, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.6 Basic Authentication +---------------------------- + +To illustrate creating and installing a handler we will use the +‘HTTPBasicAuthHandler’. For a more detailed discussion of this subject +– including an explanation of how Basic Authentication works - see the +Basic Authentication Tutorial(1). + +When authentication is required, the server sends a header (as well as +the 401 error code) requesting authentication. This specifies the +authentication scheme and a ‘realm’. The header looks like: +‘WWW-Authenticate: SCHEME realm="REALM"’. + +e.g. + + WWW-Authenticate: Basic realm="cPanel Users" + +The client should then retry the request with the appropriate name and +password for the realm included as a header in the request. This is +‘basic authentication’. In order to simplify this process we can create +an instance of ‘HTTPBasicAuthHandler’ and an opener to use this handler. + +The ‘HTTPBasicAuthHandler’ uses an object called a password manager to +handle the mapping of URLs and realms to passwords and usernames. If +you know what the realm is (from the authentication header sent by the +server), then you can use a ‘HTTPPasswordMgr’. Frequently one doesn’t +care what the realm is. In that case, it is convenient to use +‘HTTPPasswordMgrWithDefaultRealm’. This allows you to specify a default +username and password for a URL. This will be supplied in the absence of +you providing an alternative combination for a specific realm. We +indicate this by providing ‘None’ as the realm argument to the +‘add_password’ method. + +The top-level URL is the first URL that requires authentication. URLs +“deeper” than the URL you pass to .add_password() will also match. + + # create a password manager + password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm() + + # Add the username and password. + # If we knew the realm, we could use it instead of None. + top_level_url = "http://example.com/foo/" + password_mgr.add_password(None, top_level_url, username, password) + + handler = urllib.request.HTTPBasicAuthHandler(password_mgr) + + # create "opener" (OpenerDirector instance) + opener = urllib.request.build_opener(handler) + + # use the opener to fetch a URL + opener.open(a_url) + + # Install the opener. + # Now all calls to urllib.request.urlopen use our opener. + urllib.request.install_opener(opener) + + Note: In the above example we only supplied our + ‘HTTPBasicAuthHandler’ to ‘build_opener’. By default openers have + the handlers for normal situations – ‘ProxyHandler’ (if a proxy + setting such as an ‘http_proxy’ environment variable is set), + ‘UnknownHandler’, ‘HTTPHandler’, ‘HTTPDefaultErrorHandler’, + ‘HTTPRedirectHandler’, ‘FTPHandler’, ‘FileHandler’, ‘DataHandler’, + ‘HTTPErrorProcessor’. + +‘top_level_url’ is in fact `either' a full URL (including the ‘http:’ +scheme component and the hostname and optionally the port number) e.g. +‘"http://example.com/"’ `or' an “authority” (i.e. the hostname, +optionally including the port number) e.g. ‘"example.com"’ or +‘"example.com:8080"’ (the latter example includes a port number). The +authority, if present, must NOT contain the “userinfo” component - for +example ‘"joe:password@example.com"’ is not correct. + + ---------- Footnotes ---------- + + (1) http://www.voidspace.org.uk/python/articles/authentication.shtml + + +File: python.info, Node: Proxies, Next: Sockets and Layers, Prev: Basic Authentication, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.7 Proxies +--------------- + +`urllib' will auto-detect your proxy settings and use those. This is +through the ‘ProxyHandler’, which is part of the normal handler chain +when a proxy setting is detected. Normally that’s a good thing, but +there are occasions when it may not be helpful (1). One way to do this +is to setup our own ‘ProxyHandler’, with no proxies defined. This is +done using similar steps to setting up a Basic Authentication(2) +handler: + + >>> proxy_support = urllib.request.ProxyHandler({}) + >>> opener = urllib.request.build_opener(proxy_support) + >>> urllib.request.install_opener(opener) + + Note: Currently ‘urllib.request’ `does not' support fetching of + ‘https’ locations through a proxy. However, this can be enabled by + extending urllib.request as shown in the recipe (3). + + Note: ‘HTTP_PROXY’ will be ignored if a variable ‘REQUEST_METHOD’ + is set; see the documentation on *note getproxies(): 2947. + + ---------- Footnotes ---------- + + (1) (5) In my case I have to use a proxy to access the internet at +work. If you attempt to fetch `localhost' URLs through this proxy it +blocks them. IE is set to use the proxy, which urllib picks up on. In +order to test scripts with a localhost server, I have to prevent urllib +from using the proxy. + + (2) http://www.voidspace.org.uk/python/articles/authentication.shtml + + (3) (6) urllib opener for SSL proxy (CONNECT method): ASPN Cookbook +Recipe (https://code.activestate.com/recipes/456195/). + + +File: python.info, Node: Sockets and Layers, Next: Footnotes, Prev: Proxies, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.8 Sockets and Layers +-------------------------- + +The Python support for fetching resources from the web is layered. +urllib uses the *note http.client: 94. library, which in turn uses the +socket library. + +As of Python 2.3 you can specify how long a socket should wait for a +response before timing out. This can be useful in applications which +have to fetch web pages. By default the socket module has `no timeout' +and can hang. Currently, the socket timeout is not exposed at the +http.client or urllib.request levels. However, you can set the default +timeout globally for all sockets using + + import socket + import urllib.request + + # timeout in seconds + timeout = 10 + socket.setdefaulttimeout(timeout) + + # this call to urllib.request.urlopen now uses the default timeout + # we have set in the socket module + req = urllib.request.Request('http://www.voidspace.org.uk') + response = urllib.request.urlopen(req) + +__________________________________________________________________ + + +File: python.info, Node: Footnotes, Prev: Sockets and Layers, Up: HOWTO Fetch Internet Resources Using The urllib Package + +10.12.9 Footnotes +----------------- + +This document was reviewed and revised by John Lee. + + +File: python.info, Node: Argparse Tutorial, Next: An introduction to the ipaddress module, Prev: HOWTO Fetch Internet Resources Using The urllib Package, Up: Python HOWTOs + +10.13 Argparse Tutorial +======================= + + +author: Tshepang Lekhonkhobe + +This tutorial is intended to be a gentle introduction to *note argparse: +6, the recommended command-line parsing module in the Python standard +library. + + Note: There are two other modules that fulfill the same task, + namely *note getopt: 87. (an equivalent for ‘getopt()’ from the C + language) and the deprecated *note optparse: c3. Note also that + *note argparse: 6. is based on *note optparse: c3, and therefore + very similar in terms of usage. + +* Menu: + +* Concepts:: +* The basics:: +* Introducing Positional arguments:: +* Introducing Optional arguments:: +* Combining Positional and Optional arguments:: +* Getting a little more advanced:: +* Conclusion:: + + +File: python.info, Node: Concepts, Next: The basics, Up: Argparse Tutorial + +10.13.1 Concepts +---------------- + +Let’s show the sort of functionality that we are going to explore in +this introductory tutorial by making use of the ‘ls’ command: + + $ ls + cpython devguide prog.py pypy rm-unused-function.patch + $ ls pypy + ctypes_configure demo dotviewer include lib_pypy lib-python ... + $ ls -l + total 20 + drwxr-xr-x 19 wena wena 4096 Feb 18 18:51 cpython + drwxr-xr-x 4 wena wena 4096 Feb 8 12:04 devguide + -rwxr-xr-x 1 wena wena 535 Feb 19 00:05 prog.py + drwxr-xr-x 14 wena wena 4096 Feb 7 00:59 pypy + -rw-r--r-- 1 wena wena 741 Feb 18 01:01 rm-unused-function.patch + $ ls --help + Usage: ls [OPTION]... [FILE]... + List information about the FILEs (the current directory by default). + Sort entries alphabetically if none of -cftuvSUX nor --sort is specified. + ... + +A few concepts we can learn from the four commands: + + * The ‘ls’ command is useful when run without any options at all. It + defaults to displaying the contents of the current directory. + + * If we want beyond what it provides by default, we tell it a bit + more. In this case, we want it to display a different directory, + ‘pypy’. What we did is specify what is known as a positional + argument. It’s named so because the program should know what to do + with the value, solely based on where it appears on the command + line. This concept is more relevant to a command like ‘cp’, whose + most basic usage is ‘cp SRC DEST’. The first position is `what you + want copied,' and the second position is `where you want it copied + to'. + + * Now, say we want to change behaviour of the program. In our + example, we display more info for each file instead of just showing + the file names. The ‘-l’ in that case is known as an optional + argument. + + * That’s a snippet of the help text. It’s very useful in that you + can come across a program you have never used before, and can + figure out how it works simply by reading its help text. + + +File: python.info, Node: The basics, Next: Introducing Positional arguments, Prev: Concepts, Up: Argparse Tutorial + +10.13.2 The basics +------------------ + +Let us start with a very simple example which does (almost) nothing: + + import argparse + parser = argparse.ArgumentParser() + parser.parse_args() + +Following is a result of running the code: + + $ python3 prog.py + $ python3 prog.py --help + usage: prog.py [-h] + + optional arguments: + -h, --help show this help message and exit + $ python3 prog.py --verbose + usage: prog.py [-h] + prog.py: error: unrecognized arguments: --verbose + $ python3 prog.py foo + usage: prog.py [-h] + prog.py: error: unrecognized arguments: foo + +Here is what is happening: + + * Running the script without any options results in nothing displayed + to stdout. Not so useful. + + * The second one starts to display the usefulness of the *note + argparse: 6. module. We have done almost nothing, but already we + get a nice help message. + + * The ‘--help’ option, which can also be shortened to ‘-h’, is the + only option we get for free (i.e. no need to specify it). + Specifying anything else results in an error. But even then, we do + get a useful usage message, also for free. + + +File: python.info, Node: Introducing Positional arguments, Next: Introducing Optional arguments, Prev: The basics, Up: Argparse Tutorial + +10.13.3 Introducing Positional arguments +---------------------------------------- + +An example: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("echo") + args = parser.parse_args() + print(args.echo) + +And running the code: + + $ python3 prog.py + usage: prog.py [-h] echo + prog.py: error: the following arguments are required: echo + $ python3 prog.py --help + usage: prog.py [-h] echo + + positional arguments: + echo + + optional arguments: + -h, --help show this help message and exit + $ python3 prog.py foo + foo + +Here is what’s happening: + + * We’ve added the ‘add_argument()’ method, which is what we use to + specify which command-line options the program is willing to + accept. In this case, I’ve named it ‘echo’ so that it’s in line + with its function. + + * Calling our program now requires us to specify an option. + + * The ‘parse_args()’ method actually returns some data from the + options specified, in this case, ‘echo’. + + * The variable is some form of ‘magic’ that *note argparse: 6. + performs for free (i.e. no need to specify which variable that + value is stored in). You will also notice that its name matches + the string argument given to the method, ‘echo’. + +Note however that, although the help display looks nice and all, it +currently is not as helpful as it can be. For example we see that we +got ‘echo’ as a positional argument, but we don’t know what it does, +other than by guessing or by reading the source code. So, let’s make it +a bit more useful: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("echo", help="echo the string you use here") + args = parser.parse_args() + print(args.echo) + +And we get: + + $ python3 prog.py -h + usage: prog.py [-h] echo + + positional arguments: + echo echo the string you use here + + optional arguments: + -h, --help show this help message and exit + +Now, how about doing something even more useful: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", help="display a square of a given number") + args = parser.parse_args() + print(args.square**2) + +Following is a result of running the code: + + $ python3 prog.py 4 + Traceback (most recent call last): + File "prog.py", line 5, in + print(args.square**2) + TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int' + +That didn’t go so well. That’s because *note argparse: 6. treats the +options we give it as strings, unless we tell it otherwise. So, let’s +tell *note argparse: 6. to treat that input as an integer: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", help="display a square of a given number", + type=int) + args = parser.parse_args() + print(args.square**2) + +Following is a result of running the code: + + $ python3 prog.py 4 + 16 + $ python3 prog.py four + usage: prog.py [-h] square + prog.py: error: argument square: invalid int value: 'four' + +That went well. The program now even helpfully quits on bad illegal +input before proceeding. + + +File: python.info, Node: Introducing Optional arguments, Next: Combining Positional and Optional arguments, Prev: Introducing Positional arguments, Up: Argparse Tutorial + +10.13.4 Introducing Optional arguments +-------------------------------------- + +So far we have been playing with positional arguments. Let us have a +look on how to add optional ones: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("--verbosity", help="increase output verbosity") + args = parser.parse_args() + if args.verbosity: + print("verbosity turned on") + +And the output: + + $ python3 prog.py --verbosity 1 + verbosity turned on + $ python3 prog.py + $ python3 prog.py --help + usage: prog.py [-h] [--verbosity VERBOSITY] + + optional arguments: + -h, --help show this help message and exit + --verbosity VERBOSITY + increase output verbosity + $ python3 prog.py --verbosity + usage: prog.py [-h] [--verbosity VERBOSITY] + prog.py: error: argument --verbosity: expected one argument + +Here is what is happening: + + * The program is written so as to display something when + ‘--verbosity’ is specified and display nothing when not. + + * To show that the option is actually optional, there is no error + when running the program without it. Note that by default, if an + optional argument isn’t used, the relevant variable, in this case + ‘args.verbosity’, is given ‘None’ as a value, which is the reason + it fails the truth test of the *note if: de7. statement. + + * The help message is a bit different. + + * When using the ‘--verbosity’ option, one must also specify some + value, any value. + +The above example accepts arbitrary integer values for ‘--verbosity’, +but for our simple program, only two values are actually useful, ‘True’ +or ‘False’. Let’s modify the code accordingly: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("--verbose", help="increase output verbosity", + action="store_true") + args = parser.parse_args() + if args.verbose: + print("verbosity turned on") + +And the output: + + $ python3 prog.py --verbose + verbosity turned on + $ python3 prog.py --verbose 1 + usage: prog.py [-h] [--verbose] + prog.py: error: unrecognized arguments: 1 + $ python3 prog.py --help + usage: prog.py [-h] [--verbose] + + optional arguments: + -h, --help show this help message and exit + --verbose increase output verbosity + +Here is what is happening: + + * The option is now more of a flag than something that requires a + value. We even changed the name of the option to match that idea. + Note that we now specify a new keyword, ‘action’, and give it the + value ‘"store_true"’. This means that, if the option is specified, + assign the value ‘True’ to ‘args.verbose’. Not specifying it + implies ‘False’. + + * It complains when you specify a value, in true spirit of what flags + actually are. + + * Notice the different help text. + +* Menu: + +* Short options:: + + +File: python.info, Node: Short options, Up: Introducing Optional arguments + +10.13.4.1 Short options +....................... + +If you are familiar with command line usage, you will notice that I +haven’t yet touched on the topic of short versions of the options. It’s +quite simple: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("-v", "--verbose", help="increase output verbosity", + action="store_true") + args = parser.parse_args() + if args.verbose: + print("verbosity turned on") + +And here goes: + + $ python3 prog.py -v + verbosity turned on + $ python3 prog.py --help + usage: prog.py [-h] [-v] + + optional arguments: + -h, --help show this help message and exit + -v, --verbose increase output verbosity + +Note that the new ability is also reflected in the help text. + + +File: python.info, Node: Combining Positional and Optional arguments, Next: Getting a little more advanced, Prev: Introducing Optional arguments, Up: Argparse Tutorial + +10.13.5 Combining Positional and Optional arguments +--------------------------------------------------- + +Our program keeps growing in complexity: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbose", action="store_true", + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbose: + print("the square of {} equals {}".format(args.square, answer)) + else: + print(answer) + +And now the output: + + $ python3 prog.py + usage: prog.py [-h] [-v] square + prog.py: error: the following arguments are required: square + $ python3 prog.py 4 + 16 + $ python3 prog.py 4 --verbose + the square of 4 equals 16 + $ python3 prog.py --verbose 4 + the square of 4 equals 16 + + * We’ve brought back a positional argument, hence the complaint. + + * Note that the order does not matter. + +How about we give this program of ours back the ability to have multiple +verbosity values, and actually get to use them: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", type=int, + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity == 2: + print("the square of {} equals {}".format(args.square, answer)) + elif args.verbosity == 1: + print("{}^2 == {}".format(args.square, answer)) + else: + print(answer) + +And the output: + + $ python3 prog.py 4 + 16 + $ python3 prog.py 4 -v + usage: prog.py [-h] [-v VERBOSITY] square + prog.py: error: argument -v/--verbosity: expected one argument + $ python3 prog.py 4 -v 1 + 4^2 == 16 + $ python3 prog.py 4 -v 2 + the square of 4 equals 16 + $ python3 prog.py 4 -v 3 + 16 + +These all look good except the last one, which exposes a bug in our +program. Let’s fix it by restricting the values the ‘--verbosity’ +option can accept: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", type=int, choices=[0, 1, 2], + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity == 2: + print("the square of {} equals {}".format(args.square, answer)) + elif args.verbosity == 1: + print("{}^2 == {}".format(args.square, answer)) + else: + print(answer) + +And the output: + + $ python3 prog.py 4 -v 3 + usage: prog.py [-h] [-v {0,1,2}] square + prog.py: error: argument -v/--verbosity: invalid choice: 3 (choose from 0, 1, 2) + $ python3 prog.py 4 -h + usage: prog.py [-h] [-v {0,1,2}] square + + positional arguments: + square display a square of a given number + + optional arguments: + -h, --help show this help message and exit + -v {0,1,2}, --verbosity {0,1,2} + increase output verbosity + +Note that the change also reflects both in the error message as well as +the help string. + +Now, let’s use a different approach of playing with verbosity, which is +pretty common. It also matches the way the CPython executable handles +its own verbosity argument (check the output of ‘python --help’): + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display the square of a given number") + parser.add_argument("-v", "--verbosity", action="count", + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity == 2: + print("the square of {} equals {}".format(args.square, answer)) + elif args.verbosity == 1: + print("{}^2 == {}".format(args.square, answer)) + else: + print(answer) + +We have introduced another action, “count”, to count the number of +occurrences of a specific optional arguments: + + $ python3 prog.py 4 + 16 + $ python3 prog.py 4 -v + 4^2 == 16 + $ python3 prog.py 4 -vv + the square of 4 equals 16 + $ python3 prog.py 4 --verbosity --verbosity + the square of 4 equals 16 + $ python3 prog.py 4 -v 1 + usage: prog.py [-h] [-v] square + prog.py: error: unrecognized arguments: 1 + $ python3 prog.py 4 -h + usage: prog.py [-h] [-v] square + + positional arguments: + square display a square of a given number + + optional arguments: + -h, --help show this help message and exit + -v, --verbosity increase output verbosity + $ python3 prog.py 4 -vvv + 16 + + * Yes, it’s now more of a flag (similar to ‘action="store_true"’) in + the previous version of our script. That should explain the + complaint. + + * It also behaves similar to “store_true” action. + + * Now here’s a demonstration of what the “count” action gives. + You’ve probably seen this sort of usage before. + + * And if you don’t specify the ‘-v’ flag, that flag is considered to + have ‘None’ value. + + * As should be expected, specifying the long form of the flag, we + should get the same output. + + * Sadly, our help output isn’t very informative on the new ability + our script has acquired, but that can always be fixed by improving + the documentation for our script (e.g. via the ‘help’ keyword + argument). + + * That last output exposes a bug in our program. + +Let’s fix: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", action="count", + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + + # bugfix: replace == with >= + if args.verbosity >= 2: + print("the square of {} equals {}".format(args.square, answer)) + elif args.verbosity >= 1: + print("{}^2 == {}".format(args.square, answer)) + else: + print(answer) + +And this is what it gives: + + $ python3 prog.py 4 -vvv + the square of 4 equals 16 + $ python3 prog.py 4 -vvvv + the square of 4 equals 16 + $ python3 prog.py 4 + Traceback (most recent call last): + File "prog.py", line 11, in + if args.verbosity >= 2: + TypeError: '>=' not supported between instances of 'NoneType' and 'int' + + * First output went well, and fixes the bug we had before. That is, + we want any value >= 2 to be as verbose as possible. + + * Third output not so good. + +Let’s fix that bug: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", action="count", default=0, + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity >= 2: + print("the square of {} equals {}".format(args.square, answer)) + elif args.verbosity >= 1: + print("{}^2 == {}".format(args.square, answer)) + else: + print(answer) + +We’ve just introduced yet another keyword, ‘default’. We’ve set it to +‘0’ in order to make it comparable to the other int values. Remember +that by default, if an optional argument isn’t specified, it gets the +‘None’ value, and that cannot be compared to an int value (hence the +*note TypeError: 192. exception). + +And: + + $ python3 prog.py 4 + 16 + +You can go quite far just with what we’ve learned so far, and we have +only scratched the surface. The *note argparse: 6. module is very +powerful, and we’ll explore a bit more of it before we end this +tutorial. + + +File: python.info, Node: Getting a little more advanced, Next: Conclusion, Prev: Combining Positional and Optional arguments, Up: Argparse Tutorial + +10.13.6 Getting a little more advanced +-------------------------------------- + +What if we wanted to expand our tiny program to perform other powers, +not just squares: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + parser.add_argument("-v", "--verbosity", action="count", default=0) + args = parser.parse_args() + answer = args.x**args.y + if args.verbosity >= 2: + print("{} to the power {} equals {}".format(args.x, args.y, answer)) + elif args.verbosity >= 1: + print("{}^{} == {}".format(args.x, args.y, answer)) + else: + print(answer) + +Output: + + $ python3 prog.py + usage: prog.py [-h] [-v] x y + prog.py: error: the following arguments are required: x, y + $ python3 prog.py -h + usage: prog.py [-h] [-v] x y + + positional arguments: + x the base + y the exponent + + optional arguments: + -h, --help show this help message and exit + -v, --verbosity + $ python3 prog.py 4 2 -v + 4^2 == 16 + +Notice that so far we’ve been using verbosity level to `change' the text +that gets displayed. The following example instead uses verbosity level +to display `more' text instead: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + parser.add_argument("-v", "--verbosity", action="count", default=0) + args = parser.parse_args() + answer = args.x**args.y + if args.verbosity >= 2: + print("Running '{}'".format(__file__)) + if args.verbosity >= 1: + print("{}^{} == ".format(args.x, args.y), end="") + print(answer) + +Output: + + $ python3 prog.py 4 2 + 16 + $ python3 prog.py 4 2 -v + 4^2 == 16 + $ python3 prog.py 4 2 -vv + Running 'prog.py' + 4^2 == 16 + +* Menu: + +* Conflicting options:: + + +File: python.info, Node: Conflicting options, Up: Getting a little more advanced + +10.13.6.1 Conflicting options +............................. + +So far, we have been working with two methods of an *note +argparse.ArgumentParser: 695. instance. Let’s introduce a third one, +‘add_mutually_exclusive_group()’. It allows for us to specify options +that conflict with each other. Let’s also change the rest of the +program so that the new functionality makes more sense: we’ll introduce +the ‘--quiet’ option, which will be the opposite of the ‘--verbose’ one: + + import argparse + + parser = argparse.ArgumentParser() + group = parser.add_mutually_exclusive_group() + group.add_argument("-v", "--verbose", action="store_true") + group.add_argument("-q", "--quiet", action="store_true") + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + args = parser.parse_args() + answer = args.x**args.y + + if args.quiet: + print(answer) + elif args.verbose: + print("{} to the power {} equals {}".format(args.x, args.y, answer)) + else: + print("{}^{} == {}".format(args.x, args.y, answer)) + +Our program is now simpler, and we’ve lost some functionality for the +sake of demonstration. Anyways, here’s the output: + + $ python3 prog.py 4 2 + 4^2 == 16 + $ python3 prog.py 4 2 -q + 16 + $ python3 prog.py 4 2 -v + 4 to the power 2 equals 16 + $ python3 prog.py 4 2 -vq + usage: prog.py [-h] [-v | -q] x y + prog.py: error: argument -q/--quiet: not allowed with argument -v/--verbose + $ python3 prog.py 4 2 -v --quiet + usage: prog.py [-h] [-v | -q] x y + prog.py: error: argument -q/--quiet: not allowed with argument -v/--verbose + +That should be easy to follow. I’ve added that last output so you can +see the sort of flexibility you get, i.e. mixing long form options with +short form ones. + +Before we conclude, you probably want to tell your users the main +purpose of your program, just in case they don’t know: + + import argparse + + parser = argparse.ArgumentParser(description="calculate X to the power of Y") + group = parser.add_mutually_exclusive_group() + group.add_argument("-v", "--verbose", action="store_true") + group.add_argument("-q", "--quiet", action="store_true") + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + args = parser.parse_args() + answer = args.x**args.y + + if args.quiet: + print(answer) + elif args.verbose: + print("{} to the power {} equals {}".format(args.x, args.y, answer)) + else: + print("{}^{} == {}".format(args.x, args.y, answer)) + +Note that slight difference in the usage text. Note the ‘[-v | -q]’, +which tells us that we can either use ‘-v’ or ‘-q’, but not both at the +same time: + + $ python3 prog.py --help + usage: prog.py [-h] [-v | -q] x y + + calculate X to the power of Y + + positional arguments: + x the base + y the exponent + + optional arguments: + -h, --help show this help message and exit + -v, --verbose + -q, --quiet + + +File: python.info, Node: Conclusion, Prev: Getting a little more advanced, Up: Argparse Tutorial + +10.13.7 Conclusion +------------------ + +The *note argparse: 6. module offers a lot more than shown here. Its +docs are quite detailed and thorough, and full of examples. Having gone +through this tutorial, you should easily digest them without feeling +overwhelmed. + + +File: python.info, Node: An introduction to the ipaddress module, Next: Argument Clinic How-To, Prev: Argparse Tutorial, Up: Python HOWTOs + +10.14 An introduction to the ipaddress module +============================================= + + +author: Peter Moody + + +author: Nick Coghlan + +Overview +........ + +This document aims to provide a gentle introduction to the *note +ipaddress: a2. module. It is aimed primarily at users that aren’t +already familiar with IP networking terminology, but may also be useful +to network engineers wanting an overview of how *note ipaddress: a2. +represents IP network addressing concepts. + +* Menu: + +* Creating Address/Network/Interface objects:: +* Inspecting Address/Network/Interface Objects:: +* Networks as lists of Addresses:: +* Comparisons: Comparisons<4>. +* Using IP Addresses with other modules:: +* Getting more detail when instance creation fails:: + + +File: python.info, Node: Creating Address/Network/Interface objects, Next: Inspecting Address/Network/Interface Objects, Up: An introduction to the ipaddress module + +10.14.1 Creating Address/Network/Interface objects +-------------------------------------------------- + +Since *note ipaddress: a2. is a module for inspecting and manipulating +IP addresses, the first thing you’ll want to do is create some objects. +You can use *note ipaddress: a2. to create objects from strings and +integers. + +* Menu: + +* A Note on IP Versions:: +* IP Host Addresses:: +* Defining Networks:: +* Host Interfaces:: + + +File: python.info, Node: A Note on IP Versions, Next: IP Host Addresses, Up: Creating Address/Network/Interface objects + +10.14.1.1 A Note on IP Versions +............................... + +For readers that aren’t particularly familiar with IP addressing, it’s +important to know that the Internet Protocol is currently in the process +of moving from version 4 of the protocol to version 6. This transition +is occurring largely because version 4 of the protocol doesn’t provide +enough addresses to handle the needs of the whole world, especially +given the increasing number of devices with direct connections to the +internet. + +Explaining the details of the differences between the two versions of +the protocol is beyond the scope of this introduction, but readers need +to at least be aware that these two versions exist, and it will +sometimes be necessary to force the use of one version or the other. + + +File: python.info, Node: IP Host Addresses, Next: Defining Networks, Prev: A Note on IP Versions, Up: Creating Address/Network/Interface objects + +10.14.1.2 IP Host Addresses +........................... + +Addresses, often referred to as “host addresses” are the most basic unit +when working with IP addressing. The simplest way to create addresses +is to use the *note ipaddress.ip_address(): 2c16. factory function, +which automatically determines whether to create an IPv4 or IPv6 address +based on the passed in value: + + >>> ipaddress.ip_address('192.0.2.1') + IPv4Address('192.0.2.1') + >>> ipaddress.ip_address('2001:DB8::1') + IPv6Address('2001:db8::1') + +Addresses can also be created directly from integers. Values that will +fit within 32 bits are assumed to be IPv4 addresses: + + >>> ipaddress.ip_address(3221225985) + IPv4Address('192.0.2.1') + >>> ipaddress.ip_address(42540766411282592856903984951653826561) + IPv6Address('2001:db8::1') + +To force the use of IPv4 or IPv6 addresses, the relevant classes can be +invoked directly. This is particularly useful to force creation of IPv6 +addresses for small integers: + + >>> ipaddress.ip_address(1) + IPv4Address('0.0.0.1') + >>> ipaddress.IPv4Address(1) + IPv4Address('0.0.0.1') + >>> ipaddress.IPv6Address(1) + IPv6Address('::1') + + +File: python.info, Node: Defining Networks, Next: Host Interfaces, Prev: IP Host Addresses, Up: Creating Address/Network/Interface objects + +10.14.1.3 Defining Networks +........................... + +Host addresses are usually grouped together into IP networks, so *note +ipaddress: a2. provides a way to create, inspect and manipulate network +definitions. IP network objects are constructed from strings that +define the range of host addresses that are part of that network. The +simplest form for that information is a “network address/network prefix” +pair, where the prefix defines the number of leading bits that are +compared to determine whether or not an address is part of the network +and the network address defines the expected value of those bits. + +As for addresses, a factory function is provided that determines the +correct IP version automatically: + + >>> ipaddress.ip_network('192.0.2.0/24') + IPv4Network('192.0.2.0/24') + >>> ipaddress.ip_network('2001:db8::0/96') + IPv6Network('2001:db8::/96') + +Network objects cannot have any host bits set. The practical effect of +this is that ‘192.0.2.1/24’ does not describe a network. Such +definitions are referred to as interface objects since the +ip-on-a-network notation is commonly used to describe network interfaces +of a computer on a given network and are described further in the next +section. + +By default, attempting to create a network object with host bits set +will result in *note ValueError: 1fb. being raised. To request that the +additional bits instead be coerced to zero, the flag ‘strict=False’ can +be passed to the constructor: + + >>> ipaddress.ip_network('192.0.2.1/24') + Traceback (most recent call last): + ... + ValueError: 192.0.2.1/24 has host bits set + >>> ipaddress.ip_network('192.0.2.1/24', strict=False) + IPv4Network('192.0.2.0/24') + +While the string form offers significantly more flexibility, networks +can also be defined with integers, just like host addresses. In this +case, the network is considered to contain only the single address +identified by the integer, so the network prefix includes the entire +network address: + + >>> ipaddress.ip_network(3221225984) + IPv4Network('192.0.2.0/32') + >>> ipaddress.ip_network(42540766411282592856903984951653826560) + IPv6Network('2001:db8::/128') + +As with addresses, creation of a particular kind of network can be +forced by calling the class constructor directly instead of using the +factory function. + + +File: python.info, Node: Host Interfaces, Prev: Defining Networks, Up: Creating Address/Network/Interface objects + +10.14.1.4 Host Interfaces +......................... + +As mentioned just above, if you need to describe an address on a +particular network, neither the address nor the network classes are +sufficient. Notation like ‘192.0.2.1/24’ is commonly used by network +engineers and the people who write tools for firewalls and routers as +shorthand for “the host ‘192.0.2.1’ on the network ‘192.0.2.0/24’”, +Accordingly, *note ipaddress: a2. provides a set of hybrid classes that +associate an address with a particular network. The interface for +creation is identical to that for defining network objects, except that +the address portion isn’t constrained to being a network address. + + >>> ipaddress.ip_interface('192.0.2.1/24') + IPv4Interface('192.0.2.1/24') + >>> ipaddress.ip_interface('2001:db8::1/96') + IPv6Interface('2001:db8::1/96') + +Integer inputs are accepted (as with networks), and use of a particular +IP version can be forced by calling the relevant constructor directly. + + +File: python.info, Node: Inspecting Address/Network/Interface Objects, Next: Networks as lists of Addresses, Prev: Creating Address/Network/Interface objects, Up: An introduction to the ipaddress module + +10.14.2 Inspecting Address/Network/Interface Objects +---------------------------------------------------- + +You’ve gone to the trouble of creating an +IPv(4|6)(Address|Network|Interface) object, so you probably want to get +information about it. *note ipaddress: a2. tries to make doing this +easy and intuitive. + +Extracting the IP version: + + >>> addr4 = ipaddress.ip_address('192.0.2.1') + >>> addr6 = ipaddress.ip_address('2001:db8::1') + >>> addr6.version + 6 + >>> addr4.version + 4 + +Obtaining the network from an interface: + + >>> host4 = ipaddress.ip_interface('192.0.2.1/24') + >>> host4.network + IPv4Network('192.0.2.0/24') + >>> host6 = ipaddress.ip_interface('2001:db8::1/96') + >>> host6.network + IPv6Network('2001:db8::/96') + +Finding out how many individual addresses are in a network: + + >>> net4 = ipaddress.ip_network('192.0.2.0/24') + >>> net4.num_addresses + 256 + >>> net6 = ipaddress.ip_network('2001:db8::0/96') + >>> net6.num_addresses + 4294967296 + +Iterating through the “usable” addresses on a network: + + >>> net4 = ipaddress.ip_network('192.0.2.0/24') + >>> for x in net4.hosts(): + ... print(x) + 192.0.2.1 + 192.0.2.2 + 192.0.2.3 + 192.0.2.4 + ... + 192.0.2.252 + 192.0.2.253 + 192.0.2.254 + +Obtaining the netmask (i.e. set bits corresponding to the network +prefix) or the hostmask (any bits that are not part of the netmask): + + >>> net4 = ipaddress.ip_network('192.0.2.0/24') + >>> net4.netmask + IPv4Address('255.255.255.0') + >>> net4.hostmask + IPv4Address('0.0.0.255') + >>> net6 = ipaddress.ip_network('2001:db8::0/96') + >>> net6.netmask + IPv6Address('ffff:ffff:ffff:ffff:ffff:ffff::') + >>> net6.hostmask + IPv6Address('::ffff:ffff') + +Exploding or compressing the address: + + >>> addr6.exploded + '2001:0db8:0000:0000:0000:0000:0000:0001' + >>> addr6.compressed + '2001:db8::1' + >>> net6.exploded + '2001:0db8:0000:0000:0000:0000:0000:0000/96' + >>> net6.compressed + '2001:db8::/96' + +While IPv4 doesn’t support explosion or compression, the associated +objects still provide the relevant properties so that version neutral +code can easily ensure the most concise or most verbose form is used for +IPv6 addresses while still correctly handling IPv4 addresses. + + +File: python.info, Node: Networks as lists of Addresses, Next: Comparisons<4>, Prev: Inspecting Address/Network/Interface Objects, Up: An introduction to the ipaddress module + +10.14.3 Networks as lists of Addresses +-------------------------------------- + +It’s sometimes useful to treat networks as lists. This means it is +possible to index them like this: + + >>> net4[1] + IPv4Address('192.0.2.1') + >>> net4[-1] + IPv4Address('192.0.2.255') + >>> net6[1] + IPv6Address('2001:db8::1') + >>> net6[-1] + IPv6Address('2001:db8::ffff:ffff') + +It also means that network objects lend themselves to using the list +membership test syntax like this: + + if address in network: + # do something + +Containment testing is done efficiently based on the network prefix: + + >>> addr4 = ipaddress.ip_address('192.0.2.1') + >>> addr4 in ipaddress.ip_network('192.0.2.0/24') + True + >>> addr4 in ipaddress.ip_network('192.0.3.0/24') + False + + +File: python.info, Node: Comparisons<4>, Next: Using IP Addresses with other modules, Prev: Networks as lists of Addresses, Up: An introduction to the ipaddress module + +10.14.4 Comparisons +------------------- + +*note ipaddress: a2. provides some simple, hopefully intuitive ways to +compare objects, where it makes sense: + + >>> ipaddress.ip_address('192.0.2.1') < ipaddress.ip_address('192.0.2.2') + True + +A *note TypeError: 192. exception is raised if you try to compare +objects of different versions or different types. + + +File: python.info, Node: Using IP Addresses with other modules, Next: Getting more detail when instance creation fails, Prev: Comparisons<4>, Up: An introduction to the ipaddress module + +10.14.5 Using IP Addresses with other modules +--------------------------------------------- + +Other modules that use IP addresses (such as *note socket: ef.) usually +won’t accept objects from this module directly. Instead, they must be +coerced to an integer or string that the other module will accept: + + >>> addr4 = ipaddress.ip_address('192.0.2.1') + >>> str(addr4) + '192.0.2.1' + >>> int(addr4) + 3221225985 + + +File: python.info, Node: Getting more detail when instance creation fails, Prev: Using IP Addresses with other modules, Up: An introduction to the ipaddress module + +10.14.6 Getting more detail when instance creation fails +-------------------------------------------------------- + +When creating address/network/interface objects using the +version-agnostic factory functions, any errors will be reported as *note +ValueError: 1fb. with a generic error message that simply says the +passed in value was not recognized as an object of that type. The lack +of a specific error is because it’s necessary to know whether the value +is `supposed' to be IPv4 or IPv6 in order to provide more detail on why +it has been rejected. + +To support use cases where it is useful to have access to this +additional detail, the individual class constructors actually raise the +*note ValueError: 1fb. subclasses *note ipaddress.AddressValueError: +2c1f. and *note ipaddress.NetmaskValueError: 2c44. to indicate exactly +which part of the definition failed to parse correctly. + +The error messages are significantly more detailed when using the class +constructors directly. For example: + + >>> ipaddress.ip_address("192.168.0.256") + Traceback (most recent call last): + ... + ValueError: '192.168.0.256' does not appear to be an IPv4 or IPv6 address + >>> ipaddress.IPv4Address("192.168.0.256") + Traceback (most recent call last): + ... + ipaddress.AddressValueError: Octet 256 (> 255) not permitted in '192.168.0.256' + + >>> ipaddress.ip_network("192.168.0.1/64") + Traceback (most recent call last): + ... + ValueError: '192.168.0.1/64' does not appear to be an IPv4 or IPv6 network + >>> ipaddress.IPv4Network("192.168.0.1/64") + Traceback (most recent call last): + ... + ipaddress.NetmaskValueError: '64' is not a valid netmask + +However, both of the module specific exceptions have *note ValueError: +1fb. as their parent class, so if you’re not concerned with the +particular type of error, you can still write code like the following: + + try: + network = ipaddress.IPv4Network(address) + except ValueError: + print('address/netmask is invalid for IPv4:', address) + + +File: python.info, Node: Argument Clinic How-To, Next: Instrumenting CPython with DTrace and SystemTap, Prev: An introduction to the ipaddress module, Up: Python HOWTOs + +10.15 Argument Clinic How-To +============================ + + +author: Larry Hastings + +Abstract +........ + +Argument Clinic is a preprocessor for CPython C files. Its purpose is +to automate all the boilerplate involved with writing argument parsing +code for “builtins”. This document shows you how to convert your first +C function to work with Argument Clinic, and then introduces some +advanced topics on Argument Clinic usage. + +Currently Argument Clinic is considered internal-only for CPython. Its +use is not supported for files outside CPython, and no guarantees are +made regarding backwards compatibility for future versions. In other +words: if you maintain an external C extension for CPython, you’re +welcome to experiment with Argument Clinic in your own code. But the +version of Argument Clinic that ships with the next version of CPython +`could' be totally incompatible and break all your code. + +* Menu: + +* The Goals Of Argument Clinic:: +* Basic Concepts And Usage:: +* Converting Your First Function:: +* Advanced Topics:: + + +File: python.info, Node: The Goals Of Argument Clinic, Next: Basic Concepts And Usage, Up: Argument Clinic How-To + +10.15.1 The Goals Of Argument Clinic +------------------------------------ + +Argument Clinic’s primary goal is to take over responsibility for all +argument parsing code inside CPython. This means that, when you convert +a function to work with Argument Clinic, that function should no longer +do any of its own argument parsing—the code generated by Argument Clinic +should be a “black box” to you, where CPython calls in at the top, and +your code gets called at the bottom, with ‘PyObject *args’ (and maybe +‘PyObject *kwargs’) magically converted into the C variables and types +you need. + +In order for Argument Clinic to accomplish its primary goal, it must be +easy to use. Currently, working with CPython’s argument parsing library +is a chore, requiring maintaining redundant information in a surprising +number of places. When you use Argument Clinic, you don’t have to +repeat yourself. + +Obviously, no one would want to use Argument Clinic unless it’s solving +their problem—and without creating new problems of its own. So it’s +paramount that Argument Clinic generate correct code. It’d be nice if +the code was faster, too, but at the very least it should not introduce +a major speed regression. (Eventually Argument Clinic `should' make a +major speedup possible—we could rewrite its code generator to produce +tailor-made argument parsing code, rather than calling the +general-purpose CPython argument parsing library. That would make for +the fastest argument parsing possible!) + +Additionally, Argument Clinic must be flexible enough to work with any +approach to argument parsing. Python has some functions with some very +strange parsing behaviors; Argument Clinic’s goal is to support all of +them. + +Finally, the original motivation for Argument Clinic was to provide +introspection “signatures” for CPython builtins. It used to be, the +introspection query functions would throw an exception if you passed in +a builtin. With Argument Clinic, that’s a thing of the past! + +One idea you should keep in mind, as you work with Argument Clinic: the +more information you give it, the better job it’ll be able to do. +Argument Clinic is admittedly relatively simple right now. But as it +evolves it will get more sophisticated, and it should be able to do many +interesting and smart things with all the information you give it. + + +File: python.info, Node: Basic Concepts And Usage, Next: Converting Your First Function, Prev: The Goals Of Argument Clinic, Up: Argument Clinic How-To + +10.15.2 Basic Concepts And Usage +-------------------------------- + +Argument Clinic ships with CPython; you’ll find it in +‘Tools/clinic/clinic.py’. If you run that script, specifying a C file +as an argument: + + $ python3 Tools/clinic/clinic.py foo.c + +Argument Clinic will scan over the file looking for lines that look +exactly like this: + + /*[clinic input] + +When it finds one, it reads everything up to a line that looks exactly +like this: + + [clinic start generated code]*/ + +Everything in between these two lines is input for Argument Clinic. All +of these lines, including the beginning and ending comment lines, are +collectively called an Argument Clinic “block”. + +When Argument Clinic parses one of these blocks, it generates output. +This output is rewritten into the C file immediately after the block, +followed by a comment containing a checksum. The Argument Clinic block +now looks like this: + + /*[clinic input] + ... clinic input goes here ... + [clinic start generated code]*/ + ... clinic output goes here ... + /*[clinic end generated code: checksum=...]*/ + +If you run Argument Clinic on the same file a second time, Argument +Clinic will discard the old output and write out the new output with a +fresh checksum line. However, if the input hasn’t changed, the output +won’t change either. + +You should never modify the output portion of an Argument Clinic block. +Instead, change the input until it produces the output you want. +(That’s the purpose of the checksum—to detect if someone changed the +output, as these edits would be lost the next time Argument Clinic +writes out fresh output.) + +For the sake of clarity, here’s the terminology we’ll use with Argument +Clinic: + + * The first line of the comment (‘/*[clinic input]’) is the `start + line'. + + * The last line of the initial comment (‘[clinic start generated + code]*/’) is the `end line'. + + * The last line (‘/*[clinic end generated code: checksum=...]*/’) is + the `checksum line'. + + * In between the start line and the end line is the `input'. + + * In between the end line and the checksum line is the `output'. + + * All the text collectively, from the start line to the checksum line + inclusively, is the `block'. (A block that hasn’t been + successfully processed by Argument Clinic yet doesn’t have output + or a checksum line, but it’s still considered a block.) + + +File: python.info, Node: Converting Your First Function, Next: Advanced Topics, Prev: Basic Concepts And Usage, Up: Argument Clinic How-To + +10.15.3 Converting Your First Function +-------------------------------------- + +The best way to get a sense of how Argument Clinic works is to convert a +function to work with it. Here, then, are the bare minimum steps you’d +need to follow to convert a function to work with Argument Clinic. Note +that for code you plan to check in to CPython, you really should take +the conversion farther, using some of the advanced concepts you’ll see +later on in the document (like “return converters” and “self +converters”). But we’ll keep it simple for this walkthrough so you can +learn. + +Let’s dive in! + + 0. Make sure you’re working with a freshly updated checkout of the + CPython trunk. + + 1. Find a Python builtin that calls either *note PyArg_ParseTuple(): + 26a. or *note PyArg_ParseTupleAndKeywords(): 5ca, and hasn’t been + converted to work with Argument Clinic yet. For my example I’m + using ‘_pickle.Pickler.dump()’. + + 2. If the call to the ‘PyArg_Parse’ function uses any of the following + format units: + + O& + O! + es + es# + et + et# + + or if it has multiple calls to *note PyArg_ParseTuple(): 26a, you + should choose a different function. Argument Clinic `does' support + all of these scenarios. But these are advanced topics—let’s do + something simpler for your first function. + + Also, if the function has multiple calls to *note + PyArg_ParseTuple(): 26a. or *note PyArg_ParseTupleAndKeywords(): + 5ca. where it supports different types for the same argument, or if + the function uses something besides PyArg_Parse functions to parse + its arguments, it probably isn’t suitable for conversion to + Argument Clinic. Argument Clinic doesn’t support generic functions + or polymorphic parameters. + + 3. Add the following boilerplate above the function, creating our + block: + + /*[clinic input] + [clinic start generated code]*/ + + 4. Cut the docstring and paste it in between the ‘[clinic]’ lines, + removing all the junk that makes it a properly quoted C string. + When you’re done you should have just the text, based at the left + margin, with no line wider than 80 characters. (Argument Clinic + will preserve indents inside the docstring.) + + If the old docstring had a first line that looked like a function + signature, throw that line away. (The docstring doesn’t need it + anymore—when you use ‘help()’ on your builtin in the future, the + first line will be built automatically based on the function’s + signature.) + + Sample: + + /*[clinic input] + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + 5. If your docstring doesn’t have a “summary” line, Argument Clinic + will complain. So let’s make sure it has one. The “summary” line + should be a paragraph consisting of a single 80-column line at the + beginning of the docstring. + + (Our example docstring consists solely of a summary line, so the + sample code doesn’t have to change for this step.) + + 6. Above the docstring, enter the name of the function, followed by a + blank line. This should be the Python name of the function, and + should be the full dotted path to the function—it should start with + the name of the module, include any sub-modules, and if the + function is a method on a class it should include the class name + too. + + Sample: + + /*[clinic input] + _pickle.Pickler.dump + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + 7. If this is the first time that module or class has been used with + Argument Clinic in this C file, you must declare the module and/or + class. Proper Argument Clinic hygiene prefers declaring these in a + separate block somewhere near the top of the C file, in the same + way that include files and statics go at the top. (In our sample + code we’ll just show the two blocks next to each other.) + + The name of the class and module should be the same as the one seen + by Python. Check the name defined in the *note PyModuleDef: 3888. + or *note PyTypeObject: 2d6. as appropriate. + + When you declare a class, you must also specify two aspects of its + type in C: the type declaration you’d use for a pointer to an + instance of this class, and a pointer to the *note PyTypeObject: + 2d6. for this class. + + Sample: + + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + + /*[clinic input] + _pickle.Pickler.dump + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + 8. Declare each of the parameters to the function. Each parameter + should get its own line. All the parameter lines should be + indented from the function name and the docstring. + + The general form of these parameter lines is as follows: + + name_of_parameter: converter + + If the parameter has a default value, add that after the converter: + + name_of_parameter: converter = default_value + + Argument Clinic’s support for “default values” is quite + sophisticated; please see *note the section below on default + values: 3f69. for more information. + + Add a blank line below the parameters. + + What’s a “converter”? It establishes both the type of the variable + used in C, and the method to convert the Python value into a C + value at runtime. For now you’re going to use what’s called a + “legacy converter”—a convenience syntax intended to make porting + old code into Argument Clinic easier. + + For each parameter, copy the “format unit” for that parameter from + the ‘PyArg_Parse()’ format argument and specify `that' as its + converter, as a quoted string. (“format unit” is the formal name + for the one-to-three character substring of the ‘format’ parameter + that tells the argument parsing function what the type of the + variable is and how to convert it. For more on format units please + see *note Parsing arguments and building values: 2d0.) + + For multicharacter format units like ‘z#’, use the entire + two-or-three character string. + + Sample: + + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + + /*[clinic input] + _pickle.Pickler.dump + + obj: 'O' + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + 9. If your function has ‘|’ in the format string, meaning some + parameters have default values, you can ignore it. Argument Clinic + infers which parameters are optional based on whether or not they + have default values. + + If your function has ‘$’ in the format string, meaning it takes + keyword-only arguments, specify ‘*’ on a line by itself before the + first keyword-only argument, indented the same as the parameter + lines. + + (‘_pickle.Pickler.dump’ has neither, so our sample is unchanged.) + + 10. If the existing C function calls *note PyArg_ParseTuple(): 26a. + (as opposed to *note PyArg_ParseTupleAndKeywords(): 5ca.), then all + its arguments are positional-only. + + To mark all parameters as positional-only in Argument Clinic, add a + ‘/’ on a line by itself after the last parameter, indented the same + as the parameter lines. + + Currently this is all-or-nothing; either all parameters are + positional-only, or none of them are. (In the future Argument + Clinic may relax this restriction.) + + Sample: + + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + + /*[clinic input] + _pickle.Pickler.dump + + obj: 'O' + / + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + 11. It’s helpful to write a per-parameter docstring for each + parameter. But per-parameter docstrings are optional; you can skip + this step if you prefer. + + Here’s how to add a per-parameter docstring. The first line of the + per-parameter docstring must be indented further than the parameter + definition. The left margin of this first line establishes the + left margin for the whole per-parameter docstring; all the text you + write will be outdented by this amount. You can write as much text + as you like, across multiple lines if you wish. + + Sample: + + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + + /*[clinic input] + _pickle.Pickler.dump + + obj: 'O' + The object to be pickled. + / + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + 12. Save and close the file, then run ‘Tools/clinic/clinic.py’ on it. + With luck everything worked—your block now has output, and a ‘.c.h’ + file has been generated! Reopen the file in your text editor to + see: + + /*[clinic input] + _pickle.Pickler.dump + + obj: 'O' + The object to be pickled. + / + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + static PyObject * + _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ + + Obviously, if Argument Clinic didn’t produce any output, it’s + because it found an error in your input. Keep fixing your errors + and retrying until Argument Clinic processes your file without + complaint. + + For readability, most of the glue code has been generated to a + ‘.c.h’ file. You’ll need to include that in your original ‘.c’ + file, typically right after the clinic module block: + + #include "clinic/_pickle.c.h" + + 13. Double-check that the argument-parsing code Argument Clinic + generated looks basically the same as the existing code. + + First, ensure both places use the same argument-parsing function. + The existing code must call either *note PyArg_ParseTuple(): 26a. + or *note PyArg_ParseTupleAndKeywords(): 5ca.; ensure that the code + generated by Argument Clinic calls the `exact' same function. + + Second, the format string passed in to *note PyArg_ParseTuple(): + 26a. or *note PyArg_ParseTupleAndKeywords(): 5ca. should be + `exactly' the same as the hand-written one in the existing + function, up to the colon or semi-colon. + + (Argument Clinic always generates its format strings with a ‘:’ + followed by the name of the function. If the existing code’s + format string ends with ‘;’, to provide usage help, this change is + harmless—don’t worry about it.) + + Third, for parameters whose format units require two arguments + (like a length variable, or an encoding string, or a pointer to a + conversion function), ensure that the second argument is `exactly' + the same between the two invocations. + + Fourth, inside the output portion of the block you’ll find a + preprocessor macro defining the appropriate static *note + PyMethodDef: e1b. structure for this builtin: + + #define __PICKLE_PICKLER_DUMP_METHODDEF \ + {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, + + This static structure should be `exactly' the same as the existing + static *note PyMethodDef: e1b. structure for this builtin. + + If any of these items differ in `any way', adjust your Argument + Clinic function specification and rerun ‘Tools/clinic/clinic.py’ + until they `are' the same. + + 14. Notice that the last line of its output is the declaration of your + “impl” function. This is where the builtin’s implementation goes. + Delete the existing prototype of the function you’re modifying, but + leave the opening curly brace. Now delete its argument parsing + code and the declarations of all the variables it dumps the + arguments into. Notice how the Python arguments are now arguments + to this impl function; if the implementation used different names + for these variables, fix it. + + Let’s reiterate, just because it’s kind of weird. Your code should + now look like this: + + static return_type + your_function_impl(...) + /*[clinic end generated code: checksum=...]*/ + { + ... + + Argument Clinic generated the checksum line and the function + prototype just above it. You should write the opening (and + closing) curly braces for the function, and the implementation + inside. + + Sample: + + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ + + /*[clinic input] + _pickle.Pickler.dump + + obj: 'O' + The object to be pickled. + / + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + + PyDoc_STRVAR(__pickle_Pickler_dump__doc__, + "Write a pickled representation of obj to the open file.\n" + "\n" + ... + static PyObject * + _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ + { + /* Check whether the Pickler was initialized correctly (issue3664). + Developers often forget to call __init__() in their subclasses, which + would trigger a segfault without this check. */ + if (self->write == NULL) { + PyErr_Format(PicklingError, + "Pickler.__init__() was not called by %s.__init__()", + Py_TYPE(self)->tp_name); + return NULL; + } + + if (_Pickler_ClearBuffer(self) < 0) + return NULL; + + ... + + 15. Remember the macro with the *note PyMethodDef: e1b. structure for + this function? Find the existing *note PyMethodDef: e1b. structure + for this function and replace it with a reference to the macro. + (If the builtin is at module scope, this will probably be very near + the end of the file; if the builtin is a class method, this will + probably be below but relatively near to the implementation.) + + Note that the body of the macro contains a trailing comma. So when + you replace the existing static *note PyMethodDef: e1b. structure + with the macro, `don’t' add a comma to the end. + + Sample: + + static struct PyMethodDef Pickler_methods[] = { + __PICKLE_PICKLER_DUMP_METHODDEF + __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF + {NULL, NULL} /* sentinel */ + }; + + 16. Compile, then run the relevant portions of the regression-test + suite. This change should not introduce any new compile-time + warnings or errors, and there should be no externally-visible + change to Python’s behavior. + + Well, except for one difference: ‘inspect.signature()’ run on your + function should now provide a valid signature! + + Congratulations, you’ve ported your first function to work with + Argument Clinic! + + +File: python.info, Node: Advanced Topics, Prev: Converting Your First Function, Up: Argument Clinic How-To + +10.15.4 Advanced Topics +----------------------- + +Now that you’ve had some experience working with Argument Clinic, it’s +time for some advanced topics. + +* Menu: + +* Symbolic default values:: +* Renaming the C functions and variables generated by Argument Clinic:: +* Converting functions using PyArg_UnpackTuple:: +* Optional Groups:: +* Using real Argument Clinic converters, instead of “legacy converters”: Using real Argument Clinic converters instead of “legacy converters”. +* Py_buffer:: +* Advanced converters:: +* Parameter default values:: +* The NULL default value:: +* Expressions specified as default values:: +* Using a return converter:: +* Cloning existing functions:: +* Calling Python code:: +* Using a “self converter”:: +* Writing a custom converter:: +* Writing a custom return converter:: +* METH_O and METH_NOARGS:: +* tp_new and tp_init functions:: +* Changing and redirecting Clinic’s output:: +* The #ifdef trick:: +* Using Argument Clinic in Python files:: + + +File: python.info, Node: Symbolic default values, Next: Renaming the C functions and variables generated by Argument Clinic, Up: Advanced Topics + +10.15.4.1 Symbolic default values +................................. + +The default value you provide for a parameter can’t be any arbitrary +expression. Currently the following are explicitly supported: + + * Numeric constants (integer and float) + + * String constants + + * ‘True’, ‘False’, and ‘None’ + + * Simple symbolic constants like ‘sys.maxsize’, which must start with + the name of the module + +In case you’re curious, this is implemented in ‘from_builtin()’ in +‘Lib/inspect.py’. + +(In the future, this may need to get even more elaborate, to allow full +expressions like ‘CONSTANT - 1’.) + + +File: python.info, Node: Renaming the C functions and variables generated by Argument Clinic, Next: Converting functions using PyArg_UnpackTuple, Prev: Symbolic default values, Up: Advanced Topics + +10.15.4.2 Renaming the C functions and variables generated by Argument Clinic +............................................................................. + +Argument Clinic automatically names the functions it generates for you. +Occasionally this may cause a problem, if the generated name collides +with the name of an existing C function. There’s an easy solution: +override the names used for the C functions. Just add the keyword +‘"as"’ to your function declaration line, followed by the function name +you wish to use. Argument Clinic will use that function name for the +base (generated) function, then add ‘"_impl"’ to the end and use that +for the name of the impl function. + +For example, if we wanted to rename the C function names generated for +‘pickle.Pickler.dump’, it’d look like this: + + /*[clinic input] + pickle.Pickler.dump as pickler_dumper + + ... + +The base function would now be named ‘pickler_dumper()’, and the impl +function would now be named ‘pickler_dumper_impl()’. + +Similarly, you may have a problem where you want to give a parameter a +specific Python name, but that name may be inconvenient in C. Argument +Clinic allows you to give a parameter different names in Python and in +C, using the same ‘"as"’ syntax: + + /*[clinic input] + pickle.Pickler.dump + + obj: object + file as file_obj: object + protocol: object = NULL + * + fix_imports: bool = True + +Here, the name used in Python (in the signature and the ‘keywords’ +array) would be ‘file’, but the C variable would be named ‘file_obj’. + +You can use this to rename the ‘self’ parameter too! + + +File: python.info, Node: Converting functions using PyArg_UnpackTuple, Next: Optional Groups, Prev: Renaming the C functions and variables generated by Argument Clinic, Up: Advanced Topics + +10.15.4.3 Converting functions using PyArg_UnpackTuple +...................................................... + +To convert a function parsing its arguments with *note +PyArg_UnpackTuple(): e46, simply write out all the arguments, specifying +each as an ‘object’. You may specify the ‘type’ argument to cast the +type as appropriate. All arguments should be marked positional-only +(add a ‘/’ on a line by itself after the last argument). + +Currently the generated code will use *note PyArg_ParseTuple(): 26a, but +this will change soon. + + +File: python.info, Node: Optional Groups, Next: Using real Argument Clinic converters instead of “legacy converters”, Prev: Converting functions using PyArg_UnpackTuple, Up: Advanced Topics + +10.15.4.4 Optional Groups +......................... + +Some legacy functions have a tricky approach to parsing their arguments: +they count the number of positional arguments, then use a ‘switch’ +statement to call one of several different *note PyArg_ParseTuple(): +26a. calls depending on how many positional arguments there are. (These +functions cannot accept keyword-only arguments.) This approach was used +to simulate optional arguments back before *note +PyArg_ParseTupleAndKeywords(): 5ca. was created. + +While functions using this approach can often be converted to use *note +PyArg_ParseTupleAndKeywords(): 5ca, optional arguments, and default +values, it’s not always possible. Some of these legacy functions have +behaviors *note PyArg_ParseTupleAndKeywords(): 5ca. doesn’t directly +support. The most obvious example is the builtin function ‘range()’, +which has an optional argument on the `left' side of its required +argument! Another example is ‘curses.window.addch()’, which has a group +of two arguments that must always be specified together. (The arguments +are called ‘x’ and ‘y’; if you call the function passing in ‘x’, you +must also pass in ‘y’—and if you don’t pass in ‘x’ you may not pass in +‘y’ either.) + +In any case, the goal of Argument Clinic is to support argument parsing +for all existing CPython builtins without changing their semantics. +Therefore Argument Clinic supports this alternate approach to parsing, +using what are called `optional groups'. Optional groups are groups of +arguments that must all be passed in together. They can be to the left +or the right of the required arguments. They can `only' be used with +positional-only parameters. + + Note: Optional groups are `only' intended for use when converting + functions that make multiple calls to *note PyArg_ParseTuple(): + 26a.! Functions that use `any' other approach for parsing + arguments should `almost never' be converted to Argument Clinic + using optional groups. Functions using optional groups currently + cannot have accurate signatures in Python, because Python just + doesn’t understand the concept. Please avoid using optional groups + wherever possible. + +To specify an optional group, add a ‘[’ on a line by itself before the +parameters you wish to group together, and a ‘]’ on a line by itself +after these parameters. As an example, here’s how ‘curses.window.addch’ +uses optional groups to make the first two parameters and the last +parameter optional: + + /*[clinic input] + + curses.window.addch + + [ + x: int + X-coordinate. + y: int + Y-coordinate. + ] + + ch: object + Character to add. + + [ + attr: long + Attributes for the character. + ] + / + + ... + +Notes: + + * For every optional group, one additional parameter will be passed + into the impl function representing the group. The parameter will + be an int named ‘group_{direction}_{number}’, where ‘{direction}’ + is either ‘right’ or ‘left’ depending on whether the group is + before or after the required parameters, and ‘{number}’ is a + monotonically increasing number (starting at 1) indicating how far + away the group is from the required parameters. When the impl is + called, this parameter will be set to zero if this group was + unused, and set to non-zero if this group was used. (By used or + unused, I mean whether or not the parameters received arguments in + this invocation.) + + * If there are no required arguments, the optional groups will behave + as if they’re to the right of the required arguments. + + * In the case of ambiguity, the argument parsing code favors + parameters on the left (before the required parameters). + + * Optional groups can only contain positional-only parameters. + + * Optional groups are `only' intended for legacy code. Please do not + use optional groups for new code. + + +File: python.info, Node: Using real Argument Clinic converters instead of “legacy converters”, Next: Py_buffer, Prev: Optional Groups, Up: Advanced Topics + +10.15.4.5 Using real Argument Clinic converters, instead of “legacy converters” +............................................................................... + +To save time, and to minimize how much you need to learn to achieve your +first port to Argument Clinic, the walkthrough above tells you to use +“legacy converters”. “Legacy converters” are a convenience, designed +explicitly to make porting existing code to Argument Clinic easier. And +to be clear, their use is acceptable when porting code for Python 3.4. + +However, in the long term we probably want all our blocks to use +Argument Clinic’s real syntax for converters. Why? A couple reasons: + + * The proper converters are far easier to read and clearer in their + intent. + + * There are some format units that are unsupported as “legacy + converters”, because they require arguments, and the legacy + converter syntax doesn’t support specifying arguments. + + * In the future we may have a new argument parsing library that isn’t + restricted to what *note PyArg_ParseTuple(): 26a. supports; this + flexibility won’t be available to parameters using legacy + converters. + +Therefore, if you don’t mind a little extra effort, please use the +normal converters instead of legacy converters. + +In a nutshell, the syntax for Argument Clinic (non-legacy) converters +looks like a Python function call. However, if there are no explicit +arguments to the function (all functions take their default values), you +may omit the parentheses. Thus ‘bool’ and ‘bool()’ are exactly the same +converters. + +All arguments to Argument Clinic converters are keyword-only. All +Argument Clinic converters accept the following arguments: + + ‘c_default’ + + The default value for this parameter when defined in C. + Specifically, this will be the initializer for the variable + declared in the “parse function”. See *note the section on + default values: 3f69. for how to use this. Specified as a + string. + + ‘annotation’ + + The annotation value for this parameter. Not currently + supported, because PEP 8(1) mandates that the Python library + may not use annotations. + +In addition, some converters accept additional arguments. Here is a +list of these arguments, along with their meanings: + + ‘accept’ + + A set of Python types (and possibly pseudo-types); this + restricts the allowable Python argument to values of these + types. (This is not a general-purpose facility; as a rule it + only supports specific lists of types as shown in the legacy + converter table.) + + To accept ‘None’, add ‘NoneType’ to this set. + + ‘bitwise’ + + Only supported for unsigned integers. The native integer + value of this Python argument will be written to the parameter + without any range checking, even for negative values. + + ‘converter’ + + Only supported by the ‘object’ converter. Specifies the name + of a *note C “converter function”: 39cd. to use to convert + this object to a native type. + + ‘encoding’ + + Only supported for strings. Specifies the encoding to use + when converting this string from a Python str (Unicode) value + into a C ‘char *’ value. + + ‘subclass_of’ + + Only supported for the ‘object’ converter. Requires that the + Python value be a subclass of a Python type, as expressed in + C. + + ‘type’ + + Only supported for the ‘object’ and ‘self’ converters. + Specifies the C type that will be used to declare the + variable. Default value is ‘"PyObject *"’. + + ‘zeroes’ + + Only supported for strings. If true, embedded NUL bytes + (‘'\\0'’) are permitted inside the value. The length of the + string will be passed in to the impl function, just after the + string parameter, as a parameter named + ‘_length’. + +Please note, not every possible combination of arguments will work. +Usually these arguments are implemented by specific ‘PyArg_ParseTuple’ +`format units', with specific behavior. For example, currently you +cannot call ‘unsigned_short’ without also specifying ‘bitwise=True’. +Although it’s perfectly reasonable to think this would work, these +semantics don’t map to any existing format unit. So Argument Clinic +doesn’t support it. (Or, at least, not yet.) + +Below is a table showing the mapping of legacy converters into real +Argument Clinic converters. On the left is the legacy converter, on the +right is the text you’d replace it with. + +‘'B'’ ‘unsigned_char(bitwise=True)’ + + +‘'b'’ ‘unsigned_char’ + + +‘'c'’ ‘char’ + + +‘'C'’ ‘int(accept={str})’ + + +‘'d'’ ‘double’ + + +‘'D'’ ‘Py_complex’ + + +‘'es'’ ‘str(encoding='name_of_encoding')’ + + +‘'es#'’ ‘str(encoding='name_of_encoding', zeroes=True)’ + + +‘'et'’ ‘str(encoding='name_of_encoding', accept={bytes, bytearray, str})’ + + +‘'et#'’ ‘str(encoding='name_of_encoding', accept={bytes, bytearray, str}, zeroes=True)’ + + +‘'f'’ ‘float’ + + +‘'h'’ ‘short’ + + +‘'H'’ ‘unsigned_short(bitwise=True)’ + + +‘'i'’ ‘int’ + + +‘'I'’ ‘unsigned_int(bitwise=True)’ + + +‘'k'’ ‘unsigned_long(bitwise=True)’ + + +‘'K'’ ‘unsigned_long_long(bitwise=True)’ + + +‘'l'’ ‘long’ + + +‘'L'’ ‘long long’ + + +‘'n'’ ‘Py_ssize_t’ + + +‘'O'’ ‘object’ + + +‘'O!'’ ‘object(subclass_of='&PySomething_Type')’ + + +‘'O&'’ ‘object(converter='name_of_c_function')’ + + +‘'p'’ ‘bool’ + + +‘'S'’ ‘PyBytesObject’ + + +‘'s'’ ‘str’ + + +‘'s#'’ ‘str(zeroes=True)’ + + +‘'s*'’ ‘Py_buffer(accept={buffer, str})’ + + +‘'U'’ ‘unicode’ + + +‘'u'’ ‘Py_UNICODE’ + + +‘'u#'’ ‘Py_UNICODE(zeroes=True)’ + + +‘'w*'’ ‘Py_buffer(accept={rwbuffer})’ + + +‘'Y'’ ‘PyByteArrayObject’ + + +‘'y'’ ‘str(accept={bytes})’ + + +‘'y#'’ ‘str(accept={robuffer}, zeroes=True)’ + + +‘'y*'’ ‘Py_buffer’ + + +‘'Z'’ ‘Py_UNICODE(accept={str, NoneType})’ + + +‘'Z#'’ ‘Py_UNICODE(accept={str, NoneType}, zeroes=True)’ + + +‘'z'’ ‘str(accept={str, NoneType})’ + + +‘'z#'’ ‘str(accept={str, NoneType}, zeroes=True)’ + + +‘'z*'’ ‘Py_buffer(accept={buffer, str, NoneType})’ + + +As an example, here’s our sample ‘pickle.Pickler.dump’ using the proper +converter: + + /*[clinic input] + pickle.Pickler.dump + + obj: object + The object to be pickled. + / + + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ + +One advantage of real converters is that they’re more flexible than +legacy converters. For example, the ‘unsigned_int’ converter (and all +the ‘unsigned_’ converters) can be specified without ‘bitwise=True’. +Their default behavior performs range checking on the value, and they +won’t accept negative numbers. You just can’t do that with a legacy +converter! + +Argument Clinic will show you all the converters it has available. For +each converter it’ll show you all the parameters it accepts, along with +the default value for each parameter. Just run ‘Tools/clinic/clinic.py +--converters’ to see the full list. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0008 + + +File: python.info, Node: Py_buffer, Next: Advanced converters, Prev: Using real Argument Clinic converters instead of “legacy converters”, Up: Advanced Topics + +10.15.4.6 Py_buffer +................... + +When using the ‘Py_buffer’ converter (or the ‘'s*'’, ‘'w*'’, ‘'*y'’, or +‘'z*'’ legacy converters), you `must' not call *note PyBuffer_Release(): +d54. on the provided buffer. Argument Clinic generates code that does +it for you (in the parsing function). + + +File: python.info, Node: Advanced converters, Next: Parameter default values, Prev: Py_buffer, Up: Advanced Topics + +10.15.4.7 Advanced converters +............................. + +Remember those format units you skipped for your first time because they +were advanced? Here’s how to handle those too. + +The trick is, all those format units take arguments—either conversion +functions, or types, or strings specifying an encoding. (But “legacy +converters” don’t support arguments. That’s why we skipped them for +your first function.) The argument you specified to the format unit is +now an argument to the converter; this argument is either ‘converter’ +(for ‘O&’), ‘subclass_of’ (for ‘O!’), or ‘encoding’ (for all the format +units that start with ‘e’). + +When using ‘subclass_of’, you may also want to use the other custom +argument for ‘object()’: ‘type’, which lets you set the type actually +used for the parameter. For example, if you want to ensure that the +object is a subclass of ‘PyUnicode_Type’, you probably want to use the +converter ‘object(type='PyUnicodeObject *', +subclass_of='&PyUnicode_Type')’. + +One possible problem with using Argument Clinic: it takes away some +possible flexibility for the format units starting with ‘e’. When +writing a ‘PyArg_Parse’ call by hand, you could theoretically decide at +runtime what encoding string to pass in to *note PyArg_ParseTuple(): +26a. But now this string must be hard-coded at +Argument-Clinic-preprocessing-time. This limitation is deliberate; it +made supporting this format unit much easier, and may allow for future +optimizations. This restriction doesn’t seem unreasonable; CPython +itself always passes in static hard-coded encoding strings for +parameters whose format units start with ‘e’. + + +File: python.info, Node: Parameter default values, Next: The NULL default value, Prev: Advanced converters, Up: Advanced Topics + +10.15.4.8 Parameter default values +.................................. + +Default values for parameters can be any of a number of values. At +their simplest, they can be string, int, or float literals: + + foo: str = "abc" + bar: int = 123 + bat: float = 45.6 + +They can also use any of Python’s built-in constants: + + yep: bool = True + nope: bool = False + nada: object = None + +There’s also special support for a default value of ‘NULL’, and for +simple expressions, documented in the following sections. + + +File: python.info, Node: The NULL default value, Next: Expressions specified as default values, Prev: Parameter default values, Up: Advanced Topics + +10.15.4.9 The ‘NULL’ default value +.................................. + +For string and object parameters, you can set them to ‘None’ to indicate +that there’s no default. However, that means the C variable will be +initialized to ‘Py_None’. For convenience’s sakes, there’s a special +value called ‘NULL’ for just this reason: from Python’s perspective it +behaves like a default value of ‘None’, but the C variable is +initialized with ‘NULL’. + + +File: python.info, Node: Expressions specified as default values, Next: Using a return converter, Prev: The NULL default value, Up: Advanced Topics + +10.15.4.10 Expressions specified as default values +.................................................. + +The default value for a parameter can be more than just a literal value. +It can be an entire expression, using math operators and looking up +attributes on objects. However, this support isn’t exactly simple, +because of some non-obvious semantics. + +Consider the following example: + + foo: Py_ssize_t = sys.maxsize - 1 + +‘sys.maxsize’ can have different values on different platforms. +Therefore Argument Clinic can’t simply evaluate that expression locally +and hard-code it in C. So it stores the default in such a way that it +will get evaluated at runtime, when the user asks for the function’s +signature. + +What namespace is available when the expression is evaluated? It’s +evaluated in the context of the module the builtin came from. So, if +your module has an attribute called “‘max_widgets’”, you may simply use +it: + + foo: Py_ssize_t = max_widgets + +If the symbol isn’t found in the current module, it fails over to +looking in ‘sys.modules’. That’s how it can find ‘sys.maxsize’ for +example. (Since you don’t know in advance what modules the user will +load into their interpreter, it’s best to restrict yourself to modules +that are preloaded by Python itself.) + +Evaluating default values only at runtime means Argument Clinic can’t +compute the correct equivalent C default value. So you need to tell it +explicitly. When you use an expression, you must also specify the +equivalent expression in C, using the ‘c_default’ parameter to the +converter: + + foo: Py_ssize_t(c_default="PY_SSIZE_T_MAX - 1") = sys.maxsize - 1 + +Another complication: Argument Clinic can’t know in advance whether or +not the expression you supply is valid. It parses it to make sure it +looks legal, but it can’t `actually' know. You must be very careful +when using expressions to specify values that are guaranteed to be valid +at runtime! + +Finally, because expressions must be representable as static C values, +there are many restrictions on legal expressions. Here’s a list of +Python features you’re not permitted to use: + + * Function calls. + + * Inline if statements (‘3 if foo else 5’). + + * Automatic sequence unpacking (‘*[1, 2, 3]’). + + * List/set/dict comprehensions and generator expressions. + + * Tuple/list/set/dict literals. + + +File: python.info, Node: Using a return converter, Next: Cloning existing functions, Prev: Expressions specified as default values, Up: Advanced Topics + +10.15.4.11 Using a return converter +................................... + +By default the impl function Argument Clinic generates for you returns +‘PyObject *’. But your C function often computes some C type, then +converts it into the ‘PyObject *’ at the last moment. Argument Clinic +handles converting your inputs from Python types into native C types—why +not have it convert your return value from a native C type into a Python +type too? + +That’s what a “return converter” does. It changes your impl function to +return some C type, then adds code to the generated (non-impl) function +to handle converting that value into the appropriate ‘PyObject *’. + +The syntax for return converters is similar to that of parameter +converters. You specify the return converter like it was a return +annotation on the function itself. Return converters behave much the +same as parameter converters; they take arguments, the arguments are all +keyword-only, and if you’re not changing any of the default arguments +you can omit the parentheses. + +(If you use both ‘"as"’ `and' a return converter for your function, the +‘"as"’ should come before the return converter.) + +There’s one additional complication when using return converters: how do +you indicate an error has occurred? Normally, a function returns a +valid (non-‘NULL’) pointer for success, and ‘NULL’ for failure. But if +you use an integer return converter, all integers are valid. How can +Argument Clinic detect an error? Its solution: each return converter +implicitly looks for a special value that indicates an error. If you +return that value, and an error has been set (‘PyErr_Occurred()’ returns +a true value), then the generated code will propagate the error. +Otherwise it will encode the value you return like normal. + +Currently Argument Clinic supports only a few return converters: + + bool + int + unsigned int + long + unsigned int + size_t + Py_ssize_t + float + double + DecodeFSDefault + +None of these take parameters. For the first three, return -1 to +indicate error. For ‘DecodeFSDefault’, the return type is ‘const char +*’; return a ‘NULL’ pointer to indicate an error. + +(There’s also an experimental ‘NoneType’ converter, which lets you +return ‘Py_None’ on success or ‘NULL’ on failure, without having to +increment the reference count on ‘Py_None’. I’m not sure it adds enough +clarity to be worth using.) + +To see all the return converters Argument Clinic supports, along with +their parameters (if any), just run ‘Tools/clinic/clinic.py +--converters’ for the full list. + + +File: python.info, Node: Cloning existing functions, Next: Calling Python code, Prev: Using a return converter, Up: Advanced Topics + +10.15.4.12 Cloning existing functions +..................................... + +If you have a number of functions that look similar, you may be able to +use Clinic’s “clone” feature. When you clone an existing function, you +reuse: + + * its parameters, including + + * their names, + + * their converters, with all parameters, + + * their default values, + + * their per-parameter docstrings, + + * their `kind' (whether they’re positional only, positional or + keyword, or keyword only), and + + * its return converter. + +The only thing not copied from the original function is its docstring; +the syntax allows you to specify a new docstring. + +Here’s the syntax for cloning a function: + + /*[clinic input] + module.class.new_function [as c_basename] = module.class.existing_function + + Docstring for new_function goes here. + [clinic start generated code]*/ + +(The functions can be in different modules or classes. I wrote +‘module.class’ in the sample just to illustrate that you must use the +full path to `both' functions.) + +Sorry, there’s no syntax for partially-cloning a function, or cloning a +function then modifying it. Cloning is an all-or nothing proposition. + +Also, the function you are cloning from must have been previously +defined in the current file. + + +File: python.info, Node: Calling Python code, Next: Using a “self converter”, Prev: Cloning existing functions, Up: Advanced Topics + +10.15.4.13 Calling Python code +.............................. + +The rest of the advanced topics require you to write Python code which +lives inside your C file and modifies Argument Clinic’s runtime state. +This is simple: you simply define a Python block. + +A Python block uses different delimiter lines than an Argument Clinic +function block. It looks like this: + + /*[python input] + # python code goes here + [python start generated code]*/ + +All the code inside the Python block is executed at the time it’s +parsed. All text written to stdout inside the block is redirected into +the “output” after the block. + +As an example, here’s a Python block that adds a static integer variable +to the C code: + + /*[python input] + print('static int __ignored_unused_variable__ = 0;') + [python start generated code]*/ + static int __ignored_unused_variable__ = 0; + /*[python checksum:...]*/ + + +File: python.info, Node: Using a “self converter”, Next: Writing a custom converter, Prev: Calling Python code, Up: Advanced Topics + +10.15.4.14 Using a “self converter” +................................... + +Argument Clinic automatically adds a “self” parameter for you using a +default converter. It automatically sets the ‘type’ of this parameter +to the “pointer to an instance” you specified when you declared the +type. However, you can override Argument Clinic’s converter and specify +one yourself. Just add your own ‘self’ parameter as the first parameter +in a block, and ensure that its converter is an instance of +‘self_converter’ or a subclass thereof. + +What’s the point? This lets you override the type of ‘self’, or give it +a different default name. + +How do you specify the custom type you want to cast ‘self’ to? If you +only have one or two functions with the same type for ‘self’, you can +directly use Argument Clinic’s existing ‘self’ converter, passing in the +type you want to use as the ‘type’ parameter: + + /*[clinic input] + + _pickle.Pickler.dump + + self: self(type="PicklerObject *") + obj: object + / + + Write a pickled representation of the given object to the open file. + [clinic start generated code]*/ + +On the other hand, if you have a lot of functions that will use the same +type for ‘self’, it’s best to create your own converter, subclassing +‘self_converter’ but overwriting the ‘type’ member: + + /*[python input] + class PicklerObject_converter(self_converter): + type = "PicklerObject *" + [python start generated code]*/ + + /*[clinic input] + + _pickle.Pickler.dump + + self: PicklerObject + obj: object + / + + Write a pickled representation of the given object to the open file. + [clinic start generated code]*/ + + +File: python.info, Node: Writing a custom converter, Next: Writing a custom return converter, Prev: Using a “self converter”, Up: Advanced Topics + +10.15.4.15 Writing a custom converter +..................................... + +As we hinted at in the previous section… you can write your own +converters! A converter is simply a Python class that inherits from +‘CConverter’. The main purpose of a custom converter is if you have a +parameter using the ‘O&’ format unit—parsing this parameter means +calling a *note PyArg_ParseTuple(): 26a. “converter function”. + +Your converter class should be named ‘*something*_converter’. If the +name follows this convention, then your converter class will be +automatically registered with Argument Clinic; its name will be the name +of your class with the ‘_converter’ suffix stripped off. (This is +accomplished with a metaclass.) + +You shouldn’t subclass ‘CConverter.__init__’. Instead, you should write +a ‘converter_init()’ function. ‘converter_init()’ always accepts a +‘self’ parameter; after that, all additional parameters `must' be +keyword-only. Any arguments passed in to the converter in Argument +Clinic will be passed along to your ‘converter_init()’. + +There are some additional members of ‘CConverter’ you may wish to +specify in your subclass. Here’s the current list: + +‘type’ + + The C type to use for this variable. ‘type’ should be a Python + string specifying the type, e.g. ‘int’. If this is a pointer + type, the type string should end with ‘' *'’. + +‘default’ + + The Python default value for this parameter, as a Python value. Or + the magic value ‘unspecified’ if there is no default. + +‘py_default’ + + ‘default’ as it should appear in Python code, as a string. Or + ‘None’ if there is no default. + +‘c_default’ + + ‘default’ as it should appear in C code, as a string. Or ‘None’ if + there is no default. + +‘c_ignored_default’ + + The default value used to initialize the C variable when there is + no default, but not specifying a default may result in an + “uninitialized variable” warning. This can easily happen when + using option groups—although properly-written code will never + actually use this value, the variable does get passed in to the + impl, and the C compiler will complain about the “use” of the + uninitialized value. This value should always be a non-empty + string. + +‘converter’ + + The name of the C converter function, as a string. + +‘impl_by_reference’ + + A boolean value. If true, Argument Clinic will add a ‘&’ in front + of the name of the variable when passing it into the impl function. + +‘parse_by_reference’ + + A boolean value. If true, Argument Clinic will add a ‘&’ in front + of the name of the variable when passing it into *note + PyArg_ParseTuple(): 26a. + +Here’s the simplest example of a custom converter, from +‘Modules/zlibmodule.c’: + + /*[python input] + + class ssize_t_converter(CConverter): + type = 'Py_ssize_t' + converter = 'ssize_t_converter' + + [python start generated code]*/ + /*[python end generated code: output=da39a3ee5e6b4b0d input=35521e4e733823c7]*/ + +This block adds a converter to Argument Clinic named ‘ssize_t’. +Parameters declared as ‘ssize_t’ will be declared as type ‘Py_ssize_t’, +and will be parsed by the ‘'O&'’ format unit, which will call the +‘ssize_t_converter’ converter function. ‘ssize_t’ variables +automatically support default values. + +More sophisticated custom converters can insert custom C code to handle +initialization and cleanup. You can see more examples of custom +converters in the CPython source tree; grep the C files for the string +‘CConverter’. + + +File: python.info, Node: Writing a custom return converter, Next: METH_O and METH_NOARGS, Prev: Writing a custom converter, Up: Advanced Topics + +10.15.4.16 Writing a custom return converter +............................................ + +Writing a custom return converter is much like writing a custom +converter. Except it’s somewhat simpler, because return converters are +themselves much simpler. + +Return converters must subclass ‘CReturnConverter’. There are no +examples yet of custom return converters, because they are not widely +used yet. If you wish to write your own return converter, please read +‘Tools/clinic/clinic.py’, specifically the implementation of +‘CReturnConverter’ and all its subclasses. + + +File: python.info, Node: METH_O and METH_NOARGS, Next: tp_new and tp_init functions, Prev: Writing a custom return converter, Up: Advanced Topics + +10.15.4.17 METH_O and METH_NOARGS +................................. + +To convert a function using ‘METH_O’, make sure the function’s single +argument is using the ‘object’ converter, and mark the arguments as +positional-only: + + /*[clinic input] + meth_o_sample + + argument: object + / + [clinic start generated code]*/ + +To convert a function using ‘METH_NOARGS’, just don’t specify any +arguments. + +You can still use a self converter, a return converter, and specify a +‘type’ argument to the object converter for ‘METH_O’. + + +File: python.info, Node: tp_new and tp_init functions, Next: Changing and redirecting Clinic’s output, Prev: METH_O and METH_NOARGS, Up: Advanced Topics + +10.15.4.18 tp_new and tp_init functions +....................................... + +You can convert ‘tp_new’ and ‘tp_init’ functions. Just name them +‘__new__’ or ‘__init__’ as appropriate. Notes: + + * The function name generated for ‘__new__’ doesn’t end in ‘__new__’ + like it would by default. It’s just the name of the class, + converted into a valid C identifier. + + * No ‘PyMethodDef’ ‘#define’ is generated for these functions. + + * ‘__init__’ functions return ‘int’, not ‘PyObject *’. + + * Use the docstring as the class docstring. + + * Although ‘__new__’ and ‘__init__’ functions must always accept both + the ‘args’ and ‘kwargs’ objects, when converting you may specify + any signature for these functions that you like. (If your function + doesn’t support keywords, the parsing function generated will throw + an exception if it receives any.) + + +File: python.info, Node: Changing and redirecting Clinic’s output, Next: The #ifdef trick, Prev: tp_new and tp_init functions, Up: Advanced Topics + +10.15.4.19 Changing and redirecting Clinic’s output +................................................... + +It can be inconvenient to have Clinic’s output interspersed with your +conventional hand-edited C code. Luckily, Clinic is configurable: you +can buffer up its output for printing later (or earlier!), or write its +output to a separate file. You can also add a prefix or suffix to every +line of Clinic’s generated output. + +While changing Clinic’s output in this manner can be a boon to +readability, it may result in Clinic code using types before they are +defined, or your code attempting to use Clinic-generated code before it +is defined. These problems can be easily solved by rearranging the +declarations in your file, or moving where Clinic’s generated code goes. +(This is why the default behavior of Clinic is to output everything into +the current block; while many people consider this hampers readability, +it will never require rearranging your code to fix definition-before-use +problems.) + +Let’s start with defining some terminology: + +`field' + + A field, in this context, is a subsection of Clinic’s output. For + example, the ‘#define’ for the ‘PyMethodDef’ structure is a field, + called ‘methoddef_define’. Clinic has seven different fields it + can output per function definition: + + docstring_prototype + docstring_definition + methoddef_define + impl_prototype + parser_prototype + parser_definition + impl_definition + + All the names are of the form ‘"_"’, where ‘""’ is the + semantic object represented (the parsing function, the impl + function, the docstring, or the methoddef structure) and ‘""’ + represents what kind of statement the field is. Field names that + end in ‘"_prototype"’ represent forward declarations of that thing, + without the actual body/data of the thing; field names that end in + ‘"_definition"’ represent the actual definition of the thing, with + the body/data of the thing. (‘"methoddef"’ is special, it’s the + only one that ends with ‘"_define"’, representing that it’s a + preprocessor #define.) + +`destination' + + A destination is a place Clinic can write output to. There are + five built-in destinations: + + ‘block’ + + The default destination: printed in the output section of the + current Clinic block. + + ‘buffer’ + + A text buffer where you can save text for later. Text sent + here is appended to the end of any existing text. It’s an + error to have any text left in the buffer when Clinic finishes + processing a file. + + ‘file’ + + A separate “clinic file” that will be created automatically by + Clinic. The filename chosen for the file is + ‘{basename}.clinic{extension}’, where ‘basename’ and + ‘extension’ were assigned the output from ‘os.path.splitext()’ + run on the current file. (Example: the ‘file’ destination for + ‘_pickle.c’ would be written to ‘_pickle.clinic.c’.) + + `Important: When using a' ‘file’ `destination, you' `must + check in' `the generated file!' + + ‘two-pass’ + + A buffer like ‘buffer’. However, a two-pass buffer can only + be dumped once, and it prints out all text sent to it during + all processing, even from Clinic blocks `after' the dumping + point. + + ‘suppress’ + + The text is suppressed—thrown away. + +Clinic defines five new directives that let you reconfigure its output. + +The first new directive is ‘dump’: + + dump + +This dumps the current contents of the named destination into the output +of the current block, and empties it. This only works with ‘buffer’ and +‘two-pass’ destinations. + +The second new directive is ‘output’. The most basic form of ‘output’ +is like this: + + output + +This tells Clinic to output `field' to `destination'. ‘output’ also +supports a special meta-destination, called ‘everything’, which tells +Clinic to output `all' fields to that `destination'. + +‘output’ has a number of other functions: + + output push + output pop + output preset + +‘output push’ and ‘output pop’ allow you to push and pop configurations +on an internal configuration stack, so that you can temporarily modify +the output configuration, then easily restore the previous +configuration. Simply push before your change to save the current +configuration, then pop when you wish to restore the previous +configuration. + +‘output preset’ sets Clinic’s output to one of several built-in preset +configurations, as follows: + + ‘block’ + + Clinic’s original starting configuration. Writes everything + immediately after the input block. + + Suppress the ‘parser_prototype’ and ‘docstring_prototype’, + write everything else to ‘block’. + + ‘file’ + + Designed to write everything to the “clinic file” that it can. + You then ‘#include’ this file near the top of your file. You + may need to rearrange your file to make this work, though + usually this just means creating forward declarations for + various ‘typedef’ and ‘PyTypeObject’ definitions. + + Suppress the ‘parser_prototype’ and ‘docstring_prototype’, + write the ‘impl_definition’ to ‘block’, and write everything + else to ‘file’. + + The default filename is ‘"{dirname}/clinic/{basename}.h"’. + + ‘buffer’ + + Save up most of the output from Clinic, to be written into + your file near the end. For Python files implementing modules + or builtin types, it’s recommended that you dump the buffer + just above the static structures for your module or builtin + type; these are normally very near the end. Using ‘buffer’ + may require even more editing than ‘file’, if your file has + static ‘PyMethodDef’ arrays defined in the middle of the file. + + Suppress the ‘parser_prototype’, ‘impl_prototype’, and + ‘docstring_prototype’, write the ‘impl_definition’ to ‘block’, + and write everything else to ‘file’. + + ‘two-pass’ + + Similar to the ‘buffer’ preset, but writes forward + declarations to the ‘two-pass’ buffer, and definitions to the + ‘buffer’. This is similar to the ‘buffer’ preset, but may + require less editing than ‘buffer’. Dump the ‘two-pass’ + buffer near the top of your file, and dump the ‘buffer’ near + the end just like you would when using the ‘buffer’ preset. + + Suppresses the ‘impl_prototype’, write the ‘impl_definition’ + to ‘block’, write ‘docstring_prototype’, ‘methoddef_define’, + and ‘parser_prototype’ to ‘two-pass’, write everything else to + ‘buffer’. + + ‘partial-buffer’ + + Similar to the ‘buffer’ preset, but writes more things to + ‘block’, only writing the really big chunks of generated code + to ‘buffer’. This avoids the definition-before-use problem of + ‘buffer’ completely, at the small cost of having slightly more + stuff in the block’s output. Dump the ‘buffer’ near the end, + just like you would when using the ‘buffer’ preset. + + Suppresses the ‘impl_prototype’, write the + ‘docstring_definition’ and ‘parser_definition’ to ‘buffer’, + write everything else to ‘block’. + +The third new directive is ‘destination’: + + destination [...] + +This performs an operation on the destination named ‘name’. + +There are two defined subcommands: ‘new’ and ‘clear’. + +The ‘new’ subcommand works like this: + + destination new + +This creates a new destination with name ‘’ and type ‘’. + +There are five destination types: + + ‘suppress’ + + Throws the text away. + + ‘block’ + + Writes the text to the current block. This is what Clinic + originally did. + + ‘buffer’ + + A simple text buffer, like the “buffer” builtin destination + above. + + ‘file’ + + A text file. The file destination takes an extra argument, a + template to use for building the filename, like so: + + destination new + + The template can use three strings internally that will be + replaced by bits of the filename: + + {path} + + The full path to the file, including directory and + full filename. + + {dirname} + + The name of the directory the file is in. + + {basename} + + Just the name of the file, not including the + directory. + + {basename_root} + + Basename with the extension clipped off (everything + up to but not including the last ‘.’). + + {basename_extension} + + The last ‘.’ and everything after it. If the + basename does not contain a period, this will be the + empty string. + + If there are no periods in the filename, {basename} and + {filename} are the same, and {extension} is empty. + “{basename}{extension}” is always exactly the same as + “{filename}”.” + + ‘two-pass’ + + A two-pass buffer, like the “two-pass” builtin destination + above. + +The ‘clear’ subcommand works like this: + + destination clear + +It removes all the accumulated text up to this point in the destination. +(I don’t know what you’d need this for, but I thought maybe it’d be +useful while someone’s experimenting.) + +The fourth new directive is ‘set’: + + set line_prefix "string" + set line_suffix "string" + +‘set’ lets you set two internal variables in Clinic. ‘line_prefix’ is a +string that will be prepended to every line of Clinic’s output; +‘line_suffix’ is a string that will be appended to every line of +Clinic’s output. + +Both of these support two format strings: + + ‘{block comment start}’ + + Turns into the string ‘/*’, the start-comment text sequence + for C files. + + ‘{block comment end}’ + + Turns into the string ‘*/’, the end-comment text sequence for + C files. + +The final new directive is one you shouldn’t need to use directly, +called ‘preserve’: + + preserve + +This tells Clinic that the current contents of the output should be +kept, unmodified. This is used internally by Clinic when dumping output +into ‘file’ files; wrapping it in a Clinic block lets Clinic use its +existing checksum functionality to ensure the file was not modified by +hand before it gets overwritten. + + +File: python.info, Node: The #ifdef trick, Next: Using Argument Clinic in Python files, Prev: Changing and redirecting Clinic’s output, Up: Advanced Topics + +10.15.4.20 The #ifdef trick +........................... + +If you’re converting a function that isn’t available on all platforms, +there’s a trick you can use to make life a little easier. The existing +code probably looks like this: + + #ifdef HAVE_FUNCTIONNAME + static module_functionname(...) + { + ... + } + #endif /* HAVE_FUNCTIONNAME */ + +And then in the ‘PyMethodDef’ structure at the bottom the existing code +will have: + + #ifdef HAVE_FUNCTIONNAME + {'functionname', ... }, + #endif /* HAVE_FUNCTIONNAME */ + +In this scenario, you should enclose the body of your impl function +inside the ‘#ifdef’, like so: + + #ifdef HAVE_FUNCTIONNAME + /*[clinic input] + module.functionname + ... + [clinic start generated code]*/ + static module_functionname(...) + { + ... + } + #endif /* HAVE_FUNCTIONNAME */ + +Then, remove those three lines from the ‘PyMethodDef’ structure, +replacing them with the macro Argument Clinic generated: + + MODULE_FUNCTIONNAME_METHODDEF + +(You can find the real name for this macro inside the generated code. +Or you can calculate it yourself: it’s the name of your function as +defined on the first line of your block, but with periods changed to +underscores, uppercased, and ‘"_METHODDEF"’ added to the end.) + +Perhaps you’re wondering: what if ‘HAVE_FUNCTIONNAME’ isn’t defined? +The ‘MODULE_FUNCTIONNAME_METHODDEF’ macro won’t be defined either! + +Here’s where Argument Clinic gets very clever. It actually detects that +the Argument Clinic block might be deactivated by the ‘#ifdef’. When +that happens, it generates a little extra code that looks like this: + + #ifndef MODULE_FUNCTIONNAME_METHODDEF + #define MODULE_FUNCTIONNAME_METHODDEF + #endif /* !defined(MODULE_FUNCTIONNAME_METHODDEF) */ + +That means the macro always works. If the function is defined, this +turns into the correct structure, including the trailing comma. If the +function is undefined, this turns into nothing. + +However, this causes one ticklish problem: where should Argument Clinic +put this extra code when using the “block” output preset? It can’t go +in the output block, because that could be deactivated by the ‘#ifdef’. +(That’s the whole point!) + +In this situation, Argument Clinic writes the extra code to the “buffer” +destination. This may mean that you get a complaint from Argument +Clinic: + + Warning in file "Modules/posixmodule.c" on line 12357: + Destination buffer 'buffer' not empty at end of file, emptying. + +When this happens, just open your file, find the ‘dump buffer’ block +that Argument Clinic added to your file (it’ll be at the very bottom), +then move it above the ‘PyMethodDef’ structure where that macro is used. + + +File: python.info, Node: Using Argument Clinic in Python files, Prev: The #ifdef trick, Up: Advanced Topics + +10.15.4.21 Using Argument Clinic in Python files +................................................ + +It’s actually possible to use Argument Clinic to preprocess Python +files. There’s no point to using Argument Clinic blocks, of course, as +the output wouldn’t make any sense to the Python interpreter. But using +Argument Clinic to run Python blocks lets you use Python as a Python +preprocessor! + +Since Python comments are different from C comments, Argument Clinic +blocks embedded in Python files look slightly different. They look like +this: + + #/*[python input] + #print("def foo(): pass") + #[python start generated code]*/ + def foo(): pass + #/*[python checksum:...]*/ + + +File: python.info, Node: Instrumenting CPython with DTrace and SystemTap, Prev: Argument Clinic How-To, Up: Python HOWTOs + +10.16 Instrumenting CPython with DTrace and SystemTap +===================================================== + + +author: David Malcolm + + +author: Łukasz Langa + +DTrace and SystemTap are monitoring tools, each providing a way to +inspect what the processes on a computer system are doing. They both +use domain-specific languages allowing a user to write scripts which: + + - filter which processes are to be observed + + - gather data from the processes of interest + + - generate reports on the data + +As of Python 3.6, CPython can be built with embedded “markers”, also +known as “probes”, that can be observed by a DTrace or SystemTap script, +making it easier to monitor what the CPython processes on a system are +doing. + +`CPython implementation detail:' DTrace markers are implementation +details of the CPython interpreter. No guarantees are made about probe +compatibility between versions of CPython. DTrace scripts can stop +working or work incorrectly without warning when changing CPython +versions. + +* Menu: + +* Enabling the static markers:: +* Static DTrace probes:: +* Static SystemTap markers:: +* Available static markers:: +* SystemTap Tapsets:: +* Examples: Examples<36>. + + +File: python.info, Node: Enabling the static markers, Next: Static DTrace probes, Up: Instrumenting CPython with DTrace and SystemTap + +10.16.1 Enabling the static markers +----------------------------------- + +macOS comes with built-in support for DTrace. On Linux, in order to +build CPython with the embedded markers for SystemTap, the SystemTap +development tools must be installed. + +On a Linux machine, this can be done via: + + $ yum install systemtap-sdt-devel + +or: + + $ sudo apt-get install systemtap-sdt-dev + +CPython must then be configured ‘--with-dtrace’: + + checking for --with-dtrace... yes + +On macOS, you can list available DTrace probes by running a Python +process in the background and listing all probes made available by the +Python provider: + + $ python3.6 -q & + $ sudo dtrace -l -P python$! # or: dtrace -l -m python3.6 + + ID PROVIDER MODULE FUNCTION NAME + 29564 python18035 python3.6 _PyEval_EvalFrameDefault function-entry + 29565 python18035 python3.6 dtrace_function_entry function-entry + 29566 python18035 python3.6 _PyEval_EvalFrameDefault function-return + 29567 python18035 python3.6 dtrace_function_return function-return + 29568 python18035 python3.6 collect gc-done + 29569 python18035 python3.6 collect gc-start + 29570 python18035 python3.6 _PyEval_EvalFrameDefault line + 29571 python18035 python3.6 maybe_dtrace_line line + +On Linux, you can verify if the SystemTap static markers are present in +the built binary by seeing if it contains a “.note.stapsdt” section. + + $ readelf -S ./python | grep .note.stapsdt + [30] .note.stapsdt NOTE 0000000000000000 00308d78 + +If you’ve built Python as a shared library (with –enable-shared), you +need to look instead within the shared library. For example: + + $ readelf -S libpython3.3dm.so.1.0 | grep .note.stapsdt + [29] .note.stapsdt NOTE 0000000000000000 00365b68 + +Sufficiently modern readelf can print the metadata: + + $ readelf -n ./python + + Displaying notes found at file offset 0x00000254 with length 0x00000020: + Owner Data size Description + GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag) + OS: Linux, ABI: 2.6.32 + + Displaying notes found at file offset 0x00000274 with length 0x00000024: + Owner Data size Description + GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) + Build ID: df924a2b08a7e89f6e11251d4602022977af2670 + + Displaying notes found at file offset 0x002d6c30 with length 0x00000144: + Owner Data size Description + stapsdt 0x00000031 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: gc__start + Location: 0x00000000004371c3, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6bf6 + Arguments: -4@%ebx + stapsdt 0x00000030 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: gc__done + Location: 0x00000000004374e1, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6bf8 + Arguments: -8@%rax + stapsdt 0x00000045 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: function__entry + Location: 0x000000000053db6c, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6be8 + Arguments: 8@%rbp 8@%r12 -4@%eax + stapsdt 0x00000046 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: function__return + Location: 0x000000000053dba8, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6bea + Arguments: 8@%rbp 8@%r12 -4@%eax + +The above metadata contains information for SystemTap describing how it +can patch strategically-placed machine code instructions to enable the +tracing hooks used by a SystemTap script. + + +File: python.info, Node: Static DTrace probes, Next: Static SystemTap markers, Prev: Enabling the static markers, Up: Instrumenting CPython with DTrace and SystemTap + +10.16.2 Static DTrace probes +---------------------------- + +The following example DTrace script can be used to show the call/return +hierarchy of a Python script, only tracing within the invocation of a +function called “start”. In other words, import-time function +invocations are not going to be listed: + + self int indent; + + python$target:::function-entry + /copyinstr(arg1) == "start"/ + { + self->trace = 1; + } + + python$target:::function-entry + /self->trace/ + { + printf("%d\t%*s:", timestamp, 15, probename); + printf("%*s", self->indent, ""); + printf("%s:%s:%d\n", basename(copyinstr(arg0)), copyinstr(arg1), arg2); + self->indent++; + } + + python$target:::function-return + /self->trace/ + { + self->indent--; + printf("%d\t%*s:", timestamp, 15, probename); + printf("%*s", self->indent, ""); + printf("%s:%s:%d\n", basename(copyinstr(arg0)), copyinstr(arg1), arg2); + } + + python$target:::function-return + /copyinstr(arg1) == "start"/ + { + self->trace = 0; + } + +It can be invoked like this: + + $ sudo dtrace -q -s call_stack.d -c "python3.6 script.py" + +The output looks like this: + + 156641360502280 function-entry:call_stack.py:start:23 + 156641360518804 function-entry: call_stack.py:function_1:1 + 156641360532797 function-entry: call_stack.py:function_3:9 + 156641360546807 function-return: call_stack.py:function_3:10 + 156641360563367 function-return: call_stack.py:function_1:2 + 156641360578365 function-entry: call_stack.py:function_2:5 + 156641360591757 function-entry: call_stack.py:function_1:1 + 156641360605556 function-entry: call_stack.py:function_3:9 + 156641360617482 function-return: call_stack.py:function_3:10 + 156641360629814 function-return: call_stack.py:function_1:2 + 156641360642285 function-return: call_stack.py:function_2:6 + 156641360656770 function-entry: call_stack.py:function_3:9 + 156641360669707 function-return: call_stack.py:function_3:10 + 156641360687853 function-entry: call_stack.py:function_4:13 + 156641360700719 function-return: call_stack.py:function_4:14 + 156641360719640 function-entry: call_stack.py:function_5:18 + 156641360732567 function-return: call_stack.py:function_5:21 + 156641360747370 function-return:call_stack.py:start:28 + + +File: python.info, Node: Static SystemTap markers, Next: Available static markers, Prev: Static DTrace probes, Up: Instrumenting CPython with DTrace and SystemTap + +10.16.3 Static SystemTap markers +-------------------------------- + +The low-level way to use the SystemTap integration is to use the static +markers directly. This requires you to explicitly state the binary file +containing them. + +For example, this SystemTap script can be used to show the call/return +hierarchy of a Python script: + + probe process("python").mark("function__entry") { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + + printf("%s => %s in %s:%d\\n", + thread_indent(1), funcname, filename, lineno); + } + + probe process("python").mark("function__return") { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + + printf("%s <= %s in %s:%d\\n", + thread_indent(-1), funcname, filename, lineno); + } + +It can be invoked like this: + + $ stap \ + show-call-hierarchy.stp \ + -c "./python test.py" + +The output looks like this: + + 11408 python(8274): => __contains__ in Lib/_abcoll.py:362 + 11414 python(8274): => __getitem__ in Lib/os.py:425 + 11418 python(8274): => encode in Lib/os.py:490 + 11424 python(8274): <= encode in Lib/os.py:493 + 11428 python(8274): <= __getitem__ in Lib/os.py:426 + 11433 python(8274): <= __contains__ in Lib/_abcoll.py:366 + +where the columns are: + + - time in microseconds since start of script + + - name of executable + + - PID of process + +and the remainder indicates the call/return hierarchy as the script +executes. + +For a ‘–enable-shared’ build of CPython, the markers are contained +within the libpython shared library, and the probe’s dotted path needs +to reflect this. For example, this line from the above example: + + probe process("python").mark("function__entry") { + +should instead read: + + probe process("python").library("libpython3.6dm.so.1.0").mark("function__entry") { + +(assuming a debug build of CPython 3.6) + + +File: python.info, Node: Available static markers, Next: SystemTap Tapsets, Prev: Static SystemTap markers, Up: Instrumenting CPython with DTrace and SystemTap + +10.16.4 Available static markers +-------------------------------- + + -- Object: function__entry(str filename, str funcname, int lineno) + + This marker indicates that execution of a Python function has + begun. It is only triggered for pure-Python (bytecode) functions. + + The filename, function name, and line number are provided back to + the tracing script as positional arguments, which must be accessed + using ‘$arg1’, ‘$arg2’, ‘$arg3’: + + * ‘$arg1’ : ‘(const char *)’ filename, accessible using + ‘user_string($arg1)’ + + * ‘$arg2’ : ‘(const char *)’ function name, accessible + using ‘user_string($arg2)’ + + * ‘$arg3’ : ‘int’ line number + + -- Object: function__return(str filename, str funcname, int lineno) + + This marker is the converse of ‘function__entry()’, and indicates + that execution of a Python function has ended (either via ‘return’, + or via an exception). It is only triggered for pure-Python + (bytecode) functions. + + The arguments are the same as for ‘function__entry()’ + + -- Object: line(str filename, str funcname, int lineno) + + This marker indicates a Python line is about to be executed. It is + the equivalent of line-by-line tracing with a Python profiler. It + is not triggered within C functions. + + The arguments are the same as for ‘function__entry()’. + + -- Object: gc__start(int generation) + + Fires when the Python interpreter starts a garbage collection + cycle. ‘arg0’ is the generation to scan, like *note gc.collect(): + 22e. + + -- Object: gc__done(long collected) + + Fires when the Python interpreter finishes a garbage collection + cycle. ‘arg0’ is the number of collected objects. + + -- Object: import__find__load__start(str modulename) + + Fires before *note importlib: 9b. attempts to find and load the + module. ‘arg0’ is the module name. + + New in version 3.7. + + -- Object: import__find__load__done(str modulename, int found) + + Fires after *note importlib: 9b.’s find_and_load function is + called. ‘arg0’ is the module name, ‘arg1’ indicates if module was + successfully loaded. + + New in version 3.7. + + -- Object: audit(str event, void *tuple) + + Fires when *note sys.audit(): 31ea. or *note PySys_Audit(): 31eb. + is called. ‘arg0’ is the event name as C string, ‘arg1’ is a *note + PyObject: 4ba. pointer to a tuple object. + + New in version 3.8. + + +File: python.info, Node: SystemTap Tapsets, Next: Examples<36>, Prev: Available static markers, Up: Instrumenting CPython with DTrace and SystemTap + +10.16.5 SystemTap Tapsets +------------------------- + +The higher-level way to use the SystemTap integration is to use a +“tapset”: SystemTap’s equivalent of a library, which hides some of the +lower-level details of the static markers. + +Here is a tapset file, based on a non-shared build of CPython: + + /* + Provide a higher-level wrapping around the function__entry and + function__return markers: + \*/ + probe python.function.entry = process("python").mark("function__entry") + { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + frameptr = $arg4 + } + probe python.function.return = process("python").mark("function__return") + { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + frameptr = $arg4 + } + +If this file is installed in SystemTap’s tapset directory (e.g. +‘/usr/share/systemtap/tapset’), then these additional probepoints become +available: + + -- Object: python.function.entry(str filename, str funcname, int + lineno, frameptr) + + This probe point indicates that execution of a Python function has + begun. It is only triggered for pure-Python (bytecode) functions. + + -- Object: python.function.return(str filename, str funcname, int + lineno, frameptr) + + This probe point is the converse of ‘python.function.return’, and + indicates that execution of a Python function has ended (either via + ‘return’, or via an exception). It is only triggered for + pure-Python (bytecode) functions. + + +File: python.info, Node: Examples<36>, Prev: SystemTap Tapsets, Up: Instrumenting CPython with DTrace and SystemTap + +10.16.6 Examples +---------------- + +This SystemTap script uses the tapset above to more cleanly implement +the example given above of tracing the Python function-call hierarchy, +without needing to directly name the static markers: + + probe python.function.entry + { + printf("%s => %s in %s:%d\n", + thread_indent(1), funcname, filename, lineno); + } + + probe python.function.return + { + printf("%s <= %s in %s:%d\n", + thread_indent(-1), funcname, filename, lineno); + } + +The following script uses the tapset above to provide a top-like view of +all running CPython code, showing the top 20 most frequently-entered +bytecode frames, each second, across the whole system: + + global fn_calls; + + probe python.function.entry + { + fn_calls[pid(), filename, funcname, lineno] += 1; + } + + probe timer.ms(1000) { + printf("\033[2J\033[1;1H") /* clear screen \*/ + printf("%6s %80s %6s %30s %6s\n", + "PID", "FILENAME", "LINE", "FUNCTION", "CALLS") + foreach ([pid, filename, funcname, lineno] in fn_calls- limit 20) { + printf("%6d %80s %6d %30s %6d\n", + pid, filename, lineno, funcname, + fn_calls[pid, filename, funcname, lineno]); + } + delete fn_calls; + } + + +File: python.info, Node: Python Frequently Asked Questions, Next: Glossary, Prev: Python HOWTOs, Up: Top + +11 Python Frequently Asked Questions +************************************ + +* Menu: + +* General Python FAQ:: +* Programming FAQ:: +* Design and History FAQ:: +* Library and Extension FAQ:: +* Extending/Embedding FAQ:: +* Python on Windows FAQ:: +* Graphic User Interface FAQ:: +* “Why is Python Installed on my Computer?” FAQ:: + + +File: python.info, Node: General Python FAQ, Next: Programming FAQ, Up: Python Frequently Asked Questions + +11.1 General Python FAQ +======================= + +* Menu: + +* General Information:: +* Python in the real world:: + + +File: python.info, Node: General Information, Next: Python in the real world, Up: General Python FAQ + +11.1.1 General Information +-------------------------- + +* Menu: + +* What is Python?:: +* What is the Python Software Foundation?:: +* Are there copyright restrictions on the use of Python?:: +* Why was Python created in the first place?:: +* What is Python good for?:: +* How does the Python version numbering scheme work?:: +* How do I obtain a copy of the Python source?:: +* How do I get documentation on Python?:: +* I’ve never programmed before. Is there a Python tutorial?: I’ve never programmed before Is there a Python tutorial?. +* Is there a newsgroup or mailing list devoted to Python?:: +* How do I get a beta test version of Python?:: +* How do I submit bug reports and patches for Python?:: +* Are there any published articles about Python that I can reference?:: +* Are there any books on Python?:: +* Where in the world is www.python.org located?: Where in the world is www python org located?. +* Why is it called Python?:: +* Do I have to like “Monty Python’s Flying Circus”?:: + + +File: python.info, Node: What is Python?, Next: What is the Python Software Foundation?, Up: General Information + +11.1.1.1 What is Python? +........................ + +Python is an interpreted, interactive, object-oriented programming +language. It incorporates modules, exceptions, dynamic typing, very +high level dynamic data types, and classes. It supports multiple +programming paradigms beyond object-oriented programming, such as +procedural and functional programming. Python combines remarkable power +with very clear syntax. It has interfaces to many system calls and +libraries, as well as to various window systems, and is extensible in C +or C++. It is also usable as an extension language for applications +that need a programmable interface. Finally, Python is portable: it +runs on many Unix variants including Linux and macOS, and on Windows. + +To find out more, start with *note The Python Tutorial: e8d. The +Beginner’s Guide to Python(1) links to other introductory tutorials and +resources for learning Python. + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/BeginnersGuide + + +File: python.info, Node: What is the Python Software Foundation?, Next: Are there copyright restrictions on the use of Python?, Prev: What is Python?, Up: General Information + +11.1.1.2 What is the Python Software Foundation? +................................................ + +The Python Software Foundation is an independent non-profit organization +that holds the copyright on Python versions 2.1 and newer. The PSF’s +mission is to advance open source technology related to the Python +programming language and to publicize the use of Python. The PSF’s home +page is at ‘https://www.python.org/psf/’. + +Donations to the PSF are tax-exempt in the US. If you use Python and +find it helpful, please contribute via the PSF donation page(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/psf/donations/ + + +File: python.info, Node: Are there copyright restrictions on the use of Python?, Next: Why was Python created in the first place?, Prev: What is the Python Software Foundation?, Up: General Information + +11.1.1.3 Are there copyright restrictions on the use of Python? +............................................................... + +You can do anything you want with the source, as long as you leave the +copyrights in and display those copyrights in any documentation about +Python that you produce. If you honor the copyright rules, it’s OK to +use Python for commercial use, to sell copies of Python in source or +binary form (modified or unmodified), or to sell products that +incorporate Python in some form. We would still like to know about all +commercial use of Python, of course. + +See the PSF license page(1) to find further explanations and a link to +the full text of the license. + +The Python logo is trademarked, and in certain cases permission is +required to use it. Consult the Trademark Usage Policy(2) for more +information. + + ---------- Footnotes ---------- + + (1) https://www.python.org/psf/license/ + + (2) https://www.python.org/psf/trademarks/ + + +File: python.info, Node: Why was Python created in the first place?, Next: What is Python good for?, Prev: Are there copyright restrictions on the use of Python?, Up: General Information + +11.1.1.4 Why was Python created in the first place? +................................................... + +Here’s a `very' brief summary of what started it all, written by Guido +van Rossum: + + I had extensive experience with implementing an interpreted + language in the ABC group at CWI, and from working with this group + I had learned a lot about language design. This is the origin of + many Python features, including the use of indentation for + statement grouping and the inclusion of very-high-level data types + (although the details are all different in Python). + + I had a number of gripes about the ABC language, but also liked + many of its features. It was impossible to extend the ABC language + (or its implementation) to remedy my complaints – in fact its lack + of extensibility was one of its biggest problems. I had some + experience with using Modula-2+ and talked with the designers of + Modula-3 and read the Modula-3 report. Modula-3 is the origin of + the syntax and semantics used for exceptions, and some other Python + features. + + I was working in the Amoeba distributed operating system group at + CWI. We needed a better way to do system administration than by + writing either C programs or Bourne shell scripts, since Amoeba had + its own system call interface which wasn’t easily accessible from + the Bourne shell. My experience with error handling in Amoeba made + me acutely aware of the importance of exceptions as a programming + language feature. + + It occurred to me that a scripting language with a syntax like ABC + but with access to the Amoeba system calls would fill the need. I + realized that it would be foolish to write an Amoeba-specific + language, so I decided that I needed a language that was generally + extensible. + + During the 1989 Christmas holidays, I had a lot of time on my hand, + so I decided to give it a try. During the next year, while still + mostly working on it in my own time, Python was used in the Amoeba + project with increasing success, and the feedback from colleagues + made me add many early improvements. + + In February 1991, after just over a year of development, I decided + to post to USENET. The rest is in the ‘Misc/HISTORY’ file. + + +File: python.info, Node: What is Python good for?, Next: How does the Python version numbering scheme work?, Prev: Why was Python created in the first place?, Up: General Information + +11.1.1.5 What is Python good for? +................................. + +Python is a high-level general-purpose programming language that can be +applied to many different classes of problems. + +The language comes with a large standard library that covers areas such +as string processing (regular expressions, Unicode, calculating +differences between files), Internet protocols (HTTP, FTP, SMTP, +XML-RPC, POP, IMAP, CGI programming), software engineering (unit +testing, logging, profiling, parsing Python code), and operating system +interfaces (system calls, filesystems, TCP/IP sockets). Look at the +table of contents for *note The Python Standard Library: e8e. to get an +idea of what’s available. A wide variety of third-party extensions are +also available. Consult the Python Package Index(1) to find packages of +interest to you. + + ---------- Footnotes ---------- + + (1) https://pypi.org + + +File: python.info, Node: How does the Python version numbering scheme work?, Next: How do I obtain a copy of the Python source?, Prev: What is Python good for?, Up: General Information + +11.1.1.6 How does the Python version numbering scheme work? +........................................................... + +Python versions are numbered A.B.C or A.B. A is the major version number +– it is only incremented for really major changes in the language. B is +the minor version number, incremented for less earth-shattering changes. +C is the micro-level – it is incremented for each bugfix release. See +PEP 6(1) for more information about bugfix releases. + +Not all releases are bugfix releases. In the run-up to a new major +release, a series of development releases are made, denoted as alpha, +beta, or release candidate. Alphas are early releases in which +interfaces aren’t yet finalized; it’s not unexpected to see an interface +change between two alpha releases. Betas are more stable, preserving +existing interfaces but possibly adding new modules, and release +candidates are frozen, making no changes except as needed to fix +critical bugs. + +Alpha, beta and release candidate versions have an additional suffix. +The suffix for an alpha version is “aN” for some small number N, the +suffix for a beta version is “bN” for some small number N, and the +suffix for a release candidate version is “cN” for some small number N. +In other words, all versions labeled 2.0aN precede the versions labeled +2.0bN, which precede versions labeled 2.0cN, and `those' precede 2.0. + +You may also find version numbers with a “+” suffix, e.g. “2.2+”. +These are unreleased versions, built directly from the CPython +development repository. In practice, after a final minor release is +made, the version is incremented to the next minor version, which +becomes the “a0” version, e.g. “2.4a0”. + +See also the documentation for *note sys.version: 5d3, *note +sys.hexversion: 335d, and *note sys.version_info: b1a. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0006 + + +File: python.info, Node: How do I obtain a copy of the Python source?, Next: How do I get documentation on Python?, Prev: How does the Python version numbering scheme work?, Up: General Information + +11.1.1.7 How do I obtain a copy of the Python source? +..................................................... + +The latest Python source distribution is always available from +python.org, at ‘https://www.python.org/downloads/’. The latest +development sources can be obtained at +‘https://github.com/python/cpython/’. + +The source distribution is a gzipped tar file containing the complete C +source, Sphinx-formatted documentation, Python library modules, example +programs, and several useful pieces of freely distributable software. +The source will compile and run out of the box on most UNIX platforms. + +Consult the Getting Started section of the Python Developer’s Guide(1) +for more information on getting the source code and compiling it. + + ---------- Footnotes ---------- + + (1) https://devguide.python.org/setup/ + + +File: python.info, Node: How do I get documentation on Python?, Next: I’ve never programmed before Is there a Python tutorial?, Prev: How do I obtain a copy of the Python source?, Up: General Information + +11.1.1.8 How do I get documentation on Python? +.............................................. + +The standard documentation for the current stable version of Python is +available at ‘https://docs.python.org/3/’. PDF, plain text, and +downloadable HTML versions are also available at +‘https://docs.python.org/3/download.html’. + +The documentation is written in reStructuredText and processed by the +Sphinx documentation tool(1). The reStructuredText source for the +documentation is part of the Python source distribution. + + ---------- Footnotes ---------- + + (1) http://sphinx-doc.org/ + + +File: python.info, Node: I’ve never programmed before Is there a Python tutorial?, Next: Is there a newsgroup or mailing list devoted to Python?, Prev: How do I get documentation on Python?, Up: General Information + +11.1.1.9 I’ve never programmed before. Is there a Python tutorial? +.................................................................. + +There are numerous tutorials and books available. The standard +documentation includes *note The Python Tutorial: e8d. + +Consult the Beginner’s Guide(1) to find information for beginning Python +programmers, including lists of tutorials. + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/BeginnersGuide + + +File: python.info, Node: Is there a newsgroup or mailing list devoted to Python?, Next: How do I get a beta test version of Python?, Prev: I’ve never programmed before Is there a Python tutorial?, Up: General Information + +11.1.1.10 Is there a newsgroup or mailing list devoted to Python? +................................................................. + +There is a newsgroup, ‘comp.lang.python’, and a mailing list, +python-list(1). The newsgroup and mailing list are gatewayed into each +other – if you can read news it’s unnecessary to subscribe to the +mailing list. ‘comp.lang.python’ is high-traffic, receiving hundreds of +postings every day, and Usenet readers are often more able to cope with +this volume. + +Announcements of new software releases and events can be found in +comp.lang.python.announce, a low-traffic moderated list that receives +about five postings per day. It’s available as the python-announce +mailing list(2). + +More info about other mailing lists and newsgroups can be found at +‘https://www.python.org/community/lists/’. + + ---------- Footnotes ---------- + + (1) https://mail.python.org/mailman/listinfo/python-list + + (2) https://mail.python.org/mailman/listinfo/python-announce-list + + +File: python.info, Node: How do I get a beta test version of Python?, Next: How do I submit bug reports and patches for Python?, Prev: Is there a newsgroup or mailing list devoted to Python?, Up: General Information + +11.1.1.11 How do I get a beta test version of Python? +..................................................... + +Alpha and beta releases are available from +‘https://www.python.org/downloads/’. All releases are announced on the +comp.lang.python and comp.lang.python.announce newsgroups and on the +Python home page at ‘https://www.python.org/’; an RSS feed of news is +available. + +You can also access the development version of Python through Git. See +The Python Developer’s Guide(1) for details. + + ---------- Footnotes ---------- + + (1) https://devguide.python.org/ + + +File: python.info, Node: How do I submit bug reports and patches for Python?, Next: Are there any published articles about Python that I can reference?, Prev: How do I get a beta test version of Python?, Up: General Information + +11.1.1.12 How do I submit bug reports and patches for Python? +............................................................. + +To report a bug or submit a patch, please use the Roundup installation +at ‘https://bugs.python.org/’. + +You must have a Roundup account to report bugs; this makes it possible +for us to contact you if we have follow-up questions. It will also +enable Roundup to send you updates as we act on your bug. If you had +previously used SourceForge to report bugs to Python, you can obtain +your Roundup password through Roundup’s password reset procedure(1). + +For more information on how Python is developed, consult the Python +Developer’s Guide(2). + + ---------- Footnotes ---------- + + (1) https://bugs.python.org/user?@template=forgotten + + (2) https://devguide.python.org/ + + +File: python.info, Node: Are there any published articles about Python that I can reference?, Next: Are there any books on Python?, Prev: How do I submit bug reports and patches for Python?, Up: General Information + +11.1.1.13 Are there any published articles about Python that I can reference? +............................................................................. + +It’s probably best to cite your favorite book about Python. + +The very first article about Python was written in 1991 and is now quite +outdated. + + Guido van Rossum and Jelke de Boer, “Interactively Testing Remote + Servers Using the Python Programming Language”, CWI Quarterly, + Volume 4, Issue 4 (December 1991), Amsterdam, pp 283–303. + + +File: python.info, Node: Are there any books on Python?, Next: Where in the world is www python org located?, Prev: Are there any published articles about Python that I can reference?, Up: General Information + +11.1.1.14 Are there any books on Python? +........................................ + +Yes, there are many, and more are being published. See the python.org +wiki at ‘https://wiki.python.org/moin/PythonBooks’ for a list. + +You can also search online bookstores for “Python” and filter out the +Monty Python references; or perhaps search for “Python” and “language”. + + +File: python.info, Node: Where in the world is www python org located?, Next: Why is it called Python?, Prev: Are there any books on Python?, Up: General Information + +11.1.1.15 Where in the world is www.python.org located? +....................................................... + +The Python project’s infrastructure is located all over the world and is +managed by the Python Infrastructure Team. Details here(1). + + ---------- Footnotes ---------- + + (1) http://infra.psf.io + + +File: python.info, Node: Why is it called Python?, Next: Do I have to like “Monty Python’s Flying Circus”?, Prev: Where in the world is www python org located?, Up: General Information + +11.1.1.16 Why is it called Python? +.................................. + +When he began implementing Python, Guido van Rossum was also reading the +published scripts from "Monty Python’s Flying Circus"(1), a BBC comedy +series from the 1970s. Van Rossum thought he needed a name that was +short, unique, and slightly mysterious, so he decided to call the +language Python. + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Monty_Python + + +File: python.info, Node: Do I have to like “Monty Python’s Flying Circus”?, Prev: Why is it called Python?, Up: General Information + +11.1.1.17 Do I have to like “Monty Python’s Flying Circus”? +........................................................... + +No, but it helps. :) + + +File: python.info, Node: Python in the real world, Prev: General Information, Up: General Python FAQ + +11.1.2 Python in the real world +------------------------------- + +* Menu: + +* How stable is Python?:: +* How many people are using Python?:: +* Have any significant projects been done in Python?:: +* What new developments are expected for Python in the future?:: +* Is it reasonable to propose incompatible changes to Python?:: +* Is Python a good language for beginning programmers?:: + + +File: python.info, Node: How stable is Python?, Next: How many people are using Python?, Up: Python in the real world + +11.1.2.1 How stable is Python? +.............................. + +Very stable. New, stable releases have been coming out roughly every 6 +to 18 months since 1991, and this seems likely to continue. As of +version 3.9, Python will have a major new release every 12 months ( PEP +602(1)). + +The developers issue “bugfix” releases of older versions, so the +stability of existing releases gradually improves. Bugfix releases, +indicated by a third component of the version number (e.g. 3.5.3, +3.6.2), are managed for stability; only fixes for known problems are +included in a bugfix release, and it’s guaranteed that interfaces will +remain the same throughout a series of bugfix releases. + +The latest stable releases can always be found on the Python download +page(2). There are two production-ready versions of Python: 2.x and +3.x. The recommended version is 3.x, which is supported by most widely +used libraries. Although 2.x is still widely used, it will not be +maintained after January 1, 2020(3). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0602 + + (2) https://www.python.org/downloads/ + + (3) https://www.python.org/dev/peps/pep-0373/ + + +File: python.info, Node: How many people are using Python?, Next: Have any significant projects been done in Python?, Prev: How stable is Python?, Up: Python in the real world + +11.1.2.2 How many people are using Python? +.......................................... + +There are probably millions of users, though it’s difficult to obtain an +exact count. + +Python is available for free download, so there are no sales figures, +and it’s available from many different sites and packaged with many +Linux distributions, so download statistics don’t tell the whole story +either. + +The comp.lang.python newsgroup is very active, but not all Python users +post to the group or even read it. + + +File: python.info, Node: Have any significant projects been done in Python?, Next: What new developments are expected for Python in the future?, Prev: How many people are using Python?, Up: Python in the real world + +11.1.2.3 Have any significant projects been done in Python? +........................................................... + +See ‘https://www.python.org/about/success’ for a list of projects that +use Python. Consulting the proceedings for past Python conferences(1) +will reveal contributions from many different companies and +organizations. + +High-profile Python projects include the Mailman mailing list manager(2) +and the Zope application server(3). Several Linux distributions, most +notably Red Hat(4), have written part or all of their installer and +system administration software in Python. Companies that use Python +internally include Google, Yahoo, and Lucasfilm Ltd. + + ---------- Footnotes ---------- + + (1) https://www.python.org/community/workshops/ + + (2) http://www.list.org + + (3) http://www.zope.org + + (4) https://www.redhat.com + + +File: python.info, Node: What new developments are expected for Python in the future?, Next: Is it reasonable to propose incompatible changes to Python?, Prev: Have any significant projects been done in Python?, Up: Python in the real world + +11.1.2.4 What new developments are expected for Python in the future? +..................................................................... + +See ‘https://www.python.org/dev/peps/’ for the Python Enhancement +Proposals (PEPs). PEPs are design documents describing a suggested new +feature for Python, providing a concise technical specification and a +rationale. Look for a PEP titled “Python X.Y Release Schedule”, where +X.Y is a version that hasn’t been publicly released yet. + +New development is discussed on the python-dev mailing list(1). + + ---------- Footnotes ---------- + + (1) https://mail.python.org/mailman/listinfo/python-dev/ + + +File: python.info, Node: Is it reasonable to propose incompatible changes to Python?, Next: Is Python a good language for beginning programmers?, Prev: What new developments are expected for Python in the future?, Up: Python in the real world + +11.1.2.5 Is it reasonable to propose incompatible changes to Python? +.................................................................... + +In general, no. There are already millions of lines of Python code +around the world, so any change in the language that invalidates more +than a very small fraction of existing programs has to be frowned upon. +Even if you can provide a conversion program, there’s still the problem +of updating all documentation; many books have been written about +Python, and we don’t want to invalidate them all at a single stroke. + +Providing a gradual upgrade path is necessary if a feature has to be +changed. PEP 5(1) describes the procedure followed for introducing +backward-incompatible changes while minimizing disruption for users. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0005 + + +File: python.info, Node: Is Python a good language for beginning programmers?, Prev: Is it reasonable to propose incompatible changes to Python?, Up: Python in the real world + +11.1.2.6 Is Python a good language for beginning programmers? +............................................................. + +Yes. + +It is still common to start students with a procedural and statically +typed language such as Pascal, C, or a subset of C++ or Java. Students +may be better served by learning Python as their first language. Python +has a very simple and consistent syntax and a large standard library +and, most importantly, using Python in a beginning programming course +lets students concentrate on important programming skills such as +problem decomposition and data type design. With Python, students can +be quickly introduced to basic concepts such as loops and procedures. +They can probably even work with user-defined objects in their very +first course. + +For a student who has never programmed before, using a statically typed +language seems unnatural. It presents additional complexity that the +student must master and slows the pace of the course. The students are +trying to learn to think like a computer, decompose problems, design +consistent interfaces, and encapsulate data. While learning to use a +statically typed language is important in the long term, it is not +necessarily the best topic to address in the students’ first programming +course. + +Many other aspects of Python make it a good first language. Like Java, +Python has a large standard library so that students can be assigned +programming projects very early in the course that `do' something. +Assignments aren’t restricted to the standard four-function calculator +and check balancing programs. By using the standard library, students +can gain the satisfaction of working on realistic applications as they +learn the fundamentals of programming. Using the standard library also +teaches students about code reuse. Third-party modules such as PyGame +are also helpful in extending the students’ reach. + +Python’s interactive interpreter enables students to test language +features while they’re programming. They can keep a window with the +interpreter running while they enter their program’s source in another +window. If they can’t remember the methods for a list, they can do +something like this: + + >>> L = [] + >>> dir(L) + ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', + '__dir__', '__doc__', '__eq__', '__format__', '__ge__', + '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', + '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', + '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', + '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', + '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', + 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', + 'reverse', 'sort'] + >>> [d for d in dir(L) if '__' not in d] + ['append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] + + >>> help(L.append) + Help on built-in function append: + + append(...) + L.append(object) -> None -- append object to end + + >>> L.append(1) + >>> L + [1] + +With the interpreter, documentation is never far from the student as +they are programming. + +There are also good IDEs for Python. IDLE is a cross-platform IDE for +Python that is written in Python using Tkinter. PythonWin is a +Windows-specific IDE. Emacs users will be happy to know that there is a +very good Python mode for Emacs. All of these programming environments +provide syntax highlighting, auto-indenting, and access to the +interactive interpreter while coding. Consult the Python wiki(1) for a +full list of Python editing environments. + +If you want to discuss Python’s use in education, you may be interested +in joining the edu-sig mailing list(2). + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/PythonEditors + + (2) https://www.python.org/community/sigs/current/edu-sig + + +File: python.info, Node: Programming FAQ, Next: Design and History FAQ, Prev: General Python FAQ, Up: Python Frequently Asked Questions + +11.2 Programming FAQ +==================== + +* Menu: + +* General Questions:: +* Core Language:: +* Numbers and strings:: +* Performance: Performance<4>. +* Sequences (Tuples/Lists): Sequences Tuples/Lists. +* Objects:: +* Modules: Modules<3>. + + +File: python.info, Node: General Questions, Next: Core Language, Up: Programming FAQ + +11.2.1 General Questions +------------------------ + +* Menu: + +* Is there a source code level debugger with breakpoints, single-stepping, etc.?: Is there a source code level debugger with breakpoints single-stepping etc ?. +* Are there tools to help find bugs or perform static analysis?:: +* How can I create a stand-alone binary from a Python script?:: +* Are there coding standards or a style guide for Python programs?:: + + +File: python.info, Node: Is there a source code level debugger with breakpoints single-stepping etc ?, Next: Are there tools to help find bugs or perform static analysis?, Up: General Questions + +11.2.1.1 Is there a source code level debugger with breakpoints, single-stepping, etc.? +....................................................................................... + +Yes. + +Several debuggers for Python are described below, and the built-in +function *note breakpoint(): 2f9. allows you to drop into any of them. + +The pdb module is a simple but adequate console-mode debugger for +Python. It is part of the standard Python library, and is *note +documented in the Library Reference Manual: c9. You can also write your +own debugger by using the code for pdb as an example. + +The IDLE interactive development environment, which is part of the +standard Python distribution (normally available as Tools/scripts/idle), +includes a graphical debugger. + +PythonWin is a Python IDE that includes a GUI debugger based on pdb. +The Pythonwin debugger colors breakpoints and has quite a few cool +features such as debugging non-Pythonwin programs. Pythonwin is +available as part of the Python for Windows Extensions(1) project and as +a part of the ActivePython distribution (see +‘https://www.activestate.com/activepython’). + +Eric(2) is an IDE built on PyQt and the Scintilla editing component. + +Pydb is a version of the standard Python debugger pdb, modified for use +with DDD (Data Display Debugger), a popular graphical debugger front +end. Pydb can be found at ‘http://bashdb.sourceforge.net/pydb/’ and DDD +can be found at ‘https://www.gnu.org/software/ddd’. + +There are a number of commercial Python IDEs that include graphical +debuggers. They include: + + * Wing IDE (‘https://wingware.com/’) + + * Komodo IDE (‘https://komodoide.com/’) + + * PyCharm (‘https://www.jetbrains.com/pycharm/’) + + ---------- Footnotes ---------- + + (1) https://sourceforge.net/projects/pywin32/ + + (2) http://eric-ide.python-projects.org/ + + +File: python.info, Node: Are there tools to help find bugs or perform static analysis?, Next: How can I create a stand-alone binary from a Python script?, Prev: Is there a source code level debugger with breakpoints single-stepping etc ?, Up: General Questions + +11.2.1.2 Are there tools to help find bugs or perform static analysis? +...................................................................... + +Yes. + +Pylint(1) and Pyflakes(2) do basic checking that will help you catch +bugs sooner. + +Static type checkers such as Mypy(3), Pyre(4), and Pytype(5) can check +type hints in Python source code. + + ---------- Footnotes ---------- + + (1) https://www.pylint.org/ + + (2) https://github.com/PyCQA/pyflakes + + (3) http://mypy-lang.org/ + + (4) https://pyre-check.org/ + + (5) https://github.com/google/pytype + + +File: python.info, Node: How can I create a stand-alone binary from a Python script?, Next: Are there coding standards or a style guide for Python programs?, Prev: Are there tools to help find bugs or perform static analysis?, Up: General Questions + +11.2.1.3 How can I create a stand-alone binary from a Python script? +.................................................................... + +You don’t need the ability to compile Python to C code if all you want +is a stand-alone program that users can download and run without having +to install the Python distribution first. There are a number of tools +that determine the set of modules required by a program and bind these +modules together with a Python binary to produce a single executable. + +One is to use the freeze tool, which is included in the Python source +tree as ‘Tools/freeze’. It converts Python byte code to C arrays; a C +compiler you can embed all your modules into a new program, which is +then linked with the standard Python modules. + +It works by scanning your source recursively for import statements (in +both forms) and looking for the modules in the standard Python path as +well as in the source directory (for built-in modules). It then turns +the bytecode for modules written in Python into C code (array +initializers that can be turned into code objects using the marshal +module) and creates a custom-made config file that only contains those +built-in modules which are actually used in the program. It then +compiles the generated C code and links it with the rest of the Python +interpreter to form a self-contained binary which acts exactly like your +script. + +Obviously, freeze requires a C compiler. There are several other +utilities which don’t. One is Thomas Heller’s py2exe (Windows only) at + + ‘http://www.py2exe.org/’ + +Another tool is Anthony Tuininga’s cx_Freeze(1). + + ---------- Footnotes ---------- + + (1) https://anthony-tuininga.github.io/cx_Freeze/ + + +File: python.info, Node: Are there coding standards or a style guide for Python programs?, Prev: How can I create a stand-alone binary from a Python script?, Up: General Questions + +11.2.1.4 Are there coding standards or a style guide for Python programs? +......................................................................... + +Yes. The coding style required for standard library modules is +documented as PEP 8(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0008 + + +File: python.info, Node: Core Language, Next: Numbers and strings, Prev: General Questions, Up: Programming FAQ + +11.2.2 Core Language +-------------------- + +* Menu: + +* Why am I getting an UnboundLocalError when the variable has a value?:: +* What are the rules for local and global variables in Python?:: +* Why do lambdas defined in a loop with different values all return the same result?:: +* How do I share global variables across modules?:: +* What are the “best practices” for using import in a module?:: +* Why are default values shared between objects?:: +* How can I pass optional or keyword parameters from one function to another?:: +* What is the difference between arguments and parameters?:: +* Why did changing list ‘y’ also change list ‘x’?:: +* How do I write a function with output parameters (call by reference)?: How do I write a function with output parameters call by reference ?. +* How do you make a higher order function in Python?:: +* How do I copy an object in Python?:: +* How can I find the methods or attributes of an object?:: +* How can my code discover the name of an object?:: +* What’s up with the comma operator’s precedence?:: +* Is there an equivalent of C’s “?;” ternary operator?: Is there an equivalent of C’s “? ” ternary operator?. +* Is it possible to write obfuscated one-liners in Python?:: +* What does the slash(/) in the parameter list of a function mean?: What does the slash / in the parameter list of a function mean?. + + +File: python.info, Node: Why am I getting an UnboundLocalError when the variable has a value?, Next: What are the rules for local and global variables in Python?, Up: Core Language + +11.2.2.1 Why am I getting an UnboundLocalError when the variable has a value? +............................................................................. + +It can be a surprise to get the UnboundLocalError in previously working +code when it is modified by adding an assignment statement somewhere in +the body of a function. + +This code: + + >>> x = 10 + >>> def bar(): + ... print(x) + >>> bar() + 10 + +works, but this code: + + >>> x = 10 + >>> def foo(): + ... print(x) + ... x += 1 + +results in an UnboundLocalError: + + >>> foo() + Traceback (most recent call last): + ... + UnboundLocalError: local variable 'x' referenced before assignment + +This is because when you make an assignment to a variable in a scope, +that variable becomes local to that scope and shadows any similarly +named variable in the outer scope. Since the last statement in foo +assigns a new value to ‘x’, the compiler recognizes it as a local +variable. Consequently when the earlier ‘print(x)’ attempts to print +the uninitialized local variable and an error results. + +In the example above you can access the outer scope variable by +declaring it global: + + >>> x = 10 + >>> def foobar(): + ... global x + ... print(x) + ... x += 1 + >>> foobar() + 10 + +This explicit declaration is required in order to remind you that +(unlike the superficially analogous situation with class and instance +variables) you are actually modifying the value of the variable in the +outer scope: + + >>> print(x) + 11 + +You can do a similar thing in a nested scope using the *note nonlocal: +c3f. keyword: + + >>> def foo(): + ... x = 10 + ... def bar(): + ... nonlocal x + ... print(x) + ... x += 1 + ... bar() + ... print(x) + >>> foo() + 10 + 11 + + +File: python.info, Node: What are the rules for local and global variables in Python?, Next: Why do lambdas defined in a loop with different values all return the same result?, Prev: Why am I getting an UnboundLocalError when the variable has a value?, Up: Core Language + +11.2.2.2 What are the rules for local and global variables in Python? +..................................................................... + +In Python, variables that are only referenced inside a function are +implicitly global. If a variable is assigned a value anywhere within +the function’s body, it’s assumed to be a local unless explicitly +declared as global. + +Though a bit surprising at first, a moment’s consideration explains +this. On one hand, requiring *note global: ed8. for assigned variables +provides a bar against unintended side-effects. On the other hand, if +‘global’ was required for all global references, you’d be using ‘global’ +all the time. You’d have to declare as global every reference to a +built-in function or to a component of an imported module. This clutter +would defeat the usefulness of the ‘global’ declaration for identifying +side-effects. + + +File: python.info, Node: Why do lambdas defined in a loop with different values all return the same result?, Next: How do I share global variables across modules?, Prev: What are the rules for local and global variables in Python?, Up: Core Language + +11.2.2.3 Why do lambdas defined in a loop with different values all return the same result? +........................................................................................... + +Assume you use a for loop to define a few different lambdas (or even +plain functions), e.g.: + + >>> squares = [] + >>> for x in range(5): + ... squares.append(lambda: x**2) + +This gives you a list that contains 5 lambdas that calculate ‘x**2’. +You might expect that, when called, they would return, respectively, +‘0’, ‘1’, ‘4’, ‘9’, and ‘16’. However, when you actually try you will +see that they all return ‘16’: + + >>> squares[2]() + 16 + >>> squares[4]() + 16 + +This happens because ‘x’ is not local to the lambdas, but is defined in +the outer scope, and it is accessed when the lambda is called — not when +it is defined. At the end of the loop, the value of ‘x’ is ‘4’, so all +the functions now return ‘4**2’, i.e. ‘16’. You can also verify this +by changing the value of ‘x’ and see how the results of the lambdas +change: + + >>> x = 8 + >>> squares[2]() + 64 + +In order to avoid this, you need to save the values in variables local +to the lambdas, so that they don’t rely on the value of the global ‘x’: + + >>> squares = [] + >>> for x in range(5): + ... squares.append(lambda n=x: n**2) + +Here, ‘n=x’ creates a new variable ‘n’ local to the lambda and computed +when the lambda is defined so that it has the same value that ‘x’ had at +that point in the loop. This means that the value of ‘n’ will be ‘0’ in +the first lambda, ‘1’ in the second, ‘2’ in the third, and so on. +Therefore each lambda will now return the correct result: + + >>> squares[2]() + 4 + >>> squares[4]() + 16 + +Note that this behaviour is not peculiar to lambdas, but applies to +regular functions too. + + +File: python.info, Node: How do I share global variables across modules?, Next: What are the “best practices” for using import in a module?, Prev: Why do lambdas defined in a loop with different values all return the same result?, Up: Core Language + +11.2.2.4 How do I share global variables across modules? +........................................................ + +The canonical way to share information across modules within a single +program is to create a special module (often called config or cfg). +Just import the config module in all modules of your application; the +module then becomes available as a global name. Because there is only +one instance of each module, any changes made to the module object get +reflected everywhere. For example: + +config.py: + + x = 0 # Default value of the 'x' configuration setting + +mod.py: + + import config + config.x = 1 + +main.py: + + import config + import mod + print(config.x) + +Note that using a module is also the basis for implementing the +Singleton design pattern, for the same reason. + + +File: python.info, Node: What are the “best practices” for using import in a module?, Next: Why are default values shared between objects?, Prev: How do I share global variables across modules?, Up: Core Language + +11.2.2.5 What are the “best practices” for using import in a module? +.................................................................... + +In general, don’t use ‘from modulename import *’. Doing so clutters the +importer’s namespace, and makes it much harder for linters to detect +undefined names. + +Import modules at the top of a file. Doing so makes it clear what other +modules your code requires and avoids questions of whether the module +name is in scope. Using one import per line makes it easy to add and +delete module imports, but using multiple imports per line uses less +screen space. + +It’s good practice if you import modules in the following order: + + 1. standard library modules – e.g. ‘sys’, ‘os’, ‘getopt’, ‘re’ + + 2. third-party library modules (anything installed in Python’s + site-packages directory) – e.g. mx.DateTime, ZODB, PIL.Image, etc. + + 3. locally-developed modules + +It is sometimes necessary to move imports to a function or class to +avoid problems with circular imports. Gordon McMillan says: + + Circular imports are fine where both modules use the “import + ” form of import. They fail when the 2nd module wants to + grab a name out of the first (“from module import name”) and the + import is at the top level. That’s because names in the 1st are + not yet available, because the first module is busy importing the + 2nd. + +In this case, if the second module is only used in one function, then +the import can easily be moved into that function. By the time the +import is called, the first module will have finished initializing, and +the second module can do its import. + +It may also be necessary to move imports out of the top level of code if +some of the modules are platform-specific. In that case, it may not +even be possible to import all of the modules at the top of the file. +In this case, importing the correct modules in the corresponding +platform-specific code is a good option. + +Only move imports into a local scope, such as inside a function +definition, if it’s necessary to solve a problem such as avoiding a +circular import or are trying to reduce the initialization time of a +module. This technique is especially helpful if many of the imports are +unnecessary depending on how the program executes. You may also want to +move imports into a function if the modules are only ever used in that +function. Note that loading a module the first time may be expensive +because of the one time initialization of the module, but loading a +module multiple times is virtually free, costing only a couple of +dictionary lookups. Even if the module name has gone out of scope, the +module is probably available in *note sys.modules: 1152. + + +File: python.info, Node: Why are default values shared between objects?, Next: How can I pass optional or keyword parameters from one function to another?, Prev: What are the “best practices” for using import in a module?, Up: Core Language + +11.2.2.6 Why are default values shared between objects? +....................................................... + +This type of bug commonly bites neophyte programmers. Consider this +function: + + def foo(mydict={}): # Danger: shared reference to one dict for all calls + ... compute something ... + mydict[key] = value + return mydict + +The first time you call this function, ‘mydict’ contains a single item. +The second time, ‘mydict’ contains two items because when ‘foo()’ begins +executing, ‘mydict’ starts out with an item already in it. + +It is often expected that a function call creates new objects for +default values. This is not what happens. Default values are created +exactly once, when the function is defined. If that object is changed, +like the dictionary in this example, subsequent calls to the function +will refer to this changed object. + +By definition, immutable objects such as numbers, strings, tuples, and +‘None’, are safe from change. Changes to mutable objects such as +dictionaries, lists, and class instances can lead to confusion. + +Because of this feature, it is good programming practice to not use +mutable objects as default values. Instead, use ‘None’ as the default +value and inside the function, check if the parameter is ‘None’ and +create a new list/dictionary/whatever if it is. For example, don’t +write: + + def foo(mydict={}): + ... + +but: + + def foo(mydict=None): + if mydict is None: + mydict = {} # create a new dict for local namespace + +This feature can be useful. When you have a function that’s +time-consuming to compute, a common technique is to cache the parameters +and the resulting value of each call to the function, and return the +cached value if the same value is requested again. This is called +“memoizing”, and can be implemented like this: + + # Callers can only provide two parameters and optionally pass _cache by keyword + def expensive(arg1, arg2, *, _cache={}): + if (arg1, arg2) in _cache: + return _cache[(arg1, arg2)] + + # Calculate the value + result = ... expensive computation ... + _cache[(arg1, arg2)] = result # Store result in the cache + return result + +You could use a global variable containing a dictionary instead of the +default value; it’s a matter of taste. + + +File: python.info, Node: How can I pass optional or keyword parameters from one function to another?, Next: What is the difference between arguments and parameters?, Prev: Why are default values shared between objects?, Up: Core Language + +11.2.2.7 How can I pass optional or keyword parameters from one function to another? +.................................................................................... + +Collect the arguments using the ‘*’ and ‘**’ specifiers in the +function’s parameter list; this gives you the positional arguments as a +tuple and the keyword arguments as a dictionary. You can then pass +these arguments when calling another function by using ‘*’ and ‘**’: + + def f(x, *args, **kwargs): + ... + kwargs['width'] = '14.3c' + ... + g(x, *args, **kwargs) + + +File: python.info, Node: What is the difference between arguments and parameters?, Next: Why did changing list ‘y’ also change list ‘x’?, Prev: How can I pass optional or keyword parameters from one function to another?, Up: Core Language + +11.2.2.8 What is the difference between arguments and parameters? +................................................................. + +*note Parameters: 11d2. are defined by the names that appear in a +function definition, whereas *note arguments: 11ca. are the values +actually passed to a function when calling it. Parameters define what +types of arguments a function can accept. For example, given the +function definition: + + def func(foo, bar=None, **kwargs): + pass + +`foo', `bar' and `kwargs' are parameters of ‘func’. However, when +calling ‘func’, for example: + + func(42, bar=314, extra=somevar) + +the values ‘42’, ‘314’, and ‘somevar’ are arguments. + + +File: python.info, Node: Why did changing list ‘y’ also change list ‘x’?, Next: How do I write a function with output parameters call by reference ?, Prev: What is the difference between arguments and parameters?, Up: Core Language + +11.2.2.9 Why did changing list ‘y’ also change list ‘x’? +........................................................ + +If you wrote code like: + + >>> x = [] + >>> y = x + >>> y.append(10) + >>> y + [10] + >>> x + [10] + +you might be wondering why appending an element to ‘y’ changed ‘x’ too. + +There are two factors that produce this result: + + 1. Variables are simply names that refer to objects. Doing ‘y = x’ + doesn’t create a copy of the list – it creates a new variable ‘y’ + that refers to the same object ‘x’ refers to. This means that + there is only one object (the list), and both ‘x’ and ‘y’ refer to + it. + + 2. Lists are *note mutable: ebe, which means that you can change their + content. + +After the call to ‘append()’, the content of the mutable object has +changed from ‘[]’ to ‘[10]’. Since both the variables refer to the same +object, using either name accesses the modified value ‘[10]’. + +If we instead assign an immutable object to ‘x’: + + >>> x = 5 # ints are immutable + >>> y = x + >>> x = x + 1 # 5 can't be mutated, we are creating a new object here + >>> x + 6 + >>> y + 5 + +we can see that in this case ‘x’ and ‘y’ are not equal anymore. This is +because integers are *note immutable: eb6, and when we do ‘x = x + 1’ we +are not mutating the int ‘5’ by incrementing its value; instead, we are +creating a new object (the int ‘6’) and assigning it to ‘x’ (that is, +changing which object ‘x’ refers to). After this assignment we have two +objects (the ints ‘6’ and ‘5’) and two variables that refer to them (‘x’ +now refers to ‘6’ but ‘y’ still refers to ‘5’). + +Some operations (for example ‘y.append(10)’ and ‘y.sort()’) mutate the +object, whereas superficially similar operations (for example ‘y = y + +[10]’ and ‘sorted(y)’) create a new object. In general in Python (and +in all cases in the standard library) a method that mutates an object +will return ‘None’ to help avoid getting the two types of operations +confused. So if you mistakenly write ‘y.sort()’ thinking it will give +you a sorted copy of ‘y’, you’ll instead end up with ‘None’, which will +likely cause your program to generate an easily diagnosed error. + +However, there is one class of operations where the same operation +sometimes has different behaviors with different types: the augmented +assignment operators. For example, ‘+=’ mutates lists but not tuples or +ints (‘a_list += [1, 2, 3]’ is equivalent to ‘a_list.extend([1, 2, 3])’ +and mutates ‘a_list’, whereas ‘some_tuple += (1, 2, 3)’ and ‘some_int += +1’ create new objects). + +In other words: + + * If we have a mutable object (*note list: 262, *note dict: 1b8, + *note set: b6f, etc.), we can use some specific operations to + mutate it and all the variables that refer to it will see the + change. + + * If we have an immutable object (*note str: 330, *note int: 184, + *note tuple: 47e, etc.), all the variables that refer to it will + always see the same value, but operations that transform that value + into a new value always return a new object. + +If you want to know if two variables refer to the same object or not, +you can use the *note is: 10be. operator, or the built-in function *note +id(): d86. + + +File: python.info, Node: How do I write a function with output parameters call by reference ?, Next: How do you make a higher order function in Python?, Prev: Why did changing list ‘y’ also change list ‘x’?, Up: Core Language + +11.2.2.10 How do I write a function with output parameters (call by reference)? +............................................................................... + +Remember that arguments are passed by assignment in Python. Since +assignment just creates references to objects, there’s no alias between +an argument name in the caller and callee, and so no call-by-reference +per se. You can achieve the desired effect in a number of ways. + + 1. By returning a tuple of the results: + + >>> def func1(a, b): + ... a = 'new-value' # a and b are local names + ... b = b + 1 # assigned to new objects + ... return a, b # return new values + ... + >>> x, y = 'old-value', 99 + >>> func1(x, y) + ('new-value', 100) + + This is almost always the clearest solution. + + 2. By using global variables. This isn’t thread-safe, and is not + recommended. + + 3. By passing a mutable (changeable in-place) object: + + >>> def func2(a): + ... a[0] = 'new-value' # 'a' references a mutable list + ... a[1] = a[1] + 1 # changes a shared object + ... + >>> args = ['old-value', 99] + >>> func2(args) + >>> args + ['new-value', 100] + + 4. By passing in a dictionary that gets mutated: + + >>> def func3(args): + ... args['a'] = 'new-value' # args is a mutable dictionary + ... args['b'] = args['b'] + 1 # change it in-place + ... + >>> args = {'a': 'old-value', 'b': 99} + >>> func3(args) + >>> args + {'a': 'new-value', 'b': 100} + + 5. Or bundle up values in a class instance: + + >>> class Namespace: + ... def __init__(self, /, **args): + ... for key, value in args.items(): + ... setattr(self, key, value) + ... + >>> def func4(args): + ... args.a = 'new-value' # args is a mutable Namespace + ... args.b = args.b + 1 # change object in-place + ... + >>> args = Namespace(a='old-value', b=99) + >>> func4(args) + >>> vars(args) + {'a': 'new-value', 'b': 100} + + There’s almost never a good reason to get this complicated. + +Your best choice is to return a tuple containing the multiple results. + + +File: python.info, Node: How do you make a higher order function in Python?, Next: How do I copy an object in Python?, Prev: How do I write a function with output parameters call by reference ?, Up: Core Language + +11.2.2.11 How do you make a higher order function in Python? +............................................................ + +You have two choices: you can use nested scopes or you can use callable +objects. For example, suppose you wanted to define ‘linear(a,b)’ which +returns a function ‘f(x)’ that computes the value ‘a*x+b’. Using nested +scopes: + + def linear(a, b): + def result(x): + return a * x + b + return result + +Or using a callable object: + + class linear: + + def __init__(self, a, b): + self.a, self.b = a, b + + def __call__(self, x): + return self.a * x + self.b + +In both cases, + + taxes = linear(0.3, 2) + +gives a callable object where ‘taxes(10e6) == 0.3 * 10e6 + 2’. + +The callable object approach has the disadvantage that it is a bit +slower and results in slightly longer code. However, note that a +collection of callables can share their signature via inheritance: + + class exponential(linear): + # __init__ inherited + def __call__(self, x): + return self.a * (x ** self.b) + +Object can encapsulate state for several methods: + + class counter: + + value = 0 + + def set(self, x): + self.value = x + + def up(self): + self.value = self.value + 1 + + def down(self): + self.value = self.value - 1 + + count = counter() + inc, dec, reset = count.up, count.down, count.set + +Here ‘inc()’, ‘dec()’ and ‘reset()’ act like functions which share the +same counting variable. + + +File: python.info, Node: How do I copy an object in Python?, Next: How can I find the methods or attributes of an object?, Prev: How do you make a higher order function in Python?, Up: Core Language + +11.2.2.12 How do I copy an object in Python? +............................................ + +In general, try *note copy.copy(): 3cb. or *note copy.deepcopy(): 3cc. +for the general case. Not all objects can be copied, but most can. + +Some objects can be copied more easily. Dictionaries have a *note +copy(): 446. method: + + newdict = olddict.copy() + +Sequences can be copied by slicing: + + new_l = l[:] + + +File: python.info, Node: How can I find the methods or attributes of an object?, Next: How can my code discover the name of an object?, Prev: How do I copy an object in Python?, Up: Core Language + +11.2.2.13 How can I find the methods or attributes of an object? +................................................................ + +For an instance x of a user-defined class, ‘dir(x)’ returns an +alphabetized list of the names containing the instance attributes and +methods and attributes defined by its class. + + +File: python.info, Node: How can my code discover the name of an object?, Next: What’s up with the comma operator’s precedence?, Prev: How can I find the methods or attributes of an object?, Up: Core Language + +11.2.2.14 How can my code discover the name of an object? +......................................................... + +Generally speaking, it can’t, because objects don’t really have names. +Essentially, assignment always binds a name to a value; the same is true +of ‘def’ and ‘class’ statements, but in that case the value is a +callable. Consider the following code: + + >>> class A: + ... pass + ... + >>> B = A + >>> a = B() + >>> b = a + >>> print(b) + <__main__.A object at 0x16D07CC> + >>> print(a) + <__main__.A object at 0x16D07CC> + +Arguably the class has a name: even though it is bound to two names and +invoked through the name B the created instance is still reported as an +instance of class A. However, it is impossible to say whether the +instance’s name is a or b, since both names are bound to the same value. + +Generally speaking it should not be necessary for your code to “know the +names” of particular values. Unless you are deliberately writing +introspective programs, this is usually an indication that a change of +approach might be beneficial. + +In comp.lang.python, Fredrik Lundh once gave an excellent analogy in +answer to this question: + + The same way as you get the name of that cat you found on your + porch: the cat (object) itself cannot tell you its name, and it + doesn’t really care – so the only way to find out what it’s called + is to ask all your neighbours (namespaces) if it’s their cat + (object)… + + ….and don’t be surprised if you’ll find that it’s known by many + names, or no name at all! + + +File: python.info, Node: What’s up with the comma operator’s precedence?, Next: Is there an equivalent of C’s “? ” ternary operator?, Prev: How can my code discover the name of an object?, Up: Core Language + +11.2.2.15 What’s up with the comma operator’s precedence? +......................................................... + +Comma is not an operator in Python. Consider this session: + + >>> "a" in "b", "a" + (False, 'a') + +Since the comma is not an operator, but a separator between expressions +the above is evaluated as if you had entered: + + ("a" in "b"), "a" + +not: + + "a" in ("b", "a") + +The same is true of the various assignment operators (‘=’, ‘+=’ etc). +They are not truly operators but syntactic delimiters in assignment +statements. + + +File: python.info, Node: Is there an equivalent of C’s “? ” ternary operator?, Next: Is it possible to write obfuscated one-liners in Python?, Prev: What’s up with the comma operator’s precedence?, Up: Core Language + +11.2.2.16 Is there an equivalent of C’s “?:” ternary operator? +.............................................................. + +Yes, there is. The syntax is as follows: + + [on_true] if [expression] else [on_false] + + x, y = 50, 25 + small = x if x < y else y + +Before this syntax was introduced in Python 2.5, a common idiom was to +use logical operators: + + [expression] and [on_true] or [on_false] + +However, this idiom is unsafe, as it can give wrong results when +`on_true' has a false boolean value. Therefore, it is always better to +use the ‘... if ... else ...’ form. + + +File: python.info, Node: Is it possible to write obfuscated one-liners in Python?, Next: What does the slash / in the parameter list of a function mean?, Prev: Is there an equivalent of C’s “? ” ternary operator?, Up: Core Language + +11.2.2.17 Is it possible to write obfuscated one-liners in Python? +.................................................................. + +Yes. Usually this is done by nesting *note lambda: c2f. within +‘lambda’. See the following three examples, due to Ulf Bartelt: + + from functools import reduce + + # Primes < 1000 + print(list(filter(None,map(lambda y:y*reduce(lambda x,y:x*y!=0, + map(lambda x,y=y:y%x,range(2,int(pow(y,0.5)+1))),1),range(2,1000))))) + + # First 10 Fibonacci numbers + print(list(map(lambda x,f=lambda x,f:(f(x-1,f)+f(x-2,f)) if x>1 else 1: + f(x,f), range(10)))) + + # Mandelbrot set + print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y, + Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,Sy=Sy,L=lambda yc,Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,i=IM, + Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro, + i=i,Sx=Sx,F=lambda xc,yc,x,y,k,f=lambda xc,yc,x,y,k,f:(k<=0)or (x*x+y*y + >=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr( + 64+F(Ru+x*(Ro-Ru)/Sx,yc,0,0,i)),range(Sx))):L(Iu+y*(Io-Iu)/Sy),range(Sy + ))))(-2.1, 0.7, -1.2, 1.2, 30, 80, 24)) + # \___ ___/ \___ ___/ | | |__ lines on screen + # V V | |______ columns on screen + # | | |__________ maximum of "iterations" + # | |_________________ range on y axis + # |____________________________ range on x axis + +Don’t try this at home, kids! + + +File: python.info, Node: What does the slash / in the parameter list of a function mean?, Prev: Is it possible to write obfuscated one-liners in Python?, Up: Core Language + +11.2.2.18 What does the slash(/) in the parameter list of a function mean? +.......................................................................... + +A slash in the argument list of a function denotes that the parameters +prior to it are positional-only. Positional-only parameters are the +ones without an externally-usable name. Upon calling a function that +accepts positional-only parameters, arguments are mapped to parameters +based solely on their position. For example, *note divmod(): 152. is a +function that accepts positional-only parameters. Its documentation +looks like this: + + >>> help(divmod) + Help on built-in function divmod in module builtins: + + divmod(x, y, /) + Return the tuple (x//y, x%y). Invariant: div*y + mod == x. + +The slash at the end of the parameter list means that both parameters +are positional-only. Thus, calling *note divmod(): 152. with keyword +arguments would lead to an error: + + >>> divmod(x=3, y=4) + Traceback (most recent call last): + File "", line 1, in + TypeError: divmod() takes no keyword arguments + + +File: python.info, Node: Numbers and strings, Next: Performance<4>, Prev: Core Language, Up: Programming FAQ + +11.2.3 Numbers and strings +-------------------------- + +* Menu: + +* How do I specify hexadecimal and octal integers?:: +* Why does -22 // 10 return -3?:: +* How do I convert a string to a number?:: +* How do I convert a number to a string?:: +* How do I modify a string in place?:: +* How do I use strings to call functions/methods?:: +* Is there an equivalent to Perl’s chomp() for removing trailing newlines from strings?: Is there an equivalent to Perl’s chomp for removing trailing newlines from strings?. +* Is there a scanf() or sscanf() equivalent?: Is there a scanf or sscanf equivalent?. +* What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean?:: + + +File: python.info, Node: How do I specify hexadecimal and octal integers?, Next: Why does -22 // 10 return -3?, Up: Numbers and strings + +11.2.3.1 How do I specify hexadecimal and octal integers? +......................................................... + +To specify an octal digit, precede the octal value with a zero, and then +a lower or uppercase “o”. For example, to set the variable “a” to the +octal value “10” (8 in decimal), type: + + >>> a = 0o10 + >>> a + 8 + +Hexadecimal is just as easy. Simply precede the hexadecimal number with +a zero, and then a lower or uppercase “x”. Hexadecimal digits can be +specified in lower or uppercase. For example, in the Python +interpreter: + + >>> a = 0xa5 + >>> a + 165 + >>> b = 0XB2 + >>> b + 178 + + +File: python.info, Node: Why does -22 // 10 return -3?, Next: How do I convert a string to a number?, Prev: How do I specify hexadecimal and octal integers?, Up: Numbers and strings + +11.2.3.2 Why does -22 // 10 return -3? +...................................... + +It’s primarily driven by the desire that ‘i % j’ have the same sign as +‘j’. If you want that, and also want: + + i == (i // j) * j + (i % j) + +then integer division has to return the floor. C also requires that +identity to hold, and then compilers that truncate ‘i // j’ need to make +‘i % j’ have the same sign as ‘i’. + +There are few real use cases for ‘i % j’ when ‘j’ is negative. When ‘j’ +is positive, there are many, and in virtually all of them it’s more +useful for ‘i % j’ to be ‘>= 0’. If the clock says 10 now, what did it +say 200 hours ago? ‘-190 % 12 == 2’ is useful; ‘-190 % 12 == -10’ is a +bug waiting to bite. + + +File: python.info, Node: How do I convert a string to a number?, Next: How do I convert a number to a string?, Prev: Why does -22 // 10 return -3?, Up: Numbers and strings + +11.2.3.3 How do I convert a string to a number? +............................................... + +For integers, use the built-in *note int(): 184. type constructor, e.g. +‘int('144') == 144’. Similarly, *note float(): 187. converts to +floating-point, e.g. ‘float('144') == 144.0’. + +By default, these interpret the number as decimal, so that ‘int('0144') +== 144’ and ‘int('0x144')’ raises *note ValueError: 1fb. ‘int(string, +base)’ takes the base to convert from as a second optional argument, so +‘int('0x144', 16) == 324’. If the base is specified as 0, the number is +interpreted using Python’s rules: a leading ‘0o’ indicates octal, and +‘0x’ indicates a hex number. + +Do not use the built-in function *note eval(): b90. if all you need is +to convert strings to numbers. *note eval(): b90. will be significantly +slower and it presents a security risk: someone could pass you a Python +expression that might have unwanted side effects. For example, someone +could pass ‘__import__('os').system("rm -rf $HOME")’ which would erase +your home directory. + +*note eval(): b90. also has the effect of interpreting numbers as Python +expressions, so that e.g. ‘eval('09')’ gives a syntax error because +Python does not allow leading ‘0’ in a decimal number (except ‘0’). + + +File: python.info, Node: How do I convert a number to a string?, Next: How do I modify a string in place?, Prev: How do I convert a string to a number?, Up: Numbers and strings + +11.2.3.4 How do I convert a number to a string? +............................................... + +To convert, e.g., the number 144 to the string ‘144’, use the built-in +type constructor *note str(): 330. If you want a hexadecimal or octal +representation, use the built-in functions *note hex(): c66. or *note +oct(): c65. For fancy formatting, see the *note Formatted string +literals: 15c. and *note Format String Syntax: d1a. sections, e.g. +‘"{:04d}".format(144)’ yields ‘'0144'’ and ‘"{:.3f}".format(1.0/3.0)’ +yields ‘'0.333'’. + + +File: python.info, Node: How do I modify a string in place?, Next: How do I use strings to call functions/methods?, Prev: How do I convert a number to a string?, Up: Numbers and strings + +11.2.3.5 How do I modify a string in place? +........................................... + +You can’t, because strings are immutable. In most situations, you +should simply construct a new string from the various parts you want to +assemble it from. However, if you need an object with the ability to +modify in-place unicode data, try using an *note io.StringIO: 82d. +object or the *note array: 7. module: + + >>> import io + >>> s = "Hello, world" + >>> sio = io.StringIO(s) + >>> sio.getvalue() + 'Hello, world' + >>> sio.seek(7) + 7 + >>> sio.write("there!") + 6 + >>> sio.getvalue() + 'Hello, there!' + + >>> import array + >>> a = array.array('u', s) + >>> print(a) + array('u', 'Hello, world') + >>> a[0] = 'y' + >>> print(a) + array('u', 'yello, world') + >>> a.tounicode() + 'yello, world' + + +File: python.info, Node: How do I use strings to call functions/methods?, Next: Is there an equivalent to Perl’s chomp for removing trailing newlines from strings?, Prev: How do I modify a string in place?, Up: Numbers and strings + +11.2.3.6 How do I use strings to call functions/methods? +........................................................ + +There are various techniques. + + * The best is to use a dictionary that maps strings to functions. + The primary advantage of this technique is that the strings do not + need to match the names of the functions. This is also the primary + technique used to emulate a case construct: + + def a(): + pass + + def b(): + pass + + dispatch = {'go': a, 'stop': b} # Note lack of parens for funcs + + dispatch[get_input()]() # Note trailing parens to call function + + * Use the built-in function *note getattr(): 448.: + + import foo + getattr(foo, 'bar')() + + Note that *note getattr(): 448. works on any object, including + classes, class instances, modules, and so on. + + This is used in several places in the standard library, like this: + + class Foo: + def do_foo(self): + ... + + def do_bar(self): + ... + + f = getattr(foo_instance, 'do_' + opname) + f() + + * Use *note locals(): 455. or *note eval(): b90. to resolve the + function name: + + def myFunc(): + print("hello") + + fname = "myFunc" + + f = locals()[fname] + f() + + f = eval(fname) + f() + + Note: Using *note eval(): b90. is slow and dangerous. If you don’t + have absolute control over the contents of the string, someone + could pass a string that resulted in an arbitrary function being + executed. + + +File: python.info, Node: Is there an equivalent to Perl’s chomp for removing trailing newlines from strings?, Next: Is there a scanf or sscanf equivalent?, Prev: How do I use strings to call functions/methods?, Up: Numbers and strings + +11.2.3.7 Is there an equivalent to Perl’s chomp() for removing trailing newlines from strings? +.............................................................................................. + +You can use ‘S.rstrip("\r\n")’ to remove all occurrences of any line +terminator from the end of the string ‘S’ without removing other +trailing whitespace. If the string ‘S’ represents more than one line, +with several empty lines at the end, the line terminators for all the +blank lines will be removed: + + >>> lines = ("line 1 \r\n" + ... "\r\n" + ... "\r\n") + >>> lines.rstrip("\n\r") + 'line 1 ' + +Since this is typically only desired when reading text one line at a +time, using ‘S.rstrip()’ this way works well. + + +File: python.info, Node: Is there a scanf or sscanf equivalent?, Next: What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean?, Prev: Is there an equivalent to Perl’s chomp for removing trailing newlines from strings?, Up: Numbers and strings + +11.2.3.8 Is there a scanf() or sscanf() equivalent? +................................................... + +Not as such. + +For simple input parsing, the easiest approach is usually to split the +line into whitespace-delimited words using the *note split(): 7a1. +method of string objects and then convert decimal strings to numeric +values using *note int(): 184. or *note float(): 187. ‘split()’ +supports an optional “sep” parameter which is useful if the line uses +something other than whitespace as a separator. + +For more complicated input parsing, regular expressions are more +powerful than C’s ‘sscanf()’ and better suited for the task. + + +File: python.info, Node: What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean?, Prev: Is there a scanf or sscanf equivalent?, Up: Numbers and strings + +11.2.3.9 What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean? +........................................................................... + +See the *note Unicode HOWTO: c3c. + + +File: python.info, Node: Performance<4>, Next: Sequences Tuples/Lists, Prev: Numbers and strings, Up: Programming FAQ + +11.2.4 Performance +------------------ + +* Menu: + +* My program is too slow. How do I speed it up?: My program is too slow How do I speed it up?. +* What is the most efficient way to concatenate many strings together?:: + + +File: python.info, Node: My program is too slow How do I speed it up?, Next: What is the most efficient way to concatenate many strings together?, Up: Performance<4> + +11.2.4.1 My program is too slow. How do I speed it up? +...................................................... + +That’s a tough one, in general. First, here are a list of things to +remember before diving further: + + * Performance characteristics vary across Python implementations. + This FAQ focuses on *note CPython: 10d7. + + * Behaviour can vary across operating systems, especially when + talking about I/O or multi-threading. + + * You should always find the hot spots in your program `before' + attempting to optimize any code (see the *note profile: d3. + module). + + * Writing benchmark scripts will allow you to iterate quickly when + searching for improvements (see the *note timeit: 10b. module). + + * It is highly recommended to have good code coverage (through unit + testing or any other technique) before potentially introducing + regressions hidden in sophisticated optimizations. + +That being said, there are many tricks to speed up Python code. Here +are some general principles which go a long way towards reaching +acceptable performance levels: + + * Making your algorithms faster (or changing to faster ones) can + yield much larger benefits than trying to sprinkle + micro-optimization tricks all over your code. + + * Use the right data structures. Study documentation for the *note + Built-in Types: 12be. and the *note collections: 1e. module. + + * When the standard library provides a primitive for doing something, + it is likely (although not guaranteed) to be faster than any + alternative you may come up with. This is doubly true for + primitives written in C, such as builtins and some extension types. + For example, be sure to use either the *note list.sort(): 445. + built-in method or the related *note sorted(): 444. function to do + sorting (and see the *note Sorting HOW TO: 12ab. for examples of + moderately advanced usage). + + * Abstractions tend to create indirections and force the interpreter + to work more. If the levels of indirection outweigh the amount of + useful work done, your program will be slower. You should avoid + excessive abstraction, especially under the form of tiny functions + or methods (which are also often detrimental to readability). + +If you have reached the limit of what pure Python can allow, there are +tools to take you further away. For example, Cython(1) can compile a +slightly modified version of Python code into a C extension, and can be +used on many different platforms. Cython can take advantage of +compilation (and optional type annotations) to make your code +significantly faster than when interpreted. If you are confident in +your C programming skills, you can also *note write a C extension +module: e90. yourself. + +See also +........ + +The wiki page devoted to performance tips(2). + + ---------- Footnotes ---------- + + (1) http://cython.org + + (2) https://wiki.python.org/moin/PythonSpeed/PerformanceTips + + +File: python.info, Node: What is the most efficient way to concatenate many strings together?, Prev: My program is too slow How do I speed it up?, Up: Performance<4> + +11.2.4.2 What is the most efficient way to concatenate many strings together? +............................................................................. + +*note str: 330. and *note bytes: 331. objects are immutable, therefore +concatenating many strings together is inefficient as each concatenation +creates a new object. In the general case, the total runtime cost is +quadratic in the total string length. + +To accumulate many *note str: 330. objects, the recommended idiom is to +place them into a list and call *note str.join(): 12dd. at the end: + + chunks = [] + for s in my_strings: + chunks.append(s) + result = ''.join(chunks) + +(another reasonably efficient idiom is to use *note io.StringIO: 82d.) + +To accumulate many *note bytes: 331. objects, the recommended idiom is +to extend a *note bytearray: 332. object using in-place concatenation +(the ‘+=’ operator): + + result = bytearray() + for b in my_bytes_objects: + result += b + + +File: python.info, Node: Sequences Tuples/Lists, Next: Objects, Prev: Performance<4>, Up: Programming FAQ + +11.2.5 Sequences (Tuples/Lists) +------------------------------- + +* Menu: + +* How do I convert between tuples and lists?:: +* What’s a negative index?:: +* How do I iterate over a sequence in reverse order?:: +* How do you remove duplicates from a list?:: +* How do you remove multiple items from a list:: +* How do you make an array in Python?:: +* How do I create a multidimensional list?:: +* How do I apply a method to a sequence of objects?:: +* Why does a_tuple[i] += [‘item’] raise an exception when the addition works?:: +* I want to do a complicated sort; can you do a Schwartzian Transform in Python?: I want to do a complicated sort can you do a Schwartzian Transform in Python?. +* How can I sort one list by values from another list?:: + + +File: python.info, Node: How do I convert between tuples and lists?, Next: What’s a negative index?, Up: Sequences Tuples/Lists + +11.2.5.1 How do I convert between tuples and lists? +................................................... + +The type constructor ‘tuple(seq)’ converts any sequence (actually, any +iterable) into a tuple with the same items in the same order. + +For example, ‘tuple([1, 2, 3])’ yields ‘(1, 2, 3)’ and ‘tuple('abc')’ +yields ‘('a', 'b', 'c')’. If the argument is a tuple, it does not make +a copy but returns the same object, so it is cheap to call *note +tuple(): 47e. when you aren’t sure that an object is already a tuple. + +The type constructor ‘list(seq)’ converts any sequence or iterable into +a list with the same items in the same order. For example, ‘list((1, 2, +3))’ yields ‘[1, 2, 3]’ and ‘list('abc')’ yields ‘['a', 'b', 'c']’. If +the argument is a list, it makes a copy just like ‘seq[:]’ would. + + +File: python.info, Node: What’s a negative index?, Next: How do I iterate over a sequence in reverse order?, Prev: How do I convert between tuples and lists?, Up: Sequences Tuples/Lists + +11.2.5.2 What’s a negative index? +................................. + +Python sequences are indexed with positive numbers and negative numbers. +For positive numbers 0 is the first index 1 is the second index and so +forth. For negative indices -1 is the last index and -2 is the +penultimate (next to last) index and so forth. Think of ‘seq[-n]’ as +the same as ‘seq[len(seq)-n]’. + +Using negative indices can be very convenient. For example ‘S[:-1]’ is +all of the string except for its last character, which is useful for +removing the trailing newline from a string. + + +File: python.info, Node: How do I iterate over a sequence in reverse order?, Next: How do you remove duplicates from a list?, Prev: What’s a negative index?, Up: Sequences Tuples/Lists + +11.2.5.3 How do I iterate over a sequence in reverse order? +........................................................... + +Use the *note reversed(): 18e. built-in function: + + for x in reversed(sequence): + ... # do something with x ... + +This won’t touch your original sequence, but build a new copy with +reversed order to iterate over. + + +File: python.info, Node: How do you remove duplicates from a list?, Next: How do you remove multiple items from a list, Prev: How do I iterate over a sequence in reverse order?, Up: Sequences Tuples/Lists + +11.2.5.4 How do you remove duplicates from a list? +.................................................. + +See the Python Cookbook for a long discussion of many ways to do this: + + ‘https://code.activestate.com/recipes/52560/’ + +If you don’t mind reordering the list, sort it and then scan from the +end of the list, deleting duplicates as you go: + + if mylist: + mylist.sort() + last = mylist[-1] + for i in range(len(mylist)-2, -1, -1): + if last == mylist[i]: + del mylist[i] + else: + last = mylist[i] + +If all elements of the list may be used as set keys (i.e. they are all +*note hashable: 10c8.) this is often faster + + mylist = list(set(mylist)) + +This converts the list into a set, thereby removing duplicates, and then +back into a list. + + +File: python.info, Node: How do you remove multiple items from a list, Next: How do you make an array in Python?, Prev: How do you remove duplicates from a list?, Up: Sequences Tuples/Lists + +11.2.5.5 How do you remove multiple items from a list +..................................................... + +As with removing duplicates, explicitly iterating in reverse with a +delete condition is one possibility. However, it is easier and faster +to use slice replacement with an implicit or explicit forward iteration. +Here are three variations.: + + mylist[:] = filter(keep_function, mylist) + mylist[:] = (x for x in mylist if keep_condition) + mylist[:] = [x for x in mylist if keep_condition] + +The list comprehension may be fastest. + + +File: python.info, Node: How do you make an array in Python?, Next: How do I create a multidimensional list?, Prev: How do you remove multiple items from a list, Up: Sequences Tuples/Lists + +11.2.5.6 How do you make an array in Python? +............................................ + +Use a list: + + ["this", 1, "is", "an", "array"] + +Lists are equivalent to C or Pascal arrays in their time complexity; the +primary difference is that a Python list can contain objects of many +different types. + +The ‘array’ module also provides methods for creating arrays of fixed +types with compact representations, but they are slower to index than +lists. Also note that the Numeric extensions and others define +array-like structures with various characteristics as well. + +To get Lisp-style linked lists, you can emulate cons cells using tuples: + + lisp_list = ("like", ("this", ("example", None) ) ) + +If mutability is desired, you could use lists instead of tuples. Here +the analogue of lisp car is ‘lisp_list[0]’ and the analogue of cdr is +‘lisp_list[1]’. Only do this if you’re sure you really need to, because +it’s usually a lot slower than using Python lists. + + +File: python.info, Node: How do I create a multidimensional list?, Next: How do I apply a method to a sequence of objects?, Prev: How do you make an array in Python?, Up: Sequences Tuples/Lists + +11.2.5.7 How do I create a multidimensional list? +................................................. + +You probably tried to make a multidimensional array like this: + + >>> A = [[None] * 2] * 3 + +This looks correct if you print it: + + >>> A + [[None, None], [None, None], [None, None]] + +But when you assign a value, it shows up in multiple places: + + >>> A[0][0] = 5 + >>> A + [[5, None], [5, None], [5, None]] + +The reason is that replicating a list with ‘*’ doesn’t create copies, it +only creates references to the existing objects. The ‘*3’ creates a +list containing 3 references to the same list of length two. Changes to +one row will show in all rows, which is almost certainly not what you +want. + +The suggested approach is to create a list of the desired length first +and then fill in each element with a newly created list: + + A = [None] * 3 + for i in range(3): + A[i] = [None] * 2 + +This generates a list containing 3 different lists of length two. You +can also use a list comprehension: + + w, h = 2, 3 + A = [[None] * w for i in range(h)] + +Or, you can use an extension that provides a matrix datatype; NumPy(1) +is the best known. + + ---------- Footnotes ---------- + + (1) http://www.numpy.org/ + + +File: python.info, Node: How do I apply a method to a sequence of objects?, Next: Why does a_tuple[i] += [‘item’] raise an exception when the addition works?, Prev: How do I create a multidimensional list?, Up: Sequences Tuples/Lists + +11.2.5.8 How do I apply a method to a sequence of objects? +.......................................................... + +Use a list comprehension: + + result = [obj.method() for obj in mylist] + + +File: python.info, Node: Why does a_tuple[i] += [‘item’] raise an exception when the addition works?, Next: I want to do a complicated sort can you do a Schwartzian Transform in Python?, Prev: How do I apply a method to a sequence of objects?, Up: Sequences Tuples/Lists + +11.2.5.9 Why does a_tuple[i] += [‘item’] raise an exception when the addition works? +.................................................................................... + +This is because of a combination of the fact that augmented assignment +operators are `assignment' operators, and the difference between mutable +and immutable objects in Python. + +This discussion applies in general when augmented assignment operators +are applied to elements of a tuple that point to mutable objects, but +we’ll use a ‘list’ and ‘+=’ as our exemplar. + +If you wrote: + + >>> a_tuple = (1, 2) + >>> a_tuple[0] += 1 + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +The reason for the exception should be immediately clear: ‘1’ is added +to the object ‘a_tuple[0]’ points to (‘1’), producing the result object, +‘2’, but when we attempt to assign the result of the computation, ‘2’, +to element ‘0’ of the tuple, we get an error because we can’t change +what an element of a tuple points to. + +Under the covers, what this augmented assignment statement is doing is +approximately this: + + >>> result = a_tuple[0] + 1 + >>> a_tuple[0] = result + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +It is the assignment part of the operation that produces the error, +since a tuple is immutable. + +When you write something like: + + >>> a_tuple = (['foo'], 'bar') + >>> a_tuple[0] += ['item'] + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +The exception is a bit more surprising, and even more surprising is the +fact that even though there was an error, the append worked: + + >>> a_tuple[0] + ['foo', 'item'] + +To see why this happens, you need to know that (a) if an object +implements an ‘__iadd__’ magic method, it gets called when the ‘+=’ +augmented assignment is executed, and its return value is what gets used +in the assignment statement; and (b) for lists, ‘__iadd__’ is equivalent +to calling ‘extend’ on the list and returning the list. That’s why we +say that for lists, ‘+=’ is a “shorthand” for ‘list.extend’: + + >>> a_list = [] + >>> a_list += [1] + >>> a_list + [1] + +This is equivalent to: + + >>> result = a_list.__iadd__([1]) + >>> a_list = result + +The object pointed to by a_list has been mutated, and the pointer to the +mutated object is assigned back to ‘a_list’. The end result of the +assignment is a no-op, since it is a pointer to the same object that +‘a_list’ was previously pointing to, but the assignment still happens. + +Thus, in our tuple example what is happening is equivalent to: + + >>> result = a_tuple[0].__iadd__(['item']) + >>> a_tuple[0] = result + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +The ‘__iadd__’ succeeds, and thus the list is extended, but even though +‘result’ points to the same object that ‘a_tuple[0]’ already points to, +that final assignment still results in an error, because tuples are +immutable. + + +File: python.info, Node: I want to do a complicated sort can you do a Schwartzian Transform in Python?, Next: How can I sort one list by values from another list?, Prev: Why does a_tuple[i] += [‘item’] raise an exception when the addition works?, Up: Sequences Tuples/Lists + +11.2.5.10 I want to do a complicated sort: can you do a Schwartzian Transform in Python? +........................................................................................ + +The technique, attributed to Randal Schwartz of the Perl community, +sorts the elements of a list by a metric which maps each element to its +“sort value”. In Python, use the ‘key’ argument for the *note +list.sort(): 445. method: + + Isorted = L[:] + Isorted.sort(key=lambda s: int(s[10:15])) + + +File: python.info, Node: How can I sort one list by values from another list?, Prev: I want to do a complicated sort can you do a Schwartzian Transform in Python?, Up: Sequences Tuples/Lists + +11.2.5.11 How can I sort one list by values from another list? +.............................................................. + +Merge them into an iterator of tuples, sort the resulting list, and then +pick out the element you want. + + >>> list1 = ["what", "I'm", "sorting", "by"] + >>> list2 = ["something", "else", "to", "sort"] + >>> pairs = zip(list1, list2) + >>> pairs = sorted(pairs) + >>> pairs + [("I'm", 'else'), ('by', 'sort'), ('sorting', 'to'), ('what', 'something')] + >>> result = [x[1] for x in pairs] + >>> result + ['else', 'sort', 'to', 'something'] + +An alternative for the last step is: + + >>> result = [] + >>> for p in pairs: result.append(p[1]) + +If you find this more legible, you might prefer to use this instead of +the final list comprehension. However, it is almost twice as slow for +long lists. Why? First, the ‘append()’ operation has to reallocate +memory, and while it uses some tricks to avoid doing that each time, it +still has to do it occasionally, and that costs quite a bit. Second, +the expression “result.append” requires an extra attribute lookup, and +third, there’s a speed reduction from having to make all those function +calls. + + +File: python.info, Node: Objects, Next: Modules<3>, Prev: Sequences Tuples/Lists, Up: Programming FAQ + +11.2.6 Objects +-------------- + +* Menu: + +* What is a class?:: +* What is a method?:: +* What is self?:: +* How do I check if an object is an instance of a given class or of a subclass of it?:: +* What is delegation?:: +* How do I call a method defined in a base class from a derived class that overrides it?:: +* How can I organize my code to make it easier to change the base class?:: +* How do I create static class data and static class methods?:: +* How can I overload constructors (or methods) in Python?: How can I overload constructors or methods in Python?. +* I try to use __spam and I get an error about _SomeClassName__spam.: I try to use __spam and I get an error about _SomeClassName__spam. +* My class defines __del__ but it is not called when I delete the object.: My class defines __del__ but it is not called when I delete the object. +* How do I get a list of all instances of a given class?:: +* Why does the result of id() appear to be not unique?: Why does the result of id appear to be not unique?. + + +File: python.info, Node: What is a class?, Next: What is a method?, Up: Objects + +11.2.6.1 What is a class? +......................... + +A class is the particular object type created by executing a class +statement. Class objects are used as templates to create instance +objects, which embody both the data (attributes) and code (methods) +specific to a datatype. + +A class can be based on one or more other classes, called its base +class(es). It then inherits the attributes and methods of its base +classes. This allows an object model to be successively refined by +inheritance. You might have a generic ‘Mailbox’ class that provides +basic accessor methods for a mailbox, and subclasses such as +‘MboxMailbox’, ‘MaildirMailbox’, ‘OutlookMailbox’ that handle various +specific mailbox formats. + + +File: python.info, Node: What is a method?, Next: What is self?, Prev: What is a class?, Up: Objects + +11.2.6.2 What is a method? +.......................... + +A method is a function on some object ‘x’ that you normally call as +‘x.name(arguments...)’. Methods are defined as functions inside the +class definition: + + class C: + def meth(self, arg): + return arg * 2 + self.attribute + + +File: python.info, Node: What is self?, Next: How do I check if an object is an instance of a given class or of a subclass of it?, Prev: What is a method?, Up: Objects + +11.2.6.3 What is self? +...................... + +Self is merely a conventional name for the first argument of a method. +A method defined as ‘meth(self, a, b, c)’ should be called as ‘x.meth(a, +b, c)’ for some instance ‘x’ of the class in which the definition +occurs; the called method will think it is called as ‘meth(x, a, b, c)’. + +See also *note Why must ‘self’ be used explicitly in method definitions +and calls?: 3fde. + + +File: python.info, Node: How do I check if an object is an instance of a given class or of a subclass of it?, Next: What is delegation?, Prev: What is self?, Up: Objects + +11.2.6.4 How do I check if an object is an instance of a given class or of a subclass of it? +............................................................................................ + +Use the built-in function ‘isinstance(obj, cls)’. You can check if an +object is an instance of any of a number of classes by providing a tuple +instead of a single class, e.g. ‘isinstance(obj, (class1, class2, +...))’, and can also check whether an object is one of Python’s built-in +types, e.g. ‘isinstance(obj, str)’ or ‘isinstance(obj, (int, float, +complex))’. + +Note that most programs do not use *note isinstance(): 44f. on +user-defined classes very often. If you are developing the classes +yourself, a more proper object-oriented style is to define methods on +the classes that encapsulate a particular behaviour, instead of checking +the object’s class and doing a different thing based on what class it +is. For example, if you have a function that does something: + + def search(obj): + if isinstance(obj, Mailbox): + ... # code to search a mailbox + elif isinstance(obj, Document): + ... # code to search a document + elif ... + +A better approach is to define a ‘search()’ method on all the classes +and just call it: + + class Mailbox: + def search(self): + ... # code to search a mailbox + + class Document: + def search(self): + ... # code to search a document + + obj.search() + + +File: python.info, Node: What is delegation?, Next: How do I call a method defined in a base class from a derived class that overrides it?, Prev: How do I check if an object is an instance of a given class or of a subclass of it?, Up: Objects + +11.2.6.5 What is delegation? +............................ + +Delegation is an object oriented technique (also called a design +pattern). Let’s say you have an object ‘x’ and want to change the +behaviour of just one of its methods. You can create a new class that +provides a new implementation of the method you’re interested in +changing and delegates all other methods to the corresponding method of +‘x’. + +Python programmers can easily implement delegation. For example, the +following class implements a class that behaves like a file but converts +all written data to uppercase: + + class UpperOut: + + def __init__(self, outfile): + self._outfile = outfile + + def write(self, s): + self._outfile.write(s.upper()) + + def __getattr__(self, name): + return getattr(self._outfile, name) + +Here the ‘UpperOut’ class redefines the ‘write()’ method to convert the +argument string to uppercase before calling the underlying +‘self._outfile.write()’ method. All other methods are delegated to the +underlying ‘self._outfile’ object. The delegation is accomplished via +the ‘__getattr__’ method; consult *note the language reference: 10db. +for more information about controlling attribute access. + +Note that for more general cases delegation can get trickier. When +attributes must be set as well as retrieved, the class must define a +*note __setattr__(): e2c. method too, and it must do so carefully. The +basic implementation of *note __setattr__(): e2c. is roughly equivalent +to the following: + + class X: + ... + def __setattr__(self, name, value): + self.__dict__[name] = value + ... + +Most *note __setattr__(): e2c. implementations must modify +‘self.__dict__’ to store local state for self without causing an +infinite recursion. + + +File: python.info, Node: How do I call a method defined in a base class from a derived class that overrides it?, Next: How can I organize my code to make it easier to change the base class?, Prev: What is delegation?, Up: Objects + +11.2.6.6 How do I call a method defined in a base class from a derived class that overrides it? +............................................................................................... + +Use the built-in *note super(): 4e2. function: + + class Derived(Base): + def meth(self): + super(Derived, self).meth() + +For version prior to 3.0, you may be using classic classes: For a class +definition such as ‘class Derived(Base): ...’ you can call method +‘meth()’ defined in ‘Base’ (or one of ‘Base’’s base classes) as +‘Base.meth(self, arguments...)’. Here, ‘Base.meth’ is an unbound +method, so you need to provide the ‘self’ argument. + + +File: python.info, Node: How can I organize my code to make it easier to change the base class?, Next: How do I create static class data and static class methods?, Prev: How do I call a method defined in a base class from a derived class that overrides it?, Up: Objects + +11.2.6.7 How can I organize my code to make it easier to change the base class? +............................................................................... + +You could define an alias for the base class, assign the real base class +to it before your class definition, and use the alias throughout your +class. Then all you have to change is the value assigned to the alias. +Incidentally, this trick is also handy if you want to decide dynamically +(e.g. depending on availability of resources) which base class to use. +Example: + + BaseAlias = + + class Derived(BaseAlias): + def meth(self): + BaseAlias.meth(self) + ... + + +File: python.info, Node: How do I create static class data and static class methods?, Next: How can I overload constructors or methods in Python?, Prev: How can I organize my code to make it easier to change the base class?, Up: Objects + +11.2.6.8 How do I create static class data and static class methods? +.................................................................... + +Both static data and static methods (in the sense of C++ or Java) are +supported in Python. + +For static data, simply define a class attribute. To assign a new value +to the attribute, you have to explicitly use the class name in the +assignment: + + class C: + count = 0 # number of times C.__init__ called + + def __init__(self): + C.count = C.count + 1 + + def getcount(self): + return C.count # or return self.count + +‘c.count’ also refers to ‘C.count’ for any ‘c’ such that ‘isinstance(c, +C)’ holds, unless overridden by ‘c’ itself or by some class on the +base-class search path from ‘c.__class__’ back to ‘C’. + +Caution: within a method of C, an assignment like ‘self.count = 42’ +creates a new and unrelated instance named “count” in ‘self’’s own dict. +Rebinding of a class-static data name must always specify the class +whether inside a method or not: + + C.count = 314 + +Static methods are possible: + + class C: + @staticmethod + def static(arg1, arg2, arg3): + # No 'self' parameter! + ... + +However, a far more straightforward way to get the effect of a static +method is via a simple module-level function: + + def getcount(): + return C.count + +If your code is structured so as to define one class (or tightly related +class hierarchy) per module, this supplies the desired encapsulation. + + +File: python.info, Node: How can I overload constructors or methods in Python?, Next: I try to use __spam and I get an error about _SomeClassName__spam, Prev: How do I create static class data and static class methods?, Up: Objects + +11.2.6.9 How can I overload constructors (or methods) in Python? +................................................................ + +This answer actually applies to all methods, but the question usually +comes up first in the context of constructors. + +In C++ you’d write + + class C { + C() { cout << "No arguments\n"; } + C(int i) { cout << "Argument is " << i << "\n"; } + } + +In Python you have to write a single constructor that catches all cases +using default arguments. For example: + + class C: + def __init__(self, i=None): + if i is None: + print("No arguments") + else: + print("Argument is", i) + +This is not entirely equivalent, but close enough in practice. + +You could also try a variable-length argument list, e.g. + + def __init__(self, *args): + ... + +The same approach works for all method definitions. + + +File: python.info, Node: I try to use __spam and I get an error about _SomeClassName__spam, Next: My class defines __del__ but it is not called when I delete the object, Prev: How can I overload constructors or methods in Python?, Up: Objects + +11.2.6.10 I try to use __spam and I get an error about _SomeClassName__spam. +............................................................................ + +Variable names with double leading underscores are “mangled” to provide +a simple but effective way to define class private variables. Any +identifier of the form ‘__spam’ (at least two leading underscores, at +most one trailing underscore) is textually replaced with +‘_classname__spam’, where ‘classname’ is the current class name with any +leading underscores stripped. + +This doesn’t guarantee privacy: an outside user can still deliberately +access the “_classname__spam” attribute, and private values are visible +in the object’s ‘__dict__’. Many Python programmers never bother to use +private variable names at all. + + +File: python.info, Node: My class defines __del__ but it is not called when I delete the object, Next: How do I get a list of all instances of a given class?, Prev: I try to use __spam and I get an error about _SomeClassName__spam, Up: Objects + +11.2.6.11 My class defines __del__ but it is not called when I delete the object. +................................................................................. + +There are several possible reasons for this. + +The del statement does not necessarily call *note __del__(): 91f. – it +simply decrements the object’s reference count, and if this reaches zero +*note __del__(): 91f. is called. + +If your data structures contain circular links (e.g. a tree where each +child has a parent reference and each parent has a list of children) the +reference counts will never go back to zero. Once in a while Python +runs an algorithm to detect such cycles, but the garbage collector might +run some time after the last reference to your data structure vanishes, +so your *note __del__(): 91f. method may be called at an inconvenient +and random time. This is inconvenient if you’re trying to reproduce a +problem. Worse, the order in which object’s *note __del__(): 91f. +methods are executed is arbitrary. You can run *note gc.collect(): 22e. +to force a collection, but there `are' pathological cases where objects +will never be collected. + +Despite the cycle collector, it’s still a good idea to define an +explicit ‘close()’ method on objects to be called whenever you’re done +with them. The ‘close()’ method can then remove attributes that refer +to subobjects. Don’t call *note __del__(): 91f. directly – *note +__del__(): 91f. should call ‘close()’ and ‘close()’ should make sure +that it can be called more than once for the same object. + +Another way to avoid cyclical references is to use the *note weakref: +128. module, which allows you to point to objects without incrementing +their reference count. Tree data structures, for instance, should use +weak references for their parent and sibling references (if they need +them!). + +Finally, if your *note __del__(): 91f. method raises an exception, a +warning message is printed to *note sys.stderr: 30f. + + +File: python.info, Node: How do I get a list of all instances of a given class?, Next: Why does the result of id appear to be not unique?, Prev: My class defines __del__ but it is not called when I delete the object, Up: Objects + +11.2.6.12 How do I get a list of all instances of a given class? +................................................................ + +Python does not keep track of all instances of a class (or of a built-in +type). You can program the class’s constructor to keep track of all +instances by keeping a list of weak references to each instance. + + +File: python.info, Node: Why does the result of id appear to be not unique?, Prev: How do I get a list of all instances of a given class?, Up: Objects + +11.2.6.13 Why does the result of ‘id()’ appear to be not unique? +................................................................ + +The *note id(): d86. builtin returns an integer that is guaranteed to be +unique during the lifetime of the object. Since in CPython, this is the +object’s memory address, it happens frequently that after an object is +deleted from memory, the next freshly created object is allocated at the +same position in memory. This is illustrated by this example: + + >>> id(1000) + 13901272 + >>> id(2000) + 13901272 + +The two ids belong to different integer objects that are created before, +and deleted immediately after execution of the ‘id()’ call. To be sure +that objects whose id you want to examine are still alive, create +another reference to the object: + + >>> a = 1000; b = 2000 + >>> id(a) + 13901272 + >>> id(b) + 13891296 + + +File: python.info, Node: Modules<3>, Prev: Objects, Up: Programming FAQ + +11.2.7 Modules +-------------- + +* Menu: + +* How do I create a .pyc file?: How do I create a pyc file?. +* How do I find the current module name?:: +* How can I have modules that mutually import each other?:: +* __import__(‘x.y.z’) returns ; how do I get z?: __import__ ‘x y z’ returns ; how do I get z?. +* When I edit an imported module and reimport it, the changes don’t show up. Why does this happen?: When I edit an imported module and reimport it the changes don’t show up Why does this happen?. + + +File: python.info, Node: How do I create a pyc file?, Next: How do I find the current module name?, Up: Modules<3> + +11.2.7.1 How do I create a .pyc file? +..................................... + +When a module is imported for the first time (or when the source file +has changed since the current compiled file was created) a ‘.pyc’ file +containing the compiled code should be created in a ‘__pycache__’ +subdirectory of the directory containing the ‘.py’ file. The ‘.pyc’ +file will have a filename that starts with the same name as the ‘.py’ +file, and ends with ‘.pyc’, with a middle component that depends on the +particular ‘python’ binary that created it. (See PEP 3147(1) for +details.) + +One reason that a ‘.pyc’ file may not be created is a permissions +problem with the directory containing the source file, meaning that the +‘__pycache__’ subdirectory cannot be created. This can happen, for +example, if you develop as one user but run as another, such as if you +are testing with a web server. + +Unless the *note PYTHONDONTWRITEBYTECODE: d39. environment variable is +set, creation of a .pyc file is automatic if you’re importing a module +and Python has the ability (permissions, free space, etc…) to create a +‘__pycache__’ subdirectory and write the compiled module to that +subdirectory. + +Running Python on a top level script is not considered an import and no +‘.pyc’ will be created. For example, if you have a top-level module +‘foo.py’ that imports another module ‘xyz.py’, when you run ‘foo’ (by +typing ‘python foo.py’ as a shell command), a ‘.pyc’ will be created for +‘xyz’ because ‘xyz’ is imported, but no ‘.pyc’ file will be created for +‘foo’ since ‘foo.py’ isn’t being imported. + +If you need to create a ‘.pyc’ file for ‘foo’ – that is, to create a +‘.pyc’ file for a module that is not imported – you can, using the *note +py_compile: d7. and *note compileall: 21. modules. + +The *note py_compile: d7. module can manually compile any module. One +way is to use the ‘compile()’ function in that module interactively: + + >>> import py_compile + >>> py_compile.compile('foo.py') + +This will write the ‘.pyc’ to a ‘__pycache__’ subdirectory in the same +location as ‘foo.py’ (or you can override that with the optional +parameter ‘cfile’). + +You can also automatically compile all files in a directory or +directories using the *note compileall: 21. module. You can do it from +the shell prompt by running ‘compileall.py’ and providing the path of a +directory containing Python files to compile: + + python -m compileall . + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + +File: python.info, Node: How do I find the current module name?, Next: How can I have modules that mutually import each other?, Prev: How do I create a pyc file?, Up: Modules<3> + +11.2.7.2 How do I find the current module name? +............................................... + +A module can find out its own module name by looking at the predefined +global variable ‘__name__’. If this has the value ‘'__main__'’, the +program is running as a script. Many modules that are usually used by +importing them also provide a command-line interface or a self-test, and +only execute this code after checking ‘__name__’: + + def main(): + print('Running test...') + ... + + if __name__ == '__main__': + main() + + +File: python.info, Node: How can I have modules that mutually import each other?, Next: __import__ ‘x y z’ returns ; how do I get z?, Prev: How do I find the current module name?, Up: Modules<3> + +11.2.7.3 How can I have modules that mutually import each other? +................................................................ + +Suppose you have the following modules: + +foo.py: + + from bar import bar_var + foo_var = 1 + +bar.py: + + from foo import foo_var + bar_var = 2 + +The problem is that the interpreter will perform the following steps: + + * main imports foo + + * Empty globals for foo are created + + * foo is compiled and starts executing + + * foo imports bar + + * Empty globals for bar are created + + * bar is compiled and starts executing + + * bar imports foo (which is a no-op since there already is a module + named foo) + + * bar.foo_var = foo.foo_var + +The last step fails, because Python isn’t done with interpreting ‘foo’ +yet and the global symbol dictionary for ‘foo’ is still empty. + +The same thing happens when you use ‘import foo’, and then try to access +‘foo.foo_var’ in global code. + +There are (at least) three possible workarounds for this problem. + +Guido van Rossum recommends avoiding all uses of ‘from import +...’, and placing all code inside functions. Initializations of global +variables and class variables should use constants or built-in functions +only. This means everything from an imported module is referenced as +‘.’. + +Jim Roskind suggests performing steps in the following order in each +module: + + * exports (globals, functions, and classes that don’t need imported + base classes) + + * ‘import’ statements + + * active code (including globals that are initialized from imported + values). + +van Rossum doesn’t like this approach much because the imports appear in +a strange place, but it does work. + +Matthias Urlichs recommends restructuring your code so that the +recursive import is not necessary in the first place. + +These solutions are not mutually exclusive. + + +File: python.info, Node: __import__ ‘x y z’ returns ; how do I get z?, Next: When I edit an imported module and reimport it the changes don’t show up Why does this happen?, Prev: How can I have modules that mutually import each other?, Up: Modules<3> + +11.2.7.4 __import__(‘x.y.z’) returns ; how do I get z? +.................................................................. + +Consider using the convenience function *note import_module(): b13. from +*note importlib: 9b. instead: + + z = importlib.import_module('x.y.z') + + +File: python.info, Node: When I edit an imported module and reimport it the changes don’t show up Why does this happen?, Prev: __import__ ‘x y z’ returns ; how do I get z?, Up: Modules<3> + +11.2.7.5 When I edit an imported module and reimport it, the changes don’t show up. Why does this happen? +......................................................................................................... + +For reasons of efficiency as well as consistency, Python only reads the +module file on the first time a module is imported. If it didn’t, in a +program consisting of many modules where each one imports the same basic +module, the basic module would be parsed and re-parsed many times. To +force re-reading of a changed module, do this: + + import importlib + import modname + importlib.reload(modname) + +Warning: this technique is not 100% fool-proof. In particular, modules +containing statements like + + from modname import some_objects + +will continue to work with the old version of the imported objects. If +the module contains class definitions, existing class instances will +`not' be updated to use the new class definition. This can result in +the following paradoxical behaviour: + + >>> import importlib + >>> import cls + >>> c = cls.C() # Create an instance of C + >>> importlib.reload(cls) + + >>> isinstance(c, cls.C) # isinstance is false?!? + False + +The nature of the problem is made clear if you print out the “identity” +of the class objects: + + >>> hex(id(c.__class__)) + '0x7352a0' + >>> hex(id(cls.C)) + '0x4198d0' + + +File: python.info, Node: Design and History FAQ, Next: Library and Extension FAQ, Prev: Programming FAQ, Up: Python Frequently Asked Questions + +11.3 Design and History FAQ +=========================== + +* Menu: + +* Why does Python use indentation for grouping of statements?:: +* Why am I getting strange results with simple arithmetic operations?:: +* Why are floating-point calculations so inaccurate?:: +* Why are Python strings immutable?:: +* Why must ‘self’ be used explicitly in method definitions and calls?:: +* Why can’t I use an assignment in an expression?:: +* Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))?: Why does Python use methods for some functionality e g list index but functions for other e g len list ?. +* Why is join() a string method instead of a list or tuple method?: Why is join a string method instead of a list or tuple method?. +* How fast are exceptions?:: +* Why isn’t there a switch or case statement in Python?:: +* Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?:: +* Why can’t lambda expressions contain statements?:: +* Can Python be compiled to machine code, C or some other language?: Can Python be compiled to machine code C or some other language?. +* How does Python manage memory?:: +* Why doesn’t CPython use a more traditional garbage collection scheme?:: +* Why isn’t all memory freed when CPython exits?:: +* Why are there separate tuple and list data types?:: +* How are lists implemented in CPython?:: +* How are dictionaries implemented in CPython?:: +* Why must dictionary keys be immutable?:: +* Why doesn’t list.sort() return the sorted list?: Why doesn’t list sort return the sorted list?. +* How do you specify and enforce an interface spec in Python?:: +* Why is there no goto?:: +* Why can’t raw strings (r-strings) end with a backslash?: Why can’t raw strings r-strings end with a backslash?. +* Why doesn’t Python have a “with” statement for attribute assignments?:: +* Why are colons required for the if/while/def/class statements?:: +* Why does Python allow commas at the end of lists and tuples?:: + + +File: python.info, Node: Why does Python use indentation for grouping of statements?, Next: Why am I getting strange results with simple arithmetic operations?, Up: Design and History FAQ + +11.3.1 Why does Python use indentation for grouping of statements? +------------------------------------------------------------------ + +Guido van Rossum believes that using indentation for grouping is +extremely elegant and contributes a lot to the clarity of the average +Python program. Most people learn to love this feature after a while. + +Since there are no begin/end brackets there cannot be a disagreement +between grouping perceived by the parser and the human reader. +Occasionally C programmers will encounter a fragment of code like this: + + if (x <= y) + x++; + y--; + z++; + +Only the ‘x++’ statement is executed if the condition is true, but the +indentation leads you to believe otherwise. Even experienced C +programmers will sometimes stare at it a long time wondering why ‘y’ is +being decremented even for ‘x > y’. + +Because there are no begin/end brackets, Python is much less prone to +coding-style conflicts. In C there are many different ways to place the +braces. If you’re used to reading and writing code that uses one style, +you will feel at least slightly uneasy when reading (or being required +to write) another style. + +Many coding styles place begin/end brackets on a line by themselves. +This makes programs considerably longer and wastes valuable screen +space, making it harder to get a good overview of a program. Ideally, a +function should fit on one screen (say, 20–30 lines). 20 lines of +Python can do a lot more work than 20 lines of C. This is not solely due +to the lack of begin/end brackets – the lack of declarations and the +high-level data types are also responsible – but the indentation-based +syntax certainly helps. + + +File: python.info, Node: Why am I getting strange results with simple arithmetic operations?, Next: Why are floating-point calculations so inaccurate?, Prev: Why does Python use indentation for grouping of statements?, Up: Design and History FAQ + +11.3.2 Why am I getting strange results with simple arithmetic operations? +-------------------------------------------------------------------------- + +See the next question. + + +File: python.info, Node: Why are floating-point calculations so inaccurate?, Next: Why are Python strings immutable?, Prev: Why am I getting strange results with simple arithmetic operations?, Up: Design and History FAQ + +11.3.3 Why are floating-point calculations so inaccurate? +--------------------------------------------------------- + +Users are often surprised by results like this: + + >>> 1.2 - 1.0 + 0.19999999999999996 + +and think it is a bug in Python. It’s not. This has little to do with +Python, and much more to do with how the underlying platform handles +floating-point numbers. + +The *note float: 187. type in CPython uses a C ‘double’ for storage. A +*note float: 187. object’s value is stored in binary floating-point with +a fixed precision (typically 53 bits) and Python uses C operations, +which in turn rely on the hardware implementation in the processor, to +perform floating-point operations. This means that as far as +floating-point operations are concerned, Python behaves like many +popular languages including C and Java. + +Many numbers that can be written easily in decimal notation cannot be +expressed exactly in binary floating-point. For example, after: + + >>> x = 1.2 + +the value stored for ‘x’ is a (very good) approximation to the decimal +value ‘1.2’, but is not exactly equal to it. On a typical machine, the +actual stored value is: + + 1.0011001100110011001100110011001100110011001100110011 (binary) + +which is exactly: + + 1.1999999999999999555910790149937383830547332763671875 (decimal) + +The typical precision of 53 bits provides Python floats with 15–16 +decimal digits of accuracy. + +For a fuller explanation, please see the *note floating point +arithmetic: fb8. chapter in the Python tutorial. + + +File: python.info, Node: Why are Python strings immutable?, Next: Why must ‘self’ be used explicitly in method definitions and calls?, Prev: Why are floating-point calculations so inaccurate?, Up: Design and History FAQ + +11.3.4 Why are Python strings immutable? +---------------------------------------- + +There are several advantages. + +One is performance: knowing that a string is immutable means we can +allocate space for it at creation time, and the storage requirements are +fixed and unchanging. This is also one of the reasons for the +distinction between tuples and lists. + +Another advantage is that strings in Python are considered as +“elemental” as numbers. No amount of activity will change the value 8 +to anything else, and in Python, no amount of activity will change the +string “eight” to anything else. + + +File: python.info, Node: Why must ‘self’ be used explicitly in method definitions and calls?, Next: Why can’t I use an assignment in an expression?, Prev: Why are Python strings immutable?, Up: Design and History FAQ + +11.3.5 Why must ‘self’ be used explicitly in method definitions and calls? +-------------------------------------------------------------------------- + +The idea was borrowed from Modula-3. It turns out to be very useful, +for a variety of reasons. + +First, it’s more obvious that you are using a method or instance +attribute instead of a local variable. Reading ‘self.x’ or +‘self.meth()’ makes it absolutely clear that an instance variable or +method is used even if you don’t know the class definition by heart. In +C++, you can sort of tell by the lack of a local variable declaration +(assuming globals are rare or easily recognizable) – but in Python, +there are no local variable declarations, so you’d have to look up the +class definition to be sure. Some C++ and Java coding standards call +for instance attributes to have an ‘m_’ prefix, so this explicitness is +still useful in those languages, too. + +Second, it means that no special syntax is necessary if you want to +explicitly reference or call the method from a particular class. In +C++, if you want to use a method from a base class which is overridden +in a derived class, you have to use the ‘::’ operator – in Python you +can write ‘baseclass.methodname(self, )’. This is +particularly useful for *note __init__(): d5e. methods, and in general +in cases where a derived class method wants to extend the base class +method of the same name and thus has to call the base class method +somehow. + +Finally, for instance variables it solves a syntactic problem with +assignment: since local variables in Python are (by definition!) those +variables to which a value is assigned in a function body (and that +aren’t explicitly declared global), there has to be some way to tell the +interpreter that an assignment was meant to assign to an instance +variable instead of to a local variable, and it should preferably be +syntactic (for efficiency reasons). C++ does this through declarations, +but Python doesn’t have declarations and it would be a pity having to +introduce them just for this purpose. Using the explicit ‘self.var’ +solves this nicely. Similarly, for using instance variables, having to +write ‘self.var’ means that references to unqualified names inside a +method don’t have to search the instance’s directories. To put it +another way, local variables and instance variables live in two +different namespaces, and you need to tell Python which namespace to +use. + + +File: python.info, Node: Why can’t I use an assignment in an expression?, Next: Why does Python use methods for some functionality e g list index but functions for other e g len list ?, Prev: Why must ‘self’ be used explicitly in method definitions and calls?, Up: Design and History FAQ + +11.3.6 Why can’t I use an assignment in an expression? +------------------------------------------------------ + +Starting in Python 3.8, you can! + +Assignment expressions using the walrus operator ‘:=’ assign a variable +in an expression: + + while chunk := fp.read(200): + print(chunk) + +See PEP 572(1) for more information. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0572 + + +File: python.info, Node: Why does Python use methods for some functionality e g list index but functions for other e g len list ?, Next: Why is join a string method instead of a list or tuple method?, Prev: Why can’t I use an assignment in an expression?, Up: Design and History FAQ + +11.3.7 Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? +----------------------------------------------------------------------------------------------------------------------- + +As Guido said: + + (a) For some operations, prefix notation just reads better than + postfix – prefix (and infix!) operations have a long tradition in + mathematics which likes notations where the visuals help the + mathematician thinking about a problem. Compare the easy with + which we rewrite a formula like x*(a+b) into x*a + x*b to the + clumsiness of doing the same thing using a raw OO notation. + + (b) When I read code that says len(x) I `know' that it is asking + for the length of something. This tells me two things: the result + is an integer, and the argument is some kind of container. To the + contrary, when I read x.len(), I have to already know that x is + some kind of container implementing an interface or inheriting from + a class that has a standard len(). Witness the confusion we + occasionally have when a class that is not implementing a mapping + has a get() or keys() method, or something that isn’t a file has a + write() method. + +— ‘https://mail.python.org/pipermail/python-3000/2006-November/004643.html’ + + +File: python.info, Node: Why is join a string method instead of a list or tuple method?, Next: How fast are exceptions?, Prev: Why does Python use methods for some functionality e g list index but functions for other e g len list ?, Up: Design and History FAQ + +11.3.8 Why is join() a string method instead of a list or tuple method? +----------------------------------------------------------------------- + +Strings became much more like other standard types starting in Python +1.6, when methods were added which give the same functionality that has +always been available using the functions of the string module. Most of +these new methods have been widely accepted, but the one which appears +to make some programmers feel uncomfortable is: + + ", ".join(['1', '2', '4', '8', '16']) + +which gives the result: + + "1, 2, 4, 8, 16" + +There are two common arguments against this usage. + +The first runs along the lines of: “It looks really ugly using a method +of a string literal (string constant)”, to which the answer is that it +might, but a string literal is just a fixed value. If the methods are +to be allowed on names bound to strings there is no logical reason to +make them unavailable on literals. + +The second objection is typically cast as: “I am really telling a +sequence to join its members together with a string constant”. Sadly, +you aren’t. For some reason there seems to be much less difficulty with +having *note split(): 7a1. as a string method, since in that case it is +easy to see that + + "1, 2, 4, 8, 16".split(", ") + +is an instruction to a string literal to return the substrings delimited +by the given separator (or, by default, arbitrary runs of white space). + +*note join(): 12dd. is a string method because in using it you are +telling the separator string to iterate over a sequence of strings and +insert itself between adjacent elements. This method can be used with +any argument which obeys the rules for sequence objects, including any +new classes you might define yourself. Similar methods exist for bytes +and bytearray objects. + + +File: python.info, Node: How fast are exceptions?, Next: Why isn’t there a switch or case statement in Python?, Prev: Why is join a string method instead of a list or tuple method?, Up: Design and History FAQ + +11.3.9 How fast are exceptions? +------------------------------- + +A try/except block is extremely efficient if no exceptions are raised. +Actually catching an exception is expensive. In versions of Python +prior to 2.0 it was common to use this idiom: + + try: + value = mydict[key] + except KeyError: + mydict[key] = getvalue(key) + value = mydict[key] + +This only made sense when you expected the dict to have the key almost +all the time. If that wasn’t the case, you coded it like this: + + if key in mydict: + value = mydict[key] + else: + value = mydict[key] = getvalue(key) + +For this specific case, you could also use ‘value = dict.setdefault(key, +getvalue(key))’, but only if the ‘getvalue()’ call is cheap enough +because it is evaluated in all cases. + + +File: python.info, Node: Why isn’t there a switch or case statement in Python?, Next: Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?, Prev: How fast are exceptions?, Up: Design and History FAQ + +11.3.10 Why isn’t there a switch or case statement in Python? +------------------------------------------------------------- + +You can do this easily enough with a sequence of ‘if... elif... elif... +else’. There have been some proposals for switch statement syntax, but +there is no consensus (yet) on whether and how to do range tests. See +PEP 275(1) for complete details and the current status. + +For cases where you need to choose from a very large number of +possibilities, you can create a dictionary mapping case values to +functions to call. For example: + + def function_1(...): + ... + + functions = {'a': function_1, + 'b': function_2, + 'c': self.method_1, ...} + + func = functions[value] + func() + +For calling methods on objects, you can simplify yet further by using +the *note getattr(): 448. built-in to retrieve methods with a particular +name: + + def visit_a(self, ...): + ... + ... + + def dispatch(self, value): + method_name = 'visit_' + str(value) + method = getattr(self, method_name) + method() + +It’s suggested that you use a prefix for the method names, such as +‘visit_’ in this example. Without such a prefix, if values are coming +from an untrusted source, an attacker would be able to call any method +on your object. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0275 + + +File: python.info, Node: Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?, Next: Why can’t lambda expressions contain statements?, Prev: Why isn’t there a switch or case statement in Python?, Up: Design and History FAQ + +11.3.11 Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? +---------------------------------------------------------------------------------------------------------------- + +Answer 1: Unfortunately, the interpreter pushes at least one C stack +frame for each Python stack frame. Also, extensions can call back into +Python at almost random moments. Therefore, a complete threads +implementation requires thread support for C. + +Answer 2: Fortunately, there is Stackless Python(1), which has a +completely redesigned interpreter loop that avoids the C stack. + + ---------- Footnotes ---------- + + (1) https://github.com/stackless-dev/stackless/wiki + + +File: python.info, Node: Why can’t lambda expressions contain statements?, Next: Can Python be compiled to machine code C or some other language?, Prev: Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?, Up: Design and History FAQ + +11.3.12 Why can’t lambda expressions contain statements? +-------------------------------------------------------- + +Python lambda expressions cannot contain statements because Python’s +syntactic framework can’t handle statements nested inside expressions. +However, in Python, this is not a serious problem. Unlike lambda forms +in other languages, where they add functionality, Python lambdas are +only a shorthand notation if you’re too lazy to define a function. + +Functions are already first class objects in Python, and can be declared +in a local scope. Therefore the only advantage of using a lambda +instead of a locally-defined function is that you don’t need to invent a +name for the function – but that’s just a local variable to which the +function object (which is exactly the same type of object that a lambda +expression yields) is assigned! + + +File: python.info, Node: Can Python be compiled to machine code C or some other language?, Next: How does Python manage memory?, Prev: Why can’t lambda expressions contain statements?, Up: Design and History FAQ + +11.3.13 Can Python be compiled to machine code, C or some other language? +------------------------------------------------------------------------- + +Cython(1) compiles a modified version of Python with optional +annotations into C extensions. Nuitka(2) is an up-and-coming compiler +of Python into C++ code, aiming to support the full Python language. +For compiling to Java you can consider VOC(3). + + ---------- Footnotes ---------- + + (1) http://cython.org/ + + (2) http://www.nuitka.net/ + + (3) https://voc.readthedocs.io + + +File: python.info, Node: How does Python manage memory?, Next: Why doesn’t CPython use a more traditional garbage collection scheme?, Prev: Can Python be compiled to machine code C or some other language?, Up: Design and History FAQ + +11.3.14 How does Python manage memory? +-------------------------------------- + +The details of Python memory management depend on the implementation. +The standard implementation of Python, *note CPython: 10d7, uses +reference counting to detect inaccessible objects, and another mechanism +to collect reference cycles, periodically executing a cycle detection +algorithm which looks for inaccessible cycles and deletes the objects +involved. The *note gc: 86. module provides functions to perform a +garbage collection, obtain debugging statistics, and tune the +collector’s parameters. + +Other implementations (such as Jython(1) or PyPy(2)), however, can rely +on a different mechanism such as a full-blown garbage collector. This +difference can cause some subtle porting problems if your Python code +depends on the behavior of the reference counting implementation. + +In some Python implementations, the following code (which is fine in +CPython) will probably run out of file descriptors: + + for file in very_long_list_of_files: + f = open(file) + c = f.read(1) + +Indeed, using CPython’s reference counting and destructor scheme, each +new assignment to `f' closes the previous file. With a traditional GC, +however, those file objects will only get collected (and closed) at +varying and possibly long intervals. + +If you want to write code that will work with any Python implementation, +you should explicitly close the file or use the *note with: 6e9. +statement; this will work regardless of memory management scheme: + + for file in very_long_list_of_files: + with open(file) as f: + c = f.read(1) + + ---------- Footnotes ---------- + + (1) http://www.jython.org + + (2) http://www.pypy.org + + +File: python.info, Node: Why doesn’t CPython use a more traditional garbage collection scheme?, Next: Why isn’t all memory freed when CPython exits?, Prev: How does Python manage memory?, Up: Design and History FAQ + +11.3.15 Why doesn’t CPython use a more traditional garbage collection scheme? +----------------------------------------------------------------------------- + +For one thing, this is not a C standard feature and hence it’s not +portable. (Yes, we know about the Boehm GC library. It has bits of +assembler code for `most' common platforms, not for all of them, and +although it is mostly transparent, it isn’t completely transparent; +patches are required to get Python to work with it.) + +Traditional GC also becomes a problem when Python is embedded into other +applications. While in a standalone Python it’s fine to replace the +standard malloc() and free() with versions provided by the GC library, +an application embedding Python may want to have its `own' substitute +for malloc() and free(), and may not want Python’s. Right now, CPython +works with anything that implements malloc() and free() properly. + + +File: python.info, Node: Why isn’t all memory freed when CPython exits?, Next: Why are there separate tuple and list data types?, Prev: Why doesn’t CPython use a more traditional garbage collection scheme?, Up: Design and History FAQ + +11.3.16 Why isn’t all memory freed when CPython exits? +------------------------------------------------------ + +Objects referenced from the global namespaces of Python modules are not +always deallocated when Python exits. This may happen if there are +circular references. There are also certain bits of memory that are +allocated by the C library that are impossible to free (e.g. a tool +like Purify will complain about these). Python is, however, aggressive +about cleaning up memory on exit and does try to destroy every single +object. + +If you want to force Python to delete certain things on deallocation use +the *note atexit: c. module to run a function that will force those +deletions. + + +File: python.info, Node: Why are there separate tuple and list data types?, Next: How are lists implemented in CPython?, Prev: Why isn’t all memory freed when CPython exits?, Up: Design and History FAQ + +11.3.17 Why are there separate tuple and list data types? +--------------------------------------------------------- + +Lists and tuples, while similar in many respects, are generally used in +fundamentally different ways. Tuples can be thought of as being similar +to Pascal records or C structs; they’re small collections of related +data which may be of different types which are operated on as a group. +For example, a Cartesian coordinate is appropriately represented as a +tuple of two or three numbers. + +Lists, on the other hand, are more like arrays in other languages. They +tend to hold a varying number of objects all of which have the same type +and which are operated on one-by-one. For example, ‘os.listdir('.')’ +returns a list of strings representing the files in the current +directory. Functions which operate on this output would generally not +break if you added another file or two to the directory. + +Tuples are immutable, meaning that once a tuple has been created, you +can’t replace any of its elements with a new value. Lists are mutable, +meaning that you can always change a list’s elements. Only immutable +elements can be used as dictionary keys, and hence only tuples and not +lists can be used as keys. + + +File: python.info, Node: How are lists implemented in CPython?, Next: How are dictionaries implemented in CPython?, Prev: Why are there separate tuple and list data types?, Up: Design and History FAQ + +11.3.18 How are lists implemented in CPython? +--------------------------------------------- + +CPython’s lists are really variable-length arrays, not Lisp-style linked +lists. The implementation uses a contiguous array of references to +other objects, and keeps a pointer to this array and the array’s length +in a list head structure. + +This makes indexing a list ‘a[i]’ an operation whose cost is independent +of the size of the list or the value of the index. + +When items are appended or inserted, the array of references is resized. +Some cleverness is applied to improve the performance of appending items +repeatedly; when the array must be grown, some extra space is allocated +so the next few times don’t require an actual resize. + + +File: python.info, Node: How are dictionaries implemented in CPython?, Next: Why must dictionary keys be immutable?, Prev: How are lists implemented in CPython?, Up: Design and History FAQ + +11.3.19 How are dictionaries implemented in CPython? +---------------------------------------------------- + +CPython’s dictionaries are implemented as resizable hash tables. +Compared to B-trees, this gives better performance for lookup (the most +common operation by far) under most circumstances, and the +implementation is simpler. + +Dictionaries work by computing a hash code for each key stored in the +dictionary using the *note hash(): 9c4. built-in function. The hash +code varies widely depending on the key and a per-process seed; for +example, “Python” could hash to -539294296 while “python”, a string that +differs by a single bit, could hash to 1142331976. The hash code is +then used to calculate a location in an internal array where the value +will be stored. Assuming that you’re storing keys that all have +different hash values, this means that dictionaries take constant time – +O(1), in Big-O notation – to retrieve a key. + + +File: python.info, Node: Why must dictionary keys be immutable?, Next: Why doesn’t list sort return the sorted list?, Prev: How are dictionaries implemented in CPython?, Up: Design and History FAQ + +11.3.20 Why must dictionary keys be immutable? +---------------------------------------------- + +The hash table implementation of dictionaries uses a hash value +calculated from the key value to find the key. If the key were a +mutable object, its value could change, and thus its hash could also +change. But since whoever changes the key object can’t tell that it was +being used as a dictionary key, it can’t move the entry around in the +dictionary. Then, when you try to look up the same object in the +dictionary it won’t be found because its hash value is different. If +you tried to look up the old value it wouldn’t be found either, because +the value of the object found in that hash bin would be different. + +If you want a dictionary indexed with a list, simply convert the list to +a tuple first; the function ‘tuple(L)’ creates a tuple with the same +entries as the list ‘L’. Tuples are immutable and can therefore be used +as dictionary keys. + +Some unacceptable solutions that have been proposed: + + - Hash lists by their address (object ID). This doesn’t work because + if you construct a new list with the same value it won’t be found; + e.g.: + + mydict = {[1, 2]: '12'} + print(mydict[[1, 2]]) + + would raise a *note KeyError: 2c7. exception because the id of the + ‘[1, 2]’ used in the second line differs from that in the first + line. In other words, dictionary keys should be compared using + ‘==’, not using *note is: 10be. + + - Make a copy when using a list as a key. This doesn’t work because + the list, being a mutable object, could contain a reference to + itself, and then the copying code would run into an infinite loop. + + - Allow lists as keys but tell the user not to modify them. This + would allow a class of hard-to-track bugs in programs when you + forgot or modified a list by accident. It also invalidates an + important invariant of dictionaries: every value in ‘d.keys()’ is + usable as a key of the dictionary. + + - Mark lists as read-only once they are used as a dictionary key. + The problem is that it’s not just the top-level object that could + change its value; you could use a tuple containing a list as a key. + Entering anything as a key into a dictionary would require marking + all objects reachable from there as read-only – and again, + self-referential objects could cause an infinite loop. + +There is a trick to get around this if you need to, but use it at your +own risk: You can wrap a mutable structure inside a class instance which +has both a *note __eq__(): 33f. and a *note __hash__(): 340. method. +You must then make sure that the hash value for all such wrapper objects +that reside in a dictionary (or other hash based structure), remain +fixed while the object is in the dictionary (or other structure). + + class ListWrapper: + def __init__(self, the_list): + self.the_list = the_list + + def __eq__(self, other): + return self.the_list == other.the_list + + def __hash__(self): + l = self.the_list + result = 98767 - len(l)*555 + for i, el in enumerate(l): + try: + result = result + (hash(el) % 9999999) * 1001 + i + except Exception: + result = (result % 7777777) + i * 333 + return result + +Note that the hash computation is complicated by the possibility that +some members of the list may be unhashable and also by the possibility +of arithmetic overflow. + +Furthermore it must always be the case that if ‘o1 == o2’ (ie +‘o1.__eq__(o2) is True’) then ‘hash(o1) == hash(o2)’ (ie, ‘o1.__hash__() +== o2.__hash__()’), regardless of whether the object is in a dictionary +or not. If you fail to meet these restrictions dictionaries and other +hash based structures will misbehave. + +In the case of ListWrapper, whenever the wrapper object is in a +dictionary the wrapped list must not change to avoid anomalies. Don’t +do this unless you are prepared to think hard about the requirements and +the consequences of not meeting them correctly. Consider yourself +warned. + + +File: python.info, Node: Why doesn’t list sort return the sorted list?, Next: How do you specify and enforce an interface spec in Python?, Prev: Why must dictionary keys be immutable?, Up: Design and History FAQ + +11.3.21 Why doesn’t list.sort() return the sorted list? +------------------------------------------------------- + +In situations where performance matters, making a copy of the list just +to sort it would be wasteful. Therefore, *note list.sort(): 445. sorts +the list in place. In order to remind you of that fact, it does not +return the sorted list. This way, you won’t be fooled into accidentally +overwriting a list when you need a sorted copy but also need to keep the +unsorted version around. + +If you want to return a new list, use the built-in *note sorted(): 444. +function instead. This function creates a new list from a provided +iterable, sorts it and returns it. For example, here’s how to iterate +over the keys of a dictionary in sorted order: + + for key in sorted(mydict): + ... # do whatever with mydict[key]... + + +File: python.info, Node: How do you specify and enforce an interface spec in Python?, Next: Why is there no goto?, Prev: Why doesn’t list sort return the sorted list?, Up: Design and History FAQ + +11.3.22 How do you specify and enforce an interface spec in Python? +------------------------------------------------------------------- + +An interface specification for a module as provided by languages such as +C++ and Java describes the prototypes for the methods and functions of +the module. Many feel that compile-time enforcement of interface +specifications helps in the construction of large programs. + +Python 2.6 adds an *note abc: 4. module that lets you define Abstract +Base Classes (ABCs). You can then use *note isinstance(): 44f. and +*note issubclass(): 450. to check whether an instance or a class +implements a particular ABC. The *note collections.abc: 1f. module +defines a set of useful ABCs such as *note Iterable: 1601, *note +Container: 15ff, and *note MutableMapping: 9ef. + +For Python, many of the advantages of interface specifications can be +obtained by an appropriate test discipline for components. + +A good test suite for a module can both provide a regression test and +serve as a module interface specification and a set of examples. Many +Python modules can be run as a script to provide a simple “self test.” +Even modules which use complex external interfaces can often be tested +in isolation using trivial “stub” emulations of the external interface. +The *note doctest: 67. and *note unittest: 11b. modules or third-party +test frameworks can be used to construct exhaustive test suites that +exercise every line of code in a module. + +An appropriate testing discipline can help build large complex +applications in Python as well as having interface specifications would. +In fact, it can be better because an interface specification cannot test +certain properties of a program. For example, the ‘append()’ method is +expected to add new elements to the end of some internal list; an +interface specification cannot test that your ‘append()’ implementation +will actually do this correctly, but it’s trivial to check this property +in a test suite. + +Writing test suites is very helpful, and you might want to design your +code with an eye to making it easily tested. One increasingly popular +technique, test-directed development, calls for writing parts of the +test suite first, before you write any of the actual code. Of course +Python allows you to be sloppy and not write test cases at all. + + +File: python.info, Node: Why is there no goto?, Next: Why can’t raw strings r-strings end with a backslash?, Prev: How do you specify and enforce an interface spec in Python?, Up: Design and History FAQ + +11.3.23 Why is there no goto? +----------------------------- + +In the 1970s people realized that unrestricted goto could lead to messy +“spaghetti” code that was hard to understand and revise. In a +high-level language, it is also unneeded as long as there are ways to +branch (in Python, with ‘if’ statements and ‘or’, ‘and’, and ‘if-else’ +expressions) and loop (with ‘while’ and ‘for’ statements, possibly +containing ‘continue’ and ‘break’). + +One can also use exceptions to provide a “structured goto” that works +even across function calls. Many feel that exceptions can conveniently +emulate all reasonable uses of the “go” or “goto” constructs of C, +Fortran, and other languages. For example: + + class label(Exception): pass # declare a label + + try: + ... + if condition: raise label() # goto label + ... + except label: # where to goto + pass + ... + +This doesn’t allow you to jump into the middle of a loop, but that’s +usually considered an abuse of goto anyway. Use sparingly. + + +File: python.info, Node: Why can’t raw strings r-strings end with a backslash?, Next: Why doesn’t Python have a “with” statement for attribute assignments?, Prev: Why is there no goto?, Up: Design and History FAQ + +11.3.24 Why can’t raw strings (r-strings) end with a backslash? +--------------------------------------------------------------- + +More precisely, they can’t end with an odd number of backslashes: the +unpaired backslash at the end escapes the closing quote character, +leaving an unterminated string. + +Raw strings were designed to ease creating input for processors (chiefly +regular expression engines) that want to do their own backslash escape +processing. Such processors consider an unmatched trailing backslash to +be an error anyway, so raw strings disallow that. In return, they allow +you to pass on the string quote character by escaping it with a +backslash. These rules work well when r-strings are used for their +intended purpose. + +If you’re trying to build Windows pathnames, note that all Windows +system calls accept forward slashes too: + + f = open("/mydir/file.txt") # works fine! + +If you’re trying to build a pathname for a DOS command, try e.g. one of + + dir = r"\this\is\my\dos\dir" "\\" + dir = r"\this\is\my\dos\dir\ "[:-1] + dir = "\\this\\is\\my\\dos\\dir\\" + + +File: python.info, Node: Why doesn’t Python have a “with” statement for attribute assignments?, Next: Why are colons required for the if/while/def/class statements?, Prev: Why can’t raw strings r-strings end with a backslash?, Up: Design and History FAQ + +11.3.25 Why doesn’t Python have a “with” statement for attribute assignments? +----------------------------------------------------------------------------- + +Python has a ‘with’ statement that wraps the execution of a block, +calling code on the entrance and exit from the block. Some language +have a construct that looks like this: + + with obj: + a = 1 # equivalent to obj.a = 1 + total = total + 1 # obj.total = obj.total + 1 + +In Python, such a construct would be ambiguous. + +Other languages, such as Object Pascal, Delphi, and C++, use static +types, so it’s possible to know, in an unambiguous way, what member is +being assigned to. This is the main point of static typing – the +compiler `always' knows the scope of every variable at compile time. + +Python uses dynamic types. It is impossible to know in advance which +attribute will be referenced at runtime. Member attributes may be added +or removed from objects on the fly. This makes it impossible to know, +from a simple reading, what attribute is being referenced: a local one, +a global one, or a member attribute? + +For instance, take the following incomplete snippet: + + def foo(a): + with a: + print(x) + +The snippet assumes that “a” must have a member attribute called “x”. +However, there is nothing in Python that tells the interpreter this. +What should happen if “a” is, let us say, an integer? If there is a +global variable named “x”, will it be used inside the with block? As +you see, the dynamic nature of Python makes such choices much harder. + +The primary benefit of “with” and similar language features (reduction +of code volume) can, however, easily be achieved in Python by +assignment. Instead of: + + function(args).mydict[index][index].a = 21 + function(args).mydict[index][index].b = 42 + function(args).mydict[index][index].c = 63 + +write this: + + ref = function(args).mydict[index][index] + ref.a = 21 + ref.b = 42 + ref.c = 63 + +This also has the side-effect of increasing execution speed because name +bindings are resolved at run-time in Python, and the second version only +needs to perform the resolution once. + + +File: python.info, Node: Why are colons required for the if/while/def/class statements?, Next: Why does Python allow commas at the end of lists and tuples?, Prev: Why doesn’t Python have a “with” statement for attribute assignments?, Up: Design and History FAQ + +11.3.26 Why are colons required for the if/while/def/class statements? +---------------------------------------------------------------------- + +The colon is required primarily to enhance readability (one of the +results of the experimental ABC language). Consider this: + + if a == b + print(a) + +versus + + if a == b: + print(a) + +Notice how the second one is slightly easier to read. Notice further +how a colon sets off the example in this FAQ answer; it’s a standard +usage in English. + +Another minor reason is that the colon makes it easier for editors with +syntax highlighting; they can look for colons to decide when indentation +needs to be increased instead of having to do a more elaborate parsing +of the program text. + + +File: python.info, Node: Why does Python allow commas at the end of lists and tuples?, Prev: Why are colons required for the if/while/def/class statements?, Up: Design and History FAQ + +11.3.27 Why does Python allow commas at the end of lists and tuples? +-------------------------------------------------------------------- + +Python lets you add a trailing comma at the end of lists, tuples, and +dictionaries: + + [1, 2, 3,] + ('a', 'b', 'c',) + d = { + "A": [1, 5], + "B": [6, 7], # last trailing comma is optional but good style + } + +There are several reasons to allow this. + +When you have a literal value for a list, tuple, or dictionary spread +across multiple lines, it’s easier to add more elements because you +don’t have to remember to add a comma to the previous line. The lines +can also be reordered without creating a syntax error. + +Accidentally omitting the comma can lead to errors that are hard to +diagnose. For example: + + x = [ + "fee", + "fie" + "foo", + "fum" + ] + +This list looks like it has four elements, but it actually contains +three: “fee”, “fiefoo” and “fum”. Always adding the comma avoids this +source of error. + +Allowing the trailing comma may also make programmatic code generation +easier. + + +File: python.info, Node: Library and Extension FAQ, Next: Extending/Embedding FAQ, Prev: Design and History FAQ, Up: Python Frequently Asked Questions + +11.4 Library and Extension FAQ +============================== + +* Menu: + +* General Library Questions:: +* Common tasks:: +* Threads:: +* Input and Output: Input and Output<2>. +* Network/Internet Programming:: +* Databases:: +* Mathematics and Numerics:: + + +File: python.info, Node: General Library Questions, Next: Common tasks, Up: Library and Extension FAQ + +11.4.1 General Library Questions +-------------------------------- + +* Menu: + +* How do I find a module or application to perform task X?:: +* Where is the math.py (socket.py, regex.py, etc.) source file?: Where is the math py socket py regex py etc source file?. +* How do I make a Python script executable on Unix?:: +* Is there a curses/termcap package for Python?:: +* Is there an equivalent to C’s onexit() in Python?: Is there an equivalent to C’s onexit in Python?. +* Why don’t my signal handlers work?:: + + +File: python.info, Node: How do I find a module or application to perform task X?, Next: Where is the math py socket py regex py etc source file?, Up: General Library Questions + +11.4.1.1 How do I find a module or application to perform task X? +................................................................. + +Check *note the Library Reference: e8e. to see if there’s a relevant +standard library module. (Eventually you’ll learn what’s in the +standard library and will be able to skip this step.) + +For third-party packages, search the Python Package Index(1) or try +Google(2) or another Web search engine. Searching for “Python” plus a +keyword or two for your topic of interest will usually find something +helpful. + + ---------- Footnotes ---------- + + (1) https://pypi.org + + (2) https://www.google.com + + +File: python.info, Node: Where is the math py socket py regex py etc source file?, Next: How do I make a Python script executable on Unix?, Prev: How do I find a module or application to perform task X?, Up: General Library Questions + +11.4.1.2 Where is the math.py (socket.py, regex.py, etc.) source file? +...................................................................... + +If you can’t find a source file for a module it may be a built-in or +dynamically loaded module implemented in C, C++ or other compiled +language. In this case you may not have the source file or it may be +something like ‘mathmodule.c’, somewhere in a C source directory (not on +the Python Path). + +There are (at least) three kinds of modules in Python: + + 1. modules written in Python (.py); + + 2. modules written in C and dynamically loaded (.dll, .pyd, .so, .sl, + etc); + + 3. modules written in C and linked with the interpreter; to get a list + of these, type: + + import sys + print(sys.builtin_module_names) + + +File: python.info, Node: How do I make a Python script executable on Unix?, Next: Is there a curses/termcap package for Python?, Prev: Where is the math py socket py regex py etc source file?, Up: General Library Questions + +11.4.1.3 How do I make a Python script executable on Unix? +.......................................................... + +You need to do two things: the script file’s mode must be executable and +the first line must begin with ‘#!’ followed by the path of the Python +interpreter. + +The first is done by executing ‘chmod +x scriptfile’ or perhaps ‘chmod +755 scriptfile’. + +The second can be done in a number of ways. The most straightforward +way is to write + + #!/usr/local/bin/python + +as the very first line of your file, using the pathname for where the +Python interpreter is installed on your platform. + +If you would like the script to be independent of where the Python +interpreter lives, you can use the ‘env’ program. Almost all Unix +variants support the following, assuming the Python interpreter is in a +directory on the user’s ‘PATH’: + + #!/usr/bin/env python + +`Don’t' do this for CGI scripts. The ‘PATH’ variable for CGI scripts is +often very minimal, so you need to use the actual absolute pathname of +the interpreter. + +Occasionally, a user’s environment is so full that the ‘/usr/bin/env’ +program fails; or there’s no env program at all. In that case, you can +try the following hack (due to Alex Rezinsky): + + #! /bin/sh + """:" + exec python $0 ${1+"$@"} + """ + +The minor disadvantage is that this defines the script’s __doc__ string. +However, you can fix that by adding + + __doc__ = """...Whatever...""" + + +File: python.info, Node: Is there a curses/termcap package for Python?, Next: Is there an equivalent to C’s onexit in Python?, Prev: How do I make a Python script executable on Unix?, Up: General Library Questions + +11.4.1.4 Is there a curses/termcap package for Python? +...................................................... + +For Unix variants: The standard Python source distribution comes with a +curses module in the Modules(1) subdirectory, though it’s not compiled +by default. (Note that this is not available in the Windows +distribution – there is no curses module for Windows.) + +The *note curses: 2c. module supports basic curses features as well as +many additional functions from ncurses and SYSV curses such as colour, +alternative character set support, pads, and mouse support. This means +the module isn’t compatible with operating systems that only have BSD +curses, but there don’t seem to be any currently maintained OSes that +fall into this category. + +For Windows: use the consolelib module(2). + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Modules + + (2) http://effbot.org/zone/console-index.htm + + +File: python.info, Node: Is there an equivalent to C’s onexit in Python?, Next: Why don’t my signal handlers work?, Prev: Is there a curses/termcap package for Python?, Up: General Library Questions + +11.4.1.5 Is there an equivalent to C’s onexit() in Python? +.......................................................... + +The *note atexit: c. module provides a register function that is similar +to C’s ‘onexit()’. + + +File: python.info, Node: Why don’t my signal handlers work?, Prev: Is there an equivalent to C’s onexit in Python?, Up: General Library Questions + +11.4.1.6 Why don’t my signal handlers work? +........................................... + +The most common problem is that the signal handler is declared with the +wrong argument list. It is called as + + handler(signum, frame) + +so it should be declared with two arguments: + + def handler(signum, frame): + ... + + +File: python.info, Node: Common tasks, Next: Threads, Prev: General Library Questions, Up: Library and Extension FAQ + +11.4.2 Common tasks +------------------- + +* Menu: + +* How do I test a Python program or component?:: +* How do I create documentation from doc strings?:: +* How do I get a single keypress at a time?:: + + +File: python.info, Node: How do I test a Python program or component?, Next: How do I create documentation from doc strings?, Up: Common tasks + +11.4.2.1 How do I test a Python program or component? +..................................................... + +Python comes with two testing frameworks. The *note doctest: 67. module +finds examples in the docstrings for a module and runs them, comparing +the output with the expected output given in the docstring. + +The *note unittest: 11b. module is a fancier testing framework modelled +on Java and Smalltalk testing frameworks. + +To make testing easier, you should use good modular design in your +program. Your program should have almost all functionality encapsulated +in either functions or class methods – and this sometimes has the +surprising and delightful effect of making the program run faster +(because local variable accesses are faster than global accesses). +Furthermore the program should avoid depending on mutating global +variables, since this makes testing much more difficult to do. + +The “global main logic” of your program may be as simple as + + if __name__ == "__main__": + main_logic() + +at the bottom of the main module of your program. + +Once your program is organized as a tractable collection of functions +and class behaviours you should write test functions that exercise the +behaviours. A test suite that automates a sequence of tests can be +associated with each module. This sounds like a lot of work, but since +Python is so terse and flexible it’s surprisingly easy. You can make +coding much more pleasant and fun by writing your test functions in +parallel with the “production code”, since this makes it easy to find +bugs and even design flaws earlier. + +“Support modules” that are not intended to be the main module of a +program may include a self-test of the module. + + if __name__ == "__main__": + self_test() + +Even programs that interact with complex external interfaces may be +tested when the external interfaces are unavailable by using “fake” +interfaces implemented in Python. + + +File: python.info, Node: How do I create documentation from doc strings?, Next: How do I get a single keypress at a time?, Prev: How do I test a Python program or component?, Up: Common tasks + +11.4.2.2 How do I create documentation from doc strings? +........................................................ + +The *note pydoc: d9. module can create HTML from the doc strings in your +Python source code. An alternative for creating API documentation +purely from docstrings is epydoc(1). Sphinx(2) can also include +docstring content. + + ---------- Footnotes ---------- + + (1) http://epydoc.sourceforge.net/ + + (2) http://sphinx-doc.org + + +File: python.info, Node: How do I get a single keypress at a time?, Prev: How do I create documentation from doc strings?, Up: Common tasks + +11.4.2.3 How do I get a single keypress at a time? +.................................................. + +For Unix variants there are several solutions. It’s straightforward to +do this using curses, but curses is a fairly large module to learn. + + +File: python.info, Node: Threads, Next: Input and Output<2>, Prev: Common tasks, Up: Library and Extension FAQ + +11.4.3 Threads +-------------- + +* Menu: + +* How do I program using threads?:: +* None of my threads seem to run; why?: None of my threads seem to run why?. +* How do I parcel out work among a bunch of worker threads?:: +* What kinds of global value mutation are thread-safe?:: +* Can’t we get rid of the Global Interpreter Lock?:: + + +File: python.info, Node: How do I program using threads?, Next: None of my threads seem to run why?, Up: Threads + +11.4.3.1 How do I program using threads? +........................................ + +Be sure to use the *note threading: 109. module and not the *note +_thread: 3. module. The *note threading: 109. module builds convenient +abstractions on top of the low-level primitives provided by the *note +_thread: 3. module. + +Aahz has a set of slides from his threading tutorial that are helpful; +see ‘http://www.pythoncraft.com/OSCON2001/’. + + +File: python.info, Node: None of my threads seem to run why?, Next: How do I parcel out work among a bunch of worker threads?, Prev: How do I program using threads?, Up: Threads + +11.4.3.2 None of my threads seem to run: why? +............................................. + +As soon as the main thread exits, all threads are killed. Your main +thread is running too quickly, giving the threads no time to do any +work. + +A simple fix is to add a sleep to the end of the program that’s long +enough for all the threads to finish: + + import threading, time + + def thread_task(name, n): + for i in range(n): + print(name, i) + + for i in range(10): + T = threading.Thread(target=thread_task, args=(str(i), i)) + T.start() + + time.sleep(10) # <---------------------------! + +But now (on many platforms) the threads don’t run in parallel, but +appear to run sequentially, one at a time! The reason is that the OS +thread scheduler doesn’t start a new thread until the previous thread is +blocked. + +A simple fix is to add a tiny sleep to the start of the run function: + + def thread_task(name, n): + time.sleep(0.001) # <--------------------! + for i in range(n): + print(name, i) + + for i in range(10): + T = threading.Thread(target=thread_task, args=(str(i), i)) + T.start() + + time.sleep(10) + +Instead of trying to guess a good delay value for *note time.sleep(): +680, it’s better to use some kind of semaphore mechanism. One idea is +to use the *note queue: da. module to create a queue object, let each +thread append a token to the queue when it finishes, and let the main +thread read as many tokens from the queue as there are threads. + + +File: python.info, Node: How do I parcel out work among a bunch of worker threads?, Next: What kinds of global value mutation are thread-safe?, Prev: None of my threads seem to run why?, Up: Threads + +11.4.3.3 How do I parcel out work among a bunch of worker threads? +.................................................................. + +The easiest way is to use the new *note concurrent.futures: 22. module, +especially the *note ThreadPoolExecutor: 27a. class. + +Or, if you want fine control over the dispatching algorithm, you can +write your own logic manually. Use the *note queue: da. module to +create a queue containing a list of jobs. The *note Queue: d18. class +maintains a list of objects and has a ‘.put(obj)’ method that adds items +to the queue and a ‘.get()’ method to return them. The class will take +care of the locking necessary to ensure that each job is handed out +exactly once. + +Here’s a trivial example: + + import threading, queue, time + + # The worker thread gets jobs off the queue. When the queue is empty, it + # assumes there will be no more work and exits. + # (Realistically workers will run until terminated.) + def worker(): + print('Running worker') + time.sleep(0.1) + while True: + try: + arg = q.get(block=False) + except queue.Empty: + print('Worker', threading.currentThread(), end=' ') + print('queue empty') + break + else: + print('Worker', threading.currentThread(), end=' ') + print('running with argument', arg) + time.sleep(0.5) + + # Create queue + q = queue.Queue() + + # Start a pool of 5 workers + for i in range(5): + t = threading.Thread(target=worker, name='worker %i' % (i+1)) + t.start() + + # Begin adding work to the queue + for i in range(50): + q.put(i) + + # Give threads time to run + print('Main thread sleeping') + time.sleep(5) + +When run, this will produce the following output: + + Running worker + Running worker + Running worker + Running worker + Running worker + Main thread sleeping + Worker running with argument 0 + Worker running with argument 1 + Worker running with argument 2 + Worker running with argument 3 + Worker running with argument 4 + Worker running with argument 5 + ... + +Consult the module’s documentation for more details; the *note Queue: +d18. class provides a featureful interface. + + +File: python.info, Node: What kinds of global value mutation are thread-safe?, Next: Can’t we get rid of the Global Interpreter Lock?, Prev: How do I parcel out work among a bunch of worker threads?, Up: Threads + +11.4.3.4 What kinds of global value mutation are thread-safe? +............................................................. + +A *note global interpreter lock: 506. (GIL) is used internally to ensure +that only one thread runs in the Python VM at a time. In general, +Python offers to switch among threads only between bytecode +instructions; how frequently it switches can be set via *note +sys.setswitchinterval(): beb. Each bytecode instruction and therefore +all the C implementation code reached from each instruction is therefore +atomic from the point of view of a Python program. + +In theory, this means an exact accounting requires an exact +understanding of the PVM bytecode implementation. In practice, it means +that operations on shared variables of built-in data types (ints, lists, +dicts, etc) that “look atomic” really are. + +For example, the following operations are all atomic (L, L1, L2 are +lists, D, D1, D2 are dicts, x, y are objects, i, j are ints): + + L.append(x) + L1.extend(L2) + x = L[i] + x = L.pop() + L1[i:j] = L2 + L.sort() + x = y + x.field = y + D[x] = y + D1.update(D2) + D.keys() + +These aren’t: + + i = i+1 + L.append(L[-1]) + L[i] = L[j] + D[x] = D[x] + 1 + +Operations that replace other objects may invoke those other objects’ +*note __del__(): 91f. method when their reference count reaches zero, +and that can affect things. This is especially true for the mass +updates to dictionaries and lists. When in doubt, use a mutex! + + +File: python.info, Node: Can’t we get rid of the Global Interpreter Lock?, Prev: What kinds of global value mutation are thread-safe?, Up: Threads + +11.4.3.5 Can’t we get rid of the Global Interpreter Lock? +......................................................... + +The *note global interpreter lock: 506. (GIL) is often seen as a +hindrance to Python’s deployment on high-end multiprocessor server +machines, because a multi-threaded Python program effectively only uses +one CPU, due to the insistence that (almost) all Python code can only +run while the GIL is held. + +Back in the days of Python 1.5, Greg Stein actually implemented a +comprehensive patch set (the “free threading” patches) that removed the +GIL and replaced it with fine-grained locking. Adam Olsen recently did +a similar experiment in his python-safethread(1) project. +Unfortunately, both experiments exhibited a sharp drop in single-thread +performance (at least 30% slower), due to the amount of fine-grained +locking necessary to compensate for the removal of the GIL. + +This doesn’t mean that you can’t make good use of Python on multi-CPU +machines! You just have to be creative with dividing the work up +between multiple `processes' rather than multiple `threads'. The *note +ProcessPoolExecutor: 2a4. class in the new *note concurrent.futures: 22. +module provides an easy way of doing so; the *note multiprocessing: b7. +module provides a lower-level API in case you want more control over +dispatching of tasks. + +Judicious use of C extensions will also help; if you use a C extension +to perform a time-consuming task, the extension can release the GIL +while the thread of execution is in the C code and allow other threads +to get some work done. Some standard library modules such as *note +zlib: 144. and *note hashlib: 8d. already do this. + +It has been suggested that the GIL should be a per-interpreter-state +lock rather than truly global; interpreters then wouldn’t be able to +share objects. Unfortunately, this isn’t likely to happen either. It +would be a tremendous amount of work, because many object +implementations currently have global state. For example, small +integers and short strings are cached; these caches would have to be +moved to the interpreter state. Other object types have their own free +list; these free lists would have to be moved to the interpreter state. +And so on. + +And I doubt that it can even be done in finite time, because the same +problem exists for 3rd party extensions. It is likely that 3rd party +extensions are being written at a faster rate than you can convert them +to store all their global state in the interpreter state. + +And finally, once you have multiple interpreters not sharing any state, +what have you gained over running each interpreter in a separate +process? + + ---------- Footnotes ---------- + + (1) https://code.google.com/archive/p/python-safethread + + +File: python.info, Node: Input and Output<2>, Next: Network/Internet Programming, Prev: Threads, Up: Library and Extension FAQ + +11.4.4 Input and Output +----------------------- + +* Menu: + +* How do I delete a file? (And other file questions…): How do I delete a file? And other file questions…. +* How do I copy a file?:: +* How do I read (or write) binary data?: How do I read or write binary data?. +* I can’t seem to use os.read() on a pipe created with os.popen(); why?: I can’t seem to use os read on a pipe created with os popen ; why?. +* How do I access the serial (RS232) port?: How do I access the serial RS232 port?. +* Why doesn’t closing sys.stdout (stdin, stderr) really close it?: Why doesn’t closing sys stdout stdin stderr really close it?. + + +File: python.info, Node: How do I delete a file? And other file questions…, Next: How do I copy a file?, Up: Input and Output<2> + +11.4.4.1 How do I delete a file? (And other file questions…) +............................................................ + +Use ‘os.remove(filename)’ or ‘os.unlink(filename)’; for documentation, +see the *note os: c4. module. The two functions are identical; *note +unlink(): a3c. is simply the name of the Unix system call for this +function. + +To remove a directory, use *note os.rmdir(): a3a.; use *note os.mkdir(): +a36. to create one. ‘os.makedirs(path)’ will create any intermediate +directories in ‘path’ that don’t exist. ‘os.removedirs(path)’ will +remove intermediate directories as long as they’re empty; if you want to +delete an entire directory tree and its contents, use *note +shutil.rmtree(): 21c. + +To rename a file, use ‘os.rename(old_path, new_path)’. + +To truncate a file, open it using ‘f = open(filename, "rb+")’, and use +‘f.truncate(offset)’; offset defaults to the current seek position. +There’s also ‘os.ftruncate(fd, offset)’ for files opened with *note +os.open(): 665, where `fd' is the file descriptor (a small integer). + +The *note shutil: e9. module also contains a number of functions to work +on files including *note copyfile(): 258, *note copytree(): 21a, and +*note rmtree(): 21c. + + +File: python.info, Node: How do I copy a file?, Next: How do I read or write binary data?, Prev: How do I delete a file? And other file questions…, Up: Input and Output<2> + +11.4.4.2 How do I copy a file? +.............................. + +The *note shutil: e9. module contains a *note copyfile(): 258. function. +Note that on MacOS 9 it doesn’t copy the resource fork and Finder info. + + +File: python.info, Node: How do I read or write binary data?, Next: I can’t seem to use os read on a pipe created with os popen ; why?, Prev: How do I copy a file?, Up: Input and Output<2> + +11.4.4.3 How do I read (or write) binary data? +.............................................. + +To read or write complex binary data formats, it’s best to use the *note +struct: f8. module. It allows you to take a string containing binary +data (usually numbers) and convert it to Python objects; and vice versa. + +For example, the following code reads two 2-byte integers and one 4-byte +integer in big-endian format from a file: + + import struct + + with open(filename, "rb") as f: + s = f.read(8) + x, y, z = struct.unpack(">hhl", s) + +The ‘>’ in the format string forces big-endian data; the letter ‘h’ +reads one “short integer” (2 bytes), and ‘l’ reads one “long integer” (4 +bytes) from the string. + +For data that is more regular (e.g. a homogeneous list of ints or +floats), you can also use the *note array: 7. module. + + Note: To read and write binary data, it is mandatory to open the + file in binary mode (here, passing ‘"rb"’ to *note open(): 4f0.). + If you use ‘"r"’ instead (the default), the file will be open in + text mode and ‘f.read()’ will return *note str: 330. objects rather + than *note bytes: 331. objects. + + +File: python.info, Node: I can’t seem to use os read on a pipe created with os popen ; why?, Next: How do I access the serial RS232 port?, Prev: How do I read or write binary data?, Up: Input and Output<2> + +11.4.4.4 I can’t seem to use os.read() on a pipe created with os.popen(); why? +.............................................................................. + +*note os.read(): 668. is a low-level function which takes a file +descriptor, a small integer representing the opened file. *note +os.popen(): 2a9. creates a high-level file object, the same type +returned by the built-in *note open(): 4f0. function. Thus, to read `n' +bytes from a pipe `p' created with *note os.popen(): 2a9, you need to +use ‘p.read(n)’. + + +File: python.info, Node: How do I access the serial RS232 port?, Next: Why doesn’t closing sys stdout stdin stderr really close it?, Prev: I can’t seem to use os read on a pipe created with os popen ; why?, Up: Input and Output<2> + +11.4.4.5 How do I access the serial (RS232) port? +................................................. + +For Win32, POSIX (Linux, BSD, etc.), Jython: + + ‘http://pyserial.sourceforge.net’ + +For Unix, see a Usenet post by Mitch Chapman: + + ‘https://groups.google.com/groups?selm=34A04430.CF9@ohioee.com’ + + +File: python.info, Node: Why doesn’t closing sys stdout stdin stderr really close it?, Prev: How do I access the serial RS232 port?, Up: Input and Output<2> + +11.4.4.6 Why doesn’t closing sys.stdout (stdin, stderr) really close it? +........................................................................ + +Python *note file objects: b42. are a high-level layer of abstraction on +low-level C file descriptors. + +For most file objects you create in Python via the built-in *note +open(): 4f0. function, ‘f.close()’ marks the Python file object as being +closed from Python’s point of view, and also arranges to close the +underlying C file descriptor. This also happens automatically in ‘f’’s +destructor, when ‘f’ becomes garbage. + +But stdin, stdout and stderr are treated specially by Python, because of +the special status also given to them by C. Running ‘sys.stdout.close()’ +marks the Python-level file object as being closed, but does `not' close +the associated C file descriptor. + +To close the underlying C file descriptor for one of these three, you +should first be sure that’s what you really want to do (e.g., you may +confuse extension modules trying to do I/O). If it is, use *note +os.close(): 3d2.: + + os.close(stdin.fileno()) + os.close(stdout.fileno()) + os.close(stderr.fileno()) + +Or you can use the numeric constants 0, 1 and 2, respectively. + + +File: python.info, Node: Network/Internet Programming, Next: Databases, Prev: Input and Output<2>, Up: Library and Extension FAQ + +11.4.5 Network/Internet Programming +----------------------------------- + +* Menu: + +* What WWW tools are there for Python?:: +* How can I mimic CGI form submission (METHOD=POST)?: How can I mimic CGI form submission METHOD=POST ?. +* What module should I use to help with generating HTML?:: +* How do I send mail from a Python script?:: +* How do I avoid blocking in the connect() method of a socket?: How do I avoid blocking in the connect method of a socket?. + + +File: python.info, Node: What WWW tools are there for Python?, Next: How can I mimic CGI form submission METHOD=POST ?, Up: Network/Internet Programming + +11.4.5.1 What WWW tools are there for Python? +............................................. + +See the chapters titled *note Internet Protocols and Support: 28cd. and +*note Internet Data Handling: 24ca. in the Library Reference Manual. +Python has many modules that will help you build server-side and +client-side web systems. + +A summary of available frameworks is maintained by Paul Boddie at +‘https://wiki.python.org/moin/WebProgramming’. + +Cameron Laird maintains a useful set of pages about Python web +technologies at ‘http://phaseit.net/claird/comp.lang.python/web_python’. + + +File: python.info, Node: How can I mimic CGI form submission METHOD=POST ?, Next: What module should I use to help with generating HTML?, Prev: What WWW tools are there for Python?, Up: Network/Internet Programming + +11.4.5.2 How can I mimic CGI form submission (METHOD=POST)? +........................................................... + +I would like to retrieve web pages that are the result of POSTing a +form. Is there existing code that would let me do this easily? + +Yes. Here’s a simple example that uses urllib.request: + + #!/usr/local/bin/python + + import urllib.request + + # build the query string + qs = "First=Josephine&MI=Q&Last=Public" + + # connect and send the server a path + req = urllib.request.urlopen('http://www.some-server.out-there' + '/cgi-bin/some-cgi-script', data=qs) + with req: + msg, hdrs = req.read(), req.info() + +Note that in general for percent-encoded POST operations, query strings +must be quoted using *note urllib.parse.urlencode(): 78b. For example, +to send ‘name=Guy Steele, Jr.’: + + >>> import urllib.parse + >>> urllib.parse.urlencode({'name': 'Guy Steele, Jr.'}) + 'name=Guy+Steele%2C+Jr.' + +See also +........ + +*note HOWTO Fetch Internet Resources Using The urllib Package: 29b5. for +extensive examples. + + +File: python.info, Node: What module should I use to help with generating HTML?, Next: How do I send mail from a Python script?, Prev: How can I mimic CGI form submission METHOD=POST ?, Up: Network/Internet Programming + +11.4.5.3 What module should I use to help with generating HTML? +............................................................... + +You can find a collection of useful links on the Web Programming wiki +page(1). + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/WebProgramming + + +File: python.info, Node: How do I send mail from a Python script?, Next: How do I avoid blocking in the connect method of a socket?, Prev: What module should I use to help with generating HTML?, Up: Network/Internet Programming + +11.4.5.4 How do I send mail from a Python script? +................................................. + +Use the standard library module *note smtplib: ed. + +Here’s a very simple interactive mail sender that uses it. This method +will work on any host that supports an SMTP listener. + + import sys, smtplib + + fromaddr = input("From: ") + toaddrs = input("To: ").split(',') + print("Enter message, end with ^D:") + msg = '' + while True: + line = sys.stdin.readline() + if not line: + break + msg += line + + # The actual mail send + server = smtplib.SMTP('localhost') + server.sendmail(fromaddr, toaddrs, msg) + server.quit() + +A Unix-only alternative uses sendmail. The location of the sendmail +program varies between systems; sometimes it is ‘/usr/lib/sendmail’, +sometimes ‘/usr/sbin/sendmail’. The sendmail manual page will help you +out. Here’s some sample code: + + import os + + SENDMAIL = "/usr/sbin/sendmail" # sendmail location + p = os.popen("%s -t -i" % SENDMAIL, "w") + p.write("To: receiver@example.com\n") + p.write("Subject: test\n") + p.write("\n") # blank line separating headers from body + p.write("Some text\n") + p.write("some more text\n") + sts = p.close() + if sts != 0: + print("Sendmail exit status", sts) + + +File: python.info, Node: How do I avoid blocking in the connect method of a socket?, Prev: How do I send mail from a Python script?, Up: Network/Internet Programming + +11.4.5.5 How do I avoid blocking in the connect() method of a socket? +..................................................................... + +The *note select: e5. module is commonly used to help with asynchronous +I/O on sockets. + +To prevent the TCP connect from blocking, you can set the socket to +non-blocking mode. Then when you do the ‘connect()’, you will either +connect immediately (unlikely) or get an exception that contains the +error number as ‘.errno’. ‘errno.EINPROGRESS’ indicates that the +connection is in progress, but hasn’t finished yet. Different OSes will +return different values, so you’re going to have to check what’s +returned on your system. + +You can use the ‘connect_ex()’ method to avoid creating an exception. +It will just return the errno value. To poll, you can call +‘connect_ex()’ again later – ‘0’ or ‘errno.EISCONN’ indicate that you’re +connected – or you can pass this socket to select to check if it’s +writable. + + Note: The *note asyncore: b. module presents a framework-like + approach to the problem of writing non-blocking networking code. + The third-party Twisted(1) library is a popular and feature-rich + alternative. + + ---------- Footnotes ---------- + + (1) https://twistedmatrix.com/trac/ + + +File: python.info, Node: Databases, Next: Mathematics and Numerics, Prev: Network/Internet Programming, Up: Library and Extension FAQ + +11.4.6 Databases +---------------- + +* Menu: + +* Are there any interfaces to database packages in Python?:: +* How do you implement persistent objects in Python?:: + + +File: python.info, Node: Are there any interfaces to database packages in Python?, Next: How do you implement persistent objects in Python?, Up: Databases + +11.4.6.1 Are there any interfaces to database packages in Python? +................................................................. + +Yes. + +Interfaces to disk-based hashes such as *note DBM: 35. and *note GDBM: +34. are also included with standard Python. There is also the *note +sqlite3: f2. module, which provides a lightweight disk-based relational +database. + +Support for most relational databases is available. See the +DatabaseProgramming wiki page(1) for details. + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/DatabaseProgramming + + +File: python.info, Node: How do you implement persistent objects in Python?, Prev: Are there any interfaces to database packages in Python?, Up: Databases + +11.4.6.2 How do you implement persistent objects in Python? +........................................................... + +The *note pickle: ca. library module solves this in a very general way +(though you still can’t store things like open files, sockets or +windows), and the *note shelve: e7. library module uses pickle and +(g)dbm to create persistent mappings containing arbitrary Python +objects. + + +File: python.info, Node: Mathematics and Numerics, Prev: Databases, Up: Library and Extension FAQ + +11.4.7 Mathematics and Numerics +------------------------------- + +* Menu: + +* How do I generate random numbers in Python?:: + + +File: python.info, Node: How do I generate random numbers in Python?, Up: Mathematics and Numerics + +11.4.7.1 How do I generate random numbers in Python? +.................................................... + +The standard module *note random: dc. implements a random number +generator. Usage is simple: + + import random + random.random() + +This returns a random floating point number in the range [0, 1). + +There are also many other specialized generators in this module, such +as: + + * ‘randrange(a, b)’ chooses an integer in the range [a, b). + + * ‘uniform(a, b)’ chooses a floating point number in the range [a, + b). + + * ‘normalvariate(mean, sdev)’ samples the normal (Gaussian) + distribution. + +Some higher-level functions operate on sequences directly, such as: + + * ‘choice(S)’ chooses random element from a given sequence + + * ‘shuffle(L)’ shuffles a list in-place, i.e. permutes it randomly + +There’s also a ‘Random’ class you can instantiate to create independent +multiple random number generators. + + +File: python.info, Node: Extending/Embedding FAQ, Next: Python on Windows FAQ, Prev: Library and Extension FAQ, Up: Python Frequently Asked Questions + +11.5 Extending/Embedding FAQ +============================ + +* Menu: + +* Can I create my own functions in C?:: +* Can I create my own functions in C++?:: +* Writing C is hard; are there any alternatives?:: +* How can I execute arbitrary Python statements from C?:: +* How can I evaluate an arbitrary Python expression from C?:: +* How do I extract C values from a Python object?:: +* How do I use Py_BuildValue() to create a tuple of arbitrary length?: How do I use Py_BuildValue to create a tuple of arbitrary length?. +* How do I call an object’s method from C?:: +* How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?: How do I catch the output from PyErr_Print or anything that prints to stdout/stderr ?. +* How do I access a module written in Python from C?:: +* How do I interface to C++ objects from Python?:: +* I added a module using the Setup file and the make fails; why?:: +* How do I debug an extension?:: +* I want to compile a Python module on my Linux system, but some files are missing. Why?: I want to compile a Python module on my Linux system but some files are missing Why?. +* How do I tell “incomplete input” from “invalid input”?:: +* How do I find undefined g++ symbols __builtin_new or __pure_virtual?:: +* Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?: Can I create an object class with some methods implemented in C and others in Python e g through inheritance ?. + + +File: python.info, Node: Can I create my own functions in C?, Next: Can I create my own functions in C++?, Up: Extending/Embedding FAQ + +11.5.1 Can I create my own functions in C? +------------------------------------------ + +Yes, you can create built-in modules containing functions, variables, +exceptions and even new types in C. This is explained in the document +*note Extending and Embedding the Python Interpreter: e90. + +Most intermediate or advanced Python books will also cover this topic. + + +File: python.info, Node: Can I create my own functions in C++?, Next: Writing C is hard; are there any alternatives?, Prev: Can I create my own functions in C?, Up: Extending/Embedding FAQ + +11.5.2 Can I create my own functions in C++? +-------------------------------------------- + +Yes, using the C compatibility features found in C++. Place ‘extern "C" +{ ... }’ around the Python include files and put ‘extern "C"’ before +each function that is going to be called by the Python interpreter. +Global or static C++ objects with constructors are probably not a good +idea. + + +File: python.info, Node: Writing C is hard; are there any alternatives?, Next: How can I execute arbitrary Python statements from C?, Prev: Can I create my own functions in C++?, Up: Extending/Embedding FAQ + +11.5.3 Writing C is hard; are there any alternatives? +----------------------------------------------------- + +There are a number of alternatives to writing your own C extensions, +depending on what you’re trying to do. + +Cython(1) and its relative Pyrex(2) are compilers that accept a slightly +modified form of Python and generate the corresponding C code. Cython +and Pyrex make it possible to write an extension without having to learn +Python’s C API. + +If you need to interface to some C or C++ library for which no Python +extension currently exists, you can try wrapping the library’s data +types and functions with a tool such as SWIG(3). SIP(4), CXX(5) +Boost(6), or Weave(7) are also alternatives for wrapping C++ libraries. + + ---------- Footnotes ---------- + + (1) http://cython.org + + (2) https://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/ + + (3) http://www.swig.org + + (4) https://riverbankcomputing.com/software/sip/intro + + (5) http://cxx.sourceforge.net/ + + (6) http://www.boost.org/libs/python/doc/index.html + + (7) https://github.com/scipy/weave + + +File: python.info, Node: How can I execute arbitrary Python statements from C?, Next: How can I evaluate an arbitrary Python expression from C?, Prev: Writing C is hard; are there any alternatives?, Up: Extending/Embedding FAQ + +11.5.4 How can I execute arbitrary Python statements from C? +------------------------------------------------------------ + +The highest-level function to do this is *note PyRun_SimpleString(): +38c8. which takes a single string argument to be executed in the context +of the module ‘__main__’ and returns ‘0’ for success and ‘-1’ when an +exception occurred (including *note SyntaxError: 458.). If you want +more control, use *note PyRun_String(): 391b.; see the source for *note +PyRun_SimpleString(): 38c8. in ‘Python/pythonrun.c’. + + +File: python.info, Node: How can I evaluate an arbitrary Python expression from C?, Next: How do I extract C values from a Python object?, Prev: How can I execute arbitrary Python statements from C?, Up: Extending/Embedding FAQ + +11.5.5 How can I evaluate an arbitrary Python expression from C? +---------------------------------------------------------------- + +Call the function *note PyRun_String(): 391b. from the previous question +with the start symbol *note Py_eval_input: 3929.; it parses an +expression, evaluates it and returns its value. + + +File: python.info, Node: How do I extract C values from a Python object?, Next: How do I use Py_BuildValue to create a tuple of arbitrary length?, Prev: How can I evaluate an arbitrary Python expression from C?, Up: Extending/Embedding FAQ + +11.5.6 How do I extract C values from a Python object? +------------------------------------------------------ + +That depends on the object’s type. If it’s a tuple, *note +PyTuple_Size(): 3b99. returns its length and *note PyTuple_GetItem(): +385c. returns the item at a specified index. Lists have similar +functions, ‘PyListSize()’ and *note PyList_GetItem(): 385d. + +For bytes, *note PyBytes_Size(): 3b20. returns its length and *note +PyBytes_AsStringAndSize(): 3b24. provides a pointer to its value and its +length. Note that Python bytes objects may contain null bytes so C’s +‘strlen()’ should not be used. + +To test the type of an object, first make sure it isn’t ‘NULL’, and then +use *note PyBytes_Check(): d1f, *note PyTuple_Check(): 3b95, *note +PyList_Check(): 3baa, etc. + +There is also a high-level API to Python objects which is provided by +the so-called ‘abstract’ interface – read ‘Include/abstract.h’ for +further details. It allows interfacing with any kind of Python sequence +using calls like *note PySequence_Length(): 3a47, *note +PySequence_GetItem(): 38f6, etc. as well as many other useful protocols +such as numbers (*note PyNumber_Index(): 3a3e. et al.) and mappings in +the PyMapping APIs. + + +File: python.info, Node: How do I use Py_BuildValue to create a tuple of arbitrary length?, Next: How do I call an object’s method from C?, Prev: How do I extract C values from a Python object?, Up: Extending/Embedding FAQ + +11.5.7 How do I use Py_BuildValue() to create a tuple of arbitrary length? +-------------------------------------------------------------------------- + +You can’t. Use *note PyTuple_Pack(): 3b98. instead. + + +File: python.info, Node: How do I call an object’s method from C?, Next: How do I catch the output from PyErr_Print or anything that prints to stdout/stderr ?, Prev: How do I use Py_BuildValue to create a tuple of arbitrary length?, Up: Extending/Embedding FAQ + +11.5.8 How do I call an object’s method from C? +----------------------------------------------- + +The *note PyObject_CallMethod(): 3a0e. function can be used to call an +arbitrary method of an object. The parameters are the object, the name +of the method to call, a format string like that used with *note +Py_BuildValue(): 2ce, and the argument values: + + PyObject * + PyObject_CallMethod(PyObject *object, const char *method_name, + const char *arg_format, ...); + +This works for any object that has methods – whether built-in or +user-defined. You are responsible for eventually *note Py_DECREF(): +266.’ing the return value. + +To call, e.g., a file object’s “seek” method with arguments 10, 0 +(assuming the file object pointer is “f”): + + res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0); + if (res == NULL) { + ... an exception occurred ... + } + else { + Py_DECREF(res); + } + +Note that since *note PyObject_CallObject(): 384f. `always' wants a +tuple for the argument list, to call a function without arguments, pass +“()” for the format, and to call a function with one argument, surround +the argument in parentheses, e.g. “(i)”. + + +File: python.info, Node: How do I catch the output from PyErr_Print or anything that prints to stdout/stderr ?, Next: How do I access a module written in Python from C?, Prev: How do I call an object’s method from C?, Up: Extending/Embedding FAQ + +11.5.9 How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)? +----------------------------------------------------------------------------------------------- + +In Python code, define an object that supports the ‘write()’ method. +Assign this object to *note sys.stdout: 30e. and *note sys.stderr: 30f. +Call print_error, or just allow the standard traceback mechanism to +work. Then, the output will go wherever your ‘write()’ method sends it. + +The easiest way to do this is to use the *note io.StringIO: 82d. class: + + >>> import io, sys + >>> sys.stdout = io.StringIO() + >>> print('foo') + >>> print('hello world!') + >>> sys.stderr.write(sys.stdout.getvalue()) + foo + hello world! + +A custom object to do the same would look like this: + + >>> import io, sys + >>> class StdoutCatcher(io.TextIOBase): + ... def __init__(self): + ... self.data = [] + ... def write(self, stuff): + ... self.data.append(stuff) + ... + >>> import sys + >>> sys.stdout = StdoutCatcher() + >>> print('foo') + >>> print('hello world!') + >>> sys.stderr.write(''.join(sys.stdout.data)) + foo + hello world! + + +File: python.info, Node: How do I access a module written in Python from C?, Next: How do I interface to C++ objects from Python?, Prev: How do I catch the output from PyErr_Print or anything that prints to stdout/stderr ?, Up: Extending/Embedding FAQ + +11.5.10 How do I access a module written in Python from C? +---------------------------------------------------------- + +You can get a pointer to the module object as follows: + + module = PyImport_ImportModule(""); + +If the module hasn’t been imported yet (i.e. it is not yet present in +*note sys.modules: 1152.), this initializes the module; otherwise it +simply returns the value of ‘sys.modules[""]’. Note that it +doesn’t enter the module into any namespace – it only ensures it has +been initialized and is stored in *note sys.modules: 1152. + +You can then access the module’s attributes (i.e. any name defined in +the module) as follows: + + attr = PyObject_GetAttrString(module, ""); + +Calling *note PyObject_SetAttrString(): 3a05. to assign to variables in +the module also works. + + +File: python.info, Node: How do I interface to C++ objects from Python?, Next: I added a module using the Setup file and the make fails; why?, Prev: How do I access a module written in Python from C?, Up: Extending/Embedding FAQ + +11.5.11 How do I interface to C++ objects from Python? +------------------------------------------------------ + +Depending on your requirements, there are many approaches. To do this +manually, begin by reading *note the “Extending and Embedding” document: +e90. Realize that for the Python run-time system, there isn’t a whole +lot of difference between C and C++ – so the strategy of building a new +Python type around a C structure (pointer) type will also work for C++ +objects. + +For C++ libraries, see *note Writing C is hard; are there any +alternatives?: 4035. + + +File: python.info, Node: I added a module using the Setup file and the make fails; why?, Next: How do I debug an extension?, Prev: How do I interface to C++ objects from Python?, Up: Extending/Embedding FAQ + +11.5.12 I added a module using the Setup file and the make fails; why? +---------------------------------------------------------------------- + +Setup must end in a newline, if there is no newline there, the build +process fails. (Fixing this requires some ugly shell script hackery, +and this bug is so minor that it doesn’t seem worth the effort.) + + +File: python.info, Node: How do I debug an extension?, Next: I want to compile a Python module on my Linux system but some files are missing Why?, Prev: I added a module using the Setup file and the make fails; why?, Up: Extending/Embedding FAQ + +11.5.13 How do I debug an extension? +------------------------------------ + +When using GDB with dynamically loaded extensions, you can’t set a +breakpoint in your extension until your extension is loaded. + +In your ‘.gdbinit’ file (or interactively), add the command: + + br _PyImport_LoadDynamicModule + +Then, when you run GDB: + + $ gdb /local/bin/python + gdb) run myscript.py + gdb) continue # repeat until your extension is loaded + gdb) finish # so that your extension is loaded + gdb) br myfunction.c:50 + gdb) continue + + +File: python.info, Node: I want to compile a Python module on my Linux system but some files are missing Why?, Next: How do I tell “incomplete input” from “invalid input”?, Prev: How do I debug an extension?, Up: Extending/Embedding FAQ + +11.5.14 I want to compile a Python module on my Linux system, but some files are missing. Why? +---------------------------------------------------------------------------------------------- + +Most packaged versions of Python don’t include the +‘/usr/lib/python2.`x'/config/’ directory, which contains various files +required for compiling Python extensions. + +For Red Hat, install the python-devel RPM to get the necessary files. + +For Debian, run ‘apt-get install python-dev’. + + +File: python.info, Node: How do I tell “incomplete input” from “invalid input”?, Next: How do I find undefined g++ symbols __builtin_new or __pure_virtual?, Prev: I want to compile a Python module on my Linux system but some files are missing Why?, Up: Extending/Embedding FAQ + +11.5.15 How do I tell “incomplete input” from “invalid input”? +-------------------------------------------------------------- + +Sometimes you want to emulate the Python interactive interpreter’s +behavior, where it gives you a continuation prompt when the input is +incomplete (e.g. you typed the start of an “if” statement or you didn’t +close your parentheses or triple string quotes), but it gives you a +syntax error message immediately when the input is invalid. + +In Python you can use the *note codeop: 1d. module, which approximates +the parser’s behavior sufficiently. IDLE uses this, for example. + +The easiest way to do it in C is to call *note PyRun_InteractiveLoop(): +390e. (perhaps in a separate thread) and let the Python interpreter +handle the input for you. You can also set the *note +PyOS_ReadlineFunctionPointer(): 96c. to point at your custom input +function. See ‘Modules/readline.c’ and ‘Parser/myreadline.c’ for more +hints. + +However sometimes you have to run the embedded Python interpreter in the +same thread as your rest application and you can’t allow the *note +PyRun_InteractiveLoop(): 390e. to stop while waiting for user input. +The one solution then is to call ‘PyParser_ParseString()’ and test for +‘e.error’ equal to ‘E_EOF’, which means the input is incomplete. Here’s +a sample code fragment, untested, inspired by code from Alex Farber: + + #define PY_SSIZE_T_CLEAN + #include + #include + #include + #include + #include + #include + + int testcomplete(char *code) + /* code should end in \n */ + /* return -1 for error, 0 for incomplete, 1 for complete */ + { + node *n; + perrdetail e; + + n = PyParser_ParseString(code, &_PyParser_Grammar, + Py_file_input, &e); + if (n == NULL) { + if (e.error == E_EOF) + return 0; + return -1; + } + + PyNode_Free(n); + return 1; + } + +Another solution is trying to compile the received string with *note +Py_CompileString(): 3921. If it compiles without errors, try to execute +the returned code object by calling *note PyEval_EvalCode(): 3925. +Otherwise save the input for later. If the compilation fails, find out +if it’s an error or just more input is required - by extracting the +message string from the exception tuple and comparing it to the string +“unexpected EOF while parsing”. Here is a complete example using the +GNU readline library (you may want to ignore `SIGINT' while calling +readline()): + + #include + #include + + #define PY_SSIZE_T_CLEAN + #include + #include + #include + #include + + int main (int argc, char* argv[]) + { + int i, j, done = 0; /* lengths of line, code */ + char ps1[] = ">>> "; + char ps2[] = "... "; + char *prompt = ps1; + char *msg, *line, *code = NULL; + PyObject *src, *glb, *loc; + PyObject *exc, *val, *trb, *obj, *dum; + + Py_Initialize (); + loc = PyDict_New (); + glb = PyDict_New (); + PyDict_SetItemString (glb, "__builtins__", PyEval_GetBuiltins ()); + + while (!done) + { + line = readline (prompt); + + if (NULL == line) /* Ctrl-D pressed */ + { + done = 1; + } + else + { + i = strlen (line); + + if (i > 0) + add_history (line); /* save non-empty lines */ + + if (NULL == code) /* nothing in code yet */ + j = 0; + else + j = strlen (code); + + code = realloc (code, i + j + 2); + if (NULL == code) /* out of memory */ + exit (1); + + if (0 == j) /* code was empty, so */ + code[0] = '\0'; /* keep strncat happy */ + + strncat (code, line, i); /* append line to code */ + code[i + j] = '\n'; /* append '\n' to code */ + code[i + j + 1] = '\0'; + + src = Py_CompileString (code, "", Py_single_input); + + if (NULL != src) /* compiled just fine - */ + { + if (ps1 == prompt || /* ">>> " or */ + '\n' == code[i + j - 1]) /* "... " and double '\n' */ + { /* so execute it */ + dum = PyEval_EvalCode (src, glb, loc); + Py_XDECREF (dum); + Py_XDECREF (src); + free (code); + code = NULL; + if (PyErr_Occurred ()) + PyErr_Print (); + prompt = ps1; + } + } /* syntax error or E_EOF? */ + else if (PyErr_ExceptionMatches (PyExc_SyntaxError)) + { + PyErr_Fetch (&exc, &val, &trb); /* clears exception! */ + + if (PyArg_ParseTuple (val, "sO", &msg, &obj) && + !strcmp (msg, "unexpected EOF while parsing")) /* E_EOF */ + { + Py_XDECREF (exc); + Py_XDECREF (val); + Py_XDECREF (trb); + prompt = ps2; + } + else /* some other syntax error */ + { + PyErr_Restore (exc, val, trb); + PyErr_Print (); + free (code); + code = NULL; + prompt = ps1; + } + } + else /* some non-syntax error */ + { + PyErr_Print (); + free (code); + code = NULL; + prompt = ps1; + } + + free (line); + } + } + + Py_XDECREF(glb); + Py_XDECREF(loc); + Py_Finalize(); + exit(0); + } + + +File: python.info, Node: How do I find undefined g++ symbols __builtin_new or __pure_virtual?, Next: Can I create an object class with some methods implemented in C and others in Python e g through inheritance ?, Prev: How do I tell “incomplete input” from “invalid input”?, Up: Extending/Embedding FAQ + +11.5.16 How do I find undefined g++ symbols __builtin_new or __pure_virtual? +---------------------------------------------------------------------------- + +To dynamically load g++ extension modules, you must recompile Python, +relink it using g++ (change LINKCC in the Python Modules Makefile), and +link your extension module using g++ (e.g., ‘g++ -shared -o mymodule.so +mymodule.o’). + + +File: python.info, Node: Can I create an object class with some methods implemented in C and others in Python e g through inheritance ?, Prev: How do I find undefined g++ symbols __builtin_new or __pure_virtual?, Up: Extending/Embedding FAQ + +11.5.17 Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)? +------------------------------------------------------------------------------------------------------------------------ + +Yes, you can inherit from built-in classes such as *note int: 184, *note +list: 262, *note dict: 1b8, etc. + +The Boost Python Library (BPL, +‘http://www.boost.org/libs/python/doc/index.html’) provides a way of +doing this from C++ (i.e. you can inherit from an extension class +written in C++ using the BPL). + + +File: python.info, Node: Python on Windows FAQ, Next: Graphic User Interface FAQ, Prev: Extending/Embedding FAQ, Up: Python Frequently Asked Questions + +11.6 Python on Windows FAQ +========================== + +* Menu: + +* How do I run a Python program under Windows?:: +* How do I make Python scripts executable?:: +* Why does Python sometimes take so long to start?:: +* How do I make an executable from a Python script?:: +* Is a *.pyd file the same as a DLL?: Is a * pyd file the same as a DLL?. +* How can I embed Python into a Windows application?:: +* How do I keep editors from inserting tabs into my Python source?:: +* How do I check for a keypress without blocking?:: + + +File: python.info, Node: How do I run a Python program under Windows?, Next: How do I make Python scripts executable?, Up: Python on Windows FAQ + +11.6.1 How do I run a Python program under Windows? +--------------------------------------------------- + +This is not necessarily a straightforward question. If you are already +familiar with running programs from the Windows command line then +everything will seem obvious; otherwise, you might need a little more +guidance. + +Unless you use some sort of integrated development environment, you will +end up `typing' Windows commands into what is variously referred to as a +“DOS window” or “Command prompt window”. Usually you can create such a +window from your search bar by searching for ‘cmd’. You should be able +to recognize when you have started such a window because you will see a +Windows “command prompt”, which usually looks like this: + + C:\> + +The letter may be different, and there might be other things after it, +so you might just as easily see something like: + + D:\YourName\Projects\Python> + +depending on how your computer has been set up and what else you have +recently done with it. Once you have started such a window, you are +well on the way to running Python programs. + +You need to realize that your Python scripts have to be processed by +another program called the Python `interpreter'. The interpreter reads +your script, compiles it into bytecodes, and then executes the bytecodes +to run your program. So, how do you arrange for the interpreter to +handle your Python? + +First, you need to make sure that your command window recognises the +word “py” as an instruction to start the interpreter. If you have +opened a command window, you should try entering the command ‘py’ and +hitting return: + + C:\Users\YourName> py + +You should then see something like: + + Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel)] on win32 + Type "help", "copyright", "credits" or "license" for more information. + >>> + +You have started the interpreter in “interactive mode”. That means you +can enter Python statements or expressions interactively and have them +executed or evaluated while you wait. This is one of Python’s strongest +features. Check it by entering a few expressions of your choice and +seeing the results: + + >>> print("Hello") + Hello + >>> "Hello" * 3 + 'HelloHelloHello' + +Many people use the interactive mode as a convenient yet highly +programmable calculator. When you want to end your interactive Python +session, call the *note exit(): 12b9. function or hold the ‘Ctrl’ key +down while you enter a ‘Z’, then hit the “‘Enter’” key to get back to +your Windows command prompt. + +You may also find that you have a Start-menu entry such as Start ‣ +Programs ‣ Python 3.x ‣ Python (command line) that results in you seeing +the ‘>>>’ prompt in a new window. If so, the window will disappear +after you call the *note exit(): 12b9. function or enter the ‘Ctrl-Z’ +character; Windows is running a single “python” command in the window, +and closes it when you terminate the interpreter. + +Now that we know the ‘py’ command is recognized, you can give your +Python script to it. You’ll have to give either an absolute or a +relative path to the Python script. Let’s say your Python script is +located in your desktop and is named ‘hello.py’, and your command prompt +is nicely opened in your home directory so you’re seeing something +similar to: + + C:\Users\YourName> + +So now you’ll ask the ‘py’ command to give your script to Python by +typing ‘py’ followed by your script path: + + C:\Users\YourName> py Desktop\hello.py + hello + + +File: python.info, Node: How do I make Python scripts executable?, Next: Why does Python sometimes take so long to start?, Prev: How do I run a Python program under Windows?, Up: Python on Windows FAQ + +11.6.2 How do I make Python scripts executable? +----------------------------------------------- + +On Windows, the standard Python installer already associates the .py +extension with a file type (Python.File) and gives that file type an +open command that runs the interpreter (‘D:\Program +Files\Python\python.exe "%1" %*’). This is enough to make scripts +executable from the command prompt as ‘foo.py’. If you’d rather be able +to execute the script by simple typing ‘foo’ with no extension you need +to add .py to the PATHEXT environment variable. + + +File: python.info, Node: Why does Python sometimes take so long to start?, Next: How do I make an executable from a Python script?, Prev: How do I make Python scripts executable?, Up: Python on Windows FAQ + +11.6.3 Why does Python sometimes take so long to start? +------------------------------------------------------- + +Usually Python starts very quickly on Windows, but occasionally there +are bug reports that Python suddenly begins to take a long time to start +up. This is made even more puzzling because Python will work fine on +other Windows systems which appear to be configured identically. + +The problem may be caused by a misconfiguration of virus checking +software on the problem machine. Some virus scanners have been known to +introduce startup overhead of two orders of magnitude when the scanner +is configured to monitor all reads from the filesystem. Try checking +the configuration of virus scanning software on your systems to ensure +that they are indeed configured identically. McAfee, when configured to +scan all file system read activity, is a particular offender. + + +File: python.info, Node: How do I make an executable from a Python script?, Next: Is a * pyd file the same as a DLL?, Prev: Why does Python sometimes take so long to start?, Up: Python on Windows FAQ + +11.6.4 How do I make an executable from a Python script? +-------------------------------------------------------- + +See cx_Freeze(1) for a distutils extension that allows you to create +console and GUI executables from Python code. py2exe(2), the most +popular extension for building Python 2.x-based executables, does not +yet support Python 3 but a version that does is in development. + + ---------- Footnotes ---------- + + (1) https://cx-freeze.readthedocs.io/en/latest/ + + (2) http://www.py2exe.org/ + + +File: python.info, Node: Is a * pyd file the same as a DLL?, Next: How can I embed Python into a Windows application?, Prev: How do I make an executable from a Python script?, Up: Python on Windows FAQ + +11.6.5 Is a ‘*.pyd’ file the same as a DLL? +------------------------------------------- + +Yes, .pyd files are dll’s, but there are a few differences. If you have +a DLL named ‘foo.pyd’, then it must have a function ‘PyInit_foo()’. You +can then write Python “import foo”, and Python will search for foo.pyd +(as well as foo.py, foo.pyc) and if it finds it, will attempt to call +‘PyInit_foo()’ to initialize it. You do not link your .exe with +foo.lib, as that would cause Windows to require the DLL to be present. + +Note that the search path for foo.pyd is PYTHONPATH, not the same as the +path that Windows uses to search for foo.dll. Also, foo.pyd need not be +present to run your program, whereas if you linked your program with a +dll, the dll is required. Of course, foo.pyd is required if you want to +say ‘import foo’. In a DLL, linkage is declared in the source code with +‘__declspec(dllexport)’. In a .pyd, linkage is defined in a list of +available functions. + + +File: python.info, Node: How can I embed Python into a Windows application?, Next: How do I keep editors from inserting tabs into my Python source?, Prev: Is a * pyd file the same as a DLL?, Up: Python on Windows FAQ + +11.6.6 How can I embed Python into a Windows application? +--------------------------------------------------------- + +Embedding the Python interpreter in a Windows app can be summarized as +follows: + + 1. Do _not_ build Python into your .exe file directly. On Windows, + Python must be a DLL to handle importing modules that are + themselves DLL’s. (This is the first key undocumented fact.) + Instead, link to ‘python`NN'.dll’; it is typically installed in + ‘C:\Windows\System’. `NN' is the Python version, a number such as + “33” for Python 3.3. + + You can link to Python in two different ways. Load-time linking + means linking against ‘python`NN'.lib’, while run-time linking + means linking against ‘python`NN'.dll’. (General note: + ‘python`NN'.lib’ is the so-called “import lib” corresponding to + ‘python`NN'.dll’. It merely defines symbols for the linker.) + + Run-time linking greatly simplifies link options; everything + happens at run time. Your code must load ‘python`NN'.dll’ using + the Windows ‘LoadLibraryEx()’ routine. The code must also use + access routines and data in ‘python`NN'.dll’ (that is, Python’s C + API’s) using pointers obtained by the Windows ‘GetProcAddress()’ + routine. Macros can make using these pointers transparent to any C + code that calls routines in Python’s C API. + + Borland note: convert ‘python`NN'.lib’ to OMF format using + Coff2Omf.exe first. + + 2. If you use SWIG, it is easy to create a Python “extension module” + that will make the app’s data and methods available to Python. + SWIG will handle just about all the grungy details for you. The + result is C code that you link `into' your .exe file (!) You do + _not_ have to create a DLL file, and this also simplifies linking. + + 3. SWIG will create an init function (a C function) whose name depends + on the name of the extension module. For example, if the name of + the module is leo, the init function will be called initleo(). If + you use SWIG shadow classes, as you should, the init function will + be called initleoc(). This initializes a mostly hidden helper + class used by the shadow class. + + The reason you can link the C code in step 2 into your .exe file is + that calling the initialization function is equivalent to importing + the module into Python! (This is the second key undocumented + fact.) + + 4. In short, you can use the following code to initialize the Python + interpreter with your extension module. + + #include "python.h" + ... + Py_Initialize(); // Initialize Python. + initmyAppc(); // Initialize (import) the helper class. + PyRun_SimpleString("import myApp"); // Import the shadow class. + + 5. There are two problems with Python’s C API which will become + apparent if you use a compiler other than MSVC, the compiler used + to build pythonNN.dll. + + Problem 1: The so-called “Very High Level” functions that take FILE + * arguments will not work in a multi-compiler environment because + each compiler’s notion of a struct FILE will be different. From an + implementation standpoint these are very _low_ level functions. + + Problem 2: SWIG generates the following code when generating + wrappers to void functions: + + Py_INCREF(Py_None); + _resultobj = Py_None; + return _resultobj; + + Alas, Py_None is a macro that expands to a reference to a complex + data structure called _Py_NoneStruct inside pythonNN.dll. Again, + this code will fail in a mult-compiler environment. Replace such + code by: + + return Py_BuildValue(""); + + It may be possible to use SWIG’s ‘%typemap’ command to make the + change automatically, though I have not been able to get this to + work (I’m a complete SWIG newbie). + + 6. Using a Python shell script to put up a Python interpreter window + from inside your Windows app is not a good idea; the resulting + window will be independent of your app’s windowing system. Rather, + you (or the wxPythonWindow class) should create a “native” + interpreter window. It is easy to connect that window to the + Python interpreter. You can redirect Python’s i/o to _any_ object + that supports read and write, so all you need is a Python object + (defined in your extension module) that contains read() and write() + methods. + + +File: python.info, Node: How do I keep editors from inserting tabs into my Python source?, Next: How do I check for a keypress without blocking?, Prev: How can I embed Python into a Windows application?, Up: Python on Windows FAQ + +11.6.7 How do I keep editors from inserting tabs into my Python source? +----------------------------------------------------------------------- + +The FAQ does not recommend using tabs, and the Python style guide, PEP +8(1), recommends 4 spaces for distributed Python code; this is also the +Emacs python-mode default. + +Under any editor, mixing tabs and spaces is a bad idea. MSVC is no +different in this respect, and is easily configured to use spaces: Take +Tools ‣ Options ‣ Tabs, and for file type “Default” set “Tab size” and +“Indent size” to 4, and select the “Insert spaces” radio button. + +Python raises *note IndentationError: e78. or *note TabError: e77. if +mixed tabs and spaces are causing problems in leading whitespace. You +may also run the *note tabnanny: 100. module to check a directory tree +in batch mode. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0008 + + +File: python.info, Node: How do I check for a keypress without blocking?, Prev: How do I keep editors from inserting tabs into my Python source?, Up: Python on Windows FAQ + +11.6.8 How do I check for a keypress without blocking? +------------------------------------------------------ + +Use the *note msvcrt: b6. module. This is a standard Windows-specific +extension module. It defines a function ‘kbhit()’ which checks whether +a keyboard hit is present, and ‘getch()’ which gets one character +without echoing it. + + +File: python.info, Node: Graphic User Interface FAQ, Next: “Why is Python Installed on my Computer?” FAQ, Prev: Python on Windows FAQ, Up: Python Frequently Asked Questions + +11.7 Graphic User Interface FAQ +=============================== + +* Menu: + +* General GUI Questions:: +* What platform-independent GUI toolkits exist for Python?:: +* What platform-specific GUI toolkits exist for Python?:: +* Tkinter questions:: + + +File: python.info, Node: General GUI Questions, Next: What platform-independent GUI toolkits exist for Python?, Up: Graphic User Interface FAQ + +11.7.1 General GUI Questions +---------------------------- + + +File: python.info, Node: What platform-independent GUI toolkits exist for Python?, Next: What platform-specific GUI toolkits exist for Python?, Prev: General GUI Questions, Up: Graphic User Interface FAQ + +11.7.2 What platform-independent GUI toolkits exist for Python? +--------------------------------------------------------------- + +Depending on what platform(s) you are aiming at, there are several. +Some of them haven’t been ported to Python 3 yet. At least *note +Tkinter: 4055. and *note Qt: 4056. are known to be Python 3-compatible. + +* Menu: + +* Tkinter:: +* wxWidgets:: +* Qt:: +* Gtk+:: +* Kivy:: +* FLTK:: +* OpenGL:: + + +File: python.info, Node: Tkinter, Next: wxWidgets, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.1 Tkinter +................ + +Standard builds of Python include an object-oriented interface to the +Tcl/Tk widget set, called *note tkinter: 2e7f. This is probably the +easiest to install (since it comes included with most binary +distributions(1) of Python) and use. For more info about Tk, including +pointers to the source, see the Tcl/Tk home page(2). Tcl/Tk is fully +portable to the Mac OS X, Windows, and Unix platforms. + + ---------- Footnotes ---------- + + (1) https://www.python.org/downloads/ + + (2) https://www.tcl.tk + + +File: python.info, Node: wxWidgets, Next: Qt, Prev: Tkinter, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.2 wxWidgets +.................. + +wxWidgets (‘https://www.wxwidgets.org’) is a free, portable GUI class +library written in C++ that provides a native look and feel on a number +of platforms, with Windows, Mac OS X, GTK, X11, all listed as current +stable targets. Language bindings are available for a number of +languages including Python, Perl, Ruby, etc. + +wxPython(1) is the Python binding for wxwidgets. While it often lags +slightly behind the official wxWidgets releases, it also offers a number +of features via pure Python extensions that are not available in other +language bindings. There is an active wxPython user and developer +community. + +Both wxWidgets and wxPython are free, open source, software with +permissive licences that allow their use in commercial products as well +as in freeware or shareware. + + ---------- Footnotes ---------- + + (1) https://www.wxpython.org + + +File: python.info, Node: Qt, Next: Gtk+, Prev: wxWidgets, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.3 Qt +........... + +There are bindings available for the Qt toolkit (using either PyQt(1) or +PySide(2)) and for KDE (PyKDE4(3)). PyQt is currently more mature than +PySide, but you must buy a PyQt license from Riverbank Computing(4) if +you want to write proprietary applications. PySide is free for all +applications. + +Qt 4.5 upwards is licensed under the LGPL license; also, commercial +licenses are available from The Qt Company(5). + + ---------- Footnotes ---------- + + (1) https://riverbankcomputing.com/software/pyqt/intro + + (2) https://wiki.qt.io/PySide + + (3) https://techbase.kde.org/Languages/Python/Using_PyKDE_4 + + (4) https://www.riverbankcomputing.com/commercial/license-faq + + (5) https://www.qt.io/licensing/ + + +File: python.info, Node: Gtk+, Next: Kivy, Prev: Qt, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.4 Gtk+ +............. + +The GObject introspection bindings(1) for Python allow you to write GTK+ +3 applications. There is also a Python GTK+ 3 Tutorial(2). + +The older PyGtk bindings for the Gtk+ 2 toolkit(3) have been implemented +by James Henstridge; see <‘http://www.pygtk.org’>. + + ---------- Footnotes ---------- + + (1) https://wiki.gnome.org/Projects/PyGObject + + (2) https://python-gtk-3-tutorial.readthedocs.io + + (3) https://www.gtk.org + + +File: python.info, Node: Kivy, Next: FLTK, Prev: Gtk+, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.5 Kivy +............. + +Kivy(1) is a cross-platform GUI library supporting both desktop +operating systems (Windows, macOS, Linux) and mobile devices (Android, +iOS). It is written in Python and Cython, and can use a range of +windowing backends. + +Kivy is free and open source software distributed under the MIT license. + + ---------- Footnotes ---------- + + (1) https://kivy.org/ + + +File: python.info, Node: FLTK, Next: OpenGL, Prev: Kivy, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.6 FLTK +............. + +Python bindings for the FLTK toolkit(1), a simple yet powerful and +mature cross-platform windowing system, are available from the PyFLTK +project(2). + + ---------- Footnotes ---------- + + (1) http://www.fltk.org + + (2) http://pyfltk.sourceforge.net + + +File: python.info, Node: OpenGL, Prev: FLTK, Up: What platform-independent GUI toolkits exist for Python? + +11.7.2.7 OpenGL +............... + +For OpenGL bindings, see PyOpenGL(1). + + ---------- Footnotes ---------- + + (1) http://pyopengl.sourceforge.net + + +File: python.info, Node: What platform-specific GUI toolkits exist for Python?, Next: Tkinter questions, Prev: What platform-independent GUI toolkits exist for Python?, Up: Graphic User Interface FAQ + +11.7.3 What platform-specific GUI toolkits exist for Python? +------------------------------------------------------------ + +By installing the PyObjc Objective-C bridge(1), Python programs can use +Mac OS X’s Cocoa libraries. + +*note Pythonwin: 4047. by Mark Hammond includes an interface to the +Microsoft Foundation Classes and a Python programming environment that’s +written mostly in Python using the MFC classes. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/pyobjc/ + + +File: python.info, Node: Tkinter questions, Prev: What platform-specific GUI toolkits exist for Python?, Up: Graphic User Interface FAQ + +11.7.4 Tkinter questions +------------------------ + +* Menu: + +* How do I freeze Tkinter applications?:: +* Can I have Tk events handled while waiting for I/O?:: +* I can’t get key bindings to work in Tkinter; why?: I can’t get key bindings to work in Tkinter why?. + + +File: python.info, Node: How do I freeze Tkinter applications?, Next: Can I have Tk events handled while waiting for I/O?, Up: Tkinter questions + +11.7.4.1 How do I freeze Tkinter applications? +.............................................. + +Freeze is a tool to create stand-alone applications. When freezing +Tkinter applications, the applications will not be truly stand-alone, as +the application will still need the Tcl and Tk libraries. + +One solution is to ship the application with the Tcl and Tk libraries, +and point to them at run-time using the ‘TCL_LIBRARY’ and ‘TK_LIBRARY’ +environment variables. + +To get truly stand-alone applications, the Tcl scripts that form the +library have to be integrated into the application as well. One tool +supporting that is SAM (stand-alone modules), which is part of the Tix +distribution (‘http://tix.sourceforge.net/’). + +Build Tix with SAM enabled, perform the appropriate call to +‘Tclsam_init()’, etc. inside Python’s ‘Modules/tkappinit.c’, and link +with libtclsam and libtksam (you might include the Tix libraries as +well). + + +File: python.info, Node: Can I have Tk events handled while waiting for I/O?, Next: I can’t get key bindings to work in Tkinter why?, Prev: How do I freeze Tkinter applications?, Up: Tkinter questions + +11.7.4.2 Can I have Tk events handled while waiting for I/O? +............................................................ + +On platforms other than Windows, yes, and you don’t even need threads! +But you’ll have to restructure your I/O code a bit. Tk has the +equivalent of Xt’s ‘XtAddInput()’ call, which allows you to register a +callback function which will be called from the Tk mainloop when I/O is +possible on a file descriptor. See *note File Handlers: 2e99. + + +File: python.info, Node: I can’t get key bindings to work in Tkinter why?, Prev: Can I have Tk events handled while waiting for I/O?, Up: Tkinter questions + +11.7.4.3 I can’t get key bindings to work in Tkinter: why? +.......................................................... + +An often-heard complaint is that event handlers bound to events with the +‘bind()’ method don’t get handled even when the appropriate key is +pressed. + +The most common cause is that the widget to which the binding applies +doesn’t have “keyboard focus”. Check out the Tk documentation for the +focus command. Usually a widget is given the keyboard focus by clicking +in it (but not for labels; see the takefocus option). + + +File: python.info, Node: “Why is Python Installed on my Computer?” FAQ, Prev: Graphic User Interface FAQ, Up: Python Frequently Asked Questions + +11.8 “Why is Python Installed on my Computer?” FAQ +================================================== + +* Menu: + +* What is Python?: What is Python?<2>. +* Why is Python installed on my machine?:: +* Can I delete Python?:: + + +File: python.info, Node: What is Python?<2>, Next: Why is Python installed on my machine?, Up: “Why is Python Installed on my Computer?” FAQ + +11.8.1 What is Python? +---------------------- + +Python is a programming language. It’s used for many different +applications. It’s used in some high schools and colleges as an +introductory programming language because Python is easy to learn, but +it’s also used by professional software developers at places such as +Google, NASA, and Lucasfilm Ltd. + +If you wish to learn more about Python, start with the Beginner’s Guide +to Python(1). + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/BeginnersGuide + + +File: python.info, Node: Why is Python installed on my machine?, Next: Can I delete Python?, Prev: What is Python?<2>, Up: “Why is Python Installed on my Computer?” FAQ + +11.8.2 Why is Python installed on my machine? +--------------------------------------------- + +If you find Python installed on your system but don’t remember +installing it, there are several possible ways it could have gotten +there. + + * Perhaps another user on the computer wanted to learn programming + and installed it; you’ll have to figure out who’s been using the + machine and might have installed it. + + * A third-party application installed on the machine might have been + written in Python and included a Python installation. There are + many such applications, from GUI programs to network servers and + administrative scripts. + + * Some Windows machines also have Python installed. At this writing + we’re aware of computers from Hewlett-Packard and Compaq that + include Python. Apparently some of HP/Compaq’s administrative + tools are written in Python. + + * Many Unix-compatible operating systems, such as Mac OS X and some + Linux distributions, have Python installed by default; it’s + included in the base installation. + + +File: python.info, Node: Can I delete Python?, Prev: Why is Python installed on my machine?, Up: “Why is Python Installed on my Computer?” FAQ + +11.8.3 Can I delete Python? +--------------------------- + +That depends on where Python came from. + +If someone installed it deliberately, you can remove it without hurting +anything. On Windows, use the Add/Remove Programs icon in the Control +Panel. + +If Python was installed by a third-party application, you can also +remove it, but that application will no longer work. You should use +that application’s uninstaller rather than removing Python directly. + +If Python came with your operating system, removing it is not +recommended. If you remove it, whatever tools were written in Python +will no longer run, and some of them might be important to you. +Reinstalling the whole system would then be required to fix things +again. + + +File: python.info, Node: Glossary, Next: About these documents, Prev: Python Frequently Asked Questions, Up: Top + +12 Glossary +*********** + +‘>>>’ + + The default Python prompt of the interactive shell. Often seen for + code examples which can be executed interactively in the + interpreter. + +‘...’ + + Can refer to: + + * The default Python prompt of the interactive shell when + entering the code for an indented code block, when within a + pair of matching left and right delimiters (parentheses, + square brackets, curly braces or triple quotes), or after + specifying a decorator. + + * The *note Ellipsis: 12b6. built-in constant. + +2to3 + + A tool that tries to convert Python 2.x code to Python 3.x code by + handling most of the incompatibilities which can be detected by + parsing the source and traversing the parse tree. + + 2to3 is available in the standard library as *note lib2to3: a7.; a + standalone entry point is provided as ‘Tools/scripts/2to3’. See + *note 2to3 - Automated Python 2 to 3 code translation: c76. + +abstract base class + + Abstract base classes complement *note duck-typing: 4069. by + providing a way to define interfaces when other techniques like + *note hasattr(): 447. would be clumsy or subtly wrong (for example + with *note magic methods: 10de.). ABCs introduce virtual + subclasses, which are classes that don’t inherit from a class but + are still recognized by *note isinstance(): 44f. and *note + issubclass(): 450.; see the *note abc: 4. module documentation. + Python comes with many built-in ABCs for data structures (in the + *note collections.abc: 1f. module), numbers (in the *note numbers: + c1. module), streams (in the *note io: a1. module), import finders + and loaders (in the *note importlib.abc: 9c. module). You can + create your own ABCs with the *note abc: 4. module. + +annotation + + A label associated with a variable, a class attribute or a function + parameter or return value, used by convention as a *note type hint: + 406b. + + Annotations of local variables cannot be accessed at runtime, but + annotations of global variables, class attributes, and functions + are stored in the ‘__annotations__’ special attribute of modules, + classes, and functions, respectively. + + See *note variable annotation: 61f, *note function annotation: ef1, + PEP 484(1) and PEP 526(2), which describe this functionality. + +argument + + A value passed to a *note function: 11c9. (or *note method: 406c.) + when calling the function. There are two kinds of argument: + + * `keyword argument': an argument preceded by an identifier + (e.g. ‘name=’) in a function call or passed as a value in a + dictionary preceded by ‘**’. For example, ‘3’ and ‘5’ are + both keyword arguments in the following calls to *note + complex(): 189.: + + complex(real=3, imag=5) + complex(**{'real': 3, 'imag': 5}) + + * `positional argument': an argument that is not a keyword + argument. Positional arguments can appear at the beginning of + an argument list and/or be passed as elements of an *note + iterable: bac. preceded by ‘*’. For example, ‘3’ and ‘5’ are + both positional arguments in the following calls: + + complex(3, 5) + complex(*(3, 5)) + + Arguments are assigned to the named local variables in a function + body. See the *note Calls: 64c. section for the rules governing + this assignment. Syntactically, any expression can be used to + represent an argument; the evaluated value is assigned to the local + variable. + + See also the *note parameter: 11d2. glossary entry, the FAQ + question on *note the difference between arguments and parameters: + 3fb4, and PEP 362(3). + +asynchronous context manager + + An object which controls the environment seen in an *note async + with: 643. statement by defining *note __aenter__(): 113b. and + *note __aexit__(): 113c. methods. Introduced by PEP 492(4). + +asynchronous generator + + A function which returns an *note asynchronous generator iterator: + 335c. It looks like a coroutine function defined with *note async + def: 284. except that it contains *note yield: 18f. expressions for + producing a series of values usable in an *note async for: 2e3. + loop. + + Usually refers to an asynchronous generator function, but may refer + to an `asynchronous generator iterator' in some contexts. In cases + where the intended meaning isn’t clear, using the full terms avoids + ambiguity. + + An asynchronous generator function may contain *note await: 1a3. + expressions as well as *note async for: 2e3, and *note async with: + 643. statements. + +asynchronous generator iterator + + An object created by a *note asynchronous generator: 11a9. + function. + + This is an *note asynchronous iterator: 645. which when called + using the *note __anext__(): 1138. method returns an awaitable + object which will execute the body of the asynchronous generator + function until the next *note yield: 18f. expression. + + Each *note yield: 18f. temporarily suspends processing, remembering + the location execution state (including local variables and pending + try-statements). When the `asynchronous generator iterator' + effectively resumes with another awaitable returned by *note + __anext__(): 1138, it picks up where it left off. See PEP 492(5) + and PEP 525(6). + +asynchronous iterable + + An object, that can be used in an *note async for: 2e3. statement. + Must return an *note asynchronous iterator: 645. from its *note + __aiter__(): 487. method. Introduced by PEP 492(7). + +asynchronous iterator + + An object that implements the *note __aiter__(): 487. and *note + __anext__(): 1138. methods. ‘__anext__’ must return an *note + awaitable: 519. object. *note async for: 2e3. resolves the + awaitables returned by an asynchronous iterator’s *note + __anext__(): 1138. method until it raises a *note + StopAsyncIteration: 10cd. exception. Introduced by PEP 492(8). + +attribute + + A value associated with an object which is referenced by name using + dotted expressions. For example, if an object `o' has an attribute + `a' it would be referenced as `o.a'. + +awaitable + + An object that can be used in an *note await: 1a3. expression. Can + be a *note coroutine: 1a6. or an object with an *note __await__(): + 642. method. See also PEP 492(9). + +BDFL + + Benevolent Dictator For Life, a.k.a. Guido van Rossum(10), + Python’s creator. + +binary file + + A *note file object: b42. able to read and write *note bytes-like + objects: 5e8. Examples of binary files are files opened in binary + mode (‘'rb'’, ‘'wb'’ or ‘'rb+'’), ‘sys.stdin.buffer’, + ‘sys.stdout.buffer’, and instances of *note io.BytesIO: 79b. and + *note gzip.GzipFile: 6dd. + + See also *note text file: f3a. for a file object able to read and + write *note str: 330. objects. + +bytes-like object + + An object that supports the *note Buffer Protocol: 1291. and can + export a C-*note contiguous: 1360. buffer. This includes all *note + bytes: 331, *note bytearray: 332, and *note array.array: 451. + objects, as well as many common *note memoryview: 25c. objects. + Bytes-like objects can be used for various operations that work + with binary data; these include compression, saving to a binary + file, and sending over a socket. + + Some operations need the binary data to be mutable. The + documentation often refers to these as “read-write bytes-like + objects”. Example mutable buffer objects include *note bytearray: + 332. and a *note memoryview: 25c. of a *note bytearray: 332. Other + operations require the binary data to be stored in immutable + objects (“read-only bytes-like objects”); examples of these include + *note bytes: 331. and a *note memoryview: 25c. of a *note bytes: + 331. object. + +bytecode + + Python source code is compiled into bytecode, the internal + representation of a Python program in the CPython interpreter. The + bytecode is also cached in ‘.pyc’ files so that executing the same + file is faster the second time (recompilation from source to + bytecode can be avoided). This “intermediate language” is said to + run on a *note virtual machine: 247d. that executes the machine + code corresponding to each bytecode. Do note that bytecodes are + not expected to work between different Python virtual machines, nor + to be stable between Python releases. + + A list of bytecode instructions can be found in the documentation + for *note the dis module: 35dc. + +callback + + A subroutine function which is passed as an argument to be executed + at some point in the future. + +class + + A template for creating user-defined objects. Class definitions + normally contain method definitions which operate on instances of + the class. + +class variable + + A variable defined in a class and intended to be modified only at + class level (i.e., not in an instance of the class). + +coercion + + The implicit conversion of an instance of one type to another + during an operation which involves two arguments of the same type. + For example, ‘int(3.15)’ converts the floating point number to the + integer ‘3’, but in ‘3+4.5’, each argument is of a different type + (one int, one float), and both must be converted to the same type + before they can be added or it will raise a *note TypeError: 192. + Without coercion, all arguments of even compatible types would have + to be normalized to the same value by the programmer, e.g., + ‘float(3)+4.5’ rather than just ‘3+4.5’. + +complex number + + An extension of the familiar real number system in which all + numbers are expressed as a sum of a real part and an imaginary + part. Imaginary numbers are real multiples of the imaginary unit + (the square root of ‘-1’), often written ‘i’ in mathematics or ‘j’ + in engineering. Python has built-in support for complex numbers, + which are written with this latter notation; the imaginary part is + written with a ‘j’ suffix, e.g., ‘3+1j’. To get access to complex + equivalents of the *note math: b1. module, use *note cmath: 19. + Use of complex numbers is a fairly advanced mathematical feature. + If you’re not aware of a need for them, it’s almost certain you can + safely ignore them. + +context manager + + An object which controls the environment seen in a *note with: 6e9. + statement by defining *note __enter__(): c95. and *note __exit__(): + c96. methods. See PEP 343(11). + +context variable + + A variable which can have different values depending on its + context. This is similar to Thread-Local Storage in which each + execution thread may have a different value for a variable. + However, with context variables, there may be several contexts in + one execution thread and the main usage for context variables is to + keep track of variables in concurrent asynchronous tasks. See + *note contextvars: 25. + +contiguous + + A buffer is considered contiguous exactly if it is either + `C-contiguous' or `Fortran contiguous'. Zero-dimensional buffers + are C and Fortran contiguous. In one-dimensional arrays, the items + must be laid out in memory next to each other, in order of + increasing indexes starting from zero. In multidimensional + C-contiguous arrays, the last index varies the fastest when + visiting items in order of memory address. However, in Fortran + contiguous arrays, the first index varies the fastest. + +coroutine + + Coroutines are a more generalized form of subroutines. Subroutines + are entered at one point and exited at another point. Coroutines + can be entered, exited, and resumed at many different points. They + can be implemented with the *note async def: 284. statement. See + also PEP 492(12). + +coroutine function + + A function which returns a *note coroutine: 1a6. object. A + coroutine function may be defined with the *note async def: 284. + statement, and may contain *note await: 1a3, *note async for: 2e3, + and *note async with: 643. keywords. These were introduced by PEP + 492(13). + +CPython + + The canonical implementation of the Python programming language, as + distributed on python.org(14). The term “CPython” is used when + necessary to distinguish this implementation from others such as + Jython or IronPython. + +decorator + + A function returning another function, usually applied as a + function transformation using the ‘@wrapper’ syntax. Common + examples for decorators are *note classmethod(): 1d8. and *note + staticmethod(): 1d9. + + The decorator syntax is merely syntactic sugar, the following two + function definitions are semantically equivalent: + + def f(...): + ... + f = staticmethod(f) + + @staticmethod + def f(...): + ... + + The same concept exists for classes, but is less commonly used + there. See the documentation for *note function definitions: ef0. + and *note class definitions: c6a. for more about decorators. + +descriptor + + Any object which defines the methods *note __get__(): 10dd, *note + __set__(): 10e1, or *note __delete__(): 10e2. When a class + attribute is a descriptor, its special binding behavior is + triggered upon attribute lookup. Normally, using `a.b' to get, set + or delete an attribute looks up the object named `b' in the class + dictionary for `a', but if `b' is a descriptor, the respective + descriptor method gets called. Understanding descriptors is a key + to a deep understanding of Python because they are the basis for + many features including functions, methods, properties, class + methods, static methods, and reference to super classes. + + For more information about descriptors’ methods, see *note + Implementing Descriptors: 4e9. + +dictionary + + An associative array, where arbitrary keys are mapped to values. + The keys can be any object with *note __hash__(): 340. and *note + __eq__(): 33f. methods. Called a hash in Perl. + +dictionary comprehension + + A compact way to process all or part of the elements in an iterable + and return a dictionary with the results. ‘results = {n: n ** 2 + for n in range(10)}’ generates a dictionary containing key ‘n’ + mapped to value ‘n ** 2’. See *note Displays for lists, sets and + dictionaries: 1191. + +dictionary view + + The objects returned from *note dict.keys(): c2a, *note + dict.values(): c2c, and *note dict.items(): c2b. are called + dictionary views. They provide a dynamic view on the dictionary’s + entries, which means that when the dictionary changes, the view + reflects these changes. To force the dictionary view to become a + full list use ‘list(dictview)’. See *note Dictionary view objects: + 1381. + +docstring + + A string literal which appears as the first expression in a class, + function or module. While ignored when the suite is executed, it + is recognized by the compiler and put into the ‘__doc__’ attribute + of the enclosing class, function or module. Since it is available + via introspection, it is the canonical place for documentation of + the object. + +duck-typing + + A programming style which does not look at an object’s type to + determine if it has the right interface; instead, the method or + attribute is simply called or used (“If it looks like a duck and + quacks like a duck, it must be a duck.”) By emphasizing interfaces + rather than specific types, well-designed code improves its + flexibility by allowing polymorphic substitution. Duck-typing + avoids tests using *note type(): 608. or *note isinstance(): 44f. + (Note, however, that duck-typing can be complemented with *note + abstract base classes: b33.) Instead, it typically employs *note + hasattr(): 447. tests or *note EAFP: 1c20. programming. + +EAFP + + Easier to ask for forgiveness than permission. This common Python + coding style assumes the existence of valid keys or attributes and + catches exceptions if the assumption proves false. This clean and + fast style is characterized by the presence of many *note try: d72. + and *note except: b3e. statements. The technique contrasts with + the *note LBYL: 4075. style common to many other languages such as + C. + +expression + + A piece of syntax which can be evaluated to some value. In other + words, an expression is an accumulation of expression elements like + literals, names, attribute access, operators or function calls + which all return a value. In contrast to many other languages, not + all language constructs are expressions. There are also *note + statement: 1847.s which cannot be used as expressions, such as + *note while: ec1. Assignments are also statements, not + expressions. + +extension module + + A module written in C or C++, using Python’s C API to interact with + the core and with user code. + +f-string + + String literals prefixed with ‘'f'’ or ‘'F'’ are commonly called + “f-strings” which is short for *note formatted string literals: + 15c. See also PEP 498(15). + +file object + + An object exposing a file-oriented API (with methods such as + ‘read()’ or ‘write()’) to an underlying resource. Depending on the + way it was created, a file object can mediate access to a real + on-disk file or to another type of storage or communication device + (for example standard input/output, in-memory buffers, sockets, + pipes, etc.). File objects are also called `file-like objects' or + `streams'. + + There are actually three categories of file objects: raw *note + binary files: 1966, buffered *note binary files: 1966. and *note + text files: f3a. Their interfaces are defined in the *note io: a1. + module. The canonical way to create a file object is by using the + *note open(): 4f0. function. + +file-like object + + A synonym for *note file object: b42. + +finder + + An object that tries to find the *note loader: 1160. for a module + that is being imported. + + Since Python 3.3, there are two types of finder: *note meta path + finders: 9b1. for use with *note sys.meta_path: 5e6, and *note path + entry finders: 9b2. for use with *note sys.path_hooks: 95c. + + See PEP 302(16), PEP 420(17) and PEP 451(18) for much more detail. + +floor division + + Mathematical division that rounds down to nearest integer. The + floor division operator is ‘//’. For example, the expression ‘11 + // 4’ evaluates to ‘2’ in contrast to the ‘2.75’ returned by float + true division. Note that ‘(-11) // 4’ is ‘-3’ because that is + ‘-2.75’ rounded `downward'. See PEP 238(19). + +function + + A series of statements which returns some value to a caller. It + can also be passed zero or more *note arguments: 11ca. which may be + used in the execution of the body. See also *note parameter: 11d2, + *note method: 406c, and the *note Function definitions: ef0. + section. + +function annotation + + An *note annotation: 406a. of a function parameter or return value. + + Function annotations are usually used for *note type hints: 406b.: + for example, this function is expected to take two *note int: 184. + arguments and is also expected to have an *note int: 184. return + value: + + def sum_two_numbers(a: int, b: int) -> int: + return a + b + + Function annotation syntax is explained in section *note Function + definitions: ef0. + + See *note variable annotation: 61f. and PEP 484(20), which describe + this functionality. + +__future__ + + A pseudo-module which programmers can use to enable new language + features which are not compatible with the current interpreter. + + By importing the *note __future__: 0. module and evaluating its + variables, you can see when a new feature was first added to the + language and when it becomes the default: + + >>> import __future__ + >>> __future__.division + _Feature((2, 2, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 8192) + +garbage collection + + The process of freeing memory when it is not used anymore. Python + performs garbage collection via reference counting and a cyclic + garbage collector that is able to detect and break reference + cycles. The garbage collector can be controlled using the *note + gc: 86. module. + +generator + + A function which returns a *note generator iterator: 457. It looks + like a normal function except that it contains *note yield: 18f. + expressions for producing a series of values usable in a for-loop + or that can be retrieved one at a time with the *note next(): 682. + function. + + Usually refers to a generator function, but may refer to a + `generator iterator' in some contexts. In cases where the intended + meaning isn’t clear, using the full terms avoids ambiguity. + +generator iterator + + An object created by a *note generator: 9a2. function. + + Each *note yield: 18f. temporarily suspends processing, remembering + the location execution state (including local variables and pending + try-statements). When the `generator iterator' resumes, it picks + up where it left off (in contrast to functions which start fresh on + every invocation). + +generator expression + + An expression that returns an iterator. It looks like a normal + expression followed by a ‘for’ clause defining a loop variable, + range, and an optional ‘if’ clause. The combined expression + generates values for an enclosing function: + + >>> sum(i*i for i in range(10)) # sum of squares 0, 1, 4, ... 81 + 285 + +generic function + + A function composed of multiple functions implementing the same + operation for different types. Which implementation should be used + during a call is determined by the dispatch algorithm. + + See also the *note single dispatch: 1cb. glossary entry, the *note + functools.singledispatch(): 38a. decorator, and PEP 443(21). + +GIL + + See *note global interpreter lock: 506. + +global interpreter lock + + The mechanism used by the *note CPython: 10d7. interpreter to + assure that only one thread executes Python *note bytecode: 614. at + a time. This simplifies the CPython implementation by making the + object model (including critical built-in types such as *note dict: + 1b8.) implicitly safe against concurrent access. Locking the + entire interpreter makes it easier for the interpreter to be + multi-threaded, at the expense of much of the parallelism afforded + by multi-processor machines. + + However, some extension modules, either standard or third-party, + are designed so as to release the GIL when doing + computationally-intensive tasks such as compression or hashing. + Also, the GIL is always released when doing I/O. + + Past efforts to create a “free-threaded” interpreter (one which + locks shared data at a much finer granularity) have not been + successful because performance suffered in the common + single-processor case. It is believed that overcoming this + performance issue would make the implementation much more + complicated and therefore costlier to maintain. + +hash-based pyc + + A bytecode cache file that uses the hash rather than the + last-modified time of the corresponding source file to determine + its validity. See *note Cached bytecode invalidation: 329. + +hashable + + An object is `hashable' if it has a hash value which never changes + during its lifetime (it needs a *note __hash__(): 340. method), and + can be compared to other objects (it needs an *note __eq__(): 33f. + method). Hashable objects which compare equal must have the same + hash value. + + Hashability makes an object usable as a dictionary key and a set + member, because these data structures use the hash value + internally. + + Most of Python’s immutable built-in objects are hashable; mutable + containers (such as lists or dictionaries) are not; immutable + containers (such as tuples and frozensets) are only hashable if + their elements are hashable. Objects which are instances of + user-defined classes are hashable by default. They all compare + unequal (except with themselves), and their hash value is derived + from their *note id(): d86. + +IDLE + + An Integrated Development Environment for Python. IDLE is a basic + editor and interpreter environment which ships with the standard + distribution of Python. + +immutable + + An object with a fixed value. Immutable objects include numbers, + strings and tuples. Such an object cannot be altered. A new + object has to be created if a different value has to be stored. + They play an important role in places where a constant hash value + is needed, for example as a key in a dictionary. + +import path + + A list of locations (or *note path entries: 1175.) that are + searched by the *note path based finder: 1165. for modules to + import. During import, this list of locations usually comes from + *note sys.path: 488, but for subpackages it may also come from the + parent package’s ‘__path__’ attribute. + +importing + + The process by which Python code in one module is made available to + Python code in another module. + +importer + + An object that both finds and loads a module; both a *note finder: + 115f. and *note loader: 1160. object. + +interactive + + Python has an interactive interpreter which means you can enter + statements and expressions at the interpreter prompt, immediately + execute them and see their results. Just launch ‘python’ with no + arguments (possibly by selecting it from your computer’s main + menu). It is a very powerful way to test out new ideas or inspect + modules and packages (remember ‘help(x)’). + +interpreted + + Python is an interpreted language, as opposed to a compiled one, + though the distinction can be blurry because of the presence of the + bytecode compiler. This means that source files can be run + directly without explicitly creating an executable which is then + run. Interpreted languages typically have a shorter + development/debug cycle than compiled ones, though their programs + generally also run more slowly. See also *note interactive: 407a. + +interpreter shutdown + + When asked to shut down, the Python interpreter enters a special + phase where it gradually releases all allocated resources, such as + modules and various critical internal structures. It also makes + several calls to the *note garbage collector: f9f. This can + trigger the execution of code in user-defined destructors or + weakref callbacks. Code executed during the shutdown phase can + encounter various exceptions as the resources it relies on may not + function anymore (common examples are library modules or the + warnings machinery). + + The main reason for interpreter shutdown is that the ‘__main__’ + module or the script being run has finished executing. + +iterable + + An object capable of returning its members one at a time. Examples + of iterables include all sequence types (such as *note list: 262, + *note str: 330, and *note tuple: 47e.) and some non-sequence types + like *note dict: 1b8, *note file objects: b42, and objects of any + classes you define with an *note __iter__(): 50f. method or with a + *note __getitem__(): 27c. method that implements *note Sequence: + ebc. semantics. + + Iterables can be used in a *note for: c30. loop and in many other + places where a sequence is needed (*note zip(): c32, *note map(): + c2d, …). When an iterable object is passed as an argument to the + built-in function *note iter(): d27, it returns an iterator for the + object. This iterator is good for one pass over the set of values. + When using iterables, it is usually not necessary to call *note + iter(): d27. or deal with iterator objects yourself. The ‘for’ + statement does that automatically for you, creating a temporary + unnamed variable to hold the iterator for the duration of the loop. + See also *note iterator: 112e, *note sequence: ebc, and *note + generator: 9a2. + +iterator + + An object representing a stream of data. Repeated calls to the + iterator’s *note __next__(): c64. method (or passing it to the + built-in function *note next(): 682.) return successive items in + the stream. When no more data are available a *note StopIteration: + 486. exception is raised instead. At this point, the iterator + object is exhausted and any further calls to its ‘__next__()’ + method just raise *note StopIteration: 486. again. Iterators are + required to have an *note __iter__(): 50f. method that returns the + iterator object itself so every iterator is also iterable and may + be used in most places where other iterables are accepted. One + notable exception is code which attempts multiple iteration passes. + A container object (such as a *note list: 262.) produces a fresh + new iterator each time you pass it to the *note iter(): d27. + function or use it in a *note for: c30. loop. Attempting this with + an iterator will just return the same exhausted iterator object + used in the previous iteration pass, making it appear like an empty + container. + + More information can be found in *note Iterator Types: 10fe. + +key function + + A key function or collation function is a callable that returns a + value used for sorting or ordering. For example, *note + locale.strxfrm(): 2d93. is used to produce a sort key that is aware + of locale specific sort conventions. + + A number of tools in Python accept key functions to control how + elements are ordered or grouped. They include *note min(): 809, + *note max(): 80a, *note sorted(): 444, *note list.sort(): 445, + *note heapq.merge(): 6df, *note heapq.nsmallest(): 1613, *note + heapq.nlargest(): 1612, and *note itertools.groupby(): 17f3. + + There are several ways to create a key function. For example. the + *note str.lower(): 12ee. method can serve as a key function for + case insensitive sorts. Alternatively, a key function can be built + from a *note lambda: c2f. expression such as ‘lambda r: (r[0], + r[2])’. Also, the *note operator: c2. module provides three key + function constructors: *note attrgetter(): 71b, *note itemgetter(): + 261, and *note methodcaller(): 71c. See the *note Sorting HOW TO: + 12ab. for examples of how to create and use key functions. + +keyword argument + + See *note argument: 11ca. + +lambda + + An anonymous inline function consisting of a single *note + expression: 3350. which is evaluated when the function is called. + The syntax to create a lambda function is ‘lambda [parameters]: + expression’ + +LBYL + + Look before you leap. This coding style explicitly tests for + pre-conditions before making calls or lookups. This style + contrasts with the *note EAFP: 1c20. approach and is characterized + by the presence of many *note if: de7. statements. + + In a multi-threaded environment, the LBYL approach can risk + introducing a race condition between “the looking” and “the + leaping”. For example, the code, ‘if key in mapping: return + mapping[key]’ can fail if another thread removes `key' from + `mapping' after the test, but before the lookup. This issue can be + solved with locks or by using the EAFP approach. + +list + + A built-in Python *note sequence: ebc. Despite its name it is more + akin to an array in other languages than to a linked list since + access to elements is O(1). + +list comprehension + + A compact way to process all or part of the elements in a sequence + and return a list with the results. ‘result = ['{:#04x}'.format(x) + for x in range(256) if x % 2 == 0]’ generates a list of strings + containing even hex numbers (0x..) in the range from 0 to 255. + The *note if: de7. clause is optional. If omitted, all elements in + ‘range(256)’ are processed. + +loader + + An object that loads a module. It must define a method named + ‘load_module()’. A loader is typically returned by a *note finder: + 115f. See PEP 302(22) for details and *note importlib.abc.Loader: + 7c2. for an *note abstract base class: b33. + +magic method + + An informal synonym for *note special method: 338b. + +mapping + + A container object that supports arbitrary key lookups and + implements the methods specified in the *note Mapping: 15f3. or + *note MutableMapping: 9ef. *note abstract base classes: 15d6. + Examples include *note dict: 1b8, *note collections.defaultdict: + b3a, *note collections.OrderedDict: 1b9. and *note + collections.Counter: 9da. + +meta path finder + + A *note finder: 115f. returned by a search of *note sys.meta_path: + 5e6. Meta path finders are related to, but different from *note + path entry finders: 9b2. + + See *note importlib.abc.MetaPathFinder: 9b3. for the methods that + meta path finders implement. + +metaclass + + The class of a class. Class definitions create a class name, a + class dictionary, and a list of base classes. The metaclass is + responsible for taking those three arguments and creating the + class. Most object oriented programming languages provide a + default implementation. What makes Python special is that it is + possible to create custom metaclasses. Most users never need this + tool, but when the need arises, metaclasses can provide powerful, + elegant solutions. They have been used for logging attribute + access, adding thread-safety, tracking object creation, + implementing singletons, and many other tasks. + + More information can be found in *note Metaclasses: 10ea. + +method + + A function which is defined inside a class body. If called as an + attribute of an instance of that class, the method will get the + instance object as its first *note argument: 11ca. (which is + usually called ‘self’). See *note function: 11c9. and *note nested + scope: 1295. + +method resolution order + + Method Resolution Order is the order in which base classes are + searched for a member during lookup. See The Python 2.3 Method + Resolution Order(23) for details of the algorithm used by the + Python interpreter since the 2.3 release. + +module + + An object that serves as an organizational unit of Python code. + Modules have a namespace containing arbitrary Python objects. + Modules are loaded into Python by the process of *note importing: + 1151. + + See also *note package: 1155. + +module spec + + A namespace containing the import-related information used to load + a module. An instance of *note importlib.machinery.ModuleSpec: + 116b. + +MRO + + See *note method resolution order: 12ae. + +mutable + + Mutable objects can change their value but keep their *note id(): + d86. See also *note immutable: eb6. + +named tuple + + The term “named tuple” applies to any type or class that inherits + from tuple and whose indexable elements are also accessible using + named attributes. The type or class may have other features as + well. + + Several built-in types are named tuples, including the values + returned by *note time.localtime(): 155c. and *note os.stat(): 1f1. + Another example is *note sys.float_info: 12c6.: + + >>> sys.float_info[1] # indexed access + 1024 + >>> sys.float_info.max_exp # named field access + 1024 + >>> isinstance(sys.float_info, tuple) # kind of tuple + True + + Some named tuples are built-in types (such as the above examples). + Alternatively, a named tuple can be created from a regular class + definition that inherits from *note tuple: 47e. and that defines + named fields. Such a class can be written by hand or it can be + created with the factory function *note collections.namedtuple(): + 1b7. The latter technique also adds some extra methods that may + not be found in hand-written or built-in named tuples. + +namespace + + The place where a variable is stored. Namespaces are implemented + as dictionaries. There are the local, global and built-in + namespaces as well as nested namespaces in objects (in methods). + Namespaces support modularity by preventing naming conflicts. For + instance, the functions *note builtins.open: 4f0. and *note + os.open(): 665. are distinguished by their namespaces. Namespaces + also aid readability and maintainability by making it clear which + module implements a function. For instance, writing *note + random.seed(): c08. or *note itertools.islice(): 3a2. makes it + clear that those functions are implemented by the *note random: dc. + and *note itertools: a3. modules, respectively. + +namespace package + + A PEP 420(24) *note package: 1155. which serves only as a container + for subpackages. Namespace packages may have no physical + representation, and specifically are not like a *note regular + package: 1157. because they have no ‘__init__.py’ file. + + See also *note module: 1150. + +nested scope + + The ability to refer to a variable in an enclosing definition. For + instance, a function defined inside another function can refer to + variables in the outer function. Note that nested scopes by + default work only for reference and not for assignment. Local + variables both read and write in the innermost scope. Likewise, + global variables read and write to the global namespace. The *note + nonlocal: c3f. allows writing to outer scopes. + +new-style class + + Old name for the flavor of classes now used for all class objects. + In earlier Python versions, only new-style classes could use + Python’s newer, versatile features like *note __slots__: e2d, + descriptors, properties, *note __getattribute__(): 449, class + methods, and static methods. + +object + + Any data with state (attributes or value) and defined behavior + (methods). Also the ultimate base class of any *note new-style + class: 196e. + +package + + A Python *note module: 1150. which can contain submodules or + recursively, subpackages. Technically, a package is a Python + module with an ‘__path__’ attribute. + + See also *note regular package: 1157. and *note namespace package: + 1158. + +parameter + + A named entity in a *note function: 11c9. (or method) definition + that specifies an *note argument: 11ca. (or in some cases, + arguments) that the function can accept. There are five kinds of + parameter: + + * `positional-or-keyword': specifies an argument that can be + passed either *note positionally: 11ca. or as a *note keyword + argument: 11ca. This is the default kind of parameter, for + example `foo' and `bar' in the following: + + def func(foo, bar=None): ... + + * `positional-only': specifies an argument that can be supplied + only by position. Positional-only parameters can be defined + by including a ‘/’ character in the parameter list of the + function definition after them, for example `posonly1' and + `posonly2' in the following: + + def func(posonly1, posonly2, /, positional_or_keyword): ... + + * `keyword-only': specifies an argument that can be supplied + only by keyword. Keyword-only parameters can be defined by + including a single var-positional parameter or bare ‘*’ in the + parameter list of the function definition before them, for + example `kw_only1' and `kw_only2' in the following: + + def func(arg, *, kw_only1, kw_only2): ... + + * `var-positional': specifies that an arbitrary sequence of + positional arguments can be provided (in addition to any + positional arguments already accepted by other parameters). + Such a parameter can be defined by prepending the parameter + name with ‘*’, for example `args' in the following: + + def func(*args, **kwargs): ... + + * `var-keyword': specifies that arbitrarily many keyword + arguments can be provided (in addition to any keyword + arguments already accepted by other parameters). Such a + parameter can be defined by prepending the parameter name with + ‘**’, for example `kwargs' in the example above. + + Parameters can specify both optional and required arguments, as + well as default values for some optional arguments. + + See also the *note argument: 11ca. glossary entry, the FAQ question + on *note the difference between arguments and parameters: 3fb4, the + *note inspect.Parameter: 6f4. class, the *note Function + definitions: ef0. section, and PEP 362(25). + +path entry + + A single location on the *note import path: 1162. which the *note + path based finder: 1165. consults to find modules for importing. + +path entry finder + + A *note finder: 115f. returned by a callable on *note + sys.path_hooks: 95c. (i.e. a *note path entry hook: 1177.) which + knows how to locate modules given a *note path entry: 1175. + + See *note importlib.abc.PathEntryFinder: 9b4. for the methods that + path entry finders implement. + +path entry hook + + A callable on the ‘sys.path_hook’ list which returns a *note path + entry finder: 9b2. if it knows how to find modules on a specific + *note path entry: 1175. + +path based finder + + One of the default *note meta path finders: 9b1. which searches an + *note import path: 1162. for modules. + +path-like object + + An object representing a file system path. A path-like object is + either a *note str: 330. or *note bytes: 331. object representing a + path, or an object implementing the *note os.PathLike: 4eb. + protocol. An object that supports the *note os.PathLike: 4eb. + protocol can be converted to a *note str: 330. or *note bytes: 331. + file system path by calling the *note os.fspath(): 4ed. function; + *note os.fsdecode(): 4ee. and *note os.fsencode(): 4ef. can be used + to guarantee a *note str: 330. or *note bytes: 331. result instead, + respectively. Introduced by PEP 519(26). + +PEP + + Python Enhancement Proposal. A PEP is a design document providing + information to the Python community, or describing a new feature + for Python or its processes or environment. PEPs should provide a + concise technical specification and a rationale for proposed + features. + + PEPs are intended to be the primary mechanisms for proposing major + new features, for collecting community input on an issue, and for + documenting the design decisions that have gone into Python. The + PEP author is responsible for building consensus within the + community and documenting dissenting opinions. + + See PEP 1(27). + +portion + + A set of files in a single directory (possibly stored in a zip + file) that contribute to a namespace package, as defined in PEP + 420(28). + +positional argument + + See *note argument: 11ca. + +provisional API + + A provisional API is one which has been deliberately excluded from + the standard library’s backwards compatibility guarantees. While + major changes to such interfaces are not expected, as long as they + are marked provisional, backwards incompatible changes (up to and + including removal of the interface) may occur if deemed necessary + by core developers. Such changes will not be made gratuitously – + they will occur only if serious fundamental flaws are uncovered + that were missed prior to the inclusion of the API. + + Even for provisional APIs, backwards incompatible changes are seen + as a “solution of last resort” - every attempt will still be made + to find a backwards compatible resolution to any identified + problems. + + This process allows the standard library to continue to evolve over + time, without locking in problematic design errors for extended + periods of time. See PEP 411(29) for more details. + +provisional package + + See *note provisional API: 348. + +Python 3000 + + Nickname for the Python 3.x release line (coined long ago when the + release of version 3 was something in the distant future.) This is + also abbreviated “Py3k”. + +Pythonic + + An idea or piece of code which closely follows the most common + idioms of the Python language, rather than implementing code using + concepts common to other languages. For example, a common idiom in + Python is to loop over all elements of an iterable using a *note + for: c30. statement. Many other languages don’t have this type of + construct, so people unfamiliar with Python sometimes use a + numerical counter instead: + + for i in range(len(food)): + print(food[i]) + + As opposed to the cleaner, Pythonic method: + + for piece in food: + print(piece) + +qualified name + + A dotted name showing the “path” from a module’s global scope to a + class, function or method defined in that module, as defined in PEP + 3155(30). For top-level functions and classes, the qualified name + is the same as the object’s name: + + >>> class C: + ... class D: + ... def meth(self): + ... pass + ... + >>> C.__qualname__ + 'C' + >>> C.D.__qualname__ + 'C.D' + >>> C.D.meth.__qualname__ + 'C.D.meth' + + When used to refer to modules, the `fully qualified name' means the + entire dotted path to the module, including any parent packages, + e.g. ‘email.mime.text’: + + >>> import email.mime.text + >>> email.mime.text.__name__ + 'email.mime.text' + +reference count + + The number of references to an object. When the reference count of + an object drops to zero, it is deallocated. Reference counting is + generally not visible to Python code, but it is a key element of + the *note CPython: 10d7. implementation. The *note sys: fd. module + defines a *note getrefcount(): 335a. function that programmers can + call to return the reference count for a particular object. + +regular package + + A traditional *note package: 1155, such as a directory containing + an ‘__init__.py’ file. + + See also *note namespace package: 1158. + +__slots__ + + A declaration inside a class that saves memory by pre-declaring + space for instance attributes and eliminating instance + dictionaries. Though popular, the technique is somewhat tricky to + get right and is best reserved for rare cases where there are large + numbers of instances in a memory-critical application. + +sequence + + An *note iterable: bac. which supports efficient element access + using integer indices via the *note __getitem__(): 27c. special + method and defines a *note __len__(): dcb. method that returns the + length of the sequence. Some built-in sequence types are *note + list: 262, *note str: 330, *note tuple: 47e, and *note bytes: 331. + Note that *note dict: 1b8. also supports *note __getitem__(): 27c. + and *note __len__(): dcb, but is considered a mapping rather than a + sequence because the lookups use arbitrary *note immutable: eb6. + keys rather than integers. + + The *note collections.abc.Sequence: 12db. abstract base class + defines a much richer interface that goes beyond just *note + __getitem__(): 27c. and *note __len__(): dcb, adding ‘count()’, + ‘index()’, *note __contains__(): d28, and *note __reversed__(): + 52f. Types that implement this expanded interface can be + registered explicitly using *note register(): 9d1. + +set comprehension + + A compact way to process all or part of the elements in an iterable + and return a set with the results. ‘results = {c for c in + 'abracadabra' if c not in 'abc'}’ generates the set of strings + ‘{'r', 'd'}’. See *note Displays for lists, sets and dictionaries: + 1191. + +single dispatch + + A form of *note generic function: 1ca. dispatch where the + implementation is chosen based on the type of a single argument. + +slice + + An object usually containing a portion of a *note sequence: ebc. A + slice is created using the subscript notation, ‘[]’ with colons + between numbers when several are given, such as in + ‘variable_name[1:3:5]’. The bracket (subscript) notation uses + *note slice: e04. objects internally. + +special method + + A method that is called implicitly by Python to execute a certain + operation on a type, such as addition. Such methods have names + starting and ending with double underscores. Special methods are + documented in *note Special method names: 50e. + +statement + + A statement is part of a suite (a “block” of code). A statement is + either an *note expression: 3350. or one of several constructs with + a keyword, such as *note if: de7, *note while: ec1. or *note for: + c30. + +text encoding + + A codec which encodes Unicode strings to bytes. + +text file + + A *note file object: b42. able to read and write *note str: 330. + objects. Often, a text file actually accesses a byte-oriented + datastream and handles the *note text encoding: 12a5. + automatically. Examples of text files are files opened in text + mode (‘'r'’ or ‘'w'’), *note sys.stdin: 30d, *note sys.stdout: 30e, + and instances of *note io.StringIO: 82d. + + See also *note binary file: 1966. for a file object able to read + and write *note bytes-like objects: 5e8. + +triple-quoted string + + A string which is bound by three instances of either a quotation + mark (”) or an apostrophe (‘). While they don’t provide any + functionality not available with single-quoted strings, they are + useful for a number of reasons. They allow you to include + unescaped single and double quotes within a string and they can + span multiple lines without the use of the continuation character, + making them especially useful when writing docstrings. + +type + + The type of a Python object determines what kind of object it is; + every object has a type. An object’s type is accessible as its + *note __class__: e0a. attribute or can be retrieved with + ‘type(obj)’. + +type alias + + A synonym for a type, created by assigning the type to an + identifier. + + Type aliases are useful for simplifying *note type hints: 406b. + For example: + + from typing import List, Tuple + + def remove_gray_shades( + colors: List[Tuple[int, int, int]]) -> List[Tuple[int, int, int]]: + pass + + could be made more readable like this: + + from typing import List, Tuple + + Color = Tuple[int, int, int] + + def remove_gray_shades(colors: List[Color]) -> List[Color]: + pass + + See *note typing: 119. and PEP 484(31), which describe this + functionality. + +type hint + + An *note annotation: 406a. that specifies the expected type for a + variable, a class attribute, or a function parameter or return + value. + + Type hints are optional and are not enforced by Python but they are + useful to static type analysis tools, and aid IDEs with code + completion and refactoring. + + Type hints of global variables, class attributes, and functions, + but not local variables, can be accessed using *note + typing.get_type_hints(): 30a. + + See *note typing: 119. and PEP 484(32), which describe this + functionality. + +universal newlines + + A manner of interpreting text streams in which all of the following + are recognized as ending a line: the Unix end-of-line convention + ‘'\n'’, the Windows convention ‘'\r\n'’, and the old Macintosh + convention ‘'\r'’. See PEP 278(33) and PEP 3116(34), as well as + *note bytes.splitlines(): 134e. for an additional use. + +variable annotation + + An *note annotation: 406a. of a variable or a class attribute. + + When annotating a variable or a class attribute, assignment is + optional: + + class C: + field: 'annotation' + + Variable annotations are usually used for *note type hints: 406b.: + for example this variable is expected to take *note int: 184. + values: + + count: int = 0 + + Variable annotation syntax is explained in section *note Annotated + assignment statements: 121e. + + See *note function annotation: ef1, PEP 484(35) and PEP 526(36), + which describe this functionality. + +virtual environment + + A cooperatively isolated runtime environment that allows Python + users and applications to install and upgrade Python distribution + packages without interfering with the behaviour of other Python + applications running on the same system. + + See also *note venv: 125. + +virtual machine + + A computer defined entirely in software. Python’s virtual machine + executes the *note bytecode: 614. emitted by the bytecode compiler. + +Zen of Python + + Listing of Python design principles and philosophies that are + helpful in understanding and using the language. The listing can + be found by typing “‘import this’” at the interactive prompt. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0484 + + (2) https://www.python.org/dev/peps/pep-0526 + + (3) https://www.python.org/dev/peps/pep-0362 + + (4) https://www.python.org/dev/peps/pep-0492 + + (5) https://www.python.org/dev/peps/pep-0492 + + (6) https://www.python.org/dev/peps/pep-0525 + + (7) https://www.python.org/dev/peps/pep-0492 + + (8) https://www.python.org/dev/peps/pep-0492 + + (9) https://www.python.org/dev/peps/pep-0492 + + (10) https://gvanrossum.github.io/ + + (11) https://www.python.org/dev/peps/pep-0343 + + (12) https://www.python.org/dev/peps/pep-0492 + + (13) https://www.python.org/dev/peps/pep-0492 + + (14) https://www.python.org + + (15) https://www.python.org/dev/peps/pep-0498 + + (16) https://www.python.org/dev/peps/pep-0302 + + (17) https://www.python.org/dev/peps/pep-0420 + + (18) https://www.python.org/dev/peps/pep-0451 + + (19) https://www.python.org/dev/peps/pep-0238 + + (20) https://www.python.org/dev/peps/pep-0484 + + (21) https://www.python.org/dev/peps/pep-0443 + + (22) https://www.python.org/dev/peps/pep-0302 + + (23) https://www.python.org/download/releases/2.3/mro/ + + (24) https://www.python.org/dev/peps/pep-0420 + + (25) https://www.python.org/dev/peps/pep-0362 + + (26) https://www.python.org/dev/peps/pep-0519 + + (27) https://www.python.org/dev/peps/pep-0001 + + (28) https://www.python.org/dev/peps/pep-0420 + + (29) https://www.python.org/dev/peps/pep-0411 + + (30) https://www.python.org/dev/peps/pep-3155 + + (31) https://www.python.org/dev/peps/pep-0484 + + (32) https://www.python.org/dev/peps/pep-0484 + + (33) https://www.python.org/dev/peps/pep-0278 + + (34) https://www.python.org/dev/peps/pep-3116 + + (35) https://www.python.org/dev/peps/pep-0484 + + (36) https://www.python.org/dev/peps/pep-0526 + + +File: python.info, Node: About these documents, Next: Dealing with Bugs, Prev: Glossary, Up: Top + +13 About these documents +************************ + +These documents are generated from reStructuredText(1) sources by +Sphinx(2), a document processor specifically written for the Python +documentation. + +Development of the documentation and its toolchain is an entirely +volunteer effort, just like Python itself. If you want to contribute, +please take a look at the *note Dealing with Bugs: 38d7. page for +information on how to do so. New volunteers are always welcome! + +Many thanks go to: + + * Fred L. Drake, Jr., the creator of the original Python + documentation toolset and writer of much of the content; + + * the Docutils(3) project for creating reStructuredText and the + Docutils suite; + + * Fredrik Lundh for his Alternative Python Reference(4) project from + which Sphinx got many good ideas. + +* Menu: + +* Contributors to the Python Documentation:: + + ---------- Footnotes ---------- + + (1) http://docutils.sourceforge.net/rst.html + + (2) http://sphinx-doc.org/ + + (3) http://docutils.sourceforge.net/ + + (4) http://effbot.org/zone/pyref.htm + + +File: python.info, Node: Contributors to the Python Documentation, Up: About these documents + +13.1 Contributors to the Python Documentation +============================================= + +Many people have contributed to the Python language, the Python standard +library, and the Python documentation. See Misc/ACKS(1) in the Python +source distribution for a partial list of contributors. + +It is only with the input and contributions of the Python community that +Python has such wonderful documentation – Thank You! + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/tree/3.8/Misc/ACKS + + +File: python.info, Node: Dealing with Bugs, Next: Copyright, Prev: About these documents, Up: Top + +14 Dealing with Bugs +******************** + +Python is a mature programming language which has established a +reputation for stability. In order to maintain this reputation, the +developers would like to know of any deficiencies you find in Python. + +It can be sometimes faster to fix bugs yourself and contribute patches +to Python as it streamlines the process and involves less people. Learn +how to *note contribute: 408e. + +* Menu: + +* Documentation bugs:: +* Using the Python issue tracker:: +* Getting started contributing to Python yourself:: + + +File: python.info, Node: Documentation bugs, Next: Using the Python issue tracker, Up: Dealing with Bugs + +14.1 Documentation bugs +======================= + +If you find a bug in this documentation or would like to propose an +improvement, please submit a bug report on the *note tracker: 4090. If +you have a suggestion on how to fix it, include that as well. + +If you’re short on time, you can also email documentation bug reports to + (behavioral bugs can be sent to +). ‘docs@’ is a mailing list run by volunteers; +your request will be noticed, though it may take a while to be +processed. + +See also +........ + +Documentation bugs(1) + + A list of documentation bugs that have been submitted to the Python + issue tracker. + +Issue Tracking(2) + + Overview of the process involved in reporting an improvement on the + tracker. + +Helping with Documentation(3) + + Comprehensive guide for individuals that are interested in + contributing to Python documentation. + + ---------- Footnotes ---------- + + (1) +https://bugs.python.org/issue?@filter=status&@filter=components&components=4&status=1&@columns=id,activity,title,status&@sort=-activity + + (2) https://devguide.python.org/tracker/ + + (3) +https://devguide.python.org/docquality/#helping-with-documentation + + +File: python.info, Node: Using the Python issue tracker, Next: Getting started contributing to Python yourself, Prev: Documentation bugs, Up: Dealing with Bugs + +14.2 Using the Python issue tracker +=================================== + +Bug reports for Python itself should be submitted via the Python Bug +Tracker (‘https://bugs.python.org/’). The bug tracker offers a Web form +which allows pertinent information to be entered and submitted to the +developers. + +The first step in filing a report is to determine whether the problem +has already been reported. The advantage in doing so, aside from saving +the developers time, is that you learn what has been done to fix it; it +may be that the problem has already been fixed for the next release, or +additional information is needed (in which case you are welcome to +provide it if you can!). To do this, search the bug database using the +search box on the top of the page. + +If the problem you’re reporting is not already in the bug tracker, go +back to the Python Bug Tracker and log in. If you don’t already have a +tracker account, select the “Register” link or, if you use OpenID, one +of the OpenID provider logos in the sidebar. It is not possible to +submit a bug report anonymously. + +Being now logged in, you can submit a bug. Select the “Create New” link +in the sidebar to open the bug reporting form. + +The submission form has a number of fields. For the “Title” field, +enter a `very' short description of the problem; less than ten words is +good. In the “Type” field, select the type of your problem; also select +the “Component” and “Versions” to which the bug relates. + +In the “Comment” field, describe the problem in detail, including what +you expected to happen and what did happen. Be sure to include whether +any extension modules were involved, and what hardware and software +platform you were using (including version information as appropriate). + +Each bug report will be assigned to a developer who will determine what +needs to be done to correct the problem. You will receive an update +each time action is taken on the bug. + +See also +........ + +How to Report Bugs Effectively(1) + + Article which goes into some detail about how to create a useful + bug report. This describes what kind of information is useful and + why it is useful. + +Bug Report Writing Guidelines(2) + + Information about writing a good bug report. Some of this is + specific to the Mozilla project, but describes general good + practices. + + ---------- Footnotes ---------- + + (1) https://www.chiark.greenend.org.uk/~sgtatham/bugs.html + + (2) +https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writing_guidelines + + +File: python.info, Node: Getting started contributing to Python yourself, Prev: Using the Python issue tracker, Up: Dealing with Bugs + +14.3 Getting started contributing to Python yourself +==================================================== + +Beyond just reporting bugs that you find, you are also welcome to submit +patches to fix them. You can find more information on how to get +started patching Python in the Python Developer’s Guide(1). If you have +questions, the core-mentorship mailing list(2) is a friendly place to +get answers to any and all questions pertaining to the process of fixing +issues in Python. + + ---------- Footnotes ---------- + + (1) https://devguide.python.org/ + + (2) +https://mail.python.org/mailman3/lists/core-mentorship.python.org/ + + +File: python.info, Node: Copyright, Next: History and License, Prev: Dealing with Bugs, Up: Top + +15 Copyright +************ + +Python and this documentation is: + +Copyright © 2001-2021 Python Software Foundation. All rights reserved. + +Copyright © 2000 BeOpen.com. All rights reserved. + +Copyright © 1995-2000 Corporation for National Research Initiatives. +All rights reserved. + +Copyright © 1991-1995 Stichting Mathematisch Centrum. All rights +reserved. + +__________________________________________________________________ + +See *note History and License: 4096. for complete license and +permissions information. + + +File: python.info, Node: History and License, Next: Distributing Python Modules Legacy version, Prev: Copyright, Up: Top + +16 History and License +********************** + +* Menu: + +* History of the software:: +* Terms and conditions for accessing or otherwise using Python:: +* Licenses and Acknowledgements for Incorporated Software:: + + +File: python.info, Node: History of the software, Next: Terms and conditions for accessing or otherwise using Python, Up: History and License + +16.1 History of the software +============================ + +Python was created in the early 1990s by Guido van Rossum at Stichting +Mathematisch Centrum (CWI, see ‘https://www.cwi.nl/’) in the Netherlands +as a successor of a language called ABC. Guido remains Python’s +principal author, although it includes many contributions from others. + +In 1995, Guido continued his work on Python at the Corporation for +National Research Initiatives (CNRI, see +‘https://www.cnri.reston.va.us/’) in Reston, Virginia where he released +several versions of the software. + +In May 2000, Guido and the Python core development team moved to +BeOpen.com to form the BeOpen PythonLabs team. In October of the same +year, the PythonLabs team moved to Digital Creations (now Zope +Corporation; see ‘https://www.zope.org/’). In 2001, the Python Software +Foundation (PSF, see ‘https://www.python.org/psf/’) was formed, a +non-profit organization created specifically to own Python-related +Intellectual Property. Zope Corporation is a sponsoring member of the +PSF. + +All Python releases are Open Source (see ‘https://opensource.org/’ for +the Open Source Definition). Historically, most, but not all, Python +releases have also been GPL-compatible; the table below summarizes the +various releases. + +Release Derived from Year Owner GPL compatible? + +------------------------------------------------------------------------------------------------ + +0.9.0 thru 1.2 n/a 1991-1995 CWI yes + + +1.3 thru 1.5.2 1.2 1995-1999 CNRI yes + + +1.6 1.5.2 2000 CNRI no + + +2.0 1.6 2000 BeOpen.com no + + +1.6.1 1.6 2001 CNRI no + + +2.1 2.0+1.6.1 2001 PSF no + + +2.0.1 2.0+1.6.1 2001 PSF yes + + +2.1.1 2.1+2.0.1 2001 PSF yes + + +2.1.2 2.1.1 2002 PSF yes + + +2.1.3 2.1.2 2002 PSF yes + + +2.2 and above 2.1.1 2001-now PSF yes + + + Note: GPL-compatible doesn’t mean that we’re distributing Python + under the GPL. All Python licenses, unlike the GPL, let you + distribute a modified version without making your changes open + source. The GPL-compatible licenses make it possible to combine + Python with other software that is released under the GPL; the + others don’t. + +Thanks to the many outside volunteers who have worked under Guido’s +direction to make these releases possible. + + +File: python.info, Node: Terms and conditions for accessing or otherwise using Python, Next: Licenses and Acknowledgements for Incorporated Software, Prev: History of the software, Up: History and License + +16.2 Terms and conditions for accessing or otherwise using Python +================================================================= + +Python software and documentation are licensed under the *note PSF +License Agreement: 409b. + +Starting with Python 3.8.6, examples, recipes, and other code in the +documentation are dual licensed under the PSF License Agreement and the +*note Zero-Clause BSD license: 409c. + +Some software incorporated into Python is under different licenses. The +licenses are listed with code falling under that license. See *note +Licenses and Acknowledgements for Incorporated Software: 409d. for an +incomplete list of these licenses. + +* Menu: + +* PSF LICENSE AGREEMENT FOR PYTHON 3.8.9: PSF LICENSE AGREEMENT FOR PYTHON 3 8 9. +* BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0: BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 0. +* CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1: CNRI LICENSE AGREEMENT FOR PYTHON 1 6 1. +* CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2: CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 2. +* ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3.8.9 DOCUMENTATION: ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3 8 9 DOCUMENTATION. + + +File: python.info, Node: PSF LICENSE AGREEMENT FOR PYTHON 3 8 9, Next: BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 0, Up: Terms and conditions for accessing or otherwise using Python + +16.2.1 PSF LICENSE AGREEMENT FOR PYTHON 3.8.9 +--------------------------------------------- + + 1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and + the Individual or Organization ("Licensee") accessing and otherwise using Python + 3.8.9 software in source or binary form and its associated documentation. + + 2. Subject to the terms and conditions of this License Agreement, PSF hereby + grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, + analyze, test, perform and/or display publicly, prepare derivative works, + distribute, and otherwise use Python 3.8.9 alone or in any derivative + version, provided, however, that PSF's License Agreement and PSF's notice of + copyright, i.e., "Copyright © 2001-2021 Python Software Foundation; All Rights + Reserved" are retained in Python 3.8.9 alone or in any derivative version + prepared by Licensee. + + 3. In the event Licensee prepares a derivative work that is based on or + incorporates Python 3.8.9 or any part thereof, and wants to make the + derivative work available to others as provided herein, then Licensee hereby + agrees to include in any such work a brief summary of the changes made to Python + 3.8.9. + + 4. PSF is making Python 3.8.9 available to Licensee on an "AS IS" basis. + PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF + EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY REPRESENTATION OR + WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE + USE OF PYTHON 3.8.9 WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. + + 5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON 3.8.9 + FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF + MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 3.8.9, OR ANY DERIVATIVE + THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. + + 6. This License Agreement will automatically terminate upon a material breach of + its terms and conditions. + + 7. Nothing in this License Agreement shall be deemed to create any relationship + of agency, partnership, or joint venture between PSF and Licensee. This License + Agreement does not grant permission to use PSF trademarks or trade name in a + trademark sense to endorse or promote products or services of Licensee, or any + third party. + + 8. By copying, installing or otherwise using Python 3.8.9, Licensee agrees + to be bound by the terms and conditions of this License Agreement. + + +File: python.info, Node: BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 0, Next: CNRI LICENSE AGREEMENT FOR PYTHON 1 6 1, Prev: PSF LICENSE AGREEMENT FOR PYTHON 3 8 9, Up: Terms and conditions for accessing or otherwise using Python + +16.2.2 BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0 +-------------------------------------------------- + +BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1 + + 1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an office at + 160 Saratoga Avenue, Santa Clara, CA 95051, and the Individual or Organization + ("Licensee") accessing and otherwise using this software in source or binary + form and its associated documentation ("the Software"). + + 2. Subject to the terms and conditions of this BeOpen Python License Agreement, + BeOpen hereby grants Licensee a non-exclusive, royalty-free, world-wide license + to reproduce, analyze, test, perform and/or display publicly, prepare derivative + works, distribute, and otherwise use the Software alone or in any derivative + version, provided, however, that the BeOpen Python License is retained in the + Software, alone or in any derivative version prepared by Licensee. + + 3. BeOpen is making the Software available to Licensee on an "AS IS" basis. + BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF + EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND DISCLAIMS ANY REPRESENTATION OR + WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE + USE OF THE SOFTWARE WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. + + 4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE SOFTWARE FOR + ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF USING, + MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY DERIVATIVE THEREOF, EVEN IF + ADVISED OF THE POSSIBILITY THEREOF. + + 5. This License Agreement will automatically terminate upon a material breach of + its terms and conditions. + + 6. This License Agreement shall be governed by and interpreted in all respects + by the law of the State of California, excluding conflict of law provisions. + Nothing in this License Agreement shall be deemed to create any relationship of + agency, partnership, or joint venture between BeOpen and Licensee. This License + Agreement does not grant permission to use BeOpen trademarks or trade names in a + trademark sense to endorse or promote products or services of Licensee, or any + third party. As an exception, the "BeOpen Python" logos available at + ‘http://www.pythonlabs.com/logos.html’ may be used according to the permissions + granted on that web page. + + 7. By copying, installing or otherwise using the software, Licensee agrees to be + bound by the terms and conditions of this License Agreement. + + +File: python.info, Node: CNRI LICENSE AGREEMENT FOR PYTHON 1 6 1, Next: CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 2, Prev: BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 0, Up: Terms and conditions for accessing or otherwise using Python + +16.2.3 CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1 +---------------------------------------------- + + 1. This LICENSE AGREEMENT is between the Corporation for National Research + Initiatives, having an office at 1895 Preston White Drive, Reston, VA 20191 + ("CNRI"), and the Individual or Organization ("Licensee") accessing and + otherwise using Python 1.6.1 software in source or binary form and its + associated documentation. + + 2. Subject to the terms and conditions of this License Agreement, CNRI hereby + grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, + analyze, test, perform and/or display publicly, prepare derivative works, + distribute, and otherwise use Python 1.6.1 alone or in any derivative version, + provided, however, that CNRI's License Agreement and CNRI's notice of copyright, + i.e., "Copyright © 1995-2001 Corporation for National Research Initiatives; All + Rights Reserved" are retained in Python 1.6.1 alone or in any derivative version + prepared by Licensee. Alternately, in lieu of CNRI's License Agreement, + Licensee may substitute the following text (omitting the quotes): "Python 1.6.1 + is made available subject to the terms and conditions in CNRI's License + Agreement. This Agreement together with Python 1.6.1 may be located on the + Internet using the following unique, persistent identifier (known as a handle): + 1895.22/1013. This Agreement may also be obtained from a proxy server on the + Internet using the following URL: ‘http://hdl.handle.net/1895.22/1013’." + + 3. In the event Licensee prepares a derivative work that is based on or + incorporates Python 1.6.1 or any part thereof, and wants to make the derivative + work available to others as provided herein, then Licensee hereby agrees to + include in any such work a brief summary of the changes made to Python 1.6.1. + + 4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS" basis. CNRI + MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, + BUT NOT LIMITATION, CNRI MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY + OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF + PYTHON 1.6.1 WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. + + 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON 1.6.1 FOR + ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF + MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1, OR ANY DERIVATIVE + THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. + + 6. This License Agreement will automatically terminate upon a material breach of + its terms and conditions. + + 7. This License Agreement shall be governed by the federal intellectual property + law of the United States, including without limitation the federal copyright + law, and, to the extent such U.S. federal law does not apply, by the law of the + Commonwealth of Virginia, excluding Virginia's conflict of law provisions. + Notwithstanding the foregoing, with regard to derivative works based on Python + 1.6.1 that incorporate non-separable material that was previously distributed + under the GNU General Public License (GPL), the law of the Commonwealth of + Virginia shall govern this License Agreement only as to issues arising under or + with respect to Paragraphs 4, 5, and 7 of this License Agreement. Nothing in + this License Agreement shall be deemed to create any relationship of agency, + partnership, or joint venture between CNRI and Licensee. This License Agreement + does not grant permission to use CNRI trademarks or trade name in a trademark + sense to endorse or promote products or services of Licensee, or any third + party. + + 8. By clicking on the "ACCEPT" button where indicated, or by copying, installing + or otherwise using Python 1.6.1, Licensee agrees to be bound by the terms and + conditions of this License Agreement. + + +File: python.info, Node: CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 2, Next: ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3 8 9 DOCUMENTATION, Prev: CNRI LICENSE AGREEMENT FOR PYTHON 1 6 1, Up: Terms and conditions for accessing or otherwise using Python + +16.2.4 CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2 +--------------------------------------------------------- + + Copyright © 1991 - 1995, Stichting Mathematisch Centrum Amsterdam, The + Netherlands. All rights reserved. + + Permission to use, copy, modify, and distribute this software and its + documentation for any purpose and without fee is hereby granted, provided that + the above copyright notice appear in all copies and that both that copyright + notice and this permission notice appear in supporting documentation, and that + the name of Stichting Mathematisch Centrum or CWI not be used in advertising or + publicity pertaining to distribution of the software without specific, written + prior permission. + + STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS + SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO + EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT + OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, + DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS + ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS + SOFTWARE. + + +File: python.info, Node: ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3 8 9 DOCUMENTATION, Prev: CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 2, Up: Terms and conditions for accessing or otherwise using Python + +16.2.5 ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3.8.9 DOCUMENTATION +------------------------------------------------------------------------- + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH + REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY + AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, + INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM + LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR + OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR + PERFORMANCE OF THIS SOFTWARE. + + +File: python.info, Node: Licenses and Acknowledgements for Incorporated Software, Prev: Terms and conditions for accessing or otherwise using Python, Up: History and License + +16.3 Licenses and Acknowledgements for Incorporated Software +============================================================ + +This section is an incomplete, but growing list of licenses and +acknowledgements for third-party software incorporated in the Python +distribution. + +* Menu: + +* Mersenne Twister:: +* Sockets: Sockets<2>. +* Asynchronous socket services:: +* Cookie management:: +* Execution tracing:: +* UUencode and UUdecode functions:: +* XML Remote Procedure Calls:: +* test_epoll:: +* Select kqueue:: +* SipHash24:: +* strtod and dtoa:: +* OpenSSL:: +* expat:: +* libffi:: +* zlib: zlib<3>. +* cfuhash:: +* libmpdec:: +* W3C C14N test suite:: + + +File: python.info, Node: Mersenne Twister, Next: Sockets<2>, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.1 Mersenne Twister +----------------------- + +The ‘_random’ module includes code based on a download from +‘http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html’. +The following are the verbatim comments from the original code: + + A C-program for MT19937, with initialization improved 2002/1/26. + Coded by Takuji Nishimura and Makoto Matsumoto. + + Before using, initialize the state by using init_genrand(seed) + or init_by_array(init_key, key_length). + + Copyright (C) 1997 - 2002, Makoto Matsumoto and Takuji Nishimura, + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + 3. The names of its contributors may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR + PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + + Any feedback is very welcome. + http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html + email: m-mat @ math.sci.hiroshima-u.ac.jp (remove space) + + +File: python.info, Node: Sockets<2>, Next: Asynchronous socket services, Prev: Mersenne Twister, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.2 Sockets +-------------- + +The *note socket: ef. module uses the functions, ‘getaddrinfo()’, and +‘getnameinfo()’, which are coded in separate source files from the WIDE +Project, ‘http://www.wide.ad.jp/’. + + Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + 3. Neither the name of the project nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND + ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE + FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + SUCH DAMAGE. + + +File: python.info, Node: Asynchronous socket services, Next: Cookie management, Prev: Sockets<2>, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.3 Asynchronous socket services +----------------------------------- + +The *note asynchat: 9. and *note asyncore: b. modules contain the +following notice: + + Copyright 1996 by Sam Rushing + + All Rights Reserved + + Permission to use, copy, modify, and distribute this software and + its documentation for any purpose and without fee is hereby + granted, provided that the above copyright notice appear in all + copies and that both that copyright notice and this permission + notice appear in supporting documentation, and that the name of Sam + Rushing not be used in advertising or publicity pertaining to + distribution of the software without specific, written prior + permission. + + SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, + INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN + NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR + CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS + OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, + NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN + CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + + +File: python.info, Node: Cookie management, Next: Execution tracing, Prev: Asynchronous socket services, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.4 Cookie management +------------------------ + +The *note http.cookies: 96. module contains the following notice: + + Copyright 2000 by Timothy O'Malley + + All Rights Reserved + + Permission to use, copy, modify, and distribute this software + and its documentation for any purpose and without fee is hereby + granted, provided that the above copyright notice appear in all + copies and that both that copyright notice and this permission + notice appear in supporting documentation, and that the name of + Timothy O'Malley not be used in advertising or publicity + pertaining to distribution of the software without specific, written + prior permission. + + Timothy O'Malley DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS + SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY + AND FITNESS, IN NO EVENT SHALL Timothy O'Malley BE LIABLE FOR + ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, + WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS + ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR + PERFORMANCE OF THIS SOFTWARE. + + +File: python.info, Node: Execution tracing, Next: UUencode and UUdecode functions, Prev: Cookie management, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.5 Execution tracing +------------------------ + +The *note trace: 112. module contains the following notice: + + portions copyright 2001, Autonomous Zones Industries, Inc., all rights... + err... reserved and offered to the public under the terms of the + Python 2.2 license. + Author: Zooko O'Whielacronx + http://zooko.com/ + mailto:zooko@zooko.com + + Copyright 2000, Mojam Media, Inc., all rights reserved. + Author: Skip Montanaro + + Copyright 1999, Bioreason, Inc., all rights reserved. + Author: Andrew Dalke + + Copyright 1995-1997, Automatrix, Inc., all rights reserved. + Author: Skip Montanaro + + Copyright 1991-1995, Stichting Mathematisch Centrum, all rights reserved. + + + Permission to use, copy, modify, and distribute this Python software and + its associated documentation for any purpose without fee is hereby + granted, provided that the above copyright notice appears in all copies, + and that both that copyright notice and this permission notice appear in + supporting documentation, and that the name of neither Automatrix, + Bioreason or Mojam Media be used in advertising or publicity pertaining to + distribution of the software without specific, written prior permission. + + +File: python.info, Node: UUencode and UUdecode functions, Next: XML Remote Procedure Calls, Prev: Execution tracing, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.6 UUencode and UUdecode functions +-------------------------------------- + +The *note uu: 123. module contains the following notice: + + Copyright 1994 by Lance Ellinghouse + Cathedral City, California Republic, United States of America. + All Rights Reserved + Permission to use, copy, modify, and distribute this software and its + documentation for any purpose and without fee is hereby granted, + provided that the above copyright notice appear in all copies and that + both that copyright notice and this permission notice appear in + supporting documentation, and that the name of Lance Ellinghouse + not be used in advertising or publicity pertaining to distribution + of the software without specific, written prior permission. + LANCE ELLINGHOUSE DISCLAIMS ALL WARRANTIES WITH REGARD TO + THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND + FITNESS, IN NO EVENT SHALL LANCE ELLINGHOUSE CENTRUM BE LIABLE + FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT + OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + + Modified by Jack Jansen, CWI, July 1995: + - Use binascii module to do the actual line-by-line conversion + between ascii and binary. This results in a 1000-fold speedup. The C + version is still 5 times faster, though. + - Arguments more compliant with Python standard + + +File: python.info, Node: XML Remote Procedure Calls, Next: test_epoll, Prev: UUencode and UUdecode functions, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.7 XML Remote Procedure Calls +--------------------------------- + +The *note xmlrpc.client: 13f. module contains the following notice: + + The XML-RPC client interface is + + Copyright (c) 1999-2002 by Secret Labs AB + Copyright (c) 1999-2002 by Fredrik Lundh + + By obtaining, using, and/or copying this software and/or its + associated documentation, you agree that you have read, understood, + and will comply with the following terms and conditions: + + Permission to use, copy, modify, and distribute this software and + its associated documentation for any purpose and without fee is + hereby granted, provided that the above copyright notice appears in + all copies, and that both that copyright notice and this permission + notice appear in supporting documentation, and that the name of + Secret Labs AB or the author not be used in advertising or publicity + pertaining to distribution of the software without specific, written + prior permission. + + SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD + TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- + ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR + BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY + DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, + WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS + ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE + OF THIS SOFTWARE. + + +File: python.info, Node: test_epoll, Next: Select kqueue, Prev: XML Remote Procedure Calls, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.8 test_epoll +----------------- + +The ‘test_epoll’ module contains the following notice: + + Copyright (c) 2001-2006 Twisted Matrix Laboratories. + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + "Software"), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be + included in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE + LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION + OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + +File: python.info, Node: Select kqueue, Next: SipHash24, Prev: test_epoll, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.9 Select kqueue +-------------------- + +The *note select: e5. module contains the following notice for the +kqueue interface: + + Copyright (c) 2000 Doug White, 2006 James Knight, 2007 Christian Heimes + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + SUCH DAMAGE. + + +File: python.info, Node: SipHash24, Next: strtod and dtoa, Prev: Select kqueue, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.10 SipHash24 +----------------- + +The file ‘Python/pyhash.c’ contains Marek Majkowski’ implementation of +Dan Bernstein’s SipHash24 algorithm. It contains the following note: + + + Copyright (c) 2013 Marek Majkowski + + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + + Original location: + https://github.com/majek/csiphash/ + + Solution inspired by code from: + Samuel Neves (supercop/crypto_auth/siphash24/little) + djb (supercop/crypto_auth/siphash24/little2) + Jean-Philippe Aumasson (https://131002.net/siphash/siphash24.c) + + +File: python.info, Node: strtod and dtoa, Next: OpenSSL, Prev: SipHash24, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.11 strtod and dtoa +----------------------- + +The file ‘Python/dtoa.c’, which supplies C functions dtoa and strtod for +conversion of C doubles to and from strings, is derived from the file of +the same name by David M. Gay, currently available from +‘http://www.netlib.org/fp/’. The original file, as retrieved on March +16, 2009, contains the following copyright and licensing notice: + + /**************************************************************** + * + * The author of this software is David M. Gay. + * + * Copyright (c) 1991, 2000, 2001 by Lucent Technologies. + * + * Permission to use, copy, modify, and distribute this software for any + * purpose without fee is hereby granted, provided that this entire notice + * is included in all copies of any software which is or includes a copy + * or modification of this software and in all copies of the supporting + * documentation for such software. + * + * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED + * WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR LUCENT MAKES ANY + * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY + * OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE. + * + ***************************************************************/ + + +File: python.info, Node: OpenSSL, Next: expat, Prev: strtod and dtoa, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.12 OpenSSL +--------------- + +The modules *note hashlib: 8d, *note posix: d1, *note ssl: f3, *note +crypt: 29. use the OpenSSL library for added performance if made +available by the operating system. Additionally, the Windows and Mac OS +X installers for Python may include a copy of the OpenSSL libraries, so +we include a copy of the OpenSSL license here: + + LICENSE ISSUES + ============== + + The OpenSSL toolkit stays under a dual license, i.e. both the conditions of + the OpenSSL License and the original SSLeay license apply to the toolkit. + See below for the actual license texts. Actually both licenses are BSD-style + Open Source licenses. In case of any license issues related to OpenSSL + please contact openssl-core@openssl.org. + + OpenSSL License + --------------- + + /* ==================================================================== + * Copyright (c) 1998-2008 The OpenSSL Project. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * + * 3. All advertising materials mentioning features or use of this + * software must display the following acknowledgment: + * "This product includes software developed by the OpenSSL Project + * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" + * + * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to + * endorse or promote products derived from this software without + * prior written permission. For written permission, please contact + * openssl-core@openssl.org. + * + * 5. Products derived from this software may not be called "OpenSSL" + * nor may "OpenSSL" appear in their names without prior written + * permission of the OpenSSL Project. + * + * 6. Redistributions of any form whatsoever must retain the following + * acknowledgment: + * "This product includes software developed by the OpenSSL Project + * for use in the OpenSSL Toolkit (http://www.openssl.org/)" + * + * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY + * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR + * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + * ==================================================================== + * + * This product includes cryptographic software written by Eric Young + * (eay@cryptsoft.com). This product includes software written by Tim + * Hudson (tjh@cryptsoft.com). + * + */ + + Original SSLeay License + ----------------------- + + /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) + * All rights reserved. + * + * This package is an SSL implementation written + * by Eric Young (eay@cryptsoft.com). + * The implementation was written so as to conform with Netscapes SSL. + * + * This library is free for commercial and non-commercial use as long as + * the following conditions are aheared to. The following conditions + * apply to all code found in this distribution, be it the RC4, RSA, + * lhash, DES, etc., code; not just the SSL code. The SSL documentation + * included with this distribution is covered by the same copyright terms + * except that the holder is Tim Hudson (tjh@cryptsoft.com). + * + * Copyright remains Eric Young's, and as such any Copyright notices in + * the code are not to be removed. + * If this package is used in a product, Eric Young should be given attribution + * as the author of the parts of the library used. + * This can be in the form of a textual message at program startup or + * in documentation (online or textual) provided with the package. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * "This product includes cryptographic software written by + * Eric Young (eay@cryptsoft.com)" + * The word 'cryptographic' can be left out if the rouines from the library + * being used are not cryptographic related :-). + * 4. If you include any Windows specific code (or a derivative thereof) from + * the apps directory (application code) you must include an acknowledgement: + * "This product includes software written by Tim Hudson (tjh@cryptsoft.com)" + * + * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * The licence and distribution terms for any publically available version or + * derivative of this code cannot be changed. i.e. this code cannot simply be + * copied and put under another distribution licence + * [including the GNU Public Licence.] + */ + + +File: python.info, Node: expat, Next: libffi, Prev: OpenSSL, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.13 expat +------------- + +The ‘pyexpat’ extension is built using an included copy of the expat +sources unless the build is configured ‘--with-system-expat’: + + Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd + and Clark Cooper + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + "Software"), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY + CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, + TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE + SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + +File: python.info, Node: libffi, Next: zlib<3>, Prev: expat, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.14 libffi +-------------- + +The ‘_ctypes’ extension is built using an included copy of the libffi +sources unless the build is configured ‘--with-system-libffi’: + + Copyright (c) 1996-2008 Red Hat, Inc and others. + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + ``Software''), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + DEALINGS IN THE SOFTWARE. + + +File: python.info, Node: zlib<3>, Next: cfuhash, Prev: libffi, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.15 zlib +------------ + +The *note zlib: 144. extension is built using an included copy of the +zlib sources if the zlib version found on the system is too old to be +used for the build: + + Copyright (C) 1995-2011 Jean-loup Gailly and Mark Adler + + This software is provided 'as-is', without any express or implied + warranty. In no event will the authors be held liable for any damages + arising from the use of this software. + + Permission is granted to anyone to use this software for any purpose, + including commercial applications, and to alter it and redistribute it + freely, subject to the following restrictions: + + 1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. + + 2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. + + 3. This notice may not be removed or altered from any source distribution. + + Jean-loup Gailly Mark Adler + jloup@gzip.org madler@alumni.caltech.edu + + +File: python.info, Node: cfuhash, Next: libmpdec, Prev: zlib<3>, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.16 cfuhash +--------------- + +The implementation of the hash table used by the *note tracemalloc: 114. +is based on the cfuhash project: + + Copyright (c) 2005 Don Owens + All rights reserved. + + This code is released under the BSD license: + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + + * Neither the name of the author nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + + +File: python.info, Node: libmpdec, Next: W3C C14N test suite, Prev: cfuhash, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.17 libmpdec +---------------- + +The ‘_decimal’ module is built using an included copy of the libmpdec +library unless the build is configured ‘--with-system-libmpdec’: + + Copyright (c) 2008-2016 Stefan Krah. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS "AS IS" AND + ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + SUCH DAMAGE. + + +File: python.info, Node: W3C C14N test suite, Prev: libmpdec, Up: Licenses and Acknowledgements for Incorporated Software + +16.3.18 W3C C14N test suite +--------------------------- + +The C14N 2.0 test suite in the *note test: 105. package +(‘Lib/test/xmltestdata/c14n-20/’) was retrieved from the W3C website at +‘https://www.w3.org/TR/xml-c14n2-testcases/’ and is distributed under +the 3-clause BSD license: + + Copyright (c) 2013 W3C(R) (MIT, ERCIM, Keio, Beihang), All Rights + Reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of works must retain the original copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the original + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials + provided with the distribution. + + * Neither the name of the W3C nor the names of its contributors + may be used to endorse or promote products derived from this + work without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + + +File: python.info, Node: Distributing Python Modules Legacy version, Next: Installing Python Modules Legacy version, Prev: History and License, Up: Top + +17 Distributing Python Modules (Legacy version) +*********************************************** + + +Authors: Greg Ward, Anthony Baxter + + +Email: + +See also +........ + +*note Distributing Python Modules: 7f6. + + The up to date module distribution documentations + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + + Note: This guide only covers the basic tools for building and + distributing extensions that are provided as part of this version + of Python. Third party tools offer easier to use and more secure + alternatives. Refer to the quick recommendations section(1) in the + Python Packaging User Guide for more information. + +This document describes the Python Distribution Utilities (“Distutils”) +from the module developer’s point of view, describing the underlying +capabilities that ‘setuptools’ builds on to allow Python developers to +make Python modules and extensions readily available to a wider +audience. + +* Menu: + +* An Introduction to Distutils:: +* Writing the Setup Script:: +* Writing the Setup Configuration File:: +* Creating a Source Distribution:: +* Creating Built Distributions:: +* Distutils Examples:: +* Extending Distutils:: +* Command Reference:: +* API Reference:: + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/guides/tool-recommendations/ + + +File: python.info, Node: An Introduction to Distutils, Next: Writing the Setup Script, Up: Distributing Python Modules Legacy version + +17.1 An Introduction to Distutils +================================= + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +This document covers using the Distutils to distribute your Python +modules, concentrating on the role of developer/distributor: if you’re +looking for information on installing Python modules, you should refer +to the *note Installing Python Modules (Legacy version): 7f7. chapter. + +* Menu: + +* Concepts & Terminology:: +* A Simple Example: A Simple Example<2>. +* General Python terminology:: +* Distutils-specific terminology:: + + +File: python.info, Node: Concepts & Terminology, Next: A Simple Example<2>, Up: An Introduction to Distutils + +17.1.1 Concepts & Terminology +----------------------------- + +Using the Distutils is quite simple, both for module developers and for +users/administrators installing third-party modules. As a developer, +your responsibilities (apart from writing solid, well-documented and +well-tested code, of course!) are: + + * write a setup script (‘setup.py’ by convention) + + * (optional) write a setup configuration file + + * create a source distribution + + * (optional) create one or more built (binary) distributions + +Each of these tasks is covered in this document. + +Not all module developers have access to a multitude of platforms, so +it’s not always feasible to expect them to create a multitude of built +distributions. It is hoped that a class of intermediaries, called +`packagers', will arise to address this need. Packagers will take +source distributions released by module developers, build them on one or +more platforms, and release the resulting built distributions. Thus, +users on the most popular platforms will be able to install most popular +Python module distributions in the most natural way for their platform, +without having to run a single setup script or compile a line of code. + + +File: python.info, Node: A Simple Example<2>, Next: General Python terminology, Prev: Concepts & Terminology, Up: An Introduction to Distutils + +17.1.2 A Simple Example +----------------------- + +The setup script is usually quite simple, although since it’s written in +Python, there are no arbitrary limits to what you can do with it, though +you should be careful about putting arbitrarily expensive operations in +your setup script. Unlike, say, Autoconf-style configure scripts, the +setup script may be run multiple times in the course of building and +installing your module distribution. + +If all you want to do is distribute a module called ‘foo’, contained in +a file ‘foo.py’, then your setup script can be as simple as this: + + from distutils.core import setup + setup(name='foo', + version='1.0', + py_modules=['foo'], + ) + +Some observations: + + * most information that you supply to the Distutils is supplied as + keyword arguments to the ‘setup()’ function + + * those keyword arguments fall into two categories: package metadata + (name, version number) and information about what’s in the package + (a list of pure Python modules, in this case) + + * modules are specified by module name, not filename (the same will + hold true for packages and extensions) + + * it’s recommended that you supply a little more metadata, in + particular your name, email address and a URL for the project (see + section *note Writing the Setup Script: 40bf. for an example) + +To create a source distribution for this module, you would create a +setup script, ‘setup.py’, containing the above code, and run this +command from a terminal: + + python setup.py sdist + +For Windows, open a command prompt window (Start ‣ Accessories) and +change the command to: + + setup.py sdist + +‘sdist’ will create an archive file (e.g., tarball on Unix, ZIP file on +Windows) containing your setup script ‘setup.py’, and your module +‘foo.py’. The archive file will be named ‘foo-1.0.tar.gz’ (or ‘.zip’), +and will unpack into a directory ‘foo-1.0’. + +If an end-user wishes to install your ‘foo’ module, all they have to do +is download ‘foo-1.0.tar.gz’ (or ‘.zip’), unpack it, and—from the +‘foo-1.0’ directory—run + + python setup.py install + +which will ultimately copy ‘foo.py’ to the appropriate directory for +third-party modules in their Python installation. + +This simple example demonstrates some fundamental concepts of the +Distutils. First, both developers and installers have the same basic +user interface, i.e. the setup script. The difference is which +Distutils `commands' they use: the ‘sdist’ command is almost exclusively +for module developers, while ‘install’ is more often for installers +(although most developers will want to install their own code +occasionally). + +If you want to make things really easy for your users, you can create +one or more built distributions for them. For instance, if you are +running on a Windows machine, and want to make things easy for other +Windows users, you can create an executable installer (the most +appropriate type of built distribution for this platform) with the +‘bdist_wininst’ command. For example: + + python setup.py bdist_wininst + +will create an executable installer, ‘foo-1.0.win32.exe’, in the current +directory. + +Other useful built distribution formats are RPM, implemented by the +‘bdist_rpm’ command, Solaris ‘pkgtool’ (‘bdist_pkgtool’), and HP-UX +‘swinstall’ (‘bdist_sdux’). For example, the following command will +create an RPM file called ‘foo-1.0.noarch.rpm’: + + python setup.py bdist_rpm + +(The ‘bdist_rpm’ command uses the ‘rpm’ executable, therefore this has +to be run on an RPM-based system such as Red Hat Linux, SuSE Linux, or +Mandrake Linux.) + +You can find out what distribution formats are available at any time by +running + + python setup.py bdist --help-formats + + +File: python.info, Node: General Python terminology, Next: Distutils-specific terminology, Prev: A Simple Example<2>, Up: An Introduction to Distutils + +17.1.3 General Python terminology +--------------------------------- + +If you’re reading this document, you probably have a good idea of what +modules, extensions, and so forth are. Nevertheless, just to be sure +that everyone is operating from a common starting point, we offer the +following glossary of common Python terms: + +module + + the basic unit of code reusability in Python: a block of code + imported by some other code. Three types of modules concern us + here: pure Python modules, extension modules, and packages. + +pure Python module + + a module written in Python and contained in a single ‘.py’ file + (and possibly associated ‘.pyc’ files). Sometimes referred to as a + “pure module.” + +extension module + + a module written in the low-level language of the Python + implementation: C/C++ for Python, Java for Jython. Typically + contained in a single dynamically loadable pre-compiled file, e.g. + a shared object (‘.so’) file for Python extensions on Unix, a DLL + (given the ‘.pyd’ extension) for Python extensions on Windows, or a + Java class file for Jython extensions. (Note that currently, the + Distutils only handles C/C++ extensions for Python.) + +package + + a module that contains other modules; typically contained in a + directory in the filesystem and distinguished from other + directories by the presence of a file ‘__init__.py’. + +root package + + the root of the hierarchy of packages. (This isn’t really a + package, since it doesn’t have an ‘__init__.py’ file. But we have + to call it something.) The vast majority of the standard library + is in the root package, as are many small, standalone third-party + modules that don’t belong to a larger module collection. Unlike + regular packages, modules in the root package can be found in many + directories: in fact, every directory listed in ‘sys.path’ + contributes modules to the root package. + + +File: python.info, Node: Distutils-specific terminology, Prev: General Python terminology, Up: An Introduction to Distutils + +17.1.4 Distutils-specific terminology +------------------------------------- + +The following terms apply more specifically to the domain of +distributing Python modules using the Distutils: + +module distribution + + a collection of Python modules distributed together as a single + downloadable resource and meant to be installed `en masse'. + Examples of some well-known module distributions are NumPy, SciPy, + Pillow, or mxBase. (This would be called a `package', except that + term is already taken in the Python context: a single module + distribution may contain zero, one, or many Python packages.) + +pure module distribution + + a module distribution that contains only pure Python modules and + packages. Sometimes referred to as a “pure distribution.” + +non-pure module distribution + + a module distribution that contains at least one extension module. + Sometimes referred to as a “non-pure distribution.” + +distribution root + + the top-level directory of your source tree (or source + distribution); the directory where ‘setup.py’ exists. Generally + ‘setup.py’ will be run from this directory. + + +File: python.info, Node: Writing the Setup Script, Next: Writing the Setup Configuration File, Prev: An Introduction to Distutils, Up: Distributing Python Modules Legacy version + +17.2 Writing the Setup Script +============================= + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +The setup script is the centre of all activity in building, +distributing, and installing modules using the Distutils. The main +purpose of the setup script is to describe your module distribution to +the Distutils, so that the various commands that operate on your modules +do the right thing. As we saw in section *note A Simple Example: 40be. +above, the setup script consists mainly of a call to ‘setup()’, and most +information supplied to the Distutils by the module developer is +supplied as keyword arguments to ‘setup()’. + +Here’s a slightly more involved example, which we’ll follow for the next +couple of sections: the Distutils’ own setup script. (Keep in mind that +although the Distutils are included with Python 1.6 and later, they also +have an independent existence so that Python 1.5.2 users can use them to +install other module distributions. The Distutils’ own setup script, +shown here, is used to install the package into Python 1.5.2.) + + #!/usr/bin/env python + + from distutils.core import setup + + setup(name='Distutils', + version='1.0', + description='Python Distribution Utilities', + author='Greg Ward', + author_email='gward@python.net', + url='https://www.python.org/sigs/distutils-sig/', + packages=['distutils', 'distutils.command'], + ) + +There are only two differences between this and the trivial one-file +distribution presented in section *note A Simple Example: 40be.: more +metadata, and the specification of pure Python modules by package, +rather than by module. This is important since the Distutils consist of +a couple of dozen modules split into (so far) two packages; an explicit +list of every module would be tedious to generate and difficult to +maintain. For more information on the additional meta-data, see section +*note Additional meta-data: 40c6. + +Note that any pathnames (files or directories) supplied in the setup +script should be written using the Unix convention, i.e. +slash-separated. The Distutils will take care of converting this +platform-neutral representation into whatever is appropriate on your +current platform before actually using the pathname. This makes your +setup script portable across operating systems, which of course is one +of the major goals of the Distutils. In this spirit, all pathnames in +this document are slash-separated. + +This, of course, only applies to pathnames given to Distutils functions. +If you, for example, use standard Python functions such as *note +glob.glob(): 5c6. or *note os.listdir(): a41. to specify files, you +should be careful to write portable code instead of hardcoding path +separators: + + glob.glob(os.path.join('mydir', 'subdir', '*.html')) + os.listdir(os.path.join('mydir', 'subdir')) + +* Menu: + +* Listing whole packages:: +* Listing individual modules:: +* Describing extension modules:: +* Relationships between Distributions and Packages:: +* Installing Scripts:: +* Installing Package Data:: +* Installing Additional Files:: +* Additional meta-data:: +* Debugging the setup script:: + + +File: python.info, Node: Listing whole packages, Next: Listing individual modules, Up: Writing the Setup Script + +17.2.1 Listing whole packages +----------------------------- + +The ‘packages’ option tells the Distutils to process (build, distribute, +install, etc.) all pure Python modules found in each package mentioned +in the ‘packages’ list. In order to do this, of course, there has to be +a correspondence between package names and directories in the +filesystem. The default correspondence is the most obvious one, i.e. +package *note distutils: 39. is found in the directory ‘distutils’ +relative to the distribution root. Thus, when you say ‘packages = +['foo']’ in your setup script, you are promising that the Distutils will +find a file ‘foo/__init__.py’ (which might be spelled differently on +your system, but you get the idea) relative to the directory where your +setup script lives. If you break this promise, the Distutils will issue +a warning but still process the broken package anyway. + +If you use a different convention to lay out your source directory, +that’s no problem: you just have to supply the ‘package_dir’ option to +tell the Distutils about your convention. For example, say you keep all +Python source under ‘lib’, so that modules in the “root package” (i.e., +not in any package at all) are in ‘lib’, modules in the ‘foo’ package +are in ‘lib/foo’, and so forth. Then you would put + + package_dir = {'': 'lib'} + +in your setup script. The keys to this dictionary are package names, +and an empty package name stands for the root package. The values are +directory names relative to your distribution root. In this case, when +you say ‘packages = ['foo']’, you are promising that the file +‘lib/foo/__init__.py’ exists. + +Another possible convention is to put the ‘foo’ package right in ‘lib’, +the ‘foo.bar’ package in ‘lib/bar’, etc. This would be written in the +setup script as + + package_dir = {'foo': 'lib'} + +A ‘package: dir’ entry in the ‘package_dir’ dictionary implicitly +applies to all packages below `package', so the ‘foo.bar’ case is +automatically handled here. In this example, having ‘packages = ['foo', +'foo.bar']’ tells the Distutils to look for ‘lib/__init__.py’ and +‘lib/bar/__init__.py’. (Keep in mind that although ‘package_dir’ +applies recursively, you must explicitly list all packages in +‘packages’: the Distutils will `not' recursively scan your source tree +looking for any directory with an ‘__init__.py’ file.) + + +File: python.info, Node: Listing individual modules, Next: Describing extension modules, Prev: Listing whole packages, Up: Writing the Setup Script + +17.2.2 Listing individual modules +--------------------------------- + +For a small module distribution, you might prefer to list all modules +rather than listing packages—especially the case of a single module that +goes in the “root package” (i.e., no package at all). This simplest +case was shown in section *note A Simple Example: 40be.; here is a +slightly more involved example: + + py_modules = ['mod1', 'pkg.mod2'] + +This describes two modules, one of them in the “root” package, the other +in the ‘pkg’ package. Again, the default package/directory layout +implies that these two modules can be found in ‘mod1.py’ and +‘pkg/mod2.py’, and that ‘pkg/__init__.py’ exists as well. And again, +you can override the package/directory correspondence using the +‘package_dir’ option. + + +File: python.info, Node: Describing extension modules, Next: Relationships between Distributions and Packages, Prev: Listing individual modules, Up: Writing the Setup Script + +17.2.3 Describing extension modules +----------------------------------- + +Just as writing Python extension modules is a bit more complicated than +writing pure Python modules, describing them to the Distutils is a bit +more complicated. Unlike pure modules, it’s not enough just to list +modules or packages and expect the Distutils to go out and find the +right files; you have to specify the extension name, source file(s), and +any compile/link requirements (include directories, libraries to link +with, etc.). + +All of this is done through another keyword argument to ‘setup()’, the +‘ext_modules’ option. ‘ext_modules’ is just a list of *note Extension: +40cd. instances, each of which describes a single extension module. +Suppose your distribution includes a single extension, called ‘foo’ and +implemented by ‘foo.c’. If no additional instructions to the +compiler/linker are needed, describing this extension is quite simple: + + Extension('foo', ['foo.c']) + +The ‘Extension’ class can be imported from *note distutils.core: 54. +along with ‘setup()’. Thus, the setup script for a module distribution +that contains only this one extension and nothing else might be: + + from distutils.core import setup, Extension + setup(name='foo', + version='1.0', + ext_modules=[Extension('foo', ['foo.c'])], + ) + +The ‘Extension’ class (actually, the underlying extension-building +machinery implemented by the ‘build_ext’ command) supports a great deal +of flexibility in describing Python extensions, which is explained in +the following sections. + +* Menu: + +* Extension names and packages:: +* Extension source files:: +* Preprocessor options:: +* Library options:: +* Other options:: + + +File: python.info, Node: Extension names and packages, Next: Extension source files, Up: Describing extension modules + +17.2.3.1 Extension names and packages +..................................... + +The first argument to the *note Extension: 40cd. constructor is always +the name of the extension, including any package names. For example, + + Extension('foo', ['src/foo1.c', 'src/foo2.c']) + +describes an extension that lives in the root package, while + + Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c']) + +describes the same extension in the ‘pkg’ package. The source files and +resulting object code are identical in both cases; the only difference +is where in the filesystem (and therefore where in Python’s namespace +hierarchy) the resulting extension lives. + +If you have a number of extensions all in the same package (or all under +the same base package), use the ‘ext_package’ keyword argument to +‘setup()’. For example, + + setup(..., + ext_package='pkg', + ext_modules=[Extension('foo', ['foo.c']), + Extension('subpkg.bar', ['bar.c'])], + ) + +will compile ‘foo.c’ to the extension ‘pkg.foo’, and ‘bar.c’ to +‘pkg.subpkg.bar’. + + +File: python.info, Node: Extension source files, Next: Preprocessor options, Prev: Extension names and packages, Up: Describing extension modules + +17.2.3.2 Extension source files +............................... + +The second argument to the *note Extension: 40cd. constructor is a list +of source files. Since the Distutils currently only support C, C++, and +Objective-C extensions, these are normally C/C++/Objective-C source +files. (Be sure to use appropriate extensions to distinguish C++ source +files: ‘.cc’ and ‘.cpp’ seem to be recognized by both Unix and Windows +compilers.) + +However, you can also include SWIG interface (‘.i’) files in the list; +the ‘build_ext’ command knows how to deal with SWIG extensions: it will +run SWIG on the interface file and compile the resulting C/C++ file into +your extension. + +This warning notwithstanding, options to SWIG can be currently passed +like this: + + setup(..., + ext_modules=[Extension('_foo', ['foo.i'], + swig_opts=['-modern', '-I../include'])], + py_modules=['foo'], + ) + +Or on the commandline like this: + + > python setup.py build_ext --swig-opts="-modern -I../include" + +On some platforms, you can include non-source files that are processed +by the compiler and included in your extension. Currently, this just +means Windows message text (‘.mc’) files and resource definition (‘.rc’) +files for Visual C++. These will be compiled to binary resource +(‘.res’) files and linked into the executable. + + +File: python.info, Node: Preprocessor options, Next: Library options, Prev: Extension source files, Up: Describing extension modules + +17.2.3.3 Preprocessor options +............................. + +Three optional arguments to *note Extension: 40cd. will help if you need +to specify include directories to search or preprocessor macros to +define/undefine: ‘include_dirs’, ‘define_macros’, and ‘undef_macros’. + +For example, if your extension requires header files in the ‘include’ +directory under your distribution root, use the ‘include_dirs’ option: + + Extension('foo', ['foo.c'], include_dirs=['include']) + +You can specify absolute directories there; if you know that your +extension will only be built on Unix systems with X11R6 installed to +‘/usr’, you can get away with + + Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11']) + +You should avoid this sort of non-portable usage if you plan to +distribute your code: it’s probably better to write C code like + + #include + +If you need to include header files from some other Python extension, +you can take advantage of the fact that header files are installed in a +consistent way by the Distutils ‘install_headers’ command. For example, +the Numerical Python header files are installed (on a standard Unix +installation) to ‘/usr/local/include/python1.5/Numerical’. (The exact +location will differ according to your platform and Python +installation.) Since the Python include +directory—‘/usr/local/include/python1.5’ in this case—is always included +in the search path when building Python extensions, the best approach is +to write C code like + + #include + +If you must put the ‘Numerical’ include directory right into your header +search path, though, you can find that directory using the Distutils +*note distutils.sysconfig: 62. module: + + from distutils.sysconfig import get_python_inc + incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical') + setup(..., + Extension(..., include_dirs=[incdir]), + ) + +Even though this is quite portable—it will work on any Python +installation, regardless of platform—it’s probably easier to just write +your C code in the sensible way. + +You can define and undefine pre-processor macros with the +‘define_macros’ and ‘undef_macros’ options. ‘define_macros’ takes a +list of ‘(name, value)’ tuples, where ‘name’ is the name of the macro to +define (a string) and ‘value’ is its value: either a string or ‘None’. +(Defining a macro ‘FOO’ to ‘None’ is the equivalent of a bare ‘#define +FOO’ in your C source: with most compilers, this sets ‘FOO’ to the +string ‘1’.) ‘undef_macros’ is just a list of macros to undefine. + +For example: + + Extension(..., + define_macros=[('NDEBUG', '1'), + ('HAVE_STRFTIME', None)], + undef_macros=['HAVE_FOO', 'HAVE_BAR']) + +is the equivalent of having this at the top of every C source file: + + #define NDEBUG 1 + #define HAVE_STRFTIME + #undef HAVE_FOO + #undef HAVE_BAR + + +File: python.info, Node: Library options, Next: Other options, Prev: Preprocessor options, Up: Describing extension modules + +17.2.3.4 Library options +........................ + +You can also specify the libraries to link against when building your +extension, and the directories to search for those libraries. The +‘libraries’ option is a list of libraries to link against, +‘library_dirs’ is a list of directories to search for libraries at +link-time, and ‘runtime_library_dirs’ is a list of directories to search +for shared (dynamically loaded) libraries at run-time. + +For example, if you need to link against libraries known to be in the +standard library search path on target systems + + Extension(..., + libraries=['gdbm', 'readline']) + +If you need to link with libraries in a non-standard location, you’ll +have to include the location in ‘library_dirs’: + + Extension(..., + library_dirs=['/usr/X11R6/lib'], + libraries=['X11', 'Xt']) + +(Again, this sort of non-portable construct should be avoided if you +intend to distribute your code.) + + +File: python.info, Node: Other options, Prev: Library options, Up: Describing extension modules + +17.2.3.5 Other options +...................... + +There are still some other options which can be used to handle special +cases. + +The ‘optional’ option is a boolean; if it is true, a build failure in +the extension will not abort the build process, but instead simply not +install the failing extension. + +The ‘extra_objects’ option is a list of object files to be passed to the +linker. These files must not have extensions, as the default extension +for the compiler is used. + +‘extra_compile_args’ and ‘extra_link_args’ can be used to specify +additional command line options for the respective compiler and linker +command lines. + +‘export_symbols’ is only useful on Windows. It can contain a list of +symbols (functions or variables) to be exported. This option is not +needed when building compiled extensions: Distutils will automatically +add ‘initmodule’ to the list of exported symbols. + +The ‘depends’ option is a list of files that the extension depends on +(for example header files). The build command will call the compiler on +the sources to rebuild extension if any on this files has been modified +since the previous build. + + +File: python.info, Node: Relationships between Distributions and Packages, Next: Installing Scripts, Prev: Describing extension modules, Up: Writing the Setup Script + +17.2.4 Relationships between Distributions and Packages +------------------------------------------------------- + +A distribution may relate to packages in three specific ways: + + 1. It can require packages or modules. + + 2. It can provide packages or modules. + + 3. It can obsolete packages or modules. + +These relationships can be specified using keyword arguments to the +*note distutils.core.setup(): 38b8. function. + +Dependencies on other Python modules and packages can be specified by +supplying the `requires' keyword argument to ‘setup()’. The value must +be a list of strings. Each string specifies a package that is required, +and optionally what versions are sufficient. + +To specify that any version of a module or package is required, the +string should consist entirely of the module or package name. Examples +include ‘'mymodule'’ and ‘'xml.parsers.expat'’. + +If specific versions are required, a sequence of qualifiers can be +supplied in parentheses. Each qualifier may consist of a comparison +operator and a version number. The accepted comparison operators are: + + < > == + <= >= != + +These can be combined by using multiple qualifiers separated by commas +(and optional whitespace). In this case, all of the qualifiers must be +matched; a logical AND is used to combine the evaluations. + +Let’s look at a bunch of examples: + +Requires Expression Explanation + +--------------------------------------------------------------------------------- + +‘==1.0’ Only version ‘1.0’ is compatible + + +‘>1.0, !=1.5.1, <2.0’ Any version after ‘1.0’ and before ‘2.0’ is + compatible, except ‘1.5.1’ + + +Now that we can specify dependencies, we also need to be able to specify +what we provide that other distributions can require. This is done +using the `provides' keyword argument to ‘setup()’. The value for this +keyword is a list of strings, each of which names a Python module or +package, and optionally identifies the version. If the version is not +specified, it is assumed to match that of the distribution. + +Some examples: + +Provides Expression Explanation + +----------------------------------------------------------------------------- + +‘mypkg’ Provide ‘mypkg’, using the distribution version + + +‘mypkg (1.1)’ Provide ‘mypkg’ version 1.1, regardless of the + distribution version + + +A package can declare that it obsoletes other packages using the +`obsoletes' keyword argument. The value for this is similar to that of +the `requires' keyword: a list of strings giving module or package +specifiers. Each specifier consists of a module or package name +optionally followed by one or more version qualifiers. Version +qualifiers are given in parentheses after the module or package name. + +The versions identified by the qualifiers are those that are obsoleted +by the distribution being described. If no qualifiers are given, all +versions of the named module or package are understood to be obsoleted. + + +File: python.info, Node: Installing Scripts, Next: Installing Package Data, Prev: Relationships between Distributions and Packages, Up: Writing the Setup Script + +17.2.5 Installing Scripts +------------------------- + +So far we have been dealing with pure and non-pure Python modules, which +are usually not run by themselves but imported by scripts. + +Scripts are files containing Python source code, intended to be started +from the command line. Scripts don’t require Distutils to do anything +very complicated. The only clever feature is that if the first line of +the script starts with ‘#!’ and contains the word “python”, the +Distutils will adjust the first line to refer to the current interpreter +location. By default, it is replaced with the current interpreter +location. The ‘--executable’ (or ‘-e’) option will allow the +interpreter path to be explicitly overridden. + +The ‘scripts’ option simply is a list of files to be handled in this +way. From the PyXML setup script: + + setup(..., + scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val'] + ) + +Changed in version 3.1: All the scripts will also be added to the +‘MANIFEST’ file if no template is provided. See *note Specifying the +files to distribute: 38bb. + + +File: python.info, Node: Installing Package Data, Next: Installing Additional Files, Prev: Installing Scripts, Up: Writing the Setup Script + +17.2.6 Installing Package Data +------------------------------ + +Often, additional files need to be installed into a package. These +files are often data that’s closely related to the package’s +implementation, or text files containing documentation that might be of +interest to programmers using the package. These files are called +`package data'. + +Package data can be added to packages using the ‘package_data’ keyword +argument to the ‘setup()’ function. The value must be a mapping from +package name to a list of relative path names that should be copied into +the package. The paths are interpreted as relative to the directory +containing the package (information from the ‘package_dir’ mapping is +used if appropriate); that is, the files are expected to be part of the +package in the source directories. They may contain glob patterns as +well. + +The path names may contain directory portions; any necessary directories +will be created in the installation. + +For example, if a package should contain a subdirectory with several +data files, the files can be arranged like this in the source tree: + + setup.py + src/ + mypkg/ + __init__.py + module.py + data/ + tables.dat + spoons.dat + forks.dat + +The corresponding call to ‘setup()’ might be: + + setup(..., + packages=['mypkg'], + package_dir={'mypkg': 'src/mypkg'}, + package_data={'mypkg': ['data/*.dat']}, + ) + +Changed in version 3.1: All the files that match ‘package_data’ will be +added to the ‘MANIFEST’ file if no template is provided. See *note +Specifying the files to distribute: 38bb. + + +File: python.info, Node: Installing Additional Files, Next: Additional meta-data, Prev: Installing Package Data, Up: Writing the Setup Script + +17.2.7 Installing Additional Files +---------------------------------- + +The ‘data_files’ option can be used to specify additional files needed +by the module distribution: configuration files, message catalogs, data +files, anything which doesn’t fit in the previous categories. + +‘data_files’ specifies a sequence of (`directory', `files') pairs in the +following way: + + setup(..., + data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']), + ('config', ['cfg/data.cfg'])], + ) + +Each (`directory', `files') pair in the sequence specifies the +installation directory and the files to install there. + +Each file name in `files' is interpreted relative to the ‘setup.py’ +script at the top of the package source distribution. Note that you can +specify the directory where the data files will be installed, but you +cannot rename the data files themselves. + +The `directory' should be a relative path. It is interpreted relative +to the installation prefix (Python’s ‘sys.prefix’ for system +installations; ‘site.USER_BASE’ for user installations). Distutils +allows `directory' to be an absolute installation path, but this is +discouraged since it is incompatible with the wheel packaging format. +No directory information from `files' is used to determine the final +location of the installed file; only the name of the file is used. + +You can specify the ‘data_files’ options as a simple sequence of files +without specifying a target directory, but this is not recommended, and +the ‘install’ command will print a warning in this case. To install +data files directly in the target directory, an empty string should be +given as the directory. + +Changed in version 3.1: All the files that match ‘data_files’ will be +added to the ‘MANIFEST’ file if no template is provided. See *note +Specifying the files to distribute: 38bb. + + +File: python.info, Node: Additional meta-data, Next: Debugging the setup script, Prev: Installing Additional Files, Up: Writing the Setup Script + +17.2.8 Additional meta-data +--------------------------- + +The setup script may include additional meta-data beyond the name and +version. This information includes: + +Meta-Data Description Value Notes + +---------------------------------------------------------------------------------------------- + +‘name’ name of the package short string (1) + + +‘version’ version of this release short string (1)(2) + + +‘author’ package author’s name short string (3) + + +‘author_email’ email address of the package email address (3) + author + + +‘maintainer’ package maintainer’s name short string (3) + + +‘maintainer_email’ email address of the package email address (3) + maintainer + + +‘url’ home page for the package URL (1) + + +‘description’ short, summary description of short string + the package + + +‘long_description’ longer description of the long string (4) + package + + +‘download_url’ location where the package URL + may be downloaded + + +‘classifiers’ a list of classifiers list of strings (6)(7) + + +‘platforms’ a list of platforms list of strings (6)(8) + + +‘keywords’ a list of keywords list of strings (6)(8) + + +‘license’ license for the package short string (5) + + +Notes: + + 1. These fields are required. + + 2. It is recommended that versions take the form + `major.minor[.patch[.sub]]'. + + 3. Either the author or the maintainer must be identified. If + maintainer is provided, distutils lists it as the author in + ‘PKG-INFO’. + + 4. The ‘long_description’ field is used by PyPI when you publish a + package, to build its project page. + + 5. The ‘license’ field is a text indicating the license covering the + package where the license is not a selection from the “License” + Trove classifiers. See the ‘Classifier’ field. Notice that + there’s a ‘licence’ distribution option which is deprecated but + still acts as an alias for ‘license’. + + 6. This field must be a list. + + 7. The valid classifiers are listed on PyPI(1). + + 8. To preserve backward compatibility, this field also accepts a + string. If you pass a comma-separated string ‘'foo, bar'’, it will + be converted to ‘['foo', 'bar']’, Otherwise, it will be converted + to a list of one string. + +‘short string’ + + A single line of text, not more than 200 characters. + +‘long string’ + + Multiple lines of plain text in reStructuredText format (see + ‘http://docutils.sourceforge.net/’). + +‘list of strings’ + + See below. + +Encoding the version information is an art in itself. Python packages +generally adhere to the version format `major.minor[.patch][sub]'. The +major number is 0 for initial, experimental releases of software. It is +incremented for releases that represent major milestones in a package. +The minor number is incremented when important new features are added to +the package. The patch number increments when bug-fix releases are +made. Additional trailing version information is sometimes used to +indicate sub-releases. These are “a1,a2,…,aN” (for alpha releases, +where functionality and API may change), “b1,b2,…,bN” (for beta +releases, which only fix bugs) and “pr1,pr2,…,prN” (for final +pre-release release testing). Some examples: + +0.1.0 + + the first, experimental release of a package + +1.0.1a2 + + the second alpha release of the first patch version of 1.0 + +‘classifiers’ must be specified in a list: + + setup(..., + classifiers=[ + 'Development Status :: 4 - Beta', + 'Environment :: Console', + 'Environment :: Web Environment', + 'Intended Audience :: End Users/Desktop', + 'Intended Audience :: Developers', + 'Intended Audience :: System Administrators', + 'License :: OSI Approved :: Python Software Foundation License', + 'Operating System :: MacOS :: MacOS X', + 'Operating System :: Microsoft :: Windows', + 'Operating System :: POSIX', + 'Programming Language :: Python', + 'Topic :: Communications :: Email', + 'Topic :: Office/Business', + 'Topic :: Software Development :: Bug Tracking', + ], + ) + +Changed in version 3.7: *note setup: 38b8. now warns when ‘classifiers’, +‘keywords’ or ‘platforms’ fields are not specified as a list or a +string. + + ---------- Footnotes ---------- + + (1) https://pypi.org/classifiers + + +File: python.info, Node: Debugging the setup script, Prev: Additional meta-data, Up: Writing the Setup Script + +17.2.9 Debugging the setup script +--------------------------------- + +Sometimes things go wrong, and the setup script doesn’t do what the +developer wants. + +Distutils catches any exceptions when running the setup script, and +print a simple error message before the script is terminated. The +motivation for this behaviour is to not confuse administrators who don’t +know much about Python and are trying to install a package. If they get +a big long traceback from deep inside the guts of Distutils, they may +think the package or the Python installation is broken because they +don’t read all the way down to the bottom and see that it’s a permission +problem. + +On the other hand, this doesn’t help the developer to find the cause of +the failure. For this purpose, the ‘DISTUTILS_DEBUG’ environment +variable can be set to anything except an empty string, and distutils +will now print detailed information about what it is doing, dump the +full traceback when an exception occurs, and print the whole command +line when an external program (like a C compiler) fails. + + +File: python.info, Node: Writing the Setup Configuration File, Next: Creating a Source Distribution, Prev: Writing the Setup Script, Up: Distributing Python Modules Legacy version + +17.3 Writing the Setup Configuration File +========================================= + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +Often, it’s not possible to write down everything needed to build a +distribution `a priori': you may need to get some information from the +user, or from the user’s system, in order to proceed. As long as that +information is fairly simple—a list of directories to search for C +header files or libraries, for example—then providing a configuration +file, ‘setup.cfg’, for users to edit is a cheap and easy way to solicit +it. Configuration files also let you provide default values for any +command option, which the installer can then override either on the +command-line or by editing the config file. + +The setup configuration file is a useful middle-ground between the setup +script—which, ideally, would be opaque to installers (1)—and the +command-line to the setup script, which is outside of your control and +entirely up to the installer. In fact, ‘setup.cfg’ (and any other +Distutils configuration files present on the target system) are +processed after the contents of the setup script, but before the +command-line. This has several useful consequences: + + * installers can override some of what you put in ‘setup.py’ by + editing ‘setup.cfg’ + + * you can provide non-standard defaults for options that are not + easily set in ‘setup.py’ + + * installers can override anything in ‘setup.cfg’ using the + command-line options to ‘setup.py’ + +The basic syntax of the configuration file is simple: + + [command] + option=value + ... + +where `command' is one of the Distutils commands (e.g. ‘build_py’, +‘install’), and `option' is one of the options that command supports. +Any number of options can be supplied for each command, and any number +of command sections can be included in the file. Blank lines are +ignored, as are comments, which run from a ‘'#'’ character until the end +of the line. Long option values can be split across multiple lines +simply by indenting the continuation lines. + +You can find out the list of options supported by a particular command +with the universal ‘--help’ option, e.g. + + $ python setup.py --help build_ext + [...] + Options for 'build_ext' command: + --build-lib (-b) directory for compiled extension modules + --build-temp (-t) directory for temporary files (build by-products) + --inplace (-i) ignore build-lib and put compiled extensions into the + source directory alongside your pure Python modules + --include-dirs (-I) list of directories to search for header files + --define (-D) C preprocessor macros to define + --undef (-U) C preprocessor macros to undefine + --swig-opts list of SWIG command line options + [...] + +Note that an option spelled ‘--foo-bar’ on the command-line is spelled +‘foo_bar’ in configuration files. For example, say you want your +extensions to be built “in-place”—that is, you have an extension +‘pkg.ext’, and you want the compiled extension file (‘ext.so’ on Unix, +say) to be put in the same source directory as your pure Python modules +‘pkg.mod1’ and ‘pkg.mod2’. You can always use the ‘--inplace’ option on +the command-line to ensure this: + + python setup.py build_ext --inplace + +But this requires that you always specify the ‘build_ext’ command +explicitly, and remember to provide ‘--inplace’. An easier way is to +“set and forget” this option, by encoding it in ‘setup.cfg’, the +configuration file for this distribution: + + [build_ext] + inplace=1 + +This will affect all builds of this module distribution, whether or not +you explicitly specify ‘build_ext’. If you include ‘setup.cfg’ in your +source distribution, it will also affect end-user builds—which is +probably a bad idea for this option, since always building extensions +in-place would break installation of the module distribution. In +certain peculiar cases, though, modules are built right in their +installation directory, so this is conceivably a useful ability. +(Distributing extensions that expect to be built in their installation +directory is almost always a bad idea, though.) + +Another example: certain commands take a lot of options that don’t +change from run to run; for example, ‘bdist_rpm’ needs to know +everything required to generate a “spec” file for creating an RPM +distribution. Some of this information comes from the setup script, and +some is automatically generated by the Distutils (such as the list of +files installed). But some of it has to be supplied as options to +‘bdist_rpm’, which would be very tedious to do on the command-line for +every run. Hence, here is a snippet from the Distutils’ own +‘setup.cfg’: + + [bdist_rpm] + release = 1 + packager = Greg Ward + doc_files = CHANGES.txt + README.txt + USAGE.txt + doc/ + examples/ + +Note that the ‘doc_files’ option is simply a whitespace-separated string +split across multiple lines for readability. + +See also +........ + +*note Syntax of config files: 40e1. in “Installing Python Modules” + + More information on the configuration files is available in the + manual for system administrators. + + ---------- Footnotes ---------- + + (1) (1) This ideal probably won’t be achieved until +auto-configuration is fully supported by the Distutils. + + +File: python.info, Node: Creating a Source Distribution, Next: Creating Built Distributions, Prev: Writing the Setup Configuration File, Up: Distributing Python Modules Legacy version + +17.4 Creating a Source Distribution +=================================== + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +As shown in section *note A Simple Example: 40be, you use the ‘sdist’ +command to create a source distribution. In the simplest case, + + python setup.py sdist + +(assuming you haven’t specified any ‘sdist’ options in the setup script +or config file), ‘sdist’ creates the archive of the default format for +the current platform. The default format is a gzip’ed tar file +(‘.tar.gz’) on Unix, and ZIP file on Windows. + +You can specify as many formats as you like using the ‘--formats’ +option, for example: + + python setup.py sdist --formats=gztar,zip + +to create a gzipped tarball and a zip file. The available formats are: + +Format Description Notes + +------------------------------------------------------------ + +‘zip’ zip file (‘.zip’) (1),(3) + + +‘gztar’ gzip’ed tar file (2) + (‘.tar.gz’) + + +‘bztar’ bzip2’ed tar file + (‘.tar.bz2’) + + +‘xztar’ xz’ed tar file (‘.tar.xz’) + + +‘ztar’ compressed tar file (4) + (‘.tar.Z’) + + +‘tar’ tar file (‘.tar’) + + +Changed in version 3.5: Added support for the ‘xztar’ format. + +Notes: + + 1. default on Windows + + 2. default on Unix + + 3. requires either external ‘zip’ utility or *note zipfile: 142. + module (part of the standard Python library since Python 1.6) + + 4. requires the ‘compress’ program. Notice that this format is now + pending for deprecation and will be removed in the future versions + of Python. + +When using any ‘tar’ format (‘gztar’, ‘bztar’, ‘xztar’, ‘ztar’ or +‘tar’), under Unix you can specify the ‘owner’ and ‘group’ names that +will be set for each member of the archive. + +For example, if you want all files of the archive to be owned by root: + + python setup.py sdist --owner=root --group=root + +* Menu: + +* Specifying the files to distribute:: +* Manifest-related options:: + + +File: python.info, Node: Specifying the files to distribute, Next: Manifest-related options, Up: Creating a Source Distribution + +17.4.1 Specifying the files to distribute +----------------------------------------- + +If you don’t supply an explicit list of files (or instructions on how to +generate one), the ‘sdist’ command puts a minimal default set into the +source distribution: + + * all Python source files implied by the ‘py_modules’ and ‘packages’ + options + + * all C source files mentioned in the ‘ext_modules’ or ‘libraries’ + options + + * scripts identified by the ‘scripts’ option See *note Installing + Scripts: 40d4. + + * anything that looks like a test script: ‘test/test*.py’ (currently, + the Distutils don’t do anything with test scripts except include + them in source distributions, but in the future there will be a + standard for testing Python module distributions) + + * Any of the standard README files (‘README’, ‘README.txt’, or + ‘README.rst’), ‘setup.py’ (or whatever you called your setup + script), and ‘setup.cfg’. + + * all files that matches the ‘package_data’ metadata. See *note + Installing Package Data: 40d6. + + * all files that matches the ‘data_files’ metadata. See *note + Installing Additional Files: 40d8. + +Sometimes this is enough, but usually you will want to specify +additional files to distribute. The typical way to do this is to write +a `manifest template', called ‘MANIFEST.in’ by default. The manifest +template is just a list of instructions for how to generate your +manifest file, ‘MANIFEST’, which is the exact list of files to include +in your source distribution. The ‘sdist’ command processes this +template and generates a manifest based on its instructions and what it +finds in the filesystem. + +If you prefer to roll your own manifest file, the format is simple: one +filename per line, regular files (or symlinks to them) only. If you do +supply your own ‘MANIFEST’, you must specify everything: the default set +of files described above does not apply in this case. + +Changed in version 3.1: An existing generated ‘MANIFEST’ will be +regenerated without ‘sdist’ comparing its modification time to the one +of ‘MANIFEST.in’ or ‘setup.py’. + +Changed in version 3.1.3: ‘MANIFEST’ files start with a comment +indicating they are generated. Files without this comment are not +overwritten or removed. + +Changed in version 3.2.2: ‘sdist’ will read a ‘MANIFEST’ file if no +‘MANIFEST.in’ exists, like it used to do. + +Changed in version 3.7: ‘README.rst’ is now included in the list of +distutils standard READMEs. + +The manifest template has one command per line, where each command +specifies a set of files to include or exclude from the source +distribution. For an example, again we turn to the Distutils’ own +manifest template: + + include *.txt + recursive-include examples *.txt *.py + prune examples/sample?/build + +The meanings should be fairly clear: include all files in the +distribution root matching ‘*.txt’, all files anywhere under the +‘examples’ directory matching ‘*.txt’ or ‘*.py’, and exclude all +directories matching ‘examples/sample?/build’. All of this is done +`after' the standard include set, so you can exclude files from the +standard set with explicit instructions in the manifest template. (Or, +you can use the ‘--no-defaults’ option to disable the standard set +entirely.) There are several other commands available in the manifest +template mini-language; see section *note Creating a source +distribution; the sdist command: 40e6. + +The order of commands in the manifest template matters: initially, we +have the list of default files as described above, and each command in +the template adds to or removes from that list of files. Once we have +fully processed the manifest template, we remove files that should not +be included in the source distribution: + + * all files in the Distutils “build” tree (default ‘build/’) + + * all files in directories named ‘RCS’, ‘CVS’, ‘.svn’, ‘.hg’, ‘.git’, + ‘.bzr’ or ‘_darcs’ + +Now we have our complete list of files, which is written to the manifest +for future reference, and then used to build the source distribution +archive(s). + +You can disable the default set of included files with the +‘--no-defaults’ option, and you can disable the standard exclude set +with ‘--no-prune’. + +Following the Distutils’ own manifest template, let’s trace how the +‘sdist’ command builds the list of files to include in the Distutils +source distribution: + + 1. include all Python source files in the ‘distutils’ and + ‘distutils/command’ subdirectories (because packages corresponding + to those two directories were mentioned in the ‘packages’ option in + the setup script—see section *note Writing the Setup Script: 40bf.) + + 2. include ‘README.txt’, ‘setup.py’, and ‘setup.cfg’ (standard files) + + 3. include ‘test/test*.py’ (standard files) + + 4. include ‘*.txt’ in the distribution root (this will find + ‘README.txt’ a second time, but such redundancies are weeded out + later) + + 5. include anything matching ‘*.txt’ or ‘*.py’ in the sub-tree under + ‘examples’, + + 6. exclude all files in the sub-trees starting at directories matching + ‘examples/sample?/build’—this may exclude files included by the + previous two steps, so it’s important that the ‘prune’ command in + the manifest template comes after the ‘recursive-include’ command + + 7. exclude the entire ‘build’ tree, and any ‘RCS’, ‘CVS’, ‘.svn’, + ‘.hg’, ‘.git’, ‘.bzr’ and ‘_darcs’ directories + +Just like in the setup script, file and directory names in the manifest +template should always be slash-separated; the Distutils will take care +of converting them to the standard representation on your platform. +That way, the manifest template is portable across operating systems. + + +File: python.info, Node: Manifest-related options, Prev: Specifying the files to distribute, Up: Creating a Source Distribution + +17.4.2 Manifest-related options +------------------------------- + +The normal course of operations for the ‘sdist’ command is as follows: + + * if the manifest file (‘MANIFEST’ by default) exists and the first + line does not have a comment indicating it is generated from + ‘MANIFEST.in’, then it is used as is, unaltered + + * if the manifest file doesn’t exist or has been previously + automatically generated, read ‘MANIFEST.in’ and create the manifest + + * if neither ‘MANIFEST’ nor ‘MANIFEST.in’ exist, create a manifest + with just the default file set + + * use the list of files now in ‘MANIFEST’ (either just generated or + read in) to create the source distribution archive(s) + +There are a couple of options that modify this behaviour. First, use +the ‘--no-defaults’ and ‘--no-prune’ to disable the standard “include” +and “exclude” sets. + +Second, you might just want to (re)generate the manifest, but not create +a source distribution: + + python setup.py sdist --manifest-only + +‘-o’ is a shortcut for ‘--manifest-only’. + + +File: python.info, Node: Creating Built Distributions, Next: Distutils Examples, Prev: Creating a Source Distribution, Up: Distributing Python Modules Legacy version + +17.5 Creating Built Distributions +================================= + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +A “built distribution” is what you’re probably used to thinking of +either as a “binary package” or an “installer” (depending on your +background). It’s not necessarily binary, though, because it might +contain only Python source code and/or byte-code; and we don’t call it a +package, because that word is already spoken for in Python. (And +“installer” is a term specific to the world of mainstream desktop +systems.) + +A built distribution is how you make life as easy as possible for +installers of your module distribution: for users of RPM-based Linux +systems, it’s a binary RPM; for Windows users, it’s an executable +installer; for Debian-based Linux users, it’s a Debian package; and so +forth. Obviously, no one person will be able to create built +distributions for every platform under the sun, so the Distutils are +designed to enable module developers to concentrate on their +specialty—writing code and creating source distributions—while an +intermediary species called `packagers' springs up to turn source +distributions into built distributions for as many platforms as there +are packagers. + +Of course, the module developer could be their own packager; or the +packager could be a volunteer “out there” somewhere who has access to a +platform which the original developer does not; or it could be software +periodically grabbing new source distributions and turning them into +built distributions for as many platforms as the software has access to. +Regardless of who they are, a packager uses the setup script and the +‘bdist’ command family to generate built distributions. + +As a simple example, if I run the following command in the Distutils +source tree: + + python setup.py bdist + +then the Distutils builds my module distribution (the Distutils itself +in this case), does a “fake” installation (also in the ‘build’ +directory), and creates the default type of built distribution for my +platform. The default format for built distributions is a “dumb” tar +file on Unix, and a simple executable installer on Windows. (That tar +file is considered “dumb” because it has to be unpacked in a specific +location to work.) + +Thus, the above command on a Unix system creates +‘Distutils-1.0.`plat'.tar.gz’; unpacking this tarball from the right +place installs the Distutils just as though you had downloaded the +source distribution and run ‘python setup.py install’. (The “right +place” is either the root of the filesystem or Python’s ‘`prefix'’ +directory, depending on the options given to the ‘bdist_dumb’ command; +the default is to make dumb distributions relative to ‘`prefix'’.) + +Obviously, for pure Python distributions, this isn’t any simpler than +just running ‘python setup.py install’—but for non-pure distributions, +which include extensions that would need to be compiled, it can mean the +difference between someone being able to use your extensions or not. +And creating “smart” built distributions, such as an RPM package or an +executable installer for Windows, is far more convenient for users even +if your distribution doesn’t include any extensions. + +The ‘bdist’ command has a ‘--formats’ option, similar to the ‘sdist’ +command, which you can use to select the types of built distribution to +generate: for example, + + python setup.py bdist --format=zip + +would, when run on a Unix system, create +‘Distutils-1.0.`plat'.zip’—again, this archive would be unpacked from +the root directory to install the Distutils. + +The available formats for built distributions are: + +Format Description Notes + +------------------------------------------------------------------- + +‘gztar’ gzipped tar file (‘.tar.gz’) (1) + + +‘bztar’ bzipped tar file (‘.tar.bz2’) + + +‘xztar’ xzipped tar file (‘.tar.xz’) + + +‘ztar’ compressed tar file (‘.tar.Z’) (3) + + +‘tar’ tar file (‘.tar’) + + +‘zip’ zip file (‘.zip’) (2),(4) + + +‘rpm’ RPM (5) + + +‘pkgtool’ Solaris ‘pkgtool’ + + +‘sdux’ HP-UX ‘swinstall’ + + +‘wininst’ self-extracting ZIP file for (4) + Windows + + +‘msi’ Microsoft Installer. + + +Changed in version 3.5: Added support for the ‘xztar’ format. + +Notes: + + 1. default on Unix + + 2. default on Windows + + 3. requires external ‘compress’ utility. + + 4. requires either external ‘zip’ utility or *note zipfile: 142. + module (part of the standard Python library since Python 1.6) + + 5. requires external ‘rpm’ utility, version 3.0.4 or better (use ‘rpm + --version’ to find out which version you have) + +You don’t have to use the ‘bdist’ command with the ‘--formats’ option; +you can also use the command that directly implements the format you’re +interested in. Some of these ‘bdist’ “sub-commands” actually generate +several similar formats; for instance, the ‘bdist_dumb’ command +generates all the “dumb” archive formats (‘tar’, ‘gztar’, ‘bztar’, +‘xztar’, ‘ztar’, and ‘zip’), and ‘bdist_rpm’ generates both binary and +source RPMs. The ‘bdist’ sub-commands, and the formats generated by +each, are: + +Command Formats + +------------------------------------------------------------------------- + +‘bdist_dumb’ tar, gztar, bztar, xztar, ztar, zip + + +‘bdist_rpm’ rpm, srpm + + +‘bdist_wininst’ wininst + + +‘bdist_msi’ msi + + + Note: bdist_wininst is deprecated since Python 3.8. + +The following sections give details on the individual ‘bdist_*’ +commands. + +* Menu: + +* Creating RPM packages:: +* Creating Windows Installers:: +* Cross-compiling on Windows:: +* Vista User Access Control (UAC): Vista User Access Control UAC. + + +File: python.info, Node: Creating RPM packages, Next: Creating Windows Installers, Up: Creating Built Distributions + +17.5.1 Creating RPM packages +---------------------------- + +The RPM format is used by many popular Linux distributions, including +Red Hat, SuSE, and Mandrake. If one of these (or any of the other +RPM-based Linux distributions) is your usual environment, creating RPM +packages for other users of that same distribution is trivial. +Depending on the complexity of your module distribution and differences +between Linux distributions, you may also be able to create RPMs that +work on different RPM-based distributions. + +The usual way to create an RPM of your module distribution is to run the +‘bdist_rpm’ command: + + python setup.py bdist_rpm + +or the ‘bdist’ command with the ‘--format’ option: + + python setup.py bdist --formats=rpm + +The former allows you to specify RPM-specific options; the latter allows +you to easily specify multiple formats in one run. If you need to do +both, you can explicitly specify multiple ‘bdist_*’ commands and their +options: + + python setup.py bdist_rpm --packager="John Doe " \ + bdist_wininst --target-version="2.0" + +Creating RPM packages is driven by a ‘.spec’ file, much as using the +Distutils is driven by the setup script. To make your life easier, the +‘bdist_rpm’ command normally creates a ‘.spec’ file based on the +information you supply in the setup script, on the command line, and in +any Distutils configuration files. Various options and sections in the +‘.spec’ file are derived from options in the setup script as follows: + +RPM ‘.spec’ file option or section Distutils setup script option + +-------------------------------------------------------------------------------------------------- + +Name ‘name’ + + +Summary (in preamble) ‘description’ + + +Version ‘version’ + + +Vendor ‘author’ and ‘author_email’, or — & ‘maintainer’ + and ‘maintainer_email’ + + +Copyright ‘license’ + + +Url ‘url’ + + +%description (section) ‘long_description’ + + +Additionally, there are many options in ‘.spec’ files that don’t have +corresponding options in the setup script. Most of these are handled +through options to the ‘bdist_rpm’ command as follows: + +RPM ‘.spec’ file option or ‘bdist_rpm’ option default value +section + +---------------------------------------------------------------------------------------------------- + +Release ‘release’ “1” + + +Group ‘group’ “Development/Libraries” + + +Vendor ‘vendor’ (see above) + + +Packager ‘packager’ (none) + + +Provides ‘provides’ (none) + + +Requires ‘requires’ (none) + + +Conflicts ‘conflicts’ (none) + + +Obsoletes ‘obsoletes’ (none) + + +Distribution ‘distribution_name’ (none) + + +BuildRequires ‘build_requires’ (none) + + +Icon ‘icon’ (none) + + +Obviously, supplying even a few of these options on the command-line +would be tedious and error-prone, so it’s usually best to put them in +the setup configuration file, ‘setup.cfg’—see section *note Writing the +Setup Configuration File: 40de. If you distribute or package many +Python module distributions, you might want to put options that apply to +all of them in your personal Distutils configuration file +(‘~/.pydistutils.cfg’). If you want to temporarily disable this file, +you can pass the ‘--no-user-cfg’ option to ‘setup.py’. + +There are three steps to building a binary RPM package, all of which are +handled automatically by the Distutils: + + 1. create a ‘.spec’ file, which describes the package (analogous to + the Distutils setup script; in fact, much of the information in the + setup script winds up in the ‘.spec’ file) + + 2. create the source RPM + + 3. create the “binary” RPM (which may or may not contain binary code, + depending on whether your module distribution contains Python + extensions) + +Normally, RPM bundles the last two steps together; when you use the +Distutils, all three steps are typically bundled together. + +If you wish, you can separate these three steps. You can use the +‘--spec-only’ option to make ‘bdist_rpm’ just create the ‘.spec’ file +and exit; in this case, the ‘.spec’ file will be written to the +“distribution directory”—normally ‘dist/’, but customizable with the +‘--dist-dir’ option. (Normally, the ‘.spec’ file winds up deep in the +“build tree,” in a temporary directory created by ‘bdist_rpm’.) + + +File: python.info, Node: Creating Windows Installers, Next: Cross-compiling on Windows, Prev: Creating RPM packages, Up: Creating Built Distributions + +17.5.2 Creating Windows Installers +---------------------------------- + + Warning: bdist_wininst is deprecated since Python 3.8. + +Executable installers are the natural format for binary distributions on +Windows. They display a nice graphical user interface, display some +information about the module distribution to be installed taken from the +metadata in the setup script, let the user select a few options, and +start or cancel the installation. + +Since the metadata is taken from the setup script, creating Windows +installers is usually as easy as running: + + python setup.py bdist_wininst + +or the ‘bdist’ command with the ‘--formats’ option: + + python setup.py bdist --formats=wininst + +If you have a pure module distribution (only containing pure Python +modules and packages), the resulting installer will be version +independent and have a name like ‘foo-1.0.win32.exe’. Note that +creating ‘wininst’ binary distributions in only supported on Windows +systems. + +If you have a non-pure distribution, the extensions can only be created +on a Windows platform, and will be Python version dependent. The +installer filename will reflect this and now has the form +‘foo-1.0.win32-py2.0.exe’. You have to create a separate installer for +every Python version you want to support. + +The installer will try to compile pure modules into *note bytecode: 614. +after installation on the target system in normal and optimizing mode. +If you don’t want this to happen for some reason, you can run the +‘bdist_wininst’ command with the ‘--no-target-compile’ and/or the +‘--no-target-optimize’ option. + +By default the installer will display the cool “Python Powered” logo +when it is run, but you can also supply your own 152x261 bitmap which +must be a Windows ‘.bmp’ file with the ‘--bitmap’ option. + +The installer will also display a large title on the desktop background +window when it is run, which is constructed from the name of your +distribution and the version number. This can be changed to another +text by using the ‘--title’ option. + +The installer file will be written to the “distribution directory” — +normally ‘dist/’, but customizable with the ‘--dist-dir’ option. + + +File: python.info, Node: Cross-compiling on Windows, Next: Vista User Access Control UAC, Prev: Creating Windows Installers, Up: Creating Built Distributions + +17.5.3 Cross-compiling on Windows +--------------------------------- + +Starting with Python 2.6, distutils is capable of cross-compiling +between Windows platforms. In practice, this means that with the +correct tools installed, you can use a 32bit version of Windows to +create 64bit extensions and vice-versa. + +To build for an alternate platform, specify the ‘--plat-name’ option to +the build command. Valid values are currently ‘win32’, and ‘win-amd64’. +For example, on a 32bit version of Windows, you could execute: + + python setup.py build --plat-name=win-amd64 + +to build a 64bit version of your extension. The Windows Installers also +support this option, so the command: + + python setup.py build --plat-name=win-amd64 bdist_wininst + +would create a 64bit installation executable on your 32bit version of +Windows. + +To cross-compile, you must download the Python source code and +cross-compile Python itself for the platform you are targeting - it is +not possible from a binary installation of Python (as the .lib etc file +for other platforms are not included.) In practice, this means the user +of a 32 bit operating system will need to use Visual Studio 2008 to open +the ‘PCbuild/PCbuild.sln’ solution in the Python source tree and build +the “x64” configuration of the ‘pythoncore’ project before +cross-compiling extensions is possible. + +Note that by default, Visual Studio 2008 does not install 64bit +compilers or tools. You may need to reexecute the Visual Studio setup +process and select these tools (using Control Panel->[Add/Remove] +Programs is a convenient way to check or modify your existing install.) + +* Menu: + +* The Postinstallation script:: + + +File: python.info, Node: The Postinstallation script, Up: Cross-compiling on Windows + +17.5.3.1 The Postinstallation script +.................................... + +Starting with Python 2.3, a postinstallation script can be specified +with the ‘--install-script’ option. The basename of the script must be +specified, and the script filename must also be listed in the scripts +argument to the setup function. + +This script will be run at installation time on the target system after +all the files have been copied, with ‘argv[1]’ set to ‘-install’, and +again at uninstallation time before the files are removed with ‘argv[1]’ +set to ‘-remove’. + +The installation script runs embedded in the windows installer, every +output (‘sys.stdout’, ‘sys.stderr’) is redirected into a buffer and will +be displayed in the GUI after the script has finished. + +Some functions especially useful in this context are available as +additional built-in functions in the installation script. + + -- Function: directory_created (path) + -- Function: file_created (path) + + These functions should be called when a directory or file is + created by the postinstall script at installation time. It will + register `path' with the uninstaller, so that it will be removed + when the distribution is uninstalled. To be safe, directories are + only removed if they are empty. + + -- Function: get_special_folder_path (csidl_string) + + This function can be used to retrieve special folder locations on + Windows like the Start Menu or the Desktop. It returns the full + path to the folder. `csidl_string' must be one of the following + strings: + + "CSIDL_APPDATA" + + "CSIDL_COMMON_STARTMENU" + "CSIDL_STARTMENU" + + "CSIDL_COMMON_DESKTOPDIRECTORY" + "CSIDL_DESKTOPDIRECTORY" + + "CSIDL_COMMON_STARTUP" + "CSIDL_STARTUP" + + "CSIDL_COMMON_PROGRAMS" + "CSIDL_PROGRAMS" + + "CSIDL_FONTS" + + If the folder cannot be retrieved, *note OSError: 1d3. is raised. + + Which folders are available depends on the exact Windows version, + and probably also the configuration. For details refer to + Microsoft’s documentation of the ‘SHGetSpecialFolderPath()’ + function. + + -- Function: create_shortcut (target, description, filename[, + arguments[, workdir[, iconpath[, iconindex]]]]) + + This function creates a shortcut. `target' is the path to the + program to be started by the shortcut. `description' is the + description of the shortcut. `filename' is the title of the + shortcut that the user will see. `arguments' specifies the command + line arguments, if any. `workdir' is the working directory for the + program. `iconpath' is the file containing the icon for the + shortcut, and `iconindex' is the index of the icon in the file + `iconpath'. Again, for details consult the Microsoft documentation + for the ‘IShellLink’ interface. + + +File: python.info, Node: Vista User Access Control UAC, Prev: Cross-compiling on Windows, Up: Creating Built Distributions + +17.5.4 Vista User Access Control (UAC) +-------------------------------------- + +Starting with Python 2.6, bdist_wininst supports a +‘--user-access-control’ option. The default is ‘none’ (meaning no UAC +handling is done), and other valid values are ‘auto’ (meaning prompt for +UAC elevation if Python was installed for all users) and ‘force’ +(meaning always prompt for elevation). + + Note: bdist_wininst is deprecated since Python 3.8. + + +File: python.info, Node: Distutils Examples, Next: Extending Distutils, Prev: Creating Built Distributions, Up: Distributing Python Modules Legacy version + +17.6 Distutils Examples +======================= + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +This chapter provides a number of basic examples to help get started +with distutils. Additional information about using distutils can be +found in the Distutils Cookbook. + +See also +........ + +Distutils Cookbook(1) + + Collection of recipes showing how to achieve more control over + distutils. + +* Menu: + +* Pure Python distribution (by module): Pure Python distribution by module. +* Pure Python distribution (by package): Pure Python distribution by package. +* Single extension module:: +* Checking a package:: +* Reading the metadata:: + + ---------- Footnotes ---------- + + (1) https://wiki.python.org/moin/Distutils/Cookbook + + +File: python.info, Node: Pure Python distribution by module, Next: Pure Python distribution by package, Up: Distutils Examples + +17.6.1 Pure Python distribution (by module) +------------------------------------------- + +If you’re just distributing a couple of modules, especially if they +don’t live in a particular package, you can specify them individually +using the ‘py_modules’ option in the setup script. + +In the simplest case, you’ll have two files to worry about: a setup +script and the single module you’re distributing, ‘foo.py’ in this +example: + + / + setup.py + foo.py + +(In all diagrams in this section, `' will refer to the +distribution root directory.) A minimal setup script to describe this +situation would be: + + from distutils.core import setup + setup(name='foo', + version='1.0', + py_modules=['foo'], + ) + +Note that the name of the distribution is specified independently with +the ‘name’ option, and there’s no rule that says it has to be the same +as the name of the sole module in the distribution (although that’s +probably a good convention to follow). However, the distribution name +is used to generate filenames, so you should stick to letters, digits, +underscores, and hyphens. + +Since ‘py_modules’ is a list, you can of course specify multiple +modules, eg. if you’re distributing modules ‘foo’ and ‘bar’, your setup +might look like this: + + / + setup.py + foo.py + bar.py + +and the setup script might be + + from distutils.core import setup + setup(name='foobar', + version='1.0', + py_modules=['foo', 'bar'], + ) + +You can put module source files into another directory, but if you have +enough modules to do that, it’s probably easier to specify modules by +package rather than listing them individually. + + +File: python.info, Node: Pure Python distribution by package, Next: Single extension module, Prev: Pure Python distribution by module, Up: Distutils Examples + +17.6.2 Pure Python distribution (by package) +-------------------------------------------- + +If you have more than a couple of modules to distribute, especially if +they are in multiple packages, it’s probably easier to specify whole +packages rather than individual modules. This works even if your +modules are not in a package; you can just tell the Distutils to process +modules from the root package, and that works the same as any other +package (except that you don’t have to have an ‘__init__.py’ file). + +The setup script from the last example could also be written as + + from distutils.core import setup + setup(name='foobar', + version='1.0', + packages=[''], + ) + +(The empty string stands for the root package.) + +If those two files are moved into a subdirectory, but remain in the root +package, e.g.: + + / + setup.py + src/ foo.py + bar.py + +then you would still specify the root package, but you have to tell the +Distutils where source files in the root package live: + + from distutils.core import setup + setup(name='foobar', + version='1.0', + package_dir={'': 'src'}, + packages=[''], + ) + +More typically, though, you will want to distribute multiple modules in +the same package (or in sub-packages). For example, if the ‘foo’ and +‘bar’ modules belong in package ‘foobar’, one way to layout your source +tree is + + / + setup.py + foobar/ + __init__.py + foo.py + bar.py + +This is in fact the default layout expected by the Distutils, and the +one that requires the least work to describe in your setup script: + + from distutils.core import setup + setup(name='foobar', + version='1.0', + packages=['foobar'], + ) + +If you want to put modules in directories not named for their package, +then you need to use the ‘package_dir’ option again. For example, if +the ‘src’ directory holds modules in the ‘foobar’ package: + + / + setup.py + src/ + __init__.py + foo.py + bar.py + +an appropriate setup script would be + + from distutils.core import setup + setup(name='foobar', + version='1.0', + package_dir={'foobar': 'src'}, + packages=['foobar'], + ) + +Or, you might put modules from your main package right in the +distribution root: + + / + setup.py + __init__.py + foo.py + bar.py + +in which case your setup script would be + + from distutils.core import setup + setup(name='foobar', + version='1.0', + package_dir={'foobar': ''}, + packages=['foobar'], + ) + +(The empty string also stands for the current directory.) + +If you have sub-packages, they must be explicitly listed in ‘packages’, +but any entries in ‘package_dir’ automatically extend to sub-packages. +(In other words, the Distutils does `not' scan your source tree, trying +to figure out which directories correspond to Python packages by looking +for ‘__init__.py’ files.) Thus, if the default layout grows a +sub-package: + + / + setup.py + foobar/ + __init__.py + foo.py + bar.py + subfoo/ + __init__.py + blah.py + +then the corresponding setup script would be + + from distutils.core import setup + setup(name='foobar', + version='1.0', + packages=['foobar', 'foobar.subfoo'], + ) + + +File: python.info, Node: Single extension module, Next: Checking a package, Prev: Pure Python distribution by package, Up: Distutils Examples + +17.6.3 Single extension module +------------------------------ + +Extension modules are specified using the ‘ext_modules’ option. +‘package_dir’ has no effect on where extension source files are found; +it only affects the source for pure Python modules. The simplest case, +a single extension module in a single C source file, is: + + / + setup.py + foo.c + +If the ‘foo’ extension belongs in the root package, the setup script for +this could be + + from distutils.core import setup + from distutils.extension import Extension + setup(name='foobar', + version='1.0', + ext_modules=[Extension('foo', ['foo.c'])], + ) + +If the extension actually belongs in a package, say ‘foopkg’, then + +With exactly the same source tree layout, this extension can be put in +the ‘foopkg’ package simply by changing the name of the extension: + + from distutils.core import setup + from distutils.extension import Extension + setup(name='foobar', + version='1.0', + ext_modules=[Extension('foopkg.foo', ['foo.c'])], + ) + + +File: python.info, Node: Checking a package, Next: Reading the metadata, Prev: Single extension module, Up: Distutils Examples + +17.6.4 Checking a package +------------------------- + +The ‘check’ command allows you to verify if your package meta-data meet +the minimum requirements to build a distribution. + +To run it, just call it using your ‘setup.py’ script. If something is +missing, ‘check’ will display a warning. + +Let’s take an example with a simple script: + + from distutils.core import setup + + setup(name='foobar') + +Running the ‘check’ command will display some warnings: + + $ python setup.py check + running check + warning: check: missing required meta-data: version, url + warning: check: missing meta-data: either (author and author_email) or + (maintainer and maintainer_email) must be supplied + +If you use the reStructuredText syntax in the ‘long_description’ field +and docutils(1) is installed you can check if the syntax is fine with +the ‘check’ command, using the ‘restructuredtext’ option. + +For example, if the ‘setup.py’ script is changed like this: + + from distutils.core import setup + + desc = """\ + My description + ============== + + This is the description of the ``foobar`` package. + """ + + setup(name='foobar', version='1', author='tarek', + author_email='tarek@ziade.org', + url='http://example.com', long_description=desc) + +Where the long description is broken, ‘check’ will be able to detect it +by using the ‘docutils’ parser: + + $ python setup.py check --restructuredtext + running check + warning: check: Title underline too short. (line 2) + warning: check: Could not finish the parsing. + + ---------- Footnotes ---------- + + (1) http://docutils.sourceforge.net + + +File: python.info, Node: Reading the metadata, Prev: Checking a package, Up: Distutils Examples + +17.6.5 Reading the metadata +--------------------------- + +The *note distutils.core.setup(): 38b8. function provides a command-line +interface that allows you to query the metadata fields of a project +through the ‘setup.py’ script of a given project: + + $ python setup.py --name + distribute + +This call reads the ‘name’ metadata by running the *note +distutils.core.setup(): 38b8. function. Although, when a source or +binary distribution is created with Distutils, the metadata fields are +written in a static file called ‘PKG-INFO’. When a Distutils-based +project is installed in Python, the ‘PKG-INFO’ file is copied alongside +the modules and packages of the distribution under +‘NAME-VERSION-pyX.X.egg-info’, where ‘NAME’ is the name of the project, +‘VERSION’ its version as defined in the Metadata, and ‘pyX.X’ the major +and minor version of Python like ‘2.7’ or ‘3.2’. + +You can read back this static file, by using the +‘distutils.dist.DistributionMetadata’ class and its ‘read_pkg_file()’ +method: + + >>> from distutils.dist import DistributionMetadata + >>> metadata = DistributionMetadata() + >>> metadata.read_pkg_file(open('distribute-0.6.8-py2.7.egg-info')) + >>> metadata.name + 'distribute' + >>> metadata.version + '0.6.8' + >>> metadata.description + 'Easily download, build, install, upgrade, and uninstall Python packages' + +Notice that the class can also be instantiated with a metadata file path +to loads its values: + + >>> pkg_info_path = 'distribute-0.6.8-py2.7.egg-info' + >>> DistributionMetadata(pkg_info_path).name + 'distribute' + + +File: python.info, Node: Extending Distutils, Next: Command Reference, Prev: Distutils Examples, Up: Distributing Python Modules Legacy version + +17.7 Extending Distutils +======================== + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +Distutils can be extended in various ways. Most extensions take the +form of new commands or replacements for existing commands. New +commands may be written to support new types of platform-specific +packaging, for example, while replacements for existing commands may be +made to modify details of how the command operates on a package. + +Most extensions of the distutils are made within ‘setup.py’ scripts that +want to modify existing commands; many simply add a few file extensions +that should be copied into packages in addition to ‘.py’ files as a +convenience. + +Most distutils command implementations are subclasses of the *note +distutils.cmd.Command: 4107. class. New commands may directly inherit +from ‘Command’, while replacements often derive from ‘Command’ +indirectly, directly subclassing the command they are replacing. +Commands are required to derive from ‘Command’. + +* Menu: + +* Integrating new commands:: +* Adding new distribution types:: + + +File: python.info, Node: Integrating new commands, Next: Adding new distribution types, Up: Extending Distutils + +17.7.1 Integrating new commands +------------------------------- + +There are different ways to integrate new command implementations into +distutils. The most difficult is to lobby for the inclusion of the new +features in distutils itself, and wait for (and require) a version of +Python that provides that support. This is really hard for many +reasons. + +The most common, and possibly the most reasonable for most needs, is to +include the new implementations with your ‘setup.py’ script, and cause +the *note distutils.core.setup(): 38b8. function use them: + + from distutils.command.build_py import build_py as _build_py + from distutils.core import setup + + class build_py(_build_py): + """Specialized Python source builder.""" + + # implement whatever needs to be different... + + setup(cmdclass={'build_py': build_py}, + ...) + +This approach is most valuable if the new implementations must be used +to use a particular package, as everyone interested in the package will +need to have the new command implementation. + +Beginning with Python 2.4, a third option is available, intended to +allow new commands to be added which can support existing ‘setup.py’ +scripts without requiring modifications to the Python installation. +This is expected to allow third-party extensions to provide support for +additional packaging systems, but the commands can be used for anything +distutils commands can be used for. A new configuration option, +‘command_packages’ (command-line option ‘--command-packages’), can be +used to specify additional packages to be searched for modules +implementing commands. Like all distutils options, this can be +specified on the command line or in a configuration file. This option +can only be set in the ‘[global]’ section of a configuration file, or +before any commands on the command line. If set in a configuration +file, it can be overridden from the command line; setting it to an empty +string on the command line causes the default to be used. This should +never be set in a configuration file provided with a package. + +This new option can be used to add any number of packages to the list of +packages searched for command implementations; multiple package names +should be separated by commas. When not specified, the search is only +performed in the *note distutils.command: 3e. package. When ‘setup.py’ +is run with the option ‘--command-packages distcmds,buildcmds’, however, +the packages *note distutils.command: 3e, ‘distcmds’, and ‘buildcmds’ +will be searched in that order. New commands are expected to be +implemented in modules of the same name as the command by classes +sharing the same name. Given the example command line option above, the +command ‘bdist_openpkg’ could be implemented by the class +‘distcmds.bdist_openpkg.bdist_openpkg’ or +‘buildcmds.bdist_openpkg.bdist_openpkg’. + + +File: python.info, Node: Adding new distribution types, Prev: Integrating new commands, Up: Extending Distutils + +17.7.2 Adding new distribution types +------------------------------------ + +Commands that create distributions (files in the ‘dist/’ directory) need +to add ‘(command, filename)’ pairs to ‘self.distribution.dist_files’ so +that ‘upload’ can upload it to PyPI. The `filename' in the pair contains +no path information, only the name of the file itself. In dry-run mode, +pairs should still be added to represent what would have been created. + + +File: python.info, Node: Command Reference, Next: API Reference, Prev: Extending Distutils, Up: Distributing Python Modules Legacy version + +17.8 Command Reference +====================== + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +* Menu: + +* Installing modules; the install command family: Installing modules the install command family. +* Creating a source distribution; the sdist command: Creating a source distribution the sdist command. + + +File: python.info, Node: Installing modules the install command family, Next: Creating a source distribution the sdist command, Up: Command Reference + +17.8.1 Installing modules: the ‘install’ command family +------------------------------------------------------- + +The install command ensures that the build commands have been run and +then runs the subcommands ‘install_lib’, ‘install_data’ and +‘install_scripts’. + +* Menu: + +* install_data:: +* install_scripts:: + + +File: python.info, Node: install_data, Next: install_scripts, Up: Installing modules the install command family + +17.8.1.1 ‘install_data’ +....................... + +This command installs all data files provided with the distribution. + + +File: python.info, Node: install_scripts, Prev: install_data, Up: Installing modules the install command family + +17.8.1.2 ‘install_scripts’ +.......................... + +This command installs all (Python) scripts in the distribution. + + +File: python.info, Node: Creating a source distribution the sdist command, Prev: Installing modules the install command family, Up: Command Reference + +17.8.2 Creating a source distribution: the ‘sdist’ command +---------------------------------------------------------- + +The manifest template commands are: + +Command Description + +---------------------------------------------------------------------------------------------------- + +‘include pat1 pat2 ...’ include all files matching any of the listed + patterns + + +‘exclude pat1 pat2 ...’ exclude all files matching any of the listed + patterns + + +‘recursive-include dir pat1 pat2 ...’ include all files under `dir' matching any of the + listed patterns + + +‘recursive-exclude dir pat1 pat2 ...’ exclude all files under `dir' matching any of the + listed patterns + + +‘global-include pat1 pat2 ...’ include all files anywhere in the source tree + matching — & any of the listed patterns + + +‘global-exclude pat1 pat2 ...’ exclude all files anywhere in the source tree + matching — & any of the listed patterns + + +‘prune dir’ exclude all files under `dir' + + +‘graft dir’ include all files under `dir' + + +The patterns here are Unix-style “glob” patterns: ‘*’ matches any +sequence of regular filename characters, ‘?’ matches any single regular +filename character, and ‘[range]’ matches any of the characters in +`range' (e.g., ‘a-z’, ‘a-zA-Z’, ‘a-f0-9_.’). The definition of “regular +filename character” is platform-specific: on Unix it is anything except +slash; on Windows anything except backslash or colon. + + +File: python.info, Node: API Reference, Prev: Command Reference, Up: Distributing Python Modules Legacy version + +17.9 API Reference +================== + +See also +........ + +New and changed setup.py arguments in setuptools(1) + + The ‘setuptools’ project adds new capabilities to the ‘setup’ + function and other APIs, makes the API consistent across different + Python versions, and is hence recommended over using ‘distutils’ + directly. + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + +* Menu: + +* distutils.core — Core Distutils functionality: distutils core — Core Distutils functionality. +* distutils.ccompiler — CCompiler base class: distutils ccompiler — CCompiler base class. +* distutils.unixccompiler — Unix C Compiler: distutils unixccompiler — Unix C Compiler. +* distutils.msvccompiler — Microsoft Compiler: distutils msvccompiler — Microsoft Compiler. +* distutils.bcppcompiler — Borland Compiler: distutils bcppcompiler — Borland Compiler. +* distutils.cygwincompiler — Cygwin Compiler: distutils cygwincompiler — Cygwin Compiler. +* distutils.archive_util — Archiving utilities: distutils archive_util — Archiving utilities. +* distutils.dep_util — Dependency checking: distutils dep_util — Dependency checking. +* distutils.dir_util — Directory tree operations: distutils dir_util — Directory tree operations. +* distutils.file_util — Single file operations: distutils file_util — Single file operations. +* distutils.util — Miscellaneous other utility functions: distutils util — Miscellaneous other utility functions. +* distutils.dist — The Distribution class: distutils dist — The Distribution class. +* distutils.extension — The Extension class: distutils extension — The Extension class. +* distutils.debug — Distutils debug mode: distutils debug — Distutils debug mode. +* distutils.errors — Distutils exceptions: distutils errors — Distutils exceptions. +* distutils.fancy_getopt — Wrapper around the standard getopt module: distutils fancy_getopt — Wrapper around the standard getopt module. +* distutils.filelist — The FileList class: distutils filelist — The FileList class. +* distutils.log — Simple PEP 282-style logging: distutils log — Simple PEP 282-style logging. +* distutils.spawn — Spawn a sub-process: distutils spawn — Spawn a sub-process. +* distutils.sysconfig — System configuration information: distutils sysconfig — System configuration information. +* distutils.text_file — The TextFile class: distutils text_file — The TextFile class. +* distutils.version — Version number classes: distutils version — Version number classes. +* distutils.cmd — Abstract base class for Distutils commands: distutils cmd — Abstract base class for Distutils commands. +* Creating a new Distutils command:: +* distutils.command — Individual Distutils commands: distutils command — Individual Distutils commands. +* distutils.command.bdist — Build a binary installer: distutils command bdist — Build a binary installer. +* distutils.command.bdist_packager — Abstract base class for packagers: distutils command bdist_packager — Abstract base class for packagers. +* distutils.command.bdist_dumb — Build a “dumb” installer: distutils command bdist_dumb — Build a “dumb” installer. +* distutils.command.bdist_msi — Build a Microsoft Installer binary package: distutils command bdist_msi — Build a Microsoft Installer binary package. +* distutils.command.bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM: distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM. +* distutils.command.bdist_wininst — Build a Windows installer: distutils command bdist_wininst — Build a Windows installer. +* distutils.command.sdist — Build a source distribution: distutils command sdist — Build a source distribution. +* distutils.command.build — Build all files of a package: distutils command build — Build all files of a package. +* distutils.command.build_clib — Build any C libraries in a package: distutils command build_clib — Build any C libraries in a package. +* distutils.command.build_ext — Build any extensions in a package: distutils command build_ext — Build any extensions in a package. +* distutils.command.build_py — Build the .py/.pyc files of a package: distutils command build_py — Build the py/ pyc files of a package. +* distutils.command.build_scripts — Build the scripts of a package: distutils command build_scripts — Build the scripts of a package. +* distutils.command.clean — Clean a package build area: distutils command clean — Clean a package build area. +* distutils.command.config — Perform package configuration: distutils command config — Perform package configuration. +* distutils.command.install — Install a package: distutils command install — Install a package. +* distutils.command.install_data — Install data files from a package: distutils command install_data — Install data files from a package. +* distutils.command.install_headers — Install C/C++ header files from a package: distutils command install_headers — Install C/C++ header files from a package. +* distutils.command.install_lib — Install library files from a package: distutils command install_lib — Install library files from a package. +* distutils.command.install_scripts — Install script files from a package: distutils command install_scripts — Install script files from a package. +* distutils.command.register — Register a module with the Python Package Index: distutils command register — Register a module with the Python Package Index. +* distutils.command.check — Check the meta-data of a package: distutils command check — Check the meta-data of a package. + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#new-and-changed-setup-keywords + + +File: python.info, Node: distutils core — Core Distutils functionality, Next: distutils ccompiler — CCompiler base class, Up: API Reference + +17.9.1 ‘distutils.core’ — Core Distutils functionality +------------------------------------------------------ + +The *note distutils.core: 54. module is the only module that needs to be +installed to use the Distutils. It provides the *note setup(): 38b8. +(which is called from the setup script). Indirectly provides the +‘distutils.dist.Distribution’ and *note distutils.cmd.Command: 4107. +class. + + -- Function: distutils.core.setup (arguments) + + The basic do-everything function that does most everything you + could ever ask for from a Distutils method. + + The setup function takes a large number of arguments. These are + laid out in the following table. + + argument name value type + + -------------------------------------------------------------------------------------------------------------------------------- + + `name' The name of the package a string + + + `version' The version number of the package; a string + see *note distutils.version: 66. + + + `description' A single line describing the a string + package + + + `long_description' Longer description of the package a string + + + `author' The name of the package author a string + + + `author_email' The email address of the package a string + author + + + `maintainer' The name of the current a string + maintainer, if different from the + author. Note that if the + maintainer is provided, distutils + will use it as the author in + ‘PKG-INFO’ + + + `maintainer_email' The email address of the current a string + maintainer, if different from the + author + + + `url' A URL for the package (homepage) a string + + + `download_url' A URL to download the package a string + + + `packages' A list of Python packages that a list of strings + distutils will manipulate + + + `py_modules' A list of Python modules that a list of strings + distutils will manipulate + + + `scripts' A list of standalone script files a list of strings + to be built and installed + + + `ext_modules' A list of Python extensions to be a list of instances of *note distutils.core.Extension: 40cd. + built + + + `classifiers' A list of categories for the a list of strings; valid classifiers are listed on PyPI(1). + package + + + `distclass' the *note Distribution: 4118. a subclass of *note distutils.core.Distribution: 4118. + class to use + + + `script_name' The name of the setup.py script - a string + defaults to ‘sys.argv[0]’ + + + `script_args' Arguments to supply to the setup a list of strings + script + + + `options' default options for the setup a dictionary + script + + + `license' The license for the package a string + + + `keywords' Descriptive meta-data, see PEP a list of strings or a comma-separated string + 314(2) + + + `platforms' a list of strings or a comma-separated string + + + `cmdclass' A mapping of command names to a dictionary + *note Command: 4119. subclasses + + + `data_files' A list of data files to install a list + + + `package_dir' A mapping of package to directory a dictionary + names + + + -- Function: distutils.core.run_setup (script_name[, script_args=None, + stop_after='run']) + + Run a setup script in a somewhat controlled environment, and return + the ‘distutils.dist.Distribution’ instance that drives things. + This is useful if you need to find out the distribution meta-data + (passed as keyword args from `script' to *note setup(): 38b8.), or + the contents of the config files or command-line. + + `script_name' is a file that will be read and run with *note + exec(): c44. ‘sys.argv[0]’ will be replaced with `script' for the + duration of the call. `script_args' is a list of strings; if + supplied, ‘sys.argv[1:]’ will be replaced by `script_args' for the + duration of the call. + + `stop_after' tells *note setup(): 38b8. when to stop processing; + possible values: + + value description + + ---------------------------------------------------------------------- + + `init' Stop after the *note Distribution: 4118. + instance has been created and populated with + the keyword arguments to *note setup(): 38b8. + + + `config' Stop after config files have been parsed (and + their data stored in the + *note Distribution: 4118. instance) + + + `commandline' Stop after the command-line (‘sys.argv[1:]’ or + `script_args') have been parsed (and the data + stored in the *note Distribution: 4118. + instance.) + + + `run' Stop after all commands have been run (the same + as if *note setup(): 38b8. had been called in + the usual way). This is the default value. + + +In addition, the *note distutils.core: 54. module exposed a number of +classes that live elsewhere. + + * ‘Extension’ from *note distutils.extension: 5b. + + * *note Command: 4107. from *note distutils.cmd: 3d. + + * ‘Distribution’ from *note distutils.dist: 59. + +A short description of each of these follows, but see the relevant +module for the full reference. + + -- Class: distutils.core.Extension + + The Extension class describes a single C or C++ extension module in + a setup script. It accepts the following keyword arguments in its + constructor: + + argument name value type + + -------------------------------------------------------------------------------------------------- + + `name' the full name of the extension, a string + including any packages — ie. + `not' a filename or pathname, but + Python dotted name + + + `sources' list of source filenames, relative a list of strings + to the distribution root (where + the setup script lives), in Unix + form (slash-separated) for + portability. Source files may be + C, C++, SWIG (.i), + platform-specific resource files, + or whatever else is recognized by + the ‘build_ext’ command as source + for a Python extension. + + + `include_dirs' list of directories to search for a list of strings + C/C++ header files (in Unix form + for portability) + + + `define_macros' list of macros to define; each a list of tuples + macro is defined using a 2-tuple + ‘(name, value)’, where `value' is + either the string to define it to + or ‘None’ to define it without a + particular value (equivalent of + ‘#define FOO’ in source or ‘-DFOO’ + on Unix C compiler command line) + + + `undef_macros' list of macros to undefine a list of strings + explicitly + + + `library_dirs' list of directories to search for a list of strings + C/C++ libraries at link time + + + `libraries' list of library names (not a list of strings + filenames or paths) to link + against + + + `runtime_library_dirs' list of directories to search for a list of strings + C/C++ libraries at run time (for + shared extensions, this is when + the extension is loaded) + + + `extra_objects' list of extra files to link with a list of strings + (eg. object files not implied by + ‘sources’, static library that + must be explicitly specified, + binary resource files, etc.) + + + `extra_compile_args' any extra platform- and a list of strings + compiler-specific information to + use when compiling the source + files in ‘sources’. For platforms + and compilers where a command line + makes sense, this is typically a + list of command-line arguments, + but for other platforms it could + be anything. + + + `extra_link_args' any extra platform- and a list of strings + compiler-specific information to + use when linking object files + together to create the extension + (or to create a new static Python + interpreter). Similar + interpretation as for + ‘extra_compile_args’. + + + `export_symbols' list of symbols to be exported a list of strings + from a shared extension. Not used + on all platforms, and not + generally necessary for Python + extensions, which typically export + exactly one symbol: ‘init’ + + extension_name. + + + `depends' list of files that the extension a list of strings + depends on + + + `language' extension language (i.e. ‘'c'’, a string + ‘'c++'’, ‘'objc'’). Will be + detected from the source + extensions if not provided. + + + `optional' specifies that a build failure in a boolean + the extension should not abort the + build process, but simply skip the + extension. + + + Changed in version 3.8: On Unix, C extensions are no longer linked + to libpython except on Android and Cygwin. + + -- Class: distutils.core.Distribution + + A *note Distribution: 4118. describes how to build, install and + package up a Python software package. + + See the *note setup(): 38b8. function for a list of keyword + arguments accepted by the Distribution constructor. *note setup(): + 38b8. creates a Distribution instance. + + Changed in version 3.7: *note Distribution: 4118. now warns if + ‘classifiers’, ‘keywords’ and ‘platforms’ fields are not specified + as a list or a string. + + -- Class: distutils.core.Command + + A *note Command: 4119. class (or rather, an instance of one of its + subclasses) implement a single distutils command. + + ---------- Footnotes ---------- + + (1) https://pypi.org/classifiers + + (2) https://www.python.org/dev/peps/pep-0314 + + +File: python.info, Node: distutils ccompiler — CCompiler base class, Next: distutils unixccompiler — Unix C Compiler, Prev: distutils core — Core Distutils functionality, Up: API Reference + +17.9.2 ‘distutils.ccompiler’ — CCompiler base class +--------------------------------------------------- + +This module provides the abstract base class for the *note CCompiler: +411c. classes. A *note CCompiler: 411c. instance can be used for all +the compile and link steps needed to build a single project. Methods +are provided to set options for the compiler — macro definitions, +include directories, link path, libraries and the like. + +This module provides the following functions. + + -- Function: distutils.ccompiler.gen_lib_options (compiler, + library_dirs, runtime_library_dirs, libraries) + + Generate linker options for searching library directories and + linking with specific libraries. `libraries' and `library_dirs' + are, respectively, lists of library names (not filenames!) and + search directories. Returns a list of command-line options + suitable for use with some compiler (depending on the two format + strings passed in). + + -- Function: distutils.ccompiler.gen_preprocess_options (macros, + include_dirs) + + Generate C pre-processor options (‘-D’, ‘-U’, ‘-I’) as used by at + least two types of compilers: the typical Unix compiler and Visual + C++. `macros' is the usual thing, a list of 1- or 2-tuples, where + ‘(name,)’ means undefine (‘-U’) macro `name', and ‘(name, value)’ + means define (‘-D’) macro `name' to `value'. `include_dirs' is + just a list of directory names to be added to the header file + search path (‘-I’). Returns a list of command-line options + suitable for either Unix compilers or Visual C++. + + -- Function: distutils.ccompiler.get_default_compiler (osname, + platform) + + Determine the default compiler to use for the given platform. + + `osname' should be one of the standard Python OS names (i.e. the + ones returned by ‘os.name’) and `platform' the common value + returned by ‘sys.platform’ for the platform in question. + + The default values are ‘os.name’ and ‘sys.platform’ in case the + parameters are not given. + + -- Function: distutils.ccompiler.new_compiler (plat=None, + compiler=None, verbose=0, dry_run=0, force=0) + + Factory function to generate an instance of some CCompiler subclass + for the supplied platform/compiler combination. `plat' defaults to + ‘os.name’ (eg. ‘'posix'’, ‘'nt'’), and `compiler' defaults to the + default compiler for that platform. Currently only ‘'posix'’ and + ‘'nt'’ are supported, and the default compilers are “traditional + Unix interface” (‘UnixCCompiler’ class) and Visual C++ + (‘MSVCCompiler’ class). Note that it’s perfectly possible to ask + for a Unix compiler object under Windows, and a Microsoft compiler + object under Unix—if you supply a value for `compiler', `plat' is + ignored. + + -- Function: distutils.ccompiler.show_compilers () + + Print list of available compilers (used by the ‘--help-compiler’ + options to ‘build’, ‘build_ext’, ‘build_clib’). + + -- Class: distutils.ccompiler.CCompiler ([verbose=0, dry_run=0, + force=0]) + + The abstract base class *note CCompiler: 411c. defines the + interface that must be implemented by real compiler classes. The + class also has some utility methods used by several compiler + classes. + + The basic idea behind a compiler abstraction class is that each + instance can be used for all the compile/link steps in building a + single project. Thus, attributes common to all of those compile + and link steps — include directories, macros to define, libraries + to link against, etc. — are attributes of the compiler instance. + To allow for variability in how individual files are treated, most + of those attributes may be varied on a per-compilation or per-link + basis. + + The constructor for each subclass creates an instance of the + Compiler object. Flags are `verbose' (show verbose output), + `dry_run' (don’t actually execute the steps) and `force' (rebuild + everything, regardless of dependencies). All of these flags + default to ‘0’ (off). Note that you probably don’t want to + instantiate *note CCompiler: 411c. or one of its subclasses + directly - use the ‘distutils.CCompiler.new_compiler()’ factory + function instead. + + The following methods allow you to manually alter compiler options + for the instance of the Compiler class. + + -- Method: add_include_dir (dir) + + Add `dir' to the list of directories that will be searched for + header files. The compiler is instructed to search + directories in the order in which they are supplied by + successive calls to *note add_include_dir(): 4122. + + -- Method: set_include_dirs (dirs) + + Set the list of directories that will be searched to `dirs' (a + list of strings). Overrides any preceding calls to *note + add_include_dir(): 4122.; subsequent calls to *note + add_include_dir(): 4122. add to the list passed to *note + set_include_dirs(): 4123. This does not affect any list of + standard include directories that the compiler may search by + default. + + -- Method: add_library (libname) + + Add `libname' to the list of libraries that will be included + in all links driven by this compiler object. Note that + `libname' should *not* be the name of a file containing a + library, but the name of the library itself: the actual + filename will be inferred by the linker, the compiler, or the + compiler class (depending on the platform). + + The linker will be instructed to link against libraries in the + order they were supplied to *note add_library(): 4124. and/or + *note set_libraries(): 4125. It is perfectly valid to + duplicate library names; the linker will be instructed to link + against libraries as many times as they are mentioned. + + -- Method: set_libraries (libnames) + + Set the list of libraries to be included in all links driven + by this compiler object to `libnames' (a list of strings). + This does not affect any standard system libraries that the + linker may include by default. + + -- Method: add_library_dir (dir) + + Add `dir' to the list of directories that will be searched for + libraries specified to *note add_library(): 4124. and *note + set_libraries(): 4125. The linker will be instructed to + search for libraries in the order they are supplied to *note + add_library_dir(): 4126. and/or *note set_library_dirs(): + 4127. + + -- Method: set_library_dirs (dirs) + + Set the list of library search directories to `dirs' (a list + of strings). This does not affect any standard library search + path that the linker may search by default. + + -- Method: add_runtime_library_dir (dir) + + Add `dir' to the list of directories that will be searched for + shared libraries at runtime. + + -- Method: set_runtime_library_dirs (dirs) + + Set the list of directories to search for shared libraries at + runtime to `dirs' (a list of strings). This does not affect + any standard search path that the runtime linker may search by + default. + + -- Method: define_macro (name[, value=None]) + + Define a preprocessor macro for all compilations driven by + this compiler object. The optional parameter `value' should + be a string; if it is not supplied, then the macro will be + defined without an explicit value and the exact outcome + depends on the compiler used. + + -- Method: undefine_macro (name) + + Undefine a preprocessor macro for all compilations driven by + this compiler object. If the same macro is defined by *note + define_macro(): 412a. and undefined by *note undefine_macro(): + 412b. the last call takes precedence (including multiple + redefinitions or undefinitions). If the macro is + redefined/undefined on a per-compilation basis (ie. in the + call to *note compile(): 1b4.), then that takes precedence. + + -- Method: add_link_object (object) + + Add `object' to the list of object files (or analogues, such + as explicitly named library files or the output of “resource + compilers”) to be included in every link driven by this + compiler object. + + -- Method: set_link_objects (objects) + + Set the list of object files (or analogues) to be included in + every link to `objects'. This does not affect any standard + object files that the linker may include by default (such as + system libraries). + + The following methods implement methods for autodetection of + compiler options, providing some functionality similar to GNU + ‘autoconf’. + + -- Method: detect_language (sources) + + Detect the language of a given file, or list of files. Uses + the instance attributes ‘language_map’ (a dictionary), and + ‘language_order’ (a list) to do the job. + + -- Method: find_library_file (dirs, lib[, debug=0]) + + Search the specified list of directories for a static or + shared library file `lib' and return the full path to that + file. If `debug' is true, look for a debugging version (if + that makes sense on the current platform). Return ‘None’ if + `lib' wasn’t found in any of the specified directories. + + -- Method: has_function (funcname[, includes=None, + include_dirs=None, libraries=None, library_dirs=None]) + + Return a boolean indicating whether `funcname' is supported on + the current platform. The optional arguments can be used to + augment the compilation environment by providing additional + include files and paths and libraries and paths. + + -- Method: library_dir_option (dir) + + Return the compiler option to add `dir' to the list of + directories searched for libraries. + + -- Method: library_option (lib) + + Return the compiler option to add `lib' to the list of + libraries linked into the shared library or executable. + + -- Method: runtime_library_dir_option (dir) + + Return the compiler option to add `dir' to the list of + directories searched for runtime libraries. + + -- Method: set_executables (**args) + + Define the executables (and options for them) that will be run + to perform the various stages of compilation. The exact set + of executables that may be specified here depends on the + compiler class (via the ‘executables’ class attribute), but + most will have: + + attribute description + + ------------------------------------------------------------------ + + `compiler' the C/C++ compiler + + + `linker_so' linker used to create shared objects and + libraries + + + `linker_exe' linker used to create binary executables + + + `archiver' static library creator + + + On platforms with a command-line (Unix, DOS/Windows), each of + these is a string that will be split into executable name and + (optional) list of arguments. (Splitting the string is done + similarly to how Unix shells operate: words are delimited by + spaces, but quotes and backslashes can override this. See + *note distutils.util.split_quoted(): 4135.) + + The following methods invoke stages in the build process. + + -- Method: compile (sources[, output_dir=None, macros=None, + include_dirs=None, debug=0, extra_preargs=None, + extra_postargs=None, depends=None]) + + Compile one or more source files. Generates object files + (e.g. transforms a ‘.c’ file to a ‘.o’ file.) + + `sources' must be a list of filenames, most likely C/C++ + files, but in reality anything that can be handled by a + particular compiler and compiler class (eg. ‘MSVCCompiler’ + can handle resource files in `sources'). Return a list of + object filenames, one per source filename in `sources'. + Depending on the implementation, not all source files will + necessarily be compiled, but all corresponding object + filenames will be returned. + + If `output_dir' is given, object files will be put under it, + while retaining their original path component. That is, + ‘foo/bar.c’ normally compiles to ‘foo/bar.o’ (for a Unix + implementation); if `output_dir' is `build', then it would + compile to ‘build/foo/bar.o’. + + `macros', if given, must be a list of macro definitions. A + macro definition is either a ‘(name, value)’ 2-tuple or a + ‘(name,)’ 1-tuple. The former defines a macro; if the value + is ‘None’, the macro is defined without an explicit value. + The 1-tuple case undefines a macro. Later + definitions/redefinitions/undefinitions take precedence. + + `include_dirs', if given, must be a list of strings, the + directories to add to the default include file search path for + this compilation only. + + `debug' is a boolean; if true, the compiler will be instructed + to output debug symbols in (or alongside) the object file(s). + + `extra_preargs' and `extra_postargs' are + implementation-dependent. On platforms that have the notion + of a command-line (e.g. Unix, DOS/Windows), they are most + likely lists of strings: extra command-line arguments to + prepend/append to the compiler command line. On other + platforms, consult the implementation class documentation. In + any event, they are intended as an escape hatch for those + occasions when the abstract compiler framework doesn’t cut the + mustard. + + `depends', if given, is a list of filenames that all targets + depend on. If a source file is older than any file in + depends, then the source file will be recompiled. This + supports dependency tracking, but only at a coarse + granularity. + + Raises ‘CompileError’ on failure. + + -- Method: create_static_lib (objects, output_libname[, + output_dir=None, debug=0, target_lang=None]) + + Link a bunch of stuff together to create a static library + file. The “bunch of stuff” consists of the list of object + files supplied as `objects', the extra object files supplied + to *note add_link_object(): 412c. and/or *note + set_link_objects(): 412d, the libraries supplied to *note + add_library(): 4124. and/or *note set_libraries(): 4125, and + the libraries supplied as `libraries' (if any). + + `output_libname' should be a library name, not a filename; the + filename will be inferred from the library name. `output_dir' + is the directory where the library file will be put. + + `debug' is a boolean; if true, debugging information will be + included in the library (note that on most platforms, it is + the compile step where this matters: the `debug' flag is + included here just for consistency). + + `target_lang' is the target language for which the given + objects are being compiled. This allows specific linkage time + treatment of certain languages. + + Raises ‘LibError’ on failure. + + -- Method: link (target_desc, objects, output_filename[, + output_dir=None, libraries=None, library_dirs=None, + runtime_library_dirs=None, export_symbols=None, debug=0, + extra_preargs=None, extra_postargs=None, build_temp=None, + target_lang=None]) + + Link a bunch of stuff together to create an executable or + shared library file. + + The “bunch of stuff” consists of the list of object files + supplied as `objects'. `output_filename' should be a + filename. If `output_dir' is supplied, `output_filename' is + relative to it (i.e. `output_filename' can provide directory + components if needed). + + `libraries' is a list of libraries to link against. These are + library names, not filenames, since they’re translated into + filenames in a platform-specific way (eg. `foo' becomes + ‘libfoo.a’ on Unix and ‘foo.lib’ on DOS/Windows). However, + they can include a directory component, which means the linker + will look in that specific directory rather than searching all + the normal locations. + + `library_dirs', if supplied, should be a list of directories + to search for libraries that were specified as bare library + names (ie. no directory component). These are on top of the + system default and those supplied to *note add_library_dir(): + 4126. and/or *note set_library_dirs(): 4127. + `runtime_library_dirs' is a list of directories that will be + embedded into the shared library and used to search for other + shared libraries that *it* depends on at run-time. (This may + only be relevant on Unix.) + + `export_symbols' is a list of symbols that the shared library + will export. (This appears to be relevant only on Windows.) + + `debug' is as for *note compile(): 1b4. and *note + create_static_lib(): 4137, with the slight distinction that it + actually matters on most platforms (as opposed to *note + create_static_lib(): 4137, which includes a `debug' flag + mostly for form’s sake). + + `extra_preargs' and `extra_postargs' are as for *note + compile(): 1b4. (except of course that they supply + command-line arguments for the particular linker being used). + + `target_lang' is the target language for which the given + objects are being compiled. This allows specific linkage time + treatment of certain languages. + + Raises ‘LinkError’ on failure. + + -- Method: link_executable (objects, output_progname[, + output_dir=None, libraries=None, library_dirs=None, + runtime_library_dirs=None, debug=0, extra_preargs=None, + extra_postargs=None, target_lang=None]) + + Link an executable. `output_progname' is the name of the file + executable, while `objects' are a list of object filenames to + link in. Other arguments are as for the *note link(): 4138. + method. + + -- Method: link_shared_lib (objects, output_libname[, + output_dir=None, libraries=None, library_dirs=None, + runtime_library_dirs=None, export_symbols=None, debug=0, + extra_preargs=None, extra_postargs=None, build_temp=None, + target_lang=None]) + + Link a shared library. `output_libname' is the name of the + output library, while `objects' is a list of object filenames + to link in. Other arguments are as for the *note link(): + 4138. method. + + -- Method: link_shared_object (objects, output_filename[, + output_dir=None, libraries=None, library_dirs=None, + runtime_library_dirs=None, export_symbols=None, debug=0, + extra_preargs=None, extra_postargs=None, build_temp=None, + target_lang=None]) + + Link a shared object. `output_filename' is the name of the + shared object that will be created, while `objects' is a list + of object filenames to link in. Other arguments are as for + the *note link(): 4138. method. + + -- Method: preprocess (source[, output_file=None, macros=None, + include_dirs=None, extra_preargs=None, + extra_postargs=None]) + + Preprocess a single C/C++ source file, named in `source'. + Output will be written to file named `output_file', or + `stdout' if `output_file' not supplied. `macros' is a list of + macro definitions as for *note compile(): 1b4, which will + augment the macros set with *note define_macro(): 412a. and + *note undefine_macro(): 412b. `include_dirs' is a list of + directory names that will be added to the default list, in the + same way as *note add_include_dir(): 4122. + + Raises ‘PreprocessError’ on failure. + + The following utility methods are defined by the *note CCompiler: + 411c. class, for use by the various concrete subclasses. + + -- Method: executable_filename (basename[, strip_dir=0, + output_dir='']) + + Returns the filename of the executable for the given + `basename'. Typically for non-Windows platforms this is the + same as the basename, while Windows will get a ‘.exe’ added. + + -- Method: library_filename (libname[, lib_type='static', + strip_dir=0, output_dir='']) + + Returns the filename for the given library name on the current + platform. On Unix a library with `lib_type' of ‘'static'’ + will typically be of the form ‘liblibname.a’, while a + `lib_type' of ‘'dynamic'’ will be of the form ‘liblibname.so’. + + -- Method: object_filenames (source_filenames[, strip_dir=0, + output_dir='']) + + Returns the name of the object files for the given source + files. `source_filenames' should be a list of filenames. + + -- Method: shared_object_filename (basename[, strip_dir=0, + output_dir='']) + + Returns the name of a shared object file for the given file + name `basename'. + + -- Method: execute (func, args[, msg=None, level=1]) + + Invokes *note distutils.util.execute(): 4142. This method + invokes a Python function `func' with the given arguments + `args', after logging and taking into account the `dry_run' + flag. + + -- Method: spawn (cmd) + + Invokes ‘distutils.util.spawn()’. This invokes an external + process to run the given command. + + -- Method: mkpath (name[, mode=511]) + + Invokes *note distutils.dir_util.mkpath(): 4145. This creates + a directory and any missing ancestor directories. + + -- Method: move_file (src, dst) + + Invokes *note distutils.file_util.move_file(): 4147. Renames + `src' to `dst'. + + -- Method: announce (msg[, level=1]) + + Write a message using ‘distutils.log.debug()’. + + -- Method: warn (msg) + + Write a warning message `msg' to standard error. + + -- Method: debug_print (msg) + + If the `debug' flag is set on this *note CCompiler: 411c. + instance, print `msg' to standard output, otherwise do + nothing. + + +File: python.info, Node: distutils unixccompiler — Unix C Compiler, Next: distutils msvccompiler — Microsoft Compiler, Prev: distutils ccompiler — CCompiler base class, Up: API Reference + +17.9.3 ‘distutils.unixccompiler’ — Unix C Compiler +-------------------------------------------------- + +This module provides the ‘UnixCCompiler’ class, a subclass of +‘CCompiler’ that handles the typical Unix-style command-line C compiler: + + * macros defined with ‘-Dname[=value]’ + + * macros undefined with ‘-Uname’ + + * include search directories specified with ‘-Idir’ + + * libraries specified with ‘-llib’ + + * library search directories specified with ‘-Ldir’ + + * compile handled by ‘cc’ (or similar) executable with ‘-c’ option: + compiles ‘.c’ to ‘.o’ + + * link static library handled by ‘ar’ command (possibly with + ‘ranlib’) + + * link shared library handled by ‘cc’ ‘-shared’ + + +File: python.info, Node: distutils msvccompiler — Microsoft Compiler, Next: distutils bcppcompiler — Borland Compiler, Prev: distutils unixccompiler — Unix C Compiler, Up: API Reference + +17.9.4 ‘distutils.msvccompiler’ — Microsoft Compiler +---------------------------------------------------- + +This module provides ‘MSVCCompiler’, an implementation of the abstract +‘CCompiler’ class for Microsoft Visual Studio. Typically, extension +modules need to be compiled with the same compiler that was used to +compile Python. For Python 2.3 and earlier, the compiler was Visual +Studio 6. For Python 2.4 and 2.5, the compiler is Visual Studio .NET +2003. + +‘MSVCCompiler’ will normally choose the right compiler, linker etc. on +its own. To override this choice, the environment variables +`DISTUTILS_USE_SDK' and `MSSdk' must be both set. `MSSdk' indicates +that the current environment has been setup by the SDK’s ‘SetEnv.Cmd’ +script, or that the environment variables had been registered when the +SDK was installed; `DISTUTILS_USE_SDK' indicates that the distutils user +has made an explicit choice to override the compiler selection by +‘MSVCCompiler’. + + +File: python.info, Node: distutils bcppcompiler — Borland Compiler, Next: distutils cygwincompiler — Cygwin Compiler, Prev: distutils msvccompiler — Microsoft Compiler, Up: API Reference + +17.9.5 ‘distutils.bcppcompiler’ — Borland Compiler +-------------------------------------------------- + +This module provides ‘BorlandCCompiler’, a subclass of the abstract +‘CCompiler’ class for the Borland C++ compiler. + + +File: python.info, Node: distutils cygwincompiler — Cygwin Compiler, Next: distutils archive_util — Archiving utilities, Prev: distutils bcppcompiler — Borland Compiler, Up: API Reference + +17.9.6 ‘distutils.cygwincompiler’ — Cygwin Compiler +--------------------------------------------------- + +This module provides the ‘CygwinCCompiler’ class, a subclass of +‘UnixCCompiler’ that handles the Cygwin port of the GNU C compiler to +Windows. It also contains the Mingw32CCompiler class which handles the +mingw32 port of GCC (same as cygwin in no-cygwin mode). + + +File: python.info, Node: distutils archive_util — Archiving utilities, Next: distutils dep_util — Dependency checking, Prev: distutils cygwincompiler — Cygwin Compiler, Up: API Reference + +17.9.7 ‘distutils.archive_util’ — Archiving utilities +----------------------------------------------------- + +This module provides a few functions for creating archive files, such as +tarballs or zipfiles. + + -- Function: distutils.archive_util.make_archive (base_name, format[, + root_dir=None, base_dir=None, verbose=0, dry_run=0]) + + Create an archive file (eg. ‘zip’ or ‘tar’). `base_name' is the + name of the file to create, minus any format-specific extension; + `format' is the archive format: one of ‘zip’, ‘tar’, ‘gztar’, + ‘bztar’, ‘xztar’, or ‘ztar’. `root_dir' is a directory that will + be the root directory of the archive; ie. we typically ‘chdir’ + into `root_dir' before creating the archive. `base_dir' is the + directory where we start archiving from; ie. `base_dir' will be + the common prefix of all files and directories in the archive. + `root_dir' and `base_dir' both default to the current directory. + Returns the name of the archive file. + + Changed in version 3.5: Added support for the ‘xztar’ format. + + -- Function: distutils.archive_util.make_tarball (base_name, base_dir[, + compress='gzip', verbose=0, dry_run=0]) + + ‘Create an (optional compressed) archive as a tar file from all + files in and under `base_dir'. `compress' must be ‘'gzip'’ (the + default), ‘'bzip2'’, ‘'xz'’, ‘'compress'’, or ‘None’. For the + ‘'compress'’ method the compression utility named by ‘compress’ + must be on the default program search path, so this is probably + Unix-specific. The output tar file will be named ‘base_dir.tar’, + possibly plus the appropriate compression extension (‘.gz’, ‘.bz2’, + ‘.xz’ or ‘.Z’). Return the output filename. + + Changed in version 3.5: Added support for the ‘xz’ compression. + + -- Function: distutils.archive_util.make_zipfile (base_name, base_dir[, + verbose=0, dry_run=0]) + + Create a zip file from all files in and under `base_dir'. The + output zip file will be named `base_name' + ‘.zip’. Uses either + the *note zipfile: 142. Python module (if available) or the InfoZIP + ‘zip’ utility (if installed and found on the default search path). + If neither tool is available, raises ‘DistutilsExecError’. Returns + the name of the output zip file. + + +File: python.info, Node: distutils dep_util — Dependency checking, Next: distutils dir_util — Directory tree operations, Prev: distutils archive_util — Archiving utilities, Up: API Reference + +17.9.8 ‘distutils.dep_util’ — Dependency checking +------------------------------------------------- + +This module provides functions for performing simple, timestamp-based +dependency of files and groups of files; also, functions based entirely +on such timestamp dependency analysis. + + -- Function: distutils.dep_util.newer (source, target) + + Return true if `source' exists and is more recently modified than + `target', or if `source' exists and `target' doesn’t. Return false + if both exist and `target' is the same age or newer than `source'. + Raise ‘DistutilsFileError’ if `source' does not exist. + + -- Function: distutils.dep_util.newer_pairwise (sources, targets) + + Walk two filename lists in parallel, testing if each source is + newer than its corresponding target. Return a pair of lists + (`sources', `targets') where source is newer than target, according + to the semantics of *note newer(): 4154. + + -- Function: distutils.dep_util.newer_group (sources, target[, + missing='error']) + + Return true if `target' is out-of-date with respect to any file + listed in `sources'. In other words, if `target' exists and is + newer than every file in `sources', return false; otherwise return + true. `missing' controls what we do when a source file is missing; + the default (‘'error'’) is to blow up with an *note OSError: 1d3. + from inside *note os.stat(): 1f1.; if it is ‘'ignore'’, we silently + drop any missing source files; if it is ‘'newer'’, any missing + source files make us assume that `target' is out-of-date (this is + handy in “dry-run” mode: it’ll make you pretend to carry out + commands that wouldn’t work because inputs are missing, but that + doesn’t matter because you’re not actually going to run the + commands). + + +File: python.info, Node: distutils dir_util — Directory tree operations, Next: distutils file_util — Single file operations, Prev: distutils dep_util — Dependency checking, Up: API Reference + +17.9.9 ‘distutils.dir_util’ — Directory tree operations +------------------------------------------------------- + +This module provides functions for operating on directories and trees of +directories. + + -- Function: distutils.dir_util.mkpath (name[, mode=0o777, verbose=0, + dry_run=0]) + + Create a directory and any missing ancestor directories. If the + directory already exists (or if `name' is the empty string, which + means the current directory, which of course exists), then do + nothing. Raise ‘DistutilsFileError’ if unable to create some + directory along the way (eg. some sub-path exists, but is a file + rather than a directory). If `verbose' is true, print a one-line + summary of each mkdir to stdout. Return the list of directories + actually created. + + -- Function: distutils.dir_util.create_tree (base_dir, files[, + mode=0o777, verbose=0, dry_run=0]) + + Create all the empty directories under `base_dir' needed to put + `files' there. `base_dir' is just the name of a directory which + doesn’t necessarily exist yet; `files' is a list of filenames to be + interpreted relative to `base_dir'. `base_dir' + the directory + portion of every file in `files' will be created if it doesn’t + already exist. `mode', `verbose' and `dry_run' flags are as for + *note mkpath(): 4145. + + -- Function: distutils.dir_util.copy_tree (src, dst[, preserve_mode=1, + preserve_times=1, preserve_symlinks=0, update=0, verbose=0, + dry_run=0]) + + Copy an entire directory tree `src' to a new location `dst'. Both + `src' and `dst' must be directory names. If `src' is not a + directory, raise ‘DistutilsFileError’. If `dst' does not exist, it + is created with *note mkpath(): 4145. The end result of the copy + is that every file in `src' is copied to `dst', and directories + under `src' are recursively copied to `dst'. Return the list of + files that were copied or might have been copied, using their + output name. The return value is unaffected by `update' or + `dry_run': it is simply the list of all files under `src', with the + names changed to be under `dst'. + + `preserve_mode' and `preserve_times' are the same as for *note + distutils.file_util.copy_file(): 415a.; note that they only apply + to regular files, not to directories. If `preserve_symlinks' is + true, symlinks will be copied as symlinks (on platforms that + support them!); otherwise (the default), the destination of the + symlink will be copied. `update' and `verbose' are the same as for + ‘copy_file()’. + + Files in `src' that begin with ‘.nfs’ are skipped (more information + on these files is available in answer D2 of the NFS FAQ page(1)). + + Changed in version 3.3.1: NFS files are ignored. + + -- Function: distutils.dir_util.remove_tree (directory[, verbose=0, + dry_run=0]) + + Recursively remove `directory' and all files and directories + underneath it. Any errors are ignored (apart from being reported + to ‘sys.stdout’ if `verbose' is true). + + ---------- Footnotes ---------- + + (1) http://nfs.sourceforge.net/#section_d + + +File: python.info, Node: distutils file_util — Single file operations, Next: distutils util — Miscellaneous other utility functions, Prev: distutils dir_util — Directory tree operations, Up: API Reference + +17.9.10 ‘distutils.file_util’ — Single file operations +------------------------------------------------------ + +This module contains some utility functions for operating on individual +files. + + -- Function: distutils.file_util.copy_file (src, dst[, preserve_mode=1, + preserve_times=1, update=0, link=None, verbose=0, dry_run=0]) + + Copy file `src' to `dst'. If `dst' is a directory, then `src' is + copied there with the same name; otherwise, it must be a filename. + (If the file exists, it will be ruthlessly clobbered.) If + `preserve_mode' is true (the default), the file’s mode (type and + permission bits, or whatever is analogous on the current platform) + is copied. If `preserve_times' is true (the default), the + last-modified and last-access times are copied as well. If + `update' is true, `src' will only be copied if `dst' does not + exist, or if `dst' does exist but is older than `src'. + + `link' allows you to make hard links (using *note os.link(): a35.) + or symbolic links (using *note os.symlink(): a3b.) instead of + copying: set it to ‘'hard'’ or ‘'sym'’; if it is ‘None’ (the + default), files are copied. Don’t set `link' on systems that don’t + support it: *note copy_file(): 415a. doesn’t check if hard or + symbolic linking is available. It uses ‘_copy_file_contents()’ to + copy file contents. + + Return a tuple ‘(dest_name, copied)’: `dest_name' is the actual + name of the output file, and `copied' is true if the file was + copied (or would have been copied, if `dry_run' true). + + -- Function: distutils.file_util.move_file (src, dst[, verbose, + dry_run]) + + Move file `src' to `dst'. If `dst' is a directory, the file will + be moved into it with the same name; otherwise, `src' is just + renamed to `dst'. Returns the new full name of the file. + + Warning: Handles cross-device moves on Unix using *note + copy_file(): 415a. What about other systems? + + -- Function: distutils.file_util.write_file (filename, contents) + + Create a file called `filename' and write `contents' (a sequence of + strings without line terminators) to it. + + +File: python.info, Node: distutils util — Miscellaneous other utility functions, Next: distutils dist — The Distribution class, Prev: distutils file_util — Single file operations, Up: API Reference + +17.9.11 ‘distutils.util’ — Miscellaneous other utility functions +---------------------------------------------------------------- + +This module contains other assorted bits and pieces that don’t fit into +any other utility module. + + -- Function: distutils.util.get_platform () + + Return a string that identifies the current platform. This is used + mainly to distinguish platform-specific build directories and + platform-specific built distributions. Typically includes the OS + name and version and the architecture (as supplied by + ‘os.uname()’), although the exact information included depends on + the OS; e.g., on Linux, the kernel version isn’t particularly + important. + + Examples of returned values: + + * ‘linux-i586’ + + * ‘linux-alpha’ + + * ‘solaris-2.6-sun4u’ + + For non-POSIX platforms, currently just returns ‘sys.platform’. + + For Mac OS X systems the OS version reflects the minimal version on + which binaries will run (that is, the value of + ‘MACOSX_DEPLOYMENT_TARGET’ during the build of Python), not the OS + version of the current system. + + For universal binary builds on Mac OS X the architecture value + reflects the universal binary status instead of the architecture of + the current processor. For 32-bit universal binaries the + architecture is ‘fat’, for 64-bit universal binaries the + architecture is ‘fat64’, and for 4-way universal binaries the + architecture is ‘universal’. Starting from Python 2.7 and Python + 3.2 the architecture ‘fat3’ is used for a 3-way universal build + (ppc, i386, x86_64) and ‘intel’ is used for a universal build with + the i386 and x86_64 architectures + + Examples of returned values on Mac OS X: + + * ‘macosx-10.3-ppc’ + + * ‘macosx-10.3-fat’ + + * ‘macosx-10.5-universal’ + + * ‘macosx-10.6-intel’ + + -- Function: distutils.util.convert_path (pathname) + + Return ‘pathname’ as a name that will work on the native + filesystem, i.e. split it on ‘/’ and put it back together again + using the current directory separator. Needed because filenames in + the setup script are always supplied in Unix style, and have to be + converted to the local convention before we can actually use them + in the filesystem. Raises *note ValueError: 1fb. on non-Unix-ish + systems if `pathname' either starts or ends with a slash. + + -- Function: distutils.util.change_root (new_root, pathname) + + Return `pathname' with `new_root' prepended. If `pathname' is + relative, this is equivalent to ‘os.path.join(new_root,pathname)’ + Otherwise, it requires making `pathname' relative and then joining + the two, which is tricky on DOS/Windows. + + -- Function: distutils.util.check_environ () + + Ensure that ‘os.environ’ has all the environment variables we + guarantee that users can use in config files, command-line options, + etc. Currently this includes: + + * ‘HOME’ - user’s home directory (Unix only) + + * ‘PLAT’ - description of the current platform, including + hardware and OS (see *note get_platform(): 415f.) + + -- Function: distutils.util.subst_vars (s, local_vars) + + Perform shell/Perl-style variable substitution on `s'. Every + occurrence of ‘$’ followed by a name is considered a variable, and + variable is substituted by the value found in the `local_vars' + dictionary, or in ‘os.environ’ if it’s not in `local_vars'. + `os.environ' is first checked/augmented to guarantee that it + contains certain values: see *note check_environ(): 4162. Raise + *note ValueError: 1fb. for any variables not found in either + `local_vars' or ‘os.environ’. + + Note that this is not a fully-fledged string interpolation + function. A valid ‘$variable’ can consist only of upper and lower + case letters, numbers and an underscore. No { } or ( ) style + quoting is available. + + -- Function: distutils.util.split_quoted (s) + + Split a string up according to Unix shell-like rules for quotes and + backslashes. In short: words are delimited by spaces, as long as + those spaces are not escaped by a backslash, or inside a quoted + string. Single and double quotes are equivalent, and the quote + characters can be backslash-escaped. The backslash is stripped + from any two-character escape sequence, leaving only the escaped + character. The quote characters are stripped from any quoted + string. Returns a list of words. + + -- Function: distutils.util.execute (func, args[, msg=None, verbose=0, + dry_run=0]) + + Perform some action that affects the outside world (for instance, + writing to the filesystem). Such actions are special because they + are disabled by the `dry_run' flag. This method takes care of all + that bureaucracy for you; all you have to do is supply the function + to call and an argument tuple for it (to embody the “external + action” being performed), and an optional message to print. + + -- Function: distutils.util.strtobool (val) + + Convert a string representation of truth to true (1) or false (0). + + True values are ‘y’, ‘yes’, ‘t’, ‘true’, ‘on’ and ‘1’; false values + are ‘n’, ‘no’, ‘f’, ‘false’, ‘off’ and ‘0’. Raises *note + ValueError: 1fb. if `val' is anything else. + + -- Function: distutils.util.byte_compile (py_files[, optimize=0, + force=0, prefix=None, base_dir=None, verbose=1, dry_run=0, + direct=None]) + + Byte-compile a collection of Python source files to ‘.pyc’ files in + a ‘__pycache__’ subdirectory (see PEP 3147(1) and PEP 488(2)). + `py_files' is a list of files to compile; any files that don’t end + in ‘.py’ are silently skipped. `optimize' must be one of the + following: + + * ‘0’ - don’t optimize + + * ‘1’ - normal optimization (like ‘python -O’) + + * ‘2’ - extra optimization (like ‘python -OO’) + + If `force' is true, all files are recompiled regardless of + timestamps. + + The source filename encoded in each *note bytecode: 614. file + defaults to the filenames listed in `py_files'; you can modify + these with `prefix' and `basedir'. `prefix' is a string that will + be stripped off of each source filename, and `base_dir' is a + directory name that will be prepended (after `prefix' is stripped). + You can supply either or both (or neither) of `prefix' and + `base_dir', as you wish. + + If `dry_run' is true, doesn’t actually do anything that would + affect the filesystem. + + Byte-compilation is either done directly in this interpreter + process with the standard *note py_compile: d7. module, or + indirectly by writing a temporary script and executing it. + Normally, you should let *note byte_compile(): 4165. figure out to + use direct compilation or not (see the source for details). The + `direct' flag is used by the script generated in indirect mode; + unless you know what you’re doing, leave it set to ‘None’. + + Changed in version 3.2.3: Create ‘.pyc’ files with an *note import + magic tag: 3824. in their name, in a ‘__pycache__’ subdirectory + instead of files without tag in the current directory. + + Changed in version 3.5: Create ‘.pyc’ files according to PEP + 488(3). + + -- Function: distutils.util.rfc822_escape (header) + + Return a version of `header' escaped for inclusion in an RFC 822(4) + header, by ensuring there are 8 spaces space after each newline. + Note that it does no other modification of the string. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-3147 + + (2) https://www.python.org/dev/peps/pep-0488 + + (3) https://www.python.org/dev/peps/pep-0488 + + (4) https://tools.ietf.org/html/rfc822.html + + +File: python.info, Node: distutils dist — The Distribution class, Next: distutils extension — The Extension class, Prev: distutils util — Miscellaneous other utility functions, Up: API Reference + +17.9.12 ‘distutils.dist’ — The Distribution class +------------------------------------------------- + +This module provides the *note Distribution: 4118. class, which +represents the module distribution being built/installed/distributed. + + +File: python.info, Node: distutils extension — The Extension class, Next: distutils debug — Distutils debug mode, Prev: distutils dist — The Distribution class, Up: API Reference + +17.9.13 ‘distutils.extension’ — The Extension class +--------------------------------------------------- + +This module provides the ‘Extension’ class, used to describe C/C++ +extension modules in setup scripts. + + +File: python.info, Node: distutils debug — Distutils debug mode, Next: distutils errors — Distutils exceptions, Prev: distutils extension — The Extension class, Up: API Reference + +17.9.14 ‘distutils.debug’ — Distutils debug mode +------------------------------------------------ + +This module provides the DEBUG flag. + + +File: python.info, Node: distutils errors — Distutils exceptions, Next: distutils fancy_getopt — Wrapper around the standard getopt module, Prev: distutils debug — Distutils debug mode, Up: API Reference + +17.9.15 ‘distutils.errors’ — Distutils exceptions +------------------------------------------------- + +Provides exceptions used by the Distutils modules. Note that Distutils +modules may raise standard exceptions; in particular, SystemExit is +usually raised for errors that are obviously the end-user’s fault (eg. +bad command-line arguments). + +This module is safe to use in ‘from ... import *’ mode; it only exports +symbols whose names start with ‘Distutils’ and end with ‘Error’. + + +File: python.info, Node: distutils fancy_getopt — Wrapper around the standard getopt module, Next: distutils filelist — The FileList class, Prev: distutils errors — Distutils exceptions, Up: API Reference + +17.9.16 ‘distutils.fancy_getopt’ — Wrapper around the standard getopt module +---------------------------------------------------------------------------- + +This module provides a wrapper around the standard *note getopt: 87. +module that provides the following additional features: + + * short and long options are tied together + + * options have help strings, so *note fancy_getopt(): 416c. could + potentially create a complete usage summary + + * options set attributes of a passed-in object + + * boolean options can have “negative aliases” — eg. if ‘--quiet’ is + the “negative alias” of ‘--verbose’, then ‘--quiet’ on the command + line sets `verbose' to false. + + -- Function: distutils.fancy_getopt.fancy_getopt (options, + negative_opt, object, args) + + Wrapper function. `options' is a list of ‘(long_option, + short_option, help_string)’ 3-tuples as described in the + constructor for *note FancyGetopt: 416d. `negative_opt' should be + a dictionary mapping option names to option names, both the key and + value should be in the `options' list. `object' is an object which + will be used to store values (see the *note getopt(): 87. method of + the *note FancyGetopt: 416d. class). `args' is the argument list. + Will use ‘sys.argv[1:]’ if you pass ‘None’ as `args'. + + -- Function: distutils.fancy_getopt.wrap_text (text, width) + + Wraps `text' to less than `width' wide. + + -- Class: distutils.fancy_getopt.FancyGetopt ([option_table=None]) + + The option_table is a list of 3-tuples: ‘(long_option, + short_option, help_string)’ + + If an option takes an argument, its `long_option' should have ‘'='’ + appended; `short_option' should just be a single character, no + ‘':'’ in any case. `short_option' should be ‘None’ if a + `long_option' doesn’t have a corresponding `short_option'. All + option tuples must have long options. + +The *note FancyGetopt: 416d. class provides the following methods: + + -- Method: FancyGetopt.getopt ([args=None, object=None]) + + Parse command-line options in args. Store as attributes on + `object'. + + If `args' is ‘None’ or not supplied, uses ‘sys.argv[1:]’. If + `object' is ‘None’ or not supplied, creates a new ‘OptionDummy’ + instance, stores option values there, and returns a tuple ‘(args, + object)’. If `object' is supplied, it is modified in place and + *note getopt(): 87. just returns `args'; in both cases, the + returned `args' is a modified copy of the passed-in `args' list, + which is left untouched. + + -- Method: FancyGetopt.get_option_order () + + Returns the list of ‘(option, value)’ tuples processed by the + previous run of *note getopt(): 87. Raises *note RuntimeError: 2ba. + if *note getopt(): 87. hasn’t been called yet. + + -- Method: FancyGetopt.generate_help ([header=None]) + + Generate help text (a list of strings, one per suggested line of + output) from the option table for this *note FancyGetopt: 416d. + object. + + If supplied, prints the supplied `header' at the top of the help. + + +File: python.info, Node: distutils filelist — The FileList class, Next: distutils log — Simple PEP 282-style logging, Prev: distutils fancy_getopt — Wrapper around the standard getopt module, Up: API Reference + +17.9.17 ‘distutils.filelist’ — The FileList class +------------------------------------------------- + +This module provides the ‘FileList’ class, used for poking about the +filesystem and building lists of files. + + +File: python.info, Node: distutils log — Simple PEP 282-style logging, Next: distutils spawn — Spawn a sub-process, Prev: distutils filelist — The FileList class, Up: API Reference + +17.9.18 ‘distutils.log’ — Simple `PEP 282'-style logging +-------------------------------------------------------- + + +File: python.info, Node: distutils spawn — Spawn a sub-process, Next: distutils sysconfig — System configuration information, Prev: distutils log — Simple PEP 282-style logging, Up: API Reference + +17.9.19 ‘distutils.spawn’ — Spawn a sub-process +----------------------------------------------- + +This module provides the ‘spawn()’ function, a front-end to various +platform-specific functions for launching another program in a +sub-process. Also provides ‘find_executable()’ to search the path for a +given executable name. + + +File: python.info, Node: distutils sysconfig — System configuration information, Next: distutils text_file — The TextFile class, Prev: distutils spawn — Spawn a sub-process, Up: API Reference + +17.9.20 ‘distutils.sysconfig’ — System configuration information +---------------------------------------------------------------- + +The *note distutils.sysconfig: 62. module provides access to Python’s +low-level configuration information. The specific configuration +variables available depend heavily on the platform and configuration. +The specific variables depend on the build process for the specific +version of Python being run; the variables are those found in the +‘Makefile’ and configuration header that are installed with Python on +Unix systems. The configuration header is called ‘pyconfig.h’ for +Python versions starting with 2.2, and ‘config.h’ for earlier versions +of Python. + +Some additional functions are provided which perform some useful +manipulations for other parts of the *note distutils: 39. package. + + -- Data: distutils.sysconfig.PREFIX + + The result of ‘os.path.normpath(sys.prefix)’. + + -- Data: distutils.sysconfig.EXEC_PREFIX + + The result of ‘os.path.normpath(sys.exec_prefix)’. + + -- Function: distutils.sysconfig.get_config_var (name) + + Return the value of a single variable. This is equivalent to + ‘get_config_vars().get(name)’. + + -- Function: distutils.sysconfig.get_config_vars (...) + + Return a set of variable definitions. If there are no arguments, + this returns a dictionary mapping names of configuration variables + to values. If arguments are provided, they should be strings, and + the return value will be a sequence giving the associated values. + If a given name does not have a corresponding value, ‘None’ will be + included for that variable. + + -- Function: distutils.sysconfig.get_config_h_filename () + + Return the full path name of the configuration header. For Unix, + this will be the header generated by the ‘configure’ script; for + other platforms the header will have been supplied directly by the + Python source distribution. The file is a platform-specific text + file. + + -- Function: distutils.sysconfig.get_makefile_filename () + + Return the full path name of the ‘Makefile’ used to build Python. + For Unix, this will be a file generated by the ‘configure’ script; + the meaning for other platforms will vary. The file is a + platform-specific text file, if it exists. This function is only + useful on POSIX platforms. + + -- Function: distutils.sysconfig.get_python_inc ([plat_specific[, + prefix]]) + + Return the directory for either the general or platform-dependent C + include files. If `plat_specific' is true, the platform-dependent + include directory is returned; if false or omitted, the + platform-independent directory is returned. If `prefix' is given, + it is used as either the prefix instead of *note PREFIX: 4176, or + as the exec-prefix instead of *note EXEC_PREFIX: 4177. if + `plat_specific' is true. + + -- Function: distutils.sysconfig.get_python_lib ([plat_specific[, + standard_lib[, prefix]]]) + + Return the directory for either the general or platform-dependent + library installation. If `plat_specific' is true, the + platform-dependent include directory is returned; if false or + omitted, the platform-independent directory is returned. If + `prefix' is given, it is used as either the prefix instead of *note + PREFIX: 4176, or as the exec-prefix instead of *note EXEC_PREFIX: + 4177. if `plat_specific' is true. If `standard_lib' is true, the + directory for the standard library is returned rather than the + directory for the installation of third-party extensions. + +The following function is only intended for use within the *note +distutils: 39. package. + + -- Function: distutils.sysconfig.customize_compiler (compiler) + + Do any platform-specific customization of a *note + distutils.ccompiler.CCompiler: 411c. instance. + + This function is only needed on Unix at this time, but should be + called consistently to support forward-compatibility. It inserts + the information that varies across Unix flavors and is stored in + Python’s ‘Makefile’. This information includes the selected + compiler, compiler and linker options, and the extension used by + the linker for shared objects. + +This function is even more special-purpose, and should only be used from +Python’s own build procedures. + + -- Function: distutils.sysconfig.set_python_build () + + Inform the *note distutils.sysconfig: 62. module that it is being + used as part of the build process for Python. This changes a lot + of relative locations for files, allowing them to be located in the + build area rather than in an installed Python. + + +File: python.info, Node: distutils text_file — The TextFile class, Next: distutils version — Version number classes, Prev: distutils sysconfig — System configuration information, Up: API Reference + +17.9.21 ‘distutils.text_file’ — The TextFile class +-------------------------------------------------- + +This module provides the *note TextFile: 4181. class, which gives an +interface to text files that (optionally) takes care of stripping +comments, ignoring blank lines, and joining lines with backslashes. + + -- Class: distutils.text_file.TextFile ([filename=None, file=None, + **options]) + + This class provides a file-like object that takes care of all the + things you commonly want to do when processing a text file that has + some line-by-line syntax: strip comments (as long as ‘#’ is your + comment character), skip blank lines, join adjacent lines by + escaping the newline (ie. backslash at end of line), strip leading + and/or trailing whitespace. All of these are optional and + independently controllable. + + The class provides a *note warn(): 4182. method so you can generate + warning messages that report physical line number, even if the + logical line in question spans multiple physical lines. Also + provides *note unreadline(): 4183. for implementing line-at-a-time + lookahead. + + *note TextFile: 4181. instances are create with either `filename', + `file', or both. *note RuntimeError: 2ba. is raised if both are + ‘None’. `filename' should be a string, and `file' a file object + (or something that provides *note readline(): de. and *note + close(): 4184. methods). It is recommended that you supply at + least `filename', so that *note TextFile: 4181. can include it in + warning messages. If `file' is not supplied, *note TextFile: 4181. + creates its own using the *note open(): 4f0. built-in function. + + The options are all boolean, and affect the values returned by + *note readline(): de. + + option name description default + + -------------------------------------------------------------------------- + + `strip_comments' strip from ‘'#'’ to end-of-line, true + as well as any whitespace leading + up to the ‘'#'’—unless it is + escaped by a backslash + + + `lstrip_ws' strip leading whitespace from each false + line before returning it + + + `rstrip_ws' strip trailing whitespace true + (including line terminator!) from + each line before returning it. + + + `skip_blanks' skip lines that are empty *after* true + stripping comments and whitespace. + (If both lstrip_ws and rstrip_ws + are false, then some lines may + consist of solely whitespace: + these will *not* be skipped, even + if `skip_blanks' is true.) + + + `join_lines' if a backslash is the last false + non-newline character on a line + after stripping comments and + whitespace, join the following + line to it to form one logical + line; if N consecutive lines end + with a backslash, then N+1 + physical lines will be joined to + form one logical line. + + + `collapse_join' strip leading whitespace from false + lines that are joined to their + predecessor; only matters if + ‘(join_lines and not lstrip_ws)’ + + + Note that since `rstrip_ws' can strip the trailing newline, the + semantics of *note readline(): de. must differ from those of the + built-in file object’s *note readline(): de. method! In + particular, *note readline(): de. returns ‘None’ for end-of-file: + an empty string might just be a blank line (or an all-whitespace + line), if `rstrip_ws' is true but `skip_blanks' is not. + + -- Method: open (filename) + + Open a new file `filename'. This overrides any `file' or + `filename' constructor arguments. + + -- Method: close () + + Close the current file and forget everything we know about it + (including the filename and the current line number). + + -- Method: warn (msg[, line=None]) + + Print (to stderr) a warning message tied to the current + logical line in the current file. If the current logical line + in the file spans multiple physical lines, the warning refers + to the whole range, such as ‘"lines 3-5"’. If `line' is + supplied, it overrides the current line number; it may be a + list or tuple to indicate a range of physical lines, or an + integer for a single physical line. + + -- Method: readline () + + Read and return a single logical line from the current file + (or from an internal buffer if lines have previously been + “unread” with *note unreadline(): 4183.). If the `join_lines' + option is true, this may involve reading multiple physical + lines concatenated into a single string. Updates the current + line number, so calling *note warn(): 4182. after *note + readline(): de. emits a warning about the physical line(s) + just read. Returns ‘None’ on end-of-file, since the empty + string can occur if `rstrip_ws' is true but `strip_blanks' is + not. + + -- Method: readlines () + + Read and return the list of all logical lines remaining in the + current file. This updates the current line number to the + last line of the file. + + -- Method: unreadline (line) + + Push `line' (a string) onto an internal buffer that will be + checked by future *note readline(): de. calls. Handy for + implementing a parser with line-at-a-time lookahead. Note + that lines that are “unread” with *note unreadline(): 4183. + are not subsequently re-cleansed (whitespace stripped, or + whatever) when read with *note readline(): de. If multiple + calls are made to *note unreadline(): 4183. before a call to + *note readline(): de, the lines will be returned most in most + recent first order. + + +File: python.info, Node: distutils version — Version number classes, Next: distutils cmd — Abstract base class for Distutils commands, Prev: distutils text_file — The TextFile class, Up: API Reference + +17.9.22 ‘distutils.version’ — Version number classes +---------------------------------------------------- + + +File: python.info, Node: distutils cmd — Abstract base class for Distutils commands, Next: Creating a new Distutils command, Prev: distutils version — Version number classes, Up: API Reference + +17.9.23 ‘distutils.cmd’ — Abstract base class for Distutils commands +-------------------------------------------------------------------- + +This module supplies the abstract base class *note Command: 4107. + + -- Class: distutils.cmd.Command (dist) + + Abstract base class for defining command classes, the “worker bees” + of the Distutils. A useful analogy for command classes is to think + of them as subroutines with local variables called `options'. The + options are declared in *note initialize_options(): 418a. and + defined (given their final values) in *note finalize_options(): + 418b, both of which must be defined by every command class. The + distinction between the two is necessary because option values + might come from the outside world (command line, config file, …), + and any options dependent on other options must be computed after + these outside influences have been processed — hence *note + finalize_options(): 418b. The body of the subroutine, where it + does all its work based on the values of its options, is the *note + run(): 418c. method, which must also be implemented by every + command class. + + The class constructor takes a single argument `dist', a *note + Distribution: 4118. instance. + + +File: python.info, Node: Creating a new Distutils command, Next: distutils command — Individual Distutils commands, Prev: distutils cmd — Abstract base class for Distutils commands, Up: API Reference + +17.9.24 Creating a new Distutils command +---------------------------------------- + +This section outlines the steps to create a new Distutils command. + +A new command lives in a module in the *note distutils.command: 3e. +package. There is a sample template in that directory called +‘command_template’. Copy this file to a new module with the same name +as the new command you’re implementing. This module should implement a +class with the same name as the module (and the command). So, for +instance, to create the command ‘peel_banana’ (so that users can run +‘setup.py peel_banana’), you’d copy ‘command_template’ to +‘distutils/command/peel_banana.py’, then edit it so that it’s +implementing the class ‘peel_banana’, a subclass of *note +distutils.cmd.Command: 4107. + +Subclasses of *note Command: 4107. must define the following methods. + + -- Method: Command.initialize_options () + + Set default values for all the options that this command supports. + Note that these defaults may be overridden by other commands, by + the setup script, by config files, or by the command-line. Thus, + this is not the place to code dependencies between options; + generally, *note initialize_options(): 418a. implementations are + just a bunch of ‘self.foo = None’ assignments. + + -- Method: Command.finalize_options () + + Set final values for all the options that this command supports. + This is always called as late as possible, ie. after any option + assignments from the command-line or from other commands have been + done. Thus, this is the place to code option dependencies: if + `foo' depends on `bar', then it is safe to set `foo' from `bar' as + long as `foo' still has the same value it was assigned in *note + initialize_options(): 418a. + + -- Method: Command.run () + + A command’s raison d’etre: carry out the action it exists to + perform, controlled by the options initialized in *note + initialize_options(): 418a, customized by other commands, the setup + script, the command-line, and config files, and finalized in *note + finalize_options(): 418b. All terminal output and filesystem + interaction should be done by *note run(): 418c. + + -- Attribute: Command.sub_commands + + `sub_commands' formalizes the notion of a “family” of commands, + e.g. ‘install’ as the parent with sub-commands ‘install_lib’, + ‘install_headers’, etc. The parent of a family of commands defines + `sub_commands' as a class attribute; it’s a list of 2-tuples + ‘(command_name, predicate)’, with `command_name' a string and + `predicate' a function, a string or ‘None’. `predicate' is a + method of the parent command that determines whether the + corresponding command is applicable in the current situation. + (E.g. ‘install_headers’ is only applicable if we have any C header + files to install.) If `predicate' is ‘None’, that command is + always applicable. + + `sub_commands' is usually defined at the `end' of a class, because + predicates can be methods of the class, so they must already have + been defined. The canonical example is the ‘install’ command. + + +File: python.info, Node: distutils command — Individual Distutils commands, Next: distutils command bdist — Build a binary installer, Prev: Creating a new Distutils command, Up: API Reference + +17.9.25 ‘distutils.command’ — Individual Distutils commands +----------------------------------------------------------- + + +File: python.info, Node: distutils command bdist — Build a binary installer, Next: distutils command bdist_packager — Abstract base class for packagers, Prev: distutils command — Individual Distutils commands, Up: API Reference + +17.9.26 ‘distutils.command.bdist’ — Build a binary installer +------------------------------------------------------------ + + +File: python.info, Node: distutils command bdist_packager — Abstract base class for packagers, Next: distutils command bdist_dumb — Build a “dumb” installer, Prev: distutils command bdist — Build a binary installer, Up: API Reference + +17.9.27 ‘distutils.command.bdist_packager’ — Abstract base class for packagers +------------------------------------------------------------------------------ + + +File: python.info, Node: distutils command bdist_dumb — Build a “dumb” installer, Next: distutils command bdist_msi — Build a Microsoft Installer binary package, Prev: distutils command bdist_packager — Abstract base class for packagers, Up: API Reference + +17.9.28 ‘distutils.command.bdist_dumb’ — Build a “dumb” installer +----------------------------------------------------------------- + + +File: python.info, Node: distutils command bdist_msi — Build a Microsoft Installer binary package, Next: distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM, Prev: distutils command bdist_dumb — Build a “dumb” installer, Up: API Reference + +17.9.29 ‘distutils.command.bdist_msi’ — Build a Microsoft Installer binary package +---------------------------------------------------------------------------------- + + -- Class: distutils.command.bdist_msi.bdist_msi + + Builds a Windows Installer(1) (.msi) binary package. + + In most cases, the ‘bdist_msi’ installer is a better choice than + the ‘bdist_wininst’ installer, because it provides better support + for Win64 platforms, allows administrators to perform + non-interactive installations, and allows installation through + group policies. + + ---------- Footnotes ---------- + + (1) https://msdn.microsoft.com/en-us/library/cc185688(VS.85).aspx + + +File: python.info, Node: distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM, Next: distutils command bdist_wininst — Build a Windows installer, Prev: distutils command bdist_msi — Build a Microsoft Installer binary package, Up: API Reference + +17.9.30 ‘distutils.command.bdist_rpm’ — Build a binary distribution as a Redhat RPM and SRPM +-------------------------------------------------------------------------------------------- + + +File: python.info, Node: distutils command bdist_wininst — Build a Windows installer, Next: distutils command sdist — Build a source distribution, Prev: distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM, Up: API Reference + +17.9.31 ‘distutils.command.bdist_wininst’ — Build a Windows installer +--------------------------------------------------------------------- + +Deprecated since version 3.8: Use bdist_wheel (wheel packages) instead. + + +File: python.info, Node: distutils command sdist — Build a source distribution, Next: distutils command build — Build all files of a package, Prev: distutils command bdist_wininst — Build a Windows installer, Up: API Reference + +17.9.32 ‘distutils.command.sdist’ — Build a source distribution +--------------------------------------------------------------- + + +File: python.info, Node: distutils command build — Build all files of a package, Next: distutils command build_clib — Build any C libraries in a package, Prev: distutils command sdist — Build a source distribution, Up: API Reference + +17.9.33 ‘distutils.command.build’ — Build all files of a package +---------------------------------------------------------------- + + +File: python.info, Node: distutils command build_clib — Build any C libraries in a package, Next: distutils command build_ext — Build any extensions in a package, Prev: distutils command build — Build all files of a package, Up: API Reference + +17.9.34 ‘distutils.command.build_clib’ — Build any C libraries in a package +--------------------------------------------------------------------------- + + +File: python.info, Node: distutils command build_ext — Build any extensions in a package, Next: distutils command build_py — Build the py/ pyc files of a package, Prev: distutils command build_clib — Build any C libraries in a package, Up: API Reference + +17.9.35 ‘distutils.command.build_ext’ — Build any extensions in a package +------------------------------------------------------------------------- + + +File: python.info, Node: distutils command build_py — Build the py/ pyc files of a package, Next: distutils command build_scripts — Build the scripts of a package, Prev: distutils command build_ext — Build any extensions in a package, Up: API Reference + +17.9.36 ‘distutils.command.build_py’ — Build the .py/.pyc files of a package +---------------------------------------------------------------------------- + + -- Class: distutils.command.build_py.build_py + + -- Class: distutils.command.build_py.build_py_2to3 + + Alternative implementation of build_py which also runs the 2to3 + conversion library on each .py file that is going to be installed. + To use this in a setup.py file for a distribution that is designed + to run with both Python 2.x and 3.x, add: + + try: + from distutils.command.build_py import build_py_2to3 as build_py + except ImportError: + from distutils.command.build_py import build_py + + to your setup.py, and later: + + cmdclass = {'build_py': build_py} + + to the invocation of setup(). + + +File: python.info, Node: distutils command build_scripts — Build the scripts of a package, Next: distutils command clean — Clean a package build area, Prev: distutils command build_py — Build the py/ pyc files of a package, Up: API Reference + +17.9.37 ‘distutils.command.build_scripts’ — Build the scripts of a package +-------------------------------------------------------------------------- + + +File: python.info, Node: distutils command clean — Clean a package build area, Next: distutils command config — Perform package configuration, Prev: distutils command build_scripts — Build the scripts of a package, Up: API Reference + +17.9.38 ‘distutils.command.clean’ — Clean a package build area +-------------------------------------------------------------- + +This command removes the temporary files created by ‘build’ and its +subcommands, like intermediary compiled object files. With the ‘--all’ +option, the complete build directory will be removed. + +Extension modules built *note in place: 40e0. will not be cleaned, as +they are not in the build directory. + + +File: python.info, Node: distutils command config — Perform package configuration, Next: distutils command install — Install a package, Prev: distutils command clean — Clean a package build area, Up: API Reference + +17.9.39 ‘distutils.command.config’ — Perform package configuration +------------------------------------------------------------------ + + +File: python.info, Node: distutils command install — Install a package, Next: distutils command install_data — Install data files from a package, Prev: distutils command config — Perform package configuration, Up: API Reference + +17.9.40 ‘distutils.command.install’ — Install a package +------------------------------------------------------- + + +File: python.info, Node: distutils command install_data — Install data files from a package, Next: distutils command install_headers — Install C/C++ header files from a package, Prev: distutils command install — Install a package, Up: API Reference + +17.9.41 ‘distutils.command.install_data’ — Install data files from a package +---------------------------------------------------------------------------- + + +File: python.info, Node: distutils command install_headers — Install C/C++ header files from a package, Next: distutils command install_lib — Install library files from a package, Prev: distutils command install_data — Install data files from a package, Up: API Reference + +17.9.42 ‘distutils.command.install_headers’ — Install C/C++ header files from a package +--------------------------------------------------------------------------------------- + + +File: python.info, Node: distutils command install_lib — Install library files from a package, Next: distutils command install_scripts — Install script files from a package, Prev: distutils command install_headers — Install C/C++ header files from a package, Up: API Reference + +17.9.43 ‘distutils.command.install_lib’ — Install library files from a package +------------------------------------------------------------------------------ + + +File: python.info, Node: distutils command install_scripts — Install script files from a package, Next: distutils command register — Register a module with the Python Package Index, Prev: distutils command install_lib — Install library files from a package, Up: API Reference + +17.9.44 ‘distutils.command.install_scripts’ — Install script files from a package +--------------------------------------------------------------------------------- + + +File: python.info, Node: distutils command register — Register a module with the Python Package Index, Next: distutils command check — Check the meta-data of a package, Prev: distutils command install_scripts — Install script files from a package, Up: API Reference + +17.9.45 ‘distutils.command.register’ — Register a module with the Python Package Index +-------------------------------------------------------------------------------------- + +The ‘register’ command registers the package with the Python Package +Index. This is described in more detail in PEP 301(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0301 + + +File: python.info, Node: distutils command check — Check the meta-data of a package, Prev: distutils command register — Register a module with the Python Package Index, Up: API Reference + +17.9.46 ‘distutils.command.check’ — Check the meta-data of a package +-------------------------------------------------------------------- + +The ‘check’ command performs some tests on the meta-data of a package. +For example, it verifies that all required meta-data are provided as the +arguments passed to the ‘setup()’ function. + + +File: python.info, Node: Installing Python Modules Legacy version, Next: Python Module Index, Prev: Distributing Python Modules Legacy version, Up: Top + +18 Installing Python Modules (Legacy version) +********************************************* + + +Author: Greg Ward + +See also +........ + +*note Installing Python Modules: 7f5. + + The up to date module installation documentation. For regular + Python usage, you almost certainly want that document rather than + this one. + + Note: This document is being retained solely until the ‘setuptools’ + documentation at + ‘https://setuptools.readthedocs.io/en/latest/setuptools.html’ + independently covers all of the relevant information currently + included here. + + Note: This guide only covers the basic tools for building and + distributing extensions that are provided as part of this version + of Python. Third party tools offer easier to use and more secure + alternatives. Refer to the quick recommendations section(1) in the + Python Packaging User Guide for more information. + +* Menu: + +* Introduction: Introduction<17>. +* Standard Build and Install:: +* Alternate Installation:: +* Custom Installation:: +* Distutils Configuration Files:: +* Building Extensions; Tips and Tricks: Building Extensions Tips and Tricks. + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/guides/tool-recommendations/ + + +File: python.info, Node: Introduction<17>, Next: Standard Build and Install, Up: Installing Python Modules Legacy version + +18.1 Introduction +================= + +In Python 2.0, the ‘distutils’ API was first added to the standard +library. This provided Linux distro maintainers with a standard way of +converting Python projects into Linux distro packages, and system +administrators with a standard way of installing them directly onto +target systems. + +In the many years since Python 2.0 was released, tightly coupling the +build system and package installer to the language runtime release cycle +has turned out to be problematic, and it is now recommended that +projects use the ‘pip’ package installer and the ‘setuptools’ build +system, rather than using ‘distutils’ directly. + +See *note Installing Python Modules: 7f5. and *note Distributing Python +Modules: 7f6. for more details. + +This legacy documentation is being retained only until we’re confident +that the ‘setuptools’ documentation covers everything needed. + +* Menu: + +* Distutils based source distributions:: + + +File: python.info, Node: Distutils based source distributions, Up: Introduction<17> + +18.1.1 Distutils based source distributions +------------------------------------------- + +If you download a module source distribution, you can tell pretty +quickly if it was packaged and distributed in the standard way, i.e. +using the Distutils. First, the distribution’s name and version number +will be featured prominently in the name of the downloaded archive, e.g. +‘foo-1.0.tar.gz’ or ‘widget-0.9.7.zip’. Next, the archive will unpack +into a similarly-named directory: ‘foo-1.0’ or ‘widget-0.9.7’. +Additionally, the distribution will contain a setup script ‘setup.py’, +and a file named ‘README.txt’ or possibly just ‘README’, which should +explain that building and installing the module distribution is a simple +matter of running one command from a terminal: + + python setup.py install + +For Windows, this command should be run from a command prompt window +(Start ‣ Accessories): + + setup.py install + +If all these things are true, then you already know how to build and +install the modules you’ve just downloaded: Run the command above. +Unless you need to install things in a non-standard way or customize the +build process, you don’t really need this manual. Or rather, the above +command is everything you need to get out of this manual. + + +File: python.info, Node: Standard Build and Install, Next: Alternate Installation, Prev: Introduction<17>, Up: Installing Python Modules Legacy version + +18.2 Standard Build and Install +=============================== + +As described in section *note Distutils based source distributions: +41ad, building and installing a module distribution using the Distutils +is usually one simple command to run from a terminal: + + python setup.py install + +* Menu: + +* Platform variations:: +* Splitting the job up:: +* How building works:: +* How installation works:: + + +File: python.info, Node: Platform variations, Next: Splitting the job up, Up: Standard Build and Install + +18.2.1 Platform variations +-------------------------- + +You should always run the setup command from the distribution root +directory, i.e. the top-level subdirectory that the module source +distribution unpacks into. For example, if you’ve just downloaded a +module source distribution ‘foo-1.0.tar.gz’ onto a Unix system, the +normal thing to do is: + + gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0 + cd foo-1.0 + python setup.py install + +On Windows, you’d probably download ‘foo-1.0.zip’. If you downloaded +the archive file to ‘C:\Temp’, then it would unpack into +‘C:\Temp\foo-1.0’; you can use either an archive manipulator with a +graphical user interface (such as WinZip) or a command-line tool (such +as ‘unzip’ or ‘pkunzip’) to unpack the archive. Then, open a command +prompt window and run: + + cd c:\Temp\foo-1.0 + python setup.py install + + +File: python.info, Node: Splitting the job up, Next: How building works, Prev: Platform variations, Up: Standard Build and Install + +18.2.2 Splitting the job up +--------------------------- + +Running ‘setup.py install’ builds and installs all modules in one run. +If you prefer to work incrementally—especially useful if you want to +customize the build process, or if things are going wrong—you can use +the setup script to do one thing at a time. This is particularly +helpful when the build and install will be done by different users—for +example, you might want to build a module distribution and hand it off +to a system administrator for installation (or do it yourself, with +super-user privileges). + +For example, you can build everything in one step, and then install +everything in a second step, by invoking the setup script twice: + + python setup.py build + python setup.py install + +If you do this, you will notice that running the ‘install’ command first +runs the ‘build’ command, which—in this case—quickly notices that it has +nothing to do, since everything in the ‘build’ directory is up-to-date. + +You may not need this ability to break things down often if all you do +is install modules downloaded off the ‘net, but it’s very handy for more +advanced tasks. If you get into distributing your own Python modules +and extensions, you’ll run lots of individual Distutils commands on +their own. + + +File: python.info, Node: How building works, Next: How installation works, Prev: Splitting the job up, Up: Standard Build and Install + +18.2.3 How building works +------------------------- + +As implied above, the ‘build’ command is responsible for putting the +files to install into a `build directory'. By default, this is ‘build’ +under the distribution root; if you’re excessively concerned with speed, +or want to keep the source tree pristine, you can change the build +directory with the ‘--build-base’ option. For example: + + python setup.py build --build-base=/path/to/pybuild/foo-1.0 + +(Or you could do this permanently with a directive in your system or +personal Distutils configuration file; see section *note Distutils +Configuration Files: 41b6.) Normally, this isn’t necessary. + +The default layout for the build tree is as follows: + + --- build/ --- lib/ + or + --- build/ --- lib./ + temp./ + +where ‘’ expands to a brief description of the current OS/hardware +platform and Python version. The first form, with just a ‘lib’ +directory, is used for “pure module distributions”—that is, module +distributions that include only pure Python modules. If a module +distribution contains any extensions (modules written in C/C++), then +the second form, with two ‘’ directories, is used. In that case, +the ‘temp.`plat'’ directory holds temporary files generated by the +compile/link process that don’t actually get installed. In either case, +the ‘lib’ (or ‘lib.`plat'’) directory contains all Python modules (pure +Python and extensions) that will be installed. + +In the future, more directories will be added to handle Python scripts, +documentation, binary executables, and whatever else is needed to handle +the job of installing Python modules and applications. + + +File: python.info, Node: How installation works, Prev: How building works, Up: Standard Build and Install + +18.2.4 How installation works +----------------------------- + +After the ‘build’ command runs (whether you run it explicitly, or the +‘install’ command does it for you), the work of the ‘install’ command is +relatively simple: all it has to do is copy everything under ‘build/lib’ +(or ‘build/lib.`plat'’) to your chosen installation directory. + +If you don’t choose an installation directory—i.e., if you just run +‘setup.py install’—then the ‘install’ command installs to the standard +location for third-party Python modules. This location varies by +platform and by how you built/installed Python itself. On Unix (and Mac +OS X, which is also Unix-based), it also depends on whether the module +distribution being installed is pure Python or contains extensions +(“non-pure”): + +Platform Standard installation location Default value Notes + +--------------------------------------------------------------------------------------------------------------------------------------------------- + +Unix (pure) ‘`prefix'/lib/python`X.Y'/site-packages’ ‘/usr/local/lib/python`X.Y'/site-packages’ (1) + + +Unix (non-pure) ‘`exec-prefix'/lib/python`X.Y'/site-packages’ ‘/usr/local/lib/python`X.Y'/site-packages’ (1) + + +Windows ‘`prefix'\Lib\site-packages’ ‘C:\Python`XY'\Lib\site-packages’ (2) + + +Notes: + + 1. Most Linux distributions include Python as a standard part of the + system, so ‘`prefix'’ and ‘`exec-prefix'’ are usually both ‘/usr’ + on Linux. If you build Python yourself on Linux (or any Unix-like + system), the default ‘`prefix'’ and ‘`exec-prefix'’ are + ‘/usr/local’. + + 2. The default installation directory on Windows was ‘C:\Program + Files\Python’ under Python 1.6a1, 1.5.2, and earlier. + +‘`prefix'’ and ‘`exec-prefix'’ stand for the directories that Python is +installed to, and where it finds its libraries at run-time. They are +always the same under Windows, and very often the same under Unix and +Mac OS X. You can find out what your Python installation uses for +‘`prefix'’ and ‘`exec-prefix'’ by running Python in interactive mode and +typing a few simple commands. Under Unix, just type ‘python’ at the +shell prompt. Under Windows, choose Start ‣ Programs ‣ Python X.Y ‣ +Python (command line). Once the interpreter is started, you type Python +code at the prompt. For example, on my Linux system, I type the three +Python statements shown below, and get the output as shown, to find out +my ‘`prefix'’ and ‘`exec-prefix'’: + + Python 2.4 (#26, Aug 7 2004, 17:19:02) + Type "help", "copyright", "credits" or "license" for more information. + >>> import sys + >>> sys.prefix + '/usr' + >>> sys.exec_prefix + '/usr' + +A few other placeholders are used in this document: ‘`X.Y'’ stands for +the version of Python, for example ‘3.2’; ‘`abiflags'’ will be replaced +by the value of *note sys.abiflags: 264. or the empty string for +platforms which don’t define ABI flags; ‘`distname'’ will be replaced by +the name of the module distribution being installed. Dots and +capitalization are important in the paths; for example, a value that +uses ‘python3.2’ on UNIX will typically use ‘Python32’ on Windows. + +If you don’t want to install modules to the standard location, or if you +don’t have permission to write there, then you need to read about +alternate installations in section *note Alternate Installation: 41b9. +If you want to customize your installation directories more heavily, see +section *note Custom Installation: 41ba. on custom installations. + + +File: python.info, Node: Alternate Installation, Next: Custom Installation, Prev: Standard Build and Install, Up: Installing Python Modules Legacy version + +18.3 Alternate Installation +=========================== + +Often, it is necessary or desirable to install modules to a location +other than the standard location for third-party Python modules. For +example, on a Unix system you might not have permission to write to the +standard third-party module directory. Or you might wish to try out a +module before making it a standard part of your local Python +installation. This is especially true when upgrading a distribution +already present: you want to make sure your existing base of scripts +still works with the new version before actually upgrading. + +The Distutils ‘install’ command is designed to make installing module +distributions to an alternate location simple and painless. The basic +idea is that you supply a base directory for the installation, and the +‘install’ command picks a set of directories (called an `installation +scheme') under this base directory in which to install files. The +details differ across platforms, so read whichever of the following +sections applies to you. + +Note that the various alternate installation schemes are mutually +exclusive: you can pass ‘--user’, or ‘--home’, or ‘--prefix’ and +‘--exec-prefix’, or ‘--install-base’ and ‘--install-platbase’, but you +can’t mix from these groups. + +* Menu: + +* Alternate installation; the user scheme: Alternate installation the user scheme. +* Alternate installation; the home scheme: Alternate installation the home scheme. +* Alternate installation; Unix (the prefix scheme): Alternate installation Unix the prefix scheme. +* Alternate installation; Windows (the prefix scheme): Alternate installation Windows the prefix scheme. + + +File: python.info, Node: Alternate installation the user scheme, Next: Alternate installation the home scheme, Up: Alternate Installation + +18.3.1 Alternate installation: the user scheme +---------------------------------------------- + +This scheme is designed to be the most convenient solution for users +that don’t have write permission to the global site-packages directory +or don’t want to install into it. It is enabled with a simple option: + + python setup.py install --user + +Files will be installed into subdirectories of *note site.USER_BASE: +ff3. (written as ‘`userbase'’ hereafter). This scheme installs pure +Python modules and extension modules in the same location (also known as +*note site.USER_SITE: fe1.). Here are the values for UNIX, including +Mac OS X: + +Type of file Installation directory + +------------------------------------------------------------------------------------ + +modules ‘`userbase'/lib/python`X.Y'/site-packages’ + + +scripts ‘`userbase'/bin’ + + +data ‘`userbase'’ + + +C headers ‘`userbase'/include/python`X.Y'`abiflags'/`distname'’ + + +And here are the values used on Windows: + +Type of file Installation directory + +------------------------------------------------------------------------------------ + +modules ‘`userbase'\Python`XY'\site-packages’ + + +scripts ‘`userbase'\Python`XY'\Scripts’ + + +data ‘`userbase'’ + + +C headers ‘`userbase'\Python`XY'\Include{distname}’ + + +The advantage of using this scheme compared to the other ones described +below is that the user site-packages directory is under normal +conditions always included in *note sys.path: 488. (see *note site: eb. +for more information), which means that there is no additional step to +perform after running the ‘setup.py’ script to finalize the +installation. + +The ‘build_ext’ command also has a ‘--user’ option to add +‘`userbase'/include’ to the compiler search path for header files and +‘`userbase'/lib’ to the compiler search path for libraries as well as to +the runtime search path for shared C libraries (rpath). + + +File: python.info, Node: Alternate installation the home scheme, Next: Alternate installation Unix the prefix scheme, Prev: Alternate installation the user scheme, Up: Alternate Installation + +18.3.2 Alternate installation: the home scheme +---------------------------------------------- + +The idea behind the “home scheme” is that you build and maintain a +personal stash of Python modules. This scheme’s name is derived from +the idea of a “home” directory on Unix, since it’s not unusual for a +Unix user to make their home directory have a layout similar to ‘/usr/’ +or ‘/usr/local/’. This scheme can be used by anyone, regardless of the +operating system they are installing for. + +Installing a new module distribution is as simple as + + python setup.py install --home= + +where you can supply any directory you like for the ‘--home’ option. On +Unix, lazy typists can just type a tilde (‘~’); the ‘install’ command +will expand this to your home directory: + + python setup.py install --home=~ + +To make Python find the distributions installed with this scheme, you +may have to *note modify Python’s search path: 41bf. or edit +‘sitecustomize’ (see *note site: eb.) to call *note site.addsitedir(): +343f. or edit *note sys.path: 488. + +The ‘--home’ option defines the installation base directory. Files are +installed to the following directories under the installation base as +follows: + +Type of file Installation directory + +------------------------------------------------------------------------------------ + +modules ‘`home'/lib/python’ + + +scripts ‘`home'/bin’ + + +data ‘`home'’ + + +C headers ‘`home'/include/python/`distname'’ + + +(Mentally replace slashes with backslashes if you’re on Windows.) + + +File: python.info, Node: Alternate installation Unix the prefix scheme, Next: Alternate installation Windows the prefix scheme, Prev: Alternate installation the home scheme, Up: Alternate Installation + +18.3.3 Alternate installation: Unix (the prefix scheme) +------------------------------------------------------- + +The “prefix scheme” is useful when you wish to use one Python +installation to perform the build/install (i.e., to run the setup +script), but install modules into the third-party module directory of a +different Python installation (or something that looks like a different +Python installation). If this sounds a trifle unusual, it is—that’s why +the user and home schemes come before. However, there are at least two +known cases where the prefix scheme will be useful. + +First, consider that many Linux distributions put Python in ‘/usr’, +rather than the more traditional ‘/usr/local’. This is entirely +appropriate, since in those cases Python is part of “the system” rather +than a local add-on. However, if you are installing Python modules from +source, you probably want them to go in ‘/usr/local/lib/python2.`X'’ +rather than ‘/usr/lib/python2.`X'’. This can be done with + + /usr/bin/python setup.py install --prefix=/usr/local + +Another possibility is a network filesystem where the name used to write +to a remote directory is different from the name used to read it: for +example, the Python interpreter accessed as ‘/usr/local/bin/python’ +might search for modules in ‘/usr/local/lib/python2.`X'’, but those +modules would have to be installed to, say, +‘/mnt/`@server'/export/lib/python2.`X'’. This could be done with + + /usr/local/bin/python setup.py install --prefix=/mnt/@server/export + +In either case, the ‘--prefix’ option defines the installation base, and +the ‘--exec-prefix’ option defines the platform-specific installation +base, which is used for platform-specific files. (Currently, this just +means non-pure module distributions, but could be expanded to C +libraries, binary executables, etc.) If ‘--exec-prefix’ is not +supplied, it defaults to ‘--prefix’. Files are installed as follows: + +Type of file Installation directory + +------------------------------------------------------------------------------------- + +Python modules ‘`prefix'/lib/python`X.Y'/site-packages’ + + +extension modules ‘`exec-prefix'/lib/python`X.Y'/site-packages’ + + +scripts ‘`prefix'/bin’ + + +data ‘`prefix'’ + + +C headers ‘`prefix'/include/python`X.Y'`abiflags'/`distname'’ + + +There is no requirement that ‘--prefix’ or ‘--exec-prefix’ actually +point to an alternate Python installation; if the directories listed +above do not already exist, they are created at installation time. + +Incidentally, the real reason the prefix scheme is important is simply +that a standard Unix installation uses the prefix scheme, but with +‘--prefix’ and ‘--exec-prefix’ supplied by Python itself as ‘sys.prefix’ +and ‘sys.exec_prefix’. Thus, you might think you’ll never use the +prefix scheme, but every time you run ‘python setup.py install’ without +any other options, you’re using it. + +Note that installing extensions to an alternate Python installation has +no effect on how those extensions are built: in particular, the Python +header files (‘Python.h’ and friends) installed with the Python +interpreter used to run the setup script will be used in compiling +extensions. It is your responsibility to ensure that the interpreter +used to run extensions installed in this way is compatible with the +interpreter used to build them. The best way to do this is to ensure +that the two interpreters are the same version of Python (possibly +different builds, or possibly copies of the same build). (Of course, if +your ‘--prefix’ and ‘--exec-prefix’ don’t even point to an alternate +Python installation, this is immaterial.) + + +File: python.info, Node: Alternate installation Windows the prefix scheme, Prev: Alternate installation Unix the prefix scheme, Up: Alternate Installation + +18.3.4 Alternate installation: Windows (the prefix scheme) +---------------------------------------------------------- + +Windows has no concept of a user’s home directory, and since the +standard Python installation under Windows is simpler than under Unix, +the ‘--prefix’ option has traditionally been used to install additional +packages in separate locations on Windows. + + python setup.py install --prefix="\Temp\Python" + +to install modules to the ‘\Temp\Python’ directory on the current drive. + +The installation base is defined by the ‘--prefix’ option; the +‘--exec-prefix’ option is not supported under Windows, which means that +pure Python modules and extension modules are installed into the same +location. Files are installed as follows: + +Type of file Installation directory + +----------------------------------------------------------------------------------- + +modules ‘`prefix'\Lib\site-packages’ + + +scripts ‘`prefix'\Scripts’ + + +data ‘`prefix'’ + + +C headers ‘`prefix'\Include{distname}’ + + + +File: python.info, Node: Custom Installation, Next: Distutils Configuration Files, Prev: Alternate Installation, Up: Installing Python Modules Legacy version + +18.4 Custom Installation +======================== + +Sometimes, the alternate installation schemes described in section *note +Alternate Installation: 41b9. just don’t do what you want. You might +want to tweak just one or two directories while keeping everything under +the same base directory, or you might want to completely redefine the +installation scheme. In either case, you’re creating a `custom +installation scheme'. + +To create a custom installation scheme, you start with one of the +alternate schemes and override some of the installation directories used +for the various types of files, using these options: + +Type of file Override option + +------------------------------------------------------- + +Python modules ‘--install-purelib’ + + +extension modules ‘--install-platlib’ + + +all modules ‘--install-lib’ + + +scripts ‘--install-scripts’ + + +data ‘--install-data’ + + +C headers ‘--install-headers’ + + +These override options can be relative, absolute, or explicitly defined +in terms of one of the installation base directories. (There are two +installation base directories, and they are normally the same—they only +differ when you use the Unix “prefix scheme” and supply different +‘--prefix’ and ‘--exec-prefix’ options; using ‘--install-lib’ will +override values computed or given for ‘--install-purelib’ and +‘--install-platlib’, and is recommended for schemes that don’t make a +difference between Python and extension modules.) + +For example, say you’re installing a module distribution to your home +directory under Unix—but you want scripts to go in ‘~/scripts’ rather +than ‘~/bin’. As you might expect, you can override this directory with +the ‘--install-scripts’ option; in this case, it makes most sense to +supply a relative path, which will be interpreted relative to the +installation base directory (your home directory, in this case): + + python setup.py install --home=~ --install-scripts=scripts + +Another Unix example: suppose your Python installation was built and +installed with a prefix of ‘/usr/local/python’, so under a standard +installation scripts will wind up in ‘/usr/local/python/bin’. If you +want them in ‘/usr/local/bin’ instead, you would supply this absolute +directory for the ‘--install-scripts’ option: + + python setup.py install --install-scripts=/usr/local/bin + +(This performs an installation using the “prefix scheme”, where the +prefix is whatever your Python interpreter was installed with— +‘/usr/local/python’ in this case.) + +If you maintain Python on Windows, you might want third-party modules to +live in a subdirectory of ‘`prefix'’, rather than right in ‘`prefix'’ +itself. This is almost as easy as customizing the script installation +directory—you just have to remember that there are two types of modules +to worry about, Python and extension modules, which can conveniently be +both controlled by one option: + + python setup.py install --install-lib=Site + +The specified installation directory is relative to ‘`prefix'’. Of +course, you also have to ensure that this directory is in Python’s +module search path, such as by putting a ‘.pth’ file in a site directory +(see *note site: eb.). See section *note Modifying Python’s Search +Path: 41bf. to find out how to modify Python’s search path. + +If you want to define an entire installation scheme, you just have to +supply all of the installation directory options. The recommended way +to do this is to supply relative paths; for example, if you want to +maintain all Python module-related files under ‘python’ in your home +directory, and you want a separate directory for each platform that you +use your home directory from, you might define the following +installation scheme: + + python setup.py install --home=~ \ + --install-purelib=python/lib \ + --install-platlib=python/lib.$PLAT \ + --install-scripts=python/scripts + --install-data=python/data + +or, equivalently, + + python setup.py install --home=~/python \ + --install-purelib=lib \ + --install-platlib='lib.$PLAT' \ + --install-scripts=scripts + --install-data=data + +‘$PLAT’ is not (necessarily) an environment variable—it will be expanded +by the Distutils as it parses your command line options, just as it does +when parsing your configuration file(s). + +Obviously, specifying the entire installation scheme every time you +install a new module distribution would be very tedious. Thus, you can +put these options into your Distutils config file (see section *note +Distutils Configuration Files: 41b6.): + + [install] + install-base=$HOME + install-purelib=python/lib + install-platlib=python/lib.$PLAT + install-scripts=python/scripts + install-data=python/data + +or, equivalently, + + [install] + install-base=$HOME/python + install-purelib=lib + install-platlib=lib.$PLAT + install-scripts=scripts + install-data=data + +Note that these two are `not' equivalent if you supply a different +installation base directory when you run the setup script. For example, + + python setup.py install --install-base=/tmp + +would install pure modules to ‘/tmp/python/lib’ in the first case, and +to ‘/tmp/lib’ in the second case. (For the second case, you probably +want to supply an installation base of ‘/tmp/python’.) + +You probably noticed the use of ‘$HOME’ and ‘$PLAT’ in the sample +configuration file input. These are Distutils configuration variables, +which bear a strong resemblance to environment variables. In fact, you +can use environment variables in config files on platforms that have +such a notion but the Distutils additionally define a few extra +variables that may not be in your environment, such as ‘$PLAT’. (And of +course, on systems that don’t have environment variables, such as Mac OS +9, the configuration variables supplied by the Distutils are the only +ones you can use.) See section *note Distutils Configuration Files: +41b6. for details. + + Note: When a *note virtual environment: 3321. is activated, any + options that change the installation path will be ignored from all + distutils configuration files to prevent inadvertently installing + projects outside of the virtual environment. + +* Menu: + +* Modifying Python’s Search Path:: + + +File: python.info, Node: Modifying Python’s Search Path, Up: Custom Installation + +18.4.1 Modifying Python’s Search Path +------------------------------------- + +When the Python interpreter executes an *note import: c1d. statement, it +searches for both Python code and extension modules along a search path. +A default value for the path is configured into the Python binary when +the interpreter is built. You can determine the path by importing the +*note sys: fd. module and printing the value of ‘sys.path’. + + $ python + Python 2.2 (#11, Oct 3 2002, 13:31:27) + [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2 + Type "help", "copyright", "credits" or "license" for more information. + >>> import sys + >>> sys.path + ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', + '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', + '/usr/local/lib/python2.3/site-packages'] + >>> + +The null string in ‘sys.path’ represents the current working directory. + +The expected convention for locally installed packages is to put them in +the ‘`…'/site-packages/’ directory, but you may want to install Python +modules into some arbitrary directory. For example, your site may have +a convention of keeping all software related to the web server under +‘/www’. Add-on Python modules might then belong in ‘/www/python’, and +in order to import them, this directory must be added to ‘sys.path’. +There are several different ways to add the directory. + +The most convenient way is to add a path configuration file to a +directory that’s already on Python’s path, usually to the +‘.../site-packages/’ directory. Path configuration files have an +extension of ‘.pth’, and each line must contain a single path that will +be appended to ‘sys.path’. (Because the new paths are appended to +‘sys.path’, modules in the added directories will not override standard +modules. This means you can’t use this mechanism for installing fixed +versions of standard modules.) + +Paths can be absolute or relative, in which case they’re relative to the +directory containing the ‘.pth’ file. See the documentation of the +*note site: eb. module for more information. + +A slightly less convenient way is to edit the ‘site.py’ file in Python’s +standard library, and modify ‘sys.path’. ‘site.py’ is automatically +imported when the Python interpreter is executed, unless the *note -S: +b24. switch is supplied to suppress this behaviour. So you could simply +edit ‘site.py’ and add two lines to it: + + import sys + sys.path.append('/www/python/') + +However, if you reinstall the same major version of Python (perhaps when +upgrading from 2.2 to 2.2.2, for example) ‘site.py’ will be overwritten +by the stock version. You’d have to remember that it was modified and +save a copy before doing the installation. + +There are two environment variables that can modify ‘sys.path’. *note +PYTHONHOME: 4d6. sets an alternate value for the prefix of the Python +installation. For example, if *note PYTHONHOME: 4d6. is set to +‘/www/python’, the search path will be set to ‘['', +'/www/python/lib/pythonX.Y/', '/www/python/lib/pythonX.Y/plat-linux2', +...]’. + +The *note PYTHONPATH: 953. variable can be set to a list of paths that +will be added to the beginning of ‘sys.path’. For example, if *note +PYTHONPATH: 953. is set to ‘/www/python:/opt/py’, the search path will +begin with ‘['/www/python', '/opt/py']’. (Note that directories must +exist in order to be added to ‘sys.path’; the *note site: eb. module +removes paths that don’t exist.) + +Finally, ‘sys.path’ is just a regular Python list, so any Python +application can modify it by adding or removing entries. + + +File: python.info, Node: Distutils Configuration Files, Next: Building Extensions Tips and Tricks, Prev: Custom Installation, Up: Installing Python Modules Legacy version + +18.5 Distutils Configuration Files +================================== + +As mentioned above, you can use Distutils configuration files to record +personal or site preferences for any Distutils options. That is, any +option to any command can be stored in one of two or three (depending on +your platform) configuration files, which will be consulted before the +command-line is parsed. This means that configuration files will +override default values, and the command-line will in turn override +configuration files. Furthermore, if multiple configuration files +apply, values from “earlier” files are overridden by “later” files. + +* Menu: + +* Location and names of config files:: +* Syntax of config files:: + + +File: python.info, Node: Location and names of config files, Next: Syntax of config files, Up: Distutils Configuration Files + +18.5.1 Location and names of config files +----------------------------------------- + +The names and locations of the configuration files vary slightly across +platforms. On Unix and Mac OS X, the three configuration files (in the +order they are processed) are: + +Type of file Location and filename Notes + +---------------------------------------------------------------------------------------------- + +system ‘`prefix'/lib/python`ver'/distutils/distutils.cfg’ (1) + + +personal ‘$HOME/.pydistutils.cfg’ (2) + + +local ‘setup.cfg’ (3) + + +And on Windows, the configuration files are: + +Type of file Location and filename Notes + +------------------------------------------------------------------------------------- + +system ‘`prefix'\Lib\distutils\distutils.cfg’ (4) + + +personal ‘%HOME%\pydistutils.cfg’ (5) + + +local ‘setup.cfg’ (3) + + +On all platforms, the “personal” file can be temporarily disabled by +passing the ‘–no-user-cfg’ option. + +Notes: + + 1. Strictly speaking, the system-wide configuration file lives in the + directory where the Distutils are installed; under Python 1.6 and + later on Unix, this is as shown. For Python 1.5.2, the Distutils + will normally be installed to + ‘`prefix'/lib/python1.5/site-packages/distutils’, so the system + configuration file should be put there under Python 1.5.2. + + 2. On Unix, if the ‘HOME’ environment variable is not defined, the + user’s home directory will be determined with the ‘getpwuid()’ + function from the standard *note pwd: d6. module. This is done by + the *note os.path.expanduser(): 1fe. function used by Distutils. + + 3. I.e., in the current directory (usually the location of the setup + script). + + 4. (See also note (1).) Under Python 1.6 and later, Python’s default + “installation prefix” is ‘C:\Python’, so the system configuration + file is normally ‘C:\Python\Lib\distutils\distutils.cfg’. Under + Python 1.5.2, the default prefix was ‘C:\Program Files\Python’, and + the Distutils were not part of the standard library—so the system + configuration file would be ‘C:\Program + Files\Python\distutils\distutils.cfg’ in a standard Python 1.5.2 + installation under Windows. + + 5. On Windows, if the ‘HOME’ environment variable is not defined, + ‘USERPROFILE’ then ‘HOMEDRIVE’ and ‘HOMEPATH’ will be tried. This + is done by the *note os.path.expanduser(): 1fe. function used by + Distutils. + + +File: python.info, Node: Syntax of config files, Prev: Location and names of config files, Up: Distutils Configuration Files + +18.5.2 Syntax of config files +----------------------------- + +The Distutils configuration files all have the same syntax. The config +files are grouped into sections. There is one section for each +Distutils command, plus a ‘global’ section for global options that +affect every command. Each section consists of one option per line, +specified as ‘option=value’. + +For example, the following is a complete config file that just forces +all commands to run quietly by default: + + [global] + verbose=0 + +If this is installed as the system config file, it will affect all +processing of any Python module distribution by any user on the current +system. If it is installed as your personal config file (on systems +that support them), it will affect only module distributions processed +by you. And if it is used as the ‘setup.cfg’ for a particular module +distribution, it affects only that distribution. + +You could override the default “build base” directory and make the +‘build*’ commands always forcibly rebuild all files with the following: + + [build] + build-base=blib + force=1 + +which corresponds to the command-line arguments + + python setup.py build --build-base=blib --force + +except that including the ‘build’ command on the command-line means that +command will be run. Including a particular command in config files has +no such implication; it only means that if the command is run, the +options in the config file will apply. (Or if other commands that +derive values from it are run, they will use the values in the config +file.) + +You can find out the complete list of options for any command using the +‘--help’ option, e.g.: + + python setup.py build --help + +and you can find out the complete list of global options by using +‘--help’ without a command: + + python setup.py --help + +See also the “Reference” section of the “Distributing Python Modules” +manual. + + +File: python.info, Node: Building Extensions Tips and Tricks, Prev: Distutils Configuration Files, Up: Installing Python Modules Legacy version + +18.6 Building Extensions: Tips and Tricks +========================================= + +Whenever possible, the Distutils try to use the configuration +information made available by the Python interpreter used to run the +‘setup.py’ script. For example, the same compiler and linker flags used +to compile Python will also be used for compiling extensions. Usually +this will work well, but in complicated situations this might be +inappropriate. This section discusses how to override the usual +Distutils behaviour. + +* Menu: + +* Tweaking compiler/linker flags:: +* Using non-Microsoft compilers on Windows:: + + +File: python.info, Node: Tweaking compiler/linker flags, Next: Using non-Microsoft compilers on Windows, Up: Building Extensions Tips and Tricks + +18.6.1 Tweaking compiler/linker flags +------------------------------------- + +Compiling a Python extension written in C or C++ will sometimes require +specifying custom flags for the compiler and linker in order to use a +particular library or produce a special kind of object code. This is +especially true if the extension hasn’t been tested on your platform, or +if you’re trying to cross-compile Python. + +In the most general case, the extension author might have foreseen that +compiling the extensions would be complicated, and provided a ‘Setup’ +file for you to edit. This will likely only be done if the module +distribution contains many separate extension modules, or if they often +require elaborate sets of compiler flags in order to work. + +A ‘Setup’ file, if present, is parsed in order to get a list of +extensions to build. Each line in a ‘Setup’ describes a single module. +Lines have the following structure: + + module ... [sourcefile ...] [cpparg ...] [library ...] + +Let’s examine each of the fields in turn. + + * `module' is the name of the extension module to be built, and + should be a valid Python identifier. You can’t just change this in + order to rename a module (edits to the source code would also be + needed), so this should be left alone. + + * `sourcefile' is anything that’s likely to be a source code file, at + least judging by the filename. Filenames ending in ‘.c’ are + assumed to be written in C, filenames ending in ‘.C’, ‘.cc’, and + ‘.c++’ are assumed to be C++, and filenames ending in ‘.m’ or ‘.mm’ + are assumed to be in Objective C. + + * `cpparg' is an argument for the C preprocessor, and is anything + starting with ‘-I’, ‘-D’, ‘-U’ or ‘-C’. + + * `library' is anything ending in ‘.a’ or beginning with ‘-l’ or + ‘-L’. + +If a particular platform requires a special library on your platform, +you can add it by editing the ‘Setup’ file and running ‘python setup.py +build’. For example, if the module defined by the line + + foo foomodule.c + +must be linked with the math library ‘libm.a’ on your platform, simply +add ‘-lm’ to the line: + + foo foomodule.c -lm + +Arbitrary switches intended for the compiler or the linker can be +supplied with the ‘-Xcompiler’ `arg' and ‘-Xlinker’ `arg' options: + + foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm + +The next option after ‘-Xcompiler’ and ‘-Xlinker’ will be appended to +the proper command line, so in the above example the compiler will be +passed the ‘-o32’ option, and the linker will be passed ‘-shared’. If a +compiler option requires an argument, you’ll have to supply multiple +‘-Xcompiler’ options; for example, to pass ‘-x c++’ the ‘Setup’ file +would have to contain ‘-Xcompiler -x -Xcompiler c++’. + +Compiler flags can also be supplied through setting the ‘CFLAGS’ +environment variable. If set, the contents of ‘CFLAGS’ will be added to +the compiler flags specified in the ‘Setup’ file. + + +File: python.info, Node: Using non-Microsoft compilers on Windows, Prev: Tweaking compiler/linker flags, Up: Building Extensions Tips and Tricks + +18.6.2 Using non-Microsoft compilers on Windows +----------------------------------------------- + +* Menu: + +* Borland/CodeGear C++:: +* GNU C / Cygwin / MinGW:: + + +File: python.info, Node: Borland/CodeGear C++, Next: GNU C / Cygwin / MinGW, Up: Using non-Microsoft compilers on Windows + +18.6.2.1 Borland/CodeGear C++ +............................. + +This subsection describes the necessary steps to use Distutils with the +Borland C++ compiler version 5.5. First you have to know that Borland’s +object file format (OMF) is different from the format used by the Python +version you can download from the Python or ActiveState Web site. +(Python is built with Microsoft Visual C++, which uses COFF as the +object file format.) For this reason you have to convert Python’s +library ‘python25.lib’ into the Borland format. You can do this as +follows: + + coff2omf python25.lib python25_bcpp.lib + +The ‘coff2omf’ program comes with the Borland compiler. The file +‘python25.lib’ is in the ‘Libs’ directory of your Python installation. +If your extension uses other libraries (zlib, …) you have to convert +them too. + +The converted files have to reside in the same directories as the normal +libraries. + +How does Distutils manage to use these libraries with their changed +names? If the extension needs a library (eg. ‘foo’) Distutils checks +first if it finds a library with suffix ‘_bcpp’ (eg. ‘foo_bcpp.lib’) +and then uses this library. In the case it doesn’t find such a special +library it uses the default name (‘foo.lib’.) (1) + +To let Distutils compile your extension with Borland C++ you now have to +type: + + python setup.py build --compiler=bcpp + +If you want to use the Borland C++ compiler as the default, you could +specify this in your personal or system-wide configuration file for +Distutils (see section *note Distutils Configuration Files: 41b6.) + +See also +........ + +C++Builder Compiler(2) + + Information about the free C++ compiler from Borland, including + links to the download pages. + +Creating Python Extensions Using Borland’s Free Compiler(3) + + Document describing how to use Borland’s free command-line C++ + compiler to build Python. + + ---------- Footnotes ---------- + + (1) (1) This also means you could replace all existing COFF-libraries +with OMF-libraries of the same name. + + (2) https://www.embarcadero.com/products + + (3) http://www.cyberus.ca/~g_will/pyExtenDL.shtml + + +File: python.info, Node: GNU C / Cygwin / MinGW, Prev: Borland/CodeGear C++, Up: Using non-Microsoft compilers on Windows + +18.6.2.2 GNU C / Cygwin / MinGW +............................... + +This section describes the necessary steps to use Distutils with the GNU +C/C++ compilers in their Cygwin and MinGW distributions. (1) For a +Python interpreter that was built with Cygwin, everything should work +without any of these following steps. + +Not all extensions can be built with MinGW or Cygwin, but many can. +Extensions most likely to not work are those that use C++ or depend on +Microsoft Visual C extensions. + +To let Distutils compile your extension with Cygwin you have to type: + + python setup.py build --compiler=cygwin + +and for Cygwin in no-cygwin mode (2) or for MinGW type: + + python setup.py build --compiler=mingw32 + +If you want to use any of these options/compilers as default, you should +consider writing it in your personal or system-wide configuration file +for Distutils (see section *note Distutils Configuration Files: 41b6.) + +* Menu: + +* Older Versions of Python and MinGW:: + + ---------- Footnotes ---------- + + (1) (2) Check ‘https://www.sourceware.org/cygwin/’ for more +information + + (2) (3) Then you have no POSIX emulation available, but you also +don’t need ‘cygwin1.dll’. + + +File: python.info, Node: Older Versions of Python and MinGW, Up: GNU C / Cygwin / MinGW + +18.6.2.3 Older Versions of Python and MinGW +........................................... + +The following instructions only apply if you’re using a version of +Python inferior to 2.4.1 with a MinGW inferior to 3.0.0 (with +binutils-2.13.90-20030111-1). + +These compilers require some special libraries. This task is more +complex than for Borland’s C++, because there is no program to convert +the library. First you have to create a list of symbols which the +Python DLL exports. (You can find a good program for this task at +‘https://sourceforge.net/projects/mingw/files/MinGW/Extension/pexports/’). + + pexports python25.dll >python25.def + +The location of an installed ‘python25.dll’ will depend on the +installation options and the version and language of Windows. In a +“just for me” installation, it will appear in the root of the +installation directory. In a shared installation, it will be located in +the system directory. + +Then you can create from these information an import library for gcc. + + /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a + +The resulting library has to be placed in the same directory as +‘python25.lib’. (Should be the ‘libs’ directory under your Python +installation directory.) + +If your extension uses other libraries (zlib,…) you might have to +convert them too. The converted files have to reside in the same +directories as the normal libraries do. + +See also +........ + +Building Python modules on MS Windows platform with MinGW(1) + + Information about building the required libraries for the MinGW + environment. + + ---------- Footnotes ---------- + + (1) http://old.zope.org/Members/als/tips/win32_mingw_modules + + +File: python.info, Node: Python Module Index, Next: Index, Prev: Installing Python Modules Legacy version, Up: Top + +Python Module Index +******************* + +* Menu: + +* __future__: 0. Future statement definitions +* __main__: 1. The environment where the top-level script is run. +* _dummy_thread: 2. Drop-in replacement for the _thread module. +* _thread: 3. Low-level threading API. +* abc: 4. Abstract base classes according to :pep:‘3119‘. +* aifc: 5. Read and write audio files in AIFF or AIFC format. +* argparse: 6. Command-line option and argument parsing library. +* array: 7. Space efficient arrays of uniformly typed numeric + values. +* ast: 8. Abstract Syntax Tree classes and manipulation. +* asynchat: 9. Support for asynchronous command/response protocols. +* asyncio: a. Asynchronous I/O. +* asyncore: b. A base class for developing asynchronous socket + handling services. +* atexit: c. Register and execute cleanup functions. +* audioop: d. Manipulate raw audio data. +* base64: e. RFC 3548: Base16, Base32, Base64 Data + Encodings; Base85 and Ascii85 +* bdb: f. Debugger framework. +* binascii: 10. Tools for converting between binary and various ASCII- + encoded binary representations. +* binhex: 11. Encode and decode files in binhex4 format. +* bisect: 12. Array bisection algorithms for binary searching. +* builtins: 13. The module that provides the built-in namespace. +* bz2: 14. Interfaces for bzip2 compression and decompression. +* calendar: 15. Functions for working with calendars, including + some emulation of the Unix cal program. +* cgi: 16. Helpers for running Python scripts via the Common + Gateway Interface. +* cgitb: 17. Configurable traceback handler for CGI scripts. +* chunk: 18. Module to read IFF chunks. +* cmath: 19. Mathematical functions for complex numbers. +* cmd: 1a. Build line-oriented command interpreters. +* code: 1b. Facilities to implement read-eval-print loops. +* codecs: 1c. Encode and decode data and streams. +* codeop: 1d. Compile (possibly incomplete) Python code. +* collections: 1e. Container datatypes +* collections.abc: 1f. Abstract base classes for containers +* colorsys: 20. Conversion functions between RGB and other color + systems. +* compileall: 21. Tools for byte-compiling all Python source files in a + directory tree. +* concurrent.futures: 22. Execute computations concurrently using threads or + processes. +* configparser: 23. Configuration file parser. +* contextlib: 24. Utilities for with-statement contexts. +* contextvars: 25. Context Variables +* copy: 26. Shallow and deep copy operations. +* copyreg: 27. Register pickle support functions. +* cProfile: 28. +* crypt: 29. The crypt() function used to check Unix passwords. +* csv: 2a. Write and read tabular data to and from delimited + files. +* ctypes: 2b. A foreign function library for Python. +* curses: 2c. An interface to the curses library, providing + portable terminal handling. +* curses.ascii: 2d. Constants and set-membership functions for ASCII + characters. +* curses.panel: 2e. A panel stack extension that adds depth to curses + windows. +* curses.textpad: 2f. Emacs-like input editing in a curses window. +* dataclasses: 30. Generate special methods on user-defined classes. +* datetime: 31. Basic date and time types. +* dbm: 32. Interfaces to various Unix "database" formats. +* dbm.dumb: 33. Portable implementation of the simple DBM interface. +* dbm.gnu: 34. GNU’s reinterpretation of dbm. +* dbm.ndbm: 35. The standard "database" interface, based on + ndbm. +* decimal: 36. Implementation of the General Decimal Arithmetic + Specification. +* difflib: 37. Helpers for computing differences between objects. +* dis: 38. Disassembler for Python bytecode. +* distutils: 39. Support for building and installing Python modules + into an existing Python installation. +* distutils.archive_util: 3a. Utility functions for creating archive files + (tarballs, zip files, ...) +* distutils.bcppcompiler: 3b. +* distutils.ccompiler: 3c. Abstract CCompiler class +* distutils.cmd: 3d. Provides the abstract base class + :class:‘~distutils.cmd.Command‘. This class is + subclassed by the modules in the distutils.command + subpackage. +* distutils.command: 3e. Contains one module for each standard Distutils + command. +* distutils.command.bdist: 3f. Build a binary installer for a package +* distutils.command.bdist_dumb: 40. Build a "dumb" installer - a simple archive of + files +* distutils.command.bdist_msi: 41. Build a binary distribution as a Windows MSI + file +* distutils.command.bdist_packager: 42. Abstract base class for packagers +* distutils.command.bdist_rpm: 43. Build a binary distribution as a Redhat RPM and + SRPM +* distutils.command.bdist_wininst: 44. Build a Windows installer +* distutils.command.build: 45. Build all files of a package +* distutils.command.build_clib: 46. Build any C libraries in a package +* distutils.command.build_ext: 47. Build any extensions in a package +* distutils.command.build_py: 48. Build the .py/.pyc files of a package +* distutils.command.build_scripts: 49. Build the scripts of a package +* distutils.command.check: 4a. Check the meta-data of a package +* distutils.command.clean: 4b. Clean a package build area +* distutils.command.config: 4c. Perform package configuration +* distutils.command.install: 4d. Install a package +* distutils.command.install_data: 4e. Install data files from a package +* distutils.command.install_headers: 4f. Install C/C++ header files from a package +* distutils.command.install_lib: 50. Install library files from a package +* distutils.command.install_scripts: 51. Install script files from a package +* distutils.command.register: 52. Register a module with the Python Package Index +* distutils.command.sdist: 53. Build a source distribution +* distutils.core: 54. The core Distutils functionality +* distutils.cygwinccompiler: 55. +* distutils.debug: 56. Provides the debug flag for distutils +* distutils.dep_util: 57. Utility functions for simple dependency checking +* distutils.dir_util: 58. Utility functions for operating on directories and + directory trees +* distutils.dist: 59. Provides the Distribution class, which + represents the module distribution being + built/installed/distributed +* distutils.errors: 5a. Provides standard distutils exceptions +* distutils.extension: 5b. Provides the Extension class, used to describe + C/C++ extension modules in setup scripts +* distutils.fancy_getopt: 5c. Additional getopt functionality +* distutils.file_util: 5d. Utility functions for operating on single files +* distutils.filelist: 5e. The FileList class, used for poking about the + file system and building lists of files. +* distutils.log: 5f. A simple logging mechanism, :pep:‘282‘-style +* distutils.msvccompiler: 60. Microsoft Compiler +* distutils.spawn: 61. Provides the spawn() function +* distutils.sysconfig: 62. Low-level access to configuration information of the + Python interpreter. +* distutils.text_file: 63. Provides the TextFile class, a simple interface + to text files +* distutils.unixccompiler: 64. UNIX C Compiler +* distutils.util: 65. Miscellaneous other utility functions +* distutils.version: 66. Implements classes that represent module version + numbers. +* doctest: 67. Test pieces of code within docstrings. +* dummy_threading: 68. Drop-in replacement for the threading module. +* email: 69. Package supporting the parsing, + manipulating, and generating email messages. +* email.charset: 6a. Character Sets +* email.contentmanager: 6b. Storing and Retrieving Content from MIME Parts +* email.encoders: 6c. Encoders for email message payloads. +* email.errors: 6d. The exception classes used by the email package. +* email.generator: 6e. Generate flat text email messages from a message + structure. +* email.header: 6f. Representing non-ASCII headers +* email.headerregistry: 70. Automatic Parsing of headers based on the field name +* email.iterators: 71. Iterate over a message object tree. +* email.message: 72. The base class representing email messages. +* email.mime: 73. Build MIME messages. +* email.parser: 74. Parse flat text email messages to produce a message + object structure. +* email.policy: 75. Controlling the parsing and generating of messages +* email.utils: 76. Miscellaneous email package utilities. +* encodings.idna: 77. Internationalized Domain Names implementation +* encodings.mbcs: 78. Windows ANSI codepage +* encodings.utf_8_sig: 79. UTF-8 codec with BOM signature +* ensurepip: 7a. Bootstrapping the "pip" installer into an existing + Python installation or virtual environment. +* enum: 7b. Implementation of an enumeration class. +* errno: 7c. Standard errno system symbols. +* faulthandler: 7d. Dump the Python traceback. +* fcntl: 7e. The fcntl() and ioctl() system calls. +* filecmp: 7f. Compare files efficiently. +* fileinput: 80. Loop over standard input or a list of files. +* fnmatch: 81. Unix shell style filename pattern matching. +* formatter: 82. Generic output formatter and device interface. +* fractions: 83. Rational numbers. +* ftplib: 84. FTP protocol client (requires sockets). +* functools: 85. Higher-order functions and operations on callable + objects. +* gc: 86. Interface to the cycle-detecting garbage collector. +* getopt: 87. Portable parser for command line options; support both + short and long option names. +* getpass: 88. Portable reading of passwords and retrieval of the + userid. +* gettext: 89. Multilingual internationalization services. +* glob: 8a. Unix shell style pathname pattern expansion. +* grp: 8b. The group database (getgrnam() and friends). +* gzip: 8c. Interfaces for gzip compression and decompression + using file objects. +* hashlib: 8d. Secure hash and message digest algorithms. +* heapq: 8e. Heap queue algorithm (a.k.a. priority queue). +* hmac: 8f. Keyed-Hashing for Message Authentication (HMAC) + implementation +* html: 90. Helpers for manipulating HTML. +* html.entities: 91. Definitions of HTML general entities. +* html.parser: 92. A simple parser that can handle HTML and XHTML. +* http: 93. HTTP status codes and messages +* http.client: 94. HTTP and HTTPS protocol client (requires sockets). +* http.cookiejar: 95. Classes for automatic handling of HTTP cookies. +* http.cookies: 96. Support for HTTP state management (cookies). +* http.server: 97. HTTP server and request handlers. +* imaplib: 98. IMAP4 protocol client (requires sockets). +* imghdr: 99. Determine the type of image contained in a file or + byte stream. +* imp: 9a. Access the implementation of the import statement. +* importlib: 9b. The implementation of the import machinery. +* importlib.abc: 9c. Abstract base classes related to import +* importlib.machinery: 9d. Importers and path hooks +* importlib.resources: 9e. Package resource reading, opening, and + access +* importlib.util: 9f. Utility code for importers +* inspect: a0. Extract information and source code from live objects. +* io: a1. Core tools for working with streams. +* ipaddress: a2. IPv4/IPv6 manipulation library. +* itertools: a3. Functions creating iterators for efficient looping. +* json: a4. Encode and decode the JSON format. +* json.tool: a5. A command line to validate and pretty-print JSON. +* keyword: a6. Test whether a string is a keyword in Python. +* lib2to3: a7. The 2to3 library +* linecache: a8. Provides random access to individual lines from text + files. +* locale: a9. Internationalization services. +* logging: aa. Flexible event logging system for applications. +* logging.config: ab. Configuration of the logging module. +* logging.handlers: ac. Handlers for the logging module. +* lzma: ad. A Python wrapper for the liblzma compression library. +* mailbox: ae. Manipulate mailboxes in various formats +* mailcap: af. Mailcap file handling. +* marshal: b0. Convert Python objects to streams of bytes and back + (with different constraints). +* math: b1. Mathematical functions (sin() etc.). +* mimetypes: b2. Mapping of filename extensions to MIME types. +* mmap: b3. Interface to memory-mapped files for Unix and Windows. +* modulefinder: b4. Find modules used by a script. +* msilib: b5. Creation of Microsoft Installer files, and CAB + files. +* msvcrt: b6. Miscellaneous useful routines from the MS VC++ + runtime. +* multiprocessing: b7. Process-based parallelism. +* multiprocessing.connection: b8. API for dealing with sockets. +* multiprocessing.dummy: b9. Dumb wrapper around threading. +* multiprocessing.managers: ba. Share data between process with shared objects. +* multiprocessing.pool: bb. Create pools of processes. +* multiprocessing.shared_memory: bc. Provides shared memory for direct access + across processes. +* multiprocessing.sharedctypes: bd. Allocate ctypes objects from shared memory. +* netrc: be. Loading of .netrc files. +* nis: bf. Interface to Sun’s NIS (Yellow Pages) library. +* nntplib: c0. NNTP protocol client (requires sockets). +* numbers: c1. Numeric abstract base classes (Complex, + Real, Integral, etc.). +* operator: c2. Functions corresponding to the standard operators. +* optparse: c3. Command-line option parsing library. +* os: c4. Miscellaneous operating system interfaces. +* os.path: c5. Operations on pathnames. +* ossaudiodev: c6. Access to OSS-compatible audio devices. +* parser: c7. Access parse trees for Python source code. +* pathlib: c8. Object-oriented filesystem paths +* pdb: c9. The Python debugger for interactive interpreters. +* pickle: ca. Convert Python objects to streams of bytes and back. +* pickletools: cb. Contains extensive comments about the pickle protocols + and pickle-machine opcodes, as well as some + useful functions. +* pipes: cc. A Python interface to Unix shell pipelines. +* pkgutil: cd. Utilities for the import system. +* platform: ce. Retrieves as much platform identifying data as + possible. +* plistlib: cf. Generate and parse Mac OS X plist files. +* poplib: d0. POP3 protocol client (requires sockets). +* posix: d1. The most common POSIX system calls (normally used via + module os). +* pprint: d2. Data pretty printer. +* profile: d3. Python source profiler. +* pstats: d4. Statistics object for use with the profiler. +* pty: d5. Pseudo-Terminal Handling for Linux. +* pwd: d6. The password database (getpwnam() and friends). +* py_compile: d7. Generate byte-code files from Python source files. +* pyclbr: d8. Supports information extraction for a Python module + browser. +* pydoc: d9. Documentation generator and online help system. +* queue: da. A synchronized queue class. +* quopri: db. Encode and decode files using the MIME quoted- + printable encoding. +* random: dc. Generate pseudo-random numbers with various common + distributions. +* re: dd. Regular expression operations. +* readline: de. GNU readline support for Python. +* reprlib: df. Alternate repr() implementation with size limits. +* resource: e0. An interface to provide resource usage information on + the current process. +* rlcompleter: e1. Python identifier completion, suitable for the + GNU readline library. +* runpy: e2. Locate and run Python modules without importing them + first. +* sched: e3. General purpose event scheduler. +* secrets: e4. Generate secure random numbers for managing secrets. +* select: e5. Wait for I/O completion on multiple streams. +* selectors: e6. High-level I/O multiplexing. +* shelve: e7. Python object persistence. +* shlex: e8. Simple lexical analysis for Unix shell-like languages. +* shutil: e9. High-level file operations, including copying. +* signal: ea. Set handlers for asynchronous events. +* site: eb. Module responsible for site-specific configuration. +* smtpd: ec. A SMTP server implementation in Python. +* smtplib: ed. SMTP protocol client (requires sockets). +* sndhdr: ee. Determine type of a sound file. +* socket: ef. Low-level networking interface. +* socketserver: f0. A framework for network servers. +* spwd: f1. The shadow password database (getspnam() and friends). +* sqlite3: f2. A DB-API 2.0 implementation using SQLite 3.x. +* ssl: f3. TLS/SSL wrapper for socket objects +* stat: f4. Utilities for interpreting the results of + os.stat(), os.lstat() and os.fstat(). +* statistics: f5. Mathematical statistics functions +* string: f6. Common string operations. +* stringprep: f7. String preparation, as per RFC 3453 +* struct: f8. Interpret bytes as packed binary data. +* subprocess: f9. Subprocess management. +* sunau: fa. Provide an interface to the Sun AU sound format. +* symbol: fb. Constants representing internal nodes of the parse + tree. +* symtable: fc. Interface to the compiler’s internal symbol tables. +* sys: fd. Access system-specific parameters and functions. +* sysconfig: fe. Python’s configuration information +* syslog: ff. An interface to the Unix syslog library routines. +* tabnanny: 100. Tool for detecting white space related problems in + Python source files in a directory tree. +* tarfile: 101. Read and write tar-format archive files. +* telnetlib: 102. Telnet client class. +* tempfile: 103. Generate temporary files and directories. +* termios: 104. POSIX style tty control. +* test: 105. Regression tests package containing the testing suite + for Python. +* test.support: 106. Support for Python’s regression test suite. +* test.support.script_helper: 107. Support for Python’s script execution tests. +* textwrap: 108. Text wrapping and filling +* threading: 109. Thread-based parallelism. +* time: 10a. Time access and conversions. +* timeit: 10b. Measure the execution time of small code snippets. +* tkinter: 10c. Interface to Tcl/Tk for graphical user interfaces +* tkinter.scrolledtext: 10d. Text widget with a vertical scroll bar. +* tkinter.tix: 10e. Tk Extension Widgets for Tkinter +* tkinter.ttk: 10f. Tk themed widget set +* token: 110. Constants representing terminal nodes of the parse + tree. +* tokenize: 111. Lexical scanner for Python source code. +* trace: 112. Trace or track Python statement execution. +* traceback: 113. Print or retrieve a stack traceback. +* tracemalloc: 114. Trace memory allocations. +* tty: 115. Utility functions that perform common terminal control + operations. +* turtle: 116. An educational framework for simple graphics + applications +* turtledemo: 117. A viewer for example turtle scripts +* types: 118. Names for built-in types. +* typing: 119. Support for type hints (see :pep:‘484‘). +* unicodedata: 11a. Access the Unicode Database. +* unittest: 11b. Unit testing framework for Python. +* unittest.mock: 11c. Mock object library. +* urllib: 11d. +* urllib.error: 11e. Exception classes raised by urllib.request. +* urllib.parse: 11f. Parse URLs into or assemble them from components. +* urllib.request: 120. Extensible library for opening URLs. +* urllib.response: 121. Response classes used by urllib. +* urllib.robotparser: 122. Load a robots.txt file and answer questions about + fetchability of other URLs. +* uu: 123. Encode and decode files in uuencode format. +* uuid: 124. UUID objects (universally unique identifiers) + according to RFC 4122 +* venv: 125. Creation of virtual environments. +* warnings: 126. Issue warning messages and control their disposition. +* wave: 127. Provide an interface to the WAV sound format. +* weakref: 128. Support for weak references and weak dictionaries. +* webbrowser: 129. Easy-to-use controller for Web browsers. +* winreg: 12a. Routines and objects for manipulating the Windows + registry. +* winsound: 12b. Access to the sound-playing machinery for Windows. +* wsgiref: 12c. WSGI Utilities and Reference Implementation. +* wsgiref.handlers: 12d. WSGI server/gateway base classes. +* wsgiref.headers: 12e. WSGI response header tools. +* wsgiref.simple_server: 12f. A simple WSGI HTTP server. +* wsgiref.util: 130. WSGI environment utilities. +* wsgiref.validate: 131. WSGI conformance checker. +* xdrlib: 132. Encoders and decoders for the External Data + Representation (XDR). +* xml: 133. Package containing XML processing modules +* xml.dom: 134. Document Object Model API for Python. +* xml.dom.minidom: 135. Minimal Document Object Model (DOM) implementation. +* xml.dom.pulldom: 136. Support for building partial DOM trees from SAX + events. +* xml.etree.ElementTree: 137. Implementation of the ElementTree API. +* xml.parsers.expat: 138. An interface to the Expat non-validating XML parser. +* xml.parsers.expat.errors: 139. +* xml.parsers.expat.model: 13a. +* xml.sax: 13b. Package containing SAX2 base classes and convenience + functions. +* xml.sax.handler: 13c. Base classes for SAX event handlers. +* xml.sax.saxutils: 13d. Convenience functions and classes for use with SAX. +* xml.sax.xmlreader: 13e. Interface which SAX-compliant XML parsers must + implement. +* xmlrpc.client: 13f. XML-RPC client access. +* xmlrpc.server: 140. Basic XML-RPC server implementations. +* zipapp: 141. Manage executable Python zip archives +* zipfile: 142. Read and write ZIP-format archive files. +* zipimport: 143. Support for importing Python modules from ZIP + archives. +* zlib: 144. Low-level interface to compression and decompression + routines compatible with gzip. + + +File: python.info, Node: Index, Prev: Python Module Index, Up: Top + +Index +***** + +[index] +* Menu: + +* ! (exclamation); in a command interpreter: Cmd Objects. (line 27) +* ! (exclamation); in curses module: curses ascii — Utilities for ASCII characters. + (line 226) +* ! (exclamation); in formatted string literal: String literal concatenation. + (line 24) +* ! (exclamation); in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 10) +* ! (exclamation); in glob-style wildcards <1>: fnmatch — Unix filename pattern matching. + (line 15) +* ! (exclamation); in string formatting: Format String Syntax. + (line 13) +* ! (exclamation); in struct format strings: Byte Order Size and Alignment. + (line 10) +* ! (pdb command): Debugger Commands. (line 311) +* " (double quote); string literal: Literals. (line 8) +* """; string literal: String and Bytes literals. + (line 36) +* # (hash); comment: An Informal Introduction to Python. + (line 14) +* # (hash); comment <1>: Comments. (line 6) +* # (hash); comment <2>: site — Site-specific configuration hook. + (line 45) +* # (hash); in doctests: Option Flags. (line 183) +* # (hash); in printf-style formatting: printf-style String Formatting. + (line 66) +* # (hash); in printf-style formatting <1>: printf-style Bytes Formatting. + (line 63) +* # (hash); in regular expressions: Module Contents. (line 123) +* # (hash); in string formatting: Format Specification Mini-Language. + (line 87) +* # (hash); source encoding declaration: Encoding declarations. + (line 6) +* $ (dollar); environment variables expansion: os path — Common pathname manipulations. + (line 158) +* $ (dollar); in regular expressions: Regular Expression Syntax. + (line 57) +* $ (dollar); in template strings: Template strings. (line 13) +* $ (dollar); interpolation in configuration files: Interpolation of values. + (line 40) +* % (percent); datetime format: timezone Objects. (line 71) +* % (percent); datetime format <1>: Functions<2>. (line 230) +* % (percent); datetime format <2>: Functions<2>. (line 362) +* % (percent); environment variables expansion (Windows): os path — Common pathname manipulations. + (line 158) +* % (percent); environment variables expansion (Windows) <1>: Functions<11>. + (line 233) +* % (percent); interpolation in configuration files: Interpolation of values. + (line 10) +* % (percent); printf-style formatting: printf-style String Formatting. + (line 6) +* % (percent); printf-style formatting <1>: printf-style Bytes Formatting. + (line 6) +* %=; augmented assignment: Augmented assignment statements. + (line 6) +* &=; augmented assignment: Augmented assignment statements. + (line 6) +* ’ (single quote); string literal: Literals. (line 8) +* ’’’; string literal: String and Bytes literals. + (line 36) +* () (parentheses); call: Slicings. (line 38) +* () (parentheses); class definition: Class definitions. (line 6) +* () (parentheses); function definition: Function definitions. + (line 6) +* () (parentheses); generator expression: Generator expressions. + (line 6) +* () (parentheses); in assignment target list: Assignment statements. + (line 35) +* () (parentheses); in printf-style formatting: printf-style String Formatting. + (line 26) +* () (parentheses); in printf-style formatting <1>: printf-style Bytes Formatting. + (line 23) +* () (parentheses); in regular expressions: Regular Expression Syntax. + (line 199) +* () (parentheses); tuple display: Parenthesized forms. + (line 6) +* (?!; in regular expressions: Regular Expression Syntax. + (line 316) +* (?#; in regular expressions: Regular Expression Syntax. + (line 305) +* (?; in regular expressions: Regular Expression Syntax. + (line 208) +* (?;; in regular expressions: Regular Expression Syntax. + (line 232) +* (?: Calls. (line 74) +* * (asterisk); in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 10) +* * (asterisk); in glob-style wildcards <1>: fnmatch — Unix filename pattern matching. + (line 15) +* * (asterisk); in printf-style formatting: printf-style String Formatting. + (line 26) +* * (asterisk); in printf-style formatting <1>: printf-style Bytes Formatting. + (line 23) +* * (asterisk); in regular expressions: Regular Expression Syntax. + (line 68) +* **; function definition: Function definitions. + (line 78) +* **; in dictionary displays: Dictionary displays. + (line 23) +* **; in function calls: Unpacking Argument Lists. + (line 19) +* **; in function calls <1>: Calls. (line 100) +* **; in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 41) +* **=; augmented assignment: Augmented assignment statements. + (line 6) +* *=; augmented assignment: Augmented assignment statements. + (line 6) +* *?; in regular expressions: Regular Expression Syntax. + (line 85) +* + (plus); binary operator: Binary arithmetic operations. + (line 60) +* + (plus); binary operator <1>: Numeric Types — int float complex. + (line 27) +* + (plus); in argparse module: nargs. (line 68) +* + (plus); in doctests: Option Flags. (line 183) +* + (plus); in printf-style formatting: printf-style String Formatting. + (line 66) +* + (plus); in printf-style formatting <1>: printf-style Bytes Formatting. + (line 63) +* + (plus); in regular expressions: Regular Expression Syntax. + (line 74) +* + (plus); in string formatting: Format Specification Mini-Language. + (line 71) +* + (plus); unary operator: Unary arithmetic and bitwise operations. + (line 13) +* + (plus); unary operator <1>: Numeric Types — int float complex. + (line 27) +* +=; augmented assignment: Augmented assignment statements. + (line 6) +* +?; in regular expressions: Regular Expression Syntax. + (line 85) +* , (comma): Parenthesized forms. + (line 20) +* , (comma); argument list: Slicings. (line 38) +* , (comma); expression list: List displays. (line 6) +* , (comma); expression list <1>: Set displays. (line 6) +* , (comma); expression list <2>: Expression lists. (line 6) +* , (comma); expression list <3>: The assert statement. + (line 6) +* , (comma); expression list <4>: Class definitions. (line 6) +* , (comma); identifier list: The global statement. + (line 6) +* , (comma); identifier list <1>: The nonlocal statement. + (line 6) +* , (comma); import statement: The import statement. + (line 6) +* , (comma); in dictionary displays: Dictionary displays. + (line 6) +* , (comma); in string formatting: Format Specification Mini-Language. + (line 99) +* , (comma); in target list: Assignment statements. + (line 35) +* , (comma); parameter list: Function definitions. + (line 6) +* , (comma); slicing: Slicings. (line 6) +* , (comma); with statement: The with statement. (line 6) +* - (minus); binary operator: Binary arithmetic operations. + (line 66) +* - (minus); binary operator <1>: Numeric Types — int float complex. + (line 27) +* - (minus); in doctests: Option Flags. (line 182) +* - (minus); in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 10) +* - (minus); in glob-style wildcards <1>: fnmatch — Unix filename pattern matching. + (line 15) +* - (minus); in printf-style formatting: printf-style String Formatting. + (line 66) +* - (minus); in printf-style formatting <1>: printf-style Bytes Formatting. + (line 63) +* - (minus); in regular expressions: Regular Expression Syntax. + (line 143) +* - (minus); in string formatting: Format Specification Mini-Language. + (line 71) +* - (minus); unary operator: Unary arithmetic and bitwise operations. + (line 10) +* - (minus); unary operator <1>: Numeric Types — int float complex. + (line 27) +* -=; augmented assignment: Augmented assignment statements. + (line 6) +* ->; function annotations: Function Annotations. + (line 6) +* ->; function annotations <1>: Function definitions. + (line 89) +* . (dot); attribute reference: Attribute references. + (line 6) +* . (dot); in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 10) +* . (dot); in numeric literal: Integer literals. (line 40) +* . (dot); in pathnames: Miscellaneous System Information. + (line 81) +* . (dot); in pathnames <1>: Miscellaneous System Information. + (line 109) +* . (dot); in printf-style formatting: printf-style String Formatting. + (line 26) +* . (dot); in printf-style formatting <1>: printf-style Bytes Formatting. + (line 23) +* . (dot); in regular expressions: Regular Expression Syntax. + (line 46) +* . (dot); in string formatting: Format String Syntax. + (line 13) +* . (dot); in Tkinter: A Very Quick Look at Tcl/Tk. + (line 44) +* ...: Glossary. (line 10) +* ...; ellipsis literal: The standard type hierarchy. + (line 41) +* ...; ellipsis literal <1>: Built-in Constants. (line 52) +* ...; ellipsis literal <2>: The Null Object. (line 12) +* ...; in doctests: Option Flags. (line 49) +* ...; interpreter prompt: How are Docstring Examples Recognized?. + (line 26) +* ...; interpreter prompt <1>: sys — System-specific parameters and functions. + (line 1188) +* ...; placeholder: textwrap — Text wrapping and filling. + (line 266) +* ...; placeholder <1>: pprint — Data pretty printer. + (line 27) +* ...; placeholder <2>: reprlib — Alternate repr implementation. + (line 41) +* ..; in pathnames: Miscellaneous System Information. + (line 87) +* .ini; file: configparser — Configuration file parser. + (line 8) +* .pdbrc; file: Debugger Commands. (line 37) +* / (slash); in pathnames: Miscellaneous System Information. + (line 93) +* / (slash); in pathnames <1>: Miscellaneous System Information. + (line 102) +* //=; augmented assignment: Augmented assignment statements. + (line 6) +* /=; augmented assignment: Augmented assignment statements. + (line 6) +* 0b; integer literal: Numeric literals. (line 14) +* 0o; integer literal: Numeric literals. (line 14) +* 0x; integer literal: Numeric literals. (line 14) +* 2-digit years: time — Time access and conversions. + (line 35) +* 2to3: Glossary. (line 23) +* ; (colon); annotated variable: Annotated assignment statements. + (line 6) +* ; (colon); compound statement: The if statement. (line 6) +* ; (colon); compound statement <1>: The while statement. + (line 6) +* ; (colon); compound statement <2>: The for statement. (line 6) +* ; (colon); compound statement <3>: The try statement. (line 6) +* ; (colon); compound statement <4>: The with statement. (line 6) +* ; (colon); compound statement <5>: Function definitions. + (line 6) +* ; (colon); compound statement <6>: Class definitions. (line 6) +* ; (colon); function annotations: Function Annotations. + (line 6) +* ; (colon); function annotations <1>: Function definitions. + (line 89) +* ; (colon); in dictionary expressions: Dictionary displays. + (line 6) +* ; (colon); in formatted string literal: String literal concatenation. + (line 24) +* ; (colon); in SQL statements: Cursor Objects. (line 11) +* ; (colon); in string formatting: Format String Syntax. + (line 13) +* ; (colon); lambda expression: Lambdas. (line 6) +* ; (colon); path separator (POSIX): Miscellaneous System Information. + (line 115) +* ; (colon); slicing: Slicings. (line 6) +* ; (semicolon): Compound statements. + (line 18) +* ; (semicolon) <1>: Miscellaneous System Information. + (line 115) +* < (less); in string formatting: Format Specification Mini-Language. + (line 42) +* < (less); in struct format strings: Byte Order Size and Alignment. + (line 10) +* <<=; augmented assignment: Augmented assignment statements. + (line 6) +* : Option Flags. (line 30) +* = (equals); assignment statement: Assignment statements. + (line 6) +* = (equals); class definition: Metaclasses. (line 6) +* = (equals); for help in debugging using string literals: String literal concatenation. + (line 23) +* = (equals); function definition: Function definitions. + (line 53) +* = (equals); in function calls: Slicings. (line 37) +* = (equals); in string formatting: Format Specification Mini-Language. + (line 42) +* = (equals); in struct format strings: Byte Order Size and Alignment. + (line 10) +* > (greater); in string formatting: Format Specification Mini-Language. + (line 42) +* > (greater); in struct format strings: Byte Order Size and Alignment. + (line 10) +* >>=; augmented assignment: Augmented assignment statements. + (line 6) +* >>>: Glossary. (line 6) +* >>>; interpreter prompt: How are Docstring Examples Recognized?. + (line 26) +* >>>; interpreter prompt <1>: sys — System-specific parameters and functions. + (line 1188) +* ? (question mark); in a command interpreter: Cmd Objects. (line 27) +* ? (question mark); in argparse module: nargs. (line 23) +* ? (question mark); in AST grammar: Node classes. (line 22) +* ? (question mark); in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 10) +* ? (question mark); in glob-style wildcards <1>: fnmatch — Unix filename pattern matching. + (line 15) +* ? (question mark); in regular expressions: Regular Expression Syntax. + (line 80) +* ? (question mark); in SQL statements: Cursor Objects. (line 11) +* ? (question mark); in struct format strings: Format Characters. + (line 86) +* ? (question mark); in struct format strings <1>: Format Characters. + (line 161) +* ? (question mark); replacement character: Error Handlers. (line 28) +* ??; in regular expressions: Regular Expression Syntax. + (line 85) +* @ (at); class definition: Class definitions. (line 44) +* @ (at); function definition: Function definitions. + (line 33) +* @ (at); in struct format strings: Byte Order Size and Alignment. + (line 10) +* [] (square brackets); in assignment target list: Assignment statements. + (line 35) +* [] (square brackets); in glob-style wildcards: glob — Unix style pathname pattern expansion. + (line 10) +* [] (square brackets); in glob-style wildcards <1>: fnmatch — Unix filename pattern matching. + (line 15) +* [] (square brackets); in regular expressions: Regular Expression Syntax. + (line 136) +* [] (square brackets); in string formatting: Format String Syntax. + (line 13) +* [] (square brackets); list expression: List displays. (line 6) +* [] (square brackets); subscription: Subscriptions. (line 6) +* \ (backslash); escape sequence: String and Bytes literals. + (line 72) +* \ (backslash); escape sequence <1>: Error Handlers. (line 28) +* \ (backslash); in pathnames (Windows): Miscellaneous System Information. + (line 93) +* \ (backslash); in regular expressions: Regular Expression Syntax. + (line 121) +* \ (backslash); in regular expressions <1>: Regular Expression Syntax. + (line 155) +* \ (backslash); in regular expressions <2>: Regular Expression Syntax. + (line 374) +* \a; escape sequence: String and Bytes literals. + (line 72) +* \A; in regular expressions: Regular Expression Syntax. + (line 386) +* \a; in regular expressions: Regular Expression Syntax. + (line 486) +* \b; escape sequence: String and Bytes literals. + (line 72) +* \b; in regular expressions: Regular Expression Syntax. + (line 390) +* \B; in regular expressions: Regular Expression Syntax. + (line 407) +* \b; in regular expressions <1>: Regular Expression Syntax. + (line 486) +* \d; in regular expressions: Regular Expression Syntax. + (line 418) +* \D; in regular expressions: Regular Expression Syntax. + (line 431) +* \f; escape sequence: String and Bytes literals. + (line 72) +* \f; in regular expressions: Regular Expression Syntax. + (line 486) +* \g; in regular expressions: Module Contents. (line 282) +* \n; escape sequence: String and Bytes literals. + (line 72) +* \N; escape sequence: String and Bytes literals. + (line 72) +* \N; escape sequence <1>: Error Handlers. (line 28) +* \n; in regular expressions: Regular Expression Syntax. + (line 486) +* \N; in regular expressions: Regular Expression Syntax. + (line 486) +* \r; escape sequence: String and Bytes literals. + (line 72) +* \r; in regular expressions: Regular Expression Syntax. + (line 486) +* \s; in regular expressions: Regular Expression Syntax. + (line 437) +* \S; in regular expressions: Regular Expression Syntax. + (line 452) +* \t; escape sequence: String and Bytes literals. + (line 72) +* \t; in regular expressions: Regular Expression Syntax. + (line 486) +* \u; escape sequence: String and Bytes literals. + (line 72) +* \U; escape sequence: String and Bytes literals. + (line 72) +* \u; escape sequence <1>: Error Handlers. (line 28) +* \U; escape sequence <1>: Error Handlers. (line 28) +* \u; in regular expressions: Regular Expression Syntax. + (line 486) +* \U; in regular expressions: Regular Expression Syntax. + (line 486) +* \v; escape sequence: String and Bytes literals. + (line 72) +* \v; in regular expressions: Regular Expression Syntax. + (line 486) +* \w; in regular expressions: Regular Expression Syntax. + (line 458) +* \W; in regular expressions: Regular Expression Syntax. + (line 474) +* \x; escape sequence: String and Bytes literals. + (line 72) +* \x; escape sequence <1>: Error Handlers. (line 28) +* \x; in regular expressions: Regular Expression Syntax. + (line 486) +* \Z; in regular expressions: Regular Expression Syntax. + (line 482) +* \\; escape sequence: String and Bytes literals. + (line 72) +* \\; in regular expressions: Regular Expression Syntax. + (line 486) +* ^ (caret); in curses module: curses ascii — Utilities for ASCII characters. + (line 226) +* ^ (caret); in regular expressions: Regular Expression Syntax. + (line 52) +* ^ (caret); in regular expressions <1>: Regular Expression Syntax. + (line 160) +* ^ (caret); in string formatting: Format Specification Mini-Language. + (line 42) +* ^ (caret); marker: What About Exceptions?. + (line 93) +* ^ (caret); marker <1>: traceback — Print or retrieve a stack traceback. + (line 46) +* ^=; augmented assignment: Augmented assignment statements. + (line 6) +* _ (underscore); gettext: GNU gettext API. (line 42) +* _ (underscore); in numeric literal: Numeric literals. (line 13) +* _ (underscore); in numeric literal <1>: Integer literals. (line 39) +* _ (underscore); in string formatting: Format Specification Mini-Language. + (line 105) +* _, identifiers: Keywords. (line 18) +* _anonymous_ (ctypes.Structure attribute): Structured data types. + (line 74) +* _asdict() (collections.somenamedtuple method): namedtuple Factory Function for Tuples with Named Fields. + (line 107) +* _b_base_ (ctypes._CData attribute): Data types. (line 71) +* _b_needsfree_ (ctypes._CData attribute): Data types. (line 78) +* _callmethod() (multiprocessing.managers.BaseProxy method): Proxy Objects. + (line 96) +* _CData (class in ctypes): Data types. (line 6) +* _clear_type_cache() (in module sys): sys — System-specific parameters and functions. + (line 158) +* _current_frames() (in module sys): sys — System-specific parameters and functions. + (line 167) +* _debugmallocstats() (in module sys): sys — System-specific parameters and functions. + (line 224) +* _dummy_thread (module): _dummy_thread — Drop-in replacement for the _thread module. + (line 6) +* _enablelegacywindowsfsencoding() (in module sys): sys — System-specific parameters and functions. + (line 1449) +* _exit() (in module os): Process Management. (line 120) +* _fields (ast.AST attribute): Node classes. (line 22) +* _fields (collections.somenamedtuple attribute): namedtuple Factory Function for Tuples with Named Fields. + (line 137) +* _fields_ (ctypes.Structure attribute): Structured data types. + (line 31) +* _field_defaults (collections.somenamedtuple attribute): namedtuple Factory Function for Tuples with Named Fields. + (line 150) +* _flush() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 116) +* _frozen (C type): Importing Modules<2>. + (line 260) +* _FuncPtr (class in ctypes): Foreign functions. (line 12) +* _getframe() (in module sys): sys — System-specific parameters and functions. + (line 724) +* _getvalue() (multiprocessing.managers.BaseProxy method): Proxy Objects. + (line 138) +* _get_child_mock() (unittest.mock.Mock method): The Mock Class. + (line 286) +* _handle (ctypes.PyDLL attribute): Loading shared libraries. + (line 145) +* _inittab (C type): Importing Modules<2>. + (line 292) +* _length_ (ctypes.Array attribute): Arrays and pointers. + (line 18) +* _make() (collections.somenamedtuple class method): namedtuple Factory Function for Tuples with Named Fields. + (line 98) +* _makeResult() (unittest.TextTestRunner method): Loading and running tests. + (line 466) +* _name (ctypes.PyDLL attribute): Loading shared libraries. + (line 149) +* _objects (ctypes._CData attribute): Data types. (line 83) +* _pack_ (ctypes.Structure attribute): Structured data types. + (line 67) +* _parse() (gettext.NullTranslations method): The NullTranslations class. + (line 21) +* _Pointer (class in ctypes): Arrays and pointers. + (line 31) +* _PyBytes_Resize (C function): Bytes Objects<2>. (line 182) +* _PyCFunctionFast (C type): Common Object Structures. + (line 109) +* _PyCFunctionFastWithKeywords (C type): Common Object Structures. + (line 114) +* _PyImport_Fini (C function): Importing Modules<2>. + (line 238) +* _PyImport_Init (C function): Importing Modules<2>. + (line 230) +* _PyObject_FastCallDict (C function): Object Protocol. (line 403) +* _PyObject_New (C function): Allocating Objects on the Heap. + (line 6) +* _PyObject_NewVar (C function): Allocating Objects on the Heap. + (line 9) +* _PyObject_Vectorcall (C function): Object Protocol. (line 348) +* _PyTuple_Resize (C function): Tuple Objects. (line 88) +* _Py_c_diff (C function): Complex Numbers as C Structures. + (line 27) +* _Py_c_neg (C function): Complex Numbers as C Structures. + (line 33) +* _Py_c_pow (C function): Complex Numbers as C Structures. + (line 53) +* _Py_c_prod (C function): Complex Numbers as C Structures. + (line 38) +* _Py_c_quot (C function): Complex Numbers as C Structures. + (line 44) +* _Py_c_sum (C function): Complex Numbers as C Structures. + (line 22) +* _Py_InitializeMain (C function): Multi-Phase Initialization Private Provisional API. + (line 44) +* _Py_NoneStruct (C variable): Allocating Objects on the Heap. + (line 59) +* _Py_TPFLAGS_HAVE_VECTORCALL (built-in variable): PyTypeObject Slots. + (line 628) +* _replace() (collections.somenamedtuple method): namedtuple Factory Function for Tuples with Named Fields. + (line 125) +* _setroot() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 15) +* _SimpleCData (class in ctypes): Fundamental data types<2>. + (line 6) +* _structure() (in module email.iterators): email iterators Iterators. + (line 44) +* _thread (module): _thread — Low-level threading API. + (line 6) +* _type_ (ctypes.Array attribute): Arrays and pointers. + (line 24) +* _type_ (ctypes._Pointer attribute): Arrays and pointers. + (line 46) +* _write() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 108) +* _xoptions (in module sys): sys — System-specific parameters and functions. + (line 1679) +* __, identifiers: Keywords. (line 17) +* __abs__() (in module operator): operator — Standard operators as functions. + (line 73) +* __abs__() (object method): Emulating numeric types. + (line 111) +* __add__() (in module operator): operator — Standard operators as functions. + (line 78) +* __add__() (object method): Emulating numeric types. + (line 11) +* __aenter__() (object method): Asynchronous Context Managers. + (line 12) +* __aexit__() (object method): Asynchronous Context Managers. + (line 17) +* __aiter__() (object method): Asynchronous Iterators. + (line 12) +* __all__: Importing * From a Package. + (line 6) +* __all__ (optional module attribute): The import statement. + (line 83) +* __all__ (package variable): Importing Modules<2>. + (line 7) +* __and__() (in module operator): operator — Standard operators as functions. + (line 83) +* __and__() (object method): Emulating numeric types. + (line 11) +* __anext__() (agen method): Asynchronous generator-iterator methods. + (line 10) +* __anext__() (object method): Asynchronous Iterators. + (line 16) +* __annotations__ (class attribute): The standard type hierarchy. + (line 588) +* __annotations__ (function attribute): The standard type hierarchy. + (line 304) +* __annotations__ (module attribute): The standard type hierarchy. + (line 536) +* __await__() (object method): Awaitable Objects. (line 15) +* __bases__ (class attribute): The standard type hierarchy. + (line 588) +* __bases__ (class attribute) <1>: Special Attributes. (line 19) +* __bool__() (object method): Basic customization. + (line 302) +* __bool__() (object method) <1>: Emulating container types. + (line 39) +* __breakpointhook__ (in module sys): sys — System-specific parameters and functions. + (line 336) +* __bytes__() (email.message.EmailMessage method): email message Representing an email message. + (line 129) +* __bytes__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 138) +* __bytes__() (object method): Basic customization. + (line 144) +* __cached__: Import-related module attributes. + (line 69) +* __callback__ (weakref.ref attribute): weakref — Weak references. + (line 121) +* __call__() (email.headerregistry.HeaderRegistry method): email headerregistry Custom Header Objects. + (line 389) +* __call__() (object method): Emulating callable objects. + (line 6) +* __call__() (object method) <1>: Calls. (line 147) +* __call__() (weakref.finalize method): weakref — Weak references. + (line 253) +* __cause__ (exception attribute): The raise statement. + (line 30) +* __cause__ (traceback.TracebackException attribute): TracebackException Objects. + (line 21) +* __ceil__() (fractions.Fraction method): fractions — Rational numbers. + (line 158) +* __ceil__() (object method): Emulating numeric types. + (line 141) +* __classcell__ (class namespace entry): Creating the class object. + (line 6) +* __class_getitem__() (object class method): Emulating generic types. + (line 9) +* __class__ (instance attribute): The standard type hierarchy. + (line 625) +* __class__ (instance attribute) <1>: Special Attributes. (line 15) +* __class__ (method cell): Creating the class object. + (line 6) +* __class__ (module attribute): Customizing module attribute access. + (line 6) +* __class__ (unittest.mock.Mock attribute): The Mock Class. (line 537) +* __closure__ (function attribute): The standard type hierarchy. + (line 304) +* __code__ (function attribute): The standard type hierarchy. + (line 304) +* __code__ (function object attribute): Code Objects. (line 6) +* __complex__() (object method): Emulating numeric types. + (line 119) +* __concat__() (in module operator): operator — Standard operators as functions. + (line 172) +* __contains__() (email.message.EmailMessage method): email message Representing an email message. + (line 179) +* __contains__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 300) +* __contains__() (in module operator): operator — Standard operators as functions. + (line 177) +* __contains__() (mailbox.Mailbox method): Mailbox objects. (line 195) +* __contains__() (object method): Emulating container types. + (line 148) +* __context__ (exception attribute): The raise statement. + (line 30) +* __context__ (traceback.TracebackException attribute): TracebackException Objects. + (line 25) +* __copy__() (copy protocol): copy — Shallow and deep copy operations. + (line 73) +* __debug__: The assert statement. + (line 21) +* __debug__ (built-in variable): Built-in Constants. (line 58) +* __deepcopy__() (copy protocol): copy — Shallow and deep copy operations. + (line 73) +* __defaults__ (function attribute): The standard type hierarchy. + (line 304) +* __delattr__() (object method): Customizing attribute access. + (line 68) +* __delete__() (object method): Implementing Descriptors. + (line 40) +* __delitem__() (email.message.EmailMessage method): email message Representing an email message. + (line 226) +* __delitem__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 334) +* __delitem__() (in module operator): operator — Standard operators as functions. + (line 187) +* __delitem__() (mailbox.Mailbox method): Mailbox objects. (line 76) +* __delitem__() (mailbox.MH method): MH. (line 76) +* __delitem__() (object method): Emulating container types. + (line 102) +* __del__() (io.IOBase method): I/O Base Classes. (line 164) +* __del__() (object method): Basic customization. + (line 54) +* __dict__ (class attribute): The standard type hierarchy. + (line 588) +* __dict__ (function attribute): The standard type hierarchy. + (line 304) +* __dict__ (instance attribute): The standard type hierarchy. + (line 625) +* __dict__ (module attribute): The standard type hierarchy. + (line 547) +* __dict__ (module attribute) <1>: Module Objects. (line 42) +* __dict__ (object attribute): Special Attributes. (line 10) +* __dir__ (module attribute): Customizing module attribute access. + (line 6) +* __dir__() (object method): Customizing attribute access. + (line 77) +* __dir__() (unittest.mock.Mock method): The Mock Class. (line 277) +* __displayhook__ (in module sys): sys — System-specific parameters and functions. + (line 336) +* __divmod__() (object method): Emulating numeric types. + (line 11) +* __doc__ (class attribute): The standard type hierarchy. + (line 588) +* __doc__ (function attribute): The standard type hierarchy. + (line 304) +* __doc__ (method attribute): The standard type hierarchy. + (line 391) +* __doc__ (module attribute): The standard type hierarchy. + (line 536) +* __doc__ (module attribute) <1>: Module Objects. (line 23) +* __doc__ (types.ModuleType attribute): Standard Interpreter Types. + (line 126) +* __enter__() (contextmanager method): Context Manager Types. + (line 12) +* __enter__() (object method): With Statement Context Managers. + (line 21) +* __enter__() (winreg.PyHKEY method): Registry Handle Objects. + (line 56) +* __eq__() (email.charset.Charset method): email charset Representing character sets. + (line 150) +* __eq__() (email.header.Header method): email header Internationalized headers. + (line 170) +* __eq__() (in module operator): operator — Standard operators as functions. + (line 24) +* __eq__() (instance method): Comparisons<2>. (line 50) +* __eq__() (memoryview method): Memory Views. (line 108) +* __eq__() (object method): Basic customization. + (line 173) +* __excepthook__ (in module sys): sys — System-specific parameters and functions. + (line 336) +* __exit__() (contextmanager method): Context Manager Types. + (line 32) +* __exit__() (object method): With Statement Context Managers. + (line 27) +* __exit__() (winreg.PyHKEY method): Registry Handle Objects. + (line 56) +* __file__: Import-related module attributes. + (line 67) +* __file__ (module attribute): The standard type hierarchy. + (line 536) +* __file__ (module attribute) <1>: Module Objects. (line 23) +* __file__ (module attribute) <2>: Module Objects. (line 81) +* __float__() (object method): Emulating numeric types. + (line 119) +* __floordiv__() (in module operator): operator — Standard operators as functions. + (line 88) +* __floordiv__() (object method): Emulating numeric types. + (line 11) +* __floor__() (fractions.Fraction method): fractions — Rational numbers. + (line 148) +* __floor__() (object method): Emulating numeric types. + (line 141) +* __format__: Built-in Functions. (line 643) +* __format__() (datetime.date method): date Objects. (line 274) +* __format__() (datetime.datetime method): datetime Objects. (line 706) +* __format__() (datetime.time method): time Objects. (line 215) +* __format__() (object method): Basic customization. + (line 150) +* __fspath__() (os.PathLike method): Process Parameters. (line 120) +* __func__ (method attribute): The standard type hierarchy. + (line 391) +* __future__: Glossary. (line 484) +* __future__ (module): __future__ — Future statement definitions. + (line 6) +* __future__; future statement: Future statements. (line 6) +* __getattribute__() (object method): Customizing attribute access. + (line 32) +* __getattr__ (module attribute): Customizing module attribute access. + (line 6) +* __getattr__() (object method): Customizing attribute access. + (line 10) +* __getitem__() (email.headerregistry.HeaderRegistry method): email headerregistry Custom Header Objects. + (line 384) +* __getitem__() (email.message.EmailMessage method): email message Representing an email message. + (line 189) +* __getitem__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 309) +* __getitem__() (in module operator): operator — Standard operators as functions. + (line 192) +* __getitem__() (mailbox.Mailbox method): Mailbox objects. (line 140) +* __getitem__() (mapping object method): Special method names. + (line 6) +* __getitem__() (object method): Emulating container types. + (line 76) +* __getitem__() (re.Match method): Match Objects. (line 83) +* __getnewargs_ex__() (object method): Pickling Class Instances. + (line 29) +* __getnewargs__() (object method): Pickling Class Instances. + (line 47) +* __getstate__() (copy protocol): Handling Stateful Objects. + (line 6) +* __getstate__() (object method): Pickling Class Instances. + (line 61) +* __get__() (object method): Implementing Descriptors. + (line 13) +* __ge__() (in module operator): operator — Standard operators as functions. + (line 24) +* __ge__() (instance method): Comparisons<2>. (line 50) +* __ge__() (object method): Basic customization. + (line 173) +* __globals__ (function attribute): The standard type hierarchy. + (line 304) +* __gt__() (in module operator): operator — Standard operators as functions. + (line 24) +* __gt__() (instance method): Comparisons<2>. (line 50) +* __gt__() (object method): Basic customization. + (line 173) +* __hash__() (object method): Basic customization. + (line 221) +* __iadd__() (in module operator): In-place Operators. (line 36) +* __iadd__() (object method): Emulating numeric types. + (line 77) +* __iand__() (in module operator): In-place Operators. (line 41) +* __iand__() (object method): Emulating numeric types. + (line 77) +* __iconcat__() (in module operator): In-place Operators. (line 46) +* __ifloordiv__() (in module operator): In-place Operators. (line 52) +* __ifloordiv__() (object method): Emulating numeric types. + (line 77) +* __ilshift__() (in module operator): In-place Operators. (line 57) +* __ilshift__() (object method): Emulating numeric types. + (line 77) +* __imatmul__() (in module operator): In-place Operators. (line 72) +* __imatmul__() (object method): Emulating numeric types. + (line 77) +* __imod__() (in module operator): In-place Operators. (line 62) +* __imod__() (object method): Emulating numeric types. + (line 77) +* __import__() (built-in function): Built-in Functions. (line 1772) +* __import__() (in module importlib): Functions<10>. (line 6) +* __imul__() (in module operator): In-place Operators. (line 67) +* __imul__() (object method): Emulating numeric types. + (line 77) +* __index__() (in module operator): operator — Standard operators as functions. + (line 93) +* __index__() (object method): Emulating numeric types. + (line 127) +* __init_subclass__() (object class method): Customizing class creation. + (line 13) +* __init__() (difflib.HtmlDiff method): difflib — Helpers for computing deltas. + (line 91) +* __init__() (logging.Handler method): Handler Objects. (line 13) +* __init__() (logging.logging.Formatter method): Formatters. (line 13) +* __init__() (object method): Basic customization. + (line 38) +* __instancecheck__() (class method): Customizing instance and subclass checks. + (line 15) +* __interactivehook__ (in module sys): sys — System-specific parameters and functions. + (line 953) +* __int__() (object method): Emulating numeric types. + (line 119) +* __invert__() (in module operator): operator — Standard operators as functions. + (line 98) +* __invert__() (object method): Emulating numeric types. + (line 111) +* __inv__() (in module operator): operator — Standard operators as functions. + (line 98) +* __ior__() (in module operator): In-place Operators. (line 79) +* __ior__() (object method): Emulating numeric types. + (line 77) +* __ipow__() (in module operator): In-place Operators. (line 84) +* __ipow__() (object method): Emulating numeric types. + (line 77) +* __irshift__() (in module operator): In-place Operators. (line 89) +* __irshift__() (object method): Emulating numeric types. + (line 77) +* __isub__() (in module operator): In-place Operators. (line 94) +* __isub__() (object method): Emulating numeric types. + (line 77) +* __iter__() (container method): Iterator Types. (line 14) +* __iter__() (iterator method): Iterator Types. (line 29) +* __iter__() (mailbox.Mailbox method): Mailbox objects. (line 114) +* __iter__() (object method): Emulating container types. + (line 116) +* __iter__() (unittest.TestSuite method): Grouping tests. (line 60) +* __itruediv__() (in module operator): In-place Operators. (line 99) +* __itruediv__() (object method): Emulating numeric types. + (line 77) +* __ixor__() (in module operator): In-place Operators. (line 104) +* __ixor__() (object method): Emulating numeric types. + (line 77) +* __kwdefaults__ (function attribute): The standard type hierarchy. + (line 304) +* __length_hint__() (object method): Emulating container types. + (line 52) +* __len__() (email.message.EmailMessage method): email message Representing an email message. + (line 175) +* __len__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 296) +* __len__() (mailbox.Mailbox method): Mailbox objects. (line 200) +* __len__() (mapping object method): Basic customization. + (line 304) +* __len__() (object method): Emulating container types. + (line 37) +* __le__() (in module operator): operator — Standard operators as functions. + (line 24) +* __le__() (instance method): Comparisons<2>. (line 50) +* __le__() (object method): Basic customization. + (line 173) +* __loader__: Import-related module attributes. + (line 16) +* __loader__ (module attribute): Module Objects. (line 23) +* __loader__ (types.ModuleType attribute): Standard Interpreter Types. + (line 130) +* __lshift__() (in module operator): operator — Standard operators as functions. + (line 106) +* __lshift__() (object method): Emulating numeric types. + (line 11) +* __lt__() (in module operator): operator — Standard operators as functions. + (line 24) +* __lt__() (instance method): Comparisons<2>. (line 50) +* __lt__() (object method): Basic customization. + (line 173) +* __main__ (module): __main__ — Top-level script environment. + (line 6) +* __matmul__() (in module operator): operator — Standard operators as functions. + (line 121) +* __matmul__() (object method): Emulating numeric types. + (line 11) +* __missing__(): Mapping Types — dict. + (line 96) +* __missing__() (collections.defaultdict method): defaultdict objects. + (line 22) +* __missing__() (object method): Emulating container types. + (line 111) +* __module__ (class attribute): The standard type hierarchy. + (line 588) +* __module__ (function attribute): The standard type hierarchy. + (line 304) +* __module__ (method attribute): The standard type hierarchy. + (line 391) +* __mod__() (in module operator): operator — Standard operators as functions. + (line 111) +* __mod__() (object method): Emulating numeric types. + (line 11) +* __mro__ (class attribute): Special Attributes. (line 35) +* __mul__() (in module operator): operator — Standard operators as functions. + (line 116) +* __mul__() (object method): Emulating numeric types. + (line 11) +* __name__: Import-related module attributes. + (line 10) +* __name__ (class attribute): The standard type hierarchy. + (line 588) +* __name__ (definition attribute): Special Attributes. (line 23) +* __name__ (function attribute): The standard type hierarchy. + (line 304) +* __name__ (method attribute): The standard type hierarchy. + (line 391) +* __name__ (module attribute): The standard type hierarchy. + (line 536) +* __name__ (module attribute) <1>: Module Objects. (line 23) +* __name__ (module attribute) <2>: Module Objects. (line 55) +* __name__ (types.ModuleType attribute): Standard Interpreter Types. + (line 148) +* __neg__() (in module operator): operator — Standard operators as functions. + (line 128) +* __neg__() (object method): Emulating numeric types. + (line 111) +* __new__() (object method): Basic customization. + (line 6) +* __next__() (csv.csvreader method): Reader Objects. (line 9) +* __next__() (generator method): Generator-iterator methods. + (line 12) +* __next__() (iterator method): Iterator Types. (line 37) +* __ne__() (email.charset.Charset method): email charset Representing character sets. + (line 155) +* __ne__() (email.header.Header method): email header Internationalized headers. + (line 175) +* __ne__() (in module operator): operator — Standard operators as functions. + (line 24) +* __ne__() (instance method): Comparisons<2>. (line 50) +* __ne__() (object method): Basic customization. + (line 173) +* __not__() (in module operator): operator — Standard operators as functions. + (line 49) +* __or__() (in module operator): operator — Standard operators as functions. + (line 133) +* __or__() (object method): Emulating numeric types. + (line 11) +* __package__: Import-related module attributes. + (line 23) +* __package__ (module attribute): Module Objects. (line 23) +* __package__ (types.ModuleType attribute): Standard Interpreter Types. + (line 153) +* __path__: Import-related module attributes. + (line 56) +* __pos__() (in module operator): operator — Standard operators as functions. + (line 138) +* __pos__() (object method): Emulating numeric types. + (line 111) +* __pow__() (in module operator): operator — Standard operators as functions. + (line 143) +* __pow__() (object method): Emulating numeric types. + (line 11) +* __prepare__ (metaclass method): Preparing the class namespace. + (line 6) +* __qualname__ (definition attribute): Special Attributes. (line 28) +* __radd__() (object method): Emulating numeric types. + (line 41) +* __rand__() (object method): Emulating numeric types. + (line 41) +* __rdivmod__() (object method): Emulating numeric types. + (line 41) +* __reduce_ex__() (object method): Pickling Class Instances. + (line 161) +* __reduce__() (object method): Pickling Class Instances. + (line 106) +* __repr__() (multiprocessing.managers.BaseProxy method): Proxy Objects. + (line 145) +* __repr__() (netrc.netrc method): netrc Objects. (line 16) +* __repr__() (object method): Basic customization. + (line 113) +* __reversed__() (object method): Emulating container types. + (line 127) +* __rfloordiv__() (object method): Emulating numeric types. + (line 41) +* __rlshift__() (object method): Emulating numeric types. + (line 41) +* __rmatmul__() (object method): Emulating numeric types. + (line 41) +* __rmod__() (object method): Emulating numeric types. + (line 41) +* __rmul__() (object method): Emulating numeric types. + (line 41) +* __ror__() (object method): Emulating numeric types. + (line 41) +* __round__() (fractions.Fraction method): fractions — Rational numbers. + (line 163) +* __round__() (object method): Emulating numeric types. + (line 141) +* __rpow__() (object method): Emulating numeric types. + (line 41) +* __rrshift__() (object method): Emulating numeric types. + (line 41) +* __rshift__() (in module operator): operator — Standard operators as functions. + (line 148) +* __rshift__() (object method): Emulating numeric types. + (line 11) +* __rsub__() (object method): Emulating numeric types. + (line 41) +* __rtruediv__() (object method): Emulating numeric types. + (line 41) +* __rxor__() (object method): Emulating numeric types. + (line 41) +* __self__ (method attribute): The standard type hierarchy. + (line 391) +* __setattr__() (object method): Customizing attribute access. + (line 53) +* __setitem__() (email.message.EmailMessage method): email message Representing an email message. + (line 204) +* __setitem__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 320) +* __setitem__() (in module operator): operator — Standard operators as functions. + (line 201) +* __setitem__() (mailbox.Mailbox method): Mailbox objects. (line 90) +* __setitem__() (mailbox.Maildir method): Maildir. (line 91) +* __setitem__() (object method): Emulating container types. + (line 93) +* __setstate__() (copy protocol): Handling Stateful Objects. + (line 6) +* __setstate__() (object method): Pickling Class Instances. + (line 70) +* __set_name__() (object method): Implementing Descriptors. + (line 45) +* __set__() (object method): Implementing Descriptors. + (line 31) +* __slots__: Glossary. (line 1142) +* __spec__: Import-related module attributes. + (line 40) +* __spec__ (types.ModuleType attribute): Standard Interpreter Types. + (line 174) +* __stderr__ (in module sys): sys — System-specific parameters and functions. + (line 1526) +* __stdin__ (in module sys): sys — System-specific parameters and functions. + (line 1526) +* __stdout__ (in module sys): sys — System-specific parameters and functions. + (line 1526) +* __str__() (datetime.date method): date Objects. (line 247) +* __str__() (datetime.datetime method): datetime Objects. (line 676) +* __str__() (datetime.time method): time Objects. (line 205) +* __str__() (email.charset.Charset method): email charset Representing character sets. + (line 145) +* __str__() (email.header.Header method): email header Internationalized headers. + (line 158) +* __str__() (email.headerregistry.Address method): email headerregistry Custom Header Objects. + (line 447) +* __str__() (email.headerregistry.Group method): email headerregistry Custom Header Objects. + (line 482) +* __str__() (email.message.EmailMessage method): email message Representing an email message. + (line 95) +* __str__() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 101) +* __str__() (multiprocessing.managers.BaseProxy method): Proxy Objects. + (line 149) +* __str__() (object method): Basic customization. + (line 129) +* __subclasscheck__() (class method): Customizing instance and subclass checks. + (line 21) +* __subclasses__() (class method): Special Attributes. (line 46) +* __subclasshook__() (abc.ABCMeta method): abc — Abstract Base Classes. + (line 92) +* __sub__() (in module operator): operator — Standard operators as functions. + (line 153) +* __sub__() (object method): Emulating numeric types. + (line 11) +* __suppress_context__ (traceback.TracebackException attribute): TracebackException Objects. + (line 30) +* __traceback__ (exception attribute): The raise statement. + (line 6) +* __truediv__() (in module operator): operator — Standard operators as functions. + (line 158) +* __truediv__() (object method): Emulating numeric types. + (line 11) +* __trunc__() (object method): Emulating numeric types. + (line 141) +* __unraisablehook__ (in module sys): sys — System-specific parameters and functions. + (line 336) +* __xor__() (in module operator): operator — Standard operators as functions. + (line 164) +* __xor__() (object method): Emulating numeric types. + (line 11) +* {} (curly brackets); dictionary expression: Dictionary displays. + (line 6) +* {} (curly brackets); in formatted string literal: String literal concatenation. + (line 24) +* {} (curly brackets); in regular expressions: Regular Expression Syntax. + (line 95) +* {} (curly brackets); in string formatting: Format String Syntax. + (line 13) +* {} (curly brackets); set expression: Set displays. (line 6) +* | (vertical bar); in regular expressions: Regular Expression Syntax. + (line 186) +* |=; augmented assignment: Augmented assignment statements. + (line 6) +* ~ (tilde); home directory expansion: os path — Common pathname manipulations. + (line 135) +* A (in module re): Module Contents. (line 44) +* a-LAW: audioop — Manipulate raw audio data. + (line 17) +* A-LAW: aifc — Read and write AIFF and AIFC files. + (line 160) +* A-LAW <1>: sndhdr — Determine type of sound file. + (line 8) +* a2b_base64() (in module binascii): binascii — Convert between binary and ASCII. + (line 41) +* a2b_hex() (in module binascii): binascii — Convert between binary and ASCII. + (line 161) +* a2b_hqx() (in module binascii): binascii — Convert between binary and ASCII. + (line 76) +* a2b_qp() (in module binascii): binascii — Convert between binary and ASCII. + (line 55) +* a2b_uu() (in module binascii): binascii — Convert between binary and ASCII. + (line 26) +* a85decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 163) +* a85encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 140) +* ABC (class in abc): abc — Abstract Base Classes. + (line 26) +* abc (module): abc — Abstract Base Classes. + (line 6) +* ABCMeta (class in abc): abc — Abstract Base Classes. + (line 52) +* abiflags (in module sys): sys — System-specific parameters and functions. + (line 12) +* abort(): Process Control. (line 8) +* abort() (asyncio.DatagramTransport method): Datagram Transports. + (line 15) +* abort() (asyncio.WriteTransport method): Write-only Transports. + (line 6) +* abort() (ftplib.FTP method): FTP Objects. (line 61) +* abort() (in module os): Process Management. (line 16) +* abort() (threading.Barrier method): Barrier Objects. (line 82) +* above() (curses.panel.Panel method): Panel Objects. (line 13) +* ABOVE_NORMAL_PRIORITY_CLASS (in module subprocess): Windows Constants. + (line 52) +* abs() (built-in function): Built-in Functions. (line 55) +* abs() (decimal.Context method): Context objects. (line 230) +* abs() (in module operator): operator — Standard operators as functions. + (line 73) +* abspath() (in module os.path): os path — Common pathname manipulations. + (line 54) +* abstract base class: Glossary. (line 32) +* AbstractAsyncContextManager (class in contextlib): Utilities. + (line 19) +* AbstractBasicAuthHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 332) +* AbstractChildWatcher (class in asyncio): Process Watchers. (line 39) +* abstractclassmethod() (in module abc): abc — Abstract Base Classes. + (line 237) +* AbstractContextManager (class in contextlib): Utilities. (line 8) +* AbstractDigestAuthHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 371) +* AbstractEventLoop (class in asyncio): Event Loop Implementations. + (line 40) +* AbstractEventLoopPolicy (class in asyncio): Policy Objects. + (line 8) +* AbstractFormatter (class in formatter): Formatter Implementations. + (line 18) +* abstractmethod() (in module abc): abc — Abstract Base Classes. + (line 161) +* abstractproperty() (in module abc): abc — Abstract Base Classes. + (line 281) +* AbstractSet (class in typing): Classes functions and decorators. + (line 209) +* abstractstaticmethod() (in module abc): abc — Abstract Base Classes. + (line 259) +* AbstractWriter (class in formatter): Writer Implementations. + (line 16) +* accept() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 231) +* accept() (multiprocessing.connection.Listener method): Listeners and Clients. + (line 84) +* accept() (socket.socket method): Socket Objects. (line 13) +* access() (in module os): Files and Directories. + (line 51) +* accumulate() (in module itertools): Itertool functions. (line 10) +* aclose() (agen method): Asynchronous generator-iterator methods. + (line 55) +* aclose() (contextlib.AsyncExitStack method): Utilities. (line 477) +* acos() (in module cmath): Trigonometric functions<2>. + (line 6) +* acos() (in module math): Trigonometric functions. + (line 6) +* acosh() (in module cmath): Hyperbolic functions<2>. + (line 6) +* acosh() (in module math): Hyperbolic functions. + (line 9) +* acquire() (asyncio.Condition method): Condition. (line 48) +* acquire() (asyncio.Lock method): Lock. (line 36) +* acquire() (asyncio.Semaphore method): Semaphore. (line 43) +* acquire() (logging.Handler method): Handler Objects. (line 26) +* acquire() (multiprocessing.Lock method): Synchronization primitives. + (line 64) +* acquire() (multiprocessing.RLock method): Synchronization primitives. + (line 118) +* acquire() (threading.Condition method): Condition Objects. (line 85) +* acquire() (threading.Lock method): Lock Objects. (line 43) +* acquire() (threading.RLock method): RLock Objects. (line 35) +* acquire() (threading.Semaphore method): Semaphore Objects. (line 34) +* acquire() (_thread.lock method): _thread — Low-level threading API. + (line 126) +* acquire_lock() (in module imp): imp — Access the import internals. + (line 277) +* Action (class in argparse): Action classes. (line 11) +* action (optparse.Option attribute): Option attributes. (line 11) +* ACTIONS (optparse.Option attribute): Adding new actions. (line 31) +* active_children() (in module multiprocessing): Miscellaneous<3>. + (line 6) +* active_count() (in module threading): threading — Thread-based parallelism. + (line 33) +* add() (decimal.Context method): Context objects. (line 234) +* add() (frozenset method): Set Types — set frozenset. + (line 195) +* add() (in module audioop): audioop — Manipulate raw audio data. + (line 31) +* add() (in module operator): operator — Standard operators as functions. + (line 78) +* add() (mailbox.Mailbox method): Mailbox objects. (line 59) +* add() (mailbox.Maildir method): Maildir. (line 91) +* add() (msilib.RadioButtonGroup method): GUI classes. (line 34) +* add() (pstats.Stats method): The Stats Class. (line 49) +* add() (tarfile.TarFile method): TarFile Objects. (line 200) +* add() (tkinter.ttk.Notebook method): ttk Notebook. (line 8) +* addAsyncCleanup() (unittest.IsolatedAsyncioTestCase method): Test cases. + (line 840) +* addaudithook() (in module sys): sys — System-specific parameters and functions. + (line 23) +* addch() (curses.window method): Window Objects. (line 9) +* addClassCleanup() (unittest.TestCase class method): Test cases. + (line 776) +* addCleanup() (unittest.TestCase method): Test cases. (line 746) +* addcomponent() (turtle.Shape method): Public classes. (line 66) +* addError() (unittest.TestResult method): Loading and running tests. + (line 352) +* addExpectedFailure() (unittest.TestResult method): Loading and running tests. + (line 388) +* addFailure() (unittest.TestResult method): Loading and running tests. + (line 363) +* addfile() (tarfile.TarFile method): TarFile Objects. (line 218) +* addFilter() (logging.Handler method): Handler Objects. (line 52) +* addFilter() (logging.Logger method): Logger Objects. (line 251) +* addHandler() (logging.Logger method): Logger Objects. (line 268) +* addition: Binary arithmetic operations. + (line 60) +* addLevelName() (in module logging): Module-Level Functions. + (line 190) +* addModuleCleanup() (in module unittest): setUpModule and tearDownModule. + (line 22) +* addnstr() (curses.window method): Window Objects. (line 23) +* AddPackagePath() (in module modulefinder): modulefinder — Find modules used by a script. + (line 16) +* addr (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 52) +* Address (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 402) +* address (email.headerregistry.SingleAddressHeader attribute): email headerregistry Custom Header Objects. + (line 222) +* address (multiprocessing.connection.Listener attribute): Listeners and Clients. + (line 99) +* address (multiprocessing.managers.BaseManager attribute): Managers. + (line 120) +* addresses (email.headerregistry.AddressHeader attribute): email headerregistry Custom Header Objects. + (line 195) +* addresses (email.headerregistry.Group attribute): email headerregistry Custom Header Objects. + (line 477) +* AddressHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 180) +* addressof() (in module ctypes): Utility functions. (line 6) +* AddressValueError: Custom Exceptions. (line 9) +* address_exclude() (ipaddress.IPv4Network method): Network objects. + (line 157) +* address_exclude() (ipaddress.IPv6Network method): Network objects. + (line 338) +* address_family (socketserver.BaseServer attribute): Server Objects<2>. + (line 65) +* address_string() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 323) +* addr_spec (email.headerregistry.Address attribute): email headerregistry Custom Header Objects. + (line 441) +* addshape() (in module turtle): Settings and special methods. + (line 71) +* addsitedir() (in module site): Module contents<3>. (line 51) +* addSkip() (unittest.TestResult method): Loading and running tests. + (line 380) +* addstr() (curses.window method): Window Objects. (line 31) +* addSubTest() (unittest.TestResult method): Loading and running tests. + (line 406) +* addSuccess() (unittest.TestResult method): Loading and running tests. + (line 374) +* addTest() (unittest.TestSuite method): Grouping tests. (line 25) +* addTests() (unittest.TestSuite method): Grouping tests. (line 30) +* addTypeEqualityFunc() (unittest.TestCase method): Test cases. + (line 564) +* addUnexpectedSuccess() (unittest.TestResult method): Loading and running tests. + (line 398) +* add_alias() (in module email.charset): email charset Representing character sets. + (line 193) +* add_alternative() (email.message.EmailMessage method): email message Representing an email message. + (line 640) +* add_argument() (argparse.ArgumentParser method): The add_argument method. + (line 6) +* add_argument_group() (argparse.ArgumentParser method): Argument groups. + (line 6) +* add_attachment() (email.message.EmailMessage method): email message Representing an email message. + (line 652) +* add_cgi_vars() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 132) +* add_charset() (in module email.charset): email charset Representing character sets. + (line 164) +* add_child_handler() (asyncio.AbstractChildWatcher method): Process Watchers. + (line 41) +* add_codec() (in module email.charset): email charset Representing character sets. + (line 202) +* add_cookie_header() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 11) +* add_data() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 74) +* add_dll_directory() (in module os): Process Management. (line 24) +* add_done_callback() (asyncio.Future method): Future Object. + (line 77) +* add_done_callback() (asyncio.Task method): Task Object. (line 148) +* add_done_callback() (concurrent.futures.Future method): Future Objects. + (line 70) +* add_fallback() (gettext.NullTranslations method): The NullTranslations class. + (line 28) +* add_file() (msilib.Directory method): Directory Objects. (line 30) +* add_flag() (mailbox.MaildirMessage method): MaildirMessage. + (line 76) +* add_flag() (mailbox.mboxMessage method): mboxMessage. (line 79) +* add_flag() (mailbox.MMDFMessage method): MMDFMessage. (line 78) +* add_flowing_data() (formatter.formatter method): The Formatter Interface. + (line 43) +* add_folder() (mailbox.Maildir method): Maildir. (line 71) +* add_folder() (mailbox.MH method): MH. (line 42) +* add_get_handler() (email.contentmanager.ContentManager method): email contentmanager Managing MIME Content. + (line 75) +* add_handler() (urllib.request.OpenerDirector method): OpenerDirector Objects. + (line 8) +* add_header() (email.message.EmailMessage method): email message Representing an email message. + (line 260) +* add_header() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 368) +* add_header() (urllib.request.Request method): Request Objects. + (line 79) +* add_header() (wsgiref.headers.Headers method): wsgiref headers – WSGI response header tools. + (line 61) +* add_history() (in module readline): History list. (line 38) +* add_hor_rule() (formatter.formatter method): The Formatter Interface. + (line 36) +* add_include_dir() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 96) +* add_label() (mailbox.BabylMessage method): BabylMessage. (line 55) +* add_label_data() (formatter.formatter method): The Formatter Interface. + (line 59) +* add_library() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 113) +* add_library_dir() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 135) +* add_line_break() (formatter.formatter method): The Formatter Interface. + (line 31) +* add_link_object() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 180) +* add_literal_data() (formatter.formatter method): The Formatter Interface. + (line 53) +* add_mutually_exclusive_group() (argparse.ArgumentParser method): Mutual exclusion. + (line 6) +* add_option() (optparse.OptionParser method): Defining options. + (line 14) +* add_parent() (urllib.request.BaseHandler method): BaseHandler Objects. + (line 10) +* add_password() (urllib.request.HTTPPasswordMgr method): HTTPPasswordMgr Objects. + (line 9) +* add_password() (urllib.request.HTTPPasswordMgrWithPriorAuth method): HTTPPasswordMgrWithPriorAuth Objects. + (line 10) +* add_reader() (asyncio.loop method): Watching file descriptors. + (line 6) +* add_related() (email.message.EmailMessage method): email message Representing an email message. + (line 627) +* add_runtime_library_dir() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 150) +* add_section() (configparser.ConfigParser method): ConfigParser Objects. + (line 86) +* add_section() (configparser.RawConfigParser method): RawConfigParser Objects. + (line 26) +* add_sequence() (mailbox.MHMessage method): MHMessage. (line 41) +* add_set_handler() (email.contentmanager.ContentManager method): email contentmanager Managing MIME Content. + (line 80) +* add_signal_handler() (asyncio.loop method): Unix signals. (line 6) +* add_stream() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 104) +* add_subparsers() (argparse.ArgumentParser method): Sub-commands. + (line 6) +* add_tables() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 95) +* add_type() (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 107) +* add_unredirected_header() (urllib.request.Request method): Request Objects. + (line 90) +* add_writer() (asyncio.loop method): Watching file descriptors. + (line 16) +* adjusted() (decimal.Decimal method): Decimal objects. (line 114) +* adler32() (in module zlib): zlib — Compression compatible with gzip. + (line 28) +* ADPCM, Intel/DVI: audioop — Manipulate raw audio data. + (line 17) +* adpcm2lin() (in module audioop): audioop — Manipulate raw audio data. + (line 38) +* AF_ALG (in module socket): Constants<8>. (line 183) +* AF_CAN (in module socket): Constants<8>. (line 93) +* AF_INET (in module socket): Constants<8>. (line 11) +* AF_INET6 (in module socket): Constants<8>. (line 11) +* AF_LINK (in module socket): Constants<8>. (line 207) +* AF_PACKET (in module socket): Constants<8>. (line 144) +* AF_QIPCRTR (in module socket): Constants<8>. (line 235) +* AF_RDS (in module socket): Constants<8>. (line 154) +* AF_UNIX (in module socket): Constants<8>. (line 11) +* AF_VSOCK (in module socket): Constants<8>. (line 194) +* aifc (module): aifc — Read and write AIFF and AIFC files. + (line 6) +* aifc() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 134) +* AIFF: aifc — Read and write AIFF and AIFC files. + (line 8) +* AIFF <1>: chunk — Read IFF chunked data. + (line 8) +* aiff() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 128) +* AIFF-C: aifc — Read and write AIFF and AIFC files. + (line 8) +* AIFF-C <1>: chunk — Read IFF chunked data. + (line 8) +* alarm() (in module signal): Module contents<2>. (line 224) +* alaw2lin() (in module audioop): audioop — Manipulate raw audio data. + (line 45) +* AlertDescription (class in ssl): Constants<9>. (line 486) +* ALERT_DESCRIPTION_HANDSHAKE_FAILURE (in module ssl): Constants<9>. + (line 472) +* ALERT_DESCRIPTION_INTERNAL_ERROR (in module ssl): Constants<9>. + (line 472) +* algorithms_available (in module hashlib): Hash algorithms. (line 81) +* algorithms_guaranteed (in module hashlib): Hash algorithms. + (line 72) +* alias (pdb command): Debugger Commands. (line 283) +* alignment() (in module ctypes): Utility functions. (line 14) +* alive (weakref.finalize attribute): weakref — Weak references. + (line 270) +* all() (built-in function): Built-in Functions. (line 62) +* allocate_lock() (in module _thread): _thread — Low-level threading API. + (line 68) +* allocfunc (C type): Slot Type typedefs. (line 6) +* allowed_domains() (http.cookiejar.DefaultCookiePolicy method): DefaultCookiePolicy Objects. + (line 65) +* allow_reuse_address (socketserver.BaseServer attribute): Server Objects<2>. + (line 92) +* all_errors (in module ftplib): ftplib — FTP protocol client. + (line 135) +* all_features (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 90) +* all_frames (tracemalloc.Filter attribute): Filter. (line 57) +* all_properties (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 122) +* all_suffixes() (in module importlib.machinery): importlib machinery – Importers and path hooks. + (line 57) +* all_tasks() (asyncio.Task class method): Task Object. (line 232) +* all_tasks() (in module asyncio): Introspection. (line 16) +* alt() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 218) +* altsep (in module os): Miscellaneous System Information. + (line 102) +* altzone (in module time): Timezone Constants. (line 6) +* ALT_DIGITS (in module locale): locale — Internationalization services. + (line 296) +* ALWAYS_EQ (in module test.support): test support — Utilities for the Python test suite. + (line 145) +* ALWAYS_TYPED_ACTIONS (optparse.Option attribute): Adding new actions. + (line 43) +* AMPER (in module token): token — Constants used with Python parse trees. + (line 102) +* AMPEREQUAL (in module token): token — Constants used with Python parse trees. + (line 190) +* and_() (in module operator): operator — Standard operators as functions. + (line 83) +* annotated; assignment: Annotated assignment statements. + (line 6) +* annotation: Glossary. (line 47) +* annotation (inspect.Parameter attribute): Introspecting callables with the Signature object. + (line 175) +* announce() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 503) +* anonymous; function: Lambdas. (line 6) +* answer_challenge() (in module multiprocessing.connection): Listeners and Clients. + (line 25) +* anticipate_failure() (in module test.support): test support — Utilities for the Python test suite. + (line 568) +* Any (in module typing): Classes functions and decorators. + (line 773) +* ANY (in module unittest.mock): ANY. (line 6) +* any() (built-in function): Built-in Functions. (line 73) +* AnyStr (in module typing): Classes functions and decorators. + (line 950) +* api_version (in module sys): sys — System-specific parameters and functions. + (line 1643) +* apop() (poplib.POP3 method): POP3 Objects. (line 41) +* APPDATA: PEP 370 Per-user site-packages Directory. + (line 26) +* append() (array.array method): array — Efficient arrays of numeric values. + (line 113) +* append() (collections.deque method): deque objects. (line 35) +* append() (email.header.Header method): email header Internationalized headers. + (line 96) +* append() (imaplib.IMAP4 method): IMAP4 Objects. (line 33) +* append() (msilib.CAB method): CAB Objects. (line 15) +* append() (pipes.Template method): Template Objects. (line 22) +* append() (sequence method): Mutable Sequence Types. + (line 16) +* append() (xml.etree.ElementTree.Element method): Element Objects. + (line 87) +* appendChild() (xml.dom.Node method): Node Objects. (line 115) +* appendleft() (collections.deque method): deque objects. (line 39) +* append_history_file() (in module readline): History file. (line 20) +* application_uri() (in module wsgiref.util): wsgiref util – WSGI environment utilities. + (line 32) +* apply (2to3 fixer): Fixers. (line 10) +* apply() (multiprocessing.pool.Pool method): Process Pools. (line 63) +* apply_async() (multiprocessing.pool.Pool method): Process Pools. + (line 71) +* apply_defaults() (inspect.BoundArguments method): Introspecting callables with the Signature object. + (line 312) +* architecture() (in module platform): Cross Platform. (line 6) +* archive (zipimport.zipimporter attribute): zipimporter Objects. + (line 69) +* aRepr (in module reprlib): reprlib — Alternate repr implementation. + (line 24) +* argparse (module): argparse — Parser for command-line options arguments and sub-commands. + (line 6) +* args (BaseException attribute): Base classes. (line 17) +* args (functools.partial attribute): partial Objects. (line 15) +* args (inspect.BoundArguments attribute): Introspecting callables with the Signature object. + (line 298) +* args (pdb command): Debugger Commands. (line 229) +* args (subprocess.CompletedProcess attribute): Using the subprocess Module. + (line 96) +* args (subprocess.Popen attribute): Popen Objects. (line 97) +* args_from_interpreter_flags() (in module test.support): test support — Utilities for the Python test suite. + (line 392) +* argtypes (ctypes._FuncPtr attribute): Foreign functions. (line 36) +* argument: Glossary. (line 61) +* argument; call semantics: Slicings. (line 38) +* argument; difference from parameter: How can I pass optional or keyword parameters from one function to another?. + (line 17) +* argument; function definition: Function definitions. + (line 53) +* ArgumentDefaultsHelpFormatter (class in argparse): formatter_class. + (line 10) +* ArgumentError: Foreign functions. (line 83) +* ArgumentParser (class in argparse): ArgumentParser objects. + (line 6) +* arguments (inspect.BoundArguments attribute): Introspecting callables with the Signature object. + (line 282) +* argv (in module sys): sys — System-specific parameters and functions. + (line 56) +* argv (in module sys) <1>: Process-wide parameters. + (line 217) +* arithmetic: Numeric Types — int float complex. + (line 27) +* arithmetic; conversion: Arithmetic conversions. + (line 6) +* ArithmeticError: Base classes. (line 43) +* array (class in array): array — Efficient arrays of numeric values. + (line 75) +* Array (class in ctypes): Arrays and pointers. + (line 6) +* array (module): array — Efficient arrays of numeric values. + (line 6) +* Array() (in module multiprocessing): Shared ctypes Objects. + (line 42) +* Array() (in module multiprocessing.sharedctypes): The multiprocessing sharedctypes module. + (line 51) +* Array() (multiprocessing.managers.SyncManager method): Managers. + (line 196) +* arrays: array — Efficient arrays of numeric values. + (line 6) +* arraysize (sqlite3.Cursor attribute): Cursor Objects. (line 214) +* article() (nntplib.NNTP method): Methods<3>. (line 241) +* as; except clause: The try statement. (line 40) +* as; import statement: The import statement. + (line 38) +* as; with statement: The with statement. (line 6) +* ASCII: Notation. (line 31) +* ASCII <1>: Literals. (line 8) +* ASCII (in module re): Module Contents. (line 44) +* ascii() (built-in function): Built-in Functions. (line 84) +* ascii() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 209) +* ascii_letters (in module string): String constants. (line 8) +* ascii_lowercase (in module string): String constants. (line 14) +* ascii_uppercase (in module string): String constants. (line 19) +* asctime() (in module time): Functions<2>. (line 6) +* asdict() (in module dataclasses): Module-level decorators classes and functions. + (line 269) +* asend() (agen method): Asynchronous generator-iterator methods. + (line 28) +* asin() (in module cmath): Trigonometric functions<2>. + (line 13) +* asin() (in module math): Trigonometric functions. + (line 10) +* asinh() (in module cmath): Hyperbolic functions<2>. + (line 12) +* asinh() (in module math): Hyperbolic functions. + (line 13) +* assertAlmostEqual() (unittest.TestCase method): Test cases. + (line 484) +* assertCountEqual() (unittest.TestCase method): Test cases. (line 543) +* assertDictEqual() (unittest.TestCase method): Test cases. (line 652) +* assertEqual() (unittest.TestCase method): Test cases. (line 177) +* assertFalse() (unittest.TestCase method): Test cases. (line 202) +* assertGreater() (unittest.TestCase method): Test cases. (line 509) +* assertGreaterEqual() (unittest.TestCase method): Test cases. + (line 509) +* assertIn() (unittest.TestCase method): Test cases. (line 229) +* AssertionError: Concrete exceptions. + (line 8) +* assertIs() (unittest.TestCase method): Test cases. (line 214) +* assertIsInstance() (unittest.TestCase method): Test cases. (line 236) +* assertIsNone() (unittest.TestCase method): Test cases. (line 222) +* assertIsNot() (unittest.TestCase method): Test cases. (line 214) +* assertIsNotNone() (unittest.TestCase method): Test cases. (line 222) +* assertLess() (unittest.TestCase method): Test cases. (line 509) +* assertLessEqual() (unittest.TestCase method): Test cases. (line 509) +* assertListEqual() (unittest.TestCase method): Test cases. (line 629) +* assertLogs() (unittest.TestCase method): Test cases. (line 405) +* assertMultiLineEqual() (unittest.TestCase method): Test cases. + (line 604) +* assertNotAlmostEqual() (unittest.TestCase method): Test cases. + (line 484) +* assertNotEqual() (unittest.TestCase method): Test cases. (line 197) +* assertNotIn() (unittest.TestCase method): Test cases. (line 229) +* assertNotIsInstance() (unittest.TestCase method): Test cases. + (line 236) +* assertNotRegex() (unittest.TestCase method): Test cases. (line 522) +* assertRaises() (unittest.TestCase method): Test cases. (line 271) +* assertRaisesRegex() (unittest.TestCase method): Test cases. + (line 312) +* assertRegex() (unittest.TestCase method): Test cases. (line 522) +* asserts (2to3 fixer): Fixers. (line 15) +* assertSequenceEqual() (unittest.TestCase method): Test cases. + (line 614) +* assertSetEqual() (unittest.TestCase method): Test cases. (line 640) +* assertTrue() (unittest.TestCase method): Test cases. (line 202) +* assertTupleEqual() (unittest.TestCase method): Test cases. (line 629) +* assertWarns() (unittest.TestCase method): Test cases. (line 339) +* assertWarnsRegex() (unittest.TestCase method): Test cases. (line 380) +* assert_any_await() (unittest.mock.AsyncMock method): The Mock Class. + (line 809) +* assert_any_call() (unittest.mock.Mock method): The Mock Class. + (line 147) +* assert_awaited() (unittest.mock.AsyncMock method): The Mock Class. + (line 739) +* assert_awaited_once() (unittest.mock.AsyncMock method): The Mock Class. + (line 759) +* assert_awaited_once_with() (unittest.mock.AsyncMock method): The Mock Class. + (line 792) +* assert_awaited_with() (unittest.mock.AsyncMock method): The Mock Class. + (line 775) +* assert_called() (unittest.mock.Mock method): The Mock Class. + (line 95) +* assert_called_once() (unittest.mock.Mock method): The Mock Class. + (line 106) +* assert_called_once_with() (unittest.mock.Mock method): The Mock Class. + (line 133) +* assert_called_with() (unittest.mock.Mock method): The Mock Class. + (line 123) +* assert_has_awaits() (unittest.mock.AsyncMock method): The Mock Class. + (line 826) +* assert_has_calls() (unittest.mock.Mock method): The Mock Class. + (line 163) +* assert_line_data() (formatter.formatter method): The Formatter Interface. + (line 143) +* assert_not_awaited() (unittest.mock.AsyncMock method): The Mock Class. + (line 853) +* assert_not_called() (unittest.mock.Mock method): The Mock Class. + (line 184) +* assert_python_failure() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 50) +* assert_python_ok() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 37) +* assignment; statement: The standard type hierarchy. + (line 192) +* assignment; statement <1>: Assignment statements. + (line 6) +* AST (class in ast): Node classes. (line 6) +* ast (module): ast — Abstract Syntax Trees. + (line 6) +* astimezone() (datetime.datetime method): datetime Objects. (line 426) +* astuple() (in module dataclasses): Module-level decorators classes and functions. + (line 294) +* ASYNC (in module token): token — Constants used with Python parse trees. + (line 246) +* async for; in comprehensions: Displays for lists sets and dictionaries. + (line 14) +* AsyncContextManager (class in typing): Classes functions and decorators. + (line 340) +* asynccontextmanager() (in module contextlib): Utilities. (line 91) +* AsyncExitStack (class in contextlib): Utilities. (line 453) +* AsyncGenerator (class in collections.abc): Collections Abstract Base Classes. + (line 244) +* AsyncGenerator (class in typing): Classes functions and decorators. + (line 423) +* AsyncGeneratorType (in module types): Standard Interpreter Types. + (line 43) +* asynchat (module): asynchat — Asynchronous socket command/response handler. + (line 6) +* asynchronous context manager: Glossary. (line 94) +* asynchronous generator: Glossary. (line 100) +* asynchronous generator iterator: Glossary. (line 117) +* asynchronous generator; asynchronous iterator: The standard type hierarchy. + (line 467) +* asynchronous generator; function: The standard type hierarchy. + (line 467) +* asynchronous iterable: Glossary. (line 134) +* asynchronous iterator: Glossary. (line 140) +* asyncio (module): asyncio — Asynchronous I/O. + (line 6) +* asyncio.subprocess.DEVNULL (in module asyncio): Constants<7>. + (line 23) +* asyncio.subprocess.PIPE (in module asyncio): Constants<7>. (line 6) +* asyncio.subprocess.Process (class in asyncio): Interacting with Subprocesses. + (line 11) +* asyncio.subprocess.STDOUT (in module asyncio): Constants<7>. + (line 17) +* AsyncIterable (class in collections.abc): Collections Abstract Base Classes. + (line 230) +* AsyncIterable (class in typing): Classes functions and decorators. + (line 320) +* AsyncIterator (class in collections.abc): Collections Abstract Base Classes. + (line 237) +* AsyncIterator (class in typing): Classes functions and decorators. + (line 326) +* AsyncMock (class in unittest.mock): The Mock Class. (line 667) +* asyncore (module): asyncore — Asynchronous socket handler. + (line 6) +* AsyncResult (class in multiprocessing.pool): Process Pools. + (line 191) +* asyncSetUp() (unittest.IsolatedAsyncioTestCase method): Test cases. + (line 816) +* asyncTearDown() (unittest.IsolatedAsyncioTestCase method): Test cases. + (line 825) +* async_chat (class in asynchat): asynchat — Asynchronous socket command/response handler. + (line 28) +* async_chat.ac_in_buffer_size (in module asynchat): asynchat — Asynchronous socket command/response handler. + (line 47) +* async_chat.ac_out_buffer_size (in module asynchat): asynchat — Asynchronous socket command/response handler. + (line 51) +* as_bytes() (email.message.EmailMessage method): email message Representing an email message. + (line 107) +* as_bytes() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 106) +* as_completed() (in module asyncio): Waiting Primitives. (line 86) +* as_completed() (in module concurrent.futures): Module Functions. + (line 43) +* as_integer_ratio() (decimal.Decimal method): Decimal objects. + (line 122) +* as_integer_ratio() (float method): Additional Methods on Float. + (line 9) +* as_integer_ratio() (fractions.Fraction method): fractions — Rational numbers. + (line 98) +* as_integer_ratio() (int method): Additional Methods on Integer Types. + (line 98) +* AS_IS (in module formatter): The Formatter Interface. + (line 12) +* as_posix() (pathlib.PurePath method): Methods and properties. + (line 142) +* as_string() (email.message.EmailMessage method): email message Representing an email message. + (line 61) +* as_string() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 60) +* as_tuple() (decimal.Decimal method): Decimal objects. (line 136) +* as_uri() (pathlib.PurePath method): Methods and properties. + (line 153) +* AT (in module token): token — Constants used with Python parse trees. + (line 222) +* atan() (in module cmath): Trigonometric functions<2>. + (line 18) +* atan() (in module math): Trigonometric functions. + (line 14) +* atan2() (in module math): Trigonometric functions. + (line 18) +* atanh() (in module cmath): Hyperbolic functions<2>. + (line 19) +* atanh() (in module math): Hyperbolic functions. + (line 17) +* ATEQUAL (in module token): token — Constants used with Python parse trees. + (line 226) +* atexit (module): atexit — Exit handlers. + (line 6) +* atexit (weakref.finalize attribute): weakref — Weak references. + (line 275) +* athrow() (agen method): Asynchronous generator-iterator methods. + (line 43) +* atof() (in module locale): locale — Internationalization services. + (line 448) +* atoi() (in module locale): locale — Internationalization services. + (line 453) +* atom: Atoms. (line 6) +* attach() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 167) +* attach_loop() (asyncio.AbstractChildWatcher method): Process Watchers. + (line 59) +* attach_mock() (unittest.mock.Mock method): The Mock Class. (line 235) +* AttlistDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 208) +* attrgetter() (in module operator): operator — Standard operators as functions. + (line 221) +* attrib (xml.etree.ElementTree.Element attribute): Element Objects. + (line 45) +* attribute: The standard type hierarchy. + (line 13) +* attribute <1>: Glossary. (line 149) +* attribute; assignment: Assignment statements. + (line 6) +* attribute; assignment <1>: Assignment statements. + (line 78) +* attribute; deletion: The del statement<2>. + (line 20) +* attribute; reference: Attribute references. + (line 6) +* AttributeError: Concrete exceptions. + (line 12) +* attributes (xml.dom.Node attribute): Node Objects. (line 26) +* AttributesImpl (class in xml.sax.xmlreader): xml sax xmlreader — Interface for XML parsers. + (line 69) +* AttributesNSImpl (class in xml.sax.xmlreader): xml sax xmlreader — Interface for XML parsers. + (line 80) +* attroff() (curses.window method): Window Objects. (line 53) +* attron() (curses.window method): Window Objects. (line 58) +* attrset() (curses.window method): Window Objects. (line 63) +* at_eof() (asyncio.StreamReader method): StreamReader. (line 64) +* Audio Interchange File Format: aifc — Read and write AIFF and AIFC files. + (line 8) +* Audio Interchange File Format <1>: chunk — Read IFF chunked data. + (line 8) +* AUDIODEV: ossaudiodev — Access to OSS-compatible audio devices. + (line 54) +* audioop (module): audioop — Manipulate raw audio data. + (line 6) +* AUDIO_FILE_ENCODING_ADPCM_G721 (in module sunau): sunau — Read and write Sun AU files. + (line 98) +* AUDIO_FILE_ENCODING_ADPCM_G722 (in module sunau): sunau — Read and write Sun AU files. + (line 98) +* AUDIO_FILE_ENCODING_ADPCM_G723_3 (in module sunau): sunau — Read and write Sun AU files. + (line 98) +* AUDIO_FILE_ENCODING_ADPCM_G723_5 (in module sunau): sunau — Read and write Sun AU files. + (line 98) +* AUDIO_FILE_ENCODING_ALAW_8 (in module sunau): sunau — Read and write Sun AU files. + (line 88) +* AUDIO_FILE_ENCODING_DOUBLE (in module sunau): sunau — Read and write Sun AU files. + (line 98) +* AUDIO_FILE_ENCODING_FLOAT (in module sunau): sunau — Read and write Sun AU files. + (line 98) +* AUDIO_FILE_ENCODING_LINEAR_16 (in module sunau): sunau — Read and write Sun AU files. + (line 88) +* AUDIO_FILE_ENCODING_LINEAR_24 (in module sunau): sunau — Read and write Sun AU files. + (line 88) +* AUDIO_FILE_ENCODING_LINEAR_32 (in module sunau): sunau — Read and write Sun AU files. + (line 88) +* AUDIO_FILE_ENCODING_LINEAR_8 (in module sunau): sunau — Read and write Sun AU files. + (line 88) +* AUDIO_FILE_ENCODING_MULAW_8 (in module sunau): sunau — Read and write Sun AU files. + (line 88) +* AUDIO_FILE_MAGIC (in module sunau): sunau — Read and write Sun AU files. + (line 82) +* audit events: Debugging and Profiling. + (line 11) +* audit() (in module sys): sys — System-specific parameters and functions. + (line 74) +* auditing: sys — System-specific parameters and functions. + (line 76) +* augmented; assignment: Augmented assignment statements. + (line 6) +* auth() (ftplib.FTP_TLS method): FTP_TLS Objects. (line 14) +* auth() (smtplib.SMTP method): SMTP Objects. (line 141) +* authenticate() (imaplib.IMAP4 method): IMAP4 Objects. (line 37) +* AuthenticationError: Process and exceptions. + (line 216) +* authenticators() (netrc.netrc method): netrc Objects. (line 8) +* authkey (multiprocessing.Process attribute): Process and exceptions. + (line 118) +* auto (class in enum): Module Contents<2>. (line 39) +* autorange() (timeit.Timer method): Python Interface. (line 90) +* avg() (in module audioop): audioop — Manipulate raw audio data. + (line 51) +* avgpp() (in module audioop): audioop — Manipulate raw audio data. + (line 55) +* avoids_symlink_attacks (shutil.rmtree attribute): Directory and files operations. + (line 293) +* AWAIT (in module token): token — Constants used with Python parse trees. + (line 244) +* await; in comprehensions: Displays for lists sets and dictionaries. + (line 45) +* awaitable: Glossary. (line 155) +* Awaitable (class in collections.abc): Collections Abstract Base Classes. + (line 194) +* Awaitable (class in typing): Classes functions and decorators. + (line 298) +* await_args (unittest.mock.AsyncMock attribute): The Mock Class. + (line 882) +* await_args_list (unittest.mock.AsyncMock attribute): The Mock Class. + (line 900) +* await_count (unittest.mock.AsyncMock attribute): The Mock Class. + (line 866) +* b"; bytes literal: String and Bytes literals. + (line 43) +* b’; bytes literal: String and Bytes literals. + (line 43) +* b16decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 128) +* b16encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 123) +* b2a_base64() (in module binascii): binascii — Convert between binary and ASCII. + (line 46) +* b2a_hex() (in module binascii): binascii — Convert between binary and ASCII. + (line 131) +* b2a_hqx() (in module binascii): binascii — Convert between binary and ASCII. + (line 100) +* b2a_qp() (in module binascii): binascii — Convert between binary and ASCII. + (line 62) +* b2a_uu() (in module binascii): binascii — Convert between binary and ASCII. + (line 32) +* b32decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 103) +* b32encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 98) +* b64decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 55) +* b64encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 43) +* b85decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 195) +* b85encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 184) +* Babyl (class in mailbox): Babyl. (line 6) +* BabylMessage (class in mailbox): BabylMessage. (line 6) +* back() (in module turtle): Turtle motion. (line 24) +* backslash character: Explicit line joining. + (line 6) +* backslashreplace_errors() (in module codecs): Error Handlers. + (line 151) +* backup() (sqlite3.Connection method): Connection Objects. (line 403) +* backward() (in module turtle): Turtle motion. (line 24) +* BadGzipFile: gzip — Support for gzip files. + (line 63) +* BadStatusLine: http client — HTTP protocol client. + (line 174) +* BadZipFile: zipfile — Work with ZIP archives. + (line 24) +* BadZipfile: zipfile — Work with ZIP archives. + (line 30) +* Balloon (class in tkinter.tix): Basic Widgets. (line 6) +* Barrier (class in multiprocessing): Synchronization primitives. + (line 13) +* Barrier (class in threading): Barrier Objects. (line 36) +* Barrier() (multiprocessing.managers.SyncManager method): Managers. + (line 143) +* base64 (module): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 6) +* base64; encoding: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 8) +* BaseCGIHandler (class in wsgiref.handlers): wsgiref handlers – server/gateway base classes. + (line 53) +* BaseCookie (class in http.cookies): http cookies — HTTP state management. + (line 38) +* BaseException: Base classes. (line 9) +* BaseHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 272) +* BaseHandler (class in wsgiref.handlers): wsgiref handlers – server/gateway base classes. + (line 88) +* BaseHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 37) +* BaseHTTPRequestHandler (class in http.server): http server — HTTP servers. + (line 46) +* BaseManager (class in multiprocessing.managers): Managers. (line 23) +* basename() (in module os.path): os path — Common pathname manipulations. + (line 62) +* BaseProtocol (class in asyncio): Base Protocols. (line 6) +* BaseProxy (class in multiprocessing.managers): Proxy Objects. + (line 92) +* BaseRequestHandler (class in socketserver): Request Handler Objects. + (line 6) +* BaseRotatingHandler (class in logging.handlers): BaseRotatingHandler. + (line 12) +* BaseSelector (class in selectors): Classes<3>. (line 53) +* BaseServer (class in socketserver): Server Objects<2>. (line 6) +* basestring (2to3 fixer): Fixers. (line 60) +* BaseTransport (class in asyncio): Transports Hierarchy. + (line 6) +* base_exec_prefix (in module sys): sys — System-specific parameters and functions. + (line 105) +* base_prefix (in module sys): sys — System-specific parameters and functions. + (line 119) +* basicConfig() (in module logging): Module-Level Functions. + (line 244) +* BasicContext (class in decimal): Context objects. (line 47) +* BasicInterpolation (class in configparser): Interpolation of values. + (line 10) +* BasicTestRunner (class in test.support): test support — Utilities for the Python test suite. + (line 1102) +* baudrate() (in module curses): Functions<3>. (line 18) +* bbox() (tkinter.ttk.Treeview method): ttk Treeview. (line 8) +* BDADDR_ANY (in module socket): Constants<8>. (line 218) +* BDADDR_LOCAL (in module socket): Constants<8>. (line 218) +* Bdb (class in bdb): bdb — Debugger framework. + (line 79) +* bdb (module): bdb — Debugger framework. + (line 6) +* BdbQuit: bdb — Debugger framework. + (line 15) +* BDFL: Glossary. (line 161) +* bdist_msi (class in distutils.command.bdist_msi): distutils command bdist_msi — Build a Microsoft Installer binary package. + (line 6) +* beep() (in module curses): Functions<3>. (line 26) +* Beep() (in module winsound): winsound — Sound-playing interface for Windows. + (line 12) +* BEFORE_ASYNC_WITH (opcode): Python Bytecode Instructions. + (line 295) +* begin_fill() (in module turtle): Filling. (line 16) +* BEGIN_FINALLY (opcode): Python Bytecode Instructions. + (line 406) +* begin_poly() (in module turtle): Special Turtle methods. + (line 6) +* below() (curses.panel.Panel method): Panel Objects. (line 17) +* BELOW_NORMAL_PRIORITY_CLASS (in module subprocess): Windows Constants. + (line 59) +* benchmarking: Functions<2>. (line 182) +* benchmarking <1>: Functions<2>. (line 200) +* benchmarking <2>: Functions<2>. (line 479) +* Benchmarking: timeit — Measure execution time of small code snippets. + (line 8) +* betavariate() (in module random): Real-valued distributions. + (line 33) +* bgcolor() (in module turtle): Window control. (line 6) +* bgpic() (in module turtle): Window control. (line 21) +* bias() (in module audioop): audioop — Manipulate raw audio data. + (line 61) +* bidirectional() (in module unicodedata): unicodedata — Unicode Database. + (line 54) +* bigaddrspacetest() (in module test.support): test support — Utilities for the Python test suite. + (line 676) +* BigEndianStructure (class in ctypes): Structured data types. + (line 10) +* bigmemtest() (in module test.support): test support — Utilities for the Python test suite. + (line 660) +* bin() (built-in function): Built-in Functions. (line 92) +* Binary (class in msilib): msilib — Read and write Microsoft Installer files. + (line 89) +* Binary (class in xmlrpc.client): Binary Objects. (line 6) +* binary file: Glossary. (line 166) +* binary literal: Numeric literals. (line 6) +* binary mode: Built-in Functions. (line 1217) +* binary semaphores: _thread — Low-level threading API. + (line 6) +* binary; arithmetic; operation: Binary arithmetic operations. + (line 6) +* binary; bitwise; operation: Binary bitwise operations. + (line 6) +* binary; literals: Numeric Types — int float complex. + (line 19) +* binaryfunc (C type): Slot Type typedefs. (line 111) +* BinaryIO (class in typing): Classes functions and decorators. + (line 471) +* BINARY_ADD (opcode): Python Bytecode Instructions. + (line 157) +* BINARY_AND (opcode): Python Bytecode Instructions. + (line 177) +* BINARY_FLOOR_DIVIDE (opcode): Python Bytecode Instructions. + (line 145) +* BINARY_LSHIFT (opcode): Python Bytecode Instructions. + (line 169) +* BINARY_MATRIX_MULTIPLY (opcode): Python Bytecode Instructions. + (line 139) +* BINARY_MODULO (opcode): Python Bytecode Instructions. + (line 153) +* BINARY_MULTIPLY (opcode): Python Bytecode Instructions. + (line 135) +* BINARY_OR (opcode): Python Bytecode Instructions. + (line 185) +* BINARY_POWER (opcode): Python Bytecode Instructions. + (line 131) +* BINARY_RSHIFT (opcode): Python Bytecode Instructions. + (line 173) +* BINARY_SUBSCR (opcode): Python Bytecode Instructions. + (line 165) +* BINARY_SUBTRACT (opcode): Python Bytecode Instructions. + (line 161) +* BINARY_TRUE_DIVIDE (opcode): Python Bytecode Instructions. + (line 149) +* BINARY_XOR (opcode): Python Bytecode Instructions. + (line 181) +* binascii (module): binascii — Convert between binary and ASCII. + (line 6) +* bind (widgets): Bindings and Events. + (line 6) +* bind() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 222) +* bind() (inspect.Signature method): Introspecting callables with the Signature object. + (line 96) +* bind() (socket.socket method): Socket Objects. (line 30) +* binding; name: Binding of names. (line 6) +* binding; name <1>: Assignment statements. + (line 6) +* bindtextdomain() (in module gettext): GNU gettext API. (line 14) +* bindtextdomain() (in module locale): Access to message catalogs. + (line 14) +* bind_partial() (inspect.Signature method): Introspecting callables with the Signature object. + (line 103) +* bind_port() (in module test.support): test support — Utilities for the Python test suite. + (line 804) +* bind_textdomain_codeset() (in module gettext): GNU gettext API. + (line 26) +* bind_unix_socket() (in module test.support): test support — Utilities for the Python test suite. + (line 821) +* binhex (module): binhex — Encode and decode binhex4 files. + (line 6) +* binhex() (in module binhex): binhex — Encode and decode binhex4 files. + (line 16) +* bisect (module): bisect — Array bisection algorithm. + (line 6) +* bisect() (in module bisect): bisect — Array bisection algorithm. + (line 34) +* bisect_left() (in module bisect): bisect — Array bisection algorithm. + (line 20) +* bisect_right() (in module bisect): bisect — Array bisection algorithm. + (line 34) +* bitmap() (msilib.Dialog method): GUI classes. (line 60) +* bitwise; and: Binary bitwise operations. + (line 12) +* bitwise; operations: Bitwise Operations on Integer Types. + (line 6) +* bitwise; or: Binary bitwise operations. + (line 18) +* bitwise; xor: Binary bitwise operations. + (line 15) +* bit_length() (int method): Additional Methods on Integer Types. + (line 9) +* bk() (in module turtle): Turtle motion. (line 24) +* bkgd() (curses.window method): Window Objects. (line 68) +* bkgdset() (curses.window method): Window Objects. (line 80) +* blake2b() (in module hashlib): Creating hash objects. + (line 8) +* blake2b, blake2s: BLAKE2. (line 6) +* blake2b.MAX_DIGEST_SIZE (in module hashlib): Constants<5>. (line 24) +* blake2b.MAX_KEY_SIZE (in module hashlib): Constants<5>. (line 18) +* blake2b.PERSON_SIZE (in module hashlib): Constants<5>. (line 12) +* blake2b.SALT_SIZE (in module hashlib): Constants<5>. (line 6) +* blake2s() (in module hashlib): Creating hash objects. + (line 13) +* blake2s.MAX_DIGEST_SIZE (in module hashlib): Constants<5>. (line 26) +* blake2s.MAX_KEY_SIZE (in module hashlib): Constants<5>. (line 20) +* blake2s.PERSON_SIZE (in module hashlib): Constants<5>. (line 14) +* blake2s.SALT_SIZE (in module hashlib): Constants<5>. (line 8) +* blank line: Blank lines. (line 6) +* block: Structure of a program. + (line 6) +* blocked_domains() (http.cookiejar.DefaultCookiePolicy method): DefaultCookiePolicy Objects. + (line 52) +* BlockingIOError: OS exceptions. (line 9) +* BlockingIOError <1>: High-level Module Interface. + (line 38) +* blocksize (http.client.HTTPConnection attribute): HTTPConnection Objects. + (line 115) +* block_size (hmac.HMAC attribute): hmac — Keyed-Hashing for Message Authentication. + (line 94) +* BNF: Notation. (line 6) +* BNF <1>: Expressions. (line 6) +* body() (nntplib.NNTP method): Methods<3>. (line 272) +* body_encode() (email.charset.Charset method): email charset Representing character sets. + (line 135) +* body_encoding (email.charset.Charset attribute): email charset Representing character sets. + (line 67) +* body_line_iterator() (in module email.iterators): email iterators Iterators. + (line 14) +* BOM (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_BE (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_LE (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF16 (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF16_BE (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF16_LE (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF32 (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF32_BE (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF32_LE (in module codecs): codecs — Codec registry and base classes. + (line 241) +* BOM_UTF8 (in module codecs): codecs — Codec registry and base classes. + (line 241) +* bool (built-in class): Built-in Functions. (line 114) +* Boolean; operation: Boolean operations. (line 6) +* Boolean; operations: Truth Value Testing. + (line 6) +* Boolean; operations <1>: Boolean Operations — and or not. + (line 6) +* Boolean; type: Built-in Functions. (line 124) +* Boolean; values: Boolean Values. (line 15) +* BOOLEAN_STATES (configparser.ConfigParser attribute): Customizing Parser Behaviour. + (line 245) +* bootstrap() (in module ensurepip): Module API. (line 13) +* border() (curses.window method): Window Objects. (line 91) +* bottom() (curses.panel.Panel method): Panel Objects. (line 21) +* bottom_panel() (in module curses.panel): Functions<4>. (line 8) +* BoundArguments (class in inspect): Introspecting callables with the Signature object. + (line 276) +* BoundaryError: email errors Exception and Defect classes. + (line 38) +* BoundedSemaphore (class in asyncio): BoundedSemaphore. (line 6) +* BoundedSemaphore (class in multiprocessing): Synchronization primitives. + (line 19) +* BoundedSemaphore (class in threading): Semaphore Objects. (line 69) +* BoundedSemaphore() (multiprocessing.managers.SyncManager method): Managers. + (line 150) +* box() (curses.window method): Window Objects. (line 130) +* bpformat() (bdb.Breakpoint method): bdb — Debugger framework. + (line 55) +* bpprint() (bdb.Breakpoint method): bdb — Debugger framework. + (line 74) +* break (pdb command): Debugger Commands. (line 71) +* Breakpoint (class in bdb): bdb — Debugger framework. + (line 22) +* breakpoint() (built-in function): Built-in Functions. (line 126) +* breakpointhook() (in module sys): sys — System-specific parameters and functions. + (line 187) +* breakpoints: Help menu Shell and Editor. + (line 29) +* break_anywhere() (bdb.Bdb method): bdb — Debugger framework. + (line 204) +* break_here() (bdb.Bdb method): bdb — Debugger framework. + (line 197) +* break_long_words (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 238) +* break_on_hyphens (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 248) +* broadcast_address (ipaddress.IPv4Network attribute): Network objects. + (line 91) +* broadcast_address (ipaddress.IPv6Network attribute): Network objects. + (line 308) +* broken (threading.Barrier attribute): Barrier Objects. (line 101) +* BrokenBarrierError: Barrier Objects. (line 106) +* BrokenExecutor: Exception classes. (line 14) +* BrokenPipeError: OS exceptions. (line 37) +* BrokenProcessPool: Exception classes. (line 37) +* BrokenThreadPool: Exception classes. (line 29) +* BROWSER: webbrowser — Convenient Web-browser controller. + (line 20) +* BROWSER <1>: webbrowser — Convenient Web-browser controller. + (line 95) +* BsdDbShelf (class in shelve): Restrictions. (line 52) +* buf (multiprocessing.shared_memory.SharedMemory attribute): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 87) +* buffer (2to3 fixer): Fixers. (line 64) +* buffer (io.TextIOBase attribute): Text I/O<2>. (line 30) +* buffer (unittest.TestResult attribute): Loading and running tests. + (line 277) +* buffer interface; (see buffer protocol): Iterator Protocol. + (line 45) +* buffer object; (see buffer protocol): Iterator Protocol. (line 44) +* buffer protocol: Iterator Protocol. (line 45) +* buffer protocol; binary sequence types: printf-style String Formatting. + (line 189) +* buffer protocol; str (built-in class): Text Sequence Type — str. + (line 60) +* buffer size, I/O: Built-in Functions. (line 1217) +* BufferedIOBase (class in io): I/O Base Classes. (line 223) +* BufferedProtocol (class in asyncio): Base Protocols. (line 15) +* BufferedRandom (class in io): Buffered Streams. (line 134) +* BufferedReader (class in io): Buffered Streams. (line 56) +* BufferedRWPair (class in io): Buffered Streams. (line 147) +* BufferedWriter (class in io): Buffered Streams. (line 95) +* BufferError: Base classes. (line 49) +* BufferingHandler (class in logging.handlers): MemoryHandler. + (line 19) +* BufferTooShort: Process and exceptions. + (line 208) +* buffer_info() (array.array method): array — Efficient arrays of numeric values. + (line 117) +* buffer_size (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 84) +* buffer_text (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 91) +* buffer_updated() (asyncio.BufferedProtocol method): Buffered Streaming Protocols. + (line 35) +* buffer_used (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 100) +* bufsize() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 214) +* BUILD_CONST_KEY_MAP (opcode): Python Bytecode Instructions. + (line 558) +* BUILD_LIST (opcode): Python Bytecode Instructions. + (line 540) +* BUILD_LIST_UNPACK (opcode): Python Bytecode Instructions. + (line 590) +* BUILD_MAP (opcode): Python Bytecode Instructions. + (line 548) +* BUILD_MAP_UNPACK (opcode): Python Bytecode Instructions. + (line 606) +* BUILD_MAP_UNPACK_WITH_CALL (opcode): Python Bytecode Instructions. + (line 614) +* build_opener() (in module urllib.request): urllib request — Extensible library for opening URLs. + (line 125) +* build_py (class in distutils.command.build_py): distutils command build_py — Build the py/ pyc files of a package. + (line 6) +* build_py_2to3 (class in distutils.command.build_py): distutils command build_py — Build the py/ pyc files of a package. + (line 8) +* BUILD_SET (opcode): Python Bytecode Instructions. + (line 544) +* BUILD_SET_UNPACK (opcode): Python Bytecode Instructions. + (line 598) +* BUILD_SLICE (opcode): Python Bytecode Instructions. + (line 861) +* BUILD_STRING (opcode): Python Bytecode Instructions. + (line 567) +* BUILD_TUPLE (opcode): Python Bytecode Instructions. + (line 535) +* BUILD_TUPLE_UNPACK (opcode): Python Bytecode Instructions. + (line 574) +* BUILD_TUPLE_UNPACK_WITH_CALL (opcode): Python Bytecode Instructions. + (line 582) +* built-in function; abs: Emulating numeric types. + (line 116) +* built-in function; abs <1>: Number Protocol. (line 86) +* built-in function; ascii: Object Protocol. (line 177) +* built-in function; bytes: Basic customization. + (line 146) +* built-in function; bytes <1>: Object Protocol. (line 198) +* built-in function; call: Calls. (line 132) +* built-in function; chr: The standard type hierarchy. + (line 156) +* built-in function; classmethod: Common Object Structures. + (line 227) +* built-in function; compile: The global statement. + (line 27) +* built-in function; compile <1>: Code Objects. (line 6) +* built-in function; compile <2>: Standard Interpreter Types. + (line 52) +* built-in function; compile <3>: Queries on ST Objects. + (line 14) +* built-in function; compile <4>: Importing Modules<2>. + (line 119) +* built-in function; complex: Emulating numeric types. + (line 123) +* built-in function; complex <1>: Numeric Types — int float complex. + (line 27) +* built-in function; divmod: Emulating numeric types. + (line 26) +* built-in function; divmod <1>: Emulating numeric types. + (line 56) +* built-in function; divmod <2>: Number Protocol. (line 59) +* built-in function; eval: The global statement. + (line 27) +* built-in function; eval <1>: Expression input. (line 6) +* built-in function; eval <2>: Code Objects. (line 14) +* built-in function; eval <3>: pprint — Data pretty printer. + (line 132) +* built-in function; eval <4>: PrettyPrinter Objects. + (line 26) +* built-in function; eval <5>: Converting ST Objects. + (line 45) +* built-in function; exec: The global statement. + (line 27) +* built-in function; exec <1>: Built-in Functions. (line 519) +* built-in function; exec <2>: Code Objects. (line 14) +* built-in function; exec <3>: Converting ST Objects. + (line 45) +* built-in function; float: Emulating numeric types. + (line 123) +* built-in function; float <1>: Numeric Types — int float complex. + (line 27) +* built-in function; float <2>: Number Protocol. (line 238) +* built-in function; hash: Basic customization. + (line 223) +* built-in function; hash <1>: Immutable Sequence Types. + (line 6) +* built-in function; hash <2>: Object Protocol. (line 425) +* built-in function; hash <3>: PyTypeObject Slots. (line 316) +* built-in function; help: Operating System Interface. + (line 20) +* built-in function; id: Objects values and types. + (line 11) +* built-in function; int: Emulating numeric types. + (line 123) +* built-in function; int <1>: Numeric Types — int float complex. + (line 27) +* built-in function; int <2>: Number Protocol. (line 231) +* built-in function; len: The standard type hierarchy. + (line 127) +* built-in function; len <1>: The standard type hierarchy. + (line 219) +* built-in function; len <2>: The standard type hierarchy. + (line 249) +* built-in function; len <3>: Emulating container types. + (line 39) +* built-in function; len <4>: Common Sequence Operations. + (line 21) +* built-in function; len <5>: Mapping Types — dict. + (line 6) +* built-in function; len <6>: Object Protocol. (line 471) +* built-in function; len <7>: Sequence Protocol. (line 17) +* built-in function; len <8>: Mapping Protocol. (line 20) +* built-in function; len <9>: List Objects. (line 39) +* built-in function; len <10>: Dictionary Objects. (line 136) +* built-in function; len <11>: Set Objects. (line 90) +* built-in function; max: Common Sequence Operations. + (line 21) +* built-in function; min: Common Sequence Operations. + (line 21) +* built-in function; open: Reading and Writing Files. + (line 6) +* built-in function; open <1>: The standard type hierarchy. + (line 630) +* built-in function; ord: The standard type hierarchy. + (line 156) +* built-in function; pow: Emulating numeric types. + (line 26) +* built-in function; pow <1>: Emulating numeric types. + (line 26) +* built-in function; pow <2>: Emulating numeric types. + (line 56) +* built-in function; pow <3>: Emulating numeric types. + (line 66) +* built-in function; pow <4>: Number Protocol. (line 68) +* built-in function; pow <5>: Number Protocol. (line 183) +* built-in function; print: Basic customization. + (line 150) +* built-in function; range: The for statement. (line 38) +* built-in function; repr: Expression statements. + (line 17) +* built-in function; repr <1>: Finalization and De-allocation. + (line 78) +* built-in function; repr <2>: Object Protocol. (line 165) +* built-in function; repr <3>: PyTypeObject Slots. (line 257) +* built-in function; round: Emulating numeric types. + (line 146) +* built-in function; slice: The standard type hierarchy. + (line 799) +* built-in function; slice <1>: Python Bytecode Instructions. + (line 863) +* built-in function; staticmethod: Common Object Structures. + (line 234) +* built-in function; tuple: Sequence Protocol. (line 119) +* built-in function; tuple <1>: List Objects. (line 126) +* built-in function; type: Objects values and types. + (line 11) +* built-in function; type <1>: Metaclasses. (line 6) +* built-in function; type <2>: Type Objects. (line 6) +* built-in function; type <3>: Object Protocol. (line 452) +* built-in function; __import__: Importing Modules<2>. + (line 38) +* built-in method; call: Calls. (line 132) +* built-in; method: The standard type hierarchy. + (line 498) +* built-in; types: Built-in Types. (line 9) +* BuiltinFunctionType (in module types): Standard Interpreter Types. + (line 81) +* BuiltinImporter (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 67) +* BuiltinMethodType (in module types): Standard Interpreter Types. + (line 81) +* builtins (module): builtins — Built-in objects. + (line 6) +* builtin_module_names (in module sys): sys — System-specific parameters and functions. + (line 140) +* ButtonBox (class in tkinter.tix): Basic Widgets. (line 13) +* bye() (in module turtle): Methods specific to Screen not inherited from TurtleScreen. + (line 6) +* byref() (in module ctypes): Utility functions. (line 19) +* byte: The standard type hierarchy. + (line 183) +* bytearray: The standard type hierarchy. + (line 207) +* bytearray (built-in class): Bytearray Objects. (line 9) +* bytearray; formatting: printf-style Bytes Formatting. + (line 6) +* bytearray; interpolation: printf-style Bytes Formatting. + (line 6) +* bytearray; methods: Bytes and Bytearray Operations. + (line 6) +* bytecode: The standard type hierarchy. + (line 649) +* bytecode <1>: Glossary. (line 196) +* Bytecode (class in dis): Bytecode analysis. (line 12) +* Bytecode.codeobj (in module dis): Bytecode analysis. (line 39) +* Bytecode.first_line (in module dis): Bytecode analysis. (line 43) +* BYTECODE_SUFFIXES (in module importlib.machinery): importlib machinery – Importers and path hooks. + (line 40) +* byteorder (in module sys): sys — System-specific parameters and functions. + (line 133) +* bytes: The standard type hierarchy. + (line 183) +* bytes (built-in class): Bytes Objects. (line 12) +* bytes (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 84) +* bytes literal: Literals. (line 8) +* bytes-like object: Glossary. (line 177) +* bytes; formatting: printf-style Bytes Formatting. + (line 6) +* bytes; interpolation: printf-style Bytes Formatting. + (line 6) +* bytes; methods: Bytes and Bytearray Operations. + (line 6) +* bytes; str (built-in class): Text Sequence Type — str. + (line 60) +* BytesFeedParser (class in email.parser): FeedParser API. (line 28) +* BytesGenerator (class in email.generator): email generator Generating MIME documents. + (line 43) +* BytesHeaderParser (class in email.parser): Parser API. (line 67) +* BytesIO (class in io): Buffered Streams. (line 9) +* BytesParser (class in email.parser): Parser API. (line 17) +* ByteString (class in collections.abc): Collections Abstract Base Classes. + (line 159) +* ByteString (class in typing): Classes functions and decorators. + (line 237) +* byteswap() (array.array method): array — Efficient arrays of numeric values. + (line 136) +* byteswap() (in module audioop): audioop — Manipulate raw audio data. + (line 66) +* BytesWarning: Warnings. (line 54) +* bytes_le (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 89) +* byte_compile() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 127) +* bz2 (module): bz2 — Support for bzip2 compression. + (line 6) +* BZ2Compressor (class in bz2): Incremental de compression. + (line 6) +* BZ2Decompressor (class in bz2): Incremental de compression. + (line 33) +* BZ2File (class in bz2): De compression of files. + (line 40) +* C: String and Bytes literals. + (line 72) +* C-contiguous: shape strides suboffsets. + (line 26) +* C-contiguous <1>: Glossary. (line 273) +* C14NWriterTarget (class in xml.etree.ElementTree): TreeBuilder Objects. + (line 93) +* C; language: The standard type hierarchy. + (line 6) +* C; language <1>: The standard type hierarchy. + (line 107) +* C; language <2>: The standard type hierarchy. + (line 485) +* C; language <3>: Comparisons. (line 6) +* C; language <4>: Numeric Types — int float complex. + (line 6) +* C; language <5>: Numeric Types — int float complex. + (line 104) +* C; structures: struct — Interpret bytes as packed binary data. + (line 8) +* CAB (class in msilib): CAB Objects. (line 6) +* cached (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 403) +* cached_property() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 16) +* CacheFTPHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 430) +* cache_from_source() (in module imp): imp — Access the import internals. + (line 205) +* cache_from_source() (in module importlib.util): importlib util – Utility code for importers. + (line 21) +* calcobjsize() (in module test.support): test support — Utilities for the Python test suite. + (line 531) +* calcsize() (in module struct): Functions and Exceptions. + (line 55) +* calcvobjsize() (in module test.support): test support — Utilities for the Python test suite. + (line 536) +* Calendar (class in calendar): calendar — General calendar-related functions. + (line 27) +* calendar (module): calendar — General calendar-related functions. + (line 6) +* calendar() (in module calendar): calendar — General calendar-related functions. + (line 349) +* call: Slicings. (line 38) +* call() (in module subprocess): Older high-level API. + (line 10) +* call() (in module unittest.mock): call. (line 6) +* call; instance: Emulating callable objects. + (line 8) +* Callable (class in collections.abc): Collections Abstract Base Classes. + (line 116) +* Callable (in module typing): Classes functions and decorators. + (line 863) +* callable() (built-in function): Built-in Functions. (line 193) +* CallableProxyType (in module weakref): weakref — Weak references. + (line 298) +* callback: Glossary. (line 211) +* callback (optparse.Option attribute): Option attributes. (line 60) +* callback() (contextlib.ExitStack method): Utilities. (line 414) +* callbacks (in module gc): gc — Garbage Collector interface. + (line 229) +* callback_args (optparse.Option attribute): Option attributes. + (line 66) +* callback_kwargs (optparse.Option attribute): Option attributes. + (line 66) +* called (unittest.mock.Mock attribute): The Mock Class. (line 296) +* CalledProcessError: Using the subprocess Module. + (line 190) +* calloc(): Overview<3>. (line 33) +* call_args (unittest.mock.Mock attribute): The Mock Class. (line 415) +* call_args_list (unittest.mock.Mock attribute): The Mock Class. + (line 459) +* call_at() (asyncio.loop method): Scheduling delayed callbacks. + (line 37) +* call_count (unittest.mock.Mock attribute): The Mock Class. (line 308) +* call_exception_handler() (asyncio.loop method): Error Handling API. + (line 37) +* CALL_FINALLY (opcode): Python Bytecode Instructions. + (line 703) +* CALL_FUNCTION (opcode): Python Bytecode Instructions. + (line 769) +* CALL_FUNCTION_EX (opcode): Python Bytecode Instructions. + (line 799) +* CALL_FUNCTION_KW (opcode): Python Bytecode Instructions. + (line 782) +* call_later() (asyncio.loop method): Scheduling delayed callbacks. + (line 10) +* call_list() (unittest.mock.call method): call. (line 20) +* CALL_METHOD (opcode): Python Bytecode Instructions. + (line 829) +* call_soon() (asyncio.loop method): Scheduling callbacks. + (line 6) +* call_soon_threadsafe() (asyncio.loop method): Scheduling callbacks. + (line 23) +* call_tracing() (in module sys): sys — System-specific parameters and functions. + (line 147) +* cancel() (asyncio.Future method): Future Object. (line 109) +* cancel() (asyncio.Handle method): Callback Handles. (line 11) +* cancel() (asyncio.Task method): Task Object. (line 52) +* cancel() (concurrent.futures.Future method): Future Objects. + (line 16) +* cancel() (sched.scheduler method): Scheduler Objects. (line 41) +* cancel() (threading.Timer method): Timer Objects. (line 35) +* cancelled() (asyncio.Future method): Future Object. (line 67) +* cancelled() (asyncio.Handle method): Callback Handles. (line 16) +* cancelled() (asyncio.Task method): Task Object. (line 104) +* cancelled() (concurrent.futures.Future method): Future Objects. + (line 24) +* CancelledError: Exception classes. (line 6) +* CancelledError <1>: Exceptions<9>. (line 17) +* cancel_dump_traceback_later() (in module faulthandler): Dumping the tracebacks after a timeout. + (line 29) +* cancel_join_thread() (multiprocessing.Queue method): Pipes and Queues. + (line 182) +* CannotSendHeader: http client — HTTP protocol client. + (line 166) +* CannotSendRequest: http client — HTTP protocol client. + (line 162) +* canonic() (bdb.Bdb method): bdb — Debugger framework. + (line 99) +* canonical() (decimal.Context method): Context objects. (line 238) +* canonical() (decimal.Decimal method): Decimal objects. (line 141) +* canonicalize() (in module xml.etree.ElementTree): Functions<6>. + (line 6) +* CAN_BCM (in module socket): Constants<8>. (line 107) +* can_change_color() (in module curses): Functions<3>. (line 30) +* can_fetch() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 33) +* CAN_ISOTP (in module socket): Constants<8>. (line 135) +* CAN_RAW_FD_FRAMES (in module socket): Constants<8>. (line 122) +* can_symlink() (in module test.support): test support — Utilities for the Python test suite. + (line 546) +* can_write_eof() (asyncio.StreamWriter method): StreamWriter. + (line 47) +* can_write_eof() (asyncio.WriteTransport method): Write-only Transports. + (line 14) +* can_xattr() (in module test.support): test support — Utilities for the Python test suite. + (line 550) +* capa() (poplib.POP3 method): POP3 Objects. (line 24) +* capitalize() (bytearray method): Bytes and Bytearray Operations. + (line 403) +* capitalize() (bytes method): Bytes and Bytearray Operations. + (line 403) +* capitalize() (str method): String Methods<2>. (line 22) +* captured_stderr() (in module test.support): test support — Utilities for the Python test suite. + (line 402) +* captured_stdin() (in module test.support): test support — Utilities for the Python test suite. + (line 402) +* captured_stdout() (in module test.support): test support — Utilities for the Python test suite. + (line 402) +* captureWarnings() (in module logging): Integration with the warnings module. + (line 9) +* capwords() (in module string): Helper functions. (line 6) +* casefold() (str method): String Methods<2>. (line 32) +* cast() (in module ctypes): Utility functions. (line 33) +* cast() (in module typing): Classes functions and decorators. + (line 618) +* cast() (memoryview method): Memory Views. (line 269) +* cat() (in module nis): nis — Interface to Sun’s NIS Yellow Pages. + (line 29) +* catch_threading_exception() (in module test.support): test support — Utilities for the Python test suite. + (line 826) +* catch_unraisable_exception() (in module test.support): test support — Utilities for the Python test suite. + (line 861) +* catch_warnings (class in warnings): Available Context Managers. + (line 6) +* category() (in module unicodedata): unicodedata — Unicode Database. + (line 49) +* cbreak() (in module curses): Functions<3>. (line 35) +* CC: New Improved and Deprecated Modules<4>. + (line 58) +* ccc() (ftplib.FTP_TLS method): FTP_TLS Objects. (line 23) +* CCompiler (class in distutils.ccompiler): distutils ccompiler — CCompiler base class. + (line 67) +* cdf() (statistics.NormalDist method): NormalDist objects. (line 80) +* CDLL (class in ctypes): Loading shared libraries. + (line 9) +* ceil() (in module math): Numeric Types — int float complex. + (line 104) +* ceil() (in module math) <1>: Number-theoretic and representation functions. + (line 6) +* CellType (in module types): Standard Interpreter Types. + (line 70) +* center() (bytearray method): Bytes and Bytearray Operations. + (line 236) +* center() (bytes method): Bytes and Bytearray Operations. + (line 236) +* center() (str method): String Methods<2>. (line 48) +* CertificateError: Exceptions<12>. (line 94) +* certificates: SSL Contexts. (line 669) +* CERT_NONE (in module ssl): Constants<9>. (line 11) +* CERT_OPTIONAL (in module ssl): Constants<9>. (line 25) +* CERT_REQUIRED (in module ssl): Constants<9>. (line 43) +* cert_store_stats() (ssl.SSLContext method): SSL Contexts. (line 67) +* cert_time_to_seconds() (in module ssl): Certificate handling. + (line 48) +* CFLAGS: New Improved and Deprecated Modules<4>. + (line 58) +* CFLAGS <1>: Tweaking compiler/linker flags. + (line 66) +* CFLAGS <2>: Tweaking compiler/linker flags. + (line 67) +* CFUNCTYPE() (in module ctypes): Function prototypes. + (line 15) +* cgi (module): cgi — Common Gateway Interface support. + (line 6) +* CGI; debugging: Debugging CGI scripts. + (line 6) +* CGI; exceptions: cgitb — Traceback manager for CGI scripts. + (line 8) +* CGI; protocol: cgi — Common Gateway Interface support. + (line 8) +* CGI; security: Caring about security. + (line 6) +* CGI; tracebacks: cgitb — Traceback manager for CGI scripts. + (line 8) +* CGIHandler (class in wsgiref.handlers): wsgiref handlers – server/gateway base classes. + (line 11) +* CGIHTTPRequestHandler (class in http.server): http server — HTTP servers. + (line 450) +* cgitb (module): cgitb — Traceback manager for CGI scripts. + (line 6) +* CGIXMLRPCRequestHandler (class in xmlrpc.server): xmlrpc server — Basic XML-RPC servers. + (line 45) +* cgi_directories (http.server.CGIHTTPRequestHandler attribute): http server — HTTP servers. + (line 476) +* chain() (in module itertools): Itertool functions. (line 85) +* chaining; comparisons: Comparisons. (line 17) +* chaining; comparisons <1>: Comparisons<2>. (line 6) +* ChainMap (class in collections): ChainMap objects. (line 16) +* ChainMap (class in typing): Classes functions and decorators. + (line 382) +* change_cwd() (in module test.support): test support — Utilities for the Python test suite. + (line 436) +* change_root() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 64) +* channels() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 133) +* CHANNEL_BINDING_TYPES (in module ssl): Constants<9>. (line 433) +* channel_class (smtpd.SMTPServer attribute): SMTPServer Objects. + (line 80) +* character: The standard type hierarchy. + (line 156) +* character <1>: Subscriptions. (line 38) +* character <2>: unicodedata — Unicode Database. + (line 6) +* CharacterDataHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 240) +* characters() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 129) +* characters_written (BlockingIOError attribute): OS exceptions. + (line 18) +* Charset (class in email.charset): email charset Representing character sets. + (line 24) +* charset() (gettext.NullTranslations method): The NullTranslations class. + (line 83) +* CHAR_MAX (in module locale): locale — Internationalization services. + (line 504) +* chdir() (in module os): Files and Directories. + (line 115) +* check (lzma.LZMADecompressor attribute): Compressing and decompressing data in memory. + (line 159) +* check() (imaplib.IMAP4 method): IMAP4 Objects. (line 58) +* check() (in module tabnanny): tabnanny — Detection of ambiguous indentation. + (line 17) +* checkbox() (msilib.Dialog method): GUI classes. (line 78) +* checkcache() (in module linecache): linecache — Random access to text lines. + (line 43) +* CHECKED_HASH (py_compile.PycInvalidationMode attribute): py_compile — Compile Python source files. + (line 108) +* checkfuncname() (in module bdb): bdb — Debugger framework. + (line 386) +* CheckList (class in tkinter.tix): Hierarchical ListBox. + (line 13) +* checksizeof() (in module test.support): test support — Utilities for the Python test suite. + (line 541) +* checksum; Cyclic Redundancy Check: zlib — Compression compatible with gzip. + (line 117) +* check_call() (in module subprocess): Older high-level API. + (line 36) +* check_environ() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 71) +* check_free_after_iterating() (in module test.support): test support — Utilities for the Python test suite. + (line 956) +* check_hostname (ssl.SSLContext attribute): SSL Contexts. (line 492) +* check_impl_detail() (in module test.support): test support — Utilities for the Python test suite. + (line 287) +* check_no_resource_warning() (in module test.support): test support — Utilities for the Python test suite. + (line 365) +* check_output() (doctest.OutputChecker method): OutputChecker objects. + (line 17) +* check_output() (in module subprocess): Older high-level API. + (line 64) +* check_returncode() (subprocess.CompletedProcess method): Using the subprocess Module. + (line 125) +* check_syntax_error() (in module test.support): test support — Utilities for the Python test suite. + (line 686) +* check_syntax_warning() (in module test.support): test support — Utilities for the Python test suite. + (line 697) +* check_unused_args() (string.Formatter method): Custom String Formatting. + (line 91) +* check_warnings() (in module test.support): test support — Utilities for the Python test suite. + (line 296) +* check__all__() (in module test.support): test support — Utilities for the Python test suite. + (line 968) +* chflags() (in module os): Files and Directories. + (line 134) +* chgat() (curses.window method): Window Objects. (line 136) +* childNodes (xml.dom.Node attribute): Node Objects. (line 49) +* ChildProcessError: OS exceptions. (line 24) +* children (pyclbr.Class attribute): Class Objects<2>. (line 31) +* children (pyclbr.Function attribute): Function Objects. (line 31) +* chmod() (in module os): Files and Directories. + (line 175) +* chmod() (pathlib.Path method): Methods<2>. (line 47) +* choice() (in module random): Functions for sequences. + (line 6) +* choice() (in module secrets): Random numbers. (line 15) +* choices (optparse.Option attribute): Option attributes. (line 55) +* choices() (in module random): Functions for sequences. + (line 11) +* chown() (in module os): Files and Directories. + (line 237) +* chown() (in module shutil): Directory and files operations. + (line 354) +* chr() (built-in function): Built-in Functions. (line 205) +* chroot() (in module os): Files and Directories. + (line 260) +* Chunk (class in chunk): chunk — Read IFF chunked data. + (line 49) +* chunk (module): chunk — Read IFF chunked data. + (line 6) +* cipher() (ssl.SSLSocket method): SSL Sockets. (line 209) +* cipher; DES: crypt — Function to check Unix passwords. + (line 8) +* circle() (in module turtle): Turtle motion. (line 177) +* CIRCUMFLEX (in module token): token — Constants used with Python parse trees. + (line 154) +* CIRCUMFLEXEQUAL (in module token): token — Constants used with Python parse trees. + (line 198) +* Clamped (class in decimal): Signals. (line 20) +* class: Glossary. (line 216) +* Class (class in symtable): Examining Symbol Tables. + (line 93) +* Class browser: File menu Shell and Editor. + (line 22) +* class instance; attribute: The standard type hierarchy. + (line 600) +* class instance; attribute; assignment: The standard type hierarchy. + (line 616) +* class instance; call: Calls. (line 141) +* class object; call: The standard type hierarchy. + (line 573) +* class object; call <1>: The standard type hierarchy. + (line 585) +* class object; call <2>: Calls. (line 137) +* class variable: Glossary. (line 222) +* class; attribute: The standard type hierarchy. + (line 573) +* class; attribute; assignment: The standard type hierarchy. + (line 582) +* class; body: Executing the class body. + (line 6) +* class; constructor: Basic customization. + (line 40) +* class; definition: The return statement. + (line 6) +* class; definition <1>: Class definitions. (line 6) +* class; instance: The standard type hierarchy. + (line 600) +* class; name: Class definitions. (line 6) +* classmethod() (built-in function): Built-in Functions. (line 216) +* ClassMethodDescriptorType (in module types): Standard Interpreter Types. + (line 110) +* ClassVar (in module typing): Classes functions and decorators. + (line 903) +* clause: Compound statements. + (line 18) +* CLD_CONTINUED (in module os): Process Management. (line 758) +* CLD_DUMPED (in module os): Process Management. (line 758) +* CLD_EXITED (in module os): Process Management. (line 758) +* CLD_TRAPPED (in module os): Process Management. (line 758) +* clean() (mailbox.Maildir method): Maildir. (line 82) +* cleandoc() (in module inspect): Retrieving source code. + (line 65) +* CleanImport (class in test.support): test support — Utilities for the Python test suite. + (line 1060) +* cleanup functions: Process Control. (line 26) +* clear (pdb command): Debugger Commands. (line 93) +* Clear Breakpoint: Help menu Shell and Editor. + (line 30) +* clear() (asyncio.Event method): Event. (line 59) +* clear() (collections.deque method): deque objects. (line 43) +* clear() (curses.window method): Window Objects. (line 152) +* clear() (dict method): Mapping Types — dict. + (line 142) +* clear() (email.message.EmailMessage method): email message Representing an email message. + (line 668) +* clear() (frame method): The standard type hierarchy. + (line 741) +* clear() (frozenset method): Set Types — set frozenset. + (line 213) +* clear() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 77) +* clear() (in module turtle): Window control. (line 39) +* clear() (mailbox.Mailbox method): Mailbox objects. (line 204) +* clear() (sequence method): Mutable Sequence Types. + (line 16) +* clear() (threading.Event method): Event Objects. (line 33) +* clear() (xml.etree.ElementTree.Element method): Element Objects. + (line 58) +* clearcache() (in module linecache): linecache — Random access to text lines. + (line 38) +* ClearData() (msilib.Record method): Record Objects. (line 37) +* clearok() (curses.window method): Window Objects. (line 157) +* clearscreen() (in module turtle): Window control. (line 39) +* clearstamp() (in module turtle): Turtle motion. (line 249) +* clearstamps() (in module turtle): Turtle motion. (line 268) +* clear_all_breaks() (bdb.Bdb method): bdb — Debugger framework. + (line 305) +* clear_all_file_breaks() (bdb.Bdb method): bdb — Debugger framework. + (line 300) +* clear_bpbynumber() (bdb.Bdb method): bdb — Debugger framework. + (line 294) +* clear_break() (bdb.Bdb method): bdb — Debugger framework. + (line 289) +* clear_cache() (in module filecmp): filecmp — File and Directory Comparisons. + (line 53) +* clear_content() (email.message.EmailMessage method): email message Representing an email message. + (line 672) +* clear_flags() (decimal.Context method): Context objects. (line 152) +* clear_frames() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 150) +* clear_history() (in module readline): History list. (line 8) +* clear_session_cookies() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 89) +* clear_traces() (in module tracemalloc): Functions<9>. (line 6) +* clear_traps() (decimal.Context method): Context objects. (line 156) +* Client() (in module multiprocessing.connection): Listeners and Clients. + (line 34) +* client_address (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 66) +* CLOCK_BOOTTIME (in module time): Clock ID Constants. (line 9) +* clock_getres() (in module time): Functions<2>. (line 38) +* clock_gettime() (in module time): Functions<2>. (line 48) +* clock_gettime_ns() (in module time): Functions<2>. (line 58) +* CLOCK_HIGHRES (in module time): Clock ID Constants. (line 23) +* CLOCK_MONOTONIC (in module time): Clock ID Constants. (line 34) +* CLOCK_MONOTONIC_RAW (in module time): Clock ID Constants. (line 43) +* CLOCK_PROCESS_CPUTIME_ID (in module time): Clock ID Constants. + (line 53) +* CLOCK_PROF (in module time): Clock ID Constants. (line 61) +* CLOCK_REALTIME (in module time): Clock ID Constants. (line 100) +* clock_settime() (in module time): Functions<2>. (line 67) +* clock_settime_ns() (in module time): Functions<2>. (line 76) +* CLOCK_THREAD_CPUTIME_ID (in module time): Clock ID Constants. + (line 69) +* CLOCK_UPTIME (in module time): Clock ID Constants. (line 77) +* CLOCK_UPTIME_RAW (in module time): Clock ID Constants. (line 87) +* clone() (email.generator.BytesGenerator method): email generator Generating MIME documents. + (line 112) +* clone() (email.generator.Generator method): email generator Generating MIME documents. + (line 208) +* clone() (email.policy.Policy method): email policy Policy Objects. + (line 207) +* clone() (in module turtle): Special Turtle methods. + (line 32) +* clone() (pipes.Template method): Template Objects. (line 12) +* cloneNode() (xml.dom.Node method): Node Objects. (line 147) +* close() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 116) +* close() (asyncio.AbstractChildWatcher method): Process Watchers. + (line 77) +* close() (asyncio.BaseTransport method): Base Transport. (line 6) +* close() (asyncio.loop method): Running and stopping the loop. + (line 43) +* close() (asyncio.Server method): Server Objects. (line 29) +* close() (asyncio.StreamWriter method): StreamWriter. (line 37) +* close() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 59) +* close() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 243) +* close() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 78) +* close() (contextlib.ExitStack method): Utilities. (line 446) +* close() (coroutine method): Coroutine Objects. (line 45) +* close() (dbm.dumb.dumbdbm method): dbm dumb — Portable DBM implementation. + (line 83) +* close() (dbm.gnu.gdbm method): dbm gnu — GNU’s reinterpretation of dbm. + (line 118) +* close() (dbm.ndbm.ndbm method): dbm ndbm — Interface based on ndbm. + (line 66) +* close() (distutils.text_file.TextFile method): distutils text_file — The TextFile class. + (line 96) +* close() (email.parser.BytesFeedParser method): FeedParser API. + (line 65) +* close() (ftplib.FTP method): FTP Objects. (line 246) +* close() (generator method): Generator-iterator methods. + (line 47) +* close() (html.parser.HTMLParser method): HTMLParser Methods. + (line 15) +* close() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 111) +* close() (imaplib.IMAP4 method): IMAP4 Objects. (line 62) +* close() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 127) +* close() (in module os): File Descriptor Operations. + (line 21) +* close() (in module os) <1>: Sub-interpreter support. + (line 76) +* close() (in module socket): Other functions<2>. (line 9) +* close() (io.IOBase method): I/O Base Classes. (line 49) +* close() (logging.FileHandler method): FileHandler. (line 23) +* close() (logging.Handler method): Handler Objects. (line 73) +* close() (logging.handlers.MemoryHandler method): MemoryHandler. + (line 55) +* close() (logging.handlers.NTEventLogHandler method): NTEventLogHandler. + (line 29) +* close() (logging.handlers.SocketHandler method): SocketHandler. + (line 20) +* close() (logging.handlers.SysLogHandler method): SysLogHandler. + (line 39) +* close() (mailbox.Mailbox method): Mailbox objects. (line 262) +* close() (mailbox.Maildir method): Maildir. (line 113) +* close() (mailbox.MH method): MH. (line 104) +* close() (mmap.mmap method): mmap — Memory-mapped file support. + (line 165) +* Close() (msilib.Database method): Database Objects. (line 22) +* Close() (msilib.View method): View Objects. (line 35) +* close() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 35) +* close() (multiprocessing.connection.Listener method): Listeners and Clients. + (line 91) +* close() (multiprocessing.pool.Pool method): Process Pools. (line 168) +* close() (multiprocessing.Process method): Process and exceptions. + (line 173) +* close() (multiprocessing.Queue method): Pipes and Queues. (line 163) +* close() (multiprocessing.shared_memory.SharedMemory method): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 65) +* close() (os.scandir method): Files and Directories. + (line 783) +* close() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 22) +* close() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 8) +* close() (select.devpoll method): /dev/poll Polling Objects. + (line 12) +* close() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 65) +* close() (select.kqueue method): Kqueue Objects. (line 6) +* close() (selectors.BaseSelector method): Classes<3>. (line 137) +* close() (shelve.Shelf method): shelve — Python object persistence. + (line 71) +* close() (socket.socket method): Socket Objects. (line 39) +* close() (sqlite3.Connection method): Connection Objects. (line 45) +* close() (sqlite3.Cursor method): Cursor Objects. (line 169) +* close() (sunau.AU_read method): AU_read Objects. (line 9) +* close() (sunau.AU_write method): AU_write Objects. (line 59) +* close() (tarfile.TarFile method): TarFile Objects. (line 248) +* close() (telnetlib.Telnet method): Telnet Objects. (line 89) +* close() (urllib.request.BaseHandler method): BaseHandler Objects. + (line 14) +* close() (wave.Wave_read method): Wave_read Objects. (line 9) +* close() (wave.Wave_write method): Wave_write Objects. (line 23) +* Close() (winreg.PyHKEY method): Registry Handle Objects. + (line 34) +* close() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 28) +* close() (xml.etree.ElementTree.XMLParser method): XMLParser Objects. + (line 20) +* close() (xml.etree.ElementTree.XMLPullParser method): XMLPullParser Objects. + (line 22) +* close() (xml.sax.xmlreader.IncrementalParser method): IncrementalParser Objects. + (line 13) +* close() (zipfile.ZipFile method): ZipFile Objects. (line 86) +* closed (http.client.HTTPResponse attribute): HTTPResponse Objects. + (line 65) +* closed (io.IOBase attribute): I/O Base Classes. (line 59) +* closed (mmap.mmap attribute): mmap — Memory-mapped file support. + (line 171) +* closed (ossaudiodev.oss_audio_device attribute): Audio Device Objects. + (line 230) +* closed (select.devpoll attribute): /dev/poll Polling Objects. + (line 18) +* closed (select.epoll attribute): Edge and Level Trigger Polling epoll Objects. + (line 69) +* closed (select.kqueue attribute): Kqueue Objects. (line 10) +* CloseKey() (in module winreg): Functions<11>. (line 8) +* closelog() (in module syslog): syslog — Unix syslog library routines. + (line 59) +* closerange() (in module os): File Descriptor Operations. + (line 32) +* close_connection (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 75) +* close_when_done() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 73) +* closing() (in module contextlib): Utilities. (line 121) +* clrtobot() (curses.window method): Window Objects. (line 162) +* clrtoeol() (curses.window method): Window Objects. (line 168) +* cmath (module): cmath — Mathematical functions for complex numbers. + (line 6) +* Cmd (class in cmd): cmd — Support for line-oriented command interpreters. + (line 15) +* cmd (module): cmd — Support for line-oriented command interpreters. + (line 6) +* cmd (subprocess.CalledProcessError attribute): Using the subprocess Module. + (line 201) +* cmd (subprocess.TimeoutExpired attribute): Using the subprocess Module. + (line 164) +* cmdloop() (cmd.Cmd method): Cmd Objects. (line 8) +* cmdqueue (cmd.Cmd attribute): Cmd Objects. (line 135) +* cmp() (in module filecmp): filecmp — File and Directory Comparisons. + (line 16) +* cmpfiles() (in module filecmp): filecmp — File and Directory Comparisons. + (line 33) +* cmp_op (in module dis): Opcode collections. (line 17) +* cmp_to_key() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 48) +* CMSG_LEN() (in module socket): Other functions<2>. (line 297) +* CMSG_SPACE() (in module socket): Other functions<2>. (line 312) +* code (module): code — Interpreter base classes. + (line 6) +* code (SystemExit attribute): Concrete exceptions. + (line 304) +* code (urllib.error.HTTPError attribute): urllib error — Exception classes raised by urllib request. + (line 38) +* code (xml.etree.ElementTree.ParseError attribute): Exceptions<15>. + (line 14) +* code (xml.parsers.expat.ExpatError attribute): ExpatError Exceptions. + (line 9) +* code object: The standard type hierarchy. + (line 649) +* code object <1>: Methods. (line 42) +* code object <2>: marshal — Internal Python object serialization. + (line 30) +* code object <3>: Cell Objects. (line 52) +* code; block: Execution model. (line 6) +* CodecInfo (class in codecs): codecs — Codec registry and base classes. + (line 58) +* Codecs: codecs — Codec registry and base classes. + (line 8) +* codecs (module): codecs — Codec registry and base classes. + (line 6) +* Codecs; decode: codecs — Codec registry and base classes. + (line 8) +* Codecs; encode: codecs — Codec registry and base classes. + (line 8) +* coded_value (http.cookies.Morsel attribute): Morsel Objects. + (line 56) +* codeop (module): codeop — Compile Python code. + (line 6) +* codepoint2name (in module html.entities): html entities — Definitions of HTML general entities. + (line 35) +* codes (in module xml.parsers.expat.errors): Expat error constants. + (line 16) +* CODESET (in module locale): locale — Internationalization services. + (line 184) +* CodeType (class in types): Standard Interpreter Types. + (line 50) +* code_info() (in module dis): Analysis functions. (line 11) +* coding; style: Intermezzo Coding Style. + (line 6) +* coercion: Glossary. (line 227) +* collapse_addresses() (in module ipaddress): Other Module Level Functions. + (line 43) +* collapse_rfc2231_value() (in module email.utils): email utils Miscellaneous utilities. + (line 187) +* collect() (in module gc): gc — Garbage Collector interface. + (line 34) +* Collection (class in collections.abc): Collections Abstract Base Classes. + (line 131) +* Collection (class in typing): Classes functions and decorators. + (line 203) +* collections (module): collections — Container datatypes. + (line 6) +* collections.abc (module): collections abc — Abstract Base Classes for Containers. + (line 6) +* collect_incoming_data() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 78) +* colno (json.JSONDecodeError attribute): Exceptions<13>. (line 27) +* colno (re.error attribute): Module Contents. (line 379) +* COLON (in module token): token — Constants used with Python parse trees. + (line 70) +* COLONEQUAL (in module token): token — Constants used with Python parse trees. + (line 238) +* color() (in module turtle): Color control. (line 104) +* colormode() (in module turtle): Settings and special methods. + (line 34) +* colorsys (module): colorsys — Conversions between color systems. + (line 6) +* color_content() (in module curses): Functions<3>. (line 45) +* color_pair() (in module curses): Functions<3>. (line 53) +* COLS: curses<2>. (line 7) +* COLS <1>: Functions<3>. (line 526) +* column() (tkinter.ttk.Treeview method): ttk Treeview. (line 35) +* COLUMNS: Functions<3>. (line 550) +* COLUMNS <1>: Functions<3>. (line 553) +* columns (os.terminal_size attribute): Querying the size of a terminal. + (line 30) +* col_offset (ast.AST attribute): Node classes. (line 39) +* comb() (in module math): Number-theoretic and representation functions. + (line 12) +* combinations() (in module itertools): Itertool functions. (line 110) +* combinations_with_replacement() (in module itertools): Itertool functions. + (line 161) +* combine() (datetime.datetime class method): datetime Objects. + (line 170) +* combining() (in module unicodedata): unicodedata — Unicode Database. + (line 59) +* ComboBox (class in tkinter.tix): Basic Widgets. (line 18) +* Combobox (class in tkinter.ttk): ttk Combobox. (line 6) +* comma: Parenthesized forms. + (line 20) +* COMMA (in module token): token — Constants used with Python parse trees. + (line 74) +* Command (class in distutils.cmd): distutils cmd — Abstract base class for Distutils commands. + (line 8) +* Command (class in distutils.core): distutils core — Core Distutils functionality. + (line 307) +* command (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 88) +* command line: Complete Python programs. + (line 25) +* command line option; -?: Generic options. (line 6) +* command line option; -b: Miscellaneous options. + (line 6) +* command line option; -B: Miscellaneous options. + (line 16) +* command line option; -c : Interface options. (line 36) +* command line option; –check-hash-based-pycs default|always|never: Miscellaneous options. + (line 21) +* command line option; -d: Miscellaneous options. + (line 35) +* command line option; -E: Miscellaneous options. + (line 40) +* command line option; -h: Generic options. (line 6) +* command line option; –help: Generic options. (line 6) +* command line option; -i: Miscellaneous options. + (line 45) +* command line option; -I: Miscellaneous options. + (line 56) +* command line option; -J: Options you shouldn’t use. + (line 6) +* command line option; -m : Interface options. (line 50) +* command line option; -O: Miscellaneous options. + (line 66) +* command line option; -OO: Miscellaneous options. + (line 76) +* command line option; -q: Miscellaneous options. + (line 85) +* command line option; -R: Miscellaneous options. + (line 92) +* command line option; -s: Miscellaneous options. + (line 117) +* command line option; -S: Miscellaneous options. + (line 127) +* command line option; -u: Miscellaneous options. + (line 135) +* command line option; -V: Generic options. (line 12) +* command line option; -v: Miscellaneous options. + (line 145) +* command line option; –version: Generic options. (line 12) +* command line option; -W arg: Miscellaneous options. + (line 152) +* command line option; -x: Miscellaneous options. + (line 191) +* command line option; -X: Miscellaneous options. + (line 196) +* CommandCompiler (class in codeop): codeop — Compile Python code. + (line 64) +* commands (pdb command): Debugger Commands. (line 126) +* comment: Comments. (line 6) +* comment (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 58) +* COMMENT (in module token): token — Constants used with Python parse trees. + (line 261) +* comment (zipfile.ZipFile attribute): ZipFile Objects. (line 308) +* comment (zipfile.ZipInfo attribute): ZipInfo Objects. (line 88) +* Comment() (in module xml.etree.ElementTree): Functions<6>. (line 72) +* comment() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 49) +* commenters (shlex.shlex attribute): shlex Objects. (line 85) +* CommentHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 295) +* comment_url (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 63) +* commit() (msilib.CAB method): CAB Objects. (line 24) +* Commit() (msilib.Database method): Database Objects. (line 11) +* commit() (sqlite3.Connection method): Connection Objects. (line 32) +* common (filecmp.dircmp attribute): The dircmp class. (line 59) +* Common Gateway Interface: cgi — Common Gateway Interface support. + (line 8) +* commonpath() (in module os.path): os path — Common pathname manipulations. + (line 73) +* commonprefix() (in module os.path): os path — Common pathname manipulations. + (line 88) +* common_dirs (filecmp.dircmp attribute): The dircmp class. (line 71) +* common_files (filecmp.dircmp attribute): The dircmp class. (line 75) +* common_funny (filecmp.dircmp attribute): The dircmp class. (line 79) +* common_types (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 144) +* communicate() (asyncio.asyncio.subprocess.Process method): Interacting with Subprocesses. + (line 50) +* communicate() (subprocess.Popen method): Popen Objects. (line 35) +* compare() (decimal.Context method): Context objects. (line 242) +* compare() (decimal.Decimal method): Decimal objects. (line 147) +* compare() (difflib.Differ method): Differ Objects. (line 38) +* compare_digest() (in module hmac): hmac — Keyed-Hashing for Message Authentication. + (line 109) +* compare_digest() (in module secrets): Other functions. (line 6) +* compare_networks() (ipaddress.IPv4Network method): Network objects. + (line 234) +* compare_networks() (ipaddress.IPv6Network method): Network objects. + (line 348) +* COMPARE_OP (opcode): Python Bytecode Instructions. + (line 630) +* compare_signal() (decimal.Context method): Context objects. + (line 246) +* compare_signal() (decimal.Decimal method): Decimal objects. + (line 158) +* compare_to() (tracemalloc.Snapshot method): Snapshot. (line 13) +* compare_total() (decimal.Context method): Context objects. (line 250) +* compare_total() (decimal.Decimal method): Decimal objects. (line 165) +* compare_total_mag() (decimal.Context method): Context objects. + (line 254) +* compare_total_mag() (decimal.Decimal method): Decimal objects. + (line 190) +* comparison: Comparisons. (line 6) +* comparisons: Basic customization. + (line 180) +* COMPARISON_FLAGS (in module doctest): Option Flags. (line 115) +* Compat32 (class in email.policy): email policy Policy Objects. + (line 547) +* compat32 (in module email.policy): email policy Policy Objects. + (line 602) +* Compile (class in codeop): codeop — Compile Python code. + (line 55) +* compile() (built-in function): Built-in Functions. (line 243) +* compile() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 269) +* compile() (in module py_compile): py_compile — Compile Python source files. + (line 24) +* compile() (in module re): Module Contents. (line 14) +* compile() (parser.ST method): ST Objects. (line 17) +* compileall (module): compileall — Byte-compile Python libraries. + (line 6) +* compileall command line option; -b: Command-line use. (line 52) +* compileall command line option; -d destdir: Command-line use. + (line 32) +* compileall command line option; -f: Command-line use. (line 22) +* compileall command line option; -i list: Command-line use. (line 46) +* compileall command line option; –invalidation-mode [timestamp|checked-hash|unchecked-hash]: Command-line use. + (line 73) +* compileall command line option; -j N: Command-line use. (line 67) +* compileall command line option; -l: Command-line use. (line 17) +* compileall command line option; -q: Command-line use. (line 26) +* compileall command line option; -r: Command-line use. (line 60) +* compileall command line option; -x regex: Command-line use. + (line 40) +* compileall command line option; directory ...: Command-line use. + (line 9) +* compileall command line option; file ...: Command-line use. + (line 9) +* compilest() (in module parser): Converting ST Objects. + (line 43) +* compile_command() (in module code): code — Interpreter base classes. + (line 47) +* compile_command() (in module codeop): codeop — Compile Python code. + (line 29) +* compile_dir() (in module compileall): Public functions. (line 6) +* compile_file() (in module compileall): Public functions. (line 78) +* compile_path() (in module compileall): Public functions. (line 127) +* complete() (rlcompleter.Completer method): Completer Objects. + (line 8) +* completedefault() (cmd.Cmd method): Cmd Objects. (line 80) +* CompletedProcess (class in subprocess): Using the subprocess Module. + (line 91) +* complete_statement() (in module sqlite3): Module functions and constants. + (line 147) +* complex (built-in class): Built-in Functions. (line 326) +* Complex (class in numbers): The numeric tower. (line 6) +* complex literal: Numeric literals. (line 6) +* complex number: Glossary. (line 239) +* complex number; literals: Numeric Types — int float complex. + (line 19) +* complex; number: The standard type hierarchy. + (line 119) +* compound; statement: Compound statements. + (line 6) +* comprehensions: Displays for lists sets and dictionaries. + (line 6) +* compress() (bz2.BZ2Compressor method): Incremental de compression. + (line 15) +* compress() (in module bz2): One-shot de compression. + (line 6) +* compress() (in module gzip): gzip — Support for gzip files. + (line 172) +* compress() (in module itertools): Itertool functions. (line 212) +* compress() (in module lzma): Compressing and decompressing data in memory. + (line 183) +* compress() (in module zlib): zlib — Compression compatible with gzip. + (line 45) +* compress() (lzma.LZMACompressor method): Compressing and decompressing data in memory. + (line 81) +* compress() (zlib.Compress method): zlib — Compression compatible with gzip. + (line 199) +* compressed (ipaddress.IPv4Address attribute): Address objects. + (line 52) +* compressed (ipaddress.IPv4Network attribute): Network objects. + (line 107) +* compressed (ipaddress.IPv6Address attribute): Address objects. + (line 147) +* compressed (ipaddress.IPv6Network attribute): Network objects. + (line 316) +* compression() (ssl.SSLSocket method): SSL Sockets. (line 227) +* CompressionError: tarfile — Read and write tar archive files. + (line 191) +* compressobj() (in module zlib): zlib — Compression compatible with gzip. + (line 60) +* compress_size (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 139) +* compress_type (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 84) +* COMSPEC: Process Management. (line 664) +* COMSPEC <1>: Popen Constructor. (line 90) +* concat() (in module operator): operator — Standard operators as functions. + (line 172) +* concatenation; operation: Common Sequence Operations. + (line 21) +* concurrent.futures (module): concurrent futures — Launching parallel tasks. + (line 6) +* Condition (class in asyncio): Condition. (line 6) +* Condition (class in multiprocessing): Synchronization primitives. + (line 32) +* Condition (class in threading): Condition Objects. (line 72) +* condition (pdb command): Debugger Commands. (line 119) +* condition() (msilib.Control method): GUI classes. (line 24) +* Condition() (multiprocessing.managers.SyncManager method): Managers. + (line 155) +* Conditional; expression: Boolean operations. (line 6) +* conditional; expression: Conditional expressions. + (line 6) +* ConfigParser (class in configparser): ConfigParser Objects. + (line 6) +* configparser (module): configparser — Configuration file parser. + (line 6) +* configuration information: sysconfig — Provide access to Python’s configuration information. + (line 10) +* configuration; file: configparser — Configuration file parser. + (line 8) +* configure() (tkinter.ttk.Style method): Ttk Styling. (line 24) +* configure_mock() (unittest.mock.Mock method): The Mock Class. + (line 242) +* confstr() (in module os): Miscellaneous System Information. + (line 6) +* confstr_names (in module os): Miscellaneous System Information. + (line 28) +* conjugate() (complex number method): Numeric Types — int float complex. + (line 94) +* conjugate() (decimal.Decimal method): Decimal objects. (line 202) +* conjugate() (numbers.Complex method): The numeric tower. (line 23) +* conn (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 48) +* connect() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 194) +* connect() (ftplib.FTP method): FTP Objects. (line 22) +* connect() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 105) +* connect() (in module sqlite3): Module functions and constants. + (line 55) +* connect() (multiprocessing.managers.BaseManager method): Managers. + (line 59) +* connect() (smtplib.SMTP method): SMTP Objects. (line 33) +* connect() (socket.socket method): Socket Objects. (line 60) +* Connection (class in multiprocessing.connection): Connection Objects<2>. + (line 13) +* Connection (class in sqlite3): Connection Objects. (line 6) +* connection (sqlite3.Cursor attribute): Cursor Objects. (line 230) +* ConnectionAbortedError: OS exceptions. (line 44) +* ConnectionError: OS exceptions. (line 29) +* ConnectionRefusedError: OS exceptions. (line 50) +* ConnectionResetError: OS exceptions. (line 56) +* connection_lost() (asyncio.BaseProtocol method): Base Protocol. + (line 23) +* connection_made() (asyncio.BaseProtocol method): Base Protocol. + (line 15) +* ConnectRegistry() (in module winreg): Functions<11>. (line 17) +* connect_accepted_socket() (asyncio.loop method): Creating network servers. + (line 116) +* connect_ex() (socket.socket method): Socket Objects. (line 82) +* connect_read_pipe() (asyncio.loop method): Working with pipes. + (line 6) +* connect_write_pipe() (asyncio.loop method): Working with pipes. + (line 22) +* const (optparse.Option attribute): Option attributes. (line 50) +* constant: Literals. (line 6) +* constructor() (in module copyreg): copyreg — Register pickle support functions. + (line 17) +* consumed (asyncio.LimitOverrunError attribute): Exceptions<9>. + (line 65) +* container: Objects values and types. + (line 67) +* container <1>: The standard type hierarchy. + (line 573) +* Container (class in collections.abc): Collections Abstract Base Classes. + (line 104) +* Container (class in typing): Classes functions and decorators. + (line 191) +* container; iteration over: Iterator Types. (line 6) +* contains() (in module operator): operator — Standard operators as functions. + (line 177) +* ContentDispositionHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 280) +* ContentHandler (class in xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 18) +* ContentManager (class in email.contentmanager): email contentmanager Managing MIME Content. + (line 12) +* contents (ctypes._Pointer attribute): Arrays and pointers. + (line 50) +* contents() (importlib.abc.ResourceReader method): importlib abc – Abstract base classes related to import. + (line 340) +* contents() (in module importlib.resources): importlib resources – Resources. + (line 127) +* ContentTooShortError: urllib error — Exception classes raised by urllib request. + (line 56) +* ContentTransferEncoding (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 290) +* ContentTypeHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 267) +* content_disposition (email.headerregistry.ContentDispositionHeader attribute): email headerregistry Custom Header Objects. + (line 285) +* content_manager (email.policy.EmailPolicy attribute): email policy Policy Objects. + (line 402) +* content_type (email.headerregistry.ContentTypeHeader attribute): email headerregistry Custom Header Objects. + (line 272) +* Context (class in contextvars): Manual Context Management. + (line 20) +* Context (class in decimal): Context objects. (line 96) +* context (ssl.SSLSocket attribute): SSL Sockets. (line 318) +* context management protocol: Context Manager Types. + (line 6) +* context manager: With Statement Context Managers. + (line 14) +* context manager <1>: Context Manager Types. + (line 6) +* context manager <2>: Glossary. (line 253) +* context variable: Glossary. (line 259) +* ContextDecorator (class in contextlib): Utilities. (line 264) +* contextlib (module): contextlib — Utilities for with-statement contexts. + (line 6) +* ContextManager (class in typing): Classes functions and decorators. + (line 332) +* contextmanager() (in module contextlib): Utilities. (line 30) +* ContextVar (class in contextvars): Context Variables. (line 6) +* contextvars (module): contextvars — Context Variables. + (line 6) +* contextvars.Token (class in contextvars): Context Variables. + (line 75) +* context_diff() (in module difflib): difflib — Helpers for computing deltas. + (line 157) +* contiguous: shape strides suboffsets. + (line 26) +* contiguous <1>: Glossary. (line 269) +* contiguous (memoryview attribute): Memory Views. (line 482) +* continue (pdb command): Debugger Commands. (line 192) +* Control (class in msilib): GUI classes. (line 11) +* Control (class in tkinter.tix): Basic Widgets. (line 24) +* control() (msilib.Dialog method): GUI classes. (line 47) +* control() (select.kqueue method): Kqueue Objects. (line 22) +* controlnames (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 237) +* controls() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 23) +* ConversionError: Exceptions<6>. (line 13) +* convert_arg_line_to_args() (argparse.ArgumentParser method): Customizing file parsing. + (line 6) +* convert_field() (string.Formatter method): Custom String Formatting. + (line 108) +* convert_path() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 54) +* Cookie (class in http.cookiejar): http cookiejar — Cookie handling for HTTP clients. + (line 109) +* CookieError: http cookies — HTTP state management. + (line 33) +* CookieJar (class in http.cookiejar): http cookiejar — Cookie handling for HTTP clients. + (line 45) +* cookiejar (urllib.request.HTTPCookieProcessor attribute): HTTPCookieProcessor Objects. + (line 8) +* CookiePolicy (class in http.cookiejar): http cookiejar — Cookie handling for HTTP clients. + (line 73) +* Coordinated Universal Time: time — Time access and conversions. + (line 40) +* Copy: Help menu Shell and Editor. + (line 30) +* copy (module): copy — Shallow and deep copy operations. + (line 6) +* copy() (collections.deque method): deque objects. (line 47) +* copy() (contextvars.Context method): Manual Context Management. + (line 70) +* copy() (decimal.Context method): Context objects. (line 162) +* copy() (dict method): Mapping Types — dict. + (line 146) +* copy() (frozenset method): Set Types — set frozenset. + (line 123) +* copy() (hashlib.hash method): Hash algorithms. (line 141) +* copy() (hmac.HMAC method): hmac — Keyed-Hashing for Message Authentication. + (line 82) +* copy() (http.cookies.Morsel method): Morsel Objects. (line 102) +* copy() (imaplib.IMAP4 method): IMAP4 Objects. (line 68) +* copy() (in module copy): copy — Shallow and deep copy operations. + (line 18) +* copy() (in module multiprocessing.sharedctypes): The multiprocessing sharedctypes module. + (line 83) +* copy() (in module shutil): Directory and files operations. + (line 121) +* copy() (pipes.Template method): Template Objects. (line 48) +* copy() (sequence method): Mutable Sequence Types. + (line 16) +* copy() (types.MappingProxyType method): Standard Interpreter Types. + (line 245) +* copy() (zlib.Compress method): zlib — Compression compatible with gzip. + (line 220) +* copy() (zlib.Decompress method): zlib — Compression compatible with gzip. + (line 289) +* copy2() (in module shutil): Directory and files operations. + (line 152) +* copy; protocol: Pickling Class Instances. + (line 93) +* copyfile() (in module shutil): Directory and files operations. + (line 17) +* copyfileobj() (in module shutil): Directory and files operations. + (line 6) +* copying files: shutil — High-level file operations. + (line 8) +* copymode() (in module shutil): Directory and files operations. + (line 59) +* copyreg (module): copyreg — Register pickle support functions. + (line 6) +* copyright (built-in variable): Constants added by the site module. + (line 18) +* copyright (in module sys): sys — System-specific parameters and functions. + (line 153) +* copyright (in module sys) <1>: Process-wide parameters. + (line 188) +* copysign() (in module math): Number-theoretic and representation functions. + (line 30) +* copystat() (in module shutil): Directory and files operations. + (line 77) +* copytree() (in module shutil): Directory and files operations. + (line 191) +* copy_abs() (decimal.Context method): Context objects. (line 259) +* copy_abs() (decimal.Decimal method): Decimal objects. (line 207) +* copy_context() (in module contextvars): Manual Context Management. + (line 6) +* copy_decimal() (decimal.Context method): Context objects. (line 166) +* copy_file() (in module distutils.file_util): distutils file_util — Single file operations. + (line 9) +* copy_file_range() (in module os): File Descriptor Operations. + (line 44) +* copy_location() (in module ast): ast Helpers. (line 96) +* copy_negate() (decimal.Context method): Context objects. (line 263) +* copy_negate() (decimal.Decimal method): Decimal objects. (line 213) +* copy_sign() (decimal.Context method): Context objects. (line 267) +* copy_sign() (decimal.Decimal method): Decimal objects. (line 219) +* copy_tree() (in module distutils.dir_util): distutils dir_util — Directory tree operations. + (line 32) +* coroutine: Special method lookup. + (line 74) +* coroutine <1>: Yield expressions. (line 51) +* coroutine <2>: Glossary. (line 280) +* Coroutine (class in collections.abc): Collections Abstract Base Classes. + (line 212) +* Coroutine (class in typing): Classes functions and decorators. + (line 304) +* coroutine function: Glossary. (line 288) +* coroutine() (in module asyncio): Generator-based Coroutines. + (line 16) +* coroutine() (in module types): Coroutine Utility Functions. + (line 6) +* coroutine; function: The standard type hierarchy. + (line 458) +* CoroutineType (in module types): Standard Interpreter Types. + (line 36) +* cos() (in module cmath): Trigonometric functions<2>. + (line 25) +* cos() (in module math): Trigonometric functions. + (line 28) +* cosh() (in module cmath): Hyperbolic functions<2>. + (line 26) +* cosh() (in module math): Hyperbolic functions. + (line 21) +* count (tracemalloc.Statistic attribute): Statistic. (line 15) +* count (tracemalloc.StatisticDiff attribute): StatisticDiff. + (line 15) +* count() (array.array method): array — Efficient arrays of numeric values. + (line 144) +* count() (bytearray method): Bytes and Bytearray Operations. + (line 36) +* count() (bytes method): Bytes and Bytearray Operations. + (line 36) +* count() (collections.deque method): deque objects. (line 53) +* count() (in module itertools): Itertool functions. (line 225) +* count() (multiprocessing.shared_memory.ShareableList method): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 264) +* count() (sequence method): Common Sequence Operations. + (line 21) +* count() (str method): String Methods<2>. (line 55) +* Counter (class in collections): Counter objects. (line 23) +* Counter (class in typing): Classes functions and decorators. + (line 374) +* countOf() (in module operator): operator — Standard operators as functions. + (line 183) +* countTestCases() (unittest.TestCase method): Test cases. (line 712) +* countTestCases() (unittest.TestSuite method): Grouping tests. + (line 55) +* count_diff (tracemalloc.StatisticDiff attribute): StatisticDiff. + (line 20) +* CoverageResults (class in trace): Programmatic Interface. + (line 47) +* co_argcount (code object attribute): The standard type hierarchy. + (line 662) +* CO_ASYNC_GENERATOR (in module inspect): Code Objects Bit Flags. + (line 56) +* co_cellvars (code object attribute): The standard type hierarchy. + (line 662) +* co_code (code object attribute): The standard type hierarchy. + (line 662) +* co_consts (code object attribute): The standard type hierarchy. + (line 662) +* CO_COROUTINE (in module inspect): Code Objects Bit Flags. + (line 39) +* co_filename (code object attribute): The standard type hierarchy. + (line 662) +* co_firstlineno (code object attribute): The standard type hierarchy. + (line 662) +* co_flags (code object attribute): The standard type hierarchy. + (line 662) +* co_freevars (code object attribute): The standard type hierarchy. + (line 662) +* CO_FUTURE_DIVISION (C variable): The Very High Level Layer. + (line 422) +* CO_GENERATOR (in module inspect): Code Objects Bit Flags. + (line 30) +* CO_ITERABLE_COROUTINE (in module inspect): Code Objects Bit Flags. + (line 47) +* co_kwonlyargcount (code object attribute): The standard type hierarchy. + (line 662) +* co_lnotab (code object attribute): The standard type hierarchy. + (line 662) +* co_name (code object attribute): The standard type hierarchy. + (line 662) +* co_names (code object attribute): The standard type hierarchy. + (line 662) +* CO_NESTED (in module inspect): Code Objects Bit Flags. + (line 26) +* CO_NEWLOCALS (in module inspect): Code Objects Bit Flags. + (line 13) +* co_nlocals (code object attribute): The standard type hierarchy. + (line 662) +* CO_NOFREE (in module inspect): Code Objects Bit Flags. + (line 35) +* CO_OPTIMIZED (in module inspect): Code Objects Bit Flags. + (line 9) +* co_posonlyargcount (code object attribute): The standard type hierarchy. + (line 662) +* co_stacksize (code object attribute): The standard type hierarchy. + (line 662) +* CO_VARARGS (in module inspect): Code Objects Bit Flags. + (line 18) +* CO_VARKEYWORDS (in module inspect): Code Objects Bit Flags. + (line 22) +* co_varnames (code object attribute): The standard type hierarchy. + (line 662) +* CPP: New Improved and Deprecated Modules<4>. + (line 59) +* CPPFLAGS: New Improved and Deprecated Modules<4>. + (line 59) +* cProfile (module): profile and cProfile Module Reference. + (line 6) +* CPU time: Functions<2>. (line 200) +* CPU time <1>: Functions<2>. (line 479) +* cpu_count() (in module multiprocessing): Miscellaneous<3>. (line 13) +* cpu_count() (in module os): Miscellaneous System Information. + (line 37) +* CPython: Glossary. (line 296) +* cpython_only() (in module test.support): test support — Utilities for the Python test suite. + (line 633) +* crawl_delay() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 50) +* CRC (zipfile.ZipInfo attribute): ZipInfo Objects. (line 135) +* crc32() (in module binascii): binascii — Convert between binary and ASCII. + (line 113) +* crc32() (in module zlib): zlib — Compression compatible with gzip. + (line 115) +* crc_hqx() (in module binascii): binascii — Convert between binary and ASCII. + (line 106) +* create() (imaplib.IMAP4 method): IMAP4 Objects. (line 72) +* create() (in module venv): API<2>. (line 155) +* create() (venv.EnvBuilder method): API<2>. (line 49) +* createAttribute() (xml.dom.Document method): Document Objects. + (line 46) +* createAttributeNS() (xml.dom.Document method): Document Objects. + (line 53) +* createComment() (xml.dom.Document method): Document Objects. + (line 34) +* createDocument() (xml.dom.DOMImplementation method): DOMImplementation Objects. + (line 16) +* createDocumentType() (xml.dom.DOMImplementation method): DOMImplementation Objects. + (line 26) +* createElement() (xml.dom.Document method): Document Objects. + (line 14) +* createElementNS() (xml.dom.Document method): Document Objects. + (line 21) +* createfilehandler() (tkinter.Widget.tk method): File Handlers. + (line 27) +* CreateKey() (in module winreg): Functions<11>. (line 35) +* CreateKeyEx() (in module winreg): Functions<11>. (line 63) +* createLock() (logging.Handler method): Handler Objects. (line 20) +* createLock() (logging.NullHandler method): NullHandler. (line 24) +* createProcessingInstruction() (xml.dom.Document method): Document Objects. + (line 40) +* CreateRecord() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 55) +* createSocket() (logging.handlers.SocketHandler method): SocketHandler. + (line 72) +* createTextNode() (xml.dom.Document method): Document Objects. + (line 28) +* create_aggregate() (sqlite3.Connection method): Connection Objects. + (line 109) +* create_archive() (in module zipapp): Python API. (line 8) +* create_autospec() (in module unittest.mock): create_autospec. + (line 6) +* CREATE_BREAKAWAY_FROM_JOB (in module subprocess): Windows Constants. + (line 124) +* create_collation() (sqlite3.Connection method): Connection Objects. + (line 147) +* create_configuration() (venv.EnvBuilder method): API<2>. (line 86) +* create_connection() (asyncio.loop method): Opening network connections. + (line 6) +* create_connection() (in module socket): Creating sockets. (line 79) +* create_datagram_endpoint() (asyncio.loop method): Opening network connections. + (line 134) +* create_decimal() (decimal.Context method): Context objects. + (line 170) +* create_decimal_from_float() (decimal.Context method): Context objects. + (line 194) +* create_default_context() (in module ssl): Context creation. + (line 9) +* CREATE_DEFAULT_ERROR_MODE (in module subprocess): Windows Constants. + (line 114) +* create_empty_file() (in module test.support): test support — Utilities for the Python test suite. + (line 237) +* create_function() (sqlite3.Connection method): Connection Objects. + (line 73) +* create_future() (asyncio.loop method): Creating Futures and Tasks. + (line 6) +* create_module (C function): Multi-phase initialization. + (line 67) +* create_module() (importlib.abc.Loader method): importlib abc – Abstract base classes related to import. + (line 170) +* create_module() (importlib.machinery.ExtensionFileLoader method): importlib machinery – Importers and path hooks. + (line 319) +* CREATE_NEW_CONSOLE (in module subprocess): Windows Constants. + (line 38) +* CREATE_NEW_PROCESS_GROUP (in module subprocess): Windows Constants. + (line 43) +* CREATE_NO_WINDOW (in module subprocess): Windows Constants. + (line 99) +* create_server() (asyncio.loop method): Creating network servers. + (line 6) +* create_server() (in module socket): Creating sockets. (line 103) +* create_shortcut() (built-in function): The Postinstallation script. + (line 62) +* create_socket() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 184) +* create_static_lib() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 323) +* create_stats() (profile.Profile method): profile and cProfile Module Reference. + (line 86) +* create_string_buffer() (in module ctypes): Utility functions. + (line 40) +* create_subprocess_exec() (in module asyncio): Creating Subprocesses. + (line 6) +* create_subprocess_shell() (in module asyncio): Creating Subprocesses. + (line 25) +* create_system (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 99) +* create_task() (asyncio.loop method): Creating Futures and Tasks. + (line 17) +* create_task() (in module asyncio): Creating Tasks. (line 6) +* create_tree() (in module distutils.dir_util): distutils dir_util — Directory tree operations. + (line 21) +* create_unicode_buffer() (in module ctypes): Utility functions. + (line 58) +* create_unix_connection() (asyncio.loop method): Opening network connections. + (line 212) +* create_unix_server() (asyncio.loop method): Creating network servers. + (line 94) +* create_version (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 103) +* credits (built-in variable): Constants added by the site module. + (line 18) +* critical() (in module logging): Module-Level Functions. + (line 138) +* critical() (logging.Logger method): Logger Objects. (line 234) +* CRNCYSTR (in module locale): locale — Internationalization services. + (line 257) +* cross() (in module audioop): audioop — Manipulate raw audio data. + (line 74) +* crypt (module): crypt — Function to check Unix passwords. + (line 6) +* crypt() (in module crypt): Module Functions<2>. + (line 8) +* crypt(3): crypt — Function to check Unix passwords. + (line 8) +* crypt(3) <1>: crypt — Function to check Unix passwords. + (line 16) +* crypt(3) <2>: Module Functions<2>. + (line 32) +* cryptography: Cryptographic Services. + (line 6) +* cssclasses (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 193) +* cssclasses_weekday_head (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 213) +* cssclass_month (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 227) +* cssclass_month_head (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 220) +* cssclass_noday (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 206) +* cssclass_year (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 234) +* cssclass_year_head (calendar.HTMLCalendar attribute): calendar — General calendar-related functions. + (line 241) +* csv: csv — CSV File Reading and Writing. + (line 8) +* csv (module): csv — CSV File Reading and Writing. + (line 6) +* cte (email.headerregistry.ContentTransferEncoding attribute): email headerregistry Custom Header Objects. + (line 294) +* ctermid() (in module os): Process Parameters. (line 9) +* cte_type (email.policy.Policy attribute): email policy Policy Objects. + (line 159) +* ctime() (datetime.date method): date Objects. (line 251) +* ctime() (datetime.datetime method): datetime Objects. (line 681) +* ctime() (in module time): Functions<2>. (line 85) +* ctrl() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 213) +* CTRL_BREAK_EVENT (in module signal): Module contents<2>. (line 155) +* CTRL_C_EVENT (in module signal): Module contents<2>. (line 146) +* ctypes (module): ctypes — A foreign function library for Python. + (line 6) +* curdir (in module os): Miscellaneous System Information. + (line 81) +* currency() (in module locale): locale — Internationalization services. + (line 420) +* current() (tkinter.ttk.Combobox method): ttk Combobox. (line 8) +* CurrentByteIndex (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 157) +* CurrentColumnNumber (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 161) +* currentframe() (in module inspect): The interpreter stack. + (line 74) +* CurrentLineNumber (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 165) +* current_process() (in module multiprocessing): Miscellaneous<3>. + (line 28) +* current_task() (asyncio.Task class method): Task Object. (line 244) +* current_task() (in module asyncio): Introspection. (line 6) +* current_thread() (in module threading): threading — Thread-based parallelism. + (line 39) +* curses (module): curses — Terminal handling for character-cell displays. + (line 6) +* curses.ascii (module): curses ascii — Utilities for ASCII characters. + (line 6) +* curses.panel (module): curses panel — A panel stack extension for curses. + (line 6) +* curses.textpad (module): curses textpad — Text input widget for curses programs. + (line 6) +* Cursor (class in sqlite3): Cursor Objects. (line 6) +* cursor() (sqlite3.Connection method): Connection Objects. (line 26) +* cursyncup() (curses.window method): Window Objects. (line 172) +* curs_set() (in module curses): Functions<3>. (line 61) +* customize_compiler() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 84) +* Cut: Help menu Shell and Editor. + (line 30) +* cwd() (ftplib.FTP method): FTP Objects. (line 213) +* cwd() (pathlib.Path class method): Methods<2>. (line 17) +* cycle() (in module itertools): Itertool functions. (line 247) +* Cyclic Redundancy Check: zlib — Compression compatible with gzip. + (line 117) +* c_bool (class in ctypes): Fundamental data types<2>. + (line 202) +* C_BUILTIN (in module imp): imp — Access the import internals. + (line 333) +* c_byte (class in ctypes): Fundamental data types<2>. + (line 45) +* c_char (class in ctypes): Fundamental data types<2>. + (line 51) +* c_char_p (class in ctypes): Fundamental data types<2>. + (line 58) +* c_contiguous (memoryview attribute): Memory Views. (line 468) +* c_double (class in ctypes): Fundamental data types<2>. + (line 65) +* C_EXTENSION (in module imp): imp — Access the import internals. + (line 321) +* c_float (class in ctypes): Fundamental data types<2>. + (line 76) +* c_int (class in ctypes): Fundamental data types<2>. + (line 81) +* c_int16 (class in ctypes): Fundamental data types<2>. + (line 93) +* c_int32 (class in ctypes): Fundamental data types<2>. + (line 98) +* c_int64 (class in ctypes): Fundamental data types<2>. + (line 103) +* c_int8 (class in ctypes): Fundamental data types<2>. + (line 88) +* c_long (class in ctypes): Fundamental data types<2>. + (line 108) +* c_longdouble (class in ctypes): Fundamental data types<2>. + (line 70) +* c_longlong (class in ctypes): Fundamental data types<2>. + (line 113) +* c_short (class in ctypes): Fundamental data types<2>. + (line 119) +* c_size_t (class in ctypes): Fundamental data types<2>. + (line 124) +* c_ssize_t (class in ctypes): Fundamental data types<2>. + (line 128) +* c_ubyte (class in ctypes): Fundamental data types<2>. + (line 134) +* c_uint (class in ctypes): Fundamental data types<2>. + (line 140) +* c_uint16 (class in ctypes): Fundamental data types<2>. + (line 152) +* c_uint32 (class in ctypes): Fundamental data types<2>. + (line 157) +* c_uint64 (class in ctypes): Fundamental data types<2>. + (line 162) +* c_uint8 (class in ctypes): Fundamental data types<2>. + (line 147) +* c_ulong (class in ctypes): Fundamental data types<2>. + (line 167) +* c_ulonglong (class in ctypes): Fundamental data types<2>. + (line 172) +* c_ushort (class in ctypes): Fundamental data types<2>. + (line 178) +* c_void_p (class in ctypes): Fundamental data types<2>. + (line 184) +* c_wchar (class in ctypes): Fundamental data types<2>. + (line 189) +* c_wchar_p (class in ctypes): Fundamental data types<2>. + (line 196) +* daemon (multiprocessing.Process attribute): Process and exceptions. + (line 87) +* daemon (threading.Thread attribute): Thread Objects. (line 184) +* dangling; else: Compound statements. + (line 54) +* data: Objects values and types. + (line 6) +* Data (class in plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 169) +* data (collections.UserDict attribute): UserDict objects. (line 24) +* data (collections.UserList attribute): UserList objects. (line 29) +* data (collections.UserString attribute): UserString objects. + (line 24) +* data (select.kevent attribute): Kevent Objects. (line 175) +* data (selectors.SelectorKey attribute): Classes<3>. (line 48) +* data (urllib.request.Request attribute): Request Objects. (line 39) +* data (xml.dom.Comment attribute): Comment Objects. (line 9) +* data (xml.dom.ProcessingInstruction attribute): ProcessingInstruction Objects. + (line 14) +* data (xml.dom.Text attribute): Text and CDATASection Objects. + (line 15) +* data (xmlrpc.client.Binary attribute): Binary Objects. (line 12) +* data() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 33) +* data; tabular: csv — CSV File Reading and Writing. + (line 8) +* data; type: The standard type hierarchy. + (line 6) +* DatabaseError: Exceptions<4>. (line 15) +* databases: dbm dumb — Portable DBM implementation. + (line 8) +* dataclass() (in module dataclasses): Module-level decorators classes and functions. + (line 6) +* dataclasses (module): dataclasses — Data Classes. + (line 6) +* DatagramHandler (class in logging.handlers): DatagramHandler. + (line 10) +* DatagramProtocol (class in asyncio): Base Protocols. (line 20) +* DatagramRequestHandler (class in socketserver): Request Handler Objects. + (line 42) +* DatagramTransport (class in asyncio): Transports Hierarchy. + (line 43) +* datagram_received() (asyncio.DatagramProtocol method): Datagram Protocols. + (line 9) +* DataHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 420) +* data_open() (urllib.request.DataHandler method): DataHandler Objects. + (line 6) +* data_received() (asyncio.Protocol method): Streaming Protocols. + (line 13) +* date (class in datetime): date Objects. (line 13) +* date() (datetime.datetime method): datetime Objects. (line 391) +* date() (nntplib.NNTP method): Methods<3>. (line 298) +* DateHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 136) +* datetime (class in datetime): datetime Objects. (line 17) +* DateTime (class in xmlrpc.client): DateTime Objects. (line 6) +* datetime (email.headerregistry.DateHeader attribute): email headerregistry Custom Header Objects. + (line 145) +* datetime (module): datetime — Basic date and time types. + (line 6) +* date_time (zipfile.ZipInfo attribute): ZipInfo Objects. (line 54) +* date_time_string() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 310) +* datum: Dictionary displays. + (line 6) +* day (datetime.date attribute): date Objects. (line 110) +* day (datetime.datetime attribute): datetime Objects. (line 272) +* daylight (in module time): Timezone Constants. (line 13) +* Daylight Saving Time: time — Time access and conversions. + (line 44) +* day_abbr (in module calendar): calendar — General calendar-related functions. + (line 371) +* day_name (in module calendar): calendar — General calendar-related functions. + (line 366) +* DbfilenameShelf (class in shelve): Restrictions. (line 64) +* dbm (module): dbm — Interfaces to Unix “databases”. + (line 6) +* dbm.dumb (module): dbm dumb — Portable DBM implementation. + (line 6) +* dbm.gnu (module): dbm gnu — GNU’s reinterpretation of dbm. + (line 6) +* dbm.ndbm (module): dbm ndbm — Interface based on ndbm. + (line 6) +* dcgettext() (in module locale): Access to message catalogs. + (line 10) +* deallocation, object: Finalization and De-allocation. + (line 6) +* debug (imaplib.IMAP4 attribute): IMAP4 Objects. (line 381) +* DEBUG (in module re): Module Contents. (line 58) +* debug (pdb command): Debugger Commands. (line 335) +* debug (shlex.shlex attribute): shlex Objects. (line 162) +* debug (zipfile.ZipFile attribute): ZipFile Objects. (line 302) +* debug() (in module doctest): Debugging. (line 124) +* debug() (in module logging): Module-Level Functions. + (line 44) +* debug() (logging.Logger method): Logger Objects. (line 118) +* debug() (pipes.Template method): Template Objects. (line 16) +* debug() (unittest.TestCase method): Test cases. (line 120) +* debug() (unittest.TestSuite method): Grouping tests. (line 48) +* debugger: Debug menu Shell window only. + (line 15) +* debugger <1>: sys — System-specific parameters and functions. + (line 745) +* debugger <2>: sys — System-specific parameters and functions. + (line 1310) +* debugger; configuration; file: Debugger Commands. (line 37) +* debugging: pdb — The Python Debugger. + (line 8) +* debugging; assertions: The assert statement. + (line 6) +* DebuggingServer (class in smtpd): DebuggingServer Objects. + (line 6) +* debuglevel (http.client.HTTPResponse attribute): HTTPResponse Objects. + (line 59) +* DebugRunner (class in doctest): Debugging. (line 168) +* DEBUG_BYTECODE_SUFFIXES (in module importlib.machinery): importlib machinery – Importers and path hooks. + (line 20) +* DEBUG_COLLECTABLE (in module gc): gc — Garbage Collector interface. + (line 273) +* DEBUG_LEAK (in module gc): gc — Garbage Collector interface. + (line 293) +* debug_print() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 511) +* DEBUG_SAVEALL (in module gc): gc — Garbage Collector interface. + (line 287) +* debug_src() (in module doctest): Debugging. (line 147) +* DEBUG_STATS (in module gc): gc — Garbage Collector interface. + (line 268) +* DEBUG_UNCOLLECTABLE (in module gc): gc — Garbage Collector interface. + (line 277) +* Decimal (class in decimal): Decimal objects. (line 6) +* decimal (module): decimal — Decimal fixed point and floating point arithmetic. + (line 6) +* decimal literal: Numeric literals. (line 6) +* decimal() (in module unicodedata): unicodedata — Unicode Database. + (line 31) +* DecimalException (class in decimal): Signals. (line 28) +* decode (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 69) +* decode() (bytearray method): Bytes and Bytearray Operations. + (line 49) +* decode() (bytes method): Bytes and Bytearray Operations. + (line 49) +* decode() (codecs.Codec method): Stateless Encoding and Decoding. + (line 26) +* decode() (codecs.IncrementalDecoder method): IncrementalDecoder Objects. + (line 28) +* decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 205) +* decode() (in module codecs): codecs — Codec registry and base classes. + (line 34) +* decode() (in module quopri): quopri — Encode and decode MIME quoted-printable data. + (line 19) +* decode() (in module uu): uu — Encode and decode uuencode files. + (line 37) +* decode() (json.JSONDecoder method): Encoders and Decoders. + (line 85) +* decode() (xmlrpc.client.Binary method): Binary Objects. (line 20) +* decode() (xmlrpc.client.DateTime method): DateTime Objects. + (line 13) +* decodebytes() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 212) +* DecodedGenerator (class in email.generator): email generator Generating MIME documents. + (line 232) +* decodestring() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 220) +* decodestring() (in module quopri): quopri — Encode and decode MIME quoted-printable data. + (line 41) +* decode_header() (in module email.header): email header Internationalized headers. + (line 183) +* decode_header() (in module nntplib): Utility functions<2>. + (line 8) +* decode_params() (in module email.utils): email utils Miscellaneous utilities. + (line 204) +* decode_rfc2231() (in module email.utils): email utils Miscellaneous utilities. + (line 174) +* decode_source() (in module importlib.util): importlib util – Utility code for importers. + (line 72) +* decomposition() (in module unicodedata): unicodedata — Unicode Database. + (line 75) +* decompress() (bz2.BZ2Decompressor method): Incremental de compression. + (line 45) +* decompress() (in module bz2): One-shot de compression. + (line 16) +* decompress() (in module gzip): gzip — Support for gzip files. + (line 183) +* decompress() (in module lzma): Compressing and decompressing data in memory. + (line 192) +* decompress() (in module zlib): zlib — Compression compatible with gzip. + (line 131) +* decompress() (lzma.LZMADecompressor method): Compressing and decompressing data in memory. + (line 132) +* decompress() (zlib.Decompress method): zlib — Compression compatible with gzip. + (line 258) +* decompressobj() (in module zlib): zlib — Compression compatible with gzip. + (line 176) +* decorator: Glossary. (line 303) +* DEDENT (in module token): token — Constants used with Python parse trees. + (line 52) +* DEDENT token: Indentation. (line 33) +* DEDENT token <1>: Compound statements. + (line 54) +* dedent() (in module textwrap): textwrap — Text wrapping and filling. + (line 65) +* deepcopy() (in module copy): copy — Shallow and deep copy operations. + (line 22) +* default (in module email.policy): email policy Policy Objects. + (line 487) +* DEFAULT (in module unittest.mock): DEFAULT. (line 6) +* default (inspect.Parameter attribute): Introspecting callables with the Signature object. + (line 169) +* default (optparse.Option attribute): Option attributes. (line 36) +* default() (cmd.Cmd method): Cmd Objects. (line 74) +* default() (json.JSONEncoder method): Encoders and Decoders. + (line 197) +* default; parameter; value: Function definitions. + (line 53) +* DefaultContext (class in decimal): Context objects. (line 72) +* DefaultCookiePolicy (class in http.cookiejar): http cookiejar — Cookie handling for HTTP clients. + (line 78) +* defaultdict (class in collections): defaultdict objects. + (line 6) +* DefaultDict (class in typing): Classes functions and decorators. + (line 360) +* DefaultEventLoopPolicy (class in asyncio): Policy Objects. (line 50) +* DefaultHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 310) +* DefaultHandlerExpand() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 317) +* defaults() (configparser.ConfigParser method): ConfigParser Objects. + (line 77) +* DefaultSelector (class in selectors): Classes<3>. (line 161) +* defaultTestLoader (in module unittest): Loading and running tests. + (line 431) +* defaultTestResult() (unittest.TestCase method): Test cases. + (line 717) +* DEFAULT_BUFFER_SIZE (in module io): High-level Module Interface. + (line 6) +* default_bufsize (in module xml.dom.pulldom): xml dom pulldom — Support for building partial DOM trees. + (line 104) +* default_exception_handler() (asyncio.loop method): Error Handling API. + (line 26) +* default_factory (collections.defaultdict attribute): defaultdict objects. + (line 49) +* DEFAULT_FORMAT (in module tarfile): tarfile — Read and write tar archive files. + (line 234) +* DEFAULT_IGNORES (in module filecmp): The dircmp class. (line 105) +* default_open() (urllib.request.BaseHandler method): BaseHandler Objects. + (line 30) +* DEFAULT_PROTOCOL (in module pickle): Module Interface. (line 21) +* default_timer() (in module timeit): Python Interface. (line 32) +* defects (email.headerregistry.BaseHeader attribute): email headerregistry Custom Header Objects. + (line 52) +* defects (email.message.EmailMessage attribute): email message Representing an email message. + (line 712) +* defects (email.message.Message attribute): email message Message Representing an email message using the compat32 API. + (line 724) +* define_macro() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 162) +* defpath (in module os): Miscellaneous System Information. + (line 121) +* DefragResult (class in urllib.parse): Structured Parse Results. + (line 40) +* DefragResultBytes (class in urllib.parse): Structured Parse Results. + (line 65) +* def_prog_mode() (in module curses): Functions<3>. (line 70) +* def_shell_mode() (in module curses): Functions<3>. (line 77) +* degrees() (in module math): Angular conversion. (line 6) +* degrees() (in module turtle): Settings for measurement. + (line 6) +* delattr() (built-in function): Built-in Functions. (line 358) +* delay() (in module turtle): Animation control. (line 6) +* delayload (http.cookiejar.FileCookieJar attribute): CookieJar and FileCookieJar Objects. + (line 158) +* delay_output() (in module curses): Functions<3>. (line 85) +* delch() (curses.window method): Window Objects. (line 177) +* dele() (poplib.POP3 method): POP3 Objects. (line 67) +* delete() (ftplib.FTP method): FTP Objects. (line 206) +* delete() (imaplib.IMAP4 method): IMAP4 Objects. (line 76) +* delete() (tkinter.ttk.Treeview method): ttk Treeview. (line 79) +* deleteacl() (imaplib.IMAP4 method): IMAP4 Objects. (line 80) +* deletefilehandler() (tkinter.Widget.tk method): File Handlers. + (line 37) +* DeleteKey() (in module winreg): Functions<11>. (line 102) +* DeleteKeyEx() (in module winreg): Functions<11>. (line 124) +* deleteln() (curses.window method): Window Objects. (line 181) +* deleteMe() (bdb.Breakpoint method): bdb — Debugger framework. + (line 41) +* DeleteValue() (in module winreg): Functions<11>. (line 165) +* DELETE_ATTR (opcode): Python Bytecode Instructions. + (line 515) +* DELETE_DEREF (opcode): Python Bytecode Instructions. + (line 750) +* DELETE_FAST (opcode): Python Bytecode Instructions. + (line 720) +* DELETE_GLOBAL (opcode): Python Bytecode Instructions. + (line 523) +* DELETE_NAME (opcode): Python Bytecode Instructions. + (line 489) +* DELETE_SUBSCR (opcode): Python Bytecode Instructions. + (line 254) +* deletion; target: The del statement<2>. + (line 6) +* deletion; target; list: The del statement<2>. + (line 6) +* delimiter (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 19) +* delimiters: Delimiters. (line 6) +* delitem() (in module operator): operator — Standard operators as functions. + (line 187) +* deliver_challenge() (in module multiprocessing.connection): Listeners and Clients. + (line 15) +* delocalize() (in module locale): locale — Internationalization services. + (line 441) +* del_param() (email.message.EmailMessage method): email message Representing an email message. + (line 383) +* del_param() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 543) +* demo_app() (in module wsgiref.simple_server): wsgiref simple_server – a simple WSGI HTTP server. + (line 36) +* denominator (fractions.Fraction attribute): fractions — Rational numbers. + (line 94) +* denominator (numbers.Rational attribute): The numeric tower. + (line 51) +* DeprecationWarning: Warnings. (line 17) +* deque (class in collections): deque objects. (line 6) +* Deque (class in typing): Classes functions and decorators. + (line 247) +* dequeue() (logging.handlers.QueueListener method): QueueListener. + (line 46) +* derwin() (curses.window method): Window Objects. (line 186) +* DER_cert_to_PEM_cert() (in module ssl): Certificate handling. + (line 91) +* descrgetfunc (C type): Slot Type typedefs. (line 78) +* description (inspect.Parameter.kind attribute): Introspecting callables with the Signature object. + (line 234) +* description (sqlite3.Cursor attribute): Cursor Objects. (line 220) +* description() (nntplib.NNTP method): Methods<3>. (line 149) +* descriptions() (nntplib.NNTP method): Methods<3>. (line 135) +* descriptor: Glossary. (line 325) +* descrsetfunc (C type): Slot Type typedefs. (line 83) +* dest (optparse.Option attribute): Option attributes. (line 27) +* destructor: Basic customization. + (line 56) +* destructor <1>: Assignment statements. + (line 73) +* destructor (C type): Slot Type typedefs. (line 24) +* detach() (io.BufferedIOBase method): I/O Base Classes. (line 256) +* detach() (io.TextIOBase method): Text I/O<2>. (line 37) +* detach() (socket.socket method): Socket Objects. (line 94) +* detach() (tkinter.ttk.Treeview method): ttk Treeview. (line 85) +* detach() (weakref.finalize method): weakref — Weak references. + (line 259) +* Detach() (winreg.PyHKEY method): Registry Handle Objects. + (line 40) +* DETACHED_PROCESS (in module subprocess): Windows Constants. + (line 106) +* detect_api_mismatch() (in module test.support): test support — Utilities for the Python test suite. + (line 930) +* detect_encoding() (in module tokenize): Tokenizing Input. (line 77) +* detect_language() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 198) +* deterministic profiling: Introduction to the profilers. + (line 6) +* device_encoding() (in module os): File Descriptor Operations. + (line 67) +* devnull (in module os): Miscellaneous System Information. + (line 136) +* DEVNULL (in module subprocess): Using the subprocess Module. + (line 132) +* devpoll() (in module select): select — Waiting for I/O completion. + (line 31) +* DevpollSelector (class in selectors): Classes<3>. (line 184) +* dgettext() (in module gettext): GNU gettext API. (line 49) +* dgettext() (in module locale): Access to message catalogs. + (line 8) +* Dialect (class in csv): Module Contents<3>. (line 176) +* dialect (csv.csvreader attribute): Reader Objects. (line 18) +* dialect (csv.csvwriter attribute): Writer Objects. (line 31) +* Dialog (class in msilib): GUI classes. (line 40) +* dict (2to3 fixer): Fixers. (line 70) +* dict (built-in class): Mapping Types — dict. + (line 27) +* Dict (class in typing): Classes functions and decorators. + (line 349) +* dict() (multiprocessing.managers.SyncManager method): Managers. + (line 205) +* dictConfig() (in module logging.config): Configuration functions. + (line 13) +* dictionary: Glossary. (line 341) +* dictionary comprehension: Glossary. (line 347) +* dictionary view: Glossary. (line 355) +* dictionary; comprehensions: Dictionary displays. + (line 6) +* dictionary; display: Dictionary displays. + (line 6) +* DictReader (class in csv): Module Contents<3>. (line 102) +* DictWriter (class in csv): Module Contents<3>. (line 143) +* Differ (class in difflib): difflib — Helpers for computing deltas. + (line 50) +* difference() (frozenset method): Set Types — set frozenset. + (line 109) +* difference_update() (frozenset method): Set Types — set frozenset. + (line 182) +* difflib (module): difflib — Helpers for computing deltas. + (line 6) +* diff_bytes() (in module difflib): difflib — Helpers for computing deltas. + (line 339) +* diff_files (filecmp.dircmp attribute): The dircmp class. (line 90) +* digest() (hashlib.hash method): Hash algorithms. (line 128) +* digest() (hashlib.shake method): SHAKE variable length digests. + (line 11) +* digest() (hmac.HMAC method): hmac — Keyed-Hashing for Message Authentication. + (line 56) +* digest() (in module hmac): hmac — Keyed-Hashing for Message Authentication. + (line 31) +* digest_size (hmac.HMAC attribute): hmac — Keyed-Hashing for Message Authentication. + (line 90) +* digit() (in module unicodedata): unicodedata — Unicode Database. + (line 37) +* digits (in module string): String constants. (line 24) +* dir() (built-in function): Built-in Functions. (line 380) +* dir() (ftplib.FTP method): FTP Objects. (line 189) +* dircmp (class in filecmp): The dircmp class. (line 6) +* Directory (class in msilib): Directory Objects. (line 6) +* directory (http.server.SimpleHTTPRequestHandler attribute): http server — HTTP servers. + (line 357) +* directory; changing: Files and Directories. + (line 117) +* directory; creating: Files and Directories. + (line 454) +* directory; deleting: Directory and files operations. + (line 257) +* directory; deleting <1>: Files and Directories. + (line 644) +* directory; traversal: Files and Directories. + (line 1493) +* directory; traversal <1>: Files and Directories. + (line 1589) +* directory; walking: Files and Directories. + (line 1493) +* directory; walking <1>: Files and Directories. + (line 1589) +* directory_created() (built-in function): The Postinstallation script. + (line 23) +* DirEntry (class in os): Files and Directories. + (line 821) +* DirList (class in tkinter.tix): File Selectors. (line 6) +* dirname() (in module os.path): os path — Common pathname manipulations. + (line 106) +* DirSelectBox (class in tkinter.tix): File Selectors. (line 26) +* DirSelectDialog (class in tkinter.tix): File Selectors. (line 20) +* DirsOnSysPath (class in test.support): test support — Utilities for the Python test suite. + (line 1069) +* DirTree (class in tkinter.tix): File Selectors. (line 13) +* dis (module): dis — Disassembler for Python bytecode. + (line 6) +* dis() (dis.Bytecode method): Bytecode analysis. (line 47) +* dis() (in module dis): Analysis functions. (line 40) +* dis() (in module pickletools): Programmatic Interface<2>. + (line 6) +* disable (pdb command): Debugger Commands. (line 100) +* disable() (bdb.Breakpoint method): bdb — Debugger framework. + (line 51) +* disable() (in module faulthandler): Fault handler state. + (line 22) +* disable() (in module gc): gc — Garbage Collector interface. + (line 26) +* disable() (in module logging): Module-Level Functions. + (line 167) +* disable() (profile.Profile method): profile and cProfile Module Reference. + (line 82) +* DisableReflectionKey() (in module winreg): Functions<11>. (line 487) +* disable_faulthandler() (in module test.support): test support — Utilities for the Python test suite. + (line 470) +* disable_gc() (in module test.support): test support — Utilities for the Python test suite. + (line 482) +* disable_interspersed_args() (optparse.OptionParser method): Querying and manipulating your option parser. + (line 10) +* disassemble() (in module dis): Analysis functions. (line 81) +* discard (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 54) +* discard() (frozenset method): Set Types — set frozenset. + (line 204) +* discard() (mailbox.Mailbox method): Mailbox objects. (line 76) +* discard() (mailbox.MH method): MH. (line 76) +* discard_buffers() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 84) +* disco() (in module dis): Analysis functions. (line 81) +* discover() (unittest.TestLoader method): Loading and running tests. + (line 117) +* disk_usage() (in module shutil): Directory and files operations. + (line 340) +* dispatcher (class in asyncore): asyncore — Asynchronous socket handler. + (line 67) +* dispatcher_with_send (class in asyncore): asyncore — Asynchronous socket handler. + (line 250) +* dispatch_call() (bdb.Bdb method): bdb — Debugger framework. + (line 157) +* dispatch_exception() (bdb.Bdb method): bdb — Debugger framework. + (line 177) +* dispatch_line() (bdb.Bdb method): bdb — Debugger framework. + (line 147) +* dispatch_return() (bdb.Bdb method): bdb — Debugger framework. + (line 167) +* dispatch_table (pickle.Pickler attribute): Module Interface. + (line 176) +* display (pdb command): Debugger Commands. (line 257) +* displayhook() (in module sys): sys — System-specific parameters and functions. + (line 244) +* display_name (email.headerregistry.Address attribute): email headerregistry Custom Header Objects. + (line 426) +* display_name (email.headerregistry.Group attribute): email headerregistry Custom Header Objects. + (line 471) +* dist() (in module math): Trigonometric functions. + (line 32) +* distance() (in module turtle): Tell Turtle’s state. + (line 68) +* distb() (in module dis): Analysis functions. (line 70) +* Distribution (class in distutils.core): distutils core — Core Distutils functionality. + (line 294) +* distutils (module): distutils — Building and installing Python modules. + (line 6) +* distutils.archive_util (module): distutils archive_util — Archiving utilities. + (line 6) +* distutils.bcppcompiler (module): distutils bcppcompiler — Borland Compiler. + (line 6) +* distutils.ccompiler (module): distutils ccompiler — CCompiler base class. + (line 6) +* distutils.cmd (module): distutils cmd — Abstract base class for Distutils commands. + (line 6) +* distutils.command (module): distutils command — Individual Distutils commands. + (line 5) +* distutils.command.bdist (module): distutils command bdist — Build a binary installer. + (line 5) +* distutils.command.bdist_dumb (module): distutils command bdist_dumb — Build a “dumb” installer. + (line 5) +* distutils.command.bdist_msi (module): distutils command bdist_msi — Build a Microsoft Installer binary package. + (line 6) +* distutils.command.bdist_packager (module): distutils command bdist_packager — Abstract base class for packagers. + (line 5) +* distutils.command.bdist_rpm (module): distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM. + (line 5) +* distutils.command.bdist_wininst (module): distutils command bdist_wininst — Build a Windows installer. + (line 6) +* distutils.command.build (module): distutils command build — Build all files of a package. + (line 5) +* distutils.command.build_clib (module): distutils command build_clib — Build any C libraries in a package. + (line 5) +* distutils.command.build_ext (module): distutils command build_ext — Build any extensions in a package. + (line 5) +* distutils.command.build_py (module): distutils command build_py — Build the py/ pyc files of a package. + (line 6) +* distutils.command.build_scripts (module): distutils command build_scripts — Build the scripts of a package. + (line 5) +* distutils.command.check (module): distutils command check — Check the meta-data of a package. + (line 6) +* distutils.command.clean (module): distutils command clean — Clean a package build area. + (line 6) +* distutils.command.config (module): distutils command config — Perform package configuration. + (line 5) +* distutils.command.install (module): distutils command install — Install a package. + (line 5) +* distutils.command.install_data (module): distutils command install_data — Install data files from a package. + (line 5) +* distutils.command.install_headers (module): distutils command install_headers — Install C/C++ header files from a package. + (line 5) +* distutils.command.install_lib (module): distutils command install_lib — Install library files from a package. + (line 5) +* distutils.command.install_scripts (module): distutils command install_scripts — Install script files from a package. + (line 5) +* distutils.command.register (module): distutils command register — Register a module with the Python Package Index. + (line 6) +* distutils.command.sdist (module): distutils command sdist — Build a source distribution. + (line 5) +* distutils.core (module): distutils core — Core Distutils functionality. + (line 6) +* distutils.cygwinccompiler (module): distutils cygwincompiler — Cygwin Compiler. + (line 6) +* distutils.debug (module): distutils debug — Distutils debug mode. + (line 6) +* distutils.dep_util (module): distutils dep_util — Dependency checking. + (line 6) +* distutils.dir_util (module): distutils dir_util — Directory tree operations. + (line 6) +* distutils.dist (module): distutils dist — The Distribution class. + (line 6) +* distutils.errors (module): distutils errors — Distutils exceptions. + (line 6) +* distutils.extension (module): distutils extension — The Extension class. + (line 6) +* distutils.fancy_getopt (module): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 6) +* distutils.filelist (module): distutils filelist — The FileList class. + (line 6) +* distutils.file_util (module): distutils file_util — Single file operations. + (line 6) +* distutils.log (module): distutils log — Simple PEP 282-style logging. + (line 5) +* distutils.msvccompiler (module): distutils msvccompiler — Microsoft Compiler. + (line 6) +* distutils.spawn (module): distutils spawn — Spawn a sub-process. + (line 6) +* distutils.sysconfig (module): distutils sysconfig — System configuration information. + (line 6) +* distutils.text_file (module): distutils text_file — The TextFile class. + (line 6) +* distutils.unixccompiler (module): distutils unixccompiler — Unix C Compiler. + (line 6) +* distutils.util (module): distutils util — Miscellaneous other utility functions. + (line 6) +* distutils.version (module): distutils version — Version number classes. + (line 5) +* DISTUTILS_DEBUG: Debugging the setup script. + (line 19) +* divide() (decimal.Context method): Context objects. (line 271) +* divide_int() (decimal.Context method): Context objects. (line 275) +* division: Binary arithmetic operations. + (line 28) +* DivisionByZero (class in decimal): Signals. (line 33) +* divmod() (built-in function): Built-in Functions. (line 438) +* divmod() (decimal.Context method): Context objects. (line 279) +* DllCanUnloadNow() (in module ctypes): Utility functions. (line 76) +* DllGetClassObject() (in module ctypes): Utility functions. (line 82) +* dllhandle (in module sys): sys — System-specific parameters and functions. + (line 238) +* dngettext() (in module gettext): GNU gettext API. (line 68) +* dnpgettext() (in module gettext): GNU gettext API. (line 79) +* doc (json.JSONDecodeError attribute): Exceptions<13>. (line 15) +* DocCGIXMLRPCRequestHandler (class in xmlrpc.server): Documenting XMLRPC server. + (line 22) +* DocFileSuite() (in module doctest): Unittest API. (line 24) +* doClassCleanups() (unittest.TestCase class method): Test cases. + (line 792) +* doCleanups() (unittest.TestCase method): Test cases. (line 760) +* docmd() (smtplib.SMTP method): SMTP Objects. (line 17) +* docstring: Class definitions. (line 6) +* docstring <1>: Glossary. (line 365) +* docstring (doctest.DocTest attribute): DocTest Objects. (line 49) +* docstrings: Defining Functions. (line 21) +* docstrings <1>: Documentation Strings. + (line 6) +* DocTest (class in doctest): DocTest Objects. (line 6) +* doctest (module): doctest — Test interactive Python examples. + (line 6) +* DocTestFailure: Debugging. (line 186) +* DocTestFinder (class in doctest): DocTestFinder objects. + (line 6) +* DocTestParser (class in doctest): DocTestParser objects. + (line 6) +* DocTestRunner (class in doctest): DocTestRunner objects. + (line 6) +* DocTestSuite() (in module doctest): Unittest API. (line 99) +* doctype() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 66) +* documentation string: The standard type hierarchy. + (line 703) +* documentation strings: Defining Functions. (line 21) +* documentation strings <1>: Documentation Strings. + (line 6) +* documentation; generation: pydoc — Documentation generator and online help system. + (line 8) +* documentation; online: pydoc — Documentation generator and online help system. + (line 8) +* documentElement (xml.dom.Document attribute): Document Objects. + (line 10) +* DocXMLRPCRequestHandler (class in xmlrpc.server): Documenting XMLRPC server. + (line 27) +* DocXMLRPCServer (class in xmlrpc.server): Documenting XMLRPC server. + (line 11) +* doc_header (cmd.Cmd attribute): Cmd Objects. (line 147) +* domain (email.headerregistry.Address attribute): email headerregistry Custom Header Objects. + (line 437) +* domain (tracemalloc.DomainFilter attribute): DomainFilter. (line 20) +* domain (tracemalloc.Filter attribute): Filter. (line 30) +* domain (tracemalloc.Trace attribute): Trace. (line 15) +* DomainFilter (class in tracemalloc): DomainFilter. (line 6) +* DomainLiberal (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 156) +* DomainRFC2965Match (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 149) +* DomainStrict (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 161) +* DomainStrictNoDots (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 136) +* DomainStrictNonDomain (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 142) +* domain_initial_dot (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 86) +* domain_return_ok() (http.cookiejar.CookiePolicy method): CookiePolicy Objects. + (line 27) +* domain_specified (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 82) +* DOMEventStream (class in xml.dom.pulldom): DOMEventStream Objects. + (line 6) +* DOMException: Exceptions<16>. (line 18) +* doModuleCleanups() (in module unittest): setUpModule and tearDownModule. + (line 35) +* DomstringSizeErr: Exceptions<16>. (line 23) +* done() (asyncio.Future method): Future Object. (line 59) +* done() (asyncio.Task method): Task Object. (line 113) +* done() (concurrent.futures.Future method): Future Objects. (line 33) +* done() (in module turtle): Using screen events. + (line 106) +* done() (xdrlib.Unpacker method): Unpacker Objects. (line 26) +* DONT_ACCEPT_BLANKLINE (in module doctest): Option Flags. (line 30) +* DONT_ACCEPT_TRUE_FOR_1 (in module doctest): Option Flags. (line 18) +* dont_write_bytecode (in module sys): sys — System-specific parameters and functions. + (line 280) +* doRollover() (logging.handlers.RotatingFileHandler method): RotatingFileHandler. + (line 38) +* doRollover() (logging.handlers.TimedRotatingFileHandler method): TimedRotatingFileHandler. + (line 101) +* DOT (in module token): token — Constants used with Python parse trees. + (line 118) +* dot() (in module turtle): Turtle motion. (line 218) +* DOTALL (in module re): Module Contents. (line 113) +* doublequote (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 24) +* DOUBLESLASH (in module token): token — Constants used with Python parse trees. + (line 214) +* DOUBLESLASHEQUAL (in module token): token — Constants used with Python parse trees. + (line 218) +* DOUBLESTAR (in module token): token — Constants used with Python parse trees. + (line 166) +* DOUBLESTAREQUAL (in module token): token — Constants used with Python parse trees. + (line 210) +* doupdate() (in module curses): Functions<3>. (line 89) +* down (pdb command): Debugger Commands. (line 61) +* down() (in module turtle): Drawing state. (line 6) +* do_clear() (bdb.Bdb method): bdb — Debugger framework. + (line 234) +* do_command() (curses.textpad.Textbox method): Textbox objects. + (line 30) +* do_GET() (http.server.SimpleHTTPRequestHandler method): http server — HTTP servers. + (line 372) +* do_handshake() (ssl.SSLSocket method): SSL Sockets. (line 121) +* do_HEAD() (http.server.SimpleHTTPRequestHandler method): http server — HTTP servers. + (line 365) +* do_POST() (http.server.CGIHTTPRequestHandler method): http server — HTTP servers. + (line 484) +* dpgettext() (in module gettext): GNU gettext API. (line 75) +* drain() (asyncio.StreamWriter method): StreamWriter. (line 66) +* dropwhile() (in module itertools): Itertool functions. (line 266) +* drop_whitespace (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 190) +* dst() (datetime.datetime method): datetime Objects. (line 485) +* dst() (datetime.time method): time Objects. (line 233) +* dst() (datetime.timezone method): timezone Objects. (line 57) +* dst() (datetime.tzinfo method): tzinfo Objects. (line 65) +* DTDHandler (class in xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 24) +* duck-typing: Glossary. (line 374) +* DumbWriter (class in formatter): Writer Implementations. + (line 22) +* dummy_threading (module): dummy_threading — Drop-in replacement for the threading module. + (line 6) +* dump() (in module ast): ast Helpers. (line 199) +* dump() (in module json): Basic Usage. (line 6) +* dump() (in module marshal): marshal — Internal Python object serialization. + (line 47) +* dump() (in module pickle): Module Interface. (line 35) +* dump() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 85) +* dump() (in module xml.etree.ElementTree): Functions<6>. (line 86) +* dump() (pickle.Pickler method): Module Interface. (line 155) +* dump() (tracemalloc.Snapshot method): Snapshot. (line 29) +* dumps() (in module json): Basic Usage. (line 78) +* dumps() (in module marshal): marshal — Internal Python object serialization. + (line 72) +* dumps() (in module pickle): Module Interface. (line 47) +* dumps() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 115) +* dumps() (in module xmlrpc.client): Convenience Functions. + (line 6) +* dump_stats() (profile.Profile method): profile and cProfile Module Reference. + (line 96) +* dump_stats() (pstats.Stats method): The Stats Class. (line 59) +* dump_traceback() (in module faulthandler): Dumping the traceback. + (line 6) +* dump_traceback_later() (in module faulthandler): Dumping the tracebacks after a timeout. + (line 6) +* dup() (in module os): File Descriptor Operations. + (line 73) +* dup() (socket.socket method): Socket Objects. (line 102) +* dup2() (in module os): File Descriptor Operations. + (line 85) +* DuplicateOptionError: Exceptions<5>. (line 24) +* DuplicateSectionError: Exceptions<5>. (line 14) +* DUP_TOP (opcode): Python Bytecode Instructions. + (line 79) +* DUP_TOP_TWO (opcode): Python Bytecode Instructions. + (line 85) +* dwFlags (subprocess.STARTUPINFO attribute): Windows Popen Helpers. + (line 19) +* DynamicClassAttribute() (in module types): Additional Utility Classes and Functions. + (line 38) +* D_FMT (in module locale): locale — Internationalization services. + (line 195) +* D_T_FMT (in module locale): locale — Internationalization services. + (line 189) +* e (in module cmath): Constants<3>. (line 10) +* e (in module math): Constants<2>. (line 10) +* E2BIG (in module errno): errno — Standard errno system symbols. + (line 55) +* e; in numeric literal: Integer literals. (line 40) +* EACCES (in module errno): errno — Standard errno system symbols. + (line 79) +* EADDRINUSE (in module errno): errno — Standard errno system symbols. + (line 419) +* EADDRNOTAVAIL (in module errno): errno — Standard errno system symbols. + (line 423) +* EADV (in module errno): errno — Standard errno system symbols. + (line 299) +* EAFNOSUPPORT (in module errno): errno — Standard errno system symbols. + (line 415) +* EAFP: Glossary. (line 387) +* EAGAIN (in module errno): errno — Standard errno system symbols. + (line 71) +* EALREADY (in module errno): errno — Standard errno system symbols. + (line 483) +* east_asian_width() (in module unicodedata): unicodedata — Unicode Database. + (line 64) +* EBADE (in module errno): errno — Standard errno system symbols. + (line 235) +* EBADF (in module errno): errno — Standard errno system symbols. + (line 63) +* EBADFD (in module errno): errno — Standard errno system symbols. + (line 335) +* EBADMSG (in module errno): errno — Standard errno system symbols. + (line 323) +* EBADR (in module errno): errno — Standard errno system symbols. + (line 239) +* EBADRQC (in module errno): errno — Standard errno system symbols. + (line 251) +* EBADSLT (in module errno): errno — Standard errno system symbols. + (line 255) +* EBFONT (in module errno): errno — Standard errno system symbols. + (line 263) +* EBUSY (in module errno): errno — Standard errno system symbols. + (line 91) +* ECHILD (in module errno): errno — Standard errno system symbols. + (line 67) +* echo() (in module curses): Functions<3>. (line 105) +* echochar() (curses.window method): Window Objects. (line 196) +* ECHRNG (in module errno): errno — Standard errno system symbols. + (line 203) +* ECOMM (in module errno): errno — Standard errno system symbols. + (line 307) +* ECONNABORTED (in module errno): errno — Standard errno system symbols. + (line 439) +* ECONNREFUSED (in module errno): errno — Standard errno system symbols. + (line 471) +* ECONNRESET (in module errno): errno — Standard errno system symbols. + (line 443) +* EDEADLK (in module errno): errno — Standard errno system symbols. + (line 167) +* EDEADLOCK (in module errno): errno — Standard errno system symbols. + (line 259) +* EDESTADDRREQ (in module errno): errno — Standard errno system symbols. + (line 383) +* edit() (curses.textpad.Textbox method): Textbox objects. (line 19) +* EDOM (in module errno): errno — Standard errno system symbols. + (line 159) +* EDOTDOT (in module errno): errno — Standard errno system symbols. + (line 319) +* EDQUOT (in module errno): errno — Standard errno system symbols. + (line 515) +* EEXIST (in module errno): errno — Standard errno system symbols. + (line 95) +* EFAULT (in module errno): errno — Standard errno system symbols. + (line 83) +* EFBIG (in module errno): errno — Standard errno system symbols. + (line 135) +* effective() (in module bdb): bdb — Debugger framework. + (line 397) +* ehlo() (smtplib.SMTP method): SMTP Objects. (line 58) +* ehlo_or_helo_if_needed() (smtplib.SMTP method): SMTP Objects. + (line 75) +* EHOSTDOWN (in module errno): errno — Standard errno system symbols. + (line 475) +* EHOSTUNREACH (in module errno): errno — Standard errno system symbols. + (line 479) +* EIDRM (in module errno): errno — Standard errno system symbols. + (line 199) +* EILSEQ (in module errno): errno — Standard errno system symbols. + (line 363) +* EINPROGRESS (in module errno): errno — Standard errno system symbols. + (line 487) +* EINTR (in module errno): errno — Standard errno system symbols. + (line 38) +* EINVAL (in module errno): errno — Standard errno system symbols. + (line 115) +* EIO (in module errno): errno — Standard errno system symbols. + (line 47) +* EISCONN (in module errno): errno — Standard errno system symbols. + (line 451) +* EISDIR (in module errno): errno — Standard errno system symbols. + (line 111) +* EISNAM (in module errno): errno — Standard errno system symbols. + (line 507) +* EL2HLT (in module errno): errno — Standard errno system symbols. + (line 231) +* EL2NSYNC (in module errno): errno — Standard errno system symbols. + (line 207) +* EL3HLT (in module errno): errno — Standard errno system symbols. + (line 211) +* EL3RST (in module errno): errno — Standard errno system symbols. + (line 215) +* Element (class in xml.etree.ElementTree): Element Objects. (line 6) +* ElementDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 202) +* elements() (collections.Counter method): Counter objects. (line 66) +* ElementTree (class in xml.etree.ElementTree): ElementTree Objects. + (line 6) +* element_create() (tkinter.ttk.Style method): Ttk Styling. (line 131) +* element_names() (tkinter.ttk.Style method): Ttk Styling. (line 183) +* element_options() (tkinter.ttk.Style method): Ttk Styling. (line 187) +* ELIBACC (in module errno): errno — Standard errno system symbols. + (line 343) +* ELIBBAD (in module errno): errno — Standard errno system symbols. + (line 347) +* ELIBEXEC (in module errno): errno — Standard errno system symbols. + (line 359) +* ELIBMAX (in module errno): errno — Standard errno system symbols. + (line 355) +* ELIBSCN (in module errno): errno — Standard errno system symbols. + (line 351) +* Ellinghouse, Lance: uu — Encode and decode uuencode files. + (line 20) +* Ellipsis (built-in variable): Built-in Constants. (line 52) +* ELLIPSIS (in module doctest): Option Flags. (line 49) +* ELLIPSIS (in module token): token — Constants used with Python parse trees. + (line 234) +* ELNRNG (in module errno): errno — Standard errno system symbols. + (line 219) +* ELOOP (in module errno): errno — Standard errno system symbols. + (line 187) +* else; conditional expression: Conditional expressions. + (line 6) +* email (module): email — An email and MIME handling package. + (line 6) +* email.charset (module): email charset Representing character sets. + (line 6) +* email.contentmanager (module): email contentmanager Managing MIME Content. + (line 6) +* email.encoders (module): email encoders Encoders. + (line 6) +* email.errors (module): email errors Exception and Defect classes. + (line 6) +* email.generator (module): email generator Generating MIME documents. + (line 6) +* email.header (module): email header Internationalized headers. + (line 6) +* email.headerregistry (module): email headerregistry Custom Header Objects. + (line 6) +* email.iterators (module): email iterators Iterators. + (line 6) +* email.message (module): email message Representing an email message. + (line 6) +* email.mime (module): email mime Creating email and MIME objects from scratch. + (line 6) +* email.parser (module): email parser Parsing email messages. + (line 6) +* email.policy (module): email policy Policy Objects. + (line 6) +* email.utils (module): email utils Miscellaneous utilities. + (line 6) +* EmailMessage (class in email.message): email message Representing an email message. + (line 52) +* EmailPolicy (class in email.policy): email policy Policy Objects. + (line 343) +* EMFILE (in module errno): errno — Standard errno system symbols. + (line 123) +* emit() (logging.FileHandler method): FileHandler. (line 27) +* emit() (logging.Handler method): Handler Objects. (line 107) +* emit() (logging.handlers.BufferingHandler method): MemoryHandler. + (line 24) +* emit() (logging.handlers.DatagramHandler method): DatagramHandler. + (line 20) +* emit() (logging.handlers.HTTPHandler method): HTTPHandler. (line 36) +* emit() (logging.handlers.NTEventLogHandler method): NTEventLogHandler. + (line 38) +* emit() (logging.handlers.QueueHandler method): QueueHandler. + (line 31) +* emit() (logging.handlers.RotatingFileHandler method): RotatingFileHandler. + (line 42) +* emit() (logging.handlers.SMTPHandler method): SMTPHandler. (line 35) +* emit() (logging.handlers.SocketHandler method): SocketHandler. + (line 24) +* emit() (logging.handlers.SysLogHandler method): SysLogHandler. + (line 43) +* emit() (logging.handlers.TimedRotatingFileHandler method): TimedRotatingFileHandler. + (line 105) +* emit() (logging.handlers.WatchedFileHandler method): WatchedFileHandler. + (line 46) +* emit() (logging.NullHandler method): NullHandler. (line 16) +* emit() (logging.StreamHandler method): StreamHandler. (line 17) +* EMLINK (in module errno): errno — Standard errno system symbols. + (line 151) +* Empty: queue — A synchronized queue class. + (line 82) +* empty (inspect.Parameter attribute): Introspecting callables with the Signature object. + (line 152) +* empty (inspect.Signature attribute): Introspecting callables with the Signature object. + (line 74) +* empty() (asyncio.Queue method): Queue. (line 28) +* empty() (multiprocessing.Queue method): Pipes and Queues. (line 108) +* empty() (multiprocessing.SimpleQueue method): Pipes and Queues. + (line 209) +* empty() (queue.Queue method): Queue Objects. (line 15) +* empty() (queue.SimpleQueue method): SimpleQueue Objects. + (line 14) +* empty() (sched.scheduler method): Scheduler Objects. (line 47) +* empty; list: List displays. (line 6) +* empty; tuple: The standard type hierarchy. + (line 172) +* empty; tuple <1>: Parenthesized forms. + (line 16) +* emptyline() (cmd.Cmd method): Cmd Objects. (line 68) +* EMPTY_NAMESPACE (in module xml.dom): Module Contents<4>. (line 35) +* EMSGSIZE (in module errno): errno — Standard errno system symbols. + (line 387) +* EMULTIHOP (in module errno): errno — Standard errno system symbols. + (line 315) +* enable (pdb command): Debugger Commands. (line 107) +* enable() (bdb.Breakpoint method): bdb — Debugger framework. + (line 47) +* enable() (imaplib.IMAP4 method): IMAP4 Objects. (line 84) +* enable() (in module cgitb): cgitb — Traceback manager for CGI scripts. + (line 30) +* enable() (in module faulthandler): Fault handler state. + (line 6) +* enable() (in module gc): gc — Garbage Collector interface. + (line 22) +* enable() (profile.Profile method): profile and cProfile Module Reference. + (line 78) +* EnableReflectionKey() (in module winreg): Functions<11>. (line 505) +* enable_callback_tracebacks() (in module sqlite3): Module functions and constants. + (line 189) +* enable_interspersed_args() (optparse.OptionParser method): Querying and manipulating your option parser. + (line 31) +* enable_load_extension() (sqlite3.Connection method): Connection Objects. + (line 250) +* enable_traversal() (tkinter.ttk.Notebook method): ttk Notebook. + (line 75) +* ENABLE_USER_SITE (in module site): Module contents<3>. (line 10) +* ENAMETOOLONG (in module errno): errno — Standard errno system symbols. + (line 171) +* ENAVAIL (in module errno): errno — Standard errno system symbols. + (line 503) +* enclose() (curses.window method): Window Objects. (line 201) +* encode (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 69) +* encode() (codecs.Codec method): Stateless Encoding and Decoding. + (line 9) +* encode() (codecs.IncrementalEncoder method): IncrementalEncoder Objects. + (line 28) +* encode() (email.header.Header method): email header Internationalized headers. + (line 125) +* encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 226) +* encode() (in module codecs): codecs — Codec registry and base classes. + (line 23) +* encode() (in module quopri): quopri — Encode and decode MIME quoted-printable data. + (line 29) +* encode() (in module uu): uu — Encode and decode uuencode files. + (line 25) +* encode() (json.JSONEncoder method): Encoders and Decoders. + (line 216) +* encode() (str method): String Methods<2>. (line 61) +* encode() (xmlrpc.client.Binary method): Binary Objects. (line 25) +* encode() (xmlrpc.client.DateTime method): DateTime Objects. + (line 17) +* encodebytes() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 236) +* EncodedFile() (in module codecs): codecs — Codec registry and base classes. + (line 194) +* encodePriority() (logging.handlers.SysLogHandler method): SysLogHandler. + (line 77) +* encodestring() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 246) +* encodestring() (in module quopri): quopri — Encode and decode MIME quoted-printable data. + (line 46) +* encode_7or8bit() (in module email.encoders): email encoders Encoders. + (line 57) +* encode_base64() (in module email.encoders): email encoders Encoders. + (line 49) +* encode_noop() (in module email.encoders): email encoders Encoders. + (line 63) +* encode_quopri() (in module email.encoders): email encoders Encoders. + (line 42) +* encode_rfc2231() (in module email.utils): email utils Miscellaneous utilities. + (line 178) +* encoding (curses.window attribute): Window Objects. (line 208) +* ENCODING (in module tarfile): tarfile — Read and write tar archive files. + (line 213) +* ENCODING (in module token): token — Constants used with Python parse trees. + (line 272) +* encoding (io.TextIOBase attribute): Text I/O<2>. (line 15) +* encoding (UnicodeError attribute): Concrete exceptions. + (line 342) +* encoding declarations (source file): Encoding declarations. + (line 6) +* encodings.idna (module): encodings idna — Internationalized Domain Names in Applications. + (line 6) +* encodings.mbcs (module): encodings mbcs — Windows ANSI codepage. + (line 6) +* encodings.utf_8_sig (module): encodings utf_8_sig — UTF-8 codec with BOM signature. + (line 6) +* encodings_map (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 136) +* encodings_map (mimetypes.MimeTypes attribute): MimeTypes Objects. + (line 33) +* end (UnicodeError attribute): Concrete exceptions. + (line 358) +* end() (re.Match method): Match Objects. (line 132) +* end() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 38) +* EndCdataSectionHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 306) +* EndDoctypeDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 197) +* endDocument() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 40) +* endElement() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 95) +* EndElementHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 232) +* endElementNS() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 121) +* endheaders() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 143) +* ENDMARKER (in module token): token — Constants used with Python parse trees. + (line 40) +* EndNamespaceDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 285) +* endpos (re.Match attribute): Match Objects. (line 171) +* endPrefixMapping() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 73) +* endswith() (bytearray method): Bytes and Bytearray Operations. + (line 67) +* endswith() (bytes method): Bytes and Bytearray Operations. + (line 67) +* endswith() (str method): String Methods<2>. (line 75) +* endwin() (in module curses): Functions<3>. (line 110) +* END_ASYNC_FOR (opcode): Python Bytecode Instructions. + (line 284) +* end_col_offset (ast.AST attribute): Node classes. (line 39) +* end_fill() (in module turtle): Filling. (line 20) +* END_FINALLY (opcode): Python Bytecode Instructions. + (line 415) +* end_headers() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 265) +* end_lineno (ast.AST attribute): Node classes. (line 39) +* end_ns() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 85) +* end_paragraph() (formatter.formatter method): The Formatter Interface. + (line 26) +* end_poly() (in module turtle): Special Turtle methods. + (line 11) +* ENETDOWN (in module errno): errno — Standard errno system symbols. + (line 427) +* ENETRESET (in module errno): errno — Standard errno system symbols. + (line 435) +* ENETUNREACH (in module errno): errno — Standard errno system symbols. + (line 431) +* ENFILE (in module errno): errno — Standard errno system symbols. + (line 119) +* ENOANO (in module errno): errno — Standard errno system symbols. + (line 247) +* ENOBUFS (in module errno): errno — Standard errno system symbols. + (line 447) +* ENOCSI (in module errno): errno — Standard errno system symbols. + (line 227) +* ENODATA (in module errno): errno — Standard errno system symbols. + (line 271) +* ENODEV (in module errno): errno — Standard errno system symbols. + (line 103) +* ENOENT (in module errno): errno — Standard errno system symbols. + (line 30) +* ENOEXEC (in module errno): errno — Standard errno system symbols. + (line 59) +* ENOLCK (in module errno): errno — Standard errno system symbols. + (line 175) +* ENOLINK (in module errno): errno — Standard errno system symbols. + (line 295) +* ENOMEM (in module errno): errno — Standard errno system symbols. + (line 75) +* ENOMSG (in module errno): errno — Standard errno system symbols. + (line 195) +* ENONET (in module errno): errno — Standard errno system symbols. + (line 283) +* ENOPKG (in module errno): errno — Standard errno system symbols. + (line 287) +* ENOPROTOOPT (in module errno): errno — Standard errno system symbols. + (line 395) +* ENOSPC (in module errno): errno — Standard errno system symbols. + (line 139) +* ENOSR (in module errno): errno — Standard errno system symbols. + (line 279) +* ENOSTR (in module errno): errno — Standard errno system symbols. + (line 267) +* ENOSYS (in module errno): errno — Standard errno system symbols. + (line 179) +* ENOTBLK (in module errno): errno — Standard errno system symbols. + (line 87) +* ENOTCONN (in module errno): errno — Standard errno system symbols. + (line 455) +* ENOTDIR (in module errno): errno — Standard errno system symbols. + (line 107) +* ENOTEMPTY (in module errno): errno — Standard errno system symbols. + (line 183) +* ENOTNAM (in module errno): errno — Standard errno system symbols. + (line 499) +* ENOTSOCK (in module errno): errno — Standard errno system symbols. + (line 379) +* ENOTTY (in module errno): errno — Standard errno system symbols. + (line 127) +* ENOTUNIQ (in module errno): errno — Standard errno system symbols. + (line 331) +* enqueue() (logging.handlers.QueueHandler method): QueueHandler. + (line 53) +* enqueue_sentinel() (logging.handlers.QueueListener method): QueueListener. + (line 87) +* ensurepip (module): ensurepip — Bootstrapping the pip installer. + (line 6) +* ensure_directories() (venv.EnvBuilder method): API<2>. (line 77) +* ensure_future() (in module asyncio): Future Functions. (line 19) +* enter() (sched.scheduler method): Scheduler Objects. (line 30) +* enterabs() (sched.scheduler method): Scheduler Objects. (line 9) +* enter_async_context() (contextlib.AsyncExitStack method): Utilities. + (line 463) +* enter_context() (contextlib.ExitStack method): Utilities. (line 382) +* entities (xml.dom.DocumentType attribute): DocumentType Objects. + (line 38) +* EntityDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 258) +* entitydefs (in module html.entities): html entities — Definitions of HTML general entities. + (line 25) +* EntityResolver (class in xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 31) +* Enum (class in enum): Module Contents<2>. (line 11) +* enum (module): enum — Support for enumerations. + (line 6) +* enumerate() (built-in function): Built-in Functions. (line 450) +* enumerate() (in module threading): threading — Thread-based parallelism. + (line 108) +* EnumKey() (in module winreg): Functions<11>. (line 177) +* EnumValue() (in module winreg): Functions<11>. (line 196) +* enum_certificates() (in module ssl): Certificate handling. + (line 129) +* enum_crls() (in module ssl): Certificate handling. + (line 152) +* EnvBuilder (class in venv): API<2>. (line 11) +* environ (in module os): Process Parameters. (line 16) +* environ (in module posix): Notable Module Contents. + (line 9) +* environb (in module os): Process Parameters. (line 55) +* environment: Resolution of names. + (line 12) +* environment variable; APPDATA: PEP 370 Per-user site-packages Directory. + (line 26) +* environment variable; AUDIODEV: ossaudiodev — Access to OSS-compatible audio devices. + (line 54) +* environment variable; BROWSER: webbrowser — Convenient Web-browser controller. + (line 20) +* environment variable; BROWSER <1>: webbrowser — Convenient Web-browser controller. + (line 95) +* environment variable; CC: New Improved and Deprecated Modules<4>. + (line 58) +* environment variable; CFLAGS: New Improved and Deprecated Modules<4>. + (line 58) +* environment variable; CFLAGS <1>: Tweaking compiler/linker flags. + (line 66) +* environment variable; CFLAGS <2>: Tweaking compiler/linker flags. + (line 67) +* environment variable; COLS: curses<2>. (line 7) +* environment variable; COLS <1>: Functions<3>. (line 526) +* environment variable; COLUMNS: Functions<3>. (line 550) +* environment variable; COLUMNS <1>: Functions<3>. (line 553) +* environment variable; COMSPEC: Process Management. (line 664) +* environment variable; COMSPEC <1>: Popen Constructor. (line 90) +* environment variable; CPP: New Improved and Deprecated Modules<4>. + (line 59) +* environment variable; CPPFLAGS: New Improved and Deprecated Modules<4>. + (line 59) +* environment variable; DISTUTILS_DEBUG: Debugging the setup script. + (line 19) +* environment variable; exec_prefix: Python-related paths and files. + (line 7) +* environment variable; exec_prefix <1>: Include Files. (line 37) +* environment variable; exec_prefix <2>: Include Files. (line 48) +* environment variable; HOME: os path. (line 15) +* environment variable; HOME <1>: Changes in the Python API. + (line 116) +* environment variable; HOME <2>: os path — Common pathname manipulations. + (line 140) +* environment variable; HOME <3>: os path — Common pathname manipulations. + (line 156) +* environment variable; HOME <4>: distutils util — Miscellaneous other utility functions. + (line 77) +* environment variable; HOME <5>: Location and names of config files. + (line 50) +* environment variable; HOME <6>: Location and names of config files. + (line 67) +* environment variable; HOMEDRIVE: os path — Common pathname manipulations. + (line 147) +* environment variable; HOMEDRIVE <1>: Location and names of config files. + (line 68) +* environment variable; HOMEPATH: os path — Common pathname manipulations. + (line 147) +* environment variable; HOMEPATH <1>: Location and names of config files. + (line 68) +* environment variable; http_proxy: urllib request — Extensible library for opening URLs. + (line 85) +* environment variable; http_proxy <1>: Examples<21>. (line 97) +* environment variable; http_proxy <2>: Basic Authentication. + (line 62) +* environment variable; IDLESTARTUP: Startup and code execution. + (line 7) +* environment variable; KDEDIR: webbrowser — Convenient Web-browser controller. + (line 184) +* environment variable; LANG: GNU gettext API. (line 21) +* environment variable; LANG <1>: Class-based API. (line 26) +* environment variable; LANG <2>: locale — Internationalization services. + (line 49) +* environment variable; LANG <3>: locale — Internationalization services. + (line 310) +* environment variable; LANG <4>: locale — Internationalization services. + (line 314) +* environment variable; LANGUAGE: GNU gettext API. (line 20) +* environment variable; LANGUAGE <1>: Class-based API. (line 25) +* environment variable; LC_ALL: GNU gettext API. (line 20) +* environment variable; LC_ALL <1>: Class-based API. (line 25) +* environment variable; LC_MESSAGES: GNU gettext API. (line 20) +* environment variable; LC_MESSAGES <1>: Class-based API. (line 26) +* environment variable; LDCXXSHARED: Build and C API Changes<8>. + (line 153) +* environment variable; LDFLAGS: New Improved and Deprecated Modules<4>. + (line 59) +* environment variable; LINES: curses<2>. (line 6) +* environment variable; LINES <1>: Functions<3>. (line 125) +* environment variable; LINES <2>: Functions<3>. (line 526) +* environment variable; LINES <3>: Functions<3>. (line 550) +* environment variable; LINES <4>: Functions<3>. (line 552) +* environment variable; LNAME: getpass — Portable password input. + (line 38) +* environment variable; LOGNAME: Process Parameters. (line 221) +* environment variable; LOGNAME <1>: getpass — Portable password input. + (line 38) +* environment variable; MIXERDEV: ossaudiodev — Access to OSS-compatible audio devices. + (line 74) +* environment variable; no_proxy: urllib request — Extensible library for opening URLs. + (line 303) +* environment variable; PAGER: pydoc — Documentation generator and online help system. + (line 47) +* environment variable; PATH: Changes in the Python API. + (line 143) +* environment variable; PATH <1>: Changes in ‘python’ Command Behavior<2>. + (line 6) +* environment variable; PATH <2>: Changes in ‘python’ Command Behavior<2>. + (line 11) +* environment variable; PATH <3>: Changes in ‘python’ Command Behavior<2>. + (line 12) +* environment variable; PATH <4>: The Module Search Path. + (line 16) +* environment variable; PATH <5>: Executable Python Scripts. + (line 11) +* environment variable; PATH <6>: Environment variables. + (line 27) +* environment variable; PATH <7>: Miscellaneous. (line 16) +* environment variable; PATH <8>: Installation steps. (line 32) +* environment variable; PATH <9>: Installation steps. (line 56) +* environment variable; PATH <10>: Installing Without UI. + (line 57) +* environment variable; PATH <11>: Excursus Setting environment variables. + (line 23) +* environment variable; PATH <12>: Excursus Setting environment variables. + (line 35) +* environment variable; PATH <13>: Finding the Python executable. + (line 14) +* environment variable; PATH <14>: Finding the Python executable. + (line 21) +* environment variable; PATH <15>: Finding the Python executable. + (line 22) +* environment variable; PATH <16>: Python Launcher for Windows. + (line 13) +* environment variable; PATH <17>: From the command-line. + (line 9) +* environment variable; PATH <18>: From the command-line. + (line 38) +* environment variable; PATH <19>: Shebang Lines. (line 48) +* environment variable; PATH <20>: Shebang Lines. (line 50) +* environment variable; PATH <21>: Process Management. (line 45) +* environment variable; PATH <22>: Process Management. (line 86) +* environment variable; PATH <23>: Process Management. (line 89) +* environment variable; PATH <24>: Process Management. (line 92) +* environment variable; PATH <25>: Process Management. (line 453) +* environment variable; PATH <26>: Process Management. (line 539) +* environment variable; PATH <27>: Process Management. (line 543) +* environment variable; PATH <28>: Process Management. (line 545) +* environment variable; PATH <29>: Miscellaneous System Information. + (line 118) +* environment variable; PATH <30>: webbrowser — Convenient Web-browser controller. + (line 214) +* environment variable; PATH <31>: Installing your CGI script on a Unix system. + (line 29) +* environment variable; PATH <32>: Common problems and solutions. + (line 23) +* environment variable; PATH <33>: site — Site-specific configuration hook. + (line 59) +* environment variable; PATH <34>: Embedding Python<2>. + (line 31) +* environment variable; PATH <35>: Embedding Python<2>. + (line 37) +* environment variable; PATH <36>: How do I make a Python script executable on Unix?. + (line 24) +* environment variable; PATH <37>: How do I make a Python script executable on Unix?. + (line 28) +* environment variable; PATHEXT: Other Improvements<2>. + (line 30) +* environment variable; PATHEXT <1>: Installing Without UI. + (line 58) +* environment variable; PLAT: distutils util — Miscellaneous other utility functions. + (line 79) +* environment variable; POSIXLY_CORRECT: getopt — C-style parser for command line options. + (line 71) +* environment variable; prefix: Python-related paths and files. + (line 7) +* environment variable; prefix <1>: Include Files. (line 37) +* environment variable; prefix <2>: Include Files. (line 40) +* environment variable; prefix <3>: Include Files. (line 47) +* environment variable; PYTHON*: Other Improvements<2>. + (line 9) +* environment variable; PYTHON* <1>: Interface options. (line 80) +* environment variable; PYTHON* <2>: Interface options. (line 140) +* environment variable; PYTHON* <3>: Miscellaneous options. + (line 42) +* environment variable; PYTHON* <4>: Miscellaneous options. + (line 60) +* environment variable; PYTHON* <5>: Global configuration variables. + (line 56) +* environment variable; PYTHONASYNCIODEBUG: Environment variables. + (line 238) +* environment variable; PYTHONASYNCIODEBUG <1>: Enabling debug mode. + (line 10) +* environment variable; PYTHONASYNCIODEBUG <2>: Debug Mode. (line 11) +* environment variable; PYTHONBREAKPOINT: PEP 553 Built-in breakpoint. + (line 13) +* environment variable; PYTHONBREAKPOINT <1>: Environment variables. + (line 66) +* environment variable; PYTHONBREAKPOINT <2>: sys — System-specific parameters and functions. + (line 202) +* environment variable; PYTHONBREAKPOINT <3>: sys — System-specific parameters and functions. + (line 216) +* environment variable; PYTHONBREAKPOINT <4>: sys — System-specific parameters and functions. + (line 220) +* environment variable; PYTHONCASEOK: PEP 235 Importing Modules on Case-Insensitive Platforms. + (line 17) +* environment variable; PYTHONCASEOK <1>: Environment variables. + (line 104) +* environment variable; PYTHONCOERCECLOCALE: PEP 538 Legacy C Locale Coercion. + (line 13) +* environment variable; PYTHONCOERCECLOCALE <1>: Environment variables. + (line 322) +* environment variable; PYTHONCOERCECLOCALE <2>: Environment variables. + (line 438) +* environment variable; PYTHONCOERCECLOCALE <3>: Python Configuration. + (line 15) +* environment variable; PYTHONDEBUG: Miscellaneous options. + (line 38) +* environment variable; PYTHONDEBUG <1>: Environment variables. + (line 79) +* environment variable; PYTHONDEBUG <2>: Global configuration variables. + (line 28) +* environment variable; PYTHONDEVMODE: Development Runtime Mode -X dev. + (line 6) +* environment variable; PYTHONDEVMODE <1>: Environment variables. + (line 378) +* environment variable; PYTHONDEVMODE <2>: test support script_helper — Utilities for the Python execution tests. + (line 106) +* environment variable; PYTHONDOCS: pydoc — Documentation generator and online help system. + (line 85) +* environment variable; PYTHONDONTWRITEBYTECODE: Interpreter Changes<2>. + (line 16) +* environment variable; PYTHONDONTWRITEBYTECODE <1>: New and Improved Modules<2>. + (line 625) +* environment variable; PYTHONDONTWRITEBYTECODE <2>: Miscellaneous options. + (line 19) +* environment variable; PYTHONDONTWRITEBYTECODE <3>: Environment variables. + (line 109) +* environment variable; PYTHONDONTWRITEBYTECODE <4>: sys — System-specific parameters and functions. + (line 285) +* environment variable; PYTHONDONTWRITEBYTECODE <5>: Global configuration variables. + (line 36) +* environment variable; PYTHONDONTWRITEBYTECODE <6>: How do I create a pyc file?. + (line 21) +* environment variable; PYTHONDUMPREFS: Debug build uses the same ABI as release build. + (line 14) +* environment variable; PYTHONDUMPREFS <1>: Debug-mode variables. + (line 14) +* environment variable; PYTHONDUMPREFS <2>: PyObject Slots. (line 24) +* environment variable; PYTHONEXECUTABLE: Environment variables. + (line 182) +* environment variable; PYTHONFAULTHANDLER: faulthandler<3>. (line 11) +* environment variable; PYTHONFAULTHANDLER <1>: Environment variables. + (line 209) +* environment variable; PYTHONFAULTHANDLER <2>: faulthandler — Dump the Python traceback. + (line 14) +* environment variable; PYTHONHASHSEED: Builtin functions and types. + (line 17) +* environment variable; PYTHONHASHSEED <1>: Porting Python code. + (line 6) +* environment variable; PYTHONHASHSEED <2>: Miscellaneous options. + (line 94) +* environment variable; PYTHONHASHSEED <3>: Miscellaneous options. + (line 110) +* environment variable; PYTHONHASHSEED <4>: Environment variables. + (line 124) +* environment variable; PYTHONHASHSEED <5>: Environment variables. + (line 129) +* environment variable; PYTHONHASHSEED <6>: Basic customization. + (line 298) +* environment variable; PYTHONHASHSEED <7>: Global configuration variables. + (line 48) +* environment variable; PYTHONHASHSEED <8>: Global configuration variables. + (line 51) +* environment variable; PYTHONHOME: Summary – Release highlights<2>. + (line 107) +* environment variable; PYTHONHOME <1>: Miscellaneous options. + (line 43) +* environment variable; PYTHONHOME <2>: Environment variables. + (line 11) +* environment variable; PYTHONHOME <3>: Environment variables. + (line 19) +* environment variable; PYTHONHOME <4>: Environment variables. + (line 21) +* environment variable; PYTHONHOME <5>: Environment variables. + (line 38) +* environment variable; PYTHONHOME <6>: Finding modules. (line 50) +* environment variable; PYTHONHOME <7>: Finding modules. (line 68) +* environment variable; PYTHONHOME <8>: Finding modules. (line 102) +* environment variable; PYTHONHOME <9>: test support script_helper — Utilities for the Python execution tests. + (line 25) +* environment variable; PYTHONHOME <10>: Embedding Python<2>. + (line 38) +* environment variable; PYTHONHOME <11>: Embedding Python<2>. + (line 44) +* environment variable; PYTHONHOME <12>: Global configuration variables. + (line 57) +* environment variable; PYTHONHOME <13>: Process-wide parameters. + (line 270) +* environment variable; PYTHONHOME <14>: Process-wide parameters. + (line 284) +* environment variable; PYTHONHOME <15>: PyConfig. (line 197) +* environment variable; PYTHONHOME <16>: Modifying Python’s Search Path. + (line 60) +* environment variable; PYTHONHOME <17>: Modifying Python’s Search Path. + (line 62) +* environment variable; PYTHONINSPECT: Other Changes and Fixes<2>. + (line 13) +* environment variable; PYTHONINSPECT <1>: Miscellaneous options. + (line 53) +* environment variable; PYTHONINSPECT <2>: Environment variables. + (line 85) +* environment variable; PYTHONINSPECT <3>: Global configuration variables. + (line 68) +* environment variable; PYTHONIOENCODING: Other Improvements<2>. + (line 75) +* environment variable; PYTHONIOENCODING <1>: Interpreter Changes<2>. + (line 23) +* environment variable; PYTHONIOENCODING <2>: Environment variables. + (line 142) +* environment variable; PYTHONIOENCODING <3>: Environment variables. + (line 359) +* environment variable; PYTHONIOENCODING <4>: Environment variables. + (line 426) +* environment variable; PYTHONIOENCODING <5>: sys — System-specific parameters and functions. + (line 1503) +* environment variable; PYTHONIOENCODING <6>: Process-wide parameters. + (line 14) +* environment variable; PYTHONIOENCODING <7>: Process-wide parameters. + (line 18) +* environment variable; PYTHONLEGACYWINDOWSFSENCODING: PEP 529 Change Windows filesystem encoding to UTF-8. + (line 18) +* environment variable; PYTHONLEGACYWINDOWSFSENCODING <1>: Environment variables. + (line 295) +* environment variable; PYTHONLEGACYWINDOWSFSENCODING <2>: sys — System-specific parameters and functions. + (line 1455) +* environment variable; PYTHONLEGACYWINDOWSFSENCODING <3>: Global configuration variables. + (line 90) +* environment variable; PYTHONLEGACYWINDOWSSTDIO: PEP 528 Change Windows console encoding to UTF-8. + (line 12) +* environment variable; PYTHONLEGACYWINDOWSSTDIO <1>: Environment variables. + (line 156) +* environment variable; PYTHONLEGACYWINDOWSSTDIO <2>: Environment variables. + (line 309) +* environment variable; PYTHONLEGACYWINDOWSSTDIO <3>: sys — System-specific parameters and functions. + (line 1507) +* environment variable; PYTHONLEGACYWINDOWSSTDIO <4>: Global configuration variables. + (line 102) +* environment variable; PYTHONMALLOC: PYTHONMALLOC environment variable. + (line 6) +* environment variable; PYTHONMALLOC <1>: Changes in the C API<3>. + (line 9) +* environment variable; PYTHONMALLOC <2>: Environment variables. + (line 245) +* environment variable; PYTHONMALLOC <3>: Environment variables. + (line 286) +* environment variable; PYTHONMALLOC <4>: Overview<3>. (line 75) +* environment variable; PYTHONMALLOC <5>: Default Memory Allocators. + (line 26) +* environment variable; PYTHONMALLOC <6>: Customize Memory Allocators. + (line 129) +* environment variable; PYTHONMALLOCSTATS: Environment variables. + (line 280) +* environment variable; PYTHONMALLOCSTATS <1>: Overview<3>. (line 78) +* environment variable; PYTHONNOUSERSITE: PEP 370 Per-user site-packages Directory. + (line 30) +* environment variable; PYTHONNOUSERSITE <1>: Environment variables. + (line 160) +* environment variable; PYTHONNOUSERSITE <2>: Module contents<3>. + (line 15) +* environment variable; PYTHONNOUSERSITE <3>: Global configuration variables. + (line 124) +* environment variable; PYTHONOPTIMIZE: Miscellaneous options. + (line 71) +* environment variable; PYTHONOPTIMIZE <1>: Environment variables. + (line 60) +* environment variable; PYTHONOPTIMIZE <2>: Global configuration variables. + (line 129) +* environment variable; PYTHONPATH: Changes in ‘python’ Command Behavior<2>. + (line 8) +* environment variable; PYTHONPATH <1>: Changes in ‘python’ Command Behavior<2>. + (line 9) +* environment variable; PYTHONPATH <2>: The Module Search Path. + (line 15) +* environment variable; PYTHONPATH <3>: Standard Modules. (line 34) +* environment variable; PYTHONPATH <4>: Standard Modules. (line 35) +* environment variable; PYTHONPATH <5>: Miscellaneous options. + (line 42) +* environment variable; PYTHONPATH <6>: Environment variables. + (line 24) +* environment variable; PYTHONPATH <7>: Environment variables. + (line 32) +* environment variable; PYTHONPATH <8>: Environment variables. + (line 39) +* environment variable; PYTHONPATH <9>: Environment variables. + (line 42) +* environment variable; PYTHONPATH <10>: Excursus Setting environment variables. + (line 38) +* environment variable; PYTHONPATH <11>: Finding modules. (line 36) +* environment variable; PYTHONPATH <12>: Finding modules. (line 59) +* environment variable; PYTHONPATH <13>: Finding modules. (line 102) +* environment variable; PYTHONPATH <14>: Configuration. (line 6) +* environment variable; PYTHONPATH <15>: Installing your CGI script on a Unix system. + (line 29) +* environment variable; PYTHONPATH <16>: test support script_helper — Utilities for the Python execution tests. + (line 26) +* environment variable; PYTHONPATH <17>: sys — System-specific parameters and functions. + (line 1062) +* environment variable; PYTHONPATH <18>: sys — System-specific parameters and functions. + (line 1072) +* environment variable; PYTHONPATH <19>: Building C and C++ Extensions. + (line 9) +* environment variable; PYTHONPATH <20>: Embedding Python<2>. + (line 39) +* environment variable; PYTHONPATH <21>: Embedding Python<2>. + (line 44) +* environment variable; PYTHONPATH <22>: Global configuration variables. + (line 56) +* environment variable; PYTHONPATH <23>: PyConfig. (line 252) +* environment variable; PYTHONPATH <24>: Modifying Python’s Search Path. + (line 67) +* environment variable; PYTHONPATH <25>: Modifying Python’s Search Path. + (line 68) +* environment variable; PYTHONPROFILEIMPORTTIME: Other Language Changes<2>. + (line 47) +* environment variable; PYTHONPROFILEIMPORTTIME <1>: Miscellaneous options. + (line 223) +* environment variable; PYTHONPROFILEIMPORTTIME <2>: Environment variables. + (line 230) +* environment variable; PYTHONPYCACHEPREFIX: Parallel filesystem cache for compiled bytecode files. + (line 6) +* environment variable; PYTHONPYCACHEPREFIX <1>: Miscellaneous options. + (line 257) +* environment variable; PYTHONPYCACHEPREFIX <2>: Environment variables. + (line 115) +* environment variable; PYTHONPYCACHEPREFIX <3>: sys — System-specific parameters and functions. + (line 303) +* environment variable; PYTHONSHOWALLOCCOUNT: Two new environment variables for debug mode. + (line 11) +* environment variable; PYTHONSHOWREFCOUNT: Two new environment variables for debug mode. + (line 7) +* environment variable; PYTHONSTARTUP: sys<6>. (line 17) +* environment variable; PYTHONSTARTUP <1>: sys<6>. (line 21) +* environment variable; PYTHONSTARTUP <2>: The Interactive Startup File. + (line 8) +* environment variable; PYTHONSTARTUP <3>: Miscellaneous options. + (line 50) +* environment variable; PYTHONSTARTUP <4>: Environment variables. + (line 46) +* environment variable; PYTHONSTARTUP <5>: Example. (line 10) +* environment variable; PYTHONSTARTUP <6>: Startup and code execution. + (line 7) +* environment variable; PYTHONSTARTUP <7>: sys — System-specific parameters and functions. + (line 957) +* environment variable; PYTHONSTARTUP <8>: Readline configuration. + (line 12) +* environment variable; PYTHONTHREADDEBUG: Debug-mode variables. + (line 8) +* environment variable; PYTHONTRACEMALLOC: Environment variables. + (line 219) +* environment variable; PYTHONTRACEMALLOC <1>: tracemalloc — Trace memory allocations. + (line 25) +* environment variable; PYTHONTRACEMALLOC <2>: tracemalloc — Trace memory allocations. + (line 32) +* environment variable; PYTHONTRACEMALLOC <3>: Functions<9>. (line 68) +* environment variable; PYTHONUNBUFFERED: Miscellaneous options. + (line 140) +* environment variable; PYTHONUNBUFFERED <1>: Environment variables. + (line 93) +* environment variable; PYTHONUNBUFFERED <2>: Global configuration variables. + (line 145) +* environment variable; PYTHONUSERBASE: PEP 370 Per-user site-packages Directory. + (line 23) +* environment variable; PYTHONUSERBASE <1>: Environment variables. + (line 170) +* environment variable; PYTHONUSERBASE <2>: Module contents<3>. + (line 39) +* environment variable; PYTHONUSERBASE <3>: Module contents<3>. + (line 66) +* environment variable; PYTHONUSERSITE: test support script_helper — Utilities for the Python execution tests. + (line 26) +* environment variable; PYTHONUTF8: PEP 540 Forced UTF-8 Runtime Mode. + (line 6) +* environment variable; PYTHONUTF8 <1>: Miscellaneous options. + (line 252) +* environment variable; PYTHONUTF8 <2>: Environment variables. + (line 369) +* environment variable; PYTHONUTF8 <3>: Environment variables. + (line 386) +* environment variable; PYTHONUTF8 <4>: UTF-8 mode. (line 17) +* environment variable; PYTHONUTF8 <5>: sys — System-specific parameters and functions. + (line 1505) +* environment variable; PYTHONUTF8 <6>: Python Configuration. + (line 14) +* environment variable; PYTHONVERBOSE: Miscellaneous options. + (line 151) +* environment variable; PYTHONVERBOSE <1>: Environment variables. + (line 98) +* environment variable; PYTHONVERBOSE <2>: Global configuration variables. + (line 156) +* environment variable; PYTHONWARNINGS: warnings. (line 21) +* environment variable; PYTHONWARNINGS <1>: Other Language Changes<7>. + (line 158) +* environment variable; PYTHONWARNINGS <2>: Changes to the Handling of Deprecation Warnings. + (line 25) +* environment variable; PYTHONWARNINGS <3>: Interpreter Changes. + (line 6) +* environment variable; PYTHONWARNINGS <4>: Miscellaneous options. + (line 169) +* environment variable; PYTHONWARNINGS <5>: Environment variables. + (line 188) +* environment variable; PYTHONWARNINGS <6>: Describing Warning Filters. + (line 7) +* environment variable; PYTHONWARNINGS <7>: Describing Warning Filters. + (line 21) +* environment variable; PYTHONWARNINGS <8>: Default Warning Filter. + (line 7) +* environment variable; PYTHON_DOM: Module Contents<4>. (line 23) +* environment variable; PY_PYTHON: Customizing default Python versions. + (line 18) +* environment variable; SOURCE_DATE_EPOCH: py_compile<2>. (line 7) +* environment variable; SOURCE_DATE_EPOCH <1>: py_compile — Compile Python source files. + (line 65) +* environment variable; SOURCE_DATE_EPOCH <2>: py_compile — Compile Python source files. + (line 81) +* environment variable; SOURCE_DATE_EPOCH <3>: py_compile — Compile Python source files. + (line 85) +* environment variable; SOURCE_DATE_EPOCH <4>: Command-line use. + (line 85) +* environment variable; SSLKEYLOGFILE: Context creation. (line 33) +* environment variable; SSLKEYLOGFILE <1>: Context creation. (line 67) +* environment variable; SSL_CERT_FILE: LibreSSL support. (line 15) +* environment variable; SSL_CERT_PATH: LibreSSL support. (line 15) +* environment variable; SystemRoot: Popen Constructor. (line 234) +* environment variable; TCL_LIBRARY: How do I freeze Tkinter applications?. + (line 11) +* environment variable; TEMP: tempfile — Generate temporary files and directories. + (line 239) +* environment variable; TERM: Functions<3>. (line 440) +* environment variable; TERM <1>: Functions<3>. (line 467) +* environment variable; TK_LIBRARY: How do I freeze Tkinter applications?. + (line 11) +* environment variable; TMP: tempfile — Generate temporary files and directories. + (line 241) +* environment variable; TMPDIR: tempfile — Generate temporary files and directories. + (line 237) +* environment variable; TZ: Functions<2>. (line 508) +* environment variable; TZ <1>: Functions<2>. (line 509) +* environment variable; TZ <2>: Functions<2>. (line 517) +* environment variable; TZ <3>: Functions<2>. (line 522) +* environment variable; TZ <4>: Functions<2>. (line 524) +* environment variable; TZ <5>: Functions<2>. (line 584) +* environment variable; USER: getpass — Portable password input. + (line 38) +* environment variable; USERNAME: Process Parameters. (line 221) +* environment variable; USERNAME <1>: getpass — Portable password input. + (line 39) +* environment variable; USERPROFILE: os path. (line 14) +* environment variable; USERPROFILE <1>: Changes in the Python API. + (line 115) +* environment variable; USERPROFILE <2>: os path — Common pathname manipulations. + (line 146) +* environment variable; USERPROFILE <3>: Location and names of config files. + (line 67) +* environment variable; USER_BASE: New and Improved Modules. + (line 469) +* environment variable; VIRTUAL_ENV: Creating virtual environments. + (line 129) +* environment variables; deleting: Process Parameters. (line 501) +* environment variables; setting: Process Parameters. (line 318) +* EnvironmentError: Concrete exceptions. + (line 394) +* Environments; virtual: venv — Creation of virtual environments. + (line 10) +* EnvironmentVarGuard (class in test.support): test support — Utilities for the Python test suite. + (line 1025) +* ENXIO (in module errno): errno — Standard errno system symbols. + (line 51) +* eof (bz2.BZ2Decompressor attribute): Incremental de compression. + (line 72) +* eof (lzma.LZMADecompressor attribute): Compressing and decompressing data in memory. + (line 165) +* eof (shlex.shlex attribute): shlex Objects. (line 178) +* eof (ssl.MemoryBIO attribute): Memory BIO Support<2>. + (line 143) +* eof (zlib.Decompress attribute): zlib — Compression compatible with gzip. + (line 248) +* EOFError: Concrete exceptions. + (line 19) +* EOFError (built-in exception): File Objects. (line 41) +* eof_received() (asyncio.BufferedProtocol method): Buffered Streaming Protocols. + (line 42) +* eof_received() (asyncio.Protocol method): Streaming Protocols. + (line 30) +* EOPNOTSUPP (in module errno): errno — Standard errno system symbols. + (line 407) +* EOVERFLOW (in module errno): errno — Standard errno system symbols. + (line 327) +* EPERM (in module errno): errno — Standard errno system symbols. + (line 26) +* EPFNOSUPPORT (in module errno): errno — Standard errno system symbols. + (line 411) +* epilogue (email.message.EmailMessage attribute): email message Representing an email message. + (line 704) +* epilogue (email.message.Message attribute): email message Message Representing an email message using the compat32 API. + (line 714) +* EPIPE (in module errno): errno — Standard errno system symbols. + (line 155) +* epoch: time — Time access and conversions. + (line 18) +* epoll() (in module select): select — Waiting for I/O completion. + (line 50) +* EpollSelector (class in selectors): Classes<3>. (line 175) +* EPROTO (in module errno): errno — Standard errno system symbols. + (line 311) +* EPROTONOSUPPORT (in module errno): errno — Standard errno system symbols. + (line 399) +* EPROTOTYPE (in module errno): errno — Standard errno system symbols. + (line 391) +* eq() (in module operator): operator — Standard operators as functions. + (line 24) +* EQEQUAL (in module token): token — Constants used with Python parse trees. + (line 134) +* EQUAL (in module token): token — Constants used with Python parse trees. + (line 114) +* ERA (in module locale): locale — Internationalization services. + (line 264) +* ERANGE (in module errno): errno — Standard errno system symbols. + (line 163) +* erase() (curses.window method): Window Objects. (line 218) +* erasechar() (in module curses): Functions<3>. (line 114) +* ERA_D_FMT (in module locale): locale — Internationalization services. + (line 286) +* ERA_D_T_FMT (in module locale): locale — Internationalization services. + (line 281) +* ERA_T_FMT (in module locale): locale — Internationalization services. + (line 291) +* EREMCHG (in module errno): errno — Standard errno system symbols. + (line 339) +* EREMOTE (in module errno): errno — Standard errno system symbols. + (line 291) +* EREMOTEIO (in module errno): errno — Standard errno system symbols. + (line 511) +* ERESTART (in module errno): errno — Standard errno system symbols. + (line 367) +* erf() (in module math): Special functions. (line 6) +* erfc() (in module math): Special functions. (line 20) +* EROFS (in module errno): errno — Standard errno system symbols. + (line 147) +* ERR (in module curses): Constants<6>. (line 8) +* errcheck (ctypes._FuncPtr attribute): Foreign functions. (line 59) +* errcode (xmlrpc.client.ProtocolError attribute): ProtocolError Objects. + (line 17) +* errmsg (xmlrpc.client.ProtocolError attribute): ProtocolError Objects. + (line 21) +* errno (module): errno — Standard errno system symbols. + (line 6) +* errno (OSError attribute): Concrete exceptions. + (line 135) +* error: Module Contents. (line 353) +* error <1>: Functions and Exceptions. + (line 8) +* error <2>: copy — Shallow and deep copy operations. + (line 26) +* Error: Directory and files operations. + (line 400) +* error <3>: dbm — Interfaces to Unix “databases”. + (line 16) +* error <4>: dbm gnu — GNU’s reinterpretation of dbm. + (line 21) +* error <5>: dbm ndbm — Interface based on ndbm. + (line 21) +* error <6>: dbm dumb — Portable DBM implementation. + (line 24) +* Error <1>: Exceptions<4>. (line 10) +* error <7>: zlib — Compression compatible with gzip. + (line 24) +* Error <2>: Module Contents<3>. (line 265) +* Error <3>: Exceptions<5>. (line 6) +* Error <4>: Exceptions<6>. (line 8) +* error <8>: os — Miscellaneous operating system interfaces. + (line 42) +* error <9>: getopt — C-style parser for command line options. + (line 85) +* error <10>: Functions<3>. (line 8) +* error <11>: _thread — Low-level threading API. + (line 21) +* error <12>: Exceptions<11>. (line 6) +* error <13>: select — Waiting for I/O completion. + (line 24) +* Error <5>: Exceptions<14>. (line 9) +* Error <6>: binhex — Encode and decode binhex4 files. + (line 33) +* Error <7>: binascii — Convert between binary and ASCII. + (line 174) +* Error <8>: uu — Encode and decode uuencode files. + (line 51) +* error <14>: xml parsers expat — Fast XML parsing using Expat. + (line 31) +* Error <9>: webbrowser — Convenient Web-browser controller. + (line 44) +* error <15>: audioop — Manipulate raw audio data. + (line 26) +* Error <10>: sunau — Read and write Sun AU files. + (line 75) +* Error <11>: wave — Read and write WAV files. + (line 55) +* Error <12>: locale — Internationalization services. + (line 22) +* error <16>: resource — Resource usage information. + (line 17) +* error <17>: nis — Interface to Sun’s NIS Yellow Pages. + (line 54) +* error handling: Exceptions<2>. (line 6) +* error() (argparse.ArgumentParser method): Exiting methods. (line 18) +* error() (in module logging): Module-Level Functions. + (line 133) +* error() (logging.Logger method): Logger Objects. (line 229) +* error() (urllib.request.OpenerDirector method): OpenerDirector Objects. + (line 58) +* error() (xml.sax.handler.ErrorHandler method): ErrorHandler Objects. + (line 16) +* ErrorByteIndex (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 132) +* errorcode (in module errno): errno — Standard errno system symbols. + (line 13) +* ErrorCode (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 136) +* ErrorColumnNumber (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 142) +* ErrorHandler (class in xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 38) +* ErrorLineNumber (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 146) +* errors: Exceptions<2>. (line 6) +* errors (io.TextIOBase attribute): Text I/O<2>. (line 20) +* errors (unittest.TestLoader attribute): Loading and running tests. + (line 17) +* errors (unittest.TestResult attribute): Loading and running tests. + (line 237) +* Errors; logging: logging — Logging facility for Python. + (line 8) +* ErrorString() (in module xml.parsers.expat): xml parsers expat — Fast XML parsing using Expat. + (line 42) +* ERRORTOKEN (in module token): token — Constants used with Python parse trees. + (line 252) +* error_body (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 260) +* error_content_type (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 149) +* error_headers (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 253) +* error_leader() (shlex.shlex method): shlex Objects. (line 69) +* error_message_format (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 141) +* error_output() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 225) +* error_perm: ftplib — FTP protocol client. + (line 124) +* error_proto: ftplib — FTP protocol client. + (line 129) +* error_proto <1>: poplib — POP3 protocol client. + (line 83) +* error_received() (asyncio.DatagramProtocol method): Datagram Protocols. + (line 15) +* error_reply: ftplib — FTP protocol client. + (line 114) +* error_status (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 247) +* error_temp: ftplib — FTP protocol client. + (line 119) +* escape (shlex.shlex attribute): shlex Objects. (line 110) +* escape sequence: String and Bytes literals. + (line 72) +* escape() (in module glob): glob — Unix style pathname pattern expansion. + (line 62) +* escape() (in module html): html — HyperText Markup Language support. + (line 12) +* escape() (in module re): Module Contents. (line 316) +* escape() (in module xml.sax.saxutils): xml sax saxutils — SAX Utilities. + (line 14) +* escapechar (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 35) +* escapedquotes (shlex.shlex attribute): shlex Objects. (line 122) +* ESHUTDOWN (in module errno): errno — Standard errno system symbols. + (line 459) +* ESOCKTNOSUPPORT (in module errno): errno — Standard errno system symbols. + (line 403) +* ESPIPE (in module errno): errno — Standard errno system symbols. + (line 143) +* ESRCH (in module errno): errno — Standard errno system symbols. + (line 34) +* ESRMNT (in module errno): errno — Standard errno system symbols. + (line 303) +* ESTALE (in module errno): errno — Standard errno system symbols. + (line 491) +* ESTRPIPE (in module errno): errno — Standard errno system symbols. + (line 371) +* ETIME (in module errno): errno — Standard errno system symbols. + (line 275) +* ETIMEDOUT (in module errno): errno — Standard errno system symbols. + (line 467) +* Etiny() (decimal.Context method): Context objects. (line 213) +* ETOOMANYREFS (in module errno): errno — Standard errno system symbols. + (line 463) +* Etop() (decimal.Context method): Context objects. (line 219) +* ETXTBSY (in module errno): errno — Standard errno system symbols. + (line 131) +* EUCLEAN (in module errno): errno — Standard errno system symbols. + (line 495) +* EUNATCH (in module errno): errno — Standard errno system symbols. + (line 223) +* EUSERS (in module errno): errno — Standard errno system symbols. + (line 375) +* eval() (built-in function): Built-in Functions. (line 473) +* evaluation; order: Evaluation order. (line 6) +* Event (class in asyncio): Event. (line 6) +* Event (class in multiprocessing): Synchronization primitives. + (line 42) +* Event (class in threading): Event Objects. (line 14) +* event scheduling: sched — Event scheduler. + (line 8) +* event() (msilib.Control method): GUI classes. (line 16) +* Event() (multiprocessing.managers.SyncManager method): Managers. + (line 166) +* events (selectors.SelectorKey attribute): Classes<3>. (line 44) +* events (widgets): Bindings and Events. + (line 6) +* EWOULDBLOCK (in module errno): errno — Standard errno system symbols. + (line 191) +* Example (class in doctest): Example Objects. (line 6) +* example (doctest.DocTestFailure attribute): Debugging. (line 200) +* example (doctest.UnexpectedException attribute): Debugging. + (line 221) +* examples (doctest.DocTest attribute): DocTest Objects. (line 17) +* excel (class in csv): Module Contents<3>. (line 183) +* excel_tab (class in csv): Module Contents<3>. (line 189) +* except (2to3 fixer): Fixers. (line 82) +* excepthook() (in module sys): cgitb — Traceback manager for CGI scripts. + (line 33) +* excepthook() (in module sys) <1>: sys — System-specific parameters and functions. + (line 309) +* excepthook() (in module threading): threading — Thread-based parallelism. + (line 46) +* exception: Exceptions<2>. (line 6) +* exception <1>: The raise statement. + (line 6) +* Exception: Base classes. (line 37) +* EXCEPTION (in module tkinter): File Handlers. (line 41) +* exception handler: Exceptions<2>. (line 6) +* exception() (asyncio.Future method): Future Object. (line 117) +* exception() (asyncio.Task method): Task Object. (line 134) +* exception() (concurrent.futures.Future method): Future Objects. + (line 54) +* exception() (in module logging): Module-Level Functions. + (line 143) +* exception() (logging.Logger method): Logger Objects. (line 244) +* exception; AssertionError: The assert statement. + (line 21) +* exception; AttributeError: Attribute references. + (line 10) +* exception; chaining: The raise statement. + (line 30) +* exception; GeneratorExit: Generator-iterator methods. + (line 47) +* exception; GeneratorExit <1>: Asynchronous generator-iterator methods. + (line 55) +* exception; handler: The standard type hierarchy. + (line 757) +* exception; ImportError: The import statement. + (line 6) +* exception; NameError: Identifiers Names. (line 10) +* exception; StopAsyncIteration: Asynchronous generator-iterator methods. + (line 10) +* exception; StopIteration: Generator-iterator methods. + (line 12) +* exception; StopIteration <1>: The yield statement. + (line 6) +* exception; TypeError: Unary arithmetic and bitwise operations. + (line 19) +* exception; ValueError: Shifting operations. + (line 15) +* exception; ZeroDivisionError: Binary arithmetic operations. + (line 28) +* exceptions; in CGI scripts: cgitb — Traceback manager for CGI scripts. + (line 8) +* exclusive; or: Binary bitwise operations. + (line 15) +* exc_info (doctest.UnexpectedException attribute): Debugging. + (line 225) +* exc_info (in module sys): The standard type hierarchy. + (line 757) +* exc_info() (in module sys): sys — System-specific parameters and functions. + (line 352) +* exc_info() (in module sys) <1>: Exceptions<18>. (line 45) +* exc_msg (doctest.Example attribute): Example Objects. (line 31) +* exc_type (traceback.TracebackException attribute): TracebackException Objects. + (line 38) +* EXDEV (in module errno): errno — Standard errno system symbols. + (line 99) +* exec (2to3 fixer): Fixers. (line 86) +* exec() (built-in function): Built-in Functions. (line 519) +* execfile (2to3 fixer): Fixers. (line 90) +* execl() (in module os): Process Management. (line 53) +* execle() (in module os): Process Management. (line 53) +* execlp() (in module os): Process Management. (line 53) +* execlpe() (in module os): Process Management. (line 53) +* executable (in module sys): sys — System-specific parameters and functions. + (line 391) +* executable (in module sys) <1>: Process-wide parameters. + (line 111) +* Executable Zip Files: zipapp — Manage executable Python zip archives. + (line 10) +* executable_filename() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 451) +* execute() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 481) +* execute() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 109) +* Execute() (msilib.View method): View Objects. (line 6) +* execute() (sqlite3.Connection method): Connection Objects. (line 52) +* execute() (sqlite3.Cursor method): Cursor Objects. (line 11) +* executemany() (sqlite3.Connection method): Connection Objects. + (line 59) +* executemany() (sqlite3.Cursor method): Cursor Objects. (line 46) +* executescript() (sqlite3.Connection method): Connection Objects. + (line 66) +* executescript() (sqlite3.Cursor method): Cursor Objects. (line 100) +* execution model: Execution model. (line 6) +* execution; frame: Structure of a program. + (line 16) +* execution; frame <1>: Class definitions. (line 6) +* execution; stack: The standard type hierarchy. + (line 757) +* ExecutionLoader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 452) +* Executor (class in concurrent.futures): Executor Objects. (line 6) +* execv() (in module os): Process Management. (line 53) +* execve() (in module os): Process Management. (line 53) +* execvp() (in module os): Process Management. (line 53) +* execvpe() (in module os): Process Management. (line 53) +* exec_module (C function): Multi-phase initialization. + (line 103) +* exec_module() (importlib.abc.InspectLoader method): importlib abc – Abstract base classes related to import. + (line 439) +* exec_module() (importlib.abc.Loader method): importlib abc – Abstract base classes related to import. + (line 182) +* exec_module() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 586) +* exec_module() (importlib.machinery.ExtensionFileLoader method): importlib machinery – Importers and path hooks. + (line 326) +* exec_prefix: Python-related paths and files. + (line 7) +* exec_prefix <1>: Include Files. (line 37) +* exec_prefix <2>: Include Files. (line 48) +* EXEC_PREFIX (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 23) +* exec_prefix (in module sys): sys — System-specific parameters and functions. + (line 374) +* ExFileSelectBox (class in tkinter.tix): File Selectors. (line 34) +* EXFULL (in module errno): errno — Standard errno system symbols. + (line 243) +* exists() (in module os.path): os path — Common pathname manipulations. + (line 114) +* exists() (pathlib.Path method): Methods<2>. (line 58) +* exists() (tkinter.ttk.Treeview method): ttk Treeview. (line 95) +* exists() (zipfile.Path method): Path Objects. (line 47) +* exit (built-in variable): Constants added by the site module. + (line 11) +* exit(): Process Control. (line 17) +* exit() (argparse.ArgumentParser method): Exiting methods. (line 6) +* exit() (in module sys): sys — System-specific parameters and functions. + (line 398) +* exit() (in module _thread): _thread — Low-level threading API. + (line 63) +* exitcode (multiprocessing.Process attribute): Process and exceptions. + (line 112) +* exitfunc (2to3 fixer): Fixers. (line 96) +* exitonclick() (in module turtle): Methods specific to Screen not inherited from TurtleScreen. + (line 10) +* ExitStack (class in contextlib): Utilities. (line 342) +* exp() (decimal.Context method): Context objects. (line 284) +* exp() (decimal.Decimal method): Decimal objects. (line 232) +* exp() (in module cmath): Power and logarithmic functions<2>. + (line 6) +* exp() (in module math): Power and logarithmic functions. + (line 6) +* expand() (re.Match method): Match Objects. (line 17) +* ExpandEnvironmentStrings() (in module winreg): Functions<11>. + (line 233) +* expandNode() (xml.dom.pulldom.DOMEventStream method): DOMEventStream Objects. + (line 21) +* expandtabs() (bytearray method): Bytes and Bytearray Operations. + (line 414) +* expandtabs() (bytes method): Bytes and Bytearray Operations. + (line 414) +* expandtabs() (str method): String Methods<2>. (line 82) +* expanduser() (in module os.path): os path — Common pathname manipulations. + (line 135) +* expanduser() (pathlib.Path method): Methods<2>. (line 75) +* expandvars() (in module os.path): os path — Common pathname manipulations. + (line 158) +* expand_tabs (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 156) +* Expat: xml parsers expat — Fast XML parsing using Expat. + (line 12) +* ExpatError: xml parsers expat — Fast XML parsing using Expat. + (line 25) +* expect() (telnetlib.Telnet method): Telnet Objects. (line 121) +* expected (asyncio.IncompleteReadError attribute): Exceptions<9>. + (line 50) +* expectedFailure() (in module unittest): Skipping tests and expected failures. + (line 98) +* expectedFailures (unittest.TestResult attribute): Loading and running tests. + (line 257) +* expires (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 49) +* exploded (ipaddress.IPv4Address attribute): Address objects. + (line 54) +* exploded (ipaddress.IPv4Network attribute): Network objects. + (line 109) +* exploded (ipaddress.IPv6Address attribute): Address objects. + (line 155) +* exploded (ipaddress.IPv6Network attribute): Network objects. + (line 318) +* expm1() (in module math): Power and logarithmic functions. + (line 12) +* expovariate() (in module random): Real-valued distributions. + (line 38) +* expr() (in module parser): Creating ST Objects. + (line 10) +* expression: Expressions. (line 6) +* expression <1>: Glossary. (line 397) +* expression; list: Expression lists. (line 6) +* expression; list <1>: Expression statements. + (line 6) +* expression; list <2>: Expression statements. + (line 6) +* expression; statement: Expression statements. + (line 6) +* expunge() (imaplib.IMAP4 method): IMAP4 Objects. (line 93) +* extend() (array.array method): array — Efficient arrays of numeric values. + (line 148) +* extend() (collections.deque method): deque objects. (line 59) +* extend() (sequence method): Mutable Sequence Types. + (line 16) +* extend() (xml.etree.ElementTree.Element method): Element Objects. + (line 93) +* ExtendedContext (class in decimal): Context objects. (line 58) +* ExtendedInterpolation (class in configparser): Interpolation of values. + (line 40) +* EXTENDED_ARG (opcode): Python Bytecode Instructions. + (line 868) +* extendleft() (collections.deque method): deque objects. (line 64) +* extend_path() (in module pkgutil): pkgutil — Package extension utility. + (line 19) +* Extension (class in distutils.core): distutils core — Core Distutils functionality. + (line 176) +* extension module: Glossary. (line 408) +* extension; module: The standard type hierarchy. + (line 6) +* ExtensionFileLoader (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 300) +* extensions_map (http.server.SimpleHTTPRequestHandler attribute): http server — HTTP servers. + (line 349) +* EXTENSION_SUFFIXES (in module importlib.machinery): importlib machinery – Importers and path hooks. + (line 50) +* External Data Representation: Data stream format. (line 6) +* External Data Representation <1>: xdrlib — Encode and decode XDR data. + (line 8) +* ExternalClashError: Exceptions<14>. (line 25) +* ExternalEntityParserCreate() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 44) +* ExternalEntityRefHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 333) +* external_attr (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 127) +* extra (zipfile.ZipInfo attribute): ZipInfo Objects. (line 93) +* extract() (tarfile.TarFile method): TarFile Objects. (line 162) +* extract() (traceback.StackSummary class method): StackSummary Objects. + (line 13) +* extract() (zipfile.ZipFile method): ZipFile Objects. (line 158) +* extractall() (tarfile.TarFile method): TarFile Objects. (line 136) +* extractall() (zipfile.ZipFile method): ZipFile Objects. (line 185) +* ExtractError: tarfile — Read and write tar archive files. + (line 201) +* extractfile() (tarfile.TarFile method): TarFile Objects. (line 190) +* extract_cookies() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 30) +* extract_stack() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 96) +* extract_tb() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 83) +* extract_version (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 107) +* extsep (in module os): Miscellaneous System Information. + (line 109) +* EX_CANTCREAT (in module os): Process Management. (line 202) +* EX_CONFIG (in module os): Process Management. (line 238) +* EX_DATAERR (in module os): Process Management. (line 151) +* EX_IOERR (in module os): Process Management. (line 209) +* EX_NOHOST (in module os): Process Management. (line 170) +* EX_NOINPUT (in module os): Process Management. (line 157) +* EX_NOPERM (in module os): Process Management. (line 231) +* EX_NOTFOUND (in module os): Process Management. (line 245) +* EX_NOUSER (in module os): Process Management. (line 164) +* EX_OK (in module os): Process Management. (line 138) +* EX_OSERR (in module os): Process Management. (line 188) +* EX_OSFILE (in module os): Process Management. (line 195) +* EX_PROTOCOL (in module os): Process Management. (line 224) +* EX_SOFTWARE (in module os): Process Management. (line 182) +* EX_TEMPFAIL (in module os): Process Management. (line 216) +* EX_UNAVAILABLE (in module os): Process Management. (line 176) +* EX_USAGE (in module os): Process Management. (line 144) +* f"; formatted string literal: String and Bytes literals. + (line 62) +* f’; formatted string literal: String and Bytes literals. + (line 62) +* f-string: String literal concatenation. + (line 24) +* f-string <1>: Glossary. (line 413) +* fabs() (in module math): Number-theoretic and representation functions. + (line 36) +* factorial() (in module math): Number-theoretic and representation functions. + (line 40) +* factory() (importlib.util.LazyLoader class method): importlib util – Utility code for importers. + (line 255) +* fail() (unittest.TestCase method): Test cases. (line 664) +* failfast (unittest.TestResult attribute): Loading and running tests. + (line 288) +* failureException (unittest.TestCase attribute): Test cases. + (line 669) +* failures (unittest.TestResult attribute): Loading and running tests. + (line 243) +* FAIL_FAST (in module doctest): Option Flags. (line 151) +* FakePath (class in test.support): test support — Utilities for the Python test suite. + (line 1112) +* False: The standard type hierarchy. + (line 93) +* false: Truth Value Testing. + (line 6) +* False <1>: Truth Value Testing. + (line 23) +* False <2>: Boolean Values. (line 15) +* False (Built-in object): Truth Value Testing. + (line 15) +* False (built-in variable): Built-in Constants. (line 8) +* family (socket.socket attribute): Socket Objects. (line 595) +* FancyGetopt (class in distutils.fancy_getopt): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 36) +* FancyURLopener (class in urllib.request): Legacy interface. + (line 147) +* fancy_getopt() (in module distutils.fancy_getopt): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 20) +* fast (pickle.Pickler attribute): Module Interface. (line 214) +* FastChildWatcher (class in asyncio): Process Watchers. (line 132) +* fatalError() (xml.sax.handler.ErrorHandler method): ErrorHandler Objects. + (line 24) +* Fault (class in xmlrpc.client): Fault Objects. (line 6) +* faultCode (xmlrpc.client.Fault attribute): Fault Objects. (line 11) +* faulthandler (module): faulthandler — Dump the Python traceback. + (line 6) +* faultString (xmlrpc.client.Fault attribute): Fault Objects. + (line 15) +* fchdir() (in module os): Files and Directories. + (line 268) +* fchmod() (in module os): File Descriptor Operations. + (line 97) +* fchown() (in module os): File Descriptor Operations. + (line 108) +* FCICreate() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 26) +* fcntl (module): fcntl — The fcntl and ioctl system calls. + (line 6) +* fcntl() (in module fcntl): fcntl — The fcntl and ioctl system calls. + (line 28) +* fd (selectors.SelectorKey attribute): Classes<3>. (line 40) +* fd() (in module turtle): Turtle motion. (line 6) +* fdatasync() (in module os): File Descriptor Operations. + (line 120) +* fdopen() (in module os): File Object Creation. + (line 9) +* fd_count() (in module test.support): test support — Utilities for the Python test suite. + (line 242) +* Feature (class in msilib): Features<3>. (line 6) +* feature_external_ges (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 76) +* feature_external_pes (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 83) +* feature_namespaces (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 48) +* feature_namespace_prefixes (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 55) +* feature_string_interning (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 62) +* feature_validation (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 69) +* feed() (email.parser.BytesFeedParser method): FeedParser API. + (line 56) +* feed() (html.parser.HTMLParser method): HTMLParser Methods. + (line 8) +* feed() (xml.etree.ElementTree.XMLParser method): XMLParser Objects. + (line 27) +* feed() (xml.etree.ElementTree.XMLPullParser method): XMLPullParser Objects. + (line 18) +* feed() (xml.sax.xmlreader.IncrementalParser method): IncrementalParser Objects. + (line 9) +* FeedParser (class in email.parser): FeedParser API. (line 71) +* fetch() (imaplib.IMAP4 method): IMAP4 Objects. (line 99) +* Fetch() (msilib.View method): View Objects. (line 18) +* fetchall() (sqlite3.Cursor method): Cursor Objects. (line 162) +* fetchmany() (sqlite3.Cursor method): Cursor Objects. (line 142) +* fetchone() (sqlite3.Cursor method): Cursor Objects. (line 137) +* fflags (select.kevent attribute): Kevent Objects. (line 90) +* Field (class in dataclasses): Module-level decorators classes and functions. + (line 241) +* field() (in module dataclasses): Module-level decorators classes and functions. + (line 153) +* fieldnames (csv.csvreader attribute): Reader Objects. (line 30) +* fields (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 94) +* fields() (in module dataclasses): Module-level decorators classes and functions. + (line 261) +* field_size_limit() (in module csv): Module Contents<3>. (line 95) +* file (pyclbr.Class attribute): Class Objects<2>. (line 9) +* file (pyclbr.Function attribute): Function Objects. (line 9) +* file object: Glossary. (line 419) +* file object; io module: Overview. (line 6) +* file object; open() built-in function: Built-in Functions. (line 1023) +* file-like object: Glossary. (line 435) +* file; byte-code: py_compile — Compile Python source files. + (line 8) +* file; byte-code <1>: imp — Access the import internals. + (line 19) +* file; copying: shutil — High-level file operations. + (line 8) +* file; large files: Large File Support. (line 6) +* file; mime.types: mimetypes — Map filenames to MIME types. + (line 124) +* file; modes: Built-in Functions. (line 1047) +* filecmp (module): filecmp — File and Directory Comparisons. + (line 6) +* fileConfig() (in module logging.config): Configuration functions. + (line 61) +* FileCookieJar (class in http.cookiejar): http cookiejar — Cookie handling for HTTP clients. + (line 56) +* FileEntry (class in tkinter.tix): File Selectors. (line 50) +* FileExistsError: OS exceptions. (line 61) +* FileFinder (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 160) +* FileHandler (class in logging): FileHandler. (line 10) +* FileHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 416) +* FileInput (class in fileinput): fileinput — Iterate over lines from multiple input streams. + (line 134) +* fileinput (module): fileinput — Iterate over lines from multiple input streams. + (line 6) +* FileIO (class in io): Raw File I/O. (line 6) +* filelineno() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 99) +* FileLoader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 471) +* filemode() (in module stat): stat — Interpreting stat results. + (line 121) +* filename (doctest.DocTest attribute): DocTest Objects. (line 36) +* filename (http.cookiejar.FileCookieJar attribute): CookieJar and FileCookieJar Objects. + (line 153) +* filename (OSError attribute): Concrete exceptions. + (line 157) +* filename (traceback.TracebackException attribute): TracebackException Objects. + (line 42) +* filename (tracemalloc.Frame attribute): Frame. (line 13) +* filename (zipfile.ZipFile attribute): ZipFile Objects. (line 298) +* filename (zipfile.ZipInfo attribute): ZipInfo Objects. (line 50) +* filename() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 81) +* filename2 (OSError attribute): Concrete exceptions. + (line 157) +* filenames; pathname expansion: glob — Unix style pathname pattern expansion. + (line 8) +* filenames; wildcard expansion: fnmatch — Unix filename pattern matching. + (line 8) +* filename_only (in module tabnanny): tabnanny — Detection of ambiguous indentation. + (line 31) +* filename_pattern (tracemalloc.Filter attribute): Filter. (line 53) +* fileno() (http.client.HTTPResponse method): HTTPResponse Objects. + (line 36) +* fileno() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 86) +* fileno() (io.IOBase method): I/O Base Classes. (line 63) +* fileno() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 31) +* fileno() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 28) +* fileno() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 14) +* fileno() (select.devpoll method): /dev/poll Polling Objects. + (line 24) +* fileno() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 73) +* fileno() (select.kqueue method): Kqueue Objects. (line 14) +* fileno() (selectors.DevpollSelector method): Classes<3>. (line 188) +* fileno() (selectors.EpollSelector method): Classes<3>. (line 179) +* fileno() (selectors.KqueueSelector method): Classes<3>. (line 199) +* fileno() (socket.socket method): Socket Objects. (line 110) +* fileno() (socketserver.BaseServer method): Server Objects<2>. + (line 14) +* fileno() (telnetlib.Telnet method): Telnet Objects. (line 97) +* FileNotFoundError: OS exceptions. (line 66) +* fileobj (selectors.SelectorKey attribute): Classes<3>. (line 36) +* FileSelectBox (class in tkinter.tix): File Selectors. (line 42) +* FileType (class in argparse): FileType objects. (line 6) +* FileWrapper (class in wsgiref.util): wsgiref util – WSGI environment utilities. + (line 114) +* FILE_ATTRIBUTE_ARCHIVE (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_COMPRESSED (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_DEVICE (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_DIRECTORY (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_ENCRYPTED (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_HIDDEN (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_INTEGRITY_STREAM (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_NORMAL (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_NOT_CONTENT_INDEXED (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_NO_SCRUB_DATA (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_OFFLINE (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_READONLY (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_REPARSE_POINT (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_SPARSE_FILE (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_SYSTEM (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_TEMPORARY (in module stat): stat — Interpreting stat results. + (line 394) +* FILE_ATTRIBUTE_VIRTUAL (in module stat): stat — Interpreting stat results. + (line 394) +* file_created() (built-in function): The Postinstallation script. + (line 23) +* file_dispatcher (class in asyncore): asyncore — Asynchronous socket handler. + (line 256) +* file_open() (urllib.request.FileHandler method): FileHandler Objects. + (line 6) +* file_size (zipfile.ZipInfo attribute): ZipInfo Objects. (line 143) +* file_wrapper (class in asyncore): asyncore — Asynchronous socket handler. + (line 266) +* fill() (in module textwrap): textwrap — Text wrapping and filling. + (line 29) +* fill() (textwrap.TextWrapper method): textwrap — Text wrapping and filling. + (line 285) +* fillcolor() (in module turtle): Color control. (line 58) +* filling() (in module turtle): Filling. (line 6) +* filter (2to3 fixer): Fixers. (line 101) +* Filter (class in logging): Filter Objects. (line 13) +* Filter (class in tracemalloc): Filter. (line 6) +* filter (select.kevent attribute): Kevent Objects. (line 15) +* filter() (built-in function): Built-in Functions. (line 565) +* filter() (in module curses): Functions<3>. (line 121) +* filter() (in module fnmatch): fnmatch — Unix filename pattern matching. + (line 65) +* filter() (logging.Filter method): Filter Objects. (line 20) +* filter() (logging.Handler method): Handler Objects. (line 60) +* filter() (logging.Logger method): Logger Objects. (line 259) +* filterfalse() (in module itertools): Itertool functions. (line 284) +* filterwarnings() (in module warnings): Available Functions. + (line 77) +* FILTER_DIR (in module unittest.mock): FILTER_DIR. (line 6) +* filter_traces() (tracemalloc.Snapshot method): Snapshot. (line 35) +* Final (in module typing): Classes functions and decorators. + (line 931) +* final() (in module typing): Classes functions and decorators. + (line 691) +* finalization, of objects: Finalization and De-allocation. + (line 6) +* finalize (class in weakref): weakref — Weak references. + (line 226) +* finalizer: Basic customization. + (line 56) +* finalize_options() (distutils.cmd.Command method): Creating a new Distutils command. + (line 30) +* find() (bytearray method): Bytes and Bytearray Operations. + (line 78) +* find() (bytes method): Bytes and Bytearray Operations. + (line 78) +* find() (doctest.DocTestFinder method): DocTestFinder objects. + (line 32) +* find() (in module gettext): Class-based API. (line 14) +* find() (mmap.mmap method): mmap — Memory-mapped file support. + (line 177) +* find() (str method): String Methods<2>. (line 103) +* find() (xml.etree.ElementTree.Element method): Element Objects. + (line 101) +* find() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 21) +* findall() (in module re): Module Contents. (line 219) +* findall() (re.Pattern method): Regular Expression Objects. + (line 79) +* findall() (xml.etree.ElementTree.Element method): Element Objects. + (line 110) +* findall() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 26) +* findCaller() (logging.Logger method): Logger Objects. (line 276) +* finder: Finders and loaders. + (line 6) +* finder <1>: Glossary. (line 439) +* Finder (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 28) +* finder; find_spec: The meta path. (line 6) +* findfactor() (in module audioop): audioop — Manipulate raw audio data. + (line 79) +* findfile() (in module test.support): test support — Utilities for the Python test suite. + (line 228) +* findfit() (in module audioop): audioop — Manipulate raw audio data. + (line 88) +* finditer() (in module re): Module Contents. (line 231) +* finditer() (re.Pattern method): Regular Expression Objects. + (line 85) +* findlabels() (in module dis): Analysis functions. (line 136) +* findlinestarts() (in module dis): Analysis functions. (line 125) +* findmatch() (in module mailcap): mailcap — Mailcap file handling. + (line 24) +* findmax() (in module audioop): audioop — Manipulate raw audio data. + (line 99) +* findtext() (xml.etree.ElementTree.Element method): Element Objects. + (line 119) +* findtext() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 31) +* find_class() (pickle protocol): Restricting Globals. + (line 6) +* find_class() (pickle.Unpickler method): Module Interface. (line 286) +* find_library() (in module ctypes.util): Utility functions. (line 89) +* find_library_file() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 204) +* find_loader() (importlib.abc.PathEntryFinder method): importlib abc – Abstract base classes related to import. + (line 119) +* find_loader() (importlib.machinery.FileFinder method): importlib machinery – Importers and path hooks. + (line 197) +* find_loader() (in module importlib): Functions<10>. (line 40) +* find_loader() (in module pkgutil): pkgutil — Package extension utility. + (line 77) +* find_longest_match() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 64) +* find_module() (imp.NullImporter method): imp — Access the import internals. + (line 355) +* find_module() (importlib.abc.Finder method): importlib abc – Abstract base classes related to import. + (line 35) +* find_module() (importlib.abc.MetaPathFinder method): importlib abc – Abstract base classes related to import. + (line 67) +* find_module() (importlib.abc.PathEntryFinder method): importlib abc – Abstract base classes related to import. + (line 143) +* find_module() (importlib.machinery.PathFinder class method): importlib machinery – Importers and path hooks. + (line 139) +* find_module() (in module imp): imp — Access the import internals. + (line 41) +* find_module() (zipimport.zipimporter method): zipimporter Objects. + (line 19) +* find_msvcrt() (in module ctypes.util): Utility functions. (line 98) +* find_spec() (importlib.abc.MetaPathFinder method): importlib abc – Abstract base classes related to import. + (line 52) +* find_spec() (importlib.abc.PathEntryFinder method): importlib abc – Abstract base classes related to import. + (line 106) +* find_spec() (importlib.machinery.FileFinder method): importlib machinery – Importers and path hooks. + (line 190) +* find_spec() (importlib.machinery.PathFinder class method): importlib machinery – Importers and path hooks. + (line 116) +* find_spec() (in module importlib.util): importlib util – Utility code for importers. + (line 97) +* find_unused_port() (in module test.support): test support — Utilities for the Python test suite. + (line 888) +* find_user_password() (urllib.request.HTTPPasswordMgr method): HTTPPasswordMgr Objects. + (line 16) +* find_user_password() (urllib.request.HTTPPasswordMgrWithPriorAuth method): HTTPPasswordMgrWithPriorAuth Objects. + (line 19) +* finish() (socketserver.BaseRequestHandler method): Request Handler Objects. + (line 35) +* finish_request() (socketserver.BaseServer method): Server Objects<2>. + (line 125) +* firstChild (xml.dom.Node attribute): Node Objects. (line 54) +* firstkey() (dbm.gnu.gdbm method): dbm gnu — GNU’s reinterpretation of dbm. + (line 85) +* firstweekday() (in module calendar): calendar — General calendar-related functions. + (line 297) +* fix_missing_locations() (in module ast): ast Helpers. (line 81) +* fix_sentence_endings (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 211) +* Flag (class in enum): Module Contents<2>. (line 28) +* flags (in module sys): sys — System-specific parameters and functions. + (line 429) +* flags (re.Pattern attribute): Regular Expression Objects. + (line 101) +* flags (select.kevent attribute): Kevent Objects. (line 52) +* flag_bits (zipfile.ZipInfo attribute): ZipInfo Objects. (line 115) +* flash() (in module curses): Functions<3>. (line 132) +* flatten() (email.generator.BytesGenerator method): email generator Generating MIME documents. + (line 80) +* flatten() (email.generator.Generator method): email generator Generating MIME documents. + (line 175) +* flattening; objects: pickle — Python object serialization. + (line 8) +* float (built-in class): Built-in Functions. (line 582) +* floating point literal: Numeric literals. (line 6) +* floating point; literals: Numeric Types — int float complex. + (line 19) +* floating point; number: The standard type hierarchy. + (line 107) +* FloatingPointError: Concrete exceptions. + (line 26) +* FloatOperation (class in decimal): Signals. (line 100) +* float_info (in module sys): sys — System-specific parameters and functions. + (line 499) +* float_repr_style (in module sys): sys — System-specific parameters and functions. + (line 578) +* flock() (in module fcntl): fcntl — The fcntl and ioctl system calls. + (line 108) +* floor division: Glossary. (line 450) +* floor() (in module math): Numeric Types — int float complex. + (line 104) +* floor() (in module math) <1>: Number-theoretic and representation functions. + (line 45) +* floordiv() (in module operator): operator — Standard operators as functions. + (line 88) +* flush() (bz2.BZ2Compressor method): Incremental de compression. + (line 25) +* flush() (formatter.writer method): The Writer Interface. + (line 13) +* flush() (io.BufferedWriter method): Buffered Streams. (line 121) +* flush() (io.IOBase method): I/O Base Classes. (line 69) +* flush() (logging.Handler method): Handler Objects. (line 68) +* flush() (logging.handlers.BufferingHandler method): MemoryHandler. + (line 30) +* flush() (logging.handlers.MemoryHandler method): MemoryHandler. + (line 60) +* flush() (logging.StreamHandler method): StreamHandler. (line 25) +* flush() (lzma.LZMACompressor method): Compressing and decompressing data in memory. + (line 91) +* flush() (mailbox.Mailbox method): Mailbox objects. (line 242) +* flush() (mailbox.Maildir method): Maildir. (line 102) +* flush() (mailbox.MH method): MH. (line 99) +* flush() (mmap.mmap method): mmap — Memory-mapped file support. + (line 187) +* flush() (zlib.Compress method): zlib — Compression compatible with gzip. + (line 207) +* flush() (zlib.Decompress method): zlib — Compression compatible with gzip. + (line 279) +* flushinp() (in module curses): Functions<3>. (line 139) +* FlushKey() (in module winreg): Functions<11>. (line 244) +* flush_headers() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 274) +* flush_softspace() (formatter.formatter method): The Formatter Interface. + (line 83) +* fma() (decimal.Context method): Context objects. (line 288) +* fma() (decimal.Decimal method): Decimal objects. (line 270) +* fmean() (in module statistics): Function details. (line 50) +* fmod() (in module math): Number-theoretic and representation functions. + (line 51) +* FMT_BINARY (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 200) +* FMT_XML (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 194) +* fnmatch (module): fnmatch — Unix filename pattern matching. + (line 6) +* fnmatch() (in module fnmatch): fnmatch — Unix filename pattern matching. + (line 40) +* fnmatchcase() (in module fnmatch): fnmatch — Unix filename pattern matching. + (line 59) +* focus() (tkinter.ttk.Treeview method): ttk Treeview. (line 99) +* fold (datetime.datetime attribute): datetime Objects. (line 298) +* fold (datetime.time attribute): time Objects. (line 71) +* fold() (email.headerregistry.BaseHeader method): email headerregistry Custom Header Objects. + (line 72) +* fold() (email.policy.Compat32 method): email policy Policy Objects. + (line 584) +* fold() (email.policy.EmailPolicy method): email policy Policy Objects. + (line 450) +* fold() (email.policy.Policy method): email policy Policy Objects. + (line 319) +* fold_binary() (email.policy.Compat32 method): email policy Policy Objects. + (line 592) +* fold_binary() (email.policy.EmailPolicy method): email policy Policy Objects. + (line 470) +* fold_binary() (email.policy.Policy method): email policy Policy Objects. + (line 334) +* for; in comprehensions: Displays for lists sets and dictionaries. + (line 14) +* forget() (in module test.support): test support — Utilities for the Python test suite. + (line 162) +* forget() (tkinter.ttk.Notebook method): ttk Notebook. (line 18) +* fork() (in module os): Process Management. (line 251) +* fork() (in module pty): pty — Pseudo-terminal utilities. + (line 20) +* ForkingMixIn (class in socketserver): Server Creation Notes. + (line 28) +* ForkingTCPServer (class in socketserver): Server Creation Notes. + (line 62) +* ForkingUDPServer (class in socketserver): Server Creation Notes. + (line 62) +* forkpty() (in module os): Process Management. (line 270) +* Form (class in tkinter.tix): Form Geometry Manager. + (line 9) +* format (memoryview attribute): Memory Views. (line 416) +* format (multiprocessing.shared_memory.ShareableList attribute): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 273) +* format (struct.Struct attribute): Classes<2>. (line 55) +* format() (built-in function): Built-in Functions. (line 643) +* format() (built-in function); __str__() (object method): Basic customization. + (line 129) +* format() (in module locale): locale — Internationalization services. + (line 407) +* format() (logging.Formatter method): Formatter Objects. (line 49) +* format() (logging.Handler method): Handler Objects. (line 102) +* format() (pprint.PrettyPrinter method): PrettyPrinter Objects. + (line 40) +* format() (str method): String Methods<2>. (line 117) +* format() (string.Formatter method): Custom String Formatting. + (line 17) +* format() (traceback.StackSummary method): StackSummary Objects. + (line 36) +* format() (traceback.TracebackException method): TracebackException Objects. + (line 73) +* format() (tracemalloc.Traceback method): Traceback. (line 25) +* formataddr() (in module email.utils): email utils Miscellaneous utilities. + (line 67) +* formatargspec() (in module inspect): Classes and functions<2>. + (line 92) +* formatargvalues() (in module inspect): Classes and functions<2>. + (line 121) +* formatdate() (in module email.utils): email utils Miscellaneous utilities. + (line 139) +* FormatError: Exceptions<14>. (line 32) +* FormatError() (in module ctypes): Utility functions. (line 108) +* formatException() (logging.Formatter method): Formatter Objects. + (line 110) +* formatmonth() (calendar.HTMLCalendar method): calendar — General calendar-related functions. + (line 169) +* formatmonth() (calendar.TextCalendar method): calendar — General calendar-related functions. + (line 134) +* formatStack() (logging.Formatter method): Formatter Objects. + (line 118) +* formatted string literal: String literal concatenation. + (line 24) +* Formatter (class in logging): Formatter Objects. (line 26) +* Formatter (class in string): Custom String Formatting. + (line 13) +* formatter (module): formatter — Generic output formatting. + (line 6) +* formatTime() (logging.Formatter method): Formatter Objects. + (line 73) +* formatting, string (%): printf-style String Formatting. + (line 6) +* formatting; bytearray (%): printf-style Bytes Formatting. + (line 6) +* formatting; bytes (%): printf-style Bytes Formatting. + (line 6) +* formatwarning() (in module warnings): Available Functions. + (line 68) +* formatyear() (calendar.HTMLCalendar method): calendar — General calendar-related functions. + (line 175) +* formatyear() (calendar.TextCalendar method): calendar — General calendar-related functions. + (line 148) +* formatyearpage() (calendar.HTMLCalendar method): calendar — General calendar-related functions. + (line 180) +* format_datetime() (in module email.utils): email utils Miscellaneous utilities. + (line 160) +* format_exc() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 137) +* format_exception() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 124) +* format_exception_only() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 113) +* format_exception_only() (traceback.TracebackException method): TracebackException Objects. + (line 88) +* format_field() (string.Formatter method): Custom String Formatting. + (line 102) +* format_help() (argparse.ArgumentParser method): Printing help. + (line 30) +* format_list() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 103) +* format_map() (str method): String Methods<2>. (line 147) +* format_stack() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 146) +* format_stack_entry() (bdb.Bdb method): bdb — Debugger framework. + (line 344) +* format_string() (in module locale): locale — Internationalization services. + (line 390) +* format_tb() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 142) +* format_usage() (argparse.ArgumentParser method): Printing help. + (line 25) +* FORMAT_VALUE (opcode): Python Bytecode Instructions. + (line 876) +* Fortran contiguous: shape strides suboffsets. + (line 25) +* Fortran contiguous <1>: Glossary. (line 273) +* forward() (in module turtle): Turtle motion. (line 6) +* ForwardRef (class in typing): Classes functions and decorators. + (line 597) +* FOR_ITER (opcode): Python Bytecode Instructions. + (line 686) +* found_terminator() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 89) +* fpathconf() (in module os): File Descriptor Operations. + (line 129) +* fqdn (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 90) +* Fraction (class in fractions): fractions — Rational numbers. + (line 16) +* fractions (module): fractions — Rational numbers. + (line 6) +* Frame (class in tracemalloc): Frame. (line 6) +* frame (tkinter.scrolledtext.ScrolledText attribute): tkinter scrolledtext — Scrolled Text Widget. + (line 24) +* FrameSummary (class in traceback): FrameSummary Objects. + (line 11) +* FrameType (in module types): Standard Interpreter Types. + (line 190) +* free(): Overview<3>. (line 33) +* free; variable: Binding of names. (line 26) +* freefunc (C type): Slot Type typedefs. (line 37) +* freeze utility: Importing Modules<2>. + (line 262) +* freeze() (in module gc): gc — Garbage Collector interface. + (line 182) +* freeze_support() (in module multiprocessing): Miscellaneous<3>. + (line 43) +* frexp() (in module math): Number-theoretic and representation functions. + (line 67) +* from; import statement: Binding of names. (line 9) +* from; import statement <1>: The import statement. + (line 51) +* from; yield from expression: Yield expressions. (line 63) +* frombuf() (tarfile.TarInfo class method): TarInfo Objects. (line 19) +* frombytes() (array.array method): array — Efficient arrays of numeric values. + (line 156) +* fromfd() (in module socket): Creating sockets. (line 146) +* fromfd() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 77) +* fromfd() (select.kqueue method): Kqueue Objects. (line 18) +* fromfile() (array.array method): array — Efficient arrays of numeric values. + (line 165) +* fromhex() (bytearray class method): Bytearray Objects. (line 36) +* fromhex() (bytes class method): Bytes Objects. (line 62) +* fromhex() (float class method): Additional Methods on Float. + (line 39) +* fromisocalendar() (datetime.date class method): date Objects. + (line 77) +* fromisocalendar() (datetime.datetime class method): datetime Objects. + (line 222) +* fromisoformat() (datetime.date class method): date Objects. + (line 63) +* fromisoformat() (datetime.datetime class method): datetime Objects. + (line 187) +* fromisoformat() (datetime.time class method): time Objects. + (line 113) +* fromkeys() (collections.Counter method): Counter objects. (line 106) +* fromkeys() (dict class method): Mapping Types — dict. + (line 150) +* fromlist() (array.array method): array — Efficient arrays of numeric values. + (line 174) +* fromordinal() (datetime.date class method): date Objects. (line 54) +* fromordinal() (datetime.datetime class method): datetime Objects. + (line 161) +* fromshare() (in module socket): Creating sockets. (line 163) +* fromstring() (array.array method): array — Efficient arrays of numeric values. + (line 180) +* fromstring() (in module xml.etree.ElementTree): Functions<6>. + (line 99) +* fromstringlist() (in module xml.etree.ElementTree): Functions<6>. + (line 107) +* fromtarfile() (tarfile.TarInfo class method): TarInfo Objects. + (line 26) +* fromtimestamp() (datetime.date class method): date Objects. + (line 35) +* fromtimestamp() (datetime.datetime class method): datetime Objects. + (line 93) +* fromunicode() (array.array method): array — Efficient arrays of numeric values. + (line 186) +* fromutc() (datetime.timezone method): timezone Objects. (line 61) +* fromutc() (datetime.tzinfo method): tzinfo Objects. (line 163) +* from_address() (ctypes._CData method): Data types. (line 42) +* from_buffer() (ctypes._CData method): Data types. (line 19) +* from_buffer_copy() (ctypes._CData method): Data types. (line 31) +* from_bytes() (int class method): Additional Methods on Integer Types. + (line 67) +* from_callable() (inspect.Signature class method): Introspecting callables with the Signature object. + (line 127) +* from_decimal() (fractions.Fraction method): fractions — Rational numbers. + (line 116) +* from_exception() (traceback.TracebackException class method): TracebackException Objects. + (line 63) +* from_file() (zipfile.ZipInfo class method): ZipInfo Objects. + (line 14) +* from_float() (decimal.Decimal method): Decimal objects. (line 243) +* from_float() (fractions.Fraction method): fractions — Rational numbers. + (line 105) +* from_iterable() (itertools.chain class method): Itertool functions. + (line 98) +* from_list() (traceback.StackSummary class method): StackSummary Objects. + (line 29) +* from_param() (ctypes._CData method): Data types. (line 51) +* from_samples() (statistics.NormalDist class method): NormalDist objects. + (line 45) +* from_traceback() (dis.Bytecode class method): Bytecode analysis. + (line 33) +* FrozenImporter (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 80) +* FrozenInstanceError: Exceptions<17>. (line 6) +* frozenset (built-in class): Set Types — set frozenset. + (line 33) +* FrozenSet (class in typing): Classes functions and decorators. + (line 278) +* fsdecode() (in module os): Process Parameters. (line 88) +* fsencode() (in module os): Process Parameters. (line 75) +* fspath() (in module os): Process Parameters. (line 101) +* fstat() (in module os): File Descriptor Operations. + (line 151) +* fstatvfs() (in module os): File Descriptor Operations. + (line 163) +* fstring: String literal concatenation. + (line 24) +* fsum() (in module math): Number-theoretic and representation functions. + (line 75) +* fsync() (in module os): File Descriptor Operations. + (line 171) +* fs_is_case_insensitive() (in module test.support): test support — Utilities for the Python test suite. + (line 925) +* FS_NONASCII (in module test.support): test support — Utilities for the Python test suite. + (line 48) +* FTP: urllib request Restrictions. + (line 37) +* FTP (class in ftplib): ftplib — FTP protocol client. + (line 39) +* FTP; ftplib (standard module): ftplib — FTP protocol client. + (line 8) +* FTP; protocol: urllib request Restrictions. + (line 6) +* FTP; protocol <1>: ftplib — FTP protocol client. + (line 8) +* FTPHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 426) +* ftplib (module): ftplib — FTP protocol client. + (line 6) +* ftp_open() (urllib.request.FTPHandler method): FTPHandler Objects. + (line 6) +* FTP_TLS (class in ftplib): ftplib — FTP protocol client. + (line 72) +* ftruncate() (in module os): File Descriptor Operations. + (line 184) +* Full: queue — A synchronized queue class. + (line 88) +* full() (asyncio.Queue method): Queue. (line 32) +* full() (multiprocessing.Queue method): Pipes and Queues. (line 114) +* full() (queue.Queue method): Queue Objects. (line 22) +* fullmatch() (in module re): Module Contents. (line 167) +* fullmatch() (re.Pattern method): Regular Expression Objects. + (line 56) +* full_url (urllib.request.Request attribute): Request Objects. + (line 11) +* func (functools.partial attribute): partial Objects. (line 9) +* funcattrs (2to3 fixer): Fixers. (line 105) +* function: Glossary. (line 458) +* Function (class in symtable): Examining Symbol Tables. + (line 66) +* function annotation: Glossary. (line 466) +* function; annotations: Function Annotations. + (line 6) +* function; annotations <1>: Function definitions. + (line 89) +* function; argument: The standard type hierarchy. + (line 292) +* function; call: The standard type hierarchy. + (line 292) +* function; call <1>: Calls. (line 123) +* function; call <2>: Calls. (line 132) +* function; definition: The return statement. + (line 6) +* function; definition <1>: Function definitions. + (line 6) +* function; generator: The yield statement. + (line 6) +* function; name: Function definitions. + (line 6) +* function; name <1>: Function definitions. + (line 6) +* FunctionTestCase (class in unittest): Test cases. (line 895) +* FunctionType (in module types): Standard Interpreter Types. + (line 19) +* functools (module): functools — Higher-order functions and operations on callable objects. + (line 6) +* funny_files (filecmp.dircmp attribute): The dircmp class. (line 95) +* future (2to3 fixer): Fixers. (line 111) +* Future (class in asyncio): Future Object. (line 6) +* Future (class in concurrent.futures): Future Objects. (line 10) +* future; statement: Future statements. (line 6) +* FutureWarning: Warnings. (line 40) +* fwalk() (in module os): Files and Directories. + (line 1586) +* f_back (frame attribute): The standard type hierarchy. + (line 713) +* f_builtins (frame attribute): The standard type hierarchy. + (line 713) +* f_code (frame attribute): The standard type hierarchy. + (line 713) +* f_contiguous (memoryview attribute): Memory Views. (line 475) +* f_globals (frame attribute): The standard type hierarchy. + (line 713) +* f_lasti (frame attribute): The standard type hierarchy. + (line 713) +* f_lineno (frame attribute): The standard type hierarchy. + (line 722) +* f_locals (frame attribute): The standard type hierarchy. + (line 713) +* F_LOCK (in module os): File Descriptor Operations. + (line 229) +* F_OK (in module os): Files and Directories. + (line 106) +* F_TEST (in module os): File Descriptor Operations. + (line 229) +* F_TLOCK (in module os): File Descriptor Operations. + (line 229) +* f_trace (frame attribute): The standard type hierarchy. + (line 722) +* f_trace_lines (frame attribute): The standard type hierarchy. + (line 722) +* f_trace_opcodes (frame attribute): The standard type hierarchy. + (line 722) +* F_ULOCK (in module os): File Descriptor Operations. + (line 229) +* G.722: aifc — Read and write AIFF and AIFC files. + (line 160) +* gaierror: Exceptions<11>. (line 27) +* gamma() (in module math): Special functions. (line 29) +* gammavariate() (in module random): Real-valued distributions. + (line 46) +* garbage (in module gc): gc — Garbage Collector interface. + (line 210) +* garbage collection: Objects values and types. + (line 37) +* garbage collection <1>: Glossary. (line 498) +* gather() (curses.textpad.Textbox method): Textbox objects. (line 109) +* gather() (in module asyncio): Running Tasks Concurrently. + (line 6) +* gauss() (in module random): Real-valued distributions. + (line 57) +* gc (module): gc — Garbage Collector interface. + (line 6) +* gcd() (in module fractions): fractions — Rational numbers. + (line 174) +* gcd() (in module math): Number-theoretic and representation functions. + (line 95) +* gc_collect() (in module test.support): test support — Utilities for the Python test suite. + (line 474) +* ge() (in module operator): operator — Standard operators as functions. + (line 24) +* generate_help() (distutils.fancy_getopt.FancyGetopt method): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 68) +* generate_tokens() (in module tokenize): Tokenizing Input. (line 38) +* generator: Glossary. (line 507) +* generator <1>: Glossary. (line 507) +* Generator (class in collections.abc): Collections Abstract Base Classes. + (line 150) +* Generator (class in email.generator): email generator Generating MIME documents. + (line 140) +* Generator (class in typing): Classes functions and decorators. + (line 391) +* generator expression: Glossary. (line 529) +* generator expression <1>: Glossary. (line 529) +* generator iterator: Glossary. (line 517) +* generator; expression: Generator expressions. + (line 6) +* generator; function: The standard type hierarchy. + (line 444) +* generator; function <1>: Yield expressions. (line 6) +* generator; function <2>: The yield statement. + (line 6) +* generator; iterator: The standard type hierarchy. + (line 444) +* generator; iterator <1>: The yield statement. + (line 6) +* GeneratorExit: Concrete exceptions. + (line 30) +* GeneratorType (in module types): Standard Interpreter Types. + (line 31) +* Generic (class in typing): Classes functions and decorators. + (line 47) +* generic function: Glossary. (line 538) +* generic; special; attribute: The standard type hierarchy. + (line 13) +* generic_visit() (ast.NodeVisitor method): ast Helpers. (line 136) +* genops() (in module pickletools): Programmatic Interface<2>. + (line 22) +* gen_lib_options() (in module distutils.ccompiler): distutils ccompiler — CCompiler base class. + (line 14) +* gen_preprocess_options() (in module distutils.ccompiler): distutils ccompiler — CCompiler base class. + (line 24) +* gen_uuid() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 109) +* geometric_mean() (in module statistics): Function details. (line 64) +* get() (asyncio.Queue method): Queue. (line 40) +* get() (configparser.ConfigParser method): ConfigParser Objects. + (line 188) +* get() (contextvars.Context method): Manual Context Management. + (line 85) +* get() (contextvars.ContextVar method): Context Variables. (line 30) +* get() (dict method): Mapping Types — dict. + (line 162) +* get() (email.message.EmailMessage method): email message Representing an email message. + (line 245) +* get() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 353) +* get() (in module webbrowser): webbrowser — Convenient Web-browser controller. + (line 77) +* get() (mailbox.Mailbox method): Mailbox objects. (line 140) +* get() (multiprocessing.pool.AsyncResult method): Process Pools. + (line 196) +* get() (multiprocessing.Queue method): Pipes and Queues. (line 140) +* get() (multiprocessing.SimpleQueue method): Pipes and Queues. + (line 213) +* get() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 60) +* get() (queue.Queue method): Queue Objects. (line 44) +* get() (queue.SimpleQueue method): SimpleQueue Objects. + (line 40) +* get() (tkinter.ttk.Combobox method): ttk Combobox. (line 15) +* get() (tkinter.ttk.Spinbox method): ttk Spinbox. (line 8) +* get() (types.MappingProxyType method): Standard Interpreter Types. + (line 249) +* get() (xml.etree.ElementTree.Element method): Element Objects. + (line 64) +* getacl() (imaplib.IMAP4 method): IMAP4 Objects. (line 106) +* getaddresses() (in module email.utils): email utils Miscellaneous utilities. + (line 81) +* getaddrinfo() (asyncio.loop method): DNS. (line 6) +* getaddrinfo() (in module socket): Other functions<2>. (line 17) +* getallocatedblocks() (in module sys): sys — System-specific parameters and functions. + (line 590) +* getandroidapilevel() (in module sys): sys — System-specific parameters and functions. + (line 605) +* getannotation() (imaplib.IMAP4 method): IMAP4 Objects. (line 111) +* getargspec() (in module inspect): Classes and functions<2>. + (line 17) +* getargvalues() (in module inspect): Classes and functions<2>. + (line 81) +* getatime() (in module os.path): os path — Common pathname manipulations. + (line 170) +* getattr() (built-in function): Built-in Functions. (line 676) +* getattrfunc (C type): Slot Type typedefs. (line 53) +* getAttribute() (xml.dom.Element method): Element Objects<2>. + (line 31) +* getAttributeNode() (xml.dom.Element method): Element Objects<2>. + (line 37) +* getAttributeNodeNS() (xml.dom.Element method): Element Objects<2>. + (line 47) +* getAttributeNS() (xml.dom.Element method): Element Objects<2>. + (line 41) +* getattrofunc (C type): Slot Type typedefs. (line 63) +* getattr_static() (in module inspect): Fetching attributes statically. + (line 16) +* GetBase() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 31) +* getbegyx() (curses.window method): Window Objects. (line 222) +* getbkgd() (curses.window method): Window Objects. (line 226) +* getblocking() (socket.socket method): Socket Objects. (line 153) +* getboolean() (configparser.ConfigParser method): ConfigParser Objects. + (line 222) +* getbuffer() (io.BytesIO method): Buffered Streams. (line 21) +* getbufferproc (C type): Slot Type typedefs. (line 105) +* getByteStream() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 48) +* getcallargs() (in module inspect): Classes and functions<2>. + (line 140) +* getcanvas() (in module turtle): Settings and special methods. + (line 55) +* getcapabilities() (nntplib.NNTP method): Methods<3>. (line 34) +* getcaps() (in module mailcap): mailcap — Mailcap file handling. + (line 62) +* getch() (curses.window method): Window Objects. (line 231) +* getch() (in module msvcrt): Console I/O. (line 10) +* getCharacterStream() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 64) +* getche() (in module msvcrt): Console I/O. (line 25) +* getcheckinterval() (in module sys): sys — System-specific parameters and functions. + (line 613) +* getChild() (logging.Logger method): Logger Objects. (line 106) +* getchildren() (xml.etree.ElementTree.Element method): Element Objects. + (line 130) +* getclasstree() (in module inspect): Classes and functions<2>. + (line 6) +* getclosurevars() (in module inspect): Classes and functions<2>. + (line 170) +* GetColumnInfo() (msilib.View method): View Objects. (line 12) +* getColumnNumber() (xml.sax.xmlreader.Locator method): Locator Objects. + (line 8) +* getcomments() (in module inspect): Retrieving source code. + (line 17) +* getcompname() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 73) +* getcompname() (sunau.AU_read method): AU_read Objects. (line 35) +* getcompname() (wave.Wave_read method): Wave_read Objects. (line 35) +* getcomptype() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 67) +* getcomptype() (sunau.AU_read method): AU_read Objects. (line 30) +* getcomptype() (wave.Wave_read method): Wave_read Objects. (line 31) +* getContentHandler() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 21) +* getcontext() (in module decimal): Context objects. (line 14) +* getcoroutinelocals() (in module inspect): Current State of Generators and Coroutines. + (line 73) +* getcoroutinestate() (in module inspect): Current State of Generators and Coroutines. + (line 28) +* getctime() (in module os.path): os path — Common pathname manipulations. + (line 186) +* getcwd() (in module os): Files and Directories. + (line 280) +* getcwdb() (in module os): Files and Directories. + (line 284) +* getcwdu (2to3 fixer): Fixers. (line 115) +* getdecoder() (in module codecs): codecs — Codec registry and base classes. + (line 107) +* getdefaultencoding() (in module sys): sys — System-specific parameters and functions. + (line 621) +* getdefaultlocale() (in module locale): locale — Internationalization services. + (line 301) +* getdefaulttimeout() (in module socket): Other functions<2>. + (line 332) +* getdlopenflags() (in module sys): sys — System-specific parameters and functions. + (line 626) +* getdoc() (in module inspect): Retrieving source code. + (line 6) +* getDOMImplementation() (in module xml.dom): Module Contents<4>. + (line 17) +* getDTDHandler() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 30) +* getEffectiveLevel() (logging.Logger method): Logger Objects. + (line 97) +* getegid() (in module os): Process Parameters. (line 163) +* getElementsByTagName() (xml.dom.Document method): Document Objects. + (line 61) +* getElementsByTagName() (xml.dom.Element method): Element Objects<2>. + (line 14) +* getElementsByTagNameNS() (xml.dom.Document method): Document Objects. + (line 66) +* getElementsByTagNameNS() (xml.dom.Element method): Element Objects<2>. + (line 18) +* getencoder() (in module codecs): codecs — Codec registry and base classes. + (line 99) +* getEncoding() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 32) +* getEntityResolver() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 39) +* getenv() (in module os): Process Parameters. (line 128) +* getenvb() (in module os): Process Parameters. (line 140) +* getErrorHandler() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 50) +* geteuid() (in module os): Process Parameters. (line 171) +* getEvent() (xml.dom.pulldom.DOMEventStream method): DOMEventStream Objects. + (line 11) +* getEventCategory() (logging.handlers.NTEventLogHandler method): NTEventLogHandler. + (line 43) +* getEventType() (logging.handlers.NTEventLogHandler method): NTEventLogHandler. + (line 49) +* getException() (xml.sax.SAXException method): SAXException Objects. + (line 13) +* getFeature() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 69) +* GetFieldCount() (msilib.Record method): Record Objects. (line 6) +* getfile() (in module inspect): Retrieving source code. + (line 26) +* getfilesystemencodeerrors() (in module sys): sys — System-specific parameters and functions. + (line 671) +* getfilesystemencoding() (in module sys): sys — System-specific parameters and functions. + (line 635) +* getfirst() (cgi.FieldStorage method): Higher Level Interface. + (line 56) +* getfloat() (configparser.ConfigParser method): ConfigParser Objects. + (line 215) +* getfmts() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 80) +* getfqdn() (in module socket): Other functions<2>. (line 71) +* getframeinfo() (in module inspect): The interpreter stack. + (line 46) +* getframerate() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 59) +* getframerate() (sunau.AU_read method): AU_read Objects. (line 22) +* getframerate() (wave.Wave_read method): Wave_read Objects. (line 23) +* getfullargspec() (in module inspect): Classes and functions<2>. + (line 36) +* getgeneratorlocals() (in module inspect): Current State of Generators and Coroutines. + (line 53) +* getgeneratorstate() (in module inspect): Current State of Generators and Coroutines. + (line 12) +* getgid() (in module os): Process Parameters. (line 177) +* getgrall() (in module grp): grp — The group database. + (line 57) +* getgrgid() (in module grp): grp — The group database. + (line 42) +* getgrnam() (in module grp): grp — The group database. + (line 52) +* getgrouplist() (in module os): Process Parameters. (line 183) +* getgroups() (in module os): Process Parameters. (line 193) +* getheader() (http.client.HTTPResponse method): HTTPResponse Objects. + (line 24) +* getheaders() (http.client.HTTPResponse method): HTTPResponse Objects. + (line 32) +* gethostbyaddr() (in module socket): Process Parameters. (line 471) +* gethostbyaddr() (in module socket) <1>: Other functions<2>. + (line 120) +* gethostbyname() (in module socket): Other functions<2>. (line 81) +* gethostbyname_ex() (in module socket): Other functions<2>. (line 94) +* gethostname() (in module socket): Process Parameters. (line 471) +* gethostname() (in module socket) <1>: Other functions<2>. (line 109) +* getincrementaldecoder() (in module codecs): codecs — Codec registry and base classes. + (line 123) +* getincrementalencoder() (in module codecs): codecs — Codec registry and base classes. + (line 115) +* getinfo() (zipfile.ZipFile method): ZipFile Objects. (line 91) +* getinnerframes() (in module inspect): The interpreter stack. + (line 63) +* GetInputContext() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 37) +* getint() (configparser.ConfigParser method): ConfigParser Objects. + (line 208) +* GetInteger() (msilib.Record method): Record Objects. (line 11) +* getitem() (in module operator): operator — Standard operators as functions. + (line 192) +* getiterator() (xml.etree.ElementTree.Element method): Element Objects. + (line 135) +* getiterator() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 36) +* getiterfunc (C type): Slot Type typedefs. (line 95) +* getitimer() (in module signal): Module contents<2>. (line 370) +* getkey() (curses.window method): Window Objects. (line 246) +* GetLastError() (in module ctypes): Utility functions. (line 114) +* getLength() (xml.sax.xmlreader.Attributes method): The Attributes Interface. + (line 11) +* getLevelName() (in module logging): Module-Level Functions. + (line 203) +* getline() (in module linecache): linecache — Random access to text lines. + (line 23) +* getLineNumber() (xml.sax.xmlreader.Locator method): Locator Objects. + (line 12) +* getlist() (cgi.FieldStorage method): Higher Level Interface. + (line 67) +* getloadavg() (in module os): Miscellaneous System Information. + (line 48) +* getlocale() (in module locale): locale — Internationalization services. + (line 326) +* getLogger() (in module logging): Module-Level Functions. + (line 9) +* getLoggerClass() (in module logging): Module-Level Functions. + (line 21) +* getlogin() (in module os): Process Parameters. (line 216) +* getLogRecordFactory() (in module logging): Module-Level Functions. + (line 32) +* getmark() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 92) +* getmark() (sunau.AU_read method): AU_read Objects. (line 77) +* getmark() (wave.Wave_read method): Wave_read Objects. (line 62) +* getmarkers() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 85) +* getmarkers() (sunau.AU_read method): AU_read Objects. (line 73) +* getmarkers() (wave.Wave_read method): Wave_read Objects. (line 58) +* getmaxyx() (curses.window method): Window Objects. (line 253) +* getmember() (tarfile.TarFile method): TarFile Objects. (line 101) +* getmembers() (in module inspect): Types and members. (line 276) +* getmembers() (tarfile.TarFile method): TarFile Objects. (line 109) +* getMessage() (logging.LogRecord method): LogRecord Objects. + (line 56) +* getMessage() (xml.sax.SAXException method): SAXException Objects. + (line 9) +* getMessageID() (logging.handlers.NTEventLogHandler method): NTEventLogHandler. + (line 60) +* getmodule() (in module inspect): Retrieving source code. + (line 32) +* getmodulename() (in module inspect): Types and members. (line 289) +* getmouse() (in module curses): Functions<3>. (line 145) +* getmro() (in module inspect): Classes and functions<2>. + (line 132) +* getmtime() (in module os.path): os path — Common pathname manipulations. + (line 177) +* getname() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 69) +* getName() (threading.Thread method): Thread Objects. (line 142) +* getNameByQName() (xml.sax.xmlreader.AttributesNS method): The AttributesNS Interface. + (line 16) +* getnameinfo() (asyncio.loop method): DNS. (line 11) +* getnameinfo() (in module socket): Other functions<2>. (line 134) +* getnames() (tarfile.TarFile method): TarFile Objects. (line 115) +* getNames() (xml.sax.xmlreader.Attributes method): The Attributes Interface. + (line 15) +* getnchannels() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 51) +* getnchannels() (sunau.AU_read method): AU_read Objects. (line 14) +* getnchannels() (wave.Wave_read method): Wave_read Objects. (line 15) +* getnframes() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 63) +* getnframes() (sunau.AU_read method): AU_read Objects. (line 26) +* getnframes() (wave.Wave_read method): Wave_read Objects. (line 27) +* getnode: uuid — UUID objects according to RFC 4122. + (line 178) +* getnode() (in module uuid): uuid — UUID objects according to RFC 4122. + (line 160) +* getopt (module): getopt — C-style parser for command line options. + (line 6) +* getopt() (distutils.fancy_getopt.FancyGetopt method): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 49) +* getopt() (in module getopt): getopt — C-style parser for command line options. + (line 25) +* GetoptError: getopt — C-style parser for command line options. + (line 74) +* getouterframes() (in module inspect): The interpreter stack. + (line 52) +* getoutput() (in module subprocess): Legacy Shell Invocation Functions. + (line 41) +* getpagesize() (in module resource): Resource Usage. (line 101) +* getparams() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 79) +* getparams() (sunau.AU_read method): AU_read Objects. (line 41) +* getparams() (wave.Wave_read method): Wave_read Objects. (line 40) +* getparyx() (curses.window method): Window Objects. (line 257) +* getpass (module): getpass — Portable password input. + (line 6) +* getpass() (in module getpass): getpass — Portable password input. + (line 12) +* GetPassWarning: getpass — Portable password input. + (line 29) +* getpeercert() (ssl.SSLSocket method): SSL Sockets. (line 139) +* getpeername() (socket.socket method): Socket Objects. (line 127) +* getpen() (in module turtle): Special Turtle methods. + (line 40) +* getpgid() (in module os): Process Parameters. (line 227) +* getpgrp() (in module os): Process Parameters. (line 235) +* getpid() (in module os): Process Parameters. (line 241) +* getpos() (html.parser.HTMLParser method): HTMLParser Methods. + (line 28) +* getppid() (in module os): Process Parameters. (line 245) +* getpreferredencoding() (in module locale): locale — Internationalization services. + (line 337) +* getpriority() (in module os): Process Parameters. (line 256) +* getprofile() (in module sys): sys — System-specific parameters and functions. + (line 739) +* GetProperty() (msilib.SummaryInformation method): Summary Information Objects. + (line 6) +* getProperty() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 83) +* GetPropertyCount() (msilib.SummaryInformation method): Summary Information Objects. + (line 17) +* getprotobyname() (in module socket): Other functions<2>. (line 152) +* getproxies() (in module urllib.request): urllib request — Extensible library for opening URLs. + (line 160) +* getPublicId() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 10) +* getPublicId() (xml.sax.xmlreader.Locator method): Locator Objects. + (line 16) +* getpwall() (in module pwd): pwd — The password database. + (line 65) +* getpwnam() (in module pwd): pwd — The password database. + (line 61) +* getpwuid() (in module pwd): pwd — The password database. + (line 57) +* getQNameByName() (xml.sax.xmlreader.AttributesNS method): The AttributesNS Interface. + (line 20) +* getQNames() (xml.sax.xmlreader.AttributesNS method): The AttributesNS Interface. + (line 24) +* getquota() (imaplib.IMAP4 method): IMAP4 Objects. (line 116) +* getquotaroot() (imaplib.IMAP4 method): IMAP4 Objects. (line 121) +* getrandbits() (in module random): Bookkeeping functions. + (line 41) +* getrandom() (in module os): Random numbers<2>. (line 6) +* getreader() (in module codecs): codecs — Codec registry and base classes. + (line 131) +* getrecursionlimit() (in module sys): sys — System-specific parameters and functions. + (line 688) +* getrefcount() (in module sys): sys — System-specific parameters and functions. + (line 682) +* getresgid() (in module os): Process Parameters. (line 291) +* getresponse() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 59) +* getresuid() (in module os): Process Parameters. (line 282) +* getrlimit() (in module resource): Resource Limits. (line 24) +* getroot() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 41) +* getrusage() (in module resource): Resource Usage. (line 8) +* getsample() (in module audioop): audioop — Manipulate raw audio data. + (line 108) +* getsampwidth() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 55) +* getsampwidth() (sunau.AU_read method): AU_read Objects. (line 18) +* getsampwidth() (wave.Wave_read method): Wave_read Objects. (line 19) +* getscreen() (in module turtle): Special Turtle methods. + (line 51) +* getservbyname() (in module socket): Other functions<2>. (line 161) +* getservbyport() (in module socket): Other functions<2>. (line 170) +* GetSetDescriptorType (in module types): Standard Interpreter Types. + (line 198) +* getshapes() (in module turtle): Settings and special methods. + (line 64) +* getsid() (in module os): Process Parameters. (line 432) +* getsignal() (in module signal): Module contents<2>. (line 238) +* getsitepackages() (in module site): Module contents<3>. (line 56) +* getsize() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 74) +* getsize() (in module os.path): os path — Common pathname manipulations. + (line 197) +* getsizeof() (in module sys): sys — System-specific parameters and functions. + (line 695) +* getsockname() (socket.socket method): Socket Objects. (line 135) +* getsockopt() (socket.socket method): Socket Objects. (line 141) +* getsource() (in module inspect): Retrieving source code. + (line 55) +* getsourcefile() (in module inspect): Retrieving source code. + (line 36) +* getsourcelines() (in module inspect): Retrieving source code. + (line 42) +* getspall() (in module spwd): spwd — The shadow password database. + (line 67) +* getspnam() (in module spwd): spwd — The shadow password database. + (line 59) +* getstate() (codecs.IncrementalDecoder method): IncrementalDecoder Objects. + (line 44) +* getstate() (codecs.IncrementalEncoder method): IncrementalEncoder Objects. + (line 42) +* getstate() (in module random): Bookkeeping functions. + (line 28) +* getstatusoutput() (in module subprocess): Legacy Shell Invocation Functions. + (line 11) +* getstr() (curses.window method): Window Objects. (line 263) +* GetString() (msilib.Record method): Record Objects. (line 16) +* getSubject() (logging.handlers.SMTPHandler method): SMTPHandler. + (line 39) +* GetSummaryInformation() (msilib.Database method): Database Objects. + (line 16) +* getswitchinterval() (in module sys): sys — System-specific parameters and functions. + (line 717) +* getSystemId() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 18) +* getSystemId() (xml.sax.xmlreader.Locator method): Locator Objects. + (line 20) +* getsyx() (in module curses): Functions<3>. (line 159) +* gettarinfo() (tarfile.TarFile method): TarFile Objects. (line 226) +* gettempdir() (in module tempfile): tempfile — Generate temporary files and directories. + (line 228) +* gettempdirb() (in module tempfile): tempfile — Generate temporary files and directories. + (line 256) +* gettempprefix() (in module tempfile): tempfile — Generate temporary files and directories. + (line 262) +* gettempprefixb() (in module tempfile): tempfile — Generate temporary files and directories. + (line 267) +* getTestCaseNames() (unittest.TestLoader method): Loading and running tests. + (line 111) +* gettext (module): gettext — Multilingual internationalization services. + (line 6) +* gettext() (gettext.GNUTranslations method): The GNUTranslations class. + (line 38) +* gettext() (gettext.NullTranslations method): The NullTranslations class. + (line 35) +* gettext() (in module gettext): GNU gettext API. (line 42) +* gettext() (in module locale): Access to message catalogs. + (line 6) +* gettimeout() (socket.socket method): Socket Objects. (line 162) +* gettrace() (in module sys): sys — System-specific parameters and functions. + (line 743) +* getturtle() (in module turtle): Special Turtle methods. + (line 40) +* getType() (xml.sax.xmlreader.Attributes method): The Attributes Interface. + (line 19) +* getuid() (in module os): Process Parameters. (line 300) +* geturl() (urllib.parse.urllib.parse.SplitResult method): Structured Parse Results. + (line 12) +* getuser() (in module getpass): getpass — Portable password input. + (line 34) +* getuserbase() (in module site): Module contents<3>. (line 62) +* getusersitepackages() (in module site): Module contents<3>. + (line 70) +* getvalue() (io.BytesIO method): Buffered Streams. (line 38) +* getvalue() (io.StringIO method): Text I/O<2>. (line 212) +* getValue() (xml.sax.xmlreader.Attributes method): The Attributes Interface. + (line 24) +* getValueByQName() (xml.sax.xmlreader.AttributesNS method): The AttributesNS Interface. + (line 12) +* getwch() (in module msvcrt): Console I/O. (line 20) +* getwche() (in module msvcrt): Console I/O. (line 30) +* getweakrefcount() (in module weakref): weakref — Weak references. + (line 148) +* getweakrefs() (in module weakref): weakref — Weak references. + (line 153) +* getwelcome() (ftplib.FTP method): FTP Objects. (line 42) +* getwelcome() (nntplib.NNTP method): Methods<3>. (line 28) +* getwelcome() (poplib.POP3 method): POP3 Objects. (line 20) +* getwin() (in module curses): Functions<3>. (line 165) +* getwindowsversion() (in module sys): sys — System-specific parameters and functions. + (line 754) +* getwriter() (in module codecs): codecs — Codec registry and base classes. + (line 139) +* getxattr() (in module os): Linux extended attributes. + (line 10) +* getyx() (curses.window method): Window Objects. (line 274) +* GET_AITER (opcode): Python Bytecode Instructions. + (line 268) +* get_all() (email.message.EmailMessage method): email message Representing an email message. + (line 254) +* get_all() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 362) +* get_all() (wsgiref.headers.Headers method): wsgiref headers – WSGI response header tools. + (line 51) +* get_all_breaks() (bdb.Bdb method): bdb — Debugger framework. + (line 332) +* get_all_start_methods() (in module multiprocessing): Miscellaneous<3>. + (line 70) +* GET_ANEXT (opcode): Python Bytecode Instructions. + (line 277) +* get_app() (wsgiref.simple_server.WSGIServer method): wsgiref simple_server – a simple WSGI HTTP server. + (line 66) +* get_archive_formats() (in module shutil): Archiving operations. + (line 57) +* get_args() (in module typing): Classes functions and decorators. + (line 642) +* get_asyncgen_hooks() (in module sys): sys — System-specific parameters and functions. + (line 802) +* get_attribute() (in module test.support): test support — Utilities for the Python test suite. + (line 799) +* GET_AWAITABLE (opcode): Python Bytecode Instructions. + (line 260) +* get_begidx() (in module readline): Completion. (line 40) +* get_blocking() (in module os): File Descriptor Operations. + (line 197) +* get_body() (email.message.EmailMessage method): email message Representing an email message. + (line 518) +* get_body_encoding() (email.charset.Charset method): email charset Representing character sets. + (line 96) +* get_boundary() (email.message.EmailMessage method): email message Representing an email message. + (line 403) +* get_boundary() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 579) +* get_bpbynumber() (bdb.Bdb method): bdb — Debugger framework. + (line 309) +* get_break() (bdb.Bdb method): bdb — Debugger framework. + (line 318) +* get_breaks() (bdb.Bdb method): bdb — Debugger framework. + (line 322) +* get_buffer() (asyncio.BufferedProtocol method): Buffered Streaming Protocols. + (line 23) +* get_buffer() (xdrlib.Packer method): Packer Objects. (line 8) +* get_buffer() (xdrlib.Unpacker method): Unpacker Objects. (line 22) +* get_bytes() (mailbox.Mailbox method): Mailbox objects. (line 159) +* get_cache_token() (in module abc): abc — Abstract Base Classes. + (line 325) +* get_ca_certs() (ssl.SSLContext method): SSL Contexts. (line 157) +* get_channel_binding() (ssl.SSLSocket method): SSL Sockets. (line 238) +* get_charset() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 265) +* get_charsets() (email.message.EmailMessage method): email message Representing an email message. + (line 431) +* get_charsets() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 613) +* get_children() (symtable.SymbolTable method): Examining Symbol Tables. + (line 62) +* get_children() (tkinter.ttk.Treeview method): ttk Treeview. + (line 19) +* get_child_watcher() (asyncio.AbstractEventLoopPolicy method): Policy Objects. + (line 33) +* get_child_watcher() (in module asyncio): Process Watchers. (line 25) +* get_ciphers() (ssl.SSLContext method): SSL Contexts. (line 171) +* get_clock_info() (in module time): Functions<2>. (line 97) +* get_close_matches() (in module difflib): difflib — Helpers for computing deltas. + (line 204) +* get_code() (importlib.abc.InspectLoader method): importlib abc – Abstract base classes related to import. + (line 387) +* get_code() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 581) +* get_code() (importlib.machinery.ExtensionFileLoader method): importlib machinery – Importers and path hooks. + (line 338) +* get_code() (importlib.machinery.SourcelessFileLoader method): importlib machinery – Importers and path hooks. + (line 281) +* get_code() (zipimport.zipimporter method): zipimporter Objects. + (line 28) +* get_completer() (in module readline): Completion. (line 29) +* get_completer_delims() (in module readline): Completion. (line 48) +* get_completion_type() (in module readline): Completion. (line 34) +* get_config_h_filename() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 41) +* get_config_h_filename() (in module sysconfig): Other functions<3>. + (line 68) +* get_config_var() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 27) +* get_config_var() (in module sysconfig): Configuration variables. + (line 26) +* get_config_vars() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 32) +* get_config_vars() (in module sysconfig): Configuration variables. + (line 16) +* get_content() (email.contentmanager.ContentManager method): email contentmanager Managing MIME Content. + (line 19) +* get_content() (email.message.EmailMessage method): email message Representing an email message. + (line 582) +* get_content() (in module email.contentmanager): Content Manager Instances. + (line 24) +* get_content_charset() (email.message.EmailMessage method): email message Representing an email message. + (line 424) +* get_content_charset() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 602) +* get_content_disposition() (email.message.EmailMessage method): email message Representing an email message. + (line 455) +* get_content_disposition() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 627) +* get_content_maintype() (email.message.EmailMessage method): email message Representing an email message. + (line 324) +* get_content_maintype() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 432) +* get_content_subtype() (email.message.EmailMessage method): email message Representing an email message. + (line 330) +* get_content_subtype() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 438) +* get_content_type() (email.message.EmailMessage method): email message Representing an email message. + (line 308) +* get_content_type() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 416) +* get_context() (in module multiprocessing): Miscellaneous<3>. + (line 80) +* get_coro() (asyncio.Task method): Task Object. (line 204) +* get_coroutine_origin_tracking_depth() (in module sys): sys — System-specific parameters and functions. + (line 816) +* get_count() (in module gc): gc — Garbage Collector interface. + (line 110) +* get_current_history_length() (in module readline): History list. + (line 14) +* get_data() (importlib.abc.FileLoader method): importlib abc – Abstract base classes related to import. + (line 503) +* get_data() (importlib.abc.ResourceLoader method): importlib abc – Abstract base classes related to import. + (line 368) +* get_data() (in module pkgutil): pkgutil — Package extension utility. + (line 201) +* get_data() (zipimport.zipimporter method): zipimporter Objects. + (line 33) +* get_date() (mailbox.MaildirMessage method): MaildirMessage. + (line 92) +* get_debug() (asyncio.loop method): Enabling debug mode. + (line 6) +* get_debug() (in module gc): gc — Garbage Collector interface. + (line 55) +* get_default() (argparse.ArgumentParser method): Parser defaults. + (line 33) +* get_default_compiler() (in module distutils.ccompiler): distutils ccompiler — CCompiler base class. + (line 36) +* get_default_domain() (in module nis): nis — Interface to Sun’s NIS Yellow Pages. + (line 48) +* get_default_type() (email.message.EmailMessage method): email message Representing an email message. + (line 335) +* get_default_type() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 443) +* get_default_verify_paths() (in module ssl): Certificate handling. + (line 101) +* get_dialect() (in module csv): Module Contents<3>. (line 85) +* get_docstring() (in module ast): ast Helpers. (line 61) +* get_doctest() (doctest.DocTestParser method): DocTestParser objects. + (line 13) +* get_endidx() (in module readline): Completion. (line 40) +* get_environ() (wsgiref.simple_server.WSGIRequestHandler method): wsgiref simple_server – a simple WSGI HTTP server. + (line 88) +* get_errno() (in module ctypes): Utility functions. (line 121) +* get_event_loop() (asyncio.AbstractEventLoopPolicy method): Policy Objects. + (line 12) +* get_event_loop() (in module asyncio): Event Loop. (line 39) +* get_event_loop_policy() (in module asyncio): Getting and Setting the Policy. + (line 9) +* get_examples() (doctest.DocTestParser method): DocTestParser objects. + (line 22) +* get_exception_handler() (asyncio.loop method): Error Handling API. + (line 19) +* get_exec_path() (in module os): Process Parameters. (line 153) +* get_extra_info() (asyncio.BaseTransport method): Base Transport. + (line 20) +* get_extra_info() (asyncio.StreamWriter method): StreamWriter. + (line 61) +* get_field() (string.Formatter method): Custom String Formatting. + (line 55) +* get_file() (mailbox.Babyl method): Babyl. (line 49) +* get_file() (mailbox.Mailbox method): Mailbox objects. (line 175) +* get_file() (mailbox.Maildir method): Maildir. (line 119) +* get_file() (mailbox.mbox method): mbox. (line 33) +* get_file() (mailbox.MH method): MH. (line 93) +* get_file() (mailbox.MMDF method): MMDF. (line 29) +* get_filename() (email.message.EmailMessage method): email message Representing an email message. + (line 393) +* get_filename() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 569) +* get_filename() (importlib.abc.ExecutionLoader method): importlib abc – Abstract base classes related to import. + (line 458) +* get_filename() (importlib.abc.FileLoader method): importlib abc – Abstract base classes related to import. + (line 499) +* get_filename() (importlib.machinery.ExtensionFileLoader method): importlib machinery – Importers and path hooks. + (line 346) +* get_filename() (zipimport.zipimporter method): zipimporter Objects. + (line 41) +* get_file_breaks() (bdb.Bdb method): bdb — Debugger framework. + (line 327) +* get_flags() (mailbox.MaildirMessage method): MaildirMessage. + (line 63) +* get_flags() (mailbox.mboxMessage method): mboxMessage. (line 64) +* get_flags() (mailbox.MMDFMessage method): MMDFMessage. (line 63) +* get_folder() (mailbox.Maildir method): Maildir. (line 65) +* get_folder() (mailbox.MH method): MH. (line 36) +* get_frees() (symtable.Function method): Examining Symbol Tables. + (line 88) +* get_freeze_count() (in module gc): gc — Garbage Collector interface. + (line 201) +* get_from() (mailbox.mboxMessage method): mboxMessage. (line 47) +* get_from() (mailbox.MMDFMessage method): MMDFMessage. (line 46) +* get_full_url() (urllib.request.Request method): Request Objects. + (line 106) +* get_globals() (symtable.Function method): Examining Symbol Tables. + (line 80) +* get_grouped_opcodes() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 166) +* get_handle_inheritable() (in module os): Inheritance of File Descriptors. + (line 32) +* get_header() (urllib.request.Request method): Request Objects. + (line 120) +* get_history_item() (in module readline): History list. (line 20) +* get_history_length() (in module readline): History file. (line 30) +* get_id() (symtable.SymbolTable method): Examining Symbol Tables. + (line 15) +* get_ident() (in module threading): threading — Thread-based parallelism. + (line 85) +* get_ident() (in module _thread): _thread — Low-level threading API. + (line 73) +* get_identifiers() (symtable.SymbolTable method): Examining Symbol Tables. + (line 48) +* get_importer() (in module pkgutil): pkgutil — Package extension utility. + (line 92) +* get_info() (mailbox.MaildirMessage method): MaildirMessage. + (line 102) +* get_inheritable() (in module os): Inheritance of File Descriptors. + (line 23) +* get_inheritable() (socket.socket method): Socket Objects. (line 119) +* get_instructions() (in module dis): Analysis functions. (line 110) +* get_interpreter() (in module zipapp): Python API. (line 82) +* GET_ITER (opcode): Python Bytecode Instructions. + (line 113) +* get_key() (selectors.BaseSelector method): Classes<3>. (line 145) +* get_labels() (mailbox.Babyl method): Babyl. (line 35) +* get_labels() (mailbox.BabylMessage method): BabylMessage. (line 47) +* get_last_error() (in module ctypes): Utility functions. (line 129) +* get_lineno() (symtable.SymbolTable method): Examining Symbol Tables. + (line 26) +* get_line_buffer() (in module readline): Line buffer. (line 8) +* get_loader() (in module pkgutil): pkgutil — Package extension utility. + (line 106) +* get_locals() (symtable.Function method): Examining Symbol Tables. + (line 76) +* get_logger() (in module multiprocessing): Logging<2>. (line 11) +* get_loop() (asyncio.Future method): Future Object. (line 130) +* get_loop() (asyncio.Server method): Server Objects. (line 41) +* get_magic() (in module imp): imp — Access the import internals. + (line 17) +* get_makefile_filename() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 49) +* get_makefile_filename() (in module sysconfig): Other functions<3>. + (line 72) +* get_map() (selectors.BaseSelector method): Classes<3>. (line 153) +* get_matching_blocks() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 105) +* get_message() (mailbox.Mailbox method): Mailbox objects. (line 152) +* get_method() (urllib.request.Request method): Request Objects. + (line 69) +* get_methods() (symtable.Class method): Examining Symbol Tables. + (line 98) +* get_mixed_type_key() (in module ipaddress): Other Module Level Functions. + (line 56) +* get_name() (asyncio.Task method): Task Object. (line 210) +* get_name() (symtable.Symbol method): Examining Symbol Tables. + (line 108) +* get_name() (symtable.SymbolTable method): Examining Symbol Tables. + (line 19) +* get_namespace() (symtable.Symbol method): Examining Symbol Tables. + (line 179) +* get_namespaces() (symtable.Symbol method): Examining Symbol Tables. + (line 175) +* get_native_id() (in module threading): threading — Thread-based parallelism. + (line 95) +* get_native_id() (in module _thread): _thread — Low-level threading API. + (line 81) +* get_nonlocals() (symtable.Function method): Examining Symbol Tables. + (line 84) +* get_nonstandard_attr() (http.cookiejar.Cookie method): Cookie Objects<2>. + (line 98) +* get_nowait() (asyncio.Queue method): Queue. (line 45) +* get_nowait() (multiprocessing.Queue method): Pipes and Queues. + (line 155) +* get_nowait() (queue.Queue method): Queue Objects. (line 61) +* get_nowait() (queue.SimpleQueue method): SimpleQueue Objects. + (line 51) +* get_objects() (in module gc): gc — Garbage Collector interface. + (line 59) +* get_object_traceback() (in module tracemalloc): Functions<9>. + (line 12) +* get_opcodes() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 123) +* get_option() (optparse.OptionParser method): Querying and manipulating your option parser. + (line 37) +* get_option_group() (optparse.OptionParser method): Grouping Options. + (line 103) +* get_option_order() (distutils.fancy_getopt.FancyGetopt method): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 62) +* get_origin() (in module typing): Classes functions and decorators. + (line 640) +* get_original_stdout() (in module test.support): test support — Utilities for the Python test suite. + (line 381) +* get_osfhandle() (in module msvcrt): File Operations. (line 55) +* get_output_charset() (email.charset.Charset method): email charset Representing character sets. + (line 111) +* get_param() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 478) +* get_parameters() (symtable.Function method): Examining Symbol Tables. + (line 71) +* get_params() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 457) +* get_path() (in module sysconfig): Installation paths. (line 67) +* get_paths() (in module sysconfig): Installation paths. (line 97) +* get_path_names() (in module sysconfig): Installation paths. + (line 62) +* get_payload() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 180) +* get_pid() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 6) +* get_pipe_transport() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 10) +* get_platform() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 9) +* get_platform() (in module sysconfig): Other functions<3>. (line 11) +* get_poly() (in module turtle): Special Turtle methods. + (line 17) +* get_position() (xdrlib.Unpacker method): Unpacker Objects. (line 12) +* get_protocol() (asyncio.BaseTransport method): Base Transport. + (line 87) +* get_python_inc() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 57) +* get_python_lib() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 68) +* get_python_version() (in module sysconfig): Other functions<3>. + (line 6) +* get_recsrc() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 84) +* get_referents() (in module gc): gc — Garbage Collector interface. + (line 142) +* get_referrers() (in module gc): gc — Garbage Collector interface. + (line 120) +* get_request() (socketserver.BaseServer method): Server Objects<2>. + (line 131) +* get_returncode() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 29) +* get_running_loop() (in module asyncio): Event Loop. (line 29) +* get_scheme() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 188) +* get_scheme_names() (in module sysconfig): Installation paths. + (line 57) +* get_sequences() (mailbox.MH method): MH. (line 53) +* get_sequences() (mailbox.MHMessage method): MHMessage. (line 32) +* get_server() (multiprocessing.managers.BaseManager method): Managers. + (line 46) +* get_server_certificate() (in module ssl): Certificate handling. + (line 71) +* get_shapepoly() (in module turtle): Appearance. (line 189) +* get_socket() (telnetlib.Telnet method): Telnet Objects. (line 93) +* get_source() (importlib.abc.InspectLoader method): importlib abc – Abstract base classes related to import. + (line 401) +* get_source() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 599) +* get_source() (importlib.machinery.ExtensionFileLoader method): importlib machinery – Importers and path hooks. + (line 342) +* get_source() (importlib.machinery.SourcelessFileLoader method): importlib machinery – Importers and path hooks. + (line 286) +* get_source() (zipimport.zipimporter method): zipimporter Objects. + (line 49) +* get_source_segment() (in module ast): ast Helpers. (line 70) +* get_special_folder_path() (built-in function): The Postinstallation script. + (line 32) +* get_stack() (asyncio.Task method): Task Object. (line 168) +* get_stack() (bdb.Bdb method): bdb — Debugger framework. + (line 339) +* get_starttag_text() (html.parser.HTMLParser method): HTMLParser Methods. + (line 32) +* get_start_method() (in module multiprocessing): Miscellaneous<3>. + (line 92) +* get_stats() (in module gc): gc — Garbage Collector interface. + (line 70) +* get_stderr() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 127) +* get_stderr() (wsgiref.simple_server.WSGIRequestHandler method): wsgiref simple_server – a simple WSGI HTTP server. + (line 98) +* get_stdin() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 122) +* get_string() (mailbox.Mailbox method): Mailbox objects. (line 167) +* get_subdir() (mailbox.MaildirMessage method): MaildirMessage. + (line 47) +* get_suffixes() (in module imp): imp — Access the import internals. + (line 26) +* get_symbols() (symtable.SymbolTable method): Examining Symbol Tables. + (line 57) +* get_tag() (in module imp): imp — Access the import internals. + (line 244) +* get_task_factory() (asyncio.loop method): Creating Futures and Tasks. + (line 42) +* get_terminal_size() (in module os): Querying the size of a terminal. + (line 8) +* get_terminal_size() (in module shutil): Querying the size of the output terminal. + (line 6) +* get_terminator() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 97) +* get_threshold() (in module gc): gc — Garbage Collector interface. + (line 115) +* get_token() (shlex.shlex method): shlex Objects. (line 8) +* get_traceback_limit() (in module tracemalloc): Functions<9>. + (line 22) +* get_traced_memory() (in module tracemalloc): Functions<9>. (line 32) +* get_tracemalloc_memory() (in module tracemalloc): Functions<9>. + (line 38) +* get_type() (symtable.SymbolTable method): Examining Symbol Tables. + (line 10) +* get_type_hints() (in module typing): Classes functions and decorators. + (line 627) +* get_unixfrom() (email.message.EmailMessage method): email message Representing an email message. + (line 152) +* get_unixfrom() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 162) +* get_unpack_formats() (in module shutil): Archiving operations. + (line 147) +* get_usage() (optparse.OptionParser method): Other methods. (line 22) +* get_value() (string.Formatter method): Custom String Formatting. + (line 65) +* get_version() (optparse.OptionParser method): Printing a version string. + (line 34) +* get_visible() (mailbox.BabylMessage method): BabylMessage. (line 63) +* get_wch() (curses.window method): Window Objects. (line 238) +* get_write_buffer_limits() (asyncio.WriteTransport method): Write-only Transports. + (line 23) +* get_write_buffer_size() (asyncio.WriteTransport method): Write-only Transports. + (line 19) +* GET_YIELD_FROM_ITER (opcode): Python Bytecode Instructions. + (line 117) +* gid (tarfile.TarInfo attribute): TarInfo Objects. (line 76) +* GIL: Glossary. (line 546) +* glob (module): glob — Unix style pathname pattern expansion. + (line 6) +* glob() (in module glob): glob — Unix style pathname pattern expansion. + (line 29) +* glob() (msilib.Directory method): Directory Objects. (line 39) +* glob() (pathlib.Path method): Methods<2>. (line 86) +* global interpreter lock: Thread State and the Global Interpreter Lock. + (line 6) +* global interpreter lock <1>: Glossary. (line 550) +* global; name; binding: The global statement. + (line 6) +* global; namespace: The standard type hierarchy. + (line 304) +* globals() (built-in function): Built-in Functions. (line 685) +* globs (doctest.DocTest attribute): DocTest Objects. (line 22) +* gmtime() (in module time): Functions<2>. (line 131) +* gname (tarfile.TarInfo attribute): TarInfo Objects. (line 84) +* GNOME: The Catalog constructor. + (line 6) +* GNUTranslations (class in gettext): The GNUTranslations class. + (line 33) +* GNU_FORMAT (in module tarfile): tarfile — Read and write tar archive files. + (line 226) +* gnu_getopt() (in module getopt): getopt — C-style parser for command line options. + (line 62) +* got (doctest.DocTestFailure attribute): Debugging. (line 204) +* goto() (in module turtle): Turtle motion. (line 74) +* grammar: Notation. (line 6) +* Graphical User Interface: Graphical User Interfaces with Tk. + (line 6) +* GREATER (in module token): token — Constants used with Python parse trees. + (line 110) +* GREATEREQUAL (in module token): token — Constants used with Python parse trees. + (line 146) +* Greenwich Mean Time: time — Time access and conversions. + (line 40) +* GRND_NONBLOCK (in module os): Random numbers<2>. (line 73) +* GRND_RANDOM (in module os): Random numbers<2>. (line 86) +* Group (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 457) +* group() (nntplib.NNTP method): Methods<3>. (line 158) +* group() (pathlib.Path method): Methods<2>. (line 109) +* group() (re.Match method): Match Objects. (line 29) +* groupby() (in module itertools): Itertool functions. (line 298) +* groupdict() (re.Match method): Match Objects. (line 121) +* groupindex (re.Pattern attribute): Regular Expression Objects. + (line 112) +* grouping: Indentation. (line 6) +* groups (email.headerregistry.AddressHeader attribute): email headerregistry Custom Header Objects. + (line 188) +* groups (re.Pattern attribute): Regular Expression Objects. + (line 108) +* groups() (re.Match method): Match Objects. (line 98) +* grp (module): grp — The group database. + (line 6) +* gt() (in module operator): operator — Standard operators as functions. + (line 24) +* guess_all_extensions() (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 51) +* guess_all_extensions() (mimetypes.MimeTypes method): MimeTypes Objects. + (line 65) +* guess_extension() (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 63) +* guess_extension() (mimetypes.MimeTypes method): MimeTypes Objects. + (line 55) +* guess_scheme() (in module wsgiref.util): wsgiref util – WSGI environment utilities. + (line 12) +* guess_type() (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 24) +* guess_type() (mimetypes.MimeTypes method): MimeTypes Objects. + (line 60) +* GUI: Graphical User Interfaces with Tk. + (line 6) +* gzip (module): gzip — Support for gzip files. + (line 6) +* gzip command line option; –best: Command line options. + (line 14) +* gzip command line option; -d: Command line options. + (line 18) +* gzip command line option; –decompress: Command line options. + (line 18) +* gzip command line option; –fast: Command line options. + (line 10) +* gzip command line option; -h: Command line options. + (line 22) +* gzip command line option; –help: Command line options. + (line 22) +* gzip command line option; file: Command line options. + (line 6) +* GzipFile (class in gzip): gzip — Support for gzip files. + (line 71) +* halfdelay() (in module curses): Functions<3>. (line 195) +* Handle (class in asyncio): Callback Handles. (line 6) +* handle an exception: Exceptions<2>. (line 6) +* handle() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 182) +* handle() (logging.Handler method): Handler Objects. (line 81) +* handle() (logging.handlers.QueueListener method): QueueListener. + (line 63) +* handle() (logging.Logger method): Logger Objects. (line 291) +* handle() (logging.NullHandler method): NullHandler. (line 20) +* handle() (socketserver.BaseRequestHandler method): Request Handler Objects. + (line 20) +* handle() (wsgiref.simple_server.WSGIRequestHandler method): wsgiref simple_server – a simple WSGI HTTP server. + (line 103) +* handleError() (logging.Handler method): Handler Objects. (line 88) +* handleError() (logging.handlers.SocketHandler method): SocketHandler. + (line 33) +* Handler (class in logging): Handler Objects. (line 11) +* handler() (in module cgitb): cgitb — Traceback manager for CGI scripts. + (line 66) +* handle_accept() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 143) +* handle_accepted() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 153) +* handle_charref() (html.parser.HTMLParser method): HTMLParser Methods. + (line 93) +* handle_close() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 134) +* handle_comment() (html.parser.HTMLParser method): HTMLParser Methods. + (line 102) +* handle_connect() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 128) +* handle_data() (html.parser.HTMLParser method): HTMLParser Methods. + (line 80) +* handle_decl() (html.parser.HTMLParser method): HTMLParser Methods. + (line 115) +* handle_defect() (email.policy.Policy method): email policy Policy Objects. + (line 218) +* handle_endtag() (html.parser.HTMLParser method): HTMLParser Methods. + (line 64) +* handle_entityref() (html.parser.HTMLParser method): HTMLParser Methods. + (line 86) +* handle_error() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 138) +* handle_error() (socketserver.BaseServer method): Server Objects<2>. + (line 137) +* handle_expect_100() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 195) +* handle_expt() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 122) +* handle_one_request() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 189) +* handle_pi() (html.parser.HTMLParser method): HTMLParser Methods. + (line 123) +* handle_read() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 107) +* handle_request() (socketserver.BaseServer method): Server Objects<2>. + (line 21) +* handle_request() (xmlrpc.server.CGIXMLRPCRequestHandler method): CGIXMLRPCRequestHandler. + (line 50) +* handle_startendtag() (html.parser.HTMLParser method): HTMLParser Methods. + (line 71) +* handle_starttag() (html.parser.HTMLParser method): HTMLParser Methods. + (line 45) +* handle_timeout() (socketserver.BaseServer method): Server Objects<2>. + (line 147) +* handle_write() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 112) +* harmonic_mean() (in module statistics): Function details. (line 84) +* hasattr() (built-in function): Built-in Functions. (line 692) +* hasAttribute() (xml.dom.Element method): Element Objects<2>. + (line 22) +* hasAttributeNS() (xml.dom.Element method): Element Objects<2>. + (line 26) +* hasAttributes() (xml.dom.Node method): Node Objects. (line 94) +* hasChildNodes() (xml.dom.Node method): Node Objects. (line 98) +* hascompare (in module dis): Opcode collections. (line 49) +* hasconst (in module dis): Opcode collections. (line 21) +* hasFeature() (xml.dom.DOMImplementation method): DOMImplementation Objects. + (line 11) +* hasfree (in module dis): Opcode collections. (line 25) +* hash character: Comments. (line 6) +* hash() (built-in function): Built-in Functions. (line 699) +* hash-based pyc: Glossary. (line 573) +* hash.block_size (in module hashlib): Hash algorithms. (line 99) +* hash.digest_size (in module hashlib): Hash algorithms. (line 95) +* hashable: Dictionary displays. + (line 37) +* hashable <1>: Glossary. (line 579) +* Hashable (class in collections.abc): Collections Abstract Base Classes. + (line 108) +* Hashable (class in typing): Classes functions and decorators. + (line 195) +* hasHandlers() (logging.Logger method): Logger Objects. (line 306) +* hashfunc (C type): Slot Type typedefs. (line 87) +* hashlib (module): hashlib — Secure hashes and message digests. + (line 6) +* hash_info (in module sys): sys — System-specific parameters and functions. + (line 826) +* hasjabs (in module dis): Opcode collections. (line 41) +* hasjrel (in module dis): Opcode collections. (line 37) +* haslocal (in module dis): Opcode collections. (line 45) +* hasname (in module dis): Opcode collections. (line 33) +* HAS_ALPN (in module ssl): Constants<9>. (line 350) +* has_children() (symtable.SymbolTable method): Examining Symbol Tables. + (line 39) +* has_colors() (in module curses): Functions<3>. (line 171) +* has_dualstack_ipv6() (in module socket): Creating sockets. (line 139) +* HAS_ECDH (in module ssl): Constants<9>. (line 366) +* has_exec() (symtable.SymbolTable method): Examining Symbol Tables. + (line 44) +* has_extn() (smtplib.SMTP method): SMTP Objects. (line 85) +* has_function() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 212) +* has_header() (csv.Sniffer method): Module Contents<3>. (line 218) +* has_header() (urllib.request.Request method): Request Objects. + (line 94) +* has_ic() (in module curses): Functions<3>. (line 176) +* has_il() (in module curses): Functions<3>. (line 183) +* has_ipv6 (in module socket): Constants<8>. (line 213) +* has_key (2to3 fixer): Fixers. (line 119) +* has_key() (in module curses): Functions<3>. (line 190) +* has_location (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 418) +* HAS_NEVER_CHECK_COMMON_NAME (in module ssl): Constants<9>. (line 358) +* has_nonstandard_attr() (http.cookiejar.Cookie method): Cookie Objects<2>. + (line 94) +* HAS_NPN (in module ssl): Constants<9>. (line 381) +* has_option() (configparser.ConfigParser method): ConfigParser Objects. + (line 106) +* has_option() (optparse.OptionParser method): Querying and manipulating your option parser. + (line 42) +* has_section() (configparser.ConfigParser method): ConfigParser Objects. + (line 97) +* HAS_SNI (in module ssl): Constants<9>. (line 374) +* HAS_SSLv2 (in module ssl): Constants<9>. (line 391) +* HAS_SSLv3 (in module ssl): Constants<9>. (line 398) +* has_ticket (ssl.SSLSession attribute): SSL session. (line 20) +* HAS_TLSv1 (in module ssl): Constants<9>. (line 405) +* HAS_TLSv1_1 (in module ssl): Constants<9>. (line 412) +* HAS_TLSv1_2 (in module ssl): Constants<9>. (line 419) +* HAS_TLSv1_3 (in module ssl): Constants<9>. (line 426) +* HAVE_ARGUMENT (opcode): Python Bytecode Instructions. + (line 901) +* HAVE_CONTEXTVAR (in module decimal): Constants<4>. (line 32) +* HAVE_DOCSTRINGS (in module test.support): test support — Utilities for the Python test suite. + (line 137) +* HAVE_THREADS (in module decimal): Constants<4>. (line 25) +* HCI_DATA_DIR (in module socket): Constants<8>. (line 226) +* HCI_FILTER (in module socket): Constants<8>. (line 226) +* HCI_TIME_STAMP (in module socket): Constants<8>. (line 226) +* head() (nntplib.NNTP method): Methods<3>. (line 266) +* Header (class in email.header): email header Internationalized headers. + (line 60) +* HeaderError: tarfile — Read and write tar archive files. + (line 206) +* HeaderParseError: email errors Exception and Defect classes. + (line 26) +* HeaderParser (class in email.parser): Parser API. (line 106) +* HeaderRegistry (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 299) +* Headers (class in wsgiref.headers): wsgiref headers – WSGI response header tools. + (line 9) +* headers (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 101) +* headers (urllib.error.HTTPError attribute): urllib error — Exception classes raised by urllib request. + (line 49) +* headers (xmlrpc.client.ProtocolError attribute): ProtocolError Objects. + (line 25) +* header_encode() (email.charset.Charset method): email charset Representing character sets. + (line 118) +* header_encode_lines() (email.charset.Charset method): email charset Representing character sets. + (line 125) +* header_encoding (email.charset.Charset attribute): email charset Representing character sets. + (line 59) +* header_factory (email.policy.EmailPolicy attribute): email policy Policy Objects. + (line 391) +* header_fetch_parse() (email.policy.Compat32 method): email policy Policy Objects. + (line 578) +* header_fetch_parse() (email.policy.EmailPolicy method): email policy Policy Objects. + (line 441) +* header_fetch_parse() (email.policy.Policy method): email policy Policy Objects. + (line 302) +* header_items() (urllib.request.Request method): Request Objects. + (line 125) +* header_max_count() (email.policy.EmailPolicy method): email policy Policy Objects. + (line 418) +* header_max_count() (email.policy.Policy method): email policy Policy Objects. + (line 243) +* header_offset (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 131) +* header_source_parse() (email.policy.Compat32 method): email policy Policy Objects. + (line 566) +* header_source_parse() (email.policy.EmailPolicy method): email policy Policy Objects. + (line 424) +* header_source_parse() (email.policy.Policy method): email policy Policy Objects. + (line 265) +* header_store_parse() (email.policy.Compat32 method): email policy Policy Objects. + (line 574) +* header_store_parse() (email.policy.EmailPolicy method): email policy Policy Objects. + (line 432) +* header_store_parse() (email.policy.Policy method): email policy Policy Objects. + (line 286) +* heading() (in module turtle): Tell Turtle’s state. + (line 58) +* heading() (tkinter.ttk.Treeview method): ttk Treeview. (line 105) +* heapify() (in module heapq): heapq — Heap queue algorithm. + (line 56) +* heapmin() (in module msvcrt): Other Functions. (line 6) +* heappop() (in module heapq): heapq — Heap queue algorithm. + (line 42) +* heappush() (in module heapq): heapq — Heap queue algorithm. + (line 37) +* heappushpop() (in module heapq): heapq — Heap queue algorithm. + (line 49) +* heapq (module): heapq — Heap queue algorithm. + (line 6) +* heapreplace() (in module heapq): heapq — Heap queue algorithm. + (line 60) +* helo() (smtplib.SMTP method): SMTP Objects. (line 47) +* help (optparse.Option attribute): Option attributes. (line 72) +* help (pdb command): Debugger Commands. (line 47) +* help() (built-in function): Built-in Functions. (line 712) +* help() (nntplib.NNTP method): Methods<3>. (line 214) +* help; online: pydoc — Documentation generator and online help system. + (line 8) +* herror: Exceptions<11>. (line 13) +* hex (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 127) +* hex() (built-in function): Built-in Functions. (line 734) +* hex() (bytearray method): Bytearray Objects. (line 52) +* hex() (bytes method): Bytes Objects. (line 78) +* hex() (float method): Additional Methods on Float. + (line 33) +* hex() (memoryview method): Memory Views. (line 184) +* hexadecimal literal: Numeric literals. (line 6) +* hexadecimal; literals: Numeric Types — int float complex. + (line 19) +* hexbin() (in module binhex): binhex — Encode and decode binhex4 files. + (line 23) +* hexdigest() (hashlib.hash method): Hash algorithms. (line 134) +* hexdigest() (hashlib.shake method): SHAKE variable length digests. + (line 17) +* hexdigest() (hmac.HMAC method): hmac — Keyed-Hashing for Message Authentication. + (line 69) +* hexdigits (in module string): String constants. (line 28) +* hexlify() (in module binascii): binascii — Convert between binary and ASCII. + (line 131) +* hexversion (in module sys): sys — System-specific parameters and functions. + (line 867) +* hidden() (curses.panel.Panel method): Panel Objects. (line 25) +* hide() (curses.panel.Panel method): Panel Objects. (line 30) +* hide() (tkinter.ttk.Notebook method): ttk Notebook. (line 23) +* hideturtle() (in module turtle): Visibility. (line 6) +* hide_cookie2 (http.cookiejar.CookiePolicy attribute): CookiePolicy Objects. + (line 76) +* HierarchyRequestErr: Exceptions<16>. (line 29) +* HIGHEST_PROTOCOL (in module pickle): Module Interface. (line 14) +* HIGH_PRIORITY_CLASS (in module subprocess): Windows Constants. + (line 66) +* HKEY_CLASSES_ROOT (in module winreg): HKEY_* Constants. (line 6) +* HKEY_CURRENT_CONFIG (in module winreg): HKEY_* Constants. (line 38) +* HKEY_CURRENT_USER (in module winreg): HKEY_* Constants. (line 12) +* HKEY_DYN_DATA (in module winreg): HKEY_* Constants. (line 43) +* HKEY_LOCAL_MACHINE (in module winreg): HKEY_* Constants. (line 19) +* HKEY_PERFORMANCE_DATA (in module winreg): HKEY_* Constants. + (line 31) +* HKEY_USERS (in module winreg): HKEY_* Constants. (line 25) +* hline() (curses.window method): Window Objects. (line 279) +* HList (class in tkinter.tix): Hierarchical ListBox. + (line 6) +* hls_to_rgb() (in module colorsys): colorsys — Conversions between color systems. + (line 40) +* hmac (module): hmac — Keyed-Hashing for Message Authentication. + (line 6) +* HOME: os path. (line 15) +* HOME <1>: Changes in the Python API. + (line 116) +* HOME <2>: os path — Common pathname manipulations. + (line 140) +* HOME <3>: os path — Common pathname manipulations. + (line 156) +* HOME <4>: distutils util — Miscellaneous other utility functions. + (line 77) +* HOME <5>: Location and names of config files. + (line 50) +* HOME <6>: Location and names of config files. + (line 67) +* home() (in module turtle): Turtle motion. (line 161) +* home() (pathlib.Path class method): Methods<2>. (line 25) +* HOMEDRIVE: os path — Common pathname manipulations. + (line 147) +* HOMEDRIVE <1>: Location and names of config files. + (line 68) +* HOMEPATH: os path — Common pathname manipulations. + (line 147) +* HOMEPATH <1>: Location and names of config files. + (line 68) +* hooks; import: Import hooks. (line 6) +* hooks; meta: Import hooks. (line 6) +* hooks; path: Import hooks. (line 6) +* hook_compressed() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 188) +* hook_encoded() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 199) +* host (urllib.request.Request attribute): Request Objects. (line 25) +* hostmask (ipaddress.IPv4Network attribute): Network objects. + (line 97) +* hostmask (ipaddress.IPv6Network attribute): Network objects. + (line 310) +* hostname_checks_common_name (ssl.SSLContext attribute): SSL Contexts. + (line 630) +* hosts (netrc.netrc attribute): netrc Objects. (line 23) +* hosts() (ipaddress.IPv4Network method): Network objects. (line 136) +* hosts() (ipaddress.IPv6Network method): Network objects. (line 328) +* hour (datetime.datetime attribute): datetime Objects. (line 277) +* hour (datetime.time attribute): time Objects. (line 50) +* HRESULT (class in ctypes): Fundamental data types<2>. + (line 208) +* hStdError (subprocess.STARTUPINFO attribute): Windows Popen Helpers. + (line 42) +* hStdInput (subprocess.STARTUPINFO attribute): Windows Popen Helpers. + (line 27) +* hStdOutput (subprocess.STARTUPINFO attribute): Windows Popen Helpers. + (line 35) +* hsv_to_rgb() (in module colorsys): colorsys — Conversions between color systems. + (line 48) +* ht() (in module turtle): Visibility. (line 6) +* HTML: html parser — Simple HTML and XHTML parser. + (line 8) +* HTML <1>: urllib request Restrictions. + (line 29) +* html (module): html — HyperText Markup Language support. + (line 6) +* html() (in module cgitb): cgitb — Traceback manager for CGI scripts. + (line 57) +* html.entities (module): html entities — Definitions of HTML general entities. + (line 6) +* html.parser (module): html parser — Simple HTML and XHTML parser. + (line 6) +* html5 (in module html.entities): html entities — Definitions of HTML general entities. + (line 14) +* HTMLCalendar (class in calendar): calendar — General calendar-related functions. + (line 163) +* HtmlDiff (class in difflib): difflib — Helpers for computing deltas. + (line 81) +* HTMLParser (class in html.parser): html parser — Simple HTML and XHTML parser. + (line 14) +* htonl() (in module socket): Other functions<2>. (line 199) +* htons() (in module socket): Other functions<2>. (line 206) +* HTTP (in module email.policy): email policy Policy Objects. + (line 508) +* http (module): http — HTTP modules. + (line 6) +* http.client (module): http client — HTTP protocol client. + (line 6) +* http.cookiejar (module): http cookiejar — Cookie handling for HTTP clients. + (line 6) +* http.cookies (module): http cookies — HTTP state management. + (line 6) +* http.server (module): http server — HTTP servers. + (line 6) +* HTTP; http (standard module): http — HTTP modules. + (line 8) +* HTTP; http.client (standard module): http client — HTTP protocol client. + (line 8) +* HTTP; protocol: cgi — Common Gateway Interface support. + (line 8) +* HTTP; protocol <1>: urllib request Restrictions. + (line 6) +* HTTP; protocol <2>: urllib request Restrictions. + (line 29) +* HTTP; protocol <3>: http — HTTP modules. + (line 8) +* HTTP; protocol <4>: http client — HTTP protocol client. + (line 8) +* HTTP; protocol <5>: http server — HTTP servers. + (line 8) +* HTTPBasicAuthHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 355) +* HTTPConnection (class in http.client): http client — HTTP protocol client. + (line 26) +* HTTPCookieProcessor (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 286) +* httpd: http server — HTTP servers. + (line 8) +* HTTPDefaultErrorHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 277) +* HTTPDigestAuthHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 379) +* HTTPError: urllib error — Exception classes raised by urllib request. + (line 30) +* HTTPErrorProcessor (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 439) +* HTTPException: http client — HTTP protocol client. + (line 128) +* HTTPHandler (class in logging.handlers): HTTPHandler. (line 10) +* HTTPHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 403) +* HTTPPasswordMgr (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 312) +* HTTPPasswordMgrWithDefaultRealm (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 316) +* HTTPPasswordMgrWithPriorAuth (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 322) +* HTTPRedirectHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 282) +* HTTPResponse (class in http.client): http client — HTTP protocol client. + (line 97) +* HTTPSConnection (class in http.client): http client — HTTP protocol client. + (line 57) +* HTTPServer (class in http.server): http server — HTTP servers. + (line 25) +* HTTPSHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 407) +* HTTPStatus (class in http): http — HTTP modules. + (line 28) +* https_open() (urllib.request.HTTPSHandler method): HTTPSHandler Objects. + (line 6) +* HTTPS_PORT (in module http.client): http client — HTTP protocol client. + (line 201) +* https_response() (urllib.request.HTTPErrorProcessor method): HTTPErrorProcessor Objects. + (line 18) +* http_error_301() (urllib.request.HTTPRedirectHandler method): HTTPRedirectHandler Objects. + (line 35) +* http_error_302() (urllib.request.HTTPRedirectHandler method): HTTPRedirectHandler Objects. + (line 42) +* http_error_303() (urllib.request.HTTPRedirectHandler method): HTTPRedirectHandler Objects. + (line 48) +* http_error_307() (urllib.request.HTTPRedirectHandler method): HTTPRedirectHandler Objects. + (line 54) +* http_error_401() (urllib.request.HTTPBasicAuthHandler method): HTTPBasicAuthHandler Objects. + (line 6) +* http_error_401() (urllib.request.HTTPDigestAuthHandler method): HTTPDigestAuthHandler Objects. + (line 6) +* http_error_407() (urllib.request.ProxyBasicAuthHandler method): ProxyBasicAuthHandler Objects. + (line 6) +* http_error_407() (urllib.request.ProxyDigestAuthHandler method): ProxyDigestAuthHandler Objects. + (line 6) +* http_error_auth_reqed() (urllib.request.AbstractBasicAuthHandler method): AbstractBasicAuthHandler Objects. + (line 6) +* http_error_auth_reqed() (urllib.request.AbstractDigestAuthHandler method): AbstractDigestAuthHandler Objects. + (line 6) +* http_error_default() (urllib.request.BaseHandler method): BaseHandler Objects. + (line 65) +* http_open() (urllib.request.HTTPHandler method): HTTPHandler Objects. + (line 6) +* HTTP_PORT (in module http.client): http client — HTTP protocol client. + (line 197) +* http_proxy: urllib request — Extensible library for opening URLs. + (line 85) +* http_proxy <1>: Examples<21>. (line 97) +* http_proxy <2>: Basic Authentication. + (line 62) +* http_response() (urllib.request.HTTPErrorProcessor method): HTTPErrorProcessor Objects. + (line 6) +* http_version (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 300) +* hypot() (in module math): Trigonometric functions. + (line 44) +* I (in module re): Module Contents. (line 63) +* I/O control; buffering: Built-in Functions. (line 1217) +* I/O control; buffering <1>: Socket Objects. (line 199) +* iadd() (in module operator): In-place Operators. (line 36) +* iand() (in module operator): In-place Operators. (line 41) +* iconcat() (in module operator): In-place Operators. (line 46) +* id (ssl.SSLSession attribute): SSL session. (line 12) +* id() (built-in function): Built-in Functions. (line 765) +* id() (unittest.TestCase method): Test cases. (line 727) +* idcok() (curses.window method): Window Objects. (line 286) +* ident (select.kevent attribute): Kevent Objects. (line 8) +* ident (threading.Thread attribute): Thread Objects. (line 148) +* identchars (cmd.Cmd attribute): Cmd Objects. (line 127) +* identifier: Identifiers and keywords. + (line 6) +* identifier <1>: Identifiers Names. (line 6) +* identify() (tkinter.ttk.Notebook method): ttk Notebook. (line 32) +* identify() (tkinter.ttk.Treeview method): ttk Treeview. (line 143) +* identify() (tkinter.ttk.Widget method): ttk Widget. (line 11) +* identify_column() (tkinter.ttk.Treeview method): ttk Treeview. + (line 153) +* identify_element() (tkinter.ttk.Treeview method): ttk Treeview. + (line 182) +* identify_region() (tkinter.ttk.Treeview method): ttk Treeview. + (line 160) +* identify_row() (tkinter.ttk.Treeview method): ttk Treeview. + (line 149) +* identity of an object: Objects values and types. + (line 11) +* identity; test: Membership test operations. + (line 40) +* idioms (2to3 fixer): Fixers. (line 123) +* IDLE: IDLE<3>. (line 8) +* IDLE <1>: Glossary. (line 599) +* IDLESTARTUP: Startup and code execution. + (line 7) +* IDLE_PRIORITY_CLASS (in module subprocess): Windows Constants. + (line 73) +* idlok() (curses.window method): Window Objects. (line 294) +* if; conditional expression: Conditional expressions. + (line 6) +* if; in comprehensions: Displays for lists sets and dictionaries. + (line 14) +* ifloordiv() (in module operator): In-place Operators. (line 52) +* if_indextoname() (in module socket): Other functions<2>. (line 401) +* if_nameindex() (in module socket): Other functions<2>. (line 358) +* if_nametoindex() (in module socket): Other functions<2>. (line 383) +* iglob() (in module glob): glob — Unix style pathname pattern expansion. + (line 54) +* ignorableWhitespace() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 150) +* ignore (pdb command): Debugger Commands. (line 111) +* IGNORECASE (in module re): Module Contents. (line 63) +* ignore_errors() (in module codecs): Error Handlers. (line 140) +* IGNORE_EXCEPTION_DETAIL (in module doctest): Option Flags. (line 58) +* ignore_patterns() (in module shutil): Directory and files operations. + (line 184) +* ihave() (nntplib.NNTP method): Methods<3>. (line 291) +* IISCGIHandler (class in wsgiref.handlers): wsgiref handlers – server/gateway base classes. + (line 24) +* ilshift() (in module operator): In-place Operators. (line 57) +* imag (numbers.Complex attribute): The numeric tower. (line 19) +* imaginary literal: Numeric literals. (line 6) +* imap() (multiprocessing.pool.Pool method): Process Pools. (line 126) +* IMAP4 (class in imaplib): imaplib — IMAP4 protocol client. + (line 20) +* IMAP4.abort: imaplib — IMAP4 protocol client. + (line 49) +* IMAP4.error: imaplib — IMAP4 protocol client. + (line 44) +* IMAP4.readonly: imaplib — IMAP4 protocol client. + (line 56) +* IMAP4; protocol: imaplib — IMAP4 protocol client. + (line 8) +* IMAP4_SSL (class in imaplib): imaplib — IMAP4 protocol client. + (line 65) +* IMAP4_SSL; protocol: imaplib — IMAP4 protocol client. + (line 8) +* IMAP4_stream (class in imaplib): imaplib — IMAP4 protocol client. + (line 99) +* IMAP4_stream; protocol: imaplib — IMAP4 protocol client. + (line 8) +* imaplib (module): imaplib — IMAP4 protocol client. + (line 6) +* imap_unordered() (multiprocessing.pool.Pool method): Process Pools. + (line 141) +* imatmul() (in module operator): In-place Operators. (line 72) +* imghdr (module): imghdr — Determine the type of an image. + (line 6) +* immedok() (curses.window method): Window Objects. (line 300) +* immutable: Glossary. (line 605) +* immutable object: Objects values and types. + (line 11) +* immutable; data; type: Literals<2>. (line 17) +* immutable; object: Literals<2>. (line 17) +* immutable; object <1>: Dictionary displays. + (line 37) +* immutable; sequence; types: Immutable Sequence Types. + (line 6) +* imod() (in module operator): In-place Operators. (line 62) +* imp (module): imp — Access the import internals. + (line 6) +* ImpImporter (class in pkgutil): pkgutil — Package extension utility. + (line 53) +* implementation (in module sys): sys — System-specific parameters and functions. + (line 890) +* ImpLoader (class in pkgutil): pkgutil — Package extension utility. + (line 69) +* impl_detail() (in module test.support): test support — Utilities for the Python test suite. + (line 637) +* import (2to3 fixer): Fixers. (line 139) +* import hooks: Import hooks. (line 6) +* import machinery: The import system. (line 6) +* import path: Glossary. (line 613) +* importer: Glossary. (line 626) +* ImportError: Concrete exceptions. + (line 38) +* importing: Glossary. (line 621) +* importlib (module): importlib — The implementation of import. + (line 6) +* importlib.abc (module): importlib abc – Abstract base classes related to import. + (line 6) +* importlib.machinery (module): importlib machinery – Importers and path hooks. + (line 6) +* importlib.resources (module): importlib resources – Resources. + (line 6) +* importlib.util (module): importlib util – Utility code for importers. + (line 6) +* imports (2to3 fixer): Fixers. (line 143) +* imports2 (2to3 fixer): Fixers. (line 147) +* ImportWarning: Warnings. (line 46) +* import_fresh_module() (in module test.support): test support — Utilities for the Python test suite. + (line 733) +* IMPORT_FROM (opcode): Python Bytecode Instructions. + (line 644) +* import_module() (in module importlib): Functions<10>. (line 15) +* import_module() (in module test.support): test support — Utilities for the Python test suite. + (line 718) +* IMPORT_NAME (opcode): Python Bytecode Instructions. + (line 635) +* IMPORT_STAR (opcode): Python Bytecode Instructions. + (line 366) +* ImproperConnectionState: http client — HTTP protocol client. + (line 158) +* imul() (in module operator): In-place Operators. (line 67) +* inch() (curses.window method): Window Objects. (line 308) +* inclusive (tracemalloc.DomainFilter attribute): DomainFilter. + (line 12) +* inclusive (tracemalloc.Filter attribute): Filter. (line 38) +* inclusive; or: Binary bitwise operations. + (line 18) +* Incomplete: binascii — Convert between binary and ASCII. + (line 178) +* IncompleteRead: http client — HTTP protocol client. + (line 154) +* IncompleteReadError: Exceptions<9>. (line 42) +* IncrementalDecoder (class in codecs): IncrementalDecoder Objects. + (line 11) +* incrementaldecoder (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 78) +* IncrementalEncoder (class in codecs): IncrementalEncoder Objects. + (line 11) +* incrementalencoder (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 78) +* IncrementalNewlineDecoder (class in io): Text I/O<2>. (line 234) +* IncrementalParser (class in xml.sax.xmlreader): xml sax xmlreader — Interface for XML parsers. + (line 20) +* increment_lineno() (in module ast): ast Helpers. (line 90) +* incr_item(): Exceptions<18>. (line 78) +* incr_item() <1>: Exceptions<18>. (line 123) +* indent (doctest.Example attribute): Example Objects. (line 47) +* INDENT (in module token): token — Constants used with Python parse trees. + (line 50) +* INDENT token: Indentation. (line 33) +* indent() (in module textwrap): textwrap — Text wrapping and filling. + (line 91) +* indentation: Indentation. (line 6) +* IndentationError: Concrete exceptions. + (line 258) +* index operation: The standard type hierarchy. + (line 127) +* index() (array.array method): array — Efficient arrays of numeric values. + (line 193) +* index() (bytearray method): Bytes and Bytearray Operations. + (line 99) +* index() (bytes method): Bytes and Bytearray Operations. + (line 99) +* index() (collections.deque method): deque objects. (line 70) +* index() (in module operator): operator — Standard operators as functions. + (line 93) +* index() (multiprocessing.shared_memory.ShareableList method): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 268) +* index() (sequence method): Common Sequence Operations. + (line 21) +* index() (str method): String Methods<2>. (line 162) +* index() (tkinter.ttk.Notebook method): ttk Notebook. (line 37) +* index() (tkinter.ttk.Treeview method): ttk Treeview. (line 188) +* IndexError: Concrete exceptions. + (line 59) +* indexOf() (in module operator): operator — Standard operators as functions. + (line 197) +* IndexSizeErr: Exceptions<16>. (line 34) +* indices() (slice method): The standard type hierarchy. + (line 809) +* inet_aton() (in module socket): Other functions<2>. (line 219) +* inet_ntoa() (in module socket): Other functions<2>. (line 239) +* inet_ntop() (in module socket): Other functions<2>. (line 275) +* inet_pton() (in module socket): Other functions<2>. (line 256) +* Inexact (class in decimal): Signals. (line 42) +* inf (in module cmath): Constants<3>. (line 20) +* inf (in module math): Constants<2>. (line 24) +* infile (shlex.shlex attribute): shlex Objects. (line 140) +* Infinity: Built-in Functions. (line 584) +* infj (in module cmath): Constants<3>. (line 26) +* info() (dis.Bytecode method): Bytecode analysis. (line 53) +* info() (gettext.NullTranslations method): The NullTranslations class. + (line 78) +* info() (in module logging): Module-Level Functions. + (line 119) +* info() (logging.Logger method): Logger Objects. (line 215) +* infolist() (zipfile.ZipFile method): ZipFile Objects. (line 98) +* inheritance: Class definitions. (line 6) +* ini file: configparser — Configuration file parser. + (line 8) +* init() (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 78) +* inited (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 117) +* initgroups() (in module os): Process Parameters. (line 306) +* initialize_options() (distutils.cmd.Command method): Creating a new Distutils command. + (line 21) +* initial_indent (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 199) +* initproc (C type): Slot Type typedefs. (line 45) +* initscr() (in module curses): Functions<3>. (line 229) +* init_color() (in module curses): Functions<3>. (line 204) +* init_database() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 60) +* init_pair() (in module curses): Functions<3>. (line 216) +* inode() (os.DirEntry method): Files and Directories. + (line 872) +* INPLACE_ADD (opcode): Python Bytecode Instructions. + (line 222) +* INPLACE_AND (opcode): Python Bytecode Instructions. + (line 238) +* INPLACE_FLOOR_DIVIDE (opcode): Python Bytecode Instructions. + (line 210) +* INPLACE_LSHIFT (opcode): Python Bytecode Instructions. + (line 230) +* INPLACE_MATRIX_MULTIPLY (opcode): Python Bytecode Instructions. + (line 204) +* INPLACE_MODULO (opcode): Python Bytecode Instructions. + (line 218) +* INPLACE_MULTIPLY (opcode): Python Bytecode Instructions. + (line 200) +* INPLACE_OR (opcode): Python Bytecode Instructions. + (line 246) +* INPLACE_POWER (opcode): Python Bytecode Instructions. + (line 196) +* INPLACE_RSHIFT (opcode): Python Bytecode Instructions. + (line 234) +* INPLACE_SUBTRACT (opcode): Python Bytecode Instructions. + (line 226) +* INPLACE_TRUE_DIVIDE (opcode): Python Bytecode Instructions. + (line 214) +* INPLACE_XOR (opcode): Python Bytecode Instructions. + (line 242) +* input: Expression input. (line 6) +* input (2to3 fixer): Fixers. (line 153) +* input() (built-in function): Built-in Functions. (line 778) +* input() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 54) +* InputOnly (class in tkinter.tix): Miscellaneous Widgets. + (line 6) +* InputSource (class in xml.sax.xmlreader): xml sax xmlreader — Interface for XML parsers. + (line 51) +* input_charset (email.charset.Charset attribute): email charset Representing character sets. + (line 53) +* input_codec (email.charset.Charset attribute): email charset Representing character sets. + (line 82) +* inquiry (C type): Supporting Cyclic Garbage Collection. + (line 134) +* insch() (curses.window method): Window Objects. (line 314) +* insdelln() (curses.window method): Window Objects. (line 321) +* insert() (array.array method): array — Efficient arrays of numeric values. + (line 198) +* insert() (collections.deque method): deque objects. (line 78) +* insert() (sequence method): Mutable Sequence Types. + (line 16) +* insert() (tkinter.ttk.Notebook method): ttk Notebook. (line 42) +* insert() (tkinter.ttk.Treeview method): ttk Treeview. (line 193) +* insert() (xml.etree.ElementTree.Element method): Element Objects. + (line 140) +* insertBefore() (xml.dom.Node method): Node Objects. (line 121) +* insertln() (curses.window method): Window Objects. (line 329) +* insert_text() (in module readline): Line buffer. (line 13) +* insnstr() (curses.window method): Window Objects. (line 334) +* insort() (in module bisect): bisect — Array bisection algorithm. + (line 52) +* insort_left() (in module bisect): bisect — Array bisection algorithm. + (line 45) +* insort_right() (in module bisect): bisect — Array bisection algorithm. + (line 52) +* inspect (module): inspect — Inspect live objects. + (line 6) +* inspect command line option; –details: Command Line Interface<3>. + (line 13) +* InspectLoader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 382) +* insstr() (curses.window method): Window Objects. (line 345) +* install() (gettext.NullTranslations method): The NullTranslations class. + (line 100) +* install() (in module gettext): Class-based API. (line 72) +* installHandler() (in module unittest): Signal Handling. (line 29) +* install_opener() (in module urllib.request): urllib request — Extensible library for opening URLs. + (line 116) +* install_scripts() (venv.EnvBuilder method): API<2>. (line 125) +* instance; call: Calls. (line 147) +* instate() (tkinter.ttk.Widget method): ttk Widget. (line 18) +* instr() (curses.window method): Window Objects. (line 355) +* instream (shlex.shlex attribute): shlex Objects. (line 146) +* Instruction (class in dis): Python Bytecode Instructions. + (line 10) +* Instruction.arg (in module dis): Python Bytecode Instructions. + (line 24) +* Instruction.argrepr (in module dis): Python Bytecode Instructions. + (line 32) +* Instruction.argval (in module dis): Python Bytecode Instructions. + (line 28) +* Instruction.is_jump_target (in module dis): Python Bytecode Instructions. + (line 44) +* Instruction.offset (in module dis): Python Bytecode Instructions. + (line 36) +* Instruction.opcode (in module dis): Python Bytecode Instructions. + (line 14) +* Instruction.opname (in module dis): Python Bytecode Instructions. + (line 20) +* Instruction.starts_line (in module dis): Python Bytecode Instructions. + (line 40) +* int (built-in class): Built-in Functions. (line 801) +* int (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 131) +* Int2AP() (in module imaplib): imaplib — IMAP4 protocol client. + (line 113) +* integer: The standard type hierarchy. + (line 156) +* integer literal: Numeric literals. (line 6) +* integer; literals: Numeric Types — int float complex. + (line 19) +* integer; representation: The standard type hierarchy. + (line 101) +* Integral (class in numbers): The numeric tower. (line 55) +* Integrated Development Environment: IDLE<3>. (line 8) +* IntegrityError: Exceptions<4>. (line 19) +* Intel/DVI ADPCM: audioop — Manipulate raw audio data. + (line 17) +* IntEnum (class in enum): Module Contents<2>. (line 16) +* interact (pdb command): Debugger Commands. (line 275) +* interact() (code.InteractiveConsole method): Interactive Console Objects. + (line 10) +* interact() (in module code): code — Interpreter base classes. + (line 32) +* interact() (telnetlib.Telnet method): Telnet Objects. (line 113) +* interactive: Glossary. (line 631) +* interactive mode: Complete Python programs. + (line 19) +* InteractiveConsole (class in code): code — Interpreter base classes. + (line 25) +* InteractiveInterpreter (class in code): code — Interpreter base classes. + (line 15) +* intern (2to3 fixer): Fixers. (line 157) +* intern() (in module sys): sys — System-specific parameters and functions. + (line 967) +* internal type: The standard type hierarchy. + (line 645) +* Internaldate2tuple() (in module imaplib): imaplib — IMAP4 protocol client. + (line 107) +* internalSubset (xml.dom.DocumentType attribute): DocumentType Objects. + (line 27) +* internal_attr (zipfile.ZipInfo attribute): ZipInfo Objects. + (line 123) +* Internet: Internet Protocols and Support. + (line 6) +* interpolated string literal: String literal concatenation. + (line 24) +* interpolation, string (%): printf-style String Formatting. + (line 6) +* interpolation; bytearray (%): printf-style Bytes Formatting. + (line 6) +* interpolation; bytes (%): printf-style Bytes Formatting. + (line 6) +* InterpolationDepthError: Exceptions<5>. (line 42) +* InterpolationError: Exceptions<5>. (line 37) +* InterpolationMissingOptionError: Exceptions<5>. (line 49) +* InterpolationSyntaxError: Exceptions<5>. (line 54) +* interpreted: Glossary. (line 640) +* interpreter: Top-level components. + (line 6) +* interpreter lock: Thread State and the Global Interpreter Lock. + (line 6) +* interpreter prompts: sys — System-specific parameters and functions. + (line 1188) +* interpreter shutdown: Glossary. (line 650) +* interpreter_requires_environment() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 9) +* interrupt() (sqlite3.Connection method): Connection Objects. + (line 188) +* InterruptedError: OS exceptions. (line 71) +* interrupt_main() (in module _thread): _thread — Low-level threading API. + (line 53) +* intersection() (frozenset method): Set Types — set frozenset. + (line 102) +* intersection_update() (frozenset method): Set Types — set frozenset. + (line 175) +* IntFlag (class in enum): Module Contents<2>. (line 21) +* intro (cmd.Cmd attribute): Cmd Objects. (line 142) +* int_info (in module sys): sys — System-specific parameters and functions. + (line 933) +* InuseAttributeErr: Exceptions<16>. (line 39) +* inv() (in module operator): operator — Standard operators as functions. + (line 98) +* InvalidAccessErr: Exceptions<16>. (line 44) +* invalidate_caches() (importlib.abc.MetaPathFinder method): importlib abc – Abstract base classes related to import. + (line 86) +* invalidate_caches() (importlib.abc.PathEntryFinder method): importlib abc – Abstract base classes related to import. + (line 151) +* invalidate_caches() (importlib.machinery.FileFinder method): importlib machinery – Importers and path hooks. + (line 202) +* invalidate_caches() (importlib.machinery.PathFinder class method): importlib machinery – Importers and path hooks. + (line 146) +* invalidate_caches() (in module importlib): Functions<10>. (line 62) +* InvalidCharacterErr: Exceptions<16>. (line 49) +* InvalidModificationErr: Exceptions<16>. (line 57) +* InvalidOperation (class in decimal): Signals. (line 50) +* InvalidStateErr: Exceptions<16>. (line 61) +* InvalidStateError: Exception classes. (line 22) +* InvalidStateError <1>: Exceptions<9>. (line 28) +* InvalidURL: http client — HTTP protocol client. + (line 137) +* inversion: Unary arithmetic and bitwise operations. + (line 15) +* invert() (in module operator): operator — Standard operators as functions. + (line 98) +* invocation: The standard type hierarchy. + (line 292) +* inv_cdf() (statistics.NormalDist method): NormalDist objects. + (line 86) +* in_dll() (ctypes._CData method): Data types. (line 63) +* in_table_a1() (in module stringprep): stringprep — Internet String Preparation. + (line 40) +* in_table_b1() (in module stringprep): stringprep — Internet String Preparation. + (line 45) +* in_table_c11() (in module stringprep): stringprep — Internet String Preparation. + (line 60) +* in_table_c11_c12() (in module stringprep): stringprep — Internet String Preparation. + (line 69) +* in_table_c12() (in module stringprep): stringprep — Internet String Preparation. + (line 64) +* in_table_c21() (in module stringprep): stringprep — Internet String Preparation. + (line 74) +* in_table_c21_c22() (in module stringprep): stringprep — Internet String Preparation. + (line 84) +* in_table_c22() (in module stringprep): stringprep — Internet String Preparation. + (line 79) +* in_table_c3() (in module stringprep): stringprep — Internet String Preparation. + (line 89) +* in_table_c4() (in module stringprep): stringprep — Internet String Preparation. + (line 93) +* in_table_c5() (in module stringprep): stringprep — Internet String Preparation. + (line 98) +* in_table_c6() (in module stringprep): stringprep — Internet String Preparation. + (line 102) +* in_table_c7() (in module stringprep): stringprep — Internet String Preparation. + (line 107) +* in_table_c8() (in module stringprep): stringprep — Internet String Preparation. + (line 112) +* in_table_c9() (in module stringprep): stringprep — Internet String Preparation. + (line 117) +* in_table_d1() (in module stringprep): stringprep — Internet String Preparation. + (line 121) +* in_table_d2() (in module stringprep): stringprep — Internet String Preparation. + (line 126) +* in_transaction (sqlite3.Connection attribute): Connection Objects. + (line 18) +* IO (class in typing): Classes functions and decorators. + (line 471) +* io (module): io — Core tools for working with streams. + (line 6) +* IOBase (class in io): I/O Base Classes. (line 6) +* ioctl() (in module fcntl): fcntl — The fcntl and ioctl system calls. + (line 54) +* ioctl() (socket.socket method): Socket Objects. (line 168) +* IOCTL_VM_SOCKETS_GET_LOCAL_CID (in module socket): Constants<8>. + (line 194) +* IOError: Concrete exceptions. + (line 396) +* ior() (in module operator): In-place Operators. (line 79) +* IO_REPARSE_TAG_APPEXECLINK (in module stat): stat — Interpreting stat results. + (line 418) +* IO_REPARSE_TAG_MOUNT_POINT (in module stat): stat — Interpreting stat results. + (line 418) +* IO_REPARSE_TAG_SYMLINK (in module stat): stat — Interpreting stat results. + (line 418) +* ip (ipaddress.IPv4Interface attribute): Interface objects. (line 19) +* ip (ipaddress.IPv6Interface attribute): Interface objects. (line 74) +* ipaddress (module): ipaddress — IPv4/IPv6 manipulation library. + (line 6) +* ipow() (in module operator): In-place Operators. (line 84) +* IPv4Address (class in ipaddress): Address objects. (line 13) +* IPv4Interface (class in ipaddress): Interface objects. (line 9) +* IPv4Network (class in ipaddress): Network objects. (line 13) +* ipv4_mapped (ipaddress.IPv6Address attribute): Address objects. + (line 195) +* IPv6Address (class in ipaddress): Address objects. (line 124) +* IPv6Interface (class in ipaddress): Interface objects. (line 64) +* IPv6Network (class in ipaddress): Network objects. (line 250) +* IPV6_ENABLED (in module test.support): test support — Utilities for the Python test suite. + (line 83) +* ip_address() (in module ipaddress): Convenience factory functions. + (line 9) +* ip_interface() (in module ipaddress): Convenience factory functions. + (line 37) +* ip_network() (in module ipaddress): Convenience factory functions. + (line 23) +* irshift() (in module operator): In-place Operators. (line 89) +* isabs() (in module os.path): os path — Common pathname manipulations. + (line 204) +* isabstract() (in module inspect): Types and members. (line 419) +* IsADirectoryError: OS exceptions. (line 81) +* isalnum() (bytearray method): Bytes and Bytearray Operations. + (line 441) +* isalnum() (bytes method): Bytes and Bytearray Operations. + (line 441) +* isalnum() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 131) +* isalnum() (str method): String Methods<2>. (line 167) +* isalpha() (bytearray method): Bytes and Bytearray Operations. + (line 459) +* isalpha() (bytes method): Bytes and Bytearray Operations. + (line 459) +* isalpha() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 136) +* isalpha() (str method): String Methods<2>. (line 174) +* isascii() (bytearray method): Bytes and Bytearray Operations. + (line 474) +* isascii() (bytes method): Bytes and Bytearray Operations. + (line 474) +* isascii() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 141) +* isascii() (str method): String Methods<2>. (line 184) +* isasyncgen() (in module inspect): Types and members. (line 389) +* isasyncgenfunction() (in module inspect): Types and members. + (line 372) +* isatty() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 88) +* isatty() (in module os): File Descriptor Operations. + (line 209) +* isatty() (io.IOBase method): I/O Base Classes. (line 74) +* isawaitable() (in module inspect): Types and members. (line 353) +* isblank() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 145) +* isblk() (tarfile.TarInfo method): TarInfo Objects. (line 120) +* isbuiltin() (in module inspect): Types and members. (line 409) +* ischr() (tarfile.TarInfo method): TarInfo Objects. (line 116) +* isclass() (in module inspect): Types and members. (line 309) +* isclose() (in module cmath): Classification functions. + (line 23) +* isclose() (in module math): Number-theoretic and representation functions. + (line 104) +* iscntrl() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 149) +* iscode() (in module inspect): Types and members. (line 405) +* iscoroutine() (in module asyncio): Generator-based Coroutines. + (line 36) +* iscoroutine() (in module inspect): Types and members. (line 346) +* iscoroutinefunction() (in module asyncio): Generator-based Coroutines. + (line 43) +* iscoroutinefunction() (in module inspect): Types and members. + (line 335) +* isctrl() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 190) +* isDaemon() (threading.Thread method): Thread Objects. (line 197) +* isdatadescriptor() (in module inspect): Types and members. (line 441) +* isdecimal() (str method): String Methods<2>. (line 192) +* isdev() (tarfile.TarInfo method): TarInfo Objects. (line 128) +* isdigit() (bytearray method): Bytes and Bytearray Operations. + (line 482) +* isdigit() (bytes method): Bytes and Bytearray Operations. + (line 482) +* isdigit() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 154) +* isdigit() (str method): String Methods<2>. (line 200) +* isdir() (in module os.path): os path — Common pathname manipulations. + (line 220) +* isdir() (tarfile.TarInfo method): TarInfo Objects. (line 104) +* isdisjoint() (frozenset method): Set Types — set frozenset. + (line 68) +* isdown() (in module turtle): Drawing state. (line 86) +* iselement() (in module xml.etree.ElementTree): Functions<6>. + (line 118) +* isenabled() (in module gc): gc — Garbage Collector interface. + (line 30) +* isEnabledFor() (logging.Logger method): Logger Objects. (line 89) +* isendwin() (in module curses): Functions<3>. (line 242) +* ISEOF() (in module token): token — Constants used with Python parse trees. + (line 34) +* isexpr() (in module parser): Queries on ST Objects. + (line 12) +* isexpr() (parser.ST method): ST Objects. (line 21) +* isfifo() (tarfile.TarInfo method): TarInfo Objects. (line 124) +* isfile() (in module os.path): os path — Common pathname manipulations. + (line 212) +* isfile() (tarfile.TarInfo method): TarInfo Objects. (line 96) +* isfinite() (in module cmath): Classification functions. + (line 6) +* isfinite() (in module math): Number-theoretic and representation functions. + (line 137) +* isfirstline() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 106) +* isframe() (in module inspect): Types and members. (line 401) +* isfunction() (in module inspect): Types and members. (line 318) +* isfuture() (in module asyncio): Future Functions. (line 6) +* isgenerator() (in module inspect): Types and members. (line 331) +* isgeneratorfunction() (in module inspect): Types and members. + (line 323) +* isgetsetdescriptor() (in module inspect): Types and members. + (line 454) +* isgraph() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 159) +* isidentifier() (str method): String Methods<2>. (line 210) +* isinf() (in module cmath): Classification functions. + (line 13) +* isinf() (in module math): Number-theoretic and representation functions. + (line 144) +* isinstance (2to3 fixer): Fixers. (line 161) +* isinstance() (built-in function): Built-in Functions. (line 843) +* iskeyword() (in module keyword): keyword — Testing for Python keywords. + (line 13) +* isleap() (in module calendar): calendar — General calendar-related functions. + (line 301) +* islice() (in module itertools): Itertool functions. (line 355) +* islink() (in module os.path): os path — Common pathname manipulations. + (line 228) +* islnk() (tarfile.TarInfo method): TarInfo Objects. (line 112) +* islower() (bytearray method): Bytes and Bytearray Operations. + (line 496) +* islower() (bytes method): Bytes and Bytearray Operations. + (line 496) +* islower() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 163) +* islower() (str method): String Methods<2>. (line 228) +* ismemberdescriptor() (in module inspect): Types and members. + (line 463) +* ismeta() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 194) +* ismethod() (in module inspect): Types and members. (line 314) +* ismethoddescriptor() (in module inspect): Types and members. + (line 423) +* ismodule() (in module inspect): Types and members. (line 305) +* ismount() (in module os.path): os path — Common pathname manipulations. + (line 236) +* isnan() (in module cmath): Classification functions. + (line 18) +* isnan() (in module math): Number-theoretic and representation functions. + (line 149) +* ISNONTERMINAL() (in module token): token — Constants used with Python parse trees. + (line 30) +* isnumeric() (str method): String Methods<2>. (line 234) +* isocalendar() (datetime.date method): date Objects. (line 213) +* isocalendar() (datetime.datetime method): datetime Objects. + (line 594) +* isoformat() (datetime.date method): date Objects. (line 236) +* isoformat() (datetime.datetime method): datetime Objects. (line 599) +* isoformat() (datetime.time method): time Objects. (line 151) +* IsolatedAsyncioTestCase (class in unittest): Test cases. (line 809) +* isolation_level (sqlite3.Connection attribute): Connection Objects. + (line 11) +* isoweekday() (datetime.date method): date Objects. (line 206) +* isoweekday() (datetime.datetime method): datetime Objects. (line 588) +* isprint() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 167) +* isprintable() (str method): String Methods<2>. (line 244) +* ispunct() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 171) +* isqrt() (in module math): Number-theoretic and representation functions. + (line 154) +* isreadable() (in module pprint): pprint — Data pretty printer. + (line 130) +* isreadable() (pprint.PrettyPrinter method): PrettyPrinter Objects. + (line 24) +* isrecursive() (in module pprint): pprint — Data pretty printer. + (line 139) +* isrecursive() (pprint.PrettyPrinter method): PrettyPrinter Objects. + (line 32) +* isreg() (tarfile.TarInfo method): TarInfo Objects. (line 100) +* isReservedKey() (http.cookies.Morsel method): Morsel Objects. + (line 68) +* isroutine() (in module inspect): Types and members. (line 414) +* isSameNode() (xml.dom.Node method): Node Objects. (line 102) +* isspace() (bytearray method): Bytes and Bytearray Operations. + (line 513) +* isspace() (bytes method): Bytes and Bytearray Operations. + (line 513) +* isspace() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 176) +* isspace() (str method): String Methods<2>. (line 256) +* isstdin() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 111) +* issubclass() (built-in function): Built-in Functions. (line 854) +* issubset() (frozenset method): Set Types — set frozenset. + (line 74) +* issuite() (in module parser): Queries on ST Objects. + (line 22) +* issuite() (parser.ST method): ST Objects. (line 25) +* issuperset() (frozenset method): Set Types — set frozenset. + (line 85) +* issym() (tarfile.TarInfo method): TarInfo Objects. (line 108) +* ISTERMINAL() (in module token): token — Constants used with Python parse trees. + (line 26) +* istitle() (bytearray method): Bytes and Bytearray Operations. + (line 521) +* istitle() (bytes method): Bytes and Bytearray Operations. + (line 521) +* istitle() (str method): String Methods<2>. (line 266) +* istraceback() (in module inspect): Types and members. (line 397) +* isub() (in module operator): In-place Operators. (line 94) +* isupper() (bytearray method): Bytes and Bytearray Operations. + (line 535) +* isupper() (bytes method): Bytes and Bytearray Operations. + (line 535) +* isupper() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 181) +* isupper() (str method): String Methods<2>. (line 273) +* isvisible() (in module turtle): Visibility. (line 22) +* isxdigit() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 185) +* is_() (in module operator): operator — Standard operators as functions. + (line 63) +* is_absolute() (pathlib.PurePath method): Methods and properties. + (line 165) +* is_active() (asyncio.AbstractChildWatcher method): Process Watchers. + (line 68) +* is_alive() (multiprocessing.Process method): Process and exceptions. + (line 79) +* is_alive() (threading.Thread method): Thread Objects. (line 175) +* is_android (in module test.support): test support — Utilities for the Python test suite. + (line 40) +* is_annotated() (symtable.Symbol method): Examining Symbol Tables. + (line 142) +* is_assigned() (symtable.Symbol method): Examining Symbol Tables. + (line 153) +* is_attachment() (email.message.EmailMessage method): email message Representing an email message. + (line 445) +* is_authenticated() (urllib.request.HTTPPasswordMgrWithPriorAuth method): HTTPPasswordMgrWithPriorAuth Objects. + (line 30) +* is_blocked() (http.cookiejar.DefaultCookiePolicy method): DefaultCookiePolicy Objects. + (line 60) +* is_block_device() (pathlib.Path method): Methods<2>. (line 169) +* is_canonical() (decimal.Context method): Context objects. (line 292) +* is_canonical() (decimal.Decimal method): Decimal objects. (line 278) +* IS_CHARACTER_JUNK() (in module difflib): difflib — Helpers for computing deltas. + (line 364) +* is_char_device() (pathlib.Path method): Methods<2>. (line 178) +* is_check_supported() (in module lzma): Miscellaneous<2>. (line 6) +* is_closed() (asyncio.loop method): Running and stopping the loop. + (line 39) +* is_closing() (asyncio.BaseTransport method): Base Transport. + (line 16) +* is_closing() (asyncio.StreamWriter method): StreamWriter. (line 81) +* is_dataclass() (in module dataclasses): Module-level decorators classes and functions. + (line 373) +* is_declared_global() (symtable.Symbol method): Examining Symbol Tables. + (line 133) +* is_dir() (os.DirEntry method): Files and Directories. + (line 883) +* is_dir() (pathlib.Path method): Methods<2>. (line 114) +* is_dir() (zipfile.Path method): Path Objects. (line 39) +* is_dir() (zipfile.ZipInfo method): ZipInfo Objects. (line 42) +* is_enabled() (in module faulthandler): Fault handler state. + (line 27) +* is_expired() (http.cookiejar.Cookie method): Cookie Objects<2>. + (line 109) +* is_fifo() (pathlib.Path method): Methods<2>. (line 161) +* is_file() (os.DirEntry method): Files and Directories. + (line 912) +* is_file() (pathlib.Path method): Methods<2>. (line 123) +* is_file() (zipfile.Path method): Path Objects. (line 43) +* is_finalizing() (in module sys): sys — System-specific parameters and functions. + (line 982) +* is_finite() (decimal.Context method): Context objects. (line 296) +* is_finite() (decimal.Decimal method): Decimal objects. (line 285) +* is_free() (symtable.Symbol method): Examining Symbol Tables. + (line 148) +* is_global (ipaddress.IPv4Address attribute): Address objects. + (line 97) +* is_global (ipaddress.IPv6Address attribute): Address objects. + (line 175) +* is_global() (symtable.Symbol method): Examining Symbol Tables. + (line 125) +* is_hop_by_hop() (in module wsgiref.util): wsgiref util – WSGI environment utilities. + (line 109) +* is_imported() (symtable.Symbol method): Examining Symbol Tables. + (line 116) +* is_infinite() (decimal.Context method): Context objects. (line 300) +* is_infinite() (decimal.Decimal method): Decimal objects. (line 290) +* is_integer() (float method): Additional Methods on Float. + (line 16) +* is_jython (in module test.support): test support — Utilities for the Python test suite. + (line 36) +* is_linetouched() (curses.window method): Window Objects. (line 365) +* IS_LINE_JUNK() (in module difflib): difflib — Helpers for computing deltas. + (line 357) +* is_link_local (ipaddress.IPv4Address attribute): Address objects. + (line 119) +* is_link_local (ipaddress.IPv4Network attribute): Network objects. + (line 80) +* is_link_local (ipaddress.IPv6Address attribute): Address objects. + (line 183) +* is_link_local (ipaddress.IPv6Network attribute): Network objects. + (line 304) +* is_local() (symtable.Symbol method): Examining Symbol Tables. + (line 138) +* is_loopback (ipaddress.IPv4Address attribute): Address objects. + (line 114) +* is_loopback (ipaddress.IPv4Network attribute): Network objects. + (line 78) +* is_loopback (ipaddress.IPv6Address attribute): Address objects. + (line 181) +* is_loopback (ipaddress.IPv6Network attribute): Network objects. + (line 302) +* is_mount() (pathlib.Path method): Methods<2>. (line 132) +* is_multicast (ipaddress.IPv4Address attribute): Address objects. + (line 86) +* is_multicast (ipaddress.IPv4Network attribute): Network objects. + (line 70) +* is_multicast (ipaddress.IPv6Address attribute): Address objects. + (line 171) +* is_multicast (ipaddress.IPv6Network attribute): Network objects. + (line 294) +* is_multipart() (email.message.EmailMessage method): email message Representing an email message. + (line 134) +* is_multipart() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 145) +* is_namespace() (symtable.Symbol method): Examining Symbol Tables. + (line 157) +* is_nan() (decimal.Context method): Context objects. (line 304) +* is_nan() (decimal.Decimal method): Decimal objects. (line 295) +* is_nested() (symtable.SymbolTable method): Examining Symbol Tables. + (line 35) +* is_nonlocal() (symtable.Symbol method): Examining Symbol Tables. + (line 129) +* is_normal() (decimal.Context method): Context objects. (line 309) +* is_normal() (decimal.Decimal method): Decimal objects. (line 300) +* is_normalized() (in module unicodedata): unicodedata — Unicode Database. + (line 117) +* is_not() (in module operator): operator — Standard operators as functions. + (line 67) +* is_not_allowed() (http.cookiejar.DefaultCookiePolicy method): DefaultCookiePolicy Objects. + (line 74) +* is_optimized() (symtable.SymbolTable method): Examining Symbol Tables. + (line 31) +* is_package() (importlib.abc.InspectLoader method): importlib abc – Abstract base classes related to import. + (line 413) +* is_package() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 604) +* is_package() (importlib.machinery.ExtensionFileLoader method): importlib machinery – Importers and path hooks. + (line 333) +* is_package() (importlib.machinery.SourceFileLoader method): importlib machinery – Importers and path hooks. + (line 232) +* is_package() (importlib.machinery.SourcelessFileLoader method): importlib machinery – Importers and path hooks. + (line 276) +* is_package() (zipimport.zipimporter method): zipimporter Objects. + (line 56) +* is_parameter() (symtable.Symbol method): Examining Symbol Tables. + (line 121) +* is_private (ipaddress.IPv4Address attribute): Address objects. + (line 91) +* is_private (ipaddress.IPv4Network attribute): Network objects. + (line 72) +* is_private (ipaddress.IPv6Address attribute): Address objects. + (line 173) +* is_private (ipaddress.IPv6Network attribute): Network objects. + (line 296) +* is_python_build() (in module sysconfig): Other functions<3>. + (line 50) +* is_qnan() (decimal.Context method): Context objects. (line 314) +* is_qnan() (decimal.Decimal method): Decimal objects. (line 306) +* is_reading() (asyncio.ReadTransport method): Read-only Transports. + (line 6) +* is_referenced() (symtable.Symbol method): Examining Symbol Tables. + (line 112) +* is_reserved (ipaddress.IPv4Address attribute): Address objects. + (line 110) +* is_reserved (ipaddress.IPv4Network attribute): Network objects. + (line 76) +* is_reserved (ipaddress.IPv6Address attribute): Address objects. + (line 179) +* is_reserved (ipaddress.IPv6Network attribute): Network objects. + (line 300) +* is_reserved() (pathlib.PurePath method): Methods and properties. + (line 184) +* is_resource() (importlib.abc.ResourceReader method): importlib abc – Abstract base classes related to import. + (line 334) +* is_resource() (in module importlib.resources): importlib resources – Resources. + (line 120) +* is_resource_enabled() (in module test.support): test support — Utilities for the Python test suite. + (line 197) +* is_running() (asyncio.loop method): Running and stopping the loop. + (line 35) +* is_safe (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 151) +* is_serving() (asyncio.Server method): Server Objects. (line 89) +* is_set() (asyncio.Event method): Event. (line 66) +* is_set() (threading.Event method): Event Objects. (line 23) +* is_signed() (decimal.Context method): Context objects. (line 319) +* is_signed() (decimal.Decimal method): Decimal objects. (line 311) +* is_site_local (ipaddress.IPv6Address attribute): Address objects. + (line 187) +* is_site_local (ipaddress.IPv6Network attribute): Network objects. + (line 353) +* is_snan() (decimal.Context method): Context objects. (line 323) +* is_snan() (decimal.Decimal method): Decimal objects. (line 317) +* is_socket() (pathlib.Path method): Methods<2>. (line 152) +* is_subnormal() (decimal.Context method): Context objects. (line 328) +* is_subnormal() (decimal.Decimal method): Decimal objects. (line 322) +* is_symlink() (os.DirEntry method): Files and Directories. + (line 928) +* is_symlink() (pathlib.Path method): Methods<2>. (line 144) +* is_tarfile() (in module tarfile): tarfile — Read and write tar archive files. + (line 175) +* is_term_resized() (in module curses): Functions<3>. (line 237) +* is_tracing() (in module tracemalloc): Functions<9>. (line 43) +* is_tracked() (in module gc): gc — Garbage Collector interface. + (line 157) +* is_unspecified (ipaddress.IPv4Address attribute): Address objects. + (line 105) +* is_unspecified (ipaddress.IPv4Network attribute): Network objects. + (line 74) +* is_unspecified (ipaddress.IPv6Address attribute): Address objects. + (line 177) +* is_unspecified (ipaddress.IPv6Network attribute): Network objects. + (line 298) +* is_wintouched() (curses.window method): Window Objects. (line 372) +* is_zero() (decimal.Context method): Context objects. (line 332) +* is_zero() (decimal.Decimal method): Decimal objects. (line 327) +* is_zipfile() (in module zipfile): zipfile — Work with ZIP archives. + (line 71) +* item selection: The standard type hierarchy. + (line 127) +* item() (tkinter.ttk.Treeview method): ttk Treeview. (line 212) +* item() (xml.dom.NamedNodeMap method): NamedNodeMap Objects. + (line 12) +* item() (xml.dom.NodeList method): NodeList Objects. (line 15) +* itemgetter() (in module operator): operator — Standard operators as functions. + (line 258) +* items() (configparser.ConfigParser method): ConfigParser Objects. + (line 235) +* items() (contextvars.Context method): Manual Context Management. + (line 108) +* items() (dict method): Mapping Types — dict. + (line 168) +* items() (email.message.EmailMessage method): email message Representing an email message. + (line 240) +* items() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 348) +* items() (mailbox.Mailbox method): Mailbox objects. (line 129) +* items() (types.MappingProxyType method): Standard Interpreter Types. + (line 256) +* items() (xml.etree.ElementTree.Element method): Element Objects. + (line 71) +* itemsize (array.array attribute): array — Efficient arrays of numeric values. + (line 108) +* itemsize (memoryview attribute): Memory Views. (line 428) +* ItemsView (class in collections.abc): Collections Abstract Base Classes. + (line 187) +* ItemsView (class in typing): Classes functions and decorators. + (line 290) +* iter() (built-in function): Built-in Functions. (line 862) +* iter() (xml.etree.ElementTree.Element method): Element Objects. + (line 146) +* iter() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 45) +* iterable: Glossary. (line 665) +* Iterable (class in collections.abc): Collections Abstract Base Classes. + (line 120) +* Iterable (class in typing): Classes functions and decorators. + (line 147) +* iterable; unpacking: Expression lists. (line 16) +* iterator: Glossary. (line 687) +* Iterator (class in collections.abc): Collections Abstract Base Classes. + (line 137) +* Iterator (class in typing): Classes functions and decorators. + (line 151) +* iterator protocol: Iterator Types. (line 6) +* iterdecode() (in module codecs): codecs — Codec registry and base classes. + (line 225) +* iterdir() (pathlib.Path method): Methods<2>. (line 187) +* iterdir() (zipfile.Path method): Path Objects. (line 35) +* iterdump() (sqlite3.Connection method): Connection Objects. + (line 385) +* iterencode() (in module codecs): codecs — Codec registry and base classes. + (line 213) +* iterencode() (json.JSONEncoder method): Encoders and Decoders. + (line 224) +* iterfind() (xml.etree.ElementTree.Element method): Element Objects. + (line 157) +* iterfind() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 52) +* iteritems() (mailbox.Mailbox method): Mailbox objects. (line 129) +* iterkeys() (mailbox.Mailbox method): Mailbox objects. (line 107) +* itermonthdates() (calendar.Calendar method): calendar — General calendar-related functions. + (line 45) +* itermonthdays() (calendar.Calendar method): calendar — General calendar-related functions. + (line 53) +* itermonthdays2() (calendar.Calendar method): calendar — General calendar-related functions. + (line 61) +* itermonthdays3() (calendar.Calendar method): calendar — General calendar-related functions. + (line 69) +* itermonthdays4() (calendar.Calendar method): calendar — General calendar-related functions. + (line 79) +* iternextfunc (C type): Slot Type typedefs. (line 99) +* iterparse() (in module xml.etree.ElementTree): Functions<6>. + (line 124) +* itertext() (xml.etree.ElementTree.Element method): Element Objects. + (line 166) +* itertools (2to3 fixer): Fixers. (line 174) +* itertools (module): itertools — Functions creating iterators for efficient looping. + (line 6) +* itertools_imports (2to3 fixer): Fixers. (line 168) +* itervalues() (mailbox.Mailbox method): Mailbox objects. (line 114) +* iterweekdays() (calendar.Calendar method): calendar — General calendar-related functions. + (line 39) +* iter_attachments() (email.message.EmailMessage method): email message Representing an email message. + (line 560) +* iter_child_nodes() (in module ast): ast Helpers. (line 107) +* iter_fields() (in module ast): ast Helpers. (line 102) +* iter_importers() (in module pkgutil): pkgutil — Package extension utility. + (line 123) +* iter_modules() (in module pkgutil): pkgutil — Package extension utility. + (line 141) +* iter_parts() (email.message.EmailMessage method): email message Representing an email message. + (line 576) +* iter_unpack() (in module struct): Functions and Exceptions. + (line 42) +* iter_unpack() (struct.Struct method): Classes<2>. (line 47) +* ItimerError: Module contents<2>. (line 211) +* ITIMER_PROF (in module signal): Module contents<2>. (line 178) +* ITIMER_REAL (in module signal): Module contents<2>. (line 168) +* ITIMER_VIRTUAL (in module signal): Module contents<2>. (line 173) +* itruediv() (in module operator): In-place Operators. (line 99) +* ixor() (in module operator): In-place Operators. (line 104) +* j; in numeric literal: Floating point literals. + (line 28) +* Jansen, Jack: uu — Encode and decode uuencode files. + (line 20) +* Java; language: The standard type hierarchy. + (line 107) +* java_ver() (in module platform): Java Platform. (line 6) +* join() (asyncio.Queue method): Queue. (line 50) +* join() (bytearray method): Bytes and Bytearray Operations. + (line 111) +* join() (bytes method): Bytes and Bytearray Operations. + (line 111) +* join() (in module os.path): os path — Common pathname manipulations. + (line 254) +* join() (in module shlex): shlex — Simple lexical analysis. + (line 30) +* join() (multiprocessing.JoinableQueue method): Pipes and Queues. + (line 242) +* join() (multiprocessing.pool.Pool method): Process Pools. (line 180) +* join() (multiprocessing.Process method): Process and exceptions. + (line 51) +* join() (queue.Queue method): Queue Objects. (line 83) +* join() (str method): String Methods<2>. (line 279) +* join() (threading.Thread method): Thread Objects. (line 110) +* JoinableQueue (class in multiprocessing): Pipes and Queues. + (line 221) +* joinpath() (pathlib.PurePath method): Methods and properties. + (line 198) +* join_thread() (in module test.support): test support — Utilities for the Python test suite. + (line 787) +* join_thread() (multiprocessing.Queue method): Pipes and Queues. + (line 170) +* json (module): json — JSON encoder and decoder. + (line 6) +* json.tool (module): Command Line Interface<2>. + (line 6) +* json.tool command line option; -h: Command line options<2>. + (line 41) +* json.tool command line option; –help: Command line options<2>. + (line 41) +* json.tool command line option; –json-lines: Command line options<2>. + (line 35) +* json.tool command line option; –sort-keys: Command line options<2>. + (line 29) +* json.tool command line option; infile: Command line options<2>. + (line 6) +* json.tool command line option; outfile: Command line options<2>. + (line 24) +* JSONDecodeError: Exceptions<13>. (line 6) +* JSONDecoder (class in json): Encoders and Decoders. + (line 6) +* JSONEncoder (class in json): Encoders and Decoders. + (line 102) +* js_output() (http.cookies.BaseCookie method): Cookie Objects. + (line 32) +* js_output() (http.cookies.Morsel method): Morsel Objects. (line 79) +* jump (pdb command): Debugger Commands. (line 196) +* JUMP_ABSOLUTE (opcode): Python Bytecode Instructions. + (line 682) +* JUMP_FORWARD (opcode): Python Bytecode Instructions. + (line 650) +* JUMP_IF_FALSE_OR_POP (opcode): Python Bytecode Instructions. + (line 675) +* JUMP_IF_TRUE_OR_POP (opcode): Python Bytecode Instructions. + (line 668) +* kbhit() (in module msvcrt): Console I/O. (line 6) +* KDEDIR: webbrowser — Convenient Web-browser controller. + (line 184) +* kevent() (in module select): select — Waiting for I/O completion. + (line 103) +* key: Dictionary displays. + (line 6) +* key (http.cookies.Morsel attribute): Morsel Objects. (line 60) +* key function: Glossary. (line 709) +* key/datum pair: Dictionary displays. + (line 6) +* KeyboardInterrupt: Concrete exceptions. + (line 70) +* KeyboardInterrupt (built-in exception): Signal Handling<2>. + (line 8) +* KeyboardInterrupt (built-in exception) <1>: Signal Handling<2>. + (line 20) +* KeyError: Concrete exceptions. + (line 65) +* keylog_filename (ssl.SSLContext attribute): SSL Contexts. (line 529) +* keyname() (in module curses): Functions<3>. (line 247) +* keypad() (curses.window method): Window Objects. (line 377) +* keyrefs() (weakref.WeakKeyDictionary method): weakref — Weak references. + (line 174) +* keys() (contextvars.Context method): Manual Context Management. + (line 100) +* keys() (dict method): Mapping Types — dict. + (line 173) +* keys() (email.message.EmailMessage method): email message Representing an email message. + (line 232) +* keys() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 340) +* keys() (mailbox.Mailbox method): Mailbox objects. (line 107) +* keys() (sqlite3.Row method): Row Objects. (line 18) +* keys() (types.MappingProxyType method): Standard Interpreter Types. + (line 261) +* keys() (xml.etree.ElementTree.Element method): Element Objects. + (line 76) +* KeysView (class in collections.abc): Collections Abstract Base Classes. + (line 187) +* KeysView (class in typing): Classes functions and decorators. + (line 286) +* keyword: Keywords. (line 6) +* keyword (module): keyword — Testing for Python keywords. + (line 6) +* keyword argument: Glossary. (line 731) +* keyword; as: The import statement. + (line 6) +* keyword; as <1>: The try statement. (line 6) +* keyword; as <2>: The with statement. (line 6) +* keyword; async: Coroutine function definition. + (line 9) +* keyword; await: Calls. (line 149) +* keyword; await <1>: Coroutine function definition. + (line 9) +* keyword; elif: The if statement. (line 6) +* keyword; else: The break statement. + (line 12) +* keyword; else <1>: The if statement. (line 6) +* keyword; else <2>: The while statement. + (line 6) +* keyword; else <3>: The for statement. (line 6) +* keyword; else <4>: The try statement. (line 6) +* keyword; else <5>: The try statement. (line 79) +* keyword; except: The try statement. (line 6) +* keyword; finally: The return statement. + (line 17) +* keyword; finally <1>: The break statement. + (line 18) +* keyword; finally <2>: The continue statement. + (line 6) +* keyword; finally <3>: The try statement. (line 6) +* keyword; finally <4>: The try statement. (line 85) +* keyword; from: Yield expressions. (line 6) +* keyword; from <1>: The import statement. + (line 6) +* keyword; in: The for statement. (line 6) +* keyword; yield: Yield expressions. (line 6) +* keywords (functools.partial attribute): partial Objects. (line 20) +* KEY_ALL_ACCESS (in module winreg): Access Rights. (line 8) +* KEY_CREATE_LINK (in module winreg): Access Rights. (line 51) +* KEY_CREATE_SUB_KEY (in module winreg): Access Rights. (line 38) +* KEY_ENUMERATE_SUB_KEYS (in module winreg): Access Rights. (line 42) +* KEY_EXECUTE (in module winreg): Access Rights. (line 26) +* KEY_NOTIFY (in module winreg): Access Rights. (line 46) +* KEY_QUERY_VALUE (in module winreg): Access Rights. (line 30) +* KEY_READ (in module winreg): Access Rights. (line 20) +* KEY_SET_VALUE (in module winreg): Access Rights. (line 34) +* KEY_WOW64_32KEY (in module winreg): 64-bit Specific. (line 13) +* KEY_WOW64_64KEY (in module winreg): 64-bit Specific. (line 8) +* KEY_WRITE (in module winreg): Access Rights. (line 15) +* kill() (asyncio.asyncio.subprocess.Process method): Interacting with Subprocesses. + (line 100) +* kill() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 35) +* kill() (in module os): Process Management. (line 287) +* kill() (multiprocessing.Process method): Process and exceptions. + (line 166) +* kill() (subprocess.Popen method): Popen Objects. (line 89) +* killchar() (in module curses): Functions<3>. (line 257) +* killpg() (in module os): Process Management. (line 309) +* kill_python() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 67) +* kind (inspect.Parameter attribute): Introspecting callables with the Signature object. + (line 181) +* knownfiles (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 122) +* kqueue() (in module select): select — Waiting for I/O completion. + (line 92) +* KqueueSelector (class in selectors): Classes<3>. (line 195) +* kwargs (inspect.BoundArguments attribute): Introspecting callables with the Signature object. + (line 303) +* kwlist (in module keyword): keyword — Testing for Python keywords. + (line 17) +* L (in module re): Module Contents. (line 81) +* LabelEntry (class in tkinter.tix): Basic Widgets. (line 31) +* LabelFrame (class in tkinter.tix): Basic Widgets. (line 37) +* lambda: Glossary. (line 735) +* lambda; expression: Lambdas. (line 6) +* lambda; expression <1>: Function definitions. + (line 104) +* lambda; form: Lambdas. (line 6) +* LambdaType (in module types): Standard Interpreter Types. + (line 19) +* LANG: GNU gettext API. (line 21) +* LANG <1>: Class-based API. (line 26) +* LANG <2>: locale — Internationalization services. + (line 49) +* LANG <3>: locale — Internationalization services. + (line 310) +* LANG <4>: locale — Internationalization services. + (line 314) +* LANGUAGE: GNU gettext API. (line 20) +* LANGUAGE <1>: Class-based API. (line 25) +* large files: Large File Support. (line 6) +* LARGEST (in module test.support): test support — Utilities for the Python test suite. + (line 150) +* LargeZipFile: zipfile — Work with ZIP archives. + (line 37) +* last() (nntplib.NNTP method): Methods<3>. (line 237) +* lastChild (xml.dom.Node attribute): Node Objects. (line 59) +* lastcmd (cmd.Cmd attribute): Cmd Objects. (line 131) +* lastgroup (re.Match attribute): Match Objects. (line 186) +* lastindex (re.Match attribute): Match Objects. (line 178) +* lastResort (in module logging): Module-Level Attributes. + (line 6) +* lastrowid (sqlite3.Cursor attribute): Cursor Objects. (line 199) +* last_accepted (multiprocessing.connection.Listener attribute): Listeners and Clients. + (line 103) +* last_traceback (in module sys): The standard type hierarchy. + (line 757) +* last_traceback (in module sys) <1>: sys — System-specific parameters and functions. + (line 989) +* last_type (in module sys): sys — System-specific parameters and functions. + (line 989) +* last_value (in module sys): sys — System-specific parameters and functions. + (line 989) +* layout() (tkinter.ttk.Style method): Ttk Styling. (line 96) +* lazycache() (in module linecache): linecache — Random access to text lines. + (line 50) +* LazyLoader (class in importlib.util): importlib util – Utility code for importers. + (line 225) +* LBRACE (in module token): token — Constants used with Python parse trees. + (line 126) +* LBYL: Glossary. (line 742) +* lchflags() (in module os): Files and Directories. + (line 292) +* lchmod() (in module os): Files and Directories. + (line 306) +* lchmod() (pathlib.Path method): Methods<2>. (line 208) +* lchown() (in module os): Files and Directories. + (line 321) +* LC_ALL: GNU gettext API. (line 20) +* LC_ALL <1>: Class-based API. (line 25) +* LC_ALL (in module locale): locale — Internationalization services. + (line 495) +* LC_COLLATE (in module locale): locale — Internationalization services. + (line 464) +* LC_CTYPE (in module locale): locale — Internationalization services. + (line 458) +* LC_MESSAGES: GNU gettext API. (line 20) +* LC_MESSAGES <1>: Class-based API. (line 26) +* LC_MESSAGES (in module locale): locale — Internationalization services. + (line 480) +* LC_MONETARY (in module locale): locale — Internationalization services. + (line 475) +* LC_NUMERIC (in module locale): locale — Internationalization services. + (line 487) +* LC_TIME (in module locale): locale — Internationalization services. + (line 470) +* LDCXXSHARED: Build and C API Changes<8>. + (line 153) +* ldexp() (in module math): Number-theoretic and representation functions. + (line 167) +* LDFLAGS: New Improved and Deprecated Modules<4>. + (line 59) +* ldgettext() (in module gettext): GNU gettext API. (line 90) +* ldngettext() (in module gettext): GNU gettext API. (line 94) +* le() (in module operator): operator — Standard operators as functions. + (line 24) +* leading whitespace: Indentation. (line 6) +* leapdays() (in module calendar): calendar — General calendar-related functions. + (line 306) +* leaveok() (curses.window method): Window Objects. (line 384) +* left (filecmp.dircmp attribute): The dircmp class. (line 41) +* left() (in module turtle): Turtle motion. (line 57) +* LEFTSHIFT (in module token): token — Constants used with Python parse trees. + (line 158) +* LEFTSHIFTEQUAL (in module token): token — Constants used with Python parse trees. + (line 202) +* left_list (filecmp.dircmp attribute): The dircmp class. (line 49) +* left_only (filecmp.dircmp attribute): The dircmp class. (line 63) +* len() (built-in function): Built-in Functions. (line 889) +* lenfunc (C type): Slot Type typedefs. (line 103) +* length (xml.dom.NamedNodeMap attribute): NamedNodeMap Objects. + (line 8) +* length (xml.dom.NodeList attribute): NodeList Objects. (line 21) +* length_hint() (in module operator): operator — Standard operators as functions. + (line 206) +* LESS (in module token): token — Constants used with Python parse trees. + (line 106) +* LESSEQUAL (in module token): token — Constants used with Python parse trees. + (line 142) +* lexical analysis: Lexical analysis. (line 6) +* lexical definitions: Notation. (line 31) +* lexists() (in module os.path): os path — Common pathname manipulations. + (line 127) +* lgamma() (in module math): Special functions. (line 35) +* lgettext() (gettext.GNUTranslations method): The GNUTranslations class. + (line 93) +* lgettext() (gettext.NullTranslations method): The NullTranslations class. + (line 63) +* lgettext() (in module gettext): GNU gettext API. (line 88) +* lib2to3 (module): lib2to3 - 2to3’s library. + (line 6) +* libc_ver() (in module platform): Unix Platforms. (line 6) +* library (in module dbm.ndbm): dbm ndbm — Interface based on ndbm. + (line 27) +* library (ssl.SSLError attribute): Exceptions<12>. (line 19) +* LibraryLoader (class in ctypes): Loading shared libraries. + (line 158) +* library_dir_option() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 220) +* library_filename() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 459) +* library_option() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 225) +* license (built-in variable): Constants added by the site module. + (line 24) +* LifoQueue (class in asyncio): LIFO Queue. (line 6) +* LifoQueue (class in queue): queue — A synchronized queue class. + (line 42) +* light-weight processes: _thread — Low-level threading API. + (line 6) +* LimitOverrunError: Exceptions<9>. (line 59) +* limit_denominator() (fractions.Fraction method): fractions — Rational numbers. + (line 126) +* lin2adpcm() (in module audioop): audioop — Manipulate raw audio data. + (line 112) +* lin2alaw() (in module audioop): audioop — Manipulate raw audio data. + (line 126) +* lin2lin() (in module audioop): audioop — Manipulate raw audio data. + (line 133) +* lin2ulaw() (in module audioop): audioop — Manipulate raw audio data. + (line 148) +* line continuation: Explicit line joining. + (line 6) +* line joining: Logical lines. (line 6) +* line joining <1>: Explicit line joining. + (line 6) +* line structure: Line structure. (line 6) +* line() (msilib.Dialog method): GUI classes. (line 64) +* line-buffered I/O: Built-in Functions. (line 1217) +* linecache (module): linecache — Random access to text lines. + (line 6) +* lineno (ast.AST attribute): Node classes. (line 39) +* lineno (doctest.DocTest attribute): DocTest Objects. (line 42) +* lineno (doctest.Example attribute): Example Objects. (line 41) +* lineno (json.JSONDecodeError attribute): Exceptions<13>. (line 23) +* lineno (pyclbr.Class attribute): Class Objects<2>. (line 21) +* lineno (pyclbr.Function attribute): Function Objects. (line 21) +* lineno (re.error attribute): Module Contents. (line 375) +* lineno (shlex.shlex attribute): shlex Objects. (line 169) +* lineno (traceback.TracebackException attribute): TracebackException Objects. + (line 46) +* lineno (tracemalloc.Filter attribute): Filter. (line 48) +* lineno (tracemalloc.Frame attribute): Frame. (line 17) +* lineno (xml.parsers.expat.ExpatError attribute): ExpatError Exceptions. + (line 27) +* lineno() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 92) +* LINES: curses<2>. (line 6) +* LINES <1>: Functions<3>. (line 125) +* LINES <2>: Functions<3>. (line 526) +* LINES <3>: Functions<3>. (line 550) +* LINES <4>: Functions<3>. (line 552) +* lines (os.terminal_size attribute): Querying the size of a terminal. + (line 34) +* linesep (email.policy.Policy attribute): email policy Policy Objects. + (line 152) +* linesep (in module os): Miscellaneous System Information. + (line 127) +* lineterminator (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 43) +* LineTooLong: http client — HTTP protocol client. + (line 179) +* line_buffering (io.TextIOWrapper attribute): Text I/O<2>. (line 162) +* line_num (csv.csvreader attribute): Reader Objects. (line 22) +* link() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 349) +* link() (in module os): Files and Directories. + (line 335) +* linkname (tarfile.TarInfo attribute): TarInfo Objects. (line 67) +* link_executable() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 401) +* link_shared_lib() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 411) +* link_shared_object() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 422) +* link_to() (pathlib.Path method): Methods<2>. (line 431) +* list: Glossary. (line 756) +* list (built-in class): Lists<2>. (line 10) +* List (class in typing): Classes functions and decorators. + (line 255) +* list (pdb command): Debugger Commands. (line 206) +* list comprehension: Glossary. (line 762) +* list() (imaplib.IMAP4 method): IMAP4 Objects. (line 126) +* list() (multiprocessing.managers.SyncManager method): Managers. + (line 214) +* list() (nntplib.NNTP method): Methods<3>. (line 104) +* list() (poplib.POP3 method): POP3 Objects. (line 56) +* list() (tarfile.TarFile method): TarFile Objects. (line 120) +* list; comprehensions: List displays. (line 6) +* list; display: List displays. (line 6) +* listdir() (in module os): Files and Directories. + (line 357) +* listen() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 215) +* listen() (in module logging.config): Configuration functions. + (line 107) +* listen() (in module turtle): Using screen events. + (line 6) +* listen() (socket.socket method): Socket Objects. (line 186) +* Listener (class in multiprocessing.connection): Listeners and Clients. + (line 50) +* listMethods() (xmlrpc.client.ServerProxy.system method): ServerProxy Objects. + (line 17) +* ListNoteBook (class in tkinter.tix): Manager Widgets. (line 13) +* listxattr() (in module os): Linux extended attributes. + (line 26) +* LIST_APPEND (opcode): Python Bytecode Instructions. + (line 322) +* list_dialects() (in module csv): Module Contents<3>. (line 91) +* list_folders() (mailbox.Maildir method): Maildir. (line 61) +* list_folders() (mailbox.MH method): MH. (line 32) +* literal: Literals. (line 6) +* literal <1>: Literals<2>. (line 6) +* Literal (in module typing): Classes functions and decorators. + (line 880) +* literal_eval() (in module ast): ast Helpers. (line 43) +* LittleEndianStructure (class in ctypes): Structured data types. + (line 14) +* ljust() (bytearray method): Bytes and Bytearray Operations. + (line 248) +* ljust() (bytes method): Bytes and Bytearray Operations. + (line 248) +* ljust() (str method): String Methods<2>. (line 287) +* LK_LOCK (in module msvcrt): File Operations. (line 19) +* LK_NBLCK (in module msvcrt): File Operations. (line 27) +* LK_NBRLCK (in module msvcrt): File Operations. (line 27) +* LK_RLCK (in module msvcrt): File Operations. (line 19) +* LK_UNLCK (in module msvcrt): File Operations. (line 33) +* ll (pdb command): Debugger Commands. (line 222) +* LMTP (class in smtplib): smtplib — SMTP protocol client. + (line 100) +* ln() (decimal.Context method): Context objects. (line 336) +* ln() (decimal.Decimal method): Decimal objects. (line 332) +* LNAME: getpass — Portable password input. + (line 38) +* lngettext() (gettext.GNUTranslations method): The GNUTranslations class. + (line 95) +* lngettext() (gettext.NullTranslations method): The NullTranslations class. + (line 65) +* lngettext() (in module gettext): GNU gettext API. (line 92) +* load() (http.cookiejar.FileCookieJar method): CookieJar and FileCookieJar Objects. + (line 125) +* load() (http.cookies.BaseCookie method): Cookie Objects. (line 40) +* load() (in module json): Basic Usage. (line 94) +* load() (in module marshal): marshal — Internal Python object serialization. + (line 60) +* load() (in module pickle): Module Interface. (line 58) +* load() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 44) +* load() (pickle.Unpickler method): Module Interface. (line 267) +* load() (tracemalloc.Snapshot class method): Snapshot. (line 50) +* loader: Finders and loaders. + (line 6) +* loader <1>: Glossary. (line 771) +* Loader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 158) +* loader (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 374) +* LoadError: http cookiejar — Cookie handling for HTTP clients. + (line 34) +* loader_state (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 398) +* LoadKey() (in module winreg): Functions<11>. (line 263) +* LoadLibrary() (ctypes.LibraryLoader method): Loading shared libraries. + (line 169) +* loads() (in module json): Basic Usage. (line 149) +* loads() (in module marshal): marshal — Internal Python object serialization. + (line 82) +* loads() (in module pickle): Module Interface. (line 76) +* loads() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 77) +* loads() (in module xmlrpc.client): Convenience Functions. + (line 19) +* loadTestsFromModule() (unittest.TestLoader method): Loading and running tests. + (line 41) +* loadTestsFromName() (unittest.TestLoader method): Loading and running tests. + (line 70) +* loadTestsFromNames() (unittest.TestLoader method): Loading and running tests. + (line 104) +* loadTestsFromTestCase() (unittest.TestLoader method): Loading and running tests. + (line 29) +* LOAD_ATTR (opcode): Python Bytecode Instructions. + (line 626) +* LOAD_BUILD_CLASS (opcode): Python Bytecode Instructions. + (line 433) +* load_cert_chain() (ssl.SSLContext method): SSL Contexts. (line 80) +* LOAD_CLASSDEREF (opcode): Python Bytecode Instructions. + (line 737) +* LOAD_CLOSURE (opcode): Python Bytecode Instructions. + (line 724) +* LOAD_CONST (opcode): Python Bytecode Instructions. + (line 527) +* load_default_certs() (ssl.SSLContext method): SSL Contexts. + (line 111) +* LOAD_DEREF (opcode): Python Bytecode Instructions. + (line 731) +* load_dh_params() (ssl.SSLContext method): SSL Contexts. (line 355) +* load_extension() (sqlite3.Connection method): Connection Objects. + (line 291) +* LOAD_FAST (opcode): Python Bytecode Instructions. + (line 711) +* LOAD_GLOBAL (opcode): Python Bytecode Instructions. + (line 693) +* LOAD_METHOD (opcode): Python Bytecode Instructions. + (line 818) +* load_module() (importlib.abc.FileLoader method): importlib abc – Abstract base classes related to import. + (line 492) +* load_module() (importlib.abc.InspectLoader method): importlib abc – Abstract base classes related to import. + (line 445) +* load_module() (importlib.abc.Loader method): importlib abc – Abstract base classes related to import. + (line 195) +* load_module() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 592) +* load_module() (importlib.machinery.SourceFileLoader method): importlib machinery – Importers and path hooks. + (line 247) +* load_module() (importlib.machinery.SourcelessFileLoader method): importlib machinery – Importers and path hooks. + (line 291) +* load_module() (in module imp): imp — Access the import internals. + (line 90) +* load_module() (zipimport.zipimporter method): zipimporter Objects. + (line 62) +* LOAD_NAME (opcode): Python Bytecode Instructions. + (line 531) +* load_package_tests() (in module test.support): test support — Utilities for the Python test suite. + (line 910) +* load_verify_locations() (ssl.SSLContext method): SSL Contexts. + (line 128) +* local (class in threading): Thread-Local Data. (line 15) +* localcontext() (in module decimal): Context objects. (line 25) +* LOCALE (in module re): Module Contents. (line 81) +* locale (module): locale — Internationalization services. + (line 6) +* localeconv() (in module locale): locale — Internationalization services. + (line 53) +* LocaleHTMLCalendar (class in calendar): calendar — General calendar-related functions. + (line 272) +* LocaleTextCalendar (class in calendar): calendar — General calendar-related functions. + (line 264) +* localName (xml.dom.Attr attribute): Attr Objects. (line 13) +* localName (xml.dom.Node attribute): Node Objects. (line 64) +* locals() (built-in function): Built-in Functions. (line 901) +* localtime() (in module email.utils): email utils Miscellaneous utilities. + (line 13) +* localtime() (in module time): Functions<2>. (line 141) +* Locator (class in xml.sax.xmlreader): xml sax xmlreader — Interface for XML parsers. + (line 43) +* Lock (class in asyncio): Lock. (line 6) +* Lock (class in multiprocessing): Synchronization primitives. + (line 46) +* Lock (class in threading): Lock Objects. (line 33) +* lock() (mailbox.Babyl method): Babyl. (line 59) +* lock() (mailbox.Mailbox method): Mailbox objects. (line 249) +* lock() (mailbox.Maildir method): Maildir. (line 107) +* lock() (mailbox.mbox method): mbox. (line 39) +* lock() (mailbox.MH method): MH. (line 84) +* lock() (mailbox.MMDF method): MMDF. (line 35) +* Lock() (multiprocessing.managers.SyncManager method): Managers. + (line 171) +* lock, interpreter: Thread State and the Global Interpreter Lock. + (line 6) +* locked() (asyncio.Condition method): Condition. (line 64) +* locked() (asyncio.Lock method): Lock. (line 59) +* locked() (asyncio.Semaphore method): Semaphore. (line 51) +* locked() (threading.Lock method): Lock Objects. (line 87) +* locked() (_thread.lock method): _thread — Low-level threading API. + (line 156) +* lockf() (in module fcntl): fcntl — The fcntl and ioctl system calls. + (line 120) +* lockf() (in module os): File Descriptor Operations. + (line 214) +* locking() (in module msvcrt): File Operations. (line 6) +* LockType (in module _thread): _thread — Low-level threading API. + (line 28) +* lock_held() (in module imp): imp — Access the import internals. + (line 257) +* log() (in module cmath): Power and logarithmic functions<2>. + (line 11) +* log() (in module logging): Module-Level Functions. + (line 150) +* log() (in module math): Power and logarithmic functions. + (line 28) +* log() (logging.Logger method): Logger Objects. (line 239) +* log10() (decimal.Context method): Context objects. (line 340) +* log10() (decimal.Decimal method): Decimal objects. (line 338) +* log10() (in module cmath): Power and logarithmic functions<2>. + (line 18) +* log10() (in module math): Power and logarithmic functions. + (line 55) +* log1p() (in module math): Power and logarithmic functions. + (line 36) +* log2() (in module math): Power and logarithmic functions. + (line 41) +* logb() (decimal.Context method): Context objects. (line 344) +* logb() (decimal.Decimal method): Decimal objects. (line 344) +* Logger (class in logging): Logger Objects. (line 23) +* LoggerAdapter (class in logging): LoggerAdapter Objects. + (line 11) +* logging (module): logging — Logging facility for Python. + (line 6) +* logging.config (module): logging config — Logging configuration. + (line 6) +* logging.handlers (module): logging handlers — Logging handlers. + (line 6) +* logical line: Logical lines. (line 6) +* logical_and() (decimal.Context method): Context objects. (line 348) +* logical_and() (decimal.Decimal method): Decimal objects. (line 352) +* logical_invert() (decimal.Context method): Context objects. + (line 353) +* logical_invert() (decimal.Decimal method): Decimal objects. + (line 358) +* logical_or() (decimal.Context method): Context objects. (line 357) +* logical_or() (decimal.Decimal method): Decimal objects. (line 363) +* logical_xor() (decimal.Context method): Context objects. (line 362) +* logical_xor() (decimal.Decimal method): Decimal objects. (line 369) +* login() (ftplib.FTP method): FTP Objects. (line 48) +* login() (imaplib.IMAP4 method): IMAP4 Objects. (line 132) +* login() (nntplib.NNTP method): Methods<3>. (line 48) +* login() (smtplib.SMTP method): SMTP Objects. (line 102) +* login_cram_md5() (imaplib.IMAP4 method): IMAP4 Objects. (line 137) +* LOGNAME: Process Parameters. (line 221) +* LOGNAME <1>: getpass — Portable password input. + (line 38) +* lognormvariate() (in module random): Real-valued distributions. + (line 63) +* logout() (imaplib.IMAP4 method): IMAP4 Objects. (line 143) +* LogRecord (class in logging): LogRecord Objects. (line 11) +* log_date_time_string() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 319) +* log_error() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 288) +* log_exception() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 209) +* log_message() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 294) +* log_request() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 281) +* log_to_stderr() (in module multiprocessing): Logging<2>. (line 24) +* long (2to3 fixer): Fixers. (line 181) +* longMessage (unittest.TestCase attribute): Test cases. (line 678) +* longname() (in module curses): Functions<3>. (line 264) +* LONG_MAX: Integer Objects. (line 107) +* lookup() (in module codecs): codecs — Codec registry and base classes. + (line 47) +* lookup() (in module unicodedata): unicodedata — Unicode Database. + (line 16) +* lookup() (symtable.SymbolTable method): Examining Symbol Tables. + (line 52) +* lookup() (tkinter.ttk.Style method): Ttk Styling. (line 81) +* LookupError: Base classes. (line 54) +* lookup_error() (in module codecs): Error Handlers. (line 117) +* loop control; target: The break statement. + (line 12) +* loop() (in module asyncore): asyncore — Asynchronous socket handler. + (line 49) +* loop; over mutable sequence: The for statement. (line 44) +* loop; statement: The break statement. + (line 6) +* loop; statement <1>: The continue statement. + (line 6) +* loop; statement <2>: The while statement. + (line 6) +* loop; statement <3>: The for statement. (line 6) +* lower() (bytearray method): Bytes and Bytearray Operations. + (line 553) +* lower() (bytes method): Bytes and Bytearray Operations. + (line 553) +* lower() (str method): String Methods<2>. (line 294) +* LPAR (in module token): token — Constants used with Python parse trees. + (line 54) +* lpAttributeList (subprocess.STARTUPINFO attribute): Windows Popen Helpers. + (line 60) +* lru_cache() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 73) +* lseek() (in module os): File Descriptor Operations. + (line 240) +* lshift() (in module operator): operator — Standard operators as functions. + (line 106) +* LSQB (in module token): token — Constants used with Python parse trees. + (line 62) +* lstat() (in module os): Files and Directories. + (line 394) +* lstat() (pathlib.Path method): Methods<2>. (line 214) +* lstrip() (bytearray method): Bytes and Bytearray Operations. + (line 260) +* lstrip() (bytes method): Bytes and Bytearray Operations. + (line 260) +* lstrip() (str method): String Methods<2>. (line 302) +* lsub() (imaplib.IMAP4 method): IMAP4 Objects. (line 150) +* lt() (in module operator): operator — Standard operators as functions. + (line 24) +* lt() (in module turtle): Turtle motion. (line 57) +* LWPCookieJar (class in http.cookiejar): FileCookieJar subclasses and co-operation with web browsers. + (line 28) +* lzma (module): lzma — Compression using the LZMA algorithm. + (line 6) +* LZMACompressor (class in lzma): Compressing and decompressing data in memory. + (line 6) +* LZMADecompressor (class in lzma): Compressing and decompressing data in memory. + (line 100) +* LZMAError: lzma — Compression using the LZMA algorithm. + (line 24) +* LZMAFile (class in lzma): Reading and writing compressed files. + (line 45) +* M (in module re): Module Contents. (line 101) +* machine() (in module platform): Cross Platform. (line 36) +* macros (netrc.netrc attribute): netrc Objects. (line 29) +* mac_ver() (in module platform): Mac OS Platform. (line 6) +* madvise() (mmap.mmap method): mmap — Memory-mapped file support. + (line 205) +* MADV_AUTOSYNC (in module mmap): MADV_* Constants. (line 6) +* MADV_CORE (in module mmap): MADV_* Constants. (line 6) +* MADV_DODUMP (in module mmap): MADV_* Constants. (line 6) +* MADV_DOFORK (in module mmap): MADV_* Constants. (line 6) +* MADV_DONTDUMP (in module mmap): MADV_* Constants. (line 6) +* MADV_DONTFORK (in module mmap): MADV_* Constants. (line 6) +* MADV_DONTNEED (in module mmap): MADV_* Constants. (line 6) +* MADV_FREE (in module mmap): MADV_* Constants. (line 6) +* MADV_HUGEPAGE (in module mmap): MADV_* Constants. (line 6) +* MADV_HWPOISON (in module mmap): MADV_* Constants. (line 6) +* MADV_MERGEABLE (in module mmap): MADV_* Constants. (line 6) +* MADV_NOCORE (in module mmap): MADV_* Constants. (line 6) +* MADV_NOHUGEPAGE (in module mmap): MADV_* Constants. (line 6) +* MADV_NORMAL (in module mmap): MADV_* Constants. (line 6) +* MADV_NOSYNC (in module mmap): MADV_* Constants. (line 6) +* MADV_PROTECT (in module mmap): MADV_* Constants. (line 6) +* MADV_RANDOM (in module mmap): MADV_* Constants. (line 6) +* MADV_REMOVE (in module mmap): MADV_* Constants. (line 6) +* MADV_SEQUENTIAL (in module mmap): MADV_* Constants. (line 6) +* MADV_SOFT_OFFLINE (in module mmap): MADV_* Constants. (line 6) +* MADV_UNMERGEABLE (in module mmap): MADV_* Constants. (line 6) +* MADV_WILLNEED (in module mmap): MADV_* Constants. (line 6) +* magic method: Glossary. (line 778) +* magic; method: Glossary. (line 782) +* MagicMock (class in unittest.mock): Magic Mock. (line 9) +* MAGIC_NUMBER (in module importlib.util): importlib util – Utility code for importers. + (line 13) +* Mailbox (class in mailbox): Mailbox objects. (line 6) +* mailbox (module): mailbox — Manipulate mailboxes in various formats. + (line 6) +* mailcap (module): mailcap — Mailcap file handling. + (line 6) +* Maildir (class in mailbox): Maildir. (line 6) +* MaildirMessage (class in mailbox): MaildirMessage. (line 6) +* mailfrom (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 74) +* MailmanProxy (class in smtpd): MailmanProxy Objects. + (line 6) +* main(): Process-wide parameters. + (line 9) +* main() <1>: Process-wide parameters. + (line 36) +* main() <2>: Process-wide parameters. + (line 217) +* main() (in module py_compile): py_compile — Compile Python source files. + (line 124) +* main() (in module site): Module contents<3>. (line 41) +* main() (in module unittest): Loading and running tests. + (line 489) +* mainloop() (in module turtle): Using screen events. + (line 106) +* maintype (email.headerregistry.ContentTypeHeader attribute): email headerregistry Custom Header Objects. + (line 276) +* main_thread() (in module threading): threading — Thread-based parallelism. + (line 115) +* major (email.headerregistry.MIMEVersionHeader attribute): email headerregistry Custom Header Objects. + (line 247) +* major() (in module os): Files and Directories. + (line 533) +* makedev() (in module os): Files and Directories. + (line 543) +* makedirs() (in module os): Files and Directories. + (line 452) +* makeelement() (xml.etree.ElementTree.Element method): Element Objects. + (line 174) +* makefile() (socket method): The standard type hierarchy. + (line 630) +* makefile() (socket.socket method): Socket Objects. (line 196) +* makeLogRecord() (in module logging): Module-Level Functions. + (line 236) +* makePickle() (logging.handlers.SocketHandler method): SocketHandler. + (line 45) +* makeRecord() (logging.Logger method): Logger Objects. (line 300) +* makeSocket() (logging.handlers.DatagramHandler method): DatagramHandler. + (line 28) +* makeSocket() (logging.handlers.SocketHandler method): SocketHandler. + (line 39) +* maketrans() (bytearray static method): Bytes and Bytearray Operations. + (line 121) +* maketrans() (bytes static method): Bytes and Bytearray Operations. + (line 121) +* maketrans() (str static method): String Methods<2>. (line 315) +* make_alternative() (email.message.EmailMessage method): email message Representing an email message. + (line 607) +* make_archive() (in module distutils.archive_util): distutils archive_util — Archiving utilities. + (line 9) +* make_archive() (in module shutil): Archiving operations. + (line 14) +* make_bad_fd() (in module test.support): test support — Utilities for the Python test suite. + (line 681) +* make_cookies() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 60) +* make_dataclass() (in module dataclasses): Module-level decorators classes and functions. + (line 309) +* make_file() (difflib.HtmlDiff method): difflib — Helpers for computing deltas. + (line 111) +* MAKE_FUNCTION (opcode): Python Bytecode Instructions. + (line 841) +* make_header() (in module email.header): email header Internationalized headers. + (line 200) +* make_legacy_pyc() (in module test.support): test support — Utilities for the Python test suite. + (line 190) +* make_mixed() (email.message.EmailMessage method): email message Representing an email message. + (line 617) +* make_msgid() (in module email.utils): email utils Miscellaneous utilities. + (line 29) +* make_parser() (in module xml.sax): xml sax — Support for SAX2 parsers. + (line 28) +* make_pkg() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 87) +* make_related() (email.message.EmailMessage method): email message Representing an email message. + (line 598) +* make_script() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 72) +* make_server() (in module wsgiref.simple_server): wsgiref simple_server – a simple WSGI HTTP server. + (line 14) +* make_table() (difflib.HtmlDiff method): difflib — Helpers for computing deltas. + (line 143) +* make_tarball() (in module distutils.archive_util): distutils archive_util — Archiving utilities. + (line 25) +* make_zipfile() (in module distutils.archive_util): distutils archive_util — Archiving utilities. + (line 39) +* make_zip_pkg() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 94) +* make_zip_script() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 79) +* malloc(): Overview<3>. (line 33) +* mangle_from_ (email.policy.Compat32 attribute): email policy Policy Objects. + (line 559) +* mangle_from_ (email.policy.Policy attribute): email policy Policy Objects. + (line 186) +* map (2to3 fixer): Fixers. (line 185) +* map() (built-in function): Built-in Functions. (line 913) +* map() (concurrent.futures.Executor method): Executor Objects. + (line 22) +* map() (multiprocessing.pool.Pool method): Process Pools. (line 90) +* map() (tkinter.ttk.Style method): Ttk Styling. (line 48) +* mapLogRecord() (logging.handlers.HTTPHandler method): HTTPHandler. + (line 27) +* mapping: Glossary. (line 782) +* Mapping (class in collections.abc): Collections Abstract Base Classes. + (line 182) +* Mapping (class in typing): Classes functions and decorators. + (line 217) +* mapping() (msilib.Control method): GUI classes. (line 20) +* MappingProxyType (class in types): Standard Interpreter Types. + (line 217) +* MappingView (class in collections.abc): Collections Abstract Base Classes. + (line 187) +* MappingView (class in typing): Classes functions and decorators. + (line 282) +* mapPriority() (logging.handlers.SysLogHandler method): SysLogHandler. + (line 183) +* maps (collections.ChainMap attribute): ChainMap objects. (line 39) +* maps() (in module nis): nis — Interface to Sun’s NIS Yellow Pages. + (line 41) +* MAP_ADD (opcode): Python Bytecode Instructions. + (line 327) +* map_async() (multiprocessing.pool.Pool method): Process Pools. + (line 107) +* map_table_b2() (in module stringprep): stringprep — Internet String Preparation. + (line 50) +* map_table_b3() (in module stringprep): stringprep — Internet String Preparation. + (line 55) +* map_to_type() (email.headerregistry.HeaderRegistry method): email headerregistry Custom Header Objects. + (line 376) +* marshal (module): marshal — Internal Python object serialization. + (line 6) +* marshalling; objects: pickle — Python object serialization. + (line 8) +* masking; operations: Bitwise Operations on Integer Types. + (line 6) +* Match (class in typing): Classes functions and decorators. + (line 479) +* match() (in module nis): nis — Interface to Sun’s NIS Yellow Pages. + (line 16) +* match() (in module re): Module Contents. (line 152) +* match() (pathlib.PurePath method): Methods and properties. + (line 212) +* match() (re.Pattern method): Regular Expression Objects. + (line 37) +* Matcher (class in test.support): test support — Utilities for the Python test suite. + (line 1086) +* matches() (test.support.Matcher method): test support — Utilities for the Python test suite. + (line 1088) +* match_hostname() (in module ssl): Certificate handling. + (line 6) +* match_test() (in module test.support): test support — Utilities for the Python test suite. + (line 246) +* match_value() (test.support.Matcher method): test support — Utilities for the Python test suite. + (line 1092) +* math (module): math — Mathematical functions. + (line 6) +* matmul() (in module operator): operator — Standard operators as functions. + (line 121) +* matrix multiplication: Binary arithmetic operations. + (line 23) +* max (datetime.date attribute): date Objects. (line 91) +* max (datetime.datetime attribute): datetime Objects. (line 252) +* max (datetime.time attribute): time Objects. (line 37) +* max (datetime.timedelta attribute): timedelta Objects. (line 78) +* max() (built-in function): Built-in Functions. (line 923) +* max() (decimal.Context method): Context objects. (line 367) +* max() (decimal.Decimal method): Decimal objects. (line 375) +* max() (in module audioop): audioop — Manipulate raw audio data. + (line 155) +* maxarray (reprlib.Repr attribute): Repr Objects. (line 15) +* maxdeque (reprlib.Repr attribute): Repr Objects. (line 15) +* maxdict (reprlib.Repr attribute): Repr Objects. (line 15) +* maxDiff (unittest.TestCase attribute): Test cases. (line 695) +* maxfrozenset (reprlib.Repr attribute): Repr Objects. (line 15) +* MAXIMUM_SUPPORTED (ssl.TLSVersion attribute): Constants<9>. + (line 527) +* maximum_version (ssl.SSLContext attribute): SSL Contexts. (line 542) +* maxlen (collections.deque attribute): deque objects. (line 120) +* maxlevel (reprlib.Repr attribute): Repr Objects. (line 10) +* maxlist (reprlib.Repr attribute): Repr Objects. (line 15) +* maxlong (reprlib.Repr attribute): Repr Objects. (line 27) +* maxother (reprlib.Repr attribute): Repr Objects. (line 40) +* maxpp() (in module audioop): audioop — Manipulate raw audio data. + (line 160) +* maxset (reprlib.Repr attribute): Repr Objects. (line 15) +* maxsize (asyncio.Queue attribute): Queue. (line 24) +* maxsize (in module sys): sys — System-specific parameters and functions. + (line 1005) +* maxstring (reprlib.Repr attribute): Repr Objects. (line 32) +* maxtuple (reprlib.Repr attribute): Repr Objects. (line 15) +* maxunicode (in module sys): sys — System-specific parameters and functions. + (line 1011) +* MAXYEAR (in module datetime): Constants. (line 13) +* max_count (email.headerregistry.BaseHeader attribute): email headerregistry Custom Header Objects. + (line 60) +* MAX_EMAX (in module decimal): Constants<4>. (line 16) +* MAX_INTERPOLATION_DEPTH (in module configparser): ConfigParser Objects. + (line 321) +* max_lines (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 258) +* max_line_length (email.policy.Policy attribute): email policy Policy Objects. + (line 145) +* max_mag() (decimal.Context method): Context objects. (line 371) +* max_mag() (decimal.Decimal method): Decimal objects. (line 382) +* max_memuse (in module test.support): test support — Utilities for the Python test suite. + (line 122) +* MAX_PREC (in module decimal): Constants<4>. (line 13) +* max_prefixlen (ipaddress.IPv4Address attribute): Address objects. + (line 43) +* max_prefixlen (ipaddress.IPv4Network attribute): Network objects. + (line 65) +* max_prefixlen (ipaddress.IPv6Address attribute): Address objects. + (line 169) +* max_prefixlen (ipaddress.IPv6Network attribute): Network objects. + (line 292) +* MAX_Py_ssize_t (in module test.support): test support — Utilities for the Python test suite. + (line 118) +* mbox (class in mailbox): mbox. (line 6) +* mboxMessage (class in mailbox): mboxMessage. (line 6) +* MB_ICONASTERISK (in module winsound): winsound — Sound-playing interface for Windows. + (line 128) +* MB_ICONEXCLAMATION (in module winsound): winsound — Sound-playing interface for Windows. + (line 132) +* MB_ICONHAND (in module winsound): winsound — Sound-playing interface for Windows. + (line 136) +* MB_ICONQUESTION (in module winsound): winsound — Sound-playing interface for Windows. + (line 140) +* MB_OK (in module winsound): winsound — Sound-playing interface for Windows. + (line 144) +* mean (statistics.NormalDist attribute): NormalDist objects. + (line 21) +* mean() (in module statistics): Function details. (line 10) +* median (statistics.NormalDist attribute): NormalDist objects. + (line 26) +* median() (in module statistics): Function details. (line 121) +* median_grouped() (in module statistics): Function details. (line 184) +* median_high() (in module statistics): Function details. (line 166) +* median_low() (in module statistics): Function details. (line 148) +* MemberDescriptorType (in module types): Standard Interpreter Types. + (line 206) +* membership; test: Membership test operations. + (line 37) +* memfd_create() (in module os): Files and Directories. + (line 1643) +* memmove() (in module ctypes): Utility functions. (line 137) +* MemoryBIO (class in ssl): Memory BIO Support<2>. + (line 134) +* MemoryError: Concrete exceptions. + (line 78) +* MemoryHandler (class in logging.handlers): MemoryHandler. (line 40) +* memoryview (built-in class): Memory Views. (line 10) +* memset() (in module ctypes): Utility functions. (line 143) +* merge() (in module heapq): heapq — Heap queue algorithm. + (line 79) +* Message (class in email.message): email message Message Representing an email message using the compat32 API. + (line 49) +* Message (class in mailbox): Message objects. (line 6) +* message digest, MD5: hashlib — Secure hashes and message digests. + (line 8) +* MessageBeep() (in module winsound): winsound — Sound-playing interface for Windows. + (line 31) +* MessageClass (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 163) +* MessageError: email errors Exception and Defect classes. + (line 13) +* MessageParseError: email errors Exception and Defect classes. + (line 19) +* messages (in module xml.parsers.expat.errors): Expat error constants. + (line 22) +* message_factory (email.policy.Policy attribute): email policy Policy Objects. + (line 195) +* message_from_binary_file() (in module email): Parser API. (line 129) +* message_from_bytes() (in module email): Parser API. (line 116) +* message_from_file() (in module email): Parser API. (line 152) +* message_from_string() (in module email): Parser API. (line 142) +* meta hooks: Import hooks. (line 6) +* meta path finder: Glossary. (line 791) +* meta() (in module curses): Functions<3>. (line 271) +* metaclass: Metaclasses. (line 6) +* metaclass <1>: Glossary. (line 800) +* metaclass (2to3 fixer): Fixers. (line 191) +* metaclass hint: Determining the appropriate metaclass. + (line 6) +* MetaPathFinder (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 45) +* metavar (optparse.Option attribute): Option attributes. (line 80) +* MetavarTypeHelpFormatter (class in argparse): formatter_class. + (line 10) +* meta_path (in module sys): sys — System-specific parameters and functions. + (line 1021) +* Meter (class in tkinter.tix): Basic Widgets. (line 44) +* method: Glossary. (line 815) +* method (urllib.request.Request attribute): Request Objects. + (line 52) +* method resolution order: Glossary. (line 823) +* method; call: Calls. (line 132) +* methodattrs (2to3 fixer): Fixers. (line 196) +* methodcaller() (in module operator): operator — Standard operators as functions. + (line 310) +* MethodDescriptorType (in module types): Standard Interpreter Types. + (line 103) +* methodHelp() (xmlrpc.client.ServerProxy.system method): ServerProxy Objects. + (line 43) +* methods (in module crypt): Module Attributes. (line 8) +* methods (pyclbr.Class attribute): Class Objects<2>. (line 46) +* methodSignature() (xmlrpc.client.ServerProxy.system method): ServerProxy Objects. + (line 22) +* MethodType (in module types): Standard Interpreter Types. + (line 77) +* MethodType (in module types) <1>: Function Objects<3>. + (line 14) +* MethodType (in module types) <2>: Method Objects<2>. (line 12) +* MethodWrapperType (in module types): Standard Interpreter Types. + (line 96) +* METHOD_BLOWFISH (in module crypt): Hashing Methods. (line 22) +* method_calls (unittest.mock.Mock attribute): The Mock Class. + (line 482) +* METHOD_CRYPT (in module crypt): Hashing Methods. (line 34) +* METHOD_MD5 (in module crypt): Hashing Methods. (line 29) +* METHOD_SHA256 (in module crypt): Hashing Methods. (line 17) +* METHOD_SHA512 (in module crypt): Hashing Methods. (line 11) +* METH_CLASS (built-in variable): Common Object Structures. + (line 225) +* METH_COEXIST (built-in variable): Common Object Structures. + (line 242) +* METH_FASTCALL (built-in variable): Common Object Structures. + (line 176) +* METH_NOARGS (built-in variable): Common Object Structures. + (line 203) +* METH_O (built-in variable): Common Object Structures. + (line 212) +* METH_STATIC (built-in variable): Common Object Structures. + (line 232) +* METH_VARARGS (built-in variable): Common Object Structures. + (line 157) +* MFD_ALLOW_SEALING (in module os): Files and Directories. + (line 1662) +* MFD_CLOEXEC (in module os): Files and Directories. + (line 1662) +* MFD_HUGETLB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_16GB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_16MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_1GB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_1MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_256MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_2GB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_2MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_32MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_512KB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_512MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_64KB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_8MB (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_MASK (in module os): Files and Directories. + (line 1662) +* MFD_HUGE_SHIFT (in module os): Files and Directories. + (line 1662) +* MH (class in mailbox): MH. (line 6) +* MHMessage (class in mailbox): MHMessage. (line 6) +* microsecond (datetime.datetime attribute): datetime Objects. + (line 289) +* microsecond (datetime.time attribute): time Objects. (line 62) +* MIME; base64 encoding: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 8) +* MIME; content type: mimetypes — Map filenames to MIME types. + (line 8) +* MIME; headers: mimetypes — Map filenames to MIME types. + (line 26) +* MIME; headers <1>: cgi — Common Gateway Interface support. + (line 8) +* MIME; quoted-printable encoding: quopri — Encode and decode MIME quoted-printable data. + (line 8) +* MIMEApplication (class in email.mime.application): email mime Creating email and MIME objects from scratch. + (line 95) +* MIMEAudio (class in email.mime.audio): email mime Creating email and MIME objects from scratch. + (line 125) +* MIMEBase (class in email.mime.base): email mime Creating email and MIME objects from scratch. + (line 30) +* MIMEImage (class in email.mime.image): email mime Creating email and MIME objects from scratch. + (line 158) +* MIMEMessage (class in email.mime.message): email mime Creating email and MIME objects from scratch. + (line 191) +* MIMEMultipart (class in email.mime.multipart): email mime Creating email and MIME objects from scratch. + (line 66) +* MIMENonMultipart (class in email.mime.nonmultipart): email mime Creating email and MIME objects from scratch. + (line 55) +* MIMEPart (class in email.message): email message Representing an email message. + (line 718) +* MIMEText (class in email.mime.text): email mime Creating email and MIME objects from scratch. + (line 209) +* MimeTypes (class in mimetypes): MimeTypes Objects. (line 10) +* mimetypes (module): mimetypes — Map filenames to MIME types. + (line 6) +* MIMEVersionHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 234) +* min (datetime.date attribute): date Objects. (line 87) +* min (datetime.datetime attribute): datetime Objects. (line 247) +* min (datetime.time attribute): time Objects. (line 33) +* min (datetime.timedelta attribute): timedelta Objects. (line 73) +* min() (built-in function): Built-in Functions. (line 955) +* min() (decimal.Context method): Context objects. (line 375) +* min() (decimal.Decimal method): Decimal objects. (line 387) +* MINEQUAL (in module token): token — Constants used with Python parse trees. + (line 174) +* MINIMUM_SUPPORTED (ssl.TLSVersion attribute): Constants<9>. + (line 525) +* minimum_version (ssl.SSLContext attribute): SSL Contexts. (line 563) +* minmax() (in module audioop): audioop — Manipulate raw audio data. + (line 164) +* minor (email.headerregistry.MIMEVersionHeader attribute): email headerregistry Custom Header Objects. + (line 251) +* minor() (in module os): Files and Directories. + (line 538) +* minus: Unary arithmetic and bitwise operations. + (line 10) +* MINUS (in module token): token — Constants used with Python parse trees. + (line 86) +* minus() (decimal.Context method): Context objects. (line 383) +* minute (datetime.datetime attribute): datetime Objects. (line 281) +* minute (datetime.time attribute): time Objects. (line 54) +* MINYEAR (in module datetime): Constants. (line 8) +* MIN_EMIN (in module decimal): Constants<4>. (line 19) +* MIN_ETINY (in module decimal): Constants<4>. (line 22) +* min_mag() (decimal.Context method): Context objects. (line 379) +* min_mag() (decimal.Decimal method): Decimal objects. (line 394) +* mirrored() (in module unicodedata): unicodedata — Unicode Database. + (line 69) +* misc_header (cmd.Cmd attribute): Cmd Objects. (line 152) +* MISSING (contextvars.contextvars.Token.Token attribute): Context Variables. + (line 94) +* MissingSectionHeaderError: Exceptions<5>. (line 60) +* missing_compiler_executable() (in module test.support): test support — Utilities for the Python test suite. + (line 961) +* MISSING_C_DOCSTRINGS (in module test.support): test support — Utilities for the Python test suite. + (line 132) +* MIXERDEV: ossaudiodev — Access to OSS-compatible audio devices. + (line 74) +* mkd() (ftplib.FTP method): FTP Objects. (line 217) +* mkdir() (in module os): Files and Directories. + (line 427) +* mkdir() (pathlib.Path method): Methods<2>. (line 220) +* mkdtemp() (in module tempfile): tempfile — Generate temporary files and directories. + (line 200) +* mkfifo() (in module os): Files and Directories. + (line 492) +* mknod() (in module os): Files and Directories. + (line 513) +* mkpath() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 493) +* mkpath() (in module distutils.dir_util): distutils dir_util — Directory tree operations. + (line 9) +* mksalt() (in module crypt): Module Functions<2>. + (line 39) +* mkstemp() (in module tempfile): tempfile — Generate temporary files and directories. + (line 142) +* mktemp() (in module tempfile): Deprecated functions and variables. + (line 15) +* mktime() (in module time): Functions<2>. (line 148) +* mktime_tz() (in module email.utils): email utils Miscellaneous utilities. + (line 133) +* mlsd() (ftplib.FTP method): FTP Objects. (line 165) +* mmap (class in mmap): mmap — Memory-mapped file support. + (line 48) +* mmap (module): mmap — Memory-mapped file support. + (line 6) +* MMDF (class in mailbox): MMDF. (line 6) +* MMDFMessage (class in mailbox): MMDFMessage. (line 6) +* Mock (class in unittest.mock): The Mock Class. (line 24) +* mock_add_spec() (unittest.mock.Mock method): The Mock Class. + (line 226) +* mock_calls (unittest.mock.Mock attribute): The Mock Class. (line 500) +* mock_open() (in module unittest.mock): mock_open. (line 6) +* mod() (in module operator): operator — Standard operators as functions. + (line 111) +* mode (io.FileIO attribute): Raw File I/O. (line 57) +* mode (ossaudiodev.oss_audio_device attribute): Audio Device Objects. + (line 238) +* mode (statistics.NormalDist attribute): NormalDist objects. + (line 31) +* mode (tarfile.TarInfo attribute): TarInfo Objects. (line 55) +* mode() (in module statistics): Function details. (line 229) +* mode() (in module turtle): Settings and special methods. + (line 6) +* modf() (in module math): Number-theoretic and representation functions. + (line 172) +* modified() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 45) +* Modify() (msilib.View method): View Objects. (line 23) +* modify() (select.devpoll method): /dev/poll Polling Objects. + (line 49) +* modify() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 85) +* modify() (select.poll method): Polling Objects. (line 60) +* modify() (selectors.BaseSelector method): Classes<3>. (line 94) +* module: Glossary. (line 830) +* module (pyclbr.Class attribute): Class Objects<2>. (line 13) +* module (pyclbr.Function attribute): Function Objects. (line 13) +* module spec: Finders and loaders. + (line 6) +* module spec <1>: Glossary. (line 839) +* module; array: The standard type hierarchy. + (line 213) +* module; array <1>: Binary Sequence Types — bytes bytearray memoryview. + (line 6) +* module; base64: binascii — Convert between binary and ASCII. + (line 6) +* module; bdb: pdb — The Python Debugger. + (line 17) +* module; binhex: binascii — Convert between binary and ASCII. + (line 6) +* module; builtins: The dir Function. (line 43) +* module; builtins <1>: Complete Python programs. + (line 6) +* module; builtins <2>: Embedding Python<2>. + (line 12) +* module; builtins <3>: Initializing and finalizing the interpreter. + (line 8) +* module; builtins <4>: Sub-interpreter support. + (line 26) +* module; cmd: pdb — The Python Debugger. + (line 17) +* module; copy: copyreg — Register pickle support functions. + (line 8) +* module; crypt: pwd — The password database. + (line 43) +* module; dbm.gnu: The standard type hierarchy. + (line 280) +* module; dbm.gnu <1>: Restrictions. (line 6) +* module; dbm.ndbm: The standard type hierarchy. + (line 280) +* module; dbm.ndbm <1>: Restrictions. (line 6) +* module; errno: Concrete exceptions. + (line 118) +* module; glob: fnmatch — Unix filename pattern matching. + (line 34) +* module; imp: Built-in Functions. (line 1775) +* module; importing: The import statement. + (line 6) +* module; io: The standard type hierarchy. + (line 630) +* module; json: Saving structured data with json. + (line 6) +* module; math: Numeric Types — int float complex. + (line 104) +* module; math <1>: Constants<3>. (line 47) +* module; namespace: The standard type hierarchy. + (line 536) +* module; os: posix — The most common POSIX system calls. + (line 12) +* module; pickle: copy — Shallow and deep copy operations. + (line 68) +* module; pickle <1>: copyreg — Register pickle support functions. + (line 8) +* module; pickle <2>: shelve — Python object persistence. + (line 8) +* module; pickle <3>: marshal — Internal Python object serialization. + (line 15) +* module; pty: File Descriptor Operations. + (line 355) +* module; pwd: os path — Common pathname manipulations. + (line 140) +* module; pyexpat: xml parsers expat — Fast XML parsing using Expat. + (line 20) +* module; re: String Methods<2>. (line 6) +* module; re <1>: fnmatch — Unix filename pattern matching. + (line 8) +* module; search; path: The Module Search Path. + (line 6) +* module; search; path <1>: linecache — Random access to text lines. + (line 30) +* module; search; path <2>: sys — System-specific parameters and functions. + (line 1061) +* module; search; path <3>: site — Site-specific configuration hook. + (line 14) +* module; search; path <4>: Embedding Python<2>. + (line 12) +* module; search; path <5>: Initializing and finalizing the interpreter. + (line 8) +* module; search; path <6>: Process-wide parameters. + (line 120) +* module; search; path <7>: Process-wide parameters. + (line 133) +* module; shelve: marshal — Internal Python object serialization. + (line 15) +* module; signal: _thread — Low-level threading API. + (line 173) +* module; signal <1>: Signal Handling<2>. (line 8) +* module; sitecustomize: site — Site-specific configuration hook. + (line 94) +* module; socket: Internet Protocols and Support. + (line 6) +* module; stat: Files and Directories. + (line 1002) +* module; string: locale — Internationalization services. + (line 460) +* module; struct: Socket Objects. (line 551) +* module; sys: Standard Modules. (line 6) +* module; sys <1>: The try statement. (line 69) +* module; sys <2>: Complete Python programs. + (line 6) +* module; sys <3>: Built-in Functions. (line 1217) +* module; sys <4>: Embedding Python<2>. + (line 12) +* module; sys <5>: Initializing and finalizing the interpreter. + (line 8) +* module; sys <6>: Sub-interpreter support. + (line 26) +* module; types: Type Objects. (line 6) +* module; urllib.request: http client — HTTP protocol client. + (line 8) +* module; usercustomize: site — Site-specific configuration hook. + (line 105) +* module; uu: binascii — Convert between binary and ASCII. + (line 6) +* module; _locale: locale — Internationalization services. + (line 16) +* module; _thread: High-level API. (line 45) +* module; __main__: Resolution of names. + (line 50) +* module; __main__ <1>: Complete Python programs. + (line 6) +* module; __main__ <2>: Complete Python programs. + (line 19) +* module; __main__ <3>: runpy — Locating and executing Python modules. + (line 30) +* module; __main__ <4>: runpy — Locating and executing Python modules. + (line 98) +* module; __main__ <5>: Embedding Python<2>. + (line 12) +* module; __main__ <6>: Initializing and finalizing the interpreter. + (line 8) +* module; __main__ <7>: Sub-interpreter support. + (line 26) +* ModuleFinder (class in modulefinder): modulefinder — Find modules used by a script. + (line 26) +* modulefinder (module): modulefinder — Find modules used by a script. + (line 6) +* ModuleInfo (class in pkgutil): pkgutil — Package extension utility. + (line 13) +* ModuleNotFoundError: Concrete exceptions. + (line 51) +* modules (in module sys): sys — System-specific parameters and functions. + (line 1051) +* modules (in module sys) <1>: Importing Modules<2>. + (line 7) +* modules (in module sys) <2>: Initializing and finalizing the interpreter. + (line 8) +* modules (modulefinder.ModuleFinder attribute): modulefinder — Find modules used by a script. + (line 49) +* ModuleSpec (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 352) +* modules_cleanup() (in module test.support): test support — Utilities for the Python test suite. + (line 773) +* modules_setup() (in module test.support): test support — Utilities for the Python test suite. + (line 769) +* ModuleType (class in types): Standard Interpreter Types. + (line 117) +* ModuleType (in module types): Module Objects. (line 8) +* module_for_loader() (in module importlib.util): importlib util – Utility code for importers. + (line 133) +* module_from_spec() (in module importlib.util): importlib util – Utility code for importers. + (line 117) +* module_repr() (importlib.abc.Loader method): importlib abc – Abstract base classes related to import. + (line 272) +* modulo: Binary arithmetic operations. + (line 35) +* monotonic() (in module time): Functions<2>. (line 161) +* monotonic_ns() (in module time): Functions<2>. (line 174) +* month (datetime.date attribute): date Objects. (line 106) +* month (datetime.datetime attribute): datetime Objects. (line 268) +* month() (in module calendar): calendar — General calendar-related functions. + (line 339) +* monthcalendar() (in module calendar): calendar — General calendar-related functions. + (line 328) +* monthdatescalendar() (calendar.Calendar method): calendar — General calendar-related functions. + (line 89) +* monthdays2calendar() (calendar.Calendar method): calendar — General calendar-related functions. + (line 95) +* monthdayscalendar() (calendar.Calendar method): calendar — General calendar-related functions. + (line 101) +* monthrange() (in module calendar): calendar — General calendar-related functions. + (line 323) +* month_abbr (in module calendar): calendar — General calendar-related functions. + (line 383) +* month_name (in module calendar): calendar — General calendar-related functions. + (line 376) +* Morsel (class in http.cookies): Morsel Objects. (line 6) +* most_common() (collections.Counter method): Counter objects. + (line 77) +* mouseinterval() (in module curses): Functions<3>. (line 276) +* mousemask() (in module curses): Functions<3>. (line 283) +* move() (curses.panel.Panel method): Panel Objects. (line 35) +* move() (curses.window method): Window Objects. (line 393) +* move() (in module shutil): Directory and files operations. + (line 302) +* move() (mmap.mmap method): mmap — Memory-mapped file support. + (line 218) +* move() (tkinter.ttk.Treeview method): ttk Treeview. (line 221) +* move_file() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 498) +* move_file() (in module distutils.file_util): distutils file_util — Single file operations. + (line 34) +* move_to_end() (collections.OrderedDict method): OrderedDict objects. + (line 53) +* MozillaCookieJar (class in http.cookiejar): FileCookieJar subclasses and co-operation with web browsers. + (line 9) +* MRO: Glossary. (line 845) +* mro() (class method): Special Attributes. (line 40) +* msg (http.client.HTTPResponse attribute): HTTPResponse Objects. + (line 40) +* msg (json.JSONDecodeError attribute): Exceptions<13>. (line 11) +* msg (re.error attribute): Module Contents. (line 362) +* msg (traceback.TracebackException attribute): TracebackException Objects. + (line 59) +* msg() (telnetlib.Telnet method): Telnet Objects. (line 78) +* msi: msilib — Read and write Microsoft Installer files. + (line 8) +* msilib (module): msilib — Read and write Microsoft Installer files. + (line 6) +* msvcrt (module): msvcrt — Useful routines from the MS VC++ runtime. + (line 6) +* mtime (gzip.GzipFile attribute): gzip — Support for gzip files. + (line 140) +* mtime (tarfile.TarInfo attribute): TarInfo Objects. (line 51) +* mtime() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 39) +* mt_interact() (telnetlib.Telnet method): Telnet Objects. (line 117) +* mul() (in module audioop): audioop — Manipulate raw audio data. + (line 169) +* mul() (in module operator): operator — Standard operators as functions. + (line 116) +* MultiCall (class in xmlrpc.client): MultiCall Objects. (line 9) +* MULTILINE (in module re): Module Contents. (line 101) +* MultiLoopChildWatcher (class in asyncio): Process Watchers. + (line 100) +* multimode() (in module statistics): Function details. (line 257) +* MultipartConversionError: email errors Exception and Defect classes. + (line 42) +* multiplication: Binary arithmetic operations. + (line 16) +* multiply() (decimal.Context method): Context objects. (line 388) +* multiprocessing (module): multiprocessing — Process-based parallelism. + (line 6) +* multiprocessing.connection (module): Listeners and Clients. + (line 6) +* multiprocessing.dummy (module): The multiprocessing dummy module. + (line 6) +* multiprocessing.Manager() (in module multiprocessing.sharedctypes): Managers. + (line 12) +* multiprocessing.managers (module): Managers. (line 18) +* multiprocessing.pool (module): Process Pools. (line 6) +* multiprocessing.sharedctypes (module): The multiprocessing sharedctypes module. + (line 6) +* multiprocessing.shared_memory (module): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 6) +* mutable: Glossary. (line 849) +* mutable object: Objects values and types. + (line 11) +* mutable sequence; loop over: The for statement. (line 44) +* mutable; sequence; types: Mutable Sequence Types. + (line 6) +* MutableMapping (class in collections.abc): Collections Abstract Base Classes. + (line 182) +* MutableMapping (class in typing): Classes functions and decorators. + (line 225) +* MutableSequence (class in collections.abc): Collections Abstract Base Classes. + (line 159) +* MutableSequence (class in typing): Classes functions and decorators. + (line 233) +* MutableSet (class in collections.abc): Collections Abstract Base Classes. + (line 177) +* MutableSet (class in typing): Classes functions and decorators. + (line 213) +* mvderwin() (curses.window method): Window Objects. (line 397) +* mvwin() (curses.window method): Window Objects. (line 404) +* myrights() (imaplib.IMAP4 method): IMAP4 Objects. (line 157) +* name: Identifiers and keywords. + (line 6) +* name <1>: Binding of names. (line 6) +* name <2>: Identifiers Names. (line 6) +* name (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 65) +* name (contextvars.ContextVar attribute): Context Variables. + (line 24) +* name (doctest.DocTest attribute): DocTest Objects. (line 30) +* name (email.headerregistry.BaseHeader attribute): email headerregistry Custom Header Objects. + (line 45) +* name (hashlib.hash attribute): Hash algorithms. (line 105) +* name (hmac.HMAC attribute): hmac — Keyed-Hashing for Message Authentication. + (line 100) +* name (http.cookiejar.Cookie attribute): Cookie Objects<2>. (line 28) +* name (importlib.abc.FileLoader attribute): importlib abc – Abstract base classes related to import. + (line 484) +* name (importlib.machinery.ExtensionFileLoader attribute): importlib machinery – Importers and path hooks. + (line 311) +* name (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 368) +* name (importlib.machinery.SourceFileLoader attribute): importlib machinery – Importers and path hooks. + (line 224) +* name (importlib.machinery.SourcelessFileLoader attribute): importlib machinery – Importers and path hooks. + (line 268) +* name (in module os): os — Miscellaneous operating system interfaces. + (line 46) +* NAME (in module token): token — Constants used with Python parse trees. + (line 42) +* name (inspect.Parameter attribute): Introspecting callables with the Signature object. + (line 157) +* name (io.FileIO attribute): Raw File I/O. (line 61) +* name (multiprocessing.Process attribute): Process and exceptions. + (line 67) +* name (multiprocessing.shared_memory.SharedMemory attribute): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 91) +* name (os.DirEntry attribute): Files and Directories. + (line 847) +* name (ossaudiodev.oss_audio_device attribute): Audio Device Objects. + (line 234) +* name (pyclbr.Class attribute): Class Objects<2>. (line 17) +* name (pyclbr.Function attribute): Function Objects. (line 17) +* name (tarfile.TarInfo attribute): TarInfo Objects. (line 43) +* name (threading.Thread attribute): Thread Objects. (line 136) +* name (xml.dom.Attr attribute): Attr Objects. (line 8) +* name (xml.dom.DocumentType attribute): DocumentType Objects. + (line 33) +* name (zipfile.Path attribute): Path Objects. (line 21) +* name() (in module unicodedata): unicodedata — Unicode Database. + (line 25) +* name2codepoint (in module html.entities): html entities — Definitions of HTML general entities. + (line 30) +* name; binding: The import statement. + (line 6) +* name; binding <1>: The import statement. + (line 51) +* name; binding <2>: Function definitions. + (line 6) +* name; binding <3>: Function definitions. + (line 6) +* name; binding <4>: Class definitions. (line 6) +* name; mangling: Private Variables. (line 13) +* name; mangling <1>: Identifiers Names. (line 14) +* Named Shared Memory: multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 10) +* named tuple: Glossary. (line 854) +* NamedTemporaryFile() (in module tempfile): tempfile — Generate temporary files and directories. + (line 71) +* NamedTuple (class in typing): Classes functions and decorators. + (line 488) +* namedtuple() (in module collections): namedtuple Factory Function for Tuples with Named Fields. + (line 11) +* NameError: Concrete exceptions. + (line 89) +* NameError (built-in exception): Resolution of names. + (line 16) +* namelist() (zipfile.ZipFile method): ZipFile Objects. (line 105) +* nameprep() (in module encodings.idna): encodings idna — Internationalized Domain Names in Applications. + (line 47) +* namer (logging.handlers.BaseRotatingHandler attribute): BaseRotatingHandler. + (line 17) +* namereplace_errors() (in module codecs): Error Handlers. (line 157) +* namespace: Naming and binding. (line 6) +* namespace <1>: Glossary. (line 880) +* Namespace (class in argparse): The Namespace object. + (line 6) +* Namespace (class in multiprocessing.managers): Managers. (line 226) +* namespace package: Glossary. (line 894) +* namespace() (imaplib.IMAP4 method): IMAP4 Objects. (line 162) +* Namespace() (multiprocessing.managers.SyncManager method): Managers. + (line 176) +* NamespaceErr: Exceptions<16>. (line 66) +* namespaceURI (xml.dom.Node attribute): Node Objects. (line 74) +* NAMESPACE_DNS (in module uuid): uuid — UUID objects according to RFC 4122. + (line 203) +* NAMESPACE_OID (in module uuid): uuid — UUID objects according to RFC 4122. + (line 212) +* NAMESPACE_URL (in module uuid): uuid — UUID objects according to RFC 4122. + (line 208) +* NAMESPACE_X500 (in module uuid): uuid — UUID objects according to RFC 4122. + (line 216) +* NaN: Built-in Functions. (line 584) +* nan (in module cmath): Constants<3>. (line 33) +* nan (in module math): Constants<2>. (line 31) +* nanj (in module cmath): Constants<3>. (line 40) +* NannyNag: tabnanny — Detection of ambiguous indentation. + (line 37) +* napms() (in module curses): Functions<3>. (line 292) +* nargs (optparse.Option attribute): Option attributes. (line 42) +* native_id (threading.Thread attribute): Thread Objects. (line 156) +* nbytes (memoryview attribute): Memory Views. (line 375) +* ncurses_version (in module curses): Constants<6>. (line 23) +* ndiff() (in module difflib): difflib — Helpers for computing deltas. + (line 232) +* ndim (memoryview attribute): Memory Views. (line 441) +* ne (2to3 fixer): Fixers. (line 201) +* ne() (in module operator): operator — Standard operators as functions. + (line 24) +* needs_input (bz2.BZ2Decompressor attribute): Incremental de compression. + (line 85) +* needs_input (lzma.LZMADecompressor attribute): Compressing and decompressing data in memory. + (line 175) +* neg() (in module operator): operator — Standard operators as functions. + (line 128) +* negation: Unary arithmetic and bitwise operations. + (line 10) +* nested scope: Glossary. (line 903) +* netmask (ipaddress.IPv4Network attribute): Network objects. + (line 101) +* netmask (ipaddress.IPv6Network attribute): Network objects. + (line 312) +* NetmaskValueError: Custom Exceptions. (line 13) +* netrc (class in netrc): netrc — netrc file processing. + (line 13) +* netrc (module): netrc — netrc file processing. + (line 6) +* NetrcParseError: netrc — netrc file processing. + (line 36) +* netscape (http.cookiejar.CookiePolicy attribute): CookiePolicy Objects. + (line 68) +* network (ipaddress.IPv4Interface attribute): Interface objects. + (line 28) +* network (ipaddress.IPv6Interface attribute): Interface objects. + (line 76) +* Network News Transfer Protocol: nntplib — NNTP protocol client. + (line 8) +* network_address (ipaddress.IPv4Network attribute): Network objects. + (line 86) +* network_address (ipaddress.IPv6Network attribute): Network objects. + (line 306) +* new() (in module hashlib): Hash algorithms. (line 55) +* new() (in module hmac): hmac — Keyed-Hashing for Message Authentication. + (line 12) +* new-style class: Glossary. (line 913) +* newer() (in module distutils.dep_util): distutils dep_util — Dependency checking. + (line 10) +* newer_group() (in module distutils.dep_util): distutils dep_util — Dependency checking. + (line 24) +* newer_pairwise() (in module distutils.dep_util): distutils dep_util — Dependency checking. + (line 17) +* newfunc (C type): Slot Type typedefs. (line 41) +* newgroups() (nntplib.NNTP method): Methods<3>. (line 80) +* NEWLINE (in module token): token — Constants used with Python parse trees. + (line 48) +* NEWLINE token: Logical lines. (line 6) +* NEWLINE token <1>: Compound statements. + (line 54) +* newlines (io.TextIOBase attribute): Text I/O<2>. (line 24) +* newnews() (nntplib.NNTP method): Methods<3>. (line 95) +* newpad() (in module curses): Functions<3>. (line 296) +* NewType() (in module typing): Classes functions and decorators. + (line 607) +* newwin() (in module curses): Functions<3>. (line 316) +* new_alignment() (formatter.writer method): The Writer Interface. + (line 17) +* new_child() (collections.ChainMap method): ChainMap objects. + (line 46) +* new_class() (in module types): Dynamic Type Creation. + (line 6) +* new_compiler() (in module distutils.ccompiler): distutils ccompiler — CCompiler base class. + (line 48) +* new_event_loop() (asyncio.AbstractEventLoopPolicy method): Policy Objects. + (line 27) +* new_event_loop() (in module asyncio): Event Loop. (line 61) +* new_font() (formatter.writer method): The Writer Interface. + (line 25) +* new_margin() (formatter.writer method): The Writer Interface. + (line 35) +* new_module() (in module imp): imp — Access the import internals. + (line 124) +* new_panel() (in module curses.panel): Functions<4>. (line 12) +* new_spacing() (formatter.writer method): The Writer Interface. + (line 42) +* new_styles() (formatter.writer method): The Writer Interface. + (line 46) +* next (2to3 fixer): Fixers. (line 205) +* next (pdb command): Debugger Commands. (line 168) +* next() (built-in function): Built-in Functions. (line 982) +* next() (nntplib.NNTP method): Methods<3>. (line 233) +* next() (tarfile.TarFile method): TarFile Objects. (line 130) +* next() (tkinter.ttk.Treeview method): ttk Treeview. (line 232) +* nextfile() (in module fileinput): fileinput — Iterate over lines from multiple input streams. + (line 116) +* nextkey() (dbm.gnu.gdbm method): dbm gnu — GNU’s reinterpretation of dbm. + (line 93) +* nextSibling (xml.dom.Node attribute): Node Objects. (line 42) +* next_minus() (decimal.Context method): Context objects. (line 392) +* next_minus() (decimal.Decimal method): Decimal objects. (line 399) +* next_plus() (decimal.Context method): Context objects. (line 396) +* next_plus() (decimal.Decimal method): Decimal objects. (line 405) +* next_toward() (decimal.Context method): Context objects. (line 400) +* next_toward() (decimal.Decimal method): Decimal objects. (line 411) +* ngettext() (gettext.GNUTranslations method): The GNUTranslations class. + (line 47) +* ngettext() (gettext.NullTranslations method): The NullTranslations class. + (line 41) +* ngettext() (in module gettext): GNU gettext API. (line 54) +* nice() (in module os): Process Management. (line 318) +* nis (module): nis — Interface to Sun’s NIS Yellow Pages. + (line 6) +* NL (in module token): token — Constants used with Python parse trees. + (line 265) +* nl() (in module curses): Functions<3>. (line 326) +* nlargest() (in module heapq): heapq — Heap queue algorithm. + (line 106) +* nlst() (ftplib.FTP method): FTP Objects. (line 179) +* nl_langinfo() (in module locale): locale — Internationalization services. + (line 172) +* NNTP (class in nntplib): nntplib — NNTP protocol client. + (line 54) +* NNTP; protocol: nntplib — NNTP protocol client. + (line 8) +* NNTPDataError: nntplib — NNTP protocol client. + (line 153) +* NNTPError: nntplib — NNTP protocol client. + (line 122) +* nntplib (module): nntplib — NNTP protocol client. + (line 6) +* NNTPPermanentError: nntplib — NNTP protocol client. + (line 143) +* NNTPProtocolError: nntplib — NNTP protocol client. + (line 148) +* NNTPReplyError: nntplib — NNTP protocol client. + (line 133) +* NNTPTemporaryError: nntplib — NNTP protocol client. + (line 138) +* nntp_implementation (nntplib.NNTP attribute): Attributes. (line 14) +* NNTP_SSL (class in nntplib): nntplib — NNTP protocol client. + (line 93) +* nntp_version (nntplib.NNTP attribute): Attributes. (line 6) +* nocbreak() (in module curses): Functions<3>. (line 332) +* NoDataAllowedErr: Exceptions<16>. (line 83) +* node() (in module platform): Cross Platform. (line 41) +* nodelay() (curses.window method): Window Objects. (line 408) +* nodeName (xml.dom.Node attribute): Node Objects. (line 79) +* NodeTransformer (class in ast): ast Helpers. (line 154) +* nodeType (xml.dom.Node attribute): Node Objects. (line 8) +* nodeValue (xml.dom.Node attribute): Node Objects. (line 88) +* NodeVisitor (class in ast): ast Helpers. (line 119) +* noecho() (in module curses): Functions<3>. (line 337) +* NOEXPR (in module locale): locale — Internationalization services. + (line 251) +* NoModificationAllowedErr: Exceptions<16>. (line 88) +* nonblock() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 75) +* NonCallableMagicMock (class in unittest.mock): Magic Mock. (line 21) +* NonCallableMock (class in unittest.mock): The Mock Class. (line 558) +* None (Built-in object): Truth Value Testing. + (line 15) +* None (built-in variable): Built-in Constants. (line 18) +* nonl() (in module curses): Functions<3>. (line 341) +* nonzero (2to3 fixer): Fixers. (line 211) +* noop() (imaplib.IMAP4 method): IMAP4 Objects. (line 166) +* noop() (poplib.POP3 method): POP3 Objects. (line 78) +* NoOptionError: Exceptions<5>. (line 32) +* NOP (opcode): Python Bytecode Instructions. + (line 55) +* noqiflush() (in module curses): Functions<3>. (line 351) +* noraw() (in module curses): Functions<3>. (line 359) +* NoReturn (in module typing): Classes functions and decorators. + (line 781) +* NormalDist (class in statistics): NormalDist objects. (line 13) +* normalize() (decimal.Context method): Context objects. (line 404) +* normalize() (decimal.Decimal method): Decimal objects. (line 419) +* normalize() (in module locale): locale — Internationalization services. + (line 356) +* normalize() (in module unicodedata): unicodedata — Unicode Database. + (line 81) +* normalize() (xml.dom.Node method): Node Objects. (line 141) +* NORMALIZE_WHITESPACE (in module doctest): Option Flags. (line 39) +* normalvariate() (in module random): Real-valued distributions. + (line 70) +* NORMAL_PRIORITY_CLASS (in module subprocess): Windows Constants. + (line 80) +* normcase() (in module os.path): os path — Common pathname manipulations. + (line 275) +* normpath() (in module os.path): os path — Common pathname manipulations. + (line 284) +* NoSectionError: Exceptions<5>. (line 10) +* NoSuchMailboxError: Exceptions<14>. (line 13) +* NotADirectoryError: OS exceptions. (line 86) +* notation: Notation. (line 6) +* notationDecl() (xml.sax.handler.DTDHandler method): DTDHandler Objects. + (line 8) +* NotationDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 272) +* notations (xml.dom.DocumentType attribute): DocumentType Objects. + (line 46) +* NotConnected: http client — HTTP protocol client. + (line 133) +* NoteBook (class in tkinter.tix): Manager Widgets. (line 22) +* Notebook (class in tkinter.ttk): ttk Notebook. (line 6) +* NotEmptyError: Exceptions<14>. (line 20) +* NOTEQUAL (in module token): token — Constants used with Python parse trees. + (line 138) +* NotFoundErr: Exceptions<16>. (line 72) +* notify() (asyncio.Condition method): Condition. (line 55) +* notify() (threading.Condition method): Condition Objects. (line 152) +* notify_all() (asyncio.Condition method): Condition. (line 68) +* notify_all() (threading.Condition method): Condition Objects. + (line 172) +* notimeout() (curses.window method): Window Objects. (line 412) +* NotImplemented (built-in variable): Built-in Constants. (line 25) +* NotImplementedError: Concrete exceptions. + (line 95) +* NotStandaloneHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 323) +* NotSupportedErr: Exceptions<16>. (line 78) +* NotSupportedError: Exceptions<4>. (line 40) +* not_() (in module operator): operator — Standard operators as functions. + (line 49) +* noutrefresh() (curses.window method): Window Objects. (line 420) +* now() (datetime.datetime class method): datetime Objects. (line 60) +* no_proxy: urllib request — Extensible library for opening URLs. + (line 303) +* no_tracing() (in module test.support): test support — Utilities for the Python test suite. + (line 643) +* no_type_check() (in module typing): Classes functions and decorators. + (line 716) +* no_type_check_decorator() (in module typing): Classes functions and decorators. + (line 726) +* npgettext() (gettext.GNUTranslations method): The GNUTranslations class. + (line 79) +* npgettext() (gettext.NullTranslations method): The NullTranslations class. + (line 55) +* npgettext() (in module gettext): GNU gettext API. (line 77) +* NSIG (in module signal): Module contents<2>. (line 164) +* nsmallest() (in module heapq): heapq — Heap queue algorithm. + (line 114) +* NTEventLogHandler (class in logging.handlers): NTEventLogHandler. + (line 11) +* ntohl() (in module socket): Other functions<2>. (line 179) +* ntohs() (in module socket): Other functions<2>. (line 186) +* ntransfercmd() (ftplib.FTP method): FTP Objects. (line 157) +* NT_OFFSET (in module token): token — Constants used with Python parse trees. + (line 256) +* null; operation: The pass statement. (line 6) +* null; operation <1>: The pass statement. (line 6) +* nullcontext() (in module contextlib): Utilities. (line 147) +* NullFormatter (class in formatter): Formatter Implementations. + (line 10) +* NullHandler (class in logging): NullHandler. (line 12) +* NullImporter (class in imp): imp — Access the import internals. + (line 345) +* NullTranslations (class in gettext): The NullTranslations class. + (line 13) +* NullWriter (class in formatter): Writer Implementations. + (line 10) +* number: Numeric literals. (line 6) +* Number (class in numbers): numbers — Numeric abstract base classes. + (line 15) +* NUMBER (in module token): token — Constants used with Python parse trees. + (line 44) +* numbers (module): numbers — Numeric abstract base classes. + (line 6) +* number_class() (decimal.Context method): Context objects. (line 408) +* number_class() (decimal.Decimal method): Decimal objects. (line 428) +* numerator (fractions.Fraction attribute): fractions — Rational numbers. + (line 90) +* numerator (numbers.Rational attribute): The numeric tower. (line 47) +* numeric literal: Numeric literals. (line 6) +* numeric() (in module unicodedata): unicodedata — Unicode Database. + (line 43) +* numeric; conversions: Numeric Types — int float complex. + (line 104) +* numeric; literals: Numeric Types — int float complex. + (line 19) +* Numerical Python: Built-in Functions. (line 1486) +* numinput() (in module turtle): Input methods. (line 22) +* numliterals (2to3 fixer): Fixers. (line 215) +* num_addresses (ipaddress.IPv4Network attribute): Network objects. + (line 128) +* num_addresses (ipaddress.IPv6Network attribute): Network objects. + (line 324) +* num_tickets (ssl.SSLContext attribute): SSL Contexts. (line 573) +* N_TOKENS (in module token): token — Constants used with Python parse trees. + (line 254) +* n_waiting (threading.Barrier attribute): Barrier Objects. (line 97) +* obj (memoryview attribute): Memory Views. (line 364) +* object: Objects values and types. + (line 6) +* object <1>: Glossary. (line 921) +* object (built-in class): Built-in Functions. (line 989) +* object (UnicodeError attribute): Concrete exceptions. + (line 350) +* object.__slots__ (built-in variable): __slots__. (line 13) +* object; asynchronous-generator: Asynchronous generator functions. + (line 55) +* object; Boolean: The standard type hierarchy. + (line 93) +* object; Boolean <1>: Numeric Types — int float complex. + (line 6) +* object; built-in function: The standard type hierarchy. + (line 485) +* object; built-in function <1>: Calls. (line 132) +* object; built-in method: The standard type hierarchy. + (line 498) +* object; built-in method <1>: Calls. (line 132) +* object; bytearray: Mutable Sequence Types. + (line 6) +* object; bytearray <1>: Binary Sequence Types — bytes bytearray memoryview. + (line 6) +* object; bytearray <2>: Bytearray Objects. (line 6) +* object; bytearray <3>: Byte Array Objects. (line 6) +* object; bytes: Binary Sequence Types — bytes bytearray memoryview. + (line 6) +* object; bytes <1>: Bytes Objects. (line 6) +* object; bytes <2>: Bytes Objects<2>. (line 9) +* object; callable: The standard type hierarchy. + (line 292) +* object; callable <1>: Slicings. (line 38) +* object; Capsule: Capsules<2>. (line 6) +* object; class: The standard type hierarchy. + (line 573) +* object; class <1>: Calls. (line 137) +* object; class <2>: Class definitions. (line 6) +* object; class instance: The standard type hierarchy. + (line 573) +* object; class instance <1>: The standard type hierarchy. + (line 600) +* object; class instance <2>: Calls. (line 141) +* object; code: The standard type hierarchy. + (line 649) +* object; code <1>: Methods. (line 43) +* object; code <2>: marshal — Internal Python object serialization. + (line 30) +* object; code <3>: Cell Objects. (line 53) +* object; complex: The standard type hierarchy. + (line 119) +* object; complex number: Numeric Types — int float complex. + (line 6) +* object; complex number <1>: Complex Number Objects. + (line 6) +* object; deallocation: Finalization and De-allocation. + (line 6) +* object; dictionary: The standard type hierarchy. + (line 260) +* object; dictionary <1>: The standard type hierarchy. + (line 573) +* object; dictionary <2>: Basic customization. + (line 223) +* object; dictionary <3>: Dictionary displays. + (line 6) +* object; dictionary <4>: Subscriptions. (line 6) +* object; dictionary <5>: Assignment statements. + (line 116) +* object; dictionary <6>: Mapping Types — dict. + (line 6) +* object; dictionary <7>: Dictionary Objects. (line 6) +* object; Ellipsis: The standard type hierarchy. + (line 41) +* object; file: Reading and Writing Files. + (line 6) +* object; file <1>: File Objects. (line 6) +* object; finalization: Finalization and De-allocation. + (line 6) +* object; floating point: The standard type hierarchy. + (line 107) +* object; floating point <1>: Numeric Types — int float complex. + (line 6) +* object; floating point <2>: Floating Point Objects. + (line 6) +* object; frame: The standard type hierarchy. + (line 709) +* object; frozenset: The standard type hierarchy. + (line 242) +* object; frozenset <1>: Set Objects. (line 6) +* object; function: The standard type hierarchy. + (line 297) +* object; function <1>: The standard type hierarchy. + (line 485) +* object; function <2>: Calls. (line 123) +* object; function <3>: Calls. (line 132) +* object; function <4>: Function definitions. + (line 6) +* object; function <5>: Function Objects<3>. + (line 6) +* object; generator: The standard type hierarchy. + (line 687) +* object; generator <1>: Generator expressions. + (line 6) +* object; generator <2>: Yield expressions. (line 108) +* object; immutable: The standard type hierarchy. + (line 146) +* object; immutable sequence: The standard type hierarchy. + (line 146) +* object; instance: The standard type hierarchy. + (line 573) +* object; instance <1>: The standard type hierarchy. + (line 600) +* object; instance <2>: Calls. (line 141) +* object; instancemethod: Instance Method Objects. + (line 6) +* object; integer: The standard type hierarchy. + (line 77) +* object; integer <1>: Numeric Types — int float complex. + (line 6) +* object; integer <2>: Integer Objects. (line 6) +* object; io.StringIO: Text Sequence Type — str. + (line 35) +* object; list: The standard type hierarchy. + (line 200) +* object; list <1>: List displays. (line 6) +* object; list <2>: Attribute references. + (line 10) +* object; list <3>: Subscriptions. (line 6) +* object; list <4>: Slicings. (line 6) +* object; list <5>: Assignment statements. + (line 108) +* object; list <6>: Mutable Sequence Types. + (line 6) +* object; list <7>: Lists<2>. (line 6) +* object; list <8>: List Objects. (line 6) +* object; long integer: Integer Objects. (line 6) +* object; mapping: The standard type hierarchy. + (line 249) +* object; mapping <1>: The standard type hierarchy. + (line 621) +* object; mapping <2>: Subscriptions. (line 6) +* object; mapping <3>: Assignment statements. + (line 116) +* object; mapping <4>: Mapping Types — dict. + (line 6) +* object; mapping <5>: Container Objects. (line 6) +* object; memoryview: Binary Sequence Types — bytes bytearray memoryview. + (line 6) +* object; memoryview <1>: Ellipsis Object. (line 11) +* object; method: Instance Objects. (line 31) +* object; method <1>: The standard type hierarchy. + (line 388) +* object; method <2>: The standard type hierarchy. + (line 498) +* object; method <3>: Calls. (line 132) +* object; method <4>: Methods. (line 6) +* object; method <5>: Method Objects<2>. (line 6) +* object; module: The standard type hierarchy. + (line 521) +* object; module <1>: Attribute references. + (line 10) +* object; module <2>: Module Objects. (line 6) +* object; mutable: The standard type hierarchy. + (line 192) +* object; mutable <1>: Assignment statements. + (line 6) +* object; mutable <2>: Assignment statements. + (line 103) +* object; mutable sequence: The standard type hierarchy. + (line 192) +* object; None: The standard type hierarchy. + (line 20) +* object; None <1>: Expression statements. + (line 17) +* object; None <2>: The None Object. (line 6) +* object; NotImplemented: The standard type hierarchy. + (line 28) +* object; numeric: The standard type hierarchy. + (line 47) +* object; numeric <1>: The standard type hierarchy. + (line 621) +* object; numeric <2>: Comparisons<2>. (line 43) +* object; numeric <3>: Numeric Types — int float complex. + (line 6) +* object; numeric <4>: Numeric Objects. (line 6) +* object; range: Ranges. (line 6) +* object; sequence: The standard type hierarchy. + (line 127) +* object; sequence <1>: The standard type hierarchy. + (line 621) +* object; sequence <2>: Subscriptions. (line 6) +* object; sequence <3>: Slicings. (line 6) +* object; sequence <4>: Membership test operations. + (line 37) +* object; sequence <5>: Assignment statements. + (line 108) +* object; sequence <6>: The for statement. (line 6) +* object; sequence <7>: Common Sequence Operations. + (line 6) +* object; sequence <8>: Sequence Objects. (line 6) +* object; set: The standard type hierarchy. + (line 236) +* object; set <1>: Set displays. (line 6) +* object; set <2>: Set Types — set frozenset. + (line 6) +* object; set <3>: Set Objects. (line 6) +* object; set type: The standard type hierarchy. + (line 219) +* object; slice: Emulating container types. + (line 64) +* object; socket: socket — Low-level networking interface. + (line 17) +* object; string: Subscriptions. (line 6) +* object; string <1>: Slicings. (line 6) +* object; string <2>: Ranges. (line 124) +* object; traceback: The standard type hierarchy. + (line 757) +* object; traceback <1>: The raise statement. + (line 21) +* object; traceback <2>: The try statement. (line 69) +* object; traceback <3>: sys — System-specific parameters and functions. + (line 365) +* object; traceback <4>: traceback — Print or retrieve a stack traceback. + (line 16) +* object; tuple: The standard type hierarchy. + (line 172) +* object; tuple <1>: Subscriptions. (line 6) +* object; tuple <2>: Slicings. (line 6) +* object; tuple <3>: Expression lists. (line 11) +* object; tuple <4>: Immutable Sequence Types. + (line 6) +* object; tuple <5>: Tuples. (line 6) +* object; tuple <6>: Tuple Objects. (line 6) +* object; type: Built-in Functions. (line 1666) +* object; type <1>: Objects Types and Reference Counts. + (line 6) +* object; type <2>: Type Objects<2>. (line 6) +* object; user-defined function: The standard type hierarchy. + (line 297) +* object; user-defined function <1>: Calls. (line 123) +* object; user-defined function <2>: Function definitions. + (line 6) +* object; user-defined method: The standard type hierarchy. + (line 388) +* objects; comparing: Comparisons<2>. (line 43) +* object_filenames() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 467) +* objobjargproc (C type): Slot Type typedefs. (line 122) +* objobjproc (C type): Slot Type typedefs. (line 120) +* obufcount() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 218) +* obuffree() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 223) +* oct() (built-in function): Built-in Functions. (line 999) +* octal literal: Numeric literals. (line 6) +* octal; literals: Numeric Types — int float complex. + (line 19) +* octdigits (in module string): String constants. (line 32) +* offset (traceback.TracebackException attribute): TracebackException Objects. + (line 54) +* offset (xml.parsers.expat.ExpatError attribute): ExpatError Exceptions. + (line 32) +* OK (in module curses): Constants<6>. (line 13) +* old_value (contextvars.contextvars.Token.Token attribute): Context Variables. + (line 87) +* OleDLL (class in ctypes): Loading shared libraries. + (line 32) +* onclick() (in module turtle): Using screen events. + (line 56) +* ondrag() (in module turtle): Using events. (line 57) +* onecmd() (cmd.Cmd method): Cmd Objects. (line 57) +* onkey() (in module turtle): Using screen events. + (line 12) +* onkeypress() (in module turtle): Using screen events. + (line 35) +* onkeyrelease() (in module turtle): Using screen events. + (line 12) +* onrelease() (in module turtle): Using events. (line 30) +* onscreenclick() (in module turtle): Using screen events. + (line 56) +* ontimer() (in module turtle): Using screen events. + (line 86) +* OP (in module token): token — Constants used with Python parse trees. + (line 242) +* open() (built-in function): Built-in Functions. (line 1023) +* open() (distutils.text_file.TextFile method): distutils text_file — The TextFile class. + (line 91) +* open() (imaplib.IMAP4 method): IMAP4 Objects. (line 170) +* open() (in module aifc): aifc — Read and write AIFF and AIFC files. + (line 30) +* open() (in module bz2): De compression of files. + (line 6) +* open() (in module codecs): codecs — Codec registry and base classes. + (line 167) +* open() (in module dbm): dbm — Interfaces to Unix “databases”. + (line 35) +* open() (in module dbm.dumb): dbm dumb — Portable DBM implementation. + (line 30) +* open() (in module dbm.gnu): dbm gnu — GNU’s reinterpretation of dbm. + (line 27) +* open() (in module dbm.ndbm): dbm ndbm — Interface based on ndbm. + (line 31) +* open() (in module gzip): gzip — Support for gzip files. + (line 28) +* open() (in module io): High-level Module Interface. + (line 12) +* open() (in module lzma): Reading and writing compressed files. + (line 6) +* open() (in module os): File Descriptor Operations. + (line 259) +* open() (in module ossaudiodev): ossaudiodev — Access to OSS-compatible audio devices. + (line 42) +* open() (in module shelve): shelve — Python object persistence. + (line 17) +* open() (in module sunau): sunau — Read and write Sun AU files. + (line 48) +* open() (in module tarfile): tarfile — Read and write tar archive files. + (line 35) +* open() (in module tokenize): Tokenizing Input. (line 100) +* open() (in module wave): wave — Read and write WAV files. + (line 17) +* open() (in module webbrowser): webbrowser — Convenient Web-browser controller. + (line 50) +* open() (pathlib.Path method): Methods<2>. (line 245) +* open() (pipes.Template method): Template Objects. (line 42) +* open() (tarfile.TarFile class method): TarFile Objects. (line 96) +* open() (telnetlib.Telnet method): Telnet Objects. (line 65) +* open() (urllib.request.OpenerDirector method): OpenerDirector Objects. + (line 46) +* open() (urllib.request.URLopener method): Legacy interface. + (line 98) +* open() (webbrowser.controller method): Browser Controller Objects. + (line 9) +* open() (zipfile.Path method): Path Objects. (line 25) +* open() (zipfile.ZipFile method): ZipFile Objects. (line 109) +* OpenDatabase() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 45) +* OpenerDirector (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 266) +* openfp() (in module sunau): sunau — Read and write Sun AU files. + (line 66) +* openfp() (in module wave): wave — Read and write WAV files. + (line 48) +* OpenKey() (in module winreg): Functions<11>. (line 291) +* OpenKeyEx() (in module winreg): Functions<11>. (line 291) +* openlog() (in module syslog): syslog — Unix syslog library routines. + (line 36) +* openmixer() (in module ossaudiodev): ossaudiodev — Access to OSS-compatible audio devices. + (line 70) +* openpty() (in module os): File Descriptor Operations. + (line 353) +* openpty() (in module pty): pty — Pseudo-terminal utilities. + (line 29) +* OpenSSL; (use in module hashlib): Hash algorithms. (line 21) +* OpenSSL; (use in module ssl): ssl — TLS/SSL wrapper for socket objects. + (line 8) +* OPENSSL_VERSION (in module ssl): Constants<9>. (line 441) +* OPENSSL_VERSION_INFO (in module ssl): Constants<9>. (line 451) +* OPENSSL_VERSION_NUMBER (in module ssl): Constants<9>. (line 461) +* OpenView() (msilib.Database method): Database Objects. (line 6) +* open_binary() (in module importlib.resources): importlib resources – Resources. + (line 53) +* open_code() (in module io): High-level Module Interface. + (line 22) +* open_connection() (in module asyncio): Streams. (line 42) +* open_new() (in module webbrowser): webbrowser — Convenient Web-browser controller. + (line 67) +* open_new() (webbrowser.controller method): Browser Controller Objects. + (line 15) +* open_new_tab() (in module webbrowser): webbrowser — Convenient Web-browser controller. + (line 72) +* open_new_tab() (webbrowser.controller method): Browser Controller Objects. + (line 21) +* open_osfhandle() (in module msvcrt): File Operations. (line 44) +* open_resource() (importlib.abc.ResourceReader method): importlib abc – Abstract base classes related to import. + (line 319) +* open_text() (in module importlib.resources): importlib resources – Resources. + (line 64) +* open_unix_connection() (in module asyncio): Streams. (line 98) +* open_unknown() (urllib.request.URLopener method): Legacy interface. + (line 109) +* open_urlresource() (in module test.support): test support — Utilities for the Python test suite. + (line 714) +* OperationalError: Exceptions<4>. (line 32) +* operations on; dictionary; type: Mapping Types — dict. + (line 6) +* operations on; integer; types: Bitwise Operations on Integer Types. + (line 6) +* operations on; list; type: Mutable Sequence Types. + (line 16) +* operations on; mapping; types: Mapping Types — dict. + (line 6) +* operations on; numeric; types: Numeric Types — int float complex. + (line 94) +* operations on; sequence; types: Common Sequence Operations. + (line 21) +* operations on; sequence; types <1>: Mutable Sequence Types. + (line 16) +* operator (2to3 fixer): Fixers. (line 219) +* operator (module): operator — Standard operators as functions. + (line 6) +* operator; !=: Comparisons. (line 6) +* operator; != <1>: Comparisons<2>. (line 6) +* operator; % (percent): Binary arithmetic operations. + (line 35) +* operator; % (percent) <1>: Numeric Types — int float complex. + (line 27) +* operator; & (ampersand): Binary bitwise operations. + (line 12) +* operator; & (ampersand) <1>: Bitwise Operations on Integer Types. + (line 6) +* operator; * (asterisk): Binary arithmetic operations. + (line 16) +* operator; * (asterisk) <1>: Numeric Types — int float complex. + (line 27) +* operator; **: The power operator. (line 6) +* operator; ** <1>: Numeric Types — int float complex. + (line 27) +* operator; + (plus): Unary arithmetic and bitwise operations. + (line 13) +* operator; + (plus) <1>: Binary arithmetic operations. + (line 60) +* operator; + (plus) <2>: Numeric Types — int float complex. + (line 27) +* operator; - (minus): Unary arithmetic and bitwise operations. + (line 10) +* operator; - (minus) <1>: Binary arithmetic operations. + (line 66) +* operator; - (minus) <2>: Numeric Types — int float complex. + (line 27) +* operator; / (slash): Binary arithmetic operations. + (line 28) +* operator; / (slash) <1>: Numeric Types — int float complex. + (line 27) +* operator; //: Binary arithmetic operations. + (line 28) +* operator; // <1>: Numeric Types — int float complex. + (line 27) +* operator; < (less): Comparisons. (line 6) +* operator; < (less) <1>: Comparisons<2>. (line 6) +* operator; <<: Shifting operations. + (line 6) +* operator; << <1>: Bitwise Operations on Integer Types. + (line 6) +* operator; <=: Comparisons. (line 6) +* operator; <= <1>: Comparisons<2>. (line 6) +* operator; ==: Comparisons. (line 6) +* operator; == <1>: Comparisons<2>. (line 6) +* operator; > (greater): Comparisons. (line 6) +* operator; > (greater) <1>: Comparisons<2>. (line 6) +* operator; >=: Comparisons. (line 6) +* operator; >= <1>: Comparisons<2>. (line 6) +* operator; >>: Shifting operations. + (line 6) +* operator; >> <1>: Bitwise Operations on Integer Types. + (line 6) +* operator; @ (at): Binary arithmetic operations. + (line 23) +* operator; and: Boolean operations. (line 21) +* operator; and <1>: Truth Value Testing. + (line 23) +* operator; and <2>: Boolean Operations — and or not. + (line 22) +* operator; comparison: Comparisons<2>. (line 6) +* operator; in: Membership test operations. + (line 37) +* operator; in <1>: Comparisons<2>. (line 64) +* operator; in <2>: Common Sequence Operations. + (line 21) +* operator; is: Membership test operations. + (line 40) +* operator; is <1>: Comparisons<2>. (line 6) +* operator; is not: Membership test operations. + (line 40) +* operator; is not <1>: Comparisons<2>. (line 6) +* operator; not: Boolean operations. (line 18) +* operator; not <1>: Boolean Operations — and or not. + (line 22) +* operator; not in: Membership test operations. + (line 37) +* operator; not in <1>: Comparisons<2>. (line 64) +* operator; not in <2>: Common Sequence Operations. + (line 21) +* operator; or: Boolean operations. (line 25) +* operator; or <1>: Truth Value Testing. + (line 23) +* operator; or <2>: Boolean Operations — and or not. + (line 22) +* operator; overloading: Special method names. + (line 6) +* operator; precedence: Operator precedence. + (line 6) +* operator; ^ (caret): Binary bitwise operations. + (line 15) +* operator; ^ (caret) <1>: Bitwise Operations on Integer Types. + (line 6) +* operator; | (vertical bar): Binary bitwise operations. + (line 18) +* operator; | (vertical bar) <1>: Bitwise Operations on Integer Types. + (line 6) +* operator; ~ (tilde): Unary arithmetic and bitwise operations. + (line 15) +* operator; ~ (tilde) <1>: Bitwise Operations on Integer Types. + (line 6) +* operators: Operators. (line 6) +* opmap (in module dis): Opcode collections. (line 13) +* opname (in module dis): Opcode collections. (line 9) +* optimize() (in module pickletools): Programmatic Interface<2>. + (line 31) +* OPTIMIZED_BYTECODE_SUFFIXES (in module importlib.machinery): importlib machinery – Importers and path hooks. + (line 30) +* optim_args_from_interpreter_flags() (in module test.support): test support — Utilities for the Python test suite. + (line 397) +* Optional (in module typing): Classes functions and decorators. + (line 828) +* OptionGroup (class in optparse): Grouping Options. (line 12) +* OptionMenu (class in tkinter.tix): Basic Widgets. (line 49) +* OptionParser (class in optparse): Creating the parser. + (line 9) +* Options (class in ssl): Constants<9>. (line 340) +* options (doctest.Example attribute): Example Objects. (line 53) +* options (ssl.SSLContext attribute): SSL Contexts. (line 584) +* options() (configparser.ConfigParser method): ConfigParser Objects. + (line 102) +* optionxform() (configparser.ConfigParser method): ConfigParser Objects. + (line 279) +* optparse (module): optparse — Parser for command line options. + (line 6) +* OP_ALL (in module ssl): Constants<9>. (line 209) +* OP_CIPHER_SERVER_PREFERENCE (in module ssl): Constants<9>. (line 298) +* OP_ENABLE_MIDDLEBOX_COMPAT (in module ssl): Constants<9>. (line 322) +* OP_NO_COMPRESSION (in module ssl): Constants<9>. (line 331) +* OP_NO_RENEGOTIATION (in module ssl): Constants<9>. (line 288) +* OP_NO_SSLv2 (in module ssl): Constants<9>. (line 217) +* OP_NO_SSLv3 (in module ssl): Constants<9>. (line 227) +* OP_NO_TICKET (in module ssl): Constants<9>. (line 344) +* OP_NO_TLSv1 (in module ssl): Constants<9>. (line 237) +* OP_NO_TLSv1_1 (in module ssl): Constants<9>. (line 249) +* OP_NO_TLSv1_2 (in module ssl): Constants<9>. (line 261) +* OP_NO_TLSv1_3 (in module ssl): Constants<9>. (line 273) +* OP_SINGLE_DH_USE (in module ssl): Constants<9>. (line 306) +* OP_SINGLE_ECDH_USE (in module ssl): Constants<9>. (line 314) +* ord() (built-in function): Built-in Functions. (line 1264) +* OrderedDict (class in collections): OrderedDict objects. + (line 39) +* OrderedDict (class in typing): Classes functions and decorators. + (line 367) +* ordered_attributes (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 107) +* origin (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 381) +* origin_req_host (urllib.request.Request attribute): Request Objects. + (line 30) +* origin_server (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 288) +* or_() (in module operator): operator — Standard operators as functions. + (line 133) +* os (module): os — Miscellaneous operating system interfaces. + (line 6) +* os.path (module): os path — Common pathname manipulations. + (line 6) +* OSError: Concrete exceptions. + (line 113) +* ossaudiodev (module): ossaudiodev — Access to OSS-compatible audio devices. + (line 6) +* OSSAudioError: ossaudiodev — Access to OSS-compatible audio devices. + (line 29) +* os_environ (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 166) +* output: Expression statements. + (line 17) +* output (subprocess.CalledProcessError attribute): Using the subprocess Module. + (line 205) +* output (subprocess.TimeoutExpired attribute): Using the subprocess Module. + (line 172) +* output (unittest.TestCase attribute): Test cases. (line 433) +* output() (http.cookies.BaseCookie method): Cookie Objects. (line 24) +* output() (http.cookies.Morsel method): Morsel Objects. (line 72) +* OutputChecker (class in doctest): OutputChecker objects. + (line 6) +* OutputString() (http.cookies.Morsel method): Morsel Objects. + (line 87) +* output_charset (email.charset.Charset attribute): email charset Representing character sets. + (line 74) +* output_charset() (gettext.NullTranslations method): The NullTranslations class. + (line 87) +* output_codec (email.charset.Charset attribute): email charset Representing character sets. + (line 88) +* output_difference() (doctest.OutputChecker method): OutputChecker objects. + (line 27) +* over() (nntplib.NNTP method): Methods<3>. (line 167) +* Overflow (class in decimal): Signals. (line 67) +* OverflowError: Concrete exceptions. + (line 177) +* OverflowError (built-in exception): Integer Objects. (line 107) +* OverflowError (built-in exception) <1>: Integer Objects. (line 146) +* OverflowError (built-in exception) <2>: Integer Objects. (line 188) +* OverflowError (built-in exception) <3>: Integer Objects. (line 199) +* OverflowError (built-in exception) <4>: Integer Objects. (line 210) +* OverflowError (built-in exception) <5>: Integer Objects. (line 222) +* overlap() (statistics.NormalDist method): NormalDist objects. + (line 96) +* overlaps() (ipaddress.IPv4Network method): Network objects. + (line 152) +* overlaps() (ipaddress.IPv6Network method): Network objects. + (line 336) +* overlay() (curses.window method): Window Objects. (line 427) +* overload() (in module typing): Classes functions and decorators. + (line 661) +* overwrite() (curses.window method): Window Objects. (line 440) +* owner() (pathlib.Path method): Methods<2>. (line 257) +* O_APPEND (in module os): File Descriptor Operations. + (line 304) +* O_ASYNC (in module os): File Descriptor Operations. + (line 336) +* O_BINARY (in module os): File Descriptor Operations. + (line 326) +* O_CLOEXEC (in module os): File Descriptor Operations. + (line 314) +* O_CREAT (in module os): File Descriptor Operations. + (line 304) +* O_DIRECT (in module os): File Descriptor Operations. + (line 336) +* O_DIRECTORY (in module os): File Descriptor Operations. + (line 336) +* O_DSYNC (in module os): File Descriptor Operations. + (line 314) +* O_EXCL (in module os): File Descriptor Operations. + (line 304) +* O_EXLOCK (in module os): File Descriptor Operations. + (line 336) +* O_NDELAY (in module os): File Descriptor Operations. + (line 314) +* O_NOATIME (in module os): File Descriptor Operations. + (line 336) +* O_NOCTTY (in module os): File Descriptor Operations. + (line 314) +* O_NOFOLLOW (in module os): File Descriptor Operations. + (line 336) +* O_NOINHERIT (in module os): File Descriptor Operations. + (line 326) +* O_NONBLOCK (in module os): File Descriptor Operations. + (line 314) +* O_PATH (in module os): File Descriptor Operations. + (line 336) +* O_RANDOM (in module os): File Descriptor Operations. + (line 326) +* O_RDONLY (in module os): File Descriptor Operations. + (line 304) +* O_RDWR (in module os): File Descriptor Operations. + (line 304) +* O_RSYNC (in module os): File Descriptor Operations. + (line 314) +* O_SEQUENTIAL (in module os): File Descriptor Operations. + (line 326) +* O_SHLOCK (in module os): File Descriptor Operations. + (line 336) +* O_SHORT_LIVED (in module os): File Descriptor Operations. + (line 326) +* O_SYNC (in module os): File Descriptor Operations. + (line 314) +* O_TEMPORARY (in module os): File Descriptor Operations. + (line 326) +* O_TEXT (in module os): File Descriptor Operations. + (line 326) +* O_TMPFILE (in module os): File Descriptor Operations. + (line 336) +* O_TRUNC (in module os): File Descriptor Operations. + (line 304) +* O_WRONLY (in module os): File Descriptor Operations. + (line 304) +* p (pdb command): Debugger Commands. (line 233) +* pack() (in module struct): Functions and Exceptions. + (line 13) +* pack() (mailbox.MH method): MH. (line 64) +* pack() (struct.Struct method): Classes<2>. (line 25) +* package: Packages<2>. (line 6) +* package <1>: site — Site-specific configuration hook. + (line 64) +* package <2>: Glossary. (line 927) +* Package (in module importlib.resources): importlib resources – Resources. + (line 37) +* package variable; __all__: Importing Modules<2>. + (line 7) +* package; namespace: Namespace packages. (line 6) +* package; portion: Namespace packages. (line 6) +* package; regular: Regular packages. (line 6) +* packed (ipaddress.IPv4Address attribute): Address objects. (line 65) +* packed (ipaddress.IPv6Address attribute): Address objects. (line 163) +* Packer (class in xdrlib): xdrlib — Encode and decode XDR data. + (line 18) +* packing (widgets): The Packer. (line 6) +* packing; binary; data: struct — Interpret bytes as packed binary data. + (line 8) +* pack_array() (xdrlib.Packer method): Packer Objects. (line 86) +* pack_bytes() (xdrlib.Packer method): Packer Objects. (line 54) +* pack_double() (xdrlib.Packer method): Packer Objects. (line 26) +* pack_farray() (xdrlib.Packer method): Packer Objects. (line 78) +* pack_float() (xdrlib.Packer method): Packer Objects. (line 22) +* pack_fopaque() (xdrlib.Packer method): Packer Objects. (line 38) +* pack_fstring() (xdrlib.Packer method): Packer Objects. (line 32) +* pack_into() (in module struct): Functions and Exceptions. + (line 19) +* pack_into() (struct.Struct method): Classes<2>. (line 30) +* pack_list() (xdrlib.Packer method): Packer Objects. (line 61) +* pack_opaque() (xdrlib.Packer method): Packer Objects. (line 49) +* pack_string() (xdrlib.Packer method): Packer Objects. (line 43) +* PAGER: pydoc — Documentation generator and online help system. + (line 47) +* pair_content() (in module curses): Functions<3>. (line 364) +* pair_number() (in module curses): Functions<3>. (line 370) +* PanedWindow (class in tkinter.tix): Manager Widgets. (line 6) +* parameter: Glossary. (line 936) +* Parameter (class in inspect): Introspecting callables with the Signature object. + (line 142) +* parameter; call semantics: Calls. (line 25) +* parameter; difference from argument: How can I pass optional or keyword parameters from one function to another?. + (line 16) +* parameter; function definition: The with statement. (line 103) +* ParameterizedMIMEHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 255) +* parameters (inspect.Signature attribute): Introspecting callables with the Signature object. + (line 79) +* params (email.headerregistry.ParameterizedMIMEHeader attribute): email headerregistry Custom Header Objects. + (line 263) +* pardir (in module os): Miscellaneous System Information. + (line 87) +* paren (2to3 fixer): Fixers. (line 251) +* parent (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 409) +* parent (pyclbr.Class attribute): Class Objects<2>. (line 25) +* parent (pyclbr.Function attribute): Function Objects. (line 25) +* parent (urllib.request.BaseHandler attribute): BaseHandler Objects. + (line 25) +* parent() (tkinter.ttk.Treeview method): ttk Treeview. (line 237) +* parenthesized form: Parenthesized forms. + (line 6) +* parentNode (xml.dom.Node attribute): Node Objects. (line 17) +* parents (collections.ChainMap attribute): ChainMap objects. + (line 59) +* parent_process() (in module multiprocessing): Miscellaneous<3>. + (line 35) +* paretovariate() (in module random): Real-valued distributions. + (line 83) +* parse() (doctest.DocTestParser method): DocTestParser objects. + (line 29) +* parse() (email.parser.BytesParser method): Parser API. (line 34) +* parse() (email.parser.Parser method): Parser API. (line 86) +* parse() (in module ast): ast Helpers. (line 9) +* parse() (in module cgi): Functions<8>. (line 9) +* parse() (in module xml.dom.minidom): xml dom minidom — Minimal DOM implementation. + (line 37) +* parse() (in module xml.dom.pulldom): xml dom pulldom — Support for building partial DOM trees. + (line 86) +* parse() (in module xml.etree.ElementTree): Functions<6>. (line 160) +* parse() (in module xml.sax): xml sax — Support for SAX2 parsers. + (line 39) +* parse() (string.Formatter method): Custom String Formatting. + (line 40) +* parse() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 29) +* parse() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 59) +* Parse() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 8) +* parse() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 8) +* parseaddr() (in module email.utils): email utils Miscellaneous utilities. + (line 59) +* parsebytes() (email.parser.BytesParser method): Parser API. + (line 54) +* parsedate() (in module email.utils): email utils Miscellaneous utilities. + (line 96) +* parsedate_to_datetime() (in module email.utils): email utils Miscellaneous utilities. + (line 119) +* parsedate_tz() (in module email.utils): email utils Miscellaneous utilities. + (line 108) +* ParseError (class in xml.etree.ElementTree): Exceptions<15>. + (line 6) +* ParseFile() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 16) +* ParseFlags() (in module imaplib): imaplib — IMAP4 protocol client. + (line 118) +* parser: Lexical analysis. (line 6) +* Parser (class in email.parser): Parser API. (line 75) +* parser (module): parser — Access Python parse trees. + (line 6) +* ParserCreate() (in module xml.parsers.expat): xml parsers expat — Fast XML parsing using Expat. + (line 46) +* ParserError: Exceptions and Error Handling. + (line 11) +* ParseResult (class in urllib.parse): Structured Parse Results. + (line 48) +* ParseResultBytes (class in urllib.parse): Structured Parse Results. + (line 73) +* parsestr() (email.parser.Parser method): Parser API. (line 96) +* parseString() (in module xml.dom.minidom): xml dom minidom — Minimal DOM implementation. + (line 50) +* parseString() (in module xml.dom.pulldom): xml dom pulldom — Support for building partial DOM trees. + (line 99) +* parseString() (in module xml.sax): xml sax — Support for SAX2 parsers. + (line 50) +* parse_and_bind() (in module readline): Init file. (line 8) +* parse_args() (argparse.ArgumentParser method): The parse_args method. + (line 6) +* PARSE_COLNAMES (in module sqlite3): Module functions and constants. + (line 38) +* parse_config_h() (in module sysconfig): Other functions<3>. + (line 57) +* PARSE_DECLTYPES (in module sqlite3): Module functions and constants. + (line 25) +* parse_header() (in module cgi): Functions<8>. (line 41) +* parse_headers() (in module http.client): http client — HTTP protocol client. + (line 108) +* parse_intermixed_args() (argparse.ArgumentParser method): Intermixed parsing. + (line 6) +* parse_known_args() (argparse.ArgumentParser method): Partial parsing. + (line 6) +* parse_known_intermixed_args() (argparse.ArgumentParser method): Intermixed parsing. + (line 9) +* parse_multipart() (in module cgi): Functions<8>. (line 19) +* parse_qs() (in module urllib.parse): URL Parsing. (line 140) +* parse_qsl() (in module urllib.parse): URL Parsing. (line 186) +* parsing; Python source code: parser — Access Python parse trees. + (line 6) +* ParsingError: Exceptions<5>. (line 65) +* partial (asyncio.IncompleteReadError attribute): Exceptions<9>. + (line 54) +* partial() (imaplib.IMAP4 method): IMAP4 Objects. (line 182) +* partial() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 215) +* partialmethod (class in functools): functools — Higher-order functions and operations on callable objects. + (line 246) +* parties (threading.Barrier attribute): Barrier Objects. (line 93) +* partition() (bytearray method): Bytes and Bytearray Operations. + (line 132) +* partition() (bytes method): Bytes and Bytearray Operations. + (line 132) +* partition() (str method): String Methods<2>. (line 331) +* pass_() (poplib.POP3 method): POP3 Objects. (line 36) +* Paste: Help menu Shell and Editor. + (line 30) +* patch() (in module test.support): test support — Utilities for the Python test suite. + (line 943) +* patch() (in module unittest.mock): patch. (line 10) +* patch.dict() (in module unittest.mock): patch dict. (line 6) +* patch.multiple() (in module unittest.mock): patch multiple. + (line 6) +* patch.object() (in module unittest.mock): patch object. (line 6) +* patch.stopall() (in module unittest.mock): patch methods start and stop. + (line 70) +* PATH: Changes in the Python API. + (line 143) +* PATH <1>: Changes in ‘python’ Command Behavior<2>. + (line 6) +* PATH <2>: Changes in ‘python’ Command Behavior<2>. + (line 11) +* PATH <3>: Changes in ‘python’ Command Behavior<2>. + (line 12) +* PATH <4>: The Module Search Path. + (line 16) +* PATH <5>: Executable Python Scripts. + (line 11) +* PATH <6>: Environment variables. + (line 27) +* PATH <7>: Miscellaneous. (line 16) +* PATH <8>: Installation steps. (line 32) +* PATH <9>: Installation steps. (line 56) +* PATH <10>: Installing Without UI. + (line 57) +* PATH <11>: Excursus Setting environment variables. + (line 23) +* PATH <12>: Excursus Setting environment variables. + (line 35) +* PATH <13>: Finding the Python executable. + (line 14) +* PATH <14>: Finding the Python executable. + (line 21) +* PATH <15>: Finding the Python executable. + (line 22) +* PATH <16>: Python Launcher for Windows. + (line 13) +* PATH <17>: From the command-line. + (line 9) +* PATH <18>: From the command-line. + (line 38) +* PATH <19>: Shebang Lines. (line 48) +* PATH <20>: Shebang Lines. (line 50) +* PATH <21>: Process Management. (line 45) +* PATH <22>: Process Management. (line 86) +* PATH <23>: Process Management. (line 89) +* PATH <24>: Process Management. (line 92) +* PATH <25>: Process Management. (line 453) +* PATH <26>: Process Management. (line 539) +* PATH <27>: Process Management. (line 543) +* PATH <28>: Process Management. (line 545) +* PATH <29>: Miscellaneous System Information. + (line 118) +* PATH <30>: webbrowser — Convenient Web-browser controller. + (line 214) +* PATH <31>: Installing your CGI script on a Unix system. + (line 29) +* PATH <32>: Common problems and solutions. + (line 23) +* PATH <33>: site — Site-specific configuration hook. + (line 59) +* PATH <34>: Embedding Python<2>. + (line 31) +* PATH <35>: Embedding Python<2>. + (line 37) +* PATH <36>: How do I make a Python script executable on Unix?. + (line 24) +* PATH <37>: How do I make a Python script executable on Unix?. + (line 28) +* Path (class in pathlib): Concrete paths. (line 11) +* Path (class in zipfile): Path Objects. (line 6) +* path (http.cookiejar.Cookie attribute): Cookie Objects<2>. (line 41) +* path (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 92) +* path (importlib.abc.FileLoader attribute): importlib abc – Abstract base classes related to import. + (line 488) +* path (importlib.machinery.ExtensionFileLoader attribute): importlib machinery – Importers and path hooks. + (line 315) +* path (importlib.machinery.FileFinder attribute): importlib machinery – Importers and path hooks. + (line 186) +* path (importlib.machinery.SourceFileLoader attribute): importlib machinery – Importers and path hooks. + (line 228) +* path (importlib.machinery.SourcelessFileLoader attribute): importlib machinery – Importers and path hooks. + (line 272) +* path (in module sys): sys — System-specific parameters and functions. + (line 1059) +* path (in module sys) <1>: Embedding Python<2>. + (line 12) +* path (in module sys) <2>: Initializing and finalizing the interpreter. + (line 8) +* path (in module sys) <3>: Process-wide parameters. + (line 120) +* path (in module sys) <4>: Process-wide parameters. + (line 133) +* path (os.DirEntry attribute): Files and Directories. + (line 857) +* path based finder: The Path Based Finder. + (line 6) +* path based finder <1>: Glossary. (line 1008) +* Path browser: File menu Shell and Editor. + (line 22) +* path entry: Glossary. (line 988) +* path entry finder: Glossary. (line 993) +* path entry hook: Glossary. (line 1002) +* path hooks: Import hooks. (line 6) +* path() (in module importlib.resources): importlib resources – Resources. + (line 105) +* path-like object: Glossary. (line 1013) +* path; configuration; file: site — Site-specific configuration hook. + (line 64) +* path; operations: pathlib — Object-oriented filesystem paths. + (line 10) +* path; operations <1>: os path — Common pathname manipulations. + (line 9) +* pathconf() (in module os): Files and Directories. + (line 548) +* pathconf_names (in module os): Files and Directories. + (line 572) +* PathEntryFinder (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 96) +* PATHEXT: Other Improvements<2>. + (line 30) +* PATHEXT <1>: Installing Without UI. + (line 58) +* PathFinder (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 107) +* pathlib (module): pathlib — Object-oriented filesystem paths. + (line 6) +* PathLike (class in os): Process Parameters. (line 113) +* pathname2url() (in module urllib.request): urllib request — Extensible library for opening URLs. + (line 147) +* pathsep (in module os): Miscellaneous System Information. + (line 115) +* path_hook() (importlib.machinery.FileFinder class method): importlib machinery – Importers and path hooks. + (line 206) +* path_hooks (in module sys): sys — System-specific parameters and functions. + (line 1084) +* path_importer_cache (in module sys): sys — System-specific parameters and functions. + (line 1093) +* path_mtime() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 554) +* path_return_ok() (http.cookiejar.CookiePolicy method): CookiePolicy Objects. + (line 56) +* path_stats() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 532) +* path_stats() (importlib.machinery.SourceFileLoader method): importlib machinery – Importers and path hooks. + (line 237) +* Pattern (class in typing): Classes functions and decorators. + (line 479) +* pattern (re.error attribute): Module Contents. (line 366) +* pattern (re.Pattern attribute): Regular Expression Objects. + (line 118) +* pause() (in module signal): Module contents<2>. (line 265) +* pause_reading() (asyncio.ReadTransport method): Read-only Transports. + (line 12) +* pause_writing() (asyncio.BaseProtocol method): Base Protocol. + (line 40) +* PAX_FORMAT (in module tarfile): tarfile — Read and write tar archive files. + (line 230) +* pax_headers (tarfile.TarFile attribute): TarFile Objects. (line 253) +* pax_headers (tarfile.TarInfo attribute): TarInfo Objects. (line 88) +* pbkdf2_hmac() (in module hashlib): Key derivation. (line 11) +* pd() (in module turtle): Drawing state. (line 6) +* Pdb (class in pdb): pdb — The Python Debugger. + (line 17) +* Pdb (class in pdb) <1>: pdb — The Python Debugger. + (line 146) +* pdb (module): pdb — The Python Debugger. + (line 6) +* pdf() (statistics.NormalDist method): NormalDist objects. (line 68) +* peek() (bz2.BZ2File method): De compression of files. + (line 77) +* peek() (gzip.GzipFile method): gzip — Support for gzip files. + (line 125) +* peek() (io.BufferedReader method): Buffered Streams. (line 73) +* peek() (lzma.LZMAFile method): Reading and writing compressed files. + (line 85) +* peek() (weakref.finalize method): weakref — Weak references. + (line 265) +* peer (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 95) +* PEM_cert_to_DER_cert() (in module ssl): Certificate handling. + (line 96) +* pen() (in module turtle): Drawing state. (line 33) +* pencolor() (in module turtle): Color control. (line 6) +* pending (ssl.MemoryBIO attribute): Memory BIO Support<2>. + (line 139) +* pending() (ssl.SSLSocket method): SSL Sockets. (line 313) +* PendingDeprecationWarning: Warnings. (line 22) +* pendown() (in module turtle): Drawing state. (line 6) +* pensize() (in module turtle): Drawing state. (line 18) +* penup() (in module turtle): Drawing state. (line 12) +* PEP: Glossary. (line 1025) +* PERCENT (in module token): token — Constants used with Python parse trees. + (line 122) +* PERCENTEQUAL (in module token): token — Constants used with Python parse trees. + (line 186) +* Performance: timeit — Measure execution time of small code snippets. + (line 8) +* perf_counter() (in module time): Functions<2>. (line 180) +* perf_counter_ns() (in module time): Functions<2>. (line 191) +* perm() (in module math): Number-theoretic and representation functions. + (line 177) +* PermissionError: OS exceptions. (line 92) +* permutations() (in module itertools): Itertool functions. (line 400) +* Persist() (msilib.SummaryInformation method): Summary Information Objects. + (line 29) +* persistence: pickle — Python object serialization. + (line 8) +* persistent; objects: pickle — Python object serialization. + (line 8) +* persistent_id (pickle protocol): Persistence of External Objects. + (line 6) +* persistent_id() (pickle.Pickler method): Module Interface. (line 160) +* persistent_load (pickle protocol): Persistence of External Objects. + (line 6) +* persistent_load() (pickle.Unpickler method): Module Interface. + (line 274) +* pformat() (in module pprint): pprint — Data pretty printer. + (line 81) +* pformat() (pprint.PrettyPrinter method): PrettyPrinter Objects. + (line 8) +* PF_CAN (in module socket): Constants<8>. (line 93) +* PF_PACKET (in module socket): Constants<8>. (line 144) +* PF_RDS (in module socket): Constants<8>. (line 154) +* pgettext() (gettext.GNUTranslations method): The GNUTranslations class. + (line 68) +* pgettext() (gettext.NullTranslations method): The NullTranslations class. + (line 47) +* pgettext() (in module gettext): GNU gettext API. (line 73) +* PGO (in module test.support): test support — Utilities for the Python test suite. + (line 91) +* phase() (in module cmath): Conversions to and from polar coordinates. + (line 22) +* Philbrick, Geoff: Keyword Parameters for Extension Functions. + (line 25) +* physical line: Logical lines. (line 6) +* physical line <1>: Explicit line joining. + (line 6) +* physical line <2>: String and Bytes literals. + (line 72) +* pi (in module cmath): Constants<3>. (line 6) +* pi (in module math): Constants<2>. (line 6) +* pi() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 56) +* pickle (module): pickle — Python object serialization. + (line 6) +* pickle() (in module copyreg): copyreg — Register pickle support functions. + (line 23) +* PickleBuffer (class in pickle): Module Interface. (line 301) +* PickleError: Module Interface. (line 95) +* Pickler (class in pickle): Module Interface. (line 121) +* pickletools (module): pickletools — Tools for pickle developers. + (line 6) +* pickletools command line option; -a: Command line options<3>. + (line 6) +* pickletools command line option; –annotate: Command line options<3>. + (line 6) +* pickletools command line option; –indentlevel=: Command line options<3>. + (line 14) +* pickletools command line option; -l: Command line options<3>. + (line 14) +* pickletools command line option; -m: Command line options<3>. + (line 18) +* pickletools command line option; –memo: Command line options<3>. + (line 18) +* pickletools command line option; -o: Command line options<3>. + (line 10) +* pickletools command line option; –output=: Command line options<3>. + (line 10) +* pickletools command line option; -p: Command line options<3>. + (line 23) +* pickletools command line option; –preamble=: Command line options<3>. + (line 23) +* pickling; objects: pickle — Python object serialization. + (line 8) +* PicklingError: Module Interface. (line 100) +* pid (asyncio.asyncio.subprocess.Process attribute): Interacting with Subprocesses. + (line 132) +* pid (multiprocessing.Process attribute): Process and exceptions. + (line 107) +* pid (subprocess.Popen attribute): Popen Objects. (line 138) +* PIPE (in module subprocess): Using the subprocess Module. + (line 140) +* Pipe() (in module multiprocessing): Pipes and Queues. (line 74) +* pipe() (in module os): File Descriptor Operations. + (line 365) +* pipe2() (in module os): File Descriptor Operations. + (line 376) +* pipes (module): pipes — Interface to shell pipelines. + (line 6) +* PIPE_BUF (in module select): select — Waiting for I/O completion. + (line 157) +* pipe_connection_lost() (asyncio.SubprocessProtocol method): Subprocess Protocols. + (line 19) +* pipe_data_received() (asyncio.SubprocessProtocol method): Subprocess Protocols. + (line 10) +* PIPE_MAX_SIZE (in module test.support): test support — Utilities for the Python test suite. + (line 95) +* pkgutil (module): pkgutil — Package extension utility. + (line 6) +* PKG_DIRECTORY (in module imp): imp — Access the import internals. + (line 327) +* placeholder (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 266) +* PLAT: distutils util — Miscellaneous other utility functions. + (line 79) +* platform (in module sys): sys — System-specific parameters and functions. + (line 1106) +* platform (in module sys) <1>: Process-wide parameters. + (line 172) +* platform (module): platform — Access to underlying platform’s identifying data. + (line 6) +* platform() (in module platform): Cross Platform. (line 46) +* PlaySound() (in module winsound): winsound — Sound-playing interface for Windows. + (line 20) +* plist; file: plistlib — Generate and parse Mac OS X plist files. + (line 8) +* plistlib (module): plistlib — Generate and parse Mac OS X plist files. + (line 6) +* plock() (in module os): Process Management. (line 325) +* plus: Unary arithmetic and bitwise operations. + (line 13) +* PLUS (in module token): token — Constants used with Python parse trees. + (line 82) +* plus() (decimal.Context method): Context objects. (line 412) +* PLUSEQUAL (in module token): token — Constants used with Python parse trees. + (line 170) +* pm() (in module pdb): pdb — The Python Debugger. + (line 136) +* POINTER() (in module ctypes): Utility functions. (line 149) +* pointer() (in module ctypes): Utility functions. (line 155) +* polar() (in module cmath): Conversions to and from polar coordinates. + (line 42) +* Policy (class in email.policy): email policy Policy Objects. + (line 128) +* poll() (in module select): select — Waiting for I/O completion. + (line 84) +* poll() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 42) +* poll() (select.devpoll method): /dev/poll Polling Objects. + (line 65) +* poll() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 93) +* poll() (select.poll method): Polling Objects. (line 77) +* poll() (subprocess.Popen method): Popen Objects. (line 8) +* PollSelector (class in selectors): Classes<3>. (line 171) +* Pool (class in multiprocessing.pool): Process Pools. (line 9) +* pop() (array.array method): array — Efficient arrays of numeric values. + (line 204) +* pop() (collections.deque method): deque objects. (line 87) +* pop() (dict method): Mapping Types — dict. + (line 178) +* pop() (frozenset method): Set Types — set frozenset. + (line 208) +* pop() (mailbox.Mailbox method): Mailbox objects. (line 208) +* pop() (sequence method): Mutable Sequence Types. + (line 16) +* POP3 (class in poplib): poplib — POP3 protocol client. + (line 29) +* POP3; protocol: poplib — POP3 protocol client. + (line 8) +* POP3_SSL (class in poplib): poplib — POP3 protocol client. + (line 45) +* Popen (class in subprocess): Popen Constructor. (line 11) +* popen() (in module os): The standard type hierarchy. + (line 630) +* popen() (in module os) <1>: Process Management. (line 332) +* popen() (in module os) <2>: select — Waiting for I/O completion. + (line 138) +* popitem() (collections.OrderedDict method): OrderedDict objects. + (line 46) +* popitem() (dict method): Mapping Types — dict. + (line 184) +* popitem() (mailbox.Mailbox method): Mailbox objects. (line 217) +* popleft() (collections.deque method): deque objects. (line 92) +* poplib (module): poplib — POP3 protocol client. + (line 6) +* PopupMenu (class in tkinter.tix): Basic Widgets. (line 53) +* pop_alignment() (formatter.formatter method): The Formatter Interface. + (line 97) +* pop_all() (contextlib.ExitStack method): Utilities. (line 426) +* POP_BLOCK (opcode): Python Bytecode Instructions. + (line 372) +* POP_EXCEPT (opcode): Python Bytecode Instructions. + (line 377) +* POP_FINALLY (opcode): Python Bytecode Instructions. + (line 385) +* pop_font() (formatter.formatter method): The Formatter Interface. + (line 109) +* POP_JUMP_IF_FALSE (opcode): Python Bytecode Instructions. + (line 661) +* POP_JUMP_IF_TRUE (opcode): Python Bytecode Instructions. + (line 654) +* pop_margin() (formatter.formatter method): The Formatter Interface. + (line 121) +* pop_source() (shlex.shlex method): shlex Objects. (line 63) +* pop_style() (formatter.formatter method): The Formatter Interface. + (line 132) +* POP_TOP (opcode): Python Bytecode Instructions. + (line 59) +* port (http.cookiejar.Cookie attribute): Cookie Objects<2>. (line 36) +* portion: Glossary. (line 1041) +* port_specified (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 77) +* pos (json.JSONDecodeError attribute): Exceptions<13>. (line 19) +* pos (re.error attribute): Module Contents. (line 370) +* pos (re.Match attribute): Match Objects. (line 164) +* pos() (in module operator): operator — Standard operators as functions. + (line 138) +* pos() (in module turtle): Tell Turtle’s state. + (line 6) +* position (xml.etree.ElementTree.ParseError attribute): Exceptions<15>. + (line 20) +* position() (in module turtle): Tell Turtle’s state. + (line 6) +* positional argument: Glossary. (line 1047) +* posix (module): posix — The most common POSIX system calls. + (line 6) +* POSIX Shared Memory: multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 10) +* POSIX; I/O control: termios — POSIX style tty control. + (line 6) +* POSIXLY_CORRECT: getopt — C-style parser for command line options. + (line 71) +* PosixPath (class in pathlib): Concrete paths. (line 22) +* posix_fadvise() (in module os): File Descriptor Operations. + (line 396) +* POSIX_FADV_DONTNEED (in module os): File Descriptor Operations. + (line 410) +* POSIX_FADV_NOREUSE (in module os): File Descriptor Operations. + (line 410) +* POSIX_FADV_NORMAL (in module os): File Descriptor Operations. + (line 410) +* POSIX_FADV_RANDOM (in module os): File Descriptor Operations. + (line 410) +* POSIX_FADV_SEQUENTIAL (in module os): File Descriptor Operations. + (line 410) +* POSIX_FADV_WILLNEED (in module os): File Descriptor Operations. + (line 410) +* posix_fallocate() (in module os): File Descriptor Operations. + (line 387) +* posix_spawn() (in module os): Process Management. (line 355) +* posix_spawnp() (in module os): Process Management. (line 445) +* POSIX_SPAWN_CLOSE (in module os): Process Management. (line 383) +* POSIX_SPAWN_DUP2 (in module os): Process Management. (line 389) +* POSIX_SPAWN_OPEN (in module os): Process Management. (line 377) +* post() (nntplib.NNTP method): Methods<3>. (line 278) +* post() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 179) +* postcmd() (cmd.Cmd method): Cmd Objects. (line 96) +* postloop() (cmd.Cmd method): Cmd Objects. (line 114) +* post_handshake_auth (ssl.SSLContext attribute): SSL Contexts. + (line 602) +* post_mortem() (in module pdb): pdb — The Python Debugger. + (line 129) +* post_setup() (venv.EnvBuilder method): API<2>. (line 104) +* pow() (built-in function): Built-in Functions. (line 1271) +* pow() (in module math): Power and logarithmic functions. + (line 60) +* pow() (in module operator): operator — Standard operators as functions. + (line 143) +* power() (decimal.Context method): Context objects. (line 418) +* power; operation: The power operator. (line 6) +* pp (pdb command): Debugger Commands. (line 242) +* pp() (in module pprint): pprint — Data pretty printer. + (line 93) +* pprint (module): pprint — Data pretty printer. + (line 6) +* pprint() (in module pprint): pprint — Data pretty printer. + (line 103) +* pprint() (pprint.PrettyPrinter method): PrettyPrinter Objects. + (line 14) +* prcal() (in module calendar): calendar — General calendar-related functions. + (line 344) +* pread() (in module os): File Descriptor Operations. + (line 424) +* preadv() (in module os): File Descriptor Operations. + (line 437) +* preamble (email.message.EmailMessage attribute): email message Representing an email message. + (line 680) +* preamble (email.message.Message attribute): email message Message Representing an email message using the compat32 API. + (line 690) +* precmd() (cmd.Cmd method): Cmd Objects. (line 86) +* prefix: Python-related paths and files. + (line 7) +* prefix <1>: Include Files. (line 37) +* prefix <2>: Include Files. (line 40) +* prefix <3>: Include Files. (line 47) +* PREFIX (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 19) +* prefix (in module sys): sys — System-specific parameters and functions. + (line 1168) +* prefix (xml.dom.Attr attribute): Attr Objects. (line 18) +* prefix (xml.dom.Node attribute): Node Objects. (line 69) +* prefix (zipimport.zipimporter attribute): zipimporter Objects. + (line 74) +* PREFIXES (in module site): Module contents<3>. (line 6) +* prefixlen (ipaddress.IPv4Network attribute): Network objects. + (line 132) +* prefixlen (ipaddress.IPv6Network attribute): Network objects. + (line 326) +* preloop() (cmd.Cmd method): Cmd Objects. (line 108) +* prepare() (logging.handlers.QueueHandler method): QueueHandler. + (line 40) +* prepare() (logging.handlers.QueueListener method): QueueListener. + (line 54) +* prepare_class() (in module types): Dynamic Type Creation. + (line 23) +* prepare_input_source() (in module xml.sax.saxutils): xml sax saxutils — SAX Utilities. + (line 77) +* prepend() (pipes.Template method): Template Objects. (line 37) +* preprocess() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 433) +* PrettyPrinter (class in pprint): pprint — Data pretty printer. + (line 27) +* prev() (tkinter.ttk.Treeview method): ttk Treeview. (line 242) +* previousSibling (xml.dom.Node attribute): Node Objects. (line 32) +* primary: Primaries. (line 6) +* print (2to3 fixer): Fixers. (line 257) +* print() (built-in function): Built-in Functions. (line 1306) +* print() (built-in function); __str__() (object method): Basic customization. + (line 129) +* printable (in module string): String constants. (line 41) +* printdir() (zipfile.ZipFile method): ZipFile Objects. (line 207) +* printf-style formatting: printf-style String Formatting. + (line 6) +* printf-style formatting <1>: printf-style Bytes Formatting. + (line 6) +* print_callees() (pstats.Stats method): The Stats Class. (line 217) +* print_callers() (pstats.Stats method): The Stats Class. (line 196) +* print_directory() (in module cgi): Functions<8>. (line 60) +* print_environ() (in module cgi): Functions<8>. (line 52) +* print_environ_usage() (in module cgi): Functions<8>. (line 64) +* print_exc() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 60) +* print_exc() (timeit.Timer method): Python Interface. (line 133) +* print_exception() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 33) +* PRINT_EXPR (opcode): Python Bytecode Instructions. + (line 311) +* print_form() (in module cgi): Functions<8>. (line 56) +* print_help() (argparse.ArgumentParser method): Printing help. + (line 16) +* print_last() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 65) +* print_stack() (asyncio.Task method): Task Object. (line 190) +* print_stack() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 72) +* print_stats() (profile.Profile method): profile and cProfile Module Reference. + (line 91) +* print_stats() (pstats.Stats method): The Stats Class. (line 164) +* print_tb() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 22) +* print_usage() (argparse.ArgumentParser method): Printing help. + (line 10) +* print_usage() (optparse.OptionParser method): Other methods. + (line 15) +* print_version() (optparse.OptionParser method): Printing a version string. + (line 26) +* PriorityQueue (class in asyncio): Priority Queue. (line 6) +* PriorityQueue (class in queue): queue — A synchronized queue class. + (line 50) +* PRIO_PGRP (in module os): Process Parameters. (line 271) +* PRIO_PROCESS (in module os): Process Parameters. (line 271) +* PRIO_USER (in module os): Process Parameters. (line 271) +* private; names: Identifiers Names. (line 14) +* prlimit() (in module resource): Resource Limits. (line 56) +* prmonth() (calendar.TextCalendar method): calendar — General calendar-related functions. + (line 143) +* prmonth() (in module calendar): calendar — General calendar-related functions. + (line 335) +* ProactorEventLoop (class in asyncio): Event Loop Implementations. + (line 29) +* procedure; call: Expression statements. + (line 17) +* Process (class in multiprocessing): Process and exceptions. + (line 6) +* process() (logging.LoggerAdapter method): LoggerAdapter Objects. + (line 16) +* process; group: Process Parameters. (line 179) +* process; group <1>: Process Parameters. (line 237) +* process; id: Process Parameters. (line 243) +* process; id of parent: Process Parameters. (line 247) +* process; killing: Process Management. (line 289) +* process; killing <1>: Process Management. (line 311) +* process; scheduling priority: Process Parameters. (line 258) +* process; scheduling priority <1>: Process Parameters. (line 389) +* process; signalling: Process Management. (line 289) +* process; signalling <1>: Process Management. (line 311) +* ProcessError: Process and exceptions. + (line 204) +* processes, light-weight: _thread — Low-level threading API. + (line 6) +* ProcessingInstruction() (in module xml.etree.ElementTree): Functions<6>. + (line 167) +* processingInstruction() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 164) +* ProcessingInstructionHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 236) +* ProcessLookupError: OS exceptions. (line 98) +* processor time: Functions<2>. (line 200) +* processor time <1>: Functions<2>. (line 479) +* processor() (in module platform): Cross Platform. (line 67) +* ProcessPoolExecutor (class in concurrent.futures): ProcessPoolExecutor. + (line 21) +* process_exited() (asyncio.SubprocessProtocol method): Subprocess Protocols. + (line 26) +* process_message() (smtpd.SMTPServer method): SMTPServer Objects. + (line 41) +* process_request() (socketserver.BaseServer method): Server Objects<2>. + (line 156) +* process_time() (in module time): Functions<2>. (line 198) +* process_time_ns() (in module time): Functions<2>. (line 208) +* process_tokens() (in module tabnanny): tabnanny — Detection of ambiguous indentation. + (line 42) +* prod() (in module math): Number-theoretic and representation functions. + (line 194) +* product() (in module itertools): Itertool functions. (line 461) +* Profile (class in profile): profile and cProfile Module Reference. + (line 35) +* profile (module): profile and cProfile Module Reference. + (line 6) +* profile function: threading — Thread-based parallelism. + (line 132) +* profile function <1>: sys — System-specific parameters and functions. + (line 741) +* profile function <2>: sys — System-specific parameters and functions. + (line 1227) +* profiler: sys — System-specific parameters and functions. + (line 741) +* profiler <1>: sys — System-specific parameters and functions. + (line 1227) +* profiling, deterministic: Introduction to the profilers. + (line 6) +* program: Complete Python programs. + (line 6) +* ProgrammingError: Exceptions<4>. (line 25) +* Progressbar (class in tkinter.ttk): ttk Progressbar. (line 6) +* prompt (cmd.Cmd attribute): Cmd Objects. (line 123) +* prompts, interpreter: sys — System-specific parameters and functions. + (line 1188) +* prompt_user_passwd() (urllib.request.FancyURLopener method): Legacy interface. + (line 184) +* propagate (logging.Logger attribute): Logger Objects. (line 25) +* property (built-in class): Built-in Functions. (line 1331) +* property list: plistlib — Generate and parse Mac OS X plist files. + (line 8) +* PropertyMock (class in unittest.mock): The Mock Class. (line 627) +* property_declaration_handler (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 101) +* property_dom_node (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 108) +* property_lexical_handler (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 94) +* property_xml_string (in module xml.sax.handler): xml sax handler — Base classes for SAX handlers. + (line 115) +* proto (socket.socket attribute): Socket Objects. (line 603) +* Protocol (class in asyncio): Base Protocols. (line 10) +* Protocol (class in typing): Classes functions and decorators. + (line 71) +* protocol (ssl.SSLContext attribute): SSL Contexts. (line 625) +* protocol; context management: Context Manager Types. + (line 6) +* protocol; iterator: Iterator Types. (line 6) +* protocol; Telnet: telnetlib — Telnet client. + (line 8) +* ProtocolError (class in xmlrpc.client): ProtocolError Objects. + (line 6) +* PROTOCOL_SSLv2 (in module ssl): Constants<9>. (line 150) +* PROTOCOL_SSLv23 (in module ssl): Constants<9>. (line 143) +* PROTOCOL_SSLv3 (in module ssl): Constants<9>. (line 163) +* PROTOCOL_TLS (in module ssl): Constants<9>. (line 118) +* PROTOCOL_TLSv1 (in module ssl): Constants<9>. (line 177) +* PROTOCOL_TLSv1_1 (in module ssl): Constants<9>. (line 185) +* PROTOCOL_TLSv1_2 (in module ssl): Constants<9>. (line 196) +* PROTOCOL_TLS_CLIENT (in module ssl): Constants<9>. (line 126) +* PROTOCOL_TLS_SERVER (in module ssl): Constants<9>. (line 135) +* protocol_version (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 154) +* PROTOCOL_VERSION (imaplib.IMAP4 attribute): IMAP4 Objects. (line 376) +* prot_c() (ftplib.FTP_TLS method): FTP_TLS Objects. (line 35) +* prot_p() (ftplib.FTP_TLS method): FTP_TLS Objects. (line 31) +* provisional API: Glossary. (line 1051) +* provisional package: Glossary. (line 1071) +* proxy() (in module weakref): weakref — Weak references. + (line 131) +* proxyauth() (imaplib.IMAP4 method): IMAP4 Objects. (line 187) +* ProxyBasicAuthHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 364) +* ProxyDigestAuthHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 396) +* ProxyHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 290) +* ProxyType (in module weakref): weakref — Weak references. + (line 294) +* ProxyTypes (in module weakref): weakref — Weak references. + (line 302) +* pryear() (calendar.TextCalendar method): calendar — General calendar-related functions. + (line 158) +* ps1 (in module sys): sys — System-specific parameters and functions. + (line 1185) +* ps2 (in module sys): sys — System-specific parameters and functions. + (line 1185) +* pstats (module): The Stats Class. (line 7) +* pstdev() (in module statistics): Function details. (line 271) +* pthreads: _thread — Low-level threading API. + (line 16) +* pthread_getcpuclockid() (in module time): Functions<2>. (line 21) +* pthread_kill() (in module signal): Module contents<2>. (line 282) +* pthread_sigmask() (in module signal): Module contents<2>. (line 311) +* pty (module): pty — Pseudo-terminal utilities. + (line 6) +* pu() (in module turtle): Drawing state. (line 12) +* publicId (xml.dom.DocumentType attribute): DocumentType Objects. + (line 17) +* PullDom (class in xml.dom.pulldom): xml dom pulldom — Support for building partial DOM trees. + (line 78) +* punctuation (in module string): String constants. (line 36) +* punctuation_chars (shlex.shlex attribute): shlex Objects. (line 183) +* PurePath (class in pathlib): Pure paths. (line 10) +* PurePath.anchor (in module pathlib): Methods and properties. + (line 40) +* PurePath.drive (in module pathlib): Methods and properties. + (line 8) +* PurePath.name (in module pathlib): Methods and properties. + (line 94) +* PurePath.parent (in module pathlib): Methods and properties. + (line 66) +* PurePath.parents (in module pathlib): Methods and properties. + (line 53) +* PurePath.parts (in module pathlib): Accessing individual parts. + (line 9) +* PurePath.root (in module pathlib): Methods and properties. + (line 24) +* PurePath.stem (in module pathlib): Methods and properties. + (line 131) +* PurePath.suffix (in module pathlib): Methods and properties. + (line 109) +* PurePath.suffixes (in module pathlib): Methods and properties. + (line 120) +* PurePosixPath (class in pathlib): Pure paths. (line 68) +* PureProxy (class in smtpd): PureProxy Objects. (line 6) +* PureWindowsPath (class in pathlib): Pure paths. (line 78) +* purge() (in module re): Module Contents. (line 349) +* Purpose.CLIENT_AUTH (in module ssl): Constants<9>. (line 502) +* Purpose.SERVER_AUTH (in module ssl): Constants<9>. (line 493) +* push() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 101) +* push() (code.InteractiveConsole method): Interactive Console Objects. + (line 28) +* push() (contextlib.ExitStack method): Utilities. (line 393) +* pushbutton() (msilib.Dialog method): GUI classes. (line 68) +* push_alignment() (formatter.formatter method): The Formatter Interface. + (line 90) +* push_async_callback() (contextlib.AsyncExitStack method): Utilities. + (line 473) +* push_async_exit() (contextlib.AsyncExitStack method): Utilities. + (line 468) +* push_font() (formatter.formatter method): The Formatter Interface. + (line 101) +* push_margin() (formatter.formatter method): The Formatter Interface. + (line 113) +* push_source() (shlex.shlex method): shlex Objects. (line 56) +* push_style() (formatter.formatter method): The Formatter Interface. + (line 125) +* push_token() (shlex.shlex method): shlex Objects. (line 16) +* push_with_producer() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 109) +* put() (asyncio.Queue method): Queue. (line 62) +* put() (multiprocessing.Queue method): Pipes and Queues. (line 120) +* put() (multiprocessing.SimpleQueue method): Pipes and Queues. + (line 217) +* put() (queue.Queue method): Queue Objects. (line 29) +* put() (queue.SimpleQueue method): SimpleQueue Objects. + (line 20) +* putch() (in module msvcrt): Console I/O. (line 35) +* putenv() (in module os): Process Parameters. (line 316) +* putheader() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 136) +* putp() (in module curses): Functions<3>. (line 376) +* putrequest() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 125) +* putwch() (in module msvcrt): Console I/O. (line 39) +* putwin() (curses.window method): Window Objects. (line 453) +* put_nowait() (asyncio.Queue method): Queue. (line 67) +* put_nowait() (multiprocessing.Queue method): Pipes and Queues. + (line 136) +* put_nowait() (queue.Queue method): Queue Objects. (line 40) +* put_nowait() (queue.SimpleQueue method): SimpleQueue Objects. + (line 35) +* pvariance() (in module statistics): Function details. (line 280) +* pwd (module): pwd — The password database. + (line 6) +* pwd() (ftplib.FTP method): FTP Objects. (line 221) +* pwrite() (in module os): File Descriptor Operations. + (line 495) +* pwritev() (in module os): File Descriptor Operations. + (line 506) +* PyAnySet_Check (C function): Set Objects. (line 55) +* PyAnySet_CheckExact (C function): Set Objects. (line 60) +* PyArg_Parse (C function): API Functions. (line 51) +* PyArg_ParseTuple (C function): API Functions. (line 6) +* PyArg_ParseTuple(): Extracting Parameters in Extension Functions. + (line 6) +* PyArg_ParseTupleAndKeywords (C function): API Functions. (line 20) +* PyArg_ParseTupleAndKeywords(): Keyword Parameters for Extension Functions. + (line 6) +* PyArg_UnpackTuple (C function): API Functions. (line 63) +* PyArg_ValidateKeywordArguments (C function): API Functions. + (line 42) +* PyArg_VaParse (C function): API Functions. (line 13) +* PyArg_VaParseTupleAndKeywords (C function): API Functions. (line 34) +* PyASCIIObject (C type): Unicode Type. (line 29) +* PyAsyncMethods (C type): Async Object Structures. + (line 8) +* PyAsyncMethods.am_aiter (C member): Async Object Structures. + (line 34) +* PyAsyncMethods.am_anext (C member): Async Object Structures. + (line 46) +* PyAsyncMethods.am_await (C member): Async Object Structures. + (line 22) +* PyBool_Check (C function): Boolean Objects. (line 11) +* PyBool_FromLong (C function): Boolean Objects. (line 37) +* PyBufferProcs: Buffer Protocol. (line 20) +* PyBufferProcs (C type): Buffer Object Structures. + (line 6) +* PyBufferProcs.bf_getbuffer (C member): Buffer Object Structures. + (line 12) +* PyBufferProcs.bf_releasebuffer (C member): Buffer Object Structures. + (line 60) +* PyBuffer_FillContiguousStrides (C function): Buffer-related functions. + (line 77) +* PyBuffer_FillInfo (C function): Buffer-related functions. + (line 87) +* PyBuffer_FromContiguous (C function): Buffer-related functions. + (line 59) +* PyBuffer_GetPointer (C function): Buffer-related functions. + (line 53) +* PyBuffer_IsContiguous (C function): Buffer-related functions. + (line 46) +* PyBuffer_Release (C function): Buffer-related functions. + (line 32) +* PyBuffer_SizeFromFormat (C function): Buffer-related functions. + (line 41) +* PyBuffer_ToContiguous (C function): Buffer-related functions. + (line 66) +* PyBUF_ANY_CONTIGUOUS (C macro): contiguity requests. + (line 20) +* PyBUF_CONTIG (C macro): compound requests. (line 36) +* PyBUF_CONTIG_RO (C macro): compound requests. (line 39) +* PyBUF_C_CONTIGUOUS (C macro): contiguity requests. + (line 14) +* PyBUF_FORMAT (C macro): readonly format. (line 14) +* PyBUF_FULL (C macro): compound requests. (line 18) +* PyBUF_FULL_RO (C macro): compound requests. (line 21) +* PyBUF_F_CONTIGUOUS (C macro): contiguity requests. + (line 17) +* PyBUF_INDIRECT (C macro): shape strides suboffsets. + (line 14) +* PyBUF_ND (C macro): shape strides suboffsets. + (line 20) +* PyBUF_RECORDS (C macro): compound requests. (line 24) +* PyBUF_RECORDS_RO (C macro): compound requests. (line 27) +* PyBUF_SIMPLE (C macro): shape strides suboffsets. + (line 23) +* PyBUF_STRIDED (C macro): compound requests. (line 30) +* PyBUF_STRIDED_RO (C macro): compound requests. (line 33) +* PyBUF_STRIDES (C macro): shape strides suboffsets. + (line 17) +* PyBUF_WRITABLE (C macro): readonly format. (line 6) +* PyByteArrayObject (C type): Byte Array Objects. (line 6) +* PyByteArray_AsString (C function): Direct API functions. + (line 23) +* PyByteArray_AS_STRING (C function): Macros. (line 8) +* PyByteArray_Check (C function): Type check macros. (line 6) +* PyByteArray_CheckExact (C function): Type check macros. (line 11) +* PyByteArray_Concat (C function): Direct API functions. + (line 15) +* PyByteArray_FromObject (C function): Direct API functions. + (line 6) +* PyByteArray_FromStringAndSize (C function): Direct API functions. + (line 10) +* PyByteArray_GET_SIZE (C function): Macros. (line 12) +* PyByteArray_Resize (C function): Direct API functions. + (line 29) +* PyByteArray_Size (C function): Direct API functions. + (line 19) +* PyByteArray_Type (C variable): Byte Array Objects. (line 11) +* PyBytesObject (C type): Bytes Objects<2>. (line 9) +* PyBytes_AsString (C function): Bytes Objects<2>. (line 126) +* PyBytes_AsStringAndSize (C function): Bytes Objects<2>. (line 143) +* PyBytes_AS_STRING (C function): Bytes Objects<2>. (line 138) +* PyBytes_Check (C function): Bytes Objects<2>. (line 20) +* PyBytes_CheckExact (C function): Bytes Objects<2>. (line 25) +* PyBytes_Concat (C function): Bytes Objects<2>. (line 165) +* PyBytes_ConcatAndDel (C function): Bytes Objects<2>. (line 175) +* PyBytes_FromFormat (C function): Bytes Objects<2>. (line 43) +* PyBytes_FromFormatV (C function): Bytes Objects<2>. (line 106) +* PyBytes_FromObject (C function): Bytes Objects<2>. (line 113) +* PyBytes_FromString (C function): Bytes Objects<2>. (line 30) +* PyBytes_FromStringAndSize (C function): Bytes Objects<2>. (line 35) +* PyBytes_GET_SIZE (C function): Bytes Objects<2>. (line 121) +* PyBytes_Size (C function): Bytes Objects<2>. (line 117) +* PyBytes_Type (C variable): Bytes Objects<2>. (line 14) +* pycache_prefix (in module sys): sys — System-specific parameters and functions. + (line 288) +* PyCallable_Check (C function): Object Protocol. (line 249) +* PyCallIter_Check (C function): Iterator Objects. (line 34) +* PyCallIter_New (C function): Iterator Objects. (line 38) +* PyCallIter_Type (C variable): Iterator Objects. (line 28) +* PyCapsule (C type): Capsules<2>. (line 11) +* PyCapsule_CheckExact (C function): Capsules<2>. (line 29) +* PyCapsule_Destructor (C type): Capsules<2>. (line 20) +* PyCapsule_GetContext (C function): Capsules<2>. (line 76) +* PyCapsule_GetDestructor (C function): Capsules<2>. (line 65) +* PyCapsule_GetName (C function): Capsules<2>. (line 86) +* PyCapsule_GetPointer (C function): Capsules<2>. (line 53) +* PyCapsule_Import (C function): Capsules<2>. (line 96) +* PyCapsule_IsValid (C function): Capsules<2>. (line 110) +* PyCapsule_New (C function): Capsules<2>. (line 33) +* PyCapsule_SetContext (C function): Capsules<2>. (line 126) +* PyCapsule_SetDestructor (C function): Capsules<2>. (line 134) +* PyCapsule_SetName (C function): Capsules<2>. (line 143) +* PyCapsule_SetPointer (C function): Capsules<2>. (line 153) +* PyCellObject (C type): Cell Objects. (line 16) +* PyCell_Check (C function): Cell Objects. (line 24) +* PyCell_Get (C function): Cell Objects. (line 32) +* PyCell_GET (C function): Cell Objects. (line 36) +* PyCell_New (C function): Cell Objects. (line 28) +* PyCell_Set (C function): Cell Objects. (line 41) +* PyCell_SET (C function): Cell Objects. (line 48) +* PyCell_Type (C variable): Cell Objects. (line 20) +* PyCFunction (C type): Common Object Structures. + (line 95) +* PyCFunctionWithKeywords (C type): Common Object Structures. + (line 104) +* PycInvalidationMode (class in py_compile): py_compile — Compile Python source files. + (line 91) +* pyclbr (module): pyclbr — Python module browser support. + (line 6) +* PyCodec_BackslashReplaceErrors (C function): Registry API for Unicode encoding error handlers. + (line 50) +* PyCodec_Decode (C function): Codec registry and support functions. + (line 29) +* PyCodec_Decoder (C function): Codec lookup API. (line 15) +* PyCodec_Encode (C function): Codec registry and support functions. + (line 19) +* PyCodec_Encoder (C function): Codec lookup API. (line 11) +* PyCodec_IgnoreErrors (C function): Registry API for Unicode encoding error handlers. + (line 37) +* PyCodec_IncrementalDecoder (C function): Codec lookup API. (line 24) +* PyCodec_IncrementalEncoder (C function): Codec lookup API. (line 19) +* PyCodec_KnownEncoding (C function): Codec registry and support functions. + (line 14) +* PyCodec_LookupError (C function): Registry API for Unicode encoding error handlers. + (line 28) +* PyCodec_NameReplaceErrors (C function): Registry API for Unicode encoding error handlers. + (line 54) +* PyCodec_Register (C function): Codec registry and support functions. + (line 6) +* PyCodec_RegisterError (C function): Registry API for Unicode encoding error handlers. + (line 6) +* PyCodec_ReplaceErrors (C function): Registry API for Unicode encoding error handlers. + (line 41) +* PyCodec_StreamReader (C function): Codec lookup API. (line 29) +* PyCodec_StreamWriter (C function): Codec lookup API. (line 35) +* PyCodec_StrictErrors (C function): Registry API for Unicode encoding error handlers. + (line 34) +* PyCodec_XMLCharRefReplaceErrors (C function): Registry API for Unicode encoding error handlers. + (line 45) +* PyCodeObject (C type): Code Objects<2>. (line 10) +* PyCode_Check (C function): Code Objects<2>. (line 20) +* PyCode_GetNumFree (C function): Code Objects<2>. (line 24) +* PyCode_New (C function): Code Objects<2>. (line 28) +* PyCode_NewEmpty (C function): Code Objects<2>. (line 56) +* PyCode_NewWithPosOnlyArgs (C function): Code Objects<2>. (line 42) +* PyCode_Type (C variable): Code Objects<2>. (line 15) +* PyCompactUnicodeObject (C type): Unicode Type. (line 29) +* PyCompileError: py_compile — Compile Python source files. + (line 19) +* PyCompilerFlags (C type): The Very High Level Layer. + (line 396) +* PyCompilerFlags.cf_feature_version (C member): The Very High Level Layer. + (line 412) +* PyCompilerFlags.cf_flags (C member): The Very High Level Layer. + (line 408) +* PyComplexObject (C type): Complex Numbers as Python Objects. + (line 6) +* PyComplex_AsCComplex (C function): Complex Numbers as Python Objects. + (line 44) +* PyComplex_Check (C function): Complex Numbers as Python Objects. + (line 17) +* PyComplex_CheckExact (C function): Complex Numbers as Python Objects. + (line 22) +* PyComplex_FromCComplex (C function): Complex Numbers as Python Objects. + (line 27) +* PyComplex_FromDoubles (C function): Complex Numbers as Python Objects. + (line 31) +* PyComplex_ImagAsDouble (C function): Complex Numbers as Python Objects. + (line 40) +* PyComplex_RealAsDouble (C function): Complex Numbers as Python Objects. + (line 36) +* PyComplex_Type (C variable): Complex Numbers as Python Objects. + (line 11) +* PyConfig (C type): PyConfig. (line 6) +* PyConfig.argv (C member): PyConfig. (line 99) +* PyConfig.base_executable (C member): PyConfig. (line 111) +* PyConfig.base_exec_prefix (C member): PyConfig. (line 107) +* PyConfig.base_prefix (C member): PyConfig. (line 116) +* PyConfig.buffered_stdio (C member): PyConfig. (line 120) +* PyConfig.bytes_warning (C member): PyConfig. (line 127) +* PyConfig.check_hash_pycs_mode (C member): PyConfig. (line 134) +* PyConfig.configure_c_stdio (C member): PyConfig. (line 144) +* PyConfig.dev_mode (C member): PyConfig. (line 150) +* PyConfig.dump_refs (C member): PyConfig. (line 154) +* PyConfig.executable (C member): PyConfig. (line 165) +* PyConfig.exec_prefix (C member): PyConfig. (line 161) +* PyConfig.faulthandler (C member): PyConfig. (line 169) +* PyConfig.filesystem_encoding (C member): PyConfig. (line 174) +* PyConfig.filesystem_errors (C member): PyConfig. (line 178) +* PyConfig.hash_seed (C member): PyConfig. (line 183) +* PyConfig.home (C member): PyConfig. (line 193) +* PyConfig.import_time (C member): PyConfig. (line 200) +* PyConfig.inspect (C member): PyConfig. (line 204) +* PyConfig.install_signal_handlers (C member): PyConfig. (line 208) +* PyConfig.interactive (C member): PyConfig. (line 212) +* PyConfig.isolated (C member): PyConfig. (line 216) +* PyConfig.legacy_windows_stdio (C member): PyConfig. (line 230) +* PyConfig.malloc_stats (C member): PyConfig. (line 239) +* PyConfig.module_search_paths (C member): PyConfig. (line 255) +* PyConfig.module_search_paths_set (C member): PyConfig. (line 257) +* PyConfig.optimization_level (C member): PyConfig. (line 264) +* PyConfig.parser_debug (C member): PyConfig. (line 280) +* PyConfig.parse_argv (C member): PyConfig. (line 274) +* PyConfig.pathconfig_warnings (C member): PyConfig. (line 285) +* PyConfig.prefix (C member): PyConfig. (line 291) +* PyConfig.program_name (C member): PyConfig. (line 295) +* PyConfig.pycache_prefix (C member): PyConfig. (line 300) +* PyConfig.pythonpath_env (C member): PyConfig. (line 247) +* PyConfig.quiet (C member): PyConfig. (line 306) +* PyConfig.run_command (C member): PyConfig. (line 311) +* PyConfig.run_filename (C member): PyConfig. (line 316) +* PyConfig.run_module (C member): PyConfig. (line 320) +* PyConfig.show_alloc_count (C member): PyConfig. (line 325) +* PyConfig.show_ref_count (C member): PyConfig. (line 333) +* PyConfig.site_import (C member): PyConfig. (line 342) +* PyConfig.skip_source_first_line (C member): PyConfig. (line 346) +* PyConfig.stdio_encoding (C member): PyConfig. (line 350) +* PyConfig.stdio_errors (C member): PyConfig. (line 352) +* PyConfig.tracemalloc (C member): PyConfig. (line 357) +* PyConfig.user_site_directory (C member): PyConfig. (line 365) +* PyConfig.use_environment (C member): PyConfig. (line 361) +* PyConfig.use_hash_seed (C member): PyConfig. (line 185) +* PyConfig.verbose (C member): PyConfig. (line 369) +* PyConfig.warnoptions (C member): PyConfig. (line 373) +* PyConfig.write_bytecode (C member): PyConfig. (line 384) +* PyConfig.xoptions (C member): PyConfig. (line 391) +* PyConfig_Clear (C function): PyConfig. (line 69) +* PyConfig_InitIsolatedConfig (C function): PyConfig. (line 17) +* PyConfig_InitPythonConfig (C function): PyConfig. (line 12) +* PyConfig_Read (C function): PyConfig. (line 61) +* PyConfig_SetArgv (C function): PyConfig. (line 37) +* PyConfig_SetBytesArgv (C function): PyConfig. (line 44) +* PyConfig_SetBytesString (C function): PyConfig. (line 29) +* PyConfig_SetString (C function): PyConfig. (line 22) +* PyConfig_SetWideStringList (C function): PyConfig. (line 53) +* PyContext (C type): Context Variables Objects. + (line 24) +* PyContextToken (C type): Context Variables Objects. + (line 34) +* PyContextToken_CheckExact (C function): Context Variables Objects. + (line 62) +* PyContextToken_Type (C variable): Context Variables Objects. + (line 46) +* PyContextVar (C type): Context Variables Objects. + (line 29) +* PyContextVar_CheckExact (C function): Context Variables Objects. + (line 57) +* PyContextVar_Get (C function): Context Variables Objects. + (line 107) +* PyContextVar_New (C function): Context Variables Objects. + (line 99) +* PyContextVar_Reset (C function): Context Variables Objects. + (line 132) +* PyContextVar_Set (C function): Context Variables Objects. + (line 126) +* PyContextVar_Type (C variable): Context Variables Objects. + (line 42) +* PyContext_CheckExact (C function): Context Variables Objects. + (line 52) +* PyContext_ClearFreeList (C function): Context Variables Objects. + (line 92) +* PyContext_Copy (C function): Context Variables Objects. + (line 73) +* PyContext_CopyCurrent (C function): Context Variables Objects. + (line 77) +* PyContext_Enter (C function): Context Variables Objects. + (line 81) +* PyContext_Exit (C function): Context Variables Objects. + (line 86) +* PyContext_New (C function): Context Variables Objects. + (line 69) +* PyContext_Type (C variable): Context Variables Objects. + (line 38) +* PyCoroObject (C type): Coroutine Objects<2>. + (line 11) +* PyCoro_CheckExact (C function): Coroutine Objects<2>. + (line 19) +* PyCoro_New (C function): Coroutine Objects<2>. + (line 24) +* PyCoro_Type (C variable): Coroutine Objects<2>. + (line 15) +* PyDateTime_Check (C function): DateTime Objects<2>. + (line 35) +* PyDateTime_CheckExact (C function): DateTime Objects<2>. + (line 40) +* PyDateTime_DATE_GET_HOUR (C function): DateTime Objects<2>. + (line 158) +* PyDateTime_DATE_GET_MICROSECOND (C function): DateTime Objects<2>. + (line 170) +* PyDateTime_DATE_GET_MINUTE (C function): DateTime Objects<2>. + (line 162) +* PyDateTime_DATE_GET_SECOND (C function): DateTime Objects<2>. + (line 166) +* PyDateTime_DELTA_GET_DAYS (C function): DateTime Objects<2>. + (line 200) +* PyDateTime_DELTA_GET_MICROSECONDS (C function): DateTime Objects<2>. + (line 212) +* PyDateTime_DELTA_GET_SECONDS (C function): DateTime Objects<2>. + (line 206) +* PyDateTime_FromDateAndTime (C function): DateTime Objects<2>. + (line 81) +* PyDateTime_FromDateAndTimeAndFold (C function): DateTime Objects<2>. + (line 88) +* PyDateTime_FromTimestamp (C function): DateTime Objects<2>. + (line 221) +* PyDateTime_GET_DAY (C function): DateTime Objects<2>. + (line 150) +* PyDateTime_GET_MONTH (C function): DateTime Objects<2>. + (line 146) +* PyDateTime_GET_YEAR (C function): DateTime Objects<2>. + (line 142) +* PyDateTime_TimeZone_UTC (C variable): DateTime Objects<2>. + (line 16) +* PyDateTime_TIME_GET_HOUR (C function): DateTime Objects<2>. + (line 180) +* PyDateTime_TIME_GET_MICROSECOND (C function): DateTime Objects<2>. + (line 192) +* PyDateTime_TIME_GET_MINUTE (C function): DateTime Objects<2>. + (line 184) +* PyDateTime_TIME_GET_SECOND (C function): DateTime Objects<2>. + (line 188) +* PyDate_Check (C function): DateTime Objects<2>. + (line 25) +* PyDate_CheckExact (C function): DateTime Objects<2>. + (line 30) +* PyDate_FromDate (C function): DateTime Objects<2>. + (line 77) +* PyDate_FromTimestamp (C function): DateTime Objects<2>. + (line 226) +* PyDelta_Check (C function): DateTime Objects<2>. + (line 55) +* PyDelta_CheckExact (C function): DateTime Objects<2>. + (line 60) +* PyDelta_FromDSU (C function): DateTime Objects<2>. + (line 112) +* PyDescr_IsData (C function): Descriptor Objects. (line 37) +* PyDescr_NewClassMethod (C function): Descriptor Objects. (line 32) +* PyDescr_NewGetSet (C function): Descriptor Objects. (line 13) +* PyDescr_NewMember (C function): Descriptor Objects. (line 18) +* PyDescr_NewMethod (C function): Descriptor Objects. (line 23) +* PyDescr_NewWrapper (C function): Descriptor Objects. (line 28) +* PyDictObject (C type): Dictionary Objects. (line 6) +* PyDictProxy_New (C function): Dictionary Objects. (line 31) +* PyDict_Check (C function): Dictionary Objects. (line 17) +* PyDict_CheckExact (C function): Dictionary Objects. (line 22) +* PyDict_Clear (C function): Dictionary Objects. (line 37) +* PyDict_ClearFreeList (C function): Dictionary Objects. (line 223) +* PyDict_Contains (C function): Dictionary Objects. (line 41) +* PyDict_Copy (C function): Dictionary Objects. (line 47) +* PyDict_DelItem (C function): Dictionary Objects. (line 69) +* PyDict_DelItemString (C function): Dictionary Objects. (line 76) +* PyDict_GetItem (C function): Dictionary Objects. (line 82) +* PyDict_GetItemString (C function): Dictionary Objects. (line 99) +* PyDict_GetItemWithError (C function): Dictionary Objects. (line 91) +* PyDict_Items (C function): Dictionary Objects. (line 122) +* PyDict_Keys (C function): Dictionary Objects. (line 126) +* PyDict_Merge (C function): Dictionary Objects. (line 188) +* PyDict_MergeFromSeq2 (C function): Dictionary Objects. (line 207) +* PyDict_New (C function): Dictionary Objects. (line 27) +* PyDict_Next (C function): Dictionary Objects. (line 139) +* PyDict_SetDefault (C function): Dictionary Objects. (line 110) +* PyDict_SetItem (C function): Dictionary Objects. (line 51) +* PyDict_SetItemString (C function): Dictionary Objects. (line 60) +* PyDict_Size (C function): Dictionary Objects. (line 134) +* PyDict_Type (C variable): Dictionary Objects. (line 11) +* PyDict_Update (C function): Dictionary Objects. (line 199) +* PyDict_Values (C function): Dictionary Objects. (line 130) +* PyDLL (class in ctypes): Loading shared libraries. + (line 61) +* pydoc (module): pydoc — Documentation generator and online help system. + (line 6) +* PyDoc_STR (C macro): Useful macros. (line 99) +* PyDoc_STRVAR (C macro): Useful macros. (line 81) +* PyErr_BadArgument (C function): Raising exceptions. (line 46) +* PyErr_BadInternalCall (C function): Raising exceptions. (line 197) +* PyErr_CheckSignals (C function): Signal Handling<2>. (line 6) +* PyErr_Clear (C function): Printing and clearing. + (line 6) +* PyErr_Clear(): Exceptions<18>. (line 25) +* PyErr_Clear() <1>: Exceptions<18>. (line 123) +* PyErr_ExceptionMatches (C function): Querying the error indicator. + (line 20) +* PyErr_ExceptionMatches(): Exceptions<18>. (line 123) +* PyErr_Fetch (C function): Querying the error indicator. + (line 36) +* PyErr_Fetch(): Finalization and De-allocation. + (line 21) +* PyErr_Format (C function): Raising exceptions. (line 25) +* PyErr_FormatV (C function): Raising exceptions. (line 33) +* PyErr_GetExcInfo (C function): Querying the error indicator. + (line 97) +* PyErr_GivenExceptionMatches (C function): Querying the error indicator. + (line 26) +* PyErr_NewException (C function): Exception Classes. (line 6) +* PyErr_NewExceptionWithDoc (C function): Exception Classes. (line 24) +* PyErr_NoMemory (C function): Raising exceptions. (line 52) +* PyErr_NormalizeException (C function): Querying the error indicator. + (line 78) +* PyErr_Occurred (C function): Querying the error indicator. + (line 6) +* PyErr_Occurred(): Exceptions<18>. (line 12) +* PyErr_Print (C function): Printing and clearing. + (line 26) +* PyErr_PrintEx (C function): Printing and clearing. + (line 11) +* PyErr_ResourceWarning (C function): Issuing warnings. (line 80) +* PyErr_Restore (C function): Querying the error indicator. + (line 59) +* PyErr_Restore(): Finalization and De-allocation. + (line 21) +* PyErr_SetExcFromWindowsErr (C function): Raising exceptions. + (line 114) +* PyErr_SetExcFromWindowsErrWithFilename (C function): Raising exceptions. + (line 152) +* PyErr_SetExcFromWindowsErrWithFilenameObject (C function): Raising exceptions. + (line 132) +* PyErr_SetExcFromWindowsErrWithFilenameObjects (C function): Raising exceptions. + (line 141) +* PyErr_SetExcInfo (C function): Querying the error indicator. + (line 114) +* PyErr_SetFromErrno (C function): Raising exceptions. (line 58) +* PyErr_SetFromErrnoWithFilename (C function): Raising exceptions. + (line 93) +* PyErr_SetFromErrnoWithFilenameObject (C function): Raising exceptions. + (line 73) +* PyErr_SetFromErrnoWithFilenameObjects (C function): Raising exceptions. + (line 82) +* PyErr_SetFromWindowsErr (C function): Raising exceptions. (line 100) +* PyErr_SetFromWindowsErrWithFilename (C function): Raising exceptions. + (line 123) +* PyErr_SetImportError (C function): Raising exceptions. (line 161) +* PyErr_SetImportErrorSubclass (C function): Issuing warnings. + (line 41) +* PyErr_SetInterrupt (C function): Signal Handling<2>. (line 18) +* PyErr_SetNone (C function): Raising exceptions. (line 42) +* PyErr_SetObject (C function): Raising exceptions. (line 19) +* PyErr_SetString (C function): Raising exceptions. (line 10) +* PyErr_SetString(): Exceptions<18>. (line 25) +* PyErr_SyntaxLocation (C function): Raising exceptions. (line 191) +* PyErr_SyntaxLocationEx (C function): Raising exceptions. (line 182) +* PyErr_SyntaxLocationObject (C function): Raising exceptions. + (line 171) +* PyErr_WarnEx (C function): Issuing warnings. (line 19) +* PyErr_WarnExplicit (C function): Issuing warnings. (line 63) +* PyErr_WarnExplicitObject (C function): Issuing warnings. (line 50) +* PyErr_WarnFormat (C function): Issuing warnings. (line 71) +* PyErr_WriteUnraisable (C function): Printing and clearing. + (line 30) +* PyEval_AcquireLock (C function): Low-level API. (line 137) +* PyEval_AcquireThread (C function): Low-level API. (line 104) +* PyEval_AcquireThread(): High-level API. (line 29) +* PyEval_EvalCode (C function): The Very High Level Layer. + (line 329) +* PyEval_EvalCodeEx (C function): The Very High Level Layer. + (line 335) +* PyEval_EvalFrame (C function): The Very High Level Layer. + (line 354) +* PyEval_EvalFrameEx (C function): The Very High Level Layer. + (line 359) +* PyEval_GetBuiltins (C function): Reflection. (line 6) +* PyEval_GetFrame (C function): Reflection. (line 21) +* PyEval_GetFuncDesc (C function): Reflection. (line 34) +* PyEval_GetFuncName (C function): Reflection. (line 29) +* PyEval_GetGlobals (C function): Reflection. (line 16) +* PyEval_GetLocals (C function): Reflection. (line 11) +* PyEval_InitThreads (C function): High-level API. (line 27) +* PyEval_InitThreads(): Initializing and finalizing the interpreter. + (line 8) +* PyEval_MergeCompilerFlags (C function): The Very High Level Layer. + (line 373) +* PyEval_ReleaseLock (C function): Low-level API. (line 159) +* PyEval_ReleaseThread (C function): Low-level API. (line 126) +* PyEval_ReleaseThread(): High-level API. (line 29) +* PyEval_RestoreThread (C function): High-level API. (line 62) +* PyEval_RestoreThread(): Releasing the GIL from extension code. + (line 33) +* PyEval_RestoreThread() <1>: High-level API. (line 29) +* PyEval_SaveThread (C function): High-level API. (line 55) +* PyEval_SaveThread(): Releasing the GIL from extension code. + (line 33) +* PyEval_SaveThread() <1>: High-level API. (line 29) +* PyEval_SetProfile (C function): Profiling and Tracing. + (line 116) +* PyEval_SetTrace (C function): Profiling and Tracing. + (line 127) +* PyEval_ThreadsInitialized (C function): High-level API. (line 45) +* PyException_GetCause (C function): Exception Objects. (line 32) +* PyException_GetContext (C function): Exception Objects. (line 18) +* PyException_GetTraceback (C function): Exception Objects. (line 6) +* PyException_SetCause (C function): Exception Objects. (line 38) +* PyException_SetContext (C function): Exception Objects. (line 25) +* PyException_SetTraceback (C function): Exception Objects. (line 12) +* PyExc_ArithmeticError: Standard Exceptions. + (line 11) +* PyExc_AssertionError: Standard Exceptions. + (line 11) +* PyExc_AttributeError: Standard Exceptions. + (line 11) +* PyExc_BaseException: Standard Exceptions. + (line 11) +* PyExc_BlockingIOError: Standard Exceptions. + (line 11) +* PyExc_BrokenPipeError: Standard Exceptions. + (line 11) +* PyExc_BufferError: Standard Exceptions. + (line 11) +* PyExc_BytesWarning: Standard Warning Categories. + (line 11) +* PyExc_ChildProcessError: Standard Exceptions. + (line 11) +* PyExc_ConnectionAbortedError: Standard Exceptions. + (line 11) +* PyExc_ConnectionError: Standard Exceptions. + (line 11) +* PyExc_ConnectionRefusedError: Standard Exceptions. + (line 11) +* PyExc_ConnectionResetError: Standard Exceptions. + (line 11) +* PyExc_DeprecationWarning: Standard Warning Categories. + (line 11) +* PyExc_EnvironmentError: Standard Exceptions. + (line 190) +* PyExc_EOFError: Standard Exceptions. + (line 11) +* PyExc_Exception: Standard Exceptions. + (line 11) +* PyExc_FileExistsError: Standard Exceptions. + (line 11) +* PyExc_FileNotFoundError: Standard Exceptions. + (line 11) +* PyExc_FloatingPointError: Standard Exceptions. + (line 11) +* PyExc_FutureWarning: Standard Warning Categories. + (line 11) +* PyExc_GeneratorExit: Standard Exceptions. + (line 11) +* PyExc_ImportError: Standard Exceptions. + (line 11) +* PyExc_ImportWarning: Standard Warning Categories. + (line 11) +* PyExc_IndentationError: Standard Exceptions. + (line 11) +* PyExc_IndexError: Standard Exceptions. + (line 11) +* PyExc_InterruptedError: Standard Exceptions. + (line 11) +* PyExc_IOError: Standard Exceptions. + (line 190) +* PyExc_IsADirectoryError: Standard Exceptions. + (line 11) +* PyExc_KeyboardInterrupt: Standard Exceptions. + (line 11) +* PyExc_KeyError: Standard Exceptions. + (line 11) +* PyExc_LookupError: Standard Exceptions. + (line 11) +* PyExc_MemoryError: Standard Exceptions. + (line 11) +* PyExc_ModuleNotFoundError: Standard Exceptions. + (line 11) +* PyExc_NameError: Standard Exceptions. + (line 11) +* PyExc_NotADirectoryError: Standard Exceptions. + (line 11) +* PyExc_NotImplementedError: Standard Exceptions. + (line 11) +* PyExc_OSError: Standard Exceptions. + (line 11) +* PyExc_OverflowError: Standard Exceptions. + (line 11) +* PyExc_PendingDeprecationWarning: Standard Warning Categories. + (line 11) +* PyExc_PermissionError: Standard Exceptions. + (line 11) +* PyExc_ProcessLookupError: Standard Exceptions. + (line 11) +* PyExc_RecursionError: Standard Exceptions. + (line 11) +* PyExc_ReferenceError: Standard Exceptions. + (line 11) +* PyExc_ResourceWarning: Standard Warning Categories. + (line 11) +* PyExc_RuntimeError: Standard Exceptions. + (line 11) +* PyExc_RuntimeWarning: Standard Warning Categories. + (line 11) +* PyExc_StopAsyncIteration: Standard Exceptions. + (line 11) +* PyExc_StopIteration: Standard Exceptions. + (line 11) +* PyExc_SyntaxError: Standard Exceptions. + (line 11) +* PyExc_SyntaxWarning: Standard Warning Categories. + (line 11) +* PyExc_SystemError: Standard Exceptions. + (line 11) +* PyExc_SystemExit: Standard Exceptions. + (line 11) +* PyExc_TabError: Standard Exceptions. + (line 11) +* PyExc_TimeoutError: Standard Exceptions. + (line 11) +* PyExc_TypeError: Standard Exceptions. + (line 11) +* PyExc_UnboundLocalError: Standard Exceptions. + (line 11) +* PyExc_UnicodeDecodeError: Standard Exceptions. + (line 11) +* PyExc_UnicodeEncodeError: Standard Exceptions. + (line 11) +* PyExc_UnicodeError: Standard Exceptions. + (line 11) +* PyExc_UnicodeTranslateError: Standard Exceptions. + (line 11) +* PyExc_UnicodeWarning: Standard Warning Categories. + (line 11) +* PyExc_UserWarning: Standard Warning Categories. + (line 11) +* PyExc_ValueError: Standard Exceptions. + (line 11) +* PyExc_Warning: Standard Warning Categories. + (line 11) +* PyExc_WindowsError: Standard Exceptions. + (line 190) +* PyExc_ZeroDivisionError: Standard Exceptions. + (line 11) +* PyFile_FromFd (C function): File Objects. (line 15) +* PyFile_GetLine (C function): File Objects. (line 40) +* PyFile_SetOpenCodeHook (C function): File Objects. (line 54) +* PyFile_WriteObject (C function): File Objects. (line 84) +* PyFile_WriteString (C function): File Objects. (line 92) +* PyFloatObject (C type): Floating Point Objects. + (line 6) +* PyFloat_AsDouble (C function): Floating Point Objects. + (line 35) +* PyFloat_AS_DOUBLE (C function): Floating Point Objects. + (line 47) +* PyFloat_Check (C function): Floating Point Objects. + (line 17) +* PyFloat_CheckExact (C function): Floating Point Objects. + (line 22) +* PyFloat_ClearFreeList (C function): Floating Point Objects. + (line 68) +* PyFloat_FromDouble (C function): Floating Point Objects. + (line 31) +* PyFloat_FromString (C function): Floating Point Objects. + (line 27) +* PyFloat_GetInfo (C function): Floating Point Objects. + (line 52) +* PyFloat_GetMax (C function): Floating Point Objects. + (line 58) +* PyFloat_GetMin (C function): Floating Point Objects. + (line 63) +* PyFloat_Type (C variable): Floating Point Objects. + (line 11) +* PyFrameObject (C type): The Very High Level Layer. + (line 349) +* PyFrame_GetLineNumber (C function): Reflection. (line 25) +* PyFrozenSet_Check (C function): Set Objects. (line 50) +* PyFrozenSet_CheckExact (C function): Set Objects. (line 65) +* PyFrozenSet_New (C function): Set Objects. (line 78) +* PyFrozenSet_Type (C variable): Set Objects. (line 36) +* PyFunctionObject (C type): Function Objects<3>. + (line 8) +* PyFunction_Check (C function): Function Objects<3>. + (line 18) +* PyFunction_GetAnnotations (C function): Function Objects<3>. + (line 85) +* PyFunction_GetClosure (C function): Function Objects<3>. + (line 72) +* PyFunction_GetCode (C function): Function Objects<3>. + (line 45) +* PyFunction_GetDefaults (C function): Function Objects<3>. + (line 59) +* PyFunction_GetGlobals (C function): Function Objects<3>. + (line 49) +* PyFunction_GetModule (C function): Function Objects<3>. + (line 53) +* PyFunction_New (C function): Function Objects<3>. + (line 23) +* PyFunction_NewWithQualName (C function): Function Objects<3>. + (line 34) +* PyFunction_SetAnnotations (C function): Function Objects<3>. + (line 89) +* PyFunction_SetClosure (C function): Function Objects<3>. + (line 77) +* PyFunction_SetDefaults (C function): Function Objects<3>. + (line 64) +* PyFunction_Type (C variable): Function Objects<3>. + (line 12) +* PYFUNCTYPE() (in module ctypes): Function prototypes. + (line 34) +* PyGenObject (C type): Generator Objects. (line 11) +* PyGen_Check (C function): Generator Objects. (line 19) +* PyGen_CheckExact (C function): Generator Objects. (line 23) +* PyGen_New (C function): Generator Objects. (line 28) +* PyGen_NewWithQualName (C function): Generator Objects. (line 33) +* PyGen_Type (C variable): Generator Objects. (line 15) +* PyGetSetDef (C type): Common Object Structures. + (line 358) +* PyGILState_Check (C function): High-level API. (line 142) +* PyGILState_Ensure (C function): High-level API. (line 92) +* PyGILState_GetThisThreadState (C function): High-level API. + (line 134) +* PyGILState_Release (C function): High-level API. (line 123) +* PyImport_AddModule (C function): Importing Modules<2>. + (line 111) +* PyImport_AddModuleObject (C function): Importing Modules<2>. + (line 95) +* PyImport_AppendInittab (C function): Importing Modules<2>. + (line 281) +* PyImport_Cleanup (C function): Importing Modules<2>. + (line 234) +* PyImport_ExecCodeModule (C function): Importing Modules<2>. + (line 116) +* PyImport_ExecCodeModuleEx (C function): Importing Modules<2>. + (line 152) +* PyImport_ExecCodeModuleObject (C function): Importing Modules<2>. + (line 162) +* PyImport_ExecCodeModuleWithPathnames (C function): Importing Modules<2>. + (line 172) +* PyImport_ExtendInittab (C function): Importing Modules<2>. + (line 307) +* PyImport_FrozenModules (C variable): Importing Modules<2>. + (line 273) +* PyImport_GetImporter (C function): Importing Modules<2>. + (line 219) +* PyImport_GetMagicNumber (C function): Importing Modules<2>. + (line 188) +* PyImport_GetMagicTag (C function): Importing Modules<2>. + (line 197) +* PyImport_GetModule (C function): Importing Modules<2>. + (line 211) +* PyImport_GetModuleDict (C function): Importing Modules<2>. + (line 206) +* PyImport_Import (C function): Importing Modules<2>. + (line 80) +* PyImport_ImportFrozenModule (C function): Importing Modules<2>. + (line 255) +* PyImport_ImportFrozenModuleObject (C function): Importing Modules<2>. + (line 242) +* PyImport_ImportModule (C function): Importing Modules<2>. + (line 6) +* PyImport_ImportModuleEx (C function): Importing Modules<2>. + (line 35) +* PyImport_ImportModuleLevel (C function): Importing Modules<2>. + (line 69) +* PyImport_ImportModuleLevelObject (C function): Importing Modules<2>. + (line 52) +* PyImport_ImportModuleNoBlock (C function): Importing Modules<2>. + (line 25) +* PyImport_ReloadModule (C function): Importing Modules<2>. + (line 90) +* PyIndex_Check (C function): Number Protocol. (line 272) +* PyInit_modulename (C function): Building C and C++ Extensions. + (line 16) +* PyInstanceMethod_Check (C function): Instance Method Objects. + (line 15) +* PyInstanceMethod_Function (C function): Instance Method Objects. + (line 25) +* PyInstanceMethod_GET_FUNCTION (C function): Instance Method Objects. + (line 29) +* PyInstanceMethod_New (C function): Instance Method Objects. + (line 20) +* PyInstanceMethod_Type (C variable): Instance Method Objects. + (line 10) +* PyInterpreterState (C type): High-level API. (line 9) +* PyInterpreterState_Clear (C function): Low-level API. (line 21) +* PyInterpreterState_Delete (C function): Low-level API. (line 30) +* PyInterpreterState_GetDict (C function): Low-level API. (line 63) +* PyInterpreterState_GetID (C function): Low-level API. (line 55) +* PyInterpreterState_Head (C function): Advanced Debugger Support. + (line 9) +* PyInterpreterState_Main (C function): Advanced Debugger Support. + (line 14) +* PyInterpreterState_New (C function): Low-level API. (line 12) +* PyInterpreterState_Next (C function): Advanced Debugger Support. + (line 18) +* PyInterpreterState_ThreadHead (C function): Advanced Debugger Support. + (line 25) +* PyIter_Check (C function): Iterator Protocol. (line 8) +* PyIter_Next (C function): Iterator Protocol. (line 12) +* PyListObject (C type): List Objects. (line 6) +* PyList_Append (C function): List Objects. (line 91) +* PyList_AsTuple (C function): List Objects. (line 125) +* PyList_Check (C function): List Objects. (line 17) +* PyList_CheckExact (C function): List Objects. (line 22) +* PyList_ClearFreeList (C function): List Objects. (line 131) +* PyList_GetItem (C function): List Objects. (line 46) +* PyList_GetItem(): Reference Count Details. + (line 117) +* PyList_GetSlice (C function): List Objects. (line 97) +* PyList_GET_ITEM (C function): List Objects. (line 54) +* PyList_GET_SIZE (C function): List Objects. (line 42) +* PyList_Insert (C function): List Objects. (line 83) +* PyList_New (C function): List Objects. (line 27) +* PyList_Reverse (C function): List Objects. (line 120) +* PyList_SetItem (C function): List Objects. (line 58) +* PyList_SetItem(): Reference Count Details. + (line 26) +* PyList_SetSlice (C function): List Objects. (line 105) +* PyList_SET_ITEM (C function): List Objects. (line 70) +* PyList_Size (C function): List Objects. (line 37) +* PyList_Sort (C function): List Objects. (line 115) +* PyList_Type (C variable): List Objects. (line 11) +* PyLongObject (C type): Integer Objects. (line 13) +* PyLong_AsDouble (C function): Integer Objects. (line 272) +* PyLong_AsLong (C function): Integer Objects. (line 105) +* PyLong_AsLongAndOverflow (C function): Integer Objects. (line 123) +* PyLong_AsLongLong (C function): Integer Objects. (line 144) +* PyLong_AsLongLongAndOverflow (C function): Integer Objects. + (line 162) +* PyLong_AsSize_t (C function): Integer Objects. (line 208) +* PyLong_AsSsize_t (C function): Integer Objects. (line 186) +* PyLong_AsUnsignedLong (C function): Integer Objects. (line 197) +* PyLong_AsUnsignedLongLong (C function): Integer Objects. (line 219) +* PyLong_AsUnsignedLongLongMask (C function): Integer Objects. + (line 252) +* PyLong_AsUnsignedLongMask (C function): Integer Objects. (line 234) +* PyLong_AsVoidPtr (C function): Integer Objects. (line 283) +* PyLong_Check (C function): Integer Objects. (line 24) +* PyLong_CheckExact (C function): Integer Objects. (line 29) +* PyLong_FromDouble (C function): Integer Objects. (line 66) +* PyLong_FromLong (C function): Integer Objects. (line 34) +* PyLong_FromLongLong (C function): Integer Objects. (line 56) +* PyLong_FromSize_t (C function): Integer Objects. (line 52) +* PyLong_FromSsize_t (C function): Integer Objects. (line 48) +* PyLong_FromString (C function): Integer Objects. (line 70) +* PyLong_FromUnicode (C function): Integer Objects. (line 84) +* PyLong_FromUnicodeObject (C function): Integer Objects. (line 93) +* PyLong_FromUnsignedLong (C function): Integer Objects. (line 44) +* PyLong_FromUnsignedLongLong (C function): Integer Objects. (line 60) +* PyLong_FromVoidPtr (C function): Integer Objects. (line 100) +* PyLong_Type (C variable): Integer Objects. (line 18) +* PyMappingMethods (C type): Mapping Object Structures. + (line 6) +* PyMappingMethods.mp_ass_subscript (C member): Mapping Object Structures. + (line 25) +* PyMappingMethods.mp_length (C member): Mapping Object Structures. + (line 11) +* PyMappingMethods.mp_subscript (C member): Mapping Object Structures. + (line 17) +* PyMapping_Check (C function): Mapping Protocol. (line 9) +* PyMapping_DelItem (C function): Mapping Protocol. (line 39) +* PyMapping_DelItemString (C function): Mapping Protocol. (line 45) +* PyMapping_GetItemString (C function): Mapping Protocol. (line 23) +* PyMapping_HasKey (C function): Mapping Protocol. (line 52) +* PyMapping_HasKeyString (C function): Mapping Protocol. (line 62) +* PyMapping_Items (C function): Mapping Protocol. (line 88) +* PyMapping_Keys (C function): Mapping Protocol. (line 74) +* PyMapping_Length (C function): Mapping Protocol. (line 17) +* PyMapping_SetItemString (C function): Mapping Protocol. (line 31) +* PyMapping_Size (C function): Mapping Protocol. (line 17) +* PyMapping_Values (C function): Mapping Protocol. (line 81) +* PyMarshal_ReadLastObjectFromFile (C function): Data marshalling support. + (line 68) +* PyMarshal_ReadLongFromFile (C function): Data marshalling support. + (line 42) +* PyMarshal_ReadObjectFromFile (C function): Data marshalling support. + (line 60) +* PyMarshal_ReadObjectFromString (C function): Data marshalling support. + (line 83) +* PyMarshal_ReadShortFromFile (C function): Data marshalling support. + (line 51) +* PyMarshal_WriteLongToFile (C function): Data marshalling support. + (line 20) +* PyMarshal_WriteObjectToFile (C function): Data marshalling support. + (line 28) +* PyMarshal_WriteObjectToString (C function): Data marshalling support. + (line 34) +* PyMemAllocatorDomain (C type): Customize Memory Allocators. + (line 36) +* PyMemAllocatorEx (C type): Customize Memory Allocators. + (line 8) +* PyMemberDef (C type): Common Object Structures. + (line 255) +* PyMemoryView_Check (C function): MemoryView objects. (line 41) +* PyMemoryView_FromBuffer (C function): MemoryView objects. (line 26) +* PyMemoryView_FromMemory (C function): MemoryView objects. (line 17) +* PyMemoryView_FromObject (C function): MemoryView objects. (line 10) +* PyMemoryView_GetContiguous (C function): MemoryView objects. + (line 31) +* PyMemoryView_GET_BASE (C function): MemoryView objects. (line 53) +* PyMemoryView_GET_BUFFER (C function): MemoryView objects. (line 46) +* PyMem_Calloc (C function): Memory Interface. (line 28) +* PyMem_Del (C function): Memory Interface. (line 86) +* PYMEM_DOMAIN_MEM (C macro): Customize Memory Allocators. + (line 52) +* PYMEM_DOMAIN_OBJ (C macro): Customize Memory Allocators. + (line 64) +* PYMEM_DOMAIN_RAW (C macro): Customize Memory Allocators. + (line 40) +* PyMem_Free (C function): Memory Interface. (line 57) +* PyMem_GetAllocator (C function): Customize Memory Allocators. + (line 76) +* PyMem_Malloc (C function): Memory Interface. (line 19) +* PyMem_New (C function): Memory Interface. (line 70) +* PyMem_RawCalloc (C function): Raw Memory Interface. + (line 24) +* PyMem_RawFree (C function): Raw Memory Interface. + (line 53) +* PyMem_RawMalloc (C function): Raw Memory Interface. + (line 15) +* PyMem_RawRealloc (C function): Raw Memory Interface. + (line 36) +* PyMem_Realloc (C function): Memory Interface. (line 40) +* PyMem_Resize (C function): Memory Interface. (line 76) +* PyMem_SetAllocator (C function): Customize Memory Allocators. + (line 82) +* PyMem_SetupDebugHooks (C function): Customize Memory Allocators. + (line 99) +* PyMethodDef (C type): Common Object Structures. + (line 119) +* PyMethod_Check (C function): Method Objects<2>. (line 16) +* PyMethod_ClearFreeList (C function): Method Objects<2>. (line 43) +* PyMethod_Function (C function): Method Objects<2>. (line 27) +* PyMethod_GET_FUNCTION (C function): Method Objects<2>. (line 31) +* PyMethod_GET_SELF (C function): Method Objects<2>. (line 39) +* PyMethod_New (C function): Method Objects<2>. (line 21) +* PyMethod_Self (C function): Method Objects<2>. (line 35) +* PyMethod_Type (C variable): Method Objects<2>. (line 10) +* PyModuleDef (C type): Initializing C modules. + (line 17) +* PyModuleDef.m_base (C member): Initializing C modules. + (line 23) +* PyModuleDef.m_clear (C member): Initializing C modules. + (line 82) +* PyModuleDef.m_doc (C member): Initializing C modules. + (line 31) +* PyModuleDef.m_free (C member): Initializing C modules. + (line 90) +* PyModuleDef.m_methods (C member): Initializing C modules. + (line 57) +* PyModuleDef.m_name (C member): Initializing C modules. + (line 27) +* PyModuleDef.m_reload (C member): Initializing C modules. + (line 72) +* PyModuleDef.m_size (C member): Initializing C modules. + (line 36) +* PyModuleDef.m_slots (C member): Initializing C modules. + (line 63) +* PyModuleDef.m_traverse (C member): Initializing C modules. + (line 74) +* PyModuleDef_Init (C function): Multi-phase initialization. + (line 33) +* PyModuleDef_Slot (C type): Multi-phase initialization. + (line 45) +* PyModuleDef_Slot.slot (C member): Multi-phase initialization. + (line 47) +* PyModuleDef_Slot.value (C member): Multi-phase initialization. + (line 51) +* PyModule_AddFunctions (C function): Low-level module creation functions. + (line 53) +* PyModule_AddIntConstant (C function): Support functions. (line 34) +* PyModule_AddIntMacro (C function): Support functions. (line 51) +* PyModule_AddObject (C function): Support functions. (line 11) +* PyModule_AddStringConstant (C function): Support functions. + (line 42) +* PyModule_AddStringMacro (C function): Support functions. (line 58) +* PyModule_Check (C function): Module Objects. (line 12) +* PyModule_CheckExact (C function): Module Objects. (line 17) +* PyModule_Create (C function): Single-phase initialization. + (line 10) +* PyModule_Create2 (C function): Single-phase initialization. + (line 16) +* PyModule_ExecDef (C function): Low-level module creation functions. + (line 36) +* PyModule_FromDefAndSpec (C function): Low-level module creation functions. + (line 11) +* PyModule_FromDefAndSpec2 (C function): Low-level module creation functions. + (line 21) +* PyModule_GetDef (C function): Module Objects. (line 74) +* PyModule_GetDict (C function): Module Objects. (line 41) +* PyModule_GetFilename (C function): Module Objects. (line 91) +* PyModule_GetFilenameObject (C function): Module Objects. (line 80) +* PyModule_GetName (C function): Module Objects. (line 63) +* PyModule_GetNameObject (C function): Module Objects. (line 54) +* PyModule_GetState (C function): Module Objects. (line 68) +* PyModule_New (C function): Module Objects. (line 36) +* PyModule_NewObject (C function): Module Objects. (line 22) +* PyModule_SetDocString (C function): Low-level module creation functions. + (line 44) +* PyModule_Type (C variable): Module Objects. (line 6) +* PyNumberMethods (C type): Number Object Structures. + (line 6) +* PyNumberMethods.nb_absolute (C member): Number Object Structures. + (line 85) +* PyNumberMethods.nb_add (C member): Number Object Structures. + (line 69) +* PyNumberMethods.nb_and (C member): Number Object Structures. + (line 95) +* PyNumberMethods.nb_bool (C member): Number Object Structures. + (line 87) +* PyNumberMethods.nb_divmod (C member): Number Object Structures. + (line 77) +* PyNumberMethods.nb_float (C member): Number Object Structures. + (line 105) +* PyNumberMethods.nb_floor_divide (C member): Number Object Structures. + (line 127) +* PyNumberMethods.nb_index (C member): Number Object Structures. + (line 135) +* PyNumberMethods.nb_inplace_add (C member): Number Object Structures. + (line 107) +* PyNumberMethods.nb_inplace_and (C member): Number Object Structures. + (line 121) +* PyNumberMethods.nb_inplace_floor_divide (C member): Number Object Structures. + (line 131) +* PyNumberMethods.nb_inplace_lshift (C member): Number Object Structures. + (line 117) +* PyNumberMethods.nb_inplace_matrix_multiply (C member): Number Object Structures. + (line 139) +* PyNumberMethods.nb_inplace_multiply (C member): Number Object Structures. + (line 111) +* PyNumberMethods.nb_inplace_or (C member): Number Object Structures. + (line 125) +* PyNumberMethods.nb_inplace_power (C member): Number Object Structures. + (line 115) +* PyNumberMethods.nb_inplace_remainder (C member): Number Object Structures. + (line 113) +* PyNumberMethods.nb_inplace_rshift (C member): Number Object Structures. + (line 119) +* PyNumberMethods.nb_inplace_subtract (C member): Number Object Structures. + (line 109) +* PyNumberMethods.nb_inplace_true_divide (C member): Number Object Structures. + (line 133) +* PyNumberMethods.nb_inplace_xor (C member): Number Object Structures. + (line 123) +* PyNumberMethods.nb_int (C member): Number Object Structures. + (line 101) +* PyNumberMethods.nb_invert (C member): Number Object Structures. + (line 89) +* PyNumberMethods.nb_lshift (C member): Number Object Structures. + (line 91) +* PyNumberMethods.nb_matrix_multiply (C member): Number Object Structures. + (line 137) +* PyNumberMethods.nb_multiply (C member): Number Object Structures. + (line 73) +* PyNumberMethods.nb_negative (C member): Number Object Structures. + (line 81) +* PyNumberMethods.nb_or (C member): Number Object Structures. + (line 99) +* PyNumberMethods.nb_positive (C member): Number Object Structures. + (line 83) +* PyNumberMethods.nb_power (C member): Number Object Structures. + (line 79) +* PyNumberMethods.nb_remainder (C member): Number Object Structures. + (line 75) +* PyNumberMethods.nb_reserved (C member): Number Object Structures. + (line 103) +* PyNumberMethods.nb_rshift (C member): Number Object Structures. + (line 93) +* PyNumberMethods.nb_subtract (C member): Number Object Structures. + (line 71) +* PyNumberMethods.nb_true_divide (C member): Number Object Structures. + (line 129) +* PyNumberMethods.nb_xor (C member): Number Object Structures. + (line 97) +* PyNumber_Absolute (C function): Number Protocol. (line 85) +* PyNumber_Add (C function): Number Protocol. (line 13) +* PyNumber_And (C function): Number Protocol. (line 106) +* PyNumber_AsSsize_t (C function): Number Protocol. (line 257) +* PyNumber_Check (C function): Number Protocol. (line 6) +* PyNumber_Divmod (C function): Number Protocol. (line 58) +* PyNumber_Float (C function): Number Protocol. (line 237) +* PyNumber_FloorDivide (C function): Number Protocol. (line 37) +* PyNumber_Index (C function): Number Protocol. (line 244) +* PyNumber_InPlaceAdd (C function): Number Protocol. (line 121) +* PyNumber_InPlaceAnd (C function): Number Protocol. (line 209) +* PyNumber_InPlaceFloorDivide (C function): Number Protocol. (line 154) +* PyNumber_InPlaceLshift (C function): Number Protocol. (line 193) +* PyNumber_InPlaceMatrixMultiply (C function): Number Protocol. + (line 144) +* PyNumber_InPlaceMultiply (C function): Number Protocol. (line 136) +* PyNumber_InPlaceOr (C function): Number Protocol. (line 223) +* PyNumber_InPlacePower (C function): Number Protocol. (line 181) +* PyNumber_InPlaceRemainder (C function): Number Protocol. (line 173) +* PyNumber_InPlaceRshift (C function): Number Protocol. (line 201) +* PyNumber_InPlaceSubtract (C function): Number Protocol. (line 128) +* PyNumber_InPlaceTrueDivide (C function): Number Protocol. (line 162) +* PyNumber_InPlaceXor (C function): Number Protocol. (line 216) +* PyNumber_Invert (C function): Number Protocol. (line 91) +* PyNumber_Long (C function): Number Protocol. (line 230) +* PyNumber_Lshift (C function): Number Protocol. (line 96) +* PyNumber_MatrixMultiply (C function): Number Protocol. (line 28) +* PyNumber_Multiply (C function): Number Protocol. (line 23) +* PyNumber_Negative (C function): Number Protocol. (line 76) +* PyNumber_Or (C function): Number Protocol. (line 116) +* PyNumber_Positive (C function): Number Protocol. (line 81) +* PyNumber_Power (C function): Number Protocol. (line 65) +* PyNumber_Remainder (C function): Number Protocol. (line 52) +* PyNumber_Rshift (C function): Number Protocol. (line 101) +* PyNumber_Subtract (C function): Number Protocol. (line 18) +* PyNumber_ToBase (C function): Number Protocol. (line 249) +* PyNumber_TrueDivide (C function): Number Protocol. (line 43) +* PyNumber_Xor (C function): Number Protocol. (line 111) +* PyObject (C type): Common Object Structures. + (line 17) +* PyObject.ob_refcnt (C member): PyObject Slots. (line 30) +* PyObject.ob_type (C member): PyObject Slots. (line 43) +* PyObject._ob_next (C member): PyObject Slots. (line 13) +* PyObject._ob_prev (C member): PyObject Slots. (line 13) +* PyObjectArenaAllocator (C type): Customize pymalloc Arena Allocator. + (line 8) +* PyObject_AsCharBuffer (C function): Old Buffer Protocol. + (line 21) +* PyObject_ASCII (C function): Object Protocol. (line 176) +* PyObject_AsFileDescriptor (C function): File Objects. (line 32) +* PyObject_AsReadBuffer (C function): Old Buffer Protocol. + (line 31) +* PyObject_AsWriteBuffer (C function): Old Buffer Protocol. + (line 50) +* PyObject_Bytes (C function): Object Protocol. (line 197) +* PyObject_Call (C function): Object Protocol. (line 254) +* PyObject_CallFunction (C function): Object Protocol. (line 281) +* PyObject_CallFunctionObjArgs (C function): Object Protocol. + (line 323) +* PyObject_CallMethod (C function): Object Protocol. (line 300) +* PyObject_CallMethodObjArgs (C function): Object Protocol. (line 336) +* PyObject_CallObject (C function): Object Protocol. (line 269) +* PyObject_CallObject(): Calling Python Functions from C. + (line 64) +* PyObject_Calloc (C function): Object allocators. (line 25) +* PyObject_CheckBuffer (C function): Buffer-related functions. + (line 6) +* PyObject_CheckReadBuffer (C function): Old Buffer Protocol. + (line 40) +* PyObject_Del (C function): Allocating Objects on the Heap. + (line 51) +* PyObject_DelAttr (C function): Object Protocol. (line 110) +* PyObject_DelAttrString (C function): Object Protocol. (line 116) +* PyObject_DelItem (C function): Object Protocol. (line 501) +* PyObject_Dir (C function): Object Protocol. (line 507) +* PyObject_Free (C function): Object allocators. (line 54) +* PyObject_GC_Del (C function): Supporting Cyclic Garbage Collection. + (line 71) +* PyObject_GC_New (C function): Supporting Cyclic Garbage Collection. + (line 34) +* PyObject_GC_NewVar (C function): Supporting Cyclic Garbage Collection. + (line 39) +* PyObject_GC_Resize (C function): Supporting Cyclic Garbage Collection. + (line 46) +* PyObject_GC_Track (C function): Supporting Cyclic Garbage Collection. + (line 54) +* PyObject_GC_UnTrack (C function): Supporting Cyclic Garbage Collection. + (line 76) +* PyObject_GenericGetAttr (C function): Object Protocol. (line 63) +* PyObject_GenericGetDict (C function): Object Protocol. (line 123) +* PyObject_GenericSetAttr (C function): Object Protocol. (line 98) +* PyObject_GenericSetDict (C function): Object Protocol. (line 131) +* PyObject_GetArenaAllocator (C function): Customize pymalloc Arena Allocator. + (line 26) +* PyObject_GetAttr (C function): Object Protocol. (line 48) +* PyObject_GetAttrString (C function): Object Protocol. (line 55) +* PyObject_GetBuffer (C function): Buffer-related functions. + (line 13) +* PyObject_GetItem (C function): Object Protocol. (line 487) +* PyObject_GetIter (C function): Object Protocol. (line 516) +* PyObject_HasAttr (C function): Object Protocol. (line 25) +* PyObject_HasAttrString (C function): Object Protocol. (line 36) +* PyObject_Hash (C function): Object Protocol. (line 423) +* PyObject_HashNotImplemented (C function): Object Protocol. (line 432) +* PyObject_HEAD (C macro): Common Object Structures. + (line 37) +* PyObject_HEAD_INIT (C macro): Common Object Structures. + (line 78) +* PyObject_Init (C function): Allocating Objects on the Heap. + (line 14) +* PyObject_InitVar (C function): Allocating Objects on the Heap. + (line 23) +* PyObject_IsInstance (C function): Object Protocol. (line 227) +* PyObject_IsSubclass (C function): Object Protocol. (line 206) +* PyObject_IsTrue (C function): Object Protocol. (line 439) +* PyObject_Length (C function): Object Protocol. (line 468) +* PyObject_LengthHint (C function): Object Protocol. (line 476) +* PyObject_Malloc (C function): Object allocators. (line 16) +* PyObject_New (C function): Allocating Objects on the Heap. + (line 30) +* PyObject_NewVar (C function): Allocating Objects on the Heap. + (line 38) +* PyObject_Not (C function): Object Protocol. (line 445) +* PyObject_Print (C function): Object Protocol. (line 17) +* PyObject_Realloc (C function): Object allocators. (line 37) +* PyObject_Repr (C function): Object Protocol. (line 164) +* PyObject_RichCompare (C function): Object Protocol. (line 139) +* PyObject_RichCompareBool (C function): Object Protocol. (line 149) +* PyObject_SetArenaAllocator (C function): Customize pymalloc Arena Allocator. + (line 31) +* PyObject_SetAttr (C function): Object Protocol. (line 75) +* PyObject_SetAttrString (C function): Object Protocol. (line 87) +* PyObject_SetItem (C function): Object Protocol. (line 492) +* PyObject_Size (C function): Object Protocol. (line 468) +* PyObject_Str (C function): Object Protocol. (line 186) +* PyObject_Type (C function): Object Protocol. (line 451) +* PyObject_TypeCheck (C function): Object Protocol. (line 463) +* PyObject_VAR_HEAD (C macro): Common Object Structures. + (line 47) +* PyOS_AfterFork (C function): Operating System Utilities. + (line 75) +* PyOS_AfterFork_Child (C function): Operating System Utilities. + (line 53) +* PyOS_AfterFork_Parent (C function): Operating System Utilities. + (line 39) +* PyOS_BeforeFork (C function): Operating System Utilities. + (line 26) +* PyOS_CheckStack (C function): Operating System Utilities. + (line 85) +* PyOS_double_to_string (C function): String conversion and formatting. + (line 87) +* PyOS_FSPath (C function): Operating System Utilities. + (line 6) +* PyOS_getsig (C function): Operating System Utilities. + (line 93) +* PyOS_InputHook (C variable): The Very High Level Layer. + (line 165) +* PyOS_ReadlineFunctionPointer (C variable): The Very High Level Layer. + (line 175) +* PyOS_setsig (C function): Operating System Utilities. + (line 100) +* PyOS_snprintf (C function): String conversion and formatting. + (line 8) +* PyOS_stricmp (C function): String conversion and formatting. + (line 122) +* PyOS_string_to_double (C function): String conversion and formatting. + (line 54) +* PyOS_strnicmp (C function): String conversion and formatting. + (line 127) +* PyOS_vsnprintf (C function): String conversion and formatting. + (line 15) +* PyParser_SimpleParseFile (C function): The Very High Level Layer. + (line 221) +* PyParser_SimpleParseFileFlags (C function): The Very High Level Layer. + (line 229) +* PyParser_SimpleParseString (C function): The Very High Level Layer. + (line 196) +* PyParser_SimpleParseStringFlags (C function): The Very High Level Layer. + (line 203) +* PyParser_SimpleParseStringFlagsFilename (C function): The Very High Level Layer. + (line 211) +* PyPI; (see Python Package Index (PyPI)): Installing the tools. + (line 25) +* PyPreConfig (C type): PyPreConfig. (line 6) +* PyPreConfig.allocator (C member): PyPreConfig. (line 32) +* PyPreConfig.coerce_c_locale (C member): PyPreConfig. (line 68) +* PyPreConfig.coerce_c_locale_warn (C member): PyPreConfig. (line 73) +* PyPreConfig.configure_locale (C member): PyPreConfig. (line 62) +* PyPreConfig.dev_mode (C member): PyPreConfig. (line 77) +* PyPreConfig.isolated (C member): PyPreConfig. (line 81) +* PyPreConfig.legacy_windows_fs_encoding (C member): PyPreConfig. + (line 85) +* PyPreConfig.parse_argv (C member): PyPreConfig. (line 94) +* PyPreConfig.use_environment (C member): PyPreConfig. (line 101) +* PyPreConfig.utf8_mode (C member): PyPreConfig. (line 105) +* PyPreConfig_InitIsolatedConfig (C function): PyPreConfig. (line 24) +* PyPreConfig_InitPythonConfig (C function): PyPreConfig. (line 18) +* PyProperty_Type (C variable): Descriptor Objects. (line 9) +* PyRun_AnyFile (C function): The Very High Level Layer. + (line 48) +* PyRun_AnyFileEx (C function): The Very High Level Layer. + (line 61) +* PyRun_AnyFileExFlags (C function): The Very High Level Layer. + (line 68) +* PyRun_AnyFileFlags (C function): The Very High Level Layer. + (line 54) +* PyRun_File (C function): The Very High Level Layer. + (line 255) +* PyRun_FileEx (C function): The Very High Level Layer. + (line 262) +* PyRun_FileExFlags (C function): The Very High Level Layer. + (line 276) +* PyRun_FileFlags (C function): The Very High Level Layer. + (line 269) +* PyRun_InteractiveLoop (C function): The Very High Level Layer. + (line 149) +* PyRun_InteractiveLoopFlags (C function): The Very High Level Layer. + (line 156) +* PyRun_InteractiveOne (C function): The Very High Level Layer. + (line 127) +* PyRun_InteractiveOneFlags (C function): The Very High Level Layer. + (line 134) +* PyRun_SimpleFile (C function): The Very High Level Layer. + (line 100) +* PyRun_SimpleFileEx (C function): The Very High Level Layer. + (line 106) +* PyRun_SimpleFileExFlags (C function): The Very High Level Layer. + (line 113) +* PyRun_SimpleString (C function): The Very High Level Layer. + (line 79) +* PyRun_SimpleStringFlags (C function): The Very High Level Layer. + (line 85) +* PyRun_String (C function): The Very High Level Layer. + (line 236) +* PyRun_StringFlags (C function): The Very High Level Layer. + (line 243) +* PySeqIter_Check (C function): Iterator Objects. (line 18) +* PySeqIter_New (C function): Iterator Objects. (line 22) +* PySeqIter_Type (C variable): Iterator Objects. (line 12) +* PySequenceMethods (C type): Sequence Object Structures. + (line 6) +* PySequenceMethods.sq_ass_item (C member): Sequence Object Structures. + (line 45) +* PySequenceMethods.sq_concat (C member): Sequence Object Structures. + (line 18) +* PySequenceMethods.sq_contains (C member): Sequence Object Structures. + (line 54) +* PySequenceMethods.sq_inplace_concat (C member): Sequence Object Structures. + (line 61) +* PySequenceMethods.sq_inplace_repeat (C member): Sequence Object Structures. + (line 71) +* PySequenceMethods.sq_item (C member): Sequence Object Structures. + (line 31) +* PySequenceMethods.sq_length (C member): Sequence Object Structures. + (line 11) +* PySequenceMethods.sq_repeat (C member): Sequence Object Structures. + (line 24) +* PySequence_Check (C function): Sequence Protocol. (line 6) +* PySequence_Concat (C function): Sequence Protocol. (line 20) +* PySequence_Contains (C function): Sequence Protocol. (line 99) +* PySequence_Count (C function): Sequence Protocol. (line 91) +* PySequence_DelItem (C function): Sequence Protocol. (line 70) +* PySequence_DelSlice (C function): Sequence Protocol. (line 83) +* PySequence_Fast (C function): Sequence Protocol. (line 127) +* PySequence_Fast_GET_ITEM (C function): Sequence Protocol. (line 149) +* PySequence_Fast_GET_SIZE (C function): Sequence Protocol. (line 141) +* PySequence_Fast_ITEMS (C function): Sequence Protocol. (line 156) +* PySequence_GetItem (C function): Sequence Protocol. (line 47) +* PySequence_GetItem(): Reference Count Details. + (line 117) +* PySequence_GetSlice (C function): Sequence Protocol. (line 52) +* PySequence_Index (C function): Sequence Protocol. (line 105) +* PySequence_InPlaceConcat (C function): Sequence Protocol. (line 31) +* PySequence_InPlaceRepeat (C function): Sequence Protocol. (line 39) +* PySequence_ITEM (C function): Sequence Protocol. (line 166) +* PySequence_Length (C function): Sequence Protocol. (line 14) +* PySequence_List (C function): Sequence Protocol. (line 112) +* PySequence_Repeat (C function): Sequence Protocol. (line 25) +* PySequence_SetItem (C function): Sequence Protocol. (line 58) +* PySequence_SetSlice (C function): Sequence Protocol. (line 75) +* PySequence_Size (C function): Sequence Protocol. (line 14) +* PySequence_Tuple (C function): Sequence Protocol. (line 118) +* PySetObject (C type): Set Objects. (line 19) +* PySet_Add (C function): Set Objects. (line 108) +* PySet_Check (C function): Set Objects. (line 45) +* PySet_Clear (C function): Set Objects. (line 140) +* PySet_ClearFreeList (C function): Set Objects. (line 144) +* PySet_Contains (C function): Set Objects. (line 99) +* PySet_Discard (C function): Set Objects. (line 123) +* PySet_GET_SIZE (C function): Set Objects. (line 95) +* PySet_New (C function): Set Objects. (line 70) +* PySet_Pop (C function): Set Objects. (line 133) +* PySet_Size (C function): Set Objects. (line 88) +* PySet_Type (C variable): Set Objects. (line 31) +* PySignal_SetWakeupFd (C function): Signal Handling<2>. (line 28) +* PySlice_AdjustIndices (C function): Slice Objects. (line 101) +* PySlice_Check (C function): Slice Objects. (line 11) +* PySlice_GetIndices (C function): Slice Objects. (line 25) +* PySlice_GetIndicesEx (C function): Slice Objects. (line 44) +* PySlice_New (C function): Slice Objects. (line 15) +* PySlice_Type (C variable): Slice Objects. (line 6) +* PySlice_Unpack (C function): Slice Objects. (line 87) +* PyState_AddModule (C function): Module lookup. (line 23) +* PyState_FindModule (C function): Module lookup. (line 15) +* PyState_RemoveModule (C function): Module lookup. (line 45) +* PyStatus (C type): PyStatus. (line 6) +* PyStatus.err_msg (C member): PyStatus. (line 20) +* PyStatus.exitcode (C member): PyStatus. (line 16) +* PyStatus.func (C member): PyStatus. (line 24) +* PyStatus_Error (C function): PyStatus. (line 34) +* PyStatus_Exception (C function): PyStatus. (line 48) +* PyStatus_Exit (C function): PyStatus. (line 42) +* PyStatus_IsError (C function): PyStatus. (line 54) +* PyStatus_IsExit (C function): PyStatus. (line 58) +* PyStatus_NoMemory (C function): PyStatus. (line 38) +* PyStatus_Ok (C function): PyStatus. (line 30) +* PyStructSequence_Desc (C type): Struct Sequence Objects. + (line 34) +* PyStructSequence_Field (C type): Struct Sequence Objects. + (line 57) +* PyStructSequence_GetItem (C function): Struct Sequence Objects. + (line 86) +* PyStructSequence_GET_ITEM (C function): Struct Sequence Objects. + (line 93) +* PyStructSequence_InitType (C function): Struct Sequence Objects. + (line 19) +* PyStructSequence_InitType2 (C function): Struct Sequence Objects. + (line 25) +* PyStructSequence_New (C function): Struct Sequence Objects. + (line 81) +* PyStructSequence_NewType (C function): Struct Sequence Objects. + (line 11) +* PyStructSequence_SetItem (C function): Struct Sequence Objects. + (line 99) +* PyStructSequence_SET_ITEM (C function): Struct Sequence Objects. + (line 108) +* PyStructSequence_UnnamedField (C variable): Struct Sequence Objects. + (line 77) +* PySys_AddAuditHook (C function): System Functions. (line 129) +* PySys_AddWarnOption (C function): System Functions. (line 27) +* PySys_AddWarnOptionUnicode (C function): System Functions. (line 33) +* PySys_AddXOption (C function): System Functions. (line 87) +* PySys_Audit (C function): System Functions. (line 103) +* PySys_FormatStderr (C function): System Functions. (line 80) +* PySys_FormatStdout (C function): System Functions. (line 72) +* PySys_GetObject (C function): System Functions. (line 11) +* PySys_GetXOptions (C function): System Functions. (line 96) +* PySys_ResetWarnOptions (C function): System Functions. (line 22) +* PySys_SetArgv (C function): Process-wide parameters. + (line 255) +* PySys_SetArgv(): Initializing and finalizing the interpreter. + (line 8) +* PySys_SetArgvEx (C function): Process-wide parameters. + (line 214) +* PySys_SetArgvEx(): Embedding Python<2>. + (line 17) +* PySys_SetArgvEx() <1>: Initializing and finalizing the interpreter. + (line 8) +* PySys_SetObject (C function): System Functions. (line 16) +* PySys_SetPath (C function): System Functions. (line 43) +* PySys_WriteStderr (C function): System Functions. (line 67) +* PySys_WriteStdout (C function): System Functions. (line 49) +* Python 3000: Glossary. (line 1075) +* Python Editor: IDLE<3>. (line 8) +* Python Enhancement Proposals; PEP 1: New Development Process. + (line 67) +* Python Enhancement Proposals; PEP 1 <1>: New Development Process. + (line 82) +* Python Enhancement Proposals; PEP 1 <2>: Glossary. (line 1041) +* Python Enhancement Proposals; PEP 100: Unicode<2>. (line 15) +* Python Enhancement Proposals; PEP 11: PEP 538 Legacy C Locale Coercion. + (line 19) +* Python Enhancement Proposals; PEP 11 <1>: Unsupported Operating Systems. + (line 6) +* Python Enhancement Proposals; PEP 11 <2>: Library Changes. (line 15) +* Python Enhancement Proposals; PEP 11 <3>: Using Python on Windows. + (line 18) +* Python Enhancement Proposals; PEP 11 <4>: Other Platforms. (line 8) +* Python Enhancement Proposals; PEP 201: New Development Process. + (line 86) +* Python Enhancement Proposals; PEP 205: PEP 205 Weak References. + (line 82) +* Python Enhancement Proposals; PEP 205 <1>: weakref — Weak references. + (line 311) +* Python Enhancement Proposals; PEP 207: PEP 207 Rich Comparisons. + (line 71) +* Python Enhancement Proposals; PEP 207 <1>: PEP 207 Rich Comparisons. + (line 77) +* Python Enhancement Proposals; PEP 208: PEP 208 New Coercion Model. + (line 30) +* Python Enhancement Proposals; PEP 217: PEP 217 Interactive Display Hook. + (line 27) +* Python Enhancement Proposals; PEP 218: PEP 218 Built-In Set Objects. + (line 50) +* Python Enhancement Proposals; PEP 218 <1>: Other Language Changes<12>. + (line 12) +* Python Enhancement Proposals; PEP 218 <2>: PEP 218 A Standard Set Datatype. + (line 80) +* Python Enhancement Proposals; PEP 227: PEP 227 Nested Scopes. + (line 96) +* Python Enhancement Proposals; PEP 227 <1>: PEP 227 Nested Scopes<2>. + (line 96) +* Python Enhancement Proposals; PEP 227 <2>: __future__ — Future statement definitions. + (line 71) +* Python Enhancement Proposals; PEP 229: PEP 229 New Build System. + (line 39) +* Python Enhancement Proposals; PEP 230: PEP 230 Warning Framework. + (line 73) +* Python Enhancement Proposals; PEP 232: PEP 232 Function Attributes. + (line 36) +* Python Enhancement Proposals; PEP 234: PEP 234 Iterators. (line 124) +* Python Enhancement Proposals; PEP 235: Introduction<12>. (line 37) +* Python Enhancement Proposals; PEP 236: PEP 227 Nested Scopes<2>. + (line 87) +* Python Enhancement Proposals; PEP 236 <1>: PEP 227 Nested Scopes<2>. + (line 88) +* Python Enhancement Proposals; PEP 236 <2>: PEP 236 __future__ Directives. + (line 29) +* Python Enhancement Proposals; PEP 236 <3>: Future statements. + (line 85) +* Python Enhancement Proposals; PEP 237: Integers. (line 6) +* Python Enhancement Proposals; PEP 237 <1>: PEP 237 Unifying Long Integers and Integers. + (line 22) +* Python Enhancement Proposals; PEP 237 <2>: Other Language Changes<12>. + (line 18) +* Python Enhancement Proposals; PEP 237 <3>: PEP 237 Unifying Long Integers and Integers<2>. + (line 37) +* Python Enhancement Proposals; PEP 237 <4>: printf-style String Formatting. + (line 182) +* Python Enhancement Proposals; PEP 237 <5>: printf-style Bytes Formatting. + (line 189) +* Python Enhancement Proposals; PEP 238: Integers. (line 10) +* Python Enhancement Proposals; PEP 238 <1>: PEP 238 Changing the Division Operator. + (line 21) +* Python Enhancement Proposals; PEP 238 <2>: PEP 238 Changing the Division Operator. + (line 27) +* Python Enhancement Proposals; PEP 238 <3>: PEP 238 Changing the Division Operator. + (line 68) +* Python Enhancement Proposals; PEP 238 <4>: __future__ — Future statement definitions. + (line 77) +* Python Enhancement Proposals; PEP 238 <5>: The Very High Level Layer. + (line 425) +* Python Enhancement Proposals; PEP 238 <6>: Glossary. (line 458) +* Python Enhancement Proposals; PEP 241: PEP 241 Metadata in Python Packages. + (line 15) +* Python Enhancement Proposals; PEP 241 <1>: PEP 241 Metadata in Python Packages. + (line 28) +* Python Enhancement Proposals; PEP 241 <2>: PEP 241 Metadata in Python Packages. + (line 35) +* Python Enhancement Proposals; PEP 243: PEP 241 Metadata in Python Packages. + (line 39) +* Python Enhancement Proposals; PEP 247: hmac<2>. (line 20) +* Python Enhancement Proposals; PEP 249: The sqlite3 package. + (line 18) +* Python Enhancement Proposals; PEP 249 <1>: The sqlite3 package. + (line 110) +* Python Enhancement Proposals; PEP 249 <2>: sqlite3 — DB-API 2 0 interface for SQLite databases. + (line 18) +* Python Enhancement Proposals; PEP 249 <3>: sqlite3 — DB-API 2 0 interface for SQLite databases. + (line 113) +* Python Enhancement Proposals; PEP 252: Related Links. (line 17) +* Python Enhancement Proposals; PEP 252 <1>: Related Links. (line 17) +* Python Enhancement Proposals; PEP 252 <2>: Implementing Descriptors. + (line 24) +* Python Enhancement Proposals; PEP 253: Old and New Classes. + (line 28) +* Python Enhancement Proposals; PEP 253 <1>: Multiple Inheritance The Diamond Rule. + (line 8) +* Python Enhancement Proposals; PEP 253 <2>: Related Links. (line 17) +* Python Enhancement Proposals; PEP 253 <3>: Related Links. (line 19) +* Python Enhancement Proposals; PEP 253 <4>: Related Links. (line 21) +* Python Enhancement Proposals; PEP 255: PEP 255 Simple Generators. + (line 45) +* Python Enhancement Proposals; PEP 255 <1>: PEP 255 Simple Generators. + (line 131) +* Python Enhancement Proposals; PEP 255 <2>: PEP 255 Simple Generators<2>. + (line 43) +* Python Enhancement Proposals; PEP 255 <3>: PEP 255 Simple Generators<2>. + (line 129) +* Python Enhancement Proposals; PEP 255 <4>: Yield expressions. + (line 88) +* Python Enhancement Proposals; PEP 255 <5>: __future__ — Future statement definitions. + (line 74) +* Python Enhancement Proposals; PEP 261: Unicode Changes. (line 18) +* Python Enhancement Proposals; PEP 261 <1>: Unicode Changes. + (line 61) +* Python Enhancement Proposals; PEP 263: Other Language Changes<11>. + (line 98) +* Python Enhancement Proposals; PEP 263 <1>: PEP 263 Source Code Encodings. + (line 29) +* Python Enhancement Proposals; PEP 263 <2>: Introduction<12>. + (line 41) +* Python Enhancement Proposals; PEP 263 <3>: Tokenizing Input. + (line 35) +* Python Enhancement Proposals; PEP 263 <4>: Tokenizing Input. + (line 89) +* Python Enhancement Proposals; PEP 263 <5>: Unicode Literals in Python Source Code. + (line 48) +* Python Enhancement Proposals; PEP 264: Other Changes and Fixes<3>. + (line 55) +* Python Enhancement Proposals; PEP 273: PEP 273 Importing Modules from ZIP Archives. + (line 41) +* Python Enhancement Proposals; PEP 273 <1>: PEP 273 Importing Modules from ZIP Archives. + (line 44) +* Python Enhancement Proposals; PEP 273 <2>: zipimport — Import modules from Zip archives. + (line 42) +* Python Enhancement Proposals; PEP 273 <3>: zipimport — Import modules from Zip archives. + (line 45) +* Python Enhancement Proposals; PEP 274: New Syntax. (line 38) +* Python Enhancement Proposals; PEP 275: Why isn’t there a switch or case statement in Python?. + (line 8) +* Python Enhancement Proposals; PEP 277: PEP 277 Unicode file name support for Windows NT. + (line 32) +* Python Enhancement Proposals; PEP 278: PEP 278 Universal Newline Support. + (line 33) +* Python Enhancement Proposals; PEP 278 <1>: Glossary. (line 1287) +* Python Enhancement Proposals; PEP 279: PEP 279 enumerate. (line 27) +* Python Enhancement Proposals; PEP 282: PEP 282 The logging Package. + (line 101) +* Python Enhancement Proposals; PEP 282 <1>: PEP 282 The logging Package. + (line 107) +* Python Enhancement Proposals; PEP 282 <2>: Archiving operations. + (line 45) +* Python Enhancement Proposals; PEP 282 <3>: Integration with the warnings module. + (line 36) +* Python Enhancement Proposals; PEP 282 <4>: distutils log — Simple PEP 282-style logging. + (line 6) +* Python Enhancement Proposals; PEP 285: PEP 285 A Boolean Type. + (line 47) +* Python Enhancement Proposals; PEP 285 <1>: PEP 285 A Boolean Type. + (line 70) +* Python Enhancement Proposals; PEP 288: PEP 342 New Generator Features. + (line 146) +* Python Enhancement Proposals; PEP 289: PEP 289 Generator Expressions. + (line 56) +* Python Enhancement Proposals; PEP 289 <1>: Other Language Changes<12>. + (line 15) +* Python Enhancement Proposals; PEP 289 <2>: Python documentation. + (line 12) +* Python Enhancement Proposals; PEP 292: PEP 292 Simpler String Substitutions. + (line 45) +* Python Enhancement Proposals; PEP 292 <1>: Template strings. + (line 6) +* Python Enhancement Proposals; PEP 293: PEP 293 Codec Error Handling Callbacks. + (line 33) +* Python Enhancement Proposals; PEP 3000: Python 3 0. (line 43) +* Python Enhancement Proposals; PEP 301: PEP 301 Package Index and Metadata for Distutils. + (line 45) +* Python Enhancement Proposals; PEP 301 <1>: distutils command register — Register a module with the Python Package Index. + (line 7) +* Python Enhancement Proposals; PEP 302: Using importlib as the Implementation of Import. + (line 13) +* Python Enhancement Proposals; PEP 302 <1>: Visible Changes. + (line 18) +* Python Enhancement Proposals; PEP 302 <2>: Porting Python code. + (line 71) +* Python Enhancement Proposals; PEP 302 <3>: New Improved and Deprecated Modules<2>. + (line 106) +* Python Enhancement Proposals; PEP 302 <4>: New Improved and Removed Modules. + (line 289) +* Python Enhancement Proposals; PEP 302 <5>: PEP 273 Importing Modules from ZIP Archives. + (line 46) +* Python Enhancement Proposals; PEP 302 <6>: PEP 302 New Import Hooks. + (line 13) +* Python Enhancement Proposals; PEP 302 <7>: PEP 302 New Import Hooks. + (line 38) +* Python Enhancement Proposals; PEP 302 <8>: PEP 302 New Import Hooks. + (line 62) +* Python Enhancement Proposals; PEP 302 <9>: The import system. + (line 42) +* Python Enhancement Proposals; PEP 302 <10>: References. (line 10) +* Python Enhancement Proposals; PEP 302 <11>: Built-in Functions. + (line 1783) +* Python Enhancement Proposals; PEP 302 <12>: linecache — Random access to text lines. + (line 31) +* Python Enhancement Proposals; PEP 302 <13>: sys — System-specific parameters and functions. + (line 1091) +* Python Enhancement Proposals; PEP 302 <14>: sys — System-specific parameters and functions. + (line 1101) +* Python Enhancement Proposals; PEP 302 <15>: zipimport — Import modules from Zip archives. + (line 47) +* Python Enhancement Proposals; PEP 302 <16>: zipimport — Import modules from Zip archives. + (line 49) +* Python Enhancement Proposals; PEP 302 <17>: pkgutil — Package extension utility. + (line 55) +* Python Enhancement Proposals; PEP 302 <18>: pkgutil — Package extension utility. + (line 57) +* Python Enhancement Proposals; PEP 302 <19>: pkgutil — Package extension utility. + (line 58) +* Python Enhancement Proposals; PEP 302 <20>: pkgutil — Package extension utility. + (line 66) +* Python Enhancement Proposals; PEP 302 <21>: pkgutil — Package extension utility. + (line 74) +* Python Enhancement Proposals; PEP 302 <22>: pkgutil — Package extension utility. + (line 87) +* Python Enhancement Proposals; PEP 302 <23>: pkgutil — Package extension utility. + (line 103) +* Python Enhancement Proposals; PEP 302 <24>: pkgutil — Package extension utility. + (line 118) +* Python Enhancement Proposals; PEP 302 <25>: pkgutil — Package extension utility. + (line 138) +* Python Enhancement Proposals; PEP 302 <26>: pkgutil — Package extension utility. + (line 159) +* Python Enhancement Proposals; PEP 302 <27>: pkgutil — Package extension utility. + (line 198) +* Python Enhancement Proposals; PEP 302 <28>: runpy — Locating and executing Python modules. + (line 32) +* Python Enhancement Proposals; PEP 302 <29>: Introduction<12>. + (line 45) +* Python Enhancement Proposals; PEP 302 <30>: importlib abc – Abstract base classes related to import. + (line 38) +* Python Enhancement Proposals; PEP 302 <31>: importlib abc – Abstract base classes related to import. + (line 160) +* Python Enhancement Proposals; PEP 302 <32>: importlib abc – Abstract base classes related to import. + (line 361) +* Python Enhancement Proposals; PEP 302 <33>: importlib abc – Abstract base classes related to import. + (line 385) +* Python Enhancement Proposals; PEP 302 <34>: importlib abc – Abstract base classes related to import. + (line 456) +* Python Enhancement Proposals; PEP 302 <35>: imp — Access the import internals. + (line 347) +* Python Enhancement Proposals; PEP 302 <36>: Glossary. (line 450) +* Python Enhancement Proposals; PEP 302 <37>: Glossary. (line 777) +* Python Enhancement Proposals; PEP 305: PEP 305 Comma-separated Files. + (line 42) +* Python Enhancement Proposals; PEP 305 <1>: csv — CSV File Reading and Writing. + (line 38) +* Python Enhancement Proposals; PEP 307: PEP 307 Pickle Enhancements. + (line 8) +* Python Enhancement Proposals; PEP 307 <1>: PEP 307 Pickle Enhancements. + (line 30) +* Python Enhancement Proposals; PEP 307 <2>: PEP 307 Pickle Enhancements. + (line 42) +* Python Enhancement Proposals; PEP 307 <3>: Data stream format. + (line 32) +* Python Enhancement Proposals; PEP 308: PEP 308 Conditional Expressions. + (line 72) +* Python Enhancement Proposals; PEP 308 <1>: PEP 308 Conditional Expressions. + (line 78) +* Python Enhancement Proposals; PEP 308 <2>: Conditional expressions. + (line 17) +* Python Enhancement Proposals; PEP 309: PEP 309 Partial Function Application. + (line 73) +* Python Enhancement Proposals; PEP 3100: Python 3 0. (line 44) +* Python Enhancement Proposals; PEP 3101: PEP 3101 A New Approach To String Formatting. + (line 6) +* Python Enhancement Proposals; PEP 3101 <1>: PEP 3101 A New Approach To String Formatting. + (line 9) +* Python Enhancement Proposals; PEP 3101 <2>: PEP 3101 Advanced String Formatting. + (line 162) +* Python Enhancement Proposals; PEP 3101 <3>: Custom String Formatting. + (line 8) +* Python Enhancement Proposals; PEP 3101 <4>: Custom String Formatting. + (line 60) +* Python Enhancement Proposals; PEP 3102: New Syntax. (line 13) +* Python Enhancement Proposals; PEP 3104: New Syntax. (line 24) +* Python Enhancement Proposals; PEP 3104 <1>: The nonlocal statement. + (line 26) +* Python Enhancement Proposals; PEP 3105: Print Is A Function. + (line 8) +* Python Enhancement Proposals; PEP 3105 <1>: PEP 3105 print As a Function. + (line 35) +* Python Enhancement Proposals; PEP 3105 <2>: __future__ — Future statement definitions. + (line 87) +* Python Enhancement Proposals; PEP 3106: PEP 3106 Dictionary Views. + (line 58) +* Python Enhancement Proposals; PEP 3107: PEP 563 Postponed Evaluation of Annotations. + (line 7) +* Python Enhancement Proposals; PEP 3107 <1>: PEP 484 - Type Hints. + (line 7) +* Python Enhancement Proposals; PEP 3107 <2>: New Syntax. (line 6) +* Python Enhancement Proposals; PEP 3107 <3>: Function Annotations. + (line 7) +* Python Enhancement Proposals; PEP 3107 <4>: Function definitions. + (line 123) +* Python Enhancement Proposals; PEP 3108: Library Changes. (line 7) +* Python Enhancement Proposals; PEP 3108 <1>: Library Changes. + (line 17) +* Python Enhancement Proposals; PEP 3108 <2>: Library Changes. + (line 95) +* Python Enhancement Proposals; PEP 3109: Changed Syntax. (line 6) +* Python Enhancement Proposals; PEP 3109 <1>: Changes To Exceptions. + (line 29) +* Python Enhancement Proposals; PEP 3110: Changed Syntax. (line 16) +* Python Enhancement Proposals; PEP 3110 <1>: Changes To Exceptions. + (line 35) +* Python Enhancement Proposals; PEP 3110 <2>: PEP 3110 Exception-Handling Changes. + (line 48) +* Python Enhancement Proposals; PEP 3111: Builtins. (line 12) +* Python Enhancement Proposals; PEP 3112: PEP 3112 Byte Literals. + (line 75) +* Python Enhancement Proposals; PEP 3112 <1>: __future__ — Future statement definitions. + (line 90) +* Python Enhancement Proposals; PEP 3113: Removed Syntax. (line 6) +* Python Enhancement Proposals; PEP 3114: Operators And Special Methods. + (line 19) +* Python Enhancement Proposals; PEP 3115: types<4>. (line 10) +* Python Enhancement Proposals; PEP 3115 <1>: Changed Syntax. + (line 18) +* Python Enhancement Proposals; PEP 3115 <2>: Preparing the class namespace. + (line 21) +* Python Enhancement Proposals; PEP 3115 <3>: Class definitions. + (line 71) +* Python Enhancement Proposals; PEP 3115 <4>: Dynamic Type Creation. + (line 54) +* Python Enhancement Proposals; PEP 3116: Optimizations<7>. (line 8) +* Python Enhancement Proposals; PEP 3116 <1>: PEP 3116 New I/O Library. + (line 69) +* Python Enhancement Proposals; PEP 3116 <2>: Glossary. (line 1287) +* Python Enhancement Proposals; PEP 3118: Pickle protocol 5 with out-of-band data buffers. + (line 13) +* Python Enhancement Proposals; PEP 3118 <1>: PEP 3118 New memoryview implementation and buffer protocol documentation. + (line 6) +* Python Enhancement Proposals; PEP 3118 <2>: API changes. (line 29) +* Python Enhancement Proposals; PEP 3118 <3>: Build and C API Changes<4>. + (line 8) +* Python Enhancement Proposals; PEP 3118 <4>: Build and C API Changes<7>. + (line 12) +* Python Enhancement Proposals; PEP 3118 <5>: PEP 3118 Revised Buffer Protocol. + (line 46) +* Python Enhancement Proposals; PEP 3118 <6>: Memory Views. (line 110) +* Python Enhancement Proposals; PEP 3119: PEP 3119 Abstract Base Classes. + (line 139) +* Python Enhancement Proposals; PEP 3119 <1>: Customizing instance and subclass checks. + (line 35) +* Python Enhancement Proposals; PEP 3119 <2>: Collections Abstract Base Classes. + (line 316) +* Python Enhancement Proposals; PEP 3119 <3>: abc — Abstract Base Classes. + (line 11) +* Python Enhancement Proposals; PEP 3119 <4>: Object Protocol. + (line 218) +* Python Enhancement Proposals; PEP 3119 <5>: Object Protocol. + (line 238) +* Python Enhancement Proposals; PEP 3120: Text Vs Data Instead Of Unicode Vs 8-bit. + (line 107) +* Python Enhancement Proposals; PEP 3120 <1>: Lexical analysis. + (line 12) +* Python Enhancement Proposals; PEP 3120 <2>: Introduction<12>. + (line 77) +* Python Enhancement Proposals; PEP 3121: Build and C API Changes<7>. + (line 14) +* Python Enhancement Proposals; PEP 3121 <1>: Initializing C modules. + (line 55) +* Python Enhancement Proposals; PEP 3123: Build and C API Changes<7>. + (line 16) +* Python Enhancement Proposals; PEP 3127: PEP 3127 Integer Literal Support and Syntax. + (line 47) +* Python Enhancement Proposals; PEP 3129: PEP 3129 Class Decorators. + (line 24) +* Python Enhancement Proposals; PEP 3129 <1>: Class definitions. + (line 77) +* Python Enhancement Proposals; PEP 3131: Text Vs Data Instead Of Unicode Vs 8-bit. + (line 109) +* Python Enhancement Proposals; PEP 3131 <1>: Identifiers and keywords. + (line 10) +* Python Enhancement Proposals; PEP 3131 <2>: Identifiers and keywords. + (line 19) +* Python Enhancement Proposals; PEP 3132: New Syntax. (line 28) +* Python Enhancement Proposals; PEP 3132 <1>: Assignment statements. + (line 159) +* Python Enhancement Proposals; PEP 3134: Changed Syntax. (line 6) +* Python Enhancement Proposals; PEP 3134 <1>: Changes To Exceptions. + (line 40) +* Python Enhancement Proposals; PEP 3134 <2>: Changes To Exceptions. + (line 60) +* Python Enhancement Proposals; PEP 3135: Builtins. (line 6) +* Python Enhancement Proposals; PEP 3135 <1>: Creating the class object. + (line 54) +* Python Enhancement Proposals; PEP 3137: PEP 3137 The memoryview Object. + (line 49) +* Python Enhancement Proposals; PEP 3138: Text Vs Data Instead Of Unicode Vs 8-bit. + (line 103) +* Python Enhancement Proposals; PEP 314: PEP 314 Metadata for Python Software Packages v1 1. + (line 42) +* Python Enhancement Proposals; PEP 314 <1>: distutils core — Core Distutils functionality. + (line 103) +* Python Enhancement Proposals; PEP 3141: PEP 3141 A Type Hierarchy for Numbers. + (line 43) +* Python Enhancement Proposals; PEP 3141 <1>: numbers — Numeric abstract base classes. + (line 10) +* Python Enhancement Proposals; PEP 3141 <2>: abc — Abstract Base Classes. + (line 12) +* Python Enhancement Proposals; PEP 3144: ipaddress<4>. (line 10) +* Python Enhancement Proposals; PEP 3147: PEP 421 Adding sys implementation. + (line 21) +* Python Enhancement Proposals; PEP 3147 <1>: PEP 3147 PYC Repository Directories. + (line 69) +* Python Enhancement Proposals; PEP 3147 <2>: “Compiled” Python files. + (line 47) +* Python Enhancement Proposals; PEP 3147 <3>: Import-related module attributes. + (line 79) +* Python Enhancement Proposals; PEP 3147 <4>: test support — Utilities for the Python test suite. + (line 192) +* Python Enhancement Proposals; PEP 3147 <5>: runpy — Locating and executing Python modules. + (line 87) +* Python Enhancement Proposals; PEP 3147 <6>: Introduction<12>. + (line 81) +* Python Enhancement Proposals; PEP 3147 <7>: importlib util – Utility code for importers. + (line 24) +* Python Enhancement Proposals; PEP 3147 <8>: importlib util – Utility code for importers. + (line 60) +* Python Enhancement Proposals; PEP 3147 <9>: importlib util – Utility code for importers. + (line 64) +* Python Enhancement Proposals; PEP 3147 <10>: py_compile — Compile Python source files. + (line 30) +* Python Enhancement Proposals; PEP 3147 <11>: py_compile — Compile Python source files. + (line 69) +* Python Enhancement Proposals; PEP 3147 <12>: Command-line use. + (line 56) +* Python Enhancement Proposals; PEP 3147 <13>: Public functions. + (line 36) +* Python Enhancement Proposals; PEP 3147 <14>: Public functions. + (line 101) +* Python Enhancement Proposals; PEP 3147 <15>: imp — Access the import internals. + (line 200) +* Python Enhancement Proposals; PEP 3147 <16>: imp — Access the import internals. + (line 207) +* Python Enhancement Proposals; PEP 3147 <17>: imp — Access the import internals. + (line 230) +* Python Enhancement Proposals; PEP 3147 <18>: imp — Access the import internals. + (line 234) +* Python Enhancement Proposals; PEP 3147 <19>: imp — Access the import internals. + (line 246) +* Python Enhancement Proposals; PEP 3147 <20>: Importing Modules<2>. + (line 199) +* Python Enhancement Proposals; PEP 3147 <21>: How do I create a pyc file?. + (line 12) +* Python Enhancement Proposals; PEP 3147 <22>: distutils util — Miscellaneous other utility functions. + (line 132) +* Python Enhancement Proposals; PEP 3148: PEP 3148 The concurrent futures module. + (line 51) +* Python Enhancement Proposals; PEP 3148 <1>: Module Functions. + (line 61) +* Python Enhancement Proposals; PEP 3149: PEP 3149 ABI Version Tagged so Files. + (line 33) +* Python Enhancement Proposals; PEP 3149 <1>: sys — System-specific parameters and functions. + (line 15) +* Python Enhancement Proposals; PEP 3151: PEP 3151 Reworking the OS and IO exception hierarchy. + (line 84) +* Python Enhancement Proposals; PEP 3151 <1>: OS exceptions. (line 114) +* Python Enhancement Proposals; PEP 3151 <2>: Exceptions<11>. + (line 10) +* Python Enhancement Proposals; PEP 3151 <3>: select — Waiting for I/O completion. + (line 28) +* Python Enhancement Proposals; PEP 3151 <4>: resource — Resource usage information. + (line 21) +* Python Enhancement Proposals; PEP 3151 <5>: Standard Exceptions. + (line 181) +* Python Enhancement Proposals; PEP 3154: Summary – Release Highlights<2>. + (line 57) +* Python Enhancement Proposals; PEP 3154 <1>: pickle<4>. (line 16) +* Python Enhancement Proposals; PEP 3154 <2>: Data stream format. + (line 42) +* Python Enhancement Proposals; PEP 3155: PEP 3155 Qualified name for classes and functions. + (line 62) +* Python Enhancement Proposals; PEP 3155 <1>: Glossary. (line 1104) +* Python Enhancement Proposals; PEP 3156: Summary – Release Highlights<2>. + (line 31) +* Python Enhancement Proposals; PEP 3156 <1>: Summary – Release Highlights<2>. + (line 44) +* Python Enhancement Proposals; PEP 3156 <2>: asyncio<6>. (line 6) +* Python Enhancement Proposals; PEP 3156 <3>: asyncio<6>. (line 17) +* Python Enhancement Proposals; PEP 3156 <4>: selectors<2>. (line 6) +* Python Enhancement Proposals; PEP 318: PEP 318 Decorators for Functions and Methods. + (line 101) +* Python Enhancement Proposals; PEP 318 <1>: PEP 318 Decorators for Functions and Methods. + (line 125) +* Python Enhancement Proposals; PEP 318 <2>: Other Language Changes<12>. + (line 9) +* Python Enhancement Proposals; PEP 318 <3>: Class definitions. + (line 80) +* Python Enhancement Proposals; PEP 322: PEP 322 Reverse Iteration. + (line 35) +* Python Enhancement Proposals; PEP 322 <1>: Other Language Changes<12>. + (line 13) +* Python Enhancement Proposals; PEP 324: PEP 324 New subprocess Module. + (line 86) +* Python Enhancement Proposals; PEP 324 <1>: subprocess — Subprocess management. + (line 25) +* Python Enhancement Proposals; PEP 325: PEP 342 New Generator Features. + (line 147) +* Python Enhancement Proposals; PEP 327: The Context type. (line 56) +* Python Enhancement Proposals; PEP 328: Porting Python code. + (line 36) +* Python Enhancement Proposals; PEP 328 <1>: Removed Syntax. (line 29) +* Python Enhancement Proposals; PEP 328 <2>: PEP 328 Absolute and Relative Imports. + (line 6) +* Python Enhancement Proposals; PEP 328 <3>: PEP 328 Absolute and Relative Imports. + (line 82) +* Python Enhancement Proposals; PEP 328 <4>: PEP 328 Multi-line Imports. + (line 35) +* Python Enhancement Proposals; PEP 328 <5>: Other Language Changes<12>. + (line 21) +* Python Enhancement Proposals; PEP 328 <6>: References. (line 20) +* Python Enhancement Proposals; PEP 328 <7>: Built-in Functions. + (line 1801) +* Python Enhancement Proposals; PEP 328 <8>: __future__ — Future statement definitions. + (line 80) +* Python Enhancement Proposals; PEP 328 <9>: Introduction<12>. + (line 49) +* Python Enhancement Proposals; PEP 331: PEP 331 Locale-Independent Float/String Conversions. + (line 38) +* Python Enhancement Proposals; PEP 333: The wsgiref package. + (line 8) +* Python Enhancement Proposals; PEP 333 <1>: The wsgiref package. + (line 31) +* Python Enhancement Proposals; PEP 3333: PEP 3333 Python Web Server Gateway Interface v1 0 1. + (line 50) +* Python Enhancement Proposals; PEP 3333 <1>: wsgiref — WSGI Utilities and Reference Implementation. + (line 24) +* Python Enhancement Proposals; PEP 3333 <2>: wsgiref util – WSGI environment utilities. + (line 8) +* Python Enhancement Proposals; PEP 3333 <3>: wsgiref util – WSGI environment utilities. + (line 10) +* Python Enhancement Proposals; PEP 3333 <4>: wsgiref util – WSGI environment utilities. + (line 28) +* Python Enhancement Proposals; PEP 3333 <5>: wsgiref util – WSGI environment utilities. + (line 74) +* Python Enhancement Proposals; PEP 3333 <6>: wsgiref headers – WSGI response header tools. + (line 12) +* Python Enhancement Proposals; PEP 3333 <7>: wsgiref simple_server – a simple WSGI HTTP server. + (line 21) +* Python Enhancement Proposals; PEP 3333 <8>: wsgiref simple_server – a simple WSGI HTTP server. + (line 96) +* Python Enhancement Proposals; PEP 3333 <9>: wsgiref validate — WSGI conformance checker. + (line 13) +* Python Enhancement Proposals; PEP 3333 <10>: wsgiref validate — WSGI conformance checker. + (line 41) +* Python Enhancement Proposals; PEP 3333 <11>: wsgiref handlers – server/gateway base classes. + (line 234) +* Python Enhancement Proposals; PEP 3333 <12>: wsgiref handlers – server/gateway base classes. + (line 250) +* Python Enhancement Proposals; PEP 3333 <13>: wsgiref handlers – server/gateway base classes. + (line 257) +* Python Enhancement Proposals; PEP 3333 <14>: wsgiref handlers – server/gateway base classes. + (line 266) +* Python Enhancement Proposals; PEP 3333 <15>: wsgiref handlers – server/gateway base classes. + (line 308) +* Python Enhancement Proposals; PEP 338: PEP 338 Executing Modules as Scripts. + (line 21) +* Python Enhancement Proposals; PEP 338 <1>: Interface options. + (line 100) +* Python Enhancement Proposals; PEP 338 <2>: References. (line 24) +* Python Enhancement Proposals; PEP 338 <3>: runpy — Locating and executing Python modules. + (line 171) +* Python Enhancement Proposals; PEP 339: Build and C API Changes<10>. + (line 45) +* Python Enhancement Proposals; PEP 341: PEP 341 Unified try/except/finally. + (line 44) +* Python Enhancement Proposals; PEP 342: PEP 342 New Generator Features. + (line 42) +* Python Enhancement Proposals; PEP 342 <1>: PEP 342 New Generator Features. + (line 140) +* Python Enhancement Proposals; PEP 342 <2>: Porting to Python 2 5. + (line 15) +* Python Enhancement Proposals; PEP 342 <3>: Yield expressions. + (line 93) +* Python Enhancement Proposals; PEP 342 <4>: Collections Abstract Base Classes. + (line 152) +* Python Enhancement Proposals; PEP 342 <5>: Passing values into a generator. + (line 25) +* Python Enhancement Proposals; PEP 342 <6>: Python documentation. + (line 14) +* Python Enhancement Proposals; PEP 343: The contextlib module. + (line 64) +* Python Enhancement Proposals; PEP 343 <1>: PEP 342 New Generator Features. + (line 129) +* Python Enhancement Proposals; PEP 343 <2>: The contextlib module<2>. + (line 64) +* Python Enhancement Proposals; PEP 343 <3>: With Statement Context Managers. + (line 45) +* Python Enhancement Proposals; PEP 343 <4>: The with statement. + (line 98) +* Python Enhancement Proposals; PEP 343 <5>: Using a context manager as a function decorator. + (line 51) +* Python Enhancement Proposals; PEP 343 <6>: __future__ — Future statement definitions. + (line 84) +* Python Enhancement Proposals; PEP 343 <7>: Glossary. (line 259) +* Python Enhancement Proposals; PEP 347: Build and C API Changes<10>. + (line 10) +* Python Enhancement Proposals; PEP 352: Changes To Exceptions. + (line 9) +* Python Enhancement Proposals; PEP 352 <1>: Deprecations and Removals. + (line 9) +* Python Enhancement Proposals; PEP 352 <2>: PEP 352 Exceptions as New-Style Classes. + (line 57) +* Python Enhancement Proposals; PEP 353: PEP 353 Using ssize_t as the index type. + (line 48) +* Python Enhancement Proposals; PEP 353 <1>: PEP 353 Using ssize_t as the index type. + (line 54) +* Python Enhancement Proposals; PEP 353 <2>: Build and C API Changes<10>. + (line 20) +* Python Enhancement Proposals; PEP 356: What’s New in Python 2 5. + (line 10) +* Python Enhancement Proposals; PEP 357: PEP 357 The ‘__index__’ method. + (line 39) +* Python Enhancement Proposals; PEP 361: What’s New in Python 2 6. + (line 10) +* Python Enhancement Proposals; PEP 362: PEP 362 Function Signature Object. + (line 19) +* Python Enhancement Proposals; PEP 362 <1>: Introspecting callables with the Signature object. + (line 343) +* Python Enhancement Proposals; PEP 362 <2>: Glossary. (line 94) +* Python Enhancement Proposals; PEP 362 <3>: Glossary. (line 988) +* Python Enhancement Proposals; PEP 366: Visible Changes. (line 24) +* Python Enhancement Proposals; PEP 366 <1>: Import-related module attributes. + (line 30) +* Python Enhancement Proposals; PEP 366 <2>: Import-related module attributes. + (line 34) +* Python Enhancement Proposals; PEP 366 <3>: References. (line 17) +* Python Enhancement Proposals; PEP 366 <4>: References. (line 21) +* Python Enhancement Proposals; PEP 366 <5>: runpy — Locating and executing Python modules. + (line 175) +* Python Enhancement Proposals; PEP 366 <6>: Introduction<12>. + (line 53) +* Python Enhancement Proposals; PEP 370: PEP 370 Per-user site-packages Directory. + (line 36) +* Python Enhancement Proposals; PEP 370 <1>: Miscellaneous options. + (line 125) +* Python Enhancement Proposals; PEP 370 <2>: Environment variables. + (line 168) +* Python Enhancement Proposals; PEP 370 <3>: Environment variables. + (line 180) +* Python Enhancement Proposals; PEP 370 <4>: Command Line Interface<4>. + (line 37) +* Python Enhancement Proposals; PEP 371: PEP 371 The multiprocessing Package. + (line 145) +* Python Enhancement Proposals; PEP 372: PEP 372 Ordered Dictionaries. + (line 31) +* Python Enhancement Proposals; PEP 372 <1>: PEP 372 Adding an Ordered Dictionary to collections. + (line 97) +* Python Enhancement Proposals; PEP 373: The Future for Python 2 x. + (line 23) +* Python Enhancement Proposals; PEP 378: PEP 378 Format Specifier for Thousands Separator. + (line 32) +* Python Enhancement Proposals; PEP 378 <1>: PEP 378 Format Specifier for Thousands Separator<2>. + (line 37) +* Python Enhancement Proposals; PEP 378 <2>: Format Specification Mini-Language. + (line 103) +* Python Enhancement Proposals; PEP 380: PEP 380 Syntax for Delegating to a Subgenerator. + (line 66) +* Python Enhancement Proposals; PEP 380 <1>: Yield expressions. + (line 98) +* Python Enhancement Proposals; PEP 383: Error Handlers. (line 58) +* Python Enhancement Proposals; PEP 383 <1>: Socket families. + (line 15) +* Python Enhancement Proposals; PEP 383 <2>: Locale Encoding. + (line 14) +* Python Enhancement Proposals; PEP 383 <3>: Locale Encoding. + (line 50) +* Python Enhancement Proposals; PEP 383 <4>: File System Encoding. + (line 9) +* Python Enhancement Proposals; PEP 384: PEP 384 Defining a Stable ABI. + (line 24) +* Python Enhancement Proposals; PEP 384 <1>: Stable Application Binary Interface. + (line 36) +* Python Enhancement Proposals; PEP 385: Code Repository. (line 13) +* Python Enhancement Proposals; PEP 389: PEP 389 Argparse Command Line Parsing Module. + (line 92) +* Python Enhancement Proposals; PEP 389 <1>: PEP 389 The argparse Module for Parsing Command Lines. + (line 100) +* Python Enhancement Proposals; PEP 391: PEP 391 Dictionary Based Configuration for Logging. + (line 52) +* Python Enhancement Proposals; PEP 391 <1>: PEP 391 Dictionary-Based Configuration For Logging. + (line 94) +* Python Enhancement Proposals; PEP 392: What’s New In Python 3 2. + (line 16) +* Python Enhancement Proposals; PEP 393: PEP 393 Flexible String Representation. + (line 16) +* Python Enhancement Proposals; PEP 393 <1>: Functionality. (line 6) +* Python Enhancement Proposals; PEP 393 <2>: Performance and resource usage. + (line 29) +* Python Enhancement Proposals; PEP 393 <3>: Optimizations<5>. + (line 8) +* Python Enhancement Proposals; PEP 393 <4>: Build and C API Changes<4>. + (line 12) +* Python Enhancement Proposals; PEP 393 <5>: Deprecated Python modules functions and methods<4>. + (line 10) +* Python Enhancement Proposals; PEP 393 <6>: Deprecated functions and types of the C API<3>. + (line 6) +* Python Enhancement Proposals; PEP 393 <7>: Porting C code. (line 17) +* Python Enhancement Proposals; PEP 393 <8>: Encodings and Unicode. + (line 7) +* Python Enhancement Proposals; PEP 393 <9>: sys — System-specific parameters and functions. + (line 1016) +* Python Enhancement Proposals; PEP 393 <10>: Unicode Objects. + (line 6) +* Python Enhancement Proposals; PEP 393 <11>: Deprecated Py_UNICODE APIs. + (line 8) +* Python Enhancement Proposals; PEP 395: Open issues. (line 18) +* Python Enhancement Proposals; PEP 397: PEP 486 Make the Python Launcher aware of virtual environments. + (line 6) +* Python Enhancement Proposals; PEP 397 <1>: PEP 397 Python Launcher for Windows. + (line 31) +* Python Enhancement Proposals; PEP 397 <2>: Python Launcher for Windows. + (line 18) +* Python Enhancement Proposals; PEP 397 <3>: Creating virtual environments. + (line 195) +* Python Enhancement Proposals; PEP 398: What’s New In Python 3 3. + (line 13) +* Python Enhancement Proposals; PEP 4: Library Changes. (line 13) +* Python Enhancement Proposals; PEP 405: PEP 405 Virtual Environments. + (line 22) +* Python Enhancement Proposals; PEP 405 <1>: venv — Creation of virtual environments. + (line 19) +* Python Enhancement Proposals; PEP 409: PEP 409 Suppressing exception context. + (line 63) +* Python Enhancement Proposals; PEP 411: sys — System-specific parameters and functions. + (line 814) +* Python Enhancement Proposals; PEP 411 <1>: sys — System-specific parameters and functions. + (line 824) +* Python Enhancement Proposals; PEP 411 <2>: sys — System-specific parameters and functions. + (line 1427) +* Python Enhancement Proposals; PEP 411 <3>: sys — System-specific parameters and functions. + (line 1447) +* Python Enhancement Proposals; PEP 411 <4>: Glossary. (line 1071) +* Python Enhancement Proposals; PEP 412: PEP 412 Key-Sharing Dictionary. + (line 15) +* Python Enhancement Proposals; PEP 414: PEP 414 Explicit Unicode literals. + (line 17) +* Python Enhancement Proposals; PEP 414 <1>: String and Bytes literals. + (line 60) +* Python Enhancement Proposals; PEP 418: time<5>. (line 6) +* Python Enhancement Proposals; PEP 420: PEP 420 Implicit Namespace Packages. + (line 9) +* Python Enhancement Proposals; PEP 420 <1>: PEP 420 Implicit Namespace Packages. + (line 14) +* Python Enhancement Proposals; PEP 420 <2>: The import system. + (line 45) +* Python Enhancement Proposals; PEP 420 <3>: Namespace packages. + (line 27) +* Python Enhancement Proposals; PEP 420 <4>: module __path__. + (line 21) +* Python Enhancement Proposals; PEP 420 <5>: module __path__. + (line 21) +* Python Enhancement Proposals; PEP 420 <6>: References. (line 11) +* Python Enhancement Proposals; PEP 420 <7>: References. (line 13) +* Python Enhancement Proposals; PEP 420 <8>: References. (line 13) +* Python Enhancement Proposals; PEP 420 <9>: Introduction<12>. + (line 57) +* Python Enhancement Proposals; PEP 420 <10>: Glossary. (line 450) +* Python Enhancement Proposals; PEP 420 <11>: Glossary. (line 898) +* Python Enhancement Proposals; PEP 420 <12>: Glossary. (line 1046) +* Python Enhancement Proposals; PEP 421: SimpleNamespace. (line 16) +* Python Enhancement Proposals; PEP 421 <1>: sys — System-specific parameters and functions. + (line 926) +* Python Enhancement Proposals; PEP 421 <2>: sys — System-specific parameters and functions. + (line 931) +* Python Enhancement Proposals; PEP 424: Other Language Changes<5>. + (line 57) +* Python Enhancement Proposals; PEP 424 <1>: operator<2>. (line 8) +* Python Enhancement Proposals; PEP 427: Key terms. (line 36) +* Python Enhancement Proposals; PEP 428: Summary – Release Highlights<2>. + (line 39) +* Python Enhancement Proposals; PEP 428 <1>: pathlib<5>. (line 17) +* Python Enhancement Proposals; PEP 428 <2>: pathlib — Object-oriented filesystem paths. + (line 39) +* Python Enhancement Proposals; PEP 429: What’s New In Python 3 4. + (line 16) +* Python Enhancement Proposals; PEP 432: Other CPython implementation changes. + (line 7) +* Python Enhancement Proposals; PEP 432 <1>: Multi-Phase Initialization Private Provisional API. + (line 7) +* Python Enhancement Proposals; PEP 432 <2>: Multi-Phase Initialization Private Provisional API. + (line 57) +* Python Enhancement Proposals; PEP 434: PEP 434 IDLE Enhancement Exception for All Branches. + (line 6) +* Python Enhancement Proposals; PEP 435: Summary – Release Highlights<2>. + (line 36) +* Python Enhancement Proposals; PEP 435 <1>: enum<5>. (line 6) +* Python Enhancement Proposals; PEP 435 <2>: enum<5>. (line 15) +* Python Enhancement Proposals; PEP 436: Summary – Release Highlights<2>. + (line 120) +* Python Enhancement Proposals; PEP 436 <1>: PEP 436 Argument Clinic. + (line 6) +* Python Enhancement Proposals; PEP 436 <2>: PEP 436 Argument Clinic. + (line 28) +* Python Enhancement Proposals; PEP 441: zipapp<2>. (line 6) +* Python Enhancement Proposals; PEP 441 <1>: zipapp<2>. (line 24) +* Python Enhancement Proposals; PEP 442: Summary – Release Highlights<2>. + (line 113) +* Python Enhancement Proposals; PEP 442 <1>: Summary – Release Highlights<2>. + (line 115) +* Python Enhancement Proposals; PEP 442 <2>: PEP 442 Safe Object Finalization. + (line 6) +* Python Enhancement Proposals; PEP 442 <3>: PEP 442 Safe Object Finalization. + (line 21) +* Python Enhancement Proposals; PEP 442 <4>: gc — Garbage Collector interface. + (line 225) +* Python Enhancement Proposals; PEP 442 <5>: Finalization and De-allocation. + (line 77) +* Python Enhancement Proposals; PEP 442 <6>: PyTypeObject Slots. + (line 1387) +* Python Enhancement Proposals; PEP 443: Summary – Release Highlights<2>. + (line 55) +* Python Enhancement Proposals; PEP 443 <1>: functools<4>. (line 22) +* Python Enhancement Proposals; PEP 443 <2>: Glossary. (line 546) +* Python Enhancement Proposals; PEP 445: Summary – Release Highlights<2>. + (line 118) +* Python Enhancement Proposals; PEP 445 <1>: PEP 445 Customization of CPython Memory Allocators. + (line 6) +* Python Enhancement Proposals; PEP 445 <2>: PEP 445 Customization of CPython Memory Allocators. + (line 12) +* Python Enhancement Proposals; PEP 445 <3>: Other Improvements<2>. + (line 63) +* Python Enhancement Proposals; PEP 446: Summary – Release Highlights<2>. + (line 14) +* Python Enhancement Proposals; PEP 446 <1>: Summary – Release Highlights<2>. + (line 78) +* Python Enhancement Proposals; PEP 446 <2>: PEP 446 Newly Created File Descriptors Are Non-Inheritable. + (line 6) +* Python Enhancement Proposals; PEP 446 <3>: PEP 446 Newly Created File Descriptors Are Non-Inheritable. + (line 26) +* Python Enhancement Proposals; PEP 448: PEP 448 - Additional Unpacking Generalizations. + (line 6) +* Python Enhancement Proposals; PEP 448 <1>: PEP 448 - Additional Unpacking Generalizations. + (line 39) +* Python Enhancement Proposals; PEP 448 <2>: Dictionary displays. + (line 29) +* Python Enhancement Proposals; PEP 448 <3>: Calls. (line 113) +* Python Enhancement Proposals; PEP 448 <4>: Expression lists. + (line 22) +* Python Enhancement Proposals; PEP 450: Summary – Release Highlights<2>. + (line 47) +* Python Enhancement Proposals; PEP 450 <1>: statistics<3>. (line 6) +* Python Enhancement Proposals; PEP 450 <2>: statistics<3>. (line 14) +* Python Enhancement Proposals; PEP 451: PEP 489 Multi-phase extension module initialization. + (line 7) +* Python Enhancement Proposals; PEP 451 <1>: Summary – Release Highlights<2>. + (line 22) +* Python Enhancement Proposals; PEP 451 <2>: PEP 451 A ModuleSpec Type for the Import System. + (line 6) +* Python Enhancement Proposals; PEP 451 <3>: References. (line 26) +* Python Enhancement Proposals; PEP 451 <4>: sys — System-specific parameters and functions. + (line 1046) +* Python Enhancement Proposals; PEP 451 <5>: pkgutil — Package extension utility. + (line 90) +* Python Enhancement Proposals; PEP 451 <6>: pkgutil — Package extension utility. + (line 121) +* Python Enhancement Proposals; PEP 451 <7>: runpy — Locating and executing Python modules. + (line 91) +* Python Enhancement Proposals; PEP 451 <8>: runpy — Locating and executing Python modules. + (line 164) +* Python Enhancement Proposals; PEP 451 <9>: runpy — Locating and executing Python modules. + (line 179) +* Python Enhancement Proposals; PEP 451 <10>: Introduction<12>. + (line 61) +* Python Enhancement Proposals; PEP 451 <11>: Multi-phase initialization. + (line 72) +* Python Enhancement Proposals; PEP 451 <12>: Glossary. (line 450) +* Python Enhancement Proposals; PEP 453: Summary – Release Highlights<2>. + (line 12) +* Python Enhancement Proposals; PEP 453 <1>: Summary – Release Highlights<2>. + (line 34) +* Python Enhancement Proposals; PEP 453 <2>: Bootstrapping pip By Default. + (line 6) +* Python Enhancement Proposals; PEP 453 <3>: Documentation Changes. + (line 21) +* Python Enhancement Proposals; PEP 453 <4>: ensurepip. (line 7) +* Python Enhancement Proposals; PEP 453 <5>: venv<4>. (line 13) +* Python Enhancement Proposals; PEP 453 <6>: Other Improvements<2>. + (line 73) +* Python Enhancement Proposals; PEP 453 <7>: PEP 477 Backport ensurepip PEP 453 to Python 2 7. + (line 6) +* Python Enhancement Proposals; PEP 453 <8>: Bootstrapping pip By Default<2>. + (line 6) +* Python Enhancement Proposals; PEP 453 <9>: Documentation Changes<2>. + (line 21) +* Python Enhancement Proposals; PEP 453 <10>: ensurepip — Bootstrapping the pip installer. + (line 33) +* Python Enhancement Proposals; PEP 454: Summary – Release Highlights<2>. + (line 50) +* Python Enhancement Proposals; PEP 454 <1>: tracemalloc<3>. (line 6) +* Python Enhancement Proposals; PEP 454 <2>: tracemalloc<3>. (line 22) +* Python Enhancement Proposals; PEP 456: Summary – Release Highlights<2>. + (line 75) +* Python Enhancement Proposals; PEP 456 <1>: PEP 456 Secure and Interchangeable Hash Algorithm. + (line 6) +* Python Enhancement Proposals; PEP 461: PEP 461 - percent formatting support for bytes and bytearray. + (line 6) +* Python Enhancement Proposals; PEP 461 <1>: PEP 461 - percent formatting support for bytes and bytearray. + (line 41) +* Python Enhancement Proposals; PEP 461 <2>: printf-style Bytes Formatting. + (line 198) +* Python Enhancement Proposals; PEP 465: PEP 465 - A dedicated infix operator for matrix multiplication. + (line 6) +* Python Enhancement Proposals; PEP 465 <1>: PEP 465 - A dedicated infix operator for matrix multiplication. + (line 45) +* Python Enhancement Proposals; PEP 465 <2>: Build and C API Changes<3>. + (line 43) +* Python Enhancement Proposals; PEP 466: PEP 466 Network Security Enhancements for Python 2 7. + (line 6) +* Python Enhancement Proposals; PEP 466 <1>: PEP 466 Network Security Enhancements for Python 2 7. + (line 11) +* Python Enhancement Proposals; PEP 466 <2>: PEP 466 Network Security Enhancements for Python 2 7. + (line 21) +* Python Enhancement Proposals; PEP 466 <3>: PEP 466 Network Security Enhancements for Python 2 7. + (line 32) +* Python Enhancement Proposals; PEP 468: PEP 468 Preserving Keyword Argument Order. + (line 12) +* Python Enhancement Proposals; PEP 468 <1>: OrderedDict objects. + (line 85) +* Python Enhancement Proposals; PEP 471: PEP 471 - os scandir function – a better and faster directory iterator. + (line 6) +* Python Enhancement Proposals; PEP 471 <1>: PEP 471 - os scandir function – a better and faster directory iterator. + (line 29) +* Python Enhancement Proposals; PEP 475: PEP 475 Retry system calls failing with EINTR. + (line 32) +* Python Enhancement Proposals; PEP 475 <1>: PEP 475 Retry system calls failing with EINTR. + (line 77) +* Python Enhancement Proposals; PEP 475 <2>: Changes in the Python API<4>. + (line 6) +* Python Enhancement Proposals; PEP 475 <3>: Built-in Functions. + (line 1251) +* Python Enhancement Proposals; PEP 475 <4>: OS exceptions. (line 78) +* Python Enhancement Proposals; PEP 475 <5>: File Descriptor Operations. + (line 293) +* Python Enhancement Proposals; PEP 475 <6>: File Descriptor Operations. + (line 573) +* Python Enhancement Proposals; PEP 475 <7>: File Descriptor Operations. + (line 692) +* Python Enhancement Proposals; PEP 475 <8>: Process Management. + (line 804) +* Python Enhancement Proposals; PEP 475 <9>: Functions<2>. (line 228) +* Python Enhancement Proposals; PEP 475 <10>: Socket Objects. + (line 28) +* Python Enhancement Proposals; PEP 475 <11>: Socket Objects. + (line 80) +* Python Enhancement Proposals; PEP 475 <12>: Socket Objects. + (line 234) +* Python Enhancement Proposals; PEP 475 <13>: Socket Objects. + (line 248) +* Python Enhancement Proposals; PEP 475 <14>: Socket Objects. + (line 323) +* Python Enhancement Proposals; PEP 475 <15>: Socket Objects. + (line 395) +* Python Enhancement Proposals; PEP 475 <16>: Socket Objects. + (line 414) +* Python Enhancement Proposals; PEP 475 <17>: Socket Objects. + (line 433) +* Python Enhancement Proposals; PEP 475 <18>: Socket Objects. + (line 475) +* Python Enhancement Proposals; PEP 475 <19>: select — Waiting for I/O completion. + (line 154) +* Python Enhancement Proposals; PEP 475 <20>: /dev/poll Polling Objects. + (line 82) +* Python Enhancement Proposals; PEP 475 <21>: Edge and Level Trigger Polling epoll Objects. + (line 99) +* Python Enhancement Proposals; PEP 475 <22>: Polling Objects. + (line 94) +* Python Enhancement Proposals; PEP 475 <23>: Kqueue Objects. + (line 36) +* Python Enhancement Proposals; PEP 475 <24>: Classes<3>. (line 133) +* Python Enhancement Proposals; PEP 475 <25>: Module contents<2>. + (line 516) +* Python Enhancement Proposals; PEP 475 <26>: Module contents<2>. + (line 534) +* Python Enhancement Proposals; PEP 476: PEP 476 Enabling certificate verification by default for stdlib http clients<2>. + (line 6) +* Python Enhancement Proposals; PEP 477: PEP 477 Backport ensurepip PEP 453 to Python 2 7. + (line 6) +* Python Enhancement Proposals; PEP 478: What’s New In Python 3 5. + (line 17) +* Python Enhancement Proposals; PEP 479: Changes in Python Behavior. + (line 11) +* Python Enhancement Proposals; PEP 479 <1>: PEP 479 Change StopIteration handling inside generators. + (line 12) +* Python Enhancement Proposals; PEP 479 <2>: PEP 479 Change StopIteration handling inside generators. + (line 47) +* Python Enhancement Proposals; PEP 479 <3>: Concrete exceptions. + (line 234) +* Python Enhancement Proposals; PEP 479 <4>: Concrete exceptions. + (line 236) +* Python Enhancement Proposals; PEP 479 <5>: __future__ — Future statement definitions. + (line 93) +* Python Enhancement Proposals; PEP 483: PEP 484 - Type Hints. + (line 47) +* Python Enhancement Proposals; PEP 483 <1>: typing — Support for type hints. + (line 21) +* Python Enhancement Proposals; PEP 484: ast. (line 17) +* Python Enhancement Proposals; PEP 484 <1>: ast. (line 20) +* Python Enhancement Proposals; PEP 484 <2>: PEP 560 Core Support for typing module and Generic Types. + (line 6) +* Python Enhancement Proposals; PEP 484 <3>: PEP 526 Syntax for variable annotations. + (line 6) +* Python Enhancement Proposals; PEP 484 <4>: PEP 484 - Type Hints. + (line 16) +* Python Enhancement Proposals; PEP 484 <5>: PEP 484 - Type Hints. + (line 41) +* Python Enhancement Proposals; PEP 484 <6>: Function Annotations. + (line 8) +* Python Enhancement Proposals; PEP 484 <7>: Emulating generic types. + (line 6) +* Python Enhancement Proposals; PEP 484 <8>: Annotated assignment statements. + (line 45) +* Python Enhancement Proposals; PEP 484 <9>: Function definitions. + (line 127) +* Python Enhancement Proposals; PEP 484 <10>: typing — Support for type hints. + (line 16) +* Python Enhancement Proposals; PEP 484 <11>: typing — Support for type hints. + (line 20) +* Python Enhancement Proposals; PEP 484 <12>: NewType. (line 64) +* Python Enhancement Proposals; PEP 484 <13>: Nominal vs structural subtyping. + (line 6) +* Python Enhancement Proposals; PEP 484 <14>: Nominal vs structural subtyping. + (line 14) +* Python Enhancement Proposals; PEP 484 <15>: Classes functions and decorators. + (line 40) +* Python Enhancement Proposals; PEP 484 <16>: Classes functions and decorators. + (line 45) +* Python Enhancement Proposals; PEP 484 <17>: Classes functions and decorators. + (line 134) +* Python Enhancement Proposals; PEP 484 <18>: Classes functions and decorators. + (line 688) +* Python Enhancement Proposals; PEP 484 <19>: ast Helpers. (line 16) +* Python Enhancement Proposals; PEP 484 <20>: ast Helpers. (line 26) +* Python Enhancement Proposals; PEP 484 <21>: Glossary. (line 61) +* Python Enhancement Proposals; PEP 484 <22>: Glossary. (line 483) +* Python Enhancement Proposals; PEP 484 <23>: Glossary. (line 1262) +* Python Enhancement Proposals; PEP 484 <24>: Glossary. (line 1279) +* Python Enhancement Proposals; PEP 484 <25>: Glossary. (line 1309) +* Python Enhancement Proposals; PEP 485: PEP 485 A function for testing approximate equality. + (line 6) +* Python Enhancement Proposals; PEP 485 <1>: PEP 485 A function for testing approximate equality. + (line 36) +* Python Enhancement Proposals; PEP 485 <2>: Number-theoretic and representation functions. + (line 135) +* Python Enhancement Proposals; PEP 485 <3>: Classification functions. + (line 54) +* Python Enhancement Proposals; PEP 486: PEP 486 Make the Python Launcher aware of virtual environments. + (line 6) +* Python Enhancement Proposals; PEP 486 <1>: PEP 486 Make the Python Launcher aware of virtual environments. + (line 14) +* Python Enhancement Proposals; PEP 487: PEP 487 Simpler customization of class creation. + (line 32) +* Python Enhancement Proposals; PEP 487 <1>: PEP 487 Descriptor Protocol Enhancements. + (line 6) +* Python Enhancement Proposals; PEP 487 <2>: PEP 487 Descriptor Protocol Enhancements. + (line 33) +* Python Enhancement Proposals; PEP 487 <3>: Changes in the Python API<3>. + (line 104) +* Python Enhancement Proposals; PEP 488: PEP 488 Elimination of PYO files. + (line 6) +* Python Enhancement Proposals; PEP 488 <1>: PEP 488 Elimination of PYO files. + (line 20) +* Python Enhancement Proposals; PEP 488 <2>: Changes in the Python API<4>. + (line 79) +* Python Enhancement Proposals; PEP 488 <3>: Miscellaneous options. + (line 71) +* Python Enhancement Proposals; PEP 488 <4>: Miscellaneous options. + (line 73) +* Python Enhancement Proposals; PEP 488 <5>: Miscellaneous options. + (line 80) +* Python Enhancement Proposals; PEP 488 <6>: Miscellaneous options. + (line 82) +* Python Enhancement Proposals; PEP 488 <7>: test support — Utilities for the Python test suite. + (line 192) +* Python Enhancement Proposals; PEP 488 <8>: Introduction<12>. + (line 65) +* Python Enhancement Proposals; PEP 488 <9>: importlib util – Utility code for importers. + (line 24) +* Python Enhancement Proposals; PEP 488 <10>: importlib util – Utility code for importers. + (line 64) +* Python Enhancement Proposals; PEP 488 <11>: py_compile — Compile Python source files. + (line 31) +* Python Enhancement Proposals; PEP 488 <12>: distutils util — Miscellaneous other utility functions. + (line 132) +* Python Enhancement Proposals; PEP 488 <13>: distutils util — Miscellaneous other utility functions. + (line 169) +* Python Enhancement Proposals; PEP 489: PEP 489 Multi-phase extension module initialization. + (line 6) +* Python Enhancement Proposals; PEP 489 <1>: PEP 489 Multi-phase extension module initialization. + (line 18) +* Python Enhancement Proposals; PEP 489 <2>: Build and C API Changes<3>. + (line 37) +* Python Enhancement Proposals; PEP 489 <3>: Introduction<12>. + (line 69) +* Python Enhancement Proposals; PEP 489 <4>: importlib machinery – Importers and path hooks. + (line 77) +* Python Enhancement Proposals; PEP 489 <5>: importlib machinery – Importers and path hooks. + (line 322) +* Python Enhancement Proposals; PEP 489 <6>: importlib machinery – Importers and path hooks. + (line 328) +* Python Enhancement Proposals; PEP 489 <7>: The Module’s Method Table and Initialization Function. + (line 129) +* Python Enhancement Proposals; PEP 489 <8>: Building C and C++ Extensions. + (line 40) +* Python Enhancement Proposals; PEP 489 <9>: Multi-phase initialization. + (line 108) +* Python Enhancement Proposals; PEP 492: PEP 525 Asynchronous Generators. + (line 6) +* Python Enhancement Proposals; PEP 492 <1>: New Keywords. (line 7) +* Python Enhancement Proposals; PEP 492 <2>: PEP 492 - Coroutines with async and await syntax. + (line 6) +* Python Enhancement Proposals; PEP 492 <3>: PEP 492 - Coroutines with async and await syntax. + (line 97) +* Python Enhancement Proposals; PEP 492 <4>: New Keywords<2>. + (line 7) +* Python Enhancement Proposals; PEP 492 <5>: Changes in the C API<4>. + (line 25) +* Python Enhancement Proposals; PEP 492 <6>: Awaitable Objects. + (line 27) +* Python Enhancement Proposals; PEP 492 <7>: Yield expressions. + (line 105) +* Python Enhancement Proposals; PEP 492 <8>: The async with statement. + (line 45) +* Python Enhancement Proposals; PEP 492 <9>: Collections Abstract Base Classes. + (line 247) +* Python Enhancement Proposals; PEP 492 <10>: Code Objects Bit Flags. + (line 42) +* Python Enhancement Proposals; PEP 492 <11>: Code Objects Bit Flags. + (line 51) +* Python Enhancement Proposals; PEP 492 <12>: Glossary. (line 100) +* Python Enhancement Proposals; PEP 492 <13>: Glossary. (line 133) +* Python Enhancement Proposals; PEP 492 <14>: Glossary. (line 140) +* Python Enhancement Proposals; PEP 492 <15>: Glossary. (line 149) +* Python Enhancement Proposals; PEP 492 <16>: Glossary. (line 161) +* Python Enhancement Proposals; PEP 492 <17>: Glossary. (line 288) +* Python Enhancement Proposals; PEP 492 <18>: Glossary. (line 295) +* Python Enhancement Proposals; PEP 493: PEP 493 HTTPS verification migration tools for Python 2 7. + (line 6) +* Python Enhancement Proposals; PEP 494: What’s New In Python 3 6. + (line 17) +* Python Enhancement Proposals; PEP 495: PEP 495 Local Time Disambiguation. + (line 13) +* Python Enhancement Proposals; PEP 495 <1>: PEP 495 Local Time Disambiguation. + (line 36) +* Python Enhancement Proposals; PEP 498: PEP 498 Formatted string literals. + (line 6) +* Python Enhancement Proposals; PEP 498 <1>: PEP 498 Formatted string literals. + (line 27) +* Python Enhancement Proposals; PEP 498 <2>: Formatted string literals. + (line 141) +* Python Enhancement Proposals; PEP 498 <3>: Glossary. (line 419) +* Python Enhancement Proposals; PEP 5: PEP 230 Warning Framework. + (line 66) +* Python Enhancement Proposals; PEP 5 <1>: Is it reasonable to propose incompatible changes to Python?. + (line 14) +* Python Enhancement Proposals; PEP 506: secrets. (line 19) +* Python Enhancement Proposals; PEP 506 <1>: secrets — Generate secure random numbers for managing secrets. + (line 24) +* Python Enhancement Proposals; PEP 511: Changes in the Python API<3>. + (line 32) +* Python Enhancement Proposals; PEP 515: PEP 515 Underscores in Numeric Literals. + (line 6) +* Python Enhancement Proposals; PEP 515 <1>: PEP 515 Underscores in Numeric Literals. + (line 32) +* Python Enhancement Proposals; PEP 515 <2>: Format Specification Mini-Language. + (line 111) +* Python Enhancement Proposals; PEP 519: PEP 519 Adding a file system path protocol. + (line 62) +* Python Enhancement Proposals; PEP 519 <1>: Glossary. (line 1025) +* Python Enhancement Proposals; PEP 520: PEP 520 Preserving Class Attribute Definition Order. + (line 17) +* Python Enhancement Proposals; PEP 523: PEP 523 Adding a frame evaluation API to CPython. + (line 12) +* Python Enhancement Proposals; PEP 523 <1>: PEP 523 Adding a frame evaluation API to CPython. + (line 26) +* Python Enhancement Proposals; PEP 524: Summary – Release highlights<2>. + (line 77) +* Python Enhancement Proposals; PEP 524 <1>: os<3>. (line 20) +* Python Enhancement Proposals; PEP 524 <2>: os<3>. (line 25) +* Python Enhancement Proposals; PEP 524 <3>: Random numbers<2>. + (line 42) +* Python Enhancement Proposals; PEP 525: PEP 525 Asynchronous Generators. + (line 23) +* Python Enhancement Proposals; PEP 525 <1>: Yield expressions. + (line 103) +* Python Enhancement Proposals; PEP 525 <2>: Collections Abstract Base Classes. + (line 247) +* Python Enhancement Proposals; PEP 525 <3>: sys — System-specific parameters and functions. + (line 811) +* Python Enhancement Proposals; PEP 525 <4>: sys — System-specific parameters and functions. + (line 1422) +* Python Enhancement Proposals; PEP 525 <5>: Code Objects Bit Flags. + (line 60) +* Python Enhancement Proposals; PEP 525 <6>: Glossary. (line 134) +* Python Enhancement Proposals; PEP 526: ast. (line 18) +* Python Enhancement Proposals; PEP 526 <1>: PEP 563 Postponed Evaluation of Annotations. + (line 8) +* Python Enhancement Proposals; PEP 526 <2>: PEP 526 Syntax for variable annotations. + (line 30) +* Python Enhancement Proposals; PEP 526 <3>: typing<2>. (line 23) +* Python Enhancement Proposals; PEP 526 <4>: Annotated assignment statements. + (line 39) +* Python Enhancement Proposals; PEP 526 <5>: Function definitions. + (line 131) +* Python Enhancement Proposals; PEP 526 <6>: typing — Support for type hints. + (line 17) +* Python Enhancement Proposals; PEP 526 <7>: Classes functions and decorators. + (line 535) +* Python Enhancement Proposals; PEP 526 <8>: Classes functions and decorators. + (line 573) +* Python Enhancement Proposals; PEP 526 <9>: Classes functions and decorators. + (line 907) +* Python Enhancement Proposals; PEP 526 <10>: dataclasses — Data Classes. + (line 16) +* Python Enhancement Proposals; PEP 526 <11>: Class variables. + (line 8) +* Python Enhancement Proposals; PEP 526 <12>: ast Helpers. (line 16) +* Python Enhancement Proposals; PEP 526 <13>: Glossary. (line 61) +* Python Enhancement Proposals; PEP 526 <14>: Glossary. (line 1309) +* Python Enhancement Proposals; PEP 528: PEP 528 Change Windows console encoding to UTF-8. + (line 17) +* Python Enhancement Proposals; PEP 528 <1>: UTF-8 mode. (line 42) +* Python Enhancement Proposals; PEP 528 <2>: Global configuration variables. + (line 105) +* Python Enhancement Proposals; PEP 529: Changes in the Python API. + (line 7) +* Python Enhancement Proposals; PEP 529 <1>: PEP 529 Change Windows filesystem encoding to UTF-8. + (line 22) +* Python Enhancement Proposals; PEP 529 <2>: Environment variables. + (line 307) +* Python Enhancement Proposals; PEP 529 <3>: UTF-8 mode. (line 45) +* Python Enhancement Proposals; PEP 529 <4>: Files and Directories. + (line 289) +* Python Enhancement Proposals; PEP 529 <5>: sys — System-specific parameters and functions. + (line 666) +* Python Enhancement Proposals; PEP 529 <6>: sys — System-specific parameters and functions. + (line 1461) +* Python Enhancement Proposals; PEP 529 <7>: File System Encoding. + (line 9) +* Python Enhancement Proposals; PEP 529 <8>: Global configuration variables. + (line 93) +* Python Enhancement Proposals; PEP 530: PEP 530 Asynchronous Comprehensions. + (line 6) +* Python Enhancement Proposals; PEP 530 <1>: PEP 530 Asynchronous Comprehensions. + (line 19) +* Python Enhancement Proposals; PEP 530 <2>: Displays for lists sets and dictionaries. + (line 53) +* Python Enhancement Proposals; PEP 538: PEP 538 Legacy C Locale Coercion. + (line 11) +* Python Enhancement Proposals; PEP 538 <1>: PEP 538 Legacy C Locale Coercion. + (line 36) +* Python Enhancement Proposals; PEP 538 <2>: PEP 538 Legacy C Locale Coercion. + (line 48) +* Python Enhancement Proposals; PEP 538 <3>: PEP 540 Forced UTF-8 Runtime Mode. + (line 27) +* Python Enhancement Proposals; PEP 538 <4>: Environment variables. + (line 376) +* Python Enhancement Proposals; PEP 538 <5>: Python Configuration. + (line 13) +* Python Enhancement Proposals; PEP 539: PEP 539 New C API for Thread-Local Storage. + (line 12) +* Python Enhancement Proposals; PEP 539 <1>: PEP 539 New C API for Thread-Local Storage. + (line 30) +* Python Enhancement Proposals; PEP 539 <2>: Thread Specific Storage TSS API. + (line 15) +* Python Enhancement Proposals; PEP 540: PEP 540 Forced UTF-8 Runtime Mode. + (line 18) +* Python Enhancement Proposals; PEP 540 <1>: PEP 540 Forced UTF-8 Runtime Mode. + (line 35) +* Python Enhancement Proposals; PEP 540 <2>: Environment variables. + (line 445) +* Python Enhancement Proposals; PEP 540 <3>: Python Configuration. + (line 13) +* Python Enhancement Proposals; PEP 544: typing. (line 32) +* Python Enhancement Proposals; PEP 544 <1>: typing — Support for type hints. + (line 17) +* Python Enhancement Proposals; PEP 544 <2>: Nominal vs structural subtyping. + (line 23) +* Python Enhancement Proposals; PEP 544 <3>: Classes functions and decorators. + (line 92) +* Python Enhancement Proposals; PEP 545: PEP 545 Python Documentation Translations. + (line 6) +* Python Enhancement Proposals; PEP 545 <1>: PEP 545 Python Documentation Translations. + (line 20) +* Python Enhancement Proposals; PEP 552: PEP 552 Hash-based pyc Files. + (line 15) +* Python Enhancement Proposals; PEP 552 <1>: PEP 552 Hash-based pyc Files. + (line 35) +* Python Enhancement Proposals; PEP 552 <2>: Introduction<12>. + (line 73) +* Python Enhancement Proposals; PEP 552 <3>: py_compile — Compile Python source files. + (line 81) +* Python Enhancement Proposals; PEP 552 <4>: PyConfig. (line 137) +* Python Enhancement Proposals; PEP 553: PEP 553 Built-in breakpoint. + (line 20) +* Python Enhancement Proposals; PEP 557: dataclasses. (line 26) +* Python Enhancement Proposals; PEP 557 <1>: dataclasses — Data Classes. + (line 13) +* Python Enhancement Proposals; PEP 560: PEP 560 Core Support for typing module and Generic Types. + (line 19) +* Python Enhancement Proposals; PEP 560 <1>: types. (line 12) +* Python Enhancement Proposals; PEP 560 <2>: Resolving MRO entries. + (line 15) +* Python Enhancement Proposals; PEP 560 <3>: Emulating generic types. + (line 22) +* Python Enhancement Proposals; PEP 560 <4>: Dynamic Type Creation. + (line 60) +* Python Enhancement Proposals; PEP 560 <5>: Dynamic Type Creation. + (line 74) +* Python Enhancement Proposals; PEP 562: PEP 562 Customization of Access to Module Attributes. + (line 16) +* Python Enhancement Proposals; PEP 562 <1>: Customizing module attribute access. + (line 52) +* Python Enhancement Proposals; PEP 563: PEP 563 Postponed Evaluation of Annotations. + (line 51) +* Python Enhancement Proposals; PEP 563 <1>: Future statements. + (line 34) +* Python Enhancement Proposals; PEP 563 <2>: Function definitions. + (line 136) +* Python Enhancement Proposals; PEP 563 <3>: __future__ — Future statement definitions. + (line 97) +* Python Enhancement Proposals; PEP 564: PEP 564 New Time Functions With Nanosecond Resolution. + (line 8) +* Python Enhancement Proposals; PEP 564 <1>: PEP 564 New Time Functions With Nanosecond Resolution. + (line 33) +* Python Enhancement Proposals; PEP 564 <2>: time<2>. (line 6) +* Python Enhancement Proposals; PEP 565: PEP 565 Show DeprecationWarning in __main__. + (line 41) +* Python Enhancement Proposals; PEP 566: Using importlib metadata. + (line 20) +* Python Enhancement Proposals; PEP 566 <1>: Distributions. (line 28) +* Python Enhancement Proposals; PEP 567: contextvars. (line 20) +* Python Enhancement Proposals; PEP 567 <1>: asyncio<2>. (line 19) +* Python Enhancement Proposals; PEP 567 <2>: contextvars — Context Variables. + (line 18) +* Python Enhancement Proposals; PEP 567 <3>: Scheduling callbacks. + (line 32) +* Python Enhancement Proposals; PEP 567 <4>: Scheduling delayed callbacks. + (line 31) +* Python Enhancement Proposals; PEP 567 <5>: Scheduling delayed callbacks. + (line 49) +* Python Enhancement Proposals; PEP 567 <6>: Future Object. (line 100) +* Python Enhancement Proposals; PEP 570: Positional-only parameters. + (line 70) +* Python Enhancement Proposals; PEP 570 <1>: Changes in the Python API. + (line 159) +* Python Enhancement Proposals; PEP 572: Assignment expressions. + (line 41) +* Python Enhancement Proposals; PEP 572 <1>: CPython bytecode changes. + (line 28) +* Python Enhancement Proposals; PEP 572 <2>: Dictionary displays. + (line 47) +* Python Enhancement Proposals; PEP 572 <3>: Assignment expressions<2>. + (line 22) +* Python Enhancement Proposals; PEP 572 <4>: Why can’t I use an assignment in an expression?. + (line 14) +* Python Enhancement Proposals; PEP 574: Pickle protocol 5 with out-of-band data buffers. + (line 17) +* Python Enhancement Proposals; PEP 574 <1>: Data stream format. + (line 46) +* Python Enhancement Proposals; PEP 574 <2>: Example<4>. (line 63) +* Python Enhancement Proposals; PEP 578: PEP 578 Python Runtime Audit Hooks. + (line 12) +* Python Enhancement Proposals; PEP 578 <1>: sys — System-specific parameters and functions. + (line 44) +* Python Enhancement Proposals; PEP 578 <2>: System Functions. + (line 153) +* Python Enhancement Proposals; PEP 586: typing. (line 18) +* Python Enhancement Proposals; PEP 586 <1>: typing — Support for type hints. + (line 17) +* Python Enhancement Proposals; PEP 586 <2>: Classes functions and decorators. + (line 898) +* Python Enhancement Proposals; PEP 587: PEP 587 Python Initialization Configuration. + (line 6) +* Python Enhancement Proposals; PEP 587 <1>: PEP 587 Python Initialization Configuration. + (line 83) +* Python Enhancement Proposals; PEP 587 <2>: Python Initialization Configuration. + (line 81) +* Python Enhancement Proposals; PEP 589: typing. (line 8) +* Python Enhancement Proposals; PEP 589 <1>: typing — Support for type hints. + (line 17) +* Python Enhancement Proposals; PEP 589 <2>: Classes functions and decorators. + (line 592) +* Python Enhancement Proposals; PEP 590: Vectorcall a fast calling protocol for CPython. + (line 14) +* Python Enhancement Proposals; PEP 591: typing. (line 25) +* Python Enhancement Proposals; PEP 591 <1>: typing — Support for type hints. + (line 17) +* Python Enhancement Proposals; PEP 591 <2>: Classes functions and decorators. + (line 711) +* Python Enhancement Proposals; PEP 591 <3>: Classes functions and decorators. + (line 945) +* Python Enhancement Proposals; PEP 6: How does the Python version numbering scheme work?. + (line 9) +* Python Enhancement Proposals; PEP 602: How stable is Python?. + (line 8) +* Python Enhancement Proposals; PEP 623: Unicode Objects. (line 33) +* Python Enhancement Proposals; PEP 628: cmath. (line 7) +* Python Enhancement Proposals; PEP 628 <1>: math<3>. (line 7) +* Python Enhancement Proposals; PEP 7: Build and C API Changes<2>. + (line 9) +* Python Enhancement Proposals; PEP 7 <1>: Coding standards. (line 7) +* Python Enhancement Proposals; PEP 7 <2>: Useful macros. (line 87) +* Python Enhancement Proposals; PEP 7 <3>: Useful macros. (line 105) +* Python Enhancement Proposals; PEP 8: Library Changes. (line 26) +* Python Enhancement Proposals; PEP 8 <1>: Intermezzo Coding Style. + (line 13) +* Python Enhancement Proposals; PEP 8 <2>: Editors and IDEs. (line 7) +* Python Enhancement Proposals; PEP 8 <3>: Value comparisons. + (line 61) +* Python Enhancement Proposals; PEP 8 <4>: Using real Argument Clinic converters instead of “legacy converters”. + (line 50) +* Python Enhancement Proposals; PEP 8 <5>: Are there coding standards or a style guide for Python programs?. + (line 7) +* Python Enhancement Proposals; PEP 8 <6>: How do I keep editors from inserting tabs into my Python source?. + (line 6) +* Python Package Index (PyPI): Installing the tools. + (line 26) +* PYTHON*: Other Improvements<2>. + (line 9) +* PYTHON* <1>: Interface options. (line 80) +* PYTHON* <2>: Interface options. (line 140) +* PYTHON* <3>: Miscellaneous options. + (line 42) +* PYTHON* <4>: Miscellaneous options. + (line 60) +* PYTHON* <5>: Global configuration variables. + (line 56) +* PYTHONASYNCIODEBUG: Enabling debug mode. + (line 10) +* PYTHONASYNCIODEBUG <1>: Debug Mode. (line 11) +* PYTHONBREAKPOINT: PEP 553 Built-in breakpoint. + (line 13) +* PYTHONBREAKPOINT <1>: sys — System-specific parameters and functions. + (line 202) +* PYTHONBREAKPOINT <2>: sys — System-specific parameters and functions. + (line 216) +* PYTHONBREAKPOINT <3>: sys — System-specific parameters and functions. + (line 220) +* PYTHONCASEOK: PEP 235 Importing Modules on Case-Insensitive Platforms. + (line 17) +* PYTHONCOERCECLOCALE: PEP 538 Legacy C Locale Coercion. + (line 13) +* PYTHONCOERCECLOCALE <1>: Environment variables. + (line 438) +* PYTHONCOERCECLOCALE <2>: Python Configuration. + (line 15) +* PYTHONDEBUG: Miscellaneous options. + (line 38) +* PYTHONDEBUG <1>: Global configuration variables. + (line 28) +* PYTHONDEVMODE: Development Runtime Mode -X dev. + (line 6) +* PYTHONDEVMODE <1>: test support script_helper — Utilities for the Python execution tests. + (line 106) +* PYTHONDOCS: pydoc — Documentation generator and online help system. + (line 85) +* PYTHONDONTWRITEBYTECODE: Interpreter Changes<2>. + (line 16) +* PYTHONDONTWRITEBYTECODE <1>: New and Improved Modules<2>. + (line 625) +* PYTHONDONTWRITEBYTECODE <2>: Miscellaneous options. + (line 19) +* PYTHONDONTWRITEBYTECODE <3>: sys — System-specific parameters and functions. + (line 285) +* PYTHONDONTWRITEBYTECODE <4>: Global configuration variables. + (line 36) +* PYTHONDONTWRITEBYTECODE <5>: How do I create a pyc file?. + (line 21) +* PYTHONDUMPREFS: Debug build uses the same ABI as release build. + (line 14) +* PYTHONDUMPREFS <1>: PyObject Slots. (line 24) +* PYTHONFAULTHANDLER: faulthandler<3>. (line 11) +* PYTHONFAULTHANDLER <1>: faulthandler — Dump the Python traceback. + (line 14) +* PYTHONHASHSEED: Builtin functions and types. + (line 17) +* PYTHONHASHSEED <1>: Porting Python code. + (line 6) +* PYTHONHASHSEED <2>: Miscellaneous options. + (line 94) +* PYTHONHASHSEED <3>: Miscellaneous options. + (line 110) +* PYTHONHASHSEED <4>: Environment variables. + (line 129) +* PYTHONHASHSEED <5>: Basic customization. + (line 298) +* PYTHONHASHSEED <6>: Global configuration variables. + (line 48) +* PYTHONHASHSEED <7>: Global configuration variables. + (line 51) +* PYTHONHOME: Summary – Release highlights<2>. + (line 107) +* PYTHONHOME <1>: Miscellaneous options. + (line 43) +* PYTHONHOME <2>: Environment variables. + (line 19) +* PYTHONHOME <3>: Environment variables. + (line 21) +* PYTHONHOME <4>: Environment variables. + (line 38) +* PYTHONHOME <5>: Finding modules. (line 50) +* PYTHONHOME <6>: Finding modules. (line 68) +* PYTHONHOME <7>: Finding modules. (line 102) +* PYTHONHOME <8>: test support script_helper — Utilities for the Python execution tests. + (line 25) +* PYTHONHOME <9>: Embedding Python<2>. + (line 38) +* PYTHONHOME <10>: Embedding Python<2>. + (line 44) +* PYTHONHOME <11>: Global configuration variables. + (line 57) +* PYTHONHOME <12>: Process-wide parameters. + (line 270) +* PYTHONHOME <13>: Process-wide parameters. + (line 284) +* PYTHONHOME <14>: PyConfig. (line 197) +* PYTHONHOME <15>: Modifying Python’s Search Path. + (line 60) +* PYTHONHOME <16>: Modifying Python’s Search Path. + (line 62) +* Pythonic: Glossary. (line 1081) +* PYTHONINSPECT: Other Changes and Fixes<2>. + (line 13) +* PYTHONINSPECT <1>: Miscellaneous options. + (line 53) +* PYTHONINSPECT <2>: Global configuration variables. + (line 68) +* PYTHONIOENCODING: Other Improvements<2>. + (line 75) +* PYTHONIOENCODING <1>: Interpreter Changes<2>. + (line 23) +* PYTHONIOENCODING <2>: Environment variables. + (line 359) +* PYTHONIOENCODING <3>: Environment variables. + (line 426) +* PYTHONIOENCODING <4>: sys — System-specific parameters and functions. + (line 1503) +* PYTHONIOENCODING <5>: Process-wide parameters. + (line 14) +* PYTHONIOENCODING <6>: Process-wide parameters. + (line 18) +* PYTHONLEGACYWINDOWSFSENCODING: PEP 529 Change Windows filesystem encoding to UTF-8. + (line 18) +* PYTHONLEGACYWINDOWSFSENCODING <1>: sys — System-specific parameters and functions. + (line 1455) +* PYTHONLEGACYWINDOWSFSENCODING <2>: Global configuration variables. + (line 90) +* PYTHONLEGACYWINDOWSSTDIO: PEP 528 Change Windows console encoding to UTF-8. + (line 12) +* PYTHONLEGACYWINDOWSSTDIO <1>: Environment variables. + (line 156) +* PYTHONLEGACYWINDOWSSTDIO <2>: sys — System-specific parameters and functions. + (line 1507) +* PYTHONLEGACYWINDOWSSTDIO <3>: Global configuration variables. + (line 102) +* PYTHONMALLOC: PYTHONMALLOC environment variable. + (line 6) +* PYTHONMALLOC <1>: Changes in the C API<3>. + (line 9) +* PYTHONMALLOC <2>: Environment variables. + (line 286) +* PYTHONMALLOC <3>: Overview<3>. (line 75) +* PYTHONMALLOC <4>: Default Memory Allocators. + (line 26) +* PYTHONMALLOC <5>: Customize Memory Allocators. + (line 129) +* PYTHONMALLOCSTATS: Overview<3>. (line 78) +* PYTHONNOUSERSITE: PEP 370 Per-user site-packages Directory. + (line 30) +* PYTHONNOUSERSITE <1>: Module contents<3>. (line 15) +* PYTHONNOUSERSITE <2>: Global configuration variables. + (line 124) +* PYTHONOPTIMIZE: Miscellaneous options. + (line 71) +* PYTHONOPTIMIZE <1>: Global configuration variables. + (line 129) +* PYTHONPATH: Changes in ‘python’ Command Behavior<2>. + (line 8) +* PYTHONPATH <1>: Changes in ‘python’ Command Behavior<2>. + (line 9) +* PYTHONPATH <2>: The Module Search Path. + (line 15) +* PYTHONPATH <3>: Standard Modules. (line 34) +* PYTHONPATH <4>: Standard Modules. (line 35) +* PYTHONPATH <5>: Miscellaneous options. + (line 42) +* PYTHONPATH <6>: Environment variables. + (line 32) +* PYTHONPATH <7>: Environment variables. + (line 39) +* PYTHONPATH <8>: Environment variables. + (line 42) +* PYTHONPATH <9>: Excursus Setting environment variables. + (line 38) +* PYTHONPATH <10>: Finding modules. (line 36) +* PYTHONPATH <11>: Finding modules. (line 59) +* PYTHONPATH <12>: Finding modules. (line 102) +* PYTHONPATH <13>: Configuration. (line 6) +* PYTHONPATH <14>: Path entry finders. (line 6) +* PYTHONPATH <15>: Installing your CGI script on a Unix system. + (line 29) +* PYTHONPATH <16>: test support script_helper — Utilities for the Python execution tests. + (line 26) +* PYTHONPATH <17>: sys — System-specific parameters and functions. + (line 1062) +* PYTHONPATH <18>: sys — System-specific parameters and functions. + (line 1072) +* PYTHONPATH <19>: Building C and C++ Extensions. + (line 9) +* PYTHONPATH <20>: Embedding Python<2>. + (line 39) +* PYTHONPATH <21>: Embedding Python<2>. + (line 44) +* PYTHONPATH <22>: Global configuration variables. + (line 56) +* PYTHONPATH <23>: PyConfig. (line 252) +* PYTHONPATH <24>: Modifying Python’s Search Path. + (line 67) +* PYTHONPATH <25>: Modifying Python’s Search Path. + (line 68) +* PYTHONPROFILEIMPORTTIME: Other Language Changes<2>. + (line 47) +* PYTHONPROFILEIMPORTTIME <1>: Miscellaneous options. + (line 223) +* PYTHONPYCACHEPREFIX: Parallel filesystem cache for compiled bytecode files. + (line 6) +* PYTHONPYCACHEPREFIX <1>: Miscellaneous options. + (line 257) +* PYTHONPYCACHEPREFIX <2>: sys — System-specific parameters and functions. + (line 303) +* PYTHONSHOWALLOCCOUNT: Two new environment variables for debug mode. + (line 11) +* PYTHONSHOWREFCOUNT: Two new environment variables for debug mode. + (line 7) +* PYTHONSTARTUP: sys<6>. (line 17) +* PYTHONSTARTUP <1>: sys<6>. (line 21) +* PYTHONSTARTUP <2>: The Interactive Startup File. + (line 8) +* PYTHONSTARTUP <3>: Miscellaneous options. + (line 50) +* PYTHONSTARTUP <4>: Example. (line 10) +* PYTHONSTARTUP <5>: Startup and code execution. + (line 7) +* PYTHONSTARTUP <6>: sys — System-specific parameters and functions. + (line 957) +* PYTHONSTARTUP <7>: Readline configuration. + (line 12) +* PYTHONTRACEMALLOC: tracemalloc — Trace memory allocations. + (line 25) +* PYTHONTRACEMALLOC <1>: tracemalloc — Trace memory allocations. + (line 32) +* PYTHONTRACEMALLOC <2>: Functions<9>. (line 68) +* PYTHONUNBUFFERED: Miscellaneous options. + (line 140) +* PYTHONUNBUFFERED <1>: Global configuration variables. + (line 145) +* PYTHONUSERBASE: PEP 370 Per-user site-packages Directory. + (line 23) +* PYTHONUSERBASE <1>: Module contents<3>. (line 39) +* PYTHONUSERBASE <2>: Module contents<3>. (line 66) +* PYTHONUSERSITE: test support script_helper — Utilities for the Python execution tests. + (line 26) +* PYTHONUTF8: PEP 540 Forced UTF-8 Runtime Mode. + (line 6) +* PYTHONUTF8 <1>: Miscellaneous options. + (line 252) +* PYTHONUTF8 <2>: Environment variables. + (line 369) +* PYTHONUTF8 <3>: UTF-8 mode. (line 17) +* PYTHONUTF8 <4>: sys — System-specific parameters and functions. + (line 1505) +* PYTHONUTF8 <5>: Python Configuration. + (line 14) +* PYTHONVERBOSE: Miscellaneous options. + (line 151) +* PYTHONVERBOSE <1>: Global configuration variables. + (line 156) +* PYTHONWARNINGS: warnings. (line 21) +* PYTHONWARNINGS <1>: Other Language Changes<7>. + (line 158) +* PYTHONWARNINGS <2>: Changes to the Handling of Deprecation Warnings. + (line 25) +* PYTHONWARNINGS <3>: Interpreter Changes. + (line 6) +* PYTHONWARNINGS <4>: Miscellaneous options. + (line 169) +* PYTHONWARNINGS <5>: Describing Warning Filters. + (line 7) +* PYTHONWARNINGS <6>: Describing Warning Filters. + (line 21) +* PYTHONWARNINGS <7>: Default Warning Filter. + (line 7) +* python_branch() (in module platform): Cross Platform. (line 86) +* python_build() (in module platform): Cross Platform. (line 76) +* python_compiler() (in module platform): Cross Platform. (line 81) +* PYTHON_DOM: Module Contents<4>. (line 23) +* python_implementation() (in module platform): Cross Platform. + (line 90) +* python_is_optimized() (in module test.support): test support — Utilities for the Python test suite. + (line 203) +* python_revision() (in module platform): Cross Platform. (line 95) +* python_version() (in module platform): Cross Platform. (line 100) +* python_version_tuple() (in module platform): Cross Platform. + (line 107) +* PyThreadState: Thread State and the Global Interpreter Lock. + (line 23) +* PyThreadState <1>: Thread State and the Global Interpreter Lock. + (line 23) +* PyThreadState (C type): High-level API. (line 21) +* PyThreadState_Clear (C function): Low-level API. (line 44) +* PyThreadState_Delete (C function): Low-level API. (line 49) +* PyThreadState_Get (C function): High-level API. (line 76) +* PyThreadState_GetDict (C function): Low-level API. (line 78) +* PyThreadState_New (C function): Low-level API. (line 37) +* PyThreadState_Next (C function): Advanced Debugger Support. + (line 32) +* PyThreadState_SetAsyncExc (C function): Low-level API. (line 87) +* PyThreadState_Swap (C function): High-level API. (line 82) +* PyThread_create_key (C function): Thread Local Storage TLS API. + (line 18) +* PyThread_delete_key (C function): Thread Local Storage TLS API. + (line 20) +* PyThread_delete_key_value (C function): Thread Local Storage TLS API. + (line 26) +* PyThread_get_key_value (C function): Thread Local Storage TLS API. + (line 24) +* PyThread_ReInitTLS (C function): Thread Local Storage TLS API. + (line 28) +* PyThread_set_key_value (C function): Thread Local Storage TLS API. + (line 22) +* PyThread_tss_alloc (C function): Dynamic Allocation. (line 11) +* PyThread_tss_create (C function): Methods<4>. (line 16) +* PyThread_tss_delete (C function): Methods<4>. (line 24) +* PyThread_tss_free (C function): Dynamic Allocation. (line 17) +* PyThread_tss_get (C function): Methods<4>. (line 39) +* PyThread_tss_is_created (C function): Methods<4>. (line 11) +* PyThread_tss_set (C function): Methods<4>. (line 33) +* PyTimeZone_FromOffset (C function): DateTime Objects<2>. + (line 120) +* PyTimeZone_FromOffsetAndName (C function): DateTime Objects<2>. + (line 128) +* PyTime_Check (C function): DateTime Objects<2>. + (line 45) +* PyTime_CheckExact (C function): DateTime Objects<2>. + (line 50) +* PyTime_FromTime (C function): DateTime Objects<2>. + (line 98) +* PyTime_FromTimeAndFold (C function): DateTime Objects<2>. + (line 103) +* PyTraceMalloc_Track (C function): tracemalloc C API. (line 8) +* PyTraceMalloc_Untrack (C function): tracemalloc C API. (line 20) +* PyTrace_CALL (C variable): Profiling and Tracing. + (line 61) +* PyTrace_C_CALL (C variable): Profiling and Tracing. + (line 93) +* PyTrace_C_EXCEPTION (C variable): Profiling and Tracing. + (line 98) +* PyTrace_C_RETURN (C variable): Profiling and Tracing. + (line 103) +* PyTrace_EXCEPTION (C variable): Profiling and Tracing. + (line 69) +* PyTrace_LINE (C variable): Profiling and Tracing. + (line 81) +* PyTrace_OPCODE (C variable): Profiling and Tracing. + (line 108) +* PyTrace_RETURN (C variable): Profiling and Tracing. + (line 88) +* PyTupleObject (C type): Tuple Objects. (line 6) +* PyTuple_Check (C function): Tuple Objects. (line 17) +* PyTuple_CheckExact (C function): Tuple Objects. (line 22) +* PyTuple_ClearFreeList (C function): Tuple Objects. (line 103) +* PyTuple_GetItem (C function): Tuple Objects. (line 48) +* PyTuple_GetSlice (C function): Tuple Objects. (line 57) +* PyTuple_GET_ITEM (C function): Tuple Objects. (line 53) +* PyTuple_GET_SIZE (C function): Tuple Objects. (line 43) +* PyTuple_New (C function): Tuple Objects. (line 27) +* PyTuple_Pack (C function): Tuple Objects. (line 31) +* PyTuple_SetItem (C function): Tuple Objects. (line 64) +* PyTuple_SetItem(): Reference Count Details. + (line 26) +* PyTuple_SET_ITEM (C function): Tuple Objects. (line 76) +* PyTuple_Size (C function): Tuple Objects. (line 38) +* PyTuple_Type (C variable): Tuple Objects. (line 11) +* PyTypeObject (C type): Type Objects<2>. (line 6) +* PyTypeObject.tp_alloc (C member): PyTypeObject Slots. (line 1172) +* PyTypeObject.tp_allocs (C member): PyTypeObject Slots. (line 1394) +* PyTypeObject.tp_as_async (C member): PyTypeObject Slots. (line 240) +* PyTypeObject.tp_as_buffer (C member): PyTypeObject Slots. (line 441) +* PyTypeObject.tp_as_mapping (C member): PyTypeObject Slots. (line 303) +* PyTypeObject.tp_as_number (C member): PyTypeObject Slots. (line 281) +* PyTypeObject.tp_as_sequence (C member): PyTypeObject Slots. + (line 292) +* PyTypeObject.tp_base (C member): PyTypeObject Slots. (line 985) +* PyTypeObject.tp_bases (C member): PyTypeObject Slots. (line 1283) +* PyTypeObject.tp_basicsize (C member): PyTypeObject Slots. (line 47) +* PyTypeObject.tp_cache (C member): PyTypeObject Slots. (line 1305) +* PyTypeObject.tp_call (C member): PyTypeObject Slots. (line 351) +* PyTypeObject.tp_clear (C member): PyTypeObject Slots. (line 722) +* PyTypeObject.tp_dealloc (C member): PyTypeObject Slots. (line 99) +* PyTypeObject.tp_del (C member): PyTypeObject Slots. (line 1330) +* PyTypeObject.tp_descr_get (C member): PyTypeObject Slots. (line 1042) +* PyTypeObject.tp_descr_set (C member): PyTypeObject Slots. (line 1054) +* PyTypeObject.tp_dict (C member): PyTypeObject Slots. (line 1017) +* PyTypeObject.tp_dictoffset (C member): PyTypeObject Slots. (line 1069) +* PyTypeObject.tp_doc (C member): PyTypeObject Slots. (line 650) +* PyTypeObject.tp_finalize (C member): PyTypeObject Slots. (line 1342) +* PyTypeObject.tp_flags (C member): PyTypeObject Slots. (line 452) +* PyTypeObject.tp_free (C member): PyTypeObject Slots. (line 1231) +* PyTypeObject.tp_frees (C member): PyTypeObject Slots. (line 1398) +* PyTypeObject.tp_getattr (C member): PyTypeObject Slots. (line 201) +* PyTypeObject.tp_getattro (C member): PyTypeObject Slots. (line 389) +* PyTypeObject.tp_getset (C member): PyTypeObject Slots. (line 970) +* PyTypeObject.tp_hash (C member): PyTypeObject Slots. (line 314) +* PyTypeObject.tp_init (C member): PyTypeObject Slots. (line 1135) +* PyTypeObject.tp_is_gc (C member): PyTypeObject Slots. (line 1254) +* PyTypeObject.tp_itemsize (C member): PyTypeObject Slots. (line 47) +* PyTypeObject.tp_iter (C member): PyTypeObject Slots. (line 902) +* PyTypeObject.tp_iternext (C member): PyTypeObject Slots. (line 918) +* PyTypeObject.tp_maxalloc (C member): PyTypeObject Slots. (line 1402) +* PyTypeObject.tp_members (C member): PyTypeObject Slots. (line 955) +* PyTypeObject.tp_methods (C member): PyTypeObject Slots. (line 940) +* PyTypeObject.tp_mro (C member): PyTypeObject Slots. (line 1294) +* PyTypeObject.tp_name (C member): PyTypeObject Slots. (line 12) +* PyTypeObject.tp_new (C member): PyTypeObject Slots. (line 1195) +* PyTypeObject.tp_next (C member): PyTypeObject Slots. (line 1411) +* PyTypeObject.tp_prev (C member): PyTypeObject Slots. (line 1406) +* PyTypeObject.tp_repr (C member): PyTypeObject Slots. (line 255) +* PyTypeObject.tp_richcompare (C member): PyTypeObject Slots. + (line 791) +* PyTypeObject.tp_setattr (C member): PyTypeObject Slots. (line 220) +* PyTypeObject.tp_setattro (C member): PyTypeObject Slots. (line 414) +* PyTypeObject.tp_str (C member): PyTypeObject Slots. (line 363) +* PyTypeObject.tp_subclasses (C member): PyTypeObject Slots. (line 1313) +* PyTypeObject.tp_traverse (C member): PyTypeObject Slots. (line 660) +* PyTypeObject.tp_vectorcall_offset (C member): PyTypeObject Slots. + (line 141) +* PyTypeObject.tp_version_tag (C member): PyTypeObject Slots. + (line 1334) +* PyTypeObject.tp_weaklist (C member): PyTypeObject Slots. (line 1321) +* PyTypeObject.tp_weaklistoffset (C member): PyTypeObject Slots. + (line 866) +* PyType_Check (C function): Type Objects<2>. (line 15) +* PyType_CheckExact (C function): Type Objects<2>. (line 21) +* PyType_ClearCache (C function): Type Objects<2>. (line 26) +* PyType_FromSpec (C function): Creating Heap-Allocated Types. + (line 26) +* PyType_FromSpecWithBases (C function): Creating Heap-Allocated Types. + (line 9) +* PyType_GenericAlloc (C function): Type Objects<2>. (line 68) +* PyType_GenericNew (C function): Type Objects<2>. (line 76) +* PyType_GetFlags (C function): Type Objects<2>. (line 30) +* PyType_GetSlot (C function): Type Objects<2>. (line 90) +* PyType_HasFeature (C function): Type Objects<2>. (line 49) +* PyType_IsSubtype (C function): Type Objects<2>. (line 59) +* PyType_IS_GC (C function): Type Objects<2>. (line 54) +* PyType_Modified (C function): Type Objects<2>. (line 43) +* PyType_Ready (C function): Type Objects<2>. (line 83) +* PyType_Slot (C type): Creating Heap-Allocated Types. + (line 59) +* PyType_Slot.PyType_Slot.pfunc (C member): Creating Heap-Allocated Types. + (line 109) +* PyType_Slot.PyType_Slot.slot (C member): Creating Heap-Allocated Types. + (line 64) +* PyType_Spec (C type): Creating Heap-Allocated Types. + (line 30) +* PyType_Spec.PyType_Spec.basicsize (C member): Creating Heap-Allocated Types. + (line 39) +* PyType_Spec.PyType_Spec.flags (C member): Creating Heap-Allocated Types. + (line 47) +* PyType_Spec.PyType_Spec.itemsize (C member): Creating Heap-Allocated Types. + (line 41) +* PyType_Spec.PyType_Spec.name (C member): Creating Heap-Allocated Types. + (line 34) +* PyType_Spec.PyType_Spec.slots (C member): Creating Heap-Allocated Types. + (line 54) +* PyType_Type (C variable): Type Objects<2>. (line 10) +* PyTZInfo_Check (C function): DateTime Objects<2>. + (line 65) +* PyTZInfo_CheckExact (C function): DateTime Objects<2>. + (line 70) +* PyUnicodeDecodeError_Create (C function): Unicode Exception Objects. + (line 9) +* PyUnicodeDecodeError_GetEncoding (C function): Unicode Exception Objects. + (line 47) +* PyUnicodeDecodeError_GetEnd (C function): Unicode Exception Objects. + (line 88) +* PyUnicodeDecodeError_GetObject (C function): Unicode Exception Objects. + (line 54) +* PyUnicodeDecodeError_GetReason (C function): Unicode Exception Objects. + (line 115) +* PyUnicodeDecodeError_GetStart (C function): Unicode Exception Objects. + (line 61) +* PyUnicodeDecodeError_SetEnd (C function): Unicode Exception Objects. + (line 102) +* PyUnicodeDecodeError_SetReason (C function): Unicode Exception Objects. + (line 122) +* PyUnicodeDecodeError_SetStart (C function): Unicode Exception Objects. + (line 75) +* PyUnicodeEncodeError_Create (C function): Unicode Exception Objects. + (line 18) +* PyUnicodeEncodeError_GetEncoding (C function): Unicode Exception Objects. + (line 47) +* PyUnicodeEncodeError_GetEnd (C function): Unicode Exception Objects. + (line 88) +* PyUnicodeEncodeError_GetObject (C function): Unicode Exception Objects. + (line 54) +* PyUnicodeEncodeError_GetReason (C function): Unicode Exception Objects. + (line 115) +* PyUnicodeEncodeError_GetStart (C function): Unicode Exception Objects. + (line 61) +* PyUnicodeEncodeError_SetEnd (C function): Unicode Exception Objects. + (line 102) +* PyUnicodeEncodeError_SetReason (C function): Unicode Exception Objects. + (line 122) +* PyUnicodeEncodeError_SetStart (C function): Unicode Exception Objects. + (line 75) +* PyUnicodeObject (C type): Unicode Type. (line 29) +* PyUnicodeTranslateError_Create (C function): Unicode Exception Objects. + (line 32) +* PyUnicodeTranslateError_GetEnd (C function): Unicode Exception Objects. + (line 88) +* PyUnicodeTranslateError_GetObject (C function): Unicode Exception Objects. + (line 54) +* PyUnicodeTranslateError_GetReason (C function): Unicode Exception Objects. + (line 115) +* PyUnicodeTranslateError_GetStart (C function): Unicode Exception Objects. + (line 61) +* PyUnicodeTranslateError_SetEnd (C function): Unicode Exception Objects. + (line 102) +* PyUnicodeTranslateError_SetReason (C function): Unicode Exception Objects. + (line 122) +* PyUnicodeTranslateError_SetStart (C function): Unicode Exception Objects. + (line 75) +* PyUnicode_1BYTE_DATA (C function): Unicode Type. (line 80) +* PyUnicode_1BYTE_KIND (C macro): Unicode Type. (line 93) +* PyUnicode_2BYTE_DATA (C function): Unicode Type. (line 80) +* PyUnicode_2BYTE_KIND (C macro): Unicode Type. (line 93) +* PyUnicode_4BYTE_DATA (C function): Unicode Type. (line 80) +* PyUnicode_4BYTE_KIND (C macro): Unicode Type. (line 93) +* PyUnicode_AsASCIIString (C function): ASCII Codecs. (line 16) +* PyUnicode_AsCharmapString (C function): Character Map Codecs. + (line 30) +* PyUnicode_AsEncodedString (C function): Generic Codecs. (line 17) +* PyUnicode_AsLatin1String (C function): Latin-1 Codecs. (line 17) +* PyUnicode_AsMBCSString (C function): MBCS codecs for Windows. + (line 28) +* PyUnicode_AsRawUnicodeEscapeString (C function): Raw-Unicode-Escape Codecs. + (line 14) +* PyUnicode_AsUCS4 (C function): Creating and accessing Unicode strings. + (line 251) +* PyUnicode_AsUCS4Copy (C function): Creating and accessing Unicode strings. + (line 261) +* PyUnicode_AsUnicode (C function): Deprecated Py_UNICODE APIs. + (line 35) +* PyUnicode_AsUnicodeAndSize (C function): Deprecated Py_UNICODE APIs. + (line 63) +* PyUnicode_AsUnicodeCopy (C function): Deprecated Py_UNICODE APIs. + (line 79) +* PyUnicode_AsUnicodeEscapeString (C function): Unicode-Escape Codecs. + (line 15) +* PyUnicode_AsUTF16String (C function): UTF-16 Codecs. (line 50) +* PyUnicode_AsUTF32String (C function): UTF-32 Codecs. (line 49) +* PyUnicode_AsUTF8 (C function): UTF-8 Codecs. (line 53) +* PyUnicode_AsUTF8AndSize (C function): UTF-8 Codecs. (line 30) +* PyUnicode_AsUTF8String (C function): UTF-8 Codecs. (line 25) +* PyUnicode_AsWideChar (C function): wchar_t Support. (line 16) +* PyUnicode_AsWideCharString (C function): wchar_t Support. (line 31) +* PyUnicode_AS_DATA (C function): Unicode Type. (line 186) +* PyUnicode_AS_UNICODE (C function): Unicode Type. (line 186) +* PyUnicode_Check (C function): Unicode Type. (line 48) +* PyUnicode_CheckExact (C function): Unicode Type. (line 53) +* PyUnicode_ClearFreeList (C function): Unicode Type. (line 162) +* PyUnicode_Compare (C function): Methods and Slot Functions. + (line 87) +* PyUnicode_CompareWithASCIIString (C function): Methods and Slot Functions. + (line 95) +* PyUnicode_Concat (C function): Methods and Slot Functions. + (line 12) +* PyUnicode_Contains (C function): Methods and Slot Functions. + (line 126) +* PyUnicode_CopyCharacters (C function): Creating and accessing Unicode strings. + (line 190) +* PyUnicode_Count (C function): Methods and Slot Functions. + (line 74) +* PyUnicode_DATA (C function): Unicode Type. (line 114) +* PyUnicode_Decode (C function): Generic Codecs. (line 8) +* PyUnicode_DecodeASCII (C function): ASCII Codecs. (line 9) +* PyUnicode_DecodeCharmap (C function): Character Map Codecs. + (line 15) +* PyUnicode_DecodeFSDefault (C function): File System Encoding. + (line 65) +* PyUnicode_DecodeFSDefaultAndSize (C function): File System Encoding. + (line 43) +* PyUnicode_DecodeLatin1 (C function): Latin-1 Codecs. (line 10) +* PyUnicode_DecodeLocale (C function): Locale Encoding. (line 37) +* PyUnicode_DecodeLocaleAndSize (C function): Locale Encoding. + (line 9) +* PyUnicode_DecodeMBCS (C function): MBCS codecs for Windows. + (line 12) +* PyUnicode_DecodeMBCSStateful (C function): MBCS codecs for Windows. + (line 19) +* PyUnicode_DecodeRawUnicodeEscape (C function): Raw-Unicode-Escape Codecs. + (line 8) +* PyUnicode_DecodeUnicodeEscape (C function): Unicode-Escape Codecs. + (line 8) +* PyUnicode_DecodeUTF16 (C function): UTF-16 Codecs. (line 8) +* PyUnicode_DecodeUTF16Stateful (C function): UTF-16 Codecs. (line 37) +* PyUnicode_DecodeUTF32 (C function): UTF-32 Codecs. (line 8) +* PyUnicode_DecodeUTF32Stateful (C function): UTF-32 Codecs. (line 36) +* PyUnicode_DecodeUTF7 (C function): UTF-7 Codecs. (line 8) +* PyUnicode_DecodeUTF7Stateful (C function): UTF-7 Codecs. (line 15) +* PyUnicode_DecodeUTF8 (C function): UTF-8 Codecs. (line 8) +* PyUnicode_DecodeUTF8Stateful (C function): UTF-8 Codecs. (line 15) +* PyUnicode_Encode (C function): Generic Codecs. (line 26) +* PyUnicode_EncodeASCII (C function): ASCII Codecs. (line 21) +* PyUnicode_EncodeCharmap (C function): Character Map Codecs. + (line 44) +* PyUnicode_EncodeCodePage (C function): MBCS codecs for Windows. + (line 33) +* PyUnicode_EncodeFSDefault (C function): File System Encoding. + (line 79) +* PyUnicode_EncodeLatin1 (C function): Latin-1 Codecs. (line 23) +* PyUnicode_EncodeLocale (C function): Locale Encoding. (line 45) +* PyUnicode_EncodeMBCS (C function): MBCS codecs for Windows. + (line 43) +* PyUnicode_EncodeRawUnicodeEscape (C function): Raw-Unicode-Escape Codecs. + (line 21) +* PyUnicode_EncodeUnicodeEscape (C function): Unicode-Escape Codecs. + (line 22) +* PyUnicode_EncodeUTF16 (C function): UTF-16 Codecs. (line 56) +* PyUnicode_EncodeUTF32 (C function): UTF-32 Codecs. (line 55) +* PyUnicode_EncodeUTF7 (C function): UTF-7 Codecs. (line 25) +* PyUnicode_EncodeUTF8 (C function): UTF-8 Codecs. (line 63) +* PyUnicode_Fill (C function): Creating and accessing Unicode strings. + (line 203) +* PyUnicode_Find (C function): Methods and Slot Functions. + (line 48) +* PyUnicode_FindChar (C function): Methods and Slot Functions. + (line 59) +* PyUnicode_Format (C function): Methods and Slot Functions. + (line 121) +* PyUnicode_FromEncodedObject (C function): Creating and accessing Unicode strings. + (line 167) +* PyUnicode_FromFormat (C function): Creating and accessing Unicode strings. + (line 50) +* PyUnicode_FromFormatV (C function): Creating and accessing Unicode strings. + (line 160) +* PyUnicode_FromKindAndData (C function): Creating and accessing Unicode strings. + (line 21) +* PyUnicode_FromObject (C function): Deprecated Py_UNICODE APIs. + (line 104) +* PyUnicode_FromString (C function): Creating and accessing Unicode strings. + (line 46) +* PyUnicode_FromString(): Dictionary Objects. (line 64) +* PyUnicode_FromStringAndSize (C function): Creating and accessing Unicode strings. + (line 32) +* PyUnicode_FromUnicode (C function): Deprecated Py_UNICODE APIs. + (line 13) +* PyUnicode_FromWideChar (C function): wchar_t Support. (line 8) +* PyUnicode_FSConverter (C function): File System Encoding. + (line 13) +* PyUnicode_FSDecoder (C function): File System Encoding. + (line 30) +* PyUnicode_GetLength (C function): Creating and accessing Unicode strings. + (line 184) +* PyUnicode_GetSize (C function): Deprecated Py_UNICODE APIs. + (line 94) +* PyUnicode_GET_DATA_SIZE (C function): Unicode Type. (line 176) +* PyUnicode_GET_LENGTH (C function): Unicode Type. (line 72) +* PyUnicode_GET_SIZE (C function): Unicode Type. (line 166) +* PyUnicode_InternFromString (C function): Methods and Slot Functions. + (line 148) +* PyUnicode_InternInPlace (C function): Methods and Slot Functions. + (line 135) +* PyUnicode_Join (C function): Methods and Slot Functions. + (line 34) +* PyUnicode_KIND (C function): Unicode Type. (line 105) +* PyUnicode_MAX_CHAR_VALUE (C macro): Unicode Type. (line 153) +* PyUnicode_New (C function): Creating and accessing Unicode strings. + (line 9) +* PyUnicode_READ (C function): Unicode Type. (line 134) +* PyUnicode_ReadChar (C function): Creating and accessing Unicode strings. + (line 232) +* PyUnicode_READY (C function): Unicode Type. (line 58) +* PyUnicode_READ_CHAR (C function): Unicode Type. (line 144) +* PyUnicode_Replace (C function): Methods and Slot Functions. + (line 80) +* PyUnicode_RichCompare (C function): Methods and Slot Functions. + (line 106) +* PyUnicode_Split (C function): Methods and Slot Functions. + (line 17) +* PyUnicode_Splitlines (C function): Methods and Slot Functions. + (line 27) +* PyUnicode_Substring (C function): Creating and accessing Unicode strings. + (line 242) +* PyUnicode_Tailmatch (C function): Methods and Slot Functions. + (line 39) +* PyUnicode_TransformDecimalToASCII (C function): Deprecated Py_UNICODE APIs. + (line 52) +* PyUnicode_Translate (C function): Character Map Codecs. + (line 59) +* PyUnicode_TranslateCharmap (C function): Character Map Codecs. + (line 76) +* PyUnicode_Type (C variable): Unicode Type. (line 40) +* PyUnicode_WCHAR_KIND (C macro): Unicode Type. (line 93) +* PyUnicode_WRITE (C function): Unicode Type. (line 121) +* PyUnicode_WriteChar (C function): Creating and accessing Unicode strings. + (line 218) +* PyVarObject (C type): Common Object Structures. + (line 28) +* PyVarObject.ob_size (C member): PyVarObject Slots. (line 6) +* PyVarObject_HEAD_INIT (C macro): Common Object Structures. + (line 86) +* PyVectorcall_Call (C function): PyTypeObject Slots. (line 164) +* PyVectorcall_NARGS (C function): Object Protocol. (line 395) +* PyWeakref_Check (C function): Weak Reference Objects<2>. + (line 11) +* PyWeakref_CheckProxy (C function): Weak Reference Objects<2>. + (line 19) +* PyWeakref_CheckRef (C function): Weak Reference Objects<2>. + (line 15) +* PyWeakref_GetObject (C function): Weak Reference Objects<2>. + (line 50) +* PyWeakref_GET_OBJECT (C function): Weak Reference Objects<2>. + (line 60) +* PyWeakref_NewProxy (C function): Weak Reference Objects<2>. + (line 36) +* PyWeakref_NewRef (C function): Weak Reference Objects<2>. + (line 23) +* PyWideStringList (C type): PyWideStringList. (line 6) +* PyWideStringList.items (C member): PyWideStringList. (line 41) +* PyWideStringList.length (C member): PyWideStringList. (line 37) +* PyWideStringList_Append (C function): PyWideStringList. (line 15) +* PyWideStringList_Insert (C function): PyWideStringList. (line 22) +* PyWrapper_New (C function): Descriptor Objects. (line 43) +* PyZipFile (class in zipfile): PyZipFile Objects. (line 10) +* Py_ABS (C macro): Useful macros. (line 21) +* Py_AddPendingCall (C function): Asynchronous Notifications. + (line 10) +* Py_AddPendingCall(): Asynchronous Notifications. + (line 12) +* Py_AtExit (C function): Process Control. (line 24) +* Py_BEGIN_ALLOW_THREADS: Releasing the GIL from extension code. + (line 21) +* Py_BEGIN_ALLOW_THREADS (C macro): High-level API. (line 158) +* Py_BLOCK_THREADS (C macro): High-level API. (line 172) +* Py_buffer (C type): Buffer structure. (line 25) +* Py_buffer.buf (C member): Buffer structure. (line 27) +* Py_buffer.format (C member): Buffer structure. (line 87) +* Py_buffer.internal (C member): Buffer structure. (line 154) +* Py_buffer.itemsize (C member): Buffer structure. (line 67) +* Py_buffer.len (C member): Buffer structure. (line 50) +* Py_buffer.ndim (C member): Buffer structure. (line 96) +* Py_buffer.obj (C member): Buffer structure. (line 38) +* Py_buffer.readonly (C member): Buffer structure. (line 62) +* Py_buffer.shape (C member): Buffer structure. (line 109) +* Py_buffer.strides (C member): Buffer structure. (line 122) +* Py_buffer.suboffsets (C member): Buffer structure. (line 135) +* Py_BuildValue (C function): Building values. (line 6) +* Py_BytesMain (C function): The Very High Level Layer. + (line 41) +* Py_BytesWarningFlag (C variable): Global configuration variables. + (line 15) +* Py_CHARMASK (C macro): Useful macros. (line 52) +* Py_CLEAR (C function): Reference Counting. (line 47) +* py_compile (module): py_compile — Compile Python source files. + (line 6) +* PY_COMPILED (in module imp): imp — Access the import internals. + (line 315) +* Py_CompileString (C function): The Very High Level Layer. + (line 286) +* Py_CompileString(): The Very High Level Layer. + (line 380) +* Py_CompileString() <1>: The Very High Level Layer. + (line 385) +* Py_CompileString() <2>: The Very High Level Layer. + (line 392) +* Py_CompileStringExFlags (C function): The Very High Level Layer. + (line 320) +* Py_CompileStringFlags (C function): The Very High Level Layer. + (line 292) +* Py_CompileStringObject (C function): The Very High Level Layer. + (line 299) +* Py_complex (C type): Complex Numbers as C Structures. + (line 10) +* Py_DebugFlag (C variable): Global configuration variables. + (line 23) +* Py_DecodeLocale (C function): Operating System Utilities. + (line 108) +* Py_DECREF (C function): Reference Counting. (line 20) +* Py_DECREF(): Reference Counts<2>. + (line 17) +* Py_DEPRECATED (C macro): Useful macros. (line 70) +* Py_DontWriteBytecodeFlag (C variable): Global configuration variables. + (line 31) +* Py_Ellipsis (C variable): Ellipsis Object. (line 6) +* Py_EncodeLocale (C function): Operating System Utilities. + (line 161) +* Py_EndInterpreter (C function): Sub-interpreter support. + (line 76) +* Py_END_ALLOW_THREADS: Releasing the GIL from extension code. + (line 21) +* Py_END_ALLOW_THREADS (C macro): High-level API. (line 165) +* Py_EnterRecursiveCall (C function): Recursion Control. (line 11) +* Py_eval_input (C variable): The Very High Level Layer. + (line 378) +* Py_Exit (C function): Process Control. (line 15) +* Py_ExitStatusException (C function): PyStatus. (line 62) +* Py_False (C variable): Boolean Objects. (line 15) +* Py_FatalError (C function): Process Control. (line 6) +* Py_FatalError(): Process-wide parameters. + (line 217) +* Py_FdIsInteractive (C function): Operating System Utilities. + (line 17) +* Py_file_input (C variable): The Very High Level Layer. + (line 383) +* Py_Finalize (C function): Initializing and finalizing the interpreter. + (line 78) +* Py_FinalizeEx (C function): Initializing and finalizing the interpreter. + (line 39) +* Py_FinalizeEx(): Process Control. (line 17) +* Py_FinalizeEx() <1>: Process Control. (line 26) +* Py_FinalizeEx() <2>: Initializing and finalizing the interpreter. + (line 8) +* Py_FinalizeEx() <3>: Sub-interpreter support. + (line 49) +* Py_FinalizeEx() <4>: Sub-interpreter support. + (line 78) +* PY_FROZEN (in module imp): imp — Access the import internals. + (line 339) +* Py_FrozenFlag (C variable): Global configuration variables. + (line 39) +* Py_GetBuildInfo (C function): Process-wide parameters. + (line 203) +* Py_GetCompiler (C function): Process-wide parameters. + (line 192) +* Py_GetCopyright (C function): Process-wide parameters. + (line 181) +* Py_GETENV (C macro): Useful macros. (line 57) +* Py_GetExecPrefix (C function): Process-wide parameters. + (line 71) +* Py_GetExecPrefix(): Embedding Python<2>. + (line 42) +* Py_GetPath (C function): Process-wide parameters. + (line 118) +* Py_GetPath(): Embedding Python<2>. + (line 42) +* Py_GetPath() <1>: Process-wide parameters. + (line 36) +* Py_GetPath() <2>: Process-wide parameters. + (line 133) +* Py_GetPlatform (C function): Process-wide parameters. + (line 170) +* Py_GetPrefix (C function): Process-wide parameters. + (line 57) +* Py_GetPrefix(): Embedding Python<2>. + (line 42) +* Py_GetProgramFullPath (C function): Process-wide parameters. + (line 109) +* Py_GetProgramFullPath(): Embedding Python<2>. + (line 42) +* Py_GetProgramName (C function): Process-wide parameters. + (line 51) +* Py_GetPythonHome (C function): Process-wide parameters. + (line 281) +* Py_GetVersion (C function): Process-wide parameters. + (line 157) +* Py_HashRandomizationFlag (C variable): Global configuration variables. + (line 46) +* Py_IgnoreEnvironmentFlag (C variable): Global configuration variables. + (line 54) +* Py_INCREF (C function): Reference Counting. (line 9) +* Py_INCREF(): Reference Counts<2>. + (line 17) +* Py_Initialize (C function): Initializing and finalizing the interpreter. + (line 6) +* Py_Initialize(): Embedding Python<2>. + (line 12) +* Py_Initialize() <1>: Process-wide parameters. + (line 9) +* Py_Initialize() <2>: Process-wide parameters. + (line 36) +* Py_Initialize() <3>: Sub-interpreter support. + (line 49) +* Py_InitializeEx (C function): Initializing and finalizing the interpreter. + (line 26) +* Py_InitializeFromConfig (C function): Initialization with PyConfig. + (line 8) +* Py_InspectFlag (C variable): Global configuration variables. + (line 61) +* Py_InteractiveFlag (C variable): Global configuration variables. + (line 71) +* Py_IsInitialized (C function): Initializing and finalizing the interpreter. + (line 32) +* Py_IsInitialized(): Embedding Python<2>. + (line 51) +* Py_IsolatedFlag (C variable): Global configuration variables. + (line 75) +* Py_LeaveRecursiveCall (C function): Recursion Control. (line 29) +* Py_LegacyWindowsFSEncodingFlag (C variable): Global configuration variables. + (line 85) +* Py_LegacyWindowsStdioFlag (C variable): Global configuration variables. + (line 97) +* Py_Main (C function): The Very High Level Layer. + (line 24) +* Py_MAX (C macro): Useful macros. (line 33) +* Py_MEMBER_SIZE (C macro): Useful macros. (line 46) +* Py_MIN (C macro): Useful macros. (line 27) +* Py_mod_create (C macro): Multi-phase initialization. + (line 61) +* Py_mod_exec (C macro): Multi-phase initialization. + (line 96) +* Py_NewInterpreter (C function): Sub-interpreter support. + (line 24) +* Py_None (C variable): The None Object. (line 11) +* Py_NoSiteFlag (C variable): Global configuration variables. + (line 109) +* Py_NotImplemented (C variable): Object Protocol. (line 6) +* Py_NoUserSiteDirectory (C variable): Global configuration variables. + (line 119) +* py_object (class in ctypes): Fundamental data types<2>. + (line 213) +* Py_OptimizeFlag (C variable): Global configuration variables. + (line 127) +* Py_PreInitialize (C function): Preinitialization with PyPreConfig. + (line 8) +* Py_PreInitializeFromArgs (C function): Preinitialization with PyPreConfig. + (line 19) +* Py_PreInitializeFromBytesArgs (C function): Preinitialization with PyPreConfig. + (line 12) +* Py_PRINT_RAW: File Objects. (line 87) +* PY_PYTHON: Customizing default Python versions. + (line 18) +* Py_QuietFlag (C variable): Global configuration variables. + (line 132) +* Py_REFCNT (C macro): Common Object Structures. + (line 64) +* Py_ReprEnter (C function): Recursion Control. (line 41) +* Py_ReprLeave (C function): Recursion Control. (line 59) +* Py_RETURN_FALSE (C macro): Boolean Objects. (line 27) +* Py_RETURN_NONE (C macro): The None Object. (line 17) +* Py_RETURN_NOTIMPLEMENTED (C macro): Object Protocol. (line 11) +* Py_RETURN_RICHCOMPARE (C macro): PyTypeObject Slots. (line 835) +* Py_RETURN_TRUE (C macro): Boolean Objects. (line 32) +* Py_RunMain (C function): Py_RunMain. (line 6) +* Py_SetPath (C function): Process-wide parameters. + (line 131) +* Py_SetPath(): Process-wide parameters. + (line 120) +* Py_SetProgramName (C function): Process-wide parameters. + (line 34) +* Py_SetProgramName(): Embedding Python<2>. + (line 42) +* Py_SetProgramName() <1>: Initializing and finalizing the interpreter. + (line 8) +* Py_SetProgramName() <2>: Process-wide parameters. + (line 53) +* Py_SetProgramName() <3>: Process-wide parameters. + (line 111) +* Py_SetPythonHome (C function): Process-wide parameters. + (line 267) +* Py_SetStandardStreamEncoding (C function): Process-wide parameters. + (line 6) +* Py_single_input (C variable): The Very High Level Layer. + (line 390) +* Py_SIZE (C macro): Common Object Structures. + (line 71) +* PY_SOURCE (in module imp): imp — Access the import internals. + (line 309) +* PY_SSIZE_T_MAX: Integer Objects. (line 188) +* Py_STRINGIFY (C macro): Useful macros. (line 39) +* Py_TPFLAGS_BASETYPE (built-in variable): PyTypeObject Slots. + (line 507) +* Py_TPFLAGS_BASE_EXC_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 604) +* Py_TPFLAGS_BYTES_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 598) +* Py_TPFLAGS_DEFAULT (built-in variable): PyTypeObject Slots. + (line 556) +* Py_TPFLAGS_DICT_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 602) +* Py_TPFLAGS_HAVE_FINALIZE (built-in variable): PyTypeObject Slots. + (line 617) +* Py_TPFLAGS_HAVE_GC (built-in variable): PyTypeObject Slots. + (line 535) +* Py_TPFLAGS_HEAPTYPE (built-in variable): PyTypeObject Slots. + (line 492) +* Py_TPFLAGS_LIST_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 594) +* Py_TPFLAGS_LONG_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 592) +* Py_TPFLAGS_METHOD_DESCRIPTOR (built-in variable): PyTypeObject Slots. + (line 568) +* Py_TPFLAGS_READY (built-in variable): PyTypeObject Slots. (line 517) +* Py_TPFLAGS_READYING (built-in variable): PyTypeObject Slots. + (line 526) +* Py_TPFLAGS_TUPLE_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 596) +* Py_TPFLAGS_TYPE_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 606) +* Py_TPFLAGS_UNICODE_SUBCLASS (built-in variable): PyTypeObject Slots. + (line 600) +* Py_tracefunc (C type): Profiling and Tracing. + (line 18) +* Py_True (C variable): Boolean Objects. (line 21) +* Py_tss_NEEDS_INIT (C macro): Thread Specific Storage TSS API. + (line 28) +* Py_tss_t (C type): Thread Specific Storage TSS API. + (line 17) +* Py_TYPE (C macro): Common Object Structures. + (line 57) +* Py_UCS1 (C type): Unicode Type. (line 9) +* Py_UCS2 (C type): Unicode Type. (line 9) +* Py_UCS4 (C type): Unicode Type. (line 9) +* Py_UNBLOCK_THREADS (C macro): High-level API. (line 178) +* Py_UnbufferedStdioFlag (C variable): Global configuration variables. + (line 141) +* Py_UNICODE (C type): Unicode Type. (line 20) +* Py_UNICODE_ISALNUM (C function): Unicode Character Properties. + (line 52) +* Py_UNICODE_ISALPHA (C function): Unicode Character Properties. + (line 47) +* Py_UNICODE_ISDECIMAL (C function): Unicode Character Properties. + (line 35) +* Py_UNICODE_ISDIGIT (C function): Unicode Character Properties. + (line 39) +* Py_UNICODE_ISLINEBREAK (C function): Unicode Character Properties. + (line 30) +* Py_UNICODE_ISLOWER (C function): Unicode Character Properties. + (line 15) +* Py_UNICODE_ISNUMERIC (C function): Unicode Character Properties. + (line 43) +* Py_UNICODE_ISPRINTABLE (C function): Unicode Character Properties. + (line 57) +* Py_UNICODE_ISSPACE (C function): Unicode Character Properties. + (line 10) +* Py_UNICODE_ISTITLE (C function): Unicode Character Properties. + (line 25) +* Py_UNICODE_ISUPPER (C function): Unicode Character Properties. + (line 20) +* Py_UNICODE_IS_HIGH_SURROGATE (C macro): Unicode Character Properties. + (line 114) +* Py_UNICODE_IS_LOW_SURROGATE (C macro): Unicode Character Properties. + (line 118) +* Py_UNICODE_IS_SURROGATE (C macro): Unicode Character Properties. + (line 110) +* Py_UNICODE_JOIN_SURROGATES (C macro): Unicode Character Properties. + (line 122) +* Py_UNICODE_TODECIMAL (C function): Unicode Character Properties. + (line 91) +* Py_UNICODE_TODIGIT (C function): Unicode Character Properties. + (line 97) +* Py_UNICODE_TOLOWER (C function): Unicode Character Properties. + (line 70) +* Py_UNICODE_TONUMERIC (C function): Unicode Character Properties. + (line 103) +* Py_UNICODE_TOTITLE (C function): Unicode Character Properties. + (line 84) +* Py_UNICODE_TOUPPER (C function): Unicode Character Properties. + (line 77) +* Py_UNREACHABLE (C macro): Useful macros. (line 11) +* Py_UNUSED (C macro): Useful macros. (line 62) +* Py_VaBuildValue (C function): Building values. (line 200) +* PY_VECTORCALL_ARGUMENTS_OFFSET (C macro): Object Protocol. (line 381) +* Py_VerboseFlag (C variable): Global configuration variables. + (line 148) +* Py_VISIT (C function): Supporting Cyclic Garbage Collection. + (line 117) +* Py_XDECREF (C function): Reference Counting. (line 40) +* Py_XDECREF(): Exceptions<18>. (line 123) +* Py_XINCREF (C function): Reference Counting. (line 15) +* P_ALL (in module os): Process Management. (line 736) +* P_DETACH (in module os): Process Management. (line 599) +* P_NOWAIT (in module os): Process Management. (line 579) +* P_NOWAITO (in module os): Process Management. (line 579) +* P_OVERLAY (in module os): Process Management. (line 599) +* P_PGID (in module os): Process Management. (line 736) +* P_PID (in module os): Process Management. (line 736) +* P_WAIT (in module os): Process Management. (line 589) +* qiflush() (in module curses): Functions<3>. (line 382) +* QName (class in xml.etree.ElementTree): QName Objects. (line 6) +* qsize() (asyncio.Queue method): Queue. (line 74) +* qsize() (multiprocessing.Queue method): Pipes and Queues. (line 98) +* qsize() (queue.Queue method): Queue Objects. (line 9) +* qsize() (queue.SimpleQueue method): SimpleQueue Objects. + (line 9) +* qualified name: Glossary. (line 1100) +* quantiles() (in module statistics): Function details. (line 400) +* quantiles() (statistics.NormalDist method): NormalDist objects. + (line 103) +* quantize() (decimal.Context method): Context objects. (line 459) +* quantize() (decimal.Decimal method): Decimal objects. (line 462) +* QueryInfoKey() (in module winreg): Functions<11>. (line 324) +* QueryReflectionKey() (in module winreg): Functions<11>. (line 521) +* QueryValue() (in module winreg): Functions<11>. (line 353) +* QueryValueEx() (in module winreg): Functions<11>. (line 373) +* Queue (class in asyncio): Queue. (line 6) +* Queue (class in multiprocessing): Pipes and Queues. (line 84) +* Queue (class in queue): queue — A synchronized queue class. + (line 34) +* queue (module): queue — A synchronized queue class. + (line 6) +* queue (sched.scheduler attribute): Scheduler Objects. (line 74) +* Queue() (multiprocessing.managers.SyncManager method): Managers. + (line 181) +* QueueEmpty: Exceptions<8>. (line 6) +* QueueFull: Exceptions<8>. (line 11) +* QueueHandler (class in logging.handlers): QueueHandler. (line 21) +* QueueListener (class in logging.handlers): QueueListener. (line 25) +* quick_ratio() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 202) +* quit (built-in variable): Constants added by the site module. + (line 11) +* quit (pdb command): Debugger Commands. (line 331) +* quit() (ftplib.FTP method): FTP Objects. (line 237) +* quit() (nntplib.NNTP method): Methods<3>. (line 22) +* quit() (poplib.POP3 method): POP3 Objects. (line 82) +* quit() (smtplib.SMTP method): SMTP Objects. (line 344) +* quopri (module): quopri — Encode and decode MIME quoted-printable data. + (line 6) +* quote() (in module email.utils): email utils Miscellaneous utilities. + (line 47) +* quote() (in module shlex): shlex — Simple lexical analysis. + (line 44) +* quote() (in module urllib.parse): URL Quoting. (line 13) +* quoteattr() (in module xml.sax.saxutils): xml sax saxutils — SAX Utilities. + (line 34) +* quotechar (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 52) +* quoted-printable; encoding: quopri — Encode and decode MIME quoted-printable data. + (line 8) +* quotes (shlex.shlex attribute): shlex Objects. (line 115) +* QUOTE_ALL (in module csv): Module Contents<3>. (line 234) +* quote_from_bytes() (in module urllib.parse): URL Quoting. (line 52) +* QUOTE_MINIMAL (in module csv): Module Contents<3>. (line 238) +* QUOTE_NONE (in module csv): Module Contents<3>. (line 252) +* QUOTE_NONNUMERIC (in module csv): Module Contents<3>. (line 244) +* quote_plus() (in module urllib.parse): URL Quoting. (line 41) +* quoting (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 58) +* r"; raw string literal: String and Bytes literals. + (line 48) +* r’; raw string literal: String and Bytes literals. + (line 48) +* radians() (in module math): Angular conversion. (line 10) +* radians() (in module turtle): Settings for measurement. + (line 28) +* RadioButtonGroup (class in msilib): GUI classes. (line 29) +* radiogroup() (msilib.Dialog method): GUI classes. (line 73) +* radix() (decimal.Context method): Context objects. (line 464) +* radix() (decimal.Decimal method): Decimal objects. (line 490) +* RADIXCHAR (in module locale): locale — Internationalization services. + (line 232) +* raise (2to3 fixer): Fixers. (line 261) +* raise an exception: Exceptions<2>. (line 6) +* raise_on_defect (email.policy.Policy attribute): email policy Policy Objects. + (line 180) +* raise_signal() (in module signal): Module contents<2>. (line 276) +* RAISE_VARARGS (opcode): Python Bytecode Instructions. + (line 757) +* raising; exception: The raise statement. + (line 6) +* randbelow() (in module secrets): Random numbers. (line 19) +* randbits() (in module secrets): Random numbers. (line 23) +* randint() (in module random): Functions for integers. + (line 23) +* Random (class in random): Alternative Generator. + (line 6) +* random (module): random — Generate pseudo-random numbers. + (line 6) +* random() (in module random): Real-valued distributions. + (line 11) +* randrange() (in module random): Functions for integers. + (line 6) +* RAND_add() (in module ssl): Random generation. (line 66) +* RAND_bytes() (in module ssl): Random generation. (line 6) +* RAND_egd() (in module ssl): Random generation. (line 50) +* RAND_pseudo_bytes() (in module ssl): Random generation. (line 23) +* RAND_status() (in module ssl): Random generation. (line 43) +* range (built-in class): Ranges. (line 10) +* RARROW (in module token): token — Constants used with Python parse trees. + (line 230) +* ratecv() (in module audioop): audioop — Manipulate raw audio data. + (line 175) +* ratio() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 178) +* Rational (class in numbers): The numeric tower. (line 41) +* raw (io.BufferedIOBase attribute): I/O Base Classes. (line 249) +* raw string: String and Bytes literals. + (line 36) +* raw() (in module curses): Functions<3>. (line 389) +* raw() (pickle.PickleBuffer method): Module Interface. (line 317) +* RawArray() (in module multiprocessing.sharedctypes): The multiprocessing sharedctypes module. + (line 16) +* RawConfigParser (class in configparser): RawConfigParser Objects. + (line 6) +* RawDescriptionHelpFormatter (class in argparse): formatter_class. + (line 10) +* RawIOBase (class in io): I/O Base Classes. (line 170) +* RawPen (class in turtle): Public classes. (line 6) +* RawTextHelpFormatter (class in argparse): formatter_class. (line 10) +* RawTurtle (class in turtle): Public classes. (line 6) +* RawValue() (in module multiprocessing.sharedctypes): The multiprocessing sharedctypes module. + (line 33) +* raw_data_manager (in module email.contentmanager): Content Manager Instances. + (line 11) +* raw_decode() (json.JSONDecoder method): Encoders and Decoders. + (line 93) +* raw_input (2to3 fixer): Fixers. (line 268) +* raw_input() (code.InteractiveConsole method): Interactive Console Objects. + (line 44) +* RBRACE (in module token): token — Constants used with Python parse trees. + (line 130) +* rcpttos (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 79) +* re (module): re — Regular expression operations. + (line 6) +* re (re.Match attribute): Match Objects. (line 191) +* read() (asyncio.StreamReader method): StreamReader. (line 15) +* read() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 105) +* read() (codecs.StreamReader method): StreamReader Objects. + (line 34) +* read() (configparser.ConfigParser method): ConfigParser Objects. + (line 113) +* read() (http.client.HTTPResponse method): HTTPResponse Objects. + (line 13) +* read() (imaplib.IMAP4 method): IMAP4 Objects. (line 192) +* read() (in module os): File Descriptor Operations. + (line 555) +* read() (io.BufferedIOBase method): I/O Base Classes. (line 270) +* read() (io.BufferedReader method): Buffered Streams. (line 80) +* read() (io.RawIOBase method): I/O Base Classes. (line 183) +* read() (io.TextIOBase method): Text I/O<2>. (line 52) +* read() (mimetypes.MimeTypes method): MimeTypes Objects. (line 70) +* read() (mmap.mmap method): mmap — Memory-mapped file support. + (line 225) +* read() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 32) +* read() (ssl.MemoryBIO method): Memory BIO Support<2>. + (line 148) +* read() (ssl.SSLSocket method): SSL Sockets. (line 71) +* read() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 25) +* read() (zipfile.ZipFile method): ZipFile Objects. (line 215) +* read1() (io.BufferedIOBase method): I/O Base Classes. (line 287) +* read1() (io.BufferedReader method): Buffered Streams. (line 86) +* read1() (io.BytesIO method): Buffered Streams. (line 43) +* READABLE (in module tkinter): File Handlers. (line 41) +* readable() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 164) +* readable() (io.IOBase method): I/O Base Classes. (line 79) +* readall() (io.RawIOBase method): I/O Base Classes. (line 198) +* reader() (in module csv): Module Contents<3>. (line 8) +* ReadError: tarfile — Read and write tar archive files. + (line 186) +* readexactly() (asyncio.StreamReader method): StreamReader. (line 34) +* readfp() (configparser.ConfigParser method): ConfigParser Objects. + (line 300) +* readfp() (mimetypes.MimeTypes method): MimeTypes Objects. (line 78) +* readframes() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 97) +* readframes() (sunau.AU_read method): AU_read Objects. (line 47) +* readframes() (wave.Wave_read method): Wave_read Objects. (line 46) +* readinto() (http.client.HTTPResponse method): HTTPResponse Objects. + (line 17) +* readinto() (io.BufferedIOBase method): I/O Base Classes. (line 298) +* readinto() (io.RawIOBase method): I/O Base Classes. (line 203) +* readinto1() (io.BufferedIOBase method): I/O Base Classes. (line 311) +* readinto1() (io.BytesIO method): Buffered Streams. (line 49) +* readline (module): readline — GNU readline interface. + (line 6) +* readline() (asyncio.StreamReader method): StreamReader. (line 23) +* readline() (codecs.StreamReader method): StreamReader Objects. + (line 60) +* readline() (distutils.text_file.TextFile method): distutils text_file — The TextFile class. + (line 111) +* readline() (imaplib.IMAP4 method): IMAP4 Objects. (line 197) +* readline() (io.IOBase method): I/O Base Classes. (line 84) +* readline() (io.TextIOBase method): Text I/O<2>. (line 58) +* readline() (mmap.mmap method): mmap — Memory-mapped file support. + (line 240) +* readlines() (codecs.StreamReader method): StreamReader Objects. + (line 71) +* readlines() (distutils.text_file.TextFile method): distutils text_file — The TextFile class. + (line 124) +* readlines() (io.IOBase method): I/O Base Classes. (line 93) +* readlink() (in module os): Files and Directories. + (line 581) +* readmodule() (in module pyclbr): pyclbr — Python module browser support. + (line 19) +* readmodule_ex() (in module pyclbr): pyclbr — Python module browser support. + (line 31) +* READONLY: Generic Attribute Management. + (line 92) +* readonly (memoryview attribute): Memory Views. (line 412) +* readPlist() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 126) +* readPlistFromBytes() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 149) +* ReadTransport (class in asyncio): Transports Hierarchy. + (line 20) +* readuntil() (asyncio.StreamReader method): StreamReader. (line 43) +* readv() (in module os): File Descriptor Operations. + (line 636) +* ready() (multiprocessing.pool.AsyncResult method): Process Pools. + (line 209) +* read_all() (telnetlib.Telnet method): Telnet Objects. (line 17) +* read_binary() (in module importlib.resources): importlib resources – Resources. + (line 80) +* read_byte() (mmap.mmap method): mmap — Memory-mapped file support. + (line 235) +* read_bytes() (pathlib.Path method): Methods<2>. (line 262) +* read_bytes() (zipfile.Path method): Path Objects. (line 58) +* read_dict() (configparser.ConfigParser method): ConfigParser Objects. + (line 172) +* read_eager() (telnetlib.Telnet method): Telnet Objects. (line 35) +* read_environ() (in module wsgiref.handlers): wsgiref handlers – server/gateway base classes. + (line 306) +* read_events() (xml.etree.ElementTree.XMLPullParser method): XMLPullParser Objects. + (line 29) +* read_file() (configparser.ConfigParser method): ConfigParser Objects. + (line 150) +* read_history_file() (in module readline): History file. (line 8) +* read_init_file() (in module readline): Init file. (line 13) +* read_lazy() (telnetlib.Telnet method): Telnet Objects. (line 43) +* read_mime_types() (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 99) +* READ_RESTRICTED: Generic Attribute Management. + (line 92) +* read_sb_data() (telnetlib.Telnet method): Telnet Objects. (line 59) +* read_some() (telnetlib.Telnet method): Telnet Objects. (line 21) +* read_string() (configparser.ConfigParser method): ConfigParser Objects. + (line 162) +* read_text() (in module importlib.resources): importlib resources – Resources. + (line 91) +* read_text() (pathlib.Path method): Methods<2>. (line 275) +* read_text() (zipfile.Path method): Path Objects. (line 52) +* read_token() (shlex.shlex method): shlex Objects. (line 20) +* read_until() (telnetlib.Telnet method): Telnet Objects. (line 8) +* read_very_eager() (telnetlib.Telnet method): Telnet Objects. + (line 27) +* read_very_lazy() (telnetlib.Telnet method): Telnet Objects. + (line 51) +* read_windows_registry() (mimetypes.MimeTypes method): MimeTypes Objects. + (line 86) +* Real (class in numbers): The numeric tower. (line 28) +* real (numbers.Complex attribute): The numeric tower. (line 15) +* Real Media File Format: chunk — Read IFF chunked data. + (line 8) +* realloc(): Overview<3>. (line 33) +* realpath() (in module os.path): os path — Common pathname manipulations. + (line 295) +* REALTIME_PRIORITY_CLASS (in module subprocess): Windows Constants. + (line 87) +* real_max_memuse (in module test.support): test support — Utilities for the Python test suite. + (line 127) +* real_quick_ratio() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 207) +* reap_children() (in module test.support): test support — Utilities for the Python test suite. + (line 792) +* reap_threads() (in module test.support): test support — Utilities for the Python test suite. + (line 655) +* reason (http.client.HTTPResponse attribute): HTTPResponse Objects. + (line 55) +* reason (ssl.SSLError attribute): Exceptions<12>. (line 27) +* reason (UnicodeError attribute): Concrete exceptions. + (line 346) +* reason (urllib.error.HTTPError attribute): urllib error — Exception classes raised by urllib request. + (line 45) +* reason (urllib.error.URLError attribute): urllib error — Exception classes raised by urllib request. + (line 22) +* reattach() (tkinter.ttk.Treeview method): ttk Treeview. (line 247) +* rebinding; name: Assignment statements. + (line 6) +* reccontrols() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 54) +* received_data (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 84) +* received_lines (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 57) +* recent() (imaplib.IMAP4 method): IMAP4 Objects. (line 202) +* reconfigure() (io.TextIOWrapper method): Text I/O<2>. (line 173) +* records (unittest.TestCase attribute): Test cases. (line 428) +* record_original_stdout() (in module test.support): test support — Utilities for the Python test suite. + (line 376) +* rect() (in module cmath): Conversions to and from polar coordinates. + (line 48) +* rectangle() (in module curses.textpad): curses textpad — Text input widget for curses programs. + (line 15) +* RecursionError: Concrete exceptions. + (line 187) +* recursive_repr() (in module reprlib): reprlib — Alternate repr implementation. + (line 41) +* recv() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 204) +* recv() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 24) +* recv() (socket.socket method): Socket Objects. (line 219) +* recvfrom() (socket.socket method): Socket Objects. (line 236) +* recvfrom_into() (socket.socket method): Socket Objects. (line 362) +* recvmsg() (socket.socket method): Socket Objects. (line 254) +* recvmsg_into() (socket.socket method): Socket Objects. (line 325) +* recv_bytes() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 65) +* recv_bytes_into() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 79) +* recv_into() (socket.socket method): Socket Objects. (line 372) +* redirect_request() (urllib.request.HTTPRedirectHandler method): HTTPRedirectHandler Objects. + (line 15) +* redirect_stderr() (in module contextlib): Utilities. (line 255) +* redirect_stdout() (in module contextlib): Utilities. (line 216) +* redisplay() (in module readline): Line buffer. (line 19) +* redrawln() (curses.window method): Window Objects. (line 459) +* redrawwin() (curses.window method): Window Objects. (line 465) +* reduce (2to3 fixer): Fixers. (line 272) +* reduce() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 291) +* reducer_override() (pickle.Pickler method): Module Interface. + (line 200) +* ref (class in weakref): weakref — Weak references. + (line 85) +* refcount_test() (in module test.support): test support — Utilities for the Python test suite. + (line 648) +* reference count: Glossary. (line 1127) +* reference counting: Objects values and types. + (line 37) +* ReferenceError: Concrete exceptions. + (line 196) +* ReferenceType (in module weakref): weakref — Weak references. + (line 290) +* refold_source (email.policy.EmailPolicy attribute): email policy Policy Objects. + (line 371) +* refresh() (curses.window method): Window Objects. (line 470) +* register() (abc.ABCMeta method): abc — Abstract Base Classes. + (line 69) +* register() (in module atexit): atexit — Exit handlers. + (line 22) +* register() (in module codecs): codecs — Codec registry and base classes. + (line 150) +* register() (in module faulthandler): Dumping the traceback on a user signal. + (line 6) +* register() (in module webbrowser): webbrowser — Convenient Web-browser controller. + (line 83) +* register() (multiprocessing.managers.BaseManager method): Managers. + (line 75) +* register() (select.devpoll method): /dev/poll Polling Objects. + (line 30) +* register() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 81) +* register() (select.poll method): Polling Objects. (line 15) +* register() (selectors.BaseSelector method): Classes<3>. (line 65) +* registerDOMImplementation() (in module xml.dom): Module Contents<4>. + (line 8) +* registerResult() (in module unittest): Signal Handling. (line 35) +* register_adapter() (in module sqlite3): Module functions and constants. + (line 140) +* register_archive_format() (in module shutil): Archiving operations. + (line 81) +* register_at_fork() (in module os): Process Management. (line 464) +* register_converter() (in module sqlite3): Module functions and constants. + (line 131) +* register_defect() (email.policy.Policy method): email policy Policy Objects. + (line 229) +* register_dialect() (in module csv): Module Contents<3>. (line 70) +* register_error() (in module codecs): Error Handlers. (line 88) +* register_function() (xmlrpc.server.CGIXMLRPCRequestHandler method): CGIXMLRPCRequestHandler. + (line 9) +* register_function() (xmlrpc.server.SimpleXMLRPCServer method): SimpleXMLRPCServer Objects. + (line 10) +* register_instance() (xmlrpc.server.CGIXMLRPCRequestHandler method): CGIXMLRPCRequestHandler. + (line 26) +* register_instance() (xmlrpc.server.SimpleXMLRPCServer method): SimpleXMLRPCServer Objects. + (line 27) +* register_introspection_functions() (xmlrpc.server.CGIXMLRPCRequestHandler method): CGIXMLRPCRequestHandler. + (line 41) +* register_introspection_functions() (xmlrpc.server.SimpleXMLRPCServer method): SimpleXMLRPCServer Objects. + (line 55) +* register_multicall_functions() (xmlrpc.server.CGIXMLRPCRequestHandler method): CGIXMLRPCRequestHandler. + (line 46) +* register_multicall_functions() (xmlrpc.server.SimpleXMLRPCServer method): SimpleXMLRPCServer Objects. + (line 60) +* register_namespace() (in module xml.etree.ElementTree): Functions<6>. + (line 182) +* register_optionflag() (in module doctest): Option Flags. (line 172) +* register_shape() (in module turtle): Settings and special methods. + (line 71) +* register_unpack_format() (in module shutil): Archiving operations. + (line 126) +* regular package: Glossary. (line 1135) +* REG_BINARY (in module winreg): Value Types. (line 8) +* REG_DWORD (in module winreg): Value Types. (line 12) +* REG_DWORD_BIG_ENDIAN (in module winreg): Value Types. (line 21) +* REG_DWORD_LITTLE_ENDIAN (in module winreg): Value Types. (line 16) +* REG_EXPAND_SZ (in module winreg): Value Types. (line 25) +* REG_FULL_RESOURCE_DESCRIPTOR (in module winreg): Value Types. + (line 60) +* REG_LINK (in module winreg): Value Types. (line 30) +* REG_MULTI_SZ (in module winreg): Value Types. (line 34) +* REG_NONE (in module winreg): Value Types. (line 39) +* REG_QWORD (in module winreg): Value Types. (line 43) +* REG_QWORD_LITTLE_ENDIAN (in module winreg): Value Types. (line 49) +* REG_RESOURCE_LIST (in module winreg): Value Types. (line 56) +* REG_RESOURCE_REQUIREMENTS_LIST (in module winreg): Value Types. + (line 64) +* REG_SZ (in module winreg): Value Types. (line 68) +* relative; import: The import statement. + (line 98) +* relative; URL: urllib parse — Parse URLs into components. + (line 8) +* relative_to() (pathlib.PurePath method): Methods and properties. + (line 242) +* release() (asyncio.Condition method): Condition. (line 79) +* release() (asyncio.Lock method): Lock. (line 50) +* release() (asyncio.Semaphore method): Semaphore. (line 55) +* release() (in module platform): Cross Platform. (line 115) +* release() (logging.Handler method): Handler Objects. (line 31) +* release() (memoryview method): Memory Views. (line 235) +* release() (multiprocessing.Lock method): Synchronization primitives. + (line 92) +* release() (multiprocessing.RLock method): Synchronization primitives. + (line 148) +* release() (pickle.PickleBuffer method): Module Interface. (line 325) +* release() (threading.Condition method): Condition Objects. (line 91) +* release() (threading.Lock method): Lock Objects. (line 73) +* release() (threading.RLock method): RLock Objects. (line 66) +* release() (threading.Semaphore method): Semaphore Objects. (line 63) +* release() (_thread.lock method): _thread — Low-level threading API. + (line 151) +* releasebufferproc (C type): Slot Type typedefs. (line 107) +* release_lock() (in module imp): imp — Access the import internals. + (line 295) +* reload (2to3 fixer): Fixers. (line 276) +* reload() (in module imp): imp — Access the import internals. + (line 132) +* reload() (in module importlib): Functions<10>. (line 73) +* relpath() (in module os.path): os path — Common pathname manipulations. + (line 310) +* remainder() (decimal.Context method): Context objects. (line 468) +* remainder() (in module math): Number-theoretic and representation functions. + (line 205) +* remainder_near() (decimal.Context method): Context objects. + (line 475) +* remainder_near() (decimal.Decimal method): Decimal objects. + (line 496) +* RemoteDisconnected: http client — HTTP protocol client. + (line 184) +* remove() (array.array method): array — Efficient arrays of numeric values. + (line 210) +* remove() (collections.deque method): deque objects. (line 97) +* remove() (frozenset method): Set Types — set frozenset. + (line 199) +* remove() (in module os): Files and Directories. + (line 619) +* remove() (mailbox.Mailbox method): Mailbox objects. (line 76) +* remove() (mailbox.MH method): MH. (line 76) +* remove() (sequence method): Mutable Sequence Types. + (line 16) +* remove() (xml.etree.ElementTree.Element method): Element Objects. + (line 180) +* removeAttribute() (xml.dom.Element method): Element Objects<2>. + (line 52) +* removeAttributeNode() (xml.dom.Element method): Element Objects<2>. + (line 57) +* removeAttributeNS() (xml.dom.Element method): Element Objects<2>. + (line 62) +* removeChild() (xml.dom.Node method): Node Objects. (line 128) +* removedirs() (in module os): Files and Directories. + (line 642) +* removeFilter() (logging.Handler method): Handler Objects. (line 56) +* removeFilter() (logging.Logger method): Logger Objects. (line 255) +* removeHandler() (in module unittest): Signal Handling. (line 52) +* removeHandler() (logging.Logger method): Logger Objects. (line 272) +* removeResult() (in module unittest): Signal Handling. (line 46) +* removexattr() (in module os): Linux extended attributes. + (line 41) +* remove_child_handler() (asyncio.AbstractChildWatcher method): Process Watchers. + (line 52) +* remove_done_callback() (asyncio.Future method): Future Object. + (line 102) +* remove_done_callback() (asyncio.Task method): Task Object. (line 158) +* remove_flag() (mailbox.MaildirMessage method): MaildirMessage. + (line 84) +* remove_flag() (mailbox.mboxMessage method): mboxMessage. (line 85) +* remove_flag() (mailbox.MMDFMessage method): MMDFMessage. (line 84) +* remove_folder() (mailbox.Maildir method): Maildir. (line 76) +* remove_folder() (mailbox.MH method): MH. (line 47) +* remove_header() (urllib.request.Request method): Request Objects. + (line 99) +* remove_history_item() (in module readline): History list. (line 26) +* remove_label() (mailbox.BabylMessage method): BabylMessage. + (line 59) +* remove_option() (configparser.ConfigParser method): ConfigParser Objects. + (line 266) +* remove_option() (optparse.OptionParser method): Querying and manipulating your option parser. + (line 47) +* remove_pyc() (msilib.Directory method): Directory Objects. (line 45) +* remove_reader() (asyncio.loop method): Watching file descriptors. + (line 12) +* remove_section() (configparser.ConfigParser method): ConfigParser Objects. + (line 273) +* remove_sequence() (mailbox.MHMessage method): MHMessage. (line 46) +* remove_signal_handler() (asyncio.loop method): Unix signals. + (line 26) +* remove_tree() (in module distutils.dir_util): distutils dir_util — Directory tree operations. + (line 60) +* remove_writer() (asyncio.loop method): Watching file descriptors. + (line 25) +* rename() (ftplib.FTP method): FTP Objects. (line 202) +* rename() (imaplib.IMAP4 method): IMAP4 Objects. (line 207) +* rename() (in module os): Files and Directories. + (line 659) +* rename() (pathlib.Path method): Methods<2>. (line 290) +* renames (2to3 fixer): Fixers. (line 280) +* renames() (in module os): Files and Directories. + (line 693) +* reopenIfNeeded() (logging.handlers.WatchedFileHandler method): WatchedFileHandler. + (line 37) +* reorganize() (dbm.gnu.gdbm method): dbm gnu — GNU’s reinterpretation of dbm. + (line 104) +* repeat() (in module itertools): Itertool functions. (line 493) +* repeat() (in module timeit): Python Interface. (line 18) +* repeat() (timeit.Timer method): Python Interface. (line 108) +* repetition; operation: Common Sequence Operations. + (line 21) +* replace() (bytearray method): Bytes and Bytearray Operations. + (line 145) +* replace() (bytes method): Bytes and Bytearray Operations. + (line 145) +* replace() (curses.panel.Panel method): Panel Objects. (line 39) +* replace() (datetime.date method): date Objects. (line 166) +* replace() (datetime.datetime method): datetime Objects. (line 413) +* replace() (datetime.time method): time Objects. (line 139) +* replace() (in module dataclasses): Module-level decorators classes and functions. + (line 345) +* replace() (in module os): Files and Directories. + (line 711) +* replace() (inspect.Parameter method): Introspecting callables with the Signature object. + (line 253) +* replace() (inspect.Signature method): Introspecting callables with the Signature object. + (line 111) +* replace() (pathlib.Path method): Methods<2>. (line 314) +* replace() (str method): String Methods<2>. (line 339) +* replace() (types.CodeType method): Standard Interpreter Types. + (line 63) +* replaceChild() (xml.dom.Node method): Node Objects. (line 135) +* ReplacePackage() (in module modulefinder): modulefinder — Find modules used by a script. + (line 21) +* replace_errors() (in module codecs): Error Handlers. (line 133) +* replace_header() (email.message.EmailMessage method): email message Representing an email message. + (line 301) +* replace_header() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 409) +* replace_history_item() (in module readline): History list. (line 32) +* replace_whitespace (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 171) +* report() (filecmp.dircmp method): The dircmp class. (line 18) +* report() (modulefinder.ModuleFinder method): modulefinder — Find modules used by a script. + (line 38) +* REPORTING_FLAGS (in module doctest): Option Flags. (line 164) +* REPORT_CDIFF (in module doctest): Option Flags. (line 126) +* report_failure() (doctest.DocTestRunner method): DocTestRunner objects. + (line 70) +* report_full_closure() (filecmp.dircmp method): The dircmp class. + (line 28) +* REPORT_NDIFF (in module doctest): Option Flags. (line 131) +* REPORT_ONLY_FIRST_FAILURE (in module doctest): Option Flags. + (line 140) +* report_partial_closure() (filecmp.dircmp method): The dircmp class. + (line 23) +* report_start() (doctest.DocTestRunner method): DocTestRunner objects. + (line 48) +* report_success() (doctest.DocTestRunner method): DocTestRunner objects. + (line 59) +* REPORT_UDIFF (in module doctest): Option Flags. (line 121) +* report_unexpected_exception() (doctest.DocTestRunner method): DocTestRunner objects. + (line 81) +* repr (2to3 fixer): Fixers. (line 284) +* Repr (class in reprlib): reprlib — Alternate repr implementation. + (line 17) +* repr() (built-in function): Built-in Functions. (line 1418) +* repr() (built-in function); __repr__() (object method): Basic customization. + (line 113) +* repr() (in module reprlib): reprlib — Alternate repr implementation. + (line 31) +* repr() (reprlib.Repr method): Repr Objects. (line 47) +* repr1() (reprlib.Repr method): Repr Objects. (line 52) +* reprfunc (C type): Slot Type typedefs. (line 49) +* reprlib (module): reprlib — Alternate repr implementation. + (line 6) +* Request (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 182) +* request() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 8) +* RequestHandlerClass (socketserver.BaseServer attribute): Server Objects<2>. + (line 71) +* requestline (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 81) +* request_queue_size (socketserver.BaseServer attribute): Server Objects<2>. + (line 98) +* request_rate() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 60) +* request_uri() (in module wsgiref.util): wsgiref util – WSGI environment utilities. + (line 25) +* request_version (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 96) +* requires() (in module test.support): test support — Utilities for the Python test suite. + (line 211) +* requires_bz2() (in module test.support): test support — Utilities for the Python test suite. + (line 617) +* requires_docstrings() (in module test.support): test support — Utilities for the Python test suite. + (line 629) +* requires_freebsd_version() (in module test.support): test support — Utilities for the Python test suite. + (line 587) +* requires_gzip() (in module test.support): test support — Utilities for the Python test suite. + (line 613) +* requires_IEEE_754() (in module test.support): test support — Utilities for the Python test suite. + (line 605) +* requires_linux_version() (in module test.support): test support — Utilities for the Python test suite. + (line 593) +* requires_lzma() (in module test.support): test support — Utilities for the Python test suite. + (line 621) +* requires_mac_version() (in module test.support): test support — Utilities for the Python test suite. + (line 599) +* requires_resource() (in module test.support): test support — Utilities for the Python test suite. + (line 625) +* requires_zlib() (in module test.support): test support — Utilities for the Python test suite. + (line 609) +* reserved (zipfile.ZipInfo attribute): ZipInfo Objects. (line 111) +* reserved word: Keywords. (line 6) +* RESERVED_FUTURE (in module uuid): uuid — UUID objects according to RFC 4122. + (line 236) +* RESERVED_MICROSOFT (in module uuid): uuid — UUID objects according to RFC 4122. + (line 232) +* RESERVED_NCS (in module uuid): uuid — UUID objects according to RFC 4122. + (line 224) +* reset() (bdb.Bdb method): bdb — Debugger framework. + (line 106) +* reset() (codecs.IncrementalDecoder method): IncrementalDecoder Objects. + (line 40) +* reset() (codecs.IncrementalEncoder method): IncrementalEncoder Objects. + (line 35) +* reset() (codecs.StreamReader method): StreamReader Objects. + (line 83) +* reset() (codecs.StreamWriter method): StreamWriter Objects. + (line 41) +* reset() (contextvars.ContextVar method): Context Variables. + (line 58) +* reset() (html.parser.HTMLParser method): HTMLParser Methods. + (line 23) +* reset() (in module turtle): Window control. (line 51) +* reset() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 172) +* reset() (pipes.Template method): Template Objects. (line 8) +* reset() (threading.Barrier method): Barrier Objects. (line 71) +* reset() (xdrlib.Packer method): Packer Objects. (line 12) +* reset() (xdrlib.Unpacker method): Unpacker Objects. (line 8) +* reset() (xml.dom.pulldom.DOMEventStream method): DOMEventStream Objects. + (line 37) +* reset() (xml.sax.xmlreader.IncrementalParser method): IncrementalParser Objects. + (line 19) +* resetbuffer() (code.InteractiveConsole method): Interactive Console Objects. + (line 40) +* resetlocale() (in module locale): locale — Internationalization services. + (line 367) +* resetscreen() (in module turtle): Window control. (line 51) +* resetty() (in module curses): Functions<3>. (line 405) +* resetwarnings() (in module warnings): Available Functions. + (line 99) +* reset_mock() (unittest.mock.AsyncMock method): The Mock Class. + (line 860) +* reset_mock() (unittest.mock.Mock method): The Mock Class. (line 198) +* reset_prog_mode() (in module curses): Functions<3>. (line 395) +* reset_shell_mode() (in module curses): Functions<3>. (line 400) +* resize() (curses.window method): Window Objects. (line 489) +* resize() (in module ctypes): Utility functions. (line 163) +* resize() (mmap.mmap method): mmap — Memory-mapped file support. + (line 245) +* resizemode() (in module turtle): Appearance. (line 24) +* resizeterm() (in module curses): Functions<3>. (line 420) +* resize_term() (in module curses): Functions<3>. (line 410) +* resolution (datetime.date attribute): date Objects. (line 95) +* resolution (datetime.datetime attribute): datetime Objects. + (line 257) +* resolution (datetime.time attribute): time Objects. (line 42) +* resolution (datetime.timedelta attribute): timedelta Objects. + (line 84) +* resolve() (pathlib.Path method): Methods<2>. (line 327) +* resolveEntity() (xml.sax.handler.EntityResolver method): EntityResolver Objects. + (line 6) +* resolve_bases() (in module types): Dynamic Type Creation. + (line 58) +* resolve_name() (in module importlib.util): importlib util – Utility code for importers. + (line 80) +* Resource (in module importlib.resources): importlib resources – Resources. + (line 45) +* resource (module): resource — Resource usage information. + (line 6) +* ResourceDenied: test support — Utilities for the Python test suite. + (line 22) +* ResourceLoader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 358) +* ResourceReader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 287) +* ResourceWarning: Warnings. (line 59) +* resource_path() (importlib.abc.ResourceReader method): importlib abc – Abstract base classes related to import. + (line 327) +* response (nntplib.NNTPError attribute): nntplib — NNTP protocol client. + (line 128) +* response() (imaplib.IMAP4 method): IMAP4 Objects. (line 211) +* ResponseNotReady: http client — HTTP protocol client. + (line 170) +* responses (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 169) +* responses (in module http.client): http client — HTTP protocol client. + (line 205) +* restart (pdb command): Debugger Commands. (line 322) +* restore() (in module difflib): difflib — Helpers for computing deltas. + (line 273) +* RESTRICTED: Generic Attribute Management. + (line 92) +* restricted; execution: Builtins and restricted execution. + (line 6) +* restype (ctypes._FuncPtr attribute): Foreign functions. (line 22) +* result() (asyncio.Future method): Future Object. (line 28) +* result() (asyncio.Task method): Task Object. (line 120) +* result() (concurrent.futures.Future method): Future Objects. + (line 38) +* results() (trace.Trace method): Programmatic Interface. + (line 40) +* resume_reading() (asyncio.ReadTransport method): Read-only Transports. + (line 21) +* resume_writing() (asyncio.BaseProtocol method): Base Protocol. + (line 44) +* retr() (poplib.POP3 method): POP3 Objects. (line 62) +* retrbinary() (ftplib.FTP method): FTP Objects. (line 84) +* retrieve() (urllib.request.URLopener method): Legacy interface. + (line 113) +* retrlines() (ftplib.FTP method): FTP Objects. (line 96) +* return (pdb command): Debugger Commands. (line 188) +* returncode (asyncio.asyncio.subprocess.Process attribute): Interacting with Subprocesses. + (line 140) +* returncode (subprocess.CalledProcessError attribute): Using the subprocess Module. + (line 196) +* returncode (subprocess.CompletedProcess attribute): Using the subprocess Module. + (line 101) +* returncode (subprocess.Popen attribute): Popen Objects. (line 145) +* return_annotation (inspect.Signature attribute): Introspecting callables with the Signature object. + (line 90) +* return_ok() (http.cookiejar.CookiePolicy method): CookiePolicy Objects. + (line 18) +* RETURN_VALUE (opcode): Python Bytecode Instructions. + (line 342) +* return_value (unittest.mock.Mock attribute): The Mock Class. + (line 321) +* retval (pdb command): Debugger Commands. (line 341) +* reverse() (array.array method): array — Efficient arrays of numeric values. + (line 214) +* reverse() (collections.deque method): deque objects. (line 102) +* reverse() (in module audioop): audioop — Manipulate raw audio data. + (line 188) +* reverse() (sequence method): Mutable Sequence Types. + (line 16) +* reversed() (built-in function): Built-in Functions. (line 1430) +* reverse_order() (pstats.Stats method): The Stats Class. (line 157) +* reverse_pointer (ipaddress.IPv4Address attribute): Address objects. + (line 71) +* reverse_pointer (ipaddress.IPv6Address attribute): Address objects. + (line 165) +* Reversible (class in collections.abc): Collections Abstract Base Classes. + (line 143) +* Reversible (class in typing): Classes functions and decorators. + (line 155) +* revert() (http.cookiejar.FileCookieJar method): CookieJar and FileCookieJar Objects. + (line 141) +* rewind() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 103) +* rewind() (sunau.AU_read method): AU_read Objects. (line 53) +* rewind() (wave.Wave_read method): Wave_read Objects. (line 51) +* rfc2109 (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 68) +* rfc2109_as_netscape (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 83) +* rfc2965 (http.cookiejar.CookiePolicy attribute): CookiePolicy Objects. + (line 72) +* rfc822_escape() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 172) +* RFC; RFC 1014: xdrlib — Encode and decode XDR data. + (line 11) +* RFC; RFC 1014 <1>: xdrlib — Encode and decode XDR data. + (line 32) +* RFC; RFC 1123: Functions<2>. (line 615) +* RFC; RFC 1321: hashlib — Secure hashes and message digests. + (line 13) +* RFC; RFC 1422: Certificates. (line 38) +* RFC; RFC 1422 <1>: LibreSSL support. (line 29) +* RFC; RFC 1521: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 270) +* RFC; RFC 1521 <1>: quopri — Encode and decode MIME quoted-printable data. + (line 11) +* RFC; RFC 1521 <2>: quopri — Encode and decode MIME quoted-printable data. + (line 37) +* RFC; RFC 1522: binascii — Convert between binary and ASCII. + (line 71) +* RFC; RFC 1522 <1>: quopri — Encode and decode MIME quoted-printable data. + (line 25) +* RFC; RFC 1522 <2>: quopri — Encode and decode MIME quoted-printable data. + (line 39) +* RFC; RFC 1524: mailcap — Mailcap file handling. + (line 19) +* RFC; RFC 1524 <1>: mailcap — Mailcap file handling. + (line 38) +* RFC; RFC 1730: imaplib — IMAP4 protocol client. + (line 14) +* RFC; RFC 1738: URL Quoting. (line 176) +* RFC; RFC 1750: Random generation. (line 70) +* RFC; RFC 1766: locale — Internationalization services. + (line 322) +* RFC; RFC 1766 <1>: locale — Internationalization services. + (line 333) +* RFC; RFC 1808: urllib. (line 23) +* RFC; RFC 1808 <1>: URL Parsing. (line 33) +* RFC; RFC 1808 <2>: URL Quoting. (line 170) +* RFC; RFC 1832: xdrlib — Encode and decode XDR data. + (line 36) +* RFC; RFC 1832 <1>: xdrlib — Encode and decode XDR data. + (line 38) +* RFC; RFC 1869: smtplib — SMTP protocol client. + (line 13) +* RFC; RFC 1869 <1>: smtplib — SMTP protocol client. + (line 184) +* RFC; RFC 1870: smtpd<3>. (line 7) +* RFC; RFC 1870 <1>: smtpd — SMTP Server. + (line 26) +* RFC; RFC 1870 <2>: SMTPChannel Objects. + (line 125) +* RFC; RFC 1939: poplib — POP3 protocol client. + (line 11) +* RFC; RFC 1939 <1>: poplib — POP3 protocol client. + (line 13) +* RFC; RFC 2033: New and Improved Modules<2>. + (line 569) +* RFC; RFC 2045: email — An email and MIME handling package. + (line 16) +* RFC; RFC 2045 <1>: email message Representing an email message. + (line 316) +* RFC; RFC 2045 <2>: email message Representing an email message. + (line 318) +* RFC; RFC 2045 <3>: email message Representing an email message. + (line 321) +* RFC; RFC 2045 <4>: email headerregistry Custom Header Objects. + (line 239) +* RFC; RFC 2045 <5>: email headerregistry Custom Header Objects. + (line 297) +* RFC; RFC 2045 <6>: email message Message Representing an email message using the compat32 API. + (line 422) +* RFC; RFC 2045 <7>: email message Message Representing an email message using the compat32 API. + (line 426) +* RFC; RFC 2045 <8>: email message Message Representing an email message using the compat32 API. + (line 429) +* RFC; RFC 2045 <9>: email message Message Representing an email message using the compat32 API. + (line 526) +* RFC; RFC 2045 <10>: email header Internationalized headers. + (line 31) +* RFC; RFC 2045 <11>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 30) +* RFC; RFC 2045 <12>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 31) +* RFC; RFC 2045 <13>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 234) +* RFC; RFC 2045 <14>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 242) +* RFC; RFC 2045#section-6.8: Binary Objects. (line 31) +* RFC; RFC 2046: email — An email and MIME handling package. + (line 16) +* RFC; RFC 2046 <1>: Content Manager Instances. + (line 85) +* RFC; RFC 2046 <2>: email header Internationalized headers. + (line 32) +* RFC; RFC 2047: PEP 3333 Python Web Server Gateway Interface v1 0 1. + (line 21) +* RFC; RFC 2047 <1>: email<5>. (line 27) +* RFC; RFC 2047 <2>: email — An email and MIME handling package. + (line 16) +* RFC; RFC 2047 <3>: email policy Policy Objects. + (line 347) +* RFC; RFC 2047 <4>: email policy Policy Objects. + (line 353) +* RFC; RFC 2047 <5>: email headerregistry Custom Header Objects. + (line 78) +* RFC; RFC 2047 <6>: email headerregistry Custom Header Objects. + (line 123) +* RFC; RFC 2047 <7>: email headerregistry Custom Header Objects. + (line 128) +* RFC; RFC 2047 <8>: email header Internationalized headers. + (line 32) +* RFC; RFC 2047 <9>: email header Internationalized headers. + (line 55) +* RFC; RFC 2047 <10>: email header Internationalized headers. + (line 117) +* RFC; RFC 2047 <11>: email header Internationalized headers. + (line 141) +* RFC; RFC 2047 <12>: email utils Miscellaneous utilities. + (line 74) +* RFC; RFC 2060: imaplib — IMAP4 protocol client. + (line 13) +* RFC; RFC 2060 <1>: IMAP4 Objects. (line 310) +* RFC; RFC 2068: http cookies — HTTP state management. + (line 16) +* RFC; RFC 2104: New and Improved Modules<3>. + (line 35) +* RFC; RFC 2104 <1>: hmac — Keyed-Hashing for Message Authentication. + (line 10) +* RFC; RFC 2109: http cookies — HTTP state management. + (line 15) +* RFC; RFC 2109 <1>: http cookies — HTTP state management. + (line 35) +* RFC; RFC 2109 <2>: http cookies — HTTP state management. + (line 65) +* RFC; RFC 2109 <3>: Morsel Objects. (line 8) +* RFC; RFC 2109 <4>: Morsel Objects. (line 11) +* RFC; RFC 2109 <5>: Morsel Objects. (line 98) +* RFC; RFC 2109 <6>: Morsel Objects. (line 110) +* RFC; RFC 2109 <7>: http cookiejar — Cookie handling for HTTP clients. + (line 17) +* RFC; RFC 2109 <8>: http cookiejar — Cookie handling for HTTP clients. + (line 99) +* RFC; RFC 2109 <9>: http cookiejar — Cookie handling for HTTP clients. + (line 111) +* RFC; RFC 2109 <10>: http cookiejar — Cookie handling for HTTP clients. + (line 138) +* RFC; RFC 2109 <11>: DefaultCookiePolicy Objects. + (line 86) +* RFC; RFC 2109 <12>: Cookie Objects<2>. (line 11) +* RFC; RFC 2109 <13>: Cookie Objects<2>. (line 23) +* RFC; RFC 2109 <14>: Cookie Objects<2>. (line 70) +* RFC; RFC 2183: email — An email and MIME handling package. + (line 16) +* RFC; RFC 2183 <1>: email message Representing an email message. + (line 460) +* RFC; RFC 2183 <2>: email message Message Representing an email message using the compat32 API. + (line 632) +* RFC; RFC 2231: email — An email and MIME handling package. + (line 16) +* RFC; RFC 2231 <1>: email message Representing an email message. + (line 280) +* RFC; RFC 2231 <2>: email message Representing an email message. + (line 283) +* RFC; RFC 2231 <3>: email message Representing an email message. + (line 365) +* RFC; RFC 2231 <4>: email message Message Representing an email message using the compat32 API. + (line 385) +* RFC; RFC 2231 <5>: email message Message Representing an email message using the compat32 API. + (line 389) +* RFC; RFC 2231 <6>: email message Message Representing an email message using the compat32 API. + (line 491) +* RFC; RFC 2231 <7>: email message Message Representing an email message using the compat32 API. + (line 499) +* RFC; RFC 2231 <8>: email message Message Representing an email message using the compat32 API. + (line 533) +* RFC; RFC 2231 <9>: email header Internationalized headers. + (line 32) +* RFC; RFC 2231 <10>: email utils Miscellaneous utilities. + (line 176) +* RFC; RFC 2231 <11>: email utils Miscellaneous utilities. + (line 181) +* RFC; RFC 2231 <12>: email utils Miscellaneous utilities. + (line 190) +* RFC; RFC 2231 <13>: email utils Miscellaneous utilities. + (line 197) +* RFC; RFC 2231 <14>: email utils Miscellaneous utilities. + (line 206) +* RFC; RFC 2295: HTTP status codes. (line 175) +* RFC; RFC 2342: New and Improved Modules<3>. + (line 93) +* RFC; RFC 2342 <1>: IMAP4 Objects. (line 164) +* RFC; RFC 2368: URL Quoting. (line 166) +* RFC; RFC 2373: Address objects. (line 89) +* RFC; RFC 2373 <1>: Address objects. (line 108) +* RFC; RFC 2373 <2>: Address objects. (line 117) +* RFC; RFC 2396: urllib parse. (line 6) +* RFC; RFC 2396 <1>: urllib. (line 24) +* RFC; RFC 2396 <2>: URL Parsing. (line 245) +* RFC; RFC 2396 <3>: URL Quoting. (line 24) +* RFC; RFC 2396 <4>: URL Quoting. (line 161) +* RFC; RFC 2397: DataHandler Objects. + (line 9) +* RFC; RFC 2449: POP3 Objects. (line 26) +* RFC; RFC 2487: New and Improved Modules<3>. + (line 85) +* RFC; RFC 2518: HTTP status codes. (line 19) +* RFC; RFC 2595: poplib — POP3 protocol client. + (line 14) +* RFC; RFC 2595 <1>: POP3 Objects. (line 115) +* RFC; RFC 2616: PEP 3333 Python Web Server Gateway Interface v1 0 1. + (line 20) +* RFC; RFC 2616 <1>: wsgiref util – WSGI environment utilities. + (line 112) +* RFC; RFC 2616 <2>: wsgiref validate — WSGI conformance checker. + (line 27) +* RFC; RFC 2616 <3>: HTTPRedirectHandler Objects. + (line 8) +* RFC; RFC 2616 <4>: HTTPRedirectHandler Objects. + (line 28) +* RFC; RFC 2616 <5>: Legacy interface. (line 164) +* RFC; RFC 2616 <6>: urllib error — Exception classes raised by urllib request. + (line 40) +* RFC; RFC 2616 <7>: Introduction<16>. (line 34) +* RFC; RFC 2616 <8>: HTTPError. (line 15) +* RFC; RFC 2616 <9>: Error Codes. (line 12) +* RFC; RFC 2732: urllib parse<2>. (line 10) +* RFC; RFC 2732 <1>: New and Improved Modules. + (line 638) +* RFC; RFC 2732 <2>: URL Quoting. (line 157) +* RFC; RFC 2774: HTTP status codes. (line 184) +* RFC; RFC 2818: ssl<9>. (line 16) +* RFC; RFC 2818 <1>: Certificate handling. + (line 11) +* RFC; RFC 2821: email — An email and MIME handling package. + (line 12) +* RFC; RFC 2822: New and Improved Modules<3>. + (line 97) +* RFC; RFC 2822 <1>: Batteries Included. (line 16) +* RFC; RFC 2822 <2>: Functions<2>. (line 346) +* RFC; RFC 2822 <3>: Functions<2>. (line 616) +* RFC; RFC 2822 <4>: email message Message Representing an email message using the compat32 API. + (line 274) +* RFC; RFC 2822 <5>: email header Internationalized headers. + (line 20) +* RFC; RFC 2822 <6>: email header Internationalized headers. + (line 23) +* RFC; RFC 2822 <7>: email header Internationalized headers. + (line 31) +* RFC; RFC 2822 <8>: email header Internationalized headers. + (line 88) +* RFC; RFC 2822 <9>: email header Internationalized headers. + (line 116) +* RFC; RFC 2822 <10>: email header Internationalized headers. + (line 134) +* RFC; RFC 2822 <11>: email utils Miscellaneous utilities. + (line 31) +* RFC; RFC 2822 <12>: email utils Miscellaneous utilities. + (line 98) +* RFC; RFC 2822 <13>: email utils Miscellaneous utilities. + (line 101) +* RFC; RFC 2822 <14>: email utils Miscellaneous utilities. + (line 224) +* RFC; RFC 2822 <15>: email utils Miscellaneous utilities. + (line 142) +* RFC; RFC 2822 <16>: Message objects. (line 17) +* RFC; RFC 2822 <17>: http client — HTTP protocol client. + (line 112) +* RFC; RFC 2822 <18>: http server — HTTP servers. + (line 108) +* RFC; RFC 2964: http cookiejar — Cookie handling for HTTP clients. + (line 151) +* RFC; RFC 2965: urllib request — Extensible library for opening URLs. + (line 225) +* RFC; RFC 2965 <1>: urllib request — Extensible library for opening URLs. + (line 233) +* RFC; RFC 2965 <2>: Request Objects. (line 50) +* RFC; RFC 2965 <3>: http cookiejar — Cookie handling for HTTP clients. + (line 16) +* RFC; RFC 2965 <4>: http cookiejar — Cookie handling for HTTP clients. + (line 99) +* RFC; RFC 2965 <5>: http cookiejar — Cookie handling for HTTP clients. + (line 111) +* RFC; RFC 2965 <6>: http cookiejar — Cookie handling for HTTP clients. + (line 140) +* RFC; RFC 2965 <7>: http cookiejar — Cookie handling for HTTP clients. + (line 142) +* RFC; RFC 2965 <8>: http cookiejar — Cookie handling for HTTP clients. + (line 149) +* RFC; RFC 2965 <9>: FileCookieJar subclasses and co-operation with web browsers. + (line 16) +* RFC; RFC 2965 <10>: CookiePolicy Objects. + (line 74) +* RFC; RFC 2965 <11>: CookiePolicy Objects. + (line 79) +* RFC; RFC 2965 <12>: DefaultCookiePolicy Objects. + (line 8) +* RFC; RFC 2965 <13>: DefaultCookiePolicy Objects. + (line 90) +* RFC; RFC 2965 <14>: DefaultCookiePolicy Objects. + (line 101) +* RFC; RFC 2965 <15>: DefaultCookiePolicy Objects. + (line 105) +* RFC; RFC 2965 <16>: DefaultCookiePolicy Objects. + (line 114) +* RFC; RFC 2965 <17>: DefaultCookiePolicy Objects. + (line 151) +* RFC; RFC 2965 <18>: Cookie Objects<2>. (line 23) +* RFC; RFC 2965 <19>: Examples<24>. (line 25) +* RFC; RFC 2980: nntplib — NNTP protocol client. + (line 13) +* RFC; RFC 2980 <1>: Methods<3>. (line 317) +* RFC; RFC 3056: Address objects. (line 205) +* RFC; RFC 3171: Address objects. (line 88) +* RFC; RFC 3207: New and Improved Modules<2>. + (line 574) +* RFC; RFC 3229: HTTP status codes. (line 49) +* RFC; RFC 3280: SSL Sockets. (line 152) +* RFC; RFC 3330: Address objects. (line 116) +* RFC; RFC 3339: Displaying the date/time in messages. + (line 18) +* RFC; RFC 3454: stringprep — Internet String Preparation. + (line 17) +* RFC; RFC 3454 <1>: stringprep — Internet String Preparation. + (line 26) +* RFC; RFC 3490: Text Encodings. (line 14) +* RFC; RFC 3490 <1>: encodings idna — Internationalized Domain Names in Applications. + (line 6) +* RFC; RFC 3490 <2>: encodings idna — Internationalized Domain Names in Applications. + (line 54) +* RFC; RFC 3490 <3>: encodings idna — Internationalized Domain Names in Applications. + (line 59) +* RFC; RFC 3490#section-3.1: encodings idna — Internationalized Domain Names in Applications. + (line 24) +* RFC; RFC 3492: Text Encodings. (line 36) +* RFC; RFC 3492 <1>: encodings idna — Internationalized Domain Names in Applications. + (line 7) +* RFC; RFC 3493: Example<8>. (line 227) +* RFC; RFC 3501: IMAP4 Objects. (line 321) +* RFC; RFC 3542: Other functions<2>. (line 302) +* RFC; RFC 3548: New Improved and Deprecated Modules<3>. + (line 16) +* RFC; RFC 3548 <1>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 12) +* RFC; RFC 3548 <2>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 16) +* RFC; RFC 3548 <3>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 24) +* RFC; RFC 3548 <4>: base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 112) +* RFC; RFC 3548 <5>: binascii — Convert between binary and ASCII. + (line 50) +* RFC; RFC 3659: FTP Objects. (line 168) +* RFC; RFC 3879: Address objects. (line 190) +* RFC; RFC 3927: Address objects. (line 121) +* RFC; RFC 3977: nntplib — NNTP protocol client. + (line 13) +* RFC; RFC 3977 <1>: Attributes. (line 10) +* RFC; RFC 3977 <2>: Methods<3>. (line 36) +* RFC; RFC 3977 <3>: Methods<3>. (line 138) +* RFC; RFC 3977 <4>: Methods<3>. (line 318) +* RFC; RFC 3986: urllib parse. (line 6) +* RFC; RFC 3986 <1>: urllib. (line 22) +* RFC; RFC 3986 <2>: New and Improved Modules. + (line 617) +* RFC; RFC 3986 <3>: Porting to Python 2 7. + (line 80) +* RFC; RFC 3986 <4>: URL Parsing. (line 130) +* RFC; RFC 3986 <5>: URL Parsing. (line 342) +* RFC; RFC 3986 <6>: URL Quoting. (line 24) +* RFC; RFC 3986 <7>: URL Quoting. (line 149) +* RFC; RFC 4086: LibreSSL support. (line 33) +* RFC; RFC 4122: New Improved and Removed Modules. + (line 417) +* RFC; RFC 4122 <1>: New Improved and Removed Modules. + (line 422) +* RFC; RFC 4122 <2>: uuid — UUID objects according to RFC 4122. + (line 6) +* RFC; RFC 4122 <3>: uuid — UUID objects according to RFC 4122. + (line 13) +* RFC; RFC 4122 <4>: uuid — UUID objects according to RFC 4122. + (line 71) +* RFC; RFC 4122 <5>: uuid — UUID objects according to RFC 4122. + (line 137) +* RFC; RFC 4122 <6>: uuid — UUID objects according to RFC 4122. + (line 166) +* RFC; RFC 4122 <7>: uuid — UUID objects according to RFC 4122. + (line 230) +* RFC; RFC 4122 <8>: uuid — UUID objects according to RFC 4122. + (line 243) +* RFC; RFC 4180: csv — CSV File Reading and Writing. + (line 13) +* RFC; RFC 4193: Address objects. (line 192) +* RFC; RFC 4217: ftplib — FTP protocol client. + (line 77) +* RFC; RFC 4291: Address objects. (line 135) +* RFC; RFC 4380: Address objects. (line 212) +* RFC; RFC 4627: json — JSON encoder and decoder. + (line 11) +* RFC; RFC 4627 <1>: Top-level Non-Object Non-Array Values. + (line 6) +* RFC; RFC 4642: nntplib — NNTP protocol client. + (line 105) +* RFC; RFC 4918: HTTP status codes. (line 43) +* RFC; RFC 4918 <1>: HTTP status codes. (line 133) +* RFC; RFC 4918 <2>: HTTP status codes. (line 136) +* RFC; RFC 4918 <3>: HTTP status codes. (line 139) +* RFC; RFC 4918 <4>: HTTP status codes. (line 178) +* RFC; RFC 4954: SMTP Objects. (line 135) +* RFC; RFC 4954 <1>: SMTP Objects. (line 158) +* RFC; RFC 5161: imaplib. (line 11) +* RFC; RFC 5161 <1>: IMAP4 Objects. (line 86) +* RFC; RFC 5233: email — An email and MIME handling package. + (line 15) +* RFC; RFC 5233 <1>: email message Message Representing an email message using the compat32 API. + (line 21) +* RFC; RFC 5246: Constants<9>. (line 477) +* RFC; RFC 5246 <1>: LibreSSL support. (line 41) +* RFC; RFC 5280: Other Changes. (line 20) +* RFC; RFC 5280 <1>: Changes in the Python API<4>. + (line 95) +* RFC; RFC 5280 <2>: Certificate handling. + (line 11) +* RFC; RFC 5280 <3>: Certificate handling. + (line 64) +* RFC; RFC 5280 <4>: LibreSSL support. (line 37) +* RFC; RFC 5321: smtpd<3>. (line 6) +* RFC; RFC 5321 <1>: email headerregistry Custom Header Objects. + (line 453) +* RFC; RFC 5321 <2>: smtpd — SMTP Server. + (line 26) +* RFC; RFC 5321 <3>: SMTPServer Objects. (line 50) +* RFC; RFC 5321 <4>: SMTPServer Objects. (line 78) +* RFC; RFC 5322: email message Representing an email message. + (line 20) +* RFC; RFC 5322 <1>: email message Representing an email message. + (line 31) +* RFC; RFC 5322 <2>: Parser API. (line 41) +* RFC; RFC 5322 <3>: email generator Generating MIME documents. + (line 102) +* RFC; RFC 5322 <4>: email generator Generating MIME documents. + (line 195) +* RFC; RFC 5322 <5>: email policy Policy Objects. + (line 148) +* RFC; RFC 5322 <6>: email policy Policy Objects. + (line 326) +* RFC; RFC 5322 <7>: email policy Policy Objects. + (line 347) +* RFC; RFC 5322 <8>: email policy Policy Objects. + (line 353) +* RFC; RFC 5322 <9>: email policy Policy Objects. + (line 365) +* RFC; RFC 5322 <10>: email policy Policy Objects. + (line 398) +* RFC; RFC 5322 <11>: email errors Exception and Defect classes. + (line 28) +* RFC; RFC 5322 <12>: email headerregistry Custom Header Objects. + (line 16) +* RFC; RFC 5322 <13>: email headerregistry Custom Header Objects. + (line 117) +* RFC; RFC 5322 <14>: email headerregistry Custom Header Objects. + (line 122) +* RFC; RFC 5322 <15>: email headerregistry Custom Header Objects. + (line 123) +* RFC; RFC 5322 <16>: email headerregistry Custom Header Objects. + (line 138) +* RFC; RFC 5322 <17>: email headerregistry Custom Header Objects. + (line 159) +* RFC; RFC 5322 <18>: email headerregistry Custom Header Objects. + (line 416) +* RFC; RFC 5322 <19>: email headerregistry Custom Header Objects. + (line 450) +* RFC; RFC 5322 <20>: email headerregistry Custom Header Objects. + (line 484) +* RFC; RFC 5322 <21>: SMTP Objects. (line 317) +* RFC; RFC 5424: SysLogHandler. (line 53) +* RFC; RFC 5424 <1>: Inserting a BOM into messages sent to a SysLogHandler. + (line 6) +* RFC; RFC 5424 <2>: Inserting a BOM into messages sent to a SysLogHandler. + (line 19) +* RFC; RFC 5424 <3>: Inserting a BOM into messages sent to a SysLogHandler. + (line 43) +* RFC; RFC 5424#section-6: Inserting a BOM into messages sent to a SysLogHandler. + (line 9) +* RFC; RFC 5735: Address objects. (line 107) +* RFC; RFC 5842: HTTP status codes. (line 46) +* RFC; RFC 5842 <1>: HTTP status codes. (line 181) +* RFC; RFC 5929: SSL Sockets. (line 247) +* RFC; RFC 6066: Constants<9>. (line 377) +* RFC; RFC 6066 <1>: SSL Contexts. (line 293) +* RFC; RFC 6066 <2>: LibreSSL support. (line 45) +* RFC; RFC 6125: Certificate handling. + (line 11) +* RFC; RFC 6125 <1>: Certificate handling. + (line 29) +* RFC; RFC 6152: smtpd. (line 17) +* RFC; RFC 6152 <1>: SMTPServer Objects. (line 35) +* RFC; RFC 6531: email<2>. (line 19) +* RFC; RFC 6531 <1>: smtpd. (line 23) +* RFC; RFC 6531 <2>: smtplib. (line 15) +* RFC; RFC 6531 <3>: email message Representing an email message. + (line 103) +* RFC; RFC 6531 <4>: email policy Policy Objects. + (line 369) +* RFC; RFC 6531 <5>: smtplib — SMTP protocol client. + (line 63) +* RFC; RFC 6531 <6>: smtpd — SMTP Server. + (line 26) +* RFC; RFC 6531 <7>: SMTPServer Objects. (line 26) +* RFC; RFC 6531 <8>: SMTPChannel Objects. + (line 21) +* RFC; RFC 6532: email<2>. (line 18) +* RFC; RFC 6532 <1>: email — An email and MIME handling package. + (line 15) +* RFC; RFC 6532 <2>: email message Representing an email message. + (line 20) +* RFC; RFC 6532 <3>: Parser API. (line 42) +* RFC; RFC 6532 <4>: email policy Policy Objects. + (line 367) +* RFC; RFC 6585: HTTP status codes. (line 145) +* RFC; RFC 6585 <1>: HTTP status codes. (line 148) +* RFC; RFC 6585 <2>: HTTP status codes. (line 151) +* RFC; RFC 6585 <3>: HTTP status codes. (line 187) +* RFC; RFC 6855: imaplib. (line 12) +* RFC; RFC 6855 <1>: imaplib. (line 14) +* RFC; RFC 6855 <2>: IMAP4 Objects. (line 88) +* RFC; RFC 6855 <3>: IMAP4 Objects. (line 90) +* RFC; RFC 6856: poplib. (line 6) +* RFC; RFC 6856 <1>: POP3 Objects. (line 108) +* RFC; RFC 7159: json — JSON encoder and decoder. + (line 10) +* RFC; RFC 7159 <1>: Standard Compliance and Interoperability. + (line 6) +* RFC; RFC 7159 <2>: Top-level Non-Object Non-Array Values. + (line 9) +* RFC; RFC 7230: urllib request — Extensible library for opening URLs. + (line 197) +* RFC; RFC 7230 <1>: HTTPConnection Objects. + (line 151) +* RFC; RFC 7231: HTTP status codes. (line 13) +* RFC; RFC 7231 <1>: HTTP status codes. (line 16) +* RFC; RFC 7231 <2>: HTTP status codes. (line 22) +* RFC; RFC 7231 <3>: HTTP status codes. (line 25) +* RFC; RFC 7231 <4>: HTTP status codes. (line 28) +* RFC; RFC 7231 <5>: HTTP status codes. (line 31) +* RFC; RFC 7231 <6>: HTTP status codes. (line 34) +* RFC; RFC 7231 <7>: HTTP status codes. (line 37) +* RFC; RFC 7231 <8>: HTTP status codes. (line 52) +* RFC; RFC 7231 <9>: HTTP status codes. (line 55) +* RFC; RFC 7231 <10>: HTTP status codes. (line 58) +* RFC; RFC 7231 <11>: HTTP status codes. (line 61) +* RFC; RFC 7231 <12>: HTTP status codes. (line 67) +* RFC; RFC 7231 <13>: HTTP status codes. (line 70) +* RFC; RFC 7231 <14>: HTTP status codes. (line 76) +* RFC; RFC 7231 <15>: HTTP status codes. (line 82) +* RFC; RFC 7231 <16>: HTTP status codes. (line 85) +* RFC; RFC 7231 <17>: HTTP status codes. (line 88) +* RFC; RFC 7231 <18>: HTTP status codes. (line 91) +* RFC; RFC 7231 <19>: HTTP status codes. (line 94) +* RFC; RFC 7231 <20>: HTTP status codes. (line 100) +* RFC; RFC 7231 <21>: HTTP status codes. (line 103) +* RFC; RFC 7231 <22>: HTTP status codes. (line 106) +* RFC; RFC 7231 <23>: HTTP status codes. (line 109) +* RFC; RFC 7231 <24>: HTTP status codes. (line 115) +* RFC; RFC 7231 <25>: HTTP status codes. (line 118) +* RFC; RFC 7231 <26>: HTTP status codes. (line 121) +* RFC; RFC 7231 <27>: HTTP status codes. (line 127) +* RFC; RFC 7231 <28>: HTTP status codes. (line 142) +* RFC; RFC 7231 <29>: HTTP status codes. (line 157) +* RFC; RFC 7231 <30>: HTTP status codes. (line 160) +* RFC; RFC 7231 <31>: HTTP status codes. (line 163) +* RFC; RFC 7231 <32>: HTTP status codes. (line 166) +* RFC; RFC 7231 <33>: HTTP status codes. (line 169) +* RFC; RFC 7231 <34>: HTTP status codes. (line 172) +* RFC; RFC 7232: HTTP status codes. (line 64) +* RFC; RFC 7232 <1>: HTTP status codes. (line 112) +* RFC; RFC 7233: HTTP status codes. (line 40) +* RFC; RFC 7233 <1>: HTTP status codes. (line 124) +* RFC; RFC 7235: HTTP status codes. (line 79) +* RFC; RFC 7235 <1>: HTTP status codes. (line 97) +* RFC; RFC 7238: HTTP status codes. (line 73) +* RFC; RFC 7301: Application-Layer Protocol Negotiation Support. + (line 10) +* RFC; RFC 7301 <1>: Constants<9>. (line 354) +* RFC; RFC 7301 <2>: SSL Contexts. (line 258) +* RFC; RFC 7525: LibreSSL support. (line 53) +* RFC; RFC 7540: HTTP status codes. (line 130) +* RFC; RFC 7693: BLAKE2. (line 6) +* RFC; RFC 7725: HTTP status codes. (line 154) +* RFC; RFC 7914: Key derivation. (line 48) +* RFC; RFC 821: smtplib — SMTP protocol client. + (line 12) +* RFC; RFC 821 <1>: smtplib — SMTP protocol client. + (line 179) +* RFC; RFC 822: New and Improved Modules<3>. + (line 97) +* RFC; RFC 822 <1>: Functions<2>. (line 610) +* RFC; RFC 822 <2>: Functions<2>. (line 613) +* RFC; RFC 822 <3>: email Examples. (line 36) +* RFC; RFC 822 <4>: email header Internationalized headers. + (line 21) +* RFC; RFC 822 <5>: HTTPConnection Objects. + (line 138) +* RFC; RFC 822 <6>: SMTP Objects. (line 94) +* RFC; RFC 822 <7>: SMTP Objects. (line 232) +* RFC; RFC 822 <8>: SMTP Objects. (line 233) +* RFC; RFC 822 <9>: SMTP Example. (line 10) +* RFC; RFC 822 <10>: The GNUTranslations class. + (line 13) +* RFC; RFC 822 <11>: distutils util — Miscellaneous other utility functions. + (line 174) +* RFC; RFC 8305: Opening network connections. + (line 77) +* RFC; RFC 8305 <1>: Opening network connections. + (line 86) +* RFC; RFC 854: telnetlib — Telnet client. + (line 11) +* RFC; RFC 854 <1>: telnetlib — Telnet client. + (line 58) +* RFC; RFC 959: ftplib — FTP protocol client. + (line 16) +* RFC; RFC 959 <1>: FTP Objects. (line 148) +* RFC; RFC 977: nntplib — NNTP protocol client. + (line 13) +* RFC_4122 (in module uuid): uuid — UUID objects according to RFC 4122. + (line 228) +* rfile (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 110) +* rfind() (bytearray method): Bytes and Bytearray Operations. + (line 159) +* rfind() (bytes method): Bytes and Bytearray Operations. + (line 159) +* rfind() (mmap.mmap method): mmap — Memory-mapped file support. + (line 251) +* rfind() (str method): String Methods<2>. (line 345) +* rgb_to_hls() (in module colorsys): colorsys — Conversions between color systems. + (line 36) +* rgb_to_hsv() (in module colorsys): colorsys — Conversions between color systems. + (line 44) +* rgb_to_yiq() (in module colorsys): colorsys — Conversions between color systems. + (line 28) +* rglob() (pathlib.Path method): Methods<2>. (line 355) +* richcmpfunc (C type): Slot Type typedefs. (line 91) +* right (filecmp.dircmp attribute): The dircmp class. (line 45) +* right() (in module turtle): Turtle motion. (line 40) +* RIGHTSHIFT (in module token): token — Constants used with Python parse trees. + (line 162) +* RIGHTSHIFTEQUAL (in module token): token — Constants used with Python parse trees. + (line 206) +* right_list (filecmp.dircmp attribute): The dircmp class. (line 54) +* right_only (filecmp.dircmp attribute): The dircmp class. (line 67) +* rindex() (bytearray method): Bytes and Bytearray Operations. + (line 173) +* rindex() (bytes method): Bytes and Bytearray Operations. + (line 173) +* rindex() (str method): String Methods<2>. (line 352) +* rjust() (bytearray method): Bytes and Bytearray Operations. + (line 283) +* rjust() (bytes method): Bytes and Bytearray Operations. + (line 283) +* rjust() (str method): String Methods<2>. (line 357) +* rlcompleter (module): rlcompleter — Completion function for GNU readline. + (line 6) +* rlecode_hqx() (in module binascii): binascii — Convert between binary and ASCII. + (line 95) +* rledecode_hqx() (in module binascii): binascii — Convert between binary and ASCII. + (line 83) +* RLIMIT_AS (in module resource): Resource Limits. (line 147) +* RLIMIT_CORE (in module resource): Resource Limits. (line 92) +* RLIMIT_CPU (in module resource): Resource Limits. (line 99) +* RLIMIT_DATA (in module resource): Resource Limits. (line 111) +* RLIMIT_FSIZE (in module resource): Resource Limits. (line 107) +* RLIMIT_MEMLOCK (in module resource): Resource Limits. (line 139) +* RLIMIT_MSGQUEUE (in module resource): Resource Limits. (line 152) +* RLIMIT_NICE (in module resource): Resource Limits. (line 160) +* RLIMIT_NOFILE (in module resource): Resource Limits. (line 130) +* RLIMIT_NPROC (in module resource): Resource Limits. (line 126) +* RLIMIT_NPTS (in module resource): Resource Limits. (line 215) +* RLIMIT_OFILE (in module resource): Resource Limits. (line 135) +* RLIMIT_RSS (in module resource): Resource Limits. (line 121) +* RLIMIT_RTPRIO (in module resource): Resource Limits. (line 169) +* RLIMIT_RTTIME (in module resource): Resource Limits. (line 177) +* RLIMIT_SBSIZE (in module resource): Resource Limits. (line 194) +* RLIMIT_SIGPENDING (in module resource): Resource Limits. (line 186) +* RLIMIT_STACK (in module resource): Resource Limits. (line 115) +* RLIMIT_SWAP (in module resource): Resource Limits. (line 204) +* RLIMIT_VMEM (in module resource): Resource Limits. (line 143) +* RLIM_INFINITY (in module resource): Resource Limits. (line 20) +* RLock (class in multiprocessing): Synchronization primitives. + (line 102) +* RLock (class in threading): RLock Objects. (line 23) +* RLock() (multiprocessing.managers.SyncManager method): Managers. + (line 186) +* rmd() (ftplib.FTP method): FTP Objects. (line 225) +* rmdir() (in module os): Files and Directories. + (line 732) +* rmdir() (in module test.support): test support — Utilities for the Python test suite. + (line 177) +* rmdir() (pathlib.Path method): Methods<2>. (line 367) +* RMFF: chunk — Read IFF chunked data. + (line 8) +* rms() (in module audioop): audioop — Manipulate raw audio data. + (line 193) +* rmtree() (in module shutil): Directory and files operations. + (line 255) +* rmtree() (in module test.support): test support — Utilities for the Python test suite. + (line 183) +* RobotFileParser (class in urllib.robotparser): urllib robotparser — Parser for robots txt. + (line 16) +* robots.txt: urllib robotparser — Parser for robots txt. + (line 8) +* rollback() (sqlite3.Connection method): Connection Objects. + (line 40) +* rotate() (collections.deque method): deque objects. (line 109) +* rotate() (decimal.Context method): Context objects. (line 481) +* rotate() (decimal.Decimal method): Decimal objects. (line 515) +* rotate() (logging.handlers.BaseRotatingHandler method): BaseRotatingHandler. + (line 58) +* RotatingFileHandler (class in logging.handlers): RotatingFileHandler. + (line 9) +* rotation_filename() (logging.handlers.BaseRotatingHandler method): BaseRotatingHandler. + (line 41) +* rotator (logging.handlers.BaseRotatingHandler attribute): BaseRotatingHandler. + (line 32) +* ROT_FOUR (opcode): Python Bytecode Instructions. + (line 72) +* ROT_THREE (opcode): Python Bytecode Instructions. + (line 67) +* ROT_TWO (opcode): Python Bytecode Instructions. + (line 63) +* round() (built-in function): Built-in Functions. (line 1437) +* Rounded (class in decimal): Signals. (line 77) +* ROUND_05UP (in module decimal): Rounding modes. (line 34) +* ROUND_CEILING (in module decimal): Rounding modes. (line 6) +* ROUND_DOWN (in module decimal): Rounding modes. (line 10) +* ROUND_FLOOR (in module decimal): Rounding modes. (line 14) +* ROUND_HALF_DOWN (in module decimal): Rounding modes. (line 18) +* ROUND_HALF_EVEN (in module decimal): Rounding modes. (line 22) +* ROUND_HALF_UP (in module decimal): Rounding modes. (line 26) +* ROUND_UP (in module decimal): Rounding modes. (line 30) +* Row (class in sqlite3): Row Objects. (line 6) +* rowcount (sqlite3.Cursor attribute): Cursor Objects. (line 178) +* row_factory (sqlite3.Connection attribute): Connection Objects. + (line 301) +* RPAR (in module token): token — Constants used with Python parse trees. + (line 58) +* rpartition() (bytearray method): Bytes and Bytearray Operations. + (line 185) +* rpartition() (bytes method): Bytes and Bytearray Operations. + (line 185) +* rpartition() (str method): String Methods<2>. (line 364) +* rpc_paths (xmlrpc.server.SimpleXMLRPCRequestHandler attribute): SimpleXMLRPCServer Objects. + (line 64) +* rpop() (poplib.POP3 method): POP3 Objects. (line 46) +* rset() (poplib.POP3 method): POP3 Objects. (line 74) +* rshift() (in module operator): operator — Standard operators as functions. + (line 148) +* rsplit() (bytearray method): Bytes and Bytearray Operations. + (line 295) +* rsplit() (bytes method): Bytes and Bytearray Operations. + (line 295) +* rsplit() (str method): String Methods<2>. (line 372) +* RSQB (in module token): token — Constants used with Python parse trees. + (line 66) +* rstrip() (bytearray method): Bytes and Bytearray Operations. + (line 306) +* rstrip() (bytes method): Bytes and Bytearray Operations. + (line 306) +* rstrip() (str method): String Methods<2>. (line 381) +* rt() (in module turtle): Turtle motion. (line 40) +* RTLD_DEEPBIND (in module os): Miscellaneous System Information. + (line 141) +* RTLD_GLOBAL (in module os): Miscellaneous System Information. + (line 141) +* RTLD_LAZY (in module os): Miscellaneous System Information. + (line 141) +* RTLD_LOCAL (in module os): Miscellaneous System Information. + (line 141) +* RTLD_NODELETE (in module os): Miscellaneous System Information. + (line 141) +* RTLD_NOLOAD (in module os): Miscellaneous System Information. + (line 141) +* RTLD_NOW (in module os): Miscellaneous System Information. + (line 141) +* ruler (cmd.Cmd attribute): Cmd Objects. (line 164) +* run (pdb command): Debugger Commands. (line 322) +* Run script: Format menu Editor window only. + (line 53) +* run() (bdb.Bdb method): bdb — Debugger framework. + (line 363) +* run() (contextvars.Context method): Manual Context Management. + (line 31) +* run() (distutils.cmd.Command method): Creating a new Distutils command. + (line 40) +* run() (doctest.DocTestRunner method): DocTestRunner objects. + (line 95) +* run() (in module asyncio): Running an asyncio Program. + (line 6) +* run() (in module pdb): pdb — The Python Debugger. + (line 93) +* run() (in module profile): profile and cProfile Module Reference. + (line 9) +* run() (in module subprocess): Using the subprocess Module. + (line 15) +* run() (multiprocessing.Process method): Process and exceptions. + (line 33) +* run() (pdb.Pdb method): pdb — The Python Debugger. + (line 180) +* run() (profile.Profile method): profile and cProfile Module Reference. + (line 100) +* run() (sched.scheduler method): Scheduler Objects. (line 51) +* run() (test.support.BasicTestRunner method): test support — Utilities for the Python test suite. + (line 1104) +* run() (threading.Thread method): Thread Objects. (line 100) +* run() (trace.Trace method): Programmatic Interface. + (line 22) +* run() (unittest.IsolatedAsyncioTestCase method): Test cases. + (line 845) +* run() (unittest.TestCase method): Test cases. (line 83) +* run() (unittest.TestSuite method): Grouping tests. (line 41) +* run() (unittest.TextTestRunner method): Loading and running tests. + (line 481) +* run() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 98) +* runcall() (bdb.Bdb method): bdb — Debugger framework. + (line 380) +* runcall() (in module pdb): pdb — The Python Debugger. + (line 112) +* runcall() (pdb.Pdb method): pdb — The Python Debugger. + (line 180) +* runcall() (profile.Profile method): profile and cProfile Module Reference. + (line 109) +* runcode() (code.InteractiveInterpreter method): Interactive Interpreter Objects. + (line 33) +* runctx() (bdb.Bdb method): bdb — Debugger framework. + (line 375) +* runctx() (in module profile): profile and cProfile Module Reference. + (line 23) +* runctx() (profile.Profile method): profile and cProfile Module Reference. + (line 104) +* runctx() (trace.Trace method): Programmatic Interface. + (line 28) +* runeval() (bdb.Bdb method): bdb — Debugger framework. + (line 369) +* runeval() (in module pdb): pdb — The Python Debugger. + (line 105) +* runeval() (pdb.Pdb method): pdb — The Python Debugger. + (line 180) +* runfunc() (trace.Trace method): Programmatic Interface. + (line 35) +* running() (concurrent.futures.Future method): Future Objects. + (line 28) +* runpy (module): runpy — Locating and executing Python modules. + (line 6) +* runsource() (code.InteractiveInterpreter method): Interactive Interpreter Objects. + (line 6) +* RuntimeError: Concrete exceptions. + (line 204) +* RuntimeWarning: Warnings. (line 36) +* runtime_checkable() (in module typing): Classes functions and decorators. + (line 752) +* runtime_library_dir_option() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 230) +* run_coroutine_threadsafe() (in module asyncio): Scheduling From Other Threads. + (line 6) +* run_docstring_examples() (in module doctest): Basic API. (line 128) +* run_doctest() (in module test.support): test support — Utilities for the Python test suite. + (line 270) +* run_forever() (asyncio.loop method): Running and stopping the loop. + (line 16) +* run_in_executor() (asyncio.loop method): Executing code in thread or process pools. + (line 6) +* run_in_subinterp() (in module test.support): test support — Utilities for the Python test suite. + (line 951) +* run_module() (in module runpy): runpy — Locating and executing Python modules. + (line 27) +* run_path() (in module runpy): runpy — Locating and executing Python modules. + (line 95) +* run_python_until_end() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 30) +* run_script() (modulefinder.ModuleFinder method): modulefinder — Find modules used by a script. + (line 44) +* run_setup() (in module distutils.core): distutils core — Core Distutils functionality. + (line 121) +* run_unittest() (in module test.support): test support — Utilities for the Python test suite. + (line 254) +* run_until_complete() (asyncio.loop method): Running and stopping the loop. + (line 6) +* run_with_locale() (in module test.support): test support — Utilities for the Python test suite. + (line 574) +* run_with_tz() (in module test.support): test support — Utilities for the Python test suite. + (line 582) +* RUSAGE_BOTH (in module resource): Resource Usage. (line 122) +* RUSAGE_CHILDREN (in module resource): Resource Usage. (line 116) +* RUSAGE_SELF (in module resource): Resource Usage. (line 110) +* RUSAGE_THREAD (in module resource): Resource Usage. (line 128) +* RWF_DSYNC (in module os): File Descriptor Operations. + (line 535) +* RWF_HIPRI (in module os): File Descriptor Operations. + (line 482) +* RWF_NOWAIT (in module os): File Descriptor Operations. + (line 467) +* RWF_SYNC (in module os): File Descriptor Operations. + (line 545) +* R_OK (in module os): Files and Directories. + (line 106) +* S (in module re): Module Contents. (line 113) +* safe (uuid.SafeUUID attribute): uuid — UUID objects according to RFC 4122. + (line 31) +* SafeChildWatcher (class in asyncio): Process Watchers. (line 118) +* saferepr() (in module pprint): pprint — Data pretty printer. + (line 145) +* SafeUUID (class in uuid): uuid — UUID objects according to RFC 4122. + (line 27) +* safe_substitute() (string.Template method): Template strings. + (line 50) +* samefile() (in module os.path): os path — Common pathname manipulations. + (line 323) +* samefile() (pathlib.Path method): Methods<2>. (line 371) +* SameFileError: Directory and files operations. + (line 52) +* sameopenfile() (in module os.path): os path — Common pathname manipulations. + (line 339) +* samestat() (in module os.path): os path — Common pathname manipulations. + (line 350) +* same_files (filecmp.dircmp attribute): The dircmp class. (line 85) +* same_quantum() (decimal.Context method): Context objects. (line 485) +* same_quantum() (decimal.Decimal method): Decimal objects. (line 527) +* sample() (in module random): Functions for sequences. + (line 67) +* samples() (statistics.NormalDist method): NormalDist objects. + (line 58) +* save() (http.cookiejar.FileCookieJar method): CookieJar and FileCookieJar Objects. + (line 105) +* SAVEDCWD (in module test.support): test support — Utilities for the Python test suite. + (line 87) +* SaveKey() (in module winreg): Functions<11>. (line 400) +* SaveSignals (class in test.support): test support — Utilities for the Python test suite. + (line 1081) +* savetty() (in module curses): Functions<3>. (line 427) +* SAX2DOM (class in xml.dom.pulldom): xml dom pulldom — Support for building partial DOM trees. + (line 82) +* SAXException: xml sax — Support for SAX2 parsers. + (line 86) +* SAXNotRecognizedException: xml sax — Support for SAX2 parsers. + (line 111) +* SAXNotSupportedException: xml sax — Support for SAX2 parsers. + (line 118) +* SAXParseException: xml sax — Support for SAX2 parsers. + (line 103) +* scaleb() (decimal.Context method): Context objects. (line 489) +* scaleb() (decimal.Decimal method): Decimal objects. (line 537) +* scandir() (in module os): Files and Directories. + (line 749) +* scanf(): Simulating scanf. (line 6) +* sched (module): sched — Event scheduler. + (line 6) +* scheduler (class in sched): sched — Event scheduler. + (line 13) +* SCHED_BATCH (in module os): Interface to the scheduler. + (line 19) +* SCHED_FIFO (in module os): Interface to the scheduler. + (line 32) +* sched_getaffinity() (in module os): Interface to the scheduler. + (line 109) +* sched_getparam() (in module os): Interface to the scheduler. + (line 87) +* sched_getscheduler() (in module os): Interface to the scheduler. + (line 75) +* sched_get_priority_max() (in module os): Interface to the scheduler. + (line 63) +* sched_get_priority_min() (in module os): Interface to the scheduler. + (line 58) +* SCHED_IDLE (in module os): Interface to the scheduler. + (line 24) +* SCHED_OTHER (in module os): Interface to the scheduler. + (line 15) +* sched_param (class in os): Interface to the scheduler. + (line 46) +* sched_priority (os.sched_param attribute): Interface to the scheduler. + (line 54) +* SCHED_RESET_ON_FORK (in module os): Interface to the scheduler. + (line 40) +* SCHED_RR (in module os): Interface to the scheduler. + (line 36) +* sched_rr_get_interval() (in module os): Interface to the scheduler. + (line 93) +* sched_setaffinity() (in module os): Interface to the scheduler. + (line 102) +* sched_setparam() (in module os): Interface to the scheduler. + (line 81) +* sched_setscheduler() (in module os): Interface to the scheduler. + (line 68) +* SCHED_SPORADIC (in module os): Interface to the scheduler. + (line 28) +* sched_yield() (in module os): Interface to the scheduler. + (line 98) +* schema (in module msilib): Precomputed tables. (line 10) +* scope: Naming and binding. (line 6) +* scope <1>: Resolution of names. + (line 6) +* Screen (class in turtle): Public classes. (line 30) +* screensize() (in module turtle): Window control. (line 61) +* script_from_examples() (in module doctest): Debugging. (line 72) +* scroll() (curses.window method): Window Objects. (line 497) +* ScrolledCanvas (class in turtle): Public classes. (line 34) +* scrollok() (curses.window method): Window Objects. (line 501) +* scrypt() (in module hashlib): Key derivation. (line 44) +* seal() (in module unittest.mock): Sealing mocks. (line 6) +* search() (imaplib.IMAP4 method): IMAP4 Objects. (line 216) +* search() (in module re): Module Contents. (line 143) +* search() (re.Pattern method): Regular Expression Objects. + (line 9) +* second (datetime.datetime attribute): datetime Objects. (line 285) +* second (datetime.time attribute): time Objects. (line 58) +* seconds since the epoch: time — Time access and conversions. + (line 25) +* secrets (module): secrets — Generate secure random numbers for managing secrets. + (line 6) +* SECTCRE (configparser.ConfigParser attribute): Customizing Parser Behaviour. + (line 300) +* sections() (configparser.ConfigParser method): ConfigParser Objects. + (line 81) +* secure (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 45) +* secure hash algorithm, SHA1, SHA224, SHA256, SHA384, SHA512: hashlib — Secure hashes and message digests. + (line 8) +* Secure Sockets Layer: ssl — TLS/SSL wrapper for socket objects. + (line 8) +* see() (tkinter.ttk.Treeview method): ttk Treeview. (line 251) +* seed() (in module random): Bookkeeping functions. + (line 6) +* seek() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 92) +* seek() (io.IOBase method): I/O Base Classes. (line 104) +* seek() (io.TextIOBase method): Text I/O<2>. (line 66) +* seek() (mmap.mmap method): mmap — Memory-mapped file support. + (line 261) +* seekable() (io.IOBase method): I/O Base Classes. (line 129) +* SEEK_CUR (in module os): File Descriptor Operations. + (line 249) +* SEEK_END (in module os): File Descriptor Operations. + (line 249) +* SEEK_SET (in module os): File Descriptor Operations. + (line 249) +* seen_greeting (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 69) +* Select (class in tkinter.tix): Basic Widgets. (line 59) +* select (module): select — Waiting for I/O completion. + (line 6) +* select() (imaplib.IMAP4 method): IMAP4 Objects. (line 233) +* select() (in module select): select — Waiting for I/O completion. + (line 110) +* select() (selectors.BaseSelector method): Classes<3>. (line 108) +* select() (tkinter.ttk.Notebook method): ttk Notebook. (line 53) +* selected_alpn_protocol() (ssl.SSLSocket method): SSL Sockets. + (line 253) +* selected_npn_protocol() (ssl.SSLSocket method): SSL Sockets. + (line 263) +* selection() (tkinter.ttk.Treeview method): ttk Treeview. (line 259) +* selection_add() (tkinter.ttk.Treeview method): ttk Treeview. + (line 274) +* selection_remove() (tkinter.ttk.Treeview method): ttk Treeview. + (line 281) +* selection_set() (tkinter.ttk.Treeview method): ttk Treeview. + (line 267) +* selection_toggle() (tkinter.ttk.Treeview method): ttk Treeview. + (line 288) +* selector (urllib.request.Request attribute): Request Objects. + (line 34) +* SelectorEventLoop (class in asyncio): Event Loop Implementations. + (line 12) +* SelectorKey (class in selectors): Classes<3>. (line 29) +* selectors (module): selectors — High-level I/O multiplexing. + (line 6) +* SelectSelector (class in selectors): Classes<3>. (line 167) +* Semaphore (class in asyncio): Semaphore. (line 6) +* Semaphore (class in multiprocessing): Synchronization primitives. + (line 167) +* Semaphore (class in threading): Semaphore Objects. (line 19) +* Semaphore() (multiprocessing.managers.SyncManager method): Managers. + (line 191) +* semaphores, binary: _thread — Low-level threading API. + (line 6) +* SEMI (in module token): token — Constants used with Python parse trees. + (line 78) +* send() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 200) +* send() (coroutine method): Coroutine Objects. (line 23) +* send() (generator method): Generator-iterator methods. + (line 27) +* send() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 170) +* send() (imaplib.IMAP4 method): IMAP4 Objects. (line 240) +* send() (logging.handlers.DatagramHandler method): DatagramHandler. + (line 34) +* send() (logging.handlers.SocketHandler method): SocketHandler. + (line 63) +* send() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 15) +* send() (socket.socket method): Socket Objects. (line 381) +* sendall() (socket.socket method): Socket Objects. (line 397) +* sendcmd() (ftplib.FTP method): FTP Objects. (line 66) +* sendfile() (asyncio.loop method): Transferring files. (line 6) +* sendfile() (in module os): File Descriptor Operations. + (line 576) +* sendfile() (socket.socket method): Socket Objects. (line 488) +* sendfile() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 275) +* SendfileNotAvailableError: Exceptions<9>. (line 35) +* sendmail() (smtplib.SMTP method): SMTP Objects. (line 229) +* sendmsg() (socket.socket method): Socket Objects. (line 435) +* sendmsg_afalg() (socket.socket method): Socket Objects. (line 477) +* sendto() (asyncio.DatagramTransport method): Datagram Transports. + (line 6) +* sendto() (socket.socket method): Socket Objects. (line 416) +* send_bytes() (multiprocessing.connection.Connection method): Connection Objects<2>. + (line 54) +* send_error() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 206) +* send_flowing_data() (formatter.writer method): The Writer Interface. + (line 76) +* send_header() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 242) +* send_hor_rule() (formatter.writer method): The Writer Interface. + (line 68) +* send_label_data() (formatter.writer method): The Writer Interface. + (line 92) +* send_line_break() (formatter.writer method): The Writer Interface. + (line 54) +* send_literal_data() (formatter.writer method): The Writer Interface. + (line 83) +* send_message() (smtplib.SMTP method): SMTP Objects. (line 307) +* send_paragraph() (formatter.writer method): The Writer Interface. + (line 58) +* send_response() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 226) +* send_response_only() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 255) +* send_signal() (asyncio.asyncio.subprocess.Process method): Interacting with Subprocesses. + (line 81) +* send_signal() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 44) +* send_signal() (subprocess.Popen method): Popen Objects. (line 74) +* sentinel (in module unittest.mock): sentinel. (line 6) +* sentinel (multiprocessing.Process attribute): Process and exceptions. + (line 133) +* sep (in module os): Miscellaneous System Information. + (line 93) +* sequence: Glossary. (line 1150) +* Sequence (class in collections.abc): Collections Abstract Base Classes. + (line 159) +* Sequence (class in typing): Classes functions and decorators. + (line 229) +* sequence (in module msilib): Precomputed tables. (line 16) +* sequence2st() (in module parser): Creating ST Objects. + (line 26) +* sequence; item: Subscriptions. (line 6) +* sequence; iteration: Iterator Types. (line 6) +* SequenceMatcher (class in difflib): SequenceMatcher Objects. + (line 8) +* serializing; objects: pickle — Python object serialization. + (line 8) +* Server (class in asyncio): Server Objects. (line 12) +* server (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 71) +* ServerProxy (class in xmlrpc.client): xmlrpc client — XML-RPC client access. + (line 24) +* server_activate() (socketserver.BaseServer method): Server Objects<2>. + (line 164) +* server_address (socketserver.BaseServer attribute): Server Objects<2>. + (line 76) +* server_bind() (socketserver.BaseServer method): Server Objects<2>. + (line 170) +* server_close() (socketserver.BaseServer method): Server Objects<2>. + (line 61) +* server_hostname (ssl.SSLSocket attribute): SSL Sockets. (line 335) +* server_side (ssl.SSLSocket attribute): SSL Sockets. (line 328) +* server_software (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 176) +* server_version (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 127) +* server_version (http.server.SimpleHTTPRequestHandler attribute): http server — HTTP servers. + (line 344) +* serve_forever() (asyncio.Server method): Server Objects. (line 63) +* serve_forever() (socketserver.BaseServer method): Server Objects<2>. + (line 32) +* service_actions() (socketserver.BaseServer method): Server Objects<2>. + (line 45) +* session (ssl.SSLSocket attribute): SSL Sockets. (line 348) +* session_reused (ssl.SSLSocket attribute): SSL Sockets. (line 358) +* session_stats() (ssl.SSLContext method): SSL Contexts. (line 480) +* set (built-in class): Set Types — set frozenset. + (line 33) +* Set (class in collections.abc): Collections Abstract Base Classes. + (line 177) +* Set (class in typing): Classes functions and decorators. + (line 272) +* Set Breakpoint: Help menu Shell and Editor. + (line 30) +* set comprehension: Glossary. (line 1169) +* set() (asyncio.Event method): Event. (line 52) +* set() (configparser.ConfigParser method): ConfigParser Objects. + (line 250) +* set() (configparser.RawConfigParser method): RawConfigParser Objects. + (line 37) +* set() (contextvars.ContextVar method): Context Variables. (line 46) +* set() (http.cookies.Morsel method): Morsel Objects. (line 64) +* set() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 72) +* set() (test.support.EnvironmentVarGuard method): test support — Utilities for the Python test suite. + (line 1036) +* set() (threading.Event method): Event Objects. (line 27) +* set() (tkinter.ttk.Combobox method): ttk Combobox. (line 19) +* set() (tkinter.ttk.Spinbox method): ttk Spinbox. (line 12) +* set() (tkinter.ttk.Treeview method): ttk Treeview. (line 295) +* set() (xml.etree.ElementTree.Element method): Element Objects. + (line 81) +* set; comprehensions: Set displays. (line 6) +* set; display: Set displays. (line 6) +* setacl() (imaplib.IMAP4 method): IMAP4 Objects. (line 247) +* setannotation() (imaplib.IMAP4 method): IMAP4 Objects. (line 252) +* setattr() (built-in function): Built-in Functions. (line 1473) +* setattrfunc (C type): Slot Type typedefs. (line 57) +* setAttribute() (xml.dom.Element method): Element Objects<2>. + (line 67) +* setAttributeNode() (xml.dom.Element method): Element Objects<2>. + (line 71) +* setAttributeNodeNS() (xml.dom.Element method): Element Objects<2>. + (line 79) +* setAttributeNS() (xml.dom.Element method): Element Objects<2>. + (line 87) +* setattrofunc (C type): Slot Type typedefs. (line 69) +* SetBase() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 22) +* setblocking() (socket.socket method): Socket Objects. (line 512) +* setByteStream() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 36) +* setcbreak() (in module tty): tty — Terminal control functions. + (line 24) +* setCharacterStream() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 55) +* setcheckinterval() (in module sys): sys — System-specific parameters and functions. + (line 1196) +* setcomptype() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 158) +* setcomptype() (sunau.AU_write method): AU_write Objects. (line 28) +* setcomptype() (wave.Wave_write method): Wave_write Objects. + (line 51) +* setContentHandler() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 25) +* setcontext() (in module decimal): Context objects. (line 18) +* setDaemon() (threading.Thread method): Thread Objects. (line 197) +* setdefault() (dict method): Mapping Types — dict. + (line 205) +* setdefault() (http.cookies.Morsel method): Morsel Objects. (line 108) +* setdefaulttimeout() (in module socket): Other functions<2>. + (line 339) +* setdlopenflags() (in module sys): sys — System-specific parameters and functions. + (line 1212) +* setDocumentLocator() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 10) +* setDTDHandler() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 34) +* setegid() (in module os): Process Parameters. (line 338) +* setEncoding() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 22) +* setEntityResolver() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 43) +* setErrorHandler() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 54) +* seteuid() (in module os): Process Parameters. (line 344) +* setFeature() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 76) +* setfirstweekday() (in module calendar): calendar — General calendar-related functions. + (line 287) +* setfmt() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 125) +* setFormatter() (logging.Handler method): Handler Objects. (line 48) +* setframerate() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 148) +* setframerate() (sunau.AU_write method): AU_write Objects. (line 19) +* setframerate() (wave.Wave_write method): Wave_write Objects. + (line 38) +* setgid() (in module os): Process Parameters. (line 350) +* setgroups() (in module os): Process Parameters. (line 356) +* seth() (in module turtle): Turtle motion. (line 132) +* setheading() (in module turtle): Turtle motion. (line 132) +* sethostname() (in module socket): Other functions<2>. (line 346) +* SetInteger() (msilib.Record method): Record Objects. (line 32) +* setitem() (in module operator): operator — Standard operators as functions. + (line 201) +* setitimer() (in module signal): Module contents<2>. (line 348) +* setLevel() (logging.Handler method): Handler Objects. (line 35) +* setLevel() (logging.Logger method): Logger Objects. (line 50) +* setlocale() (in module locale): locale — Internationalization services. + (line 27) +* setLocale() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 60) +* setLoggerClass() (in module logging): Module-Level Functions. + (line 343) +* setlogmask() (in module syslog): syslog — Unix syslog library routines. + (line 73) +* setLogRecordFactory() (in module logging): Module-Level Functions. + (line 355) +* setmark() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 176) +* setMaxConns() (urllib.request.CacheFTPHandler method): CacheFTPHandler Objects. + (line 13) +* setmode() (in module msvcrt): File Operations. (line 38) +* setName() (threading.Thread method): Thread Objects. (line 142) +* setnchannels() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 140) +* setnchannels() (sunau.AU_write method): AU_write Objects. (line 9) +* setnchannels() (wave.Wave_write method): Wave_write Objects. + (line 30) +* setnframes() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 152) +* setnframes() (sunau.AU_write method): AU_write Objects. (line 23) +* setnframes() (wave.Wave_write method): Wave_write Objects. (line 45) +* SetParamEntityParsing() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 54) +* setparameters() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 189) +* setparams() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 168) +* setparams() (sunau.AU_write method): AU_write Objects. (line 33) +* setparams() (wave.Wave_write method): Wave_write Objects. (line 56) +* setpassword() (zipfile.ZipFile method): ZipFile Objects. (line 211) +* setpgid() (in module os): Process Parameters. (line 379) +* setpgrp() (in module os): Process Parameters. (line 371) +* setpos() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 108) +* setpos() (in module turtle): Turtle motion. (line 74) +* setpos() (sunau.AU_read method): AU_read Objects. (line 60) +* setpos() (wave.Wave_read method): Wave_read Objects. (line 69) +* setposition() (in module turtle): Turtle motion. (line 74) +* setpriority() (in module os): Process Parameters. (line 387) +* setprofile() (in module sys): sys — System-specific parameters and functions. + (line 1225) +* setprofile() (in module threading): threading — Thread-based parallelism. + (line 130) +* SetProperty() (msilib.SummaryInformation method): Summary Information Objects. + (line 22) +* setProperty() (xml.sax.xmlreader.XMLReader method): XMLReader Objects. + (line 90) +* setPublicId() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 6) +* setquota() (imaplib.IMAP4 method): IMAP4 Objects. (line 257) +* setraw() (in module tty): tty — Terminal control functions. + (line 18) +* setrecursionlimit() (in module sys): sys — System-specific parameters and functions. + (line 1277) +* setregid() (in module os): Process Parameters. (line 404) +* setresgid() (in module os): Process Parameters. (line 410) +* setresuid() (in module os): Process Parameters. (line 418) +* setreuid() (in module os): Process Parameters. (line 426) +* setrlimit() (in module resource): Resource Limits. (line 31) +* setsampwidth() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 144) +* setsampwidth() (sunau.AU_write method): AU_write Objects. (line 13) +* setsampwidth() (wave.Wave_write method): Wave_write Objects. + (line 34) +* setscrreg() (curses.window method): Window Objects. (line 511) +* setsid() (in module os): Process Parameters. (line 439) +* setsockopt() (socket.socket method): Socket Objects. (line 545) +* setstate() (codecs.IncrementalDecoder method): IncrementalDecoder Objects. + (line 60) +* setstate() (codecs.IncrementalEncoder method): IncrementalEncoder Objects. + (line 51) +* setstate() (in module random): Bookkeeping functions. + (line 34) +* setStream() (logging.StreamHandler method): StreamHandler. (line 32) +* SetStream() (msilib.Record method): Record Objects. (line 26) +* SetString() (msilib.Record method): Record Objects. (line 21) +* setswitchinterval() (in module sys): sys — System-specific parameters and functions. + (line 1295) +* setswitchinterval() (in module sys) <1>: Thread State and the Global Interpreter Lock. + (line 15) +* setswitchinterval() (in module test.support): test support — Utilities for the Python test suite. + (line 281) +* setSystemId() (xml.sax.xmlreader.InputSource method): InputSource Objects. + (line 14) +* setsyx() (in module curses): Functions<3>. (line 432) +* setTarget() (logging.handlers.MemoryHandler method): MemoryHandler. + (line 67) +* settiltangle() (in module turtle): Appearance. (line 118) +* settimeout() (socket.socket method): Socket Objects. (line 529) +* setTimeout() (urllib.request.CacheFTPHandler method): CacheFTPHandler Objects. + (line 9) +* settrace() (in module sys): sys — System-specific parameters and functions. + (line 1308) +* settrace() (in module threading): threading — Thread-based parallelism. + (line 123) +* setuid() (in module os): Process Parameters. (line 446) +* setundobuffer() (in module turtle): Special Turtle methods. + (line 61) +* setup() (in module distutils.core): distutils core — Core Distutils functionality. + (line 12) +* setup() (in module turtle): Methods specific to Screen not inherited from TurtleScreen. + (line 20) +* setup() (socketserver.BaseRequestHandler method): Request Handler Objects. + (line 14) +* setUp() (unittest.TestCase method): Test cases. (line 33) +* setUpClass() (unittest.TestCase method): Test cases. (line 55) +* setupterm() (in module curses): Functions<3>. (line 437) +* SETUP_ANNOTATIONS (opcode): Python Bytecode Instructions. + (line 357) +* SETUP_ASYNC_WITH (opcode): Python Bytecode Instructions. + (line 303) +* setup_environ() (wsgiref.handlers.BaseHandler method): wsgiref handlers – server/gateway base classes. + (line 196) +* SETUP_FINALLY (opcode): Python Bytecode Instructions. + (line 697) +* setup_python() (venv.EnvBuilder method): API<2>. (line 91) +* setup_scripts() (venv.EnvBuilder method): API<2>. (line 99) +* setup_testing_defaults() (in module wsgiref.util): wsgiref util – WSGI environment utilities. + (line 68) +* SETUP_WITH (opcode): Python Bytecode Instructions. + (line 438) +* SetValue() (in module winreg): Functions<11>. (line 424) +* SetValueEx() (in module winreg): Functions<11>. (line 455) +* setworldcoordinates() (in module turtle): Window control. (line 88) +* setx() (in module turtle): Turtle motion. (line 104) +* setxattr() (in module os): Linux extended attributes. + (line 57) +* sety() (in module turtle): Turtle motion. (line 118) +* SET_ADD (opcode): Python Bytecode Instructions. + (line 317) +* set_all(): Reference Count Details. + (line 98) +* set_allowed_domains() (http.cookiejar.DefaultCookiePolicy method): DefaultCookiePolicy Objects. + (line 70) +* set_alpn_protocols() (ssl.SSLContext method): SSL Contexts. + (line 252) +* set_app() (wsgiref.simple_server.WSGIServer method): wsgiref simple_server – a simple WSGI HTTP server. + (line 61) +* set_asyncgen_hooks() (in module sys): sys — System-specific parameters and functions. + (line 1404) +* set_authorizer() (sqlite3.Connection method): Connection Objects. + (line 194) +* set_auto_history() (in module readline): History list. (line 43) +* set_blocked_domains() (http.cookiejar.DefaultCookiePolicy method): DefaultCookiePolicy Objects. + (line 56) +* set_blocking() (in module os): File Descriptor Operations. + (line 612) +* set_boundary() (email.message.EmailMessage method): email message Representing an email message. + (line 411) +* set_boundary() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 587) +* set_break() (bdb.Bdb method): bdb — Debugger framework. + (line 281) +* set_charset() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 235) +* set_children() (tkinter.ttk.Treeview method): ttk Treeview. + (line 25) +* set_child_watcher() (asyncio.AbstractEventLoopPolicy method): Policy Objects. + (line 42) +* set_child_watcher() (in module asyncio): Process Watchers. (line 29) +* set_ciphers() (ssl.SSLContext method): SSL Contexts. (line 238) +* set_completer() (in module readline): Completion. (line 14) +* set_completer_delims() (in module readline): Completion. (line 48) +* set_completion_display_matches_hook() (in module readline): Completion. + (line 57) +* set_content() (email.contentmanager.ContentManager method): email contentmanager Managing MIME Content. + (line 40) +* set_content() (email.message.EmailMessage method): email message Representing an email message. + (line 590) +* set_content() (in module email.contentmanager): Content Manager Instances. + (line 35) +* set_continue() (bdb.Bdb method): bdb — Debugger framework. + (line 266) +* set_cookie() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 72) +* set_cookie_if_ok() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 68) +* set_coroutine_origin_tracking_depth() (in module sys): sys — System-specific parameters and functions. + (line 1429) +* set_current() (msilib.Feature method): Features<3>. (line 14) +* set_data() (importlib.abc.SourceLoader method): importlib abc – Abstract base classes related to import. + (line 568) +* set_data() (importlib.machinery.SourceFileLoader method): importlib machinery – Importers and path hooks. + (line 242) +* set_date() (mailbox.MaildirMessage method): MaildirMessage. + (line 97) +* set_debug() (asyncio.loop method): Enabling debug mode. + (line 14) +* set_debug() (in module gc): gc — Garbage Collector interface. + (line 48) +* set_debuglevel() (ftplib.FTP method): FTP Objects. (line 13) +* set_debuglevel() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 71) +* set_debuglevel() (nntplib.NNTP method): Methods<3>. (line 307) +* set_debuglevel() (poplib.POP3 method): POP3 Objects. (line 11) +* set_debuglevel() (smtplib.SMTP method): SMTP Objects. (line 8) +* set_debuglevel() (telnetlib.Telnet method): Telnet Objects. + (line 84) +* set_defaults() (argparse.ArgumentParser method): Parser defaults. + (line 6) +* set_defaults() (optparse.OptionParser method): Other methods. + (line 27) +* set_default_executor() (asyncio.loop method): Executing code in thread or process pools. + (line 65) +* set_default_type() (email.message.EmailMessage method): email message Representing an email message. + (line 342) +* set_default_type() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 450) +* set_default_verify_paths() (ssl.SSLContext method): SSL Contexts. + (line 229) +* set_ecdh_curve() (ssl.SSLContext method): SSL Contexts. (line 368) +* set_errno() (in module ctypes): Utility functions. (line 171) +* set_event_loop() (asyncio.AbstractEventLoopPolicy method): Policy Objects. + (line 23) +* set_event_loop() (in module asyncio): Event Loop. (line 57) +* set_event_loop_policy() (in module asyncio): Getting and Setting the Policy. + (line 13) +* set_exception() (asyncio.Future method): Future Object. (line 52) +* set_exception() (concurrent.futures.Future method): Future Objects. + (line 123) +* set_exception_handler() (asyncio.loop method): Error Handling API. + (line 8) +* set_executable() (in module multiprocessing): Miscellaneous<3>. + (line 107) +* set_executables() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 235) +* set_flags() (mailbox.MaildirMessage method): MaildirMessage. + (line 72) +* set_flags() (mailbox.mboxMessage method): mboxMessage. (line 72) +* set_flags() (mailbox.MMDFMessage method): MMDFMessage. (line 71) +* set_from() (mailbox.mboxMessage method): mboxMessage. (line 53) +* set_from() (mailbox.MMDFMessage method): MMDFMessage. (line 52) +* set_handle_inheritable() (in module os): Inheritance of File Descriptors. + (line 38) +* set_history_length() (in module readline): History file. (line 30) +* set_include_dirs() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 103) +* set_info() (mailbox.MaildirMessage method): MaildirMessage. + (line 108) +* set_inheritable() (in module os): Inheritance of File Descriptors. + (line 28) +* set_inheritable() (socket.socket method): Socket Objects. (line 505) +* set_labels() (mailbox.BabylMessage method): BabylMessage. (line 51) +* set_last_error() (in module ctypes): Utility functions. (line 180) +* set_libraries() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 128) +* set_library_dirs() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 144) +* set_link_objects() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 187) +* set_literal (2to3 fixer): Fixers. (line 288) +* set_loader() (in module importlib.util): importlib util – Utility code for importers. + (line 168) +* set_match_tests() (in module test.support): test support — Utilities for the Python test suite. + (line 250) +* set_memlimit() (in module test.support): test support — Utilities for the Python test suite. + (line 371) +* set_name() (asyncio.Task method): Task Object. (line 220) +* set_next() (bdb.Bdb method): bdb — Debugger framework. + (line 248) +* set_nonstandard_attr() (http.cookiejar.Cookie method): Cookie Objects<2>. + (line 103) +* set_npn_protocols() (ssl.SSLContext method): SSL Contexts. (line 272) +* set_ok() (http.cookiejar.CookiePolicy method): CookiePolicy Objects. + (line 9) +* set_option_negotiation_callback() (telnetlib.Telnet method): Telnet Objects. + (line 143) +* set_output_charset() (gettext.NullTranslations method): The NullTranslations class. + (line 94) +* set_package() (in module importlib.util): importlib util – Utility code for importers. + (line 183) +* set_param() (email.message.EmailMessage method): email message Representing an email message. + (line 351) +* set_param() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 518) +* set_pasv() (ftplib.FTP method): FTP Objects. (line 107) +* set_payload() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 225) +* set_policy() (http.cookiejar.CookieJar method): CookieJar and FileCookieJar Objects. + (line 56) +* set_position() (xdrlib.Unpacker method): Unpacker Objects. (line 16) +* set_pre_input_hook() (in module readline): Startup hooks. (line 14) +* set_progress_handler() (sqlite3.Connection method): Connection Objects. + (line 218) +* set_protocol() (asyncio.BaseTransport method): Base Transport. + (line 80) +* set_proxy() (urllib.request.Request method): Request Objects. + (line 114) +* set_python_build() (in module distutils.sysconfig): distutils sysconfig — System configuration information. + (line 99) +* set_quit() (bdb.Bdb method): bdb — Debugger framework. + (line 271) +* set_recsrc() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 89) +* set_result() (asyncio.Future method): Future Object. (line 45) +* set_result() (concurrent.futures.Future method): Future Objects. + (line 111) +* set_return() (bdb.Bdb method): bdb — Debugger framework. + (line 252) +* set_running_or_notify_cancel() (concurrent.futures.Future method): Future Objects. + (line 89) +* set_runtime_library_dirs() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 155) +* set_seq1() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 54) +* set_seq2() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 59) +* set_seqs() (difflib.SequenceMatcher method): SequenceMatcher Objects. + (line 44) +* set_sequences() (mailbox.MH method): MH. (line 58) +* set_sequences() (mailbox.MHMessage method): MHMessage. (line 37) +* set_servername_callback (ssl.SSLContext attribute): SSL Contexts. + (line 339) +* set_server_documentation() (xmlrpc.server.DocCGIXMLRPCRequestHandler method): DocCGIXMLRPCRequestHandler. + (line 24) +* set_server_documentation() (xmlrpc.server.DocXMLRPCServer method): DocXMLRPCServer Objects. + (line 24) +* set_server_name() (xmlrpc.server.DocCGIXMLRPCRequestHandler method): DocCGIXMLRPCRequestHandler. + (line 18) +* set_server_name() (xmlrpc.server.DocXMLRPCServer method): DocXMLRPCServer Objects. + (line 18) +* set_server_title() (xmlrpc.server.DocCGIXMLRPCRequestHandler method): DocCGIXMLRPCRequestHandler. + (line 13) +* set_server_title() (xmlrpc.server.DocXMLRPCServer method): DocXMLRPCServer Objects. + (line 13) +* set_spacing() (formatter.formatter method): The Formatter Interface. + (line 139) +* set_startup_hook() (in module readline): Startup hooks. (line 6) +* set_start_method() (in module multiprocessing): Miscellaneous<3>. + (line 120) +* set_step() (bdb.Bdb method): bdb — Debugger framework. + (line 244) +* set_subdir() (mailbox.MaildirMessage method): MaildirMessage. + (line 58) +* set_task_factory() (asyncio.loop method): Creating Futures and Tasks. + (line 31) +* set_terminator() (asynchat.async_chat method): asynchat — Asynchronous socket command/response handler. + (line 117) +* set_threshold() (in module gc): gc — Garbage Collector interface. + (line 89) +* set_trace() (bdb.Bdb method): bdb — Debugger framework. + (line 261) +* set_trace() (in module bdb): bdb — Debugger framework. + (line 404) +* set_trace() (in module pdb): pdb — The Python Debugger. + (line 119) +* set_trace() (pdb.Pdb method): pdb — The Python Debugger. + (line 180) +* set_trace_callback() (sqlite3.Connection method): Connection Objects. + (line 232) +* set_tunnel() (http.client.HTTPConnection method): HTTPConnection Objects. + (line 81) +* set_type() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 551) +* set_unittest_reportflags() (in module doctest): Unittest API. + (line 165) +* set_unixfrom() (email.message.EmailMessage method): email message Representing an email message. + (line 146) +* set_unixfrom() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 157) +* set_until() (bdb.Bdb method): bdb — Debugger framework. + (line 256) +* set_url() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 21) +* set_usage() (optparse.OptionParser method): Other methods. (line 8) +* set_userptr() (curses.panel.Panel method): Panel Objects. (line 43) +* set_visible() (mailbox.BabylMessage method): BabylMessage. (line 68) +* set_wakeup_fd() (in module signal): Module contents<2>. (line 377) +* set_write_buffer_limits() (asyncio.WriteTransport method): Write-only Transports. + (line 33) +* SF_APPEND (in module stat): stat — Interpreting stat results. + (line 374) +* SF_ARCHIVED (in module stat): stat — Interpreting stat results. + (line 366) +* SF_IMMUTABLE (in module stat): stat — Interpreting stat results. + (line 370) +* SF_MNOWAIT (in module os): File Descriptor Operations. + (line 625) +* SF_NODISKIO (in module os): File Descriptor Operations. + (line 625) +* SF_NOUNLINK (in module stat): stat — Interpreting stat results. + (line 378) +* SF_SNAPSHOT (in module stat): stat — Interpreting stat results. + (line 382) +* SF_SYNC (in module os): File Descriptor Operations. + (line 625) +* Shape (class in turtle): Public classes. (line 43) +* shape (memoryview attribute): Memory Views. (line 446) +* shape() (in module turtle): Appearance. (line 6) +* shapesize() (in module turtle): Appearance. (line 52) +* shapetransform() (in module turtle): Appearance. (line 157) +* share() (socket.socket method): Socket Objects. (line 575) +* ShareableList (class in multiprocessing.shared_memory): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 243) +* ShareableList() (multiprocessing.managers.SharedMemoryManager method): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 203) +* Shared Memory: multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 10) +* SharedMemory (class in multiprocessing.shared_memory): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 33) +* SharedMemory() (multiprocessing.managers.SharedMemoryManager method): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 198) +* SharedMemoryManager (class in multiprocessing.managers): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 171) +* shared_ciphers() (ssl.SSLSocket method): SSL Sockets. (line 216) +* shared_object_filename() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 474) +* shearfactor() (in module turtle): Appearance. (line 84) +* Shelf (class in shelve): Restrictions. (line 23) +* shelve (module): shelve — Python object persistence. + (line 6) +* shield() (in module asyncio): Shielding From Cancellation. + (line 6) +* shift() (decimal.Context method): Context objects. (line 494) +* shift() (decimal.Decimal method): Decimal objects. (line 543) +* shifting; operation: Shifting operations. + (line 6) +* shifting; operations: Bitwise Operations on Integer Types. + (line 6) +* shift_path_info() (in module wsgiref.util): wsgiref util – WSGI environment utilities. + (line 38) +* shlex (class in shlex): shlex — Simple lexical analysis. + (line 82) +* shlex (module): shlex — Simple lexical analysis. + (line 6) +* shm (multiprocessing.shared_memory.ShareableList attribute): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 278) +* shortDescription() (unittest.TestCase method): Test cases. (line 733) +* shorten() (in module textwrap): textwrap — Text wrapping and filling. + (line 40) +* shouldFlush() (logging.handlers.BufferingHandler method): MemoryHandler. + (line 35) +* shouldFlush() (logging.handlers.MemoryHandler method): MemoryHandler. + (line 71) +* shouldStop (unittest.TestResult attribute): Loading and running tests. + (line 268) +* show() (curses.panel.Panel method): Panel Objects. (line 49) +* showsyntaxerror() (code.InteractiveInterpreter method): Interactive Interpreter Objects. + (line 44) +* showtraceback() (code.InteractiveInterpreter method): Interactive Interpreter Objects. + (line 53) +* showturtle() (in module turtle): Visibility. (line 15) +* showwarning() (in module warnings): Available Functions. + (line 56) +* show_code() (in module dis): Analysis functions. (line 26) +* show_compilers() (in module distutils.ccompiler): distutils ccompiler — CCompiler base class. + (line 62) +* shuffle() (in module random): Functions for sequences. + (line 49) +* shutdown() (concurrent.futures.Executor method): Executor Objects. + (line 56) +* shutdown() (imaplib.IMAP4 method): IMAP4 Objects. (line 262) +* shutdown() (in module logging): Module-Level Functions. + (line 332) +* shutdown() (multiprocessing.managers.BaseManager method): Managers. + (line 67) +* shutdown() (socket.socket method): Socket Objects. (line 568) +* shutdown() (socketserver.BaseServer method): Server Objects<2>. + (line 54) +* shutdown_asyncgens() (asyncio.loop method): Running and stopping the loop. + (line 56) +* shutil (module): shutil — High-level file operations. + (line 6) +* side_effect (unittest.mock.Mock attribute): The Mock Class. + (line 347) +* SIGABRT (in module signal): Module contents<2>. (line 28) +* SIGALRM (in module signal): Module contents<2>. (line 32) +* SIGBREAK (in module signal): Module contents<2>. (line 38) +* SIGBUS (in module signal): Module contents<2>. (line 44) +* SIGCHLD (in module signal): Module contents<2>. (line 50) +* SIGCLD (in module signal): Module contents<2>. (line 56) +* SIGCONT (in module signal): Module contents<2>. (line 60) +* SIGFPE (in module signal): Module contents<2>. (line 66) +* SIGHUP (in module signal): Module contents<2>. (line 76) +* SIGILL (in module signal): Module contents<2>. (line 83) +* SIGINT: Signal Handling<2>. (line 8) +* SIGINT <1>: Signal Handling<2>. (line 20) +* SIGINT (in module signal): Module contents<2>. (line 87) +* siginterrupt() (in module signal): Module contents<2>. (line 419) +* SIGKILL (in module signal): Module contents<2>. (line 93) +* signal (module): signal — Set handlers for asynchronous events. + (line 6) +* signal() (in module signal): Module contents<2>. (line 433) +* Signature (class in inspect): Introspecting callables with the Signature object. + (line 50) +* signature (inspect.BoundArguments attribute): Introspecting callables with the Signature object. + (line 308) +* signature() (in module inspect): Introspecting callables with the Signature object. + (line 12) +* sigpending() (in module signal): Module contents<2>. (line 460) +* SIGPIPE (in module signal): Module contents<2>. (line 101) +* SIGSEGV (in module signal): Module contents<2>. (line 109) +* SIGTERM (in module signal): Module contents<2>. (line 113) +* sigtimedwait() (in module signal): Module contents<2>. (line 518) +* SIGUSR1 (in module signal): Module contents<2>. (line 117) +* SIGUSR2 (in module signal): Module contents<2>. (line 123) +* sigwait() (in module signal): Module contents<2>. (line 474) +* sigwaitinfo() (in module signal): Module contents<2>. (line 490) +* SIGWINCH (in module signal): Module contents<2>. (line 129) +* SIG_BLOCK (in module signal): Module contents<2>. (line 186) +* SIG_DFL (in module signal): Module contents<2>. (line 15) +* SIG_IGN (in module signal): Module contents<2>. (line 23) +* SIG_SETMASK (in module signal): Module contents<2>. (line 201) +* SIG_UNBLOCK (in module signal): Module contents<2>. (line 193) +* Simple Mail Transfer Protocol: smtplib — SMTP protocol client. + (line 8) +* simple; statement: Simple statements. (line 6) +* SimpleCookie (class in http.cookies): http cookies — HTTP state management. + (line 48) +* simplefilter() (in module warnings): Available Functions. + (line 90) +* SimpleHandler (class in wsgiref.handlers): wsgiref handlers – server/gateway base classes. + (line 70) +* SimpleHTTPRequestHandler (class in http.server): http server — HTTP servers. + (line 331) +* SimpleNamespace (class in types): Additional Utility Classes and Functions. + (line 6) +* SimpleQueue (class in multiprocessing): Pipes and Queues. (line 204) +* SimpleQueue (class in queue): queue — A synchronized queue class. + (line 75) +* SimpleXMLRPCRequestHandler (class in xmlrpc.server): xmlrpc server — Basic XML-RPC servers. + (line 58) +* SimpleXMLRPCServer (class in xmlrpc.server): xmlrpc server — Basic XML-RPC servers. + (line 19) +* sin() (in module cmath): Trigonometric functions<2>. + (line 29) +* sin() (in module math): Trigonometric functions. + (line 57) +* single dispatch: Glossary. (line 1177) +* SingleAddressHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 217) +* singledispatch() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 319) +* singledispatchmethod (class in functools): functools — Higher-order functions and operations on callable objects. + (line 436) +* singleton; tuple: The standard type hierarchy. + (line 172) +* sinh() (in module cmath): Hyperbolic functions<2>. + (line 30) +* sinh() (in module math): Hyperbolic functions. + (line 25) +* SIO_KEEPALIVE_VALS (in module socket): Constants<8>. (line 167) +* SIO_LOOPBACK_FAST_PATH (in module socket): Constants<8>. (line 167) +* SIO_RCVALL (in module socket): Constants<8>. (line 167) +* site (module): site — Site-specific configuration hook. + (line 6) +* site command line option; –user-base: Command Line Interface<4>. + (line 18) +* site command line option; –user-site: Command Line Interface<4>. + (line 22) +* site-packages; directory: site — Site-specific configuration hook. + (line 24) +* site_maps() (urllib.robotparser.RobotFileParser method): urllib robotparser — Parser for robots txt. + (line 71) +* sixtofour (ipaddress.IPv6Address attribute): Address objects. + (line 202) +* size (multiprocessing.shared_memory.SharedMemory attribute): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 96) +* size (struct.Struct attribute): Classes<2>. (line 62) +* size (tarfile.TarInfo attribute): TarInfo Objects. (line 47) +* size (tracemalloc.Statistic attribute): Statistic. (line 19) +* size (tracemalloc.StatisticDiff attribute): StatisticDiff. (line 26) +* size (tracemalloc.Trace attribute): Trace. (line 23) +* size() (ftplib.FTP method): FTP Objects. (line 229) +* size() (mmap.mmap method): mmap — Memory-mapped file support. + (line 269) +* Sized (class in collections.abc): Collections Abstract Base Classes. + (line 112) +* Sized (class in typing): Classes functions and decorators. + (line 199) +* sizeof() (in module ctypes): Utility functions. (line 189) +* size_diff (tracemalloc.StatisticDiff attribute): StatisticDiff. + (line 32) +* SIZE_MAX: Integer Objects. (line 210) +* SKIP (in module doctest): Option Flags. (line 103) +* skip() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 113) +* skip() (in module unittest): Skipping tests and expected failures. + (line 85) +* skipIf() (in module unittest): Skipping tests and expected failures. + (line 90) +* skipinitialspace (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 65) +* skipped (unittest.TestResult attribute): Loading and running tests. + (line 250) +* skippedEntity() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 176) +* SkipTest: Skipping tests and expected failures. + (line 104) +* skipTest() (unittest.TestCase method): Test cases. (line 97) +* skipUnless() (in module unittest): Skipping tests and expected failures. + (line 94) +* skip_unless_bind_unix_socket() (in module test.support): test support — Utilities for the Python test suite. + (line 563) +* skip_unless_symlink() (in module test.support): test support — Utilities for the Python test suite. + (line 554) +* skip_unless_xattr() (in module test.support): test support — Utilities for the Python test suite. + (line 559) +* SLASH (in module token): token — Constants used with Python parse trees. + (line 94) +* SLASHEQUAL (in module token): token — Constants used with Python parse trees. + (line 182) +* slave() (nntplib.NNTP method): Methods<3>. (line 303) +* sleep() (in module asyncio): Sleeping. (line 6) +* sleep() (in module time): Functions<2>. (line 215) +* slice: Slicings. (line 6) +* slice <1>: Glossary. (line 1182) +* slice (built-in class): Built-in Functions. (line 1482) +* slice; assignment: Mutable Sequence Types. + (line 16) +* slice; operation: Common Sequence Operations. + (line 21) +* slicing: The standard type hierarchy. + (line 133) +* slicing <1>: The standard type hierarchy. + (line 192) +* slicing <2>: Slicings. (line 6) +* slicing; assignment: Assignment statements. + (line 126) +* SMALLEST (in module test.support): test support — Utilities for the Python test suite. + (line 155) +* SMTP (class in smtplib): smtplib — SMTP protocol client. + (line 16) +* SMTP (in module email.policy): email policy Policy Objects. + (line 493) +* SMTP; protocol: smtplib — SMTP protocol client. + (line 8) +* SMTPAuthenticationError: smtplib — SMTP protocol client. + (line 171) +* SMTPChannel (class in smtpd): SMTPChannel Objects. + (line 6) +* SMTPConnectError: smtplib — SMTP protocol client. + (line 156) +* smtpd (module): smtpd — SMTP Server. + (line 6) +* SMTPDataError: smtplib — SMTP protocol client. + (line 152) +* SMTPException: smtplib — SMTP protocol client. + (line 117) +* SMTPHandler (class in logging.handlers): SMTPHandler. (line 10) +* SMTPHeloError: smtplib — SMTP protocol client. + (line 161) +* smtplib (module): smtplib — SMTP protocol client. + (line 6) +* SMTPNotSupportedError: smtplib — SMTP protocol client. + (line 165) +* SMTPRecipientsRefused: smtplib — SMTP protocol client. + (line 145) +* SMTPResponseException: smtplib — SMTP protocol client. + (line 131) +* SMTPSenderRefused: smtplib — SMTP protocol client. + (line 139) +* SMTPServer (class in smtpd): SMTPServer Objects. (line 6) +* SMTPServerDisconnected: smtplib — SMTP protocol client. + (line 125) +* SMTPUTF8 (in module email.policy): email policy Policy Objects. + (line 499) +* smtp_server (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 44) +* SMTP_SSL (class in smtplib): smtplib — SMTP protocol client. + (line 66) +* smtp_state (smtpd.SMTPChannel attribute): SMTPChannel Objects. + (line 63) +* Snapshot (class in tracemalloc): Snapshot. (line 6) +* sndhdr (module): sndhdr — Determine type of sound file. + (line 6) +* SND_ALIAS (in module winsound): winsound — Sound-playing interface for Windows. + (line 47) +* SND_ASYNC (in module winsound): winsound — Sound-playing interface for Windows. + (line 109) +* SND_FILENAME (in module winsound): winsound — Sound-playing interface for Windows. + (line 42) +* SND_LOOP (in module winsound): winsound — Sound-playing interface for Windows. + (line 88) +* SND_MEMORY (in module winsound): winsound — Sound-playing interface for Windows. + (line 94) +* SND_NODEFAULT (in module winsound): winsound — Sound-playing interface for Windows. + (line 113) +* SND_NOSTOP (in module winsound): winsound — Sound-playing interface for Windows. + (line 118) +* SND_NOWAIT (in module winsound): winsound — Sound-playing interface for Windows. + (line 122) +* SND_PURGE (in module winsound): winsound — Sound-playing interface for Windows. + (line 103) +* sniff() (csv.Sniffer method): Module Contents<3>. (line 211) +* Sniffer (class in csv): Module Contents<3>. (line 204) +* sni_callback (ssl.SSLContext attribute): SSL Contexts. (line 288) +* socket (module): socket — Low-level networking interface. + (line 6) +* socket (socketserver.BaseServer attribute): Server Objects<2>. + (line 85) +* socket() (imaplib.IMAP4 method): IMAP4 Objects. (line 268) +* socket() (in module socket): Creating sockets. (line 8) +* socket() (in module socket) <1>: select — Waiting for I/O completion. + (line 138) +* SocketHandler (class in logging.handlers): SocketHandler. (line 10) +* socketpair() (in module socket): Creating sockets. (line 60) +* sockets (asyncio.Server attribute): Server Objects. (line 99) +* socketserver (module): socketserver — A framework for network servers. + (line 6) +* SocketType (in module socket): Creating sockets. (line 173) +* socket_type (socketserver.BaseServer attribute): Server Objects<2>. + (line 108) +* sock_accept() (asyncio.loop method): Working with socket objects directly. + (line 74) +* SOCK_CLOEXEC (in module socket): Constants<8>. (line 32) +* sock_connect() (asyncio.loop method): Working with socket objects directly. + (line 55) +* SOCK_DGRAM (in module socket): Constants<8>. (line 21) +* SOCK_MAX_SIZE (in module test.support): test support — Utilities for the Python test suite. + (line 100) +* SOCK_NONBLOCK (in module socket): Constants<8>. (line 32) +* SOCK_RAW (in module socket): Constants<8>. (line 21) +* SOCK_RDM (in module socket): Constants<8>. (line 21) +* sock_recv() (asyncio.loop method): Working with socket objects directly. + (line 12) +* sock_recv_into() (asyncio.loop method): Working with socket objects directly. + (line 26) +* sock_sendall() (asyncio.loop method): Working with socket objects directly. + (line 37) +* sock_sendfile() (asyncio.loop method): Working with socket objects directly. + (line 97) +* SOCK_SEQPACKET (in module socket): Constants<8>. (line 21) +* SOCK_STREAM (in module socket): Constants<8>. (line 21) +* SOL_ALG (in module socket): Constants<8>. (line 183) +* SOL_RDS (in module socket): Constants<8>. (line 154) +* SOMAXCONN (in module socket): Constants<8>. (line 48) +* sort() (imaplib.IMAP4 method): IMAP4 Objects. (line 272) +* sort() (list method): Lists<2>. (line 39) +* sortdict() (in module test.support): test support — Utilities for the Python test suite. + (line 224) +* sorted() (built-in function): Built-in Functions. (line 1498) +* sortTestMethodsUsing (unittest.TestLoader attribute): Loading and running tests. + (line 186) +* sort_stats() (pstats.Stats method): The Stats Class. (line 67) +* source (doctest.Example attribute): Example Objects. (line 17) +* source (pdb command): Debugger Commands. (line 251) +* source (shlex.shlex attribute): shlex Objects. (line 151) +* source character set: Encoding declarations. + (line 6) +* SourceFileLoader (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 216) +* sourcehook() (shlex.shlex method): shlex Objects. (line 26) +* SourcelessFileLoader (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 256) +* SourceLoader (class in importlib.abc): importlib abc – Abstract base classes related to import. + (line 507) +* SOURCE_DATE_EPOCH: py_compile<2>. (line 7) +* SOURCE_DATE_EPOCH <1>: py_compile — Compile Python source files. + (line 65) +* SOURCE_DATE_EPOCH <2>: py_compile — Compile Python source files. + (line 81) +* SOURCE_DATE_EPOCH <3>: py_compile — Compile Python source files. + (line 85) +* SOURCE_DATE_EPOCH <4>: Command-line use. (line 85) +* source_from_cache() (in module imp): imp — Access the import internals. + (line 228) +* source_from_cache() (in module importlib.util): importlib util – Utility code for importers. + (line 58) +* source_hash() (in module importlib.util): importlib util – Utility code for importers. + (line 217) +* SOURCE_SUFFIXES (in module importlib.machinery): importlib machinery – Importers and path hooks. + (line 13) +* source_to_code() (importlib.abc.InspectLoader static method): importlib abc – Abstract base classes related to import. + (line 422) +* space: Indentation. (line 6) +* space; in printf-style formatting: printf-style String Formatting. + (line 66) +* space; in printf-style formatting <1>: printf-style Bytes Formatting. + (line 63) +* space; in string formatting: Format Specification Mini-Language. + (line 71) +* span() (re.Match method): Match Objects. (line 157) +* spawn() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 488) +* spawn() (in module pty): pty — Pseudo-terminal utilities. + (line 36) +* spawnl() (in module os): Process Management. (line 500) +* spawnle() (in module os): Process Management. (line 500) +* spawnlp() (in module os): Process Management. (line 500) +* spawnlpe() (in module os): Process Management. (line 500) +* spawnv() (in module os): Process Management. (line 500) +* spawnve() (in module os): Process Management. (line 500) +* spawnvp() (in module os): Process Management. (line 500) +* spawnvpe() (in module os): Process Management. (line 500) +* spawn_python() (in module test.support.script_helper): test support script_helper — Utilities for the Python execution tests. + (line 59) +* special method: Glossary. (line 1190) +* special; attribute: The standard type hierarchy. + (line 13) +* special; method: Glossary. (line 1194) +* specified_attributes (xml.parsers.expat.xmlparser attribute): XMLParser Objects<2>. + (line 117) +* spec_from_file_location() (in module importlib.util): importlib util – Utility code for importers. + (line 205) +* spec_from_loader() (in module importlib.util): importlib util – Utility code for importers. + (line 194) +* speed() (in module turtle): Turtle motion. (line 302) +* speed() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 140) +* Spinbox (class in tkinter.ttk): ttk Spinbox. (line 6) +* split() (bytearray method): Bytes and Bytearray Operations. + (line 329) +* split() (bytes method): Bytes and Bytearray Operations. + (line 329) +* split() (in module os.path): os path — Common pathname manipulations. + (line 364) +* split() (in module re): Module Contents. (line 176) +* split() (in module shlex): shlex — Simple lexical analysis. + (line 17) +* split() (re.Pattern method): Regular Expression Objects. + (line 74) +* split() (str method): String Methods<2>. (line 394) +* splitdrive() (in module os.path): os path — Common pathname manipulations. + (line 379) +* splitext() (in module os.path): os path — Common pathname manipulations. + (line 400) +* splitlines() (bytearray method): Bytes and Bytearray Operations. + (line 572) +* splitlines() (bytes method): Bytes and Bytearray Operations. + (line 572) +* splitlines() (str method): String Methods<2>. (line 434) +* SplitResult (class in urllib.parse): Structured Parse Results. + (line 55) +* SplitResultBytes (class in urllib.parse): Structured Parse Results. + (line 82) +* split_quoted() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 98) +* SpooledTemporaryFile() (in module tempfile): tempfile — Generate temporary files and directories. + (line 93) +* sprintf-style formatting: printf-style String Formatting. + (line 6) +* sprintf-style formatting <1>: printf-style Bytes Formatting. + (line 6) +* spwd (module): spwd — The shadow password database. + (line 6) +* sqlite3 (module): sqlite3 — DB-API 2 0 interface for SQLite databases. + (line 6) +* sqlite_version (in module sqlite3): Module functions and constants. + (line 16) +* sqlite_version_info (in module sqlite3): Module functions and constants. + (line 20) +* sqrt() (decimal.Context method): Context objects. (line 498) +* sqrt() (decimal.Decimal method): Decimal objects. (line 554) +* sqrt() (in module cmath): Power and logarithmic functions<2>. + (line 23) +* sqrt() (in module math): Power and logarithmic functions. + (line 74) +* ssizeargfunc (C type): Slot Type typedefs. (line 116) +* ssizeobjargproc (C type): Slot Type typedefs. (line 118) +* SSL: ssl — TLS/SSL wrapper for socket objects. + (line 8) +* ssl (module): ssl — TLS/SSL wrapper for socket objects. + (line 6) +* SSLCertVerificationError: Exceptions<12>. (line 79) +* SSLContext (class in ssl): SSL Contexts. (line 14) +* SSLEOFError: Exceptions<12>. (line 71) +* SSLError: Exceptions<12>. (line 6) +* SSLErrorNumber (class in ssl): Constants<9>. (line 511) +* SSLKEYLOGFILE: Context creation. (line 33) +* SSLKEYLOGFILE <1>: Context creation. (line 67) +* SSLObject (class in ssl): Memory BIO Support<2>. + (line 30) +* sslobject_class (ssl.SSLContext attribute): SSL Contexts. (line 471) +* SSLSession (class in ssl): SSL session. (line 8) +* SSLSocket (class in ssl): SSL Sockets. (line 6) +* sslsocket_class (ssl.SSLContext attribute): SSL Contexts. (line 445) +* SSLSyscallError: Exceptions<12>. (line 62) +* SSLv3 (ssl.TLSVersion attribute): Constants<9>. (line 533) +* SSLWantReadError: Exceptions<12>. (line 44) +* SSLWantWriteError: Exceptions<12>. (line 53) +* SSLZeroReturnError: Exceptions<12>. (line 35) +* SSL_CERT_FILE: LibreSSL support. (line 15) +* SSL_CERT_PATH: LibreSSL support. (line 15) +* ssl_version (ftplib.FTP_TLS attribute): FTP_TLS Objects. (line 9) +* st() (in module turtle): Visibility. (line 15) +* st2list() (in module parser): Converting ST Objects. + (line 11) +* st2tuple() (in module parser): Converting ST Objects. + (line 31) +* stack (traceback.TracebackException attribute): TracebackException Objects. + (line 34) +* stack viewer: Debug menu Shell window only. + (line 15) +* stack() (in module inspect): The interpreter stack. + (line 84) +* stack; trace: The standard type hierarchy. + (line 757) +* stackable; streams: codecs — Codec registry and base classes. + (line 8) +* StackSummary (class in traceback): StackSummary Objects. + (line 11) +* stack_effect() (in module dis): Analysis functions. (line 141) +* stack_size() (in module threading): threading — Thread-based parallelism. + (line 137) +* stack_size() (in module _thread): _thread — Low-level threading API. + (line 94) +* stamp() (in module turtle): Turtle motion. (line 238) +* Standard C: String and Bytes literals. + (line 72) +* standard input: Complete Python programs. + (line 25) +* standard; output: Expression statements. + (line 17) +* standarderror (2to3 fixer): Fixers. (line 293) +* standard_b64decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 79) +* standard_b64encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 74) +* standend() (curses.window method): Window Objects. (line 516) +* standout() (curses.window method): Window Objects. (line 521) +* STAR (in module token): token — Constants used with Python parse trees. + (line 90) +* STAREQUAL (in module token): token — Constants used with Python parse trees. + (line 178) +* starmap() (in module itertools): Itertool functions. (line 518) +* starmap() (multiprocessing.pool.Pool method): Process Pools. + (line 148) +* starmap_async() (multiprocessing.pool.Pool method): Process Pools. + (line 159) +* start (range attribute): Ranges. (line 60) +* start (slice object attribute): The standard type hierarchy. + (line 803) +* start (slice object attribute) <1>: Slicings. (line 26) +* start (UnicodeError attribute): Concrete exceptions. + (line 354) +* start() (in module tracemalloc): Functions<9>. (line 50) +* start() (logging.handlers.QueueListener method): QueueListener. + (line 71) +* start() (multiprocessing.managers.BaseManager method): Managers. + (line 40) +* start() (multiprocessing.Process method): Process and exceptions. + (line 43) +* start() (re.Match method): Match Objects. (line 132) +* start() (threading.Thread method): Thread Objects. (line 89) +* start() (tkinter.ttk.Progressbar method): ttk Progressbar. (line 8) +* start() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 43) +* StartCdataSectionHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 300) +* StartDoctypeDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 186) +* startDocument() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 32) +* startElement() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 82) +* StartElementHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 224) +* startElementNS() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 102) +* startfile() (in module os): Process Management. (line 611) +* STARTF_USESHOWWINDOW (in module subprocess): Windows Constants. + (line 33) +* STARTF_USESTDHANDLES (in module subprocess): Windows Constants. + (line 27) +* StartNamespaceDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 279) +* startPrefixMapping() (xml.sax.handler.ContentHandler method): ContentHandler Objects. + (line 49) +* startswith() (bytearray method): Bytes and Bytearray Operations. + (line 198) +* startswith() (bytes method): Bytes and Bytearray Operations. + (line 198) +* startswith() (str method): String Methods<2>. (line 507) +* startTest() (unittest.TestResult method): Loading and running tests. + (line 331) +* startTestRun() (unittest.TestResult method): Loading and running tests. + (line 340) +* starttls() (imaplib.IMAP4 method): IMAP4 Objects. (line 290) +* starttls() (nntplib.NNTP method): Methods<3>. (line 62) +* starttls() (smtplib.SMTP method): SMTP Objects. (line 185) +* STARTUPINFO (class in subprocess): Windows Popen Helpers. + (line 9) +* start_color() (in module curses): Functions<3>. (line 445) +* start_component() (msilib.Directory method): Directory Objects. + (line 20) +* start_new_thread() (in module _thread): _thread — Low-level threading API. + (line 32) +* start_ns() (xml.etree.ElementTree.TreeBuilder method): TreeBuilder Objects. + (line 75) +* start_server() (in module asyncio): Streams. (line 65) +* start_serving() (asyncio.Server method): Server Objects. (line 47) +* start_threads() (in module test.support): test support — Utilities for the Python test suite. + (line 526) +* start_tls() (asyncio.loop method): TLS Upgrade. (line 6) +* start_unix_server() (in module asyncio): Streams. (line 118) +* stat (module): stat — Interpreting stat results. + (line 6) +* stat() (in module os): Files and Directories. + (line 976) +* stat() (nntplib.NNTP method): Methods<3>. (line 219) +* stat() (os.DirEntry method): Files and Directories. + (line 947) +* stat() (pathlib.Path method): Methods<2>. (line 35) +* stat() (poplib.POP3 method): POP3 Objects. (line 51) +* state() (tkinter.ttk.Widget method): ttk Widget. (line 25) +* statement: Glossary. (line 1197) +* statement grouping: Indentation. (line 6) +* statement; assert: The assert statement. + (line 6) +* statement; assert <1>: Concrete exceptions. + (line 10) +* statement; assignment, annotated: Annotated assignment statements. + (line 6) +* statement; assignment, augmented: Augmented assignment statements. + (line 6) +* statement; async def: Coroutines<2>. (line 8) +* statement; async for: Coroutine function definition. + (line 26) +* statement; async with: The async for statement. + (line 43) +* statement; break: The break statement. + (line 6) +* statement; break <1>: The while statement. + (line 17) +* statement; break <2>: The for statement. (line 22) +* statement; break <3>: The try statement. (line 79) +* statement; break <4>: The try statement. (line 108) +* statement; class: Class definitions. (line 6) +* statement; continue: The continue statement. + (line 6) +* statement; continue <1>: The while statement. + (line 17) +* statement; continue <2>: The for statement. (line 22) +* statement; continue <3>: The try statement. (line 79) +* statement; continue <4>: The try statement. (line 108) +* statement; def: Function definitions. + (line 6) +* statement; del: Basic customization. + (line 56) +* statement; del <1>: The del statement<2>. + (line 6) +* statement; del <2>: Mutable Sequence Types. + (line 16) +* statement; del <3>: Mapping Types — dict. + (line 6) +* statement; except: Built-in Exceptions. + (line 6) +* statement; for: for Statements. (line 6) +* statement; for <1>: The break statement. + (line 6) +* statement; for <2>: The continue statement. + (line 6) +* statement; for <3>: The for statement. (line 6) +* statement; global: The del statement<2>. + (line 15) +* statement; global <1>: The global statement. + (line 6) +* statement; if: The if statement. (line 6) +* statement; if <1>: Truth Value Testing. + (line 6) +* statement; import: The standard type hierarchy. + (line 521) +* statement; import <1>: The import statement. + (line 6) +* statement; import <2>: Built-in Functions. (line 1775) +* statement; import <3>: site — Site-specific configuration hook. + (line 45) +* statement; import <4>: imp — Access the import internals. + (line 11) +* statement; nonlocal: The nonlocal statement. + (line 6) +* statement; pass: The pass statement. (line 6) +* statement; raise: The raise statement. + (line 6) +* statement; raise <1>: Built-in Exceptions. + (line 14) +* statement; return: The return statement. + (line 6) +* statement; return <1>: The try statement. (line 79) +* statement; return <2>: The try statement. (line 108) +* statement; try: The standard type hierarchy. + (line 780) +* statement; try <1>: The try statement. (line 6) +* statement; try <2>: Built-in Exceptions. + (line 6) +* statement; while: The break statement. + (line 6) +* statement; while <1>: The continue statement. + (line 6) +* statement; while <2>: The while statement. + (line 6) +* statement; while <3>: Truth Value Testing. + (line 6) +* statement; with: With Statement Context Managers. + (line 14) +* statement; with <1>: The with statement. (line 6) +* statement; yield: The yield statement. + (line 6) +* staticmethod() (built-in function): Built-in Functions. (line 1525) +* Statistic (class in tracemalloc): Statistic. (line 6) +* StatisticDiff (class in tracemalloc): StatisticDiff. (line 6) +* statistics (module): statistics — Mathematical statistics functions. + (line 6) +* statistics() (tracemalloc.Snapshot method): Snapshot. (line 56) +* StatisticsError: Exceptions<3>. (line 8) +* Stats (class in pstats): The Stats Class. (line 9) +* status (http.client.HTTPResponse attribute): HTTPResponse Objects. + (line 51) +* status() (imaplib.IMAP4 method): IMAP4 Objects. (line 303) +* statvfs() (in module os): Files and Directories. + (line 1236) +* stat_result (class in os): Files and Directories. + (line 1031) +* StdButtonBox (class in tkinter.tix): Basic Widgets. (line 65) +* stderr (asyncio.asyncio.subprocess.Process attribute): Interacting with Subprocesses. + (line 120) +* stderr (in module sys): The standard type hierarchy. + (line 630) +* stderr (in module sys) <1>: sys — System-specific parameters and functions. + (line 1463) +* stderr (in module sys) <2>: Sub-interpreter support. + (line 26) +* stderr (subprocess.CalledProcessError attribute): Using the subprocess Module. + (line 214) +* stderr (subprocess.CompletedProcess attribute): Using the subprocess Module. + (line 119) +* stderr (subprocess.Popen attribute): Popen Objects. (line 123) +* stderr (subprocess.TimeoutExpired attribute): Using the subprocess Module. + (line 181) +* stdev (statistics.NormalDist attribute): NormalDist objects. + (line 35) +* stdev() (in module statistics): Function details. (line 335) +* stdin (asyncio.asyncio.subprocess.Process attribute): Interacting with Subprocesses. + (line 110) +* stdin (in module sys): The standard type hierarchy. + (line 630) +* stdin (in module sys) <1>: sys — System-specific parameters and functions. + (line 1463) +* stdin (in module sys) <2>: Sub-interpreter support. + (line 26) +* stdin (subprocess.Popen attribute): Popen Objects. (line 104) +* stdin; stdout; sdterr: Process-wide parameters. + (line 9) +* stdio: The standard type hierarchy. + (line 630) +* stdout (asyncio.asyncio.subprocess.Process attribute): Interacting with Subprocesses. + (line 115) +* STDOUT (in module subprocess): Using the subprocess Module. + (line 147) +* stdout (in module sys): The standard type hierarchy. + (line 630) +* stdout (in module sys) <1>: sys — System-specific parameters and functions. + (line 1463) +* stdout (in module sys) <2>: Sub-interpreter support. + (line 26) +* stdout (subprocess.CalledProcessError attribute): Using the subprocess Module. + (line 210) +* stdout (subprocess.CompletedProcess attribute): Using the subprocess Module. + (line 109) +* stdout (subprocess.Popen attribute): Popen Objects. (line 113) +* stdout (subprocess.TimeoutExpired attribute): Using the subprocess Module. + (line 177) +* STD_ERROR_HANDLE (in module subprocess): Windows Constants. + (line 18) +* STD_INPUT_HANDLE (in module subprocess): Windows Constants. + (line 8) +* STD_OUTPUT_HANDLE (in module subprocess): Windows Constants. + (line 13) +* step (pdb command): Debugger Commands. (line 162) +* step (range attribute): Ranges. (line 69) +* step (slice object attribute): The standard type hierarchy. + (line 803) +* step (slice object attribute) <1>: Slicings. (line 26) +* step() (tkinter.ttk.Progressbar method): ttk Progressbar. (line 15) +* stereocontrols() (ossaudiodev.oss_mixer_device method): Mixer Device Objects. + (line 44) +* stls() (poplib.POP3 method): POP3 Objects. (line 113) +* stop (range attribute): Ranges. (line 65) +* stop (slice object attribute): The standard type hierarchy. + (line 803) +* stop (slice object attribute) <1>: Slicings. (line 26) +* stop() (asyncio.loop method): Running and stopping the loop. + (line 31) +* stop() (in module tracemalloc): Functions<9>. (line 76) +* stop() (logging.handlers.QueueListener method): QueueListener. + (line 78) +* stop() (tkinter.ttk.Progressbar method): ttk Progressbar. (line 21) +* stop() (unittest.TestResult method): Loading and running tests. + (line 311) +* StopAsyncIteration: Concrete exceptions. + (line 240) +* StopIteration: Concrete exceptions. + (line 210) +* stopListening() (in module logging.config): Configuration functions. + (line 164) +* stopTest() (unittest.TestResult method): Loading and running tests. + (line 335) +* stopTestRun() (unittest.TestResult method): Loading and running tests. + (line 346) +* stop_here() (bdb.Bdb method): bdb — Debugger framework. + (line 191) +* storbinary() (ftplib.FTP method): FTP Objects. (line 112) +* store() (imaplib.IMAP4 method): IMAP4 Objects. (line 307) +* STORE_ACTIONS (optparse.Option attribute): Adding new actions. + (line 35) +* STORE_ATTR (opcode): Python Bytecode Instructions. + (line 510) +* STORE_DEREF (opcode): Python Bytecode Instructions. + (line 745) +* STORE_FAST (opcode): Python Bytecode Instructions. + (line 716) +* STORE_GLOBAL (opcode): Python Bytecode Instructions. + (line 519) +* STORE_NAME (opcode): Python Bytecode Instructions. + (line 483) +* STORE_SUBSCR (opcode): Python Bytecode Instructions. + (line 250) +* storlines() (ftplib.FTP method): FTP Objects. (line 126) +* str (built-in class): Text Sequence Type — str. + (line 44) +* str (built-in class); (see also string): Ranges. (line 125) +* str() (in module locale): locale — Internationalization services. + (line 435) +* strcoll() (in module locale): locale — Internationalization services. + (line 375) +* StreamError: tarfile — Read and write tar archive files. + (line 196) +* StreamHandler (class in logging): StreamHandler. (line 11) +* StreamReader (class in asyncio): StreamReader. (line 6) +* StreamReader (class in codecs): StreamReader Objects. + (line 10) +* streamreader (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 87) +* StreamReaderWriter (class in codecs): StreamReaderWriter Objects. + (line 12) +* StreamRecoder (class in codecs): StreamRecoder Objects. + (line 13) +* StreamRequestHandler (class in socketserver): Request Handler Objects. + (line 42) +* streams: codecs — Codec registry and base classes. + (line 8) +* StreamWriter (class in asyncio): StreamWriter. (line 6) +* StreamWriter (class in codecs): StreamWriter Objects. + (line 10) +* streamwriter (codecs.CodecInfo attribute): codecs — Codec registry and base classes. + (line 87) +* strerror (OSError attribute): Concrete exceptions. + (line 151) +* strerror(): Raising exceptions. (line 59) +* strerror() (in module os): Process Parameters. (line 452) +* strftime() (datetime.date method): date Objects. (line 267) +* strftime() (datetime.datetime method): datetime Objects. (line 700) +* strftime() (datetime.time method): time Objects. (line 209) +* strftime() (in module time): Functions<2>. (line 230) +* strict (csv.Dialect attribute): Dialects and Formatting Parameters. + (line 70) +* strict (in module email.policy): email policy Policy Objects. + (line 514) +* strict_domain (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 95) +* strict_errors() (in module codecs): Error Handlers. (line 128) +* strict_ns_domain (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 117) +* strict_ns_set_initial_dollar (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 122) +* strict_ns_set_path (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 127) +* strict_ns_unverifiable (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 112) +* strict_rfc2965_unverifiable (http.cookiejar.DefaultCookiePolicy attribute): DefaultCookiePolicy Objects. + (line 103) +* strides (memoryview attribute): Memory Views. (line 454) +* STRING (in module token): token — Constants used with Python parse trees. + (line 46) +* string (module): string — Common string operations. + (line 6) +* string (re.Match attribute): Match Objects. (line 196) +* string literal: Literals. (line 8) +* string; conversion: Basic customization. + (line 150) +* string; conversion <1>: Expression statements. + (line 17) +* string; format() (built-in function): Built-in Functions. (line 643) +* string; formatted literal: String literal concatenation. + (line 24) +* string; formatting, printf: printf-style String Formatting. + (line 6) +* string; immutable sequences: The standard type hierarchy. + (line 154) +* string; interpolated literal: String literal concatenation. + (line 24) +* string; interpolation, printf: printf-style String Formatting. + (line 6) +* string; item: Subscriptions. (line 38) +* string; methods: Text Sequence Type — str. + (line 85) +* string; object representation: Finalization and De-allocation. + (line 79) +* string; PyObject_Str (C function): Object Protocol. (line 186) +* string; str (built-in class): Text Sequence Type — str. + (line 44) +* string; str() (built-in function): Built-in Functions. (line 1558) +* string; text sequence type: Ranges. (line 125) +* string; __format__() (object method): Basic customization. + (line 150) +* string; __str__() (object method): Basic customization. + (line 129) +* StringIO (class in io): Text I/O<2>. (line 193) +* stringprep (module): stringprep — Internet String Preparation. + (line 6) +* strings, documentation: Defining Functions. (line 21) +* strings, documentation <1>: Documentation Strings. + (line 6) +* string_at() (in module ctypes): Utility functions. (line 194) +* strip() (bytearray method): Bytes and Bytearray Operations. + (line 375) +* strip() (bytes method): Bytes and Bytearray Operations. + (line 375) +* strip() (str method): String Methods<2>. (line 514) +* stripspaces (curses.textpad.Textbox attribute): Textbox objects. + (line 115) +* strip_dirs() (pstats.Stats method): The Stats Class. (line 34) +* strip_python_strerr() (in module test.support): test support — Utilities for the Python test suite. + (line 386) +* strptime() (datetime.datetime class method): datetime Objects. + (line 231) +* strptime() (in module time): Functions<2>. (line 362) +* strsignal() (in module signal): Module contents<2>. (line 249) +* strtobool() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 119) +* Struct (class in struct): Classes<2>. (line 8) +* struct (module): struct — Interpret bytes as packed binary data. + (line 6) +* Structure (class in ctypes): Structured data types. + (line 21) +* struct_time (class in time): Functions<2>. (line 396) +* strxfrm() (in module locale): locale — Internationalization services. + (line 382) +* STType (in module parser): ST Objects. (line 10) +* Style (class in tkinter.ttk): Ttk Styling. (line 20) +* ST_ATIME (in module stat): stat — Interpreting stat results. + (line 163) +* st_atime (os.stat_result attribute): Files and Directories. + (line 1076) +* st_atime_ns (os.stat_result attribute): Files and Directories. + (line 1092) +* st_birthtime (os.stat_result attribute): Files and Directories. + (line 1160) +* st_blksize (os.stat_result attribute): Files and Directories. + (line 1138) +* st_blocks (os.stat_result attribute): Files and Directories. + (line 1133) +* st_creator (os.stat_result attribute): Files and Directories. + (line 1178) +* ST_CTIME (in module stat): stat — Interpreting stat results. + (line 171) +* st_ctime (os.stat_result attribute): Files and Directories. + (line 1084) +* st_ctime_ns (os.stat_result attribute): Files and Directories. + (line 1102) +* ST_DEV (in module stat): stat — Interpreting stat results. + (line 142) +* st_dev (os.stat_result attribute): Files and Directories. + (line 1052) +* st_file_attributes (os.stat_result attribute): Files and Directories. + (line 1188) +* st_flags (os.stat_result attribute): Files and Directories. + (line 1148) +* st_fstype (os.stat_result attribute): Files and Directories. + (line 1167) +* st_gen (os.stat_result attribute): Files and Directories. + (line 1156) +* ST_GID (in module stat): stat — Interpreting stat results. + (line 154) +* st_gid (os.stat_result attribute): Files and Directories. + (line 1064) +* ST_INO (in module stat): stat — Interpreting stat results. + (line 138) +* st_ino (os.stat_result attribute): Files and Directories. + (line 1043) +* ST_MODE (in module stat): stat — Interpreting stat results. + (line 134) +* st_mode (os.stat_result attribute): Files and Directories. + (line 1039) +* ST_MTIME (in module stat): stat — Interpreting stat results. + (line 167) +* st_mtime (os.stat_result attribute): Files and Directories. + (line 1080) +* st_mtime_ns (os.stat_result attribute): Files and Directories. + (line 1097) +* ST_NLINK (in module stat): stat — Interpreting stat results. + (line 146) +* st_nlink (os.stat_result attribute): Files and Directories. + (line 1056) +* st_rdev (os.stat_result attribute): Files and Directories. + (line 1144) +* st_reparse_tag (os.stat_result attribute): Files and Directories. + (line 1195) +* st_rsize (os.stat_result attribute): Files and Directories. + (line 1174) +* ST_SIZE (in module stat): stat — Interpreting stat results. + (line 158) +* st_size (os.stat_result attribute): Files and Directories. + (line 1068) +* st_type (os.stat_result attribute): Files and Directories. + (line 1182) +* ST_UID (in module stat): stat — Interpreting stat results. + (line 150) +* st_uid (os.stat_result attribute): Files and Directories. + (line 1060) +* sub() (in module operator): operator — Standard operators as functions. + (line 153) +* sub() (in module re): Module Contents. (line 241) +* sub() (re.Pattern method): Regular Expression Objects. + (line 91) +* subclassing; immutable types: Basic customization. + (line 8) +* subdirs (filecmp.dircmp attribute): The dircmp class. (line 100) +* SubElement() (in module xml.etree.ElementTree): Functions<6>. + (line 192) +* submit() (concurrent.futures.Executor method): Executor Objects. + (line 12) +* submodule_search_locations (importlib.machinery.ModuleSpec attribute): importlib machinery – Importers and path hooks. + (line 391) +* subn() (in module re): Module Contents. (line 306) +* subn() (re.Pattern method): Regular Expression Objects. + (line 96) +* subnets() (ipaddress.IPv4Network method): Network objects. (line 170) +* subnets() (ipaddress.IPv6Network method): Network objects. (line 340) +* subnet_of() (ipaddress.IPv4Network method): Network objects. + (line 212) +* subnet_of() (ipaddress.IPv6Network method): Network objects. + (line 344) +* Subnormal (class in decimal): Signals. (line 86) +* suboffsets (memoryview attribute): Memory Views. (line 463) +* subpad() (curses.window method): Window Objects. (line 525) +* subprocess (module): subprocess — Subprocess management. + (line 6) +* SubprocessError: Using the subprocess Module. + (line 153) +* SubprocessProtocol (class in asyncio): Base Protocols. (line 24) +* SubprocessTransport (class in asyncio): Transports Hierarchy. + (line 50) +* subprocess_exec() (asyncio.loop method): Running Subprocesses. + (line 15) +* subprocess_shell() (asyncio.loop method): Running Subprocesses. + (line 112) +* subscribe() (imaplib.IMAP4 method): IMAP4 Objects. (line 331) +* subscript; assignment: Mutable Sequence Types. + (line 16) +* subscript; operation: Common Sequence Operations. + (line 21) +* subscription: The standard type hierarchy. + (line 127) +* subscription <1>: The standard type hierarchy. + (line 192) +* subscription <2>: The standard type hierarchy. + (line 249) +* subscription <3>: Subscriptions. (line 6) +* subscription; assignment: Assignment statements. + (line 103) +* subsequent_indent (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 205) +* substitute() (string.Template method): Template strings. (line 40) +* subst_vars() (in module distutils.util): distutils util — Miscellaneous other utility functions. + (line 82) +* subTest() (unittest.TestCase method): Test cases. (line 105) +* subtract() (collections.Counter method): Counter objects. (line 88) +* subtract() (decimal.Context method): Context objects. (line 502) +* subtraction: Binary arithmetic operations. + (line 66) +* subtype (email.headerregistry.ContentTypeHeader attribute): email headerregistry Custom Header Objects. + (line 278) +* subwin() (curses.window method): Window Objects. (line 532) +* sub_commands (distutils.cmd.Command attribute): Creating a new Distutils command. + (line 49) +* successful() (multiprocessing.pool.AsyncResult method): Process Pools. + (line 213) +* suffix_map (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 128) +* suffix_map (mimetypes.MimeTypes attribute): MimeTypes Objects. + (line 23) +* suite: Compound statements. + (line 18) +* suite() (in module parser): Creating ST Objects. + (line 18) +* suiteClass (unittest.TestLoader attribute): Loading and running tests. + (line 192) +* sum() (built-in function): Built-in Functions. (line 1568) +* summarize() (doctest.DocTestRunner method): DocTestRunner objects. + (line 116) +* summarize_address_range() (in module ipaddress): Other Module Level Functions. + (line 27) +* sum_list(): Reference Count Details. + (line 143) +* sum_sequence(): Reference Count Details. + (line 170) +* sum_sequence() <1>: Exceptions<18>. (line 65) +* sunau (module): sunau — Read and write Sun AU files. + (line 6) +* super (pyclbr.Class attribute): Class Objects<2>. (line 38) +* super() (built-in function): Built-in Functions. (line 1584) +* supernet() (ipaddress.IPv4Network method): Network objects. + (line 196) +* supernet() (ipaddress.IPv6Network method): Network objects. + (line 342) +* supernet_of() (ipaddress.IPv4Network method): Network objects. + (line 223) +* supernet_of() (ipaddress.IPv6Network method): Network objects. + (line 346) +* SupportsAbs (class in typing): Classes functions and decorators. + (line 181) +* SupportsBytes (class in typing): Classes functions and decorators. + (line 171) +* SupportsComplex (class in typing): Classes functions and decorators. + (line 167) +* SupportsFloat (class in typing): Classes functions and decorators. + (line 163) +* SupportsIndex (class in typing): Classes functions and decorators. + (line 175) +* SupportsInt (class in typing): Classes functions and decorators. + (line 159) +* SupportsRound (class in typing): Classes functions and decorators. + (line 186) +* supports_bytes_environ (in module os): Process Parameters. (line 458) +* supports_dir_fd (in module os): Files and Directories. + (line 1279) +* supports_effective_ids (in module os): Files and Directories. + (line 1304) +* supports_fd (in module os): Files and Directories. + (line 1323) +* supports_follow_symlinks (in module os): Files and Directories. + (line 1342) +* supports_unicode_filenames (in module os.path): os path — Common pathname manipulations. + (line 409) +* suppress() (in module contextlib): Utilities. (line 178) +* SuppressCrashReport (class in test.support): test support — Utilities for the Python test suite. + (line 1045) +* swapcase() (bytearray method): Bytes and Bytearray Operations. + (line 596) +* swapcase() (bytes method): Bytes and Bytearray Operations. + (line 596) +* swapcase() (str method): String Methods<2>. (line 538) +* swap_attr() (in module test.support): test support — Utilities for the Python test suite. + (line 487) +* swap_item() (in module test.support): test support — Utilities for the Python test suite. + (line 504) +* SW_HIDE (in module subprocess): Windows Constants. (line 23) +* Symbol (class in symtable): Examining Symbol Tables. + (line 103) +* symbol (module): symbol — Constants used with Python parse trees. + (line 6) +* SymbolTable (class in symtable): Examining Symbol Tables. + (line 6) +* symlink() (in module os): Files and Directories. + (line 1365) +* symlink_to() (pathlib.Path method): Methods<2>. (line 390) +* symmetric_difference() (frozenset method): Set Types — set frozenset. + (line 116) +* symmetric_difference_update() (frozenset method): Set Types — set frozenset. + (line 188) +* symtable (module): symtable — Access to the compiler’s symbol tables. + (line 6) +* symtable() (in module symtable): Generating Symbol Tables. + (line 6) +* sym_name (in module symbol): symbol — Constants used with Python parse trees. + (line 19) +* sync() (dbm.dumb.dumbdbm method): dbm dumb — Portable DBM implementation. + (line 78) +* sync() (dbm.gnu.gdbm method): dbm gnu — GNU’s reinterpretation of dbm. + (line 113) +* sync() (in module os): Files and Directories. + (line 1407) +* sync() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 165) +* sync() (shelve.Shelf method): shelve — Python object persistence. + (line 63) +* syncdown() (curses.window method): Window Objects. (line 542) +* synchronized() (in module multiprocessing.sharedctypes): The multiprocessing sharedctypes module. + (line 88) +* SyncManager (class in multiprocessing.managers): Managers. (line 133) +* syncok() (curses.window method): Window Objects. (line 548) +* syncup() (curses.window method): Window Objects. (line 553) +* syntax: Notation. (line 6) +* SyntaxErr: Exceptions<16>. (line 93) +* SyntaxError: Concrete exceptions. + (line 247) +* SyntaxWarning: Warnings. (line 32) +* sys (module): sys — System-specific parameters and functions. + (line 6) +* sys.exc_info: The standard type hierarchy. + (line 757) +* sys.last_traceback: The standard type hierarchy. + (line 757) +* sys.meta_path: The meta path. (line 6) +* sys.modules: The module cache. (line 6) +* sys.path: Path entry finders. (line 6) +* sys.path_hooks: Path entry finders. (line 6) +* sys.path_importer_cache: Path entry finders. (line 6) +* sys.stderr: The standard type hierarchy. + (line 630) +* sys.stdin: The standard type hierarchy. + (line 630) +* sys.stdout: The standard type hierarchy. + (line 630) +* sysconf() (in module os): Miscellaneous System Information. + (line 56) +* sysconfig (module): sysconfig — Provide access to Python’s configuration information. + (line 6) +* sysconf_names (in module os): Miscellaneous System Information. + (line 66) +* syslog (module): syslog — Unix syslog library routines. + (line 6) +* syslog() (in module syslog): syslog — Unix syslog library routines. + (line 18) +* SysLogHandler (class in logging.handlers): SysLogHandler. (line 10) +* system() (in module os): Process Management. (line 647) +* system() (in module platform): Cross Platform. (line 120) +* SystemError: Concrete exceptions. + (line 268) +* SystemError (built-in exception): Module Objects. (line 55) +* SystemError (built-in exception) <1>: Module Objects. (line 81) +* SystemExit: Concrete exceptions. + (line 282) +* SystemExit (built-in exception): Exceptions<2>. (line 26) +* systemId (xml.dom.DocumentType attribute): DocumentType Objects. + (line 22) +* SystemRandom (class in random): Alternative Generator. + (line 11) +* SystemRandom (class in secrets): Random numbers. (line 9) +* SystemRoot: Popen Constructor. (line 234) +* system_alias() (in module platform): Cross Platform. (line 126) +* system_must_validate_cert() (in module test.support): test support — Utilities for the Python test suite. + (line 219) +* sys_exc (2to3 fixer): Fixers. (line 297) +* sys_version (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 134) +* S_ENFMT (in module stat): stat — Interpreting stat results. + (line 317) +* S_IEXEC (in module stat): stat — Interpreting stat results. + (line 331) +* S_IFBLK (in module stat): stat — Interpreting stat results. + (line 206) +* S_IFCHR (in module stat): stat — Interpreting stat results. + (line 214) +* S_IFDIR (in module stat): stat — Interpreting stat results. + (line 210) +* S_IFDOOR (in module stat): stat — Interpreting stat results. + (line 222) +* S_IFIFO (in module stat): stat — Interpreting stat results. + (line 218) +* S_IFLNK (in module stat): stat — Interpreting stat results. + (line 198) +* S_IFMT() (in module stat): stat — Interpreting stat results. + (line 78) +* S_IFPORT (in module stat): stat — Interpreting stat results. + (line 228) +* S_IFREG (in module stat): stat — Interpreting stat results. + (line 202) +* S_IFSOCK (in module stat): stat — Interpreting stat results. + (line 194) +* S_IFWHT (in module stat): stat — Interpreting stat results. + (line 234) +* S_IMODE() (in module stat): stat — Interpreting stat results. + (line 71) +* S_IREAD (in module stat): stat — Interpreting stat results. + (line 323) +* S_IRGRP (in module stat): stat — Interpreting stat results. + (line 289) +* S_IROTH (in module stat): stat — Interpreting stat results. + (line 305) +* S_IRUSR (in module stat): stat — Interpreting stat results. + (line 273) +* S_IRWXG (in module stat): stat — Interpreting stat results. + (line 285) +* S_IRWXO (in module stat): stat — Interpreting stat results. + (line 301) +* S_IRWXU (in module stat): stat — Interpreting stat results. + (line 269) +* S_ISBLK() (in module stat): stat — Interpreting stat results. + (line 30) +* S_ISCHR() (in module stat): stat — Interpreting stat results. + (line 25) +* S_ISDIR() (in module stat): stat — Interpreting stat results. + (line 21) +* S_ISDOOR() (in module stat): stat — Interpreting stat results. + (line 50) +* S_ISFIFO() (in module stat): stat — Interpreting stat results. + (line 38) +* S_ISGID (in module stat): stat — Interpreting stat results. + (line 251) +* S_ISLNK() (in module stat): stat — Interpreting stat results. + (line 42) +* S_ISPORT() (in module stat): stat — Interpreting stat results. + (line 56) +* S_ISREG() (in module stat): stat — Interpreting stat results. + (line 34) +* S_ISSOCK() (in module stat): stat — Interpreting stat results. + (line 46) +* S_ISUID (in module stat): stat — Interpreting stat results. + (line 247) +* S_ISVTX (in module stat): stat — Interpreting stat results. + (line 262) +* S_ISWHT() (in module stat): stat — Interpreting stat results. + (line 62) +* S_IWGRP (in module stat): stat — Interpreting stat results. + (line 293) +* S_IWOTH (in module stat): stat — Interpreting stat results. + (line 309) +* S_IWRITE (in module stat): stat — Interpreting stat results. + (line 327) +* S_IWUSR (in module stat): stat — Interpreting stat results. + (line 277) +* S_IXGRP (in module stat): stat — Interpreting stat results. + (line 297) +* S_IXOTH (in module stat): stat — Interpreting stat results. + (line 313) +* S_IXUSR (in module stat): stat — Interpreting stat results. + (line 281) +* tab: Indentation. (line 6) +* tab() (tkinter.ttk.Notebook method): ttk Notebook. (line 62) +* TabError: Concrete exceptions. + (line 263) +* tabnanny (module): tabnanny — Detection of ambiguous indentation. + (line 6) +* tabs() (tkinter.ttk.Notebook method): ttk Notebook. (line 71) +* tabsize (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 162) +* tag (xml.etree.ElementTree.Element attribute): Element Objects. + (line 16) +* tagName (xml.dom.Element attribute): Element Objects<2>. (line 9) +* tag_bind() (tkinter.ttk.Treeview method): ttk Treeview. (line 303) +* tag_configure() (tkinter.ttk.Treeview method): ttk Treeview. + (line 309) +* tag_has() (tkinter.ttk.Treeview method): ttk Treeview. (line 319) +* tail (xml.etree.ElementTree.Element attribute): Element Objects. + (line 21) +* takewhile() (in module itertools): Itertool functions. (line 533) +* take_snapshot() (in module tracemalloc): Functions<9>. (line 88) +* tan() (in module cmath): Trigonometric functions<2>. + (line 33) +* tan() (in module math): Trigonometric functions. + (line 61) +* tanh() (in module cmath): Hyperbolic functions<2>. + (line 34) +* tanh() (in module math): Hyperbolic functions. + (line 29) +* TarError: tarfile — Read and write tar archive files. + (line 182) +* TarFile (class in tarfile): TarFile Objects. (line 22) +* tarfile (module): tarfile — Read and write tar archive files. + (line 6) +* tarfile command line option; -c ... : Command-line options<2>. + (line 11) +* tarfile command line option; –create ... : Command-line options<2>. + (line 11) +* tarfile command line option; -e []: Command-line options<2>. + (line 16) +* tarfile command line option; –extract []: Command-line options<2>. + (line 16) +* tarfile command line option; -l : Command-line options<2>. + (line 6) +* tarfile command line option; –list : Command-line options<2>. + (line 6) +* tarfile command line option; -t : Command-line options<2>. + (line 22) +* tarfile command line option; –test : Command-line options<2>. + (line 22) +* tarfile command line option; -v: Command-line options<2>. + (line 27) +* tarfile command line option; –verbose: Command-line options<2>. + (line 27) +* target: Assignment statements. + (line 27) +* target (xml.dom.ProcessingInstruction attribute): ProcessingInstruction Objects. + (line 9) +* target; list: Assignment statements. + (line 27) +* target; list <1>: The for statement. (line 6) +* target; list; assignment: Assignment statements. + (line 35) +* TarInfo (class in tarfile): TarInfo Objects. (line 15) +* Task (class in asyncio): Task Object. (line 6) +* task_done() (asyncio.Queue method): Queue. (line 78) +* task_done() (multiprocessing.JoinableQueue method): Pipes and Queues. + (line 227) +* task_done() (queue.Queue method): Queue Objects. (line 68) +* tau (in module cmath): Constants<3>. (line 14) +* tau (in module math): Constants<2>. (line 14) +* tbreak (pdb command): Debugger Commands. (line 88) +* tb_frame (traceback attribute): The standard type hierarchy. + (line 780) +* tb_lasti (traceback attribute): The standard type hierarchy. + (line 780) +* tb_lineno (traceback attribute): The standard type hierarchy. + (line 780) +* tb_locals (unittest.TestResult attribute): Loading and running tests. + (line 295) +* tb_next (traceback attribute): The standard type hierarchy. + (line 789) +* tcdrain() (in module termios): termios — POSIX style tty control. + (line 53) +* tcflow() (in module termios): termios — POSIX style tty control. + (line 64) +* tcflush() (in module termios): termios — POSIX style tty control. + (line 58) +* tcgetattr() (in module termios): termios — POSIX style tty control. + (line 26) +* tcgetpgrp() (in module os): File Descriptor Operations. + (line 653) +* Tcl() (in module tkinter): Tkinter Modules. (line 33) +* TCL_LIBRARY: How do I freeze Tkinter applications?. + (line 11) +* TCPServer (class in socketserver): socketserver — A framework for network servers. + (line 15) +* tcsendbreak() (in module termios): termios — POSIX style tty control. + (line 47) +* tcsetattr() (in module termios): termios — POSIX style tty control. + (line 37) +* tcsetpgrp() (in module os): File Descriptor Operations. + (line 660) +* tearDown() (unittest.TestCase method): Test cases. (line 41) +* tearDownClass() (unittest.TestCase method): Test cases. (line 69) +* tee() (in module itertools): Itertool functions. (line 546) +* tell() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 112) +* tell() (chunk.Chunk method): chunk — Read IFF chunked data. + (line 101) +* tell() (io.IOBase method): I/O Base Classes. (line 135) +* tell() (io.TextIOBase method): Text I/O<2>. (line 88) +* tell() (mmap.mmap method): mmap — Memory-mapped file support. + (line 274) +* tell() (sunau.AU_read method): AU_read Objects. (line 65) +* tell() (sunau.AU_write method): AU_write Objects. (line 39) +* tell() (wave.Wave_read method): Wave_read Objects. (line 73) +* tell() (wave.Wave_write method): Wave_write Objects. (line 62) +* Telnet (class in telnetlib): telnetlib — Telnet client. + (line 25) +* telnetlib (module): telnetlib — Telnet client. + (line 6) +* TEMP: tempfile — Generate temporary files and directories. + (line 239) +* tempdir (in module tempfile): tempfile — Generate temporary files and directories. + (line 281) +* tempfile (module): tempfile — Generate temporary files and directories. + (line 6) +* Template (class in pipes): pipes — Interface to shell pipelines. + (line 19) +* Template (class in string): Template strings. (line 35) +* template (string.Template attribute): Template strings. (line 70) +* temporary; file: tempfile — Generate temporary files and directories. + (line 8) +* temporary; file name: tempfile — Generate temporary files and directories. + (line 8) +* TemporaryDirectory() (in module tempfile): tempfile — Generate temporary files and directories. + (line 119) +* TemporaryFile() (in module tempfile): tempfile — Generate temporary files and directories. + (line 28) +* temp_cwd() (in module test.support): test support — Utilities for the Python test suite. + (line 445) +* temp_dir() (in module test.support): test support — Utilities for the Python test suite. + (line 426) +* temp_umask() (in module test.support): test support — Utilities for the Python test suite. + (line 459) +* teredo (ipaddress.IPv6Address attribute): Address objects. (line 209) +* TERM: Functions<3>. (line 440) +* TERM <1>: Functions<3>. (line 467) +* termattrs() (in module curses): Functions<3>. (line 459) +* terminal_size (class in os): Querying the size of a terminal. + (line 25) +* terminate() (asyncio.asyncio.subprocess.Process method): Interacting with Subprocesses. + (line 90) +* terminate() (asyncio.SubprocessTransport method): Subprocess Transports. + (line 49) +* terminate() (multiprocessing.pool.Pool method): Process Pools. + (line 174) +* terminate() (multiprocessing.Process method): Process and exceptions. + (line 149) +* terminate() (subprocess.Popen method): Popen Objects. (line 83) +* termination model: Exceptions<2>. (line 20) +* termios (module): termios — POSIX style tty control. + (line 6) +* termname() (in module curses): Functions<3>. (line 465) +* ternary; operator: Conditional expressions. + (line 6) +* ternaryfunc (C type): Slot Type typedefs. (line 113) +* test (doctest.DocTestFailure attribute): Debugging. (line 195) +* test (doctest.UnexpectedException attribute): Debugging. (line 216) +* test (module): test — Regression tests package for Python. + (line 6) +* test() (in module cgi): Functions<8>. (line 46) +* test.support (module): test support — Utilities for the Python test suite. + (line 6) +* test.support.script_helper (module): test support script_helper — Utilities for the Python execution tests. + (line 6) +* TestCase (class in unittest): Test cases. (line 6) +* TestFailed: test support — Utilities for the Python test suite. + (line 16) +* testfile() (in module doctest): Basic API. (line 12) +* TESTFN (in module test.support): test support — Utilities for the Python test suite. + (line 52) +* TESTFN_ENCODING (in module test.support): test support — Utilities for the Python test suite. + (line 62) +* TESTFN_NONASCII (in module test.support): test support — Utilities for the Python test suite. + (line 78) +* TESTFN_UNDECODABLE (in module test.support): test support — Utilities for the Python test suite. + (line 72) +* TESTFN_UNENCODABLE (in module test.support): test support — Utilities for the Python test suite. + (line 66) +* TESTFN_UNICODE (in module test.support): test support — Utilities for the Python test suite. + (line 58) +* TestHandler (class in test.support): test support — Utilities for the Python test suite. + (line 1108) +* TestLoader (class in unittest): Loading and running tests. + (line 6) +* testMethodPrefix (unittest.TestLoader attribute): Loading and running tests. + (line 178) +* testmod() (in module doctest): Basic API. (line 90) +* testNamePatterns (unittest.TestLoader attribute): Loading and running tests. + (line 200) +* TestResult (class in unittest): Loading and running tests. + (line 217) +* tests (in module imghdr): imghdr — Determine the type of an image. + (line 75) +* testsource() (in module doctest): Debugging. (line 106) +* testsRun (unittest.TestResult attribute): Loading and running tests. + (line 273) +* TestSuite (class in unittest): Grouping tests. (line 6) +* testzip() (zipfile.ZipFile method): ZipFile Objects. (line 232) +* TEST_DATA_DIR (in module test.support): test support — Utilities for the Python test suite. + (line 114) +* TEST_HOME_DIR (in module test.support): test support — Utilities for the Python test suite. + (line 110) +* TEST_HTTP_URL (in module test.support): test support — Utilities for the Python test suite. + (line 141) +* TEST_SUPPORT_DIR (in module test.support): test support — Utilities for the Python test suite. + (line 105) +* Text (class in typing): Classes functions and decorators. + (line 457) +* text (in module msilib): Precomputed tables. (line 23) +* text (traceback.TracebackException attribute): TracebackException Objects. + (line 50) +* text (xml.etree.ElementTree.Element attribute): Element Objects. + (line 21) +* text encoding: Glossary. (line 1204) +* text file: Glossary. (line 1208) +* text mode: Built-in Functions. (line 1217) +* text() (in module cgitb): cgitb — Traceback manager for CGI scripts. + (line 48) +* text() (msilib.Dialog method): GUI classes. (line 56) +* Textbox (class in curses.textpad): Textbox objects. (line 8) +* TextCalendar (class in calendar): calendar — General calendar-related functions. + (line 128) +* textdomain() (in module gettext): GNU gettext API. (line 36) +* textdomain() (in module locale): Access to message catalogs. + (line 12) +* TextFile (class in distutils.text_file): distutils text_file — The TextFile class. + (line 10) +* textinput() (in module turtle): Input methods. (line 6) +* TextIO (class in typing): Classes functions and decorators. + (line 471) +* TextIOBase (class in io): Text I/O<2>. (line 6) +* TextIOWrapper (class in io): Text I/O<2>. (line 99) +* TextTestResult (class in unittest): Loading and running tests. + (line 422) +* TextTestRunner (class in unittest): Loading and running tests. + (line 437) +* textwrap (module): textwrap — Text wrapping and filling. + (line 6) +* TextWrapper (class in textwrap): textwrap — Text wrapping and filling. + (line 129) +* text_factory (sqlite3.Connection attribute): Connection Objects. + (line 336) +* theme_create() (tkinter.ttk.Style method): Ttk Styling. (line 191) +* theme_names() (tkinter.ttk.Style method): Ttk Styling. (line 239) +* theme_settings() (tkinter.ttk.Style method): Ttk Styling. (line 201) +* theme_use() (tkinter.ttk.Style method): Ttk Styling. (line 243) +* THOUSEP (in module locale): locale — Internationalization services. + (line 236) +* Thread (class in threading): Thread Objects. (line 57) +* thread() (imaplib.IMAP4 method): IMAP4 Objects. (line 335) +* ThreadedChildWatcher (class in asyncio): Process Watchers. (line 84) +* threading (module): threading — Thread-based parallelism. + (line 6) +* ThreadingHTTPServer (class in http.server): http server — HTTP servers. + (line 32) +* ThreadingMixIn (class in socketserver): Server Creation Notes. + (line 28) +* ThreadingTCPServer (class in socketserver): Server Creation Notes. + (line 62) +* ThreadingUDPServer (class in socketserver): Server Creation Notes. + (line 62) +* threading_cleanup() (in module test.support): test support — Utilities for the Python test suite. + (line 782) +* threading_setup() (in module test.support): test support — Utilities for the Python test suite. + (line 778) +* ThreadPool (class in multiprocessing.pool): The multiprocessing dummy module. + (line 16) +* ThreadPoolExecutor (class in concurrent.futures): ThreadPoolExecutor. + (line 39) +* threads; POSIX: _thread — Low-level threading API. + (line 16) +* thread_info (in module sys): sys — System-specific parameters and functions. + (line 1547) +* thread_time() (in module time): Functions<2>. (line 477) +* thread_time_ns() (in module time): Functions<2>. (line 491) +* throw (2to3 fixer): Fixers. (line 302) +* throw() (coroutine method): Coroutine Objects. (line 34) +* throw() (generator method): Generator-iterator methods. + (line 38) +* ticket_lifetime_hint (ssl.SSLSession attribute): SSL session. + (line 18) +* tigetflag() (in module curses): Functions<3>. (line 470) +* tigetnum() (in module curses): Functions<3>. (line 477) +* tigetstr() (in module curses): Functions<3>. (line 484) +* TILDE (in module token): token — Constants used with Python parse trees. + (line 150) +* tilt() (in module turtle): Appearance. (line 102) +* tiltangle() (in module turtle): Appearance. (line 137) +* time (class in datetime): time Objects. (line 10) +* time (module): time — Time access and conversions. + (line 6) +* time (ssl.SSLSession attribute): SSL session. (line 14) +* time() (asyncio.loop method): Scheduling delayed callbacks. + (line 56) +* time() (datetime.datetime method): datetime Objects. (line 395) +* time() (in module time): Functions<2>. (line 454) +* Time2Internaldate() (in module imaplib): imaplib — IMAP4 protocol client. + (line 122) +* timedelta (class in datetime): timedelta Objects. (line 9) +* TimedRotatingFileHandler (class in logging.handlers): TimedRotatingFileHandler. + (line 10) +* timegm() (in module calendar): calendar — General calendar-related functions. + (line 355) +* timeit (module): timeit — Measure execution time of small code snippets. + (line 6) +* timeit command line option; -h: Command-Line Interface<4>. + (line 44) +* timeit command line option; –help: Command-Line Interface<4>. + (line 44) +* timeit command line option; -n N: Command-Line Interface<4>. + (line 13) +* timeit command line option; –number=N: Command-Line Interface<4>. + (line 13) +* timeit command line option; -p: Command-Line Interface<4>. + (line 25) +* timeit command line option; –process: Command-Line Interface<4>. + (line 25) +* timeit command line option; -r N: Command-Line Interface<4>. + (line 17) +* timeit command line option; –repeat=N: Command-Line Interface<4>. + (line 17) +* timeit command line option; -s S: Command-Line Interface<4>. + (line 21) +* timeit command line option; –setup=S: Command-Line Interface<4>. + (line 21) +* timeit command line option; -u: Command-Line Interface<4>. + (line 33) +* timeit command line option; –unit=U: Command-Line Interface<4>. + (line 33) +* timeit command line option; -v: Command-Line Interface<4>. + (line 40) +* timeit command line option; –verbose: Command-Line Interface<4>. + (line 40) +* timeit() (in module timeit): Python Interface. (line 8) +* timeit() (timeit.Timer method): Python Interface. (line 69) +* timeout: Exceptions<11>. (line 40) +* timeout (socketserver.BaseServer attribute): Server Objects<2>. + (line 114) +* timeout (ssl.SSLSession attribute): SSL session. (line 16) +* timeout (subprocess.TimeoutExpired attribute): Using the subprocess Module. + (line 168) +* timeout() (curses.window method): Window Objects. (line 558) +* TimeoutError: OS exceptions. (line 103) +* TimeoutError <1>: Process and exceptions. + (line 220) +* TimeoutError <2>: Exception classes. (line 10) +* TimeoutError <3>: Exceptions<9>. (line 10) +* TimeoutExpired: Using the subprocess Module. + (line 159) +* TIMEOUT_MAX (in module threading): threading — Thread-based parallelism. + (line 161) +* TIMEOUT_MAX (in module _thread): _thread — Low-level threading API. + (line 116) +* Timer (class in threading): Timer Objects. (line 25) +* Timer (class in timeit): Python Interface. (line 39) +* TimerHandle (class in asyncio): Callback Handles. (line 22) +* times() (in module os): Process Management. (line 680) +* TIMESTAMP (py_compile.PycInvalidationMode attribute): py_compile — Compile Python source files. + (line 101) +* timestamp() (datetime.datetime method): datetime Objects. (line 546) +* timetuple() (datetime.date method): date Objects. (line 179) +* timetuple() (datetime.datetime method): datetime Objects. (line 501) +* timetz() (datetime.datetime method): datetime Objects. (line 404) +* timezone (class in datetime): timezone Objects. (line 14) +* timezone (in module time): Timezone Constants. (line 17) +* time_ns() (in module time): Functions<2>. (line 498) +* title() (bytearray method): Bytes and Bytearray Operations. + (line 621) +* title() (bytes method): Bytes and Bytearray Operations. + (line 621) +* title() (in module turtle): Methods specific to Screen not inherited from TurtleScreen. + (line 49) +* title() (str method): String Methods<2>. (line 544) +* Tix: tkinter tix — Extension widgets for Tk. + (line 8) +* tixCommand (class in tkinter.tix): Tix Commands. (line 6) +* tix_addbitmapdir() (tkinter.tix.tixCommand method): Tix Commands. + (line 47) +* tix_cget() (tkinter.tix.tixCommand method): Tix Commands. (line 32) +* tix_configure() (tkinter.tix.tixCommand method): Tix Commands. + (line 20) +* tix_filedialog() (tkinter.tix.tixCommand method): Tix Commands. + (line 57) +* tix_getbitmap() (tkinter.tix.tixCommand method): Tix Commands. + (line 37) +* tix_getimage() (tkinter.tix.tixCommand method): Tix Commands. + (line 68) +* tix_option_get() (tkinter.tix.tixCommand method): Tix Commands. + (line 82) +* tix_resetoptions() (tkinter.tix.tixCommand method): Tix Commands. + (line 86) +* Tk: Graphical User Interfaces with Tk. + (line 6) +* Tk (class in tkinter): Tkinter Modules. (line 25) +* Tk (class in tkinter.tix): Using Tix. (line 6) +* Tk Option Data Types: Tk Option Data Types. + (line 6) +* Tkinter: Graphical User Interfaces with Tk. + (line 6) +* tkinter (module): tkinter — Python interface to Tcl/Tk. + (line 6) +* tkinter.scrolledtext (module): tkinter scrolledtext — Scrolled Text Widget. + (line 6) +* tkinter.tix (module): tkinter tix — Extension widgets for Tk. + (line 6) +* tkinter.ttk (module): tkinter ttk — Tk themed widgets. + (line 6) +* TK_LIBRARY: How do I freeze Tkinter applications?. + (line 11) +* TList (class in tkinter.tix): Tabular ListBox. (line 6) +* TLS: ssl — TLS/SSL wrapper for socket objects. + (line 8) +* TLSv1 (ssl.TLSVersion attribute): Constants<9>. (line 535) +* TLSv1_1 (ssl.TLSVersion attribute): Constants<9>. (line 537) +* TLSv1_2 (ssl.TLSVersion attribute): Constants<9>. (line 539) +* TLSv1_3 (ssl.TLSVersion attribute): Constants<9>. (line 541) +* TLSVersion (class in ssl): Constants<9>. (line 517) +* TMP: tempfile — Generate temporary files and directories. + (line 241) +* TMPDIR: tempfile — Generate temporary files and directories. + (line 237) +* ToASCII() (in module encodings.idna): encodings idna — Internationalized Domain Names in Applications. + (line 52) +* tobuf() (tarfile.TarInfo method): TarInfo Objects. (line 31) +* tobytes() (array.array method): array — Efficient arrays of numeric values. + (line 218) +* tobytes() (memoryview method): Memory Views. (line 159) +* today() (datetime.date class method): date Objects. (line 29) +* today() (datetime.datetime class method): datetime Objects. + (line 47) +* tofile() (array.array method): array — Efficient arrays of numeric values. + (line 227) +* token: Lexical analysis. (line 6) +* token (module): token — Constants used with Python parse trees. + (line 6) +* token (shlex.shlex attribute): shlex Objects. (line 173) +* TokenError: Tokenizing Input. (line 107) +* tokenize (module): tokenize — Tokenizer for Python source. + (line 6) +* tokenize command line option; -e: Command-Line Usage<2>. + (line 19) +* tokenize command line option; –exact: Command-Line Usage<2>. + (line 19) +* tokenize command line option; -h: Command-Line Usage<2>. + (line 15) +* tokenize command line option; –help: Command-Line Usage<2>. + (line 15) +* tokenize() (in module tokenize): Tokenizing Input. (line 8) +* token_bytes() (in module secrets): Generating tokens. (line 10) +* token_hex() (in module secrets): Generating tokens. (line 19) +* token_urlsafe() (in module secrets): Generating tokens. (line 28) +* tok_name (in module token): token — Constants used with Python parse trees. + (line 20) +* tolist() (array.array method): array — Efficient arrays of numeric values. + (line 232) +* tolist() (memoryview method): Memory Views. (line 200) +* tolist() (parser.ST method): ST Objects. (line 29) +* tomono() (in module audioop): audioop — Manipulate raw audio data. + (line 200) +* toordinal() (datetime.date method): date Objects. (line 194) +* toordinal() (datetime.datetime method): datetime Objects. (line 541) +* top() (curses.panel.Panel method): Panel Objects. (line 53) +* top() (poplib.POP3 method): POP3 Objects. (line 86) +* toprettyxml() (xml.dom.minidom.Node method): DOM Objects. (line 58) +* top_panel() (in module curses.panel): Functions<4>. (line 19) +* toreadonly() (memoryview method): Memory Views. (line 216) +* tostereo() (in module audioop): audioop — Manipulate raw audio data. + (line 206) +* tostring() (array.array method): array — Efficient arrays of numeric values. + (line 236) +* tostring() (in module xml.etree.ElementTree): Functions<6>. + (line 205) +* tostringlist() (in module xml.etree.ElementTree): Functions<6>. + (line 227) +* total_changes (sqlite3.Connection attribute): Connection Objects. + (line 379) +* total_ordering() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 172) +* total_seconds() (datetime.timedelta method): timedelta Objects. + (line 245) +* totuple() (parser.ST method): ST Objects. (line 33) +* touch() (pathlib.Path method): Methods<2>. (line 409) +* touchline() (curses.window method): Window Objects. (line 568) +* touchwin() (curses.window method): Window Objects. (line 575) +* tounicode() (array.array method): array — Efficient arrays of numeric values. + (line 242) +* ToUnicode() (in module encodings.idna): encodings idna — Internationalized Domain Names in Applications. + (line 57) +* towards() (in module turtle): Tell Turtle’s state. + (line 15) +* toxml() (xml.dom.minidom.Node method): DOM Objects. (line 43) +* to_bytes() (int method): Additional Methods on Integer Types. + (line 35) +* to_eng_string() (decimal.Context method): Context objects. (line 506) +* to_eng_string() (decimal.Decimal method): Decimal objects. (line 558) +* to_integral() (decimal.Decimal method): Decimal objects. (line 571) +* to_integral_exact() (decimal.Context method): Context objects. + (line 516) +* to_integral_exact() (decimal.Decimal method): Decimal objects. + (line 577) +* to_integral_value() (decimal.Decimal method): Decimal objects. + (line 585) +* to_sci_string() (decimal.Context method): Context objects. (line 520) +* tparm() (in module curses): Functions<3>. (line 491) +* Trace (class in trace): Programmatic Interface. + (line 6) +* Trace (class in tracemalloc): Trace. (line 6) +* trace (module): trace — Trace or track Python statement execution. + (line 6) +* trace command line option; -c: Main options. (line 12) +* trace command line option; -C: Modifiers. (line 11) +* trace command line option; –count: Main options. (line 12) +* trace command line option; –coverdir=: Modifiers. (line 11) +* trace command line option; -f: Modifiers. (line 6) +* trace command line option; –file=: Modifiers. (line 6) +* trace command line option; -g: Modifiers. (line 33) +* trace command line option; –help: Command-Line Usage. (line 15) +* trace command line option; –ignore-dir=: Filters. (line 14) +* trace command line option; –ignore-module=: Filters. (line 8) +* trace command line option; -l: Main options. (line 23) +* trace command line option; –listfuncs: Main options. (line 23) +* trace command line option; -m: Modifiers. (line 17) +* trace command line option; –missing: Modifiers. (line 17) +* trace command line option; –no-report: Modifiers. (line 27) +* trace command line option; -r: Main options. (line 27) +* trace command line option; -R: Modifiers. (line 27) +* trace command line option; –report: Main options. (line 27) +* trace command line option; -s: Modifiers. (line 22) +* trace command line option; –summary: Modifiers. (line 22) +* trace command line option; -t: Main options. (line 19) +* trace command line option; -T: Main options. (line 33) +* trace command line option; –timing: Modifiers. (line 33) +* trace command line option; –trace: Main options. (line 19) +* trace command line option; –trackcalls: Main options. (line 33) +* trace command line option; –version: Command-Line Usage. (line 19) +* trace function: threading — Thread-based parallelism. + (line 125) +* trace function <1>: sys — System-specific parameters and functions. + (line 745) +* trace function <2>: sys — System-specific parameters and functions. + (line 1310) +* trace() (in module inspect): The interpreter stack. + (line 94) +* Traceback (class in tracemalloc): Traceback. (line 6) +* traceback (module): traceback — Print or retrieve a stack traceback. + (line 6) +* traceback (tracemalloc.Statistic attribute): Statistic. (line 23) +* traceback (tracemalloc.StatisticDiff attribute): StatisticDiff. + (line 38) +* traceback (tracemalloc.Trace attribute): Trace. (line 27) +* TracebackException (class in traceback): TracebackException Objects. + (line 11) +* tracebacklimit (in module sys): sys — System-specific parameters and functions. + (line 1581) +* tracebacks; in CGI scripts: cgitb — Traceback manager for CGI scripts. + (line 8) +* TracebackType (class in types): Standard Interpreter Types. + (line 182) +* traceback_limit (tracemalloc.Snapshot attribute): Snapshot. + (line 83) +* traceback_limit (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 219) +* tracemalloc (module): tracemalloc — Trace memory allocations. + (line 6) +* tracer() (in module turtle): Animation control. (line 23) +* traces (tracemalloc.Snapshot attribute): Snapshot. (line 89) +* trace_dispatch() (bdb.Bdb method): bdb — Debugger framework. + (line 111) +* trailing; comma: Expression lists. (line 24) +* transfercmd() (ftplib.FTP method): FTP Objects. (line 135) +* TransientResource (class in test.support): test support — Utilities for the Python test suite. + (line 1016) +* transient_internet() (in module test.support): test support — Utilities for the Python test suite. + (line 463) +* translate() (bytearray method): Bytes and Bytearray Operations. + (line 210) +* translate() (bytes method): Bytes and Bytearray Operations. + (line 210) +* translate() (in module fnmatch): fnmatch — Unix filename pattern matching. + (line 71) +* translate() (str method): String Methods<2>. (line 575) +* translation() (in module gettext): Class-based API. (line 44) +* transport (asyncio.StreamWriter attribute): StreamWriter. (line 57) +* Transport (class in asyncio): Transports Hierarchy. + (line 29) +* Transport Layer Security: ssl — TLS/SSL wrapper for socket objects. + (line 8) +* traverseproc (C type): Supporting Cyclic Garbage Collection. + (line 102) +* Tree (class in tkinter.tix): Hierarchical ListBox. + (line 20) +* TreeBuilder (class in xml.etree.ElementTree): TreeBuilder Objects. + (line 6) +* Treeview (class in tkinter.ttk): ttk Treeview. (line 6) +* triangular() (in module random): Real-valued distributions. + (line 25) +* triple-quoted string: String and Bytes literals. + (line 36) +* triple-quoted string <1>: Glossary. (line 1220) +* True: The standard type hierarchy. + (line 93) +* true: Truth Value Testing. + (line 10) +* True <1>: Truth Value Testing. + (line 23) +* True <2>: Boolean Values. (line 15) +* True (built-in variable): Built-in Constants. (line 13) +* truediv() (in module operator): operator — Standard operators as functions. + (line 158) +* trunc() (in module math): Numeric Types — int float complex. + (line 104) +* trunc() (in module math) <1>: Number-theoretic and representation functions. + (line 227) +* truncate() (in module os): Files and Directories. + (line 1415) +* truncate() (io.IOBase method): I/O Base Classes. (line 139) +* truth() (in module operator): operator — Standard operators as functions. + (line 57) +* truth; value: Truth Value Testing. + (line 6) +* ttk: tkinter ttk — Tk themed widgets. + (line 8) +* tty (module): tty — Terminal control functions. + (line 6) +* tty; I/O control: termios — POSIX style tty control. + (line 6) +* ttyname() (in module os): File Descriptor Operations. + (line 668) +* tuple (built-in class): Tuples. (line 12) +* Tuple (in module typing): Classes functions and decorators. + (line 849) +* tuple2st() (in module parser): Creating ST Objects. + (line 51) +* tuple_params (2to3 fixer): Fixers. (line 306) +* Turtle (class in turtle): Public classes. (line 16) +* turtle (module): turtle — Turtle graphics. + (line 6) +* turtledemo (module): turtledemo — Demo scripts. + (line 6) +* turtles() (in module turtle): Settings and special methods. + (line 96) +* TurtleScreen (class in turtle): Public classes. (line 22) +* turtlesize() (in module turtle): Appearance. (line 52) +* type: The standard type hierarchy. + (line 6) +* type <1>: Glossary. (line 1230) +* type (built-in class): Built-in Functions. (line 1662) +* Type (class in typing): Classes functions and decorators. + (line 105) +* type (optparse.Option attribute): Option attributes. (line 19) +* type (socket.socket attribute): Socket Objects. (line 599) +* type (tarfile.TarInfo attribute): TarInfo Objects. (line 59) +* type (urllib.request.Request attribute): Request Objects. (line 21) +* type alias: Glossary. (line 1237) +* type hint: Glossary. (line 1263) +* type of an object: Objects values and types. + (line 11) +* type; hierarchy: The standard type hierarchy. + (line 6) +* typeahead() (in module curses): Functions<3>. (line 499) +* typecode (array.array attribute): array — Efficient arrays of numeric values. + (line 104) +* typecodes (in module array): array — Efficient arrays of numeric values. + (line 91) +* TypedDict (class in typing): Classes functions and decorators. + (line 550) +* TYPED_ACTIONS (optparse.Option attribute): Adding new actions. + (line 39) +* typed_subpart_iterator() (in module email.iterators): email iterators Iterators. + (line 26) +* TypeError: Concrete exceptions. + (line 309) +* types (2to3 fixer): Fixers. (line 311) +* types (module): types — Dynamic type creation and names for built-in types. + (line 6) +* TYPES (optparse.Option attribute): Adding new types. (line 11) +* types, internal: The standard type hierarchy. + (line 645) +* types_map (in module mimetypes): mimetypes — Map filenames to MIME types. + (line 140) +* types_map (mimetypes.MimeTypes attribute): MimeTypes Objects. + (line 39) +* types_map_inv (mimetypes.MimeTypes attribute): MimeTypes Objects. + (line 47) +* TypeVar (class in typing): Classes functions and decorators. + (line 8) +* TYPE_CHECKER (optparse.Option attribute): Adding new types. + (line 16) +* TYPE_CHECKING (in module typing): Classes functions and decorators. + (line 966) +* type_check_only() (in module typing): Classes functions and decorators. + (line 734) +* TYPE_COMMENT (in module token): token — Constants used with Python parse trees. + (line 250) +* TYPE_IGNORE (in module token): token — Constants used with Python parse trees. + (line 248) +* typing (module): typing — Support for type hints. + (line 6) +* TZ: Functions<2>. (line 508) +* TZ <1>: Functions<2>. (line 509) +* TZ <2>: Functions<2>. (line 517) +* TZ <3>: Functions<2>. (line 522) +* TZ <4>: Functions<2>. (line 524) +* TZ <5>: Functions<2>. (line 584) +* tzinfo (class in datetime): tzinfo Objects. (line 6) +* tzinfo (datetime.datetime attribute): datetime Objects. (line 293) +* tzinfo (datetime.time attribute): time Objects. (line 66) +* tzname (in module time): Timezone Constants. (line 23) +* tzname() (datetime.datetime method): datetime Objects. (line 495) +* tzname() (datetime.time method): time Objects. (line 243) +* tzname() (datetime.timezone method): timezone Objects. (line 42) +* tzname() (datetime.tzinfo method): tzinfo Objects. (line 123) +* tzset() (in module time): Functions<2>. (line 505) +* T_FMT (in module locale): locale — Internationalization services. + (line 201) +* T_FMT_AMPM (in module locale): locale — Internationalization services. + (line 207) +* u"; string literal: Literals. (line 8) +* u’; string literal: Literals. (line 8) +* u-LAW: audioop — Manipulate raw audio data. + (line 17) +* u-LAW <1>: aifc — Read and write AIFF and AIFC files. + (line 160) +* u-LAW <2>: sndhdr — Determine type of sound file. + (line 8) +* ucd_3_2_0 (in module unicodedata): unicodedata — Unicode Database. + (line 131) +* udata (select.kevent attribute): Kevent Objects. (line 179) +* UDPServer (class in socketserver): socketserver — A framework for network servers. + (line 25) +* UF_APPEND (in module stat): stat — Interpreting stat results. + (line 346) +* UF_COMPRESSED (in module stat): stat — Interpreting stat results. + (line 358) +* UF_HIDDEN (in module stat): stat — Interpreting stat results. + (line 362) +* UF_IMMUTABLE (in module stat): stat — Interpreting stat results. + (line 342) +* UF_NODUMP (in module stat): stat — Interpreting stat results. + (line 338) +* UF_NOUNLINK (in module stat): stat — Interpreting stat results. + (line 354) +* UF_OPAQUE (in module stat): stat — Interpreting stat results. + (line 350) +* UID (class in plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 181) +* uid (tarfile.TarInfo attribute): TarInfo Objects. (line 72) +* uid() (imaplib.IMAP4 method): IMAP4 Objects. (line 358) +* uidl() (poplib.POP3 method): POP3 Objects. (line 98) +* ulaw2lin() (in module audioop): audioop — Manipulate raw audio data. + (line 213) +* ULONG_MAX: Integer Objects. (line 199) +* umask() (in module os): Process Parameters. (line 465) +* unalias (pdb command): Debugger Commands. (line 307) +* uname (tarfile.TarInfo attribute): TarInfo Objects. (line 80) +* uname() (in module os): Process Parameters. (line 469) +* uname() (in module platform): Cross Platform. (line 137) +* unary; arithmetic; operation: Unary arithmetic and bitwise operations. + (line 6) +* unary; bitwise; operation: Unary arithmetic and bitwise operations. + (line 6) +* unaryfunc (C type): Slot Type typedefs. (line 109) +* UNARY_INVERT (opcode): Python Bytecode Instructions. + (line 109) +* UNARY_NEGATIVE (opcode): Python Bytecode Instructions. + (line 101) +* UNARY_NOT (opcode): Python Bytecode Instructions. + (line 105) +* UNARY_POSITIVE (opcode): Python Bytecode Instructions. + (line 97) +* unbinding; name: The del statement<2>. + (line 15) +* UnboundLocalError: Resolution of names. + (line 16) +* UnboundLocalError <1>: Concrete exceptions. + (line 327) +* unbuffered I/O: Built-in Functions. (line 1217) +* UNC paths; and os.makedirs(): Files and Directories. + (line 454) +* UNCHECKED_HASH (py_compile.PycInvalidationMode attribute): py_compile — Compile Python source files. + (line 114) +* unconsumed_tail (zlib.Decompress attribute): zlib — Compression compatible with gzip. + (line 239) +* unctrl() (in module curses): Functions<3>. (line 511) +* unctrl() (in module curses.ascii): curses ascii — Utilities for ASCII characters. + (line 226) +* undefine_macro() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 170) +* Underflow (class in decimal): Signals. (line 93) +* undisplay (pdb command): Debugger Commands. (line 267) +* undo() (in module turtle): Turtle motion. (line 291) +* undobufferentries() (in module turtle): Special Turtle methods. + (line 74) +* undoc_header (cmd.Cmd attribute): Cmd Objects. (line 158) +* unescape() (in module html): html — HyperText Markup Language support. + (line 23) +* unescape() (in module xml.sax.saxutils): xml sax saxutils — SAX Utilities. + (line 24) +* UnexpectedException: Debugging. (line 208) +* unexpectedSuccesses (unittest.TestResult attribute): Loading and running tests. + (line 263) +* unfreeze() (in module gc): gc — Garbage Collector interface. + (line 194) +* ungetch() (in module curses): Functions<3>. (line 518) +* ungetch() (in module msvcrt): Console I/O. (line 44) +* ungetmouse() (in module curses): Functions<3>. (line 540) +* ungetwch() (in module msvcrt): Console I/O. (line 50) +* unget_wch() (in module curses): Functions<3>. (line 531) +* unhexlify() (in module binascii): binascii — Convert between binary and ASCII. + (line 161) +* Unicode: The standard type hierarchy. + (line 156) +* Unicode <1>: unicodedata — Unicode Database. + (line 6) +* Unicode <2>: codecs — Codec registry and base classes. + (line 8) +* unicode (2to3 fixer): Fixers. (line 316) +* Unicode Consortium: String and Bytes literals. + (line 36) +* Unicode; database: unicodedata — Unicode Database. + (line 6) +* unicodedata (module): unicodedata — Unicode Database. + (line 6) +* UnicodeDecodeError: Concrete exceptions. + (line 367) +* UnicodeEncodeError: Concrete exceptions. + (line 362) +* UnicodeError: Concrete exceptions. + (line 333) +* UnicodeTranslateError: Concrete exceptions. + (line 372) +* UnicodeWarning: Warnings. (line 50) +* unidata_version (in module unicodedata): unicodedata — Unicode Database. + (line 127) +* unified_diff() (in module difflib): difflib — Helpers for computing deltas. + (line 295) +* uniform() (in module random): Real-valued distributions. + (line 16) +* UnimplementedFileMode: http client — HTTP protocol client. + (line 150) +* Union (class in ctypes): Structured data types. + (line 6) +* Union (in module typing): Classes functions and decorators. + (line 795) +* union() (frozenset method): Set Types — set frozenset. + (line 96) +* unique() (in module enum): Ensuring unique enumeration values. + (line 10) +* unittest (module): unittest — Unit testing framework. + (line 6) +* unittest command line option; -b: Command-line options<3>. + (line 8) +* unittest command line option; –buffer: Command-line options<3>. + (line 8) +* unittest command line option; -c: Command-line options<3>. + (line 15) +* unittest command line option; –catch: Command-line options<3>. + (line 15) +* unittest command line option; -f: Command-line options<3>. + (line 24) +* unittest command line option; –failfast: Command-line options<3>. + (line 24) +* unittest command line option; -k: Command-line options<3>. + (line 28) +* unittest command line option; –locals: Command-line options<3>. + (line 45) +* unittest-discover command line option; -p: Test Discovery. (line 35) +* unittest-discover command line option; –pattern pattern: Test Discovery. + (line 35) +* unittest-discover command line option; -s: Test Discovery. (line 31) +* unittest-discover command line option; –start-directory directory: Test Discovery. + (line 31) +* unittest-discover command line option; -t: Test Discovery. (line 39) +* unittest-discover command line option; –top-level-directory directory: Test Discovery. + (line 39) +* unittest-discover command line option; -v: Test Discovery. (line 27) +* unittest-discover command line option; –verbose: Test Discovery. + (line 27) +* unittest.mock (module): unittest mock — mock object library. + (line 6) +* universal newlines: Glossary. (line 1280) +* universal newlines; bytearray.splitlines method: Bytes and Bytearray Operations. + (line 572) +* universal newlines; bytes.splitlines method: Bytes and Bytearray Operations. + (line 572) +* universal newlines; csv.reader function: Module Contents<3>. + (line 8) +* universal newlines; importlib.abc.InspectLoader.get_source method: importlib abc – Abstract base classes related to import. + (line 398) +* universal newlines; io.IncrementalNewlineDecoder class: Text I/O<2>. + (line 234) +* universal newlines; io.TextIOWrapper class: Text I/O<2>. (line 123) +* universal newlines; open() built-in function: Built-in Functions. + (line 1155) +* universal newlines; str.splitlines method: String Methods<2>. + (line 434) +* universal newlines; subprocess module: Frequently Used Arguments. + (line 33) +* universal newlines; What’s new: PEP 3116 New I/O Library. + (line 42) +* universal newlines; What’s new <1>: New Improved and Removed Modules. + (line 138) +* universal newlines; What’s new <2>: PEP 324 New subprocess Module. + (line 36) +* universal newlines; What’s new <3>: PEP 277 Unicode file name support for Windows NT. + (line 36) +* UNIX: Complete Python programs. + (line 25) +* UNIX; file control: fcntl — The fcntl and ioctl system calls. + (line 6) +* UNIX; I/O control: fcntl — The fcntl and ioctl system calls. + (line 6) +* UnixDatagramServer (class in socketserver): socketserver — A framework for network servers. + (line 32) +* UnixStreamServer (class in socketserver): socketserver — A framework for network servers. + (line 32) +* unix_dialect (class in csv): Module Contents<3>. (line 195) +* unix_shell (in module test.support): test support — Utilities for the Python test suite. + (line 44) +* unknown (uuid.SafeUUID attribute): uuid — UUID objects according to RFC 4122. + (line 40) +* UnknownHandler (class in urllib.request): urllib request — Extensible library for opening URLs. + (line 435) +* UnknownProtocol: http client — HTTP protocol client. + (line 142) +* UnknownTransferEncoding: http client — HTTP protocol client. + (line 146) +* unknown_decl() (html.parser.HTMLParser method): HTMLParser Methods. + (line 137) +* unknown_open() (urllib.request.BaseHandler method): BaseHandler Objects. + (line 55) +* unknown_open() (urllib.request.UnknownHandler method): UnknownHandler Objects. + (line 6) +* unlink() (in module os): Files and Directories. + (line 1433) +* unlink() (in module test.support): test support — Utilities for the Python test suite. + (line 171) +* unlink() (multiprocessing.shared_memory.SharedMemory method): multiprocessing shared_memory — Provides shared memory for direct access across processes. + (line 73) +* unlink() (pathlib.Path method): Methods<2>. (line 418) +* unlink() (xml.dom.minidom.Node method): DOM Objects. (line 10) +* unload() (in module test.support): test support — Utilities for the Python test suite. + (line 167) +* unlock() (mailbox.Babyl method): Babyl. (line 59) +* unlock() (mailbox.Mailbox method): Mailbox objects. (line 258) +* unlock() (mailbox.Maildir method): Maildir. (line 107) +* unlock() (mailbox.mbox method): mbox. (line 39) +* unlock() (mailbox.MH method): MH. (line 84) +* unlock() (mailbox.MMDF method): MMDF. (line 35) +* unpack() (in module struct): Functions and Exceptions. + (line 26) +* unpack() (struct.Struct method): Classes<2>. (line 35) +* Unpacker (class in xdrlib): xdrlib — Encode and decode XDR data. + (line 24) +* unpacking; dictionary: Dictionary displays. + (line 23) +* unpacking; in function calls: Calls. (line 74) +* unpack_archive() (in module shutil): Archiving operations. + (line 105) +* unpack_array() (xdrlib.Unpacker method): Unpacker Objects. (line 91) +* unpack_bytes() (xdrlib.Unpacker method): Unpacker Objects. (line 70) +* unpack_double() (xdrlib.Unpacker method): Unpacker Objects. + (line 40) +* UNPACK_EX (opcode): Python Bytecode Instructions. + (line 499) +* unpack_farray() (xdrlib.Unpacker method): Unpacker Objects. + (line 85) +* unpack_float() (xdrlib.Unpacker method): Unpacker Objects. (line 36) +* unpack_fopaque() (xdrlib.Unpacker method): Unpacker Objects. + (line 54) +* unpack_from() (in module struct): Functions and Exceptions. + (line 34) +* unpack_from() (struct.Struct method): Classes<2>. (line 41) +* unpack_fstring() (xdrlib.Unpacker method): Unpacker Objects. + (line 48) +* unpack_list() (xdrlib.Unpacker method): Unpacker Objects. (line 77) +* unpack_opaque() (xdrlib.Unpacker method): Unpacker Objects. + (line 65) +* UNPACK_SEQUENCE (opcode): Python Bytecode Instructions. + (line 494) +* unpack_string() (xdrlib.Unpacker method): Unpacker Objects. + (line 59) +* unparsedEntityDecl() (xml.sax.handler.DTDHandler method): DTDHandler Objects. + (line 12) +* UnparsedEntityDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 249) +* Unpickler (class in pickle): Module Interface. (line 226) +* UnpicklingError: Module Interface. (line 108) +* unquote() (in module email.utils): email utils Miscellaneous utilities. + (line 52) +* unquote() (in module urllib.parse): URL Quoting. (line 60) +* unquote_plus() (in module urllib.parse): URL Quoting. (line 76) +* unquote_to_bytes() (in module urllib.parse): URL Quoting. (line 86) +* unraisablehook() (in module sys): sys — System-specific parameters and functions. + (line 1589) +* unreachable object: Objects values and types. + (line 37) +* unreadline() (distutils.text_file.TextFile method): distutils text_file — The TextFile class. + (line 130) +* unrecognized escape sequence: String and Bytes literals. + (line 156) +* unregister() (in module atexit): atexit — Exit handlers. + (line 43) +* unregister() (in module faulthandler): Dumping the traceback on a user signal. + (line 23) +* unregister() (select.devpoll method): /dev/poll Polling Objects. + (line 55) +* unregister() (select.epoll method): Edge and Level Trigger Polling epoll Objects. + (line 89) +* unregister() (select.poll method): Polling Objects. (line 67) +* unregister() (selectors.BaseSelector method): Classes<3>. (line 80) +* unregister_archive_format() (in module shutil): Archiving operations. + (line 100) +* unregister_dialect() (in module csv): Module Contents<3>. (line 79) +* unregister_unpack_format() (in module shutil): Archiving operations. + (line 143) +* unsafe (uuid.SafeUUID attribute): uuid — UUID objects according to RFC 4122. + (line 36) +* unset() (test.support.EnvironmentVarGuard method): test support — Utilities for the Python test suite. + (line 1041) +* unsetenv() (in module os): Process Parameters. (line 499) +* UnstructuredHeader (class in email.headerregistry): email headerregistry Custom Header Objects. + (line 115) +* unsubscribe() (imaplib.IMAP4 method): IMAP4 Objects. (line 365) +* UnsupportedOperation: High-level Module Interface. + (line 43) +* until (pdb command): Debugger Commands. (line 177) +* untokenize() (in module tokenize): Tokenizing Input. (line 57) +* untouchwin() (curses.window method): Window Objects. (line 580) +* unused_data (bz2.BZ2Decompressor attribute): Incremental de compression. + (line 78) +* unused_data (lzma.LZMADecompressor attribute): Compressing and decompressing data in memory. + (line 169) +* unused_data (zlib.Decompress attribute): zlib — Compression compatible with gzip. + (line 231) +* unverifiable (urllib.request.Request attribute): Request Objects. + (line 47) +* unwrap() (in module inspect): Classes and functions<2>. + (line 186) +* unwrap() (in module urllib.parse): URL Parsing. (line 370) +* unwrap() (ssl.SSLSocket method): SSL Sockets. (line 272) +* up (pdb command): Debugger Commands. (line 66) +* up() (in module turtle): Drawing state. (line 12) +* update() (collections.Counter method): Counter objects. (line 111) +* update() (dict method): Mapping Types — dict. + (line 211) +* update() (frozenset method): Set Types — set frozenset. + (line 169) +* update() (hashlib.hash method): Hash algorithms. (line 117) +* update() (hmac.HMAC method): hmac — Keyed-Hashing for Message Authentication. + (line 47) +* update() (http.cookies.Morsel method): Morsel Objects. (line 94) +* update() (in module turtle): Animation control. (line 46) +* update() (mailbox.Mailbox method): Mailbox objects. (line 227) +* update() (mailbox.Maildir method): Maildir. (line 91) +* update() (trace.CoverageResults method): Programmatic Interface. + (line 52) +* update_authenticated() (urllib.request.HTTPPasswordMgrWithPriorAuth method): HTTPPasswordMgrWithPriorAuth Objects. + (line 24) +* update_lines_cols() (in module curses): Functions<3>. (line 524) +* update_panels() (in module curses.panel): Functions<4>. (line 23) +* update_visible() (mailbox.BabylMessage method): BabylMessage. + (line 76) +* update_wrapper() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 485) +* upper() (bytearray method): Bytes and Bytearray Operations. + (line 664) +* upper() (bytes method): Bytes and Bytearray Operations. + (line 664) +* upper() (str method): String Methods<2>. (line 593) +* urandom() (in module os): Random numbers<2>. (line 29) +* URL: cgi — Common Gateway Interface support. + (line 8) +* URL <1>: urllib parse — Parse URLs into components. + (line 8) +* URL <2>: urllib robotparser — Parser for robots txt. + (line 8) +* URL <3>: http server — HTTP servers. + (line 8) +* url (xmlrpc.client.ProtocolError attribute): ProtocolError Objects. + (line 13) +* url2pathname() (in module urllib.request): urllib request — Extensible library for opening URLs. + (line 154) +* URL; parsing: urllib parse — Parse URLs into components. + (line 8) +* urlcleanup() (in module urllib.request): Legacy interface. (line 62) +* urldefrag() (in module urllib.parse): URL Parsing. (line 344) +* urlencode() (in module urllib.parse): URL Quoting. (line 98) +* URLError: urllib error — Exception classes raised by urllib request. + (line 17) +* urljoin() (in module urllib.parse): URL Parsing. (line 314) +* urllib (2to3 fixer): Fixers. (line 320) +* urllib (module): urllib — URL handling modules. + (line 6) +* urllib.error (module): urllib error — Exception classes raised by urllib request. + (line 6) +* urllib.parse (module): urllib parse — Parse URLs into components. + (line 6) +* urllib.request (module): urllib request — Extensible library for opening URLs. + (line 6) +* urllib.response (module): urllib response — Response classes used by urllib. + (line 6) +* urllib.robotparser (module): urllib robotparser — Parser for robots txt. + (line 6) +* urlopen() (in module urllib.request): urllib request — Extensible library for opening URLs. + (line 22) +* URLopener (class in urllib.request): Legacy interface. (line 67) +* urlparse() (in module urllib.parse): URL Parsing. (line 9) +* urlretrieve() (in module urllib.request): Legacy interface. + (line 10) +* urlsafe_b64decode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 91) +* urlsafe_b64encode() (in module base64): base64 — Base16 Base32 Base64 Base85 Data Encodings. + (line 84) +* urlsplit() (in module urllib.parse): URL Parsing. (line 238) +* urlunparse() (in module urllib.parse): URL Parsing. (line 230) +* urlunsplit() (in module urllib.parse): URL Parsing. (line 305) +* urn (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 135) +* UseForeignDTD() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 63) +* USER: getpass — Portable password input. + (line 38) +* user() (poplib.POP3 method): POP3 Objects. (line 31) +* user-defined; function: The standard type hierarchy. + (line 297) +* user-defined; function; call: Calls. (line 123) +* user-defined; method: The standard type hierarchy. + (line 388) +* user; effective id: Process Parameters. (line 173) +* user; id: Process Parameters. (line 302) +* user; id, setting: Process Parameters. (line 448) +* UserDict (class in collections): UserDict objects. (line 12) +* UserList (class in collections): UserList objects. (line 16) +* USERNAME: Process Parameters. (line 221) +* USERNAME <1>: getpass — Portable password input. + (line 39) +* username (email.headerregistry.Address attribute): email headerregistry Custom Header Objects. + (line 432) +* USERPROFILE: os path. (line 14) +* USERPROFILE <1>: Changes in the Python API. + (line 115) +* USERPROFILE <2>: os path — Common pathname manipulations. + (line 146) +* USERPROFILE <3>: Location and names of config files. + (line 67) +* userptr() (curses.panel.Panel method): Panel Objects. (line 57) +* UserString (class in collections): UserString objects. (line 12) +* UserWarning: Warnings. (line 13) +* USER_BASE: New and Improved Modules. + (line 469) +* USER_BASE (in module site): Module contents<3>. (line 30) +* user_call() (bdb.Bdb method): bdb — Debugger framework. + (line 212) +* user_exception() (bdb.Bdb method): bdb — Debugger framework. + (line 229) +* user_line() (bdb.Bdb method): bdb — Debugger framework. + (line 218) +* user_return() (bdb.Bdb method): bdb — Debugger framework. + (line 224) +* USER_SITE (in module site): Module contents<3>. (line 19) +* use_default_colors() (in module curses): Functions<3>. (line 555) +* use_env() (in module curses): Functions<3>. (line 545) +* use_rawinput (cmd.Cmd attribute): Cmd Objects. (line 169) +* USTAR_FORMAT (in module tarfile): tarfile — Read and write tar archive files. + (line 222) +* UTC: time — Time access and conversions. + (line 40) +* utc (datetime.timezone attribute): timezone Objects. (line 68) +* utcfromtimestamp() (datetime.datetime class method): datetime Objects. + (line 124) +* utcnow() (datetime.datetime class method): datetime Objects. + (line 77) +* utcoffset() (datetime.datetime method): datetime Objects. (line 475) +* utcoffset() (datetime.time method): time Objects. (line 223) +* utcoffset() (datetime.timezone method): timezone Objects. (line 30) +* utcoffset() (datetime.tzinfo method): tzinfo Objects. (line 38) +* utctimetuple() (datetime.datetime method): datetime Objects. + (line 520) +* utf8 (email.policy.EmailPolicy attribute): email policy Policy Objects. + (line 363) +* utf8() (poplib.POP3 method): POP3 Objects. (line 105) +* utf8_enabled (imaplib.IMAP4 attribute): IMAP4 Objects. (line 387) +* utime() (in module os): Files and Directories. + (line 1447) +* uu (module): uu — Encode and decode uuencode files. + (line 6) +* UUID (class in uuid): uuid — UUID objects according to RFC 4122. + (line 45) +* uuid (module): uuid — UUID objects according to RFC 4122. + (line 6) +* uuid1: uuid — UUID objects according to RFC 4122. + (line 186) +* uuid1() (in module uuid): uuid — UUID objects according to RFC 4122. + (line 178) +* uuid3: uuid — UUID objects according to RFC 4122. + (line 191) +* uuid3() (in module uuid): uuid — UUID objects according to RFC 4122. + (line 186) +* uuid4: uuid — UUID objects according to RFC 4122. + (line 195) +* uuid4() (in module uuid): uuid — UUID objects according to RFC 4122. + (line 191) +* uuid5: uuid — UUID objects according to RFC 4122. + (line 200) +* uuid5() (in module uuid): uuid — UUID objects according to RFC 4122. + (line 195) +* UuidCreate() (in module msilib): msilib — Read and write Microsoft Installer files. + (line 39) +* v4_int_to_packed() (in module ipaddress): Other Module Level Functions. + (line 8) +* v6_int_to_packed() (in module ipaddress): Other Module Level Functions. + (line 20) +* validator() (in module wsgiref.validate): wsgiref validate — WSGI conformance checker. + (line 22) +* valid_signals() (in module signal): Module contents<2>. (line 257) +* value (ctypes._SimpleCData attribute): Fundamental data types<2>. + (line 17) +* value (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 32) +* value (http.cookies.Morsel attribute): Morsel Objects. (line 52) +* value (xml.dom.Attr attribute): Attr Objects. (line 23) +* value of an object: Objects values and types. + (line 11) +* Value() (in module multiprocessing): Shared ctypes Objects. + (line 9) +* Value() (in module multiprocessing.sharedctypes): The multiprocessing sharedctypes module. + (line 67) +* Value() (multiprocessing.managers.SyncManager method): Managers. + (line 200) +* ValueError: Concrete exceptions. + (line 377) +* valuerefs() (weakref.WeakValueDictionary method): weakref — Weak references. + (line 188) +* values() (contextvars.Context method): Manual Context Management. + (line 104) +* values() (dict method): Mapping Types — dict. + (line 222) +* values() (email.message.EmailMessage method): email message Representing an email message. + (line 236) +* values() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 344) +* values() (mailbox.Mailbox method): Mailbox objects. (line 114) +* values() (types.MappingProxyType method): Standard Interpreter Types. + (line 265) +* ValuesView (class in collections.abc): Collections Abstract Base Classes. + (line 187) +* ValuesView (class in typing): Classes functions and decorators. + (line 294) +* value_decode() (http.cookies.BaseCookie method): Cookie Objects. + (line 6) +* value_encode() (http.cookies.BaseCookie method): Cookie Objects. + (line 13) +* var (contextvars.contextvars.Token.Token attribute): Context Variables. + (line 82) +* variable annotation: Glossary. (line 1288) +* variance (statistics.NormalDist attribute): NormalDist objects. + (line 40) +* variance() (in module statistics): Function details. (line 344) +* variant (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 139) +* vars() (built-in function): Built-in Functions. (line 1703) +* VBAR (in module token): token — Constants used with Python parse trees. + (line 98) +* vbar (tkinter.scrolledtext.ScrolledText attribute): tkinter scrolledtext — Scrolled Text Widget. + (line 28) +* VBAREQUAL (in module token): token — Constants used with Python parse trees. + (line 194) +* Vec2D (class in turtle): Public classes. (line 86) +* vectorcallfunc (C type): Slot Type typedefs. (line 26) +* venv (module): venv — Creation of virtual environments. + (line 6) +* VERBOSE (in module re): Module Contents. (line 120) +* verbose (in module tabnanny): tabnanny — Detection of ambiguous indentation. + (line 26) +* verbose (in module test.support): test support — Utilities for the Python test suite. + (line 30) +* verify() (smtplib.SMTP method): SMTP Objects. (line 91) +* VerifyFlags (class in ssl): Constants<9>. (line 112) +* VerifyMode (class in ssl): Constants<9>. (line 64) +* verify_client_post_handshake() (ssl.SSLSocket method): SSL Sockets. + (line 281) +* verify_code (ssl.SSLCertVerificationError attribute): Exceptions<12>. + (line 86) +* VERIFY_CRL_CHECK_CHAIN (in module ssl): Constants<9>. (line 89) +* VERIFY_CRL_CHECK_LEAF (in module ssl): Constants<9>. (line 78) +* VERIFY_DEFAULT (in module ssl): Constants<9>. (line 70) +* verify_flags (ssl.SSLContext attribute): SSL Contexts. (line 640) +* verify_message (ssl.SSLCertVerificationError attribute): Exceptions<12>. + (line 90) +* verify_mode (ssl.SSLContext attribute): SSL Contexts. (line 656) +* verify_request() (socketserver.BaseServer method): Server Objects<2>. + (line 175) +* VERIFY_X509_STRICT (in module ssl): Constants<9>. (line 96) +* VERIFY_X509_TRUSTED_FIRST (in module ssl): Constants<9>. (line 103) +* version (email.headerregistry.MIMEVersionHeader attribute): email headerregistry Custom Header Objects. + (line 242) +* version (http.client.HTTPResponse attribute): HTTPResponse Objects. + (line 46) +* version (http.cookiejar.Cookie attribute): Cookie Objects<2>. + (line 20) +* version (in module curses): Constants<6>. (line 18) +* version (in module marshal): marshal — Internal Python object serialization. + (line 90) +* version (in module sqlite3): Module functions and constants. + (line 6) +* version (in module sys): sys — System-specific parameters and functions. + (line 1634) +* version (in module sys) <1>: Process-wide parameters. + (line 164) +* version (in module sys) <2>: Process-wide parameters. + (line 199) +* version (in module sys) <3>: Process-wide parameters. + (line 210) +* version (ipaddress.IPv4Address attribute): Address objects. + (line 39) +* version (ipaddress.IPv4Network attribute): Network objects. + (line 63) +* version (ipaddress.IPv6Address attribute): Address objects. + (line 167) +* version (ipaddress.IPv6Network attribute): Network objects. + (line 290) +* version (urllib.request.URLopener attribute): Legacy interface. + (line 139) +* version (uuid.UUID attribute): uuid — UUID objects according to RFC 4122. + (line 146) +* version() (in module ensurepip): Module API. (line 8) +* version() (in module platform): Cross Platform. (line 132) +* version() (ssl.SSLSocket method): SSL Sockets. (line 303) +* version_info (in module sqlite3): Module functions and constants. + (line 11) +* version_info (in module sys): sys — System-specific parameters and functions. + (line 1649) +* version_string() (http.server.BaseHTTPRequestHandler method): http server — HTTP servers. + (line 304) +* vformat() (string.Formatter method): Custom String Formatting. + (line 26) +* virtual environment: Glossary. (line 1310) +* virtual machine: Glossary. (line 1319) +* VIRTUAL_ENV: Creating virtual environments. + (line 129) +* visit() (ast.NodeVisitor method): ast Helpers. (line 129) +* visitproc (C type): Supporting Cyclic Garbage Collection. + (line 91) +* vline() (curses.window method): Window Objects. (line 585) +* voidcmd() (ftplib.FTP method): FTP Objects. (line 74) +* volume (zipfile.ZipInfo attribute): ZipInfo Objects. (line 119) +* vonmisesvariate() (in module random): Real-valued distributions. + (line 75) +* wait() (asyncio.asyncio.subprocess.Process method): Interacting with Subprocesses. + (line 38) +* wait() (asyncio.Condition method): Condition. (line 86) +* wait() (asyncio.Event method): Event. (line 45) +* wait() (in module asyncio): Waiting Primitives. (line 6) +* wait() (in module concurrent.futures): Module Functions. (line 6) +* wait() (in module multiprocessing.connection): Listeners and Clients. + (line 113) +* wait() (in module os): Process Management. (line 709) +* wait() (multiprocessing.pool.AsyncResult method): Process Pools. + (line 204) +* wait() (subprocess.Popen method): Popen Objects. (line 13) +* wait() (threading.Barrier method): Barrier Objects. (line 43) +* wait() (threading.Condition method): Condition Objects. (line 97) +* wait() (threading.Event method): Event Objects. (line 39) +* wait3() (in module os): Process Management. (line 807) +* wait4() (in module os): Process Management. (line 818) +* waitid() (in module os): Process Management. (line 719) +* waitpid() (in module os): Process Management. (line 770) +* wait_closed() (asyncio.Server method): Server Objects. (line 95) +* wait_closed() (asyncio.StreamWriter method): StreamWriter. (line 88) +* wait_for() (asyncio.Condition method): Condition. (line 98) +* wait_for() (in module asyncio): Timeouts. (line 6) +* wait_for() (threading.Condition method): Condition Objects. + (line 128) +* wait_threads_exit() (in module test.support): test support — Utilities for the Python test suite. + (line 521) +* walk() (email.message.EmailMessage method): email message Representing an email message. + (line 467) +* walk() (email.message.Message method): email message Message Representing an email message using the compat32 API. + (line 636) +* walk() (in module ast): ast Helpers. (line 112) +* walk() (in module os): Files and Directories. + (line 1490) +* walk_packages() (in module pkgutil): pkgutil — Package extension utility. + (line 162) +* walk_stack() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 157) +* walk_tb() (in module traceback): traceback — Print or retrieve a stack traceback. + (line 166) +* want (doctest.Example attribute): Example Objects. (line 23) +* warn() (distutils.ccompiler.CCompiler method): distutils ccompiler — CCompiler base class. + (line 507) +* warn() (distutils.text_file.TextFile method): distutils text_file — The TextFile class. + (line 101) +* warn() (in module warnings): Available Functions. + (line 6) +* Warning: Warnings. (line 9) +* Warning <1>: Exceptions<4>. (line 6) +* warning() (in module logging): Module-Level Functions. + (line 124) +* warning() (logging.Logger method): Logger Objects. (line 220) +* warning() (xml.sax.handler.ErrorHandler method): ErrorHandler Objects. + (line 29) +* warnings: warnings — Warning control. + (line 8) +* warnings (module): warnings — Warning control. + (line 6) +* WarningsRecorder (class in test.support): test support — Utilities for the Python test suite. + (line 1097) +* warnoptions (in module sys): sys — System-specific parameters and functions. + (line 1662) +* warn_explicit() (in module warnings): Available Functions. + (line 32) +* wasSuccessful() (unittest.TestResult method): Loading and running tests. + (line 302) +* WatchedFileHandler (class in logging.handlers): WatchedFileHandler. + (line 24) +* wave (module): wave — Read and write WAV files. + (line 6) +* WCONTINUED (in module os): Process Management. (line 837) +* WCOREDUMP() (in module os): Process Management. (line 857) +* WeakKeyDictionary (class in weakref): weakref — Weak references. + (line 158) +* WeakMethod (class in weakref): weakref — Weak references. + (line 197) +* weakref (module): weakref — Weak references. + (line 6) +* WeakSet (class in weakref): weakref — Weak references. + (line 192) +* WeakValueDictionary (class in weakref): weakref — Weak references. + (line 178) +* webbrowser (module): webbrowser — Convenient Web-browser controller. + (line 6) +* weekday() (datetime.date method): date Objects. (line 200) +* weekday() (datetime.datetime method): datetime Objects. (line 582) +* weekday() (in module calendar): calendar — General calendar-related functions. + (line 313) +* weekheader() (in module calendar): calendar — General calendar-related functions. + (line 318) +* weibullvariate() (in module random): Real-valued distributions. + (line 87) +* WEXITED (in module os): Process Management. (line 747) +* WEXITSTATUS() (in module os): Process Management. (line 903) +* wfile (http.server.BaseHTTPRequestHandler attribute): http server — HTTP servers. + (line 115) +* what() (in module imghdr): imghdr — Determine the type of an image. + (line 15) +* what() (in module sndhdr): sndhdr — Determine type of sound file. + (line 25) +* whathdr() (in module sndhdr): sndhdr — Determine type of sound file. + (line 34) +* whatis (pdb command): Debugger Commands. (line 247) +* when() (asyncio.TimerHandle method): Callback Handles. (line 29) +* where (pdb command): Debugger Commands. (line 55) +* which() (in module shutil): Directory and files operations. + (line 370) +* whichdb() (in module dbm): dbm — Interfaces to Unix “databases”. + (line 23) +* whitespace (in module string): String constants. (line 47) +* whitespace (shlex.shlex attribute): shlex Objects. (line 104) +* whitespace_split (shlex.shlex attribute): shlex Objects. (line 128) +* Widget (class in tkinter.ttk): ttk Widget. (line 9) +* width (textwrap.TextWrapper attribute): textwrap — Text wrapping and filling. + (line 149) +* width() (in module turtle): Drawing state. (line 18) +* WIFCONTINUED() (in module os): Process Management. (line 867) +* WIFEXITED() (in module os): Process Management. (line 895) +* WIFSIGNALED() (in module os): Process Management. (line 888) +* WIFSTOPPED() (in module os): Process Management. (line 877) +* win32_edition() (in module platform): Windows Platform. (line 21) +* win32_is_iot() (in module platform): Windows Platform. (line 29) +* win32_ver() (in module platform): Windows Platform. (line 6) +* WinDLL (class in ctypes): Loading shared libraries. + (line 46) +* window manager (widgets): The Window Manager. (line 6) +* window() (curses.panel.Panel method): Panel Objects. (line 62) +* Windows: Complete Python programs. + (line 25) +* Windows ini file: configparser — Configuration file parser. + (line 8) +* WindowsError: Concrete exceptions. + (line 398) +* WindowsPath (class in pathlib): Concrete paths. (line 32) +* WindowsProactorEventLoopPolicy (class in asyncio): Policy Objects. + (line 68) +* WindowsRegistryFinder (class in importlib.machinery): importlib machinery – Importers and path hooks. + (line 92) +* WindowsSelectorEventLoopPolicy (class in asyncio): Policy Objects. + (line 61) +* window_height() (in module turtle): Settings and special methods. + (line 103) +* window_width() (in module turtle): Settings and special methods. + (line 110) +* winerror (OSError attribute): Concrete exceptions. + (line 139) +* WinError() (in module ctypes): Utility functions. (line 203) +* WINFUNCTYPE() (in module ctypes): Function prototypes. + (line 25) +* winreg (module): winreg — Windows registry access. + (line 6) +* WinSock: select — Waiting for I/O completion. + (line 146) +* winsound (module): winsound — Sound-playing interface for Windows. + (line 6) +* winver (in module sys): sys — System-specific parameters and functions. + (line 1668) +* WITH_CLEANUP_FINISH (opcode): Python Bytecode Instructions. + (line 468) +* WITH_CLEANUP_START (opcode): Python Bytecode Instructions. + (line 452) +* with_hostmask (ipaddress.IPv4Interface attribute): Interface objects. + (line 55) +* with_hostmask (ipaddress.IPv4Network attribute): Network objects. + (line 123) +* with_hostmask (ipaddress.IPv6Interface attribute): Interface objects. + (line 82) +* with_hostmask (ipaddress.IPv6Network attribute): Network objects. + (line 322) +* with_name() (pathlib.PurePath method): Methods and properties. + (line 259) +* with_netmask (ipaddress.IPv4Interface attribute): Interface objects. + (line 46) +* with_netmask (ipaddress.IPv4Network attribute): Network objects. + (line 118) +* with_netmask (ipaddress.IPv6Interface attribute): Interface objects. + (line 80) +* with_netmask (ipaddress.IPv6Network attribute): Network objects. + (line 320) +* with_prefixlen (ipaddress.IPv4Interface attribute): Interface objects. + (line 37) +* with_prefixlen (ipaddress.IPv4Network attribute): Network objects. + (line 105) +* with_prefixlen (ipaddress.IPv6Interface attribute): Interface objects. + (line 78) +* with_prefixlen (ipaddress.IPv6Network attribute): Network objects. + (line 314) +* with_pymalloc() (in module test.support): test support — Utilities for the Python test suite. + (line 207) +* with_suffix() (pathlib.PurePath method): Methods and properties. + (line 275) +* with_traceback() (BaseException method): Base classes. (line 25) +* WNOHANG (in module os): Process Management. (line 829) +* WNOWAIT (in module os): Process Management. (line 747) +* wordchars (shlex.shlex attribute): shlex Objects. (line 91) +* World Wide Web: Internet Protocols and Support. + (line 6) +* World Wide Web <1>: urllib parse — Parse URLs into components. + (line 8) +* World Wide Web <2>: urllib robotparser — Parser for robots txt. + (line 8) +* wrap() (in module textwrap): textwrap — Text wrapping and filling. + (line 16) +* wrap() (textwrap.TextWrapper method): textwrap — Text wrapping and filling. + (line 276) +* wrapper() (in module curses): Functions<3>. (line 564) +* WrapperDescriptorType (in module types): Standard Interpreter Types. + (line 88) +* wraps() (in module functools): functools — Higher-order functions and operations on callable objects. + (line 533) +* wrap_bio() (ssl.SSLContext method): SSL Contexts. (line 454) +* wrap_future() (in module asyncio): Future Functions. (line 44) +* wrap_socket() (in module ssl): Certificate handling. + (line 167) +* wrap_socket() (ssl.SSLContext method): SSL Contexts. (line 390) +* wrap_text() (in module distutils.fancy_getopt): distutils fancy_getopt — Wrapper around the standard getopt module. + (line 32) +* WRITABLE (in module tkinter): File Handlers. (line 41) +* writable() (asyncore.dispatcher method): asyncore — Asynchronous socket handler. + (line 172) +* writable() (io.IOBase method): I/O Base Classes. (line 152) +* write() (asyncio.StreamWriter method): StreamWriter. (line 15) +* write() (asyncio.WriteTransport method): Write-only Transports. + (line 60) +* write() (code.InteractiveInterpreter method): Interactive Interpreter Objects. + (line 63) +* write() (codecs.StreamWriter method): StreamWriter Objects. + (line 31) +* write() (configparser.ConfigParser method): ConfigParser Objects. + (line 257) +* write() (email.generator.BytesGenerator method): email generator Generating MIME documents. + (line 118) +* write() (email.generator.Generator method): email generator Generating MIME documents. + (line 214) +* write() (in module os): File Descriptor Operations. + (line 676) +* write() (in module turtle): More drawing control. + (line 29) +* write() (io.BufferedIOBase method): I/O Base Classes. (line 324) +* write() (io.BufferedWriter method): Buffered Streams. (line 127) +* write() (io.RawIOBase method): I/O Base Classes. (line 211) +* write() (io.TextIOBase method): Text I/O<2>. (line 94) +* write() (mmap.mmap method): mmap — Memory-mapped file support. + (line 278) +* write() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 39) +* write() (ssl.MemoryBIO method): Memory BIO Support<2>. + (line 153) +* write() (ssl.SSLSocket method): SSL Sockets. (line 91) +* write() (telnetlib.Telnet method): Telnet Objects. (line 101) +* write() (xml.etree.ElementTree.ElementTree method): ElementTree Objects. + (line 67) +* write() (zipfile.ZipFile method): ZipFile Objects. (line 242) +* writeall() (ossaudiodev.oss_audio_device method): Audio Device Objects. + (line 51) +* writeframes() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 187) +* writeframes() (sunau.AU_write method): AU_write Objects. (line 52) +* writeframes() (wave.Wave_write method): Wave_write Objects. + (line 75) +* writeframesraw() (aifc.aifc method): aifc — Read and write AIFF and AIFC files. + (line 195) +* writeframesraw() (sunau.AU_write method): AU_write Objects. + (line 45) +* writeframesraw() (wave.Wave_write method): Wave_write Objects. + (line 68) +* writeheader() (csv.DictWriter method): Writer Objects. (line 37) +* writelines() (asyncio.StreamWriter method): StreamWriter. (line 26) +* writelines() (asyncio.WriteTransport method): Write-only Transports. + (line 67) +* writelines() (codecs.StreamWriter method): StreamWriter Objects. + (line 35) +* writelines() (io.IOBase method): I/O Base Classes. (line 158) +* writePlist() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 142) +* writePlistToBytes() (in module plistlib): plistlib — Generate and parse Mac OS X plist files. + (line 161) +* writepy() (zipfile.PyZipFile method): PyZipFile Objects. (line 20) +* writer (formatter.formatter attribute): The Formatter Interface. + (line 22) +* writer() (in module csv): Module Contents<3>. (line 40) +* writerow() (csv.csvwriter method): Writer Objects. (line 15) +* writerows() (csv.csvwriter method): Writer Objects. (line 23) +* writestr() (zipfile.ZipFile method): ZipFile Objects. (line 266) +* WriteTransport (class in asyncio): Transports Hierarchy. + (line 11) +* writev() (in module os): File Descriptor Operations. + (line 695) +* writexml() (xml.dom.minidom.Node method): DOM Objects. (line 27) +* write_byte() (mmap.mmap method): mmap — Memory-mapped file support. + (line 294) +* write_bytes() (pathlib.Path method): Methods<2>. (line 437) +* write_docstringdict() (in module turtle): Translation of docstrings into different languages. + (line 10) +* write_eof() (asyncio.StreamWriter method): StreamWriter. (line 52) +* write_eof() (asyncio.WriteTransport method): Write-only Transports. + (line 74) +* write_eof() (ssl.MemoryBIO method): Memory BIO Support<2>. + (line 161) +* write_file() (in module distutils.file_util): distutils file_util — Single file operations. + (line 44) +* write_history_file() (in module readline): History file. (line 14) +* WRITE_RESTRICTED: Generic Attribute Management. + (line 92) +* write_results() (trace.CoverageResults method): Programmatic Interface. + (line 57) +* write_text() (pathlib.Path method): Methods<2>. (line 452) +* write_through (io.TextIOWrapper attribute): Text I/O<2>. (line 166) +* writing; values: Expression statements. + (line 17) +* WrongDocumentErr: Exceptions<16>. (line 97) +* wsgiref (module): wsgiref — WSGI Utilities and Reference Implementation. + (line 6) +* wsgiref.handlers (module): wsgiref handlers – server/gateway base classes. + (line 6) +* wsgiref.headers (module): wsgiref headers – WSGI response header tools. + (line 6) +* wsgiref.simple_server (module): wsgiref simple_server – a simple WSGI HTTP server. + (line 6) +* wsgiref.util (module): wsgiref util – WSGI environment utilities. + (line 6) +* wsgiref.validate (module): wsgiref validate — WSGI conformance checker. + (line 6) +* WSGIRequestHandler (class in wsgiref.simple_server): wsgiref simple_server – a simple WSGI HTTP server. + (line 75) +* WSGIServer (class in wsgiref.simple_server): wsgiref simple_server – a simple WSGI HTTP server. + (line 45) +* wsgi_file_wrapper (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 269) +* wsgi_multiprocess (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 153) +* wsgi_multithread (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 146) +* wsgi_run_once (wsgiref.handlers.BaseHandler attribute): wsgiref handlers – server/gateway base classes. + (line 160) +* wShowWindow (subprocess.STARTUPINFO attribute): Windows Popen Helpers. + (line 49) +* WSTOPPED (in module os): Process Management. (line 747) +* WSTOPSIG() (in module os): Process Management. (line 912) +* wstring_at() (in module ctypes): Utility functions. (line 214) +* ws_comma (2to3 fixer): Fixers. (line 325) +* WTERMSIG() (in module os): Process Management. (line 921) +* WUNTRACED (in module os): Process Management. (line 845) +* WWW: Internet Protocols and Support. + (line 6) +* WWW <1>: urllib parse — Parse URLs into components. + (line 8) +* WWW <2>: urllib robotparser — Parser for robots txt. + (line 8) +* WWW; server: cgi — Common Gateway Interface support. + (line 8) +* WWW; server <1>: http server — HTTP servers. + (line 8) +* W_OK (in module os): Files and Directories. + (line 106) +* X (in module re): Module Contents. (line 120) +* X509 certificate: SSL Contexts. (line 668) +* xatom() (imaplib.IMAP4 method): IMAP4 Objects. (line 369) +* XATTR_CREATE (in module os): Linux extended attributes. + (line 87) +* XATTR_REPLACE (in module os): Linux extended attributes. + (line 93) +* XATTR_SIZE_MAX (in module os): Linux extended attributes. + (line 82) +* xcor() (in module turtle): Tell Turtle’s state. + (line 34) +* XDR: xdrlib — Encode and decode XDR data. + (line 8) +* xdrlib (module): xdrlib — Encode and decode XDR data. + (line 6) +* xhdr() (nntplib.NNTP method): Methods<3>. (line 320) +* XHTML: html parser — Simple HTML and XHTML parser. + (line 8) +* XHTML_NAMESPACE (in module xml.dom): Module Contents<4>. (line 53) +* xml (module): XML Processing Modules. + (line 6) +* XML() (in module xml.etree.ElementTree): Functions<6>. (line 253) +* xml.dom (module): xml dom — The Document Object Model API. + (line 6) +* xml.dom.minidom (module): xml dom minidom — Minimal DOM implementation. + (line 6) +* xml.dom.pulldom (module): xml dom pulldom — Support for building partial DOM trees. + (line 6) +* xml.etree.ElementInclude.default_loader() (in module xml.etree.ElementTree): Functions<7>. + (line 6) +* xml.etree.ElementInclude.include() (in module xml.etree.ElementTree): Functions<7>. + (line 17) +* xml.etree.ElementTree (module): xml etree ElementTree — The ElementTree XML API. + (line 6) +* xml.parsers.expat (module): xml parsers expat — Fast XML parsing using Expat. + (line 6) +* xml.parsers.expat.errors (module): Expat error constants. + (line 6) +* xml.parsers.expat.model (module): Content Model Descriptions. + (line 6) +* xml.sax (module): xml sax — Support for SAX2 parsers. + (line 6) +* xml.sax.handler (module): xml sax handler — Base classes for SAX handlers. + (line 6) +* xml.sax.saxutils (module): xml sax saxutils — SAX Utilities. + (line 6) +* xml.sax.xmlreader (module): xml sax xmlreader — Interface for XML parsers. + (line 6) +* xmlcharrefreplace_errors() (in module codecs): Error Handlers. + (line 145) +* XmlDeclHandler() (xml.parsers.expat.xmlparser method): XMLParser Objects<2>. + (line 175) +* XMLFilterBase (class in xml.sax.saxutils): xml sax saxutils — SAX Utilities. + (line 68) +* XMLGenerator (class in xml.sax.saxutils): xml sax saxutils — SAX Utilities. + (line 52) +* XMLID() (in module xml.etree.ElementTree): Functions<6>. (line 261) +* XMLNS_NAMESPACE (in module xml.dom): Module Contents<4>. (line 47) +* XMLParser (class in xml.etree.ElementTree): XMLParser Objects. + (line 6) +* XMLParserType (in module xml.parsers.expat): xml parsers expat — Fast XML parsing using Expat. + (line 35) +* XMLPullParser (class in xml.etree.ElementTree): XMLPullParser Objects. + (line 6) +* XMLReader (class in xml.sax.xmlreader): xml sax xmlreader — Interface for XML parsers. + (line 16) +* xmlrpc.client (module): xmlrpc client — XML-RPC client access. + (line 6) +* xmlrpc.server (module): xmlrpc server — Basic XML-RPC servers. + (line 6) +* XML_ERROR_ABORTED (in module xml.parsers.expat.errors): Expat error constants. + (line 180) +* XML_ERROR_ASYNC_ENTITY (in module xml.parsers.expat.errors): Expat error constants. + (line 29) +* XML_ERROR_ATTRIBUTE_EXTERNAL_ENTITY_REF (in module xml.parsers.expat.errors): Expat error constants. + (line 31) +* XML_ERROR_BAD_CHAR_REF (in module xml.parsers.expat.errors): Expat error constants. + (line 37) +* XML_ERROR_BINARY_ENTITY_REF (in module xml.parsers.expat.errors): Expat error constants. + (line 42) +* XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING (in module xml.parsers.expat.errors): Expat error constants. + (line 135) +* XML_ERROR_DUPLICATE_ATTRIBUTE (in module xml.parsers.expat.errors): Expat error constants. + (line 47) +* XML_ERROR_ENTITY_DECLARED_IN_PE (in module xml.parsers.expat.errors): Expat error constants. + (line 126) +* XML_ERROR_EXTERNAL_ENTITY_HANDLING (in module xml.parsers.expat.errors): Expat error constants. + (line 116) +* XML_ERROR_FEATURE_REQUIRES_XML_DTD (in module xml.parsers.expat.errors): Expat error constants. + (line 128) +* XML_ERROR_FINISHED (in module xml.parsers.expat.errors): Expat error constants. + (line 184) +* XML_ERROR_INCOMPLETE_PE (in module xml.parsers.expat.errors): Expat error constants. + (line 152) +* XML_ERROR_INCORRECT_ENCODING (in module xml.parsers.expat.errors): Expat error constants. + (line 51) +* XML_ERROR_INVALID_TOKEN (in module xml.parsers.expat.errors): Expat error constants. + (line 53) +* XML_ERROR_JUNK_AFTER_DOC_ELEMENT (in module xml.parsers.expat.errors): Expat error constants. + (line 59) +* XML_ERROR_MISPLACED_XML_PI (in module xml.parsers.expat.errors): Expat error constants. + (line 64) +* XML_ERROR_NOT_STANDALONE (in module xml.parsers.expat.errors): Expat error constants. + (line 118) +* XML_ERROR_NOT_SUSPENDED (in module xml.parsers.expat.errors): Expat error constants. + (line 175) +* XML_ERROR_NO_ELEMENTS (in module xml.parsers.expat.errors): Expat error constants. + (line 69) +* XML_ERROR_NO_MEMORY (in module xml.parsers.expat.errors): Expat error constants. + (line 74) +* XML_ERROR_PARAM_ENTITY_REF (in module xml.parsers.expat.errors): Expat error constants. + (line 78) +* XML_ERROR_PARTIAL_CHAR (in module xml.parsers.expat.errors): Expat error constants. + (line 82) +* XML_ERROR_PUBLICID (in module xml.parsers.expat.errors): Expat error constants. + (line 165) +* XML_ERROR_RECURSIVE_ENTITY_REF (in module xml.parsers.expat.errors): Expat error constants. + (line 86) +* XML_ERROR_SUSPENDED (in module xml.parsers.expat.errors): Expat error constants. + (line 169) +* XML_ERROR_SUSPEND_PE (in module xml.parsers.expat.errors): Expat error constants. + (line 190) +* XML_ERROR_SYNTAX (in module xml.parsers.expat.errors): Expat error constants. + (line 91) +* XML_ERROR_TAG_MISMATCH (in module xml.parsers.expat.errors): Expat error constants. + (line 95) +* XML_ERROR_TEXT_DECL (in module xml.parsers.expat.errors): Expat error constants. + (line 160) +* XML_ERROR_UNBOUND_PREFIX (in module xml.parsers.expat.errors): Expat error constants. + (line 142) +* XML_ERROR_UNCLOSED_CDATA_SECTION (in module xml.parsers.expat.errors): Expat error constants. + (line 112) +* XML_ERROR_UNCLOSED_TOKEN (in module xml.parsers.expat.errors): Expat error constants. + (line 99) +* XML_ERROR_UNDECLARING_PREFIX (in module xml.parsers.expat.errors): Expat error constants. + (line 147) +* XML_ERROR_UNDEFINED_ENTITY (in module xml.parsers.expat.errors): Expat error constants. + (line 104) +* XML_ERROR_UNEXPECTED_STATE (in module xml.parsers.expat.errors): Expat error constants. + (line 124) +* XML_ERROR_UNKNOWN_ENCODING (in module xml.parsers.expat.errors): Expat error constants. + (line 108) +* XML_ERROR_XML_DECL (in module xml.parsers.expat.errors): Expat error constants. + (line 156) +* XML_NAMESPACE (in module xml.dom): Module Contents<4>. (line 42) +* xor() (in module operator): operator — Standard operators as functions. + (line 164) +* xover() (nntplib.NNTP method): Methods<3>. (line 336) +* xpath() (nntplib.NNTP method): Methods<3>. (line 344) +* xrange (2to3 fixer): Fixers. (line 330) +* xreadlines (2to3 fixer): Fixers. (line 335) +* xview() (tkinter.ttk.Treeview method): ttk Treeview. (line 327) +* X_OK (in module os): Files and Directories. + (line 106) +* ycor() (in module turtle): Tell Turtle’s state. + (line 46) +* year (datetime.date attribute): date Objects. (line 102) +* year (datetime.datetime attribute): datetime Objects. (line 264) +* Year 2038: time — Time access and conversions. + (line 30) +* yeardatescalendar() (calendar.Calendar method): calendar — General calendar-related functions. + (line 106) +* yeardays2calendar() (calendar.Calendar method): calendar — General calendar-related functions. + (line 114) +* yeardayscalendar() (calendar.Calendar method): calendar — General calendar-related functions. + (line 121) +* YESEXPR (in module locale): locale — Internationalization services. + (line 241) +* yield; examples: Generator-iterator methods. + (line 57) +* yield; expression: Yield expressions. (line 6) +* yield; yield from (in What’s New): PEP 3151 Reworking the OS and IO exception hierarchy. + (line 87) +* YIELD_FROM (opcode): Python Bytecode Instructions. + (line 350) +* YIELD_VALUE (opcode): Python Bytecode Instructions. + (line 346) +* yiq_to_rgb() (in module colorsys): colorsys — Conversions between color systems. + (line 32) +* yview() (tkinter.ttk.Treeview method): ttk Treeview. (line 331) +* Zen of Python: Glossary. (line 1325) +* ZeroDivisionError: Concrete exceptions. + (line 384) +* zfill() (bytearray method): Bytes and Bytearray Operations. + (line 683) +* zfill() (bytes method): Bytes and Bytearray Operations. + (line 683) +* zfill() (str method): String Methods<2>. (line 604) +* zip (2to3 fixer): Fixers. (line 339) +* zip() (built-in function): Built-in Functions. (line 1723) +* zipapp (module): zipapp — Manage executable Python zip archives. + (line 6) +* zipapp command line option; -c: Command-Line Interface<5>. + (line 45) +* zipapp command line option; –compress: Command-Line Interface<5>. + (line 45) +* zipapp command line option; -h: Command-Line Interface<5>. + (line 61) +* zipapp command line option; –help: Command-Line Interface<5>. + (line 61) +* zipapp command line option; –info: Command-Line Interface<5>. + (line 55) +* zipapp command line option; -m : Command-Line Interface<5>. + (line 35) +* zipapp command line option; –main=: Command-Line Interface<5>. + (line 35) +* zipapp command line option; -o : Command-Line Interface<5>. + (line 18) +* zipapp command line option; –output=: Command-Line Interface<5>. + (line 18) +* zipapp command line option; -p : Command-Line Interface<5>. + (line 29) +* zipapp command line option; –python=: Command-Line Interface<5>. + (line 29) +* ZipFile (class in zipfile): ZipFile Objects. (line 6) +* zipfile (module): zipfile — Work with ZIP archives. + (line 6) +* zipfile command line option; -c ... : Command-line options. + (line 11) +* zipfile command line option; –create ... : Command-line options. + (line 11) +* zipfile command line option; -e : Command-line options. + (line 16) +* zipfile command line option; –extract : Command-line options. + (line 16) +* zipfile command line option; -l : Command-line options. + (line 6) +* zipfile command line option; –list : Command-line options. + (line 6) +* zipfile command line option; -t : Command-line options. + (line 21) +* zipfile command line option; –test : Command-line options. + (line 21) +* zipimport (module): zipimport — Import modules from Zip archives. + (line 6) +* zipimporter (class in zipimport): zipimporter Objects. + (line 8) +* ZipImportError: zipimport — Import modules from Zip archives. + (line 55) +* ZipInfo (class in zipfile): zipfile — Work with ZIP archives. + (line 58) +* ZIP_BZIP2 (in module zipfile): zipfile — Work with ZIP archives. + (line 88) +* ZIP_DEFLATED (in module zipfile): zipfile — Work with ZIP archives. + (line 83) +* zip_longest() (in module itertools): Itertool functions. (line 586) +* ZIP_LZMA (in module zipfile): zipfile — Work with ZIP archives. + (line 95) +* ZIP_STORED (in module zipfile): zipfile — Work with ZIP archives. + (line 79) +* zlib (module): zlib — Compression compatible with gzip. + (line 6) +* ZLIB_RUNTIME_VERSION (in module zlib): zlib — Compression compatible with gzip. + (line 309) +* ZLIB_VERSION (in module zlib): zlib — Compression compatible with gzip. + (line 302) + + + + + + +Tag Table: +Node: Top344 +Ref: contents doc545 +Ref: 145545 +Node: What’s New in Python166588 +Ref: whatsnew/index doc166690 +Ref: 146166690 +Ref: whatsnew/index python-documentation-contents166690 +Ref: 147166690 +Ref: whatsnew/index what-s-new-in-python166690 +Ref: 148166690 +Ref: whatsnew/index whatsnew-index166690 +Ref: 149166690 +Node: What’s New In Python 3 8167965 +Ref: whatsnew/3 8 doc168085 +Ref: 14a168085 +Ref: whatsnew/3 8 what-s-new-in-python-3-8168085 +Ref: 14b168085 +Node: Summary – Release highlights168946 +Ref: whatsnew/3 8 summary-release-highlights169060 +Ref: 14d169060 +Node: New Features169133 +Ref: whatsnew/3 8 new-features169278 +Ref: 14e169278 +Node: Assignment expressions169884 +Ref: whatsnew/3 8 assignment-expressions169990 +Ref: 14f169990 +Ref: Assignment expressions-Footnote-1171496 +Ref: Assignment expressions-Footnote-2171593 +Ref: Assignment expressions-Footnote-3171642 +Node: Positional-only parameters171685 +Ref: whatsnew/3 8 positional-only-parameters171853 +Ref: 151171853 +Ref: Positional-only parameters-Footnote-1174535 +Ref: Positional-only parameters-Footnote-2174587 +Ref: Positional-only parameters-Footnote-3174636 +Node: Parallel filesystem cache for compiled bytecode files174679 +Ref: whatsnew/3 8 parallel-filesystem-cache-for-compiled-bytecode-files174871 +Ref: 153174871 +Ref: Parallel filesystem cache for compiled bytecode files-Footnote-1175513 +Node: Debug build uses the same ABI as release build175556 +Ref: whatsnew/3 8 debug-build-uses-the-same-abi-as-release-build175788 +Ref: 158175788 +Ref: Debug build uses the same ABI as release build-Footnote-1178123 +Ref: Debug build uses the same ABI as release build-Footnote-2178166 +Ref: Debug build uses the same ABI as release build-Footnote-3178209 +Ref: Debug build uses the same ABI as release build-Footnote-4178252 +Node: f-strings support = for self-documenting expressions and debugging178295 +Ref: whatsnew/3 8 f-strings-support-for-self-documenting-expressions-and-debugging178508 +Ref: 15a178508 +Ref: f-strings support = for self-documenting expressions and debugging-Footnote-1179597 +Node: PEP 578 Python Runtime Audit Hooks179640 +Ref: whatsnew/3 8 pep-578-python-runtime-audit-hooks179850 +Ref: 15d179850 +Ref: PEP 578 Python Runtime Audit Hooks-Footnote-1180333 +Node: PEP 587 Python Initialization Configuration180382 +Ref: whatsnew/3 8 pep-587-python-initialization-configuration180572 +Ref: 15e180572 +Ref: PEP 587 Python Initialization Configuration-Footnote-1182559 +Ref: PEP 587 Python Initialization Configuration-Footnote-2182608 +Ref: PEP 587 Python Initialization Configuration-Footnote-3182657 +Node: Vectorcall a fast calling protocol for CPython182700 +Ref: whatsnew/3 8 vectorcall-a-fast-calling-protocol-for-cpython182903 +Ref: 17e182903 +Ref: Vectorcall a fast calling protocol for CPython-Footnote-1183463 +Ref: Vectorcall a fast calling protocol for CPython-Footnote-2183512 +Node: Pickle protocol 5 with out-of-band data buffers183555 +Ref: whatsnew/3 8 pickle-protocol-5-with-out-of-band-data-buffers183706 +Ref: 17f183706 +Ref: Pickle protocol 5 with out-of-band data buffers-Footnote-1184458 +Ref: Pickle protocol 5 with out-of-band data buffers-Footnote-2184507 +Ref: Pickle protocol 5 with out-of-band data buffers-Footnote-3184556 +Node: Other Language Changes184599 +Ref: whatsnew/3 8 other-language-changes184725 +Ref: 180184725 +Ref: Other Language Changes-Footnote-1192102 +Ref: Other Language Changes-Footnote-2192145 +Ref: Other Language Changes-Footnote-3192188 +Ref: Other Language Changes-Footnote-4192231 +Ref: Other Language Changes-Footnote-5192274 +Ref: Other Language Changes-Footnote-6192317 +Ref: Other Language Changes-Footnote-7192360 +Ref: Other Language Changes-Footnote-8192403 +Ref: Other Language Changes-Footnote-9192446 +Ref: Other Language Changes-Footnote-10192489 +Ref: Other Language Changes-Footnote-11192533 +Ref: Other Language Changes-Footnote-12192579 +Ref: Other Language Changes-Footnote-13192623 +Ref: Other Language Changes-Footnote-14192693 +Ref: Other Language Changes-Footnote-15192753 +Ref: Other Language Changes-Footnote-16192797 +Ref: Other Language Changes-Footnote-17192841 +Node: New Modules192885 +Ref: whatsnew/3 8 new-modules193015 +Ref: 19d193015 +Ref: New Modules-Footnote-1194088 +Node: Improved Modules194131 +Ref: whatsnew/3 8 improved-modules194252 +Ref: 19e194252 +Node: ast194876 +Ref: whatsnew/3 8 ast194948 +Ref: 19f194948 +Ref: ast-Footnote-1195980 +Ref: ast-Footnote-2196023 +Ref: ast-Footnote-3196072 +Ref: ast-Footnote-4196121 +Ref: ast-Footnote-5196170 +Node: asyncio196213 +Ref: whatsnew/3 8 asyncio196302 +Ref: 1a4196302 +Ref: asyncio-Footnote-1199458 +Ref: asyncio-Footnote-2199501 +Ref: asyncio-Footnote-3199544 +Ref: asyncio-Footnote-4199587 +Ref: asyncio-Footnote-5199630 +Ref: asyncio-Footnote-6199673 +Ref: asyncio-Footnote-7199716 +Ref: asyncio-Footnote-8199759 +Ref: asyncio-Footnote-9199802 +Ref: asyncio-Footnote-10199855 +Node: builtins199899 +Ref: whatsnew/3 8 builtins199996 +Ref: 1b3199996 +Ref: builtins-Footnote-1200495 +Node: collections200538 +Ref: whatsnew/3 8 collections200636 +Ref: 1b5200636 +Ref: collections-Footnote-1201158 +Node: cProfile201201 +Ref: whatsnew/3 8 cprofile201294 +Ref: 1ba201294 +Ref: cProfile-Footnote-1201644 +Node: csv201687 +Ref: whatsnew/3 8 csv201775 +Ref: 1bc201775 +Ref: csv-Footnote-1202092 +Node: curses202135 +Ref: whatsnew/3 8 curses202221 +Ref: 1be202221 +Ref: curses-Footnote-1202465 +Node: ctypes202508 +Ref: whatsnew/3 8 ctypes202599 +Ref: 1c0202599 +Ref: ctypes-Footnote-1203088 +Node: datetime203131 +Ref: whatsnew/3 8 datetime203225 +Ref: 1c3203225 +Ref: datetime-Footnote-1203653 +Node: functools203696 +Ref: whatsnew/3 8 functools203786 +Ref: 1c6203786 +Ref: functools-Footnote-1205328 +Ref: functools-Footnote-2205371 +Ref: functools-Footnote-3205414 +Node: gc205457 +Ref: whatsnew/3 8 gc205546 +Ref: 1cc205546 +Ref: gc-Footnote-1205779 +Node: gettext205822 +Ref: whatsnew/3 8 gettext205906 +Ref: 1ce205906 +Ref: gettext-Footnote-1206107 +Node: gzip206149 +Ref: whatsnew/3 8 gzip206247 +Ref: 1d0206247 +Ref: gzip-Footnote-1206664 +Ref: gzip-Footnote-2206707 +Node: IDLE and idlelib206749 +Ref: whatsnew/3 8 idle-and-idlelib206847 +Ref: 1d4206847 +Ref: IDLE and idlelib-Footnote-1208816 +Ref: IDLE and idlelib-Footnote-2208861 +Ref: IDLE and idlelib-Footnote-3208903 +Ref: IDLE and idlelib-Footnote-4208946 +Ref: IDLE and idlelib-Footnote-5208989 +Ref: IDLE and idlelib-Footnote-6209032 +Ref: IDLE and idlelib-Footnote-7209074 +Ref: IDLE and idlelib-Footnote-8209117 +Node: inspect209160 +Ref: whatsnew/3 8 inspect209256 +Ref: 1d5209256 +Ref: inspect-Footnote-1210022 +Node: io210065 +Ref: whatsnew/3 8 io210154 +Ref: 1da210154 +Ref: io-Footnote-1210490 +Node: itertools210533 +Ref: whatsnew/3 8 itertools210624 +Ref: 1dc210624 +Ref: itertools-Footnote-1211002 +Node: json tool211045 +Ref: whatsnew/3 8 json-tool211141 +Ref: 1de211141 +Ref: json tool-Footnote-1211348 +Node: logging211391 +Ref: whatsnew/3 8 logging211482 +Ref: 1df211482 +Ref: logging-Footnote-1212198 +Node: math212241 +Ref: whatsnew/3 8 math212327 +Ref: 1e1212327 +Ref: math-Footnote-1214076 +Ref: math-Footnote-2214119 +Ref: math-Footnote-3214162 +Ref: math-Footnote-4214205 +Ref: math-Footnote-5214248 +Ref: math-Footnote-6214291 +Ref: math-Footnote-7214334 +Ref: math-Footnote-8214377 +Node: mmap214420 +Ref: whatsnew/3 8 mmap214514 +Ref: 1eb214514 +Ref: mmap-Footnote-1214744 +Node: multiprocessing214787 +Ref: whatsnew/3 8 multiprocessing214879 +Ref: 1ee214879 +Ref: multiprocessing-Footnote-1215183 +Ref: multiprocessing-Footnote-2215226 +Node: os215269 +Ref: whatsnew/3 8 os215364 +Ref: 1ef215364 +Ref: os-Footnote-1216872 +Ref: os-Footnote-2216915 +Ref: os-Footnote-3216958 +Node: os path217001 +Ref: whatsnew/3 8 os-path217088 +Ref: 1f5217088 +Ref: os path-Footnote-1218111 +Ref: os path-Footnote-2218154 +Ref: os path-Footnote-3218197 +Node: pathlib218240 +Ref: whatsnew/3 8 pathlib218331 +Ref: 200218331 +Ref: pathlib-Footnote-1219057 +Ref: pathlib-Footnote-2219100 +Node: pickle219143 +Ref: whatsnew/3 8 pickle219235 +Ref: 20c219235 +Ref: pickle-Footnote-1219573 +Node: plistlib219616 +Ref: whatsnew/3 8 plistlib219707 +Ref: 20f219707 +Ref: plistlib-Footnote-1219945 +Node: pprint219988 +Ref: whatsnew/3 8 pprint220083 +Ref: 211220083 +Ref: pprint-Footnote-1221148 +Node: py_compile221191 +Ref: whatsnew/3 8 py-compile221283 +Ref: 214221283 +Ref: py_compile-Footnote-1221472 +Node: shlex221515 +Ref: whatsnew/3 8 shlex221607 +Ref: 216221607 +Ref: shlex-Footnote-1221811 +Node: shutil221854 +Ref: whatsnew/3 8 shutil221942 +Ref: 219221942 +Ref: shutil-Footnote-1222596 +Ref: shutil-Footnote-2222639 +Ref: shutil-Footnote-3222682 +Node: socket222725 +Ref: whatsnew/3 8 socket222811 +Ref: 21d222811 +Ref: socket-Footnote-1223386 +Ref: socket-Footnote-2223429 +Node: ssl223472 +Ref: whatsnew/3 8 ssl223562 +Ref: 223223562 +Ref: ssl-Footnote-1223824 +Node: statistics223867 +Ref: whatsnew/3 8 statistics223954 +Ref: 226223954 +Ref: statistics-Footnote-1225685 +Ref: statistics-Footnote-2225728 +Ref: statistics-Footnote-3225771 +Ref: statistics-Footnote-4225814 +Ref: statistics-Footnote-5225857 +Node: sys225900 +Ref: whatsnew/3 8 sys225991 +Ref: 22c225991 +Ref: sys-Footnote-1226434 +Node: tarfile226477 +Ref: whatsnew/3 8 tarfile226567 +Ref: 22f226567 +Ref: tarfile-Footnote-1226986 +Node: threading227029 +Ref: whatsnew/3 8 threading227124 +Ref: 230227124 +Ref: threading-Footnote-1227860 +Ref: threading-Footnote-2227905 +Node: tokenize227948 +Ref: whatsnew/3 8 tokenize228043 +Ref: 236228043 +Ref: tokenize-Footnote-1228369 +Node: tkinter228412 +Ref: whatsnew/3 8 tkinter228502 +Ref: 237228502 +Ref: tkinter-Footnote-1229045 +Ref: tkinter-Footnote-2229088 +Ref: tkinter-Footnote-3229131 +Node: time229174 +Ref: whatsnew/3 8 time229262 +Ref: 238229262 +Ref: time-Footnote-1229441 +Node: typing229484 +Ref: whatsnew/3 8 typing229576 +Ref: 23a229576 +Ref: typing-Footnote-1231050 +Ref: typing-Footnote-2231099 +Ref: typing-Footnote-3231148 +Ref: typing-Footnote-4231197 +Node: unicodedata231246 +Ref: whatsnew/3 8 unicodedata231342 +Ref: 245231342 +Ref: unicodedata-Footnote-1231771 +Ref: unicodedata-Footnote-2231832 +Ref: unicodedata-Footnote-3231875 +Node: unittest231918 +Ref: whatsnew/3 8 unittest232012 +Ref: 247232012 +Ref: unittest-Footnote-1233302 +Ref: unittest-Footnote-2233345 +Ref: unittest-Footnote-3233388 +Ref: unittest-Footnote-4233431 +Node: venv233474 +Ref: whatsnew/3 8 venv233564 +Ref: 24e233564 +Ref: venv-Footnote-1233816 +Node: weakref233859 +Ref: whatsnew/3 8 weakref233944 +Ref: 24f233944 +Ref: weakref-Footnote-1234237 +Node: xml234280 +Ref: whatsnew/3 8 xml234367 +Ref: 251234367 +Ref: xml-Footnote-1235541 +Ref: xml-Footnote-2235584 +Ref: xml-Footnote-3235627 +Ref: xml-Footnote-4235670 +Ref: xml-Footnote-5235713 +Node: xmlrpc235756 +Ref: whatsnew/3 8 xmlrpc235827 +Ref: 254235827 +Ref: xmlrpc-Footnote-1236219 +Node: Optimizations236262 +Ref: whatsnew/3 8 optimizations236395 +Ref: 256236395 +Ref: Optimizations-Footnote-1240762 +Ref: Optimizations-Footnote-2240805 +Ref: Optimizations-Footnote-3240848 +Ref: Optimizations-Footnote-4240891 +Ref: Optimizations-Footnote-5240934 +Ref: Optimizations-Footnote-6240977 +Ref: Optimizations-Footnote-7241020 +Ref: Optimizations-Footnote-8241063 +Ref: Optimizations-Footnote-9241106 +Ref: Optimizations-Footnote-10241149 +Ref: Optimizations-Footnote-11241193 +Ref: Optimizations-Footnote-12241237 +Ref: Optimizations-Footnote-13241281 +Node: Build and C API Changes241325 +Ref: whatsnew/3 8 build-and-c-api-changes241452 +Ref: 263241452 +Ref: Build and C API Changes-Footnote-1247043 +Ref: Build and C API Changes-Footnote-2247086 +Ref: Build and C API Changes-Footnote-3247129 +Ref: Build and C API Changes-Footnote-4247172 +Ref: Build and C API Changes-Footnote-5247215 +Ref: Build and C API Changes-Footnote-6247258 +Ref: Build and C API Changes-Footnote-7247301 +Ref: Build and C API Changes-Footnote-8247344 +Ref: Build and C API Changes-Footnote-9247387 +Ref: Build and C API Changes-Footnote-10247430 +Ref: Build and C API Changes-Footnote-11247474 +Ref: Build and C API Changes-Footnote-12247518 +Node: Deprecated247562 +Ref: whatsnew/3 8 deprecated247700 +Ref: 277247700 +Ref: Deprecated-Footnote-1253564 +Ref: Deprecated-Footnote-2253607 +Ref: Deprecated-Footnote-3253650 +Ref: Deprecated-Footnote-4253693 +Ref: Deprecated-Footnote-5253735 +Ref: Deprecated-Footnote-6253778 +Ref: Deprecated-Footnote-7253821 +Ref: Deprecated-Footnote-8253864 +Ref: Deprecated-Footnote-9253907 +Ref: Deprecated-Footnote-10253950 +Ref: Deprecated-Footnote-11253994 +Ref: Deprecated-Footnote-12254038 +Ref: Deprecated-Footnote-13254082 +Node: API and Feature Removals254126 +Ref: whatsnew/3 8 api-and-feature-removals254262 +Ref: 2a8254262 +Ref: API and Feature Removals-Footnote-1257096 +Ref: API and Feature Removals-Footnote-2257139 +Ref: API and Feature Removals-Footnote-3257182 +Ref: API and Feature Removals-Footnote-4257225 +Ref: API and Feature Removals-Footnote-5257268 +Ref: API and Feature Removals-Footnote-6257311 +Ref: API and Feature Removals-Footnote-7257354 +Ref: API and Feature Removals-Footnote-8257397 +Ref: API and Feature Removals-Footnote-9257440 +Ref: API and Feature Removals-Footnote-10257483 +Ref: API and Feature Removals-Footnote-11257527 +Node: Porting to Python 3 8257571 +Ref: whatsnew/3 8 porting-to-python-3-8257728 +Ref: 2ae257728 +Node: Changes in Python behavior258039 +Ref: whatsnew/3 8 changes-in-python-behavior258157 +Ref: 2af258157 +Ref: Changes in Python behavior-Footnote-1260537 +Ref: Changes in Python behavior-Footnote-2260580 +Ref: Changes in Python behavior-Footnote-3260623 +Ref: Changes in Python behavior-Footnote-4260666 +Ref: Changes in Python behavior-Footnote-5260709 +Ref: Changes in Python behavior-Footnote-6260752 +Node: Changes in the Python API260795 +Ref: whatsnew/3 8 changes-in-the-python-api260942 +Ref: 2b7260942 +Ref: whatsnew/3 8 bpo-36085-whatsnew267908 +Ref: 2ca267908 +Ref: Changes in the Python API-Footnote-1269458 +Ref: Changes in the Python API-Footnote-2269507 +Ref: Changes in the Python API-Footnote-3269550 +Ref: Changes in the Python API-Footnote-4269593 +Ref: Changes in the Python API-Footnote-5269636 +Ref: Changes in the Python API-Footnote-6269679 +Ref: Changes in the Python API-Footnote-7269722 +Ref: Changes in the Python API-Footnote-8269765 +Ref: Changes in the Python API-Footnote-9269808 +Ref: Changes in the Python API-Footnote-10269851 +Ref: Changes in the Python API-Footnote-11269895 +Ref: Changes in the Python API-Footnote-12269939 +Ref: Changes in the Python API-Footnote-13269983 +Ref: Changes in the Python API-Footnote-14270027 +Ref: Changes in the Python API-Footnote-15270071 +Ref: Changes in the Python API-Footnote-16270115 +Ref: Changes in the Python API-Footnote-17270159 +Ref: Changes in the Python API-Footnote-18270202 +Ref: Changes in the Python API-Footnote-19270246 +Ref: Changes in the Python API-Footnote-20270290 +Ref: Changes in the Python API-Footnote-21270334 +Ref: Changes in the Python API-Footnote-22270378 +Ref: Changes in the Python API-Footnote-23270422 +Ref: Changes in the Python API-Footnote-24270466 +Ref: Changes in the Python API-Footnote-25270510 +Ref: Changes in the Python API-Footnote-26270554 +Ref: Changes in the Python API-Footnote-27270598 +Node: Changes in the C API270648 +Ref: whatsnew/3 8 changes-in-the-c-api270793 +Ref: 2cb270793 +Ref: Changes in the C API-Footnote-1276620 +Ref: Changes in the C API-Footnote-2276663 +Ref: Changes in the C API-Footnote-3276706 +Ref: Changes in the C API-Footnote-4276749 +Ref: Changes in the C API-Footnote-5276792 +Ref: Changes in the C API-Footnote-6276835 +Ref: Changes in the C API-Footnote-7276878 +Ref: Changes in the C API-Footnote-8276921 +Node: CPython bytecode changes276964 +Ref: whatsnew/3 8 cpython-bytecode-changes277099 +Ref: 2da277099 +Ref: CPython bytecode changes-Footnote-1278461 +Ref: CPython bytecode changes-Footnote-2278504 +Ref: CPython bytecode changes-Footnote-3278547 +Ref: CPython bytecode changes-Footnote-4278596 +Node: Demos and Tools278639 +Ref: whatsnew/3 8 demos-and-tools278745 +Ref: 2e5278745 +Ref: Demos and Tools-Footnote-1282042 +Ref: Demos and Tools-Footnote-2282085 +Ref: Demos and Tools-Footnote-3282213 +Node: Notable changes in Python 3 8 1282263 +Ref: whatsnew/3 8 notable-changes-in-python-3-8-1282427 +Ref: 2e6282427 +Ref: Notable changes in Python 3 8 1-Footnote-1282938 +Node: Notable changes in Python 3 8 2282981 +Ref: whatsnew/3 8 notable-changes-in-python-3-8-2283155 +Ref: 2e8283155 +Ref: Notable changes in Python 3 8 2-Footnote-1283478 +Node: Notable changes in Python 3 8 3283521 +Ref: whatsnew/3 8 notable-changes-in-python-3-8-3283695 +Ref: 2e9283695 +Ref: Notable changes in Python 3 8 3-Footnote-1284083 +Node: Notable changes in Python 3 8 8284126 +Ref: whatsnew/3 8 notable-changes-in-python-3-8-8284300 +Ref: 2ea284300 +Ref: Notable changes in Python 3 8 8-Footnote-1285030 +Node: Notable changes in Python 3 8 9285073 +Ref: whatsnew/3 8 notable-changes-in-python-3-8-9285207 +Ref: 2ef285207 +Ref: Notable changes in Python 3 8 9-Footnote-1285679 +Node: What’s New In Python 3 7285722 +Ref: whatsnew/3 7 doc285877 +Ref: 2f1285877 +Ref: whatsnew/3 7 what-s-new-in-python-3-7285877 +Ref: 2f2285877 +Node: Summary – Release Highlights287099 +Ref: whatsnew/3 7 summary-release-highlights287216 +Ref: 2f3287216 +Ref: Summary – Release Highlights-Footnote-1289369 +Ref: Summary – Release Highlights-Footnote-2289449 +Ref: Summary – Release Highlights-Footnote-3289485 +Ref: Summary – Release Highlights-Footnote-4289521 +Node: New Features<2>289557 +Ref: whatsnew/3 7 new-features289708 +Ref: 308289708 +Node: PEP 563 Postponed Evaluation of Annotations290806 +Ref: whatsnew/3 7 pep-563-postponed-evaluation-of-annotations290942 +Ref: 309290942 +Ref: whatsnew/3 7 whatsnew37-pep563290942 +Ref: 2f4290942 +Ref: PEP 563 Postponed Evaluation of Annotations-Footnote-1292706 +Ref: PEP 563 Postponed Evaluation of Annotations-Footnote-2292755 +Ref: PEP 563 Postponed Evaluation of Annotations-Footnote-3292804 +Node: PEP 538 Legacy C Locale Coercion292853 +Ref: whatsnew/3 7 pep-538-legacy-c-locale-coercion293031 +Ref: 30b293031 +Ref: whatsnew/3 7 whatsnew37-pep538293031 +Ref: 2ff293031 +Ref: PEP 538 Legacy C Locale Coercion-Footnote-1295418 +Ref: PEP 538 Legacy C Locale Coercion-Footnote-2295467 +Ref: PEP 538 Legacy C Locale Coercion-Footnote-3295516 +Ref: PEP 538 Legacy C Locale Coercion-Footnote-4295565 +Node: PEP 540 Forced UTF-8 Runtime Mode295614 +Ref: whatsnew/3 7 pep-540-forced-utf-8-runtime-mode295776 +Ref: 310295776 +Ref: whatsnew/3 7 whatsnew37-pep540295776 +Ref: 300295776 +Ref: PEP 540 Forced UTF-8 Runtime Mode-Footnote-1297332 +Ref: PEP 540 Forced UTF-8 Runtime Mode-Footnote-2297381 +Ref: PEP 540 Forced UTF-8 Runtime Mode-Footnote-3297430 +Node: PEP 553 Built-in breakpoint297479 +Ref: whatsnew/3 7 pep-553-built-in-breakpoint297651 +Ref: 312297651 +Ref: whatsnew/3 7 whatsnew37-pep553297651 +Ref: 2f8297651 +Ref: PEP 553 Built-in breakpoint-Footnote-1298490 +Node: PEP 539 New C API for Thread-Local Storage298539 +Ref: whatsnew/3 7 pep-539-new-c-api-for-thread-local-storage298730 +Ref: 315298730 +Ref: whatsnew/3 7 whatsnew37-pep539298730 +Ref: 304298730 +Ref: PEP 539 New C API for Thread-Local Storage-Footnote-1300249 +Ref: PEP 539 New C API for Thread-Local Storage-Footnote-2300298 +Node: PEP 562 Customization of Access to Module Attributes300347 +Ref: whatsnew/3 7 pep-562-customization-of-access-to-module-attributes300564 +Ref: 319300564 +Ref: whatsnew/3 7 whatsnew37-pep562300564 +Ref: 2fa300564 +Ref: PEP 562 Customization of Access to Module Attributes-Footnote-1301151 +Node: PEP 564 New Time Functions With Nanosecond Resolution301200 +Ref: whatsnew/3 7 pep-564-new-time-functions-with-nanosecond-resolution301418 +Ref: 31c301418 +Ref: whatsnew/3 7 whatsnew37-pep564301418 +Ref: 2fe301418 +Ref: PEP 564 New Time Functions With Nanosecond Resolution-Footnote-1302504 +Ref: PEP 564 New Time Functions With Nanosecond Resolution-Footnote-2302553 +Ref: PEP 564 New Time Functions With Nanosecond Resolution-Footnote-3302638 +Node: PEP 565 Show DeprecationWarning in __main__302687 +Ref: whatsnew/3 7 pep-565-show-deprecationwarning-in-main302909 +Ref: 324302909 +Ref: whatsnew/3 7 whatsnew37-pep565302909 +Ref: 303302909 +Ref: PEP 565 Show DeprecationWarning in __main__-Footnote-1304841 +Node: PEP 560 Core Support for typing module and Generic Types304890 +Ref: whatsnew/3 7 pep-560-core-support-for-typing-module-and-generic-types305087 +Ref: 326305087 +Ref: whatsnew/3 7 whatsnew37-pep560305087 +Ref: 2fb305087 +Ref: PEP 560 Core Support for typing module and Generic Types-Footnote-1306042 +Ref: PEP 560 Core Support for typing module and Generic Types-Footnote-2306091 +Node: PEP 552 Hash-based pyc Files306140 +Ref: whatsnew/3 7 pep-552-hash-based-pyc-files306335 +Ref: 328306335 +Ref: whatsnew/3 7 whatsnew37-pep552306335 +Ref: 301306335 +Ref: PEP 552 Hash-based pyc Files-Footnote-1308009 +Ref: PEP 552 Hash-based pyc Files-Footnote-2308050 +Ref: PEP 552 Hash-based pyc Files-Footnote-3308099 +Node: PEP 545 Python Documentation Translations308148 +Ref: whatsnew/3 7 pep-545-python-documentation-translations308318 +Ref: 32a308318 +Ref: whatsnew/3 7 whatsnew37-pep545308318 +Ref: 305308318 +Ref: PEP 545 Python Documentation Translations-Footnote-1308902 +Ref: PEP 545 Python Documentation Translations-Footnote-2308951 +Node: Development Runtime Mode -X dev309000 +Ref: whatsnew/3 7 development-runtime-mode-x-dev309133 +Ref: 32b309133 +Ref: whatsnew/3 7 whatsnew37-devmode309133 +Ref: 302309133 +Node: Other Language Changes<2>309601 +Ref: whatsnew/3 7 other-language-changes309736 +Ref: 32d309736 +Ref: Other Language Changes<2>-Footnote-1312125 +Ref: Other Language Changes<2>-Footnote-2312168 +Ref: Other Language Changes<2>-Footnote-3312211 +Ref: Other Language Changes<2>-Footnote-4312254 +Ref: Other Language Changes<2>-Footnote-5312297 +Ref: Other Language Changes<2>-Footnote-6312340 +Ref: Other Language Changes<2>-Footnote-7312383 +Ref: Other Language Changes<2>-Footnote-8312426 +Ref: Other Language Changes<2>-Footnote-9312469 +Ref: Other Language Changes<2>-Footnote-10312512 +Node: New Modules<2>312556 +Ref: whatsnew/3 7 new-modules312695 +Ref: 339312695 +Node: contextvars312818 +Ref: whatsnew/3 7 contextvars312900 +Ref: 33a312900 +Ref: whatsnew/3 7 whatsnew37-pep567312900 +Ref: 2f5312900 +Ref: contextvars-Footnote-1313629 +Node: dataclasses313678 +Ref: whatsnew/3 7 dataclasses313788 +Ref: 33c313788 +Ref: whatsnew/3 7 whatsnew37-pep557313788 +Ref: 2f6313788 +Ref: dataclasses-Footnote-1314459 +Node: importlib resources314508 +Ref: whatsnew/3 7 importlib-resources314598 +Ref: 341314598 +Ref: whatsnew/3 7 whatsnew37-importlib-resources314598 +Ref: 2f7314598 +Ref: importlib resources-Footnote-1315346 +Ref: importlib resources-Footnote-2315389 +Node: Improved Modules<2>315450 +Ref: whatsnew/3 7 improved-modules315577 +Ref: 343315577 +Node: argparse316728 +Ref: whatsnew/3 7 argparse316811 +Ref: 344316811 +Ref: argparse-Footnote-1317045 +Node: asyncio<2>317088 +Ref: whatsnew/3 7 asyncio317188 +Ref: 346317188 +Ref: whatsnew/3 7 whatsnew37-asyncio317188 +Ref: 2fd317188 +Ref: asyncio<2>-Footnote-1323490 +Ref: asyncio<2>-Footnote-2323533 +Ref: asyncio<2>-Footnote-3323582 +Ref: asyncio<2>-Footnote-4323625 +Ref: asyncio<2>-Footnote-5323668 +Ref: asyncio<2>-Footnote-6323711 +Ref: asyncio<2>-Footnote-7323754 +Ref: asyncio<2>-Footnote-8323797 +Ref: asyncio<2>-Footnote-9323840 +Ref: asyncio<2>-Footnote-10323883 +Ref: asyncio<2>-Footnote-11323927 +Ref: asyncio<2>-Footnote-12323971 +Ref: asyncio<2>-Footnote-13324015 +Ref: asyncio<2>-Footnote-14324059 +Ref: asyncio<2>-Footnote-15324103 +Ref: asyncio<2>-Footnote-16324147 +Ref: asyncio<2>-Footnote-17324191 +Ref: asyncio<2>-Footnote-18324235 +Ref: asyncio<2>-Footnote-19324279 +Ref: asyncio<2>-Footnote-20324323 +Ref: asyncio<2>-Footnote-21324367 +Ref: asyncio<2>-Footnote-22324411 +Ref: asyncio<2>-Footnote-23324455 +Ref: asyncio<2>-Footnote-24324499 +Node: binascii324543 +Ref: whatsnew/3 7 binascii324643 +Ref: 36c324643 +Ref: binascii-Footnote-1324922 +Node: calendar324965 +Ref: whatsnew/3 7 calendar325069 +Ref: 36e325069 +Ref: calendar-Footnote-1325319 +Node: collections<2>325362 +Ref: whatsnew/3 7 collections325468 +Ref: 370325468 +Ref: collections<2>-Footnote-1325659 +Node: compileall325702 +Ref: whatsnew/3 7 compileall325818 +Ref: 371325818 +Ref: compileall-Footnote-1326215 +Node: concurrent futures326258 +Ref: whatsnew/3 7 concurrent-futures326370 +Ref: 373326370 +Ref: concurrent futures-Footnote-1326812 +Ref: concurrent futures-Footnote-2326855 +Node: contextlib326898 +Ref: whatsnew/3 7 contextlib327011 +Ref: 374327011 +Ref: contextlib-Footnote-1327541 +Ref: contextlib-Footnote-2327584 +Ref: contextlib-Footnote-3327627 +Ref: contextlib-Footnote-4327670 +Node: cProfile<2>327713 +Ref: whatsnew/3 7 cprofile327813 +Ref: 37a327813 +Ref: cProfile<2>-Footnote-1328039 +Node: crypt328082 +Ref: whatsnew/3 7 crypt328183 +Ref: 37b328183 +Ref: crypt-Footnote-1328515 +Ref: crypt-Footnote-2328558 +Node: datetime<2>328601 +Ref: whatsnew/3 7 datetime328694 +Ref: 37d328694 +Ref: datetime<2>-Footnote-1329105 +Ref: datetime<2>-Footnote-2329148 +Node: dbm329190 +Ref: whatsnew/3 7 dbm329285 +Ref: 381329285 +Node: decimal329430 +Ref: whatsnew/3 7 decimal329517 +Ref: 382329517 +Ref: decimal-Footnote-1329738 +Node: dis329781 +Ref: whatsnew/3 7 dis329874 +Ref: 383329874 +Ref: dis-Footnote-1330277 +Node: distutils330320 +Ref: whatsnew/3 7 distutils330410 +Ref: 385330410 +Ref: distutils-Footnote-1330658 +Node: enum330701 +Ref: whatsnew/3 7 enum330800 +Ref: 386330800 +Ref: enum-Footnote-1331468 +Ref: enum-Footnote-2331511 +Node: functools<2>331554 +Ref: whatsnew/3 7 functools331649 +Ref: 389331649 +Ref: functools<2>-Footnote-1331878 +Node: gc<2>331921 +Ref: whatsnew/3 7 gc332016 +Ref: 38b332016 +Ref: gc<2>-Footnote-1332555 +Node: hmac332598 +Ref: whatsnew/3 7 hmac332692 +Ref: 38f332692 +Ref: hmac-Footnote-1332950 +Node: http client332993 +Ref: whatsnew/3 7 http-client333093 +Ref: 391333093 +Ref: http client-Footnote-1333350 +Node: http server333393 +Ref: whatsnew/3 7 http-server333505 +Ref: 394333505 +Ref: http server-Footnote-1334383 +Ref: http server-Footnote-2334426 +Ref: http server-Footnote-3334469 +Node: idlelib and IDLE334512 +Ref: whatsnew/3 7 idlelib-and-idle334622 +Ref: 397334622 +Ref: idlelib and IDLE-Footnote-1337631 +Ref: idlelib and IDLE-Footnote-2337674 +Ref: idlelib and IDLE-Footnote-3337719 +Ref: idlelib and IDLE-Footnote-4337762 +Ref: idlelib and IDLE-Footnote-5337805 +Ref: idlelib and IDLE-Footnote-6337848 +Ref: idlelib and IDLE-Footnote-7337891 +Ref: idlelib and IDLE-Footnote-8337934 +Ref: idlelib and IDLE-Footnote-9337977 +Ref: idlelib and IDLE-Footnote-10338020 +Ref: idlelib and IDLE-Footnote-11338066 +Ref: idlelib and IDLE-Footnote-12338109 +Ref: idlelib and IDLE-Footnote-13338153 +Node: importlib338197 +Ref: whatsnew/3 7 importlib338301 +Ref: 398338301 +Ref: importlib-Footnote-1339172 +Ref: importlib-Footnote-2339215 +Ref: importlib-Footnote-3339258 +Node: io<2>339301 +Ref: whatsnew/3 7 io339398 +Ref: 39c339398 +Ref: io<2>-Footnote-1339663 +Ref: io<2>-Footnote-2339706 +Node: ipaddress339749 +Ref: whatsnew/3 7 ipaddress339849 +Ref: 39e339849 +Ref: ipaddress-Footnote-1340169 +Node: itertools<2>340212 +Ref: whatsnew/3 7 itertools340313 +Ref: 3a1340313 +Ref: itertools<2>-Footnote-1340550 +Node: locale340593 +Ref: whatsnew/3 7 locale340695 +Ref: 3a3340695 +Ref: locale-Footnote-1341109 +Node: logging<2>341152 +Ref: whatsnew/3 7 logging341249 +Ref: 3a6341249 +Ref: logging<2>-Footnote-1341773 +Ref: logging<2>-Footnote-2341816 +Ref: logging<2>-Footnote-3341859 +Node: math<2>341902 +Ref: whatsnew/3 7 math342002 +Ref: 3aa342002 +Ref: math<2>-Footnote-1342215 +Node: mimetypes342258 +Ref: whatsnew/3 7 mimetypes342354 +Ref: 3ac342354 +Ref: mimetypes-Footnote-1342569 +Node: msilib342612 +Ref: whatsnew/3 7 msilib342719 +Ref: 3ad342719 +Ref: msilib-Footnote-1342942 +Node: multiprocessing<2>342985 +Ref: whatsnew/3 7 multiprocessing343088 +Ref: 3af343088 +Ref: multiprocessing<2>-Footnote-1343735 +Ref: multiprocessing<2>-Footnote-2343778 +Ref: multiprocessing<2>-Footnote-3343821 +Node: os<2>343864 +Ref: whatsnew/3 7 os343971 +Ref: 3b3343971 +Ref: os<2>-Footnote-1345269 +Ref: os<2>-Footnote-2345312 +Ref: os<2>-Footnote-3345355 +Ref: os<2>-Footnote-4345398 +Ref: os<2>-Footnote-5345441 +Ref: os<2>-Footnote-6345484 +Ref: os<2>-Footnote-7345527 +Node: pathlib<2>345570 +Ref: whatsnew/3 7 pathlib345662 +Ref: 3c0345662 +Ref: pathlib<2>-Footnote-1345922 +Node: pdb345965 +Ref: whatsnew/3 7 pdb346065 +Ref: 3c1346065 +Ref: pdb-Footnote-1346475 +Ref: pdb-Footnote-2346518 +Node: py_compile<2>346561 +Ref: whatsnew/3 7 py-compile346656 +Ref: 3c3346656 +Ref: py_compile<2>-Footnote-1347110 +Ref: py_compile<2>-Footnote-2347151 +Node: pydoc347194 +Ref: whatsnew/3 7 pydoc347291 +Ref: 3c4347291 +Ref: pydoc-Footnote-1347515 +Node: queue347558 +Ref: whatsnew/3 7 queue347644 +Ref: 3c5347644 +Ref: queue-Footnote-1347849 +Node: re347892 +Ref: whatsnew/3 7 re347979 +Ref: 3c7347979 +Ref: re-Footnote-1349055 +Ref: re-Footnote-2349098 +Ref: re-Footnote-3349141 +Ref: re-Footnote-4349184 +Ref: re-Footnote-5349227 +Node: signal349270 +Ref: whatsnew/3 7 signal349361 +Ref: 3cd349361 +Ref: signal-Footnote-1349680 +Node: socket<2>349723 +Ref: whatsnew/3 7 socket349824 +Ref: 3cf349824 +Ref: socket<2>-Footnote-1350912 +Ref: socket<2>-Footnote-2350955 +Ref: socket<2>-Footnote-3350998 +Ref: socket<2>-Footnote-4351041 +Ref: socket<2>-Footnote-5351084 +Ref: socket<2>-Footnote-6351127 +Node: socketserver351170 +Ref: whatsnew/3 7 socketserver351272 +Ref: 3d4351272 +Node: sqlite3351748 +Ref: whatsnew/3 7 sqlite3351847 +Ref: 3d7351847 +Ref: sqlite3-Footnote-1352289 +Ref: sqlite3-Footnote-2352332 +Node: ssl<2>352375 +Ref: whatsnew/3 7 ssl352468 +Ref: 3db352468 +Ref: ssl<2>-Footnote-1355517 +Ref: ssl<2>-Footnote-2355560 +Ref: ssl<2>-Footnote-3355603 +Ref: ssl<2>-Footnote-4355646 +Ref: ssl<2>-Footnote-5355689 +Ref: ssl<2>-Footnote-6355732 +Ref: ssl<2>-Footnote-7355775 +Ref: ssl<2>-Footnote-8355818 +Ref: ssl<2>-Footnote-9355861 +Ref: ssl<2>-Footnote-10355904 +Ref: ssl<2>-Footnote-11355948 +Ref: ssl<2>-Footnote-12355992 +Ref: ssl<2>-Footnote-13356036 +Node: string356080 +Ref: whatsnew/3 7 string356176 +Ref: 3ea356176 +Ref: string-Footnote-1356453 +Node: subprocess356498 +Ref: whatsnew/3 7 subprocess356594 +Ref: 3ec356594 +Ref: subprocess-Footnote-1357916 +Ref: subprocess-Footnote-2357959 +Ref: subprocess-Footnote-3358002 +Ref: subprocess-Footnote-4358045 +Node: sys<2>358088 +Ref: whatsnew/3 7 sys358185 +Ref: 3f0358185 +Ref: sys<2>-Footnote-1358925 +Ref: sys<2>-Footnote-2358968 +Ref: sys<2>-Footnote-3359011 +Node: time<2>359054 +Ref: whatsnew/3 7 time359151 +Ref: 3f4359151 +Ref: time<2>-Footnote-1360361 +Ref: time<2>-Footnote-2360410 +Node: tkinter<2>360453 +Ref: whatsnew/3 7 tkinter360555 +Ref: 3fc360555 +Ref: tkinter<2>-Footnote-1360736 +Node: tracemalloc360779 +Ref: whatsnew/3 7 tracemalloc360879 +Ref: 3fe360879 +Ref: tracemalloc-Footnote-1361345 +Node: types361388 +Ref: whatsnew/3 7 types361492 +Ref: 401361492 +Ref: types-Footnote-1362007 +Ref: types-Footnote-2362050 +Ref: types-Footnote-3362093 +Ref: types-Footnote-4362142 +Node: unicodedata<2>362185 +Ref: whatsnew/3 7 unicodedata362289 +Ref: 407362289 +Ref: unicodedata<2>-Footnote-1362492 +Node: unittest<2>362547 +Ref: whatsnew/3 7 unittest362659 +Ref: 408362659 +Ref: unittest<2>-Footnote-1363064 +Node: unittest mock363107 +Ref: whatsnew/3 7 unittest-mock363217 +Ref: 409363217 +Ref: unittest mock-Footnote-1363735 +Ref: unittest mock-Footnote-2363778 +Node: urllib parse363821 +Ref: whatsnew/3 7 urllib-parse363922 +Ref: 40c363922 +Ref: urllib parse-Footnote-1364235 +Ref: urllib parse-Footnote-2364284 +Ref: urllib parse-Footnote-3364333 +Node: uu364376 +Ref: whatsnew/3 7 uu364468 +Ref: 40e364468 +Ref: uu-Footnote-1364740 +Node: uuid364783 +Ref: whatsnew/3 7 uuid364871 +Ref: 410364871 +Ref: uuid-Footnote-1365511 +Ref: uuid-Footnote-2365554 +Node: warnings365597 +Ref: whatsnew/3 7 warnings365692 +Ref: 414365692 +Ref: warnings-Footnote-1367146 +Ref: warnings-Footnote-2367189 +Ref: warnings-Footnote-3367232 +Ref: warnings-Footnote-4367275 +Node: xml etree367318 +Ref: whatsnew/3 7 xml-etree367422 +Ref: 419367422 +Ref: xml etree-Footnote-1367763 +Node: xmlrpc server367806 +Ref: whatsnew/3 7 xmlrpc-server367908 +Ref: 41b367908 +Ref: xmlrpc server-Footnote-1368117 +Node: zipapp368159 +Ref: whatsnew/3 7 zipapp368259 +Ref: 41c368259 +Ref: zipapp-Footnote-1368783 +Ref: zipapp-Footnote-2368826 +Node: zipfile368869 +Ref: whatsnew/3 7 zipfile368947 +Ref: 41e368947 +Ref: zipfile-Footnote-1369312 +Ref: zipfile-Footnote-2369355 +Node: C API Changes369398 +Ref: whatsnew/3 7 c-api-changes369524 +Ref: 420369524 +Ref: C API Changes-Footnote-1374036 +Ref: C API Changes-Footnote-2374079 +Ref: C API Changes-Footnote-3374122 +Ref: C API Changes-Footnote-4374165 +Ref: C API Changes-Footnote-5374208 +Ref: C API Changes-Footnote-6374251 +Ref: C API Changes-Footnote-7374294 +Ref: C API Changes-Footnote-8374337 +Ref: C API Changes-Footnote-9374380 +Ref: C API Changes-Footnote-10374423 +Ref: C API Changes-Footnote-11374467 +Ref: C API Changes-Footnote-12374511 +Ref: C API Changes-Footnote-13374555 +Ref: C API Changes-Footnote-14374599 +Ref: C API Changes-Footnote-15374643 +Ref: C API Changes-Footnote-16374686 +Ref: C API Changes-Footnote-17374730 +Ref: C API Changes-Footnote-18374774 +Ref: C API Changes-Footnote-19374818 +Ref: C API Changes-Footnote-20374862 +Node: Build Changes374906 +Ref: whatsnew/3 7 build-changes375029 +Ref: 441375029 +Ref: Build Changes-Footnote-1376163 +Ref: Build Changes-Footnote-2376206 +Ref: Build Changes-Footnote-3376249 +Node: Optimizations<2>376292 +Ref: whatsnew/3 7 optimizations376438 +Ref: 442376438 +Ref: whatsnew/3 7 whatsnew37-perf376438 +Ref: 306376438 +Ref: whatsnew/3 7 whatsnew37-asyncio-perf377132 +Ref: 347377132 +Ref: Optimizations<2>-Footnote-1381686 +Ref: Optimizations<2>-Footnote-2381729 +Ref: Optimizations<2>-Footnote-3381772 +Ref: Optimizations<2>-Footnote-4381815 +Ref: Optimizations<2>-Footnote-5381858 +Ref: Optimizations<2>-Footnote-6381901 +Ref: Optimizations<2>-Footnote-7381944 +Ref: Optimizations<2>-Footnote-8381987 +Ref: Optimizations<2>-Footnote-9382030 +Ref: Optimizations<2>-Footnote-10382073 +Ref: Optimizations<2>-Footnote-11382117 +Ref: Optimizations<2>-Footnote-12382161 +Ref: Optimizations<2>-Footnote-13382205 +Ref: Optimizations<2>-Footnote-14382249 +Ref: Optimizations<2>-Footnote-15382293 +Ref: Optimizations<2>-Footnote-16382337 +Ref: Optimizations<2>-Footnote-17382381 +Ref: Optimizations<2>-Footnote-18382425 +Ref: Optimizations<2>-Footnote-19382469 +Ref: Optimizations<2>-Footnote-20382513 +Ref: Optimizations<2>-Footnote-21382557 +Ref: Optimizations<2>-Footnote-22382601 +Ref: Optimizations<2>-Footnote-23382645 +Ref: Optimizations<2>-Footnote-24382689 +Ref: Optimizations<2>-Footnote-25382733 +Ref: Optimizations<2>-Footnote-26382777 +Ref: Optimizations<2>-Footnote-27382821 +Ref: Optimizations<2>-Footnote-28382865 +Ref: Optimizations<2>-Footnote-29382909 +Ref: Optimizations<2>-Footnote-30382953 +Node: Other CPython Implementation Changes382997 +Ref: whatsnew/3 7 other-cpython-implementation-changes383156 +Ref: 454383156 +Ref: Other CPython Implementation Changes-Footnote-1384557 +Ref: Other CPython Implementation Changes-Footnote-2384600 +Ref: Other CPython Implementation Changes-Footnote-3384643 +Ref: Other CPython Implementation Changes-Footnote-4384686 +Ref: Other CPython Implementation Changes-Footnote-5384729 +Node: Deprecated Python Behavior384772 +Ref: whatsnew/3 7 deprecated-python-behavior384962 +Ref: 456384962 +Ref: Deprecated Python Behavior-Footnote-1386071 +Ref: Deprecated Python Behavior-Footnote-2386114 +Node: Deprecated Python modules functions and methods386157 +Ref: whatsnew/3 7 deprecated-python-modules-functions-and-methods386354 +Ref: 459386354 +Node: aifc386752 +Ref: whatsnew/3 7 aifc386859 +Ref: 45a386859 +Ref: aifc-Footnote-1387085 +Node: asyncio<3>387128 +Ref: whatsnew/3 7 id2387258 +Ref: 45c387258 +Ref: whatsnew/3 7 whatsnew37-asyncio-deprecated387258 +Ref: 36b387258 +Ref: asyncio<3>-Footnote-1387794 +Ref: asyncio<3>-Footnote-2387837 +Node: collections<3>387880 +Ref: whatsnew/3 7 id3388012 +Ref: 45d388012 +Ref: collections<3>-Footnote-1388391 +Node: dbm<2>388434 +Ref: whatsnew/3 7 id4388563 +Ref: 45e388563 +Ref: dbm<2>-Footnote-1388964 +Node: enum<2>389007 +Ref: whatsnew/3 7 id5389132 +Ref: 45f389132 +Ref: enum<2>-Footnote-1389582 +Node: gettext<2>389625 +Ref: whatsnew/3 7 gettext389756 +Ref: 460389756 +Ref: gettext<2>-Footnote-1390001 +Node: importlib<2>390044 +Ref: whatsnew/3 7 id6390177 +Ref: 461390177 +Ref: importlib<2>-Footnote-1390703 +Node: locale<2>390746 +Ref: whatsnew/3 7 id7390876 +Ref: 467390876 +Ref: locale<2>-Footnote-1391085 +Node: macpath391128 +Ref: whatsnew/3 7 macpath391258 +Ref: 469391258 +Ref: macpath-Footnote-1391448 +Node: threading<2>391490 +Ref: whatsnew/3 7 threading391620 +Ref: 46a391620 +Ref: threading<2>-Footnote-1391928 +Node: socket<3>391971 +Ref: whatsnew/3 7 id8392100 +Ref: 46b392100 +Ref: socket<3>-Footnote-1392445 +Node: ssl<3>392488 +Ref: whatsnew/3 7 id9392610 +Ref: 46e392610 +Ref: ssl<3>-Footnote-1392828 +Node: sunau392871 +Ref: whatsnew/3 7 sunau392990 +Ref: 470392990 +Ref: sunau-Footnote-1393229 +Node: sys<3>393272 +Ref: whatsnew/3 7 id10393389 +Ref: 473393389 +Ref: sys<3>-Footnote-1393706 +Node: wave393749 +Ref: whatsnew/3 7 wave393852 +Ref: 474393852 +Ref: wave-Footnote-1394087 +Node: Deprecated functions and types of the C API394130 +Ref: whatsnew/3 7 deprecated-functions-and-types-of-the-c-api394326 +Ref: 477394326 +Ref: Deprecated functions and types of the C API-Footnote-1394984 +Ref: Deprecated functions and types of the C API-Footnote-2395027 +Node: Platform Support Removals395070 +Ref: whatsnew/3 7 id11395246 +Ref: 479395246 +Ref: whatsnew/3 7 platform-support-removals395246 +Ref: 3df395246 +Ref: Platform Support Removals-Footnote-1396658 +Ref: Platform Support Removals-Footnote-2396721 +Node: API and Feature Removals<2>396800 +Ref: whatsnew/3 7 api-and-feature-removals396948 +Ref: 47a396948 +Ref: API and Feature Removals<2>-Footnote-1399917 +Ref: API and Feature Removals<2>-Footnote-2399960 +Ref: API and Feature Removals<2>-Footnote-3400003 +Node: Module Removals400046 +Ref: whatsnew/3 7 module-removals400189 +Ref: 482400189 +Ref: Module Removals-Footnote-1400524 +Node: Windows-only Changes400567 +Ref: whatsnew/3 7 windows-only-changes400704 +Ref: 483400704 +Ref: Windows-only Changes-Footnote-1401581 +Ref: Windows-only Changes-Footnote-2401624 +Node: Porting to Python 3 7401667 +Ref: whatsnew/3 7 porting-to-python-3-7401820 +Ref: 484401820 +Ref: whatsnew/3 7 porting-to-python-37401820 +Ref: 307401820 +Node: Changes in Python Behavior402282 +Ref: whatsnew/3 7 changes-in-python-behavior402403 +Ref: 485402403 +Ref: Changes in Python Behavior-Footnote-1404269 +Ref: Changes in Python Behavior-Footnote-2404312 +Ref: Changes in Python Behavior-Footnote-3404361 +Ref: Changes in Python Behavior-Footnote-4404404 +Ref: Changes in Python Behavior-Footnote-5404447 +Ref: Changes in Python Behavior-Footnote-6404490 +Node: Changes in the Python API<2>404533 +Ref: whatsnew/3 7 changes-in-the-python-api404686 +Ref: 489404686 +Ref: Changes in the Python API<2>-Footnote-1414237 +Ref: Changes in the Python API<2>-Footnote-2414280 +Ref: Changes in the Python API<2>-Footnote-3414323 +Ref: Changes in the Python API<2>-Footnote-4414366 +Ref: Changes in the Python API<2>-Footnote-5414409 +Ref: Changes in the Python API<2>-Footnote-6414452 +Ref: Changes in the Python API<2>-Footnote-7414495 +Ref: Changes in the Python API<2>-Footnote-8414538 +Ref: Changes in the Python API<2>-Footnote-9414581 +Ref: Changes in the Python API<2>-Footnote-10414624 +Ref: Changes in the Python API<2>-Footnote-11414668 +Ref: Changes in the Python API<2>-Footnote-12414712 +Ref: Changes in the Python API<2>-Footnote-13414756 +Ref: Changes in the Python API<2>-Footnote-14414800 +Ref: Changes in the Python API<2>-Footnote-15414843 +Ref: Changes in the Python API<2>-Footnote-16414887 +Ref: Changes in the Python API<2>-Footnote-17414931 +Ref: Changes in the Python API<2>-Footnote-18414975 +Ref: Changes in the Python API<2>-Footnote-19415019 +Ref: Changes in the Python API<2>-Footnote-20415063 +Ref: Changes in the Python API<2>-Footnote-21415107 +Ref: Changes in the Python API<2>-Footnote-22415151 +Ref: Changes in the Python API<2>-Footnote-23415195 +Ref: Changes in the Python API<2>-Footnote-24415239 +Ref: Changes in the Python API<2>-Footnote-25415283 +Ref: Changes in the Python API<2>-Footnote-26415327 +Ref: Changes in the Python API<2>-Footnote-27415371 +Ref: Changes in the Python API<2>-Footnote-28415415 +Ref: Changes in the Python API<2>-Footnote-29415459 +Ref: Changes in the Python API<2>-Footnote-30415503 +Node: Changes in the C API<2>415547 +Ref: whatsnew/3 7 changes-in-the-c-api415701 +Ref: 4ab415701 +Ref: Changes in the C API<2>-Footnote-1416337 +Node: CPython bytecode changes<2>416380 +Ref: whatsnew/3 7 cpython-bytecode-changes416529 +Ref: 4ac416529 +Ref: CPython bytecode changes<2>-Footnote-1416881 +Ref: CPython bytecode changes<2>-Footnote-2416924 +Node: Windows-only Changes<2>416967 +Ref: whatsnew/3 7 id12417129 +Ref: 4af417129 +Ref: Windows-only Changes<2>-Footnote-1417450 +Node: Other CPython implementation changes417493 +Ref: whatsnew/3 7 id13417619 +Ref: 4b1417619 +Ref: Other CPython implementation changes-Footnote-1419967 +Ref: Other CPython implementation changes-Footnote-2420016 +Ref: Other CPython implementation changes-Footnote-3420059 +Node: Notable changes in Python 3 7 1420102 +Ref: whatsnew/3 7 notable-changes-in-python-3-7-1420266 +Ref: 4b6420266 +Ref: Notable changes in Python 3 7 1-Footnote-1421206 +Ref: Notable changes in Python 3 7 1-Footnote-2421249 +Ref: Notable changes in Python 3 7 1-Footnote-3421292 +Node: Notable changes in Python 3 7 2421335 +Ref: whatsnew/3 7 notable-changes-in-python-3-7-2421509 +Ref: 4bb421509 +Node: Notable changes in Python 3 7 6422004 +Ref: whatsnew/3 7 notable-changes-in-python-3-7-6422179 +Ref: 4bc422179 +Ref: Notable changes in Python 3 7 6-Footnote-1422690 +Node: Notable changes in Python 3 7 10422733 +Ref: whatsnew/3 7 notable-changes-in-python-3-7-10422868 +Ref: 4bd422868 +Ref: Notable changes in Python 3 7 10-Footnote-1423600 +Node: What’s New In Python 3 6423643 +Ref: whatsnew/3 6 doc423798 +Ref: 4be423798 +Ref: whatsnew/3 6 what-s-new-in-python-3-6423798 +Ref: 4bf423798 +Ref: What’s New In Python 3 6-Footnote-1425061 +Ref: What’s New In Python 3 6-Footnote-2425121 +Node: Summary – Release highlights<2>425170 +Ref: whatsnew/3 6 summary-release-highlights425290 +Ref: 4c0425290 +Ref: Summary – Release highlights<2>-Footnote-1429531 +Ref: Summary – Release highlights<2>-Footnote-2429611 +Ref: Summary – Release highlights<2>-Footnote-3429701 +Node: New Features<3>429750 +Ref: whatsnew/3 6 new-features429904 +Ref: 4d7429904 +Ref: whatsnew/3 6 pypy-dict-implementation429904 +Ref: 4d8429904 +Node: PEP 498 Formatted string literals431314 +Ref: whatsnew/3 6 pep-498-formatted-string-literals431447 +Ref: 4d9431447 +Ref: whatsnew/3 6 whatsnew36-pep498431447 +Ref: 4c1431447 +Ref: PEP 498 Formatted string literals-Footnote-1432431 +Ref: PEP 498 Formatted string literals-Footnote-2432480 +Node: PEP 526 Syntax for variable annotations432529 +Ref: whatsnew/3 6 pep-526-syntax-for-variable-annotations432710 +Ref: 4dc432710 +Ref: whatsnew/3 6 whatsnew36-pep526432710 +Ref: 4c3432710 +Ref: PEP 526 Syntax for variable annotations-Footnote-1433954 +Ref: PEP 526 Syntax for variable annotations-Footnote-2434003 +Ref: PEP 526 Syntax for variable annotations-Footnote-3434052 +Ref: PEP 526 Syntax for variable annotations-Footnote-4434086 +Node: PEP 515 Underscores in Numeric Literals434127 +Ref: whatsnew/3 6 pep-515-underscores-in-numeric-literals434306 +Ref: 4dd434306 +Ref: whatsnew/3 6 whatsnew36-pep515434306 +Ref: 4c2434306 +Ref: PEP 515 Underscores in Numeric Literals-Footnote-1435375 +Ref: PEP 515 Underscores in Numeric Literals-Footnote-2435424 +Node: PEP 525 Asynchronous Generators435473 +Ref: whatsnew/3 6 pep-525-asynchronous-generators435648 +Ref: 4df435648 +Ref: whatsnew/3 6 whatsnew36-pep525435648 +Ref: 4c4435648 +Ref: PEP 525 Asynchronous Generators-Footnote-1436483 +Ref: PEP 525 Asynchronous Generators-Footnote-2436532 +Node: PEP 530 Asynchronous Comprehensions436581 +Ref: whatsnew/3 6 pep-530-asynchronous-comprehensions436764 +Ref: 4e0436764 +Ref: whatsnew/3 6 whatsnew36-pep530436764 +Ref: 4c5436764 +Ref: PEP 530 Asynchronous Comprehensions-Footnote-1437322 +Ref: PEP 530 Asynchronous Comprehensions-Footnote-2437371 +Node: PEP 487 Simpler customization of class creation437420 +Ref: whatsnew/3 6 pep-487-simpler-customization-of-class-creation437612 +Ref: 4e1437612 +Ref: whatsnew/3 6 whatsnew36-pep487437612 +Ref: 4c8437612 +Ref: PEP 487 Simpler customization of class creation-Footnote-1438704 +Node: PEP 487 Descriptor Protocol Enhancements438753 +Ref: whatsnew/3 6 pep-487-descriptor-protocol-enhancements438952 +Ref: 4e6438952 +Ref: whatsnew/3 6 whatsnew36-pep487-descriptors438952 +Ref: 4e7438952 +Ref: PEP 487 Descriptor Protocol Enhancements-Footnote-1440203 +Ref: PEP 487 Descriptor Protocol Enhancements-Footnote-2440252 +Node: PEP 519 Adding a file system path protocol440301 +Ref: whatsnew/3 6 pep-519-adding-a-file-system-path-protocol440486 +Ref: 4ea440486 +Ref: whatsnew/3 6 whatsnew36-pep519440486 +Ref: 4cd440486 +Ref: PEP 519 Adding a file system path protocol-Footnote-1443313 +Node: PEP 495 Local Time Disambiguation443362 +Ref: whatsnew/3 6 pep-495-local-time-disambiguation443558 +Ref: 4f2443558 +Ref: whatsnew/3 6 whatsnew36-pep495443558 +Ref: 4ce443558 +Ref: PEP 495 Local Time Disambiguation-Footnote-1444971 +Ref: PEP 495 Local Time Disambiguation-Footnote-2445020 +Node: PEP 529 Change Windows filesystem encoding to UTF-8445069 +Ref: whatsnew/3 6 pep-529-change-windows-filesystem-encoding-to-utf-8445271 +Ref: 4f5445271 +Ref: whatsnew/3 6 whatsnew36-pep529445271 +Ref: 4d4445271 +Ref: PEP 529 Change Windows filesystem encoding to UTF-8-Footnote-1446301 +Node: PEP 528 Change Windows console encoding to UTF-8446350 +Ref: whatsnew/3 6 pep-528-change-windows-console-encoding-to-utf-8446570 +Ref: 4f9446570 +Ref: whatsnew/3 6 whatsnew36-pep528446570 +Ref: 4d3446570 +Ref: PEP 528 Change Windows console encoding to UTF-8-Footnote-1447273 +Node: PEP 520 Preserving Class Attribute Definition Order447322 +Ref: whatsnew/3 6 pep-520-preserving-class-attribute-definition-order447532 +Ref: 4fb447532 +Ref: whatsnew/3 6 whatsnew36-pep520447532 +Ref: 4c9447532 +Ref: PEP 520 Preserving Class Attribute Definition Order-Footnote-1448172 +Node: PEP 468 Preserving Keyword Argument Order448221 +Ref: whatsnew/3 6 pep-468-preserving-keyword-argument-order448406 +Ref: 4fe448406 +Ref: whatsnew/3 6 whatsnew36-pep468448406 +Ref: 4ca448406 +Ref: PEP 468 Preserving Keyword Argument Order-Footnote-1448767 +Node: New dict implementation448816 +Ref: whatsnew/3 6 new-dict-implementation448998 +Ref: 4ff448998 +Ref: whatsnew/3 6 whatsnew36-compactdict448998 +Ref: 4c7448998 +Ref: New dict implementation-Footnote-1449976 +Ref: New dict implementation-Footnote-2450056 +Ref: New dict implementation-Footnote-3450146 +Ref: New dict implementation-Footnote-4450189 +Node: PEP 523 Adding a frame evaluation API to CPython450269 +Ref: whatsnew/3 6 pep-523-adding-a-frame-evaluation-api-to-cpython450443 +Ref: 500450443 +Ref: whatsnew/3 6 whatsnew36-pep523450443 +Ref: 501450443 +Ref: PEP 523 Adding a frame evaluation API to CPython-Footnote-1451614 +Ref: PEP 523 Adding a frame evaluation API to CPython-Footnote-2451663 +Node: PYTHONMALLOC environment variable451712 +Ref: whatsnew/3 6 pythonmalloc-environment-variable451899 +Ref: 502451899 +Ref: whatsnew/3 6 whatsnew36-pythonmalloc451899 +Ref: 4cc451899 +Ref: PYTHONMALLOC environment variable-Footnote-1455082 +Ref: PYTHONMALLOC environment variable-Footnote-2455125 +Node: DTrace and SystemTap probing support455168 +Ref: whatsnew/3 6 dtrace-and-systemtap-probing-support455298 +Ref: 50b455298 +Ref: whatsnew/3 6 whatsnew36-tracing455298 +Ref: 4cb455298 +Ref: DTrace and SystemTap probing support-Footnote-1456129 +Node: Other Language Changes<3>456172 +Ref: whatsnew/3 6 other-language-changes456307 +Ref: 50d456307 +Ref: Other Language Changes<3>-Footnote-1457692 +Ref: Other Language Changes<3>-Footnote-2457735 +Ref: Other Language Changes<3>-Footnote-3457778 +Ref: Other Language Changes<3>-Footnote-4457821 +Node: New Modules<3>457864 +Ref: whatsnew/3 6 new-modules458003 +Ref: 511458003 +Node: secrets458062 +Ref: whatsnew/3 6 secrets458120 +Ref: 512458120 +Ref: whatsnew/3 6 whatsnew36-pep506458120 +Ref: 4c6458120 +Ref: secrets-Footnote-1458803 +Node: Improved Modules<3>458852 +Ref: whatsnew/3 6 improved-modules458982 +Ref: 513458982 +Node: array460363 +Ref: whatsnew/3 6 array460439 +Ref: 514460439 +Ref: array-Footnote-1460732 +Node: ast<2>460775 +Ref: whatsnew/3 6 ast460870 +Ref: 515460870 +Ref: ast<2>-Footnote-1461113 +Node: asyncio<4>461156 +Ref: whatsnew/3 6 asyncio461257 +Ref: 516461257 +Ref: asyncio<4>-Footnote-1464597 +Ref: asyncio<4>-Footnote-2464640 +Ref: asyncio<4>-Footnote-3464685 +Ref: asyncio<4>-Footnote-4464728 +Ref: asyncio<4>-Footnote-5464771 +Ref: asyncio<4>-Footnote-6464814 +Ref: asyncio<4>-Footnote-7464857 +Ref: asyncio<4>-Footnote-8464900 +Ref: asyncio<4>-Footnote-9464943 +Ref: asyncio<4>-Footnote-10464986 +Ref: asyncio<4>-Footnote-11465030 +Ref: asyncio<4>-Footnote-12465074 +Node: binascii<2>465118 +Ref: whatsnew/3 6 binascii465218 +Ref: 522465218 +Ref: binascii<2>-Footnote-1465503 +Node: cmath465546 +Ref: whatsnew/3 6 cmath465650 +Ref: 524465650 +Ref: cmath-Footnote-1466121 +Ref: cmath-Footnote-2466164 +Ref: cmath-Footnote-3466213 +Node: collections<4>466256 +Ref: whatsnew/3 6 collections466370 +Ref: 52c466370 +Ref: collections<4>-Footnote-1467471 +Ref: collections<4>-Footnote-2467514 +Ref: collections<4>-Footnote-3467557 +Ref: collections<4>-Footnote-4467600 +Ref: collections<4>-Footnote-5467643 +Ref: collections<4>-Footnote-6467686 +Node: concurrent futures<2>467729 +Ref: whatsnew/3 6 concurrent-futures467851 +Ref: 532467851 +Ref: concurrent futures<2>-Footnote-1468181 +Node: contextlib<2>468224 +Ref: whatsnew/3 6 contextlib468343 +Ref: 533468343 +Ref: contextlib<2>-Footnote-1468839 +Node: datetime<3>468882 +Ref: whatsnew/3 6 datetime468990 +Ref: 536468990 +Ref: datetime<3>-Footnote-1470007 +Ref: datetime<3>-Footnote-2470050 +Ref: datetime<3>-Footnote-3470093 +Ref: datetime<3>-Footnote-4470136 +Node: decimal<2>470179 +Ref: whatsnew/3 6 decimal470286 +Ref: 53a470286 +Ref: decimal<2>-Footnote-1470699 +Node: distutils<2>470742 +Ref: whatsnew/3 6 distutils470843 +Ref: 53c470843 +Ref: distutils<2>-Footnote-1471217 +Node: email471260 +Ref: whatsnew/3 6 email471360 +Ref: 53d471360 +Ref: email-Footnote-1472276 +Ref: email-Footnote-2472319 +Ref: email-Footnote-3472362 +Node: encodings472405 +Ref: whatsnew/3 6 encodings472500 +Ref: 543472500 +Ref: encodings-Footnote-1472795 +Node: enum<3>472838 +Ref: whatsnew/3 6 enum472940 +Ref: 544472940 +Ref: enum<3>-Footnote-1473688 +Node: faulthandler473731 +Ref: whatsnew/3 6 faulthandler473833 +Ref: 547473833 +Ref: faulthandler-Footnote-1474096 +Node: fileinput474139 +Ref: whatsnew/3 6 fileinput474241 +Ref: 549474241 +Ref: fileinput-Footnote-1474429 +Node: hashlib474472 +Ref: whatsnew/3 6 hashlib474576 +Ref: 54b474576 +Ref: hashlib-Footnote-1475594 +Ref: hashlib-Footnote-2475637 +Ref: hashlib-Footnote-3475680 +Ref: hashlib-Footnote-4475723 +Node: http client<2>475766 +Ref: whatsnew/3 6 http-client475880 +Ref: 54e475880 +Ref: http client<2>-Footnote-1476140 +Node: idlelib and IDLE<2>476183 +Ref: whatsnew/3 6 idlelib-and-idle476302 +Ref: 551476302 +Ref: idlelib and IDLE<2>-Footnote-1479776 +Ref: idlelib and IDLE<2>-Footnote-2479819 +Ref: idlelib and IDLE<2>-Footnote-3479862 +Ref: idlelib and IDLE<2>-Footnote-4479907 +Ref: idlelib and IDLE<2>-Footnote-5479950 +Ref: idlelib and IDLE<2>-Footnote-6479993 +Ref: idlelib and IDLE<2>-Footnote-7480036 +Ref: idlelib and IDLE<2>-Footnote-8480079 +Ref: idlelib and IDLE<2>-Footnote-9480122 +Ref: idlelib and IDLE<2>-Footnote-10480165 +Ref: idlelib and IDLE<2>-Footnote-11480209 +Node: importlib<3>480255 +Ref: whatsnew/3 6 importlib480370 +Ref: 552480370 +Ref: importlib<3>-Footnote-1481197 +Node: inspect<2>481240 +Ref: whatsnew/3 6 inspect481340 +Ref: 55a481340 +Ref: inspect<2>-Footnote-1482121 +Ref: inspect<2>-Footnote-2482164 +Node: json482207 +Ref: whatsnew/3 6 json482305 +Ref: 55e482305 +Ref: json-Footnote-1482577 +Node: logging<3>482620 +Ref: whatsnew/3 6 logging482715 +Ref: 561482715 +Ref: logging<3>-Footnote-1482979 +Node: math<3>483022 +Ref: whatsnew/3 6 math483131 +Ref: 563483131 +Ref: math<3>-Footnote-1483363 +Ref: math<3>-Footnote-2483406 +Node: multiprocessing<3>483455 +Ref: whatsnew/3 6 multiprocessing483559 +Ref: 564483559 +Ref: multiprocessing<3>-Footnote-1483782 +Node: os<3>483824 +Ref: whatsnew/3 6 os483931 +Ref: 566483931 +Ref: os<3>-Footnote-1484931 +Ref: os<3>-Footnote-2484974 +Ref: os<3>-Footnote-3485023 +Node: pathlib<3>485072 +Ref: whatsnew/3 6 pathlib485167 +Ref: 56a485167 +Ref: pathlib<3>-Footnote-1485401 +Node: pdb<2>485444 +Ref: whatsnew/3 6 pdb485543 +Ref: 56b485543 +Node: pickle<2>485700 +Ref: whatsnew/3 6 pickle485800 +Ref: 56d485800 +Ref: pickle<2>-Footnote-1486117 +Node: pickletools486160 +Ref: whatsnew/3 6 pickletools486262 +Ref: 56f486262 +Ref: pickletools-Footnote-1486490 +Node: pydoc<2>486533 +Ref: whatsnew/3 6 pydoc486632 +Ref: 571486632 +Ref: pydoc<2>-Footnote-1487026 +Ref: pydoc<2>-Footnote-2487068 +Node: random487111 +Ref: whatsnew/3 6 random487204 +Ref: 573487204 +Ref: random-Footnote-1487459 +Node: re<2>487502 +Ref: whatsnew/3 6 re487595 +Ref: 575487595 +Ref: re<2>-Footnote-1488300 +Ref: re<2>-Footnote-2488344 +Ref: re<2>-Footnote-3488387 +Node: readline488430 +Ref: whatsnew/3 6 readline488528 +Ref: 576488528 +Ref: readline-Footnote-1488759 +Node: rlcompleter488802 +Ref: whatsnew/3 6 rlcompleter488903 +Ref: 578488903 +Ref: rlcompleter-Footnote-1489207 +Ref: rlcompleter-Footnote-2489250 +Node: shlex<2>489293 +Ref: whatsnew/3 6 shlex489390 +Ref: 579489390 +Ref: shlex<2>-Footnote-1489680 +Node: site489725 +Ref: whatsnew/3 6 site489821 +Ref: 57c489821 +Ref: site-Footnote-1490085 +Node: sqlite3<2>490128 +Ref: whatsnew/3 6 sqlite3490225 +Ref: 57d490225 +Ref: sqlite3<2>-Footnote-1490428 +Node: socket<4>490471 +Ref: whatsnew/3 6 socket490579 +Ref: 57f490579 +Ref: socket<4>-Footnote-1491541 +Ref: socket<4>-Footnote-2491584 +Ref: socket<4>-Footnote-3491627 +Ref: socket<4>-Footnote-4491670 +Node: socketserver<2>491713 +Ref: whatsnew/3 6 socketserver491817 +Ref: 586491817 +Ref: socketserver<2>-Footnote-1492446 +Ref: socketserver<2>-Footnote-2492489 +Node: ssl<4>492532 +Ref: whatsnew/3 6 ssl492640 +Ref: 58a492640 +Ref: ssl<4>-Footnote-1493836 +Ref: ssl<4>-Footnote-2493879 +Ref: ssl<4>-Footnote-3493922 +Ref: ssl<4>-Footnote-4493965 +Ref: ssl<4>-Footnote-5494008 +Ref: ssl<4>-Footnote-6494051 +Ref: ssl<4>-Footnote-7494094 +Node: statistics<2>494137 +Ref: whatsnew/3 6 statistics494236 +Ref: 58e494236 +Ref: statistics<2>-Footnote-1494426 +Node: struct494469 +Ref: whatsnew/3 6 struct494575 +Ref: 590494575 +Ref: struct-Footnote-1494806 +Node: subprocess<2>494849 +Ref: whatsnew/3 6 subprocess494948 +Ref: 591494948 +Ref: subprocess<2>-Footnote-1495641 +Ref: subprocess<2>-Footnote-2495684 +Node: sys<4>495726 +Ref: whatsnew/3 6 sys495828 +Ref: 593495828 +Ref: sys<4>-Footnote-1496429 +Ref: sys<4>-Footnote-2496472 +Node: telnetlib496515 +Ref: whatsnew/3 6 telnetlib496611 +Ref: 596496611 +Ref: telnetlib-Footnote-1496782 +Node: time<3>496825 +Ref: whatsnew/3 6 time496921 +Ref: 598496921 +Node: timeit497060 +Ref: whatsnew/3 6 timeit497157 +Ref: 59a497157 +Ref: timeit-Footnote-1497617 +Ref: timeit-Footnote-2497659 +Node: tkinter<3>497702 +Ref: whatsnew/3 6 tkinter497801 +Ref: 59d497801 +Ref: tkinter<3>-Footnote-1498227 +Node: traceback498270 +Ref: whatsnew/3 6 traceback498377 +Ref: 59e498377 +Ref: whatsnew/3 6 whatsnew36-traceback498377 +Ref: 510498377 +Ref: traceback-Footnote-1499012 +Node: tracemalloc<2>499055 +Ref: whatsnew/3 6 tracemalloc499161 +Ref: 59f499161 +Ref: tracemalloc<2>-Footnote-1499522 +Node: typing<2>499565 +Ref: whatsnew/3 6 typing499676 +Ref: 5a1499676 +Ref: whatsnew/3 6 whatsnew36-typing499676 +Ref: 4cf499676 +Ref: typing<2>-Footnote-1501328 +Ref: typing<2>-Footnote-2501378 +Ref: typing<2>-Footnote-3501421 +Ref: typing<2>-Footnote-4501464 +Ref: typing<2>-Footnote-5501513 +Ref: typing<2>-Footnote-6501563 +Ref: typing<2>-Footnote-7501615 +Node: unicodedata<3>501667 +Ref: whatsnew/3 6 unicodedata501780 +Ref: 5a6501780 +Ref: unicodedata<3>-Footnote-1501968 +Node: unittest mock<2>502018 +Ref: whatsnew/3 6 unittest-mock502136 +Ref: 5a7502136 +Ref: unittest mock<2>-Footnote-1502653 +Ref: unittest mock<2>-Footnote-2502696 +Node: urllib request502739 +Ref: whatsnew/3 6 urllib-request502861 +Ref: 5ab502861 +Ref: urllib request-Footnote-1503226 +Node: urllib robotparser503269 +Ref: whatsnew/3 6 urllib-robotparser503382 +Ref: 5ac503382 +Ref: urllib robotparser-Footnote-1503627 +Node: venv<2>503670 +Ref: whatsnew/3 6 venv503780 +Ref: 5ae503780 +Ref: venv<2>-Footnote-1504067 +Node: warnings<2>504110 +Ref: whatsnew/3 6 warnings504208 +Ref: 5af504208 +Ref: warnings<2>-Footnote-1505483 +Ref: warnings<2>-Footnote-2505526 +Node: winreg505569 +Ref: whatsnew/3 6 winreg505668 +Ref: 5b1505668 +Ref: winreg-Footnote-1505841 +Node: winsound505884 +Ref: whatsnew/3 6 winsound505985 +Ref: 5b3505985 +Ref: winsound-Footnote-1506184 +Node: xmlrpc client506227 +Ref: whatsnew/3 6 xmlrpc-client506332 +Ref: 5b7506332 +Ref: xmlrpc client-Footnote-1506625 +Node: zipfile<2>506668 +Ref: whatsnew/3 6 zipfile506769 +Ref: 5b8506769 +Ref: zipfile<2>-Footnote-1507303 +Ref: zipfile<2>-Footnote-2507346 +Node: zlib507389 +Ref: whatsnew/3 6 zlib507468 +Ref: 5bd507468 +Ref: zlib-Footnote-1507724 +Ref: zlib-Footnote-2507767 +Node: Optimizations<3>507810 +Ref: whatsnew/3 6 optimizations507952 +Ref: 5c0507952 +Ref: Optimizations<3>-Footnote-1511821 +Ref: Optimizations<3>-Footnote-2511864 +Ref: Optimizations<3>-Footnote-3511907 +Ref: Optimizations<3>-Footnote-4511950 +Ref: Optimizations<3>-Footnote-5511993 +Ref: Optimizations<3>-Footnote-6512036 +Ref: Optimizations<3>-Footnote-7512079 +Ref: Optimizations<3>-Footnote-8512122 +Ref: Optimizations<3>-Footnote-9512165 +Ref: Optimizations<3>-Footnote-10512208 +Ref: Optimizations<3>-Footnote-11512252 +Ref: Optimizations<3>-Footnote-12512296 +Ref: Optimizations<3>-Footnote-13512340 +Ref: Optimizations<3>-Footnote-14512384 +Ref: Optimizations<3>-Footnote-15512428 +Ref: Optimizations<3>-Footnote-16512472 +Ref: Optimizations<3>-Footnote-17512516 +Ref: Optimizations<3>-Footnote-18512560 +Ref: Optimizations<3>-Footnote-19512604 +Ref: Optimizations<3>-Footnote-20512648 +Ref: Optimizations<3>-Footnote-21512692 +Node: Build and C API Changes<2>512736 +Ref: whatsnew/3 6 build-and-c-api-changes512877 +Ref: 5c8512877 +Ref: Build and C API Changes<2>-Footnote-1515330 +Ref: Build and C API Changes<2>-Footnote-2515379 +Ref: Build and C API Changes<2>-Footnote-3515422 +Ref: Build and C API Changes<2>-Footnote-4515465 +Ref: Build and C API Changes<2>-Footnote-5515508 +Ref: Build and C API Changes<2>-Footnote-6515550 +Ref: Build and C API Changes<2>-Footnote-7515593 +Ref: Build and C API Changes<2>-Footnote-8515636 +Ref: Build and C API Changes<2>-Footnote-9515679 +Ref: Build and C API Changes<2>-Footnote-10515722 +Node: Other Improvements515766 +Ref: whatsnew/3 6 other-improvements515904 +Ref: 5d0515904 +Node: Deprecated<2>516275 +Ref: whatsnew/3 6 deprecated516394 +Ref: 5d4516394 +Node: New Keywords516715 +Ref: whatsnew/3 6 new-keywords516812 +Ref: 5d5516812 +Ref: New Keywords-Footnote-1517205 +Node: Deprecated Python behavior517254 +Ref: whatsnew/3 6 deprecated-python-behavior517410 +Ref: 5d6517410 +Ref: Deprecated Python behavior-Footnote-1518617 +Ref: Deprecated Python behavior-Footnote-2518660 +Ref: Deprecated Python behavior-Footnote-3518703 +Node: Deprecated Python modules functions and methods<2>518746 +Ref: whatsnew/3 6 deprecated-python-modules-functions-and-methods518936 +Ref: 5d9518936 +Node: asynchat519245 +Ref: whatsnew/3 6 asynchat519357 +Ref: 5da519357 +Ref: asynchat-Footnote-1519545 +Node: asyncore519588 +Ref: whatsnew/3 6 asyncore519715 +Ref: 5db519715 +Ref: asyncore-Footnote-1519903 +Node: dbm<3>519946 +Ref: whatsnew/3 6 dbm520077 +Ref: 5dc520077 +Ref: dbm<3>-Footnote-1520437 +Node: distutils<3>520480 +Ref: whatsnew/3 6 id2520606 +Ref: 5dd520606 +Ref: distutils<3>-Footnote-1520926 +Node: grp520969 +Ref: whatsnew/3 6 grp521101 +Ref: 5de521101 +Ref: grp-Footnote-1521299 +Node: importlib<4>521342 +Ref: whatsnew/3 6 id3521467 +Ref: 5e0521467 +Node: os<4>522121 +Ref: whatsnew/3 6 id4522248 +Ref: 5e7522248 +Ref: os<4>-Footnote-1522542 +Ref: os<4>-Footnote-2522585 +Node: re<3>522628 +Ref: whatsnew/3 6 id5522749 +Ref: 5e9522749 +Ref: re<3>-Footnote-1523075 +Node: ssl<5>523118 +Ref: whatsnew/3 6 id6523244 +Ref: 5ea523244 +Ref: ssl<5>-Footnote-1524039 +Ref: ssl<5>-Footnote-2524082 +Ref: ssl<5>-Footnote-3524125 +Node: tkinter<4>524168 +Ref: whatsnew/3 6 id7524296 +Ref: 5eb524296 +Node: venv<3>524457 +Ref: whatsnew/3 6 id8524570 +Ref: 5ec524570 +Ref: venv<3>-Footnote-1524921 +Node: Deprecated functions and types of the C API<2>524964 +Ref: whatsnew/3 6 deprecated-functions-and-types-of-the-c-api525152 +Ref: 5ed525152 +Node: Deprecated Build Options525501 +Ref: whatsnew/3 6 deprecated-build-options525630 +Ref: 5ef525630 +Node: Removed526071 +Ref: whatsnew/3 6 removed526193 +Ref: 5f0526193 +Node: API and Feature Removals<3>526291 +Ref: whatsnew/3 6 api-and-feature-removals526362 +Ref: 5f1526362 +Ref: API and Feature Removals<3>-Footnote-1528454 +Ref: API and Feature Removals<3>-Footnote-2528497 +Node: Porting to Python 3 6528570 +Ref: whatsnew/3 6 porting-to-python-3-6528710 +Ref: 5f5528710 +Node: Changes in ‘python’ Command Behavior529096 +Ref: whatsnew/3 6 changes-in-python-command-behavior529231 +Ref: 5f6529231 +Ref: Changes in ‘python’ Command Behavior-Footnote-1529711 +Node: Changes in the Python API<3>529754 +Ref: whatsnew/3 6 changes-in-the-python-api529921 +Ref: 5f7529921 +Ref: Changes in the Python API<3>-Footnote-1540044 +Ref: Changes in the Python API<3>-Footnote-2540086 +Ref: Changes in the Python API<3>-Footnote-3540135 +Ref: Changes in the Python API<3>-Footnote-4540178 +Ref: Changes in the Python API<3>-Footnote-5540221 +Ref: Changes in the Python API<3>-Footnote-6540264 +Ref: Changes in the Python API<3>-Footnote-7540307 +Ref: Changes in the Python API<3>-Footnote-8540350 +Ref: Changes in the Python API<3>-Footnote-9540393 +Ref: Changes in the Python API<3>-Footnote-10540436 +Ref: Changes in the Python API<3>-Footnote-11540480 +Ref: Changes in the Python API<3>-Footnote-12540530 +Ref: Changes in the Python API<3>-Footnote-13540574 +Ref: Changes in the Python API<3>-Footnote-14540618 +Ref: Changes in the Python API<3>-Footnote-15540662 +Ref: Changes in the Python API<3>-Footnote-16540706 +Ref: Changes in the Python API<3>-Footnote-17540749 +Ref: Changes in the Python API<3>-Footnote-18540793 +Ref: Changes in the Python API<3>-Footnote-19540837 +Ref: Changes in the Python API<3>-Footnote-20540881 +Node: Changes in the C API<3>540925 +Ref: whatsnew/3 6 changes-in-the-c-api541079 +Ref: 611541079 +Ref: Changes in the C API<3>-Footnote-1541733 +Ref: Changes in the C API<3>-Footnote-2541776 +Node: CPython bytecode changes<3>541818 +Ref: whatsnew/3 6 cpython-bytecode-changes541935 +Ref: 613541935 +Ref: CPython bytecode changes<3>-Footnote-1543591 +Ref: CPython bytecode changes<3>-Footnote-2543634 +Ref: CPython bytecode changes<3>-Footnote-3543677 +Ref: CPython bytecode changes<3>-Footnote-4543720 +Ref: CPython bytecode changes<3>-Footnote-5543763 +Ref: CPython bytecode changes<3>-Footnote-6543806 +Ref: CPython bytecode changes<3>-Footnote-7543849 +Ref: CPython bytecode changes<3>-Footnote-8543892 +Ref: CPython bytecode changes<3>-Footnote-9543935 +Node: Notable changes in Python 3 6 2543978 +Ref: whatsnew/3 6 notable-changes-in-python-3-6-2544142 +Ref: 620544142 +Node: New make regen-all build target544306 +Ref: whatsnew/3 6 new-make-regen-all-build-target544448 +Ref: 621544448 +Ref: New make regen-all build target-Footnote-1545235 +Ref: New make regen-all build target-Footnote-2545302 +Node: Removal of make touch build target545345 +Ref: whatsnew/3 6 removal-of-make-touch-build-target545487 +Ref: 622545487 +Ref: Removal of make touch build target-Footnote-1545917 +Node: Notable changes in Python 3 6 4545960 +Ref: whatsnew/3 6 notable-changes-in-python-3-6-4546134 +Ref: 623546134 +Ref: Notable changes in Python 3 6 4-Footnote-1546507 +Ref: Notable changes in Python 3 6 4-Footnote-2546550 +Node: Notable changes in Python 3 6 5546593 +Ref: whatsnew/3 6 notable-changes-in-python-3-6-5546767 +Ref: 624546767 +Ref: Notable changes in Python 3 6 5-Footnote-1547070 +Node: Notable changes in Python 3 6 7547113 +Ref: whatsnew/3 6 notable-changes-in-python-3-6-7547288 +Ref: 625547288 +Ref: Notable changes in Python 3 6 7-Footnote-1547665 +Node: Notable changes in Python 3 6 10547708 +Ref: whatsnew/3 6 notable-changes-in-python-3-6-10547884 +Ref: 626547884 +Ref: Notable changes in Python 3 6 10-Footnote-1548397 +Node: Notable changes in Python 3 6 13548440 +Ref: whatsnew/3 6 notable-changes-in-python-3-6-13548576 +Ref: 627548576 +Ref: Notable changes in Python 3 6 13-Footnote-1549308 +Node: What’s New In Python 3 5549351 +Ref: whatsnew/3 5 doc549506 +Ref: 628549506 +Ref: whatsnew/3 5 what-s-new-in-python-3-5549506 +Ref: 629549506 +Ref: What’s New In Python 3 5-Footnote-1550445 +Ref: What’s New In Python 3 5-Footnote-2550505 +Node: Summary – Release highlights<3>550554 +Ref: whatsnew/3 5 summary-release-highlights550674 +Ref: 62a550674 +Ref: Summary – Release highlights<3>-Footnote-1554403 +Ref: Summary – Release highlights<3>-Footnote-2554445 +Ref: Summary – Release highlights<3>-Footnote-3554488 +Ref: Summary – Release highlights<3>-Footnote-4554531 +Ref: Summary – Release highlights<3>-Footnote-5554574 +Ref: Summary – Release highlights<3>-Footnote-6554617 +Ref: Summary – Release highlights<3>-Footnote-7554660 +Node: New Features<4>554703 +Ref: whatsnew/3 5 new-features554857 +Ref: 63d554857 +Node: PEP 492 - Coroutines with async and await syntax555953 +Ref: whatsnew/3 5 pep-492-coroutines-with-async-and-await-syntax556124 +Ref: 63e556124 +Ref: whatsnew/3 5 whatsnew-pep-492556124 +Ref: 62b556124 +Ref: PEP 492 - Coroutines with async and await syntax-Footnote-1559203 +Ref: PEP 492 - Coroutines with async and await syntax-Footnote-2559252 +Node: PEP 465 - A dedicated infix operator for matrix multiplication559301 +Ref: whatsnew/3 5 pep-465-a-dedicated-infix-operator-for-matrix-multiplication559527 +Ref: 647559527 +Ref: whatsnew/3 5 whatsnew-pep-465559527 +Ref: 62c559527 +Ref: PEP 465 - A dedicated infix operator for matrix multiplication-Footnote-1560930 +Ref: PEP 465 - A dedicated infix operator for matrix multiplication-Footnote-2560979 +Node: PEP 448 - Additional Unpacking Generalizations561028 +Ref: whatsnew/3 5 pep-448-additional-unpacking-generalizations561266 +Ref: 64b561266 +Ref: whatsnew/3 5 whatsnew-pep-448561266 +Ref: 62d561266 +Ref: PEP 448 - Additional Unpacking Generalizations-Footnote-1562359 +Ref: PEP 448 - Additional Unpacking Generalizations-Footnote-2562408 +Node: PEP 461 - percent formatting support for bytes and bytearray562457 +Ref: whatsnew/3 5 pep-461-percent-formatting-support-for-bytes-and-bytearray562653 +Ref: 64f562653 +Ref: whatsnew/3 5 whatsnew-pep-461562653 +Ref: 630562653 +Ref: PEP 461 - percent formatting support for bytes and bytearray-Footnote-1564189 +Ref: PEP 461 - percent formatting support for bytes and bytearray-Footnote-2564238 +Node: PEP 484 - Type Hints564287 +Ref: whatsnew/3 5 pep-484-type-hints564509 +Ref: 651564509 +Ref: whatsnew/3 5 whatsnew-pep-484564509 +Ref: 62e564509 +Ref: PEP 484 - Type Hints-Footnote-1566156 +Ref: PEP 484 - Type Hints-Footnote-2566205 +Ref: PEP 484 - Type Hints-Footnote-3566254 +Ref: PEP 484 - Type Hints-Footnote-4566283 +Ref: PEP 484 - Type Hints-Footnote-5566332 +Node: PEP 471 - os scandir function – a better and faster directory iterator566381 +Ref: whatsnew/3 5 pep-471-os-scandir-function-a-better-and-faster-directory-iterator566588 +Ref: 653566588 +Ref: whatsnew/3 5 whatsnew-pep-471566588 +Ref: 639566588 +Ref: PEP 471 - os scandir function – a better and faster directory iterator-Footnote-1567944 +Ref: PEP 471 - os scandir function – a better and faster directory iterator-Footnote-2567993 +Node: PEP 475 Retry system calls failing with EINTR568042 +Ref: whatsnew/3 5 pep-475-retry-system-calls-failing-with-eintr568284 +Ref: 656568284 +Ref: whatsnew/3 5 whatsnew-pep-475568284 +Ref: 657568284 +Ref: PEP 475 Retry system calls failing with EINTR-Footnote-1571348 +Ref: PEP 475 Retry system calls failing with EINTR-Footnote-2571397 +Node: PEP 479 Change StopIteration handling inside generators571446 +Ref: whatsnew/3 5 pep-479-change-stopiteration-handling-inside-generators571667 +Ref: 681571667 +Ref: whatsnew/3 5 whatsnew-pep-479571667 +Ref: 5d7571667 +Ref: PEP 479 Change StopIteration handling inside generators-Footnote-1573597 +Ref: PEP 479 Change StopIteration handling inside generators-Footnote-2573646 +Node: PEP 485 A function for testing approximate equality573695 +Ref: whatsnew/3 5 pep-485-a-function-for-testing-approximate-equality573933 +Ref: 684573933 +Ref: whatsnew/3 5 whatsnew-pep-485573933 +Ref: 685573933 +Ref: PEP 485 A function for testing approximate equality-Footnote-1575116 +Ref: PEP 485 A function for testing approximate equality-Footnote-2575165 +Node: PEP 486 Make the Python Launcher aware of virtual environments575214 +Ref: whatsnew/3 5 pep-486-make-the-python-launcher-aware-of-virtual-environments575429 +Ref: 688575429 +Ref: whatsnew/3 5 whatsnew-pep-486575429 +Ref: 689575429 +Ref: PEP 486 Make the Python Launcher aware of virtual environments-Footnote-1576003 +Ref: PEP 486 Make the Python Launcher aware of virtual environments-Footnote-2576052 +Ref: PEP 486 Make the Python Launcher aware of virtual environments-Footnote-3576101 +Node: PEP 488 Elimination of PYO files576150 +Ref: whatsnew/3 5 pep-488-elimination-of-pyo-files576365 +Ref: 68a576365 +Ref: whatsnew/3 5 whatsnew-pep-488576365 +Ref: 635576365 +Ref: PEP 488 Elimination of PYO files-Footnote-1577231 +Ref: PEP 488 Elimination of PYO files-Footnote-2577280 +Node: PEP 489 Multi-phase extension module initialization577329 +Ref: whatsnew/3 5 pep-489-multi-phase-extension-module-initialization577473 +Ref: 68d577473 +Ref: whatsnew/3 5 whatsnew-pep-489577473 +Ref: 636577473 +Ref: PEP 489 Multi-phase extension module initialization-Footnote-1578236 +Ref: PEP 489 Multi-phase extension module initialization-Footnote-2578285 +Ref: PEP 489 Multi-phase extension module initialization-Footnote-3578334 +Node: Other Language Changes<4>578383 +Ref: whatsnew/3 5 other-language-changes578518 +Ref: 68e578518 +Ref: Other Language Changes<4>-Footnote-1579516 +Ref: Other Language Changes<4>-Footnote-2579559 +Ref: Other Language Changes<4>-Footnote-3579602 +Ref: Other Language Changes<4>-Footnote-4579645 +Ref: Other Language Changes<4>-Footnote-5579688 +Ref: Other Language Changes<4>-Footnote-6579731 +Ref: Other Language Changes<4>-Footnote-7579774 +Node: New Modules<4>579817 +Ref: whatsnew/3 5 new-modules579956 +Ref: 690579956 +Node: typing<3>580045 +Ref: whatsnew/3 5 typing580123 +Ref: 691580123 +Node: zipapp<2>580334 +Ref: whatsnew/3 5 whatsnew-zipapp580412 +Ref: 62f580412 +Ref: whatsnew/3 5 zipapp580412 +Ref: 692580412 +Ref: zipapp<2>-Footnote-1581122 +Ref: zipapp<2>-Footnote-2581171 +Ref: zipapp<2>-Footnote-3581216 +Ref: zipapp<2>-Footnote-4581259 +Node: Improved Modules<4>581308 +Ref: whatsnew/3 5 improved-modules581448 +Ref: 693581448 +Node: argparse<2>583012 +Ref: whatsnew/3 5 argparse583098 +Ref: 694583098 +Ref: argparse<2>-Footnote-1583421 +Node: asyncio<5>583464 +Ref: whatsnew/3 5 asyncio583562 +Ref: 698583562 +Ref: asyncio<5>-Footnote-1587040 +Ref: asyncio<5>-Footnote-2587083 +Ref: asyncio<5>-Footnote-3587126 +Node: bz2587171 +Ref: whatsnew/3 5 bz2587261 +Ref: 6a2587261 +Ref: bz2-Footnote-1587516 +Node: cgi587559 +Ref: whatsnew/3 5 cgi587647 +Ref: 6a4587647 +Ref: cgi-Footnote-1587842 +Node: cmath<2>587885 +Ref: whatsnew/3 5 cmath587974 +Ref: 6a5587974 +Ref: cmath<2>-Footnote-1588188 +Node: code588231 +Ref: whatsnew/3 5 code588331 +Ref: 6a6588331 +Ref: code-Footnote-1588582 +Node: collections<5>588625 +Ref: whatsnew/3 5 collections588732 +Ref: 6a8588732 +Ref: whatsnew/3 5 whatsnew-ordereddict588773 +Ref: 637588773 +Ref: collections<5>-Footnote-1590052 +Ref: collections<5>-Footnote-2590095 +Ref: collections<5>-Footnote-3590138 +Ref: collections<5>-Footnote-4590181 +Ref: collections<5>-Footnote-5590224 +Node: collections abc590267 +Ref: whatsnew/3 5 collections-abc590383 +Ref: 6b4590383 +Ref: collections abc-Footnote-1591053 +Ref: collections abc-Footnote-2591096 +Ref: collections abc-Footnote-3591139 +Ref: collections abc-Footnote-4591182 +Node: compileall<2>591229 +Ref: whatsnew/3 5 compileall591352 +Ref: 6ba591352 +Ref: compileall<2>-Footnote-1592229 +Ref: compileall<2>-Footnote-2592272 +Ref: compileall<2>-Footnote-3592315 +Node: concurrent futures<3>592358 +Ref: whatsnew/3 5 concurrent-futures592478 +Ref: 6bd592478 +Ref: concurrent futures<3>-Footnote-1592980 +Ref: concurrent futures<3>-Footnote-2593023 +Node: configparser593066 +Ref: whatsnew/3 5 configparser593186 +Ref: 6bf593186 +Ref: configparser-Footnote-1594137 +Node: contextlib<3>594180 +Ref: whatsnew/3 5 contextlib594285 +Ref: 6c0594285 +Ref: contextlib<3>-Footnote-1594895 +Node: csv<2>594938 +Ref: whatsnew/3 5 csv595040 +Ref: 6c3595040 +Ref: csv<2>-Footnote-1595244 +Node: curses<2>595287 +Ref: whatsnew/3 5 curses595382 +Ref: 6c5595382 +Ref: curses<2>-Footnote-1595663 +Node: dbm<4>595705 +Ref: whatsnew/3 5 dbm595801 +Ref: 6c7595801 +Ref: dbm<4>-Footnote-1596004 +Node: difflib596047 +Ref: whatsnew/3 5 difflib596146 +Ref: 6c8596146 +Ref: difflib-Footnote-1596682 +Ref: difflib-Footnote-2596724 +Node: distutils<4>596767 +Ref: whatsnew/3 5 distutils596867 +Ref: 6cb596867 +Ref: distutils<4>-Footnote-1597322 +Ref: distutils<4>-Footnote-2597364 +Node: doctest597407 +Ref: whatsnew/3 5 doctest597508 +Ref: 6cc597508 +Ref: doctest-Footnote-1597793 +Node: email<2>597836 +Ref: whatsnew/3 5 email597932 +Ref: 6cf597932 +Ref: email<2>-Footnote-1599043 +Ref: email<2>-Footnote-2599086 +Ref: email<2>-Footnote-3599129 +Ref: email<2>-Footnote-4599178 +Ref: email<2>-Footnote-5599227 +Ref: email<2>-Footnote-6599270 +Node: enum<4>599313 +Ref: whatsnew/3 5 enum599417 +Ref: 6d5599417 +Ref: enum<4>-Footnote-1599808 +Node: faulthandler<2>599851 +Ref: whatsnew/3 5 faulthandler599959 +Ref: 6d6599959 +Ref: faulthandler<2>-Footnote-1600271 +Node: functools<3>600314 +Ref: whatsnew/3 5 functools600419 +Ref: 6da600419 +Ref: whatsnew/3 5 whatsnew-lrucache600458 +Ref: 63a600458 +Ref: functools<3>-Footnote-1600684 +Node: glob600727 +Ref: whatsnew/3 5 glob600824 +Ref: 6db600824 +Ref: glob-Footnote-1601076 +Node: gzip<2>601119 +Ref: whatsnew/3 5 gzip601209 +Ref: 6dc601209 +Ref: gzip<2>-Footnote-1601434 +Node: heapq601477 +Ref: whatsnew/3 5 heapq601567 +Ref: 6de601567 +Ref: heapq-Footnote-1602207 +Node: http602250 +Ref: whatsnew/3 5 http602347 +Ref: 6e1602347 +Ref: http-Footnote-1602593 +Node: http client<3>602636 +Ref: whatsnew/3 5 http-client602747 +Ref: 6e3602747 +Ref: http client<3>-Footnote-1603491 +Node: idlelib and IDLE<3>603533 +Ref: whatsnew/3 5 idlelib-and-idle603647 +Ref: 6e7603647 +Node: imaplib604041 +Ref: whatsnew/3 5 imaplib604147 +Ref: 6e8604147 +Ref: imaplib-Footnote-1605026 +Ref: imaplib-Footnote-2605068 +Ref: imaplib-Footnote-3605117 +Ref: imaplib-Footnote-4605166 +Ref: imaplib-Footnote-5605215 +Ref: imaplib-Footnote-6605258 +Node: imghdr605301 +Ref: whatsnew/3 5 imghdr605400 +Ref: 6ec605400 +Ref: imghdr-Footnote-1605694 +Ref: imghdr-Footnote-2605725 +Ref: imghdr-Footnote-3605768 +Ref: imghdr-Footnote-4605811 +Node: importlib<5>605854 +Ref: whatsnew/3 5 importlib605956 +Ref: 6ee605956 +Ref: importlib<5>-Footnote-1606791 +Ref: importlib<5>-Footnote-2606834 +Ref: importlib<5>-Footnote-3606877 +Node: inspect<3>606920 +Ref: whatsnew/3 5 inspect607021 +Ref: 6f2607021 +Ref: inspect<3>-Footnote-1608578 +Ref: inspect<3>-Footnote-2608621 +Ref: inspect<3>-Footnote-3608664 +Ref: inspect<3>-Footnote-4608707 +Ref: inspect<3>-Footnote-5608750 +Ref: inspect<3>-Footnote-6608793 +Ref: inspect<3>-Footnote-7608836 +Ref: inspect<3>-Footnote-8608879 +Node: io<3>608922 +Ref: whatsnew/3 5 io609023 +Ref: 700609023 +Ref: io<3>-Footnote-1609321 +Node: ipaddress<2>609364 +Ref: whatsnew/3 5 ipaddress609462 +Ref: 704609462 +Ref: ipaddress<2>-Footnote-1610490 +Ref: ipaddress<2>-Footnote-2610533 +Node: json<2>610576 +Ref: whatsnew/3 5 json610678 +Ref: 705610678 +Ref: json<2>-Footnote-1611178 +Ref: json<2>-Footnote-2611221 +Node: linecache611264 +Ref: whatsnew/3 5 linecache611363 +Ref: 707611363 +Ref: linecache-Footnote-1611773 +Node: locale<3>611816 +Ref: whatsnew/3 5 locale611918 +Ref: 70a611918 +Ref: locale<3>-Footnote-1612483 +Node: logging<4>612526 +Ref: whatsnew/3 5 logging612623 +Ref: 70c612623 +Ref: logging<4>-Footnote-1613593 +Ref: logging<4>-Footnote-2613636 +Node: lzma613679 +Ref: whatsnew/3 5 lzma613774 +Ref: 713613774 +Ref: lzma-Footnote-1614036 +Node: math<4>614079 +Ref: whatsnew/3 5 math614182 +Ref: 715614182 +Ref: math<4>-Footnote-1614730 +Ref: math<4>-Footnote-2614773 +Ref: math<4>-Footnote-3614816 +Node: multiprocessing<4>614859 +Ref: whatsnew/3 5 multiprocessing614966 +Ref: 718614966 +Ref: multiprocessing<4>-Footnote-1615216 +Node: operator615259 +Ref: whatsnew/3 5 operator615364 +Ref: 71a615364 +Ref: operator-Footnote-1615769 +Ref: operator-Footnote-2615812 +Node: os<5>615855 +Ref: whatsnew/3 5 os615952 +Ref: 71f615952 +Ref: os<5>-Footnote-1617800 +Ref: os<5>-Footnote-2617843 +Ref: os<5>-Footnote-3617886 +Ref: os<5>-Footnote-4617929 +Ref: os<5>-Footnote-5617972 +Ref: os<5>-Footnote-6618015 +Node: pathlib<4>618058 +Ref: whatsnew/3 5 pathlib618156 +Ref: 727618156 +Ref: pathlib<4>-Footnote-1619627 +Ref: pathlib<4>-Footnote-2619670 +Ref: pathlib<4>-Footnote-3619713 +Ref: pathlib<4>-Footnote-4619756 +Ref: pathlib<4>-Footnote-5619799 +Node: pickle<3>619842 +Ref: whatsnew/3 5 pickle619941 +Ref: 730619941 +Ref: pickle<3>-Footnote-1620257 +Node: poplib620300 +Ref: whatsnew/3 5 poplib620394 +Ref: 731620394 +Ref: poplib-Footnote-1620636 +Ref: poplib-Footnote-2620685 +Node: re<4>620728 +Ref: whatsnew/3 5 re620824 +Ref: 733620824 +Ref: re<4>-Footnote-1622060 +Ref: re<4>-Footnote-2622102 +Ref: re<4>-Footnote-3622145 +Ref: re<4>-Footnote-4622190 +Node: readline<2>622233 +Ref: whatsnew/3 5 readline622332 +Ref: 73b622332 +Ref: readline<2>-Footnote-1622595 +Node: selectors622638 +Ref: whatsnew/3 5 selectors622741 +Ref: 73d622741 +Ref: selectors-Footnote-1622963 +Node: shutil<2>623006 +Ref: whatsnew/3 5 shutil623107 +Ref: 73e623107 +Ref: shutil<2>-Footnote-1623586 +Ref: shutil<2>-Footnote-2623629 +Node: signal<2>623671 +Ref: whatsnew/3 5 signal623768 +Ref: 73f623768 +Ref: signal<2>-Footnote-1624235 +Ref: signal<2>-Footnote-2624278 +Node: smtpd624321 +Ref: whatsnew/3 5 smtpd624416 +Ref: 740624416 +Ref: smtpd-Footnote-1626100 +Ref: smtpd-Footnote-2626143 +Ref: smtpd-Footnote-3626192 +Ref: smtpd-Footnote-4626235 +Ref: smtpd-Footnote-5626284 +Ref: smtpd-Footnote-6626327 +Node: smtplib626370 +Ref: whatsnew/3 5 smtplib626462 +Ref: 741626462 +Ref: smtplib-Footnote-1627083 +Ref: smtplib-Footnote-2627126 +Ref: smtplib-Footnote-3627169 +Ref: smtplib-Footnote-4627218 +Node: sndhdr627261 +Ref: whatsnew/3 5 sndhdr627357 +Ref: 746627357 +Ref: sndhdr-Footnote-1627572 +Node: socket<5>627615 +Ref: whatsnew/3 5 socket627710 +Ref: 749627710 +Ref: socket<5>-Footnote-1628681 +Ref: socket<5>-Footnote-2628724 +Ref: socket<5>-Footnote-3628767 +Ref: socket<5>-Footnote-4628810 +Node: ssl<6>628853 +Ref: whatsnew/3 5 ssl628952 +Ref: 74d628952 +Node: Memory BIO Support629081 +Ref: whatsnew/3 5 memory-bio-support629197 +Ref: 74e629197 +Ref: whatsnew/3 5 whatsnew-sslmemorybio629197 +Ref: 638629197 +Ref: Memory BIO Support-Footnote-1630080 +Node: Application-Layer Protocol Negotiation Support630123 +Ref: whatsnew/3 5 application-layer-protocol-negotiation-support630261 +Ref: 750630261 +Ref: Application-Layer Protocol Negotiation Support-Footnote-1630966 +Ref: Application-Layer Protocol Negotiation Support-Footnote-2631009 +Node: Other Changes631058 +Ref: whatsnew/3 5 other-changes631169 +Ref: 754631169 +Ref: Other Changes-Footnote-1632762 +Ref: Other Changes-Footnote-2632805 +Ref: Other Changes-Footnote-3632848 +Ref: Other Changes-Footnote-4632891 +Ref: Other Changes-Footnote-5632940 +Ref: Other Changes-Footnote-6632983 +Ref: Other Changes-Footnote-7633026 +Ref: Other Changes-Footnote-8633069 +Node: sqlite3<3>633112 +Ref: whatsnew/3 5 sqlite3633215 +Ref: 75d633215 +Ref: sqlite3<3>-Footnote-1633544 +Ref: sqlite3<3>-Footnote-2633587 +Node: subprocess<3>633630 +Ref: whatsnew/3 5 subprocess633733 +Ref: 75f633733 +Ref: whatsnew/3 5 whatsnew-subprocess633733 +Ref: 63b633733 +Ref: subprocess<3>-Footnote-1634747 +Node: sys<5>634790 +Ref: whatsnew/3 5 sys634892 +Ref: 761634892 +Ref: sys<5>-Footnote-1635542 +Ref: sys<5>-Footnote-2635585 +Node: sysconfig635628 +Ref: whatsnew/3 5 sysconfig635727 +Ref: 764635727 +Ref: sysconfig-Footnote-1635960 +Node: tarfile<2>636003 +Ref: whatsnew/3 5 tarfile636108 +Ref: 765636108 +Ref: tarfile<2>-Footnote-1637019 +Ref: tarfile<2>-Footnote-2637062 +Ref: tarfile<2>-Footnote-3637105 +Node: threading<3>637148 +Ref: whatsnew/3 5 threading637251 +Ref: 76b637251 +Ref: threading<3>-Footnote-1637503 +Node: time<4>637546 +Ref: whatsnew/3 5 time637648 +Ref: 76e637648 +Ref: time<4>-Footnote-1637825 +Node: timeit<2>637868 +Ref: whatsnew/3 5 timeit637968 +Ref: 770637968 +Ref: timeit<2>-Footnote-1638435 +Ref: timeit<2>-Footnote-2638478 +Node: tkinter<5>638520 +Ref: whatsnew/3 5 tkinter638625 +Ref: 772638625 +Ref: tkinter<5>-Footnote-1638957 +Node: traceback<2>639000 +Ref: whatsnew/3 5 traceback639104 +Ref: 773639104 +Ref: whatsnew/3 5 whatsnew-traceback639104 +Ref: 63c639104 +Ref: traceback<2>-Footnote-1639687 +Ref: traceback<2>-Footnote-2639730 +Ref: traceback<2>-Footnote-3639773 +Node: types<2>639816 +Ref: whatsnew/3 5 types639924 +Ref: 77b639924 +Ref: types<2>-Footnote-1640368 +Ref: types<2>-Footnote-2640411 +Node: unicodedata<4>640454 +Ref: whatsnew/3 5 unicodedata640561 +Ref: 77e640561 +Ref: unicodedata<4>-Footnote-1640713 +Node: unittest<3>640763 +Ref: whatsnew/3 5 unittest640878 +Ref: 77f640878 +Ref: unittest<3>-Footnote-1641672 +Ref: unittest<3>-Footnote-2641715 +Ref: unittest<3>-Footnote-3641758 +Node: unittest mock<3>641801 +Ref: whatsnew/3 5 unittest-mock641908 +Ref: 783641908 +Ref: unittest mock<3>-Footnote-1642849 +Ref: unittest mock<3>-Footnote-2642892 +Ref: unittest mock<3>-Footnote-3642935 +Ref: unittest mock<3>-Footnote-4642978 +Ref: unittest mock<3>-Footnote-5643021 +Ref: unittest mock<3>-Footnote-6643064 +Node: urllib643107 +Ref: whatsnew/3 5 urllib643210 +Ref: 789643210 +Ref: urllib-Footnote-1644370 +Ref: urllib-Footnote-2644413 +Ref: urllib-Footnote-3644455 +Ref: urllib-Footnote-4644498 +Ref: urllib-Footnote-5644541 +Ref: urllib-Footnote-6644590 +Ref: urllib-Footnote-7644639 +Ref: urllib-Footnote-8644688 +Node: wsgiref644731 +Ref: whatsnew/3 5 wsgiref644827 +Ref: 78e644827 +Ref: wsgiref-Footnote-1645065 +Node: xmlrpc<2>645107 +Ref: whatsnew/3 5 xmlrpc645204 +Ref: 790645204 +Ref: xmlrpc<2>-Footnote-1645574 +Ref: xmlrpc<2>-Footnote-2645617 +Node: xml sax645660 +Ref: whatsnew/3 5 xml-sax645760 +Ref: 791645760 +Ref: xml sax-Footnote-1646093 +Ref: xml sax-Footnote-2646135 +Node: zipfile<3>646178 +Ref: whatsnew/3 5 zipfile646260 +Ref: 794646260 +Ref: zipfile<3>-Footnote-1646600 +Ref: zipfile<3>-Footnote-2646643 +Node: Other module-level changes646686 +Ref: whatsnew/3 5 other-module-level-changes646828 +Ref: 795646828 +Ref: Other module-level changes-Footnote-1647159 +Node: Optimizations<4>647202 +Ref: whatsnew/3 5 optimizations647351 +Ref: 796647351 +Ref: Optimizations<4>-Footnote-1650408 +Ref: Optimizations<4>-Footnote-2650451 +Ref: Optimizations<4>-Footnote-3650494 +Ref: Optimizations<4>-Footnote-4650537 +Ref: Optimizations<4>-Footnote-5650580 +Ref: Optimizations<4>-Footnote-6650623 +Ref: Optimizations<4>-Footnote-7650666 +Ref: Optimizations<4>-Footnote-8650709 +Ref: Optimizations<4>-Footnote-9650752 +Ref: Optimizations<4>-Footnote-10650795 +Ref: Optimizations<4>-Footnote-11650839 +Ref: Optimizations<4>-Footnote-12650883 +Ref: Optimizations<4>-Footnote-13650927 +Ref: Optimizations<4>-Footnote-14650971 +Ref: Optimizations<4>-Footnote-15651015 +Ref: Optimizations<4>-Footnote-16651059 +Ref: Optimizations<4>-Footnote-17651103 +Ref: Optimizations<4>-Footnote-18651147 +Ref: Optimizations<4>-Footnote-19651191 +Ref: Optimizations<4>-Footnote-20651235 +Node: Build and C API Changes<3>651279 +Ref: whatsnew/3 5 build-and-c-api-changes651415 +Ref: 7a4651415 +Ref: Build and C API Changes<3>-Footnote-1654965 +Ref: Build and C API Changes<3>-Footnote-2655008 +Ref: Build and C API Changes<3>-Footnote-3655051 +Ref: Build and C API Changes<3>-Footnote-4655094 +Ref: Build and C API Changes<3>-Footnote-5655137 +Ref: Build and C API Changes<3>-Footnote-6655180 +Ref: Build and C API Changes<3>-Footnote-7655229 +Ref: Build and C API Changes<3>-Footnote-8655272 +Ref: Build and C API Changes<3>-Footnote-9655315 +Ref: Build and C API Changes<3>-Footnote-10655364 +Node: Deprecated<3>655403 +Ref: whatsnew/3 5 deprecated655533 +Ref: 7b0655533 +Node: New Keywords<2>655810 +Ref: whatsnew/3 5 new-keywords655913 +Ref: 7b1655913 +Ref: New Keywords<2>-Footnote-1656187 +Node: Deprecated Python Behavior<2>656236 +Ref: whatsnew/3 5 deprecated-python-behavior656377 +Ref: 7b2656377 +Node: Unsupported Operating Systems656792 +Ref: whatsnew/3 5 unsupported-operating-systems656968 +Ref: 7b3656968 +Ref: Unsupported Operating Systems-Footnote-1657209 +Node: Deprecated Python modules functions and methods<3>657258 +Ref: whatsnew/3 5 deprecated-python-modules-functions-and-methods657396 +Ref: 7b4657396 +Ref: Deprecated Python modules functions and methods<3>-Footnote-1660334 +Ref: Deprecated Python modules functions and methods<3>-Footnote-2660377 +Ref: Deprecated Python modules functions and methods<3>-Footnote-3660419 +Ref: Deprecated Python modules functions and methods<3>-Footnote-4660462 +Ref: Deprecated Python modules functions and methods<3>-Footnote-5660505 +Ref: Deprecated Python modules functions and methods<3>-Footnote-6660548 +Ref: Deprecated Python modules functions and methods<3>-Footnote-7660591 +Ref: Deprecated Python modules functions and methods<3>-Footnote-8660634 +Node: Removed<2>660677 +Ref: whatsnew/3 5 removed660802 +Ref: 7ba660802 +Node: API and Feature Removals<4>660900 +Ref: whatsnew/3 5 api-and-feature-removals660974 +Ref: 7bb660974 +Ref: API and Feature Removals<4>-Footnote-1661831 +Ref: API and Feature Removals<4>-Footnote-2661873 +Node: Porting to Python 3 5661916 +Ref: whatsnew/3 5 porting-to-python-3-5662059 +Ref: 7bc662059 +Node: Changes in Python behavior<2>662404 +Ref: whatsnew/3 5 changes-in-python-behavior662528 +Ref: 7bd662528 +Node: Changes in the Python API<4>662938 +Ref: whatsnew/3 5 changes-in-the-python-api663094 +Ref: 7be663094 +Ref: Changes in the Python API<4>-Footnote-1670020 +Ref: Changes in the Python API<4>-Footnote-2670069 +Ref: Changes in the Python API<4>-Footnote-3670112 +Ref: Changes in the Python API<4>-Footnote-4670155 +Ref: Changes in the Python API<4>-Footnote-5670198 +Ref: Changes in the Python API<4>-Footnote-6670241 +Ref: Changes in the Python API<4>-Footnote-7670284 +Ref: Changes in the Python API<4>-Footnote-8670327 +Ref: Changes in the Python API<4>-Footnote-9670370 +Ref: Changes in the Python API<4>-Footnote-10670413 +Ref: Changes in the Python API<4>-Footnote-11670457 +Ref: Changes in the Python API<4>-Footnote-12670501 +Ref: Changes in the Python API<4>-Footnote-13670544 +Ref: Changes in the Python API<4>-Footnote-14670594 +Ref: Changes in the Python API<4>-Footnote-15670644 +Ref: Changes in the Python API<4>-Footnote-16670688 +Ref: Changes in the Python API<4>-Footnote-17670732 +Ref: Changes in the Python API<4>-Footnote-18670776 +Node: Changes in the C API<4>670819 +Ref: whatsnew/3 5 changes-in-the-c-api670937 +Ref: 7c9670937 +Ref: Changes in the C API<4>-Footnote-1672209 +Ref: Changes in the C API<4>-Footnote-2672252 +Ref: Changes in the C API<4>-Footnote-3672295 +Node: Notable changes in Python 3 5 4672344 +Ref: whatsnew/3 5 notable-changes-in-python-3-5-4672468 +Ref: 7ce672468 +Node: New make regen-all build target<2>672705 +Ref: whatsnew/3 5 new-make-regen-all-build-target672853 +Ref: 7cf672853 +Ref: New make regen-all build target<2>-Footnote-1673640 +Ref: New make regen-all build target<2>-Footnote-2673707 +Node: Removal of make touch build target<2>673750 +Ref: whatsnew/3 5 removal-of-make-touch-build-target673898 +Ref: 7d0673898 +Ref: Removal of make touch build target<2>-Footnote-1674328 +Node: What’s New In Python 3 4674371 +Ref: whatsnew/3 4 doc674526 +Ref: 7d1674526 +Ref: whatsnew/3 4 what-s-new-in-python-3-4674526 +Ref: 7d2674526 +Ref: What’s New In Python 3 4-Footnote-1675260 +Ref: What’s New In Python 3 4-Footnote-2675320 +Node: Summary – Release Highlights<2>675369 +Ref: whatsnew/3 4 summary-release-highlights675489 +Ref: 7d3675489 +Ref: Summary – Release Highlights<2>-Footnote-1679785 +Ref: Summary – Release Highlights<2>-Footnote-2679834 +Ref: Summary – Release Highlights<2>-Footnote-3679883 +Ref: Summary – Release Highlights<2>-Footnote-4679926 +Ref: Summary – Release Highlights<2>-Footnote-5679975 +Ref: Summary – Release Highlights<2>-Footnote-6680018 +Ref: Summary – Release Highlights<2>-Footnote-7680067 +Ref: Summary – Release Highlights<2>-Footnote-8680116 +Ref: Summary – Release Highlights<2>-Footnote-9680165 +Ref: Summary – Release Highlights<2>-Footnote-10680214 +Ref: Summary – Release Highlights<2>-Footnote-11680264 +Ref: Summary – Release Highlights<2>-Footnote-12680314 +Ref: Summary – Release Highlights<2>-Footnote-13680364 +Ref: Summary – Release Highlights<2>-Footnote-14680414 +Ref: Summary – Release Highlights<2>-Footnote-15680464 +Ref: Summary – Release Highlights<2>-Footnote-16680507 +Ref: Summary – Release Highlights<2>-Footnote-17680551 +Ref: Summary – Release Highlights<2>-Footnote-18680601 +Ref: Summary – Release Highlights<2>-Footnote-19680651 +Ref: Summary – Release Highlights<2>-Footnote-20680695 +Ref: Summary – Release Highlights<2>-Footnote-21680741 +Ref: Summary – Release Highlights<2>-Footnote-22680791 +Ref: Summary – Release Highlights<2>-Footnote-23680841 +Ref: Summary – Release Highlights<2>-Footnote-24680885 +Ref: Summary – Release Highlights<2>-Footnote-25680935 +Node: New Features<5>680985 +Ref: whatsnew/3 4 new-features681128 +Ref: 7ef681128 +Node: PEP 453 Explicit Bootstrapping of PIP in Python Installations681618 +Ref: whatsnew/3 4 pep-453-explicit-bootstrapping-of-pip-in-python-installations681798 +Ref: 7f0681798 +Ref: whatsnew/3 4 whatsnew-pep-453681798 +Ref: 7d4681798 +Node: Bootstrapping pip By Default682010 +Ref: whatsnew/3 4 bootstrapping-pip-by-default682166 +Ref: 7f1682166 +Ref: Bootstrapping pip By Default-Footnote-1685066 +Ref: Bootstrapping pip By Default-Footnote-2685115 +Node: Documentation Changes685210 +Ref: whatsnew/3 4 documentation-changes685366 +Ref: 7f4685366 +Ref: Documentation Changes-Footnote-1686317 +Ref: Documentation Changes-Footnote-2686354 +Node: PEP 446 Newly Created File Descriptors Are Non-Inheritable686403 +Ref: whatsnew/3 4 pep-446-newly-created-file-descriptors-are-non-inheritable686622 +Ref: 7f9686622 +Ref: whatsnew/3 4 whatsnew-pep-446686622 +Ref: 7d5686622 +Ref: PEP 446 Newly Created File Descriptors Are Non-Inheritable-Footnote-1687640 +Ref: PEP 446 Newly Created File Descriptors Are Non-Inheritable-Footnote-2687689 +Node: Improvements to Codec Handling687738 +Ref: whatsnew/3 4 codec-handling-improvements687943 +Ref: 7d7687943 +Ref: whatsnew/3 4 improvements-to-codec-handling687943 +Ref: 801687943 +Ref: Improvements to Codec Handling-Footnote-1692149 +Ref: Improvements to Codec Handling-Footnote-2692191 +Ref: Improvements to Codec Handling-Footnote-3692234 +Ref: Improvements to Codec Handling-Footnote-4692277 +Node: PEP 451 A ModuleSpec Type for the Import System692320 +Ref: whatsnew/3 4 pep-451-a-modulespec-type-for-the-import-system692492 +Ref: 806692492 +Ref: whatsnew/3 4 whatsnew-pep-451692492 +Ref: 7d8692492 +Ref: PEP 451 A ModuleSpec Type for the Import System-Footnote-1693443 +Ref: PEP 451 A ModuleSpec Type for the Import System-Footnote-2693492 +Node: Other Language Changes<5>693572 +Ref: whatsnew/3 4 other-language-changes693705 +Ref: 808693705 +Ref: Other Language Changes<5>-Footnote-1696342 +Ref: Other Language Changes<5>-Footnote-2696385 +Ref: Other Language Changes<5>-Footnote-3696428 +Ref: Other Language Changes<5>-Footnote-4696471 +Ref: Other Language Changes<5>-Footnote-5696516 +Ref: Other Language Changes<5>-Footnote-6696559 +Ref: Other Language Changes<5>-Footnote-7696602 +Ref: Other Language Changes<5>-Footnote-8696645 +Ref: Other Language Changes<5>-Footnote-9696688 +Ref: Other Language Changes<5>-Footnote-10696731 +Ref: Other Language Changes<5>-Footnote-11696775 +Ref: Other Language Changes<5>-Footnote-12696825 +Node: New Modules<5>696869 +Ref: whatsnew/3 4 new-modules696998 +Ref: 80d696998 +Node: asyncio<6>697209 +Ref: whatsnew/3 4 asyncio697288 +Ref: 80e697288 +Ref: whatsnew/3 4 whatsnew-asyncio697288 +Ref: 7da697288 +Ref: asyncio<6>-Footnote-1697886 +Ref: asyncio<6>-Footnote-2697935 +Node: ensurepip697984 +Ref: whatsnew/3 4 ensurepip698079 +Ref: 80f698079 +Ref: whatsnew/3 4 whatsnew-ensurepip698079 +Ref: 7db698079 +Ref: ensurepip-Footnote-1699351 +Node: enum<5>699400 +Ref: whatsnew/3 4 enum699495 +Ref: 810699495 +Ref: whatsnew/3 4 whatsnew-enum699495 +Ref: 7dc699495 +Ref: enum<5>-Footnote-1700065 +Ref: enum<5>-Footnote-2700114 +Node: pathlib<5>700163 +Ref: whatsnew/3 4 pathlib700261 +Ref: 811700261 +Ref: whatsnew/3 4 whatsnew-pathlib700261 +Ref: 7dd700261 +Ref: pathlib<5>-Footnote-1700879 +Node: selectors<2>700928 +Ref: whatsnew/3 4 selectors701032 +Ref: 812701032 +Ref: whatsnew/3 4 whatsnew-selectors701032 +Ref: 7de701032 +Ref: selectors<2>-Footnote-1701290 +Node: statistics<3>701339 +Ref: whatsnew/3 4 statistics701447 +Ref: 813701447 +Ref: whatsnew/3 4 whatsnew-statistics701447 +Ref: 7df701447 +Ref: statistics<3>-Footnote-1701912 +Ref: statistics<3>-Footnote-2701961 +Node: tracemalloc<3>702010 +Ref: whatsnew/3 4 tracemalloc702097 +Ref: 814702097 +Ref: whatsnew/3 4 whatsnew-tracemalloc702097 +Ref: 7e0702097 +Ref: tracemalloc<3>-Footnote-1702762 +Ref: tracemalloc<3>-Footnote-2702811 +Node: Improved Modules<5>702860 +Ref: whatsnew/3 4 improved-modules703004 +Ref: 815703004 +Node: abc704316 +Ref: whatsnew/3 4 abc704391 +Ref: 816704391 +Ref: abc-Footnote-1704912 +Ref: abc-Footnote-2704955 +Node: aifc<2>704998 +Ref: whatsnew/3 4 aifc705093 +Ref: 81a705093 +Ref: aifc<2>-Footnote-1705731 +Ref: aifc<2>-Footnote-2705774 +Ref: aifc<2>-Footnote-3705817 +Node: argparse<3>705859 +Ref: whatsnew/3 4 argparse705958 +Ref: 81f705958 +Ref: argparse<3>-Footnote-1706204 +Node: audioop706247 +Ref: whatsnew/3 4 audioop706345 +Ref: 821706345 +Ref: audioop-Footnote-1706884 +Ref: audioop-Footnote-2706927 +Ref: audioop-Footnote-3706970 +Node: base64707013 +Ref: whatsnew/3 4 base64707114 +Ref: 823707114 +Ref: base64-Footnote-1707945 +Ref: base64-Footnote-2707988 +Node: collections<6>708031 +Ref: whatsnew/3 4 collections708133 +Ref: 828708133 +Ref: collections<6>-Footnote-1708463 +Node: colorsys708506 +Ref: whatsnew/3 4 colorsys708615 +Ref: 82a708615 +Ref: colorsys-Footnote-1708983 +Node: contextlib<4>709026 +Ref: whatsnew/3 4 contextlib709127 +Ref: 82b709127 +Ref: contextlib<4>-Footnote-1710312 +Ref: contextlib<4>-Footnote-2710355 +Ref: contextlib<4>-Footnote-3710398 +Node: dbm<5>710441 +Ref: whatsnew/3 4 dbm710540 +Ref: 82f710540 +Ref: dbm<5>-Footnote-1710887 +Node: dis<2>710930 +Ref: whatsnew/3 4 dis711026 +Ref: 831711026 +Ref: dis<2>-Footnote-1713625 +Ref: dis<2>-Footnote-2713668 +Ref: dis<2>-Footnote-3713711 +Node: doctest<2>713754 +Ref: whatsnew/3 4 doctest713852 +Ref: 83b713852 +Ref: doctest<2>-Footnote-1714620 +Ref: doctest<2>-Footnote-2714663 +Ref: doctest<2>-Footnote-3714706 +Node: email<3>714748 +Ref: whatsnew/3 4 email714847 +Ref: 83e714847 +Ref: whatsnew/3 4 whatsnew-email-contentmanager716095 +Ref: 7e4716095 +Ref: email<3>-Footnote-1717024 +Ref: email<3>-Footnote-2717067 +Ref: email<3>-Footnote-3717110 +Ref: email<3>-Footnote-4717153 +Node: filecmp717196 +Ref: whatsnew/3 4 filecmp717297 +Ref: 846717297 +Ref: filecmp-Footnote-1718043 +Ref: filecmp-Footnote-2718086 +Node: functools<4>718129 +Ref: whatsnew/3 4 functools718227 +Ref: 84a718227 +Ref: whatsnew/3 4 whatsnew-singledispatch718674 +Ref: 7e1718674 +Ref: functools<4>-Footnote-1719633 +Ref: functools<4>-Footnote-2719675 +Ref: functools<4>-Footnote-3719724 +Ref: functools<4>-Footnote-4719767 +Node: gc<3>719810 +Ref: whatsnew/3 4 gc719908 +Ref: 84d719908 +Ref: gc<3>-Footnote-1720174 +Node: glob<2>720217 +Ref: whatsnew/3 4 glob720313 +Ref: 84f720313 +Ref: glob<2>-Footnote-1720617 +Node: hashlib<2>720659 +Ref: whatsnew/3 4 hashlib720757 +Ref: 851720757 +Ref: hashlib<2>-Footnote-1721390 +Ref: hashlib<2>-Footnote-2721435 +Ref: hashlib<2>-Footnote-3721478 +Node: hmac<2>721521 +Ref: whatsnew/3 4 hmac721616 +Ref: 853721616 +Ref: hmac<2>-Footnote-1722642 +Ref: hmac<2>-Footnote-2722685 +Ref: hmac<2>-Footnote-3722728 +Ref: hmac<2>-Footnote-4722777 +Node: html722820 +Ref: whatsnew/3 4 html722912 +Ref: 859722912 +Ref: html-Footnote-1723669 +Ref: html-Footnote-2723711 +Ref: html-Footnote-3723754 +Node: http<2>723797 +Ref: whatsnew/3 4 http723901 +Ref: 85b723901 +Ref: http<2>-Footnote-1724542 +Ref: http<2>-Footnote-2724585 +Node: idlelib and IDLE<4>724628 +Ref: whatsnew/3 4 idlelib-and-idle724740 +Ref: 85e724740 +Node: importlib<6>725134 +Ref: whatsnew/3 4 importlib725249 +Ref: 85f725249 +Ref: importlib<6>-Footnote-1727250 +Ref: importlib<6>-Footnote-2727293 +Ref: importlib<6>-Footnote-3727336 +Ref: importlib<6>-Footnote-4727379 +Ref: importlib<6>-Footnote-5727422 +Ref: importlib<6>-Footnote-6727465 +Ref: importlib<6>-Footnote-7727508 +Node: inspect<4>727551 +Ref: whatsnew/3 4 inspect727659 +Ref: 867727659 +Ref: inspect<4>-Footnote-1729373 +Ref: inspect<4>-Footnote-2729416 +Ref: inspect<4>-Footnote-3729459 +Ref: inspect<4>-Footnote-4729502 +Ref: inspect<4>-Footnote-5729545 +Ref: inspect<4>-Footnote-6729588 +Node: ipaddress<3>729631 +Ref: whatsnew/3 4 ipaddress729737 +Ref: 86b729737 +Ref: ipaddress<3>-Footnote-1730263 +Node: logging<5>730306 +Ref: whatsnew/3 4 logging730409 +Ref: 86d730409 +Ref: logging<5>-Footnote-1731539 +Ref: logging<5>-Footnote-2731581 +Ref: logging<5>-Footnote-3731624 +Node: marshal731667 +Ref: whatsnew/3 4 marshal731765 +Ref: 873731765 +Ref: whatsnew/3 4 whatsnew-marshal-3731765 +Ref: 7d9731765 +Ref: marshal-Footnote-1732444 +Ref: marshal-Footnote-2732487 +Node: mmap<2>732530 +Ref: whatsnew/3 4 mmap732636 +Ref: 874732636 +Ref: mmap<2>-Footnote-1732799 +Node: multiprocessing<5>732841 +Ref: whatsnew/3 4 multiprocessing732951 +Ref: 875732951 +Ref: whatsnew/3 4 whatsnew-multiprocessing-no-fork733002 +Ref: 7e3733002 +Ref: multiprocessing<5>-Footnote-1734814 +Ref: multiprocessing<5>-Footnote-2734856 +Ref: multiprocessing<5>-Footnote-3734899 +Ref: multiprocessing<5>-Footnote-4734941 +Node: operator<2>734984 +Ref: whatsnew/3 4 operator735092 +Ref: 87c735092 +Ref: operator<2>-Footnote-1735643 +Ref: operator<2>-Footnote-2735692 +Ref: operator<2>-Footnote-3735735 +Node: os<6>735778 +Ref: whatsnew/3 4 os735874 +Ref: 87e735874 +Ref: os<6>-Footnote-1737319 +Ref: os<6>-Footnote-2737362 +Ref: os<6>-Footnote-3737405 +Ref: os<6>-Footnote-4737447 +Node: pdb<3>737490 +Ref: whatsnew/3 4 pdb737584 +Ref: 885737584 +Ref: pdb<3>-Footnote-1738578 +Ref: pdb<3>-Footnote-2738621 +Node: pickle<4>738664 +Ref: whatsnew/3 4 pickle738764 +Ref: 888738764 +Ref: whatsnew/3 4 whatsnew-protocol-4738764 +Ref: 7e2738764 +Ref: pickle<4>-Footnote-1739359 +Node: plistlib<2>739408 +Ref: whatsnew/3 4 plistlib739511 +Ref: 88a739511 +Ref: plistlib<2>-Footnote-1740045 +Node: poplib<2>740088 +Ref: whatsnew/3 4 poplib740191 +Ref: 891740191 +Ref: poplib<2>-Footnote-1740579 +Node: pprint<2>740621 +Ref: whatsnew/3 4 pprint740716 +Ref: 894740716 +Ref: pprint<2>-Footnote-1741334 +Ref: pprint<2>-Footnote-2741377 +Node: pty741420 +Ref: whatsnew/3 4 pty741514 +Ref: 897741514 +Node: pydoc<3>741705 +Ref: whatsnew/3 4 pydoc741795 +Ref: 899741795 +Ref: pydoc<3>-Footnote-1742850 +Ref: pydoc<3>-Footnote-2742893 +Node: re<5>742936 +Ref: whatsnew/3 4 re743031 +Ref: 89a743031 +Ref: re<5>-Footnote-1743772 +Ref: re<5>-Footnote-2743815 +Ref: re<5>-Footnote-3743858 +Node: resource743901 +Ref: whatsnew/3 4 resource743994 +Ref: 89e743994 +Ref: resource-Footnote-1744845 +Ref: resource-Footnote-2744888 +Ref: resource-Footnote-3744931 +Node: select744974 +Ref: whatsnew/3 4 select745068 +Ref: 8a8745068 +Ref: select-Footnote-1745575 +Ref: select-Footnote-2745618 +Node: shelve745661 +Ref: whatsnew/3 4 shelve745756 +Ref: 8af745756 +Ref: shelve-Footnote-1746026 +Node: shutil<3>746069 +Ref: whatsnew/3 4 shutil746166 +Ref: 8b1746166 +Ref: shutil<3>-Footnote-1746537 +Node: smtpd<2>746582 +Ref: whatsnew/3 4 smtpd746683 +Ref: 8b4746683 +Ref: smtpd<2>-Footnote-1747059 +Node: smtplib<2>747102 +Ref: whatsnew/3 4 smtplib747203 +Ref: 8b6747203 +Ref: smtplib<2>-Footnote-1747563 +Node: socket<6>747605 +Ref: whatsnew/3 4 socket747708 +Ref: 8b8747708 +Ref: socket<6>-Footnote-1748489 +Ref: socket<6>-Footnote-2748532 +Node: sqlite3<4>748574 +Ref: whatsnew/3 4 sqlite3748673 +Ref: 8bd748673 +Ref: sqlite3<4>-Footnote-1748962 +Ref: sqlite3<4>-Footnote-2749002 +Node: ssl<7>749045 +Ref: whatsnew/3 4 ssl749139 +Ref: 8be749139 +Ref: whatsnew/3 4 whatsnew-tls-11-12749166 +Ref: 7e7749166 +Ref: whatsnew/3 4 whatsnew34-sslcontext749436 +Ref: 7ea749436 +Ref: whatsnew/3 4 whatsnew34-win-cert-store752110 +Ref: 7e8752110 +Ref: whatsnew/3 4 whatsnew34-sni752364 +Ref: 7e9752364 +Ref: ssl<7>-Footnote-1752813 +Ref: ssl<7>-Footnote-2752856 +Ref: ssl<7>-Footnote-3752899 +Ref: ssl<7>-Footnote-4752942 +Ref: ssl<7>-Footnote-5752985 +Ref: ssl<7>-Footnote-6753028 +Ref: ssl<7>-Footnote-7753070 +Ref: ssl<7>-Footnote-8753113 +Ref: ssl<7>-Footnote-9753156 +Ref: ssl<7>-Footnote-10753198 +Node: stat753242 +Ref: whatsnew/3 4 stat753335 +Ref: 8d2753335 +Ref: stat-Footnote-1753806 +Ref: stat-Footnote-2753849 +Node: struct<2>753892 +Ref: whatsnew/3 4 struct753992 +Ref: 8d7753992 +Ref: struct<2>-Footnote-1754322 +Node: subprocess<4>754365 +Ref: whatsnew/3 4 subprocess754469 +Ref: 8da754469 +Ref: subprocess<4>-Footnote-1754913 +Ref: subprocess<4>-Footnote-2754956 +Node: sunau<2>754999 +Ref: whatsnew/3 4 sunau755100 +Ref: 8dd755100 +Ref: sunau<2>-Footnote-1755905 +Ref: sunau<2>-Footnote-2755948 +Ref: sunau<2>-Footnote-3755991 +Ref: sunau<2>-Footnote-4756034 +Node: sys<6>756076 +Ref: whatsnew/3 4 sys756174 +Ref: 8e1756174 +Ref: sys<6>-Footnote-1757452 +Ref: sys<6>-Footnote-2757495 +Node: tarfile<3>757537 +Ref: whatsnew/3 4 tarfile757635 +Ref: 8e7757635 +Ref: tarfile<3>-Footnote-1757952 +Node: textwrap757995 +Ref: whatsnew/3 4 textwrap758099 +Ref: 8e9758099 +Ref: textwrap-Footnote-1758798 +Ref: textwrap-Footnote-2758841 +Node: threading<4>758884 +Ref: whatsnew/3 4 threading758990 +Ref: 8ee758990 +Ref: threading<4>-Footnote-1759333 +Node: traceback<3>759376 +Ref: whatsnew/3 4 traceback759482 +Ref: 8f0759482 +Ref: traceback<3>-Footnote-1759795 +Node: types<3>759840 +Ref: whatsnew/3 4 types759943 +Ref: 8f2759943 +Ref: types<3>-Footnote-1760447 +Node: urllib<2>760490 +Ref: whatsnew/3 4 urllib760592 +Ref: 8f4760592 +Ref: urllib<2>-Footnote-1761906 +Ref: urllib<2>-Footnote-2761949 +Ref: urllib<2>-Footnote-3761992 +Ref: urllib<2>-Footnote-4762035 +Ref: urllib<2>-Footnote-5762078 +Ref: urllib<2>-Footnote-6762121 +Node: unittest<4>762164 +Ref: whatsnew/3 4 unittest762265 +Ref: 8fe762265 +Ref: unittest<4>-Footnote-1765419 +Ref: unittest<4>-Footnote-2765462 +Ref: unittest<4>-Footnote-3765505 +Ref: unittest<4>-Footnote-4765548 +Ref: unittest<4>-Footnote-5765591 +Ref: unittest<4>-Footnote-6765634 +Ref: unittest<4>-Footnote-7765677 +Ref: unittest<4>-Footnote-8765720 +Ref: unittest<4>-Footnote-9765763 +Node: venv<4>765806 +Ref: whatsnew/3 4 venv765905 +Ref: 907765905 +Ref: venv<4>-Footnote-1766461 +Ref: venv<4>-Footnote-2766504 +Ref: venv<4>-Footnote-3766547 +Node: wave<2>766596 +Ref: whatsnew/3 4 wave766694 +Ref: 90a766694 +Ref: wave<2>-Footnote-1767329 +Ref: wave<2>-Footnote-2767372 +Ref: wave<2>-Footnote-3767415 +Ref: wave<2>-Footnote-4767457 +Node: weakref<2>767499 +Ref: whatsnew/3 4 weakref767602 +Ref: 90e767602 +Ref: weakref<2>-Footnote-1768221 +Ref: weakref<2>-Footnote-2768264 +Ref: weakref<2>-Footnote-3768307 +Node: xml etree<2>768350 +Ref: whatsnew/3 4 xml-etree768456 +Ref: 912768456 +Ref: xml etree<2>-Footnote-1769193 +Ref: xml etree<2>-Footnote-2769236 +Node: zipfile<4>769279 +Ref: whatsnew/3 4 zipfile769366 +Ref: 919769366 +Ref: zipfile<4>-Footnote-1769895 +Ref: zipfile<4>-Footnote-2769938 +Node: CPython Implementation Changes769981 +Ref: whatsnew/3 4 cpython-implementation-changes770124 +Ref: 91c770124 +Node: PEP 445 Customization of CPython Memory Allocators770654 +Ref: whatsnew/3 4 pep-445-customization-of-cpython-memory-allocators770812 +Ref: 91d770812 +Ref: whatsnew/3 4 whatsnew-pep-445770812 +Ref: 7ed770812 +Ref: PEP 445 Customization of CPython Memory Allocators-Footnote-1771207 +Ref: PEP 445 Customization of CPython Memory Allocators-Footnote-2771256 +Node: PEP 442 Safe Object Finalization771305 +Ref: whatsnew/3 4 pep-442-safe-object-finalization771521 +Ref: 91e771521 +Ref: whatsnew/3 4 whatsnew-pep-442771521 +Ref: 7ec771521 +Ref: PEP 442 Safe Object Finalization-Footnote-1772388 +Ref: PEP 442 Safe Object Finalization-Footnote-2772437 +Node: PEP 456 Secure and Interchangeable Hash Algorithm772486 +Ref: whatsnew/3 4 pep-456-secure-and-interchangeable-hash-algorithm772675 +Ref: 920772675 +Ref: whatsnew/3 4 whatsnew-pep-456772675 +Ref: 7e5772675 +Ref: PEP 456 Secure and Interchangeable Hash Algorithm-Footnote-1773621 +Ref: PEP 456 Secure and Interchangeable Hash Algorithm-Footnote-2773670 +Node: PEP 436 Argument Clinic773713 +Ref: whatsnew/3 4 pep-436-argument-clinic773899 +Ref: 922773899 +Ref: whatsnew/3 4 whatsnew-pep-436773899 +Ref: 7ee773899 +Ref: PEP 436 Argument Clinic-Footnote-1775005 +Ref: PEP 436 Argument Clinic-Footnote-2775054 +Node: Other Build and C API Changes775103 +Ref: whatsnew/3 4 other-build-and-c-api-changes775261 +Ref: 923775261 +Ref: Other Build and C API Changes-Footnote-1777345 +Ref: Other Build and C API Changes-Footnote-2777388 +Ref: Other Build and C API Changes-Footnote-3777431 +Ref: Other Build and C API Changes-Footnote-4777476 +Ref: Other Build and C API Changes-Footnote-5777518 +Ref: Other Build and C API Changes-Footnote-6777561 +Ref: Other Build and C API Changes-Footnote-7777634 +Ref: Other Build and C API Changes-Footnote-8777698 +Ref: Other Build and C API Changes-Footnote-9777741 +Node: Other Improvements<2>777784 +Ref: whatsnew/3 4 other-improvements777944 +Ref: 929777944 +Ref: whatsnew/3 4 other-improvements-3-4777944 +Ref: 92a777944 +Ref: whatsnew/3 4 whatsnew-isolated-mode777999 +Ref: 7d6777999 +Ref: Other Improvements<2>-Footnote-1782124 +Ref: Other Improvements<2>-Footnote-2782167 +Ref: Other Improvements<2>-Footnote-3782209 +Ref: Other Improvements<2>-Footnote-4782252 +Ref: Other Improvements<2>-Footnote-5782295 +Ref: Other Improvements<2>-Footnote-6782338 +Ref: Other Improvements<2>-Footnote-7782381 +Ref: Other Improvements<2>-Footnote-8782476 +Ref: Other Improvements<2>-Footnote-9782529 +Ref: Other Improvements<2>-Footnote-10782572 +Ref: Other Improvements<2>-Footnote-11782616 +Ref: Other Improvements<2>-Footnote-12782660 +Ref: Other Improvements<2>-Footnote-13782710 +Ref: Other Improvements<2>-Footnote-14782754 +Ref: Other Improvements<2>-Footnote-15782798 +Ref: Other Improvements<2>-Footnote-16782842 +Ref: Other Improvements<2>-Footnote-17782886 +Ref: Other Improvements<2>-Footnote-18782936 +Ref: Other Improvements<2>-Footnote-19782980 +Ref: Other Improvements<2>-Footnote-20783024 +Ref: Other Improvements<2>-Footnote-21783068 +Node: Significant Optimizations783112 +Ref: whatsnew/3 4 significant-optimizations783234 +Ref: 930783234 +Ref: Significant Optimizations-Footnote-1786037 +Ref: Significant Optimizations-Footnote-2786080 +Ref: Significant Optimizations-Footnote-3786123 +Ref: Significant Optimizations-Footnote-4786166 +Ref: Significant Optimizations-Footnote-5786209 +Ref: Significant Optimizations-Footnote-6786252 +Ref: Significant Optimizations-Footnote-7786295 +Ref: Significant Optimizations-Footnote-8786337 +Ref: Significant Optimizations-Footnote-9786380 +Ref: Significant Optimizations-Footnote-10786423 +Ref: Significant Optimizations-Footnote-11786467 +Ref: Significant Optimizations-Footnote-12786511 +Ref: Significant Optimizations-Footnote-13786555 +Node: Deprecated<4>786599 +Ref: whatsnew/3 4 deprecated786733 +Ref: 935786733 +Ref: whatsnew/3 4 deprecated-3-4786733 +Ref: 807786733 +Node: Deprecations in the Python API787173 +Ref: whatsnew/3 4 deprecations-in-the-python-api787281 +Ref: 936787281 +Node: Deprecated Features791043 +Ref: whatsnew/3 4 deprecated-features791151 +Ref: 949791151 +Ref: Deprecated Features-Footnote-1791534 +Ref: Deprecated Features-Footnote-2791577 +Node: Removed<3>791620 +Ref: whatsnew/3 4 removed791745 +Ref: 94b791745 +Node: Operating Systems No Longer Supported791901 +Ref: whatsnew/3 4 operating-systems-no-longer-supported792021 +Ref: 94c792021 +Ref: Operating Systems No Longer Supported-Footnote-1792434 +Ref: Operating Systems No Longer Supported-Footnote-2792477 +Ref: Operating Systems No Longer Supported-Footnote-3792520 +Node: API and Feature Removals<5>792563 +Ref: whatsnew/3 4 api-and-feature-removals792705 +Ref: 94d792705 +Ref: API and Feature Removals<5>-Footnote-1795159 +Ref: API and Feature Removals<5>-Footnote-2795195 +Ref: API and Feature Removals<5>-Footnote-3795238 +Ref: API and Feature Removals<5>-Footnote-4795281 +Ref: API and Feature Removals<5>-Footnote-5795324 +Ref: API and Feature Removals<5>-Footnote-6795367 +Ref: API and Feature Removals<5>-Footnote-7795409 +Node: Code Cleanups795452 +Ref: whatsnew/3 4 code-cleanups795548 +Ref: 950795548 +Ref: Code Cleanups-Footnote-1796211 +Node: Porting to Python 3 4796254 +Ref: whatsnew/3 4 porting-to-python-3-4796382 +Ref: 951796382 +Node: Changes in ‘python’ Command Behavior<2>796753 +Ref: whatsnew/3 4 changes-in-python-command-behavior796891 +Ref: 952796891 +Ref: Changes in ‘python’ Command Behavior<2>-Footnote-1798002 +Node: Changes in the Python API<5>798045 +Ref: whatsnew/3 4 changes-in-the-python-api798215 +Ref: 954798215 +Ref: Changes in the Python API<5>-Footnote-1811597 +Ref: Changes in the Python API<5>-Footnote-2811640 +Ref: Changes in the Python API<5>-Footnote-3811683 +Ref: Changes in the Python API<5>-Footnote-4811726 +Ref: Changes in the Python API<5>-Footnote-5811769 +Ref: Changes in the Python API<5>-Footnote-6811812 +Ref: Changes in the Python API<5>-Footnote-7811855 +Ref: Changes in the Python API<5>-Footnote-8811898 +Ref: Changes in the Python API<5>-Footnote-9811940 +Ref: Changes in the Python API<5>-Footnote-10811983 +Ref: Changes in the Python API<5>-Footnote-11812027 +Ref: Changes in the Python API<5>-Footnote-12812071 +Ref: Changes in the Python API<5>-Footnote-13812115 +Ref: Changes in the Python API<5>-Footnote-14812159 +Ref: Changes in the Python API<5>-Footnote-15812203 +Ref: Changes in the Python API<5>-Footnote-16812247 +Ref: Changes in the Python API<5>-Footnote-17812291 +Ref: Changes in the Python API<5>-Footnote-18812335 +Ref: Changes in the Python API<5>-Footnote-19812379 +Ref: Changes in the Python API<5>-Footnote-20812423 +Ref: Changes in the Python API<5>-Footnote-21812467 +Ref: Changes in the Python API<5>-Footnote-22812511 +Ref: Changes in the Python API<5>-Footnote-23812555 +Ref: Changes in the Python API<5>-Footnote-24812598 +Node: Changes in the C API<5>812642 +Ref: whatsnew/3 4 changes-in-the-c-api812760 +Ref: 966812760 +Ref: Changes in the C API<5>-Footnote-1814520 +Ref: Changes in the C API<5>-Footnote-2814563 +Node: Changed in 3 4 3814606 +Ref: whatsnew/3 4 changed-in-3-4-3814715 +Ref: 972814715 +Node: PEP 476 Enabling certificate verification by default for stdlib http clients814931 +Ref: whatsnew/3 4 pep-476815060 +Ref: 973815060 +Ref: whatsnew/3 4 pep-476-enabling-certificate-verification-by-default-for-stdlib-http-clients815060 +Ref: 974815060 +Node: What’s New In Python 3 3816059 +Ref: whatsnew/3 3 doc816214 +Ref: 975816214 +Ref: whatsnew/3 3 what-s-new-in-python-3-3816214 +Ref: 976816214 +Ref: What’s New In Python 3 3-Footnote-1818154 +Ref: What’s New In Python 3 3-Footnote-2818214 +Node: Summary – Release highlights<4>818263 +Ref: whatsnew/3 3 summary-release-highlights818396 +Ref: 977818396 +Node: PEP 405 Virtual Environments819725 +Ref: whatsnew/3 3 pep-405819902 +Ref: 979819902 +Ref: whatsnew/3 3 pep-405-virtual-environments819902 +Ref: 981819902 +Ref: PEP 405 Virtual Environments-Footnote-1820890 +Node: PEP 420 Implicit Namespace Packages820939 +Ref: whatsnew/3 3 pep-420-implicit-namespace-packages821155 +Ref: 982821155 +Ref: PEP 420 Implicit Namespace Packages-Footnote-1821666 +Ref: PEP 420 Implicit Namespace Packages-Footnote-2821715 +Node: PEP 3118 New memoryview implementation and buffer protocol documentation821764 +Ref: whatsnew/3 3 pep-3118-new-memoryview-implementation-and-buffer-protocol-documentation821990 +Ref: 983821990 +Ref: whatsnew/3 3 pep-3118-update821990 +Ref: 984821990 +Ref: PEP 3118 New memoryview implementation and buffer protocol documentation-Footnote-1823120 +Node: Features823169 +Ref: whatsnew/3 3 features823306 +Ref: 985823306 +Ref: Features-Footnote-1824106 +Node: API changes824149 +Ref: whatsnew/3 3 api-changes824286 +Ref: 986824286 +Ref: API changes-Footnote-1825301 +Ref: API changes-Footnote-2825344 +Node: PEP 393 Flexible String Representation825393 +Ref: whatsnew/3 3 pep-393825619 +Ref: 97c825619 +Ref: whatsnew/3 3 pep-393-flexible-string-representation825619 +Ref: 989825619 +Ref: PEP 393 Flexible String Representation-Footnote-1826678 +Node: Functionality826727 +Ref: whatsnew/3 3 functionality826854 +Ref: 98a826854 +Ref: Functionality-Footnote-1828301 +Ref: Functionality-Footnote-2828350 +Node: Performance and resource usage828393 +Ref: whatsnew/3 3 performance-and-resource-usage828520 +Ref: 98c828520 +Ref: Performance and resource usage-Footnote-1829751 +Node: PEP 397 Python Launcher for Windows829800 +Ref: whatsnew/3 3 pep-397830006 +Ref: 98d830006 +Ref: whatsnew/3 3 pep-397-python-launcher-for-windows830006 +Ref: 98e830006 +Ref: PEP 397 Python Launcher for Windows-Footnote-1831548 +Ref: PEP 397 Python Launcher for Windows-Footnote-2831590 +Node: PEP 3151 Reworking the OS and IO exception hierarchy831639 +Ref: whatsnew/3 3 pep-3151831854 +Ref: 97a831854 +Ref: whatsnew/3 3 pep-3151-reworking-the-os-and-io-exception-hierarchy831854 +Ref: 991831854 +Ref: PEP 3151 Reworking the OS and IO exception hierarchy-Footnote-1834423 +Node: PEP 380 Syntax for Delegating to a Subgenerator834472 +Ref: whatsnew/3 3 pep-380834689 +Ref: 978834689 +Ref: whatsnew/3 3 pep-380-syntax-for-delegating-to-a-subgenerator834689 +Ref: 9a1834689 +Ref: PEP 380 Syntax for Delegating to a Subgenerator-Footnote-1837132 +Node: PEP 409 Suppressing exception context837181 +Ref: whatsnew/3 3 pep-409-suppressing-exception-context837379 +Ref: 9a3837379 +Ref: PEP 409 Suppressing exception context-Footnote-1839467 +Node: PEP 414 Explicit Unicode literals839516 +Ref: whatsnew/3 3 pep-414-explicit-unicode-literals839716 +Ref: 9a4839716 +Ref: PEP 414 Explicit Unicode literals-Footnote-1840429 +Node: PEP 3155 Qualified name for classes and functions840478 +Ref: whatsnew/3 3 pep-3155-qualified-name-for-classes-and-functions840671 +Ref: 9a5840671 +Ref: PEP 3155 Qualified name for classes and functions-Footnote-1842263 +Node: PEP 412 Key-Sharing Dictionary842312 +Ref: whatsnew/3 3 pep-412842505 +Ref: 97d842505 +Ref: whatsnew/3 3 pep-412-key-sharing-dictionary842505 +Ref: 9a6842505 +Ref: PEP 412 Key-Sharing Dictionary-Footnote-1843024 +Node: PEP 362 Function Signature Object843073 +Ref: whatsnew/3 3 pep-362-function-signature-object843250 +Ref: 9a7843250 +Ref: PEP 362 Function Signature Object-Footnote-1844144 +Node: PEP 421 Adding sys implementation844193 +Ref: whatsnew/3 3 pep-421-adding-sys-implementation844387 +Ref: 9a9844387 +Ref: PEP 421 Adding sys implementation-Footnote-1845601 +Node: SimpleNamespace845650 +Ref: whatsnew/3 3 simplenamespace845735 +Ref: 9ab845735 +Ref: SimpleNamespace-Footnote-1846353 +Node: Using importlib as the Implementation of Import846402 +Ref: whatsnew/3 3 importlib846588 +Ref: 97b846588 +Ref: whatsnew/3 3 using-importlib-as-the-implementation-of-import846588 +Ref: 9ad846588 +Ref: Using importlib as the Implementation of Import-Footnote-1847869 +Ref: Using importlib as the Implementation of Import-Footnote-2847911 +Ref: Using importlib as the Implementation of Import-Footnote-3847954 +Ref: Using importlib as the Implementation of Import-Footnote-4847997 +Ref: Using importlib as the Implementation of Import-Footnote-5848040 +Node: New APIs848089 +Ref: whatsnew/3 3 new-apis848205 +Ref: 9b0848205 +Node: Visible Changes850047 +Ref: whatsnew/3 3 visible-changes850163 +Ref: 9ba850163 +Ref: Visible Changes-Footnote-1852017 +Ref: Visible Changes-Footnote-2852066 +Node: Other Language Changes<6>852115 +Ref: whatsnew/3 3 other-language-changes852295 +Ref: 9bc852295 +Ref: Other Language Changes<6>-Footnote-1854255 +Ref: Other Language Changes<6>-Footnote-2854298 +Ref: Other Language Changes<6>-Footnote-3854341 +Ref: Other Language Changes<6>-Footnote-4854384 +Ref: Other Language Changes<6>-Footnote-5854427 +Ref: Other Language Changes<6>-Footnote-6854470 +Ref: Other Language Changes<6>-Footnote-7854513 +Ref: Other Language Changes<6>-Footnote-8854556 +Node: A Finer-Grained Import Lock854599 +Ref: whatsnew/3 3 a-finer-grained-import-lock854759 +Ref: 9c0854759 +Ref: A Finer-Grained Import Lock-Footnote-1855502 +Node: Builtin functions and types855544 +Ref: whatsnew/3 3 builtin-functions-and-types855693 +Ref: 9c2855693 +Ref: Builtin functions and types-Footnote-1856926 +Node: New Modules<6>856968 +Ref: whatsnew/3 3 new-modules857109 +Ref: 9c6857109 +Node: faulthandler<3>857235 +Ref: whatsnew/3 3 faulthandler857322 +Ref: 9c7857322 +Node: ipaddress<4>858247 +Ref: whatsnew/3 3 ipaddress858350 +Ref: 9c9858350 +Ref: ipaddress<4>-Footnote-1858694 +Node: lzma<2>858743 +Ref: whatsnew/3 3 lzma858822 +Ref: 9ca858822 +Ref: lzma<2>-Footnote-1859134 +Node: Improved Modules<6>859176 +Ref: whatsnew/3 3 improved-modules859306 +Ref: 9cb859306 +Node: abc<2>860573 +Ref: whatsnew/3 3 abc860652 +Ref: 9cc860652 +Ref: abc<2>-Footnote-1861684 +Ref: abc<2>-Footnote-2861727 +Node: array<2>861770 +Ref: whatsnew/3 3 array861867 +Ref: 9d2861867 +Ref: array<2>-Footnote-1862105 +Node: base64<2>862150 +Ref: whatsnew/3 3 base64862252 +Ref: 9d3862252 +Ref: base64<2>-Footnote-1862550 +Node: binascii<3>862593 +Ref: whatsnew/3 3 binascii862693 +Ref: 9d4862693 +Ref: binascii<3>-Footnote-1862947 +Node: bz2<2>862990 +Ref: whatsnew/3 3 bz2863087 +Ref: 9d5863087 +Ref: bz2<2>-Footnote-1864058 +Ref: bz2<2>-Footnote-2864100 +Node: codecs864142 +Ref: whatsnew/3 3 codecs864242 +Ref: 9d8864242 +Ref: codecs-Footnote-1865562 +Ref: codecs-Footnote-2865605 +Ref: codecs-Footnote-3865648 +Node: collections<7>865691 +Ref: whatsnew/3 3 collections865798 +Ref: 9d9865798 +Ref: collections<7>-Footnote-1866563 +Ref: collections<7>-Footnote-2866606 +Ref: collections<7>-Footnote-3866649 +Ref: collections<7>-Footnote-4866692 +Node: contextlib<5>866735 +Ref: whatsnew/3 3 contextlib866844 +Ref: 9db866844 +Ref: contextlib<5>-Footnote-1867438 +Node: crypt<2>867481 +Ref: whatsnew/3 3 crypt867585 +Ref: 9dc867585 +Ref: crypt<2>-Footnote-1867798 +Node: curses<3>867841 +Ref: whatsnew/3 3 curses867943 +Ref: 9dd867943 +Ref: curses<3>-Footnote-1868801 +Node: datetime<4>868843 +Ref: whatsnew/3 3 datetime868947 +Ref: 9e1868947 +Ref: datetime<4>-Footnote-1869665 +Node: decimal<3>869708 +Ref: whatsnew/3 3 decimal869811 +Ref: 9e3869811 +Ref: whatsnew/3 3 new-decimal869811 +Ref: 97e869811 +Ref: decimal<3>-Footnote-1871514 +Node: Features<2>871556 +Ref: whatsnew/3 3 id1871637 +Ref: 9e4871637 +Node: API changes<2>872009 +Ref: whatsnew/3 3 id2872090 +Ref: 9e7872090 +Node: email<4>874868 +Ref: whatsnew/3 3 email874966 +Ref: 9f2874966 +Ref: whatsnew/3 3 new-email874966 +Ref: 97f874966 +Node: Policy Framework875095 +Ref: whatsnew/3 3 policy-framework875203 +Ref: 9f3875203 +Node: Provisional Policy with New Header API878211 +Ref: whatsnew/3 3 provisional-policy-with-new-header-api878345 +Ref: 9f7878345 +Node: Other API Changes882981 +Ref: whatsnew/3 3 other-api-changes883090 +Ref: 9fb883090 +Node: ftplib883981 +Ref: whatsnew/3 3 ftplib884081 +Ref: a02884081 +Ref: ftplib-Footnote-1884988 +Ref: ftplib-Footnote-2885030 +Ref: ftplib-Footnote-3885073 +Node: functools<5>885116 +Ref: whatsnew/3 3 functools885213 +Ref: a08885213 +Ref: functools<5>-Footnote-1885559 +Node: gc<4>885602 +Ref: whatsnew/3 3 gc885700 +Ref: a09885700 +Node: hmac<3>885872 +Ref: whatsnew/3 3 hmac885965 +Ref: a0b885965 +Ref: hmac<3>-Footnote-1886229 +Node: http<3>886272 +Ref: whatsnew/3 3 http886367 +Ref: a0d886367 +Ref: http<3>-Footnote-1887045 +Ref: http<3>-Footnote-2887087 +Ref: http<3>-Footnote-3887130 +Node: html<2>887173 +Ref: whatsnew/3 3 html887271 +Ref: a14887271 +Ref: html<2>-Footnote-1888269 +Ref: html<2>-Footnote-2888312 +Ref: html<2>-Footnote-3888355 +Ref: html<2>-Footnote-4888398 +Ref: html<2>-Footnote-5888441 +Ref: html<2>-Footnote-6888484 +Ref: html<2>-Footnote-7888529 +Ref: html<2>-Footnote-8888573 +Ref: html<2>-Footnote-9888616 +Ref: html<2>-Footnote-10888659 +Ref: html<2>-Footnote-11888705 +Ref: html<2>-Footnote-12888750 +Ref: html<2>-Footnote-13888794 +Ref: html<2>-Footnote-14888838 +Ref: html<2>-Footnote-15888881 +Ref: html<2>-Footnote-16888925 +Node: imaplib<2>888969 +Ref: whatsnew/3 3 imaplib889070 +Ref: a16889070 +Ref: imaplib<2>-Footnote-1889310 +Node: inspect<5>889352 +Ref: whatsnew/3 3 inspect889451 +Ref: a18889451 +Ref: inspect<5>-Footnote-1890161 +Ref: inspect<5>-Footnote-2890204 +Node: io<4>890247 +Ref: whatsnew/3 3 io890348 +Ref: a1b890348 +Ref: io<4>-Footnote-1890995 +Node: itertools<3>891038 +Ref: whatsnew/3 3 itertools891139 +Ref: a1c891139 +Node: logging<6>891295 +Ref: whatsnew/3 3 logging891398 +Ref: a1d891398 +Node: math<5>891849 +Ref: whatsnew/3 3 math891947 +Ref: a1f891947 +Ref: math<5>-Footnote-1892170 +Node: mmap<3>892213 +Ref: whatsnew/3 3 mmap892319 +Ref: a21892319 +Ref: mmap<3>-Footnote-1892654 +Node: multiprocessing<6>892697 +Ref: whatsnew/3 3 multiprocessing892803 +Ref: a23892803 +Ref: multiprocessing<6>-Footnote-1893953 +Ref: multiprocessing<6>-Footnote-2893996 +Ref: multiprocessing<6>-Footnote-3894038 +Ref: multiprocessing<6>-Footnote-4894080 +Node: nntplib894123 +Ref: whatsnew/3 3 nntplib894227 +Ref: a2b894227 +Ref: nntplib-Footnote-1894790 +Node: os<7>894832 +Ref: whatsnew/3 3 os894924 +Ref: a2d894924 +Ref: os<7>-Footnote-1902302 +Ref: os<7>-Footnote-2902345 +Ref: os<7>-Footnote-3902387 +Ref: os<7>-Footnote-4902430 +Ref: os<7>-Footnote-5902473 +Ref: os<7>-Footnote-6902516 +Ref: os<7>-Footnote-7902558 +Ref: os<7>-Footnote-8902601 +Ref: os<7>-Footnote-9902644 +Ref: os<7>-Footnote-10902687 +Ref: os<7>-Footnote-11902730 +Node: pdb<4>902774 +Ref: whatsnew/3 3 pdb902868 +Ref: a67902868 +Ref: pdb<4>-Footnote-1903149 +Node: pickle<5>903192 +Ref: whatsnew/3 3 pickle903289 +Ref: a68903289 +Ref: pickle<5>-Footnote-1903557 +Node: pydoc<4>903600 +Ref: whatsnew/3 3 pydoc903696 +Ref: a6a903696 +Node: re<6>903893 +Ref: whatsnew/3 3 re903985 +Ref: a6b903985 +Ref: re<6>-Footnote-1904179 +Node: sched904221 +Ref: whatsnew/3 3 sched904314 +Ref: a6c904314 +Ref: sched-Footnote-1905446 +Ref: sched-Footnote-2905489 +Ref: sched-Footnote-3905531 +Ref: sched-Footnote-4905574 +Ref: sched-Footnote-5905617 +Node: select<2>905660 +Ref: whatsnew/3 3 select905756 +Ref: a71905756 +Ref: select<2>-Footnote-1906019 +Node: shlex<3>906061 +Ref: whatsnew/3 3 shlex906161 +Ref: a72906161 +Node: shutil<4>906472 +Ref: whatsnew/3 3 shutil906572 +Ref: a74906572 +Ref: shutil<4>-Footnote-1908442 +Ref: shutil<4>-Footnote-2908485 +Ref: shutil<4>-Footnote-3908528 +Ref: shutil<4>-Footnote-4908571 +Ref: shutil<4>-Footnote-5908614 +Ref: shutil<4>-Footnote-6908657 +Ref: shutil<4>-Footnote-7908700 +Ref: shutil<4>-Footnote-8908742 +Node: signal<3>908784 +Ref: whatsnew/3 3 signal908884 +Ref: a78908884 +Ref: signal<3>-Footnote-1909949 +Node: smtpd<3>909991 +Ref: whatsnew/3 3 smtpd910092 +Ref: a7f910092 +Ref: smtpd<3>-Footnote-1910578 +Ref: smtpd<3>-Footnote-2910627 +Ref: smtpd<3>-Footnote-3910676 +Node: smtplib<3>910718 +Ref: whatsnew/3 3 smtplib910819 +Ref: a80910819 +Ref: smtplib<3>-Footnote-1911570 +Ref: smtplib<3>-Footnote-2911613 +Ref: smtplib<3>-Footnote-3911656 +Node: socket<7>911698 +Ref: whatsnew/3 3 socket911806 +Ref: a85911806 +Ref: socket<7>-Footnote-1913048 +Ref: socket<7>-Footnote-2913090 +Ref: socket<7>-Footnote-3913133 +Ref: socket<7>-Footnote-4913176 +Node: socketserver<3>913219 +Ref: whatsnew/3 3 socketserver913327 +Ref: a88913327 +Ref: socketserver<3>-Footnote-1913700 +Node: sqlite3<5>913743 +Ref: whatsnew/3 3 sqlite3913848 +Ref: a8c913848 +Ref: sqlite3<5>-Footnote-1914123 +Node: ssl<8>914166 +Ref: whatsnew/3 3 ssl914263 +Ref: a8e914263 +Ref: ssl<8>-Footnote-1916480 +Ref: ssl<8>-Footnote-2916523 +Ref: ssl<8>-Footnote-3916566 +Ref: ssl<8>-Footnote-4916609 +Ref: ssl<8>-Footnote-5916652 +Ref: ssl<8>-Footnote-6916695 +Ref: ssl<8>-Footnote-7916738 +Ref: ssl<8>-Footnote-8916781 +Ref: ssl<8>-Footnote-9916824 +Ref: ssl<8>-Footnote-10916867 +Ref: ssl<8>-Footnote-11916911 +Node: stat<2>916955 +Ref: whatsnew/3 3 stat917051 +Ref: a9c917051 +Ref: stat<2>-Footnote-1917346 +Node: struct<3>917389 +Ref: whatsnew/3 3 struct917492 +Ref: a9e917492 +Ref: struct<3>-Footnote-1917739 +Node: subprocess<5>917781 +Ref: whatsnew/3 3 subprocess917883 +Ref: a9f917883 +Ref: subprocess<5>-Footnote-1918218 +Ref: subprocess<5>-Footnote-2918260 +Node: sys<7>918302 +Ref: whatsnew/3 3 sys918405 +Ref: aa1918405 +Ref: sys<7>-Footnote-1918623 +Node: tarfile<4>918666 +Ref: whatsnew/3 3 tarfile918764 +Ref: aa4918764 +Ref: tarfile<4>-Footnote-1918972 +Node: tempfile919014 +Ref: whatsnew/3 3 tempfile919117 +Ref: aa5919117 +Ref: tempfile-Footnote-1919344 +Node: textwrap<2>919386 +Ref: whatsnew/3 3 textwrap919491 +Ref: aa7919491 +Ref: textwrap<2>-Footnote-1919736 +Node: threading<5>919779 +Ref: whatsnew/3 3 threading919883 +Ref: aa9919883 +Ref: threading<5>-Footnote-1920814 +Ref: threading<5>-Footnote-2920857 +Node: time<5>920899 +Ref: whatsnew/3 3 time921000 +Ref: ab0921000 +Ref: time<5>-Footnote-1921967 +Ref: time<5>-Footnote-2922016 +Node: types<4>922059 +Ref: whatsnew/3 3 types922159 +Ref: ab5922159 +Ref: types<4>-Footnote-1922495 +Ref: types<4>-Footnote-2922538 +Ref: types<4>-Footnote-3922587 +Node: unittest<5>922630 +Ref: whatsnew/3 3 unittest922732 +Ref: ab9922732 +Ref: unittest<5>-Footnote-1923145 +Node: urllib<3>923188 +Ref: whatsnew/3 3 urllib923292 +Ref: ac0923292 +Ref: urllib<3>-Footnote-1923644 +Node: webbrowser923689 +Ref: whatsnew/3 3 webbrowser923803 +Ref: ac2923803 +Ref: webbrowser-Footnote-1924345 +Ref: webbrowser-Footnote-2924388 +Node: xml etree ElementTree924431 +Ref: whatsnew/3 3 xml-etree-elementtree924543 +Ref: ac3924543 +Node: zlib<2>925062 +Ref: whatsnew/3 3 zlib925155 +Ref: ac5925155 +Ref: zlib<2>-Footnote-1925627 +Ref: zlib<2>-Footnote-2925670 +Node: Optimizations<5>925713 +Ref: whatsnew/3 3 optimizations925855 +Ref: ac8925855 +Ref: Optimizations<5>-Footnote-1926666 +Ref: Optimizations<5>-Footnote-2926715 +Ref: Optimizations<5>-Footnote-3926758 +Ref: Optimizations<5>-Footnote-4926801 +Node: Build and C API Changes<4>926844 +Ref: whatsnew/3 3 build-and-c-api-changes926980 +Ref: 987926980 +Ref: Build and C API Changes<4>-Footnote-1928853 +Ref: Build and C API Changes<4>-Footnote-2928902 +Ref: Build and C API Changes<4>-Footnote-3928951 +Node: Deprecated<5>928994 +Ref: whatsnew/3 3 deprecated929135 +Ref: ae7929135 +Node: Unsupported Operating Systems<2>929472 +Ref: whatsnew/3 3 unsupported-operating-systems929613 +Ref: ae8929613 +Node: Deprecated Python modules functions and methods<4>929967 +Ref: whatsnew/3 3 deprecated-python-modules-functions-and-methods930163 +Ref: ae9930163 +Ref: Deprecated Python modules functions and methods<4>-Footnote-1932501 +Ref: Deprecated Python modules functions and methods<4>-Footnote-2932543 +Ref: Deprecated Python modules functions and methods<4>-Footnote-3932592 +Ref: Deprecated Python modules functions and methods<4>-Footnote-4932635 +Ref: Deprecated Python modules functions and methods<4>-Footnote-5932678 +Node: Deprecated functions and types of the C API<3>932721 +Ref: whatsnew/3 3 deprecated-functions-and-types-of-the-c-api932904 +Ref: aed932904 +Ref: Deprecated functions and types of the C API<3>-Footnote-1936044 +Node: Deprecated features936093 +Ref: whatsnew/3 3 deprecated-features936217 +Ref: b10936217 +Node: Porting to Python 3 3936440 +Ref: whatsnew/3 3 porting-to-python-3-3936546 +Ref: b11936546 +Node: Porting Python code936823 +Ref: whatsnew/3 3 porting-python-code936923 +Ref: 9af936923 +Ref: whatsnew/3 3 portingpythoncode936923 +Ref: b12936923 +Ref: Porting Python code-Footnote-1943951 +Ref: Porting Python code-Footnote-2943994 +Ref: Porting Python code-Footnote-3944037 +Ref: Porting Python code-Footnote-4944080 +Ref: Porting Python code-Footnote-5944129 +Ref: Porting Python code-Footnote-6944178 +Ref: Porting Python code-Footnote-7944220 +Ref: Porting Python code-Footnote-8944265 +Ref: Porting Python code-Footnote-9944308 +Ref: Porting Python code-Footnote-10944350 +Node: Porting C code944394 +Ref: whatsnew/3 3 porting-c-code944524 +Ref: 988944524 +Ref: Porting C code-Footnote-1946016 +Node: Building C extensions946065 +Ref: whatsnew/3 3 building-c-extensions946203 +Ref: b22946203 +Ref: Building C extensions-Footnote-1946801 +Node: Command Line Switch Changes946844 +Ref: whatsnew/3 3 command-line-switch-changes946959 +Ref: b23946959 +Ref: Command Line Switch Changes-Footnote-1947524 +Ref: Command Line Switch Changes-Footnote-2947567 +Node: What’s New In Python 3 2947610 +Ref: whatsnew/3 2 doc947765 +Ref: b25947765 +Ref: whatsnew/3 2 what-s-new-in-python-3-2947765 +Ref: b26947765 +Ref: What’s New In Python 3 2-Footnote-1949113 +Ref: What’s New In Python 3 2-Footnote-2949212 +Node: PEP 384 Defining a Stable ABI949261 +Ref: whatsnew/3 2 pep-384-defining-a-stable-abi949406 +Ref: b27949406 +Ref: PEP 384 Defining a Stable ABI-Footnote-1950497 +Node: PEP 389 Argparse Command Line Parsing Module950546 +Ref: whatsnew/3 2 pep-389-argparse-command-line-parsing-module950750 +Ref: b28950750 +Ref: PEP 389 Argparse Command Line Parsing Module-Footnote-1954876 +Node: PEP 391 Dictionary Based Configuration for Logging954925 +Ref: whatsnew/3 2 pep-391-dictionary-based-configuration-for-logging955138 +Ref: b2a955138 +Ref: PEP 391 Dictionary Based Configuration for Logging-Footnote-1957429 +Node: PEP 3148 The concurrent futures module957478 +Ref: whatsnew/3 2 pep-3148-the-concurrent-futures-module957682 +Ref: b2c957682 +Ref: PEP 3148 The concurrent futures module-Footnote-1960380 +Node: PEP 3147 PYC Repository Directories960429 +Ref: whatsnew/3 2 pep-3147-pyc-repository-directories960619 +Ref: b31960619 +Ref: PEP 3147 PYC Repository Directories-Footnote-1963697 +Node: PEP 3149 ABI Version Tagged so Files963746 +Ref: whatsnew/3 2 pep-3149-abi-version-tagged-so-files963949 +Ref: b34963949 +Ref: PEP 3149 ABI Version Tagged so Files-Footnote-1965207 +Node: PEP 3333 Python Web Server Gateway Interface v1 0 1965256 +Ref: whatsnew/3 2 pep-3333-python-web-server-gateway-interface-v1-0-1965449 +Ref: b35965449 +Ref: PEP 3333 Python Web Server Gateway Interface v1 0 1-Footnote-1967811 +Ref: PEP 3333 Python Web Server Gateway Interface v1 0 1-Footnote-2967860 +Ref: PEP 3333 Python Web Server Gateway Interface v1 0 1-Footnote-3967909 +Node: Other Language Changes<7>967958 +Ref: whatsnew/3 2 other-language-changes968150 +Ref: b38968150 +Ref: Other Language Changes<7>-Footnote-1977649 +Ref: Other Language Changes<7>-Footnote-2977691 +Ref: Other Language Changes<7>-Footnote-3977733 +Ref: Other Language Changes<7>-Footnote-4977778 +Ref: Other Language Changes<7>-Footnote-5977820 +Ref: Other Language Changes<7>-Footnote-6977862 +Ref: Other Language Changes<7>-Footnote-7977904 +Ref: Other Language Changes<7>-Footnote-8977946 +Ref: Other Language Changes<7>-Footnote-9977988 +Ref: Other Language Changes<7>-Footnote-10978030 +Ref: Other Language Changes<7>-Footnote-11978074 +Ref: Other Language Changes<7>-Footnote-12978119 +Ref: Other Language Changes<7>-Footnote-13978162 +Ref: Other Language Changes<7>-Footnote-14978205 +Ref: Other Language Changes<7>-Footnote-15978249 +Ref: Other Language Changes<7>-Footnote-16978293 +Node: New Improved and Deprecated Modules978336 +Ref: whatsnew/3 2 new-improved-and-deprecated-modules978492 +Ref: b45978492 +Node: email<5>980518 +Ref: whatsnew/3 2 email980618 +Ref: b46980618 +Ref: email<5>-Footnote-1982592 +Ref: email<5>-Footnote-2982641 +Ref: email<5>-Footnote-3982683 +Node: elementtree982726 +Ref: whatsnew/3 2 elementtree982847 +Ref: b4e982847 +Ref: elementtree-Footnote-1984325 +Ref: elementtree-Footnote-2984381 +Node: functools<6>984423 +Ref: whatsnew/3 2 functools984548 +Ref: b55984548 +Ref: functools<6>-Footnote-1988003 +Ref: functools<6>-Footnote-2988055 +Ref: functools<6>-Footnote-3988107 +Ref: functools<6>-Footnote-4988150 +Ref: functools<6>-Footnote-5988193 +Ref: functools<6>-Footnote-6988235 +Ref: functools<6>-Footnote-7988277 +Ref: functools<6>-Footnote-8988319 +Node: itertools<4>988371 +Ref: whatsnew/3 2 itertools988499 +Ref: b57988499 +Node: collections<8>989189 +Ref: whatsnew/3 2 collections989317 +Ref: b59989317 +Ref: collections<8>-Footnote-1991559 +Ref: collections<8>-Footnote-2991619 +Node: threading<6>991666 +Ref: whatsnew/3 2 threading991799 +Ref: b5e991799 +Ref: threading<6>-Footnote-1994560 +Ref: threading<6>-Footnote-2994621 +Ref: threading<6>-Footnote-3994719 +Ref: threading<6>-Footnote-4994791 +Node: datetime and time994833 +Ref: whatsnew/3 2 datetime-and-time994959 +Ref: b62994959 +Ref: datetime and time-Footnote-1997747 +Ref: datetime and time-Footnote-2997792 +Ref: datetime and time-Footnote-3997834 +Ref: datetime and time-Footnote-4997876 +Ref: datetime and time-Footnote-5997918 +Ref: datetime and time-Footnote-6997963 +Ref: datetime and time-Footnote-7998005 +Node: math<6>998048 +Ref: whatsnew/3 2 math998168 +Ref: b66998168 +Ref: math<6>-Footnote-11000045 +Node: abc<3>1000098 +Ref: whatsnew/3 2 abc1000206 +Ref: b6b1000206 +Ref: abc<3>-Footnote-11000828 +Node: io<5>1000870 +Ref: whatsnew/3 2 io1000978 +Ref: b6c1000978 +Ref: io<5>-Footnote-11002143 +Node: reprlib1002185 +Ref: whatsnew/3 2 reprlib1002297 +Ref: b6e1002297 +Ref: reprlib-Footnote-11003285 +Ref: reprlib-Footnote-21003327 +Node: logging<7>1003369 +Ref: whatsnew/3 2 logging1003482 +Ref: b711003482 +Node: csv<3>1005333 +Ref: whatsnew/3 2 csv1005452 +Ref: b781005452 +Ref: csv<3>-Footnote-11006313 +Ref: csv<3>-Footnote-21006355 +Node: contextlib<6>1006400 +Ref: whatsnew/3 2 contextlib1006530 +Ref: b7c1006530 +Ref: contextlib<6>-Footnote-11008743 +Node: decimal and fractions1008785 +Ref: whatsnew/3 2 decimal-and-fractions1008912 +Ref: b7f1008912 +Ref: decimal and fractions-Footnote-11011334 +Ref: decimal and fractions-Footnote-21011376 +Ref: decimal and fractions-Footnote-31011418 +Ref: decimal and fractions-Footnote-41011460 +Ref: decimal and fractions-Footnote-51011502 +Ref: decimal and fractions-Footnote-61011544 +Node: ftp1011586 +Ref: whatsnew/3 2 ftp1011705 +Ref: b831011705 +Ref: ftp-Footnote-11013047 +Ref: ftp-Footnote-21013089 +Ref: ftp-Footnote-31013131 +Ref: ftp-Footnote-41013173 +Node: popen1013215 +Ref: whatsnew/3 2 popen1013322 +Ref: b841013322 +Ref: popen-Footnote-11013627 +Ref: popen-Footnote-21013669 +Node: select<3>1013712 +Ref: whatsnew/3 2 select1013832 +Ref: b851013832 +Ref: select<3>-Footnote-11014263 +Node: gzip and zipfile1014305 +Ref: whatsnew/3 2 gzip-and-zipfile1014430 +Ref: b871014430 +Ref: gzip and zipfile-Footnote-11015985 +Ref: gzip and zipfile-Footnote-21016027 +Ref: gzip and zipfile-Footnote-31016069 +Ref: gzip and zipfile-Footnote-41016114 +Ref: gzip and zipfile-Footnote-51016156 +Ref: gzip and zipfile-Footnote-61016198 +Node: tarfile<5>1016240 +Ref: whatsnew/3 2 tarfile1016366 +Ref: b8b1016366 +Ref: tarfile<5>-Footnote-11017937 +Node: hashlib<3>1017979 +Ref: whatsnew/3 2 hashlib1018095 +Ref: b8e1018095 +Ref: hashlib<3>-Footnote-11018850 +Node: ast<3>1018892 +Ref: whatsnew/3 2 ast1019003 +Ref: b8f1019003 +Node: os<8>1019940 +Ref: whatsnew/3 2 os1020050 +Ref: b911020050 +Node: shutil<5>1020819 +Ref: whatsnew/3 2 shutil1020933 +Ref: b951020933 +Node: sqlite3<6>1023100 +Ref: whatsnew/3 2 sqlite31023216 +Ref: b981023216 +Ref: sqlite3<6>-Footnote-11023865 +Node: html<3>1023907 +Ref: whatsnew/3 2 html1024023 +Ref: b9b1024023 +Node: socket<8>1024309 +Ref: whatsnew/3 2 socket1024421 +Ref: b9c1024421 +Ref: socket<8>-Footnote-11025063 +Ref: socket<8>-Footnote-21025105 +Node: ssl<9>1025147 +Ref: whatsnew/3 2 ssl1025256 +Ref: b9f1025256 +Ref: ssl<9>-Footnote-11027358 +Ref: ssl<9>-Footnote-21027407 +Ref: ssl<9>-Footnote-31027492 +Ref: ssl<9>-Footnote-41027534 +Ref: ssl<9>-Footnote-51027576 +Ref: ssl<9>-Footnote-61027618 +Ref: ssl<9>-Footnote-71027660 +Ref: ssl<9>-Footnote-81027702 +Ref: ssl<9>-Footnote-91027744 +Node: nntp1027786 +Ref: whatsnew/3 2 nntp1027898 +Ref: ba41027898 +Ref: nntp-Footnote-11028468 +Ref: nntp-Footnote-21028510 +Node: certificates1028552 +Ref: whatsnew/3 2 certificates1028668 +Ref: ba71028668 +Ref: certificates-Footnote-11029063 +Node: imaplib<3>1029105 +Ref: whatsnew/3 2 imaplib1029231 +Ref: ba91029231 +Ref: imaplib<3>-Footnote-11029502 +Node: http client<4>1029544 +Ref: whatsnew/3 2 http-client1029669 +Ref: bab1029669 +Ref: http client<4>-Footnote-11031091 +Node: unittest<6>1031134 +Ref: whatsnew/3 2 unittest1031258 +Ref: bae1031258 +Ref: unittest<6>-Footnote-11036039 +Ref: unittest<6>-Footnote-21036082 +Ref: unittest<6>-Footnote-31036124 +Node: random<2>1036166 +Ref: whatsnew/3 2 random1036285 +Ref: bba1036285 +Ref: random<2>-Footnote-11036947 +Node: poplib<3>1036989 +Ref: whatsnew/3 2 poplib1037108 +Ref: bc01037108 +Ref: poplib<3>-Footnote-11037460 +Node: asyncore<2>1037502 +Ref: whatsnew/3 2 asyncore1037623 +Ref: bc21037623 +Ref: asyncore<2>-Footnote-11038099 +Node: tempfile<2>1038141 +Ref: whatsnew/3 2 tempfile1038263 +Ref: bc71038263 +Ref: tempfile<2>-Footnote-11038670 +Node: inspect<6>1038712 +Ref: whatsnew/3 2 inspect1038831 +Ref: bc91038831 +Ref: inspect<6>-Footnote-11040189 +Node: pydoc<5>1040232 +Ref: whatsnew/3 2 pydoc1040346 +Ref: bcc1040346 +Ref: pydoc<5>-Footnote-11040663 +Node: dis<3>1040705 +Ref: whatsnew/3 2 dis1040815 +Ref: bcd1040815 +Ref: dis<3>-Footnote-11042963 +Node: dbm<6>1043005 +Ref: whatsnew/3 2 dbm1043116 +Ref: bcf1043116 +Ref: dbm<6>-Footnote-11043303 +Node: ctypes<2>1043345 +Ref: whatsnew/3 2 ctypes1043457 +Ref: bd01043457 +Node: site<2>1043573 +Ref: whatsnew/3 2 site1043691 +Ref: bd21043691 +Ref: site<2>-Footnote-11044920 +Node: sysconfig<2>1044962 +Ref: whatsnew/3 2 sysconfig1045077 +Ref: bd61045077 +Node: pdb<5>1047306 +Ref: whatsnew/3 2 pdb1047429 +Ref: bda1047429 +Node: configparser<2>1048327 +Ref: whatsnew/3 2 configparser1048453 +Ref: bdb1048453 +Node: urllib parse<2>1051317 +Ref: whatsnew/3 2 urllib-parse1051444 +Ref: bdd1051444 +Ref: urllib parse<2>-Footnote-11053452 +Ref: urllib parse<2>-Footnote-21053495 +Ref: urllib parse<2>-Footnote-31053544 +Ref: urllib parse<2>-Footnote-41053586 +Ref: urllib parse<2>-Footnote-51053628 +Node: mailbox1053670 +Ref: whatsnew/3 2 mailbox1053792 +Ref: be11053792 +Ref: mailbox-Footnote-11055364 +Node: turtledemo1055406 +Ref: whatsnew/3 2 turtledemo1055504 +Ref: be81055504 +Ref: turtledemo-Footnote-11055939 +Node: Multi-threading1055982 +Ref: whatsnew/3 2 multi-threading1056129 +Ref: be91056129 +Ref: Multi-threading-Footnote-11057766 +Ref: Multi-threading-Footnote-21057845 +Ref: Multi-threading-Footnote-31057887 +Ref: Multi-threading-Footnote-41057931 +Node: Optimizations<6>1057973 +Ref: whatsnew/3 2 optimizations1058092 +Ref: bed1058092 +Ref: Optimizations<6>-Footnote-11061531 +Ref: Optimizations<6>-Footnote-21061573 +Ref: Optimizations<6>-Footnote-31061615 +Ref: Optimizations<6>-Footnote-41061657 +Ref: Optimizations<6>-Footnote-51061703 +Ref: Optimizations<6>-Footnote-61061745 +Ref: Optimizations<6>-Footnote-71061787 +Ref: Optimizations<6>-Footnote-81061830 +Ref: Optimizations<6>-Footnote-91061872 +Ref: Optimizations<6>-Footnote-101061914 +Ref: Optimizations<6>-Footnote-111061957 +Ref: Optimizations<6>-Footnote-121062000 +Ref: Optimizations<6>-Footnote-131062043 +Ref: Optimizations<6>-Footnote-141062089 +Ref: Optimizations<6>-Footnote-151062132 +Ref: Optimizations<6>-Footnote-161062176 +Node: Unicode1062219 +Ref: whatsnew/3 2 unicode1062329 +Ref: bf01062329 +Ref: Unicode-Footnote-11062879 +Ref: Unicode-Footnote-21062929 +Ref: Unicode-Footnote-31062973 +Node: Codecs1063044 +Ref: whatsnew/3 2 codecs1063151 +Ref: bf11063151 +Ref: Codecs-Footnote-11064080 +Node: Documentation1064125 +Ref: whatsnew/3 2 documentation1064229 +Ref: bf21064229 +Ref: Documentation-Footnote-11065873 +Ref: Documentation-Footnote-21065941 +Ref: Documentation-Footnote-31066017 +Ref: Documentation-Footnote-41066059 +Node: IDLE1066101 +Ref: whatsnew/3 2 idle1066214 +Ref: bf61066214 +Ref: IDLE-Footnote-11066598 +Ref: IDLE-Footnote-21066640 +Node: Code Repository1066682 +Ref: whatsnew/3 2 code-repository1066808 +Ref: bf71066808 +Ref: Code Repository-Footnote-11067414 +Ref: Code Repository-Footnote-21067453 +Ref: Code Repository-Footnote-31067502 +Ref: Code Repository-Footnote-41067556 +Node: Build and C API Changes<5>1067600 +Ref: whatsnew/3 2 build-and-c-api-changes1067743 +Ref: bf81067743 +Ref: Build and C API Changes<5>-Footnote-11071880 +Ref: Build and C API Changes<5>-Footnote-21071923 +Ref: Build and C API Changes<5>-Footnote-31071965 +Ref: Build and C API Changes<5>-Footnote-41072007 +Ref: Build and C API Changes<5>-Footnote-51072049 +Ref: Build and C API Changes<5>-Footnote-61072091 +Ref: Build and C API Changes<5>-Footnote-71072133 +Ref: Build and C API Changes<5>-Footnote-81072175 +Ref: Build and C API Changes<5>-Footnote-91072217 +Ref: Build and C API Changes<5>-Footnote-101072259 +Ref: Build and C API Changes<5>-Footnote-111072302 +Ref: Build and C API Changes<5>-Footnote-121072345 +Ref: Build and C API Changes<5>-Footnote-131072388 +Ref: Build and C API Changes<5>-Footnote-141072450 +Ref: Build and C API Changes<5>-Footnote-151072530 +Node: Porting to Python 3 21072587 +Ref: whatsnew/3 2 porting-to-python-3-21072706 +Ref: c011072706 +Ref: Porting to Python 3 2-Footnote-11079549 +Ref: Porting to Python 3 2-Footnote-21079591 +Ref: Porting to Python 3 2-Footnote-31079633 +Ref: Porting to Python 3 2-Footnote-41079678 +Ref: Porting to Python 3 2-Footnote-51079721 +Ref: Porting to Python 3 2-Footnote-61079764 +Ref: Porting to Python 3 2-Footnote-71079807 +Node: What’s New In Python 3 11079849 +Ref: whatsnew/3 1 doc1080004 +Ref: c131080004 +Ref: whatsnew/3 1 what-s-new-in-python-3-11080004 +Ref: c141080004 +Node: PEP 372 Ordered Dictionaries1080630 +Ref: whatsnew/3 1 pep-372-ordered-dictionaries1080778 +Ref: c151080778 +Ref: PEP 372 Ordered Dictionaries-Footnote-11082327 +Ref: PEP 372 Ordered Dictionaries-Footnote-21082354 +Node: PEP 378 Format Specifier for Thousands Separator1082403 +Ref: whatsnew/3 1 pep-378-format-specifier-for-thousands-separator1082585 +Ref: c161082585 +Ref: PEP 378 Format Specifier for Thousands Separator-Footnote-11083824 +Node: Other Language Changes<8>1083873 +Ref: whatsnew/3 1 other-language-changes1084065 +Ref: c171084065 +Ref: Other Language Changes<8>-Footnote-11088862 +Ref: Other Language Changes<8>-Footnote-21088907 +Ref: Other Language Changes<8>-Footnote-31088949 +Ref: Other Language Changes<8>-Footnote-41088991 +Ref: Other Language Changes<8>-Footnote-51089033 +Ref: Other Language Changes<8>-Footnote-61089078 +Ref: Other Language Changes<8>-Footnote-71089120 +Node: New Improved and Deprecated Modules<2>1089162 +Ref: whatsnew/3 1 new-improved-and-deprecated-modules1089322 +Ref: c181089322 +Ref: New Improved and Deprecated Modules<2>-Footnote-11097850 +Ref: New Improved and Deprecated Modules<2>-Footnote-21097895 +Ref: New Improved and Deprecated Modules<2>-Footnote-31097937 +Ref: New Improved and Deprecated Modules<2>-Footnote-41097979 +Ref: New Improved and Deprecated Modules<2>-Footnote-51098021 +Ref: New Improved and Deprecated Modules<2>-Footnote-61098063 +Ref: New Improved and Deprecated Modules<2>-Footnote-71098112 +Ref: New Improved and Deprecated Modules<2>-Footnote-81098154 +Ref: New Improved and Deprecated Modules<2>-Footnote-91098196 +Ref: New Improved and Deprecated Modules<2>-Footnote-101098238 +Ref: New Improved and Deprecated Modules<2>-Footnote-111098281 +Ref: New Improved and Deprecated Modules<2>-Footnote-121098324 +Ref: New Improved and Deprecated Modules<2>-Footnote-131098367 +Node: Optimizations<7>1098410 +Ref: whatsnew/3 1 optimizations1098552 +Ref: c1e1098552 +Ref: Optimizations<7>-Footnote-11100729 +Ref: Optimizations<7>-Footnote-21100778 +Ref: Optimizations<7>-Footnote-31100820 +Ref: Optimizations<7>-Footnote-41100862 +Ref: Optimizations<7>-Footnote-51100904 +Ref: Optimizations<7>-Footnote-61100929 +Ref: Optimizations<7>-Footnote-71100971 +Node: IDLE<2>1101013 +Ref: whatsnew/3 1 idle1101143 +Ref: c1f1101143 +Ref: IDLE<2>-Footnote-11101358 +Node: Build and C API Changes<6>1101400 +Ref: whatsnew/3 1 build-and-c-api-changes1101535 +Ref: c201101535 +Ref: Build and C API Changes<6>-Footnote-11103760 +Ref: Build and C API Changes<6>-Footnote-21103802 +Ref: Build and C API Changes<6>-Footnote-31103844 +Ref: Build and C API Changes<6>-Footnote-41103886 +Ref: Build and C API Changes<6>-Footnote-51103928 +Node: Porting to Python 3 11103970 +Ref: whatsnew/3 1 porting-to-python-3-11104089 +Ref: c241104089 +Node: What’s New In Python 3 01105219 +Ref: whatsnew/3 0 doc1105374 +Ref: c251105374 +Ref: whatsnew/3 0 what-s-new-in-python-3-01105374 +Ref: c261105374 +Node: Common Stumbling Blocks1107185 +Ref: whatsnew/3 0 common-stumbling-blocks1107306 +Ref: c271107306 +Node: Print Is A Function1107674 +Ref: whatsnew/3 0 print-is-a-function1107798 +Ref: c281107798 +Ref: Print Is A Function-Footnote-11109482 +Node: Views And Iterators Instead Of Lists1109531 +Ref: whatsnew/3 0 views-and-iterators-instead-of-lists1109684 +Ref: c291109684 +Node: Ordering Comparisons1111524 +Ref: whatsnew/3 0 ordering-comparisons1111666 +Ref: c331111666 +Node: Integers1113117 +Ref: whatsnew/3 0 integers1113263 +Ref: c351113263 +Ref: Integers-Footnote-11114485 +Ref: Integers-Footnote-21114534 +Node: Text Vs Data Instead Of Unicode Vs 8-bit1114583 +Ref: whatsnew/3 0 text-vs-data-instead-of-unicode-vs-8-bit1114700 +Ref: c361114700 +Ref: Text Vs Data Instead Of Unicode Vs 8-bit-Footnote-11121284 +Ref: Text Vs Data Instead Of Unicode Vs 8-bit-Footnote-21121333 +Ref: Text Vs Data Instead Of Unicode Vs 8-bit-Footnote-31121382 +Node: Overview Of Syntax Changes1121431 +Ref: whatsnew/3 0 overview-of-syntax-changes1121598 +Ref: c3d1121598 +Node: New Syntax1121808 +Ref: whatsnew/3 0 new-syntax1121904 +Ref: c3e1121904 +Ref: New Syntax-Footnote-11124450 +Ref: New Syntax-Footnote-21124499 +Ref: New Syntax-Footnote-31124548 +Ref: New Syntax-Footnote-41124597 +Ref: New Syntax-Footnote-51124646 +Node: Changed Syntax1124695 +Ref: whatsnew/3 0 changed-syntax1124814 +Ref: c411124814 +Ref: Changed Syntax-Footnote-11126507 +Ref: Changed Syntax-Footnote-21126556 +Ref: Changed Syntax-Footnote-31126605 +Ref: Changed Syntax-Footnote-41126654 +Node: Removed Syntax1126703 +Ref: whatsnew/3 0 removed-syntax1126803 +Ref: c431126803 +Ref: Removed Syntax-Footnote-11127993 +Ref: Removed Syntax-Footnote-21128042 +Node: Changes Already Present In Python 2 61128091 +Ref: whatsnew/3 0 changes-already-present-in-python-2-61128250 +Ref: c461128250 +Node: Library Changes1132087 +Ref: whatsnew/3 0 library-changes1132264 +Ref: c591132264 +Ref: Library Changes-Footnote-11138492 +Ref: Library Changes-Footnote-21138541 +Ref: Library Changes-Footnote-31138590 +Ref: Library Changes-Footnote-41138639 +Ref: Library Changes-Footnote-51138688 +Ref: Library Changes-Footnote-61138737 +Node: PEP 3101 A New Approach To String Formatting1138786 +Ref: whatsnew/3 0 pep-3101-a-new-approach-to-string-formatting1138947 +Ref: c5d1138947 +Ref: PEP 3101 A New Approach To String Formatting-Footnote-11139399 +Node: Changes To Exceptions1139448 +Ref: whatsnew/3 0 changes-to-exceptions1139621 +Ref: c5e1139621 +Ref: Changes To Exceptions-Footnote-11143125 +Ref: Changes To Exceptions-Footnote-21143174 +Ref: Changes To Exceptions-Footnote-31143223 +Ref: Changes To Exceptions-Footnote-41143272 +Ref: Changes To Exceptions-Footnote-51143321 +Node: Miscellaneous Other Changes1143370 +Ref: whatsnew/3 0 miscellaneous-other-changes1143525 +Ref: c601143525 +Node: Operators And Special Methods1143651 +Ref: whatsnew/3 0 operators-and-special-methods1143761 +Ref: c611143761 +Ref: Operators And Special Methods-Footnote-11145401 +Node: Builtins1145450 +Ref: whatsnew/3 0 builtins1145560 +Ref: c691145560 +Ref: Builtins-Footnote-11148031 +Ref: Builtins-Footnote-21148080 +Node: Build and C API Changes<7>1148129 +Ref: whatsnew/3 0 build-and-c-api-changes1148274 +Ref: c711148274 +Ref: Build and C API Changes<7>-Footnote-11149307 +Ref: Build and C API Changes<7>-Footnote-21149356 +Ref: Build and C API Changes<7>-Footnote-31149405 +Node: Performance1149454 +Ref: whatsnew/3 0 performance1149593 +Ref: c741149593 +Node: Porting To Python 3 01149912 +Ref: whatsnew/3 0 porting-to-python-3-01150016 +Ref: c751150016 +Node: What’s New in Python 2 71151617 +Ref: whatsnew/2 7 doc1151772 +Ref: c781151772 +Ref: whatsnew/2 7 what-s-new-in-python-2-71151772 +Ref: c791151772 +Node: The Future for Python 2 x1154235 +Ref: whatsnew/2 7 the-future-for-python-2-x1154379 +Ref: c7a1154379 +Ref: whatsnew/2 7 whatsnew27-python311154379 +Ref: c7b1154379 +Ref: The Future for Python 2 x-Footnote-11157726 +Ref: The Future for Python 2 x-Footnote-21157775 +Node: Changes to the Handling of Deprecation Warnings1157812 +Ref: whatsnew/2 7 changes-to-the-handling-of-deprecation-warnings1157984 +Ref: c7e1157984 +Ref: Changes to the Handling of Deprecation Warnings-Footnote-11159559 +Node: Python 3 1 Features1159601 +Ref: whatsnew/2 7 python-3-1-features1159799 +Ref: c7f1159799 +Node: PEP 372 Adding an Ordered Dictionary to collections1161718 +Ref: whatsnew/2 7 pep-03721161920 +Ref: c801161920 +Ref: whatsnew/2 7 pep-372-adding-an-ordered-dictionary-to-collections1161920 +Ref: c831161920 +Ref: PEP 372 Adding an Ordered Dictionary to collections-Footnote-11165524 +Ref: PEP 372 Adding an Ordered Dictionary to collections-Footnote-21165551 +Node: PEP 378 Format Specifier for Thousands Separator<2>1165600 +Ref: whatsnew/2 7 pep-03781165836 +Ref: c811165836 +Ref: whatsnew/2 7 pep-378-format-specifier-for-thousands-separator1165836 +Ref: c851165836 +Ref: PEP 378 Format Specifier for Thousands Separator<2>-Footnote-11167353 +Node: PEP 389 The argparse Module for Parsing Command Lines1167402 +Ref: whatsnew/2 7 pep-389-the-argparse-module-for-parsing-command-lines1167637 +Ref: c861167637 +Ref: PEP 389 The argparse Module for Parsing Command Lines-Footnote-11171661 +Node: PEP 391 Dictionary-Based Configuration For Logging1171710 +Ref: whatsnew/2 7 pep-391-dictionary-based-configuration-for-logging1171919 +Ref: c871171919 +Ref: PEP 391 Dictionary-Based Configuration For Logging-Footnote-11175787 +Node: PEP 3106 Dictionary Views1175836 +Ref: whatsnew/2 7 pep-3106-dictionary-views1176022 +Ref: c8d1176022 +Ref: PEP 3106 Dictionary Views-Footnote-11178139 +Ref: PEP 3106 Dictionary Views-Footnote-21178188 +Node: PEP 3137 The memoryview Object1178230 +Ref: whatsnew/2 7 pep-3137-the-memoryview-object1178391 +Ref: c8e1178391 +Ref: PEP 3137 The memoryview Object-Footnote-11180056 +Ref: PEP 3137 The memoryview Object-Footnote-21180105 +Node: Other Language Changes<9>1180147 +Ref: whatsnew/2 7 other-language-changes1180307 +Ref: c8f1180307 +Ref: Other Language Changes<9>-Footnote-11190263 +Ref: Other Language Changes<9>-Footnote-21190305 +Ref: Other Language Changes<9>-Footnote-31190347 +Ref: Other Language Changes<9>-Footnote-41190389 +Ref: Other Language Changes<9>-Footnote-51190431 +Ref: Other Language Changes<9>-Footnote-61190473 +Ref: Other Language Changes<9>-Footnote-71190515 +Ref: Other Language Changes<9>-Footnote-81190557 +Ref: Other Language Changes<9>-Footnote-91190599 +Ref: Other Language Changes<9>-Footnote-101190641 +Ref: Other Language Changes<9>-Footnote-111190684 +Ref: Other Language Changes<9>-Footnote-121190727 +Ref: Other Language Changes<9>-Footnote-131190770 +Ref: Other Language Changes<9>-Footnote-141190813 +Ref: Other Language Changes<9>-Footnote-151190859 +Ref: Other Language Changes<9>-Footnote-161190902 +Ref: Other Language Changes<9>-Footnote-171190945 +Ref: Other Language Changes<9>-Footnote-181190988 +Ref: Other Language Changes<9>-Footnote-191191034 +Ref: Other Language Changes<9>-Footnote-201191077 +Ref: Other Language Changes<9>-Footnote-211191120 +Ref: Other Language Changes<9>-Footnote-221191163 +Ref: Other Language Changes<9>-Footnote-231191206 +Ref: Other Language Changes<9>-Footnote-241191249 +Node: Interpreter Changes1191292 +Ref: whatsnew/2 7 interpreter-changes1191398 +Ref: c921191398 +Ref: whatsnew/2 7 new-27-interpreter1191398 +Ref: c931191398 +Ref: Interpreter Changes-Footnote-11192052 +Node: Optimizations<8>1192094 +Ref: whatsnew/2 7 optimizations1192200 +Ref: c941192200 +Ref: Optimizations<8>-Footnote-11197047 +Ref: Optimizations<8>-Footnote-21197089 +Ref: Optimizations<8>-Footnote-31197131 +Ref: Optimizations<8>-Footnote-41197173 +Ref: Optimizations<8>-Footnote-51197215 +Ref: Optimizations<8>-Footnote-61197257 +Ref: Optimizations<8>-Footnote-71197302 +Ref: Optimizations<8>-Footnote-81197344 +Ref: Optimizations<8>-Footnote-91197386 +Ref: Optimizations<8>-Footnote-101197428 +Ref: Optimizations<8>-Footnote-111197471 +Ref: Optimizations<8>-Footnote-121197514 +Ref: Optimizations<8>-Footnote-131197557 +Node: New and Improved Modules1197600 +Ref: whatsnew/2 7 new-and-improved-modules1197756 +Ref: c971197756 +Ref: New and Improved Modules-Footnote-11232932 +Ref: New and Improved Modules-Footnote-21232974 +Ref: New and Improved Modules-Footnote-31233016 +Ref: New and Improved Modules-Footnote-41233069 +Ref: New and Improved Modules-Footnote-51233111 +Ref: New and Improved Modules-Footnote-61233153 +Ref: New and Improved Modules-Footnote-71233198 +Ref: New and Improved Modules-Footnote-81233240 +Ref: New and Improved Modules-Footnote-91233282 +Ref: New and Improved Modules-Footnote-101233324 +Ref: New and Improved Modules-Footnote-111233367 +Ref: New and Improved Modules-Footnote-121233410 +Ref: New and Improved Modules-Footnote-131233453 +Ref: New and Improved Modules-Footnote-141233493 +Ref: New and Improved Modules-Footnote-151233536 +Ref: New and Improved Modules-Footnote-161233579 +Ref: New and Improved Modules-Footnote-171233622 +Ref: New and Improved Modules-Footnote-181233665 +Ref: New and Improved Modules-Footnote-191233708 +Ref: New and Improved Modules-Footnote-201233751 +Ref: New and Improved Modules-Footnote-211233794 +Ref: New and Improved Modules-Footnote-221233837 +Ref: New and Improved Modules-Footnote-231233880 +Ref: New and Improved Modules-Footnote-241233923 +Ref: New and Improved Modules-Footnote-251233966 +Ref: New and Improved Modules-Footnote-261234009 +Ref: New and Improved Modules-Footnote-271234055 +Ref: New and Improved Modules-Footnote-281234098 +Ref: New and Improved Modules-Footnote-291234141 +Ref: New and Improved Modules-Footnote-301234184 +Ref: New and Improved Modules-Footnote-311234227 +Ref: New and Improved Modules-Footnote-321234270 +Ref: New and Improved Modules-Footnote-331234313 +Ref: New and Improved Modules-Footnote-341234356 +Ref: New and Improved Modules-Footnote-351234399 +Ref: New and Improved Modules-Footnote-361234442 +Ref: New and Improved Modules-Footnote-371234485 +Ref: New and Improved Modules-Footnote-381234528 +Ref: New and Improved Modules-Footnote-391234571 +Ref: New and Improved Modules-Footnote-401234614 +Ref: New and Improved Modules-Footnote-411234657 +Ref: New and Improved Modules-Footnote-421234700 +Ref: New and Improved Modules-Footnote-431234743 +Ref: New and Improved Modules-Footnote-441234786 +Ref: New and Improved Modules-Footnote-451234829 +Ref: New and Improved Modules-Footnote-461234872 +Ref: New and Improved Modules-Footnote-471234915 +Ref: New and Improved Modules-Footnote-481234958 +Ref: New and Improved Modules-Footnote-491235004 +Ref: New and Improved Modules-Footnote-501235047 +Ref: New and Improved Modules-Footnote-511235090 +Ref: New and Improved Modules-Footnote-521235133 +Ref: New and Improved Modules-Footnote-531235176 +Ref: New and Improved Modules-Footnote-541235219 +Ref: New and Improved Modules-Footnote-551235262 +Ref: New and Improved Modules-Footnote-561235305 +Ref: New and Improved Modules-Footnote-571235348 +Ref: New and Improved Modules-Footnote-581235391 +Ref: New and Improved Modules-Footnote-591235434 +Ref: New and Improved Modules-Footnote-601235477 +Ref: New and Improved Modules-Footnote-611235520 +Ref: New and Improved Modules-Footnote-621235563 +Ref: New and Improved Modules-Footnote-631235606 +Ref: New and Improved Modules-Footnote-641235649 +Ref: New and Improved Modules-Footnote-651235692 +Ref: New and Improved Modules-Footnote-661235735 +Ref: New and Improved Modules-Footnote-671235778 +Ref: New and Improved Modules-Footnote-681235821 +Ref: New and Improved Modules-Footnote-691235864 +Ref: New and Improved Modules-Footnote-701235910 +Ref: New and Improved Modules-Footnote-711235953 +Ref: New and Improved Modules-Footnote-721235996 +Ref: New and Improved Modules-Footnote-731236082 +Ref: New and Improved Modules-Footnote-741236125 +Ref: New and Improved Modules-Footnote-751236168 +Ref: New and Improved Modules-Footnote-761236211 +Ref: New and Improved Modules-Footnote-771236254 +Ref: New and Improved Modules-Footnote-781236297 +Ref: New and Improved Modules-Footnote-791236343 +Ref: New and Improved Modules-Footnote-801236386 +Ref: New and Improved Modules-Footnote-811236429 +Ref: New and Improved Modules-Footnote-821236472 +Ref: New and Improved Modules-Footnote-831236515 +Ref: New and Improved Modules-Footnote-841236558 +Ref: New and Improved Modules-Footnote-851236601 +Ref: New and Improved Modules-Footnote-861236647 +Ref: New and Improved Modules-Footnote-871236693 +Ref: New and Improved Modules-Footnote-881236736 +Ref: New and Improved Modules-Footnote-891236786 +Ref: New and Improved Modules-Footnote-901236836 +Ref: New and Improved Modules-Footnote-911236879 +Ref: New and Improved Modules-Footnote-921236922 +Ref: New and Improved Modules-Footnote-931236965 +Ref: New and Improved Modules-Footnote-941237008 +Ref: New and Improved Modules-Footnote-951237051 +Ref: New and Improved Modules-Footnote-961237094 +Ref: New and Improved Modules-Footnote-971237137 +Node: New module importlib1237180 +Ref: whatsnew/2 7 importlib-section1237290 +Ref: c821237290 +Ref: whatsnew/2 7 new-module-importlib1237290 +Ref: cbc1237290 +Node: New module sysconfig1238830 +Ref: whatsnew/2 7 new-module-sysconfig1238974 +Ref: cbd1238974 +Node: ttk Themed Widgets for Tk1240300 +Ref: whatsnew/2 7 ttk-themed-widgets-for-tk1240447 +Ref: cc01240447 +Ref: ttk Themed Widgets for Tk-Footnote-11241519 +Ref: ttk Themed Widgets for Tk-Footnote-21241561 +Node: Updated module unittest1241603 +Ref: whatsnew/2 7 unittest-section1241760 +Ref: cc11241760 +Ref: whatsnew/2 7 updated-module-unittest1241760 +Ref: cc21241760 +Ref: Updated module unittest-Footnote-11250615 +Ref: Updated module unittest-Footnote-21250641 +Ref: Updated module unittest-Footnote-31250678 +Ref: Updated module unittest-Footnote-41250720 +Ref: Updated module unittest-Footnote-51250762 +Ref: Updated module unittest-Footnote-61250807 +Ref: Updated module unittest-Footnote-71250849 +Ref: Updated module unittest-Footnote-81250891 +Ref: Updated module unittest-Footnote-91250933 +Ref: Updated module unittest-Footnote-101250975 +Ref: Updated module unittest-Footnote-111251018 +Ref: Updated module unittest-Footnote-121251061 +Ref: Updated module unittest-Footnote-131251104 +Ref: Updated module unittest-Footnote-141251147 +Node: Updated module ElementTree 1 31251190 +Ref: whatsnew/2 7 elementtree-section1251313 +Ref: ce51251313 +Ref: whatsnew/2 7 updated-module-elementtree-1-31251313 +Ref: ce61251313 +Ref: Updated module ElementTree 1 3-Footnote-11255268 +Node: Build and C API Changes<8>1255310 +Ref: whatsnew/2 7 build-and-c-api-changes1255464 +Ref: ce81255464 +Ref: Build and C API Changes<8>-Footnote-11264135 +Ref: Build and C API Changes<8>-Footnote-21264205 +Ref: Build and C API Changes<8>-Footnote-31264247 +Ref: Build and C API Changes<8>-Footnote-41264289 +Ref: Build and C API Changes<8>-Footnote-51264331 +Ref: Build and C API Changes<8>-Footnote-61264373 +Ref: Build and C API Changes<8>-Footnote-71264415 +Ref: Build and C API Changes<8>-Footnote-81264457 +Ref: Build and C API Changes<8>-Footnote-91264526 +Ref: Build and C API Changes<8>-Footnote-101264568 +Ref: Build and C API Changes<8>-Footnote-111264611 +Ref: Build and C API Changes<8>-Footnote-121264654 +Ref: Build and C API Changes<8>-Footnote-131264697 +Ref: Build and C API Changes<8>-Footnote-141264743 +Ref: Build and C API Changes<8>-Footnote-151264789 +Ref: Build and C API Changes<8>-Footnote-161264832 +Ref: Build and C API Changes<8>-Footnote-171264875 +Ref: Build and C API Changes<8>-Footnote-181264918 +Ref: Build and C API Changes<8>-Footnote-191264961 +Ref: Build and C API Changes<8>-Footnote-201265004 +Ref: Build and C API Changes<8>-Footnote-211265050 +Ref: Build and C API Changes<8>-Footnote-221265093 +Node: Capsules1265136 +Ref: whatsnew/2 7 capsules1265245 +Ref: cee1265245 +Ref: whatsnew/2 7 whatsnew27-capsules1265245 +Ref: cef1265245 +Ref: Capsules-Footnote-11267244 +Node: Port-Specific Changes Windows1267286 +Ref: whatsnew/2 7 port-specific-changes-windows1267434 +Ref: cf21267434 +Ref: Port-Specific Changes Windows-Footnote-11269126 +Ref: Port-Specific Changes Windows-Footnote-21269168 +Ref: Port-Specific Changes Windows-Footnote-31269210 +Ref: Port-Specific Changes Windows-Footnote-41269252 +Ref: Port-Specific Changes Windows-Footnote-51269297 +Ref: Port-Specific Changes Windows-Footnote-61269339 +Node: Port-Specific Changes Mac OS X1269381 +Ref: whatsnew/2 7 port-specific-changes-mac-os-x1269550 +Ref: cf41269550 +Ref: Port-Specific Changes Mac OS X-Footnote-11270595 +Ref: Port-Specific Changes Mac OS X-Footnote-21270637 +Node: Port-Specific Changes FreeBSD1270680 +Ref: whatsnew/2 7 port-specific-changes-freebsd1270811 +Ref: cf51270811 +Ref: Port-Specific Changes FreeBSD-Footnote-11271171 +Node: Other Changes and Fixes1271213 +Ref: whatsnew/2 7 other-changes-and-fixes1271364 +Ref: cf61271364 +Ref: Other Changes and Fixes-Footnote-11273552 +Ref: Other Changes and Fixes-Footnote-21273594 +Ref: Other Changes and Fixes-Footnote-31273639 +Ref: Other Changes and Fixes-Footnote-41273681 +Ref: Other Changes and Fixes-Footnote-51273723 +Node: Porting to Python 2 71273765 +Ref: whatsnew/2 7 porting-to-python-2-71273943 +Ref: cf71273943 +Ref: Porting to Python 2 7-Footnote-11279510 +Ref: Porting to Python 2 7-Footnote-21279552 +Ref: Porting to Python 2 7-Footnote-31279594 +Ref: Porting to Python 2 7-Footnote-41279636 +Ref: Porting to Python 2 7-Footnote-51279678 +Ref: Porting to Python 2 7-Footnote-61279720 +Ref: Porting to Python 2 7-Footnote-71279762 +Ref: Porting to Python 2 7-Footnote-81279804 +Ref: Porting to Python 2 7-Footnote-91279846 +Ref: Porting to Python 2 7-Footnote-101279888 +Ref: Porting to Python 2 7-Footnote-111279931 +Ref: Porting to Python 2 7-Footnote-121279974 +Ref: Porting to Python 2 7-Footnote-131280017 +Ref: Porting to Python 2 7-Footnote-141280067 +Node: New Features Added to Python 2 7 Maintenance Releases1280110 +Ref: whatsnew/2 7 new-features-added-to-python-2-7-maintenance-releases1280281 +Ref: cf81280281 +Ref: whatsnew/2 7 py27-maintenance-enhancements1280281 +Ref: c7c1280281 +Node: Two new environment variables for debug mode1281746 +Ref: whatsnew/2 7 two-new-environment-variables-for-debug-mode1281940 +Ref: cf91281940 +Ref: Two new environment variables for debug mode-Footnote-11282619 +Ref: Two new environment variables for debug mode-Footnote-21282662 +Node: PEP 434 IDLE Enhancement Exception for All Branches1282705 +Ref: whatsnew/2 7 pep-434-idle-enhancement-exception-for-all-branches1282960 +Ref: cfa1282960 +Ref: PEP 434 IDLE Enhancement Exception for All Branches-Footnote-11283480 +Node: PEP 466 Network Security Enhancements for Python 2 71283529 +Ref: whatsnew/2 7 pep-466-network-security-enhancements-for-python-2-71283788 +Ref: cfb1283788 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-11286017 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-21286066 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-31286115 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-41286158 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-51286201 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-61286250 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-71286293 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-81286336 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-91286385 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-101286428 +Ref: PEP 466 Network Security Enhancements for Python 2 7-Footnote-111286472 +Node: PEP 477 Backport ensurepip PEP 453 to Python 2 71286516 +Ref: whatsnew/2 7 pep-477-backport-ensurepip-pep-453-to-python-2-71286803 +Ref: cfe1286803 +Ref: PEP 477 Backport ensurepip PEP 453 to Python 2 7-Footnote-11287299 +Ref: PEP 477 Backport ensurepip PEP 453 to Python 2 7-Footnote-21287348 +Node: Bootstrapping pip By Default<2>1287397 +Ref: whatsnew/2 7 bootstrapping-pip-by-default1287546 +Ref: cff1287546 +Ref: Bootstrapping pip By Default<2>-Footnote-11289214 +Ref: Bootstrapping pip By Default<2>-Footnote-21289263 +Node: Documentation Changes<2>1289361 +Ref: whatsnew/2 7 documentation-changes1289510 +Ref: d001289510 +Ref: whatsnew/2 7 id11289510 +Ref: d011289510 +Ref: Documentation Changes<2>-Footnote-11290465 +Ref: Documentation Changes<2>-Footnote-21290501 +Node: PEP 476 Enabling certificate verification by default for stdlib http clients<2>1290550 +Ref: whatsnew/2 7 pep-476-enabling-certificate-verification-by-default-for-stdlib-http-clients1290842 +Ref: d021290842 +Ref: PEP 476 Enabling certificate verification by default for stdlib http clients<2>-Footnote-11291923 +Node: PEP 493 HTTPS verification migration tools for Python 2 71291972 +Ref: whatsnew/2 7 pep-493-https-verification-migration-tools-for-python-2-71292250 +Ref: d031292250 +Ref: PEP 493 HTTPS verification migration tools for Python 2 7-Footnote-11293414 +Node: New make regen-all build target<3>1293463 +Ref: whatsnew/2 7 new-make-regen-all-build-target1293699 +Ref: d041293699 +Ref: New make regen-all build target<3>-Footnote-11294489 +Ref: New make regen-all build target<3>-Footnote-21294556 +Node: Removal of make touch build target<3>1294599 +Ref: whatsnew/2 7 removal-of-make-touch-build-target1294769 +Ref: d051294769 +Ref: Removal of make touch build target<3>-Footnote-11295204 +Node: Acknowledgements1295247 +Ref: whatsnew/2 7 acknowledgements1295388 +Ref: d061295388 +Ref: whatsnew/2 7 acks271295388 +Ref: d071295388 +Node: What’s New in Python 2 61295661 +Ref: whatsnew/2 6 doc1295816 +Ref: d081295816 +Ref: whatsnew/2 6 what-s-new-in-python-2-61295816 +Ref: d091295816 +Ref: whatsnew/2 6 whats-new-in-2-61295816 +Ref: c471295816 +Ref: What’s New in Python 2 6-Footnote-11298879 +Node: Python 3 01298928 +Ref: whatsnew/2 6 python-3-01299044 +Ref: d0a1299044 +Ref: Python 3 0-Footnote-11301158 +Ref: Python 3 0-Footnote-21301207 +Node: Changes to the Development Process1301256 +Ref: whatsnew/2 6 changes-to-the-development-process1301413 +Ref: d0b1301413 +Node: New Issue Tracker Roundup1301931 +Ref: whatsnew/2 6 new-issue-tracker-roundup1302090 +Ref: d0c1302090 +Ref: New Issue Tracker Roundup-Footnote-11304137 +Ref: New Issue Tracker Roundup-Footnote-21304186 +Ref: New Issue Tracker Roundup-Footnote-31304217 +Ref: New Issue Tracker Roundup-Footnote-41304257 +Ref: New Issue Tracker Roundup-Footnote-51304292 +Node: New Documentation Format reStructuredText Using Sphinx1304333 +Ref: whatsnew/2 6 new-documentation-format-restructuredtext-using-sphinx1304492 +Ref: d0d1304492 +Ref: New Documentation Format reStructuredText Using Sphinx-Footnote-11306602 +Ref: New Documentation Format reStructuredText Using Sphinx-Footnote-21306661 +Ref: New Documentation Format reStructuredText Using Sphinx-Footnote-31306710 +Ref: New Documentation Format reStructuredText Using Sphinx-Footnote-41306741 +Node: PEP 343 The ‘with’ statement1306781 +Ref: whatsnew/2 6 pep-03431306980 +Ref: c481306980 +Ref: whatsnew/2 6 pep-343-the-with-statement1306980 +Ref: d0e1306980 +Node: Writing Context Managers1310256 +Ref: whatsnew/2 6 new-26-context-managers1310379 +Ref: c491310379 +Ref: whatsnew/2 6 writing-context-managers1310379 +Ref: d0f1310379 +Node: The contextlib module1314892 +Ref: whatsnew/2 6 new-module-contextlib1315015 +Ref: c4a1315015 +Ref: whatsnew/2 6 the-contextlib-module1315015 +Ref: d101315015 +Ref: The contextlib module-Footnote-11317517 +Node: PEP 366 Explicit Relative Imports From a Main Module1317566 +Ref: whatsnew/2 6 pep-03661317771 +Ref: c4b1317771 +Ref: whatsnew/2 6 pep-366-explicit-relative-imports-from-a-main-module1317771 +Ref: d111317771 +Node: PEP 370 Per-user site-packages Directory1318534 +Ref: whatsnew/2 6 pep-03701318742 +Ref: c4c1318742 +Ref: whatsnew/2 6 pep-370-per-user-site-packages-directory1318742 +Ref: d131318742 +Ref: PEP 370 Per-user site-packages Directory-Footnote-11320201 +Node: PEP 371 The multiprocessing Package1320250 +Ref: whatsnew/2 6 pep-03711320441 +Ref: c4d1320441 +Ref: whatsnew/2 6 pep-371-the-multiprocessing-package1320441 +Ref: d171320441 +Ref: PEP 371 The multiprocessing Package-Footnote-11325580 +Node: PEP 3101 Advanced String Formatting1325629 +Ref: whatsnew/2 6 pep-31011325808 +Ref: c4e1325808 +Ref: whatsnew/2 6 pep-3101-advanced-string-formatting1325808 +Ref: d191325808 +Ref: PEP 3101 Advanced String Formatting-Footnote-11331521 +Node: PEP 3105 print As a Function1331570 +Ref: whatsnew/2 6 pep-31051331749 +Ref: c4f1331749 +Ref: whatsnew/2 6 pep-3105-print-as-a-function1331749 +Ref: d1b1331749 +Ref: PEP 3105 print As a Function-Footnote-11332903 +Node: PEP 3110 Exception-Handling Changes1332952 +Ref: whatsnew/2 6 pep-31101333118 +Ref: c501333118 +Ref: whatsnew/2 6 pep-3110-exception-handling-changes1333118 +Ref: d1c1333118 +Ref: PEP 3110 Exception-Handling Changes-Footnote-11334719 +Node: PEP 3112 Byte Literals1334768 +Ref: whatsnew/2 6 pep-31121334930 +Ref: c511334930 +Ref: whatsnew/2 6 pep-3112-byte-literals1334930 +Ref: d1d1334930 +Ref: PEP 3112 Byte Literals-Footnote-11337944 +Node: PEP 3116 New I/O Library1337993 +Ref: whatsnew/2 6 pep-31161338152 +Ref: c521338152 +Ref: whatsnew/2 6 pep-3116-new-i-o-library1338152 +Ref: d231338152 +Ref: PEP 3116 New I/O Library-Footnote-11341778 +Node: PEP 3118 Revised Buffer Protocol1341827 +Ref: whatsnew/2 6 pep-31181341994 +Ref: c531341994 +Ref: whatsnew/2 6 pep-3118-revised-buffer-protocol1341994 +Ref: d241341994 +Ref: PEP 3118 Revised Buffer Protocol-Footnote-11344133 +Node: PEP 3119 Abstract Base Classes1344182 +Ref: whatsnew/2 6 pep-31191344368 +Ref: c541344368 +Ref: whatsnew/2 6 pep-3119-abstract-base-classes1344368 +Ref: d261344368 +Ref: PEP 3119 Abstract Base Classes-Footnote-11349737 +Node: PEP 3127 Integer Literal Support and Syntax1349786 +Ref: whatsnew/2 6 pep-31271349965 +Ref: c551349965 +Ref: whatsnew/2 6 pep-3127-integer-literal-support-and-syntax1349965 +Ref: d291349965 +Ref: PEP 3127 Integer Literal Support and Syntax-Footnote-11351330 +Node: PEP 3129 Class Decorators1351379 +Ref: whatsnew/2 6 pep-31291351565 +Ref: c561351565 +Ref: whatsnew/2 6 pep-3129-class-decorators1351565 +Ref: d2a1351565 +Ref: PEP 3129 Class Decorators-Footnote-11351965 +Node: PEP 3141 A Type Hierarchy for Numbers1352014 +Ref: whatsnew/2 6 pep-31411352183 +Ref: c571352183 +Ref: whatsnew/2 6 pep-3141-a-type-hierarchy-for-numbers1352183 +Ref: d2b1352183 +Ref: PEP 3141 A Type Hierarchy for Numbers-Footnote-11354384 +Ref: PEP 3141 A Type Hierarchy for Numbers-Footnote-21354433 +Ref: PEP 3141 A Type Hierarchy for Numbers-Footnote-31354531 +Node: The fractions Module1354616 +Ref: whatsnew/2 6 the-fractions-module1354710 +Ref: d2f1354710 +Node: Other Language Changes<10>1356317 +Ref: whatsnew/2 6 other-language-changes1356488 +Ref: d301356488 +Ref: Other Language Changes<10>-Footnote-11365337 +Ref: Other Language Changes<10>-Footnote-21365382 +Ref: Other Language Changes<10>-Footnote-31365424 +Ref: Other Language Changes<10>-Footnote-41365469 +Ref: Other Language Changes<10>-Footnote-51365511 +Ref: Other Language Changes<10>-Footnote-61365553 +Ref: Other Language Changes<10>-Footnote-71365595 +Ref: Other Language Changes<10>-Footnote-81365637 +Ref: Other Language Changes<10>-Footnote-91365679 +Ref: Other Language Changes<10>-Footnote-101365721 +Ref: Other Language Changes<10>-Footnote-111365764 +Ref: Other Language Changes<10>-Footnote-121365807 +Ref: Other Language Changes<10>-Footnote-131365853 +Ref: Other Language Changes<10>-Footnote-141365899 +Ref: Other Language Changes<10>-Footnote-151365945 +Ref: Other Language Changes<10>-Footnote-161365991 +Node: Optimizations<9>1366037 +Ref: whatsnew/2 6 optimizations1366147 +Ref: d341366147 +Ref: Optimizations<9>-Footnote-11369003 +Ref: Optimizations<9>-Footnote-21369048 +Ref: Optimizations<9>-Footnote-31369093 +Ref: Optimizations<9>-Footnote-41369135 +Node: Interpreter Changes<2>1369177 +Ref: whatsnew/2 6 interpreter-changes1369287 +Ref: d351369287 +Ref: whatsnew/2 6 new-26-interpreter1369287 +Ref: d361369287 +Node: New and Improved Modules<2>1370873 +Ref: whatsnew/2 6 new-and-improved-modules1371032 +Ref: d3a1371032 +Ref: New and Improved Modules<2>-Footnote-11413464 +Ref: New and Improved Modules<2>-Footnote-21413509 +Ref: New and Improved Modules<2>-Footnote-31413562 +Ref: New and Improved Modules<2>-Footnote-41413604 +Ref: New and Improved Modules<2>-Footnote-51413648 +Ref: New and Improved Modules<2>-Footnote-61413690 +Ref: New and Improved Modules<2>-Footnote-71413735 +Ref: New and Improved Modules<2>-Footnote-81413777 +Ref: New and Improved Modules<2>-Footnote-91413830 +Ref: New and Improved Modules<2>-Footnote-101413875 +Ref: New and Improved Modules<2>-Footnote-111413921 +Ref: New and Improved Modules<2>-Footnote-121413967 +Ref: New and Improved Modules<2>-Footnote-131414010 +Ref: New and Improved Modules<2>-Footnote-141414053 +Ref: New and Improved Modules<2>-Footnote-151414098 +Ref: New and Improved Modules<2>-Footnote-161414141 +Ref: New and Improved Modules<2>-Footnote-171414187 +Ref: New and Improved Modules<2>-Footnote-181414230 +Ref: New and Improved Modules<2>-Footnote-191414276 +Ref: New and Improved Modules<2>-Footnote-201414322 +Ref: New and Improved Modules<2>-Footnote-211414368 +Ref: New and Improved Modules<2>-Footnote-221414413 +Ref: New and Improved Modules<2>-Footnote-231414459 +Ref: New and Improved Modules<2>-Footnote-241414505 +Ref: New and Improved Modules<2>-Footnote-251414548 +Ref: New and Improved Modules<2>-Footnote-261414591 +Ref: New and Improved Modules<2>-Footnote-271414637 +Ref: New and Improved Modules<2>-Footnote-281414683 +Ref: New and Improved Modules<2>-Footnote-291414728 +Ref: New and Improved Modules<2>-Footnote-301414771 +Ref: New and Improved Modules<2>-Footnote-311414814 +Ref: New and Improved Modules<2>-Footnote-321414857 +Ref: New and Improved Modules<2>-Footnote-331414900 +Ref: New and Improved Modules<2>-Footnote-341414943 +Ref: New and Improved Modules<2>-Footnote-351414986 +Ref: New and Improved Modules<2>-Footnote-361415029 +Ref: New and Improved Modules<2>-Footnote-371415079 +Ref: New and Improved Modules<2>-Footnote-381415124 +Ref: New and Improved Modules<2>-Footnote-391415174 +Ref: New and Improved Modules<2>-Footnote-401415219 +Ref: New and Improved Modules<2>-Footnote-411415262 +Ref: New and Improved Modules<2>-Footnote-421415307 +Ref: New and Improved Modules<2>-Footnote-431415353 +Ref: New and Improved Modules<2>-Footnote-441415396 +Ref: New and Improved Modules<2>-Footnote-451415439 +Ref: New and Improved Modules<2>-Footnote-461415482 +Ref: New and Improved Modules<2>-Footnote-471415528 +Ref: New and Improved Modules<2>-Footnote-481415571 +Ref: New and Improved Modules<2>-Footnote-491415614 +Ref: New and Improved Modules<2>-Footnote-501415660 +Ref: New and Improved Modules<2>-Footnote-511415703 +Ref: New and Improved Modules<2>-Footnote-521415749 +Ref: New and Improved Modules<2>-Footnote-531415792 +Ref: New and Improved Modules<2>-Footnote-541415838 +Ref: New and Improved Modules<2>-Footnote-551415881 +Ref: New and Improved Modules<2>-Footnote-561415927 +Ref: New and Improved Modules<2>-Footnote-571415970 +Ref: New and Improved Modules<2>-Footnote-581416016 +Ref: New and Improved Modules<2>-Footnote-591416062 +Ref: New and Improved Modules<2>-Footnote-601416105 +Ref: New and Improved Modules<2>-Footnote-611416148 +Ref: New and Improved Modules<2>-Footnote-621416193 +Ref: New and Improved Modules<2>-Footnote-631416239 +Node: The ast module1416285 +Ref: whatsnew/2 6 the-ast-module1416398 +Ref: d4a1416398 +Node: The future_builtins module1419059 +Ref: whatsnew/2 6 the-future-builtins-module1419223 +Ref: d4b1419223 +Node: The json module JavaScript Object Notation1420338 +Ref: whatsnew/2 6 the-json-module-javascript-object-notation1420530 +Ref: d4d1420530 +Node: The plistlib module A Property-List Parser1421542 +Ref: whatsnew/2 6 the-plistlib-module-a-property-list-parser1421727 +Ref: d4e1421727 +Node: ctypes Enhancements1423098 +Ref: whatsnew/2 6 ctypes-enhancements1423261 +Ref: d4f1423261 +Ref: ctypes Enhancements-Footnote-11425028 +Ref: ctypes Enhancements-Footnote-21425073 +Node: Improved SSL Support1425115 +Ref: whatsnew/2 6 improved-ssl-support1425227 +Ref: d501425227 +Ref: Improved SSL Support-Footnote-11426180 +Node: Deprecations and Removals1426213 +Ref: whatsnew/2 6 deprecations-and-removals1426372 +Ref: d511426372 +Ref: Deprecations and Removals-Footnote-11428227 +Node: Build and C API Changes<9>1428276 +Ref: whatsnew/2 6 build-and-c-api-changes1428429 +Ref: d531428429 +Ref: Build and C API Changes<9>-Footnote-11434466 +Ref: Build and C API Changes<9>-Footnote-21434508 +Ref: Build and C API Changes<9>-Footnote-31434553 +Ref: Build and C API Changes<9>-Footnote-41434595 +Ref: Build and C API Changes<9>-Footnote-51434637 +Node: Port-Specific Changes Windows<2>1434682 +Ref: whatsnew/2 6 port-specific-changes-windows1434819 +Ref: d5a1434819 +Ref: Port-Specific Changes Windows<2>-Footnote-11437229 +Ref: Port-Specific Changes Windows<2>-Footnote-21437273 +Ref: Port-Specific Changes Windows<2>-Footnote-31437318 +Node: Port-Specific Changes Mac OS X<2>1437360 +Ref: whatsnew/2 6 port-specific-changes-mac-os-x1437532 +Ref: d5b1437532 +Ref: Port-Specific Changes Mac OS X<2>-Footnote-11438803 +Node: Port-Specific Changes IRIX1438848 +Ref: whatsnew/2 6 port-specific-changes-irix1438979 +Ref: d5c1438979 +Node: Porting to Python 2 61439498 +Ref: whatsnew/2 6 porting-to-python-2-61439645 +Ref: d5d1439645 +Ref: Porting to Python 2 6-Footnote-11443070 +Ref: Porting to Python 2 6-Footnote-21443115 +Ref: Porting to Python 2 6-Footnote-31443160 +Node: Acknowledgements<2>1443205 +Ref: whatsnew/2 6 acknowledgements1443317 +Ref: d601443317 +Ref: whatsnew/2 6 acks1443317 +Ref: d611443317 +Node: What’s New in Python 2 51443659 +Ref: whatsnew/2 5 doc1443814 +Ref: d621443814 +Ref: whatsnew/2 5 what-s-new-in-python-2-51443814 +Ref: d631443814 +Ref: What’s New in Python 2 5-Footnote-11447417 +Node: PEP 308 Conditional Expressions1447466 +Ref: whatsnew/2 5 pep-3081447605 +Ref: d641447605 +Ref: whatsnew/2 5 pep-308-conditional-expressions1447605 +Ref: d691447605 +Ref: PEP 308 Conditional Expressions-Footnote-11451089 +Ref: PEP 308 Conditional Expressions-Footnote-21451138 +Node: PEP 309 Partial Function Application1451187 +Ref: whatsnew/2 5 pep-3091451385 +Ref: d6a1451385 +Ref: whatsnew/2 5 pep-309-partial-function-application1451385 +Ref: d6b1451385 +Ref: PEP 309 Partial Function Application-Footnote-11454232 +Node: PEP 314 Metadata for Python Software Packages v1 11454281 +Ref: whatsnew/2 5 pep-3141454485 +Ref: d6c1454485 +Ref: whatsnew/2 5 pep-314-metadata-for-python-software-packages-v1-11454485 +Ref: d6d1454485 +Ref: PEP 314 Metadata for Python Software Packages v1 1-Footnote-11456319 +Node: PEP 328 Absolute and Relative Imports1456368 +Ref: whatsnew/2 5 pep-3281456572 +Ref: d671456572 +Ref: whatsnew/2 5 pep-328-absolute-and-relative-imports1456572 +Ref: d6e1456572 +Ref: PEP 328 Absolute and Relative Imports-Footnote-11460384 +Ref: PEP 328 Absolute and Relative Imports-Footnote-21460433 +Node: PEP 338 Executing Modules as Scripts1460482 +Ref: whatsnew/2 5 pep-3381460670 +Ref: d6f1460670 +Ref: whatsnew/2 5 pep-338-executing-modules-as-scripts1460670 +Ref: d701460670 +Ref: PEP 338 Executing Modules as Scripts-Footnote-11461565 +Node: PEP 341 Unified try/except/finally1461614 +Ref: whatsnew/2 5 pep-3411461795 +Ref: d681461795 +Ref: whatsnew/2 5 pep-341-unified-try-except-finally1461795 +Ref: d711461795 +Ref: PEP 341 Unified try/except/finally-Footnote-11463562 +Node: PEP 342 New Generator Features1463611 +Ref: whatsnew/2 5 pep-3421463791 +Ref: d661463791 +Ref: whatsnew/2 5 pep-342-new-generator-features1463791 +Ref: d731463791 +Ref: PEP 342 New Generator Features-Footnote-11470408 +Ref: PEP 342 New Generator Features-Footnote-21470457 +Ref: PEP 342 New Generator Features-Footnote-31470506 +Ref: PEP 342 New Generator Features-Footnote-41470555 +Ref: PEP 342 New Generator Features-Footnote-51470604 +Node: PEP 343 The ‘with’ statement<2>1470653 +Ref: whatsnew/2 5 pep-3431470838 +Ref: d651470838 +Ref: whatsnew/2 5 pep-343-the-with-statement1470838 +Ref: d741470838 +Node: Writing Context Managers<2>1473858 +Ref: whatsnew/2 5 new-25-context-managers1473990 +Ref: d751473990 +Ref: whatsnew/2 5 writing-context-managers1473990 +Ref: d761473990 +Node: The contextlib module<2>1478386 +Ref: whatsnew/2 5 contextlibmod1478518 +Ref: d771478518 +Ref: whatsnew/2 5 the-contextlib-module1478518 +Ref: d781478518 +Ref: The contextlib module<2>-Footnote-11481011 +Node: PEP 352 Exceptions as New-Style Classes1481060 +Ref: whatsnew/2 5 pep-3521481254 +Ref: d791481254 +Ref: whatsnew/2 5 pep-352-exceptions-as-new-style-classes1481254 +Ref: d7a1481254 +Ref: PEP 352 Exceptions as New-Style Classes-Footnote-11483736 +Node: PEP 353 Using ssize_t as the index type1483785 +Ref: whatsnew/2 5 pep-3531483978 +Ref: d7c1483978 +Ref: whatsnew/2 5 pep-353-using-ssize-t-as-the-index-type1483978 +Ref: d7d1483978 +Ref: PEP 353 Using ssize_t as the index type-Footnote-11486890 +Ref: PEP 353 Using ssize_t as the index type-Footnote-21486939 +Node: PEP 357 The ‘__index__’ method1486988 +Ref: whatsnew/2 5 pep-3571487168 +Ref: d7f1487168 +Ref: whatsnew/2 5 pep-357-the-index-method1487168 +Ref: d801487168 +Ref: PEP 357 The ‘__index__’ method-Footnote-11488896 +Node: Other Language Changes<11>1488945 +Ref: whatsnew/2 5 other-lang1489118 +Ref: d821489118 +Ref: whatsnew/2 5 other-language-changes1489118 +Ref: d831489118 +Ref: Other Language Changes<11>-Footnote-11496071 +Node: Interactive Interpreter Changes1496120 +Ref: whatsnew/2 5 interactive1496240 +Ref: d891496240 +Ref: whatsnew/2 5 interactive-interpreter-changes1496240 +Ref: d8a1496240 +Node: Optimizations<10>1497020 +Ref: whatsnew/2 5 optimizations1497140 +Ref: d8d1497140 +Ref: whatsnew/2 5 opts1497140 +Ref: d8e1497140 +Node: New Improved and Removed Modules1500958 +Ref: whatsnew/2 5 modules1501124 +Ref: d8f1501124 +Ref: whatsnew/2 5 new-improved-and-removed-modules1501124 +Ref: d901501124 +Ref: New Improved and Removed Modules-Footnote-11525616 +Ref: New Improved and Removed Modules-Footnote-21525665 +Ref: New Improved and Removed Modules-Footnote-31525714 +Node: The ctypes package1525763 +Ref: whatsnew/2 5 module-ctypes1525882 +Ref: d9d1525882 +Ref: whatsnew/2 5 the-ctypes-package1525882 +Ref: d9e1525882 +Node: The ElementTree package1529099 +Ref: whatsnew/2 5 module-etree1529246 +Ref: d9f1529246 +Ref: whatsnew/2 5 the-elementtree-package1529246 +Ref: da01529246 +Node: The hashlib package1534458 +Ref: whatsnew/2 5 module-hashlib1534606 +Ref: da11534606 +Ref: whatsnew/2 5 the-hashlib-package1534606 +Ref: da21534606 +Node: The sqlite3 package1536191 +Ref: whatsnew/2 5 module-sqlite1536335 +Ref: da31536335 +Ref: whatsnew/2 5 the-sqlite3-package1536335 +Ref: da41536335 +Ref: The sqlite3 package-Footnote-11540344 +Ref: The sqlite3 package-Footnote-21540393 +Node: The wsgiref package1540442 +Ref: whatsnew/2 5 module-wsgiref1540558 +Ref: da51540558 +Ref: whatsnew/2 5 the-wsgiref-package1540558 +Ref: da61540558 +Ref: The wsgiref package-Footnote-11541479 +Ref: The wsgiref package-Footnote-21541528 +Node: Build and C API Changes<10>1541577 +Ref: whatsnew/2 5 build-and-c-api-changes1541738 +Ref: da71541738 +Ref: whatsnew/2 5 build-api1541738 +Ref: da81541738 +Ref: Build and C API Changes<10>-Footnote-11548757 +Ref: Build and C API Changes<10>-Footnote-21548806 +Ref: Build and C API Changes<10>-Footnote-31548855 +Node: Port-Specific Changes1548904 +Ref: whatsnew/2 5 port-specific-changes1548989 +Ref: db21548989 +Ref: whatsnew/2 5 ports1548989 +Ref: db31548989 +Ref: Port-Specific Changes-Footnote-11549666 +Node: Porting to Python 2 51549708 +Ref: whatsnew/2 5 porting1549856 +Ref: db41549856 +Ref: whatsnew/2 5 porting-to-python-2-51549856 +Ref: db51549856 +Ref: Porting to Python 2 5-Footnote-11552679 +Node: Acknowledgements<3>1552728 +Ref: whatsnew/2 5 acknowledgements1552840 +Ref: db61552840 +Node: What’s New in Python 2 41553334 +Ref: whatsnew/2 4 doc1553489 +Ref: db71553489 +Ref: whatsnew/2 4 what-s-new-in-python-2-41553489 +Ref: db81553489 +Node: PEP 218 Built-In Set Objects1555594 +Ref: whatsnew/2 4 pep-218-built-in-set-objects1555737 +Ref: db91555737 +Ref: PEP 218 Built-In Set Objects-Footnote-11557973 +Node: PEP 237 Unifying Long Integers and Integers1558022 +Ref: whatsnew/2 4 pep-237-unifying-long-integers-and-integers1558203 +Ref: dba1558203 +Ref: PEP 237 Unifying Long Integers and Integers-Footnote-11559244 +Node: PEP 289 Generator Expressions1559293 +Ref: whatsnew/2 4 pep-289-generator-expressions1559482 +Ref: dbb1559482 +Ref: PEP 289 Generator Expressions-Footnote-11561754 +Node: PEP 292 Simpler String Substitutions1561803 +Ref: whatsnew/2 4 pep-292-simpler-string-substitutions1561993 +Ref: dbc1561993 +Ref: PEP 292 Simpler String Substitutions-Footnote-11563792 +Node: PEP 318 Decorators for Functions and Methods1563841 +Ref: whatsnew/2 4 pep-318-decorators-for-functions-and-methods1564027 +Ref: dbd1564027 +Ref: PEP 318 Decorators for Functions and Methods-Footnote-11568548 +Ref: PEP 318 Decorators for Functions and Methods-Footnote-21568597 +Node: PEP 322 Reverse Iteration1568646 +Ref: whatsnew/2 4 pep-322-reverse-iteration1568825 +Ref: dbf1568825 +Ref: PEP 322 Reverse Iteration-Footnote-11569787 +Node: PEP 324 New subprocess Module1569836 +Ref: whatsnew/2 4 pep-324-new-subprocess-module1569996 +Ref: dc01569996 +Ref: PEP 324 New subprocess Module-Footnote-11573760 +Node: PEP 327 Decimal Data Type1573809 +Ref: whatsnew/2 4 pep-327-decimal-data-type1573970 +Ref: dc21573970 +Node: Why is Decimal needed?1574548 +Ref: whatsnew/2 4 why-is-decimal-needed1574657 +Ref: dc31574657 +Node: The Decimal type1577247 +Ref: whatsnew/2 4 the-decimal-type1577381 +Ref: dc41577381 +Node: The Context type1580553 +Ref: whatsnew/2 4 the-context-type1580656 +Ref: dc51580656 +Ref: The Context type-Footnote-11583265 +Node: PEP 328 Multi-line Imports1583314 +Ref: whatsnew/2 4 pep-328-multi-line-imports1583497 +Ref: dc61583497 +Ref: PEP 328 Multi-line Imports-Footnote-11584991 +Node: PEP 331 Locale-Independent Float/String Conversions1585040 +Ref: whatsnew/2 4 pep-331-locale-independent-float-string-conversions1585224 +Ref: dc71585224 +Ref: PEP 331 Locale-Independent Float/String Conversions-Footnote-11586964 +Node: Other Language Changes<12>1587013 +Ref: whatsnew/2 4 other-language-changes1587209 +Ref: dc81587209 +Ref: Other Language Changes<12>-Footnote-11594191 +Ref: Other Language Changes<12>-Footnote-21594240 +Ref: Other Language Changes<12>-Footnote-31594289 +Ref: Other Language Changes<12>-Footnote-41594338 +Ref: Other Language Changes<12>-Footnote-51594387 +Ref: Other Language Changes<12>-Footnote-61594436 +Node: Optimizations<11>1594485 +Ref: whatsnew/2 4 optimizations1594565 +Ref: dca1594565 +Node: New Improved and Deprecated Modules<3>1597320 +Ref: whatsnew/2 4 new-improved-and-deprecated-modules1597492 +Ref: dcc1597492 +Ref: New Improved and Deprecated Modules<3>-Footnote-11614779 +Node: cookielib1614828 +Ref: whatsnew/2 4 cookielib1614931 +Ref: dce1614931 +Node: doctest<3>1615850 +Ref: whatsnew/2 4 doctest1615953 +Ref: dcf1615953 +Node: Build and C API Changes<11>1618687 +Ref: whatsnew/2 4 build-and-c-api-changes1618854 +Ref: dd51618854 +Node: Port-Specific Changes<2>1621480 +Ref: whatsnew/2 4 port-specific-changes1621568 +Ref: ddc1621568 +Node: Porting to Python 2 41621747 +Ref: whatsnew/2 4 porting-to-python-2-41621895 +Ref: ddd1621895 +Node: Acknowledgements<4>1623863 +Ref: whatsnew/2 4 acknowledgements1623975 +Ref: ddf1623975 +Ref: whatsnew/2 4 acks1623975 +Ref: de01623975 +Node: What’s New in Python 2 31624308 +Ref: whatsnew/2 3 doc1624463 +Ref: de11624463 +Ref: whatsnew/2 3 what-s-new-in-python-2-31624463 +Ref: de21624463 +Node: PEP 218 A Standard Set Datatype1627481 +Ref: whatsnew/2 3 pep-218-a-standard-set-datatype1627609 +Ref: de41627609 +Ref: PEP 218 A Standard Set Datatype-Footnote-11630100 +Node: PEP 255 Simple Generators1630149 +Ref: whatsnew/2 3 pep-255-simple-generators1630315 +Ref: de51630315 +Ref: whatsnew/2 3 section-generators1630315 +Ref: de61630315 +Ref: PEP 255 Simple Generators-Footnote-11636497 +Ref: PEP 255 Simple Generators-Footnote-21636546 +Node: PEP 263 Source Code Encodings1636595 +Ref: whatsnew/2 3 pep-263-source-code-encodings1636773 +Ref: de81636773 +Ref: whatsnew/2 3 section-encodings1636773 +Ref: de91636773 +Ref: PEP 263 Source Code Encodings-Footnote-11637985 +Node: PEP 273 Importing Modules from ZIP Archives1638034 +Ref: whatsnew/2 3 pep-273-importing-modules-from-zip-archives1638235 +Ref: dea1638235 +Ref: PEP 273 Importing Modules from ZIP Archives-Footnote-11640268 +Ref: PEP 273 Importing Modules from ZIP Archives-Footnote-21640317 +Ref: PEP 273 Importing Modules from ZIP Archives-Footnote-31640366 +Node: PEP 277 Unicode file name support for Windows NT1640415 +Ref: whatsnew/2 3 pep-277-unicode-file-name-support-for-windows-nt1640620 +Ref: dec1640620 +Ref: PEP 277 Unicode file name support for Windows NT-Footnote-11642098 +Node: PEP 278 Universal Newline Support1642147 +Ref: whatsnew/2 3 pep-278-universal-newline-support1642326 +Ref: dee1642326 +Ref: PEP 278 Universal Newline Support-Footnote-11643841 +Node: PEP 279 enumerate1643890 +Ref: whatsnew/2 3 pep-279-enumerate1644048 +Ref: def1644048 +Ref: whatsnew/2 3 section-enumerate1644048 +Ref: df01644048 +Ref: PEP 279 enumerate-Footnote-11644904 +Node: PEP 282 The logging Package1644953 +Ref: whatsnew/2 3 pep-282-the-logging-package1645100 +Ref: df11645100 +Ref: PEP 282 The logging Package-Footnote-11649794 +Ref: PEP 282 The logging Package-Footnote-21649843 +Node: PEP 285 A Boolean Type1649892 +Ref: whatsnew/2 3 pep-285-a-boolean-type1650060 +Ref: df21650060 +Ref: whatsnew/2 3 section-bool1650060 +Ref: df31650060 +Ref: PEP 285 A Boolean Type-Footnote-11652509 +Ref: PEP 285 A Boolean Type-Footnote-21652558 +Node: PEP 293 Codec Error Handling Callbacks1652607 +Ref: whatsnew/2 3 pep-293-codec-error-handling-callbacks1652796 +Ref: df41652796 +Ref: PEP 293 Codec Error Handling Callbacks-Footnote-11654345 +Node: PEP 301 Package Index and Metadata for Distutils1654394 +Ref: whatsnew/2 3 pep-301-package-index-and-metadata-for-distutils1654585 +Ref: df71654585 +Ref: whatsnew/2 3 section-pep3011654585 +Ref: df81654585 +Ref: PEP 301 Package Index and Metadata for Distutils-Footnote-11656229 +Ref: PEP 301 Package Index and Metadata for Distutils-Footnote-21656265 +Node: PEP 302 New Import Hooks1656314 +Ref: whatsnew/2 3 pep-302-new-import-hooks1656496 +Ref: df91656496 +Ref: whatsnew/2 3 section-pep3021656496 +Ref: deb1656496 +Ref: PEP 302 New Import Hooks-Footnote-11659103 +Ref: PEP 302 New Import Hooks-Footnote-21659152 +Ref: PEP 302 New Import Hooks-Footnote-31659201 +Node: PEP 305 Comma-separated Files1659250 +Ref: whatsnew/2 3 pep-305-comma-separated-files1659411 +Ref: dfa1659411 +Ref: whatsnew/2 3 section-pep3051659411 +Ref: dfb1659411 +Ref: PEP 305 Comma-separated Files-Footnote-11660887 +Node: PEP 307 Pickle Enhancements1660936 +Ref: whatsnew/2 3 pep-307-pickle-enhancements1661088 +Ref: dfd1661088 +Ref: whatsnew/2 3 section-pep3071661088 +Ref: dfe1661088 +Ref: PEP 307 Pickle Enhancements-Footnote-11663059 +Ref: PEP 307 Pickle Enhancements-Footnote-21663108 +Ref: PEP 307 Pickle Enhancements-Footnote-31663157 +Node: Extended Slices1663206 +Ref: whatsnew/2 3 extended-slices1663355 +Ref: e021663355 +Ref: whatsnew/2 3 section-slices1663355 +Ref: e031663355 +Node: Other Language Changes<13>1666747 +Ref: whatsnew/2 3 other-language-changes1666907 +Ref: e051666907 +Ref: Other Language Changes<13>-Footnote-11674691 +Ref: Other Language Changes<13>-Footnote-21674763 +Node: String Changes1674817 +Ref: whatsnew/2 3 string-changes1674920 +Ref: e0b1674920 +Node: Optimizations<12>1677373 +Ref: whatsnew/2 3 optimizations1677476 +Ref: e0c1677476 +Node: New Improved and Deprecated Modules<4>1678923 +Ref: whatsnew/2 3 new-improved-and-deprecated-modules1679107 +Ref: e0e1679107 +Ref: New Improved and Deprecated Modules<4>-Footnote-11703711 +Node: Date/Time Type1703750 +Ref: whatsnew/2 3 date-time-type1703867 +Ref: e121703867 +Node: The optparse Module1706131 +Ref: whatsnew/2 3 the-optparse-module1706248 +Ref: e131706248 +Node: Pymalloc A Specialized Object Allocator1708308 +Ref: whatsnew/2 3 pymalloc-a-specialized-object-allocator1708493 +Ref: e141708493 +Ref: whatsnew/2 3 section-pymalloc1708493 +Ref: e151708493 +Node: Build and C API Changes<12>1712395 +Ref: whatsnew/2 3 build-and-c-api-changes1712568 +Ref: e171712568 +Node: Port-Specific Changes<3>1715990 +Ref: whatsnew/2 3 port-specific-changes1716078 +Ref: e1c1716078 +Node: Other Changes and Fixes<2>1717313 +Ref: whatsnew/2 3 other-changes-and-fixes1717468 +Ref: e1d1717468 +Ref: whatsnew/2 3 section-other1717468 +Ref: e0d1717468 +Node: Porting to Python 2 31719958 +Ref: whatsnew/2 3 porting-to-python-2-31720105 +Ref: e201720105 +Node: Acknowledgements<5>1722913 +Ref: whatsnew/2 3 acknowledgements1723025 +Ref: e211723025 +Ref: whatsnew/2 3 acks1723025 +Ref: e221723025 +Node: What’s New in Python 2 21723646 +Ref: whatsnew/2 2 doc1723801 +Ref: e231723801 +Ref: whatsnew/2 2 what-s-new-in-python-2-21723801 +Ref: e241723801 +Node: Introduction1724536 +Ref: whatsnew/2 2 introduction1724659 +Ref: e251724659 +Ref: Introduction-Footnote-11725631 +Ref: Introduction-Footnote-21725680 +Node: PEPs 252 and 253 Type and Class Changes1725729 +Ref: whatsnew/2 2 peps-252-and-253-type-and-class-changes1725878 +Ref: e261725878 +Node: Old and New Classes1729310 +Ref: whatsnew/2 2 old-and-new-classes1729425 +Ref: e281729425 +Ref: Old and New Classes-Footnote-11731830 +Node: Descriptors1731879 +Ref: whatsnew/2 2 descriptors1732040 +Ref: e291732040 +Node: Multiple Inheritance The Diamond Rule1735900 +Ref: whatsnew/2 2 multiple-inheritance-the-diamond-rule1736058 +Ref: e2a1736058 +Ref: Multiple Inheritance The Diamond Rule-Footnote-11738943 +Node: Attribute Access1738992 +Ref: whatsnew/2 2 attribute-access1739152 +Ref: e2b1739152 +Node: Related Links1742969 +Ref: whatsnew/2 2 related-links1743083 +Ref: e2e1743083 +Ref: whatsnew/2 2 sect-rellinks1743083 +Ref: e271743083 +Ref: Related Links-Footnote-11744584 +Ref: Related Links-Footnote-21744633 +Ref: Related Links-Footnote-31744682 +Ref: Related Links-Footnote-41744731 +Ref: Related Links-Footnote-51744780 +Node: PEP 234 Iterators1744829 +Ref: whatsnew/2 2 pep-234-iterators1744994 +Ref: e2f1744994 +Ref: PEP 234 Iterators-Footnote-11749882 +Node: PEP 255 Simple Generators<2>1749931 +Ref: whatsnew/2 2 pep-255-simple-generators1750103 +Ref: e321750103 +Ref: PEP 255 Simple Generators<2>-Footnote-11756160 +Ref: PEP 255 Simple Generators<2>-Footnote-21756209 +Node: PEP 237 Unifying Long Integers and Integers<2>1756258 +Ref: whatsnew/2 2 pep-237-unifying-long-integers-and-integers1756451 +Ref: e331756451 +Ref: PEP 237 Unifying Long Integers and Integers<2>-Footnote-11758176 +Node: PEP 238 Changing the Division Operator1758225 +Ref: whatsnew/2 2 pep-238-changing-the-division-operator1758405 +Ref: e341758405 +Ref: PEP 238 Changing the Division Operator-Footnote-11762057 +Ref: PEP 238 Changing the Division Operator-Footnote-21762106 +Ref: PEP 238 Changing the Division Operator-Footnote-31762155 +Node: Unicode Changes1762204 +Ref: whatsnew/2 2 unicode-changes1762359 +Ref: e361762359 +Ref: Unicode Changes-Footnote-11764996 +Ref: Unicode Changes-Footnote-21765045 +Node: PEP 227 Nested Scopes1765094 +Ref: whatsnew/2 2 pep-227-nested-scopes1765238 +Ref: e381765238 +Ref: PEP 227 Nested Scopes-Footnote-11769580 +Node: New and Improved Modules<3>1769629 +Ref: whatsnew/2 2 new-and-improved-modules1769787 +Ref: e391769787 +Ref: New and Improved Modules<3>-Footnote-11776417 +Ref: New and Improved Modules<3>-Footnote-21776466 +Ref: New and Improved Modules<3>-Footnote-31776515 +Ref: New and Improved Modules<3>-Footnote-41776564 +Ref: New and Improved Modules<3>-Footnote-51776613 +Node: Interpreter Changes and Fixes1776661 +Ref: whatsnew/2 2 interpreter-changes-and-fixes1776824 +Ref: e3a1776824 +Node: Other Changes and Fixes<3>1781581 +Ref: whatsnew/2 2 other-changes-and-fixes1781736 +Ref: e4c1781736 +Ref: Other Changes and Fixes<3>-Footnote-11788402 +Node: Acknowledgements<6>1788451 +Ref: whatsnew/2 2 acknowledgements1788568 +Ref: e4f1788568 +Node: What’s New in Python 2 11789169 +Ref: whatsnew/2 1 doc1789324 +Ref: e501789324 +Ref: whatsnew/2 1 what-s-new-in-python-2-11789324 +Ref: e511789324 +Node: Introduction<2>1790329 +Ref: whatsnew/2 1 introduction1790440 +Ref: e521790440 +Node: PEP 227 Nested Scopes<2>1791536 +Ref: whatsnew/2 1 pep-227-nested-scopes1791685 +Ref: e531791685 +Ref: PEP 227 Nested Scopes<2>-Footnote-11796046 +Ref: PEP 227 Nested Scopes<2>-Footnote-21796095 +Ref: PEP 227 Nested Scopes<2>-Footnote-31796144 +Node: PEP 236 __future__ Directives1796193 +Ref: whatsnew/2 1 pep-236-future-directives1796351 +Ref: e541796351 +Ref: PEP 236 __future__ Directives-Footnote-11797588 +Node: PEP 207 Rich Comparisons1797637 +Ref: whatsnew/2 1 pep-207-rich-comparisons1797796 +Ref: e551797796 +Ref: PEP 207 Rich Comparisons-Footnote-11801191 +Ref: PEP 207 Rich Comparisons-Footnote-21801240 +Node: PEP 230 Warning Framework1801289 +Ref: whatsnew/2 1 pep-230-warning-framework1801443 +Ref: e571801443 +Ref: PEP 230 Warning Framework-Footnote-11804723 +Ref: PEP 230 Warning Framework-Footnote-21804772 +Node: PEP 229 New Build System1804821 +Ref: whatsnew/2 1 pep-229-new-build-system1804974 +Ref: e591804974 +Ref: PEP 229 New Build System-Footnote-11806894 +Node: PEP 205 Weak References1806943 +Ref: whatsnew/2 1 pep-205-weak-references1807098 +Ref: e5a1807098 +Ref: PEP 205 Weak References-Footnote-11810211 +Node: PEP 232 Function Attributes1810260 +Ref: whatsnew/2 1 pep-232-function-attributes1810446 +Ref: e5b1810446 +Ref: PEP 232 Function Attributes-Footnote-11811957 +Node: PEP 235 Importing Modules on Case-Insensitive Platforms1812006 +Ref: whatsnew/2 1 pep-235-importing-modules-on-case-insensitive-platforms1812201 +Ref: e5c1812201 +Node: PEP 217 Interactive Display Hook1813104 +Ref: whatsnew/2 1 pep-217-interactive-display-hook1813298 +Ref: e5e1813298 +Ref: PEP 217 Interactive Display Hook-Footnote-11814226 +Node: PEP 208 New Coercion Model1814275 +Ref: whatsnew/2 1 pep-208-new-coercion-model1814449 +Ref: e601814449 +Ref: PEP 208 New Coercion Model-Footnote-11816071 +Node: PEP 241 Metadata in Python Packages1816120 +Ref: whatsnew/2 1 pep-241-metadata-in-python-packages1816289 +Ref: e611816289 +Ref: PEP 241 Metadata in Python Packages-Footnote-11818332 +Ref: PEP 241 Metadata in Python Packages-Footnote-21818381 +Ref: PEP 241 Metadata in Python Packages-Footnote-31818430 +Ref: PEP 241 Metadata in Python Packages-Footnote-41818479 +Node: New and Improved Modules<4>1818528 +Ref: whatsnew/2 1 new-and-improved-modules1818697 +Ref: e621818697 +Node: Other Changes and Fixes<4>1824530 +Ref: whatsnew/2 1 other-changes-and-fixes1824683 +Ref: e651824683 +Node: Acknowledgements<7>1830617 +Ref: whatsnew/2 1 acknowledgements1830734 +Ref: e661830734 +Node: What’s New in Python 2 01831025 +Ref: whatsnew/2 0 doc1831163 +Ref: e671831163 +Ref: whatsnew/2 0 what-s-new-in-python-2-01831163 +Ref: e681831163 +Node: Introduction<3>1831836 +Ref: whatsnew/2 0 introduction1831945 +Ref: e691831945 +Node: What About Python 1 6?1832829 +Ref: whatsnew/2 0 what-about-python-1-61832970 +Ref: e6a1832970 +Node: New Development Process1834211 +Ref: whatsnew/2 0 new-development-process1834347 +Ref: e6b1834347 +Ref: New Development Process-Footnote-11839495 +Ref: New Development Process-Footnote-21839544 +Ref: New Development Process-Footnote-31839593 +Node: Unicode<2>1839642 +Ref: whatsnew/2 0 unicode1839775 +Ref: e6c1839775 +Ref: Unicode<2>-Footnote-11845949 +Node: List Comprehensions1845998 +Ref: whatsnew/2 0 list-comprehensions1846128 +Ref: e6d1846128 +Node: Augmented Assignment1849859 +Ref: whatsnew/2 0 augmented-assignment1849993 +Ref: e6e1849993 +Node: String Methods1851539 +Ref: whatsnew/2 0 string-methods1851682 +Ref: e711851682 +Node: Garbage Collection of Cycles1853535 +Ref: whatsnew/2 0 garbage-collection-of-cycles1853676 +Ref: e721853676 +Node: Other Core Changes1857023 +Ref: whatsnew/2 0 other-core-changes1857164 +Ref: e731857164 +Node: Minor Language Changes1857449 +Ref: whatsnew/2 0 minor-language-changes1857564 +Ref: e741857564 +Node: Changes to Built-in Functions1862651 +Ref: whatsnew/2 0 changes-to-built-in-functions1862766 +Ref: e791862766 +Node: Porting to 2 01865056 +Ref: whatsnew/2 0 porting-to-2-01865196 +Ref: e7c1865196 +Node: Extending/Embedding Changes1870515 +Ref: whatsnew/2 0 extending-embedding-changes1870677 +Ref: e7d1870677 +Node: Distutils Making Modules Easy to Install1874499 +Ref: whatsnew/2 0 distutils-making-modules-easy-to-install1874658 +Ref: e7e1874658 +Node: XML Modules1878024 +Ref: whatsnew/2 0 xml-modules1878170 +Ref: e7f1878170 +Node: SAX2 Support1879069 +Ref: whatsnew/2 0 sax2-support1879149 +Ref: e801879149 +Node: DOM Support1880787 +Ref: whatsnew/2 0 dom-support1880897 +Ref: e811880897 +Node: Relationship to PyXML1883725 +Ref: whatsnew/2 0 relationship-to-pyxml1883814 +Ref: e821883814 +Node: Module changes1885104 +Ref: whatsnew/2 0 module-changes1885221 +Ref: e831885221 +Node: New modules1887693 +Ref: whatsnew/2 0 new-modules1887816 +Ref: e841887816 +Node: IDLE Improvements1891808 +Ref: whatsnew/2 0 idle-improvements1891947 +Ref: e861891947 +Node: Deleted and Deprecated Modules1893070 +Ref: whatsnew/2 0 deleted-and-deprecated-modules1893217 +Ref: e871893217 +Node: Acknowledgements<8>1893969 +Ref: whatsnew/2 0 acknowledgements1894090 +Ref: e881894090 +Ref: Acknowledgements<8>-Footnote-11894689 +Ref: Acknowledgements<8>-Footnote-21894728 +Node: Changelog1894791 +Ref: whatsnew/changelog doc1894894 +Ref: e891894894 +Ref: whatsnew/changelog changelog1894894 +Ref: 14c1894894 +Ref: whatsnew/changelog id11894894 +Ref: e8a1894894 +Node: The Python Tutorial1894960 +Ref: tutorial/index doc1895081 +Ref: e8b1895081 +Ref: tutorial/index the-python-tutorial1895081 +Ref: e8c1895081 +Ref: tutorial/index tutorial-index1895081 +Ref: e8d1895081 +Node: Whetting Your Appetite1897802 +Ref: tutorial/appetite doc1897917 +Ref: e931897917 +Ref: tutorial/appetite tut-intro1897917 +Ref: e941897917 +Ref: tutorial/appetite whetting-your-appetite1897917 +Ref: e951897917 +Node: Using the Python Interpreter1902435 +Ref: tutorial/interpreter doc1902593 +Ref: e961902593 +Ref: tutorial/interpreter tut-using1902593 +Ref: e971902593 +Ref: tutorial/interpreter using-the-python-interpreter1902593 +Ref: e981902593 +Node: Invoking the Interpreter1902739 +Ref: tutorial/interpreter invoking-the-interpreter1902872 +Ref: e991902872 +Ref: tutorial/interpreter tut-invoking1902872 +Ref: e9a1902872 +Ref: Invoking the Interpreter-Footnote-11905876 +Ref: Invoking the Interpreter-Footnote-21906076 +Node: Argument Passing1906137 +Ref: tutorial/interpreter argument-passing1906239 +Ref: ea01906239 +Ref: tutorial/interpreter tut-argpassing1906239 +Ref: ea11906239 +Node: Interactive Mode1907147 +Ref: tutorial/interpreter interactive-mode1907249 +Ref: ea21907249 +Ref: tutorial/interpreter tut-interactive1907249 +Ref: 8e31907249 +Node: The Interpreter and Its Environment1908263 +Ref: tutorial/interpreter the-interpreter-and-its-environment1908396 +Ref: ea41908396 +Ref: tutorial/interpreter tut-interp1908396 +Ref: ea51908396 +Node: Source Code Encoding1908516 +Ref: tutorial/interpreter source-code-encoding1908608 +Ref: ea61908608 +Ref: tutorial/interpreter tut-source-encoding1908608 +Ref: ea71908608 +Node: An Informal Introduction to Python1909843 +Ref: tutorial/introduction doc1910002 +Ref: ea91910002 +Ref: tutorial/introduction an-informal-introduction-to-python1910002 +Ref: eaa1910002 +Ref: tutorial/introduction tut-informal1910002 +Ref: eab1910002 +Node: Using Python as a Calculator1911305 +Ref: tutorial/introduction tut-calculator1911444 +Ref: eae1911444 +Ref: tutorial/introduction using-python-as-a-calculator1911444 +Ref: eaf1911444 +Node: Numbers1911697 +Ref: tutorial/introduction numbers1911785 +Ref: eb01911785 +Ref: tutorial/introduction tut-numbers1911785 +Ref: eb11911785 +Ref: Numbers-Footnote-11914793 +Node: Strings1914993 +Ref: tutorial/introduction strings1915095 +Ref: eb41915095 +Ref: tutorial/introduction tut-strings1915095 +Ref: eb51915095 +Ref: Strings-Footnote-11923047 +Node: Lists1923355 +Ref: tutorial/introduction lists1923441 +Ref: eba1923441 +Ref: tutorial/introduction tut-lists1923441 +Ref: ebb1923441 +Node: First Steps Towards Programming1926038 +Ref: tutorial/introduction first-steps-towards-programming1926177 +Ref: ebf1926177 +Ref: tutorial/introduction tut-firststeps1926177 +Ref: ec01926177 +Ref: First Steps Towards Programming-Footnote-11929274 +Node: More Control Flow Tools1929329 +Ref: tutorial/controlflow doc1929475 +Ref: ec21929475 +Ref: tutorial/controlflow more-control-flow-tools1929475 +Ref: ec31929475 +Ref: tutorial/controlflow tut-morecontrol1929475 +Ref: ec41929475 +Node: if Statements1930017 +Ref: tutorial/controlflow if-statements1930113 +Ref: ec51930113 +Ref: tutorial/controlflow tut-if1930113 +Ref: ec61930113 +Node: for Statements1930928 +Ref: tutorial/controlflow for-statements1931051 +Ref: ec91931051 +Ref: tutorial/controlflow tut-for1931051 +Ref: eca1931051 +Node: The range Function1932300 +Ref: tutorial/controlflow the-range-function1932465 +Ref: ecb1932465 +Ref: tutorial/controlflow tut-range1932465 +Ref: ecc1932465 +Node: break and continue Statements and else Clauses on Loops1934718 +Ref: tutorial/controlflow break-and-continue-statements-and-else-clauses-on-loops1934884 +Ref: ecf1934884 +Ref: tutorial/controlflow tut-break1934884 +Ref: ed01934884 +Node: pass Statements1937087 +Ref: tutorial/controlflow pass-statements1937253 +Ref: ed21937253 +Ref: tutorial/controlflow tut-pass1937253 +Ref: ed31937253 +Node: Defining Functions1937980 +Ref: tutorial/controlflow defining-functions1938117 +Ref: ed51938117 +Ref: tutorial/controlflow tut-functions1938117 +Ref: ed61938117 +Ref: Defining Functions-Footnote-11943122 +Node: More on Defining Functions1943324 +Ref: tutorial/controlflow more-on-defining-functions1943469 +Ref: eda1943469 +Ref: tutorial/controlflow tut-defining1943469 +Ref: edb1943469 +Node: Default Argument Values1943876 +Ref: tutorial/controlflow default-argument-values1943988 +Ref: edc1943988 +Ref: tutorial/controlflow tut-defaultargs1943988 +Ref: edd1943988 +Node: Keyword Arguments1945999 +Ref: tutorial/controlflow keyword-arguments1946138 +Ref: ede1946138 +Ref: tutorial/controlflow tut-keywordargs1946138 +Ref: edf1946138 +Node: Special parameters1949884 +Ref: tutorial/controlflow special-parameters1950024 +Ref: ee11950024 +Node: Positional-or-Keyword Arguments1951161 +Ref: tutorial/controlflow positional-or-keyword-arguments1951282 +Ref: ee21951282 +Node: Positional-Only Parameters1951496 +Ref: tutorial/controlflow positional-only-parameters1951648 +Ref: ee31951648 +Node: Keyword-Only Arguments1952282 +Ref: tutorial/controlflow keyword-only-arguments1952420 +Ref: ee41952420 +Node: Function Examples1952671 +Ref: tutorial/controlflow function-examples1952788 +Ref: ee51952788 +Node: Recap1955784 +Ref: tutorial/controlflow recap1955870 +Ref: ee61955870 +Node: Arbitrary Argument Lists1956731 +Ref: tutorial/controlflow arbitrary-argument-lists1956878 +Ref: ee71956878 +Ref: tutorial/controlflow tut-arbitraryargs1956878 +Ref: ee81956878 +Node: Unpacking Argument Lists1957917 +Ref: tutorial/controlflow tut-unpacking-arguments1958064 +Ref: ee91958064 +Ref: tutorial/controlflow unpacking-argument-lists1958064 +Ref: eea1958064 +Node: Lambda Expressions1959297 +Ref: tutorial/controlflow lambda-expressions1959441 +Ref: eeb1959441 +Ref: tutorial/controlflow tut-lambda1959441 +Ref: eec1959441 +Node: Documentation Strings1960432 +Ref: tutorial/controlflow documentation-strings1960572 +Ref: eed1960572 +Ref: tutorial/controlflow tut-docstrings1960572 +Ref: ed71960572 +Node: Function Annotations1962502 +Ref: tutorial/controlflow function-annotations1962615 +Ref: eee1962615 +Ref: tutorial/controlflow tut-annotations1962615 +Ref: eef1962615 +Ref: Function Annotations-Footnote-11963828 +Ref: Function Annotations-Footnote-21963877 +Node: Intermezzo Coding Style1963926 +Ref: tutorial/controlflow intermezzo-coding-style1964044 +Ref: ef21964044 +Ref: tutorial/controlflow tut-codingstyle1964044 +Ref: ef31964044 +Ref: Intermezzo Coding Style-Footnote-11966201 +Node: Data Structures1966250 +Ref: tutorial/datastructures doc1966369 +Ref: ef51966369 +Ref: tutorial/datastructures data-structures1966369 +Ref: ef61966369 +Ref: tutorial/datastructures tut-structures1966369 +Ref: ece1966369 +Node: More on Lists1966714 +Ref: tutorial/datastructures more-on-lists1966805 +Ref: ef71966805 +Ref: tutorial/datastructures tut-morelists1966805 +Ref: ef81966805 +Ref: More on Lists-Footnote-11970673 +Node: Using Lists as Stacks1970814 +Ref: tutorial/datastructures tut-lists-as-stacks1970915 +Ref: ef91970915 +Ref: tutorial/datastructures using-lists-as-stacks1970915 +Ref: efa1970915 +Node: Using Lists as Queues1971549 +Ref: tutorial/datastructures tut-lists-as-queues1971681 +Ref: efb1971681 +Ref: tutorial/datastructures using-lists-as-queues1971681 +Ref: efc1971681 +Node: List Comprehensions<2>1972745 +Ref: tutorial/datastructures list-comprehensions1972882 +Ref: efd1972882 +Ref: tutorial/datastructures tut-listcomps1972882 +Ref: efe1972882 +Node: Nested List Comprehensions1976095 +Ref: tutorial/datastructures nested-list-comprehensions1976202 +Ref: eff1976202 +Node: The del statement1977850 +Ref: tutorial/datastructures the-del-statement1977970 +Ref: f001977970 +Ref: tutorial/datastructures tut-del1977970 +Ref: f011977970 +Node: Tuples and Sequences1978819 +Ref: tutorial/datastructures tuples-and-sequences1978930 +Ref: f031978930 +Ref: tutorial/datastructures tut-tuples1978930 +Ref: ee01978930 +Node: Sets1981961 +Ref: tutorial/datastructures sets1982067 +Ref: f051982067 +Ref: tutorial/datastructures tut-sets1982067 +Ref: f061982067 +Node: Dictionaries1983807 +Ref: tutorial/datastructures dictionaries1983911 +Ref: f071983911 +Ref: tutorial/datastructures tut-dictionaries1983911 +Ref: f081983911 +Node: Looping Techniques1986732 +Ref: tutorial/datastructures looping-techniques1986850 +Ref: f091986850 +Ref: tutorial/datastructures tut-loopidioms1986850 +Ref: ecd1986850 +Node: More on Conditions1989030 +Ref: tutorial/datastructures more-on-conditions1989171 +Ref: f0a1989171 +Ref: tutorial/datastructures tut-conditions1989171 +Ref: f0b1989171 +Node: Comparing Sequences and Other Types1991273 +Ref: tutorial/datastructures comparing-sequences-and-other-types1991387 +Ref: f0d1991387 +Ref: tutorial/datastructures tut-comparing1991387 +Ref: f0e1991387 +Node: Modules1992974 +Ref: tutorial/modules doc1993086 +Ref: f0f1993086 +Ref: tutorial/modules modules1993086 +Ref: f101993086 +Ref: tutorial/modules tut-modules1993086 +Ref: f111993086 +Node: More on Modules1995639 +Ref: tutorial/modules more-on-modules1995723 +Ref: f121995723 +Ref: tutorial/modules tut-moremodules1995723 +Ref: f131995723 +Ref: More on Modules-Footnote-11998839 +Node: Executing modules as scripts1999047 +Ref: tutorial/modules executing-modules-as-scripts1999158 +Ref: f141999158 +Ref: tutorial/modules tut-modulesasscripts1999158 +Ref: f151999158 +Node: The Module Search Path2000046 +Ref: tutorial/modules the-module-search-path2000193 +Ref: f162000193 +Ref: tutorial/modules tut-searchpath2000193 +Ref: f172000193 +Node: “Compiled” Python files2001514 +Ref: tutorial/modules compiled-python-files2001624 +Ref: f192001624 +Ref: “Compiled” Python files-Footnote-12003933 +Node: Standard Modules2003982 +Ref: tutorial/modules standard-modules2004091 +Ref: f1a2004091 +Ref: tutorial/modules tut-standardmodules2004091 +Ref: f182004091 +Node: The dir Function2005555 +Ref: tutorial/modules the-dir-function2005657 +Ref: f1b2005657 +Ref: tutorial/modules tut-dir2005657 +Ref: f1c2005657 +Node: Packages2009822 +Ref: tutorial/modules packages2009899 +Ref: f1d2009899 +Ref: tutorial/modules tut-packages2009899 +Ref: f1e2009899 +Node: Importing * From a Package2014202 +Ref: tutorial/modules importing-from-a-package2014306 +Ref: f1f2014306 +Ref: tutorial/modules tut-pkg-import-star2014306 +Ref: f202014306 +Node: Intra-package References2016948 +Ref: tutorial/modules intra-package-references2017093 +Ref: f212017093 +Node: Packages in Multiple Directories2018106 +Ref: tutorial/modules packages-in-multiple-directories2018216 +Ref: f222018216 +Node: Input and Output2018744 +Ref: tutorial/inputoutput doc2018862 +Ref: f242018862 +Ref: tutorial/inputoutput input-and-output2018862 +Ref: f252018862 +Ref: tutorial/inputoutput tut-io2018862 +Ref: f262018862 +Node: Fancier Output Formatting2019171 +Ref: tutorial/inputoutput fancier-output-formatting2019283 +Ref: f272019283 +Ref: tutorial/inputoutput tut-formatting2019283 +Ref: f282019283 +Node: Formatted String Literals2023004 +Ref: tutorial/inputoutput formatted-string-literals2023124 +Ref: f2a2023124 +Ref: tutorial/inputoutput tut-f-strings2023124 +Ref: f292023124 +Node: The String format Method2024686 +Ref: tutorial/inputoutput the-string-format-method2024839 +Ref: f2b2024839 +Ref: tutorial/inputoutput tut-string-format2024839 +Ref: f2c2024839 +Node: Manual String Formatting2027507 +Ref: tutorial/inputoutput manual-string-formatting2027656 +Ref: f2e2027656 +Node: Old string formatting2029181 +Ref: tutorial/inputoutput old-string-formatting2029297 +Ref: f332029297 +Node: Reading and Writing Files2029847 +Ref: tutorial/inputoutput reading-and-writing-files2029959 +Ref: f342029959 +Ref: tutorial/inputoutput tut-files2029959 +Ref: f352029959 +Node: Methods of File Objects2032968 +Ref: tutorial/inputoutput methods-of-file-objects2033094 +Ref: f362033094 +Ref: tutorial/inputoutput tut-filemethods2033094 +Ref: f372033094 +Node: Saving structured data with json2036788 +Ref: tutorial/inputoutput saving-structured-data-with-json2036914 +Ref: f382036914 +Ref: tutorial/inputoutput tut-json2036914 +Ref: f392036914 +Ref: Saving structured data with json-Footnote-12039557 +Node: Errors and Exceptions2039581 +Ref: tutorial/errors doc2039699 +Ref: f3b2039699 +Ref: tutorial/errors errors-and-exceptions2039699 +Ref: f3c2039699 +Ref: tutorial/errors tut-errors2039699 +Ref: f3d2039699 +Node: Syntax Errors2040155 +Ref: tutorial/errors syntax-errors2040245 +Ref: f3e2040245 +Ref: tutorial/errors tut-syntaxerrors2040245 +Ref: f3f2040245 +Node: Exceptions2041048 +Ref: tutorial/errors exceptions2041166 +Ref: f402041166 +Ref: tutorial/errors tut-exceptions2041166 +Ref: f412041166 +Node: Handling Exceptions2042999 +Ref: tutorial/errors handling-exceptions2043122 +Ref: f442043122 +Ref: tutorial/errors tut-handling2043122 +Ref: ed12043122 +Node: Raising Exceptions2049018 +Ref: tutorial/errors raising-exceptions2049154 +Ref: f452049154 +Ref: tutorial/errors tut-raising2049154 +Ref: f462049154 +Node: User-defined Exceptions2050296 +Ref: tutorial/errors tut-userexceptions2050438 +Ref: f472050438 +Ref: tutorial/errors user-defined-exceptions2050438 +Ref: f482050438 +Node: Defining Clean-up Actions2052480 +Ref: tutorial/errors defining-clean-up-actions2052631 +Ref: f492052631 +Ref: tutorial/errors tut-cleanup2052631 +Ref: f4a2052631 +Node: Predefined Clean-up Actions2055758 +Ref: tutorial/errors predefined-clean-up-actions2055877 +Ref: f4b2055877 +Ref: tutorial/errors tut-cleanup-with2055877 +Ref: f4c2055877 +Node: Classes2056996 +Ref: tutorial/classes doc2057132 +Ref: f4d2057132 +Ref: tutorial/classes classes2057132 +Ref: f4e2057132 +Ref: tutorial/classes tut-classes2057132 +Ref: ed92057132 +Node: A Word About Names and Objects2059445 +Ref: tutorial/classes a-word-about-names-and-objects2059556 +Ref: f502059556 +Ref: tutorial/classes tut-object2059556 +Ref: f512059556 +Node: Python Scopes and Namespaces2060478 +Ref: tutorial/classes python-scopes-and-namespaces2060621 +Ref: f522060621 +Ref: tutorial/classes tut-scopes2060621 +Ref: f532060621 +Ref: Python Scopes and Namespaces-Footnote-12066704 +Node: Scopes and Namespaces Example2067098 +Ref: tutorial/classes scopes-and-namespaces-example2067192 +Ref: f552067192 +Ref: tutorial/classes tut-scopeexample2067192 +Ref: f562067192 +Node: A First Look at Classes2068554 +Ref: tutorial/classes a-first-look-at-classes2068681 +Ref: f572068681 +Ref: tutorial/classes tut-firstclasses2068681 +Ref: ef42068681 +Node: Class Definition Syntax2068966 +Ref: tutorial/classes class-definition-syntax2069071 +Ref: f582069071 +Ref: tutorial/classes tut-classdefinition2069071 +Ref: f592069071 +Node: Class Objects2070607 +Ref: tutorial/classes class-objects2070737 +Ref: f5a2070737 +Ref: tutorial/classes tut-classobjects2070737 +Ref: f5b2070737 +Node: Instance Objects2072938 +Ref: tutorial/classes instance-objects2073059 +Ref: f5c2073059 +Ref: tutorial/classes tut-instanceobjects2073059 +Ref: f5d2073059 +Node: Method Objects2074723 +Ref: tutorial/classes method-objects2074859 +Ref: f5e2074859 +Ref: tutorial/classes tut-methodobjects2074859 +Ref: f5f2074859 +Node: Class and Instance Variables2076800 +Ref: tutorial/classes class-and-instance-variables2076911 +Ref: f602076911 +Ref: tutorial/classes tut-class-and-instance-variables2076911 +Ref: f612076911 +Node: Random Remarks2078953 +Ref: tutorial/classes random-remarks2079063 +Ref: f622079063 +Ref: tutorial/classes tut-remarks2079063 +Ref: f632079063 +Node: Inheritance2082890 +Ref: tutorial/classes inheritance2082994 +Ref: f642082994 +Ref: tutorial/classes tut-inheritance2082994 +Ref: f652082994 +Node: Multiple Inheritance2085763 +Ref: tutorial/classes multiple-inheritance2085831 +Ref: f662085831 +Ref: tutorial/classes tut-multiple2085831 +Ref: f672085831 +Node: Private Variables2087832 +Ref: tutorial/classes private-variables2087935 +Ref: f682087935 +Ref: tutorial/classes tut-private2087935 +Ref: f4f2087935 +Node: Odds and Ends2090623 +Ref: tutorial/classes odds-and-ends2090724 +Ref: f692090724 +Ref: tutorial/classes tut-odds2090724 +Ref: f6a2090724 +Node: Iterators2091729 +Ref: tutorial/classes iterators2091823 +Ref: f6b2091823 +Ref: tutorial/classes tut-iterators2091823 +Ref: f6c2091823 +Node: Generators2094105 +Ref: tutorial/classes generators2094207 +Ref: f6d2094207 +Ref: tutorial/classes tut-generators2094207 +Ref: f6e2094207 +Node: Generator Expressions2095644 +Ref: tutorial/classes generator-expressions2095728 +Ref: f702095728 +Ref: tutorial/classes tut-genexps2095728 +Ref: f712095728 +Node: Brief Tour of the Standard Library2096715 +Ref: tutorial/stdlib doc2096876 +Ref: f722096876 +Ref: tutorial/stdlib brief-tour-of-the-standard-library2096876 +Ref: f732096876 +Ref: tutorial/stdlib tut-brieftour2096876 +Ref: f742096876 +Node: Operating System Interface2097273 +Ref: tutorial/stdlib operating-system-interface2097393 +Ref: f752097393 +Ref: tutorial/stdlib tut-os-interface2097393 +Ref: f762097393 +Node: File Wildcards2098658 +Ref: tutorial/stdlib file-wildcards2098809 +Ref: f772098809 +Ref: tutorial/stdlib tut-file-wildcards2098809 +Ref: f782098809 +Node: Command Line Arguments2099051 +Ref: tutorial/stdlib command-line-arguments2099224 +Ref: f792099224 +Ref: tutorial/stdlib tut-command-line-arguments2099224 +Ref: f7a2099224 +Node: Error Output Redirection and Program Termination2100332 +Ref: tutorial/stdlib error-output-redirection-and-program-termination2100514 +Ref: f7b2100514 +Ref: tutorial/stdlib tut-stderr2100514 +Ref: f7c2100514 +Node: String Pattern Matching2101037 +Ref: tutorial/stdlib string-pattern-matching2101208 +Ref: f7d2101208 +Ref: tutorial/stdlib tut-string-pattern-matching2101208 +Ref: f7e2101208 +Node: Mathematics2101847 +Ref: tutorial/stdlib mathematics2101985 +Ref: f7f2101985 +Ref: tutorial/stdlib tut-mathematics2101985 +Ref: f802101985 +Node: Internet Access2103133 +Ref: tutorial/stdlib internet-access2103263 +Ref: f812103263 +Ref: tutorial/stdlib tut-internet-access2103263 +Ref: f822103263 +Node: Dates and Times2104286 +Ref: tutorial/stdlib dates-and-times2104421 +Ref: f832104421 +Ref: tutorial/stdlib tut-dates-and-times2104421 +Ref: f842104421 +Node: Data Compression2105242 +Ref: tutorial/stdlib data-compression2105385 +Ref: f852105385 +Ref: tutorial/stdlib tut-data-compression2105385 +Ref: f862105385 +Node: Performance Measurement2105908 +Ref: tutorial/stdlib performance-measurement2106051 +Ref: f872106051 +Ref: tutorial/stdlib tut-performance-measurement2106051 +Ref: f882106051 +Node: Quality Control2106926 +Ref: tutorial/stdlib quality-control2107071 +Ref: f892107071 +Ref: tutorial/stdlib tut-quality-control2107071 +Ref: f8a2107071 +Node: Batteries Included2108639 +Ref: tutorial/stdlib batteries-included2108752 +Ref: f8b2108752 +Ref: tutorial/stdlib tut-batteries-included2108752 +Ref: f8c2108752 +Ref: Batteries Included-Footnote-12110564 +Node: Brief Tour of the Standard Library — Part II2110613 +Ref: tutorial/stdlib2 doc2110800 +Ref: f8d2110800 +Ref: tutorial/stdlib2 brief-tour-of-the-standard-library-part-ii2110800 +Ref: f8e2110800 +Ref: tutorial/stdlib2 tut-brieftourtwo2110800 +Ref: f8f2110800 +Node: Output Formatting2111274 +Ref: tutorial/stdlib2 output-formatting2111393 +Ref: f902111393 +Ref: tutorial/stdlib2 tut-output-formatting2111393 +Ref: f912111393 +Node: Templating2113506 +Ref: tutorial/stdlib2 templating2113673 +Ref: f922113673 +Ref: tutorial/stdlib2 tut-templating2113673 +Ref: f932113673 +Node: Working with Binary Data Record Layouts2116227 +Ref: tutorial/stdlib2 tut-binary-formats2116395 +Ref: f962116395 +Ref: tutorial/stdlib2 working-with-binary-data-record-layouts2116395 +Ref: f972116395 +Node: Multi-threading<2>2117569 +Ref: tutorial/stdlib2 multi-threading2117734 +Ref: f992117734 +Ref: tutorial/stdlib2 tut-multi-threading2117734 +Ref: f9a2117734 +Node: Logging2119677 +Ref: tutorial/stdlib2 logging2119818 +Ref: f9b2119818 +Ref: tutorial/stdlib2 tut-logging2119818 +Ref: f9c2119818 +Node: Weak References2120986 +Ref: tutorial/stdlib2 tut-weak-references2121137 +Ref: f9d2121137 +Ref: tutorial/stdlib2 weak-references2121137 +Ref: f9e2121137 +Node: Tools for Working with Lists2122893 +Ref: tutorial/stdlib2 tools-for-working-with-lists2123070 +Ref: fa02123070 +Ref: tutorial/stdlib2 tut-list-tools2123070 +Ref: fa12123070 +Node: Decimal Floating Point Arithmetic2125524 +Ref: tutorial/stdlib2 decimal-floating-point-arithmetic2125677 +Ref: fa22125677 +Ref: tutorial/stdlib2 tut-decimal-fp2125677 +Ref: fa32125677 +Node: Virtual Environments and Packages2127574 +Ref: tutorial/venv doc2127736 +Ref: fa42127736 +Ref: tutorial/venv tut-venv2127736 +Ref: fa52127736 +Ref: tutorial/venv virtual-environments-and-packages2127736 +Ref: fa62127736 +Node: Introduction<4>2127923 +Ref: tutorial/venv introduction2128046 +Ref: fa72128046 +Node: Creating Virtual Environments2129367 +Ref: tutorial/venv creating-virtual-environments2129525 +Ref: fa92129525 +Node: Managing Packages with pip2131588 +Ref: tutorial/venv managing-packages-with-pip2131722 +Ref: faa2131722 +Node: What Now?2136084 +Ref: tutorial/whatnow doc2136250 +Ref: fab2136250 +Ref: tutorial/whatnow tut-whatnow2136250 +Ref: fac2136250 +Ref: tutorial/whatnow what-now2136250 +Ref: fad2136250 +Ref: What Now?-Footnote-12139714 +Node: Interactive Input Editing and History Substitution2139871 +Ref: tutorial/interactive doc2140052 +Ref: faf2140052 +Ref: tutorial/interactive interactive-input-editing-and-history-substitution2140052 +Ref: fb02140052 +Ref: tutorial/interactive tut-interacting2140052 +Ref: e9d2140052 +Ref: Interactive Input Editing and History Substitution-Footnote-12140648 +Node: Tab Completion and History Editing2140709 +Ref: tutorial/interactive tab-completion-and-history-editing2140882 +Ref: fb12140882 +Ref: tutorial/interactive tut-keybindings2140882 +Ref: fb22140882 +Node: Alternatives to the Interactive Interpreter2141737 +Ref: tutorial/interactive alternatives-to-the-interactive-interpreter2141910 +Ref: fb32141910 +Ref: tutorial/interactive tut-commentary2141910 +Ref: fb42141910 +Ref: Alternatives to the Interactive Interpreter-Footnote-12142804 +Ref: Alternatives to the Interactive Interpreter-Footnote-22142833 +Node: Floating Point Arithmetic Issues and Limitations2142878 +Ref: tutorial/floatingpoint doc2143058 +Ref: fb52143058 +Ref: tutorial/floatingpoint bpython2143058 +Ref: fb62143058 +Ref: tutorial/floatingpoint floating-point-arithmetic-issues-and-limitations2143058 +Ref: fb72143058 +Ref: tutorial/floatingpoint tut-fp-issues2143058 +Ref: fb82143058 +Ref: Floating Point Arithmetic Issues and Limitations-Footnote-12151146 +Node: Representation Error2151185 +Ref: tutorial/floatingpoint representation-error2151290 +Ref: fbb2151290 +Ref: tutorial/floatingpoint tut-fp-error2151290 +Ref: fbc2151290 +Node: Appendix2154496 +Ref: tutorial/appendix doc2154617 +Ref: fbd2154617 +Ref: tutorial/appendix appendix2154617 +Ref: fbe2154617 +Ref: tutorial/appendix tut-appendix2154617 +Ref: fbf2154617 +Node: Interactive Mode<2>2154697 +Ref: tutorial/appendix interactive-mode2154761 +Ref: fc02154761 +Ref: tutorial/appendix tut-interac2154761 +Ref: ea32154761 +Node: Error Handling2154932 +Ref: tutorial/appendix error-handling2155036 +Ref: fc12155036 +Ref: tutorial/appendix tut-error2155036 +Ref: fc22155036 +Ref: Error Handling-Footnote-12156089 +Node: Executable Python Scripts2156159 +Ref: tutorial/appendix executable-python-scripts2156300 +Ref: fc32156300 +Ref: tutorial/appendix tut-scripts2156300 +Ref: ea82156300 +Node: The Interactive Startup File2157382 +Ref: tutorial/appendix the-interactive-startup-file2157534 +Ref: fc42157534 +Ref: tutorial/appendix tut-startup2157534 +Ref: fc52157534 +Node: The Customization Modules2158970 +Ref: tutorial/appendix the-customization-modules2159088 +Ref: fc62159088 +Ref: tutorial/appendix tut-customize2159088 +Ref: fc72159088 +Node: Python Setup and Usage2159991 +Ref: using/index doc2160119 +Ref: fc82160119 +Ref: using/index python-setup-and-usage2160119 +Ref: fc92160119 +Ref: using/index using-index2160119 +Ref: fca2160119 +Node: Command line and environment2160541 +Ref: using/cmdline doc2160667 +Ref: fcb2160667 +Ref: using/cmdline command-line-and-environment2160667 +Ref: fcc2160667 +Ref: using/cmdline using-on-general2160667 +Ref: e9f2160667 +Node: Command line2161035 +Ref: using/cmdline command-line2161142 +Ref: fce2161142 +Ref: using/cmdline using-on-cmdline2161142 +Ref: 92b2161142 +Node: Interface options2161536 +Ref: using/cmdline interface-options2161626 +Ref: fcf2161626 +Ref: using/cmdline using-on-interface-options2161626 +Ref: fd02161626 +Ref: using/cmdline cmdoption-c2163008 +Ref: e9e2163008 +Ref: using/cmdline cmdoption-m2163583 +Ref: 3372163583 +Ref: using/cmdline cmdarg-dash2165984 +Ref: fd42165984 +Ref: using/cmdline cmdarg-script2166400 +Ref: fd52166400 +Ref: Interface options-Footnote-12168250 +Node: Generic options2168299 +Ref: using/cmdline generic-options2168419 +Ref: fd62168419 +Ref: using/cmdline cmdoption2168468 +Ref: d8c2168468 +Ref: using/cmdline cmdoption-h2168491 +Ref: fd72168491 +Ref: using/cmdline cmdoption-help2168514 +Ref: d8b2168514 +Ref: using/cmdline cmdoption-v2168603 +Ref: 5d22168603 +Ref: using/cmdline cmdoption-version2168626 +Ref: 5d12168626 +Node: Miscellaneous options2168974 +Ref: using/cmdline miscellaneous-options2169104 +Ref: fd82169104 +Ref: using/cmdline using-on-misc-options2169104 +Ref: 92c2169104 +Ref: using/cmdline cmdoption-b2169165 +Ref: 4152169165 +Ref: using/cmdline id12169504 +Ref: d382169504 +Ref: using/cmdline cmdoption-check-hash-based-pycs2169673 +Ref: fd92169673 +Ref: using/cmdline cmdoption-d2170375 +Ref: fda2170375 +Ref: using/cmdline cmdoption-e2170529 +Ref: fdc2170529 +Ref: using/cmdline cmdoption-i2170687 +Ref: e1f2170687 +Ref: using/cmdline id22171144 +Ref: fd22171144 +Ref: using/cmdline cmdoption-o2171549 +Ref: 68b2171549 +Ref: using/cmdline cmdoption-oo2171944 +Ref: 68c2171944 +Ref: using/cmdline cmdoption-q2172257 +Ref: fdf2172257 +Ref: using/cmdline cmdoption-r2172395 +Ref: fe02172395 +Ref: using/cmdline cmdoption-s2173455 +Ref: d152173455 +Ref: using/cmdline id32173654 +Ref: b242173654 +Ref: using/cmdline cmdoption-u2173978 +Ref: fe32173978 +Ref: using/cmdline id42174261 +Ref: d882174261 +Ref: using/cmdline using-on-warnings2174634 +Ref: fe62174634 +Ref: using/cmdline cmdoption-w2174634 +Ref: 4172174634 +Ref: using/cmdline cmdoption-x2176244 +Ref: fe92176244 +Ref: using/cmdline id52176409 +Ref: 1552176409 +Ref: Miscellaneous options-Footnote-12180063 +Ref: Miscellaneous options-Footnote-22180112 +Ref: Miscellaneous options-Footnote-32180161 +Ref: Miscellaneous options-Footnote-42180210 +Ref: Miscellaneous options-Footnote-52180259 +Node: Options you shouldn’t use2180308 +Ref: using/cmdline options-you-shouldn-t-use2180414 +Ref: fed2180414 +Ref: using/cmdline cmdoption-j2180485 +Ref: d372180485 +Ref: Options you shouldn’t use-Footnote-12180582 +Node: Environment variables2180613 +Ref: using/cmdline environment-variables2180720 +Ref: fee2180720 +Ref: using/cmdline using-on-envvars2180720 +Ref: fef2180720 +Ref: using/cmdline envvar-PYTHONHOME2181013 +Ref: 4d62181013 +Ref: using/cmdline envvar-PYTHONPATH2181619 +Ref: 9532181619 +Ref: using/cmdline envvar-PYTHONSTARTUP2182680 +Ref: 8e52182680 +Ref: using/cmdline envvar-PYTHONOPTIMIZE2183366 +Ref: fde2183366 +Ref: using/cmdline envvar-PYTHONBREAKPOINT2183604 +Ref: 3142183604 +Ref: using/cmdline envvar-PYTHONDEBUG2184227 +Ref: fdb2184227 +Ref: using/cmdline envvar-PYTHONINSPECT2184462 +Ref: e1e2184462 +Ref: using/cmdline envvar-PYTHONUNBUFFERED2184745 +Ref: fe42184745 +Ref: using/cmdline envvar-PYTHONVERBOSE2184895 +Ref: fe52184895 +Ref: using/cmdline envvar-PYTHONCASEOK2185132 +Ref: e5d2185132 +Ref: using/cmdline envvar-PYTHONDONTWRITEBYTECODE2185291 +Ref: d392185291 +Ref: using/cmdline envvar-PYTHONPYCACHEPREFIX2185535 +Ref: 1542185535 +Ref: using/cmdline envvar-PYTHONHASHSEED2185876 +Ref: 9c52185876 +Ref: using/cmdline envvar-PYTHONIOENCODING2186559 +Ref: 92f2186559 +Ref: using/cmdline envvar-PYTHONNOUSERSITE2187382 +Ref: d162187382 +Ref: using/cmdline envvar-PYTHONUSERBASE2187624 +Ref: d142187624 +Ref: using/cmdline envvar-PYTHONEXECUTABLE2187983 +Ref: ff52187983 +Ref: using/cmdline envvar-PYTHONWARNINGS2188196 +Ref: 4182188196 +Ref: using/cmdline envvar-PYTHONFAULTHANDLER2189117 +Ref: 9c82189117 +Ref: using/cmdline envvar-PYTHONTRACEMALLOC2189529 +Ref: ff62189529 +Ref: using/cmdline envvar-PYTHONPROFILEIMPORTTIME2190003 +Ref: 3382190003 +Ref: using/cmdline envvar-PYTHONASYNCIODEBUG2190280 +Ref: ff72190280 +Ref: using/cmdline envvar-PYTHONMALLOC2190489 +Ref: 5032190489 +Ref: using/cmdline envvar-PYTHONMALLOCSTATS2191747 +Ref: ffa2191747 +Ref: using/cmdline envvar-PYTHONLEGACYWINDOWSFSENCODING2192362 +Ref: 4f72192362 +Ref: using/cmdline envvar-PYTHONLEGACYWINDOWSSTDIO2192877 +Ref: 4fa2192877 +Ref: using/cmdline envvar-PYTHONCOERCECLOCALE2193347 +Ref: 30c2193347 +Ref: using/cmdline envvar-PYTHONDEVMODE2196072 +Ref: 32c2196072 +Ref: using/cmdline envvar-PYTHONUTF82196299 +Ref: 3112196299 +Ref: Environment variables-Footnote-12199092 +Ref: Environment variables-Footnote-22199141 +Ref: Environment variables-Footnote-32199190 +Ref: Environment variables-Footnote-42199239 +Ref: Environment variables-Footnote-52199288 +Node: Debug-mode variables2199337 +Ref: using/cmdline debug-mode-variables2199415 +Ref: fff2199415 +Ref: using/cmdline envvar-PYTHONTHREADDEBUG2199546 +Ref: 10002199546 +Ref: using/cmdline envvar-PYTHONDUMPREFS2199718 +Ref: 1592199718 +Node: Using Python on Unix platforms2199950 +Ref: using/unix doc2200108 +Ref: 10012200108 +Ref: using/unix using-on-unix2200108 +Ref: 10022200108 +Ref: using/unix using-python-on-unix-platforms2200108 +Ref: 10032200108 +Node: Getting and installing the latest version of Python2200318 +Ref: using/unix getting-and-installing-the-latest-version-of-python2200460 +Ref: 10042200460 +Node: On Linux2200646 +Ref: using/unix on-linux2200773 +Ref: 10052200773 +Node: On FreeBSD and OpenBSD2201690 +Ref: using/unix on-freebsd-and-openbsd2201840 +Ref: 10062201840 +Node: On OpenSolaris2202330 +Ref: using/unix on-opensolaris2202463 +Ref: 10072202463 +Ref: On OpenSolaris-Footnote-12202683 +Node: Building Python2202716 +Ref: using/unix building-python2202897 +Ref: 10082202897 +Ref: using/unix building-python-on-unix2202897 +Ref: 7f32202897 +Ref: Building Python-Footnote-12203712 +Ref: Building Python-Footnote-22203761 +Ref: Building Python-Footnote-32203828 +Node: Python-related paths and files2203890 +Ref: using/unix python-related-paths-and-files2204033 +Ref: 10092204033 +Node: Miscellaneous2205603 +Ref: using/unix miscellaneous2205722 +Ref: 100a2205722 +Node: Using Python on Windows2206280 +Ref: using/windows doc2206437 +Ref: 100b2206437 +Ref: using/windows using-on-windows2206437 +Ref: 2d92206437 +Ref: using/windows using-python-on-windows2206437 +Ref: 100c2206437 +Ref: Using Python on Windows-Footnote-12208725 +Ref: Using Python on Windows-Footnote-22208775 +Node: The full installer2208824 +Ref: using/windows the-full-installer2208938 +Ref: 10102208938 +Ref: using/windows windows-full2208938 +Ref: 100d2208938 +Node: Installation steps2209145 +Ref: using/windows installation-steps2209259 +Ref: 10112209259 +Node: Removing the MAX_PATH Limitation2211369 +Ref: using/windows max-path2211513 +Ref: 4d52211513 +Ref: using/windows removing-the-max-path-limitation2211513 +Ref: 10132211513 +Node: Installing Without UI2212373 +Ref: using/windows install-quiet-option2212529 +Ref: 10142212529 +Ref: using/windows installing-without-ui2212529 +Ref: 10152212529 +Node: Installing Without Downloading2221857 +Ref: using/windows install-layout-option2222001 +Ref: 10122222001 +Ref: using/windows installing-without-downloading2222001 +Ref: 10162222001 +Node: Modifying an install2222996 +Ref: using/windows modifying-an-install2223110 +Ref: 10172223110 +Node: The Microsoft Store package2223999 +Ref: using/windows the-microsoft-store-package2224144 +Ref: 10182224144 +Ref: using/windows windows-store2224144 +Ref: e9b2224144 +Node: Known Issues2226300 +Ref: using/windows known-issues2226376 +Ref: 10192226376 +Ref: Known Issues-Footnote-12226987 +Node: The nuget org packages2227083 +Ref: using/windows the-nuget-org-packages2227232 +Ref: 101a2227232 +Ref: using/windows windows-nuget2227232 +Ref: 100e2227232 +Ref: The nuget org packages-Footnote-12229516 +Ref: The nuget org packages-Footnote-22229547 +Ref: The nuget org packages-Footnote-32229593 +Node: The embeddable package2229642 +Ref: using/windows the-embeddable-package2229783 +Ref: 101b2229783 +Ref: using/windows windows-embeddable2229783 +Ref: 100f2229783 +Ref: The embeddable package-Footnote-12231809 +Node: Python Application2231880 +Ref: using/windows python-application2231982 +Ref: 101c2231982 +Node: Embedding Python2233479 +Ref: using/windows embedding-python2233581 +Ref: 101d2233581 +Node: Alternative bundles2234369 +Ref: using/windows alternative-bundles2234506 +Ref: 101e2234506 +Ref: Alternative bundles-Footnote-12235374 +Ref: Alternative bundles-Footnote-22235424 +Ref: Alternative bundles-Footnote-32235467 +Ref: Alternative bundles-Footnote-42235517 +Node: Configuring Python2235554 +Ref: using/windows configuring-python2235679 +Ref: 101f2235679 +Node: Excursus Setting environment variables2236247 +Ref: using/windows excursus-setting-environment-variables2236378 +Ref: 10202236378 +Ref: using/windows setting-envvars2236378 +Ref: e9c2236378 +Node: Finding the Python executable2238760 +Ref: using/windows finding-the-python-executable2238891 +Ref: 10212238891 +Ref: using/windows windows-path-mod2238891 +Ref: 9902238891 +Node: UTF-8 mode2240183 +Ref: using/windows utf-8-mode2240316 +Ref: 10222240316 +Ref: using/windows win-utf8-mode2240316 +Ref: 10232240316 +Ref: UTF-8 mode-Footnote-12242116 +Ref: UTF-8 mode-Footnote-22242165 +Node: Python Launcher for Windows2242214 +Ref: using/windows launcher2242344 +Ref: 98f2242344 +Ref: using/windows python-launcher-for-windows2242344 +Ref: 10242242344 +Ref: Python Launcher for Windows-Footnote-12243145 +Node: Getting started2243194 +Ref: using/windows getting-started2243295 +Ref: 10252243295 +Node: From the command-line2243450 +Ref: using/windows from-the-command-line2243552 +Ref: 10262243552 +Node: Virtual environments2244768 +Ref: using/windows virtual-environments2244892 +Ref: 10272244892 +Node: From a script2245388 +Ref: using/windows from-a-script2245513 +Ref: 10282245513 +Node: From file associations2246559 +Ref: using/windows from-file-associations2246655 +Ref: 10292246655 +Node: Shebang Lines2247232 +Ref: using/windows shebang-lines2247368 +Ref: 102a2247368 +Node: Arguments in shebang lines2249423 +Ref: using/windows arguments-in-shebang-lines2249557 +Ref: 102b2249557 +Node: Customization2249847 +Ref: using/windows customization2249979 +Ref: 102c2249979 +Node: Customization via INI files2250106 +Ref: using/windows customization-via-ini-files2250227 +Ref: 102d2250227 +Node: Customizing default Python versions2250985 +Ref: using/windows customizing-default-python-versions2251106 +Ref: 102e2251106 +Node: Diagnostics2254965 +Ref: using/windows diagnostics2255062 +Ref: 102f2255062 +Node: Finding modules2255495 +Ref: using/windows finding-modules2255633 +Ref: 4b02255633 +Ref: using/windows id12255633 +Ref: 10302255633 +Node: Additional modules2261901 +Ref: using/windows additional-modules2262039 +Ref: 10322262039 +Node: PyWin322262453 +Ref: using/windows pywin322262533 +Ref: 10342262533 +Ref: PyWin32-Footnote-12263117 +Ref: PyWin32-Footnote-22263158 +Ref: PyWin32-Footnote-32263258 +Ref: PyWin32-Footnote-42263332 +Ref: PyWin32-Footnote-52263426 +Ref: PyWin32-Footnote-62263484 +Node: cx_Freeze2263533 +Ref: using/windows cx-freeze2263628 +Ref: 10352263628 +Ref: cx_Freeze-Footnote-12263985 +Node: WConio2264039 +Ref: using/windows wconio2264118 +Ref: 10372264118 +Ref: WConio-Footnote-12264466 +Node: Compiling Python on Windows2264526 +Ref: using/windows compiling-python-on-windows2264664 +Ref: 10382264664 +Ref: Compiling Python on Windows-Footnote-12265632 +Ref: Compiling Python on Windows-Footnote-22265681 +Ref: Compiling Python on Windows-Footnote-32265748 +Node: Other Platforms2265796 +Ref: using/windows other-platforms2265907 +Ref: 103a2265907 +Ref: Other Platforms-Footnote-12266479 +Ref: Other Platforms-Footnote-22266528 +Ref: Other Platforms-Footnote-32266569 +Ref: Other Platforms-Footnote-42266597 +Ref: Other Platforms-Footnote-52266685 +Ref: Other Platforms-Footnote-62266739 +Node: Using Python on a Macintosh2266789 +Ref: using/mac doc2266932 +Ref: 103b2266932 +Ref: using/mac using-on-mac2266932 +Ref: 103c2266932 +Ref: using/mac using-python-on-a-macintosh2266932 +Ref: 103d2266932 +Node: Getting and Installing MacPython2267462 +Ref: using/mac getting-and-installing-macpython2267574 +Ref: 103e2267574 +Ref: using/mac getting-osx2267574 +Ref: 103f2267574 +Node: How to run a Python script2269554 +Ref: using/mac how-to-run-a-python-script2269684 +Ref: 10402269684 +Node: Running scripts with a GUI2271155 +Ref: using/mac osx-gui-scripts2271307 +Ref: 10422271307 +Ref: using/mac running-scripts-with-a-gui2271307 +Ref: 10432271307 +Node: Configuration2271723 +Ref: using/mac configuration2271840 +Ref: 10442271840 +Node: The IDE2272383 +Ref: using/mac ide2272541 +Ref: 10412272541 +Ref: using/mac the-ide2272541 +Ref: 10462272541 +Node: Installing Additional Python Packages2272755 +Ref: using/mac installing-additional-python-packages2272907 +Ref: 10472272907 +Ref: using/mac mac-package-manager2272907 +Ref: 10452272907 +Node: GUI Programming on the Mac2273307 +Ref: using/mac gui-programming-on-the-mac2273495 +Ref: 10482273495 +Node: Distributing Python Applications on the Mac2274510 +Ref: using/mac distributing-python-applications-on-the-mac2274676 +Ref: 10492274676 +Node: Other Resources2274977 +Ref: using/mac other-resources2275108 +Ref: 104a2275108 +Node: Editors and IDEs2275420 +Ref: using/editors doc2275531 +Ref: 104b2275531 +Ref: using/editors editors2275531 +Ref: 104c2275531 +Ref: using/editors editors-and-ides2275531 +Ref: 104d2275531 +Ref: Editors and IDEs-Footnote-12275870 +Ref: Editors and IDEs-Footnote-22275919 +Ref: Editors and IDEs-Footnote-32275970 +Node: The Python Language Reference2276041 +Ref: reference/index doc2276177 +Ref: 104e2276177 +Ref: reference/index reference-index2276177 +Ref: e8f2276177 +Ref: reference/index the-python-language-reference2276177 +Ref: 104f2276177 +Node: Introduction<5>2277165 +Ref: reference/introduction doc2277271 +Ref: 10502277271 +Ref: reference/introduction id12277271 +Ref: 10512277271 +Ref: reference/introduction introduction2277271 +Ref: 10522277271 +Node: Alternate Implementations2279066 +Ref: reference/introduction alternate-implementations2279160 +Ref: 10532279160 +Ref: reference/introduction implementations2279160 +Ref: fcd2279160 +Ref: Alternate Implementations-Footnote-12281342 +Ref: Alternate Implementations-Footnote-22281373 +Ref: Alternate Implementations-Footnote-32281410 +Ref: Alternate Implementations-Footnote-42281441 +Node: Notation2281466 +Ref: reference/introduction id22281560 +Ref: 10542281560 +Ref: reference/introduction notation2281560 +Ref: 10552281560 +Ref: reference/introduction grammar-token-name2281723 +Ref: 10562281723 +Ref: reference/introduction grammar-token-lc-letter2281771 +Ref: 10572281771 +Node: Lexical analysis2283805 +Ref: reference/lexical_analysis doc2283930 +Ref: 10582283930 +Ref: reference/lexical_analysis lexical2283930 +Ref: 10592283930 +Ref: reference/lexical_analysis lexical-analysis2283930 +Ref: 105a2283930 +Ref: Lexical analysis-Footnote-12284575 +Node: Line structure2284624 +Ref: reference/lexical_analysis id12284712 +Ref: 105b2284712 +Ref: reference/lexical_analysis line-structure2284712 +Ref: 105c2284712 +Node: Logical lines2285018 +Ref: reference/lexical_analysis id22285105 +Ref: 105d2285105 +Ref: reference/lexical_analysis logical-lines2285105 +Ref: 105e2285105 +Node: Physical lines2285483 +Ref: reference/lexical_analysis id32285587 +Ref: 105f2285587 +Ref: reference/lexical_analysis physical-lines2285587 +Ref: 10602285587 +Node: Comments2286358 +Ref: reference/lexical_analysis comments2286470 +Ref: 10612286470 +Ref: reference/lexical_analysis id42286470 +Ref: 10622286470 +Node: Encoding declarations2286773 +Ref: reference/lexical_analysis encoding-declarations2286892 +Ref: 10632286892 +Ref: reference/lexical_analysis encodings2286892 +Ref: 10642286892 +Node: Explicit line joining2288022 +Ref: reference/lexical_analysis explicit-joining2288154 +Ref: 10652288154 +Ref: reference/lexical_analysis explicit-line-joining2288154 +Ref: 10662288154 +Node: Implicit line joining2289088 +Ref: reference/lexical_analysis implicit-joining2289210 +Ref: 10672289210 +Ref: reference/lexical_analysis implicit-line-joining2289210 +Ref: 10682289210 +Node: Blank lines2290046 +Ref: reference/lexical_analysis blank-lines2290158 +Ref: 10692290158 +Ref: reference/lexical_analysis id52290158 +Ref: 106a2290158 +Node: Indentation2290633 +Ref: reference/lexical_analysis id62290749 +Ref: 106b2290749 +Ref: reference/lexical_analysis indentation2290749 +Ref: 106c2290749 +Node: Whitespace between tokens2294135 +Ref: reference/lexical_analysis whitespace2294231 +Ref: 106d2294231 +Ref: reference/lexical_analysis whitespace-between-tokens2294231 +Ref: 106e2294231 +Node: Other tokens2294633 +Ref: reference/lexical_analysis id72294754 +Ref: 106f2294754 +Ref: reference/lexical_analysis other-tokens2294754 +Ref: 10702294754 +Node: Identifiers and keywords2295186 +Ref: reference/lexical_analysis identifiers2295301 +Ref: 10712295301 +Ref: reference/lexical_analysis identifiers-and-keywords2295301 +Ref: 10722295301 +Ref: reference/lexical_analysis grammar-token-identifier2296196 +Ref: 10732296196 +Ref: reference/lexical_analysis grammar-token-id-start2296242 +Ref: 10742296242 +Ref: reference/lexical_analysis grammar-token-id-continue2296391 +Ref: 10752296391 +Ref: reference/lexical_analysis grammar-token-xid-start2296539 +Ref: 10762296539 +Ref: reference/lexical_analysis grammar-token-xid-continue2296646 +Ref: 10772296646 +Ref: Identifiers and keywords-Footnote-12297645 +Ref: Identifiers and keywords-Footnote-22297694 +Ref: Identifiers and keywords-Footnote-32297743 +Node: Keywords2297805 +Ref: reference/lexical_analysis id82297914 +Ref: 10782297914 +Ref: reference/lexical_analysis keywords2297914 +Ref: 10792297914 +Node: Reserved classes of identifiers2298509 +Ref: reference/lexical_analysis id-classes2298618 +Ref: 107a2298618 +Ref: reference/lexical_analysis reserved-classes-of-identifiers2298618 +Ref: 107b2298618 +Node: Literals2300261 +Ref: reference/lexical_analysis id92300373 +Ref: 107d2300373 +Ref: reference/lexical_analysis literals2300373 +Ref: 107e2300373 +Node: String and Bytes literals2300668 +Ref: reference/lexical_analysis string-and-bytes-literals2300775 +Ref: 107f2300775 +Ref: reference/lexical_analysis strings2300775 +Ref: 10802300775 +Ref: reference/lexical_analysis grammar-token-stringliteral2300913 +Ref: 10812300913 +Ref: reference/lexical_analysis grammar-token-stringprefix2300979 +Ref: 10822300979 +Ref: reference/lexical_analysis grammar-token-shortstring2301119 +Ref: 10832301119 +Ref: reference/lexical_analysis grammar-token-longstring2301196 +Ref: 10842301196 +Ref: reference/lexical_analysis grammar-token-shortstringitem2301279 +Ref: 10852301279 +Ref: reference/lexical_analysis grammar-token-longstringitem2301338 +Ref: 10862301338 +Ref: reference/lexical_analysis grammar-token-shortstringchar2301396 +Ref: 10872301396 +Ref: reference/lexical_analysis grammar-token-longstringchar2301479 +Ref: 10882301479 +Ref: reference/lexical_analysis grammar-token-stringescapeseq2301538 +Ref: 10892301538 +Ref: reference/lexical_analysis grammar-token-bytesliteral2301591 +Ref: 108a2301591 +Ref: reference/lexical_analysis grammar-token-bytesprefix2301651 +Ref: 108b2301651 +Ref: reference/lexical_analysis grammar-token-shortbytes2301741 +Ref: 108c2301741 +Ref: reference/lexical_analysis grammar-token-longbytes2301815 +Ref: 108d2301815 +Ref: reference/lexical_analysis grammar-token-shortbytesitem2301895 +Ref: 108e2301895 +Ref: reference/lexical_analysis grammar-token-longbytesitem2301951 +Ref: 108f2301951 +Ref: reference/lexical_analysis grammar-token-shortbyteschar2302006 +Ref: 10902302006 +Ref: reference/lexical_analysis grammar-token-longbyteschar2302087 +Ref: 10912302087 +Ref: reference/lexical_analysis grammar-token-bytesescapeseq2302144 +Ref: 10922302144 +Ref: String and Bytes literals-Footnote-12309540 +Ref: String and Bytes literals-Footnote-22309589 +Node: String literal concatenation2309664 +Ref: reference/lexical_analysis string-concatenation2309805 +Ref: 10932309805 +Ref: reference/lexical_analysis string-literal-concatenation2309805 +Ref: 10942309805 +Node: Formatted string literals2310848 +Ref: reference/lexical_analysis f-strings2310980 +Ref: 15c2310980 +Ref: reference/lexical_analysis formatted-string-literals2310980 +Ref: 10952310980 +Ref: reference/lexical_analysis grammar-token-f-string2311591 +Ref: 10962311591 +Ref: reference/lexical_analysis grammar-token-replacement-field2311668 +Ref: 10972311668 +Ref: reference/lexical_analysis grammar-token-f-expression2311757 +Ref: 10982311757 +Ref: reference/lexical_analysis grammar-token-conversion2311952 +Ref: 10992311952 +Ref: reference/lexical_analysis grammar-token-format-spec2311995 +Ref: 109a2311995 +Ref: reference/lexical_analysis grammar-token-literal-char2312065 +Ref: 109b2312065 +Ref: Formatted string literals-Footnote-12317195 +Node: Numeric literals2317244 +Ref: reference/lexical_analysis numbers2317364 +Ref: 109c2317364 +Ref: reference/lexical_analysis numeric-literals2317364 +Ref: 109d2317364 +Node: Integer literals2317798 +Ref: reference/lexical_analysis integer-literals2317916 +Ref: 109e2317916 +Ref: reference/lexical_analysis integers2317916 +Ref: 109f2317916 +Ref: reference/lexical_analysis grammar-token-integer2318037 +Ref: 10a02318037 +Ref: reference/lexical_analysis grammar-token-decinteger2318109 +Ref: 10a12318109 +Ref: reference/lexical_analysis grammar-token-bininteger2318179 +Ref: 10a22318179 +Ref: reference/lexical_analysis grammar-token-octinteger2318235 +Ref: 10a32318235 +Ref: reference/lexical_analysis grammar-token-hexinteger2318291 +Ref: 10a42318291 +Ref: reference/lexical_analysis grammar-token-nonzerodigit2318347 +Ref: 10a52318347 +Ref: reference/lexical_analysis grammar-token-digit2318379 +Ref: 10a62318379 +Ref: reference/lexical_analysis grammar-token-bindigit2318411 +Ref: 10a72318411 +Ref: reference/lexical_analysis grammar-token-octdigit2318443 +Ref: 10a82318443 +Ref: reference/lexical_analysis grammar-token-hexdigit2318475 +Ref: 10a92318475 +Node: Floating point literals2319329 +Ref: reference/lexical_analysis floating2319449 +Ref: 10aa2319449 +Ref: reference/lexical_analysis floating-point-literals2319449 +Ref: 10ab2319449 +Ref: reference/lexical_analysis grammar-token-floatnumber2319591 +Ref: 10ac2319591 +Ref: reference/lexical_analysis grammar-token-pointfloat2319641 +Ref: 10ad2319641 +Ref: reference/lexical_analysis grammar-token-exponentfloat2319701 +Ref: 10ae2319701 +Ref: reference/lexical_analysis grammar-token-digitpart2319758 +Ref: 10af2319758 +Ref: reference/lexical_analysis grammar-token-fraction2319802 +Ref: 10b02319802 +Ref: reference/lexical_analysis grammar-token-exponent2319839 +Ref: 10b12319839 +Node: Imaginary literals2320405 +Ref: reference/lexical_analysis imaginary2320500 +Ref: 10b22320500 +Ref: reference/lexical_analysis imaginary-literals2320500 +Ref: 10b32320500 +Ref: reference/lexical_analysis grammar-token-imagnumber2320627 +Ref: 10b42320627 +Node: Operators2321090 +Ref: reference/lexical_analysis id112321188 +Ref: 10b52321188 +Ref: reference/lexical_analysis operators2321188 +Ref: 10b62321188 +Node: Delimiters2321425 +Ref: reference/lexical_analysis delimiters2321506 +Ref: 10b72321506 +Ref: reference/lexical_analysis id122321506 +Ref: 10b82321506 +Node: Data model2322422 +Ref: reference/datamodel doc2322547 +Ref: 10b92322547 +Ref: reference/datamodel data-model2322547 +Ref: 10ba2322547 +Ref: reference/datamodel datamodel2322547 +Ref: 10bb2322547 +Node: Objects values and types2322715 +Ref: reference/datamodel objects2322822 +Ref: 10bc2322822 +Ref: reference/datamodel objects-values-and-types2322822 +Ref: 10bd2322822 +Ref: Objects values and types-Footnote-12327631 +Node: The standard type hierarchy2327862 +Ref: reference/datamodel the-standard-type-hierarchy2327998 +Ref: 10bf2327998 +Ref: reference/datamodel types2327998 +Ref: 10c02327998 +Ref: reference/datamodel frame-objects2364268 +Ref: 10d32364268 +Ref: reference/datamodel frame clear2366012 +Ref: 80b2366012 +Ref: reference/datamodel traceback-objects2366566 +Ref: 3362366566 +Ref: reference/datamodel slice indices2369240 +Ref: 95f2369240 +Node: Special method names2371010 +Ref: reference/datamodel special-method-names2371132 +Ref: 10d42371132 +Ref: reference/datamodel specialnames2371132 +Ref: 50e2371132 +Ref: Special method names-Footnote-12372964 +Node: Basic customization2373246 +Ref: reference/datamodel basic-customization2373359 +Ref: 10d52373359 +Ref: reference/datamodel customization2373359 +Ref: 10d62373359 +Ref: reference/datamodel object __new__2373416 +Ref: 8892373416 +Ref: reference/datamodel object __init__2374990 +Ref: d5e2374990 +Ref: reference/datamodel object __del__2375849 +Ref: 91f2375849 +Ref: reference/datamodel object __repr__2378980 +Ref: 33e2378980 +Ref: reference/datamodel object __str__2379826 +Ref: e372379826 +Ref: reference/datamodel object __bytes__2380477 +Ref: 10d82380477 +Ref: reference/datamodel object __format__2380661 +Ref: 94e2380661 +Ref: reference/datamodel richcmpfuncs2381770 +Ref: 10da2381770 +Ref: reference/datamodel object __lt__2381770 +Ref: c342381770 +Ref: reference/datamodel object __le__2381810 +Ref: ca02381810 +Ref: reference/datamodel object __eq__2381850 +Ref: 33f2381850 +Ref: reference/datamodel object __ne__2381890 +Ref: e562381890 +Ref: reference/datamodel object __gt__2381930 +Ref: ca12381930 +Ref: reference/datamodel object __ge__2381970 +Ref: ca22381970 +Ref: reference/datamodel object __hash__2384490 +Ref: 3402384490 +Ref: reference/datamodel object __bool__2388704 +Ref: c682388704 +Node: Customizing attribute access2389154 +Ref: reference/datamodel attribute-access2389302 +Ref: 10db2389302 +Ref: reference/datamodel customizing-attribute-access2389302 +Ref: 10dc2389302 +Ref: reference/datamodel object __getattr__2389534 +Ref: 31a2389534 +Ref: reference/datamodel object __getattribute__2390780 +Ref: 4492390780 +Ref: reference/datamodel object __setattr__2391873 +Ref: e2c2391873 +Ref: reference/datamodel object __delattr__2392546 +Ref: 10d12392546 +Ref: reference/datamodel object __dir__2392930 +Ref: 31b2392930 +Node: Customizing module attribute access2393258 +Ref: reference/datamodel customizing-module-attribute-access2393391 +Ref: 10df2393391 +Ref: Customizing module attribute access-Footnote-12395517 +Node: Implementing Descriptors2395566 +Ref: reference/datamodel descriptors2395728 +Ref: 4e92395728 +Ref: reference/datamodel implementing-descriptors2395728 +Ref: 10e02395728 +Ref: reference/datamodel object __get__2396208 +Ref: 10dd2396208 +Ref: reference/datamodel object __set__2397139 +Ref: 10e12397139 +Ref: reference/datamodel object __delete__2397497 +Ref: 10e22397497 +Ref: reference/datamodel object __set_name__2397633 +Ref: 4e82397633 +Ref: Implementing Descriptors-Footnote-12398836 +Node: Invoking Descriptors2398885 +Ref: reference/datamodel descriptor-invocation2399021 +Ref: 10e32399021 +Ref: reference/datamodel invoking-descriptors2399021 +Ref: 10e42399021 +Node: __slots__2402236 +Ref: reference/datamodel id32402339 +Ref: 10e52402339 +Ref: reference/datamodel slots2402339 +Ref: 10e62402339 +Ref: reference/datamodel object __slots__2402697 +Ref: e2d2402697 +Node: Notes on using __slots__2403050 +Ref: reference/datamodel notes-on-using-slots2403120 +Ref: 10e72403120 +Node: Customizing class creation2405908 +Ref: reference/datamodel class-customization2406077 +Ref: 4e52406077 +Ref: reference/datamodel customizing-class-creation2406077 +Ref: 10e82406077 +Ref: reference/datamodel object __init_subclass__2406544 +Ref: 4e32406544 +Node: Metaclasses2408082 +Ref: reference/datamodel id42408186 +Ref: 10e92408186 +Ref: reference/datamodel metaclasses2408186 +Ref: 10ea2408186 +Node: Resolving MRO entries2409241 +Ref: reference/datamodel resolving-mro-entries2409391 +Ref: 10eb2409391 +Ref: Resolving MRO entries-Footnote-12409918 +Node: Determining the appropriate metaclass2409967 +Ref: reference/datamodel determining-the-appropriate-metaclass2410135 +Ref: 10ec2410135 +Node: Preparing the class namespace2411066 +Ref: reference/datamodel prepare2411237 +Ref: 4fd2411237 +Ref: reference/datamodel preparing-the-class-namespace2411237 +Ref: 10ed2411237 +Ref: Preparing the class namespace-Footnote-12412142 +Node: Executing the class body2412191 +Ref: reference/datamodel executing-the-class-body2412350 +Ref: 10ee2412350 +Node: Creating the class object2413085 +Ref: reference/datamodel class-object-creation2413235 +Ref: 4e42413235 +Ref: reference/datamodel creating-the-class-object2413235 +Ref: 10ef2413235 +Ref: Creating the class object-Footnote-12415706 +Node: Uses for metaclasses2415755 +Ref: reference/datamodel uses-for-metaclasses2415872 +Ref: 10f02415872 +Node: Customizing instance and subclass checks2416183 +Ref: reference/datamodel customizing-instance-and-subclass-checks2416347 +Ref: 10f12416347 +Ref: reference/datamodel class __instancecheck__2416837 +Ref: 10f22416837 +Ref: reference/datamodel class __subclasscheck__2417066 +Ref: 10f32417066 +Ref: Customizing instance and subclass checks-Footnote-12418019 +Node: Emulating generic types2418068 +Ref: reference/datamodel emulating-generic-types2418232 +Ref: 10f42418232 +Ref: reference/datamodel object __class_getitem__2418430 +Ref: 3272418430 +Ref: Emulating generic types-Footnote-12418958 +Ref: Emulating generic types-Footnote-22419007 +Node: Emulating callable objects2419056 +Ref: reference/datamodel callable-types2419205 +Ref: 10f52419205 +Ref: reference/datamodel emulating-callable-objects2419205 +Ref: 10f62419205 +Ref: reference/datamodel object __call__2419278 +Ref: 10ce2419278 +Node: Emulating container types2419509 +Ref: reference/datamodel emulating-container-types2419658 +Ref: 10f72419658 +Ref: reference/datamodel sequence-types2419658 +Ref: 10f82419658 +Ref: reference/datamodel object __len__2421868 +Ref: dcb2421868 +Ref: reference/datamodel object __length_hint__2422606 +Ref: 80c2422606 +Ref: reference/datamodel object __getitem__2423393 +Ref: 27c2423393 +Ref: reference/datamodel object __setitem__2424289 +Ref: c622424289 +Ref: reference/datamodel object __delitem__2424751 +Ref: c632424751 +Ref: reference/datamodel object __missing__2425177 +Ref: b3b2425177 +Ref: reference/datamodel object __iter__2425368 +Ref: 50f2425368 +Ref: reference/datamodel object __reversed__2425846 +Ref: 52f2425846 +Ref: reference/datamodel object __contains__2426830 +Ref: d282426830 +Node: Emulating numeric types2427394 +Ref: reference/datamodel emulating-numeric-types2427548 +Ref: 11012427548 +Ref: reference/datamodel numeric-types2427548 +Ref: 11022427548 +Ref: reference/datamodel object __add__2427863 +Ref: 10f92427863 +Ref: reference/datamodel object __sub__2427904 +Ref: 11032427904 +Ref: reference/datamodel object __mul__2427945 +Ref: 10fb2427945 +Ref: reference/datamodel object __matmul__2427986 +Ref: 6482427986 +Ref: reference/datamodel object __truediv__2428030 +Ref: 7862428030 +Ref: reference/datamodel object __floordiv__2428075 +Ref: e352428075 +Ref: reference/datamodel object __mod__2428121 +Ref: 11042428121 +Ref: reference/datamodel object __divmod__2428162 +Ref: 7872428162 +Ref: reference/datamodel object __pow__2428206 +Ref: 11052428206 +Ref: reference/datamodel object __lshift__2428257 +Ref: 11062428257 +Ref: reference/datamodel object __rshift__2428301 +Ref: 11072428301 +Ref: reference/datamodel object __and__2428345 +Ref: 11082428345 +Ref: reference/datamodel object __xor__2428386 +Ref: 11092428386 +Ref: reference/datamodel object __or__2428427 +Ref: 110a2428427 +Ref: reference/datamodel object __radd__2429410 +Ref: 10fa2429410 +Ref: reference/datamodel object __rsub__2429452 +Ref: 110b2429452 +Ref: reference/datamodel object __rmul__2429494 +Ref: 10fc2429494 +Ref: reference/datamodel object __rmatmul__2429536 +Ref: 6492429536 +Ref: reference/datamodel object __rtruediv__2429581 +Ref: 110c2429581 +Ref: reference/datamodel object __rfloordiv__2429627 +Ref: 110d2429627 +Ref: reference/datamodel object __rmod__2429674 +Ref: 6af2429674 +Ref: reference/datamodel object __rdivmod__2429716 +Ref: 110e2429716 +Ref: reference/datamodel object __rpow__2429761 +Ref: 110f2429761 +Ref: reference/datamodel object __rlshift__2429813 +Ref: 11102429813 +Ref: reference/datamodel object __rrshift__2429858 +Ref: 11112429858 +Ref: reference/datamodel object __rand__2429903 +Ref: 11122429903 +Ref: reference/datamodel object __rxor__2429945 +Ref: 11132429945 +Ref: reference/datamodel object __ror__2429987 +Ref: 11142429987 +Ref: reference/datamodel object __iadd__2431240 +Ref: e6f2431240 +Ref: reference/datamodel object __isub__2431282 +Ref: e702431282 +Ref: reference/datamodel object __imul__2431324 +Ref: 10fd2431324 +Ref: reference/datamodel object __imatmul__2431366 +Ref: 64a2431366 +Ref: reference/datamodel object __itruediv__2431411 +Ref: 11152431411 +Ref: reference/datamodel object __ifloordiv__2431457 +Ref: 11162431457 +Ref: reference/datamodel object __imod__2431504 +Ref: 11172431504 +Ref: reference/datamodel object __ipow__2431546 +Ref: 11182431546 +Ref: reference/datamodel object __ilshift__2431598 +Ref: 11192431598 +Ref: reference/datamodel object __irshift__2431643 +Ref: 111a2431643 +Ref: reference/datamodel object __iand__2431688 +Ref: 111b2431688 +Ref: reference/datamodel object __ixor__2431730 +Ref: 111c2431730 +Ref: reference/datamodel object __ior__2431772 +Ref: 111d2431772 +Ref: reference/datamodel object __neg__2433115 +Ref: 111f2433115 +Ref: reference/datamodel object __pos__2433149 +Ref: 11202433149 +Ref: reference/datamodel object __abs__2433183 +Ref: 11212433183 +Ref: reference/datamodel object __invert__2433217 +Ref: 11222433217 +Ref: reference/datamodel object __complex__2433369 +Ref: 18d2433369 +Ref: reference/datamodel object __int__2433407 +Ref: 18b2433407 +Ref: reference/datamodel object __float__2433441 +Ref: 18c2433441 +Ref: reference/datamodel object __index__2433649 +Ref: 18a2433649 +Ref: reference/datamodel object __round__2434323 +Ref: 11242434323 +Ref: reference/datamodel object __trunc__2434370 +Ref: 11252434370 +Ref: reference/datamodel object __floor__2434406 +Ref: 11262434406 +Ref: reference/datamodel object __ceil__2434442 +Ref: 11272434442 +Ref: Emulating numeric types-Footnote-12435005 +Ref: Emulating numeric types-Footnote-22435324 +Node: With Statement Context Managers2435553 +Ref: reference/datamodel context-managers2435703 +Ref: 11282435703 +Ref: reference/datamodel with-statement-context-managers2435703 +Ref: 11292435703 +Ref: reference/datamodel object __enter__2436466 +Ref: c952436466 +Ref: reference/datamodel object __exit__2436716 +Ref: c962436716 +Ref: With Statement Context Managers-Footnote-12437604 +Node: Special method lookup2437653 +Ref: reference/datamodel special-lookup2437771 +Ref: 10de2437771 +Ref: reference/datamodel special-method-lookup2437771 +Ref: 112b2437771 +Node: Coroutines2440492 +Ref: reference/datamodel coroutines2440578 +Ref: 112c2440578 +Node: Awaitable Objects2440728 +Ref: reference/datamodel awaitable-objects2440818 +Ref: 112d2440818 +Ref: reference/datamodel object __await__2441297 +Ref: 6422441297 +Ref: Awaitable Objects-Footnote-12441713 +Node: Coroutine Objects2441762 +Ref: reference/datamodel coroutine-objects2441883 +Ref: 10cc2441883 +Ref: reference/datamodel id72441883 +Ref: 112f2441883 +Ref: reference/datamodel coroutine send2442753 +Ref: 11312442753 +Ref: reference/datamodel coroutine throw2443292 +Ref: 11332443292 +Ref: reference/datamodel coroutine close2443888 +Ref: 11352443888 +Node: Asynchronous Iterators2444510 +Ref: reference/datamodel async-iterators2444643 +Ref: 6462444643 +Ref: reference/datamodel asynchronous-iterators2444643 +Ref: 11372444643 +Ref: reference/datamodel object __aiter__2444867 +Ref: 4872444867 +Ref: reference/datamodel object __anext__2444957 +Ref: 11382444957 +Node: Asynchronous Context Managers2445839 +Ref: reference/datamodel async-context-managers2445946 +Ref: 11392445946 +Ref: reference/datamodel asynchronous-context-managers2445946 +Ref: 113a2445946 +Ref: reference/datamodel object __aenter__2446251 +Ref: 113b2446251 +Ref: reference/datamodel object __aexit__2446410 +Ref: 113c2446410 +Node: Execution model2446885 +Ref: reference/executionmodel doc2447011 +Ref: 113d2447011 +Ref: reference/executionmodel execmodel2447011 +Ref: 113e2447011 +Ref: reference/executionmodel execution-model2447011 +Ref: 113f2447011 +Node: Structure of a program2447141 +Ref: reference/executionmodel prog-structure2447242 +Ref: 11402447242 +Ref: reference/executionmodel structure-of-a-program2447242 +Ref: 11412447242 +Node: Naming and binding2448146 +Ref: reference/executionmodel naming2448269 +Ref: 11422448269 +Ref: reference/executionmodel naming-and-binding2448269 +Ref: 11432448269 +Node: Binding of names2448451 +Ref: reference/executionmodel bind-names2448550 +Ref: 11442448550 +Ref: reference/executionmodel binding-of-names2448550 +Ref: 11452448550 +Node: Resolution of names2450034 +Ref: reference/executionmodel resolution-of-names2450175 +Ref: 11462450175 +Ref: reference/executionmodel resolve-names2450175 +Ref: 11472450175 +Node: Builtins and restricted execution2453565 +Ref: reference/executionmodel builtins-and-restricted-execution2453723 +Ref: 11482453723 +Ref: reference/executionmodel restrict-exec2453723 +Ref: 11492453723 +Node: Interaction with dynamic features2454568 +Ref: reference/executionmodel dynamic-features2454698 +Ref: 114a2454698 +Ref: reference/executionmodel interaction-with-dynamic-features2454698 +Ref: 114b2454698 +Ref: Interaction with dynamic features-Footnote-12455490 +Node: Exceptions<2>2455632 +Ref: reference/executionmodel exceptions2455724 +Ref: 114c2455724 +Ref: reference/executionmodel id22455724 +Ref: 114d2455724 +Node: The import system2457882 +Ref: reference/import doc2458009 +Ref: 114e2458009 +Ref: reference/import importsystem2458009 +Ref: 10cf2458009 +Ref: reference/import the-import-system2458009 +Ref: 114f2458009 +Ref: The import system-Footnote-12460659 +Ref: The import system-Footnote-22460704 +Ref: The import system-Footnote-32460753 +Node: importlib<7>2460802 +Ref: reference/import importlib2460888 +Ref: 11532460888 +Node: Packages<2>2461263 +Ref: reference/import packages2461367 +Ref: 11542461367 +Node: Regular packages2462804 +Ref: reference/import regular-packages2462895 +Ref: 11562462895 +Node: Namespace packages2464107 +Ref: reference/import namespace-packages2464198 +Ref: 11592464198 +Ref: Namespace packages-Footnote-12465575 +Node: Searching2465624 +Ref: reference/import searching2465723 +Ref: 115b2465723 +Node: The module cache2466548 +Ref: reference/import the-module-cache2466638 +Ref: 115c2466638 +Node: Finders and loaders2468202 +Ref: reference/import finders-and-loaders2468313 +Ref: 115d2468313 +Ref: reference/import id22468313 +Ref: 115e2468313 +Node: Import hooks2470070 +Ref: reference/import import-hooks2470178 +Ref: 11632470178 +Node: The meta path2471050 +Ref: reference/import the-meta-path2471130 +Ref: 11642471130 +Node: Loading2474129 +Ref: reference/import loading2474238 +Ref: 11662474238 +Ref: Loading-Footnote-12477544 +Node: Loaders2477928 +Ref: reference/import loaders2477998 +Ref: 11682477998 +Node: Submodules2481255 +Ref: reference/import submodules2481345 +Ref: 11692481345 +Node: Module spec2482649 +Ref: reference/import module-spec2482764 +Ref: 116a2482764 +Node: Import-related module attributes2483603 +Ref: reference/import import-mod-attrs2483723 +Ref: 11672483723 +Ref: reference/import import-related-module-attributes2483723 +Ref: 116c2483723 +Ref: reference/import __name__2483961 +Ref: d122483961 +Ref: reference/import __loader__2484159 +Ref: 9562484159 +Ref: reference/import __package__2484475 +Ref: 9552484475 +Ref: reference/import __spec__2485302 +Ref: 116d2485302 +Ref: reference/import __path__2485911 +Ref: f232485911 +Ref: reference/import __file__2486403 +Ref: 10d02486403 +Ref: reference/import __cached__2486428 +Ref: b322486428 +Ref: Import-related module attributes-Footnote-12487417 +Ref: Import-related module attributes-Footnote-22487466 +Ref: Import-related module attributes-Footnote-32487515 +Node: module __path__2487564 +Ref: reference/import module-path2487685 +Ref: 11712487685 +Ref: reference/import package-path-rules2487685 +Ref: 11702487685 +Ref: module __path__-Footnote-12488867 +Ref: module __path__-Footnote-22488916 +Node: Module reprs2488965 +Ref: reference/import module-reprs2489082 +Ref: 11722489082 +Node: Cached bytecode invalidation2490753 +Ref: reference/import cached-bytecode-invalidation2490846 +Ref: 11732490846 +Ref: reference/import pyc-invalidation2490846 +Ref: 3292490846 +Node: The Path Based Finder2492196 +Ref: reference/import the-path-based-finder2492332 +Ref: 11742492332 +Node: Path entry finders2494836 +Ref: reference/import path-entry-finders2494947 +Ref: 11762494947 +Ref: Path entry finders-Footnote-12500008 +Node: Path entry finder protocol2500266 +Ref: reference/import path-entry-finder-protocol2500377 +Ref: 11782500377 +Node: Replacing the standard import system2502545 +Ref: reference/import replacing-the-standard-import-system2502698 +Ref: 11792502698 +Node: Package Relative Imports2503701 +Ref: reference/import package-relative-imports2503868 +Ref: 117a2503868 +Ref: reference/import relativeimports2503868 +Ref: 117b2503868 +Node: Special considerations for __main__2505071 +Ref: reference/import special-considerations-for-main2505213 +Ref: 117c2505213 +Node: __main__ __spec__2505816 +Ref: reference/import id52505905 +Ref: 117d2505905 +Ref: reference/import main-spec2505905 +Ref: 116f2505905 +Node: Open issues2507301 +Ref: reference/import open-issues2507429 +Ref: 117e2507429 +Ref: Open issues-Footnote-12508065 +Node: References2508114 +Ref: reference/import references2508198 +Ref: 117f2508198 +Ref: References-Footnote-12509431 +Ref: References-Footnote-22509483 +Ref: References-Footnote-32509532 +Ref: References-Footnote-42509581 +Ref: References-Footnote-52509630 +Ref: References-Footnote-62509679 +Ref: References-Footnote-72509728 +Ref: References-Footnote-82509777 +Ref: References-Footnote-92509826 +Ref: References-Footnote-102509875 +Node: Expressions2509925 +Ref: reference/expressions doc2510054 +Ref: 11802510054 +Ref: reference/expressions expressions2510054 +Ref: 11812510054 +Ref: reference/expressions id12510054 +Ref: 11822510054 +Ref: reference/expressions grammar-token-name2510348 +Ref: 11832510348 +Node: Arithmetic conversions2510914 +Ref: reference/expressions arithmetic-conversions2510998 +Ref: 11842510998 +Ref: reference/expressions conversions2510998 +Ref: 11852510998 +Node: Atoms2511701 +Ref: reference/expressions atoms2511803 +Ref: 11862511803 +Ref: reference/expressions id22511803 +Ref: 11872511803 +Ref: reference/expressions grammar-token-atom2512050 +Ref: 11882512050 +Ref: reference/expressions grammar-token-enclosure2512102 +Ref: 11892512102 +Node: Identifiers Names2512529 +Ref: reference/expressions atom-identifiers2512608 +Ref: 107c2512608 +Ref: reference/expressions identifiers-names2512608 +Ref: 118a2512608 +Node: Literals<2>2513902 +Ref: reference/expressions atom-literals2514009 +Ref: 118b2514009 +Ref: reference/expressions literals2514009 +Ref: 118c2514009 +Ref: reference/expressions grammar-token-literal2514117 +Ref: 118d2514117 +Node: Parenthesized forms2514824 +Ref: reference/expressions parenthesized2514954 +Ref: 118e2514954 +Ref: reference/expressions parenthesized-forms2514954 +Ref: 118f2514954 +Ref: reference/expressions grammar-token-parenth-form2515089 +Ref: 11902515089 +Node: Displays for lists sets and dictionaries2515853 +Ref: reference/expressions comprehensions2515985 +Ref: 11912515985 +Ref: reference/expressions displays-for-lists-sets-and-dictionaries2515985 +Ref: 11922515985 +Ref: reference/expressions grammar-token-comprehension2516431 +Ref: 11932516431 +Ref: reference/expressions grammar-token-comp-for2516485 +Ref: 11942516485 +Ref: reference/expressions grammar-token-comp-iter2516561 +Ref: 11952516561 +Ref: reference/expressions grammar-token-comp-if2516603 +Ref: 11962516603 +Ref: Displays for lists sets and dictionaries-Footnote-12518794 +Node: List displays2518843 +Ref: reference/expressions list-displays2518968 +Ref: 11972518968 +Ref: reference/expressions lists2518968 +Ref: 11982518968 +Ref: reference/expressions grammar-token-list-display2519100 +Ref: 11992519100 +Node: Set displays2519546 +Ref: reference/expressions set2519650 +Ref: 119a2519650 +Ref: reference/expressions set-displays2519650 +Ref: 119b2519650 +Ref: reference/expressions grammar-token-set-display2519830 +Ref: 119c2519830 +Node: Dictionary displays2520362 +Ref: reference/expressions dict2520474 +Ref: 64e2520474 +Ref: reference/expressions dictionary-displays2520474 +Ref: 119d2520474 +Ref: reference/expressions grammar-token-dict-display2520625 +Ref: 119e2520625 +Ref: reference/expressions grammar-token-key-datum-list2520699 +Ref: 119f2520699 +Ref: reference/expressions grammar-token-key-datum2520760 +Ref: 11a02520760 +Ref: reference/expressions grammar-token-dict-comprehension2520829 +Ref: 11a12520829 +Ref: Dictionary displays-Footnote-12522644 +Ref: Dictionary displays-Footnote-22522693 +Node: Generator expressions2522742 +Ref: reference/expressions generator-expressions2522859 +Ref: 11a22522859 +Ref: reference/expressions genexpr2522859 +Ref: 11a32522859 +Ref: reference/expressions grammar-token-generator-expression2522992 +Ref: 11a42522992 +Node: Yield expressions2524934 +Ref: reference/expressions yield-expressions2525023 +Ref: 11a52525023 +Ref: reference/expressions yieldexpr2525023 +Ref: 11a62525023 +Ref: reference/expressions grammar-token-yield-atom2525076 +Ref: 11a72525076 +Ref: reference/expressions grammar-token-yield-expression2525127 +Ref: 11a82525127 +Ref: Yield expressions-Footnote-12530011 +Ref: Yield expressions-Footnote-22530060 +Ref: Yield expressions-Footnote-32530109 +Ref: Yield expressions-Footnote-42530158 +Ref: Yield expressions-Footnote-52530207 +Node: Generator-iterator methods2530256 +Ref: reference/expressions generator-iterator-methods2530353 +Ref: 11ac2530353 +Ref: reference/expressions generator-methods2530353 +Ref: 11302530353 +Ref: reference/expressions generator __next__2530695 +Ref: f6f2530695 +Ref: reference/expressions generator send2531456 +Ref: 11322531456 +Ref: reference/expressions generator throw2532038 +Ref: 11342532038 +Ref: reference/expressions generator close2532524 +Ref: 11362532524 +Node: Examples2533109 +Ref: reference/expressions examples2533247 +Ref: 11ad2533247 +Node: Asynchronous generator functions2534306 +Ref: reference/expressions asynchronous-generator-functions2534457 +Ref: 11aa2534457 +Ref: reference/expressions id32534457 +Ref: 11ae2534457 +Ref: Asynchronous generator functions-Footnote-12537526 +Node: Asynchronous generator-iterator methods2537605 +Ref: reference/expressions asynchronous-generator-iterator-methods2537739 +Ref: 11b32537739 +Ref: reference/expressions asynchronous-generator-methods2537739 +Ref: 11b42537739 +Ref: reference/expressions agen __anext__2537981 +Ref: 11af2537981 +Ref: reference/expressions agen asend2538912 +Ref: 11b02538912 +Ref: reference/expressions agen athrow2539777 +Ref: 11b52539777 +Ref: reference/expressions agen aclose2540465 +Ref: 11b12540465 +Node: Primaries2541480 +Ref: reference/expressions id42541576 +Ref: 11b62541576 +Ref: reference/expressions primaries2541576 +Ref: 11b72541576 +Ref: reference/expressions grammar-token-primary2541698 +Ref: 11b82541698 +Node: Attribute references2541844 +Ref: reference/expressions attribute-references2541932 +Ref: 11b92541932 +Ref: reference/expressions id52541932 +Ref: 11ba2541932 +Ref: reference/expressions grammar-token-attributeref2542061 +Ref: 11bb2542061 +Node: Subscriptions2542640 +Ref: reference/expressions id62542745 +Ref: 11bc2542745 +Ref: reference/expressions subscriptions2542745 +Ref: 11bd2542745 +Ref: reference/expressions grammar-token-subscription2542892 +Ref: 11be2542892 +Node: Slicings2544417 +Ref: reference/expressions id72544507 +Ref: 11bf2544507 +Ref: reference/expressions slicings2544507 +Ref: 11c02544507 +Ref: reference/expressions grammar-token-slicing2544755 +Ref: 11c12544755 +Ref: reference/expressions grammar-token-slice-list2544804 +Ref: 11c22544804 +Ref: reference/expressions grammar-token-slice-item2544861 +Ref: 11c32544861 +Ref: reference/expressions grammar-token-proper-slice2544909 +Ref: 11c42544909 +Ref: reference/expressions grammar-token-lower-bound2544980 +Ref: 11c52544980 +Ref: reference/expressions grammar-token-upper-bound2545013 +Ref: 11c62545013 +Ref: reference/expressions grammar-token-stride2545046 +Ref: 11c72545046 +Node: Calls2546264 +Ref: reference/expressions calls2546332 +Ref: 64c2546332 +Ref: reference/expressions id82546332 +Ref: 11c82546332 +Ref: reference/expressions grammar-token-call2546481 +Ref: 11cb2546481 +Ref: reference/expressions grammar-token-argument-list2546565 +Ref: 11cc2546565 +Ref: reference/expressions grammar-token-positional-arguments2546829 +Ref: 11cd2546829 +Ref: reference/expressions grammar-token-positional-item2546898 +Ref: 11ce2546898 +Ref: reference/expressions grammar-token-starred-and-keywords2546967 +Ref: 11cf2546967 +Ref: reference/expressions grammar-token-keywords-arguments2547100 +Ref: 11d02547100 +Ref: reference/expressions grammar-token-keyword-item2547235 +Ref: 11d12547235 +Ref: Calls-Footnote-12553271 +Node: Await expression2553320 +Ref: reference/expressions await2553429 +Ref: 1a32553429 +Ref: reference/expressions await-expression2553429 +Ref: 11d32553429 +Ref: reference/expressions grammar-token-await-expr2553618 +Ref: 11d42553618 +Node: The power operator2553676 +Ref: reference/expressions power2553815 +Ref: 11d52553815 +Ref: reference/expressions the-power-operator2553815 +Ref: 11d62553815 +Ref: reference/expressions grammar-token-power2554012 +Ref: 11d72554012 +Node: Unary arithmetic and bitwise operations2555061 +Ref: reference/expressions unary2555212 +Ref: 11d82555212 +Ref: reference/expressions unary-arithmetic-and-bitwise-operations2555212 +Ref: 11d92555212 +Ref: reference/expressions grammar-token-u-expr2555374 +Ref: 11da2555374 +Node: Binary arithmetic operations2555894 +Ref: reference/expressions binary2556046 +Ref: 11db2556046 +Ref: reference/expressions binary-arithmetic-operations2556046 +Ref: 11dc2556046 +Ref: reference/expressions grammar-token-m-expr2556390 +Ref: 11dd2556390 +Ref: reference/expressions grammar-token-a-expr2556546 +Ref: 11de2556546 +Ref: Binary arithmetic operations-Footnote-12559399 +Ref: Binary arithmetic operations-Footnote-22560016 +Node: Shifting operations2560303 +Ref: reference/expressions shifting2560441 +Ref: 11e02560441 +Ref: reference/expressions shifting-operations2560441 +Ref: 11e12560441 +Ref: reference/expressions grammar-token-shift-expr2560571 +Ref: 11e22560571 +Node: Binary bitwise operations2560935 +Ref: reference/expressions binary-bitwise-operations2561056 +Ref: 11e32561056 +Ref: reference/expressions bitwise2561056 +Ref: 11e42561056 +Ref: reference/expressions grammar-token-and-expr2561191 +Ref: 11e52561191 +Ref: reference/expressions grammar-token-xor-expr2561246 +Ref: 11e62561246 +Ref: reference/expressions grammar-token-or-expr2561297 +Ref: 11e72561297 +Node: Comparisons2561635 +Ref: reference/expressions comparisons2561755 +Ref: 11e82561755 +Ref: reference/expressions id112561755 +Ref: 11e92561755 +Ref: reference/expressions grammar-token-comparison2562050 +Ref: 11ea2562050 +Ref: reference/expressions grammar-token-comp-operator2562106 +Ref: 11eb2562106 +Node: Value comparisons2563046 +Ref: reference/expressions value-comparisons2563146 +Ref: 11ec2563146 +Ref: reference/expressions in2570994 +Ref: 7a32570994 +Ref: reference/expressions not-in2570994 +Ref: 10ff2570994 +Ref: Value comparisons-Footnote-12571032 +Ref: Value comparisons-Footnote-22571081 +Node: Membership test operations2572197 +Ref: reference/expressions membership-test-details2572326 +Ref: 11002572326 +Ref: reference/expressions membership-test-operations2572326 +Ref: 11ee2572326 +Ref: reference/expressions is2574220 +Ref: 10be2574220 +Node: Identity comparisons2574220 +Ref: reference/expressions identity-comparisons2574323 +Ref: 11ef2574323 +Ref: reference/expressions is-not2574323 +Ref: 11f02574323 +Ref: reference/expressions booleans2574669 +Ref: 11f12574669 +Ref: reference/expressions and2574669 +Ref: 11f22574669 +Ref: reference/expressions or2574669 +Ref: 11f32574669 +Ref: Identity comparisons-Footnote-12574707 +Node: Boolean operations2575014 +Ref: reference/expressions boolean-operations2575134 +Ref: 11f42575134 +Ref: reference/expressions not2575134 +Ref: 11f52575134 +Ref: reference/expressions grammar-token-or-test2575187 +Ref: 11f62575187 +Ref: reference/expressions grammar-token-and-test2575238 +Ref: 11f72575238 +Ref: reference/expressions grammar-token-not-test2575291 +Ref: 11f82575291 +Node: Assignment expressions<2>2576734 +Ref: reference/expressions assignment-expressions2576866 +Ref: 11f92576866 +Ref: reference/expressions grammar-token-assignment-expression2576927 +Ref: 11fa2576927 +Ref: Assignment expressions<2>-Footnote-12577575 +Node: Conditional expressions2577624 +Ref: reference/expressions conditional-expressions2577745 +Ref: 11fc2577745 +Ref: reference/expressions if-expr2577745 +Ref: 11fd2577745 +Ref: reference/expressions grammar-token-conditional-expression2577808 +Ref: 11fe2577808 +Ref: reference/expressions grammar-token-expression2577881 +Ref: 11fb2577881 +Ref: reference/expressions grammar-token-expression-nocond2577950 +Ref: 11ff2577950 +Ref: reference/expressions lambdas2578398 +Ref: 12002578398 +Ref: Conditional expressions-Footnote-12578436 +Node: Lambdas2578485 +Ref: reference/expressions id142578597 +Ref: 12012578597 +Ref: reference/expressions lambda2578597 +Ref: c2f2578597 +Ref: reference/expressions grammar-token-lambda-expr2578628 +Ref: 12022578628 +Ref: reference/expressions grammar-token-lambda-expr-nocond2578697 +Ref: 12032578697 +Node: Expression lists2579244 +Ref: reference/expressions expression-lists2579349 +Ref: 12042579349 +Ref: reference/expressions exprlists2579349 +Ref: 64d2579349 +Ref: reference/expressions grammar-token-expression-list2579398 +Ref: 11ab2579398 +Ref: reference/expressions grammar-token-starred-list2579461 +Ref: 12052579461 +Ref: reference/expressions grammar-token-starred-expression2579528 +Ref: 12062579528 +Ref: reference/expressions grammar-token-starred-item2579604 +Ref: 12072579604 +Ref: Expression lists-Footnote-12580572 +Node: Evaluation order2580621 +Ref: reference/expressions evalorder2580738 +Ref: 12082580738 +Ref: reference/expressions evaluation-order2580738 +Ref: 12092580738 +Node: Operator precedence2581253 +Ref: reference/expressions operator-precedence2581345 +Ref: 120a2581345 +Ref: reference/expressions operator-summary2581345 +Ref: 120b2581345 +Ref: Operator precedence-Footnote-12585896 +Ref: Operator precedence-Footnote-22585994 +Node: Simple statements2586147 +Ref: reference/simple_stmts doc2586278 +Ref: 120c2586278 +Ref: reference/simple_stmts simple2586278 +Ref: 120d2586278 +Ref: reference/simple_stmts simple-statements2586278 +Ref: 120e2586278 +Ref: reference/simple_stmts grammar-token-simple-stmt2586502 +Ref: 120f2586502 +Node: Expression statements2587447 +Ref: reference/simple_stmts expression-statements2587552 +Ref: 12102587552 +Ref: reference/simple_stmts exprstmts2587552 +Ref: 12112587552 +Ref: reference/simple_stmts grammar-token-expression-stmt2587942 +Ref: 12122587942 +Node: Assignment statements2588369 +Ref: reference/simple_stmts assignment2588503 +Ref: 12132588503 +Ref: reference/simple_stmts assignment-statements2588503 +Ref: 12142588503 +Ref: reference/simple_stmts grammar-token-assignment-stmt2588674 +Ref: 12152588674 +Ref: reference/simple_stmts grammar-token-target-list2588758 +Ref: 12162588758 +Ref: reference/simple_stmts grammar-token-target2588810 +Ref: 12172588810 +Ref: reference/simple_stmts attr-target-note2592450 +Ref: 12182592450 +Ref: Assignment statements-Footnote-12596507 +Node: Augmented assignment statements2596556 +Ref: reference/simple_stmts augassign2596685 +Ref: 12192596685 +Ref: reference/simple_stmts augmented-assignment-statements2596685 +Ref: 121a2596685 +Ref: reference/simple_stmts grammar-token-augmented-assignment-stmt2596882 +Ref: 121b2596882 +Ref: reference/simple_stmts grammar-token-augtarget2596970 +Ref: 121c2596970 +Ref: reference/simple_stmts grammar-token-augop2597056 +Ref: 121d2597056 +Node: Annotated assignment statements2598798 +Ref: reference/simple_stmts annassign2598927 +Ref: 121e2598927 +Ref: reference/simple_stmts annotated-assignment-statements2598927 +Ref: 121f2598927 +Ref: reference/simple_stmts grammar-token-annotated-assignment-stmt2599162 +Ref: 12202599162 +Ref: Annotated assignment statements-Footnote-12601109 +Ref: Annotated assignment statements-Footnote-22601158 +Node: The assert statement2601207 +Ref: reference/simple_stmts assert2601338 +Ref: e062601338 +Ref: reference/simple_stmts the-assert-statement2601338 +Ref: 12212601338 +Ref: reference/simple_stmts grammar-token-assert-stmt2601488 +Ref: 12222601488 +Node: The pass statement2602572 +Ref: reference/simple_stmts pass2602702 +Ref: ed42602702 +Ref: reference/simple_stmts the-pass-statement2602702 +Ref: 12242602702 +Ref: reference/simple_stmts grammar-token-pass-stmt2602761 +Ref: 12252602761 +Node: The del statement<2>2603111 +Ref: reference/simple_stmts del2603241 +Ref: f022603241 +Ref: reference/simple_stmts the-del-statement2603241 +Ref: 12262603241 +Ref: reference/simple_stmts grammar-token-del-stmt2603298 +Ref: 12272603298 +Node: The return statement2604218 +Ref: reference/simple_stmts return2604349 +Ref: 1902604349 +Ref: reference/simple_stmts the-return-statement2604349 +Ref: 12282604349 +Ref: reference/simple_stmts grammar-token-return-stmt2604412 +Ref: 12292604412 +Node: The yield statement2605537 +Ref: reference/simple_stmts the-yield-statement2605667 +Ref: 122a2605667 +Ref: reference/simple_stmts yield2605667 +Ref: 18f2605667 +Ref: reference/simple_stmts grammar-token-yield-stmt2605728 +Ref: 122b2605728 +Node: The raise statement2606563 +Ref: reference/simple_stmts raise2606692 +Ref: c422606692 +Ref: reference/simple_stmts the-raise-statement2606692 +Ref: 122c2606692 +Ref: reference/simple_stmts grammar-token-raise-stmt2606753 +Ref: 122d2606753 +Node: The break statement2610087 +Ref: reference/simple_stmts break2610219 +Ref: 2db2610219 +Ref: reference/simple_stmts the-break-statement2610219 +Ref: 122e2610219 +Ref: reference/simple_stmts grammar-token-break-stmt2610280 +Ref: 122f2610280 +Node: The continue statement2610873 +Ref: reference/simple_stmts continue2611006 +Ref: 1812611006 +Ref: reference/simple_stmts the-continue-statement2611006 +Ref: 12302611006 +Ref: reference/simple_stmts grammar-token-continue-stmt2611075 +Ref: 12312611075 +Ref: reference/simple_stmts import2611540 +Ref: c1d2611540 +Node: The import statement2611542 +Ref: reference/simple_stmts from2611676 +Ref: c452611676 +Ref: reference/simple_stmts the-import-statement2611676 +Ref: 12322611676 +Ref: reference/simple_stmts grammar-token-import-stmt2611741 +Ref: 12332611741 +Ref: reference/simple_stmts grammar-token-module2612198 +Ref: 12342612198 +Ref: reference/simple_stmts grammar-token-relative-module2612252 +Ref: 12352612252 +Node: Future statements2617501 +Ref: reference/simple_stmts future2617575 +Ref: 12362617575 +Ref: reference/simple_stmts future-statements2617575 +Ref: 12372617575 +Ref: reference/simple_stmts grammar-token-future-stmt2618101 +Ref: 12382618101 +Ref: reference/simple_stmts grammar-token-feature2618377 +Ref: 12392618377 +Ref: Future statements-Footnote-12621094 +Ref: Future statements-Footnote-22621143 +Node: The global statement2621192 +Ref: reference/simple_stmts global2621326 +Ref: ed82621326 +Ref: reference/simple_stmts the-global-statement2621326 +Ref: 123a2621326 +Ref: reference/simple_stmts grammar-token-global-stmt2621391 +Ref: 123b2621391 +Node: The nonlocal statement2622936 +Ref: reference/simple_stmts nonlocal2623041 +Ref: c3f2623041 +Ref: reference/simple_stmts the-nonlocal-statement2623041 +Ref: 123c2623041 +Ref: reference/simple_stmts grammar-token-nonlocal-stmt2623110 +Ref: 123d2623110 +Ref: The nonlocal statement-Footnote-12624084 +Node: Compound statements2624133 +Ref: reference/compound_stmts doc2624273 +Ref: 123e2624273 +Ref: reference/compound_stmts compound2624273 +Ref: 123f2624273 +Ref: reference/compound_stmts compound-statements2624273 +Ref: 12402624273 +Ref: reference/compound_stmts grammar-token-compound-stmt2626041 +Ref: 12412626041 +Ref: reference/compound_stmts grammar-token-suite2626398 +Ref: 12422626398 +Ref: reference/compound_stmts grammar-token-statement2626474 +Ref: 12432626474 +Ref: reference/compound_stmts grammar-token-stmt-list2626531 +Ref: 12442626531 +Ref: reference/compound_stmts if2627056 +Ref: de72627056 +Ref: reference/compound_stmts elif2627056 +Ref: ec72627056 +Node: The if statement2627255 +Ref: reference/compound_stmts else2627355 +Ref: ec82627355 +Ref: reference/compound_stmts the-if-statement2627355 +Ref: 12452627355 +Ref: reference/compound_stmts grammar-token-if-stmt2627475 +Ref: 12462627475 +Node: The while statement2628022 +Ref: reference/compound_stmts the-while-statement2628148 +Ref: 12472628148 +Ref: reference/compound_stmts while2628148 +Ref: ec12628148 +Ref: reference/compound_stmts grammar-token-while-stmt2628307 +Ref: 12482628307 +Node: The for statement2628907 +Ref: reference/compound_stmts for2629034 +Ref: c302629034 +Ref: reference/compound_stmts the-for-statement2629034 +Ref: 12492629034 +Ref: reference/compound_stmts grammar-token-for-stmt2629233 +Ref: 124a2629233 +Ref: reference/compound_stmts try2631993 +Ref: d722631993 +Ref: reference/compound_stmts except2631993 +Ref: b3e2631993 +Node: The try statement2631994 +Ref: reference/compound_stmts finally2632120 +Ref: 1822632120 +Ref: reference/compound_stmts the-try-statement2632120 +Ref: 124b2632120 +Ref: reference/compound_stmts grammar-token-try-stmt2632284 +Ref: 124c2632284 +Ref: reference/compound_stmts grammar-token-try1-stmt2632325 +Ref: 124d2632325 +Ref: reference/compound_stmts grammar-token-try2-stmt2632511 +Ref: 124e2632511 +Ref: reference/compound_stmts with2637785 +Ref: 6e92637785 +Ref: The try statement-Footnote-12637821 +Node: The with statement2638023 +Ref: reference/compound_stmts as2638152 +Ref: 124f2638152 +Ref: reference/compound_stmts the-with-statement2638152 +Ref: 12502638152 +Ref: reference/compound_stmts grammar-token-with-stmt2638521 +Ref: 12512638521 +Ref: reference/compound_stmts grammar-token-with-item2638584 +Ref: 12522638584 +Ref: reference/compound_stmts function2641439 +Ref: ef02641439 +Ref: The with statement-Footnote-12641475 +Node: Function definitions2641524 +Ref: reference/compound_stmts def2641653 +Ref: dbe2641653 +Ref: reference/compound_stmts function-definitions2641653 +Ref: 12532641653 +Ref: reference/compound_stmts grammar-token-funcdef2641826 +Ref: 12542641826 +Ref: reference/compound_stmts grammar-token-decorators2641977 +Ref: 12552641977 +Ref: reference/compound_stmts grammar-token-decorator2642023 +Ref: 12562642023 +Ref: reference/compound_stmts grammar-token-dotted-name2642114 +Ref: 12572642114 +Ref: reference/compound_stmts grammar-token-parameter-list2642178 +Ref: 12582642178 +Ref: reference/compound_stmts grammar-token-parameter-list-no-posonly2642353 +Ref: 12592642353 +Ref: reference/compound_stmts grammar-token-parameter-list-starargs2642514 +Ref: 125a2642514 +Ref: reference/compound_stmts grammar-token-parameter2642672 +Ref: 125b2642672 +Ref: reference/compound_stmts grammar-token-defparameter2642735 +Ref: 125c2642735 +Ref: reference/compound_stmts grammar-token-funcname2642797 +Ref: 125d2642797 +Ref: Function definitions-Footnote-12648489 +Ref: Function definitions-Footnote-22648687 +Ref: Function definitions-Footnote-32648736 +Ref: Function definitions-Footnote-42648785 +Ref: Function definitions-Footnote-52648834 +Node: Class definitions2648883 +Ref: reference/compound_stmts class2649007 +Ref: c6a2649007 +Ref: reference/compound_stmts class-definitions2649007 +Ref: 125f2649007 +Ref: reference/compound_stmts grammar-token-classdef2649155 +Ref: 12602649155 +Ref: reference/compound_stmts grammar-token-inheritance2649231 +Ref: 12612649231 +Ref: reference/compound_stmts grammar-token-classname2649276 +Ref: 12622649276 +Ref: Class definitions-Footnote-12651993 +Ref: Class definitions-Footnote-22652181 +Ref: Class definitions-Footnote-32652230 +Ref: Class definitions-Footnote-42652279 +Node: Coroutines<2>2652328 +Ref: reference/compound_stmts async2652423 +Ref: 1a22652423 +Ref: reference/compound_stmts coroutines2652423 +Ref: 12632652423 +Node: Coroutine function definition2652580 +Ref: reference/compound_stmts async-def2652691 +Ref: 2842652691 +Ref: reference/compound_stmts coroutine-function-definition2652691 +Ref: 12642652691 +Ref: reference/compound_stmts grammar-token-async-funcdef2652768 +Ref: 12652652768 +Node: The async for statement2653626 +Ref: reference/compound_stmts async-for2653770 +Ref: 2e32653770 +Ref: reference/compound_stmts the-async-for-statement2653770 +Ref: 12662653770 +Ref: reference/compound_stmts grammar-token-async-for-stmt2653843 +Ref: 12672653843 +Node: The async with statement2654771 +Ref: reference/compound_stmts async-with2654877 +Ref: 6432654877 +Ref: reference/compound_stmts the-async-with-statement2654877 +Ref: 12682654877 +Ref: reference/compound_stmts grammar-token-async-with-stmt2654952 +Ref: 12692654952 +Ref: The async with statement-Footnote-12656083 +Node: Top-level components2656132 +Ref: reference/toplevel_components doc2656281 +Ref: 126a2656281 +Ref: reference/toplevel_components top-level2656281 +Ref: 126b2656281 +Ref: reference/toplevel_components top-level-components2656281 +Ref: 126c2656281 +Node: Complete Python programs2656674 +Ref: reference/toplevel_components complete-python-programs2656774 +Ref: 126d2656774 +Ref: reference/toplevel_components programs2656774 +Ref: 116e2656774 +Node: File input2658156 +Ref: reference/toplevel_components file-input2658282 +Ref: 126e2658282 +Ref: reference/toplevel_components id12658282 +Ref: 126f2658282 +Ref: reference/toplevel_components grammar-token-file-input2658379 +Ref: 12702658379 +Node: Interactive input2658655 +Ref: reference/toplevel_components interactive2658773 +Ref: 12712658773 +Ref: reference/toplevel_components interactive-input2658773 +Ref: 12722658773 +Ref: reference/toplevel_components grammar-token-interactive-input2658888 +Ref: 12732658888 +Node: Expression input2659121 +Ref: reference/toplevel_components expression-input2659220 +Ref: 12742659220 +Ref: reference/toplevel_components id22659220 +Ref: 12752659220 +Ref: reference/toplevel_components grammar-token-eval-input2659422 +Ref: 12762659422 +Node: Full Grammar specification2659468 +Ref: reference/grammar doc2659589 +Ref: 12772659589 +Ref: reference/grammar full-grammar-specification2659589 +Ref: 12782659589 +Node: The Python Standard Library2670581 +Ref: library/index doc2670741 +Ref: 12792670741 +Ref: library/index library-index2670741 +Ref: e8e2670741 +Ref: library/index the-python-standard-library2670741 +Ref: 127a2670741 +Ref: The Python Standard Library-Footnote-12673432 +Node: Introduction<6>2673457 +Ref: library/intro doc2673563 +Ref: 127b2673563 +Ref: library/intro introduction2673563 +Ref: 127c2673563 +Ref: library/intro library-intro2673563 +Ref: 127d2673563 +Node: Notes on availability2676246 +Ref: library/intro availability2676319 +Ref: ffb2676319 +Ref: library/intro notes-on-availability2676319 +Ref: 127e2676319 +Node: Built-in Functions2676708 +Ref: library/functions doc2676841 +Ref: 127f2676841 +Ref: library/functions built-in-funcs2676841 +Ref: bf32676841 +Ref: library/functions built-in-functions2676841 +Ref: 12802676841 +Ref: library/functions abs2681402 +Ref: f542681402 +Ref: library/functions all2681689 +Ref: d852681689 +Ref: library/functions any2682002 +Ref: d842682002 +Ref: library/functions ascii2682326 +Ref: d4c2682326 +Ref: library/functions bin2682684 +Ref: c402682684 +Ref: library/functions bool2683352 +Ref: 1832683352 +Ref: library/functions breakpoint2683943 +Ref: 2f92683943 +Ref: library/functions func-bytearray2684773 +Ref: 12872684773 +Ref: library/functions func-bytes2686160 +Ref: 10d92686160 +Ref: library/functions callable2686859 +Ref: b442686859 +Ref: library/functions chr2687412 +Ref: 10c72687412 +Ref: library/functions classmethod2687868 +Ref: 1d82687868 +Ref: library/functions compile2688860 +Ref: 1b42688860 +Ref: library/functions complex2693079 +Ref: 1892693079 +Ref: library/functions delattr2694595 +Ref: 12812694595 +Ref: library/functions func-dict2694964 +Ref: 12842694964 +Ref: library/functions dir2695421 +Ref: d332695421 +Ref: library/functions divmod2698160 +Ref: 1522698160 +Ref: library/functions enumerate2698801 +Ref: de32698801 +Ref: library/functions eval2699694 +Ref: b902699694 +Ref: library/functions exec2702018 +Ref: c442702018 +Ref: library/functions filter2704623 +Ref: c2e2704623 +Ref: library/functions float2705424 +Ref: 1872705424 +Ref: library/functions grammar-token-sign2706044 +Ref: 12972706044 +Ref: library/functions grammar-token-infinity2706083 +Ref: 12982706083 +Ref: library/functions grammar-token-nan2706131 +Ref: 12992706131 +Ref: library/functions grammar-token-numeric-value2706166 +Ref: 129a2706166 +Ref: library/functions grammar-token-numeric-string2706224 +Ref: 129b2706224 +Ref: library/functions format2707741 +Ref: 4db2707741 +Ref: library/functions func-frozenset2708832 +Ref: 12892708832 +Ref: library/functions getattr2709307 +Ref: 4482709307 +Ref: library/functions globals2709770 +Ref: 128c2709770 +Ref: library/functions hasattr2710046 +Ref: 4472710046 +Ref: library/functions hash2710385 +Ref: 9c42710385 +Ref: library/functions help2710971 +Ref: 5722710971 +Ref: library/functions hex2712023 +Ref: c662712023 +Ref: library/functions id2713090 +Ref: d862713090 +Ref: library/functions input2713546 +Ref: c6b2713546 +Ref: library/functions int2714400 +Ref: 1842714400 +Ref: library/functions isinstance2716476 +Ref: 44f2716476 +Ref: library/functions issubclass2717064 +Ref: 4502717064 +Ref: library/functions iter2717450 +Ref: d272717450 +Ref: library/functions len2718823 +Ref: 1502718823 +Ref: library/functions func-list2719056 +Ref: 128a2719056 +Ref: library/functions locals2719277 +Ref: 4552719277 +Ref: library/functions map2719804 +Ref: c2d2719804 +Ref: library/functions max2720330 +Ref: 80a2720330 +Ref: library/functions func-memoryview2721524 +Ref: 12822721524 +Ref: library/functions min2721684 +Ref: 8092721684 +Ref: library/functions next2722869 +Ref: 6822722869 +Ref: library/functions object2723137 +Ref: 2b02723137 +Ref: library/functions oct2723556 +Ref: c652723556 +Ref: library/functions open2724328 +Ref: 4f02724328 +Ref: library/functions filemodes2725820 +Ref: 12a02725820 +Ref: library/functions open-newline-parameter2730995 +Ref: 12a22730995 +Ref: library/functions ord2736058 +Ref: 10c62736058 +Ref: library/functions pow2736380 +Ref: 19a2736380 +Ref: library/functions print2738007 +Ref: 8862738007 +Ref: library/functions property2739193 +Ref: 1d72739193 +Ref: library/functions func-range2741901 +Ref: 128b2741901 +Ref: library/functions repr2742161 +Ref: 7cc2742161 +Ref: library/functions reversed2742768 +Ref: 18e2742768 +Ref: library/functions round2743083 +Ref: c6d2743083 +Ref: library/functions func-set2744392 +Ref: 12832744392 +Ref: library/functions setattr2744850 +Ref: 12852744850 +Ref: library/functions slice2745267 +Ref: e042745267 +Ref: library/functions sorted2746048 +Ref: 4442746048 +Ref: library/functions staticmethod2747136 +Ref: 1d92747136 +Ref: library/functions func-str2748374 +Ref: 12862748374 +Ref: library/functions sum2748705 +Ref: 1e52748705 +Ref: library/functions super2749414 +Ref: 4e22749414 +Ref: library/functions func-tuple2753130 +Ref: 12882753130 +Ref: library/functions type2753357 +Ref: 6082753357 +Ref: library/functions vars2755051 +Ref: f2d2755051 +Ref: library/functions zip2755946 +Ref: c322755946 +Ref: library/functions __import__2757889 +Ref: 6102757889 +Ref: Built-in Functions-Footnote-12761064 +Ref: Built-in Functions-Footnote-22761274 +Ref: Built-in Functions-Footnote-32761323 +Ref: Built-in Functions-Footnote-42761400 +Ref: Built-in Functions-Footnote-52761449 +Node: Built-in Constants2761498 +Ref: library/constants doc2761630 +Ref: 12b32761630 +Ref: library/constants built-in-constants2761630 +Ref: 12b42761630 +Ref: library/constants built-in-consts2761630 +Ref: 12b52761630 +Ref: library/constants False2761749 +Ref: 3882761749 +Ref: library/constants True2761899 +Ref: 4992761899 +Ref: library/constants None2762046 +Ref: 1572762046 +Ref: library/constants NotImplemented2762325 +Ref: 84c2762325 +Ref: library/constants Ellipsis2763658 +Ref: 12b62763658 +Ref: library/constants __debug__2763854 +Ref: fdd2763854 +Node: Constants added by the site module2764323 +Ref: library/constants constants-added-by-the-site-module2764412 +Ref: 12b72764412 +Ref: library/constants quit2764780 +Ref: 12b82764780 +Ref: library/constants exit2764807 +Ref: 12b92764807 +Ref: library/constants copyright2765023 +Ref: 12ba2765023 +Ref: library/constants credits2765043 +Ref: 12bb2765043 +Ref: library/constants license2765164 +Ref: 12bc2765164 +Node: Built-in Types2765391 +Ref: library/stdtypes doc2765524 +Ref: 12bd2765524 +Ref: library/stdtypes bltin-types2765524 +Ref: 12be2765524 +Ref: library/stdtypes built-in-types2765524 +Ref: 12bf2765524 +Node: Truth Value Testing2766946 +Ref: library/stdtypes truth2767058 +Ref: 128d2767058 +Ref: library/stdtypes truth-value-testing2767058 +Ref: 12c02767058 +Ref: Truth Value Testing-Footnote-12768130 +Node: Boolean Operations — and or not2768270 +Ref: library/stdtypes boolean2768405 +Ref: 12c12768405 +Ref: library/stdtypes boolean-operations-and-or-not2768405 +Ref: 12c22768405 +Node: Comparisons<2>2769734 +Ref: library/stdtypes comparisons2769885 +Ref: 12c32769885 +Ref: library/stdtypes stdcomparisons2769885 +Ref: 12c42769885 +Node: Numeric Types — int float complex2772329 +Ref: library/stdtypes numeric-types-int-float-complex2772461 +Ref: 12c52772461 +Ref: library/stdtypes typesnumeric2772461 +Ref: eb32772461 +Ref: Numeric Types — int float complex-Footnote-12780503 +Node: Bitwise Operations on Integer Types2780622 +Ref: library/stdtypes bitstring-ops2780773 +Ref: 12c72780773 +Ref: library/stdtypes bitwise-operations-on-integer-types2780773 +Ref: 12c82780773 +Node: Additional Methods on Integer Types2783127 +Ref: library/stdtypes additional-methods-on-integer-types2783314 +Ref: 12c92783314 +Ref: library/stdtypes int bit_length2783540 +Ref: 12ca2783540 +Ref: library/stdtypes int to_bytes2784455 +Ref: 12cb2784455 +Ref: library/stdtypes int from_bytes2785865 +Ref: 12cd2785865 +Ref: library/stdtypes int as_integer_ratio2787077 +Ref: 1862787077 +Node: Additional Methods on Float2787391 +Ref: library/stdtypes additional-methods-on-float2787567 +Ref: 12ce2787567 +Ref: library/stdtypes float as_integer_ratio2787779 +Ref: fb92787779 +Ref: library/stdtypes float is_integer2788035 +Ref: 12cf2788035 +Ref: library/stdtypes float hex2788662 +Ref: fba2788662 +Ref: library/stdtypes float fromhex2788912 +Ref: 12d02788912 +Node: Hashing of numeric types2790550 +Ref: library/stdtypes hashing-of-numeric-types2790682 +Ref: 12d12790682 +Ref: library/stdtypes numeric-hash2790682 +Ref: 12d22790682 +Node: Iterator Types2794940 +Ref: library/stdtypes iterator-types2795093 +Ref: 12d32795093 +Ref: library/stdtypes typeiter2795093 +Ref: 10fe2795093 +Ref: library/stdtypes container __iter__2795475 +Ref: 12d42795475 +Ref: library/stdtypes iterator __iter__2796209 +Ref: 12d52796209 +Ref: library/stdtypes iterator __next__2796547 +Ref: c642796547 +Node: Generator Types2797317 +Ref: library/stdtypes generator-types2797383 +Ref: 12d62797383 +Ref: library/stdtypes id32797383 +Ref: 12d72797383 +Node: Sequence Types — list tuple range2797878 +Ref: library/stdtypes sequence-types-list-tuple-range2798022 +Ref: 12d82798022 +Ref: library/stdtypes typesseq2798022 +Ref: f042798022 +Node: Common Sequence Operations2798488 +Ref: library/stdtypes common-sequence-operations2798619 +Ref: 12d92798619 +Ref: library/stdtypes typesseq-common2798619 +Ref: 12da2798619 +Ref: Common Sequence Operations-Footnote-12807030 +Node: Immutable Sequence Types2807113 +Ref: library/stdtypes immutable-sequence-types2807275 +Ref: 12df2807275 +Ref: library/stdtypes typesseq-immutable2807275 +Ref: 12e02807275 +Node: Mutable Sequence Types2807808 +Ref: library/stdtypes mutable-sequence-types2807952 +Ref: 12e12807952 +Ref: library/stdtypes typesseq-mutable2807952 +Ref: 128f2807952 +Node: Lists<2>2813615 +Ref: library/stdtypes lists2813741 +Ref: 12e22813741 +Ref: library/stdtypes typesseq-list2813741 +Ref: 129e2813741 +Ref: library/stdtypes list2813924 +Ref: 2622813924 +Ref: library/stdtypes list sort2815111 +Ref: 4452815111 +Node: Tuples2817428 +Ref: library/stdtypes tuples2817538 +Ref: 12e32817538 +Ref: library/stdtypes typesseq-tuple2817538 +Ref: 12b12817538 +Ref: library/stdtypes tuple2817901 +Ref: 47e2817901 +Node: Ranges2819456 +Ref: library/stdtypes ranges2819549 +Ref: 12e42819549 +Ref: library/stdtypes typesseq-range2819549 +Ref: 12a92819549 +Ref: library/stdtypes range2819738 +Ref: 9be2819738 +Ref: library/stdtypes range start2821708 +Ref: 12e52821708 +Ref: library/stdtypes range stop2821838 +Ref: 12e62821838 +Ref: library/stdtypes range step2821909 +Ref: 12e72821909 +Ref: Ranges-Footnote-12823915 +Node: Text Sequence Type — str2823967 +Ref: library/stdtypes text-sequence-type-str2824149 +Ref: 12e82824149 +Ref: library/stdtypes textseq2824149 +Ref: eb72824149 +Ref: library/stdtypes str2825815 +Ref: 3302825815 +Node: String Methods<2>2827950 +Ref: library/stdtypes id52828069 +Ref: 12ea2828069 +Ref: library/stdtypes string-methods2828069 +Ref: eb82828069 +Ref: library/stdtypes str capitalize2828910 +Ref: 12ed2828910 +Ref: library/stdtypes str casefold2829276 +Ref: 6b02829276 +Ref: library/stdtypes str center2829896 +Ref: f312829896 +Ref: library/stdtypes str count2830168 +Ref: 12ef2830168 +Ref: library/stdtypes str encode2830404 +Ref: c372830404 +Ref: library/stdtypes str endswith2831142 +Ref: 7c72831142 +Ref: library/stdtypes str expandtabs2831477 +Ref: 12f02831477 +Ref: library/stdtypes str find2832592 +Ref: 79f2832592 +Ref: library/stdtypes str format2833141 +Ref: 4da2833141 +Ref: library/stdtypes str format_map2834653 +Ref: 6b12834653 +Ref: library/stdtypes str index2835164 +Ref: 12f12835164 +Ref: library/stdtypes str isalnum2835310 +Ref: 12f22835310 +Ref: library/stdtypes str isalpha2835650 +Ref: 12f32835650 +Ref: library/stdtypes str isascii2836148 +Ref: 3332836148 +Ref: library/stdtypes str isdecimal2836388 +Ref: 12f42836388 +Ref: library/stdtypes str isdigit2836776 +Ref: 12f52836776 +Ref: library/stdtypes str isidentifier2837284 +Ref: 12f62837284 +Ref: library/stdtypes str islower2837837 +Ref: 12f82837837 +Ref: library/stdtypes str isnumeric2838021 +Ref: 12f92838021 +Ref: library/stdtypes str isprintable2838501 +Ref: 6b22838501 +Ref: library/stdtypes str isspace2839119 +Ref: 12fa2839119 +Ref: library/stdtypes str istitle2839541 +Ref: 12fb2839541 +Ref: library/stdtypes str isupper2839822 +Ref: 12fc2839822 +Ref: library/stdtypes str join2840006 +Ref: 12dd2840006 +Ref: library/stdtypes str ljust2840334 +Ref: f302840334 +Ref: library/stdtypes str lower2840621 +Ref: 12ee2840621 +Ref: library/stdtypes str lstrip2840840 +Ref: 12fd2840840 +Ref: library/stdtypes str maketrans2841354 +Ref: 6b32841354 +Ref: library/stdtypes str partition2842083 +Ref: 7a22842083 +Ref: library/stdtypes str replace2842422 +Ref: 12ff2842422 +Ref: library/stdtypes str rfind2842663 +Ref: 7a02842663 +Ref: library/stdtypes str rindex2842960 +Ref: 13002842960 +Ref: library/stdtypes str rjust2843115 +Ref: f2f2843115 +Ref: library/stdtypes str rpartition2843403 +Ref: 13012843403 +Ref: library/stdtypes str rsplit2843742 +Ref: 13022843742 +Ref: library/stdtypes str rstrip2844192 +Ref: 13032844192 +Ref: library/stdtypes str split2844696 +Ref: 7a12844696 +Ref: library/stdtypes str splitlines2846355 +Ref: 13042846355 +Ref: library/stdtypes str startswith2849001 +Ref: 7c62849001 +Ref: library/stdtypes str strip2849338 +Ref: 13052849338 +Ref: library/stdtypes str swapcase2850346 +Ref: 13062850346 +Ref: library/stdtypes str title2850563 +Ref: 13072850563 +Ref: library/stdtypes str translate2851616 +Ref: 12fe2851616 +Ref: library/stdtypes str upper2852521 +Ref: 13092852521 +Ref: library/stdtypes str zfill2852986 +Ref: f322852986 +Ref: String Methods<2>-Footnote-12853507 +Ref: String Methods<2>-Footnote-22853683 +Ref: String Methods<2>-Footnote-32853859 +Ref: String Methods<2>-Footnote-42854035 +Node: printf-style String Formatting2854211 +Ref: library/stdtypes old-string-formatting2854330 +Ref: eb92854330 +Ref: library/stdtypes printf-style-string-formatting2854330 +Ref: 130a2854330 +Ref: printf-style String Formatting-Footnote-12863541 +Ref: printf-style String Formatting-Footnote-22863672 +Node: Binary Sequence Types — bytes bytearray memoryview2863721 +Ref: library/stdtypes binary-sequence-types-bytes-bytearray-memoryview2863895 +Ref: 130c2863895 +Ref: library/stdtypes binaryseq2863895 +Ref: 12922863895 +Node: Bytes Objects2864587 +Ref: library/stdtypes bytes-objects2864715 +Ref: 130d2864715 +Ref: library/stdtypes typebytes2864715 +Ref: 12942864715 +Ref: library/stdtypes bytes2865055 +Ref: 3312865055 +Ref: library/stdtypes bytes fromhex2867310 +Ref: 32e2867310 +Ref: library/stdtypes bytes hex2867917 +Ref: 6312867917 +Node: Bytearray Objects2870114 +Ref: library/stdtypes bytearray-objects2870281 +Ref: 130e2870281 +Ref: library/stdtypes typebytearray2870281 +Ref: 12932870281 +Ref: library/stdtypes bytearray2870421 +Ref: 3322870421 +Ref: library/stdtypes bytearray fromhex2871459 +Ref: 32f2871459 +Ref: library/stdtypes bytearray hex2872095 +Ref: 6322872095 +Node: Bytes and Bytearray Operations2873146 +Ref: library/stdtypes bytes-and-bytearray-operations2873329 +Ref: 130f2873329 +Ref: library/stdtypes bytes-methods2873329 +Ref: 12902873329 +Ref: library/stdtypes bytes count2874547 +Ref: 13102874547 +Ref: library/stdtypes bytearray count2874593 +Ref: 13112874593 +Ref: library/stdtypes bytes decode2875056 +Ref: c382875056 +Ref: library/stdtypes bytearray decode2875117 +Ref: 13122875117 +Ref: library/stdtypes bytes endswith2876002 +Ref: 13132876002 +Ref: library/stdtypes bytearray endswith2876054 +Ref: 13142876054 +Ref: library/stdtypes bytes find2876481 +Ref: 13152876481 +Ref: library/stdtypes bytearray find2876526 +Ref: 13162876526 +Ref: library/stdtypes bytes index2877332 +Ref: 13172877332 +Ref: library/stdtypes bytearray index2877378 +Ref: 13182877378 +Ref: library/stdtypes bytes join2877752 +Ref: 12de2877752 +Ref: library/stdtypes bytearray join2877786 +Ref: 13192877786 +Ref: library/stdtypes bytes maketrans2878222 +Ref: c092878222 +Ref: library/stdtypes bytearray maketrans2878268 +Ref: c0a2878268 +Ref: library/stdtypes bytes partition2878634 +Ref: 131b2878634 +Ref: library/stdtypes bytearray partition2878668 +Ref: 131c2878668 +Ref: library/stdtypes bytes replace2879155 +Ref: 131d2879155 +Ref: library/stdtypes bytearray replace2879201 +Ref: 131e2879201 +Ref: library/stdtypes bytes rfind2879721 +Ref: 131f2879721 +Ref: library/stdtypes bytearray rfind2879767 +Ref: 13202879767 +Ref: library/stdtypes bytes rindex2880297 +Ref: 13212880297 +Ref: library/stdtypes bytearray rindex2880344 +Ref: 13222880344 +Ref: library/stdtypes bytes rpartition2880727 +Ref: 13232880727 +Ref: library/stdtypes bytearray rpartition2880762 +Ref: 13242880762 +Ref: library/stdtypes bytes startswith2881249 +Ref: 13252881249 +Ref: library/stdtypes bytearray startswith2881303 +Ref: 13262881303 +Ref: library/stdtypes bytes translate2881739 +Ref: 131a2881739 +Ref: library/stdtypes bytearray translate2881790 +Ref: c912881790 +Ref: library/stdtypes bytes center2882811 +Ref: 13272882811 +Ref: library/stdtypes bytearray center2882856 +Ref: 13282882856 +Ref: library/stdtypes bytes ljust2883357 +Ref: 13292883357 +Ref: library/stdtypes bytearray ljust2883401 +Ref: 132a2883401 +Ref: library/stdtypes bytes lstrip2883907 +Ref: 132b2883907 +Ref: library/stdtypes bytearray lstrip2883942 +Ref: 132c2883942 +Ref: library/stdtypes bytes rjust2884843 +Ref: 132d2884843 +Ref: library/stdtypes bytearray rjust2884887 +Ref: 132e2884887 +Ref: library/stdtypes bytes rsplit2885394 +Ref: 132f2885394 +Ref: library/stdtypes bytearray rsplit2885443 +Ref: 13302885443 +Ref: library/stdtypes bytes rstrip2885957 +Ref: 13322885957 +Ref: library/stdtypes bytearray rstrip2885992 +Ref: 13332885992 +Ref: library/stdtypes bytes split2886883 +Ref: 13342886883 +Ref: library/stdtypes bytearray split2886931 +Ref: 13312886931 +Ref: library/stdtypes bytes strip2888854 +Ref: 13352888854 +Ref: library/stdtypes bytearray strip2888888 +Ref: 13362888888 +Ref: library/stdtypes bytes capitalize2890075 +Ref: 13372890075 +Ref: library/stdtypes bytearray capitalize2890107 +Ref: 13382890107 +Ref: library/stdtypes bytes expandtabs2890516 +Ref: 13392890516 +Ref: library/stdtypes bytearray expandtabs2890557 +Ref: 133a2890557 +Ref: library/stdtypes bytes isalnum2891899 +Ref: 133b2891899 +Ref: library/stdtypes bytearray isalnum2891928 +Ref: 133c2891928 +Ref: library/stdtypes bytes isalpha2892490 +Ref: 133d2892490 +Ref: library/stdtypes bytearray isalpha2892519 +Ref: 133e2892519 +Ref: library/stdtypes bytes isascii2892956 +Ref: 133f2892956 +Ref: library/stdtypes bytearray isascii2892985 +Ref: 13402892985 +Ref: library/stdtypes bytes isdigit2893200 +Ref: 13412893200 +Ref: library/stdtypes bytearray isdigit2893229 +Ref: 13422893229 +Ref: library/stdtypes bytes islower2893601 +Ref: 13432893601 +Ref: library/stdtypes bytearray islower2893630 +Ref: 13442893630 +Ref: library/stdtypes bytes isspace2894168 +Ref: 13452894168 +Ref: library/stdtypes bytearray isspace2894197 +Ref: 13462894197 +Ref: library/stdtypes bytes istitle2894529 +Ref: 13472894529 +Ref: library/stdtypes bytearray istitle2894558 +Ref: 13482894558 +Ref: library/stdtypes bytes isupper2894931 +Ref: 134a2894931 +Ref: library/stdtypes bytearray isupper2894960 +Ref: 134b2894960 +Ref: library/stdtypes bytes lower2895514 +Ref: 134c2895514 +Ref: library/stdtypes bytearray lower2895541 +Ref: 134d2895541 +Ref: library/stdtypes bytes splitlines2896180 +Ref: 134e2896180 +Ref: library/stdtypes bytearray splitlines2896226 +Ref: 134f2896226 +Ref: library/stdtypes bytes swapcase2897175 +Ref: 13502897175 +Ref: library/stdtypes bytearray swapcase2897205 +Ref: 13512897205 +Ref: library/stdtypes bytes title2898135 +Ref: 13492898135 +Ref: library/stdtypes bytearray title2898162 +Ref: 13522898162 +Ref: library/stdtypes bytes upper2899787 +Ref: 13532899787 +Ref: library/stdtypes bytearray upper2899814 +Ref: 13542899814 +Ref: library/stdtypes bytes zfill2900453 +Ref: 13552900453 +Ref: library/stdtypes bytearray zfill2900485 +Ref: 13562900485 +Node: printf-style Bytes Formatting2901195 +Ref: library/stdtypes bytes-formatting2901373 +Ref: 6502901373 +Ref: library/stdtypes printf-style-bytes-formatting2901373 +Ref: 13572901373 +Ref: printf-style Bytes Formatting-Footnote-12910866 +Ref: printf-style Bytes Formatting-Footnote-22910997 +Ref: printf-style Bytes Formatting-Footnote-32911046 +Node: Memory Views2911095 +Ref: library/stdtypes memory-views2911234 +Ref: 13582911234 +Ref: library/stdtypes typememoryview2911234 +Ref: 129f2911234 +Ref: library/stdtypes memoryview2911432 +Ref: 25c2911432 +Ref: library/stdtypes memoryview __eq__2915196 +Ref: 135c2915196 +Ref: library/stdtypes memoryview tobytes2917198 +Ref: 135d2917198 +Ref: library/stdtypes memoryview hex2918301 +Ref: 6332918301 +Ref: library/stdtypes memoryview tolist2918822 +Ref: 13592918822 +Ref: library/stdtypes memoryview toreadonly2919384 +Ref: 135e2919384 +Ref: library/stdtypes memoryview release2919996 +Ref: b3d2919996 +Ref: library/stdtypes memoryview cast2921362 +Ref: 135f2921362 +Ref: library/stdtypes memoryview obj2924381 +Ref: 13612924381 +Ref: library/stdtypes memoryview nbytes2924619 +Ref: 13622924619 +Ref: library/stdtypes memoryview readonly2925745 +Ref: 13632925745 +Ref: library/stdtypes memoryview format2925837 +Ref: 135b2925837 +Ref: library/stdtypes memoryview itemsize2926375 +Ref: 135a2926375 +Ref: library/stdtypes memoryview ndim2926751 +Ref: 13642926751 +Ref: library/stdtypes memoryview shape2926892 +Ref: 13652926892 +Ref: library/stdtypes memoryview strides2927147 +Ref: 13662927147 +Ref: library/stdtypes memoryview suboffsets2927437 +Ref: 13672927437 +Ref: library/stdtypes memoryview c_contiguous2927562 +Ref: 13682927562 +Ref: library/stdtypes memoryview f_contiguous2927714 +Ref: 13692927714 +Ref: library/stdtypes memoryview contiguous2927872 +Ref: 136a2927872 +Ref: Memory Views-Footnote-12928056 +Node: Set Types — set frozenset2928105 +Ref: library/stdtypes set-types-set-frozenset2928275 +Ref: 136b2928275 +Ref: library/stdtypes types-set2928275 +Ref: 129c2928275 +Ref: library/stdtypes set2929825 +Ref: b6f2929825 +Ref: library/stdtypes frozenset2929853 +Ref: bee2929853 +Ref: library/stdtypes frozenset isdisjoint2930889 +Ref: 136c2930889 +Ref: library/stdtypes frozenset issubset2931097 +Ref: 136d2931097 +Ref: library/stdtypes frozenset issuperset2931376 +Ref: 136e2931376 +Ref: library/stdtypes frozenset union2931659 +Ref: 136f2931659 +Ref: library/stdtypes frozenset intersection2931800 +Ref: 13702931800 +Ref: library/stdtypes frozenset difference2931963 +Ref: 13712931963 +Ref: library/stdtypes frozenset symmetric_difference2932129 +Ref: 13722932129 +Ref: library/stdtypes frozenset copy2932301 +Ref: 13732932301 +Ref: library/stdtypes frozenset update2934542 +Ref: 13742934542 +Ref: library/stdtypes frozenset intersection_update2934674 +Ref: 13752934674 +Ref: library/stdtypes frozenset difference_update2934846 +Ref: 13762934846 +Ref: library/stdtypes frozenset symmetric_difference_update2934991 +Ref: 13772934991 +Ref: library/stdtypes frozenset add2935173 +Ref: 13782935173 +Ref: library/stdtypes frozenset remove2935244 +Ref: 13792935244 +Ref: library/stdtypes frozenset discard2935401 +Ref: 137a2935401 +Ref: library/stdtypes frozenset pop2935498 +Ref: 137b2935498 +Ref: library/stdtypes frozenset clear2935647 +Ref: 137c2935647 +Node: Mapping Types — dict2936207 +Ref: library/stdtypes mapping-types-dict2936346 +Ref: 137d2936346 +Ref: library/stdtypes typesmapping2936346 +Ref: 2fc2936346 +Ref: library/stdtypes dict2937579 +Ref: 1b82937579 +Ref: library/stdtypes dict clear2941895 +Ref: 137e2941895 +Ref: library/stdtypes dict copy2941971 +Ref: 4462941971 +Ref: library/stdtypes dict fromkeys2942049 +Ref: 137f2942049 +Ref: library/stdtypes dict get2942591 +Ref: 13802942591 +Ref: library/stdtypes dict items2942846 +Ref: c2b2942846 +Ref: library/stdtypes dict keys2943020 +Ref: c2a2943020 +Ref: library/stdtypes dict pop2943165 +Ref: 13822943165 +Ref: library/stdtypes dict popitem2943418 +Ref: 13832943418 +Ref: library/stdtypes dict setdefault2944191 +Ref: 9bf2944191 +Ref: library/stdtypes dict update2944421 +Ref: dc92944421 +Ref: library/stdtypes dict values2944905 +Ref: c2c2944905 +Node: Dictionary view objects2947012 +Ref: library/stdtypes dict-views2947094 +Ref: 13812947094 +Ref: library/stdtypes dictionary-view-objects2947094 +Ref: 13842947094 +Node: Context Manager Types2949996 +Ref: library/stdtypes context-manager-types2950128 +Ref: 13862950128 +Ref: library/stdtypes typecontextmanager2950128 +Ref: 112a2950128 +Ref: library/stdtypes contextmanager __enter__2950493 +Ref: 13872950493 +Ref: library/stdtypes contextmanager __exit__2951492 +Ref: 13892951492 +Node: Other Built-in Types2953878 +Ref: library/stdtypes other-built-in-types2954006 +Ref: 138a2954006 +Ref: library/stdtypes typesother2954006 +Ref: 138b2954006 +Node: Modules<2>2954411 +Ref: library/stdtypes modules2954514 +Ref: 138c2954514 +Ref: library/stdtypes typesmodules2954514 +Ref: 138d2954514 +Node: Classes and Class Instances2955667 +Ref: library/stdtypes classes-and-class-instances2955788 +Ref: 138e2955788 +Ref: library/stdtypes typesobjects2955788 +Ref: 138f2955788 +Node: Functions2955952 +Ref: library/stdtypes functions2956070 +Ref: 13902956070 +Ref: library/stdtypes typesfunctions2956070 +Ref: 13912956070 +Node: Methods2956533 +Ref: library/stdtypes methods2956636 +Ref: 13922956636 +Ref: library/stdtypes typesmethods2956636 +Ref: 13932956636 +Node: Code Objects2958407 +Ref: library/stdtypes bltin-code-objects2958513 +Ref: 13942958513 +Ref: library/stdtypes code-objects2958513 +Ref: 13952958513 +Node: Type Objects2959219 +Ref: library/stdtypes bltin-type-objects2959333 +Ref: 12b22959333 +Ref: library/stdtypes type-objects2959333 +Ref: 13962959333 +Node: The Null Object2959684 +Ref: library/stdtypes bltin-null-object2959805 +Ref: 13972959805 +Ref: library/stdtypes the-null-object2959805 +Ref: 13982959805 +Node: The Ellipsis Object2960120 +Ref: library/stdtypes bltin-ellipsis-object2960254 +Ref: 13992960254 +Ref: library/stdtypes the-ellipsis-object2960254 +Ref: 139a2960254 +Node: The NotImplemented Object2960622 +Ref: library/stdtypes bltin-notimplemented-object2960755 +Ref: 139b2960755 +Ref: library/stdtypes the-notimplemented-object2960755 +Ref: 139c2960755 +Node: Boolean Values2961158 +Ref: library/stdtypes bltin-boolean-values2961288 +Ref: 128e2961288 +Ref: library/stdtypes boolean-values2961288 +Ref: 139d2961288 +Node: Internal Objects2961911 +Ref: library/stdtypes internal-objects2962007 +Ref: 139e2962007 +Ref: library/stdtypes typesinternal2962007 +Ref: 139f2962007 +Node: Special Attributes2962203 +Ref: library/stdtypes special-attributes2962301 +Ref: 13a02962301 +Ref: library/stdtypes specialattrs2962301 +Ref: 13a12962301 +Ref: library/stdtypes object __dict__2962540 +Ref: 4fc2962540 +Ref: library/stdtypes instance __class__2962671 +Ref: e0a2962671 +Ref: library/stdtypes class __bases__2962757 +Ref: e092962757 +Ref: library/stdtypes definition __name__2962840 +Ref: c672962840 +Ref: library/stdtypes definition __qualname__2962963 +Ref: 10c92962963 +Ref: library/stdtypes class __mro__2963139 +Ref: 12af2963139 +Ref: library/stdtypes class mro2963293 +Ref: 13a22963293 +Ref: library/stdtypes class __subclasses__2963527 +Ref: 13a32963527 +Node: Built-in Exceptions2963790 +Ref: library/exceptions doc2963929 +Ref: 13a42963929 +Ref: library/exceptions bltin-exceptions2963929 +Ref: f432963929 +Ref: library/exceptions built-in-exceptions2963929 +Ref: 13a52963929 +Node: Base classes2967216 +Ref: library/exceptions base-classes2967312 +Ref: 13a62967312 +Ref: library/exceptions BaseException2967431 +Ref: 1a82967431 +Ref: library/exceptions BaseException args2967816 +Ref: 13a72967816 +Ref: library/exceptions BaseException with_traceback2968184 +Ref: 13a82968184 +Ref: library/exceptions Exception2968593 +Ref: 1a92968593 +Ref: library/exceptions ArithmeticError2968777 +Ref: 13a92968777 +Ref: library/exceptions BufferError2969005 +Ref: 13ab2969005 +Ref: library/exceptions LookupError2969117 +Ref: 13082969117 +Node: Concrete exceptions2969383 +Ref: library/exceptions concrete-exceptions2969496 +Ref: 13ad2969496 +Ref: library/exceptions AssertionError2969619 +Ref: 12232969619 +Ref: library/exceptions AttributeError2969707 +Ref: 39b2969707 +Ref: library/exceptions EOFError2969977 +Ref: c6c2969977 +Ref: library/exceptions FloatingPointError2970254 +Ref: 13aa2970254 +Ref: library/exceptions GeneratorExit2970315 +Ref: d322970315 +Ref: library/exceptions ImportError2970637 +Ref: 3342970637 +Ref: library/exceptions ModuleNotFoundError2971212 +Ref: 39a2971212 +Ref: library/exceptions IndexError2971474 +Ref: e752971474 +Ref: library/exceptions KeyError2971701 +Ref: 2c72971701 +Ref: library/exceptions KeyboardInterrupt2971818 +Ref: 1972971818 +Ref: library/exceptions MemoryError2972208 +Ref: 13af2972208 +Ref: library/exceptions NameError2972789 +Ref: d7b2972789 +Ref: library/exceptions NotImplementedError2973011 +Ref: 60e2973011 +Ref: library/exceptions OSError2973855 +Ref: 1d32973855 +Ref: library/exceptions OSError errno2974892 +Ref: 13b12974892 +Ref: library/exceptions OSError winerror2974984 +Ref: 13b22974984 +Ref: library/exceptions OSError strerror2975563 +Ref: 13b32975563 +Ref: library/exceptions OSError filename2975799 +Ref: 13b42975799 +Ref: library/exceptions OSError filename22975828 +Ref: 13b52975828 +Ref: library/exceptions OverflowError2976806 +Ref: 9602976806 +Ref: library/exceptions RecursionError2977299 +Ref: 6342977299 +Ref: library/exceptions ReferenceError2977621 +Ref: e4d2977621 +Ref: library/exceptions RuntimeError2977947 +Ref: 2ba2977947 +Ref: library/exceptions StopIteration2978149 +Ref: 4862978149 +Ref: library/exceptions StopAsyncIteration2979480 +Ref: 10cd2979480 +Ref: library/exceptions SyntaxError2979672 +Ref: 4582979672 +Ref: library/exceptions IndentationError2980194 +Ref: e782980194 +Ref: library/exceptions TabError2980347 +Ref: e772980347 +Ref: library/exceptions SystemError2980508 +Ref: 5fb2980508 +Ref: library/exceptions SystemExit2981151 +Ref: 5fc2981151 +Ref: library/exceptions SystemExit code2982447 +Ref: 13b72982447 +Ref: library/exceptions TypeError2982589 +Ref: 1922982589 +Ref: library/exceptions UnboundLocalError2983418 +Ref: e762983418 +Ref: library/exceptions UnicodeError2983636 +Ref: c3b2983636 +Ref: library/exceptions UnicodeError encoding2983997 +Ref: 13b82983997 +Ref: library/exceptions UnicodeError reason2984086 +Ref: 13b92984086 +Ref: library/exceptions UnicodeError object2984171 +Ref: 13ba2984171 +Ref: library/exceptions UnicodeError start2984267 +Ref: 13bb2984267 +Ref: library/exceptions UnicodeError end2984359 +Ref: 13bc2984359 +Ref: library/exceptions UnicodeEncodeError2984455 +Ref: 1fc2984455 +Ref: library/exceptions UnicodeDecodeError2984607 +Ref: 1fd2984607 +Ref: library/exceptions UnicodeTranslateError2984759 +Ref: 13bd2984759 +Ref: library/exceptions ValueError2984917 +Ref: 1fb2984917 +Ref: library/exceptions ZeroDivisionError2985169 +Ref: f422985169 +Ref: library/exceptions EnvironmentError2985526 +Ref: 9932985526 +Ref: library/exceptions IOError2985559 +Ref: 9922985559 +Ref: library/exceptions WindowsError2985583 +Ref: 9942985583 +Ref: Concrete exceptions-Footnote-12985709 +Ref: Concrete exceptions-Footnote-22985758 +Node: OS exceptions2985807 +Ref: library/exceptions os-exceptions2985876 +Ref: 13b02985876 +Ref: library/exceptions BlockingIOError2986037 +Ref: 9972986037 +Ref: library/exceptions BlockingIOError characters_written2986393 +Ref: 13be2986393 +Ref: library/exceptions ChildProcessError2986648 +Ref: 9982986648 +Ref: library/exceptions ConnectionError2986787 +Ref: 6e62986787 +Ref: library/exceptions BrokenPipeError2987035 +Ref: 99d2987035 +Ref: library/exceptions ConnectionAbortedError2987334 +Ref: 99e2987334 +Ref: library/exceptions ConnectionRefusedError2987536 +Ref: 99f2987536 +Ref: library/exceptions ConnectionResetError2987738 +Ref: 9a02987738 +Ref: library/exceptions FileExistsError2987921 +Ref: 9582987921 +Ref: library/exceptions FileNotFoundError2988077 +Ref: 7c02988077 +Ref: library/exceptions InterruptedError2988229 +Ref: 6592988229 +Ref: library/exceptions IsADirectoryError2988635 +Ref: 9992988635 +Ref: library/exceptions NotADirectoryError2988814 +Ref: 99a2988814 +Ref: library/exceptions PermissionError2989028 +Ref: 5ff2989028 +Ref: library/exceptions ProcessLookupError2989246 +Ref: 99b2989246 +Ref: library/exceptions TimeoutError2989378 +Ref: 99c2989378 +Ref: OS exceptions-Footnote-12989715 +Ref: OS exceptions-Footnote-22989764 +Node: Warnings2989813 +Ref: library/exceptions warning-categories-as-exceptions2989933 +Ref: 13bf2989933 +Ref: library/exceptions warnings2989933 +Ref: 13c02989933 +Ref: library/exceptions Warning2990094 +Ref: 13c22990094 +Ref: library/exceptions UserWarning2990159 +Ref: 13c32990159 +Ref: library/exceptions DeprecationWarning2990241 +Ref: 2782990241 +Ref: library/exceptions PendingDeprecationWarning2990399 +Ref: 2792990399 +Ref: library/exceptions SyntaxWarning2990793 +Ref: 1912990793 +Ref: library/exceptions RuntimeWarning2990875 +Ref: 2c02990875 +Ref: library/exceptions FutureWarning2990968 +Ref: 3252990968 +Ref: library/exceptions ImportWarning2991155 +Ref: 5d82991155 +Ref: library/exceptions UnicodeWarning2991258 +Ref: d872991258 +Ref: library/exceptions BytesWarning2991339 +Ref: 4b52991339 +Ref: library/exceptions ResourceWarning2991458 +Ref: 4d02991458 +Node: Exception hierarchy2991619 +Ref: library/exceptions exception-hierarchy2991711 +Ref: 13c42991711 +Ref: library/text stringservices2993955 +Ref: 12e92993955 +Node: Text Processing Services2993956 +Ref: library/text doc2994101 +Ref: 13c52994101 +Ref: library/text text-processing-services2994101 +Ref: 13c62994101 +Ref: library/text textservices2994101 +Ref: 12ec2994101 +Node: string — Common string operations2994888 +Ref: library/string doc2995029 +Ref: 13c82995029 +Ref: library/string module-string2995029 +Ref: f62995029 +Ref: library/string string-common-string-operations2995029 +Ref: 13c92995029 +Ref: string — Common string operations-Footnote-12995471 +Node: String constants2995536 +Ref: library/string string-constants2995657 +Ref: 13ca2995657 +Ref: library/string string ascii_letters2995751 +Ref: c5c2995751 +Ref: library/string string ascii_lowercase2995950 +Ref: 13cb2995950 +Ref: library/string string ascii_uppercase2996110 +Ref: 13cc2996110 +Ref: library/string string digits2996270 +Ref: 13cd2996270 +Ref: library/string string hexdigits2996332 +Ref: 13ce2996332 +Ref: library/string string octdigits2996409 +Ref: 13cf2996409 +Ref: library/string string punctuation2996472 +Ref: 13d02996472 +Ref: library/string string printable2996647 +Ref: 13d12996647 +Ref: library/string string whitespace2996875 +Ref: 13d22996875 +Node: Custom String Formatting2997081 +Ref: library/string custom-string-formatting2997231 +Ref: 13d32997231 +Ref: library/string string-formatting2997231 +Ref: 12eb2997231 +Ref: library/string string Formatter2997678 +Ref: 7b52997678 +Ref: library/string string Formatter format2997779 +Ref: 48c2997779 +Ref: library/string string Formatter vformat2998140 +Ref: 13d42998140 +Ref: library/string string Formatter parse2998851 +Ref: 13d52998851 +Ref: library/string string Formatter get_field2999607 +Ref: 13d62999607 +Ref: library/string string Formatter get_value3000156 +Ref: 13d73000156 +Ref: library/string string Formatter check_unused_args3001392 +Ref: 13d83001392 +Ref: library/string string Formatter format_field3001996 +Ref: 13d93001996 +Ref: library/string string Formatter convert_field3002222 +Ref: 13da3002222 +Ref: Custom String Formatting-Footnote-13002597 +Ref: Custom String Formatting-Footnote-23002646 +Node: Format String Syntax3002695 +Ref: library/string format-string-syntax3002845 +Ref: 13db3002845 +Ref: library/string formatstrings3002845 +Ref: d1a3002845 +Ref: library/string grammar-token-replacement-field3003646 +Ref: 13dc3003646 +Ref: library/string grammar-token-field-name3003734 +Ref: 13dd3003734 +Ref: library/string grammar-token-arg-name3003821 +Ref: 13de3003821 +Ref: library/string grammar-token-attribute-name3003875 +Ref: 13df3003875 +Ref: library/string grammar-token-element-index3003918 +Ref: 13e03003918 +Ref: library/string grammar-token-index-string3003972 +Ref: 13e13003972 +Ref: library/string grammar-token-conversion3004040 +Ref: 13e23004040 +Ref: library/string grammar-token-format-spec3004088 +Ref: 13e33004088 +Node: Format Specification Mini-Language3008085 +Ref: library/string format-specification-mini-language3008200 +Ref: 13e53008200 +Ref: library/string formatspec3008200 +Ref: 4de3008200 +Ref: library/string id13009097 +Ref: 13e63009097 +Ref: library/string grammar-token-fill3009190 +Ref: 13e73009190 +Ref: library/string grammar-token-align3009231 +Ref: 13e83009231 +Ref: library/string grammar-token-sign3009278 +Ref: 13e93009278 +Ref: library/string grammar-token-width3009319 +Ref: 13ea3009319 +Ref: library/string grammar-token-grouping-option3009351 +Ref: 13eb3009351 +Ref: library/string grammar-token-precision3009386 +Ref: 13ec3009386 +Ref: library/string grammar-token-type3009418 +Ref: 13ed3009418 +Ref: Format Specification Mini-Language-Footnote-13022169 +Ref: Format Specification Mini-Language-Footnote-23022218 +Node: Format examples3022267 +Ref: library/string format-examples3022382 +Ref: 13ee3022382 +Ref: library/string formatexamples3022382 +Ref: 13e43022382 +Node: Template strings3026910 +Ref: library/string id23027052 +Ref: 13ef3027052 +Ref: library/string template-strings3027052 +Ref: 130b3027052 +Ref: library/string string Template3028513 +Ref: 3eb3028513 +Ref: library/string string Template substitute3028633 +Ref: f943028633 +Ref: library/string string Template safe_substitute3029118 +Ref: f953029118 +Ref: library/string string Template template3030152 +Ref: 13f03030152 +Ref: Template strings-Footnote-13033928 +Ref: Template strings-Footnote-23033977 +Node: Helper functions3034028 +Ref: library/string helper-functions3034141 +Ref: 13f13034141 +Ref: library/string string capwords3034192 +Ref: 13f23034192 +Node: re — Regular expression operations3034678 +Ref: library/re doc3034868 +Ref: 13f33034868 +Ref: library/re module-re3034868 +Ref: dd3034868 +Ref: library/re re-regular-expression-operations3034868 +Ref: 13f43034868 +Ref: re — Regular expression operations-Footnote-13037442 +Ref: re — Regular expression operations-Footnote-23037503 +Node: Regular Expression Syntax3037543 +Ref: library/re re-syntax3037665 +Ref: 13f53037665 +Ref: library/re regular-expression-syntax3037665 +Ref: 13f63037665 +Ref: Regular Expression Syntax-Footnote-13061278 +Node: Module Contents3061320 +Ref: library/re contents-of-module-re3061477 +Ref: 14013061477 +Ref: library/re module-contents3061477 +Ref: 14023061477 +Ref: library/re re compile3061892 +Ref: 44a3061892 +Ref: library/re re A3063020 +Ref: 13fb3063020 +Ref: library/re re ASCII3063035 +Ref: 3c83063035 +Ref: library/re re DEBUG3063664 +Ref: 14063063664 +Ref: library/re re I3063779 +Ref: 13fc3063779 +Ref: library/re re IGNORECASE3063794 +Ref: 14073063794 +Ref: library/re re L3064783 +Ref: 13fd3064783 +Ref: library/re re LOCALE3064798 +Ref: 3c93064798 +Ref: library/re re M3065739 +Ref: 13fe3065739 +Ref: library/re re MULTILINE3065754 +Ref: 13fa3065754 +Ref: library/re re S3066339 +Ref: 13ff3066339 +Ref: library/re re DOTALL3066354 +Ref: 13f93066354 +Ref: library/re re X3066590 +Ref: 14003066590 +Ref: library/re re VERBOSE3066605 +Ref: 14083066605 +Ref: library/re re search3067621 +Ref: bb23067621 +Ref: library/re re match3068025 +Ref: bb33068025 +Ref: library/re re fullmatch3068676 +Ref: 89b3068676 +Ref: library/re re split3069005 +Ref: 3ca3069005 +Ref: library/re re findall3070874 +Ref: 9633070874 +Ref: library/re re finditer3071417 +Ref: 140a3071417 +Ref: library/re re sub3071855 +Ref: 47b3071855 +Ref: library/re re subn3075036 +Ref: 7343075036 +Ref: library/re re escape3075372 +Ref: 4953075372 +Ref: library/re re purge3076858 +Ref: 140b3076858 +Ref: library/re re error3076927 +Ref: 7353076927 +Ref: library/re re error msg3077360 +Ref: 7363077360 +Ref: library/re re error pattern3077427 +Ref: 7373077427 +Ref: library/re re error pos3077499 +Ref: 7383077499 +Ref: library/re re error lineno3077614 +Ref: 7393077614 +Ref: library/re re error colno3077706 +Ref: 73a3077706 +Node: Regular Expression Objects3077858 +Ref: library/re re-objects3078003 +Ref: 89c3078003 +Ref: library/re regular-expression-objects3078003 +Ref: 140c3078003 +Ref: library/re re Pattern search3078157 +Ref: 14053078157 +Ref: library/re re Pattern match3079621 +Ref: 14043079621 +Ref: library/re re Pattern fullmatch3080480 +Ref: 140d3080480 +Ref: library/re re Pattern split3081277 +Ref: 140e3081277 +Ref: library/re re Pattern findall3081411 +Ref: 140f3081411 +Ref: library/re re Pattern finditer3081672 +Ref: 14103081672 +Ref: library/re re Pattern sub3081936 +Ref: 14113081936 +Ref: library/re re Pattern subn3082069 +Ref: 14123082069 +Ref: library/re re Pattern flags3082204 +Ref: 14133082204 +Ref: library/re re Pattern groups3082471 +Ref: 14143082471 +Ref: library/re re Pattern groupindex3082555 +Ref: 14153082555 +Ref: library/re re Pattern pattern3082769 +Ref: 14163082769 +Node: Match Objects3083028 +Ref: library/re id23083185 +Ref: 14173083185 +Ref: library/re match-objects3083185 +Ref: 89d3083185 +Ref: library/re re Match expand3083598 +Ref: 14183083598 +Ref: library/re re Match group3084120 +Ref: 14193084120 +Ref: library/re re Match __getitem__3086388 +Ref: 141a3086388 +Ref: library/re re Match groups3086863 +Ref: 141b3086863 +Ref: library/re re Match groupdict3087709 +Ref: 141c3087709 +Ref: library/re re Match start3088176 +Ref: 141d3088176 +Ref: library/re re Match end3088210 +Ref: 141e3088210 +Ref: library/re re Match span3089257 +Ref: 141f3089257 +Ref: library/re re Match pos3089510 +Ref: 14203089510 +Ref: library/re re Match endpos3089770 +Ref: 14213089770 +Ref: library/re re Match lastindex3090024 +Ref: 14223090024 +Ref: library/re re Match lastgroup3090423 +Ref: 14233090423 +Ref: library/re re Match re3090596 +Ref: 14243090596 +Ref: library/re re Match string3090763 +Ref: 14253090763 +Node: Regular Expression Examples3091001 +Ref: library/re re-examples3091123 +Ref: bf43091123 +Ref: library/re regular-expression-examples3091123 +Ref: 14263091123 +Node: Checking for a Pair3091467 +Ref: library/re checking-for-a-pair3091575 +Ref: 14273091575 +Node: Simulating scanf3093672 +Ref: library/re simulating-scanf3093804 +Ref: 14283093804 +Node: search vs match3095872 +Ref: library/re id33096003 +Ref: 14293096003 +Ref: library/re search-vs-match3096003 +Ref: 14093096003 +Node: Making a Phonebook3097217 +Ref: library/re making-a-phonebook3097344 +Ref: 142a3097344 +Node: Text Munging3099523 +Ref: library/re text-munging3099654 +Ref: 142b3099654 +Node: Finding all Adverbs3100478 +Ref: library/re finding-all-adverbs3100630 +Ref: 142c3100630 +Node: Finding all Adverbs and their Positions3101085 +Ref: library/re finding-all-adverbs-and-their-positions3101244 +Ref: 142d3101244 +Node: Raw String Notation3101946 +Ref: library/re raw-string-notation3102105 +Ref: 142e3102105 +Node: Writing a Tokenizer3103032 +Ref: library/re writing-a-tokenizer3103143 +Ref: 142f3103143 +Ref: library/re frie093106514 +Ref: 13f73106514 +Ref: Writing a Tokenizer-Footnote-13106794 +Node: difflib — Helpers for computing deltas3106849 +Ref: library/difflib doc3107042 +Ref: 14303107042 +Ref: library/difflib difflib-helpers-for-computing-deltas3107042 +Ref: 14313107042 +Ref: library/difflib module-difflib3107042 +Ref: 373107042 +Ref: library/difflib difflib Differ3109539 +Ref: 14323109539 +Ref: library/difflib difflib HtmlDiff3110666 +Ref: 14333110666 +Ref: library/difflib difflib HtmlDiff __init__3111036 +Ref: 14343111036 +Ref: library/difflib difflib HtmlDiff make_file3111849 +Ref: 6c93111849 +Ref: library/difflib difflib HtmlDiff make_table3113532 +Ref: 14363113532 +Ref: library/difflib difflib context_diff3114114 +Ref: 14373114114 +Ref: library/difflib difflib get_close_matches3116056 +Ref: 143b3116056 +Ref: library/difflib difflib ndiff3117220 +Ref: 14353117220 +Ref: library/difflib difflib restore3118870 +Ref: 143e3118870 +Ref: library/difflib difflib unified_diff3119613 +Ref: 14403119613 +Ref: library/difflib difflib diff_bytes3121532 +Ref: 6ca3121532 +Ref: library/difflib difflib IS_LINE_JUNK3122424 +Ref: 143c3122424 +Ref: library/difflib difflib IS_CHARACTER_JUNK3122726 +Ref: 143d3122726 +Ref: difflib — Helpers for computing deltas-Footnote-13123386 +Ref: difflib — Helpers for computing deltas-Footnote-23123452 +Ref: difflib — Helpers for computing deltas-Footnote-33123541 +Node: SequenceMatcher Objects3123573 +Ref: library/difflib sequence-matcher3123706 +Ref: 14413123706 +Ref: library/difflib sequencematcher-objects3123706 +Ref: 14423123706 +Ref: library/difflib difflib SequenceMatcher3123832 +Ref: 94f3123832 +Ref: library/difflib difflib SequenceMatcher set_seqs3125356 +Ref: 14433125356 +Ref: library/difflib difflib SequenceMatcher set_seq13125774 +Ref: 14453125774 +Ref: library/difflib difflib SequenceMatcher set_seq23125917 +Ref: 14443125917 +Ref: library/difflib difflib SequenceMatcher find_longest_match3126060 +Ref: 14463126060 +Ref: library/difflib difflib SequenceMatcher get_matching_blocks3128143 +Ref: 14473128143 +Ref: library/difflib difflib SequenceMatcher get_opcodes3129060 +Ref: 14483129060 +Ref: library/difflib difflib SequenceMatcher get_grouped_opcodes3131047 +Ref: 14493131047 +Ref: library/difflib difflib SequenceMatcher ratio3131479 +Ref: 144a3131479 +Ref: library/difflib difflib SequenceMatcher quick_ratio3132502 +Ref: 144b3132502 +Ref: library/difflib difflib SequenceMatcher real_quick_ratio3132622 +Ref: 144c3132622 +Node: SequenceMatcher Examples3133133 +Ref: library/difflib id13133289 +Ref: 144d3133289 +Ref: library/difflib sequencematcher-examples3133289 +Ref: 144e3133289 +Ref: SequenceMatcher Examples-Footnote-13134994 +Node: Differ Objects3135047 +Ref: library/difflib differ-objects3135194 +Ref: 144f3135194 +Ref: library/difflib id23135194 +Ref: 14503135194 +Ref: library/difflib difflib Differ compare3136600 +Ref: 143f3136600 +Node: Differ Example3137124 +Ref: library/difflib differ-example3137282 +Ref: 14513137282 +Ref: library/difflib differ-examples3137282 +Ref: 14523137282 +Node: A command-line interface to difflib3139540 +Ref: library/difflib a-command-line-interface-to-difflib3139675 +Ref: 14533139675 +Ref: library/difflib difflib-interface3139675 +Ref: 143a3139675 +Node: textwrap — Text wrapping and filling3142430 +Ref: library/textwrap doc3142619 +Ref: 14543142619 +Ref: library/textwrap module-textwrap3142619 +Ref: 1083142619 +Ref: library/textwrap textwrap-text-wrapping-and-filling3142619 +Ref: 14553142619 +Ref: library/textwrap textwrap wrap3143151 +Ref: 14563143151 +Ref: library/textwrap textwrap fill3143652 +Ref: 14583143652 +Ref: library/textwrap textwrap shorten3144012 +Ref: 8ed3144012 +Ref: library/textwrap textwrap dedent3145147 +Ref: 145d3145147 +Ref: library/textwrap textwrap indent3146083 +Ref: aa83146083 +Ref: library/textwrap textwrap TextWrapper3147419 +Ref: 8ea3147419 +Ref: library/textwrap textwrap TextWrapper width3148095 +Ref: 145f3148095 +Ref: library/textwrap textwrap TextWrapper expand_tabs3148417 +Ref: 145a3148417 +Ref: library/textwrap textwrap TextWrapper tabsize3148620 +Ref: 14593148620 +Ref: library/textwrap textwrap TextWrapper replace_whitespace3148908 +Ref: 145c3148908 +Ref: library/textwrap textwrap TextWrapper drop_whitespace3149871 +Ref: 145b3149871 +Ref: library/textwrap textwrap TextWrapper initial_indent3150285 +Ref: 14603150285 +Ref: library/textwrap textwrap TextWrapper subsequent_indent3150520 +Ref: 14613150520 +Ref: library/textwrap textwrap TextWrapper fix_sentence_endings3150747 +Ref: 14623150747 +Ref: library/textwrap textwrap TextWrapper break_long_words3151968 +Ref: 145e3151968 +Ref: library/textwrap textwrap TextWrapper break_on_hyphens3152458 +Ref: 14633152458 +Ref: library/textwrap textwrap TextWrapper max_lines3152990 +Ref: 8eb3152990 +Ref: library/textwrap textwrap TextWrapper placeholder3153235 +Ref: 8ec3153235 +Ref: library/textwrap textwrap TextWrapper wrap3153549 +Ref: 14573153549 +Ref: library/textwrap textwrap TextWrapper fill3153971 +Ref: 14643153971 +Ref: textwrap — Text wrapping and filling-Footnote-13154158 +Node: unicodedata — Unicode Database3154225 +Ref: library/unicodedata doc3154416 +Ref: 14653154416 +Ref: library/unicodedata module-unicodedata3154416 +Ref: 11a3154416 +Ref: library/unicodedata unicodedata-unicode-database3154416 +Ref: 14663154416 +Ref: library/unicodedata unicodedata lookup3154938 +Ref: 9bd3154938 +Ref: library/unicodedata unicodedata name3155255 +Ref: 14673155255 +Ref: library/unicodedata unicodedata decimal3155481 +Ref: 14683155481 +Ref: library/unicodedata unicodedata digit3155724 +Ref: 14693155724 +Ref: library/unicodedata unicodedata numeric3155962 +Ref: bf93155962 +Ref: library/unicodedata unicodedata category3156202 +Ref: 146a3156202 +Ref: library/unicodedata unicodedata bidirectional3156327 +Ref: 146b3156327 +Ref: library/unicodedata unicodedata combining3156519 +Ref: 146c3156519 +Ref: library/unicodedata unicodedata east_asian_width3156706 +Ref: 146d3156706 +Ref: library/unicodedata unicodedata mirrored3156839 +Ref: 146e3156839 +Ref: library/unicodedata unicodedata decomposition3157097 +Ref: 146f3157097 +Ref: library/unicodedata unicodedata normalize3157312 +Ref: 11ed3157312 +Ref: library/unicodedata unicodedata is_normalized3159174 +Ref: 2463159174 +Ref: library/unicodedata unicodedata unidata_version3159481 +Ref: 14703159481 +Ref: library/unicodedata unicodedata ucd_3_2_03159583 +Ref: d9c3159583 +Ref: unicodedata — Unicode Database-Footnote-13160362 +Ref: unicodedata — Unicode Database-Footnote-23160411 +Ref: unicodedata — Unicode Database-Footnote-33160457 +Ref: unicodedata — Unicode Database-Footnote-43160532 +Node: stringprep — Internet String Preparation3160610 +Ref: library/stringprep doc3160798 +Ref: 14713160798 +Ref: library/stringprep module-stringprep3160798 +Ref: f73160798 +Ref: library/stringprep stringprep-internet-string-preparation3160798 +Ref: 14723160798 +Ref: library/stringprep stringprep in_table_a13162720 +Ref: 14733162720 +Ref: library/stringprep stringprep in_table_b13162857 +Ref: 14743162857 +Ref: library/stringprep stringprep map_table_b23162983 +Ref: 14753162983 +Ref: library/stringprep stringprep map_table_b33163140 +Ref: 14763163140 +Ref: library/stringprep stringprep in_table_c113163309 +Ref: 14773163309 +Ref: library/stringprep stringprep in_table_c123163429 +Ref: 14783163429 +Ref: library/stringprep stringprep in_table_c11_c123163558 +Ref: 14793163558 +Ref: library/stringprep stringprep in_table_c213163705 +Ref: 147a3163705 +Ref: library/stringprep stringprep in_table_c223163832 +Ref: 147b3163832 +Ref: library/stringprep stringprep in_table_c21_c223163963 +Ref: 147c3163963 +Ref: library/stringprep stringprep in_table_c33164112 +Ref: 147d3164112 +Ref: library/stringprep stringprep in_table_c43164218 +Ref: 147e3164218 +Ref: library/stringprep stringprep in_table_c53164343 +Ref: 147f3164343 +Ref: library/stringprep stringprep in_table_c63164453 +Ref: 14803164453 +Ref: library/stringprep stringprep in_table_c73164581 +Ref: 14813164581 +Ref: library/stringprep stringprep in_table_c83164723 +Ref: 14823164723 +Ref: library/stringprep stringprep in_table_c93164866 +Ref: 14833164866 +Ref: library/stringprep stringprep in_table_d13164979 +Ref: 14843164979 +Ref: library/stringprep stringprep in_table_d23165137 +Ref: 14853165137 +Ref: stringprep — Internet String Preparation-Footnote-13165319 +Ref: stringprep — Internet String Preparation-Footnote-23165388 +Ref: stringprep — Internet String Preparation-Footnote-33165437 +Node: readline — GNU readline interface3165486 +Ref: library/readline doc3165694 +Ref: 14863165694 +Ref: library/readline module-readline3165694 +Ref: de3165694 +Ref: library/readline readline-gnu-readline-interface3165694 +Ref: 14873165694 +Ref: readline — GNU readline interface-Footnote-13167615 +Node: Init file3167685 +Ref: library/readline init-file3167786 +Ref: 14883167786 +Ref: library/readline readline parse_and_bind3167896 +Ref: 14893167896 +Ref: library/readline readline read_init_file3168077 +Ref: 148a3168077 +Node: Line buffer3168299 +Ref: library/readline line-buffer3168421 +Ref: 148b3168421 +Ref: library/readline readline get_line_buffer3168515 +Ref: 148c3168515 +Ref: library/readline readline insert_text3168666 +Ref: 148d3168666 +Ref: library/readline readline redisplay3168876 +Ref: 148e3168876 +Node: History file3169081 +Ref: library/readline history-file3169206 +Ref: 148f3169206 +Ref: library/readline readline read_history_file3169301 +Ref: 14903169301 +Ref: library/readline readline write_history_file3169539 +Ref: 14913169539 +Ref: library/readline readline append_history_file3169796 +Ref: 73c3169796 +Ref: library/readline readline get_history_length3170203 +Ref: 14923170203 +Ref: library/readline readline set_history_length3170248 +Ref: 14933170248 +Node: History list3170611 +Ref: library/readline history-list3170738 +Ref: 14943170738 +Ref: library/readline readline clear_history3170840 +Ref: 14953170840 +Ref: library/readline readline get_current_history_length3171088 +Ref: 14963171088 +Ref: library/readline readline get_history_item3171353 +Ref: 14973171353 +Ref: library/readline readline remove_history_item3171563 +Ref: 14983171563 +Ref: library/readline readline replace_history_item3171783 +Ref: 14993171783 +Ref: library/readline readline add_history3172014 +Ref: 149a3172014 +Ref: library/readline readline set_auto_history3172198 +Ref: 5773172198 +Node: Startup hooks3172664 +Ref: library/readline startup-hooks3172789 +Ref: 149b3172789 +Ref: library/readline readline set_startup_hook3172834 +Ref: 149c3172834 +Ref: library/readline readline set_pre_input_hook3173238 +Ref: 149d3173238 +Node: Completion3173810 +Ref: library/readline completion3173930 +Ref: 149e3173930 +Ref: library/readline readline set_completer3174416 +Ref: 149f3174416 +Ref: library/readline readline get_completer3175207 +Ref: 14a03175207 +Ref: library/readline readline get_completion_type3175340 +Ref: 14a13175340 +Ref: library/readline readline get_begidx3175540 +Ref: 14a23175540 +Ref: library/readline readline get_endidx3175577 +Ref: 14a33175577 +Ref: library/readline readline set_completer_delims3175835 +Ref: 14a43175835 +Ref: library/readline readline get_completer_delims3175888 +Ref: 14a53175888 +Ref: library/readline readline set_completion_display_matches_hook3176209 +Ref: 14a63176209 +Node: Example3176798 +Ref: library/readline example3176896 +Ref: 14a73176896 +Ref: library/readline readline-example3176896 +Ref: 14a83176896 +Node: rlcompleter — Completion function for GNU readline3179536 +Ref: library/rlcompleter doc3179693 +Ref: 14aa3179693 +Ref: library/rlcompleter module-rlcompleter3179693 +Ref: e13179693 +Ref: library/rlcompleter rlcompleter-completion-function-for-gnu-readline3179693 +Ref: 14ab3179693 +Ref: rlcompleter — Completion function for GNU readline-Footnote-13181158 +Node: Completer Objects3181228 +Ref: library/rlcompleter completer-objects3181334 +Ref: 14ac3181334 +Ref: library/rlcompleter id13181334 +Ref: 14ad3181334 +Ref: library/rlcompleter rlcompleter Completer complete3181433 +Ref: 14ae3181433 +Node: Binary Data Services3182189 +Ref: library/binary doc3182325 +Ref: 14af3182325 +Ref: library/binary binary-data-services3182325 +Ref: 14b03182325 +Ref: library/binary binaryservices3182325 +Ref: 13c73182325 +Node: struct — Interpret bytes as packed binary data3183091 +Ref: library/struct doc3183247 +Ref: 14b13183247 +Ref: library/struct module-struct3183247 +Ref: f83183247 +Ref: library/struct struct-interpret-bytes-as-packed-binary-data3183247 +Ref: 14b23183247 +Ref: struct — Interpret bytes as packed binary data-Footnote-13185054 +Node: Functions and Exceptions3185119 +Ref: library/struct functions-and-exceptions3185251 +Ref: 14b63185251 +Ref: library/struct struct error3185377 +Ref: cb43185377 +Ref: library/struct struct pack3185503 +Ref: c0b3185503 +Ref: library/struct struct pack_into3185745 +Ref: 14b73185745 +Ref: library/struct struct unpack3186044 +Ref: f983186044 +Ref: library/struct struct unpack_from3186411 +Ref: 14b93186411 +Ref: library/struct struct iter_unpack3186804 +Ref: 8d83186804 +Ref: library/struct struct calcsize3187324 +Ref: 14b83187324 +Node: Format Strings3187523 +Ref: library/struct format-strings3187674 +Ref: 14ba3187674 +Ref: library/struct struct-format-strings3187674 +Ref: 14b33187674 +Node: Byte Order Size and Alignment3188163 +Ref: library/struct byte-order-size-and-alignment3188269 +Ref: 14bc3188269 +Ref: library/struct struct-alignment3188269 +Ref: 14b43188269 +Ref: Byte Order Size and Alignment-Footnote-13191426 +Node: Format Characters3191470 +Ref: library/struct format-characters3191596 +Ref: 14bb3191596 +Ref: library/struct id13191596 +Ref: 14be3191596 +Ref: Format Characters-Footnote-13202582 +Ref: Format Characters-Footnote-23202654 +Node: Examples<2>3202730 +Ref: library/struct examples3202818 +Ref: 14bf3202818 +Ref: library/struct struct-examples3202818 +Ref: 14bd3202818 +Node: Classes<2>3204531 +Ref: library/struct classes3204649 +Ref: 14c03204649 +Ref: library/struct struct-objects3204649 +Ref: 14c13204649 +Ref: library/struct struct Struct3204745 +Ref: 14b53204745 +Ref: library/struct struct Struct pack3205487 +Ref: 14c23205487 +Ref: library/struct struct Struct pack_into3205668 +Ref: 14c43205668 +Ref: library/struct struct Struct unpack3205826 +Ref: 14c53205826 +Ref: library/struct struct Struct unpack_from3206025 +Ref: 14c63206025 +Ref: library/struct struct Struct iter_unpack3206283 +Ref: 8d93206283 +Ref: library/struct struct Struct format3206534 +Ref: 4923206534 +Ref: library/struct struct Struct size3206749 +Ref: 14c33206749 +Node: codecs — Codec registry and base classes3206950 +Ref: library/codecs doc3207106 +Ref: 14c73207106 +Ref: library/codecs codecs-codec-registry-and-base-classes3207106 +Ref: 14c83207106 +Ref: library/codecs ietf-rfc-17003207106 +Ref: 14c93207106 +Ref: library/codecs module-codecs3207106 +Ref: 1c3207106 +Ref: library/codecs codecs encode3207967 +Ref: 8023207967 +Ref: library/codecs codecs decode3208462 +Ref: 8033208462 +Ref: library/codecs codecs lookup3209022 +Ref: 13ac3209022 +Ref: library/codecs codecs CodecInfo3209520 +Ref: 14cb3209520 +Ref: library/codecs codecs CodecInfo name3209816 +Ref: 14cc3209816 +Ref: library/codecs codecs CodecInfo encode3209879 +Ref: 14cd3209879 +Ref: library/codecs codecs CodecInfo decode3209906 +Ref: 14ce3209906 +Ref: library/codecs codecs CodecInfo incrementalencoder3210284 +Ref: 14d23210284 +Ref: library/codecs codecs CodecInfo incrementaldecoder3210323 +Ref: 14d33210323 +Ref: library/codecs codecs CodecInfo streamwriter3210662 +Ref: 14d63210662 +Ref: library/codecs codecs CodecInfo streamreader3210695 +Ref: 14d73210695 +Ref: library/codecs codecs getencoder3211149 +Ref: 14da3211149 +Ref: library/codecs codecs getdecoder3211361 +Ref: 14db3211361 +Ref: library/codecs codecs getincrementalencoder3211573 +Ref: 14dc3211573 +Ref: library/codecs codecs getincrementaldecoder3211879 +Ref: 14dd3211879 +Ref: library/codecs codecs getreader3212185 +Ref: 14de3212185 +Ref: library/codecs codecs getwriter3212431 +Ref: 14df3212431 +Ref: library/codecs codecs register3212760 +Ref: 14e03212760 +Ref: library/codecs codecs open3213564 +Ref: ffe3213564 +Ref: library/codecs codecs EncodedFile3214817 +Ref: 14e23214817 +Ref: library/codecs codecs iterencode3215674 +Ref: 14e43215674 +Ref: library/codecs codecs iterdecode3216204 +Ref: 14e53216204 +Ref: library/codecs codecs BOM3216926 +Ref: 14e63216926 +Ref: library/codecs codecs BOM_BE3216947 +Ref: 14e73216947 +Ref: library/codecs codecs BOM_LE3216971 +Ref: 14e83216971 +Ref: library/codecs codecs BOM_UTF83216995 +Ref: 14e93216995 +Ref: library/codecs codecs BOM_UTF163217021 +Ref: 14ea3217021 +Ref: library/codecs codecs BOM_UTF16_BE3217048 +Ref: 14eb3217048 +Ref: library/codecs codecs BOM_UTF16_LE3217078 +Ref: 14ec3217078 +Ref: library/codecs codecs BOM_UTF323217108 +Ref: 14ed3217108 +Ref: library/codecs codecs BOM_UTF32_BE3217135 +Ref: 14ee3217135 +Ref: library/codecs codecs BOM_UTF32_LE3217165 +Ref: 14ef3217165 +Ref: codecs — Codec registry and base classes-Footnote-13218312 +Node: Codec Base Classes3218377 +Ref: library/codecs codec-base-classes3218504 +Ref: 14ca3218504 +Ref: library/codecs id13218504 +Ref: 14f03218504 +Ref: library/codecs surrogateescape3219097 +Ref: 14f13219097 +Node: Error Handlers3219234 +Ref: library/codecs error-handlers3219343 +Ref: ffd3219343 +Ref: library/codecs id23219343 +Ref: 14f23219343 +Ref: library/codecs codecs register_error3223427 +Ref: df53223427 +Ref: library/codecs codecs lookup_error3224968 +Ref: df63224968 +Ref: library/codecs codecs strict_errors3225263 +Ref: 14f33225263 +Ref: library/codecs codecs replace_errors3225434 +Ref: 14f53225434 +Ref: library/codecs codecs ignore_errors3225738 +Ref: 14f43225738 +Ref: library/codecs codecs xmlcharrefreplace_errors3225931 +Ref: 14f63225931 +Ref: library/codecs codecs backslashreplace_errors3226197 +Ref: 14f73226197 +Ref: library/codecs codecs namereplace_errors3226427 +Ref: 14f83226427 +Ref: Error Handlers-Footnote-13226736 +Node: Stateless Encoding and Decoding3226785 +Ref: library/codecs codec-objects3226936 +Ref: 14d13226936 +Ref: library/codecs stateless-encoding-and-decoding3226936 +Ref: 14f93226936 +Ref: library/codecs codecs Codec encode3227147 +Ref: 14cf3227147 +Ref: library/codecs codecs Codec decode3227889 +Ref: 14d03227889 +Node: Incremental Encoding and Decoding3228816 +Ref: library/codecs incremental-encoding-and-decoding3228981 +Ref: 14fa3228981 +Node: IncrementalEncoder Objects3229821 +Ref: library/codecs incremental-encoder-objects3229952 +Ref: 14fd3229952 +Ref: library/codecs incrementalencoder-objects3229952 +Ref: 14fe3229952 +Ref: library/codecs codecs IncrementalEncoder3230251 +Ref: 14d43230251 +Ref: library/codecs codecs IncrementalEncoder encode3231035 +Ref: 14fb3231035 +Ref: library/codecs codecs IncrementalEncoder reset3231328 +Ref: 14ff3231328 +Ref: library/codecs codecs IncrementalEncoder getstate3231597 +Ref: 15003231597 +Ref: library/codecs codecs IncrementalEncoder setstate3232010 +Ref: 15013232010 +Node: IncrementalDecoder Objects3232177 +Ref: library/codecs incremental-decoder-objects3232308 +Ref: 15023232308 +Ref: library/codecs incrementaldecoder-objects3232308 +Ref: 15033232308 +Ref: library/codecs codecs IncrementalDecoder3232607 +Ref: 14d53232607 +Ref: library/codecs codecs IncrementalDecoder decode3233391 +Ref: 14fc3233391 +Ref: library/codecs codecs IncrementalDecoder reset3234020 +Ref: 15043234020 +Ref: library/codecs codecs IncrementalDecoder getstate3234098 +Ref: 15053234098 +Ref: library/codecs codecs IncrementalDecoder setstate3235045 +Ref: 15063235045 +Node: Stream Encoding and Decoding3235211 +Ref: library/codecs stream-encoding-and-decoding3235336 +Ref: 15073235336 +Node: StreamWriter Objects3235763 +Ref: library/codecs stream-writer-objects3235877 +Ref: 15083235877 +Ref: library/codecs streamwriter-objects3235877 +Ref: 15093235877 +Ref: library/codecs codecs StreamWriter3236132 +Ref: 14d83236132 +Ref: library/codecs codecs StreamWriter write3237088 +Ref: 150a3237088 +Ref: library/codecs codecs StreamWriter writelines3237186 +Ref: 150b3237186 +Ref: library/codecs codecs StreamWriter reset3237426 +Ref: 150c3237426 +Node: StreamReader Objects3237898 +Ref: library/codecs stream-reader-objects3238047 +Ref: 150d3238047 +Ref: library/codecs streamreader-objects3238047 +Ref: 150e3238047 +Ref: library/codecs codecs StreamReader3238302 +Ref: 14d93238302 +Ref: library/codecs codecs StreamReader read3239371 +Ref: 150f3239371 +Ref: library/codecs codecs StreamReader readline3240623 +Ref: 15103240623 +Ref: library/codecs codecs StreamReader readlines3240968 +Ref: 15113240968 +Ref: library/codecs codecs StreamReader reset3241413 +Ref: 15123241413 +Node: StreamReaderWriter Objects3241807 +Ref: library/codecs stream-reader-writer3241957 +Ref: 15133241957 +Ref: library/codecs streamreaderwriter-objects3241957 +Ref: 15143241957 +Ref: library/codecs codecs StreamReaderWriter3242293 +Ref: 14e13242293 +Node: StreamRecoder Objects3242948 +Ref: library/codecs stream-recoder-objects3243069 +Ref: 15153243069 +Ref: library/codecs streamrecoder-objects3243069 +Ref: 15163243069 +Ref: library/codecs codecs StreamRecoder3243420 +Ref: 14e33243420 +Node: Encodings and Unicode3244551 +Ref: library/codecs encodings-and-unicode3244705 +Ref: 15173244705 +Ref: library/codecs encodings-overview3244705 +Ref: 15183244705 +Ref: Encodings and Unicode-Footnote-13251404 +Node: Standard Encodings3251453 +Ref: library/codecs id33251614 +Ref: 15193251614 +Ref: library/codecs standard-encodings3251614 +Ref: 68f3251614 +Node: Python Specific Encodings3273638 +Ref: library/codecs python-specific-encodings3273843 +Ref: 151a3273843 +Node: Text Encodings3274431 +Ref: library/codecs text-encodings3274533 +Ref: 151b3274533 +Ref: Text Encodings-Footnote-13277905 +Ref: Text Encodings-Footnote-23277954 +Node: Binary Transforms3278003 +Ref: library/codecs binary-transforms3278129 +Ref: 8043278129 +Ref: library/codecs id43278129 +Ref: 151c3278129 +Ref: Binary Transforms-Footnote-13281934 +Node: Text Transforms3282079 +Ref: library/codecs id63282182 +Ref: 15253282182 +Ref: library/codecs text-transforms3282182 +Ref: 8053282182 +Node: encodings idna — Internationalized Domain Names in Applications3282926 +Ref: library/codecs encodings-idna-internationalized-domain-names-in-applications3283153 +Ref: 15263283153 +Ref: library/codecs module-encodings idna3283153 +Ref: 773283153 +Ref: library/codecs encodings idna nameprep3285662 +Ref: 15273285662 +Ref: library/codecs encodings idna ToASCII3285850 +Ref: 15283285850 +Ref: library/codecs encodings idna ToUnicode3286010 +Ref: 15293286010 +Ref: encodings idna — Internationalized Domain Names in Applications-Footnote-13286157 +Ref: encodings idna — Internationalized Domain Names in Applications-Footnote-23286206 +Ref: encodings idna — Internationalized Domain Names in Applications-Footnote-33286255 +Ref: encodings idna — Internationalized Domain Names in Applications-Footnote-43286316 +Ref: encodings idna — Internationalized Domain Names in Applications-Footnote-53286365 +Node: encodings mbcs — Windows ANSI codepage3286414 +Ref: library/codecs encodings-mbcs-windows-ansi-codepage3286670 +Ref: 152a3286670 +Ref: library/codecs module-encodings mbcs3286670 +Ref: 783286670 +Node: encodings utf_8_sig — UTF-8 codec with BOM signature3287067 +Ref: library/codecs encodings-utf-8-sig-utf-8-codec-with-bom-signature3287249 +Ref: 152b3287249 +Ref: library/codecs module-encodings utf_8_sig3287249 +Ref: 793287249 +Node: Data Types3287694 +Ref: library/datatypes doc3287838 +Ref: 152c3287838 +Ref: library/datatypes data-types3287838 +Ref: 152d3287838 +Ref: library/datatypes datatypes3287838 +Ref: 152e3287838 +Node: datetime — Basic date and time types3289108 +Ref: library/datetime doc3289249 +Ref: 152f3289249 +Ref: library/datetime datetime-basic-date-and-time-types3289249 +Ref: 15303289249 +Ref: library/datetime module-datetime3289249 +Ref: 313289249 +Ref: datetime — Basic date and time types-Footnote-13290223 +Ref: datetime — Basic date and time types-Footnote-23290290 +Node: Aware and Naive Objects3290341 +Ref: library/datetime aware-and-naive-objects3290457 +Ref: 15313290457 +Ref: library/datetime datetime-naive-aware3290457 +Ref: 15323290457 +Ref: Aware and Naive Objects-Footnote-13292383 +Node: Constants3292444 +Ref: library/datetime constants3292584 +Ref: 15333292584 +Ref: library/datetime datetime MINYEAR3292686 +Ref: 15343292686 +Ref: library/datetime datetime MAXYEAR3292846 +Ref: 15353292846 +Node: Available Types3293008 +Ref: library/datetime available-types3293142 +Ref: 15363293142 +Node: Common Properties3294924 +Ref: library/datetime common-properties3295044 +Ref: 15473295044 +Node: Determining if an Object is Aware or Naive3295454 +Ref: library/datetime determining-if-an-object-is-aware-or-naive3295574 +Ref: 15483295574 +Node: timedelta Objects3296309 +Ref: library/datetime datetime-timedelta3296446 +Ref: 15493296446 +Ref: library/datetime timedelta-objects3296446 +Ref: 154a3296446 +Ref: library/datetime datetime timedelta3296605 +Ref: 1953296605 +Ref: library/datetime datetime timedelta min3298857 +Ref: 154b3298857 +Ref: library/datetime datetime timedelta max3298975 +Ref: 154c3298975 +Ref: library/datetime datetime timedelta resolution3299157 +Ref: 154d3299157 +Ref: library/datetime datetime timedelta total_seconds3306560 +Ref: c9c3306560 +Node: Examples of usage timedelta3307082 +Ref: library/datetime examples-of-usage-timedelta3307163 +Ref: 154e3307163 +Node: date Objects3308104 +Ref: library/datetime date-objects3308242 +Ref: 154f3308242 +Ref: library/datetime datetime-date3308242 +Ref: 15503308242 +Ref: library/datetime datetime date3308564 +Ref: 1933308564 +Ref: library/datetime datetime date today3308983 +Ref: 15513308983 +Ref: library/datetime datetime date fromtimestamp3309126 +Ref: 15523309126 +Ref: library/datetime datetime date fromordinal3310064 +Ref: 15533310064 +Ref: library/datetime datetime date fromisoformat3310397 +Ref: 15543310397 +Ref: library/datetime datetime date fromisocalendar3310825 +Ref: 1c43310825 +Ref: library/datetime datetime date min3311117 +Ref: 15573311117 +Ref: library/datetime datetime date max3311208 +Ref: 15583311208 +Ref: library/datetime datetime date resolution3311299 +Ref: 15593311299 +Ref: library/datetime datetime date year3311466 +Ref: 15373311466 +Ref: library/datetime datetime date month3311563 +Ref: 15383311563 +Ref: library/datetime datetime date day3311624 +Ref: 15393311624 +Ref: library/datetime datetime date replace3314114 +Ref: 155a3314114 +Ref: library/datetime datetime date timetuple3314499 +Ref: 155b3314499 +Ref: library/datetime datetime date toordinal3314994 +Ref: 155d3314994 +Ref: library/datetime datetime date weekday3315216 +Ref: 155e3315216 +Ref: library/datetime datetime date isoweekday3315441 +Ref: 155f3315441 +Ref: library/datetime datetime date isocalendar3315701 +Ref: 15563315701 +Ref: library/datetime datetime date isoformat3316550 +Ref: 15553316550 +Ref: library/datetime datetime date __str__3316839 +Ref: 15603316839 +Ref: library/datetime datetime date ctime3316941 +Ref: 15613316941 +Ref: library/datetime datetime date strftime3317415 +Ref: 5383317415 +Ref: library/datetime datetime date __format__3317723 +Ref: 15643317723 +Ref: date Objects-Footnote-13318181 +Ref: date Objects-Footnote-23318494 +Node: Examples of Usage date3318672 +Ref: library/datetime examples-of-usage-date3318743 +Ref: 15653318743 +Node: datetime Objects3320635 +Ref: library/datetime datetime-datetime3320768 +Ref: 15663320768 +Ref: library/datetime datetime-objects3320768 +Ref: 15673320768 +Ref: library/datetime datetime datetime3321217 +Ref: 1943321217 +Ref: library/datetime datetime datetime today3322080 +Ref: 15683322080 +Ref: library/datetime datetime datetime now3322438 +Ref: 15693322438 +Ref: library/datetime datetime datetime utcnow3323119 +Ref: 156b3323119 +Ref: library/datetime datetime datetime fromtimestamp3323862 +Ref: 156a3323862 +Ref: library/datetime datetime datetime utcfromtimestamp3325581 +Ref: 156c3325581 +Ref: library/datetime datetime datetime fromordinal3327253 +Ref: 156d3327253 +Ref: library/datetime datetime datetime combine3327658 +Ref: 5393327658 +Ref: library/datetime datetime datetime fromisoformat3328447 +Ref: 37e3328447 +Ref: library/datetime datetime datetime fromisocalendar3329971 +Ref: 1c53329971 +Ref: library/datetime datetime datetime strptime3330350 +Ref: 156f3330350 +Ref: library/datetime datetime datetime min3330915 +Ref: 15703330915 +Ref: library/datetime datetime datetime max3331047 +Ref: 15713331047 +Ref: library/datetime datetime datetime resolution3331199 +Ref: 15723331199 +Ref: library/datetime datetime datetime year3331394 +Ref: 153f3331394 +Ref: library/datetime datetime datetime month3331495 +Ref: 15403331495 +Ref: library/datetime datetime datetime day3331560 +Ref: 15413331560 +Ref: library/datetime datetime datetime hour3331671 +Ref: 15423331671 +Ref: library/datetime datetime datetime minute3331727 +Ref: 15433331727 +Ref: library/datetime datetime datetime second3331785 +Ref: 15443331785 +Ref: library/datetime datetime datetime microsecond3331843 +Ref: 15453331843 +Ref: library/datetime datetime datetime tzinfo3331911 +Ref: 15463331911 +Ref: library/datetime datetime datetime fold3332072 +Ref: 4f43332072 +Ref: library/datetime datetime datetime date3336703 +Ref: 15733336703 +Ref: library/datetime datetime datetime time3336801 +Ref: 15743336801 +Ref: library/datetime datetime datetime timetz3337108 +Ref: 15753337108 +Ref: library/datetime datetime datetime replace3337398 +Ref: 15763337398 +Ref: library/datetime datetime datetime astimezone3337980 +Ref: 1963337980 +Ref: library/datetime datetime datetime utcoffset3340312 +Ref: 15773340312 +Ref: library/datetime datetime datetime dst3340706 +Ref: 15783340706 +Ref: library/datetime datetime datetime tzname3341088 +Ref: 157a3341088 +Ref: library/datetime datetime datetime timetuple3341321 +Ref: 157b3341321 +Ref: library/datetime datetime datetime utctimetuple3342175 +Ref: 157c3342175 +Ref: library/datetime datetime datetime toordinal3343302 +Ref: 157d3343302 +Ref: library/datetime datetime datetime timestamp3343444 +Ref: 9e23343444 +Ref: library/datetime datetime datetime weekday3344952 +Ref: 157e3344952 +Ref: library/datetime datetime datetime isoweekday3345156 +Ref: 157f3345156 +Ref: library/datetime datetime datetime isocalendar3345390 +Ref: 156e3345390 +Ref: library/datetime datetime datetime isoformat3345544 +Ref: 37f3345544 +Ref: library/datetime datetime datetime __str__3348528 +Ref: 15803348528 +Ref: library/datetime datetime datetime ctime3348667 +Ref: 15813348667 +Ref: library/datetime datetime datetime strftime3349300 +Ref: 5373349300 +Ref: library/datetime datetime datetime __format__3349544 +Ref: 15823349544 +Ref: datetime Objects-Footnote-13350022 +Node: Examples of Usage datetime3350110 +Ref: library/datetime examples-of-usage-datetime3350189 +Ref: 15833350189 +Node: time Objects3354517 +Ref: library/datetime datetime-time3354652 +Ref: 15843354652 +Ref: library/datetime time-objects3354652 +Ref: 15853354652 +Ref: library/datetime datetime time3354860 +Ref: 4f33354860 +Ref: library/datetime datetime time min3355525 +Ref: 15863355525 +Ref: library/datetime datetime time max3355624 +Ref: 15873355624 +Ref: library/datetime datetime time resolution3355734 +Ref: 15883355734 +Ref: library/datetime datetime time hour3356002 +Ref: 153a3356002 +Ref: library/datetime datetime time minute3356054 +Ref: 153b3356054 +Ref: library/datetime datetime time second3356108 +Ref: 153c3356108 +Ref: library/datetime datetime time microsecond3356162 +Ref: 153d3356162 +Ref: library/datetime datetime time tzinfo3356226 +Ref: 153e3356226 +Ref: library/datetime datetime time fold3356377 +Ref: 15893356377 +Ref: library/datetime datetime time fromisoformat3358352 +Ref: 158a3358352 +Ref: library/datetime datetime time replace3359262 +Ref: 158c3359262 +Ref: library/datetime datetime time isoformat3359796 +Ref: 158b3359796 +Ref: library/datetime datetime time __str__3361807 +Ref: 158e3361807 +Ref: library/datetime datetime time strftime3361909 +Ref: 158f3361909 +Ref: library/datetime datetime time __format__3362140 +Ref: 15903362140 +Ref: library/datetime datetime time utcoffset3362502 +Ref: 158d3362502 +Ref: library/datetime datetime time dst3362892 +Ref: 15913362892 +Ref: library/datetime datetime time tzname3363271 +Ref: 15923363271 +Ref: time Objects-Footnote-13363600 +Node: Examples of Usage time3363643 +Ref: library/datetime examples-of-usage-time3363714 +Ref: 15933363714 +Node: tzinfo Objects3364596 +Ref: library/datetime datetime-tzinfo3364731 +Ref: 15943364731 +Ref: library/datetime tzinfo-objects3364731 +Ref: 15953364731 +Ref: library/datetime datetime tzinfo3364788 +Ref: 3803364788 +Ref: library/datetime datetime tzinfo utcoffset3366430 +Ref: 15963366430 +Ref: library/datetime datetime tzinfo dst3367660 +Ref: 15973367660 +Ref: library/datetime datetime tzinfo tzname3370242 +Ref: 15983370242 +Ref: library/datetime datetime tzinfo fromutc3372381 +Ref: 15793372381 +Ref: tzinfo Objects-Footnote-13384882 +Ref: tzinfo Objects-Footnote-23384940 +Node: timezone Objects3384980 +Ref: library/datetime datetime-timezone3385133 +Ref: 159a3385133 +Ref: library/datetime timezone-objects3385133 +Ref: 159b3385133 +Ref: library/datetime datetime timezone3385553 +Ref: a013385553 +Ref: library/datetime datetime timezone utcoffset3386181 +Ref: 159c3386181 +Ref: library/datetime datetime timezone tzname3386571 +Ref: 159d3386571 +Ref: library/datetime datetime timezone dst3387253 +Ref: 159e3387253 +Ref: library/datetime datetime timezone fromutc3387317 +Ref: 159f3387317 +Ref: library/datetime datetime timezone utc3387512 +Ref: 15993387512 +Node: strftime and strptime Behavior3387595 +Ref: library/datetime strftime-and-strptime-behavior3387725 +Ref: 15a03387725 +Ref: library/datetime strftime-strptime-behavior3387725 +Ref: 15633387725 +Node: strftime and strptime Format Codes3389911 +Ref: library/datetime strftime-and-strptime-format-codes3390037 +Ref: 15a13390037 +Node: Technical Detail3399998 +Ref: library/datetime technical-detail3400124 +Ref: 15a23400124 +Ref: Technical Detail-Footnote-13405786 +Node: calendar — General calendar-related functions3405896 +Ref: library/calendar doc3406081 +Ref: 15a43406081 +Ref: library/calendar calendar-general-calendar-related-functions3406081 +Ref: 15a53406081 +Ref: library/calendar module-calendar3406081 +Ref: 153406081 +Ref: library/calendar calendar Calendar3407284 +Ref: 15a73407284 +Ref: library/calendar calendar Calendar iterweekdays3407791 +Ref: 15a83407791 +Ref: library/calendar calendar Calendar itermonthdates3408040 +Ref: 4a63408040 +Ref: library/calendar calendar Calendar itermonthdays3408412 +Ref: 15aa3408412 +Ref: library/calendar calendar Calendar itermonthdays23408797 +Ref: 15ab3408797 +Ref: library/calendar calendar Calendar itermonthdays33409147 +Ref: 4a73409147 +Ref: library/calendar calendar Calendar itermonthdays43409527 +Ref: 4a83409527 +Ref: library/calendar calendar Calendar monthdatescalendar3409927 +Ref: 15ac3409927 +Ref: library/calendar calendar Calendar monthdays2calendar3410147 +Ref: 15ad3410147 +Ref: library/calendar calendar Calendar monthdayscalendar3410375 +Ref: 15ae3410375 +Ref: library/calendar calendar Calendar yeardatescalendar3410562 +Ref: 15af3410562 +Ref: library/calendar calendar Calendar yeardays2calendar3410956 +Ref: 15b03410956 +Ref: library/calendar calendar Calendar yeardayscalendar3411272 +Ref: 15b13411272 +Ref: library/calendar calendar TextCalendar3411557 +Ref: 15b23411557 +Ref: library/calendar calendar TextCalendar formatmonth3411741 +Ref: 15b33411741 +Ref: library/calendar calendar TextCalendar prmonth3412190 +Ref: 15b43412190 +Ref: library/calendar calendar TextCalendar formatyear3412336 +Ref: 15b53412336 +Ref: library/calendar calendar TextCalendar pryear3412873 +Ref: 15b63412873 +Ref: library/calendar calendar HTMLCalendar3413028 +Ref: 36f3413028 +Ref: library/calendar calendar HTMLCalendar formatmonth3413199 +Ref: 15b73413199 +Ref: library/calendar calendar HTMLCalendar formatyear3413449 +Ref: 15b83413449 +Ref: library/calendar calendar HTMLCalendar formatyearpage3413630 +Ref: 15b93413630 +Ref: library/calendar calendar HTMLCalendar cssclasses3414270 +Ref: 15ba3414270 +Ref: library/calendar calendar HTMLCalendar cssclass_noday3414684 +Ref: 15bb3414684 +Ref: library/calendar calendar HTMLCalendar cssclasses_weekday_head3414843 +Ref: 15bc3414843 +Ref: library/calendar calendar HTMLCalendar cssclass_month_head3415056 +Ref: 15bd3415056 +Ref: library/calendar calendar HTMLCalendar cssclass_month3415250 +Ref: 15be3415250 +Ref: library/calendar calendar HTMLCalendar cssclass_year3415457 +Ref: 15bf3415457 +Ref: library/calendar calendar HTMLCalendar cssclass_year_head3415670 +Ref: 15c03415670 +Ref: library/calendar calendar LocaleTextCalendar3416577 +Ref: 15c13416577 +Ref: library/calendar calendar LocaleHTMLCalendar3416941 +Ref: 15c23416941 +Ref: library/calendar calendar setfirstweekday3417629 +Ref: 15a63417629 +Ref: library/calendar calendar firstweekday3418055 +Ref: 15a93418055 +Ref: library/calendar calendar isleap3418165 +Ref: 15c33418165 +Ref: library/calendar calendar leapdays3418293 +Ref: 15c43418293 +Ref: library/calendar calendar weekday3418518 +Ref: 15c53418518 +Ref: library/calendar calendar weekheader3418717 +Ref: 15c63418717 +Ref: library/calendar calendar monthrange3418877 +Ref: 15c73418877 +Ref: library/calendar calendar monthcalendar3419046 +Ref: 15c83419046 +Ref: library/calendar calendar prmonth3419327 +Ref: 15c93419327 +Ref: library/calendar calendar month3419459 +Ref: 15ca3419459 +Ref: library/calendar calendar prcal3419652 +Ref: 15cb3419652 +Ref: library/calendar calendar calendar3419796 +Ref: 15cc3419796 +Ref: library/calendar calendar timegm3420011 +Ref: 15cd3420011 +Ref: library/calendar calendar day_name3420487 +Ref: 15ce3420487 +Ref: library/calendar calendar day_abbr3420596 +Ref: 15cf3420596 +Ref: library/calendar calendar month_name3420717 +Ref: 15d03420717 +Ref: library/calendar calendar month_abbr3420975 +Ref: 15d13420975 +Ref: calendar — General calendar-related functions-Footnote-13421508 +Node: collections — Container datatypes3421575 +Ref: library/collections doc3421778 +Ref: 15d23421778 +Ref: library/collections collections-container-datatypes3421778 +Ref: 15d33421778 +Ref: library/collections module-collections3421778 +Ref: 1e3421778 +Ref: collections — Container datatypes-Footnote-13424009 +Node: ChainMap objects3424089 +Ref: library/collections chainmap-objects3424201 +Ref: 15d73424201 +Ref: library/collections collections ChainMap3424588 +Ref: 4a93424588 +Ref: library/collections collections ChainMap maps3425621 +Ref: 15d83425621 +Ref: library/collections collections ChainMap new_child3425924 +Ref: 8293425924 +Ref: library/collections collections ChainMap parents3426562 +Ref: 15d93426562 +Ref: ChainMap objects-Footnote-13428415 +Ref: ChainMap objects-Footnote-23428510 +Ref: ChainMap objects-Footnote-33428557 +Ref: ChainMap objects-Footnote-43428636 +Ref: ChainMap objects-Footnote-53428689 +Node: ChainMap Examples and Recipes3428742 +Ref: library/collections chainmap-examples-and-recipes3428824 +Ref: 15da3428824 +Node: Counter objects3431938 +Ref: library/collections counter-objects3432072 +Ref: 15db3432072 +Ref: library/collections collections Counter3432777 +Ref: 9da3432777 +Ref: library/collections collections Counter elements3434747 +Ref: c9a3434747 +Ref: library/collections collections Counter most_common3435170 +Ref: c993435170 +Ref: library/collections collections Counter subtract3435630 +Ref: b5a3435630 +Ref: library/collections collections Counter fromkeys3436321 +Ref: 15dc3436321 +Ref: library/collections collections Counter update3436450 +Ref: 15dd3436450 +Ref: Counter objects-Footnote-13440836 +Ref: Counter objects-Footnote-23440915 +Ref: Counter objects-Footnote-33440962 +Node: deque objects3441055 +Ref: library/collections deque-objects3441192 +Ref: 15de3441192 +Ref: library/collections collections deque3441245 +Ref: 5313441245 +Ref: library/collections collections deque append3442688 +Ref: 15df3442688 +Ref: library/collections collections deque appendleft3442768 +Ref: 15e03442768 +Ref: library/collections collections deque clear3442851 +Ref: 15e13442851 +Ref: library/collections collections deque copy3442950 +Ref: 6ab3442950 +Ref: library/collections collections deque count3443054 +Ref: b5c3443054 +Ref: library/collections collections deque extend3443173 +Ref: 15e23443173 +Ref: library/collections collections deque extendleft3443315 +Ref: 15e33443315 +Ref: library/collections collections deque index3443562 +Ref: 6a93443562 +Ref: library/collections collections deque insert3443834 +Ref: 6aa3443834 +Ref: library/collections collections deque pop3444079 +Ref: 15e43444079 +Ref: library/collections collections deque popleft3444249 +Ref: 15e53444249 +Ref: library/collections collections deque remove3444422 +Ref: 15e63444422 +Ref: library/collections collections deque reverse3444563 +Ref: b5d3444563 +Ref: library/collections collections deque rotate3444715 +Ref: 15e73444715 +Ref: library/collections collections deque maxlen3445124 +Ref: c9b3445124 +Node: deque Recipes3447775 +Ref: library/collections deque-recipes3447838 +Ref: 15e83447838 +Ref: deque Recipes-Footnote-13450301 +Node: defaultdict objects3450362 +Ref: library/collections defaultdict-objects3450540 +Ref: 15e93450540 +Ref: library/collections collections defaultdict3450605 +Ref: b3a3450605 +Ref: library/collections collections defaultdict __missing__3451386 +Ref: 15eb3451386 +Ref: library/collections collections defaultdict default_factory3452575 +Ref: 15ea3452575 +Node: defaultdict Examples3452846 +Ref: library/collections defaultdict-examples3452922 +Ref: 15ec3452922 +Node: namedtuple Factory Function for Tuples with Named Fields3455585 +Ref: library/collections namedtuple-factory-function-for-tuples-with-named-fields3455769 +Ref: 15ed3455769 +Ref: library/collections collections namedtuple3456150 +Ref: 1b73456150 +Ref: library/collections collections somenamedtuple _make3460059 +Ref: 15ee3460059 +Ref: library/collections collections somenamedtuple _asdict3460290 +Ref: 1b63460290 +Ref: library/collections collections somenamedtuple _replace3460976 +Ref: 15ef3460976 +Ref: library/collections collections somenamedtuple _fields3461376 +Ref: 15f03461376 +Ref: library/collections collections somenamedtuple _field_defaults3461872 +Ref: 15f13461872 +Node: OrderedDict objects3464639 +Ref: library/collections ordereddict-objects3464820 +Ref: 15f23464820 +Ref: library/collections collections OrderedDict3466243 +Ref: 1b93466243 +Ref: library/collections collections OrderedDict popitem3466439 +Ref: c843466439 +Ref: library/collections collections OrderedDict move_to_end3466697 +Ref: b5b3466697 +Ref: OrderedDict objects-Footnote-13468241 +Ref: OrderedDict objects-Footnote-23468326 +Node: OrderedDict Examples and Recipes3468375 +Ref: library/collections ordereddict-examples-and-recipes3468463 +Ref: 15f53468463 +Node: UserDict objects3469803 +Ref: library/collections userdict-objects3469944 +Ref: 15f63469944 +Ref: library/collections collections UserDict3470310 +Ref: 15d43470310 +Ref: library/collections collections UserDict data3470900 +Ref: 15f73470900 +Node: UserList objects3471028 +Ref: library/collections userlist-objects3471168 +Ref: 15f83471168 +Ref: library/collections collections UserList3471683 +Ref: 15d53471683 +Ref: library/collections collections UserList data3472288 +Ref: 15f93472288 +Node: UserString objects3473082 +Ref: library/collections userstring-objects3473197 +Ref: 15fa3473197 +Ref: library/collections collections UserString3473559 +Ref: 6ad3473559 +Ref: library/collections collections UserString data3474153 +Ref: 15fb3474153 +Node: collections abc — Abstract Base Classes for Containers3474453 +Ref: library/collections abc doc3474639 +Ref: 15fc3474639 +Ref: library/collections abc collections-abc-abstract-base-classes-for-containers3474639 +Ref: 15fd3474639 +Ref: library/collections abc module-collections abc3474639 +Ref: 1f3474639 +Ref: collections abc — Abstract Base Classes for Containers-Footnote-13475251 +Node: Collections Abstract Base Classes3475327 +Ref: library/collections abc collections-abstract-base-classes3475453 +Ref: 15d63475453 +Ref: library/collections abc id13475453 +Ref: 15fe3475453 +Ref: library/collections abc collections abc Container3484413 +Ref: 15ff3484413 +Ref: library/collections abc collections abc Hashable3484525 +Ref: 16003484525 +Ref: library/collections abc collections abc Sized3484632 +Ref: 16033484632 +Ref: library/collections abc collections abc Callable3484735 +Ref: 16043484735 +Ref: library/collections abc collections abc Iterable3484843 +Ref: 16013484843 +Ref: library/collections abc collections abc Collection3485323 +Ref: 52d3485323 +Ref: library/collections abc collections abc Iterator3485436 +Ref: 16023485436 +Ref: library/collections abc collections abc Reversible3485632 +Ref: 52e3485632 +Ref: library/collections abc collections abc Generator3485790 +Ref: 6b53485790 +Ref: library/collections abc collections abc Sequence3486108 +Ref: 12db3486108 +Ref: library/collections abc collections abc MutableSequence3486144 +Ref: 6ac3486144 +Ref: library/collections abc collections abc ByteString3486187 +Ref: 16053486187 +Ref: library/collections abc collections abc Set3486922 +Ref: 13853486922 +Ref: library/collections abc collections abc MutableSet3486953 +Ref: 16063486953 +Ref: library/collections abc collections abc Mapping3487035 +Ref: 15f33487035 +Ref: library/collections abc collections abc MutableMapping3487070 +Ref: 9ef3487070 +Ref: library/collections abc collections abc MappingView3487171 +Ref: 16073487171 +Ref: library/collections abc collections abc ItemsView3487210 +Ref: 16083487210 +Ref: library/collections abc collections abc KeysView3487247 +Ref: 16093487247 +Ref: library/collections abc collections abc ValuesView3487283 +Ref: 160a3487283 +Ref: library/collections abc collections abc Awaitable3487389 +Ref: 6b63487389 +Ref: library/collections abc collections abc Coroutine3488165 +Ref: 6b73488165 +Ref: library/collections abc collections abc AsyncIterable3489059 +Ref: 6b93489059 +Ref: library/collections abc collections abc AsyncIterator3489253 +Ref: 6b83489253 +Ref: library/collections abc collections abc AsyncGenerator3489467 +Ref: 5303489467 +Ref: Collections Abstract Base Classes-Footnote-13492508 +Ref: Collections Abstract Base Classes-Footnote-23492557 +Ref: Collections Abstract Base Classes-Footnote-33492606 +Ref: Collections Abstract Base Classes-Footnote-43492655 +Ref: Collections Abstract Base Classes-Footnote-53492708 +Node: heapq — Heap queue algorithm3492757 +Ref: library/heapq doc3492944 +Ref: 160b3492944 +Ref: library/heapq heapq-heap-queue-algorithm3492944 +Ref: 160c3492944 +Ref: library/heapq module-heapq3492944 +Ref: 8e3492944 +Ref: library/heapq heapq heappush3494499 +Ref: 160e3494499 +Ref: library/heapq heapq heappop3494624 +Ref: 160f3494624 +Ref: library/heapq heapq heappushpop3494888 +Ref: 16103494888 +Ref: library/heapq heapq heapify3495161 +Ref: 160d3495161 +Ref: library/heapq heapq heapreplace3495258 +Ref: 16113495258 +Ref: library/heapq heapq merge3496105 +Ref: 6df3496105 +Ref: library/heapq heapq nlargest3497269 +Ref: 16123497269 +Ref: library/heapq heapq nsmallest3497667 +Ref: 16133497667 +Ref: heapq — Heap queue algorithm-Footnote-13498536 +Node: Basic Examples3498600 +Ref: library/heapq basic-examples3498725 +Ref: 16143498725 +Ref: Basic Examples-Footnote-13499692 +Node: Priority Queue Implementation Notes3499739 +Ref: library/heapq priority-queue-implementation-notes3499879 +Ref: 16153499879 +Ref: Priority Queue Implementation Notes-Footnote-13502815 +Node: Theory3502868 +Ref: library/heapq theory3502985 +Ref: 16163502985 +Ref: Theory-Footnote-13507145 +Node: bisect — Array bisection algorithm3507805 +Ref: library/bisect doc3507980 +Ref: 16173507980 +Ref: library/bisect bisect-array-bisection-algorithm3507980 +Ref: 16183507980 +Ref: library/bisect module-bisect3507980 +Ref: 123507980 +Ref: library/bisect bisect bisect_left3508677 +Ref: 16193508677 +Ref: library/bisect bisect bisect_right3509423 +Ref: 161a3509423 +Ref: library/bisect bisect bisect3509481 +Ref: 161b3509481 +Ref: library/bisect bisect insort_left3509908 +Ref: 161c3509908 +Ref: library/bisect bisect insort_right3510219 +Ref: 161d3510219 +Ref: library/bisect bisect insort3510277 +Ref: 161e3510277 +Ref: bisect — Array bisection algorithm-Footnote-13510795 +Ref: bisect — Array bisection algorithm-Footnote-23510860 +Node: Searching Sorted Lists3510930 +Ref: library/bisect searching-sorted-lists3511048 +Ref: 161f3511048 +Node: Other Examples3512270 +Ref: library/bisect other-examples3512388 +Ref: 16203512388 +Ref: library/bisect bisect-example3512435 +Ref: 16213512435 +Node: array — Efficient arrays of numeric values3513760 +Ref: library/array doc3513932 +Ref: 16223513932 +Ref: library/array array-efficient-arrays-of-numeric-values3513932 +Ref: 16233513932 +Ref: library/array module-array3513932 +Ref: 73513932 +Ref: library/array array array3518476 +Ref: 4513518476 +Ref: library/array array typecodes3519196 +Ref: 16283519196 +Ref: library/array array array typecode3519714 +Ref: 16293519714 +Ref: library/array array array itemsize3519800 +Ref: 162a3519800 +Ref: library/array array array append3519912 +Ref: 162b3519912 +Ref: library/array array array buffer_info3520006 +Ref: 162c3520006 +Ref: library/array array array byteswap3521036 +Ref: 162d3521036 +Ref: library/array array array count3521363 +Ref: 162e3521363 +Ref: library/array array array extend3521452 +Ref: 16273521452 +Ref: library/array array array frombytes3521812 +Ref: 16253521812 +Ref: library/array array array fromfile3522134 +Ref: 162f3522134 +Ref: library/array array array fromlist3522549 +Ref: 16243522549 +Ref: library/array array array fromstring3522746 +Ref: 16303522746 +Ref: library/array array array fromunicode3522899 +Ref: 16263522899 +Ref: library/array array array index3523213 +Ref: 16313523213 +Ref: library/array array array insert3523349 +Ref: 16323523349 +Ref: library/array array array pop3523536 +Ref: 16333523536 +Ref: library/array array array remove3523750 +Ref: 16343523750 +Ref: library/array array array reverse3523837 +Ref: 16353523837 +Ref: library/array array array tobytes3523918 +Ref: 16363523918 +Ref: library/array array array tofile3524251 +Ref: 16373524251 +Ref: library/array array array tolist3524364 +Ref: 16393524364 +Ref: library/array array array tostring3524458 +Ref: 16383524458 +Ref: library/array array array tounicode3524607 +Ref: 163a3524607 +Ref: array — Efficient arrays of numeric values-Footnote-13525983 +Node: weakref — Weak references3526019 +Ref: library/weakref doc3526215 +Ref: 163b3526215 +Ref: library/weakref module-weakref3526215 +Ref: 1283526215 +Ref: library/weakref weakref-weak-references3526215 +Ref: 163c3526215 +Ref: library/weakref weakref ref3530004 +Ref: 9103530004 +Ref: library/weakref weakref ref __callback__3531814 +Ref: 9113531814 +Ref: library/weakref weakref proxy3532179 +Ref: 2503532179 +Ref: library/weakref weakref getweakrefcount3533045 +Ref: 16403533045 +Ref: library/weakref weakref getweakrefs3533178 +Ref: 16413533178 +Ref: library/weakref weakref WeakKeyDictionary3533312 +Ref: 163d3533312 +Ref: library/weakref weakref WeakKeyDictionary keyrefs3534156 +Ref: 16423534156 +Ref: library/weakref weakref WeakValueDictionary3534259 +Ref: 163e3534259 +Ref: library/weakref weakref WeakValueDictionary valuerefs3534635 +Ref: 16433534635 +Ref: library/weakref weakref WeakSet3534744 +Ref: cb83534744 +Ref: library/weakref weakref WeakMethod3534928 +Ref: 90f3534928 +Ref: library/weakref weakref finalize3535863 +Ref: 29d3535863 +Ref: library/weakref weakref finalize __call__3537162 +Ref: 16443537162 +Ref: library/weakref weakref finalize detach3537373 +Ref: 16453537373 +Ref: library/weakref weakref finalize peek3537574 +Ref: 16463537574 +Ref: library/weakref weakref finalize alive3537743 +Ref: 16473537743 +Ref: library/weakref weakref finalize atexit3537858 +Ref: 16483537858 +Ref: library/weakref weakref ReferenceType3538457 +Ref: 16493538457 +Ref: library/weakref weakref ProxyType3538541 +Ref: 164a3538541 +Ref: library/weakref weakref CallableProxyType3538639 +Ref: 164b3538639 +Ref: library/weakref weakref ProxyTypes3538731 +Ref: 164c3538731 +Ref: weakref — Weak references-Footnote-13539361 +Ref: weakref — Weak references-Footnote-23539427 +Node: Weak Reference Objects3539476 +Ref: library/weakref weak-reference-objects3539581 +Ref: 164d3539581 +Ref: library/weakref weakref-objects3539581 +Ref: 164e3539581 +Node: Example<2>3542139 +Ref: library/weakref example3542270 +Ref: 164f3542270 +Ref: library/weakref weakref-example3542270 +Ref: 16503542270 +Node: Finalizer Objects3542811 +Ref: library/weakref finalize-examples3542961 +Ref: 16513542961 +Ref: library/weakref finalizer-objects3542961 +Ref: 16523542961 +Node: Comparing finalizers with __del__ methods3544812 +Ref: library/weakref comparing-finalizers-with-del-methods3544943 +Ref: 16533544943 +Node: types — Dynamic type creation and names for built-in types3547767 +Ref: library/types doc3547960 +Ref: 16543547960 +Ref: library/types module-types3547960 +Ref: 1183547960 +Ref: library/types types-dynamic-type-creation-and-names-for-built-in-types3547960 +Ref: 16553547960 +Ref: types — Dynamic type creation and names for built-in types-Footnote-13548761 +Node: Dynamic Type Creation3548825 +Ref: library/types dynamic-type-creation3548978 +Ref: 16563548978 +Ref: library/types types new_class3549039 +Ref: ab73549039 +Ref: library/types types prepare_class3549751 +Ref: ab83549751 +Ref: library/types types resolve_bases3550920 +Ref: 4063550920 +Ref: Dynamic Type Creation-Footnote-13551591 +Ref: Dynamic Type Creation-Footnote-23551640 +Ref: Dynamic Type Creation-Footnote-33551689 +Node: Standard Interpreter Types3551738 +Ref: library/types standard-interpreter-types3551940 +Ref: 16573551940 +Ref: library/types types FunctionType3552494 +Ref: 16583552494 +Ref: library/types types LambdaType3552523 +Ref: 16593552523 +Ref: library/types types GeneratorType3552878 +Ref: 165a3552878 +Ref: library/types types CoroutineType3553004 +Ref: 77d3553004 +Ref: library/types types AsyncGeneratorType3553159 +Ref: 165b3553159 +Ref: library/types types CodeType3553343 +Ref: 1983553343 +Ref: library/types types CodeType replace3553920 +Ref: 165c3553920 +Ref: library/types types CellType3554084 +Ref: 10cb3554084 +Ref: library/types types MethodType3554245 +Ref: 165d3554245 +Ref: library/types types BuiltinFunctionType3554332 +Ref: 165e3554332 +Ref: library/types types BuiltinMethodType3554368 +Ref: 165f3554368 +Ref: library/types types WrapperDescriptorType3554591 +Ref: 4023554591 +Ref: library/types types MethodWrapperType3554806 +Ref: 4033554806 +Ref: library/types types MethodDescriptorType3555007 +Ref: 4043555007 +Ref: library/types types ClassMethodDescriptorType3555162 +Ref: 4053555162 +Ref: library/types types ModuleType3555347 +Ref: 6f13555347 +Ref: library/types types ModuleType __doc__3555710 +Ref: 16603555710 +Ref: library/types types ModuleType __loader__3555817 +Ref: 16613555817 +Ref: library/types types ModuleType __name__3556562 +Ref: 16633556562 +Ref: library/types types ModuleType __package__3556705 +Ref: 16653556705 +Ref: library/types types ModuleType __spec__3557718 +Ref: 16673557718 +Ref: library/types types TracebackType3557944 +Ref: 3353557944 +Ref: library/types types FrameType3558255 +Ref: 16683558255 +Ref: library/types types GetSetDescriptorType3558496 +Ref: 16693558496 +Ref: library/types types MemberDescriptorType3558857 +Ref: 166a3558857 +Ref: library/types types MappingProxyType3559375 +Ref: ab63559375 +Ref: library/types types MappingProxyType copy3560208 +Ref: 166b3560208 +Ref: library/types types MappingProxyType get3560294 +Ref: 166c3560294 +Ref: library/types types MappingProxyType items3560567 +Ref: 166d3560567 +Ref: library/types types MappingProxyType keys3560697 +Ref: 166e3560697 +Ref: library/types types MappingProxyType values3560788 +Ref: 166f3560788 +Node: Additional Utility Classes and Functions3560883 +Ref: library/types additional-utility-classes-and-functions3561091 +Ref: 16703561091 +Ref: library/types types SimpleNamespace3561190 +Ref: 9ac3561190 +Ref: library/types types DynamicClassAttribute3562451 +Ref: 8f33562451 +Node: Coroutine Utility Functions3563108 +Ref: library/types coroutine-utility-functions3563281 +Ref: 16713563281 +Ref: library/types types coroutine3563354 +Ref: 77c3563354 +Node: copy — Shallow and deep copy operations3564178 +Ref: library/copy doc3564374 +Ref: 16723564374 +Ref: library/copy copy-shallow-and-deep-copy-operations3564374 +Ref: 16733564374 +Ref: library/copy module-copy3564374 +Ref: 263564374 +Ref: library/copy copy copy3564929 +Ref: 3cb3564929 +Ref: library/copy copy deepcopy3564994 +Ref: 3cc3564994 +Ref: library/copy copy error3565068 +Ref: 16743565068 +Ref: library/copy shallow-vs-deep-copy3565135 +Ref: ebd3565135 +Ref: copy — Shallow and deep copy operations-Footnote-13567782 +Node: pprint — Data pretty printer3567845 +Ref: library/pprint doc3568022 +Ref: 16753568022 +Ref: library/pprint module-pprint3568022 +Ref: d23568022 +Ref: library/pprint pprint-data-pretty-printer3568022 +Ref: 16763568022 +Ref: library/pprint pprint PrettyPrinter3569020 +Ref: 8953569020 +Ref: library/pprint pprint pformat3571692 +Ref: 8963571692 +Ref: library/pprint pprint pp3572156 +Ref: 2123572156 +Ref: library/pprint pprint pprint3572581 +Ref: 2133572581 +Ref: library/pprint pprint isreadable3573662 +Ref: 16773573662 +Ref: library/pprint pprint isrecursive3573970 +Ref: 16783573970 +Ref: library/pprint pprint saferepr3574122 +Ref: 16793574122 +Ref: pprint — Data pretty printer-Footnote-13574706 +Node: PrettyPrinter Objects3574771 +Ref: library/pprint id13574878 +Ref: 167a3574878 +Ref: library/pprint prettyprinter-objects3574878 +Ref: 167b3574878 +Ref: library/pprint pprint PrettyPrinter pformat3575006 +Ref: 167c3575006 +Ref: library/pprint pprint PrettyPrinter pprint3575205 +Ref: 167d3575205 +Ref: library/pprint pprint PrettyPrinter isreadable3575592 +Ref: 167e3575592 +Ref: library/pprint pprint PrettyPrinter isrecursive3575994 +Ref: 167f3575994 +Ref: library/pprint pprint PrettyPrinter format3576310 +Ref: 16803576310 +Node: Example<3>3577469 +Ref: library/pprint example3577576 +Ref: 16813577576 +Ref: library/pprint pprint-example3577576 +Ref: 16823577576 +Ref: Example<3>-Footnote-13584886 +Node: reprlib — Alternate repr implementation3584911 +Ref: library/reprlib doc3585080 +Ref: 16833585080 +Ref: library/reprlib module-reprlib3585080 +Ref: df3585080 +Ref: library/reprlib reprlib-alternate-repr-implementation3585080 +Ref: 16843585080 +Ref: library/reprlib reprlib Repr3585576 +Ref: 16853585576 +Ref: library/reprlib reprlib aRepr3585860 +Ref: 16863585860 +Ref: library/reprlib reprlib repr3586143 +Ref: 16873586143 +Ref: library/reprlib reprlib recursive_repr3586545 +Ref: b703586545 +Ref: reprlib — Alternate repr implementation-Footnote-13587347 +Node: Repr Objects3587413 +Ref: library/reprlib id13587536 +Ref: 16893587536 +Ref: library/reprlib repr-objects3587536 +Ref: 168a3587536 +Ref: library/reprlib reprlib Repr maxlevel3587777 +Ref: 168b3587777 +Ref: library/reprlib reprlib Repr maxdict3587901 +Ref: 168c3587901 +Ref: library/reprlib reprlib Repr maxlist3587929 +Ref: 168d3587929 +Ref: library/reprlib reprlib Repr maxtuple3587957 +Ref: 168e3587957 +Ref: library/reprlib reprlib Repr maxset3587986 +Ref: 168f3587986 +Ref: library/reprlib reprlib Repr maxfrozenset3588013 +Ref: 16903588013 +Ref: library/reprlib reprlib Repr maxdeque3588046 +Ref: 16913588046 +Ref: library/reprlib reprlib Repr maxarray3588075 +Ref: 16923588075 +Ref: library/reprlib reprlib Repr maxlong3588303 +Ref: 16933588303 +Ref: library/reprlib reprlib Repr maxstring3588472 +Ref: 16943588472 +Ref: library/reprlib reprlib Repr maxother3588829 +Ref: 16953588829 +Ref: library/reprlib reprlib Repr repr3589108 +Ref: 16883589108 +Ref: library/reprlib reprlib Repr repr13589248 +Ref: 16963589248 +Node: Subclassing Repr Objects3590066 +Ref: library/reprlib subclassing-repr-objects3590189 +Ref: 16973590189 +Ref: library/reprlib subclassing-reprs3590189 +Ref: 16983590189 +Node: enum — Support for enumerations3590872 +Ref: library/enum doc3591002 +Ref: 16993591002 +Ref: library/enum enum-support-for-enumerations3591002 +Ref: 169a3591002 +Ref: library/enum module-enum3591002 +Ref: 7b3591002 +Ref: enum — Support for enumerations-Footnote-13592199 +Node: Module Contents<2>3592262 +Ref: library/enum module-contents3592375 +Ref: 169b3592375 +Ref: library/enum enum Enum3592698 +Ref: 3873592698 +Ref: library/enum enum IntEnum3592856 +Ref: 58d3592856 +Ref: library/enum enum IntFlag3592981 +Ref: 14033592981 +Ref: library/enum enum Flag3593244 +Ref: 5453593244 +Ref: library/enum enum auto3593543 +Ref: 5463593543 +Node: Creating an Enum3593729 +Ref: library/enum creating-an-enum3593914 +Ref: 169e3593914 +Node: Programmatic access to enumeration members and their attributes3596360 +Ref: library/enum programmatic-access-to-enumeration-members-and-their-attributes3596562 +Ref: 16a03596562 +Node: Duplicating enum members and values3597320 +Ref: library/enum duplicating-enum-members-and-values3597540 +Ref: 16a13597540 +Node: Ensuring unique enumeration values3598631 +Ref: library/enum ensuring-unique-enumeration-values3598810 +Ref: 16a23598810 +Ref: library/enum enum unique3599109 +Ref: 169c3599109 +Node: Using automatic values3599656 +Ref: library/enum using-automatic-values3599809 +Ref: 16a33599809 +Node: Iteration3601022 +Ref: library/enum iteration3601155 +Ref: 16a43601155 +Node: Comparisons<3>3602017 +Ref: library/enum comparisons3602174 +Ref: 16a53602174 +Node: Allowed members and attributes of enumerations3603098 +Ref: library/enum allowed-members-and-attributes-of-enumerations3603273 +Ref: 16a73603273 +Node: Restricted Enum subclassing3605110 +Ref: library/enum restricted-enum-subclassing3605279 +Ref: 16a93605279 +Node: Pickling3606380 +Ref: library/enum pickling3606517 +Ref: 16ab3606517 +Node: Functional API3607161 +Ref: library/enum functional-api3607291 +Ref: 169d3607291 +Node: Derived Enumerations3610551 +Ref: library/enum derived-enumerations3610704 +Ref: 16ad3610704 +Node: IntEnum3610821 +Ref: library/enum intenum3610901 +Ref: 16a63610901 +Node: IntFlag3612028 +Ref: library/enum intflag3612121 +Ref: 16ae3612121 +Node: Flag3613659 +Ref: library/enum flag3613751 +Ref: 16af3613751 +Node: Others3615869 +Ref: library/enum others3615945 +Ref: 16b03615945 +Node: When to use __new__ vs __init__3617746 +Ref: library/enum when-to-use-new-vs-init3617905 +Ref: 16b13617905 +Node: Interesting examples3619023 +Ref: library/enum interesting-examples3619186 +Ref: 16b23619186 +Node: Omitting values3619633 +Ref: library/enum omitting-values3619725 +Ref: 16b33619725 +Node: Using auto3620777 +Ref: library/enum using-auto3620860 +Ref: 16b43620860 +Node: Using object3621114 +Ref: library/enum using-object3621232 +Ref: 16b53621232 +Node: Using a descriptive string3621498 +Ref: library/enum using-a-descriptive-string3621628 +Ref: 16b63621628 +Node: Using a custom __new__3621951 +Ref: library/enum using-a-custom-new3622060 +Ref: 16b73622060 +Node: OrderedEnum3623763 +Ref: library/enum orderedenum3623881 +Ref: 16aa3623881 +Node: DuplicateFreeEnum3625063 +Ref: library/enum duplicatefreeenum3625172 +Ref: 16b83625172 +Node: Planet3626251 +Ref: library/enum planet3626359 +Ref: 16a83626359 +Node: TimePeriod3627421 +Ref: library/enum timeperiod3627503 +Ref: 16b93627503 +Node: How are Enums different?3628118 +Ref: library/enum how-are-enums-different3628241 +Ref: 169f3628241 +Node: Enum Classes3628533 +Ref: library/enum enum-classes3628641 +Ref: 16ba3628641 +Node: Enum Members aka instances3629219 +Ref: library/enum enum-members-aka-instances3629348 +Ref: 16bb3629348 +Node: Finer Points3629744 +Ref: library/enum finer-points3629852 +Ref: 16bc3629852 +Node: Supported __dunder__ names3630096 +Ref: library/enum supported-dunder-names3630204 +Ref: 16bd3630204 +Node: Supported _sunder_ names3630627 +Ref: library/enum supported-sunder-names3630760 +Ref: 16be3630760 +Node: Enum member type3632262 +Ref: library/enum enum-member-type3632410 +Ref: 16bf3632410 +Node: Boolean value of Enum classes and members3633125 +Ref: library/enum boolean-value-of-enum-classes-and-members3633274 +Ref: 16c03633274 +Node: Enum classes with methods3633847 +Ref: library/enum enum-classes-with-methods3634005 +Ref: 16c13634005 +Node: Combining members of Flag3634541 +Ref: library/enum combining-members-of-flag3634649 +Ref: 16c23634649 +Node: Numeric and Mathematical Modules3635269 +Ref: library/numeric doc3635423 +Ref: 16c33635423 +Ref: library/numeric numeric3635423 +Ref: 16c43635423 +Ref: library/numeric numeric-and-mathematical-modules3635423 +Ref: 16c53635423 +Node: numbers — Numeric abstract base classes3636328 +Ref: library/numbers doc3636478 +Ref: 16c63636478 +Ref: library/numbers module-numbers3636478 +Ref: c13636478 +Ref: library/numbers numbers-numeric-abstract-base-classes3636478 +Ref: 16c73636478 +Ref: library/numbers numbers Number3636901 +Ref: 10c23636901 +Ref: numbers — Numeric abstract base classes-Footnote-13637197 +Ref: numbers — Numeric abstract base classes-Footnote-23637263 +Node: The numeric tower3637312 +Ref: library/numbers the-numeric-tower3637443 +Ref: 16c83637443 +Ref: library/numbers numbers Complex3637496 +Ref: 10c53637496 +Ref: library/numbers numbers Complex real3637936 +Ref: 16c93637936 +Ref: library/numbers numbers Complex imag3638029 +Ref: 16ca3638029 +Ref: library/numbers numbers Complex conjugate3638127 +Ref: 16cb3638127 +Ref: library/numbers numbers Real3638286 +Ref: 10c43638286 +Ref: library/numbers numbers Rational3638793 +Ref: c583638793 +Ref: library/numbers numbers Rational numerator3639030 +Ref: 16cc3639030 +Ref: library/numbers numbers Rational denominator3639082 +Ref: 16cd3639082 +Ref: library/numbers numbers Integral3639136 +Ref: 10c33639136 +Node: Notes for type implementors3639471 +Ref: library/numbers notes-for-type-implementors3639602 +Ref: 16ce3639602 +Node: Adding More Numeric ABCs3640452 +Ref: library/numbers adding-more-numeric-abcs3640587 +Ref: 16cf3640587 +Node: Implementing the arithmetic operations3640933 +Ref: library/numbers id13641068 +Ref: 16d03641068 +Ref: library/numbers implementing-the-arithmetic-operations3641068 +Ref: 10c13641068 +Node: math — Mathematical functions3645864 +Ref: library/math doc3646075 +Ref: 16d13646075 +Ref: library/math math-mathematical-functions3646075 +Ref: 16d23646075 +Ref: library/math module-math3646075 +Ref: b13646075 +Node: Number-theoretic and representation functions3647245 +Ref: library/math number-theoretic-and-representation-functions3647398 +Ref: 16d33647398 +Ref: library/math math ceil3647507 +Ref: d2d3647507 +Ref: library/math math comb3647739 +Ref: 1e73647739 +Ref: library/math math copysign3648350 +Ref: d3d3648350 +Ref: library/math math fabs3648564 +Ref: 16d43648564 +Ref: library/math math factorial3648633 +Ref: 1ea3648633 +Ref: library/math math floor3648784 +Ref: d2c3648784 +Ref: library/math math fmod3649012 +Ref: 11df3649012 +Ref: library/math math frexp3649998 +Ref: 16d53649998 +Ref: library/math math fsum3650363 +Ref: d3e3650363 +Ref: library/math math gcd3651202 +Ref: 7163651202 +Ref: library/math math isclose3651509 +Ref: 6863651509 +Ref: library/math math isfinite3652855 +Ref: b673652855 +Ref: library/math math isinf3653058 +Ref: d3b3653058 +Ref: library/math math isnan3653187 +Ref: d3c3653187 +Ref: library/math math isqrt3653305 +Ref: 1e83653305 +Ref: library/math math ldexp3653823 +Ref: 16d63653823 +Ref: library/math math modf3653958 +Ref: 16d73653958 +Ref: library/math math perm3654098 +Ref: 1e63654098 +Ref: library/math math prod3654639 +Ref: 1e43654639 +Ref: library/math math remainder3655015 +Ref: 3ab3655015 +Ref: library/math math trunc3656041 +Ref: d2e3656041 +Ref: Number-theoretic and representation functions-Footnote-13656929 +Ref: Number-theoretic and representation functions-Footnote-23656982 +Node: Power and logarithmic functions3657031 +Ref: library/math power-and-logarithmic-functions3657216 +Ref: 16d83657216 +Ref: library/math math exp3657297 +Ref: ca93657297 +Ref: library/math math expm13657515 +Ref: b683657515 +Ref: library/math math log3658128 +Ref: e113658128 +Ref: library/math math log1p3658363 +Ref: d423658363 +Ref: library/math math log23658526 +Ref: a203658526 +Ref: library/math math log103658866 +Ref: 16d93658866 +Ref: library/math math pow3658998 +Ref: 16da3658998 +Ref: library/math math sqrt3659694 +Ref: 1e93659694 +Ref: Power and logarithmic functions-Footnote-13659796 +Node: Trigonometric functions3659855 +Ref: library/math trigonometric-functions3660013 +Ref: 16db3660013 +Ref: library/math math acos3660078 +Ref: 16dc3660078 +Ref: library/math math asin3660155 +Ref: 16dd3660155 +Ref: library/math math atan3660230 +Ref: 16de3660230 +Ref: library/math math atan23660308 +Ref: 16df3660308 +Ref: library/math math cos3660820 +Ref: e103660820 +Ref: library/math math dist3660888 +Ref: 1e23660888 +Ref: library/math math hypot3661217 +Ref: 1e33661217 +Ref: library/math math sin3661742 +Ref: e0f3661742 +Ref: library/math math tan3661808 +Ref: 16e03661808 +Node: Angular conversion3661877 +Ref: library/math angular-conversion3662024 +Ref: 16e13662024 +Ref: library/math math degrees3662079 +Ref: 16e23662079 +Ref: library/math math radians3662160 +Ref: 16e33662160 +Node: Hyperbolic functions3662241 +Ref: library/math hyperbolic-functions3662382 +Ref: 16e43662382 +Ref: library/math math acosh3662554 +Ref: d3f3662554 +Ref: library/math math asinh3662635 +Ref: d403662635 +Ref: library/math math atanh3662714 +Ref: d413662714 +Ref: library/math math cosh3662796 +Ref: 16e53662796 +Ref: library/math math sinh3662868 +Ref: 16e63662868 +Ref: library/math math tanh3662938 +Ref: 16e73662938 +Ref: Hyperbolic functions-Footnote-13663047 +Node: Special functions3663105 +Ref: library/math special-functions3663240 +Ref: 16e83663240 +Ref: library/math math erf3663293 +Ref: 4523663293 +Ref: library/math math erfc3663713 +Ref: 4533663713 +Ref: library/math math gamma3664012 +Ref: b693664012 +Ref: library/math math lgamma3664111 +Ref: b6a3664111 +Ref: Special functions-Footnote-13664296 +Ref: Special functions-Footnote-23664349 +Ref: Special functions-Footnote-33664441 +Ref: Special functions-Footnote-43664494 +Ref: Special functions-Footnote-53664553 +Node: Constants<2>3664606 +Ref: library/math constants3664712 +Ref: 16e93664712 +Ref: library/math math pi3664749 +Ref: 16ea3664749 +Ref: library/math math e3664844 +Ref: 16eb3664844 +Ref: library/math math tau3664937 +Ref: 16ec3664937 +Ref: library/math math inf3665315 +Ref: 5283665315 +Ref: library/math math nan3665504 +Ref: 5293665504 +Ref: Constants<2>-Footnote-13666834 +Ref: Constants<2>-Footnote-23666886 +Node: cmath — Mathematical functions for complex numbers3666914 +Ref: library/cmath doc3667145 +Ref: 16ed3667145 +Ref: library/cmath cmath-mathematical-functions-for-complex-numbers3667145 +Ref: 16ee3667145 +Ref: library/cmath module-cmath3667145 +Ref: 193667145 +Node: Conversions to and from polar coordinates3668437 +Ref: library/cmath conversions-to-and-from-polar-coordinates3668610 +Ref: 16ef3668610 +Ref: library/cmath cmath phase3669462 +Ref: 16f03669462 +Ref: library/cmath cmath polar3670368 +Ref: 16f13670368 +Ref: library/cmath cmath rect3670621 +Ref: 16f23670621 +Node: Power and logarithmic functions<2>3670795 +Ref: library/cmath power-and-logarithmic-functions3671003 +Ref: 16f33671003 +Ref: library/cmath cmath exp3671084 +Ref: 16f43671084 +Ref: library/cmath cmath log3671205 +Ref: 16f53671205 +Ref: library/cmath cmath log103671478 +Ref: 16f63671478 +Ref: library/cmath cmath sqrt3671609 +Ref: 16f73671609 +Node: Trigonometric functions<2>3671733 +Ref: library/cmath trigonometric-functions3671923 +Ref: 16f83671923 +Ref: library/cmath cmath acos3671988 +Ref: 16f93671988 +Ref: library/cmath cmath asin3672257 +Ref: 16fa3672257 +Ref: library/cmath cmath atan3672380 +Ref: 16fb3672380 +Ref: library/cmath cmath cos3672684 +Ref: 16fc3672684 +Ref: library/cmath cmath sin3672745 +Ref: 16fd3672745 +Ref: library/cmath cmath tan3672804 +Ref: 16fe3672804 +Node: Hyperbolic functions<2>3672866 +Ref: library/cmath hyperbolic-functions3673046 +Ref: 16ff3673046 +Ref: library/cmath cmath acosh3673105 +Ref: 17003673105 +Ref: library/cmath cmath asinh3673297 +Ref: 17013673297 +Ref: library/cmath cmath atanh3673614 +Ref: 17023673614 +Ref: library/cmath cmath cosh3673913 +Ref: 17033673913 +Ref: library/cmath cmath sinh3673986 +Ref: 17043673986 +Ref: library/cmath cmath tanh3674057 +Ref: 17053674057 +Node: Classification functions3674131 +Ref: library/cmath classification-functions3674297 +Ref: 17063674297 +Ref: library/cmath cmath isfinite3674364 +Ref: 17073674364 +Ref: library/cmath cmath isinf3674536 +Ref: 17083674536 +Ref: library/cmath cmath isnan3674687 +Ref: 17093674687 +Ref: library/cmath cmath isclose3674832 +Ref: 6873674832 +Ref: Classification functions-Footnote-13676215 +Node: Constants<3>3676264 +Ref: library/cmath constants3676398 +Ref: 170a3676398 +Ref: library/cmath cmath pi3676435 +Ref: 170b3676435 +Ref: library/cmath cmath e3676505 +Ref: 170c3676505 +Ref: library/cmath cmath tau3676573 +Ref: 5253676573 +Ref: library/cmath cmath inf3676670 +Ref: 5263676670 +Ref: library/cmath cmath infj3676792 +Ref: 52a3676792 +Ref: library/cmath cmath nan3676973 +Ref: 5273676973 +Ref: library/cmath cmath nanj3677115 +Ref: 52b3677115 +Node: decimal — Decimal fixed point and floating point arithmetic3678526 +Ref: library/decimal doc3678756 +Ref: 170d3678756 +Ref: library/decimal decimal-decimal-fixed-point-and-floating-point-arithmetic3678756 +Ref: 170e3678756 +Ref: library/decimal module-decimal3678756 +Ref: 363678756 +Ref: decimal — Decimal fixed point and floating point arithmetic-Footnote-13683879 +Ref: decimal — Decimal fixed point and floating point arithmetic-Footnote-23683945 +Node: Quick-start Tutorial3683998 +Ref: library/decimal decimal-tutorial3684140 +Ref: 171e3684140 +Ref: library/decimal quick-start-tutorial3684140 +Ref: 171f3684140 +Node: Decimal objects3691360 +Ref: library/decimal decimal-decimal3691526 +Ref: 17223691526 +Ref: library/decimal decimal-objects3691526 +Ref: 17233691526 +Ref: library/decimal decimal Decimal3691575 +Ref: 1883691575 +Ref: library/decimal decimal Decimal adjusted3696701 +Ref: 17243696701 +Ref: library/decimal decimal Decimal as_integer_ratio3697052 +Ref: 53b3697052 +Ref: library/decimal decimal Decimal as_tuple3697491 +Ref: 17253697491 +Ref: library/decimal decimal Decimal canonical3697649 +Ref: 17263697649 +Ref: library/decimal decimal Decimal compare3697888 +Ref: 17273697888 +Ref: library/decimal decimal Decimal compare_signal3698321 +Ref: 17283698321 +Ref: library/decimal decimal Decimal compare_total3698633 +Ref: 17293698633 +Ref: library/decimal decimal Decimal compare_total_mag3699910 +Ref: 172a3699910 +Ref: library/decimal decimal Decimal conjugate3700515 +Ref: 172b3700515 +Ref: library/decimal decimal Decimal copy_abs3700648 +Ref: 172c3700648 +Ref: library/decimal decimal Decimal copy_negate3700862 +Ref: 172d3700862 +Ref: library/decimal decimal Decimal copy_sign3701073 +Ref: 172e3701073 +Ref: library/decimal decimal Decimal exp3701610 +Ref: 9ed3701610 +Ref: library/decimal decimal Decimal from_float3702044 +Ref: b803702044 +Ref: library/decimal decimal Decimal fma3703146 +Ref: 172f3703146 +Ref: library/decimal decimal Decimal is_canonical3703390 +Ref: 17303703390 +Ref: library/decimal decimal Decimal is_finite3703665 +Ref: 17313703665 +Ref: library/decimal decimal Decimal is_infinite3703840 +Ref: 17323703840 +Ref: library/decimal decimal Decimal is_nan3704007 +Ref: 17333704007 +Ref: library/decimal decimal Decimal is_normal3704159 +Ref: 17343704159 +Ref: library/decimal decimal Decimal is_qnan3704383 +Ref: 17353704383 +Ref: library/decimal decimal Decimal is_signed3704522 +Ref: 17363704522 +Ref: library/decimal decimal Decimal is_snan3704725 +Ref: 17373704725 +Ref: library/decimal decimal Decimal is_subnormal3704867 +Ref: 17383704867 +Ref: library/decimal decimal Decimal is_zero3705021 +Ref: 17393705021 +Ref: library/decimal decimal Decimal ln3705177 +Ref: 9ee3705177 +Ref: library/decimal decimal Decimal log103705385 +Ref: 173a3705385 +Ref: library/decimal decimal Decimal logb3705588 +Ref: 173b3705588 +Ref: library/decimal decimal Decimal logical_and3705975 +Ref: 173c3705975 +Ref: library/decimal decimal Decimal logical_invert3706240 +Ref: 173e3706240 +Ref: library/decimal decimal Decimal logical_or3706419 +Ref: 173f3706419 +Ref: library/decimal decimal Decimal logical_xor3706681 +Ref: 17403706681 +Ref: library/decimal decimal Decimal max3706949 +Ref: 17413706949 +Ref: library/decimal decimal Decimal max_mag3707253 +Ref: 17423707253 +Ref: library/decimal decimal Decimal min3707434 +Ref: 17433707434 +Ref: library/decimal decimal Decimal min_mag3707738 +Ref: 17443707738 +Ref: library/decimal decimal Decimal next_minus3707919 +Ref: 17453707919 +Ref: library/decimal decimal Decimal next_plus3708157 +Ref: 17463708157 +Ref: library/decimal decimal Decimal next_toward3708394 +Ref: 17473708394 +Ref: library/decimal decimal Decimal normalize3708761 +Ref: 17483708761 +Ref: library/decimal decimal Decimal number_class3709217 +Ref: 17493709217 +Ref: library/decimal decimal Decimal quantize3710387 +Ref: 9f13710387 +Ref: library/decimal decimal Decimal radix3711680 +Ref: 174a3711680 +Ref: library/decimal decimal Decimal remainder_near3711897 +Ref: 174b3711897 +Ref: library/decimal decimal Decimal rotate3712741 +Ref: 174c3712741 +Ref: library/decimal decimal Decimal same_quantum3713414 +Ref: 174d3713414 +Ref: library/decimal decimal Decimal scaleb3713826 +Ref: 174e3713826 +Ref: library/decimal decimal Decimal shift3714077 +Ref: 174f3714077 +Ref: library/decimal decimal Decimal sqrt3714683 +Ref: 17503714683 +Ref: library/decimal decimal Decimal to_eng_string3714790 +Ref: 17513714790 +Ref: library/decimal decimal Decimal to_integral3715266 +Ref: 17523715266 +Ref: library/decimal decimal Decimal to_integral_exact3715496 +Ref: 17543715496 +Ref: library/decimal decimal Decimal to_integral_value3715926 +Ref: 17533715926 +Node: Logical operands3716280 +Ref: library/decimal logical-operands3716348 +Ref: 17553716348 +Ref: library/decimal logical-operands-label3716348 +Ref: 173d3716348 +Node: Context objects3716695 +Ref: library/decimal context-objects3716853 +Ref: 17563716853 +Ref: library/decimal decimal-context3716853 +Ref: 17573716853 +Ref: library/decimal decimal getcontext3717237 +Ref: 17203717237 +Ref: library/decimal decimal setcontext3717330 +Ref: 17213717330 +Ref: library/decimal decimal localcontext3717564 +Ref: 13883717564 +Ref: library/decimal decimal BasicContext3718489 +Ref: 9e93718489 +Ref: library/decimal decimal ExtendedContext3718941 +Ref: 9ea3718941 +Ref: library/decimal decimal DefaultContext3719574 +Ref: 9e83719574 +Ref: library/decimal decimal Context3720676 +Ref: 9f03720676 +Ref: library/decimal decimal Context clear_flags3723595 +Ref: 175c3723595 +Ref: library/decimal decimal Context clear_traps3723675 +Ref: 175d3723675 +Ref: library/decimal decimal Context copy3723786 +Ref: 175e3723786 +Ref: library/decimal decimal Context copy_decimal3723858 +Ref: 175f3723858 +Ref: library/decimal decimal Context create_decimal3723949 +Ref: 9ec3723949 +Ref: library/decimal decimal Context create_decimal_from_float3725053 +Ref: 17603725053 +Ref: library/decimal decimal Context Etiny3725816 +Ref: 17613725816 +Ref: library/decimal decimal Context Etop3726046 +Ref: 17623726046 +Ref: library/decimal decimal Context abs3726552 +Ref: 17633726552 +Ref: library/decimal decimal Context add3726624 +Ref: 17643726624 +Ref: library/decimal decimal Context canonical3726695 +Ref: c9d3726695 +Ref: library/decimal decimal Context compare3726775 +Ref: 17653726775 +Ref: library/decimal decimal Context compare_signal3726853 +Ref: 17663726853 +Ref: library/decimal decimal Context compare_total3726957 +Ref: 17673726957 +Ref: library/decimal decimal Context compare_total_mag3727066 +Ref: 17683727066 +Ref: library/decimal decimal Context copy_abs3727204 +Ref: 17693727204 +Ref: library/decimal decimal Context copy_negate3727292 +Ref: 176a3727292 +Ref: library/decimal decimal Context copy_sign3727383 +Ref: 176b3727383 +Ref: library/decimal decimal Context divide3727462 +Ref: 176c3727462 +Ref: library/decimal decimal Context divide_int3727532 +Ref: 176d3727532 +Ref: library/decimal decimal Context divmod3727631 +Ref: 176e3727631 +Ref: library/decimal decimal Context exp3727748 +Ref: 176f3727748 +Ref: library/decimal decimal Context fma3727807 +Ref: 17703727807 +Ref: library/decimal decimal Context is_canonical3727891 +Ref: c9e3727891 +Ref: library/decimal decimal Context is_finite3728008 +Ref: 17713728008 +Ref: library/decimal decimal Context is_infinite3728119 +Ref: 17723728119 +Ref: library/decimal decimal Context is_nan3728234 +Ref: 17733728234 +Ref: library/decimal decimal Context is_normal3728360 +Ref: 17743728360 +Ref: library/decimal decimal Context is_qnan3728490 +Ref: 17753728490 +Ref: library/decimal decimal Context is_signed3728614 +Ref: 17763728614 +Ref: library/decimal decimal Context is_snan3728727 +Ref: 17773728727 +Ref: library/decimal decimal Context is_subnormal3728855 +Ref: 17783728855 +Ref: library/decimal decimal Context is_zero3728972 +Ref: 17793728972 +Ref: library/decimal decimal Context ln3729081 +Ref: 177a3729081 +Ref: library/decimal decimal Context log103729164 +Ref: 177b3729164 +Ref: library/decimal decimal Context logb3729241 +Ref: 177c3729241 +Ref: library/decimal decimal Context logical_and3729341 +Ref: 177d3729341 +Ref: library/decimal decimal Context logical_invert3729468 +Ref: 177e3729468 +Ref: library/decimal decimal Context logical_or3729546 +Ref: 177f3729546 +Ref: library/decimal decimal Context logical_xor3729671 +Ref: 17803729671 +Ref: library/decimal decimal Context max3729798 +Ref: 17813729798 +Ref: library/decimal decimal Context max_mag3729895 +Ref: 17823729895 +Ref: library/decimal decimal Context min3729996 +Ref: 17833729996 +Ref: library/decimal decimal Context min_mag3730093 +Ref: 17843730093 +Ref: library/decimal decimal Context minus3730194 +Ref: 17853730194 +Ref: library/decimal decimal Context multiply3730307 +Ref: 17863730307 +Ref: library/decimal decimal Context next_minus3730387 +Ref: 17873730387 +Ref: library/decimal decimal Context next_plus3730490 +Ref: 17883730490 +Ref: library/decimal decimal Context next_toward3730592 +Ref: 17893730592 +Ref: library/decimal decimal Context normalize3730701 +Ref: 178a3730701 +Ref: library/decimal decimal Context number_class3730778 +Ref: 178b3730778 +Ref: library/decimal decimal Context plus3730867 +Ref: 178c3730867 +Ref: library/decimal decimal Context power3731084 +Ref: 178d3731084 +Ref: library/decimal decimal Context quantize3732919 +Ref: 178e3732919 +Ref: library/decimal decimal Context radix3733042 +Ref: 178f3733042 +Ref: library/decimal decimal Context remainder3733120 +Ref: 17903733120 +Ref: library/decimal decimal Context remainder_near3733315 +Ref: 17913733315 +Ref: library/decimal decimal Context rotate3733535 +Ref: 17923733535 +Ref: library/decimal decimal Context same_quantum3733620 +Ref: 17933733620 +Ref: library/decimal decimal Context scaleb3733732 +Ref: 17943733732 +Ref: library/decimal decimal Context shift3733850 +Ref: 17953733850 +Ref: library/decimal decimal Context sqrt3733934 +Ref: 17963733934 +Ref: library/decimal decimal Context subtract3734031 +Ref: 17973734031 +Ref: library/decimal decimal Context to_eng_string3734119 +Ref: 17983734119 +Ref: library/decimal decimal Context to_integral_exact3734483 +Ref: 17993734483 +Ref: library/decimal decimal Context to_sci_string3734556 +Ref: 179a3734556 +Node: Constants<4>3734660 +Ref: library/decimal constants3734817 +Ref: 179b3734817 +Ref: library/decimal decimal-rounding-modes3734817 +Ref: 179c3734817 +Ref: library/decimal decimal MAX_PREC3735254 +Ref: 17583735254 +Ref: library/decimal decimal MAX_EMAX3735389 +Ref: 175b3735389 +Ref: library/decimal decimal MIN_EMIN3735524 +Ref: 175a3735524 +Ref: library/decimal decimal MIN_ETINY3735660 +Ref: 179d3735660 +Ref: library/decimal decimal HAVE_THREADS3735788 +Ref: 9e63735788 +Ref: library/decimal decimal HAVE_CONTEXTVAR3735939 +Ref: 179e3735939 +Node: Rounding modes3736294 +Ref: library/decimal rounding-modes3736443 +Ref: 17593736443 +Ref: library/decimal decimal ROUND_CEILING3736490 +Ref: 170f3736490 +Ref: library/decimal decimal ROUND_DOWN3736559 +Ref: 17103736559 +Ref: library/decimal decimal ROUND_FLOOR3736615 +Ref: 17113736615 +Ref: library/decimal decimal ROUND_HALF_DOWN3736683 +Ref: 17123736683 +Ref: library/decimal decimal ROUND_HALF_EVEN3736771 +Ref: 17133736771 +Ref: library/decimal decimal ROUND_HALF_UP3736870 +Ref: 17143736870 +Ref: library/decimal decimal ROUND_UP3736958 +Ref: 17153736958 +Ref: library/decimal decimal ROUND_05UP3737014 +Ref: 17163737014 +Node: Signals3737172 +Ref: library/decimal decimal-signals3737329 +Ref: 179f3737329 +Ref: library/decimal signals3737329 +Ref: 17a03737329 +Ref: library/decimal decimal Clamped3738046 +Ref: 17173738046 +Ref: library/decimal decimal DecimalException3738340 +Ref: 17a13738340 +Ref: library/decimal decimal DivisionByZero3738464 +Ref: 17183738464 +Ref: library/decimal decimal Inexact3738802 +Ref: 17193738802 +Ref: library/decimal decimal InvalidOperation3739077 +Ref: 9eb3739077 +Ref: library/decimal decimal Overflow3739518 +Ref: 171c3739518 +Ref: library/decimal decimal Rounded3739921 +Ref: 171a3739921 +Ref: library/decimal decimal Subnormal3740259 +Ref: 171b3740259 +Ref: library/decimal decimal Underflow3740481 +Ref: 171d3740481 +Ref: library/decimal decimal FloatOperation3740706 +Ref: 9e53740706 +Node: Floating Point Notes3741978 +Ref: library/decimal decimal-notes3742141 +Ref: 17a23742141 +Ref: library/decimal floating-point-notes3742141 +Ref: 17a33742141 +Node: Mitigating round-off error with increased precision3742285 +Ref: library/decimal mitigating-round-off-error-with-increased-precision3742416 +Ref: 17a43742416 +Node: Special values3744105 +Ref: library/decimal special-values3744236 +Ref: 17a53744236 +Node: Working with threads3747357 +Ref: library/decimal decimal-threads3747520 +Ref: 17a63747520 +Ref: library/decimal working-with-threads3747520 +Ref: 17a73747520 +Node: Recipes3748893 +Ref: library/decimal decimal-recipes3749047 +Ref: 17a83749047 +Ref: library/decimal recipes3749047 +Ref: 17a93749047 +Node: Decimal FAQ3753892 +Ref: library/decimal decimal-faq3754017 +Ref: 17aa3754017 +Ref: library/decimal id13754017 +Ref: 17ab3754017 +Ref: Decimal FAQ-Footnote-13760328 +Ref: Decimal FAQ-Footnote-23760395 +Ref: Decimal FAQ-Footnote-33760453 +Node: fractions — Rational numbers3760556 +Ref: library/fractions doc3760775 +Ref: 17ac3760775 +Ref: library/fractions fractions-rational-numbers3760775 +Ref: 17ad3760775 +Ref: library/fractions module-fractions3760775 +Ref: 833760775 +Ref: library/fractions fractions Fraction3761155 +Ref: 1853761155 +Ref: library/fractions fractions Fraction numerator3764329 +Ref: 17af3764329 +Ref: library/fractions fractions Fraction denominator3764413 +Ref: 17b03764413 +Ref: library/fractions fractions Fraction as_integer_ratio3764501 +Ref: 17b13764501 +Ref: library/fractions fractions Fraction from_float3764693 +Ref: b813764693 +Ref: library/fractions fractions Fraction from_decimal3765152 +Ref: b823765152 +Ref: library/fractions fractions Fraction limit_denominator3765551 +Ref: 17ae3765551 +Ref: library/fractions fractions Fraction __floor__3766425 +Ref: 17b23766425 +Ref: library/fractions fractions Fraction __ceil__3766725 +Ref: 17b33766725 +Ref: library/fractions fractions Fraction __round__3766905 +Ref: 17b43766905 +Ref: library/fractions fractions gcd3767375 +Ref: 7173767375 +Ref: fractions — Rational numbers-Footnote-13767976 +Node: random — Generate pseudo-random numbers3768044 +Ref: library/random doc3768250 +Ref: 17b53768250 +Ref: library/random module-random3768250 +Ref: dc3768250 +Ref: library/random random-generate-pseudo-random-numbers3768250 +Ref: 17b63768250 +Ref: random — Generate pseudo-random numbers-Footnote-13771240 +Ref: random — Generate pseudo-random numbers-Footnote-23771305 +Node: Bookkeeping functions3771358 +Ref: library/random bookkeeping-functions3771488 +Ref: 17ba3771488 +Ref: library/random random seed3771549 +Ref: c083771549 +Ref: library/random random getstate3772439 +Ref: 17bb3772439 +Ref: library/random random setstate3772637 +Ref: 17bc3772637 +Ref: library/random random getrandbits3772915 +Ref: 9333772915 +Node: Functions for integers3773275 +Ref: library/random functions-for-integers3773437 +Ref: 17bd3773437 +Ref: library/random random randrange3773500 +Ref: bbb3773500 +Ref: library/random random randint3774202 +Ref: bbc3774202 +Node: Functions for sequences3774343 +Ref: library/random functions-for-sequences3774509 +Ref: 17be3774509 +Ref: library/random random choice3774574 +Ref: bbd3774574 +Ref: library/random random choices3774729 +Ref: 5743774729 +Ref: library/random random shuffle3776603 +Ref: bbe3776603 +Ref: library/random random sample3777369 +Ref: bbf3777369 +Node: Real-valued distributions3778438 +Ref: library/random real-valued-distributions3778603 +Ref: 17bf3778603 +Ref: library/random random random3778942 +Ref: 17b73778942 +Ref: library/random random uniform3779055 +Ref: 17c03779055 +Ref: library/random random triangular3779394 +Ref: 17c13779394 +Ref: library/random random betavariate3779751 +Ref: 17c23779751 +Ref: library/random random expovariate3779939 +Ref: 17c33779939 +Ref: library/random random gammavariate3780318 +Ref: 17c43780318 +Ref: library/random random gauss3780720 +Ref: 17c53780720 +Ref: library/random random lognormvariate3780944 +Ref: 17c73780944 +Ref: library/random random normalvariate3781243 +Ref: 17c63781243 +Ref: library/random random vonmisesvariate3781381 +Ref: 17c83781381 +Ref: library/random random paretovariate3781723 +Ref: 17c93781723 +Ref: library/random random weibullvariate3781827 +Ref: 17ca3781827 +Node: Alternative Generator3781978 +Ref: library/random alternative-generator3782144 +Ref: 17cb3782144 +Ref: library/random random Random3782205 +Ref: 17b83782205 +Ref: library/random random SystemRandom3782354 +Ref: 17b93782354 +Node: Notes on Reproducibility3782851 +Ref: library/random notes-on-reproducibility3783012 +Ref: 17cc3783012 +Node: Examples and Recipes3783710 +Ref: library/random examples-and-recipes3783841 +Ref: 17cd3783841 +Ref: library/random random-examples3783841 +Ref: b583783841 +Ref: Examples and Recipes-Footnote-13789270 +Ref: Examples and Recipes-Footnote-23789335 +Ref: Examples and Recipes-Footnote-33789416 +Ref: Examples and Recipes-Footnote-43789462 +Ref: Examples and Recipes-Footnote-53789514 +Ref: Examples and Recipes-Footnote-63789569 +Ref: Examples and Recipes-Footnote-73789645 +Ref: Examples and Recipes-Footnote-83789680 +Ref: Examples and Recipes-Footnote-93789758 +Node: statistics — Mathematical statistics functions3789793 +Ref: library/statistics doc3789960 +Ref: 17ce3789960 +Ref: library/statistics module-statistics3789960 +Ref: f53789960 +Ref: library/statistics statistics-mathematical-statistics-functions3789960 +Ref: 17cf3789960 +Ref: statistics — Mathematical statistics functions-Footnote-13791254 +Ref: statistics — Mathematical statistics functions-Footnote-23791323 +Ref: statistics — Mathematical statistics functions-Footnote-33791349 +Node: Averages and measures of central location3791380 +Ref: library/statistics averages-and-measures-of-central-location3791533 +Ref: 17d03791533 +Node: Measures of spread3793056 +Ref: library/statistics measures-of-spread3793234 +Ref: 17d53793234 +Node: Function details3793860 +Ref: library/statistics function-details3794010 +Ref: 17da3794010 +Ref: library/statistics statistics mean3794212 +Ref: 1993794212 +Ref: library/statistics statistics fmean3795818 +Ref: 2273795818 +Ref: library/statistics statistics geometric_mean3796231 +Ref: 2283796231 +Ref: library/statistics statistics harmonic_mean3796918 +Ref: 58f3796918 +Ref: library/statistics statistics median3798388 +Ref: 17d13798388 +Ref: library/statistics statistics median_low3799383 +Ref: 17d23799383 +Ref: library/statistics statistics median_high3800024 +Ref: 17d33800024 +Ref: library/statistics statistics median_grouped3800662 +Ref: 17d43800662 +Ref: library/statistics statistics mode3802321 +Ref: 2bb3802321 +Ref: library/statistics statistics multimode3803487 +Ref: 2293803487 +Ref: library/statistics statistics pstdev3803906 +Ref: 17d63803906 +Ref: library/statistics statistics pvariance3804206 +Ref: 17d73804206 +Ref: library/statistics statistics stdev3806472 +Ref: 17d83806472 +Ref: library/statistics statistics variance3806764 +Ref: 17d93806764 +Ref: library/statistics statistics quantiles3809029 +Ref: 22a3809029 +Ref: Function details-Footnote-13811797 +Ref: Function details-Footnote-23811892 +Node: Exceptions<3>3811972 +Ref: library/statistics exceptions3812122 +Ref: 17dc3812122 +Ref: library/statistics statistics StatisticsError3812193 +Ref: 17db3812193 +Node: NormalDist objects3812317 +Ref: library/statistics normaldist-objects3812442 +Ref: 17dd3812442 +Ref: library/statistics statistics NormalDist3812830 +Ref: 22b3812830 +Ref: library/statistics statistics NormalDist mean3813095 +Ref: 17de3813095 +Ref: library/statistics statistics NormalDist median3813216 +Ref: 17df3813216 +Ref: library/statistics statistics NormalDist mode3813330 +Ref: 17e03813330 +Ref: library/statistics statistics NormalDist stdev3813430 +Ref: 17e13813430 +Ref: library/statistics statistics NormalDist variance3813555 +Ref: 17e23813555 +Ref: library/statistics statistics NormalDist from_samples3813721 +Ref: 17e33813721 +Ref: library/statistics statistics NormalDist samples3814329 +Ref: 17e43814329 +Ref: library/statistics statistics NormalDist pdf3814734 +Ref: 17e53814734 +Ref: library/statistics statistics NormalDist cdf3815335 +Ref: 17e63815335 +Ref: library/statistics statistics NormalDist inv_cdf3815578 +Ref: 17e73815578 +Ref: library/statistics statistics NormalDist overlap3816017 +Ref: 17e83816017 +Ref: library/statistics statistics NormalDist quantiles3816273 +Ref: 17e93816273 +Ref: NormalDist objects-Footnote-13817934 +Ref: NormalDist objects-Footnote-23817998 +Ref: NormalDist objects-Footnote-33818058 +Ref: NormalDist objects-Footnote-43818112 +Ref: NormalDist objects-Footnote-53818169 +Ref: NormalDist objects-Footnote-63818223 +Ref: NormalDist objects-Footnote-73818268 +Ref: NormalDist objects-Footnote-83818324 +Ref: NormalDist objects-Footnote-93818381 +Ref: NormalDist objects-Footnote-103818428 +Ref: NormalDist objects-Footnote-113818496 +Ref: NormalDist objects-Footnote-123818568 +Ref: NormalDist objects-Footnote-133818625 +Ref: NormalDist objects-Footnote-143818717 +Ref: NormalDist objects-Footnote-153818764 +Node: NormalDist Examples and Recipes3818849 +Ref: library/statistics normaldist-examples-and-recipes3818935 +Ref: 17ea3818935 +Ref: NormalDist Examples and Recipes-Footnote-13823505 +Ref: NormalDist Examples and Recipes-Footnote-23823576 +Ref: NormalDist Examples and Recipes-Footnote-33823623 +Ref: NormalDist Examples and Recipes-Footnote-43823668 +Ref: NormalDist Examples and Recipes-Footnote-53823725 +Ref: NormalDist Examples and Recipes-Footnote-63823788 +Ref: NormalDist Examples and Recipes-Footnote-73823869 +Ref: NormalDist Examples and Recipes-Footnote-83823925 +Node: Functional Programming Modules3823995 +Ref: library/functional doc3824164 +Ref: 17eb3824164 +Ref: library/functional functional-programming-modules3824164 +Ref: 17ec3824164 +Node: itertools — Functions creating iterators for efficient looping3824647 +Ref: library/itertools doc3824858 +Ref: 17ed3824858 +Ref: library/itertools itertools-functions-creating-iterators-for-efficient-looping3824858 +Ref: 17ee3824858 +Ref: library/itertools module-itertools3824858 +Ref: a33824858 +Node: Itertool functions3835248 +Ref: library/itertools itertool-functions3835393 +Ref: 17f73835393 +Ref: library/itertools itertools-functions3835393 +Ref: 17f83835393 +Ref: library/itertools itertools accumulate3835640 +Ref: 1dd3835640 +Ref: library/itertools itertools chain3839158 +Ref: 12ad3839158 +Ref: library/itertools itertools chain from_iterable3839668 +Ref: 17f13839668 +Ref: library/itertools itertools combinations3840107 +Ref: ca63840107 +Ref: library/itertools itertools combinations_with_replacement3842081 +Ref: c193842081 +Ref: library/itertools itertools compress3844060 +Ref: c1a3844060 +Ref: library/itertools itertools count3844576 +Ref: c1b3844576 +Ref: library/itertools itertools cycle3845420 +Ref: 17ef3845420 +Ref: library/itertools itertools dropwhile3846140 +Ref: 17f23846140 +Ref: library/itertools itertools filterfalse3846841 +Ref: 12963846841 +Ref: library/itertools itertools groupby3847408 +Ref: 17f33847408 +Ref: library/itertools itertools islice3850144 +Ref: 3a23850144 +Ref: library/itertools itertools permutations3852232 +Ref: 17f63852232 +Ref: library/itertools itertools product3854678 +Ref: ca73854678 +Ref: library/itertools itertools repeat3856054 +Ref: 17f03856054 +Ref: library/itertools itertools starmap3856924 +Ref: a283856924 +Ref: library/itertools itertools takewhile3857610 +Ref: 17f43857610 +Ref: library/itertools itertools tee3858052 +Ref: 17f53858052 +Ref: library/itertools itertools zip_longest3859781 +Ref: c313859781 +Ref: Itertool functions-Footnote-13861257 +Node: Itertools Recipes3861315 +Ref: library/itertools id13861460 +Ref: 17fa3861460 +Ref: library/itertools itertools-recipes3861460 +Ref: bf53861460 +Ref: Itertools Recipes-Footnote-13870500 +Node: functools — Higher-order functions and operations on callable objects3870549 +Ref: library/functools doc3870813 +Ref: 17fb3870813 +Ref: library/functools functools-higher-order-functions-and-operations-on-callable-objects3870813 +Ref: 17fc3870813 +Ref: library/functools module-functools3870813 +Ref: 853870813 +Ref: library/functools functools cached_property3871356 +Ref: 1c83871356 +Ref: library/functools functools cmp_to_key3872661 +Ref: b563872661 +Ref: library/functools functools lru_cache3873705 +Ref: 1c73873705 +Ref: library/functools functools total_ordering3877807 +Ref: 84b3877807 +Ref: library/functools functools partial3879720 +Ref: 7c83879720 +Ref: library/functools functools partialmethod3881103 +Ref: 29c3881103 +Ref: library/functools functools reduce3882847 +Ref: c6f3882847 +Ref: library/functools functools singledispatch3884054 +Ref: 38a3884054 +Ref: library/functools functools singledispatchmethod3888257 +Ref: 1c93888257 +Ref: library/functools functools update_wrapper3889878 +Ref: 95a3889878 +Ref: library/functools functools wraps3892325 +Ref: 86a3892325 +Ref: functools — Higher-order functions and operations on callable objects-Footnote-13893623 +Ref: functools — Higher-order functions and operations on callable objects-Footnote-23893691 +Ref: functools — Higher-order functions and operations on callable objects-Footnote-33893755 +Ref: functools — Higher-order functions and operations on callable objects-Footnote-43893810 +Ref: functools — Higher-order functions and operations on callable objects-Footnote-53893868 +Node: partial Objects3893911 +Ref: library/functools id13894034 +Ref: 17fe3894034 +Ref: library/functools partial-objects3894034 +Ref: 17fd3894034 +Ref: library/functools functools partial func3894215 +Ref: 17ff3894215 +Ref: library/functools functools partial args3894404 +Ref: 18003894404 +Ref: library/functools functools partial keywords3894576 +Ref: 18013894576 +Node: operator — Standard operators as functions3895138 +Ref: library/operator doc3895329 +Ref: 18023895329 +Ref: library/operator module-operator3895329 +Ref: c23895329 +Ref: library/operator operator-standard-operators-as-functions3895329 +Ref: 18033895329 +Ref: library/operator operator lt3896262 +Ref: 18043896262 +Ref: library/operator operator le3896295 +Ref: 18053896295 +Ref: library/operator operator eq3896328 +Ref: 18063896328 +Ref: library/operator operator ne3896361 +Ref: 18073896361 +Ref: library/operator operator ge3896394 +Ref: 18083896394 +Ref: library/operator operator gt3896427 +Ref: 18093896427 +Ref: library/operator operator __lt__3896460 +Ref: 180a3896460 +Ref: library/operator operator __le__3896497 +Ref: 180b3896497 +Ref: library/operator operator __eq__3896534 +Ref: 180c3896534 +Ref: library/operator operator __ne__3896571 +Ref: 180d3896571 +Ref: library/operator operator __ge__3896608 +Ref: 180e3896608 +Ref: library/operator operator __gt__3896645 +Ref: 180f3896645 +Ref: library/operator operator not_3897385 +Ref: 18103897385 +Ref: library/operator operator __not__3897419 +Ref: 18113897419 +Ref: library/operator operator truth3897740 +Ref: 18123897740 +Ref: library/operator operator is_3897930 +Ref: 18133897930 +Ref: library/operator operator is_not3898016 +Ref: 18143898016 +Ref: library/operator operator abs3898173 +Ref: 18153898173 +Ref: library/operator operator __abs__3898206 +Ref: 18163898206 +Ref: library/operator operator add3898286 +Ref: 18173898286 +Ref: library/operator operator __add__3898320 +Ref: 18183898320 +Ref: library/operator operator and_3898410 +Ref: 18193898410 +Ref: library/operator operator __and__3898445 +Ref: 181a3898445 +Ref: library/operator operator floordiv3898529 +Ref: 181b3898529 +Ref: library/operator operator __floordiv__3898568 +Ref: 181c3898568 +Ref: library/operator operator index3898639 +Ref: 11233898639 +Ref: library/operator operator __index__3898672 +Ref: 181d3898672 +Ref: library/operator operator inv3898788 +Ref: 181e3898788 +Ref: library/operator operator invert3898821 +Ref: 181f3898821 +Ref: library/operator operator __inv__3898857 +Ref: 18203898857 +Ref: library/operator operator __invert__3898894 +Ref: 18213898894 +Ref: library/operator operator lshift3899029 +Ref: 18223899029 +Ref: library/operator operator __lshift__3899066 +Ref: 18233899066 +Ref: library/operator operator mod3899146 +Ref: 18243899146 +Ref: library/operator operator __mod__3899180 +Ref: 18253899180 +Ref: library/operator operator mul3899245 +Ref: 17f93899245 +Ref: library/operator operator __mul__3899279 +Ref: 18263899279 +Ref: library/operator operator matmul3899369 +Ref: 71d3899369 +Ref: library/operator operator __matmul__3899406 +Ref: 18273899406 +Ref: library/operator operator neg3899500 +Ref: 18283899500 +Ref: library/operator operator __neg__3899533 +Ref: 18293899533 +Ref: library/operator operator or_3899612 +Ref: 182a3899612 +Ref: library/operator operator __or__3899646 +Ref: 182b3899646 +Ref: library/operator operator pos3899728 +Ref: 182c3899728 +Ref: library/operator operator __pos__3899761 +Ref: 182d3899761 +Ref: library/operator operator pow3899841 +Ref: 182e3899841 +Ref: library/operator operator __pow__3899875 +Ref: 182f3899875 +Ref: library/operator operator rshift3899966 +Ref: 18303899966 +Ref: library/operator operator __rshift__3900003 +Ref: 18313900003 +Ref: library/operator operator sub3900084 +Ref: 18323900084 +Ref: library/operator operator __sub__3900118 +Ref: 18333900118 +Ref: library/operator operator truediv3900183 +Ref: 18343900183 +Ref: library/operator operator __truediv__3900221 +Ref: 18353900221 +Ref: library/operator operator xor3900370 +Ref: 18363900370 +Ref: library/operator operator __xor__3900404 +Ref: 18373900404 +Ref: library/operator operator concat3900577 +Ref: 18383900577 +Ref: library/operator operator __concat__3900614 +Ref: 18393900614 +Ref: library/operator operator contains3900708 +Ref: 183a3900708 +Ref: library/operator operator __contains__3900747 +Ref: 183b3900747 +Ref: library/operator operator countOf3900876 +Ref: 183c3900876 +Ref: library/operator operator delitem3900969 +Ref: 183d3900969 +Ref: library/operator operator __delitem__3901007 +Ref: 183e3901007 +Ref: library/operator operator getitem3901094 +Ref: 183f3901094 +Ref: library/operator operator __getitem__3901132 +Ref: 18403901132 +Ref: library/operator operator indexOf3901219 +Ref: 18413901219 +Ref: library/operator operator setitem3901323 +Ref: 18423901323 +Ref: library/operator operator __setitem__3901364 +Ref: 18433901364 +Ref: library/operator operator length_hint3901458 +Ref: 87d3901458 +Ref: library/operator operator attrgetter3902030 +Ref: 71b3902030 +Ref: library/operator operator itemgetter3903331 +Ref: 2613903331 +Ref: library/operator operator methodcaller3905162 +Ref: 71c3905162 +Ref: operator — Standard operators as functions-Footnote-13905961 +Node: Mapping Operators to Functions3906028 +Ref: library/operator mapping-operators-to-functions3906166 +Ref: 18443906166 +Ref: library/operator operator-map3906166 +Ref: 18453906166 +Node: In-place Operators3913590 +Ref: library/operator in-place-operators3913728 +Ref: 18463913728 +Ref: library/operator operator iadd3915036 +Ref: 18483915036 +Ref: library/operator operator __iadd__3915071 +Ref: 18493915071 +Ref: library/operator operator iand3915169 +Ref: 184a3915169 +Ref: library/operator operator __iand__3915204 +Ref: 184b3915204 +Ref: library/operator operator iconcat3915302 +Ref: 184c3915302 +Ref: library/operator operator __iconcat__3915340 +Ref: 184d3915340 +Ref: library/operator operator ifloordiv3915475 +Ref: 184e3915475 +Ref: library/operator operator __ifloordiv__3915515 +Ref: 184f3915515 +Ref: library/operator operator ilshift3915624 +Ref: 18503915624 +Ref: library/operator operator __ilshift__3915662 +Ref: 18513915662 +Ref: library/operator operator imod3915767 +Ref: 18523915767 +Ref: library/operator operator __imod__3915802 +Ref: 18533915802 +Ref: library/operator operator imul3915900 +Ref: 18543915900 +Ref: library/operator operator __imul__3915935 +Ref: 18553915935 +Ref: library/operator operator imatmul3916033 +Ref: 71e3916033 +Ref: library/operator operator __imatmul__3916071 +Ref: 18563916071 +Ref: library/operator operator ior3916201 +Ref: 18573916201 +Ref: library/operator operator __ior__3916235 +Ref: 18583916235 +Ref: library/operator operator ipow3916331 +Ref: 18593916331 +Ref: library/operator operator __ipow__3916366 +Ref: 185a3916366 +Ref: library/operator operator irshift3916465 +Ref: 185b3916465 +Ref: library/operator operator __irshift__3916503 +Ref: 185c3916503 +Ref: library/operator operator isub3916608 +Ref: 185d3916608 +Ref: library/operator operator __isub__3916643 +Ref: 185e3916643 +Ref: library/operator operator itruediv3916741 +Ref: 185f3916741 +Ref: library/operator operator __itruediv__3916780 +Ref: 18603916780 +Ref: library/operator operator ixor3916886 +Ref: 18613916886 +Ref: library/operator operator __ixor__3916921 +Ref: 18623916921 +Node: File and Directory Access3917019 +Ref: library/filesys doc3917172 +Ref: 18633917172 +Ref: library/filesys file-and-directory-access3917172 +Ref: 18643917172 +Ref: library/filesys filesys3917172 +Ref: 18653917172 +Node: pathlib — Object-oriented filesystem paths3918074 +Ref: library/pathlib doc3918230 +Ref: 18663918230 +Ref: library/pathlib module-pathlib3918230 +Ref: c83918230 +Ref: library/pathlib pathlib-object-oriented-filesystem-paths3918230 +Ref: 18673918230 +Ref: pathlib — Object-oriented filesystem paths-Footnote-13919929 +Ref: pathlib — Object-oriented filesystem paths-Footnote-23919995 +Node: Basic use3920044 +Ref: library/pathlib basic-use3920153 +Ref: 186c3920153 +Node: Pure paths3921099 +Ref: library/pathlib id13921231 +Ref: 186d3921231 +Ref: library/pathlib pure-paths3921231 +Ref: 18683921231 +Ref: library/pathlib pathlib PurePath3921446 +Ref: 186e3921446 +Ref: library/pathlib pathlib PurePosixPath3923630 +Ref: 186f3923630 +Ref: library/pathlib pathlib PureWindowsPath3923924 +Ref: 186b3923924 +Node: General properties3924517 +Ref: library/pathlib general-properties3924603 +Ref: 18713924603 +Node: Operators<2>3925455 +Ref: library/pathlib operators3925576 +Ref: 18723925576 +Node: Accessing individual parts3926786 +Ref: library/pathlib accessing-individual-parts3926911 +Ref: 18733926911 +Ref: library/pathlib pathlib PurePath parts3927074 +Ref: 18743927074 +Node: Methods and properties3927472 +Ref: library/pathlib methods-and-properties3927576 +Ref: 18753927576 +Ref: library/pathlib pathlib PurePath drive3927699 +Ref: 18763927699 +Ref: library/pathlib pathlib PurePath root3928117 +Ref: 18773928117 +Ref: library/pathlib pathlib PurePath anchor3928505 +Ref: 18783928505 +Ref: library/pathlib pathlib PurePath parents3928867 +Ref: 18793928867 +Ref: library/pathlib pathlib PurePath parent3929231 +Ref: 187a3929231 +Ref: library/pathlib pathlib PurePath name3930050 +Ref: 187c3930050 +Ref: library/pathlib pathlib PurePath suffix3930438 +Ref: 187d3930438 +Ref: library/pathlib pathlib PurePath suffixes3930731 +Ref: 187e3930731 +Ref: library/pathlib pathlib PurePath stem3931041 +Ref: 187f3931041 +Ref: library/pathlib pathlib PurePath as_posix3931335 +Ref: 18803931335 +Ref: library/pathlib pathlib PurePath as_uri3931600 +Ref: 18813931600 +Ref: library/pathlib pathlib PurePath is_absolute3931954 +Ref: 18823931954 +Ref: library/pathlib pathlib PurePath is_reserved3932546 +Ref: 18833932546 +Ref: library/pathlib pathlib PurePath joinpath3933018 +Ref: 18843933018 +Ref: library/pathlib pathlib PurePath match3933595 +Ref: 18853933595 +Ref: library/pathlib pathlib PurePath relative_to3934516 +Ref: 18863934516 +Ref: library/pathlib pathlib PurePath with_name3935178 +Ref: 18873935178 +Ref: library/pathlib pathlib PurePath with_suffix3935892 +Ref: 18883935892 +Node: Concrete paths3936562 +Ref: library/pathlib concrete-paths3936725 +Ref: 18693936725 +Ref: library/pathlib id23936725 +Ref: 18893936725 +Ref: library/pathlib pathlib Path3936997 +Ref: 2013936997 +Ref: library/pathlib pathlib PosixPath3937381 +Ref: 188a3937381 +Ref: library/pathlib pathlib WindowsPath3937691 +Ref: 186a3937691 +Node: Methods<2>3938673 +Ref: library/pathlib methods3938734 +Ref: 188b3938734 +Ref: library/pathlib pathlib Path cwd3939341 +Ref: 188c3939341 +Ref: library/pathlib pathlib Path home3939563 +Ref: 72b3939563 +Ref: library/pathlib pathlib Path stat3939842 +Ref: 188e3939842 +Ref: library/pathlib pathlib Path chmod3940190 +Ref: 18903940190 +Ref: library/pathlib pathlib Path exists3940453 +Ref: 2023940453 +Ref: library/pathlib pathlib Path expanduser3940925 +Ref: 72a3940925 +Ref: library/pathlib pathlib Path glob3941246 +Ref: 18913941246 +Ref: library/pathlib pathlib Path group3942151 +Ref: 18923942151 +Ref: library/pathlib pathlib Path is_dir3942325 +Ref: 2033942325 +Ref: library/pathlib pathlib Path is_file3942666 +Ref: 2043942666 +Ref: library/pathlib pathlib Path is_mount3943014 +Ref: 2053943014 +Ref: library/pathlib pathlib Path is_symlink3943518 +Ref: 2063943518 +Ref: library/pathlib pathlib Path is_socket3943767 +Ref: 20a3943767 +Ref: library/pathlib pathlib Path is_fifo3944115 +Ref: 2093944115 +Ref: library/pathlib pathlib Path is_block_device3944442 +Ref: 2073944442 +Ref: library/pathlib pathlib Path is_char_device3944798 +Ref: 2083944798 +Ref: library/pathlib pathlib Path iterdir3945161 +Ref: 18933945161 +Ref: library/pathlib pathlib Path lchmod3945914 +Ref: 18943945914 +Ref: library/pathlib pathlib Path lstat3946103 +Ref: 18953946103 +Ref: library/pathlib pathlib Path mkdir3946289 +Ref: 7293946289 +Ref: library/pathlib pathlib Path open3947356 +Ref: 18963947356 +Ref: library/pathlib pathlib Path owner3947709 +Ref: 18973947709 +Ref: library/pathlib pathlib Path read_bytes3947882 +Ref: 72f3947882 +Ref: library/pathlib pathlib Path read_text3948191 +Ref: 72d3948191 +Ref: library/pathlib pathlib Path rename3948624 +Ref: 18983948624 +Ref: library/pathlib pathlib Path replace3949447 +Ref: 18993949447 +Ref: library/pathlib pathlib Path resolve3949956 +Ref: 187b3949956 +Ref: library/pathlib pathlib Path rglob3950888 +Ref: 189a3950888 +Ref: library/pathlib pathlib Path rmdir3951287 +Ref: 189b3951287 +Ref: library/pathlib pathlib Path samefile3951373 +Ref: 7283951373 +Ref: library/pathlib pathlib Path symlink_to3951924 +Ref: 189c3951924 +Ref: library/pathlib pathlib Path touch3952579 +Ref: 189d3952579 +Ref: library/pathlib pathlib Path unlink3952997 +Ref: 189e3952997 +Ref: library/pathlib pathlib Path link_to3953492 +Ref: 20b3953492 +Ref: library/pathlib pathlib Path write_bytes3953613 +Ref: 72e3953613 +Ref: library/pathlib pathlib Path write_text3953994 +Ref: 72c3953994 +Node: Correspondence to tools in the os module3954468 +Ref: library/pathlib correspondence-to-tools-in-the-os-module3954612 +Ref: 189f3954612 +Node: os path — Common pathname manipulations3958916 +Ref: library/os path doc3959141 +Ref: 18a53959141 +Ref: library/os path module-os path3959141 +Ref: c53959141 +Ref: library/os path os-path-common-pathname-manipulations3959141 +Ref: 18a63959141 +Ref: library/os path os path abspath3961537 +Ref: cb03961537 +Ref: library/os path os path basename3961863 +Ref: 18a23961863 +Ref: library/os path os path commonpath3962374 +Ref: 7253962374 +Ref: library/os path os path commonprefix3962892 +Ref: 7263962892 +Ref: library/os path os path dirname3963540 +Ref: 18a33963540 +Ref: library/os path os path exists3963812 +Ref: 1f63963812 +Ref: library/os path os path lexists3964400 +Ref: 1f73964400 +Ref: library/os path os path expanduser3964706 +Ref: 1fe3964706 +Ref: library/os path os path expandvars3965729 +Ref: d443965729 +Ref: library/os path os path getatime3966221 +Ref: 18a83966221 +Ref: library/os path os path getmtime3966522 +Ref: 18a93966522 +Ref: library/os path os path getctime3966898 +Ref: 18aa3966898 +Ref: library/os path os path getsize3967389 +Ref: 18ab3967389 +Ref: library/os path os path isabs3967619 +Ref: 18a13967619 +Ref: library/os path os path isfile3967928 +Ref: 1f93967928 +Ref: library/os path os path isdir3968233 +Ref: 1f83968233 +Ref: library/os path os path islink3968534 +Ref: 1f43968534 +Ref: library/os path os path ismount3968838 +Ref: 1fa3968838 +Ref: library/os path os path join3969726 +Ref: 18703969726 +Ref: library/os path os path normcase3970782 +Ref: 18ac3970782 +Ref: library/os path os path normpath3971121 +Ref: caf3971121 +Ref: library/os path os path realpath3971626 +Ref: 1ff3971626 +Ref: library/os path os path relpath3972179 +Ref: 18a03972179 +Ref: library/os path os path samefile3972647 +Ref: 8823972647 +Ref: library/os path os path sameopenfile3973208 +Ref: 18ae3973208 +Ref: library/os path os path samestat3973517 +Ref: 8813973517 +Ref: library/os path os path split3974069 +Ref: 18a73974069 +Ref: library/os path os path splitdrive3974858 +Ref: 47d3974858 +Ref: library/os path os path splitext3975755 +Ref: 18a43975755 +Ref: library/os path os path supports_unicode_filenames3976153 +Ref: ded3976153 +Ref: os path — Common pathname manipulations-Footnote-13976361 +Ref: os path — Common pathname manipulations-Footnote-23976429 +Node: fileinput — Iterate over lines from multiple input streams3976494 +Ref: library/fileinput doc3976709 +Ref: 18af3976709 +Ref: library/fileinput fileinput-iterate-over-lines-from-multiple-input-streams3976709 +Ref: 18b03976709 +Ref: library/fileinput module-fileinput3976709 +Ref: 803976709 +Ref: library/fileinput fileinput input3978873 +Ref: 2ad3978873 +Ref: library/fileinput fileinput filename3979975 +Ref: 18b13979975 +Ref: library/fileinput fileinput fileno3980131 +Ref: 18b23980131 +Ref: library/fileinput fileinput lineno3980330 +Ref: 18b33980330 +Ref: library/fileinput fileinput filelineno3980603 +Ref: 18b43980603 +Ref: library/fileinput fileinput isfirstline3980869 +Ref: 18b53980869 +Ref: library/fileinput fileinput isstdin3981021 +Ref: 18b63981021 +Ref: library/fileinput fileinput nextfile3981164 +Ref: 18b73981164 +Ref: library/fileinput fileinput close3981704 +Ref: 18b83981704 +Ref: library/fileinput fileinput FileInput3981875 +Ref: 27f3981875 +Ref: library/fileinput fileinput hook_compressed3984489 +Ref: 18ba3984489 +Ref: library/fileinput fileinput hook_encoded3984986 +Ref: 54a3984986 +Ref: fileinput — Iterate over lines from multiple input streams-Footnote-13985406 +Node: stat — Interpreting stat results3985474 +Ref: library/stat doc3985690 +Ref: 18bb3985690 +Ref: library/stat module-stat3985690 +Ref: f43985690 +Ref: library/stat stat-interpreting-stat-results3985690 +Ref: 18bc3985690 +Ref: library/stat stat S_ISDIR3986368 +Ref: 18bd3986368 +Ref: library/stat stat S_ISCHR3986458 +Ref: 18be3986458 +Ref: library/stat stat S_ISBLK3986573 +Ref: 18bf3986573 +Ref: library/stat stat S_ISREG3986679 +Ref: 18c03986679 +Ref: library/stat stat S_ISFIFO3986772 +Ref: 18c13986772 +Ref: library/stat stat S_ISLNK3986871 +Ref: 18c23986871 +Ref: library/stat stat S_ISSOCK3986965 +Ref: 18c33986965 +Ref: library/stat stat S_ISDOOR3987053 +Ref: 18c43987053 +Ref: library/stat stat S_ISPORT3987165 +Ref: 18c53987165 +Ref: library/stat stat S_ISWHT3987284 +Ref: 18c63987284 +Ref: library/stat stat S_IMODE3987489 +Ref: 18c73987489 +Ref: library/stat stat S_IFMT3987756 +Ref: 18c83987756 +Ref: library/stat stat filemode3989286 +Ref: a9d3989286 +Ref: library/stat stat ST_MODE3989697 +Ref: 8d33989697 +Ref: library/stat stat ST_INO3989750 +Ref: 18c93989750 +Ref: library/stat stat ST_DEV3989793 +Ref: 18ca3989793 +Ref: library/stat stat ST_NLINK3989847 +Ref: 18cb3989847 +Ref: library/stat stat ST_UID3989908 +Ref: 18cc3989908 +Ref: library/stat stat ST_GID3989959 +Ref: 18cd3989959 +Ref: library/stat stat ST_SIZE3990011 +Ref: 18ce3990011 +Ref: library/stat stat ST_ATIME3990123 +Ref: 18cf3990123 +Ref: library/stat stat ST_MTIME3990175 +Ref: 18d03990175 +Ref: library/stat stat ST_CTIME3990233 +Ref: 18d13990233 +Ref: library/stat stat S_IFSOCK3991275 +Ref: 18d23991275 +Ref: library/stat stat S_IFLNK3991314 +Ref: 18d33991314 +Ref: library/stat stat S_IFREG3991359 +Ref: 18d43991359 +Ref: library/stat stat S_IFBLK3991403 +Ref: 18d53991403 +Ref: library/stat stat S_IFDIR3991447 +Ref: 18d63991447 +Ref: library/stat stat S_IFCHR3991488 +Ref: 18d73991488 +Ref: library/stat stat S_IFIFO3991536 +Ref: 18d83991536 +Ref: library/stat stat S_IFDOOR3991572 +Ref: 8d43991572 +Ref: library/stat stat S_IFPORT3991635 +Ref: 8d53991635 +Ref: library/stat stat S_IFWHT3991704 +Ref: 8d63991704 +Ref: library/stat stat S_ISUID3992022 +Ref: 18d93992022 +Ref: library/stat stat S_ISGID3992065 +Ref: 18da3992065 +Ref: library/stat stat S_ISVTX3992642 +Ref: 18dd3992642 +Ref: library/stat stat S_IRWXU3992891 +Ref: 18de3992891 +Ref: library/stat stat S_IRUSR3992954 +Ref: 18df3992954 +Ref: library/stat stat S_IWUSR3993011 +Ref: 18e03993011 +Ref: library/stat stat S_IXUSR3993069 +Ref: 18e13993069 +Ref: library/stat stat S_IRWXG3993129 +Ref: 18e23993129 +Ref: library/stat stat S_IRGRP3993187 +Ref: 18e33993187 +Ref: library/stat stat S_IWGRP3993244 +Ref: 18e43993244 +Ref: library/stat stat S_IXGRP3993302 +Ref: 18db3993302 +Ref: library/stat stat S_IRWXO3993362 +Ref: 18e53993362 +Ref: library/stat stat S_IROTH3993440 +Ref: 18e63993440 +Ref: library/stat stat S_IWOTH3993499 +Ref: 18e73993499 +Ref: library/stat stat S_IXOTH3993559 +Ref: 18e83993559 +Ref: library/stat stat S_ENFMT3993621 +Ref: 18dc3993621 +Ref: library/stat stat S_IREAD3993854 +Ref: 18e93993854 +Ref: library/stat stat S_IWRITE3993925 +Ref: 18ea3993925 +Ref: library/stat stat S_IEXEC3993997 +Ref: 18eb3993997 +Ref: library/stat stat UF_NODUMP3994154 +Ref: 18ec3994154 +Ref: library/stat stat UF_IMMUTABLE3994208 +Ref: 18ed3994208 +Ref: library/stat stat UF_APPEND3994272 +Ref: 18ee3994272 +Ref: library/stat stat UF_OPAQUE3994338 +Ref: 18ef3994338 +Ref: library/stat stat UF_NOUNLINK3994429 +Ref: 18f03994429 +Ref: library/stat stat UF_COMPRESSED3994503 +Ref: 18f13994503 +Ref: library/stat stat UF_HIDDEN3994587 +Ref: 18f23994587 +Ref: library/stat stat SF_ARCHIVED3994679 +Ref: 18f33994679 +Ref: library/stat stat SF_IMMUTABLE3994739 +Ref: 18f43994739 +Ref: library/stat stat SF_APPEND3994803 +Ref: 18f53994803 +Ref: library/stat stat SF_NOUNLINK3994869 +Ref: 18f63994869 +Ref: library/stat stat SF_SNAPSHOT3994943 +Ref: 18f73994943 +Ref: library/stat stat FILE_ATTRIBUTE_ARCHIVE3995340 +Ref: 18f83995340 +Ref: library/stat stat FILE_ATTRIBUTE_COMPRESSED3995378 +Ref: 18f93995378 +Ref: library/stat stat FILE_ATTRIBUTE_DEVICE3995419 +Ref: 18fa3995419 +Ref: library/stat stat FILE_ATTRIBUTE_DIRECTORY3995456 +Ref: 18fb3995456 +Ref: library/stat stat FILE_ATTRIBUTE_ENCRYPTED3995496 +Ref: 18fc3995496 +Ref: library/stat stat FILE_ATTRIBUTE_HIDDEN3995536 +Ref: 18fd3995536 +Ref: library/stat stat FILE_ATTRIBUTE_INTEGRITY_STREAM3995573 +Ref: 18fe3995573 +Ref: library/stat stat FILE_ATTRIBUTE_NORMAL3995620 +Ref: 18ff3995620 +Ref: library/stat stat FILE_ATTRIBUTE_NOT_CONTENT_INDEXED3995657 +Ref: 19003995657 +Ref: library/stat stat FILE_ATTRIBUTE_NO_SCRUB_DATA3995707 +Ref: 19013995707 +Ref: library/stat stat FILE_ATTRIBUTE_OFFLINE3995751 +Ref: 19023995751 +Ref: library/stat stat FILE_ATTRIBUTE_READONLY3995789 +Ref: 19033995789 +Ref: library/stat stat FILE_ATTRIBUTE_REPARSE_POINT3995828 +Ref: 19043995828 +Ref: library/stat stat FILE_ATTRIBUTE_SPARSE_FILE3995872 +Ref: 19053995872 +Ref: library/stat stat FILE_ATTRIBUTE_SYSTEM3995914 +Ref: 19063995914 +Ref: library/stat stat FILE_ATTRIBUTE_TEMPORARY3995951 +Ref: 19073995951 +Ref: library/stat stat FILE_ATTRIBUTE_VIRTUAL3995991 +Ref: 19083995991 +Ref: library/stat stat IO_REPARSE_TAG_SYMLINK3996261 +Ref: 19093996261 +Ref: library/stat stat IO_REPARSE_TAG_MOUNT_POINT3996299 +Ref: 190a3996299 +Ref: library/stat stat IO_REPARSE_TAG_APPEXECLINK3996341 +Ref: 190b3996341 +Ref: stat — Interpreting stat results-Footnote-13996446 +Ref: stat — Interpreting stat results-Footnote-23996509 +Node: filecmp — File and Directory Comparisons3996589 +Ref: library/filecmp doc3996798 +Ref: 190c3996798 +Ref: library/filecmp filecmp-file-and-directory-comparisons3996798 +Ref: 190d3996798 +Ref: library/filecmp module-filecmp3996798 +Ref: 7f3996798 +Ref: library/filecmp filecmp cmp3997267 +Ref: 190e3997267 +Ref: library/filecmp filecmp cmpfiles3997954 +Ref: 190f3997954 +Ref: library/filecmp filecmp clear_cache3998899 +Ref: 8473998899 +Ref: filecmp — File and Directory Comparisons-Footnote-13999214 +Node: The dircmp class3999280 +Ref: library/filecmp dircmp-objects3999375 +Ref: 19103999375 +Ref: library/filecmp the-dircmp-class3999375 +Ref: 19113999375 +Ref: library/filecmp filecmp dircmp3999436 +Ref: 8493999436 +Ref: library/filecmp filecmp dircmp report3999971 +Ref: 19123999971 +Ref: library/filecmp filecmp dircmp report_partial_closure4000088 +Ref: 19134000088 +Ref: library/filecmp filecmp dircmp report_full_closure4000229 +Ref: 19144000229 +Ref: library/filecmp filecmp dircmp left4000748 +Ref: 19154000748 +Ref: library/filecmp filecmp dircmp right4000804 +Ref: 19164000804 +Ref: library/filecmp filecmp dircmp left_list4000861 +Ref: 19174000861 +Ref: library/filecmp filecmp dircmp right_list4000979 +Ref: 19184000979 +Ref: library/filecmp filecmp dircmp common4001098 +Ref: 19194001098 +Ref: library/filecmp filecmp dircmp left_only4001183 +Ref: 191a4001183 +Ref: library/filecmp filecmp dircmp right_only4001263 +Ref: 191b4001263 +Ref: library/filecmp filecmp dircmp common_dirs4001344 +Ref: 191c4001344 +Ref: library/filecmp filecmp dircmp common_files4001424 +Ref: 191d4001424 +Ref: library/filecmp filecmp dircmp common_funny4001496 +Ref: 191e4001496 +Ref: library/filecmp filecmp dircmp same_files4001699 +Ref: 191f4001699 +Ref: library/filecmp filecmp dircmp diff_files4001845 +Ref: 19204001845 +Ref: library/filecmp filecmp dircmp funny_files4002010 +Ref: 19214002010 +Ref: library/filecmp filecmp dircmp subdirs4002128 +Ref: 19224002128 +Ref: library/filecmp filecmp DEFAULT_IGNORES4002264 +Ref: 8484002264 +Node: tempfile — Generate temporary files and directories4002948 +Ref: library/tempfile doc4003169 +Ref: 19234003169 +Ref: library/tempfile module-tempfile4003169 +Ref: 1034003169 +Ref: library/tempfile tempfile-generate-temporary-files-and-directories4003169 +Ref: 19244003169 +Ref: library/tempfile tempfile TemporaryFile4004324 +Ref: 19254004324 +Ref: library/tempfile tempfile NamedTemporaryFile4006377 +Ref: d494006377 +Ref: library/tempfile tempfile SpooledTemporaryFile4007546 +Ref: aa64007546 +Ref: library/tempfile tempfile TemporaryDirectory4008777 +Ref: bc84008777 +Ref: library/tempfile tempfile mkstemp4009761 +Ref: 19264009761 +Ref: library/tempfile tempfile mkdtemp4012643 +Ref: 19274012643 +Ref: library/tempfile tempfile gettempdir4013757 +Ref: 192d4013757 +Ref: library/tempfile tempfile gettempdirb4014766 +Ref: 192f4014766 +Ref: library/tempfile tempfile gettempprefix4014905 +Ref: 192b4014905 +Ref: library/tempfile tempfile gettempprefixb4015064 +Ref: 192c4015064 +Ref: library/tempfile tempfile tempdir4015564 +Ref: 192e4015564 +Ref: tempfile — Generate temporary files and directories-Footnote-14016090 +Node: Examples<3>4016157 +Ref: library/tempfile examples4016301 +Ref: 19304016301 +Ref: library/tempfile tempfile-examples4016301 +Ref: 19294016301 +Node: Deprecated functions and variables4017219 +Ref: library/tempfile deprecated-functions-and-variables4017363 +Ref: 19314017363 +Ref: library/tempfile tempfile mktemp4017993 +Ref: 19324017993 +Node: glob — Unix style pathname pattern expansion4019126 +Ref: library/glob doc4019347 +Ref: 19334019347 +Ref: library/glob glob-unix-style-pathname-pattern-expansion4019347 +Ref: 19344019347 +Ref: library/glob module-glob4019347 +Ref: 8a4019347 +Ref: library/glob glob glob4020460 +Ref: 5c64020460 +Ref: library/glob glob iglob4021729 +Ref: 5c74021729 +Ref: library/glob glob escape4022041 +Ref: 8504022041 +Ref: glob — Unix style pathname pattern expansion-Footnote-14023495 +Node: fnmatch — Unix filename pattern matching4023558 +Ref: library/fnmatch doc4023767 +Ref: 19384023767 +Ref: library/fnmatch fnmatch-unix-filename-pattern-matching4023767 +Ref: 19394023767 +Ref: library/fnmatch module-fnmatch4023767 +Ref: 814023767 +Ref: library/fnmatch fnmatch fnmatch4025102 +Ref: 19354025102 +Ref: library/fnmatch fnmatch fnmatchcase4025789 +Ref: 193b4025789 +Ref: library/fnmatch fnmatch filter4026036 +Ref: 193a4026036 +Ref: library/fnmatch fnmatch translate4026284 +Ref: 193c4026284 +Ref: fnmatch — Unix filename pattern matching-Footnote-14026864 +Node: linecache — Random access to text lines4026930 +Ref: library/linecache doc4027130 +Ref: 193d4027130 +Ref: library/linecache linecache-random-access-to-text-lines4027130 +Ref: 193e4027130 +Ref: library/linecache module-linecache4027130 +Ref: a84027130 +Ref: library/linecache linecache getline4027960 +Ref: 7094027960 +Ref: library/linecache linecache clearcache4028729 +Ref: 19414028729 +Ref: library/linecache linecache checkcache4028898 +Ref: 19424028898 +Ref: library/linecache linecache lazycache4029176 +Ref: 7084029176 +Ref: linecache — Random access to text lines-Footnote-14029723 +Ref: linecache — Random access to text lines-Footnote-24029791 +Node: shutil — High-level file operations4029840 +Ref: library/shutil doc4029989 +Ref: 19434029989 +Ref: library/shutil module-shutil4029989 +Ref: e94029989 +Ref: library/shutil shutil-high-level-file-operations4029989 +Ref: 19444029989 +Ref: shutil — High-level file operations-Footnote-14031092 +Node: Directory and files operations4031157 +Ref: library/shutil directory-and-files-operations4031290 +Ref: 19454031290 +Ref: library/shutil file-operations4031290 +Ref: 19464031290 +Ref: library/shutil shutil copyfileobj4031373 +Ref: 25d4031373 +Ref: library/shutil shutil copyfile4031956 +Ref: 2584031956 +Ref: library/shutil shutil SameFileError4033556 +Ref: 8b34033556 +Ref: library/shutil shutil copymode4033726 +Ref: 19474033726 +Ref: library/shutil shutil copystat4034620 +Ref: a774034620 +Ref: library/shutil shutil copy4036726 +Ref: 2594036726 +Ref: library/shutil shutil copy24038157 +Ref: 25a4038157 +Ref: library/shutil shutil ignore_patterns4039670 +Ref: 19494039670 +Ref: library/shutil shutil copytree4039972 +Ref: 21a4039972 +Ref: library/shutil shutil rmtree4043122 +Ref: 21c4043122 +Ref: library/shutil shutil rmtree avoids_symlink_attacks4045046 +Ref: 194a4045046 +Ref: library/shutil shutil move4045386 +Ref: 25b4045386 +Ref: library/shutil shutil disk_usage4047224 +Ref: a754047224 +Ref: library/shutil shutil chown4047671 +Ref: a764047671 +Ref: library/shutil shutil which4048156 +Ref: 194b4048156 +Ref: library/shutil shutil Error4049483 +Ref: 8b24049483 +Node: Platform-dependent efficient copy operations4049819 +Ref: library/shutil platform-dependent-efficient-copy-operations4049955 +Ref: 194d4049955 +Ref: library/shutil shutil-platform-dependent-efficient-copy-operations4049955 +Ref: 25e4049955 +Ref: Platform-dependent efficient copy operations-Footnote-14051079 +Ref: Platform-dependent efficient copy operations-Footnote-24051122 +Node: copytree example4051170 +Ref: library/shutil copytree-example4051329 +Ref: 194e4051329 +Ref: library/shutil shutil-copytree-example4051329 +Ref: 194f4051329 +Node: rmtree example4053407 +Ref: library/shutil rmtree-example4053513 +Ref: 19504053513 +Ref: library/shutil shutil-rmtree-example4053513 +Ref: 19514053513 +Node: Archiving operations4054060 +Ref: library/shutil archiving-operations4054242 +Ref: b964054242 +Ref: library/shutil id14054242 +Ref: 19524054242 +Ref: library/shutil shutil make_archive4054550 +Ref: 21b4054550 +Ref: library/shutil shutil get_archive_formats4056523 +Ref: 19544056523 +Ref: library/shutil shutil register_archive_format4057365 +Ref: 19554057365 +Ref: library/shutil shutil unregister_archive_format4058211 +Ref: 19564058211 +Ref: library/shutil shutil unpack_archive4058346 +Ref: b974058346 +Ref: library/shutil shutil register_unpack_format4059306 +Ref: 19574059306 +Ref: library/shutil shutil unregister_unpack_format4060069 +Ref: 19594060069 +Ref: library/shutil shutil get_unpack_formats4060193 +Ref: 19584060193 +Ref: Archiving operations-Footnote-14061146 +Node: Archiving example4061195 +Ref: library/shutil archiving-example4061309 +Ref: 195a4061309 +Ref: library/shutil shutil-archiving-example4061309 +Ref: 195b4061309 +Node: Archiving example with base_dir4062402 +Ref: library/shutil archiving-example-with-base-dir4062516 +Ref: 195c4062516 +Ref: library/shutil shutil-archiving-example-with-basedir4062516 +Ref: 19534062516 +Node: Querying the size of the output terminal4063620 +Ref: library/shutil querying-the-size-of-the-output-terminal4063763 +Ref: 195d4063763 +Ref: library/shutil shutil get_terminal_size4063866 +Ref: a4a4063866 +Ref: Querying the size of the output terminal-Footnote-14065393 +Node: Data Persistence4065475 +Ref: library/persistence doc4065628 +Ref: 19604065628 +Ref: library/persistence data-persistence4065628 +Ref: 19614065628 +Ref: library/persistence other-environment-variables4065628 +Ref: 19624065628 +Ref: library/persistence persistence4065628 +Ref: 19634065628 +Node: pickle — Python object serialization4066459 +Ref: library/pickle doc4066604 +Ref: 19644066604 +Ref: library/pickle module-pickle4066604 +Ref: ca4066604 +Ref: library/pickle pickle-python-object-serialization4066604 +Ref: 19654066604 +Ref: pickle — Python object serialization-Footnote-14068439 +Ref: pickle — Python object serialization-Footnote-24068504 +Node: Relationship to other Python modules4068572 +Ref: library/pickle relationship-to-other-python-modules4068710 +Ref: 19684068710 +Node: Comparison with marshal4068866 +Ref: library/pickle comparison-with-marshal4068991 +Ref: 19694068991 +Node: Comparison with json4071219 +Ref: library/pickle comparison-with-json4071344 +Ref: 19674071344 +Ref: library/pickle id24071344 +Ref: 196a4071344 +Ref: Comparison with json-Footnote-14072509 +Node: Data stream format4072533 +Ref: library/pickle data-stream-format4072696 +Ref: 196c4072696 +Ref: library/pickle pickle-protocols4072696 +Ref: 56e4072696 +Ref: Data stream format-Footnote-14075666 +Ref: Data stream format-Footnote-24075715 +Ref: Data stream format-Footnote-34075764 +Node: Module Interface4075813 +Ref: library/pickle module-interface4075974 +Ref: 196f4075974 +Ref: library/pickle pickle HIGHEST_PROTOCOL4076426 +Ref: e004076426 +Ref: library/pickle pickle DEFAULT_PROTOCOL4076698 +Ref: 19724076698 +Ref: library/pickle pickle dump4077206 +Ref: 19714077206 +Ref: library/pickle pickle dumps4077704 +Ref: dff4077704 +Ref: library/pickle pickle load4078141 +Ref: 5c24078141 +Ref: library/pickle pickle loads4078889 +Ref: 5c34078889 +Ref: library/pickle pickle PickleError4079622 +Ref: 19734079622 +Ref: library/pickle pickle PicklingError4079756 +Ref: 19744079756 +Ref: library/pickle pickle UnpicklingError4080035 +Ref: 19764080035 +Ref: library/pickle pickle Pickler4080544 +Ref: 20d4080544 +Ref: library/pickle pickle Pickler dump4082098 +Ref: 19794082098 +Ref: library/pickle pickle Pickler persistent_id4082240 +Ref: 197a4082240 +Ref: library/pickle pickle Pickler dispatch_table4082916 +Ref: a694082916 +Ref: library/pickle pickle Pickler reducer_override4084080 +Ref: 20e4084080 +Ref: library/pickle pickle Pickler fast4084695 +Ref: 19804084695 +Ref: library/pickle pickle Unpickler4085180 +Ref: 19704085180 +Ref: library/pickle pickle Unpickler load4087284 +Ref: 19824087284 +Ref: library/pickle pickle Unpickler persistent_load4087579 +Ref: 197b4087579 +Ref: library/pickle pickle Unpickler find_class4088026 +Ref: 19834088026 +Ref: library/pickle pickle PickleBuffer4088705 +Ref: 19774088705 +Ref: library/pickle pickle PickleBuffer raw4089323 +Ref: 19854089323 +Ref: library/pickle pickle PickleBuffer release4089666 +Ref: 19864089666 +Node: What can be pickled and unpickled?4089782 +Ref: library/pickle pickle-picklable4089949 +Ref: 19754089949 +Ref: library/pickle what-can-be-pickled-and-unpickled4089949 +Ref: 19874089949 +Ref: What can be pickled and unpickled?-Footnote-14092677 +Ref: What can be pickled and unpickled?-Footnote-24092813 +Node: Pickling Class Instances4092953 +Ref: library/pickle pickle-inst4093158 +Ref: 196b4093158 +Ref: library/pickle pickling-class-instances4093158 +Ref: 19884093158 +Ref: library/pickle object __getnewargs_ex__4094090 +Ref: 19894094090 +Ref: library/pickle object __getnewargs__4094922 +Ref: 6ae4094922 +Ref: library/pickle object __getstate__4095488 +Ref: e014095488 +Ref: library/pickle object __setstate__4095927 +Ref: 19c4095927 +Ref: library/pickle object __reduce__4097766 +Ref: 19b4097766 +Ref: library/pickle object __reduce_ex__4100631 +Ref: 16ac4100631 +Ref: Pickling Class Instances-Footnote-14101292 +Node: Persistence of External Objects4101391 +Ref: library/pickle persistence-of-external-objects4101507 +Ref: 198b4101507 +Ref: library/pickle pickle-persistent4101507 +Ref: 197c4101507 +Ref: Persistence of External Objects-Footnote-14106175 +Node: Dispatch Tables4106439 +Ref: library/pickle dispatch-tables4106589 +Ref: 198c4106589 +Ref: library/pickle pickle-dispatch4106589 +Ref: 197e4106589 +Node: Handling Stateful Objects4107816 +Ref: library/pickle handling-stateful-objects4107926 +Ref: 198d4107926 +Ref: library/pickle pickle-state4107926 +Ref: 198a4107926 +Node: Custom Reduction for Types Functions and Other Objects4110213 +Ref: library/pickle custom-reduction-for-types-functions-and-other-objects4110403 +Ref: 198e4110403 +Ref: library/pickle reducer-override4110403 +Ref: 197f4110403 +Node: Out-of-band Buffers4112495 +Ref: library/pickle out-of-band-buffers4112680 +Ref: 198f4112680 +Ref: library/pickle pickle-oob4112680 +Ref: 19784112680 +Node: Provider API4113528 +Ref: library/pickle provider-api4113617 +Ref: 19904113617 +Node: Consumer API4114224 +Ref: library/pickle consumer-api4114332 +Ref: 19914114332 +Node: Example<4>4115634 +Ref: library/pickle example4115721 +Ref: 19924115721 +Ref: Example<4>-Footnote-14118105 +Node: Restricting Globals4118154 +Ref: library/pickle pickle-restrict4118299 +Ref: 19844118299 +Ref: library/pickle restricting-globals4118299 +Ref: 19934118299 +Node: Performance<2>4121075 +Ref: library/pickle performance4121212 +Ref: 19944121212 +Node: Examples<4>4121488 +Ref: library/pickle examples4121597 +Ref: 19954121597 +Ref: library/pickle pickle-example4121597 +Ref: 19964121597 +Node: copyreg — Register pickle support functions4122815 +Ref: library/copyreg doc4123005 +Ref: 19974123005 +Ref: library/copyreg copyreg-register-pickle-support-functions4123005 +Ref: 19984123005 +Ref: library/copyreg module-copyreg4123005 +Ref: 274123005 +Ref: library/copyreg copyreg constructor4123597 +Ref: 19994123597 +Ref: library/copyreg copyreg pickle4123800 +Ref: 197d4123800 +Ref: copyreg — Register pickle support functions-Footnote-14124753 +Node: Example<5>4124819 +Ref: library/copyreg example4124911 +Ref: 199a4124911 +Node: shelve — Python object persistence4125463 +Ref: library/shelve doc4125663 +Ref: 199b4125663 +Ref: library/shelve module-shelve4125663 +Ref: e74125663 +Ref: library/shelve shelve-python-object-persistence4125663 +Ref: 199c4125663 +Ref: library/shelve shelve open4126248 +Ref: 199d4126248 +Ref: library/shelve shelve Shelf sync4128464 +Ref: 199f4128464 +Ref: library/shelve shelve Shelf close4128785 +Ref: 19a04128785 +Ref: shelve — Python object persistence-Footnote-14129167 +Ref: shelve — Python object persistence-Footnote-24129232 +Node: Restrictions4129285 +Ref: library/shelve restrictions4129389 +Ref: 19a14129389 +Ref: library/shelve shelve Shelf4130414 +Ref: 8b04130414 +Ref: library/shelve shelve BsdDbShelf4131650 +Ref: 19a24131650 +Ref: library/shelve shelve DbfilenameShelf4132356 +Ref: 19a34132356 +Ref: Restrictions-Footnote-14132971 +Node: Example<6>4133024 +Ref: library/shelve example4133128 +Ref: 19a44133128 +Ref: library/shelve shelve-example4133128 +Ref: 199e4133128 +Node: marshal — Internal Python object serialization4134807 +Ref: library/marshal doc4135004 +Ref: 19a54135004 +Ref: library/marshal marshal-internal-python-object-serialization4135004 +Ref: 19a64135004 +Ref: library/marshal module-marshal4135004 +Ref: b04135004 +Ref: library/marshal marshal dump4137411 +Ref: 19a74137411 +Ref: library/marshal marshal load4137959 +Ref: 19a84137959 +Ref: library/marshal marshal dumps4138511 +Ref: 79c4138511 +Ref: library/marshal marshal loads4138911 +Ref: 19a94138911 +Ref: library/marshal marshal version4139212 +Ref: 19aa4139212 +Ref: marshal — Internal Python object serialization-Footnote-14139566 +Node: dbm — Interfaces to Unix “databases”4139950 +Ref: library/dbm doc4140164 +Ref: 19ab4140164 +Ref: library/dbm dbm-interfaces-to-unix-databases4140164 +Ref: 19ac4140164 +Ref: library/dbm module-dbm4140164 +Ref: 324140164 +Ref: library/dbm dbm error4140674 +Ref: 19ad4140674 +Ref: library/dbm dbm whichdb4140946 +Ref: 19ae4140946 +Ref: library/dbm dbm open4141509 +Ref: 8304141509 +Ref: dbm — Interfaces to Unix “databases”-Footnote-14145152 +Ref: dbm — Interfaces to Unix “databases”-Footnote-24145223 +Node: dbm gnu — GNU’s reinterpretation of dbm4145276 +Ref: library/dbm dbm-gnu-gnu-s-reinterpretation-of-dbm4145443 +Ref: 19af4145443 +Ref: library/dbm module-dbm gnu4145443 +Ref: 344145443 +Ref: library/dbm dbm gnu error4146259 +Ref: 2c54146259 +Ref: library/dbm dbm gnu open4146463 +Ref: 19b04146463 +Ref: library/dbm dbm gnu gdbm firstkey4148541 +Ref: 19b14148541 +Ref: library/dbm dbm gnu gdbm nextkey4148885 +Ref: 19b24148885 +Ref: library/dbm dbm gnu gdbm reorganize4149266 +Ref: 19b34149266 +Ref: library/dbm dbm gnu gdbm sync4149709 +Ref: 19b44149709 +Ref: library/dbm dbm gnu gdbm close4149874 +Ref: 19b54149874 +Ref: dbm gnu — GNU’s reinterpretation of dbm-Footnote-14149984 +Node: dbm ndbm — Interface based on ndbm4150050 +Ref: library/dbm dbm-ndbm-interface-based-on-ndbm4150266 +Ref: 19b64150266 +Ref: library/dbm module-dbm ndbm4150266 +Ref: 354150266 +Ref: library/dbm dbm ndbm error4151022 +Ref: 2c64151022 +Ref: library/dbm dbm ndbm library4151228 +Ref: 19b74151228 +Ref: library/dbm dbm ndbm open4151314 +Ref: 19b84151314 +Ref: library/dbm dbm ndbm ndbm close4152599 +Ref: 19b94152599 +Ref: dbm ndbm — Interface based on ndbm-Footnote-14152709 +Node: dbm dumb — Portable DBM implementation4152776 +Ref: library/dbm dbm-dumb-portable-dbm-implementation4152940 +Ref: 19ba4152940 +Ref: library/dbm module-dbm dumb4152940 +Ref: 334152940 +Ref: library/dbm dbm dumb error4153765 +Ref: 2c44153765 +Ref: library/dbm dbm dumb open4153971 +Ref: 2bf4153971 +Ref: library/dbm dbm dumb dumbdbm sync4155916 +Ref: 19bb4155916 +Ref: library/dbm dbm dumb dumbdbm close4156079 +Ref: 19bc4156079 +Ref: dbm dumb — Portable DBM implementation-Footnote-14156195 +Node: sqlite3 — DB-API 2 0 interface for SQLite databases4156262 +Ref: library/sqlite3 doc4156419 +Ref: 19bd4156419 +Ref: library/sqlite3 module-sqlite34156419 +Ref: f24156419 +Ref: library/sqlite3 sqlite3-db-api-2-0-interface-for-sqlite-databases4156419 +Ref: 19be4156419 +Ref: sqlite3 — DB-API 2 0 interface for SQLite databases-Footnote-14160767 +Ref: sqlite3 — DB-API 2 0 interface for SQLite databases-Footnote-24160831 +Ref: sqlite3 — DB-API 2 0 interface for SQLite databases-Footnote-34160880 +Node: Module functions and constants4160929 +Ref: library/sqlite3 module-functions-and-constants4161076 +Ref: 19c34161076 +Ref: library/sqlite3 sqlite3-module-contents4161076 +Ref: 19c44161076 +Ref: library/sqlite3 sqlite3 version4161157 +Ref: 19c54161157 +Ref: library/sqlite3 sqlite3 version_info4161291 +Ref: 19c64161291 +Ref: library/sqlite3 sqlite3 sqlite_version4161441 +Ref: 19c74161441 +Ref: library/sqlite3 sqlite3 sqlite_version_info4161545 +Ref: 19c84161545 +Ref: library/sqlite3 sqlite3 PARSE_DECLTYPES4161670 +Ref: 19c94161670 +Ref: library/sqlite3 sqlite3 PARSE_COLNAMES4162272 +Ref: 19ca4162272 +Ref: library/sqlite3 sqlite3 connect4163166 +Ref: 3da4163166 +Ref: library/sqlite3 sqlite3 register_converter4166783 +Ref: 19cd4166783 +Ref: library/sqlite3 sqlite3 register_adapter4167264 +Ref: 19cf4167264 +Ref: library/sqlite3 sqlite3 complete_statement4167589 +Ref: 19d04167589 +Ref: library/sqlite3 sqlite3 enable_callback_tracebacks4168923 +Ref: 19d14168923 +Ref: Module functions and constants-Footnote-14169374 +Node: Connection Objects4169414 +Ref: library/sqlite3 connection-objects4169584 +Ref: 19d24169584 +Ref: library/sqlite3 sqlite3-connection-objects4169584 +Ref: 19d34169584 +Ref: library/sqlite3 sqlite3 Connection4169641 +Ref: 3d84169641 +Ref: library/sqlite3 sqlite3 Connection isolation_level4169754 +Ref: 19cc4169754 +Ref: library/sqlite3 sqlite3 Connection in_transaction4170064 +Ref: 19d54170064 +Ref: library/sqlite3 sqlite3 Connection cursor4170290 +Ref: 19d64170290 +Ref: library/sqlite3 sqlite3 Connection commit4170530 +Ref: 19d74170530 +Ref: library/sqlite3 sqlite3 Connection rollback4170923 +Ref: 19d84170923 +Ref: library/sqlite3 sqlite3 Connection close4171070 +Ref: 19d94171070 +Ref: library/sqlite3 sqlite3 Connection execute4171357 +Ref: 19da4171357 +Ref: library/sqlite3 sqlite3 Connection executemany4171653 +Ref: 19db4171653 +Ref: library/sqlite3 sqlite3 Connection executescript4171957 +Ref: 19dd4171957 +Ref: library/sqlite3 sqlite3 Connection create_function4172258 +Ref: 19df4172258 +Ref: library/sqlite3 sqlite3 Connection create_aggregate4173670 +Ref: 19e14173670 +Ref: library/sqlite3 sqlite3 Connection create_collation4175002 +Ref: 19e24175002 +Ref: library/sqlite3 sqlite3 Connection interrupt4176612 +Ref: 19e34176612 +Ref: library/sqlite3 sqlite3 Connection set_authorizer4176852 +Ref: 19e44176852 +Ref: library/sqlite3 sqlite3 Connection set_progress_handler4178212 +Ref: 19e54178212 +Ref: library/sqlite3 sqlite3 Connection set_trace_callback4178870 +Ref: a8d4178870 +Ref: library/sqlite3 sqlite3 Connection enable_load_extension4179637 +Ref: b994179637 +Ref: library/sqlite3 sqlite3 Connection load_extension4181451 +Ref: b9a4181451 +Ref: library/sqlite3 sqlite3 Connection row_factory4181791 +Ref: 19e74181791 +Ref: library/sqlite3 sqlite3 Connection text_factory4183157 +Ref: 19e84183157 +Ref: library/sqlite3 sqlite3 Connection total_changes4184874 +Ref: 19e94184874 +Ref: library/sqlite3 sqlite3 Connection iterdump4185070 +Ref: 19ea4185070 +Ref: library/sqlite3 sqlite3 Connection backup4185717 +Ref: 3d94185717 +Ref: Connection Objects-Footnote-14188130 +Ref: Connection Objects-Footnote-24188176 +Ref: Connection Objects-Footnote-34188475 +Node: Cursor Objects4188774 +Ref: library/sqlite3 cursor-objects4188925 +Ref: 19eb4188925 +Ref: library/sqlite3 sqlite3-cursor-objects4188925 +Ref: 19ec4188925 +Ref: library/sqlite3 sqlite3 Cursor4188974 +Ref: 19bf4188974 +Ref: library/sqlite3 sqlite3 Cursor execute4189085 +Ref: 19c04189085 +Ref: library/sqlite3 sqlite3 Cursor executemany4190362 +Ref: 19dc4190362 +Ref: library/sqlite3 sqlite3 Cursor executescript4192142 +Ref: 19de4192142 +Ref: library/sqlite3 sqlite3 Cursor fetchone4193256 +Ref: 19c14193256 +Ref: library/sqlite3 sqlite3 Cursor fetchmany4193432 +Ref: 19ee4193432 +Ref: library/sqlite3 sqlite3 Cursor fetchall4194409 +Ref: 19c24194409 +Ref: library/sqlite3 sqlite3 Cursor close4194693 +Ref: 19ef4194693 +Ref: library/sqlite3 sqlite3 Cursor rowcount4194998 +Ref: 19f14194998 +Ref: library/sqlite3 sqlite3 Cursor lastrowid4195992 +Ref: 57e4195992 +Ref: library/sqlite3 sqlite3 Cursor arraysize4196641 +Ref: 19f24196641 +Ref: library/sqlite3 sqlite3 Cursor description4196870 +Ref: 19cb4196870 +Ref: library/sqlite3 sqlite3 Cursor connection4197251 +Ref: 19f34197251 +Node: Row Objects4197747 +Ref: library/sqlite3 row-objects4197893 +Ref: 19f44197893 +Ref: library/sqlite3 sqlite3-row-objects4197893 +Ref: 19f54197893 +Ref: library/sqlite3 sqlite3 Row4197936 +Ref: 75e4197936 +Ref: library/sqlite3 sqlite3 Row keys4198396 +Ref: 19f64198396 +Node: Exceptions<4>4199636 +Ref: library/sqlite3 exceptions4199791 +Ref: 19f74199791 +Ref: library/sqlite3 sqlite3-exceptions4199791 +Ref: 19f84199791 +Ref: library/sqlite3 sqlite3 Warning4199832 +Ref: 19ed4199832 +Ref: library/sqlite3 sqlite3 Error4199906 +Ref: 19f94199906 +Ref: library/sqlite3 sqlite3 DatabaseError4200045 +Ref: 19fa4200045 +Ref: library/sqlite3 sqlite3 IntegrityError4200151 +Ref: 19fb4200151 +Ref: library/sqlite3 sqlite3 ProgrammingError4200363 +Ref: 19f04200363 +Ref: library/sqlite3 sqlite3 OperationalError4200633 +Ref: 19e64200633 +Ref: library/sqlite3 sqlite3 NotSupportedError4200997 +Ref: 19e04200997 +Node: SQLite and Python types4201343 +Ref: library/sqlite3 sqlite-and-python-types4201511 +Ref: 19fc4201511 +Ref: library/sqlite3 sqlite3-types4201511 +Ref: 19ce4201511 +Node: Introduction<7>4201781 +Ref: library/sqlite3 introduction4201932 +Ref: 19fd4201932 +Node: Using adapters to store additional Python types in SQLite databases4203819 +Ref: library/sqlite3 using-adapters-to-store-additional-python-types-in-sqlite-databases4204026 +Ref: 19fe4204026 +Node: Letting your object adapt itself4204624 +Ref: library/sqlite3 letting-your-object-adapt-itself4204800 +Ref: 19ff4204800 +Node: Registering an adapter callable4205913 +Ref: library/sqlite3 registering-an-adapter-callable4206089 +Ref: 1a004206089 +Node: Converting SQLite values to custom Python types4207405 +Ref: library/sqlite3 converting-sqlite-values-to-custom-python-types4207628 +Ref: 1a014207628 +Node: Default adapters and converters4210257 +Ref: library/sqlite3 default-adapters-and-converters4210404 +Ref: 1a024210404 +Node: Controlling Transactions4211962 +Ref: library/sqlite3 controlling-transactions4212142 +Ref: 1a034212142 +Ref: library/sqlite3 sqlite3-controlling-transactions4212142 +Ref: 19d44212142 +Node: Using sqlite3 efficiently4213798 +Ref: library/sqlite3 using-sqlite3-efficiently4213946 +Ref: 1a044213946 +Node: Using shortcut methods4214160 +Ref: library/sqlite3 using-shortcut-methods4214298 +Ref: 1a054214298 +Node: Accessing columns by name instead of by index4215596 +Ref: library/sqlite3 accessing-columns-by-name-instead-of-by-index4215784 +Ref: 1a064215784 +Node: Using the connection as a context manager4216508 +Ref: library/sqlite3 using-the-connection-as-a-context-manager4216665 +Ref: 1a074216665 +Node: Data Compression and Archiving4217776 +Ref: library/archiving doc4217916 +Ref: 196d4217916 +Ref: library/archiving archiving4217916 +Ref: 1a084217916 +Ref: library/archiving data-compression-and-archiving4217916 +Ref: 1a094217916 +Node: zlib — Compression compatible with gzip4218508 +Ref: library/zlib doc4218656 +Ref: 1a0a4218656 +Ref: library/zlib module-zlib4218656 +Ref: 1444218656 +Ref: library/zlib zlib-compression-compatible-with-gzip4218656 +Ref: 1a0b4218656 +Ref: library/zlib zlib error4219621 +Ref: 1a0c4219621 +Ref: library/zlib zlib adler324219712 +Ref: 1a0d4219712 +Ref: library/zlib zlib compress4220625 +Ref: 5be4220625 +Ref: library/zlib zlib compressobj4221399 +Ref: 1a0e4221399 +Ref: library/zlib zlib crc324224133 +Ref: 1a0f4224133 +Ref: library/zlib zlib decompress4224961 +Ref: 5bf4224961 +Ref: library/zlib decompress-wbits4225391 +Ref: 1a104225391 +Ref: library/zlib zlib decompressobj4227126 +Ref: 1a114227126 +Ref: library/zlib zlib Compress compress4228125 +Ref: 1a124228125 +Ref: library/zlib zlib Compress flush4228483 +Ref: 1a134228483 +Ref: library/zlib zlib Compress copy4229238 +Ref: 1a144229238 +Ref: library/zlib zlib Decompress unused_data4229606 +Ref: 1a154229606 +Ref: library/zlib zlib Decompress unconsumed_tail4229952 +Ref: 1a164229952 +Ref: library/zlib zlib Decompress eof4230406 +Ref: ac64230406 +Ref: library/zlib zlib Decompress decompress4230689 +Ref: 1a174230689 +Ref: library/zlib zlib Decompress flush4231716 +Ref: 1a184231716 +Ref: library/zlib zlib Decompress copy4232121 +Ref: 1a194232121 +Ref: library/zlib zlib ZLIB_VERSION4232599 +Ref: 1a1a4232599 +Ref: library/zlib zlib ZLIB_RUNTIME_VERSION4232853 +Ref: ac74232853 +Node: gzip — Support for gzip files4233291 +Ref: library/gzip doc4233485 +Ref: 1a1b4233485 +Ref: library/gzip gzip-support-for-gzip-files4233485 +Ref: 1a1c4233485 +Ref: library/gzip module-gzip4233485 +Ref: 8c4233485 +Ref: library/gzip gzip open4234480 +Ref: 1a1d4234480 +Ref: library/gzip gzip BadGzipFile4236011 +Ref: 1d24236011 +Ref: library/gzip gzip GzipFile4236256 +Ref: 6dd4236256 +Ref: library/gzip gzip GzipFile peek4239077 +Ref: b884239077 +Ref: library/gzip gzip GzipFile mtime4239701 +Ref: 1a1e4239701 +Ref: library/gzip gzip compress4241036 +Ref: 1d14241036 +Ref: library/gzip gzip decompress4241425 +Ref: b894241425 +Ref: gzip — Support for gzip files-Footnote-14241689 +Node: Examples of usage4241752 +Ref: library/gzip examples-of-usage4241868 +Ref: 1a234241868 +Ref: library/gzip gzip-usage-examples4241868 +Ref: 1a244241868 +Node: Command Line Interface4242786 +Ref: library/gzip command-line-interface4242902 +Ref: 1a254242902 +Node: Command line options4243322 +Ref: library/gzip command-line-options4243401 +Ref: 1a264243401 +Ref: library/gzip cmdoption-gzip-arg-file4243462 +Ref: 1a274243462 +Ref: library/gzip cmdoption-gzip-fast4243554 +Ref: 1a284243554 +Ref: library/gzip cmdoption-gzip-best4243649 +Ref: 1a294243649 +Ref: library/gzip cmdoption-gzip-d4243744 +Ref: 1a2a4243744 +Ref: library/gzip cmdoption-gzip-h4243815 +Ref: 1a2b4243815 +Node: bz2 — Support for bzip2 compression4243876 +Ref: library/bz2 doc4244074 +Ref: 1a2c4244074 +Ref: library/bz2 bz2-support-for-bzip2-compression4244074 +Ref: 1a2d4244074 +Ref: library/bz2 module-bz24244074 +Ref: 144244074 +Ref: bz2 — Support for bzip2 compression-Footnote-14245114 +Node: De compression of files4245176 +Ref: library/bz2 de-compression-of-files4245308 +Ref: 1a304245308 +Ref: library/bz2 bz2 open4245377 +Ref: 9d64245377 +Ref: library/bz2 bz2 BZ2File4246814 +Ref: 9314246814 +Ref: library/bz2 bz2 BZ2File peek4248364 +Ref: 1a314248364 +Node: Incremental de compression4249828 +Ref: library/bz2 incremental-de-compression4249992 +Ref: 1a324249992 +Ref: library/bz2 bz2 BZ2Compressor4250067 +Ref: 1a2e4250067 +Ref: library/bz2 bz2 BZ2Compressor compress4250407 +Ref: 1a334250407 +Ref: library/bz2 bz2 BZ2Compressor flush4250755 +Ref: 1a344250755 +Ref: library/bz2 bz2 BZ2Decompressor4250984 +Ref: 1a2f4250984 +Ref: library/bz2 bz2 BZ2Decompressor decompress4251535 +Ref: 6a34251535 +Ref: library/bz2 bz2 BZ2Decompressor eof4252893 +Ref: 1a374252893 +Ref: library/bz2 bz2 BZ2Decompressor unused_data4253017 +Ref: 1a364253017 +Ref: library/bz2 bz2 BZ2Decompressor needs_input4253239 +Ref: 1a354253239 +Node: One-shot de compression4253461 +Ref: library/bz2 one-shot-de-compression4253622 +Ref: 1a384253622 +Ref: library/bz2 bz2 compress4253691 +Ref: 151f4253691 +Ref: library/bz2 bz2 decompress4253990 +Ref: 9d74253990 +Node: Examples of usage<2>4254348 +Ref: library/bz2 bz2-usage-examples4254474 +Ref: 1a394254474 +Ref: library/bz2 examples-of-usage4254474 +Ref: 1a3a4254474 +Node: lzma — Compression using the LZMA algorithm4257394 +Ref: library/lzma doc4257595 +Ref: 1a3b4257595 +Ref: library/lzma lzma-compression-using-the-lzma-algorithm4257595 +Ref: 1a3c4257595 +Ref: library/lzma module-lzma4257595 +Ref: ad4257595 +Ref: library/lzma lzma LZMAError4258436 +Ref: 1a3d4258436 +Ref: lzma — Compression using the LZMA algorithm-Footnote-14258855 +Node: Reading and writing compressed files4258918 +Ref: library/lzma reading-and-writing-compressed-files4259089 +Ref: 1a3e4259089 +Ref: library/lzma lzma open4259182 +Ref: 1a3f4259182 +Ref: library/lzma lzma LZMAFile4260949 +Ref: 9324260949 +Ref: library/lzma lzma LZMAFile peek4262826 +Ref: 1a424262826 +Node: Compressing and decompressing data in memory4263707 +Ref: library/lzma compressing-and-decompressing-data-in-memory4263903 +Ref: 1a434263903 +Ref: library/lzma lzma LZMACompressor4264012 +Ref: 1a414264012 +Ref: library/lzma lzma LZMACompressor compress4267090 +Ref: 1a464267090 +Ref: library/lzma lzma LZMACompressor flush4267563 +Ref: 1a474267563 +Ref: library/lzma lzma LZMADecompressor4267843 +Ref: 1a404267843 +Ref: library/lzma lzma LZMADecompressor decompress4269371 +Ref: 7144269371 +Ref: library/lzma lzma LZMADecompressor check4270732 +Ref: 1a4b4270732 +Ref: library/lzma lzma LZMADecompressor eof4270965 +Ref: 1a4c4270965 +Ref: library/lzma lzma LZMADecompressor unused_data4271058 +Ref: 1a4a4271058 +Ref: library/lzma lzma LZMADecompressor needs_input4271229 +Ref: 1a494271229 +Ref: library/lzma lzma compress4271451 +Ref: 1a444271451 +Ref: library/lzma lzma decompress4271797 +Ref: 1a484271797 +Node: Miscellaneous<2>4272304 +Ref: library/lzma miscellaneous4272495 +Ref: 1a4d4272495 +Ref: library/lzma lzma is_check_supported4272542 +Ref: 1a4e4272542 +Node: Specifying custom filter chains4272904 +Ref: library/lzma filter-chain-specs4273062 +Ref: 1a454273062 +Ref: library/lzma specifying-custom-filter-chains4273062 +Ref: 1a4f4273062 +Node: Examples<5>4275911 +Ref: library/lzma examples4276044 +Ref: 1a504276044 +Node: zipfile — Work with ZIP archives4277412 +Ref: library/zipfile doc4277620 +Ref: 1a514277620 +Ref: library/zipfile module-zipfile4277620 +Ref: 1424277620 +Ref: library/zipfile zipfile-work-with-zip-archives4277620 +Ref: 1a524277620 +Ref: library/zipfile zipfile BadZipFile4278483 +Ref: 1a534278483 +Ref: library/zipfile zipfile BadZipfile4278586 +Ref: 1a544278586 +Ref: library/zipfile zipfile LargeZipFile4278747 +Ref: 1a554278747 +Ref: library/zipfile zipfile ZipInfo4279299 +Ref: 5ba4279299 +Ref: library/zipfile zipfile is_zipfile4279997 +Ref: cba4279997 +Ref: library/zipfile zipfile ZIP_STORED4280290 +Ref: 1a5b4280290 +Ref: library/zipfile zipfile ZIP_DEFLATED4280383 +Ref: 1a5c4280383 +Ref: library/zipfile zipfile ZIP_BZIP24280529 +Ref: 1a5d4280529 +Ref: library/zipfile zipfile ZIP_LZMA4280692 +Ref: 1a5e4280692 +Ref: zipfile — Work with ZIP archives-Footnote-14281698 +Ref: zipfile — Work with ZIP archives-Footnote-24281764 +Ref: zipfile — Work with ZIP archives-Footnote-34281832 +Ref: zipfile — Work with ZIP archives-Footnote-44281900 +Node: ZipFile Objects4281933 +Ref: library/zipfile id14282040 +Ref: 1a5f4282040 +Ref: library/zipfile zipfile-objects4282040 +Ref: 1a564282040 +Ref: library/zipfile zipfile ZipFile4282091 +Ref: 41f4282091 +Ref: library/zipfile zipfile ZipFile close4286082 +Ref: 1a604286082 +Ref: library/zipfile zipfile ZipFile getinfo4286253 +Ref: 1a584286253 +Ref: library/zipfile zipfile ZipFile infolist4286512 +Ref: 1a594286512 +Ref: library/zipfile zipfile ZipFile namelist4286770 +Ref: 1a614286770 +Ref: library/zipfile zipfile ZipFile open4286851 +Ref: 5bc4286851 +Ref: library/zipfile zipfile ZipFile extract4289238 +Ref: 1a654289238 +Ref: library/zipfile zipfile ZipFile extractall4290597 +Ref: 1a664290597 +Ref: library/zipfile zipfile ZipFile printdir4291615 +Ref: 1a674291615 +Ref: library/zipfile zipfile ZipFile setpassword4291717 +Ref: 1a684291717 +Ref: library/zipfile zipfile ZipFile read4291820 +Ref: cb94291820 +Ref: library/zipfile zipfile ZipFile testzip4292722 +Ref: 1a694292722 +Ref: library/zipfile zipfile ZipFile write4293092 +Ref: 60f4293092 +Ref: library/zipfile zipfile ZipFile writestr4294258 +Ref: cbb4294258 +Ref: library/zipfile zipfile ZipFile filename4295898 +Ref: 1a6a4295898 +Ref: library/zipfile zipfile ZipFile debug4295959 +Ref: 1a6b4295959 +Ref: library/zipfile zipfile ZipFile comment4296179 +Ref: 1a6c4296179 +Node: Path Objects4296502 +Ref: library/zipfile id24296635 +Ref: 1a6d4296635 +Ref: library/zipfile path-objects4296635 +Ref: 1a574296635 +Ref: library/zipfile zipfile Path4296680 +Ref: 1a6e4296680 +Ref: library/zipfile zipfile Path name4297226 +Ref: 1a6f4297226 +Ref: library/zipfile zipfile Path open4297284 +Ref: 1a704297284 +Ref: library/zipfile zipfile Path iterdir4297665 +Ref: 1a714297665 +Ref: library/zipfile zipfile Path is_dir4297749 +Ref: 1a724297749 +Ref: library/zipfile zipfile Path is_file4297848 +Ref: 1a734297848 +Ref: library/zipfile zipfile Path exists4297943 +Ref: 1a744297943 +Ref: library/zipfile zipfile Path read_text4298071 +Ref: 1a754298071 +Ref: library/zipfile zipfile Path read_bytes4298303 +Ref: 1a764298303 +Node: PyZipFile Objects4298373 +Ref: library/zipfile id34298506 +Ref: 1a774298506 +Ref: library/zipfile pyzipfile-objects4298506 +Ref: 1a784298506 +Ref: library/zipfile zipfile PyZipFile4298708 +Ref: 91b4298708 +Ref: library/zipfile zipfile PyZipFile writepy4299032 +Ref: 91a4299032 +Node: ZipInfo Objects4301859 +Ref: library/zipfile id44302002 +Ref: 1a794302002 +Ref: library/zipfile zipinfo-objects4302002 +Ref: 1a5a4302002 +Ref: library/zipfile zipfile ZipInfo from_file4302370 +Ref: 5b94302370 +Ref: library/zipfile zipfile ZipInfo is_dir4303437 +Ref: 5bb4303437 +Ref: library/zipfile zipfile ZipInfo filename4303637 +Ref: 1a7a4303637 +Ref: library/zipfile zipfile ZipInfo date_time4303709 +Ref: 1a7b4303709 +Ref: library/zipfile zipfile ZipInfo compress_type4304491 +Ref: 1a7c4304491 +Ref: library/zipfile zipfile ZipInfo comment4304579 +Ref: 1a7d4304579 +Ref: library/zipfile zipfile ZipInfo extra4304695 +Ref: 1a7e4304695 +Ref: library/zipfile zipfile ZipInfo create_system4304899 +Ref: 1a7f4304899 +Ref: library/zipfile zipfile ZipInfo create_version4304977 +Ref: 1a804304977 +Ref: library/zipfile zipfile ZipInfo extract_version4305063 +Ref: 1a814305063 +Ref: library/zipfile zipfile ZipInfo reserved4305150 +Ref: 1a824305150 +Ref: library/zipfile zipfile ZipInfo flag_bits4305203 +Ref: 1a834305203 +Ref: library/zipfile zipfile ZipInfo volume4305258 +Ref: 1a844305258 +Ref: library/zipfile zipfile ZipInfo internal_attr4305325 +Ref: 1a854305325 +Ref: library/zipfile zipfile ZipInfo external_attr4305390 +Ref: 1a864305390 +Ref: library/zipfile zipfile ZipInfo header_offset4305460 +Ref: 1a874305460 +Ref: library/zipfile zipfile ZipInfo CRC4305536 +Ref: 1a884305536 +Ref: library/zipfile zipfile ZipInfo compress_size4305603 +Ref: 1a894305603 +Ref: library/zipfile zipfile ZipInfo file_size4305676 +Ref: 1a644305676 +Ref: ZipInfo Objects-Footnote-14305783 +Node: Command-Line Interface4305851 +Ref: library/zipfile command-line-interface4305999 +Ref: 1a8a4305999 +Ref: library/zipfile zipfile-commandline4305999 +Ref: 1a8b4305999 +Node: Command-line options4306772 +Ref: library/zipfile command-line-options4306851 +Ref: 1a8f4306851 +Ref: library/zipfile cmdoption-zipfile-l4306912 +Ref: 1a8e4306912 +Ref: library/zipfile cmdoption-zipfile-list4306945 +Ref: 1a904306945 +Ref: library/zipfile cmdoption-zipfile-c4307014 +Ref: 1a8c4307014 +Ref: library/zipfile cmdoption-zipfile-create4307071 +Ref: 1a914307071 +Ref: library/zipfile cmdoption-zipfile-e4307175 +Ref: 1a8d4307175 +Ref: library/zipfile cmdoption-zipfile-extract4307221 +Ref: 1a924307221 +Ref: library/zipfile cmdoption-zipfile-t4307320 +Ref: 1a934307320 +Ref: library/zipfile cmdoption-zipfile-test4307353 +Ref: 1a944307353 +Node: Decompression pitfalls4307439 +Ref: library/zipfile decompression-pitfalls4307563 +Ref: 1a954307563 +Node: From file itself4307846 +Ref: library/zipfile from-file-itself4307953 +Ref: 1a964307953 +Node: File System limitations4308132 +Ref: library/zipfile file-system-limitations4308269 +Ref: 1a974308269 +Node: Resources limitations4308571 +Ref: library/zipfile resources-limitations4308704 +Ref: 1a984308704 +Ref: Resources limitations-Footnote-14308991 +Node: Interruption4309038 +Ref: library/zipfile interruption4309179 +Ref: 1a994309179 +Node: Default behaviors of extraction4309385 +Ref: library/zipfile default-behaviors-of-extraction4309496 +Ref: 1a9a4309496 +Node: tarfile — Read and write tar archive files4309761 +Ref: library/tarfile doc4309915 +Ref: 1a9b4309915 +Ref: library/tarfile module-tarfile4309915 +Ref: 1014309915 +Ref: library/tarfile pkzip-application-note4309915 +Ref: 1a9c4309915 +Ref: library/tarfile tarfile-read-and-write-tar-archive-files4309915 +Ref: 1a9d4309915 +Ref: library/tarfile tarfile open4311177 +Ref: 7664311177 +Ref: library/tarfile tarfile is_tarfile4317464 +Ref: 1aa24317464 +Ref: library/tarfile tarfile TarError4317685 +Ref: 1aa34317685 +Ref: library/tarfile tarfile ReadError4317775 +Ref: 1a9f4317775 +Ref: library/tarfile tarfile CompressionError4317947 +Ref: 1aa04317947 +Ref: library/tarfile tarfile StreamError4318097 +Ref: 1aa44318097 +Ref: library/tarfile tarfile ExtractError4318236 +Ref: 1aa54318236 +Ref: library/tarfile tarfile HeaderError4318406 +Ref: 1aa64318406 +Ref: library/tarfile tarfile ENCODING4318591 +Ref: 1aa84318591 +Ref: library/tarfile tarfile USTAR_FORMAT4318939 +Ref: 1aaa4318939 +Ref: library/tarfile tarfile GNU_FORMAT4319006 +Ref: 1aab4319006 +Ref: library/tarfile tarfile PAX_FORMAT4319058 +Ref: 1aac4319058 +Ref: library/tarfile tarfile DEFAULT_FORMAT4319121 +Ref: 1aad4319121 +Ref: tarfile — Read and write tar archive files-Footnote-14319971 +Ref: tarfile — Read and write tar archive files-Footnote-24320037 +Node: TarFile Objects4320109 +Ref: library/tarfile id14320229 +Ref: 1aae4320229 +Ref: library/tarfile tarfile-objects4320229 +Ref: 1a9e4320229 +Ref: library/tarfile tarfile TarFile4321097 +Ref: b8c4321097 +Ref: library/tarfile tarfile TarFile open4324487 +Ref: 1ab14324487 +Ref: library/tarfile tarfile TarFile getmember4324652 +Ref: 1ab24324652 +Ref: library/tarfile tarfile TarFile getmembers4324975 +Ref: 76a4324975 +Ref: library/tarfile tarfile TarFile getnames4325162 +Ref: 1ab34325162 +Ref: library/tarfile tarfile TarFile list4325323 +Ref: 7694325323 +Ref: library/tarfile tarfile TarFile next4325773 +Ref: 1ab44325773 +Ref: library/tarfile tarfile TarFile extractall4325993 +Ref: 7674325993 +Ref: library/tarfile tarfile TarFile extract4327304 +Ref: 7684327304 +Ref: library/tarfile tarfile TarFile extractfile4328519 +Ref: 1ab54328519 +Ref: library/tarfile tarfile TarFile add4328915 +Ref: 47c4328915 +Ref: library/tarfile tarfile TarFile addfile4329821 +Ref: 1ab64329821 +Ref: library/tarfile tarfile TarFile gettarinfo4330188 +Ref: 1ab74330188 +Ref: library/tarfile tarfile TarFile close4331329 +Ref: 1abb4331329 +Ref: library/tarfile tarfile TarFile pax_headers4331471 +Ref: 1abc4331471 +Node: TarInfo Objects4331576 +Ref: library/tarfile id24331730 +Ref: 1abd4331730 +Ref: library/tarfile tarinfo-objects4331730 +Ref: 1aaf4331730 +Ref: library/tarfile tarfile TarInfo4332210 +Ref: b8d4332210 +Ref: library/tarfile tarfile TarInfo frombuf4332291 +Ref: 1aa74332291 +Ref: library/tarfile tarfile TarInfo fromtarfile4332504 +Ref: 1abe4332504 +Ref: library/tarfile tarfile TarInfo tobuf4332684 +Ref: 1abf4332684 +Ref: library/tarfile tarfile TarInfo name4333123 +Ref: 1aba4333123 +Ref: library/tarfile tarfile TarInfo size4333186 +Ref: 1ab94333186 +Ref: library/tarfile tarfile TarInfo mtime4333236 +Ref: 1ac04333236 +Ref: library/tarfile tarfile TarInfo mode4333299 +Ref: 1ac14333299 +Ref: library/tarfile tarfile TarInfo type4333351 +Ref: 1ac24333351 +Ref: library/tarfile tarfile TarInfo linkname4333729 +Ref: 1ac34333729 +Ref: library/tarfile tarfile TarInfo uid4333897 +Ref: 1ac44333897 +Ref: library/tarfile tarfile TarInfo gid4333986 +Ref: 1ac54333986 +Ref: library/tarfile tarfile TarInfo uname4334076 +Ref: 1ac64334076 +Ref: library/tarfile tarfile TarInfo gname4334123 +Ref: 1ac74334123 +Ref: library/tarfile tarfile TarInfo pax_headers4334171 +Ref: 1ac84334171 +Ref: library/tarfile tarfile TarInfo isfile4334371 +Ref: 1ac94334371 +Ref: library/tarfile tarfile TarInfo isreg4334479 +Ref: 1aca4334479 +Ref: library/tarfile tarfile TarInfo isdir4334545 +Ref: 1acb4334545 +Ref: library/tarfile tarfile TarInfo issym4334627 +Ref: 1acc4334627 +Ref: library/tarfile tarfile TarInfo islnk4334713 +Ref: 1acd4334713 +Ref: library/tarfile tarfile TarInfo ischr4334795 +Ref: 1ace4334795 +Ref: library/tarfile tarfile TarInfo isblk4334884 +Ref: 1acf4334884 +Ref: library/tarfile tarfile TarInfo isfifo4334969 +Ref: 1ad04334969 +Ref: library/tarfile tarfile TarInfo isdev4335047 +Ref: 1ad14335047 +Node: Command-Line Interface<2>4335168 +Ref: library/tarfile command-line-interface4335318 +Ref: 1ad24335318 +Ref: library/tarfile tarfile-commandline4335318 +Ref: 8e84335318 +Node: Command-line options<2>4336272 +Ref: library/tarfile command-line-options4336357 +Ref: 1ad64336357 +Ref: library/tarfile cmdoption-tarfile-l4336418 +Ref: 1ad54336418 +Ref: library/tarfile cmdoption-tarfile-list4336451 +Ref: 1ad74336451 +Ref: library/tarfile cmdoption-tarfile-c4336520 +Ref: 1ad34336520 +Ref: library/tarfile cmdoption-tarfile-create4336577 +Ref: 1ad84336577 +Ref: library/tarfile cmdoption-tarfile-e4336681 +Ref: 1ad44336681 +Ref: library/tarfile cmdoption-tarfile-extract4336729 +Ref: 1ad94336729 +Ref: library/tarfile cmdoption-tarfile-t4336873 +Ref: 1ada4336873 +Ref: library/tarfile cmdoption-tarfile-test4336906 +Ref: 1adb4336906 +Ref: library/tarfile cmdoption-tarfile-v4336992 +Ref: 1adc4336992 +Node: Examples<6>4337049 +Ref: library/tarfile examples4337205 +Ref: 1add4337205 +Ref: library/tarfile tar-examples4337205 +Ref: 1aa14337205 +Node: Supported tar formats4339078 +Ref: library/tarfile supported-tar-formats4339223 +Ref: 1ade4339223 +Ref: library/tarfile tar-formats4339223 +Ref: 1aa94339223 +Node: Unicode issues4341445 +Ref: library/tarfile tar-unicode4341570 +Ref: 1ab04341570 +Ref: library/tarfile unicode-issues4341570 +Ref: 1adf4341570 +Node: File Formats4343542 +Ref: library/fileformats doc4343688 +Ref: 1ae14343688 +Ref: library/fileformats file-formats4343688 +Ref: 1ae24343688 +Ref: library/fileformats fileformats4343688 +Ref: 1ae34343688 +Node: csv — CSV File Reading and Writing4344154 +Ref: library/csv doc4344290 +Ref: 1ae44344290 +Ref: library/csv csv-csv-file-reading-and-writing4344290 +Ref: 1ae54344290 +Ref: library/csv module-csv4344290 +Ref: 2a4344290 +Ref: csv — CSV File Reading and Writing-Footnote-14346200 +Ref: csv — CSV File Reading and Writing-Footnote-24346262 +Ref: csv — CSV File Reading and Writing-Footnote-34346311 +Node: Module Contents<3>4346360 +Ref: library/csv csv-contents4346494 +Ref: 1ae74346494 +Ref: library/csv module-contents4346494 +Ref: 1ae84346494 +Ref: library/csv csv reader4346605 +Ref: 1ae64346605 +Ref: library/csv csv writer4348275 +Ref: dfc4348275 +Ref: library/csv csv register_dialect4350032 +Ref: 1aec4350032 +Ref: library/csv csv unregister_dialect4350505 +Ref: 1aed4350505 +Ref: library/csv csv get_dialect4350712 +Ref: 1aef4350712 +Ref: library/csv csv list_dialects4350943 +Ref: 1aea4350943 +Ref: library/csv csv field_size_limit4351030 +Ref: 1af04351030 +Ref: library/csv csv DictReader4351265 +Ref: 1bd4351265 +Ref: library/csv csv DictWriter4352866 +Ref: b7a4352866 +Ref: library/csv csv Dialect4354529 +Ref: 1ae94354529 +Ref: library/csv csv excel4354773 +Ref: 1af14354773 +Ref: library/csv csv excel_tab4354956 +Ref: 1af24354956 +Ref: library/csv csv unix_dialect4355161 +Ref: b794355161 +Ref: library/csv csv Sniffer4355462 +Ref: 1af34355462 +Ref: library/csv csv Sniffer sniff4355631 +Ref: 1af44355631 +Ref: library/csv csv Sniffer has_header4355953 +Ref: 1af54355953 +Ref: library/csv csv QUOTE_ALL4356502 +Ref: 1af64356502 +Ref: library/csv csv QUOTE_MINIMAL4356591 +Ref: 1af74356591 +Ref: library/csv csv QUOTE_NONNUMERIC4356812 +Ref: 1af84356812 +Ref: library/csv csv QUOTE_NONE4357007 +Ref: 1af94357007 +Ref: library/csv csv Error4357507 +Ref: 1aee4357507 +Ref: Module Contents<3>-Footnote-14357633 +Ref: Module Contents<3>-Footnote-24357978 +Node: Dialects and Formatting Parameters4358323 +Ref: library/csv csv-fmt-params4358480 +Ref: 1aeb4358480 +Ref: library/csv dialects-and-formatting-parameters4358480 +Ref: 1afa4358480 +Ref: library/csv csv Dialect delimiter4359273 +Ref: 1afb4359273 +Ref: library/csv csv Dialect doublequote4359393 +Ref: 1afc4359393 +Ref: library/csv csv Dialect escapechar4359865 +Ref: 1afd4359865 +Ref: library/csv csv Dialect lineterminator4360245 +Ref: 1afe4360245 +Ref: library/csv csv Dialect quotechar4360602 +Ref: 1aff4360602 +Ref: library/csv csv Dialect quoting4360832 +Ref: 1b004360832 +Ref: library/csv csv Dialect skipinitialspace4361107 +Ref: 1b014361107 +Ref: library/csv csv Dialect strict4361276 +Ref: 1b024361276 +Node: Reader Objects4361416 +Ref: library/csv reader-objects4361569 +Ref: 1b034361569 +Ref: library/csv csv csvreader __next__4361763 +Ref: 1b044361763 +Ref: library/csv csv csvreader dialect4362143 +Ref: 1b054362143 +Ref: library/csv csv csvreader line_num4362244 +Ref: 1b064362244 +Ref: library/csv csv csvreader fieldnames4362496 +Ref: 1b074362496 +Node: Writer Objects4362700 +Ref: library/csv writer-objects4362830 +Ref: 1b084362830 +Ref: library/csv csv csvwriter writerow4363431 +Ref: 6c44363431 +Ref: library/csv csv csvwriter writerows4363744 +Ref: 1b094363744 +Ref: library/csv csv csvwriter dialect4364007 +Ref: 1b0a4364007 +Ref: library/csv csv DictWriter writeheader4364162 +Ref: b7b4364162 +Node: Examples<7>4364646 +Ref: library/csv csv-examples4364753 +Ref: 1b0b4364753 +Ref: library/csv examples4364753 +Ref: 1b0c4364753 +Node: configparser — Configuration file parser4366765 +Ref: library/configparser doc4366941 +Ref: 1b0d4366941 +Ref: library/configparser configparser-configuration-file-parser4366941 +Ref: 1b0e4366941 +Ref: library/configparser module-configparser4366941 +Ref: 234366941 +Ref: configparser — Configuration file parser-Footnote-14368235 +Node: Quick Start4368306 +Ref: library/configparser quick-start4368424 +Ref: 1b0f4368424 +Ref: Quick Start-Footnote-14371105 +Ref: Quick Start-Footnote-24371311 +Node: Supported Datatypes4371517 +Ref: library/configparser supported-datatypes4371659 +Ref: 1b134371659 +Ref: Supported Datatypes-Footnote-14372981 +Ref: Supported Datatypes-Footnote-24373187 +Node: Fallback Values4373393 +Ref: library/configparser fallback-values4373552 +Ref: 1b174373552 +Node: Supported INI File Structure4375050 +Ref: library/configparser supported-ini-file-structure4375213 +Ref: 1b104375213 +Ref: Supported INI File Structure-Footnote-14377489 +Ref: Supported INI File Structure-Footnote-24377695 +Ref: Supported INI File Structure-Footnote-34377901 +Ref: Supported INI File Structure-Footnote-44378107 +Node: Interpolation of values4378313 +Ref: library/configparser interpolation-of-values4378484 +Ref: 1b184378484 +Ref: library/configparser configparser BasicInterpolation4378720 +Ref: 1b194378720 +Ref: library/configparser configparser ExtendedInterpolation4379999 +Ref: bdc4379999 +Ref: Interpolation of values-Footnote-14381480 +Node: Mapping Protocol Access4381686 +Ref: library/configparser mapping-protocol-access4381857 +Ref: 1b114381857 +Ref: Mapping Protocol Access-Footnote-14384746 +Node: Customizing Parser Behaviour4384952 +Ref: library/configparser customizing-parser-behaviour4385119 +Ref: 1b124385119 +Ref: library/configparser configparser ConfigParser BOOLEAN_STATES4395862 +Ref: 1b1b4395862 +Ref: library/configparser configparser ConfigParser SECTCRE4398048 +Ref: 1b1c4398048 +Node: Legacy API Examples4399290 +Ref: library/configparser legacy-api-examples4399454 +Ref: 1b1d4399454 +Node: ConfigParser Objects4403272 +Ref: library/configparser configparser-objects4403431 +Ref: 1b1e4403431 +Ref: library/configparser id114403431 +Ref: 1b1f4403431 +Ref: library/configparser configparser ConfigParser4403492 +Ref: 4aa4403492 +Ref: library/configparser configparser ConfigParser defaults4407295 +Ref: 1b224407295 +Ref: library/configparser configparser ConfigParser sections4407395 +Ref: 1b234407395 +Ref: library/configparser configparser ConfigParser add_section4407538 +Ref: c044407538 +Ref: library/configparser configparser ConfigParser has_section4408027 +Ref: 1b244408027 +Ref: library/configparser configparser ConfigParser options4408203 +Ref: 1b254408203 +Ref: library/configparser configparser ConfigParser has_option4408313 +Ref: 1b264408313 +Ref: library/configparser configparser ConfigParser read4408613 +Ref: 1b274408613 +Ref: library/configparser configparser ConfigParser read_file4410338 +Ref: 1b284410338 +Ref: library/configparser configparser ConfigParser read_string4410819 +Ref: 1b2a4410819 +Ref: library/configparser configparser ConfigParser read_dict4411166 +Ref: 1b214411166 +Ref: library/configparser configparser ConfigParser get4411861 +Ref: c024411861 +Ref: library/configparser configparser ConfigParser getint4412817 +Ref: 1b154412817 +Ref: library/configparser configparser ConfigParser getfloat4413105 +Ref: 1b164413105 +Ref: library/configparser configparser ConfigParser getboolean4413408 +Ref: 1b144413408 +Ref: library/configparser configparser ConfigParser items4414138 +Ref: 1b2b4414138 +Ref: library/configparser configparser ConfigParser set4414776 +Ref: c034414776 +Ref: library/configparser configparser ConfigParser write4415061 +Ref: 1b1a4415061 +Ref: library/configparser configparser ConfigParser remove_option4415502 +Ref: 1b2d4415502 +Ref: library/configparser configparser ConfigParser remove_section4415816 +Ref: 1b2e4415816 +Ref: library/configparser configparser ConfigParser optionxform4416028 +Ref: 1b204416028 +Ref: library/configparser configparser ConfigParser readfp4417001 +Ref: 1b294417001 +Ref: library/configparser configparser MAX_INTERPOLATION_DEPTH4417769 +Ref: 1b2f4417769 +Node: RawConfigParser Objects4417995 +Ref: library/configparser id134418148 +Ref: 1b304418148 +Ref: library/configparser rawconfigparser-objects4418148 +Ref: 1b314418148 +Ref: library/configparser configparser RawConfigParser4418217 +Ref: 8714418217 +Ref: library/configparser configparser RawConfigParser add_section4419199 +Ref: 1b324419199 +Ref: library/configparser configparser RawConfigParser set4419673 +Ref: 1b334419673 +Node: Exceptions<5>4420500 +Ref: library/configparser exceptions4420624 +Ref: 1b344420624 +Ref: library/configparser configparser Error4420667 +Ref: 1b354420667 +Ref: library/configparser configparser NoSectionError4420769 +Ref: 1b2c4420769 +Ref: library/configparser configparser DuplicateSectionError4420875 +Ref: c054420875 +Ref: library/configparser configparser DuplicateOptionError4421287 +Ref: c064421287 +Ref: library/configparser configparser NoOptionError4421647 +Ref: 1b364421647 +Ref: library/configparser configparser InterpolationError4421781 +Ref: 1b374421781 +Ref: library/configparser configparser InterpolationDepthError4421926 +Ref: 1b384421926 +Ref: library/configparser configparser InterpolationMissingOptionError4422186 +Ref: 1b394422186 +Ref: library/configparser configparser InterpolationSyntaxError4422375 +Ref: 1b3a4422375 +Ref: library/configparser configparser MissingSectionHeaderError4422603 +Ref: 1b3b4422603 +Ref: library/configparser configparser ParsingError4422748 +Ref: 1b3c4422748 +Node: netrc — netrc file processing4423005 +Ref: library/netrc doc4423182 +Ref: 1b3d4423182 +Ref: library/netrc module-netrc4423182 +Ref: be4423182 +Ref: library/netrc netrc-netrc-file-processing4423182 +Ref: 1b3e4423182 +Ref: library/netrc netrc netrc4423501 +Ref: 1b3f4423501 +Ref: library/netrc netrc NetrcParseError4424762 +Ref: 1b404424762 +Ref: netrc — netrc file processing-Footnote-14425232 +Node: netrc Objects4425296 +Ref: library/netrc id14425377 +Ref: 1b414425377 +Ref: library/netrc netrc-objects4425377 +Ref: 1b424425377 +Ref: library/netrc netrc netrc authenticators4425482 +Ref: 1b434425482 +Ref: library/netrc netrc netrc __repr__4425828 +Ref: 1b444425828 +Ref: library/netrc netrc netrc hosts4426052 +Ref: 1b454426052 +Ref: library/netrc netrc netrc macros4426250 +Ref: 1b464426250 +Node: xdrlib — Encode and decode XDR data4426656 +Ref: library/xdrlib doc4426843 +Ref: 1b474426843 +Ref: library/xdrlib module-xdrlib4426843 +Ref: 1324426843 +Ref: library/xdrlib xdrlib-encode-and-decode-xdr-data4426843 +Ref: 1b484426843 +Ref: library/xdrlib xdrlib Packer4427448 +Ref: 1b494427448 +Ref: library/xdrlib xdrlib Unpacker4427631 +Ref: 1b4a4427631 +Ref: xdrlib — Encode and decode XDR data-Footnote-14428284 +Ref: xdrlib — Encode and decode XDR data-Footnote-24428349 +Ref: xdrlib — Encode and decode XDR data-Footnote-34428398 +Ref: xdrlib — Encode and decode XDR data-Footnote-44428447 +Ref: xdrlib — Encode and decode XDR data-Footnote-54428496 +Node: Packer Objects4428545 +Ref: library/xdrlib packer-objects4428658 +Ref: 1b4b4428658 +Ref: library/xdrlib xdr-packer-objects4428658 +Ref: 1b4c4428658 +Ref: library/xdrlib xdrlib Packer get_buffer4428766 +Ref: 1b4d4428766 +Ref: library/xdrlib xdrlib Packer reset4428851 +Ref: 1b4e4428851 +Ref: library/xdrlib xdrlib Packer pack_float4429287 +Ref: 1b4f4429287 +Ref: library/xdrlib xdrlib Packer pack_double4429390 +Ref: 1b504429390 +Ref: library/xdrlib xdrlib Packer pack_fstring4429566 +Ref: 1b514429566 +Ref: library/xdrlib xdrlib Packer pack_fopaque4429817 +Ref: 1b524429817 +Ref: library/xdrlib xdrlib Packer pack_string4429953 +Ref: 1b534429953 +Ref: library/xdrlib xdrlib Packer pack_opaque4430172 +Ref: 1b544430172 +Ref: library/xdrlib xdrlib Packer pack_bytes4430306 +Ref: 1b554430306 +Ref: library/xdrlib xdrlib Packer pack_list4430490 +Ref: 1b564430490 +Ref: library/xdrlib xdrlib Packer pack_farray4431180 +Ref: 1b574431180 +Ref: library/xdrlib xdrlib Packer pack_array4431542 +Ref: 1b584431542 +Node: Unpacker Objects4431793 +Ref: library/xdrlib unpacker-objects4431928 +Ref: 1b594431928 +Ref: library/xdrlib xdr-unpacker-objects4431928 +Ref: 1b5a4431928 +Ref: library/xdrlib xdrlib Unpacker reset4432044 +Ref: 1b5b4432044 +Ref: library/xdrlib xdrlib Unpacker get_position4432133 +Ref: 1b5c4432133 +Ref: library/xdrlib xdrlib Unpacker set_position4432233 +Ref: 1b5d4432233 +Ref: library/xdrlib xdrlib Unpacker get_buffer4432442 +Ref: 1b5e4432442 +Ref: library/xdrlib xdrlib Unpacker done4432536 +Ref: 1b5f4432536 +Ref: library/xdrlib xdrlib Unpacker unpack_float4432923 +Ref: 1b614432923 +Ref: library/xdrlib xdrlib Unpacker unpack_double4433017 +Ref: 1b624433017 +Ref: library/xdrlib xdrlib Unpacker unpack_fstring4433234 +Ref: 1b634433234 +Ref: library/xdrlib xdrlib Unpacker unpack_fopaque4433445 +Ref: 1b644433445 +Ref: library/xdrlib xdrlib Unpacker unpack_string4433595 +Ref: 1b654433595 +Ref: library/xdrlib xdrlib Unpacker unpack_opaque4433832 +Ref: 1b664433832 +Ref: library/xdrlib xdrlib Unpacker unpack_bytes4433982 +Ref: 1b674433982 +Ref: library/xdrlib xdrlib Unpacker unpack_list4434183 +Ref: 1b684434183 +Ref: library/xdrlib xdrlib Unpacker unpack_farray4434588 +Ref: 1b694434588 +Ref: library/xdrlib xdrlib Unpacker unpack_array4434859 +Ref: 1b6a4434859 +Node: Exceptions<6>4435134 +Ref: library/xdrlib exceptions4435246 +Ref: 1b6b4435246 +Ref: library/xdrlib xdr-exceptions4435246 +Ref: 1b6c4435246 +Ref: library/xdrlib xdrlib Error4435344 +Ref: 1b604435344 +Ref: library/xdrlib xdrlib ConversionError4435511 +Ref: 1b6d4435511 +Node: plistlib — Generate and parse Mac OS X plist files4435902 +Ref: library/plistlib doc4436049 +Ref: 1b6e4436049 +Ref: library/plistlib module-plistlib4436049 +Ref: cf4436049 +Ref: library/plistlib plistlib-generate-and-parse-mac-os-x-plist-files4436049 +Ref: 1b6f4436049 +Ref: library/plistlib plistlib load4437426 +Ref: 88b4437426 +Ref: library/plistlib plistlib loads4438594 +Ref: 88d4438594 +Ref: library/plistlib plistlib dump4438834 +Ref: 88c4438834 +Ref: library/plistlib plistlib dumps4439932 +Ref: 88e4439932 +Ref: library/plistlib plistlib readPlist4440269 +Ref: 47f4440269 +Ref: library/plistlib plistlib writePlist4440907 +Ref: 9454440907 +Ref: library/plistlib plistlib readPlistFromBytes4441166 +Ref: 4804441166 +Ref: library/plistlib plistlib writePlistToBytes4441595 +Ref: 9464441595 +Ref: library/plistlib plistlib Data4441823 +Ref: 9474441823 +Ref: library/plistlib plistlib UID4442240 +Ref: 2104442240 +Ref: library/plistlib plistlib FMT_XML4442644 +Ref: 88f4442644 +Ref: library/plistlib plistlib FMT_BINARY4442736 +Ref: 8904442736 +Ref: plistlib — Generate and parse Mac OS X plist files-Footnote-14442904 +Ref: plistlib — Generate and parse Mac OS X plist files-Footnote-24442971 +Node: Examples<8>4443070 +Ref: library/plistlib examples4443170 +Ref: 1b704443170 +Node: Cryptographic Services4443912 +Ref: library/crypto doc4444061 +Ref: 1b714444061 +Ref: library/crypto crypto4444061 +Ref: 1b724444061 +Ref: library/crypto cryptographic-services4444061 +Ref: 1b734444061 +Node: hashlib — Secure hashes and message digests4444546 +Ref: library/hashlib doc4444708 +Ref: 1b744444708 +Ref: library/hashlib hashlib-secure-hashes-and-message-digests4444708 +Ref: 1b754444708 +Ref: library/hashlib module-hashlib4444708 +Ref: 8d4444708 +Ref: hashlib — Secure hashes and message digests-Footnote-14445729 +Ref: hashlib — Secure hashes and message digests-Footnote-24445795 +Node: Hash algorithms4445844 +Ref: library/hashlib hash-algorithms4445979 +Ref: 1b764445979 +Ref: library/hashlib id14445979 +Ref: 1b774445979 +Ref: library/hashlib hashlib new4448225 +Ref: 1b784448225 +Ref: library/hashlib hashlib algorithms_guaranteed4448903 +Ref: cfc4448903 +Ref: library/hashlib hashlib algorithms_available4449236 +Ref: cfd4449236 +Ref: library/hashlib hashlib hash digest_size4449772 +Ref: 1b794449772 +Ref: library/hashlib hashlib hash block_size4449847 +Ref: 1b7a4449847 +Ref: library/hashlib hashlib hash name4449981 +Ref: 8524449981 +Ref: library/hashlib hashlib hash update4450402 +Ref: 1b7b4450402 +Ref: library/hashlib hashlib hash digest4450889 +Ref: 1b7c4450889 +Ref: library/hashlib hashlib hash hexdigest4451129 +Ref: 1b7d4451129 +Ref: library/hashlib hashlib hash copy4451400 +Ref: 1b7e4451400 +Node: SHAKE variable length digests4451587 +Ref: library/hashlib shake-variable-length-digests4451745 +Ref: 1b7f4451745 +Ref: library/hashlib hashlib shake digest4452074 +Ref: 1b804452074 +Ref: library/hashlib hashlib shake hexdigest4452298 +Ref: 1b814452298 +Node: Key derivation4452576 +Ref: library/hashlib key-derivation4452725 +Ref: 1b824452725 +Ref: library/hashlib hashlib pbkdf2_hmac4453036 +Ref: 7e64453036 +Ref: library/hashlib hashlib scrypt4454467 +Ref: 4d24454467 +Ref: Key derivation-Footnote-14455235 +Ref: Key derivation-Footnote-24455297 +Node: BLAKE24455346 +Ref: library/hashlib blake24455457 +Ref: 1b834455457 +Ref: BLAKE2-Footnote-14456179 +Ref: BLAKE2-Footnote-24456206 +Ref: BLAKE2-Footnote-34456255 +Node: Creating hash objects4456333 +Ref: library/hashlib creating-hash-objects4456418 +Ref: 1b844456418 +Ref: library/hashlib hashlib blake2b4456545 +Ref: 54c4456545 +Ref: library/hashlib hashlib blake2s4456749 +Ref: 54d4456749 +Ref: Creating hash objects-Footnote-14459681 +Node: Constants<5>4459728 +Ref: library/hashlib constants4459833 +Ref: 1b854459833 +Ref: library/hashlib hashlib blake2b SALT_SIZE4459872 +Ref: 1b864459872 +Ref: library/hashlib hashlib blake2s SALT_SIZE4459901 +Ref: 1b874459901 +Ref: library/hashlib hashlib blake2b PERSON_SIZE4459986 +Ref: 1b884459986 +Ref: library/hashlib hashlib blake2s PERSON_SIZE4460017 +Ref: 1b894460017 +Ref: library/hashlib hashlib blake2b MAX_KEY_SIZE4460122 +Ref: 1b8a4460122 +Ref: library/hashlib hashlib blake2s MAX_KEY_SIZE4460154 +Ref: 1b8b4460154 +Ref: library/hashlib hashlib blake2b MAX_DIGEST_SIZE4460205 +Ref: 1b8c4460205 +Ref: library/hashlib hashlib blake2s MAX_DIGEST_SIZE4460240 +Ref: 1b8d4460240 +Node: Examples<9>4460331 +Ref: library/hashlib examples4460422 +Ref: 1b8e4460422 +Node: Simple hashing4460596 +Ref: library/hashlib simple-hashing4460695 +Ref: 1b8f4460695 +Node: Using different digest sizes4462117 +Ref: library/hashlib using-different-digest-sizes4462238 +Ref: 1b904462238 +Node: Keyed hashing4463379 +Ref: library/hashlib keyed-hashing4463504 +Ref: 1b914463504 +Ref: Keyed hashing-Footnote-14465515 +Node: Randomized hashing4465593 +Ref: library/hashlib randomized-hashing4465705 +Ref: 1b924465705 +Ref: Randomized hashing-Footnote-14468309 +Ref: Randomized hashing-Footnote-24468376 +Node: Personalization4468407 +Ref: library/hashlib personalization4468515 +Ref: 1b934468515 +Ref: Personalization-Footnote-14470510 +Node: Tree mode4470578 +Ref: library/hashlib tree-mode4470659 +Ref: 1b944470659 +Node: Credits4471910 +Ref: library/hashlib credits4471980 +Ref: 1b954471980 +Ref: Credits-Footnote-14474214 +Ref: Credits-Footnote-24474241 +Ref: Credits-Footnote-34474310 +Ref: Credits-Footnote-44474344 +Ref: Credits-Footnote-54474381 +Ref: Credits-Footnote-64474424 +Node: hmac — Keyed-Hashing for Message Authentication4474467 +Ref: library/hmac doc4474701 +Ref: 1b964474701 +Ref: library/hmac hmac-keyed-hashing-for-message-authentication4474701 +Ref: 1b974474701 +Ref: library/hmac module-hmac4474701 +Ref: 8f4474701 +Ref: library/hmac hmac new4474993 +Ref: 8544474993 +Ref: library/hmac hmac digest4475920 +Ref: 3904475920 +Ref: library/hmac hmac HMAC update4476561 +Ref: 8554476561 +Ref: library/hmac hmac HMAC digest4476909 +Ref: 1b984476909 +Ref: library/hmac hmac HMAC hexdigest4477518 +Ref: 1b994477518 +Ref: library/hmac hmac HMAC copy4478106 +Ref: 1b9a4478106 +Ref: library/hmac hmac HMAC digest_size4478344 +Ref: 8584478344 +Ref: library/hmac hmac HMAC block_size4478431 +Ref: 8564478431 +Ref: library/hmac hmac HMAC name4478551 +Ref: 8574478551 +Ref: library/hmac hmac compare_digest4478744 +Ref: a0c4478744 +Ref: hmac — Keyed-Hashing for Message Authentication-Footnote-14479566 +Ref: hmac — Keyed-Hashing for Message Authentication-Footnote-24479629 +Node: secrets — Generate secure random numbers for managing secrets4479678 +Ref: library/secrets doc4479858 +Ref: 1b9b4479858 +Ref: library/secrets module-secrets4479858 +Ref: e44479858 +Ref: library/secrets secrets-generate-secure-random-numbers-for-managing-secrets4479858 +Ref: 1b9c4479858 +Ref: secrets — Generate secure random numbers for managing secrets-Footnote-14480722 +Ref: secrets — Generate secure random numbers for managing secrets-Footnote-24480788 +Node: Random numbers4480837 +Ref: library/secrets random-numbers4480977 +Ref: 1b9d4480977 +Ref: library/secrets secrets SystemRandom4481150 +Ref: 1b9e4481150 +Ref: library/secrets secrets choice4481367 +Ref: 1b9f4481367 +Ref: library/secrets secrets randbelow4481474 +Ref: 1ba04481474 +Ref: library/secrets secrets randbits4481560 +Ref: 1ba14481560 +Node: Generating tokens4481638 +Ref: library/secrets generating-tokens4481802 +Ref: 1ba24481802 +Ref: library/secrets secrets token_bytes4482021 +Ref: 1ba34482021 +Ref: library/secrets secrets token_hex4482314 +Ref: 1ba44482314 +Ref: library/secrets secrets token_urlsafe4482650 +Ref: 1ba54482650 +Node: How many bytes should tokens use?4483080 +Ref: library/secrets how-many-bytes-should-tokens-use4483167 +Ref: 1ba64483167 +Ref: How many bytes should tokens use?-Footnote-14484194 +Node: Other functions4484251 +Ref: library/secrets other-functions4484427 +Ref: 1ba74484427 +Ref: library/secrets secrets compare_digest4484478 +Ref: 1ba84484478 +Ref: Other functions-Footnote-14484771 +Node: Recipes and best practices4484828 +Ref: library/secrets recipes-and-best-practices4484978 +Ref: 1ba94484978 +Ref: Recipes and best practices-Footnote-14486718 +Ref: Recipes and best practices-Footnote-24486773 +Node: Generic Operating System Services4486803 +Ref: library/allos doc4486960 +Ref: 1baa4486960 +Ref: library/allos allos4486960 +Ref: 1bab4486960 +Ref: library/allos generic-operating-system-services4486960 +Ref: 1bac4486960 +Node: os — Miscellaneous operating system interfaces4488511 +Ref: library/os doc4488680 +Ref: 1bad4488680 +Ref: library/os module-os4488680 +Ref: c44488680 +Ref: library/os os-miscellaneous-operating-system-interfaces4488680 +Ref: 1bae4488680 +Ref: library/os os error4490472 +Ref: 1baf4490472 +Ref: library/os os name4490560 +Ref: 1bb04490560 +Ref: library/os os-filenames4491003 +Ref: 1ae04491003 +Ref: os — Miscellaneous operating system interfaces-Footnote-14491408 +Node: File Names Command Line Arguments and Environment Variables4491469 +Ref: library/os file-names-command-line-arguments-and-environment-variables4491640 +Ref: 1bb14491640 +Ref: library/os filesystem-encoding4491640 +Ref: 1bb24491640 +Node: Process Parameters4492644 +Ref: library/os os-procinfo4492844 +Ref: 1bb34492844 +Ref: library/os process-parameters4492844 +Ref: 1bb44492844 +Ref: library/os os ctermid4492998 +Ref: 1bb54492998 +Ref: library/os os environ4493154 +Ref: b374493154 +Ref: library/os os environb4495045 +Ref: b944495045 +Ref: library/os os fsencode4495631 +Ref: 4ef4495631 +Ref: library/os os fsdecode4496061 +Ref: 4ee4496061 +Ref: library/os os fspath4496495 +Ref: 4ed4496495 +Ref: library/os os PathLike4496906 +Ref: 4eb4496906 +Ref: library/os os PathLike __fspath__4497083 +Ref: 4ec4497083 +Ref: library/os os getenv4497353 +Ref: 1bb84497353 +Ref: library/os os getenvb4497832 +Ref: b934497832 +Ref: library/os os get_exec_path4498217 +Ref: 1bb94498217 +Ref: library/os os getegid4498596 +Ref: 1bba4498596 +Ref: library/os os geteuid4498826 +Ref: 1bbb4498826 +Ref: library/os os getgid4498949 +Ref: 1bbc4498949 +Ref: library/os os getgrouplist4499070 +Ref: a5b4499070 +Ref: library/os os getgroups4499383 +Ref: 1bbd4499383 +Ref: library/os os getlogin4500608 +Ref: 1bbf4500608 +Ref: library/os os getpgid4501089 +Ref: 1bc14501089 +Ref: library/os os getpgrp4501313 +Ref: 1bc24501313 +Ref: library/os os getpid4501430 +Ref: 1bc34501430 +Ref: library/os os getppid4501495 +Ref: 1bc44501495 +Ref: library/os os getpriority4501864 +Ref: a464501864 +Ref: library/os os PRIO_PROCESS4502495 +Ref: 1bc54502495 +Ref: library/os os PRIO_PGRP4502521 +Ref: 1bc64502521 +Ref: library/os os PRIO_USER4502544 +Ref: 1bc74502544 +Ref: library/os os getresuid4502729 +Ref: cab4502729 +Ref: library/os os getresgid4502939 +Ref: caa4502939 +Ref: library/os os getuid4503150 +Ref: 1bc84503150 +Ref: library/os os initgroups4503267 +Ref: cae4503267 +Ref: library/os os putenv4503554 +Ref: 1bb64503554 +Ref: library/os os setegid4504510 +Ref: 1bca4504510 +Ref: library/os os seteuid4504635 +Ref: 1bcb4504635 +Ref: library/os os setgid4504759 +Ref: 1bcc4504759 +Ref: library/os os setgroups4504871 +Ref: 1bbe4504871 +Ref: library/os os setpgrp4505518 +Ref: 1bcd4505518 +Ref: library/os os setpgid4505753 +Ref: 1bce4505753 +Ref: library/os os setpriority4506016 +Ref: a474506016 +Ref: library/os os setregid4506792 +Ref: 1bcf4506792 +Ref: library/os os setresgid4506934 +Ref: cac4506934 +Ref: library/os os setresuid4507117 +Ref: cad4507117 +Ref: library/os os setreuid4507299 +Ref: 1bd04507299 +Ref: library/os os getsid4507440 +Ref: 1bd14507440 +Ref: library/os os setsid4507597 +Ref: 1bd24507597 +Ref: library/os os setuid4507751 +Ref: 1bd34507751 +Ref: library/os os strerror4507863 +Ref: 1bd44507863 +Ref: library/os os supports_bytes_environ4508106 +Ref: b924508106 +Ref: library/os os umask4508271 +Ref: 1bd54508271 +Ref: library/os os uname4508369 +Ref: a5d4508369 +Ref: library/os os unsetenv4509420 +Ref: d434509420 +Node: File Object Creation4510156 +Ref: library/os file-object-creation4510323 +Ref: 1bd74510323 +Ref: library/os os-newstreams4510323 +Ref: 1bd84510323 +Ref: library/os os fdopen4510498 +Ref: 10d24510498 +Node: File Descriptor Operations4510825 +Ref: library/os file-descriptor-operations4510995 +Ref: 1bd94510995 +Ref: library/os os-fd-ops4510995 +Ref: 1bda4510995 +Ref: library/os os close4511836 +Ref: 3d24511836 +Ref: library/os os closerange4512282 +Ref: 1bde4512282 +Ref: library/os os copy_file_range4512628 +Ref: 1bdf4512628 +Ref: library/os os device_encoding4513654 +Ref: 1be14513654 +Ref: library/os os dup4513847 +Ref: 1be24513847 +Ref: library/os os dup24514216 +Ref: 3be4514216 +Ref: library/os os fchmod4514675 +Ref: 65c4514675 +Ref: library/os os fchown4515080 +Ref: 65d4515080 +Ref: library/os os fdatasync4515539 +Ref: 65e4515539 +Ref: library/os os fpathconf4515769 +Ref: 1be34515769 +Ref: library/os os fstat4516798 +Ref: 65f4516798 +Ref: library/os os fstatvfs4517058 +Ref: 6604517058 +Ref: library/os os fsync4517331 +Ref: 6614517331 +Ref: library/os os ftruncate4517815 +Ref: 6624517815 +Ref: library/os os get_blocking4518255 +Ref: 7214518255 +Ref: library/os os isatty4518594 +Ref: 1be64518594 +Ref: library/os os lockf4518745 +Ref: a5a4518745 +Ref: library/os os F_LOCK4519260 +Ref: 1be74519260 +Ref: library/os os F_TLOCK4519280 +Ref: 1be84519280 +Ref: library/os os F_ULOCK4519301 +Ref: 1be94519301 +Ref: library/os os F_TEST4519322 +Ref: 1bea4519322 +Ref: library/os os lseek4519475 +Ref: a5e4519475 +Ref: library/os os SEEK_SET4519945 +Ref: d944519945 +Ref: library/os os SEEK_CUR4519967 +Ref: d954519967 +Ref: library/os os SEEK_END4519989 +Ref: d964519989 +Ref: library/os os open4520250 +Ref: 6654520250 +Ref: library/os os O_RDONLY4522315 +Ref: 1beb4522315 +Ref: library/os os O_WRONLY4522337 +Ref: 1bec4522337 +Ref: library/os os O_RDWR4522359 +Ref: 1bee4522359 +Ref: library/os os O_APPEND4522379 +Ref: 1bef4522379 +Ref: library/os os O_CREAT4522401 +Ref: 1bf04522401 +Ref: library/os os O_EXCL4522422 +Ref: 192a4522422 +Ref: library/os os O_TRUNC4522442 +Ref: 1bf14522442 +Ref: library/os os O_DSYNC4522525 +Ref: 1bf24522525 +Ref: library/os os O_RSYNC4522546 +Ref: 1bf34522546 +Ref: library/os os O_SYNC4522567 +Ref: 1bf44522567 +Ref: library/os os O_NDELAY4522587 +Ref: 1bf54522587 +Ref: library/os os O_NONBLOCK4522609 +Ref: 7234522609 +Ref: library/os os O_NOCTTY4522633 +Ref: 1bf64522633 +Ref: library/os os O_CLOEXEC4522655 +Ref: 9c34522655 +Ref: library/os os O_BINARY4522799 +Ref: 1bed4522799 +Ref: library/os os O_NOINHERIT4522821 +Ref: 1bf74522821 +Ref: library/os os O_SHORT_LIVED4522846 +Ref: 1bf84522846 +Ref: library/os os O_TEMPORARY4522873 +Ref: 1bf94522873 +Ref: library/os os O_RANDOM4522898 +Ref: 1bfa4522898 +Ref: library/os os O_SEQUENTIAL4522920 +Ref: 1bfb4522920 +Ref: library/os os O_TEXT4522946 +Ref: 1bfc4522946 +Ref: library/os os O_ASYNC4523024 +Ref: 1bfd4523024 +Ref: library/os os O_DIRECT4523045 +Ref: 1bfe4523045 +Ref: library/os os O_DIRECTORY4523067 +Ref: 1bff4523067 +Ref: library/os os O_NOFOLLOW4523092 +Ref: 1c004523092 +Ref: library/os os O_NOATIME4523116 +Ref: 1c014523116 +Ref: library/os os O_PATH4523139 +Ref: 8834523139 +Ref: library/os os O_TMPFILE4523159 +Ref: 8844523159 +Ref: library/os os O_SHLOCK4523182 +Ref: d974523182 +Ref: library/os os O_EXLOCK4523204 +Ref: d984523204 +Ref: library/os os openpty4523495 +Ref: 1c024523495 +Ref: library/os os pipe4523944 +Ref: 1bdc4523944 +Ref: library/os os pipe24524283 +Ref: a2e4524283 +Ref: library/os os posix_fallocate4524673 +Ref: 6674524673 +Ref: library/os os posix_fadvise4524930 +Ref: 6664524930 +Ref: library/os os POSIX_FADV_NORMAL4525531 +Ref: 1c034525531 +Ref: library/os os POSIX_FADV_SEQUENTIAL4525562 +Ref: 1c044525562 +Ref: library/os os POSIX_FADV_RANDOM4525597 +Ref: 1c054525597 +Ref: library/os os POSIX_FADV_NOREUSE4525628 +Ref: 1c064525628 +Ref: library/os os POSIX_FADV_WILLNEED4525660 +Ref: 1c074525660 +Ref: library/os os POSIX_FADV_DONTNEED4525693 +Ref: 1c084525693 +Ref: library/os os pread4525927 +Ref: 3b94525927 +Ref: library/os os preadv4526313 +Ref: 3b74526313 +Ref: library/os os RWF_NOWAIT4527367 +Ref: 1c0a4527367 +Ref: library/os os RWF_HIPRI4527875 +Ref: 1c094527875 +Ref: library/os os pwrite4528272 +Ref: 3bc4528272 +Ref: library/os os pwritev4528555 +Ref: 3ba4528555 +Ref: library/os os RWF_DSYNC4529572 +Ref: 1c0d4529572 +Ref: library/os os RWF_SYNC4529847 +Ref: 1c0e4529847 +Ref: library/os os read4530120 +Ref: 6684530120 +Ref: library/os os sendfile4531061 +Ref: 3594531061 +Ref: library/os os set_blocking4532412 +Ref: 7224532412 +Ref: library/os os SF_NODISKIO4532777 +Ref: 1c0f4532777 +Ref: library/os os SF_MNOWAIT4532802 +Ref: 1c104532802 +Ref: library/os os SF_SYNC4532826 +Ref: 1c114532826 +Ref: library/os os readv4533011 +Ref: 3b84533011 +Ref: library/os os tcgetpgrp4533626 +Ref: 1c124533626 +Ref: library/os os tcsetpgrp4533840 +Ref: 1c134533840 +Ref: library/os os ttyname4534068 +Ref: 1c144534068 +Ref: library/os os write4534318 +Ref: 66e4534318 +Ref: library/os os writev4535166 +Ref: 3bb4535166 +Ref: File Descriptor Operations-Footnote-14535859 +Ref: File Descriptor Operations-Footnote-24535908 +Ref: File Descriptor Operations-Footnote-34535971 +Ref: File Descriptor Operations-Footnote-44536020 +Node: Querying the size of a terminal4536069 +Ref: library/os querying-the-size-of-a-terminal4536203 +Ref: 1c154536203 +Ref: library/os terminal-size4536203 +Ref: 1c164536203 +Ref: library/os os get_terminal_size4536307 +Ref: a494536307 +Ref: library/os os terminal_size4536943 +Ref: 195f4536943 +Ref: library/os os terminal_size columns4537064 +Ref: 1c174537064 +Ref: library/os os terminal_size lines4537148 +Ref: 1c184537148 +Node: Inheritance of File Descriptors4537231 +Ref: library/os fd-inheritance4537365 +Ref: 7fa4537365 +Ref: library/os inheritance-of-file-descriptors4537365 +Ref: 1c194537365 +Ref: library/os os get_inheritable4538328 +Ref: 7fb4538328 +Ref: library/os os set_inheritable4538455 +Ref: 7fc4538455 +Ref: library/os os get_handle_inheritable4538578 +Ref: 7fd4538578 +Ref: library/os os set_handle_inheritable4538743 +Ref: 7fe4538743 +Node: Files and Directories4538909 +Ref: library/os files-and-directories4539077 +Ref: 1c1b4539077 +Ref: library/os os-file-dir4539077 +Ref: 1bb74539077 +Ref: library/os path-fd4539226 +Ref: 3b54539226 +Ref: library/os dir-fd4540170 +Ref: a2f4540170 +Ref: library/os follow-symlinks4540885 +Ref: a304540885 +Ref: library/os os access4541469 +Ref: a314541469 +Ref: library/os os F_OK4544017 +Ref: 1c1c4544017 +Ref: library/os os R_OK4544035 +Ref: 1c1d4544035 +Ref: library/os os W_OK4544053 +Ref: 1c1e4544053 +Ref: library/os os X_OK4544071 +Ref: 1c1f4544071 +Ref: library/os os chdir4544260 +Ref: a3f4544260 +Ref: library/os os chflags4544932 +Ref: a324544932 +Ref: library/os os chmod4545997 +Ref: a334545997 +Ref: library/os os chown4547753 +Ref: a344547753 +Ref: library/os os chroot4548645 +Ref: 1c214548645 +Ref: library/os os fchdir4548850 +Ref: 65b4548850 +Ref: library/os os getcwd4549253 +Ref: 188d4549253 +Ref: library/os os getcwdb4549347 +Ref: 2b84549347 +Ref: library/os os lchflags4549655 +Ref: 1c224549655 +Ref: library/os os lchmod4550133 +Ref: 1c234550133 +Ref: library/os os lchown4550702 +Ref: 1c244550702 +Ref: library/os os link4551213 +Ref: a354551213 +Ref: library/os os listdir4551991 +Ref: a414551991 +Ref: library/os os lstat4553467 +Ref: 1f24553467 +Ref: library/os os mkdir4554603 +Ref: a364554603 +Ref: library/os mkdir-modebits4554807 +Ref: 1c254554807 +Ref: library/os os makedirs4555655 +Ref: 3bd4555655 +Ref: library/os os mkfifo4557374 +Ref: 6634557374 +Ref: library/os os mknod4558259 +Ref: 6644558259 +Ref: library/os os major4559129 +Ref: 1c284559129 +Ref: library/os os minor4559297 +Ref: 1c294559297 +Ref: library/os os makedev4559465 +Ref: 1c274559465 +Ref: library/os os pathconf4559586 +Ref: a424559586 +Ref: library/os os pathconf_names4560683 +Ref: 1c2a4560683 +Ref: library/os os readlink4561005 +Ref: 1f34561005 +Ref: library/os os remove4562503 +Ref: a374562503 +Ref: library/os os removedirs4563356 +Ref: 1c2b4563356 +Ref: library/os os rename4564198 +Ref: a384564198 +Ref: library/os os renames4565692 +Ref: 1c2c4565692 +Ref: library/os os replace4566479 +Ref: a394566479 +Ref: library/os os rmdir4567346 +Ref: a3a4567346 +Ref: library/os os scandir4567981 +Ref: 25f4567981 +Ref: library/os os scandir close4569757 +Ref: 5674569757 +Ref: library/os os DirEntry4571281 +Ref: 4f14571281 +Ref: library/os os DirEntry name4572466 +Ref: 1c2f4572466 +Ref: library/os os DirEntry path4572837 +Ref: 1c304572837 +Ref: library/os os DirEntry inode4573572 +Ref: 1c314573572 +Ref: library/os os DirEntry is_dir4573931 +Ref: 1c2d4573931 +Ref: library/os os DirEntry is_file4575366 +Ref: 6554575366 +Ref: library/os os DirEntry is_symlink4576089 +Ref: 1c324576089 +Ref: library/os os DirEntry stat4576940 +Ref: 1c2e4576940 +Ref: library/os os stat4578360 +Ref: 1f14578360 +Ref: library/os os stat_result4580942 +Ref: 188f4580942 +Ref: library/os os stat_result st_mode4581192 +Ref: 1c334581192 +Ref: library/os os stat_result st_ino4581287 +Ref: 1c344581287 +Ref: library/os os stat_result st_dev4581536 +Ref: 1c354581536 +Ref: library/os os stat_result st_nlink4581628 +Ref: 1c364581628 +Ref: library/os os stat_result st_uid4581691 +Ref: 1c374581691 +Ref: library/os os stat_result st_gid4581765 +Ref: 1c384581765 +Ref: library/os os stat_result st_size4581840 +Ref: 1c394581840 +Ref: library/os os stat_result st_atime4582094 +Ref: 1c3a4582094 +Ref: library/os os stat_result st_mtime4582184 +Ref: 1a204582184 +Ref: library/os os stat_result st_ctime4582288 +Ref: 1c3b4582288 +Ref: library/os os stat_result st_atime_ns4582485 +Ref: 1c3c4582485 +Ref: library/os os stat_result st_mtime_ns4582606 +Ref: 1c3d4582606 +Ref: library/os os stat_result st_ctime_ns4582741 +Ref: 1c3e4582741 +Ref: library/os os stat_result st_blocks4584191 +Ref: 1c3f4584191 +Ref: library/os os stat_result st_blksize4584365 +Ref: 1c404584365 +Ref: library/os os stat_result st_rdev4584567 +Ref: 1c414584567 +Ref: library/os os stat_result st_flags4584642 +Ref: 1c424584642 +Ref: library/os os stat_result st_gen4584867 +Ref: 1c434584867 +Ref: library/os os stat_result st_birthtime4584930 +Ref: 1c444584930 +Ref: library/os os stat_result st_fstype4585085 +Ref: 3bf4585085 +Ref: library/os os stat_result st_rsize4585293 +Ref: 1c454585293 +Ref: library/os os stat_result st_creator4585357 +Ref: 1c464585357 +Ref: library/os os stat_result st_type4585421 +Ref: 1c474585421 +Ref: library/os os stat_result st_file_attributes4585543 +Ref: 7204585543 +Ref: library/os os stat_result st_reparse_tag4585850 +Ref: 1c484585850 +Ref: library/os os statvfs4587700 +Ref: a434587700 +Ref: library/os os supports_dir_fd4589722 +Ref: a3e4589722 +Ref: library/os os supports_effective_ids4590881 +Ref: a454590881 +Ref: library/os os supports_fd4591619 +Ref: a444591619 +Ref: library/os os supports_follow_symlinks4592441 +Ref: 19484592441 +Ref: library/os os symlink4593575 +Ref: a3b4593575 +Ref: library/os os sync4595227 +Ref: a594595227 +Ref: library/os os truncate4595358 +Ref: 7244595358 +Ref: library/os os unlink4595878 +Ref: a3c4595878 +Ref: library/os os utime4596402 +Ref: a3d4596402 +Ref: library/os os walk4598265 +Ref: 6544598265 +Ref: library/os os fwalk4603311 +Ref: 3b44603311 +Ref: library/os os memfd_create4605748 +Ref: 1f04605748 +Ref: library/os os MFD_CLOEXEC4606621 +Ref: 1c494606621 +Ref: library/os os MFD_ALLOW_SEALING4606646 +Ref: 1c4a4606646 +Ref: library/os os MFD_HUGETLB4606677 +Ref: 1c4b4606677 +Ref: library/os os MFD_HUGE_SHIFT4606702 +Ref: 1c4c4606702 +Ref: library/os os MFD_HUGE_MASK4606730 +Ref: 1c4d4606730 +Ref: library/os os MFD_HUGE_64KB4606757 +Ref: 1c4e4606757 +Ref: library/os os MFD_HUGE_512KB4606784 +Ref: 1c4f4606784 +Ref: library/os os MFD_HUGE_1MB4606812 +Ref: 1c504606812 +Ref: library/os os MFD_HUGE_2MB4606838 +Ref: 1c514606838 +Ref: library/os os MFD_HUGE_8MB4606864 +Ref: 1c524606864 +Ref: library/os os MFD_HUGE_16MB4606890 +Ref: 1c534606890 +Ref: library/os os MFD_HUGE_32MB4606917 +Ref: 1c544606917 +Ref: library/os os MFD_HUGE_256MB4606944 +Ref: 1c554606944 +Ref: library/os os MFD_HUGE_512MB4606972 +Ref: 1c564606972 +Ref: library/os os MFD_HUGE_1GB4607000 +Ref: 1c574607000 +Ref: library/os os MFD_HUGE_2GB4607026 +Ref: 1c584607026 +Ref: library/os os MFD_HUGE_16GB4607052 +Ref: 1c594607052 +Ref: Files and Directories-Footnote-14607391 +Ref: Files and Directories-Footnote-24607440 +Ref: Files and Directories-Footnote-34607483 +Ref: Files and Directories-Footnote-44607562 +Ref: Files and Directories-Footnote-54607643 +Ref: Files and Directories-Footnote-64607732 +Ref: Files and Directories-Footnote-74607821 +Node: Linux extended attributes4607879 +Ref: library/os linux-extended-attributes4607962 +Ref: 1c5a4607962 +Ref: library/os os getxattr4608104 +Ref: a4b4608104 +Ref: library/os os listxattr4608763 +Ref: a4c4608763 +Ref: library/os os removexattr4609370 +Ref: a4d4609370 +Ref: library/os os setxattr4610031 +Ref: a4e4610031 +Ref: library/os os XATTR_SIZE_MAX4611241 +Ref: 1c5d4611241 +Ref: library/os os XATTR_CREATE4611377 +Ref: 1c5c4611377 +Ref: library/os os XATTR_REPLACE4611548 +Ref: 1c5b4611548 +Node: Process Management4611730 +Ref: library/os os-process4611898 +Ref: 1c5e4611898 +Ref: library/os process-management4611898 +Ref: 1c5f4611898 +Ref: library/os os abort4612511 +Ref: 1c614612511 +Ref: library/os os add_dll_directory4612886 +Ref: 1c24612886 +Ref: library/os os execl4614034 +Ref: 1c604614034 +Ref: library/os os execle4614081 +Ref: 1c624614081 +Ref: library/os os execlp4614134 +Ref: 1c634614134 +Ref: library/os os execlpe4614182 +Ref: 1c644614182 +Ref: library/os os execv4614236 +Ref: 1bc94614236 +Ref: library/os os execve4614272 +Ref: a404614272 +Ref: library/os os execvp4614314 +Ref: 1c654614314 +Ref: library/os os execvpe4614351 +Ref: 1c664614351 +Ref: library/os os _exit4617570 +Ref: 13b64617570 +Ref: library/os os EX_OK4618318 +Ref: 1c674618318 +Ref: library/os os EX_USAGE4618422 +Ref: 1c684618422 +Ref: library/os os EX_DATAERR4618603 +Ref: 1c694618603 +Ref: library/os os EX_NOINPUT4618723 +Ref: 1c6a4618723 +Ref: library/os os EX_NOUSER4618867 +Ref: 1c6b4618867 +Ref: library/os os EX_NOHOST4618988 +Ref: 1c6c4618988 +Ref: library/os os EX_UNAVAILABLE4619109 +Ref: 1c6d4619109 +Ref: library/os os EX_SOFTWARE4619243 +Ref: 1c6e4619243 +Ref: library/os os EX_OSERR4619375 +Ref: 1c6f4619375 +Ref: library/os os EX_OSFILE4619556 +Ref: 1c704619556 +Ref: library/os os EX_CANTCREAT4619736 +Ref: 1c714619736 +Ref: library/os os EX_IOERR4619884 +Ref: 1c724619884 +Ref: library/os os EX_TEMPFAIL4620030 +Ref: 1c734620030 +Ref: library/os os EX_PROTOCOL4620303 +Ref: 1c744620303 +Ref: library/os os EX_NOPERM4620465 +Ref: 1c754620465 +Ref: library/os os EX_CONFIG4620670 +Ref: 1c764620670 +Ref: library/os os EX_NOTFOUND4620812 +Ref: 1c774620812 +Ref: library/os os fork4620948 +Ref: 9614620948 +Ref: library/os os forkpty4621619 +Ref: 1c784621619 +Ref: library/os os kill4622349 +Ref: cf34622349 +Ref: library/os os killpg4623244 +Ref: 1c7b4623244 +Ref: library/os os nice4623478 +Ref: a484623478 +Ref: library/os os plock4623639 +Ref: 1c7c4623639 +Ref: library/os os popen4623841 +Ref: 2a94623841 +Ref: library/os os posix_spawn4625121 +Ref: 2574625121 +Ref: library/os os POSIX_SPAWN_OPEN4626127 +Ref: 1c7e4626127 +Ref: library/os os POSIX_SPAWN_CLOSE4626300 +Ref: 1c7f4626300 +Ref: library/os os POSIX_SPAWN_DUP24626423 +Ref: 1c804626423 +Ref: library/os os posix_spawnp4629038 +Ref: 1c7d4629038 +Ref: library/os os register_at_fork4629753 +Ref: 3b64629753 +Ref: library/os os spawnl4631167 +Ref: 1c1a4631167 +Ref: library/os os spawnle4631209 +Ref: 1c824631209 +Ref: library/os os spawnlp4631257 +Ref: 1c834631257 +Ref: library/os os spawnlpe4631300 +Ref: 1c844631300 +Ref: library/os os spawnv4631349 +Ref: 1c854631349 +Ref: library/os os spawnve4631392 +Ref: 1c864631392 +Ref: library/os os spawnvp4631441 +Ref: 1c874631441 +Ref: library/os os spawnvpe4631485 +Ref: 1c884631485 +Ref: library/os os P_NOWAIT4635355 +Ref: 1c894635355 +Ref: library/os os P_NOWAITO4635377 +Ref: 1c8b4635377 +Ref: library/os os P_WAIT4635722 +Ref: 1c8a4635722 +Ref: library/os os P_DETACH4636138 +Ref: 1c8c4636138 +Ref: library/os os P_OVERLAY4636160 +Ref: 1c8d4636160 +Ref: library/os os startfile4636651 +Ref: 1c8e4636651 +Ref: library/os os system4638355 +Ref: dc14638355 +Ref: library/os os times4639911 +Ref: a5c4639911 +Ref: library/os os wait4640940 +Ref: 66b4640940 +Ref: library/os os waitid4641350 +Ref: 66c4641350 +Ref: library/os os P_PID4642167 +Ref: 1c8f4642167 +Ref: library/os os P_PGID4642186 +Ref: 1c904642186 +Ref: library/os os P_ALL4642206 +Ref: 1c914642206 +Ref: library/os os WEXITED4642405 +Ref: 1c924642405 +Ref: library/os os WSTOPPED4642426 +Ref: 1c934642426 +Ref: library/os os WNOWAIT4642448 +Ref: 1c964642448 +Ref: library/os os CLD_EXITED4642649 +Ref: 1c974642649 +Ref: library/os os CLD_DUMPED4642673 +Ref: 1c984642673 +Ref: library/os os CLD_TRAPPED4642697 +Ref: 1c994642697 +Ref: library/os os CLD_CONTINUED4642722 +Ref: 1c9a4642722 +Ref: library/os os waitpid4642920 +Ref: 66d4642920 +Ref: library/os os wait34644794 +Ref: 6694644794 +Ref: library/os os wait44645290 +Ref: 66a4645290 +Ref: library/os os WNOHANG4645752 +Ref: 1c954645752 +Ref: library/os os WCONTINUED4645989 +Ref: 1c944645989 +Ref: library/os os WUNTRACED4646221 +Ref: 1c9b4646221 +Ref: library/os os WCOREDUMP4646655 +Ref: 1c9c4646655 +Ref: library/os os WIFCONTINUED4646923 +Ref: 1c9e4646923 +Ref: library/os os WIFSTOPPED4647240 +Ref: 1ca04647240 +Ref: library/os os WIFSIGNALED4647631 +Ref: 1c9d4647631 +Ref: library/os os WIFEXITED4647810 +Ref: 1ca14647810 +Ref: library/os os WEXITSTATUS4648078 +Ref: 1ca24648078 +Ref: library/os os WSTOPSIG4648279 +Ref: 1ca34648279 +Ref: library/os os WTERMSIG4648498 +Ref: 1ca44648498 +Ref: Process Management-Footnote-14648777 +Ref: Process Management-Footnote-24648849 +Ref: Process Management-Footnote-34648958 +Node: Interface to the scheduler4649007 +Ref: library/os interface-to-the-scheduler4649186 +Ref: 1ca54649186 +Ref: library/os os SCHED_OTHER4649571 +Ref: 1ca64649571 +Ref: library/os os SCHED_BATCH4649634 +Ref: 1ca74649634 +Ref: library/os os SCHED_IDLE4649783 +Ref: 1ca84649783 +Ref: library/os os SCHED_SPORADIC4649877 +Ref: 1ca94649877 +Ref: library/os os SCHED_FIFO4649960 +Ref: 1caa4649960 +Ref: library/os os SCHED_RR4650031 +Ref: 1cab4650031 +Ref: library/os os SCHED_RESET_ON_FORK4650093 +Ref: 1cac4650093 +Ref: library/os os sched_param4650315 +Ref: 1c814650315 +Ref: library/os os sched_param sched_priority4650607 +Ref: 1cad4650607 +Ref: library/os os sched_get_priority_min4650703 +Ref: a504650703 +Ref: library/os os sched_get_priority_max4650868 +Ref: a4f4650868 +Ref: library/os os sched_setscheduler4651033 +Ref: a574651033 +Ref: library/os os sched_getscheduler4651319 +Ref: a534651319 +Ref: library/os os sched_setparam4651541 +Ref: a564651541 +Ref: library/os os sched_getparam4651753 +Ref: a524651753 +Ref: library/os os sched_rr_get_interval4651953 +Ref: a544651953 +Ref: library/os os sched_yield4652125 +Ref: a584652125 +Ref: library/os os sched_setaffinity4652196 +Ref: a554652196 +Ref: library/os os sched_getaffinity4652459 +Ref: a514652459 +Node: Miscellaneous System Information4652613 +Ref: library/os miscellaneous-system-information4652791 +Ref: 1cae4652791 +Ref: library/os os-path4652791 +Ref: 1caf4652791 +Ref: library/os os confstr4652878 +Ref: 1cb04652878 +Ref: library/os os confstr_names4653913 +Ref: 1cb14653913 +Ref: library/os os cpu_count4654205 +Ref: 87f4654205 +Ref: library/os os getloadavg4654531 +Ref: 1cb24654531 +Ref: library/os os sysconf4654784 +Ref: 1c0b4654784 +Ref: library/os os sysconf_names4655204 +Ref: 1cb34655204 +Ref: library/os os curdir4655695 +Ref: 18ad4655695 +Ref: library/os os pardir4655896 +Ref: 1c264655896 +Ref: library/os os sep4656097 +Ref: 19364656097 +Ref: library/os os altsep4656509 +Ref: 19374656509 +Ref: library/os os extsep4656808 +Ref: 1cb44656808 +Ref: library/os os pathsep4656995 +Ref: ff04656995 +Ref: library/os os defpath4657239 +Ref: 194c4657239 +Ref: library/os os linesep4657448 +Ref: 12a64657448 +Ref: library/os os devnull4657861 +Ref: 1cb54657861 +Ref: library/os os RTLD_LAZY4658035 +Ref: a5f4658035 +Ref: library/os os RTLD_NOW4658058 +Ref: a604658058 +Ref: library/os os RTLD_GLOBAL4658080 +Ref: a614658080 +Ref: library/os os RTLD_LOCAL4658105 +Ref: a624658105 +Ref: library/os os RTLD_NODELETE4658129 +Ref: a634658129 +Ref: library/os os RTLD_NOLOAD4658156 +Ref: a644658156 +Ref: library/os os RTLD_DEEPBIND4658181 +Ref: a654658181 +Node: Random numbers<2>4658424 +Ref: library/os random-numbers4658567 +Ref: 1cb64658567 +Ref: library/os os getrandom4658618 +Ref: 5694658618 +Ref: library/os os urandom4659452 +Ref: 4d14659452 +Ref: library/os os GRND_NONBLOCK4661376 +Ref: 1cb84661376 +Ref: library/os os GRND_RANDOM4661849 +Ref: 1cb74661849 +Ref: Random numbers<2>-Footnote-14662068 +Ref: Random numbers<2>-Footnote-24662130 +Node: io — Core tools for working with streams4662179 +Ref: library/io doc4662393 +Ref: 1cb94662393 +Ref: library/io io-core-tools-for-working-with-streams4662393 +Ref: 1cba4662393 +Ref: library/io module-io4662393 +Ref: a14662393 +Ref: io — Core tools for working with streams-Footnote-14662739 +Node: Overview4662800 +Ref: library/io io-overview4662923 +Ref: 12a14662923 +Ref: library/io overview4662923 +Ref: 1cbb4662923 +Node: Text I/O4664180 +Ref: library/io text-i-o4664252 +Ref: 1cbc4664252 +Node: Binary I/O4664943 +Ref: library/io binary-i-o4665031 +Ref: 1cbd4665031 +Node: Raw I/O4665900 +Ref: library/io raw-i-o4665971 +Ref: 1cbe4665971 +Node: High-level Module Interface4666424 +Ref: library/io high-level-module-interface4666571 +Ref: 1cbf4666571 +Ref: library/io io DEFAULT_BUFFER_SIZE4666646 +Ref: 12a34666646 +Ref: library/io io open4666881 +Ref: 65a4666881 +Ref: library/io io open_code4667321 +Ref: 1cc04667321 +Ref: library/io io BlockingIOError4667990 +Ref: 1cc24667990 +Ref: library/io io UnsupportedOperation4668121 +Ref: 1cc34668121 +Node: Class hierarchy4668468 +Ref: library/io class-hierarchy4668621 +Ref: 1cc44668621 +Node: I/O Base Classes4673284 +Ref: library/io i-o-base-classes4673373 +Ref: 1cc64673373 +Ref: library/io io IOBase4673426 +Ref: 1db4673426 +Ref: library/io io IOBase close4675339 +Ref: 1bdd4675339 +Ref: library/io io IOBase closed4675749 +Ref: 1cc74675749 +Ref: library/io io IOBase fileno4675824 +Ref: 1bdb4675824 +Ref: library/io io IOBase flush4676045 +Ref: 1cc84676045 +Ref: library/io io IOBase isatty4676205 +Ref: 12a44676205 +Ref: library/io io IOBase readable4676345 +Ref: 1cc94676345 +Ref: library/io io IOBase readline4676507 +Ref: 13ae4676507 +Ref: library/io io IOBase readlines4676876 +Ref: 14384676876 +Ref: library/io io IOBase seek4677335 +Ref: 1a624677335 +Ref: library/io io IOBase seekable4678331 +Ref: 1cca4678331 +Ref: library/io io IOBase tell4678563 +Ref: 1a634678563 +Ref: library/io io IOBase truncate4678636 +Ref: ca54678636 +Ref: library/io io IOBase writable4679211 +Ref: 1ccb4679211 +Ref: library/io io IOBase writelines4679411 +Ref: 14394679411 +Ref: library/io io IOBase __del__4679631 +Ref: 1ccc4679631 +Ref: library/io io RawIOBase4679849 +Ref: a134679849 +Ref: library/io io RawIOBase read4680352 +Ref: 7024680352 +Ref: library/io io RawIOBase readall4681033 +Ref: 1ccd4681033 +Ref: library/io io RawIOBase readinto4681189 +Ref: 7034681189 +Ref: library/io io RawIOBase write4681527 +Ref: 1cce4681527 +Ref: library/io io BufferedIOBase4682163 +Ref: 5884682163 +Ref: library/io io BufferedIOBase raw4683380 +Ref: 1cd04683380 +Ref: library/io io BufferedIOBase detach4683650 +Ref: 1cd14683650 +Ref: library/io io BufferedIOBase read4684084 +Ref: 1a224684084 +Ref: library/io io BufferedIOBase read14684889 +Ref: 1a214684889 +Ref: library/io io BufferedIOBase readinto4685369 +Ref: 1ccf4685369 +Ref: library/io io BufferedIOBase readinto14685900 +Ref: 7014685900 +Ref: library/io io BufferedIOBase write4686392 +Ref: 5894686392 +Node: Raw File I/O4687209 +Ref: library/io raw-file-i-o4687323 +Ref: 1cd24687323 +Ref: library/io io FileIO4687368 +Ref: ca44687368 +Ref: library/io io FileIO mode4689759 +Ref: 1cd34689759 +Ref: library/io io FileIO name4689834 +Ref: 1ab84689834 +Node: Buffered Streams4689979 +Ref: library/io buffered-streams4690088 +Ref: 1cd44690088 +Ref: library/io io BytesIO4690232 +Ref: 79b4690232 +Ref: library/io io BytesIO getbuffer4690715 +Ref: b6d4690715 +Ref: library/io io BytesIO getvalue4691282 +Ref: 1a1f4691282 +Ref: library/io io BytesIO read14691404 +Ref: 1cd54691404 +Ref: library/io io BytesIO readinto14691583 +Ref: 1cd64691583 +Ref: library/io io BufferedReader4691734 +Ref: b8a4691734 +Ref: library/io io BufferedReader peek4692533 +Ref: 1cd74692533 +Ref: library/io io BufferedReader read4692804 +Ref: 1cd84692804 +Ref: library/io io BufferedReader read14693000 +Ref: 1cd94693000 +Ref: library/io io BufferedWriter4693334 +Ref: 12a74693334 +Ref: library/io io BufferedWriter flush4694407 +Ref: 1cda4694407 +Ref: library/io io BufferedWriter write4694591 +Ref: 1cdb4694591 +Ref: library/io io BufferedRandom4694878 +Ref: 12a84694878 +Ref: library/io io BufferedRWPair4695485 +Ref: 1cc54695485 +Node: Text I/O<2>4696425 +Ref: library/io id14696513 +Ref: 1cdc4696513 +Ref: library/io io TextIOBase4696552 +Ref: c394696552 +Ref: library/io io TextIOBase encoding4696895 +Ref: 1cdd4696895 +Ref: library/io io TextIOBase errors4697055 +Ref: 1cde4697055 +Ref: library/io io TextIOBase newlines4697139 +Ref: 1cdf4697139 +Ref: library/io io TextIOBase buffer4697383 +Ref: c3a4697383 +Ref: library/io io TextIOBase detach4697656 +Ref: 1ce04697656 +Ref: library/io io TextIOBase read4698171 +Ref: 1ce14698171 +Ref: library/io io TextIOBase readline4698376 +Ref: 18b94698376 +Ref: library/io io TextIOBase seek4698639 +Ref: 1ce24698639 +Ref: library/io io TextIOBase tell4699607 +Ref: 1ce34699607 +Ref: library/io io TextIOBase write4699811 +Ref: 1ce44699811 +Ref: library/io io TextIOWrapper4699940 +Ref: 5f34699940 +Ref: library/io io TextIOWrapper line_buffering4703364 +Ref: 1ce64703364 +Ref: library/io io TextIOWrapper write_through4703446 +Ref: 1ce74703446 +Ref: library/io io TextIOWrapper reconfigure4703604 +Ref: 39d4703604 +Ref: library/io io StringIO4704379 +Ref: 82d4704379 +Ref: library/io io StringIO getvalue4705264 +Ref: 1ce84705264 +Ref: library/io io IncrementalNewlineDecoder4705912 +Ref: 1ce94705912 +Node: Performance<3>4706094 +Ref: library/io performance4706211 +Ref: 1cea4706211 +Node: Binary I/O<2>4706460 +Ref: library/io id24706544 +Ref: 1ceb4706544 +Node: Text I/O<3>4707196 +Ref: library/io id34707307 +Ref: 1cec4707307 +Node: Multi-threading<3>4707884 +Ref: library/io multi-threading4707992 +Ref: 1ced4707992 +Node: Reentrancy4708525 +Ref: library/io reentrancy4708613 +Ref: 1cee4708613 +Node: time — Time access and conversions4709405 +Ref: library/time doc4709642 +Ref: 1cef4709642 +Ref: library/time module-time4709642 +Ref: 10a4709642 +Ref: library/time time-time-access-and-conversions4709642 +Ref: 1cf04709642 +Ref: library/time epoch4710342 +Ref: 1cf14710342 +Ref: time — Time access and conversions-Footnote-14714715 +Node: Functions<2>4714765 +Ref: library/time functions4714877 +Ref: 1cf24714877 +Ref: library/time time-functions4714877 +Ref: 1cf34714877 +Ref: library/time time asctime4714916 +Ref: b634714916 +Ref: library/time time pthread_getcpuclockid4715584 +Ref: 3fb4715584 +Ref: library/time time clock_getres4716190 +Ref: ab24716190 +Ref: library/time time clock_gettime4716464 +Ref: ab34716464 +Ref: library/time time clock_gettime_ns4716731 +Ref: 31e4716731 +Ref: library/time time clock_settime4716930 +Ref: ab44716930 +Ref: library/time time clock_settime_ns4717188 +Ref: 31f4717188 +Ref: library/time time ctime4717390 +Ref: 15624717390 +Ref: library/time time get_clock_info4717957 +Ref: ab14717957 +Ref: library/time time gmtime4719088 +Ref: b3f4719088 +Ref: library/time time localtime4719567 +Ref: 155c4719567 +Ref: library/time time mktime4719856 +Ref: b644719856 +Ref: library/time time monotonic4720566 +Ref: 76f4720566 +Ref: library/time time monotonic_ns4721038 +Ref: 3204721038 +Ref: library/time time perf_counter4721180 +Ref: 2aa4721180 +Ref: library/time time perf_counter_ns4721621 +Ref: 3214721621 +Ref: library/time time process_time4721774 +Ref: 2ab4721774 +Ref: library/time time process_time_ns4722190 +Ref: 3224722190 +Ref: library/time time sleep4722343 +Ref: 6804722343 +Ref: library/time time strftime4723105 +Ref: b654723105 +Ref: library/time time strptime4730185 +Ref: d914730185 +Ref: library/time time struct_time4731886 +Ref: 5994731886 +Ref: library/time time time4734647 +Ref: 31d4734647 +Ref: library/time time thread_time4735946 +Ref: 3f94735946 +Ref: library/time time thread_time_ns4736495 +Ref: 3fa4736495 +Ref: library/time time time_ns4736646 +Ref: 3234736646 +Ref: library/time time tzset4736833 +Ref: 1cf74736833 +Ref: Functions<2>-Footnote-14740751 +Ref: Functions<2>-Footnote-24740800 +Ref: Functions<2>-Footnote-34740850 +Ref: Functions<2>-Footnote-44740899 +Ref: Functions<2>-Footnote-54741536 +Ref: Functions<2>-Footnote-64741586 +Node: Clock ID Constants4741634 +Ref: library/time clock-id-constants4741773 +Ref: 1cf84741773 +Ref: library/time time-clock-id-constants4741773 +Ref: 1cf54741773 +Ref: library/time time CLOCK_BOOTTIME4741933 +Ref: 3f54741933 +Ref: library/time time CLOCK_HIGHRES4742415 +Ref: 1cf94742415 +Ref: library/time time CLOCK_MONOTONIC4742740 +Ref: 3f64742740 +Ref: library/time time CLOCK_MONOTONIC_RAW4742941 +Ref: 1cfa4742941 +Ref: library/time time CLOCK_PROCESS_CPUTIME_ID4743226 +Ref: 1cfb4743226 +Ref: library/time time CLOCK_PROF4743385 +Ref: 3f74743385 +Ref: library/time time CLOCK_THREAD_CPUTIME_ID4743561 +Ref: 1cfc4743561 +Ref: library/time time CLOCK_UPTIME4743703 +Ref: 3f84743703 +Ref: library/time time CLOCK_UPTIME_RAW4743989 +Ref: 2394743989 +Ref: library/time time CLOCK_REALTIME4744386 +Ref: 1cf64744386 +Node: Timezone Constants4744574 +Ref: library/time time-timezone-constants4744692 +Ref: 1cfd4744692 +Ref: library/time timezone-constants4744692 +Ref: 1cfe4744692 +Ref: library/time time altzone4744749 +Ref: 1cff4744749 +Ref: library/time time daylight4745037 +Ref: 1d004745037 +Ref: library/time time timezone4745123 +Ref: 1d014745123 +Ref: library/time time tzname4745321 +Ref: 1d024745321 +Node: argparse — Parser for command-line options arguments and sub-commands4746506 +Ref: library/argparse doc4746751 +Ref: 1d034746751 +Ref: library/argparse argparse-parser-for-command-line-options-arguments-and-sub-commands4746751 +Ref: 1d044746751 +Ref: library/argparse module-argparse4746751 +Ref: 64746751 +Ref: argparse — Parser for command-line options arguments and sub-commands-Footnote-14747851 +Node: Example<7>4747918 +Ref: library/argparse example4748067 +Ref: 1d064748067 +Node: Creating a parser4749684 +Ref: library/argparse creating-a-parser4749773 +Ref: 1d074749773 +Node: Adding arguments4750136 +Ref: library/argparse adding-arguments4750251 +Ref: 1d084750251 +Node: Parsing arguments4751398 +Ref: library/argparse parsing-arguments4751487 +Ref: 1d0b4751487 +Node: ArgumentParser objects4752217 +Ref: library/argparse argumentparser-objects4752398 +Ref: 1d0d4752398 +Ref: library/argparse argparse ArgumentParser4752463 +Ref: 6954752463 +Node: prog4754911 +Ref: library/argparse prog4754988 +Ref: 1d0e4754988 +Node: usage4756850 +Ref: library/argparse usage4756947 +Ref: 1d0f4756947 +Node: description4758117 +Ref: library/argparse description4758216 +Ref: 1d104758216 +Node: epilog4758979 +Ref: library/argparse epilog4759080 +Ref: 1d114759080 +Node: parents4759859 +Ref: library/argparse parents4759964 +Ref: 1d124759964 +Node: formatter_class4761418 +Ref: library/argparse formatter-class4761529 +Ref: 1d134761529 +Ref: library/argparse argparse RawDescriptionHelpFormatter4761747 +Ref: 1d194761747 +Ref: library/argparse argparse RawTextHelpFormatter4761795 +Ref: 1d1a4761795 +Ref: library/argparse argparse ArgumentDefaultsHelpFormatter4761836 +Ref: 1d1b4761836 +Ref: library/argparse argparse MetavarTypeHelpFormatter4761886 +Ref: 1d1c4761886 +Node: prefix_chars4765403 +Ref: library/argparse prefix-chars4765528 +Ref: 1d144765528 +Node: fromfile_prefix_chars4766266 +Ref: library/argparse fromfile-prefix-chars4766392 +Ref: 1d154766392 +Node: argument_default4767667 +Ref: library/argparse argument-default4767793 +Ref: 1d164767793 +Node: allow_abbrev4768680 +Ref: library/argparse allow-abbrev4768801 +Ref: 6974768801 +Ref: library/argparse id14768801 +Ref: 1d214768801 +Node: conflict_handler4769452 +Ref: library/argparse conflict-handler4769565 +Ref: 1d174769565 +Node: add_help4771123 +Ref: library/argparse add-help4771215 +Ref: 1d184771215 +Node: The add_argument method4772793 +Ref: library/argparse the-add-argument-method4772985 +Ref: 1d224772985 +Ref: library/argparse argparse ArgumentParser add_argument4773058 +Ref: 1d094773058 +Node: name or flags4774848 +Ref: library/argparse name-or-flags4774936 +Ref: 1d234774936 +Node: action4776085 +Ref: library/argparse action4776187 +Ref: 1d244776187 +Node: nargs4781794 +Ref: library/argparse nargs4781888 +Ref: 1d254781888 +Ref: library/argparse argparse-remainder4785676 +Ref: 1d2d4785676 +Node: const4786498 +Ref: library/argparse const4786593 +Ref: 1d264786593 +Node: default4787788 +Ref: library/argparse default4787882 +Ref: 1d274787882 +Node: type4789800 +Ref: library/argparse type4789896 +Ref: 1d1d4789896 +Node: choices4792521 +Ref: library/argparse choices4792618 +Ref: 1d284792618 +Node: required4794108 +Ref: library/argparse required4794205 +Ref: 1d294794205 +Node: help4795165 +Ref: library/argparse help4795262 +Ref: 1d2a4795262 +Node: metavar4797374 +Ref: library/argparse metavar4797467 +Ref: 1d2b4797467 +Node: dest4799709 +Ref: library/argparse dest4799812 +Ref: 1d1e4799812 +Node: Action classes4801447 +Ref: library/argparse action-classes4801534 +Ref: 1d2e4801534 +Ref: library/argparse argparse Action4801816 +Ref: 1d2c4801816 +Node: The parse_args method4803604 +Ref: library/argparse the-parse-args-method4803789 +Ref: 1d2f4803789 +Ref: library/argparse argparse ArgumentParser parse_args4803858 +Ref: 1d0a4803858 +Node: Option value syntax4804734 +Ref: library/argparse option-value-syntax4804837 +Ref: 1d324804837 +Node: Invalid arguments4806234 +Ref: library/argparse invalid-arguments4806368 +Ref: 1d334806368 +Node: Arguments containing -4807353 +Ref: library/argparse arguments-containing4807506 +Ref: 1d344807506 +Node: Argument abbreviations prefix matching4809511 +Ref: library/argparse argument-abbreviations-prefix-matching4809662 +Ref: 1d354809662 +Ref: library/argparse prefix-matching4809662 +Ref: 6964809662 +Node: Beyond sys argv4810590 +Ref: library/argparse args4810739 +Ref: 1d304810739 +Ref: library/argparse beyond-sys-argv4810739 +Ref: 1d364810739 +Node: The Namespace object4811689 +Ref: library/argparse namespace4811791 +Ref: 1d314811791 +Ref: library/argparse the-namespace-object4811791 +Ref: 1d374811791 +Ref: library/argparse argparse Namespace4811854 +Ref: 1d0c4811854 +Node: Other utilities4812878 +Ref: library/argparse other-utilities4813063 +Ref: 1d384813063 +Node: Sub-commands4813335 +Ref: library/argparse sub-commands4813424 +Ref: 1d394813424 +Ref: library/argparse argparse ArgumentParser add_subparsers4813471 +Ref: 4a44813471 +Node: FileType objects4822365 +Ref: library/argparse filetype-objects4822478 +Ref: 1d3a4822478 +Ref: library/argparse argparse FileType4822533 +Ref: 8204822533 +Node: Argument groups4823983 +Ref: library/argparse argument-groups4824100 +Ref: 1d3b4824100 +Ref: library/argparse argparse ArgumentParser add_argument_group4824153 +Ref: 1d3c4824153 +Node: Mutual exclusion4826258 +Ref: library/argparse mutual-exclusion4826374 +Ref: 1d3d4826374 +Ref: library/argparse argparse ArgumentParser add_mutually_exclusive_group4826429 +Ref: 1d3e4826429 +Node: Parser defaults4828024 +Ref: library/argparse parser-defaults4828138 +Ref: 1d3f4828138 +Ref: library/argparse argparse ArgumentParser set_defaults4828191 +Ref: 1d204828191 +Ref: library/argparse argparse ArgumentParser get_default4829315 +Ref: 1d404829315 +Node: Printing help4829672 +Ref: library/argparse printing-help4829785 +Ref: 1d414829785 +Ref: library/argparse argparse ArgumentParser print_usage4830014 +Ref: 1d424830014 +Ref: library/argparse argparse ArgumentParser print_help4830246 +Ref: 1d434830246 +Ref: library/argparse argparse ArgumentParser format_usage4830600 +Ref: 1d444830600 +Ref: library/argparse argparse ArgumentParser format_help4830779 +Ref: 1d454830779 +Node: Partial parsing4830992 +Ref: library/argparse partial-parsing4831114 +Ref: 1d464831114 +Ref: library/argparse argparse ArgumentParser parse_known_args4831167 +Ref: 1d474831167 +Node: Customizing file parsing4832206 +Ref: library/argparse customizing-file-parsing4832330 +Ref: 1d484832330 +Ref: library/argparse argparse ArgumentParser convert_arg_line_to_args4832401 +Ref: 1d1f4832401 +Node: Exiting methods4833306 +Ref: library/argparse exiting-methods4833433 +Ref: 1d494833433 +Ref: library/argparse argparse ArgumentParser exit4833486 +Ref: 1d4a4833486 +Ref: library/argparse argparse ArgumentParser error4834018 +Ref: 1d4b4834018 +Node: Intermixed parsing4834206 +Ref: library/argparse intermixed-parsing4834300 +Ref: 1d4c4834300 +Ref: library/argparse argparse ArgumentParser parse_intermixed_args4834359 +Ref: 3454834359 +Ref: library/argparse argparse ArgumentParser parse_known_intermixed_args4834447 +Ref: 1d4d4834447 +Node: Upgrading optparse code4835980 +Ref: library/argparse id24836135 +Ref: 1d4e4836135 +Ref: library/argparse upgrading-optparse-code4836135 +Ref: b294836135 +Node: getopt — C-style parser for command line options4838633 +Ref: library/getopt doc4838881 +Ref: 1d514838881 +Ref: library/getopt getopt-c-style-parser-for-command-line-options4838881 +Ref: 1d524838881 +Ref: library/getopt module-getopt4838881 +Ref: 874838881 +Ref: library/getopt getopt getopt4839893 +Ref: 1d534839893 +Ref: library/getopt getopt gnu_getopt4842080 +Ref: 1d554842080 +Ref: library/getopt getopt GetoptError4842660 +Ref: 1d544842660 +Ref: library/getopt getopt error4843240 +Ref: 1d564843240 +Ref: getopt — C-style parser for command line options-Footnote-14845682 +Node: logging — Logging facility for Python4845747 +Ref: library/logging doc4845964 +Ref: 1d574845964 +Ref: library/logging logging-logging-facility-for-python4845964 +Ref: 1d584845964 +Ref: library/logging module-logging4845964 +Ref: aa4845964 +Ref: logging — Logging facility for Python-Footnote-14847733 +Node: Logger Objects4847809 +Ref: library/logging logger4847922 +Ref: 1d594847922 +Ref: library/logging logger-objects4847922 +Ref: 1d5a4847922 +Ref: library/logging logging Logger4849015 +Ref: 3a74849015 +Ref: library/logging logging Logger propagate4849042 +Ref: 1d5c4849042 +Ref: library/logging logging Logger setLevel4850331 +Ref: 1d5d4850331 +Ref: library/logging logging Logger isEnabledFor4852293 +Ref: 1d604852293 +Ref: library/logging logging Logger getEffectiveLevel4852636 +Ref: 1d5f4852636 +Ref: library/logging logging Logger getChild4853107 +Ref: c8b4853107 +Ref: library/logging logging Logger debug4853606 +Ref: 7104853606 +Ref: library/logging logging Logger info4858784 +Ref: 1d634858784 +Ref: library/logging logging Logger warning4858963 +Ref: 1d654858963 +Ref: library/logging logging Logger error4859369 +Ref: 1d664859369 +Ref: library/logging logging Logger critical4859550 +Ref: 70f4859550 +Ref: library/logging logging Logger log4859737 +Ref: 70d4859737 +Ref: library/logging logging Logger exception4859933 +Ref: 70e4859933 +Ref: library/logging logging Logger addFilter4860248 +Ref: 1d674860248 +Ref: library/logging logging Logger removeFilter4860347 +Ref: 1d684860347 +Ref: library/logging logging Logger filter4860454 +Ref: 1d694860454 +Ref: library/logging logging Logger addHandler4860891 +Ref: 1d6a4860891 +Ref: library/logging logging Logger removeHandler4860988 +Ref: 1d6b4860988 +Ref: library/logging logging Logger findCaller4861093 +Ref: 1d6c4861093 +Ref: library/logging logging Logger handle4861895 +Ref: 1d6d4861895 +Ref: library/logging logging Logger makeRecord4862294 +Ref: 1d6e4862294 +Ref: library/logging logging Logger hasHandlers4862556 +Ref: 1d6f4862556 +Node: Logging Levels4863175 +Ref: library/logging levels4863312 +Ref: 1d5e4863312 +Ref: library/logging logging-levels4863312 +Ref: 1d704863312 +Node: Handler Objects4864182 +Ref: library/logging handler4864322 +Ref: 1d714864322 +Ref: library/logging handler-objects4864322 +Ref: 1d724864322 +Ref: library/logging logging Handler4864648 +Ref: 1d624864648 +Ref: library/logging logging Handler __init__4864676 +Ref: 1d734864676 +Ref: library/logging logging Handler createLock4864971 +Ref: 1d744864971 +Ref: library/logging logging Handler acquire4865159 +Ref: 1d754865159 +Ref: library/logging logging Handler release4865273 +Ref: 1d764865273 +Ref: library/logging logging Handler setLevel4865375 +Ref: 1d774865375 +Ref: library/logging logging Handler setFormatter4865947 +Ref: 1d784865947 +Ref: library/logging logging Handler addFilter4866054 +Ref: 1d794866054 +Ref: library/logging logging Handler removeFilter4866154 +Ref: 1d7a4866154 +Ref: library/logging logging Handler filter4866262 +Ref: 1d7b4866262 +Ref: library/logging logging Handler flush4866661 +Ref: 1d7c4866661 +Ref: library/logging logging Handler close4866829 +Ref: 1d7d4866829 +Ref: library/logging logging Handler handle4867187 +Ref: 1d7f4867187 +Ref: library/logging logging Handler handleError4867465 +Ref: 1d804867465 +Ref: library/logging logging Handler format4868253 +Ref: 1d824868253 +Ref: library/logging logging Handler emit4868421 +Ref: 1d814868421 +Node: Formatter Objects4868735 +Ref: library/logging formatter-objects4868875 +Ref: 1d834868875 +Ref: library/logging id14868875 +Ref: 1d844868875 +Ref: library/logging logging Formatter4870039 +Ref: 1d614870039 +Ref: library/logging logging Formatter format4871197 +Ref: 1d884871197 +Ref: library/logging logging Formatter formatTime4872611 +Ref: 1d864872611 +Ref: library/logging logging Formatter formatException4874908 +Ref: 1d894874908 +Ref: library/logging logging Formatter formatStack4875247 +Ref: 1d8a4875247 +Node: Filter Objects4875539 +Ref: library/logging filter4875681 +Ref: 1d8c4875681 +Ref: library/logging filter-objects4875681 +Ref: 1d8d4875681 +Ref: library/logging logging Filter4876195 +Ref: b774876195 +Ref: library/logging logging Filter filter4876485 +Ref: 1d8e4876485 +Node: LogRecord Objects4878466 +Ref: library/logging log-record4878611 +Ref: 1d914878611 +Ref: library/logging logrecord-objects4878611 +Ref: 1d924878611 +Ref: library/logging logging LogRecord4878904 +Ref: 9064878904 +Ref: library/logging logging LogRecord getMessage4880731 +Ref: 1d944880731 +Node: LogRecord attributes4882229 +Ref: library/logging id24882381 +Ref: 1d974882381 +Ref: library/logging logrecord-attributes4882381 +Ref: 1d854882381 +Node: LoggerAdapter Objects4890331 +Ref: library/logging logger-adapter4890479 +Ref: 1d994890479 +Ref: library/logging loggeradapter-objects4890479 +Ref: 1d9a4890479 +Ref: library/logging logging LoggerAdapter4890759 +Ref: c8c4890759 +Ref: library/logging logging LoggerAdapter process4890952 +Ref: 1d9c4890952 +Node: Thread Safety4892164 +Ref: library/logging thread-safety4892314 +Ref: 1d9d4892314 +Node: Module-Level Functions4892973 +Ref: library/logging module-level-functions4893125 +Ref: 1d9e4893125 +Ref: library/logging logging getLogger4893283 +Ref: 1d5b4893283 +Ref: library/logging logging getLoggerClass4893871 +Ref: 1d9f4893871 +Ref: library/logging logging getLogRecordFactory4894349 +Ref: 1d954894349 +Ref: library/logging logging debug4894808 +Ref: 1d644894808 +Ref: library/logging logging info4898655 +Ref: 1d8f4898655 +Ref: library/logging logging warning4898833 +Ref: 1da14898833 +Ref: library/logging logging error4899225 +Ref: 1da24899225 +Ref: library/logging logging critical4899405 +Ref: 1da34899405 +Ref: library/logging logging exception4899591 +Ref: 1da44899591 +Ref: library/logging logging log4899898 +Ref: 1da54899898 +Ref: library/logging logging disable4900826 +Ref: 1da64900826 +Ref: library/logging logging addLevelName4902042 +Ref: 1da74902042 +Ref: library/logging logging getLevelName4902700 +Ref: 1da94902700 +Ref: library/logging logging makeLogRecord4904307 +Ref: 1d934904307 +Ref: library/logging logging basicConfig4904672 +Ref: 1e04904672 +Ref: library/logging logging shutdown4908998 +Ref: 1d7e4908998 +Ref: library/logging logging setLoggerClass4909440 +Ref: 1da04909440 +Ref: library/logging logging setLogRecordFactory4910074 +Ref: 1d964910074 +Ref: Module-Level Functions-Footnote-14911407 +Node: Module-Level Attributes4911450 +Ref: library/logging module-level-attributes4911625 +Ref: 1daa4911625 +Ref: library/logging logging lastResort4911694 +Ref: b764911694 +Node: Integration with the warnings module4912303 +Ref: library/logging integration-with-the-warnings-module4912447 +Ref: 1dab4912447 +Ref: library/logging logging captureWarnings4912669 +Ref: 1dac4912669 +Ref: Integration with the warnings module-Footnote-14914054 +Ref: Integration with the warnings module-Footnote-24914103 +Node: logging config — Logging configuration4914156 +Ref: library/logging config doc4914360 +Ref: 1dae4914360 +Ref: library/logging config logging-config-logging-configuration4914360 +Ref: 1daf4914360 +Ref: library/logging config module-logging config4914360 +Ref: ab4914360 +Ref: logging config — Logging configuration-Footnote-14914980 +Node: Configuration functions4915053 +Ref: library/logging config configuration-functions4915193 +Ref: 1db04915193 +Ref: library/logging config logging-config-api4915193 +Ref: c884915193 +Ref: library/logging config logging config dictConfig4915635 +Ref: b2b4915635 +Ref: library/logging config logging config fileConfig4917753 +Ref: 3a94917753 +Ref: library/logging config logging config listen4919984 +Ref: 8724919984 +Ref: library/logging config logging config stopListening4923322 +Ref: 1db44923322 +Node: Configuration dictionary schema4923566 +Ref: library/logging config configuration-dictionary-schema4923740 +Ref: 1db54923740 +Ref: library/logging config logging-config-dictschema4923740 +Ref: 1db14923740 +Node: Dictionary Schema Details4924678 +Ref: library/logging config dictionary-schema-details4924805 +Ref: 1db74924805 +Node: Incremental Configuration4929782 +Ref: library/logging config incremental-configuration4929936 +Ref: 1dbb4929936 +Ref: library/logging config logging-config-dict-incremental4929936 +Ref: 1dba4929936 +Node: Object connections4931381 +Ref: library/logging config logging-config-dict-connections4931530 +Ref: 1db64931530 +Ref: library/logging config object-connections4931530 +Ref: 1dbc4931530 +Node: User-defined objects4933925 +Ref: library/logging config logging-config-dict-userdef4934075 +Ref: 1db84934075 +Ref: library/logging config user-defined-objects4934075 +Ref: 1dbd4934075 +Node: Access to external objects4937341 +Ref: library/logging config access-to-external-objects4937499 +Ref: 1dbe4937499 +Ref: library/logging config logging-config-dict-externalobj4937499 +Ref: 1dbf4937499 +Node: Access to internal objects4938798 +Ref: library/logging config access-to-internal-objects4938974 +Ref: 1dc04938974 +Ref: library/logging config logging-config-dict-internalobj4938974 +Ref: 1dc14938974 +Node: Import resolution and custom importers4942328 +Ref: library/logging config import-resolution-and-custom-importers4942469 +Ref: 1dc34942469 +Ref: library/logging config logging-import-resolution4942469 +Ref: 1dc44942469 +Node: Configuration file format4943447 +Ref: library/logging config configuration-file-format4943589 +Ref: 1dc54943589 +Ref: library/logging config logging-config-fileformat4943589 +Ref: 1db24943589 +Node: logging handlers — Logging handlers4951284 +Ref: library/logging handlers doc4951484 +Ref: 1dc64951484 +Ref: library/logging handlers logging-handlers-logging-handlers4951484 +Ref: 1dc74951484 +Ref: library/logging handlers module-logging handlers4951484 +Ref: ac4951484 +Ref: logging handlers — Logging handlers-Footnote-14952548 +Node: StreamHandler4952624 +Ref: library/logging handlers stream-handler4952731 +Ref: 1dc94952731 +Ref: library/logging handlers streamhandler4952731 +Ref: 1dca4952731 +Ref: library/logging handlers logging StreamHandler4953044 +Ref: b754953044 +Ref: library/logging handlers logging StreamHandler emit4953281 +Ref: 1dcb4953281 +Ref: library/logging handlers logging StreamHandler flush4953614 +Ref: 1dcc4953614 +Ref: library/logging handlers logging StreamHandler setStream4953907 +Ref: 3a84953907 +Node: FileHandler4954706 +Ref: library/logging handlers file-handler4954833 +Ref: 1dcd4954833 +Ref: library/logging handlers filehandler4954833 +Ref: 1dce4954833 +Ref: library/logging handlers logging FileHandler4955067 +Ref: 1dc84955067 +Ref: library/logging handlers logging FileHandler close4955717 +Ref: 1dd04955717 +Ref: library/logging handlers logging FileHandler emit4955772 +Ref: 1dcf4955772 +Node: NullHandler4955847 +Ref: library/logging handlers null-handler4955979 +Ref: 1dd14955979 +Ref: library/logging handlers nullhandler4955979 +Ref: 1dd24955979 +Ref: library/logging handlers logging NullHandler4956237 +Ref: c1c4956237 +Ref: library/logging handlers logging NullHandler emit4956336 +Ref: 1dd34956336 +Ref: library/logging handlers logging NullHandler handle4956405 +Ref: 1dd44956405 +Ref: library/logging handlers logging NullHandler createLock4956476 +Ref: 1dd54956476 +Node: WatchedFileHandler4956758 +Ref: library/logging handlers watched-file-handler4956898 +Ref: 1dd74956898 +Ref: library/logging handlers watchedfilehandler4956898 +Ref: 1dd84956898 +Ref: library/logging handlers logging handlers WatchedFileHandler4957913 +Ref: 1dd94957913 +Ref: library/logging handlers logging handlers WatchedFileHandler reopenIfNeeded4958586 +Ref: 5624958586 +Ref: library/logging handlers logging handlers WatchedFileHandler emit4958877 +Ref: 1dda4958877 +Node: BaseRotatingHandler4959045 +Ref: library/logging handlers base-rotating-handler4959193 +Ref: 1ddb4959193 +Ref: library/logging handlers baserotatinghandler4959193 +Ref: 1ddc4959193 +Ref: library/logging handlers logging handlers BaseRotatingHandler4959580 +Ref: 1ddd4959580 +Ref: library/logging handlers logging handlers BaseRotatingHandler namer4959756 +Ref: 1dde4959756 +Ref: library/logging handlers logging handlers BaseRotatingHandler rotator4960383 +Ref: 1de04960383 +Ref: library/logging handlers logging handlers BaseRotatingHandler rotation_filename4960666 +Ref: 1ddf4960666 +Ref: library/logging handlers logging handlers BaseRotatingHandler rotate4961235 +Ref: 1de14961235 +Node: RotatingFileHandler4962545 +Ref: library/logging handlers rotating-file-handler4962699 +Ref: 1de34962699 +Ref: library/logging handlers rotatingfilehandler4962699 +Ref: 1de44962699 +Ref: library/logging handlers logging handlers RotatingFileHandler4962891 +Ref: 1db94962891 +Ref: library/logging handlers logging handlers RotatingFileHandler doRollover4964693 +Ref: 1de64964693 +Ref: library/logging handlers logging handlers RotatingFileHandler emit4964773 +Ref: 1de54964773 +Node: TimedRotatingFileHandler4964905 +Ref: library/logging handlers timed-rotating-file-handler4965053 +Ref: 1de74965053 +Ref: library/logging handlers timedrotatingfilehandler4965053 +Ref: 1de84965053 +Ref: library/logging handlers logging handlers TimedRotatingFileHandler4965286 +Ref: 86e4965286 +Ref: library/logging handlers logging handlers TimedRotatingFileHandler doRollover4970266 +Ref: 1dea4970266 +Ref: library/logging handlers logging handlers TimedRotatingFileHandler emit4970346 +Ref: 1de94970346 +Node: SocketHandler4970473 +Ref: library/logging handlers socket-handler4970617 +Ref: 1deb4970617 +Ref: library/logging handlers sockethandler4970617 +Ref: 1dec4970617 +Ref: library/logging handlers logging handlers SocketHandler4970829 +Ref: 86f4970829 +Ref: library/logging handlers logging handlers SocketHandler close4971236 +Ref: 1ded4971236 +Ref: library/logging handlers logging handlers SocketHandler emit4971293 +Ref: 1dee4971293 +Ref: library/logging handlers logging handlers SocketHandler handleError4971728 +Ref: 1def4971728 +Ref: library/logging handlers logging handlers SocketHandler makeSocket4971958 +Ref: 1df04971958 +Ref: library/logging handlers logging handlers SocketHandler makePickle4972201 +Ref: 1df14972201 +Ref: library/logging handlers logging handlers SocketHandler send4973011 +Ref: 1df24973011 +Ref: library/logging handlers logging handlers SocketHandler createSocket4973332 +Ref: 1df34973332 +Node: DatagramHandler4974532 +Ref: library/logging handlers datagram-handler4974665 +Ref: 1df44974665 +Ref: library/logging handlers datagramhandler4974665 +Ref: 1df54974665 +Ref: library/logging handlers logging handlers DatagramHandler4974901 +Ref: 8704974901 +Ref: library/logging handlers logging handlers DatagramHandler emit4975312 +Ref: 1df64975312 +Ref: library/logging handlers logging handlers DatagramHandler makeSocket4975666 +Ref: 1df74975666 +Ref: library/logging handlers logging handlers DatagramHandler send4975852 +Ref: 1df84975852 +Node: SysLogHandler4976067 +Ref: library/logging handlers syslog-handler4976204 +Ref: 1df94976204 +Ref: library/logging handlers sysloghandler4976204 +Ref: 1dfa4976204 +Ref: library/logging handlers logging handlers SysLogHandler4976410 +Ref: a1e4976410 +Ref: library/logging handlers logging handlers SysLogHandler close4978073 +Ref: 1dfb4978073 +Ref: library/logging handlers logging handlers SysLogHandler emit4978149 +Ref: 1dfc4978149 +Ref: library/logging handlers logging handlers SysLogHandler encodePriority4980031 +Ref: 1dfd4980031 +Ref: library/logging handlers logging handlers SysLogHandler mapPriority4984055 +Ref: 1dfe4984055 +Ref: SysLogHandler-Footnote-14984545 +Ref: SysLogHandler-Footnote-24984588 +Ref: SysLogHandler-Footnote-34984637 +Node: NTEventLogHandler4984680 +Ref: library/logging handlers nt-eventlog-handler4984813 +Ref: 1dff4984813 +Ref: library/logging handlers nteventloghandler4984813 +Ref: 1e004984813 +Ref: library/logging handlers logging handlers NTEventLogHandler4985148 +Ref: 1e014985148 +Ref: library/logging handlers logging handlers NTEventLogHandler close4986209 +Ref: 1e024986209 +Ref: library/logging handlers logging handlers NTEventLogHandler emit4986611 +Ref: 1e034986611 +Ref: library/logging handlers logging handlers NTEventLogHandler getEventCategory4986769 +Ref: 1e044986769 +Ref: library/logging handlers logging handlers NTEventLogHandler getEventType4986971 +Ref: 1e054986971 +Ref: library/logging handlers logging handlers NTEventLogHandler getMessageID4987564 +Ref: 1e064987564 +Node: SMTPHandler4987994 +Ref: library/logging handlers smtp-handler4988127 +Ref: 1e074988127 +Ref: library/logging handlers smtphandler4988127 +Ref: 1e084988127 +Ref: library/logging handlers logging handlers SMTPHandler4988324 +Ref: 1e094988324 +Ref: library/logging handlers logging handlers SMTPHandler emit4989577 +Ref: 1e0a4989577 +Ref: library/logging handlers logging handlers SMTPHandler getSubject4989681 +Ref: 1e0b4989681 +Node: MemoryHandler4989827 +Ref: library/logging handlers memory-handler4989954 +Ref: 1e0c4989954 +Ref: library/logging handlers memoryhandler4989954 +Ref: 1e0d4989954 +Ref: library/logging handlers logging handlers BufferingHandler4990657 +Ref: 1e0e4990657 +Ref: library/logging handlers logging handlers BufferingHandler emit4990853 +Ref: 1e0f4990853 +Ref: library/logging handlers logging handlers BufferingHandler flush4991043 +Ref: 1e114991043 +Ref: library/logging handlers logging handlers BufferingHandler shouldFlush4991196 +Ref: 1e104991196 +Ref: library/logging handlers logging handlers MemoryHandler4991379 +Ref: 1dc24991379 +Ref: library/logging handlers logging handlers MemoryHandler close4992194 +Ref: 1e134992194 +Ref: library/logging handlers logging handlers MemoryHandler flush4992322 +Ref: 1e144992322 +Ref: library/logging handlers logging handlers MemoryHandler setTarget4992593 +Ref: 1e124992593 +Ref: library/logging handlers logging handlers MemoryHandler shouldFlush4992683 +Ref: 1e154992683 +Node: HTTPHandler4992809 +Ref: library/logging handlers http-handler4992937 +Ref: 1e164992937 +Ref: library/logging handlers httphandler4992937 +Ref: 1e174992937 +Ref: library/logging handlers logging handlers HTTPHandler4993168 +Ref: 7114993168 +Ref: library/logging handlers logging handlers HTTPHandler mapLogRecord4994128 +Ref: 1e184994128 +Ref: library/logging handlers logging handlers HTTPHandler emit4994584 +Ref: 1e194994584 +Node: QueueHandler4995318 +Ref: library/logging handlers queue-handler4995446 +Ref: 1e1a4995446 +Ref: library/logging handlers queuehandler4995446 +Ref: 1e1b4995446 +Ref: library/logging handlers logging handlers QueueHandler4996185 +Ref: 1e1c4996185 +Ref: library/logging handlers logging handlers QueueHandler emit4996682 +Ref: 1e1e4996682 +Ref: library/logging handlers logging handlers QueueHandler prepare4997161 +Ref: 1e1f4997161 +Ref: library/logging handlers logging handlers QueueHandler enqueue4997708 +Ref: 1e1d4997708 +Node: QueueListener4997954 +Ref: library/logging handlers queue-listener4998062 +Ref: 1e204998062 +Ref: library/logging handlers queuelistener4998062 +Ref: 1e214998062 +Ref: library/logging handlers logging handlers QueueListener4999082 +Ref: 7124999082 +Ref: library/logging handlers logging handlers QueueListener dequeue5000134 +Ref: 1e225000134 +Ref: library/logging handlers logging handlers QueueListener prepare5000415 +Ref: 1e235000415 +Ref: library/logging handlers logging handlers QueueListener handle5000732 +Ref: 1e245000732 +Ref: library/logging handlers logging handlers QueueListener start5000994 +Ref: 1e255000994 +Ref: library/logging handlers logging handlers QueueListener stop5001157 +Ref: 1e265001157 +Ref: library/logging handlers logging handlers QueueListener enqueue_sentinel5001468 +Ref: 1e275001468 +Node: getpass — Portable password input5001965 +Ref: library/getpass doc5002181 +Ref: 1e285002181 +Ref: library/getpass getpass-portable-password-input5002181 +Ref: 1e295002181 +Ref: library/getpass module-getpass5002181 +Ref: 885002181 +Ref: library/getpass getpass getpass5002431 +Ref: 1e2a5002431 +Ref: library/getpass getpass GetPassWarning5003280 +Ref: 1e2b5003280 +Ref: library/getpass getpass getuser5003408 +Ref: 1bc05003408 +Ref: getpass — Portable password input-Footnote-15003995 +Node: curses — Terminal handling for character-cell displays5004061 +Ref: library/curses doc5004296 +Ref: 1e2c5004296 +Ref: library/curses curses-terminal-handling-for-character-cell-displays5004296 +Ref: 1e2d5004296 +Ref: library/curses module-curses5004296 +Ref: 2c5004296 +Ref: curses — Terminal handling for character-cell displays-Footnote-15006520 +Node: Functions<3>5006583 +Ref: library/curses curses-functions5006711 +Ref: 1e2f5006711 +Ref: library/curses functions5006711 +Ref: 1e305006711 +Ref: library/curses curses error5006815 +Ref: 1e315006815 +Ref: library/curses curses baudrate5007177 +Ref: 1e325007177 +Ref: library/curses curses beep5007529 +Ref: 1e335007529 +Ref: library/curses curses can_change_color5007595 +Ref: 1e345007595 +Ref: library/curses curses cbreak5007767 +Ref: 1e355007767 +Ref: library/curses curses color_content5008245 +Ref: 1e375008245 +Ref: library/curses curses color_pair5008628 +Ref: 1e385008628 +Ref: library/curses curses curs_set5009003 +Ref: 1e3a5009003 +Ref: library/curses curses def_prog_mode5009431 +Ref: 1e3b5009431 +Ref: library/curses curses def_shell_mode5009759 +Ref: 1e3d5009759 +Ref: library/curses curses delay_output5010105 +Ref: 1e3f5010105 +Ref: library/curses curses doupdate5010195 +Ref: 1e405010195 +Ref: library/curses curses echo5011020 +Ref: 1e445011020 +Ref: library/curses curses endwin5011156 +Ref: 1e455011156 +Ref: library/curses curses erasechar5011259 +Ref: 1e465011259 +Ref: library/curses curses filter5011527 +Ref: 1e475011527 +Ref: library/curses curses flash5012138 +Ref: 1e495012138 +Ref: library/curses curses flushinp5012401 +Ref: 1e4a5012401 +Ref: library/curses curses getmouse5012587 +Ref: 1e4b5012587 +Ref: library/curses curses getsyx5013393 +Ref: 1e4d5013393 +Ref: library/curses curses getwin5013604 +Ref: 1e4f5013604 +Ref: library/curses curses has_colors5013840 +Ref: 1e505013840 +Ref: library/curses curses has_ic5013972 +Ref: 1e515013972 +Ref: library/curses curses has_il5014228 +Ref: 1e525014228 +Ref: library/curses curses has_key5014525 +Ref: 1e535014525 +Ref: library/curses curses halfdelay5014679 +Ref: 1e545014679 +Ref: library/curses curses init_color5015117 +Ref: 1e565015117 +Ref: library/curses curses init_pair5015774 +Ref: 1e575015774 +Ref: library/curses curses initscr5016495 +Ref: 1e485016495 +Ref: library/curses curses is_term_resized5016770 +Ref: 1e5a5016770 +Ref: library/curses curses isendwin5016941 +Ref: 1e5c5016941 +Ref: library/curses curses keyname5017099 +Ref: 1e5d5017099 +Ref: library/curses curses killchar5017630 +Ref: 1e5e5017630 +Ref: library/curses curses longname5017901 +Ref: 1e5f5017901 +Ref: library/curses curses meta5018176 +Ref: 1e605018176 +Ref: library/curses curses mouseinterval5018333 +Ref: 1e615018333 +Ref: library/curses curses mousemask5018632 +Ref: 1e625018632 +Ref: library/curses curses napms5019052 +Ref: 1e635019052 +Ref: library/curses curses newpad5019120 +Ref: 1e645019120 +Ref: library/curses curses newwin5020208 +Ref: 1e655020208 +Ref: library/curses curses nl5020575 +Ref: 1e665020575 +Ref: library/curses curses nocbreak5020790 +Ref: 1e555020790 +Ref: library/curses curses noecho5020911 +Ref: 1e675020911 +Ref: library/curses curses nonl5021010 +Ref: 1e685021010 +Ref: library/curses curses noqiflush5021503 +Ref: 1e695021503 +Ref: library/curses curses noraw5021893 +Ref: 1e6a5021893 +Ref: library/curses curses pair_content5022008 +Ref: 1e6b5022008 +Ref: library/curses curses pair_number5022234 +Ref: 1e395022234 +Ref: library/curses curses putp5022426 +Ref: 1e6c5022426 +Ref: library/curses curses qiflush5022672 +Ref: 1e6d5022672 +Ref: library/curses curses raw5022938 +Ref: 1e365022938 +Ref: library/curses curses reset_prog_mode5023180 +Ref: 1e3c5023180 +Ref: library/curses curses reset_shell_mode5023328 +Ref: 1e3e5023328 +Ref: library/curses curses resetty5023476 +Ref: 1e6e5023476 +Ref: library/curses curses resize_term5023618 +Ref: 1e5b5023618 +Ref: library/curses curses resizeterm5024157 +Ref: 1e705024157 +Ref: library/curses curses savetty5024430 +Ref: 1e6f5024430 +Ref: library/curses curses setsyx5024565 +Ref: 1e715024565 +Ref: library/curses curses setupterm5024735 +Ref: 1e725024735 +Ref: library/curses curses start_color5025152 +Ref: 1e735025152 +Ref: library/curses curses termattrs5025832 +Ref: 1e745025832 +Ref: library/curses curses termname5026062 +Ref: 1e755026062 +Ref: library/curses curses tigetflag5026211 +Ref: 1e765026211 +Ref: library/curses curses tigetnum5026527 +Ref: 1e775026527 +Ref: library/curses curses tigetstr5026843 +Ref: 1e785026843 +Ref: library/curses curses tparm5027153 +Ref: 1e795027153 +Ref: library/curses curses typeahead5027490 +Ref: 1e7a5027490 +Ref: library/curses curses unctrl5028083 +Ref: 1e7b5028083 +Ref: library/curses curses ungetch5028364 +Ref: 1e7c5028364 +Ref: library/curses curses update_lines_cols5028541 +Ref: 6c65028541 +Ref: library/curses curses unget_wch5028700 +Ref: 9e05028700 +Ref: library/curses curses ungetmouse5028918 +Ref: 1e7d5028918 +Ref: library/curses curses use_env5029077 +Ref: 1e7e5029077 +Ref: library/curses curses use_default_colors5029587 +Ref: 1e585029587 +Ref: library/curses curses wrapper5030020 +Ref: 2a15030020 +Node: Window Objects5030831 +Ref: library/curses curses-window-objects5030980 +Ref: 1e595030980 +Ref: library/curses window-objects5030980 +Ref: 1e7f5030980 +Ref: library/curses curses window addch5031162 +Ref: 1e805031162 +Ref: library/curses curses window addnstr5031765 +Ref: 1e815031765 +Ref: library/curses curses window addstr5032024 +Ref: 1e425032024 +Ref: library/curses curses window attroff5033024 +Ref: 1e825033024 +Ref: library/curses curses window attron5033169 +Ref: 1e835033169 +Ref: library/curses curses window attrset5033310 +Ref: 1e845033310 +Ref: library/curses curses window bkgd5033458 +Ref: 1e855033458 +Ref: library/curses curses window bkgdset5033903 +Ref: 1e865033903 +Ref: library/curses curses window border5034469 +Ref: 1e875034469 +Ref: library/curses curses window box5036421 +Ref: 1e885036421 +Ref: library/curses curses window chgat5036649 +Ref: 1e895036649 +Ref: library/curses curses window clear5037273 +Ref: 1e8b5037273 +Ref: library/curses curses window clearok5037428 +Ref: 1e8d5037428 +Ref: library/curses curses window clrtobot5037573 +Ref: 1e8e5037573 +Ref: library/curses curses window clrtoeol5037770 +Ref: 1e8f5037770 +Ref: library/curses curses window cursyncup5037850 +Ref: 1e905037850 +Ref: library/curses curses window delch5038018 +Ref: 1e915038018 +Ref: library/curses curses window deleteln5038097 +Ref: 1e925038097 +Ref: library/curses curses window derwin5038221 +Ref: 1e935038221 +Ref: library/curses curses window echochar5038631 +Ref: 1e955038631 +Ref: library/curses curses window enclose5038785 +Ref: 1e965038785 +Ref: library/curses curses window encoding5039089 +Ref: 9de5039089 +Ref: library/curses curses window erase5039473 +Ref: 1e8c5039473 +Ref: library/curses curses window getbegyx5039526 +Ref: 1e975039526 +Ref: library/curses curses window getbkgd5039630 +Ref: 1e985039630 +Ref: library/curses curses window getch5039745 +Ref: 1e4c5039745 +Ref: library/curses curses window get_wch5040065 +Ref: 9df5040065 +Ref: library/curses curses window getkey5040331 +Ref: 1e995040331 +Ref: library/curses curses window getmaxyx5040639 +Ref: 1e9a5040639 +Ref: library/curses curses window getparyx5040744 +Ref: 1e9b5040744 +Ref: library/curses curses window getstr5040948 +Ref: 1e9c5040948 +Ref: library/curses curses window getyx5041164 +Ref: 1e9d5041164 +Ref: library/curses curses window hline5041305 +Ref: 1e9e5041305 +Ref: library/curses curses window idcok5041491 +Ref: 1e9f5041491 +Ref: library/curses curses window idlok5041837 +Ref: 1ea05041837 +Ref: library/curses curses window immedok5042030 +Ref: 1ea15042030 +Ref: library/curses curses window inch5042379 +Ref: 1ea25042379 +Ref: library/curses curses window insch5042567 +Ref: 1ea35042567 +Ref: library/curses curses window insdelln5042781 +Ref: 1ea45042781 +Ref: library/curses curses window insertln5043160 +Ref: 1ea55043160 +Ref: library/curses curses window insnstr5043290 +Ref: 1ea65043290 +Ref: library/curses curses window insstr5043803 +Ref: 1ea75043803 +Ref: library/curses curses window instr5044222 +Ref: 1ea85044222 +Ref: library/curses curses window is_linetouched5044618 +Ref: 1ea95044618 +Ref: library/curses curses window is_wintouched5044901 +Ref: 1eaa5044901 +Ref: library/curses curses window keypad5045081 +Ref: 1eab5045081 +Ref: library/curses curses window leaveok5045349 +Ref: 1e4e5045349 +Ref: library/curses curses window move5045700 +Ref: 1eac5045700 +Ref: library/curses curses window mvderwin5045783 +Ref: 1ead5045783 +Ref: library/curses curses window mvwin5046060 +Ref: 1eae5046060 +Ref: library/curses curses window nodelay5046176 +Ref: 1eaf5046176 +Ref: library/curses curses window notimeout5046285 +Ref: 1eb05046285 +Ref: library/curses curses window noutrefresh5046549 +Ref: 1e415046549 +Ref: library/curses curses window overlay5046823 +Ref: 1eb15046823 +Ref: library/curses curses window overwrite5047472 +Ref: 1eb25047472 +Ref: library/curses curses window putwin5048124 +Ref: 1eb35048124 +Ref: library/curses curses window redrawln5048330 +Ref: 1eb45048330 +Ref: library/curses curses window redrawwin5048536 +Ref: 1eb55048536 +Ref: library/curses curses window refresh5048682 +Ref: 1e435048682 +Ref: library/curses curses window resize5049703 +Ref: 1eb65049703 +Ref: library/curses curses window scroll5050055 +Ref: 1eb75050055 +Ref: library/curses curses window scrollok5050163 +Ref: 1eb85050163 +Ref: library/curses curses window setscrreg5050699 +Ref: 1eb95050699 +Ref: library/curses curses window standend5050868 +Ref: 1eba5050868 +Ref: library/curses curses window standout5051019 +Ref: 1ebb5051019 +Ref: library/curses curses window subpad5051089 +Ref: 1ebc5051089 +Ref: library/curses curses window subwin5051328 +Ref: 1e945051328 +Ref: library/curses curses window syncdown5051686 +Ref: 1ebd5051686 +Ref: library/curses curses window syncok5051932 +Ref: 1ebe5051932 +Ref: library/curses curses window syncup5052095 +Ref: 1ebf5052095 +Ref: library/curses curses window timeout5052221 +Ref: 1ec05052221 +Ref: library/curses curses window touchline5052724 +Ref: 1e8a5052724 +Ref: library/curses curses window touchwin5053032 +Ref: 1ec15053032 +Ref: library/curses curses window untouchwin5053157 +Ref: 1ec25053157 +Ref: library/curses curses window vline5053290 +Ref: 1ec35053290 +Ref: Window Objects-Footnote-15053510 +Node: Constants<6>5053553 +Ref: library/curses constants5053681 +Ref: 1ec45053681 +Ref: library/curses curses ERR5053788 +Ref: 1ec55053788 +Ref: library/curses curses OK5053933 +Ref: 1ec65053933 +Ref: library/curses curses version5054076 +Ref: 1ec75054076 +Ref: library/curses curses ncurses_version5054213 +Ref: 1bf5054213 +Node: curses textpad — Text input widget for curses programs5072772 +Ref: library/curses curses-textpad-text-input-widget-for-curses-programs5073019 +Ref: 1ec85073019 +Ref: library/curses module-curses textpad5073019 +Ref: 2f5073019 +Ref: library/curses curses textpad rectangle5073609 +Ref: 1eca5073609 +Node: Textbox objects5074332 +Ref: library/curses curses-textpad-objects5074440 +Ref: 1ecb5074440 +Ref: library/curses textbox-objects5074440 +Ref: 1ecc5074440 +Ref: library/curses curses textpad Textbox5074556 +Ref: 1ec95074556 +Ref: library/curses curses textpad Textbox edit5075035 +Ref: 1ece5075035 +Ref: library/curses curses textpad Textbox do_command5075606 +Ref: 1ecf5075606 +Ref: library/curses curses textpad Textbox gather5079130 +Ref: 1ed05079130 +Ref: library/curses curses textpad Textbox stripspaces5079323 +Ref: 1ecd5079323 +Node: curses ascii — Utilities for ASCII characters5079734 +Ref: library/curses ascii doc5079976 +Ref: 1ed15079976 +Ref: library/curses ascii curses-ascii-utilities-for-ascii-characters5079976 +Ref: 1ed25079976 +Ref: library/curses ascii module-curses ascii5079976 +Ref: 2d5079976 +Ref: library/curses ascii curses ascii isalnum5083658 +Ref: 1ed35083658 +Ref: library/curses ascii curses ascii isalpha5083805 +Ref: 1ed45083805 +Ref: library/curses ascii curses ascii isascii5083950 +Ref: 1ed55083950 +Ref: library/curses ascii curses ascii isblank5084059 +Ref: 1ed65084059 +Ref: library/curses ascii curses ascii iscntrl5084172 +Ref: 1ed75084172 +Ref: library/curses ascii curses ascii isdigit5084298 +Ref: 1ed85084298 +Ref: library/curses ascii curses ascii isgraph5084462 +Ref: 1ed95084462 +Ref: library/curses ascii curses ascii islower5084563 +Ref: 1eda5084563 +Ref: library/curses ascii curses ascii isprint5084651 +Ref: 1edb5084651 +Ref: library/curses ascii curses ascii ispunct5084755 +Ref: 1edc5084755 +Ref: library/curses ascii curses ascii isspace5084898 +Ref: 1edd5084898 +Ref: library/curses ascii curses ascii isupper5085066 +Ref: 1ede5085066 +Ref: library/curses ascii curses ascii isxdigit5085150 +Ref: 1edf5085150 +Ref: library/curses ascii curses ascii isctrl5085293 +Ref: 1ee05085293 +Ref: library/curses ascii curses ascii ismeta5085402 +Ref: 1ee15085402 +Ref: library/curses ascii curses ascii ascii5086002 +Ref: 1ee25086002 +Ref: library/curses ascii curses ascii ctrl5086109 +Ref: 1ee35086109 +Ref: library/curses ascii curses ascii alt5086277 +Ref: 1ee45086277 +Ref: library/curses ascii curses ascii unctrl5086549 +Ref: 1ee55086549 +Ref: library/curses ascii curses ascii controlnames5087104 +Ref: 1ee65087104 +Node: curses panel — A panel stack extension for curses5087350 +Ref: library/curses panel doc5087599 +Ref: 1ee75087599 +Ref: library/curses panel curses-panel-a-panel-stack-extension-for-curses5087599 +Ref: 1ee85087599 +Ref: library/curses panel module-curses panel5087599 +Ref: 2e5087599 +Node: Functions<4>5088077 +Ref: library/curses panel cursespanel-functions5088199 +Ref: 1ee95088199 +Ref: library/curses panel functions5088199 +Ref: 1eea5088199 +Ref: library/curses panel curses panel bottom_panel5088309 +Ref: 1eeb5088309 +Ref: library/curses panel curses panel new_panel5088404 +Ref: 1eec5088404 +Ref: library/curses panel curses panel top_panel5088706 +Ref: 1eed5088706 +Ref: library/curses panel curses panel update_panels5088795 +Ref: 1eee5088795 +Node: Panel Objects5089006 +Ref: library/curses panel curses-panel-objects5089128 +Ref: 1eef5089128 +Ref: library/curses panel panel-objects5089128 +Ref: 1ef05089128 +Ref: library/curses panel curses panel Panel above5089484 +Ref: 1ef15089484 +Ref: library/curses panel curses panel Panel below5089561 +Ref: 1ef25089561 +Ref: library/curses panel curses panel Panel bottom5089638 +Ref: 1ef35089638 +Ref: library/curses panel curses panel Panel hidden5089716 +Ref: 1ef45089716 +Ref: library/curses panel curses panel Panel hide5089836 +Ref: 1ef55089836 +Ref: library/curses panel curses panel Panel move5089970 +Ref: 1ef65089970 +Ref: library/curses panel curses panel Panel replace5090062 +Ref: 1ef75090062 +Ref: library/curses panel curses panel Panel set_userptr5090166 +Ref: 1ef85090166 +Ref: library/curses panel curses panel Panel show5090360 +Ref: 1ef95090360 +Ref: library/curses panel curses panel Panel top5090443 +Ref: 1efa5090443 +Ref: library/curses panel curses panel Panel userptr5090511 +Ref: 1efb5090511 +Ref: library/curses panel curses panel Panel window5090626 +Ref: 1efc5090626 +Node: platform — Access to underlying platform’s identifying data5090714 +Ref: library/platform doc5090955 +Ref: 1efd5090955 +Ref: library/platform module-platform5090955 +Ref: ce5090955 +Ref: library/platform platform-access-to-underlying-platform-s-identifying-data5090955 +Ref: 1efe5090955 +Ref: platform — Access to underlying platform’s identifying data-Footnote-15091450 +Node: Cross Platform5091517 +Ref: library/platform cross-platform5091653 +Ref: 1eff5091653 +Ref: library/platform platform architecture5091704 +Ref: 1f005091704 +Ref: library/platform platform machine5093045 +Ref: 1f015093045 +Ref: library/platform platform node5093201 +Ref: 1f025093201 +Ref: library/platform platform platform5093376 +Ref: 1f035093376 +Ref: library/platform platform processor5094311 +Ref: 1f065094311 +Ref: library/platform platform python_build5094632 +Ref: 1f075094632 +Ref: library/platform platform python_compiler5094779 +Ref: 1f085094779 +Ref: library/platform platform python_branch5094902 +Ref: 1f095094902 +Ref: library/platform platform python_implementation5095016 +Ref: 1f0a5095016 +Ref: library/platform platform python_revision5095220 +Ref: 1f0b5095220 +Ref: library/platform platform python_version5095343 +Ref: 1f0c5095343 +Ref: library/platform platform python_version_tuple5095592 +Ref: 1f0d5095592 +Ref: library/platform platform release5095872 +Ref: 1f0e5095872 +Ref: library/platform platform system5096047 +Ref: 1f0f5096047 +Ref: library/platform platform system_alias5096259 +Ref: 1f045096259 +Ref: library/platform platform version5096542 +Ref: 1f105096542 +Ref: library/platform platform uname5096719 +Ref: 1f115096719 +Node: Java Platform5097412 +Ref: library/platform java-platform5097573 +Ref: 1f125097573 +Ref: library/platform platform java_ver5097622 +Ref: 1f135097622 +Node: Windows Platform5098093 +Ref: library/platform windows-platform5098255 +Ref: 1f145098255 +Ref: library/platform platform win32_ver5098310 +Ref: 1f155098310 +Ref: library/platform platform win32_edition5099009 +Ref: 1f165099009 +Ref: library/platform platform win32_is_iot5099282 +Ref: 1f175099282 +Node: Mac OS Platform5099474 +Ref: library/platform mac-os-platform5099637 +Ref: 1f185099637 +Ref: library/platform platform mac_ver5099690 +Ref: 1f055099690 +Node: Unix Platforms5100072 +Ref: library/platform unix-platforms5100210 +Ref: 1f195100210 +Ref: library/platform platform libc_ver5100261 +Ref: 1f1a5100261 +Node: errno — Standard errno system symbols5100878 +Ref: library/errno doc5101116 +Ref: 1f1b5101116 +Ref: library/errno errno-standard-errno-system-symbols5101116 +Ref: 1f1c5101116 +Ref: library/errno module-errno5101116 +Ref: 7c5101116 +Ref: library/errno errno errorcode5101528 +Ref: 1f1d5101528 +Ref: library/errno errno EPERM5102049 +Ref: 1f1e5102049 +Ref: library/errno errno ENOENT5102102 +Ref: 1f1f5102102 +Ref: library/errno errno ESRCH5102158 +Ref: 1f205102158 +Ref: library/errno errno EINTR5102203 +Ref: 6585102203 +Ref: library/errno errno EIO5102359 +Ref: 1f215102359 +Ref: library/errno errno ENXIO5102396 +Ref: 1f225102396 +Ref: library/errno errno E2BIG5102451 +Ref: 1f235102451 +Ref: library/errno errno ENOEXEC5102498 +Ref: 1f245102498 +Ref: library/errno errno EBADF5102547 +Ref: 1f255102547 +Ref: library/errno errno ECHILD5102592 +Ref: 1f265102592 +Ref: library/errno errno EAGAIN5102641 +Ref: 1c0c5102641 +Ref: library/errno errno ENOMEM5102681 +Ref: 1f275102681 +Ref: library/errno errno EACCES5102725 +Ref: 1f285102725 +Ref: library/errno errno EFAULT5102773 +Ref: 1f295102773 +Ref: library/errno errno ENOTBLK5102815 +Ref: 1f2a5102815 +Ref: library/errno errno EBUSY5102868 +Ref: 1f2b5102868 +Ref: library/errno errno EEXIST5102921 +Ref: 1f2c5102921 +Ref: library/errno errno EXDEV5102963 +Ref: 1be05102963 +Ref: library/errno errno ENODEV5103010 +Ref: 1f2d5103010 +Ref: library/errno errno ENOTDIR5103055 +Ref: 1f2e5103055 +Ref: library/errno errno EISDIR5103102 +Ref: 1f2f5103102 +Ref: library/errno errno EINVAL5103147 +Ref: 1be45103147 +Ref: library/errno errno ENFILE5103194 +Ref: 1f305103194 +Ref: library/errno errno EMFILE5103244 +Ref: 1f315103244 +Ref: library/errno errno ENOTTY5103294 +Ref: 1f325103294 +Ref: library/errno errno ETXTBSY5103341 +Ref: 1f335103341 +Ref: library/errno errno EFBIG5103387 +Ref: 1f345103387 +Ref: library/errno errno ENOSPC5103431 +Ref: 1f355103431 +Ref: library/errno errno ESPIPE5103485 +Ref: 1f365103485 +Ref: library/errno errno EROFS5103528 +Ref: 1f375103528 +Ref: library/errno errno EMLINK5103579 +Ref: 1f385103579 +Ref: library/errno errno EPIPE5103624 +Ref: 1f395103624 +Ref: library/errno errno EDOM5103665 +Ref: 1f3a5103665 +Ref: library/errno errno ERANGE5103729 +Ref: 1f3b5103729 +Ref: library/errno errno EDEADLK5103789 +Ref: 1f3c5103789 +Ref: library/errno errno ENAMETOOLONG5103850 +Ref: 1f3d5103850 +Ref: library/errno errno ENOLCK5103905 +Ref: 1f3e5103905 +Ref: library/errno errno ENOSYS5103961 +Ref: 1f3f5103961 +Ref: library/errno errno ENOTEMPTY5104016 +Ref: 1f405104016 +Ref: library/errno errno ELOOP5104069 +Ref: 1f415104069 +Ref: library/errno errno EWOULDBLOCK5104134 +Ref: 1f425104134 +Ref: library/errno errno ENOMSG5104191 +Ref: 1f435104191 +Ref: library/errno errno EIDRM5104248 +Ref: 1f445104248 +Ref: library/errno errno ECHRNG5104296 +Ref: 1f455104296 +Ref: library/errno errno EL2NSYNC5104354 +Ref: 1f465104354 +Ref: library/errno errno EL3HLT5104411 +Ref: 1f475104411 +Ref: library/errno errno EL3RST5104456 +Ref: 1f485104456 +Ref: library/errno errno ELNRNG5104500 +Ref: 1f495104500 +Ref: library/errno errno EUNATCH5104555 +Ref: 1f4a5104555 +Ref: library/errno errno ENOCSI5104615 +Ref: 1f4b5104615 +Ref: library/errno errno EL2HLT5104672 +Ref: 1f4c5104672 +Ref: library/errno errno EBADE5104717 +Ref: 1f4d5104717 +Ref: library/errno errno EBADR5104763 +Ref: 1f4e5104763 +Ref: library/errno errno EXFULL5104819 +Ref: 1f4f5104819 +Ref: library/errno errno ENOANO5104863 +Ref: 1f505104863 +Ref: library/errno errno EBADRQC5104902 +Ref: 1f515104902 +Ref: library/errno errno EBADSLT5104954 +Ref: 1f525104954 +Ref: library/errno errno EDEADLOCK5104998 +Ref: 1f535104998 +Ref: library/errno errno EBFONT5105059 +Ref: 1f545105059 +Ref: library/errno errno ENOSTR5105110 +Ref: 1f555105110 +Ref: library/errno errno ENODATA5105160 +Ref: 1f565105160 +Ref: library/errno errno ETIME5105209 +Ref: 1f575105209 +Ref: library/errno errno ENOSR5105252 +Ref: 1f585105252 +Ref: library/errno errno ENONET5105306 +Ref: 1f595105306 +Ref: library/errno errno ENOPKG5105366 +Ref: 1f5a5105366 +Ref: library/errno errno EREMOTE5105418 +Ref: 1f5b5105418 +Ref: library/errno errno ENOLINK5105466 +Ref: 1f5c5105466 +Ref: library/errno errno EADV5105519 +Ref: 1f5d5105519 +Ref: library/errno errno ESRMNT5105563 +Ref: 1f5e5105563 +Ref: library/errno errno ECOMM5105607 +Ref: 1f5f5105607 +Ref: library/errno errno EPROTO5105664 +Ref: 1f605105664 +Ref: library/errno errno EMULTIHOP5105709 +Ref: 1f615105709 +Ref: library/errno errno EDOTDOT5105761 +Ref: 1f625105761 +Ref: library/errno errno EBADMSG5105811 +Ref: 1f635105811 +Ref: library/errno errno EOVERFLOW5105861 +Ref: 1f645105861 +Ref: library/errno errno ENOTUNIQ5105932 +Ref: 1f655105932 +Ref: library/errno errno EBADFD5105991 +Ref: 1f665105991 +Ref: library/errno errno EREMCHG5106050 +Ref: 1f675106050 +Ref: library/errno errno ELIBACC5106104 +Ref: 1f685106104 +Ref: library/errno errno ELIBBAD5106174 +Ref: 1f695106174 +Ref: library/errno errno ELIBSCN5106242 +Ref: 1f6a5106242 +Ref: library/errno errno ELIBMAX5106305 +Ref: 1f6b5106305 +Ref: library/errno errno ELIBEXEC5106384 +Ref: 1f6c5106384 +Ref: library/errno errno EILSEQ5106454 +Ref: 1f6d5106454 +Ref: library/errno errno ERESTART5106506 +Ref: 1f6e5106506 +Ref: library/errno errno ESTRPIPE5106582 +Ref: 1f6f5106582 +Ref: library/errno errno EUSERS5106633 +Ref: 1f705106633 +Ref: library/errno errno ENOTSOCK5106678 +Ref: 1f715106678 +Ref: library/errno errno EDESTADDRREQ5106741 +Ref: 1f725106741 +Ref: library/errno errno EMSGSIZE5106806 +Ref: 1f735106806 +Ref: library/errno errno EPROTOTYPE5106855 +Ref: 1f745106855 +Ref: library/errno errno ENOPROTOOPT5106920 +Ref: 1f755106920 +Ref: library/errno errno EPROTONOSUPPORT5106978 +Ref: 1f765106978 +Ref: library/errno errno ESOCKTNOSUPPORT5107040 +Ref: 1f775107040 +Ref: library/errno errno EOPNOTSUPP5107105 +Ref: 1f785107105 +Ref: library/errno errno EPFNOSUPPORT5107185 +Ref: 1f795107185 +Ref: library/errno errno EAFNOSUPPORT5107251 +Ref: 1f7a5107251 +Ref: library/errno errno EADDRINUSE5107328 +Ref: 1f7b5107328 +Ref: library/errno errno EADDRNOTAVAIL5107385 +Ref: 1f7c5107385 +Ref: library/errno errno ENETDOWN5107454 +Ref: 1f7d5107454 +Ref: library/errno errno ENETUNREACH5107502 +Ref: 1f7e5107502 +Ref: library/errno errno ENETRESET5107560 +Ref: 1f7f5107560 +Ref: library/errno errno ECONNABORTED5107637 +Ref: 1f805107637 +Ref: library/errno errno ECONNRESET5107706 +Ref: 1f815107706 +Ref: library/errno errno ENOBUFS5107765 +Ref: 1f825107765 +Ref: library/errno errno EISCONN5107822 +Ref: 1f835107822 +Ref: library/errno errno ENOTCONN5107893 +Ref: 1f845107893 +Ref: library/errno errno ESHUTDOWN5107961 +Ref: 1f855107961 +Ref: library/errno errno ETOOMANYREFS5108040 +Ref: 1f865108040 +Ref: library/errno errno ETIMEDOUT5108111 +Ref: 1f875108111 +Ref: library/errno errno ECONNREFUSED5108165 +Ref: 1f885108165 +Ref: library/errno errno EHOSTDOWN5108220 +Ref: 1f895108220 +Ref: library/errno errno EHOSTUNREACH5108266 +Ref: 1f8a5108266 +Ref: library/errno errno EALREADY5108319 +Ref: 1f8b5108319 +Ref: library/errno errno EINPROGRESS5108381 +Ref: 1f8c5108381 +Ref: library/errno errno ESTALE5108442 +Ref: 1f8d5108442 +Ref: library/errno errno EUCLEAN5108494 +Ref: 1f8e5108494 +Ref: library/errno errno ENOTNAM5108550 +Ref: 1f8f5108550 +Ref: library/errno errno ENAVAIL5108609 +Ref: 1f905108609 +Ref: library/errno errno EISNAM5108670 +Ref: 1f915108670 +Ref: library/errno errno EREMOTEIO5108721 +Ref: 1f925108721 +Ref: library/errno errno EDQUOT5108771 +Ref: 1f935108771 +Node: ctypes — A foreign function library for Python5108816 +Ref: library/ctypes doc5108982 +Ref: 1f945108982 +Ref: library/ctypes ctypes-a-foreign-function-library-for-python5108982 +Ref: 1f955108982 +Ref: library/ctypes module-ctypes5108982 +Ref: 2b5108982 +Node: ctypes tutorial5109435 +Ref: library/ctypes ctypes-ctypes-tutorial5109560 +Ref: 1f965109560 +Ref: library/ctypes ctypes-tutorial5109560 +Ref: 1f975109560 +Node: Loading dynamic link libraries5110929 +Ref: library/ctypes ctypes-loading-dynamic-link-libraries5111056 +Ref: 1f9a5111056 +Ref: library/ctypes loading-dynamic-link-libraries5111056 +Ref: 1f9b5111056 +Node: Accessing functions from loaded dlls5113102 +Ref: library/ctypes accessing-functions-from-loaded-dlls5113255 +Ref: 1f9c5113255 +Ref: library/ctypes ctypes-accessing-functions-from-loaded-dlls5113255 +Ref: 1f9d5113255 +Node: Calling functions5115510 +Ref: library/ctypes calling-functions5115655 +Ref: 1f9e5115655 +Ref: library/ctypes ctypes-calling-functions5115655 +Ref: 1f9f5115655 +Node: Fundamental data types5118127 +Ref: library/ctypes ctypes-fundamental-data-types5118263 +Ref: 1fa05118263 +Ref: library/ctypes fundamental-data-types5118263 +Ref: 1fa15118263 +Node: Calling functions continued5126309 +Ref: library/ctypes calling-functions-continued5126477 +Ref: 1fb65126477 +Ref: library/ctypes ctypes-calling-functions-continued5126477 +Ref: 1fb75126477 +Node: Calling functions with your own custom data types5127561 +Ref: library/ctypes calling-functions-with-your-own-custom-data-types5127765 +Ref: 1fb85127765 +Ref: library/ctypes ctypes-calling-functions-with-own-custom-data-types5127765 +Ref: 1fb95127765 +Node: Specifying the required argument types function prototypes5128608 +Ref: library/ctypes ctypes-specifying-required-argument-types5128797 +Ref: 1fba5128797 +Ref: library/ctypes specifying-the-required-argument-types-function-prototypes5128797 +Ref: 1fbb5128797 +Node: Return types5130623 +Ref: library/ctypes ctypes-return-types5130814 +Ref: 1fbc5130814 +Ref: library/ctypes return-types5130814 +Ref: 1fbd5130814 +Node: Passing pointers or passing parameters by reference5133443 +Ref: library/ctypes ctypes-passing-pointers5133597 +Ref: 1fbf5133597 +Ref: library/ctypes passing-pointers-or-passing-parameters-by-reference5133597 +Ref: 1fc05133597 +Node: Structures and unions5134691 +Ref: library/ctypes ctypes-structures-unions5134873 +Ref: 1fc35134873 +Ref: library/ctypes structures-and-unions5134873 +Ref: 1fc45134873 +Ref: library/ctypes ctypes-structureunion-alignment-byte-order5136959 +Ref: 1fc75136959 +Node: Structure/union alignment and byte order5137287 +Ref: library/ctypes structure-union-alignment-and-byte-order5137453 +Ref: 1fc85137453 +Node: Bit fields in structures and unions5138233 +Ref: library/ctypes bit-fields-in-structures-and-unions5138384 +Ref: 1fcb5138384 +Ref: library/ctypes ctypes-bit-fields-in-structures-unions5138384 +Ref: 1fcc5138384 +Node: Arrays5138971 +Ref: library/ctypes arrays5139090 +Ref: 1fcd5139090 +Ref: library/ctypes ctypes-arrays5139090 +Ref: 1fce5139090 +Node: Pointers5140379 +Ref: library/ctypes ctypes-pointers5140479 +Ref: 1fcf5140479 +Ref: library/ctypes pointers5140479 +Ref: 1fd05140479 +Node: Type conversions5143142 +Ref: library/ctypes ctypes-type-conversions5143252 +Ref: 1fd35143252 +Ref: library/ctypes type-conversions5143252 +Ref: 1fd45143252 +Node: Incomplete Types5145854 +Ref: library/ctypes ctypes-incomplete-types5145974 +Ref: 1fd65145974 +Ref: library/ctypes incomplete-types5145974 +Ref: 1fd75145974 +Node: Callback functions5147581 +Ref: library/ctypes callback-functions5147720 +Ref: 1fd85147720 +Ref: library/ctypes ctypes-callback-functions5147720 +Ref: 1fd95147720 +Node: Accessing values exported from dlls5151594 +Ref: library/ctypes accessing-values-exported-from-dlls5151726 +Ref: 1fdd5151726 +Ref: library/ctypes ctypes-accessing-values-exported-from-dlls5151726 +Ref: 1fde5151726 +Node: Surprises5154529 +Ref: library/ctypes ctypes-surprises5154668 +Ref: 1fe15154668 +Ref: library/ctypes surprises5154668 +Ref: 1fe25154668 +Node: Variable-sized data types5156843 +Ref: library/ctypes ctypes-variable-sized-data-types5156938 +Ref: 1fe35156938 +Ref: library/ctypes variable-sized-data-types5156938 +Ref: 1fe45156938 +Node: ctypes reference5158341 +Ref: library/ctypes ctypes-ctypes-reference5158466 +Ref: 1fe65158466 +Ref: library/ctypes ctypes-reference5158466 +Ref: 1fe75158466 +Node: Finding shared libraries5158777 +Ref: library/ctypes ctypes-finding-shared-libraries5158887 +Ref: 1fe85158887 +Ref: library/ctypes finding-shared-libraries5158887 +Ref: 1fe95158887 +Node: Loading shared libraries5161412 +Ref: library/ctypes ctypes-loading-shared-libraries5161548 +Ref: 1fea5161548 +Ref: library/ctypes loading-shared-libraries5161548 +Ref: 1feb5161548 +Ref: library/ctypes ctypes CDLL5161751 +Ref: 1c15161751 +Ref: library/ctypes ctypes OleDLL5162788 +Ref: 1fec5162788 +Ref: library/ctypes ctypes WinDLL5163478 +Ref: 1fee5163478 +Ref: library/ctypes ctypes PyDLL5164149 +Ref: 1fef5164149 +Ref: library/ctypes ctypes PyDLL _handle5167872 +Ref: 1ff45167872 +Ref: library/ctypes ctypes PyDLL _name5167954 +Ref: 1ff55167954 +Ref: library/ctypes ctypes LibraryLoader5168296 +Ref: 1ff65168296 +Ref: library/ctypes ctypes LibraryLoader LoadLibrary5168761 +Ref: 1ff75168761 +Ref: Loading shared libraries-Footnote-15170336 +Node: Foreign functions5170402 +Ref: library/ctypes ctypes-foreign-functions5170533 +Ref: 1ff85170533 +Ref: library/ctypes foreign-functions5170533 +Ref: 1ff95170533 +Ref: library/ctypes ctypes _FuncPtr5170947 +Ref: 1ffa5170947 +Ref: library/ctypes ctypes _FuncPtr restype5171248 +Ref: 1ffb5171248 +Ref: library/ctypes ctypes _FuncPtr argtypes5171944 +Ref: 1ffd5171944 +Ref: library/ctypes ctypes _FuncPtr errcheck5173183 +Ref: 1ffc5173183 +Ref: library/ctypes ctypes ArgumentError5174169 +Ref: 1ffe5174169 +Node: Function prototypes5174809 +Ref: library/ctypes ctypes-function-prototypes5174933 +Ref: 1fff5174933 +Ref: library/ctypes function-prototypes5174933 +Ref: 20005174933 +Ref: library/ctypes ctypes CFUNCTYPE5175516 +Ref: 1fda5175516 +Ref: library/ctypes ctypes WINFUNCTYPE5176027 +Ref: 1fdb5176027 +Ref: library/ctypes ctypes PYFUNCTYPE5176479 +Ref: 20015176479 +Node: Utility functions5182112 +Ref: library/ctypes ctypes-utility-functions5182229 +Ref: 20025182229 +Ref: library/ctypes utility-functions5182229 +Ref: 20035182229 +Ref: library/ctypes ctypes addressof5182288 +Ref: 20045182288 +Ref: library/ctypes ctypes alignment5182531 +Ref: 20055182531 +Ref: library/ctypes ctypes byref5182691 +Ref: 1fc15182691 +Ref: library/ctypes ctypes cast5183205 +Ref: 1fd55183205 +Ref: library/ctypes ctypes create_string_buffer5183499 +Ref: 1fb45183499 +Ref: library/ctypes ctypes create_unicode_buffer5184304 +Ref: 1fb55184304 +Ref: library/ctypes ctypes DllCanUnloadNow5185123 +Ref: 20065185123 +Ref: library/ctypes ctypes DllGetClassObject5185368 +Ref: 20075185368 +Ref: library/ctypes ctypes util find_library5185628 +Ref: 60a5185628 +Ref: library/ctypes ctypes util find_msvcrt5186020 +Ref: 20085186020 +Ref: library/ctypes ctypes FormatError5186476 +Ref: 20095186476 +Ref: library/ctypes ctypes GetLastError5186715 +Ref: 1fbe5186715 +Ref: library/ctypes ctypes get_errno5186989 +Ref: 1ff05186989 +Ref: library/ctypes ctypes get_last_error5187241 +Ref: 1ff25187241 +Ref: library/ctypes ctypes memmove5187516 +Ref: 200a5187516 +Ref: library/ctypes ctypes memset5187760 +Ref: 200b5187760 +Ref: library/ctypes ctypes POINTER5188013 +Ref: 1fd25188013 +Ref: library/ctypes ctypes pointer5188261 +Ref: 1fc25188261 +Ref: library/ctypes ctypes resize5188581 +Ref: 1fe55188581 +Ref: library/ctypes ctypes set_errno5188919 +Ref: 1ff15188919 +Ref: library/ctypes ctypes set_last_error5189226 +Ref: 1ff35189226 +Ref: library/ctypes ctypes sizeof5189556 +Ref: 200c5189556 +Ref: library/ctypes ctypes string_at5189727 +Ref: 200d5189727 +Ref: library/ctypes ctypes WinError5190096 +Ref: 200e5190096 +Ref: library/ctypes ctypes wstring_at5190577 +Ref: 200f5190577 +Node: Data types5190996 +Ref: library/ctypes ctypes-data-types5191119 +Ref: 20105191119 +Ref: library/ctypes data-types5191119 +Ref: 20115191119 +Ref: library/ctypes ctypes _CData5191164 +Ref: 20125191164 +Ref: library/ctypes ctypes _CData from_buffer5191799 +Ref: 20155191799 +Ref: library/ctypes ctypes _CData from_buffer_copy5192385 +Ref: 20165192385 +Ref: library/ctypes ctypes _CData from_address5192929 +Ref: 20175192929 +Ref: library/ctypes ctypes _CData from_param5193274 +Ref: 20185193274 +Ref: library/ctypes ctypes _CData in_dll5193828 +Ref: 20195193828 +Ref: library/ctypes ctypes _CData _b_base_5194124 +Ref: 201a5194124 +Ref: library/ctypes ctypes _CData _b_needsfree_5194429 +Ref: 201b5194429 +Ref: library/ctypes ctypes _CData _objects5194603 +Ref: 20135194603 +Node: Fundamental data types<2>5194915 +Ref: library/ctypes ctypes-fundamental-data-types-25195042 +Ref: 201c5195042 +Ref: library/ctypes id15195042 +Ref: 201d5195042 +Ref: library/ctypes ctypes _SimpleCData5195111 +Ref: 201e5195111 +Ref: library/ctypes ctypes _SimpleCData value5195584 +Ref: 201f5195584 +Ref: library/ctypes ctypes c_byte5196939 +Ref: 1fa55196939 +Ref: library/ctypes ctypes c_char5197157 +Ref: 1fa35197157 +Ref: library/ctypes ctypes c_char_p5197403 +Ref: 1fb15197403 +Ref: library/ctypes ctypes c_double5197704 +Ref: 1faf5197704 +Ref: library/ctypes ctypes c_longdouble5197839 +Ref: 1fb05197839 +Ref: library/ctypes ctypes c_float5198092 +Ref: 1fae5198092 +Ref: library/ctypes ctypes c_int5198225 +Ref: 1f985198225 +Ref: library/ctypes ctypes c_int85198495 +Ref: 20205198495 +Ref: library/ctypes ctypes c_int165198624 +Ref: 20215198624 +Ref: library/ctypes ctypes c_int325198756 +Ref: 20225198756 +Ref: library/ctypes ctypes c_int645198886 +Ref: 20235198886 +Ref: library/ctypes ctypes c_long5199021 +Ref: 1f995199021 +Ref: library/ctypes ctypes c_longlong5199191 +Ref: 1fab5199191 +Ref: library/ctypes ctypes c_short5199375 +Ref: 1fa75199375 +Ref: library/ctypes ctypes c_size_t5199547 +Ref: 1fad5199547 +Ref: library/ctypes ctypes c_ssize_t5199621 +Ref: bd15199621 +Ref: library/ctypes ctypes c_ubyte5199723 +Ref: 1fa65199723 +Ref: library/ctypes ctypes c_uint5199943 +Ref: 1fa95199943 +Ref: library/ctypes ctypes c_uint85200218 +Ref: 20245200218 +Ref: library/ctypes ctypes c_uint165200351 +Ref: 20255200351 +Ref: library/ctypes ctypes c_uint325200487 +Ref: 20265200487 +Ref: library/ctypes ctypes c_uint645200621 +Ref: 20275200621 +Ref: library/ctypes ctypes c_ulong5200760 +Ref: 1faa5200760 +Ref: library/ctypes ctypes c_ulonglong5200933 +Ref: 1fac5200933 +Ref: library/ctypes ctypes c_ushort5201120 +Ref: 1fa85201120 +Ref: library/ctypes ctypes c_void_p5201300 +Ref: 1fb35201300 +Ref: library/ctypes ctypes c_wchar5201471 +Ref: 1fa45201471 +Ref: library/ctypes ctypes c_wchar_p5201736 +Ref: 1fb25201736 +Ref: library/ctypes ctypes c_bool5201950 +Ref: 1fa25201950 +Ref: library/ctypes ctypes HRESULT5202177 +Ref: 1fed5202177 +Ref: library/ctypes ctypes py_object5202338 +Ref: 20285202338 +Node: Structured data types5202734 +Ref: library/ctypes ctypes-structured-data-types5202870 +Ref: 20295202870 +Ref: library/ctypes structured-data-types5202870 +Ref: 202a5202870 +Ref: library/ctypes ctypes Union5202937 +Ref: 1fc65202937 +Ref: library/ctypes ctypes BigEndianStructure5203035 +Ref: 1fc95203035 +Ref: library/ctypes ctypes LittleEndianStructure5203156 +Ref: 1fca5203156 +Ref: library/ctypes ctypes Structure5203414 +Ref: 1fc55203414 +Ref: library/ctypes ctypes Structure _fields_5203838 +Ref: 202b5203838 +Ref: library/ctypes ctypes Structure _pack_5205432 +Ref: 202c5205432 +Ref: library/ctypes ctypes Structure _anonymous_5205720 +Ref: 202d5205720 +Node: Arrays and pointers5208076 +Ref: library/ctypes arrays-and-pointers5208178 +Ref: 202e5208178 +Ref: library/ctypes ctypes-arrays-pointers5208178 +Ref: 202f5208178 +Ref: library/ctypes ctypes Array5208241 +Ref: 20305208241 +Ref: library/ctypes ctypes Array _length_5208751 +Ref: 20315208751 +Ref: library/ctypes ctypes Array _type_5208979 +Ref: 20325208979 +Ref: library/ctypes ctypes _Pointer5209177 +Ref: 20335209177 +Ref: library/ctypes ctypes _Pointer _type_5209835 +Ref: 20345209835 +Ref: library/ctypes ctypes _Pointer contents5209905 +Ref: 1fd15209905 +Node: Concurrent Execution5210095 +Ref: library/concurrency doc5210271 +Ref: 20355210271 +Ref: library/concurrency concurrency5210271 +Ref: 20365210271 +Ref: library/concurrency concurrent-execution5210271 +Ref: 20375210271 +Node: threading — Thread-based parallelism5211376 +Ref: library/threading doc5211525 +Ref: 20385211525 +Ref: library/threading module-threading5211525 +Ref: 1095211525 +Ref: library/threading threading-thread-based-parallelism5211525 +Ref: 20395211525 +Ref: library/threading threading active_count5212746 +Ref: 203a5212746 +Ref: library/threading threading current_thread5212959 +Ref: 203c5212959 +Ref: library/threading threading excepthook5213277 +Ref: 2315213277 +Ref: library/threading threading get_ident5214611 +Ref: aaf5214611 +Ref: library/threading threading get_native_id5215005 +Ref: 2335215005 +Ref: library/threading threading enumerate5215465 +Ref: 203b5215465 +Ref: library/threading threading main_thread5215784 +Ref: 8ef5215784 +Ref: library/threading threading settrace5216009 +Ref: 203d5216009 +Ref: library/threading threading setprofile5216276 +Ref: 203e5216276 +Ref: library/threading threading stack_size5216549 +Ref: 203f5216549 +Ref: library/threading threading TIMEOUT_MAX5217843 +Ref: 20405217843 +Ref: threading — Thread-based parallelism-Footnote-15219200 +Node: Thread-Local Data5219268 +Ref: library/threading thread-local-data5219383 +Ref: 20425219383 +Ref: library/threading threading local5219735 +Ref: 1fdc5219735 +Node: Thread Objects5219931 +Ref: library/threading id15220067 +Ref: 20435220067 +Ref: library/threading thread-objects5220067 +Ref: 20445220067 +Ref: library/threading threading Thread5222747 +Ref: 2355222747 +Ref: library/threading threading Thread start5223981 +Ref: 1db35223981 +Ref: library/threading threading Thread run5224358 +Ref: 2325224358 +Ref: library/threading threading Thread join5224770 +Ref: b605224770 +Ref: library/threading threading Thread name5226050 +Ref: 20465226050 +Ref: library/threading threading Thread getName5226268 +Ref: 20485226268 +Ref: library/threading threading Thread setName5226296 +Ref: 20495226296 +Ref: library/threading threading Thread ident5226430 +Ref: 1cf45226430 +Ref: library/threading threading Thread native_id5226821 +Ref: 2345226821 +Ref: library/threading threading Thread is_alive5227680 +Ref: 20455227680 +Ref: library/threading threading Thread daemon5228018 +Ref: 20475228018 +Ref: library/threading threading Thread isDaemon5228601 +Ref: 204a5228601 +Ref: library/threading threading Thread setDaemon5228630 +Ref: 204b5228630 +Node: Lock Objects5228768 +Ref: library/threading id25228900 +Ref: 204c5228900 +Ref: library/threading lock-objects5228900 +Ref: 204d5228900 +Ref: library/threading threading Lock5230317 +Ref: 20505230317 +Ref: library/threading threading Lock acquire5230719 +Ref: 76c5230719 +Ref: library/threading threading Lock release5232050 +Ref: 204e5232050 +Ref: library/threading threading Lock locked5232535 +Ref: 20515232535 +Node: RLock Objects5232611 +Ref: library/threading id35232746 +Ref: 20525232746 +Ref: library/threading rlock-objects5232746 +Ref: 20535232746 +Ref: library/threading threading RLock5233656 +Ref: bef5233656 +Ref: library/threading threading RLock acquire5234178 +Ref: 76d5234178 +Ref: library/threading threading RLock release5235705 +Ref: 20545235705 +Node: Condition Objects5236398 +Ref: library/threading condition-objects5236538 +Ref: 20555236538 +Ref: library/threading id45236538 +Ref: 20565236538 +Ref: library/threading threading Condition5239663 +Ref: aaa5239663 +Ref: library/threading threading Condition acquire5240206 +Ref: 20575240206 +Ref: library/threading threading Condition release5240418 +Ref: 20585240418 +Ref: library/threading threading Condition wait5240601 +Ref: 20415240601 +Ref: library/threading threading Condition wait_for5242132 +Ref: 205b5242132 +Ref: library/threading threading Condition notify5243104 +Ref: 20595243104 +Ref: library/threading threading Condition notify_all5244006 +Ref: 205a5244006 +Node: Semaphore Objects5244345 +Ref: library/threading id55244485 +Ref: 205c5244485 +Ref: library/threading semaphore-objects5244485 +Ref: 205d5244485 +Ref: library/threading threading Semaphore5245181 +Ref: aab5245181 +Ref: library/threading threading Semaphore acquire5245870 +Ref: bec5245870 +Ref: library/threading threading Semaphore release5247159 +Ref: 205e5247159 +Ref: library/threading threading BoundedSemaphore5247400 +Ref: aac5247400 +Node: Semaphore Example5247963 +Ref: library/threading semaphore-example5248034 +Ref: 205f5248034 +Ref: library/threading semaphore-examples5248034 +Ref: 20605248034 +Node: Event Objects5248903 +Ref: library/threading event-objects5249039 +Ref: 20615249039 +Ref: library/threading id65249039 +Ref: 20625249039 +Ref: library/threading threading Event5249440 +Ref: aad5249440 +Ref: library/threading threading Event is_set5249830 +Ref: 20655249830 +Ref: library/threading threading Event set5249929 +Ref: 20635249929 +Ref: library/threading threading Event clear5250156 +Ref: 20645250156 +Ref: library/threading threading Event wait5250384 +Ref: cb75250384 +Node: Timer Objects5251259 +Ref: library/threading id75251393 +Ref: 20665251393 +Ref: library/threading timer-objects5251393 +Ref: 20675251393 +Ref: library/threading threading Timer5252142 +Ref: aae5252142 +Ref: library/threading threading Timer cancel5252591 +Ref: 20685252591 +Node: Barrier Objects5252781 +Ref: library/threading barrier-objects5252961 +Ref: 20695252961 +Ref: library/threading threading Barrier5253916 +Ref: b5f5253916 +Ref: library/threading threading Barrier wait5254260 +Ref: 206a5254260 +Ref: library/threading threading Barrier reset5255407 +Ref: 206b5255407 +Ref: library/threading threading Barrier abort5255843 +Ref: 206c5255843 +Ref: library/threading threading Barrier parties5256332 +Ref: 206d5256332 +Ref: library/threading threading Barrier n_waiting5256424 +Ref: 206e5256424 +Ref: library/threading threading Barrier broken5256522 +Ref: 206f5256522 +Ref: library/threading threading BrokenBarrierError5256639 +Ref: b615256639 +Node: Using locks conditions and semaphores in the with statement5256816 +Ref: library/threading using-locks-conditions-and-semaphores-in-the-with-statement5256974 +Ref: 20705256974 +Ref: library/threading with-locks5256974 +Ref: 204f5256974 +Node: multiprocessing — Process-based parallelism5257819 +Ref: library/multiprocessing doc5258068 +Ref: 20715258068 +Ref: library/multiprocessing module-multiprocessing5258068 +Ref: b75258068 +Ref: library/multiprocessing multiprocessing-process-based-parallelism5258068 +Ref: 20725258068 +Ref: multiprocessing — Process-based parallelism-Footnote-15258435 +Node: Introduction<8>5258507 +Ref: library/multiprocessing introduction5258622 +Ref: 20735258622 +Node: The Process class5260155 +Ref: library/multiprocessing the-process-class5260259 +Ref: 20745260259 +Node: Contexts and start methods5261440 +Ref: library/multiprocessing contexts-and-start-methods5261589 +Ref: 20775261589 +Ref: library/multiprocessing multiprocessing-start-methods5261662 +Ref: 8765261662 +Ref: Contexts and start methods-Footnote-15266140 +Node: Exchanging objects between processes5266183 +Ref: library/multiprocessing exchanging-objects-between-processes5266348 +Ref: 207a5266348 +Node: Synchronization between processes5268066 +Ref: library/multiprocessing synchronization-between-processes5268236 +Ref: 207d5268236 +Node: Sharing state between processes5268957 +Ref: library/multiprocessing sharing-state-between-processes5269114 +Ref: 207e5269114 +Node: Using a pool of workers5272199 +Ref: library/multiprocessing using-a-pool-of-workers5272314 +Ref: 20895272314 +Node: Reference5275431 +Ref: library/multiprocessing reference5275577 +Ref: 208a5275577 +Node: Process and exceptions5276106 +Ref: library/multiprocessing process-and-exceptions5276199 +Ref: 208b5276199 +Ref: library/multiprocessing multiprocessing Process5276272 +Ref: 3b25276272 +Ref: library/multiprocessing multiprocessing Process run5277641 +Ref: 20785277641 +Ref: library/multiprocessing multiprocessing Process start5278053 +Ref: 20755278053 +Ref: library/multiprocessing multiprocessing Process join5278302 +Ref: 208e5278302 +Ref: library/multiprocessing multiprocessing Process name5279024 +Ref: 208c5279024 +Ref: library/multiprocessing multiprocessing Process is_alive5279489 +Ref: 20905279489 +Ref: library/multiprocessing multiprocessing Process daemon5279725 +Ref: 208d5279725 +Ref: library/multiprocessing multiprocessing Process pid5280606 +Ref: 20915280606 +Ref: library/multiprocessing multiprocessing Process exitcode5280731 +Ref: 208f5280731 +Ref: library/multiprocessing multiprocessing Process authkey5280959 +Ref: 20925280959 +Ref: library/multiprocessing multiprocessing Process sentinel5281487 +Ref: a255281487 +Ref: library/multiprocessing multiprocessing Process terminate5282124 +Ref: 20945282124 +Ref: library/multiprocessing multiprocessing Process kill5282926 +Ref: 3b15282926 +Ref: library/multiprocessing multiprocessing Process close5283081 +Ref: 3b05283081 +Ref: library/multiprocessing multiprocessing ProcessError5284319 +Ref: 20955284319 +Ref: library/multiprocessing multiprocessing BufferTooShort5284431 +Ref: 20965284431 +Ref: library/multiprocessing multiprocessing AuthenticationError5284740 +Ref: 20975284740 +Ref: library/multiprocessing multiprocessing TimeoutError5284844 +Ref: 20985284844 +Node: Pipes and Queues5284954 +Ref: library/multiprocessing pipes-and-queues5285072 +Ref: 20995285072 +Ref: library/multiprocessing multiprocessing Pipe5288496 +Ref: 207c5288496 +Ref: library/multiprocessing multiprocessing Queue5288918 +Ref: 207b5288918 +Ref: library/multiprocessing multiprocessing Queue qsize5289516 +Ref: 20a75289516 +Ref: library/multiprocessing multiprocessing Queue empty5289866 +Ref: 20a25289866 +Ref: library/multiprocessing multiprocessing Queue full5290063 +Ref: 20a85290063 +Ref: library/multiprocessing multiprocessing Queue put5290258 +Ref: 20a95290258 +Ref: library/multiprocessing multiprocessing Queue put_nowait5291086 +Ref: 20aa5291086 +Ref: library/multiprocessing multiprocessing Queue get5291169 +Ref: 20ab5291169 +Ref: library/multiprocessing multiprocessing Queue get_nowait5291957 +Ref: 20a35291957 +Ref: library/multiprocessing multiprocessing Queue close5292203 +Ref: 20ac5292203 +Ref: library/multiprocessing multiprocessing Queue join_thread5292499 +Ref: 20ad5292499 +Ref: library/multiprocessing multiprocessing Queue cancel_join_thread5293050 +Ref: 20a45293050 +Ref: library/multiprocessing multiprocessing SimpleQueue5294195 +Ref: 209a5294195 +Ref: library/multiprocessing multiprocessing SimpleQueue empty5294331 +Ref: 20ae5294331 +Ref: library/multiprocessing multiprocessing SimpleQueue get5294433 +Ref: 20af5294433 +Ref: library/multiprocessing multiprocessing SimpleQueue put5294511 +Ref: 20b05294511 +Ref: library/multiprocessing multiprocessing JoinableQueue5294578 +Ref: 209b5294578 +Ref: library/multiprocessing multiprocessing JoinableQueue task_done5294798 +Ref: 209e5294798 +Ref: library/multiprocessing multiprocessing JoinableQueue join5295486 +Ref: 20b15295486 +Ref: Pipes and Queues-Footnote-15295990 +Node: Miscellaneous<3>5296032 +Ref: library/multiprocessing miscellaneous5296149 +Ref: 20b25296149 +Ref: library/multiprocessing multiprocessing active_children5296198 +Ref: 20b35296198 +Ref: library/multiprocessing multiprocessing cpu_count5296415 +Ref: 8805296415 +Ref: library/multiprocessing multiprocessing current_process5296796 +Ref: 20b45296796 +Ref: library/multiprocessing multiprocessing parent_process5296994 +Ref: 20b55296994 +Ref: library/multiprocessing multiprocessing freeze_support5297261 +Ref: 20b65297261 +Ref: library/multiprocessing multiprocessing get_all_start_methods5298297 +Ref: 8775298297 +Ref: library/multiprocessing multiprocessing get_context5298715 +Ref: 87a5298715 +Ref: library/multiprocessing multiprocessing get_start_method5299149 +Ref: 8785299149 +Ref: library/multiprocessing multiprocessing set_executable5299750 +Ref: 20b75299750 +Ref: library/multiprocessing multiprocessing set_start_method5300214 +Ref: 8795300214 +Node: Connection Objects<2>5300878 +Ref: library/multiprocessing connection-objects5301005 +Ref: 20b85301005 +Ref: library/multiprocessing multiprocessing connection Connection5301324 +Ref: 20a65301324 +Ref: library/multiprocessing multiprocessing connection Connection send5301374 +Ref: 20ba5301374 +Ref: library/multiprocessing multiprocessing connection Connection recv5301701 +Ref: 20bb5301701 +Ref: library/multiprocessing multiprocessing connection Connection fileno5301993 +Ref: 20bc5301993 +Ref: library/multiprocessing multiprocessing connection Connection close5302093 +Ref: 20bd5302093 +Ref: library/multiprocessing multiprocessing connection Connection poll5302245 +Ref: 20be5302245 +Ref: library/multiprocessing multiprocessing connection Connection send_bytes5302731 +Ref: 20bf5302731 +Ref: library/multiprocessing multiprocessing connection Connection recv_bytes5303190 +Ref: 20c05303190 +Ref: library/multiprocessing multiprocessing connection Connection recv_bytes_into5303830 +Ref: 20c15303830 +Node: Synchronization primitives5306345 +Ref: library/multiprocessing synchronization-primitives5306477 +Ref: 20c25306477 +Ref: library/multiprocessing multiprocessing Barrier5306848 +Ref: 20885306848 +Ref: library/multiprocessing multiprocessing BoundedSemaphore5307006 +Ref: 20855307006 +Ref: library/multiprocessing multiprocessing Condition5307507 +Ref: 20865307507 +Ref: library/multiprocessing multiprocessing Event5307841 +Ref: 20875307841 +Ref: library/multiprocessing multiprocessing Lock5307920 +Ref: 20825307920 +Ref: library/multiprocessing multiprocessing Lock acquire5308763 +Ref: 20c35308763 +Ref: library/multiprocessing multiprocessing Lock release5310289 +Ref: 20c45310289 +Ref: library/multiprocessing multiprocessing RLock5310658 +Ref: 20835310658 +Ref: library/multiprocessing multiprocessing RLock acquire5311389 +Ref: 20c55311389 +Ref: library/multiprocessing multiprocessing RLock release5313038 +Ref: 20c65313038 +Ref: library/multiprocessing multiprocessing Semaphore5314016 +Ref: 20845314016 +Ref: Synchronization primitives-Footnote-15315441 +Node: Shared ctypes Objects5315483 +Ref: library/multiprocessing shared-ctypes-objects5315602 +Ref: 20c75315602 +Ref: library/multiprocessing multiprocessing Value5315779 +Ref: 207f5315779 +Ref: library/multiprocessing multiprocessing Array5317208 +Ref: 20805317208 +Node: The multiprocessing sharedctypes module5318651 +Ref: library/multiprocessing module-multiprocessing sharedctypes5318748 +Ref: bd5318748 +Ref: library/multiprocessing the-multiprocessing-sharedctypes-module5318748 +Ref: 20c85318748 +Ref: library/multiprocessing multiprocessing sharedctypes RawArray5319379 +Ref: 20c95319379 +Ref: library/multiprocessing multiprocessing sharedctypes RawValue5320208 +Ref: 20cb5320208 +Ref: library/multiprocessing multiprocessing sharedctypes Array5320992 +Ref: 20ca5320992 +Ref: library/multiprocessing multiprocessing sharedctypes Value5321759 +Ref: 20cc5321759 +Ref: library/multiprocessing multiprocessing sharedctypes copy5322510 +Ref: 20cd5322510 +Ref: library/multiprocessing multiprocessing sharedctypes synchronized5322672 +Ref: 7195322672 +Node: Managers5325688 +Ref: library/multiprocessing managers5325794 +Ref: 20ce5325794 +Ref: library/multiprocessing multiprocessing-managers5325794 +Ref: 209f5325794 +Ref: library/multiprocessing multiprocessing sharedctypes multiprocessing Manager5326144 +Ref: 20cf5326144 +Ref: library/multiprocessing module-multiprocessing managers5326464 +Ref: ba5326464 +Ref: library/multiprocessing multiprocessing managers BaseManager5326654 +Ref: 20d15326654 +Ref: library/multiprocessing multiprocessing managers BaseManager start5327373 +Ref: 20d25327373 +Ref: library/multiprocessing multiprocessing managers BaseManager get_server5327608 +Ref: 20d35327608 +Ref: library/multiprocessing multiprocessing managers BaseManager connect5328152 +Ref: 20d55328152 +Ref: library/multiprocessing multiprocessing managers BaseManager shutdown5328433 +Ref: 20d65328433 +Ref: library/multiprocessing multiprocessing managers BaseManager register5328669 +Ref: 20d75328669 +Ref: library/multiprocessing multiprocessing managers BaseManager address5330985 +Ref: 20d45330985 +Ref: library/multiprocessing multiprocessing managers SyncManager5331516 +Ref: 20d05331516 +Ref: library/multiprocessing multiprocessing managers SyncManager Barrier5331951 +Ref: 20da5331951 +Ref: library/multiprocessing multiprocessing managers SyncManager BoundedSemaphore5332140 +Ref: 20db5332140 +Ref: library/multiprocessing multiprocessing managers SyncManager Condition5332295 +Ref: 20dc5332295 +Ref: library/multiprocessing multiprocessing managers SyncManager Event5332668 +Ref: 20dd5332668 +Ref: library/multiprocessing multiprocessing managers SyncManager Lock5332794 +Ref: 20de5332794 +Ref: library/multiprocessing multiprocessing managers SyncManager Namespace5332919 +Ref: 20df5332919 +Ref: library/multiprocessing multiprocessing managers SyncManager Queue5333044 +Ref: 20e05333044 +Ref: library/multiprocessing multiprocessing managers SyncManager RLock5333175 +Ref: 20e15333175 +Ref: library/multiprocessing multiprocessing managers SyncManager Semaphore5333301 +Ref: 20e25333301 +Ref: library/multiprocessing multiprocessing managers SyncManager Array5333442 +Ref: 20e35333442 +Ref: library/multiprocessing multiprocessing managers SyncManager Value5333541 +Ref: 20e45333541 +Ref: library/multiprocessing multiprocessing managers SyncManager dict5333686 +Ref: 20e55333686 +Ref: library/multiprocessing multiprocessing managers SyncManager list5333867 +Ref: 20e65333867 +Ref: library/multiprocessing multiprocessing managers Namespace5334273 +Ref: 20815334273 +Node: Customized managers5335054 +Ref: library/multiprocessing customized-managers5335149 +Ref: 20e75335149 +Node: Using a remote manager5335921 +Ref: library/multiprocessing using-a-remote-manager5336016 +Ref: 20e85336016 +Node: Proxy Objects5338169 +Ref: library/multiprocessing multiprocessing-proxy-objects5338267 +Ref: 5655338267 +Ref: library/multiprocessing proxy-objects5338267 +Ref: 20e95338267 +Ref: library/multiprocessing multiprocessing managers BaseProxy5341526 +Ref: 20d85341526 +Ref: library/multiprocessing multiprocessing managers BaseProxy _callmethod5341647 +Ref: 20d95341647 +Ref: library/multiprocessing multiprocessing managers BaseProxy _getvalue5343228 +Ref: 20ea5343228 +Ref: library/multiprocessing multiprocessing managers BaseProxy __repr__5343388 +Ref: 20eb5343388 +Ref: library/multiprocessing multiprocessing managers BaseProxy __str__5343474 +Ref: 20ec5343474 +Node: Cleanup5343579 +Ref: library/multiprocessing cleanup5343636 +Ref: 20ed5343636 +Node: Process Pools5343925 +Ref: library/multiprocessing module-multiprocessing pool5344036 +Ref: bb5344036 +Ref: library/multiprocessing process-pools5344036 +Ref: 20ee5344036 +Ref: library/multiprocessing multiprocessing pool Pool5344197 +Ref: 87b5344197 +Ref: library/multiprocessing multiprocessing pool Pool apply5346805 +Ref: 20f15346805 +Ref: library/multiprocessing multiprocessing pool Pool apply_async5347176 +Ref: 20f25347176 +Ref: library/multiprocessing multiprocessing pool Pool map5348028 +Ref: a295348028 +Ref: library/multiprocessing multiprocessing pool Pool map_async5348812 +Ref: a2a5348812 +Ref: library/multiprocessing multiprocessing pool Pool imap5349666 +Ref: 20f45349666 +Ref: library/multiprocessing multiprocessing pool Pool imap_unordered5350376 +Ref: 20f55350376 +Ref: library/multiprocessing multiprocessing pool Pool starmap5350702 +Ref: a265350702 +Ref: library/multiprocessing multiprocessing pool Pool starmap_async5351054 +Ref: a275351054 +Ref: library/multiprocessing multiprocessing pool Pool close5351405 +Ref: 20ef5351405 +Ref: library/multiprocessing multiprocessing pool Pool terminate5351592 +Ref: 20f05351592 +Ref: library/multiprocessing multiprocessing pool Pool join5351826 +Ref: 20f65351826 +Ref: library/multiprocessing multiprocessing pool AsyncResult5352270 +Ref: 20f35352270 +Ref: library/multiprocessing multiprocessing pool AsyncResult get5352427 +Ref: 20f75352427 +Ref: library/multiprocessing multiprocessing pool AsyncResult wait5352792 +Ref: 20f85352792 +Ref: library/multiprocessing multiprocessing pool AsyncResult ready5352916 +Ref: 20f95352916 +Ref: library/multiprocessing multiprocessing pool AsyncResult successful5352993 +Ref: 20fa5352993 +Node: Listeners and Clients5354287 +Ref: library/multiprocessing listeners-and-clients5354404 +Ref: 20fb5354404 +Ref: library/multiprocessing module-multiprocessing connection5354404 +Ref: b85354404 +Ref: library/multiprocessing multiprocessing-listeners-clients5354404 +Ref: 20b95354404 +Ref: library/multiprocessing multiprocessing connection deliver_challenge5354940 +Ref: 20fc5354940 +Ref: library/multiprocessing multiprocessing connection answer_challenge5355346 +Ref: 20fd5355346 +Ref: library/multiprocessing multiprocessing connection Client5355660 +Ref: 20fe5355660 +Ref: library/multiprocessing multiprocessing connection Listener5356414 +Ref: 21005356414 +Ref: library/multiprocessing multiprocessing connection Listener accept5358174 +Ref: 21015358174 +Ref: library/multiprocessing multiprocessing connection Listener close5358456 +Ref: 21025358456 +Ref: library/multiprocessing multiprocessing connection Listener address5358756 +Ref: 21035358756 +Ref: library/multiprocessing multiprocessing connection Listener last_accepted5358852 +Ref: 21045358852 +Ref: library/multiprocessing multiprocessing connection wait5359268 +Ref: a245359268 +Node: Address Formats5363308 +Ref: library/multiprocessing address-formats5363381 +Ref: 21055363381 +Ref: library/multiprocessing multiprocessing-address-formats5363381 +Ref: 20ff5363381 +Node: Authentication keys5364102 +Ref: library/multiprocessing authentication-keys5364216 +Ref: 21065364216 +Ref: library/multiprocessing multiprocessing-auth-keys5364216 +Ref: 20935364216 +Node: Logging<2>5365400 +Ref: library/multiprocessing logging5365525 +Ref: 21075365525 +Ref: library/multiprocessing multiprocessing get_logger5365793 +Ref: 21085365793 +Ref: library/multiprocessing multiprocessing log_to_stderr5366305 +Ref: 21095366305 +Node: The multiprocessing dummy module5367316 +Ref: library/multiprocessing module-multiprocessing dummy5367413 +Ref: b95367413 +Ref: library/multiprocessing the-multiprocessing-dummy-module5367413 +Ref: 210a5367413 +Ref: library/multiprocessing multiprocessing pool ThreadPool5367938 +Ref: 210b5367938 +Node: Programming guidelines5369773 +Ref: library/multiprocessing multiprocessing-programming5369916 +Ref: 20765369916 +Ref: library/multiprocessing programming-guidelines5369916 +Ref: 210c5369916 +Node: All start methods5370164 +Ref: library/multiprocessing all-start-methods5370287 +Ref: 210d5370287 +Ref: All start methods-Footnote-15376383 +Ref: All start methods-Footnote-25376425 +Ref: All start methods-Footnote-35376467 +Node: The spawn and forkserver start methods5376509 +Ref: library/multiprocessing the-spawn-and-forkserver-start-methods5376632 +Ref: 210e5376632 +Node: Examples<10>5378709 +Ref: library/multiprocessing examples5378834 +Ref: 210f5378834 +Ref: library/multiprocessing multiprocessing-examples5378834 +Ref: 20a55378834 +Node: multiprocessing shared_memory — Provides shared memory for direct access across processes5387745 +Ref: library/multiprocessing shared_memory doc5387978 +Ref: 21105387978 +Ref: library/multiprocessing shared_memory module-multiprocessing shared_memory5387978 +Ref: bc5387978 +Ref: library/multiprocessing shared_memory multiprocessing-shared-memory-provides-shared-memory-for-direct-access-across-processes5387978 +Ref: 21115387978 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory SharedMemory5389544 +Ref: 20795389544 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory SharedMemory close5391219 +Ref: 21125391219 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory SharedMemory unlink5391575 +Ref: 21135391575 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory SharedMemory buf5392355 +Ref: 21145392355 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory SharedMemory name5392444 +Ref: 21155392444 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory SharedMemory size5392555 +Ref: 21165392555 +Ref: library/multiprocessing shared_memory multiprocessing managers SharedMemoryManager5395645 +Ref: 21175395645 +Ref: library/multiprocessing shared_memory multiprocessing managers SharedMemoryManager SharedMemory5397060 +Ref: 21185397060 +Ref: library/multiprocessing shared_memory multiprocessing managers SharedMemoryManager ShareableList5397216 +Ref: 21195397216 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory ShareableList5399062 +Ref: 211a5399062 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory ShareableList count5400237 +Ref: 211b5400237 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory ShareableList index5400330 +Ref: 211c5400330 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory ShareableList format5400490 +Ref: 211d5400490 +Ref: library/multiprocessing shared_memory multiprocessing shared_memory ShareableList shm5400644 +Ref: 211e5400644 +Ref: multiprocessing shared_memory — Provides shared memory for direct access across processes-Footnote-15402320 +Ref: multiprocessing shared_memory — Provides shared memory for direct access across processes-Footnote-25402409 +Node: The concurrent package5402440 +Ref: library/concurrent doc5402675 +Ref: 211f5402675 +Ref: library/concurrent the-concurrent-package5402675 +Ref: 21205402675 +Node: concurrent futures — Launching parallel tasks5402863 +Ref: library/concurrent futures doc5403043 +Ref: 21215403043 +Ref: library/concurrent futures concurrent-futures-launching-parallel-tasks5403043 +Ref: 21225403043 +Ref: library/concurrent futures module-concurrent futures5403043 +Ref: 225403043 +Ref: concurrent futures — Launching parallel tasks-Footnote-15403887 +Ref: concurrent futures — Launching parallel tasks-Footnote-25403972 +Node: Executor Objects5404058 +Ref: library/concurrent futures executor-objects5404185 +Ref: 21245404185 +Ref: library/concurrent futures concurrent futures Executor5404238 +Ref: 21235404238 +Ref: library/concurrent futures concurrent futures Executor submit5404437 +Ref: 2a35404437 +Ref: library/concurrent futures concurrent futures Executor map5404877 +Ref: 6be5404877 +Ref: library/concurrent futures concurrent futures Executor shutdown5406518 +Ref: b2e5406518 +Node: ThreadPoolExecutor5408183 +Ref: library/concurrent futures threadpoolexecutor5408338 +Ref: 21265408338 +Ref: library/concurrent futures concurrent futures ThreadPoolExecutor5409406 +Ref: 27a5409406 +Node: ThreadPoolExecutor Example5411190 +Ref: library/concurrent futures id15411271 +Ref: 21285411271 +Ref: library/concurrent futures threadpoolexecutor-example5411271 +Ref: b2f5411271 +Node: ProcessPoolExecutor5412470 +Ref: library/concurrent futures processpoolexecutor5412623 +Ref: 21295412623 +Ref: library/concurrent futures concurrent futures ProcessPoolExecutor5413357 +Ref: 2a45413357 +Node: ProcessPoolExecutor Example5415168 +Ref: library/concurrent futures id25415251 +Ref: 212b5415251 +Ref: library/concurrent futures processpoolexecutor-example5415251 +Ref: b305415251 +Node: Future Objects5416157 +Ref: library/concurrent futures future-objects5416308 +Ref: 212c5416308 +Ref: library/concurrent futures concurrent futures Future5416517 +Ref: b2d5416517 +Ref: library/concurrent futures concurrent futures Future cancel5416756 +Ref: 212d5416756 +Ref: library/concurrent futures concurrent futures Future cancelled5417095 +Ref: 212e5417095 +Ref: library/concurrent futures concurrent futures Future running5417205 +Ref: 212f5417205 +Ref: library/concurrent futures concurrent futures Future done5417353 +Ref: 21305417353 +Ref: library/concurrent futures concurrent futures Future result5417493 +Ref: 21315417493 +Ref: library/concurrent futures concurrent futures Future exception5418227 +Ref: 21325418227 +Ref: library/concurrent futures concurrent futures Future add_done_callback5418966 +Ref: 21335418966 +Ref: library/concurrent futures concurrent futures Future set_running_or_notify_cancel5419862 +Ref: 21345419862 +Ref: library/concurrent futures concurrent futures Future set_result5420951 +Ref: 21375420951 +Ref: library/concurrent futures concurrent futures Future set_exception5421407 +Ref: 21385421407 +Node: Module Functions5421898 +Ref: library/concurrent futures module-functions5422047 +Ref: 213a5422047 +Ref: library/concurrent futures concurrent futures wait5422100 +Ref: 21365422100 +Ref: library/concurrent futures concurrent futures as_completed5424009 +Ref: 21355424009 +Ref: Module Functions-Footnote-15425066 +Node: Exception classes5425115 +Ref: library/concurrent futures exception-classes5425241 +Ref: 213b5425241 +Ref: library/concurrent futures concurrent futures CancelledError5425296 +Ref: 1aa5425296 +Ref: library/concurrent futures concurrent futures TimeoutError5425387 +Ref: 21255425387 +Ref: library/concurrent futures concurrent futures BrokenExecutor5425499 +Ref: 213c5425499 +Ref: library/concurrent futures concurrent futures InvalidStateError5425755 +Ref: 21395425755 +Ref: library/concurrent futures concurrent futures thread BrokenThreadPool5425937 +Ref: 21275425937 +Ref: library/concurrent futures concurrent futures process BrokenProcessPool5426189 +Ref: 212a5426189 +Node: subprocess — Subprocess management5426549 +Ref: library/subprocess doc5426732 +Ref: 213d5426732 +Ref: library/subprocess module-subprocess5426732 +Ref: f95426732 +Ref: library/subprocess subprocess-subprocess-management5426732 +Ref: 213e5426732 +Ref: subprocess — Subprocess management-Footnote-15427673 +Ref: subprocess — Subprocess management-Footnote-25427742 +Node: Using the subprocess Module5427791 +Ref: library/subprocess using-the-subprocess-module5427923 +Ref: 213f5427923 +Ref: library/subprocess subprocess run5428391 +Ref: 3ed5428391 +Ref: library/subprocess subprocess CompletedProcess5432175 +Ref: 7605432175 +Ref: library/subprocess subprocess CompletedProcess args5432308 +Ref: 21445432308 +Ref: library/subprocess subprocess CompletedProcess returncode5432430 +Ref: 21455432430 +Ref: library/subprocess subprocess CompletedProcess stdout5432704 +Ref: 21465432704 +Ref: library/subprocess subprocess CompletedProcess stderr5433135 +Ref: 21475433135 +Ref: library/subprocess subprocess CompletedProcess check_returncode5433377 +Ref: 21485433377 +Ref: library/subprocess subprocess DEVNULL5433541 +Ref: aa05433541 +Ref: library/subprocess subprocess PIPE5433785 +Ref: 3ee5433785 +Ref: library/subprocess subprocess STDOUT5434049 +Ref: 21495434049 +Ref: library/subprocess subprocess SubprocessError5434256 +Ref: 214a5434256 +Ref: library/subprocess subprocess TimeoutExpired5434385 +Ref: 21435434385 +Ref: library/subprocess subprocess TimeoutExpired cmd5434544 +Ref: 214b5434544 +Ref: library/subprocess subprocess TimeoutExpired timeout5434630 +Ref: 214c5434630 +Ref: library/subprocess subprocess TimeoutExpired output5434690 +Ref: 214d5434690 +Ref: library/subprocess subprocess TimeoutExpired stdout5434861 +Ref: 214e5434861 +Ref: library/subprocess subprocess TimeoutExpired stderr5434956 +Ref: 214f5434956 +Ref: library/subprocess subprocess CalledProcessError5435199 +Ref: cb55435199 +Ref: library/subprocess subprocess CalledProcessError returncode5435419 +Ref: 21515435419 +Ref: library/subprocess subprocess CalledProcessError cmd5435587 +Ref: 21525435587 +Ref: library/subprocess subprocess CalledProcessError output5435673 +Ref: 21535435673 +Ref: library/subprocess subprocess CalledProcessError stdout5435844 +Ref: 21545435844 +Ref: library/subprocess subprocess CalledProcessError stderr5435939 +Ref: 21555435939 +Node: Frequently Used Arguments5436247 +Ref: library/subprocess frequently-used-arguments5436362 +Ref: 21415436362 +Ref: library/subprocess id15436362 +Ref: 21565436362 +Node: Popen Constructor5440531 +Ref: library/subprocess popen-constructor5440668 +Ref: 215b5440668 +Ref: library/subprocess subprocess Popen5440963 +Ref: 2b95440963 +Ref: Popen Constructor-Footnote-15454778 +Node: Exceptions<7>5454838 +Ref: library/subprocess exceptions5454941 +Ref: 216b5454941 +Node: Security Considerations5455934 +Ref: library/subprocess security-considerations5456088 +Ref: 215a5456088 +Ref: Security Considerations-Footnote-15456816 +Node: Popen Objects5456886 +Ref: library/subprocess popen-objects5457034 +Ref: 216c5457034 +Ref: library/subprocess subprocess Popen poll5457151 +Ref: 216d5457151 +Ref: library/subprocess subprocess Popen wait5457311 +Ref: 5925457311 +Ref: library/subprocess subprocess Popen communicate5458240 +Ref: 21425458240 +Ref: library/subprocess subprocess Popen send_signal5459991 +Ref: 216e5459991 +Ref: library/subprocess subprocess Popen terminate5460333 +Ref: 216f5460333 +Ref: library/subprocess subprocess Popen kill5460539 +Ref: 21705460539 +Ref: library/subprocess subprocess Popen args5460773 +Ref: 21715460773 +Ref: library/subprocess subprocess Popen stdin5460956 +Ref: 21575460956 +Ref: library/subprocess subprocess Popen stdout5461387 +Ref: 21585461387 +Ref: library/subprocess subprocess Popen stderr5461890 +Ref: 21595461890 +Ref: library/subprocess subprocess Popen pid5462655 +Ref: 21725462655 +Ref: library/subprocess subprocess Popen returncode5462837 +Ref: 216a5462837 +Node: Windows Popen Helpers5463188 +Ref: library/subprocess windows-popen-helpers5463333 +Ref: 21735463333 +Ref: library/subprocess subprocess STARTUPINFO5463487 +Ref: 215d5463487 +Ref: library/subprocess subprocess STARTUPINFO dwFlags5463906 +Ref: 21745463906 +Ref: library/subprocess subprocess STARTUPINFO hStdInput5464220 +Ref: 21755464220 +Ref: library/subprocess subprocess STARTUPINFO hStdOutput5464542 +Ref: 21775464542 +Ref: library/subprocess subprocess STARTUPINFO hStdError5464853 +Ref: 21785464853 +Ref: library/subprocess subprocess STARTUPINFO wShowWindow5465161 +Ref: 21795465161 +Ref: library/subprocess subprocess STARTUPINFO lpAttributeList5465653 +Ref: 49a5465653 +Ref: Windows Popen Helpers-Footnote-15466906 +Ref: Windows Popen Helpers-Footnote-25466978 +Ref: Windows Popen Helpers-Footnote-35467050 +Node: Windows Constants5467139 +Ref: library/subprocess windows-constants5467214 +Ref: 217c5467214 +Ref: library/subprocess subprocess STD_INPUT_HANDLE5467336 +Ref: 217d5467336 +Ref: library/subprocess subprocess STD_OUTPUT_HANDLE5467473 +Ref: 217e5467473 +Ref: library/subprocess subprocess STD_ERROR_HANDLE5467621 +Ref: 217f5467621 +Ref: library/subprocess subprocess SW_HIDE5467767 +Ref: 217b5467767 +Ref: library/subprocess subprocess STARTF_USESTDHANDLES5467856 +Ref: 21765467856 +Ref: library/subprocess subprocess STARTF_USESHOWWINDOW5468087 +Ref: 217a5468087 +Ref: library/subprocess subprocess CREATE_NEW_CONSOLE5468240 +Ref: 215e5468240 +Ref: library/subprocess subprocess CREATE_NEW_PROCESS_GROUP5468387 +Ref: 215f5468387 +Ref: library/subprocess subprocess ABOVE_NORMAL_PRIORITY_CLASS5468709 +Ref: 21605468709 +Ref: library/subprocess subprocess BELOW_NORMAL_PRIORITY_CLASS5468913 +Ref: 21615468913 +Ref: library/subprocess subprocess HIGH_PRIORITY_CLASS5469116 +Ref: 21625469116 +Ref: library/subprocess subprocess IDLE_PRIORITY_CLASS5469302 +Ref: 21635469302 +Ref: library/subprocess subprocess NORMAL_PRIORITY_CLASS5469498 +Ref: 21645469498 +Ref: library/subprocess subprocess REALTIME_PRIORITY_CLASS5469700 +Ref: 21655469700 +Ref: library/subprocess subprocess CREATE_NO_WINDOW5470234 +Ref: 21665470234 +Ref: library/subprocess subprocess DETACHED_PROCESS5470416 +Ref: 21675470416 +Ref: library/subprocess subprocess CREATE_DEFAULT_ERROR_MODE5470670 +Ref: 21685470670 +Ref: library/subprocess subprocess CREATE_BREAKAWAY_FROM_JOB5471070 +Ref: 21695471070 +Node: Older high-level API5471267 +Ref: library/subprocess call-function-trio5471451 +Ref: 21405471451 +Ref: library/subprocess older-high-level-api5471451 +Ref: 21805471451 +Ref: library/subprocess subprocess call5471699 +Ref: 3ef5471699 +Ref: library/subprocess subprocess check_call5472753 +Ref: 21505472753 +Ref: library/subprocess subprocess check_output5473964 +Ref: 8db5473964 +Node: Replacing Older Functions with the subprocess Module5476195 +Ref: library/subprocess replacing-older-functions-with-the-subprocess-module5476391 +Ref: 21815476391 +Ref: library/subprocess subprocess-replacements5476391 +Ref: aea5476391 +Node: Replacing /bin/sh shell command substitution5477558 +Ref: library/subprocess replacing-bin-sh-shell-command-substitution5477724 +Ref: 21825477724 +Node: Replacing shell pipeline5477929 +Ref: library/subprocess replacing-shell-pipeline5478123 +Ref: 21835478123 +Node: Replacing os system5478772 +Ref: library/subprocess replacing-os-system5478951 +Ref: 21845478951 +Node: Replacing the os spawn family5479586 +Ref: library/subprocess replacing-the-os-spawn-family5479779 +Ref: 21855479779 +Node: Replacing os popen os popen2 os popen35480410 +Ref: library/subprocess replacing-os-popen-os-popen2-os-popen35480626 +Ref: 21865480626 +Node: Replacing functions from the popen2 module5481921 +Ref: library/subprocess replacing-functions-from-the-popen2-module5482099 +Ref: 21875482099 +Node: Legacy Shell Invocation Functions5483413 +Ref: library/subprocess legacy-shell-invocation-functions5483594 +Ref: 21885483594 +Ref: library/subprocess subprocess getstatusoutput5483958 +Ref: 8dc5483958 +Ref: library/subprocess subprocess getoutput5485136 +Ref: 21895485136 +Node: Notes5485582 +Ref: library/subprocess notes5485702 +Ref: 218a5485702 +Node: Converting an argument sequence to a string on Windows5485804 +Ref: library/subprocess converting-an-argument-sequence-to-a-string-on-windows5485900 +Ref: 218b5485900 +Ref: library/subprocess converting-argument-sequence5485900 +Ref: 215c5485900 +Node: sched — Event scheduler5487060 +Ref: library/sched doc5487232 +Ref: 218c5487232 +Ref: library/sched module-sched5487232 +Ref: e35487232 +Ref: library/sched sched-event-scheduler5487232 +Ref: 218d5487232 +Ref: library/sched sched scheduler5487502 +Ref: a6e5487502 +Ref: sched — Event scheduler-Footnote-15489162 +Node: Scheduler Objects5489226 +Ref: library/sched id15489305 +Ref: 218e5489305 +Ref: library/sched scheduler-objects5489305 +Ref: 218f5489305 +Ref: library/sched sched scheduler enterabs5489436 +Ref: a705489436 +Ref: library/sched sched scheduler enter5490320 +Ref: a6f5490320 +Ref: library/sched sched scheduler cancel5490729 +Ref: 21905490729 +Ref: library/sched sched scheduler empty5490916 +Ref: 21915490916 +Ref: library/sched sched scheduler run5491001 +Ref: a6d5491001 +Ref: library/sched sched scheduler queue5492053 +Ref: 21925492053 +Node: queue — A synchronized queue class5492312 +Ref: library/queue doc5492481 +Ref: 21935492481 +Ref: library/queue module-queue5492481 +Ref: da5492481 +Ref: library/queue queue-a-synchronized-queue-class5492481 +Ref: 21945492481 +Ref: library/queue queue Queue5493804 +Ref: d185493804 +Ref: library/queue queue LifoQueue5494168 +Ref: 21955494168 +Ref: library/queue queue PriorityQueue5494536 +Ref: 21965494536 +Ref: library/queue queue SimpleQueue5495531 +Ref: 3c65495531 +Ref: library/queue queue Empty5495705 +Ref: 20a05495705 +Ref: library/queue queue Full5495890 +Ref: 20a15495890 +Ref: queue — A synchronized queue class-Footnote-15496161 +Node: Queue Objects5496225 +Ref: library/queue queue-objects5496339 +Ref: 219b5496339 +Ref: library/queue queueobjects5496339 +Ref: 219c5496339 +Ref: library/queue queue Queue qsize5496518 +Ref: 219d5496518 +Ref: library/queue queue Queue empty5496747 +Ref: 219e5496747 +Ref: library/queue queue Queue full5497076 +Ref: 219f5497076 +Ref: library/queue queue Queue put5497401 +Ref: 21995497401 +Ref: library/queue queue Queue put_nowait5497993 +Ref: 219a5497993 +Ref: library/queue queue Queue get5498074 +Ref: 21975498074 +Ref: library/queue queue Queue get_nowait5498978 +Ref: 21985498978 +Ref: library/queue queue Queue task_done5499171 +Ref: 209c5499171 +Ref: library/queue queue Queue join5499817 +Ref: 209d5499817 +Node: SimpleQueue Objects5500866 +Ref: library/queue simplequeue-objects5500980 +Ref: 21a05500980 +Ref: library/queue queue SimpleQueue qsize5501116 +Ref: 21a15501116 +Ref: library/queue queue SimpleQueue empty5501282 +Ref: 21a25501282 +Ref: library/queue queue SimpleQueue put5501499 +Ref: 21a35501499 +Ref: library/queue queue SimpleQueue put_nowait5502264 +Ref: 21a45502264 +Ref: library/queue queue SimpleQueue get5502413 +Ref: 21a55502413 +Ref: library/queue queue SimpleQueue get_nowait5502995 +Ref: 21a65502995 +Node: contextvars — Context Variables5503443 +Ref: library/contextvars doc5503622 +Ref: 21a75503622 +Ref: library/contextvars contextvars-context-variables5503622 +Ref: 21a85503622 +Ref: library/contextvars module-contextvars5503622 +Ref: 255503622 +Ref: contextvars — Context Variables-Footnote-15504479 +Node: Context Variables5504528 +Ref: library/contextvars context-variables5504649 +Ref: 21ac5504649 +Ref: library/contextvars contextvars ContextVar5504704 +Ref: 21a95504704 +Ref: library/contextvars contextvars ContextVar name5505411 +Ref: 21ae5505411 +Ref: library/contextvars contextvars ContextVar get5505538 +Ref: 21ad5505538 +Ref: library/contextvars contextvars ContextVar set5506025 +Ref: 21af5506025 +Ref: library/contextvars contextvars ContextVar reset5506421 +Ref: 21b15506421 +Ref: library/contextvars contextvars contextvars Token5506942 +Ref: 21b05506942 +Ref: library/contextvars contextvars contextvars Token Token var5507216 +Ref: 21b25507216 +Ref: library/contextvars contextvars contextvars Token Token old_value5507360 +Ref: 21b35507360 +Ref: library/contextvars contextvars contextvars Token Token MISSING5507651 +Ref: 21b45507651 +Node: Manual Context Management5507744 +Ref: library/contextvars manual-context-management5507889 +Ref: 21b55507889 +Ref: library/contextvars contextvars copy_context5507960 +Ref: 21aa5507960 +Ref: library/contextvars contextvars Context5508436 +Ref: 21ab5508436 +Ref: library/contextvars contextvars Context run5508774 +Ref: 21b65508774 +Ref: library/contextvars contextvars Context copy5510201 +Ref: 21b75510201 +Ref: library/contextvars contextvars Context get5510644 +Ref: 21b85510644 +Ref: library/contextvars contextvars Context keys5511086 +Ref: 21b95511086 +Ref: library/contextvars contextvars Context values5511177 +Ref: 21ba5511177 +Ref: library/contextvars contextvars Context items5511280 +Ref: 21bb5511280 +Node: asyncio support5511419 +Ref: library/contextvars asyncio-support5511538 +Ref: 21bc5511538 +Node: _thread — Low-level threading API5513113 +Ref: library/_thread doc5513316 +Ref: 21bd5513316 +Ref: library/_thread module-_thread5513316 +Ref: 35513316 +Ref: library/_thread thread-low-level-threading-api5513316 +Ref: 21be5513316 +Ref: library/_thread _thread error5514033 +Ref: 21bf5514033 +Ref: library/_thread _thread LockType5514201 +Ref: 21c05514201 +Ref: library/_thread _thread start_new_thread5514269 +Ref: 21c15514269 +Ref: library/_thread _thread interrupt_main5515161 +Ref: 21c25515161 +Ref: library/_thread _thread exit5515544 +Ref: 21c65515544 +Ref: library/_thread _thread allocate_lock5515693 +Ref: 21c75515693 +Ref: library/_thread _thread get_ident5515841 +Ref: 21c85515841 +Ref: library/_thread _thread get_native_id5516207 +Ref: 21c95516207 +Ref: library/_thread _thread stack_size5516665 +Ref: 21ca5516665 +Ref: library/_thread _thread TIMEOUT_MAX5517907 +Ref: 21cb5517907 +Ref: library/_thread _thread lock acquire5518187 +Ref: 21cc5518187 +Ref: library/_thread _thread lock release5519270 +Ref: 21cd5519270 +Ref: library/_thread _thread lock locked5519412 +Ref: 21ce5519412 +Node: _dummy_thread — Drop-in replacement for the _thread module5520807 +Ref: library/_dummy_thread doc5521041 +Ref: 21cf5521041 +Ref: library/_dummy_thread dummy-thread-drop-in-replacement-for-the-thread-module5521041 +Ref: 21d05521041 +Ref: library/_dummy_thread module-_dummy_thread5521041 +Ref: 25521041 +Ref: _dummy_thread — Drop-in replacement for the _thread module-Footnote-15521838 +Node: dummy_threading — Drop-in replacement for the threading module5521910 +Ref: library/dummy_threading doc5522100 +Ref: 21d15522100 +Ref: library/dummy_threading dummy-threading-drop-in-replacement-for-the-threading-module5522100 +Ref: 21d25522100 +Ref: library/dummy_threading module-dummy_threading5522100 +Ref: 685522100 +Ref: dummy_threading — Drop-in replacement for the threading module-Footnote-15522879 +Node: Networking and Interprocess Communication5522953 +Ref: library/ipc doc5523118 +Ref: 21d35523118 +Ref: library/ipc ipc5523118 +Ref: 21d45523118 +Ref: library/ipc networking-and-interprocess-communication5523118 +Ref: 21d55523118 +Node: asyncio — Asynchronous I/O5524022 +Ref: library/asyncio doc5524178 +Ref: 21d65524178 +Ref: library/asyncio asyncio-asynchronous-i-o5524178 +Ref: 21d75524178 +Ref: library/asyncio module-asyncio5524178 +Ref: a5524178 +Node: Coroutines and Tasks5525982 +Ref: library/asyncio-task doc5526083 +Ref: 21e15526083 +Ref: library/asyncio-task coroutines-and-tasks5526083 +Ref: 21e25526083 +Node: Coroutines<3>5526541 +Ref: library/asyncio-task coroutine5526630 +Ref: 21d85526630 +Ref: library/asyncio-task coroutines5526630 +Ref: 21e35526630 +Node: Awaitables5529055 +Ref: library/asyncio-task asyncio-awaitables5529179 +Ref: 21e45529179 +Ref: library/asyncio-task awaitables5529179 +Ref: 21e55529179 +Node: Running an asyncio Program5531816 +Ref: library/asyncio-task running-an-asyncio-program5531941 +Ref: 21e85531941 +Ref: library/asyncio-task asyncio run5532014 +Ref: 1a55532014 +Ref: Running an asyncio Program-Footnote-15532958 +Node: Creating Tasks5533032 +Ref: library/asyncio-task creating-tasks5533155 +Ref: 21e95533155 +Ref: library/asyncio-task asyncio create_task5533204 +Ref: 1ae5533204 +Node: Sleeping5534184 +Ref: library/asyncio-task sleeping5534307 +Ref: 21ea5534307 +Ref: library/asyncio-task asyncio sleep5534344 +Ref: 2855534344 +Ref: library/asyncio-task asyncio-example-sleep5534730 +Ref: 21eb5534730 +Node: Running Tasks Concurrently5535235 +Ref: library/asyncio-task running-tasks-concurrently5535371 +Ref: 21ec5535371 +Ref: library/asyncio-task asyncio gather5535444 +Ref: 2865535444 +Ref: library/asyncio-task asyncio-example-gather5536836 +Ref: 21ed5536836 +Node: Shielding From Cancellation5538492 +Ref: library/asyncio-task shielding-from-cancellation5538628 +Ref: 21ee5538628 +Ref: library/asyncio-task asyncio shield5538703 +Ref: 2875538703 +Node: Timeouts5539886 +Ref: library/asyncio-task timeouts5540014 +Ref: 21f05540014 +Ref: library/asyncio-task asyncio wait_for5540051 +Ref: 2885540051 +Ref: library/asyncio-task asyncio-example-waitfor5540904 +Ref: 21f25540904 +Node: Waiting Primitives5541586 +Ref: library/asyncio-task waiting-primitives5541716 +Ref: 21f35541716 +Ref: library/asyncio-task asyncio wait5541775 +Ref: 2895541775 +Ref: library/asyncio-task asyncio-example-wait-coroutine5544069 +Ref: 21f45544069 +Ref: library/asyncio-task asyncio as_completed5544974 +Ref: 28a5544974 +Node: Scheduling From Other Threads5545609 +Ref: library/asyncio-task scheduling-from-other-threads5545744 +Ref: 21f55545744 +Ref: library/asyncio-task asyncio run_coroutine_threadsafe5545825 +Ref: 51a5545825 +Node: Introspection5547258 +Ref: library/asyncio-task introspection5547386 +Ref: 21f75547386 +Ref: library/asyncio-task asyncio current_task5547435 +Ref: 3505547435 +Ref: library/asyncio-task asyncio all_tasks5547712 +Ref: 3515547712 +Node: Task Object5547970 +Ref: library/asyncio-task task-object5548095 +Ref: 21f85548095 +Ref: library/asyncio-task asyncio Task5548140 +Ref: 1ad5548140 +Ref: library/asyncio-task asyncio Task cancel5550156 +Ref: 21ef5550156 +Ref: library/asyncio-task asyncio-example-task-cancel5550865 +Ref: 21fd5550865 +Ref: library/asyncio-task asyncio Task cancelled5552122 +Ref: 21f95552122 +Ref: library/asyncio-task asyncio Task done5552432 +Ref: 21fe5552432 +Ref: library/asyncio-task asyncio Task result5552648 +Ref: 21ff5552648 +Ref: library/asyncio-task asyncio Task exception5553136 +Ref: 22015553136 +Ref: library/asyncio-task asyncio Task add_done_callback5553623 +Ref: 22025553623 +Ref: library/asyncio-task asyncio Task remove_done_callback5553938 +Ref: 22035553938 +Ref: library/asyncio-task asyncio Task get_stack5554236 +Ref: 22055554236 +Ref: library/asyncio-task asyncio Task print_stack5555232 +Ref: 22065555232 +Ref: library/asyncio-task asyncio Task get_coro5555724 +Ref: 1ac5555724 +Ref: library/asyncio-task asyncio Task get_name5555856 +Ref: 1b15555856 +Ref: library/asyncio-task asyncio Task set_name5556128 +Ref: 1b05556128 +Ref: library/asyncio-task asyncio Task all_tasks5556462 +Ref: 3535556462 +Ref: library/asyncio-task asyncio Task current_task5556950 +Ref: 3525556950 +Node: Generator-based Coroutines5557376 +Ref: library/asyncio-task asyncio-generator-based-coro5557479 +Ref: 21e65557479 +Ref: library/asyncio-task generator-based-coroutines5557479 +Ref: 22075557479 +Ref: library/asyncio-task asyncio coroutine5557950 +Ref: 2825557950 +Ref: library/asyncio-task asyncio iscoroutine5558516 +Ref: 22085558516 +Ref: library/asyncio-task asyncio iscoroutinefunction5558762 +Ref: 22095558762 +Node: Streams5559079 +Ref: library/asyncio-stream doc5559215 +Ref: 220a5559215 +Ref: library/asyncio-stream asyncio-streams5559215 +Ref: 21d95559215 +Ref: library/asyncio-stream streams5559215 +Ref: 220b5559215 +Ref: library/asyncio-stream asyncio-example-stream5559552 +Ref: 220c5559552 +Ref: library/asyncio-stream asyncio open_connection5560298 +Ref: 3635560298 +Ref: library/asyncio-stream asyncio start_server5561225 +Ref: 3645561225 +Ref: library/asyncio-stream asyncio open_unix_connection5562524 +Ref: 22105562524 +Ref: library/asyncio-stream asyncio start_unix_server5563165 +Ref: 22125563165 +Ref: Streams-Footnote-15563891 +Node: StreamReader5563965 +Ref: library/asyncio-stream streamreader5564042 +Ref: 22135564042 +Ref: library/asyncio-stream asyncio StreamReader5564089 +Ref: 220e5564089 +Ref: library/asyncio-stream asyncio StreamReader read5564372 +Ref: 22145564372 +Ref: library/asyncio-stream asyncio StreamReader readline5564653 +Ref: 22155564653 +Ref: library/asyncio-stream asyncio StreamReader readexactly5565009 +Ref: 51f5565009 +Ref: library/asyncio-stream asyncio StreamReader readuntil5565305 +Ref: 51e5565305 +Ref: library/asyncio-stream asyncio StreamReader at_eof5566154 +Ref: 22195566154 +Node: StreamWriter5566277 +Ref: library/asyncio-stream streamwriter5566375 +Ref: 221a5566375 +Ref: library/asyncio-stream asyncio StreamWriter5566422 +Ref: 220f5566422 +Ref: library/asyncio-stream asyncio StreamWriter write5566704 +Ref: 221b5566704 +Ref: library/asyncio-stream asyncio StreamWriter writelines5567074 +Ref: 221c5567074 +Ref: library/asyncio-stream asyncio StreamWriter close5567467 +Ref: 221d5567467 +Ref: library/asyncio-stream asyncio StreamWriter can_write_eof5567724 +Ref: 221e5567724 +Ref: library/asyncio-stream asyncio StreamWriter write_eof5567895 +Ref: 221f5567895 +Ref: library/asyncio-stream asyncio StreamWriter transport5568023 +Ref: 22205568023 +Ref: library/asyncio-stream asyncio StreamWriter get_extra_info5568106 +Ref: 22215568106 +Ref: library/asyncio-stream asyncio StreamWriter drain5568280 +Ref: 22225568280 +Ref: library/asyncio-stream asyncio StreamWriter is_closing5568873 +Ref: 3575568873 +Ref: library/asyncio-stream asyncio StreamWriter wait_closed5569034 +Ref: 3565569034 +Node: Examples<11>5569268 +Ref: library/asyncio-stream examples5569345 +Ref: 220d5569345 +Node: TCP echo client using streams5569542 +Ref: library/asyncio-stream asyncio-tcp-echo-client-streams5569658 +Ref: 22235569658 +Ref: library/asyncio-stream tcp-echo-client-using-streams5569658 +Ref: 22245569658 +Node: TCP echo server using streams5570380 +Ref: library/asyncio-stream asyncio-tcp-echo-server-streams5570521 +Ref: 22265570521 +Ref: library/asyncio-stream tcp-echo-server-using-streams5570521 +Ref: 22275570521 +Node: Get HTTP headers5571495 +Ref: library/asyncio-stream get-http-headers5571661 +Ref: 22295571661 +Node: Register an open socket to wait for data using streams5572896 +Ref: library/asyncio-stream asyncio-example-create-connection-streams5573024 +Ref: 222a5573024 +Ref: library/asyncio-stream register-an-open-socket-to-wait-for-data-using-streams5573024 +Ref: 222b5573024 +Node: Synchronization Primitives5574413 +Ref: library/asyncio-sync doc5574541 +Ref: 222f5574541 +Ref: library/asyncio-sync asyncio-sync5574541 +Ref: 21dc5574541 +Ref: library/asyncio-sync synchronization-primitives5574541 +Ref: 22305574541 +Ref: Synchronization Primitives-Footnote-15575582 +Node: Lock5575654 +Ref: library/asyncio-sync lock5575735 +Ref: 22315575735 +Ref: library/asyncio-sync asyncio Lock5575766 +Ref: 28b5575766 +Ref: library/asyncio-sync asyncio Lock acquire5576478 +Ref: 22325576478 +Ref: library/asyncio-sync asyncio Lock release5576972 +Ref: 22335576972 +Ref: library/asyncio-sync asyncio Lock locked5577189 +Ref: 22345577189 +Node: Event5577271 +Ref: library/asyncio-sync event5577370 +Ref: 22355577370 +Ref: library/asyncio-sync asyncio Event5577403 +Ref: 28c5577403 +Ref: library/asyncio-sync asyncio-example-sync-event5577982 +Ref: 22395577982 +Ref: library/asyncio-sync asyncio Event wait5578611 +Ref: 22385578611 +Ref: library/asyncio-sync asyncio Event set5578821 +Ref: 22365578821 +Ref: library/asyncio-sync asyncio Event clear5578961 +Ref: 22375578961 +Ref: library/asyncio-sync asyncio Event is_set5579151 +Ref: 223a5579151 +Node: Condition5579229 +Ref: library/asyncio-sync condition5579333 +Ref: 223b5579333 +Ref: library/asyncio-sync asyncio Condition5579374 +Ref: 28d5579374 +Ref: library/asyncio-sync asyncio Condition acquire5580655 +Ref: 223c5580655 +Ref: library/asyncio-sync asyncio Condition notify5580858 +Ref: 223d5580858 +Ref: library/asyncio-sync asyncio Condition locked5581217 +Ref: 223e5581217 +Ref: library/asyncio-sync asyncio Condition notify_all5581310 +Ref: 223f5581310 +Ref: library/asyncio-sync asyncio Condition release5581688 +Ref: 22405581688 +Ref: library/asyncio-sync asyncio Condition wait5581850 +Ref: 22415581850 +Ref: library/asyncio-sync asyncio Condition wait_for5582320 +Ref: 22425582320 +Node: Semaphore5582574 +Ref: library/asyncio-sync semaphore5582689 +Ref: 22435582689 +Ref: library/asyncio-sync asyncio Semaphore5582730 +Ref: 28e5582730 +Ref: library/asyncio-sync asyncio Semaphore acquire5583891 +Ref: 22445583891 +Ref: library/asyncio-sync asyncio Semaphore locked5584179 +Ref: 22465584179 +Ref: library/asyncio-sync asyncio Semaphore release5584283 +Ref: 22455584283 +Node: BoundedSemaphore5584597 +Ref: library/asyncio-sync boundedsemaphore5584694 +Ref: 22475584694 +Ref: library/asyncio-sync asyncio BoundedSemaphore5584749 +Ref: 28f5584749 +Node: Subprocesses5585460 +Ref: library/asyncio-subprocess doc5585587 +Ref: 22485585587 +Ref: library/asyncio-subprocess asyncio-subprocess5585587 +Ref: 21da5585587 +Ref: library/asyncio-subprocess subprocesses5585587 +Ref: 22495585587 +Ref: library/asyncio-subprocess asyncio-example-subprocess-shell5585875 +Ref: 224a5585875 +Ref: Subprocesses-Footnote-15587171 +Ref: Subprocesses-Footnote-25587249 +Node: Creating Subprocesses5587332 +Ref: library/asyncio-subprocess creating-subprocesses5587423 +Ref: 224c5587423 +Ref: library/asyncio-subprocess asyncio create_subprocess_exec5587488 +Ref: 2915587488 +Ref: library/asyncio-subprocess asyncio create_subprocess_shell5588145 +Ref: 2925588145 +Ref: Creating Subprocesses-Footnote-15589748 +Node: Constants<7>5589818 +Ref: library/asyncio-subprocess constants5589947 +Ref: 22545589947 +Ref: library/asyncio-subprocess asyncio asyncio subprocess PIPE5589988 +Ref: 22555589988 +Ref: library/asyncio-subprocess asyncio asyncio subprocess STDOUT5590434 +Ref: 22595590434 +Ref: library/asyncio-subprocess asyncio asyncio subprocess DEVNULL5590620 +Ref: 225a5590620 +Node: Interacting with Subprocesses5590901 +Ref: library/asyncio-subprocess interacting-with-subprocesses5591000 +Ref: 225b5591000 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process5591331 +Ref: 224d5591331 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process wait5592309 +Ref: 225d5592309 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process communicate5592818 +Ref: 225c5592818 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process send_signal5594054 +Ref: 22605594054 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process terminate5594446 +Ref: 22615594446 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process kill5594735 +Ref: 22635594735 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process stdin5594965 +Ref: 22565594965 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process stdout5595127 +Ref: 22575595127 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process stderr5595292 +Ref: 22585595292 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process pid5595795 +Ref: 22645595795 +Ref: library/asyncio-subprocess asyncio asyncio subprocess Process returncode5596034 +Ref: 225f5596034 +Node: Subprocess and Threads5596392 +Ref: library/asyncio-subprocess asyncio-subprocess-threads5596501 +Ref: 225e5596501 +Ref: library/asyncio-subprocess subprocess-and-threads5596501 +Ref: 22655596501 +Node: Examples<12>5597371 +Ref: library/asyncio-subprocess examples5597480 +Ref: 224b5597480 +Ref: library/asyncio-subprocess asyncio-example-create-subprocess-exec5597664 +Ref: 22695597664 +Node: Queues5598467 +Ref: library/asyncio-queue doc5598581 +Ref: 226b5598581 +Ref: library/asyncio-queue asyncio-queues5598581 +Ref: 21db5598581 +Ref: library/asyncio-queue queues5598581 +Ref: 226c5598581 +Ref: Queues-Footnote-15599272 +Node: Queue5599345 +Ref: library/asyncio-queue queue5599416 +Ref: 226e5599416 +Ref: library/asyncio-queue asyncio Queue5599449 +Ref: 2905599449 +Ref: library/asyncio-queue asyncio Queue maxsize5600107 +Ref: 22715600107 +Ref: library/asyncio-queue asyncio Queue empty5600185 +Ref: 22725600185 +Ref: library/asyncio-queue asyncio Queue full5600287 +Ref: 22735600287 +Ref: library/asyncio-queue asyncio Queue get5600543 +Ref: 226f5600543 +Ref: library/asyncio-queue asyncio Queue get_nowait5600694 +Ref: 22745600694 +Ref: library/asyncio-queue asyncio Queue join5600830 +Ref: 69f5600830 +Ref: library/asyncio-queue asyncio Queue put5601328 +Ref: 22765601328 +Ref: library/asyncio-queue asyncio Queue put_nowait5601499 +Ref: 22775601499 +Ref: library/asyncio-queue asyncio Queue qsize5601683 +Ref: 22705601683 +Ref: library/asyncio-queue asyncio Queue task_done5601762 +Ref: 6a05601762 +Node: Priority Queue5602445 +Ref: library/asyncio-queue priority-queue5602535 +Ref: 22795602535 +Ref: library/asyncio-queue asyncio PriorityQueue5602586 +Ref: 227a5602586 +Node: LIFO Queue5602793 +Ref: library/asyncio-queue lifo-queue5602891 +Ref: 227b5602891 +Ref: library/asyncio-queue asyncio LifoQueue5602934 +Ref: 227c5602934 +Node: Exceptions<8>5603077 +Ref: library/asyncio-queue exceptions5603173 +Ref: 227d5603173 +Ref: library/asyncio-queue asyncio QueueEmpty5603216 +Ref: 22755603216 +Ref: library/asyncio-queue asyncio QueueFull5603358 +Ref: 22785603358 +Node: Examples<13>5603515 +Ref: library/asyncio-queue examples5603592 +Ref: 226d5603592 +Ref: library/asyncio-queue asyncio-example-queue-dist5603631 +Ref: 227e5603631 +Node: Exceptions<9>5605389 +Ref: library/asyncio-exceptions doc5605501 +Ref: 227f5605501 +Ref: library/asyncio-exceptions asyncio-exceptions5605501 +Ref: 22805605501 +Ref: library/asyncio-exceptions exceptions5605501 +Ref: 22815605501 +Ref: library/asyncio-exceptions asyncio TimeoutError5605657 +Ref: 21f15605657 +Ref: library/asyncio-exceptions asyncio CancelledError5605860 +Ref: 1a75605860 +Ref: library/asyncio-exceptions asyncio InvalidStateError5606215 +Ref: 22005606215 +Ref: library/asyncio-exceptions asyncio SendfileNotAvailableError5606454 +Ref: 22825606454 +Ref: library/asyncio-exceptions asyncio IncompleteReadError5606639 +Ref: 22165606639 +Ref: library/asyncio-exceptions asyncio IncompleteReadError expected5606854 +Ref: 22835606854 +Ref: library/asyncio-exceptions asyncio IncompleteReadError partial5606949 +Ref: 22175606949 +Ref: library/asyncio-exceptions asyncio LimitOverrunError5607072 +Ref: 22185607072 +Ref: library/asyncio-exceptions asyncio LimitOverrunError consumed5607234 +Ref: 22845607234 +Ref: Exceptions<9>-Footnote-15607353 +Node: Event Loop5607431 +Ref: library/asyncio-eventloop doc5607544 +Ref: 22855607544 +Ref: library/asyncio-eventloop event-loop5607544 +Ref: 22865607544 +Ref: library/asyncio-eventloop asyncio get_running_loop5608374 +Ref: 3545608374 +Ref: library/asyncio-eventloop asyncio get_event_loop5608659 +Ref: 3555608659 +Ref: library/asyncio-eventloop asyncio set_event_loop5609379 +Ref: 22875609379 +Ref: library/asyncio-eventloop asyncio new_event_loop5609492 +Ref: 22885609492 +Ref: Event Loop-Footnote-15610761 +Ref: Event Loop-Footnote-25610834 +Node: Event Loop Methods5610913 +Ref: library/asyncio-eventloop asyncio-event-loop5611003 +Ref: 6445611003 +Ref: library/asyncio-eventloop event-loop-methods5611003 +Ref: 228a5611003 +Node: Running and stopping the loop5611586 +Ref: library/asyncio-eventloop running-and-stopping-the-loop5611699 +Ref: 22915611699 +Ref: library/asyncio-eventloop asyncio loop run_until_complete5611780 +Ref: 5185611780 +Ref: library/asyncio-eventloop asyncio loop run_forever5612096 +Ref: 22925612096 +Ref: library/asyncio-eventloop asyncio loop stop5612816 +Ref: 5205612816 +Ref: library/asyncio-eventloop asyncio loop is_running5612869 +Ref: 22935612869 +Ref: library/asyncio-eventloop asyncio loop is_closed5612965 +Ref: 69b5612965 +Ref: library/asyncio-eventloop asyncio loop close5613050 +Ref: 22945613050 +Ref: library/asyncio-eventloop asyncio loop shutdown_asyncgens5613455 +Ref: 5215613455 +Node: Scheduling callbacks5614139 +Ref: library/asyncio-eventloop scheduling-callbacks5614289 +Ref: 22955614289 +Ref: library/asyncio-eventloop asyncio loop call_soon5614352 +Ref: 3495614352 +Ref: library/asyncio-eventloop asyncio loop call_soon_threadsafe5615026 +Ref: 34a5615026 +Ref: library/asyncio-eventloop asyncio-pass-keywords5615415 +Ref: 22975615415 +Ref: Scheduling callbacks-Footnote-15615910 +Node: Scheduling delayed callbacks5615959 +Ref: library/asyncio-eventloop asyncio-delayed-calls5616106 +Ref: 22985616106 +Ref: library/asyncio-eventloop scheduling-delayed-callbacks5616106 +Ref: 22995616106 +Ref: library/asyncio-eventloop asyncio loop call_later5616339 +Ref: 34b5616339 +Ref: library/asyncio-eventloop asyncio loop call_at5617510 +Ref: 34c5617510 +Ref: library/asyncio-eventloop asyncio loop time5618270 +Ref: 229a5618270 +Ref: Scheduling delayed callbacks-Footnote-15618705 +Ref: Scheduling delayed callbacks-Footnote-25618754 +Node: Creating Futures and Tasks5618803 +Ref: library/asyncio-eventloop creating-futures-and-tasks5618957 +Ref: 229b5618957 +Ref: library/asyncio-eventloop asyncio loop create_future5619032 +Ref: 51c5619032 +Ref: library/asyncio-eventloop asyncio loop create_task5619388 +Ref: 1af5619388 +Ref: library/asyncio-eventloop asyncio loop set_task_factory5619907 +Ref: 69d5619907 +Ref: library/asyncio-eventloop asyncio loop get_task_factory5620377 +Ref: 69e5620377 +Node: Opening network connections5620487 +Ref: library/asyncio-eventloop opening-network-connections5620637 +Ref: 229c5620637 +Ref: library/asyncio-eventloop asyncio loop create_connection5620714 +Ref: 1b25620714 +Ref: library/asyncio-eventloop asyncio loop create_datagram_endpoint5626532 +Ref: 2e75626532 +Ref: library/asyncio-eventloop asyncio loop create_unix_connection5629986 +Ref: 22115629986 +Ref: Opening network connections-Footnote-15630950 +Ref: Opening network connections-Footnote-25630999 +Node: Creating network servers5631048 +Ref: library/asyncio-eventloop creating-network-servers5631190 +Ref: 22a65631190 +Ref: library/asyncio-eventloop asyncio loop create_server5631261 +Ref: 35d5631261 +Ref: library/asyncio-eventloop asyncio loop create_unix_server5634906 +Ref: 35e5634906 +Ref: library/asyncio-eventloop asyncio loop connect_accepted_socket5635764 +Ref: 3655635764 +Node: Transferring files5636794 +Ref: library/asyncio-eventloop transferring-files5636920 +Ref: 22a75636920 +Ref: library/asyncio-eventloop asyncio loop sendfile5636979 +Ref: 22a85636979 +Node: TLS Upgrade5638037 +Ref: library/asyncio-eventloop tls-upgrade5638164 +Ref: 22a95638164 +Ref: library/asyncio-eventloop asyncio loop start_tls5638209 +Ref: 34e5638209 +Node: Watching file descriptors5639433 +Ref: library/asyncio-eventloop watching-file-descriptors5639578 +Ref: 22aa5639578 +Ref: library/asyncio-eventloop asyncio loop add_reader5639651 +Ref: 222e5639651 +Ref: library/asyncio-eventloop asyncio loop remove_reader5639869 +Ref: 22ab5639869 +Ref: library/asyncio-eventloop asyncio loop add_writer5639976 +Ref: 22ac5639976 +Ref: library/asyncio-eventloop asyncio loop remove_writer5640296 +Ref: 22ad5640296 +Node: Working with socket objects directly5640491 +Ref: library/asyncio-eventloop working-with-socket-objects-directly5640628 +Ref: 22af5640628 +Ref: library/asyncio-eventloop asyncio loop sock_recv5641079 +Ref: 49e5641079 +Ref: library/asyncio-eventloop asyncio loop sock_recv_into5641548 +Ref: 34f5641548 +Ref: library/asyncio-eventloop asyncio loop sock_sendall5641855 +Ref: 49f5641855 +Ref: library/asyncio-eventloop asyncio loop sock_connect5642612 +Ref: 6a15642612 +Ref: library/asyncio-eventloop asyncio loop sock_accept5643233 +Ref: 4a05643233 +Ref: library/asyncio-eventloop asyncio loop sock_sendfile5644057 +Ref: 3585644057 +Node: DNS5645258 +Ref: library/asyncio-eventloop dns5645388 +Ref: 22b05645388 +Ref: library/asyncio-eventloop asyncio loop getaddrinfo5645417 +Ref: 4a15645417 +Ref: library/asyncio-eventloop asyncio loop getnameinfo5645583 +Ref: 4a25645583 +Node: Working with pipes5645967 +Ref: library/asyncio-eventloop working-with-pipes5646073 +Ref: 22b35646073 +Ref: library/asyncio-eventloop asyncio loop connect_read_pipe5646132 +Ref: 22505646132 +Ref: library/asyncio-eventloop asyncio loop connect_write_pipe5646707 +Ref: 22515646707 +Node: Unix signals5647545 +Ref: library/asyncio-eventloop unix-signals5647689 +Ref: 22b65647689 +Ref: library/asyncio-eventloop asyncio loop add_signal_handler5647736 +Ref: 21de5647736 +Ref: library/asyncio-eventloop asyncio loop remove_signal_handler5648524 +Ref: 22b75648524 +Node: Executing code in thread or process pools5648828 +Ref: library/asyncio-eventloop executing-code-in-thread-or-process-pools5648972 +Ref: 22b85648972 +Ref: library/asyncio-eventloop asyncio loop run_in_executor5649077 +Ref: 21e75649077 +Ref: library/asyncio-eventloop asyncio loop set_default_executor5651221 +Ref: 27b5651221 +Node: Error Handling API5651705 +Ref: library/asyncio-eventloop error-handling-api5651856 +Ref: 22b95651856 +Ref: library/asyncio-eventloop asyncio loop set_exception_handler5651981 +Ref: 22ba5651981 +Ref: library/asyncio-eventloop asyncio loop get_exception_handler5652509 +Ref: 51d5652509 +Ref: library/asyncio-eventloop asyncio loop default_exception_handler5652683 +Ref: 22bc5652683 +Ref: library/asyncio-eventloop asyncio loop call_exception_handler5653058 +Ref: 22bb5653058 +Node: Enabling debug mode5653937 +Ref: library/asyncio-eventloop enabling-debug-mode5654067 +Ref: 22bd5654067 +Ref: library/asyncio-eventloop asyncio loop get_debug5654128 +Ref: 69a5654128 +Ref: library/asyncio-eventloop asyncio loop set_debug5654382 +Ref: 6995654382 +Node: Running Subprocesses5654651 +Ref: library/asyncio-eventloop running-subprocesses5654754 +Ref: 22be5654754 +Ref: library/asyncio-eventloop asyncio loop subprocess_exec5655214 +Ref: 21dd5655214 +Ref: library/asyncio-eventloop asyncio loop subprocess_shell5659206 +Ref: 224e5659206 +Ref: Running Subprocesses-Footnote-15660526 +Node: Callback Handles5660596 +Ref: library/asyncio-eventloop callback-handles5660709 +Ref: 228b5660709 +Ref: library/asyncio-eventloop asyncio Handle5660764 +Ref: 228c5660764 +Ref: library/asyncio-eventloop asyncio Handle cancel5660909 +Ref: 22c15660909 +Ref: library/asyncio-eventloop asyncio Handle cancelled5661062 +Ref: 3665661062 +Ref: library/asyncio-eventloop asyncio TimerHandle5661184 +Ref: 228d5661184 +Ref: library/asyncio-eventloop asyncio TimerHandle when5661380 +Ref: 3625661380 +Node: Server Objects5661625 +Ref: library/asyncio-eventloop server-objects5661746 +Ref: 228e5661746 +Ref: library/asyncio-eventloop asyncio Server5662014 +Ref: 35c5662014 +Ref: library/asyncio-eventloop asyncio Server close5662577 +Ref: 22c25662577 +Ref: library/asyncio-eventloop asyncio Server get_loop5662969 +Ref: 35b5662969 +Ref: library/asyncio-eventloop asyncio Server start_serving5663098 +Ref: 35f5663098 +Ref: library/asyncio-eventloop asyncio Server serve_forever5663722 +Ref: 3605663722 +Ref: library/asyncio-eventloop asyncio Server is_serving5664619 +Ref: 3615664619 +Ref: library/asyncio-eventloop asyncio Server wait_closed5664755 +Ref: 22c35664755 +Ref: library/asyncio-eventloop asyncio Server sockets5664863 +Ref: 4a35664863 +Node: Event Loop Implementations5665179 +Ref: library/asyncio-eventloop asyncio-event-loops5665296 +Ref: 22c45665296 +Ref: library/asyncio-eventloop event-loop-implementations5665296 +Ref: 228f5665296 +Ref: library/asyncio-eventloop asyncio SelectorEventLoop5665624 +Ref: 22665665624 +Ref: library/asyncio-eventloop asyncio ProactorEventLoop5666139 +Ref: 1ab5666139 +Ref: library/asyncio-eventloop asyncio AbstractEventLoop5666377 +Ref: 22c55666377 +Ref: Event Loop Implementations-Footnote-15666672 +Node: Examples<14>5666758 +Ref: library/asyncio-eventloop examples5666852 +Ref: 22905666852 +Node: Hello World with call_soon5667451 +Ref: library/asyncio-eventloop asyncio-example-lowlevel-helloworld5667575 +Ref: 22c65667575 +Ref: library/asyncio-eventloop hello-world-with-call-soon5667575 +Ref: 22c75667575 +Node: Display the current date with call_later5668349 +Ref: library/asyncio-eventloop asyncio-example-call-later5668521 +Ref: 22c85668521 +Ref: library/asyncio-eventloop display-the-current-date-with-call-later5668521 +Ref: 22c95668521 +Node: Watch a file descriptor for read events5669513 +Ref: library/asyncio-eventloop asyncio-example-watch-fd5669701 +Ref: 222d5669701 +Ref: library/asyncio-eventloop watch-a-file-descriptor-for-read-events5669701 +Ref: 22ca5669701 +Node: Set signal handlers for SIGINT and SIGTERM5671021 +Ref: library/asyncio-eventloop asyncio-example-unix-signals5671160 +Ref: 22cb5671160 +Ref: library/asyncio-eventloop set-signal-handlers-for-sigint-and-sigterm5671160 +Ref: 22cc5671160 +Node: Futures5672075 +Ref: library/asyncio-future doc5672199 +Ref: 22cd5672199 +Ref: library/asyncio-future asyncio-futures5672199 +Ref: 21e05672199 +Ref: library/asyncio-future futures5672199 +Ref: 22ce5672199 +Ref: Futures-Footnote-15672566 +Ref: Futures-Footnote-25672640 +Node: Future Functions5672720 +Ref: library/asyncio-future future-functions5672802 +Ref: 22cf5672802 +Ref: library/asyncio-future asyncio isfuture5672857 +Ref: 22d05672857 +Ref: library/asyncio-future asyncio ensure_future5673163 +Ref: 5175673163 +Ref: library/asyncio-future asyncio wrap_future5674104 +Ref: 22d15674104 +Node: Future Object5674265 +Ref: library/asyncio-future future-object5674347 +Ref: 22d25674347 +Ref: library/asyncio-future asyncio Future5674396 +Ref: 4435674396 +Ref: library/asyncio-future asyncio Future result5675318 +Ref: 22d35675318 +Ref: library/asyncio-future asyncio Future set_result5675933 +Ref: 21fa5675933 +Ref: library/asyncio-future asyncio Future set_exception5676128 +Ref: 21fb5676128 +Ref: library/asyncio-future asyncio Future done5676331 +Ref: 22d45676331 +Ref: library/asyncio-future asyncio Future cancelled5676595 +Ref: 22d55676595 +Ref: library/asyncio-future asyncio Future add_done_callback5676900 +Ref: 34d5676900 +Ref: library/asyncio-future asyncio Future remove_done_callback5677889 +Ref: 22045677889 +Ref: library/asyncio-future asyncio Future cancel5678121 +Ref: 21fc5678121 +Ref: library/asyncio-future asyncio Future exception5678404 +Ref: 22d65678404 +Ref: library/asyncio-future asyncio Future get_loop5678848 +Ref: 35a5678848 +Ref: library/asyncio-future asyncio-example-future5678973 +Ref: 22d75678973 +Ref: Future Object-Footnote-15681010 +Node: Transports and Protocols5681059 +Ref: library/asyncio-protocol doc5681181 +Ref: 22d85681181 +Ref: library/asyncio-protocol asyncio-transports-protocols5681181 +Ref: 21df5681181 +Ref: library/asyncio-protocol transports-and-protocols5681181 +Ref: 22d95681181 +Node: Transports5683549 +Ref: library/asyncio-protocol asyncio-transport5683638 +Ref: 22a05683638 +Ref: library/asyncio-protocol transports5683638 +Ref: 22da5683638 +Ref: Transports-Footnote-15684380 +Node: Transports Hierarchy5684458 +Ref: library/asyncio-protocol transports-hierarchy5684548 +Ref: 22e35684548 +Ref: library/asyncio-protocol asyncio BaseTransport5684611 +Ref: 22dc5684611 +Ref: library/asyncio-protocol asyncio WriteTransport5684740 +Ref: 22b55684740 +Ref: library/asyncio-protocol asyncio ReadTransport5685071 +Ref: 22b45685071 +Ref: library/asyncio-protocol asyncio Transport5685398 +Ref: 22dd5685398 +Ref: library/asyncio-protocol asyncio DatagramTransport5686001 +Ref: 22de5686001 +Ref: library/asyncio-protocol asyncio SubprocessTransport5686244 +Ref: 22c05686244 +Node: Base Transport5686570 +Ref: library/asyncio-protocol base-transport5686689 +Ref: 22e45686689 +Ref: library/asyncio-protocol asyncio BaseTransport close5686740 +Ref: 22e55686740 +Ref: library/asyncio-protocol asyncio BaseTransport is_closing5687111 +Ref: 51b5687111 +Ref: library/asyncio-protocol asyncio BaseTransport get_extra_info5687218 +Ref: 2c85687218 +Ref: library/asyncio-protocol asyncio BaseTransport set_protocol5689375 +Ref: 22ea5689375 +Ref: library/asyncio-protocol asyncio BaseTransport get_protocol5689560 +Ref: 22eb5689560 +Node: Read-only Transports5689638 +Ref: library/asyncio-protocol read-only-transports5689758 +Ref: 22ec5689758 +Ref: library/asyncio-protocol asyncio ReadTransport is_reading5689821 +Ref: 3675689821 +Ref: library/asyncio-protocol asyncio ReadTransport pause_reading5689952 +Ref: 3695689952 +Ref: library/asyncio-protocol asyncio ReadTransport resume_reading5690321 +Ref: 3685690321 +Node: Write-only Transports5690659 +Ref: library/asyncio-protocol write-only-transports5690784 +Ref: 22ee5690784 +Ref: library/asyncio-protocol asyncio WriteTransport abort5690849 +Ref: 22ef5690849 +Ref: library/asyncio-protocol asyncio WriteTransport can_write_eof5691188 +Ref: 22f05691188 +Ref: library/asyncio-protocol asyncio WriteTransport get_write_buffer_size5691345 +Ref: 22f25691345 +Ref: library/asyncio-protocol asyncio WriteTransport get_write_buffer_limits5691472 +Ref: 69c5691472 +Ref: library/asyncio-protocol asyncio WriteTransport set_write_buffer_limits5691786 +Ref: 22f35691786 +Ref: library/asyncio-protocol asyncio WriteTransport write5693198 +Ref: 22f65693198 +Ref: library/asyncio-protocol asyncio WriteTransport writelines5693397 +Ref: 22f75693397 +Ref: library/asyncio-protocol asyncio WriteTransport write_eof5693681 +Ref: 22f15693681 +Node: Datagram Transports5693970 +Ref: library/asyncio-protocol datagram-transports5694096 +Ref: 22f85694096 +Ref: library/asyncio-protocol asyncio DatagramTransport sendto5694157 +Ref: 22f95694157 +Ref: library/asyncio-protocol asyncio DatagramTransport abort5694535 +Ref: 22fa5694535 +Node: Subprocess Transports5694877 +Ref: library/asyncio-protocol asyncio-subprocess-transports5694973 +Ref: 22525694973 +Ref: library/asyncio-protocol subprocess-transports5694973 +Ref: 22fb5694973 +Ref: library/asyncio-protocol asyncio SubprocessTransport get_pid5695038 +Ref: 22fc5695038 +Ref: library/asyncio-protocol asyncio SubprocessTransport get_pipe_transport5695136 +Ref: 22fd5695136 +Ref: library/asyncio-protocol asyncio SubprocessTransport get_returncode5695882 +Ref: 22fe5695882 +Ref: library/asyncio-protocol asyncio SubprocessTransport kill5696118 +Ref: 22ff5696118 +Ref: library/asyncio-protocol asyncio SubprocessTransport send_signal5696379 +Ref: 23015696379 +Ref: library/asyncio-protocol asyncio SubprocessTransport terminate5696538 +Ref: 23005696538 +Ref: library/asyncio-protocol asyncio SubprocessTransport close5696838 +Ref: 23025696838 +Node: Protocols5697064 +Ref: library/asyncio-protocol asyncio-protocol5697174 +Ref: 229f5697174 +Ref: library/asyncio-protocol protocols5697174 +Ref: 22db5697174 +Ref: Protocols-Footnote-15697967 +Node: Base Protocols5698044 +Ref: library/asyncio-protocol base-protocols5698126 +Ref: 23035698126 +Ref: library/asyncio-protocol asyncio BaseProtocol5698177 +Ref: 22df5698177 +Ref: library/asyncio-protocol asyncio Protocol5698269 +Ref: 22e05698269 +Ref: library/asyncio-protocol asyncio BufferedProtocol5698402 +Ref: 2c95698402 +Ref: library/asyncio-protocol asyncio DatagramProtocol5698558 +Ref: 22e15698558 +Ref: library/asyncio-protocol asyncio SubprocessProtocol5698674 +Ref: 22bf5698674 +Node: Base Protocol5698840 +Ref: library/asyncio-protocol base-protocol5698950 +Ref: 23045698950 +Ref: library/asyncio-protocol asyncio BaseProtocol connection_made5699273 +Ref: 22a15699273 +Ref: library/asyncio-protocol asyncio BaseProtocol connection_lost5699527 +Ref: 22e65699527 +Ref: library/asyncio-protocol asyncio BaseProtocol pause_writing5700070 +Ref: 22f45700070 +Ref: library/asyncio-protocol asyncio BaseProtocol resume_writing5700186 +Ref: 22f55700186 +Node: Streaming Protocols5700650 +Ref: library/asyncio-protocol streaming-protocols5700774 +Ref: 23055700774 +Ref: library/asyncio-protocol asyncio Protocol data_received5701184 +Ref: 22ed5701184 +Ref: library/asyncio-protocol asyncio Protocol eof_received5701865 +Ref: 23065701865 +Node: Buffered Streaming Protocols5702729 +Ref: library/asyncio-protocol buffered-streaming-protocols5702858 +Ref: 23075702858 +Ref: library/asyncio-protocol asyncio BufferedProtocol get_buffer5703724 +Ref: 23085703724 +Ref: library/asyncio-protocol asyncio BufferedProtocol buffer_updated5704198 +Ref: 23095704198 +Ref: library/asyncio-protocol asyncio BufferedProtocol eof_received5704399 +Ref: 230a5704399 +Node: Datagram Protocols5704971 +Ref: library/asyncio-protocol datagram-protocols5705101 +Ref: 230b5705101 +Ref: library/asyncio-protocol asyncio DatagramProtocol datagram_received5705299 +Ref: 230c5705299 +Ref: library/asyncio-protocol asyncio DatagramProtocol error_received5705564 +Ref: 230d5705564 +Node: Subprocess Protocols5706499 +Ref: library/asyncio-protocol asyncio-subprocess-protocols5706592 +Ref: 22535706592 +Ref: library/asyncio-protocol subprocess-protocols5706592 +Ref: 230e5706592 +Ref: library/asyncio-protocol asyncio SubprocessProtocol pipe_data_received5706828 +Ref: 230f5706828 +Ref: library/asyncio-protocol asyncio SubprocessProtocol pipe_connection_lost5707101 +Ref: 23105707101 +Ref: library/asyncio-protocol asyncio SubprocessProtocol process_exited5707311 +Ref: 23115707311 +Node: Examples<15>5707409 +Ref: library/asyncio-protocol examples5707500 +Ref: 22e25707500 +Node: TCP Echo Server5707755 +Ref: library/asyncio-protocol asyncio-example-tcp-echo-server-protocol5707843 +Ref: 22285707843 +Ref: library/asyncio-protocol tcp-echo-server5707843 +Ref: 23125707843 +Node: TCP Echo Client5709127 +Ref: library/asyncio-protocol asyncio-example-tcp-echo-client-protocol5709239 +Ref: 22255709239 +Ref: library/asyncio-protocol tcp-echo-client5709239 +Ref: 23135709239 +Node: UDP Echo Server5710797 +Ref: library/asyncio-protocol asyncio-udp-echo-server-protocol5710909 +Ref: 22a55710909 +Ref: library/asyncio-protocol udp-echo-server5710909 +Ref: 23145710909 +Node: UDP Echo Client5712055 +Ref: library/asyncio-protocol asyncio-udp-echo-client-protocol5712179 +Ref: 22a45712179 +Ref: library/asyncio-protocol udp-echo-client5712179 +Ref: 23155712179 +Node: Connecting Existing Sockets5713711 +Ref: library/asyncio-protocol asyncio-example-create-connection5713863 +Ref: 222c5713863 +Ref: library/asyncio-protocol connecting-existing-sockets5713863 +Ref: 23165713863 +Node: loop subprocess_exec and SubprocessProtocol5715775 +Ref: library/asyncio-protocol asyncio-example-subprocess-proto5715903 +Ref: 226a5715903 +Ref: library/asyncio-protocol loop-subprocess-exec-and-subprocessprotocol5715903 +Ref: 23175715903 +Node: Policies5717704 +Ref: library/asyncio-policy doc5717835 +Ref: 23185717835 +Ref: library/asyncio-policy asyncio-policies5717835 +Ref: 22895717835 +Ref: library/asyncio-policy policies5717835 +Ref: 23195717835 +Node: Getting and Setting the Policy5718621 +Ref: library/asyncio-policy getting-and-setting-the-policy5718719 +Ref: 231b5718719 +Ref: library/asyncio-policy asyncio get_event_loop_policy5718890 +Ref: 231c5718890 +Ref: library/asyncio-policy asyncio set_event_loop_policy5718984 +Ref: 231d5718984 +Node: Policy Objects5719165 +Ref: library/asyncio-policy policy-objects5719288 +Ref: 231e5719288 +Ref: library/asyncio-policy asyncio AbstractEventLoopPolicy5719405 +Ref: 231a5719405 +Ref: library/asyncio-policy asyncio AbstractEventLoopPolicy get_event_loop5719500 +Ref: 231f5719500 +Ref: library/asyncio-policy asyncio AbstractEventLoopPolicy set_event_loop5719789 +Ref: 23205719789 +Ref: library/asyncio-policy asyncio AbstractEventLoopPolicy new_event_loop5719894 +Ref: 23215719894 +Ref: library/asyncio-policy asyncio AbstractEventLoopPolicy get_child_watcher5720039 +Ref: 23225720039 +Ref: library/asyncio-policy asyncio AbstractEventLoopPolicy set_child_watcher5720275 +Ref: 23245720275 +Ref: library/asyncio-policy asyncio DefaultEventLoopPolicy5720480 +Ref: 23255720480 +Ref: library/asyncio-policy asyncio WindowsSelectorEventLoopPolicy5720886 +Ref: 23265720886 +Ref: library/asyncio-policy asyncio WindowsProactorEventLoopPolicy5721094 +Ref: 23275721094 +Node: Process Watchers5721301 +Ref: library/asyncio-policy asyncio-watchers5721409 +Ref: 22675721409 +Ref: library/asyncio-policy process-watchers5721409 +Ref: 23285721409 +Ref: library/asyncio-policy asyncio get_child_watcher5722280 +Ref: 232c5722280 +Ref: library/asyncio-policy asyncio set_child_watcher5722387 +Ref: 232d5722387 +Ref: library/asyncio-policy asyncio AbstractChildWatcher5722819 +Ref: 23235722819 +Ref: library/asyncio-policy asyncio AbstractChildWatcher add_child_handler5722860 +Ref: 232e5722860 +Ref: library/asyncio-policy asyncio AbstractChildWatcher remove_child_handler5723253 +Ref: 232f5723253 +Ref: library/asyncio-policy asyncio AbstractChildWatcher attach_loop5723504 +Ref: 23305723504 +Ref: library/asyncio-policy asyncio AbstractChildWatcher is_active5723768 +Ref: 23315723768 +Ref: library/asyncio-policy asyncio AbstractChildWatcher close5724004 +Ref: 23325724004 +Ref: library/asyncio-policy asyncio ThreadedChildWatcher5724163 +Ref: 22685724163 +Ref: library/asyncio-policy asyncio MultiLoopChildWatcher5724633 +Ref: 23295724633 +Ref: library/asyncio-policy asyncio SafeChildWatcher5725292 +Ref: 232a5725292 +Ref: library/asyncio-policy asyncio FastChildWatcher5725881 +Ref: 232b5725881 +Node: Custom Policies5726349 +Ref: library/asyncio-policy custom-policies5726434 +Ref: 23335726434 +Node: Platform Support5727041 +Ref: library/asyncio-platforms doc5727168 +Ref: 23345727168 +Ref: library/asyncio-platforms asyncio-platform-support5727168 +Ref: 22ae5727168 +Ref: library/asyncio-platforms platform-support5727168 +Ref: 23355727168 +Node: All Platforms5727456 +Ref: library/asyncio-platforms all-platforms5727538 +Ref: 23365727538 +Node: Windows5727702 +Ref: library/asyncio-platforms windows5727798 +Ref: 23375727798 +Ref: Windows-Footnote-15729628 +Ref: Windows-Footnote-25729711 +Ref: Windows-Footnote-35729793 +Ref: Windows-Footnote-45729874 +Node: Subprocess Support on Windows5729939 +Ref: library/asyncio-platforms asyncio-windows-subprocess5730012 +Ref: 224f5730012 +Ref: library/asyncio-platforms subprocess-support-on-windows5730012 +Ref: 23395730012 +Node: macOS5730395 +Ref: library/asyncio-platforms macos5730469 +Ref: 233a5730469 +Node: High-level API Index5731101 +Ref: library/asyncio-api-index doc5731239 +Ref: 233c5731239 +Ref: library/asyncio-api-index high-level-api-index5731239 +Ref: 233d5731239 +Node: Tasks5731517 +Ref: library/asyncio-api-index tasks5731597 +Ref: 233e5731597 +Node: Queues<2>5734370 +Ref: library/asyncio-api-index queues5734474 +Ref: 233f5734474 +Node: Subprocesses<2>5735293 +Ref: library/asyncio-api-index subprocesses5735402 +Ref: 23405735402 +Node: Streams<2>5735964 +Ref: library/asyncio-api-index streams5736079 +Ref: 23415736079 +Node: Synchronization5737514 +Ref: library/asyncio-api-index synchronization5737628 +Ref: 23425737628 +Node: Exceptions<10>5738765 +Ref: library/asyncio-api-index exceptions5738860 +Ref: 23435738860 +Node: Low-level API Index5739822 +Ref: library/asyncio-llapi-index doc5739967 +Ref: 23445739967 +Ref: library/asyncio-llapi-index low-level-api-index5739967 +Ref: 23455739967 +Node: Obtaining the Event Loop5740239 +Ref: library/asyncio-llapi-index obtaining-the-event-loop5740349 +Ref: 23465740349 +Node: Event Loop Methods<2>5741462 +Ref: library/asyncio-llapi-index event-loop-methods5741594 +Ref: 23475741594 +Node: Transports<2>5753125 +Ref: library/asyncio-llapi-index transports5753245 +Ref: 23485753245 +Node: Protocols<2>5759077 +Ref: library/asyncio-llapi-index protocols5759195 +Ref: 23495759195 +Node: Event Loop Policies5762718 +Ref: library/asyncio-llapi-index event-loop-policies5762814 +Ref: 234a5762814 +Node: Developing with asyncio5763643 +Ref: library/asyncio-dev doc5763759 +Ref: 234b5763759 +Ref: library/asyncio-dev asyncio-dev5763759 +Ref: 234c5763759 +Ref: library/asyncio-dev developing-with-asyncio5763759 +Ref: 234d5763759 +Node: Debug Mode5764170 +Ref: library/asyncio-dev asyncio-debug-mode5764279 +Ref: feb5764279 +Ref: library/asyncio-dev debug-mode5764279 +Ref: 234e5764279 +Node: Concurrency and Multithreading5765905 +Ref: library/asyncio-dev asyncio-multithreading5766044 +Ref: 21f65766044 +Ref: library/asyncio-dev concurrency-and-multithreading5766044 +Ref: 23515766044 +Node: Running Blocking Code5768356 +Ref: library/asyncio-dev asyncio-handle-blocking5768495 +Ref: 23525768495 +Ref: library/asyncio-dev running-blocking-code5768495 +Ref: 23535768495 +Node: Logging<3>5768990 +Ref: library/asyncio-dev asyncio-logger5769130 +Ref: 234f5769130 +Ref: library/asyncio-dev logging5769130 +Ref: 23545769130 +Node: Detect never-awaited coroutines5769411 +Ref: library/asyncio-dev asyncio-coroutine-not-scheduled5769563 +Ref: 23505769563 +Ref: library/asyncio-dev detect-never-awaited-coroutines5769563 +Ref: 23555769563 +Node: Detect never-retrieved exceptions5770572 +Ref: library/asyncio-dev detect-never-retrieved-exceptions5770705 +Ref: 23565770705 +Ref: Detect never-retrieved exceptions-Footnote-15772314 +Node: socket — Low-level networking interface5772378 +Ref: library/socket doc5772585 +Ref: 23575772585 +Ref: library/socket module-socket5772585 +Ref: ef5772585 +Ref: library/socket socket-low-level-networking-interface5772585 +Ref: 23585772585 +Ref: socket — Low-level networking interface-Footnote-15773900 +Node: Socket families5773965 +Ref: library/socket socket-families5774082 +Ref: 23595774082 +Ref: library/socket host-port5775278 +Ref: 235a5775278 +Ref: Socket families-Footnote-15783231 +Node: Module contents5783280 +Ref: library/socket module-contents5783420 +Ref: 23605783420 +Node: Exceptions<11>5783627 +Ref: library/socket exceptions5783714 +Ref: 23615783714 +Ref: library/socket socket error5783755 +Ref: 9955783755 +Ref: library/socket socket herror5783942 +Ref: 23625783942 +Ref: library/socket socket gaierror5784567 +Ref: 23655784567 +Ref: library/socket socket timeout5785172 +Ref: c0e5785172 +Ref: Exceptions<11>-Footnote-15785667 +Node: Constants<8>5785716 +Ref: library/socket constants5785824 +Ref: 23675785824 +Ref: library/socket socket AF_UNIX5786012 +Ref: 22a35786012 +Ref: library/socket socket AF_INET5786037 +Ref: 229d5786037 +Ref: library/socket socket AF_INET65786062 +Ref: 229e5786062 +Ref: library/socket socket SOCK_STREAM5786376 +Ref: c8a5786376 +Ref: library/socket socket SOCK_DGRAM5786405 +Ref: c895786405 +Ref: library/socket socket SOCK_RAW5786433 +Ref: 23685786433 +Ref: library/socket socket SOCK_RDM5786459 +Ref: 23695786459 +Ref: library/socket socket SOCK_SEQPACKET5786485 +Ref: 236a5786485 +Ref: library/socket socket SOCK_CLOEXEC5786784 +Ref: 4975786784 +Ref: library/socket socket SOCK_NONBLOCK5786814 +Ref: 4965786814 +Ref: library/socket socket SOMAXCONN5787243 +Ref: 74c5787243 +Ref: library/socket socket AF_CAN5788361 +Ref: 235b5788361 +Ref: library/socket socket PF_CAN5788385 +Ref: 236b5788385 +Ref: library/socket socket CAN_BCM5788643 +Ref: 8b95788643 +Ref: library/socket socket CAN_RAW_FD_FRAMES5789065 +Ref: 7c55789065 +Ref: library/socket socket CAN_ISOTP5789478 +Ref: 235c5789478 +Ref: library/socket socket AF_PACKET5789727 +Ref: 235d5789727 +Ref: library/socket socket PF_PACKET5789754 +Ref: 236c5789754 +Ref: library/socket socket AF_RDS5789968 +Ref: 236d5789968 +Ref: library/socket socket PF_RDS5789992 +Ref: 236e5789992 +Ref: library/socket socket SOL_RDS5790016 +Ref: 236f5790016 +Ref: library/socket socket SIO_RCVALL5790254 +Ref: 23705790254 +Ref: library/socket socket SIO_KEEPALIVE_VALS5790282 +Ref: 23715790282 +Ref: library/socket socket SIO_LOOPBACK_FAST_PATH5790318 +Ref: 5815790318 +Ref: library/socket socket AF_ALG5790737 +Ref: 5845790737 +Ref: library/socket socket SOL_ALG5790761 +Ref: 23725790761 +Ref: library/socket socket AF_VSOCK5790926 +Ref: 3d35790926 +Ref: library/socket socket IOCTL_VM_SOCKETS_GET_LOCAL_CID5790952 +Ref: 23735790952 +Ref: library/socket socket AF_LINK5791162 +Ref: 8ba5791162 +Ref: library/socket socket has_ipv65791256 +Ref: 23745791256 +Ref: library/socket socket BDADDR_ANY5791388 +Ref: 23755791388 +Ref: library/socket socket BDADDR_LOCAL5791416 +Ref: 23765791416 +Ref: library/socket socket HCI_FILTER5791685 +Ref: 23775791685 +Ref: library/socket socket HCI_TIME_STAMP5791713 +Ref: 23785791713 +Ref: library/socket socket HCI_DATA_DIR5791745 +Ref: 23795791745 +Ref: library/socket socket AF_QIPCRTR5792013 +Ref: 235e5792013 +Ref: Constants<8>-Footnote-15792244 +Node: Functions<5>5792295 +Ref: library/socket functions5792380 +Ref: 237a5792380 +Node: Creating sockets5792489 +Ref: library/socket creating-sockets5792581 +Ref: 237b5792581 +Ref: library/socket socket socket5792698 +Ref: 6745792698 +Ref: library/socket socket socketpair5795121 +Ref: 4815795121 +Ref: library/socket socket create_connection5795858 +Ref: b9e5795858 +Ref: library/socket socket create_server5797017 +Ref: 21e5797017 +Ref: library/socket socket has_dualstack_ipv65798793 +Ref: 21f5798793 +Ref: library/socket socket fromfd5798988 +Ref: 237d5798988 +Ref: library/socket socket fromshare5799870 +Ref: 4935799870 +Ref: library/socket socket SocketType5800123 +Ref: 237f5800123 +Node: Other functions<2>5800276 +Ref: library/socket other-functions5800368 +Ref: 23805800368 +Ref: library/socket socket close5800495 +Ref: 3d15800495 +Ref: library/socket socket getaddrinfo5800767 +Ref: 22b15800767 +Ref: library/socket socket getfqdn5803515 +Ref: 23815803515 +Ref: library/socket socket gethostbyname5804032 +Ref: 23825804032 +Ref: library/socket socket gethostbyname_ex5804632 +Ref: 23635804632 +Ref: library/socket socket gethostname5805406 +Ref: 1bd65805406 +Ref: library/socket socket gethostbyaddr5805792 +Ref: 23645805792 +Ref: library/socket socket getnameinfo5806488 +Ref: 22b25806488 +Ref: library/socket socket getprotobyname5807209 +Ref: 23835807209 +Ref: library/socket socket getservbyname5807663 +Ref: 23845807663 +Ref: library/socket socket getservbyport5808074 +Ref: 23855808074 +Ref: library/socket socket ntohl5808471 +Ref: 23865808471 +Ref: library/socket socket ntohs5808728 +Ref: 46d5808728 +Ref: library/socket socket htonl5809286 +Ref: 23875809286 +Ref: library/socket socket htons5809543 +Ref: 46c5809543 +Ref: library/socket socket inet_aton5810101 +Ref: 23885810101 +Ref: library/socket socket inet_ntoa5811038 +Ref: 23895811038 +Ref: library/socket socket inet_pton5811867 +Ref: 8bb5811867 +Ref: library/socket socket inet_ntop5812701 +Ref: 8bc5812701 +Ref: library/socket socket CMSG_LEN5813724 +Ref: 238a5813724 +Ref: library/socket socket CMSG_SPACE5814398 +Ref: 238b5814398 +Ref: library/socket socket getdefaulttimeout5815291 +Ref: 237e5815291 +Ref: library/socket socket setdefaulttimeout5815569 +Ref: 23665815569 +Ref: library/socket socket sethostname5815857 +Ref: a875815857 +Ref: library/socket socket if_nameindex5816187 +Ref: 2205816187 +Ref: library/socket socket if_nametoindex5816976 +Ref: 2215816976 +Ref: library/socket socket if_indextoname5817425 +Ref: 2225817425 +Ref: Other functions<2>-Footnote-15817912 +Node: Socket Objects5817961 +Ref: library/socket id15818110 +Ref: 238c5818110 +Ref: library/socket socket-objects5818110 +Ref: 237c5818110 +Ref: library/socket socket socket accept5818462 +Ref: 6755818462 +Ref: library/socket socket socket bind5819228 +Ref: 238d5819228 +Ref: library/socket socket socket close5819528 +Ref: 6005819528 +Ref: library/socket socket socket connect5820523 +Ref: 6765820523 +Ref: library/socket socket socket connect_ex5821595 +Ref: 238f5821595 +Ref: library/socket socket socket detach5822174 +Ref: b9d5822174 +Ref: library/socket socket socket dup5822415 +Ref: 23905822415 +Ref: library/socket socket socket fileno5822597 +Ref: 23915822597 +Ref: library/socket socket socket get_inheritable5822958 +Ref: 7ff5822958 +Ref: library/socket socket socket getpeername5823220 +Ref: 22e75823220 +Ref: library/socket socket socket getsockname5823559 +Ref: 22e85823559 +Ref: library/socket socket socket getsockopt5823808 +Ref: 5825823808 +Ref: library/socket socket socket getblocking5824504 +Ref: 3d05824504 +Ref: library/socket socket socket gettimeout5824726 +Ref: 23925824726 +Ref: library/socket socket socket ioctl5824973 +Ref: 5805824973 +Ref: library/socket socket socket listen5825610 +Ref: 74b5825610 +Ref: library/socket socket socket makefile5826036 +Ref: b195826036 +Ref: library/socket socket socket recv5827151 +Ref: 6775827151 +Ref: library/socket socket socket recvfrom5827947 +Ref: 6785827947 +Ref: library/socket socket socket recvmsg5828883 +Ref: 6795828883 +Ref: library/socket socket socket recvmsg_into5832814 +Ref: a865832814 +Ref: library/socket socket socket recvfrom_into5834456 +Ref: cb35834456 +Ref: library/socket socket socket recv_into5834990 +Ref: cb25834990 +Ref: library/socket socket socket send5835437 +Ref: 67a5835437 +Ref: library/socket socket socket sendall5836250 +Ref: 67b5836250 +Ref: library/socket socket socket sendto5837224 +Ref: 67d5837224 +Ref: library/socket socket socket sendmsg5838054 +Ref: 67c5838054 +Ref: library/socket socket socket sendmsg_afalg5840219 +Ref: 5855840219 +Ref: library/socket socket socket sendfile5840556 +Ref: 74a5840556 +Ref: library/socket socket socket set_inheritable5841460 +Ref: 8005841460 +Ref: library/socket socket socket setblocking5841640 +Ref: 1be55841640 +Ref: library/socket socket socket settimeout5842212 +Ref: 235f5842212 +Ref: library/socket socket socket setsockopt5842948 +Ref: 5835842948 +Ref: library/socket socket socket shutdown5844026 +Ref: 238e5844026 +Ref: library/socket socket socket share5844322 +Ref: 4945844322 +Ref: library/socket socket socket family5845187 +Ref: 23965845187 +Ref: library/socket socket socket type5845242 +Ref: 4985845242 +Ref: library/socket socket socket proto5845293 +Ref: 23975845293 +Ref: Socket Objects-Footnote-15845385 +Ref: Socket Objects-Footnote-25845434 +Ref: Socket Objects-Footnote-35845483 +Ref: Socket Objects-Footnote-45845557 +Ref: Socket Objects-Footnote-55845606 +Ref: Socket Objects-Footnote-65845655 +Ref: Socket Objects-Footnote-75845704 +Ref: Socket Objects-Footnote-85845753 +Ref: Socket Objects-Footnote-95845802 +Ref: Socket Objects-Footnote-105845851 +Node: Notes on socket timeouts5845901 +Ref: library/socket notes-on-socket-timeouts5846045 +Ref: 23985846045 +Ref: library/socket socket-timeouts5846045 +Ref: 23955846045 +Node: Timeouts and the connect method5847402 +Ref: library/socket timeouts-and-the-connect-method5847533 +Ref: 23995847533 +Node: Timeouts and the accept method5848008 +Ref: library/socket timeouts-and-the-accept-method5848139 +Ref: 239a5848139 +Node: Example<8>5848880 +Ref: library/socket example5849001 +Ref: 239b5849001 +Ref: library/socket socket-example5849001 +Ref: 239c5849001 +Ref: Example<8>-Footnote-15856742 +Node: ssl — TLS/SSL wrapper for socket objects5856791 +Ref: library/ssl doc5857007 +Ref: 239d5857007 +Ref: library/ssl module-ssl5857007 +Ref: f35857007 +Ref: library/ssl ssl-tls-ssl-wrapper-for-socket-objects5857007 +Ref: 239e5857007 +Ref: ssl — TLS/SSL wrapper for socket objects-Footnote-15859614 +Node: Functions Constants and Exceptions5859676 +Ref: library/ssl functions-constants-and-exceptions5859809 +Ref: 239f5859809 +Node: Socket creation5860057 +Ref: library/ssl socket-creation5860168 +Ref: 23a05860168 +Node: Context creation5861935 +Ref: library/ssl context-creation5862069 +Ref: 23a15862069 +Ref: library/ssl ssl create_default_context5862211 +Ref: 8c15862211 +Ref: Context creation-Footnote-15865069 +Node: Exceptions<12>5865114 +Ref: library/ssl exceptions5865250 +Ref: 23a75865250 +Ref: library/ssl ssl SSLError5865291 +Ref: c0f5865291 +Ref: library/ssl ssl SSLError library5865863 +Ref: a985865863 +Ref: library/ssl ssl SSLError reason5866141 +Ref: a995866141 +Ref: library/ssl ssl SSLZeroReturnError5866400 +Ref: 23a85866400 +Ref: library/ssl ssl SSLWantReadError5866694 +Ref: 7565866694 +Ref: library/ssl ssl SSLWantWriteError5867001 +Ref: 7575867001 +Ref: library/ssl ssl SSLSyscallError5867305 +Ref: 23aa5867305 +Ref: library/ssl ssl SSLEOFError5867594 +Ref: 23ab5867594 +Ref: library/ssl ssl SSLCertVerificationError5867862 +Ref: 3dd5867862 +Ref: library/ssl ssl SSLCertVerificationError verify_code5868026 +Ref: 23ac5868026 +Ref: library/ssl ssl SSLCertVerificationError verify_message5868130 +Ref: 23ad5868130 +Ref: library/ssl ssl CertificateError5868228 +Ref: 23ae5868228 +Node: Random generation5868427 +Ref: library/ssl random-generation5868567 +Ref: 23af5868567 +Ref: library/ssl ssl RAND_bytes5868622 +Ref: a8f5868622 +Ref: library/ssl ssl RAND_pseudo_bytes5869294 +Ref: a905869294 +Ref: library/ssl ssl RAND_status5870144 +Ref: 23b05870144 +Ref: library/ssl ssl RAND_egd5870471 +Ref: 23b25870471 +Ref: library/ssl ssl RAND_add5871127 +Ref: 23b15871127 +Ref: Random generation-Footnote-15871569 +Ref: Random generation-Footnote-25871663 +Node: Certificate handling5871712 +Ref: library/ssl certificate-handling5871850 +Ref: 23b35871850 +Ref: library/ssl ssl match_hostname5871911 +Ref: 3dc5871911 +Ref: library/ssl ssl cert_time_to_seconds5873852 +Ref: 7585873852 +Ref: library/ssl ssl get_server_certificate5874762 +Ref: a9a5874762 +Ref: library/ssl ssl DER_cert_to_PEM_cert5875773 +Ref: 23b55875773 +Ref: library/ssl ssl PEM_cert_to_DER_cert5875955 +Ref: 23b65875955 +Ref: library/ssl ssl get_default_verify_paths5876135 +Ref: 8c25876135 +Ref: library/ssl ssl enum_certificates5877140 +Ref: 8ce5877140 +Ref: library/ssl ssl enum_crls5878016 +Ref: 8cf5878016 +Ref: library/ssl ssl wrap_socket5878546 +Ref: 46f5878546 +Ref: Certificate handling-Footnote-15880039 +Ref: Certificate handling-Footnote-25880088 +Ref: Certificate handling-Footnote-35880137 +Ref: Certificate handling-Footnote-45880186 +Ref: Certificate handling-Footnote-55880235 +Node: Constants<9>5880284 +Ref: library/ssl constants5880396 +Ref: 23b95880396 +Ref: library/ssl ssl CERT_NONE5880561 +Ref: 23ba5880561 +Ref: library/ssl ssl CERT_OPTIONAL5881178 +Ref: 23bc5881178 +Ref: library/ssl ssl CERT_REQUIRED5882083 +Ref: 23a55882083 +Ref: library/ssl ssl VerifyMode5883208 +Ref: 23be5883208 +Ref: library/ssl ssl VERIFY_DEFAULT5883324 +Ref: 8c75883324 +Ref: library/ssl ssl VERIFY_CRL_CHECK_LEAF5883577 +Ref: 8c85883577 +Ref: library/ssl ssl VERIFY_CRL_CHECK_CHAIN5884022 +Ref: 8c95884022 +Ref: library/ssl ssl VERIFY_X509_STRICT5884228 +Ref: 8ca5884228 +Ref: library/ssl ssl VERIFY_X509_TRUSTED_FIRST5884408 +Ref: 23bf5884408 +Ref: library/ssl ssl VerifyFlags5884697 +Ref: 23c05884697 +Ref: library/ssl ssl PROTOCOL_TLS5884817 +Ref: 23a25884817 +Ref: library/ssl ssl PROTOCOL_TLS_CLIENT5885047 +Ref: 23bb5885047 +Ref: library/ssl ssl PROTOCOL_TLS_SERVER5885361 +Ref: 23c15885361 +Ref: library/ssl ssl PROTOCOL_SSLv235885577 +Ref: 23c25885577 +Ref: library/ssl ssl PROTOCOL_SSLv25885730 +Ref: 23c35885730 +Ref: library/ssl ssl PROTOCOL_SSLv35886095 +Ref: 23b45886095 +Ref: library/ssl ssl PROTOCOL_TLSv15886587 +Ref: 23c45886587 +Ref: library/ssl ssl PROTOCOL_TLSv1_15886886 +Ref: 8bf5886886 +Ref: library/ssl ssl PROTOCOL_TLSv1_25887262 +Ref: 8c05887262 +Ref: library/ssl ssl OP_ALL5887764 +Ref: 23c55887764 +Ref: library/ssl ssl OP_NO_SSLv25888021 +Ref: ba05888021 +Ref: library/ssl ssl OP_NO_SSLv35888322 +Ref: 23a35888322 +Ref: library/ssl ssl OP_NO_TLSv15888623 +Ref: 23c65888623 +Ref: library/ssl ssl OP_NO_TLSv1_15889062 +Ref: 23c75889062 +Ref: library/ssl ssl OP_NO_TLSv1_25889449 +Ref: 23c85889449 +Ref: library/ssl ssl OP_NO_TLSv1_35889836 +Ref: 23c95889836 +Ref: library/ssl ssl OP_NO_RENEGOTIATION5890427 +Ref: 23ca5890427 +Ref: library/ssl ssl OP_CIPHER_SERVER_PREFERENCE5890708 +Ref: a9b5890708 +Ref: library/ssl ssl OP_SINGLE_DH_USE5890939 +Ref: 23cb5890939 +Ref: library/ssl ssl OP_SINGLE_ECDH_USE5891193 +Ref: 23cc5891193 +Ref: library/ssl ssl OP_ENABLE_MIDDLEBOX_COMPAT5891451 +Ref: 23cd5891451 +Ref: library/ssl ssl OP_NO_COMPRESSION5891728 +Ref: a965891728 +Ref: library/ssl ssl Options5891985 +Ref: 23ce5891985 +Ref: library/ssl ssl OP_NO_TICKET5892071 +Ref: 23cf5892071 +Ref: library/ssl ssl HAS_ALPN5892185 +Ref: 7535892185 +Ref: library/ssl ssl HAS_NEVER_CHECK_COMMON_NAME5892392 +Ref: 23d05892392 +Ref: library/ssl ssl HAS_ECDH5892628 +Ref: 23d15892628 +Ref: library/ssl ssl HAS_SNI5892885 +Ref: 23d25892885 +Ref: library/ssl ssl HAS_NPN5893066 +Ref: 23d35893066 +Ref: library/ssl ssl HAS_SSLv25893417 +Ref: 23d45893417 +Ref: library/ssl ssl HAS_SSLv35893554 +Ref: 23d55893554 +Ref: library/ssl ssl HAS_TLSv15893691 +Ref: 23d65893691 +Ref: library/ssl ssl HAS_TLSv1_15893828 +Ref: 3e95893828 +Ref: library/ssl ssl HAS_TLSv1_25893967 +Ref: 23d75893967 +Ref: library/ssl ssl HAS_TLSv1_35894106 +Ref: 23d85894106 +Ref: library/ssl ssl CHANNEL_BINDING_TYPES5894245 +Ref: 23d95894245 +Ref: library/ssl ssl OPENSSL_VERSION5894463 +Ref: ba15894463 +Ref: library/ssl ssl OPENSSL_VERSION_INFO5894675 +Ref: ba25894675 +Ref: library/ssl ssl OPENSSL_VERSION_NUMBER5894901 +Ref: ba35894901 +Ref: library/ssl ssl ALERT_DESCRIPTION_HANDSHAKE_FAILURE5895170 +Ref: 23da5895170 +Ref: library/ssl ssl ALERT_DESCRIPTION_INTERNAL_ERROR5895220 +Ref: 23db5895220 +Ref: library/ssl ssl AlertDescription5895612 +Ref: 23dc5895612 +Ref: library/ssl ssl Purpose SERVER_AUTH5895752 +Ref: 8cc5895752 +Ref: library/ssl ssl Purpose CLIENT_AUTH5896065 +Ref: 8cd5896065 +Ref: library/ssl ssl SSLErrorNumber5896378 +Ref: 23dd5896378 +Ref: library/ssl ssl TLSVersion5896503 +Ref: 23de5896503 +Ref: library/ssl ssl TLSVersion MINIMUM_SUPPORTED5896718 +Ref: 23df5896718 +Ref: library/ssl ssl TLSVersion MAXIMUM_SUPPORTED5896763 +Ref: 23e05896763 +Ref: library/ssl ssl TLSVersion SSLv35896986 +Ref: 23e15896986 +Ref: library/ssl ssl TLSVersion TLSv15897019 +Ref: 23e25897019 +Ref: library/ssl ssl TLSVersion TLSv1_15897052 +Ref: 23e35897052 +Ref: library/ssl ssl TLSVersion TLSv1_25897087 +Ref: 23e45897087 +Ref: library/ssl ssl TLSVersion TLSv1_35897122 +Ref: 23e55897122 +Ref: Constants<9>-Footnote-15897219 +Ref: Constants<9>-Footnote-25897268 +Ref: Constants<9>-Footnote-35897317 +Ref: Constants<9>-Footnote-45897395 +Ref: Constants<9>-Footnote-55897444 +Node: SSL Sockets5897537 +Ref: library/ssl ssl-sockets5897691 +Ref: 23e65897691 +Ref: library/ssl ssl SSLSocket5897734 +Ref: 3e25897734 +Ref: library/ssl ssl SSLSocket read5899910 +Ref: 75b5899910 +Ref: library/ssl ssl SSLSocket write5900730 +Ref: 75c5900730 +Ref: library/ssl ssl SSLSocket do_handshake5901960 +Ref: 75a5901960 +Ref: library/ssl ssl SSLSocket getpeercert5902724 +Ref: 8d15902724 +Ref: library/ssl ssl SSLSocket cipher5906337 +Ref: 22e95906337 +Ref: library/ssl ssl SSLSocket shared_ciphers5906621 +Ref: 7595906621 +Ref: library/ssl ssl SSLSocket compression5907102 +Ref: a955907102 +Ref: library/ssl ssl SSLSocket get_channel_binding5907447 +Ref: a945907447 +Ref: library/ssl ssl SSLSocket selected_alpn_protocol5908080 +Ref: 7525908080 +Ref: library/ssl ssl SSLSocket selected_npn_protocol5908491 +Ref: 23e95908491 +Ref: library/ssl ssl SSLSocket unwrap5908845 +Ref: 23e75908845 +Ref: library/ssl ssl SSLSocket verify_client_post_handshake5909268 +Ref: 2255909268 +Ref: library/ssl ssl SSLSocket version5910146 +Ref: 7555910146 +Ref: library/ssl ssl SSLSocket pending5910557 +Ref: 23ea5910557 +Ref: library/ssl ssl SSLSocket context5910695 +Ref: 23e85910695 +Ref: library/ssl ssl SSLSocket server_side5911043 +Ref: 23eb5911043 +Ref: library/ssl ssl SSLSocket server_hostname5911213 +Ref: 3e05911213 +Ref: library/ssl ssl SSLSocket session5911716 +Ref: 23ec5911716 +Ref: library/ssl ssl SSLSocket session_reused5912072 +Ref: 23ed5912072 +Ref: SSL Sockets-Footnote-15912175 +Ref: SSL Sockets-Footnote-25912224 +Node: SSL Contexts5912273 +Ref: library/ssl ssl-contexts5912405 +Ref: 23ee5912405 +Ref: library/ssl ssl SSLContext5912749 +Ref: 3e45912749 +Ref: library/ssl ssl SSLContext cert_store_stats5916736 +Ref: 8c45916736 +Ref: library/ssl ssl SSLContext load_cert_chain5917140 +Ref: a915917140 +Ref: library/ssl ssl SSLContext load_default_certs5918766 +Ref: 8cb5918766 +Ref: library/ssl ssl SSLContext load_verify_locations5919590 +Ref: 7eb5919590 +Ref: library/ssl ssl SSLContext get_ca_certs5920920 +Ref: 8c55920920 +Ref: library/ssl ssl SSLContext get_ciphers5921555 +Ref: 58c5921555 +Ref: library/ssl ssl SSLContext set_default_verify_paths5923696 +Ref: 8c35923696 +Ref: library/ssl ssl SSLContext set_ciphers5924165 +Ref: 23b85924165 +Ref: library/ssl ssl SSLContext set_alpn_protocols5924813 +Ref: 7515924813 +Ref: library/ssl ssl SSLContext set_npn_protocols5925701 +Ref: a975925701 +Ref: library/ssl ssl SSLContext sni_callback5926370 +Ref: 23f05926370 +Ref: library/ssl ssl SSLContext set_servername_callback5928938 +Ref: 8d05928938 +Ref: library/ssl ssl SSLContext load_dh_params5929619 +Ref: a925929619 +Ref: library/ssl ssl SSLContext set_ecdh_curve5930162 +Ref: a935930162 +Ref: library/ssl ssl SSLContext wrap_socket5930889 +Ref: 3e55930889 +Ref: library/ssl ssl SSLContext sslsocket_class5933676 +Ref: 23f15933676 +Ref: library/ssl ssl SSLContext wrap_bio5933972 +Ref: 3e65933972 +Ref: library/ssl ssl SSLContext sslobject_class5934711 +Ref: 23f25934711 +Ref: library/ssl ssl SSLContext session_stats5935004 +Ref: 23f35935004 +Ref: library/ssl ssl SSLContext check_hostname5935469 +Ref: 23bd5935469 +Ref: library/ssl ssl SSLContext keylog_filename5937060 +Ref: 23a65937060 +Ref: library/ssl ssl SSLContext maximum_version5937568 +Ref: 3e85937568 +Ref: library/ssl ssl SSLContext minimum_version5938551 +Ref: 3e75938551 +Ref: library/ssl ssl SSLContext num_tickets5938885 +Ref: 23f45938885 +Ref: library/ssl ssl SSLContext options5939232 +Ref: 23b75939232 +Ref: library/ssl ssl SSLContext post_handshake_auth5939979 +Ref: 2245939979 +Ref: library/ssl ssl SSLContext protocol5940962 +Ref: 23f55940962 +Ref: library/ssl ssl SSLContext hostname_checks_common_name5941098 +Ref: 3de5941098 +Ref: library/ssl ssl SSLContext verify_flags5941419 +Ref: 8c65941419 +Ref: library/ssl ssl SSLContext verify_mode5942003 +Ref: 23a45942003 +Ref: SSL Contexts-Footnote-15942509 +Ref: SSL Contexts-Footnote-25942667 +Ref: SSL Contexts-Footnote-35942758 +Ref: SSL Contexts-Footnote-45942850 +Ref: SSL Contexts-Footnote-55943008 +Ref: SSL Contexts-Footnote-65943099 +Ref: SSL Contexts-Footnote-75943191 +Ref: SSL Contexts-Footnote-85943279 +Ref: SSL Contexts-Footnote-95943344 +Ref: SSL Contexts-Footnote-105943393 +Ref: SSL Contexts-Footnote-115943472 +Ref: SSL Contexts-Footnote-125943522 +Ref: SSL Contexts-Footnote-135943599 +Node: Certificates5943676 +Ref: library/ssl certificates5943809 +Ref: 23f65943809 +Ref: library/ssl ssl-certificates5943809 +Ref: 23ef5943809 +Ref: Certificates-Footnote-15946158 +Node: Certificate chains5946207 +Ref: library/ssl certificate-chains5946298 +Ref: 23f75946298 +Node: CA certificates5947569 +Ref: library/ssl ca-certificates5947697 +Ref: 23f85947697 +Node: Combined key and certificate5948287 +Ref: library/ssl combined-key-and-certificate5948421 +Ref: 23f95948421 +Node: Self-signed certificates5949067 +Ref: library/ssl self-signed-certificates5949177 +Ref: 23fa5949177 +Node: Examples<16>5950875 +Ref: library/ssl examples5951025 +Ref: 23fb5951025 +Node: Testing for SSL support5951154 +Ref: library/ssl testing-for-ssl-support5951256 +Ref: 23fc5951256 +Node: Client-side operation5951570 +Ref: library/ssl client-side-operation5951702 +Ref: 23fd5951702 +Node: Server-side operation5956444 +Ref: library/ssl server-side-operation5956544 +Ref: 23fe5956544 +Node: Notes on non-blocking sockets5958554 +Ref: library/ssl notes-on-non-blocking-sockets5958713 +Ref: 23ff5958713 +Ref: library/ssl ssl-nonblocking5958713 +Ref: 23a95958713 +Node: Memory BIO Support<2>5961601 +Ref: library/ssl memory-bio-support5961759 +Ref: 24015961759 +Ref: library/ssl ssl SSLObject5962875 +Ref: 3e35962875 +Ref: library/ssl ssl MemoryBIO5966638 +Ref: 74f5966638 +Ref: library/ssl ssl MemoryBIO pending5966766 +Ref: 24025966766 +Ref: library/ssl ssl MemoryBIO eof5966865 +Ref: 24035966865 +Ref: library/ssl ssl MemoryBIO read5966995 +Ref: 24045966995 +Ref: library/ssl ssl MemoryBIO write5967153 +Ref: 24055967153 +Ref: library/ssl ssl MemoryBIO write_eof5967435 +Ref: 24065967435 +Node: SSL session5967727 +Ref: library/ssl ssl-session5967879 +Ref: 24075967879 +Ref: library/ssl ssl SSLSession5967945 +Ref: 58b5967945 +Ref: library/ssl ssl SSLSession id5968022 +Ref: 24085968022 +Ref: library/ssl ssl SSLSession time5968046 +Ref: 24095968046 +Ref: library/ssl ssl SSLSession timeout5968072 +Ref: 240a5968072 +Ref: library/ssl ssl SSLSession ticket_lifetime_hint5968101 +Ref: 240b5968101 +Ref: library/ssl ssl SSLSession has_ticket5968143 +Ref: 240c5968143 +Node: Security considerations5968175 +Ref: library/ssl security-considerations5968313 +Ref: 240d5968313 +Ref: library/ssl ssl-security5968313 +Ref: 22a25968313 +Node: Best defaults5968451 +Ref: library/ssl best-defaults5968548 +Ref: 240e5968548 +Node: Manual settings5969709 +Ref: library/ssl manual-settings5969831 +Ref: 240f5969831 +Node: Verifying certificates5969964 +Ref: library/ssl verifying-certificates5970064 +Ref: 24105970064 +Node: Protocol versions5971327 +Ref: library/ssl protocol-versions5971452 +Ref: 24115971452 +Node: Cipher selection5972253 +Ref: library/ssl cipher-selection5972347 +Ref: 24125972347 +Ref: Cipher selection-Footnote-15973003 +Node: Multi-processing5973088 +Ref: library/ssl multi-processing5973188 +Ref: 24135973188 +Node: TLS 1 35973743 +Ref: library/ssl ssl-tlsv1-35973886 +Ref: 3e15973886 +Ref: library/ssl tls-1-35973886 +Ref: 24145973886 +Node: LibreSSL support5975057 +Ref: library/ssl libressl-support5975168 +Ref: 24155975168 +Ref: library/ssl ssl-libressl5975168 +Ref: 24165975168 +Ref: LibreSSL support-Footnote-15976792 +Ref: LibreSSL support-Footnote-25976858 +Ref: LibreSSL support-Footnote-35976907 +Ref: LibreSSL support-Footnote-45976956 +Ref: LibreSSL support-Footnote-55977005 +Ref: LibreSSL support-Footnote-65977054 +Ref: LibreSSL support-Footnote-75977103 +Ref: LibreSSL support-Footnote-85977179 +Ref: LibreSSL support-Footnote-95977228 +Node: select — Waiting for I/O completion5977286 +Ref: library/select doc5977502 +Ref: 24175977502 +Ref: library/select module-select5977502 +Ref: e55977502 +Ref: library/select select-waiting-for-i-o-completion5977502 +Ref: 24185977502 +Ref: library/select select error5978510 +Ref: 9965978510 +Ref: library/select select devpoll5978697 +Ref: 8ab5978697 +Ref: library/select select epoll5979417 +Ref: 8a95979417 +Ref: library/select select poll5980903 +Ref: 24005980903 +Ref: library/select select kqueue5981222 +Ref: 241c5981222 +Ref: library/select select kevent5981561 +Ref: 241e5981561 +Ref: library/select select select5981838 +Ref: 6735981838 +Ref: library/select select PIPE_BUF5984107 +Ref: b865984107 +Ref: select — Waiting for I/O completion-Footnote-15984802 +Ref: select — Waiting for I/O completion-Footnote-25984851 +Node: /dev/poll Polling Objects5984900 +Ref: library/select dev-poll-polling-objects5985052 +Ref: 24205985052 +Ref: library/select devpoll-objects5985052 +Ref: 24195985052 +Ref: library/select select devpoll close5985406 +Ref: 8ad5985406 +Ref: library/select select devpoll closed5985517 +Ref: 8ae5985517 +Ref: library/select select devpoll fileno5985624 +Ref: 8ac5985624 +Ref: library/select select devpoll register5985744 +Ref: 24215985744 +Ref: library/select select devpoll modify5986728 +Ref: 24225986728 +Ref: library/select select devpoll unregister5986936 +Ref: 24235986936 +Ref: library/select select devpoll poll5987293 +Ref: 66f5987293 +Ref: /dev/poll Polling Objects-Footnote-15988436 +Node: Edge and Level Trigger Polling epoll Objects5988485 +Ref: library/select edge-and-level-trigger-polling-epoll-objects5988661 +Ref: 24245988661 +Ref: library/select epoll-objects5988661 +Ref: 241a5988661 +Ref: library/select select epoll close5991529 +Ref: 8aa5991529 +Ref: library/select select epoll closed5991618 +Ref: 24255991618 +Ref: library/select select epoll fileno5991695 +Ref: 24265991695 +Ref: library/select select epoll fromfd5991783 +Ref: 24275991783 +Ref: library/select select epoll register5991873 +Ref: 24285991873 +Ref: library/select select epoll modify5991973 +Ref: 24295991973 +Ref: library/select select epoll unregister5992058 +Ref: 242a5992058 +Ref: library/select select epoll poll5992158 +Ref: 6705992158 +Ref: Edge and Level Trigger Polling epoll Objects-Footnote-15992560 +Node: Polling Objects5992609 +Ref: library/select poll-objects5992774 +Ref: 241b5992774 +Ref: library/select polling-objects5992774 +Ref: 242b5992774 +Ref: library/select select poll register5993341 +Ref: 242c5993341 +Ref: library/select select poll modify5995318 +Ref: 242d5995318 +Ref: library/select select poll unregister5995619 +Ref: 242e5995619 +Ref: library/select select poll poll5996008 +Ref: 6725996008 +Ref: Polling Objects-Footnote-15997154 +Node: Kqueue Objects5997203 +Ref: library/select id15997338 +Ref: 242f5997338 +Ref: library/select kqueue-objects5997338 +Ref: 241d5997338 +Ref: library/select select kqueue close5997387 +Ref: 24305997387 +Ref: library/select select kqueue closed5997478 +Ref: 24315997478 +Ref: library/select select kqueue fileno5997557 +Ref: 24325997557 +Ref: library/select select kqueue fromfd5997646 +Ref: 24335997646 +Ref: library/select select kqueue control5997737 +Ref: 6715997737 +Ref: Kqueue Objects-Footnote-15998388 +Node: Kevent Objects5998437 +Ref: library/select id25998548 +Ref: 24345998548 +Ref: library/select kevent-objects5998548 +Ref: 241f5998548 +Ref: library/select select kevent ident5998663 +Ref: 24355998663 +Ref: library/select select kevent filter5998965 +Ref: 24365998965 +Ref: library/select select kevent flags6000747 +Ref: 24376000747 +Ref: library/select select kevent fflags6002408 +Ref: 24386002408 +Ref: library/select select kevent data6006153 +Ref: 24396006153 +Ref: library/select select kevent udata6006209 +Ref: 243a6006209 +Node: selectors — High-level I/O multiplexing6006264 +Ref: library/selectors doc6006478 +Ref: 243b6006478 +Ref: library/selectors module-selectors6006478 +Ref: e66006478 +Ref: library/selectors selectors-high-level-i-o-multiplexing6006478 +Ref: 243c6006478 +Ref: selectors — High-level I/O multiplexing-Footnote-16006836 +Node: Introduction<9>6006904 +Ref: library/selectors introduction6007016 +Ref: 243d6007016 +Node: Classes<3>6008183 +Ref: library/selectors classes6008316 +Ref: 24406008316 +Ref: library/selectors selectors SelectorKey6009095 +Ref: 24416009095 +Ref: library/selectors selectors SelectorKey fileobj6009373 +Ref: 24426009373 +Ref: library/selectors selectors SelectorKey fd6009437 +Ref: 24436009437 +Ref: library/selectors selectors SelectorKey events6009500 +Ref: 24446009500 +Ref: library/selectors selectors SelectorKey data6009591 +Ref: 24456009591 +Ref: library/selectors selectors BaseSelector6009757 +Ref: 243e6009757 +Ref: library/selectors selectors BaseSelector register6010433 +Ref: 24466010433 +Ref: library/selectors selectors BaseSelector unregister6011087 +Ref: 24476011087 +Ref: library/selectors selectors BaseSelector modify6011707 +Ref: 44b6011707 +Ref: library/selectors selectors BaseSelector select6012307 +Ref: 24486012307 +Ref: library/selectors selectors BaseSelector close6013606 +Ref: 24496013606 +Ref: library/selectors selectors BaseSelector get_key6013824 +Ref: 244a6013824 +Ref: library/selectors selectors BaseSelector get_map6014111 +Ref: 244b6014111 +Ref: library/selectors selectors DefaultSelector6014380 +Ref: 243f6014380 +Ref: library/selectors selectors SelectSelector6014588 +Ref: 23386014588 +Ref: library/selectors selectors PollSelector6014675 +Ref: 44d6014675 +Ref: library/selectors selectors EpollSelector6014759 +Ref: 44c6014759 +Ref: library/selectors selectors EpollSelector fileno6014844 +Ref: 244c6014844 +Ref: library/selectors selectors DevpollSelector6014984 +Ref: 44e6014984 +Ref: library/selectors selectors DevpollSelector fileno6015073 +Ref: 244d6015073 +Ref: library/selectors selectors KqueueSelector6015241 +Ref: 233b6015241 +Ref: library/selectors selectors KqueueSelector fileno6015329 +Ref: 244e6015329 +Ref: Classes<3>-Footnote-16015507 +Node: Examples<17>6015556 +Ref: library/selectors examples6015665 +Ref: 244f6015665 +Node: asyncore — Asynchronous socket handler6016692 +Ref: library/asyncore doc6016926 +Ref: 24506016926 +Ref: library/asyncore asyncore-asynchronous-socket-handler6016926 +Ref: 24516016926 +Ref: library/asyncore module-asyncore6016926 +Ref: b6016926 +Ref: library/asyncore asyncore loop6019226 +Ref: 24526019226 +Ref: library/asyncore asyncore dispatcher6020256 +Ref: bc36020256 +Ref: library/asyncore asyncore dispatcher handle_read6022175 +Ref: 24556022175 +Ref: library/asyncore asyncore dispatcher handle_write6022336 +Ref: 24566022336 +Ref: library/asyncore asyncore dispatcher handle_expt6022712 +Ref: 24576022712 +Ref: library/asyncore asyncore dispatcher handle_connect6022924 +Ref: 24586022924 +Ref: library/asyncore asyncore dispatcher handle_close6023171 +Ref: 24596023171 +Ref: library/asyncore asyncore dispatcher handle_error6023250 +Ref: 245a6023250 +Ref: library/asyncore asyncore dispatcher handle_accept6023417 +Ref: bc56023417 +Ref: library/asyncore asyncore dispatcher handle_accepted6023796 +Ref: bc46023796 +Ref: library/asyncore asyncore dispatcher readable6024284 +Ref: 24536024284 +Ref: library/asyncore asyncore dispatcher writable6024636 +Ref: 24546024636 +Ref: library/asyncore asyncore dispatcher create_socket6025143 +Ref: 245c6025143 +Ref: library/asyncore asyncore dispatcher connect6025542 +Ref: 245b6025542 +Ref: library/asyncore asyncore dispatcher send6025743 +Ref: 245d6025743 +Ref: library/asyncore asyncore dispatcher recv6025835 +Ref: 245e6025835 +Ref: library/asyncore asyncore dispatcher listen6026282 +Ref: 245f6026282 +Ref: library/asyncore asyncore dispatcher bind6026553 +Ref: 24606026553 +Ref: library/asyncore asyncore dispatcher accept6026992 +Ref: bc66026992 +Ref: library/asyncore asyncore dispatcher close6027634 +Ref: 24616027634 +Ref: library/asyncore asyncore dispatcher_with_send6027922 +Ref: 24626027922 +Ref: library/asyncore asyncore file_dispatcher6028153 +Ref: 24636028153 +Ref: library/asyncore asyncore file_wrapper6028582 +Ref: 24646028582 +Ref: asyncore — Asynchronous socket handler-Footnote-16029098 +Node: asyncore Example basic HTTP client6029165 +Ref: library/asyncore asyncore-example-16029319 +Ref: 24656029319 +Ref: library/asyncore asyncore-example-basic-http-client6029319 +Ref: 24666029319 +Node: asyncore Example basic echo server6030329 +Ref: library/asyncore asyncore-example-26030483 +Ref: 24676030483 +Ref: library/asyncore asyncore-example-basic-echo-server6030483 +Ref: 24686030483 +Node: asynchat — Asynchronous socket command/response handler6031410 +Ref: library/asynchat doc6031650 +Ref: 24696031650 +Ref: library/asynchat asynchat-asynchronous-socket-command-response-handler6031650 +Ref: 246a6031650 +Ref: library/asynchat module-asynchat6031650 +Ref: 96031650 +Ref: library/asynchat asynchat async_chat6032861 +Ref: 8b56032861 +Ref: library/asynchat asynchat async_chat ac_in_buffer_size6033776 +Ref: 246d6033776 +Ref: library/asynchat asynchat async_chat ac_out_buffer_size6033878 +Ref: 246e6033878 +Ref: library/asynchat asynchat async_chat close_when_done6035073 +Ref: 24706035073 +Ref: library/asynchat asynchat async_chat collect_incoming_data6035253 +Ref: 246b6035253 +Ref: library/asynchat asynchat async_chat discard_buffers6035485 +Ref: 24716035485 +Ref: library/asynchat asynchat async_chat found_terminator6035652 +Ref: 246c6035652 +Ref: library/asynchat asynchat async_chat get_terminator6036001 +Ref: 24726036001 +Ref: library/asynchat asynchat async_chat push6036097 +Ref: 24736036097 +Ref: library/asynchat asynchat async_chat push_with_producer6036434 +Ref: 24746036434 +Ref: library/asynchat asynchat async_chat set_terminator6036783 +Ref: 246f6036783 +Ref: asynchat — Asynchronous socket command/response handler-Footnote-16037873 +Node: asynchat Example6037940 +Ref: library/asynchat asynchat-example6038050 +Ref: 24756038050 +Ref: library/asynchat id16038050 +Ref: 24766038050 +Node: signal — Set handlers for asynchronous events6040399 +Ref: library/signal doc6040634 +Ref: 24776040634 +Ref: library/signal module-signal6040634 +Ref: ea6040634 +Ref: library/signal signal-set-handlers-for-asynchronous-events6040634 +Ref: 24786040634 +Node: General rules6040996 +Ref: library/signal general-rules6041120 +Ref: 24796041120 +Node: Execution of Python signal handlers6041936 +Ref: library/signal execution-of-python-signal-handlers6042049 +Ref: 247c6042049 +Node: Signals and threads6043208 +Ref: library/signal id16043321 +Ref: 24806043321 +Ref: library/signal signals-and-threads6043321 +Ref: 24816043321 +Node: Module contents<2>6043747 +Ref: library/signal module-contents6043890 +Ref: 24826043890 +Ref: library/signal signal SIG_DFL6044403 +Ref: 21c46044403 +Ref: library/signal signal SIG_IGN6044735 +Ref: 21c56044735 +Ref: library/signal signal SIGABRT6044856 +Ref: 24876044856 +Ref: library/signal signal SIGALRM6044922 +Ref: 24886044922 +Ref: library/signal signal SIGBREAK6045026 +Ref: 24896045026 +Ref: library/signal signal SIGBUS6045140 +Ref: 248a6045140 +Ref: library/signal signal SIGCHLD6045240 +Ref: 247b6045240 +Ref: library/signal signal SIGCLD6045347 +Ref: 248b6045347 +Ref: library/signal signal SIGCONT6045408 +Ref: 1c9f6045408 +Ref: library/signal signal SIGFPE6045526 +Ref: 247e6045526 +Ref: library/signal signal SIGHUP6045763 +Ref: 248c6045763 +Ref: library/signal signal SIGILL6045910 +Ref: 248d6045910 +Ref: library/signal signal SIGINT6045962 +Ref: 21c36045962 +Ref: library/signal signal SIGKILL6046092 +Ref: 248e6046092 +Ref: library/signal signal SIGPIPE6046223 +Ref: 247a6046223 +Ref: library/signal signal SIGSEGV6046383 +Ref: 247f6046383 +Ref: library/signal signal SIGTERM6046461 +Ref: 22626046461 +Ref: library/signal signal SIGUSR16046513 +Ref: 248f6046513 +Ref: library/signal signal SIGUSR26046606 +Ref: 24906046606 +Ref: library/signal signal SIGWINCH6046699 +Ref: 24916046699 +Ref: library/signal signal CTRL_C_EVENT6047355 +Ref: 1c796047355 +Ref: library/signal signal CTRL_BREAK_EVENT6047583 +Ref: 1c7a6047583 +Ref: library/signal signal NSIG6047819 +Ref: 24926047819 +Ref: library/signal signal ITIMER_REAL6047903 +Ref: 24936047903 +Ref: library/signal signal ITIMER_VIRTUAL6048035 +Ref: 24946048035 +Ref: library/signal signal ITIMER_PROF6048181 +Ref: 24956048181 +Ref: library/signal signal SIG_BLOCK6048520 +Ref: 24836048520 +Ref: library/signal signal SIG_UNBLOCK6048702 +Ref: 24846048702 +Ref: library/signal signal SIG_SETMASK6048893 +Ref: 24856048893 +Ref: library/signal signal ItimerError6049143 +Ref: 24966049143 +Ref: library/signal signal alarm6049666 +Ref: 24996049666 +Ref: library/signal signal getsignal6050309 +Ref: 24866050309 +Ref: library/signal signal strsignal6050881 +Ref: 249a6050881 +Ref: library/signal signal valid_signals6051139 +Ref: 249b6051139 +Ref: library/signal signal pause6051382 +Ref: 249c6051382 +Ref: library/signal signal raise_signal6051768 +Ref: 249d6051768 +Ref: library/signal signal pthread_kill6051901 +Ref: a7a6051901 +Ref: library/signal signal pthread_sigmask6053100 +Ref: a796053100 +Ref: library/signal signal setitimer6054519 +Ref: 24976054519 +Ref: library/signal signal getitimer6055537 +Ref: 24986055537 +Ref: library/signal signal set_wakeup_fd6055696 +Ref: 3ce6055696 +Ref: library/signal signal siginterrupt6057853 +Ref: a7e6057853 +Ref: library/signal signal signal6058456 +Ref: a7d6058456 +Ref: library/signal signal sigpending6059918 +Ref: a7b6059918 +Ref: library/signal signal sigwait6060376 +Ref: a7c6060376 +Ref: library/signal signal sigwaitinfo6060953 +Ref: 67f6060953 +Ref: library/signal signal sigtimedwait6062216 +Ref: 67e6062216 +Ref: Module contents<2>-Footnote-16062984 +Ref: Module contents<2>-Footnote-26063033 +Node: Example<9>6063082 +Ref: library/signal example6063227 +Ref: 249e6063227 +Ref: library/signal signal-example6063227 +Ref: 249f6063227 +Node: Note on SIGPIPE6064126 +Ref: library/signal note-on-sigpipe6064244 +Ref: 24a06064244 +Node: mmap — Memory-mapped file support6065658 +Ref: library/mmap doc6065827 +Ref: 24a16065827 +Ref: library/mmap mmap-memory-mapped-file-support6065827 +Ref: 24a26065827 +Ref: library/mmap module-mmap6065827 +Ref: b36065827 +Ref: library/mmap mmap mmap6068263 +Ref: 1ec6068263 +Ref: library/mmap mmap mmap close6073278 +Ref: 24a36073278 +Ref: library/mmap mmap mmap closed6073489 +Ref: 24a46073489 +Ref: library/mmap mmap mmap find6073593 +Ref: 24a56073593 +Ref: library/mmap mmap mmap flush6074023 +Ref: 2c36074023 +Ref: library/mmap mmap mmap madvise6074911 +Ref: 1ed6074911 +Ref: library/mmap mmap mmap move6075477 +Ref: 24a76075477 +Ref: library/mmap mmap mmap read6075755 +Ref: a226075755 +Ref: library/mmap mmap mmap read_byte6076207 +Ref: 24a86076207 +Ref: library/mmap mmap mmap readline6076355 +Ref: 24a96076355 +Ref: library/mmap mmap mmap resize6076495 +Ref: 24aa6076495 +Ref: library/mmap mmap mmap rfind6076743 +Ref: 24ab6076743 +Ref: library/mmap mmap mmap seek6077175 +Ref: 24ac6077175 +Ref: library/mmap mmap mmap size6077561 +Ref: 24ad6077561 +Ref: library/mmap mmap mmap tell6077700 +Ref: 24ae6077700 +Ref: library/mmap mmap mmap write6077787 +Ref: 24af6077787 +Ref: library/mmap mmap mmap write_byte6078494 +Ref: 24b06078494 +Node: MADV_* Constants6078840 +Ref: library/mmap madv-constants6078928 +Ref: 24b16078928 +Ref: library/mmap madvise-constants6078928 +Ref: 24a66078928 +Ref: library/mmap mmap MADV_NORMAL6078981 +Ref: 24b26078981 +Ref: library/mmap mmap MADV_RANDOM6079008 +Ref: 24b36079008 +Ref: library/mmap mmap MADV_SEQUENTIAL6079035 +Ref: 24b46079035 +Ref: library/mmap mmap MADV_WILLNEED6079066 +Ref: 24b56079066 +Ref: library/mmap mmap MADV_DONTNEED6079095 +Ref: 24b66079095 +Ref: library/mmap mmap MADV_REMOVE6079124 +Ref: 24b76079124 +Ref: library/mmap mmap MADV_DONTFORK6079151 +Ref: 24b86079151 +Ref: library/mmap mmap MADV_DOFORK6079180 +Ref: 24b96079180 +Ref: library/mmap mmap MADV_HWPOISON6079207 +Ref: 24ba6079207 +Ref: library/mmap mmap MADV_MERGEABLE6079236 +Ref: 24bb6079236 +Ref: library/mmap mmap MADV_UNMERGEABLE6079266 +Ref: 24bc6079266 +Ref: library/mmap mmap MADV_SOFT_OFFLINE6079298 +Ref: 24bd6079298 +Ref: library/mmap mmap MADV_HUGEPAGE6079331 +Ref: 24be6079331 +Ref: library/mmap mmap MADV_NOHUGEPAGE6079360 +Ref: 24bf6079360 +Ref: library/mmap mmap MADV_DONTDUMP6079391 +Ref: 24c06079391 +Ref: library/mmap mmap MADV_DODUMP6079420 +Ref: 24c16079420 +Ref: library/mmap mmap MADV_FREE6079447 +Ref: 24c26079447 +Ref: library/mmap mmap MADV_NOSYNC6079472 +Ref: 24c36079472 +Ref: library/mmap mmap MADV_AUTOSYNC6079499 +Ref: 24c46079499 +Ref: library/mmap mmap MADV_NOCORE6079528 +Ref: 24c56079528 +Ref: library/mmap mmap MADV_CORE6079555 +Ref: 24c66079555 +Ref: library/mmap mmap MADV_PROTECT6079580 +Ref: 24c76079580 +Node: Internet Data Handling6079815 +Ref: library/netdata doc6079994 +Ref: 24c86079994 +Ref: library/netdata internet-data-handling6079994 +Ref: 24c96079994 +Ref: library/netdata netdata6079994 +Ref: 24ca6079994 +Node: email — An email and MIME handling package6080703 +Ref: library/email doc6080848 +Ref: 24cb6080848 +Ref: library/email email-an-email-and-mime-handling-package6080848 +Ref: 24cc6080848 +Ref: library/email module-email6080848 +Ref: 696080848 +Ref: email — An email and MIME handling package-Footnote-16087608 +Ref: email — An email and MIME handling package-Footnote-26087681 +Ref: email — An email and MIME handling package-Footnote-36087730 +Ref: email — An email and MIME handling package-Footnote-46087779 +Ref: email — An email and MIME handling package-Footnote-56087828 +Ref: email — An email and MIME handling package-Footnote-66087877 +Ref: email — An email and MIME handling package-Footnote-76087926 +Ref: email — An email and MIME handling package-Footnote-86087975 +Ref: email — An email and MIME handling package-Footnote-96088024 +Node: email message Representing an email message6088073 +Ref: library/email message doc6088241 +Ref: 24cd6088241 +Ref: library/email message email-message-representing-an-email-message6088241 +Ref: 24ce6088241 +Ref: library/email message module-email message6088241 +Ref: 726088241 +Ref: library/email message email message EmailMessage6090729 +Ref: 5426090729 +Ref: library/email message email message EmailMessage as_string6091210 +Ref: 24d16091210 +Ref: library/email message email message EmailMessage __str__6093054 +Ref: 24d26093054 +Ref: library/email message email message EmailMessage as_bytes6093523 +Ref: 24d36093523 +Ref: library/email message email message EmailMessage __bytes__6094685 +Ref: 24d46094685 +Ref: library/email message email message EmailMessage is_multipart6094861 +Ref: 24d56094861 +Ref: library/email message email message EmailMessage set_unixfrom6095546 +Ref: 24d66095546 +Ref: library/email message email message EmailMessage get_unixfrom6095768 +Ref: 24d86095768 +Ref: library/email message email message EmailMessage __len__6096900 +Ref: 24da6096900 +Ref: library/email message email message EmailMessage __contains__6096998 +Ref: 24db6096998 +Ref: library/email message email message EmailMessage __getitem__6097388 +Ref: 24dc6097388 +Ref: library/email message email message EmailMessage __setitem__6098105 +Ref: 24df6098105 +Ref: library/email message email message EmailMessage __delitem__6099138 +Ref: 24e06099138 +Ref: library/email message email message EmailMessage keys6099364 +Ref: 24d96099364 +Ref: library/email message email message EmailMessage values6099458 +Ref: 24e16099458 +Ref: library/email message email message EmailMessage items6099548 +Ref: 24e26099548 +Ref: library/email message email message EmailMessage get6099679 +Ref: 24e36099679 +Ref: library/email message email message EmailMessage get_all6100028 +Ref: 24dd6100028 +Ref: library/email message email message EmailMessage add_header6100265 +Ref: 24e46100265 +Ref: library/email message email message EmailMessage replace_header6102229 +Ref: 24e56102229 +Ref: library/email message email message EmailMessage get_content_type6102535 +Ref: 24e66102535 +Ref: library/email message email message EmailMessage get_content_maintype6103419 +Ref: 24e86103419 +Ref: library/email message email message EmailMessage get_content_subtype6103625 +Ref: 24e96103625 +Ref: library/email message email message EmailMessage get_default_type6103818 +Ref: 24e76103818 +Ref: library/email message email message EmailMessage set_default_type6104138 +Ref: 24ea6104138 +Ref: library/email message email message EmailMessage set_param6104580 +Ref: 24eb6104580 +Ref: library/email message email message EmailMessage del_param6106194 +Ref: 24ec6106194 +Ref: library/email message email message EmailMessage get_filename6106630 +Ref: 24ed6106630 +Ref: library/email message email message EmailMessage get_boundary6107169 +Ref: 24ef6107169 +Ref: library/email message email message EmailMessage set_boundary6107533 +Ref: 24f06107533 +Ref: library/email message email message EmailMessage get_content_charset6108201 +Ref: 24f26108201 +Ref: library/email message email message EmailMessage get_charsets6108503 +Ref: 24f36108503 +Ref: library/email message email message EmailMessage is_attachment6109196 +Ref: 24f46109196 +Ref: library/email message email message EmailMessage get_content_disposition6109563 +Ref: 24f66109563 +Ref: library/email message email message EmailMessage walk6110038 +Ref: 24f76110038 +Ref: library/email message email message EmailMessage get_body6112051 +Ref: 24f86112051 +Ref: library/email message email message EmailMessage iter_attachments6114170 +Ref: 24f96114170 +Ref: library/email message email message EmailMessage iter_parts6115144 +Ref: 24fa6115144 +Ref: library/email message email message EmailMessage get_content6115357 +Ref: 24fb6115357 +Ref: library/email message email message EmailMessage set_content6115767 +Ref: 24fd6115767 +Ref: library/email message email message EmailMessage make_related6116177 +Ref: 24ff6116177 +Ref: library/email message email message EmailMessage make_alternative6116665 +Ref: 25006116665 +Ref: library/email message email message EmailMessage make_mixed6117184 +Ref: 25016117184 +Ref: library/email message email message EmailMessage add_related6117723 +Ref: 25026117723 +Ref: library/email message email message EmailMessage add_alternative6118480 +Ref: 25046118480 +Ref: library/email message email message EmailMessage add_attachment6119170 +Ref: 25056119170 +Ref: library/email message email message EmailMessage clear6120155 +Ref: 25066120155 +Ref: library/email message email message EmailMessage clear_content6120236 +Ref: 25076120236 +Ref: library/email message email message EmailMessage preamble6120496 +Ref: 25086120496 +Ref: library/email message email message EmailMessage epilogue6121758 +Ref: 25096121758 +Ref: library/email message email message EmailMessage defects6122104 +Ref: 250a6122104 +Ref: library/email message email message MIMEPart6122346 +Ref: 8436122346 +Ref: email message Representing an email message-Footnote-16122712 +Ref: email message Representing an email message-Footnote-26122784 +Ref: email message Representing an email message-Footnote-36122984 +Ref: email message Representing an email message-Footnote-46123033 +Ref: email message Representing an email message-Footnote-56123082 +Ref: email message Representing an email message-Footnote-66123131 +Ref: email message Representing an email message-Footnote-76123180 +Ref: email message Representing an email message-Footnote-86123229 +Ref: email message Representing an email message-Footnote-96123278 +Ref: email message Representing an email message-Footnote-106123327 +Ref: email message Representing an email message-Footnote-116123377 +Ref: email message Representing an email message-Footnote-126123427 +Ref: email message Representing an email message-Footnote-136123477 +Node: email parser Parsing email messages6123527 +Ref: library/email parser doc6123745 +Ref: 250b6123745 +Ref: library/email parser email-parser-parsing-email-messages6123745 +Ref: 250c6123745 +Ref: library/email parser module-email parser6123745 +Ref: 746123745 +Ref: email parser Parsing email messages-Footnote-16126130 +Node: FeedParser API6126201 +Ref: library/email parser feedparser-api6126306 +Ref: 250e6126306 +Ref: library/email parser email parser BytesFeedParser6127599 +Ref: b496127599 +Ref: library/email parser email parser BytesFeedParser feed6128798 +Ref: 250f6128798 +Ref: library/email parser email parser BytesFeedParser close6129233 +Ref: 25106129233 +Ref: library/email parser email parser FeedParser6129473 +Ref: 250d6129473 +Node: Parser API6129931 +Ref: library/email parser parser-api6130061 +Ref: 25116130061 +Ref: library/email parser email parser BytesParser6130789 +Ref: b4a6130789 +Ref: library/email parser email parser BytesParser parse6131456 +Ref: 25126131456 +Ref: library/email parser email parser BytesParser parsebytes6132496 +Ref: 25136132496 +Ref: library/email parser email parser BytesHeaderParser6133002 +Ref: 9fc6133002 +Ref: library/email parser email parser Parser6133220 +Ref: b186133220 +Ref: library/email parser email parser Parser parse6133565 +Ref: 25146133565 +Ref: library/email parser email parser Parser parsestr6134003 +Ref: 25156134003 +Ref: library/email parser email parser HeaderParser6134432 +Ref: 9fd6134432 +Ref: library/email parser email message_from_bytes6134829 +Ref: b476134829 +Ref: library/email parser email message_from_binary_file6135296 +Ref: b486135296 +Ref: library/email parser email message_from_string6135769 +Ref: 25166135769 +Ref: library/email parser email message_from_file6136164 +Ref: 25176136164 +Ref: Parser API-Footnote-16136894 +Ref: Parser API-Footnote-26136943 +Node: Additional notes6136992 +Ref: library/email parser additional-notes6137099 +Ref: 25186137099 +Node: email generator Generating MIME documents6138641 +Ref: library/email generator doc6138843 +Ref: 25196138843 +Ref: library/email generator email-generator-generating-mime-documents6138843 +Ref: 251a6138843 +Ref: library/email generator module-email generator6138843 +Ref: 6e6138843 +Ref: library/email generator email generator BytesGenerator6140966 +Ref: b4d6140966 +Ref: library/email generator email generator BytesGenerator flatten6142872 +Ref: 251b6142872 +Ref: library/email generator email generator BytesGenerator clone6144694 +Ref: 251e6144694 +Ref: library/email generator email generator BytesGenerator write6144892 +Ref: 251c6144892 +Ref: library/email generator email generator Generator6145943 +Ref: b4c6145943 +Ref: library/email generator email generator Generator flatten6147789 +Ref: 251f6147789 +Ref: library/email generator email generator Generator clone6149549 +Ref: 25216149549 +Ref: library/email generator email generator Generator write6149734 +Ref: 25206149734 +Ref: library/email generator email generator DecodedGenerator6150643 +Ref: 53e6150643 +Ref: email generator Generating MIME documents-Footnote-16152060 +Ref: email generator Generating MIME documents-Footnote-26152134 +Ref: email generator Generating MIME documents-Footnote-36152627 +Ref: email generator Generating MIME documents-Footnote-46152679 +Ref: email generator Generating MIME documents-Footnote-56152728 +Ref: email generator Generating MIME documents-Footnote-66152780 +Node: email policy Policy Objects6152829 +Ref: library/email policy doc6153037 +Ref: 25226153037 +Ref: library/email policy email-policy-policy-objects6153037 +Ref: 25236153037 +Ref: library/email policy module-email policy6153037 +Ref: 756153037 +Ref: library/email policy email policy Policy6158992 +Ref: 9f46158992 +Ref: library/email policy email policy Policy max_line_length6159812 +Ref: 25246159812 +Ref: library/email policy email policy Policy linesep6160118 +Ref: 25256160118 +Ref: library/email policy email policy Policy cte_type6160387 +Ref: 251d6160387 +Ref: library/email policy email policy Policy raise_on_defect6161498 +Ref: 25276161498 +Ref: library/email policy email policy Policy mangle_from_6161742 +Ref: 6d06161742 +Ref: library/email policy email policy Policy message_factory6162099 +Ref: 53f6162099 +Ref: library/email policy email policy Policy clone6162536 +Ref: 9f66162536 +Ref: library/email policy email policy Policy handle_defect6163010 +Ref: 25296163010 +Ref: library/email policy email policy Policy register_defect6163489 +Ref: 25286163489 +Ref: library/email policy email policy Policy header_max_count6164182 +Ref: 252a6164182 +Ref: library/email policy email policy Policy header_source_parse6165244 +Ref: 252b6165244 +Ref: library/email policy email policy Policy header_store_parse6166257 +Ref: 252c6166257 +Ref: library/email policy email policy Policy header_fetch_parse6167018 +Ref: 252d6167018 +Ref: library/email policy email policy Policy fold6167841 +Ref: 252e6167841 +Ref: library/email policy email policy Policy fold_binary6168557 +Ref: 25266168557 +Ref: library/email policy email policy EmailPolicy6168891 +Ref: 9f86168891 +Ref: library/email policy email policy EmailPolicy utf86169721 +Ref: 6d26169721 +Ref: library/email policy email policy EmailPolicy refold_source6170117 +Ref: 252f6170117 +Ref: library/email policy email policy EmailPolicy header_factory6170912 +Ref: 25306170912 +Ref: library/email policy email policy EmailPolicy content_manager6171523 +Ref: 8446171523 +Ref: library/email policy email policy EmailPolicy header_max_count6172232 +Ref: 25326172232 +Ref: library/email policy email policy EmailPolicy header_source_parse6172438 +Ref: 25346172438 +Ref: library/email policy email policy EmailPolicy header_store_parse6172823 +Ref: 25356172823 +Ref: library/email policy email policy EmailPolicy header_fetch_parse6173304 +Ref: 25366173304 +Ref: library/email policy email policy EmailPolicy fold6173730 +Ref: 25376173730 +Ref: library/email policy email policy EmailPolicy fold_binary6174893 +Ref: 25386174893 +Ref: library/email policy email policy default6175712 +Ref: 24d06175712 +Ref: library/email policy email policy SMTP6175922 +Ref: 25396175922 +Ref: library/email policy email policy SMTPUTF86176124 +Ref: 253a6176124 +Ref: library/email policy email policy HTTP6176538 +Ref: 253b6176538 +Ref: library/email policy email policy strict6176730 +Ref: 253c6176730 +Ref: library/email policy email policy Compat326178157 +Ref: 9f56178157 +Ref: library/email policy email policy Compat32 mangle_from_6178688 +Ref: 253d6178688 +Ref: library/email policy email policy Compat32 header_source_parse6178876 +Ref: 253e6178876 +Ref: library/email policy email policy Compat32 header_store_parse6179261 +Ref: 253f6179261 +Ref: library/email policy email policy Compat32 header_fetch_parse6179367 +Ref: 25406179367 +Ref: library/email policy email policy Compat32 fold6179609 +Ref: 25426179609 +Ref: library/email policy email policy Compat32 fold_binary6179952 +Ref: 25436179952 +Ref: library/email policy email policy compat326180482 +Ref: 5406180482 +Ref: email policy Policy Objects-Footnote-16180685 +Ref: email policy Policy Objects-Footnote-26180756 +Ref: email policy Policy Objects-Footnote-36180805 +Ref: email policy Policy Objects-Footnote-46180854 +Ref: email policy Policy Objects-Footnote-56180903 +Ref: email policy Policy Objects-Footnote-66180952 +Ref: email policy Policy Objects-Footnote-76181001 +Ref: email policy Policy Objects-Footnote-86181050 +Ref: email policy Policy Objects-Footnote-96181123 +Ref: email policy Policy Objects-Footnote-106181172 +Ref: email policy Policy Objects-Footnote-116181222 +Ref: email policy Policy Objects-Footnote-126181272 +Node: email errors Exception and Defect classes6181322 +Ref: library/email errors doc6181531 +Ref: 25446181531 +Ref: library/email errors email-errors-exception-and-defect-classes6181531 +Ref: 25456181531 +Ref: library/email errors module-email errors6181531 +Ref: 6d6181531 +Ref: library/email errors email errors MessageError6181835 +Ref: 25466181835 +Ref: library/email errors email errors MessageParseError6182076 +Ref: 25476182076 +Ref: library/email errors email errors HeaderParseError6182352 +Ref: 24f16182352 +Ref: library/email errors email errors BoundaryError6182995 +Ref: 25486182995 +Ref: library/email errors email errors MultipartConversionError6183075 +Ref: 25496183075 +Ref: email errors Exception and Defect classes-Footnote-16186402 +Ref: email errors Exception and Defect classes-Footnote-26186473 +Node: email headerregistry Custom Header Objects6186522 +Ref: library/email headerregistry doc6186746 +Ref: 254c6186746 +Ref: library/email headerregistry email-headerregistry-custom-header-objects6186746 +Ref: 254d6186746 +Ref: library/email headerregistry module-email headerregistry6186746 +Ref: 706186746 +Ref: library/email headerregistry email headerregistry BaseHeader6188512 +Ref: 24de6188512 +Ref: library/email headerregistry email headerregistry BaseHeader name6188825 +Ref: 25506188825 +Ref: library/email headerregistry email headerregistry BaseHeader defects6189076 +Ref: 25516189076 +Ref: library/email headerregistry email headerregistry BaseHeader max_count6189428 +Ref: 25336189428 +Ref: library/email headerregistry email headerregistry BaseHeader fold6189945 +Ref: 25526189945 +Ref: library/email headerregistry email headerregistry UnstructuredHeader6192136 +Ref: 254f6192136 +Ref: library/email headerregistry email headerregistry DateHeader6193215 +Ref: 25536193215 +Ref: library/email headerregistry email headerregistry DateHeader datetime6193578 +Ref: 25546193578 +Ref: library/email headerregistry email headerregistry AddressHeader6195146 +Ref: 25556195146 +Ref: library/email headerregistry email headerregistry AddressHeader groups6195430 +Ref: 25566195430 +Ref: library/email headerregistry email headerregistry AddressHeader addresses6195745 +Ref: 25596195745 +Ref: library/email headerregistry email headerregistry SingleAddressHeader6196907 +Ref: 255b6196907 +Ref: library/email headerregistry email headerregistry SingleAddressHeader address6197048 +Ref: 255c6197048 +Ref: library/email headerregistry email headerregistry MIMEVersionHeader6197580 +Ref: 255d6197580 +Ref: library/email headerregistry email headerregistry MIMEVersionHeader version6197972 +Ref: 255e6197972 +Ref: library/email headerregistry email headerregistry MIMEVersionHeader major6198099 +Ref: 255f6198099 +Ref: library/email headerregistry email headerregistry MIMEVersionHeader minor6198176 +Ref: 25606198176 +Ref: library/email headerregistry email headerregistry ParameterizedMIMEHeader6198253 +Ref: 25616198253 +Ref: library/email headerregistry email headerregistry ParameterizedMIMEHeader params6198636 +Ref: 25626198636 +Ref: library/email headerregistry email headerregistry ContentTypeHeader6198733 +Ref: 25636198733 +Ref: library/email headerregistry email headerregistry ContentTypeHeader content_type6198884 +Ref: 25646198884 +Ref: library/email headerregistry email headerregistry ContentTypeHeader maintype6198990 +Ref: 25656198990 +Ref: library/email headerregistry email headerregistry ContentTypeHeader subtype6199020 +Ref: 25666199020 +Ref: library/email headerregistry email headerregistry ContentDispositionHeader6199049 +Ref: 25676199049 +Ref: library/email headerregistry email headerregistry ContentDispositionHeader content_disposition6199214 +Ref: 25686199214 +Ref: library/email headerregistry email headerregistry ContentTransferEncoding6199351 +Ref: 25696199351 +Ref: library/email headerregistry email headerregistry ContentTransferEncoding cte6199466 +Ref: 256a6199466 +Ref: library/email headerregistry email headerregistry HeaderRegistry6199635 +Ref: 254e6199635 +Ref: library/email headerregistry email headerregistry HeaderRegistry map_to_type6201355 +Ref: 256b6201355 +Ref: library/email headerregistry email headerregistry HeaderRegistry __getitem__6201690 +Ref: 256c6201690 +Ref: library/email headerregistry email headerregistry HeaderRegistry __call__6201813 +Ref: 256d6201813 +Ref: library/email headerregistry email headerregistry Address6202455 +Ref: 255a6202455 +Ref: library/email headerregistry email headerregistry Address display_name6203294 +Ref: 256e6203294 +Ref: library/email headerregistry email headerregistry Address username6203520 +Ref: 256f6203520 +Ref: library/email headerregistry email headerregistry Address domain6203640 +Ref: 25706203640 +Ref: library/email headerregistry email headerregistry Address addr_spec6203720 +Ref: 25716203720 +Ref: library/email headerregistry email headerregistry Address __str__6203938 +Ref: 25726203938 +Ref: library/email headerregistry email headerregistry Group6204377 +Ref: 25576204377 +Ref: library/email headerregistry email headerregistry Group display_name6204926 +Ref: 25586204926 +Ref: library/email headerregistry email headerregistry Group addresses6205181 +Ref: 25736205181 +Ref: library/email headerregistry email headerregistry Group __str__6205329 +Ref: 25746205329 +Ref: email headerregistry Custom Header Objects-Footnote-16205769 +Ref: email headerregistry Custom Header Objects-Footnote-26205849 +Ref: email headerregistry Custom Header Objects-Footnote-36205921 +Ref: email headerregistry Custom Header Objects-Footnote-46205970 +Ref: email headerregistry Custom Header Objects-Footnote-56206019 +Ref: email headerregistry Custom Header Objects-Footnote-66206068 +Ref: email headerregistry Custom Header Objects-Footnote-76206117 +Ref: email headerregistry Custom Header Objects-Footnote-86206166 +Ref: email headerregistry Custom Header Objects-Footnote-96206215 +Ref: email headerregistry Custom Header Objects-Footnote-106206264 +Ref: email headerregistry Custom Header Objects-Footnote-116206314 +Ref: email headerregistry Custom Header Objects-Footnote-126206364 +Ref: email headerregistry Custom Header Objects-Footnote-136206414 +Ref: email headerregistry Custom Header Objects-Footnote-146206464 +Ref: email headerregistry Custom Header Objects-Footnote-156206514 +Ref: email headerregistry Custom Header Objects-Footnote-166206564 +Ref: email headerregistry Custom Header Objects-Footnote-176206614 +Node: email contentmanager Managing MIME Content6206664 +Ref: library/email contentmanager doc6206861 +Ref: 25756206861 +Ref: library/email contentmanager email-contentmanager-managing-mime-content6206861 +Ref: 25766206861 +Ref: library/email contentmanager module-email contentmanager6206861 +Ref: 6b6206861 +Ref: library/email contentmanager email contentmanager ContentManager6207118 +Ref: 25776207118 +Ref: library/email contentmanager email contentmanager ContentManager get_content6207407 +Ref: 24fc6207407 +Ref: library/email contentmanager email contentmanager ContentManager set_content6208261 +Ref: 24fe6208261 +Ref: library/email contentmanager email contentmanager ContentManager add_get_handler6209884 +Ref: 25796209884 +Ref: library/email contentmanager email contentmanager ContentManager add_set_handler6210076 +Ref: 257b6210076 +Ref: email contentmanager Managing MIME Content-Footnote-16210441 +Ref: email contentmanager Managing MIME Content-Footnote-26210521 +Node: Content Manager Instances6210593 +Ref: library/email contentmanager content-manager-instances6210697 +Ref: 257d6210697 +Ref: library/email contentmanager email contentmanager raw_data_manager6211037 +Ref: 25316211037 +Ref: library/email contentmanager email contentmanager get_content6211767 +Ref: 257a6211767 +Ref: library/email contentmanager email contentmanager set_content6212366 +Ref: 257c6212366 +Ref: Content Manager Instances-Footnote-16216785 +Node: email Examples6216834 +Ref: library/email examples doc6217063 +Ref: 257e6217063 +Ref: library/email examples email-examples6217063 +Ref: 8456217063 +Ref: library/email examples id16217063 +Ref: 257f6217063 +Ref: email Examples-Footnote-16231728 +Ref: email Examples-Footnote-26231776 +Node: email message Message Representing an email message using the compat32 API6231862 +Ref: library/email compat32-message doc6232104 +Ref: 25806232104 +Ref: library/email compat32-message compat32-message6232104 +Ref: 24cf6232104 +Ref: library/email compat32-message email-message-message-representing-an-email-message-using-the-compat32-api6232104 +Ref: 25816232104 +Ref: library/email compat32-message email message Message6234661 +Ref: 5416234661 +Ref: library/email compat32-message email message Message as_string6235194 +Ref: 83f6235194 +Ref: library/email compat32-message email message Message __str__6237342 +Ref: 25826237342 +Ref: library/email compat32-message email message Message as_bytes6237507 +Ref: 8406237507 +Ref: library/email compat32-message email message Message __bytes__6239063 +Ref: 8416239063 +Ref: library/email compat32-message email message Message is_multipart6239268 +Ref: 24f56239268 +Ref: library/email compat32-message email message Message set_unixfrom6239944 +Ref: 25836239944 +Ref: library/email compat32-message email message Message get_unixfrom6240084 +Ref: 25846240084 +Ref: library/email compat32-message email message Message attach6240241 +Ref: 25036240241 +Ref: library/email compat32-message email message Message get_payload6240859 +Ref: b4b6240859 +Ref: library/email compat32-message email message Message set_payload6243510 +Ref: 25856243510 +Ref: library/email compat32-message email message Message set_charset6243978 +Ref: 25866243978 +Ref: library/email compat32-message email message Message get_charset6245737 +Ref: 25876245737 +Ref: library/email compat32-message email message Message __len__6247189 +Ref: 25896247189 +Ref: library/email compat32-message email message Message __contains__6247287 +Ref: 258a6247287 +Ref: library/email compat32-message email message Message __getitem__6247657 +Ref: 258b6247657 +Ref: library/email compat32-message email message Message __setitem__6248192 +Ref: 258d6248192 +Ref: library/email compat32-message email message Message __delitem__6248749 +Ref: 258e6248749 +Ref: library/email compat32-message email message Message keys6248975 +Ref: 25886248975 +Ref: library/email compat32-message email message Message values6249069 +Ref: 258f6249069 +Ref: library/email compat32-message email message Message items6249159 +Ref: 25906249159 +Ref: library/email compat32-message email message Message get6249290 +Ref: 25916249290 +Ref: library/email compat32-message email message Message get_all6249614 +Ref: 258c6249614 +Ref: library/email compat32-message email message Message add_header6249851 +Ref: 25926249851 +Ref: library/email compat32-message email message Message replace_header6251817 +Ref: 25936251817 +Ref: library/email compat32-message email message Message get_content_type6252106 +Ref: 25946252106 +Ref: library/email compat32-message email message Message get_content_maintype6252964 +Ref: 25966252964 +Ref: library/email compat32-message email message Message get_content_subtype6253170 +Ref: 25976253170 +Ref: library/email compat32-message email message Message get_default_type6253363 +Ref: 25956253363 +Ref: library/email compat32-message email message Message set_default_type6253683 +Ref: 25986253683 +Ref: library/email compat32-message email message Message get_params6253970 +Ref: 25996253970 +Ref: library/email compat32-message email message Message get_param6255026 +Ref: 259a6255026 +Ref: library/email compat32-message email message Message set_param6257028 +Ref: 8426257028 +Ref: library/email compat32-message email message Message del_param6258295 +Ref: 259c6258295 +Ref: library/email compat32-message email message Message set_type6258736 +Ref: 259d6258736 +Ref: library/email compat32-message email message Message get_filename6259609 +Ref: 259e6259609 +Ref: library/email compat32-message email message Message get_boundary6260148 +Ref: 259f6260148 +Ref: library/email compat32-message email message Message set_boundary6260512 +Ref: 25a06260512 +Ref: library/email compat32-message email message Message get_content_charset6261327 +Ref: 25a16261327 +Ref: library/email compat32-message email message Message get_charsets6261815 +Ref: 25a26261815 +Ref: library/email compat32-message email message Message get_content_disposition6262517 +Ref: 6d16262517 +Ref: library/email compat32-message email message Message walk6262880 +Ref: 25a36262880 +Ref: library/email compat32-message email message Message preamble6265000 +Ref: 25a46265000 +Ref: library/email compat32-message email message Message epilogue6266262 +Ref: 25a56266262 +Ref: library/email compat32-message email message Message defects6266660 +Ref: 25a66266660 +Ref: email message Message Representing an email message using the compat32 API-Footnote-16266938 +Ref: email message Message Representing an email message using the compat32 API-Footnote-26266987 +Ref: email message Message Representing an email message using the compat32 API-Footnote-36267036 +Ref: email message Message Representing an email message using the compat32 API-Footnote-46267085 +Ref: email message Message Representing an email message using the compat32 API-Footnote-56267134 +Ref: email message Message Representing an email message using the compat32 API-Footnote-66267183 +Ref: email message Message Representing an email message using the compat32 API-Footnote-76267232 +Ref: email message Message Representing an email message using the compat32 API-Footnote-86267281 +Ref: email message Message Representing an email message using the compat32 API-Footnote-96267330 +Ref: email message Message Representing an email message using the compat32 API-Footnote-106267379 +Ref: email message Message Representing an email message using the compat32 API-Footnote-116267429 +Ref: email message Message Representing an email message using the compat32 API-Footnote-126267479 +Node: email mime Creating email and MIME objects from scratch6267529 +Ref: library/email mime doc6267795 +Ref: 25a76267795 +Ref: library/email mime email-mime-creating-email-and-mime-objects-from-scratch6267795 +Ref: 25a86267795 +Ref: library/email mime module-email mime6267795 +Ref: 736267795 +Ref: library/email mime email mime base MIMEBase6269031 +Ref: 25a96269031 +Ref: library/email mime email mime nonmultipart MIMENonMultipart6270174 +Ref: 254a6270174 +Ref: library/email mime email mime multipart MIMEMultipart6270666 +Ref: 25aa6270666 +Ref: library/email mime email mime application MIMEApplication6271979 +Ref: 25ab6271979 +Ref: library/email mime email mime audio MIMEAudio6273321 +Ref: 25ac6273321 +Ref: library/email mime email mime image MIMEImage6274908 +Ref: 254b6274908 +Ref: library/email mime email mime message MIMEMessage6276506 +Ref: 25ad6276506 +Ref: library/email mime email mime text MIMEText6277190 +Ref: 6d36277190 +Ref: email mime Creating email and MIME objects from scratch-Footnote-16278828 +Node: email header Internationalized headers6278895 +Ref: library/email header doc6279128 +Ref: 25ae6279128 +Ref: library/email header email-header-internationalized-headers6279128 +Ref: 25af6279128 +Ref: library/email header module-email header6279128 +Ref: 6f6279128 +Ref: library/email header email header Header6281825 +Ref: 25416281825 +Ref: library/email header email header Header append6283697 +Ref: 25b06283697 +Ref: library/email header email header Header encode6285031 +Ref: 25b16285031 +Ref: library/email header email header Header __str__6286672 +Ref: 25b26286672 +Ref: library/email header email header Header __eq__6287171 +Ref: 25b36287171 +Ref: library/email header email header Header __ne__6287307 +Ref: 25b46287307 +Ref: library/email header email header decode_header6287531 +Ref: 9f96287531 +Ref: library/email header email header make_header6288187 +Ref: 9fa6288187 +Ref: email header Internationalized headers-Footnote-16288884 +Ref: email header Internationalized headers-Footnote-26288955 +Ref: email header Internationalized headers-Footnote-36289004 +Ref: email header Internationalized headers-Footnote-46289052 +Ref: email header Internationalized headers-Footnote-56289101 +Ref: email header Internationalized headers-Footnote-66289150 +Ref: email header Internationalized headers-Footnote-76289199 +Ref: email header Internationalized headers-Footnote-86289248 +Ref: email header Internationalized headers-Footnote-96289297 +Ref: email header Internationalized headers-Footnote-106289346 +Ref: email header Internationalized headers-Footnote-116289396 +Ref: email header Internationalized headers-Footnote-126289446 +Ref: email header Internationalized headers-Footnote-136289496 +Ref: email header Internationalized headers-Footnote-146289546 +Ref: email header Internationalized headers-Footnote-156289596 +Node: email charset Representing character sets6289646 +Ref: library/email charset doc6289847 +Ref: 25b56289847 +Ref: library/email charset email-charset-representing-character-sets6289847 +Ref: 25b66289847 +Ref: library/email charset module-email charset6289847 +Ref: 6a6289847 +Ref: library/email charset email charset Charset6290658 +Ref: 6d46290658 +Ref: library/email charset email charset Charset input_charset6292183 +Ref: 25b76292183 +Ref: library/email charset email charset Charset header_encoding6292439 +Ref: 25b86292439 +Ref: library/email charset email charset Charset body_encoding6292825 +Ref: 25b96292825 +Ref: library/email charset email charset Charset output_charset6293102 +Ref: 25ba6293102 +Ref: library/email charset email charset Charset input_codec6293441 +Ref: 25bb6293441 +Ref: library/email charset email charset Charset output_codec6293655 +Ref: 25bc6293655 +Ref: library/email charset email charset Charset get_body_encoding6293967 +Ref: 25bd6293967 +Ref: library/email charset email charset Charset get_output_charset6294684 +Ref: 25be6294684 +Ref: library/email charset email charset Charset header_encode6294888 +Ref: 25bf6294888 +Ref: library/email charset email charset Charset header_encode_lines6295096 +Ref: 25c06295096 +Ref: library/email charset email charset Charset body_encode6295529 +Ref: 25c16295529 +Ref: library/email charset email charset Charset __str__6295861 +Ref: 25c26295861 +Ref: library/email charset email charset Charset __eq__6296032 +Ref: 25c36296032 +Ref: library/email charset email charset Charset __ne__6296168 +Ref: 25c46296168 +Ref: library/email charset email charset add_charset6296462 +Ref: 25c56296462 +Ref: library/email charset email charset add_alias6297787 +Ref: 25c76297787 +Ref: library/email charset email charset add_codec6298113 +Ref: 25c66298113 +Ref: email charset Representing character sets-Footnote-16298507 +Node: email encoders Encoders6298579 +Ref: library/email encoders doc6298777 +Ref: 25c86298777 +Ref: library/email encoders email-encoders-encoders6298777 +Ref: 25c96298777 +Ref: library/email encoders module-email encoders6298777 +Ref: 6c6298777 +Ref: library/email encoders email encoders encode_quopri6300463 +Ref: 25ca6300463 +Ref: library/email encoders email encoders encode_base646300791 +Ref: 25cb6300791 +Ref: library/email encoders email encoders encode_7or8bit6301187 +Ref: 25cc6301187 +Ref: library/email encoders email encoders encode_noop6301447 +Ref: 25cd6301447 +Ref: email encoders Encoders-Footnote-16301627 +Ref: email encoders Encoders-Footnote-26301700 +Node: email utils Miscellaneous utilities6301821 +Ref: library/email utils doc6302003 +Ref: 25ce6302003 +Ref: library/email utils email-utils-miscellaneous-utilities6302003 +Ref: 25cf6302003 +Ref: library/email utils module-email utils6302003 +Ref: 766302003 +Ref: library/email utils email utils localtime6302299 +Ref: a006302299 +Ref: library/email utils email utils make_msgid6303122 +Ref: 25d06303122 +Ref: library/email utils email utils quote6304019 +Ref: 25d16304019 +Ref: library/email utils email utils unquote6304198 +Ref: 24ee6304198 +Ref: library/email utils email utils parseaddr6304468 +Ref: 25d26304468 +Ref: library/email utils email utils formataddr6304827 +Ref: b156304827 +Ref: library/email utils email utils getaddresses6305508 +Ref: 25d36305508 +Ref: library/email utils email utils parsedate6306146 +Ref: 25d46306146 +Ref: library/email utils email utils parsedate_tz6306775 +Ref: 25d56306775 +Ref: library/email utils email utils parsedate_to_datetime6307354 +Ref: 9ff6307354 +Ref: library/email utils email utils mktime_tz6308065 +Ref: 25d66308065 +Ref: library/email utils email utils formatdate6308302 +Ref: 25d76308302 +Ref: library/email utils email utils format_datetime6309248 +Ref: 9fe6309248 +Ref: library/email utils email utils decode_rfc22316309936 +Ref: 25d86309936 +Ref: library/email utils email utils encode_rfc22316310036 +Ref: 25d96310036 +Ref: library/email utils email utils collapse_rfc2231_value6310444 +Ref: 259b6310444 +Ref: library/email utils email utils decode_params6311298 +Ref: 25da6311298 +Ref: email utils Miscellaneous utilities-Footnote-16311554 +Ref: email utils Miscellaneous utilities-Footnote-26311624 +Ref: email utils Miscellaneous utilities-Footnote-36311673 +Ref: email utils Miscellaneous utilities-Footnote-46311722 +Ref: email utils Miscellaneous utilities-Footnote-56311771 +Ref: email utils Miscellaneous utilities-Footnote-66311820 +Ref: email utils Miscellaneous utilities-Footnote-76312087 +Ref: email utils Miscellaneous utilities-Footnote-86312136 +Ref: email utils Miscellaneous utilities-Footnote-96312185 +Ref: email utils Miscellaneous utilities-Footnote-106312234 +Ref: email utils Miscellaneous utilities-Footnote-116312284 +Ref: email utils Miscellaneous utilities-Footnote-126312334 +Node: email iterators Iterators6312384 +Ref: library/email iterators doc6312534 +Ref: 25db6312534 +Ref: library/email iterators email-iterators-iterators6312534 +Ref: 25dc6312534 +Ref: library/email iterators module-email iterators6312534 +Ref: 716312534 +Ref: library/email iterators email iterators body_line_iterator6312934 +Ref: 25dd6312934 +Ref: library/email iterators email iterators typed_subpart_iterator6313511 +Ref: 25de6313511 +Ref: library/email iterators email iterators _structure6314229 +Ref: 25df6314229 +Ref: email iterators Iterators-Footnote-16315809 +Node: json — JSON encoder and decoder6315883 +Ref: library/json doc6316070 +Ref: 25e06316070 +Ref: library/json json-json-encoder-and-decoder6316070 +Ref: 25e16316070 +Ref: library/json module-json6316070 +Ref: a46316070 +Ref: json — JSON encoder and decoder-Footnote-16320273 +Ref: json — JSON encoder and decoder-Footnote-26320345 +Ref: json — JSON encoder and decoder-Footnote-36320369 +Ref: json — JSON encoder and decoder-Footnote-46320418 +Ref: json — JSON encoder and decoder-Footnote-56320467 +Ref: json — JSON encoder and decoder-Footnote-66320546 +Ref: json — JSON encoder and decoder-Footnote-76320595 +Ref: json — JSON encoder and decoder-Footnote-86320861 +Node: Basic Usage6320886 +Ref: library/json basic-usage6320997 +Ref: 25e36320997 +Ref: library/json json dump6321040 +Ref: 6046321040 +Ref: library/json json dumps6324573 +Ref: 6056324573 +Ref: library/json json load6325398 +Ref: 55f6325398 +Ref: library/json json loads6328010 +Ref: 5606328010 +Ref: Basic Usage-Footnote-16328930 +Node: Encoders and Decoders6328961 +Ref: library/json encoders-and-decoders6329095 +Ref: 25e66329095 +Ref: library/json json JSONDecoder6329158 +Ref: 6076329158 +Ref: library/json json-to-py-table6329412 +Ref: 25e56329412 +Ref: library/json json JSONDecoder decode6332312 +Ref: 25e76332312 +Ref: library/json json JSONDecoder raw_decode6332564 +Ref: 25e86332564 +Ref: library/json json JSONEncoder6332928 +Ref: 6066332928 +Ref: library/json py-to-json-table6333229 +Ref: 25e46333229 +Ref: library/json json JSONEncoder default6337537 +Ref: 25e96337537 +Ref: library/json json JSONEncoder encode6338226 +Ref: 25ea6338226 +Ref: library/json json JSONEncoder iterencode6338471 +Ref: 25eb6338471 +Node: Exceptions<13>6338732 +Ref: library/json exceptions6338895 +Ref: 25ec6338895 +Ref: library/json json JSONDecodeError6338936 +Ref: 7066338936 +Ref: library/json json JSONDecodeError msg6339077 +Ref: 25ed6339077 +Ref: library/json json JSONDecodeError doc6339144 +Ref: 25ee6339144 +Ref: library/json json JSONDecodeError pos6339212 +Ref: 25ef6339212 +Ref: library/json json JSONDecodeError lineno6339295 +Ref: 25f06339295 +Ref: library/json json JSONDecodeError colno6339367 +Ref: 25f16339367 +Node: Standard Compliance and Interoperability6339466 +Ref: library/json standard-compliance-and-interoperability6339633 +Ref: 25f26339633 +Ref: Standard Compliance and Interoperability-Footnote-16340783 +Ref: Standard Compliance and Interoperability-Footnote-26340832 +Node: Character Encodings6340911 +Ref: library/json character-encodings6341046 +Ref: 25f36341046 +Node: Infinite and NaN Number Values6342377 +Ref: library/json infinite-and-nan-number-values6342552 +Ref: 25f46342552 +Node: Repeated Names Within an Object6343340 +Ref: library/json repeated-names-within-an-object6343533 +Ref: 25f56343533 +Node: Top-level Non-Object Non-Array Values6344056 +Ref: library/json top-level-non-object-non-array-values6344245 +Ref: 25f66344245 +Ref: Top-level Non-Object Non-Array Values-Footnote-16344899 +Ref: Top-level Non-Object Non-Array Values-Footnote-26344948 +Node: Implementation Limitations6344997 +Ref: library/json implementation-limitations6345146 +Ref: 25f76345146 +Node: Command Line Interface<2>6346096 +Ref: library/json command-line-interface6346240 +Ref: 25f86346240 +Ref: library/json json-commandline6346240 +Ref: 25e26346240 +Ref: library/json module-json tool6346240 +Ref: a56346240 +Ref: Command Line Interface<2>-Footnote-16347168 +Node: Command line options<2>6347236 +Ref: library/json command-line-options6347321 +Ref: 25fa6347321 +Ref: library/json cmdoption-json-tool-arg-infile6347384 +Ref: 25fb6347384 +Ref: library/json cmdoption-json-tool-arg-outfile6347868 +Ref: 25fc6347868 +Ref: library/json cmdoption-json-tool-sort-keys6348011 +Ref: 25f96348011 +Ref: library/json cmdoption-json-tool-json-lines6348131 +Ref: 25fd6348131 +Ref: library/json cmdoption-json-tool-h6348245 +Ref: 25fe6348245 +Node: mailcap — Mailcap file handling6348306 +Ref: library/mailcap doc6348500 +Ref: 25ff6348500 +Ref: library/mailcap mailcap-mailcap-file-handling6348500 +Ref: 26006348500 +Ref: library/mailcap module-mailcap6348500 +Ref: af6348500 +Ref: library/mailcap mailcap findmatch6349479 +Ref: 26016349479 +Ref: library/mailcap mailcap getcaps6351568 +Ref: 26026351568 +Ref: mailcap — Mailcap file handling-Footnote-16352408 +Ref: mailcap — Mailcap file handling-Footnote-26352474 +Ref: mailcap — Mailcap file handling-Footnote-36352523 +Node: mailbox — Manipulate mailboxes in various formats6352572 +Ref: library/mailbox doc6352774 +Ref: 26036352774 +Ref: library/mailbox mailbox-manipulate-mailboxes-in-various-formats6352774 +Ref: 26046352774 +Ref: library/mailbox module-mailbox6352774 +Ref: ae6352774 +Ref: mailbox — Manipulate mailboxes in various formats-Footnote-16353666 +Node: Mailbox objects6353732 +Ref: library/mailbox id16353859 +Ref: 26056353859 +Ref: library/mailbox mailbox-objects6353859 +Ref: 26066353859 +Ref: library/mailbox mailbox Mailbox6353918 +Ref: be36353918 +Ref: library/mailbox mailbox Mailbox add6356720 +Ref: be26356720 +Ref: library/mailbox mailbox Mailbox remove6357510 +Ref: 26076357510 +Ref: library/mailbox mailbox Mailbox __delitem__6357540 +Ref: 260c6357540 +Ref: library/mailbox mailbox Mailbox discard6357575 +Ref: 26086357575 +Ref: library/mailbox mailbox Mailbox __setitem__6358120 +Ref: 260d6358120 +Ref: library/mailbox mailbox Mailbox iterkeys6359003 +Ref: 260e6359003 +Ref: library/mailbox mailbox Mailbox keys6359032 +Ref: 260f6359032 +Ref: library/mailbox mailbox Mailbox itervalues6359217 +Ref: 26106359217 +Ref: library/mailbox mailbox Mailbox __iter__6359248 +Ref: 26116359248 +Ref: library/mailbox mailbox Mailbox values6359277 +Ref: 26126359277 +Ref: library/mailbox mailbox Mailbox iteritems6359916 +Ref: 26136359916 +Ref: library/mailbox mailbox Mailbox items6359946 +Ref: 26146359946 +Ref: library/mailbox mailbox Mailbox get6360469 +Ref: 26156360469 +Ref: library/mailbox mailbox Mailbox __getitem__6360510 +Ref: 26166360510 +Ref: library/mailbox mailbox Mailbox get_message6361096 +Ref: 26176361096 +Ref: library/mailbox mailbox Mailbox get_bytes6361384 +Ref: be56361384 +Ref: library/mailbox mailbox Mailbox get_string6361617 +Ref: be66361617 +Ref: library/mailbox mailbox Mailbox get_file6361956 +Ref: be46361956 +Ref: library/mailbox mailbox Mailbox __contains__6362931 +Ref: 26186362931 +Ref: library/mailbox mailbox Mailbox __len__6363065 +Ref: 26196363065 +Ref: library/mailbox mailbox Mailbox clear6363148 +Ref: 261a6363148 +Ref: library/mailbox mailbox Mailbox pop6363224 +Ref: 261b6363224 +Ref: library/mailbox mailbox Mailbox popitem6363667 +Ref: 261c6363667 +Ref: library/mailbox mailbox Mailbox update6364188 +Ref: 261d6364188 +Ref: library/mailbox mailbox Mailbox flush6364906 +Ref: 261e6364906 +Ref: library/mailbox mailbox Mailbox lock6365197 +Ref: 26096365197 +Ref: library/mailbox mailbox Mailbox unlock6365616 +Ref: 260a6365616 +Ref: library/mailbox mailbox Mailbox close6365696 +Ref: 26206365696 +Node: Maildir6365948 +Ref: library/mailbox mailbox-maildir6366020 +Ref: 26216366020 +Ref: library/mailbox maildir6366020 +Ref: 26226366020 +Ref: library/mailbox mailbox Maildir6366063 +Ref: ca86366063 +Ref: library/mailbox mailbox Maildir list_folders6368912 +Ref: 26246368912 +Ref: library/mailbox mailbox Maildir get_folder6369000 +Ref: 26256369000 +Ref: library/mailbox mailbox Maildir add_folder6369240 +Ref: 26276369240 +Ref: library/mailbox mailbox Maildir remove_folder6369397 +Ref: 26286369397 +Ref: library/mailbox mailbox Maildir clean6369639 +Ref: 262a6369639 +Ref: library/mailbox mailbox Maildir add6369979 +Ref: 262b6369979 +Ref: library/mailbox mailbox Maildir __setitem__6370010 +Ref: 262c6370010 +Ref: library/mailbox mailbox Maildir update6370054 +Ref: 262d6370054 +Ref: library/mailbox mailbox Maildir flush6370467 +Ref: 262e6370467 +Ref: library/mailbox mailbox Maildir lock6370602 +Ref: 262f6370602 +Ref: library/mailbox mailbox Maildir unlock6370627 +Ref: 26306370627 +Ref: library/mailbox mailbox Maildir close6370760 +Ref: 26316370760 +Ref: library/mailbox mailbox Maildir get_file6370957 +Ref: 26326370957 +Ref: Maildir-Footnote-16371493 +Ref: Maildir-Footnote-26371541 +Node: mbox6371585 +Ref: library/mailbox mailbox-mbox6371668 +Ref: 26336371668 +Ref: library/mailbox mbox6371668 +Ref: 26346371668 +Ref: library/mailbox mailbox mbox6371705 +Ref: 260b6371705 +Ref: library/mailbox mailbox mbox get_file6373141 +Ref: 26356373141 +Ref: library/mailbox mailbox mbox lock6373354 +Ref: 26366373354 +Ref: library/mailbox mailbox mbox unlock6373379 +Ref: 26376373379 +Ref: mbox-Footnote-16373956 +Ref: mbox-Footnote-26374016 +Ref: mbox-Footnote-36374068 +Node: MH6374146 +Ref: library/mailbox mailbox-mh6374227 +Ref: 26386374227 +Ref: library/mailbox mh6374227 +Ref: 26396374227 +Ref: library/mailbox mailbox MH6374260 +Ref: 263a6374260 +Ref: library/mailbox mailbox MH list_folders6375706 +Ref: 263c6375706 +Ref: library/mailbox mailbox MH get_folder6375794 +Ref: 263d6375794 +Ref: library/mailbox mailbox MH add_folder6376031 +Ref: 263e6376031 +Ref: library/mailbox mailbox MH remove_folder6376185 +Ref: 263f6376185 +Ref: library/mailbox mailbox MH get_sequences6376427 +Ref: 26406376427 +Ref: library/mailbox mailbox MH set_sequences6376604 +Ref: 26416376604 +Ref: library/mailbox mailbox MH pack6376842 +Ref: 26426376842 +Ref: library/mailbox mailbox MH remove6377263 +Ref: 26436377263 +Ref: library/mailbox mailbox MH __delitem__6377293 +Ref: 26446377293 +Ref: library/mailbox mailbox MH discard6377328 +Ref: 26456377328 +Ref: library/mailbox mailbox MH lock6377537 +Ref: 26466377537 +Ref: library/mailbox mailbox MH unlock6377562 +Ref: 26476377562 +Ref: library/mailbox mailbox MH get_file6377937 +Ref: 26486377937 +Ref: library/mailbox mailbox MH flush6378129 +Ref: 26496378129 +Ref: library/mailbox mailbox MH close6378259 +Ref: 264a6378259 +Ref: MH-Footnote-16378726 +Ref: MH-Footnote-26378761 +Node: Babyl6378806 +Ref: library/mailbox babyl6378887 +Ref: 264b6378887 +Ref: library/mailbox mailbox-babyl6378887 +Ref: 264c6378887 +Ref: library/mailbox mailbox Babyl6378926 +Ref: 264d6378926 +Ref: library/mailbox mailbox Babyl get_labels6380495 +Ref: 264f6380495 +Ref: library/mailbox mailbox Babyl get_file6381028 +Ref: 26506381028 +Ref: library/mailbox mailbox Babyl lock6381548 +Ref: 26516381548 +Ref: library/mailbox mailbox Babyl unlock6381573 +Ref: 26526381573 +Ref: Babyl-Footnote-16381967 +Ref: Babyl-Footnote-26382011 +Node: MMDF6382089 +Ref: library/mailbox mailbox-mmdf6382159 +Ref: 26536382159 +Ref: library/mailbox mmdf6382159 +Ref: 26546382159 +Ref: library/mailbox mailbox MMDF6382196 +Ref: 26556382196 +Ref: library/mailbox mailbox MMDF get_file6383459 +Ref: 26576383459 +Ref: library/mailbox mailbox MMDF lock6383672 +Ref: 26586383672 +Ref: library/mailbox mailbox MMDF unlock6383697 +Ref: 26596383697 +Ref: MMDF-Footnote-16384134 +Ref: MMDF-Footnote-26384194 +Node: Message objects6384237 +Ref: library/mailbox mailbox-message-objects6384387 +Ref: 265a6384387 +Ref: library/mailbox message-objects6384387 +Ref: 265b6384387 +Ref: library/mailbox mailbox Message6384446 +Ref: be76384446 +Ref: Message objects-Footnote-16386492 +Node: MaildirMessage6386541 +Ref: library/mailbox mailbox-maildirmessage6386627 +Ref: 265c6386627 +Ref: library/mailbox maildirmessage6386627 +Ref: 265d6386627 +Ref: library/mailbox mailbox MaildirMessage6386684 +Ref: 26236386684 +Ref: library/mailbox mailbox MaildirMessage get_subdir6388540 +Ref: 265e6388540 +Ref: library/mailbox mailbox MaildirMessage set_subdir6389057 +Ref: 265f6389057 +Ref: library/mailbox mailbox MaildirMessage get_flags6389228 +Ref: 26606389228 +Ref: library/mailbox mailbox MaildirMessage set_flags6389700 +Ref: 26616389700 +Ref: library/mailbox mailbox MaildirMessage add_flag6389804 +Ref: 26626389804 +Ref: library/mailbox mailbox MaildirMessage remove_flag6390154 +Ref: 26636390154 +Ref: library/mailbox mailbox MaildirMessage get_date6390509 +Ref: 26646390509 +Ref: library/mailbox mailbox MaildirMessage set_date6390665 +Ref: 26656390665 +Ref: library/mailbox mailbox MaildirMessage get_info6390830 +Ref: 26666390830 +Ref: library/mailbox mailbox MaildirMessage set_info6391053 +Ref: 26676391053 +Node: mboxMessage6393871 +Ref: library/mailbox mailbox-mboxmessage6393975 +Ref: 26686393975 +Ref: library/mailbox mboxmessage6393975 +Ref: 26696393975 +Ref: library/mailbox mailbox mboxMessage6394026 +Ref: 24d76394026 +Ref: library/mailbox mailbox mboxMessage get_from6395828 +Ref: 266a6395828 +Ref: library/mailbox mailbox mboxMessage set_from6396059 +Ref: 266b6396059 +Ref: library/mailbox mailbox mboxMessage get_flags6396595 +Ref: 266c6396595 +Ref: library/mailbox mailbox mboxMessage set_flags6396943 +Ref: 266d6396943 +Ref: library/mailbox mailbox mboxMessage add_flag6397234 +Ref: 266e6397234 +Ref: library/mailbox mailbox mboxMessage remove_flag6397454 +Ref: 266f6397454 +Node: MHMessage6400455 +Ref: library/mailbox mailbox-mhmessage6400557 +Ref: 26706400557 +Ref: library/mailbox mhmessage6400557 +Ref: 26716400557 +Ref: library/mailbox mailbox MHMessage6400606 +Ref: 263b6400606 +Ref: library/mailbox mailbox MHMessage get_sequences6401572 +Ref: 26726401572 +Ref: library/mailbox mailbox MHMessage set_sequences6401695 +Ref: 26736401695 +Ref: library/mailbox mailbox MHMessage add_sequence6401803 +Ref: 26746401803 +Ref: library/mailbox mailbox MHMessage remove_sequence6401933 +Ref: 26756401933 +Node: BabylMessage6403819 +Ref: library/mailbox babylmessage6403921 +Ref: 26766403921 +Ref: library/mailbox mailbox-babylmessage6403921 +Ref: 26776403921 +Ref: library/mailbox mailbox BabylMessage6403976 +Ref: 264e6403976 +Ref: library/mailbox mailbox BabylMessage get_labels6405344 +Ref: 26786405344 +Ref: library/mailbox mailbox BabylMessage set_labels6405427 +Ref: 26796405427 +Ref: library/mailbox mailbox BabylMessage add_label6405527 +Ref: 267a6405527 +Ref: library/mailbox mailbox BabylMessage remove_label6405624 +Ref: 267b6405624 +Ref: library/mailbox mailbox BabylMessage get_visible6405729 +Ref: 267c6405729 +Ref: library/mailbox mailbox BabylMessage set_visible6405897 +Ref: 267d6405897 +Ref: library/mailbox mailbox BabylMessage update_visible6406247 +Ref: 267e6406247 +Node: MMDFMessage6408715 +Ref: library/mailbox mailbox-mmdfmessage6408799 +Ref: 267f6408799 +Ref: library/mailbox mmdfmessage6408799 +Ref: 26806408799 +Ref: library/mailbox mailbox MMDFMessage6408852 +Ref: 26566408852 +Ref: library/mailbox mailbox MMDFMessage get_from6410538 +Ref: 26816410538 +Ref: library/mailbox mailbox MMDFMessage set_from6410769 +Ref: 26826410769 +Ref: library/mailbox mailbox MMDFMessage get_flags6411305 +Ref: 26836411305 +Ref: library/mailbox mailbox MMDFMessage set_flags6411653 +Ref: 26846411653 +Ref: library/mailbox mailbox MMDFMessage add_flag6411944 +Ref: 26856411944 +Ref: library/mailbox mailbox MMDFMessage remove_flag6412164 +Ref: 26866412164 +Node: Exceptions<14>6415171 +Ref: library/mailbox exceptions6415318 +Ref: 26876415318 +Ref: library/mailbox mailbox Error6415440 +Ref: 26886415440 +Ref: library/mailbox mailbox NoSuchMailboxError6415534 +Ref: 26266415534 +Ref: library/mailbox mailbox NotEmptyError6415840 +Ref: 26296415840 +Ref: library/mailbox mailbox ExternalClashError6416002 +Ref: 261f6416002 +Ref: library/mailbox mailbox FormatError6416315 +Ref: 26896416315 +Node: Examples<18>6416512 +Ref: library/mailbox examples6416635 +Ref: 268a6416635 +Ref: library/mailbox mailbox-examples6416635 +Ref: 268b6416635 +Node: mimetypes — Map filenames to MIME types6418913 +Ref: library/mimetypes doc6419135 +Ref: 268c6419135 +Ref: library/mimetypes mimetypes-map-filenames-to-mime-types6419135 +Ref: 268d6419135 +Ref: library/mimetypes module-mimetypes6419135 +Ref: b26419135 +Ref: library/mimetypes mimetypes guess_type6420022 +Ref: 268f6420022 +Ref: library/mimetypes mimetypes guess_all_extensions6421349 +Ref: 26906421349 +Ref: library/mimetypes mimetypes guess_extension6421914 +Ref: 26916421914 +Ref: library/mimetypes mimetypes init6422618 +Ref: 268e6422618 +Ref: library/mimetypes mimetypes read_mime_types6423594 +Ref: 26936423594 +Ref: library/mimetypes mimetypes add_type6423971 +Ref: 26946423971 +Ref: library/mimetypes mimetypes inited6424428 +Ref: 26956424428 +Ref: library/mimetypes mimetypes knownfiles6424598 +Ref: 26926424598 +Ref: library/mimetypes mimetypes suffix_map6424808 +Ref: 26966424808 +Ref: library/mimetypes mimetypes encodings_map6425158 +Ref: 26976425158 +Ref: library/mimetypes mimetypes types_map6425257 +Ref: 26986425257 +Ref: library/mimetypes mimetypes common_types6425348 +Ref: 26996425348 +Ref: mimetypes — Map filenames to MIME types-Footnote-16425890 +Ref: mimetypes — Map filenames to MIME types-Footnote-26425958 +Node: MimeTypes Objects6426029 +Ref: library/mimetypes id16426124 +Ref: 269a6426124 +Ref: library/mimetypes mimetypes-objects6426124 +Ref: 269b6426124 +Ref: library/mimetypes mimetypes MimeTypes6426373 +Ref: 269c6426373 +Ref: library/mimetypes mimetypes MimeTypes suffix_map6427059 +Ref: 269f6427059 +Ref: library/mimetypes mimetypes MimeTypes encodings_map6427541 +Ref: 26a06427541 +Ref: library/mimetypes mimetypes MimeTypes types_map6427754 +Ref: 26a16427754 +Ref: library/mimetypes mimetypes MimeTypes types_map_inv6428090 +Ref: 26a26428090 +Ref: library/mimetypes mimetypes MimeTypes guess_extension6428440 +Ref: 26a36428440 +Ref: library/mimetypes mimetypes MimeTypes guess_type6428618 +Ref: 26a46428618 +Ref: library/mimetypes mimetypes MimeTypes guess_all_extensions6428785 +Ref: 26a56428785 +Ref: library/mimetypes mimetypes MimeTypes read6428973 +Ref: 269d6428973 +Ref: library/mimetypes mimetypes MimeTypes readfp6429286 +Ref: 269e6429286 +Ref: library/mimetypes mimetypes MimeTypes read_windows_registry6429617 +Ref: 26a66429617 +Node: base64 — Base16 Base32 Base64 Base85 Data Encodings6429958 +Ref: library/base64 doc6430171 +Ref: 26a76430171 +Ref: library/base64 base64-base16-base32-base64-base85-data-encodings6430171 +Ref: 26a86430171 +Ref: library/base64 module-base646430171 +Ref: e6430171 +Ref: library/base64 base64 b64encode6432038 +Ref: 26a96432038 +Ref: library/base64 base64 b64decode6432588 +Ref: 26aa6432588 +Ref: library/base64 base64 standard_b64encode6433432 +Ref: 26ab6433432 +Ref: library/base64 base64 standard_b64decode6433605 +Ref: 26ac6433605 +Ref: library/base64 base64 urlsafe_b64encode6433794 +Ref: 26ad6433794 +Ref: library/base64 base64 urlsafe_b64decode6434134 +Ref: 26ae6434134 +Ref: library/base64 base64 b32encode6434451 +Ref: 26af6434451 +Ref: library/base64 base64 b32decode6434597 +Ref: 95d6434597 +Ref: library/base64 base64 b16encode6435587 +Ref: 26b06435587 +Ref: library/base64 base64 b16decode6435733 +Ref: 26b16435733 +Ref: library/base64 base64 a85encode6436216 +Ref: 8246436216 +Ref: library/base64 base64 a85decode6437179 +Ref: 8256437179 +Ref: library/base64 base64 b85encode6438042 +Ref: 8266438042 +Ref: library/base64 base64 b85decode6438394 +Ref: 8276438394 +Ref: library/base64 base64 decode6438658 +Ref: 26b26438658 +Ref: library/base64 base64 decodebytes6438966 +Ref: 151e6438966 +Ref: library/base64 base64 decodestring6439194 +Ref: 26b36439194 +Ref: library/base64 base64 encode6439322 +Ref: 26b46439322 +Ref: library/base64 base64 encodebytes6439841 +Ref: 151d6439841 +Ref: library/base64 base64 encodestring6440226 +Ref: 26b56440226 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-16441034 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-26441099 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-36441148 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-46441197 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-56441246 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-66441295 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-76441344 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-86441393 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-96441442 +Ref: base64 — Base16 Base32 Base64 Base85 Data Encodings-Footnote-106441491 +Node: binhex — Encode and decode binhex4 files6441541 +Ref: library/binhex doc6441758 +Ref: 26b66441758 +Ref: library/binhex binhex-encode-and-decode-binhex4-files6441758 +Ref: 26b76441758 +Ref: library/binhex module-binhex6441758 +Ref: 116441758 +Ref: library/binhex binhex binhex6442180 +Ref: 26b86442180 +Ref: library/binhex binhex hexbin6442452 +Ref: 26b96442452 +Ref: library/binhex binhex Error6442850 +Ref: 26ba6442850 +Ref: binhex — Encode and decode binhex4 files-Footnote-16443287 +Node: Notes<2>6443352 +Ref: library/binhex binhex-notes6443439 +Ref: 26bb6443439 +Ref: library/binhex notes6443439 +Ref: 26bc6443439 +Node: binascii — Convert between binary and ASCII6443725 +Ref: library/binascii doc6443944 +Ref: 26bd6443944 +Ref: library/binascii binascii-convert-between-binary-and-ascii6443944 +Ref: 26be6443944 +Ref: library/binascii module-binascii6443944 +Ref: 106443944 +Ref: library/binascii binascii a2b_uu6444982 +Ref: 26bf6444982 +Ref: library/binascii binascii b2a_uu6445232 +Ref: 36d6445232 +Ref: library/binascii binascii a2b_base646445611 +Ref: 26c06445611 +Ref: library/binascii binascii b2a_base646445785 +Ref: 5236445785 +Ref: library/binascii binascii a2b_qp6446135 +Ref: 26c16446135 +Ref: library/binascii binascii b2a_qp6446432 +Ref: 26c26446432 +Ref: library/binascii binascii a2b_hqx6447200 +Ref: 26c36447200 +Ref: library/binascii binascii rledecode_hqx6447488 +Ref: 26c46447488 +Ref: library/binascii binascii rlecode_hqx6448034 +Ref: 26c66448034 +Ref: library/binascii binascii b2a_hqx6448159 +Ref: 26c76448159 +Ref: library/binascii binascii crc_hqx6448404 +Ref: 26c86448404 +Ref: library/binascii binascii crc326448709 +Ref: 26c96448709 +Ref: library/binascii binascii b2a_hex6449500 +Ref: 15206449500 +Ref: library/binascii binascii hexlify6449564 +Ref: 26ca6449564 +Ref: library/binascii binascii a2b_hex6450761 +Ref: 15216450761 +Ref: library/binascii binascii unhexlify6450801 +Ref: 26cb6450801 +Ref: library/binascii binascii Error6451329 +Ref: 95e6451329 +Ref: library/binascii binascii Incomplete6451433 +Ref: 26c56451433 +Ref: binascii — Convert between binary and ASCII-Footnote-16452050 +Ref: binascii — Convert between binary and ASCII-Footnote-26452099 +Node: quopri — Encode and decode MIME quoted-printable data6452148 +Ref: library/quopri doc6452364 +Ref: 26cc6452364 +Ref: library/quopri module-quopri6452364 +Ref: db6452364 +Ref: library/quopri quopri-encode-and-decode-mime-quoted-printable-data6452364 +Ref: 26cd6452364 +Ref: library/quopri quopri decode6453099 +Ref: 15236453099 +Ref: library/quopri quopri encode6453638 +Ref: 15226453638 +Ref: library/quopri quopri decodestring6454294 +Ref: 26ce6454294 +Ref: library/quopri quopri encodestring6454491 +Ref: 26cf6454491 +Ref: quopri — Encode and decode MIME quoted-printable data-Footnote-16454941 +Ref: quopri — Encode and decode MIME quoted-printable data-Footnote-26455006 +Ref: quopri — Encode and decode MIME quoted-printable data-Footnote-36455055 +Ref: quopri — Encode and decode MIME quoted-printable data-Footnote-46455104 +Ref: quopri — Encode and decode MIME quoted-printable data-Footnote-56455153 +Node: uu — Encode and decode uuencode files6455202 +Ref: library/uu doc6455364 +Ref: 26d06455364 +Ref: library/uu module-uu6455364 +Ref: 1236455364 +Ref: library/uu uu-encode-and-decode-uuencode-files6455364 +Ref: 26d16455364 +Ref: library/uu uu encode6456313 +Ref: 40f6456313 +Ref: library/uu uu decode6456841 +Ref: 15246456841 +Ref: library/uu uu Error6457529 +Ref: 26d26457529 +Ref: uu — Encode and decode uuencode files-Footnote-16457946 +Node: Structured Markup Processing Tools6458007 +Ref: library/markup doc6458175 +Ref: 26d36458175 +Ref: library/markup markup6458175 +Ref: 26d46458175 +Ref: library/markup structured-markup-processing-tools6458175 +Ref: 26d56458175 +Node: html — HyperText Markup Language support6459718 +Ref: library/html doc6459884 +Ref: 26d66459884 +Ref: library/html html-hypertext-markup-language-support6459884 +Ref: 26d76459884 +Ref: library/html module-html6459884 +Ref: 906459884 +Ref: library/html html escape6460150 +Ref: 9346460150 +Ref: library/html html unescape6460621 +Ref: 85a6460621 +Ref: html — HyperText Markup Language support-Footnote-16461322 +Node: html parser — Simple HTML and XHTML parser6461394 +Ref: library/html parser doc6461623 +Ref: 26d86461623 +Ref: library/html parser html-parser-simple-html-and-xhtml-parser6461623 +Ref: 26d96461623 +Ref: library/html parser module-html parser6461623 +Ref: 926461623 +Ref: library/html parser html parser HTMLParser6461999 +Ref: 7bf6461999 +Ref: html parser — Simple HTML and XHTML parser-Footnote-16463117 +Node: Example HTML Parser Application6463187 +Ref: library/html parser example-html-parser-application6463326 +Ref: 26da6463326 +Node: HTMLParser Methods6464532 +Ref: library/html parser htmlparser-methods6464692 +Ref: 26db6464692 +Ref: library/html parser html parser HTMLParser feed6464819 +Ref: 26dc6464819 +Ref: library/html parser html parser HTMLParser close6465086 +Ref: 26dd6465086 +Ref: library/html parser html parser HTMLParser reset6465447 +Ref: 26de6465447 +Ref: library/html parser html parser HTMLParser getpos6465590 +Ref: 26df6465590 +Ref: library/html parser html parser HTMLParser get_starttag_text6465669 +Ref: 26e06465669 +Ref: library/html parser html parser HTMLParser handle_starttag6466240 +Ref: 26e26466240 +Ref: library/html parser html parser HTMLParser handle_endtag6467036 +Ref: 26e36467036 +Ref: library/html parser html parser HTMLParser handle_startendtag6467243 +Ref: 26e16467243 +Ref: library/html parser html parser HTMLParser handle_data6467656 +Ref: 26e46467656 +Ref: library/html parser html parser HTMLParser handle_entityref6467858 +Ref: 26e56467858 +Ref: library/html parser html parser HTMLParser handle_charref6468162 +Ref: 26e66468162 +Ref: library/html parser html parser HTMLParser handle_comment6468608 +Ref: 26e76468608 +Ref: library/html parser html parser HTMLParser handle_decl6469134 +Ref: 26e86469134 +Ref: library/html parser html parser HTMLParser handle_pi6469418 +Ref: 26e96469418 +Ref: library/html parser html parser HTMLParser unknown_decl6470108 +Ref: 26ea6470108 +Node: Examples<19>6470470 +Ref: library/html parser examples6470590 +Ref: 26eb6470590 +Ref: library/html parser htmlparser-examples6470590 +Ref: 26ec6470590 +Node: html entities — Definitions of HTML general entities6473999 +Ref: library/html entities doc6474208 +Ref: 26ed6474208 +Ref: library/html entities html-entities-definitions-of-html-general-entities6474208 +Ref: 26ee6474208 +Ref: library/html entities module-html entities6474208 +Ref: 916474208 +Ref: library/html entities html entities html56474589 +Ref: a156474589 +Ref: library/html entities html entities entitydefs6475074 +Ref: 26f16475074 +Ref: library/html entities html entities name2codepoint6475213 +Ref: 26ef6475213 +Ref: library/html entities html entities codepoint2name6475333 +Ref: 26f06475333 +Ref: html entities — Definitions of HTML general entities-Footnote-16475480 +Ref: html entities — Definitions of HTML general entities-Footnote-26475552 +Node: XML Processing Modules6475641 +Ref: library/xml doc6475855 +Ref: 26f26475855 +Ref: library/xml module-xml6475855 +Ref: 1336475855 +Ref: library/xml xml6475855 +Ref: 26f36475855 +Ref: library/xml xml-processing-modules6475855 +Ref: 26f46475855 +Ref: XML Processing Modules-Footnote-16477315 +Node: XML vulnerabilities6477375 +Ref: library/xml id16477484 +Ref: 26f76477484 +Ref: library/xml xml-vulnerabilities6477484 +Ref: 26f56477484 +Ref: XML vulnerabilities-Footnote-16482146 +Ref: XML vulnerabilities-Footnote-26482209 +Ref: XML vulnerabilities-Footnote-36482262 +Ref: XML vulnerabilities-Footnote-46482315 +Ref: XML vulnerabilities-Footnote-56482378 +Ref: XML vulnerabilities-Footnote-66482425 +Node: The defusedxml Package6482470 +Ref: library/xml defusedxml-package6482579 +Ref: 26f66482579 +Ref: library/xml the-defusedxml-package6482579 +Ref: 26f86482579 +Ref: The defusedxml Package-Footnote-16483035 +Node: xml etree ElementTree — The ElementTree XML API6483080 +Ref: library/xml etree elementtree doc6483281 +Ref: 26f96483281 +Ref: library/xml etree elementtree dtd6483281 +Ref: 26fa6483281 +Ref: library/xml etree elementtree module-xml etree ElementTree6483281 +Ref: 1376483281 +Ref: library/xml etree elementtree xml-etree-elementtree-the-elementtree-xml-api6483281 +Ref: 26fb6483281 +Ref: xml etree ElementTree — The ElementTree XML API-Footnote-16484153 +Node: Tutorial6484234 +Ref: library/xml etree elementtree tutorial6484350 +Ref: 26fc6484350 +Node: XML tree and elements6484801 +Ref: library/xml etree elementtree xml-tree-and-elements6484887 +Ref: 26fd6484887 +Node: Parsing XML6485465 +Ref: library/xml etree elementtree elementtree-parsing-xml6485593 +Ref: 26fe6485593 +Ref: library/xml etree elementtree parsing-xml6485593 +Ref: 26ff6485593 +Node: Pull API for non-blocking parsing6488049 +Ref: library/xml etree elementtree elementtree-pull-parsing6488184 +Ref: 9146488184 +Ref: library/xml etree elementtree pull-api-for-non-blocking-parsing6488184 +Ref: 27016488184 +Node: Finding interesting elements6490081 +Ref: library/xml etree elementtree finding-interesting-elements6490226 +Ref: 27046490226 +Node: Modifying an XML File6491467 +Ref: library/xml etree elementtree modifying-an-xml-file6491601 +Ref: 27096491601 +Node: Building XML documents6494545 +Ref: library/xml etree elementtree building-xml-documents6494678 +Ref: 270d6494678 +Node: Parsing XML with Namespaces6495047 +Ref: library/xml etree elementtree parsing-xml-with-namespaces6495179 +Ref: 270f6495179 +Ref: Parsing XML with Namespaces-Footnote-16497494 +Ref: Parsing XML with Namespaces-Footnote-26497546 +Node: Additional resources6497598 +Ref: library/xml etree elementtree additional-resources6497699 +Ref: 27106497699 +Node: XPath support6497851 +Ref: library/xml etree elementtree elementtree-xpath6497988 +Ref: 41a6497988 +Ref: library/xml etree elementtree xpath-support6497988 +Ref: 27116497988 +Ref: XPath support-Footnote-16498352 +Node: Example<10>6498388 +Ref: library/xml etree elementtree example6498480 +Ref: 27126498480 +Node: Supported XPath syntax6499499 +Ref: library/xml etree elementtree supported-xpath-syntax6499591 +Ref: 27136499591 +Node: Reference<2>6503552 +Ref: library/xml etree elementtree reference6503697 +Ref: 27146503697 +Node: Functions<6>6503775 +Ref: library/xml etree elementtree elementtree-functions6503836 +Ref: 27156503836 +Ref: library/xml etree elementtree functions6503836 +Ref: 27166503836 +Ref: library/xml etree elementtree xml etree ElementTree canonicalize6503877 +Ref: 27176503877 +Ref: library/xml etree elementtree xml etree ElementTree Comment6506259 +Ref: 27186506259 +Ref: library/xml etree elementtree xml etree ElementTree dump6506944 +Ref: 27196506944 +Ref: library/xml etree elementtree xml etree ElementTree fromstring6507418 +Ref: 27006507418 +Ref: library/xml etree elementtree xml etree ElementTree fromstringlist6507777 +Ref: b4f6507777 +Ref: library/xml etree elementtree xml etree ElementTree iselement6508195 +Ref: 271b6508195 +Ref: library/xml etree elementtree xml etree ElementTree iterparse6508409 +Ref: 9486508409 +Ref: library/xml etree elementtree xml etree ElementTree parse6510269 +Ref: 271c6510269 +Ref: library/xml etree elementtree xml etree ElementTree ProcessingInstruction6510613 +Ref: 271d6510613 +Ref: library/xml etree elementtree xml etree ElementTree register_namespace6511346 +Ref: b506511346 +Ref: library/xml etree elementtree xml etree ElementTree SubElement6511775 +Ref: 270e6511775 +Ref: library/xml etree elementtree xml etree ElementTree tostring6512345 +Ref: 9156512345 +Ref: library/xml etree elementtree xml etree ElementTree tostringlist6513424 +Ref: 9166513424 +Ref: library/xml etree elementtree xml etree ElementTree XML6514673 +Ref: 271a6514673 +Ref: library/xml etree elementtree xml etree ElementTree XMLID6515067 +Ref: 271e6515067 +Ref: Functions<6>-Footnote-16515542 +Ref: Functions<6>-Footnote-26515583 +Ref: Functions<6>-Footnote-36515896 +Node: XInclude support6516209 +Ref: library/xml etree elementtree elementtree-xinclude6516353 +Ref: 271f6516353 +Ref: library/xml etree elementtree xinclude-support6516353 +Ref: 27206516353 +Ref: XInclude support-Footnote-16516713 +Node: Example<11>6516753 +Ref: library/xml etree elementtree id36516817 +Ref: 27216516817 +Node: Reference<3>6518631 +Ref: library/xml etree elementtree id46518754 +Ref: 27226518754 +Node: Functions<7>6518996 +Ref: library/xml etree elementtree elementinclude-functions6519081 +Ref: 27236519081 +Ref: library/xml etree elementtree id56519081 +Ref: 27246519081 +Ref: library/xml etree elementtree xml etree ElementTree xml etree ElementInclude default_loader6519122 +Ref: 27256519122 +Ref: library/xml etree elementtree xml etree ElementTree xml etree ElementInclude include6519709 +Ref: 27266519709 +Node: Element Objects6520317 +Ref: library/xml etree elementtree element-objects6520430 +Ref: 27276520430 +Ref: library/xml etree elementtree elementtree-element-objects6520430 +Ref: 27286520430 +Ref: library/xml etree elementtree xml etree ElementTree Element6520483 +Ref: ac46520483 +Ref: library/xml etree elementtree xml etree ElementTree Element tag6520965 +Ref: 27296520965 +Ref: library/xml etree elementtree xml etree ElementTree Element text6521110 +Ref: 27076521110 +Ref: library/xml etree elementtree xml etree ElementTree Element tail6521135 +Ref: 272a6521135 +Ref: library/xml etree elementtree xml etree ElementTree Element attrib6522250 +Ref: 272b6522250 +Ref: library/xml etree elementtree xml etree ElementTree Element clear6522794 +Ref: 272c6522794 +Ref: library/xml etree elementtree xml etree ElementTree Element get6522987 +Ref: 27086522987 +Ref: library/xml etree elementtree xml etree ElementTree Element items6523175 +Ref: 272d6523175 +Ref: library/xml etree elementtree xml etree ElementTree Element keys6523344 +Ref: 272e6523344 +Ref: library/xml etree elementtree xml etree ElementTree Element set6523486 +Ref: 270a6523486 +Ref: library/xml etree elementtree xml etree ElementTree Element append6523659 +Ref: 270b6523659 +Ref: library/xml etree elementtree xml etree ElementTree Element extend6523895 +Ref: b516523895 +Ref: library/xml etree elementtree xml etree ElementTree Element find6524145 +Ref: 27066524145 +Ref: library/xml etree elementtree xml etree ElementTree Element findall6524567 +Ref: 27056524567 +Ref: library/xml etree elementtree xml etree ElementTree Element findtext6524986 +Ref: 272f6524986 +Ref: library/xml etree elementtree xml etree ElementTree Element getchildren6525590 +Ref: 27306525590 +Ref: library/xml etree elementtree xml etree ElementTree Element getiterator6525741 +Ref: 27316525741 +Ref: library/xml etree elementtree xml etree ElementTree Element insert6525912 +Ref: 27326525912 +Ref: library/xml etree elementtree xml etree ElementTree Element iter6526123 +Ref: ce76526123 +Ref: library/xml etree elementtree xml etree ElementTree Element iterfind6526614 +Ref: b526526614 +Ref: library/xml etree elementtree xml etree ElementTree Element itertext6526946 +Ref: b536526946 +Ref: library/xml etree elementtree xml etree ElementTree Element makeelement6527169 +Ref: 27336527169 +Ref: library/xml etree elementtree xml etree ElementTree Element remove6527392 +Ref: 270c6527392 +Ref: Element Objects-Footnote-16529952 +Node: ElementTree Objects6529995 +Ref: library/xml etree elementtree elementtree-elementtree-objects6530109 +Ref: 27346530109 +Ref: library/xml etree elementtree elementtree-objects6530109 +Ref: 27356530109 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree6530170 +Ref: 9176530170 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree _setroot6530522 +Ref: 27366530522 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree find6530771 +Ref: 27376530771 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree findall6530910 +Ref: 27386530910 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree findtext6531055 +Ref: 27396531055 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree getiterator6531216 +Ref: 273a6531216 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree getroot6531392 +Ref: 273c6531392 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree iter6531472 +Ref: 273b6531472 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree iterfind6531742 +Ref: 273d6531742 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree parse6531919 +Ref: 273e6531919 +Ref: library/xml etree elementtree xml etree ElementTree ElementTree write6532264 +Ref: 9186532264 +Ref: ElementTree Objects-Footnote-16535033 +Node: QName Objects6535346 +Ref: library/xml etree elementtree elementtree-qname-objects6535464 +Ref: 273f6535464 +Ref: library/xml etree elementtree qname-objects6535464 +Ref: 27406535464 +Ref: library/xml etree elementtree xml etree ElementTree QName6535513 +Ref: 27416535513 +Node: TreeBuilder Objects6536026 +Ref: library/xml etree elementtree elementtree-treebuilder-objects6536142 +Ref: 27426536142 +Ref: library/xml etree elementtree treebuilder-objects6536142 +Ref: 27436536142 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder6536203 +Ref: 2536536203 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder close6537321 +Ref: 27446537321 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder data6537483 +Ref: 27456537483 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder end6537647 +Ref: b546537647 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder start6537780 +Ref: 27466537780 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder comment6537982 +Ref: 27476537982 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder pi6538178 +Ref: 27486538178 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder doctype6538489 +Ref: 2c16538489 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder start_ns6538813 +Ref: 27496538813 +Ref: library/xml etree elementtree xml etree ElementTree TreeBuilder end_ns6539207 +Ref: 274a6539207 +Ref: library/xml etree elementtree xml etree ElementTree C14NWriterTarget6539454 +Ref: 274b6539454 +Ref: TreeBuilder Objects-Footnote-16540002 +Node: XMLParser Objects6540043 +Ref: library/xml etree elementtree elementtree-xmlparser-objects6540167 +Ref: 274c6540167 +Ref: library/xml etree elementtree xmlparser-objects6540167 +Ref: 274d6540167 +Ref: library/xml etree elementtree xml etree ElementTree XMLParser6540224 +Ref: 2526540224 +Ref: library/xml etree elementtree xml etree ElementTree XMLParser close6540936 +Ref: 274f6540936 +Ref: library/xml etree elementtree xml etree ElementTree XMLParser feed6541192 +Ref: 274e6541192 +Ref: XMLParser Objects-Footnote-16543069 +Node: XMLPullParser Objects6543382 +Ref: library/xml etree elementtree elementtree-xmlpullparser-objects6543501 +Ref: 27506543501 +Ref: library/xml etree elementtree xmlpullparser-objects6543501 +Ref: 27516543501 +Ref: library/xml etree elementtree xml etree ElementTree XMLPullParser6543566 +Ref: 9136543566 +Ref: library/xml etree elementtree xml etree ElementTree XMLPullParser feed6544258 +Ref: 27026544258 +Ref: library/xml etree elementtree xml etree ElementTree XMLPullParser close6544340 +Ref: 27526544340 +Ref: library/xml etree elementtree xml etree ElementTree XMLPullParser read_events6544651 +Ref: 27036544651 +Node: Exceptions<15>6546385 +Ref: library/xml etree elementtree exceptions6546478 +Ref: 27536546478 +Ref: library/xml etree elementtree xml etree ElementTree ParseError6546521 +Ref: c0c6546521 +Ref: library/xml etree elementtree xml etree ElementTree ParseError code6546849 +Ref: 27546546849 +Ref: library/xml etree elementtree xml etree ElementTree ParseError position6547054 +Ref: 27556547054 +Node: xml dom — The Document Object Model API6547179 +Ref: library/xml dom doc6547404 +Ref: 27566547404 +Ref: library/xml dom module-xml dom6547404 +Ref: 1346547404 +Ref: library/xml dom xml-dom-the-document-object-model-api6547404 +Ref: 27576547404 +Ref: xml dom — The Document Object Model API-Footnote-16550878 +Ref: xml dom — The Document Object Model API-Footnote-26550954 +Ref: xml dom — The Document Object Model API-Footnote-36551020 +Ref: xml dom — The Document Object Model API-Footnote-46551067 +Node: Module Contents<4>6551126 +Ref: library/xml dom module-contents6551249 +Ref: 275a6551249 +Ref: library/xml dom xml dom registerDOMImplementation6551359 +Ref: 275b6551359 +Ref: library/xml dom xml dom getDOMImplementation6551807 +Ref: 27586551807 +Ref: library/xml dom xml dom EMPTY_NAMESPACE6552716 +Ref: 275c6552716 +Ref: library/xml dom xml dom XML_NAMESPACE6552989 +Ref: 275d6552989 +Ref: library/xml dom xml dom XMLNS_NAMESPACE6553146 +Ref: 275e6553146 +Ref: library/xml dom xml dom XHTML_NAMESPACE6553332 +Ref: 275f6553332 +Ref: Module Contents<4>-Footnote-16554067 +Ref: Module Contents<4>-Footnote-26554112 +Ref: Module Contents<4>-Footnote-36554169 +Node: Objects in the DOM6554207 +Ref: library/xml dom dom-objects6554350 +Ref: 27606554350 +Ref: library/xml dom objects-in-the-dom6554350 +Ref: 27616554350 +Node: DOMImplementation Objects6558885 +Ref: library/xml dom dom-implementation-objects6558986 +Ref: 27626558986 +Ref: library/xml dom domimplementation-objects6558986 +Ref: 276c6558986 +Ref: library/xml dom xml dom DOMImplementation hasFeature6559342 +Ref: 276d6559342 +Ref: library/xml dom xml dom DOMImplementation createDocument6559521 +Ref: 276e6559521 +Ref: library/xml dom xml dom DOMImplementation createDocumentType6560041 +Ref: 276f6560041 +Node: Node Objects6560349 +Ref: library/xml dom dom-node-objects6560475 +Ref: 27636560475 +Ref: library/xml dom node-objects6560475 +Ref: 27706560475 +Ref: library/xml dom xml dom Node nodeType6560592 +Ref: 27716560592 +Ref: library/xml dom xml dom Node parentNode6561015 +Ref: 27726561015 +Ref: library/xml dom xml dom Node attributes6561424 +Ref: 27736561424 +Ref: library/xml dom xml dom Node previousSibling6561637 +Ref: 27746561637 +Ref: library/xml dom xml dom Node nextSibling6562123 +Ref: 27756562123 +Ref: library/xml dom xml dom Node childNodes6562389 +Ref: 27766562389 +Ref: library/xml dom xml dom Node firstChild6562508 +Ref: 27776562508 +Ref: library/xml dom xml dom Node lastChild6562645 +Ref: 27786562645 +Ref: library/xml dom xml dom Node localName6562780 +Ref: 27796562780 +Ref: library/xml dom xml dom Node prefix6562944 +Ref: 277a6562944 +Ref: library/xml dom xml dom Node namespaceURI6563112 +Ref: 277b6563112 +Ref: library/xml dom xml dom Node nodeName6563275 +Ref: 277c6563275 +Ref: library/xml dom xml dom Node nodeValue6563712 +Ref: 277d6563712 +Ref: library/xml dom xml dom Node hasAttributes6563946 +Ref: 277e6563946 +Ref: library/xml dom xml dom Node hasChildNodes6564037 +Ref: 277f6564037 +Ref: library/xml dom xml dom Node isSameNode6564129 +Ref: 27806564129 +Ref: library/xml dom xml dom Node appendChild6564758 +Ref: 27816564758 +Ref: library/xml dom xml dom Node insertBefore6564966 +Ref: 27826564966 +Ref: library/xml dom xml dom Node removeChild6565307 +Ref: 27836565307 +Ref: library/xml dom xml dom Node replaceChild6565595 +Ref: 27846565595 +Ref: library/xml dom xml dom Node normalize6565809 +Ref: 27856565809 +Ref: library/xml dom xml dom Node cloneNode6566026 +Ref: 27866566026 +Node: NodeList Objects6566170 +Ref: library/xml dom dom-nodelist-objects6566291 +Ref: 27646566291 +Ref: library/xml dom nodelist-objects6566291 +Ref: 27876566291 +Ref: library/xml dom xml dom NodeList item6566772 +Ref: 27886566772 +Ref: library/xml dom xml dom NodeList length6567002 +Ref: 27896567002 +Node: DocumentType Objects6567669 +Ref: library/xml dom documenttype-objects6567794 +Ref: 278a6567794 +Ref: library/xml dom dom-documenttype-objects6567794 +Ref: 27656567794 +Ref: library/xml dom xml dom DocumentType publicId6568427 +Ref: 278b6568427 +Ref: library/xml dom xml dom DocumentType systemId6568593 +Ref: 278c6568593 +Ref: library/xml dom xml dom DocumentType internalSubset6568769 +Ref: 278d6568769 +Ref: library/xml dom xml dom DocumentType name6569022 +Ref: 278e6569022 +Ref: library/xml dom xml dom DocumentType entities6569151 +Ref: 278f6569151 +Ref: library/xml dom xml dom DocumentType notations6569534 +Ref: 27906569534 +Node: Document Objects6569913 +Ref: library/xml dom document-objects6570040 +Ref: 27916570040 +Ref: library/xml dom dom-document-objects6570040 +Ref: 27666570040 +Ref: library/xml dom xml dom Document documentElement6570287 +Ref: 27926570287 +Ref: library/xml dom xml dom Document createElement6570381 +Ref: 27936570381 +Ref: library/xml dom xml dom Document createElementNS6570670 +Ref: 27946570670 +Ref: library/xml dom xml dom Document createTextNode6571021 +Ref: 27956571021 +Ref: library/xml dom xml dom Document createComment6571240 +Ref: 27966571240 +Ref: library/xml dom xml dom Document createProcessingInstruction6571461 +Ref: 27976571461 +Ref: library/xml dom xml dom Document createAttribute6571733 +Ref: 27986571733 +Ref: library/xml dom xml dom Document createAttributeNS6572040 +Ref: 27996572040 +Ref: library/xml dom xml dom Document getElementsByTagName6572428 +Ref: 279a6572428 +Ref: library/xml dom xml dom Document getElementsByTagNameNS6572604 +Ref: 279b6572604 +Node: Element Objects<2>6572875 +Ref: library/xml dom dom-element-objects6572994 +Ref: 27676572994 +Ref: library/xml dom element-objects6572994 +Ref: 279c6572994 +Ref: library/xml dom xml dom Element tagName6573135 +Ref: 279d6573135 +Ref: library/xml dom xml dom Element getElementsByTagName6573282 +Ref: 279e6573282 +Ref: library/xml dom xml dom Element getElementsByTagNameNS6573395 +Ref: 279f6573395 +Ref: library/xml dom xml dom Element hasAttribute6573526 +Ref: 27a06573526 +Ref: library/xml dom xml dom Element hasAttributeNS6573640 +Ref: 27a16573640 +Ref: library/xml dom xml dom Element getAttribute6573804 +Ref: 27a26573804 +Ref: library/xml dom xml dom Element getAttributeNode6574017 +Ref: 27a36574017 +Ref: library/xml dom xml dom Element getAttributeNS6574138 +Ref: 27a46574138 +Ref: library/xml dom xml dom Element getAttributeNodeNS6574396 +Ref: 27a56574396 +Ref: library/xml dom xml dom Element removeAttribute6574550 +Ref: 27a66574550 +Ref: library/xml dom xml dom Element removeAttributeNode6574708 +Ref: 27a86574708 +Ref: library/xml dom xml dom Element removeAttributeNS6574900 +Ref: 27a96574900 +Ref: library/xml dom xml dom Element setAttribute6575109 +Ref: 27aa6575109 +Ref: library/xml dom xml dom Element setAttributeNode6575201 +Ref: 27ab6575201 +Ref: library/xml dom xml dom Element setAttributeNodeNS6575545 +Ref: 27ad6575545 +Ref: library/xml dom xml dom Element setAttributeNS6575918 +Ref: 27ae6575918 +Node: Attr Objects6576154 +Ref: library/xml dom attr-objects6576277 +Ref: 27af6576277 +Ref: library/xml dom dom-attr-objects6576277 +Ref: 27686576277 +Ref: library/xml dom xml dom Attr name6576392 +Ref: 27b06576392 +Ref: library/xml dom xml dom Attr localName6576504 +Ref: 27b16576504 +Ref: library/xml dom xml dom Attr prefix6576658 +Ref: 27b26576658 +Ref: library/xml dom xml dom Attr value6576778 +Ref: 27b36576778 +Node: NamedNodeMap Objects6576903 +Ref: library/xml dom dom-attributelist-objects6577023 +Ref: 27b46577023 +Ref: library/xml dom namednodemap-objects6577023 +Ref: 27b56577023 +Ref: library/xml dom xml dom NamedNodeMap length6577142 +Ref: 27b66577142 +Ref: library/xml dom xml dom NamedNodeMap item6577218 +Ref: 27b76577218 +Node: Comment Objects6577699 +Ref: library/xml dom comment-objects6577836 +Ref: 27b86577836 +Ref: library/xml dom dom-comment-objects6577836 +Ref: 27696577836 +Ref: library/xml dom xml dom Comment data6578008 +Ref: 27b96578008 +Node: Text and CDATASection Objects6578226 +Ref: library/xml dom dom-text-objects6578372 +Ref: 276a6578372 +Ref: library/xml dom text-and-cdatasection-objects6578372 +Ref: 27ba6578372 +Ref: library/xml dom xml dom Text data6578868 +Ref: 27bb6578868 +Node: ProcessingInstruction Objects6579361 +Ref: library/xml dom dom-pi-objects6579506 +Ref: 276b6579506 +Ref: library/xml dom processinginstruction-objects6579506 +Ref: 27bc6579506 +Ref: library/xml dom xml dom ProcessingInstruction target6579718 +Ref: 27bd6579718 +Ref: library/xml dom xml dom ProcessingInstruction data6579886 +Ref: 27be6579886 +Node: Exceptions<16>6580024 +Ref: library/xml dom dom-exceptions6580131 +Ref: 27bf6580131 +Ref: library/xml dom exceptions6580131 +Ref: 27c06580131 +Ref: library/xml dom xml dom DOMException6580810 +Ref: 27c16580810 +Ref: library/xml dom xml dom DomstringSizeErr6580972 +Ref: 27c26580972 +Ref: library/xml dom xml dom HierarchyRequestErr6581227 +Ref: 27c36581227 +Ref: library/xml dom xml dom IndexSizeErr6581366 +Ref: 27c46581366 +Ref: library/xml dom xml dom InuseAttributeErr6581508 +Ref: 27ac6581508 +Ref: library/xml dom xml dom InvalidAccessErr6581672 +Ref: 27c56581672 +Ref: library/xml dom xml dom InvalidCharacterErr6581805 +Ref: 27c66581805 +Ref: library/xml dom xml dom InvalidModificationErr6582165 +Ref: 27c76582165 +Ref: library/xml dom xml dom InvalidStateErr6582279 +Ref: 27c86582279 +Ref: library/xml dom xml dom NamespaceErr6582422 +Ref: 27c96582422 +Ref: library/xml dom xml dom NotFoundErr6582631 +Ref: 27a76582631 +Ref: library/xml dom xml dom NotSupportedErr6582862 +Ref: 27ca6582862 +Ref: library/xml dom xml dom NoDataAllowedErr6583004 +Ref: 27cb6583004 +Ref: library/xml dom xml dom NoModificationAllowedErr6583132 +Ref: 27cc6583132 +Ref: library/xml dom xml dom SyntaxErr6583298 +Ref: 27cd6583298 +Ref: library/xml dom xml dom WrongDocumentErr6583393 +Ref: 27ce6583393 +Ref: Exceptions<16>-Footnote-16586434 +Node: Conformance6586479 +Ref: library/xml dom conformance6586595 +Ref: 27cf6586595 +Ref: library/xml dom dom-conformance6586595 +Ref: 27596586595 +Node: Type Mapping6586852 +Ref: library/xml dom dom-type-mapping6586937 +Ref: 27d06586937 +Ref: library/xml dom type-mapping6586937 +Ref: 27d16586937 +Node: Accessor Methods6587762 +Ref: library/xml dom accessor-methods6587847 +Ref: 27d26587847 +Ref: library/xml dom dom-accessor-methods6587847 +Ref: 27d36587847 +Node: xml dom minidom — Minimal DOM implementation6589723 +Ref: library/xml dom minidom doc6589957 +Ref: 27d46589957 +Ref: library/xml dom minidom module-xml dom minidom6589957 +Ref: 1356589957 +Ref: library/xml dom minidom xml-dom-minidom-minimal-dom-implementation6589957 +Ref: 27d56589957 +Ref: library/xml dom minidom xml dom minidom parse6591300 +Ref: 27d66591300 +Ref: library/xml dom minidom xml dom minidom parseString6591869 +Ref: 27d76591869 +Ref: xml dom minidom — Minimal DOM implementation-Footnote-16594447 +Ref: xml dom minidom — Minimal DOM implementation-Footnote-26594521 +Node: DOM Objects6594568 +Ref: library/xml dom minidom dom-objects6594682 +Ref: 27d86594682 +Ref: library/xml dom minidom minidom-objects6594682 +Ref: 27d96594682 +Ref: library/xml dom minidom xml dom minidom Node unlink6594917 +Ref: 27da6594917 +Ref: library/xml dom minidom xml dom minidom Node writexml6595693 +Ref: 27db6595693 +Ref: library/xml dom minidom xml dom minidom Node toxml6596473 +Ref: 27dc6596473 +Ref: library/xml dom minidom xml dom minidom Node toprettyxml6597114 +Ref: 27dd6597114 +Ref: DOM Objects-Footnote-16597677 +Node: DOM Example6598084 +Ref: library/xml dom minidom dom-example6598235 +Ref: 27de6598235 +Ref: library/xml dom minidom id26598235 +Ref: 27df6598235 +Node: minidom and the DOM standard6600275 +Ref: library/xml dom minidom minidom-and-dom6600406 +Ref: 27e06600406 +Ref: library/xml dom minidom minidom-and-the-dom-standard6600406 +Ref: 27e16600406 +Node: xml dom pulldom — Support for building partial DOM trees6603047 +Ref: library/xml dom pulldom doc6603276 +Ref: 27e26603276 +Ref: library/xml dom pulldom module-xml dom pulldom6603276 +Ref: 1366603276 +Ref: library/xml dom pulldom xml-dom-pulldom-support-for-building-partial-dom-trees6603276 +Ref: 27e36603276 +Ref: library/xml dom pulldom xml dom pulldom PullDom6606031 +Ref: 27e56606031 +Ref: library/xml dom pulldom xml dom pulldom SAX2DOM6606152 +Ref: 27e76606152 +Ref: library/xml dom pulldom xml dom pulldom parse6606273 +Ref: 27e86606273 +Ref: library/xml dom pulldom xml dom pulldom parseString6606865 +Ref: 27ea6606865 +Ref: library/xml dom pulldom xml dom pulldom default_bufsize6607017 +Ref: 27eb6607017 +Ref: xml dom pulldom — Support for building partial DOM trees-Footnote-16607330 +Node: DOMEventStream Objects6607404 +Ref: library/xml dom pulldom domeventstream-objects6607521 +Ref: 27ec6607521 +Ref: library/xml dom pulldom id16607521 +Ref: 27ed6607521 +Ref: library/xml dom pulldom xml dom pulldom DOMEventStream6607586 +Ref: 27d6607586 +Ref: library/xml dom pulldom xml dom pulldom DOMEventStream getEvent6607753 +Ref: 27ee6607753 +Ref: library/xml dom pulldom xml dom pulldom DOMEventStream expandNode6608245 +Ref: 27e46608245 +Ref: library/xml dom pulldom xml dom pulldom DOMEventStream reset6608959 +Ref: 27ef6608959 +Node: xml sax — Support for SAX2 parsers6608986 +Ref: library/xml sax doc6609218 +Ref: 27f06609218 +Ref: library/xml sax module-xml sax6609218 +Ref: 13b6609218 +Ref: library/xml sax xml-sax-support-for-sax2-parsers6609218 +Ref: 27f16609218 +Ref: library/xml sax xml sax make_parser6610309 +Ref: 27f46610309 +Ref: library/xml sax xml sax parse6610807 +Ref: 27f56610807 +Ref: library/xml sax xml sax parseString6611402 +Ref: 7936611402 +Ref: library/xml sax xml sax SAXException6613327 +Ref: 27f96613327 +Ref: library/xml sax xml sax SAXParseException6614199 +Ref: 27f76614199 +Ref: library/xml sax xml sax SAXNotRecognizedException6614610 +Ref: 27fa6614610 +Ref: library/xml sax xml sax SAXNotSupportedException6614912 +Ref: 27fb6614912 +Ref: xml sax — Support for SAX2 parsers-Footnote-16615926 +Ref: xml sax — Support for SAX2 parsers-Footnote-26616002 +Node: SAXException Objects6616037 +Ref: library/xml sax sax-exception-objects6616130 +Ref: 27fc6616130 +Ref: library/xml sax saxexception-objects6616130 +Ref: 27fd6616130 +Ref: library/xml sax xml sax SAXException getMessage6616270 +Ref: 27fe6616270 +Ref: library/xml sax xml sax SAXException getException6616380 +Ref: 27ff6616380 +Node: xml sax handler — Base classes for SAX handlers6616484 +Ref: library/xml sax handler doc6616692 +Ref: 28006616692 +Ref: library/xml sax handler module-xml sax handler6616692 +Ref: 13c6616692 +Ref: library/xml sax handler xml-sax-handler-base-classes-for-sax-handlers6616692 +Ref: 28016616692 +Ref: library/xml sax handler xml sax handler ContentHandler6617379 +Ref: 27e66617379 +Ref: library/xml sax handler xml sax handler DTDHandler6617619 +Ref: 28026617619 +Ref: library/xml sax handler xml sax handler EntityResolver6617804 +Ref: 28036617804 +Ref: library/xml sax handler xml sax handler ErrorHandler6618086 +Ref: 27f66618086 +Ref: library/xml sax handler xml sax handler feature_namespaces6618483 +Ref: 28046618483 +Ref: library/xml sax handler xml sax handler feature_namespace_prefixes6618811 +Ref: 28056618811 +Ref: library/xml sax handler xml sax handler feature_string_interning6619244 +Ref: 28066619244 +Ref: library/xml sax handler xml sax handler feature_validation6619663 +Ref: 28076619663 +Ref: library/xml sax handler xml sax handler feature_external_ges6620006 +Ref: 27f36620006 +Ref: library/xml sax handler xml sax handler feature_external_pes6620323 +Ref: 28086620323 +Ref: library/xml sax handler xml sax handler all_features6620708 +Ref: 28096620708 +Ref: library/xml sax handler xml sax handler property_lexical_handler6620776 +Ref: 280a6620776 +Ref: library/xml sax handler xml sax handler property_declaration_handler6621100 +Ref: 280b6621100 +Ref: library/xml sax handler xml sax handler property_dom_node6621462 +Ref: 280c6621462 +Ref: library/xml sax handler xml sax handler property_xml_string6621856 +Ref: 280d6621856 +Ref: library/xml sax handler xml sax handler all_properties6622129 +Ref: 280e6622129 +Ref: xml sax handler — Base classes for SAX handlers-Footnote-16622359 +Node: ContentHandler Objects6622433 +Ref: library/xml sax handler content-handler-objects6622568 +Ref: 280f6622568 +Ref: library/xml sax handler contenthandler-objects6622568 +Ref: 28106622568 +Ref: library/xml sax handler xml sax handler ContentHandler setDocumentLocator6622822 +Ref: 28116622822 +Ref: library/xml sax handler xml sax handler ContentHandler startDocument6623897 +Ref: 28126623897 +Ref: library/xml sax handler xml sax handler ContentHandler endDocument6624175 +Ref: 28136624175 +Ref: library/xml sax handler xml sax handler ContentHandler startPrefixMapping6624543 +Ref: 28146624543 +Ref: library/xml sax handler xml sax handler ContentHandler endPrefixMapping6625745 +Ref: 28156625745 +Ref: library/xml sax handler xml sax handler ContentHandler startElement6626088 +Ref: 28166626088 +Ref: library/xml sax handler xml sax handler ContentHandler endElement6626729 +Ref: 28176626729 +Ref: library/xml sax handler xml sax handler ContentHandler startElementNS6626957 +Ref: 28196626957 +Ref: library/xml sax handler xml sax handler ContentHandler endElementNS6627945 +Ref: 281b6627945 +Ref: library/xml sax handler xml sax handler ContentHandler characters6628218 +Ref: 281c6628218 +Ref: library/xml sax handler xml sax handler ContentHandler ignorableWhitespace6629245 +Ref: 281d6629245 +Ref: library/xml sax handler xml sax handler ContentHandler processingInstruction6629912 +Ref: 281e6629912 +Ref: library/xml sax handler xml sax handler ContentHandler skippedEntity6630372 +Ref: 281f6630372 +Node: DTDHandler Objects6630872 +Ref: library/xml sax handler dtd-handler-objects6631038 +Ref: 28206631038 +Ref: library/xml sax handler dtdhandler-objects6631038 +Ref: 28216631038 +Ref: library/xml sax handler xml sax handler DTDHandler notationDecl6631163 +Ref: 28226631163 +Ref: library/xml sax handler xml sax handler DTDHandler unparsedEntityDecl6631270 +Ref: 28236631270 +Node: EntityResolver Objects6631408 +Ref: library/xml sax handler entity-resolver-objects6631572 +Ref: 28246631572 +Ref: library/xml sax handler entityresolver-objects6631572 +Ref: 28256631572 +Ref: library/xml sax handler xml sax handler EntityResolver resolveEntity6631639 +Ref: 28266631639 +Node: ErrorHandler Objects6631907 +Ref: library/xml sax handler errorhandler-objects6632044 +Ref: 28276632044 +Ref: library/xml sax handler sax-error-handler6632044 +Ref: 28286632044 +Ref: library/xml sax handler xml sax handler ErrorHandler error6632691 +Ref: 28296632691 +Ref: library/xml sax handler xml sax handler ErrorHandler fatalError6633060 +Ref: 282a6633060 +Ref: library/xml sax handler xml sax handler ErrorHandler warning6633246 +Ref: 282b6633246 +Node: xml sax saxutils — SAX Utilities6633591 +Ref: library/xml sax utils doc6633810 +Ref: 282c6633810 +Ref: library/xml sax utils module-xml sax saxutils6633810 +Ref: 13d6633810 +Ref: library/xml sax utils xml-sax-saxutils-sax-utilities6633810 +Ref: 282d6633810 +Ref: library/xml sax utils xml sax saxutils escape6634197 +Ref: 282e6634197 +Ref: library/xml sax utils xml sax saxutils unescape6634651 +Ref: 282f6634651 +Ref: library/xml sax utils xml sax saxutils quoteattr6635118 +Ref: 28306635118 +Ref: library/xml sax utils xml sax saxutils XMLGenerator6636027 +Ref: 28316636027 +Ref: library/xml sax utils xml sax saxutils XMLFilterBase6636880 +Ref: 28326636880 +Ref: library/xml sax utils xml sax saxutils prepare_input_source6637302 +Ref: 28336637302 +Ref: xml sax saxutils — SAX Utilities-Footnote-16637791 +Node: xml sax xmlreader — Interface for XML parsers6637867 +Ref: library/xml sax reader doc6638087 +Ref: 28346638087 +Ref: library/xml sax reader module-xml sax xmlreader6638087 +Ref: 13e6638087 +Ref: library/xml sax reader xml-sax-xmlreader-interface-for-xml-parsers6638087 +Ref: 28356638087 +Ref: library/xml sax reader xml sax xmlreader XMLReader6638587 +Ref: 27e96638587 +Ref: library/xml sax reader xml sax xmlreader IncrementalParser6638683 +Ref: 28366638683 +Ref: library/xml sax reader xml sax xmlreader Locator6639810 +Ref: 27f86639810 +Ref: library/xml sax reader xml sax xmlreader InputSource6640143 +Ref: 7926640143 +Ref: library/xml sax reader xml sax xmlreader AttributesImpl6640911 +Ref: 28386640911 +Ref: library/xml sax reader xml sax xmlreader AttributesNSImpl6641510 +Ref: 28396641510 +Ref: xml sax xmlreader — Interface for XML parsers-Footnote-16642267 +Node: XMLReader Objects6642344 +Ref: library/xml sax reader id16642479 +Ref: 283a6642479 +Ref: library/xml sax reader xmlreader-objects6642479 +Ref: 283b6642479 +Ref: library/xml sax reader xml sax xmlreader XMLReader parse6642606 +Ref: 28376642606 +Ref: library/xml sax reader xml sax xmlreader XMLReader getContentHandler6643195 +Ref: 283c6643195 +Ref: library/xml sax reader xml sax xmlreader XMLReader setContentHandler6643292 +Ref: 283d6643292 +Ref: library/xml sax reader xml sax xmlreader XMLReader getDTDHandler6643475 +Ref: 283e6643475 +Ref: library/xml sax reader xml sax xmlreader XMLReader setDTDHandler6643564 +Ref: 283f6643564 +Ref: library/xml sax reader xml sax xmlreader XMLReader getEntityResolver6643731 +Ref: 28406643731 +Ref: library/xml sax reader xml sax xmlreader XMLReader setEntityResolver6643828 +Ref: 28416643828 +Ref: library/xml sax reader xml sax xmlreader XMLReader getErrorHandler6644120 +Ref: 28426644120 +Ref: library/xml sax reader xml sax xmlreader XMLReader setErrorHandler6644213 +Ref: 28436644213 +Ref: library/xml sax reader xml sax xmlreader XMLReader setLocale6644417 +Ref: 28446644417 +Ref: library/xml sax reader xml sax xmlreader XMLReader getFeature6644783 +Ref: 28456644783 +Ref: library/xml sax reader xml sax xmlreader XMLReader setFeature6645065 +Ref: 27f26645065 +Ref: library/xml sax reader xml sax xmlreader XMLReader getProperty6645354 +Ref: 28466645354 +Ref: library/xml sax reader xml sax xmlreader XMLReader setProperty6645645 +Ref: 28476645645 +Node: IncrementalParser Objects6645939 +Ref: library/xml sax reader incremental-parser-objects6646098 +Ref: 28486646098 +Ref: library/xml sax reader incrementalparser-objects6646098 +Ref: 28496646098 +Ref: library/xml sax reader xml sax xmlreader IncrementalParser feed6646256 +Ref: 284a6646256 +Ref: library/xml sax reader xml sax xmlreader IncrementalParser close6646332 +Ref: 284b6646332 +Ref: library/xml sax reader xml sax xmlreader IncrementalParser reset6646571 +Ref: 284c6646571 +Node: Locator Objects6646833 +Ref: library/xml sax reader id26646994 +Ref: 284d6646994 +Ref: library/xml sax reader locator-objects6646994 +Ref: 284e6646994 +Ref: library/xml sax reader xml sax xmlreader Locator getColumnNumber6647105 +Ref: 284f6647105 +Ref: library/xml sax reader xml sax xmlreader Locator getLineNumber6647208 +Ref: 28506647208 +Ref: library/xml sax reader xml sax xmlreader Locator getPublicId6647307 +Ref: 28516647307 +Ref: library/xml sax reader xml sax xmlreader Locator getSystemId6647401 +Ref: 28526647401 +Node: InputSource Objects6647495 +Ref: library/xml sax reader input-source-objects6647655 +Ref: 28536647655 +Ref: library/xml sax reader inputsource-objects6647655 +Ref: 28546647655 +Ref: library/xml sax reader xml sax xmlreader InputSource setPublicId6647716 +Ref: 28556647716 +Ref: library/xml sax reader xml sax xmlreader InputSource getPublicId6647823 +Ref: 28566647823 +Ref: library/xml sax reader xml sax xmlreader InputSource setSystemId6647931 +Ref: 28576647931 +Ref: library/xml sax reader xml sax xmlreader InputSource getSystemId6648038 +Ref: 28586648038 +Ref: library/xml sax reader xml sax xmlreader InputSource setEncoding6648146 +Ref: 28596648146 +Ref: library/xml sax reader xml sax xmlreader InputSource getEncoding6648532 +Ref: 285a6648532 +Ref: library/xml sax reader xml sax xmlreader InputSource setByteStream6648626 +Ref: 285b6648626 +Ref: library/xml sax reader xml sax xmlreader InputSource getByteStream6649059 +Ref: 285c6649059 +Ref: library/xml sax reader xml sax xmlreader InputSource setCharacterStream6649267 +Ref: 285d6649267 +Ref: library/xml sax reader xml sax xmlreader InputSource getCharacterStream6649580 +Ref: 285e6649580 +Node: The Attributes Interface6649681 +Ref: library/xml sax reader attributes-objects6649852 +Ref: 28186649852 +Ref: library/xml sax reader the-attributes-interface6649852 +Ref: 285f6649852 +Ref: library/xml sax reader xml sax xmlreader Attributes getLength6650180 +Ref: 28606650180 +Ref: library/xml sax reader xml sax xmlreader Attributes getNames6650256 +Ref: 28616650256 +Ref: library/xml sax reader xml sax xmlreader Attributes getType6650334 +Ref: 28626650334 +Ref: library/xml sax reader xml sax xmlreader Attributes getValue6650459 +Ref: 28636650459 +Node: The AttributesNS Interface6650543 +Ref: library/xml sax reader attributes-ns-objects6650686 +Ref: 281a6650686 +Ref: library/xml sax reader the-attributesns-interface6650686 +Ref: 28646650686 +Ref: library/xml sax reader xml sax xmlreader AttributesNS getValueByQName6651019 +Ref: 28656651019 +Ref: library/xml sax reader xml sax xmlreader AttributesNS getNameByQName6651113 +Ref: 28666651113 +Ref: library/xml sax reader xml sax xmlreader AttributesNS getQNameByName6651236 +Ref: 28676651236 +Ref: library/xml sax reader xml sax xmlreader AttributesNS getQNames6651357 +Ref: 28686651357 +Node: xml parsers expat — Fast XML parsing using Expat6651448 +Ref: library/pyexpat doc6651625 +Ref: 28696651625 +Ref: library/pyexpat module-xml parsers expat6651625 +Ref: 1386651625 +Ref: library/pyexpat xml-parsers-expat-fast-xml-parsing-using-expat6651625 +Ref: 286a6651625 +Ref: library/pyexpat xml parsers expat ExpatError6652679 +Ref: c0d6652679 +Ref: library/pyexpat xml parsers expat error6652889 +Ref: 286c6652889 +Ref: library/pyexpat xml parsers expat XMLParserType6652968 +Ref: 286d6652968 +Ref: library/pyexpat xml parsers expat ErrorString6653165 +Ref: 286f6653165 +Ref: library/pyexpat xml parsers expat ParserCreate6653288 +Ref: 286e6653288 +Ref: xml parsers expat — Fast XML parsing using Expat-Footnote-16655724 +Ref: xml parsers expat — Fast XML parsing using Expat-Footnote-26656037 +Node: XMLParser Objects<2>6656070 +Ref: library/pyexpat id26656207 +Ref: 28706656207 +Ref: library/pyexpat xmlparser-objects6656207 +Ref: 28716656207 +Ref: library/pyexpat xml parsers expat xmlparser Parse6656317 +Ref: 28726656317 +Ref: library/pyexpat xml parsers expat xmlparser ParseFile6656695 +Ref: 28736656695 +Ref: library/pyexpat xml parsers expat xmlparser SetBase6656915 +Ref: 28746656915 +Ref: library/pyexpat xml parsers expat xmlparser GetBase6657331 +Ref: 28786657331 +Ref: library/pyexpat xml parsers expat xmlparser GetInputContext6657529 +Ref: 28796657529 +Ref: library/pyexpat xml parsers expat xmlparser ExternalEntityParserCreate6657812 +Ref: 287a6657812 +Ref: library/pyexpat xml parsers expat xmlparser SetParamEntityParsing6658329 +Ref: 287d6658329 +Ref: library/pyexpat xml parsers expat xmlparser UseForeignDTD6658694 +Ref: 287e6658694 +Ref: library/pyexpat xml parsers expat xmlparser buffer_size6659736 +Ref: 28816659736 +Ref: library/pyexpat xml parsers expat xmlparser buffer_text6659998 +Ref: 28826659998 +Ref: library/pyexpat xml parsers expat xmlparser buffer_used6660450 +Ref: 28846660450 +Ref: library/pyexpat xml parsers expat xmlparser ordered_attributes6660723 +Ref: 287b6660723 +Ref: library/pyexpat xml parsers expat xmlparser specified_attributes6661215 +Ref: 287c6661215 +Ref: library/pyexpat xml parsers expat xmlparser ErrorByteIndex6662021 +Ref: 28856662021 +Ref: library/pyexpat xml parsers expat xmlparser ErrorCode6662107 +Ref: 28866662107 +Ref: library/pyexpat xml parsers expat xmlparser ErrorColumnNumber6662338 +Ref: 28876662338 +Ref: library/pyexpat xml parsers expat xmlparser ErrorLineNumber6662430 +Ref: 28886662430 +Ref: library/pyexpat xml parsers expat xmlparser CurrentByteIndex6662923 +Ref: 28896662923 +Ref: library/pyexpat xml parsers expat xmlparser CurrentColumnNumber6663012 +Ref: 288a6663012 +Ref: library/pyexpat xml parsers expat xmlparser CurrentLineNumber6663107 +Ref: 288b6663107 +Ref: library/pyexpat xml parsers expat xmlparser XmlDeclHandler6663520 +Ref: 288c6663520 +Ref: library/pyexpat xml parsers expat xmlparser StartDoctypeDeclHandler6664133 +Ref: 287f6664133 +Ref: library/pyexpat xml parsers expat xmlparser EndDoctypeDeclHandler6664686 +Ref: 28806664686 +Ref: library/pyexpat xml parsers expat xmlparser ElementDeclHandler6664852 +Ref: 288d6664852 +Ref: library/pyexpat xml parsers expat xmlparser AttlistDeclHandler6665066 +Ref: 288e6665066 +Ref: library/pyexpat xml parsers expat xmlparser StartElementHandler6665969 +Ref: 288f6665969 +Ref: library/pyexpat xml parsers expat xmlparser EndElementHandler6666366 +Ref: 28906666366 +Ref: library/pyexpat xml parsers expat xmlparser ProcessingInstructionHandler6666457 +Ref: 28916666457 +Ref: library/pyexpat xml parsers expat xmlparser CharacterDataHandler6666571 +Ref: 28836666571 +Ref: library/pyexpat xml parsers expat xmlparser UnparsedEntityDeclHandler6666993 +Ref: 28776666993 +Ref: library/pyexpat xml parsers expat xmlparser EntityDeclHandler6667386 +Ref: 28946667386 +Ref: library/pyexpat xml parsers expat xmlparser NotationDeclHandler6668131 +Ref: 28766668131 +Ref: library/pyexpat xml parsers expat xmlparser StartNamespaceDeclHandler6668422 +Ref: 28956668422 +Ref: library/pyexpat xml parsers expat xmlparser EndNamespaceDeclHandler6668700 +Ref: 28966668700 +Ref: library/pyexpat xml parsers expat xmlparser CommentHandler6669223 +Ref: 28976669223 +Ref: library/pyexpat xml parsers expat xmlparser StartCdataSectionHandler6669409 +Ref: 28926669409 +Ref: library/pyexpat xml parsers expat xmlparser EndCdataSectionHandler6669646 +Ref: 28936669646 +Ref: library/pyexpat xml parsers expat xmlparser DefaultHandler6669739 +Ref: 28986669739 +Ref: library/pyexpat xml parsers expat xmlparser DefaultHandlerExpand6670024 +Ref: 28996670024 +Ref: library/pyexpat xml parsers expat xmlparser NotStandaloneHandler6670264 +Ref: 289a6670264 +Ref: library/pyexpat xml parsers expat xmlparser ExternalEntityRefHandler6670786 +Ref: 28756670786 +Node: ExpatError Exceptions6671842 +Ref: library/pyexpat expaterror-exceptions6671999 +Ref: 289b6671999 +Ref: library/pyexpat expaterror-objects6671999 +Ref: 286b6671999 +Ref: library/pyexpat xml parsers expat ExpatError code6672140 +Ref: 289c6672140 +Ref: library/pyexpat xml parsers expat ExpatError lineno6672789 +Ref: 289f6672789 +Ref: library/pyexpat xml parsers expat ExpatError offset6672916 +Ref: 28a06672916 +Node: Example<12>6673057 +Ref: library/pyexpat example6673220 +Ref: 28a16673220 +Ref: library/pyexpat expat-example6673220 +Ref: 28a26673220 +Node: Content Model Descriptions6674318 +Ref: library/pyexpat content-model-descriptions6674481 +Ref: 28a36674481 +Ref: library/pyexpat expat-content-models6674481 +Ref: 28a46674481 +Ref: library/pyexpat module-xml parsers expat model6674481 +Ref: 13a6674481 +Node: Expat error constants6676324 +Ref: library/pyexpat expat-error-constants6676467 +Ref: 28a56676467 +Ref: library/pyexpat expat-errors6676467 +Ref: 28a66676467 +Ref: library/pyexpat module-xml parsers expat errors6676467 +Ref: 1396676467 +Ref: library/pyexpat xml parsers expat errors codes6677059 +Ref: 289e6677059 +Ref: library/pyexpat xml parsers expat errors messages6677196 +Ref: 289d6677196 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_ASYNC_ENTITY6677349 +Ref: 28a76677349 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_ATTRIBUTE_EXTERNAL_ENTITY_REF6677408 +Ref: 28a86677408 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_BAD_CHAR_REF6677609 +Ref: 28a96677609 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_BINARY_ENTITY_REF6677804 +Ref: 28aa6677804 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_DUPLICATE_ATTRIBUTE6677978 +Ref: 28ab6677978 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_INCORRECT_ENCODING6678103 +Ref: 28ac6678103 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_INVALID_TOKEN6678168 +Ref: 28ad6678168 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_JUNK_AFTER_DOC_ELEMENT6678382 +Ref: 28ae6678382 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_MISPLACED_XML_PI6678531 +Ref: 28af6678531 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_NO_ELEMENTS6678684 +Ref: 28b06678684 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_NO_MEMORY6678859 +Ref: 28b16678859 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_PARAM_ENTITY_REF6678971 +Ref: 28b26678971 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_PARTIAL_CHAR6679105 +Ref: 28b36679105 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_RECURSIVE_ENTITY_REF6679218 +Ref: 28b46679218 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_SYNTAX6679420 +Ref: 28b56679420 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_TAG_MISMATCH6679526 +Ref: 28b66679526 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNCLOSED_TOKEN6679646 +Ref: 28b76679646 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNDEFINED_ENTITY6679830 +Ref: 28b86679830 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNKNOWN_ENCODING6679956 +Ref: 28b96679956 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNCLOSED_CDATA_SECTION6680074 +Ref: 28ba6680074 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_EXTERNAL_ENTITY_HANDLING6680188 +Ref: 28bb6680188 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_NOT_STANDALONE6680259 +Ref: 28bc6680259 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNEXPECTED_STATE6680522 +Ref: 28bd6680522 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_ENTITY_DECLARED_IN_PE6680585 +Ref: 28be6680585 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_FEATURE_REQUIRES_XML_DTD6680653 +Ref: 28bf6680653 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING6680959 +Ref: 28c06680959 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNBOUND_PREFIX6681232 +Ref: 28c16681232 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_UNDECLARING_PREFIX6681374 +Ref: 28c26681374 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_INCOMPLETE_PE6681535 +Ref: 28c36681535 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_XML_DECL6681649 +Ref: 28c46681649 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_TEXT_DECL6681761 +Ref: 28c56681761 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_PUBLICID6681897 +Ref: 28c66681897 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_SUSPENDED6682019 +Ref: 28c76682019 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_NOT_SUSPENDED6682243 +Ref: 28c86682243 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_ABORTED6682395 +Ref: 28c96682395 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_FINISHED6682507 +Ref: 28ca6682507 +Ref: library/pyexpat xml parsers expat errors XML_ERROR_SUSPEND_PE6682753 +Ref: 28cb6682753 +Node: Internet Protocols and Support6682810 +Ref: library/internet doc6682975 +Ref: 28cc6682975 +Ref: library/internet internet6682975 +Ref: 28cd6682975 +Ref: library/internet internet-protocols-and-support6682975 +Ref: 28ce6682975 +Node: webbrowser — Convenient Web-browser controller6685106 +Ref: library/webbrowser doc6685270 +Ref: 28cf6685270 +Ref: library/webbrowser module-webbrowser6685270 +Ref: 1296685270 +Ref: library/webbrowser webbrowser-convenient-web-browser-controller6685270 +Ref: 28d06685270 +Ref: library/webbrowser webbrowser Error6687218 +Ref: 28d26687218 +Ref: library/webbrowser webbrowser open6687349 +Ref: 28d16687349 +Ref: library/webbrowser webbrowser open_new6688156 +Ref: 28d36688156 +Ref: library/webbrowser webbrowser open_new_tab6688322 +Ref: 28d46688322 +Ref: library/webbrowser webbrowser get6688500 +Ref: 28d56688500 +Ref: library/webbrowser webbrowser register6688728 +Ref: 28d66688728 +Ref: webbrowser — Convenient Web-browser controller-Footnote-16695266 +Ref: webbrowser — Convenient Web-browser controller-Footnote-26695335 +Node: Browser Controller Objects6695472 +Ref: library/webbrowser browser-controller-objects6695583 +Ref: 28d76695583 +Ref: library/webbrowser browser-controllers6695583 +Ref: 28d86695583 +Ref: library/webbrowser webbrowser controller open6695763 +Ref: 28d96695763 +Ref: library/webbrowser webbrowser controller open_new6696027 +Ref: 28da6696027 +Ref: library/webbrowser webbrowser controller open_new_tab6696246 +Ref: 28db6696246 +Node: cgi — Common Gateway Interface support6696446 +Ref: library/cgi doc6696662 +Ref: 28dc6696662 +Ref: library/cgi cgi-common-gateway-interface-support6696662 +Ref: 28dd6696662 +Ref: library/cgi module-cgi6696662 +Ref: 166696662 +Ref: cgi — Common Gateway Interface support-Footnote-16697329 +Node: Introduction<10>6697391 +Ref: library/cgi introduction6697513 +Ref: 28de6697513 +Ref: library/cgi cgi-intro6697558 +Ref: 28df6697558 +Node: Using the cgi module6699227 +Ref: library/cgi id16699380 +Ref: 28e06699380 +Ref: library/cgi using-the-cgi-module6699380 +Ref: 28e16699380 +Node: Higher Level Interface6705288 +Ref: library/cgi higher-level-interface6705437 +Ref: 28e36705437 +Ref: library/cgi cgi FieldStorage getfirst6707751 +Ref: 28e46707751 +Ref: library/cgi cgi FieldStorage getlist6708325 +Ref: 28e26708325 +Ref: Higher Level Interface-Footnote-16708899 +Node: Functions<8>6709157 +Ref: library/cgi functions6709307 +Ref: 28e56709307 +Ref: library/cgi functions-in-cgi-module6709307 +Ref: 28e66709307 +Ref: library/cgi cgi parse6709489 +Ref: 2ed6709489 +Ref: library/cgi cgi parse_multipart6709921 +Ref: 2ee6709921 +Ref: library/cgi cgi parse_header6710899 +Ref: 28e76710899 +Ref: library/cgi cgi test6711050 +Ref: 28e86711050 +Ref: library/cgi cgi print_environ6711235 +Ref: 28e96711235 +Ref: library/cgi cgi print_form6711315 +Ref: 28ea6711315 +Ref: library/cgi cgi print_directory6711381 +Ref: 28eb6711381 +Ref: library/cgi cgi print_environ_usage6711463 +Ref: 28ec6711463 +Node: Caring about security6711579 +Ref: library/cgi caring-about-security6711750 +Ref: 28ed6711750 +Ref: library/cgi cgi-security6711750 +Ref: 28ee6711750 +Node: Installing your CGI script on a Unix system6712535 +Ref: library/cgi installing-your-cgi-script-on-a-unix-system6712717 +Ref: 28ef6712717 +Node: Testing your CGI script6714747 +Ref: library/cgi testing-your-cgi-script6714929 +Ref: 28f06714929 +Node: Debugging CGI scripts6715543 +Ref: library/cgi debugging-cgi-scripts6715711 +Ref: 28f16715711 +Node: Common problems and solutions6718860 +Ref: library/cgi common-problems-and-solutions6718996 +Ref: 28f26718996 +Node: cgitb — Traceback manager for CGI scripts6720338 +Ref: library/cgitb doc6720561 +Ref: 28f36720561 +Ref: library/cgitb cgitb-traceback-manager-for-cgi-scripts6720561 +Ref: 28f46720561 +Ref: library/cgitb module-cgitb6720561 +Ref: 176720561 +Ref: library/cgitb cgitb enable6721751 +Ref: 28f56721751 +Ref: library/cgitb cgitb text6722685 +Ref: 28f66722685 +Ref: library/cgitb cgitb html6723113 +Ref: 28f76723113 +Ref: library/cgitb cgitb handler6723541 +Ref: 28f86723541 +Ref: cgitb — Traceback manager for CGI scripts-Footnote-16724174 +Node: wsgiref — WSGI Utilities and Reference Implementation6724238 +Ref: library/wsgiref doc6724452 +Ref: 28f96724452 +Ref: library/wsgiref module-wsgiref6724452 +Ref: 12c6724452 +Ref: library/wsgiref wsgiref-wsgi-utilities-and-reference-implementation6724452 +Ref: 28fa6724452 +Ref: wsgiref — WSGI Utilities and Reference Implementation-Footnote-16726297 +Ref: wsgiref — WSGI Utilities and Reference Implementation-Footnote-26726346 +Node: wsgiref util – WSGI environment utilities6726383 +Ref: library/wsgiref module-wsgiref util6726573 +Ref: 1306726573 +Ref: library/wsgiref wsgiref-util-wsgi-environment-utilities6726573 +Ref: 28fb6726573 +Ref: library/wsgiref wsgiref util guess_scheme6727030 +Ref: 28fc6727030 +Ref: library/wsgiref wsgiref util request_uri6727682 +Ref: 28fd6727682 +Ref: library/wsgiref wsgiref util application_uri6728008 +Ref: 28fe6728008 +Ref: library/wsgiref wsgiref util shift_path_info6728274 +Ref: 28ff6728274 +Ref: library/wsgiref wsgiref util setup_testing_defaults6729942 +Ref: 29006729942 +Ref: library/wsgiref wsgiref util is_hop_by_hop6731626 +Ref: 29016731626 +Ref: library/wsgiref wsgiref util FileWrapper6731799 +Ref: 27e6731799 +Ref: wsgiref util – WSGI environment utilities-Footnote-16733072 +Ref: wsgiref util – WSGI environment utilities-Footnote-26733121 +Ref: wsgiref util – WSGI environment utilities-Footnote-36733170 +Ref: wsgiref util – WSGI environment utilities-Footnote-46733219 +Ref: wsgiref util – WSGI environment utilities-Footnote-56733268 +Node: wsgiref headers – WSGI response header tools6733317 +Ref: library/wsgiref module-wsgiref headers6733567 +Ref: 12e6733567 +Ref: library/wsgiref wsgiref-headers-wsgi-response-header-tools6733567 +Ref: 29026733567 +Ref: library/wsgiref wsgiref headers Headers6733829 +Ref: 78f6733829 +Ref: library/wsgiref wsgiref headers Headers get_all6736094 +Ref: 29036736094 +Ref: library/wsgiref wsgiref headers Headers add_header6736531 +Ref: 29046736531 +Ref: wsgiref headers – WSGI response header tools-Footnote-16737668 +Node: wsgiref simple_server – a simple WSGI HTTP server6737717 +Ref: library/wsgiref module-wsgiref simple_server6737969 +Ref: 12f6737969 +Ref: library/wsgiref wsgiref-simple-server-a-simple-wsgi-http-server6737969 +Ref: 29056737969 +Ref: library/wsgiref wsgiref simple_server make_server6738563 +Ref: 29066738563 +Ref: library/wsgiref wsgiref simple_server demo_app6739403 +Ref: 29076739403 +Ref: library/wsgiref wsgiref simple_server WSGIServer6739843 +Ref: 29086739843 +Ref: library/wsgiref wsgiref simple_server WSGIServer set_app6740596 +Ref: 290a6740596 +Ref: library/wsgiref wsgiref simple_server WSGIServer get_app6740741 +Ref: 290b6740741 +Ref: library/wsgiref wsgiref simple_server WSGIRequestHandler6741084 +Ref: 290c6741084 +Ref: library/wsgiref wsgiref simple_server WSGIRequestHandler get_environ6741700 +Ref: 290d6741700 +Ref: library/wsgiref wsgiref simple_server WSGIRequestHandler get_stderr6742206 +Ref: 290e6742206 +Ref: library/wsgiref wsgiref simple_server WSGIRequestHandler handle6742389 +Ref: 290f6742389 +Ref: wsgiref simple_server – a simple WSGI HTTP server-Footnote-16742660 +Ref: wsgiref simple_server – a simple WSGI HTTP server-Footnote-26742709 +Node: wsgiref validate — WSGI conformance checker6742758 +Ref: library/wsgiref module-wsgiref validate6743012 +Ref: 1316743012 +Ref: library/wsgiref wsgiref-validate-wsgi-conformance-checker6743012 +Ref: 29106743012 +Ref: library/wsgiref wsgiref validate validator6743919 +Ref: 29116743919 +Ref: wsgiref validate — WSGI conformance checker-Footnote-16746138 +Ref: wsgiref validate — WSGI conformance checker-Footnote-26746187 +Ref: wsgiref validate — WSGI conformance checker-Footnote-36746236 +Node: wsgiref handlers – server/gateway base classes6746285 +Ref: library/wsgiref module-wsgiref handlers6746500 +Ref: 12d6746500 +Ref: library/wsgiref wsgiref-handlers-server-gateway-base-classes6746500 +Ref: 29126746500 +Ref: library/wsgiref wsgiref handlers CGIHandler6746891 +Ref: 29136746891 +Ref: library/wsgiref wsgiref handlers IISCGIHandler6747540 +Ref: 29156747540 +Ref: library/wsgiref wsgiref handlers BaseCGIHandler6748915 +Ref: 29146748915 +Ref: library/wsgiref wsgiref handlers SimpleHandler6749782 +Ref: 29166749782 +Ref: library/wsgiref wsgiref handlers BaseHandler6750699 +Ref: 29176750699 +Ref: library/wsgiref wsgiref handlers BaseHandler run6751058 +Ref: 29186751058 +Ref: library/wsgiref wsgiref handlers BaseHandler _write6751397 +Ref: 29196751397 +Ref: library/wsgiref wsgiref handlers BaseHandler _flush6751743 +Ref: 291a6751743 +Ref: library/wsgiref wsgiref handlers BaseHandler get_stdin6751952 +Ref: 291b6751952 +Ref: library/wsgiref wsgiref handlers BaseHandler get_stderr6752117 +Ref: 291c6752117 +Ref: library/wsgiref wsgiref handlers BaseHandler add_cgi_vars6752285 +Ref: 291d6752285 +Ref: library/wsgiref wsgiref handlers BaseHandler wsgi_multithread6752837 +Ref: 291e6752837 +Ref: library/wsgiref wsgiref handlers BaseHandler wsgi_multiprocess6753128 +Ref: 291f6753128 +Ref: library/wsgiref wsgiref handlers BaseHandler wsgi_run_once6753421 +Ref: 29206753421 +Ref: library/wsgiref wsgiref handlers BaseHandler os_environ6753664 +Ref: 29216753664 +Ref: library/wsgiref wsgiref handlers BaseHandler server_software6754170 +Ref: 29226754170 +Ref: library/wsgiref wsgiref handlers BaseHandler get_scheme6754766 +Ref: 29246754766 +Ref: library/wsgiref wsgiref handlers BaseHandler setup_environ6755127 +Ref: 29256755127 +Ref: library/wsgiref wsgiref handlers BaseHandler log_exception6755787 +Ref: 29276755787 +Ref: library/wsgiref wsgiref handlers BaseHandler traceback_limit6756287 +Ref: 29286756287 +Ref: library/wsgiref wsgiref handlers BaseHandler error_output6756509 +Ref: 29296756509 +Ref: library/wsgiref wsgiref handlers BaseHandler error_status6757581 +Ref: 292a6757581 +Ref: library/wsgiref wsgiref handlers BaseHandler error_headers6757786 +Ref: 292b6757786 +Ref: library/wsgiref wsgiref handlers BaseHandler error_body6758078 +Ref: 292c6758078 +Ref: library/wsgiref wsgiref handlers BaseHandler wsgi_file_wrapper6758424 +Ref: 29266758424 +Ref: library/wsgiref wsgiref handlers BaseHandler sendfile6758632 +Ref: 292d6758632 +Ref: library/wsgiref wsgiref handlers BaseHandler origin_server6759208 +Ref: 29236759208 +Ref: library/wsgiref wsgiref handlers BaseHandler http_version6759735 +Ref: 292e6759735 +Ref: library/wsgiref wsgiref handlers read_environ6759957 +Ref: b366759957 +Ref: wsgiref handlers – server/gateway base classes-Footnote-16760871 +Ref: wsgiref handlers – server/gateway base classes-Footnote-26760920 +Ref: wsgiref handlers – server/gateway base classes-Footnote-36760969 +Ref: wsgiref handlers – server/gateway base classes-Footnote-46761018 +Ref: wsgiref handlers – server/gateway base classes-Footnote-56761067 +Node: Examples<20>6761116 +Ref: library/wsgiref examples6761277 +Ref: 292f6761277 +Node: urllib — URL handling modules6763849 +Ref: library/urllib doc6764074 +Ref: 29306764074 +Ref: library/urllib module-urllib6764074 +Ref: 11d6764074 +Ref: library/urllib urllib-url-handling-modules6764074 +Ref: 29316764074 +Ref: urllib — URL handling modules-Footnote-16764655 +Node: urllib request — Extensible library for opening URLs6764718 +Ref: library/urllib request doc6764939 +Ref: 29326764939 +Ref: library/urllib request module-urllib request6764939 +Ref: 1206764939 +Ref: library/urllib request urllib-request-extensible-library-for-opening-urls6764939 +Ref: 29336764939 +Ref: library/urllib request urllib request urlopen6765550 +Ref: 78c6765550 +Ref: library/urllib request urllib request install_opener6769862 +Ref: 293b6769862 +Ref: library/urllib request urllib request build_opener6770315 +Ref: 293c6770315 +Ref: library/urllib request urllib request pathname2url6771433 +Ref: 29446771433 +Ref: library/urllib request urllib request url2pathname6771732 +Ref: 29456771732 +Ref: library/urllib request urllib request getproxies6771989 +Ref: 29476771989 +Ref: library/urllib request urllib request Request6773148 +Ref: 8f66773148 +Ref: library/urllib request urllib request OpenerDirector6777579 +Ref: 29386777579 +Ref: library/urllib request urllib request BaseHandler6777798 +Ref: 293d6777798 +Ref: library/urllib request urllib request HTTPDefaultErrorHandler6777958 +Ref: 293f6777958 +Ref: library/urllib request urllib request HTTPRedirectHandler6778150 +Ref: 29406778150 +Ref: library/urllib request urllib request HTTPCookieProcessor6778235 +Ref: 29496778235 +Ref: library/urllib request urllib request ProxyHandler6778337 +Ref: 293a6778337 +Ref: library/urllib request urllib request HTTPPasswordMgr6779449 +Ref: 294a6779449 +Ref: library/urllib request urllib request HTTPPasswordMgrWithDefaultRealm6779566 +Ref: 294b6779566 +Ref: library/urllib request urllib request HTTPPasswordMgrWithPriorAuth6779807 +Ref: 78a6779807 +Ref: library/urllib request urllib request AbstractBasicAuthHandler6780193 +Ref: 294c6780193 +Ref: library/urllib request urllib request HTTPBasicAuthHandler6781535 +Ref: 294f6781535 +Ref: library/urllib request urllib request ProxyBasicAuthHandler6781999 +Ref: 29506781999 +Ref: library/urllib request urllib request AbstractDigestAuthHandler6782339 +Ref: 29516782339 +Ref: library/urllib request urllib request HTTPDigestAuthHandler6782749 +Ref: 29526782749 +Ref: library/urllib request urllib request ProxyDigestAuthHandler6783611 +Ref: 29536783611 +Ref: library/urllib request urllib request HTTPHandler6783952 +Ref: 293e6783952 +Ref: library/urllib request urllib request HTTPSHandler6784037 +Ref: ba86784037 +Ref: library/urllib request urllib request FileHandler6784367 +Ref: 29426784367 +Ref: library/urllib request urllib request DataHandler6784430 +Ref: 8f56784430 +Ref: library/urllib request urllib request FTPHandler6784517 +Ref: 29416784517 +Ref: library/urllib request urllib request CacheFTPHandler6784576 +Ref: 29546784576 +Ref: library/urllib request urllib request UnknownHandler6784705 +Ref: 29396784705 +Ref: library/urllib request urllib request HTTPErrorProcessor6784795 +Ref: 29436784795 +Ref: urllib request — Extensible library for opening URLs-Footnote-16785676 +Ref: urllib request — Extensible library for opening URLs-Footnote-26785749 +Ref: urllib request — Extensible library for opening URLs-Footnote-36785800 +Ref: urllib request — Extensible library for opening URLs-Footnote-46785837 +Ref: urllib request — Extensible library for opening URLs-Footnote-56785886 +Ref: urllib request — Extensible library for opening URLs-Footnote-66785935 +Node: Request Objects6785984 +Ref: library/urllib request id16786121 +Ref: 29556786121 +Ref: library/urllib request request-objects6786121 +Ref: 29566786121 +Ref: library/urllib request urllib request Request full_url6786392 +Ref: 8f86786392 +Ref: library/urllib request urllib request Request type6786687 +Ref: 29576786687 +Ref: library/urllib request urllib request Request host6786738 +Ref: 29586786738 +Ref: library/urllib request urllib request Request origin_req_host6786865 +Ref: 29596786865 +Ref: library/urllib request urllib request Request selector6786960 +Ref: 295a6786960 +Ref: library/urllib request urllib request Request data6787124 +Ref: 8f96787124 +Ref: library/urllib request urllib request Request unverifiable6787387 +Ref: 295b6787387 +Ref: library/urllib request urllib request Request method6787517 +Ref: 8f76787517 +Ref: library/urllib request urllib request Request get_method6788223 +Ref: ac16788223 +Ref: library/urllib request urllib request Request add_header6788645 +Ref: 29486788645 +Ref: library/urllib request urllib request Request add_unredirected_header6789238 +Ref: 295c6789238 +Ref: library/urllib request urllib request Request has_header6789364 +Ref: 295d6789364 +Ref: library/urllib request urllib request Request remove_header6789505 +Ref: 8fb6789505 +Ref: library/urllib request urllib request Request get_full_url6789678 +Ref: 295e6789678 +Ref: library/urllib request urllib request Request set_proxy6789835 +Ref: 295f6789835 +Ref: library/urllib request urllib request Request get_header6790088 +Ref: 29606790088 +Ref: library/urllib request urllib request Request header_items6790254 +Ref: 29616790254 +Ref: Request Objects-Footnote-16790613 +Node: OpenerDirector Objects6790662 +Ref: library/urllib request opener-director-objects6790827 +Ref: 29626790827 +Ref: library/urllib request openerdirector-objects6790827 +Ref: 29636790827 +Ref: library/urllib request urllib request OpenerDirector add_handler6790959 +Ref: 29646790959 +Ref: library/urllib request urllib request OpenerDirector open6792517 +Ref: 8fa6792517 +Ref: library/urllib request urllib request OpenerDirector error6793201 +Ref: 29696793201 +Node: BaseHandler Objects6794977 +Ref: library/urllib request base-handler-objects6795154 +Ref: 296a6795154 +Ref: library/urllib request basehandler-objects6795154 +Ref: 296b6795154 +Ref: library/urllib request urllib request BaseHandler add_parent6795394 +Ref: 296c6795394 +Ref: library/urllib request urllib request BaseHandler close6795473 +Ref: 296d6795473 +Ref: library/urllib request urllib request BaseHandler parent6795849 +Ref: 296e6795849 +Ref: library/urllib request urllib request BaseHandler default_open6796003 +Ref: 296f6796003 +Ref: library/urllib request protocol-open6796677 +Ref: 29656796677 +Ref: library/urllib request urllib request BaseHandler unknown_open6797043 +Ref: 29706797043 +Ref: library/urllib request urllib request BaseHandler http_error_default6797455 +Ref: 29716797455 +Ref: library/urllib request http-error-nnn6798262 +Ref: 29666798262 +Ref: library/urllib request protocol-request6798760 +Ref: 29676798760 +Ref: library/urllib request protocol-response6799174 +Ref: 29686799174 +Node: HTTPRedirectHandler Objects6799765 +Ref: library/urllib request http-redirect-handler6799947 +Ref: 29726799947 +Ref: library/urllib request httpredirecthandler-objects6799947 +Ref: 29736799947 +Ref: library/urllib request urllib request HTTPRedirectHandler redirect_request6800444 +Ref: 29746800444 +Ref: library/urllib request urllib request HTTPRedirectHandler http_error_3016801528 +Ref: 29756801528 +Ref: library/urllib request urllib request HTTPRedirectHandler http_error_3026801799 +Ref: 29766801799 +Ref: library/urllib request urllib request HTTPRedirectHandler http_error_3036801978 +Ref: 29776801978 +Ref: library/urllib request urllib request HTTPRedirectHandler http_error_3076802161 +Ref: 29786802161 +Ref: HTTPRedirectHandler Objects-Footnote-16802389 +Ref: HTTPRedirectHandler Objects-Footnote-26802438 +Node: HTTPCookieProcessor Objects6802487 +Ref: library/urllib request http-cookie-processor6802670 +Ref: 29796802670 +Ref: library/urllib request httpcookieprocessor-objects6802670 +Ref: 297a6802670 +Ref: library/urllib request urllib request HTTPCookieProcessor cookiejar6802809 +Ref: 297b6802809 +Node: ProxyHandler Objects6802937 +Ref: library/urllib request proxy-handler6803116 +Ref: 297d6803116 +Ref: library/urllib request proxyhandler-objects6803116 +Ref: 297e6803116 +Node: HTTPPasswordMgr Objects6803594 +Ref: library/urllib request http-password-mgr6803782 +Ref: 294d6803782 +Ref: library/urllib request httppasswordmgr-objects6803782 +Ref: 297f6803782 +Ref: library/urllib request urllib request HTTPPasswordMgr add_password6803968 +Ref: 29806803968 +Ref: library/urllib request urllib request HTTPPasswordMgr find_user_password6804315 +Ref: 29816804315 +Node: HTTPPasswordMgrWithPriorAuth Objects6804689 +Ref: library/urllib request http-password-mgr-with-prior-auth6804889 +Ref: 294e6804889 +Ref: library/urllib request httppasswordmgrwithpriorauth-objects6804889 +Ref: 29826804889 +Ref: library/urllib request urllib request HTTPPasswordMgrWithPriorAuth add_password6805143 +Ref: 29836805143 +Ref: library/urllib request urllib request HTTPPasswordMgrWithPriorAuth find_user_password6805557 +Ref: 29846805557 +Ref: library/urllib request urllib request HTTPPasswordMgrWithPriorAuth update_authenticated6805716 +Ref: 29856805716 +Ref: library/urllib request urllib request HTTPPasswordMgrWithPriorAuth is_authenticated6805912 +Ref: 29866805912 +Node: AbstractBasicAuthHandler Objects6806088 +Ref: library/urllib request abstract-basic-auth-handler6806293 +Ref: 29876806293 +Ref: library/urllib request abstractbasicauthhandler-objects6806293 +Ref: 29886806293 +Ref: library/urllib request urllib request AbstractBasicAuthHandler http_error_auth_reqed6806378 +Ref: 29896806378 +Node: HTTPBasicAuthHandler Objects6807198 +Ref: library/urllib request http-basic-auth-handler6807396 +Ref: 298a6807396 +Ref: library/urllib request httpbasicauthhandler-objects6807396 +Ref: 298b6807396 +Ref: library/urllib request urllib request HTTPBasicAuthHandler http_error_4016807475 +Ref: 298c6807475 +Node: ProxyBasicAuthHandler Objects6807632 +Ref: library/urllib request proxy-basic-auth-handler6807831 +Ref: 298d6807831 +Ref: library/urllib request proxybasicauthhandler-objects6807831 +Ref: 298e6807831 +Ref: library/urllib request urllib request ProxyBasicAuthHandler http_error_4076807912 +Ref: 298f6807912 +Node: AbstractDigestAuthHandler Objects6808070 +Ref: library/urllib request abstract-digest-auth-handler6808270 +Ref: 29906808270 +Ref: library/urllib request abstractdigestauthhandler-objects6808270 +Ref: 29916808270 +Ref: library/urllib request urllib request AbstractDigestAuthHandler http_error_auth_reqed6808359 +Ref: 29926808359 +Node: HTTPDigestAuthHandler Objects6808736 +Ref: library/urllib request http-digest-auth-handler6808937 +Ref: 29936808937 +Ref: library/urllib request httpdigestauthhandler-objects6808937 +Ref: 29946808937 +Ref: library/urllib request urllib request HTTPDigestAuthHandler http_error_4016809018 +Ref: 29956809018 +Node: ProxyDigestAuthHandler Objects6809176 +Ref: library/urllib request proxy-digest-auth-handler6809363 +Ref: 29966809363 +Ref: library/urllib request proxydigestauthhandler-objects6809363 +Ref: 29976809363 +Ref: library/urllib request urllib request ProxyDigestAuthHandler http_error_4076809446 +Ref: 29986809446 +Node: HTTPHandler Objects6809605 +Ref: library/urllib request http-handler-objects6809783 +Ref: 29996809783 +Ref: library/urllib request httphandler-objects6809783 +Ref: 299a6809783 +Ref: library/urllib request urllib request HTTPHandler http_open6809844 +Ref: 299b6809844 +Node: HTTPSHandler Objects6809986 +Ref: library/urllib request https-handler-objects6810153 +Ref: 299c6810153 +Ref: library/urllib request httpshandler-objects6810153 +Ref: 299d6810153 +Ref: library/urllib request urllib request HTTPSHandler https_open6810216 +Ref: 299e6810216 +Node: FileHandler Objects6810361 +Ref: library/urllib request file-handler-objects6810528 +Ref: 299f6810528 +Ref: library/urllib request filehandler-objects6810528 +Ref: 29a06810528 +Ref: library/urllib request urllib request FileHandler file_open6810589 +Ref: 29a16810589 +Node: DataHandler Objects6810891 +Ref: library/urllib request data-handler-objects6811056 +Ref: 29a26811056 +Ref: library/urllib request datahandler-objects6811056 +Ref: 29a36811056 +Ref: library/urllib request urllib request DataHandler data_open6811117 +Ref: 29a46811117 +Ref: DataHandler Objects-Footnote-16811676 +Node: FTPHandler Objects6811725 +Ref: library/urllib request ftp-handler-objects6811894 +Ref: 29a56811894 +Ref: library/urllib request ftphandler-objects6811894 +Ref: 29a66811894 +Ref: library/urllib request urllib request FTPHandler ftp_open6811953 +Ref: 29a76811953 +Node: CacheFTPHandler Objects6812101 +Ref: library/urllib request cacheftp-handler-objects6812273 +Ref: 29a86812273 +Ref: library/urllib request cacheftphandler-objects6812273 +Ref: 29a96812273 +Ref: library/urllib request urllib request CacheFTPHandler setTimeout6812455 +Ref: 29aa6812455 +Ref: library/urllib request urllib request CacheFTPHandler setMaxConns6812548 +Ref: 29ab6812548 +Node: UnknownHandler Objects6812648 +Ref: library/urllib request unknown-handler-objects6812828 +Ref: 29ac6812828 +Ref: library/urllib request unknownhandler-objects6812828 +Ref: 29ad6812828 +Ref: library/urllib request urllib request UnknownHandler unknown_open6812895 +Ref: 29ae6812895 +Node: HTTPErrorProcessor Objects6812986 +Ref: library/urllib request http-error-processor-objects6813155 +Ref: 29af6813155 +Ref: library/urllib request httperrorprocessor-objects6813155 +Ref: 29b06813155 +Ref: library/urllib request urllib request HTTPErrorProcessor http_response6813230 +Ref: 29b16813230 +Ref: library/urllib request urllib request HTTPErrorProcessor https_response6813700 +Ref: 29b26813700 +Node: Examples<21>6813863 +Ref: library/urllib request examples6814026 +Ref: 29b36814026 +Ref: library/urllib request urllib-request-examples6814026 +Ref: 29b46814026 +Ref: library/urllib request urllib-examples6819594 +Ref: 29b66819594 +Node: Legacy interface6821087 +Ref: library/urllib request legacy-interface6821251 +Ref: 29b76821251 +Ref: library/urllib request urllib request urlretrieve6821482 +Ref: 29b86821482 +Ref: library/urllib request urllib request urlcleanup6824144 +Ref: 29b96824144 +Ref: library/urllib request urllib request URLopener6824304 +Ref: 29356824304 +Ref: library/urllib request urllib request URLopener open6825813 +Ref: 29bb6825813 +Ref: library/urllib request urllib request URLopener open_unknown6826316 +Ref: 29bc6826316 +Ref: library/urllib request urllib request URLopener retrieve6826428 +Ref: 29bd6826428 +Ref: library/urllib request urllib request URLopener version6827977 +Ref: 29ba6827977 +Ref: library/urllib request urllib request FancyURLopener6828298 +Ref: 29366828298 +Ref: library/urllib request urllib request FancyURLopener prompt_user_passwd6830035 +Ref: 29be6830035 +Ref: Legacy interface-Footnote-16830575 +Node: urllib request Restrictions6830624 +Ref: library/urllib request urllib-request-restrictions6830767 +Ref: 29bf6830767 +Node: urllib response — Response classes used by urllib6833335 +Ref: library/urllib request module-urllib response6833568 +Ref: 1216833568 +Ref: library/urllib request urllib-response-response-classes-used-by-urllib6833568 +Ref: 29c06833568 +Node: urllib parse — Parse URLs into components6834116 +Ref: library/urllib parse doc6834354 +Ref: 29c16834354 +Ref: library/urllib parse module-urllib parse6834354 +Ref: 11f6834354 +Ref: library/urllib parse urllib-parse-parse-urls-into-components6834354 +Ref: 29c26834354 +Ref: urllib parse — Parse URLs into components-Footnote-16835629 +Node: URL Parsing6835700 +Ref: library/urllib parse url-parsing6835827 +Ref: 29c36835827 +Ref: library/urllib parse urllib parse urlparse6836000 +Ref: 5fa6836000 +Ref: library/urllib parse urllib parse parse_qs6843300 +Ref: 2eb6843300 +Ref: library/urllib parse urllib parse parse_qsl6845483 +Ref: 2ec6845483 +Ref: library/urllib parse urllib parse urlunparse6847516 +Ref: 29c56847516 +Ref: library/urllib parse urllib parse urlsplit6847918 +Ref: 5f96847918 +Ref: library/urllib parse urllib parse urlunsplit6851950 +Ref: 29c66851950 +Ref: library/urllib parse urllib parse urljoin6852394 +Ref: 78d6852394 +Ref: library/urllib parse urllib parse urldefrag6853704 +Ref: bde6853704 +Ref: library/urllib parse urllib parse unwrap6855049 +Ref: 29c76855049 +Ref: URL Parsing-Footnote-16855397 +Ref: URL Parsing-Footnote-26855446 +Ref: URL Parsing-Footnote-36855495 +Ref: URL Parsing-Footnote-46855544 +Node: Parsing ASCII Encoded Bytes6855593 +Ref: library/urllib parse id16855753 +Ref: 29c86855753 +Ref: library/urllib parse parsing-ascii-encoded-bytes6855753 +Ref: be06855753 +Node: Structured Parse Results6857879 +Ref: library/urllib parse structured-parse-results6858039 +Ref: 29c96858039 +Ref: library/urllib parse urlparse-result-object6858039 +Ref: 29c46858039 +Ref: library/urllib parse urllib parse urllib parse SplitResult geturl6858455 +Ref: 29ca6858455 +Ref: library/urllib parse urllib parse DefragResult6859578 +Ref: 29cb6859578 +Ref: library/urllib parse urllib parse ParseResult6859836 +Ref: 29cd6859836 +Ref: library/urllib parse urllib parse SplitResult6860107 +Ref: 29cf6860107 +Ref: library/urllib parse urllib parse DefragResultBytes6860512 +Ref: 29cc6860512 +Ref: library/urllib parse urllib parse ParseResultBytes6860772 +Ref: 29ce6860772 +Ref: library/urllib parse urllib parse SplitResultBytes6861071 +Ref: 29d06861071 +Node: URL Quoting6861362 +Ref: library/urllib parse url-quoting6861486 +Ref: 29d16861486 +Ref: library/urllib parse urllib parse quote6861889 +Ref: 40d6861889 +Ref: library/urllib parse urllib parse quote_plus6863249 +Ref: bdf6863249 +Ref: library/urllib parse urllib parse quote_from_bytes6863740 +Ref: 29d26863740 +Ref: library/urllib parse urllib parse unquote6864033 +Ref: 29466864033 +Ref: library/urllib parse urllib parse unquote_plus6864662 +Ref: 29d36864662 +Ref: library/urllib parse urllib parse unquote_to_bytes6865000 +Ref: 29d46865000 +Ref: library/urllib parse urllib parse urlencode6865418 +Ref: 78b6865418 +Ref: URL Quoting-Footnote-16869051 +Ref: URL Quoting-Footnote-26869100 +Ref: URL Quoting-Footnote-36869149 +Ref: URL Quoting-Footnote-46869198 +Ref: URL Quoting-Footnote-56869247 +Ref: URL Quoting-Footnote-66869296 +Ref: URL Quoting-Footnote-76869345 +Ref: URL Quoting-Footnote-86869394 +Node: urllib error — Exception classes raised by urllib request6869443 +Ref: library/urllib error doc6869674 +Ref: 29d56869674 +Ref: library/urllib error module-urllib error6869674 +Ref: 11e6869674 +Ref: library/urllib error urllib-error-exception-classes-raised-by-urllib-request6869674 +Ref: 29d66869674 +Ref: library/urllib error urllib error URLError6870173 +Ref: 29376870173 +Ref: library/urllib error urllib error URLError reason6870351 +Ref: 29d76870351 +Ref: library/urllib error urllib error HTTPError6870622 +Ref: 8fc6870622 +Ref: library/urllib error urllib error HTTPError code6870979 +Ref: 29d86870979 +Ref: library/urllib error urllib error HTTPError reason6871239 +Ref: 29da6871239 +Ref: library/urllib error urllib error HTTPError headers6871341 +Ref: 8fd6871341 +Ref: library/urllib error urllib error ContentTooShortError6871507 +Ref: 29db6871507 +Ref: urllib error — Exception classes raised by urllib request-Footnote-16871907 +Ref: urllib error — Exception classes raised by urllib request-Footnote-26871978 +Node: urllib robotparser — Parser for robots txt6872027 +Ref: library/urllib robotparser doc6872236 +Ref: 29dc6872236 +Ref: library/urllib robotparser module-urllib robotparser6872236 +Ref: 1226872236 +Ref: library/urllib robotparser urllib-robotparser-parser-for-robots-txt6872236 +Ref: 29dd6872236 +Ref: library/urllib robotparser urllib robotparser RobotFileParser6872779 +Ref: 5ad6872779 +Ref: library/urllib robotparser urllib robotparser RobotFileParser set_url6872952 +Ref: 29de6872952 +Ref: library/urllib robotparser urllib robotparser RobotFileParser read6873046 +Ref: 29df6873046 +Ref: library/urllib robotparser urllib robotparser RobotFileParser parse6873142 +Ref: 29e06873142 +Ref: library/urllib robotparser urllib robotparser RobotFileParser can_fetch6873212 +Ref: 29e16873212 +Ref: library/urllib robotparser urllib robotparser RobotFileParser mtime6873426 +Ref: 29e26873426 +Ref: library/urllib robotparser urllib robotparser RobotFileParser modified6873654 +Ref: 29e36873654 +Ref: library/urllib robotparser urllib robotparser RobotFileParser crawl_delay6873783 +Ref: 29e46873783 +Ref: library/urllib robotparser urllib robotparser RobotFileParser request_rate6874185 +Ref: 29e56874185 +Ref: library/urllib robotparser urllib robotparser RobotFileParser site_maps6874636 +Ref: 29e66874636 +Ref: urllib robotparser — Parser for robots txt-Footnote-16875556 +Node: http — HTTP modules6875634 +Ref: library/http doc6875820 +Ref: 29e76875820 +Ref: library/http http-http-modules6875820 +Ref: 29e86875820 +Ref: library/http module-http6875820 +Ref: 936875820 +Ref: library/http http HTTPStatus6876643 +Ref: 6e26876643 +Ref: http — HTTP modules-Footnote-16877383 +Node: HTTP status codes6877455 +Ref: library/http http-status-codes6877530 +Ref: 29e96877530 +Ref: library/http id16877530 +Ref: 29ea6877530 +Ref: HTTP status codes-Footnote-16890588 +Ref: HTTP status codes-Footnote-26890672 +Ref: HTTP status codes-Footnote-36890721 +Ref: HTTP status codes-Footnote-46890770 +Ref: HTTP status codes-Footnote-56890819 +Ref: HTTP status codes-Footnote-66890868 +Ref: HTTP status codes-Footnote-76890917 +Ref: HTTP status codes-Footnote-86890966 +Ref: HTTP status codes-Footnote-96891015 +Ref: HTTP status codes-Footnote-106891064 +Ref: HTTP status codes-Footnote-116891114 +Ref: HTTP status codes-Footnote-126891164 +Ref: HTTP status codes-Footnote-136891214 +Ref: HTTP status codes-Footnote-146891264 +Ref: HTTP status codes-Footnote-156891314 +Ref: HTTP status codes-Footnote-166891364 +Ref: HTTP status codes-Footnote-176891414 +Ref: HTTP status codes-Footnote-186891464 +Ref: HTTP status codes-Footnote-196891514 +Ref: HTTP status codes-Footnote-206891564 +Ref: HTTP status codes-Footnote-216891614 +Ref: HTTP status codes-Footnote-226891664 +Ref: HTTP status codes-Footnote-236891714 +Ref: HTTP status codes-Footnote-246891764 +Ref: HTTP status codes-Footnote-256891814 +Ref: HTTP status codes-Footnote-266891864 +Ref: HTTP status codes-Footnote-276891914 +Ref: HTTP status codes-Footnote-286891964 +Ref: HTTP status codes-Footnote-296892014 +Ref: HTTP status codes-Footnote-306892064 +Ref: HTTP status codes-Footnote-316892114 +Ref: HTTP status codes-Footnote-326892164 +Ref: HTTP status codes-Footnote-336892214 +Ref: HTTP status codes-Footnote-346892264 +Ref: HTTP status codes-Footnote-356892314 +Ref: HTTP status codes-Footnote-366892364 +Ref: HTTP status codes-Footnote-376892414 +Ref: HTTP status codes-Footnote-386892464 +Ref: HTTP status codes-Footnote-396892514 +Ref: HTTP status codes-Footnote-406892564 +Ref: HTTP status codes-Footnote-416892614 +Ref: HTTP status codes-Footnote-426892664 +Ref: HTTP status codes-Footnote-436892714 +Ref: HTTP status codes-Footnote-446892764 +Ref: HTTP status codes-Footnote-456892814 +Ref: HTTP status codes-Footnote-466892864 +Ref: HTTP status codes-Footnote-476892914 +Ref: HTTP status codes-Footnote-486892964 +Ref: HTTP status codes-Footnote-496893014 +Ref: HTTP status codes-Footnote-506893064 +Ref: HTTP status codes-Footnote-516893114 +Ref: HTTP status codes-Footnote-526893164 +Ref: HTTP status codes-Footnote-536893214 +Ref: HTTP status codes-Footnote-546893264 +Ref: HTTP status codes-Footnote-556893314 +Ref: HTTP status codes-Footnote-566893364 +Ref: HTTP status codes-Footnote-576893414 +Ref: HTTP status codes-Footnote-586893464 +Ref: HTTP status codes-Footnote-596893514 +Ref: HTTP status codes-Footnote-606893564 +Node: http client — HTTP protocol client6893614 +Ref: library/http client doc6893786 +Ref: 29eb6893786 +Ref: library/http client http-client-http-protocol-client6893786 +Ref: 29ec6893786 +Ref: library/http client module-http client6893786 +Ref: 946893786 +Ref: library/http client http client HTTPConnection6894478 +Ref: 3926894478 +Ref: library/http client http client HTTPSConnection6896056 +Ref: 3936896056 +Ref: library/http client http client HTTPResponse6897890 +Ref: a116897890 +Ref: library/http client http client parse_headers6898272 +Ref: 29ed6898272 +Ref: library/http client http client HTTPException6899207 +Ref: 29f06899207 +Ref: library/http client http client NotConnected6899358 +Ref: 29f16899358 +Ref: library/http client http client InvalidURL6899446 +Ref: 29f26899446 +Ref: library/http client http client UnknownProtocol6899599 +Ref: 29f36899599 +Ref: library/http client http client UnknownTransferEncoding6899690 +Ref: 29f46899690 +Ref: library/http client http client UnimplementedFileMode6899789 +Ref: 29f56899789 +Ref: library/http client http client IncompleteRead6899886 +Ref: 29f66899886 +Ref: library/http client http client ImproperConnectionState6899976 +Ref: 29f76899976 +Ref: library/http client http client CannotSendRequest6900075 +Ref: 29f86900075 +Ref: library/http client http client CannotSendHeader6900178 +Ref: 29f96900178 +Ref: library/http client http client ResponseNotReady6900280 +Ref: 29fa6900280 +Ref: library/http client http client BadStatusLine6900382 +Ref: 29fb6900382 +Ref: library/http client http client LineTooLong6900557 +Ref: 29fc6900557 +Ref: library/http client http client RemoteDisconnected6900735 +Ref: 6e56900735 +Ref: library/http client http client HTTP_PORT6901214 +Ref: 29fd6901214 +Ref: library/http client http client HTTPS_PORT6901311 +Ref: 29fe6901311 +Ref: library/http client http client responses6901411 +Ref: 29ff6901411 +Ref: http client — HTTP protocol client-Footnote-16901872 +Ref: http client — HTTP protocol client-Footnote-26901942 +Ref: http client — HTTP protocol client-Footnote-36901993 +Node: HTTPConnection Objects6902042 +Ref: library/http client httpconnection-objects6902166 +Ref: 2a006902166 +Ref: library/http client id16902166 +Ref: 2a016902166 +Ref: library/http client http client HTTPConnection request6902299 +Ref: 54f6902299 +Ref: library/http client http client HTTPConnection getresponse6905081 +Ref: 6e46905081 +Ref: library/http client http client HTTPConnection set_debuglevel6905573 +Ref: 2a026905573 +Ref: library/http client http client HTTPConnection set_tunnel6905974 +Ref: bad6905974 +Ref: library/http client http client HTTPConnection connect6906979 +Ref: 2a036906979 +Ref: library/http client http client HTTPConnection close6907209 +Ref: 2a046907209 +Ref: library/http client http client HTTPConnection blocksize6907288 +Ref: 2a056907288 +Ref: library/http client http client HTTPConnection putrequest6907572 +Ref: 2a066907572 +Ref: library/http client http client HTTPConnection putheader6908121 +Ref: 2a076908121 +Ref: library/http client http client HTTPConnection endheaders6908456 +Ref: 5506908456 +Ref: library/http client http client HTTPConnection send6909825 +Ref: 2a086909825 +Ref: HTTPConnection Objects-Footnote-16910083 +Ref: HTTPConnection Objects-Footnote-26910131 +Node: HTTPResponse Objects6910180 +Ref: library/http client httpresponse-objects6910325 +Ref: 2a096910325 +Ref: library/http client id26910325 +Ref: 2a0a6910325 +Ref: library/http client http client HTTPResponse read6910739 +Ref: 2a0b6910739 +Ref: library/http client http client HTTPResponse readinto6910852 +Ref: a126910852 +Ref: library/http client http client HTTPResponse getheader6911039 +Ref: 2a0c6911039 +Ref: library/http client http client HTTPResponse getheaders6911430 +Ref: 2a0d6911430 +Ref: library/http client http client HTTPResponse fileno6911517 +Ref: 2a0e6911517 +Ref: library/http client http client HTTPResponse msg6911609 +Ref: 29ee6911609 +Ref: library/http client http client HTTPResponse version6911814 +Ref: 2a0f6911814 +Ref: library/http client http client HTTPResponse status6911935 +Ref: 2a106911935 +Ref: library/http client http client HTTPResponse reason6912009 +Ref: 29346912009 +Ref: library/http client http client HTTPResponse debuglevel6912085 +Ref: 2a116912085 +Ref: library/http client http client HTTPResponse closed6912280 +Ref: 2a126912280 +Node: Examples<22>6912361 +Ref: library/http client examples6912503 +Ref: 2a136912503 +Node: HTTPMessage Objects6915419 +Ref: library/http client httpmessage-objects6915532 +Ref: 2a146915532 +Ref: library/http client id36915532 +Ref: 2a156915532 +Node: ftplib — FTP protocol client6915747 +Ref: library/ftplib doc6915929 +Ref: 2a166915929 +Ref: library/ftplib ftplib-ftp-protocol-client6915929 +Ref: 2a176915929 +Ref: library/ftplib module-ftplib6915929 +Ref: 846915929 +Ref: library/ftplib ftplib FTP6917464 +Ref: 2f06917464 +Ref: library/ftplib ftplib FTP_TLS6918960 +Ref: a036918960 +Ref: library/ftplib ftplib error_reply6921249 +Ref: 2a196921249 +Ref: library/ftplib ftplib error_temp6921366 +Ref: 2a1a6921366 +Ref: library/ftplib ftplib error_perm6921529 +Ref: 2a1b6921529 +Ref: library/ftplib ftplib error_proto6921692 +Ref: 2a1c6921692 +Ref: library/ftplib ftplib all_errors6921923 +Ref: 2a1d6921923 +Ref: ftplib — FTP protocol client-Footnote-16922585 +Ref: ftplib — FTP protocol client-Footnote-26922650 +Ref: ftplib — FTP protocol client-Footnote-36922698 +Node: FTP Objects6922747 +Ref: library/ftplib ftp-objects6922849 +Ref: 2a1e6922849 +Ref: library/ftplib id16922849 +Ref: 2a1f6922849 +Ref: library/ftplib ftplib FTP set_debuglevel6923186 +Ref: 2a206923186 +Ref: library/ftplib ftplib FTP connect6923650 +Ref: 2a216923650 +Ref: library/ftplib ftplib FTP getwelcome6924661 +Ref: 2a226924661 +Ref: library/ftplib ftplib FTP login6924891 +Ref: 2a236924891 +Ref: library/ftplib ftplib FTP abort6925604 +Ref: 2a246925604 +Ref: library/ftplib ftplib FTP sendcmd6925742 +Ref: 2a256925742 +Ref: library/ftplib ftplib FTP voidcmd6925966 +Ref: 2a266925966 +Ref: library/ftplib ftplib FTP retrbinary6926339 +Ref: 2a276926339 +Ref: library/ftplib ftplib FTP retrlines6927019 +Ref: 2a296927019 +Ref: library/ftplib ftplib FTP set_pasv6927641 +Ref: 2a2a6927641 +Ref: library/ftplib ftplib FTP storbinary6927792 +Ref: c9f6927792 +Ref: library/ftplib ftplib FTP storlines6928492 +Ref: 2a2b6928492 +Ref: library/ftplib ftplib FTP transfercmd6928957 +Ref: 2a286928957 +Ref: library/ftplib ftplib FTP ntransfercmd6930207 +Ref: 2a2c6930207 +Ref: library/ftplib ftplib FTP mlsd6930557 +Ref: a056930557 +Ref: library/ftplib ftplib FTP nlst6931249 +Ref: a066931249 +Ref: library/ftplib ftplib FTP dir6931660 +Ref: a076931660 +Ref: library/ftplib ftplib FTP rename6932295 +Ref: 2a2d6932295 +Ref: library/ftplib ftplib FTP delete6932394 +Ref: 2a2e6932394 +Ref: library/ftplib ftplib FTP cwd6932657 +Ref: 2a2f6932657 +Ref: library/ftplib ftplib FTP mkd6932736 +Ref: 2a306932736 +Ref: library/ftplib ftplib FTP pwd6932812 +Ref: 2a316932812 +Ref: library/ftplib ftplib FTP rmd6932902 +Ref: 2a326932902 +Ref: library/ftplib ftplib FTP size6932991 +Ref: 2a336932991 +Ref: library/ftplib ftplib FTP quit6933317 +Ref: 2a346933317 +Ref: library/ftplib ftplib FTP close6933727 +Ref: 2a356933727 +Ref: FTP Objects-Footnote-16934187 +Ref: FTP Objects-Footnote-26934235 +Node: FTP_TLS Objects6934284 +Ref: library/ftplib ftp-tls-objects6934386 +Ref: 2a366934386 +Ref: library/ftplib ftplib FTP_TLS ssl_version6934531 +Ref: 2a376934531 +Ref: library/ftplib ftplib FTP_TLS auth6934649 +Ref: 2a386934649 +Ref: library/ftplib ftplib FTP_TLS ccc6935007 +Ref: a046935007 +Ref: library/ftplib ftplib FTP_TLS prot_p6935247 +Ref: 2a186935247 +Ref: library/ftplib ftplib FTP_TLS prot_c6935315 +Ref: 2a396935315 +Node: poplib — POP3 protocol client6935387 +Ref: library/poplib doc6935566 +Ref: 2a3a6935566 +Ref: library/poplib module-poplib6935566 +Ref: d06935566 +Ref: library/poplib poplib-pop3-protocol-client6935566 +Ref: 2a3b6935566 +Ref: library/poplib poplib POP36936681 +Ref: 2a3c6936681 +Ref: library/poplib poplib POP3_SSL6937421 +Ref: bc16937421 +Ref: library/poplib poplib error_proto6939193 +Ref: b176939193 +Ref: poplib — POP3 protocol client-Footnote-16939846 +Ref: poplib — POP3 protocol client-Footnote-26939911 +Ref: poplib — POP3 protocol client-Footnote-36939960 +Ref: poplib — POP3 protocol client-Footnote-46940009 +Ref: poplib — POP3 protocol client-Footnote-56940058 +Node: POP3 Objects6940120 +Ref: library/poplib id16940221 +Ref: 2a3d6940221 +Ref: library/poplib pop3-objects6940221 +Ref: 2a3e6940221 +Ref: library/poplib poplib POP3 set_debuglevel6940455 +Ref: 2a3f6940455 +Ref: library/poplib poplib POP3 getwelcome6940920 +Ref: 2a406940920 +Ref: library/poplib poplib POP3 capa6941011 +Ref: 8926941011 +Ref: library/poplib poplib POP3 user6941200 +Ref: 2a416941200 +Ref: library/poplib poplib POP3 pass_6941318 +Ref: 2a426941318 +Ref: library/poplib poplib POP3 apop6941501 +Ref: 2a436941501 +Ref: library/poplib poplib POP3 rpop6941619 +Ref: 2a446941619 +Ref: library/poplib poplib POP3 stat6941738 +Ref: 2a456941738 +Ref: library/poplib poplib POP3 list6941870 +Ref: 2a466941870 +Ref: library/poplib poplib POP3 retr6942062 +Ref: 2a476942062 +Ref: library/poplib poplib POP3 dele6942225 +Ref: 2a486942225 +Ref: library/poplib poplib POP3 rset6942503 +Ref: 2a496942503 +Ref: library/poplib poplib POP3 noop6942578 +Ref: 2a4a6942578 +Ref: library/poplib poplib POP3 quit6942654 +Ref: b166942654 +Ref: library/poplib poplib POP3 top6942744 +Ref: 2a4b6942744 +Ref: library/poplib poplib POP3 uidl6943281 +Ref: 2a4c6943281 +Ref: library/poplib poplib POP3 utf86943565 +Ref: 7326943565 +Ref: library/poplib poplib POP3 stls6943772 +Ref: 8936943772 +Ref: POP3 Objects-Footnote-16944571 +Ref: POP3 Objects-Footnote-26944620 +Ref: POP3 Objects-Footnote-36944669 +Node: POP3 Example6944718 +Ref: library/poplib id26944819 +Ref: 2a4d6944819 +Ref: library/poplib pop3-example6944819 +Ref: 2a4e6944819 +Node: imaplib — IMAP4 protocol client6945331 +Ref: library/imaplib doc6945512 +Ref: 2a4f6945512 +Ref: library/imaplib imaplib-imap4-protocol-client6945512 +Ref: 2a506945512 +Ref: library/imaplib module-imaplib6945512 +Ref: 986945512 +Ref: library/imaplib imaplib IMAP46946177 +Ref: 60b6946177 +Ref: library/imaplib imaplib IMAP4 error6947110 +Ref: 2a526947110 +Ref: library/imaplib imaplib IMAP4 abort6947253 +Ref: 2a536947253 +Ref: library/imaplib imaplib IMAP4 readonly6947515 +Ref: 2a546947515 +Ref: library/imaplib imaplib IMAP4_SSL6947872 +Ref: a176947872 +Ref: library/imaplib imaplib IMAP4_stream6949593 +Ref: 2a516949593 +Ref: library/imaplib imaplib Internaldate2tuple6949863 +Ref: 2a556949863 +Ref: library/imaplib imaplib Int2AP6950113 +Ref: 2a566950113 +Ref: library/imaplib imaplib ParseFlags6950261 +Ref: 2a576950261 +Ref: library/imaplib imaplib Time2Internaldate6950382 +Ref: 2a586950382 +Ref: imaplib — IMAP4 protocol client-Footnote-16951725 +Ref: imaplib — IMAP4 protocol client-Footnote-26951791 +Ref: imaplib — IMAP4 protocol client-Footnote-36951840 +Node: IMAP4 Objects6951889 +Ref: library/imaplib id16951994 +Ref: 2a596951994 +Ref: library/imaplib imap4-objects6951994 +Ref: 2a5a6951994 +Ref: library/imaplib imaplib IMAP4 append6953498 +Ref: 2a5b6953498 +Ref: library/imaplib imaplib IMAP4 authenticate6953602 +Ref: 2a5c6953602 +Ref: library/imaplib imaplib IMAP4 check6954438 +Ref: 2a5d6954438 +Ref: library/imaplib imaplib IMAP4 close6954502 +Ref: 2a5e6954502 +Ref: library/imaplib imaplib IMAP4 copy6954687 +Ref: 2a5f6954687 +Ref: library/imaplib imaplib IMAP4 create6954799 +Ref: 2a606954799 +Ref: library/imaplib imaplib IMAP4 delete6954877 +Ref: 2a616954877 +Ref: library/imaplib imaplib IMAP4 deleteacl6954955 +Ref: 2a626954955 +Ref: library/imaplib imaplib IMAP4 enable6955065 +Ref: 6ea6955065 +Ref: library/imaplib imaplib IMAP4 expunge6955382 +Ref: 2a636955382 +Ref: library/imaplib imaplib IMAP4 fetch6955630 +Ref: 2a646955630 +Ref: library/imaplib imaplib IMAP4 getacl6955910 +Ref: 2a656955910 +Ref: library/imaplib imaplib IMAP4 getannotation6956064 +Ref: 2a666956064 +Ref: library/imaplib imaplib IMAP4 getquota6956265 +Ref: 2a676956265 +Ref: library/imaplib imaplib IMAP4 getquotaroot6956439 +Ref: 2a686956439 +Ref: library/imaplib imaplib IMAP4 list6956627 +Ref: 2a696956627 +Ref: library/imaplib imaplib IMAP4 login6956896 +Ref: 2a6a6956896 +Ref: library/imaplib imaplib IMAP4 login_cram_md56957029 +Ref: 2a6b6957029 +Ref: library/imaplib imaplib IMAP4 logout6957289 +Ref: 2a6c6957289 +Ref: library/imaplib imaplib IMAP4 lsub6957486 +Ref: 2a6d6957486 +Ref: library/imaplib imaplib IMAP4 myrights6957776 +Ref: 2a6e6957776 +Ref: library/imaplib imaplib IMAP4 namespace6957895 +Ref: 2a6f6957895 +Ref: library/imaplib imaplib IMAP4 noop6957984 +Ref: 2a706957984 +Ref: library/imaplib imaplib IMAP4 open6958044 +Ref: 2a716958044 +Ref: library/imaplib imaplib IMAP4 partial6958563 +Ref: 2a766958563 +Ref: library/imaplib imaplib IMAP4 proxyauth6958740 +Ref: 2a776958740 +Ref: library/imaplib imaplib IMAP4 read6958892 +Ref: 2a726958892 +Ref: library/imaplib imaplib IMAP4 readline6959008 +Ref: 2a736959008 +Ref: library/imaplib imaplib IMAP4 recent6959120 +Ref: 2a786959120 +Ref: library/imaplib imaplib IMAP4 rename6959275 +Ref: 2a796959275 +Ref: library/imaplib imaplib IMAP4 response6959383 +Ref: 2a7a6959383 +Ref: library/imaplib imaplib IMAP4 search6959541 +Ref: 2a7b6959541 +Ref: library/imaplib imaplib IMAP4 select6960216 +Ref: 2a7c6960216 +Ref: library/imaplib imaplib IMAP4 send6960511 +Ref: 2a746960511 +Ref: library/imaplib imaplib IMAP4 setacl6960724 +Ref: 2a7d6960724 +Ref: library/imaplib imaplib IMAP4 setannotation6960887 +Ref: 2a7e6960887 +Ref: library/imaplib imaplib IMAP4 setquota6961076 +Ref: 2a7f6961076 +Ref: library/imaplib imaplib IMAP4 shutdown6961250 +Ref: 2a756961250 +Ref: library/imaplib imaplib IMAP4 socket6961437 +Ref: 2a806961437 +Ref: library/imaplib imaplib IMAP4 sort6961523 +Ref: 2a816961523 +Ref: library/imaplib imaplib IMAP4 starttls6962467 +Ref: baa6962467 +Ref: library/imaplib imaplib IMAP4 status6962994 +Ref: 2a826962994 +Ref: library/imaplib imaplib IMAP4 store6963090 +Ref: 2a836963090 +Ref: library/imaplib imaplib IMAP4 subscribe6964297 +Ref: 2a846964297 +Ref: library/imaplib imaplib IMAP4 thread6964368 +Ref: 2a856964368 +Ref: library/imaplib imaplib IMAP4 uid6965495 +Ref: 2a866965495 +Ref: library/imaplib imaplib IMAP4 unsubscribe6965813 +Ref: 2a876965813 +Ref: library/imaplib imaplib IMAP4 xatom6965890 +Ref: 2a886965890 +Ref: library/imaplib imaplib IMAP4 PROTOCOL_VERSION6966094 +Ref: 2a896966094 +Ref: library/imaplib imaplib IMAP4 debug6966229 +Ref: 2a8a6966229 +Ref: library/imaplib imaplib IMAP4 utf8_enabled6966432 +Ref: 6eb6966432 +Ref: IMAP4 Objects-Footnote-16966710 +Ref: IMAP4 Objects-Footnote-26966759 +Ref: IMAP4 Objects-Footnote-36966808 +Ref: IMAP4 Objects-Footnote-46966857 +Ref: IMAP4 Objects-Footnote-56966906 +Ref: IMAP4 Objects-Footnote-66966955 +Node: IMAP4 Example6967004 +Ref: library/imaplib id26967109 +Ref: 2a8b6967109 +Ref: library/imaplib imap4-example6967109 +Ref: 2a8c6967109 +Node: nntplib — NNTP protocol client6967597 +Ref: library/nntplib doc6967779 +Ref: 2a8d6967779 +Ref: library/nntplib module-nntplib6967779 +Ref: c06967779 +Ref: library/nntplib nntplib-nntp-protocol-client6967779 +Ref: 2a8e6967779 +Ref: library/nntplib nntplib NNTP6969631 +Ref: a2c6969631 +Ref: library/nntplib nntplib NNTP_SSL6971617 +Ref: ba56971617 +Ref: library/nntplib nntplib NNTPError6972978 +Ref: 2a906972978 +Ref: library/nntplib nntplib NNTPError response6973224 +Ref: 2a916973224 +Ref: library/nntplib nntplib NNTPReplyError6973345 +Ref: 2a926973345 +Ref: library/nntplib nntplib NNTPTemporaryError6973466 +Ref: 2a936973466 +Ref: library/nntplib nntplib NNTPPermanentError6973594 +Ref: 2a8f6973594 +Ref: library/nntplib nntplib NNTPProtocolError6973722 +Ref: 2a946973722 +Ref: library/nntplib nntplib NNTPDataError6973886 +Ref: 2a956973886 +Ref: nntplib — NNTP protocol client-Footnote-16974100 +Ref: nntplib — NNTP protocol client-Footnote-26974166 +Ref: nntplib — NNTP protocol client-Footnote-36974215 +Ref: nntplib — NNTP protocol client-Footnote-46974263 +Ref: nntplib — NNTP protocol client-Footnote-56974312 +Node: NNTP Objects6974361 +Ref: library/nntplib id16974471 +Ref: 2a966974471 +Ref: library/nntplib nntp-objects6974471 +Ref: 2a976974471 +Node: Attributes6974679 +Ref: library/nntplib attributes6974757 +Ref: 2a986974757 +Ref: library/nntplib nntplib NNTP nntp_version6974800 +Ref: 2a996974800 +Ref: library/nntplib nntplib NNTP nntp_implementation6975066 +Ref: 2a9a6975066 +Ref: Attributes-Footnote-16975303 +Node: Methods<3>6975352 +Ref: library/nntplib methods6975430 +Ref: 2a9b6975430 +Ref: library/nntplib nntplib NNTP quit6976321 +Ref: 2a9c6976321 +Ref: library/nntplib nntplib NNTP getwelcome6976504 +Ref: 2a9d6976504 +Ref: library/nntplib nntplib NNTP getcapabilities6976735 +Ref: 2a9e6976735 +Ref: library/nntplib nntplib NNTP login6977196 +Ref: 2a9f6977196 +Ref: library/nntplib nntplib NNTP starttls6977814 +Ref: ba66977814 +Ref: library/nntplib nntplib NNTP newgroups6978614 +Ref: 2aa06978614 +Ref: library/nntplib nntplib NNTP newnews6979281 +Ref: 2aa16979281 +Ref: library/nntplib nntplib NNTP list6979652 +Ref: 2aa26979652 +Ref: library/nntplib nntplib NNTP descriptions6980988 +Ref: 2aa36980988 +Ref: library/nntplib nntplib NNTP description6981586 +Ref: 2aa46981586 +Ref: library/nntplib nntplib NNTP group6981954 +Ref: 2aa56981954 +Ref: library/nntplib nntplib NNTP over6982388 +Ref: 2aa66982388 +Ref: library/nntplib nntplib NNTP help6984550 +Ref: 2aa86984550 +Ref: library/nntplib nntplib NNTP stat6984704 +Ref: 2aa96984704 +Ref: library/nntplib nntplib NNTP next6985373 +Ref: 2aaa6985373 +Ref: library/nntplib nntplib NNTP last6985467 +Ref: 2aab6985467 +Ref: library/nntplib nntplib NNTP article6985561 +Ref: 2aac6985561 +Ref: library/nntplib nntplib NNTP head6986669 +Ref: 2aad6986669 +Ref: library/nntplib nntplib NNTP body6986903 +Ref: 2aae6986903 +Ref: library/nntplib nntplib NNTP post6987137 +Ref: 2aaf6987137 +Ref: library/nntplib nntplib NNTP ihave6987763 +Ref: 2ab06987763 +Ref: library/nntplib nntplib NNTP date6988039 +Ref: 2ab16988039 +Ref: library/nntplib nntplib NNTP slave6988207 +Ref: 2ab26988207 +Ref: library/nntplib nntplib NNTP set_debuglevel6988303 +Ref: 2ab36988303 +Ref: library/nntplib nntplib NNTP xhdr6988939 +Ref: 2ab46988939 +Ref: library/nntplib nntplib NNTP xover6989877 +Ref: 2ab56989877 +Ref: library/nntplib nntplib NNTP xpath6990253 +Ref: 2ab66990253 +Ref: Methods<3>-Footnote-16990603 +Ref: Methods<3>-Footnote-26990652 +Ref: Methods<3>-Footnote-36990701 +Ref: Methods<3>-Footnote-46990750 +Node: Utility functions<2>6990799 +Ref: library/nntplib utility-functions6990909 +Ref: 2ab76990909 +Ref: library/nntplib nntplib decode_header6991023 +Ref: 2aa76991023 +Node: smtplib — SMTP protocol client6991614 +Ref: library/smtplib doc6991784 +Ref: 2ab86991784 +Ref: library/smtplib module-smtplib6991784 +Ref: ed6991784 +Ref: library/smtplib smtplib-smtp-protocol-client6991784 +Ref: 2ab96991784 +Ref: library/smtplib smtplib SMTP6992273 +Ref: a816992273 +Ref: library/smtplib smtplib SMTP_SSL6994675 +Ref: a826994675 +Ref: library/smtplib smtplib LMTP6996366 +Ref: a836996366 +Ref: library/smtplib smtplib SMTPException6997198 +Ref: 8b76997198 +Ref: library/smtplib smtplib SMTPServerDisconnected6997454 +Ref: 2abd6997454 +Ref: library/smtplib smtplib SMTPResponseException6997682 +Ref: 2abe6997682 +Ref: library/smtplib smtplib SMTPSenderRefused6998047 +Ref: 2abf6998047 +Ref: library/smtplib smtplib SMTPRecipientsRefused6998288 +Ref: 2ac06998288 +Ref: library/smtplib smtplib SMTPDataError6998561 +Ref: 2ac16998561 +Ref: library/smtplib smtplib SMTPConnectError6998657 +Ref: 2abb6998657 +Ref: library/smtplib smtplib SMTPHeloError6998778 +Ref: 2ac26998778 +Ref: library/smtplib smtplib SMTPNotSupportedError6998865 +Ref: 2ac36998865 +Ref: library/smtplib smtplib SMTPAuthenticationError6999007 +Ref: 2ac46999007 +Ref: smtplib — SMTP protocol client-Footnote-16999730 +Ref: smtplib — SMTP protocol client-Footnote-26999796 +Ref: smtplib — SMTP protocol client-Footnote-36999844 +Ref: smtplib — SMTP protocol client-Footnote-46999893 +Ref: smtplib — SMTP protocol client-Footnote-56999942 +Ref: smtplib — SMTP protocol client-Footnote-66999990 +Node: SMTP Objects7000039 +Ref: library/smtplib id17000141 +Ref: 2ac57000141 +Ref: library/smtplib smtp-objects7000141 +Ref: 2ac67000141 +Ref: library/smtplib smtplib SMTP set_debuglevel7000245 +Ref: 7437000245 +Ref: library/smtplib smtplib SMTP docmd7000596 +Ref: 2ac77000596 +Ref: library/smtplib smtplib SMTP connect7001246 +Ref: 2aba7001246 +Ref: library/smtplib smtplib SMTP helo7001931 +Ref: 2ac87001931 +Ref: library/smtplib smtplib SMTP ehlo7002391 +Ref: 2ac97002391 +Ref: library/smtplib smtplib SMTP ehlo_or_helo_if_needed7003284 +Ref: 2acb7003284 +Ref: library/smtplib smtplib SMTP has_extn7003626 +Ref: 2aca7003626 +Ref: library/smtplib smtplib SMTP verify7003821 +Ref: 2acc7003821 +Ref: library/smtplib smtplib SMTP login7004240 +Ref: 2acd7004240 +Ref: library/smtplib smtplib SMTP auth7005825 +Ref: 7427005825 +Ref: library/smtplib smtplib SMTP starttls7008068 +Ref: a847008068 +Ref: library/smtplib smtplib SMTP sendmail7009778 +Ref: 7447009778 +Ref: library/smtplib smtplib SMTP send_message7013259 +Ref: 7457013259 +Ref: library/smtplib smtplib SMTP quit7015327 +Ref: 2abc7015327 +Ref: SMTP Objects-Footnote-17015790 +Ref: SMTP Objects-Footnote-27015838 +Ref: SMTP Objects-Footnote-37015887 +Ref: SMTP Objects-Footnote-47015936 +Ref: SMTP Objects-Footnote-57015984 +Ref: SMTP Objects-Footnote-67016032 +Node: SMTP Example7016081 +Ref: library/smtplib id27016183 +Ref: 2ace7016183 +Ref: library/smtplib smtp-example7016183 +Ref: 2acf7016183 +Ref: SMTP Example-Footnote-17017616 +Node: smtpd — SMTP Server7017664 +Ref: library/smtpd doc7017829 +Ref: 2ad07017829 +Ref: library/smtpd module-smtpd7017829 +Ref: ec7017829 +Ref: library/smtpd smtpd-smtp-server7017829 +Ref: 2ad17017829 +Ref: smtpd — SMTP Server-Footnote-17018832 +Ref: smtpd — SMTP Server-Footnote-27018896 +Ref: smtpd — SMTP Server-Footnote-37018936 +Ref: smtpd — SMTP Server-Footnote-47018985 +Ref: smtpd — SMTP Server-Footnote-57019034 +Node: SMTPServer Objects7019083 +Ref: library/smtpd smtpserver-objects7019191 +Ref: 2ad27019191 +Ref: library/smtpd smtpd SMTPServer7019250 +Ref: 6027019250 +Ref: library/smtpd smtpd SMTPServer process_message7021107 +Ref: 6037021107 +Ref: library/smtpd smtpd SMTPServer channel_class7022983 +Ref: 2ad37022983 +Ref: SMTPServer Objects-Footnote-17023594 +Ref: SMTPServer Objects-Footnote-27023643 +Ref: SMTPServer Objects-Footnote-37023692 +Ref: SMTPServer Objects-Footnote-47023741 +Node: DebuggingServer Objects7023790 +Ref: library/smtpd debuggingserver-objects7023924 +Ref: 2ad47023924 +Ref: library/smtpd smtpd DebuggingServer7023993 +Ref: 2ad57023993 +Node: PureProxy Objects7024195 +Ref: library/smtpd pureproxy-objects7024331 +Ref: 2ad67024331 +Ref: library/smtpd smtpd PureProxy7024388 +Ref: 2ad77024388 +Node: MailmanProxy Objects7024679 +Ref: library/smtpd mailmanproxy-objects7024811 +Ref: 2ad87024811 +Ref: library/smtpd smtpd MailmanProxy7024874 +Ref: 2ad97024874 +Node: SMTPChannel Objects7025284 +Ref: library/smtpd smtpchannel-objects7025390 +Ref: 2ada7025390 +Ref: library/smtpd smtpd SMTPChannel7025451 +Ref: 6017025451 +Ref: library/smtpd smtpd SMTPChannel smtp_server7026946 +Ref: 2adb7026946 +Ref: library/smtpd smtpd SMTPChannel conn7027050 +Ref: 2adc7027050 +Ref: library/smtpd smtpd SMTPChannel addr7027137 +Ref: 2add7027137 +Ref: library/smtpd smtpd SMTPChannel received_lines7027272 +Ref: 2ade7027272 +Ref: library/smtpd smtpd SMTPChannel smtp_state7027493 +Ref: 2adf7027493 +Ref: library/smtpd smtpd SMTPChannel seen_greeting7027703 +Ref: 2ae07027703 +Ref: library/smtpd smtpd SMTPChannel mailfrom7027836 +Ref: 2ae17027836 +Ref: library/smtpd smtpd SMTPChannel rcpttos7027982 +Ref: 2ae27027982 +Ref: library/smtpd smtpd SMTPChannel received_data7028137 +Ref: 2ae37028137 +Ref: library/smtpd smtpd SMTPChannel fqdn7028346 +Ref: 2ae47028346 +Ref: library/smtpd smtpd SMTPChannel peer7028490 +Ref: 2ae57028490 +Ref: SMTPChannel Objects-Footnote-17030946 +Ref: SMTPChannel Objects-Footnote-27030995 +Node: telnetlib — Telnet client7031044 +Ref: library/telnetlib doc7031220 +Ref: 2ae67031220 +Ref: library/telnetlib module-telnetlib7031220 +Ref: 1027031220 +Ref: library/telnetlib telnetlib-telnet-client7031220 +Ref: 2ae77031220 +Ref: library/telnetlib telnetlib Telnet7032222 +Ref: 5977032222 +Ref: telnetlib — Telnet client-Footnote-17033772 +Ref: telnetlib — Telnet client-Footnote-27033840 +Ref: telnetlib — Telnet client-Footnote-37033888 +Node: Telnet Objects7033936 +Ref: library/telnetlib id17034037 +Ref: 2aea7034037 +Ref: library/telnetlib telnet-objects7034037 +Ref: 2aeb7034037 +Ref: library/telnetlib telnetlib Telnet read_until7034146 +Ref: 2aec7034146 +Ref: library/telnetlib telnetlib Telnet read_all7034501 +Ref: 2aed7034501 +Ref: library/telnetlib telnetlib Telnet read_some7034604 +Ref: 2aee7034604 +Ref: library/telnetlib telnetlib Telnet read_very_eager7034782 +Ref: 2aef7034782 +Ref: library/telnetlib telnetlib Telnet read_eager7035093 +Ref: 2af07035093 +Ref: library/telnetlib telnetlib Telnet read_lazy7035367 +Ref: 2af17035367 +Ref: library/telnetlib telnetlib Telnet read_very_lazy7035658 +Ref: 2af27035658 +Ref: library/telnetlib telnetlib Telnet read_sb_data7035931 +Ref: 2af37035931 +Ref: library/telnetlib telnetlib Telnet open7036164 +Ref: 2ae87036164 +Ref: library/telnetlib telnetlib Telnet msg7036730 +Ref: 2af47036730 +Ref: library/telnetlib telnetlib Telnet set_debuglevel7036956 +Ref: 2af57036956 +Ref: library/telnetlib telnetlib Telnet close7037128 +Ref: 2ae97037128 +Ref: library/telnetlib telnetlib Telnet get_socket7037185 +Ref: 2af67037185 +Ref: library/telnetlib telnetlib Telnet fileno7037267 +Ref: 2af77037267 +Ref: library/telnetlib telnetlib Telnet write7037368 +Ref: 2af87037368 +Ref: library/telnetlib telnetlib Telnet interact7037837 +Ref: 2af97037837 +Ref: library/telnetlib telnetlib Telnet mt_interact7037933 +Ref: 2afa7037933 +Ref: library/telnetlib telnetlib Telnet expect7038023 +Ref: 2afb7038023 +Ref: library/telnetlib telnetlib Telnet set_option_negotiation_callback7039029 +Ref: 2afc7039029 +Node: Telnet Example7039344 +Ref: library/telnetlib id27039445 +Ref: 2afd7039445 +Ref: library/telnetlib telnet-example7039445 +Ref: 2afe7039445 +Node: uuid — UUID objects according to RFC 41227040001 +Ref: library/uuid doc7040204 +Ref: 2aff7040204 +Ref: library/uuid module-uuid7040204 +Ref: 1247040204 +Ref: library/uuid uuid-uuid-objects-according-to-rfc-41227040204 +Ref: 2b007040204 +Ref: library/uuid uuid SafeUUID7041330 +Ref: 2b047041330 +Ref: library/uuid uuid SafeUUID safe7041382 +Ref: 2b057041382 +Ref: library/uuid uuid SafeUUID unsafe7041499 +Ref: 2b067041499 +Ref: library/uuid uuid SafeUUID unknown7041596 +Ref: 2b077041596 +Ref: library/uuid uuid UUID7041737 +Ref: 2607041737 +Ref: library/uuid uuid UUID bytes7043698 +Ref: 2b097043698 +Ref: library/uuid uuid UUID bytes_le7043827 +Ref: 2b0a7043827 +Ref: library/uuid uuid UUID fields7043979 +Ref: 2b0b7043979 +Ref: library/uuid uuid UUID hex7045539 +Ref: 2b0c7045539 +Ref: library/uuid uuid UUID int7045617 +Ref: 2b087045617 +Ref: library/uuid uuid UUID urn7045679 +Ref: 2b0d7045679 +Ref: library/uuid uuid UUID variant7045757 +Ref: 2b0e7045757 +Ref: library/uuid uuid UUID version7046021 +Ref: 2b137046021 +Ref: library/uuid uuid UUID is_safe7046159 +Ref: 4117046159 +Ref: library/uuid uuid getnode7046414 +Ref: 4127046414 +Ref: library/uuid uuid uuid17047384 +Ref: 4137047384 +Ref: library/uuid uuid uuid37047736 +Ref: 2b017047736 +Ref: library/uuid uuid uuid47047905 +Ref: 2b027047905 +Ref: library/uuid uuid uuid57047964 +Ref: 2b037047964 +Ref: library/uuid uuid NAMESPACE_DNS7048266 +Ref: 2b147048266 +Ref: library/uuid uuid NAMESPACE_URL7048393 +Ref: 2b157048393 +Ref: library/uuid uuid NAMESPACE_OID7048491 +Ref: 2b167048491 +Ref: library/uuid uuid NAMESPACE_X5007048594 +Ref: 2b177048594 +Ref: library/uuid uuid RESERVED_NCS7048852 +Ref: 2b0f7048852 +Ref: library/uuid uuid RFC_41227048919 +Ref: 2b107048919 +Ref: library/uuid uuid RESERVED_MICROSOFT7048998 +Ref: 2b117048998 +Ref: library/uuid uuid RESERVED_FUTURE7049077 +Ref: 2b127049077 +Ref: uuid — UUID objects according to RFC 4122-Footnote-17049455 +Ref: uuid — UUID objects according to RFC 4122-Footnote-27049518 +Ref: uuid — UUID objects according to RFC 4122-Footnote-37049567 +Ref: uuid — UUID objects according to RFC 4122-Footnote-47049616 +Ref: uuid — UUID objects according to RFC 4122-Footnote-57049665 +Ref: uuid — UUID objects according to RFC 4122-Footnote-67049714 +Ref: uuid — UUID objects according to RFC 4122-Footnote-77049763 +Node: Example<13>7049812 +Ref: library/uuid example7049903 +Ref: 2b187049903 +Ref: library/uuid uuid-example7049903 +Ref: 2b197049903 +Node: socketserver — A framework for network servers7051162 +Ref: library/socketserver doc7051366 +Ref: 2b1a7051366 +Ref: library/socketserver module-socketserver7051366 +Ref: f07051366 +Ref: library/socketserver socketserver-a-framework-for-network-servers7051366 +Ref: 2b1b7051366 +Ref: library/socketserver socketserver TCPServer7051725 +Ref: 2b1c7051725 +Ref: library/socketserver socketserver UDPServer7052196 +Ref: 2b1f7052196 +Ref: library/socketserver socketserver UnixStreamServer7052499 +Ref: 2b207052499 +Ref: library/socketserver socketserver UnixDatagramServer7052612 +Ref: 2b217052612 +Ref: socketserver — A framework for network servers-Footnote-17054965 +Node: Server Creation Notes7055036 +Ref: library/socketserver server-creation-notes7055168 +Ref: 2b267055168 +Ref: library/socketserver socketserver ForkingMixIn7056002 +Ref: 3d57056002 +Ref: library/socketserver socketserver ThreadingMixIn7056039 +Ref: 3d67056039 +Ref: library/socketserver socketserver ForkingTCPServer7057529 +Ref: 2b287057529 +Ref: library/socketserver socketserver ForkingUDPServer7057570 +Ref: 2b297057570 +Ref: library/socketserver socketserver ThreadingTCPServer7057611 +Ref: 2b2a7057611 +Ref: library/socketserver socketserver ThreadingUDPServer7057654 +Ref: 2b277057654 +Node: Server Objects<2>7059945 +Ref: library/socketserver server-objects7060109 +Ref: 2b2c7060109 +Ref: library/socketserver socketserver BaseServer7060160 +Ref: a897060160 +Ref: library/socketserver socketserver BaseServer fileno7060559 +Ref: 2b2f7060559 +Ref: library/socketserver socketserver BaseServer handle_request7060832 +Ref: 2b247060832 +Ref: library/socketserver socketserver BaseServer serve_forever7061434 +Ref: a8b7061434 +Ref: library/socketserver socketserver BaseServer service_actions7062085 +Ref: a8a7062085 +Ref: library/socketserver socketserver BaseServer shutdown7062385 +Ref: 2b357062385 +Ref: library/socketserver socketserver BaseServer server_close7062665 +Ref: 2b257062665 +Ref: library/socketserver socketserver BaseServer address_family7062751 +Ref: 2b367062751 +Ref: library/socketserver socketserver BaseServer RequestHandlerClass7062962 +Ref: 2b2e7062962 +Ref: library/socketserver socketserver BaseServer server_address7063120 +Ref: 2b2d7063120 +Ref: library/socketserver socketserver BaseServer socket7063543 +Ref: 2b377063543 +Ref: library/socketserver socketserver BaseServer allow_reuse_address7063729 +Ref: 2b387063729 +Ref: library/socketserver socketserver BaseServer request_queue_size7063943 +Ref: 2b397063943 +Ref: library/socketserver socketserver BaseServer socket_type7064434 +Ref: 2b3a7064434 +Ref: library/socketserver socketserver BaseServer timeout7064625 +Ref: 2b337064625 +Ref: library/socketserver socketserver BaseServer finish_request7065131 +Ref: 2b3b7065131 +Ref: library/socketserver socketserver BaseServer get_request7065348 +Ref: 2b307065348 +Ref: library/socketserver socketserver BaseServer handle_error7065578 +Ref: 5fd7065578 +Ref: library/socketserver socketserver BaseServer handle_timeout7066030 +Ref: 2b347066030 +Ref: library/socketserver socketserver BaseServer process_request7066469 +Ref: 2b327066469 +Ref: library/socketserver socketserver BaseServer server_activate7066840 +Ref: 2b1e7066840 +Ref: library/socketserver socketserver BaseServer server_bind7067087 +Ref: 2b1d7067087 +Ref: library/socketserver socketserver BaseServer verify_request7067241 +Ref: 2b317067241 +Node: Request Handler Objects7067826 +Ref: library/socketserver request-handler-objects7067981 +Ref: 2b3c7067981 +Ref: library/socketserver socketserver BaseRequestHandler7068050 +Ref: 2b227068050 +Ref: library/socketserver socketserver BaseRequestHandler setup7068404 +Ref: 2b3d7068404 +Ref: library/socketserver socketserver BaseRequestHandler handle7068599 +Ref: 2b237068599 +Ref: library/socketserver socketserver BaseRequestHandler finish7069303 +Ref: 2b3e7069303 +Ref: library/socketserver socketserver StreamRequestHandler7069582 +Ref: 5877069582 +Ref: library/socketserver socketserver DatagramRequestHandler7069627 +Ref: 2b2b7069627 +Node: Examples<23>7070391 +Ref: library/socketserver examples7070520 +Ref: 2b3f7070520 +Node: socketserver TCPServer Example7070725 +Ref: library/socketserver socketserver-tcpserver-example7070843 +Ref: 2b407070843 +Node: socketserver UDPServer Example7073999 +Ref: library/socketserver socketserver-udpserver-example7074145 +Ref: 2b417074145 +Node: Asynchronous Mixins7075762 +Ref: library/socketserver asynchronous-mixins7075869 +Ref: 2b427075869 +Node: http server — HTTP servers7078215 +Ref: library/http server doc7078414 +Ref: 2b437078414 +Ref: library/http server http-server-http-servers7078414 +Ref: 2b447078414 +Ref: library/http server module-http server7078414 +Ref: 977078414 +Ref: library/http server http server HTTPServer7079224 +Ref: 29097079224 +Ref: library/http server http server ThreadingHTTPServer7079579 +Ref: 3967079579 +Ref: library/http server http server BaseHTTPRequestHandler7080127 +Ref: a0e7080127 +Ref: library/http server http server BaseHTTPRequestHandler client_address7081117 +Ref: 2b457081117 +Ref: library/http server http server BaseHTTPRequestHandler server7081259 +Ref: 2b467081259 +Ref: library/http server http server BaseHTTPRequestHandler close_connection7081328 +Ref: 2b477081328 +Ref: library/http server http server BaseHTTPRequestHandler requestline7081563 +Ref: 2b497081563 +Ref: library/http server http server BaseHTTPRequestHandler command7081878 +Ref: 2b4a7081878 +Ref: library/http server http server BaseHTTPRequestHandler path7081982 +Ref: 2b4b7081982 +Ref: library/http server http server BaseHTTPRequestHandler request_version7082046 +Ref: 2b4c7082046 +Ref: library/http server http server BaseHTTPRequestHandler headers7082182 +Ref: 29ef7082182 +Ref: library/http server http server BaseHTTPRequestHandler rfile7082609 +Ref: 2b4e7082609 +Ref: library/http server http server BaseHTTPRequestHandler wfile7082761 +Ref: 2b4f7082761 +Ref: library/http server http server BaseHTTPRequestHandler server_version7083209 +Ref: 2b507083209 +Ref: library/http server http server BaseHTTPRequestHandler sys_version7083496 +Ref: 2b517083496 +Ref: library/http server http server BaseHTTPRequestHandler error_message_format7083751 +Ref: 2b537083751 +Ref: library/http server http server BaseHTTPRequestHandler error_content_type7084112 +Ref: 2b547084112 +Ref: library/http server http server BaseHTTPRequestHandler protocol_version7084292 +Ref: 2b557084292 +Ref: library/http server http server BaseHTTPRequestHandler MessageClass7084755 +Ref: 2b4d7084755 +Ref: library/http server http server BaseHTTPRequestHandler responses7084987 +Ref: 29d97084987 +Ref: library/http server http server BaseHTTPRequestHandler handle7085552 +Ref: 2b587085552 +Ref: library/http server http server BaseHTTPRequestHandler handle_one_request7085852 +Ref: 2b487085852 +Ref: library/http server http server BaseHTTPRequestHandler handle_expect_1007086050 +Ref: 2b597086050 +Ref: library/http server http server BaseHTTPRequestHandler send_error7086556 +Ref: 85c7086556 +Ref: library/http server http server BaseHTTPRequestHandler send_response7087684 +Ref: 2b5a7087684 +Ref: library/http server http server BaseHTTPRequestHandler send_header7088500 +Ref: 2b567088500 +Ref: library/http server http server BaseHTTPRequestHandler send_response_only7089080 +Ref: 2b577089080 +Ref: library/http server http server BaseHTTPRequestHandler end_headers7089500 +Ref: a0f7089500 +Ref: library/http server http server BaseHTTPRequestHandler flush_headers7089799 +Ref: a107089799 +Ref: library/http server http server BaseHTTPRequestHandler log_request7089971 +Ref: 2b5c7089971 +Ref: library/http server http server BaseHTTPRequestHandler log_error7090268 +Ref: 2b5d7090268 +Ref: library/http server http server BaseHTTPRequestHandler log_message7090513 +Ref: 2b5e7090513 +Ref: library/http server http server BaseHTTPRequestHandler version_string7091002 +Ref: 2b527091002 +Ref: library/http server http server BaseHTTPRequestHandler date_time_string7091216 +Ref: 2b5b7091216 +Ref: library/http server http server BaseHTTPRequestHandler log_date_time_string7091604 +Ref: 2b5f7091604 +Ref: library/http server http server BaseHTTPRequestHandler address_string7091715 +Ref: 2b607091715 +Ref: library/http server http server SimpleHTTPRequestHandler7091958 +Ref: 3957091958 +Ref: library/http server http server SimpleHTTPRequestHandler server_version7092515 +Ref: 2b637092515 +Ref: library/http server http server SimpleHTTPRequestHandler extensions_map7092676 +Ref: 2b647092676 +Ref: library/http server http server SimpleHTTPRequestHandler directory7092998 +Ref: 2b657092998 +Ref: library/http server http server SimpleHTTPRequestHandler do_HEAD7093214 +Ref: 2b627093214 +Ref: library/http server http server SimpleHTTPRequestHandler do_GET7093502 +Ref: 2b617093502 +Ref: library/http server http-server-cli7095929 +Ref: 85d7095929 +Ref: library/http server http server CGIHTTPRequestHandler7096940 +Ref: 2b667096940 +Ref: library/http server http server CGIHTTPRequestHandler cgi_directories7098131 +Ref: 2b677098131 +Ref: library/http server http server CGIHTTPRequestHandler do_POST7098377 +Ref: 2b687098377 +Ref: http server — HTTP servers-Footnote-17098954 +Ref: http server — HTTP servers-Footnote-27099024 +Node: http cookies — HTTP state management7099073 +Ref: library/http cookies doc7099275 +Ref: 2b697099275 +Ref: library/http cookies http-cookies-http-state-management7099275 +Ref: 2b6a7099275 +Ref: library/http cookies module-http cookies7099275 +Ref: 967099275 +Ref: library/http cookies http cookies CookieError7100637 +Ref: 2b6b7100637 +Ref: library/http cookies http cookies BaseCookie7100804 +Ref: 2b6c7100804 +Ref: library/http cookies http cookies SimpleCookie7101193 +Ref: 2b6e7101193 +Ref: http cookies — HTTP state management-Footnote-17102023 +Ref: http cookies — HTTP state management-Footnote-27102094 +Ref: http cookies — HTTP state management-Footnote-37102143 +Ref: http cookies — HTTP state management-Footnote-47102192 +Ref: http cookies — HTTP state management-Footnote-57102241 +Node: Cookie Objects7102290 +Ref: library/http cookies cookie-objects7102402 +Ref: 2b6f7102402 +Ref: library/http cookies id17102402 +Ref: 2b707102402 +Ref: library/http cookies http cookies BaseCookie value_decode7102453 +Ref: 2b717102453 +Ref: library/http cookies http cookies BaseCookie value_encode7102725 +Ref: 2b727102725 +Ref: library/http cookies http cookies BaseCookie output7103177 +Ref: 2b737103177 +Ref: library/http cookies http cookies BaseCookie js_output7103545 +Ref: 2b747103545 +Ref: library/http cookies http cookies BaseCookie load7103824 +Ref: 2b6d7103824 +Node: Morsel Objects7104106 +Ref: library/http cookies id27104238 +Ref: 2b757104238 +Ref: library/http cookies morsel-objects7104238 +Ref: 2b767104238 +Ref: library/http cookies http cookies Morsel7104289 +Ref: 4907104289 +Ref: library/http cookies http cookies Morsel value7105641 +Ref: 48e7105641 +Ref: library/http cookies http cookies Morsel coded_value7105701 +Ref: 48f7105701 +Ref: library/http cookies http cookies Morsel key7105807 +Ref: 48d7105807 +Ref: library/http cookies http cookies Morsel set7105864 +Ref: 4917105864 +Ref: library/http cookies http cookies Morsel isReservedKey7105973 +Ref: 2b777105973 +Ref: library/http cookies http cookies Morsel output7106084 +Ref: 2b787106084 +Ref: library/http cookies http cookies Morsel js_output7106424 +Ref: 2b797106424 +Ref: library/http cookies http cookies Morsel OutputString7106698 +Ref: 2b7a7106698 +Ref: library/http cookies http cookies Morsel update7106910 +Ref: 7c47106910 +Ref: library/http cookies http cookies Morsel copy7107210 +Ref: 7c37107210 +Ref: library/http cookies http cookies Morsel setdefault7107365 +Ref: 2b7b7107365 +Ref: Morsel Objects-Footnote-17107580 +Ref: Morsel Objects-Footnote-27107629 +Ref: Morsel Objects-Footnote-37107678 +Ref: Morsel Objects-Footnote-47107727 +Node: Example<14>7107776 +Ref: library/http cookies cookie-example7107885 +Ref: 2b7c7107885 +Ref: library/http cookies example7107885 +Ref: 2b7d7107885 +Node: http cookiejar — Cookie handling for HTTP clients7109537 +Ref: library/http cookiejar doc7109754 +Ref: 2b7e7109754 +Ref: library/http cookiejar http-cookiejar-cookie-handling-for-http-clients7109754 +Ref: 2b7f7109754 +Ref: library/http cookiejar module-http cookiejar7109754 +Ref: 957109754 +Ref: library/http cookiejar http cookiejar LoadError7111322 +Ref: 2b807111322 +Ref: library/http cookiejar http cookiejar CookieJar7111697 +Ref: 297c7111697 +Ref: library/http cookiejar http cookiejar FileCookieJar7112173 +Ref: 2b817112173 +Ref: library/http cookiejar http cookiejar CookiePolicy7112917 +Ref: 2b827112917 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy7113077 +Ref: 2b867113077 +Ref: library/http cookiejar http cookiejar Cookie7114901 +Ref: 2b887114901 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-17116601 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-27116674 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-37116723 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-47116772 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-57116821 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-67116870 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-77116919 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-87116968 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-97117017 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-107117066 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-117117116 +Ref: http cookiejar — Cookie handling for HTTP clients-Footnote-127117166 +Node: CookieJar and FileCookieJar Objects7117216 +Ref: library/http cookiejar cookie-jar-objects7117407 +Ref: 2b897117407 +Ref: library/http cookiejar cookiejar-and-filecookiejar-objects7117407 +Ref: 2b8a7117407 +Ref: library/http cookiejar http cookiejar CookieJar add_cookie_header7117684 +Ref: 2b8b7117684 +Ref: library/http cookiejar http cookiejar CookieJar extract_cookies7118602 +Ref: 2b8c7118602 +Ref: library/http cookiejar http cookiejar CookieJar set_policy7119837 +Ref: 2b8e7119837 +Ref: library/http cookiejar http cookiejar CookieJar make_cookies7119941 +Ref: 2b8f7119941 +Ref: library/http cookiejar http cookiejar CookieJar set_cookie_if_ok7120228 +Ref: 2b907120228 +Ref: library/http cookiejar http cookiejar CookieJar set_cookie7120353 +Ref: 2b917120353 +Ref: library/http cookiejar http cookiejar CookieJar clear7120502 +Ref: 2b927120502 +Ref: library/http cookiejar http cookiejar CookieJar clear_session_cookies7121007 +Ref: 2b937121007 +Ref: library/http cookiejar http cookiejar FileCookieJar save7121650 +Ref: 2b947121650 +Ref: library/http cookiejar http cookiejar FileCookieJar load7122503 +Ref: 2b837122503 +Ref: library/http cookiejar http cookiejar FileCookieJar revert7123070 +Ref: 2b847123070 +Ref: library/http cookiejar http cookiejar FileCookieJar filename7123469 +Ref: 2b957123469 +Ref: library/http cookiejar http cookiejar FileCookieJar delayload7123606 +Ref: 2b967123606 +Node: FileCookieJar subclasses and co-operation with web browsers7124027 +Ref: library/http cookiejar file-cookie-jar-classes7124247 +Ref: 2b857124247 +Ref: library/http cookiejar filecookiejar-subclasses-and-co-operation-with-web-browsers7124247 +Ref: 2b977124247 +Ref: library/http cookiejar http cookiejar MozillaCookieJar7124475 +Ref: 2b987124475 +Ref: library/http cookiejar http cookiejar LWPCookieJar7125272 +Ref: 2b997125272 +Ref: FileCookieJar subclasses and co-operation with web browsers-Footnote-17125750 +Node: CookiePolicy Objects7125799 +Ref: library/http cookiejar cookie-policy-objects7126011 +Ref: 2b9a7126011 +Ref: library/http cookiejar cookiepolicy-objects7126011 +Ref: 2b9b7126011 +Ref: library/http cookiejar http cookiejar CookiePolicy set_ok7126164 +Ref: 2b8d7126164 +Ref: library/http cookiejar http cookiejar CookiePolicy return_ok7126490 +Ref: 2b9c7126490 +Ref: library/http cookiejar http cookiejar CookiePolicy domain_return_ok7126819 +Ref: 2b9d7126819 +Ref: library/http cookiejar http cookiejar CookiePolicy path_return_ok7128197 +Ref: 2b9e7128197 +Ref: library/http cookiejar http cookiejar CookiePolicy netscape7128649 +Ref: 2b9f7128649 +Ref: library/http cookiejar http cookiejar CookiePolicy rfc29657128722 +Ref: 2ba07128722 +Ref: library/http cookiejar http cookiejar CookiePolicy hide_cookie27128797 +Ref: 2ba17128797 +Ref: CookiePolicy Objects-Footnote-17129348 +Ref: CookiePolicy Objects-Footnote-27129397 +Node: DefaultCookiePolicy Objects7129446 +Ref: library/http cookiejar default-cookie-policy-objects7129616 +Ref: 2ba27129616 +Ref: library/http cookiejar defaultcookiepolicy-objects7129616 +Ref: 2ba37129616 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy blocked_domains7131934 +Ref: 2ba47131934 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy set_blocked_domains7132045 +Ref: 2ba57132045 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy is_blocked7132159 +Ref: 2ba67132159 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy allowed_domains7132301 +Ref: 2ba77132301 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy set_allowed_domains7132437 +Ref: 2ba87132437 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy is_not_allowed7132571 +Ref: 2ba97132571 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy rfc2109_as_netscape7132905 +Ref: 2b877132905 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy strict_domain7133492 +Ref: 2baa7133492 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy strict_rfc2965_unverifiable7133796 +Ref: 2bab7133796 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy strict_ns_unverifiable7134174 +Ref: 2bac7134174 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy strict_ns_domain7134323 +Ref: 2bad7134323 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy strict_ns_set_initial_dollar7134504 +Ref: 2bae7134504 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy strict_ns_set_path7134659 +Ref: 2baf7134659 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy DomainStrictNoDots7134983 +Ref: 2bb07134983 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy DomainStrictNonDomain7135228 +Ref: 2bb17135228 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy DomainRFC2965Match7135576 +Ref: 2bb27135576 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy DomainLiberal7135813 +Ref: 2bb37135813 +Ref: library/http cookiejar http cookiejar DefaultCookiePolicy DomainStrict7135961 +Ref: 2bb47135961 +Ref: DefaultCookiePolicy Objects-Footnote-17136114 +Ref: DefaultCookiePolicy Objects-Footnote-27136163 +Ref: DefaultCookiePolicy Objects-Footnote-37136212 +Ref: DefaultCookiePolicy Objects-Footnote-47136261 +Ref: DefaultCookiePolicy Objects-Footnote-57136310 +Ref: DefaultCookiePolicy Objects-Footnote-67136359 +Ref: DefaultCookiePolicy Objects-Footnote-77136408 +Node: Cookie Objects<2>7136457 +Ref: library/http cookiejar cookie-objects7136619 +Ref: 2bb57136619 +Ref: library/http cookiejar http cookiejar Cookie version7137388 +Ref: 2bb67137388 +Ref: library/http cookiejar http cookiejar Cookie name7137746 +Ref: 2bb77137746 +Ref: library/http cookiejar http cookiejar Cookie value7137804 +Ref: 2bb87137804 +Ref: library/http cookiejar http cookiejar Cookie port7137884 +Ref: 2bb97137884 +Ref: library/http cookiejar http cookiejar Cookie path7138023 +Ref: 2bba7138023 +Ref: library/http cookiejar http cookiejar Cookie secure7138118 +Ref: 2bbb7138118 +Ref: library/http cookiejar http cookiejar Cookie expires7138225 +Ref: 2bbc7138225 +Ref: library/http cookiejar http cookiejar Cookie discard7138378 +Ref: 2bbe7138378 +Ref: library/http cookiejar http cookiejar Cookie comment7138455 +Ref: 2bbf7138455 +Ref: library/http cookiejar http cookiejar Cookie comment_url7138588 +Ref: 2bc07138588 +Ref: library/http cookiejar http cookiejar Cookie rfc21097138735 +Ref: 2bc17138735 +Ref: library/http cookiejar http cookiejar Cookie port_specified7139145 +Ref: 2bc27139145 +Ref: library/http cookiejar http cookiejar Cookie domain_specified7139324 +Ref: 2bc37139324 +Ref: library/http cookiejar http cookiejar Cookie domain_initial_dot7139433 +Ref: 2bc47139433 +Ref: library/http cookiejar http cookiejar Cookie has_nonstandard_attr7139689 +Ref: 2bc57139689 +Ref: library/http cookiejar http cookiejar Cookie get_nonstandard_attr7139803 +Ref: 2bc67139803 +Ref: library/http cookiejar http cookiejar Cookie set_nonstandard_attr7139965 +Ref: 2bc77139965 +Ref: library/http cookiejar http cookiejar Cookie is_expired7140137 +Ref: 2bbd7140137 +Ref: Cookie Objects<2>-Footnote-17140430 +Ref: Cookie Objects<2>-Footnote-27140479 +Ref: Cookie Objects<2>-Footnote-37140528 +Ref: Cookie Objects<2>-Footnote-47140577 +Node: Examples<24>7140626 +Ref: library/http cookiejar examples7140752 +Ref: 2bc87140752 +Ref: Examples<24>-Footnote-17142205 +Node: xmlrpc — XMLRPC server and client modules7142254 +Ref: library/xmlrpc doc7142472 +Ref: 2bc97142472 +Ref: library/xmlrpc xmlrpc-xmlrpc-server-and-client-modules7142472 +Ref: 2bca7142472 +Node: xmlrpc client — XML-RPC client access7142974 +Ref: library/xmlrpc client doc7143180 +Ref: 2bcb7143180 +Ref: library/xmlrpc client module-xmlrpc client7143180 +Ref: 13f7143180 +Ref: library/xmlrpc client xmlrpc-client-xml-rpc-client-access7143180 +Ref: 2bcc7143180 +Ref: library/xmlrpc client xmlrpc client ServerProxy7144113 +Ref: 2557144113 +Ref: xmlrpc client — XML-RPC client access-Footnote-17151932 +Ref: xmlrpc client — XML-RPC client access-Footnote-27152004 +Ref: xmlrpc client — XML-RPC client access-Footnote-37152098 +Ref: xmlrpc client — XML-RPC client access-Footnote-47152157 +Ref: xmlrpc client — XML-RPC client access-Footnote-57152216 +Ref: xmlrpc client — XML-RPC client access-Footnote-67152262 +Node: ServerProxy Objects7152311 +Ref: library/xmlrpc client id17152431 +Ref: 2bd17152431 +Ref: library/xmlrpc client serverproxy-objects7152431 +Ref: 2bd27152431 +Ref: library/xmlrpc client xmlrpc client ServerProxy system listMethods7153087 +Ref: 2bd37153087 +Ref: library/xmlrpc client xmlrpc client ServerProxy system methodSignature7153250 +Ref: 2bd47153250 +Ref: library/xmlrpc client xmlrpc client ServerProxy system methodHelp7154274 +Ref: 2bd57154274 +Node: DateTime Objects7155349 +Ref: library/xmlrpc client datetime-objects7155492 +Ref: 2bd67155492 +Ref: library/xmlrpc client id27155492 +Ref: 2bd77155492 +Ref: library/xmlrpc client xmlrpc client DateTime7155547 +Ref: 2bcd7155547 +Ref: library/xmlrpc client xmlrpc client DateTime decode7155856 +Ref: 2bd87155856 +Ref: library/xmlrpc client xmlrpc client DateTime encode7155953 +Ref: 2bd97155953 +Node: Binary Objects7157025 +Ref: library/xmlrpc client binary-objects7157162 +Ref: 2bda7157162 +Ref: library/xmlrpc client id37157162 +Ref: 2bdb7157162 +Ref: library/xmlrpc client xmlrpc client Binary7157213 +Ref: 2bce7157213 +Ref: library/xmlrpc client xmlrpc client Binary data7157430 +Ref: 2bdc7157430 +Ref: library/xmlrpc client xmlrpc client Binary decode7157736 +Ref: 2bdd7157736 +Ref: library/xmlrpc client xmlrpc client Binary encode7157875 +Ref: 2bde7157875 +Ref: Binary Objects-Footnote-17159118 +Node: Fault Objects7159179 +Ref: library/xmlrpc client fault-objects7159321 +Ref: 2bdf7159321 +Ref: library/xmlrpc client id47159321 +Ref: 2be07159321 +Ref: library/xmlrpc client xmlrpc client Fault7159370 +Ref: 2bcf7159370 +Ref: library/xmlrpc client xmlrpc client Fault faultCode7159537 +Ref: 2be17159537 +Ref: library/xmlrpc client xmlrpc client Fault faultString7159615 +Ref: 2be27159615 +Node: ProtocolError Objects7160577 +Ref: library/xmlrpc client protocol-error-objects7160722 +Ref: 2be37160722 +Ref: library/xmlrpc client protocolerror-objects7160722 +Ref: 2be47160722 +Ref: library/xmlrpc client xmlrpc client ProtocolError7160787 +Ref: 2bd07160787 +Ref: library/xmlrpc client xmlrpc client ProtocolError url7161064 +Ref: 2be57161064 +Ref: library/xmlrpc client xmlrpc client ProtocolError errcode7161141 +Ref: 2be67161141 +Ref: library/xmlrpc client xmlrpc client ProtocolError errmsg7161197 +Ref: 2be77161197 +Ref: library/xmlrpc client xmlrpc client ProtocolError headers7161276 +Ref: 2be87161276 +Node: MultiCall Objects7162016 +Ref: library/xmlrpc client multicall-objects7162169 +Ref: 2be97162169 +Ref: library/xmlrpc client xmlrpc client MultiCall7162352 +Ref: 2bea7162352 +Ref: MultiCall Objects-Footnote-17164057 +Node: Convenience Functions7164240 +Ref: library/xmlrpc client convenience-functions7164395 +Ref: 2beb7164395 +Ref: library/xmlrpc client xmlrpc client dumps7164460 +Ref: 2bec7164460 +Ref: library/xmlrpc client xmlrpc client loads7165173 +Ref: 2bed7165173 +Node: Example of Client Usage7166041 +Ref: library/xmlrpc client example-of-client-usage7166213 +Ref: 2bee7166213 +Ref: library/xmlrpc client xmlrpc-client-example7166213 +Ref: 2bef7166213 +Node: Example of Client and Server Usage7167520 +Ref: library/xmlrpc client example-of-client-and-server-usage7167662 +Ref: 2bf07167662 +Node: xmlrpc server — Basic XML-RPC servers7167798 +Ref: library/xmlrpc server doc7168005 +Ref: 2bf27168005 +Ref: library/xmlrpc server module-xmlrpc server7168005 +Ref: 1407168005 +Ref: library/xmlrpc server xmlrpc-server-basic-xml-rpc-servers7168005 +Ref: 2bf37168005 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCServer7168687 +Ref: 2bf47168687 +Ref: library/xmlrpc server xmlrpc server CGIXMLRPCRequestHandler7170211 +Ref: 2bf57170211 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCRequestHandler7170862 +Ref: 2bf67170862 +Ref: xmlrpc server — Basic XML-RPC servers-Footnote-17171350 +Node: SimpleXMLRPCServer Objects7171422 +Ref: library/xmlrpc server simple-xmlrpc-servers7171556 +Ref: 2bf77171556 +Ref: library/xmlrpc server simplexmlrpcserver-objects7171556 +Ref: 2bf87171556 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCServer register_function7171792 +Ref: 2bf97171792 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCServer register_instance7172535 +Ref: 2bfa7172535 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCServer register_introspection_functions7174151 +Ref: 2bfb7174151 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCServer register_multicall_functions7174360 +Ref: 2bfc7174360 +Ref: library/xmlrpc server xmlrpc server SimpleXMLRPCRequestHandler rpc_paths7174489 +Ref: 2bfd7174489 +Node: SimpleXMLRPCServer Example7174909 +Ref: library/xmlrpc server id17174998 +Ref: 2bfe7174998 +Ref: library/xmlrpc server simplexmlrpcserver-example7174998 +Ref: 2bf17174998 +Node: CGIXMLRPCRequestHandler7179515 +Ref: library/xmlrpc server cgixmlrpcrequesthandler7179683 +Ref: 2bff7179683 +Ref: library/xmlrpc server xmlrpc server CGIXMLRPCRequestHandler register_function7179867 +Ref: 2c007179867 +Ref: library/xmlrpc server xmlrpc server CGIXMLRPCRequestHandler register_instance7180615 +Ref: 2c017180615 +Ref: library/xmlrpc server xmlrpc server CGIXMLRPCRequestHandler register_introspection_functions7181500 +Ref: 2c027181500 +Ref: library/xmlrpc server xmlrpc server CGIXMLRPCRequestHandler register_multicall_functions7181713 +Ref: 2c037181713 +Ref: library/xmlrpc server xmlrpc server CGIXMLRPCRequestHandler handle_request7181852 +Ref: 2c047181852 +Node: Documenting XMLRPC server7182439 +Ref: library/xmlrpc server documenting-xmlrpc-server7182604 +Ref: 2c057182604 +Ref: library/xmlrpc server xmlrpc server DocXMLRPCServer7182932 +Ref: 2c067182932 +Ref: library/xmlrpc server xmlrpc server DocCGIXMLRPCRequestHandler7183410 +Ref: 2c077183410 +Ref: library/xmlrpc server xmlrpc server DocXMLRPCRequestHandler7183545 +Ref: 2c087183545 +Node: DocXMLRPCServer Objects7183864 +Ref: library/xmlrpc server doc-xmlrpc-servers7184032 +Ref: 2c097184032 +Ref: library/xmlrpc server docxmlrpcserver-objects7184032 +Ref: 2c0a7184032 +Ref: library/xmlrpc server xmlrpc server DocXMLRPCServer set_server_title7184469 +Ref: 2c0b7184469 +Ref: library/xmlrpc server xmlrpc server DocXMLRPCServer set_server_name7184659 +Ref: 2c0c7184659 +Ref: library/xmlrpc server xmlrpc server DocXMLRPCServer set_server_documentation7184881 +Ref: 2c0d7184881 +Node: DocCGIXMLRPCRequestHandler7185137 +Ref: library/xmlrpc server doccgixmlrpcrequesthandler7185271 +Ref: 2c0e7185271 +Ref: library/xmlrpc server xmlrpc server DocCGIXMLRPCRequestHandler set_server_title7185722 +Ref: 2c0f7185722 +Ref: library/xmlrpc server xmlrpc server DocCGIXMLRPCRequestHandler set_server_name7185923 +Ref: 2c107185923 +Ref: library/xmlrpc server xmlrpc server DocCGIXMLRPCRequestHandler set_server_documentation7186156 +Ref: 2c117186156 +Node: ipaddress — IPv4/IPv6 manipulation library7186423 +Ref: library/ipaddress doc7186582 +Ref: 2c127186582 +Ref: library/ipaddress ipaddress-ipv4-ipv6-manipulation-library7186582 +Ref: 2c137186582 +Ref: library/ipaddress module-ipaddress7186582 +Ref: a27186582 +Ref: ipaddress — IPv4/IPv6 manipulation library-Footnote-17187605 +Node: Convenience factory functions7187673 +Ref: library/ipaddress convenience-factory-functions7187804 +Ref: 2c157187804 +Ref: library/ipaddress ipaddress ip_address7188007 +Ref: 2c167188007 +Ref: library/ipaddress ipaddress ip_network7188588 +Ref: 2c197188588 +Ref: library/ipaddress ipaddress ip_interface7189297 +Ref: 2c1a7189297 +Node: IP Addresses7190150 +Ref: library/ipaddress ip-addresses7190312 +Ref: 2c1d7190312 +Node: Address objects7190455 +Ref: library/ipaddress address-objects7190562 +Ref: 2c1e7190562 +Ref: library/ipaddress ipaddress IPv4Address7191005 +Ref: 2c177191005 +Ref: library/ipaddress ipaddress IPv4Address version7192069 +Ref: 2c207192069 +Ref: library/ipaddress ipaddress IPv4Address max_prefixlen7192177 +Ref: 2c217192177 +Ref: library/ipaddress ipaddress IPv4Address compressed7192515 +Ref: 2c227192515 +Ref: library/ipaddress ipaddress IPv4Address exploded7192547 +Ref: 2c237192547 +Ref: library/ipaddress ipaddress IPv4Address packed7193036 +Ref: 2c247193036 +Ref: library/ipaddress ipaddress IPv4Address reverse_pointer7193273 +Ref: 2c257193273 +Ref: library/ipaddress ipaddress IPv4Address is_multicast7193818 +Ref: 2c267193818 +Ref: library/ipaddress ipaddress IPv4Address is_private7193985 +Ref: 2c277193985 +Ref: library/ipaddress ipaddress IPv4Address is_global7194200 +Ref: 86c7194200 +Ref: library/ipaddress ipaddress IPv4Address is_unspecified7194444 +Ref: 2c287194444 +Ref: library/ipaddress ipaddress IPv4Address is_reserved7194598 +Ref: 2c297194598 +Ref: library/ipaddress ipaddress IPv4Address is_loopback7194696 +Ref: 2c2a7194696 +Ref: library/ipaddress ipaddress IPv4Address is_link_local7194848 +Ref: 2c2b7194848 +Ref: library/ipaddress ipaddress IPv6Address7194983 +Ref: 2c187194983 +Ref: library/ipaddress ipaddress IPv6Address compressed7195911 +Ref: 2c2c7195911 +Ref: library/ipaddress ipaddress IPv6Address exploded7196220 +Ref: 2c2d7196220 +Ref: library/ipaddress ipaddress IPv6Address packed7196497 +Ref: 2c2e7196497 +Ref: library/ipaddress ipaddress IPv6Address reverse_pointer7196525 +Ref: 2c2f7196525 +Ref: library/ipaddress ipaddress IPv6Address version7196562 +Ref: 2c307196562 +Ref: library/ipaddress ipaddress IPv6Address max_prefixlen7196591 +Ref: 2c317196591 +Ref: library/ipaddress ipaddress IPv6Address is_multicast7196626 +Ref: 2c327196626 +Ref: library/ipaddress ipaddress IPv6Address is_private7196660 +Ref: 2c337196660 +Ref: library/ipaddress ipaddress IPv6Address is_global7196692 +Ref: 2c347196692 +Ref: library/ipaddress ipaddress IPv6Address is_unspecified7196723 +Ref: 2c357196723 +Ref: library/ipaddress ipaddress IPv6Address is_reserved7196759 +Ref: 2c367196759 +Ref: library/ipaddress ipaddress IPv6Address is_loopback7196792 +Ref: 2c377196792 +Ref: library/ipaddress ipaddress IPv6Address is_link_local7196825 +Ref: 2c387196825 +Ref: library/ipaddress ipaddress IPv6Address is_site_local7196901 +Ref: 2c397196901 +Ref: library/ipaddress ipaddress IPv6Address ipv4_mapped7197247 +Ref: 2c3a7197247 +Ref: library/ipaddress ipaddress IPv6Address sixtofour7197519 +Ref: 2c3b7197519 +Ref: library/ipaddress ipaddress IPv6Address teredo7197809 +Ref: 2c3c7197809 +Ref: Address objects-Footnote-17198159 +Ref: Address objects-Footnote-27198208 +Ref: Address objects-Footnote-37198257 +Ref: Address objects-Footnote-47198359 +Ref: Address objects-Footnote-57198461 +Ref: Address objects-Footnote-67198563 +Ref: Address objects-Footnote-77198665 +Ref: Address objects-Footnote-87198714 +Ref: Address objects-Footnote-97198763 +Ref: Address objects-Footnote-107198812 +Ref: Address objects-Footnote-117198862 +Ref: Address objects-Footnote-127198912 +Ref: Address objects-Footnote-137198962 +Ref: Address objects-Footnote-147199012 +Ref: Address objects-Footnote-157199062 +Ref: Address objects-Footnote-167199112 +Node: Conversion to Strings and Integers7199162 +Ref: library/ipaddress conversion-to-strings-and-integers7199290 +Ref: 2c3d7199290 +Node: Operators<3>7199832 +Ref: library/ipaddress operators7199936 +Ref: 2c3e7199936 +Node: Comparison operators7200201 +Ref: library/ipaddress comparison-operators7200299 +Ref: 2c3f7200299 +Node: Arithmetic operators7200672 +Ref: library/ipaddress arithmetic-operators7200770 +Ref: 2c407200770 +Node: IP Network definitions7201273 +Ref: library/ipaddress ip-network-definitions7201423 +Ref: 2c417201423 +Node: Prefix net mask and host mask7202133 +Ref: library/ipaddress prefix-net-mask-and-host-mask7202245 +Ref: 2c427202245 +Node: Network objects7202915 +Ref: library/ipaddress network-objects7203048 +Ref: 2c437203048 +Ref: library/ipaddress ipaddress IPv4Network7203507 +Ref: 3a07203507 +Ref: library/ipaddress ipaddress IPv4Network version7205930 +Ref: 2c457205930 +Ref: library/ipaddress ipaddress IPv4Network max_prefixlen7205959 +Ref: 2c467205959 +Ref: library/ipaddress ipaddress IPv4Network is_multicast7206094 +Ref: 2c477206094 +Ref: library/ipaddress ipaddress IPv4Network is_private7206128 +Ref: 2c487206128 +Ref: library/ipaddress ipaddress IPv4Network is_unspecified7206160 +Ref: 2c497206160 +Ref: library/ipaddress ipaddress IPv4Network is_reserved7206196 +Ref: 2c4a7206196 +Ref: library/ipaddress ipaddress IPv4Network is_loopback7206229 +Ref: 2c4b7206229 +Ref: library/ipaddress ipaddress IPv4Network is_link_local7206262 +Ref: 2c4c7206262 +Ref: library/ipaddress ipaddress IPv4Network network_address7206454 +Ref: 2c4d7206454 +Ref: library/ipaddress ipaddress IPv4Network broadcast_address7206628 +Ref: 2c4e7206628 +Ref: library/ipaddress ipaddress IPv4Network hostmask7206825 +Ref: 2c4f7206825 +Ref: library/ipaddress ipaddress IPv4Network netmask7206920 +Ref: 2c507206920 +Ref: library/ipaddress ipaddress IPv4Network with_prefixlen7207013 +Ref: 2c517207013 +Ref: library/ipaddress ipaddress IPv4Network compressed7207049 +Ref: 2c527207049 +Ref: library/ipaddress ipaddress IPv4Network exploded7207081 +Ref: 2c537207081 +Ref: library/ipaddress ipaddress IPv4Network with_netmask7207382 +Ref: 2c547207382 +Ref: library/ipaddress ipaddress IPv4Network with_hostmask7207513 +Ref: 2c557207513 +Ref: library/ipaddress ipaddress IPv4Network num_addresses7207646 +Ref: 2c567207646 +Ref: library/ipaddress ipaddress IPv4Network prefixlen7207738 +Ref: 2c577207738 +Ref: library/ipaddress ipaddress IPv4Network hosts7207819 +Ref: 2c587207819 +Ref: library/ipaddress ipaddress IPv4Network overlaps7208621 +Ref: 2c597208621 +Ref: library/ipaddress ipaddress IPv4Network address_exclude7208793 +Ref: 2c5a7208793 +Ref: library/ipaddress ipaddress IPv4Network subnets7209399 +Ref: 7977209399 +Ref: library/ipaddress ipaddress IPv4Network supernet7210947 +Ref: 7987210947 +Ref: library/ipaddress ipaddress IPv4Network subnet_of7211753 +Ref: 2c5b7211753 +Ref: library/ipaddress ipaddress IPv4Network supernet_of7212050 +Ref: 2c5c7212050 +Ref: library/ipaddress ipaddress IPv4Network compare_networks7212353 +Ref: 2c5d7212353 +Ref: library/ipaddress ipaddress IPv6Network7213051 +Ref: 39f7213051 +Ref: library/ipaddress ipaddress IPv6Network version7214855 +Ref: 2c5e7214855 +Ref: library/ipaddress ipaddress IPv6Network max_prefixlen7214884 +Ref: 2c5f7214884 +Ref: library/ipaddress ipaddress IPv6Network is_multicast7214919 +Ref: 2c607214919 +Ref: library/ipaddress ipaddress IPv6Network is_private7214953 +Ref: 2c617214953 +Ref: library/ipaddress ipaddress IPv6Network is_unspecified7214985 +Ref: 2c627214985 +Ref: library/ipaddress ipaddress IPv6Network is_reserved7215021 +Ref: 2c637215021 +Ref: library/ipaddress ipaddress IPv6Network is_loopback7215054 +Ref: 2c647215054 +Ref: library/ipaddress ipaddress IPv6Network is_link_local7215087 +Ref: 2c657215087 +Ref: library/ipaddress ipaddress IPv6Network network_address7215122 +Ref: 2c667215122 +Ref: library/ipaddress ipaddress IPv6Network broadcast_address7215159 +Ref: 2c677215159 +Ref: library/ipaddress ipaddress IPv6Network hostmask7215198 +Ref: 2c687215198 +Ref: library/ipaddress ipaddress IPv6Network netmask7215228 +Ref: 2c697215228 +Ref: library/ipaddress ipaddress IPv6Network with_prefixlen7215257 +Ref: 2c6a7215257 +Ref: library/ipaddress ipaddress IPv6Network compressed7215293 +Ref: 2c6b7215293 +Ref: library/ipaddress ipaddress IPv6Network exploded7215325 +Ref: 2c6c7215325 +Ref: library/ipaddress ipaddress IPv6Network with_netmask7215355 +Ref: 2c6d7215355 +Ref: library/ipaddress ipaddress IPv6Network with_hostmask7215389 +Ref: 2c6e7215389 +Ref: library/ipaddress ipaddress IPv6Network num_addresses7215424 +Ref: 2c6f7215424 +Ref: library/ipaddress ipaddress IPv6Network prefixlen7215459 +Ref: 2c707215459 +Ref: library/ipaddress ipaddress IPv6Network hosts7215490 +Ref: 2c717215490 +Ref: library/ipaddress ipaddress IPv6Network overlaps7215846 +Ref: 2c727215846 +Ref: library/ipaddress ipaddress IPv6Network address_exclude7215881 +Ref: 2c737215881 +Ref: library/ipaddress ipaddress IPv6Network subnets7215925 +Ref: 2c747215925 +Ref: library/ipaddress ipaddress IPv6Network supernet7215987 +Ref: 2c757215987 +Ref: library/ipaddress ipaddress IPv6Network subnet_of7216050 +Ref: 2c767216050 +Ref: library/ipaddress ipaddress IPv6Network supernet_of7216086 +Ref: 2c777216086 +Ref: library/ipaddress ipaddress IPv6Network compare_networks7216124 +Ref: 2c787216124 +Ref: library/ipaddress ipaddress IPv6Network is_site_local7216266 +Ref: 2c797216266 +Node: Operators<4>7216443 +Ref: library/ipaddress id17216538 +Ref: 2c7a7216538 +Node: Logical operators7216844 +Ref: library/ipaddress logical-operators7216931 +Ref: 2c7b7216931 +Node: Iteration<2>7217136 +Ref: library/ipaddress iteration7217267 +Ref: 2c7c7217267 +Node: Networks as containers of addresses7218095 +Ref: library/ipaddress networks-as-containers-of-addresses7218200 +Ref: 2c7d7218200 +Node: Interface objects7218658 +Ref: library/ipaddress interface-objects7218824 +Ref: 2c7e7218824 +Ref: library/ipaddress ipaddress IPv4Interface7218973 +Ref: 2c1b7218973 +Ref: library/ipaddress ipaddress IPv4Interface ip7219390 +Ref: 2c7f7219390 +Ref: library/ipaddress ipaddress IPv4Interface network7219637 +Ref: 2c807219637 +Ref: library/ipaddress ipaddress IPv4Interface with_prefixlen7219894 +Ref: 2c817219894 +Ref: library/ipaddress ipaddress IPv4Interface with_netmask7220162 +Ref: 2c827220162 +Ref: library/ipaddress ipaddress IPv4Interface with_hostmask7220435 +Ref: 2c837220435 +Ref: library/ipaddress ipaddress IPv6Interface7220707 +Ref: 2c1c7220707 +Ref: library/ipaddress ipaddress IPv6Interface ip7221124 +Ref: 2c847221124 +Ref: library/ipaddress ipaddress IPv6Interface network7221148 +Ref: 2c857221148 +Ref: library/ipaddress ipaddress IPv6Interface with_prefixlen7221177 +Ref: 2c867221177 +Ref: library/ipaddress ipaddress IPv6Interface with_netmask7221213 +Ref: 2c877221213 +Ref: library/ipaddress ipaddress IPv6Interface with_hostmask7221247 +Ref: 2c887221247 +Node: Operators<5>7221421 +Ref: library/ipaddress id27221487 +Ref: 2c897221487 +Node: Logical operators<2>7221749 +Ref: library/ipaddress id37221818 +Ref: 2c8a7221818 +Node: Other Module Level Functions7222475 +Ref: library/ipaddress other-module-level-functions7222636 +Ref: 2c8b7222636 +Ref: library/ipaddress ipaddress v4_int_to_packed7222781 +Ref: 2c8c7222781 +Ref: library/ipaddress ipaddress v6_int_to_packed7223255 +Ref: 2c8d7223255 +Ref: library/ipaddress ipaddress summarize_address_range7223564 +Ref: 7997223564 +Ref: library/ipaddress ipaddress collapse_addresses7224447 +Ref: 79a7224447 +Ref: library/ipaddress ipaddress get_mixed_type_key7225015 +Ref: 2c8e7225015 +Node: Custom Exceptions7225614 +Ref: library/ipaddress custom-exceptions7225749 +Ref: 2c8f7225749 +Ref: library/ipaddress ipaddress AddressValueError7225920 +Ref: 2c1f7225920 +Ref: library/ipaddress ipaddress NetmaskValueError7226023 +Ref: 2c447226023 +Node: Multimedia Services7226127 +Ref: library/mm doc7226278 +Ref: 2c907226278 +Ref: library/mm mmedia7226278 +Ref: 2c917226278 +Ref: library/mm multimedia-services7226278 +Ref: 2c927226278 +Node: audioop — Manipulate raw audio data7226961 +Ref: library/audioop doc7227106 +Ref: 2c937227106 +Ref: library/audioop audioop-manipulate-raw-audio-data7227106 +Ref: 2c947227106 +Ref: library/audioop module-audioop7227106 +Ref: d7227106 +Ref: library/audioop audioop error7228004 +Ref: 2c957228004 +Ref: library/audioop audioop add7228134 +Ref: 2c967228134 +Ref: library/audioop audioop adpcm2lin7228465 +Ref: 2c977228465 +Ref: library/audioop audioop alaw2lin7228784 +Ref: 2c997228784 +Ref: library/audioop audioop avg7229045 +Ref: 2c9a7229045 +Ref: library/audioop audioop avgpp7229149 +Ref: 2c9b7229149 +Ref: library/audioop audioop bias7229355 +Ref: 2c9c7229355 +Ref: library/audioop audioop byteswap7229542 +Ref: 8227229542 +Ref: library/audioop audioop cross7229772 +Ref: 2c9d7229772 +Ref: library/audioop audioop findfactor7229905 +Ref: 2c9e7229905 +Ref: library/audioop audioop findfit7230316 +Ref: 2c9f7230316 +Ref: library/audioop audioop findmax7230926 +Ref: 2ca07230926 +Ref: library/audioop audioop getsample7231283 +Ref: 2ca17231283 +Ref: library/audioop audioop lin2adpcm7231401 +Ref: 2c987231401 +Ref: library/audioop audioop lin2alaw7232140 +Ref: 2ca27232140 +Ref: library/audioop audioop lin2lin7232464 +Ref: 2ca37232464 +Ref: library/audioop audioop lin2ulaw7233092 +Ref: 2ca47233092 +Ref: library/audioop audioop max7233416 +Ref: 2ca57233416 +Ref: library/audioop audioop maxpp7233545 +Ref: 2ca67233545 +Ref: library/audioop audioop minmax7233656 +Ref: 2ca77233656 +Ref: library/audioop audioop mul7233813 +Ref: 2ca87233813 +Ref: library/audioop audioop ratecv7234039 +Ref: 2ca97234039 +Ref: library/audioop audioop reverse7234617 +Ref: 2caa7234617 +Ref: library/audioop audioop rms7234746 +Ref: 2cab7234746 +Ref: library/audioop audioop tomono7234935 +Ref: 2cac7234935 +Ref: library/audioop audioop tostereo7235197 +Ref: 2cad7235197 +Ref: library/audioop audioop ulaw2lin7235512 +Ref: 2cae7235512 +Node: aifc — Read and write AIFF and AIFC files7238247 +Ref: library/aifc doc7238438 +Ref: 2caf7238438 +Ref: library/aifc aifc-read-and-write-aiff-and-aifc-files7238438 +Ref: 2cb07238438 +Ref: library/aifc module-aifc7238438 +Ref: 57238438 +Ref: library/aifc aifc open7239709 +Ref: 45b7239709 +Ref: library/aifc aifc aifc getnchannels7240768 +Ref: 2cb17240768 +Ref: library/aifc aifc aifc getsampwidth7240872 +Ref: 2cb27240872 +Ref: library/aifc aifc aifc getframerate7240960 +Ref: 2cb37240960 +Ref: library/aifc aifc aifc getnframes7241062 +Ref: 2cb47241062 +Ref: library/aifc aifc aifc getcomptype7241147 +Ref: 2cb57241147 +Ref: library/aifc aifc aifc getcompname7241342 +Ref: 2cb67241342 +Ref: library/aifc aifc aifc getparams7241571 +Ref: 81b7241571 +Ref: library/aifc aifc aifc getmarkers7241773 +Ref: 2cb77241773 +Ref: library/aifc aifc aifc getmark7242092 +Ref: 2cb87242092 +Ref: library/aifc aifc aifc readframes7242227 +Ref: 2cb97242227 +Ref: library/aifc aifc aifc rewind7242444 +Ref: 2cba7242444 +Ref: library/aifc aifc aifc setpos7242575 +Ref: 2cbb7242575 +Ref: library/aifc aifc aifc tell7242648 +Ref: 2cbc7242648 +Ref: library/aifc aifc aifc close7242713 +Ref: 81c7242713 +Ref: library/aifc aifc aifc aiff7243265 +Ref: 2cbd7243265 +Ref: library/aifc aifc aifc aifc7243469 +Ref: 2cbe7243469 +Ref: library/aifc aifc aifc setnchannels7243675 +Ref: 2cbf7243675 +Ref: library/aifc aifc aifc setsampwidth7243774 +Ref: 2cc07243774 +Ref: library/aifc aifc aifc setframerate7243863 +Ref: 2cc17243863 +Ref: library/aifc aifc aifc setnframes7243960 +Ref: 2cc27243960 +Ref: library/aifc aifc aifc setcomptype7244174 +Ref: 2cc37244174 +Ref: library/aifc aifc aifc setparams7244661 +Ref: 2cc47244661 +Ref: library/aifc aifc aifc setmark7244999 +Ref: 2cc57244999 +Ref: library/aifc aifc aifc writeframes7245353 +Ref: 81e7245353 +Ref: library/aifc aifc aifc writeframesraw7245597 +Ref: 81d7245597 +Ref: aifc — Read and write AIFF and AIFC files-Footnote-17246065 +Node: sunau — Read and write Sun AU files7246128 +Ref: library/sunau doc7246315 +Ref: 2cc67246315 +Ref: library/sunau module-sunau7246315 +Ref: fa7246315 +Ref: library/sunau sunau-read-and-write-sun-au-files7246315 +Ref: 2cc77246315 +Ref: library/sunau sunau open7247861 +Ref: 4727247861 +Ref: library/sunau sunau openfp7248316 +Ref: 4717248316 +Ref: library/sunau sunau Error7248571 +Ref: 2cc87248571 +Ref: library/sunau sunau AUDIO_FILE_MAGIC7248772 +Ref: 2cc97248772 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_MULAW_87248955 +Ref: 2cca7248955 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_LINEAR_87248999 +Ref: 2ccb7248999 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_LINEAR_167249044 +Ref: 2ccc7249044 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_LINEAR_247249090 +Ref: 2ccd7249090 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_LINEAR_327249136 +Ref: 2cce7249136 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_ALAW_87249182 +Ref: 2ccf7249182 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_FLOAT7249321 +Ref: 2cd07249321 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_DOUBLE7249363 +Ref: 2cd17249363 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_ADPCM_G7217249406 +Ref: 2cd27249406 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_ADPCM_G7227249453 +Ref: 2cd37249453 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_ADPCM_G723_37249500 +Ref: 2cd47249500 +Ref: library/sunau sunau AUDIO_FILE_ENCODING_ADPCM_G723_57249549 +Ref: 2cd57249549 +Ref: sunau — Read and write Sun AU files-Footnote-17249807 +Node: AU_read Objects7249871 +Ref: library/sunau au-read-objects7249985 +Ref: 2cd67249985 +Ref: library/sunau id17249985 +Ref: 2cd77249985 +Ref: library/sunau sunau AU_read close7250123 +Ref: 2cd87250123 +Ref: library/sunau sunau AU_read getnchannels7250259 +Ref: 2cd97250259 +Ref: library/sunau sunau AU_read getsampwidth7250363 +Ref: 2cda7250363 +Ref: library/sunau sunau AU_read getframerate7250437 +Ref: 2cdb7250437 +Ref: library/sunau sunau AU_read getnframes7250508 +Ref: 2cdc7250508 +Ref: library/sunau sunau AU_read getcomptype7250581 +Ref: 2cdd7250581 +Ref: library/sunau sunau AU_read getcompname7250732 +Ref: 2cde7250732 +Ref: library/sunau sunau AU_read getparams7250965 +Ref: 2cdf7250965 +Ref: library/sunau sunau AU_read readframes7251170 +Ref: 2ce07251170 +Ref: library/sunau sunau AU_read rewind7251409 +Ref: 2ce17251409 +Ref: library/sunau sunau AU_read setpos7251641 +Ref: 2ce27251641 +Ref: library/sunau sunau AU_read tell7251807 +Ref: 2ce37251807 +Ref: library/sunau sunau AU_read getmarkers7252091 +Ref: 2ce47252091 +Ref: library/sunau sunau AU_read getmark7252152 +Ref: 2ce57252152 +Node: AU_write Objects7252208 +Ref: library/sunau au-write-objects7252322 +Ref: 2ce67252322 +Ref: library/sunau id27252322 +Ref: 2ce77252322 +Ref: library/sunau sunau AU_write setnchannels7252463 +Ref: 2ce87252463 +Ref: library/sunau sunau AU_write setsampwidth7252536 +Ref: 8de7252536 +Ref: library/sunau sunau AU_write setframerate7252678 +Ref: 2ce97252678 +Ref: library/sunau sunau AU_write setnframes7252743 +Ref: 2cea7252743 +Ref: library/sunau sunau AU_write setcomptype7252882 +Ref: 2ceb7252882 +Ref: library/sunau sunau AU_write setparams7253043 +Ref: 2cec7253043 +Ref: library/sunau sunau AU_write tell7253259 +Ref: 2ced7253259 +Ref: library/sunau sunau AU_write writeframesraw7253446 +Ref: 8df7253446 +Ref: library/sunau sunau AU_write writeframes7253632 +Ref: 8e07253632 +Ref: library/sunau sunau AU_write close7253820 +Ref: 2cee7253820 +Node: wave — Read and write WAV files7254060 +Ref: library/wave doc7254235 +Ref: 2cef7254235 +Ref: library/wave module-wave7254235 +Ref: 1277254235 +Ref: library/wave wave-read-and-write-wav-files7254235 +Ref: 2cf07254235 +Ref: library/wave wave open7254665 +Ref: 4767254665 +Ref: library/wave wave openfp7255716 +Ref: 4757255716 +Ref: library/wave wave Error7255908 +Ref: 2cf37255908 +Ref: wave — Read and write WAV files-Footnote-17256162 +Node: Wave_read Objects7256225 +Ref: library/wave id17256339 +Ref: 2cf47256339 +Ref: library/wave wave-read-objects7256339 +Ref: 2cf57256339 +Ref: library/wave wave Wave_read close7256476 +Ref: 2cf17256476 +Ref: library/wave wave Wave_read getnchannels7256662 +Ref: 2cf67256662 +Ref: library/wave wave Wave_read getsampwidth7256780 +Ref: 2cf77256780 +Ref: library/wave wave Wave_read getframerate7256856 +Ref: 2cf87256856 +Ref: library/wave wave Wave_read getnframes7256929 +Ref: 2cf97256929 +Ref: library/wave wave Wave_read getcomptype7257004 +Ref: 2cfa7257004 +Ref: library/wave wave Wave_read getcompname7257116 +Ref: 2cfb7257116 +Ref: library/wave wave Wave_read getparams7257274 +Ref: 2cfc7257274 +Ref: library/wave wave Wave_read readframes7257481 +Ref: 2cfd7257481 +Ref: library/wave wave Wave_read rewind7257608 +Ref: 2cfe7257608 +Ref: library/wave wave Wave_read getmarkers7257835 +Ref: 2cff7257835 +Ref: library/wave wave Wave_read getmark7257898 +Ref: 2d007257898 +Ref: library/wave wave Wave_read setpos7258089 +Ref: 2d017258089 +Ref: library/wave wave Wave_read tell7258179 +Ref: 2d027258179 +Node: Wave_write Objects7258254 +Ref: library/wave id27258368 +Ref: 2d037258368 +Ref: library/wave wave-write-objects7258368 +Ref: 90b7258368 +Ref: library/wave wave Wave_write close7259309 +Ref: 2cf27259309 +Ref: library/wave wave Wave_write setnchannels7259630 +Ref: 2d067259630 +Ref: library/wave wave Wave_write setsampwidth7259705 +Ref: 2d077259705 +Ref: library/wave wave Wave_write setframerate7259787 +Ref: 2d087259787 +Ref: library/wave wave Wave_write setnframes7259967 +Ref: 2d047259967 +Ref: library/wave wave Wave_write setcomptype7260217 +Ref: 2d097260217 +Ref: library/wave wave Wave_write setparams7260406 +Ref: 2d057260406 +Ref: library/wave wave Wave_write tell7260625 +Ref: 2d0a7260625 +Ref: library/wave wave Wave_write writeframesraw7260818 +Ref: 90c7260818 +Ref: library/wave wave Wave_write writeframes7261006 +Ref: 90d7261006 +Node: chunk — Read IFF chunked data7261579 +Ref: library/chunk doc7261763 +Ref: 2d0b7261763 +Ref: library/chunk chunk-read-iff-chunked-data7261763 +Ref: 2d0c7261763 +Ref: library/chunk module-chunk7261763 +Ref: 187261763 +Ref: library/chunk chunk Chunk7263593 +Ref: 2d0d7263593 +Ref: library/chunk chunk Chunk getname7264725 +Ref: 2d0e7264725 +Ref: library/chunk chunk Chunk getsize7264852 +Ref: 2d0f7264852 +Ref: library/chunk chunk Chunk close7264923 +Ref: 2d107264923 +Ref: library/chunk chunk Chunk isatty7265287 +Ref: 2d117265287 +Ref: library/chunk chunk Chunk seek7265347 +Ref: 2d127265347 +Ref: library/chunk chunk Chunk tell7265789 +Ref: 2d137265789 +Ref: library/chunk chunk Chunk read7265870 +Ref: 2d147265870 +Ref: library/chunk chunk Chunk skip7266250 +Ref: 2d157266250 +Ref: chunk — Read IFF chunked data-Footnote-17266604 +Ref: chunk — Read IFF chunked data-Footnote-27266668 +Node: colorsys — Conversions between color systems7266782 +Ref: library/colorsys doc7266974 +Ref: 2d167266974 +Ref: library/colorsys colorsys-conversions-between-color-systems7266974 +Ref: 2d177266974 +Ref: library/colorsys module-colorsys7266974 +Ref: 207266974 +Ref: library/colorsys colorsys rgb_to_yiq7267960 +Ref: 2d187267960 +Ref: library/colorsys colorsys yiq_to_rgb7268070 +Ref: 2d197268070 +Ref: library/colorsys colorsys rgb_to_hls7268180 +Ref: 2d1a7268180 +Ref: library/colorsys colorsys hls_to_rgb7268290 +Ref: 2d1b7268290 +Ref: library/colorsys colorsys rgb_to_hsv7268400 +Ref: 2d1c7268400 +Ref: library/colorsys colorsys hsv_to_rgb7268510 +Ref: 2d1d7268510 +Ref: colorsys — Conversions between color systems-Footnote-17268822 +Node: imghdr — Determine the type of an image7268889 +Ref: library/imghdr doc7269089 +Ref: 2d1e7269089 +Ref: library/imghdr imghdr-determine-the-type-of-an-image7269089 +Ref: 2d1f7269089 +Ref: library/imghdr module-imghdr7269089 +Ref: 997269089 +Ref: library/imghdr imghdr what7269452 +Ref: 6ed7269452 +Ref: library/imghdr imghdr tests7271173 +Ref: 2d207271173 +Ref: imghdr — Determine the type of an image-Footnote-17271691 +Node: sndhdr — Determine type of sound file7271756 +Ref: library/sndhdr doc7271964 +Ref: 2d217271964 +Ref: library/sndhdr module-sndhdr7271964 +Ref: ee7271964 +Ref: library/sndhdr sndhdr-determine-type-of-sound-file7271964 +Ref: 2d227271964 +Ref: library/sndhdr sndhdr what7273208 +Ref: 7477273208 +Ref: library/sndhdr sndhdr whathdr7273526 +Ref: 7487273526 +Ref: sndhdr — Determine type of sound file-Footnote-17273909 +Node: ossaudiodev — Access to OSS-compatible audio devices7273974 +Ref: library/ossaudiodev doc7274132 +Ref: 2d237274132 +Ref: library/ossaudiodev module-ossaudiodev7274132 +Ref: c67274132 +Ref: library/ossaudiodev ossaudiodev-access-to-oss-compatible-audio-devices7274132 +Ref: 2d247274132 +Ref: library/ossaudiodev ossaudiodev OSSAudioError7275017 +Ref: 2d257275017 +Ref: library/ossaudiodev ossaudiodev open7275536 +Ref: 2d267275536 +Ref: library/ossaudiodev ossaudiodev openmixer7276935 +Ref: 2d277276935 +Ref: ossaudiodev — Access to OSS-compatible audio devices-Footnote-17277373 +Node: Audio Device Objects7277421 +Ref: library/ossaudiodev audio-device-objects7277561 +Ref: 2d287277561 +Ref: library/ossaudiodev ossaudio-device-objects7277561 +Ref: 2d297277561 +Ref: library/ossaudiodev ossaudiodev oss_audio_device close7278153 +Ref: 2d2a7278153 +Ref: library/ossaudiodev ossaudiodev oss_audio_device fileno7278377 +Ref: 2d2b7278377 +Ref: library/ossaudiodev ossaudiodev oss_audio_device read7278478 +Ref: 2d2c7278478 +Ref: library/ossaudiodev ossaudiodev oss_audio_device write7278786 +Ref: 2d2d7278786 +Ref: library/ossaudiodev ossaudiodev oss_audio_device writeall7279306 +Ref: 2d2e7279306 +Ref: library/ossaudiodev ossaudiodev oss_audio_device nonblock7280468 +Ref: 2d2f7280468 +Ref: library/ossaudiodev ossaudiodev oss_audio_device getfmts7280635 +Ref: 2d307280635 +Ref: library/ossaudiodev ossaudiodev oss_audio_device setfmt7282805 +Ref: 2d317282805 +Ref: library/ossaudiodev ossaudiodev oss_audio_device channels7283176 +Ref: 2d327283176 +Ref: library/ossaudiodev ossaudiodev oss_audio_device speed7283504 +Ref: 2d337283504 +Ref: library/ossaudiodev ossaudiodev oss_audio_device sync7284249 +Ref: 2d347284249 +Ref: library/ossaudiodev ossaudiodev oss_audio_device reset7284532 +Ref: 2d357284532 +Ref: library/ossaudiodev ossaudiodev oss_audio_device post7284796 +Ref: 2d367284796 +Ref: library/ossaudiodev ossaudiodev oss_audio_device setparameters7285209 +Ref: 2d377285209 +Ref: library/ossaudiodev ossaudiodev oss_audio_device bufsize7286272 +Ref: 2d387286272 +Ref: library/ossaudiodev ossaudiodev oss_audio_device obufcount7286372 +Ref: 2d397286372 +Ref: library/ossaudiodev ossaudiodev oss_audio_device obuffree7286506 +Ref: 2d3a7286506 +Ref: library/ossaudiodev ossaudiodev oss_audio_device closed7286731 +Ref: 2d3b7286731 +Ref: library/ossaudiodev ossaudiodev oss_audio_device name7286832 +Ref: 2d3c7286832 +Ref: library/ossaudiodev ossaudiodev oss_audio_device mode7286923 +Ref: 2d3d7286923 +Node: Mixer Device Objects7287038 +Ref: library/ossaudiodev id17287178 +Ref: 2d3e7287178 +Ref: library/ossaudiodev mixer-device-objects7287178 +Ref: 2d3f7287178 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device close7287289 +Ref: 2d407287289 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device fileno7287490 +Ref: 2d417287490 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device controls7287736 +Ref: 2d427287736 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device stereocontrols7288714 +Ref: 2d437288714 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device reccontrols7289159 +Ref: 2d447289159 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device get7289388 +Ref: 2d457289388 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device set7289883 +Ref: 2d467289883 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device get_recsrc7290446 +Ref: 2d477290446 +Ref: library/ossaudiodev ossaudiodev oss_mixer_device set_recsrc7290607 +Ref: 2d487290607 +Node: Internationalization7291004 +Ref: library/i18n doc7291143 +Ref: 2d497291143 +Ref: library/i18n i18n7291143 +Ref: 2d4a7291143 +Ref: library/i18n internationalization7291143 +Ref: 2d4b7291143 +Node: gettext — Multilingual internationalization services7291598 +Ref: library/gettext doc7291758 +Ref: 2d4c7291758 +Ref: library/gettext gettext-multilingual-internationalization-services7291758 +Ref: 2d4d7291758 +Ref: library/gettext module-gettext7291758 +Ref: 897291758 +Ref: gettext — Multilingual internationalization services-Footnote-17292735 +Node: GNU gettext API7292801 +Ref: library/gettext gnu-gettext-api7292931 +Ref: 2d4e7292931 +Ref: library/gettext gettext bindtextdomain7293475 +Ref: 2d4f7293475 +Ref: library/gettext gettext bind_textdomain_codeset7294047 +Ref: 2977294047 +Ref: library/gettext gettext textdomain7294473 +Ref: 2d517294473 +Ref: library/gettext gettext gettext7294719 +Ref: dcd7294719 +Ref: library/gettext gettext dgettext7294988 +Ref: 2d527294988 +Ref: library/gettext gettext ngettext7295127 +Ref: 2d537295127 +Ref: library/gettext gettext dngettext7295837 +Ref: 2d547295837 +Ref: library/gettext gettext pgettext7295991 +Ref: 1cf7295991 +Ref: library/gettext gettext dpgettext7296042 +Ref: 2d557296042 +Ref: library/gettext gettext npgettext7296102 +Ref: 2d567296102 +Ref: library/gettext gettext dnpgettext7296166 +Ref: 2d577296166 +Ref: library/gettext gettext lgettext7296531 +Ref: 2937296531 +Ref: library/gettext gettext ldgettext7296573 +Ref: 2947296573 +Ref: library/gettext gettext lngettext7296624 +Ref: 2957296624 +Ref: library/gettext gettext ldngettext7296679 +Ref: 2967296679 +Ref: GNU gettext API-Footnote-17298124 +Ref: GNU gettext API-Footnote-27298612 +Node: Class-based API7298677 +Ref: library/gettext class-based-api7298860 +Ref: 2d587298860 +Ref: library/gettext gettext find7299390 +Ref: 2d5a7299390 +Ref: library/gettext gettext translation7300838 +Ref: 29a7300838 +Ref: library/gettext gettext install7302274 +Ref: 29b7302274 +Ref: Class-based API-Footnote-17303369 +Node: The NullTranslations class7303439 +Ref: library/gettext the-nulltranslations-class7303551 +Ref: 2d5f7303551 +Ref: library/gettext gettext NullTranslations7303996 +Ref: 2d5d7303996 +Ref: library/gettext gettext NullTranslations _parse7304395 +Ref: 2d617304395 +Ref: library/gettext gettext NullTranslations add_fallback7304706 +Ref: 2d607304706 +Ref: library/gettext gettext NullTranslations gettext7304969 +Ref: 2d627304969 +Ref: library/gettext gettext NullTranslations ngettext7305167 +Ref: 2d637305167 +Ref: library/gettext gettext NullTranslations pgettext7305419 +Ref: 2d647305419 +Ref: library/gettext gettext NullTranslations npgettext7305677 +Ref: 2d657305677 +Ref: library/gettext gettext NullTranslations lgettext7305950 +Ref: 2d5b7305950 +Ref: library/gettext gettext NullTranslations lngettext7305987 +Ref: 2d5c7305987 +Ref: library/gettext gettext NullTranslations info7306563 +Ref: 2d667306563 +Ref: library/gettext gettext NullTranslations charset7306731 +Ref: 2d677306731 +Ref: library/gettext gettext NullTranslations output_charset7306820 +Ref: 2987306820 +Ref: library/gettext gettext NullTranslations set_output_charset7307062 +Ref: 2997307062 +Ref: library/gettext gettext NullTranslations install7307250 +Ref: 2d5e7307250 +Node: The GNUTranslations class7308489 +Ref: library/gettext the-gnutranslations-class7308641 +Ref: 2d687308641 +Ref: library/gettext gettext GNUTranslations7310151 +Ref: 2d597310151 +Ref: library/gettext gettext GNUTranslations gettext7310271 +Ref: 2d697310271 +Ref: library/gettext gettext GNUTranslations ngettext7310680 +Ref: 2d6a7310680 +Ref: library/gettext gettext GNUTranslations pgettext7311604 +Ref: 2d6b7311604 +Ref: library/gettext gettext GNUTranslations npgettext7312080 +Ref: 2d6c7312080 +Ref: library/gettext gettext GNUTranslations lgettext7312691 +Ref: 2d6d7312691 +Ref: library/gettext gettext GNUTranslations lngettext7312728 +Ref: 2d6e7312728 +Ref: The GNUTranslations class-Footnote-17313298 +Node: Solaris message catalog support7313346 +Ref: library/gettext solaris-message-catalog-support7313495 +Ref: 2d6f7313495 +Node: The Catalog constructor7313744 +Ref: library/gettext the-catalog-constructor7313859 +Ref: 2d707313859 +Node: Internationalizing your programs and modules7314517 +Ref: library/gettext internationalizing-your-programs-and-modules7314704 +Ref: 2d717314704 +Ref: Internationalizing your programs and modules-Footnote-17318282 +Ref: Internationalizing your programs and modules-Footnote-27318314 +Node: Localizing your module7318357 +Ref: library/gettext localizing-your-module7318496 +Ref: 2d727318496 +Node: Localizing your application7319091 +Ref: library/gettext localizing-your-application7319268 +Ref: 2d737319268 +Node: Changing languages on the fly7319996 +Ref: library/gettext changing-languages-on-the-fly7320172 +Ref: 2d747320172 +Node: Deferred translations7320851 +Ref: library/gettext deferred-translations7320991 +Ref: 2d757320991 +Node: Acknowledgements<9>7323230 +Ref: library/gettext acknowledgements7323393 +Ref: 2d767323393 +Node: locale — Internationalization services7323789 +Ref: library/locale doc7323949 +Ref: 2d777323949 +Ref: library/locale locale-internationalization-services7323949 +Ref: 2d787323949 +Ref: library/locale module-locale7323949 +Ref: a97323949 +Ref: library/locale locale Error7324677 +Ref: 2d797324677 +Ref: library/locale locale setlocale7324804 +Ref: 1ce57324804 +Ref: library/locale locale localeconv7325991 +Ref: 48a7325991 +Ref: library/locale locale nl_langinfo7334292 +Ref: 2d7d7334292 +Ref: library/locale locale CODESET7334813 +Ref: 2d7e7334813 +Ref: library/locale locale D_T_FMT7334947 +Ref: 2d7f7334947 +Ref: library/locale locale D_FMT7335143 +Ref: 2d807335143 +Ref: library/locale locale T_FMT7335330 +Ref: 2d817335330 +Ref: library/locale locale T_FMT_AMPM7335517 +Ref: 2d827335517 +Ref: library/locale locale RADIXCHAR7336215 +Ref: 2d837336215 +Ref: library/locale locale THOUSEP7336319 +Ref: 2d847336319 +Ref: library/locale locale YESEXPR7336439 +Ref: 2d857336439 +Ref: library/locale locale NOEXPR7336823 +Ref: 2d867336823 +Ref: library/locale locale CRNCYSTR7337008 +Ref: 2d877337008 +Ref: library/locale locale ERA7337286 +Ref: 2d887337286 +Ref: library/locale locale ERA_D_T_FMT7338060 +Ref: 2d897338060 +Ref: library/locale locale ERA_D_FMT7338231 +Ref: 2d8a7338231 +Ref: library/locale locale ERA_T_FMT7338393 +Ref: 2d8b7338393 +Ref: library/locale locale ALT_DIGITS7338555 +Ref: 2d8c7338555 +Ref: library/locale locale getdefaultlocale7338689 +Ref: ffc7338689 +Ref: library/locale locale getlocale7339896 +Ref: 15a37339896 +Ref: library/locale locale getpreferredencoding7340373 +Ref: 3a57340373 +Ref: library/locale locale normalize7341241 +Ref: 2d8f7341241 +Ref: library/locale locale resetlocale7341663 +Ref: 2d907341663 +Ref: library/locale locale strcoll7341912 +Ref: 2d917341912 +Ref: library/locale locale strxfrm7342231 +Ref: 2d937342231 +Ref: library/locale locale format_string7342575 +Ref: 3a47342575 +Ref: library/locale locale format7343271 +Ref: 4687343271 +Ref: library/locale locale currency7343747 +Ref: 2d947343747 +Ref: library/locale locale str7344377 +Ref: 2d957344377 +Ref: library/locale locale delocalize7344566 +Ref: 70b7344566 +Ref: library/locale locale atof7344744 +Ref: 2d967344744 +Ref: library/locale locale atoi7344885 +Ref: 2d977344885 +Ref: library/locale locale LC_CTYPE7345016 +Ref: 2d8e7345016 +Ref: library/locale locale LC_COLLATE7345238 +Ref: 2d927345238 +Ref: library/locale locale LC_TIME7345431 +Ref: 2d987345431 +Ref: library/locale locale LC_MONETARY7345581 +Ref: 2d7c7345581 +Ref: library/locale locale LC_MESSAGES7345754 +Ref: 2d997345754 +Ref: library/locale locale LC_NUMERIC7346052 +Ref: 2d7a7346052 +Ref: library/locale locale LC_ALL7346368 +Ref: 2d8d7346368 +Ref: library/locale locale CHAR_MAX7346793 +Ref: 2d7b7346793 +Ref: locale — Internationalization services-Footnote-17347644 +Ref: locale — Internationalization services-Footnote-27347709 +Ref: locale — Internationalization services-Footnote-37347758 +Node: Background details hints tips and caveats7347807 +Ref: library/locale background-details-hints-tips-and-caveats7347986 +Ref: 2d9a7347986 +Node: For extension writers and programs that embed Python7350184 +Ref: library/locale embedding-locale7350398 +Ref: 2d9b7350398 +Ref: library/locale for-extension-writers-and-programs-that-embed-python7350398 +Ref: 2d9c7350398 +Node: Access to message catalogs7351196 +Ref: library/locale access-to-message-catalogs7351360 +Ref: 2d9d7351360 +Ref: library/locale locale-gettext7351360 +Ref: 2d9e7351360 +Ref: library/locale locale gettext7351433 +Ref: 2d9f7351433 +Ref: library/locale locale dgettext7351469 +Ref: 2da07351469 +Ref: library/locale locale dcgettext7351514 +Ref: 2da17351514 +Ref: library/locale locale textdomain7351570 +Ref: 2da27351570 +Ref: library/locale locale bindtextdomain7351612 +Ref: 2da37351612 +Node: Program Frameworks7352544 +Ref: library/frameworks doc7352697 +Ref: 2da47352697 +Ref: library/frameworks frameworks7352697 +Ref: 2da57352697 +Ref: library/frameworks program-frameworks7352697 +Ref: 2da67352697 +Node: turtle — Turtle graphics7353146 +Ref: library/turtle doc7353290 +Ref: 2da77353290 +Ref: library/turtle module-turtle7353290 +Ref: 1167353290 +Ref: library/turtle turtle-turtle-graphics7353290 +Ref: 2da87353290 +Ref: turtle — Turtle graphics-Footnote-17353907 +Node: Introduction<11>7353972 +Ref: library/turtle introduction7354107 +Ref: 2da97354107 +Node: Overview of available Turtle and Screen methods7357838 +Ref: library/turtle overview-of-available-turtle-and-screen-methods7358037 +Ref: 2db07358037 +Node: Turtle methods7358216 +Ref: library/turtle turtle-methods7358353 +Ref: 2db17358353 +Node: Methods of TurtleScreen/Screen7361431 +Ref: library/turtle methods-of-turtlescreen-screen7361568 +Ref: 2e017361568 +Node: Methods of RawTurtle/Turtle and corresponding functions7362963 +Ref: library/turtle methods-of-rawturtle-turtle-and-corresponding-functions7363204 +Ref: 2e227363204 +Node: Turtle motion7363601 +Ref: library/turtle turtle-motion7363736 +Ref: 2e237363736 +Ref: library/turtle turtle forward7363783 +Ref: 2db27363783 +Ref: library/turtle turtle fd7363823 +Ref: 2db37363823 +Ref: library/turtle turtle back7364261 +Ref: 2db67364261 +Ref: library/turtle turtle bk7364298 +Ref: 2db57364298 +Ref: library/turtle turtle backward7364333 +Ref: 2db47364333 +Ref: library/turtle turtle right7364705 +Ref: 2db77364705 +Ref: library/turtle turtle rt7364740 +Ref: 2db87364740 +Ref: library/turtle turtle left7365206 +Ref: 2db97365206 +Ref: library/turtle turtle lt7365240 +Ref: 2dba7365240 +Ref: library/turtle turtle goto7365703 +Ref: 2dbb7365703 +Ref: library/turtle turtle setpos7365741 +Ref: 2dbc7365741 +Ref: library/turtle turtle setposition7365781 +Ref: 2dbd7365781 +Ref: library/turtle turtle setx7366548 +Ref: 2dbe7366548 +Ref: library/turtle turtle sety7366872 +Ref: 2dbf7366872 +Ref: library/turtle turtle setheading7367195 +Ref: 2dc07367195 +Ref: library/turtle turtle seth7367238 +Ref: 2dc17367238 +Ref: library/turtle turtle home7368054 +Ref: 2dc27368054 +Ref: library/turtle turtle circle7368481 +Ref: 2dc37368481 +Ref: library/turtle turtle dot7369866 +Ref: 2dc47369866 +Ref: library/turtle turtle stamp7370416 +Ref: 2dc57370416 +Ref: library/turtle turtle clearstamp7370755 +Ref: 2dc67370755 +Ref: library/turtle turtle clearstamps7371265 +Ref: 2dc77371265 +Ref: library/turtle turtle undo7371845 +Ref: 2dc87371845 +Ref: library/turtle turtle speed7372170 +Ref: 2dc97372170 +Node: Tell Turtle’s state7373169 +Ref: library/turtle tell-turtle-s-state7373337 +Ref: 2e257373337 +Ref: library/turtle turtle position7373398 +Ref: 2dca7373398 +Ref: library/turtle turtle pos7373431 +Ref: 2dcb7373431 +Ref: library/turtle turtle towards7373603 +Ref: 2dcc7373603 +Ref: library/turtle turtle xcor7374168 +Ref: 2dcd7374168 +Ref: library/turtle turtle ycor7374448 +Ref: 2dce7374448 +Ref: library/turtle turtle heading7374735 +Ref: 2dcf7374735 +Ref: library/turtle turtle distance7374983 +Ref: 2dd07374983 +Node: Settings for measurement7375572 +Ref: library/turtle settings-for-measurement7375738 +Ref: 2e267375738 +Ref: library/turtle turtle degrees7375807 +Ref: 2dd17375807 +Ref: library/turtle turtle radians7376435 +Ref: 2dd27376435 +Node: Pen control7376760 +Ref: library/turtle pen-control7376917 +Ref: 2e277376917 +Node: Drawing state7377043 +Ref: library/turtle drawing-state7377126 +Ref: 2e287377126 +Ref: library/turtle turtle pendown7377175 +Ref: 2dd37377175 +Ref: library/turtle turtle pd7377207 +Ref: 2dd47377207 +Ref: library/turtle turtle down7377234 +Ref: 2dd57377234 +Ref: library/turtle turtle penup7377313 +Ref: 2dd67377313 +Ref: library/turtle turtle pu7377343 +Ref: 2dd77377343 +Ref: library/turtle turtle up7377370 +Ref: 2dd87377370 +Ref: library/turtle turtle pensize7377448 +Ref: 2dd97377448 +Ref: library/turtle turtle width7377490 +Ref: 2dda7377490 +Ref: library/turtle turtle pen7377952 +Ref: 2ddb7377952 +Ref: library/turtle turtle isdown7379862 +Ref: 2ddc7379862 +Node: Color control7380112 +Ref: library/turtle color-control7380211 +Ref: 2e297380211 +Ref: library/turtle turtle pencolor7380260 +Ref: 2dde7380260 +Ref: library/turtle turtle fillcolor7381881 +Ref: 2ddf7381881 +Ref: library/turtle turtle color7383400 +Ref: 2ddd7383400 +Node: Filling7384604 +Ref: library/turtle filling7384710 +Ref: 2e2a7384710 +Ref: library/turtle turtle filling7384747 +Ref: 2de07384747 +Ref: library/turtle turtle begin_fill7385011 +Ref: 2de17385011 +Ref: library/turtle turtle end_fill7385108 +Ref: 2de27385108 +Node: More drawing control7385655 +Ref: library/turtle more-drawing-control7385739 +Ref: 2e2b7385739 +Ref: library/turtle turtle write7386454 +Ref: 2de57386454 +Node: Turtle state7387233 +Ref: library/turtle turtle-state7387378 +Ref: 2e2c7387378 +Node: Visibility7387465 +Ref: library/turtle visibility7387543 +Ref: 2e2d7387543 +Ref: library/turtle turtle hideturtle7387586 +Ref: 2de87387586 +Ref: library/turtle turtle ht7387621 +Ref: 2de97387621 +Ref: library/turtle turtle showturtle7387878 +Ref: 2de67387878 +Ref: library/turtle turtle st7387913 +Ref: 2de77387913 +Ref: library/turtle turtle isvisible7388007 +Ref: 2dea7388007 +Node: Appearance7388286 +Ref: library/turtle appearance7388364 +Ref: 2e2e7388364 +Ref: library/turtle turtle shape7388407 +Ref: 2deb7388407 +Ref: library/turtle turtle resizemode7389080 +Ref: 2dec7389080 +Ref: library/turtle turtle shapesize7390051 +Ref: 2ded7390051 +Ref: library/turtle turtle turtlesize7390143 +Ref: 2dee7390143 +Ref: library/turtle turtle shearfactor7391134 +Ref: 2def7391134 +Ref: library/turtle turtle tilt7391803 +Ref: 2df27391803 +Ref: library/turtle turtle settiltangle7392243 +Ref: 2df07392243 +Ref: library/turtle turtle tiltangle7392792 +Ref: 2df17392792 +Ref: library/turtle turtle shapetransform7393514 +Ref: 2df37393514 +Ref: library/turtle turtle get_shapepoly7394546 +Ref: 2df47394546 +Node: Using events7394917 +Ref: library/turtle using-events7395073 +Ref: 2e2f7395073 +Ref: library/turtle turtle onrelease7395973 +Ref: 2df67395973 +Ref: library/turtle turtle ondrag7396984 +Ref: 2df77396984 +Node: Special Turtle methods7397859 +Ref: library/turtle special-turtle-methods7398018 +Ref: 2e307398018 +Ref: library/turtle turtle begin_poly7398085 +Ref: 2df87398085 +Ref: library/turtle turtle end_poly7398228 +Ref: 2df97398228 +Ref: library/turtle turtle get_poly7398419 +Ref: 2dfa7398419 +Ref: library/turtle turtle clone7398821 +Ref: 2dfb7398821 +Ref: library/turtle turtle getturtle7399018 +Ref: 2dfc7399018 +Ref: library/turtle turtle getpen7399052 +Ref: 2dfd7399052 +Ref: library/turtle turtle getscreen7399320 +Ref: 2dfe7399320 +Ref: library/turtle turtle setundobuffer7399626 +Ref: 2dff7399626 +Ref: library/turtle turtle undobufferentries7400063 +Ref: 2e007400063 +Node: Compound shapes7400223 +Ref: library/turtle compound-shapes7400361 +Ref: 2e317400361 +Ref: library/turtle compoundshapes7400361 +Ref: 2e327400361 +Node: Methods of TurtleScreen/Screen and corresponding functions7401388 +Ref: library/turtle methods-of-turtlescreen-screen-and-corresponding-functions7401596 +Ref: 2e347401596 +Node: Window control7402076 +Ref: library/turtle window-control7402211 +Ref: 2e357402211 +Ref: library/turtle turtle bgcolor7402262 +Ref: 2e027402262 +Ref: library/turtle turtle bgpic7402675 +Ref: 2e037402675 +Ref: library/turtle turtle clear7403279 +Ref: 2de47403279 +Ref: library/turtle turtle clearscreen7403309 +Ref: 2e047403309 +Ref: library/turtle turtle reset7403791 +Ref: 2de37403791 +Ref: library/turtle turtle resetscreen7403821 +Ref: 2e057403821 +Ref: library/turtle turtle screensize7404157 +Ref: 2e067404157 +Ref: library/turtle turtle setworldcoordinates7405066 +Ref: 2e077405066 +Node: Animation control7406048 +Ref: library/turtle animation-control7406211 +Ref: 2e367406211 +Ref: library/turtle turtle delay7406268 +Ref: 2e087406268 +Ref: library/turtle turtle tracer7406707 +Ref: 2e097406707 +Ref: library/turtle turtle update7407408 +Ref: 2e0a7407408 +Node: Using screen events7407580 +Ref: library/turtle using-screen-events7407742 +Ref: 2e377407742 +Ref: library/turtle turtle listen7407803 +Ref: 2e0b7407803 +Ref: library/turtle turtle onkey7408038 +Ref: 2e0c7408038 +Ref: library/turtle turtle onkeyrelease7408076 +Ref: 2e0d7408076 +Ref: library/turtle turtle onkeypress7408698 +Ref: 2e0e7408698 +Ref: library/turtle turtle onclick7409301 +Ref: 2df57409301 +Ref: library/turtle turtle onscreenclick7409353 +Ref: 2e0f7409353 +Ref: library/turtle turtle ontimer7410541 +Ref: 2e107410541 +Ref: library/turtle turtle mainloop7411025 +Ref: 2e117411025 +Ref: library/turtle turtle done7411058 +Ref: 2e127411058 +Node: Input methods7411384 +Ref: library/turtle input-methods7411557 +Ref: 2e387411557 +Ref: library/turtle turtle textinput7411606 +Ref: 2e1c7411606 +Ref: library/turtle turtle numinput7412060 +Ref: 2e1d7412060 +Node: Settings and special methods7412980 +Ref: library/turtle settings-and-special-methods7413192 +Ref: 2e397413192 +Ref: library/turtle turtle mode7413271 +Ref: 2e137413271 +Ref: library/turtle turtle colormode7414500 +Ref: 2e147414500 +Ref: library/turtle turtle getcanvas7415140 +Ref: 2e157415140 +Ref: library/turtle turtle getshapes7415391 +Ref: 2e167415391 +Ref: library/turtle turtle register_shape7415585 +Ref: 2e177415585 +Ref: library/turtle turtle addshape7415640 +Ref: 2e187415640 +Ref: library/turtle turtle turtles7416589 +Ref: 2e197416589 +Ref: library/turtle turtle window_height7416755 +Ref: 2e1a7416755 +Ref: library/turtle turtle window_width7416892 +Ref: 2e1b7416892 +Node: Methods specific to Screen not inherited from TurtleScreen7417026 +Ref: library/turtle methods-specific-to-screen-not-inherited-from-turtlescreen7417216 +Ref: 2e3a7417216 +Ref: library/turtle screenspecific7417216 +Ref: 2e3b7417216 +Ref: library/turtle turtle bye7417357 +Ref: 2e1e7417357 +Ref: library/turtle turtle exitonclick7417424 +Ref: 2e1f7417424 +Ref: library/turtle turtle setup7417864 +Ref: 2e207417864 +Ref: library/turtle turtle title7419184 +Ref: 2e217419184 +Node: Public classes7419450 +Ref: library/turtle public-classes7419625 +Ref: 2e3c7419625 +Ref: library/turtle turtle RawTurtle7419676 +Ref: 2dad7419676 +Ref: library/turtle turtle RawPen7419713 +Ref: 2dae7419713 +Ref: library/turtle turtle Turtle7419984 +Ref: 2daf7419984 +Ref: library/turtle turtle TurtleScreen7420176 +Ref: 2daa7420176 +Ref: library/turtle turtle Screen7420363 +Ref: 2dac7420363 +Ref: library/turtle turtle ScrolledCanvas7420458 +Ref: 2dab7420458 +Ref: library/turtle turtle Shape7420753 +Ref: 2e337420753 +Ref: library/turtle turtle Shape addcomponent7421561 +Ref: 2e3d7421561 +Ref: library/turtle turtle Vec2D7422161 +Ref: 2e247422161 +Node: Help and configuration7422713 +Ref: library/turtle help-and-configuration7422857 +Ref: 2e3e7422857 +Node: How to use help7423049 +Ref: library/turtle how-to-use-help7423182 +Ref: 2e3f7423182 +Node: Translation of docstrings into different languages7425512 +Ref: library/turtle translation-of-docstrings-into-different-languages7425689 +Ref: 2e407425689 +Ref: library/turtle turtle write_docstringdict7425993 +Ref: 2e417425993 +Node: How to configure Screen and Turtles7427032 +Ref: library/turtle how-to-configure-screen-and-turtles7427185 +Ref: 2e427427185 +Node: turtledemo — Demo scripts7429968 +Ref: library/turtle module-turtledemo7430122 +Ref: 1177430122 +Ref: library/turtle turtledemo-demo-scripts7430122 +Ref: 2e437430122 +Node: Changes since Python 2 67435765 +Ref: library/turtle changes-since-python-2-67435921 +Ref: 2e447435921 +Node: Changes since Python 3 07436896 +Ref: library/turtle changes-since-python-3-07437016 +Ref: 2e457437016 +Node: cmd — Support for line-oriented command interpreters7438223 +Ref: library/cmd doc7438409 +Ref: 2e467438409 +Ref: library/cmd cmd-support-for-line-oriented-command-interpreters7438409 +Ref: 2e477438409 +Ref: library/cmd module-cmd7438409 +Ref: 1a7438409 +Ref: library/cmd cmd Cmd7438885 +Ref: 2e487438885 +Ref: cmd — Support for line-oriented command interpreters-Footnote-17440078 +Node: Cmd Objects7440140 +Ref: library/cmd cmd-objects7440262 +Ref: 2e4a7440262 +Ref: library/cmd id17440262 +Ref: 2e4b7440262 +Ref: library/cmd cmd Cmd cmdloop7440361 +Ref: 2e4c7440361 +Ref: library/cmd cmd Cmd onecmd7442924 +Ref: 2e4f7442924 +Ref: library/cmd cmd Cmd emptyline7443515 +Ref: 2e527443515 +Ref: library/cmd cmd Cmd default7443713 +Ref: 2e517443713 +Ref: library/cmd cmd Cmd completedefault7443910 +Ref: 2e537443910 +Ref: library/cmd cmd Cmd precmd7444132 +Ref: 2e507444132 +Ref: library/cmd cmd Cmd postcmd7444611 +Ref: 2e4e7444611 +Ref: library/cmd cmd Cmd preloop7445244 +Ref: 2e547445244 +Ref: library/cmd cmd Cmd postloop7445438 +Ref: 2e557445438 +Ref: library/cmd cmd Cmd prompt7445722 +Ref: 2e567445722 +Ref: library/cmd cmd Cmd identchars7445791 +Ref: 2e577445791 +Ref: library/cmd cmd Cmd lastcmd7445886 +Ref: 2e587445886 +Ref: library/cmd cmd Cmd cmdqueue7445959 +Ref: 2e597445959 +Ref: library/cmd cmd Cmd intro7446210 +Ref: 2e4d7446210 +Ref: library/cmd cmd Cmd doc_header7446364 +Ref: 2e5a7446364 +Ref: library/cmd cmd Cmd misc_header7446484 +Ref: 2e5b7446484 +Ref: library/cmd cmd Cmd undoc_header7446703 +Ref: 2e5c7446703 +Ref: library/cmd cmd Cmd ruler7446919 +Ref: 2e5d7446919 +Ref: library/cmd cmd Cmd use_rawinput7447093 +Ref: 2e497447093 +Node: Cmd Example7447550 +Ref: library/cmd cmd-example7447672 +Ref: 2e5e7447672 +Ref: library/cmd id27447672 +Ref: 2e5f7447672 +Node: shlex — Simple lexical analysis7453015 +Ref: library/shlex doc7453166 +Ref: 2e607453166 +Ref: library/shlex module-shlex7453166 +Ref: e87453166 +Ref: library/shlex shlex-simple-lexical-analysis7453166 +Ref: 2e617453166 +Ref: library/shlex shlex split7453687 +Ref: 2187453687 +Ref: library/shlex shlex join7454342 +Ref: 2177454342 +Ref: library/shlex shlex quote7454798 +Ref: a737454798 +Ref: library/shlex shlex shlex7456100 +Ref: 57a7456100 +Ref: shlex — Simple lexical analysis-Footnote-17458346 +Node: shlex Objects7458410 +Ref: library/shlex id17458515 +Ref: 2e657458515 +Ref: library/shlex shlex-objects7458515 +Ref: 2e667458515 +Ref: library/shlex shlex shlex get_token7458619 +Ref: 2e677458619 +Ref: library/shlex shlex shlex push_token7458982 +Ref: 2e687458982 +Ref: library/shlex shlex shlex read_token7459064 +Ref: 2e6a7459064 +Ref: library/shlex shlex shlex sourcehook7459303 +Ref: 2e6b7459303 +Ref: library/shlex shlex shlex push_source7460843 +Ref: 2e6d7460843 +Ref: library/shlex shlex shlex pop_source7461146 +Ref: 2e6e7461146 +Ref: library/shlex shlex shlex error_leader7461345 +Ref: 2e6f7461345 +Ref: library/shlex shlex shlex commenters7462074 +Ref: 2e627462074 +Ref: library/shlex shlex shlex wordchars7462296 +Ref: 2e647462296 +Ref: library/shlex shlex shlex whitespace7462992 +Ref: 2e727462992 +Ref: library/shlex shlex shlex escape7463190 +Ref: 2e737463190 +Ref: library/shlex shlex shlex quotes7463355 +Ref: 2e747463355 +Ref: library/shlex shlex shlex escapedquotes7463640 +Ref: 2e757463640 +Ref: library/shlex shlex shlex whitespace_split7463869 +Ref: 2e717463869 +Ref: library/shlex shlex shlex infile7464407 +Ref: 2e637464407 +Ref: library/shlex shlex shlex instream7464638 +Ref: 2e767464638 +Ref: library/shlex shlex shlex source7464763 +Ref: 2e6c7464763 +Ref: library/shlex shlex shlex debug7465353 +Ref: 2e777465353 +Ref: library/shlex shlex shlex lineno7465617 +Ref: 2e787465617 +Ref: library/shlex shlex shlex token7465713 +Ref: 2e797465713 +Ref: library/shlex shlex shlex eof7465830 +Ref: 2e697465830 +Ref: library/shlex shlex shlex punctuation_chars7466006 +Ref: 2e707466006 +Node: Parsing Rules7466415 +Ref: library/shlex parsing-rules7466563 +Ref: 2e7a7466563 +Ref: library/shlex shlex-parsing-rules7466563 +Ref: 2e7b7466563 +Node: Improved Compatibility with Shells7468582 +Ref: library/shlex improved-compatibility-with-shells7468708 +Ref: 2e7c7468708 +Ref: library/shlex improved-shell-compatibility7468708 +Ref: 57b7468708 +Node: Graphical User Interfaces with Tk7471552 +Ref: library/tk doc7471702 +Ref: 2e7d7471702 +Ref: library/tk graphical-user-interfaces-with-tk7471702 +Ref: 2e7e7471702 +Ref: library/tk tkinter7471702 +Ref: 2e7f7471702 +Node: tkinter — Python interface to Tcl/Tk7473433 +Ref: library/tkinter doc7473583 +Ref: 2e817473583 +Ref: library/tkinter module-tkinter7473583 +Ref: 10c7473583 +Ref: library/tkinter tkinter-python-interface-to-tcl-tk7473583 +Ref: 2e827473583 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-17476018 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-27476094 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-37476139 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-47476170 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-57476210 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-67476249 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-77476300 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-87476386 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-97476455 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-107476512 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-117476545 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-127476569 +Ref: tkinter — Python interface to Tcl/Tk-Footnote-137476629 +Node: Tkinter Modules7476666 +Ref: library/tkinter tkinter-modules7476787 +Ref: 2e837476787 +Ref: library/tkinter tkinter Tk7477634 +Ref: 2e847477634 +Ref: library/tkinter tkinter Tcl7477952 +Ref: 2e857477952 +Node: Tkinter Life Preserver7479492 +Ref: library/tkinter tkinter-life-preserver7479649 +Ref: 2e867479649 +Node: How To Use This Section7480543 +Ref: library/tkinter how-to-use-this-section7480662 +Ref: 2e877480662 +Node: A Simple Hello World Program7482102 +Ref: library/tkinter a-simple-hello-world-program7482221 +Ref: 2e887482221 +Node: A Very Quick Look at Tcl/Tk7483093 +Ref: library/tkinter a-very-quick-look-at-tcl-tk7483264 +Ref: 2e897483264 +Node: Mapping Basic Tk into Tkinter7486617 +Ref: library/tkinter mapping-basic-tk-into-tkinter7486796 +Ref: 2e8b7486796 +Ref: library/tkinter tkinter-basic-mapping7486796 +Ref: 2e8a7486796 +Node: How Tk and Tkinter are Related7488618 +Ref: library/tkinter how-tk-and-tkinter-are-related7488785 +Ref: 2e8d7488785 +Node: Handy Reference7490057 +Ref: library/tkinter handy-reference7490208 +Ref: 2e8e7490208 +Node: Setting Options7490460 +Ref: library/tkinter setting-options7490546 +Ref: 2e8f7490546 +Ref: library/tkinter tkinter-setting-options7490546 +Ref: 2e8c7490546 +Node: The Packer7493753 +Ref: library/tkinter the-packer7493862 +Ref: 2e907493862 +Node: Packer Options7495512 +Ref: library/tkinter packer-options7495631 +Ref: 2e917495631 +Node: Coupling Widget Variables7496331 +Ref: library/tkinter coupling-widget-variables7496458 +Ref: 2e927496458 +Node: The Window Manager7498660 +Ref: library/tkinter the-window-manager7498793 +Ref: 2e937498793 +Node: Tk Option Data Types7500187 +Ref: library/tkinter tk-option-data-types7500314 +Ref: 2e947500314 +Node: Bindings and Events7503378 +Ref: library/tkinter bindings-and-events7503506 +Ref: 2e957503506 +Node: The index Parameter7506237 +Ref: library/tkinter the-index-parameter7506351 +Ref: 2e967506351 +Node: Images7508286 +Ref: library/tkinter images7508372 +Ref: 2e977508372 +Ref: Images-Footnote-17509306 +Node: File Handlers7509340 +Ref: library/tkinter file-handlers7509452 +Ref: 2e987509452 +Ref: library/tkinter tkinter-file-handlers7509452 +Ref: 2e997509452 +Ref: library/tkinter tkinter Widget tk createfilehandler7510394 +Ref: 2e9a7510394 +Ref: library/tkinter tkinter Widget tk deletefilehandler7510829 +Ref: 2e9b7510829 +Ref: library/tkinter tkinter READABLE7510911 +Ref: 2e9c7510911 +Ref: library/tkinter tkinter WRITABLE7510938 +Ref: 2e9d7510938 +Ref: library/tkinter tkinter EXCEPTION7510965 +Ref: 2e9e7510965 +Node: tkinter ttk — Tk themed widgets7511040 +Ref: library/tkinter ttk doc7511239 +Ref: 2e9f7511239 +Ref: library/tkinter ttk module-tkinter ttk7511239 +Ref: 10f7511239 +Ref: library/tkinter ttk tkinter-ttk-tk-themed-widgets7511239 +Ref: 2ea07511239 +Ref: tkinter ttk — Tk themed widgets-Footnote-17512301 +Ref: tkinter ttk — Tk themed widgets-Footnote-27512371 +Node: Using Ttk7512424 +Ref: library/tkinter ttk using-ttk7512523 +Ref: 2ea17512523 +Ref: Using Ttk-Footnote-17513680 +Node: Ttk Widgets7513743 +Ref: library/tkinter ttk ttk-widgets7513857 +Ref: 2ea27513857 +Node: Widget7514966 +Ref: library/tkinter ttk widget7515079 +Ref: 2ea97515079 +Node: Standard Options7515396 +Ref: library/tkinter ttk standard-options7515489 +Ref: 2eaa7515489 +Node: Scrollable Widget Options7517166 +Ref: library/tkinter ttk scrollable-widget-options7517281 +Ref: 2eab7517281 +Node: Label Options7518404 +Ref: library/tkinter ttk label-options7518524 +Ref: 2eac7518524 +Node: Compatibility Options7521083 +Ref: library/tkinter ttk compatibility-options7521191 +Ref: 2eae7521191 +Node: Widget States7521727 +Ref: library/tkinter ttk widget-states7521832 +Ref: 2eb07521832 +Node: ttk Widget7523511 +Ref: library/tkinter ttk ttk-widget7523586 +Ref: 2eb17523586 +Ref: library/tkinter ttk tkinter ttk Widget7523773 +Ref: 2ea67523773 +Ref: library/tkinter ttk tkinter ttk Widget identify7523804 +Ref: 2eb27523804 +Ref: library/tkinter ttk tkinter ttk Widget instate7524047 +Ref: 2eb37524047 +Ref: library/tkinter ttk tkinter ttk Widget state7524391 +Ref: 2eaf7524391 +Node: Combobox7524779 +Ref: library/tkinter ttk combobox7524888 +Ref: 2eb47524888 +Node: Options7525583 +Ref: library/tkinter ttk options7525658 +Ref: 2eb57525658 +Node: Virtual events7528164 +Ref: library/tkinter ttk virtual-events7528260 +Ref: 2eb67528260 +Node: ttk Combobox7528440 +Ref: library/tkinter ttk ttk-combobox7528520 +Ref: 2eb77528520 +Ref: library/tkinter ttk tkinter ttk Combobox7528567 +Ref: 2ea37528567 +Ref: library/tkinter ttk tkinter ttk Combobox current7528600 +Ref: 2eb87528600 +Ref: library/tkinter ttk tkinter ttk Combobox get7528876 +Ref: 2eb97528876 +Ref: library/tkinter ttk tkinter ttk Combobox set7528955 +Ref: 2eba7528955 +Node: Spinbox7529039 +Ref: library/tkinter ttk spinbox7529150 +Ref: 2ebb7529150 +Node: Options<2>7529910 +Ref: library/tkinter ttk id17529990 +Ref: 2ebc7529990 +Node: Virtual events<2>7532376 +Ref: library/tkinter ttk id27532476 +Ref: 2ebd7532476 +Node: ttk Spinbox7532684 +Ref: library/tkinter ttk ttk-spinbox7532765 +Ref: 2ebe7532765 +Ref: library/tkinter ttk tkinter ttk Spinbox7532810 +Ref: 3fd7532810 +Ref: library/tkinter ttk tkinter ttk Spinbox get7532842 +Ref: 2ebf7532842 +Ref: library/tkinter ttk tkinter ttk Spinbox set7532920 +Ref: 2ec07532920 +Node: Notebook7533003 +Ref: library/tkinter ttk notebook7533117 +Ref: 2ec17533117 +Node: Options<3>7533477 +Ref: library/tkinter ttk id37533552 +Ref: 2ec27533552 +Node: Tab Options7534727 +Ref: library/tkinter ttk tab-options7534826 +Ref: 2ec37534826 +Node: Tab Identifiers7536901 +Ref: library/tkinter ttk tab-identifiers7537004 +Ref: 2ec57537004 +Node: Virtual Events7537538 +Ref: library/tkinter ttk id47537642 +Ref: 2ec77537642 +Node: ttk Notebook7537786 +Ref: library/tkinter ttk ttk-notebook7537866 +Ref: 2ec87537866 +Ref: library/tkinter ttk tkinter ttk Notebook7537913 +Ref: 2ea47537913 +Ref: library/tkinter ttk tkinter ttk Notebook add7537946 +Ref: 2ec97537946 +Ref: library/tkinter ttk tkinter ttk Notebook forget7538231 +Ref: 2eca7538231 +Ref: library/tkinter ttk tkinter ttk Notebook hide7538369 +Ref: 2ecb7538369 +Ref: library/tkinter ttk tkinter ttk Notebook identify7538679 +Ref: 2ecc7538679 +Ref: library/tkinter ttk tkinter ttk Notebook index7538821 +Ref: 2ec67538821 +Ref: library/tkinter ttk tkinter ttk Notebook insert7539000 +Ref: 2ecd7539000 +Ref: library/tkinter ttk tkinter ttk Notebook select7539385 +Ref: 2ece7539385 +Ref: library/tkinter ttk tkinter ttk Notebook tab7539699 +Ref: 2ecf7539699 +Ref: library/tkinter ttk tkinter ttk Notebook tabs7540045 +Ref: 2ed07540045 +Ref: library/tkinter ttk tkinter ttk Notebook enable_traversal7540133 +Ref: 2ec47540133 +Node: Progressbar7540960 +Ref: library/tkinter ttk progressbar7541076 +Ref: 2ed17541076 +Node: Options<4>7541517 +Ref: library/tkinter ttk id57541599 +Ref: 2ed27541599 +Node: ttk Progressbar7543474 +Ref: library/tkinter ttk ttk-progressbar7543556 +Ref: 2ed37543556 +Ref: library/tkinter ttk tkinter ttk Progressbar7543609 +Ref: 2ea57543609 +Ref: library/tkinter ttk tkinter ttk Progressbar start7543645 +Ref: 2ed47543645 +Ref: library/tkinter ttk tkinter ttk Progressbar step7543913 +Ref: 2ed57543913 +Ref: library/tkinter ttk tkinter ttk Progressbar stop7544060 +Ref: 2ed67544060 +Node: Separator7544245 +Ref: library/tkinter ttk separator7544361 +Ref: 2ed77544361 +Node: Options<5>7544591 +Ref: library/tkinter ttk id67544647 +Ref: 2ed87544647 +Node: Sizegrip7545036 +Ref: library/tkinter ttk sizegrip7545149 +Ref: 2ed97545149 +Node: Platform-specific notes7545499 +Ref: library/tkinter ttk platform-specific-notes7545580 +Ref: 2eda7545580 +Node: Bugs7545838 +Ref: library/tkinter ttk bugs7545919 +Ref: 2edb7545919 +Node: Treeview7546192 +Ref: library/tkinter ttk treeview7546307 +Ref: 2edc7546307 +Node: Options<6>7547665 +Ref: library/tkinter ttk id77547741 +Ref: 2ee07547741 +Node: Item Options7550387 +Ref: library/tkinter ttk item-options7550483 +Ref: 2ee17550483 +Node: Tag Options7551659 +Ref: library/tkinter ttk tag-options7551763 +Ref: 2ee27551763 +Node: Column Identifiers7552493 +Ref: library/tkinter ttk column-identifiers7552602 +Ref: 2edd7552602 +Node: Virtual Events<2>7553474 +Ref: library/tkinter ttk id87553584 +Ref: 2ee37553584 +Node: ttk Treeview7554486 +Ref: library/tkinter ttk ttk-treeview7554569 +Ref: 2ee57554569 +Ref: library/tkinter ttk tkinter ttk Treeview7554616 +Ref: 2bd7554616 +Ref: library/tkinter ttk tkinter ttk Treeview bbox7554649 +Ref: 2ee67554649 +Ref: library/tkinter ttk tkinter ttk Treeview get_children7555086 +Ref: 2ee77555086 +Ref: library/tkinter ttk tkinter ttk Treeview set_children7555252 +Ref: 2ee87555252 +Ref: library/tkinter ttk tkinter ttk Treeview column7555644 +Ref: 2ee97555644 +Ref: library/tkinter ttk tkinter ttk Treeview delete7557054 +Ref: 2eea7557054 +Ref: library/tkinter ttk tkinter ttk Treeview detach7557200 +Ref: 2eeb7557200 +Ref: library/tkinter ttk tkinter ttk Treeview exists7557511 +Ref: 2eec7557511 +Ref: library/tkinter ttk tkinter ttk Treeview focus7557621 +Ref: 2ee47557621 +Ref: library/tkinter ttk tkinter ttk Treeview heading7557814 +Ref: 2eed7557814 +Ref: library/tkinter ttk tkinter ttk Treeview identify7558915 +Ref: 2eee7558915 +Ref: library/tkinter ttk tkinter ttk Treeview identify_row7559152 +Ref: 2eef7559152 +Ref: library/tkinter ttk tkinter ttk Treeview identify_column7559247 +Ref: 2ef07559247 +Ref: library/tkinter ttk tkinter ttk Treeview identify_region7559408 +Ref: 2ef17559408 +Ref: library/tkinter ttk tkinter ttk Treeview identify_element7560044 +Ref: 2ef27560044 +Ref: library/tkinter ttk tkinter ttk Treeview index7560172 +Ref: 2ef37560172 +Ref: library/tkinter ttk tkinter ttk Treeview insert7560300 +Ref: 2ef47560300 +Ref: library/tkinter ttk tkinter ttk Treeview item7561197 +Ref: 2ef57561197 +Ref: library/tkinter ttk tkinter ttk Treeview move7561575 +Ref: 2ef67561575 +Ref: library/tkinter ttk tkinter ttk Treeview next7562013 +Ref: 2ef77562013 +Ref: library/tkinter ttk tkinter ttk Treeview parent7562168 +Ref: 2ef87562168 +Ref: library/tkinter ttk tkinter ttk Treeview prev7562319 +Ref: 2ef97562319 +Ref: library/tkinter ttk tkinter ttk Treeview reattach7562479 +Ref: 2efa7562479 +Ref: library/tkinter ttk tkinter ttk Treeview see7562581 +Ref: 2efb7562581 +Ref: library/tkinter ttk tkinter ttk Treeview selection7562837 +Ref: 2bc7562837 +Ref: library/tkinter ttk tkinter ttk Treeview selection_set7563085 +Ref: 2be7563085 +Ref: library/tkinter ttk tkinter ttk Treeview selection_add7563290 +Ref: 2efc7563290 +Ref: library/tkinter ttk tkinter ttk Treeview selection_remove7563490 +Ref: 2efd7563490 +Ref: library/tkinter ttk tkinter ttk Treeview selection_toggle7563698 +Ref: 2efe7563698 +Ref: library/tkinter ttk tkinter ttk Treeview set7563923 +Ref: 2eff7563923 +Ref: library/tkinter ttk tkinter ttk Treeview tag_bind7564288 +Ref: 2f007564288 +Ref: library/tkinter ttk tkinter ttk Treeview tag_configure7564558 +Ref: 2f017564558 +Ref: library/tkinter ttk tkinter ttk Treeview tag_has7564987 +Ref: 2f027564987 +Ref: library/tkinter ttk tkinter ttk Treeview xview7565272 +Ref: 2ede7565272 +Ref: library/tkinter ttk tkinter ttk Treeview yview7565368 +Ref: 2edf7565368 +Node: Ttk Styling7565462 +Ref: library/tkinter ttk ttk-styling7565560 +Ref: 2f037565560 +Ref: library/tkinter ttk ttkstyling7565560 +Ref: 2ea77565560 +Ref: library/tkinter ttk tkinter ttk Style7566154 +Ref: 2ea87566154 +Ref: library/tkinter ttk tkinter ttk Style configure7566243 +Ref: 2f047566243 +Ref: library/tkinter ttk tkinter ttk Style map7566970 +Ref: 2ead7566970 +Ref: library/tkinter ttk tkinter ttk Style lookup7568235 +Ref: 2f057568235 +Ref: library/tkinter ttk tkinter ttk Style layout7568756 +Ref: 2f067568756 +Ref: library/tkinter ttk tkinter ttk Style element_create7570068 +Ref: 2f087570068 +Ref: library/tkinter ttk tkinter ttk Style element_names7572190 +Ref: 2f097572190 +Ref: library/tkinter ttk tkinter ttk Style element_options7572295 +Ref: 2f0a7572295 +Ref: library/tkinter ttk tkinter ttk Style theme_create7572401 +Ref: 2f0b7572401 +Ref: library/tkinter ttk tkinter ttk Style theme_settings7572814 +Ref: 2f0c7572814 +Ref: library/tkinter ttk tkinter ttk Style theme_names7574291 +Ref: 2f0d7574291 +Ref: library/tkinter ttk tkinter ttk Style theme_use7574371 +Ref: 2f0e7574371 +Ref: Ttk Styling-Footnote-17574668 +Node: Layouts7574729 +Ref: library/tkinter ttk layouts7574784 +Ref: 2f077574784 +Node: tkinter tix — Extension widgets for Tk7576068 +Ref: library/tkinter tix doc7576274 +Ref: 2f0f7576274 +Ref: library/tkinter tix layout7576274 +Ref: 2f107576274 +Ref: library/tkinter tix layouts7576274 +Ref: 2f117576274 +Ref: library/tkinter tix module-tkinter tix7576274 +Ref: 10e7576274 +Ref: library/tkinter tix tkinter-tix-extension-widgets-for-tk7576274 +Ref: 2f127576274 +Ref: tkinter tix — Extension widgets for Tk-Footnote-17578218 +Ref: tkinter tix — Extension widgets for Tk-Footnote-27578288 +Ref: tkinter tix — Extension widgets for Tk-Footnote-37578324 +Ref: tkinter tix — Extension widgets for Tk-Footnote-47578377 +Ref: tkinter tix — Extension widgets for Tk-Footnote-57578454 +Node: Using Tix7578511 +Ref: library/tkinter tix using-tix7578617 +Ref: 2f197578617 +Ref: library/tkinter tix tkinter tix Tk7578656 +Ref: 2f1a7578656 +Node: Tix Widgets7579629 +Ref: library/tkinter tix tix-widgets7579756 +Ref: 2f1b7579756 +Ref: Tix Widgets-Footnote-17580096 +Node: Basic Widgets7580174 +Ref: library/tkinter tix basic-widgets7580258 +Ref: 2f1c7580258 +Ref: library/tkinter tix tkinter tix Balloon7580305 +Ref: 2f1d7580305 +Ref: library/tkinter tix tkinter tix ButtonBox7580583 +Ref: 2f1e7580583 +Ref: library/tkinter tix tkinter tix ComboBox7580724 +Ref: 2f147580724 +Ref: library/tkinter tix tkinter tix Control7580958 +Ref: 2f157580958 +Ref: library/tkinter tix tkinter tix LabelEntry7581271 +Ref: 2f1f7581271 +Ref: library/tkinter tix tkinter tix LabelFrame7581486 +Ref: 2f207581486 +Ref: library/tkinter tix tkinter tix Meter7581788 +Ref: 2f217581788 +Ref: library/tkinter tix tkinter tix OptionMenu7581941 +Ref: 2f227581941 +Ref: library/tkinter tix tkinter tix PopupMenu7582034 +Ref: 2f237582034 +Ref: library/tkinter tix tkinter tix Select7582282 +Ref: 2f247582282 +Ref: library/tkinter tix tkinter tix StdButtonBox7582479 +Ref: 2f257582479 +Ref: Basic Widgets-Footnote-17582651 +Ref: Basic Widgets-Footnote-27582731 +Ref: Basic Widgets-Footnote-37582813 +Ref: Basic Widgets-Footnote-47582894 +Ref: Basic Widgets-Footnote-57582974 +Ref: Basic Widgets-Footnote-67583057 +Ref: Basic Widgets-Footnote-77583140 +Ref: Basic Widgets-Footnote-87583218 +Ref: Basic Widgets-Footnote-97583301 +Ref: Basic Widgets-Footnote-107583383 +Ref: Basic Widgets-Footnote-117583463 +Node: File Selectors7583549 +Ref: library/tkinter tix file-selectors7583662 +Ref: 2f267583662 +Ref: library/tkinter tix tkinter tix DirList7583711 +Ref: 2f277583711 +Ref: library/tkinter tix tkinter tix DirTree7583971 +Ref: 2f287583971 +Ref: library/tkinter tix tkinter tix DirSelectDialog7584231 +Ref: 2f297584231 +Ref: library/tkinter tix tkinter tix DirSelectBox7584487 +Ref: 2f2a7584487 +Ref: library/tkinter tix tkinter tix ExFileSelectBox7584829 +Ref: 2f2b7584829 +Ref: library/tkinter tix tkinter tix FileSelectBox7585160 +Ref: 2f2c7585160 +Ref: library/tkinter tix tkinter tix FileEntry7585492 +Ref: 2f177585492 +Ref: File Selectors-Footnote-17585812 +Ref: File Selectors-Footnote-27585892 +Ref: File Selectors-Footnote-37585972 +Ref: File Selectors-Footnote-47586060 +Ref: File Selectors-Footnote-57586148 +Ref: File Selectors-Footnote-67586234 +Node: Hierarchical ListBox7586316 +Ref: library/tkinter tix hierarchical-listbox7586431 +Ref: 2f2d7586431 +Ref: library/tkinter tix tkinter tix HList7586492 +Ref: 2f137586492 +Ref: library/tkinter tix tkinter tix CheckList7586777 +Ref: 2f2e7586777 +Ref: library/tkinter tix tkinter tix Tree7587065 +Ref: 2f2f7587065 +Ref: Hierarchical ListBox-Footnote-17587305 +Ref: Hierarchical ListBox-Footnote-27587383 +Ref: Hierarchical ListBox-Footnote-37587465 +Node: Tabular ListBox7587542 +Ref: library/tkinter tix tabular-listbox7587658 +Ref: 2f307587658 +Ref: library/tkinter tix tkinter tix TList7587709 +Ref: 2f317587709 +Ref: Tabular ListBox-Footnote-17588195 +Node: Manager Widgets7588273 +Ref: library/tkinter tix manager-widgets7588380 +Ref: 2f327588380 +Ref: library/tkinter tix tkinter tix PanedWindow7588431 +Ref: 2f187588431 +Ref: library/tkinter tix tkinter tix ListNoteBook7588738 +Ref: 2f337588738 +Ref: library/tkinter tix tkinter tix NoteBook7589189 +Ref: 2f167589189 +Ref: Manager Widgets-Footnote-17589600 +Ref: Manager Widgets-Footnote-27589684 +Ref: Manager Widgets-Footnote-37589769 +Node: Image Types7589850 +Ref: library/tkinter tix image-types7589963 +Ref: 2f347589963 +Ref: Image Types-Footnote-17590575 +Ref: Image Types-Footnote-27590651 +Node: Miscellaneous Widgets7590729 +Ref: library/tkinter tix miscellaneous-widgets7590848 +Ref: 2f357590848 +Ref: library/tkinter tix tkinter tix InputOnly7590911 +Ref: 2f367590911 +Ref: Miscellaneous Widgets-Footnote-17591112 +Node: Form Geometry Manager7591194 +Ref: library/tkinter tix form-geometry-manager7591293 +Ref: 2f377591293 +Ref: library/tkinter tix tkinter tix Form7591439 +Ref: 2f387591439 +Ref: Form Geometry Manager-Footnote-17591590 +Node: Tix Commands7591667 +Ref: library/tkinter tix tix-commands7591776 +Ref: 2f397591776 +Ref: library/tkinter tix tkinter tix tixCommand7591823 +Ref: 2f3a7591823 +Ref: library/tkinter tix tkinter tix tixCommand tix_configure7592329 +Ref: 2f3b7592329 +Ref: library/tkinter tix tkinter tix tixCommand tix_cget7593014 +Ref: 2f3c7593014 +Ref: library/tkinter tix tkinter tix tixCommand tix_getbitmap7593189 +Ref: 2f3d7593189 +Ref: library/tkinter tix tkinter tix tixCommand tix_addbitmapdir7593740 +Ref: 2f3e7593740 +Ref: library/tkinter tix tkinter tix tixCommand tix_filedialog7594274 +Ref: 2f407594274 +Ref: library/tkinter tix tkinter tix tixCommand tix_getimage7594854 +Ref: 2f3f7594854 +Ref: library/tkinter tix tkinter tix tixCommand tix_option_get7595644 +Ref: 2f417595644 +Ref: library/tkinter tix tkinter tix tixCommand tix_resetoptions7595753 +Ref: 2f427595753 +Ref: Tix Commands-Footnote-17596586 +Node: tkinter scrolledtext — Scrolled Text Widget7596658 +Ref: library/tkinter scrolledtext doc7596838 +Ref: 2f437596838 +Ref: library/tkinter scrolledtext module-tkinter scrolledtext7596838 +Ref: 10d7596838 +Ref: library/tkinter scrolledtext tkinter-scrolledtext-scrolled-text-widget7596838 +Ref: 2f447596838 +Ref: library/tkinter scrolledtext tkinter scrolledtext ScrolledText frame7597807 +Ref: 2f457597807 +Ref: library/tkinter scrolledtext tkinter scrolledtext ScrolledText vbar7597907 +Ref: 2f467597907 +Ref: tkinter scrolledtext — Scrolled Text Widget-Footnote-17598006 +Node: IDLE<3>7598086 +Ref: library/idle doc7598265 +Ref: 2f477598265 +Ref: library/idle id17598265 +Ref: 2f487598265 +Ref: library/idle idle7598265 +Ref: 94a7598265 +Ref: IDLE<3>-Footnote-17599295 +Node: Menus7599359 +Ref: library/idle menus7599439 +Ref: 2f497599439 +Node: File menu Shell and Editor7600721 +Ref: library/idle file-menu-shell-and-editor7600824 +Ref: 2f4a7600824 +Node: Edit menu Shell and Editor7602111 +Ref: library/idle edit-menu-shell-and-editor7602253 +Ref: 2f4b7602253 +Node: Format menu Editor window only7603994 +Ref: library/idle format-menu7604137 +Ref: 2f4e7604137 +Ref: library/idle format-menu-editor-window-only7604137 +Ref: 2f4f7604137 +Node: Run menu Editor window only7605506 +Ref: library/idle run-menu-editor-window-only7605651 +Ref: 2f507605651 +Ref: library/idle run-module7605730 +Ref: 2f517605730 +Ref: library/idle run-custom7606222 +Ref: 2f537606222 +Ref: library/idle check-module7606481 +Ref: 2f527606481 +Ref: library/idle python-shell7606825 +Ref: 2f547606825 +Node: Shell menu Shell window only7606887 +Ref: library/idle shell-menu-shell-window-only7607030 +Ref: 2f557607030 +Node: Debug menu Shell window only7607547 +Ref: library/idle debug-menu-shell-window-only7607692 +Ref: 2f567607692 +Node: Options menu Shell and Editor7608648 +Ref: library/idle options-menu-shell-and-editor7608793 +Ref: 2f577608793 +Node: Window menu Shell and Editor7610410 +Ref: library/idle window-menu-shell-and-editor7610553 +Ref: 2f5a7610553 +Node: Help menu Shell and Editor7610745 +Ref: library/idle help-menu-shell-and-editor7610872 +Ref: 2f5b7610872 +Node: Context Menus7611592 +Ref: library/idle context-menus7611682 +Ref: 2f5d7611682 +Node: Editing and navigation7612856 +Ref: library/idle editing-and-navigation7612971 +Ref: 2f5e7612971 +Ref: library/idle id27612971 +Ref: 2f5f7612971 +Node: Editor windows7613196 +Ref: library/idle editor-windows7613290 +Ref: 2f607613290 +Node: Key bindings7613912 +Ref: library/idle key-bindings7614036 +Ref: 2f617614036 +Node: Automatic indentation7615424 +Ref: library/idle automatic-indentation7615545 +Ref: 2f627615545 +Node: Completions7616125 +Ref: library/idle completions7616242 +Ref: 2f4c7616242 +Ref: library/idle id37616242 +Ref: 2f637616242 +Node: Calltips7618889 +Ref: library/idle calltips7618997 +Ref: 2f4d7618997 +Ref: library/idle id47618997 +Ref: 2f647618997 +Node: Code Context7620623 +Ref: library/idle code-context7620739 +Ref: 2f597620739 +Ref: library/idle id57620739 +Ref: 2f657620739 +Node: Python Shell window7621627 +Ref: library/idle python-shell-window7621746 +Ref: 2f667621746 +Node: Text colors7622872 +Ref: library/idle text-colors7622970 +Ref: 2f677622970 +Node: Startup and code execution7623733 +Ref: library/idle startup-and-code-execution7623863 +Ref: 2f687623863 +Node: Command line usage7624994 +Ref: library/idle command-line-usage7625099 +Ref: 2f697625099 +Node: Startup failure7626139 +Ref: library/idle startup-failure7626270 +Ref: 2f6a7626270 +Node: Running user code7629457 +Ref: library/idle running-user-code7629590 +Ref: 2f6b7629590 +Node: User output in Shell7632289 +Ref: library/idle user-output-in-shell7632438 +Ref: 2f6c7632438 +Node: Developing tkinter applications7635524 +Ref: library/idle developing-tkinter-applications7635684 +Ref: 2f6d7635684 +Node: Running without a subprocess7636933 +Ref: library/idle running-without-a-subprocess7637064 +Ref: 2f6e7637064 +Node: Help and preferences7638446 +Ref: library/idle help-and-preferences7638545 +Ref: 2f6f7638545 +Node: Help sources7638692 +Ref: library/idle help-sources7638789 +Ref: 2f5c7638789 +Ref: library/idle id67638789 +Ref: 2f707638789 +Node: Setting preferences7639659 +Ref: library/idle preferences7639778 +Ref: 2f587639778 +Ref: library/idle setting-preferences7639778 +Ref: 2f717639778 +Node: IDLE on macOS7640787 +Ref: library/idle idle-on-macos7640904 +Ref: 2f727640904 +Node: Extensions7641171 +Ref: library/idle extensions7641260 +Ref: 2f737641260 +Node: Other Graphical User Interface Packages7641611 +Ref: library/othergui doc7641736 +Ref: 2f747641736 +Ref: library/othergui other-graphical-user-interface-packages7641736 +Ref: 2f757641736 +Ref: library/othergui other-gui-packages7641736 +Ref: 2e807641736 +Ref: Other Graphical User Interface Packages-Footnote-17644204 +Ref: Other Graphical User Interface Packages-Footnote-27644254 +Ref: Other Graphical User Interface Packages-Footnote-37644306 +Ref: Other Graphical User Interface Packages-Footnote-47644335 +Ref: Other Graphical User Interface Packages-Footnote-57644389 +Ref: Other Graphical User Interface Packages-Footnote-67644419 +Ref: Other Graphical User Interface Packages-Footnote-77644450 +Ref: Other Graphical User Interface Packages-Footnote-87644505 +Ref: Other Graphical User Interface Packages-Footnote-97644564 +Ref: Other Graphical User Interface Packages-Footnote-107644603 +Ref: Other Graphical User Interface Packages-Footnote-117644637 +Ref: Other Graphical User Interface Packages-Footnote-127644673 +Node: Development Tools7644726 +Ref: library/development doc7644881 +Ref: 2f767644881 +Ref: library/development development7644881 +Ref: 2f777644881 +Ref: library/development development-tools7644881 +Ref: 2f787644881 +Node: typing — Support for type hints7646130 +Ref: library/typing doc7646282 +Ref: 2f797646282 +Ref: library/typing module-typing7646282 +Ref: 1197646282 +Ref: library/typing typing-support-for-type-hints7646282 +Ref: 2f7a7646282 +Ref: typing — Support for type hints-Footnote-17647674 +Ref: typing — Support for type hints-Footnote-27647739 +Ref: typing — Support for type hints-Footnote-37647788 +Ref: typing — Support for type hints-Footnote-47647837 +Ref: typing — Support for type hints-Footnote-57647886 +Ref: typing — Support for type hints-Footnote-67647935 +Ref: typing — Support for type hints-Footnote-77647984 +Ref: typing — Support for type hints-Footnote-87648033 +Ref: typing — Support for type hints-Footnote-97648082 +Node: Type aliases7648131 +Ref: library/typing type-aliases7648229 +Ref: 2f807648229 +Node: NewType7649401 +Ref: library/typing distinct7649516 +Ref: 2f817649516 +Ref: library/typing newtype7649516 +Ref: 2f827649516 +Ref: NewType-Footnote-17652392 +Node: Callable7652441 +Ref: library/typing callable7652552 +Ref: 2f837652552 +Node: Generics7653208 +Ref: library/typing generics7653338 +Ref: 2f847653338 +Ref: library/typing id17653338 +Ref: 2f857653338 +Node: User-defined generic types7654039 +Ref: library/typing user-defined-generic-types7654173 +Ref: 2f867654173 +Node: The Any type7657443 +Ref: library/typing the-any-type7657600 +Ref: 2f877657600 +Node: Nominal vs structural subtyping7660188 +Ref: library/typing nominal-vs-structural-subtyping7660351 +Ref: 2f887660351 +Ref: Nominal vs structural subtyping-Footnote-17662003 +Ref: Nominal vs structural subtyping-Footnote-27662052 +Ref: Nominal vs structural subtyping-Footnote-37662101 +Node: Classes functions and decorators7662150 +Ref: library/typing classes-functions-and-decorators7662292 +Ref: 2f897662292 +Ref: library/typing typing TypeVar7662450 +Ref: 2f7e7662450 +Ref: library/typing typing Generic7664112 +Ref: 2f7f7664112 +Ref: library/typing typing Protocol7664819 +Ref: 23f7664819 +Ref: library/typing typing Type7665804 +Ref: 2f8a7665804 +Ref: library/typing typing Iterable7667522 +Ref: 2f8b7667522 +Ref: library/typing typing Iterator7667631 +Ref: 2f8c7667631 +Ref: library/typing typing Reversible7667741 +Ref: 2f8d7667741 +Ref: library/typing typing SupportsInt7667854 +Ref: 2417667854 +Ref: library/typing typing SupportsFloat7667938 +Ref: 2f8e7667938 +Ref: library/typing typing SupportsComplex7668026 +Ref: 2f8f7668026 +Ref: library/typing typing SupportsBytes7668118 +Ref: 2f907668118 +Ref: library/typing typing SupportsIndex7668206 +Ref: 2427668206 +Ref: library/typing typing SupportsAbs7668320 +Ref: 2f917668320 +Ref: library/typing typing SupportsRound7668446 +Ref: 2f927668446 +Ref: library/typing typing Container7668576 +Ref: 2f937668576 +Ref: library/typing typing Hashable7668687 +Ref: 2f947668687 +Ref: library/typing typing Sized7668771 +Ref: 2f957668771 +Ref: library/typing typing Collection7668849 +Ref: 5a27668849 +Ref: library/typing typing AbstractSet7669014 +Ref: 2f967669014 +Ref: library/typing typing MutableSet7669131 +Ref: 2f977669131 +Ref: library/typing typing Mapping7669245 +Ref: 2f987669245 +Ref: library/typing typing MutableMapping7669539 +Ref: 2f997669539 +Ref: library/typing typing Sequence7669661 +Ref: 2f9a7669661 +Ref: library/typing typing MutableSequence7669791 +Ref: 2f9b7669791 +Ref: library/typing typing ByteString7669911 +Ref: 2f9c7669911 +Ref: library/typing typing Deque7670282 +Ref: 2f9d7670282 +Ref: library/typing typing List7670448 +Ref: 2f9e7670448 +Ref: library/typing typing Set7670996 +Ref: 2f9f7670996 +Ref: library/typing typing FrozenSet7671246 +Ref: 2fa07671246 +Ref: library/typing typing MappingView7671364 +Ref: 2fa17671364 +Ref: library/typing typing KeysView7671487 +Ref: 2fa27671487 +Ref: library/typing typing ItemsView7671621 +Ref: 2fa37671621 +Ref: library/typing typing ValuesView7671753 +Ref: 2fa47671753 +Ref: library/typing typing Awaitable7671871 +Ref: 2fa57671871 +Ref: library/typing typing Coroutine7672009 +Ref: 2fa67672009 +Ref: library/typing typing AsyncIterable7672537 +Ref: 2fa87672537 +Ref: library/typing typing AsyncIterator7672683 +Ref: 2fa97672683 +Ref: library/typing typing ContextManager7672835 +Ref: 5357672835 +Ref: library/typing typing AsyncContextManager7673014 +Ref: 2faa7673014 +Ref: library/typing typing Dict7673208 +Ref: 2fab7673208 +Ref: library/typing typing DefaultDict7673572 +Ref: 2fac7673572 +Ref: library/typing typing OrderedDict7673754 +Ref: 2fad7673754 +Ref: library/typing typing Counter7673936 +Ref: 2fae7673936 +Ref: library/typing typing ChainMap7674114 +Ref: 2faf7674114 +Ref: library/typing typing Generator7674315 +Ref: 2fa77674315 +Ref: library/typing typing AsyncGenerator7675450 +Ref: 2fb07675450 +Ref: library/typing typing Text7676713 +Ref: 2fb17676713 +Ref: library/typing typing IO7677187 +Ref: 2fb27677187 +Ref: library/typing typing TextIO7677208 +Ref: 2fb37677208 +Ref: library/typing typing BinaryIO7677233 +Ref: 2fb47677233 +Ref: library/typing typing Pattern7677452 +Ref: 2fb57677452 +Ref: library/typing typing Match7677478 +Ref: 2fb67677478 +Ref: library/typing typing NamedTuple7677830 +Ref: 2807677830 +Ref: library/typing typing TypedDict7679770 +Ref: 23b7679770 +Ref: library/typing typing ForwardRef7681568 +Ref: 2fb77681568 +Ref: library/typing typing NewType7681926 +Ref: 5a57681926 +Ref: library/typing typing cast7682245 +Ref: 2fb87682245 +Ref: library/typing typing get_type_hints7682546 +Ref: 30a7682546 +Ref: library/typing typing get_origin7683197 +Ref: 2437683197 +Ref: library/typing typing get_args7683235 +Ref: 2447683235 +Ref: library/typing typing overload7683936 +Ref: 2fb97683936 +Ref: library/typing typing final7685204 +Ref: 23e7685204 +Ref: library/typing typing no_type_check7685884 +Ref: 2fba7685884 +Ref: library/typing typing no_type_check_decorator7686240 +Ref: 2fbb7686240 +Ref: library/typing typing type_check_only7686492 +Ref: 2fbc7686492 +Ref: library/typing typing runtime_checkable7687167 +Ref: 2407687167 +Ref: library/typing typing Any7687899 +Ref: 6527687899 +Ref: library/typing typing NoReturn7688089 +Ref: 2fbd7688089 +Ref: library/typing typing Union7688369 +Ref: 2f7b7688369 +Ref: library/typing typing Optional7689343 +Ref: 2fbe7689343 +Ref: library/typing typing Tuple7690033 +Ref: 2f7c7690033 +Ref: library/typing typing Callable7690671 +Ref: 2f7d7690671 +Ref: library/typing typing Literal7691435 +Ref: 23c7691435 +Ref: library/typing typing ClassVar7692301 +Ref: 5a37692301 +Ref: library/typing typing Final7693388 +Ref: 23d7693388 +Ref: library/typing typing AnyStr7693942 +Ref: 2fbf7693942 +Ref: library/typing typing TYPE_CHECKING7694506 +Ref: 5a47694506 +Ref: Classes functions and decorators-Footnote-17695248 +Ref: Classes functions and decorators-Footnote-27695297 +Ref: Classes functions and decorators-Footnote-37695346 +Ref: Classes functions and decorators-Footnote-47695395 +Ref: Classes functions and decorators-Footnote-57695444 +Ref: Classes functions and decorators-Footnote-67695493 +Ref: Classes functions and decorators-Footnote-77695542 +Ref: Classes functions and decorators-Footnote-87695591 +Ref: Classes functions and decorators-Footnote-97695640 +Ref: Classes functions and decorators-Footnote-107695689 +Ref: Classes functions and decorators-Footnote-117695739 +Ref: Classes functions and decorators-Footnote-127695789 +Node: pydoc — Documentation generator and online help system7695839 +Ref: library/pydoc doc7696044 +Ref: 2fc07696044 +Ref: library/pydoc module-pydoc7696044 +Ref: d97696044 +Ref: library/pydoc pydoc-documentation-generator-and-online-help-system7696044 +Ref: 2fc17696044 +Ref: pydoc — Documentation generator and online help system-Footnote-17700938 +Node: doctest — Test interactive Python examples7701002 +Ref: library/doctest doc7701209 +Ref: 2fc37701209 +Ref: library/doctest doctest-test-interactive-python-examples7701209 +Ref: 2fc47701209 +Ref: library/doctest module-doctest7701209 +Ref: 677701209 +Ref: doctest — Test interactive Python examples-Footnote-17705325 +Node: Simple Usage Checking Examples in Docstrings7705391 +Ref: library/doctest doctest-simple-testmod7705570 +Ref: 2fc57705570 +Ref: library/doctest simple-usage-checking-examples-in-docstrings7705570 +Ref: 2fc67705570 +Node: Simple Usage Checking Examples in a Text File7707346 +Ref: library/doctest doctest-simple-testfile7707546 +Ref: 2fc87707546 +Ref: library/doctest simple-usage-checking-examples-in-a-text-file7707546 +Ref: 2fc97707546 +Node: How It Works7709892 +Ref: library/doctest doctest-how-it-works7710057 +Ref: 2fcb7710057 +Ref: library/doctest how-it-works7710057 +Ref: 2fcc7710057 +Node: Which Docstrings Are Examined?7710712 +Ref: library/doctest doctest-which-docstrings7710838 +Ref: 2fcd7710838 +Ref: library/doctest which-docstrings-are-examined7710838 +Ref: 2fce7710838 +Node: How are Docstring Examples Recognized?7711685 +Ref: library/doctest doctest-finding-examples7711851 +Ref: 2fcf7711851 +Ref: library/doctest how-are-docstring-examples-recognized7711851 +Ref: 2fd07711851 +Node: What’s the Execution Context?7714939 +Ref: library/doctest doctest-execution-context7715097 +Ref: 2fd47715097 +Ref: library/doctest what-s-the-execution-context7715097 +Ref: 2fd57715097 +Node: What About Exceptions?7715814 +Ref: library/doctest doctest-exceptions7715946 +Ref: 2fd67715946 +Ref: library/doctest what-about-exceptions7715946 +Ref: 2fd77715946 +Ref: library/doctest option-flags-and-directives7720454 +Ref: 2fd97720454 +Ref: What About Exceptions?-Footnote-17720491 +Node: Option Flags7720695 +Ref: library/doctest doctest-options7720806 +Ref: 83c7720806 +Ref: library/doctest option-flags7720806 +Ref: 2fda7720806 +Ref: library/doctest doctest DONT_ACCEPT_TRUE_FOR_17721424 +Ref: 2fdb7721424 +Ref: library/doctest doctest DONT_ACCEPT_BLANKLINE7722052 +Ref: 2fdc7722052 +Ref: library/doctest doctest NORMALIZE_WHITESPACE7722505 +Ref: 2fd17722505 +Ref: library/doctest doctest ELLIPSIS7722995 +Ref: dd17722995 +Ref: library/doctest doctest IGNORE_EXCEPTION_DETAIL7723431 +Ref: 2fd87723431 +Ref: library/doctest doctest SKIP7725475 +Ref: 2fdd7725475 +Ref: library/doctest doctest COMPARISON_FLAGS7725989 +Ref: 2fde7725989 +Ref: library/doctest doctest REPORT_UDIFF7726161 +Ref: dd27726161 +Ref: library/doctest doctest REPORT_CDIFF7726317 +Ref: dd37726317 +Ref: library/doctest doctest REPORT_NDIFF7726477 +Ref: dd47726477 +Ref: library/doctest doctest REPORT_ONLY_FIRST_FAILURE7726941 +Ref: 2fdf7726941 +Ref: library/doctest doctest FAIL_FAST7727520 +Ref: 83d7727520 +Ref: library/doctest doctest REPORTING_FLAGS7727987 +Ref: 2fe07727987 +Ref: library/doctest doctest register_optionflag7728244 +Ref: 2fe17728244 +Node: Directives7728734 +Ref: library/doctest directives7728834 +Ref: 2fe47728834 +Ref: library/doctest doctest-directives7728834 +Ref: 2fd27728834 +Ref: library/doctest grammar-token-directive7729059 +Ref: 2fe57729059 +Ref: library/doctest grammar-token-directive-options7729123 +Ref: 2fe67729123 +Ref: library/doctest grammar-token-directive-option7729196 +Ref: 2fe77729196 +Ref: library/doctest grammar-token-on-or-off7729259 +Ref: 2fe87729259 +Ref: library/doctest grammar-token-directive-option-name7729301 +Ref: 2fe97729301 +Node: Warnings<2>7731203 +Ref: library/doctest doctest-warnings7731282 +Ref: 2fea7731282 +Ref: library/doctest warnings7731282 +Ref: 2feb7731282 +Node: Basic API7733171 +Ref: library/doctest basic-api7733303 +Ref: 2fec7733303 +Ref: library/doctest doctest-basic-api7733303 +Ref: 2fc77733303 +Ref: library/doctest doctest testfile7733686 +Ref: 2fca7733686 +Ref: library/doctest doctest testmod7737618 +Ref: dd07737618 +Ref: library/doctest doctest run_docstring_examples7739364 +Ref: 2fee7739364 +Node: Unittest API7740342 +Ref: library/doctest doctest-unittest-api7740474 +Ref: 2fef7740474 +Ref: library/doctest unittest-api7740474 +Ref: 2ff07740474 +Ref: library/doctest doctest DocFileSuite7741244 +Ref: 2ff17741244 +Ref: library/doctest doctest DocTestSuite7744968 +Ref: 6cd7744968 +Ref: library/doctest doctest set_unittest_reportflags7748118 +Ref: 2ff37748118 +Node: Advanced API7749314 +Ref: library/doctest advanced-api7749446 +Ref: 2ff47749446 +Ref: library/doctest doctest-advanced-api7749446 +Ref: 2ff57749446 +Node: DocTest Objects7751587 +Ref: library/doctest doctest-doctest7751675 +Ref: 2ff77751675 +Ref: library/doctest doctest-objects7751675 +Ref: 2ff87751675 +Ref: library/doctest doctest DocTest7751728 +Ref: 2ff27751728 +Ref: library/doctest doctest DocTest examples7752144 +Ref: 2ff97752144 +Ref: library/doctest doctest DocTest globs7752319 +Ref: 2ffa7752319 +Ref: library/doctest doctest DocTest name7752654 +Ref: 2ffb7752654 +Ref: library/doctest doctest DocTest filename7752847 +Ref: 2ffc7752847 +Ref: library/doctest doctest DocTest lineno7753084 +Ref: 2ffd7753084 +Ref: library/doctest doctest DocTest docstring7753362 +Ref: 2ffe7753362 +Node: Example Objects7753561 +Ref: library/doctest doctest-example7753679 +Ref: 2fff7753679 +Ref: library/doctest example-objects7753679 +Ref: 30007753679 +Ref: library/doctest doctest Example7753732 +Ref: 2ff67753732 +Ref: library/doctest doctest Example source7754171 +Ref: 30017754171 +Ref: library/doctest doctest Example want7754415 +Ref: 30027754415 +Ref: library/doctest doctest Example exc_msg7754766 +Ref: 30037754766 +Ref: library/doctest doctest Example lineno7755251 +Ref: 30057755251 +Ref: library/doctest doctest Example indent7755482 +Ref: 30067755482 +Ref: library/doctest doctest Example options7755675 +Ref: 30077755675 +Node: DocTestFinder objects7756071 +Ref: library/doctest doctest-doctestfinder7756195 +Ref: 30087756195 +Ref: library/doctest doctestfinder-objects7756195 +Ref: 30097756195 +Ref: library/doctest doctest DocTestFinder7756260 +Ref: 2fed7756260 +Ref: library/doctest doctest DocTestFinder find7757399 +Ref: 300a7757399 +Node: DocTestParser objects7759518 +Ref: library/doctest doctest-doctestparser7759648 +Ref: 300b7759648 +Ref: library/doctest doctestparser-objects7759648 +Ref: 300c7759648 +Ref: library/doctest doctest DocTestParser7759713 +Ref: 2fd37759713 +Ref: library/doctest doctest DocTestParser get_doctest7759947 +Ref: 300d7759947 +Ref: library/doctest doctest DocTestParser get_examples7760342 +Ref: 300e7760342 +Ref: library/doctest doctest DocTestParser parse7760680 +Ref: 300f7760680 +Node: DocTestRunner objects7761076 +Ref: library/doctest doctest-doctestrunner7761206 +Ref: 30107761206 +Ref: library/doctest doctestrunner-objects7761206 +Ref: 30117761206 +Ref: library/doctest doctest DocTestRunner7761271 +Ref: 2fe37761271 +Ref: library/doctest doctest DocTestRunner report_start7763379 +Ref: 30127763379 +Ref: library/doctest doctest DocTestRunner report_success7763878 +Ref: 30137763878 +Ref: library/doctest doctest DocTestRunner report_failure7764411 +Ref: 30157764411 +Ref: library/doctest doctest DocTestRunner report_unexpected_exception7764934 +Ref: 30147764934 +Ref: library/doctest doctest DocTestRunner run7765609 +Ref: 30167765609 +Ref: library/doctest doctest DocTestRunner summarize7766625 +Ref: 30177766625 +Node: OutputChecker objects7767050 +Ref: library/doctest doctest-outputchecker7767150 +Ref: 30187767150 +Ref: library/doctest outputchecker-objects7767150 +Ref: 30197767150 +Ref: library/doctest doctest OutputChecker7767215 +Ref: 2fe27767215 +Ref: library/doctest doctest OutputChecker check_output7767711 +Ref: 301a7767711 +Ref: library/doctest doctest OutputChecker output_difference7768198 +Ref: 301b7768198 +Node: Debugging7768513 +Ref: library/doctest debugging7768640 +Ref: 301c7768640 +Ref: library/doctest doctest-debugging7768640 +Ref: 301d7768640 +Ref: library/doctest doctest script_from_examples7770739 +Ref: 30207770739 +Ref: library/doctest doctest testsource7771726 +Ref: 30217771726 +Ref: library/doctest doctest debug7772523 +Ref: 301f7772523 +Ref: library/doctest doctest debug_src7773613 +Ref: 30237773613 +Ref: library/doctest doctest DebugRunner7774487 +Ref: 301e7774487 +Ref: library/doctest doctest DocTestFailure7775264 +Ref: 30257775264 +Ref: library/doctest doctest DocTestFailure test7775627 +Ref: 30267775627 +Ref: library/doctest doctest DocTestFailure example7775750 +Ref: 30277775750 +Ref: library/doctest doctest DocTestFailure got7775833 +Ref: 30287775833 +Ref: library/doctest doctest UnexpectedException7775905 +Ref: 30247775905 +Ref: library/doctest doctest UnexpectedException test7776258 +Ref: 30297776258 +Ref: library/doctest doctest UnexpectedException example7776386 +Ref: 302a7776386 +Ref: library/doctest doctest UnexpectedException exc_info7776474 +Ref: 302b7776474 +Node: Soapbox7776635 +Ref: library/doctest doctest-soapbox7776741 +Ref: 302c7776741 +Ref: library/doctest soapbox7776741 +Ref: 302d7776741 +Node: unittest — Unit testing framework7780585 +Ref: library/unittest doc7780773 +Ref: 302e7780773 +Ref: library/unittest module-unittest7780773 +Ref: 11b7780773 +Ref: library/unittest unittest-unit-testing-framework7780773 +Ref: 302f7780773 +Ref: unittest — Unit testing framework-Footnote-17783930 +Ref: unittest — Unit testing framework-Footnote-27784007 +Ref: unittest — Unit testing framework-Footnote-37784100 +Ref: unittest — Unit testing framework-Footnote-47784133 +Ref: unittest — Unit testing framework-Footnote-57784197 +Ref: unittest — Unit testing framework-Footnote-67784255 +Ref: unittest — Unit testing framework-Footnote-77784285 +Ref: unittest — Unit testing framework-Footnote-87784313 +Ref: unittest — Unit testing framework-Footnote-97784343 +Node: Basic example7784377 +Ref: library/unittest basic-example7784496 +Ref: 30317784496 +Ref: library/unittest unittest-minimal-example7784496 +Ref: 30327784496 +Node: Command-Line Interface<3>7787300 +Ref: library/unittest command-line-interface7787442 +Ref: 30347787442 +Ref: library/unittest unittest-command-line-interface7787442 +Ref: 30357787442 +Node: Command-line options<3>7788838 +Ref: library/unittest command-line-options7788923 +Ref: 30377788923 +Ref: library/unittest cmdoption-unittest-b7789037 +Ref: cc37789037 +Ref: library/unittest cmdoption-unittest-c7789304 +Ref: cc47789304 +Ref: library/unittest cmdoption-unittest-f7789644 +Ref: cc67789644 +Ref: library/unittest cmdoption-unittest-k7789735 +Ref: 30397789735 +Ref: library/unittest cmdoption-unittest-locals7790451 +Ref: 303a7790451 +Node: Test Discovery7790844 +Ref: library/unittest test-discovery7790993 +Ref: 303b7790993 +Ref: library/unittest unittest-test-discovery7790993 +Ref: 30367790993 +Ref: library/unittest cmdoption-unittest-discover-v7791888 +Ref: 303c7791888 +Ref: library/unittest cmdoption-unittest-discover-s7791944 +Ref: 303d7791944 +Ref: library/unittest cmdoption-unittest-discover-p7792050 +Ref: 303e7792050 +Ref: library/unittest cmdoption-unittest-discover-t7792152 +Ref: 303f7792152 +Node: Organizing test code7793925 +Ref: library/unittest organizing-test-code7794071 +Ref: 30417794071 +Ref: library/unittest organizing-tests7794071 +Ref: 30337794071 +Node: Re-using old test code7798825 +Ref: library/unittest legacy-unit-tests7798993 +Ref: 30437798993 +Ref: library/unittest re-using-old-test-code7798993 +Ref: 30447798993 +Node: Skipping tests and expected failures7800533 +Ref: library/unittest skipping-tests-and-expected-failures7800726 +Ref: 30457800726 +Ref: library/unittest unittest-skipping7800726 +Ref: 30467800726 +Ref: library/unittest unittest skip7803814 +Ref: 30477803814 +Ref: library/unittest unittest skipIf7803962 +Ref: 304a7803962 +Ref: library/unittest unittest skipUnless7804068 +Ref: 304b7804068 +Ref: library/unittest unittest expectedFailure7804182 +Ref: 30497804182 +Ref: library/unittest unittest SkipTest7804402 +Ref: 9037804402 +Node: Distinguishing test iterations using subtests7804890 +Ref: library/unittest distinguishing-test-iterations-using-subtests7805082 +Ref: 304c7805082 +Ref: library/unittest subtests7805082 +Ref: 9017805082 +Node: Classes and functions7807376 +Ref: library/unittest classes-and-functions7807557 +Ref: 304d7807557 +Ref: library/unittest unittest-contents7807557 +Ref: 304e7807557 +Node: Test cases7807759 +Ref: library/unittest test-cases7807850 +Ref: 304f7807850 +Ref: library/unittest testcase-objects7807850 +Ref: 30507807850 +Ref: library/unittest unittest TestCase7807893 +Ref: 8ff7807893 +Ref: library/unittest unittest TestCase setUp7809215 +Ref: ccb7809215 +Ref: library/unittest unittest TestCase tearDown7809593 +Ref: ccc7809593 +Ref: library/unittest unittest TestCase setUpClass7810367 +Ref: 24c7810367 +Ref: library/unittest unittest TestCase tearDownClass7810806 +Ref: cc97810806 +Ref: library/unittest unittest TestCase run7811254 +Ref: abe7811254 +Ref: library/unittest unittest TestCase skipTest7811881 +Ref: 30487811881 +Ref: library/unittest unittest TestCase subTest7812137 +Ref: 9007812137 +Ref: library/unittest unittest TestCase debug7812692 +Ref: 30537812692 +Ref: library/unittest assert-methods7812927 +Ref: 30307812927 +Ref: library/unittest unittest TestCase assertEqual7816774 +Ref: bb57816774 +Ref: library/unittest unittest TestCase assertNotEqual7817644 +Ref: bb67817644 +Ref: library/unittest unittest TestCase assertTrue7817823 +Ref: bb47817823 +Ref: library/unittest unittest TestCase assertFalse7817868 +Ref: cc77817868 +Ref: library/unittest unittest TestCase assertIs7818369 +Ref: ccf7818369 +Ref: library/unittest unittest TestCase assertIsNot7818421 +Ref: cd07818421 +Ref: library/unittest unittest TestCase assertIsNone7818594 +Ref: ccd7818594 +Ref: library/unittest unittest TestCase assertIsNotNone7818641 +Ref: cce7818641 +Ref: library/unittest unittest TestCase assertIn7818778 +Ref: cd87818778 +Ref: library/unittest unittest TestCase assertNotIn7818834 +Ref: cd97818834 +Ref: library/unittest unittest TestCase assertIsInstance7818986 +Ref: cd17818986 +Ref: library/unittest unittest TestCase assertNotIsInstance7819041 +Ref: cd27819041 +Ref: library/unittest unittest TestCase assertRaises7821714 +Ref: aba7821714 +Ref: library/unittest unittest TestCase assertRaisesRegex7823441 +Ref: abb7823441 +Ref: library/unittest unittest TestCase assertWarns7824438 +Ref: library/unittest unittest TestCase assertWarnsRegex7826149 +Ref: abd7826149 +Ref: library/unittest unittest TestCase assertLogs7827084 +Ref: 9057827084 +Ref: library/unittest unittest TestCase records7828137 +Ref: 30557828137 +Ref: library/unittest unittest TestCase output7828280 +Ref: 30567828280 +Ref: library/unittest unittest TestCase assertAlmostEqual7831605 +Ref: bb77831605 +Ref: library/unittest unittest TestCase assertNotAlmostEqual7831703 +Ref: bb87831703 +Ref: library/unittest unittest TestCase assertGreater7832774 +Ref: cd37832774 +Ref: library/unittest unittest TestCase assertGreaterEqual7832831 +Ref: cd47832831 +Ref: library/unittest unittest TestCase assertLess7832893 +Ref: cd57832893 +Ref: library/unittest unittest TestCase assertLessEqual7832947 +Ref: cd67832947 +Ref: library/unittest unittest TestCase assertRegex7833312 +Ref: bb17833312 +Ref: library/unittest unittest TestCase assertNotRegex7833365 +Ref: 30577833365 +Ref: library/unittest unittest TestCase assertCountEqual7834238 +Ref: baf7834238 +Ref: library/unittest type-specific-methods7834899 +Ref: 30547834899 +Ref: library/unittest unittest TestCase addTypeEqualityFunc7835217 +Ref: ce17835217 +Ref: library/unittest unittest TestCase assertMultiLineEqual7837961 +Ref: cd77837961 +Ref: library/unittest unittest TestCase assertSequenceEqual7838376 +Ref: cdd7838376 +Ref: library/unittest unittest TestCase assertListEqual7839001 +Ref: cdb7839001 +Ref: library/unittest unittest TestCase assertTupleEqual7839060 +Ref: cdc7839060 +Ref: library/unittest unittest TestCase assertSetEqual7839508 +Ref: cda7839508 +Ref: library/unittest unittest TestCase assertDictEqual7839960 +Ref: cde7839960 +Ref: library/unittest other-methods-and-attrs7840318 +Ref: 30597840318 +Ref: library/unittest unittest TestCase fail7840409 +Ref: 305a7840409 +Ref: library/unittest unittest TestCase failureException7840552 +Ref: 30587840552 +Ref: library/unittest unittest TestCase longMessage7840968 +Ref: cc87840968 +Ref: library/unittest unittest TestCase maxDiff7841706 +Ref: bb07841706 +Ref: library/unittest unittest TestCase countTestCases7842387 +Ref: 305b7842387 +Ref: library/unittest unittest TestCase defaultTestResult7842569 +Ref: 30527842569 +Ref: library/unittest unittest TestCase id7842999 +Ref: 305c7842999 +Ref: library/unittest unittest TestCase shortDescription7843194 +Ref: 305d7843194 +Ref: library/unittest unittest TestCase addCleanup7843814 +Ref: 2a27843814 +Ref: library/unittest unittest TestCase doCleanups7844414 +Ref: cca7844414 +Ref: library/unittest unittest TestCase addClassCleanup7845052 +Ref: 24b7845052 +Ref: library/unittest unittest TestCase doClassCleanups7845724 +Ref: 305f7845724 +Ref: library/unittest unittest IsolatedAsyncioTestCase7846426 +Ref: 24d7846426 +Ref: library/unittest unittest IsolatedAsyncioTestCase asyncSetUp7846637 +Ref: 30607846637 +Ref: library/unittest unittest IsolatedAsyncioTestCase asyncTearDown7847077 +Ref: 30617847077 +Ref: library/unittest unittest IsolatedAsyncioTestCase addAsyncCleanup7847923 +Ref: 30627847923 +Ref: library/unittest unittest IsolatedAsyncioTestCase run7848081 +Ref: 30637848081 +Ref: library/unittest unittest FunctionTestCase7849818 +Ref: 30427849818 +Node: Deprecated aliases7850329 +Ref: library/unittest deprecated-aliases7850394 +Ref: bb97850394 +Ref: library/unittest id17850394 +Ref: 30647850394 +Node: Grouping tests7853523 +Ref: library/unittest grouping-tests7853648 +Ref: 30657853648 +Ref: library/unittest testsuite-objects7853648 +Ref: 30667853648 +Ref: library/unittest unittest TestSuite7853699 +Ref: 6ce7853699 +Ref: library/unittest unittest TestSuite addTest7854646 +Ref: 30677854646 +Ref: library/unittest unittest TestSuite addTests7854766 +Ref: 30687854766 +Ref: library/unittest unittest TestSuite run7855142 +Ref: 30697855142 +Ref: library/unittest unittest TestSuite debug7855444 +Ref: 306a7855444 +Ref: library/unittest unittest TestSuite countTestCases7855717 +Ref: 306b7855717 +Ref: library/unittest unittest TestSuite __iter__7855881 +Ref: 9627855881 +Node: Loading and running tests7857337 +Ref: library/unittest loading-and-running-tests7857443 +Ref: 306c7857443 +Ref: library/unittest unittest TestLoader7857516 +Ref: 7827857516 +Ref: library/unittest unittest TestLoader errors7858010 +Ref: 7817858010 +Ref: library/unittest unittest TestLoader loadTestsFromTestCase7858479 +Ref: 306e7858479 +Ref: library/unittest unittest TestLoader loadTestsFromModule7859030 +Ref: 7807859030 +Ref: library/unittest unittest TestLoader loadTestsFromName7860535 +Ref: cdf7860535 +Ref: library/unittest unittest TestLoader loadTestsFromNames7862335 +Ref: 30707862335 +Ref: library/unittest unittest TestLoader getTestCaseNames7862620 +Ref: 306f7862620 +Ref: library/unittest unittest TestLoader discover7862823 +Ref: 9047862823 +Ref: library/unittest unittest TestLoader testMethodPrefix7865689 +Ref: 30717865689 +Ref: library/unittest unittest TestLoader sortTestMethodsUsing7865978 +Ref: 30727865978 +Ref: library/unittest unittest TestLoader suiteClass7866192 +Ref: ce07866192 +Ref: library/unittest unittest TestLoader testNamePatterns7866489 +Ref: 30737866489 +Ref: library/unittest unittest TestResult7867203 +Ref: abf7867203 +Ref: library/unittest unittest TestResult errors7868065 +Ref: 30747868065 +Ref: library/unittest unittest TestResult failures7868295 +Ref: 30757868295 +Ref: library/unittest unittest TestResult skipped7868584 +Ref: 30767868584 +Ref: library/unittest unittest TestResult expectedFailures7868780 +Ref: 30777868780 +Ref: library/unittest unittest TestResult unexpectedSuccesses7869022 +Ref: 30787869022 +Ref: library/unittest unittest TestResult shouldStop7869187 +Ref: 30797869187 +Ref: library/unittest unittest TestResult testsRun7869321 +Ref: 307b7869321 +Ref: library/unittest unittest TestResult buffer7869400 +Ref: 307c7869400 +Ref: library/unittest unittest TestResult failfast7869849 +Ref: 307f7869849 +Ref: library/unittest unittest TestResult tb_locals7870034 +Ref: 30807870034 +Ref: library/unittest unittest TestResult wasSuccessful7870182 +Ref: 30817870182 +Ref: library/unittest unittest TestResult stop7870519 +Ref: 307a7870519 +Ref: library/unittest unittest TestResult startTest7871478 +Ref: 307d7871478 +Ref: library/unittest unittest TestResult stopTest7871577 +Ref: 307e7871577 +Ref: library/unittest unittest TestResult startTestRun7871712 +Ref: ce37871712 +Ref: library/unittest unittest TestResult stopTestRun7871831 +Ref: ce47871831 +Ref: library/unittest unittest TestResult addError7871948 +Ref: 30837871948 +Ref: library/unittest unittest TestResult addFailure7872415 +Ref: 30847872415 +Ref: library/unittest unittest TestResult addSuccess7872873 +Ref: 30857872873 +Ref: library/unittest unittest TestResult addSkip7873015 +Ref: 30867873015 +Ref: library/unittest unittest TestResult addExpectedFailure7873311 +Ref: 30877873311 +Ref: library/unittest unittest TestResult addUnexpectedSuccess7873737 +Ref: 30887873737 +Ref: library/unittest unittest TestResult addSubTest7874043 +Ref: 30897874043 +Ref: library/unittest unittest TextTestResult7874715 +Ref: 305e7874715 +Ref: library/unittest unittest defaultTestLoader7875037 +Ref: 306d7875037 +Ref: library/unittest unittest TextTestRunner7875292 +Ref: 30827875292 +Ref: library/unittest unittest TextTestRunner _makeResult7876888 +Ref: 308b7876888 +Ref: library/unittest unittest TextTestRunner run7877540 +Ref: 308c7877540 +Ref: library/unittest unittest main7877901 +Ref: 9027877901 +Node: load_tests Protocol7880769 +Ref: library/unittest load-tests-protocol7880850 +Ref: 30407880850 +Node: Class and Module Fixtures7883522 +Ref: library/unittest class-and-module-fixtures7883673 +Ref: 30517883673 +Node: setUpClass and tearDownClass7885561 +Ref: library/unittest setupclass-and-teardownclass7885690 +Ref: 308d7885690 +Node: setUpModule and tearDownModule7886604 +Ref: library/unittest setupmodule-and-teardownmodule7886733 +Ref: 308e7886733 +Ref: library/unittest unittest addModuleCleanup7887346 +Ref: 24a7887346 +Ref: library/unittest unittest doModuleCleanups7887924 +Ref: 308f7887924 +Node: Signal Handling7888537 +Ref: library/unittest signal-handling7888658 +Ref: 30387888658 +Ref: library/unittest unittest installHandler7889892 +Ref: 30907889892 +Ref: library/unittest unittest registerResult7890137 +Ref: 30917890137 +Ref: library/unittest unittest removeResult7890631 +Ref: 30927890631 +Ref: library/unittest unittest removeHandler7890853 +Ref: cc57890853 +Node: unittest mock — mock object library7891243 +Ref: library/unittest mock doc7891420 +Ref: 30937891420 +Ref: library/unittest mock module-unittest mock7891420 +Ref: 11c7891420 +Ref: library/unittest mock unittest-mock-mock-object-library7891420 +Ref: 30947891420 +Ref: unittest mock — mock object library-Footnote-17892982 +Ref: unittest mock — mock object library-Footnote-27893054 +Node: Quick Guide7893092 +Ref: library/unittest mock quick-guide7893200 +Ref: 30957893200 +Node: The Mock Class7898575 +Ref: library/unittest mock the-mock-class7898704 +Ref: 309b7898704 +Ref: library/unittest mock unittest mock Mock7899732 +Ref: 2497899732 +Ref: library/unittest mock unittest mock Mock assert_called7903156 +Ref: 5a87903156 +Ref: library/unittest mock unittest mock Mock assert_called_once7903446 +Ref: 5a97903446 +Ref: library/unittest mock unittest mock Mock assert_called_with7904042 +Ref: 30a27904042 +Ref: library/unittest mock unittest mock Mock assert_called_once_with7904426 +Ref: 30a37904426 +Ref: library/unittest mock unittest mock Mock assert_any_call7905035 +Ref: 30a47905035 +Ref: library/unittest mock unittest mock Mock assert_has_calls7905701 +Ref: 30a57905701 +Ref: library/unittest mock unittest mock Mock assert_not_called7906546 +Ref: 7847906546 +Ref: library/unittest mock unittest mock Mock reset_mock7906981 +Ref: 5aa7906981 +Ref: library/unittest mock unittest mock Mock mock_add_spec7908117 +Ref: 30a77908117 +Ref: library/unittest mock unittest mock Mock attach_mock7908442 +Ref: 30a87908442 +Ref: library/unittest mock unittest mock Mock configure_mock7908735 +Ref: 30a17908735 +Ref: library/unittest mock unittest mock Mock __dir__7909969 +Ref: 30aa7909969 +Ref: library/unittest mock unittest mock Mock _get_child_mock7910302 +Ref: 30ac7910302 +Ref: library/unittest mock unittest mock Mock called7910702 +Ref: 30ad7910702 +Ref: library/unittest mock unittest mock Mock call_count7911002 +Ref: 30ae7911002 +Ref: library/unittest mock unittest mock Mock return_value7911333 +Ref: 30a07911333 +Ref: library/unittest mock unittest mock Mock side_effect7912139 +Ref: 309e7912139 +Ref: library/unittest mock unittest mock Mock call_args7914609 +Ref: 30af7914609 +Ref: library/unittest mock unittest mock Mock call_args_list7916436 +Ref: 30b07916436 +Ref: library/unittest mock unittest mock Mock method_calls7917479 +Ref: 30a97917479 +Ref: library/unittest mock unittest mock Mock mock_calls7918217 +Ref: 30a67918217 +Ref: library/unittest mock unittest mock Mock __class__7919755 +Ref: 30b37919755 +Ref: library/unittest mock unittest mock NonCallableMock7920528 +Ref: 309c7920528 +Ref: library/unittest mock unittest mock PropertyMock7923056 +Ref: 30b47923056 +Ref: library/unittest mock unittest mock AsyncMock7924430 +Ref: 2487924430 +Ref: library/unittest mock unittest mock AsyncMock assert_awaited7927286 +Ref: 30b57927286 +Ref: library/unittest mock unittest mock AsyncMock assert_awaited_once7928021 +Ref: 30b67928021 +Ref: library/unittest mock unittest mock AsyncMock assert_awaited_with7928586 +Ref: 30b77928586 +Ref: library/unittest mock unittest mock AsyncMock assert_awaited_once_with7929255 +Ref: 30b87929255 +Ref: library/unittest mock unittest mock AsyncMock assert_any_await7929980 +Ref: 30b97929980 +Ref: library/unittest mock unittest mock AsyncMock assert_has_awaits7930623 +Ref: 30ba7930623 +Ref: library/unittest mock unittest mock AsyncMock assert_not_awaited7931718 +Ref: 30bc7931718 +Ref: library/unittest mock unittest mock AsyncMock reset_mock7931893 +Ref: 30bd7931893 +Ref: library/unittest mock unittest mock AsyncMock await_count7932124 +Ref: 30be7932124 +Ref: library/unittest mock unittest mock AsyncMock await_args7932570 +Ref: 30bf7932570 +Ref: library/unittest mock unittest mock AsyncMock await_args_list7933189 +Ref: 30bb7933189 +Ref: The Mock Class-Footnote-17934055 +Node: Calling7934481 +Ref: library/unittest mock calling7934567 +Ref: 30c07934567 +Node: Deleting Attributes7937809 +Ref: library/unittest mock deleting-attributes7937937 +Ref: 30c17937937 +Ref: library/unittest mock id27937937 +Ref: 30c27937937 +Node: Mock names and the name attribute7938714 +Ref: library/unittest mock mock-names-and-the-name-attribute7938864 +Ref: 30c37938864 +Node: Attaching Mocks as Attributes7939454 +Ref: library/unittest mock attaching-mocks-as-attributes7939576 +Ref: 30c47939576 +Node: The patchers7941440 +Ref: library/unittest mock the-patchers7941592 +Ref: 30c57941592 +Node: patch7942207 +Ref: library/unittest mock patch7942282 +Ref: 30c67942282 +Ref: library/unittest mock unittest mock patch7942482 +Ref: 7887942482 +Node: patch object7951324 +Ref: library/unittest mock patch-object7951418 +Ref: 30c87951418 +Ref: library/unittest mock unittest mock patch object7951463 +Ref: 30c97951463 +Node: patch dict7952942 +Ref: library/unittest mock patch-dict7953045 +Ref: 30ca7953045 +Ref: library/unittest mock unittest mock patch dict7953088 +Ref: 30977953088 +Node: patch multiple7957208 +Ref: library/unittest mock patch-multiple7957327 +Ref: 30cc7957327 +Ref: library/unittest mock unittest mock patch multiple7957378 +Ref: 30cd7957378 +Node: patch methods start and stop7960086 +Ref: library/unittest mock patch-methods-start-and-stop7960209 +Ref: 30ce7960209 +Ref: library/unittest mock start-and-stop7960209 +Ref: 30cf7960209 +Ref: library/unittest mock unittest mock patch stopall7962946 +Ref: 30d07962946 +Node: patch builtins7963055 +Ref: library/unittest mock id47963175 +Ref: 30d17963175 +Ref: library/unittest mock patch-builtins7963175 +Ref: 30d27963175 +Node: TEST_PREFIX7963496 +Ref: library/unittest mock id57963612 +Ref: 30d37963612 +Ref: library/unittest mock test-prefix7963612 +Ref: 30cb7963612 +Node: Nesting Patch Decorators7964495 +Ref: library/unittest mock nesting-patch-decorators7964611 +Ref: 30d47964611 +Node: Where to patch7965528 +Ref: library/unittest mock id67965671 +Ref: 30c77965671 +Ref: library/unittest mock where-to-patch7965671 +Ref: 30967965671 +Node: Patching Descriptors and Proxy Objects7967437 +Ref: library/unittest mock patching-descriptors-and-proxy-objects7967547 +Ref: 30d57967547 +Ref: Patching Descriptors and Proxy Objects-Footnote-17967988 +Node: MagicMock and magic method support7968070 +Ref: library/unittest mock magicmock-and-magic-method-support7968215 +Ref: 30d67968215 +Node: Mocking Magic Methods7968357 +Ref: library/unittest mock magic-methods7968468 +Ref: 30987968468 +Ref: library/unittest mock mocking-magic-methods7968468 +Ref: 30d77968468 +Ref: Mocking Magic Methods-Footnote-17972366 +Ref: Mocking Magic Methods-Footnote-27972609 +Node: Magic Mock7972734 +Ref: library/unittest mock magic-mock7972845 +Ref: 30d87972845 +Ref: library/unittest mock unittest mock MagicMock7972989 +Ref: 7857972989 +Ref: library/unittest mock unittest mock NonCallableMagicMock7973456 +Ref: 309d7973456 +Node: Helpers7976826 +Ref: library/unittest mock helpers7976950 +Ref: 30d97976950 +Node: sentinel7977123 +Ref: library/unittest mock sentinel7977191 +Ref: 30da7977191 +Ref: library/unittest mock unittest mock sentinel7977230 +Ref: 40a7977230 +Node: DEFAULT7978405 +Ref: library/unittest mock default7978486 +Ref: 30db7978486 +Ref: library/unittest mock unittest mock DEFAULT7978523 +Ref: 309f7978523 +Node: call7978776 +Ref: library/unittest mock call7978864 +Ref: 30dc7978864 +Ref: library/unittest mock unittest mock call7978895 +Ref: 30b17978895 +Ref: library/unittest mock unittest mock call call_list7979436 +Ref: 30dd7979436 +Ref: library/unittest mock calls-as-tuples7980388 +Ref: 30b27980388 +Node: create_autospec7982014 +Ref: library/unittest mock create-autospec7982098 +Ref: 30de7982098 +Ref: library/unittest mock unittest mock create_autospec7982151 +Ref: 309a7982151 +Node: ANY7983451 +Ref: library/unittest mock any7983541 +Ref: 30df7983541 +Ref: library/unittest mock unittest mock ANY7983570 +Ref: 30e07983570 +Node: FILTER_DIR7984452 +Ref: library/unittest mock filter-dir7984536 +Ref: 30e17984536 +Ref: library/unittest mock unittest mock FILTER_DIR7984579 +Ref: 30ab7984579 +Node: mock_open7986632 +Ref: library/unittest mock mock-open7986725 +Ref: 30e27986725 +Ref: library/unittest mock unittest mock mock_open7986766 +Ref: 30e37986766 +Ref: mock_open-Footnote-17989588 +Node: Autospeccing7989613 +Ref: library/unittest mock auto-speccing7989709 +Ref: 30997989709 +Ref: library/unittest mock autospeccing7989709 +Ref: 30e47989709 +Ref: Autospeccing-Footnote-17999219 +Node: Sealing mocks7999464 +Ref: library/unittest mock sealing-mocks7999542 +Ref: 30e57999542 +Ref: library/unittest mock unittest mock seal7999591 +Ref: 40b7999591 +Node: unittest mock — getting started8000403 +Ref: library/unittest mock-examples doc8000592 +Ref: 30e68000592 +Ref: library/unittest mock-examples unittest-mock-getting-started8000592 +Ref: 30e78000592 +Ref: library/unittest mock-examples getting-started8000702 +Ref: 30e88000702 +Node: Using Mock8000769 +Ref: library/unittest mock-examples using-mock8000874 +Ref: 30e98000874 +Node: Mock Patching Methods8001290 +Ref: library/unittest mock-examples mock-patching-methods8001401 +Ref: 30ea8001401 +Node: Mock for Method Calls on an Object8002951 +Ref: library/unittest mock-examples mock-for-method-calls-on-an-object8003086 +Ref: 30eb8003086 +Node: Mocking Classes8004187 +Ref: library/unittest mock-examples mocking-classes8004318 +Ref: 30ec8004318 +Node: Naming your mocks8005315 +Ref: library/unittest mock-examples naming-your-mocks8005430 +Ref: 30ed8005430 +Node: Tracking all Calls8005862 +Ref: library/unittest mock-examples tracking-all-calls8005998 +Ref: 30ee8005998 +Node: Setting Return Values and Attributes8007405 +Ref: library/unittest mock-examples setting-return-values-and-attributes8007553 +Ref: 30ef8007553 +Node: Raising exceptions with mocks8009064 +Ref: library/unittest mock-examples raising-exceptions-with-mocks8009229 +Ref: 30f08009229 +Node: Side effect functions and iterables8009609 +Ref: library/unittest mock-examples side-effect-functions-and-iterables8009768 +Ref: 30f18009768 +Node: Mocking asynchronous iterators8010808 +Ref: library/unittest mock-examples mocking-asynchronous-iterators8010974 +Ref: 30f28010974 +Node: Mocking asynchronous context manager8011547 +Ref: library/unittest mock-examples mocking-asynchronous-context-manager8011717 +Ref: 30f38011717 +Node: Creating a Mock from an Existing Object8012619 +Ref: library/unittest mock-examples creating-a-mock-from-an-existing-object8012750 +Ref: 30f48012750 +Node: Patch Decorators8014560 +Ref: library/unittest mock-examples patch-decorators8014690 +Ref: 30f58014690 +Node: Further Examples8019963 +Ref: library/unittest mock-examples further-examples8020074 +Ref: 30f68020074 +Ref: library/unittest mock-examples id18020074 +Ref: 30f78020074 +Node: Mocking chained calls8020702 +Ref: library/unittest mock-examples mocking-chained-calls8020800 +Ref: 30f88020800 +Node: Partial mocking8023915 +Ref: library/unittest mock-examples partial-mocking8024048 +Ref: 30f98024048 +Ref: Partial mocking-Footnote-18026056 +Node: Mocking a Generator Method8026157 +Ref: library/unittest mock-examples mocking-a-generator-method8026313 +Ref: 30fa8026313 +Ref: Mocking a Generator Method-Footnote-18027447 +Node: Applying the same patch to every test method8027771 +Ref: library/unittest mock-examples applying-the-same-patch-to-every-test-method8027935 +Ref: 30fb8027935 +Node: Mocking Unbound Methods8030253 +Ref: library/unittest mock-examples mocking-unbound-methods8030424 +Ref: 30fc8030424 +Node: Checking multiple calls with mock8032192 +Ref: library/unittest mock-examples checking-multiple-calls-with-mock8032348 +Ref: 30fd8032348 +Node: Coping with mutable arguments8033892 +Ref: library/unittest mock-examples coping-with-mutable-arguments8034040 +Ref: 30fe8034040 +Node: Nesting Patches8038215 +Ref: library/unittest mock-examples nesting-patches8038365 +Ref: 30ff8038365 +Node: Mocking a dictionary with MagicMock8040172 +Ref: library/unittest mock-examples mocking-a-dictionary-with-magicmock8040329 +Ref: 31008040329 +Node: Mock subclasses and their attributes8042975 +Ref: library/unittest mock-examples mock-subclasses-and-their-attributes8043148 +Ref: 31018043148 +Ref: Mock subclasses and their attributes-Footnote-18045284 +Ref: Mock subclasses and their attributes-Footnote-28045457 +Ref: Mock subclasses and their attributes-Footnote-38045515 +Node: Mocking imports with patch dict8045602 +Ref: library/unittest mock-examples mocking-imports-with-patch-dict8045796 +Ref: 31028045796 +Node: Tracking order of calls and less verbose call assertions8048444 +Ref: library/unittest mock-examples tracking-order-of-calls-and-less-verbose-call-assertions8048632 +Ref: 31038048632 +Node: More complex argument matching8051971 +Ref: library/unittest mock-examples more-complex-argument-matching8052119 +Ref: 31048052119 +Ref: More complex argument matching-Footnote-18055168 +Ref: More complex argument matching-Footnote-28055211 +Node: 2to3 - Automated Python 2 to 3 code translation8055333 +Ref: library/2to3 doc8055529 +Ref: 31058055529 +Ref: library/2to3 to3-automated-python-2-to-3-code-translation8055529 +Ref: 31068055529 +Ref: library/2to3 to3-reference8055529 +Ref: c768055529 +Node: Using 2to38056192 +Ref: library/2to3 to3-using8056301 +Ref: 31078056301 +Ref: library/2to3 using-2to38056301 +Ref: 31088056301 +Node: Fixers8060554 +Ref: library/2to3 fixers8060698 +Ref: 310a8060698 +Ref: library/2to3 to3-fixers8060698 +Ref: 31098060698 +Ref: library/2to3 2to3fixer-apply8060949 +Ref: 310b8060949 +Ref: library/2to3 2to3fixer-asserts8061115 +Ref: 310c8061115 +Ref: library/2to3 2to3fixer-basestring8063380 +Ref: 310d8063380 +Ref: library/2to3 2to3fixer-buffer8063458 +Ref: 310e8063458 +Ref: library/2to3 2to3fixer-dict8063669 +Ref: 310f8063669 +Ref: library/2to3 2to3fixer-except8064281 +Ref: 31108064281 +Ref: library/2to3 2to3fixer-exec8064361 +Ref: 31118064361 +Ref: library/2to3 2to3fixer-execfile8064458 +Ref: 31128064458 +Ref: library/2to3 2to3fixer-exitfunc8064655 +Ref: 31138064655 +Ref: library/2to3 2to3fixer-filter8064771 +Ref: 31148064771 +Ref: library/2to3 2to3fixer-funcattrs8064861 +Ref: 31158064861 +Ref: library/2to3 2to3fixer-future8065045 +Ref: 31168065045 +Ref: library/2to3 2to3fixer-getcwdu8065135 +Ref: 31178065135 +Ref: library/2to3 2to3fixer-has_key8065220 +Ref: 31188065220 +Ref: library/2to3 2to3fixer-idioms8065304 +Ref: 31198065304 +Ref: library/2to3 2to3fixer-import8065838 +Ref: 311a8065838 +Ref: library/2to3 2to3fixer-imports8065930 +Ref: 311b8065930 +Ref: library/2to3 2to3fixer-imports28066008 +Ref: 311c8066008 +Ref: library/2to3 2to3fixer-input8066195 +Ref: 311d8066195 +Ref: library/2to3 2to3fixer-intern8066282 +Ref: 311e8066282 +Ref: library/2to3 2to3fixer-isinstance8066363 +Ref: 311f8066363 +Ref: library/2to3 2to3fixer-itertools_imports8066665 +Ref: 31208066665 +Ref: library/2to3 2to3fixer-itertools8066913 +Ref: 31218066913 +Ref: library/2to3 2to3fixer-long8067168 +Ref: 31228067168 +Ref: library/2to3 2to3fixer-map8067233 +Ref: 31238067233 +Ref: library/2to3 2to3fixer-metaclass8067441 +Ref: 31248067441 +Ref: library/2to3 2to3fixer-methodattrs8067603 +Ref: 31258067603 +Ref: library/2to3 2to3fixer-ne8067746 +Ref: 31268067746 +Ref: library/2to3 2to3fixer-next8067829 +Ref: 31278067829 +Ref: library/2to3 2to3fixer-nonzero8068026 +Ref: 31288068026 +Ref: library/2to3 2to3fixer-numliterals8068110 +Ref: 31298068110 +Ref: library/2to3 2to3fixer-operator8068189 +Ref: 312a8068189 +Ref: library/2to3 2to3fixer-paren8069872 +Ref: 312b8069872 +Ref: library/2to3 2to3fixer-print8070053 +Ref: 312c8070053 +Ref: library/2to3 2to3fixer-raise8070153 +Ref: 312d8070153 +Ref: library/2to3 2to3fixer-raw_input8070432 +Ref: 312e8070432 +Ref: library/2to3 2to3fixer-reduce8070514 +Ref: 312f8070514 +Ref: library/2to3 2to3fixer-reload8070612 +Ref: 31308070612 +Ref: library/2to3 2to3fixer-renames8070699 +Ref: 31318070699 +Ref: library/2to3 2to3fixer-repr8070781 +Ref: 31328070781 +Ref: library/2to3 2to3fixer-set_literal8070869 +Ref: 31338070869 +Ref: library/2to3 2to3fixer-standarderror8070999 +Ref: 31348070999 +Ref: library/2to3 2to3fixer-sys_exc8071088 +Ref: 31358071088 +Ref: library/2to3 2to3fixer-throw8071245 +Ref: 31368071245 +Ref: library/2to3 2to3fixer-tuple_params8071333 +Ref: 31378071333 +Ref: library/2to3 2to3fixer-types8071458 +Ref: 31388071458 +Ref: library/2to3 2to3fixer-unicode8071575 +Ref: 31398071575 +Ref: library/2to3 2to3fixer-urllib8071646 +Ref: 313a8071646 +Ref: library/2to3 2to3fixer-ws_comma8071774 +Ref: 313b8071774 +Ref: library/2to3 2to3fixer-xrange8071889 +Ref: 313c8071889 +Ref: library/2to3 2to3fixer-xreadlines8072036 +Ref: 313d8072036 +Ref: library/2to3 2to3fixer-zip8072134 +Ref: 313e8072134 +Node: lib2to3 - 2to3’s library8072293 +Ref: library/2to3 lib2to3-2to3-s-library8072418 +Ref: 313f8072418 +Ref: library/2to3 module-lib2to38072418 +Ref: a78072418 +Ref: lib2to3 - 2to3’s library-Footnote-18072749 +Node: test — Regression tests package for Python8072813 +Ref: library/test doc8073028 +Ref: 31408073028 +Ref: library/test module-test8073028 +Ref: 1058073028 +Ref: library/test test-regression-tests-package-for-python8073028 +Ref: 31418073028 +Node: Writing Unit Tests for the test package8074458 +Ref: library/test writing-tests8074633 +Ref: 31428074633 +Ref: library/test writing-unit-tests-for-the-test-package8074633 +Ref: 31438074633 +Node: Running tests using the command-line interface8079178 +Ref: library/test regrtest8079353 +Ref: 92d8079353 +Ref: library/test running-tests-using-the-command-line-interface8079353 +Ref: 31448079353 +Node: test support — Utilities for the Python test suite8081291 +Ref: library/test module-test support8081530 +Ref: 1068081530 +Ref: library/test test-support-utilities-for-the-python-test-suite8081530 +Ref: 31458081530 +Ref: library/test test support TestFailed8082038 +Ref: 31468082038 +Ref: library/test test support ResourceDenied8082258 +Ref: 31478082258 +Ref: library/test test support verbose8082552 +Ref: 31498082552 +Ref: library/test test support is_jython8082768 +Ref: 314a8082768 +Ref: library/test test support is_android8082857 +Ref: 314b8082857 +Ref: library/test test support unix_shell8082935 +Ref: 314c8082935 +Ref: library/test test support FS_NONASCII8083032 +Ref: 314d8083032 +Ref: library/test test support TESTFN8083135 +Ref: 314e8083135 +Ref: library/test test support TESTFN_UNICODE8083325 +Ref: 314f8083325 +Ref: library/test test support TESTFN_ENCODING8083416 +Ref: 31508083416 +Ref: library/test test support TESTFN_UNENCODABLE8083509 +Ref: 31518083509 +Ref: library/test test support TESTFN_UNDECODABLE8083749 +Ref: 31528083749 +Ref: library/test test support TESTFN_NONASCII8083991 +Ref: 31538083991 +Ref: library/test test support IPV6_ENABLED8084111 +Ref: 31548084111 +Ref: library/test test support SAVEDCWD8084228 +Ref: 31558084228 +Ref: library/test test support PGO8084299 +Ref: 31568084299 +Ref: library/test test support PIPE_MAX_SIZE8084397 +Ref: 31578084397 +Ref: library/test test support SOCK_MAX_SIZE8084545 +Ref: 31588084545 +Ref: library/test test support TEST_SUPPORT_DIR8084695 +Ref: 31598084695 +Ref: library/test test support TEST_HOME_DIR8084817 +Ref: 315a8084817 +Ref: library/test test support TEST_DATA_DIR8084914 +Ref: 315b8084914 +Ref: library/test test support MAX_Py_ssize_t8085015 +Ref: 315c8085015 +Ref: library/test test support max_memuse8085113 +Ref: 315d8085113 +Ref: library/test test support real_max_memuse8085276 +Ref: 315f8085276 +Ref: library/test test support MISSING_C_DOCSTRINGS8085448 +Ref: 31608085448 +Ref: library/test test support HAVE_DOCSTRINGS8085616 +Ref: 31618085616 +Ref: library/test test support TEST_HTTP_URL8085696 +Ref: 31628085696 +Ref: library/test test support ALWAYS_EQ8085805 +Ref: 31638085805 +Ref: library/test test support LARGEST8085921 +Ref: 31648085921 +Ref: library/test test support SMALLEST8086055 +Ref: 31658086055 +Ref: library/test test support forget8086257 +Ref: 31668086257 +Ref: library/test test support unload8086428 +Ref: 31678086428 +Ref: library/test test support unlink8086514 +Ref: 31688086514 +Ref: library/test test support rmdir8086719 +Ref: 31698086719 +Ref: library/test test support rmtree8086922 +Ref: 316a8086922 +Ref: library/test test support make_legacy_pyc8087221 +Ref: 316b8087221 +Ref: library/test test support is_resource_enabled8087556 +Ref: 316c8087556 +Ref: library/test test support python_is_optimized8087783 +Ref: 316d8087783 +Ref: library/test test support with_pymalloc8087911 +Ref: 316e8087911 +Ref: library/test test support requires8088000 +Ref: 31488088000 +Ref: library/test test support system_must_validate_cert8088367 +Ref: 316f8088367 +Ref: library/test test support sortdict8088514 +Ref: 31708088514 +Ref: library/test test support findfile8088606 +Ref: 31718088606 +Ref: library/test test support create_empty_file8088978 +Ref: 31728088978 +Ref: library/test test support fd_count8089120 +Ref: 31738089120 +Ref: library/test test support match_test8089209 +Ref: 31748089209 +Ref: library/test test support set_match_tests8089324 +Ref: 31758089324 +Ref: library/test test support run_unittest8089439 +Ref: 31768089439 +Ref: library/test test support run_doctest8090101 +Ref: 31778090101 +Ref: library/test test support setswitchinterval8090568 +Ref: 31788090568 +Ref: library/test test support check_impl_detail8090794 +Ref: 31798090794 +Ref: library/test test support check_warnings8091206 +Ref: 317a8091206 +Ref: library/test test support check_no_resource_warning8094252 +Ref: 317d8094252 +Ref: library/test test support set_memlimit8094516 +Ref: 315e8094516 +Ref: library/test test support record_original_stdout8094674 +Ref: 317e8094674 +Ref: library/test test support get_original_stdout8094840 +Ref: 317f8094840 +Ref: library/test test support strip_python_strerr8095013 +Ref: 31808095013 +Ref: library/test test support args_from_interpreter_flags8095270 +Ref: 31818095270 +Ref: library/test test support optim_args_from_interpreter_flags8095459 +Ref: 31828095459 +Ref: library/test test support captured_stdin8095641 +Ref: 31838095641 +Ref: library/test test support captured_stdout8095686 +Ref: 31848095686 +Ref: library/test test support captured_stderr8095732 +Ref: 31858095732 +Ref: library/test test support temp_dir8096457 +Ref: 31868096457 +Ref: library/test test support change_cwd8096900 +Ref: 31878096900 +Ref: library/test test support temp_cwd8097259 +Ref: 31888097259 +Ref: library/test test support temp_umask8097897 +Ref: 31898097897 +Ref: library/test test support transient_internet8098009 +Ref: 318a8098009 +Ref: library/test test support disable_faulthandler8098268 +Ref: 318b8098268 +Ref: library/test test support gc_collect8098402 +Ref: 318c8098402 +Ref: library/test test support disable_gc8098738 +Ref: 318d8098738 +Ref: library/test test support swap_attr8098884 +Ref: 318e8098884 +Ref: library/test test support swap_item8099470 +Ref: 318f8099470 +Ref: library/test test support wait_threads_exit8100054 +Ref: 31908100054 +Ref: library/test test support start_threads8100210 +Ref: 31918100210 +Ref: library/test test support calcobjsize8100366 +Ref: 31928100366 +Ref: library/test test support calcvobjsize8100538 +Ref: 31938100538 +Ref: library/test test support checksizeof8100713 +Ref: 31948100713 +Ref: library/test test support can_symlink8100884 +Ref: 31958100884 +Ref: library/test test support can_xattr8101009 +Ref: 31968101009 +Ref: library/test test support skip_unless_symlink8101123 +Ref: 31978101123 +Ref: library/test test support skip_unless_xattr8101254 +Ref: 31988101254 +Ref: library/test test support skip_unless_bind_unix_socket8101369 +Ref: 31998101369 +Ref: library/test test support anticipate_failure8101519 +Ref: 319a8101519 +Ref: library/test test support run_with_locale8101781 +Ref: 319b8101781 +Ref: library/test test support run_with_tz8102141 +Ref: 319c8102141 +Ref: library/test test support requires_freebsd_version8102303 +Ref: 319d8102303 +Ref: library/test test support requires_linux_version8102537 +Ref: 319e8102537 +Ref: library/test test support requires_mac_version8102766 +Ref: 319f8102766 +Ref: library/test test support requires_IEEE_7548102998 +Ref: 31a08102998 +Ref: library/test test support requires_zlib8103107 +Ref: 31a18103107 +Ref: library/test test support requires_gzip8103222 +Ref: 31a28103222 +Ref: library/test test support requires_bz28103336 +Ref: 31a38103336 +Ref: library/test test support requires_lzma8103448 +Ref: 31a48103448 +Ref: library/test test support requires_resource8103562 +Ref: 31a58103562 +Ref: library/test test support requires_docstrings8103687 +Ref: 31a68103687 +Ref: library/test test support cpython_only8103810 +Ref: 31a78103810 +Ref: library/test test support impl_detail8103913 +Ref: 31a88103913 +Ref: library/test test support no_tracing8104143 +Ref: 31a98104143 +Ref: library/test test support refcount_test8104273 +Ref: 31aa8104273 +Ref: library/test test support reap_threads8104584 +Ref: 31ab8104584 +Ref: library/test test support bigmemtest8104715 +Ref: 31ac8104715 +Ref: library/test test support bigaddrspacetest8105462 +Ref: 31ad8105462 +Ref: library/test test support make_bad_fd8105602 +Ref: 31ae8105602 +Ref: library/test test support check_syntax_error8105761 +Ref: 31af8105761 +Ref: library/test test support check_syntax_warning8106315 +Ref: 31b08106315 +Ref: library/test test support open_urlresource8107116 +Ref: 31b18107116 +Ref: library/test test support import_module8107245 +Ref: 31b28107245 +Ref: library/test test support import_fresh_module8107849 +Ref: 31b38107849 +Ref: library/test test support modules_setup8109450 +Ref: 31b48109450 +Ref: library/test test support modules_cleanup8109543 +Ref: 31b58109543 +Ref: library/test test support threading_setup8109707 +Ref: 31b68109707 +Ref: library/test test support threading_cleanup8109818 +Ref: 31b78109818 +Ref: library/test test support join_thread8110029 +Ref: 31b88110029 +Ref: library/test test support reap_children8110225 +Ref: 31b98110225 +Ref: library/test test support get_attribute8110499 +Ref: 31ba8110499 +Ref: library/test test support bind_port8110661 +Ref: 31bb8110661 +Ref: library/test test support bind_unix_socket8111591 +Ref: 31bc8111591 +Ref: library/test test support catch_threading_exception8111760 +Ref: 31bd8111760 +Ref: library/test test support catch_unraisable_exception8112694 +Ref: 31be8112694 +Ref: library/test test support find_unused_port8113580 +Ref: 31bf8113580 +Ref: library/test test support load_package_tests8114888 +Ref: 31c08114888 +Ref: library/test test support fs_is_case_insensitive8115518 +Ref: 31c18115518 +Ref: library/test test support detect_api_mismatch8115666 +Ref: 31c28115666 +Ref: library/test test support patch8116135 +Ref: 31c38116135 +Ref: library/test test support run_in_subinterp8116471 +Ref: 31c48116471 +Ref: library/test test support check_free_after_iterating8116640 +Ref: 31c58116640 +Ref: library/test test support missing_compiler_executable8116789 +Ref: 31c68116789 +Ref: library/test test support check__all__8117111 +Ref: 31c78117111 +Ref: library/test test support TransientResource8118975 +Ref: 31c88118975 +Ref: library/test test support EnvironmentVarGuard8119431 +Ref: 31c98119431 +Ref: library/test test support EnvironmentVarGuard set8119889 +Ref: 31ca8119889 +Ref: library/test test support EnvironmentVarGuard unset8120036 +Ref: 31cb8120036 +Ref: library/test test support SuppressCrashReport8120147 +Ref: 31cc8120147 +Ref: library/test test support CleanImport8120649 +Ref: 31cf8120649 +Ref: library/test test support DirsOnSysPath8121008 +Ref: 31d08121008 +Ref: library/test test support SaveSignals8121499 +Ref: 31d18121499 +Ref: library/test test support Matcher8121630 +Ref: 31d28121630 +Ref: library/test test support Matcher matches8121663 +Ref: 31d38121663 +Ref: library/test test support Matcher match_value8121776 +Ref: 31d48121776 +Ref: library/test test support WarningsRecorder8121915 +Ref: 31d58121915 +Ref: library/test test support BasicTestRunner8122090 +Ref: 31d68122090 +Ref: library/test test support BasicTestRunner run8122131 +Ref: 31d78122131 +Ref: library/test test support TestHandler8122205 +Ref: 31d88122205 +Ref: library/test test support FakePath8122311 +Ref: 31d98122311 +Ref: test support — Utilities for the Python test suite-Footnote-18122594 +Ref: test support — Utilities for the Python test suite-Footnote-28122643 +Ref: test support — Utilities for the Python test suite-Footnote-38122692 +Node: test support script_helper — Utilities for the Python execution tests8122772 +Ref: library/test module-test support script_helper8122958 +Ref: 1078122958 +Ref: library/test test-support-script-helper-utilities-for-the-python-execution-tests8122958 +Ref: 31da8122958 +Ref: library/test test support script_helper interpreter_requires_environment8123232 +Ref: 31db8123232 +Ref: library/test test support script_helper run_python_until_end8124207 +Ref: 31dc8124207 +Ref: library/test test support script_helper assert_python_ok8124493 +Ref: 31dd8124493 +Ref: library/test test support script_helper assert_python_failure8125007 +Ref: 31de8125007 +Ref: library/test test support script_helper spawn_python8125349 +Ref: 31df8125349 +Ref: library/test test support script_helper kill_python8125662 +Ref: 31e08125662 +Ref: library/test test support script_helper make_script8125818 +Ref: 31e18125818 +Ref: library/test test support script_helper make_zip_script8126127 +Ref: 31e28126127 +Ref: library/test test support script_helper make_pkg8126496 +Ref: 31e38126496 +Ref: library/test test support script_helper make_zip_pkg8126699 +Ref: 31e48126699 +Node: Debugging and Profiling8127356 +Ref: library/debug doc8127513 +Ref: 31e58127513 +Ref: library/debug debugging-and-profiling8127513 +Ref: 31e68127513 +Ref: library/audit_events audit-events8127971 +Ref: 31e78127971 +Node: Audit events table8128303 +Ref: library/audit_events doc8128416 +Ref: 31e88128416 +Ref: library/audit_events audit-events-table8128416 +Ref: 31e98128416 +Node: bdb — Debugger framework8181513 +Ref: library/bdb doc8181677 +Ref: 32128181677 +Ref: library/bdb bdb-debugger-framework8181677 +Ref: 32138181677 +Ref: library/bdb module-bdb8181677 +Ref: f8181677 +Ref: library/bdb bdb BdbQuit8182012 +Ref: 32148182012 +Ref: library/bdb bdb Breakpoint8182176 +Ref: 32158182176 +Ref: library/bdb bdb Breakpoint deleteMe8183044 +Ref: 32168183044 +Ref: library/bdb bdb Breakpoint enable8183260 +Ref: 32178183260 +Ref: library/bdb bdb Breakpoint disable8183331 +Ref: 32188183331 +Ref: library/bdb bdb Breakpoint bpformat8183404 +Ref: 32198183404 +Ref: library/bdb bdb Breakpoint bpprint8183838 +Ref: 321a8183838 +Ref: library/bdb bdb Bdb8184003 +Ref: c988184003 +Ref: library/bdb bdb Bdb canonic8184788 +Ref: 321b8184788 +Ref: library/bdb bdb Bdb reset8185046 +Ref: 321c8185046 +Ref: library/bdb bdb Bdb trace_dispatch8185221 +Ref: 321d8185221 +Ref: library/bdb bdb Bdb dispatch_line8186639 +Ref: 321e8186639 +Ref: library/bdb bdb Bdb dispatch_call8187113 +Ref: 32208187113 +Ref: library/bdb bdb Bdb dispatch_return8187594 +Ref: 32228187594 +Ref: library/bdb bdb Bdb dispatch_exception8188083 +Ref: 32248188083 +Ref: library/bdb bdb Bdb stop_here8188739 +Ref: 32268188739 +Ref: library/bdb bdb Bdb break_here8188950 +Ref: 32278188950 +Ref: library/bdb bdb Bdb break_anywhere8189220 +Ref: 32288189220 +Ref: library/bdb bdb Bdb user_call8189462 +Ref: 32218189462 +Ref: library/bdb bdb Bdb user_line8189699 +Ref: 321f8189699 +Ref: library/bdb bdb Bdb user_return8189907 +Ref: 32238189907 +Ref: library/bdb bdb Bdb user_exception8190087 +Ref: 32258190087 +Ref: library/bdb bdb Bdb do_clear8190269 +Ref: 32298190269 +Ref: library/bdb bdb Bdb set_step8190554 +Ref: 322a8190554 +Ref: library/bdb bdb Bdb set_next8190624 +Ref: 322b8190624 +Ref: library/bdb bdb Bdb set_return8190721 +Ref: 322c8190721 +Ref: library/bdb bdb Bdb set_until8190811 +Ref: 322d8190811 +Ref: library/bdb bdb Bdb set_trace8190982 +Ref: 322e8190982 +Ref: library/bdb bdb Bdb set_continue8191141 +Ref: 322f8191141 +Ref: library/bdb bdb Bdb set_quit8191314 +Ref: 32308191314 +Ref: library/bdb bdb Bdb set_break8191732 +Ref: 32318191732 +Ref: library/bdb bdb Bdb clear_break8192083 +Ref: 32328192083 +Ref: library/bdb bdb Bdb clear_bpbynumber8192253 +Ref: 32338192253 +Ref: library/bdb bdb Bdb clear_all_file_breaks8192476 +Ref: 32348192476 +Ref: library/bdb bdb Bdb clear_all_breaks8192635 +Ref: 32358192635 +Ref: library/bdb bdb Bdb get_bpbynumber8192717 +Ref: 32368192717 +Ref: library/bdb bdb Bdb get_break8193066 +Ref: 32378193066 +Ref: library/bdb bdb Bdb get_breaks8193183 +Ref: 32388193183 +Ref: library/bdb bdb Bdb get_file_breaks8193337 +Ref: 32398193337 +Ref: library/bdb bdb Bdb get_all_breaks8193475 +Ref: 323a8193475 +Ref: library/bdb bdb Bdb get_stack8193681 +Ref: 323b8193681 +Ref: library/bdb bdb Bdb format_stack_entry8193847 +Ref: 323c8193847 +Ref: library/bdb bdb Bdb run8194434 +Ref: 323d8194434 +Ref: library/bdb bdb Bdb runeval8194664 +Ref: 323e8194664 +Ref: library/bdb bdb Bdb runctx8194892 +Ref: 323f8194892 +Ref: library/bdb bdb Bdb runcall8195028 +Ref: 29f8195028 +Ref: library/bdb bdb checkfuncname8195194 +Ref: 32408195194 +Ref: library/bdb bdb effective8195632 +Ref: 32418195632 +Ref: library/bdb bdb set_trace8195951 +Ref: 32428195951 +Ref: bdb — Debugger framework-Footnote-18196101 +Node: faulthandler — Dump the Python traceback8196163 +Ref: library/faulthandler doc8196336 +Ref: 32438196336 +Ref: library/faulthandler faulthandler-dump-the-python-traceback8196336 +Ref: 32448196336 +Ref: library/faulthandler module-faulthandler8196336 +Ref: 7d8196336 +Node: Dumping the traceback8198378 +Ref: library/faulthandler dumping-the-traceback8198506 +Ref: 32458198506 +Ref: library/faulthandler faulthandler dump_traceback8198569 +Ref: 6d88198569 +Node: Fault handler state8198876 +Ref: library/faulthandler fault-handler-state8199051 +Ref: 32468199051 +Ref: library/faulthandler faulthandler enable8199110 +Ref: 5488199110 +Ref: library/faulthandler faulthandler disable8199800 +Ref: 32488199800 +Ref: library/faulthandler faulthandler is_enabled8199941 +Ref: 32498199941 +Node: Dumping the tracebacks after a timeout8200028 +Ref: library/faulthandler dumping-the-tracebacks-after-a-timeout8200220 +Ref: 324a8200220 +Ref: library/faulthandler faulthandler dump_traceback_later8200317 +Ref: 6d98200317 +Ref: library/faulthandler faulthandler cancel_dump_traceback_later8201349 +Ref: 324b8201349 +Node: Dumping the traceback on a user signal8201473 +Ref: library/faulthandler dumping-the-traceback-on-a-user-signal8201673 +Ref: 324c8201673 +Ref: library/faulthandler faulthandler register8201770 +Ref: 6d78201770 +Ref: library/faulthandler faulthandler unregister8202401 +Ref: 324d8202401 +Node: Issue with file descriptors8202676 +Ref: library/faulthandler faulthandler-fd8202849 +Ref: 32478202849 +Ref: library/faulthandler issue-with-file-descriptors8202849 +Ref: 324e8202849 +Node: Example<15>8203317 +Ref: library/faulthandler example8203443 +Ref: 324f8203443 +Node: pdb — The Python Debugger8203989 +Ref: library/pdb doc8204156 +Ref: 32508204156 +Ref: library/pdb debugger8204156 +Ref: 32518204156 +Ref: library/pdb module-pdb8204156 +Ref: c98204156 +Ref: library/pdb pdb-the-python-debugger8204156 +Ref: 32528204156 +Ref: library/pdb pdb run8207366 +Ref: 30228207366 +Ref: library/pdb pdb runeval8208058 +Ref: 32578208058 +Ref: library/pdb pdb runcall8208369 +Ref: 32588208369 +Ref: library/pdb pdb set_trace8208684 +Ref: 3c28208684 +Ref: library/pdb pdb post_mortem8209100 +Ref: d458209100 +Ref: library/pdb pdb pm8209391 +Ref: 32598209391 +Ref: library/pdb pdb Pdb8209735 +Ref: 56c8209735 +Ref: library/pdb pdb Pdb run8211101 +Ref: 325b8211101 +Ref: library/pdb pdb Pdb runeval8211161 +Ref: 325c8211161 +Ref: library/pdb pdb Pdb runcall8211226 +Ref: 325d8211226 +Ref: library/pdb pdb Pdb set_trace8211277 +Ref: 325e8211277 +Ref: pdb — The Python Debugger-Footnote-18211444 +Ref: pdb — The Python Debugger-Footnote-28211506 +Node: Debugger Commands8211639 +Ref: library/pdb debugger-commands8211720 +Ref: 32538211720 +Ref: library/pdb id28211720 +Ref: 325f8211720 +Ref: library/pdb pdbcommand-help8213994 +Ref: 32628213994 +Ref: library/pdb pdbcommand-where8214387 +Ref: 32638214387 +Ref: library/pdb pdbcommand-down8214577 +Ref: 32648214577 +Ref: library/pdb pdbcommand-up8214716 +Ref: 32658214716 +Ref: library/pdb pdbcommand-break8214852 +Ref: 32668214852 +Ref: library/pdb pdbcommand-tbreak8215734 +Ref: 32678215734 +Ref: library/pdb pdbcommand-clear8215945 +Ref: 32688215945 +Ref: library/pdb pdbcommand-disable8216253 +Ref: 32698216253 +Ref: library/pdb pdbcommand-enable8216578 +Ref: 326a8216578 +Ref: library/pdb pdbcommand-ignore8216668 +Ref: 326b8216668 +Ref: library/pdb pdbcommand-condition8217061 +Ref: 326c8217061 +Ref: library/pdb pdbcommand-commands8217354 +Ref: 326d8217354 +Ref: library/pdb pdbcommand-step8218989 +Ref: 32558218989 +Ref: library/pdb pdbcommand-next8219174 +Ref: 32568219174 +Ref: library/pdb pdbcommand-until8219572 +Ref: 32718219572 +Ref: library/pdb pdbcommand-return8219969 +Ref: 326e8219969 +Ref: library/pdb pdbcommand-continue8220056 +Ref: 32548220056 +Ref: library/pdb pdbcommand-jump8220156 +Ref: 326f8220156 +Ref: library/pdb pdbcommand-list8220586 +Ref: 32608220586 +Ref: library/pdb pdbcommand-ll8221324 +Ref: 32728221324 +Ref: library/pdb pdbcommand-args8221502 +Ref: 32738221502 +Ref: library/pdb pdbcommand-p8221581 +Ref: 8878221581 +Ref: library/pdb pdbcommand-pp8221850 +Ref: 32748221850 +Ref: library/pdb pdbcommand-whatis8222013 +Ref: 32758222013 +Ref: library/pdb pdbcommand-source8222090 +Ref: 32768222090 +Ref: library/pdb pdbcommand-display8222217 +Ref: 32778222217 +Ref: library/pdb pdbcommand-undisplay8222474 +Ref: 32788222474 +Ref: library/pdb pdbcommand-interact8222690 +Ref: 32798222690 +Ref: library/pdb debugger-aliases8222919 +Ref: 32618222919 +Ref: library/pdb pdbcommand-alias8222919 +Ref: 327a8222919 +Ref: library/pdb pdbcommand-unalias8224016 +Ref: 327b8224016 +Ref: library/pdb pdbcommand-!8224080 +Ref: 327c8224080 +Ref: library/pdb pdbcommand-run8224521 +Ref: 327d8224521 +Ref: library/pdb pdbcommand-restart8224552 +Ref: 327e8224552 +Ref: library/pdb pdbcommand-quit8224891 +Ref: 32708224891 +Ref: library/pdb pdbcommand-debug8224985 +Ref: 327f8224985 +Ref: library/pdb pdbcommand-retval8225184 +Ref: 32808225184 +Node: The Python Profilers8225272 +Ref: library/profile doc8225453 +Ref: 32818225453 +Ref: library/profile profile8225453 +Ref: 32828225453 +Ref: library/profile the-python-profilers8225453 +Ref: 32838225453 +Ref: The Python Profilers-Footnote-18225897 +Ref: The Python Profilers-Footnote-28225963 +Node: Introduction to the profilers8226028 +Ref: library/profile introduction-to-the-profilers8226146 +Ref: 32848226146 +Ref: library/profile profiler-introduction8226146 +Ref: 32858226146 +Node: Instant User’s Manual8227632 +Ref: library/profile instant-user-s-manual8227796 +Ref: 32868227796 +Ref: library/profile profile-instant8227796 +Ref: 32878227796 +Node: profile and cProfile Module Reference8233971 +Ref: library/profile module-cProfile8234121 +Ref: 288234121 +Ref: library/profile profile-and-cprofile-module-reference8234121 +Ref: 328c8234121 +Ref: library/profile module-profile8234232 +Ref: d38234232 +Ref: library/profile profile run8234326 +Ref: 328d8234326 +Ref: library/profile profile runctx8234961 +Ref: 328e8234961 +Ref: library/profile profile Profile8235363 +Ref: 1bb8235363 +Ref: library/profile profile Profile enable8236931 +Ref: 328f8236931 +Ref: library/profile profile Profile disable8237032 +Ref: 32908237032 +Ref: library/profile profile Profile create_stats8237133 +Ref: 32918237133 +Ref: library/profile profile Profile print_stats8237277 +Ref: 32928237277 +Ref: library/profile profile Profile dump_stats8237435 +Ref: 32938237435 +Ref: library/profile profile Profile run8237542 +Ref: 32948237542 +Ref: library/profile profile Profile runctx8237620 +Ref: 32958237620 +Ref: library/profile profile Profile runcall8237777 +Ref: 29e8237777 +Node: The Stats Class8238118 +Ref: library/profile profile-stats8238277 +Ref: 32968238277 +Ref: library/profile the-stats-class8238277 +Ref: 32978238277 +Ref: library/profile module-pstats8238409 +Ref: d48238409 +Ref: library/profile pstats Stats8238411 +Ref: 32888238411 +Ref: library/profile pstats Stats strip_dirs8239674 +Ref: 328a8239674 +Ref: library/profile pstats Stats add8240496 +Ref: 32988240496 +Ref: library/profile pstats Stats dump_stats8240989 +Ref: 32998240989 +Ref: library/profile pstats Stats sort_stats8241370 +Ref: 32898241370 +Ref: library/profile pstats Stats reverse_order8246727 +Ref: 329a8246727 +Ref: library/profile pstats Stats print_stats8247013 +Ref: 328b8247013 +Ref: library/profile pstats Stats print_callers8248457 +Ref: 329b8248457 +Ref: library/profile pstats Stats print_callees8249573 +Ref: 329c8249573 +Node: What Is Deterministic Profiling?8249954 +Ref: library/profile deterministic-profiling8250087 +Ref: 329d8250087 +Ref: library/profile what-is-deterministic-profiling8250087 +Ref: 329e8250087 +Node: Limitations8251885 +Ref: library/profile limitations8252014 +Ref: 329f8252014 +Ref: library/profile profile-limitations8252014 +Ref: 32a08252014 +Node: Calibration8253897 +Ref: library/profile calibration8254014 +Ref: 32a18254014 +Ref: library/profile profile-calibration8254014 +Ref: 32a28254014 +Node: Using a custom timer8255709 +Ref: library/profile profile-timers8255806 +Ref: 32a38255806 +Ref: library/profile using-a-custom-timer8255806 +Ref: 32a48255806 +Node: timeit — Measure execution time of small code snippets8258256 +Ref: library/timeit doc8258461 +Ref: 32a58258461 +Ref: library/timeit module-timeit8258461 +Ref: 10b8258461 +Ref: library/timeit timeit-measure-execution-time-of-small-code-snippets8258461 +Ref: 32a68258461 +Ref: timeit — Measure execution time of small code snippets-Footnote-18259228 +Node: Basic Examples<2>8259293 +Ref: library/timeit basic-examples8259428 +Ref: 32a98259428 +Node: Python Interface8260716 +Ref: library/timeit id18260885 +Ref: 32ab8260885 +Ref: library/timeit python-interface8260885 +Ref: 32a88260885 +Ref: library/timeit timeit timeit8261006 +Ref: 7718261006 +Ref: library/timeit timeit repeat8261469 +Ref: 32ad8261469 +Ref: library/timeit timeit default_timer8262059 +Ref: 32af8262059 +Ref: library/timeit timeit Timer8262264 +Ref: 32ac8262264 +Ref: library/timeit timeit Timer timeit8263711 +Ref: 59c8263711 +Ref: library/timeit timeit Timer autorange8264820 +Ref: 59b8264820 +Ref: library/timeit timeit Timer repeat8265547 +Ref: 32ae8265547 +Ref: library/timeit timeit Timer print_exc8266822 +Ref: 32b08266822 +Node: Command-Line Interface<4>8267402 +Ref: library/timeit command-line-interface8267566 +Ref: 32b18267566 +Ref: library/timeit timeit-command-line-interface8267566 +Ref: 32a78267566 +Ref: library/timeit cmdoption-timeit-n8267825 +Ref: 32b28267825 +Ref: library/timeit cmdoption-timeit-r8267911 +Ref: 32b38267911 +Ref: library/timeit cmdoption-timeit-s8268002 +Ref: 32b48268002 +Ref: library/timeit cmdoption-timeit-p8268106 +Ref: 32b58268106 +Ref: library/timeit cmdoption-timeit-u8268327 +Ref: 32b68268327 +Ref: library/timeit cmdoption-timeit-v8268472 +Ref: 32b78268472 +Ref: library/timeit cmdoption-timeit-h8268572 +Ref: 32b88268572 +Node: Examples<25>8269791 +Ref: library/timeit examples8269930 +Ref: 32b98269930 +Ref: library/timeit timeit-examples8269930 +Ref: 32aa8269930 +Node: trace — Trace or track Python statement execution8273147 +Ref: library/trace doc8273372 +Ref: 32ba8273372 +Ref: library/trace module-trace8273372 +Ref: 1128273372 +Ref: library/trace trace-trace-or-track-python-statement-execution8273372 +Ref: 32bb8273372 +Ref: trace — Trace or track Python statement execution-Footnote-18274117 +Ref: trace — Trace or track Python statement execution-Footnote-28274181 +Node: Command-Line Usage8274222 +Ref: library/trace command-line-usage8274359 +Ref: 32bc8274359 +Ref: library/trace trace-cli8274359 +Ref: 32bd8274359 +Ref: library/trace cmdoption-trace-help8274713 +Ref: 32be8274713 +Ref: library/trace cmdoption-trace-version8274771 +Ref: 32bf8274771 +Node: Main options8274995 +Ref: library/trace main-options8275080 +Ref: 32c08275080 +Ref: library/trace cmdoption-trace-c8275470 +Ref: 32c38275470 +Ref: library/trace cmdoption-trace-t8275737 +Ref: 32c28275737 +Ref: library/trace cmdoption-trace-l8275812 +Ref: 32c18275812 +Ref: library/trace cmdoption-trace-r8275910 +Ref: 32c78275910 +Ref: library/trace cmdoption-trace-trackcalls8276116 +Ref: 32c88276116 +Node: Modifiers8276226 +Ref: library/trace modifiers8276327 +Ref: 32c98276327 +Ref: library/trace cmdoption-trace-f8276366 +Ref: 32c58276366 +Ref: library/trace cmdoption-trace-coverdir8276533 +Ref: 32c48276533 +Ref: library/trace cmdoption-trace-m8276731 +Ref: 32ca8276731 +Ref: library/trace cmdoption-trace-s8276867 +Ref: 32cb8276867 +Ref: library/trace cmdoption-trace-no-report8277029 +Ref: 32c68277029 +Ref: library/trace cmdoption-trace-g8277260 +Ref: 32cc8277260 +Node: Filters8277389 +Ref: library/trace filters8277469 +Ref: 32cd8277469 +Ref: library/trace cmdoption-trace-ignore-module8277551 +Ref: 32ce8277551 +Ref: library/trace cmdoption-trace-ignore-dir8277748 +Ref: 32cf8277748 +Node: Programmatic Interface8277958 +Ref: library/trace programmatic-interface8278095 +Ref: 32d08278095 +Ref: library/trace trace-api8278095 +Ref: 32d18278095 +Ref: library/trace trace Trace8278160 +Ref: 32d28278160 +Ref: library/trace trace Trace run8279071 +Ref: 32d38279071 +Ref: library/trace trace Trace runctx8279313 +Ref: 32d48279313 +Ref: library/trace trace Trace runfunc8279627 +Ref: 2a08279627 +Ref: library/trace trace Trace results8279817 +Ref: 32d58279817 +Ref: library/trace trace CoverageResults8280126 +Ref: 32d68280126 +Ref: library/trace trace CoverageResults update8280290 +Ref: 32d78280290 +Ref: library/trace trace CoverageResults write_results8280408 +Ref: 32d88280408 +Node: tracemalloc — Trace memory allocations8281438 +Ref: library/tracemalloc doc8281598 +Ref: 32d98281598 +Ref: library/tracemalloc module-tracemalloc8281598 +Ref: 1148281598 +Ref: library/tracemalloc tracemalloc-trace-memory-allocations8281598 +Ref: 32da8281598 +Ref: tracemalloc — Trace memory allocations-Footnote-18282935 +Node: Examples<26>8283005 +Ref: library/tracemalloc examples8283106 +Ref: 32db8283106 +Node: Display the top 108283255 +Ref: library/tracemalloc display-the-top-108283350 +Ref: 32dc8283350 +Node: Compute differences8284834 +Ref: library/tracemalloc compute-differences8284973 +Ref: 32de8284973 +Node: Get the traceback of a memory block8287163 +Ref: library/tracemalloc get-the-traceback-of-a-memory-block8287294 +Ref: 32e18287294 +Node: Pretty top8289926 +Ref: library/tracemalloc pretty-top8290029 +Ref: 32e28290029 +Node: API8292416 +Ref: library/tracemalloc api8292517 +Ref: 32e38292517 +Node: Functions<9>8292688 +Ref: library/tracemalloc functions8292761 +Ref: 32e48292761 +Ref: library/tracemalloc tracemalloc clear_traces8292800 +Ref: 32e58292800 +Ref: library/tracemalloc tracemalloc get_object_traceback8292935 +Ref: 32e78292935 +Ref: library/tracemalloc tracemalloc get_traceback_limit8293343 +Ref: 32e98293343 +Ref: library/tracemalloc tracemalloc get_traced_memory8293666 +Ref: 32ea8293666 +Ref: library/tracemalloc tracemalloc get_tracemalloc_memory8293872 +Ref: 32eb8293872 +Ref: library/tracemalloc tracemalloc is_tracing8294070 +Ref: 32ec8294070 +Ref: library/tracemalloc tracemalloc start8294302 +Ref: fea8294302 +Ref: library/tracemalloc tracemalloc stop8295501 +Ref: 32e68295501 +Ref: library/tracemalloc tracemalloc take_snapshot8295934 +Ref: 32ee8295934 +Node: DomainFilter8296632 +Ref: library/tracemalloc domainfilter8296720 +Ref: 32f08296720 +Ref: library/tracemalloc tracemalloc DomainFilter8296765 +Ref: 5a08296765 +Ref: library/tracemalloc tracemalloc DomainFilter inclusive8296929 +Ref: 32f18296929 +Ref: library/tracemalloc tracemalloc DomainFilter domain8297229 +Ref: 32f28297229 +Node: Filter8297334 +Ref: library/tracemalloc filter8297415 +Ref: 32f38297415 +Ref: library/tracemalloc tracemalloc Filter8297448 +Ref: 32f48297448 +Ref: library/tracemalloc tracemalloc Filter domain8298295 +Ref: 32f58298295 +Ref: library/tracemalloc tracemalloc Filter inclusive8298566 +Ref: 32f68298566 +Ref: library/tracemalloc tracemalloc Filter lineno8298998 +Ref: 32f88298998 +Ref: library/tracemalloc tracemalloc Filter filename_pattern8299150 +Ref: 32f78299150 +Ref: library/tracemalloc tracemalloc Filter all_frames8299264 +Ref: 32f98299264 +Node: Frame8299671 +Ref: library/tracemalloc frame8299748 +Ref: 32fb8299748 +Ref: library/tracemalloc tracemalloc Frame8299781 +Ref: 32fc8299781 +Ref: library/tracemalloc tracemalloc Frame filename8299929 +Ref: 32fd8299929 +Ref: library/tracemalloc tracemalloc Frame lineno8299992 +Ref: 32fe8299992 +Node: Snapshot8300056 +Ref: library/tracemalloc snapshot8300136 +Ref: 32ff8300136 +Ref: library/tracemalloc tracemalloc Snapshot8300175 +Ref: 32ef8300175 +Ref: library/tracemalloc tracemalloc Snapshot compare_to8300353 +Ref: 32ed8300353 +Ref: library/tracemalloc tracemalloc Snapshot dump8301079 +Ref: 32df8301079 +Ref: library/tracemalloc tracemalloc Snapshot filter_traces8301215 +Ref: 33068301215 +Ref: library/tracemalloc tracemalloc Snapshot load8301892 +Ref: 32e08301892 +Ref: library/tracemalloc tracemalloc Snapshot statistics8302018 +Ref: 32dd8302018 +Ref: library/tracemalloc tracemalloc Snapshot traceback_limit8303217 +Ref: 32fa8303217 +Ref: library/tracemalloc tracemalloc Snapshot traces8303436 +Ref: 33078303436 +Node: Statistic8303731 +Ref: library/tracemalloc statistic8303819 +Ref: 330c8303819 +Ref: library/tracemalloc tracemalloc Statistic8303860 +Ref: 33088303860 +Ref: library/tracemalloc tracemalloc Statistic count8304084 +Ref: 33048304084 +Ref: library/tracemalloc tracemalloc Statistic size8304159 +Ref: 33098304159 +Ref: library/tracemalloc tracemalloc Statistic traceback8304246 +Ref: 330a8304246 +Node: StatisticDiff8304378 +Ref: library/tracemalloc statisticdiff8304463 +Ref: 330d8304463 +Ref: library/tracemalloc tracemalloc StatisticDiff8304512 +Ref: 33008304512 +Ref: library/tracemalloc tracemalloc StatisticDiff count8304812 +Ref: 330e8304812 +Ref: library/tracemalloc tracemalloc StatisticDiff count_diff8304986 +Ref: 33038304986 +Ref: library/tracemalloc tracemalloc StatisticDiff size8305208 +Ref: 33028305208 +Ref: library/tracemalloc tracemalloc StatisticDiff size_diff8305404 +Ref: 33018305404 +Ref: library/tracemalloc tracemalloc StatisticDiff traceback8305638 +Ref: 33058305638 +Node: Trace8305772 +Ref: library/tracemalloc trace8305857 +Ref: 330f8305857 +Ref: library/tracemalloc tracemalloc Trace8305890 +Ref: 330b8305890 +Ref: library/tracemalloc tracemalloc Trace domain8306123 +Ref: 33108306123 +Ref: library/tracemalloc tracemalloc Trace size8306401 +Ref: 33118306401 +Ref: library/tracemalloc tracemalloc Trace traceback8306485 +Ref: 33128306485 +Node: Traceback8306617 +Ref: library/tracemalloc traceback8306680 +Ref: 33138306680 +Ref: library/tracemalloc tracemalloc Traceback8306721 +Ref: 3ff8306721 +Ref: library/tracemalloc tracemalloc Traceback format8307442 +Ref: 4008307442 +Node: Software Packaging and Distribution8308525 +Ref: library/distribution doc8308688 +Ref: 33158308688 +Ref: library/distribution software-packaging-and-distribution8308688 +Ref: 33168308688 +Ref: Software Packaging and Distribution-Footnote-18309272 +Node: distutils — Building and installing Python modules8309297 +Ref: library/distutils doc8309475 +Ref: 33178309475 +Ref: library/distutils distutils-building-and-installing-python-modules8309475 +Ref: 33188309475 +Ref: library/distutils module-distutils8309475 +Ref: 398309475 +Ref: distutils — Building and installing Python modules-Footnote-18311382 +Ref: distutils — Building and installing Python modules-Footnote-28311435 +Ref: distutils — Building and installing Python modules-Footnote-38311464 +Node: ensurepip — Bootstrapping the pip installer8311501 +Ref: library/ensurepip doc8311729 +Ref: 33198311729 +Ref: library/ensurepip ensurepip-bootstrapping-the-pip-installer8311729 +Ref: 331a8311729 +Ref: library/ensurepip module-ensurepip8311729 +Ref: 7a8311729 +Ref: ensurepip — Bootstrapping the pip installer-Footnote-18313134 +Node: Command line interface8313183 +Ref: library/ensurepip command-line-interface8313306 +Ref: 331b8313306 +Node: Module API8315087 +Ref: library/ensurepip module-api8315210 +Ref: 331c8315210 +Ref: library/ensurepip ensurepip version8315317 +Ref: 331d8315317 +Ref: library/ensurepip ensurepip bootstrap8315477 +Ref: 31f28315477 +Node: venv — Creation of virtual environments8317252 +Ref: library/venv doc8317476 +Ref: 331e8317476 +Ref: library/venv module-venv8317476 +Ref: 1258317476 +Ref: library/venv venv-creation-of-virtual-environments8317476 +Ref: 331f8317476 +Ref: venv — Creation of virtual environments-Footnote-18318404 +Ref: venv — Creation of virtual environments-Footnote-28318465 +Ref: venv — Creation of virtual environments-Footnote-38318514 +Node: Creating virtual environments8318637 +Ref: library/venv creating-virtual-environments8318759 +Ref: 33208318759 +Ref: library/venv venv-def8325765 +Ref: 33218325765 +Ref: Creating virtual environments-Footnote-18328740 +Ref: Creating virtual environments-Footnote-28328814 +Ref: Creating virtual environments-Footnote-38328869 +Ref: Creating virtual environments-Footnote-48328914 +Ref: Creating virtual environments-Footnote-58328952 +Node: API<2>8329001 +Ref: library/venv api8329166 +Ref: 33258329166 +Ref: library/venv venv-api8329166 +Ref: 7f28329166 +Ref: library/venv venv EnvBuilder8329423 +Ref: 9088329423 +Ref: library/venv venv EnvBuilder create8331126 +Ref: 33268331126 +Ref: library/venv venv EnvBuilder ensure_directories8332421 +Ref: 33278332421 +Ref: library/venv venv EnvBuilder create_configuration8332875 +Ref: 33288332875 +Ref: library/venv venv EnvBuilder setup_python8333013 +Ref: 33298333013 +Ref: library/venv venv EnvBuilder setup_scripts8333379 +Ref: 332a8333379 +Ref: library/venv venv EnvBuilder post_setup8333528 +Ref: 332b8333528 +Ref: library/venv venv EnvBuilder install_scripts8334525 +Ref: 332c8334525 +Ref: library/venv venv create8335807 +Ref: 9098335807 +Node: An example of extending EnvBuilder8336248 +Ref: library/venv an-example-of-extending-envbuilder8336375 +Ref: 332d8336375 +Ref: An example of extending EnvBuilder-Footnote-18347110 +Node: zipapp — Manage executable Python zip archives8347157 +Ref: library/zipapp doc8347327 +Ref: 332e8347327 +Ref: library/zipapp module-zipapp8347327 +Ref: 1418347327 +Ref: library/zipapp pip8347327 +Ref: 332f8347327 +Ref: library/zipapp zipapp-manage-executable-python-zip-archives8347327 +Ref: 33308347327 +Ref: zipapp — Manage executable Python zip archives-Footnote-18348103 +Node: Basic Example8348168 +Ref: library/zipapp basic-example8348300 +Ref: 33338348300 +Node: Command-Line Interface<5>8348705 +Ref: library/zipapp command-line-interface8348856 +Ref: 33348348856 +Ref: library/zipapp zipapp-command-line-interface8348856 +Ref: 33318348856 +Ref: library/zipapp cmdoption-zipapp-o8349352 +Ref: 33358349352 +Ref: library/zipapp cmdoption-zipapp-p8349857 +Ref: 33368349857 +Ref: library/zipapp cmdoption-zipapp-m8350141 +Ref: 33378350141 +Ref: library/zipapp cmdoption-zipapp-c8350580 +Ref: 33388350580 +Ref: library/zipapp cmdoption-zipapp-info8350865 +Ref: 33398350865 +Ref: library/zipapp cmdoption-zipapp-h8351076 +Ref: 333a8351076 +Node: Python API8351152 +Ref: library/zipapp python-api8351302 +Ref: 333b8351302 +Ref: library/zipapp zipapp-python-api8351302 +Ref: 33328351302 +Ref: library/zipapp zipapp create_archive8351390 +Ref: 41d8351390 +Ref: library/zipapp zipapp get_interpreter8355173 +Ref: 333c8355173 +Node: Examples<27>8355530 +Ref: library/zipapp examples8355681 +Ref: 333d8355681 +Ref: library/zipapp zipapp-examples8355681 +Ref: 333e8355681 +Node: Specifying the Interpreter8357046 +Ref: library/zipapp specifying-the-interpreter8357231 +Ref: 333f8357231 +Ref: library/zipapp zipapp-specifying-the-interpreter8357231 +Ref: 33408357231 +Node: Creating Standalone Applications with zipapp8358470 +Ref: library/zipapp creating-standalone-applications-with-zipapp8358684 +Ref: 33418358684 +Node: Making a Windows executable8360726 +Ref: library/zipapp making-a-windows-executable8360850 +Ref: 33428360850 +Node: Caveats8363977 +Ref: library/zipapp caveats8364101 +Ref: 33438364101 +Node: The Python Zip Application Archive Format8365891 +Ref: library/zipapp the-python-zip-application-archive-format8366070 +Ref: 33448366070 +Node: Python Runtime Services8367973 +Ref: library/python doc8368139 +Ref: 33458368139 +Ref: library/python python8368139 +Ref: 33468368139 +Ref: library/python python-runtime-services8368139 +Ref: 33478368139 +Node: sys — System-specific parameters and functions8368995 +Ref: library/sys doc8369180 +Ref: 33488369180 +Ref: library/sys module-sys8369180 +Ref: fd8369180 +Ref: library/sys sys-system-specific-parameters-and-functions8369180 +Ref: 33498369180 +Ref: library/sys sys abiflags8369539 +Ref: 2648369539 +Ref: library/sys sys addaudithook8369862 +Ref: 31ec8369862 +Ref: library/sys sys argv8371424 +Ref: bfb8371424 +Ref: library/sys auditing8372310 +Ref: fd18372310 +Ref: library/sys sys audit8372310 +Ref: 31ea8372310 +Ref: library/sys sys base_exec_prefix8373710 +Ref: 33248373710 +Ref: library/sys sys base_prefix8374356 +Ref: 2d508374356 +Ref: library/sys sys byteorder8374992 +Ref: 12cc8374992 +Ref: library/sys sys builtin_module_names8375248 +Ref: 334a8375248 +Ref: library/sys sys call_tracing8375516 +Ref: 334b8375516 +Ref: library/sys sys copyright8375785 +Ref: 334c8375785 +Ref: library/sys sys _clear_type_cache8375893 +Ref: 334d8375893 +Ref: library/sys sys _current_frames8376219 +Ref: d9b8376219 +Ref: library/sys sys breakpointhook8377088 +Ref: 3138377088 +Ref: library/sys sys _debugmallocstats8378986 +Ref: 334e8378986 +Ref: library/sys sys dllhandle8379412 +Ref: 334f8379412 +Ref: library/sys sys displayhook8379533 +Ref: e5f8379533 +Ref: library/sys sys dont_write_bytecode8381060 +Ref: 33518381060 +Ref: library/sys sys pycache_prefix8381457 +Ref: 1568381457 +Ref: library/sys sys excepthook8382428 +Ref: e638382428 +Ref: library/sys sys __breakpointhook__8383753 +Ref: 33528383753 +Ref: library/sys sys __displayhook__8383786 +Ref: 33538383786 +Ref: library/sys sys __excepthook__8383816 +Ref: 33548383816 +Ref: library/sys sys __unraisablehook__8383845 +Ref: 33558383845 +Ref: library/sys sys exc_info8384353 +Ref: c5f8384353 +Ref: library/sys sys exec_prefix8385575 +Ref: 33238385575 +Ref: library/sys sys executable8386495 +Ref: 2748386495 +Ref: library/sys sys exit8386796 +Ref: ce28386796 +Ref: library/sys sys flags8388468 +Ref: b3c8388468 +Ref: library/sys sys float_info8391452 +Ref: 12c68391452 +Ref: library/sys sys float_repr_style8396405 +Ref: c908396405 +Ref: library/sys sys getallocatedblocks8396944 +Ref: 8e28396944 +Ref: library/sys sys getandroidapilevel8397581 +Ref: 3f18397581 +Ref: library/sys sys getcheckinterval8397755 +Ref: e088397755 +Ref: library/sys sys getdefaultencoding8397977 +Ref: 33598397977 +Ref: library/sys sys getdlopenflags8398120 +Ref: e4e8398120 +Ref: library/sys sys getfilesystemencoding8398431 +Ref: 4f68398431 +Ref: library/sys sys getfilesystemencodeerrors8399839 +Ref: 5948399839 +Ref: library/sys sys getrefcount8400242 +Ref: 335a8400242 +Ref: library/sys sys getrecursionlimit8400500 +Ref: e7a8400500 +Ref: library/sys sys getsizeof8400812 +Ref: 32e88400812 +Ref: library/sys sys getswitchinterval8401764 +Ref: 33588401764 +Ref: library/sys sys _getframe8401934 +Ref: e648401934 +Ref: library/sys sys getprofile8402581 +Ref: d478402581 +Ref: library/sys sys gettrace8402681 +Ref: d488402681 +Ref: library/sys sys getwindowsversion8403119 +Ref: 5958403119 +Ref: library/sys sys get_asyncgen_hooks8405422 +Ref: 335b8405422 +Ref: library/sys sys get_coroutine_origin_tracking_depth8406029 +Ref: 3f28406029 +Ref: library/sys sys hash_info8406379 +Ref: 9218406379 +Ref: library/sys sys hexversion8408080 +Ref: 335d8408080 +Ref: library/sys sys implementation8408973 +Ref: 9aa8408973 +Ref: library/sys sys int_info8411174 +Ref: c218411174 +Ref: library/sys sys __interactivehook__8412036 +Ref: 8e48412036 +Ref: library/sys sys intern8412581 +Ref: c6e8412581 +Ref: library/sys sys is_finalizing8413349 +Ref: 7628413349 +Ref: library/sys sys last_type8413530 +Ref: c5a8413530 +Ref: library/sys sys last_value8413554 +Ref: 335f8413554 +Ref: library/sys sys last_traceback8413579 +Ref: 325a8413579 +Ref: library/sys sys maxsize8414233 +Ref: b438414233 +Ref: library/sys sys maxunicode8414449 +Ref: 98b8414449 +Ref: library/sys sys meta_path8414848 +Ref: 5e68414848 +Ref: library/sys sys modules8416089 +Ref: 11528416089 +Ref: library/sys sys path8416446 +Ref: 4888416446 +Ref: library/sys sys path_hooks8417538 +Ref: 95c8417538 +Ref: library/sys sys path_importer_cache8417829 +Ref: 49d8417829 +Ref: library/sys sys platform8418355 +Ref: 2b18418355 +Ref: library/sys sys prefix8420774 +Ref: 33228420774 +Ref: library/sys sys ps18421671 +Ref: ff18421671 +Ref: library/sys sys ps28421689 +Ref: ff28421689 +Ref: library/sys sys setcheckinterval8422165 +Ref: 33578422165 +Ref: library/sys sys setdlopenflags8422954 +Ref: a668422954 +Ref: library/sys sys setprofile8423553 +Ref: e3d8423553 +Ref: library/sys sys setrecursionlimit8425692 +Ref: e7b8425692 +Ref: library/sys sys setswitchinterval8426484 +Ref: beb8426484 +Ref: library/sys sys settrace8427061 +Ref: e3e8427061 +Ref: library/sys sys set_asyncgen_hooks8431595 +Ref: 11b28431595 +Ref: library/sys sys set_coroutine_origin_tracking_depth8432697 +Ref: 3f38432697 +Ref: library/sys sys _enablelegacywindowsfsencoding8433512 +Ref: 4f88433512 +Ref: library/sys sys stdin8433977 +Ref: 30d8433977 +Ref: library/sys sys stdout8433997 +Ref: 30e8433997 +Ref: library/sys sys stderr8434018 +Ref: 30f8434018 +Ref: library/sys sys __stdin__8436907 +Ref: 33618436907 +Ref: library/sys sys __stdout__8436931 +Ref: 195e8436931 +Ref: library/sys sys __stderr__8436956 +Ref: 33628436956 +Ref: library/sys sys thread_info8437921 +Ref: aa28437921 +Ref: library/sys sys tracebacklimit8439372 +Ref: 33638439372 +Ref: library/sys sys unraisablehook8439736 +Ref: 22d8439736 +Ref: library/sys sys version8441470 +Ref: 5d38441470 +Ref: library/sys sys api_version8441872 +Ref: 33648441872 +Ref: library/sys sys version_info8442060 +Ref: b1a8442060 +Ref: library/sys sys warnoptions8442681 +Ref: 4168442681 +Ref: library/sys sys winver8442900 +Ref: 33658442900 +Ref: library/sys sys _xoptions8443337 +Ref: fec8443337 +Ref: library/sys c998444136 +Ref: 33568444136 +Ref: sys — System-specific parameters and functions-Footnote-18444347 +Ref: sys — System-specific parameters and functions-Footnote-28444396 +Ref: sys — System-specific parameters and functions-Footnote-38444445 +Ref: sys — System-specific parameters and functions-Footnote-48444494 +Ref: sys — System-specific parameters and functions-Footnote-58444546 +Ref: sys — System-specific parameters and functions-Footnote-68444595 +Ref: sys — System-specific parameters and functions-Footnote-78444644 +Ref: sys — System-specific parameters and functions-Footnote-88444693 +Ref: sys — System-specific parameters and functions-Footnote-98444742 +Ref: sys — System-specific parameters and functions-Footnote-108444791 +Ref: sys — System-specific parameters and functions-Footnote-118444841 +Ref: sys — System-specific parameters and functions-Footnote-128444891 +Ref: sys — System-specific parameters and functions-Footnote-138444941 +Ref: sys — System-specific parameters and functions-Footnote-148444991 +Ref: sys — System-specific parameters and functions-Footnote-158445041 +Ref: sys — System-specific parameters and functions-Footnote-168445121 +Ref: sys — System-specific parameters and functions-Footnote-178445171 +Ref: sys — System-specific parameters and functions-Footnote-188445221 +Node: sysconfig — Provide access to Python’s configuration information8445271 +Ref: library/sysconfig doc8445494 +Ref: 33668445494 +Ref: library/sysconfig module-sysconfig8445494 +Ref: fe8445494 +Ref: library/sysconfig sysconfig-provide-access-to-python-s-configuration-information8445494 +Ref: 33678445494 +Ref: sysconfig — Provide access to Python’s configuration information-Footnote-18446136 +Node: Configuration variables8446204 +Ref: library/sysconfig configuration-variables8446359 +Ref: 33688446359 +Ref: library/sysconfig sysconfig get_config_vars8446852 +Ref: 9658446852 +Ref: library/sysconfig sysconfig get_config_var8447223 +Ref: 9648447223 +Node: Installation paths8447666 +Ref: library/sysconfig installation-paths8447848 +Ref: 33698447848 +Ref: library/sysconfig sysconfig get_scheme_names8449823 +Ref: 336a8449823 +Ref: library/sysconfig sysconfig get_path_names8449961 +Ref: 336b8449961 +Ref: library/sysconfig sysconfig get_path8450100 +Ref: cbe8450100 +Ref: library/sysconfig sysconfig get_paths8451309 +Ref: bd98451309 +Node: Other functions<3>8451932 +Ref: library/sysconfig other-functions8452118 +Ref: 336c8452118 +Ref: library/sysconfig sysconfig get_python_version8452169 +Ref: bd88452169 +Ref: library/sysconfig sysconfig get_platform8452340 +Ref: bd78452340 +Ref: library/sysconfig sysconfig is_python_build8453346 +Ref: cbf8453346 +Ref: library/sysconfig sysconfig parse_config_h8453637 +Ref: 336d8453637 +Ref: library/sysconfig sysconfig get_config_h_filename8454031 +Ref: 336e8454031 +Ref: library/sysconfig sysconfig get_makefile_filename8454124 +Ref: 336f8454124 +Node: Using sysconfig as a script8454215 +Ref: library/sysconfig using-sysconfig-as-a-script8454374 +Ref: 33708454374 +Node: builtins — Built-in objects8455447 +Ref: library/builtins doc8455663 +Ref: 33718455663 +Ref: library/builtins builtins-built-in-objects8455663 +Ref: 33728455663 +Ref: library/builtins module-builtins8455663 +Ref: 138455663 +Node: __main__ — Top-level script environment8457164 +Ref: library/__main__ doc8457340 +Ref: 33738457340 +Ref: library/__main__ main-top-level-script-environment8457340 +Ref: 33748457340 +Ref: library/__main__ module-__main__8457340 +Ref: 18457340 +Node: warnings — Warning control8458234 +Ref: library/warnings doc8458409 +Ref: 33758458409 +Ref: library/warnings module-warnings8458409 +Ref: 1268458409 +Ref: library/warnings warnings-warning-control8458409 +Ref: 33768458409 +Ref: warnings — Warning control-Footnote-18460720 +Node: Warning Categories8460787 +Ref: library/warnings id18460898 +Ref: 337a8460898 +Ref: library/warnings warning-categories8460898 +Ref: 13c18460898 +Node: The Warnings Filter8464880 +Ref: library/warnings the-warnings-filter8465032 +Ref: 337b8465032 +Ref: library/warnings warning-filter8465032 +Ref: fe78465032 +Node: Describing Warning Filters8467767 +Ref: library/warnings describing-warning-filters8467880 +Ref: fe88467880 +Ref: library/warnings id28467880 +Ref: 337c8467880 +Node: Default Warning Filter8469675 +Ref: library/warnings default-warning-filter8469826 +Ref: 337d8469826 +Ref: library/warnings id38469826 +Ref: 337e8469826 +Node: Overriding the default filter8470913 +Ref: library/warnings overriding-the-default-filter8471029 +Ref: 337f8471029 +Ref: library/warnings warning-disable8471029 +Ref: 33808471029 +Node: Temporarily Suppressing Warnings8472464 +Ref: library/warnings temporarily-suppressing-warnings8472614 +Ref: 33818472614 +Ref: library/warnings warning-suppress8472614 +Ref: 33828472614 +Node: Testing Warnings8473632 +Ref: library/warnings testing-warnings8473809 +Ref: 33838473809 +Ref: library/warnings warning-testing8473809 +Ref: 33848473809 +Node: Updating Code For New Versions of Dependencies8475873 +Ref: library/warnings updating-code-for-new-versions-of-dependencies8476037 +Ref: 33858476037 +Ref: library/warnings warning-ignored8476037 +Ref: 308a8476037 +Node: Available Functions8477427 +Ref: library/warnings available-functions8477601 +Ref: 33868477601 +Ref: library/warnings warning-functions8477601 +Ref: 33878477601 +Ref: library/warnings warnings warn8477660 +Ref: e588477660 +Ref: library/warnings warnings warn_explicit8478879 +Ref: 5b08478879 +Ref: library/warnings warnings showwarning8480067 +Ref: 33798480067 +Ref: library/warnings warnings formatwarning8480705 +Ref: 1dad8480705 +Ref: library/warnings warnings filterwarnings8481137 +Ref: e078481137 +Ref: library/warnings warnings simplefilter8481813 +Ref: 317c8481813 +Ref: library/warnings warnings resetwarnings8482248 +Ref: 33788482248 +Node: Available Context Managers8482516 +Ref: library/warnings available-context-managers8482635 +Ref: 33888482635 +Ref: library/warnings warnings catch_warnings8482710 +Ref: 317b8482710 +Node: dataclasses — Data Classes8483959 +Ref: library/dataclasses doc8484145 +Ref: 33898484145 +Ref: library/dataclasses dataclasses-data-classes8484145 +Ref: 338a8484145 +Ref: library/dataclasses module-dataclasses8484145 +Ref: 308484145 +Ref: dataclasses — Data Classes-Footnote-18485806 +Ref: dataclasses — Data Classes-Footnote-28485876 +Ref: dataclasses — Data Classes-Footnote-38485925 +Node: Module-level decorators classes and functions8485974 +Ref: library/dataclasses module-level-decorators-classes-and-functions8486113 +Ref: 338c8486113 +Ref: library/dataclasses dataclasses dataclass8486228 +Ref: 33d8486228 +Ref: library/dataclasses dataclasses field8493200 +Ref: 338d8493200 +Ref: library/dataclasses dataclasses Field8497466 +Ref: 338e8497466 +Ref: library/dataclasses dataclasses fields8498259 +Ref: 338f8498259 +Ref: library/dataclasses dataclasses asdict8498638 +Ref: 33908498638 +Ref: library/dataclasses dataclasses astuple8499436 +Ref: 33918499436 +Ref: library/dataclasses dataclasses make_dataclass8499972 +Ref: 33928499972 +Ref: library/dataclasses dataclasses replace8501497 +Ref: 33938501497 +Ref: library/dataclasses dataclasses is_dataclass8502914 +Ref: 33948502914 +Node: Post-init processing8503369 +Ref: library/dataclasses post-init-processing8503532 +Ref: 33958503532 +Node: Class variables8504533 +Ref: library/dataclasses class-variables8504670 +Ref: 33968504670 +Ref: Class variables-Footnote-18505220 +Node: Init-only variables8505269 +Ref: library/dataclasses init-only-variables8505402 +Ref: 33978505402 +Node: Frozen instances8506613 +Ref: library/dataclasses frozen-instances8506745 +Ref: 33988506745 +Node: Inheritance<2>8507346 +Ref: library/dataclasses inheritance8507484 +Ref: 339a8507484 +Node: Default factory functions8508536 +Ref: library/dataclasses default-factory-functions8508680 +Ref: 339b8508680 +Node: Mutable default values8509358 +Ref: library/dataclasses mutable-default-values8509502 +Ref: 339c8509502 +Node: Exceptions<17>8511396 +Ref: library/dataclasses exceptions8511506 +Ref: 339d8511506 +Ref: library/dataclasses dataclasses FrozenInstanceError8511547 +Ref: 33998511547 +Node: contextlib — Utilities for with-statement contexts8511768 +Ref: library/contextlib doc8511955 +Ref: 339e8511955 +Ref: library/contextlib contextlib-utilities-for-with-statement-contexts8511955 +Ref: 339f8511955 +Ref: library/contextlib module-contextlib8511955 +Ref: 248511955 +Ref: contextlib — Utilities for with-statement contexts-Footnote-18512617 +Node: Utilities8512686 +Ref: library/contextlib utilities8512816 +Ref: 33a08512816 +Ref: library/contextlib contextlib AbstractContextManager8512888 +Ref: 5348512888 +Ref: library/contextlib contextlib AbstractAsyncContextManager8513374 +Ref: 3788513374 +Ref: library/contextlib contextlib contextmanager8513881 +Ref: b7e8513881 +Ref: library/contextlib contextlib asynccontextmanager8516849 +Ref: 3778516849 +Ref: library/contextlib contextlib closing8517859 +Ref: 33a18517859 +Ref: library/contextlib simplifying-support-for-single-optional-context-managers8518654 +Ref: 33a28518654 +Ref: library/contextlib contextlib nullcontext8518654 +Ref: 3758518654 +Ref: library/contextlib contextlib suppress8519766 +Ref: 82c8519766 +Ref: library/contextlib contextlib redirect_stdout8520871 +Ref: 6c28520871 +Ref: library/contextlib contextlib redirect_stderr8522207 +Ref: 6c18522207 +Ref: library/contextlib contextlib ContextDecorator8522469 +Ref: b7d8522469 +Ref: library/contextlib contextlib ExitStack8524863 +Ref: 3768524863 +Ref: library/contextlib contextlib ExitStack enter_context8526729 +Ref: 33a48526729 +Ref: library/contextlib contextlib ExitStack push8527161 +Ref: 33a58527161 +Ref: library/contextlib contextlib ExitStack callback8528049 +Ref: 2a58528049 +Ref: library/contextlib contextlib ExitStack pop_all8528502 +Ref: 33a68528502 +Ref: library/contextlib contextlib ExitStack close8529554 +Ref: 33a78529554 +Ref: library/contextlib contextlib AsyncExitStack8529842 +Ref: 3798529842 +Ref: library/contextlib contextlib AsyncExitStack enter_async_context8530203 +Ref: 33a98530203 +Ref: library/contextlib contextlib AsyncExitStack push_async_exit8530345 +Ref: 33aa8530345 +Ref: library/contextlib contextlib AsyncExitStack push_async_callback8530507 +Ref: 2a68530507 +Ref: library/contextlib contextlib AsyncExitStack aclose8530644 +Ref: 33a88530644 +Node: Examples and Recipes<2>8531218 +Ref: library/contextlib examples-and-recipes8531407 +Ref: 33ab8531407 +Node: Supporting a variable number of context managers8531855 +Ref: library/contextlib supporting-a-variable-number-of-context-managers8532014 +Ref: 33ac8532014 +Node: Catching exceptions from __enter__ methods8533056 +Ref: library/contextlib catching-exceptions-from-enter-methods8533266 +Ref: 33ad8533266 +Node: Cleaning up in an __enter__ implementation8534414 +Ref: library/contextlib cleaning-up-in-an-enter-implementation8534627 +Ref: 33ae8534627 +Node: Replacing any use of try-finally and flag variables8536530 +Ref: library/contextlib replacing-any-use-of-try-finally-and-flag-variables8536748 +Ref: 33af8536748 +Node: Using a context manager as a function decorator8539277 +Ref: library/contextlib using-a-context-manager-as-a-function-decorator8539444 +Ref: 33b08539444 +Ref: Using a context manager as a function decorator-Footnote-18541278 +Node: Single use reusable and reentrant context managers8541327 +Ref: library/contextlib single-use-reusable-and-reentrant-cms8541498 +Ref: 82e8541498 +Ref: library/contextlib single-use-reusable-and-reentrant-context-managers8541498 +Ref: 33b18541498 +Node: Reentrant context managers8543035 +Ref: library/contextlib reentrant-cms8543182 +Ref: 33a38543182 +Ref: library/contextlib reentrant-context-managers8543182 +Ref: 33b28543182 +Node: Reusable context managers8544698 +Ref: library/contextlib reusable-cms8544845 +Ref: 33b38544845 +Ref: library/contextlib reusable-context-managers8544845 +Ref: 33b48544845 +Node: abc — Abstract Base Classes8547523 +Ref: library/abc doc8547706 +Ref: 33b58547706 +Ref: library/abc abc-abstract-base-classes8547706 +Ref: 33b68547706 +Ref: library/abc module-abc8547706 +Ref: 48547706 +Ref: library/abc abc ABC8548671 +Ref: 8188548671 +Ref: library/abc abc ABCMeta8549516 +Ref: 8198549516 +Ref: library/abc abc ABCMeta register8550296 +Ref: 9d18550296 +Ref: library/abc abc ABCMeta __subclasshook__8550978 +Ref: 33b78550978 +Ref: library/abc abc abstractmethod8553875 +Ref: 9ce8553875 +Ref: library/abc abc abstractclassmethod8556975 +Ref: 9cf8556975 +Ref: library/abc abc abstractstaticmethod8557691 +Ref: 9d08557691 +Ref: library/abc abc abstractproperty8558409 +Ref: 9cd8558409 +Ref: library/abc abc get_cache_token8559796 +Ref: 8178559796 +Ref: abc — Abstract Base Classes-Footnote-18560209 +Ref: abc — Abstract Base Classes-Footnote-28560271 +Ref: abc — Abstract Base Classes-Footnote-38560320 +Ref: abc — Abstract Base Classes-Footnote-48560369 +Node: atexit — Exit handlers8560480 +Ref: library/atexit doc8560660 +Ref: 33b88560660 +Ref: library/atexit atexit-exit-handlers8560660 +Ref: 33b98560660 +Ref: library/atexit module-atexit8560660 +Ref: c8560660 +Ref: library/atexit atexit register8561568 +Ref: e858561568 +Ref: library/atexit atexit unregister8562619 +Ref: 33ba8562619 +Node: atexit Example8563180 +Ref: library/atexit atexit-example8563255 +Ref: 33bb8563255 +Ref: library/atexit id18563255 +Ref: 33bc8563255 +Node: traceback — Print or retrieve a stack traceback8564616 +Ref: library/traceback doc8564810 +Ref: 33bd8564810 +Ref: library/traceback module-traceback8564810 +Ref: 1138564810 +Ref: library/traceback traceback-print-or-retrieve-a-stack-traceback8564810 +Ref: 33be8564810 +Ref: library/traceback traceback print_tb8565590 +Ref: 7798565590 +Ref: library/traceback traceback print_exception8566141 +Ref: 1d8b8566141 +Ref: library/traceback traceback print_exc8567290 +Ref: 33bf8567290 +Ref: library/traceback traceback print_last8567456 +Ref: 33c08567456 +Ref: library/traceback traceback print_stack8567783 +Ref: 77a8567783 +Ref: library/traceback traceback extract_tb8568328 +Ref: 33c18568328 +Ref: library/traceback traceback extract_stack8569069 +Ref: 33c28569069 +Ref: library/traceback traceback format_list8569372 +Ref: 33c38569372 +Ref: library/traceback traceback format_exception_only8569881 +Ref: 30048569881 +Ref: library/traceback traceback format_exception8570500 +Ref: 33c48570500 +Ref: library/traceback traceback format_exc8571109 +Ref: 33c58571109 +Ref: library/traceback traceback format_tb8571269 +Ref: 33148571269 +Ref: library/traceback traceback format_stack8571385 +Ref: 33c68571385 +Ref: library/traceback traceback clear_frames8571510 +Ref: 8f18571510 +Ref: library/traceback traceback walk_stack8571719 +Ref: 7748571719 +Ref: library/traceback traceback walk_tb8572029 +Ref: 7758572029 +Ref: traceback — Print or retrieve a stack traceback-Footnote-18572461 +Node: TracebackException Objects8572529 +Ref: library/traceback tracebackexception-objects8572670 +Ref: 33c88572670 +Ref: library/traceback traceback TracebackException8572910 +Ref: 7768572910 +Ref: library/traceback traceback TracebackException __cause__8573295 +Ref: 33c98573295 +Ref: library/traceback traceback TracebackException __context__8573403 +Ref: 33ca8573403 +Ref: library/traceback traceback TracebackException __suppress_context__8573525 +Ref: 33cb8573525 +Ref: library/traceback traceback TracebackException stack8573644 +Ref: 33cc8573644 +Ref: library/traceback traceback TracebackException exc_type8573737 +Ref: 33cd8573737 +Ref: library/traceback traceback TracebackException filename8573815 +Ref: 33ce8573815 +Ref: library/traceback traceback TracebackException lineno8573916 +Ref: 33cf8573916 +Ref: library/traceback traceback TracebackException text8574017 +Ref: 33d08574017 +Ref: library/traceback traceback TracebackException offset8574109 +Ref: 33d18574109 +Ref: library/traceback traceback TracebackException msg8574229 +Ref: 33d28574229 +Ref: library/traceback traceback TracebackException from_exception8574313 +Ref: 33d38574313 +Ref: library/traceback traceback TracebackException format8574696 +Ref: 33d48574696 +Ref: library/traceback traceback TracebackException format_exception_only8575244 +Ref: 33d58575244 +Node: StackSummary Objects8575791 +Ref: library/traceback stacksummary-objects8575961 +Ref: 33d68575961 +Ref: library/traceback traceback StackSummary8576132 +Ref: 7778576132 +Ref: library/traceback traceback StackSummary extract8576167 +Ref: 33c78576167 +Ref: library/traceback traceback StackSummary from_list8576992 +Ref: 33d78576992 +Ref: library/traceback traceback StackSummary format8577295 +Ref: 33d88577295 +Node: FrameSummary Objects8577915 +Ref: library/traceback framesummary-objects8578077 +Ref: 33d98578077 +Ref: library/traceback traceback FrameSummary8578244 +Ref: 7788578244 +Node: Traceback Examples8578989 +Ref: library/traceback traceback-example8579122 +Ref: 33da8579122 +Ref: library/traceback traceback-examples8579122 +Ref: 33db8579122 +Node: __future__ — Future statement definitions8584518 +Ref: library/__future__ doc8584722 +Ref: 33dc8584722 +Ref: library/__future__ future-future-statement-definitions8584722 +Ref: 33dd8584722 +Ref: library/__future__ module-__future__8584722 +Ref: 08584722 +Ref: __future__ — Future statement definitions-Footnote-18589927 +Ref: __future__ — Future statement definitions-Footnote-28589996 +Ref: __future__ — Future statement definitions-Footnote-38590045 +Ref: __future__ — Future statement definitions-Footnote-48590094 +Ref: __future__ — Future statement definitions-Footnote-58590143 +Ref: __future__ — Future statement definitions-Footnote-68590192 +Ref: __future__ — Future statement definitions-Footnote-78590241 +Ref: __future__ — Future statement definitions-Footnote-88590290 +Ref: __future__ — Future statement definitions-Footnote-98590339 +Ref: __future__ — Future statement definitions-Footnote-108590388 +Node: gc — Garbage Collector interface8590438 +Ref: library/gc doc8590625 +Ref: 33de8590625 +Ref: library/gc gc-garbage-collector-interface8590625 +Ref: 33df8590625 +Ref: library/gc module-gc8590625 +Ref: 868590625 +Ref: library/gc gc enable8591552 +Ref: 33e08591552 +Ref: library/gc gc disable8591623 +Ref: 33e18591623 +Ref: library/gc gc isenabled8591696 +Ref: 33e28591696 +Ref: library/gc gc collect8591787 +Ref: 22e8591787 +Ref: library/gc gc set_debug8592422 +Ref: 33e38592422 +Ref: library/gc gc get_debug8592689 +Ref: 33e48592689 +Ref: library/gc gc get_objects8592768 +Ref: 1cd8592768 +Ref: library/gc gc get_stats8593184 +Ref: 84e8593184 +Ref: library/gc gc set_threshold8593878 +Ref: 33e58593878 +Ref: library/gc gc get_count8595104 +Ref: 33e68595104 +Ref: library/gc gc get_threshold8595229 +Ref: 33e78595229 +Ref: library/gc gc get_referrers8595374 +Ref: 31f58595374 +Ref: library/gc gc get_referents8596415 +Ref: 31f48596415 +Ref: library/gc gc is_tracked8597149 +Ref: ca38597149 +Ref: library/gc gc freeze8597974 +Ref: 38c8597974 +Ref: library/gc gc unfreeze8598508 +Ref: 38d8598508 +Ref: library/gc gc get_freeze_count8598667 +Ref: 38e8598667 +Ref: library/gc gc garbage8598910 +Ref: b408598910 +Ref: library/gc gc callbacks8599776 +Ref: a0a8599776 +Ref: library/gc gc DEBUG_STATS8601107 +Ref: 33ea8601107 +Ref: library/gc gc DEBUG_COLLECTABLE8601250 +Ref: 33eb8601250 +Ref: library/gc gc DEBUG_UNCOLLECTABLE8601336 +Ref: b418601336 +Ref: library/gc gc DEBUG_SAVEALL8601716 +Ref: 33e98601716 +Ref: library/gc gc DEBUG_LEAK8601908 +Ref: 33ec8601908 +Ref: gc — Garbage Collector interface-Footnote-18602152 +Ref: gc — Garbage Collector interface-Footnote-28602241 +Node: inspect — Inspect live objects8602290 +Ref: library/inspect doc8602475 +Ref: 33ed8602475 +Ref: library/inspect inspect-inspect-live-objects8602475 +Ref: 33ee8602475 +Ref: library/inspect module-inspect8602475 +Ref: a08602475 +Ref: inspect — Inspect live objects-Footnote-18603647 +Node: Types and members8603713 +Ref: library/inspect inspect-types8603830 +Ref: 33ef8603830 +Ref: library/inspect types-and-members8603830 +Ref: 33f08603830 +Ref: library/inspect inspect getmembers8617492 +Ref: 33f18617492 +Ref: library/inspect inspect getmodulename8618104 +Ref: 5f28618104 +Ref: library/inspect inspect ismodule8618781 +Ref: 33f48618781 +Ref: library/inspect inspect isclass8618873 +Ref: 33f58618873 +Ref: library/inspect inspect ismethod8619012 +Ref: 33f68619012 +Ref: library/inspect inspect isfunction8619128 +Ref: 33f78619128 +Ref: library/inspect inspect isgeneratorfunction8619306 +Ref: 33f98619306 +Ref: library/inspect inspect isgenerator8619599 +Ref: 33fa8619599 +Ref: library/inspect inspect iscoroutinefunction8619697 +Ref: 6f88619697 +Ref: library/inspect inspect iscoroutine8620088 +Ref: 6f78620088 +Ref: library/inspect inspect isawaitable8620274 +Ref: 6f98620274 +Ref: library/inspect inspect isasyncgenfunction8620724 +Ref: 33fb8620724 +Ref: library/inspect inspect isasyncgen8621234 +Ref: 33fc8621234 +Ref: library/inspect inspect istraceback8621462 +Ref: 33fd8621462 +Ref: library/inspect inspect isframe8621560 +Ref: 33fe8621560 +Ref: library/inspect inspect iscode8621650 +Ref: 33ff8621650 +Ref: library/inspect inspect isbuiltin8621738 +Ref: 34008621738 +Ref: library/inspect inspect isroutine8621874 +Ref: 34018621874 +Ref: library/inspect inspect isabstract8622009 +Ref: 34028622009 +Ref: library/inspect inspect ismethoddescriptor8622117 +Ref: 34038622117 +Ref: library/inspect inspect isdatadescriptor8623000 +Ref: 34048623000 +Ref: library/inspect inspect isgetsetdescriptor8623629 +Ref: 34058623629 +Ref: library/inspect inspect ismemberdescriptor8623980 +Ref: 34068623980 +Node: Retrieving source code8624342 +Ref: library/inspect inspect-source8624517 +Ref: 34078624517 +Ref: library/inspect retrieving-source-code8624517 +Ref: 34088624517 +Ref: library/inspect inspect getdoc8624584 +Ref: 1d68624584 +Ref: library/inspect inspect getcomments8625017 +Ref: 2fc28625017 +Ref: library/inspect inspect getfile8625441 +Ref: 340a8625441 +Ref: library/inspect inspect getmodule8625675 +Ref: 340b8625675 +Ref: library/inspect inspect getsourcefile8625775 +Ref: 340c8625775 +Ref: library/inspect inspect getsourcelines8626012 +Ref: 340d8626012 +Ref: library/inspect inspect getsource8626631 +Ref: 340e8626631 +Ref: library/inspect inspect cleandoc8627081 +Ref: 34098627081 +Node: Introspecting callables with the Signature object8627492 +Ref: library/inspect inspect-signature-object8627674 +Ref: 340f8627674 +Ref: library/inspect introspecting-callables-with-the-signature-object8627674 +Ref: 34108627674 +Ref: library/inspect inspect signature8627991 +Ref: 55b8627991 +Ref: library/inspect inspect Signature8629383 +Ref: 6f38629383 +Ref: library/inspect inspect Signature empty8630414 +Ref: 34138630414 +Ref: library/inspect inspect Signature parameters8630534 +Ref: 34118630534 +Ref: library/inspect inspect Signature return_annotation8631029 +Ref: 34148631029 +Ref: library/inspect inspect Signature bind8631248 +Ref: 34158631248 +Ref: library/inspect inspect Signature bind_partial8631525 +Ref: 34168631525 +Ref: library/inspect inspect Signature replace8631901 +Ref: 34128631901 +Ref: library/inspect inspect Signature from_callable8632604 +Ref: 6f68632604 +Ref: library/inspect inspect Parameter8633186 +Ref: 6f48633186 +Ref: library/inspect inspect Parameter empty8633541 +Ref: 34188633541 +Ref: library/inspect inspect Parameter name8633672 +Ref: 34198633672 +Ref: library/inspect inspect Parameter default8634136 +Ref: 341a8634136 +Ref: library/inspect inspect Parameter annotation8634327 +Ref: 341b8634327 +Ref: library/inspect inspect Parameter kind8634515 +Ref: 341c8634515 +Ref: library/inspect inspect Parameter kind description8637339 +Ref: 341d8637339 +Ref: library/inspect inspect Parameter replace8637874 +Ref: 34178637874 +Ref: library/inspect inspect BoundArguments8638942 +Ref: 9a88638942 +Ref: library/inspect inspect BoundArguments arguments8639146 +Ref: 341e8639146 +Ref: library/inspect inspect BoundArguments args8639884 +Ref: 341f8639884 +Ref: library/inspect inspect BoundArguments kwargs8640036 +Ref: 34208640036 +Ref: library/inspect inspect BoundArguments signature8640186 +Ref: 34218640186 +Ref: library/inspect inspect BoundArguments apply_defaults8640284 +Ref: 6f58640284 +Ref: Introspecting callables with the Signature object-Footnote-18641280 +Node: Classes and functions<2>8641329 +Ref: library/inspect classes-and-functions8641510 +Ref: 34228641510 +Ref: library/inspect inspect-classes-functions8641510 +Ref: 34238641510 +Ref: library/inspect inspect getclasstree8641575 +Ref: 34248641575 +Ref: library/inspect inspect getargspec8642154 +Ref: 55c8642154 +Ref: library/inspect inspect getfullargspec8643091 +Ref: 55d8643091 +Ref: library/inspect inspect getargvalues8645465 +Ref: 7b88645465 +Ref: library/inspect inspect formatargspec8645970 +Ref: 7b78645970 +Ref: library/inspect inspect formatargvalues8647095 +Ref: 7b98647095 +Ref: library/inspect inspect getmro8647565 +Ref: 34258647565 +Ref: library/inspect inspect getcallargs8647929 +Ref: 7b68647929 +Ref: library/inspect inspect getclosurevars8649278 +Ref: a198649278 +Ref: library/inspect inspect unwrap8649994 +Ref: 8698649994 +Node: The interpreter stack8650728 +Ref: library/inspect inspect-stack8650890 +Ref: 34268650890 +Ref: library/inspect the-interpreter-stack8650890 +Ref: 34278650890 +Ref: library/inspect inspect getframeinfo8652890 +Ref: 34288652890 +Ref: library/inspect inspect getouterframes8653117 +Ref: 6fe8653117 +Ref: library/inspect inspect getinnerframes8653613 +Ref: 6ff8653613 +Ref: library/inspect inspect currentframe8654106 +Ref: 34298654106 +Ref: library/inspect inspect stack8654507 +Ref: 6fc8654507 +Ref: library/inspect inspect trace8654901 +Ref: 6fd8654901 +Node: Fetching attributes statically8655379 +Ref: library/inspect fetching-attributes-statically8655559 +Ref: 342a8655559 +Ref: library/inspect inspect getattr_static8656141 +Ref: bcb8656141 +Node: Current State of Generators and Coroutines8657946 +Ref: library/inspect current-state-of-generators-and-coroutines8658127 +Ref: 342b8658127 +Ref: library/inspect inspect getgeneratorstate8658556 +Ref: bca8658556 +Ref: library/inspect inspect getcoroutinestate8658969 +Ref: 6fb8658969 +Ref: library/inspect inspect getgeneratorlocals8659804 +Ref: a1a8659804 +Ref: library/inspect inspect getcoroutinelocals8660678 +Ref: 6fa8660678 +Node: Code Objects Bit Flags8660913 +Ref: library/inspect code-objects-bit-flags8661089 +Ref: 342c8661089 +Ref: library/inspect inspect-module-co-flags8661089 +Ref: 33f28661089 +Ref: library/inspect inspect CO_OPTIMIZED8661252 +Ref: 342d8661252 +Ref: library/inspect inspect CO_NEWLOCALS8661339 +Ref: 342e8661339 +Ref: library/inspect inspect CO_VARARGS8661484 +Ref: 342f8661484 +Ref: library/inspect inspect CO_VARKEYWORDS8661592 +Ref: 34308661592 +Ref: library/inspect inspect CO_NESTED8661704 +Ref: 34318661704 +Ref: library/inspect inspect CO_GENERATOR8661798 +Ref: 34328661798 +Ref: library/inspect inspect CO_NOFREE8661973 +Ref: 34338661973 +Ref: library/inspect inspect CO_COROUTINE8662064 +Ref: 34348662064 +Ref: library/inspect inspect CO_ITERABLE_COROUTINE8662299 +Ref: 34358662299 +Ref: library/inspect inspect CO_ASYNC_GENERATOR8662612 +Ref: 34368662612 +Ref: Code Objects Bit Flags-Footnote-18663257 +Ref: Code Objects Bit Flags-Footnote-28663306 +Ref: Code Objects Bit Flags-Footnote-38663355 +Node: Command Line Interface<3>8663404 +Ref: library/inspect command-line-interface8663529 +Ref: 34378663529 +Ref: library/inspect inspect-module-cli8663529 +Ref: 8688663529 +Ref: library/inspect cmdoption-inspect-details8663904 +Ref: 34388663904 +Node: site — Site-specific configuration hook8664019 +Ref: library/site doc8664161 +Ref: 34398664161 +Ref: library/site module-site8664161 +Ref: eb8664161 +Ref: library/site site-site-specific-configuration-hook8664161 +Ref: 343a8664161 +Ref: site — Site-specific configuration hook-Footnote-18669929 +Node: Readline configuration8669992 +Ref: library/site readline-configuration8670119 +Ref: 343c8670119 +Ref: library/site rlcompleter-config8670119 +Ref: 8e68670119 +Node: Module contents<3>8670777 +Ref: library/site module-contents8670938 +Ref: 343d8670938 +Ref: library/site site PREFIXES8670991 +Ref: 343e8670991 +Ref: library/site site ENABLE_USER_SITE8671072 +Ref: 343b8671072 +Ref: library/site site USER_SITE8671508 +Ref: fe18671508 +Ref: library/site site USER_BASE8672060 +Ref: ff38672060 +Ref: library/site site main8672630 +Ref: fe28672630 +Ref: library/site site addsitedir8672979 +Ref: 343f8672979 +Ref: library/site site getsitepackages8673185 +Ref: bd38673185 +Ref: library/site site getuserbase8673319 +Ref: bd48673319 +Ref: library/site site getusersitepackages8673562 +Ref: bd58673562 +Node: Command Line Interface<4>8673960 +Ref: library/site command-line-interface8674090 +Ref: 34408674090 +Ref: library/site site-commandline8674090 +Ref: 34418674090 +Ref: library/site cmdoption-site-user-base8674636 +Ref: 34428674636 +Ref: library/site cmdoption-site-user-site8674718 +Ref: 34438674718 +Ref: Command Line Interface<4>-Footnote-18675329 +Node: Custom Python Interpreters8675378 +Ref: library/custominterp doc8675526 +Ref: 34448675526 +Ref: library/custominterp custom-python-interpreters8675526 +Ref: 34458675526 +Ref: library/custominterp custominterp8675526 +Ref: 34468675526 +Node: code — Interpreter base classes8676111 +Ref: library/code doc8676246 +Ref: 34478676246 +Ref: library/code code-interpreter-base-classes8676246 +Ref: 34488676246 +Ref: library/code module-code8676246 +Ref: 1b8676246 +Ref: library/code code InteractiveInterpreter8676664 +Ref: 34498676664 +Ref: library/code code InteractiveConsole8677177 +Ref: 14a98677177 +Ref: library/code code interact8677487 +Ref: 344a8677487 +Ref: library/code code compile_command8678246 +Ref: 344c8678246 +Ref: code — Interpreter base classes-Footnote-18679532 +Node: Interactive Interpreter Objects8679595 +Ref: library/code interactive-interpreter-objects8679732 +Ref: 344d8679732 +Ref: library/code interpreter-objects8679732 +Ref: 344e8679732 +Ref: library/code code InteractiveInterpreter runsource8679815 +Ref: 344f8679815 +Ref: library/code code InteractiveInterpreter runcode8681102 +Ref: 34518681102 +Ref: library/code code InteractiveInterpreter showsyntaxerror8681561 +Ref: 34508681561 +Ref: library/code code InteractiveInterpreter showtraceback8682035 +Ref: 6a78682035 +Ref: library/code code InteractiveInterpreter write8682416 +Ref: 34528682416 +Node: Interactive Console Objects8682636 +Ref: library/code console-objects8682773 +Ref: 34538682773 +Ref: library/code interactive-console-objects8682773 +Ref: 34548682773 +Ref: library/code code InteractiveConsole interact8683037 +Ref: 34558683037 +Ref: library/code code InteractiveConsole push8683890 +Ref: 34568683890 +Ref: library/code code InteractiveConsole resetbuffer8684585 +Ref: 34578684585 +Ref: library/code code InteractiveConsole raw_input8684694 +Ref: 344b8684694 +Node: codeop — Compile Python code8685054 +Ref: library/codeop doc8685189 +Ref: 34588685189 +Ref: library/codeop codeop-compile-python-code8685189 +Ref: 34598685189 +Ref: library/codeop module-codeop8685189 +Ref: 1d8685189 +Ref: library/codeop codeop compile_command8686150 +Ref: 345a8686150 +Ref: library/codeop codeop Compile8687552 +Ref: 345b8687552 +Ref: library/codeop codeop CommandCompiler8687953 +Ref: 345c8687953 +Ref: codeop — Compile Python code-Footnote-18688371 +Node: Importing Modules8688436 +Ref: library/modules doc8688585 +Ref: 345d8688585 +Ref: library/modules importing-modules8688585 +Ref: 345e8688585 +Ref: library/modules modules8688585 +Ref: 345f8688585 +Node: zipimport — Import modules from Zip archives8689128 +Ref: library/zipimport doc8689274 +Ref: 34608689274 +Ref: library/zipimport module-zipimport8689274 +Ref: 1438689274 +Ref: library/zipimport zipimport-import-modules-from-zip-archives8689274 +Ref: 34618689274 +Ref: library/zipimport zipimport ZipImportError8691377 +Ref: 34628691377 +Ref: zipimport — Import modules from Zip archives-Footnote-18691669 +Ref: zipimport — Import modules from Zip archives-Footnote-28691737 +Ref: zipimport — Import modules from Zip archives-Footnote-38691805 +Ref: zipimport — Import modules from Zip archives-Footnote-48691854 +Ref: zipimport — Import modules from Zip archives-Footnote-58691903 +Ref: zipimport — Import modules from Zip archives-Footnote-68691952 +Node: zipimporter Objects8692001 +Ref: library/zipimport id18692124 +Ref: 34638692124 +Ref: library/zipimport zipimporter-objects8692124 +Ref: 34648692124 +Ref: library/zipimport zipimport zipimporter8692247 +Ref: 34658692247 +Ref: library/zipimport zipimport zipimporter find_module8692718 +Ref: 34668692718 +Ref: library/zipimport zipimport zipimporter get_code8693142 +Ref: 34678693142 +Ref: library/zipimport zipimport zipimporter get_data8693320 +Ref: 34688693320 +Ref: library/zipimport zipimport zipimporter get_filename8693592 +Ref: 34698693592 +Ref: library/zipimport zipimport zipimporter get_source8693852 +Ref: 346a8693852 +Ref: library/zipimport zipimport zipimporter is_package8694142 +Ref: 346b8694142 +Ref: library/zipimport zipimport zipimporter load_module8694353 +Ref: 346c8694353 +Ref: library/zipimport zipimport zipimporter archive8694631 +Ref: 346d8694631 +Ref: library/zipimport zipimport zipimporter prefix8694764 +Ref: 346e8694764 +Node: Examples<28>8695171 +Ref: library/zipimport examples8695294 +Ref: 346f8695294 +Ref: library/zipimport zipimport-examples8695294 +Ref: 34708695294 +Node: pkgutil — Package extension utility8695974 +Ref: library/pkgutil doc8696175 +Ref: 34718696175 +Ref: library/pkgutil module-pkgutil8696175 +Ref: cd8696175 +Ref: library/pkgutil pkgutil-package-extension-utility8696175 +Ref: 34728696175 +Ref: library/pkgutil pkgutil ModuleInfo8696460 +Ref: 60d8696460 +Ref: library/pkgutil pkgutil extend_path8696614 +Ref: 34738696614 +Ref: library/pkgutil pkgutil ImpImporter8698286 +Ref: 34748698286 +Ref: library/pkgutil pkgutil ImpLoader8698985 +Ref: 34758698985 +Ref: library/pkgutil pkgutil find_loader8699314 +Ref: 34768699314 +Ref: library/pkgutil pkgutil get_importer8699891 +Ref: 34778699891 +Ref: library/pkgutil pkgutil get_loader8700406 +Ref: 34788700406 +Ref: library/pkgutil pkgutil iter_importers8701123 +Ref: b148701123 +Ref: library/pkgutil pkgutil iter_modules8701822 +Ref: 60c8701822 +Ref: library/pkgutil pkgutil walk_packages8702668 +Ref: 48b8702668 +Ref: library/pkgutil pkgutil get_data8704246 +Ref: 34798704246 +Ref: pkgutil — Package extension utility-Footnote-18705416 +Ref: pkgutil — Package extension utility-Footnote-28705482 +Ref: pkgutil — Package extension utility-Footnote-38705531 +Ref: pkgutil — Package extension utility-Footnote-48705580 +Ref: pkgutil — Package extension utility-Footnote-58705629 +Ref: pkgutil — Package extension utility-Footnote-68705678 +Ref: pkgutil — Package extension utility-Footnote-78705727 +Ref: pkgutil — Package extension utility-Footnote-88705776 +Ref: pkgutil — Package extension utility-Footnote-98705825 +Ref: pkgutil — Package extension utility-Footnote-108705874 +Ref: pkgutil — Package extension utility-Footnote-118705924 +Ref: pkgutil — Package extension utility-Footnote-128705974 +Ref: pkgutil — Package extension utility-Footnote-138706024 +Ref: pkgutil — Package extension utility-Footnote-148706074 +Node: modulefinder — Find modules used by a script8706124 +Ref: library/modulefinder doc8706326 +Ref: 347b8706326 +Ref: library/modulefinder module-modulefinder8706326 +Ref: b48706326 +Ref: library/modulefinder modulefinder-find-modules-used-by-a-script8706326 +Ref: 347c8706326 +Ref: library/modulefinder modulefinder AddPackagePath8706845 +Ref: 347e8706845 +Ref: library/modulefinder modulefinder ReplacePackage8706995 +Ref: 347f8706995 +Ref: library/modulefinder modulefinder ModuleFinder8707158 +Ref: 347d8707158 +Ref: library/modulefinder modulefinder ModuleFinder report8707817 +Ref: 34818707817 +Ref: library/modulefinder modulefinder ModuleFinder run_script8708032 +Ref: 34808708032 +Ref: library/modulefinder modulefinder ModuleFinder modules8708170 +Ref: 34828708170 +Ref: modulefinder — Find modules used by a script-Footnote-18708394 +Node: Example usage of ModuleFinder8708465 +Ref: library/modulefinder example-usage-of-modulefinder8708577 +Ref: 34848708577 +Ref: library/modulefinder modulefinder-example8708577 +Ref: 34838708577 +Node: runpy — Locating and executing Python modules8709948 +Ref: library/runpy doc8710155 +Ref: 34858710155 +Ref: library/runpy module-runpy8710155 +Ref: e28710155 +Ref: library/runpy runpy-locating-and-executing-python-modules8710155 +Ref: 34868710155 +Ref: library/runpy runpy run_module8711190 +Ref: fd38711190 +Ref: library/runpy runpy run_path8714452 +Ref: cb18714452 +Ref: runpy — Locating and executing Python modules-Footnote-18718737 +Ref: runpy — Locating and executing Python modules-Footnote-28718801 +Ref: runpy — Locating and executing Python modules-Footnote-38718850 +Ref: runpy — Locating and executing Python modules-Footnote-48718899 +Ref: runpy — Locating and executing Python modules-Footnote-58718948 +Ref: runpy — Locating and executing Python modules-Footnote-68718997 +Ref: runpy — Locating and executing Python modules-Footnote-78719046 +Ref: runpy — Locating and executing Python modules-Footnote-88719095 +Node: importlib — The implementation of import8719144 +Ref: library/importlib doc8719329 +Ref: 34878719329 +Ref: library/importlib importlib-the-implementation-of-import8719329 +Ref: 34888719329 +Ref: library/importlib module-importlib8719329 +Ref: 9b8719329 +Ref: importlib — The implementation of import-Footnote-18720102 +Node: Introduction<12>8720180 +Ref: library/importlib introduction8720297 +Ref: 34898720297 +Ref: Introduction<12>-Footnote-18722132 +Ref: Introduction<12>-Footnote-28722184 +Ref: Introduction<12>-Footnote-38722233 +Ref: Introduction<12>-Footnote-48722282 +Ref: Introduction<12>-Footnote-58722331 +Ref: Introduction<12>-Footnote-68722380 +Ref: Introduction<12>-Footnote-78722429 +Ref: Introduction<12>-Footnote-88722478 +Ref: Introduction<12>-Footnote-98722527 +Ref: Introduction<12>-Footnote-108722576 +Ref: Introduction<12>-Footnote-118722626 +Ref: Introduction<12>-Footnote-128722676 +Ref: Introduction<12>-Footnote-138722726 +Node: Functions<10>8722776 +Ref: library/importlib functions8722959 +Ref: 348a8722959 +Ref: library/importlib importlib __import__8722998 +Ref: 9ae8722998 +Ref: library/importlib importlib import_module8723306 +Ref: b138723306 +Ref: library/importlib importlib find_loader8724634 +Ref: 9378724634 +Ref: library/importlib importlib invalidate_caches8725638 +Ref: 49c8725638 +Ref: library/importlib importlib reload8726087 +Ref: 3998726087 +Node: importlib abc – Abstract base classes related to import8729319 +Ref: library/importlib importlib-abc-abstract-base-classes-related-to-import8729519 +Ref: 348b8729519 +Ref: library/importlib module-importlib abc8729519 +Ref: 9c8729519 +Ref: library/importlib importlib abc Finder8730352 +Ref: 9b58730352 +Ref: library/importlib importlib abc Finder find_module8730561 +Ref: 348c8730561 +Ref: library/importlib importlib abc MetaPathFinder8731011 +Ref: 9b38731011 +Ref: library/importlib importlib abc MetaPathFinder find_spec8731216 +Ref: 4638731216 +Ref: library/importlib importlib abc MetaPathFinder find_module8731969 +Ref: 4628731969 +Ref: library/importlib importlib abc MetaPathFinder invalidate_caches8732792 +Ref: 348e8732792 +Ref: library/importlib importlib abc PathEntryFinder8733201 +Ref: 9b48733201 +Ref: library/importlib importlib abc PathEntryFinder find_spec8733618 +Ref: 4658733618 +Ref: library/importlib importlib abc PathEntryFinder find_loader8734256 +Ref: 4648734256 +Ref: library/importlib importlib abc PathEntryFinder find_module8735459 +Ref: 93b8735459 +Ref: library/importlib importlib abc PathEntryFinder invalidate_caches8735732 +Ref: 348f8735732 +Ref: library/importlib importlib abc Loader8736006 +Ref: 7c28736006 +Ref: library/importlib importlib abc Loader create_module8736430 +Ref: 5548736430 +Ref: library/importlib importlib abc Loader exec_module8736863 +Ref: 5e48736863 +Ref: library/importlib importlib abc Loader load_module8737331 +Ref: 5e38737331 +Ref: library/importlib importlib abc Loader module_repr8740504 +Ref: 9418740504 +Ref: library/importlib importlib abc ResourceReader8741003 +Ref: 3428741003 +Ref: library/importlib importlib abc ResourceReader open_resource8742661 +Ref: 34908742661 +Ref: library/importlib importlib abc ResourceReader resource_path8742922 +Ref: 34918742922 +Ref: library/importlib importlib abc ResourceReader is_resource8743158 +Ref: 34928743158 +Ref: library/importlib importlib abc ResourceReader contents8743372 +Ref: 34938743372 +Ref: library/importlib importlib abc ResourceLoader8744240 +Ref: 4668744240 +Ref: library/importlib importlib abc ResourceLoader get_data8744614 +Ref: 347a8744614 +Ref: library/importlib importlib abc InspectLoader8745301 +Ref: 8608745301 +Ref: library/importlib importlib abc InspectLoader get_code8745485 +Ref: 8618745485 +Ref: library/importlib importlib abc InspectLoader get_source8746068 +Ref: 8658746068 +Ref: library/importlib importlib abc InspectLoader is_package8746650 +Ref: 34948746650 +Ref: library/importlib importlib abc InspectLoader source_to_code8747021 +Ref: 6ef8747021 +Ref: library/importlib importlib abc InspectLoader exec_module8747665 +Ref: 93f8747665 +Ref: library/importlib importlib abc InspectLoader load_module8747797 +Ref: 93c8747797 +Ref: library/importlib importlib abc ExecutionLoader8747990 +Ref: 34958747990 +Ref: library/importlib importlib abc ExecutionLoader get_filename8748237 +Ref: 34968748237 +Ref: library/importlib importlib abc FileLoader8748784 +Ref: 9b78748784 +Ref: library/importlib importlib abc FileLoader name8749278 +Ref: 34978749278 +Ref: library/importlib importlib abc FileLoader path8749361 +Ref: 34988749361 +Ref: library/importlib importlib abc FileLoader load_module8749430 +Ref: 93d8749430 +Ref: library/importlib importlib abc FileLoader get_filename8749616 +Ref: 34998749616 +Ref: library/importlib importlib abc FileLoader get_data8749710 +Ref: 349a8749710 +Ref: library/importlib importlib abc SourceLoader8749831 +Ref: 349b8749831 +Ref: library/importlib importlib abc SourceLoader path_stats8750928 +Ref: aec8750928 +Ref: library/importlib importlib abc SourceLoader path_mtime8751706 +Ref: aeb8751706 +Ref: library/importlib importlib abc SourceLoader set_data8752254 +Ref: 349c8752254 +Ref: library/importlib importlib abc SourceLoader get_code8752769 +Ref: 349d8752769 +Ref: library/importlib importlib abc SourceLoader exec_module8752892 +Ref: 9408752892 +Ref: library/importlib importlib abc SourceLoader load_module8753033 +Ref: 93e8753033 +Ref: library/importlib importlib abc SourceLoader get_source8753235 +Ref: 9598753235 +Ref: library/importlib importlib abc SourceLoader is_package8753362 +Ref: 349e8753362 +Ref: importlib abc – Abstract base classes related to import-Footnote-18753801 +Ref: importlib abc – Abstract base classes related to import-Footnote-28753873 +Ref: importlib abc – Abstract base classes related to import-Footnote-38753922 +Ref: importlib abc – Abstract base classes related to import-Footnote-48753971 +Ref: importlib abc – Abstract base classes related to import-Footnote-58754020 +Ref: importlib abc – Abstract base classes related to import-Footnote-68754069 +Node: importlib resources – Resources8754118 +Ref: library/importlib importlib-resources-resources8754353 +Ref: 349f8754353 +Ref: library/importlib module-importlib resources8754353 +Ref: 9e8754353 +Ref: library/importlib importlib resources Package8755690 +Ref: 34a08755690 +Ref: library/importlib importlib resources Resource8756045 +Ref: 34a18756045 +Ref: library/importlib importlib resources open_binary8756282 +Ref: 34a28756282 +Ref: library/importlib importlib resources open_text8756803 +Ref: 34a38756803 +Ref: library/importlib importlib resources read_binary8757516 +Ref: 34a48757516 +Ref: library/importlib importlib resources read_text8758038 +Ref: 34a58758038 +Ref: library/importlib importlib resources path8758741 +Ref: 34a68758741 +Ref: library/importlib importlib resources is_resource8759458 +Ref: 34a78759458 +Ref: library/importlib importlib resources contents8759779 +Ref: 34a88759779 +Ref: importlib resources – Resources-Footnote-18760210 +Ref: importlib resources – Resources-Footnote-28760289 +Ref: importlib resources – Resources-Footnote-38760360 +Ref: importlib resources – Resources-Footnote-48760453 +Ref: importlib resources – Resources-Footnote-58760524 +Node: importlib machinery – Importers and path hooks8760600 +Ref: library/importlib importlib-machinery-importers-and-path-hooks8760823 +Ref: 34a98760823 +Ref: library/importlib module-importlib machinery8760823 +Ref: 9d8760823 +Ref: library/importlib importlib machinery SOURCE_SUFFIXES8761154 +Ref: 34aa8761154 +Ref: library/importlib importlib machinery DEBUG_BYTECODE_SUFFIXES8761323 +Ref: 34ab8761323 +Ref: library/importlib importlib machinery OPTIMIZED_BYTECODE_SUFFIXES8761590 +Ref: 34ad8761590 +Ref: library/importlib importlib machinery BYTECODE_SUFFIXES8761857 +Ref: 34ac8761857 +Ref: library/importlib importlib machinery EXTENSION_SUFFIXES8762146 +Ref: 34ae8762146 +Ref: library/importlib importlib machinery all_suffixes8762321 +Ref: 33f38762321 +Ref: library/importlib importlib machinery BuiltinImporter8762755 +Ref: 5558762755 +Ref: library/importlib importlib machinery FrozenImporter8763320 +Ref: 9578763320 +Ref: library/importlib importlib machinery WindowsRegistryFinder8763740 +Ref: 5e58763740 +Ref: library/importlib importlib machinery PathFinder8764226 +Ref: 95b8764226 +Ref: library/importlib importlib machinery PathFinder find_spec8764541 +Ref: 93a8764541 +Ref: library/importlib importlib machinery PathFinder find_module8765718 +Ref: 9398765718 +Ref: library/importlib importlib machinery PathFinder invalidate_caches8765929 +Ref: 49b8765929 +Ref: library/importlib importlib machinery FileFinder8766541 +Ref: 9b68766541 +Ref: library/importlib importlib machinery FileFinder path8767846 +Ref: 34af8767846 +Ref: library/importlib importlib machinery FileFinder find_spec8767919 +Ref: 7c18767919 +Ref: library/importlib importlib machinery FileFinder find_loader8768094 +Ref: 34b08768094 +Ref: library/importlib importlib machinery FileFinder invalidate_caches8768229 +Ref: 34b18768229 +Ref: library/importlib importlib machinery FileFinder path_hook8768309 +Ref: 34b28768309 +Ref: library/importlib importlib machinery SourceFileLoader8768755 +Ref: 9b88768755 +Ref: library/importlib importlib machinery SourceFileLoader name8769046 +Ref: 34b38769046 +Ref: library/importlib importlib machinery SourceFileLoader path8769136 +Ref: 34b48769136 +Ref: library/importlib importlib machinery SourceFileLoader is_package8769202 +Ref: 34b58769202 +Ref: library/importlib importlib machinery SourceFileLoader path_stats8769331 +Ref: 34b68769331 +Ref: library/importlib importlib machinery SourceFileLoader set_data8769467 +Ref: 34b78769467 +Ref: library/importlib importlib machinery SourceFileLoader load_module8769606 +Ref: 5e18769606 +Ref: library/importlib importlib machinery SourcelessFileLoader8769925 +Ref: 9b98769925 +Ref: library/importlib importlib machinery SourcelessFileLoader name8770396 +Ref: 34b88770396 +Ref: library/importlib importlib machinery SourcelessFileLoader path8770480 +Ref: 34b98770480 +Ref: library/importlib importlib machinery SourcelessFileLoader is_package8770548 +Ref: 34ba8770548 +Ref: library/importlib importlib machinery SourcelessFileLoader get_code8770674 +Ref: 34bb8770674 +Ref: library/importlib importlib machinery SourcelessFileLoader get_source8770810 +Ref: 34bc8770810 +Ref: library/importlib importlib machinery SourcelessFileLoader load_module8770949 +Ref: 5e28770949 +Ref: library/importlib importlib machinery ExtensionFileLoader8771243 +Ref: 5568771243 +Ref: library/importlib importlib machinery ExtensionFileLoader name8771610 +Ref: 34bd8771610 +Ref: library/importlib importlib machinery ExtensionFileLoader path8771687 +Ref: 34be8771687 +Ref: library/importlib importlib machinery ExtensionFileLoader create_module8771754 +Ref: 34bf8771754 +Ref: library/importlib importlib machinery ExtensionFileLoader exec_module8771931 +Ref: 34c08771931 +Ref: library/importlib importlib machinery ExtensionFileLoader is_package8772089 +Ref: 34c18772089 +Ref: library/importlib importlib machinery ExtensionFileLoader get_code8772273 +Ref: 34c28772273 +Ref: library/importlib importlib machinery ExtensionFileLoader get_source8772382 +Ref: 34c38772382 +Ref: library/importlib importlib machinery ExtensionFileLoader get_filename8772498 +Ref: 8668772498 +Ref: library/importlib importlib machinery ModuleSpec8772608 +Ref: 116b8772608 +Ref: library/importlib importlib machinery ModuleSpec name8773421 +Ref: 16648773421 +Ref: library/importlib importlib machinery ModuleSpec loader8773529 +Ref: 16628773529 +Ref: library/importlib importlib machinery ModuleSpec origin8773708 +Ref: 34c48773708 +Ref: library/importlib importlib machinery ModuleSpec submodule_search_locations8774069 +Ref: 34c58774069 +Ref: library/importlib importlib machinery ModuleSpec loader_state8774234 +Ref: 34c68774234 +Ref: library/importlib importlib machinery ModuleSpec cached8774359 +Ref: 34c78774359 +Ref: library/importlib importlib machinery ModuleSpec parent8774489 +Ref: 16668774489 +Ref: library/importlib importlib machinery ModuleSpec has_location8774770 +Ref: 34c88774770 +Ref: importlib machinery – Importers and path hooks-Footnote-18774954 +Ref: importlib machinery – Importers and path hooks-Footnote-28775033 +Ref: importlib machinery – Importers and path hooks-Footnote-38775082 +Ref: importlib machinery – Importers and path hooks-Footnote-48775131 +Node: importlib util – Utility code for importers8775180 +Ref: library/importlib importlib-util-utility-code-for-importers8775382 +Ref: 34c98775382 +Ref: library/importlib module-importlib util8775382 +Ref: 9f8775382 +Ref: library/importlib importlib util MAGIC_NUMBER8775708 +Ref: 8628775708 +Ref: library/importlib importlib util cache_from_source8775951 +Ref: 5578775951 +Ref: library/importlib importlib util source_from_cache8777860 +Ref: 5588777860 +Ref: library/importlib importlib util decode_source8778485 +Ref: 8648778485 +Ref: library/importlib importlib util resolve_name8778755 +Ref: 34ca8778755 +Ref: library/importlib importlib util find_spec8779459 +Ref: 9388779459 +Ref: library/importlib importlib util module_from_spec8780351 +Ref: 6f08780351 +Ref: library/importlib importlib util module_for_loader8780984 +Ref: 9428780984 +Ref: library/importlib importlib util set_loader8782761 +Ref: 9438782761 +Ref: library/importlib importlib util set_package8783387 +Ref: 9448783387 +Ref: library/importlib importlib util spec_from_loader8783793 +Ref: 348d8783793 +Ref: library/importlib importlib util spec_from_file_location8784239 +Ref: 5598784239 +Ref: library/importlib importlib util source_hash8784713 +Ref: 34cb8784713 +Ref: library/importlib importlib util LazyLoader8784985 +Ref: 5538784985 +Ref: library/importlib importlib util LazyLoader factory8786468 +Ref: 34cd8786468 +Ref: importlib util – Utility code for importers-Footnote-18787032 +Ref: importlib util – Utility code for importers-Footnote-28787105 +Ref: importlib util – Utility code for importers-Footnote-38787154 +Ref: importlib util – Utility code for importers-Footnote-48787203 +Ref: importlib util – Utility code for importers-Footnote-58787252 +Ref: importlib util – Utility code for importers-Footnote-68787301 +Node: Examples<29>8787350 +Ref: library/importlib examples8787495 +Ref: 34ce8787495 +Ref: library/importlib importlib-examples8787495 +Ref: 34cf8787495 +Node: Importing programmatically8787760 +Ref: library/importlib importing-programmatically8787880 +Ref: 34d08787880 +Node: Checking if a module can be imported8788111 +Ref: library/importlib checking-if-a-module-can-be-imported8788272 +Ref: 34d18788272 +Node: Importing a source file directly8789057 +Ref: library/importlib importing-a-source-file-directly8789214 +Ref: 34d28789214 +Node: Setting up an importer8789773 +Ref: library/importlib setting-up-an-importer8789931 +Ref: 34d38789931 +Node: Approximating importlib import_module8791505 +Ref: library/importlib approximating-importlib-import-module8791622 +Ref: 34d48791622 +Node: Using importlib metadata8793219 +Ref: library/importlib metadata doc8793348 +Ref: 34d58793348 +Ref: library/importlib metadata using8793348 +Ref: 34d68793348 +Ref: library/importlib metadata using-importlib-metadata8793348 +Ref: 34d78793348 +Ref: Using importlib metadata-Footnote-18794691 +Ref: Using importlib metadata-Footnote-28794776 +Ref: Using importlib metadata-Footnote-38794861 +Ref: Using importlib metadata-Footnote-48794933 +Ref: Using importlib metadata-Footnote-58794971 +Node: Overview<2>8795020 +Ref: library/importlib metadata overview8795118 +Ref: 34d88795118 +Node: Functional API<2>8796540 +Ref: library/importlib metadata functional-api8796660 +Ref: 34de8796660 +Node: Entry points8796912 +Ref: library/importlib metadata entry-points8797008 +Ref: 34d98797008 +Ref: library/importlib metadata id18797008 +Ref: 34df8797008 +Ref: Entry points-Footnote-18798088 +Node: Distribution metadata8798199 +Ref: library/importlib metadata distribution-metadata8798325 +Ref: 34e08798325 +Ref: library/importlib metadata metadata8798325 +Ref: 34da8798325 +Ref: Distribution metadata-Footnote-18798809 +Node: Distribution versions8799089 +Ref: library/importlib metadata distribution-versions8799221 +Ref: 34e18799221 +Ref: library/importlib metadata version8799221 +Ref: 34db8799221 +Node: Distribution files8799430 +Ref: library/importlib metadata distribution-files8799566 +Ref: 34e28799566 +Ref: library/importlib metadata files8799566 +Ref: 34dc8799566 +Ref: Distribution files-Footnote-18800956 +Node: Distribution requirements8801053 +Ref: library/importlib metadata distribution-requirements8801159 +Ref: 34e38801159 +Ref: library/importlib metadata requirements8801159 +Ref: 34dd8801159 +Node: Distributions8801426 +Ref: library/importlib metadata distributions8801565 +Ref: 34e48801565 +Ref: Distributions-Footnote-18802472 +Node: Extending the search algorithm8802521 +Ref: library/importlib metadata extending-the-search-algorithm8802634 +Ref: 34e58802634 +Node: Python Language Services8804401 +Ref: library/language doc8804546 +Ref: 34e68804546 +Ref: library/language language8804546 +Ref: 34e78804546 +Ref: library/language python-language-services8804546 +Ref: 34e88804546 +Node: parser — Access Python parse trees8805453 +Ref: library/parser doc8805588 +Ref: 34e98805588 +Ref: library/parser module-parser8805588 +Ref: c78805588 +Ref: library/parser parser-access-python-parse-trees8805588 +Ref: 34ea8805588 +Node: Creating ST Objects8810798 +Ref: library/parser creating-st-objects8810920 +Ref: 34f08810920 +Ref: library/parser creating-sts8810920 +Ref: 34f18810920 +Ref: library/parser parser expr8811166 +Ref: 34eb8811166 +Ref: library/parser parser suite8811501 +Ref: 34ec8811501 +Ref: library/parser parser sequence2st8811838 +Ref: 34ed8811838 +Ref: library/parser parser tuple2st8813253 +Ref: 34f48813253 +Node: Converting ST Objects8813422 +Ref: library/parser converting-st-objects8813574 +Ref: 34f58813574 +Ref: library/parser converting-sts8813574 +Ref: 34f68813574 +Ref: library/parser parser st2list8813888 +Ref: 34ee8813888 +Ref: library/parser parser st2tuple8814962 +Ref: 34ef8814962 +Ref: library/parser parser compilest8815506 +Ref: 34f38815506 +Node: Queries on ST Objects8816669 +Ref: library/parser queries-on-st-objects8816831 +Ref: 34f78816831 +Ref: library/parser querying-sts8816831 +Ref: 34f88816831 +Ref: library/parser parser isexpr8817203 +Ref: 34f98817203 +Ref: library/parser parser issuite8817686 +Ref: 34fa8817686 +Node: Exceptions and Error Handling8818046 +Ref: library/parser exceptions-and-error-handling8818197 +Ref: 34fb8818197 +Ref: library/parser st-errors8818197 +Ref: 34fc8818197 +Ref: library/parser parser ParserError8818492 +Ref: 34f28818492 +Node: ST Objects8819654 +Ref: library/parser id18819812 +Ref: 34fd8819812 +Ref: library/parser st-objects8819812 +Ref: 34fe8819812 +Ref: library/parser parser STType8820000 +Ref: 34ff8820000 +Ref: library/parser parser ST compile8820184 +Ref: 35008820184 +Ref: library/parser parser ST isexpr8820280 +Ref: 35018820280 +Ref: library/parser parser ST issuite8820338 +Ref: 35028820338 +Ref: library/parser parser ST tolist8820398 +Ref: 35038820398 +Ref: library/parser parser ST totuple8820509 +Ref: 35048820509 +Node: Example Emulation of compile8820622 +Ref: library/parser example-emulation-of-compile8820742 +Ref: 35058820742 +Node: ast — Abstract Syntax Trees8821836 +Ref: library/ast doc8822033 +Ref: 35068822033 +Ref: library/ast ast-abstract-syntax-trees8822033 +Ref: 35078822033 +Ref: library/ast module-ast8822033 +Ref: 88822033 +Ref: ast — Abstract Syntax Trees-Footnote-18822978 +Node: Node classes8823040 +Ref: library/ast node-classes8823143 +Ref: 35098823143 +Ref: library/ast ast AST8823188 +Ref: 35088823188 +Ref: library/ast ast AST _fields8824011 +Ref: 350b8824011 +Ref: library/ast ast AST lineno8824813 +Ref: 350c8824813 +Ref: library/ast ast AST col_offset8824840 +Ref: 350d8824840 +Ref: library/ast ast AST end_lineno8824871 +Ref: 350e8824871 +Ref: library/ast ast AST end_col_offset8824902 +Ref: 350f8824902 +Node: Abstract Grammar8827173 +Ref: library/ast abstract-grammar8827296 +Ref: 350a8827296 +Ref: library/ast id18827296 +Ref: 35108827296 +Node: ast Helpers8833351 +Ref: library/ast ast-helpers8833453 +Ref: 35118833453 +Ref: library/ast ast parse8833641 +Ref: 1a18833641 +Ref: library/ast ast literal_eval8835442 +Ref: 4a58835442 +Ref: library/ast ast get_docstring8836303 +Ref: 35128836303 +Ref: library/ast ast get_source_segment8836722 +Ref: 1a08836722 +Ref: library/ast ast fix_missing_locations8837172 +Ref: 35138837172 +Ref: library/ast ast increment_lineno8837628 +Ref: 35148837628 +Ref: library/ast ast copy_location8837860 +Ref: 35158837860 +Ref: library/ast ast iter_fields8838094 +Ref: 35168838094 +Ref: library/ast ast iter_child_nodes8838249 +Ref: 35178838249 +Ref: library/ast ast walk8838427 +Ref: 35188838427 +Ref: library/ast ast NodeVisitor8838693 +Ref: 2818838693 +Ref: library/ast ast NodeVisitor visit8839035 +Ref: 35198839035 +Ref: library/ast ast NodeVisitor generic_visit8839315 +Ref: 351a8839315 +Ref: library/ast ast NodeTransformer8840150 +Ref: 351b8840150 +Ref: library/ast ast dump8842021 +Ref: 351c8842021 +Ref: ast Helpers-Footnote-18843489 +Ref: ast Helpers-Footnote-28843538 +Ref: ast Helpers-Footnote-38843587 +Ref: ast Helpers-Footnote-48843636 +Ref: ast Helpers-Footnote-58843684 +Ref: ast Helpers-Footnote-68843751 +Ref: ast Helpers-Footnote-78843806 +Ref: ast Helpers-Footnote-88843845 +Node: symtable — Access to the compiler’s symbol tables8843882 +Ref: library/symtable doc8844092 +Ref: 351d8844092 +Ref: library/symtable module-symtable8844092 +Ref: fc8844092 +Ref: library/symtable symtable-access-to-the-compiler-s-symbol-tables8844092 +Ref: 351e8844092 +Ref: symtable — Access to the compiler’s symbol tables-Footnote-18844674 +Node: Generating Symbol Tables8844741 +Ref: library/symtable generating-symbol-tables8844887 +Ref: 351f8844887 +Ref: library/symtable symtable symtable8844956 +Ref: 35208844956 +Node: Examining Symbol Tables8845236 +Ref: library/symtable examining-symbol-tables8845382 +Ref: 35228845382 +Ref: library/symtable symtable SymbolTable8845449 +Ref: 35218845449 +Ref: library/symtable symtable SymbolTable get_type8845551 +Ref: 35238845551 +Ref: library/symtable symtable SymbolTable get_id8845713 +Ref: 35248845713 +Ref: library/symtable symtable SymbolTable get_name8845785 +Ref: 35258845785 +Ref: library/symtable symtable SymbolTable get_lineno8846088 +Ref: 35268846088 +Ref: library/symtable symtable SymbolTable is_optimized8846213 +Ref: 35278846213 +Ref: library/symtable symtable SymbolTable is_nested8846322 +Ref: 35288846322 +Ref: library/symtable symtable SymbolTable has_children8846426 +Ref: 35298846426 +Ref: library/symtable symtable SymbolTable has_exec8846600 +Ref: 352b8846600 +Ref: library/symtable symtable SymbolTable get_identifiers8846689 +Ref: 352c8846689 +Ref: library/symtable symtable SymbolTable lookup8846786 +Ref: 352d8846786 +Ref: library/symtable symtable SymbolTable get_symbols8846909 +Ref: 352f8846909 +Ref: library/symtable symtable SymbolTable get_children8847034 +Ref: 352a8847034 +Ref: library/symtable symtable Function8847122 +Ref: 35308847122 +Ref: library/symtable symtable Function get_parameters8847247 +Ref: 35318847247 +Ref: library/symtable symtable Function get_locals8847368 +Ref: 35328847368 +Ref: library/symtable symtable Function get_globals8847471 +Ref: 35338847471 +Ref: library/symtable symtable Function get_nonlocals8847576 +Ref: 35348847576 +Ref: library/symtable symtable Function get_frees8847685 +Ref: 35358847685 +Ref: library/symtable symtable Class8847805 +Ref: 35368847805 +Ref: library/symtable symtable Class get_methods8847913 +Ref: 35378847913 +Ref: library/symtable symtable Symbol8848037 +Ref: 352e8848037 +Ref: library/symtable symtable Symbol get_name8848193 +Ref: 35388848193 +Ref: library/symtable symtable Symbol is_referenced8848262 +Ref: 35398848262 +Ref: library/symtable symtable Symbol is_imported8848362 +Ref: 353a8848362 +Ref: library/symtable symtable Symbol is_parameter8848485 +Ref: 353b8848485 +Ref: library/symtable symtable Symbol is_global8848578 +Ref: 353c8848578 +Ref: library/symtable symtable Symbol is_nonlocal8848663 +Ref: 353d8848663 +Ref: library/symtable symtable Symbol is_declared_global8848752 +Ref: cb68848752 +Ref: library/symtable symtable Symbol is_local8848889 +Ref: 353e8848889 +Ref: library/symtable symtable Symbol is_annotated8848985 +Ref: 353f8848985 +Ref: library/symtable symtable Symbol is_free8849107 +Ref: 35408849107 +Ref: library/symtable symtable Symbol is_assigned8849238 +Ref: 35418849238 +Ref: library/symtable symtable Symbol is_namespace8849343 +Ref: 35428849343 +Ref: library/symtable symtable Symbol get_namespaces8849985 +Ref: 35438849985 +Ref: library/symtable symtable Symbol get_namespace8850080 +Ref: 35448850080 +Node: symbol — Constants used with Python parse trees8850249 +Ref: library/symbol doc8850478 +Ref: 35458850478 +Ref: library/symbol module-symbol8850478 +Ref: fb8850478 +Ref: library/symbol symbol-constants-used-with-python-parse-trees8850478 +Ref: 35468850478 +Ref: library/symbol symbol sym_name8851147 +Ref: 35478851147 +Ref: symbol — Constants used with Python parse trees-Footnote-18851402 +Node: token — Constants used with Python parse trees8851467 +Ref: library/token doc8851682 +Ref: 35488851682 +Ref: library/token module-token8851682 +Ref: 1108851682 +Ref: library/token token-constants-used-with-python-parse-trees8851682 +Ref: 35498851682 +Ref: library/token token tok_name8852396 +Ref: 354a8852396 +Ref: library/token token ISTERMINAL8852614 +Ref: 354b8852614 +Ref: library/token token ISNONTERMINAL8852701 +Ref: 354c8852701 +Ref: library/token token ISEOF8852795 +Ref: 354d8852795 +Ref: library/token token ENDMARKER8852926 +Ref: 354e8852926 +Ref: library/token token NAME8852953 +Ref: 354f8852953 +Ref: library/token token NUMBER8852975 +Ref: 35508852975 +Ref: library/token token STRING8852999 +Ref: 35518852999 +Ref: library/token token NEWLINE8853023 +Ref: 35528853023 +Ref: library/token token INDENT8853048 +Ref: 35538853048 +Ref: library/token token DEDENT8853072 +Ref: 35548853072 +Ref: library/token token LPAR8853096 +Ref: 35558853096 +Ref: library/token token RPAR8853151 +Ref: 35568853151 +Ref: library/token token LSQB8853206 +Ref: 35578853206 +Ref: library/token token RSQB8853261 +Ref: 35588853261 +Ref: library/token token COLON8853316 +Ref: 35598853316 +Ref: library/token token COMMA8853372 +Ref: 355a8853372 +Ref: library/token token SEMI8853428 +Ref: 355b8853428 +Ref: library/token token PLUS8853483 +Ref: 355c8853483 +Ref: library/token token MINUS8853538 +Ref: 355d8853538 +Ref: library/token token STAR8853594 +Ref: 355e8853594 +Ref: library/token token SLASH8853649 +Ref: 355f8853649 +Ref: library/token token VBAR8853705 +Ref: 35608853705 +Ref: library/token token AMPER8853760 +Ref: 35618853760 +Ref: library/token token LESS8853816 +Ref: 35628853816 +Ref: library/token token GREATER8853871 +Ref: 35638853871 +Ref: library/token token EQUAL8853929 +Ref: 35648853929 +Ref: library/token token DOT8853985 +Ref: 35658853985 +Ref: library/token token PERCENT8854039 +Ref: 35668854039 +Ref: library/token token LBRACE8854097 +Ref: 35678854097 +Ref: library/token token RBRACE8854154 +Ref: 35688854154 +Ref: library/token token EQEQUAL8854211 +Ref: 35698854211 +Ref: library/token token NOTEQUAL8854270 +Ref: 356a8854270 +Ref: library/token token LESSEQUAL8854330 +Ref: 356b8854330 +Ref: library/token token GREATEREQUAL8854391 +Ref: 356c8854391 +Ref: library/token token TILDE8854455 +Ref: 356d8854455 +Ref: library/token token CIRCUMFLEX8854511 +Ref: 356e8854511 +Ref: library/token token LEFTSHIFT8854572 +Ref: 356f8854572 +Ref: library/token token RIGHTSHIFT8854633 +Ref: 35708854633 +Ref: library/token token DOUBLESTAR8854695 +Ref: 35718854695 +Ref: library/token token PLUSEQUAL8854757 +Ref: 35728854757 +Ref: library/token token MINEQUAL8854818 +Ref: 35738854818 +Ref: library/token token STAREQUAL8854878 +Ref: 35748854878 +Ref: library/token token SLASHEQUAL8854939 +Ref: 35758854939 +Ref: library/token token PERCENTEQUAL8855001 +Ref: 35768855001 +Ref: library/token token AMPEREQUAL8855065 +Ref: 35778855065 +Ref: library/token token VBAREQUAL8855127 +Ref: 35788855127 +Ref: library/token token CIRCUMFLEXEQUAL8855188 +Ref: 35798855188 +Ref: library/token token LEFTSHIFTEQUAL8855255 +Ref: 357a8855255 +Ref: library/token token RIGHTSHIFTEQUAL8855322 +Ref: 357b8855322 +Ref: library/token token DOUBLESTAREQUAL8855390 +Ref: 357c8855390 +Ref: library/token token DOUBLESLASH8855458 +Ref: 357d8855458 +Ref: library/token token DOUBLESLASHEQUAL8855521 +Ref: 357e8855521 +Ref: library/token token AT8855590 +Ref: 357f8855590 +Ref: library/token token ATEQUAL8855643 +Ref: 35808855643 +Ref: library/token token RARROW8855702 +Ref: 35818855702 +Ref: library/token token ELLIPSIS8855760 +Ref: 35828855760 +Ref: library/token token COLONEQUAL8855821 +Ref: 35838855821 +Ref: library/token token OP8855883 +Ref: 35848855883 +Ref: library/token token AWAIT8855903 +Ref: 35858855903 +Ref: library/token token ASYNC8855926 +Ref: 35868855926 +Ref: library/token token TYPE_IGNORE8855949 +Ref: 35878855949 +Ref: library/token token TYPE_COMMENT8855978 +Ref: 35888855978 +Ref: library/token token ERRORTOKEN8856008 +Ref: 35898856008 +Ref: library/token token N_TOKENS8856036 +Ref: 358a8856036 +Ref: library/token token NT_OFFSET8856062 +Ref: 358b8856062 +Ref: library/token token COMMENT8856207 +Ref: 358c8856207 +Ref: library/token token NL8856278 +Ref: 358d8856278 +Ref: library/token token ENCODING8856560 +Ref: 358e8856560 +Ref: token — Constants used with Python parse trees-Footnote-18857672 +Node: keyword — Testing for Python keywords8857736 +Ref: library/keyword doc8857942 +Ref: 358f8857942 +Ref: library/keyword keyword-testing-for-python-keywords8857942 +Ref: 35908857942 +Ref: library/keyword module-keyword8857942 +Ref: a68857942 +Ref: library/keyword keyword iskeyword8858233 +Ref: 12f78858233 +Ref: library/keyword keyword kwlist8858334 +Ref: 35918858334 +Ref: keyword — Testing for Python keywords-Footnote-18858638 +Node: tokenize — Tokenizer for Python source8858704 +Ref: library/tokenize doc8858909 +Ref: 35928858909 +Ref: library/tokenize module-tokenize8858909 +Ref: 1118858909 +Ref: library/tokenize tokenize-tokenizer-for-python-source8858909 +Ref: 35938858909 +Ref: tokenize — Tokenizer for Python source-Footnote-18859860 +Node: Tokenizing Input8859927 +Ref: library/tokenize tokenizing-input8860050 +Ref: 35948860050 +Ref: library/tokenize tokenize tokenize8860156 +Ref: c5b8860156 +Ref: library/tokenize tokenize generate_tokens8861590 +Ref: 35958861590 +Ref: library/tokenize tokenize untokenize8862357 +Ref: 35968862357 +Ref: library/tokenize tokenize detect_encoding8863301 +Ref: 19408863301 +Ref: library/tokenize tokenize open8864297 +Ref: 193f8864297 +Ref: library/tokenize tokenize TokenError8864466 +Ref: 35978864466 +Ref: Tokenizing Input-Footnote-18864955 +Ref: Tokenizing Input-Footnote-28865004 +Node: Command-Line Usage<2>8865053 +Ref: library/tokenize command-line-usage8865197 +Ref: 35988865197 +Ref: library/tokenize tokenize-cli8865197 +Ref: 35998865197 +Ref: library/tokenize cmdoption-tokenize-h8865461 +Ref: 359a8865461 +Ref: library/tokenize cmdoption-tokenize-e8865531 +Ref: 359b8865531 +Node: Examples<30>8865731 +Ref: library/tokenize examples8865850 +Ref: 359c8865850 +Node: tabnanny — Detection of ambiguous indentation8870344 +Ref: library/tabnanny doc8870550 +Ref: 359d8870550 +Ref: library/tabnanny module-tabnanny8870550 +Ref: 1008870550 +Ref: library/tabnanny tabnanny-detection-of-ambiguous-indentation8870550 +Ref: 359e8870550 +Ref: library/tabnanny tabnanny check8871081 +Ref: 359f8871081 +Ref: library/tabnanny tabnanny verbose8871521 +Ref: 35a08871521 +Ref: library/tabnanny tabnanny filename_only8871678 +Ref: 35a18871678 +Ref: library/tabnanny tabnanny NannyNag8871896 +Ref: 35a28871896 +Ref: library/tabnanny tabnanny process_tokens8872061 +Ref: 35a38872061 +Ref: tabnanny — Detection of ambiguous indentation-Footnote-18872360 +Node: pyclbr — Python module browser support8872427 +Ref: library/pyclbr doc8872635 +Ref: 35a48872635 +Ref: library/pyclbr module-pyclbr8872635 +Ref: d88872635 +Ref: library/pyclbr pyclbr-python-module-browser-support8872635 +Ref: 35a58872635 +Ref: library/pyclbr pyclbr readmodule8873333 +Ref: 35a68873333 +Ref: library/pyclbr pyclbr readmodule_ex8873944 +Ref: 35a78873944 +Ref: pyclbr — Python module browser support-Footnote-18875020 +Node: Function Objects8875085 +Ref: library/pyclbr function-objects8875203 +Ref: 35a88875203 +Ref: library/pyclbr pyclbr-function-objects8875203 +Ref: 35a98875203 +Ref: library/pyclbr pyclbr Function file8875370 +Ref: 35aa8875370 +Ref: library/pyclbr pyclbr Function module8875457 +Ref: 35ab8875457 +Ref: library/pyclbr pyclbr Function name8875551 +Ref: 35ac8875551 +Ref: library/pyclbr pyclbr Function lineno8875613 +Ref: 35ad8875613 +Ref: library/pyclbr pyclbr Function parent8875708 +Ref: 35ae8875708 +Ref: library/pyclbr pyclbr Function children8875838 +Ref: 35af8875838 +Node: Class Objects<2>8875985 +Ref: library/pyclbr class-objects8876103 +Ref: 35b08876103 +Ref: library/pyclbr pyclbr-class-objects8876103 +Ref: 35b18876103 +Ref: library/pyclbr pyclbr Class file8876282 +Ref: 35b28876282 +Ref: library/pyclbr pyclbr Class module8876363 +Ref: 35b38876363 +Ref: library/pyclbr pyclbr Class name8876451 +Ref: 35b48876451 +Ref: library/pyclbr pyclbr Class lineno8876507 +Ref: 35b58876507 +Ref: library/pyclbr pyclbr Class parent8876599 +Ref: 35b68876599 +Ref: library/pyclbr pyclbr Class children8876722 +Ref: 35b78876722 +Ref: library/pyclbr pyclbr Class super8876866 +Ref: 35b88876866 +Ref: library/pyclbr pyclbr Class methods8877204 +Ref: 35b98877204 +Node: py_compile — Compile Python source files8877394 +Ref: library/py_compile doc8877599 +Ref: 35ba8877599 +Ref: library/py_compile module-py_compile8877599 +Ref: d78877599 +Ref: library/py_compile py-compile-compile-python-source-files8877599 +Ref: 35bb8877599 +Ref: library/py_compile py_compile PyCompileError8878224 +Ref: 35bc8878224 +Ref: library/py_compile py_compile compile8878353 +Ref: 2158878353 +Ref: library/py_compile py_compile PycInvalidationMode8882043 +Ref: 35bd8882043 +Ref: library/py_compile py_compile PycInvalidationMode TIMESTAMP8882464 +Ref: 35bf8882464 +Ref: library/py_compile py_compile PycInvalidationMode CHECKED_HASH8882746 +Ref: 35be8882746 +Ref: library/py_compile py_compile PycInvalidationMode UNCHECKED_HASH8882992 +Ref: 35c08882992 +Ref: library/py_compile py_compile main8883441 +Ref: 35c18883441 +Ref: py_compile — Compile Python source files-Footnote-18884315 +Ref: py_compile — Compile Python source files-Footnote-28884384 +Ref: py_compile — Compile Python source files-Footnote-38884433 +Ref: py_compile — Compile Python source files-Footnote-48884482 +Ref: py_compile — Compile Python source files-Footnote-58884531 +Node: compileall — Byte-compile Python libraries8884580 +Ref: library/compileall doc8884785 +Ref: 35c28884785 +Ref: library/compileall compileall-byte-compile-python-libraries8884785 +Ref: 35c38884785 +Ref: library/compileall module-compileall8884785 +Ref: 218884785 +Ref: compileall — Byte-compile Python libraries-Footnote-18885445 +Node: Command-line use8885514 +Ref: library/compileall command-line-use8885636 +Ref: 35c48885636 +Ref: library/compileall cmdoption-compileall-arg-directory8885787 +Ref: 35c58885787 +Ref: library/compileall cmdoption-compileall-arg-file8885821 +Ref: 35c68885821 +Ref: library/compileall cmdoption-compileall-l8886077 +Ref: 35c78886077 +Ref: library/compileall cmdoption-compileall-f8886235 +Ref: 35c88886235 +Ref: library/compileall cmdoption-compileall-q8886314 +Ref: 35c98886314 +Ref: library/compileall cmdoption-compileall-d8886506 +Ref: 35ca8886506 +Ref: library/compileall cmdoption-compileall-x8886867 +Ref: 35cb8886867 +Ref: library/compileall cmdoption-compileall-i8887048 +Ref: 35cc8887048 +Ref: library/compileall cmdoption-compileall-b8887258 +Ref: 35cd8887258 +Ref: library/compileall cmdoption-compileall-r8887592 +Ref: 35ce8887592 +Ref: library/compileall cmdoption-compileall-j8887870 +Ref: 35cf8887870 +Ref: library/compileall cmdoption-compileall-invalidation-mode8888057 +Ref: 35d08888057 +Ref: Command-line use-Footnote-18889747 +Node: Public functions8889796 +Ref: library/compileall public-functions8889918 +Ref: 35d18889918 +Ref: library/compileall compileall compile_dir8889973 +Ref: 3728889973 +Ref: library/compileall compileall compile_file8893127 +Ref: 6bb8893127 +Ref: library/compileall compileall compile_path8895277 +Ref: 6bc8895277 +Ref: Public functions-Footnote-18897005 +Ref: Public functions-Footnote-28897054 +Node: dis — Disassembler for Python bytecode8897103 +Ref: library/dis doc8897309 +Ref: 35d28897309 +Ref: library/dis dis-disassembler-for-python-bytecode8897309 +Ref: 35d38897309 +Ref: library/dis module-dis8897309 +Ref: 388897309 +Ref: dis — Disassembler for Python bytecode-Footnote-18898749 +Node: Bytecode analysis8898811 +Ref: library/dis bytecode-analysis8898932 +Ref: 35d48898932 +Ref: library/dis dis Bytecode8899172 +Ref: 8378899172 +Ref: library/dis dis Bytecode from_traceback8900199 +Ref: 8398900199 +Ref: library/dis dis Bytecode codeobj8900423 +Ref: 35d58900423 +Ref: library/dis dis Bytecode first_line8900484 +Ref: 35d68900484 +Ref: library/dis dis Bytecode dis8900578 +Ref: 8388900578 +Ref: library/dis dis Bytecode info8900769 +Ref: 35d78900769 +Node: Analysis functions8901227 +Ref: library/dis analysis-functions8901385 +Ref: 35d88901385 +Ref: library/dis dis code_info8901691 +Ref: bce8901691 +Ref: library/dis dis show_code8902230 +Ref: 8328902230 +Ref: library/dis dis dis8902694 +Ref: 3848902694 +Ref: library/dis dis distb8904130 +Ref: 8338904130 +Ref: library/dis dis disassemble8904526 +Ref: 8348904526 +Ref: library/dis dis disco8904587 +Ref: 35d98904587 +Ref: library/dis dis get_instructions8905474 +Ref: 8368905474 +Ref: library/dis dis findlinestarts8906086 +Ref: 35da8906086 +Ref: library/dis dis findlabels8906574 +Ref: 35db8906574 +Ref: library/dis dis stack_effect8906744 +Ref: 83a8906744 +Ref: Analysis functions-Footnote-18907312 +Node: Python Bytecode Instructions8907389 +Ref: library/dis bytecodes8907548 +Ref: 35dc8907548 +Ref: library/dis python-bytecode-instructions8907548 +Ref: 35dd8907548 +Ref: library/dis dis Instruction8907782 +Ref: 8358907782 +Ref: library/dis dis Instruction opcode8907849 +Ref: 35de8907849 +Ref: library/dis dis Instruction opname8908042 +Ref: 35e08908042 +Ref: library/dis dis Instruction arg8908110 +Ref: 35e18908110 +Ref: library/dis dis Instruction argval8908202 +Ref: 35e28908202 +Ref: library/dis dis Instruction argrepr8908289 +Ref: 35e38908289 +Ref: library/dis dis Instruction offset8908373 +Ref: 35e48908373 +Ref: library/dis dis Instruction starts_line8908457 +Ref: 35e58908457 +Ref: library/dis dis Instruction is_jump_target8908555 +Ref: 35e68908555 +Ref: library/dis opcode-NOP8908787 +Ref: 35e78908787 +Ref: library/dis opcode-POP_TOP8908877 +Ref: 35e88908877 +Ref: library/dis opcode-ROT_TWO8908941 +Ref: 35e98908941 +Ref: library/dis opcode-ROT_THREE8909004 +Ref: 35ea8909004 +Ref: library/dis opcode-ROT_FOUR8909123 +Ref: 2dc8909123 +Ref: library/dis opcode-DUP_TOP8909275 +Ref: 35eb8909275 +Ref: library/dis opcode-DUP_TOP_TWO8909374 +Ref: 35ec8909374 +Ref: library/dis opcode-UNARY_POSITIVE8909644 +Ref: 35ed8909644 +Ref: library/dis opcode-UNARY_NEGATIVE8909707 +Ref: 35ee8909707 +Ref: library/dis opcode-UNARY_NOT8909770 +Ref: 35ef8909770 +Ref: library/dis opcode-UNARY_INVERT8909831 +Ref: 35f08909831 +Ref: library/dis opcode-GET_ITER8909892 +Ref: 35f18909892 +Ref: library/dis opcode-GET_YIELD_FROM_ITER8909954 +Ref: 35f28909954 +Ref: library/dis opcode-BINARY_POWER8910373 +Ref: 35f38910373 +Ref: library/dis opcode-BINARY_MULTIPLY8910441 +Ref: 35f48910441 +Ref: library/dis opcode-BINARY_MATRIX_MULTIPLY8910511 +Ref: 35f58910511 +Ref: library/dis opcode-BINARY_FLOOR_DIVIDE8910614 +Ref: 35f68910614 +Ref: library/dis opcode-BINARY_TRUE_DIVIDE8910689 +Ref: 35f78910689 +Ref: library/dis opcode-BINARY_MODULO8910762 +Ref: 35f88910762 +Ref: library/dis opcode-BINARY_ADD8910830 +Ref: 35f98910830 +Ref: library/dis opcode-BINARY_SUBTRACT8910895 +Ref: 35fa8910895 +Ref: library/dis opcode-BINARY_SUBSCR8910965 +Ref: 35fb8910965 +Ref: library/dis opcode-BINARY_LSHIFT8911032 +Ref: 35fc8911032 +Ref: library/dis opcode-BINARY_RSHIFT8911101 +Ref: 35fd8911101 +Ref: library/dis opcode-BINARY_AND8911170 +Ref: 35fe8911170 +Ref: library/dis opcode-BINARY_XOR8911235 +Ref: 35ff8911235 +Ref: library/dis opcode-BINARY_OR8911300 +Ref: 36008911300 +Ref: library/dis opcode-INPLACE_POWER8911641 +Ref: 36018911641 +Ref: library/dis opcode-INPLACE_MULTIPLY8911719 +Ref: 36028911719 +Ref: library/dis opcode-INPLACE_MATRIX_MULTIPLY8911799 +Ref: 36038911799 +Ref: library/dis opcode-INPLACE_FLOOR_DIVIDE8911912 +Ref: 36048911912 +Ref: library/dis opcode-INPLACE_TRUE_DIVIDE8911997 +Ref: 36058911997 +Ref: library/dis opcode-INPLACE_MODULO8912080 +Ref: 36068912080 +Ref: library/dis opcode-INPLACE_ADD8912158 +Ref: 36078912158 +Ref: library/dis opcode-INPLACE_SUBTRACT8912233 +Ref: 36088912233 +Ref: library/dis opcode-INPLACE_LSHIFT8912313 +Ref: 36098912313 +Ref: library/dis opcode-INPLACE_RSHIFT8912392 +Ref: 360a8912392 +Ref: library/dis opcode-INPLACE_AND8912471 +Ref: 360b8912471 +Ref: library/dis opcode-INPLACE_XOR8912546 +Ref: 360c8912546 +Ref: library/dis opcode-INPLACE_OR8912621 +Ref: 360d8912621 +Ref: library/dis opcode-STORE_SUBSCR8912695 +Ref: 360e8912695 +Ref: library/dis opcode-DELETE_SUBSCR8912762 +Ref: 360f8912762 +Ref: library/dis opcode-GET_AWAITABLE8912848 +Ref: 36108912848 +Ref: library/dis opcode-GET_AITER8913128 +Ref: 36118913128 +Ref: library/dis opcode-GET_ANEXT8913331 +Ref: 36128913331 +Ref: library/dis opcode-END_ASYNC_FOR8913509 +Ref: 2e28913509 +Ref: library/dis opcode-BEFORE_ASYNC_WITH8913960 +Ref: 36138913960 +Ref: library/dis opcode-SETUP_ASYNC_WITH8914187 +Ref: 36148914187 +Ref: library/dis opcode-PRINT_EXPR8914302 +Ref: 36158914302 +Ref: library/dis opcode-SET_ADD8914538 +Ref: 36168914538 +Ref: library/dis opcode-LIST_APPEND8914649 +Ref: 36178914649 +Ref: library/dis opcode-MAP_ADD8914769 +Ref: 2e48914769 +Ref: library/dis opcode-RETURN_VALUE8915281 +Ref: 36188915281 +Ref: library/dis opcode-YIELD_VALUE8915361 +Ref: 36198915361 +Ref: library/dis opcode-YIELD_FROM8915444 +Ref: 361a8915444 +Ref: library/dis opcode-SETUP_ANNOTATIONS8915580 +Ref: 61e8915580 +Ref: library/dis opcode-IMPORT_STAR8915878 +Ref: 361b8915878 +Ref: library/dis opcode-POP_BLOCK8916117 +Ref: 361c8916117 +Ref: library/dis opcode-POP_EXCEPT8916278 +Ref: 361d8916278 +Ref: library/dis opcode-POP_FINALLY8916611 +Ref: 2df8916611 +Ref: library/dis opcode-BEGIN_FINALLY8917551 +Ref: 2dd8917551 +Ref: library/dis opcode-END_FINALLY8917829 +Ref: 2e08917829 +Ref: library/dis opcode-LOAD_BUILD_CLASS8918660 +Ref: 361f8918660 +Ref: library/dis opcode-SETUP_WITH8918828 +Ref: 36208918828 +Ref: library/dis opcode-WITH_CLEANUP_START8919498 +Ref: 2e18919498 +Ref: library/dis opcode-WITH_CLEANUP_FINISH8920256 +Ref: 361e8920256 +Ref: library/dis opcode-STORE_NAME8920846 +Ref: 36228920846 +Ref: library/dis opcode-DELETE_NAME8921100 +Ref: 36258921100 +Ref: library/dis opcode-UNPACK_SEQUENCE8921248 +Ref: 36238921248 +Ref: library/dis opcode-UNPACK_EX8921384 +Ref: 36268921384 +Ref: library/dis opcode-STORE_ATTR8921887 +Ref: 36278921887 +Ref: library/dis opcode-DELETE_ATTR8922018 +Ref: 36288922018 +Ref: library/dis opcode-STORE_GLOBAL8922132 +Ref: 36248922132 +Ref: library/dis opcode-DELETE_GLOBAL8922238 +Ref: 36298922238 +Ref: library/dis opcode-LOAD_CONST8922340 +Ref: 362a8922340 +Ref: library/dis opcode-LOAD_NAME8922426 +Ref: 362b8922426 +Ref: library/dis opcode-BUILD_TUPLE8922534 +Ref: 362c8922534 +Ref: library/dis opcode-BUILD_LIST8922681 +Ref: 362d8922681 +Ref: library/dis opcode-BUILD_SET8922773 +Ref: 362e8922773 +Ref: library/dis opcode-BUILD_MAP8922863 +Ref: 362f8922863 +Ref: library/dis opcode-BUILD_CONST_KEY_MAP8923223 +Ref: 6178923223 +Ref: library/dis opcode-BUILD_STRING8923538 +Ref: 6168923538 +Ref: library/dis opcode-BUILD_TUPLE_UNPACK8923701 +Ref: 36308923701 +Ref: library/dis opcode-BUILD_TUPLE_UNPACK_WITH_CALL8923944 +Ref: 61d8923944 +Ref: library/dis opcode-BUILD_LIST_UNPACK8924226 +Ref: 36318924226 +Ref: library/dis opcode-BUILD_SET_UNPACK8924462 +Ref: 36328924462 +Ref: library/dis opcode-BUILD_MAP_UNPACK8924695 +Ref: 36338924695 +Ref: library/dis opcode-BUILD_MAP_UNPACK_WITH_CALL8924953 +Ref: 61b8924953 +Ref: library/dis opcode-LOAD_ATTR8925411 +Ref: 36348925411 +Ref: library/dis opcode-COMPARE_OP8925503 +Ref: 36358925503 +Ref: library/dis opcode-IMPORT_NAME8925635 +Ref: 36368925635 +Ref: library/dis opcode-IMPORT_FROM8926032 +Ref: 36378926032 +Ref: library/dis opcode-JUMP_FORWARD8926268 +Ref: 36388926268 +Ref: library/dis opcode-POP_JUMP_IF_TRUE8926348 +Ref: 36398926348 +Ref: library/dis opcode-POP_JUMP_IF_FALSE8926495 +Ref: 363a8926495 +Ref: library/dis opcode-JUMP_IF_TRUE_OR_POP8926644 +Ref: 363b8926644 +Ref: library/dis opcode-JUMP_IF_FALSE_OR_POP8926848 +Ref: 363c8926848 +Ref: library/dis opcode-JUMP_ABSOLUTE8927053 +Ref: 363d8927053 +Ref: library/dis opcode-FOR_ITER8927129 +Ref: 363e8927129 +Ref: library/dis opcode-LOAD_GLOBAL8927447 +Ref: 363f8927447 +Ref: library/dis opcode-SETUP_FINALLY8927547 +Ref: 36408927547 +Ref: library/dis opcode-CALL_FINALLY8927743 +Ref: 2de8927743 +Ref: library/dis opcode-LOAD_FAST8927980 +Ref: 36418927980 +Ref: library/dis opcode-STORE_FAST8928099 +Ref: 36218928099 +Ref: library/dis opcode-DELETE_FAST8928193 +Ref: 36428928193 +Ref: library/dis opcode-LOAD_CLOSURE8928276 +Ref: 36438928276 +Ref: library/dis opcode-LOAD_DEREF8928575 +Ref: 36448928575 +Ref: library/dis opcode-LOAD_CLASSDEREF8928761 +Ref: 92e8928761 +Ref: library/dis opcode-STORE_DEREF8928995 +Ref: 36458928995 +Ref: library/dis opcode-DELETE_DEREF8929121 +Ref: 36468929121 +Ref: library/dis opcode-RAISE_VARARGS8929306 +Ref: 36478929306 +Ref: library/dis opcode-CALL_FUNCTION8929731 +Ref: 6198929731 +Ref: library/dis opcode-CALL_FUNCTION_KW8930331 +Ref: 61a8930331 +Ref: library/dis opcode-CALL_FUNCTION_EX8931183 +Ref: 61c8931183 +Ref: library/dis opcode-LOAD_METHOD8932155 +Ref: 4ad8932155 +Ref: library/dis opcode-CALL_METHOD8932649 +Ref: 4ae8932649 +Ref: library/dis opcode-MAKE_FUNCTION8933182 +Ref: 6188933182 +Ref: library/dis opcode-BUILD_SLICE8933869 +Ref: 36488933869 +Ref: library/dis opcode-EXTENDED_ARG8934150 +Ref: 36498934150 +Ref: library/dis opcode-FORMAT_VALUE8934487 +Ref: 6158934487 +Ref: library/dis opcode-HAVE_ARGUMENT8935347 +Ref: 364a8935347 +Node: Opcode collections8935772 +Ref: library/dis id18935904 +Ref: 364b8935904 +Ref: library/dis opcode-collections8935904 +Ref: 35df8935904 +Ref: library/dis dis opname8936049 +Ref: 364c8936049 +Ref: library/dis dis opmap8936136 +Ref: 364d8936136 +Ref: library/dis dis cmp_op8936212 +Ref: 364e8936212 +Ref: library/dis dis hasconst8936281 +Ref: 364f8936281 +Ref: library/dis dis hasfree8936357 +Ref: 36508936357 +Ref: library/dis dis hasname8936693 +Ref: 36518936693 +Ref: library/dis dis hasjrel8936778 +Ref: 36528936778 +Ref: library/dis dis hasjabs8936863 +Ref: 36538936863 +Ref: library/dis dis haslocal8936949 +Ref: 36548936949 +Ref: library/dis dis hascompare8937031 +Ref: 36558937031 +Node: pickletools — Tools for pickle developers8937108 +Ref: library/pickletools doc8937261 +Ref: 36568937261 +Ref: library/pickletools module-pickletools8937261 +Ref: cb8937261 +Ref: library/pickletools pickletools-tools-for-pickle-developers8937261 +Ref: 36578937261 +Ref: pickletools — Tools for pickle developers-Footnote-18938052 +Node: Command line usage<2>8938122 +Ref: library/pickletools command-line-usage8938257 +Ref: 36588938257 +Node: Command line options<3>8939210 +Ref: library/pickletools command-line-options8939291 +Ref: 36598939291 +Ref: library/pickletools cmdoption-pickletools-a8939354 +Ref: 365a8939354 +Ref: library/pickletools cmdoption-pickletools-o8939448 +Ref: 365b8939448 +Ref: library/pickletools cmdoption-pickletools-l8939546 +Ref: 365c8939546 +Ref: library/pickletools cmdoption-pickletools-m8939655 +Ref: 365d8939655 +Ref: library/pickletools cmdoption-pickletools-p8939775 +Ref: 365e8939775 +Node: Programmatic Interface<2>8939925 +Ref: library/pickletools programmatic-interface8940060 +Ref: 365f8940060 +Ref: library/pickletools pickletools dis8940127 +Ref: 5708940127 +Ref: library/pickletools pickletools genops8940956 +Ref: 36608940956 +Ref: library/pickletools pickletools optimize8941393 +Ref: 19818941393 +Node: Miscellaneous Services8941673 +Ref: library/misc doc8941829 +Ref: 36618941829 +Ref: library/misc misc8941829 +Ref: 36628941829 +Ref: library/misc miscellaneous-services8941829 +Ref: 36638941829 +Node: formatter — Generic output formatting8942076 +Ref: library/formatter doc8942174 +Ref: 36648942174 +Ref: library/formatter formatter-generic-output-formatting8942174 +Ref: 36658942174 +Ref: library/formatter module-formatter8942174 +Ref: 828942174 +Node: The Formatter Interface8943757 +Ref: library/formatter formatter-interface8943890 +Ref: 36668943890 +Ref: library/formatter the-formatter-interface8943890 +Ref: 36678943890 +Ref: library/formatter formatter AS_IS8944218 +Ref: 36688944218 +Ref: library/formatter formatter formatter writer8944665 +Ref: 36698944665 +Ref: library/formatter formatter formatter end_paragraph8944760 +Ref: 366a8944760 +Ref: library/formatter formatter formatter add_line_break8944907 +Ref: 366b8944907 +Ref: library/formatter formatter formatter add_hor_rule8945056 +Ref: 366c8945056 +Ref: library/formatter formatter formatter add_flowing_data8945369 +Ref: 366d8945369 +Ref: library/formatter formatter formatter add_literal_data8945889 +Ref: 366e8945889 +Ref: library/formatter formatter formatter add_label_data8946108 +Ref: 366f8946108 +Ref: library/formatter formatter formatter flush_softspace8947535 +Ref: 36708947535 +Ref: library/formatter formatter formatter push_alignment8947800 +Ref: 36718947800 +Ref: library/formatter formatter formatter pop_alignment8948119 +Ref: 36728948119 +Ref: library/formatter formatter formatter push_font8948197 +Ref: 36738948197 +Ref: library/formatter formatter formatter pop_font8948578 +Ref: 36748948578 +Ref: library/formatter formatter formatter push_margin8948646 +Ref: 36758948646 +Ref: library/formatter formatter formatter pop_margin8949013 +Ref: 36768949013 +Ref: library/formatter formatter formatter push_style8949085 +Ref: 36778949085 +Ref: library/formatter formatter formatter pop_style8949388 +Ref: 36788949388 +Ref: library/formatter formatter formatter set_spacing8949655 +Ref: 36798949655 +Ref: library/formatter formatter formatter assert_line_data8949744 +Ref: 367a8949744 +Node: Formatter Implementations8950102 +Ref: library/formatter formatter-implementations8950264 +Ref: 367b8950264 +Ref: library/formatter formatter-impls8950264 +Ref: 367c8950264 +Ref: library/formatter formatter NullFormatter8950490 +Ref: 367d8950490 +Ref: library/formatter formatter AbstractFormatter8950878 +Ref: 367f8950878 +Node: The Writer Interface8951165 +Ref: library/formatter the-writer-interface8951326 +Ref: 36808951326 +Ref: library/formatter writer-interface8951326 +Ref: 36818951326 +Ref: library/formatter formatter writer flush8951747 +Ref: 36828951747 +Ref: library/formatter formatter writer new_alignment8951834 +Ref: 36838951834 +Ref: library/formatter formatter writer new_font8952198 +Ref: 36848952198 +Ref: library/formatter formatter writer new_margin8952724 +Ref: 36858952724 +Ref: library/formatter formatter writer new_spacing8953054 +Ref: 36868953054 +Ref: library/formatter formatter writer new_styles8953138 +Ref: 36878953138 +Ref: library/formatter formatter writer send_line_break8953472 +Ref: 36888953472 +Ref: library/formatter formatter writer send_paragraph8953541 +Ref: 36898953541 +Ref: library/formatter formatter writer send_hor_rule8954017 +Ref: 368a8954017 +Ref: library/formatter formatter writer send_flowing_data8954368 +Ref: 368b8954368 +Ref: library/formatter formatter writer send_literal_data8954668 +Ref: 368c8954668 +Ref: library/formatter formatter writer send_label_data8955098 +Ref: 368d8955098 +Node: Writer Implementations8955404 +Ref: library/formatter writer-implementations8955531 +Ref: 368e8955531 +Ref: library/formatter writer-impls8955531 +Ref: 368f8955531 +Ref: library/formatter formatter NullWriter8955789 +Ref: 367e8955789 +Ref: library/formatter formatter AbstractWriter8956034 +Ref: 36908956034 +Ref: library/formatter formatter DumbWriter8956250 +Ref: 36918956250 +Node: MS Windows Specific Services8956616 +Ref: library/windows doc8956770 +Ref: 36928956770 +Ref: library/windows ms-windows-specific-services8956770 +Ref: 36938956770 +Ref: library/windows mswin-specific-services8956770 +Ref: 10338956770 +Node: msilib — Read and write Microsoft Installer files8957134 +Ref: library/msilib doc8957310 +Ref: 36948957310 +Ref: library/msilib module-msilib8957310 +Ref: b58957310 +Ref: library/msilib msilib-read-and-write-microsoft-installer-files8957310 +Ref: 36958957310 +Ref: library/msilib msilib FCICreate8958384 +Ref: 36968958384 +Ref: library/msilib msilib UuidCreate8958885 +Ref: 36978958885 +Ref: library/msilib msilib OpenDatabase8959081 +Ref: 36988959081 +Ref: library/msilib msilib CreateRecord8959633 +Ref: 36998959633 +Ref: library/msilib msilib init_database8959798 +Ref: 369a8959798 +Ref: library/msilib msilib add_data8960369 +Ref: 369c8960369 +Ref: library/msilib msilib Binary8960943 +Ref: 369d8960943 +Ref: library/msilib msilib add_tables8961139 +Ref: 369e8961139 +Ref: library/msilib msilib add_stream8961480 +Ref: 369f8961480 +Ref: library/msilib msilib gen_uuid8961640 +Ref: 36a08961640 +Ref: msilib — Read and write Microsoft Installer files-Footnote-18962119 +Ref: msilib — Read and write Microsoft Installer files-Footnote-28962193 +Ref: msilib — Read and write Microsoft Installer files-Footnote-38962256 +Ref: msilib — Read and write Microsoft Installer files-Footnote-48962336 +Node: Database Objects8962416 +Ref: library/msilib database-objects8962541 +Ref: 36a18962541 +Ref: library/msilib id18962541 +Ref: 36a28962541 +Ref: library/msilib msilib Database OpenView8962594 +Ref: 36a38962594 +Ref: library/msilib msilib Database Commit8962744 +Ref: 36a48962744 +Ref: library/msilib msilib Database GetSummaryInformation8962880 +Ref: 36a58962880 +Ref: library/msilib msilib Database Close8963084 +Ref: 3ae8963084 +Ref: Database Objects-Footnote-18963352 +Ref: Database Objects-Footnote-28963432 +Ref: Database Objects-Footnote-38963512 +Ref: Database Objects-Footnote-48963592 +Node: View Objects8963672 +Ref: library/msilib id28963833 +Ref: 36a68963833 +Ref: library/msilib view-objects8963833 +Ref: 36a78963833 +Ref: library/msilib msilib View Execute8963878 +Ref: 36a88963878 +Ref: library/msilib msilib View GetColumnInfo8964106 +Ref: 36a98964106 +Ref: library/msilib msilib View Fetch8964331 +Ref: 36aa8964331 +Ref: library/msilib msilib View Modify8964444 +Ref: 36ab8964444 +Ref: library/msilib msilib View Close8964995 +Ref: 36ac8964995 +Ref: View Objects-Footnote-18965221 +Ref: View Objects-Footnote-28965301 +Ref: View Objects-Footnote-38965381 +Ref: View Objects-Footnote-48965461 +Ref: View Objects-Footnote-58965541 +Node: Summary Information Objects8965621 +Ref: library/msilib summary-information-objects8965780 +Ref: 36ad8965780 +Ref: library/msilib summary-objects8965780 +Ref: 36ae8965780 +Ref: library/msilib msilib SummaryInformation GetProperty8965855 +Ref: 36af8965855 +Ref: library/msilib msilib SummaryInformation GetPropertyCount8966446 +Ref: 36b08966446 +Ref: library/msilib msilib SummaryInformation SetProperty8966598 +Ref: 36b18966598 +Ref: library/msilib msilib SummaryInformation Persist8966891 +Ref: 36b28966891 +Ref: Summary Information Objects-Footnote-18967220 +Ref: Summary Information Objects-Footnote-28967300 +Ref: Summary Information Objects-Footnote-38967380 +Ref: Summary Information Objects-Footnote-48967460 +Node: Record Objects8967540 +Ref: library/msilib id38967693 +Ref: 36b38967693 +Ref: library/msilib record-objects8967693 +Ref: 36b48967693 +Ref: library/msilib msilib Record GetFieldCount8967742 +Ref: 36b58967742 +Ref: library/msilib msilib Record GetInteger8967873 +Ref: 36b68967873 +Ref: library/msilib msilib Record GetString8968010 +Ref: 36b78968010 +Ref: library/msilib msilib Record SetString8968144 +Ref: 36b88968144 +Ref: library/msilib msilib Record SetStream8968306 +Ref: 36b98968306 +Ref: library/msilib msilib Record SetInteger8968505 +Ref: 36ba8968505 +Ref: library/msilib msilib Record ClearData8968668 +Ref: 36bb8968668 +Ref: Record Objects-Footnote-18968949 +Ref: Record Objects-Footnote-28969029 +Ref: Record Objects-Footnote-38969109 +Ref: Record Objects-Footnote-48969189 +Ref: Record Objects-Footnote-58969269 +Node: Errors8969349 +Ref: library/msilib errors8969486 +Ref: 36bc8969486 +Ref: library/msilib msi-errors8969486 +Ref: 36bd8969486 +Node: CAB Objects8969634 +Ref: library/msilib cab8969774 +Ref: 36be8969774 +Ref: library/msilib cab-objects8969774 +Ref: 36bf8969774 +Ref: library/msilib msilib CAB8969817 +Ref: 36c08969817 +Ref: library/msilib msilib CAB append8970178 +Ref: 36c18970178 +Ref: library/msilib msilib CAB commit8970527 +Ref: 36c28970527 +Node: Directory Objects8970731 +Ref: library/msilib directory-objects8970876 +Ref: 36c38970876 +Ref: library/msilib msi-directory8970876 +Ref: 36c48970876 +Ref: library/msilib msilib Directory8970931 +Ref: 36c58970931 +Ref: library/msilib msilib Directory start_component8971700 +Ref: 36c68971700 +Ref: library/msilib msilib Directory add_file8972241 +Ref: 36c78972241 +Ref: library/msilib msilib Directory glob8972738 +Ref: 36c88972738 +Ref: library/msilib msilib Directory remove_pyc8972952 +Ref: 36c98972952 +Ref: Directory Objects-Footnote-18973168 +Ref: Directory Objects-Footnote-28973248 +Ref: Directory Objects-Footnote-38973328 +Ref: Directory Objects-Footnote-48973408 +Node: Features<3>8973488 +Ref: library/msilib features8973633 +Ref: 36ca8973633 +Ref: library/msilib id48973633 +Ref: 36cb8973633 +Ref: library/msilib msilib Feature8973670 +Ref: 36cc8973670 +Ref: library/msilib msilib Feature set_current8974068 +Ref: 36cd8974068 +Ref: Features<3>-Footnote-18974369 +Node: GUI classes8974449 +Ref: library/msilib gui-classes8974595 +Ref: 36ce8974595 +Ref: library/msilib msi-gui8974595 +Ref: 36cf8974595 +Ref: library/msilib msilib Control8974876 +Ref: 36d08974876 +Ref: library/msilib msilib Control event8975049 +Ref: 36d18975049 +Ref: library/msilib msilib Control mapping8975196 +Ref: 36d28975196 +Ref: library/msilib msilib Control condition8975318 +Ref: 36d38975318 +Ref: library/msilib msilib RadioButtonGroup8975457 +Ref: 36d48975457 +Ref: library/msilib msilib RadioButtonGroup add8975655 +Ref: 36d58975655 +Ref: library/msilib msilib Dialog8975925 +Ref: 36d68975925 +Ref: library/msilib msilib Dialog control8976230 +Ref: 36d78976230 +Ref: library/msilib msilib Dialog text8976593 +Ref: 36d88976593 +Ref: library/msilib msilib Dialog bitmap8976710 +Ref: 36d98976710 +Ref: library/msilib msilib Dialog line8976819 +Ref: 36da8976819 +Ref: library/msilib msilib Dialog pushbutton8976918 +Ref: 36db8976918 +Ref: library/msilib msilib Dialog radiogroup8977076 +Ref: 36dc8977076 +Ref: library/msilib msilib Dialog checkbox8977250 +Ref: 36dd8977250 +Ref: GUI classes-Footnote-18977611 +Ref: GUI classes-Footnote-28977691 +Ref: GUI classes-Footnote-38977771 +Ref: GUI classes-Footnote-48977851 +Ref: GUI classes-Footnote-58977931 +Ref: GUI classes-Footnote-68978011 +Ref: GUI classes-Footnote-78978091 +Node: Precomputed tables8978171 +Ref: library/msilib msi-tables8978297 +Ref: 36de8978297 +Ref: library/msilib precomputed-tables8978297 +Ref: 36df8978297 +Ref: library/msilib msilib schema8978510 +Ref: 369b8978510 +Ref: library/msilib msilib sequence8978727 +Ref: 36e08978727 +Ref: library/msilib msilib text8978962 +Ref: 36e18978962 +Node: msvcrt — Useful routines from the MS VC++ runtime8979103 +Ref: library/msvcrt doc8979322 +Ref: 36e28979322 +Ref: library/msvcrt module-msvcrt8979322 +Ref: b68979322 +Ref: library/msvcrt msvcrt-useful-routines-from-the-ms-vc-runtime8979322 +Ref: 36e38979322 +Node: File Operations8980337 +Ref: library/msvcrt file-operations8980460 +Ref: 36e48980460 +Ref: library/msvcrt msvcrt-files8980460 +Ref: 36e58980460 +Ref: library/msvcrt msvcrt locking8980511 +Ref: 31f78980511 +Ref: library/msvcrt msvcrt LK_LOCK8981174 +Ref: 36e68981174 +Ref: library/msvcrt msvcrt LK_RLCK8981199 +Ref: 36e78981199 +Ref: library/msvcrt msvcrt LK_NBLCK8981440 +Ref: 36e88981440 +Ref: library/msvcrt msvcrt LK_NBRLCK8981466 +Ref: 36e98981466 +Ref: library/msvcrt msvcrt LK_UNLCK8981595 +Ref: 36ea8981595 +Ref: library/msvcrt msvcrt setmode8981702 +Ref: 36eb8981702 +Ref: library/msvcrt msvcrt open_osfhandle8981941 +Ref: 31f88981941 +Ref: library/msvcrt msvcrt get_osfhandle8982439 +Ref: 31f68982439 +Node: Console I/O8982698 +Ref: library/msvcrt console-i-o8982845 +Ref: 36ec8982845 +Ref: library/msvcrt msvcrt-console8982845 +Ref: 36ed8982845 +Ref: library/msvcrt msvcrt kbhit8982888 +Ref: 36ee8982888 +Ref: library/msvcrt msvcrt getch8982980 +Ref: 36ef8982980 +Ref: library/msvcrt msvcrt getwch8983461 +Ref: 36f08983461 +Ref: library/msvcrt msvcrt getche8983573 +Ref: 36f18983573 +Ref: library/msvcrt msvcrt getwche8983720 +Ref: 36f28983720 +Ref: library/msvcrt msvcrt putch8983834 +Ref: 36f38983834 +Ref: library/msvcrt msvcrt putwch8983938 +Ref: 36f48983938 +Ref: library/msvcrt msvcrt ungetch8984062 +Ref: 36f58984062 +Ref: library/msvcrt msvcrt ungetwch8984277 +Ref: 36f68984277 +Node: Other Functions8984405 +Ref: library/msvcrt msvcrt-other8984528 +Ref: 36f78984528 +Ref: library/msvcrt other-functions8984528 +Ref: 36f88984528 +Ref: library/msvcrt msvcrt heapmin8984579 +Ref: 36f98984579 +Node: winreg — Windows registry access8984772 +Ref: library/winreg doc8984988 +Ref: 36fa8984988 +Ref: library/winreg module-winreg8984988 +Ref: 12a8984988 +Ref: library/winreg winreg-windows-registry-access8984988 +Ref: 36fb8984988 +Ref: library/winreg exception-changed8985402 +Ref: 36fd8985402 +Node: Functions<11>8985642 +Ref: library/winreg functions8985748 +Ref: 36fe8985748 +Ref: library/winreg id18985748 +Ref: 36ff8985748 +Ref: library/winreg winreg CloseKey8985832 +Ref: 37008985832 +Ref: library/winreg winreg ConnectRegistry8986153 +Ref: 31fd8986153 +Ref: library/winreg winreg CreateKey8986844 +Ref: 31fe8986844 +Ref: library/winreg winreg CreateKeyEx8987804 +Ref: 31ff8987804 +Ref: library/winreg winreg DeleteKey8989140 +Ref: 32008989140 +Ref: library/winreg winreg DeleteKeyEx8989888 +Ref: 32018989888 +Ref: library/winreg winreg DeleteValue8991350 +Ref: 32028991350 +Ref: library/winreg winreg EnumKey8991717 +Ref: 32058991717 +Ref: library/winreg winreg EnumValue8992383 +Ref: 32068992383 +Ref: library/winreg winreg ExpandEnvironmentStrings8993648 +Ref: 32078993648 +Ref: library/winreg winreg FlushKey8993998 +Ref: 37078993998 +Ref: library/winreg winreg LoadKey8994818 +Ref: 32088994818 +Ref: library/winreg winreg OpenKey8996010 +Ref: 32098996010 +Ref: library/winreg winreg OpenKeyEx8996083 +Ref: 370a8996083 +Ref: library/winreg winreg QueryInfoKey8997164 +Ref: 320b8997164 +Ref: library/winreg winreg QueryValue8998059 +Ref: 320d8998059 +Ref: library/winreg winreg QueryValueEx8998935 +Ref: 320e8998935 +Ref: library/winreg winreg SaveKey8999810 +Ref: 320f8999810 +Ref: library/winreg winreg SetValue9000855 +Ref: 32109000855 +Ref: library/winreg winreg SetValueEx9002033 +Ref: 32119002033 +Ref: library/winreg winreg DisableReflectionKey9003280 +Ref: 32039003280 +Ref: library/winreg winreg EnableReflectionKey9003911 +Ref: 32049003911 +Ref: library/winreg winreg QueryReflectionKey9004419 +Ref: 320c9004419 +Ref: Functions<11>-Footnote-19004922 +Ref: Functions<11>-Footnote-29004996 +Ref: Functions<11>-Footnote-39005073 +Node: Constants<10>9005150 +Ref: library/winreg constants9005288 +Ref: 370f9005288 +Ref: library/winreg id29005288 +Ref: 37109005288 +Node: HKEY_* Constants9005470 +Ref: library/winreg hkey-constants9005558 +Ref: 37029005558 +Ref: library/winreg id39005558 +Ref: 37119005558 +Ref: library/winreg winreg HKEY_CLASSES_ROOT9005611 +Ref: 37129005611 +Ref: library/winreg winreg HKEY_CURRENT_USER9005862 +Ref: 37139005862 +Ref: library/winreg winreg HKEY_LOCAL_MACHINE9006165 +Ref: 37099006165 +Ref: library/winreg winreg HKEY_USERS9006389 +Ref: 37089006389 +Ref: library/winreg winreg HKEY_PERFORMANCE_DATA9006598 +Ref: 37149006598 +Ref: library/winreg winreg HKEY_CURRENT_CONFIG9006865 +Ref: 37159006865 +Ref: library/winreg winreg HKEY_DYN_DATA9007000 +Ref: 37169007000 +Node: Access Rights9007092 +Ref: library/winreg access-rights9007200 +Ref: 37049007200 +Ref: library/winreg id49007200 +Ref: 37179007200 +Ref: library/winreg winreg KEY_ALL_ACCESS9007311 +Ref: 37189007311 +Ref: library/winreg winreg KEY_WRITE9007600 +Ref: 37039007600 +Ref: library/winreg winreg KEY_READ9007753 +Ref: 370b9007753 +Ref: library/winreg winreg KEY_EXECUTE9007932 +Ref: 371e9007932 +Ref: library/winreg winreg KEY_QUERY_VALUE9008004 +Ref: 37199008004 +Ref: library/winreg winreg KEY_SET_VALUE9008092 +Ref: 370d9008092 +Ref: library/winreg winreg KEY_CREATE_SUB_KEY9008183 +Ref: 371a9008183 +Ref: library/winreg winreg KEY_ENUMERATE_SUB_KEYS9008273 +Ref: 371b9008273 +Ref: library/winreg winreg KEY_NOTIFY9008373 +Ref: 371c9008373 +Ref: library/winreg winreg KEY_CREATE_LINK9008507 +Ref: 371d9008507 +Ref: Access Rights-Footnote-19008638 +Node: 64-bit Specific9008715 +Ref: library/winreg bit-access-rights9008780 +Ref: 371f9008780 +Ref: library/winreg bit-specific9008780 +Ref: 37209008780 +Ref: library/winreg winreg KEY_WOW64_64KEY9008899 +Ref: 37059008899 +Ref: library/winreg winreg KEY_WOW64_32KEY9009036 +Ref: 37219009036 +Ref: 64-bit Specific-Footnote-19009209 +Node: Value Types9009281 +Ref: library/winreg id59009364 +Ref: 37229009364 +Ref: library/winreg value-types9009364 +Ref: 370e9009364 +Ref: library/winreg winreg REG_BINARY9009459 +Ref: 37239009459 +Ref: library/winreg winreg REG_DWORD9009519 +Ref: 37249009519 +Ref: library/winreg winreg REG_DWORD_LITTLE_ENDIAN9009568 +Ref: 37259009568 +Ref: library/winreg winreg REG_DWORD_BIG_ENDIAN9009700 +Ref: 37269009700 +Ref: library/winreg winreg REG_EXPAND_SZ9009783 +Ref: 37069009783 +Ref: library/winreg winreg REG_LINK9009912 +Ref: 37279009912 +Ref: library/winreg winreg REG_MULTI_SZ9009970 +Ref: 37289009970 +Ref: library/winreg winreg REG_NONE9010136 +Ref: 37299010136 +Ref: library/winreg winreg REG_QWORD9010192 +Ref: 5b29010192 +Ref: library/winreg winreg REG_QWORD_LITTLE_ENDIAN9010269 +Ref: 372a9010269 +Ref: library/winreg winreg REG_RESOURCE_LIST9010426 +Ref: 372b9010426 +Ref: library/winreg winreg REG_FULL_RESOURCE_DESCRIPTOR9010499 +Ref: 372c9010499 +Ref: library/winreg winreg REG_RESOURCE_REQUIREMENTS_LIST9010572 +Ref: 372d9010572 +Ref: library/winreg winreg REG_SZ9010653 +Ref: 370c9010653 +Ref: Value Types-Footnote-19010746 +Node: Registry Handle Objects9010823 +Ref: library/winreg handle-object9010939 +Ref: 36fc9010939 +Ref: library/winreg registry-handle-objects9010939 +Ref: 372e9010939 +Ref: library/winreg winreg PyHKEY Close9012099 +Ref: 37019012099 +Ref: library/winreg winreg PyHKEY Detach9012231 +Ref: 320a9012231 +Ref: library/winreg winreg PyHKEY __enter__9012829 +Ref: 372f9012829 +Ref: library/winreg winreg PyHKEY __exit__9012861 +Ref: 37309012861 +Node: winsound — Sound-playing interface for Windows9013255 +Ref: library/winsound doc9013411 +Ref: 37319013411 +Ref: library/winsound module-winsound9013411 +Ref: 12b9013411 +Ref: library/winsound winsound-sound-playing-interface-for-windows9013411 +Ref: 37329013411 +Ref: library/winsound winsound Beep9013761 +Ref: 5b49013761 +Ref: library/winsound winsound PlaySound9014149 +Ref: 5b69014149 +Ref: library/winsound winsound MessageBeep9014721 +Ref: 5b59014721 +Ref: library/winsound winsound SND_FILENAME9015330 +Ref: 37339015330 +Ref: library/winsound winsound SND_ALIAS9015463 +Ref: 37349015463 +Ref: library/winsound winsound SND_LOOP9017081 +Ref: 37369017081 +Ref: library/winsound winsound SND_MEMORY9017268 +Ref: 37389017268 +Ref: library/winsound winsound SND_PURGE9017630 +Ref: 37399017630 +Ref: library/winsound winsound SND_ASYNC9017790 +Ref: 37379017790 +Ref: library/winsound winsound SND_NODEFAULT9017886 +Ref: 37359017886 +Ref: library/winsound winsound SND_NOSTOP9018009 +Ref: 373a9018009 +Ref: library/winsound winsound SND_NOWAIT9018089 +Ref: 373b9018089 +Ref: library/winsound winsound MB_ICONASTERISK9018247 +Ref: 373c9018247 +Ref: library/winsound winsound MB_ICONEXCLAMATION9018325 +Ref: 373d9018325 +Ref: library/winsound winsound MB_ICONHAND9018410 +Ref: 373e9018410 +Ref: library/winsound winsound MB_ICONQUESTION9018481 +Ref: 373f9018481 +Ref: library/winsound winsound MB_OK9018560 +Ref: 37409018560 +Node: Unix Specific Services9018628 +Ref: library/unix doc9018778 +Ref: 37419018778 +Ref: library/unix unix9018778 +Ref: 37429018778 +Ref: library/unix unix-specific-services9018778 +Ref: 37439018778 +Node: posix — The most common POSIX system calls9019629 +Ref: library/posix doc9019770 +Ref: 37449019770 +Ref: library/posix module-posix9019770 +Ref: d19019770 +Ref: library/posix posix-the-most-common-posix-system-calls9019770 +Ref: 37459019770 +Node: Large File Support9020970 +Ref: library/posix large-file-support9021101 +Ref: 37469021101 +Ref: library/posix posix-large-files9021101 +Ref: 37479021101 +Node: Notable Module Contents9022152 +Ref: library/posix notable-module-contents9022283 +Ref: 37489022283 +Ref: library/posix posix-contents9022283 +Ref: 37499022283 +Ref: library/posix posix environ9022484 +Ref: 374a9022484 +Node: pwd — The password database9023670 +Ref: library/pwd doc9023857 +Ref: 374b9023857 +Ref: library/pwd module-pwd9023857 +Ref: d69023857 +Ref: library/pwd pwd-the-password-database9023857 +Ref: 374c9023857 +Ref: library/pwd pwd getpwuid9026132 +Ref: 374d9026132 +Ref: library/pwd pwd getpwnam9026238 +Ref: 374e9026238 +Ref: library/pwd pwd getpwall9026339 +Ref: 374f9026339 +Node: spwd — The shadow password database9026653 +Ref: library/spwd doc9026822 +Ref: 37509026822 +Ref: library/spwd module-spwd9026822 +Ref: f19026822 +Ref: library/spwd spwd-the-shadow-password-database9026822 +Ref: 37519026822 +Ref: library/spwd spwd getspnam9029116 +Ref: 5fe9029116 +Ref: library/spwd spwd getspall9029373 +Ref: 37529029373 +Node: grp — The group database9029694 +Ref: library/grp doc9029876 +Ref: 37539029876 +Ref: library/grp grp-the-group-database9029876 +Ref: 37549029876 +Ref: library/grp module-grp9029876 +Ref: 8b9029876 +Ref: library/grp grp getgrgid9031420 +Ref: 5df9031420 +Ref: library/grp grp getgrnam9031769 +Ref: 37559031769 +Ref: library/grp grp getgrall9031945 +Ref: 37569031945 +Node: crypt — Function to check Unix passwords9032241 +Ref: library/crypt doc9032421 +Ref: 37579032421 +Ref: library/crypt crypt-function-to-check-unix-passwords9032421 +Ref: 37589032421 +Ref: library/crypt module-crypt9032421 +Ref: 299032421 +Ref: crypt — Function to check Unix passwords-Footnote-19033429 +Node: Hashing Methods9033493 +Ref: library/crypt hashing-methods9033613 +Ref: 37599033613 +Ref: library/crypt crypt METHOD_SHA5129033800 +Ref: 375a9033800 +Ref: library/crypt crypt METHOD_SHA2569033989 +Ref: 375b9033989 +Ref: library/crypt crypt METHOD_BLOWFISH9034148 +Ref: 375c9034148 +Ref: library/crypt crypt METHOD_MD59034329 +Ref: 375d9034329 +Ref: library/crypt crypt METHOD_CRYPT9034480 +Ref: 6099034480 +Node: Module Attributes9034624 +Ref: library/crypt module-attributes9034772 +Ref: 375e9034772 +Ref: library/crypt crypt methods9034848 +Ref: 375f9034848 +Node: Module Functions<2>9035024 +Ref: library/crypt module-functions9035169 +Ref: 37609035169 +Ref: library/crypt crypt crypt9035284 +Ref: 37619035284 +Ref: library/crypt crypt mksalt9036788 +Ref: 37c9036788 +Node: Examples<31>9037641 +Ref: library/crypt examples9037760 +Ref: 37629037760 +Node: termios — POSIX style tty control9038901 +Ref: library/termios doc9039089 +Ref: 37639039089 +Ref: library/termios module-termios9039089 +Ref: 1049039089 +Ref: library/termios termios-posix-style-tty-control9039089 +Ref: 37649039089 +Ref: library/termios termios tcgetattr9040066 +Ref: 37659040066 +Ref: library/termios termios tcsetattr9040645 +Ref: 37669040645 +Ref: library/termios termios tcsendbreak9041139 +Ref: 37679041139 +Ref: library/termios termios tcdrain9041353 +Ref: 37689041353 +Ref: library/termios termios tcflush9041476 +Ref: 37699041476 +Ref: library/termios termios tcflow9041734 +Ref: 376a9041734 +Node: Example<16>9042159 +Ref: library/termios example9042242 +Ref: 376b9042242 +Ref: library/termios termios-example9042242 +Ref: 376c9042242 +Node: tty — Terminal control functions9043000 +Ref: library/tty doc9043179 +Ref: 376d9043179 +Ref: library/tty module-tty9043179 +Ref: 1159043179 +Ref: library/tty tty-terminal-control-functions9043179 +Ref: 376e9043179 +Ref: library/tty tty setraw9043602 +Ref: 376f9043602 +Ref: library/tty tty setcbreak9043838 +Ref: 37709043838 +Ref: tty — Terminal control functions-Footnote-19044203 +Node: pty — Pseudo-terminal utilities9044265 +Ref: library/pty doc9044451 +Ref: 37719044451 +Ref: library/pty module-pty9044451 +Ref: d59044451 +Ref: library/pty pty-pseudo-terminal-utilities9044451 +Ref: 37729044451 +Ref: library/pty pty fork9045086 +Ref: 37739045086 +Ref: library/pty pty openpty9045501 +Ref: 37749045501 +Ref: library/pty pty spawn9045780 +Ref: 8989045780 +Ref: pty — Pseudo-terminal utilities-Footnote-19047814 +Ref: pty — Pseudo-terminal utilities-Footnote-29047876 +Node: Example<17>9047919 +Ref: library/pty example9048000 +Ref: 37759048000 +Node: fcntl — The fcntl and ioctl system calls9049182 +Ref: library/fcntl doc9049372 +Ref: 37769049372 +Ref: library/fcntl fcntl-the-fcntl-and-ioctl-system-calls9049372 +Ref: 37779049372 +Ref: library/fcntl module-fcntl9049372 +Ref: 7e9049372 +Ref: library/fcntl fcntl fcntl9050488 +Ref: 23939050488 +Ref: library/fcntl fcntl ioctl9051932 +Ref: dde9051932 +Ref: library/fcntl fcntl flock9054262 +Ref: 31f39054262 +Ref: library/fcntl fcntl lockf9054769 +Ref: d529054769 +Node: pipes — Interface to shell pipelines9057491 +Ref: library/pipes doc9057687 +Ref: 37789057687 +Ref: library/pipes module-pipes9057687 +Ref: cc9057687 +Ref: library/pipes pipes-interface-to-shell-pipelines9057687 +Ref: 37799057687 +Ref: library/pipes pipes Template9058236 +Ref: 377a9058236 +Ref: pipes — Interface to shell pipelines-Footnote-19058607 +Node: Template Objects9058671 +Ref: library/pipes id19058762 +Ref: 377b9058762 +Ref: library/pipes template-objects9058762 +Ref: 377c9058762 +Ref: library/pipes pipes Template reset9058854 +Ref: 377d9058854 +Ref: library/pipes pipes Template clone9058941 +Ref: 377e9058941 +Ref: library/pipes pipes Template debug9059023 +Ref: 377f9059023 +Ref: library/pipes pipes Template append9059268 +Ref: 37809059268 +Ref: library/pipes pipes Template prepend9060018 +Ref: 37819060018 +Ref: library/pipes pipes Template open9060168 +Ref: 37829060168 +Ref: library/pipes pipes Template copy9060370 +Ref: 37839060370 +Node: resource — Resource usage information9060466 +Ref: library/resource doc9060665 +Ref: 37849060665 +Ref: library/resource module-resource9060665 +Ref: e09060665 +Ref: library/resource resource-resource-usage-information9060665 +Ref: 37859060665 +Ref: library/resource resource error9061150 +Ref: 37869061150 +Ref: resource — Resource usage information-Footnote-19061424 +Node: Resource Limits9061473 +Ref: library/resource resource-limits9061587 +Ref: 37879061587 +Ref: library/resource resource RLIM_INFINITY9062466 +Ref: 37889062466 +Ref: library/resource resource getrlimit9062570 +Ref: 37899062570 +Ref: library/resource resource setrlimit9062864 +Ref: 31cd9062864 +Ref: library/resource resource prlimit9064022 +Ref: 89f9064022 +Ref: library/resource resource RLIMIT_CORE9065627 +Ref: 31ce9065627 +Ref: library/resource resource RLIMIT_CPU9065888 +Ref: 378b9065888 +Ref: library/resource resource RLIMIT_FSIZE9066248 +Ref: 378c9066248 +Ref: library/resource resource RLIMIT_DATA9066344 +Ref: 378d9066344 +Ref: library/resource resource RLIMIT_STACK9066435 +Ref: 378e9066435 +Ref: library/resource resource RLIMIT_RSS9066632 +Ref: 378f9066632 +Ref: library/resource resource RLIMIT_NPROC9066750 +Ref: 37909066750 +Ref: library/resource resource RLIMIT_NOFILE9066853 +Ref: 378a9066853 +Ref: library/resource resource RLIMIT_OFILE9066967 +Ref: 37919066967 +Ref: library/resource resource RLIMIT_MEMLOCK9067050 +Ref: 37929067050 +Ref: library/resource resource RLIMIT_VMEM9067148 +Ref: 37939067148 +Ref: library/resource resource RLIMIT_AS9067250 +Ref: 37949067250 +Ref: library/resource resource RLIMIT_MSGQUEUE9067371 +Ref: 8a09067371 +Ref: library/resource resource RLIMIT_NICE9067561 +Ref: 8a19067561 +Ref: library/resource resource RLIMIT_RTPRIO9067759 +Ref: 8a29067759 +Ref: library/resource resource RLIMIT_RTTIME9067919 +Ref: 8a39067919 +Ref: library/resource resource RLIMIT_SIGPENDING9068177 +Ref: 8a49068177 +Ref: library/resource resource RLIMIT_SBSIZE9068352 +Ref: 8a59068352 +Ref: library/resource resource RLIMIT_SWAP9068656 +Ref: 8a69068656 +Ref: library/resource resource RLIMIT_NPTS9069042 +Ref: 8a79069042 +Ref: Resource Limits-Footnote-19069258 +Node: Resource Usage9069325 +Ref: library/resource resource-usage9069439 +Ref: 37959069439 +Ref: library/resource resource getrusage9069556 +Ref: d999069556 +Ref: library/resource resource getpagesize9074049 +Ref: 37969074049 +Ref: library/resource resource RUSAGE_SELF9074353 +Ref: 37979074353 +Ref: library/resource resource RUSAGE_CHILDREN9074553 +Ref: 37989074553 +Ref: library/resource resource RUSAGE_BOTH9074752 +Ref: 37999074752 +Ref: library/resource resource RUSAGE_THREAD9074949 +Ref: 379a9074949 +Node: nis — Interface to Sun’s NIS Yellow Pages9075143 +Ref: library/nis doc9075343 +Ref: 379b9075343 +Ref: library/nis module-nis9075343 +Ref: bf9075343 +Ref: library/nis nis-interface-to-sun-s-nis-yellow-pages9075343 +Ref: 379c9075343 +Ref: library/nis nis match9075794 +Ref: d929075794 +Ref: library/nis nis cat9076341 +Ref: 379e9076341 +Ref: library/nis nis maps9076807 +Ref: d939076807 +Ref: library/nis nis get_default_domain9077035 +Ref: 379f9077035 +Ref: library/nis nis error9077180 +Ref: 379d9077180 +Node: syslog — Unix syslog library routines9077271 +Ref: library/syslog doc9077423 +Ref: 37a09077423 +Ref: library/syslog module-syslog9077423 +Ref: ff9077423 +Ref: library/syslog syslog-unix-syslog-library-routines9077423 +Ref: 37a19077423 +Ref: library/syslog syslog syslog9078004 +Ref: 31fc9078004 +Ref: library/syslog syslog openlog9078815 +Ref: 31fa9078815 +Ref: library/syslog syslog closelog9079979 +Ref: 31f99079979 +Ref: library/syslog syslog setlogmask9080512 +Ref: 31fb9080512 +Node: Examples<32>9081791 +Ref: library/syslog examples9081879 +Ref: 37a29081879 +Node: Simple example9081947 +Ref: library/syslog simple-example9082010 +Ref: 37a39082010 +Node: Superseded Modules9082522 +Ref: library/superseded doc9082664 +Ref: 37a49082664 +Ref: library/superseded superseded9082664 +Ref: 37a59082664 +Ref: library/superseded superseded-modules9082664 +Ref: 37a69082664 +Node: optparse — Parser for command line options9082954 +Ref: library/optparse doc9083097 +Ref: 37a79083097 +Ref: library/optparse module-optparse9083097 +Ref: c39083097 +Ref: library/optparse optparse-parser-for-command-line-options9083097 +Ref: 37a89083097 +Ref: optparse — Parser for command line options-Footnote-19085998 +Node: Background9086065 +Ref: library/optparse background9086176 +Ref: 37aa9086176 +Ref: library/optparse optparse-background9086176 +Ref: 37ab9086176 +Node: Terminology9086661 +Ref: library/optparse optparse-terminology9086749 +Ref: 37ac9086749 +Ref: library/optparse terminology9086749 +Ref: 37ad9086749 +Node: What are options for?9090577 +Ref: library/optparse optparse-what-options-for9090708 +Ref: 37ae9090708 +Ref: library/optparse what-are-options-for9090708 +Ref: 37af9090708 +Node: What are positional arguments for?9092393 +Ref: library/optparse optparse-what-positional-arguments-for9092504 +Ref: 37b09092504 +Ref: library/optparse what-are-positional-arguments-for9092504 +Ref: 37b19092504 +Node: Tutorial<2>9093839 +Ref: library/optparse optparse-tutorial9093974 +Ref: 37b29093974 +Ref: library/optparse tutorial9093974 +Ref: 37b39093974 +Node: Understanding option actions9096580 +Ref: library/optparse optparse-understanding-option-actions9096681 +Ref: 37b89096681 +Ref: library/optparse understanding-option-actions9096681 +Ref: 37b99096681 +Node: The store action9097272 +Ref: library/optparse optparse-store-action9097411 +Ref: 37bb9097411 +Ref: library/optparse the-store-action9097411 +Ref: 37bc9097411 +Node: Handling boolean flag options9099601 +Ref: library/optparse handling-boolean-flag-options9099725 +Ref: 37bd9099725 +Ref: library/optparse optparse-handling-boolean-options9099725 +Ref: 37be9099725 +Node: Other actions9100608 +Ref: library/optparse optparse-other-actions9100730 +Ref: 37bf9100730 +Ref: library/optparse other-actions9100730 +Ref: 37c09100730 +Node: Default values9101148 +Ref: library/optparse default-values9101256 +Ref: 37c39101256 +Ref: library/optparse optparse-default-values9101256 +Ref: 37c49101256 +Node: Generating help9103144 +Ref: library/optparse generating-help9103264 +Ref: 37c59103264 +Ref: library/optparse optparse-generating-help9103264 +Ref: 37c69103264 +Node: Grouping Options9107438 +Ref: library/optparse grouping-options9107506 +Ref: 37c79107506 +Ref: library/optparse optparse OptionGroup9107837 +Ref: 37c89107837 +Ref: library/optparse optparse OptionParser get_option_group9111335 +Ref: 37c99111335 +Node: Printing a version string9111605 +Ref: library/optparse optparse-printing-version-string9111738 +Ref: 37ca9111738 +Ref: library/optparse printing-a-version-string9111738 +Ref: 37cb9111738 +Ref: library/optparse optparse OptionParser print_version9112622 +Ref: 37cc9112622 +Ref: library/optparse optparse OptionParser get_version9112996 +Ref: 37ce9112996 +Node: How optparse handles errors9113140 +Ref: library/optparse how-optparse-handles-errors9113281 +Ref: 37cf9113281 +Ref: library/optparse optparse-how-optparse-handles-errors9113281 +Ref: 37d09113281 +Node: Putting it all together9115364 +Ref: library/optparse optparse-putting-it-all-together9115471 +Ref: 37d19115471 +Ref: library/optparse putting-it-all-together9115471 +Ref: 37d29115471 +Node: Reference Guide9116389 +Ref: library/optparse optparse-reference-guide9116530 +Ref: 37c19116530 +Ref: library/optparse reference-guide9116530 +Ref: 37d39116530 +Node: Creating the parser9116903 +Ref: library/optparse creating-the-parser9117004 +Ref: 37d49117004 +Ref: library/optparse optparse-creating-parser9117004 +Ref: 37d59117004 +Ref: library/optparse optparse OptionParser9117149 +Ref: 37a99117149 +Node: Populating the parser9120155 +Ref: library/optparse optparse-populating-parser9120281 +Ref: 37d79120281 +Ref: library/optparse populating-the-parser9120281 +Ref: 37d89120281 +Node: Defining options9121567 +Ref: library/optparse defining-options9121691 +Ref: 37d99121691 +Ref: library/optparse optparse-defining-options9121691 +Ref: 37da9121691 +Ref: library/optparse optparse OptionParser add_option9122100 +Ref: 1d4f9122100 +Node: Option attributes9124973 +Ref: library/optparse option-attributes9125099 +Ref: 37dc9125099 +Ref: library/optparse optparse-option-attributes9125099 +Ref: 37dd9125099 +Ref: library/optparse optparse Option action9125435 +Ref: 37b49125435 +Ref: library/optparse optparse Option type9125658 +Ref: 37b59125658 +Ref: library/optparse optparse Option dest9125875 +Ref: 37b69125875 +Ref: library/optparse optparse Option default9126220 +Ref: 37df9126220 +Ref: library/optparse optparse Option nargs9126414 +Ref: 37e19126414 +Ref: library/optparse optparse Option const9126647 +Ref: 37e29126647 +Ref: library/optparse optparse Option choices9126757 +Ref: 37e39126757 +Ref: library/optparse optparse Option callback9126881 +Ref: 37e49126881 +Ref: library/optparse optparse Option callback_args9127113 +Ref: 37e59127113 +Ref: library/optparse optparse Option callback_kwargs9127149 +Ref: 37e69127149 +Ref: library/optparse optparse Option help9127313 +Ref: 37b79127313 +Ref: library/optparse optparse Option metavar9127661 +Ref: 37e79127661 +Node: Standard option actions9127866 +Ref: library/optparse optparse-standard-option-actions9127997 +Ref: 37db9127997 +Ref: library/optparse standard-option-actions9127997 +Ref: 37e89127997 +Node: Standard option types9136084 +Ref: library/optparse optparse-standard-option-types9136218 +Ref: 37de9136218 +Ref: library/optparse standard-option-types9136218 +Ref: 37e99136218 +Node: Parsing arguments<2>9137689 +Ref: library/optparse optparse-parsing-arguments9137844 +Ref: 37ea9137844 +Ref: library/optparse parsing-arguments9137844 +Ref: 37eb9137844 +Node: Querying and manipulating your option parser9139214 +Ref: library/optparse optparse-querying-manipulating-option-parser9139373 +Ref: 37ec9139373 +Ref: library/optparse querying-and-manipulating-your-option-parser9139373 +Ref: 37ed9139373 +Ref: library/optparse optparse OptionParser disable_interspersed_args9139682 +Ref: 1d509139682 +Ref: library/optparse optparse OptionParser enable_interspersed_args9140472 +Ref: 37ee9140472 +Ref: library/optparse optparse OptionParser get_option9140678 +Ref: 37ef9140678 +Ref: library/optparse optparse OptionParser has_option9140851 +Ref: 37f09140851 +Ref: library/optparse optparse OptionParser remove_option9141027 +Ref: 37f19141027 +Node: Conflicts between options9141413 +Ref: library/optparse conflicts-between-options9141562 +Ref: 37f29141562 +Ref: library/optparse optparse-conflicts-between-options9141562 +Ref: 37d69141562 +Node: Cleanup<2>9144026 +Ref: library/optparse cleanup9144144 +Ref: 37f39144144 +Ref: library/optparse optparse-cleanup9144144 +Ref: 37f49144144 +Node: Other methods9144557 +Ref: library/optparse optparse-other-methods9144641 +Ref: 37f59144641 +Ref: library/optparse other-methods9144641 +Ref: 37f69144641 +Ref: library/optparse optparse OptionParser set_usage9144743 +Ref: 37f79144743 +Ref: library/optparse optparse OptionParser print_usage9145032 +Ref: 37cd9145032 +Ref: library/optparse optparse OptionParser get_usage9145368 +Ref: 37f89145368 +Ref: library/optparse optparse OptionParser set_defaults9145506 +Ref: 37e09145506 +Node: Option Callbacks9146674 +Ref: library/optparse option-callbacks9146822 +Ref: 37f99146822 +Ref: library/optparse optparse-option-callbacks9146822 +Ref: 37c29146822 +Node: Defining a callback option9148039 +Ref: library/optparse defining-a-callback-option9148151 +Ref: 37fa9148151 +Ref: library/optparse optparse-defining-callback-option9148151 +Ref: 37fb9148151 +Node: How callbacks are called9150376 +Ref: library/optparse how-callbacks-are-called9150525 +Ref: 37fc9150525 +Ref: library/optparse optparse-how-callbacks-called9150525 +Ref: 37fd9150525 +Node: Raising errors in a callback9153135 +Ref: library/optparse optparse-raising-errors-in-callback9153293 +Ref: 37fe9153293 +Ref: library/optparse raising-errors-in-a-callback9153293 +Ref: 37ff9153293 +Node: Callback example 1 trivial callback9153764 +Ref: library/optparse callback-example-1-trivial-callback9153935 +Ref: 38009153935 +Ref: library/optparse optparse-callback-example-19153935 +Ref: 38019153935 +Node: Callback example 2 check option order9154383 +Ref: library/optparse callback-example-2-check-option-order9154575 +Ref: 38029154575 +Ref: library/optparse optparse-callback-example-29154575 +Ref: 38039154575 +Node: Callback example 3 check option order generalized9155133 +Ref: library/optparse callback-example-3-check-option-order-generalized9155334 +Ref: 38049155334 +Ref: library/optparse optparse-callback-example-39155334 +Ref: 38059155334 +Node: Callback example 4 check arbitrary condition9156108 +Ref: library/optparse callback-example-4-check-arbitrary-condition9156306 +Ref: 38069156306 +Ref: library/optparse optparse-callback-example-49156306 +Ref: 38079156306 +Node: Callback example 5 fixed arguments9157101 +Ref: library/optparse callback-example-5-fixed-arguments9157287 +Ref: 38089157287 +Ref: library/optparse optparse-callback-example-59157287 +Ref: 38099157287 +Node: Callback example 6 variable arguments9158361 +Ref: library/optparse callback-example-6-variable-arguments9158494 +Ref: 380a9158494 +Ref: library/optparse optparse-callback-example-69158494 +Ref: 380b9158494 +Node: Extending optparse9160599 +Ref: library/optparse extending-optparse9160723 +Ref: 380c9160723 +Ref: library/optparse optparse-extending-optparse9160723 +Ref: 37ba9160723 +Node: Adding new types9161060 +Ref: library/optparse adding-new-types9161158 +Ref: 380d9161158 +Ref: library/optparse optparse-adding-new-types9161158 +Ref: 380e9161158 +Ref: library/optparse optparse Option TYPES9161447 +Ref: 380f9161447 +Ref: library/optparse optparse Option TYPE_CHECKER9161606 +Ref: 38109161606 +Node: Adding new actions9164617 +Ref: library/optparse adding-new-actions9164715 +Ref: 38119164715 +Ref: library/optparse optparse-adding-new-actions9164715 +Ref: 38129164715 +Ref: library/optparse optparse Option ACTIONS9165818 +Ref: 38139165818 +Ref: library/optparse optparse Option STORE_ACTIONS9165894 +Ref: 38149165894 +Ref: library/optparse optparse Option TYPED_ACTIONS9165987 +Ref: 38159165987 +Ref: library/optparse optparse Option ALWAYS_TYPED_ACTIONS9166080 +Ref: 38169166080 +Node: imp — Access the import internals9169382 +Ref: library/imp doc9169525 +Ref: 38179169525 +Ref: library/imp imp-access-the-import-internals9169525 +Ref: 38189169525 +Ref: library/imp module-imp9169525 +Ref: 9a9169525 +Ref: library/imp imp get_magic9169976 +Ref: 8639169976 +Ref: library/imp imp get_suffixes9170267 +Ref: 38199170267 +Ref: library/imp imp find_module9171005 +Ref: 381d9171005 +Ref: library/imp imp load_module9173518 +Ref: 38219173518 +Ref: library/imp imp new_module9175389 +Ref: 38229175389 +Ref: library/imp imp reload9175638 +Ref: c709175638 +Ref: library/imp imp cache_from_source9178898 +Ref: 38239178898 +Ref: library/imp imp source_from_cache9179954 +Ref: 38259179954 +Ref: library/imp imp get_tag9180689 +Ref: 38249180689 +Ref: library/imp imp lock_held9181240 +Ref: 38269181240 +Ref: library/imp imp acquire_lock9182174 +Ref: 38279182174 +Ref: library/imp imp release_lock9182875 +Ref: 38289182875 +Ref: library/imp imp PY_SOURCE9183411 +Ref: 381a9183411 +Ref: library/imp imp PY_COMPILED9183517 +Ref: 381b9183517 +Ref: library/imp imp C_EXTENSION9183639 +Ref: 381c9183639 +Ref: library/imp imp PKG_DIRECTORY9183769 +Ref: 38209183769 +Ref: library/imp imp C_BUILTIN9183885 +Ref: 381e9183885 +Ref: library/imp imp PY_FROZEN9183995 +Ref: 381f9183995 +Ref: library/imp imp NullImporter9184103 +Ref: 9bb9184103 +Ref: library/imp imp NullImporter find_module9184497 +Ref: 38299184497 +Ref: imp — Access the import internals-Footnote-19184987 +Ref: imp — Access the import internals-Footnote-29185049 +Ref: imp — Access the import internals-Footnote-39185098 +Ref: imp — Access the import internals-Footnote-49185147 +Ref: imp — Access the import internals-Footnote-59185196 +Ref: imp — Access the import internals-Footnote-69185245 +Ref: imp — Access the import internals-Footnote-79185294 +Node: Examples<33>9185343 +Ref: library/imp examples9185427 +Ref: 382a9185427 +Ref: library/imp examples-imp9185427 +Ref: 382b9185427 +Node: Undocumented Modules9186433 +Ref: library/undoc doc9186544 +Ref: 382c9186544 +Ref: library/undoc undoc9186544 +Ref: 382d9186544 +Ref: library/undoc undocumented-modules9186544 +Ref: 382e9186544 +Node: Platform specific modules9186991 +Ref: library/undoc platform-specific-modules9187073 +Ref: 382f9187073 +Node: Extending and Embedding the Python Interpreter9187464 +Ref: extending/index doc9187624 +Ref: 38309187624 +Ref: extending/index extending-and-embedding-the-python-interpreter9187624 +Ref: 38319187624 +Ref: extending/index extending-index9187624 +Ref: e909187624 +Node: Recommended third party tools9188925 +Ref: extending/index recommended-third-party-tools9189091 +Ref: 38329189091 +Ref: Recommended third party tools-Footnote-19189789 +Ref: Recommended third party tools-Footnote-29189816 +Ref: Recommended third party tools-Footnote-39189852 +Ref: Recommended third party tools-Footnote-49189880 +Ref: Recommended third party tools-Footnote-59189914 +Node: Creating extensions without third party tools9189987 +Ref: extending/index creating-extensions-without-third-party-tools9190215 +Ref: 38339190215 +Node: Extending Python with C or C++9190831 +Ref: extending/extending doc9190985 +Ref: 38349190985 +Ref: extending/extending extending-intro9190985 +Ref: 38359190985 +Ref: extending/extending extending-python-with-c-or-c9190985 +Ref: 38369190985 +Ref: Extending Python with C or C++-Footnote-19192916 +Node: A Simple Example9192953 +Ref: extending/extending a-simple-example9193077 +Ref: 38379193077 +Ref: extending/extending extending-simpleexample9193077 +Ref: 38389193077 +Ref: A Simple Example-Footnote-19196812 +Node: Intermezzo Errors and Exceptions9196966 +Ref: extending/extending extending-errors9197118 +Ref: 383a9197118 +Ref: extending/extending intermezzo-errors-and-exceptions9197118 +Ref: 383b9197118 +Node: Back to the Example9204243 +Ref: extending/extending back-to-the-example9204434 +Ref: 38429204434 +Ref: extending/extending backtoexample9204434 +Ref: 38439204434 +Node: The Module’s Method Table and Initialization Function9206096 +Ref: extending/extending methodtable9206278 +Ref: 38459206278 +Ref: extending/extending the-module-s-method-table-and-initialization-function9206278 +Ref: 38469206278 +Ref: The Module’s Method Table and Initialization Function-Footnote-19211685 +Node: Compilation and Linkage9211734 +Ref: extending/extending compilation9211928 +Ref: 38499211928 +Ref: extending/extending compilation-and-linkage9211928 +Ref: 384a9211928 +Node: Calling Python Functions from C9213402 +Ref: extending/extending calling-python-functions-from-c9213585 +Ref: 384c9213585 +Ref: extending/extending callingpython9213585 +Ref: 384d9213585 +Node: Extracting Parameters in Extension Functions9220268 +Ref: extending/extending extracting-parameters-in-extension-functions9220470 +Ref: 38519220470 +Ref: extending/extending parsetuple9220470 +Ref: 38399220470 +Node: Keyword Parameters for Extension Functions9223152 +Ref: extending/extending keyword-parameters-for-extension-functions9223348 +Ref: 38529223348 +Ref: extending/extending parsetupleandkeywords9223348 +Ref: 38539223348 +Node: Building Arbitrary Values9226063 +Ref: extending/extending building-arbitrary-values9226231 +Ref: 38549226231 +Ref: extending/extending buildvalue9226231 +Ref: 38559226231 +Node: Reference Counts9228317 +Ref: extending/extending refcounts9228468 +Ref: 384e9228468 +Ref: extending/extending reference-counts9228468 +Ref: 38569228468 +Node: Reference Counting in Python9233245 +Ref: extending/extending refcountsinpython9233350 +Ref: 38579233350 +Ref: extending/extending reference-counting-in-python9233350 +Ref: 38589233350 +Ref: Reference Counting in Python-Footnote-19235727 +Ref: Reference Counting in Python-Footnote-29235855 +Node: Ownership Rules9236030 +Ref: extending/extending ownership-rules9236152 +Ref: 38599236152 +Ref: extending/extending ownershiprules9236152 +Ref: 385a9236152 +Node: Thin Ice9238621 +Ref: extending/extending thin-ice9238728 +Ref: 38649238728 +Ref: extending/extending thinice9238728 +Ref: 38659238728 +Node: NULL Pointers9242010 +Ref: extending/extending null-pointers9242093 +Ref: 38679242093 +Ref: extending/extending nullpointers9242093 +Ref: 38689242093 +Ref: NULL Pointers-Footnote-19243718 +Node: Writing Extensions in C++9243859 +Ref: extending/extending cplusplus9244026 +Ref: 38699244026 +Ref: extending/extending writing-extensions-in-c9244026 +Ref: 386a9244026 +Node: Providing a C API for an Extension Module9244746 +Ref: extending/extending providing-a-c-api-for-an-extension-module9244888 +Ref: 386b9244888 +Ref: extending/extending using-capsules9244888 +Ref: cf19244888 +Node: Defining Extension Types Tutorial9253739 +Ref: extending/newtypes_tutorial doc9253942 +Ref: 386f9253942 +Ref: extending/newtypes_tutorial defining-extension-types-tutorial9253942 +Ref: 38709253942 +Ref: extending/newtypes_tutorial defining-new-types9253942 +Ref: 38719253942 +Node: The Basics9254586 +Ref: extending/newtypes_tutorial dnt-basics9254719 +Ref: 38729254719 +Ref: extending/newtypes_tutorial the-basics9254719 +Ref: 38739254719 +Ref: The Basics-Footnote-19264034 +Node: Adding data and methods to the Basic example9264104 +Ref: extending/newtypes_tutorial adding-data-and-methods-to-the-basic-example9264290 +Ref: 387e9264290 +Ref: Adding data and methods to the Basic example-Footnote-19280544 +Ref: Adding data and methods to the Basic example-Footnote-29280641 +Node: Providing finer control over data attributes9280782 +Ref: extending/newtypes_tutorial providing-finer-control-over-data-attributes9280994 +Ref: 38899280994 +Ref: Providing finer control over data attributes-Footnote-19290905 +Node: Supporting cyclic garbage collection9291289 +Ref: extending/newtypes_tutorial supporting-cyclic-garbage-collection9291480 +Ref: 388b9291480 +Ref: Supporting cyclic garbage collection-Footnote-19302119 +Node: Subclassing other types9302297 +Ref: extending/newtypes_tutorial subclassing-other-types9302435 +Ref: 388f9302435 +Node: Defining Extension Types Assorted Topics9307684 +Ref: extending/newtypes doc9307886 +Ref: 38929307886 +Ref: extending/newtypes defining-extension-types-assorted-topics9307886 +Ref: 38939307886 +Ref: extending/newtypes new-types-topics9307886 +Ref: 38949307886 +Ref: extending/newtypes dnt-type-methods9307983 +Ref: 38959307983 +Node: Finalization and De-allocation9312416 +Ref: extending/newtypes finalization-and-de-allocation9312551 +Ref: 38969312551 +Ref: Finalization and De-allocation-Footnote-19315692 +Node: Object Presentation9315741 +Ref: extending/newtypes object-presentation9315905 +Ref: 38999315905 +Node: Attribute Management9317559 +Ref: extending/newtypes attribute-management9317710 +Ref: 389d9317710 +Node: Generic Attribute Management9319264 +Ref: extending/newtypes generic-attribute-management9319392 +Ref: 38859319392 +Ref: extending/newtypes id19319392 +Ref: 389e9319392 +Node: Type-specific Attribute Management9323650 +Ref: extending/newtypes type-specific-attribute-management9323778 +Ref: 38a19323778 +Node: Object Comparison9325546 +Ref: extending/newtypes object-comparison9325703 +Ref: 38a49325703 +Node: Abstract Protocol Support9327473 +Ref: extending/newtypes abstract-protocol-support9327632 +Ref: 38a89327632 +Node: Weak Reference Support9333311 +Ref: extending/newtypes weak-reference-support9333469 +Ref: 38ae9333469 +Ref: extending/newtypes weakref-support9333469 +Ref: 163f9333469 +Node: More Suggestions9335385 +Ref: extending/newtypes more-suggestions9335509 +Ref: 38b09335509 +Node: Building C and C++ Extensions9336479 +Ref: extending/building doc9336688 +Ref: 38b29336688 +Ref: extending/building building9336688 +Ref: 384b9336688 +Ref: extending/building building-c-and-c-extensions9336688 +Ref: 38b39336688 +Ref: extending/building c PyInit_modulename9337184 +Ref: 38b49337184 +Ref: Building C and C++ Extensions-Footnote-19338545 +Node: Building C and C++ Extensions with distutils9338594 +Ref: extending/building building-c-and-c-extensions-with-distutils9338748 +Ref: 38b79338748 +Node: Distributing your extension modules9342725 +Ref: extending/building distributing9342879 +Ref: 38b99342879 +Ref: extending/building distributing-your-extension-modules9342879 +Ref: 38ba9342879 +Node: Building C and C++ Extensions on Windows9343730 +Ref: extending/windows doc9343890 +Ref: 38bc9343890 +Ref: extending/windows building-c-and-c-extensions-on-windows9343890 +Ref: 38bd9343890 +Ref: extending/windows building-on-windows9343890 +Ref: 10399343890 +Node: A Cookbook Approach9345159 +Ref: extending/windows a-cookbook-approach9345300 +Ref: 38be9345300 +Ref: extending/windows win-cookbook9345300 +Ref: 38bf9345300 +Ref: A Cookbook Approach-Footnote-19345935 +Node: Differences Between Unix and Windows9346012 +Ref: extending/windows differences-between-unix-and-windows9346184 +Ref: 38c09346184 +Ref: extending/windows dynamic-linking9346184 +Ref: 38c19346184 +Node: Using DLLs in Practice9349107 +Ref: extending/windows using-dlls-in-practice9349251 +Ref: 38c29349251 +Ref: extending/windows win-dlls9349251 +Ref: 38c39349251 +Node: Embedding the CPython runtime in a larger application9350806 +Ref: extending/index embedding-the-cpython-runtime-in-a-larger-application9350996 +Ref: 38c49350996 +Node: Embedding Python in Another Application9351438 +Ref: extending/embedding doc9351567 +Ref: 38c59351567 +Ref: extending/embedding embedding9351567 +Ref: 38c69351567 +Ref: extending/embedding embedding-python-in-another-application9351567 +Ref: 38c79351567 +Node: Very High Level Embedding9353910 +Ref: extending/embedding high-level-embedding9354064 +Ref: 38ca9354064 +Ref: extending/embedding very-high-level-embedding9354064 +Ref: 38cb9354064 +Node: Beyond Very High Level Embedding An overview9355828 +Ref: extending/embedding beyond-very-high-level-embedding-an-overview9356005 +Ref: 38cc9356005 +Ref: extending/embedding lower-level-embedding9356005 +Ref: 38cd9356005 +Node: Pure Embedding9357724 +Ref: extending/embedding id19357901 +Ref: 38ce9357901 +Ref: extending/embedding pure-embedding9357901 +Ref: 38cf9357901 +Node: Extending Embedded Python9362522 +Ref: extending/embedding extending-embedded-python9362678 +Ref: 38d29362678 +Ref: extending/embedding extending-with-embedding9362678 +Ref: 38d39362678 +Node: Embedding Python in C++9364658 +Ref: extending/embedding embedding-python-in-c9364845 +Ref: 38d49364845 +Ref: extending/embedding embeddingincplusplus9364845 +Ref: 38d59364845 +Node: Compiling and Linking under Unix-like systems9365217 +Ref: extending/embedding compiling9365370 +Ref: 38d09365370 +Ref: extending/embedding compiling-and-linking-under-unix-like-systems9365370 +Ref: 38d69365370 +Node: Python/C API Reference Manual9367548 +Ref: c-api/index doc9367708 +Ref: 38d89367708 +Ref: c-api/index c-api-index9367708 +Ref: e919367708 +Ref: c-api/index python-c-api-reference-manual9367708 +Ref: 38d99367708 +Node: Introduction<13>9368529 +Ref: c-api/intro doc9368655 +Ref: 38da9368655 +Ref: c-api/intro api-intro9368655 +Ref: 38db9368655 +Ref: c-api/intro introduction9368655 +Ref: 38dc9368655 +Node: Coding standards9370229 +Ref: c-api/intro coding-standards9370320 +Ref: 38dd9370320 +Ref: Coding standards-Footnote-19370759 +Node: Include Files9370808 +Ref: c-api/intro api-includes9370921 +Ref: 38de9370921 +Ref: c-api/intro include-files9370921 +Ref: 38df9370921 +Node: Useful macros9373442 +Ref: c-api/intro useful-macros9373573 +Ref: 38e09373573 +Ref: c-api/intro c Py_UNREACHABLE9373860 +Ref: 4239373860 +Ref: c-api/intro c Py_ABS9374252 +Ref: 38e19374252 +Ref: c-api/intro c Py_MIN9374347 +Ref: 38e29374347 +Ref: c-api/intro c Py_MAX9374461 +Ref: 38e39374461 +Ref: c-api/intro c Py_STRINGIFY9374575 +Ref: 38e49374575 +Ref: c-api/intro c Py_MEMBER_SIZE9374726 +Ref: 38e59374726 +Ref: c-api/intro c Py_CHARMASK9374869 +Ref: 38e69374869 +Ref: c-api/intro c Py_GETENV9375051 +Ref: 38e79375051 +Ref: c-api/intro c Py_UNUSED9375234 +Ref: 38e89375234 +Ref: c-api/intro c Py_DEPRECATED9375456 +Ref: 38e99375456 +Ref: c-api/intro c PyDoc_STRVAR9375730 +Ref: 38ea9375730 +Ref: c-api/intro c PyDoc_STR9376311 +Ref: 38eb9376311 +Ref: Useful macros-Footnote-19376860 +Ref: Useful macros-Footnote-29376909 +Node: Objects Types and Reference Counts9376958 +Ref: c-api/intro api-objects9377090 +Ref: 38ec9377090 +Ref: c-api/intro objects-types-and-reference-counts9377090 +Ref: 38ed9377090 +Node: Reference Counts<2>9378471 +Ref: c-api/intro api-refcounts9378575 +Ref: 38ee9378575 +Ref: c-api/intro reference-counts9378575 +Ref: 38ef9378575 +Node: Reference Count Details9382283 +Ref: c-api/intro api-refcountdetails9382362 +Ref: 38f09382362 +Ref: c-api/intro reference-count-details9382362 +Ref: 38f19382362 +Node: Types9389570 +Ref: c-api/intro api-types9389674 +Ref: 38f79389674 +Ref: c-api/intro types9389674 +Ref: 38f89389674 +Node: Exceptions<18>9390152 +Ref: c-api/intro api-exceptions9390290 +Ref: 38f99390290 +Ref: c-api/intro exceptions9390290 +Ref: 38fa9390290 +Node: Embedding Python<2>9396428 +Ref: c-api/intro api-embedding9396548 +Ref: 38fc9396548 +Ref: c-api/intro embedding-python9396548 +Ref: 38fd9396548 +Node: Debugging Builds9399747 +Ref: c-api/intro api-debugging9399844 +Ref: 39029399844 +Ref: c-api/intro debugging-builds9399844 +Ref: 39039399844 +Node: Stable Application Binary Interface9402312 +Ref: c-api/stable doc9402472 +Ref: 39049402472 +Ref: c-api/stable stable9402472 +Ref: 39059402472 +Ref: c-api/stable stable-application-binary-interface9402472 +Ref: 39069402472 +Ref: Stable Application Binary Interface-Footnote-19404449 +Node: The Very High Level Layer9404498 +Ref: c-api/veryhigh doc9404660 +Ref: 39079404660 +Ref: c-api/veryhigh the-very-high-level-layer9404660 +Ref: 39089404660 +Ref: c-api/veryhigh veryhigh9404660 +Ref: 39099404660 +Ref: c-api/veryhigh c Py_Main9405681 +Ref: 4b79405681 +Ref: c-api/veryhigh c Py_BytesMain9406640 +Ref: 1769406640 +Ref: c-api/veryhigh c PyRun_AnyFile9406809 +Ref: 390a9406809 +Ref: c-api/veryhigh c PyRun_AnyFileFlags9407034 +Ref: 390c9407034 +Ref: c-api/veryhigh c PyRun_AnyFileEx9407276 +Ref: 390d9407276 +Ref: c-api/veryhigh c PyRun_AnyFileExFlags9407505 +Ref: 390b9407505 +Ref: c-api/veryhigh c PyRun_SimpleString9408072 +Ref: 38c89408072 +Ref: c-api/veryhigh c PyRun_SimpleStringFlags9408299 +Ref: 390f9408299 +Ref: c-api/veryhigh c PyRun_SimpleFile9408982 +Ref: 38c99408982 +Ref: c-api/veryhigh c PyRun_SimpleFileEx9409213 +Ref: 39119409213 +Ref: c-api/veryhigh c PyRun_SimpleFileExFlags9409435 +Ref: 39109409435 +Ref: c-api/veryhigh c PyRun_InteractiveOne9410127 +Ref: 39129410127 +Ref: c-api/veryhigh c PyRun_InteractiveOneFlags9410345 +Ref: 39139410345 +Ref: c-api/veryhigh c PyRun_InteractiveLoop9411107 +Ref: 390e9411107 +Ref: c-api/veryhigh c PyRun_InteractiveLoopFlags9411327 +Ref: 39149411327 +Ref: c-api/veryhigh c PyOS_InputHook9411787 +Ref: 39159411787 +Ref: c-api/veryhigh c PyOS_ReadlineFunctionPointer9412270 +Ref: 96c9412270 +Ref: c-api/veryhigh c PyParser_SimpleParseString9413300 +Ref: 39169413300 +Ref: c-api/veryhigh c PyParser_SimpleParseStringFlags9413574 +Ref: 39189413574 +Ref: c-api/veryhigh c PyParser_SimpleParseStringFlagsFilename9413837 +Ref: 39179413837 +Ref: c-api/veryhigh c PyParser_SimpleParseFile9414356 +Ref: 39199414356 +Ref: c-api/veryhigh c PyParser_SimpleParseFileFlags9414600 +Ref: 391a9414600 +Ref: c-api/veryhigh c PyRun_String9414885 +Ref: 391b9414885 +Ref: c-api/veryhigh c PyRun_StringFlags9415157 +Ref: 391c9415157 +Ref: c-api/veryhigh c PyRun_File9415818 +Ref: 391d9415818 +Ref: c-api/veryhigh c PyRun_FileEx9416132 +Ref: 391f9416132 +Ref: c-api/veryhigh c PyRun_FileFlags9416432 +Ref: 39209416432 +Ref: c-api/veryhigh c PyRun_FileExFlags9416755 +Ref: 391e9416755 +Ref: c-api/veryhigh c Py_CompileString9417342 +Ref: 39219417342 +Ref: c-api/veryhigh c Py_CompileStringFlags9417607 +Ref: 39229417607 +Ref: c-api/veryhigh c Py_CompileStringObject9417901 +Ref: 39249417901 +Ref: c-api/veryhigh c Py_CompileStringExFlags9419018 +Ref: 39239419018 +Ref: c-api/veryhigh c PyEval_EvalCode9419392 +Ref: 39259419392 +Ref: c-api/veryhigh c PyEval_EvalCodeEx9419714 +Ref: 39269419714 +Ref: c-api/veryhigh c PyFrameObject9420391 +Ref: 9719420391 +Ref: c-api/veryhigh c PyEval_EvalFrame9420552 +Ref: 39279420552 +Ref: c-api/veryhigh c PyEval_EvalFrameEx9420787 +Ref: 9679420787 +Ref: c-api/veryhigh c PyEval_MergeCompilerFlags9421484 +Ref: 39289421484 +Ref: c-api/veryhigh c Py_eval_input9421676 +Ref: 39299421676 +Ref: c-api/veryhigh c Py_file_input9421834 +Ref: 392a9421834 +Ref: c-api/veryhigh c Py_single_input9422120 +Ref: 392b9422120 +Ref: c-api/veryhigh c PyCompilerFlags9422346 +Ref: 2cc9422346 +Ref: c-api/veryhigh c PyCompilerFlags cf_flags9422877 +Ref: 392c9422877 +Ref: c-api/veryhigh c PyCompilerFlags cf_feature_version9422937 +Ref: 392d9422937 +Ref: c-api/veryhigh c CO_FUTURE_DIVISION9423295 +Ref: 392e9423295 +Ref: The Very High Level Layer-Footnote-19423514 +Node: Reference Counting9423563 +Ref: c-api/refcounting doc9423708 +Ref: 392f9423708 +Ref: c-api/refcounting countingrefs9423708 +Ref: 39309423708 +Ref: c-api/refcounting reference-counting9423708 +Ref: 39319423708 +Ref: c-api/refcounting c Py_INCREF9423841 +Ref: 2659423841 +Ref: c-api/refcounting c Py_XINCREF9424063 +Ref: 2679424063 +Ref: c-api/refcounting c Py_DECREF9424237 +Ref: 2669424237 +Ref: c-api/refcounting c Py_XDECREF9425350 +Ref: 2689425350 +Ref: c-api/refcounting c Py_CLEAR9425628 +Ref: 388d9425628 +Node: Exception Handling9426773 +Ref: c-api/exceptions doc9426905 +Ref: 39329426905 +Ref: c-api/exceptions exception-handling9426905 +Ref: 39339426905 +Ref: c-api/exceptions exceptionhandling9426905 +Ref: 33779426905 +Node: Printing and clearing9429219 +Ref: c-api/exceptions printing-and-clearing9429322 +Ref: 39349429322 +Ref: c-api/exceptions c PyErr_Clear9429379 +Ref: 96b9429379 +Ref: c-api/exceptions c PyErr_PrintEx9429510 +Ref: 39359429510 +Ref: c-api/exceptions c PyErr_Print9430200 +Ref: 39369430200 +Ref: c-api/exceptions c PyErr_WriteUnraisable9430277 +Ref: 39379430277 +Node: Raising exceptions9430985 +Ref: c-api/exceptions raising-exceptions9431113 +Ref: 39389431113 +Ref: c-api/exceptions c PyErr_SetString9431353 +Ref: 383c9431353 +Ref: c-api/exceptions c PyErr_SetObject9431773 +Ref: 383e9431773 +Ref: c-api/exceptions c PyErr_Format9432006 +Ref: 7aa9432006 +Ref: c-api/exceptions c PyErr_FormatV9432457 +Ref: 7a99432457 +Ref: c-api/exceptions c PyErr_SetNone9432750 +Ref: 39399432750 +Ref: c-api/exceptions c PyErr_BadArgument9432871 +Ref: 393a9432871 +Ref: c-api/exceptions c PyErr_NoMemory9433124 +Ref: 38409433124 +Ref: c-api/exceptions c PyErr_SetFromErrno9433406 +Ref: 383d9433406 +Ref: c-api/exceptions c PyErr_SetFromErrnoWithFilenameObject9434286 +Ref: 393c9434286 +Ref: c-api/exceptions c PyErr_SetFromErrnoWithFilenameObjects9434765 +Ref: 393d9434765 +Ref: c-api/exceptions c PyErr_SetFromErrnoWithFilename9435161 +Ref: 393e9435161 +Ref: c-api/exceptions c PyErr_SetFromWindowsErr9435502 +Ref: 393f9435502 +Ref: c-api/exceptions c PyErr_SetExcFromWindowsErr9436273 +Ref: 39409436273 +Ref: c-api/exceptions c PyErr_SetFromWindowsErrWithFilename9436572 +Ref: 39419436572 +Ref: c-api/exceptions c PyErr_SetExcFromWindowsErrWithFilenameObject9436952 +Ref: 39429436952 +Ref: c-api/exceptions c PyErr_SetExcFromWindowsErrWithFilenameObjects9437301 +Ref: 39439437301 +Ref: c-api/exceptions c PyErr_SetExcFromWindowsErrWithFilename9437682 +Ref: 39449437682 +Ref: c-api/exceptions c PyErr_SetImportError9438027 +Ref: 5f89438027 +Ref: c-api/exceptions c PyErr_SyntaxLocationObject9438482 +Ref: 39459438482 +Ref: c-api/exceptions c PyErr_SyntaxLocationEx9438890 +Ref: 39469438890 +Ref: c-api/exceptions c PyErr_SyntaxLocation9439183 +Ref: 39479439183 +Ref: c-api/exceptions c PyErr_BadInternalCall9439366 +Ref: 39489439366 +Node: Issuing warnings9439664 +Ref: c-api/exceptions issuing-warnings9439799 +Ref: 39499439799 +Ref: c-api/exceptions c PyErr_WarnEx9440695 +Ref: db19440695 +Ref: c-api/exceptions c PyErr_SetImportErrorSubclass9441842 +Ref: 5cb9441842 +Ref: c-api/exceptions c PyErr_WarnExplicitObject9442190 +Ref: 394b9442190 +Ref: c-api/exceptions c PyErr_WarnExplicit9442731 +Ref: 394c9442731 +Ref: c-api/exceptions c PyErr_WarnFormat9443120 +Ref: 394d9443120 +Ref: c-api/exceptions c PyErr_ResourceWarning9443438 +Ref: 5cc9443438 +Node: Querying the error indicator9443758 +Ref: c-api/exceptions querying-the-error-indicator9443893 +Ref: 394e9443893 +Ref: c-api/exceptions c PyErr_Occurred9443964 +Ref: 383f9443964 +Ref: c-api/exceptions c PyErr_ExceptionMatches9444760 +Ref: 38fb9444760 +Ref: c-api/exceptions c PyErr_GivenExceptionMatches9445040 +Ref: 394f9445040 +Ref: c-api/exceptions c PyErr_Fetch9445439 +Ref: 96a9445439 +Ref: c-api/exceptions c PyErr_Restore9446359 +Ref: 38979446359 +Ref: c-api/exceptions c PyErr_NormalizeException9447352 +Ref: 39509447352 +Ref: c-api/exceptions c PyErr_GetExcInfo9448223 +Ref: 39519448223 +Ref: c-api/exceptions c PyErr_SetExcInfo9448977 +Ref: 39529448977 +Node: Signal Handling<2>9449787 +Ref: c-api/exceptions signal-handling9449923 +Ref: 39539449923 +Ref: c-api/exceptions c PyErr_CheckSignals9449968 +Ref: 393b9449968 +Ref: c-api/exceptions c PyErr_SetInterrupt9450634 +Ref: 39549450634 +Ref: c-api/exceptions c PySignal_SetWakeupFd9451030 +Ref: d469451030 +Node: Exception Classes9451700 +Ref: c-api/exceptions exception-classes9451825 +Ref: 39559451825 +Ref: c-api/exceptions c PyErr_NewException9451874 +Ref: c009451874 +Ref: c-api/exceptions c PyErr_NewExceptionWithDoc9452812 +Ref: bff9452812 +Node: Exception Objects9453217 +Ref: c-api/exceptions exception-objects9453349 +Ref: 39569453349 +Ref: c-api/exceptions c PyException_GetTraceback9453398 +Ref: 39579453398 +Ref: c-api/exceptions c PyException_SetTraceback9453706 +Ref: 39589453706 +Ref: c-api/exceptions c PyException_GetContext9453891 +Ref: 39599453891 +Ref: c-api/exceptions c PyException_SetContext9454263 +Ref: 395a9454263 +Ref: c-api/exceptions c PyException_GetCause9454558 +Ref: 395b9454558 +Ref: c-api/exceptions c PyException_SetCause9454875 +Ref: 395c9454875 +Node: Unicode Exception Objects9455293 +Ref: c-api/exceptions unicode-exception-objects9455425 +Ref: 395d9455425 +Ref: c-api/exceptions unicodeexceptions9455425 +Ref: 395e9455425 +Ref: c-api/exceptions c PyUnicodeDecodeError_Create9455572 +Ref: 395f9455572 +Ref: c-api/exceptions c PyUnicodeEncodeError_Create9455995 +Ref: 39609455995 +Ref: c-api/exceptions c PyUnicodeTranslateError_Create9456613 +Ref: 39619456613 +Ref: c-api/exceptions c PyUnicodeDecodeError_GetEncoding9457191 +Ref: 39629457191 +Ref: c-api/exceptions c PyUnicodeEncodeError_GetEncoding9457276 +Ref: 39639457276 +Ref: c-api/exceptions c PyUnicodeDecodeError_GetObject9457467 +Ref: 39649457467 +Ref: c-api/exceptions c PyUnicodeEncodeError_GetObject9457540 +Ref: 39659457540 +Ref: c-api/exceptions c PyUnicodeTranslateError_GetObject9457613 +Ref: 39669457613 +Ref: c-api/exceptions c PyUnicodeDecodeError_GetStart9457803 +Ref: 39679457803 +Ref: c-api/exceptions c PyUnicodeEncodeError_GetStart9457898 +Ref: 39689457898 +Ref: c-api/exceptions c PyUnicodeTranslateError_GetStart9457993 +Ref: 39699457993 +Ref: c-api/exceptions c PyUnicodeDecodeError_SetStart9458274 +Ref: 396a9458274 +Ref: c-api/exceptions c PyUnicodeEncodeError_SetStart9458368 +Ref: 396b9458368 +Ref: c-api/exceptions c PyUnicodeTranslateError_SetStart9458462 +Ref: 396c9458462 +Ref: c-api/exceptions c PyUnicodeDecodeError_GetEnd9458687 +Ref: 396d9458687 +Ref: c-api/exceptions c PyUnicodeEncodeError_GetEnd9458778 +Ref: 396e9458778 +Ref: c-api/exceptions c PyUnicodeTranslateError_GetEnd9458869 +Ref: 396f9458869 +Ref: c-api/exceptions c PyUnicodeDecodeError_SetEnd9459140 +Ref: 39709459140 +Ref: c-api/exceptions c PyUnicodeEncodeError_SetEnd9459230 +Ref: 39719459230 +Ref: c-api/exceptions c PyUnicodeTranslateError_SetEnd9459320 +Ref: 39729459320 +Ref: c-api/exceptions c PyUnicodeDecodeError_GetReason9459537 +Ref: 39739459537 +Ref: c-api/exceptions c PyUnicodeEncodeError_GetReason9459610 +Ref: 39749459610 +Ref: c-api/exceptions c PyUnicodeTranslateError_GetReason9459683 +Ref: 39759459683 +Ref: c-api/exceptions c PyUnicodeDecodeError_SetReason9459873 +Ref: 39769459873 +Ref: c-api/exceptions c PyUnicodeEncodeError_SetReason9459970 +Ref: 39779459970 +Ref: c-api/exceptions c PyUnicodeTranslateError_SetReason9460067 +Ref: 39789460067 +Node: Recursion Control9460298 +Ref: c-api/exceptions recursion-control9460432 +Ref: 39799460432 +Ref: c-api/exceptions c Py_EnterRecursiveCall9460736 +Ref: 397a9460736 +Ref: c-api/exceptions c Py_LeaveRecursiveCall9461482 +Ref: 397c9461482 +Ref: c-api/exceptions c Py_ReprEnter9462031 +Ref: 397d9462031 +Ref: c-api/exceptions c Py_ReprLeave9462799 +Ref: 397e9462799 +Node: Standard Exceptions9462990 +Ref: c-api/exceptions standard-exceptions9463126 +Ref: 397f9463126 +Ref: c-api/exceptions standardexceptions9463126 +Ref: 39809463126 +Ref: Standard Exceptions-Footnote-19476505 +Node: Standard Warning Categories9476554 +Ref: c-api/exceptions standard-warning-categories9476664 +Ref: 39819476664 +Ref: c-api/exceptions standardwarningcategories9476664 +Ref: 394a9476664 +Node: Utilities<2>9479793 +Ref: c-api/utilities doc9479929 +Ref: 39829479929 +Ref: c-api/utilities id19479929 +Ref: 39839479929 +Ref: c-api/utilities utilities9479929 +Ref: 39849479929 +Node: Operating System Utilities9480478 +Ref: c-api/sys doc9480578 +Ref: 39859480578 +Ref: c-api/sys operating-system-utilities9480578 +Ref: 39869480578 +Ref: c-api/sys os9480578 +Ref: 39879480578 +Ref: c-api/sys c PyOS_FSPath9480645 +Ref: 5cd9480645 +Ref: c-api/sys c Py_FdIsInteractive9481191 +Ref: 39889481191 +Ref: c-api/sys c PyOS_BeforeFork9481674 +Ref: 4329481674 +Ref: c-api/sys c PyOS_AfterFork_Parent9482207 +Ref: 4339482207 +Ref: c-api/sys c PyOS_AfterFork_Child9482833 +Ref: 2cd9482833 +Ref: c-api/sys c PyOS_AfterFork9483734 +Ref: 4319483734 +Ref: c-api/sys c PyOS_CheckStack9484151 +Ref: 397b9484151 +Ref: c-api/sys c PyOS_getsig9484535 +Ref: 398c9484535 +Ref: c-api/sys c PyOS_setsig9484844 +Ref: 398d9484844 +Ref: c-api/sys c Py_DecodeLocale9485213 +Ref: 43c9485213 +Ref: c-api/sys c Py_EncodeLocale9487262 +Ref: 43d9487262 +Node: System Functions9489062 +Ref: c-api/sys system-functions9489186 +Ref: 39929489186 +Ref: c-api/sys systemfunctions9489186 +Ref: 39939489186 +Ref: c-api/sys c PySys_GetObject9489485 +Ref: 39949489485 +Ref: c-api/sys c PySys_SetObject9489721 +Ref: 39959489721 +Ref: c-api/sys c PySys_ResetWarnOptions9489979 +Ref: 39969489979 +Ref: c-api/sys c PySys_AddWarnOption9490156 +Ref: 4b39490156 +Ref: c-api/sys c PySys_AddWarnOptionUnicode9490387 +Ref: 4b29490387 +Ref: c-api/sys c PySys_SetPath9490857 +Ref: 39979490857 +Ref: c-api/sys c PySys_WriteStdout9491123 +Ref: 39989491123 +Ref: c-api/sys c PySys_WriteStderr9491999 +Ref: 39999491999 +Ref: c-api/sys c PySys_FormatStdout9492169 +Ref: 399a9492169 +Ref: c-api/sys c PySys_FormatStderr9492441 +Ref: 399c9492441 +Ref: c-api/sys c PySys_AddXOption9492639 +Ref: 399d9492639 +Ref: c-api/sys c PySys_GetXOptions9492947 +Ref: 399e9492947 +Ref: c-api/sys c PySys_Audit9493227 +Ref: 31eb9493227 +Ref: c-api/sys c PySys_AddAuditHook9494356 +Ref: 31ed9494356 +Ref: System Functions-Footnote-19496207 +Node: Process Control9496256 +Ref: c-api/sys process-control9496374 +Ref: 39a09496374 +Ref: c-api/sys processcontrol9496374 +Ref: 39a19496374 +Ref: c-api/sys c Py_FatalError9496419 +Ref: 39a29496419 +Ref: c-api/sys c Py_Exit9496899 +Ref: 6129496899 +Ref: c-api/sys c Py_AtExit9497253 +Ref: 39a39497253 +Node: Importing Modules<2>9497903 +Ref: c-api/import doc9498029 +Ref: 39a49498029 +Ref: c-api/import importing9498029 +Ref: 39a59498029 +Ref: c-api/import importing-modules9498029 +Ref: 39a69498029 +Ref: c-api/import c PyImport_ImportModule9498078 +Ref: c739498078 +Ref: c-api/import c PyImport_ImportModuleNoBlock9499109 +Ref: 9c19499109 +Ref: c-api/import c PyImport_ImportModuleEx9499597 +Ref: b219499597 +Ref: c-api/import c PyImport_ImportModuleLevelObject9500325 +Ref: 39a79500325 +Ref: c-api/import c PyImport_ImportModuleLevel9501085 +Ref: b209501085 +Ref: c-api/import c PyImport_Import9501507 +Ref: d5f9501507 +Ref: c-api/import c PyImport_ReloadModule9502028 +Ref: 39a89502028 +Ref: c-api/import c PyImport_AddModuleObject9502290 +Ref: 39a99502290 +Ref: c-api/import c PyImport_AddModule9503117 +Ref: 38609503117 +Ref: c-api/import c PyImport_ExecCodeModule9503353 +Ref: 39aa9503353 +Ref: c-api/import c PyImport_ExecCodeModuleEx9505134 +Ref: 39ab9505134 +Ref: c-api/import c PyImport_ExecCodeModuleObject9505522 +Ref: 39ad9505522 +Ref: c-api/import c PyImport_ExecCodeModuleWithPathnames9505958 +Ref: 39ac9505958 +Ref: c-api/import c PyImport_GetMagicNumber9506602 +Ref: b1f9506602 +Ref: c-api/import c PyImport_GetMagicTag9506957 +Ref: 39ae9506957 +Ref: c-api/import c PyImport_GetModuleDict9507266 +Ref: 39af9507266 +Ref: c-api/import c PyImport_GetModule9507504 +Ref: 4219507504 +Ref: c-api/import c PyImport_GetImporter9507853 +Ref: 39b09507853 +Ref: c-api/import c _PyImport_Init9508515 +Ref: 39b19508515 +Ref: c-api/import c PyImport_Cleanup9508618 +Ref: 39b29508618 +Ref: c-api/import c _PyImport_Fini9508714 +Ref: 39b39508714 +Ref: c-api/import c PyImport_ImportFrozenModuleObject9508815 +Ref: 39b49508815 +Ref: c-api/import c PyImport_ImportFrozenModule9509419 +Ref: 39b59509419 +Ref: c-api/import c _frozen9509626 +Ref: 39b69509626 +Ref: c-api/import c PyImport_FrozenModules9510038 +Ref: 1fe09510038 +Ref: c-api/import c PyImport_AppendInittab9510442 +Ref: 38489510442 +Ref: c-api/import c _inittab9510985 +Ref: 39b89510985 +Ref: c-api/import c PyImport_ExtendInittab9511640 +Ref: 39b79511640 +Ref: Importing Modules<2>-Footnote-19512260 +Node: Data marshalling support9512309 +Ref: c-api/marshal doc9512457 +Ref: 39b99512457 +Ref: c-api/marshal data-marshalling-support9512457 +Ref: 39ba9512457 +Ref: c-api/marshal marshalling-utils9512457 +Ref: 39bb9512457 +Ref: c-api/marshal c PyMarshal_WriteLongToFile9513204 +Ref: 39bc9513204 +Ref: c-api/marshal c PyMarshal_WriteObjectToFile9513522 +Ref: 39bd9513522 +Ref: c-api/marshal c PyMarshal_WriteObjectToString9513719 +Ref: 39be9513719 +Ref: c-api/marshal c PyMarshal_ReadLongFromFile9514046 +Ref: 39bf9514046 +Ref: c-api/marshal c PyMarshal_ReadShortFromFile9514403 +Ref: 39c09514403 +Ref: c-api/marshal c PyMarshal_ReadObjectFromFile9514762 +Ref: 39c19514762 +Ref: c-api/marshal c PyMarshal_ReadLastObjectFromFile9515106 +Ref: 39c29515106 +Ref: c-api/marshal c PyMarshal_ReadObjectFromString9515883 +Ref: 39c39515883 +Node: Parsing arguments and building values9516291 +Ref: c-api/arg doc9516451 +Ref: 39c49516451 +Ref: c-api/arg arg-parsing9516451 +Ref: 2d09516451 +Ref: c-api/arg parsing-arguments-and-building-values9516451 +Ref: 39c59516451 +Node: Parsing arguments<3>9517122 +Ref: c-api/arg parsing-arguments9517240 +Ref: 39c79517240 +Node: Strings and buffers9517970 +Ref: c-api/arg strings-and-buffers9518065 +Ref: 39c89518065 +Node: Numbers<2>9530253 +Ref: c-api/arg numbers9530370 +Ref: 39ca9530370 +Node: Other objects9532488 +Ref: c-api/arg other-objects9532599 +Ref: 39cc9532599 +Ref: c-api/arg o-ampersand9533351 +Ref: 39cd9533351 +Node: API Functions9537893 +Ref: c-api/arg api-functions9537985 +Ref: 39ce9537985 +Ref: c-api/arg c PyArg_ParseTuple9538030 +Ref: 26a9538030 +Ref: c-api/arg c PyArg_VaParse9538324 +Ref: 39cf9538324 +Ref: c-api/arg c PyArg_ParseTupleAndKeywords9538553 +Ref: 5ca9538553 +Ref: c-api/arg c PyArg_VaParseTupleAndKeywords9539154 +Ref: dda9539154 +Ref: c-api/arg c PyArg_ValidateKeywordArguments9539452 +Ref: 39d09539452 +Ref: c-api/arg c PyArg_Parse9539755 +Ref: 39c69539755 +Ref: c-api/arg c PyArg_UnpackTuple9540359 +Ref: e469540359 +Node: Building values9542290 +Ref: c-api/arg building-values9542408 +Ref: 39d19542408 +Ref: c-api/arg c Py_BuildValue9542457 +Ref: 2ce9542457 +Ref: c-api/arg c Py_VaBuildValue9550009 +Ref: 39d29550009 +Node: String conversion and formatting9550263 +Ref: c-api/conversion doc9550409 +Ref: 39d39550409 +Ref: c-api/conversion string-conversion9550409 +Ref: 39d49550409 +Ref: c-api/conversion string-conversion-and-formatting9550409 +Ref: 39d59550409 +Ref: c-api/conversion c PyOS_snprintf9550550 +Ref: e499550550 +Ref: c-api/conversion c PyOS_vsnprintf9550811 +Ref: e4a9550811 +Ref: c-api/conversion c PyOS_string_to_double9552523 +Ref: c239552523 +Ref: c-api/conversion c PyOS_double_to_string9554243 +Ref: 39d69554243 +Ref: c-api/conversion c PyOS_stricmp9555835 +Ref: 39d79555835 +Ref: c-api/conversion c PyOS_strnicmp9556043 +Ref: 39d89556043 +Node: Reflection9556281 +Ref: c-api/reflection doc9556426 +Ref: 39d99556426 +Ref: c-api/reflection id19556426 +Ref: 39da9556426 +Ref: c-api/reflection reflection9556426 +Ref: 39db9556426 +Ref: c-api/reflection c PyEval_GetBuiltins9556461 +Ref: 39dc9556461 +Ref: c-api/reflection c PyEval_GetLocals9556705 +Ref: 39dd9556705 +Ref: c-api/reflection c PyEval_GetGlobals9556929 +Ref: 39de9556929 +Ref: c-api/reflection c PyEval_GetFrame9557155 +Ref: 39df9557155 +Ref: c-api/reflection c PyFrame_GetLineNumber9557352 +Ref: ceb9557352 +Ref: c-api/reflection c PyEval_GetFuncName9557484 +Ref: 39e09557484 +Ref: c-api/reflection c PyEval_GetFuncDesc9557664 +Ref: 39e19557664 +Node: Codec registry and support functions9558042 +Ref: c-api/codec doc9558146 +Ref: 39e29558146 +Ref: c-api/codec codec-registry9558146 +Ref: 5ee9558146 +Ref: c-api/codec codec-registry-and-support-functions9558146 +Ref: 39e39558146 +Ref: c-api/codec c PyCodec_Register9558233 +Ref: 39e49558233 +Ref: c-api/codec c PyCodec_KnownEncoding9558510 +Ref: 39e59558510 +Ref: c-api/codec c PyCodec_Encode9558720 +Ref: 39e69558720 +Ref: c-api/codec c PyCodec_Decode9559201 +Ref: 39e79559201 +Node: Codec lookup API9559766 +Ref: c-api/codec codec-lookup-api9559912 +Ref: 39e89559912 +Ref: c-api/codec c PyCodec_Encoder9560230 +Ref: 39e99560230 +Ref: c-api/codec c PyCodec_Decoder9560388 +Ref: 39ea9560388 +Ref: c-api/codec c PyCodec_IncrementalEncoder9560545 +Ref: 39eb9560545 +Ref: c-api/codec c PyCodec_IncrementalDecoder9560766 +Ref: 39ec9560766 +Ref: c-api/codec c PyCodec_StreamReader9560987 +Ref: 39ed9560987 +Ref: c-api/codec c PyCodec_StreamWriter9561223 +Ref: 39ee9561223 +Node: Registry API for Unicode encoding error handlers9561459 +Ref: c-api/codec registry-api-for-unicode-encoding-error-handlers9561605 +Ref: 39ef9561605 +Ref: c-api/codec c PyCodec_RegisterError9561720 +Ref: 39f09561720 +Ref: c-api/codec c PyCodec_LookupError9562812 +Ref: 39f19562812 +Ref: c-api/codec c PyCodec_StrictErrors9563121 +Ref: 39f29563121 +Ref: c-api/codec c PyCodec_IgnoreErrors9563248 +Ref: 39f39563248 +Ref: c-api/codec c PyCodec_ReplaceErrors9563407 +Ref: 39f49563407 +Ref: c-api/codec c PyCodec_XMLCharRefReplaceErrors9563577 +Ref: 39f59563577 +Ref: c-api/codec c PyCodec_BackslashReplaceErrors9563768 +Ref: 39f69563768 +Ref: c-api/codec c PyCodec_NameReplaceErrors9563975 +Ref: 7a89563975 +Node: Abstract Objects Layer9564173 +Ref: c-api/abstract doc9564313 +Ref: 39f79564313 +Ref: c-api/abstract abstract9564313 +Ref: 38a99564313 +Ref: c-api/abstract abstract-objects-layer9564313 +Ref: 39f89564313 +Node: Object Protocol9565024 +Ref: c-api/object doc9565122 +Ref: 39f99565122 +Ref: c-api/object object9565122 +Ref: 39fa9565122 +Ref: c-api/object object-protocol9565122 +Ref: 39fb9565122 +Ref: c-api/object c Py_NotImplemented9565167 +Ref: 39fc9565167 +Ref: c-api/object c Py_RETURN_NOTIMPLEMENTED9565343 +Ref: 39fd9565343 +Ref: c-api/object c PyObject_Print9565554 +Ref: 39fe9565554 +Ref: c-api/object c PyObject_HasAttr9565929 +Ref: 39ff9565929 +Ref: c-api/object c PyObject_HasAttrString9566428 +Ref: 3a019566428 +Ref: c-api/object c PyObject_GetAttr9566990 +Ref: 3a009566990 +Ref: c-api/object c PyObject_GetAttrString9567323 +Ref: 385b9567323 +Ref: c-api/object c PyObject_GenericGetAttr9567664 +Ref: 3a029567664 +Ref: c-api/object c PyObject_SetAttr9568287 +Ref: 3a039568287 +Ref: c-api/object c PyObject_SetAttrString9568783 +Ref: 3a059568783 +Ref: c-api/object c PyObject_GenericSetAttr9569293 +Ref: 3a079569293 +Ref: c-api/object c PyObject_DelAttr9569950 +Ref: 3a049569950 +Ref: c-api/object c PyObject_DelAttrString9570192 +Ref: 3a069570192 +Ref: c-api/object c PyObject_GenericGetDict9570452 +Ref: 3a089570452 +Ref: c-api/object c PyObject_GenericSetDict9570728 +Ref: 3a099570728 +Ref: c-api/object c PyObject_RichCompare9571003 +Ref: 38a69571003 +Ref: c-api/object c PyObject_RichCompareBool9571632 +Ref: 38a79571632 +Ref: c-api/object c PyObject_Repr9572405 +Ref: 9689572405 +Ref: c-api/object c PyObject_ASCII9572906 +Ref: 3a0a9572906 +Ref: c-api/object c PyObject_Str9573390 +Ref: 9699573390 +Ref: c-api/object c PyObject_Bytes9573940 +Ref: 3a0b9573940 +Ref: c-api/object c PyObject_IsSubclass9574371 +Ref: 79e9574371 +Ref: c-api/object c PyObject_IsInstance9575347 +Ref: 79d9575347 +Ref: c-api/object c PyCallable_Check9576318 +Ref: 3a0c9576318 +Ref: c-api/object c PyObject_Call9576519 +Ref: 38509576519 +Ref: c-api/object c PyObject_CallObject9577164 +Ref: 384f9577164 +Ref: c-api/object c PyObject_CallFunction9577636 +Ref: 2cf9577636 +Ref: c-api/object c PyObject_CallMethod9578451 +Ref: 3a0e9578451 +Ref: c-api/object c PyObject_CallFunctionObjArgs9579330 +Ref: 3a0d9579330 +Ref: c-api/object c PyObject_CallMethodObjArgs9579864 +Ref: 3a0f9579864 +Ref: c-api/object c _PyObject_Vectorcall9580409 +Ref: 3a109580409 +Ref: c-api/object c PY_VECTORCALL_ARGUMENTS_OFFSET9581831 +Ref: 3a139581831 +Ref: c-api/object c PyVectorcall_NARGS9582460 +Ref: 3a129582460 +Ref: c-api/object c _PyObject_FastCallDict9582715 +Ref: 3a149582715 +Ref: c-api/object c PyObject_Hash9583632 +Ref: 3a159583632 +Ref: c-api/object c PyObject_HashNotImplemented9583973 +Ref: d319583973 +Ref: c-api/object c PyObject_IsTrue9584320 +Ref: 3a169584320 +Ref: c-api/object c PyObject_Not9584562 +Ref: 3a179584562 +Ref: c-api/object c PyObject_Type9584796 +Ref: 3a189584796 +Ref: c-api/object c PyObject_TypeCheck9585434 +Ref: 38b19585434 +Ref: c-api/object c PyObject_Size9585632 +Ref: 3a199585632 +Ref: c-api/object c PyObject_Length9585687 +Ref: 3a1a9585687 +Ref: c-api/object c PyObject_LengthHint9585999 +Ref: 9289585999 +Ref: c-api/object c PyObject_GetItem9586436 +Ref: 38f59586436 +Ref: c-api/object c PyObject_SetItem9586701 +Ref: 38f39586701 +Ref: c-api/object c PyObject_DelItem9587048 +Ref: 3a1b9587048 +Ref: c-api/object c PyObject_Dir9587282 +Ref: 3a1c9587282 +Ref: c-api/object c PyObject_GetIter9587819 +Ref: 3a1d9587819 +Ref: Object Protocol-Footnote-19588227 +Ref: Object Protocol-Footnote-29588276 +Node: Number Protocol9588325 +Ref: c-api/number doc9588449 +Ref: 3a1e9588449 +Ref: c-api/number number9588449 +Ref: 3a1f9588449 +Ref: c-api/number number-protocol9588449 +Ref: 3a209588449 +Ref: c-api/number c PyNumber_Check9588494 +Ref: 26b9588494 +Ref: c-api/number c PyNumber_Add9588744 +Ref: 3a219588744 +Ref: c-api/number c PyNumber_Subtract9588994 +Ref: 3a229588994 +Ref: c-api/number c PyNumber_Multiply9589255 +Ref: 3a239589255 +Ref: c-api/number c PyNumber_MatrixMultiply9589515 +Ref: 7ae9589515 +Ref: c-api/number c PyNumber_FloorDivide9589830 +Ref: 3a249589830 +Ref: c-api/number c PyNumber_TrueDivide9590095 +Ref: 3a259590095 +Ref: c-api/number c PyNumber_Remainder9590586 +Ref: 3a269590586 +Ref: c-api/number c PyNumber_Divmod9590856 +Ref: 3a279590856 +Ref: c-api/number c PyNumber_Power9591130 +Ref: 3a289591130 +Ref: c-api/number c PyNumber_Negative9591593 +Ref: 3a299591593 +Ref: c-api/number c PyNumber_Positive9591824 +Ref: 3a2a9591824 +Ref: c-api/number c PyNumber_Absolute9592034 +Ref: 3a2b9592034 +Ref: c-api/number c PyNumber_Invert9592264 +Ref: 3a2c9592264 +Ref: c-api/number c PyNumber_Lshift9592501 +Ref: 3a2d9592501 +Ref: c-api/number c PyNumber_Rshift9592772 +Ref: 3a2e9592772 +Ref: c-api/number c PyNumber_And9593044 +Ref: 3a2f9593044 +Ref: c-api/number c PyNumber_Xor9593309 +Ref: 3a309593309 +Ref: c-api/number c PyNumber_Or9593582 +Ref: 3a319593582 +Ref: c-api/number c PyNumber_InPlaceAdd9593845 +Ref: 3a329593845 +Ref: c-api/number c PyNumber_InPlaceSubtract9594174 +Ref: 3a339594174 +Ref: c-api/number c PyNumber_InPlaceMultiply9594514 +Ref: 3a349594514 +Ref: c-api/number c PyNumber_InPlaceMatrixMultiply9594853 +Ref: 7af9594853 +Ref: c-api/number c PyNumber_InPlaceFloorDivide9595237 +Ref: 3a359595237 +Ref: c-api/number c PyNumber_InPlaceTrueDivide9595588 +Ref: 3a369595588 +Ref: c-api/number c PyNumber_InPlaceRemainder9596148 +Ref: 3a379596148 +Ref: c-api/number c PyNumber_InPlacePower9596487 +Ref: 3a389596487 +Ref: c-api/number c PyNumber_InPlaceLshift9597082 +Ref: 3a399597082 +Ref: c-api/number c PyNumber_InPlaceRshift9597432 +Ref: 3a3a9597432 +Ref: c-api/number c PyNumber_InPlaceAnd9597783 +Ref: 3a3b9597783 +Ref: c-api/number c PyNumber_InPlaceXor9598127 +Ref: 3a3c9598127 +Ref: c-api/number c PyNumber_InPlaceOr9598479 +Ref: 3a3d9598479 +Ref: c-api/number c PyNumber_Long9598821 +Ref: 26c9598821 +Ref: c-api/number c PyNumber_Float9599076 +Ref: 26d9599076 +Ref: c-api/number c PyNumber_Index9599331 +Ref: 3a3e9599331 +Ref: c-api/number c PyNumber_ToBase9599559 +Ref: 3a3f9599559 +Ref: c-api/number c PyNumber_AsSsize_t9600020 +Ref: 3a409600020 +Ref: c-api/number c PyIndex_Check9600749 +Ref: 3a419600749 +Node: Sequence Protocol9600977 +Ref: c-api/sequence doc9601102 +Ref: 3a429601102 +Ref: c-api/sequence sequence9601102 +Ref: 3a439601102 +Ref: c-api/sequence sequence-protocol9601102 +Ref: 3a449601102 +Ref: c-api/sequence c PySequence_Check9601151 +Ref: 3a459601151 +Ref: c-api/sequence c PySequence_Size9601561 +Ref: 3a469601561 +Ref: c-api/sequence c PySequence_Length9601618 +Ref: 3a479601618 +Ref: c-api/sequence c PySequence_Concat9601831 +Ref: 3a489601831 +Ref: c-api/sequence c PySequence_Repeat9602097 +Ref: 3a499602097 +Ref: c-api/sequence c PySequence_InPlaceConcat9602389 +Ref: 3a4a9602389 +Ref: c-api/sequence c PySequence_InPlaceRepeat9602735 +Ref: 3a4b9602735 +Ref: c-api/sequence c PySequence_GetItem9603096 +Ref: 38f69603096 +Ref: c-api/sequence c PySequence_GetSlice9603337 +Ref: 3a4c9603337 +Ref: c-api/sequence c PySequence_SetItem9603639 +Ref: 38f29603639 +Ref: c-api/sequence c PySequence_DelItem9604138 +Ref: 3a4d9604138 +Ref: c-api/sequence c PySequence_SetSlice9604350 +Ref: 3a4e9604350 +Ref: c-api/sequence c PySequence_DelSlice9604628 +Ref: 3a4f9604628 +Ref: c-api/sequence c PySequence_Count9604896 +Ref: 3a509604896 +Ref: c-api/sequence c PySequence_Contains9605220 +Ref: 3a519605220 +Ref: c-api/sequence c PySequence_Index9605515 +Ref: 3a529605515 +Ref: c-api/sequence c PySequence_List9605770 +Ref: 3a539605770 +Ref: c-api/sequence c PySequence_Tuple9606085 +Ref: 3a549606085 +Ref: c-api/sequence c PySequence_Fast9606493 +Ref: 3a559606493 +Ref: c-api/sequence c PySequence_Fast_GET_SIZE9607177 +Ref: 3a579607177 +Ref: c-api/sequence c PySequence_Fast_GET_ITEM9607573 +Ref: 3a589607573 +Ref: c-api/sequence c PySequence_Fast_ITEMS9607867 +Ref: 3a599607867 +Ref: c-api/sequence c PySequence_ITEM9608271 +Ref: 3a5a9608271 +Node: Mapping Protocol9608620 +Ref: c-api/mapping doc9608747 +Ref: 3a5b9608747 +Ref: c-api/mapping mapping9608747 +Ref: 3a5c9608747 +Ref: c-api/mapping mapping-protocol9608747 +Ref: 3a5d9608747 +Ref: c-api/mapping c PyMapping_Check9608904 +Ref: 3a5e9608904 +Ref: c-api/mapping c PyMapping_Size9609283 +Ref: 3a5f9609283 +Ref: c-api/mapping c PyMapping_Length9609339 +Ref: 3a609609339 +Ref: c-api/mapping c PyMapping_GetItemString9609546 +Ref: 3a619609546 +Ref: c-api/mapping c PyMapping_SetItemString9609877 +Ref: 3a629609877 +Ref: c-api/mapping c PyMapping_DelItem9610240 +Ref: 3a639610240 +Ref: c-api/mapping c PyMapping_DelItemString9610528 +Ref: 3a649610528 +Ref: c-api/mapping c PyMapping_HasKey9610781 +Ref: 3a659610781 +Ref: c-api/mapping c PyMapping_HasKeyString9611224 +Ref: 3a669611224 +Ref: c-api/mapping c PyMapping_Keys9611736 +Ref: 42c9611736 +Ref: c-api/mapping c PyMapping_Values9612007 +Ref: 42d9612007 +Ref: c-api/mapping c PyMapping_Items9612282 +Ref: 42e9612282 +Node: Iterator Protocol9612616 +Ref: c-api/iter doc9612741 +Ref: 3a679612741 +Ref: c-api/iter iterator9612741 +Ref: 3a689612741 +Ref: c-api/iter iterator-protocol9612741 +Ref: 3a699612741 +Ref: c-api/iter c PyIter_Check9612856 +Ref: 3a6a9612856 +Ref: c-api/iter c PyIter_Next9612972 +Ref: 3a6b9612972 +Node: Buffer Protocol9613938 +Ref: c-api/buffer doc9614066 +Ref: 3a6c9614066 +Ref: c-api/buffer buffer-protocol9614066 +Ref: 3a6d9614066 +Ref: c-api/buffer bufferobjects9614066 +Ref: 12919614066 +Node: Buffer structure9616588 +Ref: c-api/buffer buffer-structure9616685 +Ref: 3a6f9616685 +Ref: c-api/buffer id19616685 +Ref: 3a709616685 +Ref: c-api/buffer c Py_buffer9617760 +Ref: b1b9617760 +Ref: c-api/buffer c Py_buffer buf9617783 +Ref: 3a729617783 +Ref: c-api/buffer c Py_buffer obj9618244 +Ref: 3a749618244 +Ref: c-api/buffer c Py_buffer len9618825 +Ref: 3a779618825 +Ref: c-api/buffer c Py_buffer readonly9619425 +Ref: 3a7a9619425 +Ref: c-api/buffer c Py_buffer itemsize9619595 +Ref: 3a7b9619595 +Ref: c-api/buffer c Py_buffer format9620512 +Ref: 3a7c9620512 +Ref: c-api/buffer c Py_buffer ndim9620839 +Ref: 3a7f9620839 +Ref: c-api/buffer c Py_buffer shape9621437 +Ref: 3a7e9621437 +Ref: c-api/buffer c Py_buffer strides9621984 +Ref: 3a739621984 +Ref: c-api/buffer c Py_buffer suboffsets9622504 +Ref: 3a809622504 +Ref: c-api/buffer c Py_buffer internal9623391 +Ref: 3a829623391 +Node: Buffer request types9623766 +Ref: c-api/buffer buffer-request-types9623886 +Ref: 3a839623886 +Ref: c-api/buffer id29623886 +Ref: 3a849623886 +Node: request-independent fields9624494 +Ref: c-api/buffer request-independent-fields9624601 +Ref: 3a859624601 +Node: readonly format9624869 +Ref: c-api/buffer readonly-format9625009 +Ref: 3a869625009 +Ref: c-api/buffer c PyBUF_WRITABLE9625060 +Ref: 3a799625060 +Ref: c-api/buffer c PyBUF_FORMAT9625393 +Ref: 3a7d9625393 +Node: shape strides suboffsets9625974 +Ref: c-api/buffer shape-strides-suboffsets9626107 +Ref: 3a879626107 +Ref: c-api/buffer c PyBUF_INDIRECT9626644 +Ref: 3a889626644 +Ref: c-api/buffer c PyBUF_STRIDES9626835 +Ref: 3a899626835 +Ref: c-api/buffer c PyBUF_ND9627017 +Ref: 3a8a9627017 +Ref: c-api/buffer c PyBUF_SIMPLE9627208 +Ref: 3a789627208 +Node: contiguity requests9627310 +Ref: c-api/buffer contiguity-requests9627445 +Ref: 3a8b9627445 +Ref: c-api/buffer c PyBUF_C_CONTIGUOUS9628054 +Ref: 3a8c9628054 +Ref: c-api/buffer c PyBUF_F_CONTIGUOUS9628307 +Ref: 3a8d9628307 +Ref: c-api/buffer c PyBUF_ANY_CONTIGUOUS9628562 +Ref: 3a8e9628562 +Node: compound requests9628957 +Ref: c-api/buffer compound-requests9629059 +Ref: 3a8f9629059 +Ref: c-api/buffer c PyBUF_FULL9629934 +Ref: 3a919629934 +Ref: c-api/buffer c PyBUF_FULL_RO9630264 +Ref: 3a929630264 +Ref: c-api/buffer c PyBUF_RECORDS9630591 +Ref: 3a939630591 +Ref: c-api/buffer c PyBUF_RECORDS_RO9630921 +Ref: 3a949630921 +Ref: c-api/buffer c PyBUF_STRIDED9631245 +Ref: 3a959631245 +Ref: c-api/buffer c PyBUF_STRIDED_RO9631576 +Ref: 3a969631576 +Ref: c-api/buffer c PyBUF_CONTIG9631900 +Ref: 3a979631900 +Ref: c-api/buffer c PyBUF_CONTIG_RO9632231 +Ref: 3a989632231 +Node: Complex arrays9632424 +Ref: c-api/buffer complex-arrays9632552 +Ref: 3a819632552 +Node: NumPy-style shape and strides9632758 +Ref: c-api/buffer numpy-style-shape-and-strides9632885 +Ref: 3a999632885 +Node: PIL-style shape strides and suboffsets9634765 +Ref: c-api/buffer pil-style-shape-strides-and-suboffsets9634892 +Ref: 3a9a9634892 +Node: Buffer-related functions9636118 +Ref: c-api/buffer buffer-related-functions9636217 +Ref: 3a9b9636217 +Ref: c-api/buffer c PyObject_CheckBuffer9636286 +Ref: 3a9c9636286 +Ref: c-api/buffer c PyObject_GetBuffer9636571 +Ref: d259636571 +Ref: c-api/buffer c PyBuffer_Release9637519 +Ref: d549637519 +Ref: c-api/buffer c PyBuffer_SizeFromFormat9637900 +Ref: 3a9d9637900 +Ref: c-api/buffer c PyBuffer_IsContiguous9638082 +Ref: 3a909638082 +Ref: c-api/buffer c PyBuffer_GetPointer9638421 +Ref: 3a9e9638421 +Ref: c-api/buffer c PyBuffer_FromContiguous9638660 +Ref: 3a9f9638660 +Ref: c-api/buffer c PyBuffer_ToContiguous9638968 +Ref: 3aa09638968 +Ref: c-api/buffer c PyBuffer_FillContiguousStrides9639378 +Ref: 3aa19639378 +Ref: c-api/buffer c PyBuffer_FillInfo9639766 +Ref: 3a769639766 +Node: Old Buffer Protocol9640743 +Ref: c-api/objbuffer doc9640845 +Ref: 3aa29640845 +Ref: c-api/objbuffer old-buffer-protocol9640845 +Ref: 3aa39640845 +Ref: c-api/objbuffer c PyObject_AsCharBuffer9641607 +Ref: 3aa49641607 +Ref: c-api/objbuffer c PyObject_AsReadBuffer9642081 +Ref: 3aa59642081 +Ref: c-api/objbuffer c PyObject_CheckReadBuffer9642542 +Ref: 3aa69642542 +Ref: c-api/objbuffer c PyObject_AsWriteBuffer9642987 +Ref: 3aa79642987 +Node: Concrete Objects Layer9643419 +Ref: c-api/concrete doc9643586 +Ref: 3aa89643586 +Ref: c-api/concrete concrete9643586 +Ref: 3aa99643586 +Ref: c-api/concrete concrete-objects-layer9643586 +Ref: 3aaa9643586 +Node: Fundamental Objects9644571 +Ref: c-api/concrete fundamental9644673 +Ref: 3aac9644673 +Ref: c-api/concrete fundamental-objects9644673 +Ref: 3aad9644673 +Node: Type Objects<2>9644870 +Ref: c-api/type doc9644965 +Ref: 3aae9644965 +Ref: c-api/type type-objects9644965 +Ref: 3aaf9644965 +Ref: c-api/type typeobjects9644965 +Ref: 3ab09644965 +Ref: c-api/type c PyTypeObject9645008 +Ref: 2d69645008 +Ref: c-api/type c PyType_Type9645104 +Ref: 3ab19645104 +Ref: c-api/type c PyType_Check9645259 +Ref: 3ab29645259 +Ref: c-api/type c PyType_CheckExact9645473 +Ref: 3ab39645473 +Ref: c-api/type c PyType_ClearCache9645666 +Ref: 3ab49645666 +Ref: c-api/type c PyType_GetFlags9645789 +Ref: 3ab59645789 +Ref: c-api/type c PyType_Modified9646283 +Ref: 3ab79646283 +Ref: c-api/type c PyType_HasFeature9646540 +Ref: 3ab89646540 +Ref: c-api/type c PyType_IS_GC9646731 +Ref: 3ab99646731 +Ref: c-api/type c PyType_IsSubtype9646923 +Ref: 3aba9646923 +Ref: c-api/type c PyType_GenericAlloc9647283 +Ref: 2709647283 +Ref: c-api/type c PyType_GenericNew9647627 +Ref: 387d9647627 +Ref: c-api/type c PyType_Ready9647919 +Ref: 38829647919 +Ref: c-api/type c PyType_GetSlot9648258 +Ref: 9249648258 +Node: Creating Heap-Allocated Types9648835 +Ref: c-api/type creating-heap-allocated-types9648916 +Ref: 3abb9648916 +Ref: c-api/type c PyType_FromSpecWithBases9649073 +Ref: 3abd9649073 +Ref: c-api/type c PyType_FromSpec9649732 +Ref: 2d19649732 +Ref: c-api/type c PyType_Spec9649895 +Ref: 3abf9649895 +Ref: c-api/type c PyType_Spec PyType_Spec name9649966 +Ref: 3ac09649966 +Ref: c-api/type c PyType_Spec PyType_Spec basicsize9650100 +Ref: 3ac19650100 +Ref: c-api/type c PyType_Spec PyType_Spec itemsize9650146 +Ref: 3ac29650146 +Ref: c-api/type c PyType_Spec PyType_Spec flags9650346 +Ref: 3ac39650346 +Ref: c-api/type c PyType_Spec PyType_Spec slots9650592 +Ref: 3ac49650592 +Ref: c-api/type c PyType_Slot9650765 +Ref: 3ac59650765 +Ref: c-api/type c PyType_Slot PyType_Slot slot9650896 +Ref: 3ac69650896 +Ref: c-api/type c PyType_Slot PyType_Slot pfunc9652257 +Ref: 3ad19652257 +Node: The None Object9652436 +Ref: c-api/none doc9652531 +Ref: 3ad29652531 +Ref: c-api/none noneobject9652531 +Ref: 3ad39652531 +Ref: c-api/none the-none-object9652531 +Ref: 3ad49652531 +Ref: c-api/none c Py_None9652850 +Ref: 38449652850 +Ref: c-api/none c Py_RETURN_NONE9653069 +Ref: dd69653069 +Node: Numeric Objects9653256 +Ref: c-api/concrete numeric-objects9653383 +Ref: 3ad59653383 +Ref: c-api/concrete numericobjects9653383 +Ref: 3ad69653383 +Node: Integer Objects9653532 +Ref: c-api/long doc9653623 +Ref: 3ad79653623 +Ref: c-api/long integer-objects9653623 +Ref: 3ad89653623 +Ref: c-api/long longobjects9653623 +Ref: 3ad99653623 +Ref: c-api/long c PyLongObject9653915 +Ref: 3ada9653915 +Ref: c-api/long c PyLong_Type9654025 +Ref: 3adb9654025 +Ref: c-api/long c PyLong_Check9654226 +Ref: 3adc9654226 +Ref: c-api/long c PyLong_CheckExact9654386 +Ref: 3add9654386 +Ref: c-api/long c PyLong_FromLong9654556 +Ref: 38419654556 +Ref: c-api/long c PyLong_FromUnsignedLong9655093 +Ref: 3ade9655093 +Ref: c-api/long c PyLong_FromSsize_t9655306 +Ref: 3adf9655306 +Ref: c-api/long c PyLong_FromSize_t9655508 +Ref: 3ae09655508 +Ref: c-api/long c PyLong_FromLongLong9655701 +Ref: 3ae19655701 +Ref: c-api/long c PyLong_FromUnsignedLongLong9655902 +Ref: 3ae29655902 +Ref: c-api/long c PyLong_FromDouble9656139 +Ref: 3ae39656139 +Ref: c-api/long c PyLong_FromString9656339 +Ref: 3ae49656339 +Ref: c-api/long c PyLong_FromUnicode9657203 +Ref: 3ae59657203 +Ref: c-api/long c PyLong_FromUnicodeObject9657603 +Ref: 3ae69657603 +Ref: c-api/long c PyLong_FromVoidPtr9657839 +Ref: 3ae79657839 +Ref: c-api/long c PyLong_AsLong9658087 +Ref: 2699658087 +Ref: c-api/long c PyLong_AsLongAndOverflow9658739 +Ref: bfd9658739 +Ref: c-api/long c PyLong_AsLongLong9659633 +Ref: 3ae99659633 +Ref: c-api/long c PyLong_AsLongLongAndOverflow9660304 +Ref: bfc9660304 +Ref: c-api/long c PyLong_AsSsize_t9661246 +Ref: 3aea9661246 +Ref: c-api/long c PyLong_AsUnsignedLong9661634 +Ref: 3aeb9661634 +Ref: c-api/long c PyLong_AsSize_t9662051 +Ref: 3aec9662051 +Ref: c-api/long c PyLong_AsUnsignedLongLong9662434 +Ref: c229662434 +Ref: c-api/long c PyLong_AsUnsignedLongMask9663003 +Ref: 3aed9663003 +Ref: c-api/long c PyLong_AsUnsignedLongLongMask9663741 +Ref: 3aee9663741 +Ref: c-api/long c PyLong_AsDouble9664517 +Ref: 3aef9664517 +Ref: c-api/long c PyLong_AsVoidPtr9664894 +Ref: 3ae89664894 +Node: Boolean Objects9665325 +Ref: c-api/bool doc9665447 +Ref: 3af09665447 +Ref: c-api/bool boolean-objects9665447 +Ref: 3af19665447 +Ref: c-api/bool boolobjects9665447 +Ref: 3af29665447 +Ref: c-api/bool c PyBool_Check9665749 +Ref: 3af39665749 +Ref: c-api/bool c Py_False9665852 +Ref: 3af49665852 +Ref: c-api/bool c Py_True9666049 +Ref: 3af59666049 +Ref: c-api/bool c Py_RETURN_FALSE9666244 +Ref: dd89666244 +Ref: c-api/bool c Py_RETURN_TRUE9666367 +Ref: dd79666367 +Ref: c-api/bool c PyBool_FromLong9666488 +Ref: 3af69666488 +Node: Floating Point Objects9666677 +Ref: c-api/float doc9666806 +Ref: 3af79666806 +Ref: c-api/float floating-point-objects9666806 +Ref: 3af89666806 +Ref: c-api/float floatobjects9666806 +Ref: 3af99666806 +Ref: c-api/float c PyFloatObject9666869 +Ref: 3afa9666869 +Ref: c-api/float c PyFloat_Type9666987 +Ref: 3afb9666987 +Ref: c-api/float c PyFloat_Check9667198 +Ref: 3afc9667198 +Ref: c-api/float c PyFloat_CheckExact9667361 +Ref: 3afd9667361 +Ref: c-api/float c PyFloat_FromString9667534 +Ref: 3afe9667534 +Ref: c-api/float c PyFloat_FromDouble9667743 +Ref: 3aff9667743 +Ref: c-api/float c PyFloat_AsDouble9667921 +Ref: 26e9667921 +Ref: c-api/float c PyFloat_AS_DOUBLE9668519 +Ref: 3b009668519 +Ref: c-api/float c PyFloat_GetInfo9668689 +Ref: d579668689 +Ref: c-api/float c PyFloat_GetMax9668967 +Ref: d559668967 +Ref: c-api/float c PyFloat_GetMin9669095 +Ref: d569669095 +Ref: c-api/float c PyFloat_ClearFreeList9669222 +Ref: 3b019669222 +Node: Complex Number Objects9669359 +Ref: c-api/complex doc9669464 +Ref: 3b029669464 +Ref: c-api/complex complex-number-objects9669464 +Ref: 3b039669464 +Ref: c-api/complex complexobjects9669464 +Ref: 3b049669464 +Node: Complex Numbers as C Structures9669897 +Ref: c-api/complex complex-numbers-as-c-structures9670029 +Ref: 3b059670029 +Ref: c-api/complex c Py_complex9670312 +Ref: 39cb9670312 +Ref: c-api/complex c _Py_c_sum9670704 +Ref: 3b069670704 +Ref: c-api/complex c _Py_c_diff9670880 +Ref: 3b079670880 +Ref: c-api/complex c _Py_c_neg9671079 +Ref: 3b089671079 +Ref: c-api/complex c _Py_c_prod9671254 +Ref: 3b099671254 +Ref: c-api/complex c _Py_c_quot9671445 +Ref: 3b0a9671445 +Ref: c-api/complex c _Py_c_pow9671737 +Ref: 3b0b9671737 +Node: Complex Numbers as Python Objects9672046 +Ref: c-api/complex complex-numbers-as-python-objects9672178 +Ref: 3b0c9672178 +Ref: c-api/complex c PyComplexObject9672263 +Ref: 3b0d9672263 +Ref: c-api/complex c PyComplex_Type9672383 +Ref: 3b0e9672383 +Ref: c-api/complex c PyComplex_Check9672596 +Ref: 3b0f9672596 +Ref: c-api/complex c PyComplex_CheckExact9672765 +Ref: 3b109672765 +Ref: c-api/complex c PyComplex_FromCComplex9672944 +Ref: 3b119672944 +Ref: c-api/complex c PyComplex_FromDoubles9673133 +Ref: 3b129673133 +Ref: c-api/complex c PyComplex_RealAsDouble9673334 +Ref: 3b139673334 +Ref: c-api/complex c PyComplex_ImagAsDouble9673452 +Ref: 3b149673452 +Ref: c-api/complex c PyComplex_AsCComplex9673575 +Ref: d589673575 +Node: Sequence Objects9674232 +Ref: c-api/concrete sequence-objects9674357 +Ref: 3b159674357 +Ref: c-api/concrete sequenceobjects9674357 +Ref: 3b169674357 +Node: Bytes Objects<2>9674752 +Ref: c-api/bytes doc9674848 +Ref: 3b179674848 +Ref: c-api/bytes bytes-objects9674848 +Ref: 3b189674848 +Ref: c-api/bytes bytesobjects9674848 +Ref: 3b199674848 +Ref: c-api/bytes c PyBytesObject9675014 +Ref: d1e9675014 +Ref: c-api/bytes c PyBytes_Type9675123 +Ref: 3b1a9675123 +Ref: c-api/bytes c PyBytes_Check9675322 +Ref: d1f9675322 +Ref: c-api/bytes c PyBytes_CheckExact9675477 +Ref: 3b1b9675477 +Ref: c-api/bytes c PyBytes_FromString9675643 +Ref: 3b1c9675643 +Ref: c-api/bytes c PyBytes_FromStringAndSize9675922 +Ref: d209675922 +Ref: c-api/bytes c PyBytes_FromFormat9676265 +Ref: 3b1d9676265 +Ref: c-api/bytes c PyBytes_FromFormatV9680111 +Ref: 3b1e9680111 +Ref: c-api/bytes c PyBytes_FromObject9680342 +Ref: 3b1f9680342 +Ref: c-api/bytes c PyBytes_Size9680527 +Ref: 3b209680527 +Ref: c-api/bytes c PyBytes_GET_SIZE9680640 +Ref: 3b219680640 +Ref: c-api/bytes c PyBytes_AsString9680780 +Ref: 3b229680780 +Ref: c-api/bytes c PyBytes_AS_STRING9681393 +Ref: 3b239681393 +Ref: c-api/bytes c PyBytes_AsStringAndSize9681538 +Ref: 3b249681538 +Ref: c-api/bytes c PyBytes_Concat9682540 +Ref: 3b259682540 +Ref: c-api/bytes c PyBytes_ConcatAndDel9683033 +Ref: 3b269683033 +Ref: c-api/bytes c _PyBytes_Resize9683295 +Ref: 3b279683295 +Ref: Bytes Objects<2>-Footnote-19684180 +Ref: Bytes Objects<2>-Footnote-29684310 +Ref: Bytes Objects<2>-Footnote-39684440 +Ref: Bytes Objects<2>-Footnote-49684570 +Ref: Bytes Objects<2>-Footnote-59684700 +Ref: Bytes Objects<2>-Footnote-69684830 +Ref: Bytes Objects<2>-Footnote-79684960 +Ref: Bytes Objects<2>-Footnote-89685090 +Node: Byte Array Objects9685220 +Ref: c-api/bytearray doc9685351 +Ref: 3b289685351 +Ref: c-api/bytearray byte-array-objects9685351 +Ref: 3b299685351 +Ref: c-api/bytearray bytearrayobjects9685351 +Ref: 3b2a9685351 +Ref: c-api/bytearray c PyByteArrayObject9685406 +Ref: 3b2b9685406 +Ref: c-api/bytearray c PyByteArray_Type9685523 +Ref: 3b2c9685523 +Node: Type check macros9685802 +Ref: c-api/bytearray type-check-macros9685903 +Ref: 3b2d9685903 +Ref: c-api/bytearray c PyByteArray_Check9685956 +Ref: 3b2e9685956 +Ref: c-api/bytearray c PyByteArray_CheckExact9686123 +Ref: 3b2f9686123 +Node: Direct API functions9686301 +Ref: c-api/bytearray direct-api-functions9686417 +Ref: 3b309686417 +Ref: c-api/bytearray c PyByteArray_FromObject9686476 +Ref: d219686476 +Ref: c-api/bytearray c PyByteArray_FromStringAndSize9686683 +Ref: d229686683 +Ref: c-api/bytearray c PyByteArray_Concat9686933 +Ref: 3b319686933 +Ref: c-api/bytearray c PyByteArray_Size9687122 +Ref: 3b329687122 +Ref: c-api/bytearray c PyByteArray_AsString9687267 +Ref: 3b339687267 +Ref: c-api/bytearray c PyByteArray_Resize9687501 +Ref: 3b349687501 +Node: Macros9687647 +Ref: c-api/bytearray macros9687737 +Ref: 3b359687737 +Ref: c-api/bytearray c PyByteArray_AS_STRING9687838 +Ref: 3b369687838 +Ref: c-api/bytearray c PyByteArray_GET_SIZE9687964 +Ref: 3b379687964 +Node: Unicode Objects and Codecs9688090 +Ref: c-api/unicode doc9688218 +Ref: b1c9688218 +Ref: c-api/unicode unicode-objects-and-codecs9688218 +Ref: 3b389688218 +Ref: c-api/unicode unicodeobjects9688218 +Ref: 3b399688218 +Node: Unicode Objects9688370 +Ref: c-api/unicode unicode-objects9688472 +Ref: 3b3a9688472 +Ref: Unicode Objects-Footnote-19690153 +Ref: Unicode Objects-Footnote-29690202 +Node: Unicode Type9690251 +Ref: c-api/unicode unicode-type9690352 +Ref: 3b3b9690352 +Ref: c-api/unicode c Py_UCS49690484 +Ref: ad39690484 +Ref: c-api/unicode c Py_UCS29690504 +Ref: ad29690504 +Ref: c-api/unicode c Py_UCS19690524 +Ref: ad19690524 +Ref: c-api/unicode c Py_UNICODE9690793 +Ref: aee9690793 +Ref: c-api/unicode c PyASCIIObject9691135 +Ref: ad49691135 +Ref: c-api/unicode c PyCompactUnicodeObject9691161 +Ref: ad59691161 +Ref: c-api/unicode c PyUnicodeObject9691196 +Ref: 3b3c9691196 +Ref: c-api/unicode c PyUnicode_Type9691505 +Ref: 3b3d9691505 +Ref: c-api/unicode c PyUnicode_Check9691815 +Ref: 3b3e9691815 +Ref: c-api/unicode c PyUnicode_CheckExact9691964 +Ref: 3b3f9691964 +Ref: c-api/unicode c PyUnicode_READY9692116 +Ref: ad69692116 +Ref: c-api/unicode c PyUnicode_GET_LENGTH9692634 +Ref: acc9692634 +Ref: c-api/unicode c PyUnicode_1BYTE_DATA9692881 +Ref: adb9692881 +Ref: c-api/unicode c PyUnicode_2BYTE_DATA9692941 +Ref: adc9692941 +Ref: c-api/unicode c PyUnicode_4BYTE_DATA9693001 +Ref: add9693001 +Ref: c-api/unicode c PyUnicode_WCHAR_KIND9693466 +Ref: adf9693466 +Ref: c-api/unicode c PyUnicode_1BYTE_KIND9693500 +Ref: ae09693500 +Ref: c-api/unicode c PyUnicode_2BYTE_KIND9693534 +Ref: ae19693534 +Ref: c-api/unicode c PyUnicode_4BYTE_KIND9693568 +Ref: ae29693568 +Ref: c-api/unicode c PyUnicode_KIND9693809 +Ref: ade9693809 +Ref: c-api/unicode c PyUnicode_DATA9694133 +Ref: ada9694133 +Ref: c-api/unicode c PyUnicode_WRITE9694356 +Ref: ae59694356 +Ref: c-api/unicode c PyUnicode_READ9694923 +Ref: ae39694923 +Ref: c-api/unicode c PyUnicode_READ_CHAR9695203 +Ref: ae49695203 +Ref: c-api/unicode c PyUnicode_MAX_CHAR_VALUE9695525 +Ref: ae69695525 +Ref: c-api/unicode c PyUnicode_ClearFreeList9695841 +Ref: 3b409695841 +Ref: c-api/unicode c PyUnicode_GET_SIZE9695957 +Ref: af59695957 +Ref: c-api/unicode c PyUnicode_GET_DATA_SIZE9696385 +Ref: af79696385 +Ref: c-api/unicode c PyUnicode_AS_UNICODE9696769 +Ref: af19696769 +Ref: c-api/unicode c PyUnicode_AS_DATA9696832 +Ref: af49696832 +Node: Unicode Character Properties9697903 +Ref: c-api/unicode unicode-character-properties9698051 +Ref: 3b419698051 +Ref: c-api/unicode c Py_UNICODE_ISSPACE9698315 +Ref: 3b429698315 +Ref: c-api/unicode c Py_UNICODE_ISLOWER9698461 +Ref: 3b439698461 +Ref: c-api/unicode c Py_UNICODE_ISUPPER9698606 +Ref: 3b449698606 +Ref: c-api/unicode c Py_UNICODE_ISTITLE9698752 +Ref: 3b459698752 +Ref: c-api/unicode c Py_UNICODE_ISLINEBREAK9698897 +Ref: 3b469698897 +Ref: c-api/unicode c Py_UNICODE_ISDECIMAL9699046 +Ref: 3b479699046 +Ref: c-api/unicode c Py_UNICODE_ISDIGIT9699186 +Ref: 3b489699186 +Ref: c-api/unicode c Py_UNICODE_ISNUMERIC9699322 +Ref: 3b499699322 +Ref: c-api/unicode c Py_UNICODE_ISALPHA9699462 +Ref: 3b4a9699462 +Ref: c-api/unicode c Py_UNICODE_ISALNUM9699609 +Ref: 3b4b9699609 +Ref: c-api/unicode c Py_UNICODE_ISPRINTABLE9699758 +Ref: 3b4c9699758 +Ref: c-api/unicode c Py_UNICODE_TOLOWER9700429 +Ref: 3b4d9700429 +Ref: c-api/unicode c Py_UNICODE_TOUPPER9700631 +Ref: 3b4e9700631 +Ref: c-api/unicode c Py_UNICODE_TOTITLE9700833 +Ref: 3b4f9700833 +Ref: c-api/unicode c Py_UNICODE_TODECIMAL9701035 +Ref: 3b509701035 +Ref: c-api/unicode c Py_UNICODE_TODIGIT9701256 +Ref: 3b519701256 +Ref: c-api/unicode c Py_UNICODE_TONUMERIC9701471 +Ref: 3b529701471 +Ref: c-api/unicode c Py_UNICODE_IS_SURROGATE9701724 +Ref: 3b539701724 +Ref: c-api/unicode c Py_UNICODE_IS_HIGH_SURROGATE9701834 +Ref: 3b549701834 +Ref: c-api/unicode c Py_UNICODE_IS_LOW_SURROGATE9701954 +Ref: 3b559701954 +Ref: c-api/unicode c Py_UNICODE_JOIN_SURROGATES9702072 +Ref: 3b569702072 +Node: Creating and accessing Unicode strings9702297 +Ref: c-api/unicode creating-and-accessing-unicode-strings9702459 +Ref: 3b579702459 +Ref: c-api/unicode c PyUnicode_New9702643 +Ref: acd9702643 +Ref: c-api/unicode c PyUnicode_FromKindAndData9703146 +Ref: ad79703146 +Ref: c-api/unicode c PyUnicode_FromStringAndSize9703605 +Ref: 3b589703605 +Ref: c-api/unicode c PyUnicode_FromString9704274 +Ref: 38d19704274 +Ref: c-api/unicode c PyUnicode_FromFormat9704458 +Ref: 7cb9704458 +Ref: c-api/unicode c PyUnicode_FromFormatV9711712 +Ref: 399b9711712 +Ref: c-api/unicode c PyUnicode_FromEncodedObject9711946 +Ref: 3b599711946 +Ref: c-api/unicode c PyUnicode_GetLength9712708 +Ref: acb9712708 +Ref: c-api/unicode c PyUnicode_CopyCharacters9712865 +Ref: aca9712865 +Ref: c-api/unicode c PyUnicode_Fill9713342 +Ref: afb9713342 +Ref: c-api/unicode c PyUnicode_WriteChar9713821 +Ref: ad09713821 +Ref: c-api/unicode c PyUnicode_ReadChar9714384 +Ref: acf9714384 +Ref: c-api/unicode c PyUnicode_Substring9714710 +Ref: ace9714710 +Ref: c-api/unicode c PyUnicode_AsUCS49715033 +Ref: ad89715033 +Ref: c-api/unicode c PyUnicode_AsUCS4Copy9715464 +Ref: ad99715464 +Ref: Creating and accessing Unicode strings-Footnote-19715845 +Ref: Creating and accessing Unicode strings-Footnote-29715998 +Ref: Creating and accessing Unicode strings-Footnote-39716151 +Ref: Creating and accessing Unicode strings-Footnote-49716304 +Ref: Creating and accessing Unicode strings-Footnote-59716457 +Ref: Creating and accessing Unicode strings-Footnote-69716610 +Ref: Creating and accessing Unicode strings-Footnote-79716763 +Ref: Creating and accessing Unicode strings-Footnote-89716916 +Ref: Creating and accessing Unicode strings-Footnote-99717069 +Ref: Creating and accessing Unicode strings-Footnote-109717222 +Ref: Creating and accessing Unicode strings-Footnote-119717376 +Ref: Creating and accessing Unicode strings-Footnote-129717530 +Ref: Creating and accessing Unicode strings-Footnote-139717684 +Node: Deprecated Py_UNICODE APIs9717838 +Ref: c-api/unicode deprecated-py-unicode-apis9717987 +Ref: 3b5b9717987 +Ref: c-api/unicode c PyUnicode_FromUnicode9718365 +Ref: aef9718365 +Ref: c-api/unicode c PyUnicode_AsUnicode9719411 +Ref: af29719411 +Ref: c-api/unicode c PyUnicode_TransformDecimalToASCII9720308 +Ref: b0f9720308 +Ref: c-api/unicode c PyUnicode_AsUnicodeAndSize9720861 +Ref: af39720861 +Ref: c-api/unicode c PyUnicode_AsUnicodeCopy9721597 +Ref: af89721597 +Ref: c-api/unicode c PyUnicode_GetSize9722245 +Ref: af69722245 +Ref: c-api/unicode c PyUnicode_FromObject9722631 +Ref: 3b5d9722631 +Ref: Deprecated Py_UNICODE APIs-Footnote-19723061 +Node: Locale Encoding9723110 +Ref: c-api/unicode locale-encoding9723241 +Ref: 3b5e9723241 +Ref: c-api/unicode c PyUnicode_DecodeLocaleAndSize9723375 +Ref: 43e9723375 +Ref: c-api/unicode c PyUnicode_DecodeLocale9724554 +Ref: 3b5f9724554 +Ref: c-api/unicode c PyUnicode_EncodeLocale9724832 +Ref: 43f9724832 +Ref: Locale Encoding-Footnote-19726026 +Ref: Locale Encoding-Footnote-29726075 +Node: File System Encoding9726124 +Ref: c-api/unicode file-system-encoding9726244 +Ref: 3b609726244 +Ref: c-api/unicode c PyUnicode_FSConverter9726726 +Ref: 5ce9726726 +Ref: c-api/unicode c PyUnicode_FSDecoder9727440 +Ref: 5cf9727440 +Ref: c-api/unicode c PyUnicode_DecodeFSDefaultAndSize9728001 +Ref: 39909728001 +Ref: c-api/unicode c PyUnicode_DecodeFSDefault9728788 +Ref: 3b619728788 +Ref: c-api/unicode c PyUnicode_EncodeFSDefault9729321 +Ref: 39919729321 +Ref: File System Encoding-Footnote-19730250 +Ref: File System Encoding-Footnote-29730299 +Node: wchar_t Support9730348 +Ref: c-api/unicode wchar-t-support9730444 +Ref: 3b629730444 +Ref: c-api/unicode c PyUnicode_FromWideChar9730550 +Ref: af09730550 +Ref: c-api/unicode c PyUnicode_AsWideChar9730914 +Ref: 3b5c9730914 +Ref: c-api/unicode c PyUnicode_AsWideCharString9731721 +Ref: 4389731721 +Node: Built-in Codecs9732796 +Ref: c-api/unicode built-in-codecs9732933 +Ref: 3b639732933 +Ref: c-api/unicode builtincodecs9732933 +Ref: 3b5a9732933 +Node: Generic Codecs9734361 +Ref: c-api/unicode generic-codecs9734448 +Ref: 3b649734448 +Ref: c-api/unicode c PyUnicode_Decode9734532 +Ref: 3b659734532 +Ref: c-api/unicode c PyUnicode_AsEncodedString9735053 +Ref: 3b669735053 +Ref: c-api/unicode c PyUnicode_Encode9735564 +Ref: afc9735564 +Node: UTF-8 Codecs9736306 +Ref: c-api/unicode utf-8-codecs9736415 +Ref: 3b679736415 +Ref: c-api/unicode c PyUnicode_DecodeUTF89736493 +Ref: 3b689736493 +Ref: c-api/unicode c PyUnicode_DecodeUTF8Stateful9736793 +Ref: 3b699736793 +Ref: c-api/unicode c PyUnicode_AsUTF8String9737291 +Ref: aff9737291 +Ref: c-api/unicode c PyUnicode_AsUTF8AndSize9737583 +Ref: 42a9737583 +Ref: c-api/unicode c PyUnicode_AsUTF89738516 +Ref: 42b9738516 +Ref: c-api/unicode c PyUnicode_EncodeUTF89738790 +Ref: afe9738790 +Node: UTF-32 Codecs9739412 +Ref: c-api/unicode utf-32-codecs9739520 +Ref: 3b6a9739520 +Ref: c-api/unicode c PyUnicode_DecodeUTF329739601 +Ref: 3b6b9739601 +Ref: c-api/unicode c PyUnicode_DecodeUTF32Stateful9740751 +Ref: 3b6c9740751 +Ref: c-api/unicode c PyUnicode_AsUTF32String9741373 +Ref: 3b6d9741373 +Ref: c-api/unicode c PyUnicode_EncodeUTF329741708 +Ref: b009741708 +Node: UTF-16 Codecs9742756 +Ref: c-api/unicode utf-16-codecs9742864 +Ref: 3b6e9742864 +Ref: c-api/unicode c PyUnicode_DecodeUTF169742945 +Ref: 3b6f9742945 +Ref: c-api/unicode c PyUnicode_DecodeUTF16Stateful9744175 +Ref: 3b709744175 +Ref: c-api/unicode c PyUnicode_AsUTF16String9744806 +Ref: 3b719744806 +Ref: c-api/unicode c PyUnicode_EncodeUTF169745141 +Ref: b019745141 +Node: UTF-7 Codecs9746314 +Ref: c-api/unicode utf-7-codecs9746430 +Ref: 3b729746430 +Ref: c-api/unicode c PyUnicode_DecodeUTF79746508 +Ref: 3b739746508 +Ref: c-api/unicode c PyUnicode_DecodeUTF7Stateful9746808 +Ref: 3b749746808 +Ref: c-api/unicode c PyUnicode_EncodeUTF79747308 +Ref: afd9747308 +Node: Unicode-Escape Codecs9748150 +Ref: c-api/unicode unicode-escape-codecs9748278 +Ref: 3b759748278 +Ref: c-api/unicode c PyUnicode_DecodeUnicodeEscape9748389 +Ref: 3b769748389 +Ref: c-api/unicode c PyUnicode_AsUnicodeEscapeString9748707 +Ref: b039748707 +Ref: c-api/unicode c PyUnicode_EncodeUnicodeEscape9749027 +Ref: b029749027 +Node: Raw-Unicode-Escape Codecs9749548 +Ref: c-api/unicode raw-unicode-escape-codecs9749678 +Ref: 3b779749678 +Ref: c-api/unicode c PyUnicode_DecodeRawUnicodeEscape9749801 +Ref: 3b789749801 +Ref: c-api/unicode c PyUnicode_AsRawUnicodeEscapeString9750126 +Ref: b059750126 +Ref: c-api/unicode c PyUnicode_EncodeRawUnicodeEscape9750453 +Ref: b049750453 +Node: Latin-1 Codecs9751038 +Ref: c-api/unicode latin-1-codecs9751159 +Ref: 3b799751159 +Ref: c-api/unicode c PyUnicode_DecodeLatin19751356 +Ref: 3b7a9751356 +Ref: c-api/unicode c PyUnicode_AsLatin1String9751660 +Ref: b079751660 +Ref: c-api/unicode c PyUnicode_EncodeLatin19751961 +Ref: b069751961 +Node: ASCII Codecs9752537 +Ref: c-api/unicode ascii-codecs9752653 +Ref: 3b7b9752653 +Ref: c-api/unicode c PyUnicode_DecodeASCII9752801 +Ref: 3b7c9752801 +Ref: c-api/unicode c PyUnicode_AsASCIIString9753102 +Ref: b099753102 +Ref: c-api/unicode c PyUnicode_EncodeASCII9753395 +Ref: b089753395 +Node: Character Map Codecs9753967 +Ref: c-api/unicode character-map-codecs9754092 +Ref: 3b7d9754092 +Ref: c-api/unicode c PyUnicode_DecodeCharmap9754571 +Ref: 3b7e9754571 +Ref: c-api/unicode c PyUnicode_AsCharmapString9755395 +Ref: 3b7f9755395 +Ref: c-api/unicode c PyUnicode_EncodeCharmap9756055 +Ref: b0a9756055 +Ref: c-api/unicode c PyUnicode_Translate9756753 +Ref: 3b809756753 +Ref: c-api/unicode c PyUnicode_TranslateCharmap9757592 +Ref: b0b9757592 +Node: MBCS codecs for Windows9758240 +Ref: c-api/unicode mbcs-codecs-for-windows9758368 +Ref: 3b819758368 +Ref: c-api/unicode c PyUnicode_DecodeMBCS9758728 +Ref: 3b829758728 +Ref: c-api/unicode c PyUnicode_DecodeMBCSStateful9759027 +Ref: 3b839759027 +Ref: c-api/unicode c PyUnicode_AsMBCSString9759492 +Ref: b0d9759492 +Ref: c-api/unicode c PyUnicode_EncodeCodePage9759783 +Ref: b0e9759783 +Ref: c-api/unicode c PyUnicode_EncodeMBCS9760181 +Ref: b0c9760181 +Node: Methods & Slots9760793 +Ref: c-api/unicode methods-slots9760892 +Ref: 3b849760892 +Node: Methods and Slot Functions9760943 +Ref: c-api/unicode methods-and-slot-functions9761056 +Ref: 3b859761056 +Ref: c-api/unicode unicodemethodsandslots9761056 +Ref: 3b869761056 +Ref: c-api/unicode c PyUnicode_Concat9761379 +Ref: b1d9761379 +Ref: c-api/unicode c PyUnicode_Split9761557 +Ref: 3b879761557 +Ref: c-api/unicode c PyUnicode_Splitlines9762022 +Ref: 3b889762022 +Ref: c-api/unicode c PyUnicode_Join9762372 +Ref: b1e9762372 +Ref: c-api/unicode c PyUnicode_Tailmatch9762599 +Ref: afa9762599 +Ref: c-api/unicode c PyUnicode_Find9763000 +Ref: 3b899763000 +Ref: c-api/unicode c PyUnicode_FindChar9763548 +Ref: 4409763548 +Ref: c-api/unicode c PyUnicode_Count9764227 +Ref: 3b8a9764227 +Ref: c-api/unicode c PyUnicode_Replace9764485 +Ref: 3b8b9764485 +Ref: c-api/unicode c PyUnicode_Compare9764837 +Ref: af99764837 +Ref: c-api/unicode c PyUnicode_CompareWithASCIIString9765159 +Ref: bfe9765159 +Ref: c-api/unicode c PyUnicode_RichCompare9765615 +Ref: 3b8c9765615 +Ref: c-api/unicode c PyUnicode_Format9766145 +Ref: 3b8d9766145 +Ref: c-api/unicode c PyUnicode_Contains9766371 +Ref: 3b8e9766371 +Ref: c-api/unicode c PyUnicode_InternInPlace9766676 +Ref: 3b8f9766676 +Ref: c-api/unicode c PyUnicode_InternFromString9767451 +Ref: 3b909767451 +Node: Tuple Objects9767834 +Ref: c-api/tuple doc9767967 +Ref: 3b919767967 +Ref: c-api/tuple tuple-objects9767967 +Ref: 3b929767967 +Ref: c-api/tuple tupleobjects9767967 +Ref: 3b939767967 +Ref: c-api/tuple c PyTupleObject9768014 +Ref: 399f9768014 +Ref: c-api/tuple c PyTuple_Type9768123 +Ref: 3b949768123 +Ref: c-api/tuple c PyTuple_Check9768322 +Ref: 3b959768322 +Ref: c-api/tuple c PyTuple_CheckExact9768466 +Ref: 3b969768466 +Ref: c-api/tuple c PyTuple_New9768621 +Ref: 3b979768621 +Ref: c-api/tuple c PyTuple_Pack9768786 +Ref: 3b989768786 +Ref: c-api/tuple c PyTuple_Size9769144 +Ref: 3b999769144 +Ref: c-api/tuple c PyTuple_GET_SIZE9769279 +Ref: 3b9a9769279 +Ref: c-api/tuple c PyTuple_GetItem9769466 +Ref: 385c9769466 +Ref: c-api/tuple c PyTuple_GET_ITEM9769750 +Ref: 3b9b9769750 +Ref: c-api/tuple c PyTuple_GetSlice9769946 +Ref: 3b9c9769946 +Ref: c-api/tuple c PyTuple_SetItem9770318 +Ref: 38619770318 +Ref: c-api/tuple c PyTuple_SET_ITEM9770790 +Ref: 3b9d9770790 +Ref: c-api/tuple c _PyTuple_Resize9771274 +Ref: e4b9771274 +Ref: c-api/tuple c PyTuple_ClearFreeList9772190 +Ref: 3b9e9772190 +Node: Struct Sequence Objects9772304 +Ref: c-api/tuple struct-sequence-objects9772423 +Ref: 3b9f9772423 +Ref: c-api/tuple c PyStructSequence_NewType9772731 +Ref: 3ba09772731 +Ref: c-api/tuple c PyStructSequence_InitType9773044 +Ref: 9279773044 +Ref: c-api/tuple c PyStructSequence_InitType29773222 +Ref: 9269773222 +Ref: c-api/tuple c PyStructSequence_Desc9773468 +Ref: 4299773468 +Ref: c-api/tuple c PyStructSequence_Field9775046 +Ref: 4289775046 +Ref: c-api/tuple c PyStructSequence_UnnamedField9776147 +Ref: 3ba29776147 +Ref: c-api/tuple c PyStructSequence_New9776258 +Ref: 3ba19776258 +Ref: c-api/tuple c PyStructSequence_GetItem9776479 +Ref: 3ba39776479 +Ref: c-api/tuple c PyStructSequence_GET_ITEM9776735 +Ref: 3ba49776735 +Ref: c-api/tuple c PyStructSequence_SetItem9776935 +Ref: 3ba59776935 +Ref: c-api/tuple c PyStructSequence_SET_ITEM9777281 +Ref: 3ba69777281 +Node: List Objects9777513 +Ref: c-api/list doc9777610 +Ref: 3ba79777610 +Ref: c-api/list list-objects9777610 +Ref: 3ba89777610 +Ref: c-api/list listobjects9777610 +Ref: 3ba99777610 +Ref: c-api/list c PyListObject9777655 +Ref: 3a569777655 +Ref: c-api/list c PyList_Type9777762 +Ref: 38919777762 +Ref: c-api/list c PyList_Check9777961 +Ref: 3baa9777961 +Ref: c-api/list c PyList_CheckExact9778102 +Ref: 3bab9778102 +Ref: c-api/list c PyList_New9778254 +Ref: 38f49778254 +Ref: c-api/list c PyList_Size9778768 +Ref: d7e9778768 +Ref: c-api/list c PyList_GET_SIZE9778940 +Ref: 3bac9778940 +Ref: c-api/list c PyList_GetItem9779071 +Ref: 385d9779071 +Ref: c-api/list c PyList_GET_ITEM9779492 +Ref: 3bad9779492 +Ref: c-api/list c PyList_SetItem9779681 +Ref: 38629779681 +Ref: c-api/list c PyList_SET_ITEM9780130 +Ref: 3bae9780130 +Ref: c-api/list c PyList_Insert9780649 +Ref: 3baf9780649 +Ref: c-api/list c PyList_Append9780958 +Ref: 3bb09780958 +Ref: c-api/list c PyList_GetSlice9781217 +Ref: 3bb19781217 +Ref: c-api/list c PyList_SetSlice9781613 +Ref: 3bb29781613 +Ref: c-api/list c PyList_Sort9782080 +Ref: 3bb39782080 +Ref: c-api/list c PyList_Reverse9782267 +Ref: 3bb49782267 +Ref: c-api/list c PyList_AsTuple9782467 +Ref: 3bb59782467 +Ref: c-api/list c PyList_ClearFreeList9782667 +Ref: 3bb69782667 +Node: Container Objects9782806 +Ref: c-api/concrete container-objects9782935 +Ref: 3bb79782935 +Ref: c-api/concrete mapobjects9782935 +Ref: 3bb89782935 +Node: Dictionary Objects9783033 +Ref: c-api/dict doc9783125 +Ref: 3bb99783125 +Ref: c-api/dict dictionary-objects9783125 +Ref: 3bba9783125 +Ref: c-api/dict dictobjects9783125 +Ref: 3bbb9783125 +Ref: c-api/dict c PyDictObject9783180 +Ref: 3bbc9783180 +Ref: c-api/dict c PyDict_Type9783293 +Ref: 3bbd9783293 +Ref: c-api/dict c PyDict_Check9783498 +Ref: 3aab9783498 +Ref: c-api/dict c PyDict_CheckExact9783639 +Ref: 3bbe9783639 +Ref: c-api/dict c PyDict_New9783791 +Ref: 3bbf9783791 +Ref: c-api/dict c PyDictProxy_New9783931 +Ref: 3bc09783931 +Ref: c-api/dict c PyDict_Clear9784257 +Ref: 3bc19784257 +Ref: c-api/dict c PyDict_Contains9784365 +Ref: 3bc29784365 +Ref: c-api/dict c PyDict_Copy9784659 +Ref: 3bc39784659 +Ref: c-api/dict c PyDict_SetItem9784825 +Ref: 38639784825 +Ref: c-api/dict c PyDict_SetItemString9785190 +Ref: 3bc49785190 +Ref: c-api/dict c PyDict_DelItem9785577 +Ref: 3bc59785577 +Ref: c-api/dict c PyDict_DelItemString9785906 +Ref: 3bc69785906 +Ref: c-api/dict c PyDict_GetItem9786196 +Ref: 385e9786196 +Ref: c-api/dict c PyDict_GetItemWithError9786684 +Ref: 3bc79786684 +Ref: c-api/dict c PyDict_GetItemString9787051 +Ref: 385f9787051 +Ref: c-api/dict c PyDict_SetDefault9787585 +Ref: 3bc89787585 +Ref: c-api/dict c PyDict_Items9788175 +Ref: 3bc99788175 +Ref: c-api/dict c PyDict_Keys9788352 +Ref: 3bca9788352 +Ref: c-api/dict c PyDict_Values9788527 +Ref: 3bcb9788527 +Ref: c-api/dict c PyDict_Size9788710 +Ref: 3bcc9788710 +Ref: c-api/dict c PyDict_Next9788874 +Ref: 3bcd9788874 +Ref: c-api/dict c PyDict_Merge9790779 +Ref: 3bce9790779 +Ref: c-api/dict c PyDict_Update9791328 +Ref: 3bcf9791328 +Ref: c-api/dict c PyDict_MergeFromSeq29791748 +Ref: 3bd09791748 +Ref: c-api/dict c PyDict_ClearFreeList9792430 +Ref: 3bd19792430 +Node: Set Objects9792569 +Ref: c-api/set doc9792661 +Ref: 3bd29792661 +Ref: c-api/set set-objects9792661 +Ref: 3bd39792661 +Ref: c-api/set setobjects9792661 +Ref: 3bd49792661 +Ref: c-api/set c PySetObject9793443 +Ref: 3bd59793443 +Ref: c-api/set c PySet_Type9794074 +Ref: 3bd69794074 +Ref: c-api/set c PyFrozenSet_Type9794220 +Ref: 3bd79794220 +Ref: c-api/set c PySet_Check9794524 +Ref: 3bd89794524 +Ref: c-api/set c PyFrozenSet_Check9794658 +Ref: 3bd99794658 +Ref: c-api/set c PyAnySet_Check9794804 +Ref: 3bda9794804 +Ref: c-api/set c PyAnySet_CheckExact9794974 +Ref: 3bdb9794974 +Ref: c-api/set c PyFrozenSet_CheckExact9795155 +Ref: 3bdc9795155 +Ref: c-api/set c PySet_New9795311 +Ref: dab9795311 +Ref: c-api/set c PyFrozenSet_New9795762 +Ref: dac9795762 +Ref: c-api/set c PySet_Size9796298 +Ref: db09796298 +Ref: c-api/set c PySet_GET_SIZE9796605 +Ref: 3bdd9796605 +Ref: c-api/set c PySet_Contains9796736 +Ref: daf9796736 +Ref: c-api/set c PySet_Add9797245 +Ref: dad9797245 +Ref: c-api/set c PySet_Discard9797995 +Ref: dae9797995 +Ref: c-api/set c PySet_Pop9798554 +Ref: 3bde9798554 +Ref: c-api/set c PySet_Clear9798939 +Ref: 3bdf9798939 +Ref: c-api/set c PySet_ClearFreeList9799033 +Ref: 3be09799033 +Node: Function Objects<2>9799171 +Ref: c-api/concrete function-objects9799297 +Ref: 3be19799297 +Ref: c-api/concrete otherobjects9799297 +Ref: 3be29799297 +Node: Function Objects<3>9799510 +Ref: c-api/function doc9799617 +Ref: 3be39799617 +Ref: c-api/function function-objects9799617 +Ref: 3be49799617 +Ref: c-api/function id19799617 +Ref: 3be59799617 +Ref: c-api/function c PyFunctionObject9799725 +Ref: 3be69799725 +Ref: c-api/function c PyFunction_Type9799797 +Ref: 3be79799797 +Ref: c-api/function c PyFunction_Check9800014 +Ref: 3be89800014 +Ref: c-api/function c PyFunction_New9800197 +Ref: 3be99800197 +Ref: c-api/function c PyFunction_NewWithQualName9800760 +Ref: 3bea9800760 +Ref: c-api/function c PyFunction_GetCode9801234 +Ref: 3beb9801234 +Ref: c-api/function c PyFunction_GetGlobals9801407 +Ref: 3bec9801407 +Ref: c-api/function c PyFunction_GetModule9801590 +Ref: 3bed9801590 +Ref: c-api/function c PyFunction_GetDefaults9801879 +Ref: 3bee9801879 +Ref: c-api/function c PyFunction_SetDefaults9802109 +Ref: 3bef9802109 +Ref: c-api/function c PyFunction_GetClosure9802386 +Ref: 3bf09802386 +Ref: c-api/function c PyFunction_SetClosure9802615 +Ref: 3bf19802615 +Ref: c-api/function c PyFunction_GetAnnotations9802901 +Ref: 3bf29802901 +Ref: c-api/function c PyFunction_SetAnnotations9803117 +Ref: 3bf39803117 +Node: Instance Method Objects9803397 +Ref: c-api/method doc9803530 +Ref: 3bf49803530 +Ref: c-api/method instance-method-objects9803530 +Ref: 3bf59803530 +Ref: c-api/method instancemethod-objects9803530 +Ref: 3bf69803530 +Ref: c-api/method c PyInstanceMethod_Type9803795 +Ref: 3bf79803795 +Ref: c-api/method c PyInstanceMethod_Check9803983 +Ref: 3bf89803983 +Ref: c-api/method c PyInstanceMethod_New9804186 +Ref: 3bf99804186 +Ref: c-api/method c PyInstanceMethod_Function9804452 +Ref: 3bfa9804452 +Ref: c-api/method c PyInstanceMethod_GET_FUNCTION9804636 +Ref: 3bfb9804636 +Node: Method Objects<2>9804842 +Ref: c-api/method id19804968 +Ref: 3bfc9804968 +Ref: c-api/method method-objects9804968 +Ref: 3bfd9804968 +Ref: c-api/method c PyMethod_Type9805194 +Ref: 3bfe9805194 +Ref: c-api/method c PyMethod_Check9805394 +Ref: 3bff9805394 +Ref: c-api/method c PyMethod_New9805571 +Ref: 3c009805571 +Ref: c-api/method c PyMethod_Function9805917 +Ref: 3c019805917 +Ref: c-api/method c PyMethod_GET_FUNCTION9806088 +Ref: 3c029806088 +Ref: c-api/method c PyMethod_Self9806280 +Ref: 3c039806280 +Ref: c-api/method c PyMethod_GET_SELF9806440 +Ref: 3c049806440 +Ref: c-api/method c PyMethod_ClearFreeList9806624 +Ref: 3c059806624 +Node: Cell Objects9806739 +Ref: c-api/cell doc9806857 +Ref: 3c069806857 +Ref: c-api/cell cell-objects9806857 +Ref: 3c079806857 +Ref: c-api/cell id19806857 +Ref: 3c089806857 +Ref: c-api/cell c PyCellObject9807502 +Ref: 3c099807502 +Ref: c-api/cell c PyCell_Type9807573 +Ref: 3c0a9807573 +Ref: c-api/cell c PyCell_Check9807668 +Ref: 3c0b9807668 +Ref: c-api/cell c PyCell_New9807780 +Ref: 3c0c9807780 +Ref: c-api/cell c PyCell_Get9807972 +Ref: 3c0d9807972 +Ref: c-api/cell c PyCell_GET9808109 +Ref: 3c0e9808109 +Ref: c-api/cell c PyCell_Set9808326 +Ref: 3c0f9808326 +Ref: c-api/cell c PyCell_SET9808687 +Ref: 3c109808687 +Ref: c-api/code codeobjects9808949 +Ref: 3c119808949 +Node: Code Objects<2>9808950 +Ref: c-api/code doc9809042 +Ref: 3c129809042 +Ref: c-api/code code-objects9809042 +Ref: 3c139809042 +Ref: c-api/code c PyCodeObject9809247 +Ref: 3c149809247 +Ref: c-api/code c PyCode_Type9809406 +Ref: 3c159809406 +Ref: c-api/code c PyCode_Check9809553 +Ref: 3c169809553 +Ref: c-api/code c PyCode_GetNumFree9809657 +Ref: 3c179809657 +Ref: c-api/code c PyCode_New9809766 +Ref: 2729809766 +Ref: c-api/code c PyCode_NewWithPosOnlyArgs9810428 +Ref: 2719810428 +Ref: c-api/code c PyCode_NewEmpty9810987 +Ref: cea9810987 +Node: Other Objects9811342 +Ref: c-api/concrete other-objects9811442 +Ref: 3c189811442 +Node: File Objects9811848 +Ref: c-api/file doc9811933 +Ref: 3c199811933 +Ref: c-api/file file-objects9811933 +Ref: 3c1a9811933 +Ref: c-api/file fileobjects9811933 +Ref: 3c1b9811933 +Ref: c-api/file c PyFile_FromFd9812520 +Ref: 3c1c9812520 +Ref: c-api/file c PyObject_AsFileDescriptor9813476 +Ref: 3c1d9813476 +Ref: c-api/file c PyFile_GetLine9813892 +Ref: 3c1e9813892 +Ref: c-api/file c PyFile_SetOpenCodeHook9814659 +Ref: 1cc19814659 +Ref: c-api/file c PyFile_WriteObject9815846 +Ref: 3c1f9815846 +Ref: c-api/file c PyFile_WriteString9816237 +Ref: 3c209816237 +Node: Module Objects9816444 +Ref: c-api/module doc9816554 +Ref: 3c219816554 +Ref: c-api/module module-objects9816554 +Ref: 3c229816554 +Ref: c-api/module moduleobjects9816554 +Ref: 3c239816554 +Ref: c-api/module c PyModule_Type9816601 +Ref: 3c249816601 +Ref: c-api/module c PyModule_Check9816801 +Ref: 3c259816801 +Ref: c-api/module c PyModule_CheckExact9816934 +Ref: 3c269816934 +Ref: c-api/module c PyModule_NewObject9817087 +Ref: 3c279817087 +Ref: c-api/module c PyModule_New9817686 +Ref: 3c289817686 +Ref: c-api/module c PyModule_GetDict9817905 +Ref: 3c299817905 +Ref: c-api/module c PyModule_GetNameObject9818494 +Ref: 3c2a9818494 +Ref: c-api/module c PyModule_GetName9818814 +Ref: 3c2b9818814 +Ref: c-api/module c PyModule_GetState9818985 +Ref: 3c2c9818985 +Ref: c-api/module c PyModule_GetDef9819228 +Ref: 3c2e9819228 +Ref: c-api/module c PyModule_GetFilenameObject9819464 +Ref: 3c2f9819464 +Ref: c-api/module c PyModule_GetFilename9819898 +Ref: 3c309819898 +Node: Initializing C modules9820329 +Ref: c-api/module initializing-c-modules9820424 +Ref: 3c319820424 +Ref: c-api/module initializing-modules9820424 +Ref: 38b59820424 +Ref: c-api/module c PyModuleDef9821058 +Ref: 38889821058 +Ref: c-api/module c PyModuleDef m_base9821280 +Ref: 3c329821280 +Ref: c-api/module c PyModuleDef m_name9821397 +Ref: 3c339821397 +Ref: c-api/module c PyModuleDef m_doc9821472 +Ref: 3c349821472 +Ref: c-api/module c PyModuleDef m_size9821634 +Ref: 3c2d9821634 +Ref: c-api/module c PyModuleDef m_methods9822578 +Ref: 3c359822578 +Ref: c-api/module c PyModuleDef m_slots9822793 +Ref: 3c369822793 +Ref: c-api/module c PyModuleDef m_reload9823170 +Ref: 3c379823170 +Ref: c-api/module c PyModuleDef m_traverse9823217 +Ref: 3c389823217 +Ref: c-api/module c PyModuleDef m_clear9823594 +Ref: 3c3a9823594 +Ref: c-api/module c PyModuleDef m_free9823958 +Ref: 3c3b9823958 +Ref: Initializing C modules-Footnote-19824488 +Node: Single-phase initialization9824537 +Ref: c-api/module single-phase-initialization9824658 +Ref: 3c3c9824658 +Ref: c-api/module c PyModule_Create9824937 +Ref: 38479824937 +Ref: c-api/module c PyModule_Create29825220 +Ref: 3c3d9825220 +Node: Multi-phase initialization9825922 +Ref: c-api/module id19826087 +Ref: 3c3f9826087 +Ref: c-api/module multi-phase-initialization9826087 +Ref: 38b69826087 +Ref: c-api/module c PyModuleDef_Init9827684 +Ref: 3c409827684 +Ref: c-api/module c PyModuleDef_Slot9828135 +Ref: 3c419828135 +Ref: c-api/module c PyModuleDef_Slot slot9828165 +Ref: 3c429828165 +Ref: c-api/module c PyModuleDef_Slot value9828266 +Ref: 3c439828266 +Ref: c-api/module c Py_mod_create9828484 +Ref: 3c449828484 +Ref: c-api/module c create_module9828677 +Ref: 3c459828677 +Ref: c-api/module c Py_mod_exec9830096 +Ref: 3c399830096 +Ref: c-api/module c exec_module9830368 +Ref: 3c469830368 +Ref: Multi-phase initialization-Footnote-19830655 +Ref: Multi-phase initialization-Footnote-29830704 +Node: Low-level module creation functions9830753 +Ref: c-api/module low-level-module-creation-functions9830908 +Ref: 3c479830908 +Ref: c-api/module c PyModule_FromDefAndSpec9831287 +Ref: 7ab9831287 +Ref: c-api/module c PyModule_FromDefAndSpec29831667 +Ref: 7ac9831667 +Ref: c-api/module c PyModule_ExecDef9832290 +Ref: 7ad9832290 +Ref: c-api/module c PyModule_SetDocString9832483 +Ref: 3c489832483 +Ref: c-api/module c PyModule_AddFunctions9832822 +Ref: 3c499832822 +Node: Support functions9833514 +Ref: c-api/module support-functions9833634 +Ref: 3c4a9833634 +Ref: c-api/module c PyModule_AddObject9833920 +Ref: 3c3e9833920 +Ref: c-api/module c PyModule_AddIntConstant9834835 +Ref: 3c4b9834835 +Ref: c-api/module c PyModule_AddStringConstant9835132 +Ref: 3c4c9835132 +Ref: c-api/module c PyModule_AddIntMacro9835493 +Ref: 3c4d9835493 +Ref: c-api/module c PyModule_AddStringMacro9835840 +Ref: d599835840 +Node: Module lookup9835952 +Ref: c-api/module module-lookup9836047 +Ref: 3c4e9836047 +Ref: c-api/module c PyState_FindModule9836466 +Ref: 3c4f9836466 +Ref: c-api/module c PyState_AddModule9836950 +Ref: 3c509836950 +Ref: c-api/module c PyState_RemoveModule9837860 +Ref: 3c519837860 +Node: Iterator Objects9838070 +Ref: c-api/iterator doc9838186 +Ref: 3c529838186 +Ref: c-api/iterator id19838186 +Ref: 3c539838186 +Ref: c-api/iterator iterator-objects9838186 +Ref: 3c549838186 +Ref: c-api/iterator c PySeqIter_Type9838584 +Ref: 3c559838584 +Ref: c-api/iterator c PySeqIter_Check9838818 +Ref: 3c579838818 +Ref: c-api/iterator c PySeqIter_New9838929 +Ref: 3c569838929 +Ref: c-api/iterator c PyCallIter_Type9839207 +Ref: 3c589839207 +Ref: c-api/iterator c PyCallIter_Check9839415 +Ref: 3c5a9839415 +Ref: c-api/iterator c PyCallIter_New9839528 +Ref: 3c599839528 +Node: Descriptor Objects9839958 +Ref: c-api/descriptor doc9840073 +Ref: 3c5b9840073 +Ref: c-api/descriptor descriptor-objects9840073 +Ref: 3c5c9840073 +Ref: c-api/descriptor id19840073 +Ref: 3c5d9840073 +Ref: c-api/descriptor c PyProperty_Type9840254 +Ref: 3c5e9840254 +Ref: c-api/descriptor c PyDescr_NewGetSet9840357 +Ref: 3c5f9840357 +Ref: c-api/descriptor c PyDescr_NewMember9840497 +Ref: 3c609840497 +Ref: c-api/descriptor c PyDescr_NewMethod9840635 +Ref: 3c619840635 +Ref: c-api/descriptor c PyDescr_NewWrapper9840773 +Ref: 3c629840773 +Ref: c-api/descriptor c PyDescr_NewClassMethod9840930 +Ref: 3c639840930 +Ref: c-api/descriptor c PyDescr_IsData9841068 +Ref: 3c649841068 +Ref: c-api/descriptor c PyWrapper_New9841313 +Ref: 3c659841313 +Node: Slice Objects9841415 +Ref: c-api/slice doc9841529 +Ref: 3c669841529 +Ref: c-api/slice id19841529 +Ref: 3c679841529 +Ref: c-api/slice slice-objects9841529 +Ref: 3c689841529 +Ref: c-api/slice c PySlice_Type9841576 +Ref: 3c699841576 +Ref: c-api/slice c PySlice_Check9841725 +Ref: 3c6a9841725 +Ref: c-api/slice c PySlice_New9841849 +Ref: 3c6b9841849 +Ref: c-api/slice c PySlice_GetIndices9842349 +Ref: 3c6c9842349 +Ref: c-api/slice c PySlice_GetIndicesEx9843090 +Ref: 4789843090 +Ref: c-api/slice c PySlice_Unpack9845049 +Ref: 42f9845049 +Ref: c-api/slice c PySlice_AdjustIndices9845627 +Ref: 4309845627 +Node: Ellipsis Object9846067 +Ref: c-api/slice ellipsis-object9846181 +Ref: 3c6d9846181 +Ref: c-api/slice c Py_Ellipsis9846232 +Ref: 3c6e9846232 +Ref: c-api/memoryview memoryview-objects9846492 +Ref: 3a719846492 +Node: MemoryView objects9846493 +Ref: c-api/memoryview doc9846619 +Ref: 3c6f9846619 +Ref: c-api/memoryview id19846619 +Ref: 3c709846619 +Ref: c-api/memoryview c PyMemoryView_FromObject9846833 +Ref: 3c719846833 +Ref: c-api/memoryview c PyMemoryView_FromMemory9847207 +Ref: ac99847207 +Ref: c-api/memoryview c PyMemoryView_FromBuffer9847508 +Ref: 3a759847508 +Ref: c-api/memoryview c PyMemoryView_GetContiguous9847786 +Ref: 3c729847786 +Ref: c-api/memoryview c PyMemoryView_Check9848278 +Ref: 3c739848278 +Ref: c-api/memoryview c PyMemoryView_GET_BUFFER9848477 +Ref: 3c749848477 +Ref: c-api/memoryview c PyMemoryView_GET_BASE9848777 +Ref: 3c759848777 +Node: Weak Reference Objects<2>9849146 +Ref: c-api/weakref doc9849268 +Ref: 3c769849268 +Ref: c-api/weakref weak-reference-objects9849268 +Ref: 3c779849268 +Ref: c-api/weakref weakrefobjects9849268 +Ref: 3c789849268 +Ref: c-api/weakref c PyWeakref_Check9849586 +Ref: 3c799849586 +Ref: c-api/weakref c PyWeakref_CheckRef9849693 +Ref: 3c7a9849693 +Ref: c-api/weakref c PyWeakref_CheckProxy9849787 +Ref: 3c7b9849787 +Ref: c-api/weakref c PyWeakref_NewRef9849879 +Ref: 3c7c9849879 +Ref: c-api/weakref c PyWeakref_NewProxy9850657 +Ref: 3c7d9850657 +Ref: c-api/weakref c PyWeakref_GetObject9851443 +Ref: 3c7e9851443 +Ref: c-api/weakref c PyWeakref_GET_OBJECT9851949 +Ref: 3c7f9851949 +Node: Capsules<2>9852167 +Ref: c-api/capsule doc9852288 +Ref: 3c809852288 +Ref: c-api/capsule capsules9852288 +Ref: 386e9852288 +Ref: c-api/capsule id19852288 +Ref: 3c819852288 +Ref: c-api/capsule c PyCapsule9852455 +Ref: c079852455 +Ref: c-api/capsule c PyCapsule_Destructor9852899 +Ref: 3c829852899 +Ref: c-api/capsule c PyCapsule_CheckExact9853159 +Ref: 3c839853159 +Ref: c-api/capsule c PyCapsule_New9853276 +Ref: 386c9853276 +Ref: c-api/capsule c PyCapsule_GetPointer9854189 +Ref: 3c849854189 +Ref: c-api/capsule c PyCapsule_GetDestructor9854654 +Ref: 3c859854654 +Ref: c-api/capsule c PyCapsule_GetContext9855086 +Ref: 3c869855086 +Ref: c-api/capsule c PyCapsule_GetName9855484 +Ref: 3c879855484 +Ref: c-api/capsule c PyCapsule_Import9855879 +Ref: 386d9855879 +Ref: c-api/capsule c PyCapsule_IsValid9856559 +Ref: cf09856559 +Ref: c-api/capsule c PyCapsule_SetContext9857339 +Ref: 3c889857339 +Ref: c-api/capsule c PyCapsule_SetDestructor9857574 +Ref: 3c899857574 +Ref: c-api/capsule c PyCapsule_SetName9857828 +Ref: 3c8a9857828 +Ref: c-api/capsule c PyCapsule_SetPointer9858211 +Ref: 3c8b9858211 +Node: Generator Objects9858484 +Ref: c-api/gen doc9858600 +Ref: 3c8c9858600 +Ref: c-api/gen gen-objects9858600 +Ref: 3c8d9858600 +Ref: c-api/gen generator-objects9858600 +Ref: 3c8e9858600 +Ref: c-api/gen c PyGenObject9858900 +Ref: 3c919858900 +Ref: c-api/gen c PyGen_Type9858975 +Ref: 3c929858975 +Ref: c-api/gen c PyGen_Check9859074 +Ref: 3c939859074 +Ref: c-api/gen c PyGen_CheckExact9859200 +Ref: 3c949859200 +Ref: c-api/gen c PyGen_New9859350 +Ref: 3c8f9859350 +Ref: c-api/gen c PyGen_NewWithQualName9859617 +Ref: 3c909859617 +Node: Coroutine Objects<2>9860033 +Ref: c-api/coro doc9860163 +Ref: 3c959860163 +Ref: c-api/coro coro-objects9860163 +Ref: 7cd9860163 +Ref: c-api/coro coroutine-objects9860163 +Ref: 3c969860163 +Ref: c-api/coro c PyCoroObject9860322 +Ref: 3c979860322 +Ref: c-api/coro c PyCoro_Type9860398 +Ref: 3c989860398 +Ref: c-api/coro c PyCoro_CheckExact9860498 +Ref: 3c999860498 +Ref: c-api/coro c PyCoro_New9860650 +Ref: 3c9a9860650 +Node: Context Variables Objects9861055 +Ref: c-api/contextvars doc9861187 +Ref: 3c9b9861187 +Ref: c-api/contextvars context-variables-objects9861187 +Ref: 3c9c9861187 +Ref: c-api/contextvars contextvarsobjects9861187 +Ref: 33b9861187 +Ref: c-api/contextvars contextvarsobjects-pointertype-change9861258 +Ref: 4b99861258 +Ref: c-api/contextvars c PyContext9861796 +Ref: 3c9d9861796 +Ref: c-api/contextvars c PyContextVar9861907 +Ref: 3c9e9861907 +Ref: c-api/contextvars c PyContextToken9862024 +Ref: 3c9f9862024 +Ref: c-api/contextvars c PyContext_Type9862126 +Ref: 3ca09862126 +Ref: c-api/contextvars c PyContextVar_Type9862226 +Ref: 3ca19862226 +Ref: c-api/contextvars c PyContextToken_Type9862338 +Ref: 3ca29862338 +Ref: c-api/contextvars c PyContext_CheckExact9862478 +Ref: 3ca39862478 +Ref: c-api/contextvars c PyContextVar_CheckExact9862664 +Ref: 3ca49862664 +Ref: c-api/contextvars c PyContextToken_CheckExact9862856 +Ref: 3ca59862856 +Ref: c-api/contextvars c PyContext_New9863090 +Ref: 3ca69863090 +Ref: c-api/contextvars c PyContext_Copy9863260 +Ref: 3ca79863260 +Ref: c-api/contextvars c PyContext_CopyCurrent9863464 +Ref: 3ca89863464 +Ref: c-api/contextvars c PyContext_Enter9863661 +Ref: 3ca99863661 +Ref: c-api/contextvars c PyContext_Exit9863834 +Ref: 3caa9863834 +Ref: c-api/contextvars c PyContext_ClearFreeList9864063 +Ref: 3cab9864063 +Ref: c-api/contextvars c PyContextVar_New9864262 +Ref: 3cac9864262 +Ref: c-api/contextvars c PyContextVar_Get9864660 +Ref: 3cad9864660 +Ref: c-api/contextvars c PyContextVar_Set9865299 +Ref: 3cae9865299 +Ref: c-api/contextvars c PyContextVar_Reset9865584 +Ref: 3caf9865584 +Ref: Context Variables Objects-Footnote-19865920 +Node: DateTime Objects<2>9865963 +Ref: c-api/datetime doc9866066 +Ref: 3cb09866066 +Ref: c-api/datetime datetime-objects9866066 +Ref: 3cb19866066 +Ref: c-api/datetime datetimeobjects9866066 +Ref: 3cb29866066 +Ref: c-api/datetime c PyDateTime_TimeZone_UTC9866629 +Ref: 4369866629 +Ref: c-api/datetime c PyDate_Check9866841 +Ref: 3cb39866841 +Ref: c-api/datetime c PyDate_CheckExact9867031 +Ref: 3cb49867031 +Ref: c-api/datetime c PyDateTime_Check9867184 +Ref: 3cb59867184 +Ref: c-api/datetime c PyDateTime_CheckExact9867386 +Ref: 3cb69867386 +Ref: c-api/datetime c PyTime_Check9867547 +Ref: 3cb79867547 +Ref: c-api/datetime c PyTime_CheckExact9867737 +Ref: 3cb89867737 +Ref: c-api/datetime c PyDelta_Check9867890 +Ref: 3cb99867890 +Ref: c-api/datetime c PyDelta_CheckExact9868083 +Ref: 3cba9868083 +Ref: c-api/datetime c PyTZInfo_Check9868238 +Ref: 3cbb9868238 +Ref: c-api/datetime c PyTZInfo_CheckExact9868434 +Ref: 3cbc9868434 +Ref: c-api/datetime c PyDate_FromDate9868618 +Ref: 3cbd9868618 +Ref: c-api/datetime c PyDateTime_FromDateAndTime9868816 +Ref: 3cbe9868816 +Ref: c-api/datetime c PyDateTime_FromDateAndTimeAndFold9869136 +Ref: 3cbf9869136 +Ref: c-api/datetime c PyTime_FromTime9869505 +Ref: 3cc09869505 +Ref: c-api/datetime c PyTime_FromTimeAndFold9869747 +Ref: 3cc19869747 +Ref: c-api/datetime c PyDelta_FromDSU9870043 +Ref: 3cc29870043 +Ref: c-api/datetime c PyTimeZone_FromOffset9870461 +Ref: 4349870461 +Ref: c-api/datetime c PyTimeZone_FromOffsetAndName9870737 +Ref: 4359870737 +Ref: c-api/datetime c PyDateTime_GET_YEAR9871281 +Ref: 3cc39871281 +Ref: c-api/datetime c PyDateTime_GET_MONTH9871385 +Ref: 3cc49871385 +Ref: c-api/datetime c PyDateTime_GET_DAY9871501 +Ref: 3cc59871501 +Ref: c-api/datetime c PyDateTime_DATE_GET_HOUR9871815 +Ref: 3cc69871815 +Ref: c-api/datetime c PyDateTime_DATE_GET_MINUTE9871938 +Ref: 3cc79871938 +Ref: c-api/datetime c PyDateTime_DATE_GET_SECOND9872065 +Ref: 3cc89872065 +Ref: c-api/datetime c PyDateTime_DATE_GET_MICROSECOND9872192 +Ref: 3cc99872192 +Ref: c-api/datetime c PyDateTime_TIME_GET_HOUR9872537 +Ref: 3cca9872537 +Ref: c-api/datetime c PyDateTime_TIME_GET_MINUTE9872656 +Ref: 3ccb9872656 +Ref: c-api/datetime c PyDateTime_TIME_GET_SECOND9872779 +Ref: 3ccc9872779 +Ref: c-api/datetime c PyDateTime_TIME_GET_MICROSECOND9872902 +Ref: 3ccd9872902 +Ref: c-api/datetime c PyDateTime_DELTA_GET_DAYS9873240 +Ref: 3cce9873240 +Ref: c-api/datetime c PyDateTime_DELTA_GET_SECONDS9873408 +Ref: 3ccf9873408 +Ref: c-api/datetime c PyDateTime_DELTA_GET_MICROSECONDS9873574 +Ref: 3cd09873574 +Ref: c-api/datetime c PyDateTime_FromTimestamp9873825 +Ref: 3cd19873825 +Ref: c-api/datetime c PyDate_FromTimestamp9874097 +Ref: 3cd29874097 +Node: Initialization Finalization and Threads9874357 +Ref: c-api/init doc9874537 +Ref: 3cd39874537 +Ref: c-api/init initialization9874537 +Ref: 3cd49874537 +Ref: c-api/init initialization-finalization-and-threads9874537 +Ref: 3cd59874537 +Node: Before Python Initialization9875039 +Ref: c-api/init before-python-initialization9875182 +Ref: 3cd69875182 +Ref: c-api/init pre-init-safe9875182 +Ref: 43a9875182 +Node: Global configuration variables9877184 +Ref: c-api/init global-conf-vars9877379 +Ref: 3cd79877379 +Ref: c-api/init global-configuration-variables9877379 +Ref: 3ce29877379 +Ref: c-api/init c Py_BytesWarningFlag9877850 +Ref: 4b49877850 +Ref: c-api/init c Py_DebugFlag9878132 +Ref: 3ce39878132 +Ref: c-api/init c Py_DontWriteBytecodeFlag9878360 +Ref: 3ce49878360 +Ref: c-api/init c Py_FrozenFlag9878625 +Ref: 3ce59878625 +Ref: c-api/init c Py_HashRandomizationFlag9878846 +Ref: 3ce69878846 +Ref: c-api/init c Py_IgnoreEnvironmentFlag9879132 +Ref: 4b89879132 +Ref: c-api/init c Py_InspectFlag9879372 +Ref: 3ce79879372 +Ref: c-api/init c Py_InteractiveFlag9879741 +Ref: 39899879741 +Ref: c-api/init c Py_IsolatedFlag9879821 +Ref: 3ce89879821 +Ref: c-api/init c Py_LegacyWindowsFSEncodingFlag9880092 +Ref: 398e9880092 +Ref: c-api/init c Py_LegacyWindowsStdioFlag9880473 +Ref: 3ce99880473 +Ref: c-api/init c Py_NoSiteFlag9880854 +Ref: 3cea9880854 +Ref: c-api/init c Py_NoUserSiteDirectory9881229 +Ref: 3ceb9881229 +Ref: c-api/init c Py_OptimizeFlag9881489 +Ref: 1fdf9881489 +Ref: c-api/init c Py_QuietFlag9881627 +Ref: 3cec9881627 +Ref: c-api/init c Py_UnbufferedStdioFlag9881815 +Ref: 3ced9881815 +Ref: c-api/init c Py_VerboseFlag9882022 +Ref: 3cee9882022 +Ref: Global configuration variables-Footnote-19882510 +Ref: Global configuration variables-Footnote-29882559 +Node: Initializing and finalizing the interpreter9882608 +Ref: c-api/init initializing-and-finalizing-the-interpreter9882798 +Ref: 3cef9882798 +Ref: c-api/init c Py_Initialize9882899 +Ref: 4399882899 +Ref: c-api/init c Py_InitializeEx9883870 +Ref: 3cf09883870 +Ref: c-api/init c Py_IsInitialized9884149 +Ref: 39019884149 +Ref: c-api/init c Py_FinalizeEx9884420 +Ref: 5c99884420 +Ref: c-api/init c Py_Finalize9886555 +Ref: ced9886555 +Node: Process-wide parameters9886707 +Ref: c-api/init process-wide-parameters9886911 +Ref: 3cf29886911 +Ref: c-api/init c Py_SetStandardStreamEncoding9886972 +Ref: 9259886972 +Ref: c-api/init c Py_SetProgramName9888093 +Ref: 10319888093 +Ref: c-api/init c Py_GetProgramName9889007 +Ref: 2769889007 +Ref: c-api/init c Py_GetPrefix9889239 +Ref: 38ff9889239 +Ref: c-api/init c Py_GetExecPrefix9890013 +Ref: 39009890013 +Ref: c-api/init c Py_GetProgramFullPath9892183 +Ref: 2759892183 +Ref: c-api/init c Py_GetPath9892621 +Ref: 38fe9892621 +Ref: c-api/init c Py_SetPath9893332 +Ref: 2739893332 +Ref: c-api/init c Py_GetVersion9894552 +Ref: 3ce09894552 +Ref: c-api/init c Py_GetPlatform9895114 +Ref: 3cdf9895114 +Ref: c-api/init c Py_GetCopyright9895707 +Ref: 3cde9895707 +Ref: c-api/init c Py_GetCompiler9896090 +Ref: 3cdd9896090 +Ref: c-api/init c Py_GetBuildInfo9896473 +Ref: d9a9896473 +Ref: c-api/init c PySys_SetArgvEx9896883 +Ref: bfa9896883 +Ref: c-api/init c PySys_SetArgv9898860 +Ref: cec9898860 +Ref: c-api/init c Py_SetPythonHome9899284 +Ref: 3cda9899284 +Ref: c-api/init c Py_GetPythonHome9899882 +Ref: 3ce19899882 +Ref: Process-wide parameters-Footnote-19900165 +Node: Thread State and the Global Interpreter Lock9900234 +Ref: c-api/init thread-state-and-the-global-interpreter-lock9900418 +Ref: 3cf39900418 +Ref: c-api/init threads9900418 +Ref: 3cf49900418 +Node: Releasing the GIL from extension code9901961 +Ref: c-api/init releasing-the-gil-from-extension-code9902114 +Ref: 3cf79902114 +Node: Non-Python created threads9903989 +Ref: c-api/init gilstate9904170 +Ref: 3cf89904170 +Ref: c-api/init non-python-created-threads9904170 +Ref: 3cf99904170 +Node: Cautions about fork9905928 +Ref: c-api/init cautions-about-fork9906086 +Ref: 3cfb9906086 +Ref: c-api/init fork-and-threads9906086 +Ref: 398a9906086 +Node: High-level API9908025 +Ref: c-api/init high-level-api9908170 +Ref: 3cfc9908170 +Ref: c-api/init c PyInterpreterState9908344 +Ref: 2c29908344 +Ref: c-api/init c PyThreadState9908911 +Ref: 3cf59908911 +Ref: c-api/init c PyEval_InitThreads9909147 +Ref: c129909147 +Ref: c-api/init c PyEval_ThreadsInitialized9909847 +Ref: dd99909847 +Ref: c-api/init c PyEval_SaveThread9910245 +Ref: c119910245 +Ref: c-api/init c PyEval_RestoreThread9910558 +Ref: 2b49910558 +Ref: c-api/init c PyThreadState_Get9911280 +Ref: 3cf69911280 +Ref: c-api/init c PyThreadState_Swap9911550 +Ref: 3cfd9911550 +Ref: c-api/init c PyGILState_Ensure9911920 +Ref: 2b69911920 +Ref: c-api/init c PyGILState_Release9913649 +Ref: 3cfa9913649 +Ref: c-api/init c PyGILState_GetThisThreadState9914137 +Ref: 3cfe9914137 +Ref: c-api/init c PyGILState_Check9914526 +Ref: 3cff9914526 +Ref: c-api/init c Py_BEGIN_ALLOW_THREADS9915259 +Ref: 38669915259 +Ref: c-api/init c Py_END_ALLOW_THREADS9915565 +Ref: 2b59915565 +Ref: c-api/init c Py_BLOCK_THREADS9915848 +Ref: 3d009915848 +Ref: c-api/init c Py_UNBLOCK_THREADS9916033 +Ref: 3d019916033 +Node: Low-level API9916248 +Ref: c-api/init low-level-api9916365 +Ref: 3d029916365 +Ref: c-api/init c PyInterpreterState_New9916580 +Ref: 31ef9916580 +Ref: c-api/init c PyInterpreterState_Clear9916921 +Ref: 31ee9916921 +Ref: c-api/init c PyInterpreterState_Delete9917221 +Ref: 3d039917221 +Ref: c-api/init c PyThreadState_New9917516 +Ref: 3d049917516 +Ref: c-api/init c PyThreadState_Clear9917816 +Ref: 3d059917816 +Ref: c-api/init c PyThreadState_Delete9917984 +Ref: 3d069917984 +Ref: c-api/init c PyInterpreterState_GetID9918243 +Ref: 43b9918243 +Ref: c-api/init c PyInterpreterState_GetDict9918492 +Ref: 3d079918492 +Ref: c-api/init c PyThreadState_GetDict9919001 +Ref: 3d089919001 +Ref: c-api/init c PyThreadState_SetAsyncExc9919502 +Ref: 4379919502 +Ref: c-api/init c PyEval_AcquireThread9920297 +Ref: 2b39920297 +Ref: c-api/init c PyEval_ReleaseThread9921404 +Ref: 3d099921404 +Ref: c-api/init c PyEval_AcquireLock9921973 +Ref: 2b29921973 +Ref: c-api/init c PyEval_ReleaseLock9923027 +Ref: c109923027 +Node: Sub-interpreter support9923355 +Ref: c-api/init id19923542 +Ref: 3d0a9923542 +Ref: c-api/init sub-interpreter-support9923542 +Ref: 398b9923542 +Ref: c-api/init c Py_NewInterpreter9924473 +Ref: 3cf19924473 +Ref: c-api/init c Py_EndInterpreter9927485 +Ref: 3d0d9927485 +Node: Bugs and caveats9928154 +Ref: c-api/init bugs-and-caveats9928230 +Ref: 3d0c9928230 +Node: Asynchronous Notifications9929839 +Ref: c-api/init asynchronous-notifications9930003 +Ref: 3d0e9930003 +Ref: c-api/init c Py_AddPendingCall9930248 +Ref: ce99930248 +Node: Profiling and Tracing9931907 +Ref: c-api/init profiling9932073 +Ref: 3d0f9932073 +Ref: c-api/init profiling-and-tracing9932073 +Ref: 3d109932073 +Ref: c-api/init c Py_tracefunc9932752 +Ref: 3d119932752 +Ref: c-api/init c PyTrace_CALL9934997 +Ref: 3d129934997 +Ref: c-api/init c PyTrace_EXCEPTION9935387 +Ref: 3d139935387 +Ref: c-api/init c PyTrace_LINE9935997 +Ref: 3d149935997 +Ref: c-api/init c PyTrace_RETURN9936293 +Ref: 3d159936293 +Ref: c-api/init c PyTrace_C_CALL9936446 +Ref: 3d169936446 +Ref: c-api/init c PyTrace_C_EXCEPTION9936608 +Ref: 3d179936608 +Ref: c-api/init c PyTrace_C_RETURN9936777 +Ref: 3d189936777 +Ref: c-api/init c PyTrace_OPCODE9936932 +Ref: 3d199936932 +Ref: c-api/init c PyEval_SetProfile9937266 +Ref: e3b9937266 +Ref: c-api/init c PyEval_SetTrace9937830 +Ref: e3c9937830 +Node: Advanced Debugger Support9938380 +Ref: c-api/init advanced-debugger-support9938548 +Ref: 3d1a9938548 +Ref: c-api/init advanced-debugging9938548 +Ref: 3d1b9938548 +Ref: c-api/init c PyInterpreterState_Head9938688 +Ref: e3f9938688 +Ref: c-api/init c PyInterpreterState_Main9938844 +Ref: 3d0b9938844 +Ref: c-api/init c PyInterpreterState_Next9938956 +Ref: e409938956 +Ref: c-api/init c PyInterpreterState_ThreadHead9939158 +Ref: e419939158 +Ref: c-api/init c PyThreadState_Next9939402 +Ref: e429939402 +Node: Thread Local Storage Support9939649 +Ref: c-api/init thread-local-storage9939787 +Ref: 3d1c9939787 +Ref: c-api/init thread-local-storage-support9939787 +Ref: 3d1d9939787 +Node: Thread Specific Storage TSS API9940913 +Ref: c-api/init thread-specific-storage-api9941046 +Ref: 3179941046 +Ref: c-api/init thread-specific-storage-tss-api9941046 +Ref: 3d1e9941046 +Ref: c-api/init c Py_tss_t9941436 +Ref: 3189941436 +Ref: c-api/init c Py_tss_NEEDS_INIT9941876 +Ref: 3d1f9941876 +Ref: Thread Specific Storage TSS API-Footnote-19942164 +Node: Dynamic Allocation9942213 +Ref: c-api/init dynamic-allocation9942318 +Ref: 3d209942318 +Ref: c-api/init c PyThread_tss_alloc9942599 +Ref: 3d219942599 +Ref: c-api/init c PyThread_tss_free9942819 +Ref: 3d229942819 +Node: Methods<4>9943241 +Ref: c-api/init methods9943346 +Ref: 3d249943346 +Ref: c-api/init c PyThread_tss_is_created9943652 +Ref: 3d289943652 +Ref: c-api/init c PyThread_tss_create9943841 +Ref: 3d279943841 +Ref: c-api/init c PyThread_tss_delete9944258 +Ref: 3d239944258 +Ref: c-api/init c PyThread_tss_set9944690 +Ref: 3d259944690 +Ref: c-api/init c PyThread_tss_get9944959 +Ref: 3d269944959 +Node: Thread Local Storage TLS API9945196 +Ref: c-api/init thread-local-storage-api9945329 +Ref: 3169945329 +Ref: c-api/init thread-local-storage-tls-api9945329 +Ref: 3d299945329 +Ref: c-api/init c PyThread_create_key9945953 +Ref: 3d2a9945953 +Ref: c-api/init c PyThread_delete_key9945997 +Ref: 3d2b9945997 +Ref: c-api/init c PyThread_set_key_value9946049 +Ref: 9709946049 +Ref: c-api/init c PyThread_get_key_value9946116 +Ref: 3d2c9946116 +Ref: c-api/init c PyThread_delete_key_value9946172 +Ref: 3d2d9946172 +Ref: c-api/init c PyThread_ReInitTLS9946230 +Ref: 3d2e9946230 +Node: Python Initialization Configuration9946274 +Ref: c-api/init_config doc9946449 +Ref: 3d2f9946449 +Ref: c-api/init_config init-config9946449 +Ref: 17d9946449 +Ref: c-api/init_config python-initialization-configuration9946449 +Ref: 3d309946449 +Ref: Python Initialization Configuration-Footnote-19948414 +Node: PyWideStringList9948463 +Ref: c-api/init_config pywidestringlist9948568 +Ref: 3d329948568 +Ref: c-api/init_config c PyWideStringList9948617 +Ref: 1629948617 +Ref: c-api/init_config c PyWideStringList_Append9948807 +Ref: 1749948807 +Ref: c-api/init_config c PyWideStringList_Insert9949023 +Ref: 1759949023 +Ref: c-api/init_config c PyWideStringList length9949464 +Ref: 3d339949464 +Ref: c-api/init_config c PyWideStringList items9949526 +Ref: 3d349949526 +Node: PyStatus9949585 +Ref: c-api/init_config pystatus9949710 +Ref: 3d359949710 +Ref: c-api/init_config c PyStatus9949743 +Ref: 1619949743 +Ref: c-api/init_config c PyStatus exitcode9949961 +Ref: 3d369949961 +Ref: c-api/init_config c PyStatus err_msg9950050 +Ref: 3d379950050 +Ref: c-api/init_config c PyStatus func9950116 +Ref: 3d389950116 +Ref: c-api/init_config c PyStatus_Ok9950264 +Ref: 1739950264 +Ref: c-api/init_config c PyStatus_Error9950334 +Ref: 16d9950334 +Ref: c-api/init_config c PyStatus_NoMemory9950450 +Ref: 1729950450 +Ref: c-api/init_config c PyStatus_Exit9950560 +Ref: 16f9950560 +Ref: c-api/init_config c PyStatus_Exception9950709 +Ref: 16e9950709 +Ref: c-api/init_config c PyStatus_IsError9950940 +Ref: 1709950940 +Ref: c-api/init_config c PyStatus_IsExit9951036 +Ref: 1719951036 +Ref: c-api/init_config c Py_ExitStatusException9951130 +Ref: 1779951130 +Node: PyPreConfig9952071 +Ref: c-api/init_config pypreconfig9952214 +Ref: 3d399952214 +Ref: c-api/init_config c PyPreConfig9952253 +Ref: 1609952253 +Ref: c-api/init_config c PyPreConfig_InitPythonConfig9952487 +Ref: 16c9952487 +Ref: c-api/init_config c PyPreConfig_InitIsolatedConfig9952675 +Ref: 16b9952675 +Ref: c-api/init_config c PyPreConfig allocator9952891 +Ref: 3d3c9952891 +Ref: c-api/init_config c PyPreConfig configure_locale9953988 +Ref: 3d3e9953988 +Ref: c-api/init_config c PyPreConfig coerce_c_locale9954194 +Ref: 3d3f9954194 +Ref: c-api/init_config c PyPreConfig coerce_c_locale_warn9954368 +Ref: 3d409954368 +Ref: c-api/init_config c PyPreConfig dev_mode9954480 +Ref: 3d419954480 +Ref: c-api/init_config c PyPreConfig isolated9954559 +Ref: 3d439954559 +Ref: c-api/init_config c PyPreConfig legacy_windows_fs_encoding9954638 +Ref: 3d459954638 +Ref: c-api/init_config c PyPreConfig parse_argv9954987 +Ref: 3d469954987 +Ref: c-api/init_config c PyPreConfig use_environment9955297 +Ref: 3d479955297 +Ref: c-api/init_config c PyPreConfig utf8_mode9955390 +Ref: 3d499955390 +Node: Preinitialization with PyPreConfig9955471 +Ref: c-api/init_config preinitialization-with-pypreconfig9955614 +Ref: 3d4a9955614 +Ref: c-api/init_config c Py_PreInitialize9955735 +Ref: 1799955735 +Ref: c-api/init_config c Py_PreInitializeFromBytesArgs9955871 +Ref: 17b9955871 +Ref: c-api/init_config c Py_PreInitializeFromArgs9956108 +Ref: 17a9956108 +Node: PyConfig9957879 +Ref: c-api/init_config pyconfig9958039 +Ref: 3d4b9958039 +Ref: c-api/init_config c PyConfig9958072 +Ref: 15f9958072 +Ref: c-api/init_config c PyConfig_InitPythonConfig9958183 +Ref: 1659958183 +Ref: c-api/init_config c PyConfig_InitIsolatedConfig9958340 +Ref: 1649958340 +Ref: c-api/init_config c PyConfig_SetString9958501 +Ref: 16a9958501 +Ref: c-api/init_config c PyConfig_SetBytesString9958749 +Ref: 1699958749 +Ref: c-api/init_config c PyConfig_SetArgv9959034 +Ref: 1679959034 +Ref: c-api/init_config c PyConfig_SetBytesArgv9959259 +Ref: 1689959259 +Ref: c-api/init_config c PyConfig_SetWideStringList9959517 +Ref: 3d319959517 +Ref: c-api/init_config c PyConfig_Read9959799 +Ref: 1669959799 +Ref: c-api/init_config c PyConfig_Clear9960016 +Ref: 1639960016 +Ref: c-api/init_config c PyConfig argv9961062 +Ref: 3d4d9961062 +Ref: c-api/init_config c PyConfig base_exec_prefix9961453 +Ref: 3d4e9961453 +Ref: c-api/init_config c PyConfig base_executable9961544 +Ref: 3d4f9961544 +Ref: c-api/init_config c PyConfig base_prefix9961736 +Ref: 3d519961736 +Ref: c-api/init_config c PyConfig buffered_stdio9961817 +Ref: 3d529961817 +Ref: c-api/init_config c PyConfig bytes_warning9962018 +Ref: 3d539962018 +Ref: c-api/init_config c PyConfig check_hash_pycs_mode9962331 +Ref: 3d549962331 +Ref: c-api/init_config c PyConfig configure_c_stdio9962675 +Ref: 3d559962675 +Ref: c-api/init_config c PyConfig dev_mode9962892 +Ref: 3d429962892 +Ref: c-api/init_config c PyConfig dump_refs9962977 +Ref: 3d569962977 +Ref: c-api/init_config c PyConfig exec_prefix9963180 +Ref: 3d579963180 +Ref: c-api/init_config c PyConfig executable9963261 +Ref: 3d509963261 +Ref: c-api/init_config c PyConfig faulthandler9963339 +Ref: 3d589963339 +Ref: c-api/init_config c PyConfig filesystem_encoding9963461 +Ref: 3d599963461 +Ref: c-api/init_config c PyConfig filesystem_errors9963582 +Ref: 3d5a9963582 +Ref: c-api/init_config c PyConfig hash_seed9963722 +Ref: 3d5b9963722 +Ref: c-api/init_config c PyConfig use_hash_seed9963766 +Ref: 3d5c9963766 +Ref: c-api/init_config c PyConfig home9963999 +Ref: 3d5d9963999 +Ref: c-api/init_config c PyConfig import_time9964167 +Ref: 3d5e9964167 +Ref: c-api/init_config c PyConfig inspect9964248 +Ref: 3d5f9964248 +Ref: c-api/init_config c PyConfig install_signal_handlers9964353 +Ref: 3d609964353 +Ref: c-api/init_config c PyConfig interactive9964437 +Ref: 3d619964437 +Ref: c-api/init_config c PyConfig isolated9964502 +Ref: 3d449964502 +Ref: c-api/init_config c PyConfig legacy_windows_stdio9965046 +Ref: 3d639965046 +Ref: c-api/init_config c PyConfig malloc_stats9965392 +Ref: 3d649965392 +Ref: c-api/init_config c PyConfig pythonpath_env9965630 +Ref: 3d659965630 +Ref: c-api/init_config c PyConfig module_search_paths9965877 +Ref: 3d669965877 +Ref: c-api/init_config c PyConfig module_search_paths_set9965934 +Ref: 3d679965934 +Ref: c-api/init_config c PyConfig optimization_level9966215 +Ref: 3d699966215 +Ref: c-api/init_config c PyConfig parse_argv9966491 +Ref: 3d4c9966491 +Ref: c-api/init_config c PyConfig parser_debug9966740 +Ref: 3d6a9966740 +Ref: c-api/init_config c PyConfig pathconfig_warnings9966896 +Ref: 3d6b9966896 +Ref: c-api/init_config c PyConfig prefix9967156 +Ref: 3d6c9967156 +Ref: c-api/init_config c PyConfig program_name9967227 +Ref: 3d6d9967227 +Ref: c-api/init_config c PyConfig pycache_prefix9967377 +Ref: 3d6e9967377 +Ref: c-api/init_config c PyConfig quiet9967568 +Ref: 3d6f9967568 +Ref: c-api/init_config c PyConfig run_command9967717 +Ref: 3d709967717 +Ref: c-api/init_config c PyConfig run_filename9967848 +Ref: 3d719967848 +Ref: c-api/init_config c PyConfig run_module9967968 +Ref: 3d729967968 +Ref: c-api/init_config c PyConfig show_alloc_count9968097 +Ref: 3d739968097 +Ref: c-api/init_config c PyConfig show_ref_count9968333 +Ref: 3d749968333 +Ref: c-api/init_config c PyConfig site_import9968585 +Ref: 3d759968585 +Ref: c-api/init_config c PyConfig skip_source_first_line9968678 +Ref: 3d769968678 +Ref: c-api/init_config c PyConfig stdio_encoding9968771 +Ref: 3d779968771 +Ref: c-api/init_config c PyConfig stdio_errors9968815 +Ref: 3d789968815 +Ref: c-api/init_config c PyConfig tracemalloc9968982 +Ref: 3d799968982 +Ref: c-api/init_config c PyConfig use_environment9969091 +Ref: 3d489969091 +Ref: c-api/init_config c PyConfig user_site_directory9969199 +Ref: 3d629969199 +Ref: c-api/init_config c PyConfig verbose9969315 +Ref: 3d7a9969315 +Ref: c-api/init_config c PyConfig warnoptions9969392 +Ref: 3d7b9969392 +Ref: c-api/init_config c PyConfig write_bytecode9969867 +Ref: 3d7c9969867 +Ref: c-api/init_config c PyConfig xoptions9970080 +Ref: 3d7d9970080 +Ref: PyConfig-Footnote-19970506 +Node: Initialization with PyConfig9970555 +Ref: c-api/init_config initialization-with-pyconfig9970703 +Ref: 3d7e9970703 +Ref: c-api/init_config c Py_InitializeFromConfig9970808 +Ref: 1789970808 +Node: Isolated Configuration9973522 +Ref: c-api/init_config init-isolated-conf9973682 +Ref: 3d3b9973682 +Ref: c-api/init_config isolated-configuration9973682 +Ref: 3d7f9973682 +Node: Python Configuration9974477 +Ref: c-api/init_config init-python-config9974627 +Ref: 3d3a9974627 +Ref: c-api/init_config python-configuration9974627 +Ref: 3d809974627 +Ref: Python Configuration-Footnote-19976203 +Ref: Python Configuration-Footnote-29976252 +Node: Path Configuration9976301 +Ref: c-api/init_config init-path-config9976439 +Ref: 3d689976439 +Ref: c-api/init_config path-configuration9976439 +Ref: 3d819976439 +Node: Py_RunMain9979871 +Ref: c-api/init_config py-runmain9980039 +Ref: 3d829980039 +Ref: c-api/init_config c Py_RunMain9980082 +Ref: 17c9980082 +Node: Multi-Phase Initialization Private Provisional API9980676 +Ref: c-api/init_config multi-phase-initialization-private-provisional-api9980817 +Ref: 3d839980817 +Ref: c-api/init_config c _Py_InitializeMain9982014 +Ref: 3d849982014 +Ref: Multi-Phase Initialization Private Provisional API-Footnote-19984033 +Ref: Multi-Phase Initialization Private Provisional API-Footnote-29984082 +Node: Memory Management9984131 +Ref: c-api/memory doc9984296 +Ref: 3d859984296 +Ref: c-api/memory memory9984296 +Ref: 3d3d9984296 +Ref: c-api/memory memory-management9984296 +Ref: 3d869984296 +Node: Overview<3>9984583 +Ref: c-api/memory memoryoverview9984677 +Ref: 3d879984677 +Ref: c-api/memory overview9984677 +Ref: 3d889984677 +Node: Raw Memory Interface9988592 +Ref: c-api/memory raw-memory-interface9988711 +Ref: 3d899988711 +Ref: c-api/memory c PyMem_RawMalloc9989151 +Ref: 96d9989151 +Ref: c-api/memory c PyMem_RawCalloc9989539 +Ref: 7a59989539 +Ref: c-api/memory c PyMem_RawRealloc9990040 +Ref: 96e9990040 +Ref: c-api/memory c PyMem_RawFree9990789 +Ref: 398f9990789 +Node: Memory Interface9991195 +Ref: c-api/memory memory-interface9991320 +Ref: 3d8a9991320 +Ref: c-api/memory memoryinterface9991320 +Ref: 3d8b9991320 +Ref: c-api/memory c PyMem_Malloc9991825 +Ref: 5059991825 +Ref: c-api/memory c PyMem_Calloc9992207 +Ref: 7a69992207 +Ref: c-api/memory c PyMem_Realloc9992702 +Ref: 96f9992702 +Ref: c-api/memory c PyMem_Free9993433 +Ref: da99993433 +Ref: c-api/memory c PyMem_New9993931 +Ref: 3d8c9993931 +Ref: c-api/memory c PyMem_Resize9994182 +Ref: 3d8d9994182 +Ref: c-api/memory c PyMem_Del9994650 +Ref: 3d8e9994650 +Node: Object allocators9995226 +Ref: c-api/memory object-allocators9995356 +Ref: 3d8f9995356 +Ref: c-api/memory c PyObject_Malloc9995766 +Ref: 5089995766 +Ref: c-api/memory c PyObject_Calloc9996154 +Ref: 7a79996154 +Ref: c-api/memory c PyObject_Realloc9996655 +Ref: daa9996655 +Ref: c-api/memory c PyObject_Free9997404 +Ref: 5049997404 +Node: Default Memory Allocators9997810 +Ref: c-api/memory default-memory-allocators9997951 +Ref: ff89997951 +Ref: c-api/memory id19997951 +Ref: 3d909997951 +Node: Customize Memory Allocators10000247 +Ref: c-api/memory customize-memory-allocators10000393 +Ref: 3d9110000393 +Ref: c-api/memory c PyMemAllocatorEx10000485 +Ref: 7ca10000485 +Ref: c-api/memory c PyMemAllocatorDomain10002305 +Ref: 3d9210002305 +Ref: c-api/memory c PYMEM_DOMAIN_RAW10002398 +Ref: ff910002398 +Ref: c-api/memory c PYMEM_DOMAIN_MEM10002640 +Ref: 50910002640 +Ref: c-api/memory c PYMEM_DOMAIN_OBJ10002869 +Ref: 50710002869 +Ref: c-api/memory c PyMem_GetAllocator10003110 +Ref: 3cdb10003110 +Ref: c-api/memory c PyMem_SetAllocator10003282 +Ref: 3cd810003282 +Ref: c-api/memory c PyMem_SetupDebugHooks10003928 +Ref: 50a10003928 +Node: The pymalloc allocator10006063 +Ref: c-api/memory pymalloc10006201 +Ref: 5c110006201 +Ref: c-api/memory the-pymalloc-allocator10006201 +Ref: 3d9310006201 +Node: Customize pymalloc Arena Allocator10007036 +Ref: c-api/memory customize-pymalloc-arena-allocator10007129 +Ref: 3d9410007129 +Ref: c-api/memory c PyObjectArenaAllocator10007239 +Ref: 3d9510007239 +Ref: c-api/memory c PyObject_GetArenaAllocator10008243 +Ref: 3cdc10008243 +Ref: c-api/memory c PyObject_SetArenaAllocator10008369 +Ref: 3cd910008369 +Node: tracemalloc C API10008495 +Ref: c-api/memory tracemalloc-c-api10008618 +Ref: 3d9610008618 +Ref: c-api/memory c PyTraceMalloc_Track10008690 +Ref: 42410008690 +Ref: c-api/memory c PyTraceMalloc_Untrack10009103 +Ref: 42510009103 +Node: Examples<34>10009395 +Ref: c-api/memory examples10009487 +Ref: 3d9710009487 +Ref: c-api/memory memoryexamples10009487 +Ref: 3d9810009487 +Node: Object Implementation Support10011400 +Ref: c-api/objimpl doc10011552 +Ref: 3d9910011552 +Ref: c-api/objimpl newtypes10011552 +Ref: 3d9a10011552 +Ref: c-api/objimpl object-implementation-support10011552 +Ref: 3d9b10011552 +Node: Allocating Objects on the Heap10012061 +Ref: c-api/allocation doc10012190 +Ref: 3d9c10012190 +Ref: c-api/allocation allocating-objects10012190 +Ref: 3d9d10012190 +Ref: c-api/allocation allocating-objects-on-the-heap10012190 +Ref: 3d9e10012190 +Ref: c-api/allocation c _PyObject_New10012267 +Ref: 3d9f10012267 +Ref: c-api/allocation c _PyObject_NewVar10012365 +Ref: 3da010012365 +Ref: c-api/allocation c PyObject_Init10012496 +Ref: 26f10012496 +Ref: c-api/allocation c PyObject_InitVar10012945 +Ref: 3da110012945 +Ref: c-api/allocation c PyObject_New10013236 +Ref: 2d210013236 +Ref: c-api/allocation c PyObject_NewVar10013681 +Ref: 2d310013681 +Ref: c-api/allocation c PyObject_Del10014408 +Ref: e1610014408 +Ref: c-api/allocation c _Py_NoneStruct10014799 +Ref: 3da210014799 +Node: Common Object Structures10015117 +Ref: c-api/structures doc10015270 +Ref: 3da310015270 +Ref: c-api/structures common-object-structures10015270 +Ref: 3da410015270 +Ref: c-api/structures common-structs10015270 +Ref: 3da510015270 +Ref: c-api/structures c PyObject10015861 +Ref: 4ba10015861 +Ref: c-api/structures c PyVarObject10016445 +Ref: 3da610016445 +Ref: c-api/structures c PyObject_HEAD10016825 +Ref: c7210016825 +Ref: c-api/structures c PyObject_VAR_HEAD10017086 +Ref: 3da810017086 +Ref: c-api/structures c Py_TYPE10017389 +Ref: 387610017389 +Ref: c-api/structures c Py_REFCNT10017555 +Ref: 387510017555 +Ref: c-api/structures c Py_SIZE10017727 +Ref: 3da710017727 +Ref: c-api/structures c PyObject_HEAD_INIT10017896 +Ref: 3da910017896 +Ref: c-api/structures c PyVarObject_HEAD_INIT10018114 +Ref: 3daa10018114 +Ref: c-api/structures c PyCFunction10018391 +Ref: ddb10018391 +Ref: c-api/structures c PyCFunctionWithKeywords10018830 +Ref: 3dab10018830 +Ref: c-api/structures c _PyCFunctionFast10018991 +Ref: 3dac10018991 +Ref: c-api/structures c _PyCFunctionFastWithKeywords10019136 +Ref: 3dae10019136 +Ref: c-api/structures c PyMethodDef10019303 +Ref: e1b10019303 +Ref: c-api/structures METH_VARARGS10021255 +Ref: e4810021255 +Ref: c-api/structures METH_FASTCALL10022184 +Ref: 3dad10022184 +Ref: c-api/structures METH_NOARGS10023288 +Ref: e1810023288 +Ref: c-api/structures METH_O10023687 +Ref: e4710023687 +Ref: c-api/structures METH_CLASS10024276 +Ref: e1910024276 +Ref: c-api/structures METH_STATIC10024549 +Ref: e1a10024549 +Ref: c-api/structures METH_COEXIST10024934 +Ref: 3daf10024934 +Ref: c-api/structures c PyMemberDef10025600 +Ref: 42610025600 +Ref: c-api/structures c PyGetSetDef10029724 +Ref: 42710029724 +Node: Type Objects<3>10031858 +Ref: c-api/typeobj doc10032005 +Ref: 3db010032005 +Ref: c-api/typeobj type-objects10032005 +Ref: 3db110032005 +Ref: c-api/typeobj type-structs10032005 +Ref: 387710032005 +Node: Quick Reference10033234 +Ref: c-api/typeobj quick-reference10033333 +Ref: 3db310033333 +Node: “tp slots”10033445 +Ref: c-api/typeobj tp-slots10033529 +Ref: 3db410033529 +Ref: c-api/typeobj tp-slots-table10033529 +Ref: 3db510033529 +Ref: “tp slots”-Footnote-110046725 +Ref: “tp slots”-Footnote-210046994 +Node: sub-slots10047783 +Ref: c-api/typeobj id310047889 +Ref: 3ddc10047889 +Ref: c-api/typeobj sub-slots10047889 +Ref: 3dbb10047889 +Node: slot typedefs10058074 +Ref: c-api/typeobj slot-typedefs10058157 +Ref: 3e1610058157 +Ref: c-api/typeobj slot-typedefs-table10058157 +Ref: 3db610058157 +Node: PyTypeObject Definition10065957 +Ref: c-api/typeobj pytypeobject-definition10066079 +Ref: 3e1910066079 +Node: PyObject Slots10068904 +Ref: c-api/typeobj pyobject-slots10069028 +Ref: 3e1a10069028 +Ref: c-api/typeobj c PyObject _ob_next10069450 +Ref: 3e1b10069450 +Ref: c-api/typeobj c PyObject _ob_prev10069492 +Ref: 3e1c10069492 +Ref: c-api/typeobj c PyObject ob_refcnt10070242 +Ref: 3e1d10070242 +Ref: c-api/typeobj c PyObject ob_type10070735 +Ref: 3e1e10070735 +Node: PyVarObject Slots10071822 +Ref: c-api/typeobj pyvarobject-slots10071941 +Ref: 3e1f10071941 +Ref: c-api/typeobj c PyVarObject ob_size10071996 +Ref: 3e2010071996 +Node: PyTypeObject Slots10072285 +Ref: c-api/typeobj pytypeobject-slots10072400 +Ref: 3e2110072400 +Ref: c-api/typeobj c PyTypeObject tp_name10072753 +Ref: 389b10072753 +Ref: c-api/typeobj c PyTypeObject tp_basicsize10074458 +Ref: 387910074458 +Ref: c-api/typeobj c PyTypeObject tp_itemsize10074509 +Ref: 387810074509 +Ref: c-api/typeobj c PyTypeObject tp_dealloc10077162 +Ref: 387f10077162 +Ref: c-api/typeobj c PyTypeObject tp_vectorcall_offset10079214 +Ref: 3a1110079214 +Ref: c-api/typeobj c PyVectorcall_Call10080348 +Ref: 3e2410080348 +Ref: c-api/typeobj c PyTypeObject tp_getattr10081975 +Ref: 38a210081975 +Ref: c-api/typeobj c PyTypeObject tp_setattr10082696 +Ref: 38a310082696 +Ref: c-api/typeobj c PyTypeObject tp_as_async10083437 +Ref: 3dba10083437 +Ref: c-api/typeobj c PyTypeObject tp_repr10083979 +Ref: 389a10083979 +Ref: c-api/typeobj c PyTypeObject tp_as_number10084928 +Ref: 3dbd10084928 +Ref: c-api/typeobj c PyTypeObject tp_as_sequence10085329 +Ref: 3dbe10085329 +Ref: c-api/typeobj c PyTypeObject tp_as_mapping10085740 +Ref: 3dbf10085740 +Ref: c-api/typeobj c PyTypeObject tp_hash10086146 +Ref: 38ac10086146 +Ref: c-api/typeobj c PyTypeObject tp_call10087734 +Ref: 38ad10087734 +Ref: c-api/typeobj c PyTypeObject tp_str10088134 +Ref: 389c10088134 +Ref: c-api/typeobj c PyTypeObject tp_getattro10089081 +Ref: 389f10089081 +Ref: c-api/typeobj c PyTypeObject tp_setattro10089944 +Ref: 38a010089944 +Ref: c-api/typeobj c PyTypeObject tp_as_buffer10090938 +Ref: 3dc410090938 +Ref: c-api/typeobj c PyTypeObject tp_flags10091338 +Ref: 3ab610091338 +Ref: c-api/typeobj Py_TPFLAGS_HEAPTYPE10093230 +Ref: 3abe10093230 +Ref: c-api/typeobj Py_TPFLAGS_BASETYPE10093875 +Ref: 388710093875 +Ref: c-api/typeobj Py_TPFLAGS_READY10094152 +Ref: 3e2a10094152 +Ref: c-api/typeobj Py_TPFLAGS_READYING10094342 +Ref: 3e2b10094342 +Ref: c-api/typeobj Py_TPFLAGS_HAVE_GC10094541 +Ref: 388e10094541 +Ref: c-api/typeobj Py_TPFLAGS_DEFAULT10095522 +Ref: 387a10095522 +Ref: c-api/typeobj Py_TPFLAGS_METHOD_DESCRIPTOR10095902 +Ref: 3e2d10095902 +Ref: c-api/typeobj Py_TPFLAGS_LONG_SUBCLASS10096767 +Ref: 3e2e10096767 +Ref: c-api/typeobj Py_TPFLAGS_LIST_SUBCLASS10096808 +Ref: 3e2f10096808 +Ref: c-api/typeobj Py_TPFLAGS_TUPLE_SUBCLASS10096849 +Ref: 3e3010096849 +Ref: c-api/typeobj Py_TPFLAGS_BYTES_SUBCLASS10096891 +Ref: 3e3110096891 +Ref: c-api/typeobj Py_TPFLAGS_UNICODE_SUBCLASS10096933 +Ref: 3e3210096933 +Ref: c-api/typeobj Py_TPFLAGS_DICT_SUBCLASS10096977 +Ref: 3e3310096977 +Ref: c-api/typeobj Py_TPFLAGS_BASE_EXC_SUBCLASS10097018 +Ref: 3e3410097018 +Ref: c-api/typeobj Py_TPFLAGS_TYPE_SUBCLASS10097063 +Ref: 3e3510097063 +Ref: c-api/typeobj Py_TPFLAGS_HAVE_FINALIZE10097621 +Ref: 2d810097621 +Ref: c-api/typeobj _Py_TPFLAGS_HAVE_VECTORCALL10098001 +Ref: 3e2210098001 +Ref: c-api/typeobj c PyTypeObject tp_doc10098899 +Ref: 387b10098899 +Ref: c-api/typeobj c PyTypeObject tp_traverse10099211 +Ref: 33e810099211 +Ref: c-api/typeobj c PyTypeObject tp_clear10102127 +Ref: 389810102127 +Ref: c-api/typeobj c PyTypeObject tp_richcompare10105454 +Ref: 38a510105454 +Ref: c-api/typeobj c Py_RETURN_RICHCOMPARE10106979 +Ref: 42210106979 +Ref: c-api/typeobj c PyTypeObject tp_weaklistoffset10108214 +Ref: 38af10108214 +Ref: c-api/typeobj c PyTypeObject tp_iter10109997 +Ref: e3010109997 +Ref: c-api/typeobj c PyTypeObject tp_iternext10110471 +Ref: e3110110471 +Ref: c-api/typeobj c PyTypeObject tp_methods10111263 +Ref: 388610111263 +Ref: c-api/typeobj c PyTypeObject tp_members10111759 +Ref: 388410111759 +Ref: c-api/typeobj c PyTypeObject tp_getset10112291 +Ref: 388a10112291 +Ref: c-api/typeobj c PyTypeObject tp_base10112815 +Ref: 389010112815 +Ref: c-api/typeobj c PyTypeObject tp_dict10114109 +Ref: 3aca10114109 +Ref: c-api/typeobj c PyTypeObject tp_descr_get10115087 +Ref: 3dcb10115087 +Ref: c-api/typeobj c PyTypeObject tp_descr_set10115383 +Ref: 3dcd10115383 +Ref: c-api/typeobj c PyTypeObject tp_dictoffset10115774 +Ref: 3acf10115774 +Ref: c-api/typeobj c PyTypeObject tp_init10119032 +Ref: 388310119032 +Ref: c-api/typeobj c PyTypeObject tp_alloc10120488 +Ref: 388110120488 +Ref: c-api/typeobj c PyTypeObject tp_new10121173 +Ref: 387c10121173 +Ref: c-api/typeobj c PyTypeObject tp_free10122737 +Ref: 388010122737 +Ref: c-api/typeobj c PyTypeObject tp_is_gc10123421 +Ref: 3dd310123421 +Ref: c-api/typeobj c PyTypeObject tp_bases10124572 +Ref: 3dd410124572 +Ref: c-api/typeobj c PyTypeObject tp_mro10124820 +Ref: 3acb10124820 +Ref: c-api/typeobj c PyTypeObject tp_cache10125138 +Ref: 3acc10125138 +Ref: c-api/typeobj c PyTypeObject tp_subclasses10125275 +Ref: 3acd10125275 +Ref: c-api/typeobj c PyTypeObject tp_weaklist10125448 +Ref: 3ace10125448 +Ref: c-api/typeobj c PyTypeObject tp_del10125667 +Ref: 3dd510125667 +Ref: c-api/typeobj c PyTypeObject tp_version_tag10125783 +Ref: 3dd610125783 +Ref: c-api/typeobj c PyTypeObject tp_finalize10125958 +Ref: 2d710125958 +Ref: c-api/typeobj c PyTypeObject tp_allocs10127682 +Ref: 3dd710127682 +Ref: c-api/typeobj c PyTypeObject tp_frees10127760 +Ref: 3dd810127760 +Ref: c-api/typeobj c PyTypeObject tp_maxalloc10127831 +Ref: 3dd910127831 +Ref: c-api/typeobj c PyTypeObject tp_prev10127930 +Ref: 3dda10127930 +Ref: c-api/typeobj c PyTypeObject tp_next10128073 +Ref: 3ddb10128073 +Ref: PyTypeObject Slots-Footnote-110128905 +Node: Heap Types10128954 +Ref: c-api/typeobj heap-types10129043 +Ref: 3abc10129043 +Ref: c-api/typeobj id410129043 +Ref: 3e2510129043 +Node: Number Object Structures10130207 +Ref: c-api/typeobj number-object-structures10130355 +Ref: 3e3610130355 +Ref: c-api/typeobj number-structs10130355 +Ref: 3e2710130355 +Ref: c-api/typeobj c PyNumberMethods10130420 +Ref: d8110130420 +Ref: c-api/typeobj c PyNumberMethods nb_add10132846 +Ref: 3ac810132846 +Ref: c-api/typeobj c PyNumberMethods nb_subtract10132895 +Ref: 3de310132895 +Ref: c-api/typeobj c PyNumberMethods nb_multiply10132949 +Ref: 3de510132949 +Ref: c-api/typeobj c PyNumberMethods nb_remainder10133003 +Ref: 3de710133003 +Ref: c-api/typeobj c PyNumberMethods nb_divmod10133058 +Ref: 3de910133058 +Ref: c-api/typeobj c PyNumberMethods nb_power10133110 +Ref: 3dea10133110 +Ref: c-api/typeobj c PyNumberMethods nb_negative10133162 +Ref: 3dec10133162 +Ref: c-api/typeobj c PyNumberMethods nb_positive10133215 +Ref: 3ded10133215 +Ref: c-api/typeobj c PyNumberMethods nb_absolute10133268 +Ref: 3dee10133268 +Ref: c-api/typeobj c PyNumberMethods nb_bool10133321 +Ref: 3def10133321 +Ref: c-api/typeobj c PyNumberMethods nb_invert10133368 +Ref: 3df010133368 +Ref: c-api/typeobj c PyNumberMethods nb_lshift10133419 +Ref: 3df110133419 +Ref: c-api/typeobj c PyNumberMethods nb_rshift10133471 +Ref: 3df310133471 +Ref: c-api/typeobj c PyNumberMethods nb_and10133523 +Ref: 3df510133523 +Ref: c-api/typeobj c PyNumberMethods nb_xor10133572 +Ref: 3df710133572 +Ref: c-api/typeobj c PyNumberMethods nb_or10133621 +Ref: 3df910133621 +Ref: c-api/typeobj c PyNumberMethods nb_int10133669 +Ref: 3dfb10133669 +Ref: c-api/typeobj c PyNumberMethods nb_reserved10133717 +Ref: 3dfc10133717 +Ref: c-api/typeobj c PyNumberMethods nb_float10133766 +Ref: 3dfd10133766 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_add10133816 +Ref: 3de210133816 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_subtract10133873 +Ref: 3de410133873 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_multiply10133935 +Ref: 3de610133935 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_remainder10133997 +Ref: 3de810133997 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_power10134060 +Ref: 3deb10134060 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_lshift10134120 +Ref: 3df210134120 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_rshift10134180 +Ref: 3df410134180 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_and10134240 +Ref: 3df610134240 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_xor10134297 +Ref: 3df810134297 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_or10134354 +Ref: 3dfa10134354 +Ref: c-api/typeobj c PyNumberMethods nb_floor_divide10134410 +Ref: 3dfe10134410 +Ref: c-api/typeobj c PyNumberMethods nb_true_divide10134468 +Ref: 3e0010134468 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_floor_divide10134525 +Ref: 3dff10134525 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_true_divide10134591 +Ref: 3e0110134591 +Ref: c-api/typeobj c PyNumberMethods nb_index10134656 +Ref: 3e0210134656 +Ref: c-api/typeobj c PyNumberMethods nb_matrix_multiply10134706 +Ref: 3e0310134706 +Ref: c-api/typeobj c PyNumberMethods nb_inplace_matrix_multiply10134767 +Ref: 3e0410134767 +Node: Mapping Object Structures10134836 +Ref: c-api/typeobj mapping-object-structures10134995 +Ref: 3e3710134995 +Ref: c-api/typeobj mapping-structs10134995 +Ref: 3e2910134995 +Ref: c-api/typeobj c PyMappingMethods10135062 +Ref: 38ab10135062 +Ref: c-api/typeobj c PyMappingMethods mp_length10135229 +Ref: 3e0510135229 +Ref: c-api/typeobj c PyMappingMethods mp_subscript10135484 +Ref: 3e0710135484 +Ref: c-api/typeobj c PyMappingMethods mp_ass_subscript10135843 +Ref: 3e0810135843 +Node: Sequence Object Structures10136283 +Ref: c-api/typeobj sequence-object-structures10136442 +Ref: 3e3810136442 +Ref: c-api/typeobj sequence-structs10136442 +Ref: 3e2810136442 +Ref: c-api/typeobj c PySequenceMethods10136511 +Ref: 38aa10136511 +Ref: c-api/typeobj c PySequenceMethods sq_length10136657 +Ref: 3ac910136657 +Ref: c-api/typeobj c PySequenceMethods sq_concat10136962 +Ref: 3e0a10136962 +Ref: c-api/typeobj c PySequenceMethods sq_repeat10137228 +Ref: 3e0b10137228 +Ref: c-api/typeobj c PySequenceMethods sq_item10137508 +Ref: 3e0d10137508 +Ref: c-api/typeobj c PySequenceMethods sq_ass_item10138214 +Ref: 3e0e10138214 +Ref: c-api/typeobj c PySequenceMethods sq_contains10138668 +Ref: 3e1010138668 +Ref: c-api/typeobj c PySequenceMethods sq_inplace_concat10138970 +Ref: 3e1210138970 +Ref: c-api/typeobj c PySequenceMethods sq_inplace_repeat10139484 +Ref: 3e1310139484 +Node: Buffer Object Structures10140011 +Ref: c-api/typeobj buffer-object-structures10140168 +Ref: 3e3910140168 +Ref: c-api/typeobj buffer-structs10140168 +Ref: 3a6e10140168 +Ref: c-api/typeobj c PyBufferProcs10140233 +Ref: 3dc510140233 +Ref: c-api/typeobj c PyBufferProcs bf_getbuffer10140463 +Ref: 3ad010140463 +Ref: c-api/typeobj c PyBufferProcs bf_releasebuffer10142355 +Ref: 39c910142355 +Node: Async Object Structures10143467 +Ref: c-api/typeobj async-object-structures10143616 +Ref: 3e3a10143616 +Ref: c-api/typeobj async-structs10143616 +Ref: 3e2610143616 +Ref: c-api/typeobj c PyAsyncMethods10143700 +Ref: 3ac710143700 +Ref: c-api/typeobj c PyAsyncMethods am_await10144080 +Ref: 3ddd10144080 +Ref: c-api/typeobj c PyAsyncMethods am_aiter10144422 +Ref: 3ddf10144422 +Ref: c-api/typeobj c PyAsyncMethods am_anext10144769 +Ref: 3de010144769 +Node: Slot Type typedefs10145042 +Ref: c-api/typeobj id510145179 +Ref: 3e1810145179 +Ref: c-api/typeobj slot-type-typedefs10145179 +Ref: 3e3b10145179 +Ref: c-api/typeobj c allocfunc10145232 +Ref: 3dd010145232 +Ref: c-api/typeobj c destructor10146199 +Ref: 3db710146199 +Ref: c-api/typeobj c vectorcallfunc10146244 +Ref: 3e2310146244 +Ref: c-api/typeobj c freefunc10146544 +Ref: 3dd210146544 +Ref: c-api/typeobj c newfunc10146614 +Ref: 3dd110146614 +Ref: c-api/typeobj c initproc10146715 +Ref: 3dcf10146715 +Ref: c-api/typeobj c reprfunc10146812 +Ref: 3dbc10146812 +Ref: c-api/typeobj c getattrfunc10146891 +Ref: 3db810146891 +Ref: c-api/typeobj c setattrfunc10147020 +Ref: 3db910147020 +Ref: c-api/typeobj c getattrofunc10147238 +Ref: 3dc210147238 +Ref: c-api/typeobj c setattrofunc10147407 +Ref: 3dc310147407 +Ref: c-api/typeobj c descrgetfunc10147665 +Ref: 3dcc10147665 +Ref: c-api/typeobj c descrsetfunc10147780 +Ref: 3dce10147780 +Ref: c-api/typeobj c hashfunc10147879 +Ref: 3dc010147879 +Ref: c-api/typeobj c richcmpfunc10147958 +Ref: 3dc810147958 +Ref: c-api/typeobj c getiterfunc10148064 +Ref: 3dc910148064 +Ref: c-api/typeobj c iternextfunc10148145 +Ref: 3dca10148145 +Ref: c-api/typeobj c lenfunc10148231 +Ref: 3e0610148231 +Ref: c-api/typeobj c getbufferproc10148279 +Ref: 3e1410148279 +Ref: c-api/typeobj c releasebufferproc10148344 +Ref: 3e1510148344 +Ref: c-api/typeobj c unaryfunc10148409 +Ref: 3dde10148409 +Ref: c-api/typeobj c binaryfunc10148458 +Ref: 3de110148458 +Ref: c-api/typeobj c ternaryfunc10148520 +Ref: 3dc110148520 +Ref: c-api/typeobj c ssizeargfunc10148605 +Ref: 3e0c10148605 +Ref: c-api/typeobj c ssizeobjargproc10148669 +Ref: 3e0f10148669 +Ref: c-api/typeobj c objobjproc10148730 +Ref: 3e1110148730 +Ref: c-api/typeobj c objobjargproc10148786 +Ref: 3e0910148786 +Node: Examples<35>10148857 +Ref: c-api/typeobj examples10149007 +Ref: 3e3c10149007 +Ref: c-api/typeobj typedef-examples10149007 +Ref: 3db210149007 +Node: Supporting Cyclic Garbage Collection10154344 +Ref: c-api/gcsupport doc10154467 +Ref: 3e3d10154467 +Ref: c-api/gcsupport supporting-cycle-detection10154467 +Ref: 3e2c10154467 +Ref: c-api/gcsupport supporting-cyclic-garbage-collection10154467 +Ref: 3e3e10154467 +Ref: c-api/gcsupport c PyObject_GC_New10155799 +Ref: 2d410155799 +Ref: c-api/gcsupport c PyObject_GC_NewVar10155993 +Ref: 2d510155993 +Ref: c-api/gcsupport c PyObject_GC_Resize10156220 +Ref: 3e3f10156220 +Ref: c-api/gcsupport c PyObject_GC_Track10156493 +Ref: e4410156493 +Ref: c-api/gcsupport c PyObject_GC_Del10157186 +Ref: e4310157186 +Ref: c-api/gcsupport c PyObject_GC_UnTrack10157357 +Ref: e4510157357 +Ref: c-api/gcsupport c visitproc10158035 +Ref: 3e1710158035 +Ref: c-api/gcsupport c traverseproc10158581 +Ref: 3dc610158581 +Ref: c-api/gcsupport c Py_VISIT10159319 +Ref: 388c10159319 +Ref: c-api/gcsupport c inquiry10159902 +Ref: 3dc710159902 +Node: API and ABI Versioning10160377 +Ref: c-api/apiabiversion doc10160503 +Ref: 3e4010160503 +Ref: c-api/apiabiversion api-and-abi-versioning10160503 +Ref: 3e4110160503 +Ref: c-api/apiabiversion apiabiversion10160503 +Ref: 335e10160503 +Ref: API and ABI Versioning-Footnote-110162454 +Node: Distributing Python Modules10162526 +Ref: distributing/index doc10162665 +Ref: 3e4210162665 +Ref: distributing/index distributing-index10162665 +Ref: 7f610162665 +Ref: distributing/index distributing-python-modules10162665 +Ref: 3e4310162665 +Node: Key terms10163846 +Ref: distributing/index key-terms10163967 +Ref: 3e4410163967 +Ref: Key terms-Footnote-110165907 +Ref: Key terms-Footnote-210165932 +Ref: Key terms-Footnote-310165961 +Ref: Key terms-Footnote-410165993 +Ref: Key terms-Footnote-510166029 +Ref: Key terms-Footnote-610166082 +Ref: Key terms-Footnote-710166120 +Ref: Key terms-Footnote-810166173 +Node: Open source licensing and collaboration10166222 +Ref: distributing/index open-source-licensing-and-collaboration10166372 +Ref: 3e4510166372 +Ref: distributing/index wheel10166372 +Ref: 3e4610166372 +Node: Installing the tools10167390 +Ref: distributing/index installing-the-tools10167570 +Ref: 3e4710167570 +Ref: Installing the tools-Footnote-110168472 +Node: Reading the Python Packaging User Guide10168570 +Ref: distributing/index publishing-python-packages10168723 +Ref: 3e4810168723 +Ref: distributing/index reading-the-python-packaging-user-guide10168723 +Ref: 3e4910168723 +Ref: Reading the Python Packaging User Guide-Footnote-110169128 +Ref: Reading the Python Packaging User Guide-Footnote-210169198 +Ref: Reading the Python Packaging User Guide-Footnote-310169292 +Ref: Reading the Python Packaging User Guide-Footnote-410169394 +Node: How do I…?10169454 +Ref: distributing/index how-do-i10169578 +Ref: 3e4a10169578 +Ref: distributing/index the-pypirc-file10169578 +Ref: 3e4b10169578 +Node: … choose a name for my project?10169765 +Ref: distributing/index choose-a-name-for-my-project10169900 +Ref: 3e4c10169900 +Node: … create and distribute binary extensions?10170482 +Ref: distributing/index create-and-distribute-binary-extensions10170617 +Ref: 3e4d10170617 +Ref: … create and distribute binary extensions?-Footnote-110171039 +Node: Installing Python Modules10171112 +Ref: installing/index doc10171235 +Ref: 3e4e10171235 +Ref: installing/index installing-index10171235 +Ref: 7f510171235 +Ref: installing/index installing-python-modules10171235 +Ref: 3e4f10171235 +Node: Key terms<2>10172373 +Ref: installing/index key-terms10172467 +Ref: 3e5010172467 +Ref: Key terms<2>-Footnote-110174593 +Ref: Key terms<2>-Footnote-210174618 +Ref: Key terms<2>-Footnote-310174647 +Ref: Key terms<2>-Footnote-410174679 +Ref: Key terms<2>-Footnote-510174715 +Node: Basic usage10174795 +Ref: installing/index basic-usage10174911 +Ref: 3e5110174911 +Ref: Basic usage-Footnote-110176566 +Ref: Basic usage-Footnote-210176603 +Node: How do I …?10176652 +Ref: installing/index how-do-i10176782 +Ref: 3e5210176782 +Node: … install pip in versions of Python prior to Python 3 4?10177171 +Ref: installing/index install-pip-in-versions-of-python-prior-to-python-3-410177335 +Ref: 3e5310177335 +Ref: … install pip in versions of Python prior to Python 3 4?-Footnote-110177767 +Node: … install packages just for the current user?10177854 +Ref: installing/index install-packages-just-for-the-current-user10178066 +Ref: 3e5410178066 +Node: … install scientific Python packages?10178328 +Ref: installing/index install-scientific-python-packages10178546 +Ref: 3e5510178546 +Ref: … install scientific Python packages?-Footnote-110179052 +Ref: … install scientific Python packages?-Footnote-210179098 +Node: … work with multiple versions of Python installed in parallel?10179144 +Ref: installing/index work-with-multiple-versions-of-python-installed-in-parallel10179306 +Ref: 3e5610179306 +Node: Common installation issues10180281 +Ref: installing/index common-installation-issues10180391 +Ref: 3e5710180391 +Node: Installing into the system Python on Linux10180566 +Ref: installing/index installing-into-the-system-python-on-linux10180697 +Ref: 3e5810180697 +Node: Pip not installed10181264 +Ref: installing/index pip-not-installed10181432 +Ref: 3e5910181432 +Ref: Pip not installed-Footnote-110181706 +Node: Installing binary extensions10181808 +Ref: installing/index installing-binary-extensions10181925 +Ref: 3e5a10181925 +Ref: Installing binary extensions-Footnote-110182820 +Ref: Installing binary extensions-Footnote-210182866 +Node: Python HOWTOs10182915 +Ref: howto/index doc10183044 +Ref: 3e5b10183044 +Ref: howto/index python-howtos10183044 +Ref: 3e5c10183044 +Node: Porting Python 2 Code to Python 310183918 +Ref: howto/pyporting doc10184047 +Ref: 3e5d10184047 +Ref: howto/pyporting porting-python-2-code-to-python-310184047 +Ref: 3e5e10184047 +Ref: howto/pyporting pyporting-howto10184047 +Ref: c7d10184047 +Ref: Porting Python 2 Code to Python 3-Footnote-110184926 +Ref: Porting Python 2 Code to Python 3-Footnote-210185030 +Ref: Porting Python 2 Code to Python 3-Footnote-310185076 +Node: The Short Explanation10185140 +Ref: howto/pyporting the-short-explanation10185247 +Ref: 3e5f10185247 +Ref: The Short Explanation-Footnote-110186398 +Ref: The Short Explanation-Footnote-210186440 +Ref: The Short Explanation-Footnote-310186499 +Ref: The Short Explanation-Footnote-410186548 +Ref: The Short Explanation-Footnote-510186588 +Ref: The Short Explanation-Footnote-610186636 +Ref: The Short Explanation-Footnote-710186673 +Node: Details10186703 +Ref: howto/pyporting details10186810 +Ref: 3e6010186810 +Node: Drop support for Python 2 6 and older10188286 +Ref: howto/pyporting drop-support-for-python-2-6-and-older10188446 +Ref: 3e6110188446 +Ref: Drop support for Python 2 6 and older-Footnote-110189771 +Ref: Drop support for Python 2 6 and older-Footnote-210189808 +Node: Make sure you specify the proper version support in your setup py file10189848 +Ref: howto/pyporting make-sure-you-specify-the-proper-version-support-in-your-setup-py-file10190040 +Ref: 3e6210190040 +Ref: Make sure you specify the proper version support in your setup py file-Footnote-110190642 +Node: Have good test coverage10190679 +Ref: howto/pyporting have-good-test-coverage10190876 +Ref: 3e6310190876 +Ref: Have good test coverage-Footnote-110191562 +Node: Learn the differences between Python 2 & 310191604 +Ref: howto/pyporting learn-the-differences-between-python-2-310191747 +Ref: 3e6410191747 +Ref: Learn the differences between Python 2 & 3-Footnote-110192412 +Ref: Learn the differences between Python 2 & 3-Footnote-210192447 +Node: Update your code10192503 +Ref: howto/pyporting update-your-code10192656 +Ref: 3e6510192656 +Ref: Update your code-Footnote-110194932 +Ref: Update your code-Footnote-210194991 +Ref: Update your code-Footnote-310195040 +Ref: Update your code-Footnote-410195099 +Ref: Update your code-Footnote-510195148 +Node: Division10195185 +Ref: howto/pyporting division10195278 +Ref: 3e6610195278 +Node: Text versus binary data10196334 +Ref: howto/pyporting text-versus-binary-data10196486 +Ref: 3e6710196486 +Ref: Text versus binary data-Footnote-110203001 +Node: Use feature detection instead of version detection10203038 +Ref: howto/pyporting use-feature-detection-instead-of-version-detection10203173 +Ref: 3e6810203173 +Ref: Use feature detection instead of version detection-Footnote-110204876 +Node: Prevent compatibility regressions10204920 +Ref: howto/pyporting prevent-compatibility-regressions10205077 +Ref: 3e6910205077 +Ref: Prevent compatibility regressions-Footnote-110206419 +Ref: Prevent compatibility regressions-Footnote-210206459 +Ref: Prevent compatibility regressions-Footnote-310206508 +Node: Check which dependencies block your transition10206567 +Ref: howto/pyporting check-which-dependencies-block-your-transition10206766 +Ref: 3e6a10206766 +Ref: Check which dependencies block your transition-Footnote-110207628 +Node: Update your setup py file to denote Python 3 compatibility10207676 +Ref: howto/pyporting update-your-setup-py-file-to-denote-python-3-compatibility10207887 +Ref: 3e6b10207887 +Node: Use continuous integration to stay compatible10208401 +Ref: howto/pyporting use-continuous-integration-to-stay-compatible10208610 +Ref: 3e6c10208610 +Ref: Use continuous integration to stay compatible-Footnote-110209922 +Node: Consider using optional static type checking10209959 +Ref: howto/pyporting consider-using-optional-static-type-checking10210101 +Ref: 3e6d10210101 +Ref: Consider using optional static type checking-Footnote-110210882 +Ref: Consider using optional static type checking-Footnote-210210912 +Node: Porting Extension Modules to Python 310210953 +Ref: howto/cporting doc10211121 +Ref: 3e6e10211121 +Ref: howto/cporting cporting-howto10211121 +Ref: c7710211121 +Ref: howto/cporting porting-extension-modules-to-python-310211121 +Ref: 3e6f10211121 +Ref: howto/cporting why-python-3-exists10211121 +Ref: 3e7010211121 +Ref: Porting Extension Modules to Python 3-Footnote-110211903 +Ref: Porting Extension Modules to Python 3-Footnote-210211954 +Ref: Porting Extension Modules to Python 3-Footnote-310212011 +Ref: Porting Extension Modules to Python 3-Footnote-410212038 +Node: Curses Programming with Python10212085 +Ref: howto/curses doc10212242 +Ref: 3e7110212242 +Ref: howto/curses cffi10212242 +Ref: 3e7210212242 +Ref: howto/curses curses-howto10212242 +Ref: 1e2e10212242 +Ref: howto/curses curses-programming-with-python10212242 +Ref: 3e7310212242 +Node: What is curses?10212653 +Ref: howto/curses what-is-curses10212784 +Ref: 3e7410212784 +Ref: What is curses?-Footnote-110215344 +Ref: What is curses?-Footnote-210215384 +Ref: What is curses?-Footnote-310215427 +Node: The Python curses module10215476 +Ref: howto/curses the-python-curses-module10215552 +Ref: 3e7510215552 +Node: Starting and ending a curses application10216381 +Ref: howto/curses starting-and-ending-a-curses-application10216537 +Ref: 3e7610216537 +Node: Windows and Pads10219801 +Ref: howto/curses windows-and-pads10219957 +Ref: 3e7710219957 +Node: Displaying Text10224503 +Ref: howto/curses displaying-text10224629 +Ref: 3e7810224629 +Node: Attributes and Color10228415 +Ref: howto/curses attributes-and-color10228487 +Ref: 3e7910228487 +Node: User Input10233047 +Ref: howto/curses user-input10233177 +Ref: 3e7a10233177 +Ref: User Input-Footnote-110236981 +Node: For More Information10237021 +Ref: howto/curses for-more-information10237127 +Ref: 3e7b10237127 +Ref: For More Information-Footnote-110238626 +Ref: For More Information-Footnote-210238663 +Ref: For More Information-Footnote-310238726 +Ref: For More Information-Footnote-410238770 +Ref: For More Information-Footnote-510238831 +Ref: For More Information-Footnote-610238883 +Node: Descriptor HowTo Guide10238957 +Ref: howto/descriptor doc10239105 +Ref: 3e7c10239105 +Ref: howto/descriptor descriptor-howto-guide10239105 +Ref: 3e7d10239105 +Node: Abstract10239454 +Ref: howto/descriptor abstract10239557 +Ref: 3e7e10239557 +Node: Definition and Introduction10240076 +Ref: howto/descriptor definition-and-introduction10240207 +Ref: 3e7f10240207 +Node: Descriptor Protocol10241562 +Ref: howto/descriptor descriptor-protocol10241708 +Ref: 3e8010241708 +Node: Invoking Descriptors<2>10243019 +Ref: howto/descriptor invoking-descriptors10243156 +Ref: 3e8110243156 +Ref: Invoking Descriptors<2>-Footnote-110246152 +Ref: Invoking Descriptors<2>-Footnote-210246220 +Ref: Invoking Descriptors<2>-Footnote-310246292 +Node: Descriptor Example10246372 +Ref: howto/descriptor descriptor-example10246500 +Ref: 3e8210246500 +Node: Properties10247902 +Ref: howto/descriptor properties10248028 +Ref: 3e8310248028 +Node: Functions and Methods10250755 +Ref: howto/descriptor functions-and-methods10250895 +Ref: 3e8410250895 +Node: Static Methods and Class Methods10253094 +Ref: howto/descriptor static-methods-and-class-methods10253215 +Ref: 3e8510253215 +Node: Functional Programming HOWTO10257809 +Ref: howto/functional doc10257940 +Ref: 3e8610257940 +Ref: howto/functional functional-programming-howto10257940 +Ref: 3e8710257940 +Node: Introduction<14>10258751 +Ref: howto/functional introduction10258853 +Ref: 3e8810258853 +Node: Formal provability10263567 +Ref: howto/functional formal-provability10263657 +Ref: 3e8a10263657 +Node: Modularity10265573 +Ref: howto/functional modularity10265701 +Ref: 3e8b10265701 +Node: Ease of debugging and testing10266108 +Ref: howto/functional ease-of-debugging-and-testing10266231 +Ref: 3e8c10266231 +Node: Composability10266968 +Ref: howto/functional composability10267072 +Ref: 3e8d10267072 +Node: Iterators<2>10267797 +Ref: howto/functional functional-howto-iterators10267953 +Ref: 3e8910267953 +Ref: howto/functional iterators10267953 +Ref: 3e8e10267953 +Node: Data Types That Support Iterators10271316 +Ref: howto/functional data-types-that-support-iterators10271398 +Ref: 3e8f10271398 +Node: Generator expressions and list comprehensions10273308 +Ref: howto/functional generator-expressions-and-list-comprehensions10273461 +Ref: 3e9010273461 +Node: Generators<2>10277968 +Ref: howto/functional generators10278127 +Ref: 3e9110278127 +Ref: Generators<2>-Footnote-110281985 +Node: Passing values into a generator10282065 +Ref: howto/functional passing-values-into-a-generator10282146 +Ref: 3e9210282146 +Ref: Passing values into a generator-Footnote-110286140 +Node: Built-in functions10286189 +Ref: howto/functional built-in-functions10286323 +Ref: 3e9310286323 +Ref: Built-in functions-Footnote-110290361 +Node: The itertools module10290415 +Ref: howto/functional the-itertools-module10290556 +Ref: 3e9410290556 +Node: Creating new iterators10291301 +Ref: howto/functional creating-new-iterators10291418 +Ref: 3e9510291418 +Node: Calling functions on elements10294103 +Ref: howto/functional calling-functions-on-elements10294247 +Ref: 3e9610294247 +Node: Selecting elements10295064 +Ref: howto/functional selecting-elements10295208 +Ref: 3e9710295208 +Node: Combinatoric functions10296700 +Ref: howto/functional combinatoric-functions10296832 +Ref: 3e9810296832 +Node: Grouping elements10299165 +Ref: howto/functional grouping-elements10299270 +Ref: 3e9910299270 +Node: The functools module10300822 +Ref: howto/functional the-functools-module10300986 +Ref: 3e9a10300986 +Node: The operator module10304603 +Ref: howto/functional the-operator-module10304679 +Ref: 3e9b10304679 +Node: Small functions and the lambda expression10305493 +Ref: howto/functional small-functions-and-the-lambda-expression10305674 +Ref: 3e9c10305674 +Node: Revision History and Acknowledgements10308533 +Ref: howto/functional revision-history-and-acknowledgements10308707 +Ref: 3e9d10308707 +Node: References<2>10309486 +Ref: howto/functional references10309610 +Ref: 3e9e10309610 +Node: General10309716 +Ref: howto/functional general10309797 +Ref: 3e9f10309797 +Node: Python-specific10310738 +Ref: howto/functional python-specific10310848 +Ref: 3ea010310848 +Ref: Python-specific-Footnote-110311323 +Ref: Python-specific-Footnote-210311374 +Ref: Python-specific-Footnote-310311427 +Node: Python documentation10311480 +Ref: howto/functional python-documentation10311574 +Ref: 3ea110311574 +Ref: Python documentation-Footnote-110311975 +Ref: Python documentation-Footnote-210312024 +Node: Logging HOWTO10312073 +Ref: howto/logging doc10312198 +Ref: 3ea210312198 +Ref: howto/logging logging-howto10312198 +Ref: 3ea310312198 +Node: Basic Logging Tutorial10312511 +Ref: howto/logging basic-logging-tutorial10312617 +Ref: 3ea410312617 +Ref: howto/logging logging-basic-tutorial10312617 +Ref: b7210312617 +Node: When to use logging10313403 +Ref: howto/logging when-to-use-logging10313506 +Ref: 3ea510313506 +Node: A simple example10317397 +Ref: howto/logging a-simple-example10317526 +Ref: 3ea610317526 +Ref: howto/logging howto-minimal-example10317526 +Ref: 3ea710317526 +Node: Logging to a file10318313 +Ref: howto/logging logging-to-a-file10318452 +Ref: 3ea810318452 +Node: Logging from multiple modules10320965 +Ref: howto/logging logging-from-multiple-modules10321109 +Ref: 3ea910321109 +Node: Logging variable data10322315 +Ref: howto/logging logging-variable-data10322483 +Ref: 3eaa10322483 +Node: Changing the format of displayed messages10323302 +Ref: howto/logging changing-the-format-of-displayed-messages10323477 +Ref: 3eab10323477 +Node: Displaying the date/time in messages10324473 +Ref: howto/logging displaying-the-date-time-in-messages10324637 +Ref: 3eac10324637 +Ref: Displaying the date/time in messages-Footnote-110325689 +Node: Next Steps10325738 +Ref: howto/logging next-steps10325852 +Ref: 3ead10325852 +Node: Advanced Logging Tutorial10326810 +Ref: howto/logging advanced-logging-tutorial10326942 +Ref: 3eae10326942 +Ref: howto/logging logging-advanced-tutorial10326942 +Ref: b7310326942 +Node: Logging Flow10330333 +Ref: howto/logging logging-flow10330423 +Ref: 3eaf10330423 +Node: Loggers10330598 +Ref: howto/logging loggers10330705 +Ref: 3eb010330705 +Node: Handlers10334986 +Ref: howto/logging handler-basic10335091 +Ref: 3eb110335091 +Ref: howto/logging handlers10335091 +Ref: 3eb210335091 +Node: Formatters10337159 +Ref: howto/logging formatters10337276 +Ref: 3eb410337276 +Ref: howto/logging logging logging Formatter __init__10337726 +Ref: 3eb510337726 +Node: Configuring Logging10339418 +Ref: howto/logging configuring-logging10339571 +Ref: 3eb610339571 +Node: What happens if no configuration is provided10346135 +Ref: howto/logging what-happens-if-no-configuration-is-provided10346311 +Ref: 3eb710346311 +Node: Configuring Logging for a Library10347738 +Ref: howto/logging configuring-logging-for-a-library10347886 +Ref: 3eb810347886 +Ref: howto/logging library-config10347886 +Ref: 1dd610347886 +Node: Logging Levels<2>10350316 +Ref: howto/logging logging-levels10350441 +Ref: 3eb910350441 +Node: Custom Levels10353302 +Ref: howto/logging custom-levels10353369 +Ref: 1da810353369 +Ref: howto/logging id110353369 +Ref: 3eba10353369 +Node: Useful Handlers10354084 +Ref: howto/logging id210354216 +Ref: 3ebb10354216 +Ref: howto/logging useful-handlers10354216 +Ref: 3eb310354216 +Node: Exceptions raised during logging10358224 +Ref: howto/logging exceptions-raised-during-logging10358374 +Ref: 3ebc10358374 +Ref: howto/logging logging-exceptions10358374 +Ref: 3ebd10358374 +Node: Using arbitrary objects as messages10359530 +Ref: howto/logging arbitrary-object-messages10359677 +Ref: 1d9810359677 +Ref: howto/logging using-arbitrary-objects-as-messages10359677 +Ref: 3ebe10359677 +Node: Optimization10360291 +Ref: howto/logging optimization10360397 +Ref: 3ebf10360397 +Node: Logging Cookbook10363951 +Ref: howto/logging-cookbook doc10364072 +Ref: 3ec010364072 +Ref: howto/logging-cookbook id110364072 +Ref: 3ec110364072 +Ref: howto/logging-cookbook logging-cookbook10364072 +Ref: b7410364072 +Node: Using logging in multiple modules10365683 +Ref: howto/logging-cookbook using-logging-in-multiple-modules10365807 +Ref: 3ec210365807 +Node: Logging from multiple threads10369381 +Ref: howto/logging-cookbook logging-from-multiple-threads10369546 +Ref: 3ec310369546 +Node: Multiple handlers and formatters10371293 +Ref: howto/logging-cookbook multiple-handlers-and-formatters10371457 +Ref: 3ec410371457 +Node: Logging to multiple destinations10373620 +Ref: howto/logging-cookbook logging-to-multiple-destinations10373783 +Ref: 3ec510373783 +Ref: howto/logging-cookbook multiple-destinations10373783 +Ref: 3ec610373783 +Node: Configuration server example10376634 +Ref: howto/logging-cookbook configuration-server-example10376797 +Ref: 3ec710376797 +Node: Dealing with handlers that block10378359 +Ref: howto/logging-cookbook dealing-with-handlers-that-block10378543 +Ref: 3ec810378543 +Node: Sending and receiving logging events across a network10382200 +Ref: howto/logging-cookbook network-logging10382408 +Ref: 3ec910382408 +Ref: howto/logging-cookbook sending-and-receiving-logging-events-across-a-network10382408 +Ref: 3eca10382408 +Node: Adding contextual information to your logging output10387876 +Ref: howto/logging-cookbook adding-contextual-information-to-your-logging-output10388100 +Ref: 3ecb10388100 +Ref: howto/logging-cookbook context-info10388100 +Ref: 1d9b10388100 +Node: Using LoggerAdapters to impart contextual information10389182 +Ref: howto/logging-cookbook using-loggeradapters-to-impart-contextual-information10389379 +Ref: 3ecc10389379 +Node: Using objects other than dicts to pass contextual information10392527 +Ref: howto/logging-cookbook using-objects-other-than-dicts-to-pass-contextual-information10392678 +Ref: 3ecd10392678 +Node: Using Filters to impart contextual information10393137 +Ref: howto/logging-cookbook filters-contextual10393334 +Ref: 1d9010393334 +Ref: howto/logging-cookbook using-filters-to-impart-contextual-information10393334 +Ref: 3ece10393334 +Node: Logging to a single file from multiple processes10397139 +Ref: howto/logging-cookbook logging-to-a-single-file-from-multiple-processes10397329 +Ref: 3ecf10397329 +Ref: howto/logging-cookbook multiple-processes10397329 +Ref: 3ed010397329 +Node: Using concurrent futures ProcessPoolExecutor10407759 +Ref: howto/logging-cookbook using-concurrent-futures-processpoolexecutor10407888 +Ref: 3ed110407888 +Node: Using file rotation10408894 +Ref: howto/logging-cookbook using-file-rotation10409068 +Ref: 3ed210409068 +Node: Use of alternative formatting styles10410966 +Ref: howto/logging-cookbook format-styles10411113 +Ref: 3ed310411113 +Ref: howto/logging-cookbook use-of-alternative-formatting-styles10411113 +Ref: 3ed410411113 +Node: Customizing LogRecord10418299 +Ref: howto/logging-cookbook custom-logrecord10418470 +Ref: 3ed510418470 +Ref: howto/logging-cookbook customizing-logrecord10418470 +Ref: 3ed610418470 +Node: Subclassing QueueHandler - a ZeroMQ example10422383 +Ref: howto/logging-cookbook subclassing-queuehandler-a-zeromq-example10422562 +Ref: 3ed710422562 +Ref: howto/logging-cookbook zeromq-handlers10422562 +Ref: 3ed810422562 +Node: Subclassing QueueListener - a ZeroMQ example10423898 +Ref: howto/logging-cookbook subclassing-queuelistener-a-zeromq-example10424097 +Ref: 3ed910424097 +Node: An example dictionary-based configuration10425218 +Ref: howto/logging-cookbook an-example-dictionary-based-configuration10425436 +Ref: 3eda10425436 +Ref: An example dictionary-based configuration-Footnote-110427473 +Ref: An example dictionary-based configuration-Footnote-210427559 +Node: Using a rotator and namer to customize log rotation processing10427645 +Ref: howto/logging-cookbook cookbook-rotator-namer10427859 +Ref: 1de210427859 +Ref: howto/logging-cookbook using-a-rotator-and-namer-to-customize-log-rotation-processing10427859 +Ref: 3edb10427859 +Node: A more elaborate multiprocessing example10428744 +Ref: howto/logging-cookbook a-more-elaborate-multiprocessing-example10428970 +Ref: 3edc10428970 +Node: Inserting a BOM into messages sent to a SysLogHandler10438598 +Ref: howto/logging-cookbook inserting-a-bom-into-messages-sent-to-a-sysloghandler10438793 +Ref: 3edd10438793 +Ref: Inserting a BOM into messages sent to a SysLogHandler-Footnote-110440960 +Ref: Inserting a BOM into messages sent to a SysLogHandler-Footnote-210441009 +Ref: Inserting a BOM into messages sent to a SysLogHandler-Footnote-310441068 +Ref: Inserting a BOM into messages sent to a SysLogHandler-Footnote-410441117 +Node: Implementing structured logging10441166 +Ref: howto/logging-cookbook implementing-structured-logging10441357 +Ref: 3ede10441357 +Node: Customizing handlers with dictConfig10444104 +Ref: howto/logging-cookbook custom-handlers10444304 +Ref: 3edf10444304 +Ref: howto/logging-cookbook customizing-handlers-with-dictconfig10444304 +Ref: 3ee010444304 +Node: Using particular formatting styles throughout your application10449497 +Ref: howto/logging-cookbook formatting-styles10449701 +Ref: 1d8710449701 +Ref: howto/logging-cookbook using-particular-formatting-styles-throughout-your-application10449701 +Ref: 3ee110449701 +Node: Using LogRecord factories10451702 +Ref: howto/logging-cookbook using-logrecord-factories10451863 +Ref: 3ee210451863 +Node: Using custom message objects10452839 +Ref: howto/logging-cookbook using-custom-message-objects10453000 +Ref: 3ee310453000 +Node: Configuring filters with dictConfig10455710 +Ref: howto/logging-cookbook configuring-filters-with-dictconfig10455909 +Ref: 3ee410455909 +Ref: howto/logging-cookbook filters-dictconfig10455909 +Ref: 3ee510455909 +Node: Customized exception formatting10458874 +Ref: howto/logging-cookbook custom-format-exception10459036 +Ref: 3ee610459036 +Ref: howto/logging-cookbook customized-exception-formatting10459036 +Ref: 3ee710459036 +Node: Speaking logging messages10461166 +Ref: howto/logging-cookbook speaking-logging-messages10461353 +Ref: 3ee810461353 +Ref: howto/logging-cookbook spoken-messages10461353 +Ref: 3ee910461353 +Node: Buffering logging messages and outputting them conditionally10463521 +Ref: howto/logging-cookbook buffered-logging10463725 +Ref: 3eea10463725 +Ref: howto/logging-cookbook buffering-logging-messages-and-outputting-them-conditionally10463725 +Ref: 3eeb10463725 +Node: Formatting times using UTC GMT via configuration10469302 +Ref: howto/logging-cookbook formatting-times-using-utc-gmt-via-configuration10469526 +Ref: 3eec10469526 +Ref: howto/logging-cookbook utc-formatting10469526 +Ref: 3eed10469526 +Node: Using a context manager for selective logging10471428 +Ref: howto/logging-cookbook context-manager10471626 +Ref: 3eee10471626 +Ref: howto/logging-cookbook using-a-context-manager-for-selective-logging10471626 +Ref: 3eef10471626 +Node: A CLI application starter template10476208 +Ref: howto/logging-cookbook a-cli-application-starter-template10476378 +Ref: 3ef010476378 +Ref: howto/logging-cookbook starter-template10476378 +Ref: 3ef110476378 +Node: A Qt GUI for logging10482361 +Ref: howto/logging-cookbook a-qt-gui-for-logging10482477 +Ref: 3ef210482477 +Ref: howto/logging-cookbook qt-gui10482477 +Ref: 3ef310482477 +Ref: A Qt GUI for logging-Footnote-110491871 +Ref: A Qt GUI for logging-Footnote-210491898 +Ref: A Qt GUI for logging-Footnote-310491940 +Node: Regular Expression HOWTO10491980 +Ref: howto/regex doc10492112 +Ref: 3ef410492112 +Ref: howto/regex regex-howto10492112 +Ref: 13f810492112 +Ref: howto/regex regular-expression-howto10492112 +Ref: 3ef510492112 +Node: Introduction<15>10492609 +Ref: howto/regex introduction10492710 +Ref: 3ef610492710 +Node: Simple Patterns10494315 +Ref: howto/regex simple-patterns10494450 +Ref: 3ef710494450 +Node: Matching Characters10494945 +Ref: howto/regex matching-characters10495041 +Ref: 3ef810495041 +Node: Repeating Things10499676 +Ref: howto/regex repeating-things10499772 +Ref: 3ef910499772 +Node: Using Regular Expressions10504710 +Ref: howto/regex using-regular-expressions10504847 +Ref: 3efa10504847 +Node: Compiling Regular Expressions10505340 +Ref: howto/regex compiling-regular-expressions10505460 +Ref: 3efb10505460 +Node: The Backslash Plague10506677 +Ref: howto/regex id110506824 +Ref: 3efc10506824 +Ref: howto/regex the-backslash-plague10506824 +Ref: 3efd10506824 +Node: Performing Matches10509710 +Ref: howto/regex performing-matches10509853 +Ref: 3efe10509853 +Ref: Performing Matches-Footnote-110515758 +Node: Module-Level Functions<2>10515830 +Ref: howto/regex module-level-functions10515970 +Ref: 3eff10515970 +Node: Compilation Flags10517170 +Ref: howto/regex compilation-flags10517283 +Ref: 3f0010517283 +Node: More Pattern Power10524467 +Ref: howto/regex more-pattern-power10524606 +Ref: 3f0210524606 +Node: More Metacharacters10524971 +Ref: howto/regex id210525062 +Ref: 3f0310525062 +Ref: howto/regex more-metacharacters10525062 +Ref: 3f0110525062 +Node: Grouping10529715 +Ref: howto/regex grouping10529845 +Ref: 3f0410529845 +Node: Non-capturing and Named Groups10533330 +Ref: howto/regex non-capturing-and-named-groups10533461 +Ref: 3f0510533461 +Node: Lookahead Assertions10538597 +Ref: howto/regex lookahead-assertions10538711 +Ref: 3f0610538711 +Node: Modifying Strings10542378 +Ref: howto/regex modifying-strings10542507 +Ref: 3f0710542507 +Node: Splitting Strings10543477 +Ref: howto/regex splitting-strings10543575 +Ref: 3f0810543575 +Node: Search and Replace10545893 +Ref: howto/regex search-and-replace10545991 +Ref: 3f0910545991 +Node: Common Problems10550479 +Ref: howto/regex common-problems10550598 +Ref: 3f0a10550598 +Node: Use String Methods10551036 +Ref: howto/regex use-string-methods10551134 +Ref: 3f0b10551134 +Node: match versus search10552851 +Ref: howto/regex match-versus-search10552982 +Ref: 3f0c10552982 +Node: Greedy versus Non-Greedy10554610 +Ref: howto/regex greedy-versus-non-greedy10554739 +Ref: 3f0d10554739 +Node: Using re VERBOSE10556662 +Ref: howto/regex using-re-verbose10556763 +Ref: 3f0e10556763 +Node: Feedback10558290 +Ref: howto/regex feedback10558383 +Ref: 3f0f10558383 +Node: Socket Programming HOWTO10559166 +Ref: howto/sockets doc10559296 +Ref: 3f1010559296 +Ref: howto/sockets socket-howto10559296 +Ref: 239410559296 +Ref: howto/sockets socket-programming-howto10559296 +Ref: 3f1110559296 +Node: Sockets10559894 +Ref: howto/sockets sockets10559988 +Ref: 3f1210559988 +Node: History10561148 +Ref: howto/sockets history10561199 +Ref: 3f1310561199 +Node: Creating a Socket10561774 +Ref: howto/sockets creating-a-socket10561891 +Ref: 3f1410561891 +Node: IPC10565147 +Ref: howto/sockets ipc10565204 +Ref: 3f1510565204 +Node: Using a Socket10565650 +Ref: howto/sockets using-a-socket10565773 +Ref: 3f1610565773 +Node: Binary Data10571912 +Ref: howto/sockets binary-data10571974 +Ref: 3f1710571974 +Node: Disconnecting10573050 +Ref: howto/sockets disconnecting10573176 +Ref: 3f1810573176 +Node: When Sockets Die10574652 +Ref: howto/sockets when-sockets-die10574718 +Ref: 3f1910574718 +Node: Non-blocking Sockets10575597 +Ref: howto/sockets non-blocking-sockets10575700 +Ref: 3f1a10575700 +Node: Sorting HOW TO10579518 +Ref: howto/sorting doc10579637 +Ref: 3f1b10579637 +Ref: howto/sorting sorting-how-to10579637 +Ref: 3f1c10579637 +Ref: howto/sorting sortinghowto10579637 +Ref: 12ab10579637 +Node: Sorting Basics10580269 +Ref: howto/sorting sorting-basics10580356 +Ref: 3f1d10580356 +Node: Key Functions10581158 +Ref: howto/sorting key-functions10581279 +Ref: 3f1e10581279 +Node: Operator Module Functions10582949 +Ref: howto/sorting operator-module-functions10583080 +Ref: 3f1f10583080 +Node: Ascending and Descending10584139 +Ref: howto/sorting ascending-and-descending10584289 +Ref: 3f2010584289 +Node: Sort Stability and Complex Sorts10584826 +Ref: howto/sorting sort-stability-and-complex-sorts10584993 +Ref: 3f2110584993 +Ref: Sort Stability and Complex Sorts-Footnote-110586628 +Ref: Sort Stability and Complex Sorts-Footnote-210586694 +Node: The Old Way Using Decorate-Sort-Undecorate10586740 +Ref: howto/sorting the-old-way-using-decorate-sort-undecorate10586918 +Ref: 3f2210586918 +Ref: The Old Way Using Decorate-Sort-Undecorate-Footnote-110588607 +Node: The Old Way Using the cmp Parameter10588667 +Ref: howto/sorting the-old-way-using-the-cmp-parameter10588825 +Ref: 3f2310588825 +Node: Odd and Ends10591301 +Ref: howto/sorting odd-and-ends10591408 +Ref: 3f2410591408 +Node: Unicode HOWTO10593128 +Ref: howto/unicode doc10593278 +Ref: 3f2510593278 +Ref: howto/unicode id110593278 +Ref: 3f2610593278 +Ref: howto/unicode unicode-howto10593278 +Ref: c3c10593278 +Node: Introduction to Unicode10593680 +Ref: howto/unicode introduction-to-unicode10593788 +Ref: 3f2710593788 +Node: Definitions10593922 +Ref: howto/unicode definitions10594011 +Ref: 3f2810594011 +Node: Encodings10597032 +Ref: howto/unicode encodings10597143 +Ref: 3f2910597143 +Node: References<3>10600826 +Ref: howto/unicode references10600917 +Ref: 3f2a10600917 +Ref: References<3>-Footnote-110601794 +Ref: References<3>-Footnote-210601825 +Ref: References<3>-Footnote-310601865 +Ref: References<3>-Footnote-410601917 +Ref: References<3>-Footnote-510601963 +Ref: References<3>-Footnote-610602137 +Ref: References<3>-Footnote-710602194 +Node: Python’s Unicode Support10602238 +Ref: howto/unicode python-s-unicode-support10602387 +Ref: 3f2b10602387 +Node: The String Type10602756 +Ref: howto/unicode the-string-type10602862 +Ref: 3f2c10602862 +Node: Converting to Bytes10605974 +Ref: howto/unicode converting-to-bytes10606127 +Ref: 3f2d10606127 +Node: Unicode Literals in Python Source Code10607973 +Ref: howto/unicode unicode-literals-in-python-source-code10608129 +Ref: 3f2e10608129 +Ref: Unicode Literals in Python Source Code-Footnote-110610236 +Node: Unicode Properties10610285 +Ref: howto/unicode unicode-properties10610439 +Ref: 3f2f10610439 +Ref: Unicode Properties-Footnote-110612206 +Node: Comparing Strings10612275 +Ref: howto/unicode comparing-strings10612418 +Ref: 3f3010612418 +Node: Unicode Regular Expressions10615265 +Ref: howto/unicode unicode-regular-expressions10615403 +Ref: 3f3110615403 +Node: References<4>10616544 +Ref: howto/unicode id210616656 +Ref: 3f3210616656 +Ref: References<4>-Footnote-110617443 +Ref: References<4>-Footnote-210617538 +Ref: References<4>-Footnote-310617590 +Node: Reading and Writing Unicode Data10617659 +Ref: howto/unicode reading-and-writing-unicode-data10617805 +Ref: 3f3310617805 +Node: Unicode filenames10621516 +Ref: howto/unicode unicode-filenames10621650 +Ref: 3f3410621650 +Node: Tips for Writing Unicode-aware Programs10624088 +Ref: howto/unicode tips-for-writing-unicode-aware-programs10624244 +Ref: 3f3510624244 +Node: Converting Between File Encodings10625609 +Ref: howto/unicode converting-between-file-encodings10625755 +Ref: 3f3610625755 +Node: Files in an Unknown Encoding10626526 +Ref: howto/unicode files-in-an-unknown-encoding10626672 +Ref: 3f3710626672 +Node: References<5>10627575 +Ref: howto/unicode id310627705 +Ref: 3f3810627705 +Ref: References<5>-Footnote-110628315 +Ref: References<5>-Footnote-210628388 +Ref: References<5>-Footnote-310628493 +Node: Acknowledgements<10>10628561 +Ref: howto/unicode acknowledgements10628672 +Ref: 3f3910628672 +Node: HOWTO Fetch Internet Resources Using The urllib Package10629198 +Ref: howto/urllib2 doc10629351 +Ref: 3f3a10629351 +Ref: howto/urllib2 howto-fetch-internet-resources-using-the-urllib-package10629351 +Ref: 3f3b10629351 +Ref: howto/urllib2 urllib-howto10629351 +Ref: 29b510629351 +Ref: HOWTO Fetch Internet Resources Using The urllib Package-Footnote-110629898 +Ref: HOWTO Fetch Internet Resources Using The urllib Package-Footnote-210629953 +Node: Introduction<16>10630029 +Ref: howto/urllib2 introduction10630159 +Ref: 3f3c10630159 +Ref: Introduction<16>-Footnote-110631755 +Ref: Introduction<16>-Footnote-210631828 +Node: Fetching URLs10631877 +Ref: howto/urllib2 fetching-urls10632038 +Ref: 3f3d10632038 +Node: Data10634286 +Ref: howto/urllib2 data10634356 +Ref: 3f3e10634356 +Ref: Data-Footnote-110637009 +Node: Headers10637078 +Ref: howto/urllib2 headers10637148 +Ref: 3f3f10637148 +Ref: Headers-Footnote-110638662 +Ref: Headers-Footnote-210638694 +Ref: Headers-Footnote-310638908 +Ref: Headers-Footnote-410639032 +Node: Handling Exceptions<2>10639154 +Ref: howto/urllib2 handling-exceptions10639314 +Ref: 3f4110639314 +Node: URLError10639799 +Ref: howto/urllib2 urlerror10639884 +Ref: 3f4210639884 +Node: HTTPError10640438 +Ref: howto/urllib2 httperror10640546 +Ref: 3f4310640546 +Ref: HTTPError-Footnote-110641428 +Node: Error Codes10641477 +Ref: howto/urllib2 error-codes10641534 +Ref: 3f4410641534 +Ref: Error Codes-Footnote-110646287 +Node: Wrapping it Up10646336 +Ref: howto/urllib2 wrapping-it-up10646427 +Ref: 3f4510646427 +Node: Number 110646648 +Ref: howto/urllib2 number-110646724 +Ref: 3f4610646724 +Node: Number 210647346 +Ref: howto/urllib2 number-210647422 +Ref: 3f4710647422 +Node: info and geturl10647956 +Ref: howto/urllib2 info-and-geturl10648123 +Ref: 3f4010648123 +Ref: info and geturl-Footnote-110649009 +Node: Openers and Handlers10649046 +Ref: howto/urllib2 openers-and-handlers10649211 +Ref: 3f4810649211 +Node: Basic Authentication10650861 +Ref: howto/urllib2 id510651018 +Ref: 3f4910651018 +Ref: Basic Authentication-Footnote-110654433 +Node: Proxies10654506 +Ref: howto/urllib2 proxies10654661 +Ref: 3f4a10654661 +Ref: Proxies-Footnote-110655685 +Ref: Proxies-Footnote-210655992 +Ref: Proxies-Footnote-310656065 +Node: Sockets and Layers10656192 +Ref: howto/urllib2 sockets-and-layers10656336 +Ref: 3f4b10656336 +Node: Footnotes10657357 +Ref: howto/urllib2 footnotes10657485 +Ref: 3f4c10657485 +Node: Argparse Tutorial10657575 +Ref: howto/argparse doc10657754 +Ref: 3f4d10657754 +Ref: howto/argparse argparse-tutorial10657754 +Ref: 3f4e10657754 +Ref: howto/argparse id110657834 +Ref: 1d0510657834 +Node: Concepts10658515 +Ref: howto/argparse concepts10658596 +Ref: 3f4f10658596 +Node: The basics10660686 +Ref: howto/argparse the-basics10660808 +Ref: 3f5010660808 +Node: Introducing Positional arguments10661984 +Ref: howto/argparse introducing-positional-arguments10662128 +Ref: 3f5110662128 +Node: Introducing Optional arguments10665420 +Ref: howto/argparse introducing-optional-arguments10665597 +Ref: 3f5210665597 +Node: Short options10668606 +Ref: howto/argparse short-options10668686 +Ref: 3f5310668686 +Node: Combining Positional and Optional arguments10669492 +Ref: howto/argparse combining-positional-and-optional-arguments10669667 +Ref: 3f5410669667 +Node: Getting a little more advanced10677916 +Ref: howto/argparse getting-a-little-more-advanced10678071 +Ref: 3f5510678071 +Node: Conflicting options10680097 +Ref: howto/argparse conflicting-options10680183 +Ref: 3f5610680183 +Node: Conclusion10683343 +Ref: howto/argparse conclusion10683446 +Ref: 3f5710683446 +Node: An introduction to the ipaddress module10683711 +Ref: howto/ipaddress doc10683857 +Ref: 3f5810683857 +Ref: howto/ipaddress an-introduction-to-the-ipaddress-module10683857 +Ref: 3f5910683857 +Ref: howto/ipaddress ipaddress-howto10683857 +Ref: 2c1410683857 +Node: Creating Address/Network/Interface objects10684600 +Ref: howto/ipaddress creating-address-network-interface-objects10684771 +Ref: 3f5a10684771 +Node: A Note on IP Versions10685198 +Ref: howto/ipaddress a-note-on-ip-versions10685324 +Ref: 3f5b10685324 +Node: IP Host Addresses10686107 +Ref: howto/ipaddress ip-host-addresses10686259 +Ref: 3f5c10686259 +Node: Defining Networks10687446 +Ref: howto/ipaddress defining-networks10687592 +Ref: 3f5d10687592 +Node: Host Interfaces10689950 +Ref: howto/ipaddress host-interfaces10690070 +Ref: 3f5e10690070 +Node: Inspecting Address/Network/Interface Objects10691075 +Ref: howto/ipaddress inspecting-address-network-interface-objects10691285 +Ref: 3f5f10691285 +Node: Networks as lists of Addresses10693635 +Ref: howto/ipaddress networks-as-lists-of-addresses10693817 +Ref: 3f6010693817 +Node: Comparisons<4>10694616 +Ref: howto/ipaddress comparisons10694791 +Ref: 3f6110694791 +Node: Using IP Addresses with other modules10695150 +Ref: howto/ipaddress using-ip-addresses-with-other-modules10695343 +Ref: 3f6210695343 +Node: Getting more detail when instance creation fails10695774 +Ref: howto/ipaddress getting-more-detail-when-instance-creation-fails10695944 +Ref: 3f6310695944 +Node: Argument Clinic How-To10698002 +Ref: howto/clinic doc10698178 +Ref: 3f6410698178 +Ref: howto/clinic argument-clinic-how-to10698178 +Ref: 3f6510698178 +Node: The Goals Of Argument Clinic10699215 +Ref: howto/clinic the-goals-of-argument-clinic10699335 +Ref: 3f6610699335 +Node: Basic Concepts And Usage10701701 +Ref: howto/clinic basic-concepts-and-usage10701860 +Ref: 3f6710701860 +Node: Converting Your First Function10704295 +Ref: howto/clinic converting-your-first-function10704441 +Ref: 3f6810704441 +Node: Advanced Topics10720589 +Ref: howto/clinic advanced-topics10720702 +Ref: 3f6a10720702 +Node: Symbolic default values10721682 +Ref: howto/clinic symbolic-default-values10721833 +Ref: 3f6b10721833 +Node: Renaming the C functions and variables generated by Argument Clinic10722463 +Ref: howto/clinic renaming-the-c-functions-and-variables-generated-by-argument-clinic10722667 +Ref: 3f6c10722667 +Node: Converting functions using PyArg_UnpackTuple10724330 +Ref: howto/clinic converting-functions-using-pyarg-unpacktuple10724526 +Ref: 3f6d10724526 +Node: Optional Groups10725071 +Ref: howto/clinic optional-groups10725272 +Ref: 3f6e10725272 +Node: Using real Argument Clinic converters instead of “legacy converters”10729321 +Ref: howto/clinic using-real-argument-clinic-converters-instead-of-legacy-converters10729487 +Ref: 3f6f10729487 +Ref: Using real Argument Clinic converters instead of “legacy converters”-Footnote-110738425 +Node: Py_buffer10738474 +Ref: howto/clinic py-buffer10738644 +Ref: 3f7010738644 +Node: Advanced converters10738960 +Ref: howto/clinic advanced-converters10739082 +Ref: 3f7110739082 +Node: Parameter default values10740787 +Ref: howto/clinic default-values10740922 +Ref: 3f6910740922 +Ref: howto/clinic parameter-default-values10740922 +Ref: 3f7210740922 +Node: The NULL default value10741451 +Ref: howto/clinic the-null-default-value10741606 +Ref: 3f7310741606 +Node: Expressions specified as default values10742080 +Ref: howto/clinic expressions-specified-as-default-values10742235 +Ref: 3f7410742235 +Node: Using a return converter10744630 +Ref: howto/clinic using-a-return-converter10744789 +Ref: 3f7510744789 +Node: Cloning existing functions10747434 +Ref: howto/clinic cloning-existing-functions10747573 +Ref: 3f7610747573 +Node: Calling Python code10748897 +Ref: howto/clinic calling-python-code10749040 +Ref: 3f7710749040 +Node: Using a “self converter”10749959 +Ref: howto/clinic using-a-self-converter10750102 +Ref: 3f7810750102 +Node: Writing a custom converter10751848 +Ref: howto/clinic writing-a-custom-converter10752005 +Ref: 3f7910752005 +Node: Writing a custom return converter10755709 +Ref: howto/clinic writing-a-custom-return-converter10755860 +Ref: 3f7a10755860 +Node: METH_O and METH_NOARGS10756439 +Ref: howto/clinic meth-o-and-meth-noargs10756592 +Ref: 3f7b10756592 +Node: tp_new and tp_init functions10757163 +Ref: howto/clinic tp-new-and-tp-init-functions10757325 +Ref: 3f7c10757325 +Node: Changing and redirecting Clinic’s output10758270 +Ref: howto/clinic changing-and-redirecting-clinic-s-output10758426 +Ref: 3f7d10758426 +Node: The #ifdef trick10769706 +Ref: howto/clinic the-ifdef-trick10769871 +Ref: 3f7e10769871 +Node: Using Argument Clinic in Python files10772653 +Ref: howto/clinic using-argument-clinic-in-python-files10772767 +Ref: 3f7f10772767 +Node: Instrumenting CPython with DTrace and SystemTap10773463 +Ref: howto/instrumentation doc10773591 +Ref: 3f8010773591 +Ref: howto/instrumentation instrumentation10773591 +Ref: 50c10773591 +Ref: howto/instrumentation instrumenting-cpython-with-dtrace-and-systemtap10773591 +Ref: 3f8110773591 +Node: Enabling the static markers10774789 +Ref: howto/instrumentation enabling-the-static-markers10774929 +Ref: 3f8210774929 +Node: Static DTrace probes10779076 +Ref: howto/instrumentation static-dtrace-probes10779249 +Ref: 3f8310779249 +Node: Static SystemTap markers10781690 +Ref: howto/instrumentation static-systemtap-markers10781860 +Ref: 3f8410781860 +Node: Available static markers10783905 +Ref: howto/instrumentation available-static-markers10784072 +Ref: 3f8510784072 +Node: SystemTap Tapsets10786617 +Ref: howto/instrumentation systemtap-tapsets10786772 +Ref: 3f8610786772 +Node: Examples<36>10788392 +Ref: howto/instrumentation examples10788514 +Ref: 3f8710788514 +Node: Python Frequently Asked Questions10789837 +Ref: faq/index doc10789949 +Ref: 3f8810789949 +Ref: faq/index faq-index10789949 +Ref: fae10789949 +Ref: faq/index python-frequently-asked-questions10789949 +Ref: 3f8910789949 +Node: General Python FAQ10790273 +Ref: faq/general doc10790385 +Ref: 3f8a10790385 +Ref: faq/general general-python-faq10790385 +Ref: 3f8b10790385 +Node: General Information10790497 +Ref: faq/general general-information10790604 +Ref: 3f8c10790604 +Node: What is Python?10791593 +Ref: faq/general what-is-python10791712 +Ref: 3f8d10791712 +Ref: What is Python?-Footnote-110792661 +Node: What is the Python Software Foundation?10792713 +Ref: faq/general what-is-the-python-software-foundation10792895 +Ref: 3f8e10792895 +Ref: What is the Python Software Foundation?-Footnote-110793499 +Node: Are there copyright restrictions on the use of Python?10793545 +Ref: faq/general are-there-copyright-restrictions-on-the-use-of-python10793754 +Ref: 3f8f10793754 +Ref: Are there copyright restrictions on the use of Python?-Footnote-110794626 +Ref: Are there copyright restrictions on the use of Python?-Footnote-210794670 +Node: Why was Python created in the first place?10794717 +Ref: faq/general why-was-python-created-in-the-first-place10794911 +Ref: 3f9010794911 +Node: What is Python good for?10797243 +Ref: faq/general what-is-python-good-for10797433 +Ref: 3f9110797433 +Ref: What is Python good for?-Footnote-110798302 +Node: How does the Python version numbering scheme work?10798327 +Ref: faq/general how-does-the-python-version-numbering-scheme-work10798519 +Ref: 3f9210798519 +Ref: How does the Python version numbering scheme work?-Footnote-110800392 +Node: How do I obtain a copy of the Python source?10800441 +Ref: faq/general how-do-i-obtain-a-copy-of-the-python-source10800646 +Ref: 3f9310800646 +Ref: How do I obtain a copy of the Python source?-Footnote-110801430 +Node: How do I get documentation on Python?10801473 +Ref: faq/general how-do-i-get-documentation-on-python10801686 +Ref: 3f9410801686 +Ref: How do I get documentation on Python?-Footnote-110802249 +Node: I’ve never programmed before Is there a Python tutorial?10802280 +Ref: faq/general i-ve-never-programmed-before-is-there-a-python-tutorial10802504 +Ref: 3f9510802504 +Ref: I’ve never programmed before Is there a Python tutorial?-Footnote-110802916 +Node: Is there a newsgroup or mailing list devoted to Python?10802968 +Ref: faq/general is-there-a-newsgroup-or-mailing-list-devoted-to-python10803198 +Ref: 3f9610803198 +Ref: Is there a newsgroup or mailing list devoted to Python?-Footnote-110804077 +Ref: Is there a newsgroup or mailing list devoted to Python?-Footnote-210804138 +Node: How do I get a beta test version of Python?10804208 +Ref: faq/general how-do-i-get-a-beta-test-version-of-python10804431 +Ref: 3f9710804431 +Ref: How do I get a beta test version of Python?-Footnote-110804970 +Node: How do I submit bug reports and patches for Python?10805007 +Ref: faq/general how-do-i-submit-bug-reports-and-patches-for-python10805242 +Ref: 3f9810805242 +Ref: How do I submit bug reports and patches for Python?-Footnote-110805953 +Ref: How do I submit bug reports and patches for Python?-Footnote-210806010 +Node: Are there any published articles about Python that I can reference?10806047 +Ref: faq/general are-there-any-published-articles-about-python-that-i-can-reference10806269 +Ref: 3f9910806269 +Node: Are there any books on Python?10806782 +Ref: faq/general are-there-any-books-on-python10806998 +Ref: 3f9a10806998 +Node: Where in the world is www python org located?10807375 +Ref: faq/general where-in-the-world-is-www-python-org-located10807548 +Ref: 3f9b10807548 +Ref: Where in the world is www python org located?-Footnote-110807834 +Node: Why is it called Python?10807862 +Ref: faq/general why-is-it-called-python10808060 +Ref: 3f9c10808060 +Ref: Why is it called Python?-Footnote-110808466 +Node: Do I have to like “Monty Python’s Flying Circus”?10808517 +Ref: faq/general do-i-have-to-like-monty-python-s-flying-circus10808661 +Ref: 3f9d10808661 +Node: Python in the real world10808811 +Ref: faq/general python-in-the-real-world10808918 +Ref: 3f9e10808918 +Node: How stable is Python?10809298 +Ref: faq/general how-stable-is-python10809422 +Ref: 3f9f10809422 +Ref: How stable is Python?-Footnote-110810462 +Ref: How stable is Python?-Footnote-210810511 +Ref: How stable is Python?-Footnote-310810553 +Node: How many people are using Python?10810603 +Ref: faq/general how-many-people-are-using-python10810786 +Ref: 3fa010810786 +Node: Have any significant projects been done in Python?10811292 +Ref: faq/general have-any-significant-projects-been-done-in-python10811514 +Ref: 3fa110811514 +Ref: Have any significant projects been done in Python?-Footnote-110812228 +Ref: Have any significant projects been done in Python?-Footnote-210812280 +Ref: Have any significant projects been done in Python?-Footnote-310812308 +Ref: Have any significant projects been done in Python?-Footnote-410812336 +Node: What new developments are expected for Python in the future?10812367 +Ref: faq/general what-new-developments-are-expected-for-python-in-the-future10812615 +Ref: 3fa210812615 +Ref: What new developments are expected for Python in the future?-Footnote-110813204 +Node: Is it reasonable to propose incompatible changes to Python?10813265 +Ref: faq/general is-it-reasonable-to-propose-incompatible-changes-to-python10813515 +Ref: 3fa310813515 +Ref: Is it reasonable to propose incompatible changes to Python?-Footnote-110814319 +Node: Is Python a good language for beginning programmers?10814368 +Ref: faq/general is-python-a-good-language-for-beginning-programmers10814549 +Ref: 3fa410814549 +Ref: Is Python a good language for beginning programmers?-Footnote-110818408 +Ref: Is Python a good language for beginning programmers?-Footnote-210818459 +Node: Programming FAQ10818521 +Ref: faq/programming doc10818664 +Ref: 3fa510818664 +Ref: faq/programming programming-faq10818664 +Ref: 3fa610818664 +Node: General Questions10818899 +Ref: faq/programming general-questions10818990 +Ref: 3fa710818990 +Node: Is there a source code level debugger with breakpoints single-stepping etc ?10819410 +Ref: faq/programming is-there-a-source-code-level-debugger-with-breakpoints-single-stepping-etc10819610 +Ref: 3fa810819610 +Ref: Is there a source code level debugger with breakpoints single-stepping etc ?-Footnote-110821358 +Ref: Is there a source code level debugger with breakpoints single-stepping etc ?-Footnote-210821408 +Node: Are there tools to help find bugs or perform static analysis?10821453 +Ref: faq/programming are-there-tools-to-help-find-bugs-or-perform-static-analysis10821721 +Ref: 3fa910821721 +Ref: Are there tools to help find bugs or perform static analysis?-Footnote-110822095 +Ref: Are there tools to help find bugs or perform static analysis?-Footnote-210822127 +Ref: Are there tools to help find bugs or perform static analysis?-Footnote-310822169 +Ref: Are there tools to help find bugs or perform static analysis?-Footnote-410822199 +Ref: Are there tools to help find bugs or perform static analysis?-Footnote-510822231 +Node: How can I create a stand-alone binary from a Python script?10822272 +Ref: faq/programming how-can-i-create-a-stand-alone-binary-from-a-python-script10822528 +Ref: 3faa10822528 +Ref: How can I create a stand-alone binary from a Python script?-Footnote-110824185 +Node: Are there coding standards or a style guide for Python programs?10824239 +Ref: faq/programming are-there-coding-standards-or-a-style-guide-for-python-programs10824425 +Ref: 3fab10824425 +Ref: Are there coding standards or a style guide for Python programs?-Footnote-110824699 +Node: Core Language10824748 +Ref: faq/programming core-language10824867 +Ref: 3fac10824867 +Node: Why am I getting an UnboundLocalError when the variable has a value?10826237 +Ref: faq/programming why-am-i-getting-an-unboundlocalerror-when-the-variable-has-a-value10826424 +Ref: 3fad10826424 +Node: What are the rules for local and global variables in Python?10828283 +Ref: faq/programming what-are-the-rules-for-local-and-global-variables-in-python10828561 +Ref: 3fae10828561 +Node: Why do lambdas defined in a loop with different values all return the same result?10829461 +Ref: faq/programming why-do-lambdas-defined-in-a-loop-with-different-values-all-return-the-same-result10829718 +Ref: 3faf10829718 +Node: How do I share global variables across modules?10831627 +Ref: faq/programming how-do-i-share-global-variables-across-modules10831887 +Ref: 3fb010831887 +Node: What are the “best practices” for using import in a module?10832690 +Ref: faq/programming what-are-the-best-practices-for-using-import-in-a-module10832914 +Ref: 3fb110832914 +Node: Why are default values shared between objects?10835670 +Ref: faq/programming why-are-default-values-shared-between-objects10835922 +Ref: 3fb210835922 +Node: How can I pass optional or keyword parameters from one function to another?10838310 +Ref: faq/programming how-can-i-pass-optional-or-keyword-parameters-from-one-function-to-another10838555 +Ref: 3fb310838555 +Node: What is the difference between arguments and parameters?10839142 +Ref: faq/programming faq-argument-vs-parameter10839396 +Ref: 3fb410839396 +Ref: faq/programming what-is-the-difference-between-arguments-and-parameters10839396 +Ref: 3fb510839396 +Node: Why did changing list ‘y’ also change list ‘x’?10840086 +Ref: faq/programming why-did-changing-list-y-also-change-list-x10840333 +Ref: 3fb610840333 +Node: How do I write a function with output parameters call by reference ?10843734 +Ref: faq/programming how-do-i-write-a-function-with-output-parameters-call-by-reference10843975 +Ref: 3fb710843975 +Node: How do you make a higher order function in Python?10846380 +Ref: faq/programming how-do-you-make-a-higher-order-function-in-python10846600 +Ref: 3fb810846600 +Node: How do I copy an object in Python?10848174 +Ref: faq/programming how-do-i-copy-an-object-in-python10848380 +Ref: 3fb910848380 +Node: How can I find the methods or attributes of an object?10848787 +Ref: faq/programming how-can-i-find-the-methods-or-attributes-of-an-object10848990 +Ref: 3fba10848990 +Node: How can my code discover the name of an object?10849304 +Ref: faq/programming how-can-my-code-discover-the-name-of-an-object10849524 +Ref: 3fbb10849524 +Node: What’s up with the comma operator’s precedence?10851144 +Ref: faq/programming what-s-up-with-the-comma-operator-s-precedence10851368 +Ref: 3fbc10851368 +Node: Is there an equivalent of C’s “? ” ternary operator?10851926 +Ref: faq/programming is-there-an-equivalent-of-c-s-ternary-operator10852159 +Ref: 3fbd10852159 +Node: Is it possible to write obfuscated one-liners in Python?10852754 +Ref: faq/programming is-it-possible-to-write-obfuscated-one-liners-in-python10852999 +Ref: 3fbe10852999 +Node: What does the slash / in the parameter list of a function mean?10854478 +Ref: faq/programming faq-positional-only-arguments10854656 +Ref: 129d10854656 +Ref: faq/programming what-does-the-slash-in-the-parameter-list-of-a-function-mean10854656 +Ref: 3fbf10854656 +Node: Numbers and strings10855756 +Ref: faq/programming numbers-and-strings10855872 +Ref: 3fc010855872 +Node: How do I specify hexadecimal and octal integers?10856544 +Ref: faq/programming how-do-i-specify-hexadecimal-and-octal-integers10856686 +Ref: 3fc110856686 +Node: Why does -22 // 10 return -3?10857335 +Ref: faq/programming why-does-22-10-return-310857524 +Ref: 3fc210857524 +Node: How do I convert a string to a number?10858285 +Ref: faq/programming how-do-i-convert-a-string-to-a-number10858464 +Ref: 3fc310858464 +Node: How do I convert a number to a string?10859776 +Ref: faq/programming how-do-i-convert-a-number-to-a-string10859960 +Ref: 3fc410859960 +Node: How do I modify a string in place?10860511 +Ref: faq/programming how-do-i-modify-a-string-in-place10860704 +Ref: 3fc510860704 +Node: How do I use strings to call functions/methods?10861560 +Ref: faq/programming how-do-i-use-strings-to-call-functions-methods10861800 +Ref: 3fc610861800 +Node: Is there an equivalent to Perl’s chomp for removing trailing newlines from strings?10863425 +Ref: faq/programming is-there-an-equivalent-to-perl-s-chomp-for-removing-trailing-newlines-from-strings10863669 +Ref: 3fc710863669 +Node: Is there a scanf or sscanf equivalent?10864428 +Ref: faq/programming is-there-a-scanf-or-sscanf-equivalent10864699 +Ref: 3fc810864699 +Node: What does ‘UnicodeDecodeError’ or ‘UnicodeEncodeError’ error mean?10865350 +Ref: faq/programming what-does-unicodedecodeerror-or-unicodeencodeerror-error-mean10865527 +Ref: 3fc910865527 +Node: Performance<4>10865723 +Ref: faq/programming performance10865848 +Ref: 3fca10865848 +Node: My program is too slow How do I speed it up?10866065 +Ref: faq/programming my-program-is-too-slow-how-do-i-speed-it-up10866237 +Ref: 3fcb10866237 +Ref: My program is too slow How do I speed it up?-Footnote-110869116 +Ref: My program is too slow How do I speed it up?-Footnote-210869142 +Node: What is the most efficient way to concatenate many strings together?10869207 +Ref: faq/programming efficient-string-concatenation10869379 +Ref: 3fcc10869379 +Ref: faq/programming what-is-the-most-efficient-way-to-concatenate-many-strings-together10869379 +Ref: 3fcd10869379 +Node: Sequences Tuples/Lists10870349 +Ref: faq/programming sequences-tuples-lists10870462 +Ref: 3fce10870462 +Node: How do I convert between tuples and lists?10871206 +Ref: faq/programming how-do-i-convert-between-tuples-and-lists10871342 +Ref: 3fcf10871342 +Node: What’s a negative index?10872189 +Ref: faq/programming what-s-a-negative-index10872384 +Ref: 3fd010872384 +Node: How do I iterate over a sequence in reverse order?10872964 +Ref: faq/programming how-do-i-iterate-over-a-sequence-in-reverse-order10873158 +Ref: 3fd110873158 +Node: How do you remove duplicates from a list?10873507 +Ref: faq/programming how-do-you-remove-duplicates-from-a-list10873719 +Ref: 3fd210873719 +Node: How do you remove multiple items from a list10874547 +Ref: faq/programming how-do-you-remove-multiple-items-from-a-list10874744 +Ref: 3fd310874744 +Node: How do you make an array in Python?10875292 +Ref: faq/programming how-do-you-make-an-array-in-python10875488 +Ref: 3fd410875488 +Node: How do I create a multidimensional list?10876472 +Ref: faq/programming faq-multidimensional-list10876673 +Ref: 12dc10876673 +Ref: faq/programming how-do-i-create-a-multidimensional-list10876673 +Ref: 3fd510876673 +Ref: How do I create a multidimensional list?-Footnote-110877895 +Node: How do I apply a method to a sequence of objects?10877925 +Ref: faq/programming how-do-i-apply-a-method-to-a-sequence-of-objects10878170 +Ref: 3fd610878170 +Node: Why does a_tuple[i] += [‘item’] raise an exception when the addition works?10878364 +Ref: faq/programming faq-augmented-assignment-tuple-error10878646 +Ref: 111e10878646 +Ref: faq/programming why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works10878646 +Ref: 3fd710878646 +Node: I want to do a complicated sort can you do a Schwartzian Transform in Python?10881870 +Ref: faq/programming i-want-to-do-a-complicated-sort-can-you-do-a-schwartzian-transform-in-python10882155 +Ref: 3fd810882155 +Node: How can I sort one list by values from another list?10882640 +Ref: faq/programming how-can-i-sort-one-list-by-values-from-another-list10882837 +Ref: 3fd910882837 +Node: Objects10884051 +Ref: faq/programming objects10884160 +Ref: 3fda10884160 +Node: What is a class?10885169 +Ref: faq/programming what-is-a-class10885255 +Ref: 3fdb10885255 +Node: What is a method?10885981 +Ref: faq/programming what-is-a-method10886089 +Ref: 3fdc10886089 +Node: What is self?10886398 +Ref: faq/programming what-is-self10886573 +Ref: 3fdd10886573 +Node: How do I check if an object is an instance of a given class or of a subclass of it?10887015 +Ref: faq/programming how-do-i-check-if-an-object-is-an-instance-of-a-given-class-or-of-a-subclass-of-it10887192 +Ref: 3fdf10887192 +Node: What is delegation?10888683 +Ref: faq/programming what-is-delegation10888933 +Ref: 3fe010888933 +Node: How do I call a method defined in a base class from a derived class that overrides it?10890783 +Ref: faq/programming how-do-i-call-a-method-defined-in-a-base-class-from-a-derived-class-that-overrides-it10891020 +Ref: 3fe110891020 +Node: How can I organize my code to make it easier to change the base class?10891705 +Ref: faq/programming how-can-i-organize-my-code-to-make-it-easier-to-change-the-base-class10891982 +Ref: 3fe210891982 +Node: How do I create static class data and static class methods?10892657 +Ref: faq/programming how-do-i-create-static-class-data-and-static-class-methods10892901 +Ref: 3fe310892901 +Node: How can I overload constructors or methods in Python?10894470 +Ref: faq/programming how-can-i-overload-constructors-or-methods-in-python10894709 +Ref: 3fe410894709 +Node: I try to use __spam and I get an error about _SomeClassName__spam10895616 +Ref: faq/programming i-try-to-use-spam-and-i-get-an-error-about-someclassname-spam10895866 +Ref: 3fe510895866 +Node: My class defines __del__ but it is not called when I delete the object10896667 +Ref: faq/programming my-class-defines-del-but-it-is-not-called-when-i-delete-the-object10896918 +Ref: 3fe610896918 +Node: How do I get a list of all instances of a given class?10898892 +Ref: faq/programming how-do-i-get-a-list-of-all-instances-of-a-given-class10899128 +Ref: 3fe710899128 +Node: Why does the result of id appear to be not unique?10899469 +Ref: faq/programming why-does-the-result-of-id-appear-to-be-not-unique10899626 +Ref: 3fe810899626 +Node: Modules<3>10900518 +Ref: faq/programming modules10900596 +Ref: 3fe910900596 +Node: How do I create a pyc file?10901136 +Ref: faq/programming how-do-i-create-a-pyc-file10901257 +Ref: 3fea10901257 +Ref: How do I create a pyc file?-Footnote-110903838 +Node: How do I find the current module name?10903887 +Ref: faq/programming how-do-i-find-the-current-module-name10904072 +Ref: 3feb10904072 +Node: How can I have modules that mutually import each other?10904630 +Ref: faq/programming how-can-i-have-modules-that-mutually-import-each-other10904852 +Ref: 3fec10904852 +Node: __import__ ‘x y z’ returns ; how do I get z?10906738 +Ref: faq/programming import-x-y-z-returns-module-x-how-do-i-get-z10907018 +Ref: 3fed10907018 +Node: When I edit an imported module and reimport it the changes don’t show up Why does this happen?10907308 +Ref: faq/programming when-i-edit-an-imported-module-and-reimport-it-the-changes-don-t-show-up-why-does-this-happen10907524 +Ref: 3fee10907524 +Node: Design and History FAQ10908971 +Ref: faq/design doc10909121 +Ref: 3fef10909121 +Ref: faq/design design-and-history-faq10909121 +Ref: 3ff010909121 +Node: Why does Python use indentation for grouping of statements?10911165 +Ref: faq/design why-does-python-use-indentation-for-grouping-of-statements10911359 +Ref: 3ff110911359 +Node: Why am I getting strange results with simple arithmetic operations?10913061 +Ref: faq/design why-am-i-getting-strange-results-with-simple-arithmetic-operations10913314 +Ref: 3ff210913314 +Node: Why are floating-point calculations so inaccurate?10913489 +Ref: faq/design why-are-floating-point-calculations-so-inaccurate10913716 +Ref: 3ff310913716 +Node: Why are Python strings immutable?10915254 +Ref: faq/design why-are-python-strings-immutable10915485 +Ref: 3ff410915485 +Node: Why must ‘self’ be used explicitly in method definitions and calls?10916088 +Ref: faq/design why-must-self-be-used-explicitly-in-method-definitions-and-calls10916318 +Ref: 3ff510916318 +Ref: faq/design why-self10916318 +Ref: 3fde10916318 +Node: Why can’t I use an assignment in an expression?10918808 +Ref: faq/design id110919109 +Ref: 3ff610919109 +Ref: faq/design why-can-t-i-use-an-assignment-in-an-expression10919109 +Ref: f0c10919109 +Ref: Why can’t I use an assignment in an expression?-Footnote-110919481 +Node: Why does Python use methods for some functionality e g list index but functions for other e g len list ?10919530 +Ref: faq/design why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list10919822 +Ref: 3ff710919822 +Node: Why is join a string method instead of a list or tuple method?10921161 +Ref: faq/design why-is-join-a-string-method-instead-of-a-list-or-tuple-method10921428 +Ref: 3ff810921428 +Node: How fast are exceptions?10923239 +Ref: faq/design how-fast-are-exceptions10923457 +Ref: 3ff910923457 +Node: Why isn’t there a switch or case statement in Python?10924270 +Ref: faq/design why-isn-t-there-a-switch-or-case-statement-in-python10924532 +Ref: 3ffa10924532 +Ref: Why isn’t there a switch or case statement in Python?-Footnote-110925905 +Node: Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?10925954 +Ref: faq/design can-t-you-emulate-threads-in-the-interpreter-instead-of-relying-on-an-os-specific-thread-implementation10926242 +Ref: 3ffb10926242 +Ref: Can’t you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?-Footnote-110926889 +Node: Why can’t lambda expressions contain statements?10926945 +Ref: faq/design why-can-t-lambda-expressions-contain-statements10927242 +Ref: 3ffc10927242 +Node: Can Python be compiled to machine code C or some other language?10928107 +Ref: faq/design can-python-be-compiled-to-machine-code-c-or-some-other-language10928328 +Ref: 3ffd10928328 +Ref: Can Python be compiled to machine code C or some other language?-Footnote-110928763 +Ref: Can Python be compiled to machine code C or some other language?-Footnote-210928790 +Ref: Can Python be compiled to machine code C or some other language?-Footnote-310928821 +Node: How does Python manage memory?10928856 +Ref: faq/design how-does-python-manage-memory10929098 +Ref: 3ffe10929098 +Ref: How does Python manage memory?-Footnote-110930767 +Ref: How does Python manage memory?-Footnote-210930797 +Node: Why doesn’t CPython use a more traditional garbage collection scheme?10930825 +Ref: faq/design why-doesn-t-cpython-use-a-more-traditional-garbage-collection-scheme10931051 +Ref: 3fff10931051 +Node: Why isn’t all memory freed when CPython exits?10931967 +Ref: faq/design why-isn-t-all-memory-freed-when-cpython-exits10932212 +Ref: 400010932212 +Node: Why are there separate tuple and list data types?10932907 +Ref: faq/design why-are-there-separate-tuple-and-list-data-types10933118 +Ref: 400110933118 +Node: How are lists implemented in CPython?10934352 +Ref: faq/design how-are-lists-implemented-in-cpython10934559 +Ref: 400210934559 +Node: How are dictionaries implemented in CPython?10935300 +Ref: faq/design how-are-dictionaries-implemented-in-cpython10935496 +Ref: 400310935496 +Node: Why must dictionary keys be immutable?10936447 +Ref: faq/design why-must-dictionary-keys-be-immutable10936653 +Ref: 400410936653 +Node: Why doesn’t list sort return the sorted list?10940856 +Ref: faq/design why-doesn-t-list-sort-return-the-sorted-list10941077 +Ref: 400510941077 +Node: How do you specify and enforce an interface spec in Python?10941921 +Ref: faq/design how-do-you-specify-and-enforce-an-interface-spec-in-python10942125 +Ref: 400610942125 +Node: Why is there no goto?10944458 +Ref: faq/design why-is-there-no-goto10944670 +Ref: 400710944670 +Node: Why can’t raw strings r-strings end with a backslash?10945748 +Ref: faq/design why-can-t-raw-strings-r-strings-end-with-a-backslash10945976 +Ref: 400810945976 +Node: Why doesn’t Python have a “with” statement for attribute assignments?10947077 +Ref: faq/design why-doesn-t-python-have-a-with-statement-for-attribute-assignments10947346 +Ref: 400910947346 +Node: Why are colons required for the if/while/def/class statements?10949547 +Ref: faq/design why-are-colons-required-for-the-if-while-def-class-statements10949821 +Ref: 400a10949821 +Node: Why does Python allow commas at the end of lists and tuples?10950566 +Ref: faq/design why-does-python-allow-commas-at-the-end-of-lists-and-tuples10950756 +Ref: 400b10950756 +Node: Library and Extension FAQ10951857 +Ref: faq/library doc10952015 +Ref: 400c10952015 +Ref: faq/library library-and-extension-faq10952015 +Ref: 400d10952015 +Node: General Library Questions10952264 +Ref: faq/library general-library-questions10952372 +Ref: 400e10952372 +Node: How do I find a module or application to perform task X?10952884 +Ref: faq/library how-do-i-find-a-module-or-application-to-perform-task-x10953067 +Ref: 400f10953067 +Ref: How do I find a module or application to perform task X?-Footnote-110953654 +Ref: How do I find a module or application to perform task X?-Footnote-210953679 +Node: Where is the math py socket py regex py etc source file?10953710 +Ref: faq/library where-is-the-math-py-socket-py-regex-py-etc-source-file10953951 +Ref: 401010953951 +Node: How do I make a Python script executable on Unix?10954735 +Ref: faq/library how-do-i-make-a-python-script-executable-on-unix10954965 +Ref: 401110954965 +Node: Is there a curses/termcap package for Python?10956442 +Ref: faq/library is-there-a-curses-termcap-package-for-python10956665 +Ref: 401210956665 +Ref: Is there a curses/termcap package for Python?-Footnote-110957504 +Ref: Is there a curses/termcap package for Python?-Footnote-210957563 +Node: Is there an equivalent to C’s onexit in Python?10957612 +Ref: faq/library is-there-an-equivalent-to-c-s-onexit-in-python10957822 +Ref: 401310957822 +Node: Why don’t my signal handlers work?10958042 +Ref: faq/library why-don-t-my-signal-handlers-work10958198 +Ref: 401410958198 +Node: Common tasks10958522 +Ref: faq/library common-tasks10958646 +Ref: 401510958646 +Node: How do I test a Python program or component?10958844 +Ref: faq/library how-do-i-test-a-python-program-or-component10958993 +Ref: 401610958993 +Node: How do I create documentation from doc strings?10960944 +Ref: faq/library how-do-i-create-documentation-from-doc-strings10961143 +Ref: 401710961143 +Ref: How do I create documentation from doc strings?-Footnote-110961519 +Ref: How do I create documentation from doc strings?-Footnote-210961558 +Node: How do I get a single keypress at a time?10961588 +Ref: faq/library how-do-i-get-a-single-keypress-at-a-time10961734 +Ref: 401810961734 +Node: Threads10961980 +Ref: faq/library threads10962098 +Ref: 401910962098 +Node: How do I program using threads?10962426 +Ref: faq/library how-do-i-program-using-threads10962545 +Ref: 401a10962545 +Node: None of my threads seem to run why?10962978 +Ref: faq/library none-of-my-threads-seem-to-run-why10963163 +Ref: 401b10963163 +Node: How do I parcel out work among a bunch of worker threads?10964712 +Ref: faq/library how-do-i-parcel-out-work-among-a-bunch-of-worker-threads10964918 +Ref: 401c10964918 +Node: What kinds of global value mutation are thread-safe?10967518 +Ref: faq/library what-kinds-of-global-value-mutation-are-thread-safe10967739 +Ref: 401d10967739 +Node: Can’t we get rid of the Global Interpreter Lock?10969245 +Ref: faq/library can-t-we-get-rid-of-the-global-interpreter-lock10969400 +Ref: 401e10969400 +Ref: Can’t we get rid of the Global Interpreter Lock?-Footnote-110972091 +Node: Input and Output<2>10972151 +Ref: faq/library input-and-output10972285 +Ref: 401f10972285 +Node: How do I delete a file? And other file questions…10972920 +Ref: faq/library how-do-i-delete-a-file-and-other-file-questions10973057 +Ref: 402010973057 +Node: How do I copy a file?10974307 +Ref: faq/library how-do-i-copy-a-file10974488 +Ref: 402110974488 +Node: How do I read or write binary data?10974699 +Ref: faq/library how-do-i-read-or-write-binary-data10974897 +Ref: 402210974897 +Node: I can’t seem to use os read on a pipe created with os popen ; why?10976094 +Ref: faq/library i-can-t-seem-to-use-os-read-on-a-pipe-created-with-os-popen-why10976309 +Ref: 402310976309 +Node: How do I access the serial RS232 port?10976830 +Ref: faq/library how-do-i-access-the-serial-rs232-port10977072 +Ref: 402410977072 +Node: Why doesn’t closing sys stdout stdin stderr really close it?10977384 +Ref: faq/library why-doesn-t-closing-sys-stdout-stdin-stderr-really-close-it10977549 +Ref: 402510977549 +Node: Network/Internet Programming10978776 +Ref: faq/library network-internet-programming10978912 +Ref: 402610978912 +Node: What WWW tools are there for Python?10979369 +Ref: faq/library what-www-tools-are-there-for-python10979528 +Ref: 402710979528 +Node: How can I mimic CGI form submission METHOD=POST ?10980112 +Ref: faq/library how-can-i-mimic-cgi-form-submission-method-post10980334 +Ref: 402810980334 +Node: What module should I use to help with generating HTML?10981438 +Ref: faq/library what-module-should-i-use-to-help-with-generating-html10981664 +Ref: 402910981664 +Ref: What module should I use to help with generating HTML?-Footnote-110981909 +Node: How do I send mail from a Python script?10981961 +Ref: faq/library how-do-i-send-mail-from-a-python-script10982196 +Ref: 402a10982196 +Node: How do I avoid blocking in the connect method of a socket?10983537 +Ref: faq/library how-do-i-avoid-blocking-in-the-connect-method-of-a-socket10983709 +Ref: 402b10983709 +Ref: How do I avoid blocking in the connect method of a socket?-Footnote-110984961 +Node: Databases10985001 +Ref: faq/library databases10985142 +Ref: 402c10985142 +Node: Are there any interfaces to database packages in Python?10985303 +Ref: faq/library are-there-any-interfaces-to-database-packages-in-python10985464 +Ref: 402d10985464 +Ref: Are there any interfaces to database packages in Python?-Footnote-110985970 +Node: How do you implement persistent objects in Python?10986027 +Ref: faq/library how-do-you-implement-persistent-objects-in-python10986188 +Ref: 402e10986188 +Node: Mathematics and Numerics10986589 +Ref: faq/library mathematics-and-numerics10986693 +Ref: 402f10986693 +Node: How do I generate random numbers in Python?10986816 +Ref: faq/library how-do-i-generate-random-numbers-in-python10986920 +Ref: 403010986920 +Node: Extending/Embedding FAQ10987869 +Ref: faq/extending doc10988026 +Ref: 403110988026 +Ref: faq/extending extending-embedding-faq10988026 +Ref: 403210988026 +Node: Can I create my own functions in C?10989514 +Ref: faq/extending can-i-create-my-own-functions-in-c10989655 +Ref: 403310989655 +Node: Can I create my own functions in C++?10990014 +Ref: faq/extending id110990210 +Ref: 403410990210 +Node: Writing C is hard; are there any alternatives?10990597 +Ref: faq/extending c-wrapper-software10990811 +Ref: 403510990811 +Ref: faq/extending writing-c-is-hard-are-there-any-alternatives10990811 +Ref: 403610990811 +Ref: Writing C is hard; are there any alternatives?-Footnote-110991580 +Ref: Writing C is hard; are there any alternatives?-Footnote-210991606 +Ref: Writing C is hard; are there any alternatives?-Footnote-310991673 +Ref: Writing C is hard; are there any alternatives?-Footnote-410991701 +Ref: Writing C is hard; are there any alternatives?-Footnote-510991759 +Ref: Writing C is hard; are there any alternatives?-Footnote-610991795 +Ref: Writing C is hard; are there any alternatives?-Footnote-710991851 +Node: How can I execute arbitrary Python statements from C?10991890 +Ref: faq/extending how-can-i-execute-arbitrary-python-statements-from-c10992124 +Ref: 403710992124 +Node: How can I evaluate an arbitrary Python expression from C?10992670 +Ref: faq/extending how-can-i-evaluate-an-arbitrary-python-expression-from-c10992905 +Ref: 403810992905 +Node: How do I extract C values from a Python object?10993221 +Ref: faq/extending how-do-i-extract-c-values-from-a-python-object10993468 +Ref: 403910993468 +Node: How do I use Py_BuildValue to create a tuple of arbitrary length?10994707 +Ref: faq/extending how-do-i-use-py-buildvalue-to-create-a-tuple-of-arbitrary-length10994939 +Ref: 403a10994939 +Node: How do I call an object’s method from C?10995146 +Ref: faq/extending how-do-i-call-an-object-s-method-from-c10995416 +Ref: 403b10995416 +Node: How do I catch the output from PyErr_Print or anything that prints to stdout/stderr ?10996641 +Ref: faq/extending how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr10996896 +Ref: 403c10996896 +Node: How do I access a module written in Python from C?10998109 +Ref: faq/extending how-do-i-access-a-module-written-in-python-from-c10998368 +Ref: 403d10998368 +Node: How do I interface to C++ objects from Python?10999202 +Ref: faq/extending how-do-i-interface-to-c-objects-from-python10999438 +Ref: 403e10999438 +Node: I added a module using the Setup file and the make fails; why?11000009 +Ref: faq/extending i-added-a-module-using-the-setup-file-and-the-make-fails-why11000223 +Ref: 403f11000223 +Node: How do I debug an extension?11000573 +Ref: faq/extending how-do-i-debug-an-extension11000825 +Ref: 404011000825 +Node: I want to compile a Python module on my Linux system but some files are missing Why?11001376 +Ref: faq/extending i-want-to-compile-a-python-module-on-my-linux-system-but-some-files-are-missing-why11001628 +Ref: 404111001628 +Node: How do I tell “incomplete input” from “invalid input”?11002112 +Ref: faq/extending how-do-i-tell-incomplete-input-from-invalid-input11002404 +Ref: 404211002404 +Node: How do I find undefined g++ symbols __builtin_new or __pure_virtual?11008516 +Ref: faq/extending how-do-i-find-undefined-g-symbols-builtin-new-or-pure-virtual11008834 +Ref: 404311008834 +Node: Can I create an object class with some methods implemented in C and others in Python e g through inheritance ?11009222 +Ref: faq/extending can-i-create-an-object-class-with-some-methods-implemented-in-c-and-others-in-python-e-g-through-inheritance11009469 +Ref: 404411009469 +Node: Python on Windows FAQ11010022 +Ref: faq/windows doc11010180 +Ref: 404511010180 +Ref: faq/windows python-on-windows-faq11010180 +Ref: 404611010180 +Ref: faq/windows windows-faq11010180 +Ref: 404711010180 +Node: How do I run a Python program under Windows?11010696 +Ref: faq/windows faq-run-program-under-windows11010847 +Ref: 404811010847 +Ref: faq/windows how-do-i-run-a-python-program-under-windows11010847 +Ref: 404911010847 +Node: How do I make Python scripts executable?11014439 +Ref: faq/windows how-do-i-make-python-scripts-executable11014647 +Ref: 404a11014647 +Node: Why does Python sometimes take so long to start?11015209 +Ref: faq/windows why-does-python-sometimes-take-so-long-to-start11015422 +Ref: 404b11015422 +Node: How do I make an executable from a Python script?11016301 +Ref: faq/windows how-do-i-make-an-executable-from-a-python-script11016508 +Ref: 404c11016508 +Ref: How do I make an executable from a Python script?-Footnote-111016930 +Ref: How do I make an executable from a Python script?-Footnote-211016982 +Node: Is a * pyd file the same as a DLL?11017013 +Ref: faq/windows is-a-pyd-file-the-same-as-a-dll11017222 +Ref: 404d11017222 +Node: How can I embed Python into a Windows application?11018219 +Ref: faq/windows how-can-i-embed-python-into-a-windows-application11018443 +Ref: 404e11018443 +Node: How do I keep editors from inserting tabs into my Python source?11022981 +Ref: faq/windows how-do-i-keep-editors-from-inserting-tabs-into-my-python-source11023218 +Ref: 404f11023218 +Ref: How do I keep editors from inserting tabs into my Python source?-Footnote-111024095 +Node: How do I check for a keypress without blocking?11024144 +Ref: faq/windows how-do-i-check-for-a-keypress-without-blocking11024322 +Ref: 405011024322 +Node: Graphic User Interface FAQ11024671 +Ref: faq/gui doc11024855 +Ref: 405111024855 +Ref: faq/gui graphic-user-interface-faq11024855 +Ref: 405211024855 +Node: General GUI Questions11025097 +Ref: faq/gui general-gui-questions11025246 +Ref: 405311025246 +Node: What platform-independent GUI toolkits exist for Python?11025305 +Ref: faq/gui what-platform-independent-gui-toolkits-exist-for-python11025516 +Ref: 405411025516 +Node: Tkinter11025935 +Ref: faq/gui tkinter11026053 +Ref: 405511026053 +Ref: Tkinter-Footnote-111026522 +Ref: Tkinter-Footnote-211026564 +Node: wxWidgets11026591 +Ref: faq/gui wxwidgets11026720 +Ref: 405711026720 +Ref: wxWidgets-Footnote-111027582 +Node: Qt11027615 +Ref: faq/gui qt11027741 +Ref: 405611027741 +Ref: Qt-Footnote-111028217 +Ref: Qt-Footnote-211028276 +Ref: Qt-Footnote-311028310 +Ref: Qt-Footnote-411028374 +Ref: Qt-Footnote-511028440 +Node: Gtk+11028477 +Ref: faq/gui gtk11028598 +Ref: 405811028598 +Ref: Gtk+-Footnote-111028926 +Ref: Gtk+-Footnote-211028976 +Ref: Gtk+-Footnote-311029029 +Node: Kivy11029057 +Ref: faq/gui kivy11029180 +Ref: 405911029180 +Ref: Kivy-Footnote-111029540 +Node: FLTK11029566 +Ref: faq/gui fltk11029691 +Ref: 405a11029691 +Ref: FLTK-Footnote-111029906 +Ref: FLTK-Footnote-211029934 +Node: OpenGL11029972 +Ref: faq/gui opengl11030084 +Ref: 405b11030084 +Ref: OpenGL-Footnote-111030192 +Node: What platform-specific GUI toolkits exist for Python?11030232 +Ref: faq/gui what-platform-specific-gui-toolkits-exist-for-python11030439 +Ref: 405c11030439 +Ref: What platform-specific GUI toolkits exist for Python?-Footnote-111030893 +Node: Tkinter questions11030934 +Ref: faq/gui tkinter-questions11031076 +Ref: 405d11031076 +Node: How do I freeze Tkinter applications?11031342 +Ref: faq/gui how-do-i-freeze-tkinter-applications11031493 +Ref: 405e11031493 +Node: Can I have Tk events handled while waiting for I/O?11032440 +Ref: faq/gui can-i-have-tk-events-handled-while-waiting-for-i-o11032650 +Ref: 405f11032650 +Node: I can’t get key bindings to work in Tkinter why?11033125 +Ref: faq/gui i-can-t-get-key-bindings-to-work-in-tkinter-why11033289 +Ref: 406011033289 +Node: “Why is Python Installed on my Computer?” FAQ11033841 +Ref: faq/installed doc11033995 +Ref: 406111033995 +Ref: faq/installed why-is-python-installed-on-my-computer-faq11033995 +Ref: 406211033995 +Node: What is Python?<2>11034219 +Ref: faq/installed what-is-python11034370 +Ref: 406311034370 +Ref: What is Python?<2>-Footnote-111034851 +Node: Why is Python installed on my machine?11034903 +Ref: faq/installed why-is-python-installed-on-my-machine11035083 +Ref: 406411035083 +Node: Can I delete Python?11036171 +Ref: faq/installed can-i-delete-python11036324 +Ref: 406511036324 +Node: Glossary11037052 +Ref: glossary doc11037172 +Ref: 406611037172 +Ref: glossary glossary11037172 +Ref: e9211037172 +Ref: glossary id111037172 +Ref: 406711037172 +Ref: glossary term-011037197 +Ref: eac11037197 +Ref: glossary term-111037360 +Ref: ead11037360 +Ref: glossary term-2to311037754 +Ref: 406811037754 +Ref: glossary term-abstract-base-class11038168 +Ref: b3311038168 +Ref: glossary term-annotation11039005 +Ref: 406a11039005 +Ref: glossary term-argument11039576 +Ref: 11ca11039576 +Ref: glossary term-asynchronous-context-manager11041009 +Ref: 64111041009 +Ref: glossary term-asynchronous-generator11041243 +Ref: 11a911041243 +Ref: glossary term-asynchronous-generator-iterator11041968 +Ref: 335c11041968 +Ref: glossary term-asynchronous-iterable11042718 +Ref: 64011042718 +Ref: glossary term-asynchronous-iterator11042941 +Ref: 64511042941 +Ref: glossary term-attribute11043357 +Ref: 406d11043357 +Ref: glossary term-awaitable11043557 +Ref: 51911043557 +Ref: glossary term-bdfl11043754 +Ref: 406e11043754 +Ref: glossary term-binary-file11043851 +Ref: 196611043851 +Ref: glossary term-bytes-like-object11044296 +Ref: 5e811044296 +Ref: glossary term-bytecode11045281 +Ref: 61411045281 +Ref: glossary term-callback11046011 +Ref: 229611046011 +Ref: glossary term-class11046129 +Ref: 12ac11046129 +Ref: glossary term-class-variable11046294 +Ref: 406f11046294 +Ref: glossary term-coercion11046440 +Ref: 407011046440 +Ref: glossary term-complex-number11047079 +Ref: 407111047079 +Ref: glossary term-context-manager11047850 +Ref: 56811047850 +Ref: glossary term-context-variable11048051 +Ref: 407211048051 +Ref: glossary term-contiguous11048509 +Ref: 136011048509 +Ref: glossary term-coroutine11049052 +Ref: 1a611049052 +Ref: glossary term-coroutine-function11049375 +Ref: 63f11049375 +Ref: glossary term-cpython11049689 +Ref: 10d711049689 +Ref: glossary term-decorator11049942 +Ref: 28311049942 +Ref: glossary term-descriptor11050654 +Ref: 12b011050654 +Ref: glossary term-dictionary11051453 +Ref: 407311051453 +Ref: glossary term-dictionary-comprehension11051659 +Ref: 407411051659 +Ref: glossary term-dictionary-view11052005 +Ref: 15f411052005 +Ref: glossary term-docstring11052451 +Ref: 125e11052451 +Ref: glossary term-duck-typing11052842 +Ref: 406911052842 +Ref: glossary term-eafp11053546 +Ref: 1c2011053546 +Ref: glossary term-expression11053992 +Ref: 335011053992 +Ref: glossary term-extension-module11054505 +Ref: 407611054505 +Ref: glossary term-f-string11054633 +Ref: 15b11054633 +Ref: glossary term-file-object11054828 +Ref: b4211054828 +Ref: glossary term-file-like-object11055604 +Ref: 192811055604 +Ref: glossary term-finder11055666 +Ref: 115f11055666 +Ref: glossary term-floor-division11056057 +Ref: eb211056057 +Ref: glossary term-function11056433 +Ref: 11c911056433 +Ref: glossary term-function-annotation11056740 +Ref: ef111056740 +Ref: glossary term-future11057337 +Ref: 68311057337 +Ref: glossary term-garbage-collection11057815 +Ref: f9f11057815 +Ref: glossary term-generator11058135 +Ref: 9a211058135 +Ref: glossary term-generator-iterator11058651 +Ref: 45711058651 +Ref: glossary term-generator-expression11059049 +Ref: 407711059049 +Ref: glossary term-generic-function11059430 +Ref: 1ca11059430 +Ref: glossary term-gil11059789 +Ref: bea11059789 +Ref: glossary term-global-interpreter-lock11059840 +Ref: 50611059840 +Ref: glossary term-hash-based-pyc11061017 +Ref: 407811061017 +Ref: glossary term-hashable11061231 +Ref: 10c811061231 +Ref: glossary term-idle11062146 +Ref: 407911062146 +Ref: glossary term-immutable11062324 +Ref: eb611062324 +Ref: glossary term-import-path11062668 +Ref: 116211062668 +Ref: glossary term-importing11063007 +Ref: 115111063007 +Ref: glossary term-importer11063128 +Ref: 116111063128 +Ref: glossary term-interactive11063254 +Ref: 407a11063254 +Ref: glossary term-interpreted11063676 +Ref: 407b11063676 +Ref: glossary term-interpreter-shutdown11064169 +Ref: 76311064169 +Ref: glossary term-iterable11064903 +Ref: bac11064903 +Ref: glossary term-iterator11066098 +Ref: 112e11066098 +Ref: glossary term-key-function11067320 +Ref: 6e011067320 +Ref: glossary term-keyword-argument11068491 +Ref: 5c411068491 +Ref: glossary term-lambda11068541 +Ref: 33f811068541 +Ref: glossary term-lbyl11068774 +Ref: 407511068774 +Ref: glossary term-list11069441 +Ref: 407c11069441 +Ref: glossary term-list-comprehension11069623 +Ref: 407d11069623 +Ref: glossary term-loader11070041 +Ref: 116011070041 +Ref: glossary term-magic-method11070315 +Ref: 407e11070315 +Ref: glossary term-mapping11070387 +Ref: b3911070387 +Ref: glossary term-meta-path-finder11070754 +Ref: 9b111070754 +Ref: glossary term-metaclass11071051 +Ref: 201411071051 +Ref: glossary term-method11071791 +Ref: 406c11071791 +Ref: glossary term-method-resolution-order11072101 +Ref: 12ae11072101 +Ref: glossary term-module11072378 +Ref: 115011072378 +Ref: glossary term-module-spec11072641 +Ref: 336011072641 +Ref: glossary term-mro11072807 +Ref: 257811072807 +Ref: glossary term-mutable11072859 +Ref: ebe11072859 +Ref: glossary term-named-tuple11072982 +Ref: aa311072982 +Ref: glossary term-namespace11074149 +Ref: 407f11074149 +Ref: glossary term-namespace-package11074904 +Ref: 115811074904 +Ref: glossary term-nested-scope11075227 +Ref: 129511075227 +Ref: glossary term-new-style-class11075715 +Ref: 196e11075715 +Ref: glossary term-object11076041 +Ref: 408011076041 +Ref: glossary term-package11076205 +Ref: 115511076205 +Ref: glossary term-parameter11076478 +Ref: 11d211076478 +Ref: glossary positional-only-parameter11077021 +Ref: 2a711077021 +Ref: glossary keyword-only-parameter11077421 +Ref: 2ac11077421 +Ref: glossary term-path-entry11078932 +Ref: 117511078932 +Ref: glossary term-path-entry-finder11079086 +Ref: 9b211079086 +Ref: glossary term-path-entry-hook11079409 +Ref: 117711079409 +Ref: glossary term-path-based-finder11079601 +Ref: 116511079601 +Ref: glossary term-path-like-object11079736 +Ref: 36a11079736 +Ref: glossary term-pep11080369 +Ref: 408111080369 +Ref: glossary term-portion11081023 +Ref: 115a11081023 +Ref: glossary term-positional-argument11081184 +Ref: 5c511081184 +Ref: glossary term-provisional-api11081237 +Ref: 34811081237 +Ref: glossary term-provisional-package11082241 +Ref: 98011082241 +Ref: glossary term-python-300011082300 +Ref: 408211082300 +Ref: glossary term-pythonic11082493 +Ref: 408311082493 +Ref: glossary term-qualified-name11083133 +Ref: 10ca11083133 +Ref: glossary term-reference-count11083978 +Ref: 387411083978 +Ref: glossary term-regular-package11084420 +Ref: 115711084420 +Ref: glossary term-slots11084587 +Ref: 34cc11084587 +Ref: glossary term-sequence11084933 +Ref: ebc11084933 +Ref: glossary term-set-comprehension11085940 +Ref: 408411085940 +Ref: glossary term-single-dispatch11086257 +Ref: 1cb11086257 +Ref: glossary term-slice11086408 +Ref: 12aa11086408 +Ref: glossary term-special-method11086733 +Ref: 338b11086733 +Ref: glossary term-statement11087013 +Ref: 184711087013 +Ref: glossary term-text-encoding11087256 +Ref: 12a511087256 +Ref: glossary term-text-file11087325 +Ref: f3a11087325 +Ref: glossary term-triple-quoted-string11087847 +Ref: 408511087847 +Ref: glossary term-type11088347 +Ref: 408611088347 +Ref: glossary term-type-alias11088580 +Ref: 408711088580 +Ref: glossary term-type-hint11089245 +Ref: 406b11089245 +Ref: glossary term-universal-newlines11089831 +Ref: 5f411089831 +Ref: glossary term-variable-annotation11090205 +Ref: 61f11090205 +Ref: glossary term-virtual-environment11090830 +Ref: fa811090830 +Ref: glossary term-virtual-machine11091139 +Ref: 247d11091139 +Ref: glossary term-zen-of-python11091305 +Ref: 408811091305 +Ref: Glossary-Footnote-111091571 +Ref: Glossary-Footnote-211091620 +Ref: Glossary-Footnote-311091669 +Ref: Glossary-Footnote-411091718 +Ref: Glossary-Footnote-511091767 +Ref: Glossary-Footnote-611091816 +Ref: Glossary-Footnote-711091865 +Ref: Glossary-Footnote-811091914 +Ref: Glossary-Footnote-911091963 +Ref: Glossary-Footnote-1011092012 +Ref: Glossary-Footnote-1111092051 +Ref: Glossary-Footnote-1211092101 +Ref: Glossary-Footnote-1311092151 +Ref: Glossary-Footnote-1411092201 +Ref: Glossary-Footnote-1511092233 +Ref: Glossary-Footnote-1611092283 +Ref: Glossary-Footnote-1711092333 +Ref: Glossary-Footnote-1811092383 +Ref: Glossary-Footnote-1911092433 +Ref: Glossary-Footnote-2011092483 +Ref: Glossary-Footnote-2111092533 +Ref: Glossary-Footnote-2211092583 +Ref: Glossary-Footnote-2311092633 +Ref: Glossary-Footnote-2411092692 +Ref: Glossary-Footnote-2511092742 +Ref: Glossary-Footnote-2611092792 +Ref: Glossary-Footnote-2711092842 +Ref: Glossary-Footnote-2811092892 +Ref: Glossary-Footnote-2911092942 +Ref: Glossary-Footnote-3011092992 +Ref: Glossary-Footnote-3111093042 +Ref: Glossary-Footnote-3211093092 +Ref: Glossary-Footnote-3311093142 +Ref: Glossary-Footnote-3411093192 +Ref: Glossary-Footnote-3511093242 +Ref: Glossary-Footnote-3611093292 +Node: About these documents11093342 +Ref: about doc11093446 +Ref: 408911093446 +Ref: about about-these-documents11093446 +Ref: 408a11093446 +Ref: About these documents-Footnote-111094351 +Ref: About these documents-Footnote-211094400 +Ref: About these documents-Footnote-311094431 +Ref: About these documents-Footnote-411094472 +Node: Contributors to the Python Documentation11094513 +Ref: about contributors-to-the-python-documentation11094611 +Ref: 408b11094611 +Ref: Contributors to the Python Documentation-Footnote-111095070 +Node: Dealing with Bugs11095131 +Ref: bugs doc11095236 +Ref: 408c11095236 +Ref: bugs dealing-with-bugs11095236 +Ref: 408d11095236 +Ref: bugs reporting-bugs11095236 +Ref: 38d711095236 +Node: Documentation bugs11095779 +Ref: bugs documentation-bugs11095890 +Ref: 408f11095890 +Ref: Documentation bugs-Footnote-111096837 +Ref: Documentation bugs-Footnote-211096982 +Ref: Documentation bugs-Footnote-311097027 +Node: Using the Python issue tracker11097103 +Ref: bugs using-the-python-issue-tracker11097270 +Ref: 409111097270 +Ref: bugs using-the-tracker11097270 +Ref: 409011097270 +Ref: Using the Python issue tracker-Footnote-111099676 +Ref: Using the Python issue tracker-Footnote-211099739 +Node: Getting started contributing to Python yourself11099823 +Ref: bugs contributing-to-python11099963 +Ref: 408e11099963 +Ref: bugs getting-started-contributing-to-python-yourself11099963 +Ref: 409211099963 +Ref: Getting started contributing to Python yourself-Footnote-111100482 +Ref: Getting started contributing to Python yourself-Footnote-211100519 +Node: Copyright11100595 +Ref: copyright doc11100698 +Ref: 409311100698 +Ref: copyright copyright11100698 +Ref: 409411100698 +Ref: copyright core-mentorship-mailing-list11100698 +Ref: 409511100698 +Node: History and License11101213 +Ref: license doc11101341 +Ref: 409711101341 +Ref: license history-and-license11101341 +Ref: 409611101341 +Ref: license id111101341 +Ref: 409811101341 +Node: History of the software11101551 +Ref: license history-of-the-software11101699 +Ref: 409911101699 +Node: Terms and conditions for accessing or otherwise using Python11106236 +Ref: license terms-and-conditions-for-accessing-or-otherwise-using-python11106448 +Ref: 409a11106448 +Node: PSF LICENSE AGREEMENT FOR PYTHON 3 8 911107613 +Ref: license psf-license11107800 +Ref: 409b11107800 +Ref: license psf-license-agreement-for-python-release11107800 +Ref: 409e11107800 +Node: BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 011110469 +Ref: license beopen-com-license-agreement-for-python-2-011110704 +Ref: 409f11110704 +Node: CNRI LICENSE AGREEMENT FOR PYTHON 1 6 111113412 +Ref: license cnri-license-agreement-for-python-1-6-111113659 +Ref: 40a011113659 +Node: CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 211117837 +Ref: license cwi-license-agreement-for-python-0-9-0-through-1-211118107 +Ref: 40a111118107 +Node: ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON 3 8 9 DOCUMENTATION11119377 +Ref: license bsd011119599 +Ref: 409c11119599 +Ref: license zero-clause-bsd-license-for-code-in-the-python-release-documentation11119599 +Ref: 40a211119599 +Node: Licenses and Acknowledgements for Incorporated Software11120401 +Ref: license licenses-and-acknowledgements-for-incorporated-software11120581 +Ref: 40a311120581 +Ref: license otherlicenses11120581 +Ref: 409d11120581 +Node: Mersenne Twister11121216 +Ref: license mersenne-twister11121343 +Ref: 40a411121343 +Node: Sockets<2>11123617 +Ref: license sockets11123781 +Ref: 40a511123781 +Node: Asynchronous socket services11125615 +Ref: license asynchronous-socket-services11125780 +Ref: 40a611125780 +Node: Cookie management11127000 +Ref: license cookie-management11127172 +Ref: 40a711127172 +Node: Execution tracing11128389 +Ref: license execution-tracing11128564 +Ref: 40a811128564 +Node: UUencode and UUdecode functions11129824 +Ref: license uuencode-and-uudecode-functions11130008 +Ref: 40a911130008 +Node: XML Remote Procedure Calls11131590 +Ref: license xml-remote-procedure-calls11131767 +Ref: 40aa11131767 +Node: test_epoll11133292 +Ref: license test-epoll11133451 +Ref: 40ab11133451 +Node: Select kqueue11134711 +Ref: license select-kqueue11134853 +Ref: 40ac11134853 +Node: SipHash2411136407 +Ref: license siphash2411136554 +Ref: 40ad11136554 +Node: strtod and dtoa11137737 +Ref: license strtod-and-dtoa11137878 +Ref: 40ae11137878 +Node: OpenSSL11139227 +Ref: license openssl11139364 +Ref: 40af11139364 +Node: expat11146911 +Ref: license expat11147039 +Ref: 40b011147039 +Node: libffi11148439 +Ref: license libffi11148567 +Ref: 40b111148567 +Node: zlib<3>11149910 +Ref: license zlib11150040 +Ref: 40b211150040 +Node: cfuhash11151273 +Ref: license cfuhash11151405 +Ref: 40b311151405 +Node: libmpdec11153222 +Ref: license libmpdec11153366 +Ref: 40b411153366 +Node: W3C C14N test suite11154930 +Ref: license w3c-c14n-test-suite11155058 +Ref: 40b511155058 +Node: Distributing Python Modules Legacy version11157006 +Ref: distutils/index doc11157165 +Ref: 40b611157165 +Ref: distutils/index distributing-python-modules-legacy-version11157165 +Ref: 40b711157165 +Ref: distutils/index distutils-index11157165 +Ref: 7f811157165 +Ref: Distributing Python Modules Legacy version-Footnote-111158649 +Node: An Introduction to Distutils11158715 +Ref: distutils/introduction doc11158855 +Ref: 40b811158855 +Ref: distutils/introduction an-introduction-to-distutils11158855 +Ref: 40b911158855 +Ref: distutils/introduction distutils-intro11158855 +Ref: 40ba11158855 +Node: Concepts & Terminology11159610 +Ref: distutils/introduction concepts-terminology11159725 +Ref: 40bb11159725 +Ref: distutils/introduction distutils-concepts11159725 +Ref: 40bc11159725 +Node: A Simple Example<2>11160929 +Ref: distutils/introduction a-simple-example11161079 +Ref: 40bd11161079 +Ref: distutils/introduction distutils-simple-example11161079 +Ref: 40be11161079 +Node: General Python terminology11164930 +Ref: distutils/introduction general-python-terminology11165088 +Ref: 40c011165088 +Ref: distutils/introduction python-terms11165088 +Ref: 40c111165088 +Node: Distutils-specific terminology11167083 +Ref: distutils/introduction distutils-specific-terminology11167213 +Ref: 40c211167213 +Ref: distutils/introduction distutils-term11167213 +Ref: 40c311167213 +Node: Writing the Setup Script11168370 +Ref: distutils/setupscript doc11168555 +Ref: 40c411168555 +Ref: distutils/setupscript setup-script11168555 +Ref: 40bf11168555 +Ref: distutils/setupscript writing-the-setup-script11168555 +Ref: 40c511168555 +Node: Listing whole packages11171950 +Ref: distutils/setupscript listing-packages11172068 +Ref: 40c711172068 +Ref: distutils/setupscript listing-whole-packages11172068 +Ref: 40c811172068 +Node: Listing individual modules11174528 +Ref: distutils/setupscript listing-individual-modules11174683 +Ref: 40c911174683 +Ref: distutils/setupscript listing-modules11174683 +Ref: 40ca11174683 +Node: Describing extension modules11175492 +Ref: distutils/setupscript describing-extension-modules11175673 +Ref: 40cb11175673 +Ref: distutils/setupscript describing-extensions11175673 +Ref: 40cc11175673 +Node: Extension names and packages11177412 +Ref: distutils/setupscript extension-names-and-packages11177536 +Ref: 40ce11177536 +Node: Extension source files11178635 +Ref: distutils/setupscript extension-source-files11178788 +Ref: 40cf11178788 +Node: Preprocessor options11180184 +Ref: distutils/setupscript preprocessor-options11180324 +Ref: 40d011180324 +Node: Library options11183342 +Ref: distutils/setupscript library-options11183473 +Ref: 40d111183473 +Node: Other options11184451 +Ref: distutils/setupscript other-options11184553 +Ref: 40d211184553 +Node: Relationships between Distributions and Packages11185708 +Ref: distutils/setupscript relationships-between-distributions-and-packages11185881 +Ref: 40d311185881 +Node: Installing Scripts11189259 +Ref: distutils/setupscript distutils-installing-scripts11189427 +Ref: 40d411189427 +Ref: distutils/setupscript installing-scripts11189427 +Ref: 40d511189427 +Node: Installing Package Data11190534 +Ref: distutils/setupscript distutils-installing-package-data11190681 +Ref: 40d611190681 +Ref: distutils/setupscript installing-package-data11190681 +Ref: 40d711190681 +Node: Installing Additional Files11192388 +Ref: distutils/setupscript distutils-additional-files11192537 +Ref: 40d811192537 +Ref: distutils/setupscript installing-additional-files11192537 +Ref: 40d911192537 +Node: Additional meta-data11194426 +Ref: distutils/setupscript additional-meta-data11194578 +Ref: 40da11194578 +Ref: distutils/setupscript meta-data11194578 +Ref: 40c611194578 +Ref: Additional meta-data-Footnote-111201903 +Node: Debugging the setup script11201940 +Ref: distutils/setupscript debug-setup-script11202056 +Ref: 40db11202056 +Ref: distutils/setupscript debugging-the-setup-script11202056 +Ref: 40dc11202056 +Node: Writing the Setup Configuration File11203131 +Ref: distutils/configfile doc11203318 +Ref: 40dd11203318 +Ref: distutils/configfile setup-config11203318 +Ref: 40de11203318 +Ref: distutils/configfile writing-the-setup-configuration-file11203318 +Ref: 40df11203318 +Ref: distutils/configfile distutils-build-ext-inplace11206527 +Ref: 40e011206527 +Ref: Writing the Setup Configuration File-Footnote-111208986 +Node: Creating a Source Distribution11209100 +Ref: distutils/sourcedist doc11209291 +Ref: 40e211209291 +Ref: distutils/sourcedist creating-a-source-distribution11209291 +Ref: 40e311209291 +Ref: distutils/sourcedist source-dist11209291 +Ref: 40e411209291 +Node: Specifying the files to distribute11212131 +Ref: distutils/sourcedist manifest11212265 +Ref: 38bb11212265 +Ref: distutils/sourcedist specifying-the-files-to-distribute11212265 +Ref: 40e511212265 +Node: Manifest-related options11218240 +Ref: distutils/sourcedist manifest-options11218374 +Ref: 40e711218374 +Ref: distutils/sourcedist manifest-related-options11218374 +Ref: 40e811218374 +Node: Creating Built Distributions11219477 +Ref: distutils/builtdist doc11219650 +Ref: 40e911219650 +Ref: distutils/builtdist built-dist11219650 +Ref: 40ea11219650 +Ref: distutils/builtdist creating-built-distributions11219650 +Ref: 40eb11219650 +Node: Creating RPM packages11227258 +Ref: distutils/builtdist creating-rpm-packages11227380 +Ref: 40ec11227380 +Ref: distutils/builtdist creating-rpms11227380 +Ref: 40ed11227380 +Node: Creating Windows Installers11234970 +Ref: distutils/builtdist creating-windows-installers11235127 +Ref: 40ee11235127 +Ref: distutils/builtdist creating-wininst11235127 +Ref: 40ef11235127 +Node: Cross-compiling on Windows11237359 +Ref: distutils/builtdist cross-compile-windows11237524 +Ref: 40f011237524 +Ref: distutils/builtdist cross-compiling-on-windows11237524 +Ref: 40f111237524 +Node: The Postinstallation script11239210 +Ref: distutils/builtdist postinstallation-script11239300 +Ref: 40f211239300 +Ref: distutils/builtdist the-postinstallation-script11239300 +Ref: 40f311239300 +Ref: distutils/builtdist directory_created11240205 +Ref: 40f411240205 +Ref: distutils/builtdist file_created11240244 +Ref: 40f511240244 +Ref: distutils/builtdist get_special_folder_path11240595 +Ref: 40f611240595 +Ref: distutils/builtdist create_shortcut11241485 +Ref: 40f711241485 +Node: Vista User Access Control UAC11242202 +Ref: distutils/builtdist vista-user-access-control-uac11242331 +Ref: 40f811242331 +Node: Distutils Examples11242784 +Ref: distutils/examples doc11242946 +Ref: 40f911242946 +Ref: distutils/examples distutils-examples11242946 +Ref: 40fa11242946 +Ref: distutils/examples id111242946 +Ref: 40fb11242946 +Ref: Distutils Examples-Footnote-111243829 +Node: Pure Python distribution by module11243885 +Ref: distutils/examples pure-mod11244018 +Ref: 40fc11244018 +Ref: distutils/examples pure-python-distribution-by-module11244018 +Ref: 40fd11244018 +Node: Pure Python distribution by package11245802 +Ref: distutils/examples pure-pkg11245967 +Ref: 40fe11245967 +Ref: distutils/examples pure-python-distribution-by-package11245967 +Ref: 40ff11245967 +Node: Single extension module11249736 +Ref: distutils/examples single-ext11249885 +Ref: 410011249885 +Ref: distutils/examples single-extension-module11249885 +Ref: 410111249885 +Node: Checking a package11251000 +Ref: distutils/examples checking-a-package11251134 +Ref: 410211251134 +Ref: Checking a package-Footnote-111252781 +Node: Reading the metadata11252821 +Ref: distutils/examples reading-the-metadata11252923 +Ref: 410311252923 +Node: Extending Distutils11254562 +Ref: distutils/extending doc11254713 +Ref: 410411254713 +Ref: distutils/extending docutils11254713 +Ref: 410511254713 +Ref: distutils/extending extending-distutils11254713 +Ref: 103611254713 +Ref: distutils/extending id111254713 +Ref: 410611254713 +Node: Integrating new commands11255996 +Ref: distutils/extending integrating-new-commands11256114 +Ref: 410811256114 +Node: Adding new distribution types11259014 +Ref: distutils/extending adding-new-distribution-types11259132 +Ref: 410911259132 +Node: Command Reference11259586 +Ref: distutils/commandref doc11259732 +Ref: 410a11259732 +Ref: distutils/commandref command-reference11259732 +Ref: 410b11259732 +Ref: distutils/commandref reference11259732 +Ref: 410c11259732 +Node: Installing modules the install command family11260248 +Ref: distutils/commandref install-cmd11260404 +Ref: 410d11260404 +Ref: distutils/commandref installing-modules-the-install-command-family11260404 +Ref: 410e11260404 +Node: install_data11260730 +Ref: distutils/commandref install-data11260848 +Ref: 410f11260848 +Ref: distutils/commandref install-data-cmd11260848 +Ref: 411011260848 +Node: install_scripts11260971 +Ref: distutils/commandref install-scripts11261089 +Ref: 411111261089 +Ref: distutils/commandref install-scripts-cmd11261089 +Ref: 411211261089 +Node: Creating a source distribution the sdist command11261213 +Ref: distutils/commandref creating-a-source-distribution-the-sdist-command11261369 +Ref: 411311261369 +Ref: distutils/commandref sdist-cmd11261369 +Ref: 40e611261369 +Node: API Reference11264141 +Ref: distutils/apiref doc11264259 +Ref: 411411264259 +Ref: distutils/apiref api-reference11264259 +Ref: 411511264259 +Ref: distutils/apiref id111264259 +Ref: 411611264259 +Ref: API Reference-Footnote-111270182 +Node: distutils core — Core Distutils functionality11270282 +Ref: distutils/apiref distutils-core-core-distutils-functionality11270432 +Ref: 411711270432 +Ref: distutils/apiref module-distutils core11270432 +Ref: 5411270432 +Ref: distutils/apiref distutils core setup11270839 +Ref: 38b811270839 +Ref: distutils/apiref distutils core run_setup11277750 +Ref: 411a11277750 +Ref: distutils/apiref distutils core Extension11280170 +Ref: 40cd11280170 +Ref: distutils/apiref distutils core Distribution11287594 +Ref: 411811287594 +Ref: distutils/apiref distutils core Command11288111 +Ref: 411911288111 +Ref: distutils core — Core Distutils functionality-Footnote-111288310 +Ref: distutils core — Core Distutils functionality-Footnote-211288347 +Node: distutils ccompiler — CCompiler base class11288396 +Ref: distutils/apiref distutils-ccompiler-ccompiler-base-class11288598 +Ref: 411b11288598 +Ref: distutils/apiref module-distutils ccompiler11288598 +Ref: 3c11288598 +Ref: distutils/apiref distutils ccompiler gen_lib_options11289091 +Ref: 411d11289091 +Ref: distutils/apiref distutils ccompiler gen_preprocess_options11289577 +Ref: 411e11289577 +Ref: distutils/apiref distutils ccompiler get_default_compiler11290243 +Ref: 411f11290243 +Ref: distutils/apiref distutils ccompiler new_compiler11290712 +Ref: 412011290712 +Ref: distutils/apiref distutils ccompiler show_compilers11291512 +Ref: 412111291512 +Ref: distutils/apiref distutils ccompiler CCompiler11291705 +Ref: 411c11291705 +Ref: distutils/apiref distutils ccompiler CCompiler add_include_dir11293142 +Ref: 412211293142 +Ref: distutils/apiref distutils ccompiler CCompiler set_include_dirs11293444 +Ref: 412311293444 +Ref: distutils/apiref distutils ccompiler CCompiler add_library11293917 +Ref: 412411293917 +Ref: distutils/apiref distutils ccompiler CCompiler set_libraries11294704 +Ref: 412511294704 +Ref: distutils/apiref distutils ccompiler CCompiler add_library_dir11294999 +Ref: 412611294999 +Ref: distutils/apiref distutils ccompiler CCompiler set_library_dirs11295405 +Ref: 412711295405 +Ref: distutils/apiref distutils ccompiler CCompiler add_runtime_library_dir11295646 +Ref: 412811295646 +Ref: distutils/apiref distutils ccompiler CCompiler set_runtime_library_dirs11295807 +Ref: 412911295807 +Ref: distutils/apiref distutils ccompiler CCompiler define_macro11296093 +Ref: 412a11296093 +Ref: distutils/apiref distutils ccompiler CCompiler undefine_macro11296461 +Ref: 412b11296461 +Ref: distutils/apiref distutils ccompiler CCompiler add_link_object11296984 +Ref: 412c11296984 +Ref: distutils/apiref distutils ccompiler CCompiler set_link_objects11297267 +Ref: 412d11297267 +Ref: distutils/apiref distutils ccompiler CCompiler detect_language11297710 +Ref: 412e11297710 +Ref: distutils/apiref distutils ccompiler CCompiler find_library_file11297954 +Ref: 412f11297954 +Ref: distutils/apiref distutils ccompiler CCompiler has_function11298363 +Ref: 413011298363 +Ref: distutils/apiref distutils ccompiler CCompiler library_dir_option11298765 +Ref: 413111298765 +Ref: distutils/apiref distutils ccompiler CCompiler library_option11298920 +Ref: 413211298920 +Ref: distutils/apiref distutils ccompiler CCompiler runtime_library_dir_option11299091 +Ref: 413311299091 +Ref: distutils/apiref distutils ccompiler CCompiler set_executables11299262 +Ref: 413411299262 +Ref: distutils/apiref distutils ccompiler CCompiler compile11300759 +Ref: 413611300759 +Ref: distutils/apiref distutils ccompiler CCompiler create_static_lib11303541 +Ref: 413711303541 +Ref: distutils/apiref distutils ccompiler CCompiler link11304825 +Ref: 413811304825 +Ref: distutils/apiref distutils ccompiler CCompiler link_executable11307485 +Ref: 413911307485 +Ref: distutils/apiref distutils ccompiler CCompiler link_shared_lib11307975 +Ref: 413a11307975 +Ref: distutils/apiref distutils ccompiler CCompiler link_shared_object11308517 +Ref: 413b11308517 +Ref: distutils/apiref distutils ccompiler CCompiler preprocess11309083 +Ref: 413c11309083 +Ref: distutils/apiref distutils ccompiler CCompiler executable_filename11309970 +Ref: 413d11309970 +Ref: distutils/apiref distutils ccompiler CCompiler library_filename11310274 +Ref: 413e11310274 +Ref: distutils/apiref distutils ccompiler CCompiler object_filenames11310679 +Ref: 413f11310679 +Ref: distutils/apiref distutils ccompiler CCompiler shared_object_filename11310915 +Ref: 414011310915 +Ref: distutils/apiref distutils ccompiler CCompiler execute11311110 +Ref: 414111311110 +Ref: distutils/apiref distutils ccompiler CCompiler spawn11311394 +Ref: 414311311394 +Ref: distutils/apiref distutils ccompiler CCompiler mkpath11311543 +Ref: 414411311543 +Ref: distutils/apiref distutils ccompiler CCompiler move_file11311721 +Ref: 414611311721 +Ref: distutils/apiref distutils ccompiler CCompiler announce11311859 +Ref: 414811311859 +Ref: distutils/apiref distutils ccompiler CCompiler warn11311965 +Ref: 414911311965 +Ref: distutils/apiref distutils ccompiler CCompiler debug_print11312054 +Ref: 414a11312054 +Node: distutils unixccompiler — Unix C Compiler11312243 +Ref: distutils/apiref distutils-unixccompiler-unix-c-compiler11312443 +Ref: 414b11312443 +Ref: distutils/apiref module-distutils unixccompiler11312443 +Ref: 6411312443 +Node: distutils msvccompiler — Microsoft Compiler11313210 +Ref: distutils/apiref distutils-msvccompiler-microsoft-compiler11313409 +Ref: 414c11313409 +Ref: distutils/apiref module-distutils msvccompiler11313409 +Ref: 6011313409 +Node: distutils bcppcompiler — Borland Compiler11314397 +Ref: distutils/apiref distutils-bcppcompiler-borland-compiler11314597 +Ref: 414d11314597 +Ref: distutils/apiref module-distutils bcppcompiler11314597 +Ref: 3b11314597 +Node: distutils cygwincompiler — Cygwin Compiler11314831 +Ref: distutils/apiref distutils-cygwincompiler-cygwin-compiler11315032 +Ref: 414e11315032 +Ref: distutils/apiref module-distutils cygwinccompiler11315032 +Ref: 5511315032 +Node: distutils archive_util — Archiving utilities11315414 +Ref: distutils/apiref distutils-archive-util-archiving-utilities11315614 +Ref: 414f11315614 +Ref: distutils/apiref module-distutils archive_util11315614 +Ref: 3a11315614 +Ref: distutils/apiref distutils archive_util make_archive11315825 +Ref: 415011315825 +Ref: distutils/apiref distutils archive_util make_tarball11316737 +Ref: 415111316737 +Ref: distutils/apiref distutils archive_util make_zipfile11317525 +Ref: 415211317525 +Node: distutils dep_util — Dependency checking11318039 +Ref: distutils/apiref distutils-dep-util-dependency-checking11318243 +Ref: 415311318243 +Ref: distutils/apiref module-distutils dep_util11318243 +Ref: 5711318243 +Ref: distutils/apiref distutils dep_util newer11318532 +Ref: 415411318532 +Ref: distutils/apiref distutils dep_util newer_pairwise11318872 +Ref: 415511318872 +Ref: distutils/apiref distutils dep_util newer_group11319194 +Ref: 415611319194 +Node: distutils dir_util — Directory tree operations11320095 +Ref: distutils/apiref distutils-dir-util-directory-tree-operations11320299 +Ref: 415711320299 +Ref: distutils/apiref module-distutils dir_util11320299 +Ref: 5811320299 +Ref: distutils/apiref distutils dir_util mkpath11320505 +Ref: 414511320505 +Ref: distutils/apiref distutils dir_util create_tree11321114 +Ref: 415811321114 +Ref: distutils/apiref distutils dir_util copy_tree11321675 +Ref: 415911321675 +Ref: distutils/apiref distutils dir_util remove_tree11323154 +Ref: 415b11323154 +Ref: distutils dir_util — Directory tree operations-Footnote-111323468 +Node: distutils file_util — Single file operations11323514 +Ref: distutils/apiref distutils-file-util-single-file-operations11323732 +Ref: 415c11323732 +Ref: distutils/apiref module-distutils file_util11323732 +Ref: 5d11323732 +Ref: distutils/apiref distutils file_util copy_file11323929 +Ref: 415a11323929 +Ref: distutils/apiref distutils file_util move_file11325349 +Ref: 414711325349 +Ref: distutils/apiref distutils file_util write_file11325760 +Ref: 415d11325760 +Node: distutils util — Miscellaneous other utility functions11325947 +Ref: distutils/apiref distutils-util-miscellaneous-other-utility-functions11326158 +Ref: 415e11326158 +Ref: distutils/apiref module-distutils util11326158 +Ref: 6511326158 +Ref: distutils/apiref distutils util get_platform11326396 +Ref: 415f11326396 +Ref: distutils/apiref distutils util convert_path11328104 +Ref: 416011328104 +Ref: distutils/apiref distutils util change_root11328649 +Ref: 416111328649 +Ref: distutils/apiref distutils util check_environ11328974 +Ref: 416211328974 +Ref: distutils/apiref distutils util subst_vars11329393 +Ref: 416311329393 +Ref: distutils/apiref distutils util split_quoted11330209 +Ref: 413511330209 +Ref: distutils/apiref distutils util execute11330784 +Ref: 414211330784 +Ref: distutils/apiref distutils util strtobool11331304 +Ref: 416411331304 +Ref: distutils/apiref distutils util byte_compile11331657 +Ref: 416511331657 +Ref: distutils/apiref distutils util rfc822_escape11333685 +Ref: 416611333685 +Ref: distutils util — Miscellaneous other utility functions-Footnote-111333978 +Ref: distutils util — Miscellaneous other utility functions-Footnote-211334027 +Ref: distutils util — Miscellaneous other utility functions-Footnote-311334076 +Ref: distutils util — Miscellaneous other utility functions-Footnote-411334125 +Node: distutils dist — The Distribution class11334173 +Ref: distutils/apiref distutils-dist-the-distribution-class11334381 +Ref: 416711334381 +Ref: distutils/apiref module-distutils dist11334381 +Ref: 5911334381 +Node: distutils extension — The Extension class11334623 +Ref: distutils/apiref distutils-extension-the-extension-class11334815 +Ref: 416811334815 +Ref: distutils/apiref module-distutils extension11334815 +Ref: 5b11334815 +Node: distutils debug — Distutils debug mode11335034 +Ref: distutils/apiref distutils-debug-distutils-debug-mode11335226 +Ref: 416911335226 +Ref: distutils/apiref module-distutils debug11335226 +Ref: 5611335226 +Node: distutils errors — Distutils exceptions11335369 +Ref: distutils/apiref distutils-errors-distutils-exceptions11335586 +Ref: 416a11335586 +Ref: distutils/apiref module-distutils errors11335586 +Ref: 5a11335586 +Node: distutils fancy_getopt — Wrapper around the standard getopt module11336086 +Ref: distutils/apiref distutils-fancy-getopt-wrapper-around-the-standard-getopt-module11336304 +Ref: 416b11336304 +Ref: distutils/apiref module-distutils fancy_getopt11336304 +Ref: 5c11336304 +Ref: distutils/apiref distutils fancy_getopt fancy_getopt11337010 +Ref: 416c11337010 +Ref: distutils/apiref distutils fancy_getopt wrap_text11337668 +Ref: 416e11337668 +Ref: distutils/apiref distutils fancy_getopt FancyGetopt11337776 +Ref: 416d11337776 +Ref: distutils/apiref distutils fancy_getopt FancyGetopt getopt11338341 +Ref: 416f11338341 +Ref: distutils/apiref distutils fancy_getopt FancyGetopt get_option_order11338943 +Ref: 417011338943 +Ref: distutils/apiref distutils fancy_getopt FancyGetopt generate_help11339187 +Ref: 417111339187 +Node: distutils filelist — The FileList class11339467 +Ref: distutils/apiref distutils-filelist-the-filelist-class11339690 +Ref: 417211339690 +Ref: distutils/apiref module-distutils filelist11339690 +Ref: 5e11339690 +Node: distutils log — Simple PEP 282-style logging11339911 +Ref: distutils/apiref distutils-log-simple-pep-282-style-logging11340105 +Ref: 417311340105 +Ref: distutils/apiref module-distutils log11340105 +Ref: 5f11340105 +Node: distutils spawn — Spawn a sub-process11340226 +Ref: distutils/apiref distutils-spawn-spawn-a-sub-process11340435 +Ref: 417411340435 +Ref: distutils/apiref module-distutils spawn11340435 +Ref: 6111340435 +Node: distutils sysconfig — System configuration information11340774 +Ref: distutils/apiref distutils-sysconfig-system-configuration-information11340979 +Ref: 417511340979 +Ref: distutils/apiref module-distutils sysconfig11340979 +Ref: 6211340979 +Ref: distutils/apiref distutils sysconfig PREFIX11341822 +Ref: 417611341822 +Ref: distutils/apiref distutils sysconfig EXEC_PREFIX11341916 +Ref: 417711341916 +Ref: distutils/apiref distutils sysconfig get_config_var11342020 +Ref: 417811342020 +Ref: distutils/apiref distutils sysconfig get_config_vars11342185 +Ref: 417911342185 +Ref: distutils/apiref distutils sysconfig get_config_h_filename11342639 +Ref: 417a11342639 +Ref: distutils/apiref distutils sysconfig get_makefile_filename11342999 +Ref: 417b11342999 +Ref: distutils/apiref distutils sysconfig get_python_inc11343377 +Ref: 417c11343377 +Ref: distutils/apiref distutils sysconfig get_python_lib11343908 +Ref: 417d11343908 +Ref: distutils/apiref distutils sysconfig customize_compiler11344713 +Ref: 417e11344713 +Ref: distutils/apiref distutils sysconfig set_python_build11345383 +Ref: 417f11345383 +Node: distutils text_file — The TextFile class11345706 +Ref: distutils/apiref distutils-text-file-the-textfile-class11345916 +Ref: 418011345916 +Ref: distutils/apiref module-distutils text_file11345916 +Ref: 6311345916 +Ref: distutils/apiref distutils text_file TextFile11346229 +Ref: 418111346229 +Ref: distutils/apiref distutils text_file TextFile open11350747 +Ref: 418511350747 +Ref: distutils/apiref distutils text_file TextFile close11350894 +Ref: 418411350894 +Ref: distutils/apiref distutils text_file TextFile warn11351058 +Ref: 418211351058 +Ref: distutils/apiref distutils text_file TextFile readline11351568 +Ref: 418611351568 +Ref: distutils/apiref distutils text_file TextFile readlines11352250 +Ref: 418711352250 +Ref: distutils/apiref distutils text_file TextFile unreadline11352457 +Ref: 418311352457 +Node: distutils version — Version number classes11353086 +Ref: distutils/apiref distutils-version-version-number-classes11353300 +Ref: 418811353300 +Ref: distutils/apiref module-distutils version11353300 +Ref: 6611353300 +Node: distutils cmd — Abstract base class for Distutils commands11353413 +Ref: distutils/apiref distutils-cmd-abstract-base-class-for-distutils-commands11353617 +Ref: 418911353617 +Ref: distutils/apiref module-distutils cmd11353617 +Ref: 3d11353617 +Ref: distutils/apiref distutils cmd Command11353829 +Ref: 410711353829 +Node: Creating a new Distutils command11354907 +Ref: distutils/apiref creating-a-new-distutils-command11355118 +Ref: 418d11355118 +Ref: distutils/apiref distutils cmd Command initialize_options11355987 +Ref: 418a11355987 +Ref: distutils/apiref distutils cmd Command finalize_options11356435 +Ref: 418b11356435 +Ref: distutils/apiref distutils cmd Command run11356931 +Ref: 418c11356931 +Ref: distutils/apiref distutils cmd Command sub_commands11357357 +Ref: 418e11357357 +Node: distutils command — Individual Distutils commands11358348 +Ref: distutils/apiref distutils-command-individual-distutils-commands11358551 +Ref: 418f11358551 +Ref: distutils/apiref module-distutils command11358551 +Ref: 3e11358551 +Node: distutils command bdist — Build a binary installer11358678 +Ref: distutils/apiref distutils-command-bdist-build-a-binary-installer11358919 +Ref: 419011358919 +Ref: distutils/apiref module-distutils command bdist11358919 +Ref: 3f11358919 +Node: distutils command bdist_packager — Abstract base class for packagers11359048 +Ref: distutils/apiref distutils-command-bdist-packager-abstract-base-class-for-packagers11359299 +Ref: 419111359299 +Ref: distutils/apiref module-distutils command bdist_packager11359299 +Ref: 4211359299 +Node: distutils command bdist_dumb — Build a “dumb” installer11359464 +Ref: distutils/apiref distutils-command-bdist-dumb-build-a-dumb-installer11359737 +Ref: 419211359737 +Ref: distutils/apiref module-distutils command bdist_dumb11359737 +Ref: 4011359737 +Node: distutils command bdist_msi — Build a Microsoft Installer binary package11359880 +Ref: distutils/apiref distutils-command-bdist-msi-build-a-microsoft-installer-binary-package11360167 +Ref: 419311360167 +Ref: distutils/apiref module-distutils command bdist_msi11360167 +Ref: 4111360167 +Ref: distutils/apiref distutils command bdist_msi bdist_msi11360340 +Ref: 419411360340 +Ref: distutils command bdist_msi — Build a Microsoft Installer binary package-Footnote-111360783 +Node: distutils command bdist_rpm — Build a binary distribution as a Redhat RPM and SRPM11360853 +Ref: distutils/apiref distutils-command-bdist-rpm-build-a-binary-distribution-as-a-redhat-rpm-and-srpm11361140 +Ref: 419511361140 +Ref: distutils/apiref module-distutils command bdist_rpm11361140 +Ref: 4311361140 +Node: distutils command bdist_wininst — Build a Windows installer11361333 +Ref: distutils/apiref distutils-command-bdist-wininst-build-a-windows-installer11361601 +Ref: 419611361601 +Ref: distutils/apiref module-distutils command bdist_wininst11361601 +Ref: 4411361601 +Node: distutils command sdist — Build a source distribution11361821 +Ref: distutils/apiref distutils-command-sdist-build-a-source-distribution11362061 +Ref: 419711362061 +Ref: distutils/apiref module-distutils command sdist11362061 +Ref: 5311362061 +Node: distutils command build — Build all files of a package11362196 +Ref: distutils/apiref distutils-command-build-build-all-files-of-a-package11362442 +Ref: 419811362442 +Ref: distutils/apiref module-distutils command build11362442 +Ref: 4511362442 +Node: distutils command build_clib — Build any C libraries in a package11362579 +Ref: distutils/apiref distutils-command-build-clib-build-any-c-libraries-in-a-package11362835 +Ref: 419911362835 +Ref: distutils/apiref module-distutils command build_clib11362835 +Ref: 4611362835 +Node: distutils command build_ext — Build any extensions in a package11362994 +Ref: distutils/apiref distutils-command-build-ext-build-any-extensions-in-a-package11363261 +Ref: 419a11363261 +Ref: distutils/apiref module-distutils command build_ext11363261 +Ref: 4711363261 +Node: distutils command build_py — Build the py/ pyc files of a package11363416 +Ref: distutils/apiref distutils-command-build-py-build-the-py-pyc-files-of-a-package11363682 +Ref: 419b11363682 +Ref: distutils/apiref module-distutils command build_py11363682 +Ref: 4811363682 +Ref: distutils/apiref distutils command build_py build_py11363843 +Ref: 419c11363843 +Ref: distutils/apiref distutils command build_py build_py_2to311363891 +Ref: 419d11363891 +Node: distutils command build_scripts — Build the scripts of a package11364508 +Ref: distutils/apiref distutils-command-build-scripts-build-the-scripts-of-a-package11364763 +Ref: 419e11364763 +Ref: distutils/apiref module-distutils command build_scripts11364763 +Ref: 4911364763 +Node: distutils command clean — Clean a package build area11364920 +Ref: distutils/apiref distutils-command-clean-clean-a-package-build-area11365166 +Ref: 419f11365166 +Ref: distutils/apiref module-distutils command clean11365166 +Ref: 4b11365166 +Node: distutils command config — Perform package configuration11365610 +Ref: distutils/apiref distutils-command-config-perform-package-configuration11365837 +Ref: 41a011365837 +Ref: distutils/apiref module-distutils command config11365837 +Ref: 4c11365837 +Node: distutils command install — Install a package11365978 +Ref: distutils/apiref distutils-command-install-install-a-package11366219 +Ref: 41a111366219 +Ref: distutils/apiref module-distutils command install11366219 +Ref: 4d11366219 +Node: distutils command install_data — Install data files from a package11366338 +Ref: distutils/apiref distutils-command-install-data-install-data-files-from-a-package11366600 +Ref: 41a211366600 +Ref: distutils/apiref module-distutils command install_data11366600 +Ref: 4e11366600 +Node: distutils command install_headers — Install C/C++ header files from a package11366761 +Ref: distutils/apiref distutils-command-install-headers-install-c-c-header-files-from-a-package11367046 +Ref: 41a311367046 +Ref: distutils/apiref module-distutils command install_headers11367046 +Ref: 4f11367046 +Node: distutils command install_lib — Install library files from a package11367229 +Ref: distutils/apiref distutils-command-install-lib-install-library-files-from-a-package11367519 +Ref: 41a411367519 +Ref: distutils/apiref module-distutils command install_lib11367519 +Ref: 5011367519 +Node: distutils command install_scripts — Install script files from a package11367684 +Ref: distutils/apiref distutils-command-install-scripts-install-script-files-from-a-package11367973 +Ref: 41a511367973 +Ref: distutils/apiref module-distutils command install_scripts11367973 +Ref: 5111367973 +Node: distutils command register — Register a module with the Python Package Index11368144 +Ref: distutils/apiref distutils-command-register-register-a-module-with-the-python-package-index11368423 +Ref: 41a611368423 +Ref: distutils/apiref module-distutils command register11368423 +Ref: 5211368423 +Ref: distutils command register — Register a module with the Python Package Index-Footnote-111368770 +Node: distutils command check — Check the meta-data of a package11368819 +Ref: distutils/apiref distutils-command-check-check-the-meta-data-of-a-package11369016 +Ref: 41a711369016 +Ref: distutils/apiref module-distutils command check11369016 +Ref: 4a11369016 +Node: Installing Python Modules Legacy version11369358 +Ref: install/index doc11369517 +Ref: 41a811369517 +Ref: install/index install-index11369517 +Ref: 7f711369517 +Ref: install/index installing-python-modules-legacy-version11369517 +Ref: 41a911369517 +Ref: Installing Python Modules Legacy version-Footnote-111370713 +Node: Introduction<17>11370779 +Ref: install/index inst-intro11370907 +Ref: 41aa11370907 +Ref: install/index introduction11370907 +Ref: 41ab11370907 +Node: Distutils based source distributions11371871 +Ref: install/index distutils-based-source-distributions11371960 +Ref: 41ac11371960 +Ref: install/index inst-new-standard11371960 +Ref: 41ad11371960 +Node: Standard Build and Install11373247 +Ref: install/index inst-standard-install11373406 +Ref: 41ae11373406 +Ref: install/index standard-build-and-install11373406 +Ref: 41af11373406 +Node: Platform variations11373805 +Ref: install/index inst-platform-variations11373916 +Ref: 41b011373916 +Ref: install/index platform-variations11373916 +Ref: 41b111373916 +Node: Splitting the job up11374831 +Ref: install/index inst-splitting-up11374969 +Ref: 41b211374969 +Ref: install/index splitting-the-job-up11374969 +Ref: 41b311374969 +Node: How building works11376273 +Ref: install/index how-building-works11376414 +Ref: 41b411376414 +Ref: install/index inst-how-build-works11376414 +Ref: 41b511376414 +Node: How installation works11378143 +Ref: install/index how-installation-works11378255 +Ref: 41b711378255 +Ref: install/index inst-how-install-works11378255 +Ref: 41b811378255 +Node: Alternate Installation11383053 +Ref: install/index alternate-installation11383215 +Ref: 41bb11383215 +Ref: install/index inst-alt-install11383215 +Ref: 41b911383215 +Node: Alternate installation the user scheme11384903 +Ref: install/index alternate-installation-the-user-scheme11385047 +Ref: 41bc11385047 +Ref: install/index inst-alt-install-user11385047 +Ref: ff411385047 +Node: Alternate installation the home scheme11387444 +Ref: install/index alternate-installation-the-home-scheme11387642 +Ref: 41bd11387642 +Ref: install/index inst-alt-install-home11387642 +Ref: 41be11387642 +Node: Alternate installation Unix the prefix scheme11389443 +Ref: install/index alternate-installation-unix-the-prefix-scheme11389651 +Ref: 41c011389651 +Ref: install/index inst-alt-install-prefix-unix11389651 +Ref: 41c111389651 +Node: Alternate installation Windows the prefix scheme11393686 +Ref: install/index alternate-installation-windows-the-prefix-scheme11393847 +Ref: 41c211393847 +Ref: install/index inst-alt-install-prefix-windows11393847 +Ref: 41c311393847 +Node: Custom Installation11395114 +Ref: install/index custom-installation11395279 +Ref: 41c411395279 +Ref: install/index inst-custom-install11395279 +Ref: 41ba11395279 +Node: Modifying Python’s Search Path11402268 +Ref: install/index inst-search-path11402356 +Ref: 41bf11402356 +Ref: install/index modifying-python-s-search-path11402356 +Ref: 41c511402356 +Node: Distutils Configuration Files11406078 +Ref: install/index distutils-configuration-files11406256 +Ref: 41c611406256 +Ref: install/index inst-config-files11406256 +Ref: 41b611406256 +Node: Location and names of config files11406967 +Ref: install/index inst-config-filenames11407098 +Ref: 41c711407098 +Ref: install/index location-and-names-of-config-files11407098 +Ref: 41c811407098 +Node: Syntax of config files11411026 +Ref: install/index inst-config-syntax11411157 +Ref: 40e111411157 +Ref: install/index syntax-of-config-files11411157 +Ref: 41c911411157 +Node: Building Extensions Tips and Tricks11413083 +Ref: install/index building-extensions-tips-and-tricks11413233 +Ref: 41ca11413233 +Ref: install/index inst-building-ext11413233 +Ref: 41cb11413233 +Node: Tweaking compiler/linker flags11413839 +Ref: install/index inst-tweak-flags11413990 +Ref: 41cc11413990 +Ref: install/index tweaking-compiler-linker-flags11413990 +Ref: 41cd11413990 +Node: Using non-Microsoft compilers on Windows11417059 +Ref: install/index inst-non-ms-compilers11417210 +Ref: 41ce11417210 +Ref: install/index using-non-microsoft-compilers-on-windows11417210 +Ref: 41cf11417210 +Node: Borland/CodeGear C++11417369 +Ref: install/index borland-codegear-c11417497 +Ref: 41d011417497 +Ref: Borland/CodeGear C++-Footnote-111419453 +Ref: Borland/CodeGear C++-Footnote-211419564 +Ref: Borland/CodeGear C++-Footnote-311419609 +Node: GNU C / Cygwin / MinGW11419663 +Ref: install/index gnu-c-cygwin-mingw11419791 +Ref: 41d111419791 +Ref: GNU C / Cygwin / MinGW-Footnote-111420799 +Ref: GNU C / Cygwin / MinGW-Footnote-211420879 +Node: Older Versions of Python and MinGW11420980 +Ref: install/index older-versions-of-python-and-mingw11421073 +Ref: 41d211421073 +Ref: Older Versions of Python and MinGW-Footnote-111422728 +Node: Python Module Index11422793 +Node: Index11445313 +Ref: 31f113190514 +Ref: using/cmdline audit_event_cpython_run_stdin_013190544 +Ref: using/cmdline audit_event_cpython_run_file_013190575 +Ref: 31f013190606 + +End Tag Table + + +Local Variables: +coding: utf-8 +End: diff --git a/assets/info/pythonpackagingguide.info b/assets/info/pythonpackagingguide.info new file mode 100644 index 00000000..625983df --- /dev/null +++ b/assets/info/pythonpackagingguide.info @@ -0,0 +1,15961 @@ +This is pythonpackagingguide.info, produced by makeinfo version 6.5 from +pythonpackagingguide.texi. + + Python Packaging User Guide , Dec 04, 2020 + + Python Packaging Authority + + Copyright © 2013–2020, PyPA + +INFO-DIR-SECTION Miscellaneous +START-INFO-DIR-ENTRY +* pythonpackagingguide: (pythonpackagingguide.info). One line description of project. +END-INFO-DIR-ENTRY + + + Generated by Sphinx 2.3.1. + + +File: pythonpackagingguide.info, Node: Top, Next: An Overview of Packaging for Python, Up: (dir) + +Python Packaging User Guide +*************************** + + Python Packaging User Guide , Dec 04, 2020 + + Python Packaging Authority + + Copyright © 2013–2020, PyPA + +* Menu: + +* An Overview of Packaging for Python:: +* Tutorials:: +* Guides:: +* Discussions:: +* PyPA specifications:: +* Project Summaries:: +* Glossary:: +* How to Get Support:: +* Contribute to this guide:: +* News:: +* Get started:: +* Learn more:: +* Index:: + + — The Detailed Node Listing — + +An Overview of Packaging for Python + +* Thinking about deployment:: +* Packaging Python libraries and tools:: +* Packaging Python applications:: +* What about…:: +* Wrap up:: + +Packaging Python libraries and tools + +* Python modules:: +* Python source distributions:: +* Python binary distributions:: + +Packaging Python applications + +* Depending on a framework:: +* Depending on a pre-installed Python:: +* Depending on a separate software distribution ecosystem:: +* Bringing your own Python executable:: +* Bringing your own userspace:: +* Bringing your own kernel:: +* Bringing your own hardware:: + +Depending on a framework + +* Service platforms:: +* Web browsers and mobile applications:: + +What about… + +* Operating system packages:: +* virtualenv:: +* Security:: + +Tutorials + +* Installing Packages:: +* Managing Application Dependencies:: +* Packaging Python Projects:: +* Creating Documentation:: + +Installing Packages + +* Requirements for Installing Packages:: +* Creating Virtual Environments:: +* Use pip for Installing:: +* Installing from PyPI:: +* Source Distributions vs Wheels:: +* Upgrading packages:: +* Installing to the User Site:: +* Requirements files:: +* Installing from VCS:: +* Installing from other Indexes:: +* Installing from a local src tree:: +* Installing from local archives:: +* Installing from other sources:: +* Installing Prereleases:: +* Installing Setuptools “Extras”:: + +Requirements for Installing Packages + +* Ensure you can run Python from the command line:: +* Ensure you can run pip from the command line:: +* Ensure pip, setuptools, and wheel are up to date: Ensure pip setuptools and wheel are up to date. +* Optionally, create a virtual environment: Optionally create a virtual environment. + +Managing Application Dependencies + +* Installing Pipenv:: +* Installing packages for your project:: +* Using installed packages:: +* Next steps:: +* Other Tools for Application Dependency Management:: + +Packaging Python Projects + +* A simple project:: +* Creating the package files:: +* Creating a test folder:: +* Creating setup.py: Creating setup py. +* Creating README.md: Creating README md. +* Creating a LICENSE:: +* Generating distribution archives:: +* Uploading the distribution archives:: +* Installing your newly uploaded package:: +* Next steps: Next steps<2>. + +Creating Documentation + +* Installing Sphinx:: +* Getting Started With Sphinx:: +* Other Sources:: + +Guides + +* Tool recommendations:: +* Installing packages using pip and virtual environments:: +* Installing stand alone command line tools:: +* Installing pip/setuptools/wheel with Linux Package Managers:: +* Installing scientific packages:: +* Multi-version installs:: +* Packaging and distributing projects:: +* Including files in source distributions with MANIFEST.in: Including files in source distributions with MANIFEST in. +* Single-sourcing the package version:: +* Supporting multiple Python versions:: +* Dropping support for older Python versions:: +* Packaging binary extensions:: +* Supporting Windows using Appveyor:: +* Packaging namespace packages:: +* Creating and discovering plugins:: +* Analyzing PyPI package downloads:: +* Package index mirrors and caches:: +* Hosting your own simple repository:: +* Migrating to PyPI.org: Migrating to PyPI org. +* Using TestPyPI: Using TestPyPI<2>. +* Making a PyPI-friendly README:: +* Publishing package distribution releases using GitHub Actions CI/CD workflows:: + +Tool recommendations + +* Application dependency management:: +* Installation tool recommendations:: +* Packaging tool recommendations:: +* Publishing platform migration:: + +Installing packages using pip and virtual environments + +* Installing pip:: +* Installing virtualenv:: +* Creating a virtual environment:: +* Activating a virtual environment:: +* Leaving the virtual environment:: +* Installing packages:: +* Installing specific versions:: +* Installing extras:: +* Installing from source:: +* Installing from version control systems:: +* Installing from local archives: Installing from local archives<2>. +* Using other package indexes:: +* Upgrading packages: Upgrading packages<2>. +* Using requirements files:: +* Freezing dependencies:: + +Installing pip + +* Windows:: +* Linux and macOS:: + +Installing pip/setuptools/wheel with Linux Package Managers + +* Fedora:: +* CentOS/RHEL:: +* openSUSE:: +* Debian/Ubuntu:: +* Arch Linux:: + +Installing scientific packages + +* Building from source:: +* Linux distribution packages:: +* Windows installers:: +* macOS installers and package managers:: +* SciPy distributions:: +* Spack:: +* The conda cross-platform package manager:: + +Packaging and distributing projects + +* Requirements for packaging and distributing:: +* Configuring your project:: +* Working in “development mode”:: +* Packaging your project:: +* Uploading your Project to PyPI:: + +Configuring your project + +* Initial files:: +* setup() args: setup args. +* Choosing a versioning scheme:: + +Initial files + +* setup.py: setup py. +* setup.cfg: setup cfg. +* README.rst / README.md: README rst / README md. +* MANIFEST.in: MANIFEST in. +* LICENSE.txt: LICENSE txt. +* :: + +setup() args + +* name:: +* version:: +* description:: +* url:: +* author:: +* license:: +* classifiers:: +* keywords:: +* project_urls:: +* packages:: +* py_modules:: +* install_requires:: +* python_requires:: +* package_data:: +* data_files:: +* scripts:: +* entry_points:: + +entry_points + +* console_scripts:: + +Choosing a versioning scheme + +* Standards compliance for interoperability:: +* Scheme choices:: +* Pre-release versioning:: +* Local version identifiers:: + +Scheme choices + +* Semantic versioning (preferred): Semantic versioning preferred. +* Date based versioning:: +* Serial versioning:: +* Hybrid schemes:: + +Packaging your project + +* Source distributions:: +* Wheels:: + +Wheels + +* Universal Wheels:: +* Pure Python Wheels:: +* Platform Wheels:: + +Uploading your Project to PyPI + +* Create an account:: +* Upload your distributions:: + +Including files in source distributions with MANIFEST.in + +* How files are included in an sdist:: +* MANIFEST.in commands: MANIFEST in commands. + +Supporting multiple Python versions + +* Automated testing and continuous integration:: +* Tools for single-source Python packages:: +* What’s in which Python?:: + +Dropping support for older Python versions + +* Requirements:: +* Defining the Python version required:: +* Dropping a Python release:: + +Defining the Python version required + +* 1. Download the newest version of Setuptools: 1 Download the newest version of Setuptools. +* 2. Specify the version ranges for supported Python distributions: 2 Specify the version ranges for supported Python distributions. +* 3. Validating the Metadata before publishing: 3 Validating the Metadata before publishing. +* 4. Using Twine to publish: 4 Using Twine to publish. + +Packaging binary extensions + +* An overview of binary extensions:: +* Implementing binary extensions:: +* Building binary extensions:: +* Publishing binary extensions:: +* Additional resources:: + +An overview of binary extensions + +* Use cases:: +* Disadvantages:: +* Alternatives to handcoded accelerator modules:: +* Alternatives to handcoded wrapper modules:: +* Alternatives for low level system access:: + +Building binary extensions + +* Binary extensions for Windows:: +* Binary extensions for Linux:: +* Binary extensions for macOS:: + +Additional resources + +* Cross-platform wheel generation with scikit-build:: +* Introduction to C/C++ extension modules:: + +Supporting Windows using Appveyor + +* Background:: +* Setting up:: +* Adding Appveyor support to your project:: +* Additional notes:: + +Adding Appveyor support to your project + +* appveyor.yml: appveyor yml. +* Support script:: +* Access to the built wheels:: + +Additional notes + +* Testing with tox:: +* Automatically uploading wheels:: +* External dependencies:: +* Support scripts:: + +Packaging namespace packages + +* Creating a namespace package:: + +Creating a namespace package + +* Native namespace packages:: +* pkgutil-style namespace packages:: +* pkg_resources-style namespace packages:: + +Creating and discovering plugins + +* Using naming convention:: +* Using namespace packages:: +* Using package metadata:: + +Analyzing PyPI package downloads + +* Background: Background<2>. +* Public dataset:: +* Caveats:: +* Additional tools:: +* References:: + +Public dataset + +* Getting set up:: +* Data schema:: +* Useful queries:: + +Useful queries + +* Counting package downloads:: +* Package downloads over time:: +* Python versions over time:: + +Additional tools + +* google-cloud-bigquery:: +* pypinfo:: +* pandas-gbq:: + +Package index mirrors and caches + +* Caching with pip:: +* Caching with devpi:: +* Complete mirror with bandersnatch:: + +Hosting your own simple repository + +* “Manual” repository:: + +Migrating to PyPI.org + +* Publishing releases:: +* Registering package names & metadata:: +* Using TestPyPI:: +* Registering new user accounts:: +* Browsing packages:: +* Downloading packages:: +* Managing published packages and releases:: + +Using TestPyPI + +* Registering your account:: +* Using TestPyPI with Twine:: +* Using TestPyPI with pip:: +* Setting up TestPyPI in .pypirc: Setting up TestPyPI in pypirc. + +Making a PyPI-friendly README + +* Creating a README file:: +* Including your README in your package’s metadata:: +* Validating reStructuredText markup:: + +Publishing package distribution releases using GitHub Actions CI/CD workflows + +* Saving credentials on GitHub:: +* Creating a workflow definition:: +* Defining a workflow job environment:: +* Checking out the project and building distributions:: +* Publishing the distribution to PyPI and TestPyPI:: +* That’s all, folks!: That’s all folks!. + +Discussions + +* Deploying Python applications:: +* pip vs easy_install:: +* install_requires vs requirements files:: +* Wheel vs Egg:: + +Deploying Python applications + +* Overview:: +* OS packaging & installers:: +* Application bundles:: +* Configuration management:: + +Overview + +* Supporting multiple hardware platforms:: + +OS packaging & installers + +* Windows: Windows<2>. + +Windows + +* Pynsist:: + +install_requires vs requirements files + +* install_requires: install_requires<2>. +* Requirements files: Requirements files<2>. + +PyPA specifications + +* Package Distribution Metadata:: +* Package Index Interfaces:: + +Package Distribution Metadata + +* Core metadata specifications:: +* Version specifiers:: +* Dependency specifiers:: +* Declaring build system dependencies:: +* Declaring project metadata:: +* Distribution formats:: +* Platform compatibility tags:: +* Recording installed projects:: +* Entry points specification:: + +Core metadata specifications + +* Metadata-Version:: +* Name:: +* Version:: +* Platform (multiple use): Platform multiple use. +* Supported-Platform (multiple use): Supported-Platform multiple use. +* Summary:: +* Description:: +* Description-Content-Type:: +* Keywords:: +* Home-page:: +* Download-URL:: +* Author:: +* Author-email:: +* Maintainer:: +* Maintainer-email:: +* License:: +* Classifier (multiple use): Classifier multiple use. +* Requires-Dist (multiple use): Requires-Dist multiple use. +* Requires-Python:: +* Requires-External (multiple use): Requires-External multiple use. +* Project-URL (multiple-use): Project-URL multiple-use. +* Provides-Extra (multiple use): Provides-Extra multiple use. +* Rarely Used Fields:: + +Rarely Used Fields + +* Provides-Dist (multiple use): Provides-Dist multiple use. +* Obsoletes-Dist (multiple use): Obsoletes-Dist multiple use. + +Declaring project metadata + +* name: name<2>. +* version: version<2>. +* description: description<2>. +* readme:: +* requires-python:: +* license: license<2>. +* authors/maintainers:: +* keywords: keywords<2>. +* classifiers: classifiers<2>. +* urls:: +* Entry points:: +* dependencies/optional-dependencies:: +* dynamic:: + +Distribution formats + +* Source distribution format:: +* Binary distribution format:: + +Platform compatibility tags + +* Platform tags for Windows:: +* Platform tags for macOS (Mac OS X): Platform tags for macOS Mac OS X. +* Platform tags for common Linux distributions:: +* Platform tags for other *nix platforms:: + +Platform tags for common Linux distributions + +* Manylinux compatibility support:: + +Recording installed projects + +* History and change workflow:: +* The .dist-info directory: The dist-info directory. +* The METADATA file:: +* The RECORD file:: +* The INSTALLER file:: + +Entry points specification + +* Data model:: +* File format:: +* Use for scripts:: + +Package Index Interfaces + +* The .pypirc file: The pypirc file. +* Simple repository API:: + +The .pypirc file + +* Common configurations:: + +Common configurations + +* Using a PyPI token:: +* Using another package index:: + +Project Summaries + +* PyPA Projects:: +* Non-PyPA Projects:: +* Standard Library Projects:: + +PyPA Projects + +* bandersnatch:: +* build:: +* distlib:: +* packaging:: +* pip:: +* Pipenv:: +* Pipfile:: +* Python Packaging User Guide: Python Packaging User Guide<2>. +* readme_renderer:: +* setuptools:: +* trove-classifiers:: +* twine:: +* virtualenv: virtualenv<2>. +* Warehouse:: +* wheel:: + +Non-PyPA Projects + +* bento:: +* buildout:: +* conda:: +* devpi:: +* flit:: +* enscons:: +* Hashdist:: +* hatch:: +* pex:: +* pipx:: +* pip-tools:: +* piwheels:: +* poetry:: +* pypiserver:: +* scikit-build:: +* shiv:: +* Spack: Spack<2>. +* zest.releaser: zest releaser. + +Standard Library Projects + +* ensurepip:: +* distutils:: +* venv:: + +Contribute to this guide + +* Documentation types:: +* Building the guide locally:: +* Where the guide is deployed:: +* Style guide:: + +Documentation types + +* Tutorials: Tutorials<2>. +* Guides: Guides<2>. +* Discussions: Discussions<2>. +* Specifications:: + +Style guide + +* Purpose:: +* Scope:: +* Audience:: +* Voice and tone:: +* Conventions and mechanics:: + +News + +* September 2019:: +* August 2019:: +* July 2019:: +* June 2019:: +* May 2019:: +* April 2019:: +* March 2019:: +* February 2019:: +* January 2019:: +* December 2018:: +* November 2018:: +* October 2018:: +* September 2018:: +* August 2018:: +* July 2018:: +* June 2018:: +* May 2018:: +* April 2018:: +* March 2018:: +* February 2018:: +* January 2018:: +* December 2017:: +* November 2017:: +* October 2017:: +* September 2017:: +* August 2017:: +* July 2017:: +* June 2017:: +* May 2017:: +* April 2017:: +* March 2017:: +* February 2017:: + + + +File: pythonpackagingguide.info, Node: An Overview of Packaging for Python, Next: Tutorials, Prev: Top, Up: Top + +1 An Overview of Packaging for Python +************************************* + +As a general-purpose programming language, Python is designed to be used +in many ways. You can build web sites or industrial robots or a game +for your friends to play, and much more, all using the same core +technology. + +Python’s flexibility is why the first step in every Python project must +be to think about the project’s audience and the corresponding +environment where the project will run. It might seem strange to think +about packaging before writing code, but this process does wonders for +avoiding future headaches. + +This overview provides a general-purpose decision tree for reasoning +about Python’s plethora of packaging options. Read on to choose the +best technology for your next project. + +* Menu: + +* Thinking about deployment:: +* Packaging Python libraries and tools:: +* Packaging Python applications:: +* What about…:: +* Wrap up:: + + +File: pythonpackagingguide.info, Node: Thinking about deployment, Next: Packaging Python libraries and tools, Up: An Overview of Packaging for Python + +1.1 Thinking about deployment +============================= + +Packages exist to be installed (or `deployed'), so before you package +anything, you’ll want to have some answers to the deployment questions +below: + + * Who are your software’s users? Will your software be installed by + other developers doing software development, operations people in a + datacenter, or a less software-savvy group? + + * Is your software intended to run on servers, desktops, mobile + clients (phones, tablets, etc.), or embedded in dedicated devices? + + * Is your software installed individually, or in large deployment + batches? + +Packaging is all about target environment and deployment experience. +There are many answers to the questions above and each combination of +circumstances has its own solutions. With this information, the +following overview will guide you to the packaging technologies best +suited to your project. + + +File: pythonpackagingguide.info, Node: Packaging Python libraries and tools, Next: Packaging Python applications, Prev: Thinking about deployment, Up: An Overview of Packaging for Python + +1.2 Packaging Python libraries and tools +======================================== + +You may have heard about PyPI, ‘setup.py’, and ‘wheel’ files. These are +just a few of the tools Python’s ecosystem provides for distributing +Python code to developers, which you can read about in *note Packaging +and distributing projects: 6. + +The following approaches to packaging are meant for libraries and tools +used by technical audience in a development setting. If you’re looking +for ways to package Python for a non-technical audience and/or a +production setting, skip ahead to *note Packaging Python applications: +7. + +* Menu: + +* Python modules:: +* Python source distributions:: +* Python binary distributions:: + + +File: pythonpackagingguide.info, Node: Python modules, Next: Python source distributions, Up: Packaging Python libraries and tools + +1.2.1 Python modules +-------------------- + +A Python file, provided it only relies on the standard library, can be +redistributed and reused. You will also need to ensure it’s written for +the right version of Python, and only relies on the standard library. + +This is great for sharing simple scripts and snippets between people who +both have compatible Python versions (such as via email, StackOverflow, +or GitHub gists). There are even some entire Python libraries that +offer this as an option, such as bottle.py(1) and boltons(2). + +However, this pattern won’t scale for projects that consist of multiple +files, need additional libraries, or need a specific version of Python, +hence the options below. + + ---------- Footnotes ---------- + + (1) https://bottlepy.org/docs/dev/ + + (2) +http://boltons.readthedocs.io/en/latest/architecture.html#architecture + + +File: pythonpackagingguide.info, Node: Python source distributions, Next: Python binary distributions, Prev: Python modules, Up: Packaging Python libraries and tools + +1.2.2 Python source distributions +--------------------------------- + +If your code consists of multiple Python files, it’s usually organized +into a directory structure. Any directory containing Python files can +comprise an *note Import Package: a. + +Because packages consist of multiple files, they are harder to +distribute. Most protocols support transferring only one file at a time +(when was the last time you clicked a link and it downloaded multiple +files?). It’s easier to get incomplete transfers, and harder to +guarantee code integrity at the destination. + +So long as your code contains nothing but pure Python code, and you know +your deployment environment supports your version of Python, then you +can use Python’s native packaging tools to create a `source' *note +Distribution Package: b, or `sdist' for short. + +Python’s `sdists' are compressed archives (‘.tar.gz’ files) containing +one or more packages or modules. If your code is pure-Python, and you +only depend on other Python packages, you can go here to learn more(1). + +If you rely on any non-Python code, or non-Python packages (such as +libxml2(2) in the case of lxml(3), or BLAS libraries in the case of +numpy(4)), you will need to use the format detailed in the next section, +which also has many advantages for pure-Python libraries. + + Note: Python and PyPI support multiple distributions providing + different implementations of the same package. For instance the + unmaintained-but-seminal PIL distribution(5) provides the PIL + package, and so does Pillow(6), an actively-maintained fork of PIL! + + This Python packaging superpower makes it possible for Pillow to be + a drop-in replacement for PIL, just by changing your project’s + ‘install_requires’ or ‘requirements.txt’. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/distutils/sourcedist.html + + (2) https://en.wikipedia.org/wiki/Libxml2 + + (3) https://pypi.org/project/lxml/ + + (4) https://pypi.org/project/numpy + + (5) https://pypi.org/project/PIL/ + + (6) https://pypi.org/project/Pillow/ + + +File: pythonpackagingguide.info, Node: Python binary distributions, Prev: Python source distributions, Up: Packaging Python libraries and tools + +1.2.3 Python binary distributions +--------------------------------- + +So much of Python’s practical power comes from its ability to integrate +with the software ecosystem, in particular libraries written in C, C++, +Fortran, Rust, and other languages. + +Not all developers have the right tools or experiences to build these +components written in these compiled languages, so Python created the +*note Wheel: d, a package format designed to ship libraries with +compiled artifacts. In fact, Python’s package installer, ‘pip’, always +prefers wheels because installation is always faster, so even +pure-Python packages work better with wheels. + +Binary distributions are best when they come with source distributions +to match. Even if you don’t upload wheels of your code for every +operating system, by uploading the sdist, you’re enabling users of other +platforms to still build it for themselves. Default to publishing both +sdist and wheel archives together, `unless' you’re creating artifacts +for a very specific use case where you know the recipient only needs one +or the other. + +Python and PyPI make it easy to upload both wheels and sdists together. +Just follow the *note Packaging Python Projects: e. tutorial. + +[A summary of Python's packaging capabilities for tools and libraries.] +Figure: Python’s recommended built-in library and tool packaging +technologies. Excerpted from The Packaging Gradient (2017)(1). + + ---------- Footnotes ---------- + + (1) https://www.youtube.com/watch?v=iLVNWfPWAC8 + + +File: pythonpackagingguide.info, Node: Packaging Python applications, Next: What about…, Prev: Packaging Python libraries and tools, Up: An Overview of Packaging for Python + +1.3 Packaging Python applications +================================= + +So far we’ve only discussed Python’s native distribution tools. Based +on our introduction, you would be correct to infer these built-in +approaches only target environments which have Python, and an audience +who knows how to install Python packages. + +With the variety of operating systems, configurations, and people out +there, this assumption is only safe when targeting a developer audience. + +Python’s native packaging is mostly built for distributing reusable +code, called libraries, between developers. You can piggyback `tools', +or basic applications for developers, on top of Python’s library +packaging, using technologies like setuptools entry_points(1). + +Libraries are building blocks, not complete applications. For +distributing applications, there’s a whole new world of technologies out +there. + +The next few sections organize these application packaging options +according to their dependencies on the target environment, so you can +choose the right one for your project. + +* Menu: + +* Depending on a framework:: +* Depending on a pre-installed Python:: +* Depending on a separate software distribution ecosystem:: +* Bringing your own Python executable:: +* Bringing your own userspace:: +* Bringing your own kernel:: +* Bringing your own hardware:: + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/userguide/entry_point.html + + +File: pythonpackagingguide.info, Node: Depending on a framework, Next: Depending on a pre-installed Python, Up: Packaging Python applications + +1.3.1 Depending on a framework +------------------------------ + +Some types of Python applications, like web site backends and other +network services, are common enough that they have frameworks to enable +their development and packaging. Other types of applications, like +dynamic web frontends and mobile clients, are complex enough to target +that a framework becomes more than a convenience. + +In all these cases, it makes sense to work backwards, from the +framework’s packaging and deployment story. Some frameworks include a +deployment system which wraps the technologies outlined in the rest of +the guide. In these cases, you’ll want to defer to your framework’s +packaging guide for the easiest and most reliable production experience. + +If you ever wonder how these platforms and frameworks work under the +hood, you can always read the sections beyond. + +* Menu: + +* Service platforms:: +* Web browsers and mobile applications:: + + +File: pythonpackagingguide.info, Node: Service platforms, Next: Web browsers and mobile applications, Up: Depending on a framework + +1.3.1.1 Service platforms +......................... + +If you’re developing for a “Platform-as-a-Service” or “PaaS” like Heroku +or Google App Engine, you are going to want to follow their respective +packaging guides. + + * Heroku(1) + + * Google App Engine(2) + + * PythonAnywhere(3) + + * OpenShift(4) + + * “Serverless” frameworks like Zappa(5) + +In all these setups, the platform takes care of packaging and +deployment, as long as you follow their patterns. Most software does +not fit one of these templates, hence the existence of all the other +options below. + +If you’re developing software that will be deployed to machines you own, +users’ personal computers, or any other arrangement, read on. + + ---------- Footnotes ---------- + + (1) https://devcenter.heroku.com/articles/getting-started-with-python + + (2) https://cloud.google.com/appengine/docs/python/ + + (3) https://www.pythonanywhere.com/ + + (4) https://blog.openshift.com/getting-started-python/ + + (5) https://www.zappa.io/ + + +File: pythonpackagingguide.info, Node: Web browsers and mobile applications, Prev: Service platforms, Up: Depending on a framework + +1.3.1.2 Web browsers and mobile applications +............................................ + +Python’s steady advances are leading it into new spaces. These days you +can write a mobile app or web application frontend in Python. While the +language may be familiar, the packaging and deployment practices are +brand new. + +If you’re planning on releasing to these new frontiers, you’ll want to +check out the following frameworks, and refer to their packaging guides: + + * Kivy(1) + + * Beeware(2) + + * Brython(3) + + * Flexx(4) + +If you are `not' interested in using a framework or platform, or just +wonder about some of the technologies and techniques utilized by the +frameworks above, continue reading below. + + ---------- Footnotes ---------- + + (1) https://kivy.org/#home + + (2) https://pybee.org/ + + (3) https://brython.info/ + + (4) http://flexx.readthedocs.io/en/latest/ + + +File: pythonpackagingguide.info, Node: Depending on a pre-installed Python, Next: Depending on a separate software distribution ecosystem, Prev: Depending on a framework, Up: Packaging Python applications + +1.3.2 Depending on a pre-installed Python +----------------------------------------- + +Pick an arbitrary computer, and depending on the context, there’s a very +good chance Python is already installed. Included by default in most +Linux and Mac operating systems for many years now, you can reasonably +depend on Python preexisting in your data centers or on the personal +machines of developers and data scientists. + +Technologies which support this model: + + * PEX(1) (Python EXecutable) + + * zipapp(2) (does not help manage dependencies, requires Python 3.5+) + + * shiv(3) (requires Python 3) + + Note: Of all the approaches here, depending on a pre-installed + Python relies the most on the target environment. Of course, this + also makes for the smallest package, as small as single-digit + megabytes, or even kilobytes. + + In general, decreasing the dependency on the target system + increases the size of our package, so the solutions here are + roughly arranged by increasing size of output. + + ---------- Footnotes ---------- + + (1) https://github.com/pantsbuild/pex#pex + + (2) https://docs.python.org/3/library/zipapp.html + + (3) https://github.com/linkedin/shiv#shiv + + +File: pythonpackagingguide.info, Node: Depending on a separate software distribution ecosystem, Next: Bringing your own Python executable, Prev: Depending on a pre-installed Python, Up: Packaging Python applications + +1.3.3 Depending on a separate software distribution ecosystem +------------------------------------------------------------- + +For a long time many operating systems, including Mac and Windows, +lacked built-in package management. Only recently did these OSes gain +so-called “app stores”, but even those focus on consumer applications +and offer little for developers. + +Developers long sought remedies, and in this struggle, emerged with +their own package management solutions, such as Homebrew(1). The most +relevant alternative for Python developers is a package ecosystem called +Anaconda(2). Anaconda is built around Python and is increasingly common +in academic, analytical, and other data-oriented environments, even +making its way into server-oriented environments(3). + +Instructions on building and publishing for the Anaconda ecosystem: + + * Building libraries and applications with conda(4) + + * Transitioning a native Python package to Anaconda(5) + +A similar model involves installing an alternative Python distribution, +but does not support arbitrary operating system-level packages: + + * Enthought Canopy(6) + + * ActiveState ActivePython(7) + + * WinPython(8) + + ---------- Footnotes ---------- + + (1) https://brew.sh/ + + (2) https://en.wikipedia.org/wiki/Anaconda_(Python_distribution) + + (3) +https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/ + + (4) https://conda.io/docs/user-guide/tutorials/index.html + + (5) +https://conda.io/docs/user-guide/tutorials/build-pkgs-skeleton.html + + (6) https://www.enthought.com/product/canopy/ + + (7) https://www.activestate.com/activepython + + (8) http://winpython.github.io/ + + +File: pythonpackagingguide.info, Node: Bringing your own Python executable, Next: Bringing your own userspace, Prev: Depending on a separate software distribution ecosystem, Up: Packaging Python applications + +1.3.4 Bringing your own Python executable +----------------------------------------- + +Computing as we know it is defined by the ability to execute programs. +Every operating system natively supports one or more formats of program +they can natively execute. + +There are many techniques and technologies which turn your Python +program into one of these formats, most of which involve embedding the +Python interpreter and any other dependencies into a single executable +file. + +This approach, called `freezing', offers wide compatibility and seamless +user experience, though often requires multiple technologies, and a good +amount of effort. + +A selection of Python freezers: + + * pyInstaller(1) - Cross-platform + + * cx_Freeze(2) - Cross-platform + + * constructor(3) - For command-line installers + + * py2exe(4) - Windows only + + * py2app(5) - Mac only + + * bbFreeze(6) - Windows, Linux, Python 2 only + + * osnap(7) - Windows and Mac + + * pynsist(8) - Windows only + +Most of the above imply single-user deployments. For multi-component +server applications, see Chef Omnibus(9). + + ---------- Footnotes ---------- + + (1) http://www.pyinstaller.org/ + + (2) https://marcelotduarte.github.io/cx_Freeze/ + + (3) https://github.com/conda/constructor + + (4) http://www.py2exe.org/ + + (5) https://py2app.readthedocs.io/en/latest/ + + (6) https://pypi.org/project/bbfreeze + + (7) https://github.com/jamesabel/osnap + + (8) https://pypi.org/project/pynsist/ + + (9) https://github.com/chef/omnibus#-omnibus + + +File: pythonpackagingguide.info, Node: Bringing your own userspace, Next: Bringing your own kernel, Prev: Bringing your own Python executable, Up: Packaging Python applications + +1.3.5 Bringing your own userspace +--------------------------------- + +An increasing number of operating systems – including Linux, Mac OS, and +Windows – can be set up to run applications packaged as lightweight +images, using a relatively modern arrangement often referred to as +operating-system-level virtualization(1), or `containerization'. + +These techniques are mostly Python agnostic, because they package whole +OS filesystems, not just Python or Python packages. + +Adoption is most extensive among Linux servers, where the technology +originated and where the technologies below work best: + + * AppImage(2) + + * Docker(3) + + * Flatpak(4) + + * Snapcraft(5) + + ---------- Footnotes ---------- + + (1) +https://en.wikipedia.org/wiki/Operating-system-level_virtualization + + (2) https://appimage.org/ + + (3) https://www.fullstackpython.com/docker.html + + (4) https://flatpak.org/ + + (5) https://snapcraft.io/ + + +File: pythonpackagingguide.info, Node: Bringing your own kernel, Next: Bringing your own hardware, Prev: Bringing your own userspace, Up: Packaging Python applications + +1.3.6 Bringing your own kernel +------------------------------ + +Most operating systems support some form of classical virtualization, +running applications packaged as images containing a full operating +system of their own. Running these virtual machines, or VMs, is a +mature approach, widespread in data center environments. + +These techniques are mostly reserved for larger scale deployments in +data centers, though certain complex applications can benefit from this +packaging. Technologies are Python agnostic, and include: + + * Vagrant(1) + + * VHD(2), AMI(3), and other formats(4) + + * OpenStack(5) - A cloud management system in Python, with extensive + VM support + + ---------- Footnotes ---------- + + (1) https://www.vagrantup.com/ + + (2) https://en.wikipedia.org/wiki/VHD_(file_format) + + (3) https://en.wikipedia.org/wiki/Amazon_Machine_Image + + (4) https://docs.openstack.org/image-guide/image-formats.html + + (5) https://www.redhat.com/en/topics/openstack + + +File: pythonpackagingguide.info, Node: Bringing your own hardware, Prev: Bringing your own kernel, Up: Packaging Python applications + +1.3.7 Bringing your own hardware +-------------------------------- + +The most all-encompassing way to ship your software would be to ship it +already-installed on some hardware. This way, your software’s user +would require only electricity. + +Whereas the virtual machines described above are primarily reserved for +the tech-savvy, you can find hardware appliances being used by everyone +from the most advanced data centers to the youngest children. + +Embed your code on an Adafruit(1), MicroPython(2), or more-powerful +hardware running Python, then ship it to the datacenter or your users’ +homes. They plug and play, and you can call it a day. + +[A summary of technologies used to package Python applications.] +Figure: The simplified gamut of technologies used to package Python +applications. + + ---------- Footnotes ---------- + + (1) https://github.com/adafruit/circuitpython#adafruit-circuitpython + + (2) https://micropython.org/ + + +File: pythonpackagingguide.info, Node: What about…, Next: Wrap up, Prev: Packaging Python applications, Up: An Overview of Packaging for Python + +1.4 What about… +=============== + +The sections above can only summarize so much, and you might be +wondering about some of the more conspicuous gaps. + +* Menu: + +* Operating system packages:: +* virtualenv:: +* Security:: + + +File: pythonpackagingguide.info, Node: Operating system packages, Next: virtualenv, Up: What about… + +1.4.1 Operating system packages +------------------------------- + +As mentioned in *note Depending on a separate software distribution +ecosystem: 14. above, some operating systems have package managers of +their own. If you’re very sure of the operating system you’re +targeting, you can depend directly on a format like deb(1) (for Debian, +Ubuntu, etc.) or RPM(2) (for Red Hat, Fedora, etc.), and use that +built-in package manager to take care of installation, and even +deployment. You can even use FPM(3) to generate both deb and RPMs from +the same source. + +In most deployment pipelines, the OS package manager is just one piece +of the puzzle. + + ---------- Footnotes ---------- + + (1) https://en.wikipedia.org/wiki/Deb_(file_format) + + (2) https://en.wikipedia.org/wiki/RPM_Package_Manager + + (3) https://fpm.readthedocs.io/en/latest/source/virtualenv.html + + +File: pythonpackagingguide.info, Node: virtualenv, Next: Security, Prev: Operating system packages, Up: What about… + +1.4.2 virtualenv +---------------- + +Virtualenvs(1) have been an indispensible tool for multiple generations +of Python developer, but are slowly fading from view, as they are being +wrapped by higher-level tools. With packaging in particular, +virtualenvs are used as a primitive in the dh-virtualenv tool(2) and +osnap(3), both of which wrap virtualenvs in a self-contained way. + +For production deployments, do not rely on running ‘pip install’ from +the Internet into a virtualenv, as one might do in a development +environment. The overview above is full of much better solutions. + + ---------- Footnotes ---------- + + (1) http://python-guide.readthedocs.io/en/latest/dev/virtualenvs/ + + (2) http://dh-virtualenv.readthedocs.io/en/1.0/tutorial.html + + (3) https://github.com/jamesabel/osnap + + +File: pythonpackagingguide.info, Node: Security, Prev: virtualenv, Up: What about… + +1.4.3 Security +-------------- + +The further down the gradient you come, the harder it gets to update +components of your package. Everything is more tightly bound together. + +For example, if a kernel security issue emerges, and you’re deploying +containers, the host system’s kernel can be updated without requiring a +new build on behalf of the application. If you deploy VM images, you’ll +need a new build. Whether or not this dynamic makes one option more +secure is still a bit of an old debate, going back to the +still-unsettled matter of static versus dynamic linking(1). + + ---------- Footnotes ---------- + + (1) +https://www.google.com/search?channel=fs&q=static+vs+dynamic+linking + + +File: pythonpackagingguide.info, Node: Wrap up, Prev: What about…, Up: An Overview of Packaging for Python + +1.5 Wrap up +=========== + +Packaging in Python has a bit of a reputation for being a bumpy ride. +This impression is mostly a byproduct of Python’s versatility. Once you +understand the natural boundaries between each packaging solution, you +begin to realize that the varied landscape is a small price Python +programmers pay for using one of the most balanced, flexible language +available. + + +File: pythonpackagingguide.info, Node: Tutorials, Next: Guides, Prev: An Overview of Packaging for Python, Up: Top + +2 Tutorials +*********** + +`Tutorials' are opinionated step-by-step guides to help you get familiar +with packaging concepts. For more detailed information on specific +packaging topics, see *note Guides: 22. + +* Menu: + +* Installing Packages:: +* Managing Application Dependencies:: +* Packaging Python Projects:: +* Creating Documentation:: + + +File: pythonpackagingguide.info, Node: Installing Packages, Next: Managing Application Dependencies, Up: Tutorials + +2.1 Installing Packages +======================= + +This section covers the basics of how to install Python *note packages: +b. + +It’s important to note that the term “package” in this context is being +used as a synonym for a *note distribution: b. (i.e. a bundle of +software to be installed), not to refer to the kind of *note package: a. +that you import in your Python source code (i.e. a container of +modules). It is common in the Python community to refer to a *note +distribution: b. using the term “package”. Using the term +“distribution” is often not preferred, because it can easily be confused +with a Linux distribution, or another larger software distribution like +Python itself. + +* Menu: + +* Requirements for Installing Packages:: +* Creating Virtual Environments:: +* Use pip for Installing:: +* Installing from PyPI:: +* Source Distributions vs Wheels:: +* Upgrading packages:: +* Installing to the User Site:: +* Requirements files:: +* Installing from VCS:: +* Installing from other Indexes:: +* Installing from a local src tree:: +* Installing from local archives:: +* Installing from other sources:: +* Installing Prereleases:: +* Installing Setuptools “Extras”:: + + +File: pythonpackagingguide.info, Node: Requirements for Installing Packages, Next: Creating Virtual Environments, Up: Installing Packages + +2.1.1 Requirements for Installing Packages +------------------------------------------ + +This section describes the steps to follow before installing other +Python packages. + +* Menu: + +* Ensure you can run Python from the command line:: +* Ensure you can run pip from the command line:: +* Ensure pip, setuptools, and wheel are up to date: Ensure pip setuptools and wheel are up to date. +* Optionally, create a virtual environment: Optionally create a virtual environment. + + +File: pythonpackagingguide.info, Node: Ensure you can run Python from the command line, Next: Ensure you can run pip from the command line, Up: Requirements for Installing Packages + +2.1.1.1 Ensure you can run Python from the command line +....................................................... + +Before you go any further, make sure you have Python and that the +expected version is available from your command line. You can check +this by running: + + python --version + +You should get some output like ‘Python 3.6.3’. If you do not have +Python, please install the latest 3.x version from python.org(1) or +refer to the Installing Python(2) section of the Hitchhiker’s Guide to +Python. + + Note: If you’re a newcomer and you get an error like this: + + >>> python --version + Traceback (most recent call last): + File "", line 1, in + NameError: name 'python' is not defined + + It’s because this command and other suggested commands in this + tutorial are intended to be run in a `shell' (also called a + `terminal' or `console'). See the Python for Beginners getting + started tutorial(3) for an introduction to using your operating + system’s shell and interacting with Python. + + Note: If you’re using an enhanced shell like IPython or the Jupyter + notebook, you can run system commands like those in this tutorial + by prefacing them with a ‘!’ character: + + In [1]: import sys + !{sys.executable} --version + Python 3.6.3 + + It’s recommended to write ‘{sys.executable}’ rather than plain + ‘python’ in order to ensure that commands are run in the Python + installation matching the currently running notebook (which may not + be the same Python installation that the ‘python’ command refers + to). + + Note: Due to the way most Linux distributions are handling the + Python 3 migration, Linux users using the system Python without + creating a virtual environment first should replace the ‘python’ + command in this tutorial with ‘python3’ and the ‘pip’ command with + ‘pip3 --user’. Do `not' run any of the commands in this tutorial + with ‘sudo’: if you get a permissions error, come back to the + section on creating virtual environments, set one up, and then + continue with the tutorial as written. + + ---------- Footnotes ---------- + + (1) https://python.org + + (2) http://docs.python-guide.org/en/latest/starting/installation/ + + (3) +https://opentechschool.github.io/python-beginners/en/getting_started.html#what-is-python-exactly + + +File: pythonpackagingguide.info, Node: Ensure you can run pip from the command line, Next: Ensure pip setuptools and wheel are up to date, Prev: Ensure you can run Python from the command line, Up: Requirements for Installing Packages + +2.1.1.2 Ensure you can run pip from the command line +.................................................... + +Additionally, you’ll need to make sure you have *note pip: 2b. +available. You can check this by running: + + pip --version + +If you installed Python from source, with an installer from +python.org(1), or via Homebrew(2) you should already have pip. If +you’re on Linux and installed using your OS package manager, you may +have to install pip separately, see *note Installing +pip/setuptools/wheel with Linux Package Managers: 2c. + +If ‘pip’ isn’t already installed, then first try to bootstrap it from +the standard library: + + python -m ensurepip --default-pip + +If that still doesn’t allow you to run ‘pip’: + + * Securely Download get-pip.py(3) (4) + + * Run ‘python get-pip.py’. (5) This will install or upgrade + pip. Additionally, it will install *note setuptools: 2d. and + *note wheel: 2e. if they’re not installed already. + + Warning: Be cautious if you’re using a Python install + that’s managed by your operating system or another + package manager. get-pip.py does not coordinate with + those tools, and may leave your system in an inconsistent + state. You can use ‘python get-pip.py + --prefix=/usr/local/’ to install in ‘/usr/local’ which is + designed for locally-installed software. + + ---------- Footnotes ---------- + + (1) https://python.org + + (2) https://brew.sh + + (3) https://bootstrap.pypa.io/get-pip.py + + (4) (1) “Secure” in this context means using a modern browser or a +tool like ‘curl’ that verifies SSL certificates when downloading from +https URLs. + + (5) (2) Depending on your platform, this may require root or +Administrator access. *note pip: 2b. is currently considering changing +this by making user installs the default behavior +(https://github.com/pypa/pip/issues/1668). + + +File: pythonpackagingguide.info, Node: Ensure pip setuptools and wheel are up to date, Next: Optionally create a virtual environment, Prev: Ensure you can run pip from the command line, Up: Requirements for Installing Packages + +2.1.1.3 Ensure pip, setuptools, and wheel are up to date +........................................................ + +While ‘pip’ alone is sufficient to install from pre-built binary +archives, up to date copies of the ‘setuptools’ and ‘wheel’ projects are +useful to ensure you can also install from source archives: + + python -m pip install --upgrade pip setuptools wheel + + +File: pythonpackagingguide.info, Node: Optionally create a virtual environment, Prev: Ensure pip setuptools and wheel are up to date, Up: Requirements for Installing Packages + +2.1.1.4 Optionally, create a virtual environment +................................................ + +See *note section below: 31. for details, but here’s the basic venv(1) +(2) command to use on a typical Linux system: + + python3 -m venv tutorial_env + source tutorial_env/bin/activate + +This will create a new virtual environment in the ‘tutorial_env’ +subdirectory, and configure the current shell to use it as the default +‘python’ environment. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/venv.html + + (2) (3) Beginning with Python 3.4, ‘venv’ (a stdlib alternative to +*note virtualenv: 32.) will create virtualenv environments with ‘pip’ +pre-installed, thereby making it an equal alternative to *note +virtualenv: 32. + + +File: pythonpackagingguide.info, Node: Creating Virtual Environments, Next: Use pip for Installing, Prev: Requirements for Installing Packages, Up: Installing Packages + +2.1.2 Creating Virtual Environments +----------------------------------- + +Python “Virtual Environments” allow Python *note packages: b. to be +installed in an isolated location for a particular application, rather +than being installed globally. If you are looking to safely install +global command line tools, see *note Installing stand alone command line +tools: 34. + +Imagine you have an application that needs version 1 of LibFoo, but +another application requires version 2. How can you use both these +applications? If you install everything into +/usr/lib/python3.6/site-packages (or whatever your platform’s standard +location is), it’s easy to end up in a situation where you +unintentionally upgrade an application that shouldn’t be upgraded. + +Or more generally, what if you want to install an application and leave +it be? If an application works, any change in its libraries or the +versions of those libraries can break the application. + +Also, what if you can’t install *note packages: b. into the global +site-packages directory? For instance, on a shared host. + +In all these cases, virtual environments can help you. They have their +own installation directories and they don’t share libraries with other +virtual environments. + +Currently, there are two common tools for creating Python virtual +environments: + + * venv(1) is available by default in Python 3.3 and later, and + installs *note pip: 2b. and *note setuptools: 2d. into created + virtual environments in Python 3.4 and later. + + * *note virtualenv: 32. needs to be installed separately, but + supports Python 2.7+ and Python 3.3+, and *note pip: 2b, *note + setuptools: 2d. and *note wheel: 2e. are always installed into + created virtual environments by default (regardless of Python + version). + +The basic usage is like so: + +Using venv(2): + + python3 -m venv + source /bin/activate + +Using *note virtualenv: 32.: + + virtualenv + source /bin/activate + +For more information, see the venv(3) docs or the virtualenv(4) docs. + +The use of ‘source’ under Unix shells ensures that the virtual +environment’s variables are set within the current shell, and not in a +subprocess (which then disappears, having no useful effect). + +In both of the above cases, Windows users should _not_ use the ‘source’ +command, but should rather run the ‘activate’ script directly from the +command shell like so: + + \Scripts\activate + +Managing multiple virtual environments directly can become tedious, so +the *note dependency management tutorial: 35. introduces a higher level +tool, *note Pipenv: 36, that automatically manages a separate virtual +environment for each project and application that you work on. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/venv.html + + (2) https://docs.python.org/3/library/venv.html + + (3) https://docs.python.org/3/library/venv.html + + (4) http://virtualenv.pypa.io + + +File: pythonpackagingguide.info, Node: Use pip for Installing, Next: Installing from PyPI, Prev: Creating Virtual Environments, Up: Installing Packages + +2.1.3 Use pip for Installing +---------------------------- + +*note pip: 2b. is the recommended installer. Below, we’ll cover the +most common usage scenarios. For more detail, see the pip docs(1), +which includes a complete Reference Guide(2). + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io + + (2) https://pip.pypa.io/en/latest/reference/index.html + + +File: pythonpackagingguide.info, Node: Installing from PyPI, Next: Source Distributions vs Wheels, Prev: Use pip for Installing, Up: Installing Packages + +2.1.4 Installing from PyPI +-------------------------- + +The most common usage of *note pip: 2b. is to install from the *note +Python Package Index: 39. using a *note requirement specifier: 3a. +Generally speaking, a requirement specifier is composed of a project +name followed by an optional *note version specifier: 3b. PEP 440(1) +contains a full specification(2) of the currently supported specifiers. +Below are some examples. + +To install the latest version of “SomeProject”: + + pip install "SomeProject" + +To install a specific version: + + pip install "SomeProject==1.4" + +To install greater than or equal to one version and less than another: + + pip install "SomeProject>=1,<2" + +To install a version that’s “compatible”(3) with a certain version: (4) + + pip install "SomeProject~=1.4.2" + +In this case, this means to install any version “==1.4.*” version that’s +also “>=1.4.2”. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + (2) https://www.python.org/dev/peps/pep-0440#version-specifiers + + (3) https://www.python.org/dev/peps/pep-0440#compatible-release + + (4) (4) The compatible release specifier was accepted in PEP 440 +(https://www.python.org/dev/peps/pep-0440) and support was released in +*note setuptools: 2d. v8.0 and *note pip: 2b. v6.0 + + +File: pythonpackagingguide.info, Node: Source Distributions vs Wheels, Next: Upgrading packages, Prev: Installing from PyPI, Up: Installing Packages + +2.1.5 Source Distributions vs Wheels +------------------------------------ + +*note pip: 2b. can install from either *note Source Distributions +(sdist): 3d. or *note Wheels: d, but if both are present on PyPI, pip +will prefer a compatible *note wheel: d. + +*note Wheels: d. are a pre-built *note distribution: b. format that +provides faster installation compared to *note Source Distributions +(sdist): 3d, especially when a project contains compiled extensions. + +If *note pip: 2b. does not find a wheel to install, it will locally +build a wheel and cache it for future installs, instead of rebuilding +the source distribution in the future. + + +File: pythonpackagingguide.info, Node: Upgrading packages, Next: Installing to the User Site, Prev: Source Distributions vs Wheels, Up: Installing Packages + +2.1.6 Upgrading packages +------------------------ + +Upgrade an already installed ‘SomeProject’ to the latest from PyPI. + + pip install --upgrade SomeProject + + +File: pythonpackagingguide.info, Node: Installing to the User Site, Next: Requirements files, Prev: Upgrading packages, Up: Installing Packages + +2.1.7 Installing to the User Site +--------------------------------- + +To install *note packages: b. that are isolated to the current user, use +the ‘--user’ flag: + + pip install --user SomeProject + +For more information see the User Installs(1) section from the pip docs. + +Note that the ‘--user’ flag has no effect when inside a virtual +environment - all installation commands will affect the virtual +environment. + +If ‘SomeProject’ defines any command-line scripts or console entry +points, ‘--user’ will cause them to be installed inside the user +base(2)’s binary directory, which may or may not already be present in +your shell’s ‘PATH’. (Starting in version 10, pip displays a warning +when installing any scripts to a directory outside ‘PATH’.) If the +scripts are not available in your shell after installation, you’ll need +to add the directory to your ‘PATH’: + + - On Linux and macOS you can find the user base binary directory by + running ‘python -m site --user-base’ and adding ‘bin’ to the end. + For example, this will typically print ‘~/.local’ (with ‘~’ + expanded to the absolute path to your home directory) so you’ll + need to add ‘~/.local/bin’ to your ‘PATH’. You can set your ‘PATH’ + permanently by modifying ~/.profile(3). + + - On Windows you can find the user base binary directory by running + ‘py -m site --user-site’ and replacing ‘site-packages’ with + ‘Scripts’. For example, this could return + ‘C:\Users\Username\AppData\Roaming\Python36\site-packages’ so you + would need to set your ‘PATH’ to include + ‘C:\Users\Username\AppData\Roaming\Python36\Scripts’. You can set + your user ‘PATH’ permanently in the Control Panel(4). You may need + to log out for the ‘PATH’ changes to take effect. + + ---------- Footnotes ---------- + + (1) +https://pip.readthedocs.io/en/latest/user_guide.html#user-installs + + (2) https://docs.python.org/3/library/site.html#site.USER_BASE + + (3) https://stackoverflow.com/a/14638025 + + (4) +https://msdn.microsoft.com/en-us/library/windows/desktop/bb776899(v=vs.85).aspx + + +File: pythonpackagingguide.info, Node: Requirements files, Next: Installing from VCS, Prev: Installing to the User Site, Up: Installing Packages + +2.1.8 Requirements files +------------------------ + +Install a list of requirements specified in a Requirements File(1). + + pip install -r requirements.txt + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + +File: pythonpackagingguide.info, Node: Installing from VCS, Next: Installing from other Indexes, Prev: Requirements files, Up: Installing Packages + +2.1.9 Installing from VCS +------------------------- + +Install a project from VCS in “editable” mode. For a full breakdown of +the syntax, see pip’s section on VCS Support(1). + + pip install -e git+https://git.repo/some_pkg.git#egg=SomeProject # from git + pip install -e hg+https://hg.repo/some_pkg#egg=SomeProject # from mercurial + pip install -e svn+svn://svn.repo/some_pkg/trunk/#egg=SomeProject # from svn + pip install -e git+https://git.repo/some_pkg.git@feature#egg=SomeProject # from a branch + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/reference/pip_install/#vcs-support + + +File: pythonpackagingguide.info, Node: Installing from other Indexes, Next: Installing from a local src tree, Prev: Installing from VCS, Up: Installing Packages + +2.1.10 Installing from other Indexes +------------------------------------ + +Install from an alternate index + + pip install --index-url http://my.package.repo/simple/ SomeProject + +Search an additional index during install, in addition to *note PyPI: +39. + + pip install --extra-index-url http://my.package.repo/simple SomeProject + + +File: pythonpackagingguide.info, Node: Installing from a local src tree, Next: Installing from local archives, Prev: Installing from other Indexes, Up: Installing Packages + +2.1.11 Installing from a local src tree +--------------------------------------- + +Installing from local src in Development Mode(1), i.e. in such a way +that the project appears to be installed, but yet is still editable from +the src tree. + + pip install -e + +You can also install normally from src + + pip install + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode + + +File: pythonpackagingguide.info, Node: Installing from local archives, Next: Installing from other sources, Prev: Installing from a local src tree, Up: Installing Packages + +2.1.12 Installing from local archives +------------------------------------- + +Install a particular source archive file. + + pip install ./downloads/SomeProject-1.0.4.tar.gz + +Install from a local directory containing archives (and don’t check +*note PyPI: 39.) + + pip install --no-index --find-links=file:///local/dir/ SomeProject + pip install --no-index --find-links=/local/dir/ SomeProject + pip install --no-index --find-links=relative/dir/ SomeProject + + +File: pythonpackagingguide.info, Node: Installing from other sources, Next: Installing Prereleases, Prev: Installing from local archives, Up: Installing Packages + +2.1.13 Installing from other sources +------------------------------------ + +To install from other data sources (for example Amazon S3 storage) you +can create a helper application that presents the data in a PEP 503(1) +compliant index format, and use the ‘--extra-index-url’ flag to direct +pip to use that index. + + ./s3helper --port=7777 + pip install --extra-index-url http://localhost:7777 SomeProject + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0503 + + +File: pythonpackagingguide.info, Node: Installing Prereleases, Next: Installing Setuptools “Extras”, Prev: Installing from other sources, Up: Installing Packages + +2.1.14 Installing Prereleases +----------------------------- + +Find pre-release and development versions, in addition to stable +versions. By default, pip only finds stable versions. + + pip install --pre SomeProject + + +File: pythonpackagingguide.info, Node: Installing Setuptools “Extras”, Prev: Installing Prereleases, Up: Installing Packages + +2.1.15 Installing Setuptools “Extras” +------------------------------------- + +Install setuptools extras(1). + + $ pip install SomePackage[PDF] + $ pip install SomePackage[PDF]==3.0 + $ pip install -e .[PDF]==3.0 # editable project in current directory + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies + + +File: pythonpackagingguide.info, Node: Managing Application Dependencies, Next: Packaging Python Projects, Prev: Installing Packages, Up: Tutorials + +2.2 Managing Application Dependencies +===================================== + +The *note package installation tutorial: 25. covered the basics of +getting set up to install and update Python packages. + +However, running these commands interactively can get tedious even for +your own personal projects, and things get even more difficult when +trying to set up development environments automatically for projects +with multiple contributors. + +This tutorial walks you through the use of *note Pipenv: 36. to manage +dependencies for an application. It will show you how to install and +use the necessary tools and make strong recommendations on best +practices. + +Keep in mind that Python is used for a great many different purposes, +and precisely how you want to manage your dependencies may change based +on how you decide to publish your software. The guidance presented here +is most directly applicable to the development and deployment of network +services (including web applications), but is also very well suited to +managing development and testing environments for any kind of project. + +Developers of Python libraries, or of applications that support +distribution as Python libraries, should also consider the poetry(1) +project as an alternative dependency management solution. + +* Menu: + +* Installing Pipenv:: +* Installing packages for your project:: +* Using installed packages:: +* Next steps:: +* Other Tools for Application Dependency Management:: + + ---------- Footnotes ---------- + + (1) https://github.com/python-poetry/poetry + + +File: pythonpackagingguide.info, Node: Installing Pipenv, Next: Installing packages for your project, Up: Managing Application Dependencies + +2.2.1 Installing Pipenv +----------------------- + +*note Pipenv: 36. is a dependency manager for Python projects. If +you’re familiar with Node.js’ npm(1) or Ruby’s bundler(2), it is similar +in spirit to those tools. While *note pip: 2b. alone is often +sufficient for personal use, Pipenv is recommended for collaborative +projects as it’s a higher-level tool that simplifies dependency +management for common use cases. + +Use ‘pip’ to install Pipenv: + + pip install --user pipenv + Note: This does a user installation(3) to prevent breaking any + system-wide packages. If ‘pipenv’ isn’t available in your shell + after installation, you’ll need to add the user base(4)’s binary + directory to your ‘PATH’. See *note Installing to the User Site: + 40. for more information. + + ---------- Footnotes ---------- + + (1) https://www.npmjs.com/ + + (2) http://bundler.io/ + + (3) https://pip.pypa.io/en/stable/user_guide/#user-installs + + (4) https://docs.python.org/3/library/site.html#site.USER_BASE + + +File: pythonpackagingguide.info, Node: Installing packages for your project, Next: Using installed packages, Prev: Installing Pipenv, Up: Managing Application Dependencies + +2.2.2 Installing packages for your project +------------------------------------------ + +Pipenv manages dependencies on a per-project basis. To install +packages, change into your project’s directory (or just an empty +directory for this tutorial) and run: + + cd myproject + pipenv install requests + +Pipenv will install the Requests(1) library and create a ‘Pipfile’ for +you in your project’s directory. The *note Pipfile: 51. is used to +track which dependencies your project needs in case you need to +re-install them, such as when you share your project with others. You +should get output similar to this (although the exact paths shown will +vary): + + Creating a Pipfile for this project... + Creating a virtualenv for this project... + Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6' + New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6 + Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python + Installing setuptools, pip, wheel...done. + + Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd + Installing requests... + Collecting requests + Using cached requests-2.18.4-py2.py3-none-any.whl + Collecting idna<2.7,>=2.5 (from requests) + Using cached idna-2.6-py2.py3-none-any.whl + Collecting urllib3<1.23,>=1.21.1 (from requests) + Using cached urllib3-1.22-py2.py3-none-any.whl + Collecting chardet<3.1.0,>=3.0.2 (from requests) + Using cached chardet-3.0.4-py2.py3-none-any.whl + Collecting certifi>=2017.4.17 (from requests) + Using cached certifi-2017.7.27.1-py2.py3-none-any.whl + Installing collected packages: idna, urllib3, chardet, certifi, requests + Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22 + + Adding requests to Pipfile's [packages]... + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/requests/ + + +File: pythonpackagingguide.info, Node: Using installed packages, Next: Next steps, Prev: Installing packages for your project, Up: Managing Application Dependencies + +2.2.3 Using installed packages +------------------------------ + +Now that Requests is installed you can create a simple ‘main.py’ file to +use it: + + import requests + + response = requests.get('https://httpbin.org/ip') + + print('Your IP is {0}'.format(response.json()['origin'])) + +Then you can run this script using ‘pipenv run’: + + pipenv run python main.py + +You should get output similar to this: + + Your IP is 8.8.8.8 + +Using ‘pipenv run’ ensures that your installed packages are available to +your script. It’s also possible to spawn a new shell that ensures all +commands have access to your installed packages with ‘pipenv shell’. + + +File: pythonpackagingguide.info, Node: Next steps, Next: Other Tools for Application Dependency Management, Prev: Using installed packages, Up: Managing Application Dependencies + +2.2.4 Next steps +---------------- + +Congratulations, you now know how to effectively manage dependencies and +development environments on a collaborative Python project! ✨ 🍰 ✨ + +If you’re interested in creating and distributing your own Python +packages, see the *note tutorial on packaging and distributing packages: +55. + +Note that when your application includes definitions of Python source +packages, they (and their dependencies) can be added to your ‘pipenv’ +environment with ‘pipenv install -e ’ +(e.g. ‘pipenv install -e .’ or ‘pipenv install -e src’). + + +File: pythonpackagingguide.info, Node: Other Tools for Application Dependency Management, Prev: Next steps, Up: Managing Application Dependencies + +2.2.5 Other Tools for Application Dependency Management +------------------------------------------------------- + +If you find this particular approach to managing application +dependencies isn’t working well for you or your use case, you may want +to explore these other tools and techniques to see if one of them is a +better fit: + + * poetry(1) for a tool comparable in scope to ‘pipenv’ that focuses + more directly on use cases where the repository being managed is + structured as a Python project with a valid ‘pyproject.toml’ file + (by contrast, ‘pipenv’ explicitly avoids making the assumption that + the application being worked on that’s depending on components from + PyPI will itself support distribution as a ‘pip’-installable Python + package). + + * hatch(2) for opinionated coverage of even more steps in the project + management workflow (such as incrementing versions, tagging + releases, and creating new skeleton projects from project + templates) + + * pip-tools(3) to build your own custom workflow from lower level + pieces like ‘pip-compile’ and ‘pip-sync’ + + * micropipenv(4) is a lightweight wrapper for pip to support + requirements.txt, Pipenv and Poetry lock files or converting them + to pip-tools compatible output. Designed for containerized Python + applications but not limited to them. + + ---------- Footnotes ---------- + + (1) https://github.com/python-poetry/poetry + + (2) https://github.com/ofek/hatch + + (3) https://github.com/jazzband/pip-tools + + (4) https://github.com/thoth-station/micropipenv + + +File: pythonpackagingguide.info, Node: Packaging Python Projects, Next: Creating Documentation, Prev: Managing Application Dependencies, Up: Tutorials + +2.3 Packaging Python Projects +============================= + +This tutorial walks you through how to package a simple Python project. +It will show you how to add the necessary files and structure to create +the package, how to build the package, and how to upload it to the +Python Package Index. + +* Menu: + +* A simple project:: +* Creating the package files:: +* Creating a test folder:: +* Creating setup.py: Creating setup py. +* Creating README.md: Creating README md. +* Creating a LICENSE:: +* Generating distribution archives:: +* Uploading the distribution archives:: +* Installing your newly uploaded package:: +* Next steps: Next steps<2>. + + +File: pythonpackagingguide.info, Node: A simple project, Next: Creating the package files, Up: Packaging Python Projects + +2.3.1 A simple project +---------------------- + +This tutorial uses a simple project named ‘example_pkg’. If you are +unfamiliar with Python’s modules and *note import packages: a, take a +few minutes to read over the Python documentation for packages and +modules(1). Even if you already have a project that you want to package +up, we recommend following this tutorial as-is using this example +package and then trying with your own package. + +To create this project locally, create the following file structure: + + packaging_tutorial + └── example_pkg +    └── __init__.py + +Once you create this structure, you’ll want to run all of the commands +in this tutorial within the top-level folder - so be sure to ‘cd +packaging_tutorial’. + +‘example_pkg/__init__.py’ is required to import the directory as a +package, and can simply be an empty file. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/tutorial/modules.html#packages + + +File: pythonpackagingguide.info, Node: Creating the package files, Next: Creating a test folder, Prev: A simple project, Up: Packaging Python Projects + +2.3.2 Creating the package files +-------------------------------- + +You will now create a handful of files to package up this project and +prepare it for distribution. Create the new files listed below and +place them in the project’s root directory - you will add content to +them in the following steps. + + packaging_tutorial + ├── LICENSE + ├── README.md + ├── example_pkg + │   └── __init__.py + ├── setup.py + └── tests + + +File: pythonpackagingguide.info, Node: Creating a test folder, Next: Creating setup py, Prev: Creating the package files, Up: Packaging Python Projects + +2.3.3 Creating a test folder +---------------------------- + +‘tests/’ is a placeholder for unit test files. Leave it empty for now. + + +File: pythonpackagingguide.info, Node: Creating setup py, Next: Creating README md, Prev: Creating a test folder, Up: Packaging Python Projects + +2.3.4 Creating setup.py +----------------------- + +‘setup.py’ is the build script for *note setuptools: 2d. It tells +setuptools about your package (such as the name and version) as well as +which code files to include. + +Open ‘setup.py’ and enter the following content. Update the package +name to include your username (for example, ‘example-pkg-theacodes’), +this ensures that you have a unique package name and that your package +doesn’t conflict with packages uploaded by other people following this +tutorial. + + import setuptools + + with open("README.md", "r") as fh: + long_description = fh.read() + + setuptools.setup( + name="example-pkg-YOUR-USERNAME-HERE", # Replace with your own username + version="0.0.1", + author="Example Author", + author_email="author@example.com", + description="A small example package", + long_description=long_description, + long_description_content_type="text/markdown", + url="https://github.com/pypa/sampleproject", + packages=setuptools.find_packages(), + classifiers=[ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + ], + python_requires='>=3.6', + ) + +‘setup()’ takes several arguments. This example package uses a +relatively minimal set: + + - ‘name’ is the `distribution name' of your package. This can be any + name as long as only contains letters, numbers, ‘_’ , and ‘-’. It + also must not already be taken on pypi.org. `Be sure to update + this with your username,' as this ensures you won’t try to upload a + package with the same name as one which already exists when you + upload the package. + + - ‘version’ is the package version see PEP 440(1) for more details on + versions. + + - ‘author’ and ‘author_email’ are used to identify the author of the + package. + + - ‘description’ is a short, one-sentence summary of the package. + + - ‘long_description’ is a detailed description of the package. This + is shown on the package detail package on the Python Package Index. + In this case, the long description is loaded from ‘README.md’ which + is a common pattern. + + - ‘long_description_content_type’ tells the index what type of markup + is used for the long description. In this case, it’s Markdown. + + - ‘url’ is the URL for the homepage of the project. For many + projects, this will just be a link to GitHub, GitLab, Bitbucket, or + similar code hosting service. + + - ‘packages’ is a list of all Python *note import packages: a. that + should be included in the *note Distribution Package: b. Instead + of listing each package manually, we can use ‘find_packages()’ to + automatically discover all packages and subpackages. In this case, + the list of packages will be ‘example_pkg’ as that’s the only + package present. + + - ‘classifiers’ gives the index and *note pip: 2b. some additional + metadata about your package. In this case, the package is only + compatible with Python 3, is licensed under the MIT license, and is + OS-independent. You should always include at least which + version(s) of Python your package works on, which license your + package is available under, and which operating systems your + package will work on. For a complete list of classifiers, see + ‘https://pypi.org/classifiers/’. + +There are many more than the ones mentioned here. See *note Packaging +and distributing projects: 6. for more details. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + +File: pythonpackagingguide.info, Node: Creating README md, Next: Creating a LICENSE, Prev: Creating setup py, Up: Packaging Python Projects + +2.3.5 Creating README.md +------------------------ + +Open ‘README.md’ and enter the following content. You can customize +this if you’d like. + + # Example Package + + This is a simple example package. You can use + [Github-flavored Markdown](https://guides.github.com/features/mastering-markdown/) + to write your content. + + +File: pythonpackagingguide.info, Node: Creating a LICENSE, Next: Generating distribution archives, Prev: Creating README md, Up: Packaging Python Projects + +2.3.6 Creating a LICENSE +------------------------ + +It’s important for every package uploaded to the Python Package Index to +include a license. This tells users who install your package the terms +under which they can use your package. For help picking a license, see +‘https://choosealicense.com/’. Once you have chosen a license, open +‘LICENSE’ and enter the license text. For example, if you had chosen +the MIT license: + + Copyright (c) 2018 The Python Packaging Authority + + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in all + copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE. + + +File: pythonpackagingguide.info, Node: Generating distribution archives, Next: Uploading the distribution archives, Prev: Creating a LICENSE, Up: Packaging Python Projects + +2.3.7 Generating distribution archives +-------------------------------------- + +The next step is to generate *note distribution packages: b. for the +package. These are archives that are uploaded to the Package Index and +can be installed by *note pip: 2b. + +Make sure you have the latest versions of ‘setuptools’ and ‘wheel’ +installed: + + python3 -m pip install --user --upgrade setuptools wheel + + Tip: IF you have trouble installing these, see the *note Installing + Packages: 23. tutorial. + +Now run this command from the same directory where ‘setup.py’ is +located: + + python3 setup.py sdist bdist_wheel + +This command should output a lot of text and once completed should +generate two files in the ‘dist’ directory: + + dist/ + example_pkg_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl + example_pkg_YOUR_USERNAME_HERE-0.0.1.tar.gz + + Note: If you run into trouble here, please copy the output and file + an issue over on packaging problems(1) and we’ll do our best to + help you! + +The ‘tar.gz’ file is a *note Source Archive: 62. whereas the ‘.whl’ file +is a *note Built Distribution: 63. Newer *note pip: 2b. versions +preferentially install built distributions, but will fall back to source +archives if needed. You should always upload a source archive and +provide built archives for the platforms your project is compatible +with. In this case, our example package is compatible with Python on +any platform so only one built distribution is needed. + + ---------- Footnotes ---------- + + (1) +https://github.com/pypa/packaging-problems/issues/new?title=Trouble+following+packaging+libraries+tutorial + + +File: pythonpackagingguide.info, Node: Uploading the distribution archives, Next: Installing your newly uploaded package, Prev: Generating distribution archives, Up: Packaging Python Projects + +2.3.8 Uploading the distribution archives +----------------------------------------- + +Finally, it’s time to upload your package to the Python Package Index! + +The first thing you’ll need to do is register an account on ‘Test PyPI’. +Test PyPI is a separate instance of the package index intended for +testing and experimentation. It’s great for things like this tutorial +where we don’t necessarily want to upload to the real index. To +register an account, go to ‘https://test.pypi.org/account/register/’ and +complete the steps on that page. You will also need to verify your +email address before you’re able to upload any packages. For more +details on Test PyPI, see *note Using TestPyPI: 65. + +Now you’ll create a PyPI API token(1) so you will be able to securely +upload your project. + +Go to ‘https://test.pypi.org/manage/account/#api-tokens’ and create a +new API token(2); don’t limit its scope to a particular project, since +you are creating a new project. + +`Don’t close the page until you have copied and saved the token — you +won’t see that token again.' + +Now that you are registered, you can use *note twine: 66. to upload the +distribution packages. You’ll need to install Twine: + + python3 -m pip install --user --upgrade twine + +Once installed, run Twine to upload all of the archives under ‘dist’: + + python3 -m twine upload --repository testpypi dist/* + +You will be prompted for a username and password. For the username, use +‘__token__’. For the password, use the token value, including the +‘pypi-’ prefix. + +After the command completes, you should see output similar to this: + + Uploading distributions to https://test.pypi.org/legacy/ + Enter your username: [your username] + Enter your password: + Uploading example_pkg_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl + 100%|█████████████████████| 4.65k/4.65k [00:01<00:00, 2.88kB/s] + Uploading example_pkg_YOUR_USERNAME_HERE-0.0.1.tar.gz + 100%|█████████████████████| 4.25k/4.25k [00:01<00:00, 3.05kB/s] + +Once uploaded your package should be viewable on TestPyPI, for example, +‘https://test.pypi.org/project/example-pkg-YOUR-USERNAME-HERE’ + + ---------- Footnotes ---------- + + (1) https://test.pypi.org/help/#apitoken + + (2) https://test.pypi.org/help/#apitoken + + +File: pythonpackagingguide.info, Node: Installing your newly uploaded package, Next: Next steps<2>, Prev: Uploading the distribution archives, Up: Packaging Python Projects + +2.3.9 Installing your newly uploaded package +-------------------------------------------- + +You can use *note pip: 2b. to install your package and verify that it +works. Create a new *note virtualenv: 32. (see *note Installing +Packages: 23. for detailed instructions) and install your package from +TestPyPI: + + python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps example-pkg-YOUR-USERNAME-HERE + +Make sure to specify your username in the package name! + +pip should install the package from Test PyPI and the output should look +something like this: + + Collecting example-pkg-YOUR-USERNAME-HERE + Downloading https://test-files.pythonhosted.org/packages/.../example-pkg-YOUR-USERNAME-HERE-0.0.1-py3-none-any.whl + Installing collected packages: example-pkg-YOUR-USERNAME-HERE + Successfully installed example-pkg-YOUR-USERNAME-HERE-0.0.1 + + Note: This example uses ‘--index-url’ flag to specify TestPyPI + instead of live PyPI. Additionally, it specifies ‘--no-deps’. + Since TestPyPI doesn’t have the same packages as the live PyPI, + it’s possible that attempting to install dependencies may fail or + install something unexpected. While our example package doesn’t + have any dependencies, it’s a good practice to avoid installing + dependencies when using TestPyPI. + +You can test that it was installed correctly by importing the package. +Run the Python interpreter (make sure you’re still in your virtualenv): + + python + +and from the interpreter shell import the package: + + >>> import example_pkg + +Note that the *note Import Package: a. is ‘example_pkg’ regardless of +what name you gave your *note Distribution Package: b. in ‘setup.py’ (in +this case, ‘example-pkg-YOUR-USERNAME-HERE’). + + +File: pythonpackagingguide.info, Node: Next steps<2>, Prev: Installing your newly uploaded package, Up: Packaging Python Projects + +2.3.10 Next steps +----------------- + +`Congratulations, you’ve packaged and distributed a Python project!' ✨ +🍰 ✨ + +Keep in mind that this tutorial showed you how to upload your package to +Test PyPI, which isn’t a permanent storage. The Test system +occasionally deletes packages and accounts. It is best to use Test PyPI +for testing and experiments like this tutorial. + +When you are ready to upload a real package to the Python Package Index +you can do much the same as you did in this tutorial, but with these +important differences: + + * Choose a memorable and unique name for your package. You don’t + have to append your username as you did in the tutorial. + + * Register an account on ‘https://pypi.org’ - note that these are two + separate servers and the login details from the test server are not + shared with the main server. + + * Use ‘twine upload dist/*’ to upload your package and enter your + credentials for the account you registered on the real PyPI. Now + that you’re uploading the package in production, you don’t need to + specify ‘--repository’; the package will upload to + ‘https://pypi.org/’ by default. + + * Install your package from the real PyPI using ‘pip install + [your-package]’. + +At this point if you want to read more on packaging Python libraries +here are some things you can do: + + * Read more about using *note setuptools: 2d. to package libraries in + *note Packaging and distributing projects: 6. + + * Read about *note Packaging binary extensions: 69. + + * Consider alternatives to *note setuptools: 2d. such as *note flit: + 6a, hatch(1), and poetry(2). + + ---------- Footnotes ---------- + + (1) https://github.com/ofek/hatch + + (2) https://github.com/python-poetry/poetry + + +File: pythonpackagingguide.info, Node: Creating Documentation, Prev: Packaging Python Projects, Up: Tutorials + +2.4 Creating Documentation +========================== + +This section covers the basics of how to create documentation using +Sphinx(1) and host the documentation for free in Read The Docs(2). + +* Menu: + +* Installing Sphinx:: +* Getting Started With Sphinx:: +* Other Sources:: + + ---------- Footnotes ---------- + + (1) http://sphinx-doc.org/ + + (2) https://readthedocs.org/ + + +File: pythonpackagingguide.info, Node: Installing Sphinx, Next: Getting Started With Sphinx, Up: Creating Documentation + +2.4.1 Installing Sphinx +----------------------- + +Use ‘pip’ to install Sphinx: + + pip install -U sphinx + +For other installation methods, see this installation guide(1) by +Sphinx. + + ---------- Footnotes ---------- + + (1) http://www.sphinx-doc.org/en/master/usage/installation.html + + +File: pythonpackagingguide.info, Node: Getting Started With Sphinx, Next: Other Sources, Prev: Installing Sphinx, Up: Creating Documentation + +2.4.2 Getting Started With Sphinx +--------------------------------- + +Create a ‘docs’ directory inside your project to hold your +documentation: + + cd /path/to/project + mkdir docs + +Run ‘sphinx-quickstart’ inside the ‘docs’ directory: + + cd docs + sphinx-quickstart + +This sets up a source directory, walks you through some basic +configurations, and creates an ‘index.rst’ file as well as a ‘conf.py’ +file. + +You can add some information about your project in ‘index.rst’, then +build them: + + make html + +For more details on the build process, see this guide(1) by Read The +Docs. + + ---------- Footnotes ---------- + + (1) https://docs.readthedocs.io/en/latest/intro/import-guide.html + + +File: pythonpackagingguide.info, Node: Other Sources, Prev: Getting Started With Sphinx, Up: Creating Documentation + +2.4.3 Other Sources +------------------- + +For a more detailed guide on how to use Sphinx and reStructuredText, +please see this documentation tutorial(1) on Hitchhiker’s Guide to +Python. + + ---------- Footnotes ---------- + + (1) https://docs.python-guide.org/writing/documentation/ + + +File: pythonpackagingguide.info, Node: Guides, Next: Discussions, Prev: Tutorials, Up: Top + +3 Guides +******** + +`Guides' are focused on accomplishing a specific task and assume that +you are already familiar with the basics of Python packaging. If you’re +looking for an introduction to packaging, see *note Tutorials: 20. + +* Menu: + +* Tool recommendations:: +* Installing packages using pip and virtual environments:: +* Installing stand alone command line tools:: +* Installing pip/setuptools/wheel with Linux Package Managers:: +* Installing scientific packages:: +* Multi-version installs:: +* Packaging and distributing projects:: +* Including files in source distributions with MANIFEST.in: Including files in source distributions with MANIFEST in. +* Single-sourcing the package version:: +* Supporting multiple Python versions:: +* Dropping support for older Python versions:: +* Packaging binary extensions:: +* Supporting Windows using Appveyor:: +* Packaging namespace packages:: +* Creating and discovering plugins:: +* Analyzing PyPI package downloads:: +* Package index mirrors and caches:: +* Hosting your own simple repository:: +* Migrating to PyPI.org: Migrating to PyPI org. +* Using TestPyPI: Using TestPyPI<2>. +* Making a PyPI-friendly README:: +* Publishing package distribution releases using GitHub Actions CI/CD workflows:: + + +File: pythonpackagingguide.info, Node: Tool recommendations, Next: Installing packages using pip and virtual environments, Up: Guides + +3.1 Tool recommendations +======================== + +If you’re familiar with Python packaging and installation, and just want +to know what tools are currently recommended, then here it is. + +* Menu: + +* Application dependency management:: +* Installation tool recommendations:: +* Packaging tool recommendations:: +* Publishing platform migration:: + + +File: pythonpackagingguide.info, Node: Application dependency management, Next: Installation tool recommendations, Up: Tool recommendations + +3.1.1 Application dependency management +--------------------------------------- + +Use *note Pipenv: 36. to manage library dependencies when developing +Python applications. See *note Managing Application Dependencies: 4a. +for more details on using ‘pipenv’. + +When ‘pipenv’ does not meet your use case, consider other tools like: + + * *note pip: 2b. + + * pip-tools(1) + + * Poetry(2) + + ---------- Footnotes ---------- + + (1) https://github.com/jazzband/pip-tools + + (2) https://python-poetry.org/ + + +File: pythonpackagingguide.info, Node: Installation tool recommendations, Next: Packaging tool recommendations, Prev: Application dependency management, Up: Tool recommendations + +3.1.2 Installation tool recommendations +--------------------------------------- + + * Use *note pip: 2b. to install Python *note packages: b. from *note + PyPI: 39. (1) (2) Depending on how *note pip: 2b. is installed, + you may need to also install *note wheel: 2e. to get the benefit of + wheel caching. (3) + + * Use *note virtualenv: 32, or venv(4) to isolate application + specific dependencies from a shared Python installation. (5) + + * If you’re looking for management of fully integrated cross-platform + software stacks, consider: + + * *note buildout: 80.: primarily focused on the web development + community + + * *note Spack: 81, *note Hashdist: 82, or *note conda: 83.: + primarily focused on the scientific community. + + ---------- Footnotes ---------- + + (1) (1) There are some cases where you might choose to use +‘easy_install’ (from *note setuptools: 2d.), e.g. if you need to +install from *note Eggs: 7c. (which pip doesn’t support). For a +detailed breakdown, see *note pip vs easy_install: 7d. + + (2) (2) The acceptance of PEP 453 +(https://www.python.org/dev/peps/pep-0453) means that *note pip: 2b. +will be available by default in most installations of Python 3.4 or +later. See the rationale section +(https://www.python.org/dev/peps/pep-0453#rationale) from PEP 453 +(https://www.python.org/dev/peps/pep-0453) as for why pip was chosen. + + (3) (3) get-pip.py +(https://pip.pypa.io/en/latest/installing/#get-pip) and *note +virtualenv: 32. install *note wheel: 2e, whereas *note ensurepip: 7e. +and *note venv: 7f. do not currently. Also, the common “python-pip” +package that’s found in various linux distros, does not depend on +“python-wheel” currently. + + (4) https://docs.python.org/3/library/venv.html + + (5) (4) Beginning with Python 3.4, ‘venv’ will create virtualenv +environments with ‘pip’ installed, thereby making it an equal +alternative to *note virtualenv: 32. However, using *note virtualenv: +32. will still be recommended for users that need cross-version +consistency. + + +File: pythonpackagingguide.info, Node: Packaging tool recommendations, Next: Publishing platform migration, Prev: Installation tool recommendations, Up: Tool recommendations + +3.1.3 Packaging tool recommendations +------------------------------------ + + * Use *note setuptools: 2d. to define projects and create *note + Source Distributions: 3d. (1) (2) + + * Use the ‘bdist_wheel’ *note setuptools: 2d. extension available + from the *note wheel project: 2e. to create *note wheels: d. This + is especially beneficial, if your project contains binary + extensions. + + * Use twine(3) for uploading distributions to *note PyPI: 39. + + ---------- Footnotes ---------- + + (1) (5) Although you can use pure ‘distutils’ for many projects, it +does not support defining dependencies on other projects and is missing +several convenience utilities for automatically populating distribution +metadata correctly that are provided by ‘setuptools’. Being outside the +standard library, ‘setuptools’ also offers a more consistent feature set +across different versions of Python, and (unlike ‘distutils’), recent +versions of ‘setuptools’ support all of the modern metadata fields +described in *note Core metadata specifications: 85. + +Even for projects that do choose to use ‘distutils’, when *note pip: 2b. +installs such projects directly from source (rather than installing from +a prebuilt *note wheel: d. file), it will actually build your project +using *note setuptools: 2d. instead. + + (2) (6) distribute (https://pypi.org/project/distribute) (a fork of +setuptools) was merged back into *note setuptools: 2d. in June 2013, +thereby making setuptools the default choice for packaging. + + (3) https://pypi.org/project/twine + + +File: pythonpackagingguide.info, Node: Publishing platform migration, Prev: Packaging tool recommendations, Up: Tool recommendations + +3.1.4 Publishing platform migration +----------------------------------- + +The original Python Package Index implementation (previously hosted at +pypi.python.org(1)) has been phased out in favour of an updated +implementation hosted at pypi.org(2). + +See *note Migrating to PyPI.org: 87. for more information on the status +of the migration, and what settings to change in your clients. + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) https://pypi.python.org + + (2) https://pypi.org + + +File: pythonpackagingguide.info, Node: Installing packages using pip and virtual environments, Next: Installing stand alone command line tools, Prev: Tool recommendations, Up: Guides + +3.2 Installing packages using pip and virtual environments +========================================================== + +This guide discusses how to install packages using *note pip: 2b. and a +virtual environment manager: either *note venv: 7f. for Python 3 or +*note virtualenv: 32. for Python 2. These are the lowest-level tools +for managing Python packages and are recommended if higher-level tools +do not suit your needs. + + Note: This doc uses the term `package' to refer to a *note + Distribution Package: b. which is different from a *note Import + Package: a. that which is used to import modules in your Python + source code. + +* Menu: + +* Installing pip:: +* Installing virtualenv:: +* Creating a virtual environment:: +* Activating a virtual environment:: +* Leaving the virtual environment:: +* Installing packages:: +* Installing specific versions:: +* Installing extras:: +* Installing from source:: +* Installing from version control systems:: +* Installing from local archives: Installing from local archives<2>. +* Using other package indexes:: +* Upgrading packages: Upgrading packages<2>. +* Using requirements files:: +* Freezing dependencies:: + + +File: pythonpackagingguide.info, Node: Installing pip, Next: Installing virtualenv, Up: Installing packages using pip and virtual environments + +3.2.1 Installing pip +-------------------- + +*note pip: 2b. is the reference Python package manager. It’s used to +install and update packages. You’ll need to make sure you have the +latest version of pip installed. + +* Menu: + +* Windows:: +* Linux and macOS:: + + +File: pythonpackagingguide.info, Node: Windows, Next: Linux and macOS, Up: Installing pip + +3.2.1.1 Windows +............... + +The Python installers for Windows include pip. You should be able to +access pip using: + + py -m pip --version + pip 9.0.1 from c:\python36\lib\site-packages (Python 3.6.1) + +You can make sure that pip is up-to-date by running: + + py -m pip install --upgrade pip + + +File: pythonpackagingguide.info, Node: Linux and macOS, Prev: Windows, Up: Installing pip + +3.2.1.2 Linux and macOS +....................... + +Debian and most other distributions include a python-pip(1) package, if +you want to use the Linux distribution-provided versions of pip see +*note Installing pip/setuptools/wheel with Linux Package Managers: 2c. + +You can also install pip yourself to ensure you have the latest version. +It’s recommended to use the system pip to bootstrap a user installation +of pip: + + python3 -m pip install --user --upgrade pip + +Afterwards, you should have the newest pip installed in your user site: + + python3 -m pip --version + pip 9.0.1 from $HOME/.local/lib/python3.6/site-packages (python 3.6) + + ---------- Footnotes ---------- + + (1) https://packages.debian.org/stable/python-pip + + +File: pythonpackagingguide.info, Node: Installing virtualenv, Next: Creating a virtual environment, Prev: Installing pip, Up: Installing packages using pip and virtual environments + +3.2.2 Installing virtualenv +--------------------------- + + Note: If you are using Python 3.3 or newer, the venv(1) module is + the preferred way to create and manage virtual environments. venv + is included in the Python standard library and requires no + additional installation. If you are using venv, you may skip this + section. + +*note virtualenv: 32. is used to manage Python packages for different +projects. Using virtualenv allows you to avoid installing Python +packages globally which could break system tools or other projects. You +can install virtualenv using pip. + +On macOS and Linux: + + python3 -m pip install --user virtualenv + +On Windows: + + py -m pip install --user virtualenv + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/venv.html#module-venv + + +File: pythonpackagingguide.info, Node: Creating a virtual environment, Next: Activating a virtual environment, Prev: Installing virtualenv, Up: Installing packages using pip and virtual environments + +3.2.3 Creating a virtual environment +------------------------------------ + +*note venv: 7f. (for Python 3) and *note virtualenv: 32. (for Python 2) +allow you to manage separate package installations for different +projects. They essentially allow you to create a “virtual” isolated +Python installation and install packages into that virtual installation. +When you switch projects, you can simply create a new virtual +environment and not have to worry about breaking the packages installed +in the other environments. It is always recommended to use a virtual +environment while developing Python applications. + +To create a virtual environment, go to your project’s directory and run +venv. If you are using Python 2, replace ‘venv’ with ‘virtualenv’ in +the below commands. + +On macOS and Linux: + + python3 -m venv env + +On Windows: + + py -m venv env + +The second argument is the location to create the virtual environment. +Generally, you can just create this in your project and call it ‘env’. + +venv will create a virtual Python installation in the ‘env’ folder. + + Note: You should exclude your virtual environment directory from + your version control system using ‘.gitignore’ or similar. + + +File: pythonpackagingguide.info, Node: Activating a virtual environment, Next: Leaving the virtual environment, Prev: Creating a virtual environment, Up: Installing packages using pip and virtual environments + +3.2.4 Activating a virtual environment +-------------------------------------- + +Before you can start installing or using packages in your virtual +environment you’ll need to `activate' it. Activating a virtual +environment will put the virtual environment-specific ‘python’ and ‘pip’ +executables into your shell’s ‘PATH’. + +On macOS and Linux: + + source env/bin/activate + +On Windows: + + .\env\Scripts\activate + +You can confirm you’re in the virtual environment by checking the +location of your Python interpreter, it should point to the ‘env’ +directory. + +On macOS and Linux: + + which python + .../env/bin/python + +On Windows: + + where python + .../env/bin/python.exe + +As long as your virtual environment is activated pip will install +packages into that specific environment and you’ll be able to import and +use packages in your Python application. + + +File: pythonpackagingguide.info, Node: Leaving the virtual environment, Next: Installing packages, Prev: Activating a virtual environment, Up: Installing packages using pip and virtual environments + +3.2.5 Leaving the virtual environment +------------------------------------- + +If you want to switch projects or otherwise leave your virtual +environment, simply run: + + deactivate + +If you want to re-enter the virtual environment just follow the same +instructions above about activating a virtual environment. There’s no +need to re-create the virtual environment. + + +File: pythonpackagingguide.info, Node: Installing packages, Next: Installing specific versions, Prev: Leaving the virtual environment, Up: Installing packages using pip and virtual environments + +3.2.6 Installing packages +------------------------- + +Now that you’re in your virtual environment you can install packages. +Let’s install the Requests(1) library from the *note Python Package +Index (PyPI): 39.: + + pip install requests + +pip should download requests and all of its dependencies and install +them: + + Collecting requests + Using cached requests-2.18.4-py2.py3-none-any.whl + Collecting chardet<3.1.0,>=3.0.2 (from requests) + Using cached chardet-3.0.4-py2.py3-none-any.whl + Collecting urllib3<1.23,>=1.21.1 (from requests) + Using cached urllib3-1.22-py2.py3-none-any.whl + Collecting certifi>=2017.4.17 (from requests) + Using cached certifi-2017.7.27.1-py2.py3-none-any.whl + Collecting idna<2.7,>=2.5 (from requests) + Using cached idna-2.6-py2.py3-none-any.whl + Installing collected packages: chardet, urllib3, certifi, idna, requests + Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22 + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/requests/ + + +File: pythonpackagingguide.info, Node: Installing specific versions, Next: Installing extras, Prev: Installing packages, Up: Installing packages using pip and virtual environments + +3.2.7 Installing specific versions +---------------------------------- + +pip allows you to specify which version of a package to install using +*note version specifiers: 3b. For example, to install a specific +version of ‘requests’: + + pip install requests==2.18.4 + +To install the latest ‘2.x’ release of requests: + + pip install requests>=2.0.0,<3.0.0 + +To install pre-release versions of packages, use the ‘--pre’ flag: + + pip install --pre requests + + +File: pythonpackagingguide.info, Node: Installing extras, Next: Installing from source, Prev: Installing specific versions, Up: Installing packages using pip and virtual environments + +3.2.8 Installing extras +----------------------- + +Some packages have optional extras(1). You can tell pip to install +these by specifying the extra in brackets: + + pip install requests[security] + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies + + +File: pythonpackagingguide.info, Node: Installing from source, Next: Installing from version control systems, Prev: Installing extras, Up: Installing packages using pip and virtual environments + +3.2.9 Installing from source +---------------------------- + +pip can install a package directly from source, for example: + + cd google-auth + pip install . + +Additionally, pip can install packages from source in development +mode(1), meaning that changes to the source directory will immediately +affect the installed package without needing to re-install: + + pip install --editable . + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode + + +File: pythonpackagingguide.info, Node: Installing from version control systems, Next: Installing from local archives<2>, Prev: Installing from source, Up: Installing packages using pip and virtual environments + +3.2.10 Installing from version control systems +---------------------------------------------- + +pip can install packages directly from their version control system. +For example, you can install directly from a git repository: + + git+https://github.com/GoogleCloudPlatform/google-auth-library-python.git#egg=google-auth + +For more information on supported version control systems and syntax, +see pip’s documentation on VCS Support(1). + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/reference/pip_install/#vcs-support + + +File: pythonpackagingguide.info, Node: Installing from local archives<2>, Next: Using other package indexes, Prev: Installing from version control systems, Up: Installing packages using pip and virtual environments + +3.2.11 Installing from local archives +------------------------------------- + +If you have a local copy of a *note Distribution Package: b.’s archive +(a zip, wheel, or tar file) you can install it directly with pip: + + pip install requests-2.18.4.tar.gz + +If you have a directory containing archives of multiple packages, you +can tell pip to look for packages there and not to use the *note Python +Package Index (PyPI): 39. at all: + + pip install --no-index --find-links=/local/dir/ requests + +This is useful if you are installing packages on a system with limited +connectivity or if you want to strictly control the origin of +distribution packages. + + +File: pythonpackagingguide.info, Node: Using other package indexes, Next: Upgrading packages<2>, Prev: Installing from local archives<2>, Up: Installing packages using pip and virtual environments + +3.2.12 Using other package indexes +---------------------------------- + +If you want to download packages from a different index than the *note +Python Package Index (PyPI): 39, you can use the ‘--index-url’ flag: + + pip install --index-url http://index.example.com/simple/ SomeProject + +If you want to allow packages from both the *note Python Package Index +(PyPI): 39. and a separate index, you can use the ‘--extra-index-url’ +flag instead: + + pip install --extra-index-url http://index.example.com/simple/ SomeProject + + +File: pythonpackagingguide.info, Node: Upgrading packages<2>, Next: Using requirements files, Prev: Using other package indexes, Up: Installing packages using pip and virtual environments + +3.2.13 Upgrading packages +------------------------- + +pip can upgrade packages in-place using the ‘--upgrade’ flag. For +example, to install the latest version of ‘requests’ and all of its +dependencies: + + pip install --upgrade requests + + +File: pythonpackagingguide.info, Node: Using requirements files, Next: Freezing dependencies, Prev: Upgrading packages<2>, Up: Installing packages using pip and virtual environments + +3.2.14 Using requirements files +------------------------------- + +Instead of installing packages individually, pip allows you to declare +all dependencies in a Requirements File(1). For example you could +create a ‘requirements.txt’ file containing: + + requests==2.18.4 + google-auth==1.1.0 + +And tell pip to install all of the packages in this file using the ‘-r’ +flag: + + pip install -r requirements.txt + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + +File: pythonpackagingguide.info, Node: Freezing dependencies, Prev: Using requirements files, Up: Installing packages using pip and virtual environments + +3.2.15 Freezing dependencies +---------------------------- + +Pip can export a list of all installed packages and their versions using +the ‘freeze’ command: + + pip freeze + +Which will output a list of package specifiers such as: + + cachetools==2.0.1 + certifi==2017.7.27.1 + chardet==3.0.4 + google-auth==1.1.1 + idna==2.6 + pyasn1==0.3.6 + pyasn1-modules==0.1.4 + requests==2.18.4 + rsa==3.4.2 + six==1.11.0 + urllib3==1.22 + +This is useful for creating Requirements Files(1) that can re-create the +exact versions of all packages installed in an environment. + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + +File: pythonpackagingguide.info, Node: Installing stand alone command line tools, Next: Installing pip/setuptools/wheel with Linux Package Managers, Prev: Installing packages using pip and virtual environments, Up: Guides + +3.3 Installing stand alone command line tools +============================================= + +Many packages have command line entry points. Examples of this type of +application are mypy(1), flake8(2), pipenv(3),and black(4). + +Usually you want to be able to access these from anywhere, but +installing packages and their dependencies to the same global +environment can cause version conflicts and break dependencies the +operating system has on Python packages. + +pipx(5) solves this by creating a virtual environment for each package, +while also ensuring that package’s applications are accessible through a +directory that is on your ‘$PATH’. This allows each package to be +upgraded or uninstalled without causing conflicts with other packages, +and allows you to safely run the program from anywhere. + + Note: pipx only works with Python 3.6+. + +‘pipx’ is installed with ‘pip’: + + $ python3 -m pip install --user pipx + $ python3 -m pipx ensurepath # ensures the path of the CLI application directory is on your $PATH + + Note: You may need to restart your terminal for the path updates to + take effect. + +Now you can install packages with ‘pipx install’ and access the +package’s entry point(s) from anywhere. + + $ pipx install PACKAGE + $ ENTRYPOINT_OF_PACKAGE [ARGS] + +For example + + $ pipx install cowsay + installed package cowsay 2.0, Python 3.6.2+ + These binaries are now globally available + - cowsay + done! ✨ 🌟 ✨ + $ cowsay moo + ___ + < moo > + === + \ + \ + ^__^ + (oo)\_______ + (__)\ )\/ ||----w | + || || + +To see a list of packages installed with pipx and which CLI applications +are available, use ‘pipx list’. + + $ pipx list + venvs are in /Users/user/.local/pipx/venvs + symlinks to binaries are in /Users/user/.local/bin + package black 18.9b0, Python 3.6.2+ + - black + - blackd + package cowsay 2.0, Python 3.6.2+ + - cowsay + package mypy 0.660, Python 3.6.2+ + - dmypy + - mypy + - stubgen + package nox 2018.10.17, Python 3.6.2+ + - nox + - tox-to-nox + +To upgrade or uninstall the package + + $ pipx upgrade PACKAGE + $ pipx uninstall PACKAGE + +‘pipx’ can be upgraded or uninstalled with pip + + $ pip install -U pipx + $ pip uninstall pipx + +‘pipx’ also allows you to install and run the latest version of a cli +tool in a temporary, ephemeral environment. + + $ pipx run PACKAGE [ARGS] + +For example + + $ pipx run cowsay moooo + +To see the full list of commands ‘pipx’ offers, run + + $ pipx --help + +You can learn more about ‘pipx’ at its homepage, +‘https://github.com/pipxproject/pipx’. + + ---------- Footnotes ---------- + + (1) https://github.com/python/mypy + + (2) https://github.com/PyCQA/flake8 + + (3) https://github.com/pypa/pipenv + + (4) https://github.com/ambv/black + + (5) https://github.com/pipxproject/pipx + + +File: pythonpackagingguide.info, Node: Installing pip/setuptools/wheel with Linux Package Managers, Next: Installing scientific packages, Prev: Installing stand alone command line tools, Up: Guides + +3.4 Installing pip/setuptools/wheel with Linux Package Managers +=============================================================== + + +Page Status: Incomplete + + +Last Reviewed: 2015-09-17 + +This section covers how to install *note pip: 2b, *note setuptools: 2d, +and *note wheel: 2e. using Linux package managers. + +If you’re using a Python that was downloaded from python.org(1), then +this section does not apply. See the *note Requirements for Installing +Packages: 26. section instead. + +Note that it’s common for the versions of *note pip: 2b, *note +setuptools: 2d, and *note wheel: 2e. supported by a specific Linux +Distribution to be outdated by the time it’s released to the public, and +updates generally only occur for security reasons, not for feature +updates. For certain Distributions, there are additional repositories +that can be enabled to provide newer versions. The repositories we know +about are explained below. + +Also note that it’s somewhat common for Distributions to apply patches +for the sake of security and normalization to their own standards. In +some cases, this can lead to bugs or unexpected behaviors that vary from +the original unpatched versions. When this is known, we will make note +of it below. + +* Menu: + +* Fedora:: +* CentOS/RHEL:: +* openSUSE:: +* Debian/Ubuntu:: +* Arch Linux:: + + ---------- Footnotes ---------- + + (1) https://www.python.org + + +File: pythonpackagingguide.info, Node: Fedora, Next: CentOS/RHEL, Up: Installing pip/setuptools/wheel with Linux Package Managers + +3.4.1 Fedora +------------ + + * Fedora 21: + + * Python 2: + + sudo yum upgrade python-setuptools + sudo yum install python-pip python-wheel + + * Python 3: ‘sudo yum install python3 python3-wheel’ + + * Fedora 22: + + * Python 2: + + sudo dnf upgrade python-setuptools + sudo dnf install python-pip python-wheel + + * Python 3: ‘sudo dnf install python3 python3-wheel’ + +To get newer versions of pip, setuptools, and wheel for Python 2, you +can enable the PyPA Copr Repo(1) using the Copr Repo instructions(2), +and then run: + + sudo yum|dnf upgrade python-setuptools + sudo yum|dnf install python-pip python-wheel + + ---------- Footnotes ---------- + + (1) https://copr.fedoraproject.org/coprs/pypa/pypa/ + + (2) https://fedorahosted.org/copr/wiki/HowToEnableRepo + + +File: pythonpackagingguide.info, Node: CentOS/RHEL, Next: openSUSE, Prev: Fedora, Up: Installing pip/setuptools/wheel with Linux Package Managers + +3.4.2 CentOS/RHEL +----------------- + +CentOS and RHEL don’t offer *note pip: 2b. or *note wheel: 2e. in their +core repositories, although *note setuptools: 2d. is installed by +default. + +To install pip and wheel for the system Python, there are two options: + + 1. Enable the EPEL repository(1) using these instructions(2). On EPEL + 6 and EPEL7, you can install pip like so: + + sudo yum install python-pip + + On EPEL 7 (but not EPEL 6), you can install wheel like so: + + sudo yum install python-wheel + + Since EPEL only offers extra, non-conflicting packages, EPEL does + not offer setuptools, since it’s in the core repository. + + 2. Enable the PyPA Copr Repo(3) using these instructions(4) (5). You + can install pip and wheel like so: + + sudo yum install python-pip python-wheel + + To additionally upgrade setuptools, run: + + sudo yum upgrade python-setuptools + +To install pip, wheel, and setuptools, in a parallel, non-system +environment (using yum) then there are two options: + + 1. Use the “Sofware Collections” feature to enable a parallel + collection that includes pip, setuptools, and wheel. + + * For Redhat, see here: + ‘http://developers.redhat.com/products/softwarecollections/overview/’ + + * For CentOS, see here: + ‘https://www.softwarecollections.org/en/’ + + Be aware that collections may not contain the most recent versions. + + 2. Enable the IUS repository(6) and install one of the + parallel-installable(7) Pythons, along with pip, setuptools, and + wheel, which are kept fairly up to date. + + For example, for Python 3.4 on CentOS7/RHEL7: + + sudo yum install python34u python34u-wheel + + ---------- Footnotes ---------- + + (1) https://fedoraproject.org/wiki/EPEL + + (2) +https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F + + (3) https://copr.fedoraproject.org/coprs/pypa/pypa/ + + (4) https://fedorahosted.org/copr/wiki/HowToEnableRepo + + (5) (1) Currently, there is no “copr” yum plugin available for +CentOS/RHEL, so the only option is to manually place the repo files as +described. + + (6) https://ius.io/GettingStarted/ + + (7) https://ius.io/SafeRepo/#parallel-installable-package + + +File: pythonpackagingguide.info, Node: openSUSE, Next: Debian/Ubuntu, Prev: CentOS/RHEL, Up: Installing pip/setuptools/wheel with Linux Package Managers + +3.4.3 openSUSE +-------------- + + * Python 2: + + sudo zypper install python-pip python-setuptools python-wheel + + * Python 3: + + sudo zypper install python3-pip python3-setuptools python3-wheel + + +File: pythonpackagingguide.info, Node: Debian/Ubuntu, Next: Arch Linux, Prev: openSUSE, Up: Installing pip/setuptools/wheel with Linux Package Managers + +3.4.4 Debian/Ubuntu +------------------- + + * Python 2: + + sudo apt install python-pip + + * Python 3: + + sudo apt install python3-venv python3-pip + + Warning: Recent Debian/Ubuntu versions have modified pip to use the + "User Scheme"(1) by default, which is a significant behavior change + that can be surprising to some users. + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/stable/user_guide/#user-installs + + +File: pythonpackagingguide.info, Node: Arch Linux, Prev: Debian/Ubuntu, Up: Installing pip/setuptools/wheel with Linux Package Managers + +3.4.5 Arch Linux +---------------- + + * Python 2: + + sudo pacman -S python2-pip + + * Python 3: + + sudo pacman -S python-pip + +__________________________________________________________________ + + +File: pythonpackagingguide.info, Node: Installing scientific packages, Next: Multi-version installs, Prev: Installing pip/setuptools/wheel with Linux Package Managers, Up: Guides + +3.5 Installing scientific packages +================================== + +Scientific software tends to have more complex dependencies than most, +and it will often have multiple build options to take advantage of +different kinds of hardware, or to interoperate with different pieces of +external software. + +In particular, NumPy(1), which provides the basis for most of the +software in the scientific Python stack(2) can be configured to +interoperate with different FORTRAN libraries, and can take advantage of +different levels of vectorised instructions available in modern CPUs. + +Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, +pre-built 32-bit and 64-bit binaries in the ‘wheel’ format are available +for all major operating systems (Windows, macOS, and Linux) on PyPI. +Note, however, that on Windows, NumPy binaries are linked against the +ATLAS(3) BLAS/LAPACK library, restricted to SSE2 instructions, so they +may not provide optimal linear algebra performance. + +There are a number of alternative options for obtaining scientific +Python libraries (or any other Python libraries that require a +compilation environment to install from source and don’t provide +pre-built wheel files on PyPI). + +* Menu: + +* Building from source:: +* Linux distribution packages:: +* Windows installers:: +* macOS installers and package managers:: +* SciPy distributions:: +* Spack:: +* The conda cross-platform package manager:: + + ---------- Footnotes ---------- + + (1) http://www.numpy.org/ + + (2) http://www.scipy.org/stackspec.html#stackspec + + (3) http://www.netlib.org/atlas/ + + +File: pythonpackagingguide.info, Node: Building from source, Next: Linux distribution packages, Up: Installing scientific packages + +3.5.1 Building from source +-------------------------- + +The same complexity which makes it difficult to distribute NumPy (and +many of the projects that depend on it) as wheel files also make them +difficult to build from source yourself. However, for intrepid folks +that are willing to spend the time wrangling compilers and linkers for +both C and FORTRAN, building from source is always an option. + + +File: pythonpackagingguide.info, Node: Linux distribution packages, Next: Windows installers, Prev: Building from source, Up: Installing scientific packages + +3.5.2 Linux distribution packages +--------------------------------- + +For Linux users, the system package manager will often have pre-compiled +versions of various pieces of scientific software, including NumPy and +other parts of the scientific Python stack. + +If using versions which may be several months old is acceptable, then +this is likely to be a good option (just make sure to allow access to +distributions installed into the system Python when using virtual +environments). + + +File: pythonpackagingguide.info, Node: Windows installers, Next: macOS installers and package managers, Prev: Linux distribution packages, Up: Installing scientific packages + +3.5.3 Windows installers +------------------------ + +Many Python projects that don’t (or can’t) currently publish wheel files +at least publish Windows installers, either on PyPI or on their project +download page. Using these installers allows users to avoid the need to +set up a suitable environment to build extensions locally. + +The extensions provided in these installers are typically compatible +with the CPython Windows installers published on python.org. + +For projects which don’t provide their own Windows installers (and even +some which do), Christoph Gohlke at the University of California +provides a collection of Windows installers(1). Many Python users on +Windows have reported a positive experience with these prebuilt +versions. + +As with Linux system packages, the Windows installers will only install +into a system Python installation - they do not support installation in +virtual environments. Allowing access to distributions installed into +the system Python when using virtual environments is a common approach +to working around this limitation. + +The *note Wheel: d. project also provides a ‘wheel convert’ subcommand +that can convert a Windows ‘bdist_wininst’ installer to a wheel. + + ---------- Footnotes ---------- + + (1) http://www.lfd.uci.edu/~gohlke/pythonlibs/ + + +File: pythonpackagingguide.info, Node: macOS installers and package managers, Next: SciPy distributions, Prev: Windows installers, Up: Installing scientific packages + +3.5.4 macOS installers and package managers +------------------------------------------- + +Similar to the situation on Windows, many projects (including NumPy) +publish macOS installers that are compatible with the macOS CPython +binaries published on python.org. + +macOS users also have access to Linux distribution style package +managers such as ‘MacPorts’. The SciPy site has more details on using +MacPorts to install the scientific Python stack(1) + + ---------- Footnotes ---------- + + (1) http://www.scipy.org/install.html#mac-packages + + +File: pythonpackagingguide.info, Node: SciPy distributions, Next: Spack, Prev: macOS installers and package managers, Up: Installing scientific packages + +3.5.5 SciPy distributions +------------------------- + +The SciPy site lists several distributions(1) that provide the full +SciPy stack to end users in an easy to use and update format. + +Some of these distributions may not be compatible with the standard +‘pip’ and ‘virtualenv’ based toolchain. + + ---------- Footnotes ---------- + + (1) http://www.scipy.org/install.html + + +File: pythonpackagingguide.info, Node: Spack, Next: The conda cross-platform package manager, Prev: SciPy distributions, Up: Installing scientific packages + +3.5.6 Spack +----------- + +Spack(1) is a flexible package manager designed to support multiple +versions, configurations, platforms, and compilers. It was built to +support the needs of large supercomputing centers and scientific +application teams, who must often build software many different ways. +Spack is not limited to Python; it can install packages for ‘C’, ‘C++’, +‘Fortran’, ‘R’, and other languages. It is non-destructive; installing +a new version of one package does not break existing installations, so +many configurations can coexist on the same system. + +Spack offers a simple but powerful syntax that allows users to specify +versions and configuration options concisely. Package files are written +in pure Python, and they are templated so that it is easy to swap +compilers, dependency implementations (like MPI), versions, and build +options with a single package file. Spack also generates `module' files +so that packages can be loaded and unloaded from the user’s environment. + + ---------- Footnotes ---------- + + (1) https://github.com/LLNL/spack/ + + +File: pythonpackagingguide.info, Node: The conda cross-platform package manager, Prev: Spack, Up: Installing scientific packages + +3.5.7 The conda cross-platform package manager +---------------------------------------------- + +Anaconda(1) is a Python distribution published by Anaconda, Inc. It is +a stable collection of Open Source packages for big data and scientific +use. As of the 5.0 release of Anaconda, about 200 packages are +installed by default, and a total of 400-500 can be installed and +updated from the Anaconda repository. + +‘conda’ is an open source (BSD licensed) package management system and +environment management system included in Anaconda that allows users to +install multiple versions of binary software packages and their +dependencies, and easily switch between them. It is a cross-platform +tool working on Windows, macOS, and Linux. Conda can be used to package +up and distribute all kinds of packages, it is not limited to just +Python packages. It has full support for native virtual environments. +Conda makes environments first-class citizens, making it easy to create +independent environments even for C libraries. It is written in Python, +but is Python-agnostic. Conda manages Python itself as a package, so +that ‘conda update python’ is possible, in contrast to pip, which only +manages Python packages. Conda is available in Anaconda and Miniconda +(an easy-to-install download with just Python and conda). + + ---------- Footnotes ---------- + + (1) https://www.anaconda.com/download/ + + +File: pythonpackagingguide.info, Node: Multi-version installs, Next: Packaging and distributing projects, Prev: Installing scientific packages, Up: Guides + +3.6 Multi-version installs +========================== + +easy_install allows simultaneous installation of different versions of +the same project into a single environment shared by multiple programs +which must ‘require’ the appropriate version of the project at run time +(using ‘pkg_resources’). + +For many use cases, virtual environments address this need without the +complication of the ‘require’ directive. However, the advantage of +parallel installations within the same environment is that it works for +an environment shared by multiple applications, such as the system +Python in a Linux distribution. + +The major limitation of ‘pkg_resources’ based parallel installation is +that as soon as you import ‘pkg_resources’ it locks in the `default' +version of everything which is already available on sys.path. This can +cause problems, since ‘setuptools’ created command line scripts use +‘pkg_resources’ to find the entry point to execute. This means that, +for example, you can’t use ‘require’ tests invoked through ‘nose’ or a +WSGI application invoked through ‘gunicorn’ if your application needs a +non-default version of anything that is available on the standard +‘sys.path’ - the script wrapper for the main application will lock in +the version that is available by default, so the subsequent ‘require’ +call in your own code fails with a spurious version conflict. + +This can be worked around by setting all dependencies in +‘__main__.__requires__’ before importing ‘pkg_resources’ for the first +time, but that approach does mean that standard command line invocations +of the affected tools can’t be used - it’s necessary to write a custom +wrapper script or use ‘python -c ''’ to invoke the +application’s main entry point directly. + +Refer to the pkg_resources documentation(1) for more details. + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/pkg_resources.html#workingset-objects + + +File: pythonpackagingguide.info, Node: Packaging and distributing projects, Next: Including files in source distributions with MANIFEST in, Prev: Multi-version installs, Up: Guides + +3.7 Packaging and distributing projects +======================================= + +This section covers the basics of how to configure, package and +distribute your own Python projects. It assumes that you are already +familiar with the contents of the *note Installing Packages: 23. page. + +The section does `not' aim to cover best practices for Python project +development as a whole. For example, it does not provide guidance or +tool recommendations for version control, documentation, or testing. + +For more reference material, see Building and Distributing Packages(1) +in the *note setuptools: 2d. docs, but note that some advisory content +there may be outdated. In the event of conflicts, prefer the advice in +the Python Packaging User Guide. + +* Menu: + +* Requirements for packaging and distributing:: +* Configuring your project:: +* Working in “development mode”:: +* Packaging your project:: +* Uploading your Project to PyPI:: + + ---------- Footnotes ---------- + + (1) https://setuptools.readthedocs.io/en/latest/setuptools.html + + +File: pythonpackagingguide.info, Node: Requirements for packaging and distributing, Next: Configuring your project, Up: Packaging and distributing projects + +3.7.1 Requirements for packaging and distributing +------------------------------------------------- + + 1. First, make sure you have already fulfilled the *note requirements + for installing packages: 26. + + 2. Install “twine” (1): + + pip install twine + + You’ll need this to upload your project *note distributions: b. to + *note PyPI: 39. (see *note below: b8.). + + ---------- Footnotes ---------- + + (1) (1) Depending on your platform, this may require root or +Administrator access. *note pip: 2b. is currently considering changing +this by making user installs the default behavior +(https://github.com/pypa/pip/issues/1668). + + +File: pythonpackagingguide.info, Node: Configuring your project, Next: Working in “development mode”, Prev: Requirements for packaging and distributing, Up: Packaging and distributing projects + +3.7.2 Configuring your project +------------------------------ + +* Menu: + +* Initial files:: +* setup() args: setup args. +* Choosing a versioning scheme:: + + +File: pythonpackagingguide.info, Node: Initial files, Next: setup args, Up: Configuring your project + +3.7.2.1 Initial files +..................... + +* Menu: + +* setup.py: setup py. +* setup.cfg: setup cfg. +* README.rst / README.md: README rst / README md. +* MANIFEST.in: MANIFEST in. +* LICENSE.txt: LICENSE txt. +* :: + + +File: pythonpackagingguide.info, Node: setup py, Next: setup cfg, Up: Initial files + +3.7.2.2 setup.py +................ + +The most important file is ‘setup.py’ which exists at the root of your +project directory. For an example, see the setup.py(1) in the PyPA +sample project(2). + +‘setup.py’ serves two primary functions: + + 1. It’s the file where various aspects of your project are configured. + The primary feature of ‘setup.py’ is that it contains a global + ‘setup()’ function. The keyword arguments to this function are how + specific details of your project are defined. The most relevant + arguments are explained in *note the section below: bc. + + 2. It’s the command line interface for running various commands that + relate to packaging tasks. To get a listing of available commands, + run ‘python setup.py --help-commands’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/sampleproject/blob/master/setup.py + + (2) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: setup cfg, Next: README rst / README md, Prev: setup py, Up: Initial files + +3.7.2.3 setup.cfg +................. + +‘setup.cfg’ is an ini file that contains option defaults for ‘setup.py’ +commands. For an example, see the setup.cfg(1) in the PyPA sample +project(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/sampleproject/blob/master/setup.cfg + + (2) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: README rst / README md, Next: MANIFEST in, Prev: setup cfg, Up: Initial files + +3.7.2.4 README.rst / README.md +.............................. + +All projects should contain a readme file that covers the goal of the +project. The most common format is reStructuredText(1) with an “rst” +extension, although this is not a requirement; multiple variants of +Markdown(2) are supported as well (look at ‘setup()’’s *note +long_description_content_type: bf. argument). + +For an example, see README.md(3) from the PyPA sample project(4). + + Note: Projects using *note setuptools: 2d. 0.6.27+ have standard + readme files (‘README.rst’, ‘README.txt’, or ‘README’) included in + source distributions by default. The built-in *note distutils: c0. + library adopts this behavior beginning in Python 3.7. + Additionally, *note setuptools: 2d. 36.4.0+ will include a + ‘README.md’ if found. If you are using setuptools, you don’t need + to list your readme file in ‘MANIFEST.in’. Otherwise, include it + to be explicit. + + ---------- Footnotes ---------- + + (1) http://docutils.sourceforge.net/rst.html + + (2) https://daringfireball.net/projects/markdown/ + + (3) https://github.com/pypa/sampleproject/blob/master/README.md + + (4) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: MANIFEST in, Next: LICENSE txt, Prev: README rst / README md, Up: Initial files + +3.7.2.5 MANIFEST.in +................... + +A ‘MANIFEST.in’ is needed when you need to package additional files that +are not automatically included in a source distribution. For details on +writing a ‘MANIFEST.in’ file, including a list of what’s included by +default, see “*note Including files in source distributions with +MANIFEST.in: c2.”. + +For an example, see the MANIFEST.in(1) from the PyPA sample project(2). + + Note: ‘MANIFEST.in’ does not affect binary distributions such as + wheels. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/sampleproject/blob/master/MANIFEST.in + + (2) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: LICENSE txt, Next: , Prev: MANIFEST in, Up: Initial files + +3.7.2.6 LICENSE.txt +................... + +Every package should include a license file detailing the terms of +distribution. In many jurisdictions, packages without an explicit +license can not be legally used or distributed by anyone other than the +copyright holder. If you’re unsure which license to choose, you can use +resources such as GitHub’s Choose a License(1) or consult a lawyer. + +For an example, see the LICENSE.txt(2) from the PyPA sample project(3). + + ---------- Footnotes ---------- + + (1) https://choosealicense.com/ + + (2) https://github.com/pypa/sampleproject/blob/master/LICENSE.txt + + (3) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: , Prev: LICENSE txt, Up: Initial files + +3.7.2.7 +...................... + +Although it’s not required, the most common practice is to include your +Python modules and packages under a single top-level package that has +the same *note name: c5. as your project, or something very close. + +For an example, see the sample(1) package that’s included in the PyPA +sample project(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/sampleproject/tree/master/src/sample + + (2) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: setup args, Next: Choosing a versioning scheme, Prev: Initial files, Up: Configuring your project + +3.7.2.8 setup() args +.................... + +As mentioned above, the primary feature of ‘setup.py’ is that it +contains a global ‘setup()’ function. The keyword arguments to this +function are how specific details of your project are defined. + +The most relevant arguments are explained below. Most of the snippets +given are taken from the setup.py(1) contained in the PyPA sample +project(2). + +* Menu: + +* name:: +* version:: +* description:: +* url:: +* author:: +* license:: +* classifiers:: +* keywords:: +* project_urls:: +* packages:: +* py_modules:: +* install_requires:: +* python_requires:: +* package_data:: +* data_files:: +* scripts:: +* entry_points:: + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/sampleproject/blob/master/setup.py + + (2) https://github.com/pypa/sampleproject + + +File: pythonpackagingguide.info, Node: name, Next: version, Up: setup args + +3.7.2.9 name +............ + + name='sample', + +This is the name of your project, determining how your project is listed +on *note PyPI: 39. Per PEP 508(1), valid project names must: + + - Consist only of ASCII letters, digits, underscores (‘_’), hyphens + (‘-’), and/or periods (‘.’), and + + - Start & end with an ASCII letter or digit. + +Comparison of project names is case insensitive and treats +arbitrarily-long runs of underscores, hyphens, and/or periods as equal. +For example, if you register a project named ‘cool-stuff’, users will be +able to download it or declare a dependency on it using any of the +following spellings: + + Cool-Stuff + cool.stuff + COOL_STUFF + CoOl__-.-__sTuFF + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0508 + + +File: pythonpackagingguide.info, Node: version, Next: description, Prev: name, Up: setup args + +3.7.2.10 version +................ + + version='1.2.0', + +This is the current version of your project, allowing your users to +determine whether or not they have the latest version, and to indicate +which specific versions they’ve tested their own software against. + +Versions are displayed on *note PyPI: 39. for each release if you +publish your project. + +See *note Choosing a versioning scheme: c9. for more information on ways +to use versions to convey compatibility information to your users. + +If the project code itself needs run-time access to the version, the +simplest way is to keep the version in both ‘setup.py’ and your code. +If you’d rather not duplicate the value, there are a few ways to manage +this. See the “*note Single-sourcing the package version: ca.” Advanced +Topics section. + + +File: pythonpackagingguide.info, Node: description, Next: url, Prev: version, Up: setup args + +3.7.2.11 description +.................... + + description='A sample Python project', + long_description=long_description, + long_description_content_type='text/x-rst', + +Give a short and long description for your project. + +These values will be displayed on *note PyPI: 39. if you publish your +project. On ‘pypi.org’, the user interface displays ‘description’ in +the grey banner and ‘long_description’ in the section named “Project +Description”. + +‘description’ is also displayed in lists of projects. For example, it’s +visible in the search results pages such as +‘https://pypi.org/search/?q=jupyter’, the front-page lists of trending +projects and new releases, and the list of projects you maintain within +your account profile (such as ‘https://pypi.org/user/jaraco/’). + +A content type(1) can be specified with the +‘long_description_content_type’ argument, which can be one of +‘text/plain’, ‘text/x-rst’, or ‘text/markdown’, corresponding to no +formatting, reStructuredText (reST)(2), and the Github-flavored Markdown +dialect of Markdown(3) respectively. + + ---------- Footnotes ---------- + + (1) +https://packaging.python.org/specifications/core-metadata/#description-content-type-optional + + (2) +http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#reference-names + + (3) https://daringfireball.net/projects/markdown/ + + +File: pythonpackagingguide.info, Node: url, Next: author, Prev: description, Up: setup args + +3.7.2.12 url +............ + + url='https://github.com/pypa/sampleproject', + +Give a homepage URL for your project. + + +File: pythonpackagingguide.info, Node: author, Next: license, Prev: url, Up: setup args + +3.7.2.13 author +............... + + author='A. Random Developer', + author_email='author@example.com', + +Provide details about the author. + + +File: pythonpackagingguide.info, Node: license, Next: classifiers, Prev: author, Up: setup args + +3.7.2.14 license +................ + + license='MIT', + +The ‘license’ argument doesn’t have to indicate the license under which +your package is being released, although you may optionally do so if you +want. If you’re using a standard, well-known license, then your main +indication can and should be via the ‘classifiers’ argument. +Classifiers exist for all major open-source licenses. + +The “license” argument is more typically used to indicate differences +from well-known licenses, or to include your own, unique license. As a +general rule, it’s a good idea to use a standard, well-known license, +both to avoid confusion and because some organizations avoid software +whose license is unapproved. + + +File: pythonpackagingguide.info, Node: classifiers, Next: keywords, Prev: license, Up: setup args + +3.7.2.15 classifiers +.................... + + classifiers=[ + # How mature is this project? Common values are + # 3 - Alpha + # 4 - Beta + # 5 - Production/Stable + 'Development Status :: 3 - Alpha', + + # Indicate who your project is intended for + 'Intended Audience :: Developers', + 'Topic :: Software Development :: Build Tools', + + # Pick your license as you wish (should match "license" above) + 'License :: OSI Approved :: MIT License', + + # Specify the Python versions you support here. In particular, ensure + # that you indicate whether you support Python 2, Python 3 or both. + 'Programming Language :: Python :: 2', + 'Programming Language :: Python :: 2.6', + 'Programming Language :: Python :: 2.7', + 'Programming Language :: Python :: 3', + 'Programming Language :: Python :: 3.2', + 'Programming Language :: Python :: 3.3', + 'Programming Language :: Python :: 3.4', + ], + +Provide a list of classifiers that categorize your project. For a full +listing, see ‘https://pypi.org/classifiers/’. + +Although the list of classifiers is often used to declare what Python +versions a project supports, this information is only used for searching +& browsing projects on PyPI, not for installing projects. To actually +restrict what Python versions a project can be installed on, use the +*note python_requires: d0. argument. + + +File: pythonpackagingguide.info, Node: keywords, Next: project_urls, Prev: classifiers, Up: setup args + +3.7.2.16 keywords +................. + + keywords='sample setuptools development', + +List keywords that describe your project. + + +File: pythonpackagingguide.info, Node: project_urls, Next: packages, Prev: keywords, Up: setup args + +3.7.2.17 project_urls +..................... + + project_urls={ + 'Documentation': 'https://packaging.python.org/tutorials/distributing-packages/', + 'Funding': 'https://donate.pypi.org', + 'Say Thanks!': 'http://saythanks.io/to/example', + 'Source': 'https://github.com/pypa/sampleproject/', + 'Tracker': 'https://github.com/pypa/sampleproject/issues', + }, + +List additional relevant URLs about your project. This is the place to +link to bug trackers, source repositories, or where to support package +development. The string of the key is the exact text that will be +displayed on PyPI. + + +File: pythonpackagingguide.info, Node: packages, Next: py_modules, Prev: project_urls, Up: setup args + +3.7.2.18 packages +................. + + packages=find_packages(include=['sample', 'sample.*']), + +Set ‘packages’ to a list of all *note packages: a. in your project, +including their subpackages, sub-subpackages, etc. Although the +packages can be listed manually, ‘setuptools.find_packages()’ finds them +automatically. Use the ‘include’ keyword argument to find only the +given packages. Use the ‘exclude’ keyword argument to omit packages +that are not intended to be released and installed. + + +File: pythonpackagingguide.info, Node: py_modules, Next: install_requires, Prev: packages, Up: setup args + +3.7.2.19 py_modules +................... + + py_modules=["six"], + +If your project contains any single-file Python modules that aren’t part +of a package, set ‘py_modules’ to a list of the names of the modules +(minus the ‘.py’ extension) in order to make *note setuptools: 2d. aware +of them. + + +File: pythonpackagingguide.info, Node: install_requires, Next: python_requires, Prev: py_modules, Up: setup args + +3.7.2.20 install_requires +......................... + + install_requires=['peppercorn'], + +“install_requires” should be used to specify what dependencies a project +minimally needs to run. When the project is installed by *note pip: 2b, +this is the specification that is used to install its dependencies. + +For more on using “install_requires” see *note install_requires vs +requirements files: d6. + + +File: pythonpackagingguide.info, Node: python_requires, Next: package_data, Prev: install_requires, Up: setup args + +3.7.2.21 python_requires +........................ + +If your project only runs on certain Python versions, setting the +‘python_requires’ argument to the appropriate PEP 440(1) version +specifier string will prevent *note pip: 2b. from installing the project +on other Python versions. For example, if your package is for Python 3+ +only, write: + + python_requires='>=3', + +If your package is for Python 3.3 and up but you’re not willing to +commit to Python 4 support yet, write: + + python_requires='~=3.3', + +If your package is for Python 2.6, 2.7, and all versions of Python 3 +starting with 3.3, write: + + python_requires='>=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4', + +And so on. + + Note: Support for this feature is relatively recent. Your + project’s source distributions and wheels (see *note Packaging your + project: d8.) must be built using at least version 24.2.0 of *note + setuptools: 2d. in order for the ‘python_requires’ argument to be + recognized and the appropriate metadata generated. + + In addition, only versions 9.0.0 and higher of *note pip: 2b. + recognize the ‘python_requires’ metadata. Users with earlier + versions of pip will be able to download & install projects on any + Python version regardless of the projects’ ‘python_requires’ + values. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + +File: pythonpackagingguide.info, Node: package_data, Next: data_files, Prev: python_requires, Up: setup args + +3.7.2.22 package_data +..................... + + package_data={ + 'sample': ['package_data.dat'], + }, + +Often, additional files need to be installed into a *note package: a. +These files are often data that’s closely related to the package’s +implementation, or text files containing documentation that might be of +interest to programmers using the package. These files are called +“package data”. + +The value must be a mapping from package name to a list of relative path +names that should be copied into the package. The paths are interpreted +as relative to the directory containing the package. + +For more information, see Including Data Files(1) from the setuptools +docs(2). + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files + + (2) https://setuptools.readthedocs.io + + +File: pythonpackagingguide.info, Node: data_files, Next: scripts, Prev: package_data, Up: setup args + +3.7.2.23 data_files +................... + + data_files=[('my_data', ['data/data_file'])], + +Although configuring *note package_data: da. is sufficient for most +needs, in some cases you may need to place data files `outside' of your +*note packages: a. The ‘data_files’ directive allows you to do that. +It is mostly useful if you need to install files which are used by other +programs, which may be unaware of Python packages. + +Each ‘(directory, files)’ pair in the sequence specifies the +installation directory and the files to install there. The ‘directory’ +must be a relative path (although this may change in the future, see +wheel Issue #92(1)). and it is interpreted relative to the installation +prefix (Python’s ‘sys.prefix’ for a default installation; +‘site.USER_BASE’ for a user installation). Each file name in ‘files’ is +interpreted relative to the ‘setup.py’ script at the top of the project +source distribution. + +For more information see the distutils section on Installing Additional +Files(2). + + Note: When installing packages as egg, ‘data_files’ is not + supported. So, if your project uses *note setuptools: 2d, you must + use ‘pip’ to install it. Alternatively, if you must use ‘python + setup.py’, then you need to pass the ‘--old-and-unmanageable’ + option. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/wheel/issues/92 + + (2) +http://docs.python.org/3/distutils/setupscript.html#installing-additional-files + + +File: pythonpackagingguide.info, Node: scripts, Next: entry_points, Prev: data_files, Up: setup args + +3.7.2.24 scripts +................ + +Although ‘setup()’ supports a scripts(1) keyword for pointing to +pre-made scripts to install, the recommended approach to achieve +cross-platform compatibility is to use *note console_scripts: de. entry +points (see below). + + ---------- Footnotes ---------- + + (1) +http://docs.python.org/3/distutils/setupscript.html#installing-scripts + + +File: pythonpackagingguide.info, Node: entry_points, Prev: scripts, Up: setup args + +3.7.2.25 entry_points +..................... + + entry_points={ + ... + }, + +Use this keyword to specify any plugins that your project provides for +any named entry points that may be defined by your project or others +that you depend on. + +For more information, see the section on Advertising Behavior(1) from +the *note setuptools: 2d. docs. + +The most commonly used entry point is “console_scripts” (see below). + +* Menu: + +* console_scripts:: + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/userguide/entry_point.html#dynamic-discovery-of-services-and-plugins + + +File: pythonpackagingguide.info, Node: console_scripts, Up: entry_points + +3.7.2.26 console_scripts +........................ + + entry_points={ + 'console_scripts': [ + 'sample=sample:main', + ], + }, + +Use “console_script” entry points(1) to register your script interfaces. +You can then let the toolchain handle the work of turning these +interfaces into actual scripts (2). The scripts will be generated +during the install of your *note distribution: b. + +For more information, see Automatic Script Creation(3) from the +setuptools docs(4). + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins + + (2) (2) Specifically, the “console_script” approach generates ‘.exe’ +files on Windows, which are necessary because the OS special-cases +‘.exe’ files. Script-execution features like ‘PATHEXT’ and the Python +Launcher for Windows (https://www.python.org/dev/peps/pep-0397) allow +scripts to be used in many cases, but not all. + + (3) +https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation + + (4) https://setuptools.readthedocs.io + + +File: pythonpackagingguide.info, Node: Choosing a versioning scheme, Prev: setup args, Up: Configuring your project + +3.7.2.27 Choosing a versioning scheme +..................................... + +* Menu: + +* Standards compliance for interoperability:: +* Scheme choices:: +* Pre-release versioning:: +* Local version identifiers:: + + +File: pythonpackagingguide.info, Node: Standards compliance for interoperability, Next: Scheme choices, Up: Choosing a versioning scheme + +3.7.2.28 Standards compliance for interoperability +.................................................. + +Different Python projects may use different versioning schemes based on +the needs of that particular project, but all of them are required to +comply with the flexible public version scheme(1) specified in PEP +440(2) in order to be supported in tools and libraries like ‘pip’ and +‘setuptools’. + +Here are some examples of compliant version numbers: + + 1.2.0.dev1 # Development release + 1.2.0a1 # Alpha Release + 1.2.0b1 # Beta Release + 1.2.0rc1 # Release Candidate + 1.2.0 # Final Release + 1.2.0.post1 # Post Release + 15.10 # Date based release + 23 # Serial release + +To further accommodate historical variations in approaches to version +numbering, PEP 440(3) also defines a comprehensive technique for version +normalisation(4) that maps variant spellings of different version +numbers to a standardised canonical form. + + ---------- Footnotes ---------- + + (1) +https://www.python.org/dev/peps/pep-0440#public-version-identifiers + + (2) https://www.python.org/dev/peps/pep-0440 + + (3) https://www.python.org/dev/peps/pep-0440 + + (4) https://www.python.org/dev/peps/pep-0440#normalization + + +File: pythonpackagingguide.info, Node: Scheme choices, Next: Pre-release versioning, Prev: Standards compliance for interoperability, Up: Choosing a versioning scheme + +3.7.2.29 Scheme choices +....................... + +* Menu: + +* Semantic versioning (preferred): Semantic versioning preferred. +* Date based versioning:: +* Serial versioning:: +* Hybrid schemes:: + + +File: pythonpackagingguide.info, Node: Semantic versioning preferred, Next: Date based versioning, Up: Scheme choices + +3.7.2.30 Semantic versioning (preferred) +........................................ + +For new projects, the recommended versioning scheme is based on Semantic +Versioning(1), but adopts a different approach to handling pre-releases +and build metadata. + +The essence of semantic versioning is a 3-part MAJOR.MINOR.MAINTENANCE +numbering scheme, where the project author increments: + + 1. MAJOR version when they make incompatible API changes, + + 2. MINOR version when they add functionality in a backwards-compatible + manner, and + + 3. MAINTENANCE version when they make backwards-compatible bug fixes. + +Adopting this approach as a project author allows users to make use of +“compatible release”(2) specifiers, where ‘name ~= X.Y’ requires at +least release X.Y, but also allows any later release with a matching +MAJOR version. + +Python projects adopting semantic versioning should abide by clauses 1-8 +of the Semantic Versioning 2.0.0 specification(3). + + ---------- Footnotes ---------- + + (1) http://semver.org + + (2) https://www.python.org/dev/peps/pep-0440#compatible-release + + (3) http://semver.org + + +File: pythonpackagingguide.info, Node: Date based versioning, Next: Serial versioning, Prev: Semantic versioning preferred, Up: Scheme choices + +3.7.2.31 Date based versioning +.............................. + +Semantic versioning is not a suitable choice for all projects, such as +those with a regular time based release cadence and a deprecation +process that provides warnings for a number of releases prior to removal +of a feature. + +A key advantage of date based versioning is that it is straightforward +to tell how old the base feature set of a particular release is given +just the version number. + +Version numbers for date based projects typically take the form of +YEAR.MONTH (for example, ‘12.04’, ‘15.10’). + + +File: pythonpackagingguide.info, Node: Serial versioning, Next: Hybrid schemes, Prev: Date based versioning, Up: Scheme choices + +3.7.2.32 Serial versioning +.......................... + +This is the simplest possible versioning scheme, and consists of a +single number which is incremented every release. + +While serial versioning is very easy to manage as a developer, it is the +hardest to track as an end user, as serial version numbers convey little +or no information regarding API backwards compatibility. + + +File: pythonpackagingguide.info, Node: Hybrid schemes, Prev: Serial versioning, Up: Scheme choices + +3.7.2.33 Hybrid schemes +....................... + +Combinations of the above schemes are possible. For example, a project +may combine date based versioning with serial versioning to create a +YEAR.SERIAL numbering scheme that readily conveys the approximate age of +a release, but doesn’t otherwise commit to a particular release cadence +within the year. + + +File: pythonpackagingguide.info, Node: Pre-release versioning, Next: Local version identifiers, Prev: Scheme choices, Up: Choosing a versioning scheme + +3.7.2.34 Pre-release versioning +............................... + +Regardless of the base versioning scheme, pre-releases for a given final +release may be published as: + + * zero or more dev releases (denoted with a “.devN” suffix) + + * zero or more alpha releases (denoted with a “.aN” suffix) + + * zero or more beta releases (denoted with a “.bN” suffix) + + * zero or more release candidates (denoted with a “.rcN” suffix) + +‘pip’ and other modern Python package installers ignore pre-releases by +default when deciding which versions of dependencies to install. + + +File: pythonpackagingguide.info, Node: Local version identifiers, Prev: Pre-release versioning, Up: Choosing a versioning scheme + +3.7.2.35 Local version identifiers +.................................. + +Public version identifiers are designed to support distribution via +*note PyPI: 39. Python’s software distribution tools also support the +notion of a local version identifier(1), which can be used to identify +local development builds not intended for publication, or modified +variants of a release maintained by a redistributor. + +A local version identifier takes the form ‘+’. For example: + + 1.2.0.dev1+hg.5.b11e5e6f0b0b # 5th VCS commmit since 1.2.0.dev1 release + 1.2.1+fedora.4 # Package with downstream Fedora patches applied + + ---------- Footnotes ---------- + + (1) +https://www.python.org/dev/peps/pep-0440#local-version-identifiers + + +File: pythonpackagingguide.info, Node: Working in “development mode”, Next: Packaging your project, Prev: Configuring your project, Up: Packaging and distributing projects + +3.7.3 Working in “development mode” +----------------------------------- + +Although not required, it’s common to locally install your project in +“editable” or “develop” mode while you’re working on it. This allows +your project to be both installed and editable in project form. + +Assuming you’re in the root of your project directory, then run: + + pip install -e . + +Although somewhat cryptic, ‘-e’ is short for ‘--editable’, and ‘.’ +refers to the current working directory, so together, it means to +install the current directory (i.e. your project) in editable mode. +This will also install any dependencies declared with “install_requires” +and any scripts declared with “console_scripts”. Dependencies will be +installed in the usual, non-editable mode. + +It’s fairly common to also want to install some of your dependencies in +editable mode as well. For example, supposing your project requires +“foo” and “bar”, but you want “bar” installed from VCS in editable mode, +then you could construct a requirements file like so: + + -e . + -e git+https://somerepo/bar.git#egg=bar + +The first line says to install your project and any dependencies. The +second line overrides the “bar” dependency, such that it’s fulfilled +from VCS, not PyPI. + +If, however, you want “bar” installed from a local directory in editable +mode, the requirements file should look like this, with the local paths +at the top of the file: + + -e /path/to/project/bar + -e . + +Otherwise, the dependency will be fulfilled from PyPI, due to the +installation order of the requirements file. For more on requirements +files, see the Requirements File(1) section in the pip docs. For more +on VCS installs, see the VCS Support(2) section of the pip docs. + +Lastly, if you don’t want to install any dependencies at all, you can +run: + + pip install -e . --no-deps + +For more information, see the Development Mode(3) section of the +setuptools docs(4). + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + (2) https://pip.pypa.io/en/latest/reference/pip_install/#vcs-support + + (3) +https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode + + (4) https://setuptools.readthedocs.io + + +File: pythonpackagingguide.info, Node: Packaging your project, Next: Uploading your Project to PyPI, Prev: Working in “development mode”, Up: Packaging and distributing projects + +3.7.4 Packaging your project +---------------------------- + +To have your project installable from a *note Package Index: ec. like +*note PyPI: 39, you’ll need to create a *note Distribution: b. (aka +“*note Package: b.”) for your project. + +* Menu: + +* Source distributions:: +* Wheels:: + + +File: pythonpackagingguide.info, Node: Source distributions, Next: Wheels, Up: Packaging your project + +3.7.4.1 Source distributions +............................ + +Minimally, you should create a *note Source Distribution: 3d.: + + python setup.py sdist + +A “source distribution” is unbuilt (i.e. it’s not a *note Built +Distribution: 63.), and requires a build step when installed by pip. +Even if the distribution is pure Python (i.e. contains no extensions), +it still involves a build step to build out the installation metadata +from ‘setup.py’. + + +File: pythonpackagingguide.info, Node: Wheels, Prev: Source distributions, Up: Packaging your project + +3.7.4.2 Wheels +.............. + +You should also create a wheel for your project. A wheel is a *note +built package: 63. that can be installed without needing to go through +the “build” process. Installing wheels is substantially faster for the +end user than installing from a source distribution. + +If your project is pure Python (i.e. contains no compiled extensions) +and natively supports both Python 2 and 3, then you’ll be creating +what’s called a *note *Universal Wheel* (see section below): ef. + +If your project is pure Python but does not natively support both Python +2 and 3, then you’ll be creating a *note “Pure Python Wheel” (see +section below): f0. + +If your project contains compiled extensions, then you’ll be creating +what’s called a *note *Platform Wheel* (see section below): f1. + +Before you can build wheels for your project, you’ll need to install the +‘wheel’ package: + + pip install wheel + +* Menu: + +* Universal Wheels:: +* Pure Python Wheels:: +* Platform Wheels:: + + +File: pythonpackagingguide.info, Node: Universal Wheels, Next: Pure Python Wheels, Up: Wheels + +3.7.4.3 Universal Wheels +........................ + +`Universal Wheels' are wheels that are pure Python (i.e. contain no +compiled extensions) and support Python 2 and 3. This is a wheel that +can be installed anywhere by *note pip: 2b. + +To build the wheel: + + python setup.py bdist_wheel --universal + +You can also permanently set the ‘--universal’ flag in ‘setup.cfg’: + + [bdist_wheel] + universal=1 + +Only use the ‘--universal’ setting, if: + + 1. Your project runs on Python 2 and 3 with no changes (i.e. it does + not require 2to3). + + 2. Your project does not have any C extensions. + +Beware that ‘bdist_wheel’ does not currently have any checks to warn if +you use the setting inappropriately. + +If your project has optional C extensions, it is recommended not to +publish a universal wheel, because pip will prefer the wheel over a +source installation, and prevent the possibility of building the +extension. + + +File: pythonpackagingguide.info, Node: Pure Python Wheels, Next: Platform Wheels, Prev: Universal Wheels, Up: Wheels + +3.7.4.4 Pure Python Wheels +.......................... + +`Pure Python Wheels' that are not “universal” are wheels that are pure +Python (i.e. contain no compiled extensions), but don’t natively +support both Python 2 and 3. + +To build the wheel: + + python setup.py bdist_wheel + +‘bdist_wheel’ will detect that the code is pure Python, and build a +wheel that’s named such that it’s usable on any Python installation with +the same major version (Python 2 or Python 3) as the version you used to +build the wheel. For details on the naming of wheel files, see PEP +425(1). + +If your code supports both Python 2 and 3, but with different code +(e.g., you use "2to3"(2)) you can run ‘setup.py bdist_wheel’ twice, once +with Python 2 and once with Python 3. This will produce wheels for each +version. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + (2) https://docs.python.org/2/library/2to3.html + + +File: pythonpackagingguide.info, Node: Platform Wheels, Prev: Pure Python Wheels, Up: Wheels + +3.7.4.5 Platform Wheels +....................... + +`Platform Wheels' are wheels that are specific to a certain platform +like Linux, macOS, or Windows, usually due to containing compiled +extensions. + +To build the wheel: + + python setup.py bdist_wheel + +‘bdist_wheel’ will detect that the code is not pure Python, and build a +wheel that’s named such that it’s only usable on the platform that it +was built on. For details on the naming of wheel files, see PEP 425(1). + + Note: *note PyPI: 39. currently supports uploads of platform wheels + for Windows, macOS, and the multi-distro ‘manylinux1’ ABI. Details + of the latter are defined in PEP 513(2). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + (2) https://www.python.org/dev/peps/pep-0513 + + +File: pythonpackagingguide.info, Node: Uploading your Project to PyPI, Prev: Packaging your project, Up: Packaging and distributing projects + +3.7.5 Uploading your Project to PyPI +------------------------------------ + +When you ran the command to create your distribution, a new directory +‘dist/’ was created under your project’s root directory. That’s where +you’ll find your distribution file(s) to upload. + + Note: These files are only created when you run the command to + create your distribution. This means that any time you change the + source of your project or the configuration in your ‘setup.py’ + file, you will need to rebuild these files again before you can + distribute the changes to PyPI. + + Note: Before releasing on main PyPI repo, you might prefer training + with the PyPI test site(1) which is cleaned on a semi regular + basis. See *note Using TestPyPI: f6. on how to setup your + configuration in order to use it. + + Warning: In other resources you may encounter references to using + ‘python setup.py register’ and ‘python setup.py upload’. These + methods of registering and uploading a package are `strongly + discouraged' as it may use a plaintext HTTP or unverified HTTPS + connection on some Python versions, allowing your username and + password to be intercepted during transmission. + + Tip: The reStructuredText parser used on PyPI is `not' Sphinx! + Furthermore, to ensure safety of all users, certain kinds of URLs + and directives are forbidden or stripped out (e.g., the ‘.. raw::’ + directive). `Before' trying to upload your distribution, you + should check to see if your brief / long descriptions provided in + ‘setup.py’ are valid. You can do this by following the + instructions for the pypa/readme_renderer(2) tool. + +* Menu: + +* Create an account:: +* Upload your distributions:: + + ---------- Footnotes ---------- + + (1) https://test.pypi.org/ + + (2) https://github.com/pypa/readme_renderer + + +File: pythonpackagingguide.info, Node: Create an account, Next: Upload your distributions, Up: Uploading your Project to PyPI + +3.7.5.1 Create an account +......................... + +First, you need a *note PyPI: 39. user account. You can create an +account using the form on the PyPI website(1). + +Now you’ll create a PyPI API token(2) so you will be able to securely +upload your project. + +Go to ‘https://pypi.org/manage/account/#api-tokens’ and create a new API +token(3); don’t limit its scope to a particular project, since you are +creating a new project. + +`Don’t close the page until you have copied and saved the token — you +won’t see that token again.' + + Note: To avoid having to copy and paste the token every time you + upload, you can create a ‘$HOME/.pypirc’ file: + + [pypi] + username = __token__ + password = + + `Be aware that this stores your token in plaintext.' + + For more details, see the *note specification: f8. for ‘.pypirc’. + + ---------- Footnotes ---------- + + (1) https://pypi.org/account/register/ + + (2) https://pypi.org/help/#apitoken + + (3) https://pypi.org/help/#apitoken + + +File: pythonpackagingguide.info, Node: Upload your distributions, Prev: Create an account, Up: Uploading your Project to PyPI + +3.7.5.2 Upload your distributions +................................. + +Once you have an account you can upload your distributions to *note +PyPI: 39. using *note twine: 66. + +The process for uploading a release is the same regardless of whether or +not the project already exists on PyPI - if it doesn’t exist yet, it +will be automatically created when the first release is uploaded. + +For the second and subsequent releases, PyPI only requires that the +version number of the new release differ from any previous releases. + + twine upload dist/* + +You can see if your package has successfully uploaded by navigating to +the URL ‘https://pypi.org/project/’ where ‘sampleproject’ +is the name of your project that you uploaded. It may take a minute or +two for your project to appear on the site. + +__________________________________________________________________ + + +File: pythonpackagingguide.info, Node: Including files in source distributions with MANIFEST in, Next: Single-sourcing the package version, Prev: Packaging and distributing projects, Up: Guides + +3.8 Including files in source distributions with ‘MANIFEST.in’ +============================================================== + +When building a *note source distribution: 3d. for your package, by +default only a minimal set of files are included. You may find yourself +wanting to include extra files in the source distribution, such as an +authors/contributors file, a ‘docs/’ directory, or a directory of data +files used for testing purposes. There may even be extra files that you +`need' to include; for example, if your ‘setup.py’ computes your +project’s ‘long_description’ by reading from both a README and a +changelog file, you’ll need to include both those files in the sdist so +that people that build or install from the sdist get the correct +results. + +Adding & removing files to & from the source distribution is done by +writing a ‘MANIFEST.in’ file at the project root. + +* Menu: + +* How files are included in an sdist:: +* MANIFEST.in commands: MANIFEST in commands. + + +File: pythonpackagingguide.info, Node: How files are included in an sdist, Next: MANIFEST in commands, Up: Including files in source distributions with MANIFEST in + +3.8.1 How files are included in an sdist +---------------------------------------- + +The following files are included in a source distribution by default: + + - all Python source files implied by the ‘py_modules’ and ‘packages’ + ‘setup()’ arguments + + - all C source files mentioned in the ‘ext_modules’ or ‘libraries’ + ‘setup()’ arguments + + - scripts specified by the ‘scripts’ ‘setup()’ argument + + - all files specified by the ‘package_data’ and ‘data_files’ + ‘setup()’ arguments + + - the file specified by the ‘license_file’ option in ‘setup.cfg’ + (setuptools 40.8.0+) + + - all files specified by the ‘license_files’ option in ‘setup.cfg’ + (setuptools 42.0.0+) + + - all files matching the pattern ‘test/test*.py’ + + - ‘setup.py’ (or whatever you called your setup script) + + - ‘setup.cfg’ + + - ‘README’ + + - ‘README.txt’ + + - ‘README.rst’ (Python 3.7+ or setuptools 0.6.27+) + + - ‘README.md’ (setuptools 36.4.0+) + + - ‘pyproject.toml’ (setuptools 43.0.0+) + + - ‘MANIFEST.in’ + +After adding the above files to the sdist, the commands in ‘MANIFEST.in’ +(if such a file exists) are executed in order to add and remove further +files to & from the sdist. Default files can even be removed from the +sdist with the appropriate ‘MANIFEST.in’ command. + +After processing the ‘MANIFEST.in’ file, setuptools removes the ‘build/’ +directory as well as any directories named ‘RCS’, ‘CVS’, or ‘.svn’ from +the sdist, and it adds a ‘PKG-INFO’ file and an ‘*.egg-info’ directory. +This behavior cannot be changed with ‘MANIFEST.in’. + + +File: pythonpackagingguide.info, Node: MANIFEST in commands, Prev: How files are included in an sdist, Up: Including files in source distributions with MANIFEST in + +3.8.2 ‘MANIFEST.in’ commands +---------------------------- + +A ‘MANIFEST.in’ file consists of commands, one per line, instructing +setuptools to add or remove some set of files from the sdist. The +commands are: + +Command Description + +----------------------------------------------------------------------------------------------------------------------------------------------------------- + +‘include pat1 pat2 ...’ Include all files matching any of the listed patterns + + +‘exclude pat1 pat2 ...’ Exclude all files matching any of the listed patterns + + +‘recursive-include dir-pattern pat1 pat2 ...’ Include all files under directories matching ‘dir-pattern’ that match any of the listed patterns + + +‘recursive-exclude dir-pattern pat1 pat2 ...’ Exclude all files under directories matching ‘dir-pattern’ that match any of the listed patterns + + +‘global-include pat1 pat2 ...’ Include all files anywhere in the source tree matching any of the listed patterns + + +‘global-exclude pat1 pat2 ...’ Exclude all files anywhere in the source tree matching any of the listed patterns + + +‘graft dir-pattern’ Include all files under directories matching ‘dir-pattern’ + + +‘prune dir-pattern’ Exclude all files under directories matching ‘dir-pattern’ + + +The patterns here are glob-style patterns: ‘*’ matches zero or more +regular filename characters (on Unix, everything except forward slash; +on Windows, everything except backslash and colon); ‘?’ matches a single +regular filename character, and ‘[chars]’ matches any one of the +characters between the square brackets (which may contain character +ranges, e.g., ‘[a-z]’ or ‘[a-fA-F0-9]’). Setuptools also has +undocumented support for ‘**’ matching zero or more characters including +forward slash, backslash, and colon. + +Directory patterns are relative to the root of the project directory; +e.g., ‘graft example*’ will include a directory named ‘examples’ in the +project root but will not include ‘docs/examples/’. + +File & directory names in ‘MANIFEST.in’ should be ‘/’-separated; +setuptools will automatically convert the slashes to the local +platform’s appropriate directory separator. + +Commands are processed in the order they appear in the ‘MANIFEST.in’ +file. For example, given the commands: + + graft tests + global-exclude *.py[cod] + +the contents of the directory tree ‘tests’ will first be added to the +sdist, and then after that all files in the sdist with a ‘.pyc’, ‘.pyo’, +or ‘.pyd’ extension will be removed from the sdist. If the commands +were in the opposite order, then ‘*.pyc’ files etc. would be only be +removed from what was already in the sdist before adding ‘tests’, and if +‘tests’ happened to contain any ‘*.pyc’ files, they would end up +included in the sdist because the exclusion happened before they were +included. + + +File: pythonpackagingguide.info, Node: Single-sourcing the package version, Next: Supporting multiple Python versions, Prev: Including files in source distributions with MANIFEST in, Up: Guides + +3.9 Single-sourcing the package version +======================================= + +There are many techniques to maintain a single source of truth for the +version number of your project: + + 1. Read the file in ‘setup.py’ and get the version. Example (from pip + setup.py(1)): + + import codecs + import os.path + + def read(rel_path): + here = os.path.abspath(os.path.dirname(__file__)) + with codecs.open(os.path.join(here, rel_path), 'r') as fp: + return fp.read() + + def get_version(rel_path): + for line in read(rel_path).splitlines(): + if line.startswith('__version__'): + delim = '"' if '"' in line else "'" + return line.split(delim)[1] + else: + raise RuntimeError("Unable to find version string.") + + setup( + ... + version=get_version("package/__init__.py") + ... + ) + + Note: As of the release of setuptools 46.4.0, one can + accomplish the same thing by instead placing the following in + the project’s ‘setup.cfg’ file (replacing “package” with the + import name of the package): + + [metadata] + version = attr: package.__version__ + + Earlier versions of setuptools implemented the ‘attr:’ + directive by importing the module, but setuptools 46.4.0 added + rudimentary AST analysis so that ‘attr:’ can function without + having to import any of the package’s dependencies. + + 2. Use an external build tool that either manages updating both + locations, or offers an API that both locations can use. + + Few tools you could use, in no particular order, and not + necessarily complete: bump2version(2), changes(3), + zest.releaser(4). + + 3. Set the value to a ‘__version__’ global variable in a dedicated + module in your project (e.g. ‘version.py’), then have ‘setup.py’ + read and ‘exec’ the value into a variable. + + version = {} + with open("...sample/version.py") as fp: + exec(fp.read(), version) + # later on we use: version['__version__'] + + Example using this technique: warehouse(5). + + 4. Place the value in a simple ‘VERSION’ text file and have both + ‘setup.py’ and the project code read it. + + with open(os.path.join(mypackage_root_dir, 'VERSION')) as version_file: + version = version_file.read().strip() + + An advantage with this technique is that it’s not specific to + Python. Any tool can read the version. + + Warning: With this approach you must make sure that the + ‘VERSION’ file is included in all your source and binary + distributions (e.g. add ‘include VERSION’ to your + ‘MANIFEST.in’). + + 5. Set the value in ‘setup.py’, and have the project code use the + ‘importlib.metadata’ API to fetch the value at runtime. + (‘importlib.metadata’ was introduced in Python 3.8 and is available + to older versions as the ‘importlib-metadata’ project.) An + installed project’s version can be fetched with the API as follows: + + try: + from importlib import metadata + except ImportError: + # Running on pre-3.8 Python; use importlib-metadata package + import importlib_metadata as metadata + + assert metadata.version('pip') == '1.2.0' + + Be aware that the ‘importlib.metadata’ API only knows about what’s + in the installation metadata, which is not necessarily the code + that’s currently imported. + + If a project uses this method to fetch its version at runtime, then + its ‘install_requires’ value needs to be edited to install + ‘importlib-metadata’ on pre-3.8 versions of Python like so: + + setup( + ... + install_requires=[ + ... + 'importlib-metadata ~= 1.0 ; python_version < "3.8"', + ... + ], + ... + ) + + An older (and less efficient) alternative to ‘importlib.metadata’ + is the ‘pkg_resources’ API provided by ‘setuptools’: + + import pkg_resources + assert pkg_resources.get_distribution('pip').version == '1.2.0' + + If a project uses ‘pkg_resources’ to fetch its own version at + runtime, then ‘setuptools’ must be added to the project’s + ‘install_requires’ list. + + Example using this technique: setuptools(6). + + 6. Set the value to ‘__version__’ in ‘sample/__init__.py’ and import + ‘sample’ in ‘setup.py’. + + import sample + setup( + ... + version=sample.__version__ + ... + ) + + Warning: Although this technique is common, beware that it + will fail if ‘sample/__init__.py’ imports packages from + ‘install_requires’ dependencies, which will very likely not be + installed yet when ‘setup.py’ is run. + + 7. Keep the version number in the tags of a version control system + (Git, Mercurial, etc) instead of in the code, and automatically + extract it from there using setuptools_scm(7). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/pip/blob/master/setup.py#L11 + + (2) https://pypi.org/project/bump2version + + (3) https://pypi.org/project/changes + + (4) https://pypi.org/project/zest.releaser + + (5) +https://github.com/pypa/warehouse/blob/64ca42e42d5613c8339b3ec5e1cb7765c6b23083/warehouse/__about__.py + + (6) +https://github.com/pypa/setuptools/blob/master/setuptools/version.py + + (7) https://pypi.org/project/setuptools_scm + + +File: pythonpackagingguide.info, Node: Supporting multiple Python versions, Next: Dropping support for older Python versions, Prev: Single-sourcing the package version, Up: Guides + +3.10 Supporting multiple Python versions +======================================== + + +Page Status: Incomplete + + +Last Reviewed: 2014-12-24 + + FIXME + + Useful projects/resources to reference: + + - DONE six + - DONE python-future (http://python-future.org) + - tox + - DONE Travis and Shining Panda CI (Shining Panda no longer available) + - DONE Appveyor + - DONE Ned Batchelder's "What's in Which Python" + - http://nedbatchelder.com/blog/201310/whats_in_which_python_3.html + - http://nedbatchelder.com/blog/201109/whats_in_which_python.html + - Lennart Regebro's "Porting to Python 3" + - Greg Hewgill's script to identify the minimum version of Python + required to run a particular script: + https://github.com/ghewgill/pyqver + - the Python 3 porting how to in the main docs + - cross reference to the stable ABI discussion + in the binary extensions topic (once that exists) + - mention version classifiers for distribution metadata + +In addition to the work required to create a Python package, it is often +necessary that the package must be made available on different versions +of Python. Different Python versions may contain different (or renamed) +standard library packages, and the changes between Python versions 2.x +and 3.x include changes in the language syntax. + +Performed manually, all the testing required to ensure that the package +works correctly on all the target Python versions (and OSs!) could be +very time-consuming. Fortunately, several tools are available for +dealing with this, and these will briefly be discussed here. + +* Menu: + +* Automated testing and continuous integration:: +* Tools for single-source Python packages:: +* What’s in which Python?:: + + +File: pythonpackagingguide.info, Node: Automated testing and continuous integration, Next: Tools for single-source Python packages, Up: Supporting multiple Python versions + +3.10.1 Automated testing and continuous integration +--------------------------------------------------- + +Several hosted services for automated testing are available. These +services will typically monitor your source code repository (e.g. at +Github(1) or Bitbucket(2)) and run your project’s test suite every time +a new commit is made. + +These services also offer facilities to run your project’s test suite on +`multiple versions of Python', giving rapid feedback about whether the +code will work, without the developer having to perform such tests +themselves. + +Wikipedia has an extensive comparison(3) of many continuous-integration +systems. There are two hosted services which when used in conjunction +provide automated testing across Linux, Mac and Windows: + + - Travis CI(4) provides both a Linux and a macOS environment. + The Linux environment is Ubuntu 12.04 LTS Server Edition 64 + bit while the macOS is 10.9.2 at the time of writing. + + - Appveyor(5) provides a Windows environment (Windows Server + 2012). + + TODO Either link to or provide example .yml files for these two + services. + + TODO How do we keep the Travis Linux and macOS versions up-to-date in this + document? + +Both Travis CI(6) and Appveyor(7) require a YAML(8)-formatted file as +specification for the instructions for testing. If any tests fail, the +output log for that specific configuration can be inspected. + +For Python projects that are intended to be deployed on both Python 2 +and 3 with a single-source strategy, there are a number of options. + + ---------- Footnotes ---------- + + (1) https://github.com + + (2) https://bitbucket.org + + (3) +http://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software + + (4) https://travis-ci.org + + (5) http://www.appveyor.com + + (6) https://travis-ci.org + + (7) http://www.appveyor.com + + (8) http://www.yaml.org + + +File: pythonpackagingguide.info, Node: Tools for single-source Python packages, Next: What’s in which Python?, Prev: Automated testing and continuous integration, Up: Supporting multiple Python versions + +3.10.2 Tools for single-source Python packages +---------------------------------------------- + +six(1) is a tool developed by Benjamin Peterson for wrapping over the +differences between Python 2 and Python 3. The six(2) package has +enjoyed widespread use and may be regarded as a reliable way to write a +single-source Python module that can be use in both Python 2 and 3. The +six(3) module can be used from as early as Python 2.5. A tool called +modernize(4), developed by Armin Ronacher, can be used to automatically +apply the code modifications provided by six(5). + +Similar to six(6), python-future(7) is a package that provides a +compatibility layer between Python 2 and Python 3 source code; however, +unlike six(8), this package aims to provide interoperability between +Python 2 and Python 3 with a language syntax that matches one of the two +Python versions: one may use + + - a Python 2 (by syntax) module in a Python 3 project. + + - a Python 3 (by syntax) module in a `Python 2' project. + +Because of the bi-directionality, python-future(9) offers a pathway to +converting a Python 2 package to Python 3 syntax module-by-module. +However, in contrast to six(10), python-future(11) is supported only +from Python 2.6. Similar to modernize(12) for six(13), +python-future(14) comes with two scripts called ‘futurize’ and +‘pasteurize’ that can be applied to either a Python 2 module or a Python +3 module respectively. + +Use of six(15) or python-future(16) adds an additional runtime +dependency to your package: with python-future(17), the ‘futurize’ +script can be called with the ‘--stage1’ option to apply only the +changes that Python 2.6+ already provides for forward-compatibility to +Python 3. Any remaining compatibility problems would require manual +changes. + + ---------- Footnotes ---------- + + (1) http://pythonhosted.org/six/ + + (2) http://pythonhosted.org/six/ + + (3) http://pythonhosted.org/six/ + + (4) https://pypi.org/project/modernize + + (5) http://pythonhosted.org/six/ + + (6) http://pythonhosted.org/six/ + + (7) http://python-future.org/overview.html + + (8) http://pythonhosted.org/six/ + + (9) http://python-future.org/overview.html + + (10) http://pythonhosted.org/six/ + + (11) http://python-future.org/overview.html + + (12) https://pypi.org/project/modernize + + (13) http://pythonhosted.org/six/ + + (14) http://python-future.org/overview.html + + (15) http://pythonhosted.org/six/ + + (16) http://python-future.org/overview.html + + (17) http://python-future.org/overview.html + + +File: pythonpackagingguide.info, Node: What’s in which Python?, Prev: Tools for single-source Python packages, Up: Supporting multiple Python versions + +3.10.3 What’s in which Python? +------------------------------ + +Ned Batchelder provides a list of changes in each Python release for +Python 2(1), Python 3.0-3.3(2) and Python 3.4-3.6(3). These lists may +be used to check whether any changes between Python versions may affect +your package. + + TODO These lists should be reproduced here (with permission). + + TODO The py3 list should be updated to include 3.4 + + ---------- Footnotes ---------- + + (1) https://nedbatchelder.com/blog/201109/whats_in_which_python.html + + (2) +https://nedbatchelder.com/blog/201310/whats_in_which_python_3.html + + (3) +https://nedbatchelder.com/blog/201803/whats_in_which_python_3436.html + + +File: pythonpackagingguide.info, Node: Dropping support for older Python versions, Next: Packaging binary extensions, Prev: Supporting multiple Python versions, Up: Guides + +3.11 Dropping support for older Python versions +=============================================== + +Dropping support for older Python versions is supported by the standard +*note Core metadata specifications: 85. 1.2 specification via a +“Requires-Python” attribute. + +Metadata 1.2+ clients, such as Pip 9.0+, will adhere to this +specification by matching the current Python runtime and comparing it +with the required version in the package metadata. If they do not +match, it will attempt to install the last package distribution that +supported that Python runtime. + +This mechanism can be used to drop support for older Python versions, by +amending the “Requires-Python” attribute in the package metadata. + +This guide is specifically for users of *note setuptools: 2d, other +packaging tools such as ‘flit’ may offer similar functionality but users +will need to consult relevant documentation. + +* Menu: + +* Requirements:: +* Defining the Python version required:: +* Dropping a Python release:: + + +File: pythonpackagingguide.info, Node: Requirements, Next: Defining the Python version required, Up: Dropping support for older Python versions + +3.11.1 Requirements +------------------- + +This workflow requires that: + + 1. The publisher is using the latest version of *note setuptools: 2d, + + 2. The latest version of *note twine: 66. is used to upload the + package, + + 3. The user installing the package has at least Pip 9.0, or a client + that supports the Metadata 1.2 specification. + + +File: pythonpackagingguide.info, Node: Defining the Python version required, Next: Dropping a Python release, Prev: Requirements, Up: Dropping support for older Python versions + +3.11.2 Defining the Python version required +------------------------------------------- + +* Menu: + +* 1. Download the newest version of Setuptools: 1 Download the newest version of Setuptools. +* 2. Specify the version ranges for supported Python distributions: 2 Specify the version ranges for supported Python distributions. +* 3. Validating the Metadata before publishing: 3 Validating the Metadata before publishing. +* 4. Using Twine to publish: 4 Using Twine to publish. + + +File: pythonpackagingguide.info, Node: 1 Download the newest version of Setuptools, Next: 2 Specify the version ranges for supported Python distributions, Up: Defining the Python version required + +3.11.2.1 1. Download the newest version of Setuptools +..................................................... + +Ensure that before you generate source distributions or binary +distributions, you update Setuptools and install twine. + +Steps: + + pip install --upgrade setuptools twine + +*note setuptools: 2d. version should be above 24.0.0. + + +File: pythonpackagingguide.info, Node: 2 Specify the version ranges for supported Python distributions, Next: 3 Validating the Metadata before publishing, Prev: 1 Download the newest version of Setuptools, Up: Defining the Python version required + +3.11.2.2 2. Specify the version ranges for supported Python distributions +......................................................................... + +You can specify version ranges and exclusion rules, such as at least +Python 3. Or, Python 2.7, 3.4 and beyond. + +Examples: + + Requires-Python: ">=3" + Requires-Python: ">2.7,!=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +The way to set those values is within the call to ‘setup’ within your +‘setup.py’ script. This will insert the ‘Requires-Python’ metadata +values based on the argument you provide in ‘python_requires’. + + from setuptools import setup + + + setup( + # Your setup arguments + python_requires='>=2.7', # Your supported Python ranges + ) + + +File: pythonpackagingguide.info, Node: 3 Validating the Metadata before publishing, Next: 4 Using Twine to publish, Prev: 2 Specify the version ranges for supported Python distributions, Up: Defining the Python version required + +3.11.2.3 3. Validating the Metadata before publishing +..................................................... + +Within a Python source package (the zip or the tar-gz file you download) +is a text file called PKG-INFO. + +This file is generated by Distutils or *note setuptools: 2d. when it +generates the source package. The file contains a set of keys and +values, the list of keys is part of the PyPa standard metadata format. + +You can see the contents of the generated file like this: + + tar xfO dist/my-package-1.0.0.tar.gz my-package-1.0.0/PKG-INFO + +Validate that the following is in place, before publishing the package: + + - If you have upgraded correctly, the Metadata-Version value should + be 1.2 or higher. + + - The Requires-Python field is set and matches your specification in + setup.py. + + +File: pythonpackagingguide.info, Node: 4 Using Twine to publish, Prev: 3 Validating the Metadata before publishing, Up: Defining the Python version required + +3.11.2.4 4. Using Twine to publish +.................................. + +Twine has a number of advantages, apart from being faster it is now the +supported method for publishing packages. + +Make sure you are using the newest version of Twine, at least 1.9. + + +File: pythonpackagingguide.info, Node: Dropping a Python release, Prev: Defining the Python version required, Up: Dropping support for older Python versions + +3.11.3 Dropping a Python release +-------------------------------- + +Once you have published a package with the Requires-Python metadata, you +can then make a further update removing that Python runtime from +support. + +It must be done in this order for the automated fallback to work. + +For example, you published the Requires-Python: “>=2.7” as version 1.0.0 +of your package. + +If you were then to update the version string to “>=3.5”, and publish a +new version 2.0.0 of your package, any users running Pip 9.0+ from +version 2.7 will have version 1.0.0 of the package installed, and any +>=3.5 users will receive version 2.0.0. + + +File: pythonpackagingguide.info, Node: Packaging binary extensions, Next: Supporting Windows using Appveyor, Prev: Dropping support for older Python versions, Up: Guides + +3.12 Packaging binary extensions +================================ + + +Page Status: Incomplete + + +Last Reviewed: 2013-12-08 + +One of the features of the CPython reference interpreter is that, in +addition to allowing the execution of Python code, it also exposes a +rich C API for use by other software. One of the most common uses of +this C API is to create importable C extensions that allow things which +aren’t always easy to achieve in pure Python code. + +* Menu: + +* An overview of binary extensions:: +* Implementing binary extensions:: +* Building binary extensions:: +* Publishing binary extensions:: +* Additional resources:: + + +File: pythonpackagingguide.info, Node: An overview of binary extensions, Next: Implementing binary extensions, Up: Packaging binary extensions + +3.12.1 An overview of binary extensions +--------------------------------------- + +* Menu: + +* Use cases:: +* Disadvantages:: +* Alternatives to handcoded accelerator modules:: +* Alternatives to handcoded wrapper modules:: +* Alternatives for low level system access:: + + +File: pythonpackagingguide.info, Node: Use cases, Next: Disadvantages, Up: An overview of binary extensions + +3.12.1.1 Use cases +.................. + +The typical use cases for binary extensions break down into just three +conventional categories: + + * `accelerator modules': these modules are completely self-contained, + and are created solely to run faster than the equivalent pure + Python code runs in CPython. Ideally, accelerator modules will + always have a pure Python equivalent to use as a fallback if the + accelerated version isn’t available on a given system. The CPython + standard library makes extensive use of accelerator modules. + `Example': When importing ‘datetime’, Python falls back to the + datetime.py(1) module if the C implementation ( + _datetimemodule.c(2)) is not available. + + * `wrapper modules': these modules are created to expose existing C + interfaces to Python code. They may either expose the underlying C + interface directly, or else expose a more “Pythonic” API that makes + use of Python language features to make the API easier to use. The + CPython standard library makes extensive use of wrapper modules. + `Example': functools.py(3) is a Python module wrapper for + _functoolsmodule.c(4). + + * `low-level system access': these modules are created to access + lower level features of the CPython runtime, the operating system, + or the underlying hardware. Through platform specific code, + extension modules may achieve things that aren’t possible in pure + Python code. A number of CPython standard library modules are + written in C in order to access interpreter internals that aren’t + exposed at the language level. `Example': ‘sys’, which comes from + sysmodule.c(5). + + One particularly notable feature of C extensions is that, when they + don’t need to call back into the interpreter runtime, they can + release CPython’s global interpreter lock around long-running + operations (regardless of whether those operations are CPU or IO + bound). + +Not all extension modules will fit neatly into the above categories. +The extension modules included with NumPy, for example, span all three +use cases - they move inner loops to C for speed reasons, wrap external +libraries written in C, FORTRAN and other languages, and use low level +system interfaces for both CPython and the underlying operation system +to support concurrent execution of vectorised operations and to tightly +control the exact memory layout of created objects. + + ---------- Footnotes ---------- + + (1) https://github.com/python/cpython/blob/master/Lib/datetime.py + + (2) +https://github.com/python/cpython/blob/master/Modules/_datetimemodule.c + + (3) https://github.com/python/cpython/blob/master/Lib/functools.py + + (4) +https://github.com/python/cpython/blob/master/Modules/_functoolsmodule.c + + (5) https://github.com/python/cpython/blob/master/Python/sysmodule.c + + +File: pythonpackagingguide.info, Node: Disadvantages, Next: Alternatives to handcoded accelerator modules, Prev: Use cases, Up: An overview of binary extensions + +3.12.1.2 Disadvantages +...................... + +The main disadvantage of using binary extensions is the fact that it +makes subsequent distribution of the software more difficult. One of +the advantages of using Python is that it is largely cross platform, and +the languages used to write extension modules (typically C or C++, but +really any language that can bind to the CPython C API) typically +require that custom binaries be created for different platforms. + +This means that binary extensions: + + * require that end users be able to either build them from source, or + else that someone publish pre-built binaries for common platforms + + * may not be compatible with different builds of the CPython + reference interpreter + + * often will not work correctly with alternative interpreters such as + PyPy, IronPython or Jython + + * if handcoded, make maintenance more difficult by requiring that + maintainers be familiar not only with Python, but also with the + language used to create the binary extension, as well as with the + details of the CPython C API. + + * if a pure Python fallback implementation is provided, make + maintenance more difficult by requiring that changes be implemented + in two places, and introducing additional complexity in the test + suite to ensure both versions are always executed. + +Another disadvantage of relying on binary extensions is that alternative +import mechanisms (such as the ability to import modules directly from +zipfiles) often won’t work for extension modules (as the dynamic loading +mechanisms on most platforms can only load libraries from disk). + + +File: pythonpackagingguide.info, Node: Alternatives to handcoded accelerator modules, Next: Alternatives to handcoded wrapper modules, Prev: Disadvantages, Up: An overview of binary extensions + +3.12.1.3 Alternatives to handcoded accelerator modules +...................................................... + +When extension modules are just being used to make code run faster +(after profiling has identified the code where the speed increase is +worth additional maintenance effort), a number of other alternatives +should also be considered: + + * look for existing optimised alternatives. The CPython standard + libary includes a number of optimised data structures and + algorithms (especially in the builtins and the ‘collections’ and + ‘itertools’ modules). The Python Package Index also offers + additional alternatives. Sometimes, the appropriate choice of + standard library or third party module can avoid the need to create + your own accelerator module. + + * for long running applications, the JIT compiled PyPy interpreter(1) + may offer a suitable alternative to the standard CPython runtime. + The main barrier to adopting PyPy is typically reliance on other + binary extension modules - while PyPy does emulate the CPython C + API, modules that rely on that cause problems for the PyPy JIT, and + the emulation layer can often expose latent defects in extension + modules that CPython currently tolerates (frequently around + reference counting errors - an object having one live reference + instead of two often won’t break anything, but no references + instead of one is a major problem). + + * Cython(2) is a mature static compiler that can compile most Python + code to C extension modules. The initial compilation provides some + speed increases (by bypassing the CPython interpreter layer), and + Cython’s optional static typing features can offer additional + opportunities for speed increases. Using Cython still has the + disadvantage of increasing the complexity of distributing the + resulting application, but has the benefit of having a reduced + barrier to entry for Python programmers (relative to other + languages like C or C++). + + * Numba(3) is a newer tool, created by members of the scientific + Python community, that aims to leverage LLVM to allow selective + compilation of pieces of a Python application to native machine + code at runtime. It requires that LLVM be available on the system + where the code is running, but can provide significant speed + increases, especially for operations that are amenable to + vectorisation. + + ---------- Footnotes ---------- + + (1) http://pypy.org/ + + (2) http://cython.org/ + + (3) http://numba.pydata.org/ + + +File: pythonpackagingguide.info, Node: Alternatives to handcoded wrapper modules, Next: Alternatives for low level system access, Prev: Alternatives to handcoded accelerator modules, Up: An overview of binary extensions + +3.12.1.4 Alternatives to handcoded wrapper modules +.................................................. + +The C ABI (Application Binary Interface) is a common standard for +sharing functionality between multiple applications. One of the +strengths of the CPython C API (Application Programming Interface) is +allowing Python users to tap into that functionality. However, wrapping +modules by hand is quite tedious, so a number of other alternative +approaches should be considered. + +The approaches described below don’t simplify the distribution case at +all, but they `can' significantly reduce the maintenance burden of +keeping wrapper modules up to date. + + * In addition to being useful for the creation of accelerator + modules, Cython(1) is also useful for creating wrapper modules. It + still involves wrapping the interfaces by hand, however, so may not + be a good choice for wrapping large APIs. + + * cffi(2) is a project created by some of the PyPy developers to make + it straightforward for developers that already know both Python and + C to expose their C modules to Python applications. It also makes + it relatively straightforward to wrap a C module based on its + header files, even if you don’t know C yourself. + + One of the key advantages of ‘cffi’ is that it is compatible with + the PyPy JIT, allowing CFFI wrapper modules to participate fully in + PyPy’s tracing JIT optimisations. + + * SWIG(3) is a wrapper interface generator that allows a variety of + programming languages, including Python, to interface with C `and + C++' code. + + * The standard library’s ‘ctypes’ module, while useful for getting + access to C level interfaces when header information isn’t + available, suffers from the fact that it operates solely at the C + ABI level, and thus has no automatic consistency checking between + the interface actually being exported by the library and the one + declared in the Python code. By contrast, the above alternatives + are all able to operate at the C `API' level, using C header files + to ensure consistency between the interface exported by the library + being wrapped and the one expected by the Python wrapper module. + While ‘cffi’ `can' operate directly at the C ABI level, it suffers + from the same interface inconsistency problems as ‘ctypes’ when it + is used that way. + + ---------- Footnotes ---------- + + (1) http://cython.org/ + + (2) https://cffi.readthedocs.io/ + + (3) http://www.swig.org/ + + +File: pythonpackagingguide.info, Node: Alternatives for low level system access, Prev: Alternatives to handcoded wrapper modules, Up: An overview of binary extensions + +3.12.1.5 Alternatives for low level system access +................................................. + +For applications that need low level system access (regardless of the +reason), a binary extension module often `is' the best way to go about +it. This is particularly true for low level access to the CPython +runtime itself, since some operations (like releasing the Global +Interpreter Lock) are simply invalid when the interpreter is running +code, even if a module like ‘ctypes’ or ‘cffi’ is used to obtain access +to the relevant C API interfaces. + +For cases where the extension module is manipulating the underlying +operating system or hardware (rather than the CPython runtime), it may +sometimes be better to just write an ordinary C library (or a library in +another systems programming language like C++ or Rust that can export a +C compatible ABI), and then use one of the wrapping techniques described +above to make the interface available as an importable Python module. + + +File: pythonpackagingguide.info, Node: Implementing binary extensions, Next: Building binary extensions, Prev: An overview of binary extensions, Up: Packaging binary extensions + +3.12.2 Implementing binary extensions +------------------------------------- + +The CPython Extending and Embedding(1) guide includes an introduction to +writing a custom extension module in C(2). + + mention the stable ABI (3.2+, link to the CPython C API docs) + mention the module lifecycle + mention the challenges of shared static state and subinterpreters + mention the implications of the GIL for extension modules + mention the memory allocation APIs in 3.4+ + + mention again that all this is one of the reasons why you probably + *don't* want to handcode your extension modules :) + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/extending/ + + (2) https://docs.python.org/3/extending/extending.html + + +File: pythonpackagingguide.info, Node: Building binary extensions, Next: Publishing binary extensions, Prev: Implementing binary extensions, Up: Packaging binary extensions + +3.12.3 Building binary extensions +--------------------------------- + +* Menu: + +* Binary extensions for Windows:: +* Binary extensions for Linux:: +* Binary extensions for macOS:: + + +File: pythonpackagingguide.info, Node: Binary extensions for Windows, Next: Binary extensions for Linux, Up: Building binary extensions + +3.12.3.1 Binary extensions for Windows +...................................... + +Before it is possible to build a binary extension, it is necessary to +ensure that you have a suitable compiler available. On Windows, Visual +C is used to build the official CPython interpreter, and should be used +to build compatible binary extensions. + +Python 2.7 used Visual Studio 2008, Python 3.3 and 3.4 used Visual +Studio 2010, and Python 3.5+ uses Visual Studio 2015 or later. +Unfortunately, older versions of Visual Studio are no longer easily +available from Microsoft, so for versions of Python prior to 3.5, the +compilers must be obtained differently if you do not already have a copy +of the relevant version of Visual Studio. + +To set up a build environment for binary extensions, the steps are as +follows: + + For Python 2.7 + + 1. Install “Visual C++ Compiler Package for Python 2.7”, + which is available from Microsoft’s website(1). + + 2. Use (a recent version of) setuptools in your setup.py + (pip will do this for you, in any case). + + 3. Done. + + For Python 3.4 + + 1. Install “Windows SDK for Windows 7 and .NET Framework 4” + (v7.1), which is available from Microsoft’s website(2). + + 2. Work from an SDK command prompt (with the environment + variables set, and the SDK on PATH). + + 3. Set DISTUTILS_USE_SDK=1 + + 4. Done. + + For Python 3.5 + + 1. Install Visual Studio 2015 Community Edition(3) (or any + later version, when these are released). + + 2. Done. + +Note that from Python 3.5 onwards, Visual Studio works in a backward +compatible way, which means that any future version of Visual Studio +will be able to build Python extensions for all Python versions from 3.5 +onwards. + +Building with the recommended compiler on Windows ensures that a +compatible C library is used throughout the Python process. + + ---------- Footnotes ---------- + + (1) https://www.microsoft.com/en-gb/download/details.aspx?id=44266 + + (2) https://www.microsoft.com/en-gb/download/details.aspx?id=8279 + + (3) +https://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx + + +File: pythonpackagingguide.info, Node: Binary extensions for Linux, Next: Binary extensions for macOS, Prev: Binary extensions for Windows, Up: Building binary extensions + +3.12.3.2 Binary extensions for Linux +.................................... + +Linux binaries must use a sufficiently old glibc to be compatible with +older distributions. The manylinux(1) Docker images provide a build +environment with a glibc old enough to support most current Linux +distributions on common architectures. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/manylinux + + +File: pythonpackagingguide.info, Node: Binary extensions for macOS, Prev: Binary extensions for Linux, Up: Building binary extensions + +3.12.3.3 Binary extensions for macOS +.................................... + +Binary compatibility on macOS is determined by the target minimum +deployment system, e.g. `10.9', which is often specified with the +‘MACOSX_DEPLOYMENT_TARGET’ environmental variable when building binaries +on macOS. When building with setuptools / distutils, the deployment +target is specified with the flag ‘--plat-name’, e.g. +‘macosx-10.9-x86_64’. For common deployment targets for macOS Python +distributions, see the MacPython Spinning Wheels wiki(1). + + ---------- Footnotes ---------- + + (1) https://github.com/MacPython/wiki/wiki/Spinning-wheels + + +File: pythonpackagingguide.info, Node: Publishing binary extensions, Next: Additional resources, Prev: Building binary extensions, Up: Packaging binary extensions + +3.12.4 Publishing binary extensions +----------------------------------- + +For interim guidance on this topic, see the discussion in this issue(1). + + FIXME + + cover publishing as wheel files on PyPI or a custom index server + cover creation of Windows and macOS installers + cover weak linking + mention the fact that Linux distros have a requirement to build from + source in their own build systems, so binary-only releases are strongly + discouraged + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/python-packaging-user-guide/issues/284 + + +File: pythonpackagingguide.info, Node: Additional resources, Prev: Publishing binary extensions, Up: Packaging binary extensions + +3.12.5 Additional resources +--------------------------- + +Cross-platform development and distribution of extension modules is a +complex topic, so this guide focuses primarily on providing pointers to +various tools that automate dealing with the underlying technical +challenges. The additional resources in this section are instead +intended for developers looking to understand more about the underlying +binary interfaces that those systems rely on at runtime. + +* Menu: + +* Cross-platform wheel generation with scikit-build:: +* Introduction to C/C++ extension modules:: + + +File: pythonpackagingguide.info, Node: Cross-platform wheel generation with scikit-build, Next: Introduction to C/C++ extension modules, Up: Additional resources + +3.12.5.1 Cross-platform wheel generation with scikit-build +.......................................................... + +The scikit-build(1) package helps abstract cross-platform build +operations and provides additional capabilities when creating binary +extension packages. Additional documentation is also available on the C +runtime, compiler, and build system generator(2) for Python binary +extension modules. + + ---------- Footnotes ---------- + + (1) https://scikit-build.readthedocs.io/en/latest/ + + (2) https://scikit-build.readthedocs.io/en/latest/generators.html + + +File: pythonpackagingguide.info, Node: Introduction to C/C++ extension modules, Prev: Cross-platform wheel generation with scikit-build, Up: Additional resources + +3.12.5.2 Introduction to C/C++ extension modules +................................................ + +For a more in depth explanation of how extension modules are used by +CPython on a Debian system, see the following articles: + + * What are (c)python extension modules?(1) + + * Releasing the gil(2) + + * Writing cpython extension modules using C++(3) + + ---------- Footnotes ---------- + + (1) https://thomasnyberg.com/what_are_extension_modules.html + + (2) https://thomasnyberg.com/releasing_the_gil.html + + (3) https://thomasnyberg.com/cpp_extension_modules.html + + +File: pythonpackagingguide.info, Node: Supporting Windows using Appveyor, Next: Packaging namespace packages, Prev: Packaging binary extensions, Up: Guides + +3.13 Supporting Windows using Appveyor +====================================== + + +Page Status: Incomplete + + +Last Reviewed: 2015-12-03 + +This section covers how to use the free Appveyor(1) continuous +integration service to provide Windows support for your project. This +includes testing the code on Windows, and building Windows-targeted +binaries for projects that use C extensions. + +* Menu: + +* Background:: +* Setting up:: +* Adding Appveyor support to your project:: +* Additional notes:: + + ---------- Footnotes ---------- + + (1) http://www.appveyor.com/ + + +File: pythonpackagingguide.info, Node: Background, Next: Setting up, Up: Supporting Windows using Appveyor + +3.13.1 Background +----------------- + +Many projects are developed on Unix by default, and providing Windows +support can be a challenge, because setting up a suitable Windows test +environment is non-trivial, and may require buying software licenses. + +The Appveyor service is a continuous integration service, much like the +better-known Travis(1) service that is commonly used for testing by +projects hosted on Github(2). However, unlike Travis, the build workers +on Appveyor are Windows hosts and have the necessary compilers installed +to build Python extensions. + +Windows users typically do not have access to a C compiler, and +therefore are reliant on projects that use C extensions distributing +binary wheels on PyPI in order for the distribution to be installable +via ‘pip install ’. By using Appveyor as a build service (even if +not using it for testing) it is possible for projects without a +dedicated Windows environment to provide Windows-targeted binaries. + + ---------- Footnotes ---------- + + (1) https://travis-ci.org/ + + (2) https://github.org/ + + +File: pythonpackagingguide.info, Node: Setting up, Next: Adding Appveyor support to your project, Prev: Background, Up: Supporting Windows using Appveyor + +3.13.2 Setting up +----------------- + +In order to use Appveyor to build Windows wheels for your project, you +must have an account on the service. Instructions on setting up an +account are given in the Appveyor documentation(1). The free tier of +account is perfectly adequate for open source projects. + +Appveyor provides integration with Github(2) and Bitbucket(3), so as +long as your project is hosted on one of those two services, setting up +Appveyor integration is straightforward. + +Once you have set up your Appveyor account and added your project, +Appveyor will automatically build your project each time a commit +occurs. This behaviour will be familiar to users of Travis. + + ---------- Footnotes ---------- + + (1) http://www.appveyor.com/docs + + (2) https://github.org/ + + (3) https://bitbucket.org/ + + +File: pythonpackagingguide.info, Node: Adding Appveyor support to your project, Next: Additional notes, Prev: Setting up, Up: Supporting Windows using Appveyor + +3.13.3 Adding Appveyor support to your project +---------------------------------------------- + +In order to define how Appveyor should build your project, you need to +add an ‘appveyor.yml’ file to your project. The full details of what +can be included in the file are covered in the Appveyor documentation. +This guide will provide the details necessary to set up wheel builds. + +Appveyor includes by default all of the compiler toolchains needed to +build extensions for Python. For Python 2.7, 3.5+ and 32-bit versions +of 3.3 and 3.4, the tools work out of the box. But for 64-bit versions +of Python 3.3 and 3.4, there is a small amount of additional +configuration needed to let distutils know where to find the 64-bit +compilers. (From 3.5 onwards, the version of Visual Studio used +includes 64-bit compilers with no additional setup). + +* Menu: + +* appveyor.yml: appveyor yml. +* Support script:: +* Access to the built wheels:: + + +File: pythonpackagingguide.info, Node: appveyor yml, Next: Support script, Up: Adding Appveyor support to your project + +3.13.3.1 appveyor.yml +..................... + + environment: + + matrix: + + # For Python versions available on Appveyor, see + # https://www.appveyor.com/docs/windows-images-software/#python + # The list here is complete (excluding Python 2.6, which + # isn't covered by this document) at the time of writing. + + - PYTHON: "C:\\Python27" + - PYTHON: "C:\\Python33" + - PYTHON: "C:\\Python34" + - PYTHON: "C:\\Python35" + - PYTHON: "C:\\Python27-x64" + - PYTHON: "C:\\Python33-x64" + DISTUTILS_USE_SDK: "1" + - PYTHON: "C:\\Python34-x64" + DISTUTILS_USE_SDK: "1" + - PYTHON: "C:\\Python35-x64" + - PYTHON: "C:\\Python36-x64" + + install: + # We need wheel installed to build wheels + - "%PYTHON%\\python.exe -m pip install wheel" + + build: off + + test_script: + # Put your test command here. + # If you don't need to build C extensions on 64-bit Python 3.3 or 3.4, + # you can remove "build.cmd" from the front of the command, as it's + # only needed to support those cases. + # Note that you must use the environment variable %PYTHON% to refer to + # the interpreter you're using - Appveyor does not do anything special + # to put the Python version you want to use on PATH. + - "build.cmd %PYTHON%\\python.exe setup.py test" + + after_test: + # This step builds your wheels. + # Again, you only need build.cmd if you're building C extensions for + # 64-bit Python 3.3/3.4. And you need to use %PYTHON% to get the correct + # interpreter + - "build.cmd %PYTHON%\\python.exe setup.py bdist_wheel" + + artifacts: + # bdist_wheel puts your built wheel in the dist directory + - path: dist\* + + #on_success: + # You can use this step to upload your artifacts to a public website. + # See Appveyor's documentation for more details. Or you can simply + # access your wheels from the Appveyor "artifacts" tab for your build. + +This file can be downloaded from here(1). + +The ‘appveyor.yml’ file must be located in the root directory of your +project. It is in ‘YAML’ format, and consists of a number of sections. + +The ‘environment’ section is the key to defining the Python versions for +which your wheels will be created. Appveyor comes with Python 2.6, 2.7, +3.3, 3.4 and 3.5 installed, in both 32-bit and 64-bit builds. The +example file builds for all of these environments except Python 2.6. +Installing for Python 2.6 is more complex, as it does not come with pip +included. We don’t support 2.6 in this document (as Windows users still +using Python 2 are generally able to move to Python 2.7 without too much +difficulty). + +The ‘install’ section uses pip to install any additional software that +the project may require. The only requirement for building wheels is +the ‘wheel’ project, but projects may wish to customise this code in +certain circumstances (for example, to install additional build packages +such as ‘Cython’, or test tools such as ‘tox’). + +The ‘build’ section simply switches off builds - there is no build step +needed for Python, unlike languages like ‘C#’. + +The main sections that will need to be tailored to your project are +‘test_script’ and ‘after_test’. + +The ‘test_script’ section is where you will run your project’s tests. +The supplied file runs your test suite using ‘setup.py test’. If you +are only interested in building wheels, and not in running your tests on +Windows, you can replace this section with a dummy command such as ‘echo +Skipped Tests’. You may wish to use another test tool, such as ‘nose’ +or ‘py.test’. Or you may wish to use a test driver like ‘tox’ - however +if you are using ‘tox’ there are some additional configuration changes +you will need to consider, which are described below. + +The ‘after_test’ runs once your tests have completed, and so is where +the wheels should be built. Assuming your project uses the recommended +tools (specifically, ‘setuptools’) then the ‘setup.py bdist_wheel’ +command will build your wheels. + +Note that wheels will only be built if your tests succeed. If you +expect your tests to fail on Windows, you can skip them as described +above. + + ---------- Footnotes ---------- + + (1) +https://raw.githubusercontent.com/pypa/python-packaging-user-guide/master/source/guides/appveyor-sample/appveyor.yml + + +File: pythonpackagingguide.info, Node: Support script, Next: Access to the built wheels, Prev: appveyor yml, Up: Adding Appveyor support to your project + +3.13.3.2 Support script +....................... + +The ‘appveyor.yml’ file relies on a single support script, which sets up +the environment to use the SDK compiler for 64-bit builds on Python 3.3 +and 3.4. For projects which do not need a compiler, or which don’t +support 3.3 or 3.4 on 64-bit Windows, only the ‘appveyor.yml’ file is +needed. + +build.cmd(1) is a Windows batch script that runs a single command in an +environment with the appropriate compiler for the selected Python +version. All you need to do is to set the single environment variable +‘DISTUTILS_USE_SDK’ to a value of ‘1’ and the script does the rest. It +sets up the SDK needed for 64-bit builds of Python 3.3 or 3.4, so don’t +set the environment variable for any other builds. + +You can simply download the batch file and include it in your project +unchanged. + + ---------- Footnotes ---------- + + (1) +https://raw.githubusercontent.com/pypa/python-packaging-user-guide/master/source/guides/appveyor-sample/build.cmd + + +File: pythonpackagingguide.info, Node: Access to the built wheels, Prev: Support script, Up: Adding Appveyor support to your project + +3.13.3.3 Access to the built wheels +................................... + +When your build completes, the built wheels will be available from the +Appveyor control panel for your project. They can be found by going to +the build status page for each build in turn. At the top of the build +output there is a series of links, one of which is “Artifacts”. That +page will include a list of links to the wheels for that Python version +/ architecture. You can download those wheels and upload them to PyPI +as part of your release process. + + +File: pythonpackagingguide.info, Node: Additional notes, Prev: Adding Appveyor support to your project, Up: Supporting Windows using Appveyor + +3.13.4 Additional notes +----------------------- + +* Menu: + +* Testing with tox:: +* Automatically uploading wheels:: +* External dependencies:: +* Support scripts:: + + +File: pythonpackagingguide.info, Node: Testing with tox, Next: Automatically uploading wheels, Up: Additional notes + +3.13.4.1 Testing with tox +......................... + +Many projects use the Tox(1) tool to run their tests. It ensures that +tests are run in an isolated environment using the exact files that will +be distributed by the project. + +In order to use ‘tox’ on Appveyor there are a couple of additional +considerations (in actual fact, these issues are not specific to +Appveyor, and may well affect other CI systems). + + 1. By default, ‘tox’ only passes a chosen subset of environment + variables to the test processes. Because ‘distutils’ uses + environment variables to control the compiler, this “test + isolation” feature will cause the tests to use the wrong compiler + by default. + + To force ‘tox’ to pass the necessary environment variables to the + subprocess, you need to set the ‘tox’ configuration option + ‘passenv’ to list the additional environment variables to be passed + to the subprocess. For the SDK compilers, you need + + - ‘DISTUTILS_USE_SDK’ + + - ‘MSSdk’ + + - ‘INCLUDE’ + + - ‘LIB’ + + The ‘passenv’ option can be set in your ‘tox.ini’, or if you + prefer to avoid adding Windows-specific settings to your + general project files, it can be set by setting the + ‘TOX_TESTENV_PASSENV’ environment variable. The supplied + ‘build.cmd’ script does this by default whenever + ‘DISTUTILS_USE_SDK’ is set. + + 2. When used interactively, ‘tox’ allows you to run your tests against + multiple environments (often, this means multiple Python versions). + This feature is not as useful in a CI environment like Travis or + Appveyor, where all tests are run in isolated environments for each + configuration. As a result, projects often supply an argument ‘-e + ENVNAME’ to ‘tox’ to specify which environment to use (there are + default environments for most versions of Python). + + However, this does `not' work well with a Windows CI system + like Appveyor, where there are (for example) two installations + of Python 3.4 (32-bit and 64-bit) available, but only one + ‘py34’ environment in ‘tox’. + + In order to run tests using ‘tox’, therefore, projects should + probably use the default ‘py’ environment in ‘tox’, which uses + the Python interpreter that was used to run ‘tox’. This will + ensure that when Appveyor runs the tests, they will be run + with the configured interpreter. + + In order to support running under the ‘py’ environment, it is + possible that projects with complex ‘tox’ configurations might + need to modify their ‘tox.ini’ file. Doing so is, however, + outside the scope of this document. + + ---------- Footnotes ---------- + + (1) http://tox.testrun.org + + +File: pythonpackagingguide.info, Node: Automatically uploading wheels, Next: External dependencies, Prev: Testing with tox, Up: Additional notes + +3.13.4.2 Automatically uploading wheels +....................................... + +It is possible to request Appveyor to automatically upload wheels. +There is a ‘deployment’ step available in ‘appveyor.yml’ that can be +used to (for example) copy the built artifacts to a FTP site, or an +Amazon S3 instance. Documentation on how to do this is included in the +Appveyor guides. + +Alternatively, it would be possible to add a ‘twine upload’ step to the +build. The supplied ‘appveyor.yml’ does not do this, as it is not clear +that uploading new wheels after every commit is desirable (although some +projects may wish to do this). + + +File: pythonpackagingguide.info, Node: External dependencies, Next: Support scripts, Prev: Automatically uploading wheels, Up: Additional notes + +3.13.4.3 External dependencies +.............................. + +The supplied scripts will successfully build any distribution that does +not rely on 3rd party external libraries for the build. + +It is possible to add steps to the ‘appveyor.yml’ configuration +(typically in the “install” section) to download and/or build external +libraries needed by the distribution. And if needed, it is possible to +add extra configuration for the build to supply the location of these +libraries to the compiler. However, this level of configuration is +beyond the scope of this document. + + +File: pythonpackagingguide.info, Node: Support scripts, Prev: External dependencies, Up: Additional notes + +3.13.4.4 Support scripts +........................ + +For reference, the SDK setup support script is listed here: + +‘appveyor-sample/build.cmd’ + + @echo off + :: To build extensions for 64 bit Python 3, we need to configure environment + :: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of: + :: MS Windows SDK for Windows 7 and .NET Framework 4 + :: + :: More details at: + :: https://github.com/cython/cython/wiki/CythonExtensionsOnWindows + + IF "%DISTUTILS_USE_SDK%"=="1" ( + ECHO Configuring environment to build with MSVC on a 64bit architecture + ECHO Using Windows SDK 7.1 + "C:\Program Files\Microsoft SDKs\Windows\v7.1\Setup\WindowsSdkVer.exe" -q -version:v7.1 + CALL "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /x64 /release + SET MSSdk=1 + REM Need the following to allow tox to see the SDK compiler + SET TOX_TESTENV_PASSENV=DISTUTILS_USE_SDK MSSdk INCLUDE LIB + ) ELSE ( + ECHO Using default MSVC build environment + ) + + CALL %* + + +File: pythonpackagingguide.info, Node: Packaging namespace packages, Next: Creating and discovering plugins, Prev: Supporting Windows using Appveyor, Up: Guides + +3.14 Packaging namespace packages +================================= + +Namespace packages allow you to split the sub-packages and modules +within a single *note package: a. across multiple, separate *note +distribution packages: b. (referred to as `distributions' in this +document to avoid ambiguity). For example, if you have the following +package structure: + + mynamespace/ + __init__.py + subpackage_a/ + __init__.py + ... + subpackage_b/ + __init__.py + ... + module_b.py + setup.py + +And you use this package in your code like so: + + from mynamespace import subpackage_a + from mynamespace import subpackage_b + +Then you can break these sub-packages into two separate distributions: + + mynamespace-subpackage-a/ + setup.py + mynamespace/ + subpackage_a/ + __init__.py + + mynamespace-subpackage-b/ + setup.py + mynamespace/ + subpackage_b/ + __init__.py + module_b.py + +Each sub-package can now be separately installed, used, and versioned. + +Namespace packages can be useful for a large collection of +loosely-related packages (such as a large corpus of client libraries for +multiple products from a single company). However, namespace packages +come with several caveats and are not appropriate in all cases. A +simple alternative is to use a prefix on all of your distributions such +as ‘import mynamespace_subpackage_a’ (you could even use ‘import +mynamespace_subpackage_a as subpackage_a’ to keep the import object +short). + +* Menu: + +* Creating a namespace package:: + + +File: pythonpackagingguide.info, Node: Creating a namespace package, Up: Packaging namespace packages + +3.14.1 Creating a namespace package +----------------------------------- + +There are currently three different approaches to creating namespace +packages: + + 1. Use *note native namespace packages: 134. This type of namespace + package is defined in PEP 420(1) and is available in Python 3.3 and + later. This is recommended if packages in your namespace only ever + need to support Python 3 and installation via ‘pip’. + + 2. Use *note pkgutil-style namespace packages: 135. This is + recommended for new packages that need to support Python 2 and 3 + and installation via both ‘pip’ and ‘python setup.py install’. + + 3. Use *note pkg_resources-style namespace packages: 136. This method + is recommended if you need compatibility with packages already + using this method or if your package needs to be zip-safe. + + Warning: While native namespace packages and pkgutil-style + namespace packages are largely compatible, pkg_resources-style + namespace packages are not compatible with the other methods. It’s + inadvisable to use different methods in different distributions + that provide packages to the same namespace. + +* Menu: + +* Native namespace packages:: +* pkgutil-style namespace packages:: +* pkg_resources-style namespace packages:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0420 + + +File: pythonpackagingguide.info, Node: Native namespace packages, Next: pkgutil-style namespace packages, Up: Creating a namespace package + +3.14.1.1 Native namespace packages +.................................. + +Python 3.3 added `implicit' namespace packages from PEP 420(1). All +that is required to create a native namespace package is that you just +omit ‘__init__.py’ from the namespace package directory. An example +file structure: + + setup.py + mynamespace/ + # No __init__.py here. + subpackage_a/ + # Sub-packages have __init__.py. + __init__.py + module.py + +It is extremely important that every distribution that uses the +namespace package omits the ‘__init__.py’ or uses a pkgutil-style +‘__init__.py’. If any distribution does not, it will cause the +namespace logic to fail and the other sub-packages will not be +importable. + +Because ‘mynamespace’ doesn’t contain an ‘__init__.py’, +‘setuptools.find_packages()’ won’t find the sub-package. You must use +‘setuptools.find_namespace_packages()’ instead or explicitly list all +packages in your ‘setup.py’. For example: + + from setuptools import setup, find_namespace_packages + + setup( + name='mynamespace-subpackage-a', + ... + packages=find_namespace_packages(include=['mynamespace.*']) + ) + +A complete working example of two native namespace packages can be found +in the native namespace package example project(2). + + Note: Because native and pkgutil-style namespace packages are + largely compatible, you can use native namespace packages in the + distributions that only support Python 3 and pkgutil-style + namespace packages in the distributions that need to support Python + 2 and 3. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0420 + + (2) +https://github.com/pypa/sample-namespace-packages/tree/master/native + + +File: pythonpackagingguide.info, Node: pkgutil-style namespace packages, Next: pkg_resources-style namespace packages, Prev: Native namespace packages, Up: Creating a namespace package + +3.14.1.2 pkgutil-style namespace packages +......................................... + +Python 2.3 introduced the pkgutil(1) module and the extend_path(2) +function. This can be used to declare namespace packages that need to +be compatible with both Python 2.3+ and Python 3. This is the +recommended approach for the highest level of compatibility. + +To create a pkgutil-style namespace package, you need to provide an +‘__init__.py’ file for the namespace package: + + setup.py + mynamespace/ + __init__.py # Namespace package __init__.py + subpackage_a/ + __init__.py # Sub-package __init__.py + module.py + +The ‘__init__.py’ file for the namespace package needs to contain `only' +the following: + + __path__ = __import__('pkgutil').extend_path(__path__, __name__) + +`Every' distribution that uses the namespace package must include an +identical ‘__init__.py’. If any distribution does not, it will cause +the namespace logic to fail and the other sub-packages will not be +importable. Any additional code in ‘__init__.py’ will be inaccessible. + +A complete working example of two pkgutil-style namespace packages can +be found in the pkgutil namespace example project(3). + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/pkgutil.html + + (2) +https://docs.python.org/3/library/pkgutil.html#pkgutil.extend_path + + (3) +https://github.com/pypa/sample-namespace-packages/tree/master/pkgutil + + +File: pythonpackagingguide.info, Node: pkg_resources-style namespace packages, Prev: pkgutil-style namespace packages, Up: Creating a namespace package + +3.14.1.3 pkg_resources-style namespace packages +............................................... + +Setuptools(1) provides the pkg_resources.declare_namespace(2) function +and the ‘namespace_packages’ argument to ‘setup()’. Together these can +be used to declare namespace packages. While this approach is no longer +recommended, it is widely present in most existing namespace packages. +If you are creating a new distribution within an existing namespace +package that uses this method then it’s recommended to continue using +this as the different methods are not cross-compatible and it’s not +advisable to try to migrate an existing package. + +To create a pkg_resources-style namespace package, you need to provide +an ‘__init__.py’ file for the namespace package: + + setup.py + mynamespace/ + __init__.py # Namespace package __init__.py + subpackage_a/ + __init__.py # Sub-package __init__.py + module.py + +The ‘__init__.py’ file for the namespace package needs to contain `only' +the following: + + __import__('pkg_resources').declare_namespace(__name__) + +`Every' distribution that uses the namespace package must include an +identical ‘__init__.py’. If any distribution does not, it will cause +the namespace logic to fail and the other sub-packages will not be +importable. Any additional code in ‘__init__.py’ will be inaccessible. + + Note: Some older recommendations advise the following in the + namespace package ‘__init__.py’: + + try: + __import__('pkg_resources').declare_namespace(__name__) + except ImportError: + __path__ = __import__('pkgutil').extend_path(__path__, __name__) + + The idea behind this was that in the rare case that setuptools + isn’t available packages would fall-back to the pkgutil-style + packages. This isn’t advisable because pkgutil and + pkg_resources-style namespace packages are not cross-compatible. + If the presence of setuptools is a concern then the package should + just explicitly depend on setuptools via ‘install_requires’. + +Finally, every distribution must provide the ‘namespace_packages’ +argument to ‘setup()’ in ‘setup.py’. For example: + + from setuptools import find_packages, setup + + setup( + name='mynamespace-subpackage-a', + ... + packages=find_packages() + namespace_packages=['mynamespace'] + ) + +A complete working example of two pkg_resources-style namespace packages +can be found in the pkg_resources namespace example project(3). + + ---------- Footnotes ---------- + + (1) https://setuptools.readthedocs.io + + (2) +https://setuptools.readthedocs.io/en/latest/setuptools.html#namespace-packages + + (3) +https://github.com/pypa/sample-namespace-packages/tree/master/pkg_resources + + +File: pythonpackagingguide.info, Node: Creating and discovering plugins, Next: Analyzing PyPI package downloads, Prev: Packaging namespace packages, Up: Guides + +3.15 Creating and discovering plugins +===================================== + +Often when creating a Python application or library you’ll want the +ability to provide customizations or extra features via `plugins'. +Because Python packages can be separately distributed, your application +or library may want to automatically `discover' all of the plugins +available. + +There are three major approaches to doing automatic plugin discovery: + + 1. *note Using naming convention: 13b. + + 2. *note Using namespace packages: 13c. + + 3. *note Using package metadata: 13d. + +* Menu: + +* Using naming convention:: +* Using namespace packages:: +* Using package metadata:: + + +File: pythonpackagingguide.info, Node: Using naming convention, Next: Using namespace packages, Up: Creating and discovering plugins + +3.15.1 Using naming convention +------------------------------ + +If all of the plugins for your application follow the same naming +convention, you can use pkgutil.iter_modules()(1) to discover all of the +top-level modules that match the naming convention. For example, +Flask(2) uses the naming convention ‘flask_{plugin_name}’. If you +wanted to automatically discover all of the Flask plugins installed: + + import importlib + import pkgutil + + discovered_plugins = { + name: importlib.import_module(name) + for finder, name, ispkg + in pkgutil.iter_modules() + if name.startswith('flask_') + } + +If you had both the Flask-SQLAlchemy(3) and Flask-Talisman(4) plugins +installed then ‘discovered_plugins’ would be: + + { + 'flask_sqlachemy': , + 'flask_talisman': , + } + +Using naming convention for plugins also allows you to query the Python +Package Index’s simple API(5) for all packages that conform to your +naming convention. + + ---------- Footnotes ---------- + + (1) +https://docs.python.org/2/library/pkgutil.html#pkgutil.iter_modules + + (2) https://pypi.org/project/Flask/ + + (3) https://pypi.org/project/Flask-SQLAlchemy/ + + (4) https://pypi.org/project/flask-talisman + + (5) https://www.python.org/dev/peps/pep-0503/#specification + + +File: pythonpackagingguide.info, Node: Using namespace packages, Next: Using package metadata, Prev: Using naming convention, Up: Creating and discovering plugins + +3.15.2 Using namespace packages +------------------------------- + +*note Namespace packages: 130. can be used to provide a convention for +where to place plugins and also provides a way to perform discovery. +For example, if you make the sub-package ‘myapp.plugins’ a namespace +package then other *note distributions: b. can provide modules and +packages to that namespace. Once installed, you can use +pkgutil.iter_modules()(1) to discover all modules and packages installed +under that namespace: + + import importlib + import pkgutil + + import myapp.plugins + + def iter_namespace(ns_pkg): + # Specifying the second argument (prefix) to iter_modules makes the + # returned name an absolute name instead of a relative one. This allows + # import_module to work without having to do additional modification to + # the name. + return pkgutil.iter_modules(ns_pkg.__path__, ns_pkg.__name__ + ".") + + discovered_plugins = { + name: importlib.import_module(name) + for finder, name, ispkg + in iter_namespace(myapp.plugins) + } + +Specifying ‘myapp.plugins.__path__’ to iter_modules()(2) causes it to +only look for the modules directly under that namespace. For example, +if you have installed distributions that provide the modules +‘myapp.plugins.a’ and ‘myapp.plugins.b’ then ‘discovered_plugins’ in +this case would be: + + { + 'a': , + 'b': , + } + +This sample uses a sub-package as the namespace package +(‘myapp.plugins’), but it’s also possible to use a top-level package for +this purpose (such as ‘myapp_plugins’). How to pick the namespace to +use is a matter of preference, but it’s not recommended to make your +project’s main top-level package (‘myapp’ in this case) a namespace +package for the purpose of plugins, as one bad plugin could cause the +entire namespace to break which would in turn make your project +unimportable. For the “namespace sub-package” approach to work, the +plugin packages must omit the ‘__init__.py’ for your top-level package +directory (‘myapp’ in this case) and include the namespace-package style +‘__init__.py’ in the namespace sub-package directory (‘myapp/plugins’). +This also means that plugins will need to explicitly pass a list of +packages to ‘setup()’’s ‘packages’ argument instead of using +‘setuptools.find_packages()’. + + Warning: Namespace packages are a complex feature and there are + several different ways to create them. It’s highly recommended to + read the *note Packaging namespace packages: 130. documentation and + clearly document which approach is preferred for plugins to your + project. + + ---------- Footnotes ---------- + + (1) +https://docs.python.org/2/library/pkgutil.html#pkgutil.iter_modules + + (2) +https://docs.python.org/2/library/pkgutil.html#pkgutil.iter_modules + + +File: pythonpackagingguide.info, Node: Using package metadata, Prev: Using namespace packages, Up: Creating and discovering plugins + +3.15.3 Using package metadata +----------------------------- + +Setuptools(1) provides special support(2) for plugins. By providing the +‘entry_points’ argument to ‘setup()’ in ‘setup.py’ plugins can register +themselves for discovery. + +For example if you have a package named ‘myapp-plugin-a’ and it includes +in its ‘setup.py’: + + setup( + ... + entry_points={'myapp.plugins': 'a = myapp_plugin_a'}, + ... + ) + +Then you can discover and load all of the registered entry points by +using ‘pkg_resources.iter_entry_points()’: + + import pkg_resources + + discovered_plugins = { + entry_point.name: entry_point.load() + for entry_point + in pkg_resources.iter_entry_points('myapp.plugins') + } + +In this example, ‘discovered_plugins’ would be: + + { + 'a': , + } + + Note: The ‘entry_point’ specification in ‘setup.py’ is fairly + flexible and has a lot of options. It’s recommended to read over + the entire section on entry points(3). + + ---------- Footnotes ---------- + + (1) http://setuptools.readthedocs.io + + (2) +http://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins + + (3) +http://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins + + +File: pythonpackagingguide.info, Node: Analyzing PyPI package downloads, Next: Package index mirrors and caches, Prev: Creating and discovering plugins, Up: Guides + +3.16 Analyzing PyPI package downloads +===================================== + +This section covers how to use the public PyPI download statistics +dataset to learn more about downloads of a package (or packages) hosted +on PyPI. For example, you can use it to discover the distribution of +Python versions used to download a package. + +* Menu: + +* Background: Background<2>. +* Public dataset:: +* Caveats:: +* Additional tools:: +* References:: + + +File: pythonpackagingguide.info, Node: Background<2>, Next: Public dataset, Up: Analyzing PyPI package downloads + +3.16.1 Background +----------------- + +PyPI does not display download statistics for a number of reasons: (1) + + - `Inefficient to make work with a Content Distribution Network + (CDN):' Download statistics change constantly. Including them in + project pages, which are heavily cached, would require invalidating + the cache more often, and reduce the overall effectiveness of the + cache. + + - `Highly inaccurate:' A number of things prevent the download counts + from being accurate, some of which include: + + - ‘pip’’s download cache (lowers download counts) + + - Internal or unofficial mirrors (can both raise or lower + download counts) + + - Packages not hosted on PyPI (for comparisons sake) + + - Unofficial scripts or attempts at download count inflation + (raises download counts) + + - Known historical data quality issues (lowers download counts) + + - `Not particularly useful:' Just because a project has been + downloaded a lot doesn’t mean it’s good; Similarly just because a + project hasn’t been downloaded a lot doesn’t mean it’s bad! + +In short, because it’s value is low for various reasons, and the +tradeoffs required to make it work are high, it has been not an +effective use of limited resources. + + ---------- Footnotes ---------- + + (1) (1) PyPI Download Counts deprecation email +(https://mail.python.org/pipermail/distutils-sig/2013-May/020855.html) + + +File: pythonpackagingguide.info, Node: Public dataset, Next: Caveats, Prev: Background<2>, Up: Analyzing PyPI package downloads + +3.16.2 Public dataset +--------------------- + +As an alternative, the Linehaul project(1) streams download logs from +PyPI to Google BigQuery(2) (3), where they are stored as a public +dataset. + +* Menu: + +* Getting set up:: +* Data schema:: +* Useful queries:: + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/linehaul + + (2) https://cloud.google.com/bigquery + + (3) (2) PyPI BigQuery dataset announcement email +(https://mail.python.org/pipermail/distutils-sig/2016-May/028986.html) + + +File: pythonpackagingguide.info, Node: Getting set up, Next: Data schema, Up: Public dataset + +3.16.2.1 Getting set up +....................... + +In order to use Google BigQuery(1) to query the public PyPI download +statistics dataset(2), you’ll need a Google account and to enable the +BigQuery API on a Google Cloud Platform project. You can run the up to +1TB of queries per month using the BigQuery free tier without a credit +card(3) + + - Navigate to the BigQuery web UI(4). + + - Create a new project. + + - Enable the BigQuery API(5). + +For more detailed instructions on how to get started with BigQuery, +check out the BigQuery quickstart guide(6). + + ---------- Footnotes ---------- + + (1) https://cloud.google.com/bigquery + + (2) +https://console.cloud.google.com/bigquery?p=the-psf&d=pypi&page=dataset + + (3) +https://cloud.google.com/blog/big-data/2017/01/how-to-run-a-terabyte-of-google-bigquery-queries-each-month-without-a-credit-card + + (4) https://console.cloud.google.com/bigquery + + (5) +https://console.developers.google.com/apis/library/bigquery-json.googleapis.com + + (6) +https://cloud.google.com/bigquery/docs/quickstarts/quickstart-web-ui + + +File: pythonpackagingguide.info, Node: Data schema, Next: Useful queries, Prev: Getting set up, Up: Public dataset + +3.16.2.2 Data schema +.................... + +Linehaul writes an entry in a ‘the-psf.pypi.file_downloads’ table for +each download. The table contains information about what file was +downloaded and how it was downloaded. Some useful columns from the +table schema(1) include: + +Column Description Examples + +------------------------------------------------------------------------------------- + +timestamp Date and time ‘2020-03-09 00:33:03 UTC’ + + +file.project Project name ‘pipenv’, ‘nose’ + + +file.version Package version ‘0.1.6’, ‘1.4.2’ + + +details.installer.name Installer pip, bandersnatch(2) + + +details.python Python version ‘2.7.12’, ‘3.6.4’ + + + ---------- Footnotes ---------- + + (1) +https://console.cloud.google.com/bigquery?pli=1&p=the-psf&d=pypi&t=file_downloads&page=table + + (2) /key_projects/#bandersnatch + + +File: pythonpackagingguide.info, Node: Useful queries, Prev: Data schema, Up: Public dataset + +3.16.2.3 Useful queries +....................... + +Run queries in the BigQuery web UI(1) by clicking the “Compose query” +button. + +Note that the rows are stored in a partitioned, which helps limit the +cost of queries. These example queries analyze downloads from recent +history by filtering on the ‘timestamp’ column. + +* Menu: + +* Counting package downloads:: +* Package downloads over time:: +* Python versions over time:: + + ---------- Footnotes ---------- + + (1) https://console.cloud.google.com/bigquery + + +File: pythonpackagingguide.info, Node: Counting package downloads, Next: Package downloads over time, Up: Useful queries + +3.16.2.4 Counting package downloads +................................... + +The following query counts the total number of downloads for the project +“pytest”. + + #standardSQL + SELECT COUNT(*) AS num_downloads + FROM `the-psf.pypi.file_downloads` + WHERE file.project = 'pytest' + -- Only query the last 30 days of history + AND DATE(timestamp) + BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) + AND CURRENT_DATE() + +num_downloads + +-------------------- + +20531925 + +To only count downloads from pip, filter on the ‘details.installer.name’ +column. + + #standardSQL + SELECT COUNT(*) AS num_downloads + FROM `the-psf.pypi.file_downloads` + WHERE file.project = 'pytest' + AND details.installer.name = 'pip' + -- Only query the last 30 days of history + AND DATE(timestamp) + BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) + AND CURRENT_DATE() + +num_downloads + +-------------------- + +19391645 + + +File: pythonpackagingguide.info, Node: Package downloads over time, Next: Python versions over time, Prev: Counting package downloads, Up: Useful queries + +3.16.2.5 Package downloads over time +.................................... + +To group by monthly downloads, use the ‘TIMESTAMP_TRUNC’ function. Also +filtering by this column reduces corresponding costs. + + #standardSQL + SELECT + COUNT(*) AS num_downloads, + DATE_TRUNC(DATE(timestamp), MONTH) AS `month` + FROM `the-psf.pypi.file_downloads` + WHERE + file.project = 'pytest' + -- Only query the last 6 months of history + AND DATE(timestamp) + BETWEEN DATE_TRUNC(DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH), MONTH) + AND CURRENT_DATE() + GROUP BY `month` + ORDER BY `month` DESC + +num_downloads month + +------------------------------------- + +1956741 2018-01-01 + + +2344692 2017-12-01 + + +1730398 2017-11-01 + + +2047310 2017-10-01 + + +1744443 2017-09-01 + + +1916952 2017-08-01 + + + +File: pythonpackagingguide.info, Node: Python versions over time, Prev: Package downloads over time, Up: Useful queries + +3.16.2.6 Python versions over time +.................................. + +Extract the Python version from the ‘details.python’ column. Warning: +This query processes over 500 GB of data. + + #standardSQL + SELECT + REGEXP_EXTRACT(details.python, r"[0-9]+\.[0-9]+") AS python_version, + COUNT(*) AS num_downloads, + FROM `the-psf.pypi.file_downloads` + WHERE + -- Only query the last 6 months of history + DATE(timestamp) + BETWEEN DATE_TRUNC(DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH), MONTH) + AND CURRENT_DATE() + GROUP BY `python_version` + ORDER BY `num_downloads` DESC + +python num_downloads + +--------------------------------- + +3.7 12990683561 + + +3.6 9035598511 + + +2.7 8467785320 + + +3.8 4581627740 + + +3.5 2412533601 + + +null 1641456718 + + + +File: pythonpackagingguide.info, Node: Caveats, Next: Additional tools, Prev: Public dataset, Up: Analyzing PyPI package downloads + +3.16.3 Caveats +-------------- + +In addition to the caveats listed in the background above, Linehaul +suffered from a bug which caused it to significantly under-report +download statistics prior to July 26, 2018. Downloads before this date +are proportionally accurate (e.g. the percentage of Python 2 vs. +Python 3 downloads) but total numbers are lower than actual by an order +of magnitude. + + +File: pythonpackagingguide.info, Node: Additional tools, Next: References, Prev: Caveats, Up: Analyzing PyPI package downloads + +3.16.4 Additional tools +----------------------- + +Besides using the BigQuery console, there are some additional tools +which may be useful when analyzing download statistics. + +* Menu: + +* google-cloud-bigquery:: +* pypinfo:: +* pandas-gbq:: + + +File: pythonpackagingguide.info, Node: google-cloud-bigquery, Next: pypinfo, Up: Additional tools + +3.16.4.1 ‘google-cloud-bigquery’ +................................ + +You can also access the public PyPI download statistics dataset +programmatically via the BigQuery API and the google-cloud-bigquery(1) +project, the official Python client library for BigQuery. + + from google.cloud import bigquery + + # Note: depending on where this code is being run, you may require + # additional authentication. See: + # https://cloud.google.com/bigquery/docs/authentication/ + client = bigquery.Client() + + query_job = client.query(""" + SELECT COUNT(*) AS num_downloads + FROM `the-psf.pypi.file_downloads` + WHERE file.project = 'pytest' + -- Only query the last 30 days of history + AND DATE(timestamp) + BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) + AND CURRENT_DATE()""") + + results = query_job.result() # Waits for job to complete. + for row in results: + print("{} downloads".format(row.num_downloads)) + + ---------- Footnotes ---------- + + (1) https://cloud.google.com/bigquery/docs/reference/libraries + + +File: pythonpackagingguide.info, Node: pypinfo, Next: pandas-gbq, Prev: google-cloud-bigquery, Up: Additional tools + +3.16.4.2 ‘pypinfo’ +.................. + +pypinfo(1) is a command-line tool which provides access to the dataset +and can generate several useful queries. For example, you can query the +total number of download for a package with the command ‘pypinfo +package_name’. + +Install pypinfo(2) using pip. + + pip install pypinfo + +Usage: + + $ pypinfo requests + Served from cache: False + Data processed: 6.87 GiB + Data billed: 6.87 GiB + Estimated cost: $0.04 + + | download_count | + | -------------- | + | 9,316,415 | + + ---------- Footnotes ---------- + + (1) https://github.com/ofek/pypinfo/blob/master/README.rst + + (2) https://github.com/ofek/pypinfo/blob/master/README.rst + + +File: pythonpackagingguide.info, Node: pandas-gbq, Prev: pypinfo, Up: Additional tools + +3.16.4.3 ‘pandas-gbq’ +..................... + +The pandas-gbq(1) project allows for accessing query results via +Pandas(2). + + ---------- Footnotes ---------- + + (1) https://pandas-gbq.readthedocs.io/en/latest/ + + (2) https://pandas.pydata.org/ + + +File: pythonpackagingguide.info, Node: References, Prev: Additional tools, Up: Analyzing PyPI package downloads + +3.16.5 References +----------------- + + +File: pythonpackagingguide.info, Node: Package index mirrors and caches, Next: Hosting your own simple repository, Prev: Analyzing PyPI package downloads, Up: Guides + +3.17 Package index mirrors and caches +===================================== + + +Page Status: Incomplete + + +Last Reviewed: 2014-12-24 + +Mirroring or caching of PyPI can be used to speed up local package +installation, allow offline work, handle corporate firewalls or just +plain Internet flakiness. + +Three options are available in this area: + + 1. pip provides local caching options, + + 2. devpi provides higher-level caching option, potentially shared + amongst many users or machines, and + + 3. bandersnatch provides a local complete mirror of all PyPI *note + packages: b. + +* Menu: + +* Caching with pip:: +* Caching with devpi:: +* Complete mirror with bandersnatch:: + + +File: pythonpackagingguide.info, Node: Caching with pip, Next: Caching with devpi, Up: Package index mirrors and caches + +3.17.1 Caching with pip +----------------------- + +pip provides a number of facilities for speeding up installation by +using local cached copies of *note packages: b.: + + 1. Fast & local installs(1) by downloading all the requirements for a + project and then pointing pip at those downloaded files instead of + going to PyPI. + + 2. A variation on the above which pre-builds the installation files + for the requirements using pip wheel(2): + + $ pip wheel --wheel-dir=/tmp/wheelhouse SomeProject + $ pip install --no-index --find-links=/tmp/wheelhouse SomeProject + + ---------- Footnotes ---------- + + (1) +https://pip.pypa.io/en/latest/user_guide/#installing-from-local-packages + + (2) https://pip.readthedocs.io/en/latest/reference/pip_wheel.html + + +File: pythonpackagingguide.info, Node: Caching with devpi, Next: Complete mirror with bandersnatch, Prev: Caching with pip, Up: Package index mirrors and caches + +3.17.2 Caching with devpi +------------------------- + +devpi is a caching proxy server which you run on your laptop, or some +other machine you know will always be available to you. See the devpi +documentation for getting started(1). + + ---------- Footnotes ---------- + + (1) http://doc.devpi.net/latest/quickstart-pypimirror.html + + +File: pythonpackagingguide.info, Node: Complete mirror with bandersnatch, Prev: Caching with devpi, Up: Package index mirrors and caches + +3.17.3 Complete mirror with bandersnatch +---------------------------------------- + +bandersnatch will set up a complete local mirror of all PyPI *note +packages: b. (externally-hosted packages are not mirrored). See the +bandersnatch documentation for getting that going(1). + +A benefit of devpi is that it will create a mirror which includes *note +packages: b. that are external to PyPI, unlike bandersnatch which will +only cache *note packages: b. hosted on PyPI. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/bandersnatch/ + + +File: pythonpackagingguide.info, Node: Hosting your own simple repository, Next: Migrating to PyPI org, Prev: Package index mirrors and caches, Up: Guides + +3.18 Hosting your own simple repository +======================================= + +If you wish to host your own simple repository (1), you can either use a +software package like devpi(2) or you can use simply create the proper +directory structure and use any web server that can serve static files +and generate an autoindex. + +In either case, since you’ll be hosting a repository that is likely not +in your user’s default repositories, you should instruct them in your +project’s description to configure their installer appropriately. For +example with pip: + + pip install --extra-index-url https://python.example.com/ foobar + +In addition, it is `highly' recommended that you serve your repository +with valid HTTPS. At this time, the security of your user’s +installations depends on all repositories using a valid HTTPS setup. + +* Menu: + +* “Manual” repository:: + + ---------- Footnotes ---------- + + (1) (1) For complete documentation of the simple repository protocol, +see PEP 503. + + (2) http://doc.devpi.net/latest/ + + +File: pythonpackagingguide.info, Node: “Manual” repository, Up: Hosting your own simple repository + +3.18.1 “Manual” repository +-------------------------- + +The directory layout is fairly simple, within a root directory you need +to create a directory for each project. This directory should be the +normalized name (as defined by PEP 503) of the project. Within each of +these directories simply place each of the downloadable files. If you +have the projects “Foo” (with the versions 1.0 and 2.0) and “bar” (with +the version 0.1) You should end up with a structure that looks like: + + . + ├── bar + │   └── bar-0.1.tar.gz + └── foo + ├── Foo-1.0.tar.gz + └── Foo-2.0.tar.gz + +Once you have this layout, simply configure your webserver to serve the +root directory with autoindex enabled. For an example using the built +in Web server in Twisted(1), you would simply run ‘twistd -n web --path +.’ and then instruct users to add the URL to their installer’s +configuration. + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) https://twistedmatrix.com/ + + +File: pythonpackagingguide.info, Node: Migrating to PyPI org, Next: Using TestPyPI<2>, Prev: Hosting your own simple repository, Up: Guides + +3.19 Migrating to PyPI.org +========================== + +*note pypi.org: 161. is the new, rewritten version of PyPI that has +replaced the legacy PyPI code base. It is the default version of PyPI +that people are expected to use. These are the tools and processes that +people will need to interact with ‘PyPI.org’. + +* Menu: + +* Publishing releases:: +* Registering package names & metadata:: +* Using TestPyPI:: +* Registering new user accounts:: +* Browsing packages:: +* Downloading packages:: +* Managing published packages and releases:: + + +File: pythonpackagingguide.info, Node: Publishing releases, Next: Registering package names & metadata, Up: Migrating to PyPI org + +3.19.1 Publishing releases +-------------------------- + +‘pypi.org’ is the default upload platform as of September 2016. + +Uploads through ‘pypi.python.org’ were `switched off' on `July 3, 2017'. +As of April 13th, 2018, ‘pypi.org’ is the URL for PyPI. + +The recommended way to migrate to PyPI.org for uploading is to ensure +that you are using a new enough version of your upload tool. + +The default upload settings switched to ‘pypi.org’ in the following +versions: + + * ‘twine’ 1.8.0 + + * ‘setuptools’ 27.0.0 + + * Python 2.7.13 (‘distutils’ update) + + * Python 3.4.6 (‘distutils’ update) + + * Python 3.5.3 (‘distutils’ update) + + * Python 3.6.0 (‘distutils’ update) + +In addition to ensuring you’re on a new enough version of the tool for +the tool’s default to have switched, you must also make sure that you +have not configured the tool to override its default upload URL. +Typically this is configured in a file located at ‘$HOME/.pypirc’. If +you see a file like: + + [distutils] + index-servers = + pypi + + [pypi] + repository = https://pypi.python.org/pypi + username = + password = + +Then simply delete the line starting with ‘repository’ and you will use +your upload tool’s default URL. + +If for some reason you’re unable to upgrade the version of your tool to +a version that defaults to using PyPI.org, then you may edit +‘$HOME/.pypirc’ and include the ‘repository:’ line, but use the value +‘https://upload.pypi.org/legacy/’ instead: + + [distutils] + index-servers = + pypi + + [pypi] + repository = https://upload.pypi.org/legacy/ + username = + password = + +(‘legacy’ in this URL refers to the fact that this is the new server +implementation’s emulation of the legacy server implementation’s upload +API.) + +For more details, see the *note specification: f8. for ‘.pypirc’. + + +File: pythonpackagingguide.info, Node: Registering package names & metadata, Next: Using TestPyPI, Prev: Publishing releases, Up: Migrating to PyPI org + +3.19.2 Registering package names & metadata +------------------------------------------- + +Explicit pre-registration of package names with the ‘setup.py register’ +command prior to the first upload is no longer required, and is not +currently supported by the legacy upload API emulation on PyPI.org. + +As a result, attempting explicit registration after switching to using +PyPI.org for uploads will give the following error message: + + Server response (410): This API is no longer supported, instead simply upload the file. + +The solution is to skip the registration step, and proceed directly to +uploading artifacts. + + +File: pythonpackagingguide.info, Node: Using TestPyPI, Next: Registering new user accounts, Prev: Registering package names & metadata, Up: Migrating to PyPI org + +3.19.3 Using TestPyPI +--------------------- + +Legacy TestPyPI (testpypi.python.org) is no longer available; use +test.pypi.org(1) instead. If you use TestPyPI, you must update your +‘$HOME/.pypirc’ to handle TestPyPI’s new location, by replacing +‘https://testpypi.python.org/pypi’ with ‘https://test.pypi.org/legacy/’, +for example: + + [distutils] + index-servers= + pypi + testpypi + + [testpypi] + repository = https://test.pypi.org/legacy/ + username = + password = + +For more details, see the *note specification: f8. for ‘.pypirc’. + + ---------- Footnotes ---------- + + (1) https://test.pypi.org + + +File: pythonpackagingguide.info, Node: Registering new user accounts, Next: Browsing packages, Prev: Using TestPyPI, Up: Migrating to PyPI org + +3.19.4 Registering new user accounts +------------------------------------ + +In order to help mitigate spam attacks against PyPI, new user +registration through ‘pypi.python.org’ was `switched off' on `February +20, 2018'. New user registrations at ‘pypi.org’ are open. + + +File: pythonpackagingguide.info, Node: Browsing packages, Next: Downloading packages, Prev: Registering new user accounts, Up: Migrating to PyPI org + +3.19.5 Browsing packages +------------------------ + +While ‘pypi.python.org’ is may still be used in links from other PyPA +documentation, etc, the default interface for browsing packages is +‘pypi.org’. The domain pypi.python.org now redirects to pypi.org, and +may be disabled sometime in the future. + + +File: pythonpackagingguide.info, Node: Downloading packages, Next: Managing published packages and releases, Prev: Browsing packages, Up: Migrating to PyPI org + +3.19.6 Downloading packages +--------------------------- + +‘pypi.org’ is the default host for downloading packages. + + +File: pythonpackagingguide.info, Node: Managing published packages and releases, Prev: Downloading packages, Up: Migrating to PyPI org + +3.19.7 Managing published packages and releases +----------------------------------------------- + +‘pypi.org’ provides a fully functional interface for logged in users to +manage their published packages and releases. + + +File: pythonpackagingguide.info, Node: Using TestPyPI<2>, Next: Making a PyPI-friendly README, Prev: Migrating to PyPI org, Up: Guides + +3.20 Using TestPyPI +=================== + +‘TestPyPI’ is a separate instance of the *note Python Package Index +(PyPI): 39. that allows you to try out the distribution tools and +process without worrying about affecting the real index. TestPyPI is +hosted at test.pypi.org(1) + +* Menu: + +* Registering your account:: +* Using TestPyPI with Twine:: +* Using TestPyPI with pip:: +* Setting up TestPyPI in .pypirc: Setting up TestPyPI in pypirc. + + ---------- Footnotes ---------- + + (1) https://test.pypi.org + + +File: pythonpackagingguide.info, Node: Registering your account, Next: Using TestPyPI with Twine, Up: Using TestPyPI<2> + +3.20.1 Registering your account +------------------------------- + +Because TestPyPI has a separate database from the live PyPI, you’ll need +a separate user account for specifically for TestPyPI. Go to +‘https://test.pypi.org/account/register/’ to register your account. + + Note: The database for TestPyPI may be periodically pruned, so it + is not unusual for user accounts to be deleted. + + +File: pythonpackagingguide.info, Node: Using TestPyPI with Twine, Next: Using TestPyPI with pip, Prev: Registering your account, Up: Using TestPyPI<2> + +3.20.2 Using TestPyPI with Twine +-------------------------------- + +You can upload your distributions to TestPyPI using *note twine: 66. by +specifying the ‘--repository’ flag + + $ twine upload --repository testpypi dist/* + +You can see if your package has successfully uploaded by navigating to +the URL ‘https://test.pypi.org/project/’ where +‘sampleproject’ is the name of your project that you uploaded. It may +take a minute or two for your project to appear on the site. + + +File: pythonpackagingguide.info, Node: Using TestPyPI with pip, Next: Setting up TestPyPI in pypirc, Prev: Using TestPyPI with Twine, Up: Using TestPyPI<2> + +3.20.3 Using TestPyPI with pip +------------------------------ + +You can tell pip to download packages from TestPyPI instead of PyPI by +specifying the ‘--index-url’ flag + + $ pip install --index-url https://test.pypi.org/simple/ your-package + +If you want to allow pip to also pull other packages from PyPI you can +specify ‘--extra-index-url’ to point to PyPI. This is useful when the +package you’re testing has dependencies: + + pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple your-package + + +File: pythonpackagingguide.info, Node: Setting up TestPyPI in pypirc, Prev: Using TestPyPI with pip, Up: Using TestPyPI<2> + +3.20.4 Setting up TestPyPI in ‘.pypirc’ +--------------------------------------- + +If you want to avoid entering your username, you can configure TestPyPI +in your ‘$HOME/.pypirc’: + + [testpypi] + username = + +For more details, see the *note specification: f8. for ‘.pypirc’. + + +File: pythonpackagingguide.info, Node: Making a PyPI-friendly README, Next: Publishing package distribution releases using GitHub Actions CI/CD workflows, Prev: Using TestPyPI<2>, Up: Guides + +3.21 Making a PyPI-friendly README +================================== + +README files can help your users understand your project and can be used +to set your project’s description on PyPI. This guide helps you create a +README in a PyPI-friendly format and include your README in your package +so it appears on PyPI. + +* Menu: + +* Creating a README file:: +* Including your README in your package’s metadata:: +* Validating reStructuredText markup:: + + +File: pythonpackagingguide.info, Node: Creating a README file, Next: Including your README in your package’s metadata, Up: Making a PyPI-friendly README + +3.21.1 Creating a README file +----------------------------- + +README files for Python projects are often named ‘README’, ‘README.txt’, +‘README.rst’, or ‘README.md’. + +For your README to display properly on PyPI, choose a markup language +supported by PyPI. Formats supported by PyPI’s README renderer(1) are: + + * plain text + + * reStructuredText(2) (without Sphinx extensions) + + * Markdown (GitHub Flavored Markdown(3) by default, or CommonMark(4)) + +It’s customary to save your README file in the root of your project, in +the same directory as your ‘setup.py’ file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/readme_renderer + + (2) http://docutils.sourceforge.net/rst.html + + (3) https://github.github.com/gfm/ + + (4) http://commonmark.org/ + + +File: pythonpackagingguide.info, Node: Including your README in your package’s metadata, Next: Validating reStructuredText markup, Prev: Creating a README file, Up: Making a PyPI-friendly README + +3.21.2 Including your README in your package’s metadata +------------------------------------------------------- + +To include your README’s contents as your package description, set your +project’s ‘Description’ and ‘Description-Content-Type’ metadata, +typically in your project’s ‘setup.py’ file. + +See also +........ + + * *note Description: 172. + + * *note Description-Content-Type: 173. + +For example, to set these values in a package’s ‘setup.py’ file, use +‘setup()’’s ‘long_description’ and ‘long_description_content_type’. + +Set the value of ‘long_description’ to the contents (not the path) of +the README file itself. Set the ‘long_description_content_type’ to an +accepted ‘Content-Type’-style value for your README file’s markup, such +as ‘text/plain’, ‘text/x-rst’ (for reStructuredText), or +‘text/markdown’. + + Note: If you’re using GitHub-flavored Markdown to write a project’s + description, ensure you upgrade the following tools: + + python3 -m pip install --user --upgrade setuptools wheel twine + + The minimum required versions of the respective tools are: + + - ‘setuptools >= 38.6.0’ + + - ‘wheel >= 0.31.0’ + + - ‘twine >= 1.11.0’ + + It’s recommended that you use ‘twine’ to upload the project’s + distribution packages: + + twine upload dist/* + +For example, see this ‘setup.py’ file, which reads the contents of +‘README.md’ as ‘long_description’ and identifies the markup as +GitHub-flavored Markdown: + + from setuptools import setup + + # read the contents of your README file + from os import path + this_directory = path.abspath(path.dirname(__file__)) + with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f: + long_description = f.read() + + setup( + name='an_example_package', + # other arguments omitted + long_description=long_description, + long_description_content_type='text/markdown' + ) + + +File: pythonpackagingguide.info, Node: Validating reStructuredText markup, Prev: Including your README in your package’s metadata, Up: Making a PyPI-friendly README + +3.21.3 Validating reStructuredText markup +----------------------------------------- + +If your README is written in reStructuredText, any invalid markup will +prevent it from rendering, causing PyPI to instead just show the +README’s raw source. + +Note that Sphinx extensions used in docstrings, such as directives and +roles(1) (e.g., “‘:py:func:`getattr`’” or +“‘:ref:`my-reference-label`’”), are not allowed here and will result in +error messages like “‘Error: Unknown interpreted text role "py:func".’”. + +You can check your README for markup errors before uploading as follows: + + 1. Install the latest version of twine(2); version 1.12.0 or higher is + required: + + pip install --upgrade twine + + 2. Build the sdist and wheel for your project as described under *note + Packaging your project: d8. + + 3. Run ‘twine check’ on the sdist and wheel: + + twine check dist/* + + This command will report any problems rendering your README. If + your markup renders fine, the command will output ‘Checking + distribution FILENAME: Passed’. + + ---------- Footnotes ---------- + + (1) +http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html + + (2) https://github.com/pypa/twine + + +File: pythonpackagingguide.info, Node: Publishing package distribution releases using GitHub Actions CI/CD workflows, Prev: Making a PyPI-friendly README, Up: Guides + +3.22 Publishing package distribution releases using GitHub Actions CI/CD workflows +================================================================================== + +GitHub Actions CI/CD(1) allows you to run a series of commands whenever +an event occurs on the GitHub platform. One popular choice is having a +workflow that’s triggered by a ‘push’ event. This guide shows you how +to publish a Python distribution whenever a tagged commit is pushed. It +will use the pypa/gh-action-pypi-publish GitHub Action(2). + + Attention: This guide `assumes' that you already have a project + that you know how to build distributions for and `it lives on + GitHub'. + +* Menu: + +* Saving credentials on GitHub:: +* Creating a workflow definition:: +* Defining a workflow job environment:: +* Checking out the project and building distributions:: +* Publishing the distribution to PyPI and TestPyPI:: +* That’s all, folks!: That’s all folks!. + + ---------- Footnotes ---------- + + (1) https://github.com/features/actions + + (2) https://github.com/marketplace/actions/pypi-publish + + +File: pythonpackagingguide.info, Node: Saving credentials on GitHub, Next: Creating a workflow definition, Up: Publishing package distribution releases using GitHub Actions CI/CD workflows + +3.22.1 Saving credentials on GitHub +----------------------------------- + +In this guide, we’ll demonstrate uploading to both PyPI and TestPyPI, +meaning that we’ll have two separate sets of credentials. And we’ll +need to save them in the GitHub repository settings. + +Let’s begin! 🚀 + + 1. Go to ‘https://pypi.org/manage/account/#api-tokens’ and create a + new API token(1). If you have the project on PyPI already, limit + the token scope to just that project. You can call it something + like ‘GitHub Actions CI/CD — project-org/project-repo’ in order for + it to be easily distinguishable in the token list. `Don’t close + the page just yet — you won’t see that token again.' + + 2. In a separate browser tab or window, go to the ‘Settings’ tab of + your target repository and then click on Secrets(2) in the left + sidebar. + + 3. Create a new secret called ‘pypi_password’ and copy-paste the token + from the fist step. + + 4. Now, go to ‘https://test.pypi.org/manage/account/#api-tokens’ and + repeat the steps. Save that TestPyPI token on GitHub as + ‘test_pypi_password’. + + Attention: If you don’t have a TestPyPI account, you’ll need + to create it. It’s not the same as a regular PyPI account. + + ---------- Footnotes ---------- + + (1) https://pypi.org/help/#apitoken + + (2) +https://help.github.com/en/articles/virtual-environments-for-github-actions#creating-and-using-secrets-encrypted-variables + + +File: pythonpackagingguide.info, Node: Creating a workflow definition, Next: Defining a workflow job environment, Prev: Saving credentials on GitHub, Up: Publishing package distribution releases using GitHub Actions CI/CD workflows + +3.22.2 Creating a workflow definition +------------------------------------- + +GitHub CI/CD workflows are declared in YAML files stored in the +‘.github/workflows/’ directory of your repository. + +Let’s create a ‘.github/workflows/publish-to-test-pypi.yml’ file. + +Start it with a meaningful name and define the event that should make +GitHub run this workflow: + + name: Publish Python 🐍 distributions 📦 to PyPI and TestPyPI + + on: push + + + +File: pythonpackagingguide.info, Node: Defining a workflow job environment, Next: Checking out the project and building distributions, Prev: Creating a workflow definition, Up: Publishing package distribution releases using GitHub Actions CI/CD workflows + +3.22.3 Defining a workflow job environment +------------------------------------------ + +Now, let’s add initial setup for our job. It’s a process that will +execute commands that we’ll define later. In this guide, we’ll use +Ubuntu 18.04: + + + jobs: + build-n-publish: + name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI + runs-on: ubuntu-18.04 + + + +File: pythonpackagingguide.info, Node: Checking out the project and building distributions, Next: Publishing the distribution to PyPI and TestPyPI, Prev: Defining a workflow job environment, Up: Publishing package distribution releases using GitHub Actions CI/CD workflows + +3.22.4 Checking out the project and building distributions +---------------------------------------------------------- + +Then, add the following under the ‘build-n-publish’ section: + + + steps: + - uses: actions/checkout@master + - name: Set up Python 3.7 + uses: actions/setup-python@v1 + with: + python-version: 3.7 + +This will download your repository into the CI runner and then install +and activate Python 3.7. + +And now we can build dists from source. In this example, we’ll use +‘build’ package, assuming that your project has a ‘pyproject.toml’ +properly set up (see PEP 517(1)/ PEP 518(2)). + + Tip: You can use any other method for building distributions as + long as it produces ready-to-upload artifacts saved into the + ‘dist/’ folder. + +So add this to the steps list: + + - name: Install pypa/build + run: >- + python -m + pip install + build + --user + - name: Build a binary wheel and a source tarball + run: >- + python -m + build + --sdist + --wheel + --outdir dist/ + . + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0517 + + (2) https://www.python.org/dev/peps/pep-0518 + + +File: pythonpackagingguide.info, Node: Publishing the distribution to PyPI and TestPyPI, Next: That’s all folks!, Prev: Checking out the project and building distributions, Up: Publishing package distribution releases using GitHub Actions CI/CD workflows + +3.22.5 Publishing the distribution to PyPI and TestPyPI +------------------------------------------------------- + +Finally, add the following steps at the end: + + - name: Publish distribution 📦 to Test PyPI + uses: pypa/gh-action-pypi-publish@master + with: + password: ${{ secrets.test_pypi_password }} + repository_url: https://test.pypi.org/legacy/ + - name: Publish distribution 📦 to PyPI + if: startsWith(github.ref, 'refs/tags') + uses: pypa/gh-action-pypi-publish@master + with: + password: ${{ secrets.pypi_password }} + +These two steps use the pypa/gh-action-pypi-publish(1) GitHub Action: +the first one uploads contents of the ‘dist/’ folder into TestPyPI +unconditionally and the second does that to PyPI, but only if the +current commit is tagged. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/gh-action-pypi-publish + + +File: pythonpackagingguide.info, Node: That’s all folks!, Prev: Publishing the distribution to PyPI and TestPyPI, Up: Publishing package distribution releases using GitHub Actions CI/CD workflows + +3.22.6 That’s all, folks! +------------------------- + +Now, whenever you push a tagged commit to your Git repository remote on +GitHub, this workflow will publish it to PyPI. And it’ll publish any +push to TestPyPI which is useful for providing test builds to your alpha +users as well as making sure that your release pipeline remains healthy! + + +File: pythonpackagingguide.info, Node: Discussions, Next: PyPA specifications, Prev: Guides, Up: Top + +4 Discussions +************* + +`Discussions' are focused on providing comprehensive information about a +specific topic. If you’re just trying to get stuff done, see *note +Guides: 22. + +* Menu: + +* Deploying Python applications:: +* pip vs easy_install:: +* install_requires vs requirements files:: +* Wheel vs Egg:: + + +File: pythonpackagingguide.info, Node: Deploying Python applications, Next: pip vs easy_install, Up: Discussions + +4.1 Deploying Python applications +================================= + + +Page Status: Incomplete + + +Last Reviewed: 2014-11-11 + +* Menu: + +* Overview:: +* OS packaging & installers:: +* Application bundles:: +* Configuration management:: + + +File: pythonpackagingguide.info, Node: Overview, Next: OS packaging & installers, Up: Deploying Python applications + +4.1.1 Overview +-------------- + +* Menu: + +* Supporting multiple hardware platforms:: + + +File: pythonpackagingguide.info, Node: Supporting multiple hardware platforms, Up: Overview + +4.1.1.1 Supporting multiple hardware platforms +.............................................. + + FIXME + + Meaning: x86, x64, ARM, others? + + For Python-only distributions, it *should* be straightforward to deploy on all + platforms where Python can run. + + For distributions with binary extensions, deployment is major headache. Not only + must the extensions be built on all the combinations of operating system and + hardware platform, but they must also be tested, preferably on continuous + integration platforms. The issues are similar to the "multiple Python + versions" section above, not sure whether this should be a separate section. + Even on Windows x64, both the 32 bit and 64 bit versions of Python enjoy + significant usage. + + +File: pythonpackagingguide.info, Node: OS packaging & installers, Next: Application bundles, Prev: Overview, Up: Deploying Python applications + +4.1.2 OS packaging & installers +------------------------------- + + FIXME + + - Building rpm/debs for projects + - Building rpms/debs for whole virtualenvs + - Building macOS installers for Python projects + - Building Android APKs with Kivy+P4A or P4A & Buildozer + +* Menu: + +* Windows: Windows<2>. + + +File: pythonpackagingguide.info, Node: Windows<2>, Up: OS packaging & installers + +4.1.2.1 Windows +............... + + FIXME + + - Building Windows installers for Python projects + +* Menu: + +* Pynsist:: + + +File: pythonpackagingguide.info, Node: Pynsist, Up: Windows<2> + +4.1.2.2 Pynsist +............... + +Pynsist(1) is a tool that bundles Python programs together with the +Python-interpreter into a single installer based on NSIS. In most cases, +packaging only requires the user to choose a version of the +Python-interpreter and declare the dependencies of the program. The +tool downloads the specified Python-interpreter for Windows and packages +it with all the dependencies in a single Windows-executable installer. + +The installed program can be started from a shortcut that the installer +adds to the start-menu. It uses a Python interpreter installed within +its application directory, independent of any other Python installation +on the computer. + +A big advantage of Pynsist is that the Windows packages can be built on +Linux. There are several examples for different kinds of programs +(console, GUI) in the documentation(2). The tool is released under the +MIT-licence. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/pynsist + + (2) https://pynsist.readthedocs.io + + +File: pythonpackagingguide.info, Node: Application bundles, Next: Configuration management, Prev: OS packaging & installers, Up: Deploying Python applications + +4.1.3 Application bundles +------------------------- + + FIXME + + - py2exe/py2app/PEX + - wheels kinda/sorta + + +File: pythonpackagingguide.info, Node: Configuration management, Prev: Application bundles, Up: Deploying Python applications + +4.1.4 Configuration management +------------------------------ + + FIXME + + puppet + salt + chef + ansible + fabric + + +File: pythonpackagingguide.info, Node: pip vs easy_install, Next: install_requires vs requirements files, Prev: Deploying Python applications, Up: Discussions + +4.2 pip vs easy_install +======================= + +*note setuptools: 18b. was released in 2004, as part of *note +setuptools: 2d. It was notable at the time for installing *note +packages: b. from *note PyPI: 39. using requirement specifiers, and +automatically installing dependencies. + +*note pip: 2b. came later in 2008, as alternative to *note setuptools: +18b, although still largely built on top of *note setuptools: 2d. +components. It was notable at the time for `not' installing packages as +*note Eggs: 7c. or from *note Eggs: 7c. (but rather simply as ‘flat’ +packages from *note sdists: 3d.), and introducing the idea of +Requirements Files(1), which gave users the power to easily replicate +environments. + +Here’s a breakdown of the important differences between pip and +easy_install now: + + `pip' `easy_install' + + +Installs from *note Wheels: d. Yes No + + +Uninstall Packages Yes (‘pip uninstall’) No + + +Dependency Overrides Yes (Requirements Files(2)) No + + +List Installed Packages Yes (‘pip list’ and ‘pip freeze’) No + + +PEP 438(3) Support Yes No + + +Installation format ‘Flat’ packages with ‘egg-info’ Encapsulated Egg format + metadata. + + +sys.path modification No Yes + + +Installs from *note Eggs: 7c. No Yes + + +pylauncher support(4) No Yes (5) + + +*note Multi-version installs: b5. No Yes + + +Exclude scripts during install No Yes + + +per project index Only in virtualenv Yes, via setup.cfg + + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + (2) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + (3) https://www.python.org/dev/peps/pep-0438 + + (4) https://bitbucket.org/vinay.sajip/pylauncher + + (5) (1) +‘https://setuptools.readthedocs.io/en/latest/easy_install.html#natural-script-launcher’ + + +File: pythonpackagingguide.info, Node: install_requires vs requirements files, Next: Wheel vs Egg, Prev: pip vs easy_install, Up: Discussions + +4.3 install_requires vs requirements files +========================================== + +* Menu: + +* install_requires: install_requires<2>. +* Requirements files: Requirements files<2>. + + +File: pythonpackagingguide.info, Node: install_requires<2>, Next: Requirements files<2>, Up: install_requires vs requirements files + +4.3.1 install_requires +---------------------- + +‘install_requires’ is a *note setuptools: 2d. ‘setup.py’ keyword that +should be used to specify what a project `minimally' needs to run +correctly. When the project is installed by *note pip: 2b, this is the +specification that is used to install its dependencies. + +For example, if the project requires A and B, your ‘install_requires’ +would be like so: + + install_requires=[ + 'A', + 'B' + ] + +Additionally, it’s best practice to indicate any known lower or upper +bounds. + +For example, it may be known, that your project requires at least v1 of +‘A’, and v2 of ‘B’, so it would be like so: + + install_requires=[ + 'A>=1', + 'B>=2' + ] + +It may also be known that project A follows semantic versioning, and +that v2 of ‘A’ will indicate a break in compatibility, so it makes sense +to not allow v2: + + install_requires=[ + 'A>=1,<2', + 'B>=2' + ] + +It is not considered best practice to use ‘install_requires’ to pin +dependencies to specific versions, or to specify sub-dependencies (i.e. +dependencies of your dependencies). This is overly-restrictive, and +prevents the user from gaining the benefit of dependency upgrades. + +Lastly, it’s important to understand that ‘install_requires’ is a +listing of “Abstract” requirements, i.e just names and version +restrictions that don’t determine where the dependencies will be +fulfilled from (i.e. from what index or source). The where (i.e. how +they are to be made “Concrete”) is to be determined at install time +using *note pip: 2b. options. (1) + + ---------- Footnotes ---------- + + (1) (1) For more on “Abstract” vs “Concrete” requirements, see +‘https://caremad.io/2013/07/setup-vs-requirement/’. + + +File: pythonpackagingguide.info, Node: Requirements files<2>, Prev: install_requires<2>, Up: install_requires vs requirements files + +4.3.2 Requirements files +------------------------ + +Requirements Files(1) described most simply, are just a list of pip +install(2) arguments placed into a file. + +Whereas ‘install_requires’ defines the dependencies for a single +project, Requirements Files(3) are often used to define the requirements +for a complete Python environment. + +Whereas ‘install_requires’ requirements are minimal, requirements files +often contain an exhaustive listing of pinned versions for the purpose +of achieving repeatable installations(4) of a complete environment. + +Whereas ‘install_requires’ requirements are “Abstract”, i.e. not +associated with any particular index, requirements files often contain +pip options like ‘--index-url’ or ‘--find-links’ to make requirements +“Concrete”, i.e. associated with a particular index or directory of +packages. (5) + +Whereas ‘install_requires’ metadata is automatically analyzed by pip +during an install, requirements files are not, and only are used when a +user specifically installs them using ‘pip install -r’. + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + (2) https://pip.pypa.io/en/latest/reference/pip_install/#pip-install + + (3) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + (4) https://pip.pypa.io/en/latest/user_guide/#repeatability + + (5) (1) For more on “Abstract” vs “Concrete” requirements, see +‘https://caremad.io/2013/07/setup-vs-requirement/’. + + +File: pythonpackagingguide.info, Node: Wheel vs Egg, Prev: install_requires vs requirements files, Up: Discussions + +4.4 Wheel vs Egg +================ + +*note Wheel: d. and *note Egg: 7c. are both packaging formats that aim +to support the use case of needing an install artifact that doesn’t +require building or compilation, which can be costly in testing and +production workflows. + +The *note Egg: 7c. format was introduced by *note setuptools: 2d. in +2004, whereas the *note Wheel: d. format was introduced by PEP 427(1) in +2012. + +*note Wheel: d. is currently considered the standard for *note built: +63. and *note binary: 194. packaging for Python. + +Here’s a breakdown of the important differences between *note Wheel: d. +and *note Egg: 7c. + + * *note Wheel: d. has an official PEP(2). *note Egg: 7c. did not. + + * *note Wheel: d. is a *note distribution: b. format, i.e a packaging + format. (3) *note Egg: 7c. was both a distribution format and a + runtime installation format (if left zipped), and was designed to + be importable. + + * *note Wheel: d. archives do not include .pyc files. Therefore, + when the distribution only contains Python files (i.e. no compiled + extensions), and is compatible with Python 2 and 3, it’s possible + for a wheel to be “universal”, similar to an *note sdist: 3d. + + * *note Wheel: d. uses PEP376-compliant(4) ‘.dist-info’ directories. + Egg used ‘.egg-info’. + + * *note Wheel: d. has a richer file naming convention(5). A single + wheel archive can indicate its compatibility with a number of + Python language versions and implementations, ABIs, and system + architectures. + + * *note Wheel: d. is versioned. Every wheel file contains the + version of the wheel specification and the implementation that + packaged it. + + * *note Wheel: d. is internally organized by sysconfig path type(6), + therefore making it easier to convert to other formats. + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0427 + + (2) https://www.python.org/dev/peps/pep-0427 + + (3) (1) Circumstantially, in some cases, wheels can be used as an +importable runtime format, although this is not officially supported at +this time +(https://www.python.org/dev/peps/pep-0427#is-it-possible-to-import-python-code-directly-from-a-wheel-file). + + (4) https://www.python.org/dev/peps/pep-0376 + + (5) https://www.python.org/dev/peps/pep-0425 + + (6) +http://docs.python.org/2/library/sysconfig.html#installation-paths + + +File: pythonpackagingguide.info, Node: PyPA specifications, Next: Project Summaries, Prev: Discussions, Up: Top + +5 PyPA specifications +********************* + +This is a list of currently active interoperability specifications +maintained by the Python Packaging Authority. The process for updating +these standards, and for proposing new ones, is documented on +pypa.io(1). + +* Menu: + +* Package Distribution Metadata:: +* Package Index Interfaces:: + + ---------- Footnotes ---------- + + (1) https://www.pypa.io/en/latest/specifications/ + + +File: pythonpackagingguide.info, Node: Package Distribution Metadata, Next: Package Index Interfaces, Up: PyPA specifications + +5.1 Package Distribution Metadata +================================= + +* Menu: + +* Core metadata specifications:: +* Version specifiers:: +* Dependency specifiers:: +* Declaring build system dependencies:: +* Declaring project metadata:: +* Distribution formats:: +* Platform compatibility tags:: +* Recording installed projects:: +* Entry points specification:: + + +File: pythonpackagingguide.info, Node: Core metadata specifications, Next: Version specifiers, Up: Package Distribution Metadata + +5.1.1 Core metadata specifications +---------------------------------- + +The current core metadata file format, version 2.1, is specified in PEP +566(1). It defines the following specification as the canonical source +for the core metadata file format. + +Fields defined in the following specification should be considered +valid, complete and not subject to change. The required fields are: + + - ‘Metadata-Version’ + + - ‘Name’ + + - ‘Version’ + +All the other fields are optional. + + Note: `Interpreting old metadata:' In PEP 566(2), the version + specifier field format specification was relaxed to accept the + syntax used by popular publishing tools (namely to remove the + requirement that version specifiers must be surrounded by + parentheses). Metadata consumers may want to use the more relaxed + formatting rules even for metadata files that are nominally less + than version 2.1. + +* Menu: + +* Metadata-Version:: +* Name:: +* Version:: +* Platform (multiple use): Platform multiple use. +* Supported-Platform (multiple use): Supported-Platform multiple use. +* Summary:: +* Description:: +* Description-Content-Type:: +* Keywords:: +* Home-page:: +* Download-URL:: +* Author:: +* Author-email:: +* Maintainer:: +* Maintainer-email:: +* License:: +* Classifier (multiple use): Classifier multiple use. +* Requires-Dist (multiple use): Requires-Dist multiple use. +* Requires-Python:: +* Requires-External (multiple use): Requires-External multiple use. +* Project-URL (multiple-use): Project-URL multiple-use. +* Provides-Extra (multiple use): Provides-Extra multiple use. +* Rarely Used Fields:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0566 + + (2) https://www.python.org/dev/peps/pep-0566 + + +File: pythonpackagingguide.info, Node: Metadata-Version, Next: Name, Up: Core metadata specifications + +5.1.1.1 Metadata-Version +........................ + +New in version 1.0. + +Version of the file format; legal values are “1.0”, “1.1”, “1.2” and +“2.1”. + +Automated tools consuming metadata SHOULD warn if ‘metadata_version’ is +greater than the highest version they support, and MUST fail if +‘metadata_version’ has a greater major version than the highest version +they support (as described in PEP 440(1), the major version is the value +before the first dot). + +For broader compatibility, build tools MAY choose to produce +distribution metadata using the lowest metadata version that includes +all of the needed fields. + +Example: + + Metadata-Version: 2.1 + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + +File: pythonpackagingguide.info, Node: Name, Next: Version, Prev: Metadata-Version, Up: Core metadata specifications + +5.1.1.2 Name +............ + +New in version 1.0. + +Changed in version 2.1: Added additional restrictions on format from PEP +508(1) + +The name of the distribution. The name field is the primary identifier +for a distribution. A valid name consists only of ASCII letters and +numbers, period, underscore and hyphen. It must start and end with a +letter or number. Distribution names are limited to those which match +the following regex (run with ‘re.IGNORECASE’): + + ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$ + +Example: + + Name: BeagleVote + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0508 + + +File: pythonpackagingguide.info, Node: Version, Next: Platform multiple use, Prev: Name, Up: Core metadata specifications + +5.1.1.3 Version +............... + +New in version 1.0. + +A string containing the distribution’s version number. This field must +be in the format specified in PEP 440(1). + +Example: + + Version: 1.0a2 + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + +File: pythonpackagingguide.info, Node: Platform multiple use, Next: Supported-Platform multiple use, Prev: Version, Up: Core metadata specifications + +5.1.1.4 Platform (multiple use) +............................... + +New in version 1.0. + +A Platform specification describing an operating system supported by the +distribution which is not listed in the “Operating System” Trove +classifiers. See “Classifier” below. + +Examples: + + Platform: ObscureUnix + Platform: RareDOS + + +File: pythonpackagingguide.info, Node: Supported-Platform multiple use, Next: Summary, Prev: Platform multiple use, Up: Core metadata specifications + +5.1.1.5 Supported-Platform (multiple use) +......................................... + +New in version 1.1. + +Binary distributions containing a PKG-INFO file will use the +Supported-Platform field in their metadata to specify the OS and CPU for +which the binary distribution was compiled. The semantics of the +Supported-Platform field are not specified in this PEP. + +Example: + + Supported-Platform: RedHat 7.2 + Supported-Platform: i386-win32-2791 + + +File: pythonpackagingguide.info, Node: Summary, Next: Description, Prev: Supported-Platform multiple use, Up: Core metadata specifications + +5.1.1.6 Summary +............... + +New in version 1.0. + +A one-line summary of what the distribution does. + +Example: + + Summary: A module for collecting votes from beagles. + + +File: pythonpackagingguide.info, Node: Description, Next: Description-Content-Type, Prev: Summary, Up: Core metadata specifications + +5.1.1.7 Description +................... + +New in version 1.0. + +Changed in version 2.1: This field may be specified in the message body +instead. + +A longer description of the distribution that can run to several +paragraphs. Software that deals with metadata should not assume any +maximum size for this field, though people shouldn’t include their +instruction manual as the description. + +The contents of this field can be written using reStructuredText markup +(1). For programs that work with the metadata, supporting markup is +optional; programs can also display the contents of the field as-is. +This means that authors should be conservative in the markup they use. + +To support empty lines and lines with indentation with respect to the +RFC 822 format, any CRLF character has to be suffixed by 7 spaces +followed by a pipe (“|”) char. As a result, the Description field is +encoded into a folded field that can be interpreted by RFC822 parser +(2). + +Example: + + Description: This project provides powerful math functions + |For example, you can use `sum()` to sum numbers: + | + |Example:: + | + | >>> sum(1, 2) + | 3 + | + +This encoding implies that any occurrences of a CRLF followed by 7 +spaces and a pipe char have to be replaced by a single CRLF when the +field is unfolded using a RFC822 reader. + +Alternatively, the distribution’s description may instead be provided in +the message body (i.e., after a completely blank line following the +headers, with no indentation or other special formatting necessary). + + ---------- Footnotes ---------- + + (1) (1) reStructuredText markup: ‘http://docutils.sourceforge.net/’ + + (2) (2) RFC 822 Long Header Fields: +‘http://www.freesoft.org/CIE/RFC/822/7.htm’ + + +File: pythonpackagingguide.info, Node: Description-Content-Type, Next: Keywords, Prev: Description, Up: Core metadata specifications + +5.1.1.8 Description-Content-Type +................................ + +New in version 2.1. + +A string stating the markup syntax (if any) used in the distribution’s +description, so that tools can intelligently render the description. + +Historically, PyPI supported descriptions in plain text and +reStructuredText (reST)(1), and could render reST into HTML. However, it +is common for distribution authors to write the description in +Markdown(2) (RFC 7763(3)) as many code hosting sites render Markdown +READMEs, and authors would reuse the file for the description. PyPI +didn’t recognize the format and so could not render the description +correctly. This resulted in many packages on PyPI with poorly-rendered +descriptions when Markdown is left as plain text, or worse, was +attempted to be rendered as reST. This field allows the distribution +author to specify the format of their description, opening up the +possibility for PyPI and other tools to be able to render Markdown and +other formats. + +The format of this field is the same as the ‘Content-Type’ header in +HTTP (i.e.: RFC 1341(4)). Briefly, this means that it has a +‘type/subtype’ part and then it can optionally have a number of +parameters: + +Format: + + Description-Content-Type: /; charset=[; = ...] + +The ‘type/subtype’ part has only a few legal values: + + - ‘text/plain’ + + - ‘text/x-rst’ + + - ‘text/markdown’ + +The ‘charset’ parameter can be used to specify the character encoding of +the description. The only legal value is ‘UTF-8’. If omitted, it is +assumed to be ‘UTF-8’. + +Other parameters might be specific to the chosen subtype. For example, +for the ‘markdown’ subtype, there is an optional ‘variant’ parameter +that allows specifying the variant of Markdown in use (defaults to ‘GFM’ +if not specified). Currently, two variants are recognized: + + - ‘GFM’ for Github-flavored Markdown(5) + + - ‘CommonMark’ for CommonMark(6) + +Example: + + Description-Content-Type: text/plain; charset=UTF-8 + +Example: + + Description-Content-Type: text/x-rst; charset=UTF-8 + +Example: + + Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM + +Example: + + Description-Content-Type: text/markdown + +If a ‘Description-Content-Type’ is not specified, then applications +should attempt to render it as ‘text/x-rst; charset=UTF-8’ and fall back +to ‘text/plain’ if it is not valid rst. + +If a ‘Description-Content-Type’ is an unrecognized value, then the +assumed content type is ‘text/plain’ (Although PyPI will probably reject +anything with an unrecognized value). + +If the ‘Description-Content-Type’ is ‘text/markdown’ and ‘variant’ is +not specified or is set to an unrecognized value, then the assumed +‘variant’ is ‘GFM’. + +So for the last example above, the ‘charset’ defaults to ‘UTF-8’ and the +‘variant’ defaults to ‘GFM’ and thus it is equivalent to the example +before it. + + ---------- Footnotes ---------- + + (1) +http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html + + (2) https://daringfireball.net/projects/markdown/ + + (3) https://tools.ietf.org/html/rfc7763 + + (4) https://www.w3.org/Protocols/rfc1341/4_Content-Type.html + + (5) https://tools.ietf.org/html/rfc7764#section-3.2 + + (6) https://tools.ietf.org/html/rfc7764#section-3.5 + + +File: pythonpackagingguide.info, Node: Keywords, Next: Home-page, Prev: Description-Content-Type, Up: Core metadata specifications + +5.1.1.9 Keywords +................ + +New in version 1.0. + +A list of additional keywords, separated by commas, to be used to assist +searching for the distribution in a larger catalog. + +Example: + + Keywords: dog,puppy,voting,election + + Note: The specification previously showed keywords separated by + spaces, but distutils and setuptools implemented it with commas. + These tools have been very widely used for many years, so it was + easier to update the specification to match the de facto standard. + + +File: pythonpackagingguide.info, Node: Home-page, Next: Download-URL, Prev: Keywords, Up: Core metadata specifications + +5.1.1.10 Home-page +.................. + +New in version 1.0. + +A string containing the URL for the distribution’s home page. + +Example: + + Home-page: http://www.example.com/~cschultz/bvote/ + + +File: pythonpackagingguide.info, Node: Download-URL, Next: Author, Prev: Home-page, Up: Core metadata specifications + +5.1.1.11 Download-URL +..................... + +New in version 1.1. + +A string containing the URL from which this version of the distribution +can be downloaded. (This means that the URL can’t be something like +“…/BeagleVote-latest.tgz”, but instead must be “…/BeagleVote-0.45.tgz”.) + + +File: pythonpackagingguide.info, Node: Author, Next: Author-email, Prev: Download-URL, Up: Core metadata specifications + +5.1.1.12 Author +............... + +New in version 1.0. + +A string containing the author’s name at a minimum; additional contact +information may be provided. + +Example: + + Author: C. Schultz, Universal Features Syndicate, + Los Angeles, CA + + +File: pythonpackagingguide.info, Node: Author-email, Next: Maintainer, Prev: Author, Up: Core metadata specifications + +5.1.1.13 Author-email +..................... + +New in version 1.0. + +A string containing the author’s e-mail address. It can contain a name +and e-mail address in the legal forms for a RFC-822 ‘From:’ header. + +Example: + + Author-email: "C. Schultz" + +Per RFC-822, this field may contain multiple comma-separated e-mail +addresses: + + Author-email: cschultz@example.com, snoopy@peanuts.com + + +File: pythonpackagingguide.info, Node: Maintainer, Next: Maintainer-email, Prev: Author-email, Up: Core metadata specifications + +5.1.1.14 Maintainer +................... + +New in version 1.2. + +A string containing the maintainer’s name at a minimum; additional +contact information may be provided. + +Note that this field is intended for use when a project is being +maintained by someone other than the original author: it should be +omitted if it is identical to ‘Author’. + +Example: + + Maintainer: C. Schultz, Universal Features Syndicate, + Los Angeles, CA + + +File: pythonpackagingguide.info, Node: Maintainer-email, Next: License, Prev: Maintainer, Up: Core metadata specifications + +5.1.1.15 Maintainer-email +......................... + +New in version 1.2. + +A string containing the maintainer’s e-mail address. It can contain a +name and e-mail address in the legal forms for a RFC-822 ‘From:’ header. + +Note that this field is intended for use when a project is being +maintained by someone other than the original author: it should be +omitted if it is identical to ‘Author-email’. + +Example: + + Maintainer-email: "C. Schultz" + +Per RFC-822, this field may contain multiple comma-separated e-mail +addresses: + + Maintainer-email: cschultz@example.com, snoopy@peanuts.com + + +File: pythonpackagingguide.info, Node: License, Next: Classifier multiple use, Prev: Maintainer-email, Up: Core metadata specifications + +5.1.1.16 License +................ + +New in version 1.0. + +Text indicating the license covering the distribution where the license +is not a selection from the “License” Trove classifiers. See *note +“Classifier”: 1bd. below. This field may also be used to specify a +particular version of a license which is named via the ‘Classifier’ +field, or to indicate a variation or exception to such a license. + +Examples: + + License: This software may only be obtained by sending the + author a postcard, and then the user promises not + to redistribute it. + + License: GPL version 3, excluding DRM provisions + + +File: pythonpackagingguide.info, Node: Classifier multiple use, Next: Requires-Dist multiple use, Prev: License, Up: Core metadata specifications + +5.1.1.17 Classifier (multiple use) +.................................. + +New in version 1.1. + +Each entry is a string giving a single classification value for the +distribution. Classifiers are described in PEP 301(1), and the Python +Package Index publishes a dynamic list of currently defined +classifiers(2). + +This field may be followed by an environment marker after a semicolon. + +Examples: + + Classifier: Development Status :: 4 - Beta + Classifier: Environment :: Console (Text Based) + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0301 + + (2) https://pypi.org/classifiers/ + + +File: pythonpackagingguide.info, Node: Requires-Dist multiple use, Next: Requires-Python, Prev: Classifier multiple use, Up: Core metadata specifications + +5.1.1.18 Requires-Dist (multiple use) +..................................... + +New in version 1.2. + +Changed in version 2.1: The field format specification was relaxed to +accept the syntax used by popular publishing tools. + +Each entry contains a string naming some other distutils project +required by this distribution. + +The format of a requirement string contains from one to four parts: + + * A project name, in the same format as the ‘Name:’ field. The only + mandatory part. + + * A comma-separated list of ‘extra’ names. These are defined by the + required project, referring to specific features which may need + extra dependencies. + + * A version specifier. Tools parsing the format should accept + optional parentheses around this, but tools generating it should + not use parentheses. + + * An environment marker after a semicolon. This means that the + requirement is only needed in the specified conditions. + +See PEP 508(1) for full details of the allowed format. + +The project names should correspond to names as found on the Python +Package Index(2). + +Version specifiers must follow the rules described in *note Version +specifiers: 1c2. + +Examples: + + Requires-Dist: pkginfo + Requires-Dist: PasteDeploy + Requires-Dist: zope.interface (>3.5.0) + Requires-Dist: pywin32 >1.0; sys_platform == 'win32' + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0508 + + (2) http://pypi.org/ + + +File: pythonpackagingguide.info, Node: Requires-Python, Next: Requires-External multiple use, Prev: Requires-Dist multiple use, Up: Core metadata specifications + +5.1.1.19 Requires-Python +........................ + +New in version 1.2. + +This field specifies the Python version(s) that the distribution is +guaranteed to be compatible with. Installation tools may look at this +when picking which version of a project to install. + +The value must be in the format specified in *note Version specifiers: +1c2. + +This field cannot be followed by an environment marker. + +Examples: + + Requires-Python: >=3 + Requires-Python: >2.6,!=3.0.*,!=3.1.* + Requires-Python: ~=2.6 + + +File: pythonpackagingguide.info, Node: Requires-External multiple use, Next: Project-URL multiple-use, Prev: Requires-Python, Up: Core metadata specifications + +5.1.1.20 Requires-External (multiple use) +......................................... + +New in version 1.2. + +Changed in version 2.1: The field format specification was relaxed to +accept the syntax used by popular publishing tools. + +Each entry contains a string describing some dependency in the system +that the distribution is to be used. This field is intended to serve as +a hint to downstream project maintainers, and has no semantics which are +meaningful to the ‘distutils’ distribution. + +The format of a requirement string is a name of an external dependency, +optionally followed by a version declaration within parentheses. + +This field may be followed by an environment marker after a semicolon. + +Because they refer to non-Python software releases, version numbers for +this field are `not' required to conform to the format specified in PEP +440(1): they should correspond to the version scheme used by the +external dependency. + +Notice that there is no particular rule on the strings to be used. + +Examples: + + Requires-External: C + Requires-External: libpng (>=1.5) + Requires-External: make; sys_platform != "win32" + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + +File: pythonpackagingguide.info, Node: Project-URL multiple-use, Next: Provides-Extra multiple use, Prev: Requires-External multiple use, Up: Core metadata specifications + +5.1.1.21 Project-URL (multiple-use) +................................... + +New in version 1.2. + +A string containing a browsable URL for the project and a label for it, +separated by a comma. + +Example: + + Project-URL: Bug Tracker, http://bitbucket.org/tarek/distribute/issues/ + +The label is free text limited to 32 characters. + + +File: pythonpackagingguide.info, Node: Provides-Extra multiple use, Next: Rarely Used Fields, Prev: Project-URL multiple-use, Up: Core metadata specifications + +5.1.1.22 Provides-Extra (multiple use) +...................................... + +New in version 2.1. + +A string containing the name of an optional feature. Must be a valid +Python identifier. May be used to make a dependency conditional on +whether the optional feature has been requested. + +Example: + + Provides-Extra: pdf + Requires-Dist: reportlab; extra == 'pdf' + +A second distribution requires an optional dependency by placing it +inside square brackets, and can request multiple features by separating +them with a comma (,). The requirements are evaluated for each +requested feature and added to the set of requirements for the +distribution. + +Example: + + Requires-Dist: beaglevote[pdf] + Requires-Dist: libexample[test, doc] + +Two feature names ‘test’ and ‘doc’ are reserved to mark dependencies +that are needed for running automated tests and generating +documentation, respectively. + +It is legal to specify ‘Provides-Extra:’ without referencing it in any +‘Requires-Dist:’. + + +File: pythonpackagingguide.info, Node: Rarely Used Fields, Prev: Provides-Extra multiple use, Up: Core metadata specifications + +5.1.1.23 Rarely Used Fields +........................... + +The fields in this section are currently rarely used, as their design +was inspired by comparable mechanisms in Linux package management +systems, and it isn’t at all clear how tools should interpret them in +the context of an open index server such as PyPI(1). + +As a result, popular installation tools ignore them completely, which in +turn means there is little incentive for package publishers to set them +appropriately. However, they’re retained in the metadata specification, +as they’re still potentially useful for informational purposes, and can +also be used for their originally intended purpose in combination with a +curated package repository. + +* Menu: + +* Provides-Dist (multiple use): Provides-Dist multiple use. +* Obsoletes-Dist (multiple use): Obsoletes-Dist multiple use. + + ---------- Footnotes ---------- + + (1) https://pypi.org + + +File: pythonpackagingguide.info, Node: Provides-Dist multiple use, Next: Obsoletes-Dist multiple use, Up: Rarely Used Fields + +5.1.1.24 Provides-Dist (multiple use) +..................................... + +New in version 1.2. + +Changed in version 2.1: The field format specification was relaxed to +accept the syntax used by popular publishing tools. + +Each entry contains a string naming a Distutils project which is +contained within this distribution. This field `must' include the +project identified in the ‘Name’ field, followed by the version : Name +(Version). + +A distribution may provide additional names, e.g. to indicate that +multiple projects have been bundled together. For instance, source +distributions of the ‘ZODB’ project have historically included the +‘transaction’ project, which is now available as a separate +distribution. Installing such a source distribution satisfies +requirements for both ‘ZODB’ and ‘transaction’. + +A distribution may also provide a “virtual” project name, which does not +correspond to any separately-distributed project: such a name might be +used to indicate an abstract capability which could be supplied by one +of multiple projects. E.g., multiple projects might supply RDBMS +bindings for use by a given ORM: each project might declare that it +provides ‘ORM-bindings’, allowing other projects to depend only on +having at most one of them installed. + +A version declaration may be supplied and must follow the rules +described in *note Version specifiers: 1c2. The distribution’s version +number will be implied if none is specified. + +This field may be followed by an environment marker after a semicolon. + +Examples: + + Provides-Dist: OtherProject + Provides-Dist: AnotherProject (3.4) + Provides-Dist: virtual_package; python_version >= "3.4" + + +File: pythonpackagingguide.info, Node: Obsoletes-Dist multiple use, Prev: Provides-Dist multiple use, Up: Rarely Used Fields + +5.1.1.25 Obsoletes-Dist (multiple use) +...................................... + +New in version 1.2. + +Changed in version 2.1: The field format specification was relaxed to +accept the syntax used by popular publishing tools. + +Each entry contains a string describing a distutils project’s +distribution which this distribution renders obsolete, meaning that the +two projects should not be installed at the same time. + +Version declarations can be supplied. Version numbers must be in the +format specified in *note Version specifiers: 1c2. + +This field may be followed by an environment marker after a semicolon. + +The most common use of this field will be in case a project name +changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0. When +you install Torqued Python, the Gorgon distribution should be removed. + +Examples: + + Obsoletes-Dist: Gorgon + Obsoletes-Dist: OtherProject (<3.0) + Obsoletes-Dist: Foo; os_name == "posix" + +__________________________________________________________________ + + +File: pythonpackagingguide.info, Node: Version specifiers, Next: Dependency specifiers, Prev: Core metadata specifications, Up: Package Distribution Metadata + +5.1.2 Version specifiers +------------------------ + +Version numbering requirements and the semantics for specifying +comparisons between versions are defined in PEP 440(1). + +The version specifiers section in this PEP supersedes the version +specifiers section in PEP 345(2). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440 + + (2) https://www.python.org/dev/peps/pep-0345 + + +File: pythonpackagingguide.info, Node: Dependency specifiers, Next: Declaring build system dependencies, Prev: Version specifiers, Up: Package Distribution Metadata + +5.1.3 Dependency specifiers +--------------------------- + +The dependency specifier format used to declare a dependency on another +component is defined in PEP 508(1). + +The environment markers section in this PEP supersedes the environment +markers section in PEP 345(2). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0508 + + (2) https://www.python.org/dev/peps/pep-0345 + + +File: pythonpackagingguide.info, Node: Declaring build system dependencies, Next: Declaring project metadata, Prev: Dependency specifiers, Up: Package Distribution Metadata + +5.1.4 Declaring build system dependencies +----------------------------------------- + +*note pyproject.toml: 1d7. is a build system independent file format +defined in PEP 518(1) that projects may provide in order to declare any +Python level dependencies that must be installed in order to run the +project’s build system successfully. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0518 + + +File: pythonpackagingguide.info, Node: Declaring project metadata, Next: Distribution formats, Prev: Declaring build system dependencies, Up: Package Distribution Metadata + +5.1.5 Declaring project metadata +-------------------------------- + +PEP 621(1) specifies how to write a project’s *note core metadata: 85. +in a ‘pyproject.toml’ file for packaging-related tools to consume. It +defines the following specification as the canonical source for the +format used. + +There are two kinds of metadata: `static' and `dynamic'. Static +metadata is specified in the ‘pyproject.toml’ file directly and cannot +be specified or changed by a tool. Dynamic metadata is listed via the +‘dynamic’ field (defined later in this specification) and represents +metadata that a tool will later provide. + +The fields defined in this specification MUST be in a table named +‘[project]’ in ‘pyproject.toml’. No tools may add fields to this table +which are not defined by this specification. For tools wishing to store +their own settings in ‘pyproject.toml’, they may use the ‘[tool]’ table +as defined in the *note build dependency decleration specification: 1d5. +The lack of a ‘[project]’ table implicitly means the build back-end will +dynamically provide all fields. + +The only fields required to be statically defined are: + + - ‘name’ + +The fields which are required but may be specified `either' statically +or listed as dynamic are: + + - ‘version’ + +All other fields are considered optional and my be specified statically, +listed as dynamic, or left unspecified. + +* Menu: + +* name: name<2>. +* version: version<2>. +* description: description<2>. +* readme:: +* requires-python:: +* license: license<2>. +* authors/maintainers:: +* keywords: keywords<2>. +* classifiers: classifiers<2>. +* urls:: +* Entry points:: +* dependencies/optional-dependencies:: +* dynamic:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0621 + + +File: pythonpackagingguide.info, Node: name<2>, Next: version<2>, Up: Declaring project metadata + +5.1.5.1 ‘name’ +.............. + + - TOML(1) type: string + + - Corresponding *note core metadata: 85. field: *note Name: 19c. + +The name of the project. + +Tools SHOULD normalize this name, as specified by PEP 503(2), as soon as +it is read for internal consistency. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + (2) https://www.python.org/dev/peps/pep-0503 + + +File: pythonpackagingguide.info, Node: version<2>, Next: description<2>, Prev: name<2>, Up: Declaring project metadata + +5.1.5.2 ‘version’ +................. + + - TOML(1) type: string + + - Corresponding *note core metadata: 85. field: *note Version: 19e. + +The version of the project as supported by PEP 440(2). + +Users SHOULD prefer to specify already-normalized versions. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + (2) https://www.python.org/dev/peps/pep-0440 + + +File: pythonpackagingguide.info, Node: description<2>, Next: readme, Prev: version<2>, Up: Declaring project metadata + +5.1.5.3 ‘description’ +..................... + + - TOML(1) type: string + + - Corresponding *note core metadata: 85. field: *note Summary: 1a2. + +The summary description of the project. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: readme, Next: requires-python, Prev: description<2>, Up: Declaring project metadata + +5.1.5.4 ‘readme’ +................ + + - TOML(1) type: string or table + + - Corresponding *note core metadata: 85. field: *note Description: + 1a4. and *note Description-Content-Type: 1a6. + +The full description of the project (i.e. the README). + +The field accepts either a string or a table. If it is a string then it +is a path relative to ‘pyproject.toml’ to a text file containing the +full description. Tools MUST assume the file’s encoding is UTF-8. If +the file path ends in a case-insensitive ‘.md’ suffix, then tools MUST +assume the content-type is ‘text/markdown’. If the file path ends in a +case-insensitive ‘.rst’, then tools MUST assume the content-type is +‘text/x-rst’. If a tool recognizes more extensions than this PEP, they +MAY infer the content-type for the user without specifying this field as +‘dynamic’. For all unrecognized suffixes when a content-type is not +provided, tools MUST raise an error. + +The ‘readme’ field may also take a table. The ‘file’ key has a string +value representing a path relative to ‘pyproject.toml’ to a file +containing the full description. The ‘text’ key has a string value +which is the full description. These keys are mutually-exclusive, thus +tools MUST raise an error if the metadata specifies both keys. + +A table specified in the ‘readme’ field also has a ‘content-type’ field +which takes a string specifying the content-type of the full +description. A tool MUST raise an error if the metadata does not +specify this field in the table. If the metadata does not specify the +‘charset’ parameter, then it is assumed to be UTF-8. Tools MAY support +other encodings if they choose to. Tools MAY support alternative +content-types which they can transform to a content-type as supported by +the *note core metadata: 85. Otherwise tools MUST raise an error for +unsupported content-types. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: requires-python, Next: license<2>, Prev: readme, Up: Declaring project metadata + +5.1.5.5 ‘requires-python’ +......................... + + - TOML(1) type: string + + - Corresponding *note core metadata: 85. field: *note + Requires-Python: 1c3. + +The Python version requirements of the project. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: license<2>, Next: authors/maintainers, Prev: requires-python, Up: Declaring project metadata + +5.1.5.6 ‘license’ +................. + + - TOML(1) type: table + + - Corresponding *note core metadata: 85. field: *note License: 1bb. + +The table may have one of two keys. The ‘file’ key has a string value +that is a file path relative to ‘pyproject.toml’ to the file which +contains the license for the project. Tools MUST assume the file’s +encoding is UTF-8. The ‘text’ key has a string value which is the +license of the project. These keys are mutually exclusive, so a tool +MUST raise an error if the metadata specifies both keys. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: authors/maintainers, Next: keywords<2>, Prev: license<2>, Up: Declaring project metadata + +5.1.5.7 ‘authors’/‘maintainers’ +............................... + + - TOML(1) type: Array of inline tables with string keys and values + + - Corresponding *note core metadata: 85. field: *note Author: 1b0, + *note Author-email: 1b3, *note Maintainer: 1b5, and *note + Maintainer-email: 1b8. + +The people or organizations considered to be the “authors” of the +project. The exact meaning is open to interpretation — it may list the +original or primary authors, current maintainers, or owners of the +package. + +The “maintainers” field is similar to “authors” in that its exact +meaning is open to interpretation. + +These fields accept an array of tables with 2 keys: ‘name’ and ‘email’. +Both values must be strings. The ‘name’ value MUST be a valid email +name (i.e. whatever can be put as a name, before an email, in RFC +822(2)) and not contain commas. The ‘email’ value MUST be a valid email +address. Both keys are optional. + +Using the data to fill in *note core metadata: 85. is as follows: + + 1. If only ‘name’ is provided, the value goes in *note Author: 1b0. or + *note Maintainer: 1b5. as appropriate. + + 2. If only ‘email’ is provided, the value goes in *note Author-email: + 1b3. or *note Maintainer-email: 1b8. as appropriate. + + 3. If both ‘email’ and ‘name’ are provided, the value goes in *note + Author-email: 1b3. or *note Maintainer-email: 1b8. as appropriate, + with the format ‘{name} <{email}>’. + + 4. Multiple values should be separated by commas. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + (2) https://tools.ietf.org/html/rfc822 + + +File: pythonpackagingguide.info, Node: keywords<2>, Next: classifiers<2>, Prev: authors/maintainers, Up: Declaring project metadata + +5.1.5.8 ‘keywords’ +.................. + + - TOML(1) type: array of strings + + - Corresponding *note core metadata: 85. field: *note Keywords: 1a9. + +The keywords for the project. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: classifiers<2>, Next: urls, Prev: keywords<2>, Up: Declaring project metadata + +5.1.5.9 ‘classifiers’ +..................... + + - TOML(1) type: array of strings + + - Corresponding *note core metadata: 85. field: *note Classifier: + 1bf. + +Trove classifiers which apply to the project. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: urls, Next: Entry points, Prev: classifiers<2>, Up: Declaring project metadata + +5.1.5.10 ‘urls’ +............... + + - TOML(1) type: table with keys and values of strings + + - Corresponding *note core metadata: 85. field: *note Project-URL: + 1c6. + +A table of URLs where the key is the URL label and the value is the URL +itself. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: Entry points, Next: dependencies/optional-dependencies, Prev: urls, Up: Declaring project metadata + +5.1.5.11 Entry points +..................... + + - TOML(1) type: table (‘[project.scripts]’, ‘[project.gui-scripts]’, + and ‘[project.entry-points]’) + + - *note Entry points specification: 1e6. + +There are three tables related to entry points. The ‘[project.scripts]’ +table corresponds to the ‘console_scripts’ group in the *note entry +points specification: 1e6. The key of the table is the name of the +entry point and the value is the object reference. + +The ‘[project.gui-scripts]’ table corresponds to the ‘gui_scripts’ group +in the *note entry points specification: 1e6. Its format is the same as +‘[project.scripts]’. + +The ‘[project.entry-points]’ table is a collection of tables. Each +sub-table’s name is an entry point group. The key and value semantics +are the same as ‘[project.scripts]’. Users MUST NOT create nested +sub-tables but instead keep the entry point groups to only one level +deep. + +Build back-ends MUST raise an error if the metadata defines a +‘[project.entry-points.console_scripts]’ or +‘[project.entry-points.gui_scripts]’ table, as they would be ambiguous +in the face of ‘[project.scripts]’ and ‘[project.gui-scripts]’, +respectively. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: dependencies/optional-dependencies, Next: dynamic, Prev: Entry points, Up: Declaring project metadata + +5.1.5.12 ‘dependencies’/‘optional-dependencies’ +............................................... + + - TOML(1) type: Array of PEP 508(2) strings (‘dependencies’), and a + table with values of arrays of PEP 508(3) strings + (‘optional-dependencies’) + + - Corresponding *note core metadata: 85. field: *note Requires-Dist: + 1c0. and *note Provides-Extra: 1c9. + +The (optional) dependencies of the project. + +For ‘dependencies’, it is a key whose value is an array of strings. +Each string represents a dependency of the project and MUST be formatted +as a valid PEP 508(4) string. Each string maps directly to a *note +Requires-Dist: 1c0. entry. + +For ‘optional-dependencies’, it is a table where each key specifies an +extra and whose value is an array of strings. The strings of the arrays +must be valid PEP 508(5) strings. The keys MUST be valid values for +*note Provides-Extra: 1c9. Each value in the array thus becomes a +corresponding *note Requires-Dist: 1c0. entry for the matching *note +Provides-Extra: 1c9. metadata. + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + (2) https://www.python.org/dev/peps/pep-0508 + + (3) https://www.python.org/dev/peps/pep-0508 + + (4) https://www.python.org/dev/peps/pep-0508 + + (5) https://www.python.org/dev/peps/pep-0508 + + +File: pythonpackagingguide.info, Node: dynamic, Prev: dependencies/optional-dependencies, Up: Declaring project metadata + +5.1.5.13 ‘dynamic’ +.................. + + - TOML(1) type: array of string + + - A corresponding *note core metadata: 85. field does not exist + +Specifies which fields listed by this PEP were intentionally unspecified +so another tool can/will provide such metadata dynamically. This +clearly delineates which metadata is purposefully unspecified and +expected to stay unspecified compared to being provided via tooling +later on. + + - A build back-end MUST honour statically-specified metadata (which + means the metadata did not list the field in ‘dynamic’). + + - A build back-end MUST raise an error if the metadata specifies + ‘name’ in ‘dynamic’. + + - If the *note core metadata: 85. specification lists a field as + “Required”, then the metadata MUST specify the field statically or + list it in ‘dynamic’ (build back-ends MUST raise an error + otherwise, i.e. it should not be possible for a required field to + not be listed somehow in the ‘[project]’ table). + + - If the *note core metadata: 85. specification lists a field as + “Optional”, the metadata MAY list it in ‘dynamic’ if the + expectation is a build back-end will provide the data for the field + later. + + - Build back-ends MUST raise an error if the metadata specifies a + field statically as well as being listed in ‘dynamic’. + + - If the metadata does not list a field in ‘dynamic’, then a build + back-end CANNOT fill in the requisite metadata on behalf of the + user (i.e. ‘dynamic’ is the only way to allow a tool to fill in + metadata and the user must opt into the filling in). + + - Build back-ends MUST raise an error if the metadata specifies a + field in ‘dynamic’ but the build back-end was unable to determine + the data for it (omitting the data, if determined to be the + accurate value, is acceptable). + + ---------- Footnotes ---------- + + (1) https://toml.io/ + + +File: pythonpackagingguide.info, Node: Distribution formats, Next: Platform compatibility tags, Prev: Declaring project metadata, Up: Package Distribution Metadata + +5.1.6 Distribution formats +-------------------------- + +* Menu: + +* Source distribution format:: +* Binary distribution format:: + + +File: pythonpackagingguide.info, Node: Source distribution format, Next: Binary distribution format, Up: Distribution formats + +5.1.6.1 Source distribution format +.................................. + +The accepted style of source distribution format based on +‘pyproject.toml’, defined in PEP 518(1) and adopted by PEP 517(2) has +not been implemented yet. + +There is also the legacy source distribution format, implicitly defined +by the behaviour of ‘distutils’ module in the standard library, when +executing ‘setup.py sdist’. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0518 + + (2) https://www.python.org/dev/peps/pep-0517 + + +File: pythonpackagingguide.info, Node: Binary distribution format, Prev: Source distribution format, Up: Distribution formats + +5.1.6.2 Binary distribution format +.................................. + +The binary distribution format (‘wheel’) is defined in PEP 427(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0427 + + +File: pythonpackagingguide.info, Node: Platform compatibility tags, Next: Recording installed projects, Prev: Distribution formats, Up: Package Distribution Metadata + +5.1.7 Platform compatibility tags +--------------------------------- + +Platform compatibility tags allow build tools to mark distributions as +being compatible with specific platforms, and allows installers to +understand which distributions are compatible with the system they are +running on. + +The platform compatibility tagging model used for the ‘wheel’ +distribution format is defined in PEP 425(1). + +* Menu: + +* Platform tags for Windows:: +* Platform tags for macOS (Mac OS X): Platform tags for macOS Mac OS X. +* Platform tags for common Linux distributions:: +* Platform tags for other *nix platforms:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + +File: pythonpackagingguide.info, Node: Platform tags for Windows, Next: Platform tags for macOS Mac OS X, Up: Platform compatibility tags + +5.1.7.1 Platform tags for Windows +................................. + +The scheme defined in PEP 425(1) covers public distribution of wheel +files to systems running Windows. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + +File: pythonpackagingguide.info, Node: Platform tags for macOS Mac OS X, Next: Platform tags for common Linux distributions, Prev: Platform tags for Windows, Up: Platform compatibility tags + +5.1.7.2 Platform tags for macOS (Mac OS X) +.......................................... + +The scheme defined in PEP 425(1) covers public distribution of wheel +files to systems running macOS (previously known as Mac OS X). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + +File: pythonpackagingguide.info, Node: Platform tags for common Linux distributions, Next: Platform tags for other *nix platforms, Prev: Platform tags for macOS Mac OS X, Up: Platform compatibility tags + +5.1.7.3 Platform tags for common Linux distributions +.................................................... + +The scheme defined in PEP 425(1) is insufficient for public distribution +of wheel files (and *nix wheel files in general) to Linux platforms, due +to the large ecosystem of Linux platforms and subtle differences between +them. + +Instead, PEP 513(2) defines the ‘manylinux’ standard, which represents a +common subset of Linux platforms, and allows building wheels tagged with +the ‘manylinux’ platform tag which can be used across most common Linux +distributions. + +There are multiple iterations of the ‘manylinux’ specification, each +representing the common subset of Linux platforms at a given point in +time: + + * ‘manylinux1’ ( PEP 513(3)) supports ‘x86_64’ and ‘i686’ + architectures, and is based on a compatible Linux platform from + 2007. + + * ‘manylinux2010’ ( PEP 571(4)) supports ‘x86_64’ and ‘i686’ + architectures. and updates the previous specification to be based + on a compatible Linux platform from 2010 instead. + + * ‘manylinux2014’ ( PEP 599(5)) adds support for a number of + additional architectures (‘aarch64’, ‘armv7l’, ‘ppc64’, ‘ppc64le’, + and ‘s390x’) and updates the base platform to a compatible Linux + platform from 2014. + +In general, distributions built for older versions of the specification +are forwards-compatible (meaning that ‘manylinux1’ distributions should +continue to work on modern systems) but not backwards-compatible +(meaning that ‘manylinux2010’ distributions are not expected to work on +platforms that existed before 2010). + +Package maintainers should attempt to target the most compatible +specification possible, with the caveat that the provided build +environment for ‘manylinux1’ has reached end-of-life, and the build +environment for ‘manylinux2010’ will reach end-of-life in November 2020 +(6), meaning that these images will no longer receive security updates. + +* Menu: + +* Manylinux compatibility support:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + (2) https://www.python.org/dev/peps/pep-0513 + + (3) https://www.python.org/dev/peps/pep-0513 + + (4) https://www.python.org/dev/peps/pep-0571 + + (5) https://www.python.org/dev/peps/pep-0599 + + (6) (1) ‘https://wiki.centos.org/About/Product’ + + +File: pythonpackagingguide.info, Node: Manylinux compatibility support, Up: Platform tags for common Linux distributions + +5.1.7.4 Manylinux compatibility support +....................................... + + Note: The ‘manylinux2014’ specification is relatively new and is + not yet widely recognised by install tools. + +The following table shows the minimum versions of relevant projects to +support the various ‘manylinux’ standards: + +Tool ‘manylinux1’ ‘manylinux2010’ ‘manylinux2014’ + +------------------------------------------------------------------------------ + +pip ‘>=8.1.0’ ‘>=19.0’ ‘>=19.3’ + + +auditwheel ‘>=1.0.0’ ‘>=2.0.0’ ‘>=3.0.0’ + + + +File: pythonpackagingguide.info, Node: Platform tags for other *nix platforms, Prev: Platform tags for common Linux distributions, Up: Platform compatibility tags + +5.1.7.5 Platform tags for other *nix platforms +.............................................. + +The scheme defined in PEP 425(1) is not generally sufficient for public +distribution of wheel files to other *nix platforms. Efforts are +currently (albeit intermittently) under way to define improved +compatibility tagging schemes for AIX and for Alpine Linux. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0425 + + +File: pythonpackagingguide.info, Node: Recording installed projects, Next: Entry points specification, Prev: Platform compatibility tags, Up: Package Distribution Metadata + +5.1.8 Recording installed projects +---------------------------------- + +This document specifies a common format of recording information about +Python *note projects: 1fc. installed in an environment. A common +metadata format allows tools to query, manage or uninstall projects, +regardless of how they were installed. + +Almost all information is optional. This allows tools outside the +Python ecosystem, such as Linux package managers, to integrate with +Python tooling as much as possible. For example, even if an installer +cannot easily provide a list of installed files in a format specific to +Python tooling, it should still record the name and version of the +installed project. + +* Menu: + +* History and change workflow:: +* The .dist-info directory: The dist-info directory. +* The METADATA file:: +* The RECORD file:: +* The INSTALLER file:: + + +File: pythonpackagingguide.info, Node: History and change workflow, Next: The dist-info directory, Up: Recording installed projects + +5.1.8.1 History and change workflow +................................... + +The metadata described here was first specified in PEP 376(1), and later +ammended in PEP 627(2). It was formerly known as `Database of Installed +Python Distributions'. Further amendments (except trivial language or +typography fixes) must be made through the PEP process (see PEP 1(3)). + +While this document is the normative specification, these PEPs that +introduce changes to it may include additional information such as +rationales and backwards compatibility considerations. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0376 + + (2) https://www.python.org/dev/peps/pep-0627 + + (3) https://www.python.org/dev/peps/pep-0001 + + +File: pythonpackagingguide.info, Node: The dist-info directory, Next: The METADATA file, Prev: History and change workflow, Up: Recording installed projects + +5.1.8.2 The .dist-info directory +................................ + +Each project installed from a distribution must, in addition to files, +install a “‘.dist-info’” directory located alongside importable modules +and packages (commonly, the ‘site-packages’ directory). + +This directory is named as ‘{name}-{version}.dist-info’, with name(1) +and Distribution versions(2) fields corresponding to *note Core metadata +specifications: 85. The name field must be in normalized form (see PEP +503(3) for the definition of normalization). + +This ‘.dist-info’ directory can contain these files, described in detail +below: + + * ‘METADATA’: contains project metadata + + * ‘RECORD’: records the list of installed files. + + * ‘INSTALLER’: records the name of the tool used to install the + project. + +The ‘METADATA’ file is mandatory. All other files may be omitted at the +installing tool’s discretion. Additional installer-specific files may +be present. + + Note: The *note Binary distribution format: 1ee. specification + describes additional files that may appear in the ‘.dist-info’ + directory of a *note Wheel: d. Such files may be copied to the + ‘.dist-info’ directory of an installed project. + +The previous versions of this specification also specified a ‘REQUESTED’ +file. This file is now considered a tool-specific extension, but may be +standardized again in the future. See PEP 376(4) for its original +meaning. + + ---------- Footnotes ---------- + + (1) +https://docs.python.org/2/reference/simple_stmts.html#grammar-token-name + + (2) https://docs.python.org/3/library/importlib.metadata.html#version + + (3) https://www.python.org/dev/peps/pep-0503/#normalized-names + + (4) https://www.python.org/dev/peps/pep-0376/#requested + + +File: pythonpackagingguide.info, Node: The METADATA file, Next: The RECORD file, Prev: The dist-info directory, Up: Recording installed projects + +5.1.8.3 The METADATA file +......................... + +The ‘METADATA’ file contains metadata as described in the *note Core +metadata specifications: 85. specification, version 1.1 or greater. + +The ‘METADATA’ file is mandatory. If it cannot be created, or if +required core metadata is not available, installers must report an error +and fail to install the project. + + +File: pythonpackagingguide.info, Node: The RECORD file, Next: The INSTALLER file, Prev: The METADATA file, Up: Recording installed projects + +5.1.8.4 The RECORD file +....................... + +The ‘RECORD’ file holds the list of installed files. It is a CSV file +containing one record (line) per installed file. + +The CSV dialect must be readable with the default ‘reader’ of Python’s +‘csv’ module: + + * field delimiter: ‘,’ (comma), + + * quoting char: ‘"’ (straight double quote), + + * line terminator: either ‘\r\n’ or ‘\n’. + +Each record is composed of three elements: the file’s `path', the `hash' +of the contents, and its `size'. + +The `path' may be either absolute, or relative to the directory +containing the ‘.dist-info’ directory (commonly, the ‘site-packages’ +directory). On Windows, directories may be separated either by forward- +or backslashes (‘/’ or ‘\’). + +The `hash' is either an empty string or the name of a hash algorithm +from ‘hashlib.algorithms_guaranteed’, followed by the equals character +‘=’ and the digest of the file’s contents, encoded with the +urlsafe-base64-nopad encoding (‘base64.urlsafe_b64encode(digest)’ with +trailing ‘=’ removed). + +The `size' is either the empty string, or file’s size in bytes, as a +base 10 integer. + +For any file, either or both of the `hash' and `size' fields may be left +empty. Commonly, entries for ‘.pyc’ files and the ‘RECORD’ file itself +have empty `hash' and `size'. For other files, leaving the information +out is discouraged, as it prevents verifying the integrity of the +installed project. + +If the ‘RECORD’ file is present, it must list all installed files of the +project, except ‘.pyc’ files corresponding to ‘.py’ files listed in +‘RECORD’, which are optional. Notably, the contents of the ‘.dist-info’ +directory (including the ‘RECORD’ file itself) must be listed. +Directories should not be listed. + +To completely uninstall a package, a tool needs to remove all files +listed in ‘RECORD’, all ‘.pyc’ files (of all optimization levels) +corresponding to removed ‘.py’ files, and any directories emptied by the +uninstallation. + +Here is an example snippet of a possible ‘RECORD’ file: + + /usr/bin/black,sha256=iFlOnL32lIa-RKk-MDihcbJ37wxmRbE4xk6eVYVTTeU,220 + ../../../bin/blackd,sha256=lCadt4mcU-B67O1gkQVh7-vsKgLpx6ny1le34Jz6UVo,221 + __pycache__/black.cpython-38.pyc,, + __pycache__/blackd.cpython-38.pyc,, + black-19.10b0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 + black-19.10b0.dist-info/LICENSE,sha256=nAQo8MO0d5hQz1vZbhGqqK_HLUqG1KNiI9erouWNbgA,1080 + black-19.10b0.dist-info/METADATA,sha256=UN40nGoVVTSpvLrTBwNsXgZdZIwoKFSrrDDHP6B7-A0,58841 + black-19.10b0.dist-info/RECORD,, + black.py,sha256=45IF72OgNfF8WpeNHnxV2QGfbCLubV5Xjl55cI65kYs,140161 + blackd.py,sha256=JCxaK4hLkMRwVfZMj8FRpRRYC0172-juKqbN22bISLE,6672 + blib2to3/__init__.py,sha256=9_8wL9Scv8_Cs8HJyJHGvx1vwXErsuvlsAqNZLcJQR0,8 + blib2to3/__pycache__/__init__.cpython-38.pyc,, + blib2to3/__pycache__/pygram.cpython-38.pyc,sha256=zpXgX4FHDuoeIQKO_v0sRsB-RzQFsuoKoBYvraAdoJw,1512 + blib2to3/__pycache__/pytree.cpython-38.pyc,sha256=LYLplXtG578ZjaFeoVuoX8rmxHn-BMAamCOsJMU1b9I,24910 + blib2to3/pygram.py,sha256=mXpQPqHcamFwch0RkyJsb92Wd0kUP3TW7d-u9dWhCGY,2085 + blib2to3/pytree.py,sha256=RWj3IL4U-Ljhkn4laN0C3p7IRdfvT3aIRjTV-x9hK1c,28530 + +If the ‘RECORD’ file is missing, tools that rely on ‘.dist-info’ must +not atempt to uninstall or upgrade the package. (This does not apply to +tools that rely on other sources of information, such as system package +managers in Linux distros.) + + +File: pythonpackagingguide.info, Node: The INSTALLER file, Prev: The RECORD file, Up: Recording installed projects + +5.1.8.5 The INSTALLER file +.......................... + +If present, ‘INSTALLER’ is a single-line text file naming the tool used +to install the project. If the installer is executable from the command +line, ‘INSTALLER’ should contain the command name. Otherwise, it should +contain a printable ASCII string. + +The file can be terminated by zero or more ASCII whitespace characters. + +Here are examples of two possible ‘INSTALLER’ files: + + pip + + MegaCorp Cloud Install-O-Matic + +This value should be used for informational purposes only. For example, +if a tool is asked to uninstall a project but finds no ‘RECORD’ file, it +may suggest that the tool named in ‘INSTALLER’ may be able to do the +uninstallation. + + +File: pythonpackagingguide.info, Node: Entry points specification, Prev: Recording installed projects, Up: Package Distribution Metadata + +5.1.9 Entry points specification +-------------------------------- + +`Entry points' are a mechanism for an installed distribution to +advertise components it provides to be discovered and used by other +code. For example: + + - Distributions can specify ‘console_scripts’ entry points, each + referring to a function. When `pip' (or another console_scripts + aware installer) installs the distribution, it will create a + command-line wrapper for each entry point. + + - Applications can use entry points to load plugins; e.g. Pygments + (a syntax highlighting tool) can use additional lexers and styles + from separately installed packages. For more about this, see *note + Creating and discovering plugins: 138. + +The entry point file format was originally developed to allow packages +built with setuptools to provide integration point metadata that would +be read at runtime with ‘pkg_resources’. It is now defined as a PyPA +interoperability specification in order to allow build tools other than +setuptools to publish ‘pkg_resources’ compatible entry point metadata, +and runtime libraries other than ‘pkg_resources’ to portably read +published entry point metadata (potentially with different caching and +conflict resolution strategies). + +* Menu: + +* Data model:: +* File format:: +* Use for scripts:: + + +File: pythonpackagingguide.info, Node: Data model, Next: File format, Up: Entry points specification + +5.1.9.1 Data model +.................. + +Conceptually, an entry point is defined by three required properties: + + - The `group' that an entry point belongs to indicates what sort of + object it provides. For instance, the group ‘console_scripts’ is + for entry points referring to functions which can be used as a + command, while ‘pygments.styles’ is the group for classes defining + pygments styles. The consumer typically defines the expected + interface. To avoid clashes, consumers defining a new group should + use names starting with a PyPI name owned by the consumer project, + followed by ‘.’. Group names must be one or more groups of + letters, numbers and underscores, separated by dots (regex + ‘^\w+(\.\w+)*$’). + + - The `name' identifies this entry point within its group. The + precise meaning of this is up to the consumer. For console + scripts, the name of the entry point is the command that will be + used to launch it. Within a distribution, entry point names should + be unique. If different distributions provide the same name, the + consumer decides how to handle such conflicts. The name may + contain any characters except ‘=’, but it cannot start or end with + any whitespace character, or start with ‘[’. For new entry points, + it is recommended to use only letters, numbers, underscores, dots + and dashes (regex ‘[\w.-]+’). + + - The `object reference' points to a Python object. It is either in + the form ‘importable.module’, or ‘importable.module:object.attr’. + Each of the parts delimited by dots and the colon is a valid Python + identifier. It is intended to be looked up like this: + + import importlib + modname, qualname_separator, qualname = object_ref.partition(':') + obj = importlib.import_module(modname) + if qualname_separator: + for attr in qualname.split('.'): + obj = getattr(obj, attr) + + Note: Some tools call this kind of object reference by itself an + ‘entry point’, for want of a better term, especially where it + points to a function to launch a program. + +There is also an optional property: the `extras' are a set of strings +identifying optional features of the distribution providing the entry +point. If these are specified, the entry point requires the +dependencies of those ‘extras’. See the metadata field *note +Provides-Extra (multiple use): 1c8. + +Using extras for an entry point is no longer recommended. Consumers +should support parsing them from existing distributions, but may then +ignore them. New publishing tools need not support specifying extras. +The functionality of handling extras was tied to setuptools’ model of +managing ‘egg’ packages, but newer tools such as pip and virtualenv use +a different model. + + +File: pythonpackagingguide.info, Node: File format, Next: Use for scripts, Prev: Data model, Up: Entry points specification + +5.1.9.2 File format +................... + +Entry points are defined in a file called ‘entry_points.txt’ in the +‘*.dist-info’ directory of the distribution. This is the directory +described in PEP 376(1) for installed distributions, and in PEP 427(2) +for wheels. The file uses the UTF-8 character encoding. + +The file contents are in INI format, as read by Python’s configparser(3) +module. However, configparser treats names as case-insensitive by +default, whereas entry point names are case sensitive. A case-sensitive +config parser can be made like this: + + import configparser + + class CaseSensitiveConfigParser(configparser.ConfigParser): + optionxform = staticmethod(str) + +The entry points file must always use ‘=’ to delimit names from values +(whereas configparser also allows using ‘:’). + +The sections of the config file represent entry point groups, the names +are names, and the values encode both the object reference and the +optional extras. If extras are used, they are a comma-separated list +inside square brackets. + +Within a value, readers must accept and ignore spaces (including +multiple consecutive spaces) before or after the colon, between the +object reference and the left square bracket, between the extra names +and the square brackets and colons delimiting them, and after the right +square bracket. The syntax for extras is formally specified as part of +PEP 508(4) (as ‘extras’). For tools writing the file, it is recommended +only to insert a space between the object reference and the left square +bracket. + +For example: + + [console_scripts] + foo = foomod:main + # One which depends on extras: + foobar = foomod:main_bar [bar,baz] + + # pytest plugins refer to a module, so there is no ':obj' + [pytest11] + nbval = nbval.plugin + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0376 + + (2) https://www.python.org/dev/peps/pep-0427 + + (3) +https://docs.python.org/3/library/configparser.html#module-configparser + + (4) https://www.python.org/dev/peps/pep-0508 + + +File: pythonpackagingguide.info, Node: Use for scripts, Prev: File format, Up: Entry points specification + +5.1.9.3 Use for scripts +....................... + +Two groups of entry points have special significance in packaging: +‘console_scripts’ and ‘gui_scripts’. In both groups, the name of the +entry point should be usable as a command in a system shell after the +package is installed. The object reference points to a function which +will be called with no arguments when this command is run. The function +may return an integer to be used as a process exit code, and returning +‘None’ is equivalent to returning ‘0’. + +For instance, the entry point ‘mycmd = mymod:main’ would create a +command ‘mycmd’ launching a script like this: + + import sys + from mymod import main + sys.exit(main()) + +The difference between ‘console_scripts’ and ‘gui_scripts’ only affects +Windows systems. ‘console_scripts’ are wrapped in a console executable, +so they are attached to a console and can use ‘sys.stdin’, ‘sys.stdout’ +and ‘sys.stderr’ for input and output. ‘gui_scripts’ are wrapped in a +GUI executable, so they can be started without a console, but cannot use +standard streams unless application code redirects them. Other +platforms do not have the same distinction. + +Install tools are expected to set up wrappers for both ‘console_scripts’ +and ‘gui_scripts’ in the scripts directory of the install scheme. They +are not responsible for putting this directory in the ‘PATH’ environment +variable which defines where command-line tools are found. + +As files are created from the names, and some filesystems are +case-insensitive, packages should avoid using names in these groups +which differ only in case. The behaviour of install tools when names +differ only in case is undefined. + + +File: pythonpackagingguide.info, Node: Package Index Interfaces, Prev: Package Distribution Metadata, Up: PyPA specifications + +5.2 Package Index Interfaces +============================ + +* Menu: + +* The .pypirc file: The pypirc file. +* Simple repository API:: + + +File: pythonpackagingguide.info, Node: The pypirc file, Next: Simple repository API, Up: Package Index Interfaces + +5.2.1 The ‘.pypirc’ file +------------------------ + +A ‘.pypirc’ file allows you to define the configuration for *note +package indexes: ec. (referred to here as “repositories”), so that you +don’t have to enter the URL, username, or password whenever you upload a +package with *note twine: 66. or *note flit: 6a. + +The format (originally defined by the *note distutils: c0. package) is: + + [distutils] + index-servers = + first-repository + second-repository + + [first-repository] + repository = + username = + password = + + [second-repository] + repository = + username = + password = + +The ‘distutils’ section defines an ‘index-servers’ field that lists the +name of all sections describing a repository. + +Each section describing a repository defines three fields: + + - ‘repository’: The URL of the repository. + + - ‘username’: The registered username on the repository. + + - ‘password’: The password that will used to authenticate the + username. + + Warning: Be aware that this stores your password in plain text. + For better security, consider an alternative like keyring(1), + setting environment variables, or providing the password on the + command line. + + Otherwise, set the permissions on ‘.pypirc’ so that only you can + view or modify it. For example, on Linux or macOS, run: + + chmod 600 ~/.pypirc + +* Menu: + +* Common configurations:: + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/keyring/ + + +File: pythonpackagingguide.info, Node: Common configurations, Up: The pypirc file + +5.2.1.1 Common configurations +............................. + + Note: These examples apply to *note twine: 66, and projects like + *note hatch: 20c. that use it under the hood. Other projects (e.g. + *note flit: 6a.) also use ‘.pypirc’, but with different defaults. + Please refer to each project’s documentation for more details and + usage instructions. + +Twine’s default configuration mimics a ‘.pypirc’ with repository +sections for PyPI and TestPyPI: + + [distutils] + index-servers = + pypi + testpypi + + [pypi] + repository = https://upload.pypi.org/legacy/ + + [testpypi] + repository = https://test.pypi.org/legacy/ + +Twine will add additional configuration from ‘$HOME/.pypirc’, the +command line, and environment variables to this default configuration. + +* Menu: + +* Using a PyPI token:: +* Using another package index:: + + +File: pythonpackagingguide.info, Node: Using a PyPI token, Next: Using another package index, Up: Common configurations + +5.2.1.2 Using a PyPI token +.......................... + +To set your API token(1) for PyPI, you can create a ‘$HOME/.pypirc’ +similar to: + + [pypi] + username = __token__ + password = + +For *note TestPyPI: f6, add a ‘[testpypi]’ section, using the API token +from your TestPyPI account. + + ---------- Footnotes ---------- + + (1) https://pypi.org/help/#apitoken + + +File: pythonpackagingguide.info, Node: Using another package index, Prev: Using a PyPI token, Up: Common configurations + +5.2.1.3 Using another package index +................................... + +To configure an additional repository, you’ll need to redefine the +‘index-servers’ field to include the repository name. Here is a +complete example of a ‘$HOME/.pypirc’ for PyPI, TestPyPI, and a private +repository: + + [distutils] + index-servers = + pypi + testpypi + private-repository + + [pypi] + username = __token__ + password = + + [testpypi] + username = __token__ + password = + + [private-repository] + repository = + username = + password = + + Warning: Instead of using the ‘password’ field, consider saving + your API tokens and passwords securely using keyring(1) (which is + installed by Twine): + + keyring set https://upload.pypi.org/legacy/ __token__ + keyring set https://test.pypi.org/legacy/ __token__ + keyring set + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/keyring/ + + +File: pythonpackagingguide.info, Node: Simple repository API, Prev: The pypirc file, Up: Package Index Interfaces + +5.2.2 Simple repository API +--------------------------- + +The current interface for querying available package versions and +retrieving packages from an index server is defined in PEP 503(1), with +the addition of “yank” support (allowing a kind of file deletion) as +defined in PEP 592(2). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0503 + + (2) https://www.python.org/dev/peps/pep-0592 + + +File: pythonpackagingguide.info, Node: Project Summaries, Next: Glossary, Prev: PyPA specifications, Up: Top + +6 Project Summaries +******************* + +Summaries and links for the most relevant projects in the space of +Python installation and packaging. + +* Menu: + +* PyPA Projects:: +* Non-PyPA Projects:: +* Standard Library Projects:: + + +File: pythonpackagingguide.info, Node: PyPA Projects, Next: Non-PyPA Projects, Up: Project Summaries + +6.1 PyPA Projects +================= + +* Menu: + +* bandersnatch:: +* build:: +* distlib:: +* packaging:: +* pip:: +* Pipenv:: +* Pipfile:: +* Python Packaging User Guide: Python Packaging User Guide<2>. +* readme_renderer:: +* setuptools:: +* trove-classifiers:: +* twine:: +* virtualenv: virtualenv<2>. +* Warehouse:: +* wheel:: + + +File: pythonpackagingguide.info, Node: bandersnatch, Next: build, Up: PyPA Projects + +6.1.1 bandersnatch +------------------ + +Mailing list(1) (2) | Issues(3) | GitHub(4) | PyPI(5) | Dev +IRC:#bandersnatch(6) + +‘bandersnatch’ is a PyPI mirroring client designed to efficiently create +a complete mirror of the contents of PyPI. Organizations thus save +bandwidth and latency on package downloads (especially in the context of +automated tests) and to prevent heavily loading PyPI’s Content Delivery +Network (CDN). + + ---------- Footnotes ---------- + + (1) http://mail.python.org/mailman/listinfo/distutils-sig + + (2) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (3) https://github.com/pypa/bandersnatch/issues + + (4) https://github.com/pypa/bandersnatch + + (5) https://pypi.org/project/bandersnatch + + (6) https://webchat.freenode.net/?channels=%23bandersnatch + + +File: pythonpackagingguide.info, Node: build, Next: distlib, Prev: bandersnatch, Up: PyPA Projects + +6.1.2 build +----------- + +Docs(1) | Issues(2) | GitHub(3) | PyPI(4) | User IRC:#pypa(5) | Dev +IRC:#pypa-dev(6) + +‘build’ is a PEP-517 compatible Python package builder. It provides a +CLI to build packages, as well as a Python API. + + ---------- Footnotes ---------- + + (1) https://pypa-build.readthedocs.io/ + + (2) https://github.com/pypa/build/issues + + (3) https://github.com/pypa/build + + (4) https://pypi.org/project/build + + (5) https://webchat.freenode.net/?channels=%23pypa + + (6) https://webchat.freenode.net/?channels=%23pypa-dev + + +File: pythonpackagingguide.info, Node: distlib, Next: packaging, Prev: build, Up: PyPA Projects + +6.1.3 distlib +------------- + +Docs(1) | Mailing list(2) (3) | Issues(4) | Bitbucket(5) | PyPI(6) + +‘distlib’ is a library which implements low-level functions that relate +to packaging and distribution of Python software. ‘distlib’ implements +several relevant PEPs (Python Enhancement Proposal standards) and is +useful for developers of third-party packaging tools to make and upload +binary and source *note distributions: b, achieve interoperability, +resolve dependencies, manage package resources, and do other similar +functions. + +Unlike the stricter *note packaging: 21e. project (below), which +specifically implements modern Python packaging interoperability +standards, ‘distlib’ also attempts to provide reasonable fallback +behaviours when asked to handle legacy packages and metadata that +predate the modern interoperability standards and fall into the subset +of packages that are incompatible with those standards. + + ---------- Footnotes ---------- + + (1) http://pythonhosted.org/distlib/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) https://bitbucket.org/pypa/distlib/issues?status=new&status=open + + (5) https://bitbucket.org/pypa/distlib + + (6) https://pypi.org/project/distlib + + +File: pythonpackagingguide.info, Node: packaging, Next: pip, Prev: distlib, Up: PyPA Projects + +6.1.4 packaging +--------------- + +Docs(1) | Dev list(2) | Issues(3) | GitHub(4) | PyPI(5) | User +IRC:#pypa(6) | Dev IRC:#pypa-dev(7) + +Core utilities for Python packaging used by *note pip: 2b. and *note +setuptools: 2d. + +The core utilities in the packaging library handle version handling, +specifiers, markers, requirements, tags, and similar attributes and +tasks for Python packages. Most Python users rely on this library +without needing to explicitly call it; developers of the other Python +packaging, distribution, and installation tools listed here often use +its functionality to parse, discover, and otherwise handle dependency +attributes. + +This project specifically focuses on implementing the modern Python +packaging interoperability standards defined at *note PyPA +specifications: 196, and will report errors for sufficiently old legacy +packages that are incompatible with those standards. In contrast, the +*note distlib: 21c. project is a more permissive library that attempts +to provide a plausible reading of ambiguous metadata in cases where +*note packaging: 21e. will instead report on error. + + ---------- Footnotes ---------- + + (1) https://packaging.pypa.io + + (2) https://mail.python.org/mailman3/lists/distutils-sig.python.org/ + + (3) https://github.com/pypa/packaging/issues + + (4) https://github.com/pypa/packaging + + (5) https://pypi.org/project/packaging + + (6) https://webchat.freenode.net/?channels=%23pypa + + (7) https://webchat.freenode.net/?channels=%23pypa-dev + + +File: pythonpackagingguide.info, Node: pip, Next: Pipenv, Prev: packaging, Up: PyPA Projects + +6.1.5 pip +--------- + +Docs(1) | User list(2) (3) | Dev list(4) | Issues(5) | GitHub(6) | +PyPI(7) | User IRC:#pypa(8) | Dev IRC:#pypa-dev(9) + +The most popular tool for installing Python packages, and the one +included with modern versions of Python. + +It provides the essential core features for finding, downloading, and +installing packages from PyPI and other Python package indexes, and can +be incorporated into a wide range of development workflows via its +command-line interface (CLI). + + ---------- Footnotes ---------- + + (1) https://pip.pypa.io/en/stable/ + + (2) http://groups.google.com/group/python-virtualenv + + (3) (1) pip was created by the same developer as virtualenv, and +early on adopted the virtualenv mailing list, and it’s stuck ever since. + + (4) https://mail.python.org/mailman3/lists/distutils-sig.python.org/ + + (5) https://github.com/pypa/pip/issues + + (6) https://github.com/pypa/pip + + (7) https://pypi.org/project/pip/ + + (8) https://webchat.freenode.net/?channels=%23pypa + + (9) https://webchat.freenode.net/?channels=%23pypa-dev + + +File: pythonpackagingguide.info, Node: Pipenv, Next: Pipfile, Prev: pip, Up: PyPA Projects + +6.1.6 Pipenv +------------ + +Docs(1) | Source(2) | Issues(3) | PyPI(4) + +Pipenv is a project that aims to bring the best of all packaging worlds +to the Python world. It harnesses *note Pipfile: 51, *note pip: 2b, and +*note virtualenv: 32. into one single toolchain. It features very +pretty terminal colors. + +Pipenv aims to help users manage environments, dependencies, and +imported packages on the command line. It also works well on Windows +(which other tools often underserve), makes and checkes file hashes, to +ensure compliance with hash-locked dependency specifiers, and eases +uninstallation of packages and dependencies. It is used by Python users +and system administrators, but has been less maintained since late 2018. + + ---------- Footnotes ---------- + + (1) https://pipenv.pypa.io/ + + (2) https://github.com/pypa/pipenv + + (3) https://github.com/pypa/pipenv/issues + + (4) https://pypi.org/project/pipenv + + +File: pythonpackagingguide.info, Node: Pipfile, Next: Python Packaging User Guide<2>, Prev: Pipenv, Up: PyPA Projects + +6.1.7 Pipfile +------------- + +Source(1) + +‘Pipfile’ and its sister ‘Pipfile.lock’ are a higher-level +application-centric alternative to *note pip: 2b.’s lower-level +‘requirements.txt’ file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/pipfile + + +File: pythonpackagingguide.info, Node: Python Packaging User Guide<2>, Next: readme_renderer, Prev: Pipfile, Up: PyPA Projects + +6.1.8 Python Packaging User Guide +--------------------------------- + +Docs(1) | Mailing list(2) | Issues(3) | GitHub(4) | User IRC:#pypa(5) | +Dev IRC:#pypa-dev(6) + +This guide! + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/en/latest/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) https://github.com/pypa/python-packaging-user-guide/issues + + (4) https://github.com/pypa/python-packaging-user-guide + + (5) https://webchat.freenode.net/?channels=%23pypa + + (6) https://webchat.freenode.net/?channels=%23pypa-dev + + +File: pythonpackagingguide.info, Node: readme_renderer, Next: setuptools, Prev: Python Packaging User Guide<2>, Up: PyPA Projects + +6.1.9 readme_renderer +--------------------- + +GitHub and docs(1) | PyPI(2) + +‘readme_renderer’ is a library that package developers use to render +their user documentation (README) files into HTML from markup languages +such as Markdown or reStructuredText. Developers call it on its own or +via *note twine: 66, as part of their release management process, to +check that their package descriptions will properly display on PyPI. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/readme_renderer/ + + (2) https://pypi.org/project/readme_renderer/ + + +File: pythonpackagingguide.info, Node: setuptools, Next: trove-classifiers, Prev: readme_renderer, Up: PyPA Projects + +6.1.10 setuptools +----------------- + +Docs(1) | User list(2) (3) | Dev list(4) | Issues(5) | GitHub(6) | +PyPI(7) | User IRC:#pypa(8) | Dev IRC:#pypa-dev(9) + +setuptools (which includes ‘easy_install’) is a collection of +enhancements to the Python distutils that allow you to more easily build +and distribute Python *note distributions: b, especially ones that have +dependencies on other packages. + +distribute(10) was a fork of setuptools that was merged back into +setuptools (in v0.7), thereby making setuptools the primary choice for +Python packaging. + + ---------- Footnotes ---------- + + (1) https://setuptools.readthedocs.io/en/latest/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) https://mail.python.org/mailman3/lists/distutils-sig.python.org/ + + (5) https://github.com/pypa/setuptools/issues + + (6) https://github.com/pypa/setuptools + + (7) https://pypi.org/project/setuptools + + (8) https://webchat.freenode.net/?channels=%23pypa + + (9) https://webchat.freenode.net/?channels=%23pypa-dev + + (10) https://pypi.org/project/distribute + + +File: pythonpackagingguide.info, Node: trove-classifiers, Next: twine, Prev: setuptools, Up: PyPA Projects + +6.1.11 trove-classifiers +------------------------ + +Issues(1) | GitHub(2) | PyPI(3) + +trove-classifiers is the canonical source for classifiers on PyPI(4), +which project maintainers use to systematically describe their +projects(5) so that users can better find projects that match their +needs on the PyPI. + +The trove-classifiers package contains a list of valid classifiers and +deprecated classifiers (which are paired with the classifiers that +replace them). Use this package to validate classifiers used in +packages intended for uploading to PyPI. As this list of classifiers is +published as code, you can install and import it, giving you a more +convenient workflow compared to referring to the list published on +PyPI(6). The issue tracker(7) for the project hosts discussions on +proposed classifiers and requests for new classifiers. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/trove-classifiers/issues + + (2) https://github.com/pypa/trove-classifiers + + (3) https://pypi.org/project/trove-classifiers/ + + (4) https://pypi.org/classifiers/ + + (5) +https://packaging.python.org/specifications/core-metadata/#classifier-multiple-use + + (6) https://pypi.org/classifiers/ + + (7) https://github.com/pypa/trove-classifiers/issues + + +File: pythonpackagingguide.info, Node: twine, Next: virtualenv<2>, Prev: trove-classifiers, Up: PyPA Projects + +6.1.12 twine +------------ + +Docs(1) | Mailing list(2) (3) | Issues(4) | GitHub(5) | PyPI(6) + +Twine is the primary tool developers use to upload packages to the +Python Package Index or other Python package indexes. It is a +command-line program that passes program files and metadata to a web +API. Developers use it because it’s the official PyPI upload tool, it’s +fast and secure, it’s maintained, and it reliably works. + + ---------- Footnotes ---------- + + (1) https://twine.readthedocs.io/en/latest/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) https://github.com/pypa/twine/issues + + (5) https://github.com/pypa/twine + + (6) https://pypi.org/project/twine + + +File: pythonpackagingguide.info, Node: virtualenv<2>, Next: Warehouse, Prev: twine, Up: PyPA Projects + +6.1.13 virtualenv +----------------- + +Docs(1) | User list(2) | Dev list(3) | Issues(4) | GitHub(5) | PyPI(6) | +User IRC:#pypa(7) | Dev IRC:#pypa-dev(8) + +virtualenv is a tool which uses the command-line path environment +variable to create isolated Python *note Virtual Environments: 22b, much +as *note venv: 7f. does. virtualenv provides additional functionality, +compared to *note venv: 7f, by supporting Python 2.7 and by providing +convenient features for configuring, maintaining, duplicating, and +troubleshooting the virtual environments. For more information, see the +section on *note Creating Virtual Environments: 31. + + ---------- Footnotes ---------- + + (1) https://virtualenv.pypa.io/en/stable/ + + (2) http://groups.google.com/group/python-virtualenv + + (3) https://mail.python.org/mailman3/lists/distutils-sig.python.org/ + + (4) https://github.com/pypa/virtualenv/issues + + (5) https://github.com/pypa/virtualenv + + (6) https://pypi.org/project/virtualenv/ + + (7) https://webchat.freenode.net/?channels=%23pypa + + (8) https://webchat.freenode.net/?channels=%23pypa-dev + + +File: pythonpackagingguide.info, Node: Warehouse, Next: wheel, Prev: virtualenv<2>, Up: PyPA Projects + +6.1.14 Warehouse +---------------- + +Docs(1) | Mailing list(2) (3) | Issues(4) | GitHub(5) | Dev +IRC:#pypa-dev(6) + +The current codebase powering the *note Python Package Index (PyPI): 39. +It is hosted at pypi.org(7). The default source for *note pip: 2b. +downloads. + + ---------- Footnotes ---------- + + (1) https://warehouse.pypa.io/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) https://github.com/pypa/warehouse/issues + + (5) https://github.com/pypa/warehouse + + (6) https://webchat.freenode.net/?channels=%23pypa-dev + + (7) https://pypi.org/ + + +File: pythonpackagingguide.info, Node: wheel, Prev: Warehouse, Up: PyPA Projects + +6.1.15 wheel +------------ + +Docs(1) | Mailing list(2) (3) | Issues(4) | GitHub(5) | PyPI(6) | User +IRC:#pypa(7) | Dev IRC:#pypa-dev(8) + +Primarily, the wheel project offers the ‘bdist_wheel’ *note setuptools: +2d. extension for creating *note wheel distributions: d. Additionally, +it offers its own command line utility for creating and installing +wheels. + +See also auditwheel(9), a tool that package developers use to check and +fix Python packages they are making in the binary wheel format. It +provides functionality to discover dependencies, check metadata for +compliance, and repair the wheel and metadata to properly link and +include external shared libraries in a package. + + ---------- Footnotes ---------- + + (1) https://wheel.readthedocs.io/en/latest/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) https://github.com/pypa/wheel/issues + + (5) https://github.com/pypa/wheel + + (6) https://pypi.org/project/wheel + + (7) https://webchat.freenode.net/?channels=%23pypa + + (8) https://webchat.freenode.net/?channels=%23pypa-dev + + (9) https://github.com/pypa/auditwheel + + +File: pythonpackagingguide.info, Node: Non-PyPA Projects, Next: Standard Library Projects, Prev: PyPA Projects, Up: Project Summaries + +6.2 Non-PyPA Projects +===================== + +* Menu: + +* bento:: +* buildout:: +* conda:: +* devpi:: +* flit:: +* enscons:: +* Hashdist:: +* hatch:: +* pex:: +* pipx:: +* pip-tools:: +* piwheels:: +* poetry:: +* pypiserver:: +* scikit-build:: +* shiv:: +* Spack: Spack<2>. +* zest.releaser: zest releaser. + + +File: pythonpackagingguide.info, Node: bento, Next: buildout, Up: Non-PyPA Projects + +6.2.1 bento +----------- + +Docs(1) | Mailing list(2) | Issues(3) | GitHub(4) | PyPI(5) + +Bento is a packaging tool solution for Python software, targeted as an +alternative to *note distutils: c0, *note setuptools: 2d, etc…. Bento’s +philosophy is reproducibility, extensibility and simplicity (in that +order). + + ---------- Footnotes ---------- + + (1) http://cournape.github.io/Bento/ + + (2) http://librelist.com/browser/bento + + (3) https://github.com/cournape/Bento/issues + + (4) https://github.com/cournape/Bento + + (5) https://pypi.org/project/bento + + +File: pythonpackagingguide.info, Node: buildout, Next: conda, Prev: bento, Up: Non-PyPA Projects + +6.2.2 buildout +-------------- + +Docs(1) | Mailing list(2) (3) | Issues(4) | PyPI(5) | GitHub(6) | +IRC:#buildout(7) + +Buildout is a Python-based build system for creating, assembling and +deploying applications from multiple parts, some of which may be +non-Python-based. It lets you create a buildout configuration and +reproduce the same software later. + + ---------- Footnotes ---------- + + (1) http://www.buildout.org/en/latest/ + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) https://bugs.launchpad.net/zc.buildout + + (5) https://pypi.org/project/zc.buildout + + (6) https://github.com/buildout/buildout/ + + (7) https://webchat.freenode.net/?channels=%23buildout + + +File: pythonpackagingguide.info, Node: conda, Next: devpi, Prev: buildout, Up: Non-PyPA Projects + +6.2.3 conda +----------- + +Docs(1) + +conda is the package management tool for Anaconda(2) Python +installations. Anaconda Python is a distribution from Anaconda, Inc(3) +specifically aimed at the scientific community, and in particular on +Windows where the installation of binary extensions is often difficult. + +Conda is a completely separate tool from *note pip: 2b, virtualenv and +wheel, but provides many of their combined features in terms of package +management, virtual environment management and deployment of binary +extensions. + +Conda does not install packages from PyPI and can install only from the +official Anaconda repositories, or anaconda.org (a place for +user-contributed `conda' packages), or a local (e.g. intranet) package +server. However, note that *note pip: 2b. can be installed into, and +work side-by-side with conda for managing *note distributions: b. from +PyPI. Also, conda skeleton(4) is a tool to make Python packages +installable by conda by first fetching them from PyPI and modifying +their metadata. + + ---------- Footnotes ---------- + + (1) http://conda.pydata.org/docs/ + + (2) https://docs.anaconda.com/anaconda/ + + (3) https://www.anaconda.com/download + + (4) +https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs-skeleton.html + + +File: pythonpackagingguide.info, Node: devpi, Next: flit, Prev: conda, Up: Non-PyPA Projects + +6.2.4 devpi +----------- + +Docs(1) | Mailing List(2) | Issues(3) | PyPI(4) + +devpi features a powerful PyPI-compatible server and PyPI proxy cache +with a complementary command line tool to drive packaging, testing and +release activities with Python. devpi also provides a browsable and +searchable web interface. + + ---------- Footnotes ---------- + + (1) http://doc.devpi.net/latest/ + + (2) https://groups.google.com/forum/#!forum/devpi-dev + + (3) https://bitbucket.org/hpk42/devpi/issues + + (4) https://pypi.org/project/devpi + + +File: pythonpackagingguide.info, Node: flit, Next: enscons, Prev: devpi, Up: Non-PyPA Projects + +6.2.5 flit +---------- + +Docs(1) | Issues(2) | PyPI(3) + +Flit provides a simple way to upload pure Python packages and modules to +PyPI. It focuses on making the easy things easy(4) for packaging. Flit +can generate a configuration file to quickly set up a simple project, +build source distributions and wheels, and upload them to PyPI. + +Flit uses ‘pyproject.toml’ to configure a project. Flit does not rely +on tools such as *note setuptools: 2d. to build distributions, or *note +twine: 66. to upload them to PyPI. Flit requires Python 3, but you can +use it to distribute modules for Python 2, so long as they can be +imported on Python 3. + + ---------- Footnotes ---------- + + (1) https://flit.readthedocs.io/en/latest/ + + (2) https://github.com/takluyver/flit/issues + + (3) https://pypi.org/project/flit + + (4) https://flit.readthedocs.io/en/latest/rationale.html + + +File: pythonpackagingguide.info, Node: enscons, Next: Hashdist, Prev: flit, Up: Non-PyPA Projects + +6.2.6 enscons +------------- + +Source(1) | Issues(2) | PyPI(3) + +Enscons is a Python packaging tool based on SCons(4). It builds *note +pip: 2b.-compatible source distributions and wheels without using +distutils or setuptools, including distributions with C extensions. +Enscons has a different architecture and philosophy than *note +distutils: c0. Rather than adding build features to a Python packaging +system, enscons adds Python packaging to a general purpose build system. +Enscons helps you to build sdists that can be automatically built by +*note pip: 2b, and wheels that are independent of enscons. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/dholth/enscons/src + + (2) https://bitbucket.org/dholth/enscons/issues + + (3) https://pypi.org/project/enscons + + (4) http://scons.org/ + + +File: pythonpackagingguide.info, Node: Hashdist, Next: hatch, Prev: enscons, Up: Non-PyPA Projects + +6.2.7 Hashdist +-------------- + +Docs(1) | GitHub(2) + +Hashdist is a library for building non-root software distributions. +Hashdist is trying to be “the Debian of choice for cases where Debian +technology doesn’t work”. The best way for Pythonistas to think about +Hashdist may be a more powerful hybrid of *note virtualenv: 32. and +*note buildout: 80. It is aimed at solving the problem of installing +scientific software, and making package distribution stateless, cached, +and branchable. It is used by some researchers but has been lacking in +maintenance since 2016. + + ---------- Footnotes ---------- + + (1) https://hashdist.readthedocs.io/en/latest/ + + (2) https://github.com/hashdist/hashdist/ + + +File: pythonpackagingguide.info, Node: hatch, Next: pex, Prev: Hashdist, Up: Non-PyPA Projects + +6.2.8 hatch +----------- + +GitHub and Docs(1) | PyPI(2) + +Hatch is a unified command-line tool meant to conveniently manage +dependencies and environment isolation for Python developers. Python +package developers use Hatch to configure, version, specify dependencies +for, and publish packages to PyPI. Under the hood, it uses *note twine: +66. to upload packages to PyPI, and *note pip: 2b. to download and +install packages. + + ---------- Footnotes ---------- + + (1) https://github.com/ofek/hatch + + (2) https://pypi.org/project/hatch + + +File: pythonpackagingguide.info, Node: pex, Next: pipx, Prev: hatch, Up: Non-PyPA Projects + +6.2.9 pex +--------- + +Docs(1) | GitHub(2) | PyPI(3) + +pex is both a library and tool for generating ‘.pex’ (Python EXecutable) +files, standalone Python environments in the spirit of *note virtualenv: +32. ‘.pex’ files are just carefully constructed zip files with a +‘#!/usr/bin/env python’ and special ‘__main__.py’, and are designed to +make deployment of Python applications as simple as ‘cp’. + + ---------- Footnotes ---------- + + (1) https://pex.readthedocs.io/en/latest/ + + (2) https://github.com/pantsbuild/pex/ + + (3) https://pypi.org/project/pex + + +File: pythonpackagingguide.info, Node: pipx, Next: pip-tools, Prev: pex, Up: Non-PyPA Projects + +6.2.10 pipx +----------- + +Docs(1) | GitHub(2) | PyPI(3) + +pipx is a tool to safely install and run Python CLI applications +globally. + + ---------- Footnotes ---------- + + (1) https://pipxproject.github.io/pipx/ + + (2) https://github.com/pipxproject/pipx + + (3) https://pypi.org/project/pipx/ + + +File: pythonpackagingguide.info, Node: pip-tools, Next: piwheels, Prev: pipx, Up: Non-PyPA Projects + +6.2.11 pip-tools +---------------- + +GitHub and Docs(1) | PyPI(2) + +pip-tools is a suite of tools meant for Python system administrators and +release managers who particularly want to keep their builds +deterministic yet stay up to date with new versions of their +dependencies. Users can specify particular release of their +dependencies via hash, conveniently make a properly formatted list of +requirements from information in other parts of their program, update +all dependencies (a feature *note pip: 2b. currently does not provide), +and create layers of constraints for the program to obey. + + ---------- Footnotes ---------- + + (1) https://github.com/jazzband/pip-tools/ + + (2) https://pypi.org/project/pip-tools/ + + +File: pythonpackagingguide.info, Node: piwheels, Next: poetry, Prev: pip-tools, Up: Non-PyPA Projects + +6.2.12 piwheels +--------------- + +Website(1) | Docs(2) | GitHub(3) + +piwheels is a website, and software underpinning it, that fetches source +code distribution packages from PyPI and compiles them into binary +wheels that are optimized for installation onto Raspberry Pi computers. +Raspberry Pi OS pre-configures pip to use piwheels.org as an additional +index to PyPI. + + ---------- Footnotes ---------- + + (1) https://www.piwheels.org/ + + (2) https://piwheels.readthedocs.io/ + + (3) https://github.com/piwheels/piwheels/ + + +File: pythonpackagingguide.info, Node: poetry, Next: pypiserver, Prev: piwheels, Up: Non-PyPA Projects + +6.2.13 poetry +------------- + +Docs(1) | GitHub(2) | PyPI(3) + +poetry is a command-line tool to handle dependency installation and +isolation as well as building and packaging of Python packages. It uses +‘pyproject.toml’ and, instead of depending on the resolver functionality +within *note pip: 2b, provides its own dependency resolver. It attempts +to speed users’ experience of installation and dependency resolution by +locally caching metadata about dependencies. + + ---------- Footnotes ---------- + + (1) https://python-poetry.org/ + + (2) https://github.com/python-poetry/poetry + + (3) https://pypi.org/project/poetry/ + + +File: pythonpackagingguide.info, Node: pypiserver, Next: scikit-build, Prev: poetry, Up: Non-PyPA Projects + +6.2.14 pypiserver +----------------- + +Docs(1) | GitHub(2) | PyPI(3) + +pypiserver is a minimalist application that serves as a private Python +package index within organizations, implementing a simple API and +browser interface. You can upload private packages using standard +upload tools, and users can download and install them with *note pip: +2b, without publishing them publicly. Organizations who use pypiserver +usually download packages both from pypiserver and from PyPI. + + ---------- Footnotes ---------- + + (1) https://github.com/pypiserver/pypiserver/blob/master/README.rst + + (2) https://github.com/pypiserver/pypiserver + + (3) https://pypi.org/project/pypiserver/ + + +File: pythonpackagingguide.info, Node: scikit-build, Next: shiv, Prev: pypiserver, Up: Non-PyPA Projects + +6.2.15 scikit-build +------------------- + +Docs(1) | Mailing list(2) | GitHub(3) | PyPI(4) + +Scikit-build is an improved build system generator for CPython +C/C++/Fortran/Cython extensions that integrates with *note setuptools: +2d, *note wheel: 2e. and *note pip: 2b. It internally uses cmake(5) +(available on PyPI) to provide better support for additional compilers, +build systems, cross compilation, and locating dependencies and their +associated build requirements. To speed up and parallelize the build of +large projects, the user can install ninja(6) (also available on PyPI). + + ---------- Footnotes ---------- + + (1) https://scikit-build.readthedocs.io/en/latest/ + + (2) https://groups.google.com/forum/#!forum/scikit-build + + (3) https://github.com/scikit-build/scikit-build/ + + (4) https://pypi.org/project/scikit-build + + (5) https://pypi.org/project/cmake + + (6) https://pypi.org/project/ninja + + +File: pythonpackagingguide.info, Node: shiv, Next: Spack<2>, Prev: scikit-build, Up: Non-PyPA Projects + +6.2.16 shiv +----------- + +Docs(1) | GitHub(2) | PyPI(3) + +shiv is a command line utility for building fully self contained Python +zipapps as outlined in PEP 441(4), but with all their dependencies +included. Its primary goal is making distributing Python applications +and command line tools fast & easy. + + ---------- Footnotes ---------- + + (1) https://shiv.readthedocs.io/en/latest/ + + (2) https://github.com/linkedin/shiv + + (3) https://pypi.org/project/shiv/ + + (4) https://www.python.org/dev/peps/pep-0441 + + +File: pythonpackagingguide.info, Node: Spack<2>, Next: zest releaser, Prev: shiv, Up: Non-PyPA Projects + +6.2.17 Spack +------------ + +Docs(1) | GitHub(2) | Paper(3) | Slides(4) + +A flexible package manager designed to support multiple versions, +configurations, platforms, and compilers. Spack is like Homebrew, but +packages are written in Python and parameterized to allow easy swapping +of compilers, library versions, build options, etc. Arbitrarily many +versions of packages can coexist on the same system. Spack was designed +for rapidly building high performance scientific applications on +clusters and supercomputers. + +Spack is not in PyPI (yet), but it requires no installation and can be +used immediately after cloning from GitHub. + + ---------- Footnotes ---------- + + (1) https://spack.readthedocs.io/ + + (2) https://github.com/llnl/spack/ + + (3) +http://www.computer.org/csdl/proceedings/sc/2015/3723/00/2807623.pdf + + (4) https://tgamblin.github.io/files/Gamblin-Spack-SC15-Talk.pdf + + +File: pythonpackagingguide.info, Node: zest releaser, Prev: Spack<2>, Up: Non-PyPA Projects + +6.2.18 zest.releaser +-------------------- + +Docs(1) | GitHub(2) | PyPI(3) + +‘zest.releaser’ is a Python package release tool providing an +abstraction layer on top of *note twine: 66. Python developers use +‘zest.releaser’ to automate incrementing package version numbers, +updating changelogs, tagging releases in source control, and uploading +new packages to PyPI. + + ---------- Footnotes ---------- + + (1) https://zestreleaser.readthedocs.io/en/latest/ + + (2) https://github.com/zestsoftware/zest.releaser/ + + (3) https://pypi.org/project/zest.releaser/ + + +File: pythonpackagingguide.info, Node: Standard Library Projects, Prev: Non-PyPA Projects, Up: Project Summaries + +6.3 Standard Library Projects +============================= + +* Menu: + +* ensurepip:: +* distutils:: +* venv:: + + +File: pythonpackagingguide.info, Node: ensurepip, Next: distutils, Up: Standard Library Projects + +6.3.1 ensurepip +--------------- + +Docs(1) | Issues(2) + +A package in the Python Standard Library that provides support for +bootstrapping *note pip: 2b. into an existing Python installation or +virtual environment. In most cases, end users won’t use this module, +but rather it will be used during the build of the Python distribution. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/ensurepip.html + + (2) http://bugs.python.org + + +File: pythonpackagingguide.info, Node: distutils, Next: venv, Prev: ensurepip, Up: Standard Library Projects + +6.3.2 distutils +--------------- + +Docs(1) | User list(2) (3) | Issues(4) | User IRC:#pypa(5) | Dev +IRC:#pypa-dev(6) + +The original Python packaging system, added to the standard library in +Python 2.0. + +Due to the challenges of maintaining a packaging system where feature +updates are tightly coupled to language runtime updates, direct usage of +*note distutils: c0. is now actively discouraged, with *note setuptools: +2d. being the preferred replacement. *note setuptools: 2d. not only +provides features that plain *note distutils: c0. doesn’t offer (such as +dependency declarations and entry point declarations), it also provides +a consistent build interface and feature set across all supported Python +versions. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/distutils.html + + (2) http://mail.python.org/mailman/listinfo/distutils-sig + + (3) (2) Multiple projects reuse the distutils-sig mailing list as +their user list. + + (4) http://bugs.python.org + + (5) https://webchat.freenode.net/?channels=%23pypa + + (6) https://webchat.freenode.net/?channels=%23pypa-dev + + +File: pythonpackagingguide.info, Node: venv, Prev: distutils, Up: Standard Library Projects + +6.3.3 venv +---------- + +Docs(1) | Issues(2) + +A package in the Python Standard Library (starting with Python 3.3) for +creating *note Virtual Environments: 22b. For more information, see the +section on *note Creating Virtual Environments: 31. + +__________________________________________________________________ + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/venv.html + + (2) http://bugs.python.org + + +File: pythonpackagingguide.info, Node: Glossary, Next: How to Get Support, Prev: Project Summaries, Up: Top + +7 Glossary +********** + +Binary Distribution + + A specific kind of *note Built Distribution: 63. that contains + compiled extensions. + +Built Distribution + + A *note Distribution: b. format containing files and metadata that + only need to be moved to the correct location on the target system, + to be installed. *note Wheel: d. is such a format, whereas + distutil’s *note Source Distribution: 3d. is not, in that it + requires a build step before it can be installed. This format does + not imply that Python files have to be precompiled (*note Wheel: d. + intentionally does not include compiled Python files). + +Distribution Package + + A versioned archive file that contains Python *note packages: a, + *note modules: 255, and other resource files that are used to + distribute a *note Release: 256. The archive file is what an + end-user will download from the internet and install. + + A distribution package is more commonly referred to with the single + words “package” or “distribution”, but this guide may use the + expanded term when more clarity is needed to prevent confusion with + an *note Import Package: a. (which is also commonly called a + “package”) or another kind of distribution (e.g. a Linux + distribution or the Python language distribution), which are often + referred to with the single term “distribution”. + +Egg + + A *note Built Distribution: 63. format introduced by *note + setuptools: 2d, which is being replaced by *note Wheel: d. For + details, see The Internal Structure of Python Eggs(1) and Python + Eggs(2) + +Extension Module + + A *note Module: 255. written in the low-level language of the + Python implementation: C/C++ for Python, Java for Jython. + Typically contained in a single dynamically loadable pre-compiled + file, e.g. a shared object (.so) file for Python extensions on + Unix, a DLL (given the .pyd extension) for Python extensions on + Windows, or a Java class file for Jython extensions. + +Known Good Set (KGS) + + A set of distributions at specified versions which are compatible + with each other. Typically a test suite will be run which passes + all tests before a specific set of packages is declared a known + good set. This term is commonly used by frameworks and toolkits + which are comprised of multiple individual distributions. + +Import Package + + A Python module which can contain other modules or recursively, + other packages. + + An import package is more commonly referred to with the single word + “package”, but this guide will use the expanded term when more + clarity is needed to prevent confusion with a *note Distribution + Package: b. which is also commonly called a “package”. + +Module + + The basic unit of code reusability in Python, existing in one of + two types: *note Pure Module: 259, or *note Extension Module: 257. + +Package Index + + A repository of distributions with a web interface to automate + *note package: b. discovery and consumption. + +Per Project Index + + A private or other non-canonical *note Package Index: ec. indicated + by a specific *note Project: 1fc. as the index preferred or + required to resolve dependencies of that project. + +Project + + A library, framework, script, plugin, application, or collection of + data or other resources, or some combination thereof that is + intended to be packaged into a *note Distribution: b. + + Since most projects create *note Distributions: b. using either PEP + 518(3) ‘build-system’, *note distutils: c0. or *note setuptools: + 2d, another practical way to define projects currently is something + that contains a *note pyproject.toml: 1d7, *note setup.py: 25b, or + *note setup.cfg: 25c. file at the root of the project source + directory. + + Python projects must have unique names, which are registered on + *note PyPI: 39. Each project will then contain one or more *note + Releases: 256, and each release may comprise one or more *note + distributions: b. + + Note that there is a strong convention to name a project after the + name of the package that is imported to run that project. However, + this doesn’t have to hold true. It’s possible to install a + distribution from the project ‘foo’ and have it provide a package + importable only as ‘bar’. + +Pure Module + + A *note Module: 255. written in Python and contained in a single + ‘.py’ file (and possibly associated ‘.pyc’ and/or ‘.pyo’ files). + +Python Packaging Authority (PyPA) + + PyPA is a working group that maintains many of the relevant + projects in Python packaging. They maintain a site at + ‘https://www.pypa.io’, host projects on GitHub(4) and Bitbucket(5), + and discuss issues on the distutils-sig mailing list(6) and the + Python Discourse forum(7). + +Python Package Index (PyPI) + + PyPI(8) is the default *note Package Index: ec. for the Python + community. It is open to all Python developers to consume and + distribute their distributions. + +pypi.org + + pypi.org(9) is the domain name for the *note Python Package Index + (PyPI): 39. It replaced the legacy index domain name, + ‘pypi.python.org’, in 2017. It is powered by *note Warehouse: 22d. + +pyproject.toml + + The tool-agnostic *note Project: 1fc. specification file. Defined + in PEP 518(10). + +Release + + A snapshot of a *note Project: 1fc. at a particular point in time, + denoted by a version identifier. + + Making a release may entail the publishing of multiple *note + Distributions: b. For example, if version 1.0 of a project was + released, it could be available in both a source distribution + format and a Windows installer distribution format. + +Requirement + + A specification for a *note package: b. to be installed. *note + pip: 2b, the *note PYPA: 25d. recommended installer, allows various + forms of specification that can all be considered a “requirement”. + For more information, see the pip install(11) reference. + +Requirement Specifier + + A format used by *note pip: 2b. to install packages from a *note + Package Index: ec. For an EBNF diagram of the format, see the + pkg_resources.Requirement(12) entry in the *note setuptools: 2d. + docs. For example, “foo>=1.3” is a requirement specifier, where + “foo” is the project name, and the “>=1.3” portion is the *note + Version Specifier: 3b. + +Requirements File + + A file containing a list of *note Requirements: 25e. that can be + installed using *note pip: 2b. For more information, see the *note + pip: 2b. docs on Requirements Files(13). + +setup.py + +setup.cfg + + The project specification files for *note distutils: c0. and *note + setuptools: 2d. See also *note pyproject.toml: 1d7. + +Source Archive + + An archive containing the raw source code for a *note Release: 256, + prior to creation of a *note Source Distribution: 3d. or *note + Built Distribution: 63. + +Source Distribution (or “sdist”) + + A *note distribution: b. format (usually generated using ‘python + setup.py sdist’) that provides metadata and the essential source + files needed for installing by a tool like *note pip: 2b, or for + generating a *note Built Distribution: 63. + +System Package + + A package provided in a format native to the operating system, e.g. + an rpm or dpkg file. + +Version Specifier + + The version component of a *note Requirement Specifier: 3a. For + example, the “>=1.3” portion of “foo>=1.3”. PEP 440(14) contains a + full specification(15) of the specifiers that Python packaging + currently supports. Support for PEP440 was implemented in *note + setuptools: 2d. v8.0 and *note pip: 2b. v6.0. + +Virtual Environment + + An isolated Python environment that allows packages to be installed + for use by a particular application, rather than being installed + system wide. For more information, see the section on *note + Creating Virtual Environments: 31. + +Wheel + + A *note Built Distribution: 63. format introduced by PEP 427(16), + which is intended to replace the *note Egg: 7c. format. Wheel is + currently supported by *note pip: 2b. + +Working Set + + A collection of *note distributions: b. available for importing. + These are the distributions that are on the sys.path(17) variable. + At most, one *note Distribution: b. for a project is possible in a + working set. + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/deprecated/python_eggs.html + + (2) http://peak.telecommunity.com/DevCenter/PythonEggs + + (3) https://www.python.org/dev/peps/pep-0518 + + (4) https://github.com/pypa + + (5) https://bitbucket.org/pypa + + (6) https://mail.python.org/mailman3/lists/distutils-sig.python.org/ + + (7) https://discuss.python.org/c/packaging + + (8) https://pypi.org + + (9) https://pypi.org + + (10) https://www.python.org/dev/peps/pep-0518 + + (11) https://pip.pypa.io/en/latest/reference/pip_install/#pip-install + + (12) +https://setuptools.readthedocs.io/en/latest/pkg_resources.html#requirement-objects + + (13) https://pip.pypa.io/en/latest/user_guide/#requirements-files + + (14) https://www.python.org/dev/peps/pep-0440 + + (15) https://www.python.org/dev/peps/pep-0440#version-specifiers + + (16) https://www.python.org/dev/peps/pep-0427 + + (17) https://docs.python.org/2/library/sys.html#sys.path + + +File: pythonpackagingguide.info, Node: How to Get Support, Next: Contribute to this guide, Prev: Glossary, Up: Top + +8 How to Get Support +******************** + +For support related to a specific project, see the links on the *note +Projects: 213. page. + +For something more general, or when you’re just not sure, use the +distutils-sig(1) list. + + ---------- Footnotes ---------- + + (1) http://mail.python.org/mailman/listinfo/distutils-sig + + +File: pythonpackagingguide.info, Node: Contribute to this guide, Next: News, Prev: How to Get Support, Up: Top + +9 Contribute to this guide +************************** + +The Python Packaging User Guide welcomes contributors! There are lots +of ways to help out, including: + + * Reading the guide and giving feedback + + * Reviewing new contributions + + * Revising existing content + + * Writing new content + +Most of the work on the Python Packaging User Guide takes place on the +project’s GitHub repository(1). To get started, check out the list of +open issues(2) and pull requests(3). If you’re planning to write or +edit the guide, please read the *note style guide: 266. + +By contributing to the Python Packaging User Guide, you’re expected to +follow the PSF’s Code of Conduct(4). + +* Menu: + +* Documentation types:: +* Building the guide locally:: +* Where the guide is deployed:: +* Style guide:: + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/python-packaging-user-guide/ + + (2) https://github.com/pypa/python-packaging-user-guide/issues + + (3) https://github.com/pypa/python-packaging-user-guide/pulls + + (4) https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md + + +File: pythonpackagingguide.info, Node: Documentation types, Next: Building the guide locally, Up: Contribute to this guide + +9.1 Documentation types +======================= + +This project consists of four distinct documentation types with specific +purposes. When proposing new additions to the project please pick the +appropriate documentation type. + +* Menu: + +* Tutorials: Tutorials<2>. +* Guides: Guides<2>. +* Discussions: Discussions<2>. +* Specifications:: + + +File: pythonpackagingguide.info, Node: Tutorials<2>, Next: Guides<2>, Up: Documentation types + +9.1.1 Tutorials +--------------- + +Tutorials are focused on teaching the reader new concepts by +accomplishing a goal. They are opinionated step-by-step guides. They +do not include extraneous warnings or information. example +tutorial-style document(1). + + ---------- Footnotes ---------- + + (1) https://docs.djangoproject.com/en/1.11/intro/ + + +File: pythonpackagingguide.info, Node: Guides<2>, Next: Discussions<2>, Prev: Tutorials<2>, Up: Documentation types + +9.1.2 Guides +------------ + +Guides are focused on accomplishing a specific task and can assume some +level of pre-requisite knowledge. These are similar to tutorials, but +have a narrow and clear focus and can provide lots of caveats and +additional information as needed. They may also discuss multiple +approaches to accomplishing the task. *note example guide-style +document: 130. + + +File: pythonpackagingguide.info, Node: Discussions<2>, Next: Specifications, Prev: Guides<2>, Up: Documentation types + +9.1.3 Discussions +----------------- + +Discussions are focused on understanding and information. These explore +a specific topic without a specific goal in mind. *note example +discussion-style document: 18c. + + +File: pythonpackagingguide.info, Node: Specifications, Prev: Discussions<2>, Up: Documentation types + +9.1.4 Specifications +-------------------- + +Specifications are reference documention focused on comprehensively +documenting an agreed-upon interface for interoperability between +packaging tools. *note example specification-style document: 199. + + +File: pythonpackagingguide.info, Node: Building the guide locally, Next: Where the guide is deployed, Prev: Documentation types, Up: Contribute to this guide + +9.2 Building the guide locally +============================== + +Though not required to contribute, it may be useful to build this guide +locally in order to test your changes. In order to build this guide +locally, you’ll need: + + 1. Nox(1). You can install or upgrade nox using ‘pip’: + + pip install --user nox + + 2. Python 3.6. Our build scripts are designed to work with Python 3.6 + only. See the Hitchhiker’s Guide to Python installation + instructions(2) to install Python 3.6 on your operating system. + +To build the guide, run the following bash command in the source folder: + + nox -s build + +After the process has completed you can find the HTML output in the +‘./build/html’ directory. You can open the ‘index.html’ file to view +the guide in web browser, but it’s recommended to serve the guide using +an HTTP server. + +You can build the guide and serve it via an HTTP server using the +following command: + + nox -s preview + +The guide will be browsable via ‘http://localhost:8000’. + + ---------- Footnotes ---------- + + (1) https://nox.readthedocs.io/en/latest/ + + (2) http://docs.python-guide.org/en/latest/starting/installation/ + + +File: pythonpackagingguide.info, Node: Where the guide is deployed, Next: Style guide, Prev: Building the guide locally, Up: Contribute to this guide + +9.3 Where the guide is deployed +=============================== + +The guide is deployed via ReadTheDocs and the configuration lives at +‘https://readthedocs.org/projects/python-packaging-user-guide/’. It’s +served from a custom domain and fronted by Fast.ly. + + +File: pythonpackagingguide.info, Node: Style guide, Prev: Where the guide is deployed, Up: Contribute to this guide + +9.4 Style guide +=============== + +This style guide has recommendations for how you should write the Python +Packaging User Guide. Before you start writing, please review it. By +following the style guide, your contributions will help add to a +cohesive whole and make it easier for your contributions to be accepted +into the project. + +* Menu: + +* Purpose:: +* Scope:: +* Audience:: +* Voice and tone:: +* Conventions and mechanics:: + + +File: pythonpackagingguide.info, Node: Purpose, Next: Scope, Up: Style guide + +9.4.1 Purpose +------------- + +The purpose of the Python Packaging User Guide is to be the +authoritative resource on how to package, publish, and install Python +projects using current tools. + + +File: pythonpackagingguide.info, Node: Scope, Next: Audience, Prev: Purpose, Up: Style guide + +9.4.2 Scope +----------- + +The guide is meant to answer questions and solve problems with accurate +and focused recommendations. + +The guide isn’t meant to be comprehensive and it’s not meant to replace +individual projects’ documentation. For example, pip has dozens of +commands, options, and settings. The pip documentation describes each +of them in detail, while this guide describes only the parts of pip that +are needed to complete the specific tasks described in this guide. + + +File: pythonpackagingguide.info, Node: Audience, Next: Voice and tone, Prev: Scope, Up: Style guide + +9.4.3 Audience +-------------- + +The audience of this guide is anyone who uses Python with packages. + +Don’t forget that the Python community is big and welcoming. Readers +may not share your age, gender, education, culture, and more, but they +deserve to learn about packaging just as much as you do. + +In particular, keep in mind that not all people who use Python see +themselves as programmers. The audience of this guide includes +astronomers or painters or students as well as professional software +developers. + + +File: pythonpackagingguide.info, Node: Voice and tone, Next: Conventions and mechanics, Prev: Audience, Up: Style guide + +9.4.4 Voice and tone +-------------------- + +When writing this guide, strive to write with a voice that’s +approachable and humble, even if you have all the answers. + +Imagine you’re working on a Python project with someone you know to be +smart and skilled. You like working with them and they like working +with you. That person has asked you a question and you know the answer. +How do you respond? `That' is how you should write this guide. + +Here’s a quick check: try reading aloud to get a sense for your +writing’s voice and tone. Does it sound like something you would say or +does it sound like you’re acting out a part or giving a speech? Feel +free to use contractions and don’t worry about sticking to fussy grammar +rules. You are hereby granted permission to end a sentence in a +preposition, if that’s what you want to end it with. + +When writing the guide, adjust your tone for the seriousness and +difficulty of the topic. If you’re writing an introductory tutorial, +it’s OK to make a joke, but if you’re covering a sensitive security +recommendation, you might want to avoid jokes altogether. + + +File: pythonpackagingguide.info, Node: Conventions and mechanics, Prev: Voice and tone, Up: Style guide + +9.4.5 Conventions and mechanics +------------------------------- + +`Write to the reader' + + When giving recommendations or steps to take, address the reader as + `you' or use the imperative mood. + + Wrong: To install it, the user runs… + Right: You can install it by running… + Right: To install it, run… + +`State assumptions' + + Avoid making unstated assumptions. Reading on the web means that + any page of the guide may be the first page of the guide that the + reader ever sees. If you’re going to make assumptions, then say + what assumptions that you’re going to make. + +`Cross-reference generously' + + The first time you mention a tool or practice, link to the part of + the guide that covers it, or link to a relevant document elsewhere. + Save the reader a search. + +`Respect naming practices' + + When naming tools, sites, people, and other proper nouns, use their + preferred capitalization. + + Wrong: Pip uses… + Right: pip uses… + + Wrong: …hosted on github. + Right: …hosted on GitHub. + +`Use a gender-neutral style' + + Often, you’ll address the reader directly with `you', `your' and + `yours'. Otherwise, use gender-neutral pronouns `they', `their', + and `theirs' or avoid pronouns entirely. + + Wrong: A maintainer uploads the file. Then he… + Right: A maintainer uploads the file. Then they… + Right: A maintainer uploads the file. Then the maintainer… + +`Headings' + + Write headings that use words the reader is searching for. A good + way to do this is to have your heading complete an implied + question. For example, a reader might want to know `How do I + install MyLibrary?' so a good heading might be `Install + MyLibrary'. + + In section headings, use sentence case. In other words, write + headings as you would write a typical sentence. + + Wrong: Things You Should Know About Python + Right: Things you should know about Python + +`Numbers' + + In body text, write numbers one through nine as words. For other + numbers or numbers in tables, use numerals. + + +File: pythonpackagingguide.info, Node: News, Next: Get started, Prev: Contribute to this guide, Up: Top + +10 News +******* + +* Menu: + +* September 2019:: +* August 2019:: +* July 2019:: +* June 2019:: +* May 2019:: +* April 2019:: +* March 2019:: +* February 2019:: +* January 2019:: +* December 2018:: +* November 2018:: +* October 2018:: +* September 2018:: +* August 2018:: +* July 2018:: +* June 2018:: +* May 2018:: +* April 2018:: +* March 2018:: +* February 2018:: +* January 2018:: +* December 2017:: +* November 2017:: +* October 2017:: +* September 2017:: +* August 2017:: +* July 2017:: +* June 2017:: +* May 2017:: +* April 2017:: +* March 2017:: +* February 2017:: + + +File: pythonpackagingguide.info, Node: September 2019, Next: August 2019, Up: News + +10.1 September 2019 +=================== + + - Added a guide about publishing dists via GitHub Actions. (PR + #647(1)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/647 + + +File: pythonpackagingguide.info, Node: August 2019, Next: July 2019, Prev: September 2019, Up: News + +10.2 August 2019 +================ + + - Updated to use ‘python3 -m’ when installing pipx. (PR #631(1)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/631 + + +File: pythonpackagingguide.info, Node: July 2019, Next: June 2019, Prev: August 2019, Up: News + +10.3 July 2019 +============== + + - Marked all PEP numbers with the :pep: role. (PR #629(1)) + + - Upgraded Sphinx version and removed pypa.io intersphinx. (PR + #625(2)) + + - Mentioned ‘find_namespace_packages’. (PR #622(3)) + + - Updated directory layout examples for consistency. (PR #611(4)) + + - Updated Bandersnatch link to GitHub. (PR #623(5)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/629 + + (2) https://github.com/pypa/packaging.python.org/pull/625 + + (3) https://github.com/pypa/packaging.python.org/pull/622 + + (4) https://github.com/pypa/packaging.python.org/pull/611 + + (5) https://github.com/pypa/packaging.python.org/pull/623 + + +File: pythonpackagingguide.info, Node: June 2019, Next: May 2019, Prev: July 2019, Up: News + +10.4 June 2019 +============== + + - Fixed some typos. (PR #620(1)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/620 + + +File: pythonpackagingguide.info, Node: May 2019, Next: April 2019, Prev: June 2019, Up: News + +10.5 May 2019 +============= + + - Added ‘python_requires’ usage to packaging tutorial. (PR #613(1)) + + - Added a MANIFEST.in guide page. (PR #609(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/613 + + (2) https://github.com/pypa/packaging.python.org/pull/609 + + +File: pythonpackagingguide.info, Node: April 2019, Next: March 2019, Prev: May 2019, Up: News + +10.6 April 2019 +=============== + + - Added a mention for ‘shiv’ in the key projects section. (PR + #608(1)) + + - Reduced emphasis on virtualenv. (PR #606(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/608 + + (2) https://github.com/pypa/packaging.python.org/pull/606 + + +File: pythonpackagingguide.info, Node: March 2019, Next: February 2019, Prev: April 2019, Up: News + +10.7 March 2019 +=============== + + - Moved single-sourcing guide version option to Python 3. (PR + #605(1)) + + - Covered RTD details for contributing. (PR #600(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/605 + + (2) https://github.com/pypa/packaging.python.org/pull/600 + + +File: pythonpackagingguide.info, Node: February 2019, Next: January 2019, Prev: March 2019, Up: News + +10.8 February 2019 +================== + + - Elaborate upon the differences between the tutorial and the real + packaging process. (PR #602(1)) + + - Added instructions to install Python CLI applications. (PR + #594(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/602 + + (2) https://github.com/pypa/packaging.python.org/pull/594 + + +File: pythonpackagingguide.info, Node: January 2019, Next: December 2018, Prev: February 2019, Up: News + +10.9 January 2019 +================= + + - Added ‘--no-deps’ to the packaging tutorial. (PR #593(1)) + + - Updated Sphinx and Nox. (PR #591(2)) + + - Referenced Twine from Python3. (PR #581(3)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/593 + + (2) https://github.com/pypa/packaging.python.org/pull/591 + + (3) https://github.com/pypa/packaging.python.org/pull/581 + + +File: pythonpackagingguide.info, Node: December 2018, Next: November 2018, Prev: January 2019, Up: News + +10.10 December 2018 +=================== + + - No programmers in the office! + + +File: pythonpackagingguide.info, Node: November 2018, Next: October 2018, Prev: December 2018, Up: News + +10.11 November 2018 +=================== + + - Removed landing page link to PyPI migration guide. (PR #575(1)) + + - Changed bumpversion to bump2version. (PR #572(2)) + + - Added single-sourcing package version example. (PR #573(3)) + + - Added a guide for creating documentation. (PR #568(4)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/575 + + (2) https://github.com/pypa/packaging.python.org/pull/572 + + (3) https://github.com/pypa/packaging.python.org/pull/573 + + (4) https://github.com/pypa/packaging.python.org/pull/568 + + +File: pythonpackagingguide.info, Node: October 2018, Next: September 2018, Prev: November 2018, Up: News + +10.12 October 2018 +================== + + - Updated Nox package name. (PR #566(1)) + + - Mentioned Sphinx extensions in guides. (PR #562(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/566 + + (2) https://github.com/pypa/packaging.python.org/pull/562 + + +File: pythonpackagingguide.info, Node: September 2018, Next: August 2018, Prev: October 2018, Up: News + +10.13 September 2018 +==================== + + - Added a section on checking RST markup. (PR #554(1)) + + - Updated user installs page. (PR #558(2)) + + - Updated Google BigQuery urls. (PR #556(3)) + + - Replaced tar command with working command. (PR #552(4)) + + - Changed to double quotes in the pip install SomeProject==1.4. (PR + #550(5)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/554 + + (2) https://github.com/pypa/packaging.python.org/pull/558 + + (3) https://github.com/pypa/packaging.python.org/pull/556 + + (4) https://github.com/pypa/packaging.python.org/pull/552 + + (5) https://github.com/pypa/packaging.python.org/pull/550 + + +File: pythonpackagingguide.info, Node: August 2018, Next: July 2018, Prev: September 2018, Up: News + +10.14 August 2018 +================= + + - Removed the recommendation to store passwords in cleartext. (PR + #546(1)) + + - Moved the Overview to a task based lead in along with the others. + (PR #540(2)) + + - Updated Python version supported by virtualenv. (PR #538(3)) + + - Added outline/rough draft of new Overview page. (PR #519(4)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/546 + + (2) https://github.com/pypa/packaging.python.org/pull/540 + + (3) https://github.com/pypa/packaging.python.org/pull/538 + + (4) https://github.com/pypa/packaging.python.org/pull/519 + + +File: pythonpackagingguide.info, Node: July 2018, Next: June 2018, Prev: August 2018, Up: News + +10.15 July 2018 +=============== + + - Improved binary extension docs. (PR #531(1)) + + - Added scikit-build to key projects. (PR #530(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/531 + + (2) https://github.com/pypa/packaging.python.org/pull/530 + + +File: pythonpackagingguide.info, Node: June 2018, Next: May 2018, Prev: July 2018, Up: News + +10.16 June 2018 +=============== + + - Fixed categories of interop PEP for pypa.io. (PR #527(1)) + + - Updated Markdown descriptions explanation. (PR #522(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/527 + + (2) https://github.com/pypa/packaging.python.org/pull/522 + + +File: pythonpackagingguide.info, Node: May 2018, Next: April 2018, Prev: June 2018, Up: News + +10.17 May 2018 +============== + + - Noted issues with Provides-Dist and Obsoletes-Dist. (PR #513(1)) + + - Removed outdated warning about Python version mixing with Pipenv. + (PR #501(2)) + + - Simplified packaging tutorial. (PR #498(3)) + + - Updated Windows users instructions for clarity. (PR #493(4)) + + - Updated the license section description for completeness. (PR + #492(5)) + + - Added specification-style document to contributing section. (PR + #489(6)) + + - Added documentation types to contributing guide. (PR #485(7)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/513 + + (2) https://github.com/pypa/packaging.python.org/pull/501 + + (3) https://github.com/pypa/packaging.python.org/pull/498 + + (4) https://github.com/pypa/packaging.python.org/pull/493 + + (5) https://github.com/pypa/packaging.python.org/pull/492 + + (6) https://github.com/pypa/packaging.python.org/pull/489 + + (7) https://github.com/pypa/packaging.python.org/pull/485 + + +File: pythonpackagingguide.info, Node: April 2018, Next: March 2018, Prev: May 2018, Up: News + +10.18 April 2018 +================ + + - Added README guide. (PR #461(1)) + + - Updated instructions and status for PyPI launch. (PR #475(2)) + + - Added instructions for Warehouse. (PR #471(3)) + + - Removed GPG references from publishing tutorial. (PR #466(4)) + + - Added ‘What’s in which Python 3.4–3.6?’. (PR #468(5)) + + - Added a guide for phasing out Python versions. (PR #459(6)) + + - Made default Description-Content-Type variant GFM. (PR #462(7)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/461 + + (2) https://github.com/pypa/packaging.python.org/pull/475 + + (3) https://github.com/pypa/packaging.python.org/pull/471 + + (4) https://github.com/pypa/packaging.python.org/pull/466 + + (5) https://github.com/pypa/packaging.python.org/pull/468 + + (6) https://github.com/pypa/packaging.python.org/pull/459 + + (7) https://github.com/pypa/packaging.python.org/pull/462 + + +File: pythonpackagingguide.info, Node: March 2018, Next: February 2018, Prev: April 2018, Up: News + +10.19 March 2018 +================ + + - Updated “installing scientific packages”. (PR #455(1)) + + - Added ‘long_description_content_type’ to follow PEP 556. (PR + #457(2)) + + - Clarified a long description classifier on pypi.org. (PR #456(3)) + + - Updated Core Metadata spec to follw PEP 556. (PR #412(4)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/455 + + (2) https://github.com/pypa/packaging.python.org/pull/457 + + (3) https://github.com/pypa/packaging.python.org/pull/456 + + (4) https://github.com/pypa/packaging.python.org/pull/412 + + +File: pythonpackagingguide.info, Node: February 2018, Next: January 2018, Prev: March 2018, Up: News + +10.20 February 2018 +=================== + + - Added python3-venv and python3-pip to Debian installation + instructions. (PR #445(1)) + + - Updated PyPI migration info. (PR #439(2)) + + - Added a warning about managing multiple versions with pipenv. (PR + #430(3)) + + - Added example of multiple emails to Core Metadata. (PR #429(4)) + + - Added explanation of “legacy” in test.pypi.org/legacy. (PR + #426(5)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/445 + + (2) https://github.com/pypa/packaging.python.org/pull/439 + + (3) https://github.com/pypa/packaging.python.org/pull/430 + + (4) https://github.com/pypa/packaging.python.org/pull/429 + + (5) https://github.com/pypa/packaging.python.org/pull/426 + + +File: pythonpackagingguide.info, Node: January 2018, Next: December 2017, Prev: February 2018, Up: News + +10.21 January 2018 +================== + + - Added a link to PyPI’s list of classifiers. (PR #425(1)) + + - Updated README.rst explanation. (PR #419(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/425 + + (2) https://github.com/pypa/packaging.python.org/pull/419 + + +File: pythonpackagingguide.info, Node: December 2017, Next: November 2017, Prev: January 2018, Up: News + +10.22 December 2017 +=================== + + - Replaced ‘~’ with ‘$HOME’ in guides and tutorials. (PR #418(1)) + + - Noted which fields can be used with environment markers. (PR + #416(2)) + + - Updated Requires-Python section. (PR #414(3)) + + - Added news page. (PR #404(4)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/418 + + (2) https://github.com/pypa/packaging.python.org/pull/416 + + (3) https://github.com/pypa/packaging.python.org/pull/414 + + (4) https://github.com/pypa/packaging.python.org/pull/404 + + +File: pythonpackagingguide.info, Node: November 2017, Next: October 2017, Prev: December 2017, Up: News + +10.23 November 2017 +=================== + + - Introduced a new dependency management tutorial based on Pipenv. + (PR #402(1)) + + - Updated the `Single Sourcing Package Version' tutorial to reflect + pip’s current strategy. (PR #400(2)) + + - Added documentation about the ‘py_modules’ argument to ‘setup’. + (PR #398(3)) + + - Simplified the wording for the ‘manifest.in’ section. (PR #395(4)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/402 + + (2) https://github.com/pypa/packaging.python.org/pull/400 + + (3) https://github.com/pypa/packaging.python.org/pull/398 + + (4) https://github.com/pypa/packaging.python.org/pull/395 + + +File: pythonpackagingguide.info, Node: October 2017, Next: September 2017, Prev: November 2017, Up: News + +10.24 October 2017 +================== + + - Added a specification for the ‘entry_points.txt’ file. (PR + #398(1)) + + - Created a new guide for managing packages using ‘pip’ and + ‘virtualenv’. (PR #385(2)) + + - Split the specifications page into multiple pages. (PR #386(3)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/398 + + (2) https://github.com/pypa/packaging.python.org/pull/385 + + (3) https://github.com/pypa/packaging.python.org/pull/386 + + +File: pythonpackagingguide.info, Node: September 2017, Next: August 2017, Prev: October 2017, Up: News + +10.25 September 2017 +==================== + + - Encouraged using ‘readme_renderer’ to validate ‘README.rst’. (PR + #379(1)) + + - Recommended using the ‘--user-base’ option. (PR #374(2)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/379 + + (2) https://github.com/pypa/packaging.python.org/pull/374 + + +File: pythonpackagingguide.info, Node: August 2017, Next: July 2017, Prev: September 2017, Up: News + +10.26 August 2017 +================= + + - Added a new, experimental tutorial on installing packages using + ‘Pipenv’. (PR #369(1)) + + - Added a new guide on how to use ‘TestPyPI’. (PR #366(2)) + + - Added ‘pypi.org’ as a term. (PR #365(3)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/369 + + (2) https://github.com/pypa/packaging.python.org/pull/366 + + (3) https://github.com/pypa/packaging.python.org/pull/365 + + +File: pythonpackagingguide.info, Node: July 2017, Next: June 2017, Prev: August 2017, Up: News + +10.27 July 2017 +=============== + + - Added ‘flit’ to the key projects list. (PR #358(1)) + + - Added ‘enscons’ to the list of key projects. (PR #357(2)) + + - Updated this guide’s ‘readme’ with instructions on how to build the + guide locally. (PR #356(3)) + + - Made the new ‘TestPyPI’ URL more visible, adding note to homepage + about pypi.org. (PR #354(4)) + + - Added a note about the removal of the explicit registration API. + (PR #347(5)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/358 + + (2) https://github.com/pypa/packaging.python.org/pull/357 + + (3) https://github.com/pypa/packaging.python.org/pull/356 + + (4) https://github.com/pypa/packaging.python.org/pull/354 + + (5) https://github.com/pypa/packaging.python.org/pull/347 + + +File: pythonpackagingguide.info, Node: June 2017, Next: May 2017, Prev: July 2017, Up: News + +10.28 June 2017 +=============== + + - Added a document on migrating uploads to ‘PyPI.org’. (PR #339(1)) + + - Added documentation for ‘python_requires’. (PR #338(2)) + + - Added a note about PyPI migration in the `Tool Recommendations' + tutorial. (PR #335(3)) + + - Added a note that ‘manifest.in’ does not affect wheels. (PR + #332(4)) + + - Added a license section to the distributing guide. (PR #331(5)) + + - Expanded the section on the ‘name’ argument. (PR #329(6)) + + - Adjusted the landing page. (PR #327(7), PR #326(8), PR #324(9)) + + - Updated to Sphinx 1.6.2. (PR #323(10)) + + - Switched to the PyPA theme. (PR #305(11)) + + - Re-organized the documentation into the new structure. (PR + #318(12)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/339 + + (2) https://github.com/pypa/packaging.python.org/pull/338 + + (3) https://github.com/pypa/packaging.python.org/pull/335 + + (4) https://github.com/pypa/packaging.python.org/pull/332 + + (5) https://github.com/pypa/packaging.python.org/pull/331 + + (6) https://github.com/pypa/packaging.python.org/pull/329 + + (7) https://github.com/pypa/packaging.python.org/pull/327 + + (8) https://github.com/pypa/packaging.python.org/pull/326 + + (9) https://github.com/pypa/packaging.python.org/pull/324 + + (10) https://github.com/pypa/packaging.python.org/pull/323 + + (11) https://github.com/pypa/packaging.python.org/pull/305 + + (12) https://github.com/pypa/packaging.python.org/pull/318 + + +File: pythonpackagingguide.info, Node: May 2017, Next: April 2017, Prev: June 2017, Up: News + +10.29 May 2017 +============== + + - Added documentation for the ‘Description-Content-Type’ field. (PR + #258(1)) + + - Added contributor and style guide. (PR #307(2)) + + - Documented ‘pip’ and ‘easy_install’’s differences for per-project + indexes. (PR #233(3)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/258 + + (2) https://github.com/pypa/packaging.python.org/pull/307 + + (3) https://github.com/pypa/packaging.python.org/pull/233 + + +File: pythonpackagingguide.info, Node: April 2017, Next: March 2017, Prev: May 2017, Up: News + +10.30 April 2017 +================ + + - Added travis configuration for testing pull requests. (PR #300(1)) + + - Mentioned the requirement of the ‘wheel’ package for creating + wheels (PR #299(2)) + + - Removed the ‘twine register’ reference in the `Distributing + Packages' tutorial. (PR #271(3)) + + - Added a topic on plugin discovery. (PR #294(4), PR #296(5)) + + - Added a topic on namespace packages. (PR #290(6)) + + - Added documentation explaining prominently how to install ‘pip’ in + ‘/usr/local’. (PR #230(7)) + + - Updated development mode documentation to mention that order of + local packages matters. (PR #208(8)) + + - Convert readthedocs link for their ‘.org’ -> ‘.io’ migration for + hosted projects (PR #239(9)) + + - Swaped order of ‘setup.py’ arguments for the upload command, as + order is significant. (PR #260(10)) + + - Explained how to install from unsupported sources using a helper + application. (PR #289(11)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/300 + + (2) https://github.com/pypa/packaging.python.org/pull/299 + + (3) https://github.com/pypa/packaging.python.org/pull/271 + + (4) https://github.com/pypa/packaging.python.org/pull/294 + + (5) https://github.com/pypa/packaging.python.org/pull/296 + + (6) https://github.com/pypa/packaging.python.org/pull/290 + + (7) https://github.com/pypa/packaging.python.org/pull/230 + + (8) https://github.com/pypa/packaging.python.org/pull/208 + + (9) https://github.com/pypa/packaging.python.org/pull/239 + + (10) https://github.com/pypa/packaging.python.org/pull/260 + + (11) https://github.com/pypa/packaging.python.org/pull/289 + + +File: pythonpackagingguide.info, Node: March 2017, Next: February 2017, Prev: April 2017, Up: News + +10.31 March 2017 +================ + + - Covered ‘manylinux1’ in `Platform Wheels'. (PR #283(1)) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging.python.org/pull/283 + + +File: pythonpackagingguide.info, Node: February 2017, Prev: March 2017, Up: News + +10.32 February 2017 +=================== + + - Added PEP 518(1). (PR #281(2)) + +Welcome to the `Python Packaging User Guide', a collection of tutorials +and references to help you distribute and install Python packages with +modern tools. + +This guide is maintained on GitHub(3) by the Python Packaging +Authority(4). We happily accept any *note contributions and feedback: +264. 😊 + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0518 + + (2) https://github.com/pypa/packaging.python.org/pull/281 + + (3) https://github.com/pypa/python-packaging-user-guide + + (4) https://pypa.io + + +File: pythonpackagingguide.info, Node: Get started, Next: Learn more, Prev: News, Up: Top + +11 Get started +************** + +Essential tools and concepts for working within the Python development +ecosystem are covered in our *note Tutorials: 20. section: + + * to learn how to install packages, see the *note tutorial on + installing packages: 23. + + * to learn how to manage dependencies in a version controlled + project, see the *note tutorial on managing application + dependencies: 4a. + + * to learn how to package and distribute your projects, see the *note + tutorial on packaging and distributing: e. + + * to get an overview of packaging options for Python libraries and + applications, see the *note Overview of Python Packaging: 1. + + +File: pythonpackagingguide.info, Node: Learn more, Next: Index, Prev: Get started, Up: Top + +12 Learn more +************* + +Beyond our *note Tutorials: 20, this guide has several other resources: + + * the *note Guides: 22. section for walk throughs, such as *note + Installing pip/setuptools/wheel with Linux Package Managers: 2c. or + *note Packaging binary extensions: 69. + + * the *note Discussions: 17d. section for in-depth references on + topics such as *note Deploying Python applications: 180. or *note + pip vs easy_install: 189. + + * the *note PyPA specifications: 195. section for packaging + interoperability specifications + +Additionally, there is a list of *note other projects: 213. maintained +by members of the Python Packaging Authority. + + +File: pythonpackagingguide.info, Node: Index, Prev: Learn more, Up: Top + +Index +***** + +[index] +* Menu: + +* Binary Distribution: Glossary. (line 6) +* Built Distribution: Glossary. (line 9) +* Distribution Package: Glossary. (line 19) +* Egg: Glossary. (line 34) +* environment variable; PATH: Installing to the User Site. + (line 20) +* environment variable; PATH <1>: Installing to the User Site. + (line 21) +* environment variable; PATH <2>: Installing to the User Site. + (line 23) +* Extension Module: Glossary. (line 41) +* Import Package: Glossary. (line 58) +* Known Good Set (KGS): Glossary. (line 50) +* Module: Glossary. (line 68) +* Package Index: Glossary. (line 73) +* PATH: Installing to the User Site. + (line 20) +* PATH <1>: Installing to the User Site. + (line 21) +* PATH <2>: Installing to the User Site. + (line 23) +* Per Project Index: Glossary. (line 78) +* Project: Glossary. (line 84) +* Pure Module: Glossary. (line 108) +* pypi.org: Glossary. (line 127) +* pyproject.toml: Glossary. (line 134) +* Python Enhancement Proposals; PEP 1: History and change workflow. + (line 9) +* Python Enhancement Proposals; PEP 301: Classifier multiple use. + (line 9) +* Python Enhancement Proposals; PEP 345: Version specifiers. (line 10) +* Python Enhancement Proposals; PEP 345 <1>: Dependency specifiers. + (line 10) +* Python Enhancement Proposals; PEP 376: Wheel vs Egg. (line 33) +* Python Enhancement Proposals; PEP 376 <1>: History and change workflow. + (line 6) +* Python Enhancement Proposals; PEP 376 <2>: File format. (line 8) +* Python Enhancement Proposals; PEP 397: console_scripts. (line 27) +* Python Enhancement Proposals; PEP 420: Creating a namespace package. + (line 10) +* Python Enhancement Proposals; PEP 420 <1>: Native namespace packages. + (line 6) +* Python Enhancement Proposals; PEP 425: Pure Python Wheels. (line 17) +* Python Enhancement Proposals; PEP 425 <1>: Platform Wheels. (line 16) +* Python Enhancement Proposals; PEP 425 <2>: Wheel vs Egg. (line 36) +* Python Enhancement Proposals; PEP 425 <3>: Platform compatibility tags. + (line 12) +* Python Enhancement Proposals; PEP 425 <4>: Platform tags for Windows. + (line 6) +* Python Enhancement Proposals; PEP 425 <5>: Platform tags for macOS Mac OS X. + (line 6) +* Python Enhancement Proposals; PEP 425 <6>: Platform tags for common Linux distributions. + (line 6) +* Python Enhancement Proposals; PEP 425 <7>: Platform tags for other *nix platforms. + (line 6) +* Python Enhancement Proposals; PEP 427: Wheel vs Egg. (line 12) +* Python Enhancement Proposals; PEP 427 <1>: Wheel vs Egg. (line 21) +* Python Enhancement Proposals; PEP 427 <2>: Binary distribution format. + (line 6) +* Python Enhancement Proposals; PEP 427 <3>: File format. (line 8) +* Python Enhancement Proposals; PEP 427 <4>: Glossary. (line 214) +* Python Enhancement Proposals; PEP 427#is-it-possible-to-import-python-code-directly-from-a-wheel-file: Wheel vs Egg. + (line 57) +* Python Enhancement Proposals; PEP 438: pip vs easy_install. (line 37) +* Python Enhancement Proposals; PEP 440: Installing from PyPI. + (line 9) +* Python Enhancement Proposals; PEP 440 <1>: Installing from PyPI. + (line 40) +* Python Enhancement Proposals; PEP 440 <2>: Creating setup py. + (line 49) +* Python Enhancement Proposals; PEP 440 <3>: python_requires. (line 7) +* Python Enhancement Proposals; PEP 440 <4>: Standards compliance for interoperability. + (line 8) +* Python Enhancement Proposals; PEP 440 <5>: Standards compliance for interoperability. + (line 24) +* Python Enhancement Proposals; PEP 440 <6>: Metadata-Version. + (line 14) +* Python Enhancement Proposals; PEP 440 <7>: Version. (line 9) +* Python Enhancement Proposals; PEP 440 <8>: Requires-External multiple use. + (line 22) +* Python Enhancement Proposals; PEP 440 <9>: Version specifiers. + (line 7) +* Python Enhancement Proposals; PEP 440 <10>: version<2>. (line 10) +* Python Enhancement Proposals; PEP 440 <11>: Glossary. (line 200) +* Python Enhancement Proposals; PEP 440#compatible-release: Installing from PyPI. + (line 25) +* Python Enhancement Proposals; PEP 440#compatible-release <1>: Semantic versioning preferred. + (line 20) +* Python Enhancement Proposals; PEP 440#local-version-identifiers: Local version identifiers. + (line 8) +* Python Enhancement Proposals; PEP 440#normalization: Standards compliance for interoperability. + (line 24) +* Python Enhancement Proposals; PEP 440#public-version-identifiers: Standards compliance for interoperability. + (line 8) +* Python Enhancement Proposals; PEP 440#version-specifiers: Installing from PyPI. + (line 10) +* Python Enhancement Proposals; PEP 440#version-specifiers <1>: Glossary. + (line 201) +* Python Enhancement Proposals; PEP 441: shiv. (line 9) +* Python Enhancement Proposals; PEP 453: Installation tool recommendations. + (line 30) +* Python Enhancement Proposals; PEP 453 <1>: Installation tool recommendations. + (line 34) +* Python Enhancement Proposals; PEP 453#rationale: Installation tool recommendations. + (line 33) +* Python Enhancement Proposals; PEP 503: Installing from other sources. + (line 7) +* Python Enhancement Proposals; PEP 503 <1>: name<2>. (line 12) +* Python Enhancement Proposals; PEP 503 <2>: Simple repository API. + (line 7) +* Python Enhancement Proposals; PEP 508: name. (line 9) +* Python Enhancement Proposals; PEP 508 <1>: Name. (line 8) +* Python Enhancement Proposals; PEP 508 <2>: Requires-Dist multiple use. + (line 30) +* Python Enhancement Proposals; PEP 508 <3>: Dependency specifiers. + (line 7) +* Python Enhancement Proposals; PEP 508 <4>: dependencies/optional-dependencies. + (line 6) +* Python Enhancement Proposals; PEP 508 <5>: dependencies/optional-dependencies. + (line 7) +* Python Enhancement Proposals; PEP 508 <6>: dependencies/optional-dependencies. + (line 17) +* Python Enhancement Proposals; PEP 508 <7>: dependencies/optional-dependencies. + (line 22) +* Python Enhancement Proposals; PEP 508 <8>: File format. (line 33) +* Python Enhancement Proposals; PEP 513: Platform Wheels. (line 20) +* Python Enhancement Proposals; PEP 513 <1>: Platform tags for common Linux distributions. + (line 11) +* Python Enhancement Proposals; PEP 513 <2>: Platform tags for common Linux distributions. + (line 20) +* Python Enhancement Proposals; PEP 517: Checking out the project and building distributions. + (line 21) +* Python Enhancement Proposals; PEP 517 <1>: Source distribution format. + (line 7) +* Python Enhancement Proposals; PEP 518: Checking out the project and building distributions. + (line 21) +* Python Enhancement Proposals; PEP 518 <1>: Declaring build system dependencies. + (line 7) +* Python Enhancement Proposals; PEP 518 <2>: Source distribution format. + (line 7) +* Python Enhancement Proposals; PEP 518 <3>: Glossary. (line 92) +* Python Enhancement Proposals; PEP 518 <4>: Glossary. (line 138) +* Python Enhancement Proposals; PEP 518 <5>: February 2017. (line 6) +* Python Enhancement Proposals; PEP 566: Core metadata specifications. + (line 6) +* Python Enhancement Proposals; PEP 566 <1>: Core metadata specifications. + (line 21) +* Python Enhancement Proposals; PEP 571: Platform tags for common Linux distributions. + (line 24) +* Python Enhancement Proposals; PEP 592: Simple repository API. + (line 9) +* Python Enhancement Proposals; PEP 599: Platform tags for common Linux distributions. + (line 28) +* Python Enhancement Proposals; PEP 621: Declaring project metadata. + (line 6) +* Python Enhancement Proposals; PEP 627: History and change workflow. + (line 7) +* Python Package Index (PyPI): Glossary. (line 121) +* Python Packaging Authority (PyPA): Glossary. (line 113) +* Release: Glossary. (line 138) +* Requirement: Glossary. (line 148) +* Requirement Specifier: Glossary. (line 155) +* Requirements File: Glossary. (line 164) +* setup.cfg: Glossary. (line 173) +* setup.py: Glossary. (line 170) +* Source Archive: Glossary. (line 177) +* Source Distribution (or "sdist"): Glossary. (line 183) +* System Package: Glossary. (line 190) +* Version Specifier: Glossary. (line 195) +* Virtual Environment: Glossary. (line 203) +* Wheel: Glossary. (line 210) +* Working Set: Glossary. (line 216) + + + +Tag Table: +Node: Top410 +Ref: index doc689 +Ref: 0689 +Node: An Overview of Packaging for Python14966 +Ref: overview doc15085 +Ref: 115085 +Ref: overview an-overview-of-packaging-for-python15085 +Ref: 215085 +Ref: overview python-packaging-user-guide15085 +Ref: 315085 +Node: Thinking about deployment16018 +Ref: overview thinking-about-deployment16174 +Ref: 416174 +Node: Packaging Python libraries and tools17105 +Ref: overview packaging-python-libraries-and-tools17299 +Ref: 517299 +Node: Python modules18015 +Ref: overview python-modules18152 +Ref: 818152 +Ref: Python modules-Footnote-118896 +Ref: Python modules-Footnote-218935 +Node: Python source distributions19015 +Ref: overview python-source-distributions19188 +Ref: 919188 +Ref: Python source distributions-Footnote-121021 +Ref: Python source distributions-Footnote-221081 +Ref: Python source distributions-Footnote-321127 +Ref: Python source distributions-Footnote-421166 +Ref: Python source distributions-Footnote-521205 +Ref: Python source distributions-Footnote-621243 +Node: Python binary distributions21284 +Ref: overview python-binary-distributions21434 +Ref: c21434 +Ref: Python binary distributions-Footnote-122899 +Node: Packaging Python applications22951 +Ref: overview packaging-applications23133 +Ref: 723133 +Ref: overview packaging-python-applications23133 +Ref: f23133 +Ref: Packaging Python applications-Footnote-124504 +Node: Depending on a framework24584 +Ref: overview depending-on-a-framework24732 +Ref: 1024732 +Node: Service platforms25669 +Ref: overview service-platforms25806 +Ref: 1125806 +Ref: Service platforms-Footnote-126559 +Ref: Service platforms-Footnote-226633 +Ref: Service platforms-Footnote-326689 +Ref: Service platforms-Footnote-426729 +Ref: Service platforms-Footnote-526788 +Node: Web browsers and mobile applications26818 +Ref: overview web-browsers-and-mobile-applications26955 +Ref: 1226955 +Ref: Web browsers and mobile applications-Footnote-127705 +Ref: Web browsers and mobile applications-Footnote-227736 +Ref: Web browsers and mobile applications-Footnote-327763 +Ref: Web browsers and mobile applications-Footnote-427793 +Node: Depending on a pre-installed Python27840 +Ref: overview depending-on-a-pre-installed-python28052 +Ref: 1328052 +Ref: Depending on a pre-installed Python-Footnote-129110 +Ref: Depending on a pre-installed Python-Footnote-229156 +Ref: Depending on a pre-installed Python-Footnote-329210 +Node: Depending on a separate software distribution ecosystem29256 +Ref: overview depending-on-a-separate-ecosystem29479 +Ref: 1429479 +Ref: overview depending-on-a-separate-software-distribution-ecosystem29479 +Ref: 1529479 +Ref: Depending on a separate software distribution ecosystem-Footnote-130693 +Ref: Depending on a separate software distribution ecosystem-Footnote-230718 +Ref: Depending on a separate software distribution ecosystem-Footnote-330787 +Ref: Depending on a separate software distribution ecosystem-Footnote-430870 +Ref: Depending on a separate software distribution ecosystem-Footnote-530932 +Ref: Depending on a separate software distribution ecosystem-Footnote-631009 +Ref: Depending on a separate software distribution ecosystem-Footnote-731059 +Ref: Depending on a separate software distribution ecosystem-Footnote-831108 +Node: Bringing your own Python executable31144 +Ref: overview bringing-your-own-python31359 +Ref: 1631359 +Ref: overview bringing-your-own-python-executable31359 +Ref: 1731359 +Ref: Bringing your own Python executable-Footnote-132475 +Ref: Bringing your own Python executable-Footnote-232511 +Ref: Bringing your own Python executable-Footnote-332563 +Ref: Bringing your own Python executable-Footnote-432608 +Ref: Bringing your own Python executable-Footnote-532639 +Ref: Bringing your own Python executable-Footnote-632688 +Ref: Bringing your own Python executable-Footnote-732730 +Ref: Bringing your own Python executable-Footnote-832773 +Ref: Bringing your own Python executable-Footnote-932815 +Node: Bringing your own userspace32864 +Ref: overview bringing-your-own-userspace33048 +Ref: 1833048 +Ref: Bringing your own userspace-Footnote-133751 +Ref: Bringing your own userspace-Footnote-233828 +Ref: Bringing your own userspace-Footnote-333858 +Ref: Bringing your own userspace-Footnote-433910 +Ref: Bringing your own userspace-Footnote-533939 +Node: Bringing your own kernel33969 +Ref: overview bringing-your-own-kernel34144 +Ref: 1934144 +Ref: Bringing your own kernel-Footnote-134856 +Ref: Bringing your own kernel-Footnote-234891 +Ref: Bringing your own kernel-Footnote-334947 +Ref: Bringing your own kernel-Footnote-435006 +Ref: Bringing your own kernel-Footnote-535072 +Node: Bringing your own hardware35123 +Ref: overview bringing-your-own-hardware35262 +Ref: 1a35262 +Ref: Bringing your own hardware-Footnote-136092 +Ref: Bringing your own hardware-Footnote-236165 +Node: What about…36198 +Ref: overview what-about36351 +Ref: 1b36351 +Node: Operating system packages36570 +Ref: overview operating-system-packages36678 +Ref: 1c36678 +Ref: Operating system packages-Footnote-137364 +Ref: Operating system packages-Footnote-237420 +Ref: Operating system packages-Footnote-337478 +Node: virtualenv37546 +Ref: overview virtualenv37671 +Ref: 1d37671 +Ref: virtualenv-Footnote-138291 +Ref: virtualenv-Footnote-238361 +Ref: virtualenv-Footnote-338426 +Node: Security38469 +Ref: overview security38560 +Ref: 1e38560 +Ref: Security-Footnote-139178 +Node: Wrap up39256 +Ref: overview wrap-up39371 +Ref: 1f39371 +Node: Tutorials39761 +Ref: tutorials/index doc39883 +Ref: 2039883 +Ref: tutorials/index tutorials39883 +Ref: 2139883 +Node: Installing Packages40219 +Ref: tutorials/installing-packages doc40340 +Ref: 2340340 +Ref: tutorials/installing-packages id140340 +Ref: 2440340 +Ref: tutorials/installing-packages installing-packages40340 +Ref: 2540340 +Node: Requirements for Installing Packages41523 +Ref: tutorials/installing-packages installing-requirements41667 +Ref: 2641667 +Ref: tutorials/installing-packages requirements-for-installing-packages41667 +Ref: 2741667 +Node: Ensure you can run Python from the command line42135 +Ref: tutorials/installing-packages ensure-you-can-run-python-from-the-command-line42322 +Ref: 2842322 +Ref: Ensure you can run Python from the command line-Footnote-144601 +Ref: Ensure you can run Python from the command line-Footnote-244628 +Ref: Ensure you can run Python from the command line-Footnote-344698 +Node: Ensure you can run pip from the command line44804 +Ref: tutorials/installing-packages ensure-you-can-run-pip-from-the-command-line45046 +Ref: 2945046 +Ref: tutorials/installing-packages python-org45046 +Ref: 2a45046 +Ref: Ensure you can run pip from the command line-Footnote-146545 +Ref: Ensure you can run pip from the command line-Footnote-246572 +Ref: Ensure you can run pip from the command line-Footnote-346596 +Ref: Ensure you can run pip from the command line-Footnote-446641 +Ref: Ensure you can run pip from the command line-Footnote-546802 +Node: Ensure pip setuptools and wheel are up to date47032 +Ref: tutorials/installing-packages ensure-pip-setuptools-and-wheel-are-up-to-date47266 +Ref: 2f47266 +Node: Optionally create a virtual environment47651 +Ref: tutorials/installing-packages optionally-create-a-virtual-environment47832 +Ref: 3047832 +Ref: Optionally create a virtual environment-Footnote-148327 +Ref: Optionally create a virtual environment-Footnote-248379 +Node: Creating Virtual Environments48607 +Ref: tutorials/installing-packages creating-and-using-virtual-environments48782 +Ref: 3148782 +Ref: tutorials/installing-packages creating-virtual-environments48782 +Ref: 3348782 +Ref: Creating Virtual Environments-Footnote-151559 +Ref: Creating Virtual Environments-Footnote-251611 +Ref: Creating Virtual Environments-Footnote-351663 +Ref: Creating Virtual Environments-Footnote-451715 +Node: Use pip for Installing51749 +Ref: tutorials/installing-packages use-pip-for-installing51908 +Ref: 3751908 +Ref: Use pip for Installing-Footnote-152189 +Ref: Use pip for Installing-Footnote-252217 +Node: Installing from PyPI52276 +Ref: tutorials/installing-packages installing-from-pypi52436 +Ref: 3852436 +Ref: Installing from PyPI-Footnote-153382 +Ref: Installing from PyPI-Footnote-253431 +Ref: Installing from PyPI-Footnote-353499 +Ref: Installing from PyPI-Footnote-453567 +Node: Source Distributions vs Wheels53758 +Ref: tutorials/installing-packages source-distributions-vs-wheels53914 +Ref: 3c53914 +Node: Upgrading packages54551 +Ref: tutorials/installing-packages upgrading-packages54714 +Ref: 3e54714 +Node: Installing to the User Site54878 +Ref: tutorials/installing-packages id655029 +Ref: 3f55029 +Ref: tutorials/installing-packages installing-to-the-user-site55029 +Ref: 4055029 +Ref: Installing to the User Site-Footnote-156924 +Ref: Installing to the User Site-Footnote-257000 +Ref: Installing to the User Site-Footnote-357067 +Ref: Installing to the User Site-Footnote-457112 +Node: Requirements files57201 +Ref: tutorials/installing-packages control-panel57353 +Ref: 4157353 +Ref: tutorials/installing-packages requirements-files57353 +Ref: 4257353 +Ref: Requirements files-Footnote-157547 +Node: Installing from VCS57616 +Ref: tutorials/installing-packages installing-from-vcs57770 +Ref: 4357770 +Ref: Installing from VCS-Footnote-158359 +Node: Installing from other Indexes58432 +Ref: tutorials/installing-packages installing-from-other-indexes58600 +Ref: 4458600 +Node: Installing from a local src tree58934 +Ref: tutorials/installing-packages installing-from-a-local-src-tree59113 +Ref: 4559113 +Ref: Installing from a local src tree-Footnote-159481 +Node: Installing from local archives59567 +Ref: tutorials/installing-packages installing-from-local-archives59746 +Ref: 4659746 +Node: Installing from other sources60214 +Ref: tutorials/installing-packages installing-from-other-sources60383 +Ref: 4760383 +Ref: Installing from other sources-Footnote-160833 +Node: Installing Prereleases60882 +Ref: tutorials/installing-packages installing-prereleases61055 +Ref: 4861055 +Node: Installing Setuptools “Extras”61273 +Ref: tutorials/installing-packages installing-setuptools-extras61408 +Ref: 4961408 +Ref: Installing Setuptools “Extras”-Footnote-161777 +Node: Managing Application Dependencies61909 +Ref: tutorials/managing-dependencies doc62064 +Ref: 4a62064 +Ref: tutorials/managing-dependencies managing-application-dependencies62064 +Ref: 4b62064 +Ref: tutorials/managing-dependencies managing-dependencies62064 +Ref: 3562064 +Ref: tutorials/managing-dependencies setuptools-extras62064 +Ref: 4c62064 +Ref: Managing Application Dependencies-Footnote-163547 +Node: Installing Pipenv63595 +Ref: tutorials/managing-dependencies installing-pipenv63741 +Ref: 4d63741 +Ref: tutorials/managing-dependencies pipenv-user-base64233 +Ref: 4e64233 +Ref: Installing Pipenv-Footnote-164594 +Ref: Installing Pipenv-Footnote-264625 +Ref: Installing Pipenv-Footnote-364652 +Ref: Installing Pipenv-Footnote-464716 +Node: Installing packages for your project64783 +Ref: tutorials/managing-dependencies installing-packages-for-your-project64962 +Ref: 4f64962 +Ref: tutorials/managing-dependencies user-installation64962 +Ref: 5064962 +Ref: Installing packages for your project-Footnote-166897 +Node: Using installed packages66940 +Ref: tutorials/managing-dependencies requests67112 +Ref: 5267112 +Ref: tutorials/managing-dependencies using-installed-packages67112 +Ref: 5367112 +Node: Next steps67778 +Ref: tutorials/managing-dependencies next-steps67963 +Ref: 5467963 +Node: Other Tools for Application Dependency Management68579 +Ref: tutorials/managing-dependencies other-dependency-management-tools68731 +Ref: 5668731 +Ref: tutorials/managing-dependencies other-tools-for-application-dependency-management68731 +Ref: 5768731 +Ref: Other Tools for Application Dependency Management-Footnote-170156 +Ref: Other Tools for Application Dependency Management-Footnote-270204 +Ref: Other Tools for Application Dependency Management-Footnote-370242 +Ref: Other Tools for Application Dependency Management-Footnote-470288 +Node: Packaging Python Projects70341 +Ref: tutorials/packaging-projects doc70499 +Ref: e70499 +Ref: tutorials/packaging-projects packaging-python-projects70499 +Ref: 5870499 +Node: A simple project71137 +Ref: tutorials/packaging-projects a-simple-project71264 +Ref: 5971264 +Ref: A simple project-Footnote-172179 +Node: Creating the package files72244 +Ref: tutorials/packaging-projects creating-the-package-files72402 +Ref: 5a72402 +Ref: tutorials/packaging-projects python-documentation-for-packages-and-modules72402 +Ref: 5b72402 +Node: Creating a test folder72888 +Ref: tutorials/packaging-projects creating-a-test-folder73047 +Ref: 5c73047 +Node: Creating setup py73183 +Ref: tutorials/packaging-projects creating-setup-py73334 +Ref: 5d73334 +Ref: Creating setup py-Footnote-177052 +Node: Creating README md77101 +Ref: tutorials/packaging-projects creating-readme-md77248 +Ref: 5e77248 +Node: Creating a LICENSE77587 +Ref: tutorials/packaging-projects creating-a-license77749 +Ref: 5f77749 +Node: Generating distribution archives79339 +Ref: tutorials/packaging-projects generating-archives79518 +Ref: 6079518 +Ref: tutorials/packaging-projects generating-distribution-archives79518 +Ref: 6179518 +Ref: Generating distribution archives-Footnote-181064 +Node: Uploading the distribution archives81180 +Ref: tutorials/packaging-projects uploading-the-distribution-archives81379 +Ref: 6481379 +Ref: Uploading the distribution archives-Footnote-183677 +Ref: Uploading the distribution archives-Footnote-283722 +Node: Installing your newly uploaded package83767 +Ref: tutorials/packaging-projects installing-your-newly-uploaded-package83947 +Ref: 6783947 +Node: Next steps<2>85733 +Ref: tutorials/packaging-projects next-steps85869 +Ref: 6885869 +Ref: Next steps<2>-Footnote-187572 +Ref: Next steps<2>-Footnote-287610 +Node: Creating Documentation87658 +Ref: tutorials/creating-documentation doc87774 +Ref: 6b87774 +Ref: tutorials/creating-documentation creating-documentation87774 +Ref: 6c87774 +Ref: tutorials/creating-documentation id187774 +Ref: 6d87774 +Ref: tutorials/creating-documentation poetry87774 +Ref: 6e87774 +Ref: Creating Documentation-Footnote-188083 +Ref: Creating Documentation-Footnote-288114 +Node: Installing Sphinx88147 +Ref: tutorials/creating-documentation installing-sphinx88273 +Ref: 6f88273 +Ref: tutorials/creating-documentation read-the-docs88273 +Ref: 7088273 +Ref: Installing Sphinx-Footnote-188495 +Node: Getting Started With Sphinx88563 +Ref: tutorials/creating-documentation getting-started-with-sphinx88711 +Ref: 7188711 +Ref: tutorials/creating-documentation installation-guide88711 +Ref: 7288711 +Ref: Getting Started With Sphinx-Footnote-189362 +Node: Other Sources89432 +Ref: tutorials/creating-documentation guide89554 +Ref: 7389554 +Ref: tutorials/creating-documentation other-sources89554 +Ref: 7489554 +Ref: Other Sources-Footnote-189778 +Node: Guides89839 +Ref: guides/index doc89937 +Ref: 2289937 +Ref: guides/index documentation-tutorial89937 +Ref: 7589937 +Ref: guides/index guides89937 +Ref: 7689937 +Node: Tool recommendations91174 +Ref: guides/tool-recommendations doc91314 +Ref: 7791314 +Ref: guides/tool-recommendations id191314 +Ref: 7891314 +Ref: guides/tool-recommendations tool-recommendations91314 +Ref: 7991314 +Node: Application dependency management91659 +Ref: guides/tool-recommendations application-dependency-management91805 +Ref: 7a91805 +Ref: Application dependency management-Footnote-192234 +Ref: Application dependency management-Footnote-292280 +Node: Installation tool recommendations92315 +Ref: guides/tool-recommendations installation-tool-recommendations92500 +Ref: 7b92500 +Ref: Installation tool recommendations-Footnote-193318 +Ref: Installation tool recommendations-Footnote-293573 +Ref: Installation tool recommendations-Footnote-393918 +Ref: Installation tool recommendations-Footnote-494244 +Ref: Installation tool recommendations-Footnote-594296 +Node: Packaging tool recommendations94583 +Ref: guides/tool-recommendations packaging-tool-recommendations94764 +Ref: 8494764 +Ref: Packaging tool recommendations-Footnote-195275 +Ref: Packaging tool recommendations-Footnote-296105 +Ref: Packaging tool recommendations-Footnote-396306 +Node: Publishing platform migration96345 +Ref: guides/tool-recommendations publishing-platform-migration96484 +Ref: 8696484 +Ref: Publishing platform migration-Footnote-196971 +Ref: Publishing platform migration-Footnote-297003 +Node: Installing packages using pip and virtual environments97028 +Ref: guides/installing-using-pip-and-virtual-environments doc97218 +Ref: 8897218 +Ref: guides/installing-using-pip-and-virtual-environments installing-packages-using-pip-and-virtual-environments97218 +Ref: 8997218 +Ref: guides/installing-using-pip-and-virtual-environments venv97218 +Ref: 8a97218 +Node: Installing pip98378 +Ref: guides/installing-using-pip-and-virtual-environments installing-pip98527 +Ref: 8b98527 +Node: Windows98788 +Ref: guides/installing-using-pip-and-virtual-environments windows98884 +Ref: 8c98884 +Node: Linux and macOS99189 +Ref: guides/installing-using-pip-and-virtual-environments linux-and-macos99285 +Ref: 8d99285 +Ref: Linux and macOS-Footnote-199966 +Node: Installing virtualenv100020 +Ref: guides/installing-using-pip-and-virtual-environments installing-virtualenv100208 +Ref: 8e100208 +Ref: guides/installing-using-pip-and-virtual-environments python-pip100208 +Ref: 8f100208 +Ref: Installing virtualenv-Footnote-1100962 +Node: Creating a virtual environment101026 +Ref: guides/installing-using-pip-and-virtual-environments creating-a-virtual-environment101232 +Ref: 90101232 +Node: Activating a virtual environment102458 +Ref: guides/installing-using-pip-and-virtual-environments activating-a-virtual-environment102674 +Ref: 91102674 +Node: Leaving the virtual environment103561 +Ref: guides/installing-using-pip-and-virtual-environments leaving-the-virtual-environment103766 +Ref: 92103766 +Node: Installing packages104135 +Ref: guides/installing-using-pip-and-virtual-environments installing-packages104336 +Ref: 93104336 +Ref: Installing packages-Footnote-1105377 +Node: Installing specific versions105420 +Ref: guides/installing-using-pip-and-virtual-environments installing-specific-versions105607 +Ref: 94105607 +Ref: guides/installing-using-pip-and-virtual-environments requests105607 +Ref: 95105607 +Node: Installing extras106077 +Ref: guides/installing-using-pip-and-virtual-environments installing-extras106267 +Ref: 96106267 +Ref: Installing extras-Footnote-1106501 +Node: Installing from source106633 +Ref: guides/installing-using-pip-and-virtual-environments extras106834 +Ref: 97106834 +Ref: guides/installing-using-pip-and-virtual-environments installing-from-source106834 +Ref: 98106834 +Ref: Installing from source-Footnote-1107260 +Node: Installing from version control systems107346 +Ref: guides/installing-using-pip-and-virtual-environments development-mode107563 +Ref: 99107563 +Ref: guides/installing-using-pip-and-virtual-environments installing-from-version-control-systems107563 +Ref: 9a107563 +Ref: Installing from version control systems-Footnote-1108037 +Node: Installing from local archives<2>108110 +Ref: guides/installing-using-pip-and-virtual-environments installing-from-local-archives108332 +Ref: 9b108332 +Node: Using other package indexes108987 +Ref: guides/installing-using-pip-and-virtual-environments using-other-package-indexes109191 +Ref: 9c109191 +Node: Upgrading packages<2>109723 +Ref: guides/installing-using-pip-and-virtual-environments upgrading-packages109918 +Ref: 9d109918 +Node: Using requirements files110166 +Ref: guides/installing-using-pip-and-virtual-environments using-requirements-files110355 +Ref: 9e110355 +Ref: Using requirements files-Footnote-1110812 +Node: Freezing dependencies110881 +Ref: guides/installing-using-pip-and-virtual-environments freezing-dependencies111040 +Ref: 9f111040 +Ref: Freezing dependencies-Footnote-1111672 +Node: Installing stand alone command line tools111741 +Ref: guides/installing-stand-alone-command-line-tools doc111970 +Ref: 34111970 +Ref: guides/installing-stand-alone-command-line-tools installing-stand-alone-command-line-tools111970 +Ref: a0111970 +Ref: Installing stand alone command line tools-Footnote-1114827 +Ref: Installing stand alone command line tools-Footnote-2114866 +Ref: Installing stand alone command line tools-Footnote-3114906 +Ref: Installing stand alone command line tools-Footnote-4114945 +Ref: Installing stand alone command line tools-Footnote-5114983 +Node: Installing pip/setuptools/wheel with Linux Package Managers115027 +Ref: guides/installing-using-linux-tools doc115232 +Ref: 2c115232 +Ref: guides/installing-using-linux-tools id1115232 +Ref: a1115232 +Ref: guides/installing-using-linux-tools installing-pip-setuptools-wheel-with-linux-package-managers115232 +Ref: a2115232 +Ref: Installing pip/setuptools/wheel with Linux Package Managers-Footnote-1116583 +Node: Fedora116614 +Ref: guides/installing-using-linux-tools fedora116750 +Ref: a3116750 +Ref: Fedora-Footnote-1117485 +Ref: Fedora-Footnote-2117541 +Node: CentOS/RHEL117600 +Ref: guides/installing-using-linux-tools centos-rhel117753 +Ref: a4117753 +Ref: CentOS/RHEL-Footnote-1119513 +Ref: CentOS/RHEL-Footnote-2119557 +Ref: CentOS/RHEL-Footnote-3119640 +Ref: CentOS/RHEL-Footnote-4119696 +Ref: CentOS/RHEL-Footnote-5119755 +Ref: CentOS/RHEL-Footnote-6119908 +Ref: CentOS/RHEL-Footnote-7119947 +Node: openSUSE120009 +Ref: guides/installing-using-linux-tools opensuse120169 +Ref: a5120169 +Node: Debian/Ubuntu120381 +Ref: guides/installing-using-linux-tools debian-ubuntu120540 +Ref: a6120540 +Ref: Debian/Ubuntu-Footnote-1120931 +Node: Arch Linux120995 +Ref: guides/installing-using-linux-tools arch-linux121137 +Ref: a7121137 +Node: Installing scientific packages121347 +Ref: guides/installing-scientific-packages doc121533 +Ref: a8121533 +Ref: guides/installing-scientific-packages installing-scientific-packages121533 +Ref: a9121533 +Ref: guides/installing-scientific-packages numpy-and-the-science-stack121533 +Ref: aa121533 +Ref: Installing scientific packages-Footnote-1122992 +Ref: Installing scientific packages-Footnote-2123022 +Ref: Installing scientific packages-Footnote-3123076 +Node: Building from source123113 +Ref: guides/installing-scientific-packages building-from-source123250 +Ref: ab123250 +Node: Linux distribution packages123649 +Ref: guides/installing-scientific-packages linux-distribution-packages123813 +Ref: ac123813 +Node: Windows installers124293 +Ref: guides/installing-scientific-packages windows-installers124474 +Ref: ad124474 +Ref: Windows installers-Footnote-1125725 +Node: macOS installers and package managers125776 +Ref: guides/installing-scientific-packages mac-os-x-installers-and-package-managers125949 +Ref: ae125949 +Ref: guides/installing-scientific-packages macos-installers-and-package-managers125949 +Ref: af125949 +Ref: macOS installers and package managers-Footnote-1126438 +Node: SciPy distributions126493 +Ref: guides/installing-scientific-packages scipy-distributions126653 +Ref: b0126653 +Ref: SciPy distributions-Footnote-1126990 +Node: Spack127032 +Ref: guides/installing-scientific-packages spack127195 +Ref: b1127195 +Ref: Spack-Footnote-1128241 +Node: The conda cross-platform package manager128280 +Ref: guides/installing-scientific-packages the-conda-cross-platform-package-manager128415 +Ref: b2128415 +Ref: The conda cross-platform package manager-Footnote-1129771 +Node: Multi-version installs129814 +Ref: guides/multi-version-installs doc129976 +Ref: b3129976 +Ref: guides/multi-version-installs id1129976 +Ref: b4129976 +Ref: guides/multi-version-installs multi-version-installs129976 +Ref: b5129976 +Ref: Multi-version installs-Footnote-1131886 +Node: Packaging and distributing projects131977 +Ref: guides/distributing-packages-using-setuptools doc132165 +Ref: 6132165 +Ref: guides/distributing-packages-using-setuptools distributing-packages132165 +Ref: 55132165 +Ref: guides/distributing-packages-using-setuptools packaging-and-distributing-projects132165 +Ref: b6132165 +Ref: Packaging and distributing projects-Footnote-1133133 +Node: Requirements for packaging and distributing133201 +Ref: guides/distributing-packages-using-setuptools requirements-for-packaging-and-distributing133363 +Ref: b7133363 +Ref: Requirements for packaging and distributing-Footnote-1133787 +Node: Configuring your project134017 +Ref: guides/distributing-packages-using-setuptools configuring-your-project134221 +Ref: b9134221 +Node: Initial files134373 +Ref: guides/distributing-packages-using-setuptools initial-files134480 +Ref: ba134480 +Node: setup py134706 +Ref: guides/distributing-packages-using-setuptools setup-py134796 +Ref: bb134796 +Ref: setup py-Footnote-1135627 +Ref: setup py-Footnote-2135694 +Node: setup cfg135740 +Ref: guides/distributing-packages-using-setuptools setup-cfg135861 +Ref: bd135861 +Ref: setup cfg-Footnote-1136094 +Ref: setup cfg-Footnote-2136162 +Node: README rst / README md136208 +Ref: guides/distributing-packages-using-setuptools readme-rst-readme-md136332 +Ref: be136332 +Ref: README rst / README md-Footnote-1137349 +Ref: README rst / README md-Footnote-2137398 +Ref: README rst / README md-Footnote-3137452 +Ref: README rst / README md-Footnote-4137520 +Node: MANIFEST in137566 +Ref: guides/distributing-packages-using-setuptools manifest-in137692 +Ref: c1137692 +Ref: MANIFEST in-Footnote-1138244 +Ref: MANIFEST in-Footnote-2138314 +Node: LICENSE txt138360 +Ref: guides/distributing-packages-using-setuptools license-txt138478 +Ref: c3138478 +Ref: LICENSE txt-Footnote-1138980 +Ref: LICENSE txt-Footnote-2139016 +Ref: LICENSE txt-Footnote-3139086 +Node: 139132 +Ref: guides/distributing-packages-using-setuptools your-package139230 +Ref: c4139230 +Ref: -Footnote-1139617 +Ref: -Footnote-2139686 +Node: setup args139732 +Ref: guides/distributing-packages-using-setuptools id11139876 +Ref: c6139876 +Ref: guides/distributing-packages-using-setuptools setup-args139876 +Ref: bc139876 +Ref: setup args-Footnote-1140565 +Ref: setup args-Footnote-2140632 +Node: name140678 +Ref: guides/distributing-packages-using-setuptools name140759 +Ref: c7140759 +Ref: guides/distributing-packages-using-setuptools setup-name140759 +Ref: c5140759 +Ref: name-Footnote-1141518 +Node: version141567 +Ref: guides/distributing-packages-using-setuptools version141668 +Ref: c8141668 +Node: description142476 +Ref: guides/distributing-packages-using-setuptools description142576 +Ref: bf142576 +Ref: guides/distributing-packages-using-setuptools id14142576 +Ref: cb142576 +Ref: description-Footnote-1143722 +Ref: description-Footnote-2143824 +Ref: description-Footnote-3143916 +Node: url143970 +Ref: guides/distributing-packages-using-setuptools url144069 +Ref: cc144069 +Node: author144186 +Ref: guides/distributing-packages-using-setuptools author144281 +Ref: cd144281 +Node: license144425 +Ref: guides/distributing-packages-using-setuptools license144528 +Ref: ce144528 +Node: classifiers145245 +Ref: guides/distributing-packages-using-setuptools classifiers145350 +Ref: cf145350 +Node: keywords146829 +Ref: guides/distributing-packages-using-setuptools keywords146939 +Ref: d1146939 +Node: project_urls147067 +Ref: guides/distributing-packages-using-setuptools project-urls147174 +Ref: d2147174 +Node: packages147803 +Ref: guides/distributing-packages-using-setuptools packages147912 +Ref: d3147912 +Node: py_modules148423 +Ref: guides/distributing-packages-using-setuptools py-modules148536 +Ref: d4148536 +Node: install_requires148838 +Ref: guides/distributing-packages-using-setuptools install-requires148958 +Ref: d5148958 +Node: python_requires149365 +Ref: guides/distributing-packages-using-setuptools id16149487 +Ref: d7149487 +Ref: guides/distributing-packages-using-setuptools python-requires149487 +Ref: d0149487 +Ref: python_requires-Footnote-1150847 +Node: package_data150896 +Ref: guides/distributing-packages-using-setuptools id17151012 +Ref: d9151012 +Ref: guides/distributing-packages-using-setuptools package-data151012 +Ref: da151012 +Ref: package_data-Footnote-1151744 +Ref: package_data-Footnote-2151834 +Node: data_files151876 +Ref: guides/distributing-packages-using-setuptools data-files151984 +Ref: db151984 +Ref: guides/distributing-packages-using-setuptools id18151984 +Ref: dc151984 +Ref: data_files-Footnote-1153365 +Ref: data_files-Footnote-2153413 +Node: scripts153502 +Ref: guides/distributing-packages-using-setuptools scripts153610 +Ref: dd153610 +Ref: scripts-Footnote-1153908 +Node: entry_points153988 +Ref: guides/distributing-packages-using-setuptools entry-points154077 +Ref: df154077 +Ref: entry_points-Footnote-1154566 +Node: console_scripts154688 +Ref: guides/distributing-packages-using-setuptools console-scripts154766 +Ref: de154766 +Ref: guides/distributing-packages-using-setuptools id20154766 +Ref: e0154766 +Ref: console_scripts-Footnote-1155303 +Ref: console_scripts-Footnote-2155414 +Ref: console_scripts-Footnote-3155758 +Ref: console_scripts-Footnote-4155853 +Node: Choosing a versioning scheme155895 +Ref: guides/distributing-packages-using-setuptools choosing-a-versioning-scheme156017 +Ref: c9156017 +Ref: guides/distributing-packages-using-setuptools id24156017 +Ref: e1156017 +Node: Standards compliance for interoperability156226 +Ref: guides/distributing-packages-using-setuptools standards-compliance-for-interoperability156369 +Ref: e2156369 +Ref: Standards compliance for interoperability-Footnote-1157395 +Ref: Standards compliance for interoperability-Footnote-2157472 +Ref: Standards compliance for interoperability-Footnote-3157521 +Ref: Standards compliance for interoperability-Footnote-4157570 +Node: Scheme choices157633 +Ref: guides/distributing-packages-using-setuptools scheme-choices157807 +Ref: e3157807 +Node: Semantic versioning preferred157999 +Ref: guides/distributing-packages-using-setuptools semantic-versioning-preferred158123 +Ref: e4158123 +Ref: Semantic versioning preferred-Footnote-1159117 +Ref: Semantic versioning preferred-Footnote-2159143 +Ref: Semantic versioning preferred-Footnote-3159211 +Node: Date based versioning159237 +Ref: guides/distributing-packages-using-setuptools date-based-versioning159387 +Ref: e5159387 +Node: Serial versioning159962 +Ref: guides/distributing-packages-using-setuptools serial-versioning160097 +Ref: e6160097 +Node: Hybrid schemes160474 +Ref: guides/distributing-packages-using-setuptools hybrid-schemes160579 +Ref: e7160579 +Node: Pre-release versioning160934 +Ref: guides/distributing-packages-using-setuptools pre-release-versioning161092 +Ref: e8161092 +Node: Local version identifiers161678 +Ref: guides/distributing-packages-using-setuptools local-version-identifiers161813 +Ref: e9161813 +Ref: Local version identifiers-Footnote-1162530 +Node: Working in “development mode”162606 +Ref: guides/distributing-packages-using-setuptools working-in-development-mode162789 +Ref: ea162789 +Ref: Working in “development mode”-Footnote-1164811 +Ref: Working in “development mode”-Footnote-2164880 +Ref: Working in “development mode”-Footnote-3164953 +Ref: Working in “development mode”-Footnote-4165039 +Node: Packaging your project165081 +Ref: guides/distributing-packages-using-setuptools id26165270 +Ref: eb165270 +Ref: guides/distributing-packages-using-setuptools packaging-your-project165270 +Ref: d8165270 +Node: Source distributions165559 +Ref: guides/distributing-packages-using-setuptools source-distributions165667 +Ref: ed165667 +Node: Wheels166122 +Ref: guides/distributing-packages-using-setuptools wheels166230 +Ref: ee166230 +Node: Universal Wheels167239 +Ref: guides/distributing-packages-using-setuptools id27167339 +Ref: f2167339 +Ref: guides/distributing-packages-using-setuptools universal-wheels167339 +Ref: ef167339 +Node: Pure Python Wheels168276 +Ref: guides/distributing-packages-using-setuptools id28168400 +Ref: f3168400 +Ref: guides/distributing-packages-using-setuptools pure-python-wheels168400 +Ref: f0168400 +Ref: Pure Python Wheels-Footnote-1169245 +Ref: Pure Python Wheels-Footnote-2169294 +Node: Platform Wheels169346 +Ref: guides/distributing-packages-using-setuptools id29169445 +Ref: f4169445 +Ref: guides/distributing-packages-using-setuptools platform-wheels169445 +Ref: f1169445 +Ref: Platform Wheels-Footnote-1170153 +Ref: Platform Wheels-Footnote-2170202 +Node: Uploading your Project to PyPI170251 +Ref: guides/distributing-packages-using-setuptools id30170398 +Ref: f5170398 +Ref: guides/distributing-packages-using-setuptools uploading-your-project-to-pypi170398 +Ref: b8170398 +Ref: Uploading your Project to PyPI-Footnote-1172215 +Ref: Uploading your Project to PyPI-Footnote-2172246 +Node: Create an account172294 +Ref: guides/distributing-packages-using-setuptools create-an-account172426 +Ref: f7172426 +Ref: guides/distributing-packages-using-setuptools register-your-project173348 +Ref: f9173348 +Ref: Create an account-Footnote-1173385 +Ref: Create an account-Footnote-2173428 +Ref: Create an account-Footnote-3173468 +Node: Upload your distributions173508 +Ref: guides/distributing-packages-using-setuptools api-token173640 +Ref: fa173640 +Ref: guides/distributing-packages-using-setuptools upload-your-distributions173640 +Ref: fb173640 +Node: Including files in source distributions with MANIFEST in174523 +Ref: guides/using-manifest-in doc174724 +Ref: fc174724 +Ref: guides/using-manifest-in including-files-in-source-distributions-with-manifest-in174724 +Ref: fd174724 +Ref: guides/using-manifest-in using-manifest-in174724 +Ref: c2174724 +Node: How files are included in an sdist175722 +Ref: guides/using-manifest-in how-files-are-included-in-an-sdist175892 +Ref: fe175892 +Node: MANIFEST in commands177578 +Ref: guides/using-manifest-in manifest-in-commands177748 +Ref: ff177748 +Node: Single-sourcing the package version181763 +Ref: guides/single-sourcing-package-version doc181964 +Ref: 100181964 +Ref: guides/single-sourcing-package-version single-sourcing-the-package-version181964 +Ref: 101181964 +Ref: guides/single-sourcing-package-version single-sourcing-the-version181964 +Ref: ca181964 +Ref: Single-sourcing the package version-Footnote-1187351 +Ref: Single-sourcing the package version-Footnote-2187412 +Ref: Single-sourcing the package version-Footnote-3187458 +Ref: Single-sourcing the package version-Footnote-4187499 +Ref: Single-sourcing the package version-Footnote-5187546 +Ref: Single-sourcing the package version-Footnote-6187658 +Ref: Single-sourcing the package version-Footnote-7187736 +Node: Supporting multiple Python versions187784 +Ref: guides/supporting-multiple-python-versions doc187971 +Ref: 102187971 +Ref: guides/supporting-multiple-python-versions id1187971 +Ref: 103187971 +Ref: guides/supporting-multiple-python-versions supporting-multiple-python-versions187971 +Ref: 104187971 +Node: Automated testing and continuous integration189717 +Ref: guides/supporting-multiple-python-versions automated-testing-and-continuous-integration189895 +Ref: 105189895 +Ref: Automated testing and continuous integration-Footnote-1191514 +Ref: Automated testing and continuous integration-Footnote-2191541 +Ref: Automated testing and continuous integration-Footnote-3191571 +Ref: Automated testing and continuous integration-Footnote-4191655 +Ref: Automated testing and continuous integration-Footnote-5191685 +Ref: Automated testing and continuous integration-Footnote-6191717 +Ref: Automated testing and continuous integration-Footnote-7191747 +Ref: Automated testing and continuous integration-Footnote-8191779 +Node: Tools for single-source Python packages191807 +Ref: guides/supporting-multiple-python-versions tools-for-single-source-python-packages192019 +Ref: 106192019 +Ref: Tools for single-source Python packages-Footnote-1193848 +Ref: Tools for single-source Python packages-Footnote-2193885 +Ref: Tools for single-source Python packages-Footnote-3193922 +Ref: Tools for single-source Python packages-Footnote-4193959 +Ref: Tools for single-source Python packages-Footnote-5194002 +Ref: Tools for single-source Python packages-Footnote-6194039 +Ref: Tools for single-source Python packages-Footnote-7194076 +Ref: Tools for single-source Python packages-Footnote-8194123 +Ref: Tools for single-source Python packages-Footnote-9194160 +Ref: Tools for single-source Python packages-Footnote-10194207 +Ref: Tools for single-source Python packages-Footnote-11194245 +Ref: Tools for single-source Python packages-Footnote-12194293 +Ref: Tools for single-source Python packages-Footnote-13194337 +Ref: Tools for single-source Python packages-Footnote-14194375 +Ref: Tools for single-source Python packages-Footnote-15194423 +Ref: Tools for single-source Python packages-Footnote-16194461 +Ref: Tools for single-source Python packages-Footnote-17194509 +Node: What’s in which Python?194557 +Ref: guides/supporting-multiple-python-versions what-s-in-which-python194716 +Ref: 107194716 +Ref: What’s in which Python?-Footnote-1195169 +Ref: What’s in which Python?-Footnote-2195242 +Ref: What’s in which Python?-Footnote-3195318 +Node: Dropping support for older Python versions195397 +Ref: guides/dropping-older-python-versions doc195576 +Ref: 108195576 +Ref: guides/dropping-older-python-versions dropping-support-for-older-python-versions195576 +Ref: 109195576 +Ref: guides/dropping-older-python-versions id1195576 +Ref: 10a195576 +Node: Requirements196575 +Ref: guides/dropping-older-python-versions requirements196725 +Ref: 10b196725 +Node: Defining the Python version required197073 +Ref: guides/dropping-older-python-versions defining-the-python-version-required197257 +Ref: 10c197257 +Node: 1 Download the newest version of Setuptools197730 +Ref: guides/dropping-older-python-versions download-the-newest-version-of-setuptools197932 +Ref: 10d197932 +Node: 2 Specify the version ranges for supported Python distributions198271 +Ref: guides/dropping-older-python-versions specify-the-version-ranges-for-supported-python-distributions198525 +Ref: 10e198525 +Node: 3 Validating the Metadata before publishing199263 +Ref: guides/dropping-older-python-versions validating-the-metadata-before-publishing199498 +Ref: 10f199498 +Node: 4 Using Twine to publish200305 +Ref: guides/dropping-older-python-versions using-twine-to-publish200468 +Ref: 110200468 +Node: Dropping a Python release200722 +Ref: guides/dropping-older-python-versions dropping-a-python-release200885 +Ref: 111200885 +Node: Packaging binary extensions201516 +Ref: guides/packaging-binary-extensions doc201693 +Ref: 69201693 +Ref: guides/packaging-binary-extensions binary-extensions201693 +Ref: 112201693 +Ref: guides/packaging-binary-extensions packaging-binary-extensions201693 +Ref: 113201693 +Node: An overview of binary extensions202319 +Ref: guides/packaging-binary-extensions an-overview-of-binary-extensions202468 +Ref: 114202468 +Node: Use cases202732 +Ref: guides/packaging-binary-extensions use-cases202846 +Ref: 115202846 +Ref: Use cases-Footnote-1205358 +Ref: Use cases-Footnote-2205428 +Ref: Use cases-Footnote-3205509 +Ref: Use cases-Footnote-4205580 +Ref: Use cases-Footnote-5205662 +Node: Disadvantages205735 +Ref: guides/packaging-binary-extensions disadvantages205903 +Ref: 116205903 +Node: Alternatives to handcoded accelerator modules207538 +Ref: guides/packaging-binary-extensions alternatives-to-handcoded-accelerator-modules207738 +Ref: 117207738 +Ref: Alternatives to handcoded accelerator modules-Footnote-1210252 +Ref: Alternatives to handcoded accelerator modules-Footnote-2210277 +Ref: Alternatives to handcoded accelerator modules-Footnote-3210304 +Node: Alternatives to handcoded wrapper modules210337 +Ref: guides/packaging-binary-extensions alternatives-to-handcoded-wrapper-modules210564 +Ref: 118210564 +Ref: Alternatives to handcoded wrapper modules-Footnote-1213021 +Ref: Alternatives to handcoded wrapper modules-Footnote-2213048 +Ref: Alternatives to handcoded wrapper modules-Footnote-3213085 +Node: Alternatives for low level system access213114 +Ref: guides/packaging-binary-extensions alternatives-for-low-level-system-access213287 +Ref: 119213287 +Node: Implementing binary extensions214273 +Ref: guides/packaging-binary-extensions implementing-binary-extensions214457 +Ref: 11a214457 +Ref: Implementing binary extensions-Footnote-1215100 +Ref: Implementing binary extensions-Footnote-2215145 +Node: Building binary extensions215204 +Ref: guides/packaging-binary-extensions building-binary-extensions215384 +Ref: 11b215384 +Node: Binary extensions for Windows215561 +Ref: guides/packaging-binary-extensions binary-extensions-for-windows215703 +Ref: 11c215703 +Ref: Binary extensions for Windows-Footnote-1217717 +Ref: Binary extensions for Windows-Footnote-2217788 +Ref: Binary extensions for Windows-Footnote-3217858 +Node: Binary extensions for Linux217943 +Ref: guides/packaging-binary-extensions binary-extensions-for-linux218121 +Ref: 11d218121 +Ref: Binary extensions for Linux-Footnote-1218478 +Node: Binary extensions for macOS218520 +Ref: guides/packaging-binary-extensions binary-extensions-for-macos218660 +Ref: 11e218660 +Ref: Binary extensions for macOS-Footnote-1219240 +Node: Publishing binary extensions219303 +Ref: guides/packaging-binary-extensions publishing-binary-extensions219473 +Ref: 11f219473 +Ref: Publishing binary extensions-Footnote-1219983 +Node: Additional resources220054 +Ref: guides/packaging-binary-extensions additional-resources220189 +Ref: 120220189 +Node: Cross-platform wheel generation with scikit-build220758 +Ref: guides/packaging-binary-extensions cross-platform-wheel-generation-with-scikit-build220926 +Ref: 121220926 +Ref: Cross-platform wheel generation with scikit-build-Footnote-1221374 +Ref: Cross-platform wheel generation with scikit-build-Footnote-2221429 +Node: Introduction to C/C++ extension modules221499 +Ref: guides/packaging-binary-extensions introduction-to-c-c-extension-modules221667 +Ref: 122221667 +Ref: Introduction to C/C++ extension modules-Footnote-1222055 +Ref: Introduction to C/C++ extension modules-Footnote-2222120 +Ref: Introduction to C/C++ extension modules-Footnote-3222176 +Node: Supporting Windows using Appveyor222236 +Ref: guides/supporting-windows-using-appveyor doc222399 +Ref: 123222399 +Ref: guides/supporting-windows-using-appveyor supporting-windows-using-appveyor222399 +Ref: 124222399 +Ref: Supporting Windows using Appveyor-Footnote-1222921 +Node: Background222954 +Ref: guides/supporting-windows-using-appveyor background223067 +Ref: 125223067 +Ref: Background-Footnote-1224080 +Ref: Background-Footnote-2224111 +Node: Setting up224139 +Ref: guides/supporting-windows-using-appveyor setting-up224300 +Ref: 126224300 +Ref: Setting up-Footnote-1225017 +Ref: Setting up-Footnote-2225054 +Ref: Setting up-Footnote-3225082 +Node: Adding Appveyor support to your project225113 +Ref: guides/supporting-windows-using-appveyor adding-appveyor-support-to-your-project225280 +Ref: 127225280 +Node: appveyor yml226213 +Ref: guides/supporting-windows-using-appveyor appveyor-yml226338 +Ref: 128226338 +Ref: appveyor yml-Footnote-1230703 +Node: Support script230829 +Ref: guides/supporting-windows-using-appveyor support-script230989 +Ref: 129230989 +Ref: Support script-Footnote-1231873 +Node: Access to the built wheels231996 +Ref: guides/supporting-windows-using-appveyor access-to-the-built-wheels232135 +Ref: 12a232135 +Node: Additional notes232673 +Ref: guides/supporting-windows-using-appveyor additional-notes232821 +Ref: 12b232821 +Node: Testing with tox232982 +Ref: guides/supporting-windows-using-appveyor testing-with-tox233104 +Ref: 12c233104 +Ref: Testing with tox-Footnote-1236036 +Node: Automatically uploading wheels236067 +Ref: guides/supporting-windows-using-appveyor automatically-uploading-wheels236219 +Ref: 12d236219 +Node: External dependencies236860 +Ref: guides/supporting-windows-using-appveyor external-dependencies237011 +Ref: 12e237011 +Node: Support scripts237592 +Ref: guides/supporting-windows-using-appveyor support-scripts237704 +Ref: 12f237704 +Node: Packaging namespace packages238778 +Ref: guides/packaging-namespace-packages doc238946 +Ref: 130238946 +Ref: guides/packaging-namespace-packages packaging-namespace-packages238946 +Ref: 131238946 +Ref: guides/packaging-namespace-packages tox238946 +Ref: 132238946 +Node: Creating a namespace package240602 +Ref: guides/packaging-namespace-packages creating-a-namespace-package240709 +Ref: 133240709 +Ref: Creating a namespace package-Footnote-1242041 +Node: Native namespace packages242090 +Ref: guides/packaging-namespace-packages native-namespace-packages242235 +Ref: 134242235 +Ref: Native namespace packages-Footnote-1243916 +Ref: Native namespace packages-Footnote-2243965 +Node: pkgutil-style namespace packages244043 +Ref: guides/packaging-namespace-packages pkgutil-style-namespace-packages244235 +Ref: 135244235 +Ref: pkgutil-style namespace packages-Footnote-1245500 +Ref: pkgutil-style namespace packages-Footnote-2245555 +Ref: pkgutil-style namespace packages-Footnote-3245631 +Node: pkg_resources-style namespace packages245710 +Ref: guides/packaging-namespace-packages pkg-resources-style-namespace-packages245868 +Ref: 136245868 +Ref: guides/packaging-namespace-packages pkgutil-namespace-example-project245868 +Ref: 137245868 +Ref: pkg_resources-style namespace packages-Footnote-1248492 +Ref: pkg_resources-style namespace packages-Footnote-2248534 +Ref: pkg_resources-style namespace packages-Footnote-3248622 +Node: Creating and discovering plugins248707 +Ref: guides/creating-and-discovering-plugins doc248874 +Ref: 138248874 +Ref: guides/creating-and-discovering-plugins creating-and-discovering-plugins248874 +Ref: 139248874 +Ref: guides/creating-and-discovering-plugins pkg-resources-namespace-example-project248874 +Ref: 13a248874 +Node: Using naming convention249530 +Ref: guides/creating-and-discovering-plugins using-naming-convention249669 +Ref: 13b249669 +Ref: Using naming convention-Footnote-1250755 +Ref: Using naming convention-Footnote-2250832 +Ref: Using naming convention-Footnote-3250872 +Ref: Using naming convention-Footnote-4250923 +Ref: Using naming convention-Footnote-5250971 +Node: Using namespace packages251035 +Ref: guides/creating-and-discovering-plugins simple-api251205 +Ref: 13e251205 +Ref: guides/creating-and-discovering-plugins using-namespace-packages251205 +Ref: 13c251205 +Ref: Using namespace packages-Footnote-1254013 +Ref: Using namespace packages-Footnote-2254090 +Node: Using package metadata254167 +Ref: guides/creating-and-discovering-plugins using-package-metadata254305 +Ref: 13d254305 +Ref: guides/creating-and-discovering-plugins special-support255374 +Ref: 13f255374 +Ref: Using package metadata-Footnote-1255411 +Ref: Using package metadata-Footnote-2255452 +Ref: Using package metadata-Footnote-3255562 +Node: Analyzing PyPI package downloads255672 +Ref: guides/analyzing-pypi-package-downloads doc255843 +Ref: 140255843 +Ref: guides/analyzing-pypi-package-downloads analyzing-pypi-package-downloads255843 +Ref: 141255843 +Ref: guides/analyzing-pypi-package-downloads entry-points255843 +Ref: 142255843 +Ref: guides/analyzing-pypi-package-downloads special-support255843 +Ref: 143255843 +Node: Background<2>256279 +Ref: guides/analyzing-pypi-package-downloads background256398 +Ref: 144256398 +Ref: Background<2>-Footnote-1257739 +Node: Public dataset257861 +Ref: guides/analyzing-pypi-package-downloads public-dataset257996 +Ref: 145257996 +Ref: Public dataset-Footnote-1258287 +Ref: Public dataset-Footnote-2258328 +Ref: Public dataset-Footnote-3258370 +Node: Getting set up258494 +Ref: guides/analyzing-pypi-package-downloads getting-set-up258593 +Ref: 146258593 +Ref: Getting set up-Footnote-1259188 +Ref: Getting set up-Footnote-2259230 +Ref: Getting set up-Footnote-3259311 +Ref: Getting set up-Footnote-4259449 +Ref: Getting set up-Footnote-5259499 +Ref: Getting set up-Footnote-6259588 +Node: Data schema259666 +Ref: guides/analyzing-pypi-package-downloads data-schema259788 +Ref: 147259788 +Ref: Data schema-Footnote-1261203 +Ref: Data schema-Footnote-2261305 +Node: Useful queries261341 +Ref: guides/analyzing-pypi-package-downloads useful-queries261440 +Ref: 148261440 +Ref: Useful queries-Footnote-1261904 +Node: Counting package downloads261954 +Ref: guides/analyzing-pypi-package-downloads counting-package-downloads262081 +Ref: 149262081 +Node: Package downloads over time263054 +Ref: guides/analyzing-pypi-package-downloads package-downloads-over-time263215 +Ref: 14a263215 +Node: Python versions over time264379 +Ref: guides/analyzing-pypi-package-downloads python-versions-over-time264505 +Ref: 14b264505 +Node: Caveats265521 +Ref: guides/analyzing-pypi-package-downloads caveats265659 +Ref: 14c265659 +Node: Additional tools266049 +Ref: guides/analyzing-pypi-package-downloads additional-tools266183 +Ref: 14d266183 +Node: google-cloud-bigquery266420 +Ref: guides/analyzing-pypi-package-downloads google-cloud-bigquery266524 +Ref: 14e266524 +Ref: google-cloud-bigquery-Footnote-1267533 +Node: pypinfo267600 +Ref: guides/analyzing-pypi-package-downloads pypinfo267723 +Ref: 14f267723 +Ref: pypinfo-Footnote-1268308 +Ref: pypinfo-Footnote-2268371 +Node: pandas-gbq268434 +Ref: guides/analyzing-pypi-package-downloads pandas-gbq268527 +Ref: 150268527 +Ref: pandas-gbq-Footnote-1268689 +Ref: pandas-gbq-Footnote-2268742 +Node: References268777 +Ref: guides/analyzing-pypi-package-downloads references268895 +Ref: 151268895 +Node: Package index mirrors and caches268932 +Ref: guides/index-mirrors-and-caches doc269105 +Ref: 152269105 +Ref: guides/index-mirrors-and-caches package-index-mirrors-and-caches269105 +Ref: 153269105 +Ref: guides/index-mirrors-and-caches pandas269105 +Ref: 154269105 +Ref: guides/index-mirrors-and-caches pypi-mirrors-and-caches269105 +Ref: 155269105 +Node: Caching with pip269774 +Ref: guides/index-mirrors-and-caches caching-with-pip269900 +Ref: 156269900 +Ref: Caching with pip-Footnote-1270524 +Ref: Caching with pip-Footnote-2270606 +Node: Caching with devpi270676 +Ref: guides/index-mirrors-and-caches caching-with-devpi270844 +Ref: 157270844 +Ref: Caching with devpi-Footnote-1271113 +Node: Complete mirror with bandersnatch271176 +Ref: guides/index-mirrors-and-caches complete-mirror-with-bandersnatch271319 +Ref: 158271319 +Ref: guides/index-mirrors-and-caches id1271319 +Ref: 159271319 +Ref: Complete mirror with bandersnatch-Footnote-1271819 +Node: Hosting your own simple repository271865 +Ref: guides/hosting-your-own-index doc272027 +Ref: 15a272027 +Ref: guides/hosting-your-own-index hosting-your-own-simple-repository272027 +Ref: 15b272027 +Ref: guides/hosting-your-own-index id1272027 +Ref: 15c272027 +Ref: Hosting your own simple repository-Footnote-1272937 +Ref: Hosting your own simple repository-Footnote-2273024 +Node: “Manual” repository273061 +Ref: guides/hosting-your-own-index manual-repository273169 +Ref: 15d273169 +Ref: “Manual” repository-Footnote-1274219 +Node: Migrating to PyPI org274254 +Ref: guides/migrating-to-pypi-org doc274401 +Ref: 15e274401 +Ref: guides/migrating-to-pypi-org id1274401 +Ref: 15f274401 +Ref: guides/migrating-to-pypi-org migrating-to-pypi-org274401 +Ref: 87274401 +Ref: guides/migrating-to-pypi-org twisted274401 +Ref: 160274401 +Node: Publishing releases274939 +Ref: guides/migrating-to-pypi-org publishing-releases275075 +Ref: 162275075 +Node: Registering package names & metadata277066 +Ref: guides/migrating-to-pypi-org registering-package-names-metadata277225 +Ref: 163277225 +Node: Using TestPyPI277846 +Ref: guides/migrating-to-pypi-org using-testpypi278015 +Ref: 164278015 +Ref: Using TestPyPI-Footnote-1278684 +Node: Registering new user accounts278714 +Ref: guides/migrating-to-pypi-org registering-new-user-accounts278864 +Ref: 165278864 +Node: Browsing packages279140 +Ref: guides/migrating-to-pypi-org browsing-packages279296 +Ref: 166279296 +Node: Downloading packages279604 +Ref: guides/migrating-to-pypi-org downloading-packages279771 +Ref: 167279771 +Node: Managing published packages and releases279890 +Ref: guides/migrating-to-pypi-org managing-published-packages-and-releases280031 +Ref: 168280031 +Node: Using TestPyPI<2>280251 +Ref: guides/using-testpypi doc280393 +Ref: 65280393 +Ref: guides/using-testpypi using-test-pypi280393 +Ref: f6280393 +Ref: guides/using-testpypi using-testpypi280393 +Ref: 169280393 +Ref: Using TestPyPI<2>-Footnote-1280868 +Node: Registering your account280898 +Ref: guides/using-testpypi registering-your-account281024 +Ref: 16a281024 +Node: Using TestPyPI with Twine281423 +Ref: guides/using-testpypi using-testpypi-with-twine281581 +Ref: 16b281581 +Node: Using TestPyPI with pip282084 +Ref: guides/using-testpypi using-testpypi-with-pip282247 +Ref: 16c282247 +Node: Setting up TestPyPI in pypirc282799 +Ref: guides/using-testpypi setting-up-testpypi-in-pypirc282928 +Ref: 16d282928 +Node: Making a PyPI-friendly README283244 +Ref: guides/making-a-pypi-friendly-readme doc283442 +Ref: 16e283442 +Ref: guides/making-a-pypi-friendly-readme making-a-pypi-friendly-readme283442 +Ref: 16f283442 +Node: Creating a README file283889 +Ref: guides/making-a-pypi-friendly-readme creating-a-readme-file284050 +Ref: 170284050 +Ref: Creating a README file-Footnote-1284679 +Ref: Creating a README file-Footnote-2284727 +Ref: Creating a README file-Footnote-3284776 +Ref: Creating a README file-Footnote-4284815 +Node: Including your README in your package’s metadata284846 +Ref: guides/making-a-pypi-friendly-readme including-your-readme-in-your-package-s-metadata285050 +Ref: 171285050 +Node: Validating reStructuredText markup287099 +Ref: guides/making-a-pypi-friendly-readme validating-restructuredtext-markup287272 +Ref: 174287272 +Ref: Validating reStructuredText markup-Footnote-1288403 +Ref: Validating reStructuredText markup-Footnote-2288482 +Node: Publishing package distribution releases using GitHub Actions CI/CD workflows288520 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows doc288692 +Ref: 175288692 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows publishing-package-distribution-releases-using-github-actions-ci-cd-workflows288692 +Ref: 176288692 +Ref: Publishing package distribution releases using GitHub Actions CI/CD workflows-Footnote-1289672 +Ref: Publishing package distribution releases using GitHub Actions CI/CD workflows-Footnote-2289716 +Node: Saving credentials on GitHub289776 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows saving-credentials-on-github289971 +Ref: 177289971 +Ref: Saving credentials on GitHub-Footnote-1291307 +Ref: Saving credentials on GitHub-Footnote-2291347 +Node: Creating a workflow definition291479 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows creating-a-workflow-definition291718 +Ref: 178291718 +Node: Defining a workflow job environment292173 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows defining-a-workflow-job-environment292435 +Ref: 179292435 +Node: Checking out the project and building distributions292835 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows checking-out-the-project-and-building-distributions293115 +Ref: 17a293115 +Ref: Checking out the project and building distributions-Footnote-1294351 +Ref: Checking out the project and building distributions-Footnote-2294400 +Node: Publishing the distribution to PyPI and TestPyPI294449 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows publishing-the-distribution-to-pypi-and-testpypi294713 +Ref: 17b294713 +Ref: Publishing the distribution to PyPI and TestPyPI-Footnote-1295607 +Node: That’s all folks!295662 +Ref: guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows that-s-all-folks295866 +Ref: 17c295866 +Node: Discussions296211 +Ref: discussions/index doc296319 +Ref: 17d296319 +Ref: discussions/index discussions296319 +Ref: 17e296319 +Ref: discussions/index secrets296319 +Ref: 17f296319 +Node: Deploying Python applications296632 +Ref: discussions/deploying-python-applications doc296751 +Ref: 180296751 +Ref: discussions/deploying-python-applications deploying-python-applications296751 +Ref: 181296751 +Node: Overview296980 +Ref: discussions/deploying-python-applications overview297102 +Ref: 182297102 +Node: Supporting multiple hardware platforms297186 +Ref: discussions/deploying-python-applications supporting-multiple-hardware-platforms297283 +Ref: 183297283 +Node: OS packaging & installers298059 +Ref: discussions/deploying-python-applications os-packaging-installers298209 +Ref: 184298209 +Node: Windows<2>298521 +Ref: discussions/deploying-python-applications windows298607 +Ref: 185298607 +Node: Pynsist298730 +Ref: discussions/deploying-python-applications pynsist298798 +Ref: 186298798 +Ref: Pynsist-Footnote-1299740 +Ref: Pynsist-Footnote-2299781 +Node: Application bundles299820 +Ref: discussions/deploying-python-applications application-bundles299986 +Ref: 187299986 +Node: Configuration management300103 +Ref: discussions/deploying-python-applications configuration-management300235 +Ref: 188300235 +Node: pip vs easy_install300368 +Ref: discussions/pip-vs-easy-install doc300534 +Ref: 189300534 +Ref: discussions/pip-vs-easy-install id1300534 +Ref: 18a300534 +Ref: discussions/pip-vs-easy-install pip-vs-easy-install300534 +Ref: 7d300534 +Ref: pip vs easy_install-Footnote-1304580 +Ref: pip vs easy_install-Footnote-2304649 +Ref: pip vs easy_install-Footnote-3304718 +Ref: pip vs easy_install-Footnote-4304767 +Ref: pip vs easy_install-Footnote-5304820 +Node: install_requires vs requirements files304924 +Ref: discussions/install-requires-vs-requirements doc305073 +Ref: 18c305073 +Ref: discussions/install-requires-vs-requirements id1305073 +Ref: 18d305073 +Ref: discussions/install-requires-vs-requirements install-requires-vs-requirements-files305073 +Ref: d6305073 +Ref: discussions/install-requires-vs-requirements pylauncher-support305073 +Ref: 18e305073 +Node: install_requires<2>305256 +Ref: discussions/install-requires-vs-requirements install-requires305394 +Ref: 18f305394 +Ref: install_requires<2>-Footnote-1307066 +Node: Requirements files<2>307197 +Ref: discussions/install-requires-vs-requirements requirements-files307335 +Ref: 190307335 +Ref: Requirements files<2>-Footnote-1308512 +Ref: Requirements files<2>-Footnote-2308581 +Ref: Requirements files<2>-Footnote-3308654 +Ref: Requirements files<2>-Footnote-4308723 +Ref: Requirements files<2>-Footnote-5308787 +Node: Wheel vs Egg308918 +Ref: discussions/wheel-vs-egg doc309039 +Ref: 191309039 +Ref: discussions/wheel-vs-egg id1309039 +Ref: 192309039 +Ref: discussions/wheel-vs-egg wheel-vs-egg309039 +Ref: 193309039 +Ref: Wheel vs Egg-Footnote-1310987 +Ref: Wheel vs Egg-Footnote-2311036 +Ref: Wheel vs Egg-Footnote-3311085 +Ref: Wheel vs Egg-Footnote-4311345 +Ref: Wheel vs Egg-Footnote-5311394 +Ref: Wheel vs Egg-Footnote-6311443 +Node: PyPA specifications311519 +Ref: specifications/index doc311638 +Ref: 195311638 +Ref: specifications/index packaging-specifications311638 +Ref: 196311638 +Ref: specifications/index pypa-specifications311638 +Ref: 197311638 +Ref: PyPA specifications-Footnote-1312006 +Node: Package Distribution Metadata312060 +Ref: specifications/index package-distribution-metadata312192 +Ref: 198312192 +Node: Core metadata specifications312545 +Ref: specifications/core-metadata doc312680 +Ref: 199312680 +Ref: specifications/core-metadata core-metadata312680 +Ref: 85312680 +Ref: specifications/core-metadata core-metadata-specifications312680 +Ref: 19a312680 +Ref: Core metadata specifications-Footnote-1314329 +Ref: Core metadata specifications-Footnote-2314378 +Node: Metadata-Version314427 +Ref: specifications/core-metadata metadata-version314535 +Ref: 19b314535 +Ref: Metadata-Version-Footnote-1315245 +Node: Name315294 +Ref: specifications/core-metadata core-metadata-name315418 +Ref: 19c315418 +Ref: specifications/core-metadata name315418 +Ref: 19d315418 +Ref: Name-Footnote-1315999 +Node: Version316048 +Ref: specifications/core-metadata core-metadata-version316177 +Ref: 19e316177 +Ref: specifications/core-metadata version316177 +Ref: 19f316177 +Ref: Version-Footnote-1316415 +Node: Platform multiple use316464 +Ref: specifications/core-metadata platform-multiple-use316620 +Ref: 1a0316620 +Node: Supported-Platform multiple use316953 +Ref: specifications/core-metadata supported-platform-multiple-use317109 +Ref: 1a1317109 +Node: Summary317560 +Ref: specifications/core-metadata core-metadata-summary317706 +Ref: 1a2317706 +Ref: specifications/core-metadata summary317706 +Ref: 1a3317706 +Ref: specifications/core-metadata description-optional317880 +Ref: 172317880 +Node: Description317880 +Ref: specifications/core-metadata core-metadata-description318019 +Ref: 1a4318019 +Ref: specifications/core-metadata description318019 +Ref: 1a5318019 +Ref: specifications/core-metadata description-content-type-optional319621 +Ref: 173319621 +Ref: Description-Footnote-1319659 +Ref: Description-Footnote-2319735 +Node: Description-Content-Type319823 +Ref: specifications/core-metadata core-metadata-description-content-type319963 +Ref: 1a6319963 +Ref: specifications/core-metadata description-content-type319963 +Ref: 1a7319963 +Ref: specifications/core-metadata keywords-optional322970 +Ref: 1a8322970 +Ref: Description-Content-Type-Footnote-1323008 +Ref: Description-Content-Type-Footnote-2323084 +Ref: Description-Content-Type-Footnote-3323138 +Ref: Description-Content-Type-Footnote-4323182 +Ref: Description-Content-Type-Footnote-5323247 +Ref: Description-Content-Type-Footnote-6323303 +Node: Keywords323359 +Ref: specifications/core-metadata core-metadata-keywords323497 +Ref: 1a9323497 +Ref: specifications/core-metadata keywords323497 +Ref: 1aa323497 +Node: Home-page324013 +Ref: specifications/core-metadata home-page324139 +Ref: 1ab324139 +Ref: specifications/core-metadata home-page-optional324139 +Ref: 1ac324139 +Node: Download-URL324331 +Ref: specifications/core-metadata download-url324455 +Ref: 1ad324455 +Ref: specifications/core-metadata author-optional324749 +Ref: 1ae324749 +Node: Author324750 +Ref: specifications/core-metadata author324877 +Ref: 1af324877 +Ref: specifications/core-metadata core-metadata-author324877 +Ref: 1b0324877 +Ref: specifications/core-metadata author-email-optional325159 +Ref: 1b1325159 +Node: Author-email325160 +Ref: specifications/core-metadata author-email325285 +Ref: 1b2325285 +Ref: specifications/core-metadata core-metadata-author-email325285 +Ref: 1b3325285 +Ref: specifications/core-metadata maintainer-optional325704 +Ref: 1b4325704 +Node: Maintainer325705 +Ref: specifications/core-metadata core-metadata-maintainer325840 +Ref: 1b5325840 +Ref: specifications/core-metadata maintainer325840 +Ref: 1b6325840 +Ref: specifications/core-metadata maintainer-email-optional326315 +Ref: 1b7326315 +Node: Maintainer-email326316 +Ref: specifications/core-metadata core-metadata-maintainer-email326446 +Ref: 1b8326446 +Ref: specifications/core-metadata maintainer-email326446 +Ref: 1b9326446 +Ref: specifications/core-metadata license-optional327068 +Ref: 1ba327068 +Node: License327069 +Ref: specifications/core-metadata core-metadata-license327212 +Ref: 1bb327212 +Ref: specifications/core-metadata license327212 +Ref: 1bc327212 +Ref: specifications/core-metadata metadata-classifier327849 +Ref: 1bd327849 +Node: Classifier multiple use327850 +Ref: specifications/core-metadata classifier-multiple-use328003 +Ref: 1be328003 +Ref: specifications/core-metadata core-metadata-classifier328003 +Ref: 1bf328003 +Ref: Classifier multiple use-Footnote-1328532 +Ref: Classifier multiple use-Footnote-2328581 +Node: Requires-Dist multiple use328619 +Ref: specifications/core-metadata core-metadata-requires-dist328780 +Ref: 1c0328780 +Ref: specifications/core-metadata requires-dist-multiple-use328780 +Ref: 1c1328780 +Ref: Requires-Dist multiple use-Footnote-1330167 +Ref: Requires-Dist multiple use-Footnote-2330216 +Node: Requires-Python330241 +Ref: specifications/core-metadata core-metadata-requires-python330409 +Ref: 1c3330409 +Ref: specifications/core-metadata requires-python330409 +Ref: 1c4330409 +Node: Requires-External multiple use330916 +Ref: specifications/core-metadata requires-external-multiple-use331082 +Ref: 1c5331082 +Ref: Requires-External multiple use-Footnote-1332252 +Node: Project-URL multiple-use332301 +Ref: specifications/core-metadata core-metadata-project-url332479 +Ref: 1c6332479 +Ref: specifications/core-metadata project-url-multiple-use332479 +Ref: 1c7332479 +Ref: specifications/core-metadata metadata-provides-extra332804 +Ref: 1c8332804 +Ref: specifications/core-metadata core-metadata-provides-extra332804 +Ref: 1c9332804 +Node: Provides-Extra multiple use332806 +Ref: specifications/core-metadata provides-extra-multiple-use332972 +Ref: 1ca332972 +Ref: specifications/core-metadata provides-extra-optional-multiple-use332972 +Ref: 1cb332972 +Node: Rarely Used Fields333977 +Ref: specifications/core-metadata rarely-used-fields334110 +Ref: 1cc334110 +Ref: Rarely Used Fields-Footnote-1334993 +Node: Provides-Dist multiple use335018 +Ref: specifications/core-metadata provides-dist-multiple-use335149 +Ref: 1cd335149 +Node: Obsoletes-Dist multiple use336846 +Ref: specifications/core-metadata obsoletes-dist-multiple-use336977 +Ref: 1ce336977 +Node: Version specifiers337988 +Ref: specifications/version-specifiers doc338153 +Ref: 1c2338153 +Ref: specifications/version-specifiers id1338153 +Ref: 1cf338153 +Ref: specifications/version-specifiers version-specifiers338153 +Ref: 1d0338153 +Ref: Version specifiers-Footnote-1338462 +Ref: Version specifiers-Footnote-2338511 +Node: Dependency specifiers338560 +Ref: specifications/dependency-specifiers doc338732 +Ref: 1d1338732 +Ref: specifications/dependency-specifiers dependency-specifiers338732 +Ref: 1d2338732 +Ref: specifications/dependency-specifiers id1338732 +Ref: 1d3338732 +Ref: Dependency specifiers-Footnote-1339037 +Ref: Dependency specifiers-Footnote-2339086 +Node: Declaring build system dependencies339135 +Ref: specifications/declaring-build-dependencies doc339315 +Ref: 1d4339315 +Ref: specifications/declaring-build-dependencies declaring-build-dependencies339315 +Ref: 1d5339315 +Ref: specifications/declaring-build-dependencies declaring-build-system-dependencies339315 +Ref: 1d6339315 +Ref: Declaring build system dependencies-Footnote-1339686 +Node: Declaring project metadata339735 +Ref: specifications/declaring-project-metadata doc339914 +Ref: 1d8339914 +Ref: specifications/declaring-project-metadata declaring-project-metadata339914 +Ref: 1d9339914 +Ref: specifications/declaring-project-metadata id1339914 +Ref: 1da339914 +Ref: Declaring project metadata-Footnote-1341652 +Node: name<2>341701 +Ref: specifications/declaring-project-metadata name341804 +Ref: 1db341804 +Ref: name<2>-Footnote-1342108 +Ref: name<2>-Footnote-2342133 +Node: version<2>342182 +Ref: specifications/declaring-project-metadata version342308 +Ref: 1dc342308 +Ref: version<2>-Footnote-1342601 +Ref: version<2>-Footnote-2342626 +Node: description<2>342675 +Ref: specifications/declaring-project-metadata description342800 +Ref: 1dd342800 +Ref: description<2>-Footnote-1343025 +Node: readme343050 +Ref: specifications/declaring-project-metadata readme343180 +Ref: 1de343180 +Ref: readme-Footnote-1345119 +Node: requires-python345144 +Ref: specifications/declaring-project-metadata requires-python345270 +Ref: 1df345270 +Ref: requires-python-Footnote-1345524 +Node: license<2>345549 +Ref: specifications/declaring-project-metadata license345688 +Ref: 1e0345688 +Ref: license<2>-Footnote-1346278 +Node: authors/maintainers346303 +Ref: specifications/declaring-project-metadata authors-maintainers346438 +Ref: 1e1346438 +Ref: authors/maintainers-Footnote-1348016 +Ref: authors/maintainers-Footnote-2348041 +Node: keywords<2>348084 +Ref: specifications/declaring-project-metadata keywords348223 +Ref: 1e2348223 +Ref: keywords<2>-Footnote-1348443 +Node: classifiers<2>348468 +Ref: specifications/declaring-project-metadata classifiers348592 +Ref: 1e3348592 +Ref: classifiers<2>-Footnote-1348841 +Node: urls348866 +Ref: specifications/declaring-project-metadata urls348991 +Ref: 1e4348991 +Ref: urls-Footnote-1349284 +Node: Entry points349309 +Ref: specifications/declaring-project-metadata entry-points349454 +Ref: 1e5349454 +Ref: Entry points-Footnote-1350713 +Node: dependencies/optional-dependencies350738 +Ref: specifications/declaring-project-metadata dependencies-optional-dependencies350886 +Ref: 1e7350886 +Ref: dependencies/optional-dependencies-Footnote-1351978 +Ref: dependencies/optional-dependencies-Footnote-2352003 +Ref: dependencies/optional-dependencies-Footnote-3352052 +Ref: dependencies/optional-dependencies-Footnote-4352101 +Ref: dependencies/optional-dependencies-Footnote-5352150 +Node: dynamic352199 +Ref: specifications/declaring-project-metadata dynamic352326 +Ref: 1e8352326 +Ref: dynamic-Footnote-1354257 +Node: Distribution formats354282 +Ref: specifications/distribution-formats doc354453 +Ref: 1e9354453 +Ref: specifications/distribution-formats distribution-formats354453 +Ref: 1ea354453 +Ref: specifications/distribution-formats id1354453 +Ref: 1eb354453 +Ref: specifications/distribution-formats toml354453 +Ref: 1ec354453 +Node: Source distribution format354580 +Ref: specifications/distribution-formats source-distribution-format354712 +Ref: 1ed354712 +Ref: Source distribution format-Footnote-1355156 +Ref: Source distribution format-Footnote-2355205 +Node: Binary distribution format355254 +Ref: specifications/distribution-formats binary-distribution-format355386 +Ref: 1ee355386 +Ref: specifications/distribution-formats id2355386 +Ref: 1ef355386 +Ref: Binary distribution format-Footnote-1355565 +Node: Platform compatibility tags355614 +Ref: specifications/platform-compatibility-tags doc355787 +Ref: 1f0355787 +Ref: specifications/platform-compatibility-tags id1355787 +Ref: 1f1355787 +Ref: specifications/platform-compatibility-tags platform-compatibility-tags355787 +Ref: 1f2355787 +Ref: Platform compatibility tags-Footnote-1356431 +Node: Platform tags for Windows356480 +Ref: specifications/platform-compatibility-tags platform-tags-for-windows356624 +Ref: 1f3356624 +Ref: Platform tags for Windows-Footnote-1356833 +Node: Platform tags for macOS Mac OS X356882 +Ref: specifications/platform-compatibility-tags platform-tags-for-macos-mac-os-x357079 +Ref: 1f4357079 +Ref: Platform tags for macOS Mac OS X-Footnote-1357335 +Node: Platform tags for common Linux distributions357384 +Ref: specifications/platform-compatibility-tags platform-tags-for-common-linux-distributions357594 +Ref: 1f5357594 +Ref: specifications/platform-compatibility-tags manylinux357701 +Ref: 1f6357701 +Ref: Platform tags for common Linux distributions-Footnote-1359692 +Ref: Platform tags for common Linux distributions-Footnote-2359741 +Ref: Platform tags for common Linux distributions-Footnote-3359790 +Ref: Platform tags for common Linux distributions-Footnote-4359839 +Ref: Platform tags for common Linux distributions-Footnote-5359888 +Ref: Platform tags for common Linux distributions-Footnote-6359937 +Node: Manylinux compatibility support359993 +Ref: specifications/platform-compatibility-tags manylinux-compatibility-support360119 +Ref: 1f7360119 +Node: Platform tags for other *nix platforms361047 +Ref: specifications/platform-compatibility-tags platform-tags-for-other-nix-platforms361216 +Ref: 1f8361216 +Ref: Platform tags for other *nix platforms-Footnote-1361609 +Node: Recording installed projects361658 +Ref: specifications/recording-installed-packages doc361837 +Ref: 1f9361837 +Ref: specifications/recording-installed-packages recording-installed-packages361837 +Ref: 1fa361837 +Ref: specifications/recording-installed-packages recording-installed-projects361837 +Ref: 1fb361837 +Node: History and change workflow362680 +Ref: specifications/recording-installed-packages history-and-change-workflow362818 +Ref: 1fd362818 +Ref: History and change workflow-Footnote-1363407 +Ref: History and change workflow-Footnote-2363456 +Ref: History and change workflow-Footnote-3363505 +Node: The dist-info directory363554 +Ref: specifications/recording-installed-packages the-dist-info-directory363718 +Ref: 1fe363718 +Ref: The dist-info directory-Footnote-1365231 +Ref: The dist-info directory-Footnote-2365313 +Ref: The dist-info directory-Footnote-3365387 +Ref: The dist-info directory-Footnote-4365454 +Node: The METADATA file365514 +Ref: specifications/recording-installed-packages the-metadata-file365666 +Ref: 1ff365666 +Node: The RECORD file366038 +Ref: specifications/recording-installed-packages the-record-file366185 +Ref: 200366185 +Node: The INSTALLER file369760 +Ref: specifications/recording-installed-packages the-installer-file369881 +Ref: 201369881 +Node: Entry points specification370615 +Ref: specifications/entry-points doc370758 +Ref: 202370758 +Ref: specifications/entry-points entry-points370758 +Ref: 1e6370758 +Ref: specifications/entry-points entry-points-specification370758 +Ref: 203370758 +Node: Data model372094 +Ref: specifications/entry-points data-model372201 +Ref: 204372201 +Node: File format375082 +Ref: specifications/entry-points file-format375213 +Ref: 205375213 +Ref: File format-Footnote-1377058 +Ref: File format-Footnote-2377107 +Ref: File format-Footnote-3377156 +Ref: File format-Footnote-4377237 +Node: Use for scripts377286 +Ref: specifications/entry-points use-for-scripts377398 +Ref: 206377398 +Node: Package Index Interfaces379136 +Ref: specifications/index package-index-interfaces379268 +Ref: 207379268 +Node: The pypirc file379400 +Ref: specifications/pypirc doc379520 +Ref: 208379520 +Ref: specifications/pypirc pypirc379520 +Ref: f8379520 +Ref: specifications/pypirc the-pypirc-file379520 +Ref: 209379520 +Ref: The pypirc file-Footnote-1381175 +Node: Common configurations381217 +Ref: specifications/pypirc common-configurations381304 +Ref: 20a381304 +Ref: specifications/pypirc keyring381304 +Ref: 20b381304 +Node: Using a PyPI token382192 +Ref: specifications/pypirc using-a-pypi-token382318 +Ref: 20d382318 +Ref: Using a PyPI token-Footnote-1382667 +Node: Using another package index382707 +Ref: specifications/pypirc api-token382833 +Ref: 20e382833 +Ref: specifications/pypirc using-another-package-index382833 +Ref: 20f382833 +Ref: Using another package index-Footnote-1383949 +Node: Simple repository API383991 +Ref: specifications/simple-repository-api doc384111 +Ref: 210384111 +Ref: specifications/simple-repository-api id1384111 +Ref: 211384111 +Ref: specifications/simple-repository-api simple-repository-api384111 +Ref: 212384111 +Ref: Simple repository API-Footnote-1384439 +Ref: Simple repository API-Footnote-2384488 +Node: Project Summaries384537 +Ref: key_projects doc384653 +Ref: 213384653 +Ref: key_projects project-summaries384653 +Ref: 214384653 +Ref: key_projects projects384653 +Ref: 215384653 +Node: PyPA Projects384877 +Ref: key_projects id1384984 +Ref: 216384984 +Ref: key_projects pypa-projects384984 +Ref: 217384984 +Node: bandersnatch385298 +Ref: key_projects bandersnatch385388 +Ref: 218385388 +Ref: key_projects id2385388 +Ref: 219385388 +Ref: bandersnatch-Footnote-1385852 +Ref: bandersnatch-Footnote-2385914 +Ref: bandersnatch-Footnote-3386001 +Ref: bandersnatch-Footnote-4386053 +Ref: bandersnatch-Footnote-5386098 +Ref: bandersnatch-Footnote-6386144 +Node: build386207 +Ref: key_projects build386313 +Ref: 21a386313 +Ref: key_projects id4386313 +Ref: 21b386313 +Ref: build-Footnote-1386584 +Ref: build-Footnote-2386627 +Ref: build-Footnote-3386672 +Ref: build-Footnote-4386710 +Ref: build-Footnote-5386749 +Ref: build-Footnote-6386804 +Node: distlib386863 +Ref: key_projects distlib386966 +Ref: 21c386966 +Ref: key_projects id5386966 +Ref: 21d386966 +Ref: distlib-Footnote-1387935 +Ref: distlib-Footnote-2387976 +Ref: distlib-Footnote-3388038 +Ref: distlib-Footnote-4388125 +Ref: distlib-Footnote-5388198 +Ref: distlib-Footnote-6388241 +Node: packaging388282 +Ref: key_projects id7388383 +Ref: 21f388383 +Ref: key_projects packaging388383 +Ref: 21e388383 +Ref: packaging-Footnote-1389527 +Ref: packaging-Footnote-2389561 +Ref: packaging-Footnote-3389634 +Ref: packaging-Footnote-4389683 +Ref: packaging-Footnote-5389725 +Ref: packaging-Footnote-6389768 +Ref: packaging-Footnote-7389823 +Node: pip389882 +Ref: key_projects id8389982 +Ref: 220389982 +Ref: key_projects pip389982 +Ref: 2b389982 +Ref: pip-Footnote-1390506 +Ref: pip-Footnote-2390545 +Ref: pip-Footnote-3390602 +Ref: pip-Footnote-4390746 +Ref: pip-Footnote-5390819 +Ref: pip-Footnote-6390862 +Ref: pip-Footnote-7390898 +Ref: pip-Footnote-8390936 +Ref: pip-Footnote-9390991 +Node: Pipenv391050 +Ref: key_projects id10391148 +Ref: 221391148 +Ref: key_projects pipenv391148 +Ref: 36391148 +Ref: Pipenv-Footnote-1391913 +Ref: Pipenv-Footnote-2391945 +Ref: Pipenv-Footnote-3391984 +Ref: Pipenv-Footnote-4392030 +Node: Pipfile392070 +Ref: key_projects id11392195 +Ref: 222392195 +Ref: key_projects pipfile392195 +Ref: 51392195 +Ref: Pipfile-Footnote-1392434 +Node: Python Packaging User Guide<2>392474 +Ref: key_projects python-packaging-user-guide392608 +Ref: 223392608 +Ref: Python Packaging User Guide<2>-Footnote-1392820 +Ref: Python Packaging User Guide<2>-Footnote-2392868 +Ref: Python Packaging User Guide<2>-Footnote-3392930 +Ref: Python Packaging User Guide<2>-Footnote-4392997 +Ref: Python Packaging User Guide<2>-Footnote-5393057 +Ref: Python Packaging User Guide<2>-Footnote-6393112 +Node: readme_renderer393171 +Ref: key_projects id12393308 +Ref: 224393308 +Ref: key_projects readme-renderer393308 +Ref: 225393308 +Ref: key_projects setuptools393737 +Ref: 2d393737 +Ref: readme_renderer-Footnote-1393775 +Ref: readme_renderer-Footnote-2393824 +Node: setuptools393874 +Ref: key_projects easy-install393998 +Ref: 18b393998 +Ref: key_projects id13393998 +Ref: 226393998 +Ref: setuptools-Footnote-1394590 +Ref: setuptools-Footnote-2394643 +Ref: setuptools-Footnote-3394705 +Ref: setuptools-Footnote-4394792 +Ref: setuptools-Footnote-5394865 +Ref: setuptools-Footnote-6394915 +Ref: setuptools-Footnote-7394958 +Ref: setuptools-Footnote-8395002 +Ref: setuptools-Footnote-9395057 +Ref: setuptools-Footnote-10395116 +Node: trove-classifiers395161 +Ref: key_projects id15395275 +Ref: 227395275 +Ref: key_projects trove-classifiers395275 +Ref: 228395275 +Ref: trove-classifiers-Footnote-1396149 +Ref: trove-classifiers-Footnote-2396206 +Ref: trove-classifiers-Footnote-3396256 +Ref: trove-classifiers-Footnote-4396308 +Ref: trove-classifiers-Footnote-5396346 +Ref: trove-classifiers-Footnote-6396438 +Ref: trove-classifiers-Footnote-7396476 +Node: twine396533 +Ref: key_projects id16396650 +Ref: 229396650 +Ref: key_projects twine396650 +Ref: 66396650 +Ref: twine-Footnote-1397113 +Ref: twine-Footnote-2397161 +Ref: twine-Footnote-3397223 +Ref: twine-Footnote-4397310 +Ref: twine-Footnote-5397355 +Ref: twine-Footnote-6397393 +Node: virtualenv<2>397432 +Ref: key_projects id18397541 +Ref: 22a397541 +Ref: key_projects virtualenv397541 +Ref: 32397541 +Ref: virtualenv<2>-Footnote-1398203 +Ref: virtualenv<2>-Footnote-2398249 +Ref: virtualenv<2>-Footnote-3398306 +Ref: virtualenv<2>-Footnote-4398379 +Ref: virtualenv<2>-Footnote-5398429 +Ref: virtualenv<2>-Footnote-6398472 +Ref: virtualenv<2>-Footnote-7398517 +Ref: virtualenv<2>-Footnote-8398572 +Node: Warehouse398631 +Ref: key_projects id19398740 +Ref: 22c398740 +Ref: key_projects warehouse398740 +Ref: 22d398740 +Ref: Warehouse-Footnote-1399042 +Ref: Warehouse-Footnote-2399077 +Ref: Warehouse-Footnote-3399139 +Ref: Warehouse-Footnote-4399226 +Ref: Warehouse-Footnote-5399275 +Ref: Warehouse-Footnote-6399317 +Ref: Warehouse-Footnote-7399376 +Node: wheel399402 +Ref: key_projects id21399489 +Ref: 22e399489 +Ref: key_projects wheel399489 +Ref: 2e399489 +Ref: wheel-Footnote-1400208 +Ref: wheel-Footnote-2400256 +Ref: wheel-Footnote-3400318 +Ref: wheel-Footnote-4400405 +Ref: wheel-Footnote-5400450 +Ref: wheel-Footnote-6400488 +Ref: wheel-Footnote-7400527 +Ref: wheel-Footnote-8400582 +Ref: wheel-Footnote-9400641 +Node: Non-PyPA Projects400684 +Ref: key_projects non-pypa-projects400825 +Ref: 22f400825 +Node: bento401114 +Ref: key_projects bento401204 +Ref: 230401204 +Ref: key_projects id23401204 +Ref: 231401204 +Ref: bento-Footnote-1401552 +Ref: bento-Footnote-2401593 +Ref: bento-Footnote-3401636 +Ref: bento-Footnote-4401685 +Ref: bento-Footnote-5401727 +Node: buildout401766 +Ref: key_projects buildout401870 +Ref: 80401870 +Ref: key_projects id24401870 +Ref: 232401870 +Ref: buildout-Footnote-1402258 +Ref: buildout-Footnote-2402301 +Ref: buildout-Footnote-3402363 +Ref: buildout-Footnote-4402450 +Ref: buildout-Footnote-5402497 +Ref: buildout-Footnote-6402542 +Ref: buildout-Footnote-7402588 +Node: conda402647 +Ref: key_projects conda402751 +Ref: 83402751 +Ref: key_projects id26402751 +Ref: 233402751 +Ref: conda-Footnote-1403814 +Ref: conda-Footnote-2403852 +Ref: conda-Footnote-3403896 +Ref: conda-Footnote-4403938 +Node: devpi404046 +Ref: key_projects devpi404146 +Ref: 234404146 +Ref: key_projects id27404146 +Ref: 235404146 +Ref: devpi-Footnote-1404493 +Ref: devpi-Footnote-2404530 +Ref: devpi-Footnote-3404588 +Ref: devpi-Footnote-4404637 +Node: flit404676 +Ref: key_projects flit404778 +Ref: 6a404778 +Ref: key_projects id28404778 +Ref: 236404778 +Ref: flit-Footnote-1405455 +Ref: flit-Footnote-2405502 +Ref: flit-Footnote-3405551 +Ref: flit-Footnote-4405589 +Node: enscons405650 +Ref: key_projects enscons405755 +Ref: 237405755 +Ref: key_projects id29405755 +Ref: 238405755 +Ref: enscons-Footnote-1406395 +Ref: enscons-Footnote-2406444 +Ref: enscons-Footnote-3406496 +Ref: enscons-Footnote-4406537 +Node: Hashdist406563 +Ref: key_projects hashdist406669 +Ref: 82406669 +Ref: key_projects id30406669 +Ref: 239406669 +Ref: Hashdist-Footnote-1407279 +Ref: Hashdist-Footnote-2407330 +Node: hatch407376 +Ref: key_projects hatch407478 +Ref: 20c407478 +Ref: key_projects id31407478 +Ref: 23a407478 +Ref: hatch-Footnote-1407936 +Ref: hatch-Footnote-2407974 +Node: pex408013 +Ref: key_projects id32408111 +Ref: 23b408111 +Ref: key_projects pex408111 +Ref: 23c408111 +Ref: pex-Footnote-1408561 +Ref: pex-Footnote-2408607 +Ref: pex-Footnote-3408650 +Node: pipx408687 +Ref: key_projects id33408789 +Ref: 23d408789 +Ref: key_projects pipx408789 +Ref: 23e408789 +Ref: pipx-Footnote-1408957 +Ref: pipx-Footnote-2409001 +Ref: pipx-Footnote-3409045 +Node: pip-tools409084 +Ref: key_projects id34409191 +Ref: 23f409191 +Ref: key_projects pip-tools409191 +Ref: 240409191 +Ref: pip-tools-Footnote-1409818 +Ref: pip-tools-Footnote-2409865 +Node: piwheels409909 +Ref: key_projects id35410018 +Ref: 241410018 +Ref: key_projects piwheels410018 +Ref: 242410018 +Ref: piwheels-Footnote-1410421 +Ref: piwheels-Footnote-2410455 +Ref: piwheels-Footnote-3410496 +Node: poetry410542 +Ref: key_projects id36410652 +Ref: 243410652 +Ref: key_projects poetry410652 +Ref: 244410652 +Ref: poetry-Footnote-1411159 +Ref: poetry-Footnote-2411194 +Ref: poetry-Footnote-3411242 +Node: pypiserver411283 +Ref: key_projects id37411397 +Ref: 245411397 +Ref: key_projects pypiserver411397 +Ref: 246411397 +Ref: pypiserver-Footnote-1411910 +Ref: pypiserver-Footnote-2411982 +Ref: pypiserver-Footnote-3412031 +Node: scikit-build412076 +Ref: key_projects id38412188 +Ref: 247412188 +Ref: key_projects scikit-build412188 +Ref: 248412188 +Ref: scikit-build-Footnote-1412805 +Ref: scikit-build-Footnote-2412860 +Ref: scikit-build-Footnote-3412921 +Ref: scikit-build-Footnote-4412975 +Ref: scikit-build-Footnote-5413021 +Ref: scikit-build-Footnote-6413060 +Node: shiv413099 +Ref: key_projects id39413209 +Ref: 249413209 +Ref: key_projects shiv413209 +Ref: 24a413209 +Ref: shiv-Footnote-1413548 +Ref: shiv-Footnote-2413595 +Ref: shiv-Footnote-3413636 +Ref: shiv-Footnote-4413675 +Node: Spack<2>413724 +Ref: key_projects id40413835 +Ref: 24b413835 +Ref: key_projects spack413835 +Ref: 81413835 +Ref: Spack<2>-Footnote-1414505 +Ref: Spack<2>-Footnote-2414543 +Ref: Spack<2>-Footnote-3414582 +Ref: Spack<2>-Footnote-4414660 +Node: zest releaser414729 +Ref: key_projects zest-releaser414827 +Ref: 24c414827 +Ref: key_projects zestreleaser414827 +Ref: 24d414827 +Ref: zest releaser-Footnote-1415235 +Ref: zest releaser-Footnote-2415290 +Ref: zest releaser-Footnote-3415345 +Node: Standard Library Projects415393 +Ref: key_projects standard-library-projects415512 +Ref: 24e415512 +Node: ensurepip415620 +Ref: key_projects ensurepip415723 +Ref: 7e415723 +Ref: key_projects id41415723 +Ref: 24f415723 +Ref: ensurepip-Footnote-1416094 +Ref: ensurepip-Footnote-2416151 +Node: distutils416182 +Ref: key_projects distutils416298 +Ref: c0416298 +Ref: key_projects id42416298 +Ref: 250416298 +Ref: distutils-Footnote-1417050 +Ref: distutils-Footnote-2417107 +Ref: distutils-Footnote-3417169 +Ref: distutils-Footnote-4417256 +Ref: distutils-Footnote-5417287 +Ref: distutils-Footnote-6417342 +Node: venv417401 +Ref: key_projects id44417499 +Ref: 251417499 +Ref: key_projects venv417499 +Ref: 7f417499 +Ref: venv-Footnote-1417845 +Ref: venv-Footnote-2417897 +Node: Glossary417928 +Ref: glossary doc418043 +Ref: 252418043 +Ref: glossary distribute418043 +Ref: 253418043 +Ref: glossary glossary418043 +Ref: 254418043 +Ref: glossary term-binary-distribution418066 +Ref: 194418066 +Ref: glossary term-built-distribution418180 +Ref: 63418180 +Ref: glossary term-distribution-package418685 +Ref: b418685 +Ref: glossary term-egg419456 +Ref: 7c419456 +Ref: glossary term-extension-module419678 +Ref: 257419678 +Ref: glossary term-known-good-set-kgs420094 +Ref: 258420094 +Ref: glossary term-import-package420461 +Ref: a420461 +Ref: glossary term-module420848 +Ref: 255420848 +Ref: glossary term-package-index420999 +Ref: ec420999 +Ref: glossary term-per-project-index421133 +Ref: 25a421133 +Ref: glossary term-project421346 +Ref: 1fc421346 +Ref: glossary term-pure-module422486 +Ref: 259422486 +Ref: glossary term-python-packaging-authority-pypa422652 +Ref: 25d422652 +Ref: glossary term-python-package-index-pypi422991 +Ref: 39422991 +Ref: glossary term-pypi-org423194 +Ref: 161423194 +Ref: glossary term-pyproject-toml423414 +Ref: 1d7423414 +Ref: glossary term-release423523 +Ref: 256423523 +Ref: glossary term-requirement423903 +Ref: 25e423903 +Ref: glossary term-requirement-specifier424197 +Ref: 3a424197 +Ref: glossary term-requirements-file424608 +Ref: 25f424608 +Ref: glossary term-setup-py424817 +Ref: 25b424817 +Ref: glossary term-setup-cfg424827 +Ref: 25c424827 +Ref: glossary term-source-archive424969 +Ref: 62424969 +Ref: glossary term-source-distribution-or-sdist425156 +Ref: 3d425156 +Ref: glossary term-system-package425457 +Ref: 260425457 +Ref: glossary term-version-specifier425573 +Ref: 3b425573 +Ref: glossary term-virtual-environment425933 +Ref: 22b425933 +Ref: glossary term-wheel426204 +Ref: d426204 +Ref: glossary term-working-set426397 +Ref: 261426397 +Ref: Glossary-Footnote-1426681 +Ref: Glossary-Footnote-2426762 +Ref: Glossary-Footnote-3426821 +Ref: Glossary-Footnote-4426870 +Ref: Glossary-Footnote-5426902 +Ref: Glossary-Footnote-6426937 +Ref: Glossary-Footnote-7427010 +Ref: Glossary-Footnote-8427057 +Ref: Glossary-Footnote-9427082 +Ref: Glossary-Footnote-10427107 +Ref: Glossary-Footnote-11427157 +Ref: Glossary-Footnote-12427231 +Ref: Glossary-Footnote-13427324 +Ref: Glossary-Footnote-14427394 +Ref: Glossary-Footnote-15427444 +Ref: Glossary-Footnote-16427513 +Ref: Glossary-Footnote-17427563 +Node: How to Get Support427624 +Ref: support doc427746 +Ref: 262427746 +Ref: support how-to-get-support427746 +Ref: 263427746 +Ref: How to Get Support-Footnote-1428009 +Node: Contribute to this guide428071 +Ref: contribute doc428189 +Ref: 264428189 +Ref: contribute contribute-to-this-guide428189 +Ref: 265428189 +Ref: Contribute to this guide-Footnote-1429018 +Ref: Contribute to this guide-Footnote-2429079 +Ref: Contribute to this guide-Footnote-3429146 +Ref: Contribute to this guide-Footnote-4429212 +Node: Documentation types429281 +Ref: contribute documentation-types429410 +Ref: 267429410 +Ref: contribute id4429410 +Ref: 268429410 +Node: Tutorials<2>429744 +Ref: contribute tutorials429844 +Ref: 269429844 +Ref: Tutorials<2>-Footnote-1430134 +Node: Guides<2>430188 +Ref: contribute example-tutorial-style-document430311 +Ref: 26a430311 +Ref: contribute guides430311 +Ref: 26b430311 +Node: Discussions<2>430694 +Ref: contribute discussions430819 +Ref: 26c430819 +Node: Specifications431027 +Ref: contribute specifications431134 +Ref: 26d431134 +Node: Building the guide locally431379 +Ref: contribute building-the-guide-locally431544 +Ref: 26e431544 +Ref: Building the guide locally-Footnote-1432612 +Ref: Building the guide locally-Footnote-2432658 +Node: Where the guide is deployed432728 +Ref: contribute where-the-guide-is-deployed432885 +Ref: 26f432885 +Node: Style guide433149 +Ref: contribute contributing-style-guide433271 +Ref: 266433271 +Ref: contribute style-guide433271 +Ref: 270433271 +Node: Purpose433698 +Ref: contribute purpose433781 +Ref: 271433781 +Node: Scope433971 +Ref: contribute scope434071 +Ref: 272434071 +Node: Audience434556 +Ref: contribute audience434663 +Ref: 273434663 +Node: Voice and tone435177 +Ref: contribute voice-and-tone435304 +Ref: 274435304 +Node: Conventions and mechanics436427 +Ref: contribute conventions-and-mechanics436537 +Ref: 275436537 +Node: News438731 +Ref: news doc438842 +Ref: 276438842 +Ref: news news438842 +Ref: 277438842 +Node: September 2019439381 +Ref: news september-2019439470 +Ref: 278439470 +Ref: September 2019-Footnote-1439629 +Node: August 2019439691 +Ref: news august-2019439798 +Ref: 279439798 +Ref: August 2019-Footnote-1439943 +Node: July 2019440005 +Ref: news july-2019440107 +Ref: 27a440107 +Ref: July 2019-Footnote-1440509 +Ref: July 2019-Footnote-2440571 +Ref: July 2019-Footnote-3440633 +Ref: July 2019-Footnote-4440695 +Ref: July 2019-Footnote-5440757 +Node: June 2019440819 +Ref: news june-2019440918 +Ref: 27b440918 +Ref: June 2019-Footnote-1441023 +Node: May 2019441085 +Ref: news may-2019441185 +Ref: 27c441185 +Ref: May 2019-Footnote-1441379 +Ref: May 2019-Footnote-2441441 +Node: April 2019441503 +Ref: news april-2019441604 +Ref: 27d441604 +Ref: April 2019-Footnote-1441810 +Ref: April 2019-Footnote-2441872 +Node: March 2019441934 +Ref: news march-2019442040 +Ref: 27e442040 +Ref: March 2019-Footnote-1442248 +Ref: March 2019-Footnote-2442310 +Node: February 2019442372 +Ref: news february-2019442480 +Ref: 27f442480 +Ref: February 2019-Footnote-1442744 +Ref: February 2019-Footnote-2442806 +Node: January 2019442868 +Ref: news january-2019442979 +Ref: 280442979 +Ref: January 2019-Footnote-1443216 +Ref: January 2019-Footnote-2443278 +Ref: January 2019-Footnote-3443340 +Node: December 2018443402 +Ref: news december-2018443513 +Ref: 281443513 +Node: November 2018443590 +Ref: news november-2018443701 +Ref: 282443701 +Ref: November 2018-Footnote-1444035 +Ref: November 2018-Footnote-2444097 +Ref: November 2018-Footnote-3444159 +Ref: November 2018-Footnote-4444221 +Node: October 2018444283 +Ref: news october-2018444395 +Ref: 283444395 +Ref: October 2018-Footnote-1444575 +Ref: October 2018-Footnote-2444637 +Node: September 2018444699 +Ref: news september-2018444809 +Ref: 284444809 +Ref: September 2018-Footnote-1445196 +Ref: September 2018-Footnote-2445258 +Ref: September 2018-Footnote-3445320 +Ref: September 2018-Footnote-4445382 +Ref: September 2018-Footnote-5445444 +Node: August 2018445506 +Ref: news august-2018445613 +Ref: 285445613 +Ref: August 2018-Footnote-1445997 +Ref: August 2018-Footnote-2446059 +Ref: August 2018-Footnote-3446121 +Ref: August 2018-Footnote-4446183 +Node: July 2018446245 +Ref: news july-2018446347 +Ref: 286446347 +Ref: July 2018-Footnote-1446524 +Ref: July 2018-Footnote-2446586 +Node: June 2018446648 +Ref: news june-2018446747 +Ref: 287446747 +Ref: June 2018-Footnote-1446944 +Ref: June 2018-Footnote-2447006 +Node: May 2018447068 +Ref: news may-2018447168 +Ref: 288447168 +Ref: May 2018-Footnote-1447753 +Ref: May 2018-Footnote-2447815 +Ref: May 2018-Footnote-3447877 +Ref: May 2018-Footnote-4447939 +Ref: May 2018-Footnote-5448001 +Ref: May 2018-Footnote-6448063 +Ref: May 2018-Footnote-7448125 +Node: April 2018448187 +Ref: news april-2018448288 +Ref: 289448288 +Ref: April 2018-Footnote-1448797 +Ref: April 2018-Footnote-2448859 +Ref: April 2018-Footnote-3448921 +Ref: April 2018-Footnote-4448983 +Ref: April 2018-Footnote-5449045 +Ref: April 2018-Footnote-6449107 +Ref: April 2018-Footnote-7449169 +Node: March 2018449231 +Ref: news march-2018449337 +Ref: 28a449337 +Ref: March 2018-Footnote-1449698 +Ref: March 2018-Footnote-2449760 +Ref: March 2018-Footnote-3449822 +Ref: March 2018-Footnote-4449884 +Node: February 2018449946 +Ref: news february-2018450054 +Ref: 28b450054 +Ref: February 2018-Footnote-1450519 +Ref: February 2018-Footnote-2450581 +Ref: February 2018-Footnote-3450643 +Ref: February 2018-Footnote-4450705 +Ref: February 2018-Footnote-5450767 +Node: January 2018450829 +Ref: news january-2018450940 +Ref: 28c450940 +Ref: January 2018-Footnote-1451133 +Ref: January 2018-Footnote-2451195 +Node: December 2017451257 +Ref: news december-2017451368 +Ref: 28d451368 +Ref: December 2017-Footnote-1451696 +Ref: December 2017-Footnote-2451758 +Ref: December 2017-Footnote-3451820 +Ref: December 2017-Footnote-4451882 +Node: November 2017451944 +Ref: news november-2017452055 +Ref: 28e452055 +Ref: November 2017-Footnote-1452512 +Ref: November 2017-Footnote-2452574 +Ref: November 2017-Footnote-3452636 +Ref: November 2017-Footnote-4452698 +Node: October 2017452760 +Ref: news october-2017452872 +Ref: 28f452872 +Ref: October 2017-Footnote-1453207 +Ref: October 2017-Footnote-2453269 +Ref: October 2017-Footnote-3453331 +Node: September 2017453393 +Ref: news september-2017453503 +Ref: 290453503 +Ref: September 2017-Footnote-1453744 +Ref: September 2017-Footnote-2453806 +Node: August 2017453868 +Ref: news august-2017453975 +Ref: 291453975 +Ref: August 2017-Footnote-1454271 +Ref: August 2017-Footnote-2454333 +Ref: August 2017-Footnote-3454395 +Node: July 2017454457 +Ref: news july-2017454559 +Ref: 292454559 +Ref: July 2017-Footnote-1455074 +Ref: July 2017-Footnote-2455136 +Ref: July 2017-Footnote-3455198 +Ref: July 2017-Footnote-4455260 +Ref: July 2017-Footnote-5455322 +Node: June 2017455384 +Ref: news june-2017455483 +Ref: 293455483 +Ref: June 2017-Footnote-1456267 +Ref: June 2017-Footnote-2456329 +Ref: June 2017-Footnote-3456391 +Ref: June 2017-Footnote-4456453 +Ref: June 2017-Footnote-5456515 +Ref: June 2017-Footnote-6456577 +Ref: June 2017-Footnote-7456639 +Ref: June 2017-Footnote-8456701 +Ref: June 2017-Footnote-9456763 +Ref: June 2017-Footnote-10456825 +Ref: June 2017-Footnote-11456888 +Ref: June 2017-Footnote-12456951 +Node: May 2017457014 +Ref: news may-2017457114 +Ref: 294457114 +Ref: May 2017-Footnote-1457437 +Ref: May 2017-Footnote-2457499 +Ref: May 2017-Footnote-3457561 +Node: April 2017457623 +Ref: news april-2017457724 +Ref: 295457724 +Ref: April 2017-Footnote-1458759 +Ref: April 2017-Footnote-2458821 +Ref: April 2017-Footnote-3458883 +Ref: April 2017-Footnote-4458945 +Ref: April 2017-Footnote-5459007 +Ref: April 2017-Footnote-6459069 +Ref: April 2017-Footnote-7459131 +Ref: April 2017-Footnote-8459193 +Ref: April 2017-Footnote-9459255 +Ref: April 2017-Footnote-10459317 +Ref: April 2017-Footnote-11459380 +Node: March 2017459443 +Ref: news march-2017459549 +Ref: 296459549 +Ref: March 2017-Footnote-1459687 +Node: February 2017459749 +Ref: news february-2017459836 +Ref: 297459836 +Ref: February 2017-Footnote-1460253 +Ref: February 2017-Footnote-2460302 +Ref: February 2017-Footnote-3460364 +Ref: February 2017-Footnote-4460424 +Node: Get started460448 +Ref: index get-started460545 +Ref: 298460545 +Ref: index python-packaging-authority460545 +Ref: 299460545 +Node: Learn more461212 +Ref: index learn-more461310 +Ref: 29a461310 +Node: Index461988 + +End Tag Table + + +Local Variables: +coding: utf-8 +End: diff --git a/assets/info/sed.info.gz b/assets/info/sed.info.gz new file mode 100644 index 00000000..5dd99f17 Binary files /dev/null and b/assets/info/sed.info.gz differ diff --git a/assets/info/setuptools.info b/assets/info/setuptools.info new file mode 100644 index 00000000..48a55082 --- /dev/null +++ b/assets/info/setuptools.info @@ -0,0 +1,26390 @@ +This is setuptools.info, produced by makeinfo version 6.5 from +setuptools.texi. + + setuptools 50.3.2, Dec 04, 2020 + + Python Packaging Authority + + Copyright © Python Packaging Authority + +INFO-DIR-SECTION Miscellaneous +START-INFO-DIR-ENTRY +* setuptools: (setuptools.info). One line description of project +END-INFO-DIR-ENTRY + + + Generated by Sphinx 2.3.1. + + +File: setuptools.info, Node: Top, Next: Building and Distributing Packages with Setuptools, Up: (dir) + +setuptools +********** + + setuptools 50.3.2, Dec 04, 2020 + + Python Packaging Authority + + Copyright © Python Packaging Authority + +Setuptools is a fully-featured, actively-maintained, and stable library +designed to facilitate packaging Python projects. + +Documentation content: + +* Menu: + +* Building and Distributing Packages with Setuptools:: +* Build System Support:: +* Package Discovery and Resource Access using pkg_resources:: +* Keywords:: +* Roadmap:: +* Building and Distributing Packages with Setuptools: Building and Distributing Packages with Setuptools<2>. +* Development on Setuptools:: +* Guides on backward compatibility & deprecated practice:: +* History: History<2>. +* Credits:: +* Index:: + + — The Detailed Node Listing — + +Building and Distributing Packages with Setuptools + +* Transition to PEP517:: + +Transition to PEP517 + +* setuptools Quickstart:: +* Package Discovery and Namespace Package:: +* Entry Points:: +* Dependencies Management in Setuptools:: +* Data Files Support:: +* “Development Mode”:: +* Tagging and “Daily Build” or “Snapshot” Releases:: +* Generating Source Distributions:: +* Distributing Extensions compiled with Cython:: +* Specifying Your Project’s Version:: +* Creating distutils Extensions:: +* Configuring setup() using setup.cfg files: Configuring setup using setup cfg files. +* New and Changed setup() Keywords: New and Changed setup Keywords. +* Command Reference:: +* Using setuptools to package and distribute your project:: +* Automatic Resource Extraction:: +* Defining Additional Metadata:: +* Setting the zip_safe flag:: + +setuptools Quickstart + +* Installation:: +* Python packaging at a glance:: +* Basic Use:: +* Automatic package discovery:: +* Entry points and automatic script creation:: +* Dependency management:: +* Including Data Files:: +* Development mode:: +* Uploading your package to PyPI:: +* Transitioning from setup.py to setup.cfg: Transitioning from setup py to setup cfg. +* Resources on Python packaging:: + +Package Discovery and Namespace Package + +* Using find; or find_packages: Using find or find_packages. +* Using find_namespace; or find_namespace_packages: Using find_namespace or find_namespace_packages. +* Legacy Namespace Packages:: + +Legacy Namespace Packages + +* pkg_resource style namespace package:: +* pkgutil style namespace package:: + +Entry Points + +* Console Scripts:: +* Advertising Behavior:: +* Dependency Management:: + +Dependencies Management in Setuptools + +* Build system requirement:: +* Declaring required dependency:: +* Optional dependencies:: +* Python requirement:: + +Build system requirement + +* Package requirement:: + +Declaring required dependency + +* Platform specific dependencies:: +* Dependencies that aren’t in PyPI:: + +Data Files Support + +* Accessing Data Files at Runtime:: +* Non-Package Data Files:: + +Generating Source Distributions + +* Making “Official” (Non-Snapshot) Releases: Making “Official” Non-Snapshot Releases. + +Creating distutils Extensions + +* Adding Commands:: +* Adding setup() Arguments: Adding setup Arguments. +* Customizing Distribution Options:: +* Adding new EGG-INFO Files:: +* Adding Support for Revision Control Systems:: + +Configuring setup() using setup.cfg files + +* Using a src/ layout:: +* Specifying values:: + +Specifying values + +* Metadata:: +* Options:: + +Command Reference + +* alias - Define shortcuts for commonly used commands:: +* bdist_egg - Create a Python Egg for the project:: +* develop - Deploy the project source in “Development Mode”:: +* egg_info - Create egg metadata and set build tags:: +* rotate - Delete outdated distribution files:: +* saveopts - Save used options to a configuration file:: +* setopt - Set a distutils or setuptools option in a config file:: +* test - Build package and run a unittest suite:: +* upload - Upload source and/or egg distributions to PyPI:: + +egg_info - Create egg metadata and set build tags + +* Release Tagging Options:: +* Other egg_info Options:: +* egg_info Examples:: + +saveopts - Save used options to a configuration file + +* Configuration File Options:: + +Build System Support + +* What is it?:: +* How to use it?:: + +Package Discovery and Resource Access using pkg_resources + +* Overview:: +* API Reference:: + +API Reference + +* Namespace Package Support:: +* WorkingSet Objects:: +* Environment Objects:: +* Requirement Objects:: +* Entry Points: Entry Points<2>. +* Distribution Objects:: +* ResourceManager API:: +* Metadata API:: +* Exceptions:: +* Supporting Custom Importers:: +* Utility Functions:: + +WorkingSet Objects + +* Basic WorkingSet Methods:: +* WorkingSet Methods and Attributes:: +* Receiving Change Notifications:: +* Locating Plugins:: + +Requirement Objects + +* Requirements Parsing:: +* Requirement Methods and Attributes:: + +Entry Points + +* Convenience API:: +* Creating and Parsing:: +* EntryPoint Objects:: + +Distribution Objects + +* Getting or Creating Distributions:: +* Distribution Attributes:: +* Distribution Methods:: + +ResourceManager API + +* Basic Resource Access:: +* Resource Extraction:: +* “Provider” Interface:: + +Metadata API + +* IMetadataProvider Methods:: + +Supporting Custom Importers + +* IResourceProvider:: +* Built-in Resource Providers:: + +Utility Functions + +* Parsing Utilities:: +* Platform Utilities:: +* PEP 302 Utilities:: +* File/Path Utilities:: +* History:: + +Building and Distributing Packages with Setuptools + +* Developer’s Guide:: + +Developer’s Guide + +* TRANSITIONAL NOTE:: + +TRANSITIONAL NOTE + +* setup.cfg-only projects: setup cfg-only projects. +* Configuration API:: +* Mailing List and Bug Tracker:: + +Development on Setuptools + +* Developer’s Guide for Setuptools:: +* Release Process:: + +Developer’s Guide for Setuptools + +* Recommended Reading:: +* Project Management:: +* Authoring Tickets:: +* Making a pull request:: +* Auto-Merge Requests:: +* Testing:: +* Semantic Versioning:: +* Building Documentation:: +* Vendored Dependencies:: + +Making a pull request + +* Adding change notes with your PRs:: +* Alright! So how to add a news fragment?:: +* Examples for adding changelog entries to your Pull Requests:: + +Release Process + +* Release Frequency:: +* Release Managers:: + +Guides on backward compatibility & deprecated practice + +* Supporting both Python 2 and Python 3 with Setuptools:: +* The Internal Structure of Python Eggs:: +* Easy Install:: +* Porting from Distutils:: +* “Eggsecutable” Scripts:: + +Supporting both Python 2 and Python 3 with Setuptools + +* Using 2to3:: +* Distributing Python 3 modules:: +* Advanced features:: + +Using 2to3 + +* Differential conversion:: + +The Internal Structure of Python Eggs + +* Eggs and their Formats:: +* Standard Metadata:: +* Other Technical Considerations:: + +Eggs and their Formats + +* Code and Resources:: +* Project Metadata:: +* Filename-Embedded Metadata:: +* Egg Links:: + +Standard Metadata + +* .txt File Formats: txt File Formats. +* Dependency Metadata:: +* namespace_packages.txt – Namespace Package Metadata: namespace_packages txt – Namespace Package Metadata. +* entry_points.txt – “Entry Point”/Plugin Metadata: entry_points txt – “Entry Point”/Plugin Metadata. +* The scripts Subdirectory:: +* Zip Support Metadata:: +* top_level.txt – Conflict Management Metadata: top_level txt – Conflict Management Metadata. +* SOURCES.txt – Source Files Manifest: SOURCES txt – Source Files Manifest. + +Dependency Metadata + +* requires.txt: requires txt. +* setup_requires.txt: setup_requires txt. +* dependency_links.txt: dependency_links txt. +* depends.txt – Obsolete, do not create!: depends txt – Obsolete do not create!. + +Zip Support Metadata + +* native_libs.txt: native_libs txt. +* eager_resources.txt: eager_resources txt. +* zip-safe and not-zip-safe:: + +Other Technical Considerations + +* Zip File Issues:: +* Installation and Path Management Issues:: + +Zip File Issues + +* The Extraction Process:: +* Extension Import Wrappers:: + +Installation and Path Management Issues + +* Script Wrappers:: + +Easy Install + +* Using “Easy Install”:: +* Reference Manual:: + +Using “Easy Install” + +* Installing “Easy Install”:: +* Downloading and Installing a Package:: +* Upgrading a Package:: +* Changing the Active Version:: +* Uninstalling Packages:: +* Managing Scripts:: +* Executables and Launchers:: +* Tips & Techniques:: +* Password-Protected Sites:: +* Using .pypirc Credentials: Using pypirc Credentials. + +Installing “Easy Install” + +* Troubleshooting:: +* Windows Notes:: + +Executables and Launchers + +* Windows Executable Launcher:: +* Natural Script Launcher:: + +Tips & Techniques + +* Multiple Python Versions:: +* Restricting Downloads with --allow-hosts:: +* Installing on Un-networked Machines:: +* Packaging Others’ Projects As Eggs:: +* Creating your own Package Index:: + +Using .pypirc Credentials + +* Controlling Build Options:: +* Editing and Viewing Source Packages:: +* Dealing with Installation Conflicts:: +* Compressed Installation:: + +Reference Manual + +* Configuration Files:: +* Command-Line Options:: +* Custom Installation Locations:: +* Package Index “API”:: + +Custom Installation Locations + +* Use the “–user” option:: +* Use the “–user” option and customize “PYTHONUSERBASE”:: +* Use “virtualenv”:: + +Porting from Distutils + +* Prefer Setuptools:: + +History + +* v50.3.2: v50 3 2. +* v50.3.1: v50 3 1. +* v50.3.0: v50 3 0. +* v50.2.0: v50 2 0. +* v50.1.0: v50 1 0. +* v50.0.3: v50 0 3. +* v50.0.2: v50 0 2. +* v50.0.1: v50 0 1. +* v50.0.0: v50 0 0. +* v49.6.0: v49 6 0. +* v49.5.0: v49 5 0. +* v49.4.0: v49 4 0. +* v49.3.2: v49 3 2. +* v49.3.1: v49 3 1. +* v49.3.0: v49 3 0. +* v49.2.1: v49 2 1. +* v49.2.0: v49 2 0. +* v49.1.3: v49 1 3. +* v49.1.2: v49 1 2. +* v49.1.1: v49 1 1. +* v49.0.1: v49 0 1. +* v49.1.0: v49 1 0. +* v49.0.0: v49 0 0. +* v48.0.0: v48 0 0. +* v47.3.2: v47 3 2. +* v47.3.1: v47 3 1. +* v47.3.0: v47 3 0. +* v47.2.0: v47 2 0. +* v47.1.1: v47 1 1. +* v44.1.1: v44 1 1. +* v47.1.0: v47 1 0. +* v47.0.0: v47 0 0. +* v46.4.0: v46 4 0. +* v46.3.1: v46 3 1. +* v46.3.0: v46 3 0. +* v46.2.0: v46 2 0. +* v46.1.3: v46 1 3. +* v46.1.2: v46 1 2. +* v46.1.1: v46 1 1. +* v46.1.0: v46 1 0. +* v44.1.0: v44 1 0. +* v46.0.0: v46 0 0. +* v45.3.0: v45 3 0. +* v45.2.0: v45 2 0. +* v45.1.0: v45 1 0. +* v45.0.0: v45 0 0. +* v44.0.0: v44 0 0. +* v43.0.0: v43 0 0. +* v42.0.2: v42 0 2. +* v42.0.1: v42 0 1. +* v42.0.0: v42 0 0. +* v41.6.0: v41 6 0. +* v41.5.1: v41 5 1. +* v41.5.0: v41 5 0. +* v41.4.0: v41 4 0. +* v41.3.0: v41 3 0. +* v41.2.0: v41 2 0. +* v41.1.0: v41 1 0. +* v41.0.1: v41 0 1. +* v41.0.0: v41 0 0. +* v40.9.0: v40 9 0. +* v40.8.0: v40 8 0. +* v40.7.3: v40 7 3. +* v40.7.2: v40 7 2. +* v40.7.1: v40 7 1. +* v40.7.0: v40 7 0. +* v40.6.3: v40 6 3. +* v40.6.2: v40 6 2. +* v40.6.1: v40 6 1. +* v40.6.0: v40 6 0. +* v40.5.0: v40 5 0. +* v40.4.3: v40 4 3. +* v40.4.2: v40 4 2. +* v40.4.1: v40 4 1. +* v40.4.0: v40 4 0. +* v40.3.0: v40 3 0. +* v40.2.0: v40 2 0. +* v40.1.1: v40 1 1. +* v40.1.0: v40 1 0. +* v40.0.0: v40 0 0. +* v39.2.0: v39 2 0. +* v39.1.0: v39 1 0. +* v39.0.1: v39 0 1. +* v39.0.0: v39 0 0. +* v38.7.0: v38 7 0. +* v38.6.1: v38 6 1. +* v38.6.0: v38 6 0. +* v38.5.2: v38 5 2. +* v38.5.1: v38 5 1. +* v38.5.0: v38 5 0. +* v38.4.1: v38 4 1. +* v38.4.0: v38 4 0. +* v38.3.0: v38 3 0. +* v38.2.5: v38 2 5. +* v38.2.4: v38 2 4. +* v38.2.3: v38 2 3. +* v38.2.2: v38 2 2. +* v38.2.1: v38 2 1. +* v38.2.0: v38 2 0. +* v38.1.0: v38 1 0. +* v38.0.0: v38 0 0. +* v37.0.0: v37 0 0. +* v36.8.0: v36 8 0. +* v36.7.3: v36 7 3. +* v36.7.2: v36 7 2. +* v36.7.1: v36 7 1. +* v36.7.0: v36 7 0. +* v36.6.1: v36 6 1. +* v36.6.0: v36 6 0. +* v36.5.0: v36 5 0. +* v36.4.0: v36 4 0. +* v36.3.0: v36 3 0. +* v36.2.7: v36 2 7. +* v36.2.6: v36 2 6. +* v36.2.5: v36 2 5. +* v36.2.4: v36 2 4. +* v36.2.3: v36 2 3. +* v36.2.2: v36 2 2. +* v36.2.1: v36 2 1. +* v36.2.0: v36 2 0. +* v36.1.1: v36 1 1. +* v36.1.0: v36 1 0. +* v36.0.1: v36 0 1. +* v36.0.0: v36 0 0. +* v35.0.2: v35 0 2. +* v35.0.1: v35 0 1. +* v35.0.0: v35 0 0. +* v34.4.1: v34 4 1. +* v34.4.0: v34 4 0. +* v34.3.3: v34 3 3. +* v34.3.2: v34 3 2. +* v34.3.1: v34 3 1. +* v34.3.0: v34 3 0. +* v34.2.0: v34 2 0. +* v34.1.1: v34 1 1. +* v34.1.0: v34 1 0. +* v34.0.3: v34 0 3. +* v34.0.2: v34 0 2. +* v34.0.1: v34 0 1. +* v34.0.0: v34 0 0. +* v33.1.1: v33 1 1. +* v33.1.0: v33 1 0. +* v33.0.0: v33 0 0. +* v32.3.1: v32 3 1. +* v32.3.0: v32 3 0. +* v32.2.0: v32 2 0. +* v32.1.3: v32 1 3. +* v32.1.2: v32 1 2. +* v32.1.1: v32 1 1. +* v32.1.0: v32 1 0. +* v32.0.0: v32 0 0. +* v31.0.1: v31 0 1. +* v31.0.0: v31 0 0. +* v30.4.0: v30 4 0. +* v30.3.0: v30 3 0. +* v30.2.1: v30 2 1. +* v30.2.0: v30 2 0. +* v30.1.0: v30 1 0. +* v30.0.0: v30 0 0. +* v29.0.1: v29 0 1. +* v29.0.0: v29 0 0. +* v28.8.0: v28 8 0. +* v28.7.1: v28 7 1. +* v28.7.0: v28 7 0. +* v28.6.1: v28 6 1. +* v28.6.0: v28 6 0. +* v28.5.0: v28 5 0. +* v28.4.0: v28 4 0. +* v28.3.0: v28 3 0. +* v28.1.0: v28 1 0. +* v28.0.0: v28 0 0. +* v27.3.1: v27 3 1. +* v27.3.0: v27 3 0. +* v27.2.0: v27 2 0. +* v27.1.2: v27 1 2. +* v27.1.1: v27 1 1. +* v27.1.0: v27 1 0. +* v27.0.0: v27 0 0. +* v26.1.1: v26 1 1. +* v26.1.0: v26 1 0. +* v26.0.0: v26 0 0. +* v25.4.0: v25 4 0. +* v25.3.0: v25 3 0. +* v25.2.0: v25 2 0. +* v25.1.6: v25 1 6. +* v25.1.5: v25 1 5. +* v25.1.4: v25 1 4. +* v25.1.3: v25 1 3. +* v25.1.2: v25 1 2. +* v25.1.1: v25 1 1. +* v25.1.0: v25 1 0. +* v25.0.2: v25 0 2. +* v25.0.1: v25 0 1. +* v25.0.0: v25 0 0. +* v24.3.1: v24 3 1. +* v24.3.0: v24 3 0. +* v24.2.1: v24 2 1. +* v24.2.0: v24 2 0. +* v24.1.1: v24 1 1. +* v24.1.0: v24 1 0. +* v24.0.3: v24 0 3. +* v24.0.2: v24 0 2. +* v24.0.1: v24 0 1. +* v24.0.0: v24 0 0. +* v23.2.1: v23 2 1. +* v23.1.0: v23 1 0. +* v23.0.0: v23 0 0. +* v22.0.5: v22 0 5. +* v22.0.4: v22 0 4. +* v22.0.3: v22 0 3. +* v22.0.2: v22 0 2. +* v22.0.1: v22 0 1. +* v22.0.0: v22 0 0. +* v21.2.2: v21 2 2. +* v21.2.1: v21 2 1. +* v21.2.0: v21 2 0. +* v21.1.0: v21 1 0. +* v21.0.0: v21 0 0. +* v20.10.0: v20 10 0. +* v20.9.0: v20 9 0. +* v20.8.1: v20 8 1. +* v20.8.0: v20 8 0. +* v20.7.0: v20 7 0. +* v20.6.8: v20 6 8. +* v20.6.7: v20 6 7. +* v20.6.6: v20 6 6. +* v20.6.0: v20 6 0. +* 20.5: 20 5. +* 20.4: 20 4. +* 20.3.1: 20 3 1. +* 20.3: 20 3. +* 20.2.2: 20 2 2. +* 20.2.1: 20 2 1. +* 20.2: 20 2. +* 20.1.1: 20 1 1. +* 20.1: 20 1. +* 20.0: 20 0. +* 19.7: 19 7. +* 19.6.2: 19 6 2. +* 19.6.1: 19 6 1. +* 19.6: 19 6. +* 19.5: 19 5. +* 19.4.1: 19 4 1. +* 19.4: 19 4. +* 19.3: 19 3. +* 19.2: 19 2. +* 19.1.1: 19 1 1. +* 19.1: 19 1. +* 19.0: 19 0. +* 18.8.1: 18 8 1. +* 18.8: 18 8. +* 18.7.1: 18 7 1. +* 18.7: 18 7. +* 18.6.1: 18 6 1. +* 18.6: 18 6. +* 18.5: 18 5. +* 18.4: 18 4. +* 18.3.2: 18 3 2. +* 18.3.1: 18 3 1. +* 18.3: 18 3. +* 18.2: 18 2. +* 18.1: 18 1. +* 18.0.1: 18 0 1. +* 18.0: 18 0. +* 17.1.1: 17 1 1. +* 17.1: 17 1. +* 17.0: 17 0. +* 16.0: 16 0. +* 15.2: 15 2. +* 15.1: 15 1. +* 15.0: 15 0. +* 14.3.1: 14 3 1. +* 14.3: 14 3. +* 14.2: 14 2. +* 14.1.1: 14 1 1. +* 14.1: 14 1. +* 14.0: 14 0. +* 13.0.2: 13 0 2. +* 13.0.1: 13 0 1. +* 13.0: 13 0. +* 12.4: 12 4. +* 12.3: 12 3. +* 12.2: 12 2. +* 12.1: 12 1. +* 12.0.5: 12 0 5. +* 12.0.4: 12 0 4. +* 12.0.3: 12 0 3. +* 12.0.2: 12 0 2. +* 12.0.1: 12 0 1. +* 12.0: 12 0. +* 11.3.1: 11 3 1. +* 11.3: 11 3. +* 11.2: 11 2. +* 11.1: 11 1. +* 11.0: 11 0. +* 10.2.1: 10 2 1. +* 10.2: 10 2. +* 10.1: 10 1. +* 10.0.1: 10 0 1. +* 10.0: 10 0. +* 9.1: 9 1. +* 9.0.1: 9 0 1. +* 9.0: 9 0. +* 8.4: 8 4. +* 8.3: 8 3. +* 8.2.1: 8 2 1. +* 8.2: 8 2. +* 8.1: 8 1. +* 8.0.4: 8 0 4. +* 8.0.3: 8 0 3. +* 8.0.2: 8 0 2. +* 8.0.1: 8 0 1. +* 8.0: 8 0. +* 7.0: 7 0. +* 6.1: 6 1. +* 6.0.2: 6 0 2. +* 6.0.1: 6 0 1. +* 6.0: 6 0. +* 5.8: 5 8. +* 5.7: 5 7. +* 5.6: 5 6. +* 5.5.1: 5 5 1. +* 5.5: 5 5. +* 5.4.2: 5 4 2. +* 5.4.1: 5 4 1. +* 5.4: 5 4. +* 5.3: 5 3. +* 5.2: 5 2. +* 5.1: 5 1. +* 5.0.2: 5 0 2. +* 5.0.1: 5 0 1. +* 5.0: 5 0. +* 3.7.1 and 3.8.1 and 4.0.1: 3 7 1 and 3 8 1 and 4 0 1. +* 4.0: 4 0. +* 3.8: 3 8. +* 3.7: 3 7. +* 3.6: 3 6. +* 3.5.2: 3 5 2. +* 3.5.1: 3 5 1. +* 3.5: 3 5. +* 3.4.4: 3 4 4. +* 3.4.3: 3 4 3. +* 3.4.2: 3 4 2. +* 3.4.1: 3 4 1. +* 3.4: 3 4. +* 3.3: 3 3. +* 3.2: 3 2. +* 3.1: 3 1. +* 3.0.2: 3 0 2. +* 3.0.1: 3 0 1. +* 3.0: 3 0. +* 2.2: 2 2. +* 2.1.2: 2 1 2. +* 2.1.1: 2 1 1. +* 2.1: 2 1. +* 2.0.2: 2 0 2. +* 2.0.1: 2 0 1. +* 2.0: 2 0. +* 1.4.2: 1 4 2. +* 1.4.1: 1 4 1. +* 1.4: 1 4. +* 1.3.2: 1 3 2. +* 1.3.1: 1 3 1. +* 1.3: 1 3. +* 1.2: 1 2. +* 1.1.7: 1 1 7. +* 1.1.6: 1 1 6. +* 1.1.5: 1 1 5. +* 1.1.4: 1 1 4. +* 1.1.3: 1 1 3. +* 1.1.2: 1 1 2. +* 1.1.1: 1 1 1. +* 1.1: 1 1. +* 1.0: 1 0. +* 0.9.8: 0 9 8. +* 0.9.7: 0 9 7. +* 0.9.6: 0 9 6. +* 0.9.5: 0 9 5. +* 0.9.4: 0 9 4. +* 0.9.3: 0 9 3. +* 0.9.2: 0 9 2. +* 0.9.1: 0 9 1. +* 0.9: 0 9. +* 0.8: 0 8. +* 0.7.8: 0 7 8. +* 0.7.7: 0 7 7. +* 0.7.6: 0 7 6. +* 0.7.5: 0 7 5. +* 0.7.4: 0 7 4. +* 0.7.3: 0 7 3. +* 0.7.2: 0 7 2. +* 0.7.1: 0 7 1. +* 0.7: 0 7. +* 0.7b4: 0 7b4. +* 0.6.49: 0 6 49. +* 0.6.48: 0 6 48. +* 0.6.47: 0 6 47. +* 0.6.46: 0 6 46. +* 0.6.45: 0 6 45. +* 0.6.44: 0 6 44. +* 0.6.43: 0 6 43. +* 0.6.42: 0 6 42. +* 0.6.41: 0 6 41. +* 0.6.40: 0 6 40. +* 0.6.39: 0 6 39. +* 0.6.38: 0 6 38. +* 0.6.37: 0 6 37. +* 0.6.36: 0 6 36. +* 0.6.35: 0 6 35. +* 0.6.34: 0 6 34. +* 0.6.33: 0 6 33. +* 0.6.32: 0 6 32. +* 0.6.31: 0 6 31. +* 0.6.30: 0 6 30. +* 0.6.29: 0 6 29. +* 0.6.28: 0 6 28. +* 0.6.27: 0 6 27. +* 0.6.26: 0 6 26. +* 0.6.25: 0 6 25. +* 0.6.24: 0 6 24. +* 0.6.23: 0 6 23. +* 0.6.21: 0 6 21. +* 0.6.20: 0 6 20. +* 0.6.19: 0 6 19. +* 0.6.18: 0 6 18. +* 0.6.17: 0 6 17. +* 0.6.16: 0 6 16. +* 0.6.15: 0 6 15. +* 0.6.14: 0 6 14. +* 0.6.13: 0 6 13. +* 0.6.12: 0 6 12. +* 0.6.11: 0 6 11. +* 0.6.10: 0 6 10. +* 0.6.9: 0 6 9. +* 0.6.8: 0 6 8. +* 0.6.7: 0 6 7. +* 0.6.6: 0 6 6. +* 0.6.5: 0 6 5. +* 0.6.4: 0 6 4. +* 0.6.3: 0 6 3. +* 0.6.2: 0 6 2. +* 0.6.1: 0 6 1. +* 0.6: 0 6. +* 0.6c9: 0 6c9. +* 0.6c7: 0 6c7. +* 0.6c6: 0 6c6. +* 0.6c5: 0 6c5. +* 0.6c4: 0 6c4. +* 0.6c3: 0 6c3. +* 0.6c2: 0 6c2. +* 0.6c1: 0 6c1. +* 0.6b4: 0 6b4. +* 0.6b3: 0 6b3. +* 0.6b2: 0 6b2. +* 0.6b1: 0 6b1. +* 0.6a11: 0 6a11. +* 0.6a10: 0 6a10. +* 0.6a9: 0 6a9. +* 0.6a8: 0 6a8. +* 0.6a7: 0 6a7. +* 0.6a6: 0 6a6. +* 0.6a5: 0 6a5. +* 0.6a3: 0 6a3. +* 0.6a2: 0 6a2. +* 0.6a1: 0 6a1. +* 0.5a12: 0 5a12. +* 0.5a11: 0 5a11. +* 0.5a10: 0 5a10. +* 0.5a9: 0 5a9. +* 0.5a8: 0 5a8. +* 0.5a7: 0 5a7. +* 0.5a6: 0 5a6. +* 0.5a5: 0 5a5. +* 0.5a4: 0 5a4. +* 0.5a3: 0 5a3. +* 0.5a2: 0 5a2. +* 0.5a1: 0 5a1. +* 0.4a4: 0 4a4. +* 0.4a3: 0 4a3. +* 0.4a2: 0 4a2. +* 0.4a1: 0 4a1. +* 0.3a4: 0 3a4. +* 0.3a3: 0 3a3. +* 0.3a2: 0 3a2. +* 0.3a1: 0 3a1. + +v50.3.2 + +* Documentation changes:: +* Misc:: + +v50.3.1 + +* Documentation changes: Documentation changes<2>. +* Misc: Misc<2>. + +v50.3.0 + +* Changes:: + +v50.2.0 + +* Changes: Changes<2>. + +v50.1.0 + +* Changes: Changes<3>. + +v50.0.3 + +* Misc: Misc<3>. + +v50.0.2 + +* Misc: Misc<4>. + +v50.0.1 + +* Misc: Misc<5>. + +v50.0.0 + +* Breaking Changes:: +* Changes: Changes<4>. + +v49.6.0 + +* Changes: Changes<5>. + +v49.5.0 + +* Changes: Changes<6>. + +v49.4.0 + +* Changes: Changes<7>. + +v49.3.2 + +* Documentation changes: Documentation changes<3>. +* Misc: Misc<6>. + +v49.3.1 + +* Changes: Changes<8>. + +v49.3.0 + +* Changes: Changes<9>. + +v49.2.1 + +* Misc: Misc<7>. + +v49.2.0 + +* Changes: Changes<10>. + +v49.1.3 + +* Misc: Misc<8>. + +v49.1.2 + +* Changes: Changes<11>. + +v49.1.1 + +* Misc: Misc<9>. + +v49.0.1 + +* Misc: Misc<10>. + +v49.1.0 + +* Changes: Changes<12>. + +v49.0.0 + +* Breaking Changes: Breaking Changes<2>. +* Changes: Changes<13>. +* Misc: Misc<11>. + +v48.0.0 + +* Breaking Changes: Breaking Changes<3>. + +v47.3.2 + +* Misc: Misc<12>. + +v47.3.1 + +* Misc: Misc<13>. + +v47.3.0 + +* Changes: Changes<14>. +* Misc: Misc<14>. + +v47.2.0 + +* Changes: Changes<15>. + +v47.1.1 + +* Documentation changes: Documentation changes<4>. +* Incorporate changes from v44.1.1;: Incorporate changes from v44 1 1. + +v44.1.1 + +* Misc: Misc<15>. + +v47.1.0 + +* Changes: Changes<16>. + +v47.0.0 + +* Breaking Changes: Breaking Changes<4>. +* Changes: Changes<17>. + +v46.4.0 + +* Changes: Changes<18>. + +v46.3.0 + +* Changes: Changes<19>. +* Misc: Misc<16>. + +v46.2.0 + +* Changes: Changes<20>. +* Documentation changes: Documentation changes<5>. +* Misc: Misc<17>. + +v46.1.2 + +* Misc: Misc<18>. + +v46.1.0 + +* Changes: Changes<21>. +* Incorporate changes from v44.1.0;: Incorporate changes from v44 1 0. + +v44.1.0 + +* Changes: Changes<22>. + +v46.0.0 + +* Breaking Changes: Breaking Changes<5>. +* Changes: Changes<23>. +* Documentation changes: Documentation changes<6>. +* Misc: Misc<19>. + +v45.3.0 + +* Changes: Changes<24>. + +v45.2.0 + +* Changes: Changes<25>. +* Misc: Misc<20>. + +v45.1.0 + +* Changes: Changes<26>. + +v45.0.0 + +* Breaking Changes: Breaking Changes<6>. +* Changes: Changes<27>. + +v44.0.0 + +* Breaking Changes: Breaking Changes<7>. + +v43.0.0 + +* Breaking Changes: Breaking Changes<8>. +* Changes: Changes<28>. + +v42.0.2 + +* Changes: Changes<29>. + +v42.0.1 + +* Changes: Changes<30>. + +v42.0.0 + +* Breaking Changes: Breaking Changes<9>. +* Changes: Changes<31>. + +v41.6.0 + +* Changes: Changes<32>. + +v41.5.1 + +* Changes: Changes<33>. + +v41.5.0 + +* Changes: Changes<34>. +* Documentation changes: Documentation changes<7>. +* Misc: Misc<21>. + +v41.4.0 + +* Changes: Changes<35>. + +v41.3.0 + +* Changes: Changes<36>. +* Misc: Misc<22>. + +v41.2.0 + +* Changes: Changes<37>. +* Misc: Misc<23>. + +v41.1.0 + +* Misc: Misc<24>. +* Documentation changes: Documentation changes<8>. + +v41.0.1 + +* Changes: Changes<38>. + +v41.0.0 + +* Breaking Changes: Breaking Changes<10>. + +v40.9.0 + +* Changes: Changes<39>. +* Documentation changes: Documentation changes<9>. + +v40.8.0 + +* Changes: Changes<40>. + +v40.7.3 + +* Changes: Changes<41>. + +v40.7.2 + +* Changes: Changes<42>. + +v40.7.1 + +* Changes: Changes<43>. + +v40.7.0 + +* Breaking Changes: Breaking Changes<11>. +* Changes: Changes<44>. + +v40.6.3 + +* Changes: Changes<45>. + +v40.6.2 + +* Changes: Changes<46>. + +v40.6.1 + +* Changes: Changes<47>. + +v40.6.0 + +* Deprecations:: +* Changes: Changes<48>. +* Documentation changes: Documentation changes<10>. +* Misc: Misc<25>. + +v40.5.0 + +* Changes: Changes<49>. +* Documentation changes: Documentation changes<11>. + +v40.4.3 + +* Changes: Changes<50>. + +v40.4.2 + +* Misc: Misc<26>. + +v40.4.1 + +* Changes: Changes<51>. + +v40.4.0 + +* Changes: Changes<52>. + +v40.3.0 + +* Changes: Changes<53>. +* Misc: Misc<27>. + +v40.2.0 + +* Changes: Changes<54>. + +v40.1.1 + +* Changes: Changes<55>. + +v40.1.0 + +* Changes: Changes<56>. +* Misc: Misc<28>. + +v40.0.0 + +* Breaking Changes: Breaking Changes<12>. +* Changes: Changes<57>. +* Documentation changes: Documentation changes<12>. +* Misc: Misc<29>. + +v39.2.0 + +* Changes: Changes<58>. +* Documentation changes: Documentation changes<13>. +* Misc: Misc<30>. + +1.0 + +* Backward-Incompatible Changes:: + +0.6.3 + +* setuptools:: +* bootstrapping:: + +0.6.2 + +* setuptools: setuptools<2>. +* bootstrapping: bootstrapping<2>. + +0.6.1 + +* setuptools: setuptools<3>. +* bootstrapping: bootstrapping<3>. + +0.6 + +* setuptools: setuptools<4>. +* pkg_resources:: +* easy_install:: + + + +File: setuptools.info, Node: Building and Distributing Packages with Setuptools, Next: Build System Support, Prev: Top, Up: Top + +1 Building and Distributing Packages with Setuptools +**************************************************** + +‘Setuptools’ is a collection of enhancements to the Python ‘distutils’ +that allow developers to more easily build and distribute Python +packages, especially ones that have dependencies on other packages. + +Packages built and distributed using ‘setuptools’ look to the user like +ordinary Python packages based on the ‘distutils’. + +* Menu: + +* Transition to PEP517:: + + +File: setuptools.info, Node: Transition to PEP517, Up: Building and Distributing Packages with Setuptools + +1.1 Transition to PEP517 +======================== + +Since setuptools no longer serves as the default build tool, one must +explicitly opt in (by providing a ‘pyproject.toml’ file) to use this +library. The user facing part is provided by tools such as pip and +backend interface is described *note in this document: 5. The +quickstart provides an overview of the new workflow. + +* Menu: + +* setuptools Quickstart:: +* Package Discovery and Namespace Package:: +* Entry Points:: +* Dependencies Management in Setuptools:: +* Data Files Support:: +* “Development Mode”:: +* Tagging and “Daily Build” or “Snapshot” Releases:: +* Generating Source Distributions:: +* Distributing Extensions compiled with Cython:: +* Specifying Your Project’s Version:: +* Creating distutils Extensions:: +* Configuring setup() using setup.cfg files: Configuring setup using setup cfg files. +* New and Changed setup() Keywords: New and Changed setup Keywords. +* Command Reference:: +* Using setuptools to package and distribute your project:: +* Automatic Resource Extraction:: +* Defining Additional Metadata:: +* Setting the zip_safe flag:: + + +File: setuptools.info, Node: setuptools Quickstart, Next: Package Discovery and Namespace Package, Up: Transition to PEP517 + +1.1.1 ‘setuptools’ Quickstart +----------------------------- + +* Menu: + +* Installation:: +* Python packaging at a glance:: +* Basic Use:: +* Automatic package discovery:: +* Entry points and automatic script creation:: +* Dependency management:: +* Including Data Files:: +* Development mode:: +* Uploading your package to PyPI:: +* Transitioning from setup.py to setup.cfg: Transitioning from setup py to setup cfg. +* Resources on Python packaging:: + + +File: setuptools.info, Node: Installation, Next: Python packaging at a glance, Up: setuptools Quickstart + +1.1.1.1 Installation +.................... + +To install the latest version of setuptools, use: + + pip install --upgrade setuptools + + +File: setuptools.info, Node: Python packaging at a glance, Next: Basic Use, Prev: Installation, Up: setuptools Quickstart + +1.1.1.2 Python packaging at a glance +.................................... + +The landscape of Python packaging is shifting and ‘Setuptools’ has +evolved to only provide backend support, no longer being the de-facto +packaging tool in the market. All python package must provide a +‘pyproject.toml’ and specify the backend (build system) it wants to use. +The distribution can then be generated with whatever tools that provides +a ‘build sdist’-alike functionality. While this may appear cumbersome, +given the added pieces, it in fact tremendously enhances the portability +of your package. The change is driven under 517(1). To learn more +about Python packaging in general, navigate to the bottom(2) of this +page. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0517#build-requirements + + (2) Resourcesonpythonpackaging + + +File: setuptools.info, Node: Basic Use, Next: Automatic package discovery, Prev: Python packaging at a glance, Up: setuptools Quickstart + +1.1.1.3 Basic Use +................. + +For basic use of setuptools, you will need a ‘pyproject.toml’ with the +exact following info, which declares you want to use ‘setuptools’ to +package your project: + + [build-system] + requires = ["setuptools", "wheel"] + build-backend = "setuptools.build_meta" + +Then, you will need a ‘setup.cfg’ to specify your package information, +such as metadata, contents, dependencies, etc. Here we demonstrate the +minimum + + [metadata] + name = "mypackage" + version = 0.0.1 + + [options] + packages = "mypackage" + install_requires = + requests + importlib; python_version == "2.6" + +This is what your project would look like: + + ~/mypackage/ + pyproject.toml + setup.cfg + mypackage/__init__.py + +Then, you need an installer, such as pep517(1) which you can obtain via +‘pip install pep517’. After downloading it, invoke the installer: + + python -m pep517.build + +You now have your distribution ready (e.g. a ‘tar.gz’ file and a ‘.whl’ +file in the ‘dist’ directory), which you can upload to PyPI! + +Of course, before you release your project to PyPI, you’ll want to add a +bit more information to your setup script to help people find or learn +about your project. And maybe your project will have grown by then to +include a few dependencies, and perhaps some data files and scripts. In +the next few section, we will walk through those additional but +essential information you need to specify to properly package your +project. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/pep517/ + + +File: setuptools.info, Node: Automatic package discovery, Next: Entry points and automatic script creation, Prev: Basic Use, Up: setuptools Quickstart + +1.1.1.4 Automatic package discovery +................................... + +For simple projects, it’s usually easy enough to manually add packages +to the ‘packages’ keyword in ‘setup.cfg’. However, for very large +projects , it can be a big burden to keep the package list updated. +‘setuptools’ therefore provides two convenient tools to ease the burden: +‘find:’ and ‘find_namespace:’. To use it in your project: + + [options] + packages = find: + + [options.packages.find] #optional + include=pkg1, pkg2 + exclude=pk3, pk4 + +When you pass the above information, alongside other necessary ones, +‘setuptools’ walks through the directory specified in ‘where’ (omitted +here as the package reside in current directory) and filters the +packages it can find following the ‘include’ (default to none), then +remove those that match the ‘exclude’ and return a list of Python +packages. Note that each entry in the ‘[options.packages.find]’ is +optional. The above setup also allows you to adopt a ‘src/’ layout. +For more details and advanced use, go to *note Package Discovery and +Namespace Package: c. + + +File: setuptools.info, Node: Entry points and automatic script creation, Next: Dependency management, Prev: Automatic package discovery, Up: setuptools Quickstart + +1.1.1.5 Entry points and automatic script creation +.................................................. + +Setuptools support automatic creation of scripts upon installation, that +runs code within your package if you specify them with the +‘entry_points’ keyword. This is what allows you to run commands like +‘pip install’ instead of having to type ‘python -m pip install’. To +accomplish this, add the entry_points keyword in your ‘setup.cfg’: + + [options.entry_points] + console_scripts = + main = mypkg:some_func + +When this project is installed, a ‘main’ script will be installed and +will invoke the ‘some_func’ in the ‘__init__.py’ file when called by the +user. For detailed usage, including managing the additional or optional +dependencies, go to *note Entry Points: e. + + +File: setuptools.info, Node: Dependency management, Next: Including Data Files, Prev: Entry points and automatic script creation, Up: setuptools Quickstart + +1.1.1.6 Dependency management +............................. + +‘setuptools’ supports automatically installing dependencies when a +package is installed. The simplest way to include requirement +specifiers is to use the ‘install_requires’ argument to ‘setup.cfg’. It +takes a string or list of strings containing requirement specifiers (A +version specifier is one of the operators <, >, <=, >=, == or !=, +followed by a version identifier): + + [options] + install_requires = + docutils >= 0.3 + requests <= 0.4 + +When your project is installed, all of the dependencies not already +installed will be located (via PyPI), downloaded, built (if necessary), +and installed. This, of course, is a simplified scenarios. +‘setuptools’ also provide additional keywords such as ‘setup_requires’ +that allows you to install dependencies before running the script, and +‘extras_requires’ that take care of those needed by automatically +generated scripts. It also provides mechanisms to handle dependencies +that are not in PyPI. For more advanced use, see *note Dependencies +Management in Setuptools: 10. + + +File: setuptools.info, Node: Including Data Files, Next: Development mode, Prev: Dependency management, Up: setuptools Quickstart + +1.1.1.7 Including Data Files +............................ + +The distutils have traditionally allowed installation of “data files”, +which are placed in a platform-specific location. Setuptools offers +three ways to specify data files to be included in your packages. For +the simpliest use, you can simply use the ‘include_package_data’ +keyword: + + [options] + include_package_data = True + +This tells setuptools to install any data files it finds in your +packages. The data files must be specified via the distutils’ +‘MANIFEST.in’ file. For more details, see *note Data Files Support: 13. + + +File: setuptools.info, Node: Development mode, Next: Uploading your package to PyPI, Prev: Including Data Files, Up: setuptools Quickstart + +1.1.1.8 Development mode +........................ + +‘setuptools’ allows you to install a package without copying any files +to your interpretor directory (e.g. the ‘site-packages’ directory). +This allows you to modify your source code and have the changes take +effect without you having to rebuild and reinstall. This is currently +incompatible with PEP 517 and therefore it requires a ‘setup.py’ script +with the following content: + + import setuptools + setuptools.setup() + +Then: + + pip install --editable . + +This creates a link file in your interpretor site package directory +which associate with your source code. For more information, see: (WIP) + + +File: setuptools.info, Node: Uploading your package to PyPI, Next: Transitioning from setup py to setup cfg, Prev: Development mode, Up: setuptools Quickstart + +1.1.1.9 Uploading your package to PyPI +...................................... + +After generating the distribution files, next step would be to upload +your distribution so others can use it. This functionality is provided +by twine(1) and we will only demonstrate the basic use here. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/twine/ + + +File: setuptools.info, Node: Transitioning from setup py to setup cfg, Next: Resources on Python packaging, Prev: Uploading your package to PyPI, Up: setuptools Quickstart + +1.1.1.10 Transitioning from ‘setup.py’ to ‘setup.cfg’ +..................................................... + +To avoid executing arbitary scripts and boilerplate code, we are +transitioning into a full-fledged ‘setup.cfg’ to declare your package +information instead of running ‘setup()’. This inevitably brings +challenges due to a different syntax. Here we provide a quick guide to +understanding how ‘setup.cfg’ is parsed by ‘setuptool’ to ease the pain +of transition. + + +File: setuptools.info, Node: Resources on Python packaging, Prev: Transitioning from setup py to setup cfg, Up: setuptools Quickstart + +1.1.1.11 Resources on Python packaging +...................................... + +Packaging in Python is hard. Here we provide a list of links for those +that want to learn more. + + +File: setuptools.info, Node: Package Discovery and Namespace Package, Next: Entry Points, Prev: setuptools Quickstart, Up: Transition to PEP517 + +1.1.2 Package Discovery and Namespace Package +--------------------------------------------- + + Note: a full specification for the keyword supplied to ‘setup.cfg’ + or ‘setup.py’ can be found at *note keywords reference: 1a. + + Note: the examples provided here are only to demonstrate the + functionality introduced. More metadata and options arguments need + to be supplied if you want to replicate them on your system. If + you are completely new to setuptools, the *note quickstart section: + 6. is a good place to start. + +‘Setuptools’ provide powerful tools to handle package discovery, +including support for namespace package. Normally, you would specify +the package to be included manually in the following manner: + + [options] + #... + packages = + mypkg1 + mypkg2 + + setup( + #... + packages = ['mypkg1', 'mypkg2'] + ) + +This can get tiresome reallly quickly. To speed things up, we introduce +two functions provided by setuptools: + + [options] + packages = find: + #or + packages = find_namespace: + + from setuptools import find_packages + #or + from setuptools import find_namespace_packages + +* Menu: + +* Using find; or find_packages: Using find or find_packages. +* Using find_namespace; or find_namespace_packages: Using find_namespace or find_namespace_packages. +* Legacy Namespace Packages:: + + +File: setuptools.info, Node: Using find or find_packages, Next: Using find_namespace or find_namespace_packages, Up: Package Discovery and Namespace Package + +1.1.2.1 Using ‘find:’ or ‘find_packages’ +........................................ + +Let’s start with the first tool. ‘find:’ (‘find_packages’) takes a +source directory and two lists of package name patterns to exclude and +include, and then return a list of ‘str’ representing the packages it +could find. To use it, consider the following directory + + mypkg/ + src/ + pkg1/__init__.py + pkg2/__init__.py + additional/__init__.py + + setup.cfg #or setup.py + +To have your setup.cfg or setup.py to automatically include packages +found in ‘src’ that starts with the name ‘pkg’ and not ‘additional’: + + [options] + packages = find: + package_dir = + =src + + [options.packages.find] + where = src + include = pkg* + exclude = additional + + setup( + #... + packages = find_packages( + where = 'src', + include = ['pkg*',], + exclude = ['additional',] + ), + package_dir = {"":"src"} + #... + ) + + +File: setuptools.info, Node: Using find_namespace or find_namespace_packages, Next: Legacy Namespace Packages, Prev: Using find or find_packages, Up: Package Discovery and Namespace Package + +1.1.2.2 Using ‘find_namespace:’ or ‘find_namespace_packages’ +............................................................ + +‘setuptools’ provides the ‘find_namespace:’ (‘find_namespace_packages’) +which behaves similarly to ‘find:’ but works with namespace package. +Before diving in, it is important to have a good understanding of what +namespace packages are. Here is a quick recap: + +Suppose you have two packages named as follows: + + /Users/Desktop/timmins/foo/__init__.py + /Library/timmins/bar/__init__.py + +If both ‘Desktop’ and ‘Library’ are on your ‘PYTHONPATH’, then a +namespace package called ‘timmins’ will be created automatically for you +when you invoke the import mechanism, allowing you to accomplish the +following + + >>> import timmins.foo + >>> import timmins.bar + +as if there is only one ‘timmins’ on your system. The two packages can +then be distributed separately and installed individually without +affecting the other one. Suppose you are packaging the ‘foo’ part: + + foo/ + src/ + timmins/foo/__init__.py + setup.cfg # or setup.py + +and you want the ‘foo’ to be automatically included, ‘find:’ won’t work +because timmins doesn’t contain ‘__init__.py’ directly, instead, you +have to use ‘find_namespace:’: + + [options] + package_dir = + =src + packages = find_namespace: + + [options.packages.find_namespace] + where = src + +When you install the zipped distribution, ‘timmins.foo’ would become +available to your interpreter. + +You can think of ‘find_namespace:’ as identical to ‘find:’ except it +would count a directory as a package even if it doesn’t contain +‘__init__.py’ file directly. As a result, this creates an interesting +side effect. If you organize your package like this: + + foo/ + timmins/ + foo/__init__.py + setup.cfg # or setup.py + tests/ + test_foo/__init__.py + +a naive ‘find_namespace:’ would include tests as part of your package to +be installed. A simple way to fix it is to adopt the aforementioned +‘src’ layout. + + +File: setuptools.info, Node: Legacy Namespace Packages, Prev: Using find_namespace or find_namespace_packages, Up: Package Discovery and Namespace Package + +1.1.2.3 Legacy Namespace Packages +................................. + +The fact you can create namespace package so effortlessly above is +credited to PEP 420(1). It use to be more cumbersome to accomplish the +same result. Historically, there were two methods to create namespace +packages. One is the ‘pkg_resources’ style supported by ‘setuptools’ +and the other one being ‘pkgutils’ style offered by ‘pkgutils’ module in +Python. Both are now considered deprecated despite the fact they still +linger in many existing packages. These two differ in many subtle yet +significant aspects and you can find out more on Python packaging user +guide(2) + +* Menu: + +* pkg_resource style namespace package:: +* pkgutil style namespace package:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0420/ + + (2) https://packaging.python.org/guides/packaging-namespace-packages/ + + +File: setuptools.info, Node: pkg_resource style namespace package, Next: pkgutil style namespace package, Up: Legacy Namespace Packages + +1.1.2.4 ‘pkg_resource’ style namespace package +.............................................. + +This is the method ‘setuptools’ directly supports. Starting with the +same layout, there are two pieces you need to add to it. First, an +‘__init__.py’ file directly under your namespace package directory that +contains the following: + + __import__("pkg_resources").declare_namespace(__name__) + +And the ‘namespace_packages’ keyword in your ‘setup.cfg’ or ‘setup.py’: + + [options] + namespace_packages = timmins + + setup( + # ... + namespace_packages = ['timmins'] + ) + +And your directory should look like this + + /foo/ + src/ + timmins/ + __init__.py + foo/__init__.py + setup.cfg #or setup.py + +Repeat the same for other packages and you can achieve the same result +as the previous section. + + +File: setuptools.info, Node: pkgutil style namespace package, Prev: pkg_resource style namespace package, Up: Legacy Namespace Packages + +1.1.2.5 ‘pkgutil’ style namespace package +......................................... + +This method is almost identical to the ‘pkg_resource’ except that the +‘namespace_packages’ declaration is omitted and the ‘__init__.py’ file +contains the following: + + __path__ = __import__('pkgutil').extend_path(__path__, __name__) + +The project layout remains the same and ‘setup.cfg’ remains the same. + + +File: setuptools.info, Node: Entry Points, Next: Dependencies Management in Setuptools, Prev: Package Discovery and Namespace Package, Up: Transition to PEP517 + +1.1.3 Entry Points +------------------ + +Packages may provide commands to be run at the console (console +scripts), such as the ‘pip’ command. These commands are defined for a +package as a specific kind of entry point in the ‘setup.cfg’ or +‘setup.py’. + +* Menu: + +* Console Scripts:: +* Advertising Behavior:: +* Dependency Management:: + + +File: setuptools.info, Node: Console Scripts, Next: Advertising Behavior, Up: Entry Points + +1.1.3.1 Console Scripts +....................... + +First consider an example without entry points. Imagine a package +defined thus: + + timmins/ + timmins/__init__.py + timmins/__main__.py + setup.cfg # or setup.py + #other necessary files + +with ‘__init__.py’ as: + + def helloworld(): + print("Hello world") + +and ‘__main__.py’ providing a hook: + + from . import hello_world + if __name__ == '__main__': + hello_world() + +After installing the package, the function may be invoked through the +runpy(1) module: + + python -m timmins + +Adding a console script entry point allows the package to define a +user-friendly name for installers of the package to execute. Installers +like pip will create wrapper scripts to execute a function. In the +above example, to create a command ‘hello-world’ that invokes +‘timmins.hello_world’, add a console script entry point to ‘setup.cfg’: + + [options.entry_points] + console_scripts = + hello-world = timmins:hello_world + +After installing the package, a user may invoke that function by simply +calling ‘hello-world’ on the command line. + +The syntax for entry points is specified as follows: + + = [.[.]][:.] + +where ‘name’ is the name for the script you want to create, the left +hand side of ‘:’ is the module that contains your function and the right +hand side is the object you want to invoke (e.g. a function). + +In addition to ‘console_scripts’, Setuptools supports ‘gui_scripts’, +which will launch a GUI application without running in a terminal +window. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/3/library/runpy.html + + +File: setuptools.info, Node: Advertising Behavior, Next: Dependency Management, Prev: Console Scripts, Up: Entry Points + +1.1.3.2 Advertising Behavior +............................ + +Console scripts are one use of the more general concept of entry points. +Entry points more generally allow a packager to advertise behavior for +discovery by other libraries and applications. This feature enables +“plug-in”-like functionality, where one library solicits entry points +and any number of other libraries provide those entry points. + +A good example of this plug-in behavior can be seen in pytest +plugins(1), where pytest is a test framework that allows other libraries +to extend or modify its functionality through the ‘pytest11’ entry +point. + +The console scripts work similarly, where libraries advertise their +commands and tools like ‘pip’ create wrapper scripts that invoke those +commands. + +For a project wishing to solicit entry points, Setuptools recommends the +importlib.metadata(2) module (part of stdlib since Python 3.8) or its +backport, importlib_metadata(3). + +For example, to find the console script entry points from the example +above: + + >>> from importlib import metadata + >>> eps = metadata.entry_points()['console_scripts'] + +‘eps’ is now a list of ‘EntryPoint’ objects, one of which corresponds to +the ‘hello-world = timmins:hello_world’ defined above. Each +‘EntryPoint’ contains the ‘name’, ‘group’, and ‘value’. It also +supplies a ‘.load()’ method to import and load that entry point (module +or object). + + [options.entry_points] + my.plugins = + hello-world = timmins:hello_world + +Then, a different project wishing to load ‘my.plugins’ plugins could run +the following routine to load (and invoke) such plugins: + + >>> from importlib import metadata + >>> eps = metadata.entry_points()['my.plugins'] + >>> for ep in eps: + ... plugin = ep.load() + ... plugin() + +The project soliciting the entry points needs not to have any dependency +or prior knowledge about the libraries implementing the entry points, +and downstream users are able to compose functionality by pulling +together libraries implementing the entry points. + + ---------- Footnotes ---------- + + (1) https://docs.pytest.org/en/latest/writing_plugins.html + + (2) https://docs.python.org/3/library/importlib.metadata.html + + (3) https://pypi.org/project/importlib_metadata + + +File: setuptools.info, Node: Dependency Management, Prev: Advertising Behavior, Up: Entry Points + +1.1.3.3 Dependency Management +............................. + +Some entry points may require additional dependencies to properly +function. For such an entry point, declare in square brakets any number +of dependency ‘extras’ following the entry point definition. Such entry +points will only be viable if their extras were declared and installed. +See the *note guide on dependencies management: 10. for more information +on defining extra requirements. Consider from the above example: + + [options.entry_points] + console_scripts = + hello-world = timmins:hello_world [pretty-printer] + +In this case, the ‘hello-world’ script is only viable if the +‘pretty-printer’ extra is indicated, and so a plugin host might exclude +that entry point (i.e. not install a console script) if the relevant +extra dependencies are not installed. + + +File: setuptools.info, Node: Dependencies Management in Setuptools, Next: Data Files Support, Prev: Entry Points, Up: Transition to PEP517 + +1.1.4 Dependencies Management in Setuptools +------------------------------------------- + +There are three types of dependency styles offered by setuptools: 1) +build system requirement, required dependency and 3) optional +dependency. + + Note: Packages that are added to dependency can be optionally + specified with the version by following PEP 440(1) + +* Menu: + +* Build system requirement:: +* Declaring required dependency:: +* Optional dependencies:: +* Python requirement:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440/ + + +File: setuptools.info, Node: Build system requirement, Next: Declaring required dependency, Up: Dependencies Management in Setuptools + +1.1.4.1 Build system requirement +................................ + +* Menu: + +* Package requirement:: + + +File: setuptools.info, Node: Package requirement, Up: Build system requirement + +1.1.4.2 Package requirement +........................... + +After organizing all the scripts and files and getting ready for +packaging, there needs to be a way to tell Python what programs it need +to actually do the packgaging (in our case, ‘setuptools’ of course). +Usually, you also need the ‘wheel’ package as well since it is +recommended that you upload a ‘.whl’ file to PyPI alongside your +‘.tar.gz’ file. Unlike the other two types of dependency keyword, this +one is specified in your ‘pyproject.toml’ file (if you have forgot what +this is, go to *note setuptools Quickstart: 6. or (WIP)): + + [build-system] + requires = ["setuptools", "wheel"] + #... + + Note: This used to be accomplished with the ‘setup_requires’ + keyword but is now considered deprecated in favor of the PEP 517 + style described above. To peek into how this legacy keyword is + used, consult our *note guide on deprecated practice (WIP): 2a. + + +File: setuptools.info, Node: Declaring required dependency, Next: Optional dependencies, Prev: Build system requirement, Up: Dependencies Management in Setuptools + +1.1.4.3 Declaring required dependency +..................................... + +This is where a package declares its core dependencies, without which it +won’t be able to run. ‘setuptools’ support automatically download and +install these dependencies when the package is installed. Although +there is more finess to it, let’s start with a simple example. + + [options] + #... + install_requires = + docutils + BazSpam ==1.1 + + setup( + #..., + install_requires = [ + 'docutils', + 'BazSpam ==1.1' + ] + ) + +When your project is installed (e.g. using pip), all of the +dependencies not already installed will be located (via PyPI), +downloaded, built (if necessary), and installed and 2) Any scripts in +your project will be installed with wrappers that verify the +availability of the specified dependencies at runtime. + +* Menu: + +* Platform specific dependencies:: +* Dependencies that aren’t in PyPI:: + + +File: setuptools.info, Node: Platform specific dependencies, Next: Dependencies that aren’t in PyPI, Up: Declaring required dependency + +1.1.4.4 Platform specific dependencies +...................................... + +Setuptools offer the capability to evaluate certain conditions before +blindly installing everything listed in ‘install_requires’. This is +great for platform specific dependencies. For example, the ‘enum’ +package was added in Python 3.4, therefore, package that depends on it +can elect to install it only when the Python version is older than 3.4. +To accomplish this + + [options] + #... + install_requires = + enum34;python_version<'3.4' + + setup( + #... + install_requires=[ + "enum34;python_version<'3.4'",] + ) + +Similarly, if you also wish to declare ‘pywin32’ with a minimal version +of 1.0 and only install it if the user is using a Windows operating +system: + + [options] + #... + install_requires = + enum34;python_version<'3.4' + pywin32 >= 1.0;platform_system=='Windows' + + setup( + #... + install_requires=[ + "enum34;python_version<'3.4'", + "pywin32 >= 1.0;platform_system=='Windows'" + ] + ) + +The environmental markers that may be used for testing platform types +are detailed in PEP 508(1). + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0508/ + + +File: setuptools.info, Node: Dependencies that aren’t in PyPI, Prev: Platform specific dependencies, Up: Declaring required dependency + +1.1.4.5 Dependencies that aren’t in PyPI +........................................ + + Warning: Dependency links support has been dropped by pip starting + with version 19.0 (released 2019-01-22). + +If your project depends on packages that don’t exist on PyPI, you may +still be able to depend on them, as long as they are available for +download as: + + - an egg, in the standard distutils ‘sdist’ format, + + - a single ‘.py’ file, or + + - a VCS repository (Subversion, Mercurial, or Git). + +You just need to add some URLs to the ‘dependency_links’ argument to +‘setup()’. + +The URLs must be either: + + 1. direct download URLs, + + 2. the URLs of web pages that contain direct download links, or + + 3. the repository’s URL + +In general, it’s better to link to web pages, because it is usually less +complex to update a web page than to release a new version of your +project. You can also use a SourceForge ‘showfiles.php’ link in the +case where a package you depend on is distributed via SourceForge. + +If you depend on a package that’s distributed as a single ‘.py’ file, +you must include an ‘"#egg=project-version"’ suffix to the URL, to give +a project name and version number. (Be sure to escape any dashes in the +name or version by replacing them with underscores.) EasyInstall will +recognize this suffix and automatically create a trivial ‘setup.py’ to +wrap the single ‘.py’ file as an egg. + +In the case of a VCS checkout, you should also append +‘#egg=project-version’ in order to identify for what package that +checkout should be used. You can append ‘@REV’ to the URL’s path +(before the fragment) to specify a revision. Additionally, you can also +force the VCS being used by prepending the URL with a certain prefix. +Currently available are: + + - ‘svn+URL’ for Subversion, + + - ‘git+URL’ for Git, and + + - ‘hg+URL’ for Mercurial + +A more complete example would be: + + ‘vcs+proto://host/path@revision#egg=project-version’ + +Be careful with the version. It should match the one inside the project +files. If you want to disregard the version, you have to omit it both +in the ‘requires’ and in the URL’s fragment. + +This will do a checkout (or a clone, in Git and Mercurial parlance) to a +temporary folder and run ‘setup.py bdist_egg’. + +The ‘dependency_links’ option takes the form of a list of URL strings. +For example, this will cause a search of the specified page for eggs or +source distributions, if the package’s dependencies aren’t already +installed: + + [options] + #... + dependency_links = http://peak.telecommunity.com/snapshots/ + + setup( + #... + dependency_links=[ + "http://peak.telecommunity.com/snapshots/" + ], + ) + + +File: setuptools.info, Node: Optional dependencies, Next: Python requirement, Prev: Declaring required dependency, Up: Dependencies Management in Setuptools + +1.1.4.6 Optional dependencies +............................. + +Setuptools allows you to declare dependencies that only get installed +under specific circumstances. These dependencies are specified with +‘extras_require’ keyword and are only installed if another package +depends on it (either directly or indirectly) This makes it convenient +to declare dependencies for ancillary functions such as “tests” and +“docs”. + + Note: ‘tests_require’ is now deprecated + +For example, Package-A offers optional PDF support and requires two +other dependencies for it to work: + + [metadata] + name = Package-A + + [options.extras_require] + PDF = ReportLab>=1.2; RXP + + setup( + name="Project-A", + #... + extras_require={ + "PDF": ["ReportLab>=1.2", "RXP"], + } + ) + +The name ‘PDF’ is an arbitary identifier of such a list of dependencies, +to which other components can refer and have them installed. There are +two common use cases. + +First is the console_scripts entry point: + + [metadata] + name = Project A + #... + + [options] + #... + entry_points= + [console_scripts] + rst2pdf = project_a.tools.pdfgen [PDF] + rst2html = project_a.tools.htmlgen + + setup( + name = "Project-A" + #..., + entry_points={ + "console_scripts": [ + "rst2pdf = project_a.tools.pdfgen [PDF]", + "rst2html = project_a.tools.htmlgen", + ], + } + ) + +When the script ‘rst2pdf’ is run, it will trigger the installation of +the two dependencies ‘PDF’ maps to. + +The second use case is that other package can use this “extra” for their +own dependencies. For example, if “Project-B” needs “project A” with +PDF support installed, it might declare the dependency like this: + + [metadata] + name = Project-B + #... + + [options] + #... + install_requires = + Project-A[PDF] + + setup( + name="Project-B", + install_requires=["Project-A[PDF]"], + ... + ) + +This will cause ReportLab to be installed along with project A, if +project B is installed – even if project A was already installed. In +this way, a project can encapsulate groups of optional “downstream +dependencies” under a feature name, so that packages that depend on it +don’t have to know what the downstream dependencies are. If a later +version of Project A builds in PDF support and no longer needs +ReportLab, or if it ends up needing other dependencies besides ReportLab +in order to provide PDF support, Project B’s setup information does not +need to change, but the right packages will still be installed if +needed. + + Note: Best practice: if a project ends up not needing any other + packages to support a feature, it should keep an empty requirements + list for that feature in its ‘extras_require’ argument, so that + packages depending on that feature don’t break (due to an invalid + feature name). + + +File: setuptools.info, Node: Python requirement, Prev: Optional dependencies, Up: Dependencies Management in Setuptools + +1.1.4.7 Python requirement +.......................... + +In some cases, you might need to specify the minimum required python +version. This is handled with the ‘python_requires’ keyword supplied to +‘setup.cfg’ or ‘setup.py’. + +Example WIP + + +File: setuptools.info, Node: Data Files Support, Next: “Development Mode”, Prev: Dependencies Management in Setuptools, Up: Transition to PEP517 + +1.1.5 Data Files Support +------------------------ + +The distutils have traditionally allowed installation of “data files”, +which are placed in a platform-specific location. However, the most +common use case for data files distributed with a package is for use +`by' the package, usually by including the data files in the package +directory. + +Setuptools offers three ways to specify data files to be included in +your packages. First, you can simply use the ‘include_package_data’ +keyword, e.g.: + + from setuptools import setup, find_packages + setup( + ... + include_package_data=True + ) + +This tells setuptools to install any data files it finds in your +packages. The data files must be specified via the distutils’ +‘MANIFEST.in’ file. (They can also be tracked by a revision control +system, using an appropriate plugin. See the section below on *note +Adding Support for Revision Control Systems: 32. for information on how +to write such plugins.) + +If you want finer-grained control over what files are included (for +example, if you have documentation files in your package directories and +want to exclude them from installation), then you can also use the +‘package_data’ keyword, e.g.: + + from setuptools import setup, find_packages + setup( + ... + package_data={ + # If any package contains *.txt or *.rst files, include them: + "": ["*.txt", "*.rst"], + # And include any *.msg files found in the "hello" package, too: + "hello": ["*.msg"], + } + ) + +The ‘package_data’ argument is a dictionary that maps from package names +to lists of glob patterns. The globs may include subdirectory names, if +the data files are contained in a subdirectory of the package. For +example, if the package tree looks like this: + + setup.py + src/ + mypkg/ + __init__.py + mypkg.txt + data/ + somefile.dat + otherdata.dat + +The setuptools setup file might look like this: + + from setuptools import setup, find_packages + setup( + ... + packages=find_packages("src"), # include all packages under src + package_dir={"": "src"}, # tell distutils packages are under src + + package_data={ + # If any package contains *.txt files, include them: + "": ["*.txt"], + # And include any *.dat files found in the "data" subdirectory + # of the "mypkg" package, also: + "mypkg": ["data/*.dat"], + } + ) + +Notice that if you list patterns in ‘package_data’ under the empty +string, these patterns are used to find files in every package, even +ones that also have their own patterns listed. Thus, in the above +example, the ‘mypkg.txt’ file gets included even though it’s not listed +in the patterns for ‘mypkg’. + +Also notice that if you use paths, you `must' use a forward slash (‘/’) +as the path separator, even if you are on Windows. Setuptools +automatically converts slashes to appropriate platform-specific +separators at build time. + +If datafiles are contained in a subdirectory of a package that isn’t a +package itself (no ‘__init__.py’), then the subdirectory names (or ‘*’) +are required in the ‘package_data’ argument (as shown above with +‘"data/*.dat"’). + +When building an ‘sdist’, the datafiles are also drawn from the +‘package_name.egg-info/SOURCES.txt’ file, so make sure that this is +removed if the ‘setup.py’ ‘package_data’ list is updated before calling +‘setup.py’. + +(Note: although the ‘package_data’ argument was previously only +available in ‘setuptools’, it was also added to the Python ‘distutils’ +package as of Python 2.4; there is some documentation for the feature(1) +available on the python.org website. If using the setuptools-specific +‘include_package_data’ argument, files specified by ‘package_data’ will +`not' be automatically added to the manifest unless they are listed in +the MANIFEST.in file.) + +Sometimes, the ‘include_package_data’ or ‘package_data’ options alone +aren’t sufficient to precisely define what files you want included. For +example, you may want to include package README files in your revision +control system and source distributions, but exclude them from being +installed. So, setuptools offers an ‘exclude_package_data’ option as +well, that allows you to do things like this: + + from setuptools import setup, find_packages + setup( + ... + packages=find_packages("src"), # include all packages under src + package_dir={"": "src"}, # tell distutils packages are under src + + include_package_data=True, # include everything in source control + + # ...but exclude README.txt from all packages + exclude_package_data={"": ["README.txt"]}, + ) + +The ‘exclude_package_data’ option is a dictionary mapping package names +to lists of wildcard patterns, just like the ‘package_data’ option. +And, just as with that option, a key of ‘""’ will apply the given +pattern(s) to all packages. However, any files that match these +patterns will be `excluded' from installation, even if they were listed +in ‘package_data’ or were included as a result of using +‘include_package_data’. + +In summary, the three options allow you to: + +‘include_package_data’ + + Accept all data files and directories matched by ‘MANIFEST.in’. + +‘package_data’ + + Specify additional patterns to match files that may or may not be + matched by ‘MANIFEST.in’ or found in source control. + +‘exclude_package_data’ + + Specify patterns for data files and directories that should `not' + be included when a package is installed, even if they would + otherwise have been included due to the use of the preceding + options. + +NOTE: Due to the way the distutils build process works, a data file that +you include in your project and then stop including may be “orphaned” in +your project’s build directories, requiring you to run ‘setup.py clean +--all’ to fully remove them. This may also be important for your users +and contributors if they track intermediate revisions of your project +using Subversion; be sure to let them know when you make changes that +remove files from inclusion so they can run ‘setup.py clean --all’. + +* Menu: + +* Accessing Data Files at Runtime:: +* Non-Package Data Files:: + + ---------- Footnotes ---------- + + (1) +https://docs.python.org/3/distutils/setupscript.html#installing-package-data + + +File: setuptools.info, Node: Accessing Data Files at Runtime, Next: Non-Package Data Files, Up: Data Files Support + +1.1.5.1 Accessing Data Files at Runtime +....................................... + +Typically, existing programs manipulate a package’s ‘__file__’ attribute +in order to find the location of data files. However, this manipulation +isn’t compatible with PEP 302-based import hooks, including importing +from zip files and Python Eggs. It is strongly recommended that, if you +are using data files, you should use the *note ResourceManager API: 35. +of ‘pkg_resources’ to access them. The ‘pkg_resources’ module is +distributed as part of setuptools, so if you’re using setuptools to +distribute your package, there is no reason not to use its resource +management API. See also Importlib Resources(1) for a quick example of +converting code that uses ‘__file__’ to use ‘pkg_resources’ instead. + + ---------- Footnotes ---------- + + (1) +https://docs.python.org/3/library/importlib.html#module-importlib.resources + + +File: setuptools.info, Node: Non-Package Data Files, Prev: Accessing Data Files at Runtime, Up: Data Files Support + +1.1.5.2 Non-Package Data Files +.............................. + +Historically, ‘setuptools’ by way of ‘easy_install’ would encapsulate +data files from the distribution into the egg (see the old docs(1)). As +eggs are deprecated and pip-based installs fall back to the +platform-specific location for installing data files, there is no +supported facility to reliably retrieve these resources. + +Instead, the PyPA recommends that any data files you wish to be +accessible at run time be included in the package. + + ---------- Footnotes ---------- + + (1) +https://github.com/pypa/setuptools/blob/52aacd5b276fedd6849c3a648a0014f5da563e93/docs/setuptools.txt#L970-L1001 + + +File: setuptools.info, Node: “Development Mode”, Next: Tagging and “Daily Build” or “Snapshot” Releases, Prev: Data Files Support, Up: Transition to PEP517 + +1.1.6 “Development Mode” +------------------------ + +Under normal circumstances, the ‘distutils’ assume that you are going to +build a distribution of your project, not use it in its “raw” or +“unbuilt” form. If you were to use the ‘distutils’ that way, you would +have to rebuild and reinstall your project every time you made a change +to it during development. + +Another problem that sometimes comes up with the ‘distutils’ is that you +may need to do development on two related projects at the same time. +You may need to put both projects’ packages in the same directory to run +them, but need to keep them separate for revision control purposes. How +can you do this? + +Setuptools allows you to deploy your projects for use in a common +directory or staging area, but without copying any files. Thus, you can +edit each project’s code in its checkout directory, and only need to run +build commands when you change a project’s C extensions or similarly +compiled files. You can even deploy a project into another project’s +checkout directory, if that’s your preferred way of working (as opposed +to using a common independent staging area or the site-packages +directory). + +To do this, use the ‘setup.py develop’ command. It works very similarly +to ‘setup.py install’, except that it doesn’t actually install anything. +Instead, it creates a special ‘.egg-link’ file in the deployment +directory, that links to your project’s source code. And, if your +deployment directory is Python’s ‘site-packages’ directory, it will also +update the ‘easy-install.pth’ file to include your project’s source +code, thereby making it available on ‘sys.path’ for all programs using +that Python installation. + +If you have enabled the ‘use_2to3’ flag, then of course the ‘.egg-link’ +will not link directly to your source code when run under Python 3, +since that source code would be made for Python 2 and not work under +Python 3. Instead the ‘setup.py develop’ will build Python 3 code under +the ‘build’ directory, and link there. This means that after doing code +changes you will have to run ‘setup.py build’ before these changes are +picked up by your Python 3 installation. + +In addition, the ‘develop’ command creates wrapper scripts in the target +script directory that will run your in-development scripts after +ensuring that all your ‘install_requires’ packages are available on +‘sys.path’. + +You can deploy the same project to multiple staging areas, e.g. if you +have multiple projects on the same machine that are sharing the same +project you’re doing development work. + +When you’re done with a given development task, you can remove the +project source from a staging area using ‘setup.py develop --uninstall’, +specifying the desired staging area if it’s not the default. + +There are several options to control the precise behavior of the +‘develop’ command; see the section on the *note develop: 3a. command +below for more details. + +Note that you can also apply setuptools commands to non-setuptools +projects, using commands like this: + + python -c "import setuptools; with open('setup.py') as f: exec(compile(f.read(), 'setup.py', 'exec'))" develop + +That is, you can simply list the normal setup commands and options +following the quoted part. + + +File: setuptools.info, Node: Tagging and “Daily Build” or “Snapshot” Releases, Next: Generating Source Distributions, Prev: “Development Mode”, Up: Transition to PEP517 + +1.1.7 Tagging and “Daily Build” or “Snapshot” Releases +------------------------------------------------------ + +When a set of related projects are under development, it may be +important to track finer-grained version increments than you would +normally use for e.g. “stable” releases. While stable releases might +be measured in dotted numbers with alpha/beta/etc. status codes, +development versions of a project often need to be tracked by revision +or build number or even build date. This is especially true when +projects in development need to refer to one another, and therefore may +literally need an up-to-the-minute version of something! + +To support these scenarios, ‘setuptools’ allows you to “tag” your source +and egg distributions by adding one or more of the following to the +project’s “official” version identifier: + + * A manually-specified pre-release tag, such as “build” or “dev”, or + a manually-specified post-release tag, such as a build or revision + number (‘--tag-build=STRING, -bSTRING’) + + * An 8-character representation of the build date (‘--tag-date, -d’), + as a postrelease tag + +You can add these tags by adding ‘egg_info’ and the desired options to +the command line ahead of the ‘sdist’ or ‘bdist’ commands that you want +to generate a daily build or snapshot for. See the section below on the +*note egg_info: 3d. command for more details. + +(Also, before you release your project, be sure to see the section above +on *note Specifying Your Project’s Version: 3e. for more information +about how pre- and post-release tags affect how version numbers are +interpreted. This is important in order to make sure that dependency +processing tools will know which versions of your project are newer than +others.) + +Finally, if you are creating builds frequently, and either building them +in a downloadable location or are copying them to a distribution server, +you should probably also check out the *note rotate: 3f. command, which +lets you automatically delete all but the N most-recently-modified +distributions matching a glob pattern. So, you can use a command line +like: + + setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3 + +to build an egg whose version info includes “DEV-rNNNN” (where NNNN is +the most recent Subversion revision that affected the source tree), and +then delete any egg files from the distribution directory except for the +three that were built most recently. + +If you have to manage automated builds for multiple packages, each with +different tagging and rotation policies, you may also want to check out +the *note alias: 40. command, which would let each package define an +alias like ‘daily’ that would perform the necessary tag, build, and +rotate commands. Then, a simpler script or cron job could just run +‘setup.py daily’ in each project directory. (And, you could also define +sitewide or per-user default versions of the ‘daily’ alias, so that +projects that didn’t define their own would use the appropriate +defaults.) + + +File: setuptools.info, Node: Generating Source Distributions, Next: Distributing Extensions compiled with Cython, Prev: Tagging and “Daily Build” or “Snapshot” Releases, Up: Transition to PEP517 + +1.1.8 Generating Source Distributions +------------------------------------- + +‘setuptools’ enhances the distutils’ default algorithm for source file +selection with pluggable endpoints for looking up files to include. If +you are using a revision control system, and your source distributions +only need to include files that you’re tracking in revision control, use +a corresponding plugin instead of writing a ‘MANIFEST.in’ file. See the +section below on *note Adding Support for Revision Control Systems: 32. +for information on plugins. + +If you need to include automatically generated files, or files that are +kept in an unsupported revision control system, you’ll need to create a +‘MANIFEST.in’ file to specify any files that the default file location +algorithm doesn’t catch. See the distutils documentation for more +information on the format of the ‘MANIFEST.in’ file. + +But, be sure to ignore any part of the distutils documentation that +deals with ‘MANIFEST’ or how it’s generated from ‘MANIFEST.in’; +setuptools shields you from these issues and doesn’t work the same way +in any case. Unlike the distutils, setuptools regenerates the source +distribution manifest file every time you build a source distribution, +and it builds it inside the project’s ‘.egg-info’ directory, out of the +way of your main project directory. You therefore need not worry about +whether it is up-to-date or not. + +Indeed, because setuptools’ approach to determining the contents of a +source distribution is so much simpler, its ‘sdist’ command omits nearly +all of the options that the distutils’ more complex ‘sdist’ process +requires. For all practical purposes, you’ll probably use only the +‘--formats’ option, if you use any option at all. + +* Menu: + +* Making “Official” (Non-Snapshot) Releases: Making “Official” Non-Snapshot Releases. + + +File: setuptools.info, Node: Making “Official” Non-Snapshot Releases, Up: Generating Source Distributions + +1.1.8.1 Making “Official” (Non-Snapshot) Releases +................................................. + +When you make an official release, creating source or binary +distributions, you will need to override the tag settings from +‘setup.cfg’, so that you don’t end up registering versions like +‘foobar-0.7a1.dev-r34832’. This is easy to do if you are developing on +the trunk and using tags or branches for your releases - just make the +change to ‘setup.cfg’ after branching or tagging the release, so the +trunk will still produce development snapshots. + +Alternately, if you are not branching for releases, you can override the +default version options on the command line, using something like: + + setup.py egg_info -Db "" sdist bdist_egg + +The first part of this command (‘egg_info -Db ""’) will override the +configured tag information, before creating source and binary eggs. +Thus, these commands will use the plain version from your ‘setup.py’, +without adding the build designation string. + +Of course, if you will be doing this a lot, you may wish to create a +personal alias for this operation, e.g.: + + setup.py alias -u release egg_info -Db "" + +You can then use it like this: + + setup.py release sdist bdist_egg + +Or of course you can create more elaborate aliases that do all of the +above. See the sections below on the *note egg_info: 3d. and *note +alias: 40. commands for more ideas. + + +File: setuptools.info, Node: Distributing Extensions compiled with Cython, Next: Specifying Your Project’s Version, Prev: Generating Source Distributions, Up: Transition to PEP517 + +1.1.9 Distributing Extensions compiled with Cython +-------------------------------------------------- + +‘setuptools’ will detect at build time whether Cython is installed or +not. If Cython is not found ‘setuptools’ will ignore pyx files. + +To ensure Cython is available, include Cython in the build-requires +section of your pyproject.toml: + + [build-system] + requires=[..., "cython"] + +Built with pip 10 or later, that declaration is sufficient to include +Cython in the build. For broader compatibility, declare the dependency +in your setup-requires of setup.cfg: + + [options] + setup_requires = + ... + cython + +As long as Cython is present in the build environment, ‘setuptools’ +includes transparent support for building Cython extensions, as long as +extensions are defined using ‘setuptools.Extension’. + +If you follow these rules, you can safely list ‘.pyx’ files as the +source of your ‘Extension’ objects in the setup script. If it is, then +‘setuptools’ will use it. + +Of course, for this to work, your source distributions must include the +C code generated by Cython, as well as your original ‘.pyx’ files. This +means that you will probably want to include current ‘.c’ files in your +revision control system, rebuilding them whenever you check changes in +for the ‘.pyx’ source files. This will ensure that people tracking your +project in a revision control system will be able to build it even if +they don’t have Cython installed, and that your source releases will be +similarly usable with or without Cython. + + +File: setuptools.info, Node: Specifying Your Project’s Version, Next: Creating distutils Extensions, Prev: Distributing Extensions compiled with Cython, Up: Transition to PEP517 + +1.1.10 Specifying Your Project’s Version +---------------------------------------- + +Setuptools can work well with most versioning schemes; there are, +however, a few special things to watch out for, in order to ensure that +setuptools and other tools can always tell what version of your package +is newer than another version. Knowing these things will also help you +correctly specify what versions of other projects your project depends +on. + +A version consists of an alternating series of release numbers and +pre-release or post-release tags. A release number is a series of +digits punctuated by dots, such as ‘2.4’ or ‘0.5’. Each series of +digits is treated numerically, so releases ‘2.1’ and ‘2.1.0’ are +different ways to spell the same release number, denoting the first +subrelease of release 2. But ‘2.10’ is the `tenth' subrelease of +release 2, and so is a different and newer release from ‘2.1’ or +‘2.1.0’. Leading zeros within a series of digits are also ignored, so +‘2.01’ is the same as ‘2.1’, and different from ‘2.0.1’. + +Following a release number, you can have either a pre-release or +post-release tag. Pre-release tags make a version be considered `older' +than the version they are appended to. So, revision ‘2.4’ is `newer' +than revision ‘2.4c1’, which in turn is newer than ‘2.4b1’ or ‘2.4a1’. +Postrelease tags make a version be considered `newer' than the version +they are appended to. So, revisions like ‘2.4-1’ and ‘2.4pl3’ are newer +than ‘2.4’, but are `older' than ‘2.4.1’ (which has a higher release +number). + +A pre-release tag is a series of letters that are alphabetically before +“final”. Some examples of prerelease tags would include ‘alpha’, +‘beta’, ‘a’, ‘c’, ‘dev’, and so on. You do not have to place a dot or +dash before the prerelease tag if it’s immediately after a number, but +it’s okay to do so if you prefer. Thus, ‘2.4c1’ and ‘2.4.c1’ and +‘2.4-c1’ all represent release candidate 1 of version ‘2.4’, and are +treated as identical by setuptools. + +In addition, there are three special prerelease tags that are treated as +if they were the letter ‘c’: ‘pre’, ‘preview’, and ‘rc’. So, version +‘2.4rc1’, ‘2.4pre1’ and ‘2.4preview1’ are all the exact same version as +‘2.4c1’, and are treated as identical by setuptools. + +A post-release tag is either a series of letters that are alphabetically +greater than or equal to “final”, or a dash (‘-’). Post-release tags +are generally used to separate patch numbers, port numbers, build +numbers, revision numbers, or date stamps from the release number. For +example, the version ‘2.4-r1263’ might denote Subversion revision 1263 +of a post-release patch of version ‘2.4’. Or you might use +‘2.4-20051127’ to denote a date-stamped post-release. + +Notice that after each pre or post-release tag, you are free to place +another release number, followed again by more pre- or post-release +tags. For example, ‘0.6a9.dev-r41475’ could denote Subversion revision +41475 of the in- development version of the ninth alpha of release 0.6. +Notice that ‘dev’ is a pre-release tag, so this version is a `lower' +version number than ‘0.6a9’, which would be the actual ninth alpha of +release 0.6. But the ‘-r41475’ is a post-release tag, so this version +is `newer' than ‘0.6a9.dev’. + +For the most part, setuptools’ interpretation of version numbers is +intuitive, but here are a few tips that will keep you out of trouble in +the corner cases: + + * Don’t stick adjoining pre-release tags together without a dot or + number between them. Version ‘1.9adev’ is the ‘adev’ prerelease of + ‘1.9’, `not' a development pre-release of ‘1.9a’. Use ‘.dev’ + instead, as in ‘1.9a.dev’, or separate the prerelease tags with a + number, as in ‘1.9a0dev’. ‘1.9a.dev’, ‘1.9a0dev’, and even + ‘1.9.a.dev’ are identical versions from setuptools’ point of view, + so you can use whatever scheme you prefer. + + * If you want to be certain that your chosen numbering scheme works + the way you think it will, you can use the + ‘pkg_resources.parse_version()’ function to compare different + version numbers: + + >>> from pkg_resources import parse_version + >>> parse_version("1.9.a.dev") == parse_version("1.9a0dev") + True + >>> parse_version("2.1-rc2") < parse_version("2.1") + True + >>> parse_version("0.6a9dev-r41475") < parse_version("0.6a9") + True + +Once you’ve decided on a version numbering scheme for your project, you +can have setuptools automatically tag your in-development releases with +various pre- or post-release tags. See the following sections for more +details: + + * *note Tagging and "Daily Build" or "Snapshot" Releases: 3c. + + * The *note egg_info: 3d. command + + +File: setuptools.info, Node: Creating distutils Extensions, Next: Configuring setup using setup cfg files, Prev: Specifying Your Project’s Version, Up: Transition to PEP517 + +1.1.11 Creating ‘distutils’ Extensions +-------------------------------------- + +It can be hard to add new commands or setup arguments to the distutils. +But the ‘setuptools’ package makes it a bit easier, by allowing you to +distribute a distutils extension as a separate project, and then have +projects that need the extension just refer to it in their +‘setup_requires’ argument. + +With ‘setuptools’, your distutils extension projects can hook in new +commands and ‘setup()’ arguments just by defining “entry points”. These +are mappings from command or argument names to a specification of where +to import a handler from. (See the section on *note Advertising +Behavior: 25. above for some more background on entry points.) + +* Menu: + +* Adding Commands:: +* Adding setup() Arguments: Adding setup Arguments. +* Customizing Distribution Options:: +* Adding new EGG-INFO Files:: +* Adding Support for Revision Control Systems:: + + +File: setuptools.info, Node: Adding Commands, Next: Adding setup Arguments, Up: Creating distutils Extensions + +1.1.11.1 Adding Commands +........................ + +You can add new ‘setup’ commands by defining entry points in the +‘distutils.commands’ group. For example, if you wanted to add a ‘foo’ +command, you might add something like this to your distutils extension +project’s setup script: + + setup( + # ... + entry_points={ + "distutils.commands": [ + "foo = mypackage.some_module:foo", + ], + }, + ) + +(Assuming, of course, that the ‘foo’ class in ‘mypackage.some_module’ is +a ‘setuptools.Command’ subclass.) + +Once a project containing such entry points has been activated on +‘sys.path’, (e.g. by running “install” or “develop” with a +site-packages installation directory) the command(s) will be available +to any ‘setuptools’-based setup scripts. It is not necessary to use the +‘--command-packages’ option or to monkeypatch the ‘distutils.command’ +package to install your commands; ‘setuptools’ automatically adds a +wrapper to the distutils to search for entry points in the active +distributions on ‘sys.path’. In fact, this is how setuptools’ own +commands are installed: the setuptools project’s setup script defines +entry points for them! + + +File: setuptools.info, Node: Adding setup Arguments, Next: Customizing Distribution Options, Prev: Adding Commands, Up: Creating distutils Extensions + +1.1.11.2 Adding ‘setup()’ Arguments +................................... + + Warning: Adding arguments to setup is discouraged as such arguments + are only supported through imperative execution and not supported + through declarative config. + +Sometimes, your commands may need additional arguments to the ‘setup()’ +call. You can enable this by defining entry points in the +‘distutils.setup_keywords’ group. For example, if you wanted a +‘setup()’ argument called ‘bar_baz’, you might add something like this +to your distutils extension project’s setup script: + + setup( + # ... + entry_points={ + "distutils.commands": [ + "foo = mypackage.some_module:foo", + ], + "distutils.setup_keywords": [ + "bar_baz = mypackage.some_module:validate_bar_baz", + ], + }, + ) + +The idea here is that the entry point defines a function that will be +called to validate the ‘setup()’ argument, if it’s supplied. The +‘Distribution’ object will have the initial value of the attribute set +to ‘None’, and the validation function will only be called if the +‘setup()’ call sets it to a non-None value. Here’s an example +validation function: + + def assert_bool(dist, attr, value): + """Verify that value is True, False, 0, or 1""" + if bool(value) != value: + raise DistutilsSetupError( + "%r must be a boolean value (got %r)" % (attr,value) + ) + +Your function should accept three arguments: the ‘Distribution’ object, +the attribute name, and the attribute value. It should raise a +‘DistutilsSetupError’ (from the ‘distutils.errors’ module) if the +argument is invalid. Remember, your function will only be called with +non-None values, and the default value of arguments defined this way is +always None. So, your commands should always be prepared for the +possibility that the attribute will be ‘None’ when they access it later. + +If more than one active distribution defines an entry point for the same +‘setup()’ argument, `all' of them will be called. This allows multiple +distutils extensions to define a common argument, as long as they agree +on what values of that argument are valid. + +Also note that as with commands, it is not necessary to subclass or +monkeypatch the distutils ‘Distribution’ class in order to add your +arguments; it is sufficient to define the entry points in your +extension, as long as any setup script using your extension lists your +project in its ‘setup_requires’ argument. + + +File: setuptools.info, Node: Customizing Distribution Options, Next: Adding new EGG-INFO Files, Prev: Adding setup Arguments, Up: Creating distutils Extensions + +1.1.11.3 Customizing Distribution Options +......................................... + +Plugins may wish to extend or alter the options on a Distribution object +to suit the purposes of that project. For example, a tool that infers +the ‘Distribution.version’ from SCM-metadata may need to hook into the +option finalization. To enable this feature, Setuptools offers an entry +point “setuptools.finalize_distribution_options”. That entry point must +be a callable taking one argument (the Distribution instance). + +If the callable has an ‘.order’ property, that value will be used to +determine the order in which the hook is called. Lower numbers are +called first and the default is zero (0). + +Plugins may read, alter, and set properties on the distribution, but +each plugin is encouraged to load the configuration/settings for their +behavior independently. + + +File: setuptools.info, Node: Adding new EGG-INFO Files, Next: Adding Support for Revision Control Systems, Prev: Customizing Distribution Options, Up: Creating distutils Extensions + +1.1.11.4 Adding new EGG-INFO Files +.................................. + +Some extensible applications or frameworks may want to allow third +parties to develop plugins with application or framework-specific +metadata included in the plugins’ EGG-INFO directory, for easy access +via the ‘pkg_resources’ metadata API. The easiest way to allow this is +to create a distutils extension to be used from the plugin projects’ +setup scripts (via ‘setup_requires’) that defines a new setup keyword, +and then uses that data to write an EGG-INFO file when the ‘egg_info’ +command is run. + +The ‘egg_info’ command looks for extension points in an +‘egg_info.writers’ group, and calls them to write the files. Here’s a +simple example of a distutils extension defining a setup argument +‘foo_bar’, which is a list of lines that will be written to +‘foo_bar.txt’ in the EGG-INFO directory of any project that uses the +argument: + + setup( + # ... + entry_points={ + "distutils.setup_keywords": [ + "foo_bar = setuptools.dist:assert_string_list", + ], + "egg_info.writers": [ + "foo_bar.txt = setuptools.command.egg_info:write_arg", + ], + }, + ) + +This simple example makes use of two utility functions defined by +setuptools for its own use: a routine to validate that a setup keyword +is a sequence of strings, and another one that looks up a setup argument +and writes it to a file. Here’s what the writer utility looks like: + + def write_arg(cmd, basename, filename): + argname = os.path.splitext(basename)[0] + value = getattr(cmd.distribution, argname, None) + if value is not None: + value = "\n".join(value) + "\n" + cmd.write_or_delete_file(argname, filename, value) + +As you can see, ‘egg_info.writers’ entry points must be a function +taking three arguments: a ‘egg_info’ command instance, the basename of +the file to write (e.g. ‘foo_bar.txt’), and the actual full filename +that should be written to. + +In general, writer functions should honor the command object’s ‘dry_run’ +setting when writing files, and use the ‘distutils.log’ object to do any +console output. The easiest way to conform to this requirement is to +use the ‘cmd’ object’s ‘write_file()’, ‘delete_file()’, and +‘write_or_delete_file()’ methods exclusively for your file operations. +See those methods’ docstrings for more details. + + +File: setuptools.info, Node: Adding Support for Revision Control Systems, Prev: Adding new EGG-INFO Files, Up: Creating distutils Extensions + +1.1.11.5 Adding Support for Revision Control Systems +.................................................... + +If the files you want to include in the source distribution are tracked +using Git, Mercurial or SVN, you can use the following packages to +achieve that: + + - Git and Mercurial: setuptools_scm(1) + + - SVN: setuptools_svn(2) + +If you would like to create a plugin for ‘setuptools’ to find files +tracked by another revision control system, you can do so by adding an +entry point to the ‘setuptools.file_finders’ group. The entry point +should be a function accepting a single directory name, and should yield +all the filenames within that directory (and any subdirectories thereof) +that are under revision control. + +For example, if you were going to create a plugin for a revision control +system called “foobar”, you would write a function something like this: + + def find_files_for_foobar(dirname): + # loop to yield paths that start with `dirname` + +And you would register it in a setup script using something like this: + + entry_points={ + "setuptools.file_finders": [ + "foobar = my_foobar_module:find_files_for_foobar", + ] + } + +Then, anyone who wants to use your plugin can simply install it, and +their local setuptools installation will be able to find the necessary +files. + +It is not necessary to distribute source control plugins with projects +that simply use the other source control system, or to specify the +plugins in ‘setup_requires’. When you create a source distribution with +the ‘sdist’ command, setuptools automatically records what files were +found in the ‘SOURCES.txt’ file. That way, recipients of source +distributions don’t need to have revision control at all. However, if +someone is working on a package by checking out with that system, they +will need the same plugin(s) that the original author is using. + +A few important points for writing revision control file finders: + + * Your finder function MUST return relative paths, created by + appending to the passed-in directory name. Absolute paths are NOT + allowed, nor are relative paths that reference a parent directory + of the passed-in directory. + + * Your finder function MUST accept an empty string as the directory + name, meaning the current directory. You MUST NOT convert this to + a dot; just yield relative paths. So, yielding a subdirectory + named ‘some/dir’ under the current directory should NOT be rendered + as ‘./some/dir’ or ‘/somewhere/some/dir’, but `always' as simply + ‘some/dir’ + + * Your finder function SHOULD NOT raise any errors, and SHOULD deal + gracefully with the absence of needed programs (i.e., ones + belonging to the revision control system itself. It `may', + however, use ‘distutils.log.warn()’ to inform the user of the + missing program(s). + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/setuptools_scm/ + + (2) https://pypi.org/project/setuptools_svn/ + + +File: setuptools.info, Node: Configuring setup using setup cfg files, Next: New and Changed setup Keywords, Prev: Creating distutils Extensions, Up: Transition to PEP517 + +1.1.12 Configuring setup() using setup.cfg files +------------------------------------------------ + + Note: New in 30.3.0 (8 Dec 2016). + + Important: If compatibility with legacy builds (i.e. those not + using the PEP 517(1) build API) is desired, a ‘setup.py’ file + containing a ‘setup()’ function call is still required even if your + configuration resides in ‘setup.cfg’. + +‘Setuptools’ allows using configuration files (usually ‘setup.cfg’) to +define a package’s metadata and other options that are normally supplied +to the ‘setup()’ function (declarative config). + +This approach not only allows automation scenarios but also reduces +boilerplate code in some cases. + + Note: This implementation has limited compatibility with the + distutils2-like ‘setup.cfg’ sections used by the ‘pbr’ and ‘d2to1’ + packages. + + Namely: only metadata-related keys from ‘metadata’ section are + supported (except for ‘description-file’); keys from ‘files’, + ‘entry_points’ and ‘backwards_compat’ are not supported. + + [metadata] + name = my_package + version = attr: src.VERSION + description = My package description + long_description = file: README.rst, CHANGELOG.rst, LICENSE.rst + keywords = one, two + license = BSD 3-Clause License + classifiers = + Framework :: Django + License :: OSI Approved :: BSD License + Programming Language :: Python :: 3 + Programming Language :: Python :: 3.5 + + [options] + zip_safe = False + include_package_data = True + packages = find: + scripts = + bin/first.py + bin/second.py + install_requires = + requests + importlib; python_version == "2.6" + + [options.package_data] + * = *.txt, *.rst + hello = *.msg + + [options.extras_require] + pdf = ReportLab>=1.2; RXP + rest = docutils>=0.3; pack ==1.1, ==1.3 + + [options.packages.find] + exclude = + src.subpackage1 + src.subpackage2 + + [options.data_files] + /etc/my_package = + site.d/00_default.conf + host.d/00_default.conf + data = data/img/logo.png, data/svg/icon.svg + +Metadata and options are set in the config sections of the same name. + + * Keys are the same as the keyword arguments one provides to the + ‘setup()’ function. + + * Complex values can be written comma-separated or placed one per + line in `dangling' config values. The following are equivalent: + + [metadata] + keywords = one, two + + [metadata] + keywords = + one + two + + * In some cases, complex values can be provided in dedicated + subsections for clarity. + + * Some keys allow ‘file:’, ‘attr:’, ‘find:’, and ‘find_namespace:’ + directives in order to cover common usecases. + + * Unknown keys are ignored. + +* Menu: + +* Using a src/ layout:: +* Specifying values:: + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0517 + + +File: setuptools.info, Node: Using a src/ layout, Next: Specifying values, Up: Configuring setup using setup cfg files + +1.1.12.1 Using a ‘src/’ layout +.............................. + +One commonly used package configuration has all the module source code +in a subdirectory (often called the ‘src/’ layout), like this: + + ├── src + │   └── mypackage + │   ├── __init__.py + │   └── mod1.py + ├── setup.py + └── setup.cfg + +You can set up your ‘setup.cfg’ to automatically find all your packages +in the subdirectory like this: + + # This example contains just the necessary options for a src-layout, set up + # the rest of the file as described above. + + [options] + package_dir= + =src + packages=find: + + [options.packages.find] + where=src + + +File: setuptools.info, Node: Specifying values, Prev: Using a src/ layout, Up: Configuring setup using setup cfg files + +1.1.12.2 Specifying values +.......................... + +Some values are treated as simple strings, some allow more logic. + +Type names used below: + + * ‘str’ - simple string + + * ‘list-comma’ - dangling list or string of comma-separated values + + * ‘list-semi’ - dangling list or string of semicolon-separated values + + * ‘bool’ - ‘True’ is 1, yes, true + + * ‘dict’ - list-comma where keys are separated from values by ‘=’ + + * ‘section’ - values are read from a dedicated (sub)section + +Special directives: + + * ‘attr:’ - Value is read from a module attribute. ‘attr:’ supports + callables and iterables; unsupported types are cast using ‘str()’. + + In order to support the common case of a literal value assigned to + a variable in a module containing (directly or indirectly) + third-party imports, ‘attr:’ first tries to read the value from the + module by examining the module’s AST. If that fails, ‘attr:’ falls + back to importing the module. + + * ‘file:’ - Value is read from a list of files and then concatenated + + Note: The ‘file:’ directive is sandboxed and won’t reach anything + outside the directory containing ‘setup.py’. + +* Menu: + +* Metadata:: +* Options:: + + +File: setuptools.info, Node: Metadata, Next: Options, Up: Specifying values + +1.1.12.3 Metadata +................. + + Note: The aliases given below are supported for compatibility + reasons, but their use is not advised. + +Key Aliases Type Minimum Version Notes + +------------------------------------------------------------------------------------------------------------- + +name str + + +version attr:, file:, str 39.2.0 1. + + +url home-page str + + +download_url download-url str + + +project_urls dict 38.3.0 + + +author str + + +author_email author-email str + + +maintainer str + + +maintainer_email maintainer-email str + + +classifiers classifier file:, list-comma + + +license str + + +license_file str + + +license_files list-comma + + +description summary file:, str + + +long_description long-description file:, str + + +long_description_content_type str 38.6.0 + + +keywords list-comma + + +platforms platform list-comma + + +provides list-comma + + +requires list-comma + + +obsoletes list-comma + + + Note: A version loaded using the ‘file:’ directive must comply with + PEP 440. It is easy to accidentally put something other than a + valid version string in such a file, so validation is stricter in + this case. + +Notes: 1. The ‘version’ file attribute has only been supported since +39.2.0. + + +File: setuptools.info, Node: Options, Prev: Metadata, Up: Specifying values + +1.1.12.4 Options +................ + +Key Type Minimum Version Notes + +-------------------------------------------------------------------------------------------------- + +zip_safe bool + + +setup_requires list-semi + + +install_requires list-semi + + +extras_require section + + +python_requires str + + +entry_points file:, section + + +use_2to3 bool + + +use_2to3_fixers list-comma + + +use_2to3_exclude_fixers list-comma + + +convert_2to3_doctests list-comma + + +scripts list-comma + + +eager_resources list-comma + + +dependency_links list-comma + + +tests_require list-semi + + +include_package_data bool + + +packages find:, find_namespace:, list-comma + + +package_dir dict + + +package_data section 1. + + +exclude_package_data section + + +namespace_packages list-comma + + +py_modules list-comma + + +data_files dict 40.6.0 + + + Note: `packages' - The ‘find:’ and ‘find_namespace:’ directive can + be further configured in a dedicated subsection + ‘options.packages.find’. This subsection accepts the same keys as + the ‘setuptools.find_packages’ and the + ‘setuptools.find_namespace_packages’ function: ‘where’, ‘include’, + and ‘exclude’. + + `find_namespace directive' - The ‘find_namespace:’ directive is + supported since Python >=3.3. + +Notes: 1. In the ‘package_data’ section, a key named with a single +asterisk (‘*’) refers to all packages, in lieu of the empty string used +in ‘setup.py’. + + +File: setuptools.info, Node: New and Changed setup Keywords, Next: Command Reference, Prev: Configuring setup using setup cfg files, Up: Transition to PEP517 + +1.1.13 New and Changed ‘setup()’ Keywords +----------------------------------------- + +The following keyword arguments to ‘setup()’ are added or changed by +‘setuptools’. All of them are optional; you do not have to supply them +unless you need the associated ‘setuptools’ feature. + +‘include_package_data’ + + If set to ‘True’, this tells ‘setuptools’ to automatically include + any data files it finds inside your package directories that are + specified by your ‘MANIFEST.in’ file. For more information, see + the section on *note Including Data Files: 12. + +‘exclude_package_data’ + + A dictionary mapping package names to lists of glob patterns that + should be `excluded' from your package directories. You can use + this to trim back any excess files included by + ‘include_package_data’. For a complete description and examples, + see the section on *note Including Data Files: 12. + +‘package_data’ + + A dictionary mapping package names to lists of glob patterns. For + a complete description and examples, see the section on *note + Including Data Files: 12. You do not need to use this option if + you are using ‘include_package_data’, unless you need to add e.g. + files that are generated by your setup script and build process. + (And are therefore not in source control or are files that you + don’t want to include in your source distribution.) + +‘zip_safe’ + + A boolean (True or False) flag specifying whether the project can + be safely installed and run from a zip file. If this argument is + not supplied, the ‘bdist_egg’ command will have to analyze all of + your project’s contents for possible problems each time it builds + an egg. + +‘install_requires’ + + A string or list of strings specifying what other distributions + need to be installed when this one is. See the section on *note + Declaring required dependency: 2b. for details and examples of the + format of this argument. + +‘entry_points’ + + A dictionary mapping entry point group names to strings or lists of + strings defining the entry points. Entry points are used to + support dynamic discovery of services or plugins provided by a + project. See *note Advertising Behavior: 25. for details and + examples of the format of this argument. In addition, this keyword + is used to support *note Automatic Script Creation: 21. + +‘extras_require’ + + A dictionary mapping names of “extras” (optional features of your + project) to strings or lists of strings specifying what other + distributions must be installed to support those features. See the + section on *note Declaring required dependency: 2b. for details and + examples of the format of this argument. + +‘python_requires’ + + A string corresponding to a version specifier (as defined in PEP + 440) for the Python version, used to specify the Requires-Python + defined in PEP 345. + +‘setup_requires’ + + A string or list of strings specifying what other distributions + need to be present in order for the `setup script' to run. + ‘setuptools’ will attempt to obtain these (using pip if available) + before processing the rest of the setup script or commands. This + argument is needed if you are using distutils extensions as part of + your build process; for example, extensions that process setup() + arguments and turn them into EGG-INFO metadata files. + + (Note: projects listed in ‘setup_requires’ will NOT be + automatically installed on the system where the setup script is + being run. They are simply downloaded to the ./.eggs directory if + they’re not locally available already. If you want them to be + installed, as well as being available when the setup script is run, + you should add them to ‘install_requires’ `and' ‘setup_requires’.) + +‘dependency_links’ + + A list of strings naming URLs to be searched when satisfying + dependencies. These links will be used if needed to install + packages specified by ‘setup_requires’ or ‘tests_require’. They + will also be written into the egg’s metadata for use during install + by tools that support them. + +‘namespace_packages’ + + A list of strings naming the project’s “namespace packages”. A + namespace package is a package that may be split across multiple + project distributions. For example, Zope 3’s ‘zope’ package is a + namespace package, because subpackages like ‘zope.interface’ and + ‘zope.publisher’ may be distributed separately. The egg runtime + system can automatically merge such subpackages into a single + parent package at runtime, as long as you declare them in each + project that contains any subpackages of the namespace package, and + as long as the namespace package’s ‘__init__.py’ does not contain + any code other than a namespace declaration. See the section below + on *note Using find_namespace; or find_namespace_packages: 1c. for + more information. + +‘test_suite’ + + A string naming a ‘unittest.TestCase’ subclass (or a package or + module containing one or more of them, or a method of such a + subclass), or naming a function that can be called with no + arguments and returns a ‘unittest.TestSuite’. If the named suite + is a module, and the module has an ‘additional_tests()’ function, + it is called and the results are added to the tests to be run. If + the named suite is a package, any submodules and subpackages are + recursively added to the overall test suite. + + Specifying this argument enables use of the *note test: 56. command + to run the specified test suite, e.g. via ‘setup.py test’. See + the section on the *note test: 56. command below for more details. + + New in 41.5.0: Deprecated the test command. + +‘tests_require’ + + If your project’s tests need one or more additional packages + besides those needed to install it, you can use this option to + specify them. It should be a string or list of strings specifying + what other distributions need to be present for the package’s tests + to run. When you run the ‘test’ command, ‘setuptools’ will attempt + to obtain these (using pip if available). Note that these required + projects will `not' be installed on the system where the tests are + run, but only downloaded to the project’s setup directory if + they’re not already installed locally. + + New in 41.5.0: Deprecated the test command. + +‘test_loader’ + + If you would like to use a different way of finding tests to run + than what setuptools normally uses, you can specify a module name + and class name in this argument. The named class must be + instantiable with no arguments, and its instances must support the + ‘loadTestsFromNames()’ method as defined in the Python ‘unittest’ + module’s ‘TestLoader’ class. Setuptools will pass only one test + “name” in the ‘names’ argument: the value supplied for the + ‘test_suite’ argument. The loader you specify may interpret this + string in any way it likes, as there are no restrictions on what + may be contained in a ‘test_suite’ string. + + The module name and class name must be separated by a ‘:’. The + default value of this argument is + ‘"setuptools.command.test:ScanningLoader"’. If you want to use the + default ‘unittest’ behavior, you can specify + ‘"unittest:TestLoader"’ as your ‘test_loader’ argument instead. + This will prevent automatic scanning of submodules and subpackages. + + The module and class you specify here may be contained in another + package, as long as you use the ‘tests_require’ option to ensure + that the package containing the loader class is available when the + ‘test’ command is run. + + New in 41.5.0: Deprecated the test command. + +‘eager_resources’ + + A list of strings naming resources that should be extracted + together, if any of them is needed, or if any C extensions included + in the project are imported. This argument is only useful if the + project will be installed as a zipfile, and there is a need to have + all of the listed resources be extracted to the filesystem `as a + unit'. Resources listed here should be “/”-separated paths, + relative to the source root, so to list a resource ‘foo.png’ in + package ‘bar.baz’, you would include the string ‘bar/baz/foo.png’ + in this argument. + + If you only need to obtain resources one at a time, or you don’t + have any C extensions that access other files in the project (such + as data files or shared libraries), you probably do NOT need this + argument and shouldn’t mess with it. For more details on how this + argument works, see the section below on *note Automatic Resource + Extraction: 58. + +‘use_2to3’ + + Convert the source code from Python 2 to Python 3 with 2to3 during + the build process. See *note Supporting both Python 2 and Python 3 + with Setuptools: 59. for more details. + +‘convert_2to3_doctests’ + + List of doctest source files that need to be converted with 2to3. + See *note Supporting both Python 2 and Python 3 with Setuptools: + 59. for more details. + +‘use_2to3_fixers’ + + A list of modules to search for additional fixers to be used during + the 2to3 conversion. See *note Supporting both Python 2 and Python + 3 with Setuptools: 59. for more details. + +‘project_urls’ + + An arbitrary map of URL names to hyperlinks, allowing more + extensible documentation of where various resources can be found + than the simple ‘url’ and ‘download_url’ options provide. + + +File: setuptools.info, Node: Command Reference, Next: Using setuptools to package and distribute your project, Prev: New and Changed setup Keywords, Up: Transition to PEP517 + +1.1.14 Command Reference +------------------------ + +* Menu: + +* alias - Define shortcuts for commonly used commands:: +* bdist_egg - Create a Python Egg for the project:: +* develop - Deploy the project source in “Development Mode”:: +* egg_info - Create egg metadata and set build tags:: +* rotate - Delete outdated distribution files:: +* saveopts - Save used options to a configuration file:: +* setopt - Set a distutils or setuptools option in a config file:: +* test - Build package and run a unittest suite:: +* upload - Upload source and/or egg distributions to PyPI:: + + +File: setuptools.info, Node: alias - Define shortcuts for commonly used commands, Next: bdist_egg - Create a Python Egg for the project, Up: Command Reference + +1.1.14.1 ‘alias’ - Define shortcuts for commonly used commands +.............................................................. + +Sometimes, you need to use the same commands over and over, but you +can’t necessarily set them as defaults. For example, if you produce +both development snapshot releases and “stable” releases of a project, +you may want to put the distributions in different places, or use +different ‘egg_info’ tagging options, etc. In these cases, it doesn’t +make sense to set the options in a distutils configuration file, because +the values of the options changed based on what you’re trying to do. + +Setuptools therefore allows you to define “aliases” - shortcut names for +an arbitrary string of commands and options, using ‘setup.py alias +aliasname expansion’, where aliasname is the name of the new alias, and +the remainder of the command line supplies its expansion. For example, +this command defines a sitewide alias called “daily”, that sets various +‘egg_info’ tagging options: + + setup.py alias --global-config daily egg_info --tag-build=development + +Once the alias is defined, it can then be used with other setup +commands, e.g.: + + setup.py daily bdist_egg # generate a daily-build .egg file + setup.py daily sdist # generate a daily-build source distro + setup.py daily sdist bdist_egg # generate both + +The above commands are interpreted as if the word ‘daily’ were replaced +with ‘egg_info --tag-build=development’. + +Note that setuptools will expand each alias `at most once' in a given +command line. This serves two purposes. First, if you accidentally +create an alias loop, it will have no effect; you’ll instead get an +error message about an unknown command. Second, it allows you to define +an alias for a command, that uses that command. For example, this +(project-local) alias: + + setup.py alias bdist_egg bdist_egg rotate -k1 -m.egg + +redefines the ‘bdist_egg’ command so that it always runs the ‘rotate’ +command afterwards to delete all but the newest egg file. It doesn’t +loop indefinitely on ‘bdist_egg’ because the alias is only expanded once +when used. + +You can remove a defined alias with the ‘--remove’ (or ‘-r’) option, +e.g.: + + setup.py alias --global-config --remove daily + +would delete the “daily” alias we defined above. + +Aliases can be defined on a project-specific, per-user, or sitewide +basis. The default is to define or remove a project-specific alias, but +you can use any of the *note configuration file options: 5d. (listed +under the *note saveopts: 5e. command, below) to determine which +distutils configuration file an aliases will be added to (or removed +from). + +Note that if you omit the “expansion” argument to the ‘alias’ command, +you’ll get output showing that alias’ current definition (and what +configuration file it’s defined in). If you omit the alias name as +well, you’ll get a listing of all current aliases along with their +configuration file locations. + + +File: setuptools.info, Node: bdist_egg - Create a Python Egg for the project, Next: develop - Deploy the project source in “Development Mode”, Prev: alias - Define shortcuts for commonly used commands, Up: Command Reference + +1.1.14.2 ‘bdist_egg’ - Create a Python Egg for the project +.......................................................... + + Warning: `eggs' are deprecated in favor of wheels, and not + supported by pip. + +This command generates a Python Egg (‘.egg’ file) for the project. +Python Eggs are the preferred binary distribution format for +EasyInstall, because they are cross-platform (for “pure” packages), +directly importable, and contain project metadata including scripts and +information about the project’s dependencies. They can be simply +downloaded and added to ‘sys.path’ directly, or they can be placed in a +directory on ‘sys.path’ and then automatically discovered by the egg +runtime system. + +This command runs the *note egg_info: 3d. command (if it hasn’t already +run) to update the project’s metadata (‘.egg-info’) directory. If you +have added any extra metadata files to the ‘.egg-info’ directory, those +files will be included in the new egg file’s metadata directory, for use +by the egg runtime system or by any applications or frameworks that use +that metadata. + +You won’t usually need to specify any special options for this command; +just use ‘bdist_egg’ and you’re done. But there are a few options that +may be occasionally useful: + +‘--dist-dir=DIR, -d DIR’ + + Set the directory where the ‘.egg’ file will be placed. If you + don’t supply this, then the ‘--dist-dir’ setting of the ‘bdist’ + command will be used, which is usually a directory named ‘dist’ in + the project directory. + +‘--plat-name=PLATFORM, -p PLATFORM’ + + Set the platform name string that will be embedded in the egg’s + filename (assuming the egg contains C extensions). This can be + used to override the distutils default platform name with something + more meaningful. Keep in mind, however, that the egg runtime + system expects to see eggs with distutils platform names, so it may + ignore or reject eggs with non-standard platform names. Similarly, + the EasyInstall program may ignore them when searching web pages + for download links. However, if you are cross-compiling or doing + some other unusual things, you might find a use for this option. + +‘--exclude-source-files’ + + Don’t include any modules’ ‘.py’ files in the egg, just compiled + Python, C, and data files. (Note that this doesn’t affect any + ‘.py’ files in the EGG-INFO directory or its subdirectories, since + for example there may be scripts with a ‘.py’ extension which must + still be retained.) We don’t recommend that you use this option + except for packages that are being bundled for proprietary end-user + applications, or for “embedded” scenarios where space is at an + absolute premium. On the other hand, if your package is going to + be installed and used in compressed form, you might as well exclude + the source because Python’s ‘traceback’ module doesn’t currently + understand how to display zipped source code anyway, or how to deal + with files that are in a different place from where their code was + compiled. + +There are also some options you will probably never need, but which are +there because they were copied from similar ‘bdist’ commands used as an +example for creating this one. They may be useful for testing and +debugging, however, which is why we kept them: + +‘--keep-temp, -k’ + + Keep the contents of the ‘--bdist-dir’ tree around after creating + the ‘.egg’ file. + +‘--bdist-dir=DIR, -b DIR’ + + Set the temporary directory for creating the distribution. The + entire contents of this directory are zipped to create the ‘.egg’ + file, after running various installation commands to copy the + package’s modules, data, and extensions here. + +‘--skip-build’ + + Skip doing any “build” commands; just go straight to the + install-and-compress phases. + + +File: setuptools.info, Node: develop - Deploy the project source in “Development Mode”, Next: egg_info - Create egg metadata and set build tags, Prev: bdist_egg - Create a Python Egg for the project, Up: Command Reference + +1.1.14.3 ‘develop’ - Deploy the project source in “Development Mode” +.................................................................... + +This command allows you to deploy your project’s source for use in one +or more “staging areas” where it will be available for importing. This +deployment is done in such a way that changes to the project source are +immediately available in the staging area(s), without needing to run a +build or install step after each change. + +The ‘develop’ command works by creating an ‘.egg-link’ file (named for +the project) in the given staging area. If the staging area is Python’s +‘site-packages’ directory, it also updates an ‘easy-install.pth’ file so +that the project is on ‘sys.path’ by default for all programs run using +that Python installation. + +The ‘develop’ command also installs wrapper scripts in the staging area +(or a separate directory, as specified) that will ensure the project’s +dependencies are available on ‘sys.path’ before running the project’s +source scripts. And, it ensures that any missing project dependencies +are available in the staging area, by downloading and installing them if +necessary. + +Last, but not least, the ‘develop’ command invokes the ‘build_ext -i’ +command to ensure any C extensions in the project have been built and +are up-to-date, and the ‘egg_info’ command to ensure the project’s +metadata is updated (so that the runtime and wrappers know what the +project’s dependencies are). If you make any changes to the project’s +setup script or C extensions, you should rerun the ‘develop’ command +against all relevant staging areas to keep the project’s scripts, +metadata and extensions up-to-date. Most other kinds of changes to your +project should not require any build operations or rerunning ‘develop’, +but keep in mind that even minor changes to the setup script (e.g. +changing an entry point definition) require you to re-run the ‘develop’ +or ‘test’ commands to keep the distribution updated. + +Here are some of the options that the ‘develop’ command accepts. Note +that they affect the project’s dependencies as well as the project +itself, so if you have dependencies that need to be installed and you +use ‘--exclude-scripts’ (for example), the dependencies’ scripts will +not be installed either! For this reason, you may want to use pip to +install the project’s dependencies before using the ‘develop’ command, +if you need finer control over the installation options for +dependencies. + +‘--uninstall, -u’ + + Un-deploy the current project. You may use the ‘--install-dir’ or + ‘-d’ option to designate the staging area. The created ‘.egg-link’ + file will be removed, if present and it is still pointing to the + project directory. The project directory will be removed from + ‘easy-install.pth’ if the staging area is Python’s ‘site-packages’ + directory. + + Note that this option currently does `not' uninstall script + wrappers! You must uninstall them yourself, or overwrite them by + using pip to install a different version of the package. You can + also avoid installing script wrappers in the first place, if you + use the ‘--exclude-scripts’ (aka ‘-x’) option when you run + ‘develop’ to deploy the project. + +‘--multi-version, -m’ + + “Multi-version” mode. Specifying this option prevents ‘develop’ + from adding an ‘easy-install.pth’ entry for the project(s) being + deployed, and if an entry for any version of a project already + exists, the entry will be removed upon successful deployment. In + multi-version mode, no specific version of the package is available + for importing, unless you use ‘pkg_resources.require()’ to put it + on ‘sys.path’, or you are running a wrapper script generated by + ‘setuptools’. (In which case the wrapper script calls ‘require()’ + for you.) + + Note that if you install to a directory other than ‘site-packages’, + this option is automatically in effect, because ‘.pth’ files can + only be used in ‘site-packages’ (at least in Python 2.3 and 2.4). + So, if you use the ‘--install-dir’ or ‘-d’ option (or they are set + via configuration file(s)) your project and its dependencies will + be deployed in multi- version mode. + +‘--install-dir=DIR, -d DIR’ + + Set the installation directory (staging area). If this option is + not directly specified on the command line or in a distutils + configuration file, the distutils default installation location is + used. Normally, this will be the ‘site-packages’ directory, but if + you are using distutils configuration files, setting things like + ‘prefix’ or ‘install_lib’, then those settings are taken into + account when computing the default staging area. + +‘--script-dir=DIR, -s DIR’ + + Set the script installation directory. If you don’t supply this + option (via the command line or a configuration file), but you + `have' supplied an ‘--install-dir’ (via command line or config + file), then this option defaults to the same directory, so that the + scripts will be able to find their associated package installation. + Otherwise, this setting defaults to the location where the + distutils would normally install scripts, taking any distutils + configuration file settings into account. + +‘--exclude-scripts, -x’ + + Don’t deploy script wrappers. This is useful if you don’t want to + disturb existing versions of the scripts in the staging area. + +‘--always-copy, -a’ + + Copy all needed distributions to the staging area, even if they are + already present in another directory on ‘sys.path’. By default, if + a requirement can be met using a distribution that is already + available in a directory on ‘sys.path’, it will not be copied to + the staging area. + +‘--egg-path=DIR’ + + Force the generated ‘.egg-link’ file to use a specified relative + path to the source directory. This can be useful in circumstances + where your installation directory is being shared by code running + under multiple platforms (e.g. Mac and Windows) which have + different absolute locations for the code under development, but + the same `relative' locations with respect to the installation + directory. If you use this option when installing, you must supply + the same relative path when uninstalling. + +In addition to the above options, the ‘develop’ command also accepts all +of the same options accepted by ‘easy_install’. If you’ve configured +any ‘easy_install’ settings in your ‘setup.cfg’ (or other distutils +config files), the ‘develop’ command will use them as defaults, unless +you override them in a ‘[develop]’ section or on the command line. + + +File: setuptools.info, Node: egg_info - Create egg metadata and set build tags, Next: rotate - Delete outdated distribution files, Prev: develop - Deploy the project source in “Development Mode”, Up: Command Reference + +1.1.14.4 ‘egg_info’ - Create egg metadata and set build tags +............................................................ + +This command performs two operations: it updates a project’s ‘.egg-info’ +metadata directory (used by the ‘bdist_egg’, ‘develop’, and ‘test’ +commands), and it allows you to temporarily change a project’s version +string, to support “daily builds” or “snapshot” releases. It is run +automatically by the ‘sdist’, ‘bdist_egg’, ‘develop’, and ‘test’ +commands in order to update the project’s metadata, but you can also +specify it explicitly in order to temporarily change the project’s +version string while executing other commands. (It also generates the +‘.egg-info/SOURCES.txt’ manifest file, which is used when you are +building source distributions.) + +In addition to writing the core egg metadata defined by ‘setuptools’ and +required by ‘pkg_resources’, this command can be extended to write other +metadata files as well, by defining entry points in the +‘egg_info.writers’ group. See the section on *note Adding new EGG-INFO +Files: 4b. below for more details. Note that using additional metadata +writers may require you to include a ‘setup_requires’ argument to +‘setup()’ in order to ensure that the desired writers are available on +‘sys.path’. + +* Menu: + +* Release Tagging Options:: +* Other egg_info Options:: +* egg_info Examples:: + + +File: setuptools.info, Node: Release Tagging Options, Next: Other egg_info Options, Up: egg_info - Create egg metadata and set build tags + +1.1.14.5 Release Tagging Options +................................ + +The following options can be used to modify the project’s version string +for all remaining commands on the setup command line. The options are +processed in the order shown, so if you use more than one, the requested +tags will be added in the following order: + +‘--tag-build=NAME, -b NAME’ + + Append NAME to the project’s version string. Due to the way + setuptools processes “pre-release” version suffixes beginning with + the letters “a” through “e” (like “alpha”, “beta”, and + “candidate”), you will usually want to use a tag like “.build” or + “.dev”, as this will cause the version number to be considered + `lower' than the project’s default version. (If you want to make + the version number `higher' than the default version, you can + always leave off –tag-build and then use one or both of the + following options.) + + If you have a default build tag set in your ‘setup.cfg’, you can + suppress it on the command line using ‘-b ""’ or ‘--tag-build=""’ + as an argument to the ‘egg_info’ command. + +‘--tag-date, -d’ + + Add a date stamp of the form “-YYYYMMDD” (e.g. “-20050528”) to the + project’s version number. + +‘--no-date, -D’ + + Don’t include a date stamp in the version number. This option is + included so you can override a default setting in ‘setup.cfg’. + +(Note: Because these options modify the version number used for source +and binary distributions of your project, you should first make sure +that you know how the resulting version numbers will be interpreted by +automated tools like pip. See the section above on *note Specifying +Your Project’s Version: 3e. for an explanation of pre- and post-release +tags, as well as tips on how to choose and verify a versioning scheme +for your project.) + +For advanced uses, there is one other option that can be set, to change +the location of the project’s ‘.egg-info’ directory. Commands that need +to find the project’s source directory or metadata should get it from +this setting: + + +File: setuptools.info, Node: Other egg_info Options, Next: egg_info Examples, Prev: Release Tagging Options, Up: egg_info - Create egg metadata and set build tags + +1.1.14.6 Other ‘egg_info’ Options +................................. + +‘--egg-base=SOURCEDIR, -e SOURCEDIR’ + + Specify the directory that should contain the .egg-info directory. + This should normally be the root of your project’s source tree + (which is not necessarily the same as your project directory; some + projects use a ‘src’ or ‘lib’ subdirectory as the source root). + You should not normally need to specify this directory, as it is + normally determined from the ‘package_dir’ argument to the + ‘setup()’ function, if any. If there is no ‘package_dir’ set, this + option defaults to the current directory. + + +File: setuptools.info, Node: egg_info Examples, Prev: Other egg_info Options, Up: egg_info - Create egg metadata and set build tags + +1.1.14.7 ‘egg_info’ Examples +............................ + +Creating a dated “nightly build” snapshot egg: + + setup.py egg_info --tag-date --tag-build=DEV bdist_egg + +Creating a release with no version tags, even if some default tags are +specified in ‘setup.cfg’: + + setup.py egg_info -RDb "" sdist bdist_egg + +(Notice that ‘egg_info’ must always appear on the command line `before' +any commands that you want the version changes to apply to.) + + +File: setuptools.info, Node: rotate - Delete outdated distribution files, Next: saveopts - Save used options to a configuration file, Prev: egg_info - Create egg metadata and set build tags, Up: Command Reference + +1.1.14.8 ‘rotate’ - Delete outdated distribution files +...................................................... + +As you develop new versions of your project, your distribution (‘dist’) +directory will gradually fill up with older source and/or binary +distribution files. The ‘rotate’ command lets you automatically clean +these up, keeping only the N most-recently modified files matching a +given pattern. + +‘--match=PATTERNLIST, -m PATTERNLIST’ + + Comma-separated list of glob patterns to match. This option is + `required'. The project name and ‘-*’ is prepended to the supplied + patterns, in order to match only distributions belonging to the + current project (in case you have a shared distribution directory + for multiple projects). Typically, you will use a glob pattern + like ‘.zip’ or ‘.egg’ to match files of the specified type. Note + that each supplied pattern is treated as a distinct group of files + for purposes of selecting files to delete. + +‘--keep=COUNT, -k COUNT’ + + Number of matching distributions to keep. For each group of files + identified by a pattern specified with the ‘--match’ option, delete + all but the COUNT most-recently-modified files in that group. This + option is `required'. + +‘--dist-dir=DIR, -d DIR’ + + Directory where the distributions are. This defaults to the value + of the ‘bdist’ command’s ‘--dist-dir’ option, which will usually be + the project’s ‘dist’ subdirectory. + +`Example 1': Delete all .tar.gz files from the distribution directory, +except for the 3 most recently modified ones: + + setup.py rotate --match=.tar.gz --keep=3 + +`Example 2': Delete all Python 2.3 or Python 2.4 eggs from the +distribution directory, except the most recently modified one for each +Python version: + + setup.py rotate --match=-py2.3*.egg,-py2.4*.egg --keep=1 + + +File: setuptools.info, Node: saveopts - Save used options to a configuration file, Next: setopt - Set a distutils or setuptools option in a config file, Prev: rotate - Delete outdated distribution files, Up: Command Reference + +1.1.14.9 ‘saveopts’ - Save used options to a configuration file +............................................................... + +Finding and editing ‘distutils’ configuration files can be a pain, +especially since you also have to translate the configuration options +from command-line form to the proper configuration file format. You can +avoid these hassles by using the ‘saveopts’ command. Just add it to the +command line to save the options you used. For example, this command +builds the project using the ‘mingw32’ C compiler, then saves the +–compiler setting as the default for future builds (even those run +implicitly by the ‘install’ command): + + setup.py build --compiler=mingw32 saveopts + +The ‘saveopts’ command saves all options for every command specified on +the command line to the project’s local ‘setup.cfg’ file, unless you use +one of the *note configuration file options: 5d. to change where the +options are saved. For example, this command does the same as above, +but saves the compiler setting to the site-wide (global) distutils +configuration: + + setup.py build --compiler=mingw32 saveopts -g + +Note that it doesn’t matter where you place the ‘saveopts’ command on +the command line; it will still save all the options specified for all +commands. For example, this is another valid way to spell the last +example: + + setup.py saveopts -g build --compiler=mingw32 + +Note, however, that all of the commands specified are always run, +regardless of where ‘saveopts’ is placed on the command line. + +* Menu: + +* Configuration File Options:: + + +File: setuptools.info, Node: Configuration File Options, Up: saveopts - Save used options to a configuration file + +1.1.14.10 Configuration File Options +.................................... + +Normally, settings such as options and aliases are saved to the +project’s local ‘setup.cfg’ file. But you can override this and save +them to the global or per-user configuration files, or to a +manually-specified filename. + +‘--global-config, -g’ + + Save settings to the global ‘distutils.cfg’ file inside the + ‘distutils’ package directory. You must have write access to that + directory to use this option. You also can’t combine this option + with ‘-u’ or ‘-f’. + +‘--user-config, -u’ + + Save settings to the current user’s ‘~/.pydistutils.cfg’ (POSIX) or + ‘$HOME/pydistutils.cfg’ (Windows) file. You can’t combine this + option with ‘-g’ or ‘-f’. + +‘--filename=FILENAME, -f FILENAME’ + + Save settings to the specified configuration file to use. You + can’t combine this option with ‘-g’ or ‘-u’. Note that if you + specify a non-standard filename, the ‘distutils’ and ‘setuptools’ + will not use the file’s contents. This option is mainly included + for use in testing. + +These options are used by other ‘setuptools’ commands that modify +configuration files, such as the *note alias: 40. and *note setopt: 67. +commands. + + +File: setuptools.info, Node: setopt - Set a distutils or setuptools option in a config file, Next: test - Build package and run a unittest suite, Prev: saveopts - Save used options to a configuration file, Up: Command Reference + +1.1.14.11 ‘setopt’ - Set a distutils or setuptools option in a config file +.......................................................................... + +This command is mainly for use by scripts, but it can also be used as a +quick and dirty way to change a distutils configuration option without +having to remember what file the options are in and then open an editor. + +`Example 1'. Set the default C compiler to ‘mingw32’ (using long option +names): + + setup.py setopt --command=build --option=compiler --set-value=mingw32 + +`Example 2'. Remove any setting for the distutils default package +installation directory (short option names): + + setup.py setopt -c install -o install_lib -r + +Options for the ‘setopt’ command: + +‘--command=COMMAND, -c COMMAND’ + + Command to set the option for. This option is required. + +‘--option=OPTION, -o OPTION’ + + The name of the option to set. This option is required. + +‘--set-value=VALUE, -s VALUE’ + + The value to set the option to. Not needed if ‘-r’ or ‘--remove’ + is set. + +‘--remove, -r’ + + Remove (unset) the option, instead of setting it. + +In addition to the above options, you may use any of the *note +configuration file options: 5d. (listed under the *note saveopts: 5e. +command, above) to determine which distutils configuration file the +option will be added to (or removed from). + + +File: setuptools.info, Node: test - Build package and run a unittest suite, Next: upload - Upload source and/or egg distributions to PyPI, Prev: setopt - Set a distutils or setuptools option in a config file, Up: Command Reference + +1.1.14.12 ‘test’ - Build package and run a unittest suite +......................................................... + + Warning: ‘test’ is deprecated and will be removed in a future + version. Users looking for a generic test entry point independent + of test runner are encouraged to use tox(1). + +When doing test-driven development, or running automated builds that +need testing before they are deployed for downloading or use, it’s often +useful to be able to run a project’s unit tests without actually +deploying the project anywhere, even using the ‘develop’ command. The +‘test’ command runs a project’s unit tests without actually deploying +it, by temporarily putting the project’s source on ‘sys.path’, after +first running ‘build_ext -i’ and ‘egg_info’ to ensure that any C +extensions and project metadata are up-to-date. + +To use this command, your project’s tests must be wrapped in a +‘unittest’ test suite by either a function, a ‘TestCase’ class or +method, or a module or package containing ‘TestCase’ classes. If the +named suite is a module, and the module has an ‘additional_tests()’ +function, it is called and the result (which must be a +‘unittest.TestSuite’) is added to the tests to be run. If the named +suite is a package, any submodules and subpackages are recursively added +to the overall test suite. (Note: if your project specifies a +‘test_loader’, the rules for processing the chosen ‘test_suite’ may +differ; see the *note test_loader: 57. documentation for more details.) + +Note that many test systems including ‘doctest’ support wrapping their +non-‘unittest’ tests in ‘TestSuite’ objects. So, if you are using a +test package that does not support this, we suggest you encourage its +developers to implement test suite support, as this is a convenient and +standard way to aggregate a collection of tests to be run under a common +test harness. + +By default, tests will be run in the “verbose” mode of the ‘unittest’ +package’s text test runner, but you can get the “quiet” mode (just dots) +if you supply the ‘-q’ or ‘--quiet’ option, either as a global option to +the setup script (e.g. ‘setup.py -q test’) or as an option for the +‘test’ command itself (e.g. ‘setup.py test -q’). There is one other +option available: + +‘--test-suite=NAME, -s NAME’ + + Specify the test suite (or module, class, or method) to be run + (e.g. ‘some_module.test_suite’). The default for this option can + be set by giving a ‘test_suite’ argument to the ‘setup()’ function, + e.g.: + + setup( + # ... + test_suite="my_package.tests.test_all" + ) + + If you did not set a ‘test_suite’ in your ‘setup()’ call, and do + not provide a ‘--test-suite’ option, an error will occur. + +New in 41.5.0: Deprecated the test command. + + ---------- Footnotes ---------- + + (1) https://tox.readthedocs.io + + +File: setuptools.info, Node: upload - Upload source and/or egg distributions to PyPI, Prev: test - Build package and run a unittest suite, Up: Command Reference + +1.1.14.13 ‘upload’ - Upload source and/or egg distributions to PyPI +................................................................... + +The ‘upload’ command was deprecated in version 40.0 and removed in +version 42.0. Use twine(1) instead. + +For more information on the current best practices in uploading your +packages to PyPI, see the Python Packaging User Guide’s “Packaging +Python Projects” tutorial specifically the section on uploading the +distribution archives(2). + + ---------- Footnotes ---------- + + (1) https://pypi.org/p/twine + + (2) +https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives + + +File: setuptools.info, Node: Using setuptools to package and distribute your project, Next: Automatic Resource Extraction, Prev: Command Reference, Up: Transition to PEP517 + +1.1.15 Using setuptools to package and distribute your project +-------------------------------------------------------------- + +‘setuptools’ offers a variety of functionalities that make it easy to +build and distribute your python package. Here we provide an overview +on the commonly used ones. + + +File: setuptools.info, Node: Automatic Resource Extraction, Next: Defining Additional Metadata, Prev: Using setuptools to package and distribute your project, Up: Transition to PEP517 + +1.1.16 Automatic Resource Extraction +------------------------------------ + +If you are using tools that expect your resources to be “real” files, or +your project includes non-extension native libraries or other files that +your C extensions expect to be able to access, you may need to list +those files in the ‘eager_resources’ argument to ‘setup()’, so that the +files will be extracted together, whenever a C extension in the project +is imported. + +This is especially important if your project includes shared libraries +`other' than distutils-built C extensions, and those shared libraries +use file extensions other than ‘.dll’, ‘.so’, or ‘.dylib’, which are the +extensions that setuptools 0.6a8 and higher automatically detects as +shared libraries and adds to the ‘native_libs.txt’ file for you. Any +shared libraries whose names do not end with one of those extensions +should be listed as ‘eager_resources’, because they need to be present +in the filesystem when he C extensions that link to them are used. + +The ‘pkg_resources’ runtime for compressed packages will automatically +extract `all' C extensions and ‘eager_resources’ at the same time, +whenever `any' C extension or eager resource is requested via the +‘resource_filename()’ API. (C extensions are imported using +‘resource_filename()’ internally.) This ensures that C extensions will +see all of the “real” files that they expect to see. + +Note also that you can list directory resource names in +‘eager_resources’ as well, in which case the directory’s contents +(including subdirectories) will be extracted whenever any C extension or +eager resource is requested. + +Please note that if you’re not sure whether you need to use this +argument, you don’t! It’s really intended to support projects with lots +of non-Python dependencies and as a last resort for crufty projects that +can’t otherwise handle being compressed. If your package is pure +Python, Python plus data files, or Python plus C, you really don’t need +this. You’ve got to be using either C or an external program that needs +“real” files in your project before there’s any possibility of +‘eager_resources’ being relevant to your project. + + +File: setuptools.info, Node: Defining Additional Metadata, Next: Setting the zip_safe flag, Prev: Automatic Resource Extraction, Up: Transition to PEP517 + +1.1.17 Defining Additional Metadata +----------------------------------- + +Some extensible applications and frameworks may need to define their own +kinds of metadata to include in eggs, which they can then access using +the ‘pkg_resources’ metadata APIs. Ordinarily, this is done by having +plugin developers include additional files in their +‘ProjectName.egg-info’ directory. However, since it can be tedious to +create such files by hand, you may want to create a distutils extension +that will create the necessary files from arguments to ‘setup()’, in +much the same way that ‘setuptools’ does for many of the ‘setup()’ +arguments it adds. See the section below on *note Creating distutils +Extensions: 46. for more details, especially the subsection on *note +Adding new EGG-INFO Files: 4b. + + +File: setuptools.info, Node: Setting the zip_safe flag, Prev: Defining Additional Metadata, Up: Transition to PEP517 + +1.1.18 Setting the ‘zip_safe’ flag +---------------------------------- + +For some use cases (such as bundling as part of a larger application), +Python packages may be run directly from a zip file. Not all packages, +however, are capable of running in compressed form, because they may +expect to be able to access either source code or data files as normal +operating system files. So, ‘setuptools’ can install your project as a +zipfile or a directory, and its default choice is determined by the +project’s ‘zip_safe’ flag. + +You can pass a True or False value for the ‘zip_safe’ argument to the +‘setup()’ function, or you can omit it. If you omit it, the ‘bdist_egg’ +command will analyze your project’s contents to see if it can detect any +conditions that would prevent it from working in a zipfile. It will +output notices to the console about any such conditions that it finds. + +Currently, this analysis is extremely conservative: it will consider the +project unsafe if it contains any C extensions or datafiles whatsoever. +This does `not' mean that the project can’t or won’t work as a zipfile! +It just means that the ‘bdist_egg’ authors aren’t yet comfortable +asserting that the project `will' work. If the project contains no C or +data files, and does no ‘__file__’ or ‘__path__’ introspection or source +code manipulation, then there is an extremely solid chance the project +will work when installed as a zipfile. (And if the project uses +‘pkg_resources’ for all its data file access, then C extensions and +other data files shouldn’t be a problem at all. See the *note Accessing +Data Files at Runtime: 33. section above for more information.) + +However, if ‘bdist_egg’ can’t be `sure' that your package will work, but +you’ve checked over all the warnings it issued, and you are either +satisfied it `will' work (or if you want to try it for yourself), then +you should set ‘zip_safe’ to ‘True’ in your ‘setup()’ call. If it turns +out that it doesn’t work, you can always change it to ‘False’, which +will force ‘setuptools’ to install your project as a directory rather +than as a zipfile. + +In the future, as we gain more experience with different packages and +become more satisfied with the robustness of the ‘pkg_resources’ +runtime, the “zip safety” analysis may become less conservative. +However, we strongly recommend that you determine for yourself whether +your project functions correctly when installed as a zipfile, correct +any problems if you can, and then make an explicit declaration of ‘True’ +or ‘False’ for the ‘zip_safe’ flag, so that it will not be necessary for +‘bdist_egg’ to try to guess whether your project can work as a zipfile. + + +File: setuptools.info, Node: Build System Support, Next: Package Discovery and Resource Access using pkg_resources, Prev: Building and Distributing Packages with Setuptools, Up: Top + +2 Build System Support +********************** + +* Menu: + +* What is it?:: +* How to use it?:: + + +File: setuptools.info, Node: What is it?, Next: How to use it?, Up: Build System Support + +2.1 What is it? +=============== + +Python packaging has come a long way(1). + +The traditional ‘setuptools’ way of packgaging Python modules uses a +‘setup()’ function within the ‘setup.py’ script. Commands such as +‘python setup.py bdist’ or ‘python setup.py bdist_wheel’ generate a +distribution bundle and ‘python setup.py install’ installs the +distribution. This interface makes it difficult to choose other +packaging tools without an overhaul. Because ‘setup.py’ scripts allowed +for arbitrary execution, it proved difficult to provide a reliable user +experience across environments and history. + +PEP 517(2) therefore came to rescue and specified a new standard to +package and distribute Python modules. Under PEP 517: + + a ‘pyproject.toml’ file is used to specify what program to use for + generating distribution. + + Then, two functions provided by the program, + ‘build_wheel(directory: str)’ and ‘build_sdist(directory: str)’ + create the distribution bundle at the specified ‘directory’. The + program is free to use its own configuration script or extend the + ‘.toml’ file. + + Lastly, ‘pip install *.whl’ or ‘pip install *.tar.gz’ does the + actual installation. If ‘*.whl’ is available, ‘pip’ will go ahead + and copy the files into ‘site-packages’ directory. If not, ‘pip’ + will look at ‘pyproject.toml’ and decide what program to use to + ‘build from source’ (the default is ‘setuptools’) + +With this standard, switching between packaging tools becomes a lot +easier. ‘build_meta’ implements ‘setuptools’’ build system support. + + ---------- Footnotes ---------- + + (1) https://www.bernat.tech/pep-517-518/ + + (2) https://www.python.org/dev/peps/pep-0517/ + + +File: setuptools.info, Node: How to use it?, Prev: What is it?, Up: Build System Support + +2.2 How to use it? +================== + +Starting with a package that you want to distribute. You will need your +source scripts, a ‘pyproject.toml’ file and a ‘setup.cfg’ file: + + ~/meowpkg/ + pyproject.toml + setup.cfg + meowpkg/__init__.py + +The pyproject.toml file is required to specify the build system (i.e. +what is being used to package your scripts and install from source). To +use it with setuptools, the content would be: + + [build-system] + requires = ["setuptools", "wheel"] + build-backend = "setuptools.build_meta" + +Use ‘setuptools’’ *note declarative config: 50. to specify the package +information: + + [metadata] + name = meowpkg + version = 0.0.1 + description = a package that meows + + [options] + packages = find: + +Now generate the distribution. Although the PyPA is still working to +provide a recommended tool(1) to build packages, the pep517 package(2) +provides this functionality. To build the package: + + $ pip install -q pep517 + $ mkdir dist + $ python -m pep517.build . + +And now it’s done! The ‘.whl’ file and ‘.tar.gz’ can then be +distributed and installed: + + dist/ + meowpkg-0.0.1.whl + meowpkg-0.0.1.tar.gz + + $ pip install dist/meowpkg-0.0.1.whl + +or: + + $ pip install dist/meowpkg-0.0.1.tar.gz + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging-problems/issues/219 + + (2) https://pypi.org/project/pep517 + + +File: setuptools.info, Node: Package Discovery and Resource Access using pkg_resources, Next: Keywords, Prev: Build System Support, Up: Top + +3 Package Discovery and Resource Access using ‘pkg_resources’ +************************************************************* + +The ‘pkg_resources’ module distributed with ‘setuptools’ provides an API +for Python libraries to access their resource files, and for extensible +applications and frameworks to automatically discover plugins. It also +provides runtime support for using C extensions that are inside +zipfile-format eggs, support for merging packages that have +separately-distributed modules or subpackages, and APIs for managing +Python’s current “working set” of active packages. + +* Menu: + +* Overview:: +* API Reference:: + + +File: setuptools.info, Node: Overview, Next: API Reference, Up: Package Discovery and Resource Access using pkg_resources + +3.1 Overview +============ + +The ‘pkg_resources’ module provides runtime facilities for finding, +introspecting, activating and using installed Python distributions. +Some of the more advanced features (notably the support for parallel +installation of multiple versions) rely specifically on the “egg” format +(either as a zip archive or subdirectory), while others (such as plugin +discovery) will work correctly so long as “egg-info” metadata +directories are available for relevant distributions. + +Eggs are a distribution format for Python modules, similar in concept to +Java’s “jars” or Ruby’s “gems”, or the “wheel” format defined in PEP +427. However, unlike a pure distribution format, eggs can also be +installed and added directly to ‘sys.path’ as an import location. When +installed in this way, eggs are `discoverable', meaning that they carry +metadata that unambiguously identifies their contents and dependencies. +This means that an installed egg can be `automatically' found and added +to ‘sys.path’ in response to simple requests of the form, “get me +everything I need to use docutils’ PDF support”. This feature allows +mutually conflicting versions of a distribution to co-exist in the same +Python installation, with individual applications activating the desired +version at runtime by manipulating the contents of ‘sys.path’ (this +differs from the virtual environment approach, which involves creating +isolated environments for each application). + +The following terms are needed in order to explain the capabilities +offered by this module: + +project + + A library, framework, script, plugin, application, or collection of + data or other resources, or some combination thereof. Projects are + assumed to have “relatively unique” names, e.g. names registered + with PyPI. + +release + + A snapshot of a project at a particular point in time, denoted by a + version identifier. + +distribution + + A file or files that represent a particular release. + +importable distribution + + A file or directory that, if placed on ‘sys.path’, allows Python to + import any modules contained within it. + +pluggable distribution + + An importable distribution whose filename unambiguously identifies + its release (i.e. project and version), and whose contents + unambiguously specify what releases of other projects will satisfy + its runtime requirements. + +extra + + An “extra” is an optional feature of a release, that may impose + additional runtime requirements. For example, if docutils PDF + support required a PDF support library to be present, docutils + could define its PDF support as an “extra”, and list what other + project releases need to be available in order to provide it. + +environment + + A collection of distributions potentially available for importing, + but not necessarily active. More than one distribution (i.e. + release version) for a given project may be present in an + environment. + +working set + + A collection of distributions actually available for importing, as + on ‘sys.path’. At most one distribution (release version) of a + given project may be present in a working set, as otherwise there + would be ambiguity as to what to import. + +eggs + + Eggs are pluggable distributions in one of the three formats + currently supported by ‘pkg_resources’. There are built eggs, + development eggs, and egg links. Built eggs are directories or + zipfiles whose name ends with ‘.egg’ and follows the egg naming + conventions, and contain an ‘EGG-INFO’ subdirectory (zipped or + otherwise). Development eggs are normal directories of Python code + with one or more ‘ProjectName.egg-info’ subdirectories. The + development egg format is also used to provide a default version of + a distribution that is available to software that doesn’t use + ‘pkg_resources’ to request specific versions. Egg links are + ‘*.egg-link’ files that contain the name of a built or development + egg, to support symbolic linking on platforms that do not have + native symbolic links (or where the symbolic link support is + limited). + +(For more information about these terms and concepts, see also this +architectural overview(1) of ‘pkg_resources’ and Python Eggs in +general.) + + ---------- Footnotes ---------- + + (1) +http://mail.python.org/pipermail/distutils-sig/2005-June/004652.html + + +File: setuptools.info, Node: API Reference, Prev: Overview, Up: Package Discovery and Resource Access using pkg_resources + +3.2 API Reference +================= + +* Menu: + +* Namespace Package Support:: +* WorkingSet Objects:: +* Environment Objects:: +* Requirement Objects:: +* Entry Points: Entry Points<2>. +* Distribution Objects:: +* ResourceManager API:: +* Metadata API:: +* Exceptions:: +* Supporting Custom Importers:: +* Utility Functions:: + + +File: setuptools.info, Node: Namespace Package Support, Next: WorkingSet Objects, Up: API Reference + +3.2.1 Namespace Package Support +------------------------------- + +A namespace package is a package that only contains other packages and +modules, with no direct contents of its own. Such packages can be split +across multiple, separately-packaged distributions. They are normally +used to split up large packages produced by a single organization, such +as in the ‘zope’ namespace package for Zope Corporation packages, and +the ‘peak’ namespace package for the Python Enterprise Application Kit. + +To create a namespace package, you list it in the ‘namespace_packages’ +argument to ‘setup()’, in your project’s ‘setup.py’. (See the *note +setuptools documentation on namespace packages: 1c. for more information +on this.) Also, you must add a ‘declare_namespace()’ call in the +package’s ‘__init__.py’ file(s): + +‘declare_namespace(name)’ + + Declare that the dotted package name ‘name’ is a “namespace + package” whose contained packages and modules may be spread across + multiple distributions. The named package’s ‘__path__’ will be + extended to include the corresponding package in all distributions + on ‘sys.path’ that contain a package of that name. (More + precisely, if an importer’s ‘find_module(name)’ returns a loader, + then it will also be searched for the package’s contents.) + Whenever a Distribution’s ‘activate()’ method is invoked, it checks + for the presence of namespace packages and updates their ‘__path__’ + contents accordingly. + +Applications that manipulate namespace packages or directly alter +‘sys.path’ at runtime may also need to use this API function: + +‘fixup_namespace_packages(path_item)’ + + Declare that ‘path_item’ is a newly added item on ‘sys.path’ that + may need to be used to update existing namespace packages. + Ordinarily, this is called for you when an egg is automatically + added to ‘sys.path’, but if your application modifies ‘sys.path’ to + include locations that may contain portions of a namespace package, + you will need to call this function to ensure they are added to the + existing namespace packages. + +Although by default ‘pkg_resources’ only supports namespace packages for +filesystem and zip importers, you can extend its support to other +“importers” compatible with PEP 302 using the +‘register_namespace_handler()’ function. See the section below on *note +Supporting Custom Importers: 7a. for details. + + +File: setuptools.info, Node: WorkingSet Objects, Next: Environment Objects, Prev: Namespace Package Support, Up: API Reference + +3.2.2 ‘WorkingSet’ Objects +-------------------------- + +The ‘WorkingSet’ class provides access to a collection of “active” +distributions. In general, there is only one meaningful ‘WorkingSet’ +instance: the one that represents the distributions that are currently +active on ‘sys.path’. This global instance is available under the name +‘working_set’ in the ‘pkg_resources’ module. However, specialized tools +may wish to manipulate working sets that don’t correspond to ‘sys.path’, +and therefore may wish to create other ‘WorkingSet’ instances. + +It’s important to note that the global ‘working_set’ object is +initialized from ‘sys.path’ when ‘pkg_resources’ is first imported, but +is only updated if you do all future ‘sys.path’ manipulation via +‘pkg_resources’ APIs. If you manually modify ‘sys.path’, you must +invoke the appropriate methods on the ‘working_set’ instance to keep it +in sync. Unfortunately, Python does not provide any way to detect +arbitrary changes to a list object like ‘sys.path’, so ‘pkg_resources’ +cannot automatically update the ‘working_set’ based on changes to +‘sys.path’. + +‘WorkingSet(entries=None)’ + + Create a ‘WorkingSet’ from an iterable of path entries. If + ‘entries’ is not supplied, it defaults to the value of ‘sys.path’ + at the time the constructor is called. + + Note that you will not normally construct ‘WorkingSet’ instances + yourself, but instead you will implicitly or explicitly use the + global ‘working_set’ instance. For the most part, the + ‘pkg_resources’ API is designed so that the ‘working_set’ is used + by default, such that you don’t have to explicitly refer to it most + of the time. + +All distributions available directly on ‘sys.path’ will be activated +automatically when ‘pkg_resources’ is imported. This behaviour can +cause version conflicts for applications which require non-default +versions of those distributions. To handle this situation, +‘pkg_resources’ checks for a ‘__requires__’ attribute in the ‘__main__’ +module when initializing the default working set, and uses this to +ensure a suitable version of each affected distribution is activated. +For example: + + __requires__ = ["CherryPy < 3"] # Must be set before pkg_resources import + import pkg_resources + +* Menu: + +* Basic WorkingSet Methods:: +* WorkingSet Methods and Attributes:: +* Receiving Change Notifications:: +* Locating Plugins:: + + +File: setuptools.info, Node: Basic WorkingSet Methods, Next: WorkingSet Methods and Attributes, Up: WorkingSet Objects + +3.2.2.1 Basic ‘WorkingSet’ Methods +.................................. + +The following methods of ‘WorkingSet’ objects are also available as +module- level functions in ‘pkg_resources’ that apply to the default +‘working_set’ instance. Thus, you can use e.g. +‘pkg_resources.require()’ as an abbreviation for +‘pkg_resources.working_set.require()’: + +‘require(*requirements)’ + + Ensure that distributions matching ‘requirements’ are activated + + ‘requirements’ must be a string or a (possibly-nested) sequence + thereof, specifying the distributions and versions required. The + return value is a sequence of the distributions that needed to be + activated to fulfill the requirements; all relevant distributions + are included, even if they were already activated in this working + set. + + For the syntax of requirement specifiers, see the section below on + *note Requirements Parsing: 7d. + + In general, it should not be necessary for you to call this method + directly. It’s intended more for use in quick-and-dirty scripting + and interactive interpreter hacking than for production use. If + you’re creating an actual library or application, it’s strongly + recommended that you create a “setup.py” script using ‘setuptools’, + and declare all your requirements there. That way, tools like pip + can automatically detect what requirements your package has, and + deal with them accordingly. + + Note that calling ‘require('SomePackage')’ will not install + ‘SomePackage’ if it isn’t already present. If you need to do this, + you should use the ‘resolve()’ method instead, which allows you to + pass an ‘installer’ callback that will be invoked when a needed + distribution can’t be found on the local machine. You can then + have this callback display a dialog, automatically download the + needed distribution, or whatever else is appropriate for your + application. See the documentation below on the ‘resolve()’ method + for more information, and also on the ‘obtain()’ method of + ‘Environment’ objects. + +‘run_script(requires, script_name)’ + + Locate distribution specified by ‘requires’ and run its + ‘script_name’ script. ‘requires’ must be a string containing a + requirement specifier. (See *note Requirements Parsing: 7d. below + for the syntax.) + + The script, if found, will be executed in `the caller’s globals'. + That’s because this method is intended to be called from wrapper + scripts that act as a proxy for the “real” scripts in a + distribution. A wrapper script usually doesn’t need to do anything + but invoke this function with the correct arguments. + + If you need more control over the script execution environment, you + probably want to use the ‘run_script()’ method of a ‘Distribution’ + object’s *note Metadata API: 7e. instead. + +‘iter_entry_points(group, name=None)’ + + Yield entry point objects from ‘group’ matching ‘name’ + + If ‘name’ is None, yields all entry points in ‘group’ from all + distributions in the working set, otherwise only ones matching both + ‘group’ and ‘name’ are yielded. Entry points are yielded from the + active distributions in the order that the distributions appear in + the working set. (For the global ‘working_set’, this should be the + same as the order that they are listed in ‘sys.path’.) Note that + within the entry points advertised by an individual distribution, + there is no particular ordering. + + Please see the section below on *note Entry Points: 7f. for more + information. + + +File: setuptools.info, Node: WorkingSet Methods and Attributes, Next: Receiving Change Notifications, Prev: Basic WorkingSet Methods, Up: WorkingSet Objects + +3.2.2.2 ‘WorkingSet’ Methods and Attributes +........................................... + +These methods are used to query or manipulate the contents of a specific +working set, so they must be explicitly invoked on a particular +‘WorkingSet’ instance: + +‘add_entry(entry)’ + + Add a path item to the ‘entries’, finding any distributions on it. + You should use this when you add additional items to ‘sys.path’ and + you want the global ‘working_set’ to reflect the change. This + method is also called by the ‘WorkingSet()’ constructor during + initialization. + + This method uses ‘find_distributions(entry,True)’ to find + distributions corresponding to the path entry, and then ‘add()’ + them. ‘entry’ is always appended to the ‘entries’ attribute, even + if it is already present, however. (This is because ‘sys.path’ can + contain the same value more than once, and the ‘entries’ attribute + should be able to reflect this.) + +‘__contains__(dist)’ + + True if ‘dist’ is active in this ‘WorkingSet’. Note that only one + distribution for a given project can be active in a given + ‘WorkingSet’. + +‘__iter__()’ + + Yield distributions for non-duplicate projects in the working set. + The yield order is the order in which the items’ path entries were + added to the working set. + +‘find(req)’ + + Find a distribution matching ‘req’ (a ‘Requirement’ instance). If + there is an active distribution for the requested project, this + returns it, as long as it meets the version requirement specified + by ‘req’. But, if there is an active distribution for the project + and it does `not' meet the ‘req’ requirement, ‘VersionConflict’ is + raised. If there is no active distribution for the requested + project, ‘None’ is returned. + +‘resolve(requirements, env=None, installer=None)’ + + List all distributions needed to (recursively) meet ‘requirements’ + + ‘requirements’ must be a sequence of ‘Requirement’ objects. ‘env’, + if supplied, should be an ‘Environment’ instance. If not supplied, + an ‘Environment’ is created from the working set’s ‘entries’. + ‘installer’, if supplied, will be invoked with each requirement + that cannot be met by an already-installed distribution; it should + return a ‘Distribution’ or ‘None’. (See the ‘obtain()’ method of + *note Environment Objects: 81, below, for more information on the + ‘installer’ argument.) + +‘add(dist, entry=None)’ + + Add ‘dist’ to working set, associated with ‘entry’ + + If ‘entry’ is unspecified, it defaults to ‘dist.location’. On exit + from this routine, ‘entry’ is added to the end of the working set’s + ‘.entries’ (if it wasn’t already present). + + ‘dist’ is only added to the working set if it’s for a project that + doesn’t already have a distribution active in the set. If it’s + successfully added, any callbacks registered with the ‘subscribe()’ + method will be called. (See *note Receiving Change Notifications: + 82, below.) + + Note: ‘add()’ is automatically called for you by the ‘require()’ + method, so you don’t normally need to use this method directly. + +‘entries’ + + This attribute represents a “shadow” ‘sys.path’, primarily useful + for debugging. If you are experiencing import problems, you should + check the global ‘working_set’ object’s ‘entries’ against + ‘sys.path’, to ensure that they match. If they do not, then some + part of your program is manipulating ‘sys.path’ without updating + the ‘working_set’ accordingly. IMPORTANT NOTE: do not directly + manipulate this attribute! Setting it equal to ‘sys.path’ will not + fix your problem, any more than putting black tape over an “engine + warning” light will fix your car! If this attribute is out of sync + with ‘sys.path’, it’s merely an `indicator' of the problem, not the + cause of it. + + +File: setuptools.info, Node: Receiving Change Notifications, Next: Locating Plugins, Prev: WorkingSet Methods and Attributes, Up: WorkingSet Objects + +3.2.2.3 Receiving Change Notifications +...................................... + +Extensible applications and frameworks may need to receive notification +when a new distribution (such as a plug-in component) has been added to +a working set. This is what the ‘subscribe()’ method and +‘add_activation_listener()’ function are for. + +‘subscribe(callback)’ + + Invoke ‘callback(distribution)’ once for each active distribution + that is in the set now, or gets added later. Because the callback + is invoked for already-active distributions, you do not need to + loop over the working set yourself to deal with the existing items; + just register the callback and be prepared for the fact that it + will be called immediately by this method. + + Note that callbacks `must not' allow exceptions to propagate, or + they will interfere with the operation of other callbacks and + possibly result in an inconsistent working set state. Callbacks + should use a try/except block to ignore, log, or otherwise process + any errors, especially since the code that caused the callback to + be invoked is unlikely to be able to handle the errors any better + than the callback itself. + +‘pkg_resources.add_activation_listener()’ is an alternate spelling of +‘pkg_resources.working_set.subscribe()’. + + +File: setuptools.info, Node: Locating Plugins, Prev: Receiving Change Notifications, Up: WorkingSet Objects + +3.2.2.4 Locating Plugins +........................ + +Extensible applications will sometimes have a “plugin directory” or a +set of plugin directories, from which they want to load entry points or +other metadata. The ‘find_plugins()’ method allows you to do this, by +scanning an environment for the newest version of each project that can +be safely loaded without conflicts or missing requirements. + +‘find_plugins(plugin_env, full_env=None, fallback=True)’ + + Scan ‘plugin_env’ and identify which distributions could be added + to this working set without version conflicts or missing + requirements. + + Example usage: + + distributions, errors = working_set.find_plugins( + Environment(plugin_dirlist) + ) + map(working_set.add, distributions) # add plugins+libs to sys.path + print "Couldn't load", errors # display errors + + The ‘plugin_env’ should be an ‘Environment’ instance that contains + only distributions that are in the project’s “plugin directory” or + directories. The ‘full_env’, if supplied, should be an + ‘Environment’ instance that contains all currently-available + distributions. + + If ‘full_env’ is not supplied, one is created automatically from + the ‘WorkingSet’ this method is called on, which will typically + mean that every directory on ‘sys.path’ will be scanned for + distributions. + + This method returns a 2-tuple: (‘distributions’, ‘error_info’), + where ‘distributions’ is a list of the distributions found in + ‘plugin_env’ that were loadable, along with any other distributions + that are needed to resolve their dependencies. ‘error_info’ is a + dictionary mapping unloadable plugin distributions to an exception + instance describing the error that occurred. Usually this will be + a ‘DistributionNotFound’ or ‘VersionConflict’ instance. + + Most applications will use this method mainly on the master + ‘working_set’ instance in ‘pkg_resources’, and then immediately add + the returned distributions to the working set so that they are + available on sys.path. This will make it possible to find any + entry points, and allow any other metadata tracking and hooks to be + activated. + + The resolution algorithm used by ‘find_plugins()’ is as follows. + First, the project names of the distributions present in + ‘plugin_env’ are sorted. Then, each project’s eggs are tried in + descending version order (i.e., newest version first). + + An attempt is made to resolve each egg’s dependencies. If the + attempt is successful, the egg and its dependencies are added to + the output list and to a temporary copy of the working set. The + resolution process continues with the next project name, and no + older eggs for that project are tried. + + If the resolution attempt fails, however, the error is added to the + error dictionary. If the ‘fallback’ flag is true, the next older + version of the plugin is tried, until a working version is found. + If false, the resolution process continues with the next plugin + project name. + + Some applications may have stricter fallback requirements than + others. For example, an application that has a database schema or + persistent objects may not be able to safely downgrade a version of + a package. Others may want to ensure that a new plugin + configuration is either 100% good or else revert to a known-good + configuration. (That is, they may wish to revert to a known + configuration if the ‘error_info’ return value is non-empty.) + + Note that this algorithm gives precedence to satisfying the + dependencies of alphabetically prior project names in case of + version conflicts. If two projects named “AaronsPlugin” and + “ZekesPlugin” both need different versions of “TomsLibrary”, then + “AaronsPlugin” will win and “ZekesPlugin” will be disabled due to + version conflict. + + +File: setuptools.info, Node: Environment Objects, Next: Requirement Objects, Prev: WorkingSet Objects, Up: API Reference + +3.2.3 ‘Environment’ Objects +--------------------------- + +An “environment” is a collection of ‘Distribution’ objects, usually ones +that are present and potentially importable on the current platform. +‘Environment’ objects are used by ‘pkg_resources’ to index available +distributions during dependency resolution. + +‘Environment(search_path=None, platform=get_supported_platform(), python=PY_MAJOR)’ + + Create an environment snapshot by scanning ‘search_path’ for + distributions compatible with ‘platform’ and ‘python’. + ‘search_path’ should be a sequence of strings such as might be used + on ‘sys.path’. If a ‘search_path’ isn’t supplied, ‘sys.path’ is + used. + + ‘platform’ is an optional string specifying the name of the + platform that platform-specific distributions must be compatible + with. If unspecified, it defaults to the current platform. + ‘python’ is an optional string naming the desired version of Python + (e.g. ‘'2.4'’); it defaults to the currently-running version. + + You may explicitly set ‘platform’ (and/or ‘python’) to ‘None’ if + you wish to include `all' distributions, not just those compatible + with the running platform or Python version. + + Note that ‘search_path’ is scanned immediately for distributions, + and the resulting ‘Environment’ is a snapshot of the found + distributions. It is not automatically updated if the system’s + state changes due to e.g. installation or removal of + distributions. + +‘__getitem__(project_name)’ + + Returns a list of distributions for the given project name, ordered + from newest to oldest version. (And highest to lowest format + precedence for distributions that contain the same version of the + project.) If there are no distributions for the project, returns + an empty list. + +‘__iter__()’ + + Yield the unique project names of the distributions in this + environment. The yielded names are always in lower case. + +‘add(dist)’ + + Add ‘dist’ to the environment if it matches the platform and python + version specified at creation time, and only if the distribution + hasn’t already been added. (i.e., adding the same distribution + more than once is a no-op.) + +‘remove(dist)’ + + Remove ‘dist’ from the environment. + +‘can_add(dist)’ + + Is distribution ‘dist’ acceptable for this environment? If it’s + not compatible with the ‘platform’ and ‘python’ version values + specified when the environment was created, a false value is + returned. + +‘__add__(dist_or_env)’ (‘+’ operator) + + Add a distribution or environment to an ‘Environment’ instance, + returning a `new' environment object that contains all the + distributions previously contained by both. The new environment + will have a ‘platform’ and ‘python’ of ‘None’, meaning that it will + not reject any distributions from being added to it; it will simply + accept whatever is added. If you want the added items to be + filtered for platform and Python version, or you want to add them + to the `same' environment instance, you should use in-place + addition (‘+=’) instead. + +‘__iadd__(dist_or_env)’ (‘+=’ operator) + + Add a distribution or environment to an ‘Environment’ instance + `in-place', updating the existing instance and returning it. The + ‘platform’ and ‘python’ filter attributes take effect, so + distributions in the source that do not have a suitable platform + string or Python version are silently ignored. + +‘best_match(req, working_set, installer=None)’ + + Find distribution best matching ‘req’ and usable on ‘working_set’ + + This calls the ‘find(req)’ method of the ‘working_set’ to see if a + suitable distribution is already active. (This may raise + ‘VersionConflict’ if an unsuitable version of the project is + already active in the specified ‘working_set’.) If a suitable + distribution isn’t active, this method returns the newest + distribution in the environment that meets the ‘Requirement’ in + ‘req’. If no suitable distribution is found, and ‘installer’ is + supplied, then the result of calling the environment’s ‘obtain(req, + installer)’ method will be returned. + +‘obtain(requirement, installer=None)’ + + Obtain a distro that matches requirement (e.g. via download). In + the base ‘Environment’ class, this routine just returns + ‘installer(requirement)’, unless ‘installer’ is None, in which case + None is returned instead. This method is a hook that allows + subclasses to attempt other ways of obtaining a distribution before + falling back to the ‘installer’ argument. + +‘scan(search_path=None)’ + + Scan ‘search_path’ for distributions usable on ‘platform’ + + Any distributions found are added to the environment. + ‘search_path’ should be a sequence of strings such as might be used + on ‘sys.path’. If not supplied, ‘sys.path’ is used. Only + distributions conforming to the platform/python version defined at + initialization are added. This method is a shortcut for using the + ‘find_distributions()’ function to find the distributions from each + item in ‘search_path’, and then calling ‘add()’ to add each one to + the environment. + + +File: setuptools.info, Node: Requirement Objects, Next: Entry Points<2>, Prev: Environment Objects, Up: API Reference + +3.2.4 ‘Requirement’ Objects +--------------------------- + +‘Requirement’ objects express what versions of a project are suitable +for some purpose. These objects (or their string form) are used by +various ‘pkg_resources’ APIs in order to find distributions that a +script or distribution needs. + +* Menu: + +* Requirements Parsing:: +* Requirement Methods and Attributes:: + + +File: setuptools.info, Node: Requirements Parsing, Next: Requirement Methods and Attributes, Up: Requirement Objects + +3.2.4.1 Requirements Parsing +............................ + +‘parse_requirements(s)’ + + Yield ‘Requirement’ objects for a string or iterable of lines. + Each requirement must start on a new line. See below for syntax. + +‘Requirement.parse(s)’ + + Create a ‘Requirement’ object from a string or iterable of lines. + A ‘ValueError’ is raised if the string or lines do not contain a + valid requirement specifier, or if they contain more than one + specifier. (To parse multiple specifiers from a string or iterable + of strings, use ‘parse_requirements()’ instead.) + + The syntax of a requirement specifier is defined in full in PEP + 508. + + Some examples of valid requirement specifiers: + + FooProject >= 1.2 + Fizzy [foo, bar] + PickyThing>1.6,<=1.9,!=1.8.6 + SomethingWhoseVersionIDontCareAbout + SomethingWithMarker[foo]>1.0;python_version<"2.7" + + The project name is the only required portion of a requirement + string, and if it’s the only thing supplied, the requirement will + accept any version of that project. + + The “extras” in a requirement are used to request optional features + of a project, that may require additional project distributions in + order to function. For example, if the hypothetical + “Report-O-Rama” project offered optional PDF support, it might + require an additional library in order to provide that support. + Thus, a project needing Report-O-Rama’s PDF features could use a + requirement of ‘Report-O-Rama[PDF]’ to request installation or + activation of both Report-O-Rama and any libraries it needs in + order to provide PDF support. For example, you could use: + + pip install Report-O-Rama[PDF] + + To install the necessary packages using pip, or call + ‘pkg_resources.require('Report-O-Rama[PDF]')’ to add the necessary + distributions to sys.path at runtime. + + The “markers” in a requirement are used to specify when a + requirement should be installed – the requirement will be installed + if the marker evaluates as true in the current environment. For + example, specifying ‘argparse;python_version<"3.0"’ will not + install in an Python 3 environment, but will in a Python 2 + environment. + + +File: setuptools.info, Node: Requirement Methods and Attributes, Prev: Requirements Parsing, Up: Requirement Objects + +3.2.4.2 ‘Requirement’ Methods and Attributes +............................................ + +‘__contains__(dist_or_version)’ + + Return true if ‘dist_or_version’ fits the criteria for this + requirement. If ‘dist_or_version’ is a ‘Distribution’ object, its + project name must match the requirement’s project name, and its + version must meet the requirement’s version criteria. If + ‘dist_or_version’ is a string, it is parsed using the + ‘parse_version()’ utility function. Otherwise, it is assumed to be + an already-parsed version. + + The ‘Requirement’ object’s version specifiers (‘.specs’) are + internally sorted into ascending version order, and used to + establish what ranges of versions are acceptable. Adjacent + redundant conditions are effectively consolidated (e.g. ‘">1, >2"’ + produces the same results as ‘">2"’, and ‘"<2,<3"’ produces the + same results as ‘"<2"’). ‘"!="’ versions are excised from the + ranges they fall within. The version being tested for + acceptability is then checked for membership in the resulting + ranges. + +‘__eq__(other_requirement)’ + + A requirement compares equal to another requirement if they have + case-insensitively equal project names, version specifiers, and + “extras”. (The order that extras and version specifiers are in is + also ignored.) Equal requirements also have equal hashes, so that + requirements can be used in sets or as dictionary keys. + +‘__str__()’ + + The string form of a ‘Requirement’ is a string that, if passed to + ‘Requirement.parse()’, would return an equal ‘Requirement’ object. + +‘project_name’ + + The name of the required project + +‘key’ + + An all-lowercase version of the ‘project_name’, useful for + comparison or indexing. + +‘extras’ + + A tuple of names of “extras” that this requirement calls for. + (These will be all-lowercase and normalized using the + ‘safe_extra()’ parsing utility function, so they may not exactly + equal the extras the requirement was created with.) + +‘specs’ + + A list of ‘(op,version)’ tuples, sorted in ascending parsed-version + order. The ‘op’ in each tuple is a comparison operator, + represented as a string. The ‘version’ is the (unparsed) version + number. + +‘marker’ + + An instance of ‘packaging.markers.Marker’ that allows evaluation + against the current environment. May be None if no marker + specified. + +‘url’ + + The location to download the requirement from if specified. + + +File: setuptools.info, Node: Entry Points<2>, Next: Distribution Objects, Prev: Requirement Objects, Up: API Reference + +3.2.5 Entry Points +------------------ + +Entry points are a simple way for distributions to “advertise” Python +objects (such as functions or classes) for use by other distributions. +Extensible applications and frameworks can search for entry points with +a particular name or group, either from a specific distribution or from +all active distributions on sys.path, and then inspect or load the +advertised objects at will. + +Entry points belong to “groups” which are named with a dotted name +similar to a Python package or module name. For example, the +‘setuptools’ package uses an entry point named ‘distutils.commands’ in +order to find commands defined by distutils extensions. ‘setuptools’ +treats the names of entry points defined in that group as the acceptable +commands for a setup script. + +In a similar way, other packages can define their own entry point +groups, either using dynamic names within the group (like +‘distutils.commands’), or possibly using predefined names within the +group. For example, a blogging framework that offers various pre- or +post-publishing hooks might define an entry point group and look for +entry points named “pre_process” and “post_process” within that group. + +To advertise an entry point, a project needs to use ‘setuptools’ and +provide an ‘entry_points’ argument to ‘setup()’ in its setup script, so +that the entry points will be included in the distribution’s metadata. +For more details, see the [‘setuptools’ +documentation](‘https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins’). + +Each project distribution can advertise at most one entry point of a +given name within the same entry point group. For example, a distutils +extension could advertise two different ‘distutils.commands’ entry +points, as long as they had different names. However, there is nothing +that prevents `different' projects from advertising entry points of the +same name in the same group. In some cases, this is a desirable thing, +since the application or framework that uses the entry points may be +calling them as hooks, or in some other way combining them. It is up to +the application or framework to decide what to do if multiple +distributions advertise an entry point; some possibilities include using +both entry points, displaying an error message, using the first one +found in sys.path order, etc. + +* Menu: + +* Convenience API:: +* Creating and Parsing:: +* EntryPoint Objects:: + + +File: setuptools.info, Node: Convenience API, Next: Creating and Parsing, Up: Entry Points<2> + +3.2.5.1 Convenience API +....................... + +In the following functions, the ‘dist’ argument can be a ‘Distribution’ +instance, a ‘Requirement’ instance, or a string specifying a requirement +(i.e. project name, version, etc.). If the argument is a string or +‘Requirement’, the specified distribution is located (and added to +sys.path if not already present). An error will be raised if a matching +distribution is not available. + +The ‘group’ argument should be a string containing a dotted identifier, +identifying an entry point group. If you are defining an entry point +group, you should include some portion of your package’s name in the +group name so as to avoid collision with other packages’ entry point +groups. + +‘load_entry_point(dist, group, name)’ + + Load the named entry point from the specified distribution, or + raise ‘ImportError’. + +‘get_entry_info(dist, group, name)’ + + Return an ‘EntryPoint’ object for the given ‘group’ and ‘name’ from + the specified distribution. Returns ‘None’ if the distribution has + not advertised a matching entry point. + +‘get_entry_map(dist, group=None)’ + + Return the distribution’s entry point map for ‘group’, or the full + entry map for the distribution. This function always returns a + dictionary, even if the distribution advertises no entry points. + If ‘group’ is given, the dictionary maps entry point names to the + corresponding ‘EntryPoint’ object. If ‘group’ is None, the + dictionary maps group names to dictionaries that then map entry + point names to the corresponding ‘EntryPoint’ instance in that + group. + +‘iter_entry_points(group, name=None)’ + + Yield entry point objects from ‘group’ matching ‘name’. + + If ‘name’ is None, yields all entry points in ‘group’ from all + distributions in the working set on sys.path, otherwise only ones + matching both ‘group’ and ‘name’ are yielded. Entry points are + yielded from the active distributions in the order that the + distributions appear on sys.path. (Within entry points for a + particular distribution, however, there is no particular ordering.) + + (This API is actually a method of the global ‘working_set’ object; + see the section above on *note Basic WorkingSet Methods: 7c. for + more information.) + + +File: setuptools.info, Node: Creating and Parsing, Next: EntryPoint Objects, Prev: Convenience API, Up: Entry Points<2> + +3.2.5.2 Creating and Parsing +............................ + +‘EntryPoint(name, module_name, attrs=(), extras=(), dist=None)’ + + Create an ‘EntryPoint’ instance. ‘name’ is the entry point name. + The ‘module_name’ is the (dotted) name of the module containing the + advertised object. ‘attrs’ is an optional tuple of names to look + up from the module to obtain the advertised object. For example, + an ‘attrs’ of ‘("foo","bar")’ and a ‘module_name’ of ‘"baz"’ would + mean that the advertised object could be obtained by the following + code: + + import baz + advertised_object = baz.foo.bar + + The ‘extras’ are an optional tuple of “extra feature” names that + the distribution needs in order to provide this entry point. When + the entry point is loaded, these extra features are looked up in + the ‘dist’ argument to find out what other distributions may need + to be activated on sys.path; see the ‘load()’ method for more + details. The ‘extras’ argument is only meaningful if ‘dist’ is + specified. ‘dist’ must be a ‘Distribution’ instance. + +‘EntryPoint.parse(src, dist=None)’ (classmethod) + + Parse a single entry point from string ‘src’ + + Entry point syntax follows the form: + + name = some.module:some.attr [extra1,extra2] + + The entry name and module name are required, but the ‘:attrs’ and + ‘[extras]’ parts are optional, as is the whitespace shown between + some of the items. The ‘dist’ argument is passed through to the + ‘EntryPoint()’ constructor, along with the other values parsed from + ‘src’. + +‘EntryPoint.parse_group(group, lines, dist=None)’ (classmethod) + + Parse ‘lines’ (a string or sequence of lines) to create a + dictionary mapping entry point names to ‘EntryPoint’ objects. + ‘ValueError’ is raised if entry point names are duplicated, if + ‘group’ is not a valid entry point group name, or if there are any + syntax errors. (Note: the ‘group’ parameter is used only for + validation and to create more informative error messages.) If + ‘dist’ is provided, it will be used to set the ‘dist’ attribute of + the created ‘EntryPoint’ objects. + +‘EntryPoint.parse_map(data, dist=None)’ (classmethod) + + Parse ‘data’ into a dictionary mapping group names to dictionaries + mapping entry point names to ‘EntryPoint’ objects. If ‘data’ is a + dictionary, then the keys are used as group names and the values + are passed to ‘parse_group()’ as the ‘lines’ argument. If ‘data’ + is a string or sequence of lines, it is first split into .ini-style + sections (using the ‘split_sections()’ utility function) and the + section names are used as group names. In either case, the ‘dist’ + argument is passed through to ‘parse_group()’ so that the entry + points will be linked to the specified distribution. + + +File: setuptools.info, Node: EntryPoint Objects, Prev: Creating and Parsing, Up: Entry Points<2> + +3.2.5.3 ‘EntryPoint’ Objects +............................ + +For simple introspection, ‘EntryPoint’ objects have attributes that +correspond exactly to the constructor argument names: ‘name’, +‘module_name’, ‘attrs’, ‘extras’, and ‘dist’ are all available. In +addition, the following methods are provided: + +‘load()’ + + Load the entry point, returning the advertised Python object. + Effectively calls ‘self.require()’ then returns ‘self.resolve()’. + +‘require(env=None, installer=None)’ + + Ensure that any “extras” needed by the entry point are available on + sys.path. ‘UnknownExtra’ is raised if the ‘EntryPoint’ has + ‘extras’, but no ‘dist’, or if the named extras are not defined by + the distribution. If ‘env’ is supplied, it must be an + ‘Environment’, and it will be used to search for needed + distributions if they are not already present on sys.path. If + ‘installer’ is supplied, it must be a callable taking a + ‘Requirement’ instance and returning a matching importable + ‘Distribution’ instance or None. + +‘resolve()’ + + Resolve the entry point from its module and attrs, returning the + advertised Python object. Raises ‘ImportError’ if it cannot be + obtained. + +‘__str__()’ + + The string form of an ‘EntryPoint’ is a string that could be passed + to ‘EntryPoint.parse()’ to produce an equivalent ‘EntryPoint’. + + +File: setuptools.info, Node: Distribution Objects, Next: ResourceManager API, Prev: Entry Points<2>, Up: API Reference + +3.2.6 ‘Distribution’ Objects +---------------------------- + +‘Distribution’ objects represent collections of Python code that may or +may not be importable, and may or may not have metadata and resources +associated with them. Their metadata may include information such as +what other projects the distribution depends on, what entry points the +distribution advertises, and so on. + +* Menu: + +* Getting or Creating Distributions:: +* Distribution Attributes:: +* Distribution Methods:: + + +File: setuptools.info, Node: Getting or Creating Distributions, Next: Distribution Attributes, Up: Distribution Objects + +3.2.6.1 Getting or Creating Distributions +......................................... + +Most commonly, you’ll obtain ‘Distribution’ objects from a ‘WorkingSet’ +or an ‘Environment’. (See the sections above on *note WorkingSet +Objects: 7b. and *note Environment Objects: 81, which are containers for +active distributions and available distributions, respectively.) You +can also obtain ‘Distribution’ objects from one of these high-level +APIs: + +‘find_distributions(path_item, only=False)’ + + Yield distributions accessible via ‘path_item’. If ‘only’ is true, + yield only distributions whose ‘location’ is equal to ‘path_item’. + In other words, if ‘only’ is true, this yields any distributions + that would be importable if ‘path_item’ were on ‘sys.path’. If + ‘only’ is false, this also yields distributions that are “in” or + “under” ‘path_item’, but would not be importable unless their + locations were also added to ‘sys.path’. + +‘get_distribution(dist_spec)’ + + Return a ‘Distribution’ object for a given ‘Requirement’ or string. + If ‘dist_spec’ is already a ‘Distribution’ instance, it is + returned. If it is a ‘Requirement’ object or a string that can be + parsed into one, it is used to locate and activate a matching + distribution, which is then returned. + +However, if you’re creating specialized tools for working with +distributions, or creating a new distribution format, you may also need +to create ‘Distribution’ objects directly, using one of the three +constructors below. + +These constructors all take an optional ‘metadata’ argument, which is +used to access any resources or metadata associated with the +distribution. ‘metadata’ must be an object that implements the +‘IResourceProvider’ interface, or None. If it is None, an +‘EmptyProvider’ is used instead. ‘Distribution’ objects implement both +the *note IResourceProvider: 8b. and *note IMetadataProvider Methods: +8c. by delegating them to the ‘metadata’ object. + +‘Distribution.from_location(location, basename, metadata=None, **kw)’ (classmethod) + + Create a distribution for ‘location’, which must be a string such + as a URL, filename, or other string that might be used on + ‘sys.path’. ‘basename’ is a string naming the distribution, like + ‘Foo-1.2-py2.4.egg’. If ‘basename’ ends with ‘.egg’, then the + project’s name, version, python version and platform are extracted + from the filename and used to set those properties of the created + distribution. Any additional keyword arguments are forwarded to + the ‘Distribution()’ constructor. + +‘Distribution.from_filename(filename, metadata=None**kw)’ (classmethod) + + Create a distribution by parsing a local filename. This is a + shorter way of saying + ‘Distribution.from_location(normalize_path(filename), + os.path.basename(filename), metadata)’. In other words, it creates + a distribution whose location is the normalize form of the + filename, parsing name and version information from the base + portion of the filename. Any additional keyword arguments are + forwarded to the ‘Distribution()’ constructor. + +‘Distribution(location,metadata,project_name,version,py_version,platform,precedence)’ + + Create a distribution by setting its properties. All arguments are + optional and default to None, except for ‘py_version’ (which + defaults to the current Python version) and ‘precedence’ (which + defaults to ‘EGG_DIST’; for more details see ‘precedence’ under + *note Distribution Attributes: 8d. below). Note that it’s usually + easier to use the ‘from_filename()’ or ‘from_location()’ + constructors than to specify all these arguments individually. + + +File: setuptools.info, Node: Distribution Attributes, Next: Distribution Methods, Prev: Getting or Creating Distributions, Up: Distribution Objects + +3.2.6.2 ‘Distribution’ Attributes +................................. + +location + + A string indicating the distribution’s location. For an importable + distribution, this is the string that would be added to ‘sys.path’ + to make it actively importable. For non-importable distributions, + this is simply a filename, URL, or other way of locating the + distribution. + +project_name + + A string, naming the project that this distribution is for. + Project names are defined by a project’s setup script, and they are + used to identify projects on PyPI. When a ‘Distribution’ is + constructed, the ‘project_name’ argument is passed through the + ‘safe_name()’ utility function to filter out any unacceptable + characters. + +key + + ‘dist.key’ is short for ‘dist.project_name.lower()’. It’s used for + case-insensitive comparison and indexing of distributions by + project name. + +extras + + A list of strings, giving the names of extra features defined by + the project’s dependency list (the ‘extras_require’ argument + specified in the project’s setup script). + +version + + A string denoting what release of the project this distribution + contains. When a ‘Distribution’ is constructed, the ‘version’ + argument is passed through the ‘safe_version()’ utility function to + filter out any unacceptable characters. If no ‘version’ is + specified at construction time, then attempting to access this + attribute later will cause the ‘Distribution’ to try to discover + its version by reading its ‘PKG-INFO’ metadata file. If ‘PKG-INFO’ + is unavailable or can’t be parsed, ‘ValueError’ is raised. + +parsed_version + + The ‘parsed_version’ is an object representing a “parsed” form of + the distribution’s ‘version’. ‘dist.parsed_version’ is a shortcut + for calling ‘parse_version(dist.version)’. It is used to compare + or sort distributions by version. (See the *note Parsing + Utilities: 8e. section below for more information on the + ‘parse_version()’ function.) Note that accessing ‘parsed_version’ + may result in a ‘ValueError’ if the ‘Distribution’ was constructed + without a ‘version’ and without ‘metadata’ capable of supplying the + missing version info. + +py_version + + The major/minor Python version the distribution supports, as a + string. For example, “2.7” or “3.4”. The default is the current + version of Python. + +platform + + A string representing the platform the distribution is intended + for, or ‘None’ if the distribution is “pure Python” and therefore + cross-platform. See *note Platform Utilities: 8f. below for more + information on platform strings. + +precedence + + A distribution’s ‘precedence’ is used to determine the relative + order of two distributions that have the same ‘project_name’ and + ‘parsed_version’. The default precedence is + ‘pkg_resources.EGG_DIST’, which is the highest (i.e. most + preferred) precedence. The full list of predefined precedences, + from most preferred to least preferred, is: ‘EGG_DIST’, + ‘BINARY_DIST’, ‘SOURCE_DIST’, ‘CHECKOUT_DIST’, and ‘DEVELOP_DIST’. + Normally, precedences other than ‘EGG_DIST’ are used only by the + ‘setuptools.package_index’ module, when sorting distributions found + in a package index to determine their suitability for installation. + “System” and “Development” eggs (i.e., ones that use the + ‘.egg-info’ format), however, are automatically given a precedence + of ‘DEVELOP_DIST’. + + +File: setuptools.info, Node: Distribution Methods, Prev: Distribution Attributes, Up: Distribution Objects + +3.2.6.3 ‘Distribution’ Methods +.............................. + +‘activate(path=None)’ + + Ensure distribution is importable on ‘path’. If ‘path’ is None, + ‘sys.path’ is used instead. This ensures that the distribution’s + ‘location’ is in the ‘path’ list, and it also performs any + necessary namespace package fixups or declarations. (That is, if + the distribution contains namespace packages, this method ensures + that they are declared, and that the distribution’s contents for + those namespace packages are merged with the contents provided by + any other active distributions. See the section above on *note + Namespace Package Support: 79. for more information.) + + ‘pkg_resources’ adds a notification callback to the global + ‘working_set’ that ensures this method is called whenever a + distribution is added to it. Therefore, you should not normally + need to explicitly call this method. (Note that this means that + namespace packages on ‘sys.path’ are always imported as soon as + ‘pkg_resources’ is, which is another reason why namespace packages + should not contain any code or import statements.) + +‘as_requirement()’ + + Return a ‘Requirement’ instance that matches this distribution’s + project name and version. + +‘requires(extras=())’ + + List the ‘Requirement’ objects that specify this distribution’s + dependencies. If ‘extras’ is specified, it should be a sequence of + names of “extras” defined by the distribution, and the list + returned will then include any dependencies needed to support the + named “extras”. + +‘clone(**kw)’ + + Create a copy of the distribution. Any supplied keyword arguments + override the corresponding argument to the ‘Distribution()’ + constructor, allowing you to change some of the copied + distribution’s attributes. + +‘egg_name()’ + + Return what this distribution’s standard filename should be, not + including the “.egg” extension. For example, a distribution for + project “Foo” version 1.2 that runs on Python 2.3 for Windows would + have an ‘egg_name()’ of ‘Foo-1.2-py2.3-win32’. Any dashes in the + name or version are converted to underscores. + (‘Distribution.from_location()’ will convert them back when parsing + a “.egg” file name.) + +‘__cmp__(other)’, ‘__hash__()’ + + Distribution objects are hashed and compared on the basis of their + parsed version and precedence, followed by their key (lowercase + project name), location, Python version, and platform. + +The following methods are used to access ‘EntryPoint’ objects advertised +by the distribution. See the section above on *note Entry Points: 7f. +for more detailed information about these operations: + +‘get_entry_info(group, name)’ + + Return the ‘EntryPoint’ object for ‘group’ and ‘name’, or None if + no such point is advertised by this distribution. + +‘get_entry_map(group=None)’ + + Return the entry point map for ‘group’. If ‘group’ is None, return + a dictionary mapping group names to entry point maps for all + groups. (An entry point map is a dictionary of entry point names + to ‘EntryPoint’ objects.) + +‘load_entry_point(group, name)’ + + Short for ‘get_entry_info(group, name).load()’. Returns the object + advertised by the named entry point, or raises ‘ImportError’ if the + entry point isn’t advertised by this distribution, or there is some + other import problem. + +In addition to the above methods, ‘Distribution’ objects also implement +all of the *note IResourceProvider: 8b. and *note IMetadataProvider +Methods: 8c. (which are documented in later sections): + + * ‘has_metadata(name)’ + + * ‘metadata_isdir(name)’ + + * ‘metadata_listdir(name)’ + + * ‘get_metadata(name)’ + + * ‘get_metadata_lines(name)’ + + * ‘run_script(script_name, namespace)’ + + * ‘get_resource_filename(manager, resource_name)’ + + * ‘get_resource_stream(manager, resource_name)’ + + * ‘get_resource_string(manager, resource_name)’ + + * ‘has_resource(resource_name)’ + + * ‘resource_isdir(resource_name)’ + + * ‘resource_listdir(resource_name)’ + +If the distribution was created with a ‘metadata’ argument, these +resource and metadata access methods are all delegated to that +‘metadata’ provider. Otherwise, they are delegated to an +‘EmptyProvider’, so that the distribution will appear to have no +resources or metadata. This delegation approach is used so that +supporting custom importers or new distribution formats can be done +simply by creating an appropriate *note IResourceProvider: 8b. +implementation; see the section below on *note Supporting Custom +Importers: 7a. for more details. + + +File: setuptools.info, Node: ResourceManager API, Next: Metadata API, Prev: Distribution Objects, Up: API Reference + +3.2.7 ‘ResourceManager’ API +--------------------------- + +The ‘ResourceManager’ class provides uniform access to package +resources, whether those resources exist as files and directories or are +compressed in an archive of some kind. + +Normally, you do not need to create or explicitly manage +‘ResourceManager’ instances, as the ‘pkg_resources’ module creates a +global instance for you, and makes most of its methods available as +top-level names in the ‘pkg_resources’ module namespace. So, for +example, this code actually calls the ‘resource_string()’ method of the +global ‘ResourceManager’: + + import pkg_resources + my_data = pkg_resources.resource_string(__name__, "foo.dat") + +Thus, you can use the APIs below without needing an explicit +‘ResourceManager’ instance; just import and use them as needed. + +* Menu: + +* Basic Resource Access:: +* Resource Extraction:: +* “Provider” Interface:: + + +File: setuptools.info, Node: Basic Resource Access, Next: Resource Extraction, Up: ResourceManager API + +3.2.7.1 Basic Resource Access +............................. + +In the following methods, the ‘package_or_requirement’ argument may be +either a Python package/module name (e.g. ‘foo.bar’) or a ‘Requirement’ +instance. If it is a package or module name, the named module or +package must be importable (i.e., be in a distribution or directory on +‘sys.path’), and the ‘resource_name’ argument is interpreted relative to +the named package. (Note that if a module name is used, then the +resource name is relative to the package immediately containing the +named module. Also, you should not use use a namespace package name, +because a namespace package can be spread across multiple distributions, +and is therefore ambiguous as to which distribution should be searched +for the resource.) + +If it is a ‘Requirement’, then the requirement is automatically resolved +(searching the current ‘Environment’ if necessary) and a matching +distribution is added to the ‘WorkingSet’ and ‘sys.path’ if one was not +already present. (Unless the ‘Requirement’ can’t be satisfied, in which +case an exception is raised.) The ‘resource_name’ argument is then +interpreted relative to the root of the identified distribution; i.e. +its first path segment will be treated as a peer of the top-level +modules or packages in the distribution. + +Note that resource names must be ‘/’-separated paths rooted at the +package, cannot contain relative names like ‘".."’, and cannot be +absolute. Do `not' use ‘os.path’ routines to manipulate resource paths, +as they are `not' filesystem paths. + +‘resource_exists(package_or_requirement, resource_name)’ + + Does the named resource exist? Return ‘True’ or ‘False’ + accordingly. + +‘resource_stream(package_or_requirement, resource_name)’ + + Return a readable file-like object for the specified resource; it + may be an actual file, a ‘StringIO’, or some similar object. The + stream is in “binary mode”, in the sense that whatever bytes are in + the resource will be read as-is. + +‘resource_string(package_or_requirement, resource_name)’ + + Return the specified resource as a string. The resource is read in + binary fashion, such that the returned string contains exactly the + bytes that are stored in the resource. + +‘resource_isdir(package_or_requirement, resource_name)’ + + Is the named resource a directory? Return ‘True’ or ‘False’ + accordingly. + +‘resource_listdir(package_or_requirement, resource_name)’ + + List the contents of the named resource directory, just like + ‘os.listdir’ except that it works even if the resource is in a + zipfile. + +Note that only ‘resource_exists()’ and ‘resource_isdir()’ are +insensitive as to the resource type. You cannot use +‘resource_listdir()’ on a file resource, and you can’t use +‘resource_string()’ or ‘resource_stream()’ on directory resources. +Using an inappropriate method for the resource type may result in an +exception or undefined behavior, depending on the platform and +distribution format involved. + + +File: setuptools.info, Node: Resource Extraction, Next: “Provider” Interface, Prev: Basic Resource Access, Up: ResourceManager API + +3.2.7.2 Resource Extraction +........................... + +‘resource_filename(package_or_requirement, resource_name)’ + + Sometimes, it is not sufficient to access a resource in string or + stream form, and a true filesystem filename is needed. In such + cases, you can use this method (or module-level function) to obtain + a filename for a resource. If the resource is in an archive + distribution (such as a zipped egg), it will be extracted to a + cache directory, and the filename within the cache will be + returned. If the named resource is a directory, then all resources + within that directory (including subdirectories) are also + extracted. If the named resource is a C extension or “eager + resource” (see the ‘setuptools’ documentation for details), then + all C extensions and eager resources are extracted at the same + time. + + Archived resources are extracted to a cache location that can be + managed by the following two methods: + +‘set_extraction_path(path)’ + + Set the base path where resources will be extracted to, if needed. + + If you do not call this routine before any extractions take place, + the path defaults to the return value of ‘get_default_cache()’. + (Which is based on the ‘PYTHON_EGG_CACHE’ environment variable, + with various platform-specific fallbacks. See that routine’s + documentation for more details.) + + Resources are extracted to subdirectories of this path based upon + information given by the resource provider. You may set this to a + temporary directory, but then you must call ‘cleanup_resources()’ + to delete the extracted files when done. There is no guarantee + that ‘cleanup_resources()’ will be able to remove all extracted + files. (On Windows, for example, you can’t unlink .pyd or .dll + files that are still in use.) + + Note that you may not change the extraction path for a given + resource manager once resources have been extracted, unless you + first call ‘cleanup_resources()’. + +‘cleanup_resources(force=False)’ + + Delete all extracted resource files and directories, returning a + list of the file and directory names that could not be successfully + removed. This function does not have any concurrency protection, + so it should generally only be called when the extraction path is a + temporary directory exclusive to a single process. This method is + not automatically called; you must call it explicitly or register + it as an ‘atexit’ function if you wish to ensure cleanup of a + temporary directory used for extractions. + + +File: setuptools.info, Node: “Provider” Interface, Prev: Resource Extraction, Up: ResourceManager API + +3.2.7.3 “Provider” Interface +............................ + +If you are implementing an ‘IResourceProvider’ and/or +‘IMetadataProvider’ for a new distribution archive format, you may need +to use the following ‘IResourceManager’ methods to co-ordinate +extraction of resources to the filesystem. If you’re not implementing +an archive format, however, you have no need to use these methods. +Unlike the other methods listed above, they are `not' available as +top-level functions tied to the global ‘ResourceManager’; you must +therefore have an explicit ‘ResourceManager’ instance to use them. + +‘get_cache_path(archive_name, names=())’ + + Return absolute location in cache for ‘archive_name’ and ‘names’ + + The parent directory of the resulting path will be created if it + does not already exist. ‘archive_name’ should be the base filename + of the enclosing egg (which may not be the name of the enclosing + zipfile!), including its “.egg” extension. ‘names’, if provided, + should be a sequence of path name parts “under” the egg’s + extraction location. + + This method should only be called by resource providers that need + to obtain an extraction location, and only for names they intend to + extract, as it tracks the generated names for possible cleanup + later. + +‘extraction_error()’ + + Raise an ‘ExtractionError’ describing the active exception as + interfering with the extraction process. You should call this if + you encounter any OS errors extracting the file to the cache path; + it will format the operating system exception for you, and add + other information to the ‘ExtractionError’ instance that may be + needed by programs that want to wrap or handle extraction errors + themselves. + +‘postprocess(tempname, filename)’ + + Perform any platform-specific postprocessing of ‘tempname’. + Resource providers should call this method ONLY after successfully + extracting a compressed resource. They must NOT call it on + resources that are already in the filesystem. + + ‘tempname’ is the current (temporary) name of the file, and + ‘filename’ is the name it will be renamed to by the caller after + this routine returns. + + +File: setuptools.info, Node: Metadata API, Next: Exceptions, Prev: ResourceManager API, Up: API Reference + +3.2.8 Metadata API +------------------ + +The metadata API is used to access metadata resources bundled in a +pluggable distribution. Metadata resources are virtual files or +directories containing information about the distribution, such as might +be used by an extensible application or framework to connect “plugins”. +Like other kinds of resources, metadata resource names are ‘/’-separated +and should not contain ‘..’ or begin with a ‘/’. You should not use +‘os.path’ routines to manipulate resource paths. + +The metadata API is provided by objects implementing the +‘IMetadataProvider’ or ‘IResourceProvider’ interfaces. ‘Distribution’ +objects implement this interface, as do objects returned by the +‘get_provider()’ function: + +‘get_provider(package_or_requirement)’ + + If a package name is supplied, return an ‘IResourceProvider’ for + the package. If a ‘Requirement’ is supplied, resolve it by + returning a ‘Distribution’ from the current working set (searching + the current ‘Environment’ if necessary and adding the newly found + ‘Distribution’ to the working set). If the named package can’t be + imported, or the ‘Requirement’ can’t be satisfied, an exception is + raised. + + NOTE: if you use a package name rather than a ‘Requirement’, the + object you get back may not be a pluggable distribution, depending + on the method by which the package was installed. In particular, + “development” packages and “single-version externally-managed” + packages do not have any way to map from a package name to the + corresponding project’s metadata. Do not write code that passes a + package name to ‘get_provider()’ and then tries to retrieve project + metadata from the returned object. It may appear to work when the + named package is in an ‘.egg’ file or directory, but it will fail + in other installation scenarios. If you want project metadata, you + need to ask for a `project', not a package. + +* Menu: + +* IMetadataProvider Methods:: + + +File: setuptools.info, Node: IMetadataProvider Methods, Up: Metadata API + +3.2.8.1 ‘IMetadataProvider’ Methods +................................... + +The methods provided by objects (such as ‘Distribution’ instances) that +implement the ‘IMetadataProvider’ or ‘IResourceProvider’ interfaces are: + +‘has_metadata(name)’ + + Does the named metadata resource exist? + +‘metadata_isdir(name)’ + + Is the named metadata resource a directory? + +‘metadata_listdir(name)’ + + List of metadata names in the directory (like ‘os.listdir()’) + +‘get_metadata(name)’ + + Return the named metadata resource as a string. The data is read + in binary mode; i.e., the exact bytes of the resource file are + returned. + +‘get_metadata_lines(name)’ + + Yield named metadata resource as list of non-blank non-comment + lines. This is short for calling + ‘yield_lines(provider.get_metadata(name))’. See the section on + *note yield_lines(): 95. below for more information on the syntax + it recognizes. + +‘run_script(script_name, namespace)’ + + Execute the named script in the supplied namespace dictionary. + Raises ‘ResolutionError’ if there is no script by that name in the + ‘scripts’ metadata directory. ‘namespace’ should be a Python + dictionary, usually a module dictionary if the script is being run + as a module. + + +File: setuptools.info, Node: Exceptions, Next: Supporting Custom Importers, Prev: Metadata API, Up: API Reference + +3.2.9 Exceptions +---------------- + +‘pkg_resources’ provides a simple exception hierarchy for problems that +may occur when processing requests to locate and activate packages: + + ResolutionError + DistributionNotFound + VersionConflict + UnknownExtra + + ExtractionError + +‘ResolutionError’ + + This class is used as a base class for the other three exceptions, + so that you can catch all of them with a single “except” clause. + It is also raised directly for miscellaneous requirement-resolution + problems like trying to run a script that doesn’t exist in the + distribution it was requested from. + +‘DistributionNotFound’ + + A distribution needed to fulfill a requirement could not be found. + +‘VersionConflict’ + + The requested version of a project conflicts with an + already-activated version of the same project. + +‘UnknownExtra’ + + One of the “extras” requested was not recognized by the + distribution it was requested from. + +‘ExtractionError’ + + A problem occurred extracting a resource to the Python Egg cache. + The following attributes are available on instances of this + exception: + + manager + + The resource manager that raised this exception + + cache_path + + The base directory for resource extraction + + original_error + + The exception instance that caused extraction to fail + + +File: setuptools.info, Node: Supporting Custom Importers, Next: Utility Functions, Prev: Exceptions, Up: API Reference + +3.2.10 Supporting Custom Importers +---------------------------------- + +By default, ‘pkg_resources’ supports normal filesystem imports, and +‘zipimport’ importers. If you wish to use the ‘pkg_resources’ features +with other (PEP 302-compatible) importers or module loaders, you may +need to register various handlers and support functions using these +APIs: + +‘register_finder(importer_type, distribution_finder)’ + + Register ‘distribution_finder’ to find distributions in ‘sys.path’ + items. ‘importer_type’ is the type or class of a PEP 302 + “Importer” (‘sys.path’ item handler), and ‘distribution_finder’ is + a callable that, when passed a path item, the importer instance, + and an ‘only’ flag, yields ‘Distribution’ instances found under + that path item. (The ‘only’ flag, if true, means the finder should + yield only ‘Distribution’ objects whose ‘location’ is equal to the + path item provided.) + + See the source of the ‘pkg_resources.find_on_path’ function for an + example finder function. + +‘register_loader_type(loader_type, provider_factory)’ + + Register ‘provider_factory’ to make ‘IResourceProvider’ objects for + ‘loader_type’. ‘loader_type’ is the type or class of a PEP 302 + ‘module.__loader__’, and ‘provider_factory’ is a function that, + when passed a module object, returns an *note IResourceProvider: + 8b. for that module, allowing it to be used with the *note + ResourceManager API: 35. + +‘register_namespace_handler(importer_type, namespace_handler)’ + + Register ‘namespace_handler’ to declare namespace packages for the + given ‘importer_type’. ‘importer_type’ is the type or class of a + PEP 302 “importer” (sys.path item handler), and ‘namespace_handler’ + is a callable with a signature like this: + + def namespace_handler(importer, path_entry, moduleName, module): + # return a path_entry to use for child packages + + Namespace handlers are only called if the relevant importer object + has already agreed that it can handle the relevant path item. The + handler should only return a subpath if the module ‘__path__’ does + not already contain an equivalent subpath. Otherwise, it should + return None. + + For an example namespace handler, see the source of the + ‘pkg_resources.file_ns_handler’ function, which is used for both + zipfile importing and regular importing. + +* Menu: + +* IResourceProvider:: +* Built-in Resource Providers:: + + +File: setuptools.info, Node: IResourceProvider, Next: Built-in Resource Providers, Up: Supporting Custom Importers + +3.2.10.1 IResourceProvider +.......................... + +‘IResourceProvider’ is an abstract class that documents what methods are +required of objects returned by a ‘provider_factory’ registered with +‘register_loader_type()’. ‘IResourceProvider’ is a subclass of +‘IMetadataProvider’, so objects that implement this interface must also +implement all of the *note IMetadataProvider Methods: 8c. as well as the +methods shown here. The ‘manager’ argument to the methods below must be +an object that supports the full *note ResourceManager API: 35. +documented above. + +‘get_resource_filename(manager, resource_name)’ + + Return a true filesystem path for ‘resource_name’, coordinating the + extraction with ‘manager’, if the resource must be unpacked to the + filesystem. + +‘get_resource_stream(manager, resource_name)’ + + Return a readable file-like object for ‘resource_name’. + +‘get_resource_string(manager, resource_name)’ + + Return a string containing the contents of ‘resource_name’. + +‘has_resource(resource_name)’ + + Does the package contain the named resource? + +‘resource_isdir(resource_name)’ + + Is the named resource a directory? Return a false value if the + resource does not exist or is not a directory. + +‘resource_listdir(resource_name)’ + + Return a list of the contents of the resource directory, ala + ‘os.listdir()’. Requesting the contents of a non-existent + directory may raise an exception. + +Note, by the way, that your provider classes need not (and should not) +subclass ‘IResourceProvider’ or ‘IMetadataProvider’! These classes +exist solely for documentation purposes and do not provide any useful +implementation code. You may instead wish to subclass one of the *note +built-in resource providers: 97. + + +File: setuptools.info, Node: Built-in Resource Providers, Prev: IResourceProvider, Up: Supporting Custom Importers + +3.2.10.2 Built-in Resource Providers +.................................... + +‘pkg_resources’ includes several provider classes that are automatically +used where appropriate. Their inheritance tree looks like this: + + NullProvider + EggProvider + DefaultProvider + PathMetadata + ZipProvider + EggMetadata + EmptyProvider + FileMetadata + +‘NullProvider’ + + This provider class is just an abstract base that provides for + common provider behaviors (such as running scripts), given a + definition for just a few abstract methods. + +‘EggProvider’ + + This provider class adds in some egg-specific features that are + common to zipped and unzipped eggs. + +‘DefaultProvider’ + + This provider class is used for unpacked eggs and “plain old + Python” filesystem modules. + +‘ZipProvider’ + + This provider class is used for all zipped modules, whether they + are eggs or not. + +‘EmptyProvider’ + + This provider class always returns answers consistent with a + provider that has no metadata or resources. ‘Distribution’ objects + created without a ‘metadata’ argument use an instance of this + provider class instead. Since all ‘EmptyProvider’ instances are + equivalent, there is no need to have more than one instance. + ‘pkg_resources’ therefore creates a global instance of this class + under the name ‘empty_provider’, and you may use it if you have + need of an ‘EmptyProvider’ instance. + +‘PathMetadata(path, egg_info)’ + + Create an ‘IResourceProvider’ for a filesystem-based distribution, + where ‘path’ is the filesystem location of the importable modules, + and ‘egg_info’ is the filesystem location of the distribution’s + metadata directory. ‘egg_info’ should usually be the ‘EGG-INFO’ + subdirectory of ‘path’ for an “unpacked egg”, and a + ‘ProjectName.egg-info’ subdirectory of ‘path’ for a “development + egg”. However, other uses are possible for custom purposes. + +‘EggMetadata(zipimporter)’ + + Create an ‘IResourceProvider’ for a zipfile-based distribution. + The ‘zipimporter’ should be a ‘zipimport.zipimporter’ instance, and + may represent a “basket” (a zipfile containing multiple “.egg” + subdirectories) a specific egg `within' a basket, or a zipfile egg + (where the zipfile itself is a “.egg”). It can also be a + combination, such as a zipfile egg that also contains other eggs. + +‘FileMetadata(path_to_pkg_info)’ + + Create an ‘IResourceProvider’ that provides exactly one metadata + resource: ‘PKG-INFO’. The supplied path should be a distutils + PKG-INFO file. This is basically the same as an ‘EmptyProvider’, + except that requests for ‘PKG-INFO’ will be answered using the + contents of the designated file. (This provider is used to wrap + ‘.egg-info’ files installed by vendor-supplied system packages.) + + +File: setuptools.info, Node: Utility Functions, Prev: Supporting Custom Importers, Up: API Reference + +3.2.11 Utility Functions +------------------------ + +In addition to its high-level APIs, ‘pkg_resources’ also includes +several generally-useful utility routines. These routines are used to +implement the high-level APIs, but can also be quite useful by +themselves. + +* Menu: + +* Parsing Utilities:: +* Platform Utilities:: +* PEP 302 Utilities:: +* File/Path Utilities:: +* History:: + + +File: setuptools.info, Node: Parsing Utilities, Next: Platform Utilities, Up: Utility Functions + +3.2.11.1 Parsing Utilities +.......................... + +‘parse_version(version)’ + + Parsed a project’s version string as defined by PEP 440. The + returned value will be an object that represents the version. + These objects may be compared to each other and sorted. The + sorting algorithm is as defined by PEP 440 with the addition that + any version which is not a valid PEP 440 version will be considered + less than any valid PEP 440 version and the invalid versions will + continue sorting using the original algorithm. + +‘yield_lines(strs)’ + + Yield non-empty/non-comment lines from a string/unicode or a + possibly- nested sequence thereof. If ‘strs’ is an instance of + ‘basestring’, it is split into lines, and each non-blank, + non-comment line is yielded after stripping leading and trailing + whitespace. (Lines whose first non-blank character is ‘#’ are + considered comment lines.) + + If ‘strs’ is not an instance of ‘basestring’, it is iterated over, + and each item is passed recursively to ‘yield_lines()’, so that an + arbitrarily nested sequence of strings, or sequences of sequences + of strings can be flattened out to the lines contained therein. So + for example, passing a file object or a list of strings to + ‘yield_lines’ will both work. (Note that between each string in a + sequence of strings there is assumed to be an implicit line break, + so lines cannot bridge two strings in a sequence.) + + This routine is used extensively by ‘pkg_resources’ to parse + metadata and file formats of various kinds, and most other + ‘pkg_resources’ parsing functions that yield multiple values will + use it to break up their input. However, this routine is + idempotent, so calling ‘yield_lines()’ on the output of another + call to ‘yield_lines()’ is completely harmless. + +‘split_sections(strs)’ + + Split a string (or possibly-nested iterable thereof), yielding + ‘(section, content)’ pairs found using an ‘.ini’-like syntax. Each + ‘section’ is a whitespace-stripped version of the section name + (“‘[section]’”) and each ‘content’ is a list of stripped lines + excluding blank lines and comment-only lines. If there are any + non-blank, non-comment lines before the first section header, + they’re yielded in a first ‘section’ of ‘None’. + + This routine uses ‘yield_lines()’ as its front end, so you can pass + in anything that ‘yield_lines()’ accepts, such as an open text + file, string, or sequence of strings. ‘ValueError’ is raised if a + malformed section header is found (i.e. a line starting with ‘[’ + but not ending with ‘]’). + + Note that this simplistic parser assumes that any line whose first + nonblank character is ‘[’ is a section heading, so it can’t support + .ini format variations that allow ‘[’ as the first nonblank + character on other lines. + +‘safe_name(name)’ + + Return a “safe” form of a project’s name, suitable for use in a + ‘Requirement’ string, as a distribution name, or a PyPI project + name. All non-alphanumeric runs are condensed to single “-” + characters, such that a name like “The $$$ Tree” becomes + “The-Tree”. Note that if you are generating a filename from this + value you should combine it with a call to ‘to_filename()’ so all + dashes (“-“) are replaced by underscores (“_”). See + ‘to_filename()’. + +‘safe_version(version)’ + + This will return the normalized form of any PEP 440 version. If + the version string is not PEP 440 compatible, this function behaves + similar to ‘safe_name()’ except that spaces in the input become + dots, and dots are allowed to exist in the output. As with + ‘safe_name()’, if you are generating a filename from this you + should replace any “-” characters in the output with underscores. + +‘safe_extra(extra)’ + + Return a “safe” form of an extra’s name, suitable for use in a + requirement string or a setup script’s ‘extras_require’ keyword. + This routine is similar to ‘safe_name()’ except that + non-alphanumeric runs are replaced by a single underbar (‘_’), and + the result is lowercased. + +‘to_filename(name_or_version)’ + + Escape a name or version string so it can be used in a + dash-separated filename (or ‘#egg=name-version’ tag) without + ambiguity. You should only pass in values that were returned by + ‘safe_name()’ or ‘safe_version()’. + + +File: setuptools.info, Node: Platform Utilities, Next: PEP 302 Utilities, Prev: Parsing Utilities, Up: Utility Functions + +3.2.11.2 Platform Utilities +........................... + +‘get_build_platform()’ + + Return this platform’s identifier string. For Windows, the return + value is ‘"win32"’, and for macOS it is a string of the form + ‘"macosx-10.4-ppc"’. All other platforms return the same + uname-based string that the ‘distutils.util.get_platform()’ + function returns. This string is the minimum platform version + required by distributions built on the local machine. (Backward + compatibility note: setuptools versions prior to 0.6b1 called this + function ‘get_platform()’, and the function is still available + under that name for backward compatibility reasons.) + +‘get_supported_platform()’ (New in 0.6b1) + + This is the similar to ‘get_build_platform()’, but is the maximum + platform version that the local machine supports. You will usually + want to use this value as the ‘provided’ argument to the + ‘compatible_platforms()’ function. + +‘compatible_platforms(provided, required)’ + + Return true if a distribution built on the ‘provided’ platform may + be used on the ‘required’ platform. If either platform value is + ‘None’, it is considered a wildcard, and the platforms are + therefore compatible. Likewise, if the platform strings are equal, + they’re also considered compatible, and ‘True’ is returned. + Currently, the only non-equal platform strings that are considered + compatible are macOS platform strings with the same hardware type + (e.g. ‘ppc’) and major version (e.g. ‘10’) with the ‘provided’ + platform’s minor version being less than or equal to the ‘required’ + platform’s minor version. + +‘get_default_cache()’ + + Determine the default cache location for extracting resources from + zipped eggs. This routine returns the ‘PYTHON_EGG_CACHE’ + environment variable, if set. Otherwise, on Windows, it returns a + “Python-Eggs” subdirectory of the user’s “Application Data” + directory. On all other systems, it returns + ‘os.path.expanduser("~/.python-eggs")’ if ‘PYTHON_EGG_CACHE’ is not + set. + + +File: setuptools.info, Node: PEP 302 Utilities, Next: File/Path Utilities, Prev: Platform Utilities, Up: Utility Functions + +3.2.11.3 PEP 302 Utilities +.......................... + +‘get_importer(path_item)’ + + A deprecated alias for ‘pkgutil.get_importer()’ + + +File: setuptools.info, Node: File/Path Utilities, Next: History, Prev: PEP 302 Utilities, Up: Utility Functions + +3.2.11.4 File/Path Utilities +............................ + +‘ensure_directory(path)’ + + Ensure that the parent directory (‘os.path.dirname’) of ‘path’ + actually exists, using ‘os.makedirs()’ if necessary. + +‘normalize_path(path)’ + + Return a “normalized” version of ‘path’, such that two paths + represent the same filesystem location if they have equal + ‘normalized_path()’ values. Specifically, this is a shortcut for + calling ‘os.path.realpath’ and ‘os.path.normcase’ on ‘path’. + Unfortunately, on certain platforms (notably Cygwin and macOS) the + ‘normcase’ function does not accurately reflect the platform’s + case-sensitivity, so there is always the possibility of two + apparently-different paths being equal on such platforms. + + +File: setuptools.info, Node: History, Prev: File/Path Utilities, Up: Utility Functions + +3.2.11.5 History +................ + +0.6c9 + + * Fix ‘resource_listdir('')’ always returning an empty list for + zipped eggs. + +0.6c7 + + * Fix package precedence problem where single-version eggs + installed in ‘site-packages’ would take precedence over ‘.egg’ + files (or directories) installed in ‘site-packages’. + +0.6c6 + + * Fix extracted C extensions not having executable permissions + under Cygwin. + + * Allow ‘.egg-link’ files to contain relative paths. + + * Fix cache dir defaults on Windows when multiple environment + vars are needed to construct a path. + +0.6c4 + + * Fix “dev” versions being considered newer than release + candidates. + +0.6c3 + + * Python 2.5 compatibility fixes. + +0.6c2 + + * Fix a problem with eggs specified directly on ‘PYTHONPATH’ on + case-insensitive filesystems possibly not showing up in the + default working set, due to differing normalizations of + ‘sys.path’ entries. + +0.6b3 + + * Fixed a duplicate path insertion problem on case-insensitive + filesystems. + +0.6b1 + + * Split ‘get_platform()’ into ‘get_supported_platform()’ and + ‘get_build_platform()’ to work around a Mac versioning problem + that caused the behavior of ‘compatible_platforms()’ to be + platform specific. + + * Fix entry point parsing when a standalone module name has + whitespace between it and the extras. + +0.6a11 + + * Added ‘ExtractionError’ and + ‘ResourceManager.extraction_error()’ so that cache permission + problems get a more user-friendly explanation of the problem, + and so that programs can catch and handle extraction errors if + they need to. + +0.6a10 + + * Added the ‘extras’ attribute to ‘Distribution’, the + ‘find_plugins()’ method to ‘WorkingSet’, and the ‘__add__()’ + and ‘__iadd__()’ methods to ‘Environment’. + + * ‘safe_name()’ now allows dots in project names. + + * There is a new ‘to_filename()’ function that escapes project + names and versions for safe use in constructing egg filenames + from a Distribution object’s metadata. + + * Added ‘Distribution.clone()’ method, and keyword argument + support to other ‘Distribution’ constructors. + + * Added the ‘DEVELOP_DIST’ precedence, and automatically assign + it to eggs using ‘.egg-info’ format. + +0.6a9 + + * Don’t raise an error when an invalid (unfinished) distribution + is found unless absolutely necessary. Warn about skipping + invalid/unfinished eggs when building an Environment. + + * Added support for ‘.egg-info’ files or directories with + version/platform information embedded in the filename, so that + system packagers have the option of including ‘PKG-INFO’ files + to indicate the presence of a system-installed egg, without + needing to use ‘.egg’ directories, zipfiles, or ‘.pth’ + manipulation. + + * Changed ‘parse_version()’ to remove dashes before pre-release + tags, so that ‘0.2-rc1’ is considered an `older' version than + ‘0.2’, and is equal to ‘0.2rc1’. The idea that a dash + `always' meant a post-release version was highly non-intuitive + to setuptools users and Python developers, who seem to want to + use ‘-rc’ version numbers a lot. + +0.6a8 + + * Fixed a problem with ‘WorkingSet.resolve()’ that prevented + version conflicts from being detected at runtime. + + * Improved runtime conflict warning message to identify a line + in the user’s program, rather than flagging the ‘warn()’ call + in ‘pkg_resources’. + + * Avoid giving runtime conflict warnings for namespace packages, + even if they were declared by a different package than the one + currently being activated. + + * Fix path insertion algorithm for case-insensitive filesystems. + + * Fixed a problem with nested namespace packages (e.g. + ‘peak.util’) not being set as an attribute of their parent + package. + +0.6a6 + + * Activated distributions are now inserted in ‘sys.path’ (and + the working set) just before the directory that contains them, + instead of at the end. This allows e.g. eggs in + ‘site-packages’ to override unmanaged modules in the same + location, and allows eggs found earlier on ‘sys.path’ to + override ones found later. + + * When a distribution is activated, it now checks whether any + contained non-namespace modules have already been imported and + issues a warning if a conflicting module has already been + imported. + + * Changed dependency processing so that it’s breadth-first, + allowing a depender’s preferences to override those of a + dependee, to prevent conflicts when a lower version is + acceptable to the dependee, but not the depender. + + * Fixed a problem extracting zipped files on Windows, when the + egg in question has had changed contents but still has the + same version number. + +0.6a4 + + * Fix a bug in ‘WorkingSet.resolve()’ that was introduced in + 0.6a3. + +0.6a3 + + * Added ‘safe_extra()’ parsing utility routine, and use it for + Requirement, EntryPoint, and Distribution objects’ extras + handling. + +0.6a1 + + * Enhanced performance of ‘require()’ and related operations + when all requirements are already in the working set, and + enhanced performance of directory scanning for distributions. + + * Fixed some problems using ‘pkg_resources’ w/PEP 302 loaders + other than ‘zipimport’, and the previously-broken “eager + resource” support. + + * Fixed ‘pkg_resources.resource_exists()’ not working correctly, + along with some other resource API bugs. + + * Many API changes and enhancements: + + * Added ‘EntryPoint’, ‘get_entry_map’, ‘load_entry_point’, + and ‘get_entry_info’ APIs for dynamic plugin discovery. + + * ‘list_resources’ is now ‘resource_listdir’ (and it + actually works) + + * Resource API functions like ‘resource_string()’ that + accepted a package name and resource name, will now also + accept a ‘Requirement’ object in place of the package + name (to allow access to non-package data files in an + egg). + + * ‘get_provider()’ will now accept a ‘Requirement’ instance + or a module name. If it is given a ‘Requirement’, it + will return a corresponding ‘Distribution’ (by calling + ‘require()’ if a suitable distribution isn’t already in + the working set), rather than returning a metadata and + resource provider for a specific module. (The difference + is in how resource paths are interpreted; supplying a + module name means resources path will be module-relative, + rather than relative to the distribution’s root.) + + * ‘Distribution’ objects now implement the + ‘IResourceProvider’ and ‘IMetadataProvider’ interfaces, + so you don’t need to reference the (no longer available) + ‘metadata’ attribute to get at these interfaces. + + * ‘Distribution’ and ‘Requirement’ both have a + ‘project_name’ attribute for the project name they refer + to. (Previously these were ‘name’ and ‘distname’ + attributes.) + + * The ‘path’ attribute of ‘Distribution’ objects is now + ‘location’, because it isn’t necessarily a filesystem + path (and hasn’t been for some time now). The ‘location’ + of ‘Distribution’ objects in the filesystem should always + be normalized using ‘pkg_resources.normalize_path()’; all + of the setuptools’ code that generates distributions from + the filesystem (including ‘Distribution.from_filename()’) + ensure this invariant, but if you use a more generic API + like ‘Distribution()’ or ‘Distribution.from_location()’ + you should take care that you don’t create a distribution + with an un-normalized filesystem path. + + * ‘Distribution’ objects now have an ‘as_requirement()’ + method that returns a ‘Requirement’ for the + distribution’s project name and version. + + * Distribution objects no longer have an ‘installed_on()’ + method, and the ‘install_on()’ method is now ‘activate()’ + (but may go away altogether soon). The ‘depends()’ + method has also been renamed to ‘requires()’, and + ‘InvalidOption’ is now ‘UnknownExtra’. + + * ‘find_distributions()’ now takes an additional argument + called ‘only’, that tells it to only yield distributions + whose location is the passed-in path. (It defaults to + False, so that the default behavior is unchanged.) + + * ‘AvailableDistributions’ is now called ‘Environment’, and + the ‘get()’, ‘__len__()’, and ‘__contains__()’ methods + were removed, because they weren’t particularly useful. + ‘__getitem__()’ no longer raises ‘KeyError’; it just + returns an empty list if there are no distributions for + the named project. + + * The ‘resolve()’ method of ‘Environment’ is now a method + of ‘WorkingSet’ instead, and the ‘best_match()’ method + now uses a working set instead of a path list as its + second argument. + + * There is a new ‘pkg_resources.add_activation_listener()’ + API that lets you register a callback for notifications + about distributions added to ‘sys.path’ (including the + distributions already on it). This is basically a hook + for extensible applications and frameworks to be able to + search for plugin metadata in distributions added at + runtime. + +0.5a13 + + * Fixed a bug in resource extraction from nested packages in a + zipped egg. + +0.5a12 + + * Updated extraction/cache mechanism for zipped resources to + avoid inter- process and inter-thread races during extraction. + The default cache location can now be set via the + ‘PYTHON_EGGS_CACHE’ environment variable, and the default + Windows cache is now a ‘Python-Eggs’ subdirectory of the + current user’s “Application Data” directory, if the + ‘PYTHON_EGGS_CACHE’ variable isn’t set. + +0.5a10 + + * Fix a problem with ‘pkg_resources’ being confused by + non-existent eggs on ‘sys.path’ (e.g. if a user deletes an + egg without removing it from the ‘easy-install.pth’ file). + + * Fix a problem with “basket” support in ‘pkg_resources’, where + egg-finding never actually went inside ‘.egg’ files. + + * Made ‘pkg_resources’ import the module you request resources + from, if it’s not already imported. + +0.5a4 + + * ‘pkg_resources.AvailableDistributions.resolve()’ and related + methods now accept an ‘installer’ argument: a callable taking + one argument, a ‘Requirement’ instance. The callable must + return a ‘Distribution’ object, or ‘None’ if no distribution + is found. This feature is used by EasyInstall to resolve + dependencies by recursively invoking itself. + +0.4a4 + + * Fix problems with ‘resource_listdir()’, ‘resource_isdir()’ and + resource directory extraction for zipped eggs. + +0.4a3 + + * Fixed scripts not being able to see a ‘__file__’ variable in + ‘__main__’ + + * Fixed a problem with ‘resource_isdir()’ implementation that + was introduced in 0.4a2. + +0.4a1 + + * Fixed a bug in requirements processing for exact versions + (i.e. ‘==’ and ‘!=’) when only one condition was included. + + * Added ‘safe_name()’ and ‘safe_version()’ APIs to clean up + handling of arbitrary distribution names and versions found on + PyPI. + +0.3a4 + + * ‘pkg_resources’ now supports resource directories, not just + the resources in them. In particular, there are + ‘resource_listdir()’ and ‘resource_isdir()’ APIs. + + * ‘pkg_resources’ now supports “egg baskets” – .egg zipfiles + which contain multiple distributions in subdirectories whose + names end with ‘.egg’. Having such a “basket” in a directory + on ‘sys.path’ is equivalent to having the individual eggs in + that directory, but the contained eggs can be individually + added (or not) to ‘sys.path’. Currently, however, there is no + automated way to create baskets. + + * Namespace package manipulation is now protected by the Python + import lock. + +0.3a1 + + * Initial release. + + +File: setuptools.info, Node: Keywords, Next: Roadmap, Prev: Package Discovery and Resource Access using pkg_resources, Up: Top + +4 Keywords +********** + +‘name’ + + A string specifying the name of the package. + +‘version’ + + A string specifying the version number of the package. + +‘description’ + + A string describing the package in a single line. + +‘long_description’ + + A string providing a longer description of the package. + +‘long_description_content_type’ + + A string specifying the content type is used for the + ‘long_description’ (e.g. ‘text/markdown’) + +‘author’ + + A string specifying the author of the package. + +‘author_email’ + + A string specifying the email address of the package author. + +‘maintainer’ + + A string specifying the name of the current maintainer, if + different from the author. Note that if the maintainer is + provided, setuptools will use it as the author in ‘PKG-INFO’. + +‘maintainer_email’ + + A string specifying the email address of the current maintainer, if + different from the author. + +‘url’ + + A string specifying the URL for the package homepage. + +‘download_url’ + + A string specifying the URL to download the package. + +‘packages’ + + A list of strings specifying the packages that setuptools will + manipulate. + +‘py_modules’ + + A list of strings specifying the modules that setuptools will + manipulate. + +‘scripts’ + + A list of strings specifying the standalone script files to be + built and installed. + +‘ext_package’ + + A string specifying the base package name for the extensions + provided by this package. + +‘ext_modules’ + + A list of instances of ‘setuptools.Extension’ providing the list of + Python extensions to be built. + +‘classifiers’ + + A list of strings describing the categories for the package. + +‘distclass’ + + A subclass of ‘Distribution’ to use. + +‘script_name’ + + A string specifying the name of the setup.py script – defaults to + ‘sys.argv[0]’ + +‘script_args’ + + A list of strings defining the arguments to supply to the setup + script. + +‘options’ + + A dictionary providing the default options for the setup script. + +‘license’ + + A string specifying the license of the package. + +‘keywords’ + + A list of strings or a comma-separated string providing descriptive + meta-data. See: PEP 0314(1). + +‘platforms’ + + A list of strings or comma-separated string. + +‘cmdclass’ + + A dictionary providing a mapping of command names to ‘Command’ + subclasses. + +‘data_files’ + + Warning: ‘data_files’ is deprecated. It does not work with + wheels, so it should be avoided. + + A list of strings specifying the data files to install. + +‘package_dir’ + + A dictionary providing a mapping of package to directory names. + +‘requires’ + + Warning: ‘requires’ is superseded by ‘install_requires’ and + should not be used anymore. + +‘obsoletes’ + + Warning: ‘obsoletes’ is currently ignored by ‘pip’. + + List of strings describing packages which this package renders + obsolete, meaning that the two projects should not be installed at + the same time. + + Version declarations can be supplied. Version numbers must be in + the format specified in Version specifiers (e.g. ‘foo (<3.0)’). + + This field may be followed by an environment marker after a + semicolon (e.g. ‘foo; os_name == "posix"’) + + The most common use of this field will be in case a project name + changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0. + When you install Torqued Python, the Gorgon distribution should be + removed. + +‘provides’ + + Warning: ‘provides’ is currently ignored by ‘pip’. + + List of strings describing package- and virtual package names + contained within this package. + + A package may provide additional names, e.g. to indicate that + multiple projects have been bundled together. For instance, source + distributions of the ZODB project have historically included the + transaction project, which is now available as a separate + distribution. Installing such a source distribution satisfies + requirements for both ZODB and transaction. + + A package may also provide a “virtual” project name, which does not + correspond to any separately-distributed project: such a name might + be used to indicate an abstract capability which could be supplied + by one of multiple projects. E.g., multiple projects might supply + RDBMS bindings for use by a given ORM: each project might declare + that it provides ORM-bindings, allowing other projects to depend + only on having at most one of them installed. + + A version declaration may be supplied and must follow the rules + described in Version specifiers. The distribution’s version number + will be implied if none is specified (e.g. ‘foo (<3.0)’). + + Each package may be followed by an environment marker after a + semicolon (e.g. ‘foo; os_name == "posix"’). + +‘include_package_data’ + + If set to ‘True’, this tells ‘setuptools’ to automatically include + any data files it finds inside your package directories that are + specified by your ‘MANIFEST.in’ file. For more information, see + the section on *note Including Data Files: 12. + +‘exclude_package_data’ + + A dictionary mapping package names to lists of glob patterns that + should be `excluded' from your package directories. You can use + this to trim back any excess files included by + ‘include_package_data’. For a complete description and examples, + see the section on *note Including Data Files: 12. + +‘package_data’ + + A dictionary mapping package names to lists of glob patterns. For + a complete description and examples, see the section on *note + Including Data Files: 12. You do not need to use this option if + you are using ‘include_package_data’, unless you need to add e.g. + files that are generated by your setup script and build process. + (And are therefore not in source control or are files that you + don’t want to include in your source distribution.) + +‘zip_safe’ + + A boolean (True or False) flag specifying whether the project can + be safely installed and run from a zip file. If this argument is + not supplied, the ‘bdist_egg’ command will have to analyze all of + your project’s contents for possible problems each time it builds + an egg. + +‘install_requires’ + + A string or list of strings specifying what other distributions + need to be installed when this one is. See the section on *note + Declaring required dependency: 2b. for details and examples of the + format of this argument. + +‘entry_points’ + + A dictionary mapping entry point group names to strings or lists of + strings defining the entry points. Entry points are used to + support dynamic discovery of services or plugins provided by a + project. See *note Advertising Behavior: 25. for details and + examples of the format of this argument. In addition, this keyword + is used to support *note Automatic Script Creation: 21. + +‘extras_require’ + + A dictionary mapping names of “extras” (optional features of your + project) to strings or lists of strings specifying what other + distributions must be installed to support those features. See the + section on *note Declaring required dependency: 2b. for details and + examples of the format of this argument. + +‘python_requires’ + + A string corresponding to a version specifier (as defined in PEP + 440) for the Python version, used to specify the Requires-Python + defined in PEP 345. + +‘setup_requires’ + + Warning: Using ‘setup_requires’ is discouraged in favor of + PEP-518(2) + + A string or list of strings specifying what other distributions + need to be present in order for the `setup script' to run. + ‘setuptools’ will attempt to obtain these (even going so far as to + download them using ‘EasyInstall’) before processing the rest of + the setup script or commands. This argument is needed if you are + using distutils extensions as part of your build process; for + example, extensions that process setup() arguments and turn them + into EGG-INFO metadata files. + + (Note: projects listed in ‘setup_requires’ will NOT be + automatically installed on the system where the setup script is + being run. They are simply downloaded to the ./.eggs directory if + they’re not locally available already. If you want them to be + installed, as well as being available when the setup script is run, + you should add them to ‘install_requires’ `and' ‘setup_requires’.) + +‘dependency_links’ + + Warning: ‘dependency_links’ is deprecated. It is not + supported anymore by pip. + + A list of strings naming URLs to be searched when satisfying + dependencies. These links will be used if needed to install + packages specified by ‘setup_requires’ or ‘tests_require’. They + will also be written into the egg’s metadata for use by tools like + EasyInstall to use when installing an ‘.egg’ file. + +‘namespace_packages’ + + A list of strings naming the project’s “namespace packages”. A + namespace package is a package that may be split across multiple + project distributions. For example, Zope 3’s ‘zope’ package is a + namespace package, because subpackages like ‘zope.interface’ and + ‘zope.publisher’ may be distributed separately. The egg runtime + system can automatically merge such subpackages into a single + parent package at runtime, as long as you declare them in each + project that contains any subpackages of the namespace package, and + as long as the namespace package’s ‘__init__.py’ does not contain + any code other than a namespace declaration. See the section on + *note Using find_namespace; or find_namespace_packages: 1c. for + more information. + +‘test_suite’ + + A string naming a ‘unittest.TestCase’ subclass (or a package or + module containing one or more of them, or a method of such a + subclass), or naming a function that can be called with no + arguments and returns a ‘unittest.TestSuite’. If the named suite + is a module, and the module has an ‘additional_tests()’ function, + it is called and the results are added to the tests to be run. If + the named suite is a package, any submodules and subpackages are + recursively added to the overall test suite. + + Specifying this argument enables use of the *note test - Build + package and run a unittest suite: 56. command to run the specified + test suite, e.g. via ‘setup.py test’. See the section on the + *note test - Build package and run a unittest suite: 56. command + below for more details. + + New in 41.5.0: Deprecated the test command. + +‘tests_require’ + + If your project’s tests need one or more additional packages + besides those needed to install it, you can use this option to + specify them. It should be a string or list of strings specifying + what other distributions need to be present for the package’s tests + to run. When you run the ‘test’ command, ‘setuptools’ will attempt + to obtain these (even going so far as to download them using + ‘EasyInstall’). Note that these required projects will `not' be + installed on the system where the tests are run, but only + downloaded to the project’s setup directory if they’re not already + installed locally. + + New in 41.5.0: Deprecated the test command. + +‘test_loader’ + + If you would like to use a different way of finding tests to run + than what setuptools normally uses, you can specify a module name + and class name in this argument. The named class must be + instantiable with no arguments, and its instances must support the + ‘loadTestsFromNames()’ method as defined in the Python ‘unittest’ + module’s ‘TestLoader’ class. Setuptools will pass only one test + “name” in the ‘names’ argument: the value supplied for the + ‘test_suite’ argument. The loader you specify may interpret this + string in any way it likes, as there are no restrictions on what + may be contained in a ‘test_suite’ string. + + The module name and class name must be separated by a ‘:’. The + default value of this argument is + ‘"setuptools.command.test:ScanningLoader"’. If you want to use the + default ‘unittest’ behavior, you can specify + ‘"unittest:TestLoader"’ as your ‘test_loader’ argument instead. + This will prevent automatic scanning of submodules and subpackages. + + The module and class you specify here may be contained in another + package, as long as you use the ‘tests_require’ option to ensure + that the package containing the loader class is available when the + ‘test’ command is run. + + New in 41.5.0: Deprecated the test command. + +‘eager_resources’ + + A list of strings naming resources that should be extracted + together, if any of them is needed, or if any C extensions included + in the project are imported. This argument is only useful if the + project will be installed as a zipfile, and there is a need to have + all of the listed resources be extracted to the filesystem `as a + unit'. Resources listed here should be ‘/’-separated paths, + relative to the source root, so to list a resource ‘foo.png’ in + package ‘bar.baz’, you would include the string ‘bar/baz/foo.png’ + in this argument. + + If you only need to obtain resources one at a time, or you don’t + have any C extensions that access other files in the project (such + as data files or shared libraries), you probably do NOT need this + argument and shouldn’t mess with it. For more details on how this + argument works, see the section below on *note Automatic Resource + Extraction: 58. + +‘use_2to3’ + + Convert the source code from Python 2 to Python 3 with 2to3 during + the build process. See *note Supporting both Python 2 and Python 3 + with Setuptools: 59. for more details. + +‘convert_2to3_doctests’ + + List of doctest source files that need to be converted with 2to3. + See *note Supporting both Python 2 and Python 3 with Setuptools: + 59. for more details. + +‘use_2to3_fixers’ + + A list of modules to search for additional fixers to be used during + the 2to3 conversion. See *note Supporting both Python 2 and Python + 3 with Setuptools: 59. for more details. + +‘use_2to3_exclude_fixers’ + + List of fixer names to be skipped. + +‘project_urls’ + + An arbitrary map of URL names to hyperlinks, allowing more + extensible documentation of where various resources can be found + than the simple ‘url’ and ‘download_url’ options provide. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0314/ + + (2) http://www.python.org/dev/peps/pep-0518/ + + +File: setuptools.info, Node: Roadmap, Next: Building and Distributing Packages with Setuptools<2>, Prev: Keywords, Up: Top + +5 Roadmap +********* + +Setuptools maintains a series of milestones(1) to track a roadmap of +large-scale goals. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/milestones + + +File: setuptools.info, Node: Building and Distributing Packages with Setuptools<2>, Next: Development on Setuptools, Prev: Roadmap, Up: Top + +6 Building and Distributing Packages with Setuptools +**************************************************** + +‘Setuptools’ is a collection of enhancements to the Python ‘distutils’ +that allow developers to more easily build and distribute Python +packages, especially ones that have dependencies on other packages. + +Packages built and distributed using ‘setuptools’ look to the user like +ordinary Python packages based on the ‘distutils’. + +Feature Highlights: + + * Create Python Eggs(1) - a single-file importable distribution + format + + * Enhanced support for accessing data files hosted in zipped + packages. + + * Automatically include all packages in your source tree, without + listing them individually in setup.py + + * Automatically include all relevant files in your source + distributions, without needing to create a ‘MANIFEST.in’ file, and + without having to force regeneration of the ‘MANIFEST’ file when + your source tree changes. + + * Automatically generate wrapper scripts or Windows (console and GUI) + .exe files for any number of “main” functions in your project. + (Note: this is not a py2exe replacement; the .exe files rely on the + local Python installation.) + + * Transparent Cython support, so that your setup.py can list ‘.pyx’ + files and still work even when the end-user doesn’t have Cython + installed (as long as you include the Cython-generated C in your + source distribution) + + * Command aliases - create project-specific, per-user, or site-wide + shortcut names for commonly used commands and options + + * Deploy your project in “development mode”, such that it’s available + on ‘sys.path’, yet can still be edited directly from its source + checkout. + + * Easily extend the distutils with new commands or ‘setup()’ + arguments, and distribute/reuse your extensions for multiple + projects, without copying code. + + * Create extensible applications and frameworks that automatically + discover extensions, using simple “entry points” declared in a + project’s setup script. + + * Full support for PEP 420 via ‘find_namespace_packages()’, which is + also backwards compatible to the existing ‘find_packages()’ for + Python >= 3.3. + +* Menu: + +* Developer’s Guide:: + + ---------- Footnotes ---------- + + (1) http://peak.telecommunity.com/DevCenter/PythonEggs + + +File: setuptools.info, Node: Developer’s Guide, Up: Building and Distributing Packages with Setuptools<2> + +6.1 Developer’s Guide +===================== + +* Menu: + +* TRANSITIONAL NOTE:: + + +File: setuptools.info, Node: TRANSITIONAL NOTE, Up: Developer’s Guide + +6.1.1 TRANSITIONAL NOTE +----------------------- + +Setuptools automatically calls ‘declare_namespace()’ for you at runtime, +but future versions may `not'. This is because the automatic +declaration feature has some negative side effects, such as needing to +import all namespace packages during the initialization of the +‘pkg_resources’ runtime, and also the need for ‘pkg_resources’ to be +explicitly imported before any namespace packages work at all. In some +future releases, you’ll be responsible for including your own +declaration lines, and the automatic declaration feature will be dropped +to get rid of the negative side effects. + +During the remainder of the current development cycle, therefore, +setuptools will warn you about missing ‘declare_namespace()’ calls in +your ‘__init__.py’ files, and you should correct these as soon as +possible before the compatibility support is removed. Namespace +packages without declaration lines will not work correctly once a user +has upgraded to a later version, so it’s important that you make this +change now in order to avoid having your code break in the field. Our +apologies for the inconvenience, and thank you for your patience. + +* Menu: + +* setup.cfg-only projects: setup cfg-only projects. +* Configuration API:: +* Mailing List and Bug Tracker:: + + +File: setuptools.info, Node: setup cfg-only projects, Next: Configuration API, Up: TRANSITIONAL NOTE + +6.1.1.1 setup.cfg-only projects +............................... + +New in version 40.9.0. + +If ‘setup.py’ is missing from the project directory when a PEP 517(1) +build is invoked, ‘setuptools’ emulates a dummy ‘setup.py’ file +containing only a ‘setuptools.setup()’ call. + + Note: PEP 517(2) doesn’t support editable installs so this is + currently incompatible with ‘pip install -e .’, as PEP 517(3) does + not support editable installs. + +This means that you can have a Python project with all build +configuration specified in ‘setup.cfg’, without a ‘setup.py’ file, if +you `can rely on' your project always being built by a PEP 517(4)/ PEP +518(5) compatible frontend. + +To use this feature: + + * Specify build requirements and PEP 517(6) build backend in + ‘pyproject.toml’. For example: + + [build-system] + requires = [ + "setuptools >= 40.9.0", + "wheel", + ] + build-backend = "setuptools.build_meta" + + * Use a PEP 517(7) compatible build frontend, such as ‘pip >= 19’ or + ‘pep517’. + + Warning: As PEP 517(8) is new, support is not universal, and + frontends that do support it may still have bugs. For + compatibility, you may want to put a ‘setup.py’ file + containing only a ‘setuptools.setup()’ invocation. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0517 + + (2) https://www.python.org/dev/peps/pep-0517 + + (3) https://www.python.org/dev/peps/pep-0517 + + (4) https://www.python.org/dev/peps/pep-0517 + + (5) https://www.python.org/dev/peps/pep-0518 + + (6) https://www.python.org/dev/peps/pep-0517 + + (7) https://www.python.org/dev/peps/pep-0517 + + (8) https://www.python.org/dev/peps/pep-0517 + + +File: setuptools.info, Node: Configuration API, Next: Mailing List and Bug Tracker, Prev: setup cfg-only projects, Up: TRANSITIONAL NOTE + +6.1.1.2 Configuration API +......................... + +Some automation tools may wish to access data from a configuration file. + +‘Setuptools’ exposes a ‘read_configuration()’ function for parsing +‘metadata’ and ‘options’ sections into a dictionary. + + from setuptools.config import read_configuration + + conf_dict = read_configuration("/home/user/dev/package/setup.cfg") + +By default, ‘read_configuration()’ will read only the file provided in +the first argument. To include values from other configuration files +which could be in various places, set the ‘find_others’ keyword argument +to ‘True’. + +If you have only a configuration file but not the whole package, you can +still try to get data out of it with the help of the +‘ignore_option_errors’ keyword argument. When it is set to ‘True’, all +options with errors possibly produced by directives, such as ‘attr:’ and +others, will be silently ignored. As a consequence, the resulting +dictionary will include no such options. + + +File: setuptools.info, Node: Mailing List and Bug Tracker, Prev: Configuration API, Up: TRANSITIONAL NOTE + +6.1.1.3 Mailing List and Bug Tracker +.................................... + +Please use the distutils-sig mailing list(1) for questions and +discussion about setuptools, and the setuptools bug tracker(2) ONLY for +issues you have confirmed via the list are actual bugs, and which you +have reduced to a minimal set of steps to reproduce. + + ---------- Footnotes ---------- + + (1) http://mail.python.org/pipermail/distutils-sig/ + + (2) https://github.com/pypa/setuptools/ + + +File: setuptools.info, Node: Development on Setuptools, Next: Guides on backward compatibility & deprecated practice, Prev: Building and Distributing Packages with Setuptools<2>, Up: Top + +7 Development on Setuptools +*************************** + +Setuptools is maintained by the Python community under the Python +Packaging Authority (PyPA) and led by Jason R. Coombs. + +This document describes the process by which Setuptools is developed. +This document assumes the reader has some passing familiarity with +`using' setuptools, the ‘pkg_resources’ module, and pip. It does not +attempt to explain basic concepts like inter-project dependencies, nor +does it contain detailed lexical syntax for most file formats. Neither +does it explain concepts like “namespace packages” or “resources” in any +detail, as all of these subjects are covered at length in the setuptools +developer’s guide and the ‘pkg_resources’ reference manual. + +Instead, this is `internal' documentation for how those concepts and +features are `implemented' in concrete terms. It is intended for people +who are working on the setuptools code base, who want to be able to +troubleshoot setuptools problems, want to write code that reads the file +formats involved, or want to otherwise tinker with setuptools-generated +files and directories. + +Note, however, that these are all internal implementation details and +are therefore subject to change; stick to the published API if you don’t +want to be responsible for keeping your code from breaking when +setuptools changes. You have been warned. + +* Menu: + +* Developer’s Guide for Setuptools:: +* Release Process:: + + +File: setuptools.info, Node: Developer’s Guide for Setuptools, Next: Release Process, Up: Development on Setuptools + +7.1 Developer’s Guide for Setuptools +==================================== + +If you want to know more about contributing on Setuptools, this is the +place. + +* Menu: + +* Recommended Reading:: +* Project Management:: +* Authoring Tickets:: +* Making a pull request:: +* Auto-Merge Requests:: +* Testing:: +* Semantic Versioning:: +* Building Documentation:: +* Vendored Dependencies:: + + +File: setuptools.info, Node: Recommended Reading, Next: Project Management, Up: Developer’s Guide for Setuptools + +7.1.1 Recommended Reading +------------------------- + +Please read How to write the perfect pull request(1) for some tips on +contributing to open source projects. Although the article is not +authoritative, it was authored by the maintainer of Setuptools, so +reflects his opinions and will improve the likelihood of acceptance and +quality of contribution. + + ---------- Footnotes ---------- + + (1) https://blog.jaraco.com/how-to-write-perfect-pull-request/ + + +File: setuptools.info, Node: Project Management, Next: Authoring Tickets, Prev: Recommended Reading, Up: Developer’s Guide for Setuptools + +7.1.2 Project Management +------------------------ + +Setuptools is maintained primarily in GitHub at this home(1). +Setuptools is maintained under the Python Packaging Authority (PyPA) +with several core contributors. All bugs for Setuptools are filed and +the canonical source is maintained in GitHub. + +User support and discussions are done through the issue tracker (for +specific) issues, through the distutils-sig mailing list(2), or on IRC +(Freenode) at #pypa. + +Discussions about development happen on the distutils-sig mailing list +or on Gitter(3). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools + + (2) https://mail.python.org/mailman3/lists/distutils-sig.python.org/ + + (3) https://gitter.im/pypa/setuptools + + +File: setuptools.info, Node: Authoring Tickets, Next: Making a pull request, Prev: Project Management, Up: Developer’s Guide for Setuptools + +7.1.3 Authoring Tickets +----------------------- + +Before authoring any source code, it’s often prudent to file a ticket +describing the motivation behind making changes. First search to see if +a ticket already exists for your issue. If not, create one. Try to +think from the perspective of the reader. Explain what behavior you +expected, what you got instead, and what factors might have contributed +to the unexpected behavior. In GitHub, surround a block of code or +traceback with the triple backtick “‘‘‘” so that it is formatted nicely. + +Filing a ticket provides a forum for justification, discussion, and +clarification. The ticket provides a record of the purpose for the +change and any hard decisions that were made. It provides a single +place for others to reference when trying to understand why the software +operates the way it does or why certain changes were made. + +Setuptools makes extensive use of hyperlinks to tickets in the changelog +so that system integrators and other users can get a quick summary, but +then jump to the in-depth discussion about any subject referenced. + + +File: setuptools.info, Node: Making a pull request, Next: Auto-Merge Requests, Prev: Authoring Tickets, Up: Developer’s Guide for Setuptools + +7.1.4 Making a pull request +--------------------------- + +When making a pull request, please *note include a short summary of the +changes: b1. and a reference to any issue tickets that the PR is +intended to solve. All PRs with code changes should include tests. All +changes should include a changelog entry. + +* Menu: + +* Adding change notes with your PRs:: +* Alright! So how to add a news fragment?:: +* Examples for adding changelog entries to your Pull Requests:: + + +File: setuptools.info, Node: Adding change notes with your PRs, Next: Alright! So how to add a news fragment?, Up: Making a pull request + +7.1.4.1 Adding change notes with your PRs +......................................... + +It is very important to maintain a log for news of how updating to the +new version of the software will affect end-users. This is why we +enforce collection of the change fragment files in pull requests as per +Towncrier philosophy(1). + +The idea is that when somebody makes a change, they must record the bits +that would affect end-users only including information that would be +useful to them. Then, when the maintainers publish a new release, +they’ll automatically use these records to compose a change log for the +respective version. It is important to understand that including +unnecessary low-level implementation related details generates noise +that is not particularly useful to the end-users most of the time. And +so such details should be recorded in the Git history rather than a +changelog. + + ---------- Footnotes ---------- + + (1) +https://towncrier.readthedocs.io/en/actual-freaking-docs/#philosophy + + +File: setuptools.info, Node: Alright! So how to add a news fragment?, Next: Examples for adding changelog entries to your Pull Requests, Prev: Adding change notes with your PRs, Up: Making a pull request + +7.1.4.2 Alright! So how to add a news fragment? +............................................... + +‘setuptools’ uses towncrier(1) for changelog management. To submit a +change note about your PR, add a text file into the ‘changelog.d/’ +folder. It should contain an explanation of what applying this PR will +change in the way end-users interact with the project. One sentence is +usually enough but feel free to add as many details as you feel +necessary for the users to understand what it means. + +`Use the past tense' for the text in your fragment because, combined +with others, it will be a part of the “news digest” telling the readers +`what changed' in a specific version of the library `since the previous +version'. You should also use reStructuredText syntax for highlighting +code (inline or block), linking parts of the docs or external sites. If +you wish to sign your change, feel free to add ‘-- by +:user:`github-username`’ at the end (replace ‘github-username’ with your +own!). + +Finally, name your file following the convention that Towncrier +understands: it should start with the number of an issue or a PR +followed by a dot, then add a patch type, like ‘change’, ‘doc’, ‘misc’ +etc., and add ‘.rst’ as a suffix. If you need to add more than one +fragment, you may add an optional sequence number (delimited with +another period) between the type and the suffix. + +In general the name will follow ‘..rst’ pattern, +where the categories are: + + - ‘change’: Any backwards compatible code change + + - ‘breaking’: Any backwards-compatibility breaking change + + - ‘doc’: A change to the documentation + + - ‘misc’: Changes internal to the repo like CI, test and build + changes + + - ‘deprecation’: For deprecations of an existing feature or behavior + +A pull request may have more than one of these components, for example a +code change may introduce a new feature that deprecates an old feature, +in which case two fragments should be added. It is not necessary to +make a separate documentation fragment for documentation changes +accompanying the relevant code changes. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/towncrier/ + + +File: setuptools.info, Node: Examples for adding changelog entries to your Pull Requests, Prev: Alright! So how to add a news fragment?, Up: Making a pull request + +7.1.4.3 Examples for adding changelog entries to your Pull Requests +................................................................... + +File ‘changelog.d/2395.doc.1.rst’: + + Added a ``:user:`` role to Sphinx config -- by :user:`webknjaz` + +File ‘changelog.d/1354.misc.rst’: + + Added ``towncrier`` for changelog managment -- by :user:`pganssle` + +File ‘changelog.d/2355.change.rst’: + + When pip is imported as part of a build, leave :py:mod:`distutils` + patched -- by :user:`jaraco` + + Tip: See ‘pyproject.toml’ for all available categories + (‘tool.towncrier.type’). + + +File: setuptools.info, Node: Auto-Merge Requests, Next: Testing, Prev: Making a pull request, Up: Developer’s Guide for Setuptools + +7.1.5 Auto-Merge Requests +------------------------- + +To support running all code through CI, even lightweight contributions, +the project employs Mergify to auto-merge pull requests tagged as +auto-merge. + +Use ‘hub pull-request -l auto-merge’ to create such a pull request from +the command line after pushing a new branch. + + +File: setuptools.info, Node: Testing, Next: Semantic Versioning, Prev: Auto-Merge Requests, Up: Developer’s Guide for Setuptools + +7.1.6 Testing +------------- + +The primary tests are run using tox. Make sure you have tox installed, +and invoke it: + + $ tox + +Under continuous integration, additional tests may be run. See the +‘.travis.yml’ file for full details on the tests run under Travis-CI. + + +File: setuptools.info, Node: Semantic Versioning, Next: Building Documentation, Prev: Testing, Up: Developer’s Guide for Setuptools + +7.1.7 Semantic Versioning +------------------------- + +Setuptools follows ‘semver’. + + +File: setuptools.info, Node: Building Documentation, Next: Vendored Dependencies, Prev: Semantic Versioning, Up: Developer’s Guide for Setuptools + +7.1.8 Building Documentation +---------------------------- + +Setuptools relies on the Sphinx(1) system for building documentation. +The published documentation(2) is hosted on Read the Docs. + +To build the docs locally, use tox: + + $ tox -e docs + + ---------- Footnotes ---------- + + (1) http://www.sphinx-doc.org/en/master/ + + (2) https://setuptools.readthedocs.io/en/latest/ + + +File: setuptools.info, Node: Vendored Dependencies, Prev: Building Documentation, Up: Developer’s Guide for Setuptools + +7.1.9 Vendored Dependencies +--------------------------- + +Setuptools has some dependencies, but due to bootstrapping issues(1), +those dependencies cannot be declared as they won’t be resolved soon +enough to build setuptools from source. Eventually, this limitation may +be lifted as PEP 517/518 reach ubiquitous adoption, but for now, +Setuptools cannot declare dependencies other than through +‘setuptools/_vendor/vendored.txt’ and +‘pkg_resources/_vendor/vendored.txt’ and refreshed by way of ‘paver +update_vendored’ (pavement.py). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/980 + + +File: setuptools.info, Node: Release Process, Prev: Developer’s Guide for Setuptools, Up: Development on Setuptools + +7.2 Release Process +=================== + +In order to allow for rapid, predictable releases, Setuptools uses a +mechanical technique for releases, enacted on tagged commits by +continuous integration. + +To finalize a release, run ‘tox -e finalize’, review, then push the +changes. + +If tests pass, the release will be uploaded to PyPI. + +* Menu: + +* Release Frequency:: +* Release Managers:: + + +File: setuptools.info, Node: Release Frequency, Next: Release Managers, Up: Release Process + +7.2.1 Release Frequency +----------------------- + +Some have asked why Setuptools is released so frequently. Because +Setuptools uses a mechanical release process, it’s very easy to make +releases whenever the code is stable (tests are passing). As a result, +the philosophy is to release early and often. + +While some find the frequent releases somewhat surprising, they only +empower the user. Although releases are made frequently, users can +choose the frequency at which they use those releases. If instead +Setuptools contributions were only released in batches, the user would +be constrained to only use Setuptools when those official releases were +made. With frequent releases, the user can govern exactly how often he +wishes to update. + +Frequent releases also then obviate the need for dev or beta releases in +most cases. Because releases are made early and often, bugs are +discovered and corrected quickly, in many cases before other users have +yet to encounter them. + + +File: setuptools.info, Node: Release Managers, Prev: Release Frequency, Up: Release Process + +7.2.2 Release Managers +---------------------- + +Additionally, anyone with push access to the master branch has access to +cut releases. + + +File: setuptools.info, Node: Guides on backward compatibility & deprecated practice, Next: History<2>, Prev: Development on Setuptools, Up: Top + +8 Guides on backward compatibility & deprecated practice +******************************************************** + +‘Setuptools’ has undergone tremendous changes since its first debut. As +its development continues to roll forward, many of the practice and +mechanisms it had established are now considered deprecated. But they +still remain relevant as a plethora of libraries continue to depend on +them. Many people also find it necessary to equip themselves with the +knowledge to better support backward compatibility. This guide aims to +provide the essential information for such objectives. + +* Menu: + +* Supporting both Python 2 and Python 3 with Setuptools:: +* The Internal Structure of Python Eggs:: +* Easy Install:: +* Porting from Distutils:: +* “Eggsecutable” Scripts:: + + +File: setuptools.info, Node: Supporting both Python 2 and Python 3 with Setuptools, Next: The Internal Structure of Python Eggs, Up: Guides on backward compatibility & deprecated practice + +8.1 Supporting both Python 2 and Python 3 with Setuptools +========================================================= + +Starting with Distribute version 0.6.2 and Setuptools 0.7, the +Setuptools project supported Python 3. Installing and using setuptools +for Python 3 code works exactly the same as for Python 2 code. + +Setuptools provides a facility to invoke 2to3 on the code as a part of +the build process, by setting the keyword parameter ‘use_2to3’ to True, +but the Setuptools project strongly recommends instead developing a +unified codebase using six(1), future(2), or another compatibility +library. + +* Menu: + +* Using 2to3:: +* Distributing Python 3 modules:: +* Advanced features:: + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/six/ + + (2) https://pypi.org/project/future/ + + +File: setuptools.info, Node: Using 2to3, Next: Distributing Python 3 modules, Up: Supporting both Python 2 and Python 3 with Setuptools + +8.1.1 Using 2to3 +---------------- + +Setuptools attempts to make the porting process easier by automatically +running 2to3 as a part of running tests. To do so, you need to +configure the setup.py so that you can run the unit tests with ‘python +setup.py test’. + +See *note test - Build package and run a unittest suite: 56. for more +information on this. + +Once you have the tests running under Python 2, you can add the use_2to3 +keyword parameters to setup(), and start running the tests under Python +3. The test command will now first run the build command during which +the code will be converted with 2to3, and the tests will then be run +from the build directory, as opposed from the source directory as is +normally done. + +Setuptools will convert all Python files, and also all doctests in +Python files. However, if you have doctests located in separate text +files, these will not automatically be converted. By adding them to the +‘convert_2to3_doctests’ keyword parameter Setuptools will convert them +as well. + +By default, the conversion uses all fixers in the ‘lib2to3.fixers’ +package. To use additional fixers, the parameter ‘use_2to3_fixers’ can +be set to a list of names of packages containing fixers. To exclude +fixers, the parameter ‘use_2to3_exclude_fixers’ can be set to fixer +names to be skipped. + +An example setup.py might look something like this: + + from setuptools import setup + + setup( + name='your.module', + version='1.0', + description='This is your awesome module', + author='You', + author_email='your@email', + package_dir={'': 'src'}, + packages=['your', 'you.module'], + test_suite='your.module.tests', + use_2to3=True, + convert_2to3_doctests=['src/your/module/README.txt'], + use_2to3_fixers=['your.fixers'], + use_2to3_exclude_fixers=['lib2to3.fixes.fix_import'], + ) + +* Menu: + +* Differential conversion:: + + +File: setuptools.info, Node: Differential conversion, Up: Using 2to3 + +8.1.1.1 Differential conversion +............................... + +Note that a file will only be copied and converted during the build +process if the source file has been changed. If you add a file to the +doctests that should be converted, it will not be converted the next +time you run the tests, since it hasn’t been modified. You need to +remove it from the build directory. Also if you run the build, install +or test commands before adding the use_2to3 parameter, you will have to +remove the build directory before you run the test command, as the files +otherwise will seem updated, and no conversion will happen. + +In general, if code doesn’t seem to be converted, deleting the build +directory and trying again is a good safeguard against the build +directory getting “out of sync” with the source directory. + + +File: setuptools.info, Node: Distributing Python 3 modules, Next: Advanced features, Prev: Using 2to3, Up: Supporting both Python 2 and Python 3 with Setuptools + +8.1.2 Distributing Python 3 modules +----------------------------------- + +You can distribute your modules with Python 3 support in different ways. +A normal source distribution will work, but can be slow in installing, +as the 2to3 process will be run during the install. But you can also +distribute the module in binary format, such as a binary egg. That egg +will contain the already converted code, and hence no 2to3 conversion is +needed during install. + + +File: setuptools.info, Node: Advanced features, Prev: Distributing Python 3 modules, Up: Supporting both Python 2 and Python 3 with Setuptools + +8.1.3 Advanced features +----------------------- + +If you don’t want to run the 2to3 conversion on the doctests in Python +files, you can turn that off by setting ‘setuptools.use_2to3_on_doctests += False’. + + +File: setuptools.info, Node: The Internal Structure of Python Eggs, Next: Easy Install, Prev: Supporting both Python 2 and Python 3 with Setuptools, Up: Guides on backward compatibility & deprecated practice + +8.2 The Internal Structure of Python Eggs +========================================= + +STOP! This is not the first document you should read! + +* Menu: + +* Eggs and their Formats:: +* Standard Metadata:: +* Other Technical Considerations:: + + +File: setuptools.info, Node: Eggs and their Formats, Next: Standard Metadata, Up: The Internal Structure of Python Eggs + +8.2.1 Eggs and their Formats +---------------------------- + +A “Python egg” is a logical structure embodying the release of a +specific version of a Python project, comprising its code, resources, +and metadata. There are multiple formats that can be used to physically +encode a Python egg, and others can be developed. However, a key +principle of Python eggs is that they should be discoverable and +importable. That is, it should be possible for a Python application to +easily and efficiently find out what eggs are present on a system, and +to ensure that the desired eggs’ contents are importable. + +There are two basic formats currently implemented for Python eggs: + + 1. ‘.egg’ format: a directory or zipfile `containing' the project’s + code and resources, along with an ‘EGG-INFO’ subdirectory that + contains the project’s metadata + + 2. ‘.egg-info’ format: a file or directory placed `adjacent' to the + project’s code and resources, that directly contains the project’s + metadata. + +Both formats can include arbitrary Python code and resources, including +static data files, package and non-package directories, Python modules, +C extension modules, and so on. But each format is optimized for +different purposes. + +The ‘.egg’ format is well-suited to distribution and the easy +uninstallation or upgrades of code, since the project is essentially +self-contained within a single directory or file, unmingled with any +other projects’ code or resources. It also makes it possible to have +multiple versions of a project simultaneously installed, such that +individual programs can select the versions they wish to use. + +The ‘.egg-info’ format, on the other hand, was created to support +backward-compatibility, performance, and ease of installation for system +packaging tools that expect to install all projects’ code and resources +to a single directory (e.g. ‘site-packages’). Placing the metadata in +that same directory simplifies the installation process, since it isn’t +necessary to create ‘.pth’ files or otherwise modify ‘sys.path’ to +include each installed egg. + +Its disadvantage, however, is that it provides no support for clean +uninstallation or upgrades, and of course only a single version of a +project can be installed to a given directory. Thus, support from a +package management tool is required. (This is why setuptools’ “install” +command refers to this type of egg installation as “single-version, +externally managed”.) Also, they lack sufficient data to allow them to +be copied from their installation source. easy_install can “ship” an +application by copying ‘.egg’ files or directories to a target location, +but it cannot do this for ‘.egg-info’ installs, because there is no way +to tell what code and resources belong to a particular egg – there may +be several eggs “scrambled” together in a single installation location, +and the ‘.egg-info’ format does not currently include a way to list the +files that were installed. (This may change in a future version.) + +* Menu: + +* Code and Resources:: +* Project Metadata:: +* Filename-Embedded Metadata:: +* Egg Links:: + + +File: setuptools.info, Node: Code and Resources, Next: Project Metadata, Up: Eggs and their Formats + +8.2.1.1 Code and Resources +.......................... + +The layout of the code and resources is dictated by Python’s normal +import layout, relative to the egg’s “base location”. + +For the ‘.egg’ format, the base location is the ‘.egg’ itself. That is, +adding the ‘.egg’ filename or directory name to ‘sys.path’ makes its +contents importable. + +For the ‘.egg-info’ format, however, the base location is the directory +that `contains' the ‘.egg-info’, and thus it is the directory that must +be added to ‘sys.path’ to make the egg importable. (Note that this +means that the “normal” installation of a package to a ‘sys.path’ +directory is sufficient to make it an “egg” if it has an ‘.egg-info’ +file or directory installed alongside of it.) + + +File: setuptools.info, Node: Project Metadata, Next: Filename-Embedded Metadata, Prev: Code and Resources, Up: Eggs and their Formats + +8.2.1.2 Project Metadata +........................ + +If eggs contained only code and resources, there would of course be no +difference between them and any other directory or zip file on +‘sys.path’. Thus, metadata must also be included, using a metadata file +or directory. + +For the ‘.egg’ format, the metadata is placed in an ‘EGG-INFO’ +subdirectory, directly within the ‘.egg’ file or directory. For the +‘.egg-info’ format, metadata is stored directly within the ‘.egg-info’ +directory itself. + +The minimum project metadata that all eggs must have is a standard +Python ‘PKG-INFO’ file, named ‘PKG-INFO’ and placed within the metadata +directory appropriate to the format. Because it’s possible for this to +be the only metadata file included, ‘.egg-info’ format eggs are not +required to be a directory; they can just be a ‘.egg-info’ file that +directly contains the ‘PKG-INFO’ metadata. This eliminates the need to +create a directory just to store one file. This option is `not' +available for ‘.egg’ formats, since setuptools always includes other +metadata. (In fact, setuptools itself never generates ‘.egg-info’ +files, either; the support for using files was added so that the +requirement could easily be satisfied by other tools, such as +distutils). + +In addition to the ‘PKG-INFO’ file, an egg’s metadata directory may also +include files and directories representing various forms of optional +standard metadata (see the section on *note Standard Metadata: cb, +below) or user-defined metadata required by the project. For example, +some projects may define a metadata format to describe their application +plugins, and metadata in this format would then be included by plugin +creators in their projects’ metadata directories. + + +File: setuptools.info, Node: Filename-Embedded Metadata, Next: Egg Links, Prev: Project Metadata, Up: Eggs and their Formats + +8.2.1.3 Filename-Embedded Metadata +.................................. + +To allow introspection of installed projects and runtime resolution of +inter-project dependencies, a certain amount of information is embedded +in egg filenames. At a minimum, this includes the project name, and +ideally will also include the project version number. Optionally, it +can also include the target Python version and required runtime platform +if platform-specific C code is included. The syntax of an egg filename +is as follows: + + name ["-" version ["-py" pyver ["-" required_platform]]] "." ext + +The “name” and “version” should be escaped using the ‘to_filename()’ +function provided by ‘pkg_resources’, after first processing them with +‘safe_name()’ and ‘safe_version()’ respectively. These latter two +functions can also be used to later “unescape” these parts of the +filename. (For a detailed description of these transformations, please +see the “Parsing Utilities” section of the ‘pkg_resources’ manual.) + +The “pyver” string is the Python major version, as found in the first 3 +characters of ‘sys.version’. “required_platform” is essentially a +distutils ‘get_platform()’ string, but with enhancements to properly +distinguish Mac OS versions. (See the ‘get_build_platform()’ +documentation in the “Platform Utilities” section of the ‘pkg_resources’ +manual for more details.) + +Finally, the “ext” is either ‘.egg’ or ‘.egg-info’, as appropriate for +the egg’s format. + +Normally, an egg’s filename should include at least the project name and +version, as this allows the runtime system to find desired project +versions without having to read the egg’s PKG-INFO to determine its +version number. + +Setuptools, however, only includes the version number in the filename +when an ‘.egg’ file is built using the ‘bdist_egg’ command, or when an +‘.egg-info’ directory is being installed by the ‘install_egg_info’ +command. When generating metadata for use with the original source +tree, it only includes the project name, so that the directory will not +have to be renamed each time the project’s version changes. + +This is especially important when version numbers change frequently, and +the source metadata directory is kept under version control with the +rest of the project. (As would be the case when the project’s source +includes project-defined metadata that is not generated from by +setuptools from data in the setup script.) + + +File: setuptools.info, Node: Egg Links, Prev: Filename-Embedded Metadata, Up: Eggs and their Formats + +8.2.1.4 Egg Links +................. + +In addition to the ‘.egg’ and ‘.egg-info’ formats, there is a third +egg-related extension that you may encounter on occasion: ‘.egg-link’ +files. + +These files are not eggs, strictly speaking. They simply provide a way +to reference an egg that is not physically installed in the desired +location. They exist primarily as a cross-platform alternative to +symbolic links, to support “installing” code that is being developed in +a different location than the desired installation location. For +example, if a user is developing an application plugin in their home +directory, but the plugin needs to be “installed” in an application +plugin directory, running “setup.py develop -md /path/to/app/plugins” +will install an ‘.egg-link’ file in ‘/path/to/app/plugins’, that tells +the egg runtime system where to find the actual egg (the user’s project +source directory and its ‘.egg-info’ subdirectory). + +‘.egg-link’ files are named following the format for ‘.egg’ and +‘.egg-info’ names, but only the project name is included; no version, +Python version, or platform information is included. When the runtime +searches for available eggs, ‘.egg-link’ files are opened and the actual +egg file/directory name is read from them. + +Each ‘.egg-link’ file should contain a single file or directory name, +with no newlines. This filename should be the base location of one or +more eggs. That is, the name must either end in ‘.egg’, or else it +should be the parent directory of one or more ‘.egg-info’ format eggs. + +As of setuptools 0.6c6, the path may be specified as a +platform-independent (i.e. ‘/’-separated) relative path from the +directory containing the ‘.egg-link’ file, and a second line may appear +in the file, specifying a platform-independent relative path from the +egg’s base directory to its setup script directory. This allows +installation tools such as EasyInstall to find the project’s setup +directory and build eggs or perform other setup commands on it. + + +File: setuptools.info, Node: Standard Metadata, Next: Other Technical Considerations, Prev: Eggs and their Formats, Up: The Internal Structure of Python Eggs + +8.2.2 Standard Metadata +----------------------- + +In addition to the minimum required ‘PKG-INFO’ metadata, projects can +include a variety of standard metadata files or directories, as +described below. Except as otherwise noted, these files and directories +are automatically generated by setuptools, based on information supplied +in the setup script or through analysis of the project’s code and +resources. + +Most of these files and directories are generated via “egg-info writers” +during execution of the setuptools ‘egg_info’ command, and are listed in +the ‘egg_info.writers’ entry point group defined by setuptools’ own +‘setup.py’ file. + +Project authors can register their own metadata writers as entry points +in this group (as described in the setuptools manual under “Adding new +EGG-INFO Files”) to cause setuptools to generate project-specific +metadata files or directories during execution of the ‘egg_info’ +command. It is up to project authors to document these new metadata +formats, if they create any. + +* Menu: + +* .txt File Formats: txt File Formats. +* Dependency Metadata:: +* namespace_packages.txt – Namespace Package Metadata: namespace_packages txt – Namespace Package Metadata. +* entry_points.txt – “Entry Point”/Plugin Metadata: entry_points txt – “Entry Point”/Plugin Metadata. +* The scripts Subdirectory:: +* Zip Support Metadata:: +* top_level.txt – Conflict Management Metadata: top_level txt – Conflict Management Metadata. +* SOURCES.txt – Source Files Manifest: SOURCES txt – Source Files Manifest. + + +File: setuptools.info, Node: txt File Formats, Next: Dependency Metadata, Up: Standard Metadata + +8.2.2.1 ‘.txt’ File Formats +........................... + +Files described in this section that have ‘.txt’ extensions have a +simple lexical format consisting of a sequence of text lines, each line +terminated by a linefeed character (regardless of platform). Leading +and trailing whitespace on each line is ignored, as are blank lines and +lines whose first nonblank character is a ‘#’ (comment symbol). (This +is the parsing format defined by the ‘yield_lines()’ function of the +‘pkg_resources’ module.) + +All ‘.txt’ files defined by this section follow this format, but some +are also “sectioned” files, meaning that their contents are divided into +sections, using square-bracketed section headers akin to Windows ‘.ini’ +format. Note that this does `not' imply that the lines within the +sections follow an ‘.ini’ format, however. Please see an individual +metadata file’s documentation for a description of what the lines and +section names mean in that particular file. + +Sectioned files can be parsed using the ‘split_sections()’ function; see +the “Parsing Utilities” section of the ‘pkg_resources’ manual for for +details. + + +File: setuptools.info, Node: Dependency Metadata, Next: namespace_packages txt – Namespace Package Metadata, Prev: txt File Formats, Up: Standard Metadata + +8.2.2.2 Dependency Metadata +........................... + +* Menu: + +* requires.txt: requires txt. +* setup_requires.txt: setup_requires txt. +* dependency_links.txt: dependency_links txt. +* depends.txt – Obsolete, do not create!: depends txt – Obsolete do not create!. + + +File: setuptools.info, Node: requires txt, Next: setup_requires txt, Up: Dependency Metadata + +8.2.2.3 ‘requires.txt’ +...................... + +This is a “sectioned” text file. Each section is a sequence of +“requirements”, as parsed by the ‘parse_requirements()’ function; please +see the ‘pkg_resources’ manual for the complete requirement parsing +syntax. + +The first, unnamed section (i.e., before the first section header) in +this file is the project’s core requirements, which must be installed +for the project to function. (Specified using the ‘install_requires’ +keyword to ‘setup()’). + +The remaining (named) sections describe the project’s “extra” +requirements, as specified using the ‘extras_require’ keyword to +‘setup()’. The section name is the name of the optional feature, and +the section body lists that feature’s dependencies. + +Note that it is not normally necessary to inspect this file directly; +‘pkg_resources.Distribution’ objects have a ‘requires()’ method that can +be used to obtain ‘Requirement’ objects describing the project’s core +and optional dependencies. + + +File: setuptools.info, Node: setup_requires txt, Next: dependency_links txt, Prev: requires txt, Up: Dependency Metadata + +8.2.2.4 ‘setup_requires.txt’ +............................ + +Much like ‘requires.txt’ except represents the requirements specified by +the ‘setup_requires’ parameter to the Distribution. + + +File: setuptools.info, Node: dependency_links txt, Next: depends txt – Obsolete do not create!, Prev: setup_requires txt, Up: Dependency Metadata + +8.2.2.5 ‘dependency_links.txt’ +.............................. + +A list of dependency URLs, one per line, as specified using the +‘dependency_links’ keyword to ‘setup()’. These may be direct download +URLs, or the URLs of web pages containing direct download links. Please +see the setuptools manual for more information on specifying this +option. + + +File: setuptools.info, Node: depends txt – Obsolete do not create!, Prev: dependency_links txt, Up: Dependency Metadata + +8.2.2.6 ‘depends.txt’ – Obsolete, do not create! +................................................ + +This file follows an identical format to ‘requires.txt’, but is obsolete +and should not be used. The earliest versions of setuptools required +users to manually create and maintain this file, so the runtime still +supports reading it, if it exists. The new filename was created so that +it could be automatically generated from ‘setup()’ information without +overwriting an existing hand-created ‘depends.txt’, if one was already +present in the project’s source ‘.egg-info’ directory. + + +File: setuptools.info, Node: namespace_packages txt – Namespace Package Metadata, Next: entry_points txt – “Entry Point”/Plugin Metadata, Prev: Dependency Metadata, Up: Standard Metadata + +8.2.2.7 ‘namespace_packages.txt’ – Namespace Package Metadata +............................................................. + +A list of namespace package names, one per line, as supplied to the +‘namespace_packages’ keyword to ‘setup()’. Please see the manuals for +setuptools and ‘pkg_resources’ for more information about namespace +packages. + + +File: setuptools.info, Node: entry_points txt – “Entry Point”/Plugin Metadata, Next: The scripts Subdirectory, Prev: namespace_packages txt – Namespace Package Metadata, Up: Standard Metadata + +8.2.2.8 ‘entry_points.txt’ – “Entry Point”/Plugin Metadata +.......................................................... + +This is a “sectioned” text file, whose contents encode the +‘entry_points’ keyword supplied to ‘setup()’. All sections are named, +as the section names specify the entry point groups in which the +corresponding section’s entry points are registered. + +Each section is a sequence of “entry point” lines, each parseable using +the ‘EntryPoint.parse’ classmethod; please see the ‘pkg_resources’ +manual for the complete entry point parsing syntax. + +Note that it is not necessary to parse this file directly; the +‘pkg_resources’ module provides a variety of APIs to locate and load +entry points automatically. Please see the setuptools and +‘pkg_resources’ manuals for details on the nature and uses of entry +points. + + +File: setuptools.info, Node: The scripts Subdirectory, Next: Zip Support Metadata, Prev: entry_points txt – “Entry Point”/Plugin Metadata, Up: Standard Metadata + +8.2.2.9 The ‘scripts’ Subdirectory +.................................. + +This directory is currently only created for ‘.egg’ files built by the +setuptools ‘bdist_egg’ command. It will contain copies of all of the +project’s “traditional” scripts (i.e., those specified using the +‘scripts’ keyword to ‘setup()’). This is so that they can be +reconstituted when an ‘.egg’ file is installed. + +The scripts are placed here using the distutils’ standard +‘install_scripts’ command, so any ‘#!’ lines reflect the Python +installation where the egg was built. But instead of copying the +scripts to the local script installation directory, EasyInstall writes +short wrapper scripts that invoke the original scripts from inside the +egg, after ensuring that sys.path includes the egg and any eggs it +depends on. For more about *note script wrappers: d7, see the section +below on *note Installation and Path Management Issues: d8. + + +File: setuptools.info, Node: Zip Support Metadata, Next: top_level txt – Conflict Management Metadata, Prev: The scripts Subdirectory, Up: Standard Metadata + +8.2.2.10 Zip Support Metadata +............................. + +* Menu: + +* native_libs.txt: native_libs txt. +* eager_resources.txt: eager_resources txt. +* zip-safe and not-zip-safe:: + + +File: setuptools.info, Node: native_libs txt, Next: eager_resources txt, Up: Zip Support Metadata + +8.2.2.11 ‘native_libs.txt’ +.......................... + +A list of C extensions and other dynamic link libraries contained in the +egg, one per line. Paths are ‘/’-separated and relative to the egg’s +base location. + +This file is generated as part of ‘bdist_egg’ processing, and as such +only appears in ‘.egg’ files (and ‘.egg’ directories created by +unpacking them). It is used to ensure that all libraries are extracted +from a zipped egg at the same time, in case there is any direct linkage +between them. Please see the *note Zip File Issues: db. section below +for more information on library and resource extraction from ‘.egg’ +files. + + +File: setuptools.info, Node: eager_resources txt, Next: zip-safe and not-zip-safe, Prev: native_libs txt, Up: Zip Support Metadata + +8.2.2.12 ‘eager_resources.txt’ +.............................. + +A list of resource files and/or directories, one per line, as specified +via the ‘eager_resources’ keyword to ‘setup()’. Paths are ‘/’-separated +and relative to the egg’s base location. + +Resource files or directories listed here will be extracted +simultaneously, if any of the named resources are extracted, or if any +native libraries listed in ‘native_libs.txt’ are extracted. Please see +the setuptools manual for details on what this feature is used for and +how it works, as well as the *note Zip File Issues: db. section below. + + +File: setuptools.info, Node: zip-safe and not-zip-safe, Prev: eager_resources txt, Up: Zip Support Metadata + +8.2.2.13 ‘zip-safe’ and ‘not-zip-safe’ +...................................... + +These are zero-length files, and either one or the other should exist. +If ‘zip-safe’ exists, it means that the project will work properly when +installed as an ‘.egg’ zipfile, and conversely the existence of +‘not-zip-safe’ means the project should not be installed as an ‘.egg’ +file. The ‘zip_safe’ option to setuptools’ ‘setup()’ determines which +file will be written. If the option isn’t provided, setuptools attempts +to make its own assessment of whether the package can work, based on +code and content analysis. + +If neither file is present at installation time, EasyInstall defaults to +assuming that the project should be unzipped. (Command-line options to +EasyInstall, however, take precedence even over an existing ‘zip-safe’ +or ‘not-zip-safe’ file.) + +Note that these flag files appear only in ‘.egg’ files generated by +‘bdist_egg’, and in ‘.egg’ directories created by unpacking such an +‘.egg’ file. + + +File: setuptools.info, Node: top_level txt – Conflict Management Metadata, Next: SOURCES txt – Source Files Manifest, Prev: Zip Support Metadata, Up: Standard Metadata + +8.2.2.14 ‘top_level.txt’ – Conflict Management Metadata +....................................................... + +This file is a list of the top-level module or package names provided by +the project, one Python identifier per line. + +Subpackages are not included; a project containing both a ‘foo.bar’ and +a ‘foo.baz’ would include only one line, ‘foo’, in its ‘top_level.txt’. + +This data is used by ‘pkg_resources’ at runtime to issue a warning if an +egg is added to ‘sys.path’ when its contained packages may have already +been imported. + +(It was also once used to detect conflicts with non-egg packages at +installation time, but in more recent versions, setuptools installs eggs +in such a way that they always override non-egg packages, thus +preventing a problem from arising.) + + +File: setuptools.info, Node: SOURCES txt – Source Files Manifest, Prev: top_level txt – Conflict Management Metadata, Up: Standard Metadata + +8.2.2.15 ‘SOURCES.txt’ – Source Files Manifest +.............................................. + +This file is roughly equivalent to the distutils’ ‘MANIFEST’ file. The +differences are as follows: + + * The filenames always use ‘/’ as a path separator, which must be + converted back to a platform-specific path whenever they are read. + + * The file is automatically generated by setuptools whenever the + ‘egg_info’ or ‘sdist’ commands are run, and it is `not' + user-editable. + +Although this metadata is included with distributed eggs, it is not +actually used at runtime for any purpose. Its function is to ensure +that setuptools-built `source' distributions can correctly discover what +files are part of the project’s source, even if the list had been +generated using revision control metadata on the original author’s +system. + +In other words, ‘SOURCES.txt’ has little or no runtime value for being +included in distributed eggs, and it is possible that future versions of +the ‘bdist_egg’ and ‘install_egg_info’ commands will strip it before +installation or distribution. Therefore, do not rely on its being +available outside of an original source directory or source +distribution. + + +File: setuptools.info, Node: Other Technical Considerations, Prev: Standard Metadata, Up: The Internal Structure of Python Eggs + +8.2.3 Other Technical Considerations +------------------------------------ + +* Menu: + +* Zip File Issues:: +* Installation and Path Management Issues:: + + +File: setuptools.info, Node: Zip File Issues, Next: Installation and Path Management Issues, Up: Other Technical Considerations + +8.2.3.1 Zip File Issues +....................... + +Although zip files resemble directories, they are not fully +substitutable for them. Most platforms do not support loading dynamic +link libraries contained in zipfiles, so it is not possible to directly +import C extensions from ‘.egg’ zipfiles. Similarly, there are many +existing libraries – whether in Python or C – that require actual +operating system filenames, and do not work with arbitrary “file-like” +objects or in-memory strings, and thus cannot operate directly on the +contents of zip files. + +To address these issues, the ‘pkg_resources’ module provides a “resource +API” to support obtaining either the contents of a resource, or a true +operating system filename for the resource. If the egg containing the +resource is a directory, the resource’s real filename is simply +returned. However, if the egg is a zipfile, then the resource is first +extracted to a cache directory, and the filename within the cache is +returned. + +The cache directory is determined by the ‘pkg_resources’ API; please see +the ‘set_cache_path()’ and ‘get_default_cache()’ documentation for +details. + +* Menu: + +* The Extraction Process:: +* Extension Import Wrappers:: + + +File: setuptools.info, Node: The Extraction Process, Next: Extension Import Wrappers, Up: Zip File Issues + +8.2.3.2 The Extraction Process +.............................. + +Resources are extracted to a cache subdirectory whose name is based on +the enclosing ‘.egg’ filename and the path to the resource. If there is +already a file of the correct name, size, and timestamp, its filename is +returned to the requester. Otherwise, the desired file is extracted +first to a temporary name generated using +‘mkstemp(".$extract",target_dir)’, and then its timestamp is set to +match the one in the zip file, before renaming it to its final name. +(Some collision detection and resolution code is used to handle the fact +that Windows doesn’t overwrite files when renaming.) + +If a resource directory is requested, all of its contents are +recursively extracted in this fashion, to ensure that the directory name +can be used as if it were valid all along. + +If the resource requested for extraction is listed in the +‘native_libs.txt’ or ‘eager_resources.txt’ metadata files, then `all' +resources listed in `either' file will be extracted before the requested +resource’s filename is returned, thus ensuring that all C extensions and +data used by them will be simultaneously available. + + +File: setuptools.info, Node: Extension Import Wrappers, Prev: The Extraction Process, Up: Zip File Issues + +8.2.3.3 Extension Import Wrappers +................................. + +Since Python’s built-in zip import feature does not support loading C +extension modules from zipfiles, the setuptools ‘bdist_egg’ command +generates special import wrappers to make it work. + +The wrappers are ‘.py’ files (along with corresponding ‘.pyc’ and/or +‘.pyo’ files) that have the same module name as the corresponding C +extension. These wrappers are located in the same package directory (or +top-level directory) within the zipfile, so that say, ‘foomodule.so’ +will get a corresponding ‘foo.py’, while ‘bar/baz.pyd’ will get a +corresponding ‘bar/baz.py’. + +These wrapper files contain a short stanza of Python code that asks +‘pkg_resources’ for the filename of the corresponding C extension, then +reloads the module using the obtained filename. This will cause +‘pkg_resources’ to first ensure that all of the egg’s C extensions (and +any accompanying “eager resources”) are extracted to the cache before +attempting to link to the C library. + +Note, by the way, that ‘.egg’ directories will also contain these +wrapper files. However, Python’s default import priority is such that C +extensions take precedence over same-named Python modules, so the import +wrappers are ignored unless the egg is a zipfile. + + +File: setuptools.info, Node: Installation and Path Management Issues, Prev: Zip File Issues, Up: Other Technical Considerations + +8.2.3.4 Installation and Path Management Issues +............................................... + +Python’s initial setup of ‘sys.path’ is very dependent on the Python +version and installation platform, as well as how Python was started +(i.e., script vs. ‘-c’ vs. ‘-m’ vs. interactive interpreter). In +fact, Python also provides only two relatively robust ways to affect +‘sys.path’ outside of direct manipulation in code: the ‘PYTHONPATH’ +environment variable, and ‘.pth’ files. + +However, with no cross-platform way to safely and persistently change +environment variables, this leaves ‘.pth’ files as EasyInstall’s only +real option for persistent configuration of ‘sys.path’. + +But ‘.pth’ files are rather strictly limited in what they are allowed to +do normally. They add directories only to the `end' of ‘sys.path’, +after any locally-installed ‘site-packages’ directory, and they are only +processed `in' the ‘site-packages’ directory to start with. + +This is a double whammy for users who lack write access to that +directory, because they can’t create a ‘.pth’ file that Python will +read, and even if a sympathetic system administrator adds one for them +that calls ‘site.addsitedir()’ to allow some other directory to contain +‘.pth’ files, they won’t be able to install newer versions of anything +that’s installed in the systemwide ‘site-packages’, because their paths +will still be added `after' ‘site-packages’. + +So EasyInstall applies two workarounds to solve these problems. + +The first is that EasyInstall leverages ‘.pth’ files’ “import” feature +to manipulate ‘sys.path’ and ensure that anything EasyInstall adds to a +‘.pth’ file will always appear before both the standard library and the +local ‘site-packages’ directories. Thus, it is always possible for a +user who can write a Python-read ‘.pth’ file to ensure that their +packages come first in their own environment. + +Second, when installing to a ‘PYTHONPATH’ directory (as opposed to a +“site” directory like ‘site-packages’) EasyInstall will also install a +special version of the ‘site’ module. Because it’s in a ‘PYTHONPATH’ +directory, this module will get control before the standard library +version of ‘site’ does. It will record the state of ‘sys.path’ before +invoking the “real” ‘site’ module, and then afterwards it processes any +‘.pth’ files found in ‘PYTHONPATH’ directories, including all the fixups +needed to ensure that eggs always appear before the standard library in +sys.path, but are in a relative order to one another that is defined by +their ‘PYTHONPATH’ and ‘.pth’-prescribed sequence. + +The net result of these changes is that ‘sys.path’ order will be as +follows at runtime: + + 1. The ‘sys.argv[0]’ directory, or an empty string if no script is + being executed. + + 2. All eggs installed by EasyInstall in any ‘.pth’ file in each + ‘PYTHONPATH’ directory, in order first by ‘PYTHONPATH’ order, then + normal ‘.pth’ processing order (which is to say alphabetical by + ‘.pth’ filename, then by the order of listing within each ‘.pth’ + file). + + 3. All eggs installed by EasyInstall in any ‘.pth’ file in each “site” + directory (such as ‘site-packages’), following the same ordering + rules as for the ones on ‘PYTHONPATH’. + + 4. The ‘PYTHONPATH’ directories themselves, in their original order + + 5. Any paths from ‘.pth’ files found on ‘PYTHONPATH’ that were `not' + eggs installed by EasyInstall, again following the same relative + ordering rules. + + 6. The standard library and “site” directories, along with the + contents of any ‘.pth’ files found in the “site” directories. + +Notice that sections 1, 4, and 6 comprise the “normal” Python setup for +‘sys.path’. Sections 2 and 3 are inserted to support eggs, and section +5 emulates what the “normal” semantics of ‘.pth’ files on ‘PYTHONPATH’ +would be if Python natively supported them. + +For further discussion of the tradeoffs that went into this design, as +well as notes on the actual magic inserted into ‘.pth’ files to make +them do these things, please see also the following messages to the +distutils-SIG mailing list: + + * + ‘http://mail.python.org/pipermail/distutils-sig/2006-February/006026.html’ + + * + ‘http://mail.python.org/pipermail/distutils-sig/2006-March/006123.html’ + +* Menu: + +* Script Wrappers:: + + +File: setuptools.info, Node: Script Wrappers, Up: Installation and Path Management Issues + +8.2.3.5 Script Wrappers +....................... + +EasyInstall never directly installs a project’s original scripts to a +script installation directory. Instead, it writes short wrapper scripts +that first ensure that the project’s dependencies are active on +sys.path, before invoking the original script. These wrappers have a #! +line that points to the version of Python that was used to install them, +and their second line is always a comment that indicates the type of +script wrapper, the project version required for the script to run, and +information identifying the script to be invoked. + +The format of this marker line is: + + "# EASY-INSTALL-" script_type ": " tuple_of_strings "\n" + +The ‘script_type’ is one of ‘SCRIPT’, ‘DEV-SCRIPT’, or ‘ENTRY-SCRIPT’. +The ‘tuple_of_strings’ is a comma-separated sequence of Python string +constants. For ‘SCRIPT’ and ‘DEV-SCRIPT’ wrappers, there are two +strings: the project version requirement, and the script name (as a +filename within the ‘scripts’ metadata directory). For ‘ENTRY-SCRIPT’ +wrappers, there are three: the project version requirement, the entry +point group name, and the entry point name. (See the “Automatic Script +Creation” section in the setuptools manual for more information about +entry point scripts.) + +In each case, the project version requirement string will be a string +parseable with the ‘pkg_resources’ modules’ ‘Requirement.parse()’ +classmethod. The only difference between a ‘SCRIPT’ wrapper and a +‘DEV-SCRIPT’ is that a ‘DEV-SCRIPT’ actually executes the original +source script in the project’s source tree, and is created when the +“setup.py develop” command is run. A ‘SCRIPT’ wrapper, on the other +hand, uses the “installed” script written to the ‘EGG-INFO/scripts’ +subdirectory of the corresponding ‘.egg’ zipfile or directory. +(‘.egg-info’ eggs do not have script wrappers associated with them, +except in the “setup.py develop” case.) + +The purpose of including the marker line in generated script wrappers is +to facilitate introspection of installed scripts, and their relationship +to installed eggs. For example, an uninstallation tool could use this +data to identify what scripts can safely be removed, and/or identify +what scripts would stop working if a particular egg is uninstalled. + + +File: setuptools.info, Node: Easy Install, Next: Porting from Distutils, Prev: The Internal Structure of Python Eggs, Up: Guides on backward compatibility & deprecated practice + +8.3 Easy Install +================ + + Warning: Easy Install is deprecated. Do not use it. Instead use + pip. If you think you need Easy Install, please reach out to the + PyPA team (a ticket to pip or setuptools is fine), describing your + use-case. + +Easy Install is a python module (‘easy_install’) bundled with +‘setuptools’ that lets you automatically download, build, install, and +manage Python packages. + +Please share your experiences with us! If you encounter difficulty +installing a package, please contact us via the distutils mailing +list(1). (Note: please DO NOT send private email directly to the author +of setuptools; it will be discarded. The mailing list is a searchable +archive of previously-asked and answered questions; you should begin +your research there before reporting something as a bug – and then do so +via list discussion first.) + +(Also, if you’d like to learn about how you can use ‘setuptools’ to make +your own packages work better with EasyInstall, or provide +EasyInstall-like features without requiring your users to use +EasyInstall directly, you’ll probably want to check out the full +documentation as well.) + +* Menu: + +* Using “Easy Install”:: +* Reference Manual:: + + ---------- Footnotes ---------- + + (1) http://mail.python.org/pipermail/distutils-sig/ + + +File: setuptools.info, Node: Using “Easy Install”, Next: Reference Manual, Up: Easy Install + +8.3.1 Using “Easy Install” +-------------------------- + +* Menu: + +* Installing “Easy Install”:: +* Downloading and Installing a Package:: +* Upgrading a Package:: +* Changing the Active Version:: +* Uninstalling Packages:: +* Managing Scripts:: +* Executables and Launchers:: +* Tips & Techniques:: +* Password-Protected Sites:: +* Using .pypirc Credentials: Using pypirc Credentials. + + +File: setuptools.info, Node: Installing “Easy Install”, Next: Downloading and Installing a Package, Up: Using “Easy Install” + +8.3.1.1 Installing “Easy Install” +................................. + +Please see the setuptools PyPI page(1) for download links and basic +installation instructions for each of the supported platforms. + +You will need at least Python 3.5 or 2.7. An ‘easy_install’ script will +be installed in the normal location for Python scripts on your platform. + +Note that the instructions on the setuptools PyPI page assume that you +are are installing to Python’s primary ‘site-packages’ directory. If +this is not the case, you should consult the section below on *note +Custom Installation Locations: e8. before installing. (And, on Windows, +you should not use the ‘.exe’ installer when installing to an alternate +location.) + +Note that ‘easy_install’ normally works by downloading files from the +internet. If you are behind an NTLM-based firewall that prevents Python +programs from accessing the net directly, you may wish to first install +and use the APS proxy server(2), which lets you get past such firewalls +in the same way that your web browser(s) do. + +(Alternately, if you do not wish easy_install to actually download +anything, you can restrict it from doing so with the ‘--allow-hosts’ +option; see the sections on *note restricting downloads with +–allow-hosts: e9. and *note command-line options: ea. for more details.) + +* Menu: + +* Troubleshooting:: +* Windows Notes:: + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/setuptools/ + + (2) http://ntlmaps.sf.net/ + + +File: setuptools.info, Node: Troubleshooting, Next: Windows Notes, Up: Installing “Easy Install” + +8.3.1.2 Troubleshooting +....................... + +If EasyInstall/setuptools appears to install correctly, and you can run +the ‘easy_install’ command but it fails with an ‘ImportError’, the most +likely cause is that you installed to a location other than +‘site-packages’, without taking any of the steps described in the *note +Custom Installation Locations: e8. section below. Please see that +section and follow the steps to make sure that your custom location will +work correctly. Then re-install. + +Similarly, if you can run ‘easy_install’, and it appears to be +installing packages, but then you can’t import them, the most likely +issue is that you installed EasyInstall correctly but are using it to +install packages to a non-standard location that hasn’t been properly +prepared. Again, see the section on *note Custom Installation +Locations: e8. for more details. + + +File: setuptools.info, Node: Windows Notes, Prev: Troubleshooting, Up: Installing “Easy Install” + +8.3.1.3 Windows Notes +..................... + +Installing setuptools will provide an ‘easy_install’ command according +to the techniques described in *note Executables and Launchers: ed. If +the ‘easy_install’ command is not available after installation, that +section provides details on how to configure Windows to make the +commands available. + + +File: setuptools.info, Node: Downloading and Installing a Package, Next: Upgrading a Package, Prev: Installing “Easy Install”, Up: Using “Easy Install” + +8.3.1.4 Downloading and Installing a Package +............................................ + +For basic use of ‘easy_install’, you need only supply the filename or +URL of a source distribution or .egg file (Python Egg(1)). + +`Example 1'. Install a package by name, searching PyPI for the latest +version, and automatically downloading, building, and installing it: + + easy_install SQLObject + +`Example 2'. Install or upgrade a package by name and version by +finding links on a given “download page”: + + easy_install -f http://pythonpaste.org/package_index.html SQLObject + +`Example 3'. Download a source distribution from a specified URL, +automatically building and installing it: + + easy_install http://example.com/path/to/MyPackage-1.2.3.tgz + +`Example 4'. Install an already-downloaded .egg file: + + easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg + +`Example 5'. Upgrade an already-installed package to the latest version +listed on PyPI: + + easy_install --upgrade PyProtocols + +`Example 6'. Install a source distribution that’s already downloaded +and extracted in the current directory (New in 0.5a9): + + easy_install . + +`Example 7'. (New in 0.6a1) Find a source distribution or Subversion +checkout URL for a package, and extract it or check it out to +‘~/projects/sqlobject’ (the name will always be in all-lowercase), where +it can be examined or edited. (The package will not be installed, but +it can easily be installed with ‘easy_install ~/projects/sqlobject’. +See *note Editing and Viewing Source Packages: ef. below for more +info.): + + easy_install --editable --build-directory ~/projects SQLObject + +`Example 7'. (New in 0.6.11) Install a distribution within your home +dir: + + easy_install --user SQLAlchemy + +Easy Install accepts URLs, filenames, PyPI package names (i.e., +‘distutils’ “distribution” names), and package+version specifiers. In +each case, it will attempt to locate the latest available version that +meets your criteria. + +When downloading or processing downloaded files, Easy Install recognizes +distutils source distribution files with extensions of .tgz, .tar, +.tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg +distributions as well as ‘.win32.exe’ installers built using distutils. + +By default, packages are installed to the running Python installation’s +‘site-packages’ directory, unless you provide the ‘-d’ or +‘--install-dir’ option to specify an alternative directory, or specify +an alternate location using distutils configuration files. (See *note +Configuration Files: f0, below.) + +By default, any scripts included with the package are installed to the +running Python installation’s standard script installation location. +However, if you specify an installation directory via the command line +or a config file, then the default directory for installing scripts will +be the same as the package installation directory, to ensure that the +script will have access to the installed package. You can override this +using the ‘-s’ or ‘--script-dir’ option. + +Installed packages are added to an ‘easy-install.pth’ file in the +install directory, so that Python will always use the +most-recently-installed version of the package. If you would like to be +able to select which version to use at runtime, you should use the ‘-m’ +or ‘--multi-version’ option. + + ---------- Footnotes ---------- + + (1) http://peak.telecommunity.com/DevCenter/PythonEggs + + +File: setuptools.info, Node: Upgrading a Package, Next: Changing the Active Version, Prev: Downloading and Installing a Package, Up: Using “Easy Install” + +8.3.1.5 Upgrading a Package +........................... + +You don’t need to do anything special to upgrade a package: just install +the new version, either by requesting a specific version, e.g.: + + easy_install "SomePackage==2.0" + +a version greater than the one you have now: + + easy_install "SomePackage>2.0" + +using the upgrade flag, to find the latest available version on PyPI: + + easy_install --upgrade SomePackage + +or by using a download page, direct download URL, or package filename: + + easy_install -f http://example.com/downloads ExamplePackage + + easy_install http://example.com/downloads/ExamplePackage-2.0-py2.4.egg + + easy_install my_downloads/ExamplePackage-2.0.tgz + +If you’re using ‘-m’ or ‘--multi-version’ , using the ‘require()’ +function at runtime automatically selects the newest installed version +of a package that meets your version criteria. So, installing a newer +version is the only step needed to upgrade such packages. + +If you’re installing to a directory on PYTHONPATH, or a configured +“site” directory (and not using ‘-m’), installing a package +automatically replaces any previous version in the ‘easy-install.pth’ +file, so that Python will import the most-recently installed version by +default. So, again, installing the newer version is the only upgrade +step needed. + +If you haven’t suppressed script installation (using ‘--exclude-scripts’ +or ‘-x’), then the upgraded version’s scripts will be installed, and +they will be automatically patched to ‘require()’ the corresponding +version of the package, so that you can use them even if they are +installed in multi-version mode. + +‘easy_install’ never actually deletes packages (unless you’re installing +a package with the same name and version number as an existing package), +so if you want to get rid of older versions of a package, please see +*note Uninstalling Packages: f2, below. + + +File: setuptools.info, Node: Changing the Active Version, Next: Uninstalling Packages, Prev: Upgrading a Package, Up: Using “Easy Install” + +8.3.1.6 Changing the Active Version +................................... + +If you’ve upgraded a package, but need to revert to a +previously-installed version, you can do so like this: + + easy_install PackageName==1.2.3 + +Where ‘1.2.3’ is replaced by the exact version number you wish to switch +to. If a package matching the requested name and version is not already +installed in a directory on ‘sys.path’, it will be located via PyPI and +installed. + +If you’d like to switch to the latest installed version of +‘PackageName’, you can do so like this: + + easy_install PackageName + +This will activate the latest installed version. (Note: if you have set +any ‘find_links’ via distutils configuration files, those download pages +will be checked for the latest available version of the package, and it +will be downloaded and installed if it is newer than your current +version.) + +Note that changing the active version of a package will install the +newly active version’s scripts, unless the ‘--exclude-scripts’ or ‘-x’ +option is specified. + + +File: setuptools.info, Node: Uninstalling Packages, Next: Managing Scripts, Prev: Changing the Active Version, Up: Using “Easy Install” + +8.3.1.7 Uninstalling Packages +............................. + +If you have replaced a package with another version, then you can just +delete the package(s) you don’t need by deleting the +PackageName-versioninfo.egg file or directory (found in the installation +directory). + +If you want to delete the currently installed version of a package (or +all versions of a package), you should first run: + + easy_install -m PackageName + +This will ensure that Python doesn’t continue to search for a package +you’re planning to remove. After you’ve done this, you can safely +delete the .egg files or directories, along with any scripts you wish to +remove. + + +File: setuptools.info, Node: Managing Scripts, Next: Executables and Launchers, Prev: Uninstalling Packages, Up: Using “Easy Install” + +8.3.1.8 Managing Scripts +........................ + +Whenever you install, upgrade, or change versions of a package, +EasyInstall automatically installs the scripts for the selected package +version, unless you tell it not to with ‘-x’ or ‘--exclude-scripts’. If +any scripts in the script directory have the same name, they are +overwritten. + +Thus, you do not normally need to manually delete scripts for older +versions of a package, unless the newer version of the package does not +include a script of the same name. However, if you are completely +uninstalling a package, you may wish to manually delete its scripts. + +EasyInstall’s default behavior means that you can normally only run +scripts from one version of a package at a time. If you want to keep +multiple versions of a script available, however, you can simply use the +‘--multi-version’ or ‘-m’ option, and rename the scripts that +EasyInstall creates. This works because EasyInstall installs scripts as +short code stubs that ‘require()’ the matching version of the package +the script came from, so renaming the script has no effect on what it +executes. + +For example, suppose you want to use two versions of the ‘rst2html’ tool +provided by the docutils(1) package. You might first install one +version: + + easy_install -m docutils==0.3.9 + +then rename the ‘rst2html.py’ to ‘r2h_039’, and install another version: + + easy_install -m docutils==0.3.10 + +This will create another ‘rst2html.py’ script, this one using docutils +version 0.3.10 instead of 0.3.9. You now have two scripts, each using a +different version of the package. (Notice that we used ‘-m’ for both +installations, so that Python won’t lock us out of using anything but +the most recently-installed version of the package.) + + ---------- Footnotes ---------- + + (1) http://docutils.sf.net/ + + +File: setuptools.info, Node: Executables and Launchers, Next: Tips & Techniques, Prev: Managing Scripts, Up: Using “Easy Install” + +8.3.1.9 Executables and Launchers +................................. + +On Unix systems, scripts are installed with as natural files with a “#!” +header and no extension and they launch under the Python version +indicated in the header. + +On Windows, there is no mechanism to “execute” files without extensions, +so EasyInstall provides two techniques to mirror the Unix behavior. The +behavior is indicated by the SETUPTOOLS_LAUNCHER environment variable, +which may be “executable” (default) or “natural”. + +Regardless of the technique used, the script(s) will be installed to a +Scripts directory (by default in the Python installation directory). It +is recommended for EasyInstall that you ensure this directory is in the +PATH environment variable. The easiest way to ensure the Scripts +directory is in the PATH is to run ‘Tools\Scripts\win_add2path.py’ from +the Python directory. + +Note that instead of changing your ‘PATH’ to include the Python scripts +directory, you can also retarget the installation location for scripts +so they go on a directory that’s already on the ‘PATH’. For more +information see *note Command-Line Options: ea. and *note Configuration +Files: f0. During installation, pass command line options (such as +‘--script-dir’) to control where scripts will be installed. + +* Menu: + +* Windows Executable Launcher:: +* Natural Script Launcher:: + + +File: setuptools.info, Node: Windows Executable Launcher, Next: Natural Script Launcher, Up: Executables and Launchers + +8.3.1.10 Windows Executable Launcher +.................................... + +If the “executable” launcher is used, EasyInstall will create a ‘.exe’ +launcher of the same name beside each installed script (including +‘easy_install’ itself). These small .exe files launch the script of the +same name using the Python version indicated in the ‘#!’ header. + +This behavior is currently default. To force the use of executable +launchers, set ‘SETUPTOOLS_LAUNCHER’ to “executable”. + + +File: setuptools.info, Node: Natural Script Launcher, Prev: Windows Executable Launcher, Up: Executables and Launchers + +8.3.1.11 Natural Script Launcher +................................ + +EasyInstall also supports deferring to an external launcher such as +pylauncher(1) for launching scripts. Enable this experimental +functionality by setting the ‘SETUPTOOLS_LAUNCHER’ environment variable +to “natural”. EasyInstall will then install scripts as simple scripts +with a .pya (or .pyw) extension appended. If these extensions are +associated with the pylauncher and listed in the PATHEXT environment +variable, these scripts can then be invoked simply and directly just +like any other executable. This behavior may become default in a future +version. + +EasyInstall uses the .pya extension instead of simply the typical ‘.py’ +extension. This distinct extension is necessary to prevent Python from +treating the scripts as importable modules (where name conflicts exist). +Current releases of pylauncher do not yet associate with .pya files by +default, but future versions should do so. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/pylauncher + + +File: setuptools.info, Node: Tips & Techniques, Next: Password-Protected Sites, Prev: Executables and Launchers, Up: Using “Easy Install” + +8.3.1.12 Tips & Techniques +.......................... + +* Menu: + +* Multiple Python Versions:: +* Restricting Downloads with --allow-hosts:: +* Installing on Un-networked Machines:: +* Packaging Others’ Projects As Eggs:: +* Creating your own Package Index:: + + +File: setuptools.info, Node: Multiple Python Versions, Next: Restricting Downloads with --allow-hosts, Up: Tips & Techniques + +8.3.1.13 Multiple Python Versions +................................. + +EasyInstall installs itself under two names: ‘easy_install’ and +‘easy_install-N.N’, where ‘N.N’ is the Python version used to install +it. Thus, if you install EasyInstall for both Python 3.2 and 2.7, you +can use the ‘easy_install-3.2’ or ‘easy_install-2.7’ scripts to install +packages for the respective Python version. + +Setuptools also supplies easy_install as a runnable module which may be +invoked using ‘python -m easy_install’ for any Python with Setuptools +installed. + + +File: setuptools.info, Node: Restricting Downloads with --allow-hosts, Next: Installing on Un-networked Machines, Prev: Multiple Python Versions, Up: Tips & Techniques + +8.3.1.14 Restricting Downloads with ‘--allow-hosts’ +................................................... + +You can use the ‘--allow-hosts’ (‘-H’) option to restrict what domains +EasyInstall will look for links and downloads on. ‘--allow-hosts=None’ +prevents downloading altogether. You can also use wildcards, for +example to restrict downloading to hosts in your own intranet. See the +section below on *note Command-Line Options: ea. for more details on the +‘--allow-hosts’ option. + +By default, there are no host restrictions in effect, but you can change +this default by editing the appropriate *note configuration files: f0. +and adding: + + [easy_install] + allow_hosts = *.myintranet.example.com,*.python.org + +The above example would then allow downloads only from hosts in the +‘python.org’ and ‘myintranet.example.com’ domains, unless overridden on +the command line. + + +File: setuptools.info, Node: Installing on Un-networked Machines, Next: Packaging Others’ Projects As Eggs, Prev: Restricting Downloads with --allow-hosts, Up: Tips & Techniques + +8.3.1.15 Installing on Un-networked Machines +............................................ + +Just copy the eggs or source packages you need to a directory on the +target machine, then use the ‘-f’ or ‘--find-links’ option to specify +that directory’s location. For example: + + easy_install -H None -f somedir SomePackage + +will attempt to install SomePackage using only eggs and source packages +found in ‘somedir’ and disallowing all remote access. You should of +course make sure you have all of SomePackage’s dependencies available in +somedir. + +If you have another machine of the same operating system and library +versions (or if the packages aren’t platform-specific), you can create +the directory of eggs using a command like this: + + easy_install -zmaxd somedir SomePackage + +This will tell EasyInstall to put zipped eggs or source packages for +SomePackage and all its dependencies into ‘somedir’, without creating +any scripts or .pth files. You can then copy the contents of ‘somedir’ +to the target machine. (‘-z’ means zipped eggs, ‘-m’ means +multi-version, which prevents .pth files from being used, ‘-a’ means to +copy all the eggs needed, even if they’re installed elsewhere on the +machine, and ‘-d’ indicates the directory to place the eggs in.) + +You can also build the eggs from local development packages that were +installed with the ‘setup.py develop’ command, by including the ‘-l’ +option, e.g.: + + easy_install -zmaxld somedir SomePackage + +This will use locally-available source distributions to build the eggs. + + +File: setuptools.info, Node: Packaging Others’ Projects As Eggs, Next: Creating your own Package Index, Prev: Installing on Un-networked Machines, Up: Tips & Techniques + +8.3.1.16 Packaging Others’ Projects As Eggs +........................................... + +Need to distribute a package that isn’t published in egg form? You can +use EasyInstall to build eggs for a project. You’ll want to use the +‘--zip-ok’, ‘--exclude-scripts’, and possibly ‘--no-deps’ options (‘-z’, +‘-x’ and ‘-N’, respectively). Use ‘-d’ or ‘--install-dir’ to specify +the location where you’d like the eggs placed. By placing them in a +directory that is published to the web, you can then make the eggs +available for download, either in an intranet or to the internet at +large. + +If someone distributes a package in the form of a single ‘.py’ file, you +can wrap it in an egg by tacking an ‘#egg=name-version’ suffix on the +file’s URL. So, something like this: + + easy_install -f "http://some.example.com/downloads/foo.py#egg=foo-1.0" foo + +will install the package as an egg, and this: + + easy_install -zmaxd. \ + -f "http://some.example.com/downloads/foo.py#egg=foo-1.0" foo + +will create a ‘.egg’ file in the current directory. + + +File: setuptools.info, Node: Creating your own Package Index, Prev: Packaging Others’ Projects As Eggs, Up: Tips & Techniques + +8.3.1.17 Creating your own Package Index +........................................ + +In addition to local directories and the Python Package Index, +EasyInstall can find download links on most any web page whose URL is +given to the ‘-f’ (‘--find-links’) option. In the simplest case, you +can simply have a web page with links to eggs or Python source packages, +even an automatically generated directory listing (such as the Apache +web server provides). + +If you are setting up an intranet site for package downloads, you may +want to configure the target machines to use your download site by +default, adding something like this to their *note configuration files: +f0.: + + [easy_install] + find_links = http://mypackages.example.com/somedir/ + http://turbogears.org/download/ + http://peak.telecommunity.com/dist/ + +As you can see, you can list multiple URLs separated by whitespace, +continuing on multiple lines if necessary (as long as the subsequent +lines are indented. + +If you are more ambitious, you can also create an entirely custom +package index or PyPI mirror. See the ‘--index-url’ option under *note +Command-Line Options: ea, below, and also the section on *note Package +Index "API": fc. + + +File: setuptools.info, Node: Password-Protected Sites, Next: Using pypirc Credentials, Prev: Tips & Techniques, Up: Using “Easy Install” + +8.3.1.18 Password-Protected Sites +................................. + +If a site you want to download from is password-protected using HTTP +“Basic” authentication, you can specify your credentials in the URL, +like so: + + http://some_userid:some_password@some.example.com/some_path/ + +You can do this with both index page URLs and direct download URLs. As +long as any HTML pages read by easy_install use `relative' links to +point to the downloads, the same user ID and password will be used to do +the downloading. + + +File: setuptools.info, Node: Using pypirc Credentials, Prev: Password-Protected Sites, Up: Using “Easy Install” + +8.3.1.19 Using .pypirc Credentials +.................................. + +In additional to supplying credentials in the URL, ‘easy_install’ will +also honor credentials if present in the .pypirc file. Teams +maintaining a private repository of packages may already have defined +access credentials for uploading packages according to the distutils +documentation. ‘easy_install’ will attempt to honor those if present. +Refer to the distutils documentation for Python 2.5 or later for details +on the syntax. + +* Menu: + +* Controlling Build Options:: +* Editing and Viewing Source Packages:: +* Dealing with Installation Conflicts:: +* Compressed Installation:: + + +File: setuptools.info, Node: Controlling Build Options, Next: Editing and Viewing Source Packages, Up: Using pypirc Credentials + +8.3.1.20 Controlling Build Options +.................................. + +EasyInstall respects standard distutils *note Configuration Files: f0, +so you can use them to configure build options for packages that it +installs from source. For example, if you are on Windows using the +MinGW compiler, you can configure the default compiler by putting +something like this: + + [build] + compiler = mingw32 + +into the appropriate distutils configuration file. In fact, since this +is just normal distutils configuration, it will affect any builds using +that config file, not just ones done by EasyInstall. For example, if +you add those lines to ‘distutils.cfg’ in the ‘distutils’ package +directory, it will be the default compiler for `all' packages you build. +See *note Configuration Files: f0. below for a list of the standard +configuration file locations, and links to more documentation on using +distutils configuration files. + + +File: setuptools.info, Node: Editing and Viewing Source Packages, Next: Dealing with Installation Conflicts, Prev: Controlling Build Options, Up: Using pypirc Credentials + +8.3.1.21 Editing and Viewing Source Packages +............................................ + +Sometimes a package’s source distribution contains additional +documentation, examples, configuration files, etc., that are not part of +its actual code. If you want to be able to examine these files, you can +use the ‘--editable’ option to EasyInstall, and EasyInstall will look +for a source distribution or Subversion URL for the package, then +download and extract it or check it out as a subdirectory of the +‘--build-directory’ you specify. If you then wish to install the +package after editing or configuring it, you can do so by rerunning +EasyInstall with that directory as the target. + +Note that using ‘--editable’ stops EasyInstall from actually building or +installing the package; it just finds, obtains, and possibly unpacks it +for you. This allows you to make changes to the package if necessary, +and to either install it in development mode using ‘setup.py develop’ +(if the package uses setuptools, that is), or by running ‘easy_install +projectdir’ (where ‘projectdir’ is the subdirectory EasyInstall created +for the downloaded package. + +In order to use ‘--editable’ (‘-e’ for short), you `must' also supply a +‘--build-directory’ (‘-b’ for short). The project will be placed in a +subdirectory of the build directory. The subdirectory will have the +same name as the project itself, but in all-lowercase. If a file or +directory of that name already exists, EasyInstall will print an error +message and exit. + +Also, when using ‘--editable’, you cannot use URLs or filenames as +arguments. You `must' specify project names (and optional version +requirements) so that EasyInstall knows what directory name(s) to +create. If you need to force EasyInstall to use a particular URL or +filename, you should specify it as a ‘--find-links’ item (‘-f’ for +short), and then also specify the project name, e.g.: + + easy_install -eb ~/projects \ + -fhttp://prdownloads.sourceforge.net/ctypes/ctypes-0.9.6.tar.gz?download \ + ctypes==0.9.6 + + +File: setuptools.info, Node: Dealing with Installation Conflicts, Next: Compressed Installation, Prev: Editing and Viewing Source Packages, Up: Using pypirc Credentials + +8.3.1.22 Dealing with Installation Conflicts +............................................ + +(NOTE: As of 0.6a11, this section is obsolete; it is retained here only +so that people using older versions of EasyInstall can consult it. As +of version 0.6a11, installation conflicts are handled automatically +without deleting the old or system-installed packages, and without +ignoring the issue. Instead, eggs are automatically shifted to the +front of ‘sys.path’ using special code added to the ‘easy-install.pth’ +file. So, if you are using version 0.6a11 or better of setuptools, you +do not need to worry about conflicts, and the following issues do not +apply to you.) + +EasyInstall installs distributions in a “managed” way, such that each +distribution can be independently activated or deactivated on +‘sys.path’. However, packages that were not installed by EasyInstall +are “unmanaged”, in that they usually live all in one directory and +cannot be independently activated or deactivated. + +As a result, if you are using EasyInstall to upgrade an existing +package, or to install a package with the same name as an existing +package, EasyInstall will warn you of the conflict. (This is an +improvement over ‘setup.py install’, because the ‘distutils’ just +install new packages on top of old ones, possibly combining two +unrelated packages or leaving behind modules that have been deleted in +the newer version of the package.) + +EasyInstall will stop the installation if it detects a conflict between +an existing, “unmanaged” package, and a module or package in any of the +distributions you’re installing. It will display a list of all of the +existing files and directories that would need to be deleted for the new +package to be able to function correctly. To proceed, you must manually +delete these conflicting files and directories and re-run EasyInstall. + +Of course, once you’ve replaced all of your existing “unmanaged” +packages with versions managed by EasyInstall, you won’t have any more +conflicts to worry about! + + +File: setuptools.info, Node: Compressed Installation, Prev: Dealing with Installation Conflicts, Up: Using pypirc Credentials + +8.3.1.23 Compressed Installation +................................ + +EasyInstall tries to install packages in zipped form, if it can. +Zipping packages can improve Python’s overall import performance if +you’re not using the ‘--multi-version’ option, because Python processes +zipfile entries on ‘sys.path’ much faster than it does directories. + +As of version 0.5a9, EasyInstall analyzes packages to determine whether +they can be safely installed as a zipfile, and then acts on its +analysis. (Previous versions would not install a package as a zipfile +unless you used the ‘--zip-ok’ option.) + +The current analysis approach is fairly conservative; it currently looks +for: + + * Any use of the ‘__file__’ or ‘__path__’ variables (which + should be replaced with ‘pkg_resources’ API calls) + + * Possible use of ‘inspect’ functions that expect to manipulate + source files (e.g. ‘inspect.getsource()’) + + * Top-level modules that might be scripts used with ‘python -m’ + (Python 2.4) + +If any of the above are found in the package being installed, +EasyInstall will assume that the package cannot be safely run from a +zipfile, and unzip it to a directory instead. You can override this +analysis with the ‘-zip-ok’ flag, which will tell EasyInstall to install +the package as a zipfile anyway. Or, you can use the ‘--always-unzip’ +flag, in which case EasyInstall will always unzip, even if its analysis +says the package is safe to run as a zipfile. + +Normally, however, it is simplest to let EasyInstall handle the +determination of whether to zip or unzip, and only specify overrides +when needed to work around a problem. If you find you need to override +EasyInstall’s guesses, you may want to contact the package author and +the EasyInstall maintainers, so that they can make appropriate changes +in future versions. + +(Note: If a package uses ‘setuptools’ in its setup script, the package +author has the option to declare the package safe or unsafe for zipped +usage via the ‘zip_safe’ argument to ‘setup()’. If the package author +makes such a declaration, EasyInstall believes the package’s author and +does not perform its own analysis. However, your command-line option, +if any, will still override the package author’s choice.) + + +File: setuptools.info, Node: Reference Manual, Prev: Using “Easy Install”, Up: Easy Install + +8.3.2 Reference Manual +---------------------- + +* Menu: + +* Configuration Files:: +* Command-Line Options:: +* Custom Installation Locations:: +* Package Index “API”:: + + +File: setuptools.info, Node: Configuration Files, Next: Command-Line Options, Up: Reference Manual + +8.3.2.1 Configuration Files +........................... + +(New in 0.4a2) + +You may specify default options for EasyInstall using the standard +distutils configuration files, under the command heading ‘easy_install’. +EasyInstall will look first for a ‘setup.cfg’ file in the current +directory, then a ‘~/.pydistutils.cfg’ or ‘$HOME\\pydistutils.cfg’ (on +Unix-like OSes and Windows, respectively), and finally a ‘distutils.cfg’ +file in the ‘distutils’ package directory. Here’s a simple example: + + [easy_install] + + # set the default location to install packages + install_dir = /home/me/lib/python + + # Notice that indentation can be used to continue an option + # value; this is especially useful for the "--find-links" + # option, which tells easy_install to use download links on + # these pages before consulting PyPI: + # + find_links = http://sqlobject.org/ + http://peak.telecommunity.com/dist/ + +In addition to accepting configuration for its own options under +‘[easy_install]’, EasyInstall also respects defaults specified for other +distutils commands. For example, if you don’t set an ‘install_dir’ for +‘[easy_install]’, but `have' set an ‘install_lib’ for the ‘[install]’ +command, this will become EasyInstall’s default installation directory. +Thus, if you are already using distutils configuration files to set +default install locations, build options, etc., EasyInstall will respect +your existing settings until and unless you override them explicitly in +an ‘[easy_install]’ section. + +For more information, see also the current Python documentation on the +use and location of distutils configuration files(1). + +Notice that ‘easy_install’ will use the ‘setup.cfg’ from the current +working directory only if it was triggered from ‘setup.py’ through the +‘install_requires’ option. The standalone command will not use that +file. + + ---------- Footnotes ---------- + + (1) https://docs.python.org/install/index.html#inst-config-files + + +File: setuptools.info, Node: Command-Line Options, Next: Custom Installation Locations, Prev: Configuration Files, Up: Reference Manual + +8.3.2.2 Command-Line Options +............................ + +‘--zip-ok, -z’ + + Install all packages as zip files, even if they are marked as + unsafe for running as a zipfile. This can be useful when + EasyInstall’s analysis of a non-setuptools package is too + conservative, but keep in mind that the package may not work + correctly. (Changed in 0.5a9; previously this option was required + in order for zipped installation to happen at all.) + +‘--always-unzip, -Z’ + + Don’t install any packages as zip files, even if the packages are + marked as safe for running as a zipfile. This can be useful if a + package does something unsafe, but not in a way that EasyInstall + can easily detect. EasyInstall’s default analysis is currently + very conservative, however, so you should only use this option if + you’ve had problems with a particular package, and `after' + reporting the problem to the package’s maintainer and to the + EasyInstall maintainers. + + (Note: the ‘-z/-Z’ options only affect the installation of + newly-built or downloaded packages that are not already installed + in the target directory; if you want to convert an existing + installed version from zipped to unzipped or vice versa, you’ll + need to delete the existing version first, and re-run EasyInstall.) + +‘--multi-version, -m’ + + “Multi-version” mode. Specifying this option prevents + ‘easy_install’ from adding an ‘easy-install.pth’ entry for the + package being installed, and if an entry for any version the + package already exists, it will be removed upon successful + installation. In multi-version mode, no specific version of the + package is available for importing, unless you use + ‘pkg_resources.require()’ to put it on ‘sys.path’. This can be as + simple as: + + from pkg_resources import require + require("SomePackage", "OtherPackage", "MyPackage") + + which will put the latest installed version of the specified + packages on ‘sys.path’ for you. (For more advanced uses, like + selecting specific versions and enabling optional dependencies, see + the ‘pkg_resources’ API doc.) + + Changed in 0.6a10: this option is no longer silently enabled when + installing to a non-PYTHONPATH, non-“site” directory. You must + always explicitly use this option if you want it to be active. + +‘--upgrade, -U’ (New in 0.5a4) + + By default, EasyInstall only searches online if a project/version + requirement can’t be met by distributions already installed on + sys.path or the installation directory. However, if you supply the + ‘--upgrade’ or ‘-U’ flag, EasyInstall will always check the package + index and ‘--find-links’ URLs before selecting a version to + install. In this way, you can force EasyInstall to use the latest + available version of any package it installs (subject to any + version requirements that might exclude such later versions). + +‘--install-dir=DIR, -d DIR’ + + Set the installation directory. It is up to you to ensure that + this directory is on ‘sys.path’ at runtime, and to use + ‘pkg_resources.require()’ to enable the installed package(s) that + you need. + + (New in 0.4a2) If this option is not directly specified on the + command line or in a distutils configuration file, the distutils + default installation location is used. Normally, this would be the + ‘site-packages’ directory, but if you are using distutils + configuration files, setting things like ‘prefix’ or ‘install_lib’, + then those settings are taken into account when computing the + default installation directory, as is the ‘--prefix’ option. + +‘--script-dir=DIR, -s DIR’ + + Set the script installation directory. If you don’t supply this + option (via the command line or a configuration file), but you + `have' supplied an ‘--install-dir’ (via command line or config + file), then this option defaults to the same directory, so that the + scripts will be able to find their associated package installation. + Otherwise, this setting defaults to the location where the + distutils would normally install scripts, taking any distutils + configuration file settings into account. + +‘--exclude-scripts, -x’ + + Don’t install scripts. This is useful if you need to install + multiple versions of a package, but do not want to reset the + version that will be run by scripts that are already installed. + +‘--user’ (New in 0.6.11) + + Use the user-site-packages as specified in PEP 370(1) instead of + the global site-packages. + +‘--always-copy, -a’ (New in 0.5a4) + + Copy all needed distributions to the installation directory, even + if they are already present in a directory on sys.path. In older + versions of EasyInstall, this was the default behavior, but now you + must explicitly request it. By default, EasyInstall will no longer + copy such distributions from other sys.path directories to the + installation directory, unless you explicitly gave the + distribution’s filename on the command line. + + Note that as of 0.6a10, using this option excludes “system” and + “development” eggs from consideration because they can’t be + reliably copied. This may cause EasyInstall to choose an older + version of a package than what you expected, or it may cause + downloading and installation of a fresh copy of something that’s + already installed. You will see warning messages for any eggs that + EasyInstall skips, before it falls back to an older version or + attempts to download a fresh copy. + +‘--find-links=URLS_OR_FILENAMES, -f URLS_OR_FILENAMES’ + + Scan the specified “download pages” or directories for direct links + to eggs or other distributions. Any existing file or directory + names or direct download URLs are immediately added to + EasyInstall’s search cache, and any indirect URLs (ones that don’t + point to eggs or other recognized archive formats) are added to a + list of additional places to search for download links. As soon as + EasyInstall has to go online to find a package (either because it + doesn’t exist locally, or because ‘--upgrade’ or ‘-U’ was used), + the specified URLs will be downloaded and scanned for additional + direct links. + + Eggs and archives found by way of ‘--find-links’ are only + downloaded if they are needed to meet a requirement specified on + the command line; links to unneeded packages are ignored. + + If all requested packages can be found using links on the specified + download pages, the Python Package Index will not be consulted + unless you also specified the ‘--upgrade’ or ‘-U’ option. + + (Note: if you want to refer to a local HTML file containing links, + you must use a ‘file:’ URL, as filenames that do not refer to a + directory, egg, or archive are ignored.) + + You may specify multiple URLs or file/directory names with this + option, separated by whitespace. Note that on the command line, + you will probably have to surround the URL list with quotes, so + that it is recognized as a single option value. You can also + specify URLs in a configuration file; see *note Configuration + Files: f0, above. + + Changed in 0.6a10: previously all URLs and directories passed to + this option were scanned as early as possible, but from 0.6a10 on, + only directories and direct archive links are scanned immediately; + URLs are not retrieved unless a package search was already going to + go online due to a package not being available locally, or due to + the use of the ‘--update’ or ‘-U’ option. + +‘--no-find-links’ Blocks the addition of any link. + + This parameter is useful if you want to avoid adding links defined + in a project easy_install is installing (whether it’s a requested + project or a dependency). When used, ‘--find-links’ is ignored. + + Added in Distribute 0.6.11 and Setuptools 0.7. + +‘--index-url=URL, -i URL’ (New in 0.4a1; default changed in 0.6c7) + + Specifies the base URL of the Python Package Index. The default is + ‘https://pypi.org/simple/’ if not specified. When a package is + requested that is not locally available or linked from a + ‘--find-links’ download page, the package index will be searched + for download pages for the needed package, and those download pages + will be searched for links to download an egg or source + distribution. + +‘--editable, -e’ (New in 0.6a1) + + Only find and download source distributions for the specified + projects, unpacking them to subdirectories of the specified + ‘--build-directory’. EasyInstall will not actually build or + install the requested projects or their dependencies; it will just + find and extract them for you. See *note Editing and Viewing + Source Packages: ef. above for more details. + +‘--build-directory=DIR, -b DIR’ (UPDATED in 0.6a1) + + Set the directory used to build source packages. If a package is + built from a source distribution or checkout, it will be extracted + to a subdirectory of the specified directory. The subdirectory + will have the same name as the extracted distribution’s project, + but in all-lowercase. If a file or directory of that name already + exists in the given directory, a warning will be printed to the + console, and the build will take place in a temporary directory + instead. + + This option is most useful in combination with the ‘--editable’ + option, which forces EasyInstall to `only' find and extract (but + not build and install) source distributions. See *note Editing and + Viewing Source Packages: ef, above, for more information. + +‘--verbose, -v, --quiet, -q’ (New in 0.4a4) + + Control the level of detail of EasyInstall’s progress messages. + The default detail level is “info”, which prints information only + about relatively time-consuming operations like running a setup + script, unpacking an archive, or retrieving a URL. Using ‘-q’ or + ‘--quiet’ drops the detail level to “warn”, which will only display + installation reports, warnings, and errors. Using ‘-v’ or + ‘--verbose’ increases the detail level to include individual + file-level operations, link analysis messages, and distutils + messages from any setup scripts that get run. If you include the + ‘-v’ option more than once, the second and subsequent uses are + passed down to any setup scripts, increasing the verbosity of their + reporting as well. + +‘--dry-run, -n’ (New in 0.4a4) + + Don’t actually install the package or scripts. This option is + passed down to any setup scripts run, so packages should not + actually build either. This does `not' skip downloading, nor does + it skip extracting source distributions to a temporary/build + directory. + +‘--optimize=LEVEL’, ‘-O LEVEL’ (New in 0.4a4) + + If you are installing from a source distribution, and are `not' + using the ‘--zip-ok’ option, this option controls the optimization + level for compiling installed ‘.py’ files to ‘.pyo’ files. It does + not affect the compilation of modules contained in ‘.egg’ files, + only those in ‘.egg’ directories. The optimization level can be + set to 0, 1, or 2; the default is 0 (unless it’s set under + ‘install’ or ‘install_lib’ in one of your distutils configuration + files). + +‘--record=FILENAME’ (New in 0.5a4) + + Write a record of all installed files to FILENAME. This is + basically the same as the same option for the standard distutils + “install” command, and is included for compatibility with tools + that expect to pass this option to “setup.py install”. + +‘--site-dirs=DIRLIST, -S DIRLIST’ (New in 0.6a1) + + Specify one or more custom “site” directories (separated by + commas). “Site” directories are directories where ‘.pth’ files are + processed, such as the main Python ‘site-packages’ directory. As + of 0.6a10, EasyInstall automatically detects whether a given + directory processes ‘.pth’ files (or can be made to do so), so you + should not normally need to use this option. It is is now only + necessary if you want to override EasyInstall’s judgment and force + an installation directory to be treated as if it supported ‘.pth’ + files. + +‘--no-deps, -N’ (New in 0.6a6) + + Don’t install any dependencies. This is intended as a convenience + for tools that wrap eggs in a platform-specific packaging system. + (We don’t recommend that you use it for anything else.) + +‘--allow-hosts=PATTERNS, -H PATTERNS’ (New in 0.6a6) + + Restrict downloading and spidering to hosts matching the specified + glob patterns. E.g. ‘-H *.python.org’ restricts web access so + that only packages listed and downloadable from machines in the + ‘python.org’ domain. The glob patterns must match the `entire' + user/host/port section of the target URL(s). For example, + ‘*.python.org’ will NOT accept a URL like ‘http://python.org/foo’ + or ‘http://www.python.org:8080/’. Multiple patterns can be + specified by separating them with commas. The default pattern is + ‘*’, which matches anything. + + In general, this option is mainly useful for blocking EasyInstall’s + web access altogether (e.g. ‘-Hlocalhost’), or to restrict it to + an intranet or other trusted site. EasyInstall will do the best it + can to satisfy dependencies given your host restrictions, but of + course can fail if it can’t find suitable packages. EasyInstall + displays all blocked URLs, so that you can adjust your + ‘--allow-hosts’ setting if it is more strict than you intended. + Some sites may wish to define a restrictive default setting for + this option in their *note configuration files: f0, and then + manually override the setting on the command line as needed. + +‘--prefix=DIR’ (New in 0.6a10) + + Use the specified directory as a base for computing the default + installation and script directories. On Windows, the resulting + default directories will be ‘prefix\\Lib\\site-packages’ and + ‘prefix\\Scripts’, while on other platforms the defaults will be + ‘prefix/lib/python2.X/site-packages’ (with the appropriate version + substituted) for libraries and ‘prefix/bin’ for scripts. + + Note that the ‘--prefix’ option only sets the `default' + installation and script directories, and does not override the ones + set on the command line or in a configuration file. + +‘--local-snapshots-ok, -l’ (New in 0.6c6) + + Normally, EasyInstall prefers to only install `released' versions + of projects, not in-development ones, because such projects may not + have a currently-valid version number. So, it usually only + installs them when their ‘setup.py’ directory is explicitly passed + on the command line. + + However, if this option is used, then any in-development projects + that were installed using the ‘setup.py develop’ command, will be + used to build eggs, effectively upgrading the “in-development” + project to a snapshot release. Normally, this option is used only + in conjunction with the ‘--always-copy’ option to create a + distributable snapshot of every egg needed to run an application. + + Note that if you use this option, you must make sure that there is + a valid version number (such as an SVN revision number tag) for any + in-development projects that may be used, as otherwise EasyInstall + may not be able to tell what version of the project is “newer” when + future installations or upgrades are attempted. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0370 + + +File: setuptools.info, Node: Custom Installation Locations, Next: Package Index “API”, Prev: Command-Line Options, Up: Reference Manual + +8.3.2.3 Custom Installation Locations +..................................... + +By default, EasyInstall installs python packages into Python’s main +‘site-packages’ directory, and manages them using a custom ‘.pth’ file +in that same directory. + +Very often though, a user or developer wants ‘easy_install’ to install +and manage python packages in an alternative location, usually for one +of 3 reasons: + + 1. They don’t have access to write to the main Python site-packages + directory. + + 2. They want a user-specific stash of packages, that is not visible to + other users. + + 3. They want to isolate a set of packages to a specific python + application, usually to minimize the possibility of version + conflicts. + +Historically, there have been many approaches to achieve custom +installation. The following section lists only the easiest and most +relevant approaches (1). + +*note Use the "–user" option: 104. + +*note Use the "–user" option and customize "PYTHONUSERBASE": 105. + +*note Use "virtualenv": 106. + +* Menu: + +* Use the “–user” option:: +* Use the “–user” option and customize “PYTHONUSERBASE”:: +* Use “virtualenv”:: + + ---------- Footnotes ---------- + + (1) (1) There are older ways to achieve custom installation using +various ‘easy_install’ and ‘setup.py install’ options, combined with +‘PYTHONPATH’ and/or ‘PYTHONUSERBASE’ alterations, but all of these are +effectively deprecated by the User scheme brought in by PEP-370 +(http://www.python.org/dev/peps/pep-0370/). + + +File: setuptools.info, Node: Use the “–user” option, Next: Use the “–user” option and customize “PYTHONUSERBASE”, Up: Custom Installation Locations + +8.3.2.4 Use the “–user” option +.............................. + +Python provides a User scheme for installation, which means that all +python distributions support an alternative install location that is +specific to a user (1). The Default location for each OS is explained +in the python documentation for the ‘site.USER_BASE’ variable. This +mode of installation can be turned on by specifying the ‘--user’ option +to ‘setup.py install’ or ‘easy_install’. This approach serves the need +to have a user-specific stash of packages. + + ---------- Footnotes ---------- + + (1) (3) Prior to the User scheme, there was the Home scheme, which is +still available, but requires more effort than the User scheme to get +packages recognized. + + +File: setuptools.info, Node: Use the “–user” option and customize “PYTHONUSERBASE”, Next: Use “virtualenv”, Prev: Use the “–user” option, Up: Custom Installation Locations + +8.3.2.5 Use the “–user” option and customize “PYTHONUSERBASE” +............................................................. + +The User scheme install location can be customized by setting the +‘PYTHONUSERBASE’ environment variable, which updates the value of +‘site.USER_BASE’. To isolate packages to a specific application, simply +set the OS environment of that application to a specific value of +‘PYTHONUSERBASE’, that contains just those packages. + + +File: setuptools.info, Node: Use “virtualenv”, Prev: Use the “–user” option and customize “PYTHONUSERBASE”, Up: Custom Installation Locations + +8.3.2.6 Use “virtualenv” +........................ + +“virtualenv” is a 3rd-party python package that effectively “clones” a +python installation, thereby creating an isolated location to install +packages. The evolution of “virtualenv” started before the existence of +the User installation scheme. “virtualenv” provides a version of +‘easy_install’ that is scoped to the cloned python install and is used +in the normal way. “virtualenv” does offer various features that the +User installation scheme alone does not provide, e.g. the ability to +hide the main python site-packages. + +Please refer to the virtualenv(1) documentation for more details. + + ---------- Footnotes ---------- + + (1) https://pypi.org/project/virtualenv/ + + +File: setuptools.info, Node: Package Index “API”, Prev: Custom Installation Locations, Up: Reference Manual + +8.3.2.7 Package Index “API” +........................... + +Custom package indexes (and PyPI) must follow the following rules for +EasyInstall to be able to look up and download packages: + + 1. Except where stated otherwise, “pages” are HTML or XHTML, and + “links” refer to ‘href’ attributes. + + 2. Individual project version pages’ URLs must be of the form + ‘base/projectname/version’, where ‘base’ is the package index’s + base URL. + + 3. Omitting the ‘/version’ part of a project page’s URL (but keeping + the trailing ‘/’) should result in a page that is either: + + a. The single active version of that project, as though the + version had been explicitly included, OR + + b. A page with links to all of the active version pages for that + project. + + 4. Individual project version pages should contain direct links to + downloadable distributions where possible. It is explicitly + permitted for a project’s “long_description” to include URLs, and + these should be formatted as HTML links by the package index, as + EasyInstall does no special processing to identify what parts of a + page are index-specific and which are part of the project’s + supplied description. + + 5. Where available, MD5 information should be added to download URLs + by appending a fragment identifier of the form ‘#md5=...’, where + ‘...’ is the 32-character hex MD5 digest. EasyInstall will verify + that the downloaded file’s MD5 digest matches the given value. + + 6. Individual project version pages should identify any “homepage” or + “download” URLs using ‘rel="homepage"’ and ‘rel="download"’ + attributes on the HTML elements linking to those URLs. Use of + these attributes will cause EasyInstall to always follow the + provided links, unless it can be determined by inspection that they + are downloadable distributions. If the links are not to + downloadable distributions, they are retrieved, and if they are + HTML, they are scanned for download links. They are `not' scanned + for additional “homepage” or “download” links, as these are only + processed for pages that are part of a package index site. + + 7. The root URL of the index, if retrieved with a trailing ‘/’, must + result in a page containing links to `all' projects’ active version + pages. + + (Note: This requirement is a workaround for the absence of + case-insensitive ‘safe_name()’ matching of project names in URL + paths. If project names are matched in this fashion (e.g. via the + PyPI server, mod_rewrite, or a similar mechanism), then it is not + necessary to include this all-packages listing page.) + + 8. If a package index is accessed via a ‘file://’ URL, then + EasyInstall will automatically use ‘index.html’ files, if present, + when trying to read a directory with a trailing ‘/’ on the URL. + + +File: setuptools.info, Node: Porting from Distutils, Next: “Eggsecutable” Scripts, Prev: Easy Install, Up: Guides on backward compatibility & deprecated practice + +8.4 Porting from Distutils +========================== + +Setuptools and the PyPA have a stated goal(1) to make Setuptools the +reference API for distutils. + +Since the 49.1.2 release, Setuptools includes a local, vendored copy of +distutils (from late copies of CPython) that is disabled by default. To +enable the use of this copy of distutils when invoking setuptools, set +the enviroment variable: + + SETUPTOOLS_USE_DISTUTILS=local + +This behavior is planned to become the default. + +* Menu: + +* Prefer Setuptools:: + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging-problems/issues/127 + + +File: setuptools.info, Node: Prefer Setuptools, Up: Porting from Distutils + +8.4.1 Prefer Setuptools +----------------------- + +As Distutils is deprecated, any usage of functions or objects from +distutils is similarly discouraged, and Setuptools aims to replace or +deprecate all such uses. This section describes the recommended +replacements. + +‘distutils.core.setup’ → ‘setuptools.setup’ + +‘distutils.cmd.Command’ → ‘setuptools.Command’ + +‘distutils.log’ → (no replacement yet) + +‘distutils.version.*’ → ‘packaging.version.*’ + +If a project relies on uses of ‘distutils’ that do not have a suitable +replacement above, please search the Setuptools issue tracker(1) and +file a request, describing the use-case so that Setuptools’ maintainers +can investigate. Please provide enough detail to help the maintainers +understand how distutils is used, what value it provides, and why that +behavior should be supported. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/ + + +File: setuptools.info, Node: “Eggsecutable” Scripts, Prev: Porting from Distutils, Up: Guides on backward compatibility & deprecated practice + +8.5 “Eggsecutable” Scripts +========================== + +Deprecated since version 45.3.0. + +Occasionally, there are situations where it’s desirable to make an +‘.egg’ file directly executable. You can do this by including an entry +point such as the following: + + setup( + # other arguments here... + entry_points={ + "setuptools.installation": [ + "eggsecutable = my_package.some_module:main_func", + ] + } + ) + +Any eggs built from the above setup script will include a short +executable prelude that imports and calls ‘main_func()’ from +‘my_package.some_module’. The prelude can be run on Unix-like platforms +(including Mac and Linux) by invoking the egg with ‘/bin/sh’, or by +enabling execute permissions on the ‘.egg’ file. For the executable +prelude to run, the appropriate version of Python must be available via +the ‘PATH’ environment variable, under its “long” name. That is, if the +egg is built for Python 2.3, there must be a ‘python2.3’ executable +present in a directory on ‘PATH’. + +IMPORTANT NOTE: Eggs with an “eggsecutable” header cannot be renamed, or +invoked via symlinks. They `must' be invoked using their original +filename, in order to ensure that, once running, ‘pkg_resources’ will +know what project and version is in use. The header script will check +this and exit with an error if the ‘.egg’ file has been renamed or is +invoked via a symlink that changes its base name. + + +File: setuptools.info, Node: History<2>, Next: Credits, Prev: Guides on backward compatibility & deprecated practice, Up: Top + +9 History +********* + +* Menu: + +* v50.3.2: v50 3 2. +* v50.3.1: v50 3 1. +* v50.3.0: v50 3 0. +* v50.2.0: v50 2 0. +* v50.1.0: v50 1 0. +* v50.0.3: v50 0 3. +* v50.0.2: v50 0 2. +* v50.0.1: v50 0 1. +* v50.0.0: v50 0 0. +* v49.6.0: v49 6 0. +* v49.5.0: v49 5 0. +* v49.4.0: v49 4 0. +* v49.3.2: v49 3 2. +* v49.3.1: v49 3 1. +* v49.3.0: v49 3 0. +* v49.2.1: v49 2 1. +* v49.2.0: v49 2 0. +* v49.1.3: v49 1 3. +* v49.1.2: v49 1 2. +* v49.1.1: v49 1 1. +* v49.0.1: v49 0 1. +* v49.1.0: v49 1 0. +* v49.0.0: v49 0 0. +* v48.0.0: v48 0 0. +* v47.3.2: v47 3 2. +* v47.3.1: v47 3 1. +* v47.3.0: v47 3 0. +* v47.2.0: v47 2 0. +* v47.1.1: v47 1 1. +* v44.1.1: v44 1 1. +* v47.1.0: v47 1 0. +* v47.0.0: v47 0 0. +* v46.4.0: v46 4 0. +* v46.3.1: v46 3 1. +* v46.3.0: v46 3 0. +* v46.2.0: v46 2 0. +* v46.1.3: v46 1 3. +* v46.1.2: v46 1 2. +* v46.1.1: v46 1 1. +* v46.1.0: v46 1 0. +* v44.1.0: v44 1 0. +* v46.0.0: v46 0 0. +* v45.3.0: v45 3 0. +* v45.2.0: v45 2 0. +* v45.1.0: v45 1 0. +* v45.0.0: v45 0 0. +* v44.0.0: v44 0 0. +* v43.0.0: v43 0 0. +* v42.0.2: v42 0 2. +* v42.0.1: v42 0 1. +* v42.0.0: v42 0 0. +* v41.6.0: v41 6 0. +* v41.5.1: v41 5 1. +* v41.5.0: v41 5 0. +* v41.4.0: v41 4 0. +* v41.3.0: v41 3 0. +* v41.2.0: v41 2 0. +* v41.1.0: v41 1 0. +* v41.0.1: v41 0 1. +* v41.0.0: v41 0 0. +* v40.9.0: v40 9 0. +* v40.8.0: v40 8 0. +* v40.7.3: v40 7 3. +* v40.7.2: v40 7 2. +* v40.7.1: v40 7 1. +* v40.7.0: v40 7 0. +* v40.6.3: v40 6 3. +* v40.6.2: v40 6 2. +* v40.6.1: v40 6 1. +* v40.6.0: v40 6 0. +* v40.5.0: v40 5 0. +* v40.4.3: v40 4 3. +* v40.4.2: v40 4 2. +* v40.4.1: v40 4 1. +* v40.4.0: v40 4 0. +* v40.3.0: v40 3 0. +* v40.2.0: v40 2 0. +* v40.1.1: v40 1 1. +* v40.1.0: v40 1 0. +* v40.0.0: v40 0 0. +* v39.2.0: v39 2 0. +* v39.1.0: v39 1 0. +* v39.0.1: v39 0 1. +* v39.0.0: v39 0 0. +* v38.7.0: v38 7 0. +* v38.6.1: v38 6 1. +* v38.6.0: v38 6 0. +* v38.5.2: v38 5 2. +* v38.5.1: v38 5 1. +* v38.5.0: v38 5 0. +* v38.4.1: v38 4 1. +* v38.4.0: v38 4 0. +* v38.3.0: v38 3 0. +* v38.2.5: v38 2 5. +* v38.2.4: v38 2 4. +* v38.2.3: v38 2 3. +* v38.2.2: v38 2 2. +* v38.2.1: v38 2 1. +* v38.2.0: v38 2 0. +* v38.1.0: v38 1 0. +* v38.0.0: v38 0 0. +* v37.0.0: v37 0 0. +* v36.8.0: v36 8 0. +* v36.7.3: v36 7 3. +* v36.7.2: v36 7 2. +* v36.7.1: v36 7 1. +* v36.7.0: v36 7 0. +* v36.6.1: v36 6 1. +* v36.6.0: v36 6 0. +* v36.5.0: v36 5 0. +* v36.4.0: v36 4 0. +* v36.3.0: v36 3 0. +* v36.2.7: v36 2 7. +* v36.2.6: v36 2 6. +* v36.2.5: v36 2 5. +* v36.2.4: v36 2 4. +* v36.2.3: v36 2 3. +* v36.2.2: v36 2 2. +* v36.2.1: v36 2 1. +* v36.2.0: v36 2 0. +* v36.1.1: v36 1 1. +* v36.1.0: v36 1 0. +* v36.0.1: v36 0 1. +* v36.0.0: v36 0 0. +* v35.0.2: v35 0 2. +* v35.0.1: v35 0 1. +* v35.0.0: v35 0 0. +* v34.4.1: v34 4 1. +* v34.4.0: v34 4 0. +* v34.3.3: v34 3 3. +* v34.3.2: v34 3 2. +* v34.3.1: v34 3 1. +* v34.3.0: v34 3 0. +* v34.2.0: v34 2 0. +* v34.1.1: v34 1 1. +* v34.1.0: v34 1 0. +* v34.0.3: v34 0 3. +* v34.0.2: v34 0 2. +* v34.0.1: v34 0 1. +* v34.0.0: v34 0 0. +* v33.1.1: v33 1 1. +* v33.1.0: v33 1 0. +* v33.0.0: v33 0 0. +* v32.3.1: v32 3 1. +* v32.3.0: v32 3 0. +* v32.2.0: v32 2 0. +* v32.1.3: v32 1 3. +* v32.1.2: v32 1 2. +* v32.1.1: v32 1 1. +* v32.1.0: v32 1 0. +* v32.0.0: v32 0 0. +* v31.0.1: v31 0 1. +* v31.0.0: v31 0 0. +* v30.4.0: v30 4 0. +* v30.3.0: v30 3 0. +* v30.2.1: v30 2 1. +* v30.2.0: v30 2 0. +* v30.1.0: v30 1 0. +* v30.0.0: v30 0 0. +* v29.0.1: v29 0 1. +* v29.0.0: v29 0 0. +* v28.8.0: v28 8 0. +* v28.7.1: v28 7 1. +* v28.7.0: v28 7 0. +* v28.6.1: v28 6 1. +* v28.6.0: v28 6 0. +* v28.5.0: v28 5 0. +* v28.4.0: v28 4 0. +* v28.3.0: v28 3 0. +* v28.1.0: v28 1 0. +* v28.0.0: v28 0 0. +* v27.3.1: v27 3 1. +* v27.3.0: v27 3 0. +* v27.2.0: v27 2 0. +* v27.1.2: v27 1 2. +* v27.1.1: v27 1 1. +* v27.1.0: v27 1 0. +* v27.0.0: v27 0 0. +* v26.1.1: v26 1 1. +* v26.1.0: v26 1 0. +* v26.0.0: v26 0 0. +* v25.4.0: v25 4 0. +* v25.3.0: v25 3 0. +* v25.2.0: v25 2 0. +* v25.1.6: v25 1 6. +* v25.1.5: v25 1 5. +* v25.1.4: v25 1 4. +* v25.1.3: v25 1 3. +* v25.1.2: v25 1 2. +* v25.1.1: v25 1 1. +* v25.1.0: v25 1 0. +* v25.0.2: v25 0 2. +* v25.0.1: v25 0 1. +* v25.0.0: v25 0 0. +* v24.3.1: v24 3 1. +* v24.3.0: v24 3 0. +* v24.2.1: v24 2 1. +* v24.2.0: v24 2 0. +* v24.1.1: v24 1 1. +* v24.1.0: v24 1 0. +* v24.0.3: v24 0 3. +* v24.0.2: v24 0 2. +* v24.0.1: v24 0 1. +* v24.0.0: v24 0 0. +* v23.2.1: v23 2 1. +* v23.1.0: v23 1 0. +* v23.0.0: v23 0 0. +* v22.0.5: v22 0 5. +* v22.0.4: v22 0 4. +* v22.0.3: v22 0 3. +* v22.0.2: v22 0 2. +* v22.0.1: v22 0 1. +* v22.0.0: v22 0 0. +* v21.2.2: v21 2 2. +* v21.2.1: v21 2 1. +* v21.2.0: v21 2 0. +* v21.1.0: v21 1 0. +* v21.0.0: v21 0 0. +* v20.10.0: v20 10 0. +* v20.9.0: v20 9 0. +* v20.8.1: v20 8 1. +* v20.8.0: v20 8 0. +* v20.7.0: v20 7 0. +* v20.6.8: v20 6 8. +* v20.6.7: v20 6 7. +* v20.6.6: v20 6 6. +* v20.6.0: v20 6 0. +* 20.5: 20 5. +* 20.4: 20 4. +* 20.3.1: 20 3 1. +* 20.3: 20 3. +* 20.2.2: 20 2 2. +* 20.2.1: 20 2 1. +* 20.2: 20 2. +* 20.1.1: 20 1 1. +* 20.1: 20 1. +* 20.0: 20 0. +* 19.7: 19 7. +* 19.6.2: 19 6 2. +* 19.6.1: 19 6 1. +* 19.6: 19 6. +* 19.5: 19 5. +* 19.4.1: 19 4 1. +* 19.4: 19 4. +* 19.3: 19 3. +* 19.2: 19 2. +* 19.1.1: 19 1 1. +* 19.1: 19 1. +* 19.0: 19 0. +* 18.8.1: 18 8 1. +* 18.8: 18 8. +* 18.7.1: 18 7 1. +* 18.7: 18 7. +* 18.6.1: 18 6 1. +* 18.6: 18 6. +* 18.5: 18 5. +* 18.4: 18 4. +* 18.3.2: 18 3 2. +* 18.3.1: 18 3 1. +* 18.3: 18 3. +* 18.2: 18 2. +* 18.1: 18 1. +* 18.0.1: 18 0 1. +* 18.0: 18 0. +* 17.1.1: 17 1 1. +* 17.1: 17 1. +* 17.0: 17 0. +* 16.0: 16 0. +* 15.2: 15 2. +* 15.1: 15 1. +* 15.0: 15 0. +* 14.3.1: 14 3 1. +* 14.3: 14 3. +* 14.2: 14 2. +* 14.1.1: 14 1 1. +* 14.1: 14 1. +* 14.0: 14 0. +* 13.0.2: 13 0 2. +* 13.0.1: 13 0 1. +* 13.0: 13 0. +* 12.4: 12 4. +* 12.3: 12 3. +* 12.2: 12 2. +* 12.1: 12 1. +* 12.0.5: 12 0 5. +* 12.0.4: 12 0 4. +* 12.0.3: 12 0 3. +* 12.0.2: 12 0 2. +* 12.0.1: 12 0 1. +* 12.0: 12 0. +* 11.3.1: 11 3 1. +* 11.3: 11 3. +* 11.2: 11 2. +* 11.1: 11 1. +* 11.0: 11 0. +* 10.2.1: 10 2 1. +* 10.2: 10 2. +* 10.1: 10 1. +* 10.0.1: 10 0 1. +* 10.0: 10 0. +* 9.1: 9 1. +* 9.0.1: 9 0 1. +* 9.0: 9 0. +* 8.4: 8 4. +* 8.3: 8 3. +* 8.2.1: 8 2 1. +* 8.2: 8 2. +* 8.1: 8 1. +* 8.0.4: 8 0 4. +* 8.0.3: 8 0 3. +* 8.0.2: 8 0 2. +* 8.0.1: 8 0 1. +* 8.0: 8 0. +* 7.0: 7 0. +* 6.1: 6 1. +* 6.0.2: 6 0 2. +* 6.0.1: 6 0 1. +* 6.0: 6 0. +* 5.8: 5 8. +* 5.7: 5 7. +* 5.6: 5 6. +* 5.5.1: 5 5 1. +* 5.5: 5 5. +* 5.4.2: 5 4 2. +* 5.4.1: 5 4 1. +* 5.4: 5 4. +* 5.3: 5 3. +* 5.2: 5 2. +* 5.1: 5 1. +* 5.0.2: 5 0 2. +* 5.0.1: 5 0 1. +* 5.0: 5 0. +* 3.7.1 and 3.8.1 and 4.0.1: 3 7 1 and 3 8 1 and 4 0 1. +* 4.0: 4 0. +* 3.8: 3 8. +* 3.7: 3 7. +* 3.6: 3 6. +* 3.5.2: 3 5 2. +* 3.5.1: 3 5 1. +* 3.5: 3 5. +* 3.4.4: 3 4 4. +* 3.4.3: 3 4 3. +* 3.4.2: 3 4 2. +* 3.4.1: 3 4 1. +* 3.4: 3 4. +* 3.3: 3 3. +* 3.2: 3 2. +* 3.1: 3 1. +* 3.0.2: 3 0 2. +* 3.0.1: 3 0 1. +* 3.0: 3 0. +* 2.2: 2 2. +* 2.1.2: 2 1 2. +* 2.1.1: 2 1 1. +* 2.1: 2 1. +* 2.0.2: 2 0 2. +* 2.0.1: 2 0 1. +* 2.0: 2 0. +* 1.4.2: 1 4 2. +* 1.4.1: 1 4 1. +* 1.4: 1 4. +* 1.3.2: 1 3 2. +* 1.3.1: 1 3 1. +* 1.3: 1 3. +* 1.2: 1 2. +* 1.1.7: 1 1 7. +* 1.1.6: 1 1 6. +* 1.1.5: 1 1 5. +* 1.1.4: 1 1 4. +* 1.1.3: 1 1 3. +* 1.1.2: 1 1 2. +* 1.1.1: 1 1 1. +* 1.1: 1 1. +* 1.0: 1 0. +* 0.9.8: 0 9 8. +* 0.9.7: 0 9 7. +* 0.9.6: 0 9 6. +* 0.9.5: 0 9 5. +* 0.9.4: 0 9 4. +* 0.9.3: 0 9 3. +* 0.9.2: 0 9 2. +* 0.9.1: 0 9 1. +* 0.9: 0 9. +* 0.8: 0 8. +* 0.7.8: 0 7 8. +* 0.7.7: 0 7 7. +* 0.7.6: 0 7 6. +* 0.7.5: 0 7 5. +* 0.7.4: 0 7 4. +* 0.7.3: 0 7 3. +* 0.7.2: 0 7 2. +* 0.7.1: 0 7 1. +* 0.7: 0 7. +* 0.7b4: 0 7b4. +* 0.6.49: 0 6 49. +* 0.6.48: 0 6 48. +* 0.6.47: 0 6 47. +* 0.6.46: 0 6 46. +* 0.6.45: 0 6 45. +* 0.6.44: 0 6 44. +* 0.6.43: 0 6 43. +* 0.6.42: 0 6 42. +* 0.6.41: 0 6 41. +* 0.6.40: 0 6 40. +* 0.6.39: 0 6 39. +* 0.6.38: 0 6 38. +* 0.6.37: 0 6 37. +* 0.6.36: 0 6 36. +* 0.6.35: 0 6 35. +* 0.6.34: 0 6 34. +* 0.6.33: 0 6 33. +* 0.6.32: 0 6 32. +* 0.6.31: 0 6 31. +* 0.6.30: 0 6 30. +* 0.6.29: 0 6 29. +* 0.6.28: 0 6 28. +* 0.6.27: 0 6 27. +* 0.6.26: 0 6 26. +* 0.6.25: 0 6 25. +* 0.6.24: 0 6 24. +* 0.6.23: 0 6 23. +* 0.6.21: 0 6 21. +* 0.6.20: 0 6 20. +* 0.6.19: 0 6 19. +* 0.6.18: 0 6 18. +* 0.6.17: 0 6 17. +* 0.6.16: 0 6 16. +* 0.6.15: 0 6 15. +* 0.6.14: 0 6 14. +* 0.6.13: 0 6 13. +* 0.6.12: 0 6 12. +* 0.6.11: 0 6 11. +* 0.6.10: 0 6 10. +* 0.6.9: 0 6 9. +* 0.6.8: 0 6 8. +* 0.6.7: 0 6 7. +* 0.6.6: 0 6 6. +* 0.6.5: 0 6 5. +* 0.6.4: 0 6 4. +* 0.6.3: 0 6 3. +* 0.6.2: 0 6 2. +* 0.6.1: 0 6 1. +* 0.6: 0 6. +* 0.6c9: 0 6c9. +* 0.6c7: 0 6c7. +* 0.6c6: 0 6c6. +* 0.6c5: 0 6c5. +* 0.6c4: 0 6c4. +* 0.6c3: 0 6c3. +* 0.6c2: 0 6c2. +* 0.6c1: 0 6c1. +* 0.6b4: 0 6b4. +* 0.6b3: 0 6b3. +* 0.6b2: 0 6b2. +* 0.6b1: 0 6b1. +* 0.6a11: 0 6a11. +* 0.6a10: 0 6a10. +* 0.6a9: 0 6a9. +* 0.6a8: 0 6a8. +* 0.6a7: 0 6a7. +* 0.6a6: 0 6a6. +* 0.6a5: 0 6a5. +* 0.6a3: 0 6a3. +* 0.6a2: 0 6a2. +* 0.6a1: 0 6a1. +* 0.5a12: 0 5a12. +* 0.5a11: 0 5a11. +* 0.5a10: 0 5a10. +* 0.5a9: 0 5a9. +* 0.5a8: 0 5a8. +* 0.5a7: 0 5a7. +* 0.5a6: 0 5a6. +* 0.5a5: 0 5a5. +* 0.5a4: 0 5a4. +* 0.5a3: 0 5a3. +* 0.5a2: 0 5a2. +* 0.5a1: 0 5a1. +* 0.4a4: 0 4a4. +* 0.4a3: 0 4a3. +* 0.4a2: 0 4a2. +* 0.4a1: 0 4a1. +* 0.3a4: 0 3a4. +* 0.3a3: 0 3a3. +* 0.3a2: 0 3a2. +* 0.3a1: 0 3a1. + + +File: setuptools.info, Node: v50 3 2, Next: v50 3 1, Up: History<2> + +9.1 v50.3.2 +=========== + +17 Oct 2020 + +* Menu: + +* Documentation changes:: +* Misc:: + + +File: setuptools.info, Node: Documentation changes, Next: Misc, Up: v50 3 2 + +9.1.1 Documentation changes +--------------------------- + + * #2394(1): Extended towncrier news template to include change note + categories. This allows to see what types of changes a given + version introduces – by @webknjaz(2) + + * #2427(3): Started enforcing strict syntax and reference validation + in the Sphinx docs – by @webknjaz(4) + + * #2428(5): Removed redundant Sphinx ‘Makefile’ support – by + @webknjaz(6) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2394 + + (2) https://github.com/sponsors/webknjaz + + (3) https://github.com/pypa/setuptools/issues/2427 + + (4) https://github.com/sponsors/webknjaz + + (5) https://github.com/pypa/setuptools/issues/2428 + + (6) https://github.com/sponsors/webknjaz + + +File: setuptools.info, Node: Misc, Prev: Documentation changes, Up: v50 3 2 + +9.1.2 Misc +---------- + + * #2401(1): Enabled test results reporting in AppVeyor CI – by + @webknjaz(2) + + * #2420(3): Replace Python 3.9.0 beta with 3.9.0 final on GitHub + Actions. + + * #2421(4): Python 3.9 Trove classifier got added to the dist + metadata – by @webknjaz(5) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2401 + + (2) https://github.com/sponsors/webknjaz + + (3) https://github.com/pypa/setuptools/issues/2420 + + (4) https://github.com/pypa/setuptools/issues/2421 + + (5) https://github.com/sponsors/webknjaz + + +File: setuptools.info, Node: v50 3 1, Next: v50 3 0, Prev: v50 3 2, Up: History<2> + +9.2 v50.3.1 +=========== + +14 Oct 2020 + +* Menu: + +* Documentation changes: Documentation changes<2>. +* Misc: Misc<2>. + + +File: setuptools.info, Node: Documentation changes<2>, Next: Misc<2>, Up: v50 3 1 + +9.2.1 Documentation changes +--------------------------- + + * #2093(1): Finalized doc revamp. + + * #2097(2): doc: simplify index and group deprecated files + + * #2102(3): doc overhaul step 2: break main doc into multiple + sections + + * #2111(4): doc overhaul step 3: update userguide + + * #2395(5): Added a ‘:user:’ role to Sphinx config – by @webknjaz(6) + + * #2395(7): Added an illustrative explanation about the change notes + to fragments dir – by @webknjaz(8) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2093 + + (2) https://github.com/pypa/setuptools/issues/2097 + + (3) https://github.com/pypa/setuptools/issues/2102 + + (4) https://github.com/pypa/setuptools/issues/2111 + + (5) https://github.com/pypa/setuptools/issues/2395 + + (6) https://github.com/sponsors/webknjaz + + (7) https://github.com/pypa/setuptools/issues/2395 + + (8) https://github.com/sponsors/webknjaz + + +File: setuptools.info, Node: Misc<2>, Prev: Documentation changes<2>, Up: v50 3 1 + +9.2.2 Misc +---------- + + * #2379(1): Travis CI test suite now tests against PPC64. + + * #2413(2): Suppress EOF errors (and other exceptions) when importing + lib2to3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2379 + + (2) https://github.com/pypa/setuptools/issues/2413 + + +File: setuptools.info, Node: v50 3 0, Next: v50 2 0, Prev: v50 3 1, Up: History<2> + +9.3 v50.3.0 +=========== + +05 Sep 2020 + +* Menu: + +* Changes:: + + +File: setuptools.info, Node: Changes, Up: v50 3 0 + +9.3.1 Changes +------------- + + * #2368(1): In distutils, restore support for monkeypatched + CCompiler.spawn per pypa/distutils#15(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2368 + + (2) https://github.com/pypa/distutils/issues/15 + + +File: setuptools.info, Node: v50 2 0, Next: v50 1 0, Prev: v50 3 0, Up: History<2> + +9.4 v50.2.0 +=========== + +04 Sep 2020 + +* Menu: + +* Changes: Changes<2>. + + +File: setuptools.info, Node: Changes<2>, Up: v50 2 0 + +9.4.1 Changes +------------- + + * #2355(1): When pip is imported as part of a build, leave distutils + patched. + + * #2380(2): There are some setuptools specific changes in the + ‘setuptools.command.bdist_rpm’ module that are no longer needed, + because they are part of the ‘bdist_rpm’ module in distutils in + Python 3.5.0. Therefore, code was removed from + ‘setuptools.command.bdist_rpm’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2355 + + (2) https://github.com/pypa/setuptools/issues/2380 + + +File: setuptools.info, Node: v50 1 0, Next: v50 0 3, Prev: v50 2 0, Up: History<2> + +9.5 v50.1.0 +=========== + +02 Sep 2020 + +* Menu: + +* Changes: Changes<3>. + + +File: setuptools.info, Node: Changes<3>, Up: v50 1 0 + +9.5.1 Changes +------------- + + * #2350(1): Setuptools reverts using the included distutils by + default. Platform maintainers and system integrators and others + are `strongly' encouraged to set ‘SETUPTOOLS_USE_DISTUTILS=local’ + to help identify and work through the reported issues with + distutils adoption, mainly to file issues and pull requests with + pypa/distutils such that distutils performs as needed across every + supported environment. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2350 + + +File: setuptools.info, Node: v50 0 3, Next: v50 0 2, Prev: v50 1 0, Up: History<2> + +9.6 v50.0.3 +=========== + +01 Sep 2020 + +* Menu: + +* Misc: Misc<3>. + + +File: setuptools.info, Node: Misc<3>, Up: v50 0 3 + +9.6.1 Misc +---------- + + * #2363(1): Restore link_libpython support on Python 3.7 and earlier + (see pypa/distutils#9(2)). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2363 + + (2) https://github.com/pypa/distutils/issues/9 + + +File: setuptools.info, Node: v50 0 2, Next: v50 0 1, Prev: v50 0 3, Up: History<2> + +9.7 v50.0.2 +=========== + +01 Sep 2020 + +* Menu: + +* Misc: Misc<4>. + + +File: setuptools.info, Node: Misc<4>, Up: v50 0 2 + +9.7.1 Misc +---------- + + * #2352(1): In distutils hack, use absolute import rather than + relative to avoid bpo-30876(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2352 + + (2) http://bugs.python.org/issue30876 + + +File: setuptools.info, Node: v50 0 1, Next: v50 0 0, Prev: v50 0 2, Up: History<2> + +9.8 v50.0.1 +=========== + +01 Sep 2020 + +* Menu: + +* Misc: Misc<5>. + + +File: setuptools.info, Node: Misc<5>, Up: v50 0 1 + +9.8.1 Misc +---------- + + * #2357(1): Restored Python 3.5 support in distutils.util for missing + ‘subprocess._optim_args_from_interpreter_flags’. + + * #2358(2): Restored AIX support on Python 3.8 and earlier. + + * #2361(3): Add Python 3.10 support to _distutils_hack. Get the + ‘Loader’ abstract class from importlib.abc rather than + importlib.util.abc (alias removed in Python 3.10). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2357 + + (2) https://github.com/pypa/setuptools/issues/2358 + + (3) https://github.com/pypa/setuptools/issues/2361 + + +File: setuptools.info, Node: v50 0 0, Next: v49 6 0, Prev: v50 0 1, Up: History<2> + +9.9 v50.0.0 +=========== + +20 Aug 2020 + +* Menu: + +* Breaking Changes:: +* Changes: Changes<4>. + + +File: setuptools.info, Node: Breaking Changes, Next: Changes<4>, Up: v50 0 0 + +9.9.1 Breaking Changes +---------------------- + + * #2232(1): Once again, Setuptools overrides the stdlib distutils on + import. For environments or invocations where this behavior is + undesirable, users are provided with a temporary escape hatch. If + the environment variable ‘SETUPTOOLS_USE_DISTUTILS’ is set to + ‘stdlib’, Setuptools will fall back to the legacy behavior. Use of + this escape hatch is discouraged, but it is provided to ease the + transition while proper fixes for edge cases can be addressed. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2232 + + +File: setuptools.info, Node: Changes<4>, Prev: Breaking Changes, Up: v50 0 0 + +9.9.2 Changes +------------- + + * #2334(1): In MSVC module, refine text in error message. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2334 + + +File: setuptools.info, Node: v49 6 0, Next: v49 5 0, Prev: v50 0 0, Up: History<2> + +9.10 v49.6.0 +============ + +13 Aug 2020 + +* Menu: + +* Changes: Changes<5>. + + +File: setuptools.info, Node: Changes<5>, Up: v49 6 0 + +9.10.1 Changes +-------------- + + * #2129(1): In pkg_resources, no longer detect any pathname ending in + .egg as a Python egg. Now the path must be an unpacked egg or a + zip file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2129 + + +File: setuptools.info, Node: v49 5 0, Next: v49 4 0, Prev: v49 6 0, Up: History<2> + +9.11 v49.5.0 +============ + +13 Aug 2020 + +* Menu: + +* Changes: Changes<6>. + + +File: setuptools.info, Node: Changes<6>, Up: v49 5 0 + +9.11.1 Changes +-------------- + + * #2306(1): When running as a PEP 517(2) backend, setuptools does not + try to install ‘setup_requires’ itself. They are reported as build + requirements for the frontend to install. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2306 + + (2) https://www.python.org/dev/peps/pep-0517/ + + +File: setuptools.info, Node: v49 4 0, Next: v49 3 2, Prev: v49 5 0, Up: History<2> + +9.12 v49.4.0 +============ + +13 Aug 2020 + +* Menu: + +* Changes: Changes<7>. + + +File: setuptools.info, Node: Changes<7>, Up: v49 4 0 + +9.12.1 Changes +-------------- + + * #2310(1): Updated vendored packaging version to 20.4. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2310 + + +File: setuptools.info, Node: v49 3 2, Next: v49 3 1, Prev: v49 4 0, Up: History<2> + +9.13 v49.3.2 +============ + +12 Aug 2020 + +* Menu: + +* Documentation changes: Documentation changes<3>. +* Misc: Misc<6>. + + +File: setuptools.info, Node: Documentation changes<3>, Next: Misc<6>, Up: v49 3 2 + +9.13.1 Documentation changes +---------------------------- + + * #2300(1): Improve the ‘safe_version’ function documentation + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2300 + + +File: setuptools.info, Node: Misc<6>, Prev: Documentation changes<3>, Up: v49 3 2 + +9.13.2 Misc +----------- + + * #2297(1): Once again, in stubs prefer exec_module to the deprecated + load_module. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2297 + + +File: setuptools.info, Node: v49 3 1, Next: v49 3 0, Prev: v49 3 2, Up: History<2> + +9.14 v49.3.1 +============ + +10 Aug 2020 + +* Menu: + +* Changes: Changes<8>. + + +File: setuptools.info, Node: Changes<8>, Up: v49 3 1 + +9.14.1 Changes +-------------- + + * #2316(1): Removed warning when ‘distutils’ is imported before + ‘setuptools’ when ‘distutils’ replacement is not enabled. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2316 + + +File: setuptools.info, Node: v49 3 0, Next: v49 2 1, Prev: v49 3 1, Up: History<2> + +9.15 v49.3.0 +============ + +09 Aug 2020 + +* Menu: + +* Changes: Changes<9>. + + +File: setuptools.info, Node: Changes<9>, Up: v49 3 0 + +9.15.1 Changes +-------------- + + * #2259(1): Setuptools now provides a .pth file (except for editable + installs of setuptools) to the target environment to ensure that + when enabled, the setuptools-provided distutils is preferred before + setuptools has been imported (and even if setuptools is never + imported). Honors the SETUPTOOLS_USE_DISTUTILS environment + variable. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2259 + + +File: setuptools.info, Node: v49 2 1, Next: v49 2 0, Prev: v49 3 0, Up: History<2> + +9.16 v49.2.1 +============ + +02 Aug 2020 + +* Menu: + +* Misc: Misc<7>. + + +File: setuptools.info, Node: Misc<7>, Up: v49 2 1 + +9.16.1 Misc +----------- + + * #2257(1): Fixed two flaws in + distutils._msvccompiler.MSVCCompiler.spawn. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2257 + + +File: setuptools.info, Node: v49 2 0, Next: v49 1 3, Prev: v49 2 1, Up: History<2> + +9.17 v49.2.0 +============ + +12 Jul 2020 + +* Menu: + +* Changes: Changes<10>. + + +File: setuptools.info, Node: Changes<10>, Up: v49 2 0 + +9.17.1 Changes +-------------- + + * #2230(1): Now warn the user when setuptools is imported after + distutils modules have been loaded (exempting PyPy for 3.6), + directing the users of packages to import setuptools first. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2230 + + +File: setuptools.info, Node: v49 1 3, Next: v49 1 2, Prev: v49 2 0, Up: History<2> + +9.18 v49.1.3 +============ + +12 Jul 2020 + +* Menu: + +* Misc: Misc<8>. + + +File: setuptools.info, Node: Misc<8>, Up: v49 1 3 + +9.18.1 Misc +----------- + + * #2212(1): (Distutils) Allow spawn to accept environment. Avoid + monkey-patching global state. + + * #2249(2): Fix extension loading technique in stubs. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2212 + + (2) https://github.com/pypa/setuptools/issues/2249 + + +File: setuptools.info, Node: v49 1 2, Next: v49 1 1, Prev: v49 1 3, Up: History<2> + +9.19 v49.1.2 +============ + +11 Jul 2020 + +* Menu: + +* Changes: Changes<11>. + + +File: setuptools.info, Node: Changes<11>, Up: v49 1 2 + +9.19.1 Changes +-------------- + + * #2232(1): In preparation for re-enabling a local copy of distutils, + Setuptools now honors an environment variable, + SETUPTOOLS_USE_DISTUTILS. If set to ‘stdlib’ (current default), + distutils will be used from the standard library. If set to + ‘local’ (default in a imminent backward-incompatible release), the + local copy of distutils will be used. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2232 + + +File: setuptools.info, Node: v49 1 1, Next: v49 0 1, Prev: v49 1 2, Up: History<2> + +9.20 v49.1.1 +============ + +10 Jul 2020 + +* Menu: + +* Misc: Misc<9>. + + +File: setuptools.info, Node: Misc<9>, Up: v49 1 1 + +9.20.1 Misc +----------- + + * #2094(1): Removed pkg_resources.py2_warn module, which is no longer + reachable. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2094 + + +File: setuptools.info, Node: v49 0 1, Next: v49 1 0, Prev: v49 1 1, Up: History<2> + +9.21 v49.0.1 +============ + +05 Jul 2020 + +* Menu: + +* Misc: Misc<10>. + + +File: setuptools.info, Node: Misc<10>, Up: v49 0 1 + +9.21.1 Misc +----------- + + * #2228(1): Applied fix for pypa/distutils#3(2), restoring + expectation that spawn will raise a DistutilsExecError when + attempting to execute a missing file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2228 + + (2) https://github.com/pypa/distutils/issues/3 + + +File: setuptools.info, Node: v49 1 0, Next: v49 0 0, Prev: v49 0 1, Up: History<2> + +9.22 v49.1.0 +============ + +03 Jul 2020 + +* Menu: + +* Changes: Changes<12>. + + +File: setuptools.info, Node: Changes<12>, Up: v49 1 0 + +9.22.1 Changes +-------------- + + * #2228(1): Disabled distutils adoption for now while emergent issues + are addressed. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2228 + + +File: setuptools.info, Node: v49 0 0, Next: v48 0 0, Prev: v49 1 0, Up: History<2> + +9.23 v49.0.0 +============ + +03 Jul 2020 + +* Menu: + +* Breaking Changes: Breaking Changes<2>. +* Changes: Changes<13>. +* Misc: Misc<11>. + + +File: setuptools.info, Node: Breaking Changes<2>, Next: Changes<13>, Up: v49 0 0 + +9.23.1 Breaking Changes +----------------------- + + * #2165(1): Setuptools no longer installs a site.py file during + easy_install or develop installs. As a result, .eggs on PYTHONPATH + will no longer take precedence over other packages on sys.path. If + this issue affects your production environment, please reach out to + the maintainers at #2165(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2165 + + (2) https://github.com/pypa/setuptools/issues/2165 + + +File: setuptools.info, Node: Changes<13>, Next: Misc<11>, Prev: Breaking Changes<2>, Up: v49 0 0 + +9.23.2 Changes +-------------- + + * #2137(1): Removed (private) pkg_resources.RequirementParseError, + now replaced by packaging.requirements.InvalidRequirement. Kept + the name for compatibility, but users should catch + InvalidRequirement instead. + + * #2180(2): Update vendored packaging in pkg_resources to 19.2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2137 + + (2) https://github.com/pypa/setuptools/issues/2180 + + +File: setuptools.info, Node: Misc<11>, Prev: Changes<13>, Up: v49 0 0 + +9.23.3 Misc +----------- + + * #2199(1): Fix exception causes all over the codebase by using + ‘raise new_exception from old_exception’ + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2199 + + +File: setuptools.info, Node: v48 0 0, Next: v47 3 2, Prev: v49 0 0, Up: History<2> + +9.24 v48.0.0 +============ + +03 Jul 2020 + +* Menu: + +* Breaking Changes: Breaking Changes<3>. + + +File: setuptools.info, Node: Breaking Changes<3>, Up: v48 0 0 + +9.24.1 Breaking Changes +----------------------- + + * #2143(1): Setuptools adopts distutils from the Python 3.9 standard + library and no longer depends on distutils in the standard library. + When importing ‘setuptools’ or ‘setuptools.distutils_patch’, + Setuptools will expose its bundled version as a top-level + ‘distutils’ package (and unload any previously-imported top-level + distutils package), retaining the expectation that ‘distutils’’ + objects are actually Setuptools objects. To avoid getting any + legacy behavior from the standard library, projects are advised to + always “import setuptools” prior to importing anything from + distutils. This behavior happens by default when using ‘pip + install’ or ‘pep517.build’. Workflows that rely on ‘setup.py + (anything)’ will need to first ensure setuptools is imported. One + way to achieve this behavior without modifying code is to invoke + Python thus: ‘python -c "import setuptools; + exec(open('setup.py').read())" (anything)’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2143 + + +File: setuptools.info, Node: v47 3 2, Next: v47 3 1, Prev: v48 0 0, Up: History<2> + +9.25 v47.3.2 +============ + +03 Jul 2020 + +* Menu: + +* Misc: Misc<12>. + + +File: setuptools.info, Node: Misc<12>, Up: v47 3 2 + +9.25.1 Misc +----------- + + * #2071(1): Replaced references to the deprecated imp package with + references to importlib + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2071 + + +File: setuptools.info, Node: v47 3 1, Next: v47 3 0, Prev: v47 3 2, Up: History<2> + +9.26 v47.3.1 +============ + +16 Jun 2020 + +* Menu: + +* Misc: Misc<13>. + + +File: setuptools.info, Node: Misc<13>, Up: v47 3 1 + +9.26.1 Misc +----------- + + * #1973(1): Removed ‘pkg_resources.py31compat.makedirs’ in favor of + the stdlib. Use ‘os.makedirs()’ instead. + + * #2198(2): Restore ‘__requires__’ directive in easy-install wrapper + scripts. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1973 + + (2) https://github.com/pypa/setuptools/issues/2198 + + +File: setuptools.info, Node: v47 3 0, Next: v47 2 0, Prev: v47 3 1, Up: History<2> + +9.27 v47.3.0 +============ + +15 Jun 2020 + +* Menu: + +* Changes: Changes<14>. +* Misc: Misc<14>. + + +File: setuptools.info, Node: Changes<14>, Next: Misc<14>, Up: v47 3 0 + +9.27.1 Changes +-------------- + + * #2197(1): Console script wrapper for editable installs now has a + unified template and honors importlib_metadata if present for + faster script execution on older Pythons. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2197 + + +File: setuptools.info, Node: Misc<14>, Prev: Changes<14>, Up: v47 3 0 + +9.27.2 Misc +----------- + + * #2195(1): Fix broken entry points generated by easy-install (pip + editable installs). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2195 + + +File: setuptools.info, Node: v47 2 0, Next: v47 1 1, Prev: v47 3 0, Up: History<2> + +9.28 v47.2.0 +============ + +15 Jun 2020 + +* Menu: + +* Changes: Changes<15>. + + +File: setuptools.info, Node: Changes<15>, Up: v47 2 0 + +9.28.1 Changes +-------------- + + * #2194(1): Editable-installed entry points now load significantly + faster on Python versions 3.8+. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2194 + + +File: setuptools.info, Node: v47 1 1, Next: v44 1 1, Prev: v47 2 0, Up: History<2> + +9.29 v47.1.1 +============ + +28 May 2020 + +* Menu: + +* Documentation changes: Documentation changes<4>. +* Incorporate changes from v44.1.1;: Incorporate changes from v44 1 1. + + +File: setuptools.info, Node: Documentation changes<4>, Next: Incorporate changes from v44 1 1, Up: v47 1 1 + +9.29.1 Documentation changes +---------------------------- + + * #2156(1): Update mailing list pointer in developer docs + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2156 + + +File: setuptools.info, Node: Incorporate changes from v44 1 1, Prev: Documentation changes<4>, Up: v47 1 1 + +9.29.2 Incorporate changes from v44.1.1: +---------------------------------------- + + * #2158(1): Avoid loading working set during + ‘Distribution.finalize_options’ prior to invoking + ‘_install_setup_requires’, broken since v42.0.0. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2158 + + +File: setuptools.info, Node: v44 1 1, Next: v47 1 0, Prev: v47 1 1, Up: History<2> + +9.30 v44.1.1 +============ + +28 May 2020 + +* Menu: + +* Misc: Misc<15>. + + +File: setuptools.info, Node: Misc<15>, Up: v44 1 1 + +9.30.1 Misc +----------- + + * #2158(1): Avoid loading working set during + ‘Distribution.finalize_options’ prior to invoking + ‘_install_setup_requires’, broken since v42.0.0. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2158 + + +File: setuptools.info, Node: v47 1 0, Next: v47 0 0, Prev: v44 1 1, Up: History<2> + +9.31 v47.1.0 +============ + +28 May 2020 + +* Menu: + +* Changes: Changes<16>. + + +File: setuptools.info, Node: Changes<16>, Up: v47 1 0 + +9.31.1 Changes +-------------- + + * #2070(1): In wheel-to-egg conversion, use simple + pkg_resources-style namespace declaration for packages that declare + namespace_packages. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2070 + + +File: setuptools.info, Node: v47 0 0, Next: v46 4 0, Prev: v47 1 0, Up: History<2> + +9.32 v47.0.0 +============ + +28 May 2020 + +* Menu: + +* Breaking Changes: Breaking Changes<4>. +* Changes: Changes<17>. + + +File: setuptools.info, Node: Breaking Changes<4>, Next: Changes<17>, Up: v47 0 0 + +9.32.1 Breaking Changes +----------------------- + + * #2094(1): Setuptools now actively crashes under Python 2. Python + 3.5 or later is required. Users of Python 2 should use + ‘setuptools<45’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2094 + + +File: setuptools.info, Node: Changes<17>, Prev: Breaking Changes<4>, Up: v47 0 0 + +9.32.2 Changes +-------------- + + * #1700(1): Document all supported keywords by migrating the ones + from distutils. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1700 + + +File: setuptools.info, Node: v46 4 0, Next: v46 3 1, Prev: v47 0 0, Up: History<2> + +9.33 v46.4.0 +============ + +16 May 2020 + +* Menu: + +* Changes: Changes<18>. + + +File: setuptools.info, Node: Changes<18>, Up: v46 4 0 + +9.33.1 Changes +-------------- + + * #1753(1): ‘attr:’ now extracts variables through rudimentary + examination of the AST, thereby supporting modules with third-party + imports. If examining the AST fails to find the variable, ‘attr:’ + falls back to the old behavior of importing the module. Works on + Python 3 only. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1753 + + +File: setuptools.info, Node: v46 3 1, Next: v46 3 0, Prev: v46 4 0, Up: History<2> + +9.34 v46.3.1 +============ + +15 May 2020 + +No significant changes. + + +File: setuptools.info, Node: v46 3 0, Next: v46 2 0, Prev: v46 3 1, Up: History<2> + +9.35 v46.3.0 +============ + +13 May 2020 + +* Menu: + +* Changes: Changes<19>. +* Misc: Misc<16>. + + +File: setuptools.info, Node: Changes<19>, Next: Misc<16>, Up: v46 3 0 + +9.35.1 Changes +-------------- + + * #2089(1): Package index functionality no longer attempts to remove + an md5 fragment from the index URL. This functionality, added for + distribute #163(2) is no longer relevant. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2089 + + (2) https://github.com/pypa/setuptools/issues/163 + + +File: setuptools.info, Node: Misc<16>, Prev: Changes<19>, Up: v46 3 0 + +9.35.2 Misc +----------- + + * #2041(1): Preserve file modes during pkg files copying, but clear + read only flag for target afterwards. + + * #2105(2): Filter ‘2to3’ deprecation warnings from + ‘TestDevelop.test_2to3_user_mode’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2041 + + (2) https://github.com/pypa/setuptools/issues/2105 + + +File: setuptools.info, Node: v46 2 0, Next: v46 1 3, Prev: v46 3 0, Up: History<2> + +9.36 v46.2.0 +============ + +10 May 2020 + +* Menu: + +* Changes: Changes<20>. +* Documentation changes: Documentation changes<5>. +* Misc: Misc<17>. + + +File: setuptools.info, Node: Changes<20>, Next: Documentation changes<5>, Up: v46 2 0 + +9.36.1 Changes +-------------- + + * #2040(1): Deprecated the ‘bdist_wininst’ command. Binary packages + should be built as wheels instead. + + * #2062(2): Change ‘Mac OS X’ to ‘macOS’ in code. + + * #2075(3): Stop recognizing files ending with ‘.dist-info’ as + distribution metadata. + + * #2086(4): Deprecate ‘use_2to3’ functionality. Packagers are + encouraged to use single-source solutions or build tool chains to + manage conversions outside of setuptools. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2040 + + (2) https://github.com/pypa/setuptools/issues/2062 + + (3) https://github.com/pypa/setuptools/issues/2075 + + (4) https://github.com/pypa/setuptools/issues/2086 + + +File: setuptools.info, Node: Documentation changes<5>, Next: Misc<17>, Prev: Changes<20>, Up: v46 2 0 + +9.36.2 Documentation changes +---------------------------- + + * #1698(1): Added documentation for ‘build_meta’ (a bare minimum, not + completed). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1698 + + +File: setuptools.info, Node: Misc<17>, Prev: Documentation changes<5>, Up: v46 2 0 + +9.36.3 Misc +----------- + + * #2082(1): Filter ‘lib2to3’ ‘PendingDeprecationWarning’ and + ‘DeprecationWarning’ in tests, because ‘lib2to3’ is deprecated in + Python 3.9(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2082 + + (2) https://bugs.python.org/issue40360 + + +File: setuptools.info, Node: v46 1 3, Next: v46 1 2, Prev: v46 2 0, Up: History<2> + +9.37 v46.1.3 +============ + +25 Mar 2020 + +No significant changes. + + +File: setuptools.info, Node: v46 1 2, Next: v46 1 1, Prev: v46 1 3, Up: History<2> + +9.38 v46.1.2 +============ + +25 Mar 2020 + +* Menu: + +* Misc: Misc<18>. + + +File: setuptools.info, Node: Misc<18>, Up: v46 1 2 + +9.38.1 Misc +----------- + + * #1458(1): Added template for reporting Python 2 incompatibilities. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1458 + + +File: setuptools.info, Node: v46 1 1, Next: v46 1 0, Prev: v46 1 2, Up: History<2> + +9.39 v46.1.1 +============ + +21 Mar 2020 + +No significant changes. + + +File: setuptools.info, Node: v46 1 0, Next: v44 1 0, Prev: v46 1 1, Up: History<2> + +9.40 v46.1.0 +============ + +21 Mar 2020 + +* Menu: + +* Changes: Changes<21>. +* Incorporate changes from v44.1.0;: Incorporate changes from v44 1 0. + + +File: setuptools.info, Node: Changes<21>, Next: Incorporate changes from v44 1 0, Up: v46 1 0 + +9.40.1 Changes +-------------- + + * #308(1): Allow version number normalization to be bypassed by + wrapping in a ‘setuptools.sic()’ call. + + * #1424(2): Prevent keeping files mode for package_data build. It + may break a build if user’s package data has read only flag. + + * #1431(3): In ‘easy_install.check_site_dir’, ensure the installation + directory exists. + + * #1563(4): In ‘pkg_resources’ prefer ‘find_spec’ (PEP 451(5)) to + ‘find_module’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/308 + + (2) https://github.com/pypa/setuptools/issues/1424 + + (3) https://github.com/pypa/setuptools/issues/1431 + + (4) https://github.com/pypa/setuptools/issues/1563 + + (5) https://www.python.org/dev/peps/pep-0451/ + + +File: setuptools.info, Node: Incorporate changes from v44 1 0, Prev: Changes<21>, Up: v46 1 0 + +9.40.2 Incorporate changes from v44.1.0: +---------------------------------------- + + * #1704(1): Set sys.argv[0] in setup script run by + build_meta.__legacy__ + + * #1959(2): Fix for Python 4: replace unsafe six.PY3 with six.PY2 + + * #1994(3): Fixed a bug in the + “setuptools.finalize_distribution_options” hook that lead to + ignoring the order attribute of entry points managed by this hook. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1704 + + (2) https://github.com/pypa/setuptools/issues/1959 + + (3) https://github.com/pypa/setuptools/issues/1994 + + +File: setuptools.info, Node: v44 1 0, Next: v46 0 0, Prev: v46 1 0, Up: History<2> + +9.41 v44.1.0 +============ + +21 Mar 2020 + +* Menu: + +* Changes: Changes<22>. + + +File: setuptools.info, Node: Changes<22>, Up: v44 1 0 + +9.41.1 Changes +-------------- + + * #1704(1): Set sys.argv[0] in setup script run by + build_meta.__legacy__ + + * #1959(2): Fix for Python 4: replace unsafe six.PY3 with six.PY2 + + * #1994(3): Fixed a bug in the + “setuptools.finalize_distribution_options” hook that lead to + ignoring the order attribute of entry points managed by this hook. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1704 + + (2) https://github.com/pypa/setuptools/issues/1959 + + (3) https://github.com/pypa/setuptools/issues/1994 + + +File: setuptools.info, Node: v46 0 0, Next: v45 3 0, Prev: v44 1 0, Up: History<2> + +9.42 v46.0.0 +============ + +08 Mar 2020 + +* Menu: + +* Breaking Changes: Breaking Changes<5>. +* Changes: Changes<23>. +* Documentation changes: Documentation changes<6>. +* Misc: Misc<19>. + + +File: setuptools.info, Node: Breaking Changes<5>, Next: Changes<23>, Up: v46 0 0 + +9.42.1 Breaking Changes +----------------------- + + * #65(1): Once again as in 3.0, removed the Features feature. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/65 + + +File: setuptools.info, Node: Changes<23>, Next: Documentation changes<6>, Prev: Breaking Changes<5>, Up: v46 0 0 + +9.42.2 Changes +-------------- + + * #1890(1): Fix vendored dependencies so importing + ‘setuptools.extern.some_module’ gives the same object as + ‘setuptools._vendor.some_module’. This makes Metadata picklable + again. + + * #1899(2): Test suite now fails on warnings. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1890 + + (2) https://github.com/pypa/setuptools/issues/1899 + + +File: setuptools.info, Node: Documentation changes<6>, Next: Misc<19>, Prev: Changes<23>, Up: v46 0 0 + +9.42.3 Documentation changes +---------------------------- + + * #2011(1): Fix broken link to distutils docs on package_data + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/2011 + + +File: setuptools.info, Node: Misc<19>, Prev: Documentation changes<6>, Up: v46 0 0 + +9.42.4 Misc +----------- + + * #1991(1): Include pkg_resources test data in sdist, so tests can be + executed from it. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1991 + + +File: setuptools.info, Node: v45 3 0, Next: v45 2 0, Prev: v46 0 0, Up: History<2> + +9.43 v45.3.0 +============ + +07 Mar 2020 + +* Menu: + +* Changes: Changes<24>. + + +File: setuptools.info, Node: Changes<24>, Up: v45 3 0 + +9.43.1 Changes +-------------- + + * #1557(1): Deprecated eggsecutable scripts and updated docs. + + * #1904(2): Update msvc.py to use CPython 3.8.0 mechanism to find + msvc 14+ + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1557 + + (2) https://github.com/pypa/setuptools/issues/1904 + + +File: setuptools.info, Node: v45 2 0, Next: v45 1 0, Prev: v45 3 0, Up: History<2> + +9.44 v45.2.0 +============ + +08 Feb 2020 + +* Menu: + +* Changes: Changes<25>. +* Misc: Misc<20>. + + +File: setuptools.info, Node: Changes<25>, Next: Misc<20>, Up: v45 2 0 + +9.44.1 Changes +-------------- + + * #1905(1): Fixed defect in _imp, introduced in 41.6.0 when the + ‘tests’ directory is not present. + + * #1941(2): Improve editable installs with PEP 518(3) build + isolation: + + * The ‘--user’ option is now always available. A warning is + issued if the user site directory is not available. + + * The error shown when the install directory is not in + ‘PYTHONPATH’ has been turned into a warning. + + * #1981(4): Setuptools now declares its ‘tests’ and ‘docs’ + dependencies in metadata (extras). + + * #1985(5): Add support for installing scripts in environments where + bdist_wininst is missing (i.e. Python 3.9). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1905 + + (2) https://github.com/pypa/setuptools/issues/1941 + + (3) https://www.python.org/dev/peps/pep-0518/ + + (4) https://github.com/pypa/setuptools/issues/1981 + + (5) https://github.com/pypa/setuptools/issues/1985 + + +File: setuptools.info, Node: Misc<20>, Prev: Changes<25>, Up: v45 2 0 + +9.44.2 Misc +----------- + + * #1968(1): Add flake8-2020 to check for misuse of sys.version or + sys.version_info. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1968 + + +File: setuptools.info, Node: v45 1 0, Next: v45 0 0, Prev: v45 2 0, Up: History<2> + +9.45 v45.1.0 +============ + +19 Jan 2020 + +* Menu: + +* Changes: Changes<26>. + + +File: setuptools.info, Node: Changes<26>, Up: v45 1 0 + +9.45.1 Changes +-------------- + + * #1458(1): Add minimum sunset date and preamble to Python 2 warning. + + * #1704(2): Set sys.argv[0] in setup script run by + build_meta.__legacy__ + + * #1974(3): Add Python 3 Only Trove Classifier and remove universal + wheel declaration for more complete transition from Python 2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1458 + + (2) https://github.com/pypa/setuptools/issues/1704 + + (3) https://github.com/pypa/setuptools/issues/1974 + + +File: setuptools.info, Node: v45 0 0, Next: v44 0 0, Prev: v45 1 0, Up: History<2> + +9.46 v45.0.0 +============ + +11 Jan 2020 + +* Menu: + +* Breaking Changes: Breaking Changes<6>. +* Changes: Changes<27>. + + +File: setuptools.info, Node: Breaking Changes<6>, Next: Changes<27>, Up: v45 0 0 + +9.46.1 Breaking Changes +----------------------- + + * #1458(1): Drop support for Python 2. Setuptools now requires + Python 3.5 or later. Install setuptools using pip >=9 or pin to + Setuptools <45 to maintain 2.7 support. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1458 + + +File: setuptools.info, Node: Changes<27>, Prev: Breaking Changes<6>, Up: v45 0 0 + +9.46.2 Changes +-------------- + + * #1959(1): Fix for Python 4: replace unsafe six.PY3 with six.PY2 + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1959 + + +File: setuptools.info, Node: v44 0 0, Next: v43 0 0, Prev: v45 0 0, Up: History<2> + +9.47 v44.0.0 +============ + +01 Jan 2020 + +* Menu: + +* Breaking Changes: Breaking Changes<7>. + + +File: setuptools.info, Node: Breaking Changes<7>, Up: v44 0 0 + +9.47.1 Breaking Changes +----------------------- + + * #1908(1): Drop support for Python 3.4. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1908 + + +File: setuptools.info, Node: v43 0 0, Next: v42 0 2, Prev: v44 0 0, Up: History<2> + +9.48 v43.0.0 +============ + +31 Dec 2019 + +* Menu: + +* Breaking Changes: Breaking Changes<8>. +* Changes: Changes<28>. + + +File: setuptools.info, Node: Breaking Changes<8>, Next: Changes<28>, Up: v43 0 0 + +9.48.1 Breaking Changes +----------------------- + + * #1634(1): Include ‘pyproject.toml’ in source distribution by + default. Projects relying on the previous behavior where + ‘pyproject.toml’ was excluded by default should stop relying on + that behavior or add ‘exclude pyproject.toml’ to their MANIFEST.in + file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1634 + + +File: setuptools.info, Node: Changes<28>, Prev: Breaking Changes<8>, Up: v43 0 0 + +9.48.2 Changes +-------------- + + * #1927(1): Setuptools once again declares ‘setuptools’ in the + ‘build-system.requires’ and adds PEP 517(2) build support by + declaring itself as the ‘build-backend’. It additionally specifies + ‘build-system.backend-path’ to rely on itself for those builders + that support it. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1927 + + (2) https://www.python.org/dev/peps/pep-0517/ + + +File: setuptools.info, Node: v42 0 2, Next: v42 0 1, Prev: v43 0 0, Up: History<2> + +9.49 v42.0.2 +============ + +01 Dec 2019 + +* Menu: + +* Changes: Changes<29>. + + +File: setuptools.info, Node: Changes<29>, Up: v42 0 2 + +9.49.1 Changes +-------------- + + * #1921(1): Fix support for easy_install’s ‘find-links’ option in + ‘setup.cfg’. + + * #1922(2): Build dependencies (setup_requires and tests_require) now + install transitive dependencies indicated by extras. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1921 + + (2) https://github.com/pypa/setuptools/issues/1922 + + +File: setuptools.info, Node: v42 0 1, Next: v42 0 0, Prev: v42 0 2, Up: History<2> + +9.50 v42.0.1 +============ + +25 Nov 2019 + +* Menu: + +* Changes: Changes<30>. + + +File: setuptools.info, Node: Changes<30>, Up: v42 0 1 + +9.50.1 Changes +-------------- + + * #1918(1): Fix regression in handling wheels compatibility tags. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1918 + + +File: setuptools.info, Node: v42 0 0, Next: v41 6 0, Prev: v42 0 1, Up: History<2> + +9.51 v42.0.0 +============ + +23 Nov 2019 + +* Menu: + +* Breaking Changes: Breaking Changes<9>. +* Changes: Changes<31>. + + +File: setuptools.info, Node: Breaking Changes<9>, Next: Changes<31>, Up: v42 0 0 + +9.51.1 Breaking Changes +----------------------- + + * + #1830(1), #1909(2): Mark the easy_install script and setuptools command as deprecated, and use pip(3) when available to fetch/build wheels for missing ‘setup_requires’/‘tests_require’ requirements, with the following differences in behavior: + + * support for ‘python_requires’ + + * better support for wheels (proper handling of priority + with respect to PEP 425(4) tags) + + * PEP 517(5)/518 support + + * eggs are not supported + + * no support for the ‘allow_hosts’ easy_install option + (‘index_url’/‘find_links’ are still honored) + + * pip environment variables are honored (and take + precedence over easy_install options) + + * #1898(6): Removed the “upload” and “register” commands in favor of + twine(7). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1830 + + (2) https://github.com/pypa/setuptools/issues/1909 + + (3) https://pip.pypa.io/en/stable/ + + (4) https://www.python.org/dev/peps/pep-0425/ + + (5) https://www.python.org/dev/peps/pep-0517/ + + (6) https://github.com/pypa/setuptools/issues/1898 + + (7) https://pypi.org/p/twine + + +File: setuptools.info, Node: Changes<31>, Prev: Breaking Changes<9>, Up: v42 0 0 + +9.51.2 Changes +-------------- + + * #1767(1): Add support for the ‘license_files’ option in ‘setup.cfg’ + to automatically include multiple license files in a source + distribution. + + * #1829(2): Update handling of wheels compatibility tags: * add + support for manylinux2010 * fix use of removed ‘m’ ABI flag in + Python 3.8 on Windows + + * #1861(3): Fix empty namespace package installation from wheel. + + * #1877(4): Setuptools now exposes a new entry point hook + “setuptools.finalize_distribution_options”, enabling plugins like + setuptools_scm(5) to configure options on the distribution at + finalization time. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1767 + + (2) https://github.com/pypa/setuptools/issues/1829 + + (3) https://github.com/pypa/setuptools/issues/1861 + + (4) https://github.com/pypa/setuptools/issues/1877 + + (5) https://pypi.org/project/setuptools_scm + + +File: setuptools.info, Node: v41 6 0, Next: v41 5 1, Prev: v42 0 0, Up: History<2> + +9.52 v41.6.0 +============ + +29 Oct 2019 + +* Menu: + +* Changes: Changes<32>. + + +File: setuptools.info, Node: Changes<32>, Up: v41 6 0 + +9.52.1 Changes +-------------- + + * #479(1): Replace usage of deprecated ‘imp’ module with local + re-implementation in ‘setuptools._imp’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/479 + + +File: setuptools.info, Node: v41 5 1, Next: v41 5 0, Prev: v41 6 0, Up: History<2> + +9.53 v41.5.1 +============ + +28 Oct 2019 + +* Menu: + +* Changes: Changes<33>. + + +File: setuptools.info, Node: Changes<33>, Up: v41 5 1 + +9.53.1 Changes +-------------- + + * #1891(1): Fix code for detecting Visual Studio’s version on Windows + under Python 2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1891 + + +File: setuptools.info, Node: v41 5 0, Next: v41 4 0, Prev: v41 5 1, Up: History<2> + +9.54 v41.5.0 +============ + +27 Oct 2019 + +* Menu: + +* Changes: Changes<34>. +* Documentation changes: Documentation changes<7>. +* Misc: Misc<21>. + + +File: setuptools.info, Node: Changes<34>, Next: Documentation changes<7>, Up: v41 5 0 + +9.54.1 Changes +-------------- + + * #1811(1): Improve Visual C++ 14.X support, mainly for Visual Studio + 2017 and 2019. + + * #1814(2): Fix ‘pkg_resources.Requirement’ hash/equality + implementation: take PEP 508(3) direct URL into account. + + * #1824(4): Fix tests when running under ‘python3.10’. + + * #1878(5): Formally deprecated the ‘test’ command, with the + recommendation that users migrate to ‘tox’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1811 + + (2) https://github.com/pypa/setuptools/issues/1814 + + (3) https://www.python.org/dev/peps/pep-0508/ + + (4) https://github.com/pypa/setuptools/issues/1824 + + (5) https://github.com/pypa/setuptools/issues/1878 + + +File: setuptools.info, Node: Documentation changes<7>, Next: Misc<21>, Prev: Changes<34>, Up: v41 5 0 + +9.54.2 Documentation changes +---------------------------- + + * #1860(1): Update documentation to mention the egg format is not + supported by pip and dependency links support was dropped starting + with pip 19.0. + + * #1862(2): Drop ez_setup documentation: deprecated for some time + (last updated in 2016), and still relying on easy_install + (deprecated too). + + * #1868(3): Drop most documentation references to (deprecated) + EasyInstall. + + * #1884(4): Added a trove classifier to document support for Python + 3.8. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1860 + + (2) https://github.com/pypa/setuptools/issues/1862 + + (3) https://github.com/pypa/setuptools/issues/1868 + + (4) https://github.com/pypa/setuptools/issues/1884 + + +File: setuptools.info, Node: Misc<21>, Prev: Documentation changes<7>, Up: v41 5 0 + +9.54.3 Misc +----------- + + * #1886(1): Added Python 3.8 release to the Travis test matrix. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1886 + + +File: setuptools.info, Node: v41 4 0, Next: v41 3 0, Prev: v41 5 0, Up: History<2> + +9.55 v41.4.0 +============ + +06 Oct 2019 + +* Menu: + +* Changes: Changes<35>. + + +File: setuptools.info, Node: Changes<35>, Up: v41 4 0 + +9.55.1 Changes +-------------- + + * #1847(1): In declarative config, now traps errors when invalid + ‘python_requires’ values are supplied. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1847 + + +File: setuptools.info, Node: v41 3 0, Next: v41 2 0, Prev: v41 4 0, Up: History<2> + +9.56 v41.3.0 +============ + +06 Oct 2019 + +* Menu: + +* Changes: Changes<36>. +* Misc: Misc<22>. + + +File: setuptools.info, Node: Changes<36>, Next: Misc<22>, Up: v41 3 0 + +9.56.1 Changes +-------------- + + * #1690(1): When storing extras, rely on OrderedSet to retain order + of extras as indicated by the packager, which will also be + deterministic on Python 2.7 (with PYTHONHASHSEED unset) and Python + 3.6+. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1690 + + +File: setuptools.info, Node: Misc<22>, Prev: Changes<36>, Up: v41 3 0 + +9.56.2 Misc +----------- + + * #1858(1): Fixed failing integration test triggered by + ‘long_description_content_type’ in packaging. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1858 + + +File: setuptools.info, Node: v41 2 0, Next: v41 1 0, Prev: v41 3 0, Up: History<2> + +9.57 v41.2.0 +============ + +21 Aug 2019 + +* Menu: + +* Changes: Changes<37>. +* Misc: Misc<23>. + + +File: setuptools.info, Node: Changes<37>, Next: Misc<23>, Up: v41 2 0 + +9.57.1 Changes +-------------- + + * #479(1): Remove some usage of the deprecated ‘imp’ module. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/479 + + +File: setuptools.info, Node: Misc<23>, Prev: Changes<37>, Up: v41 2 0 + +9.57.2 Misc +----------- + + * #1565(1): Changed html_sidebars from string to list of string as + per ‘https://www.sphinx-doc.org/en/master/changes.html#id58’ + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1565 + + +File: setuptools.info, Node: v41 1 0, Next: v41 0 1, Prev: v41 2 0, Up: History<2> + +9.58 v41.1.0 +============ + +13 Aug 2019 + +* Menu: + +* Misc: Misc<24>. +* Documentation changes: Documentation changes<8>. + + +File: setuptools.info, Node: Misc<24>, Next: Documentation changes<8>, Up: v41 1 0 + +9.58.1 Misc +----------- + + * #1697(1): Moved most of the constants from setup.py to setup.cfg + + * #1749(2): Fixed issue with the PEP 517(3) backend where building a + source distribution would fail if any tarball existed in the + destination directory. + + * #1750(4): Fixed an issue with PEP 517(5) backend where wheel builds + would fail if the destination directory did not already exist. + + * #1756(6): Force metadata-version >= 1.2. when project urls are + present. + + * #1769(7): Improve ‘package_data’ check: ensure the dictionary + values are lists/tuples of strings. + + * #1788(8): Changed compatibility fallback logic for ‘html.unescape’ + to avoid accessing ‘HTMLParser.unescape’ when not necessary. + ‘HTMLParser.unescape’ is deprecated and will be removed in Python + 3.9. + + * #1790(9): Added the file path to the error message when a + ‘UnicodeDecodeError’ occurs while reading a metadata file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1697 + + (2) https://github.com/pypa/setuptools/issues/1749 + + (3) https://www.python.org/dev/peps/pep-0517/ + + (4) https://github.com/pypa/setuptools/issues/1750 + + (5) https://www.python.org/dev/peps/pep-0517/ + + (6) https://github.com/pypa/setuptools/issues/1756 + + (7) https://github.com/pypa/setuptools/issues/1769 + + (8) https://github.com/pypa/setuptools/issues/1788 + + (9) https://github.com/pypa/setuptools/issues/1790 + + +File: setuptools.info, Node: Documentation changes<8>, Prev: Misc<24>, Up: v41 1 0 + +9.58.2 Documentation changes +---------------------------- + + * #1776(1): Use license classifiers rather than the license field. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1776 + + +File: setuptools.info, Node: v41 0 1, Next: v41 0 0, Prev: v41 1 0, Up: History<2> + +9.59 v41.0.1 +============ + +22 Apr 2019 + +* Menu: + +* Changes: Changes<38>. + + +File: setuptools.info, Node: Changes<38>, Up: v41 0 1 + +9.59.1 Changes +-------------- + + * #1671(1): Fixed issue with the PEP 517(2) backend that prevented + building a wheel when the ‘dist/’ directory contained existing + ‘.whl’ files. + + * #1709(3): In test.paths_on_python_path, avoid adding unnecessary + duplicates to the PYTHONPATH. + + * #1741(4): In package_index, now honor “current directory” during a + checkout of git and hg repositories under Windows + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1671 + + (2) https://www.python.org/dev/peps/pep-0517/ + + (3) https://github.com/pypa/setuptools/issues/1709 + + (4) https://github.com/pypa/setuptools/issues/1741 + + +File: setuptools.info, Node: v41 0 0, Next: v40 9 0, Prev: v41 0 1, Up: History<2> + +9.60 v41.0.0 +============ + +05 Apr 2019 + +* Menu: + +* Breaking Changes: Breaking Changes<10>. + + +File: setuptools.info, Node: Breaking Changes<10>, Up: v41 0 0 + +9.60.1 Breaking Changes +----------------------- + + * #1735(1): When parsing setup.cfg files, setuptools now requires the + files to be encoded as UTF-8. Any other encoding will lead to a + UnicodeDecodeError. This change removes support for specifying an + encoding using a ‘coding: ‘ directive in the header of the file, a + feature that was introduces in 40.7. Given the recent release of + the aforementioned feature, it is assumed that few if any projects + are utilizing the feature to specify an encoding other than UTF-8. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1735 + + +File: setuptools.info, Node: v40 9 0, Next: v40 8 0, Prev: v41 0 0, Up: History<2> + +9.61 v40.9.0 +============ + +03 Apr 2019 + +* Menu: + +* Changes: Changes<39>. +* Documentation changes: Documentation changes<9>. + + +File: setuptools.info, Node: Changes<39>, Next: Documentation changes<9>, Up: v40 9 0 + +9.61.1 Changes +-------------- + + * #1675(1): Added support for ‘setup.cfg’-only projects when using + the ‘setuptools.build_meta’ backend. Projects that have enabled + PEP 517(2) no longer need to have a ‘setup.py’ and can use the + purely declarative ‘setup.cfg’ configuration file instead. + + * #1720(3): Added support for + ‘pkg_resources.parse_requirements’-style requirements in + ‘setup_requires’ when ‘setup.py’ is invoked from the + ‘setuptools.build_meta’ build backend. + + * #1664(4): Added the path to the ‘PKG-INFO’ or ‘METADATA’ file in + the exception text when the ‘Version:’ header can’t be found. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1675 + + (2) https://www.python.org/dev/peps/pep-0517/ + + (3) https://github.com/pypa/setuptools/issues/1720 + + (4) https://github.com/pypa/setuptools/issues/1664 + + +File: setuptools.info, Node: Documentation changes<9>, Prev: Changes<39>, Up: v40 9 0 + +9.61.2 Documentation changes +---------------------------- + + * #1705(1): Removed some placeholder documentation sections referring + to deprecated features. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1705 + + +File: setuptools.info, Node: v40 8 0, Next: v40 7 3, Prev: v40 9 0, Up: History<2> + +9.62 v40.8.0 +============ + +05 Feb 2019 + +* Menu: + +* Changes: Changes<40>. + + +File: setuptools.info, Node: Changes<40>, Up: v40 8 0 + +9.62.1 Changes +-------------- + + * #1652(1): Added the ‘build_meta:__legacy__’ backend, a + “compatibility mode” PEP 517(2) backend that can be used as the + default when ‘build-backend’ is left unspecified in + ‘pyproject.toml’. + + * #1635(3): Resource paths are passed to + ‘pkg_resources.resource_string’ and similar no longer accept paths + that traverse parents, that begin with a leading ‘/’. Violations + of this expectation raise DeprecationWarnings and will become + errors. Additionally, any paths that are absolute on Windows are + strictly disallowed and will raise ValueErrors. + + * #1536(4): ‘setuptools’ will now automatically include licenses if + ‘setup.cfg’ contains a ‘license_file’ attribute, unless this file + is manually excluded inside ‘MANIFEST.in’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1652 + + (2) https://www.python.org/dev/peps/pep-0517/ + + (3) https://github.com/pypa/setuptools/issues/1635 + + (4) https://github.com/pypa/setuptools/issues/1536 + + +File: setuptools.info, Node: v40 7 3, Next: v40 7 2, Prev: v40 8 0, Up: History<2> + +9.63 v40.7.3 +============ + +03 Feb 2019 + +* Menu: + +* Changes: Changes<41>. + + +File: setuptools.info, Node: Changes<41>, Up: v40 7 3 + +9.63.1 Changes +-------------- + + * #1670(1): In package_index, revert to using a copy of splituser + from Python 3.8. Attempts to use ‘urllib.parse.urlparse’ led to + problems as reported in #1663(2) and #1668(3). This change serves + as an alternative to #1499(4) and fixes #1668(5). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1670 + + (2) https://github.com/pypa/setuptools/issues/1663 + + (3) https://github.com/pypa/setuptools/issues/1668 + + (4) https://github.com/pypa/setuptools/issues/1499 + + (5) https://github.com/pypa/setuptools/issues/1668 + + +File: setuptools.info, Node: v40 7 2, Next: v40 7 1, Prev: v40 7 3, Up: History<2> + +9.64 v40.7.2 +============ + +31 Jan 2019 + +* Menu: + +* Changes: Changes<42>. + + +File: setuptools.info, Node: Changes<42>, Up: v40 7 2 + +9.64.1 Changes +-------------- + + * #1666(1): Restore port in URL handling in package_index. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1666 + + +File: setuptools.info, Node: v40 7 1, Next: v40 7 0, Prev: v40 7 2, Up: History<2> + +9.65 v40.7.1 +============ + +28 Jan 2019 + +* Menu: + +* Changes: Changes<43>. + + +File: setuptools.info, Node: Changes<43>, Up: v40 7 1 + +9.65.1 Changes +-------------- + + * #1660(1): On Python 2, when reading config files, downcast options + from text to bytes to satisfy distutils expectations. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1660 + + +File: setuptools.info, Node: v40 7 0, Next: v40 6 3, Prev: v40 7 1, Up: History<2> + +9.66 v40.7.0 +============ + +27 Jan 2019 + +* Menu: + +* Breaking Changes: Breaking Changes<11>. +* Changes: Changes<44>. + + +File: setuptools.info, Node: Breaking Changes<11>, Next: Changes<44>, Up: v40 7 0 + +9.66.1 Breaking Changes +----------------------- + + * #1551(1): File inputs for the ‘license’ field in ‘setup.cfg’ files + now explicitly raise an error. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1551 + + +File: setuptools.info, Node: Changes<44>, Prev: Breaking Changes<11>, Up: v40 7 0 + +9.66.2 Changes +-------------- + + * #1180(1): Add support for non-ASCII in setup.cfg (#1062(2)). Add + support for native strings on some parameters (#1136(3)). + + * #1499(4): ‘setuptools.package_index’ no longer relies on the + deprecated ‘urllib.parse.splituser’ per Python #27485(5). + + * #1544(6): Added tests for PackageIndex.download (for git URLs). + + * #1625(7): In PEP 517(8) build_meta builder, ensure that sdists are + built as gztar per the spec. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1180 + + (2) https://github.com/pypa/setuptools/issues/1062 + + (3) https://github.com/pypa/setuptools/issues/1136 + + (4) https://github.com/pypa/setuptools/issues/1499 + + (5) http://bugs.python.org/issue27485 + + (6) https://github.com/pypa/setuptools/issues/1544 + + (7) https://github.com/pypa/setuptools/issues/1625 + + (8) https://www.python.org/dev/peps/pep-0517/ + + +File: setuptools.info, Node: v40 6 3, Next: v40 6 2, Prev: v40 7 0, Up: History<2> + +9.67 v40.6.3 +============ + +11 Dec 2018 + +* Menu: + +* Changes: Changes<45>. + + +File: setuptools.info, Node: Changes<45>, Up: v40 6 3 + +9.67.1 Changes +-------------- + + * #1594(1): PEP 517(2) backend no longer declares setuptools as a + dependency as it can be assumed. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1594 + + (2) https://www.python.org/dev/peps/pep-0517/ + + +File: setuptools.info, Node: v40 6 2, Next: v40 6 1, Prev: v40 6 3, Up: History<2> + +9.68 v40.6.2 +============ + +13 Nov 2018 + +* Menu: + +* Changes: Changes<46>. + + +File: setuptools.info, Node: Changes<46>, Up: v40 6 2 + +9.68.1 Changes +-------------- + + * #1592(1): Fix invalid dependency on external six module (instead of + vendored version). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1592 + + +File: setuptools.info, Node: v40 6 1, Next: v40 6 0, Prev: v40 6 2, Up: History<2> + +9.69 v40.6.1 +============ + +12 Nov 2018 + +* Menu: + +* Changes: Changes<47>. + + +File: setuptools.info, Node: Changes<47>, Up: v40 6 1 + +9.69.1 Changes +-------------- + + * #1590(1): Fixed regression where packages without ‘author’ or + ‘author_email’ fields generated malformed package metadata. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1590 + + +File: setuptools.info, Node: v40 6 0, Next: v40 5 0, Prev: v40 6 1, Up: History<2> + +9.70 v40.6.0 +============ + +12 Nov 2018 + +* Menu: + +* Deprecations:: +* Changes: Changes<48>. +* Documentation changes: Documentation changes<10>. +* Misc: Misc<25>. + + +File: setuptools.info, Node: Deprecations, Next: Changes<48>, Up: v40 6 0 + +9.70.1 Deprecations +------------------- + + * #1541(1): Officially deprecated the ‘requires’ parameter in + ‘setup()’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1541 + + +File: setuptools.info, Node: Changes<48>, Next: Documentation changes<10>, Prev: Deprecations, Up: v40 6 0 + +9.70.2 Changes +-------------- + + * #1519(1): In ‘pkg_resources.normalize_path’, additional path + normalization is now performed to ensure path values to a directory + is always the same, preventing false positives when checking + scripts have a consistent prefix to set up on Windows. + + * #1545(2): Changed the warning class of all deprecation warnings; + deprecation warning classes are no longer derived from + ‘DeprecationWarning’ and are thus visible by default. + + * #1554(3): ‘build_meta.build_sdist’ now includes ‘setup.py’ in + source distributions by default. + + * #1576(4): Started monkey-patching ‘get_metadata_version’ and + ‘read_pkg_file’ onto ‘distutils.DistributionMetadata’ to retain the + correct version on the ‘PKG-INFO’ file in the (deprecated) ‘upload’ + command. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1519 + + (2) https://github.com/pypa/setuptools/issues/1545 + + (3) https://github.com/pypa/setuptools/issues/1554 + + (4) https://github.com/pypa/setuptools/issues/1576 + + +File: setuptools.info, Node: Documentation changes<10>, Next: Misc<25>, Prev: Changes<48>, Up: v40 6 0 + +9.70.3 Documentation changes +---------------------------- + + * #1395(1): Changed Pyrex references to Cython in the documentation. + + * #1456(2): Documented that the ‘rpmbuild’ packages is required for + the ‘bdist_rpm’ command. + + * #1537(3): Documented how to use ‘setup.cfg’ for ‘src/ layouts’ + + * #1539(4): Added minimum version column in ‘setup.cfg’ metadata + table. + + * #1552(5): Fixed a minor typo in the python 2/3 compatibility + documentation. + + * #1553(6): Updated installation instructions to point to ‘pip + install’ instead of ‘ez_setup.py’. + + * #1560(7): Updated ‘setuptools’ distribution documentation to remove + some outdated information. + + * #1564(8): Documented ‘setup.cfg’ minimum version for version and + project_urls. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1395 + + (2) https://github.com/pypa/setuptools/issues/1456 + + (3) https://github.com/pypa/setuptools/issues/1537 + + (4) https://github.com/pypa/setuptools/issues/1539 + + (5) https://github.com/pypa/setuptools/issues/1552 + + (6) https://github.com/pypa/setuptools/issues/1553 + + (7) https://github.com/pypa/setuptools/issues/1560 + + (8) https://github.com/pypa/setuptools/issues/1564 + + +File: setuptools.info, Node: Misc<25>, Prev: Documentation changes<10>, Up: v40 6 0 + +9.70.4 Misc +----------- + + * #1533(1): Restricted the ‘recursive-include setuptools/_vendor’ to + contain only .py and .txt files. + + * #1572(2): Added the ‘concurrent.futures’ backport ‘futures’ to the + Python 2.7 test suite requirements. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1533 + + (2) https://github.com/pypa/setuptools/issues/1572 + + +File: setuptools.info, Node: v40 5 0, Next: v40 4 3, Prev: v40 6 0, Up: History<2> + +9.71 v40.5.0 +============ + +26 Oct 2018 + +* Menu: + +* Changes: Changes<49>. +* Documentation changes: Documentation changes<11>. + + +File: setuptools.info, Node: Changes<49>, Next: Documentation changes<11>, Up: v40 5 0 + +9.71.1 Changes +-------------- + + * #1335(1): In ‘pkg_resources.normalize_path’, fix issue on Cygwin + when cwd contains symlinks. + + * #1502(2): Deprecated support for downloads from Subversion in + package_index/easy_install. + + * #1517(3): Dropped use of six.u in favor of ‘u""’ literals. + + * #1520(4): Added support for ‘data_files’ in ‘setup.cfg’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1335 + + (2) https://github.com/pypa/setuptools/issues/1502 + + (3) https://github.com/pypa/setuptools/issues/1517 + + (4) https://github.com/pypa/setuptools/issues/1520 + + +File: setuptools.info, Node: Documentation changes<11>, Prev: Changes<49>, Up: v40 5 0 + +9.71.2 Documentation changes +---------------------------- + + * #1525(1): Fixed rendering of the deprecation warning in + easy_install doc. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1525 + + +File: setuptools.info, Node: v40 4 3, Next: v40 4 2, Prev: v40 5 0, Up: History<2> + +9.72 v40.4.3 +============ + +23 Sep 2018 + +* Menu: + +* Changes: Changes<50>. + + +File: setuptools.info, Node: Changes<50>, Up: v40 4 3 + +9.72.1 Changes +-------------- + + * #1480(1): Bump vendored pyparsing in pkg_resources to 2.2.1. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1480 + + +File: setuptools.info, Node: v40 4 2, Next: v40 4 1, Prev: v40 4 3, Up: History<2> + +9.73 v40.4.2 +============ + +21 Sep 2018 + +* Menu: + +* Misc: Misc<26>. + + +File: setuptools.info, Node: Misc<26>, Up: v40 4 2 + +9.73.1 Misc +----------- + + * #1497(1): Updated gitignore in repo. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1497 + + +File: setuptools.info, Node: v40 4 1, Next: v40 4 0, Prev: v40 4 2, Up: History<2> + +9.74 v40.4.1 +============ + +18 Sep 2018 + +* Menu: + +* Changes: Changes<51>. + + +File: setuptools.info, Node: Changes<51>, Up: v40 4 1 + +9.74.1 Changes +-------------- + + * #1480(1): Bump vendored pyparsing to 2.2.1. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1480 + + +File: setuptools.info, Node: v40 4 0, Next: v40 3 0, Prev: v40 4 1, Up: History<2> + +9.75 v40.4.0 +============ + +18 Sep 2018 + +* Menu: + +* Changes: Changes<52>. + + +File: setuptools.info, Node: Changes<52>, Up: v40 4 0 + +9.75.1 Changes +-------------- + + * #1481(1): Join the sdist ‘--dist-dir’ and the ‘build_meta’ sdist + directory argument to point to the same target (meaning the build + frontend no longer needs to clean manually the dist dir to avoid + multiple sdist presence, and setuptools no longer needs to handle + conflicts between the two). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1481 + + +File: setuptools.info, Node: v40 3 0, Next: v40 2 0, Prev: v40 4 0, Up: History<2> + +9.76 v40.3.0 +============ + +16 Sep 2018 + +* Menu: + +* Changes: Changes<53>. +* Misc: Misc<27>. + + +File: setuptools.info, Node: Changes<53>, Next: Misc<27>, Up: v40 3 0 + +9.76.1 Changes +-------------- + + * #1402(1): Fixed a bug with namespace packages under Python 3.6 when + one package in current directory hides another which is installed. + + * #1427(2): Set timestamp of ‘.egg-info’ directory whenever + ‘egg_info’ command is run. + + * #1474(3): ‘build_meta.get_requires_for_build_sdist’ now does not + include the ‘wheel’ package anymore. + + * #1486(4): Suppress warnings in pkg_resources.handle_ns. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1402 + + (2) https://github.com/pypa/setuptools/issues/1427 + + (3) https://github.com/pypa/setuptools/issues/1474 + + (4) https://github.com/pypa/setuptools/issues/1486 + + +File: setuptools.info, Node: Misc<27>, Prev: Changes<53>, Up: v40 3 0 + +9.76.2 Misc +----------- + + * #1479(1): Remove internal use of six.binary_type. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1479 + + +File: setuptools.info, Node: v40 2 0, Next: v40 1 1, Prev: v40 3 0, Up: History<2> + +9.77 v40.2.0 +============ + +21 Aug 2018 + +* Menu: + +* Changes: Changes<54>. + + +File: setuptools.info, Node: Changes<54>, Up: v40 2 0 + +9.77.1 Changes +-------------- + + * #1466(1): Fix handling of Unicode arguments in PEP 517(2) backend + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1466 + + (2) https://www.python.org/dev/peps/pep-0517/ + + +File: setuptools.info, Node: v40 1 1, Next: v40 1 0, Prev: v40 2 0, Up: History<2> + +9.78 v40.1.1 +============ + +21 Aug 2018 + +* Menu: + +* Changes: Changes<55>. + + +File: setuptools.info, Node: Changes<55>, Up: v40 1 1 + +9.78.1 Changes +-------------- + + * #1465(1): Fix regression with ‘egg_info’ command when tagging is + used. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1465 + + +File: setuptools.info, Node: v40 1 0, Next: v40 0 0, Prev: v40 1 1, Up: History<2> + +9.79 v40.1.0 +============ + +17 Aug 2018 + +* Menu: + +* Changes: Changes<56>. +* Misc: Misc<28>. + + +File: setuptools.info, Node: Changes<56>, Next: Misc<28>, Up: v40 1 0 + +9.79.1 Changes +-------------- + + * #1410(1): Deprecated ‘upload’ and ‘register’ commands. + + * #1312(2): Introduced find_namespace_packages() to find PEP 420(3) + namespace packages. + + * #1420(4): Added find_namespace: directive to config parser. + + * #1418(5): Solved race in when creating egg cache directories. + + * #1450(6): Upgraded vendored PyParsing from 2.1.10 to 2.2.0. + + * #1451(7): Upgraded vendored appdirs from 1.4.0 to 1.4.3. + + * #1388(8): Fixed “Microsoft Visual C++ Build Tools” link in + exception when Visual C++ not found. + + * #1389(9): Added support for scripts which have unicode content. + + * #1416(10): Moved several Python version checks over to using + ‘six.PY2’ and ‘six.PY3’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1410 + + (2) https://github.com/pypa/setuptools/issues/1312 + + (3) https://www.python.org/dev/peps/pep-0420/ + + (4) https://github.com/pypa/setuptools/issues/1420 + + (5) https://github.com/pypa/setuptools/issues/1418 + + (6) https://github.com/pypa/setuptools/issues/1450 + + (7) https://github.com/pypa/setuptools/issues/1451 + + (8) https://github.com/pypa/setuptools/issues/1388 + + (9) https://github.com/pypa/setuptools/issues/1389 + + (10) https://github.com/pypa/setuptools/issues/1416 + + +File: setuptools.info, Node: Misc<28>, Prev: Changes<56>, Up: v40 1 0 + +9.79.2 Misc +----------- + + * #1441(1): Removed spurious executable permissions from files that + don’t need them. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1441 + + +File: setuptools.info, Node: v40 0 0, Next: v39 2 0, Prev: v40 1 0, Up: History<2> + +9.80 v40.0.0 +============ + +09 Jul 2018 + +* Menu: + +* Breaking Changes: Breaking Changes<12>. +* Changes: Changes<57>. +* Documentation changes: Documentation changes<12>. +* Misc: Misc<29>. + + +File: setuptools.info, Node: Breaking Changes<12>, Next: Changes<57>, Up: v40 0 0 + +9.80.1 Breaking Changes +----------------------- + + * #1342(1): Drop support for Python 3.3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1342 + + +File: setuptools.info, Node: Changes<57>, Next: Documentation changes<12>, Prev: Breaking Changes<12>, Up: v40 0 0 + +9.80.2 Changes +-------------- + + * #1366(1): In package_index, fixed handling of encoded entities in + URLs. + + * #1383(2): In pkg_resources VendorImporter, avoid removing packages + imported from the root. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1366 + + (2) https://github.com/pypa/setuptools/issues/1383 + + +File: setuptools.info, Node: Documentation changes<12>, Next: Misc<29>, Prev: Changes<57>, Up: v40 0 0 + +9.80.3 Documentation changes +---------------------------- + + * #1379(1): Minor doc fixes after actually using the new release + process. + + * #1385(2): Removed section on non-package data files. + + * #1403(3): Fix developer’s guide. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1379 + + (2) https://github.com/pypa/setuptools/issues/1385 + + (3) https://github.com/pypa/setuptools/issues/1403 + + +File: setuptools.info, Node: Misc<29>, Prev: Documentation changes<12>, Up: v40 0 0 + +9.80.4 Misc +----------- + + * #1404(1): Fix PEP 518(2) configuration: set build requirements in + ‘pyproject.toml’ to ‘["wheel"]’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1404 + + (2) https://www.python.org/dev/peps/pep-0518/ + + +File: setuptools.info, Node: v39 2 0, Next: v39 1 0, Prev: v40 0 0, Up: History<2> + +9.81 v39.2.0 +============ + +19 May 2018 + +* Menu: + +* Changes: Changes<58>. +* Documentation changes: Documentation changes<13>. +* Misc: Misc<30>. + + +File: setuptools.info, Node: Changes<58>, Next: Documentation changes<13>, Up: v39 2 0 + +9.81.1 Changes +-------------- + + * #1359(1): Support using “file:” to load a PEP 440(2)-compliant + package version from a text file. + + * #1360(3): Fixed issue with a mismatch between the name of the + package and the name of the .dist-info file in wheel files + + * #1364(4): Add ‘__dir__()’ implementation to + ‘pkg_resources.Distribution()’ that includes the attributes in the + ‘_provider’ instance variable. + + * #1365(5): Take the package_dir option into account when loading the + version from a module attribute. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1359 + + (2) https://www.python.org/dev/peps/pep-0440/ + + (3) https://github.com/pypa/setuptools/issues/1360 + + (4) https://github.com/pypa/setuptools/issues/1364 + + (5) https://github.com/pypa/setuptools/issues/1365 + + +File: setuptools.info, Node: Documentation changes<13>, Next: Misc<30>, Prev: Changes<58>, Up: v39 2 0 + +9.81.2 Documentation changes +---------------------------- + + * #1353(1): Added coverage badge to README. + + * #1356(2): Made small fixes to the developer guide documentation. + + * #1357(3): Fixed warnings in documentation builds and started + enforcing that the docs build without warnings in tox. + + * #1376(4): Updated release process docs. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1353 + + (2) https://github.com/pypa/setuptools/issues/1356 + + (3) https://github.com/pypa/setuptools/issues/1357 + + (4) https://github.com/pypa/setuptools/issues/1376 + + +File: setuptools.info, Node: Misc<30>, Prev: Documentation changes<13>, Up: v39 2 0 + +9.81.3 Misc +----------- + + * #1343(1): The ‘setuptools’ specific + ‘long_description_content_type’, ‘project_urls’ and + ‘provides_extras’ fields are now set consistently after any + ‘distutils’ ‘setup_keywords’ calls, allowing them to override + values. + + * #1352(2): Added ‘tox’ environment for documentation builds. + + * #1354(3): Added ‘towncrier’ for changelog management. + + * #1355(4): Add PR template. + + * #1368(5): Fixed tests which failed without network connectivity. + + * #1369(6): Added unit tests for PEP 425(7) compatibility tags + support. + + * #1372(8): Stop testing Python 3.3 in Travis CI, now that the latest + version of ‘wheel’ no longer installs on it. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1343 + + (2) https://github.com/pypa/setuptools/issues/1352 + + (3) https://github.com/pypa/setuptools/issues/1354 + + (4) https://github.com/pypa/setuptools/issues/1355 + + (5) https://github.com/pypa/setuptools/issues/1368 + + (6) https://github.com/pypa/setuptools/issues/1369 + + (7) https://www.python.org/dev/peps/pep-0425/ + + (8) https://github.com/pypa/setuptools/issues/1372 + + +File: setuptools.info, Node: v39 1 0, Next: v39 0 1, Prev: v39 2 0, Up: History<2> + +9.82 v39.1.0 +============ + +28 Apr 2018 + + * #1340(1): Update all PyPI URLs to reflect the switch to the new + Warehouse codebase. + + * #1337(2): In ‘pkg_resources’, now support loading resources for + modules loaded by the ‘SourcelessFileLoader’. + + * #1332(3): Silence spurious wheel related warnings on Windows. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1340 + + (2) https://github.com/pypa/setuptools/issues/1337 + + (3) https://github.com/pypa/setuptools/issues/1332 + + +File: setuptools.info, Node: v39 0 1, Next: v39 0 0, Prev: v39 1 0, Up: History<2> + +9.83 v39.0.1 +============ + +18 Mar 2018 + + * #1297(1): Restore Unicode handling for Maintainer fields in + metadata. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1297 + + +File: setuptools.info, Node: v39 0 0, Next: v38 7 0, Prev: v39 0 1, Up: History<2> + +9.84 v39.0.0 +============ + +17 Mar 2018 + + * #1296(1): Setuptools now vendors its own direct dependencies, no + longer relying on the dependencies as vendored by pkg_resources. + + * #296(2): Removed long-deprecated support for iteration on Version + objects as returned by ‘pkg_resources.parse_version’. Removed the + ‘SetuptoolsVersion’ and ‘SetuptoolsLegacyVersion’ names as well. + They should not have been used, but if they were, replace with + ‘Version’ and ‘LegacyVersion’ from ‘packaging.version’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1296 + + (2) https://github.com/pypa/setuptools/issues/296 + + +File: setuptools.info, Node: v38 7 0, Next: v38 6 1, Prev: v39 0 0, Up: History<2> + +9.85 v38.7.0 +============ + +17 Mar 2018 + + * #1288(1): Add support for maintainer in PKG-INFO. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1288 + + +File: setuptools.info, Node: v38 6 1, Next: v38 6 0, Prev: v38 7 0, Up: History<2> + +9.86 v38.6.1 +============ + +17 Mar 2018 + + * #1292(1): Avoid generating ‘Provides-Extra’ in metadata when no + extra is present (but environment markers are). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1292 + + +File: setuptools.info, Node: v38 6 0, Next: v38 5 2, Prev: v38 6 1, Up: History<2> + +9.87 v38.6.0 +============ + +15 Mar 2018 + + * #1286(1): Add support for Metadata 2.1 (PEP 566(2)). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1286 + + (2) https://www.python.org/dev/peps/pep-0566/ + + +File: setuptools.info, Node: v38 5 2, Next: v38 5 1, Prev: v38 6 0, Up: History<2> + +9.88 v38.5.2 +============ + +06 Mar 2018 + + * #1285(1): Fixed RuntimeError in pkg_resources.parse_requirements on + Python 3.7 (stemming from PEP 479(2)). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1285 + + (2) https://www.python.org/dev/peps/pep-0479/ + + +File: setuptools.info, Node: v38 5 1, Next: v38 5 0, Prev: v38 5 2, Up: History<2> + +9.89 v38.5.1 +============ + +06 Feb 2018 + + * #1271(1): Revert to Cython legacy ‘build_ext’ behavior for + compatibility. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1271 + + +File: setuptools.info, Node: v38 5 0, Next: v38 4 1, Prev: v38 5 1, Up: History<2> + +9.90 v38.5.0 +============ + +04 Feb 2018 + + * #1229(1): Expand imports in ‘build_ext’ to refine detection of + Cython availability. + + * #1270(2): When Cython is available, ‘build_ext’ now uses the + new_build_ext. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1229 + + (2) https://github.com/pypa/setuptools/issues/1270 + + +File: setuptools.info, Node: v38 4 1, Next: v38 4 0, Prev: v38 5 0, Up: History<2> + +9.91 v38.4.1 +============ + +03 Feb 2018 + + * #1257(1): In bdist_egg.scan_module, fix ValueError on Python 3.7. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1257 + + +File: setuptools.info, Node: v38 4 0, Next: v38 3 0, Prev: v38 4 1, Up: History<2> + +9.92 v38.4.0 +============ + +05 Jan 2018 + + * #1231(1): Removed warning when PYTHONDONTWRITEBYTECODE is enabled. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1231 + + +File: setuptools.info, Node: v38 3 0, Next: v38 2 5, Prev: v38 4 0, Up: History<2> + +9.93 v38.3.0 +============ + +04 Jan 2018 + + * #1210(1): Add support for PEP 345(2) Project-URL metadata. + + * #1207(3): Add support for ‘long_description_type’ to setup.cfg + declarative config as intended and documented. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1210 + + (2) https://www.python.org/dev/peps/pep-0345/ + + (3) https://github.com/pypa/setuptools/issues/1207 + + +File: setuptools.info, Node: v38 2 5, Next: v38 2 4, Prev: v38 3 0, Up: History<2> + +9.94 v38.2.5 +============ + + * #1232(1): Fix trailing slash handling in + ‘pkg_resources.ZipProvider’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1232 + + +File: setuptools.info, Node: v38 2 4, Next: v38 2 3, Prev: v38 2 5, Up: History<2> + +9.95 v38.2.4 +============ + +04 Dec 2017 + + * #1220(1): Fix ‘data_files’ handling when installing from wheel. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1220 + + +File: setuptools.info, Node: v38 2 3, Next: v38 2 2, Prev: v38 2 4, Up: History<2> + +9.96 v38.2.3 +============ + +28 Nov 2017 + + * fix Travis’ Python 3.3 job. + + +File: setuptools.info, Node: v38 2 2, Next: v38 2 1, Prev: v38 2 3, Up: History<2> + +9.97 v38.2.2 +============ + +27 Nov 2017 + + * #1214(1): fix handling of namespace packages when installing from a + wheel. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1214 + + +File: setuptools.info, Node: v38 2 1, Next: v38 2 0, Prev: v38 2 2, Up: History<2> + +9.98 v38.2.1 +============ + +26 Nov 2017 + + * #1212(1): fix encoding handling of metadata when installing from a + wheel. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1212 + + +File: setuptools.info, Node: v38 2 0, Next: v38 1 0, Prev: v38 2 1, Up: History<2> + +9.99 v38.2.0 +============ + +26 Nov 2017 + + * #1200(1): easy_install now support installing from wheels: they + will be installed as standalone unzipped eggs. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1200 + + +File: setuptools.info, Node: v38 1 0, Next: v38 0 0, Prev: v38 2 0, Up: History<2> + +9.100 v38.1.0 +============= + +25 Nov 2017 + + * #1208(1): Improve error message when failing to locate scripts in + egg-info metadata. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1208 + + +File: setuptools.info, Node: v38 0 0, Next: v37 0 0, Prev: v38 1 0, Up: History<2> + +9.101 v38.0.0 +============= + +25 Nov 2017 + + * #458(1): In order to support deterministic builds, Setuptools no + longer allows packages to declare ‘install_requires’ as unordered + sequences (sets or dicts). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/458 + + +File: setuptools.info, Node: v37 0 0, Next: v36 8 0, Prev: v38 0 0, Up: History<2> + +9.102 v37.0.0 +============= + +20 Nov 2017 + + * #878(1): Drop support for Python 2.6. Python 2.6 users should rely + on ‘setuptools < 37dev’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/878 + + +File: setuptools.info, Node: v36 8 0, Next: v36 7 3, Prev: v37 0 0, Up: History<2> + +9.103 v36.8.0 +============= + +19 Nov 2017 + + * #1190(1): In SSL support for package index operations, use SNI + where available. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1190 + + +File: setuptools.info, Node: v36 7 3, Next: v36 7 2, Prev: v36 8 0, Up: History<2> + +9.104 v36.7.3 +============= + +13 Nov 2017 + + * #1175(1): Bug fixes to ‘build_meta’ module. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1175 + + +File: setuptools.info, Node: v36 7 2, Next: v36 7 1, Prev: v36 7 3, Up: History<2> + +9.105 v36.7.2 +============= + +13 Nov 2017 + + * #701(1): Fixed duplicate test discovery on Python 3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/701 + + +File: setuptools.info, Node: v36 7 1, Next: v36 7 0, Prev: v36 7 2, Up: History<2> + +9.106 v36.7.1 +============= + +11 Nov 2017 + + * #1193(1): Avoid test failures in bdist_egg when + PYTHONDONTWRITEBYTECODE is set. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1193 + + +File: setuptools.info, Node: v36 7 0, Next: v36 6 1, Prev: v36 7 1, Up: History<2> + +9.107 v36.7.0 +============= + +09 Nov 2017 + + * #1054(1): Support ‘setup_requires’ in ‘setup.cfg’ files. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1054 + + +File: setuptools.info, Node: v36 6 1, Next: v36 6 0, Prev: v36 7 0, Up: History<2> + +9.108 v36.6.1 +============= + +09 Nov 2017 + + * #1132(1): Removed redundant and costly serialization/parsing step + in ‘EntryPoint.__init__’. + + * #844(2): ‘bdist_egg --exclude-source-files’ now tested and works on + Python 3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1132 + + (2) https://github.com/pypa/setuptools/issues/844 + + +File: setuptools.info, Node: v36 6 0, Next: v36 5 0, Prev: v36 6 1, Up: History<2> + +9.109 v36.6.0 +============= + +12 Oct 2017 + + * #1143(1): Added ‘setuptools.build_meta’ module, an implementation + of PEP-517(2) for Setuptools-defined packages. + + * #1143(3): Added ‘dist_info’ command for producing dist_info + metadata. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1143 + + (2) https://www.python.org/dev/peps/pep-0517/ + + (3) https://github.com/pypa/setuptools/issues/1143 + + +File: setuptools.info, Node: v36 5 0, Next: v36 4 0, Prev: v36 6 0, Up: History<2> + +9.110 v36.5.0 +============= + +15 Sep 2017 + + * #170(1): When working with Mercurial checkouts, use + Windows-friendly syntax for suppressing output. + + * Inspired by #1134(2), performed substantial refactoring of + ‘pkg_resources.find_on_path’ to facilitate an optimization for + paths with many non-version entries. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/170 + + (2) https://github.com/pypa/setuptools/issues/1134 + + +File: setuptools.info, Node: v36 4 0, Next: v36 3 0, Prev: v36 5 0, Up: History<2> + +9.111 v36.4.0 +============= + +03 Sep 2017 + + * #1075(1): Add new ‘Description-Content-Type’ metadata field. See + here for documentation on how to use this field.(2) + + * #1068(3): Sort files and directories when building eggs for + deterministic order. + + * #196(4): Remove caching of easy_install command in fetch_build_egg. + Fixes issue where ‘pytest-runner-N.N’ would satisfy the + installation of ‘pytest’. + + * #1129(5): Fix working set dependencies handling when replacing + conflicting distributions (e.g. when using ‘setup_requires’ with a + conflicting transitive dependency, fix #1124(6)). + + * #1133(7): Improved handling of README files extensions and added + Markdown to the list of searched READMES. + + * #1135(8): Improve performance of pkg_resources import by not + invoking ‘access’ or ‘stat’ and using ‘os.listdir’ instead. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1075 + + (2) +https://packaging.python.org/specifications/#description-content-type + + (3) https://github.com/pypa/setuptools/issues/1068 + + (4) https://github.com/pypa/setuptools/issues/196 + + (5) https://github.com/pypa/setuptools/issues/1129 + + (6) https://github.com/pypa/setuptools/issues/1124 + + (7) https://github.com/pypa/setuptools/issues/1133 + + (8) https://github.com/pypa/setuptools/issues/1135 + + +File: setuptools.info, Node: v36 3 0, Next: v36 2 7, Prev: v36 4 0, Up: History<2> + +9.112 v36.3.0 +============= + +28 Aug 2017 + + * #1131(1): Make possible using several files within ‘file:’ + directive in metadata.long_description in ‘setup.cfg’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1131 + + +File: setuptools.info, Node: v36 2 7, Next: v36 2 6, Prev: v36 3 0, Up: History<2> + +9.113 v36.2.7 +============= + +02 Aug 2017 + + * fix #1105(1): Fix handling of requirements with environment markers + when declared in ‘setup.cfg’ (same treatment as for #1081(2)). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1105 + + (2) https://github.com/pypa/setuptools/issues/1081 + + +File: setuptools.info, Node: v36 2 6, Next: v36 2 5, Prev: v36 2 7, Up: History<2> + +9.114 v36.2.6 +============= + +31 Jul 2017 + + * #462(1): Don’t assume a directory is an egg by the ‘.egg’ extension + alone. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/462 + + +File: setuptools.info, Node: v36 2 5, Next: v36 2 4, Prev: v36 2 6, Up: History<2> + +9.115 v36.2.5 +============= + +30 Jul 2017 + + * #1093(1): Fix test command handler with extras_require. + + * #1112(2), #1091(3), #1115(4): Now using Trusty containers in Travis + for CI and CD. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1093 + + (2) https://github.com/pypa/setuptools/issues/1112 + + (3) https://github.com/pypa/setuptools/issues/1091 + + (4) https://github.com/pypa/setuptools/issues/1115 + + +File: setuptools.info, Node: v36 2 4, Next: v36 2 3, Prev: v36 2 5, Up: History<2> + +9.116 v36.2.4 +============= + +26 Jul 2017 + + * #1092(1): ‘pkg_resources’ now uses ‘inspect.getmro’ to resolve + classes in method resolution order. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1092 + + +File: setuptools.info, Node: v36 2 3, Next: v36 2 2, Prev: v36 2 4, Up: History<2> + +9.117 v36.2.3 +============= + +25 Jul 2017 + + * #1102(1): Restore behavior for empty extras. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1102 + + +File: setuptools.info, Node: v36 2 2, Next: v36 2 1, Prev: v36 2 3, Up: History<2> + +9.118 v36.2.2 +============= + +24 Jul 2017 + + * #1099(1): Revert commit a3ec721, restoring intended purpose of + extras as part of a requirement declaration. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1099 + + +File: setuptools.info, Node: v36 2 1, Next: v36 2 0, Prev: v36 2 2, Up: History<2> + +9.119 v36.2.1 +============= + +23 Jul 2017 + + * fix #1086(1) + + * fix #1087(2) + + * support extras specifiers in install_requires requirements + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1086 + + (2) https://github.com/pypa/setuptools/issues/1087 + + +File: setuptools.info, Node: v36 2 0, Next: v36 1 1, Prev: v36 2 1, Up: History<2> + +9.120 v36.2.0 +============= + +13 Jul 2017 + + * #1081(1): Environment markers indicated in ‘install_requires’ are + now processed and treated as nameless ‘extras_require’ with + markers, allowing their metadata in requires.txt to be correctly + generated. + + * #1053(2): Tagged commits are now released using Travis-CI build + stages, meaning releases depend on passing tests on all supported + Python versions (Linux) and not just the latest Python version. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1081 + + (2) https://github.com/pypa/setuptools/issues/1053 + + +File: setuptools.info, Node: v36 1 1, Next: v36 1 0, Prev: v36 2 0, Up: History<2> + +9.121 v36.1.1 +============= + +13 Jul 2017 + + * #1083(1): Correct ‘py31compat.makedirs’ to correctly honor + ‘exist_ok’ parameter. + + * #1083(2): Also use makedirs compatibility throughout setuptools. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1083 + + (2) https://github.com/pypa/setuptools/issues/1083 + + +File: setuptools.info, Node: v36 1 0, Next: v36 0 1, Prev: v36 1 1, Up: History<2> + +9.122 v36.1.0 +============= + +13 Jul 2017 + + * #1083(1): Avoid race condition on directory creation in + ‘pkg_resources.ensure_directory’. + + * Removed deprecation of and restored support for ‘upload_docs’ + command for sites other than PyPI. Only warehouse is dropping + support, but services like devpi(2) continue to support docs built + by setuptools’ plugins. See this comment(3) for more context on + the motivation for this change. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1083 + + (2) http://doc.devpi.net/latest/ + + (3) +https://bitbucket.org/hpk42/devpi/issues/388/support-rtd-model-for-building-uploading#comment-34292423 + + +File: setuptools.info, Node: v36 0 1, Next: v36 0 0, Prev: v36 1 0, Up: History<2> + +9.123 v36.0.1 +============= + +01 Jun 2017 + + * #1042(1): Fix import in py27compat module that still referenced six + directly, rather than through the externs module (vendored packages + hook). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1042 + + +File: setuptools.info, Node: v36 0 0, Next: v35 0 2, Prev: v36 0 1, Up: History<2> + +9.124 v36.0.0 +============= + +30 May 2017 + + * #980(1) and others: Once again, Setuptools vendors all of its + dependencies. It seems to be the case that in the Python + ecosystem, all build tools must run without any dependencies + (build, runtime, or otherwise). At such a point that a mechanism + exists that allows build tools to have dependencies, Setuptools + will adopt it. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/980 + + +File: setuptools.info, Node: v35 0 2, Next: v35 0 1, Prev: v36 0 0, Up: History<2> + +9.125 v35.0.2 +============= + +27 Apr 2017 + + * #1015(1): Fix test failures on Python 3.7. + + * #1024(2): Add workaround for Jython #2581(3) in monkey module. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1015 + + (2) https://github.com/pypa/setuptools/issues/1024 + + (3) http://bugs.jython.org/issue2581 + + +File: setuptools.info, Node: v35 0 1, Next: v35 0 0, Prev: v35 0 2, Up: History<2> + +9.126 v35.0.1 +============= + +18 Apr 2017 + + * #992(1): Revert change introduced in v34.4.1, now considered + invalid. + + * #1016(2): Revert change introduced in v35.0.0 per #1014(3), + referencing #436(4). The approach had unintended consequences, + causing sdist installs to be missing files. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/992 + + (2) https://github.com/pypa/setuptools/issues/1016 + + (3) https://github.com/pypa/setuptools/issues/1014 + + (4) https://github.com/pypa/setuptools/issues/436 + + +File: setuptools.info, Node: v35 0 0, Next: v34 4 1, Prev: v35 0 1, Up: History<2> + +9.127 v35.0.0 +============= + +15 Apr 2017 + + * #436(1): In egg_info.manifest_maker, no longer read the file list + from the manifest file, and instead re-build it on each build. In + this way, files removed from the specification will not linger in + the manifest. As a result, any files manually added to the + manifest will be removed on subsequent egg_info invocations. No + projects should be manually adding files to the manifest and should + instead use MANIFEST.in or SCM file finders to force inclusion of + files in the manifest. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/436 + + +File: setuptools.info, Node: v34 4 1, Next: v34 4 0, Prev: v35 0 0, Up: History<2> + +9.128 v34.4.1 +============= + +10 Apr 2017 + + * #1008(1): In MSVC support, use always the last version available + for Windows SDK and UCRT SDK. + + * #1008(2): In MSVC support, fix “vcruntime140.dll” returned path + with Visual Studio 2017. + + * #992(3): In msvc.msvc9_query_vcvarsall, ensure the returned dicts + have str values and not Unicode for compatibility with os.environ. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1008 + + (2) https://github.com/pypa/setuptools/issues/1008 + + (3) https://github.com/pypa/setuptools/issues/992 + + +File: setuptools.info, Node: v34 4 0, Next: v34 3 3, Prev: v34 4 1, Up: History<2> + +9.129 v34.4.0 +============= + +07 Apr 2017 + + * #995(1): In MSVC support, add support for “Microsoft Visual Studio + 2017” and “Microsoft Visual Studio Build Tools 2017”. + + * #999(2) via #1007(3): Extend support for declarative package config + in a setup.cfg file to include the options ‘python_requires’ and + ‘py_modules’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/995 + + (2) https://github.com/pypa/setuptools/issues/999 + + (3) https://github.com/pypa/setuptools/issues/1007 + + +File: setuptools.info, Node: v34 3 3, Next: v34 3 2, Prev: v34 4 0, Up: History<2> + +9.130 v34.3.3 +============= + +26 Mar 2017 + + * #967(1) (and #997(2)): Explicitly import submodules of packaging to + account for environments where the imports of those submodules is + not implied by other behavior. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/967 + + (2) https://github.com/pypa/setuptools/issues/997 + + +File: setuptools.info, Node: v34 3 2, Next: v34 3 1, Prev: v34 3 3, Up: History<2> + +9.131 v34.3.2 +============= + +11 Mar 2017 + + * #993(1): Fix documentation upload by correcting rendering of + content-type in _build_multipart on Python 3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/993 + + +File: setuptools.info, Node: v34 3 1, Next: v34 3 0, Prev: v34 3 2, Up: History<2> + +9.132 v34.3.1 +============= + +02 Mar 2017 + + * #988(1): Trap ‘os.unlink’ same as ‘os.remove’ in ‘auto_chmod’ error + handler. + + * #983(2): Fixes to invalid escape sequence deprecations on Python + 3.6. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/988 + + (2) https://github.com/pypa/setuptools/issues/983 + + +File: setuptools.info, Node: v34 3 0, Next: v34 2 0, Prev: v34 3 1, Up: History<2> + +9.133 v34.3.0 +============= + +23 Feb 2017 + + * #941(1): In the upload command, if the username is blank, default + to ‘getpass.getuser()’. + + * #971(2): Correct distutils findall monkeypatch to match appropriate + versions (namely Python 3.4.6). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/941 + + (2) https://github.com/pypa/setuptools/issues/971 + + +File: setuptools.info, Node: v34 2 0, Next: v34 1 1, Prev: v34 3 0, Up: History<2> + +9.134 v34.2.0 +============= + +12 Feb 2017 + + * #966(1): Add support for reading dist-info metadata and thus + locating Distributions from zip files. + + * #968(2): Allow ‘+’ and ‘!’ in egg fragments so that it can take + package names that contain PEP 440(3) conforming version + specifiers. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/966 + + (2) https://github.com/pypa/setuptools/issues/968 + + (3) https://www.python.org/dev/peps/pep-0440/ + + +File: setuptools.info, Node: v34 1 1, Next: v34 1 0, Prev: v34 2 0, Up: History<2> + +9.135 v34.1.1 +============= + +03 Feb 2017 + + * #953(1): More aggressively employ the compatibility issue + originally added in #706(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/953 + + (2) https://github.com/pypa/setuptools/issues/706 + + +File: setuptools.info, Node: v34 1 0, Next: v34 0 3, Prev: v34 1 1, Up: History<2> + +9.136 v34.1.0 +============= + +28 Jan 2017 + + * #930(1): ‘build_info’ now accepts two new parameters to optimize + and customize the building of C libraries. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/930 + + +File: setuptools.info, Node: v34 0 3, Next: v34 0 2, Prev: v34 1 0, Up: History<2> + +9.137 v34.0.3 +============= + +28 Jan 2017 + + * #947(1): Loosen restriction on the version of six required, + restoring compatibility with environments relying on six 1.6.0 and + later. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/947 + + +File: setuptools.info, Node: v34 0 2, Next: v34 0 1, Prev: v34 0 3, Up: History<2> + +9.138 v34.0.2 +============= + +24 Jan 2017 + + * #882(1): Ensure extras are honored when building the working set. + + * #913(2): Fix issue in develop if package directory has a trailing + slash. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/882 + + (2) https://github.com/pypa/setuptools/issues/913 + + +File: setuptools.info, Node: v34 0 1, Next: v34 0 0, Prev: v34 0 2, Up: History<2> + +9.139 v34.0.1 +============= + +23 Jan 2017 + + * #935(1): Fix glob syntax in graft. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/935 + + +File: setuptools.info, Node: v34 0 0, Next: v33 1 1, Prev: v34 0 1, Up: History<2> + +9.140 v34.0.0 +============= + +23 Jan 2017 + + * #581(1): Instead of vendoring the growing list of dependencies that + Setuptools requires to function, Setuptools now requires these + dependencies just like any other project. Unlike other projects, + however, Setuptools cannot rely on ‘setup_requires’ to demand the + dependencies it needs to install because its own machinery would be + necessary to pull those dependencies if not present (a + bootstrapping problem). As a result, Setuptools no longer supports + self upgrade or installation in the general case. Instead, users + are directed to use pip to install and upgrade using the ‘wheel’ + distributions of setuptools. + + Users are welcome to contrive other means to install or upgrade + Setuptools using other means, such as pre-installing the Setuptools + dependencies with pip or a bespoke bootstrap tool, but such usage + is not recommended and is not supported. + + As discovered in #940(2), not all versions of pip will successfully + install Setuptools from its pre-built wheel. If you encounter + issues with “No module named six” or “No module named packaging”, + especially following a line “Running setup.py egg_info for package + setuptools”, then your pip is not new enough. + + There’s an additional issue in pip where setuptools is upgraded + concurrently with other source packages, described in pip #4253(3). + The proposed workaround is to always upgrade Setuptools first prior + to upgrading other packages that would upgrade Setuptools. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/581 + + (2) https://github.com/pypa/setuptools/issues/940 + + (3) https://github.com/pypa/setuptools/issues/4253 + + +File: setuptools.info, Node: v33 1 1, Next: v33 1 0, Prev: v34 0 0, Up: History<2> + +9.141 v33.1.1 +============= + +16 Jan 2017 + + * #921(1): Correct issue where certifi fallback not being reached on + Windows. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/921 + + +File: setuptools.info, Node: v33 1 0, Next: v33 0 0, Prev: v33 1 1, Up: History<2> + +9.142 v33.1.0 +============= + +15 Jan 2017 + +Installation via pip, as indicated in the Python Packaging User’s +Guide(1), is the officially-supported mechanism for installing +Setuptools, and this recommendation is now explicit in the much more +concise README. + +Other edits and tweaks were made to the documentation. The codebase is +unchanged. + + ---------- Footnotes ---------- + + (1) https://packaging.python.org/installing/ + + +File: setuptools.info, Node: v33 0 0, Next: v32 3 1, Prev: v33 1 0, Up: History<2> + +9.143 v33.0.0 +============= + +01 Jan 2017 + + * #619(1): Removed support for the ‘tag_svn_revision’ distribution + option. If Subversion tagging support is still desired, consider + adding the functionality to setuptools_svn in setuptools_svn #2(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/619 + + (2) https://github.com/jaraco/setuptools_svn/issues/2 + + +File: setuptools.info, Node: v32 3 1, Next: v32 3 0, Prev: v33 0 0, Up: History<2> + +9.144 v32.3.1 +============= + +28 Dec 2016 + + * #866(1): Use ‘dis.Bytecode’ on Python 3.4 and later in + ‘setuptools.depends’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/866 + + +File: setuptools.info, Node: v32 3 0, Next: v32 2 0, Prev: v32 3 1, Up: History<2> + +9.145 v32.3.0 +============= + +24 Dec 2016 + + * #889(1): Backport proposed fix for disabling interpolation in + distutils.Distribution.parse_config_files. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/889 + + +File: setuptools.info, Node: v32 2 0, Next: v32 1 3, Prev: v32 3 0, Up: History<2> + +9.146 v32.2.0 +============= + +22 Dec 2016 + + * #884(1): Restore support for running the tests under + pytest-runner(2) by ensuring that PYTHONPATH is honored in tests + invoking a subprocess. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/884 + + (2) https://github.com/pytest-dev/pytest-runner + + +File: setuptools.info, Node: v32 1 3, Next: v32 1 2, Prev: v32 2 0, Up: History<2> + +9.147 v32.1.3 +============= + +21 Dec 2016 + + * #706(1): Add rmtree compatibility shim for environments where + rmtree fails when passed a unicode string. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/706 + + +File: setuptools.info, Node: v32 1 2, Next: v32 1 1, Prev: v32 1 3, Up: History<2> + +9.148 v32.1.2 +============= + +18 Dec 2016 + + * #893(1): Only release sdist in zip format as warehouse now + disallows releasing two different formats. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/893 + + +File: setuptools.info, Node: v32 1 1, Next: v32 1 0, Prev: v32 1 2, Up: History<2> + +9.149 v32.1.1 +============= + +18 Dec 2016 + + * #704(1): More selectively ensure that ‘rmtree’ is not invoked with + a byte string, enabling it to remove files that are non-ascii, even + on Python 2. + + * #712(2): In ‘sandbox.run_setup’, ensure that ‘__file__’ is always a + ‘str’, modeling the behavior observed by the interpreter when + invoking scripts and modules. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/704 + + (2) https://github.com/pypa/setuptools/issues/712 + + +File: setuptools.info, Node: v32 1 0, Next: v32 0 0, Prev: v32 1 1, Up: History<2> + +9.150 v32.1.0 +============= + +16 Dec 2016 + + * #891(1): In ‘test’ command on test failure, raise DistutilsError, + suppression invocation of subsequent commands. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/891 + + +File: setuptools.info, Node: v32 0 0, Next: v31 0 1, Prev: v32 1 0, Up: History<2> + +9.151 v32.0.0 +============= + +14 Dec 2016 + + * #890(1): Revert #849(2). ‘global-exclude .foo’ will not match all + ‘*.foo’ files any more. Package authors must add an explicit + wildcard, such as ‘global-exclude *.foo’, to match all ‘.foo’ + files. See #886(3), #849(4). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/890 + + (2) https://github.com/pypa/setuptools/issues/849 + + (3) https://github.com/pypa/setuptools/issues/886 + + (4) https://github.com/pypa/setuptools/issues/849 + + +File: setuptools.info, Node: v31 0 1, Next: v31 0 0, Prev: v32 0 0, Up: History<2> + +9.152 v31.0.1 +============= + +14 Dec 2016 + + * #885(1): Fix regression where ‘pkg_resources._rebuild_mod_path’ + would fail when a namespace package’s ‘__path__’ was not a list + with a sort attribute. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/885 + + +File: setuptools.info, Node: v31 0 0, Next: v30 4 0, Prev: v31 0 1, Up: History<2> + +9.153 v31.0.0 +============= + +11 Dec 2016 + + * #250(1): Install ‘-nspkg.pth’ files for packages installed with + ‘setup.py develop’. These .pth files allow namespace packages + installed by pip or develop to co-mingle. This change required the + removal of the change for #805(2) and pip #1924(3), introduced in + 28.3.0 and implicated in #870(4), but means that namespace packages + not in a site packages directory will no longer work on Python + earlier than 3.5, whereas before they would work on Python not + earlier than 3.3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/250 + + (2) https://github.com/pypa/setuptools/issues/805 + + (3) https://github.com/pypa/setuptools/issues/1924 + + (4) https://github.com/pypa/setuptools/issues/870 + + +File: setuptools.info, Node: v30 4 0, Next: v30 3 0, Prev: v31 0 0, Up: History<2> + +9.154 v30.4.0 +============= + +10 Dec 2016 + + * #879(1): For declarative config: + + - read_configuration() now accepts ignore_option_errors + argument. This allows scraping tools to read metadata without + a need to download entire packages. E.g. we can gather some + stats right from GitHub repos just by downloading setup.cfg. + + - packages find: directive now supports fine tuning from a + subsection. The same arguments as for find() are accepted. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/879 + + +File: setuptools.info, Node: v30 3 0, Next: v30 2 1, Prev: v30 4 0, Up: History<2> + +9.155 v30.3.0 +============= + +08 Dec 2016 + + * #394(1) via #862(2): Added support for declarative package config + in a setup.cfg file(3). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/394 + + (2) https://github.com/pypa/setuptools/issues/862 + + (3) +https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files + + +File: setuptools.info, Node: v30 2 1, Next: v30 2 0, Prev: v30 3 0, Up: History<2> + +9.156 v30.2.1 +============= + +08 Dec 2016 + + * #850(1): In test command, invoke unittest.main with indication not + to exit the process. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/850 + + +File: setuptools.info, Node: v30 2 0, Next: v30 1 0, Prev: v30 2 1, Up: History<2> + +9.157 v30.2.0 +============= + +04 Dec 2016 + + * #854(1): Bump to vendored Packaging 16.8(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/854 + + (2) https://github.com/pypa/packaging/blob/16.8/CHANGELOG.rst + + +File: setuptools.info, Node: v30 1 0, Next: v30 0 0, Prev: v30 2 0, Up: History<2> + +9.158 v30.1.0 +============= + +03 Dec 2016 + + * #846(1): Also trap ‘socket.error’ when opening URLs in + package_index. + + * #849(2): Manifest processing now matches the filename pattern + anywhere in the filename and not just at the start. Restores + behavior found prior to 28.5.0. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/846 + + (2) https://github.com/pypa/setuptools/issues/849 + + +File: setuptools.info, Node: v30 0 0, Next: v29 0 1, Prev: v30 1 0, Up: History<2> + +9.159 v30.0.0 +============= + +01 Dec 2016 + + * #864(1): Drop support for Python 3.2. Systems requiring Python 3.2 + support must use ‘setuptools < 30’. + + * #825(2): Suppress warnings for single files. + + * #830(3) via #843(4): Once again restored inclusion of data files to + sdists, but now trap TypeError caused by techniques employed rjsmin + and similar. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/864 + + (2) https://github.com/pypa/setuptools/issues/825 + + (3) https://github.com/pypa/setuptools/issues/830 + + (4) https://github.com/pypa/setuptools/issues/843 + + +File: setuptools.info, Node: v29 0 1, Next: v29 0 0, Prev: v30 0 0, Up: History<2> + +9.160 v29.0.1 +============= + +26 Nov 2016 + + * #861(1): Re-release of v29.0.1 with the executable script launchers + bundled. Now, launchers are included by default and users that + want to disable this behavior must set the environment variable + ‘SETUPTOOLS_INSTALL_WINDOWS_SPECIFIC_FILES’ to a false value like + “false” or “0”. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/861 + + +File: setuptools.info, Node: v29 0 0, Next: v28 8 0, Prev: v29 0 1, Up: History<2> + +9.161 v29.0.0 +============= + +25 Nov 2016 + + * #841(1): Drop special exception for packages invoking win32com + during the build/install process. See Distribute #118(2) for + history. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/841 + + (2) https://bitbucket.org/tarek/distribute/issue/118 + + +File: setuptools.info, Node: v28 8 0, Next: v28 7 1, Prev: v29 0 0, Up: History<2> + +9.162 v28.8.0 +============= + +04 Nov 2016 + + * #629(1): Per the discussion, refine the sorting to use version + value order for more accurate detection of the latest available + version when scanning for packages. See also #829(2). + + * #837(3): Rely on the config var “SO” for Python 3.3.0 only when + determining the ext filename. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/629 + + (2) https://github.com/pypa/setuptools/issues/829 + + (3) https://github.com/pypa/setuptools/issues/837 + + +File: setuptools.info, Node: v28 7 1, Next: v28 7 0, Prev: v28 8 0, Up: History<2> + +9.163 v28.7.1 +============= + +29 Oct 2016 + + * #827(1): Update PyPI root for dependency links. + + * #833(2): Backed out changes from #830(3) as the implementation + seems to have problems in some cases. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/827 + + (2) https://github.com/pypa/setuptools/issues/833 + + (3) https://github.com/pypa/setuptools/issues/830 + + +File: setuptools.info, Node: v28 7 0, Next: v28 6 1, Prev: v28 7 1, Up: History<2> + +9.164 v28.7.0 +============= + +28 Oct 2016 + + * #832(1): Moved much of the namespace package handling functionality + into a separate module for re-use in something like #789(2). + + * #830(3): ‘sdist’ command no longer suppresses the inclusion of data + files, re-aligning with the expectation of distutils and addressing + #274(4) and #521(5). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/832 + + (2) https://github.com/pypa/setuptools/issues/789 + + (3) https://github.com/pypa/setuptools/issues/830 + + (4) https://github.com/pypa/setuptools/issues/274 + + (5) https://github.com/pypa/setuptools/issues/521 + + +File: setuptools.info, Node: v28 6 1, Next: v28 6 0, Prev: v28 7 0, Up: History<2> + +9.165 v28.6.1 +============= + +19 Oct 2016 + + * #816(1): Fix manifest file list order in tests. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/816 + + +File: setuptools.info, Node: v28 6 0, Next: v28 5 0, Prev: v28 6 1, Up: History<2> + +9.166 v28.6.0 +============= + +16 Oct 2016 + + * #629(1): When scanning for packages, ‘pkg_resources’ now ignores + empty egg-info directories and gives precedence to packages whose + versions are lexicographically greatest, a rough approximation for + preferring the latest available version. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/629 + + +File: setuptools.info, Node: v28 5 0, Next: v28 4 0, Prev: v28 6 0, Up: History<2> + +9.167 v28.5.0 +============= + +14 Oct 2016 + + * #810(1): Tests are now invoked with tox and not setup.py test. + + * #249(2) and #450(3) via #764(4): Avoid scanning the whole tree when + building the manifest. Also fixes a long-standing bug where + patterns in ‘MANIFEST.in’ had implicit wildcard matching. This + caused ‘global-exclude .foo’ to exclude all ‘*.foo’ files, but also + ‘global-exclude bar.py’ to exclude ‘foo_bar.py’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/810 + + (2) https://github.com/pypa/setuptools/issues/249 + + (3) https://github.com/pypa/setuptools/issues/450 + + (4) https://github.com/pypa/setuptools/issues/764 + + +File: setuptools.info, Node: v28 4 0, Next: v28 3 0, Prev: v28 5 0, Up: History<2> + +9.168 v28.4.0 +============= + +14 Oct 2016 + + * #732(1): Now extras with a hyphen are honored per PEP 426(2). + + * #811(3): Update to pyparsing 2.1.10. + + * Updated ‘setuptools.command.sdist’ to re-use most of the + functionality directly from ‘distutils.command.sdist’ for the + ‘add_defaults’ method with strategic overrides. See #750(4) for + rationale. + + * #760(5) via #762(6): Look for certificate bundle where SUSE Linux + typically presents it. Use ‘certifi.where()’ to locate the bundle. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/732 + + (2) https://www.python.org/dev/peps/pep-0426/ + + (3) https://github.com/pypa/setuptools/issues/811 + + (4) https://github.com/pypa/setuptools/issues/750 + + (5) https://github.com/pypa/setuptools/issues/760 + + (6) https://github.com/pypa/setuptools/issues/762 + + +File: setuptools.info, Node: v28 3 0, Next: v28 1 0, Prev: v28 4 0, Up: History<2> + +9.169 v28.3.0 +============= + +07 Oct 2016 + + * #809(1): In ‘find_packages()’, restore support for excluding a + parent package without excluding a child package. + + * #805(2): Disable ‘-nspkg.pth’ behavior on Python 3.3+ where + PEP-420(3) functionality is adequate. Fixes pip #1924(4). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/809 + + (2) https://github.com/pypa/setuptools/issues/805 + + (3) https://www.python.org/dev/peps/pep-0420/ + + (4) https://github.com/pypa/setuptools/issues/1924 + + +File: setuptools.info, Node: v28 1 0, Next: v28 0 0, Prev: v28 3 0, Up: History<2> + +9.170 v28.1.0 +============= + +01 Oct 2016 + + * #803(1): Bump certifi to 2016.9.26. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/803 + + +File: setuptools.info, Node: v28 0 0, Next: v27 3 1, Prev: v28 1 0, Up: History<2> + +9.171 v28.0.0 +============= + +27 Sep 2016 + + * #733(1): Do not search excluded directories for packages. This + introduced a backwards incompatible change in ‘find_packages()’ so + that ‘find_packages(exclude=['foo']) == []’, excluding subpackages + of ‘foo’. Previously, ‘find_packages(exclude=['foo']) == + ['foo.bar']’, even though the parent ‘foo’ package was excluded. + + * #795(2): Bump certifi. + + * #719(3): Suppress decoding errors and instead log a warning when + metadata cannot be decoded. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/733 + + (2) https://github.com/pypa/setuptools/issues/795 + + (3) https://github.com/pypa/setuptools/issues/719 + + +File: setuptools.info, Node: v27 3 1, Next: v27 3 0, Prev: v28 0 0, Up: History<2> + +9.172 v27.3.1 +============= + +27 Sep 2016 + + * #790(1): In MSVC monkeypatching, explicitly patch each function by + name in the target module instead of inferring the module from the + function’s ‘__module__’. Improves compatibility with other + packages that might have previously patched distutils functions + (i.e. NumPy). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/790 + + +File: setuptools.info, Node: v27 3 0, Next: v27 2 0, Prev: v27 3 1, Up: History<2> + +9.173 v27.3.0 +============= + +20 Sep 2016 + + * #794(1): In test command, add installed eggs to PYTHONPATH when + invoking tests so that subprocesses will also have the dependencies + available. Fixes tox 330(2). + + * #795(3): Update vendored pyparsing 2.1.9. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/794 + + (2) https://github.com/tox-dev/tox/issues/330 + + (3) https://github.com/pypa/setuptools/issues/795 + + +File: setuptools.info, Node: v27 2 0, Next: v27 1 2, Prev: v27 3 0, Up: History<2> + +9.174 v27.2.0 +============= + +14 Sep 2016 + + * #520(1) and #513(2): Suppress ValueErrors in + fixup_namespace_packages when lookup fails. + + * Nicer, more consistent interfaces for msvc monkeypatching. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/520 + + (2) https://github.com/pypa/setuptools/issues/513 + + +File: setuptools.info, Node: v27 1 2, Next: v27 1 1, Prev: v27 2 0, Up: History<2> + +9.175 v27.1.2 +============= + +09 Sep 2016 + + * #779(1) via #781(2): Fix circular import. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/779 + + (2) https://github.com/pypa/setuptools/issues/781 + + +File: setuptools.info, Node: v27 1 1, Next: v27 1 0, Prev: v27 1 2, Up: History<2> + +9.176 v27.1.1 +============= + +09 Sep 2016 + + * #778(1): Fix MSVC monkeypatching. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/778 + + +File: setuptools.info, Node: v27 1 0, Next: v27 0 0, Prev: v27 1 1, Up: History<2> + +9.177 v27.1.0 +============= + +09 Sep 2016 + + * Introduce the (private) ‘monkey’ module to encapsulate the + distutils monkeypatching behavior. + + +File: setuptools.info, Node: v27 0 0, Next: v26 1 1, Prev: v27 1 0, Up: History<2> + +9.178 v27.0.0 +============= + +09 Sep 2016 + + * Now use Warehouse by default for ‘upload’, patching + ‘distutils.config.PyPIRCCommand’ to affect default behavior. + + Any config in .pypirc should be updated to replace + + ‘https://pypi.python.org/pypi/’ + + with + + ‘https://upload.pypi.org/legacy/’ + + Similarly, any passwords stored in the keyring should be updated to + use this new value for “system”. + + The ‘upload_docs’ command will continue to use the python.org site, + but the command is now deprecated. Users are urged to use Read The + Docs instead. + + * #776(1): Use EXT_SUFFIX for py_limited_api renaming. + + * #774(2) and #775(3): Use LegacyVersion from packaging when + detecting numpy versions. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/776 + + (2) https://github.com/pypa/setuptools/issues/774 + + (3) https://github.com/pypa/setuptools/issues/775 + + +File: setuptools.info, Node: v26 1 1, Next: v26 1 0, Prev: v27 0 0, Up: History<2> + +9.179 v26.1.1 +============= + +29 Aug 2016 + + * Re-release of 26.1.0 with pytest pinned to allow for automated + deployment and thus proper packaging environment variables, fixing + issues with missing executable launchers. + + +File: setuptools.info, Node: v26 1 0, Next: v26 0 0, Prev: v26 1 1, Up: History<2> + +9.180 v26.1.0 +============= + +28 Aug 2016 + + * #763(1): ‘pkg_resources.get_default_cache’ now defers to the + appdirs project(2) to resolve the cache directory. Adds a vendored + dependency on appdirs to pkg_resources. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/763 + + (2) https://pypi.org/project/appdirs + + +File: setuptools.info, Node: v26 0 0, Next: v25 4 0, Prev: v26 1 0, Up: History<2> + +9.181 v26.0.0 +============= + +20 Aug 2016 + + * #748(1): By default, sdists are now produced in gzipped tarfile + format by default on all platforms, adding forward compatibility + for the same behavior in Python 3.6 (See Python #27819(2)). + + * #459(3) via #736(4): On Windows with script launchers, sys.argv[0] + now reflects the name of the entry point, consistent with the + behavior in distlib and pip wrappers. + + * #752(5) via #753(6): When indicating ‘py_limited_api’ to Extension, + it must be passed as a keyword argument. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/748 + + (2) http://bugs.python.org/issue27819 + + (3) https://github.com/pypa/setuptools/issues/459 + + (4) https://github.com/pypa/setuptools/issues/736 + + (5) https://github.com/pypa/setuptools/issues/752 + + (6) https://github.com/pypa/setuptools/issues/753 + + +File: setuptools.info, Node: v25 4 0, Next: v25 3 0, Prev: v26 0 0, Up: History<2> + +9.182 v25.4.0 +============= + +19 Aug 2016 + + * Add Extension(py_limited_api=True). When set to a truthy value, + that extension gets a filename appropriate for code using + Py_LIMITED_API. When used correctly this allows a single compiled + extension to work on all future versions of CPython 3. The + py_limited_api argument only controls the filename. To be + compatible with multiple versions of Python 3, the C extension will + also need to set -DPy_LIMITED_API=… and be modified to use only the + functions in the limited API. + + +File: setuptools.info, Node: v25 3 0, Next: v25 2 0, Prev: v25 4 0, Up: History<2> + +9.183 v25.3.0 +============= + +19 Aug 2016 + + * #739(1) Fix unquoted libpaths by fixing compatibility between + ‘numpy.distutils’ and ‘distutils._msvccompiler’ for numpy < 1.11.2 + (Fix issue #728(2), error also fixed in Numpy). + + * #731(3): Bump certifi. + + * Style updates. See #740(4), #741(5), #743(6), #744(7), #742(8), + #747(9). + + * #735(10): include license file. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/739 + + (2) https://github.com/pypa/setuptools/issues/728 + + (3) https://github.com/pypa/setuptools/issues/731 + + (4) https://github.com/pypa/setuptools/issues/740 + + (5) https://github.com/pypa/setuptools/issues/741 + + (6) https://github.com/pypa/setuptools/issues/743 + + (7) https://github.com/pypa/setuptools/issues/744 + + (8) https://github.com/pypa/setuptools/issues/742 + + (9) https://github.com/pypa/setuptools/issues/747 + + (10) https://github.com/pypa/setuptools/issues/735 + + +File: setuptools.info, Node: v25 2 0, Next: v25 1 6, Prev: v25 3 0, Up: History<2> + +9.184 v25.2.0 +============= + +12 Aug 2016 + + * #612(1) via #730(2): Add a LICENSE file which needs to be provided + by the terms of the MIT license. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/612 + + (2) https://github.com/pypa/setuptools/issues/730 + + +File: setuptools.info, Node: v25 1 6, Next: v25 1 5, Prev: v25 2 0, Up: History<2> + +9.185 v25.1.6 +============= + +05 Aug 2016 + + * #725(1): revert ‘library_dir_option’ patch (Error is related to + ‘numpy.distutils’ and make errors on non Numpy users). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/725 + + +File: setuptools.info, Node: v25 1 5, Next: v25 1 4, Prev: v25 1 6, Up: History<2> + +9.186 v25.1.5 +============= + +05 Aug 2016 + + * #720(1) + + * #723(2): Improve patch for ‘library_dir_option’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/720 + + (2) https://github.com/pypa/setuptools/issues/723 + + +File: setuptools.info, Node: v25 1 4, Next: v25 1 3, Prev: v25 1 5, Up: History<2> + +9.187 v25.1.4 +============= + +04 Aug 2016 + + * #717(1) + + * #713(2) + + * #707(3): Fix Python 2 compatibility for MSVC by catching errors + properly. + + * #715(4): Fix unquoted libpaths by patching ‘library_dir_option’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/717 + + (2) https://github.com/pypa/setuptools/issues/713 + + (3) https://github.com/pypa/setuptools/issues/707 + + (4) https://github.com/pypa/setuptools/issues/715 + + +File: setuptools.info, Node: v25 1 3, Next: v25 1 2, Prev: v25 1 4, Up: History<2> + +9.188 v25.1.3 +============= + +02 Aug 2016 + + * #714(1) and #704(2): Revert fix as it breaks other components + downstream that can’t handle unicode. See #709(3), #710(4), and + #712(5). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/714 + + (2) https://github.com/pypa/setuptools/issues/704 + + (3) https://github.com/pypa/setuptools/issues/709 + + (4) https://github.com/pypa/setuptools/issues/710 + + (5) https://github.com/pypa/setuptools/issues/712 + + +File: setuptools.info, Node: v25 1 2, Next: v25 1 1, Prev: v25 1 3, Up: History<2> + +9.189 v25.1.2 +============= + +01 Aug 2016 + + * #704(1): Fix errors when installing a zip sdist that contained + files named with non-ascii characters on Windows would crash the + install when it attempted to clean up the build. + + * #646(2): MSVC compatibility - catch errors properly in + RegistryInfo.lookup. + + * #702(3): Prevent UnboundLocalError when initial working_set is + empty. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/704 + + (2) https://github.com/pypa/setuptools/issues/646 + + (3) https://github.com/pypa/setuptools/issues/702 + + +File: setuptools.info, Node: v25 1 1, Next: v25 1 0, Prev: v25 1 2, Up: History<2> + +9.190 v25.1.1 +============= + +28 Jul 2016 + + * #686(1): Fix issue in sys.path ordering by pkg_resources when + rewrite technique is “raw”. + + * #699(2): Fix typo in msvc support. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/686 + + (2) https://github.com/pypa/setuptools/issues/699 + + +File: setuptools.info, Node: v25 1 0, Next: v25 0 2, Prev: v25 1 1, Up: History<2> + +9.191 v25.1.0 +============= + +25 Jul 2016 + + * #609(1): Setuptools will now try to download a distribution from + the next possible download location if the first download fails. + This means you can now specify multiple links as ‘dependency_links’ + and all links will be tried until a working download link is + encountered. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/609 + + +File: setuptools.info, Node: v25 0 2, Next: v25 0 1, Prev: v25 1 0, Up: History<2> + +9.192 v25.0.2 +============= + +25 Jul 2016 + + * #688(1): Fix AttributeError in setup.py when invoked not from the + current directory. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/688 + + +File: setuptools.info, Node: v25 0 1, Next: v25 0 0, Prev: v25 0 2, Up: History<2> + +9.193 v25.0.1 +============= + +25 Jul 2016 + + * Cleanup of setup.py script. + + * Fixed documentation builders by allowing setup.py to be imported + without having bootstrapped the metadata. + + * More style cleanup. See #677(1), #678(2), #679(3), #681(4), + #685(5). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/677 + + (2) https://github.com/pypa/setuptools/issues/678 + + (3) https://github.com/pypa/setuptools/issues/679 + + (4) https://github.com/pypa/setuptools/issues/681 + + (5) https://github.com/pypa/setuptools/issues/685 + + +File: setuptools.info, Node: v25 0 0, Next: v24 3 1, Prev: v25 0 1, Up: History<2> + +9.194 v25.0.0 +============= + +23 Jul 2016 + + * #674(1): Default ‘sys.path’ manipulation by easy-install.pth is now + “raw”, meaning that when writing easy-install.pth during any + install operation, the ‘sys.path’ will not be rewritten and will no + longer give preference to easy_installed packages. + + To retain the old behavior when using any easy_install operation + (including ‘setup.py install’ when setuptools is present), set the + environment variable: + + SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite + + This project hopes that that few if any environments find it + necessary to retain the old behavior, and intends to drop support + for it altogether in a future release. Please report any relevant + concerns in the ticket for this change. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/674 + + +File: setuptools.info, Node: v24 3 1, Next: v24 3 0, Prev: v25 0 0, Up: History<2> + +9.195 v24.3.1 +============= + +23 Jul 2016 + + * #398(1): Fix shebang handling on Windows in script headers where + spaces in ‘sys.executable’ would produce an improperly-formatted + shebang header, introduced in 12.0 with the fix for #188(2). + + * #663(3), #670(4): More style updates. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/398 + + (2) https://github.com/pypa/setuptools/issues/188 + + (3) https://github.com/pypa/setuptools/issues/663 + + (4) https://github.com/pypa/setuptools/issues/670 + + +File: setuptools.info, Node: v24 3 0, Next: v24 2 1, Prev: v24 3 1, Up: History<2> + +9.196 v24.3.0 +============= + +21 Jul 2016 + + * #516(1): Disable ‘os.link’ to avoid hard linking in + ‘sdist.make_distribution’, avoiding errors on systems that support + hard links but not on the file system in which the build is + occurring. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/516 + + +File: setuptools.info, Node: v24 2 1, Next: v24 2 0, Prev: v24 3 0, Up: History<2> + +9.197 v24.2.1 +============= + +21 Jul 2016 + + * #667(1): Update Metadata-Version to 1.2 when ‘python_requires’ is + supplied. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/667 + + +File: setuptools.info, Node: v24 2 0, Next: v24 1 1, Prev: v24 2 1, Up: History<2> + +9.198 v24.2.0 +============= + +20 Jul 2016 + + * #631(1): Add support for ‘python_requires’ keyword. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/631 + + +File: setuptools.info, Node: v24 1 1, Next: v24 1 0, Prev: v24 2 0, Up: History<2> + +9.199 v24.1.1 +============= + +20 Jul 2016 + + * More style updates. See #660(1), #661(2), #641(3). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/660 + + (2) https://github.com/pypa/setuptools/issues/661 + + (3) https://github.com/pypa/setuptools/issues/641 + + +File: setuptools.info, Node: v24 1 0, Next: v24 0 3, Prev: v24 1 1, Up: History<2> + +9.200 v24.1.0 +============= + +20 Jul 2016 + + * #659(1): ‘setup.py’ now will fail fast and with a helpful error + message when the necessary metadata is missing. + + * More style updates. See #656(2), #635(3), #640(4), #644(5), + #650(6), #652(7), and #655(8). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/659 + + (2) https://github.com/pypa/setuptools/issues/656 + + (3) https://github.com/pypa/setuptools/issues/635 + + (4) https://github.com/pypa/setuptools/issues/640 + + (5) https://github.com/pypa/setuptools/issues/644 + + (6) https://github.com/pypa/setuptools/issues/650 + + (7) https://github.com/pypa/setuptools/issues/652 + + (8) https://github.com/pypa/setuptools/issues/655 + + +File: setuptools.info, Node: v24 0 3, Next: v24 0 2, Prev: v24 1 0, Up: History<2> + +9.201 v24.0.3 +============= + +14 Jul 2016 + + * Updated style in much of the codebase to match community + expectations. See #632(1), #633(2), #634(3), #637(4), #639(5), + #638(6), #642(7), #648(8). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/632 + + (2) https://github.com/pypa/setuptools/issues/633 + + (3) https://github.com/pypa/setuptools/issues/634 + + (4) https://github.com/pypa/setuptools/issues/637 + + (5) https://github.com/pypa/setuptools/issues/639 + + (6) https://github.com/pypa/setuptools/issues/638 + + (7) https://github.com/pypa/setuptools/issues/642 + + (8) https://github.com/pypa/setuptools/issues/648 + + +File: setuptools.info, Node: v24 0 2, Next: v24 0 1, Prev: v24 0 3, Up: History<2> + +9.202 v24.0.2 +============= + +04 Jul 2016 + + * If MSVC++14 is needed ‘setuptools.msvc’ now redirect user to Visual + C++ Build Tools web page. + + +File: setuptools.info, Node: v24 0 1, Next: v24 0 0, Prev: v24 0 2, Up: History<2> + +9.203 v24.0.1 +============= + +03 Jul 2016 + + * #625(1) and #626(2): Fixes on ‘setuptools.msvc’ mainly for Python 2 + and Linux. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/625 + + (2) https://github.com/pypa/setuptools/issues/626 + + +File: setuptools.info, Node: v24 0 0, Next: v23 2 1, Prev: v24 0 1, Up: History<2> + +9.204 v24.0.0 +============= + +02 Jul 2016 + + * Pull Request #174(1): Add more aggressive support for standalone + Microsoft Visual C++ compilers in msvc9compiler patch. + Particularly : Windows SDK 6.1 and 7.0 (MSVC++ 9.0), Windows SDK + 7.1 (MSVC++ 10.0), Visual C++ Build Tools 2015 (MSVC++14) + + * Renamed ‘setuptools.msvc9_support’ to ‘setuptools.msvc’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/174 + + +File: setuptools.info, Node: v23 2 1, Next: v23 1 0, Prev: v24 0 0, Up: History<2> + +9.205 v23.2.1 +============= + +02 Jul 2016 + +Re-release of v23.2.0, which was missing the intended commits. + + * #623(1): Remove used of deprecated ‘U’ flag when reading manifests. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/623 + + +File: setuptools.info, Node: v23 1 0, Next: v23 0 0, Prev: v23 2 1, Up: History<2> + +9.206 v23.1.0 +============= + +24 Jun 2016 + + * #619(1): Deprecated ‘tag_svn_revision’ distribution option. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/619 + + +File: setuptools.info, Node: v23 0 0, Next: v22 0 5, Prev: v23 1 0, Up: History<2> + +9.207 v23.0.0 +============= + +09 Jun 2016 + + * #611(1): Removed ARM executables for CLI and GUI script launchers + on Windows. If this was a feature you cared about, please comment + in the ticket. + + * #604(2): Removed docs building support. The project now relies on + documentation hosted at ‘https://setuptools.readthedocs.io/’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/611 + + (2) https://github.com/pypa/setuptools/issues/604 + + +File: setuptools.info, Node: v22 0 5, Next: v22 0 4, Prev: v23 0 0, Up: History<2> + +9.208 v22.0.5 +============= + +03 Jun 2016 + + * #604(1): Restore repository for upload_docs command to restore + publishing of docs during release. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/604 + + +File: setuptools.info, Node: v22 0 4, Next: v22 0 3, Prev: v22 0 5, Up: History<2> + +9.209 v22.0.4 +============= + +03 Jun 2016 + + * #589(1): Upload releases to pypi.io using the upload hostname and + legacy path. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/589 + + +File: setuptools.info, Node: v22 0 3, Next: v22 0 2, Prev: v22 0 4, Up: History<2> + +9.210 v22.0.3 +============= + +03 Jun 2016 + + * #589(1): Releases are now uploaded to pypi.io (Warehouse) even when + releases are made on Twine via Travis. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/589 + + +File: setuptools.info, Node: v22 0 2, Next: v22 0 1, Prev: v22 0 3, Up: History<2> + +9.211 v22.0.2 +============= + +03 Jun 2016 + + * #589(1): Releases are now uploaded to pypi.io (Warehouse). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/589 + + +File: setuptools.info, Node: v22 0 1, Next: v22 0 0, Prev: v22 0 2, Up: History<2> + +9.212 v22.0.1 +============= + +03 Jun 2016 + + * #190(1): On Python 2, if unicode is passed for packages to + ‘build_py’ command, it will be handled just as with text on Python + 3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/190 + + +File: setuptools.info, Node: v22 0 0, Next: v21 2 2, Prev: v22 0 1, Up: History<2> + +9.213 v22.0.0 +============= + +01 Jun 2016 + +Intended to be v21.3.0, but jaraco accidentally released as a major +bump. + + * #598(1): Setuptools now lists itself first in the User-Agent for + web requests, better following the guidelines in RFC 7231(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/598 + + (2) https://tools.ietf.org/html/rfc7231#section-5.5.3 + + +File: setuptools.info, Node: v21 2 2, Next: v21 2 1, Prev: v22 0 0, Up: History<2> + +9.214 v21.2.2 +============= + +29 May 2016 + + * Minor fixes to changelog and docs. + + +File: setuptools.info, Node: v21 2 1, Next: v21 2 0, Prev: v21 2 2, Up: History<2> + +9.215 v21.2.1 +============= + +22 May 2016 + + * #261(1): Exclude directories when resolving globs in package_data. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/261 + + +File: setuptools.info, Node: v21 2 0, Next: v21 1 0, Prev: v21 2 1, Up: History<2> + +9.216 v21.2.0 +============= + +21 May 2016 + + * #539(1): In the easy_install get_site_dirs, honor all paths found + in ‘site.getsitepackages’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/539 + + +File: setuptools.info, Node: v21 1 0, Next: v21 0 0, Prev: v21 2 0, Up: History<2> + +9.217 v21.1.0 +============= + +18 May 2016 + + * #572(1): In build_ext, now always import ‘_CONFIG_VARS’ from + ‘distutils’ rather than from ‘sysconfig’ to allow + ‘distutils.sysconfig.customize_compiler’ configure the OS X + compiler for ‘-dynamiclib’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/572 + + +File: setuptools.info, Node: v21 0 0, Next: v20 10 0, Prev: v21 1 0, Up: History<2> + +9.218 v21.0.0 +============= + +02 May 2016 + + * Removed ez_setup.py from Setuptools sdist. The bootstrap script + will be maintained in its own branch and should be generally be + retrieved from its canonical location at + ‘https://bootstrap.pypa.io/ez_setup.py’. + + +File: setuptools.info, Node: v20 10 0, Next: v20 9 0, Prev: v21 0 0, Up: History<2> + +9.219 v20.10.0 +============== + +25 Apr 2016 + + * #553(1): egg_info section is now generated in a deterministic + order, matching the order generated by earlier versions of Python. + Except on Python 2.6, order is preserved when existing settings are + present. + + * #556(2): Update to Packaging 16.7(3), restoring support for + deprecated ‘python_implmentation’ marker. + + * #555(4): Upload command now prompts for a password when uploading + to PyPI (or other repository) if no password is present in .pypirc + or in the keyring. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/553 + + (2) https://github.com/pypa/setuptools/issues/556 + + (3) https://github.com/pypa/packaging/blob/16.7/CHANGELOG.rst + + (4) https://github.com/pypa/setuptools/issues/555 + + +File: setuptools.info, Node: v20 9 0, Next: v20 8 1, Prev: v20 10 0, Up: History<2> + +9.220 v20.9.0 +============= + +16 Apr 2016 + + * #548(1): Update certify version to 2016.2.28 + + * #545(2): Safely handle deletion of non-zip eggs in rotate command. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/548 + + (2) https://github.com/pypa/setuptools/issues/545 + + +File: setuptools.info, Node: v20 8 1, Next: v20 8 0, Prev: v20 9 0, Up: History<2> + +9.221 v20.8.1 +============= + +15 Apr 2016 + + * Issue #544(1): Fix issue with extra environment marker processing + in WorkingSet due to refactor in v20.7.0. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/544 + + +File: setuptools.info, Node: v20 8 0, Next: v20 7 0, Prev: v20 8 1, Up: History<2> + +9.222 v20.8.0 +============= + +15 Apr 2016 + + * Issue #543(1): Re-release so that latest release doesn’t cause déjà + vu with distribute and setuptools 0.7 in older environments. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/543 + + +File: setuptools.info, Node: v20 7 0, Next: v20 6 8, Prev: v20 8 0, Up: History<2> + +9.223 v20.7.0 +============= + +10 Apr 2016 + + * Refactored extra environment marker processing in WorkingSet. + + * Issue #533(1): Fixed intermittent test failures. + + * Issue #536(2): In msvc9_support, trap additional exceptions that + might occur when importing ‘distutils.msvc9compiler’ in mingw + environments. + + * Issue #537(3): Provide better context when package metadata fails + to decode in UTF-8. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/533 + + (2) https://github.com/pypa/setuptools/issues/536 + + (3) https://github.com/pypa/setuptools/issues/537 + + +File: setuptools.info, Node: v20 6 8, Next: v20 6 7, Prev: v20 7 0, Up: History<2> + +9.224 v20.6.8 +============= + +09 May 2016 + + * Issue #523(1): Restored support for environment markers, now + honoring ‘extra’ environment markers. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/523 + + +File: setuptools.info, Node: v20 6 7, Next: v20 6 6, Prev: v20 6 8, Up: History<2> + +9.225 v20.6.7 +============= + +31 Mar 2016 + + * Issue #523(1): Disabled support for environment markers introduced + in v20.5. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/523 + + +File: setuptools.info, Node: v20 6 6, Next: v20 6 0, Prev: v20 6 7, Up: History<2> + +9.226 v20.6.6 +============= + +30 Mar 2016 + + * Issue #503(1): Restore support for PEP 345(2) environment markers + by updating to Packaging 16.6(3). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/503 + + (2) https://www.python.org/dev/peps/pep-0345/ + + (3) https://github.com/pypa/packaging/blob/16.6/CHANGELOG.rst + + +File: setuptools.info, Node: v20 6 0, Next: 20 5, Prev: v20 6 6, Up: History<2> + +9.227 v20.6.0 +============= + +29 Mar 2016 + + * New release process that relies on bumpversion(1) and Travis CI for + continuous deployment. + + * Project versioning semantics now follow semver(2) precisely. The + ‘v’ prefix on version numbers now also allows version numbers to be + referenced in the changelog, e.g. + ‘http://setuptools.readthedocs.io/en/latest/history.html#v20-6-0’. + + ---------- Footnotes ---------- + + (1) https://github.com/peritus/bumpversion + + (2) https://semver.org + + +File: setuptools.info, Node: 20 5, Next: 20 4, Prev: v20 6 0, Up: History<2> + +9.228 20.5 +========== + +29 Mar 2016 + + * BB Pull Request #185(1), #470(2): Add support for environment + markers in requirements in install_requires, setup_requires, + tests_require as well as adding a test for the existing + extra_requires machinery. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/185 + + (2) https://github.com/pypa/setuptools/issues/470 + + +File: setuptools.info, Node: 20 4, Next: 20 3 1, Prev: 20 5, Up: History<2> + +9.229 20.4 +========== + +29 Mar 2016 + + * Issue #422(1): Moved hosting to Github(2) from Bitbucket(3). + Issues have been migrated, though all issues and comments are + attributed to bb-migration. So if you have a particular issue or + issues to which you’ve been subscribed, you will want to “watch” + the equivalent issue in Github. The Bitbucket project will be + retained for the indefinite future, but Github now hosts the + canonical project repository. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/422 + + (2) https://github.com/pypa/setuptools + + (3) https://bitbucket.org/pypa/setuptools + + +File: setuptools.info, Node: 20 3 1, Next: 20 3, Prev: 20 4, Up: History<2> + +9.230 20.3.1 +============ + +18 Mar 2016 + + * Issue #519(1): Remove import hook when reloading the + ‘pkg_resources’ module. + + * BB Pull Request #184(2): Update documentation in ‘pkg_resources’ + around new ‘Requirement’ implementation. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/519 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/184 + + +File: setuptools.info, Node: 20 3, Next: 20 2 2, Prev: 20 3 1, Up: History<2> + +9.231 20.3 +========== + +15 Mar 2016 + + * BB Pull Request #179(1): ‘pkg_resources.Requirement’ objects are + now a subclass of ‘packaging.requirements.Requirement’, allowing + any environment markers and url (if any) to be affiliated with the + requirement + + * BB Pull Request #179(2): Restore use of RequirementParseError + exception unintentionally dropped in 20.2. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/179 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/179 + + +File: setuptools.info, Node: 20 2 2, Next: 20 2 1, Prev: 20 3, Up: History<2> + +9.232 20.2.2 +============ + +27 Feb 2016 + + * Issue #502(1): Correct regression in parsing of multiple version + specifiers separated by commas and spaces. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/502 + + +File: setuptools.info, Node: 20 2 1, Next: 20 2, Prev: 20 2 2, Up: History<2> + +9.233 20.2.1 +============ + +24 Feb 2016 + + * Issue #499(1): Restore compatibility for legacy versions by bumping + to packaging 16.4(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/499 + + (2) https://github.com/pypa/packaging/blob/16.4/CHANGELOG.rst + + +File: setuptools.info, Node: 20 2, Next: 20 1 1, Prev: 20 2 1, Up: History<2> + +9.234 20.2 +========== + +19 Feb 2016 + + * Changelog now includes release dates and links to PEPs. + + * BB Pull Request #173(1): Replace dual PEP 345(2) _markerlib + implementation and PEP 426(3) implementation of environment marker + support from packaging 16.1(4) and PEP 508(5). Fixes Issue + #122(6). See also BB Pull Request #175(7), BB Pull Request + #168(8), and BB Pull Request #164(9). Additionally: + + - ‘Requirement.parse’ no longer retains the order of + extras. + + - ‘parse_requirements’ now requires that all versions be + PEP-440(10) compliant, as revealed in #499(11). Packages + released with invalid local versions should be + re-released using the proper local version syntax, e.g. + ‘mypkg-1.0+myorg.1’. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/173 + + (2) https://www.python.org/dev/peps/pep-0345/ + + (3) https://www.python.org/dev/peps/pep-0426/ + + (4) https://github.com/pypa/packaging/blob/16.1/CHANGELOG.rst + + (5) https://www.python.org/dev/peps/pep-0508/ + + (6) https://github.com/pypa/setuptools/issues/122 + + (7) https://bitbucket.org/pypa/setuptools/pull-request/175 + + (8) https://bitbucket.org/pypa/setuptools/pull-request/168 + + (9) https://bitbucket.org/pypa/setuptools/pull-request/164 + + (10) https://www.python.org/dev/peps/pep-0440/ + + (11) https://github.com/pypa/setuptools/issues/499 + + +File: setuptools.info, Node: 20 1 1, Next: 20 1, Prev: 20 2, Up: History<2> + +9.235 20.1.1 +============ + +12 Feb 2016 + + * Update ‘upload_docs’ command to also honor keyring for password + resolution. + + +File: setuptools.info, Node: 20 1, Next: 20 0, Prev: 20 1 1, Up: History<2> + +9.236 20.1 +========== + +11 Feb 2016 + + * Added support for using passwords from keyring in the upload + command. See the upload docs(1) for details. + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#upload-upload-source-and-or-egg-distributions-to-pypi + + +File: setuptools.info, Node: 20 0, Next: 19 7, Prev: 20 1, Up: History<2> + +9.237 20.0 +========== + +07 Feb 2016 + + * Issue #118(1): Once again omit the package metadata (egg-info) from + the list of outputs in ‘--record’. This version of setuptools can + no longer be used to upgrade pip earlier than 6.0. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/118 + + +File: setuptools.info, Node: 19 7, Next: 19 6 2, Prev: 20 0, Up: History<2> + +9.238 19.7 +========== + +03 Feb 2016 + + * Off-project PR: 0dcee79(1) and f9bd9b9(2) For FreeBSD, also honor + root certificates from ca_root_nss(3). + + ---------- Footnotes ---------- + + (1) +https://github.com/pypa/setuptools/commit/0dcee791dfdcfacddaaec79b29f30a347a147413 + + (2) +https://github.com/pypa/setuptools/commit/f9bd9b9f5df54ef5a0bf8d16c3a889ab8c640580 + + (3) +https://github.com/pypa/setuptools/commit/3ae46c30225eb46e1f5aada1a19e88b79f04dc72 + + +File: setuptools.info, Node: 19 6 2, Next: 19 6 1, Prev: 19 7, Up: History<2> + +9.239 19.6.2 +============ + +31 Jan 2016 + + * Issue #491(1): Correct regression incurred in 19.4 where a + double-namespace package installed using pip would cause a + TypeError. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/491 + + +File: setuptools.info, Node: 19 6 1, Next: 19 6, Prev: 19 6 2, Up: History<2> + +9.240 19.6.1 +============ + +29 Jan 2016 + + * Restore compatibility for PyPy 3 compatibility lost in 19.4.1 + addressing Issue #487(1). + + * ‘setuptools.launch’ shim now loads scripts in a new namespace, + avoiding getting relative imports from the setuptools package on + Python 2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/487 + + +File: setuptools.info, Node: 19 6, Next: 19 5, Prev: 19 6 1, Up: History<2> + +9.241 19.6 +========== + +24 Jan 2016 + + * Added a new entry script ‘setuptools.launch’, implementing the shim + found in ‘pip.util.setuptools_build’. Use this command to launch + distutils-only packages under setuptools in the same way that pip + does, causing the setuptools monkeypatching of distutils to be + invoked prior to invoking a script. Useful for debugging or + otherwise installing a distutils-only package under setuptools when + pip isn’t available or otherwise does not expose the desired + functionality. For example: + + $ python -m setuptools.launch setup.py develop + + * Issue #488(1): Fix dual manifestation of Extension class in + extension packages installed as dependencies when Cython is + present. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/488 + + +File: setuptools.info, Node: 19 5, Next: 19 4 1, Prev: 19 6, Up: History<2> + +9.242 19.5 +========== + +23 Jan 2016 + + * Issue #486(1): Correct TypeError when getfilesystemencoding returns + None. + + * Issue #139(2): Clarified the license as MIT. + + * BB Pull Request #169(3): Removed special handling of command spec + in scripts for Jython. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/486 + + (2) https://github.com/pypa/setuptools/issues/139 + + (3) https://bitbucket.org/pypa/setuptools/pull-request/169 + + +File: setuptools.info, Node: 19 4 1, Next: 19 4, Prev: 19 5, Up: History<2> + +9.243 19.4.1 +============ + +23 Jan 2016 + + * Issue #487(1): Use direct invocation of ‘importlib.machinery’ in + ‘pkg_resources’ to avoid missing detection on relevant platforms. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/487 + + +File: setuptools.info, Node: 19 4, Next: 19 3, Prev: 19 4 1, Up: History<2> + +9.244 19.4 +========== + +16 Jan 2016 + + * Issue #341(1): Correct error in path handling of package data files + in ‘build_py’ command when package is empty. + + * Distribute #323(2), Issue #141(3), Issue #207(4), and BB Pull + Request #167(5): Another implementation of + ‘pkg_resources.WorkingSet’ and ‘pkg_resources.Distribution’ that + supports replacing an extant package with a new one, allowing for + setup_requires dependencies to supersede installed packages for the + session. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/341 + + (2) https://bitbucket.org/tarek/distribute/issue/323 + + (3) https://github.com/pypa/setuptools/issues/141 + + (4) https://github.com/pypa/setuptools/issues/207 + + (5) https://bitbucket.org/pypa/setuptools/pull-request/167 + + +File: setuptools.info, Node: 19 3, Next: 19 2, Prev: 19 4, Up: History<2> + +9.245 19.3 +========== + +06 Jan 2016 + + * Issue #229(1): Implement new technique for readily incorporating + dependencies conditionally from vendored copies or primary + locations. Adds a new dependency on six. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/229 + + +File: setuptools.info, Node: 19 2, Next: 19 1 1, Prev: 19 3, Up: History<2> + +9.246 19.2 +========== + +25 Dec 2015 + + * BB Pull Request #163(1): Add get_command_list method to + Distribution. + + * BB Pull Request #162(2): Add missing whitespace to multiline string + literals. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/163 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/162 + + +File: setuptools.info, Node: 19 1 1, Next: 19 1, Prev: 19 2, Up: History<2> + +9.247 19.1.1 +============ + +16 Dec 2015 + + * Issue #476(1): Cast version to string (using default encoding) to + avoid creating Unicode types on Python 2 clients. + + * Issue #477(2): In Powershell downloader, use explicit rendering of + strings, rather than rely on ‘repr’, which can be incorrect + (especially on Python 2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/476 + + (2) https://github.com/pypa/setuptools/issues/477 + + +File: setuptools.info, Node: 19 1, Next: 19 0, Prev: 19 1 1, Up: History<2> + +9.248 19.1 +========== + +16 Dec 2015 + + * Issue #215(1): The bootstrap script ‘ez_setup.py’ now automatically + detects the latest version of setuptools (using PyPI JSON API) + rather than hard-coding a particular value. + + * Issue #475(2): Fix incorrect usage in _translate_metadata2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/215 + + (2) https://github.com/pypa/setuptools/issues/475 + + +File: setuptools.info, Node: 19 0, Next: 18 8 1, Prev: 19 1, Up: History<2> + +9.249 19.0 +========== + +15 Dec 2015 + + * Issue #442(1): Use RawConfigParser for parsing .pypirc file. + Interpolated values are no longer honored in .pypirc files. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/442 + + +File: setuptools.info, Node: 18 8 1, Next: 18 8, Prev: 19 0, Up: History<2> + +9.250 18.8.1 +============ + +13 Dec 2015 + + * Issue #440(1): Prevent infinite recursion when a SandboxViolation + or other UnpickleableException occurs in a sandbox context with + setuptools hidden. Fixes regression introduced in Setuptools 12.0. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/440 + + +File: setuptools.info, Node: 18 8, Next: 18 7 1, Prev: 18 8 1, Up: History<2> + +9.251 18.8 +========== + +11 Dec 2015 + + * Deprecated ‘egg_info.get_pkg_info_revision’. + + * Issue #471(1): Don’t rely on repr for an HTML attribute value in + package_index. + + * Issue #419(2): Avoid errors in FileMetadata when the metadata + directory is broken. + + * Issue #472(3): Remove deprecated use of ‘U’ in mode parameter when + opening files. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/471 + + (2) https://github.com/pypa/setuptools/issues/419 + + (3) https://github.com/pypa/setuptools/issues/472 + + +File: setuptools.info, Node: 18 7 1, Next: 18 7, Prev: 18 8, Up: History<2> + +9.252 18.7.1 +============ + +01 Dec 2015 + + * Issue #469(1): Refactored logic for Issue #419(2) fix to re-use + metadata loading from Provider. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/469 + + (2) https://github.com/pypa/setuptools/issues/419 + + +File: setuptools.info, Node: 18 7, Next: 18 6 1, Prev: 18 7 1, Up: History<2> + +9.253 18.7 +========== + +28 Nov 2015 + + * Update dependency on certify. + + * BB Pull Request #160(1): Improve detection of gui script in + ‘easy_install._adjust_header’. + + * Made ‘test.test_args’ a non-data property; alternate fix for the + issue reported in BB Pull Request #155(2). + + * Issue #453(3): In ‘ez_setup’ bootstrap module, unload all + ‘pkg_resources’ modules following download. + + * BB Pull Request #158(4): Honor PEP-488(5) when excluding files for + namespace packages. + + * Issue #419(6) and BB Pull Request #144(7): Add experimental support + for reading the version info from distutils-installed metadata + rather than using the version in the filename. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/160 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/155 + + (3) https://github.com/pypa/setuptools/issues/453 + + (4) https://bitbucket.org/pypa/setuptools/pull-request/158 + + (5) https://www.python.org/dev/peps/pep-0488/ + + (6) https://github.com/pypa/setuptools/issues/419 + + (7) https://bitbucket.org/pypa/setuptools/pull-request/144 + + +File: setuptools.info, Node: 18 6 1, Next: 18 6, Prev: 18 7, Up: History<2> + +9.254 18.6.1 +============ + +24 Nov 2015 + + * Issue #464(1): Correct regression in invocation of superclass on + old-style class on Python 2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/464 + + +File: setuptools.info, Node: 18 6, Next: 18 5, Prev: 18 6 1, Up: History<2> + +9.255 18.6 +========== + +24 Nov 2015 + + * Issue #439(1): When installing entry_point scripts under + development, omit the version number of the package, allowing any + version of the package to be used. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/439 + + +File: setuptools.info, Node: 18 5, Next: 18 4, Prev: 18 6, Up: History<2> + +9.256 18.5 +========== + +01 Nov 2015 + + * In preparation for dropping support for Python 3.2, a warning is + now logged when pkg_resources is imported on Python 3.2 or earlier + Python 3 versions. + + * Add support for python_platform_implementation environment + marker(1). + + * Fix dictionary mutation during iteration(2). + + ---------- Footnotes ---------- + + (1) +https://github.com/pypa/setuptools/commit/94416707fd59a65f4a8f7f70541d6b3fc018b626 + + (2) +https://github.com/pypa/setuptools/commit/57ebfa41e0f96b97e599ecd931b7ae8a143e096e + + +File: setuptools.info, Node: 18 4, Next: 18 3 2, Prev: 18 5, Up: History<2> + +9.257 18.4 +========== + +10 Oct 2015 + + * Issue #446(1): Test command now always invokes unittest, even if no + test suite is supplied. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/446 + + +File: setuptools.info, Node: 18 3 2, Next: 18 3 1, Prev: 18 4, Up: History<2> + +9.258 18.3.2 +============ + +19 Sep 2015 + + * Correct another regression in setuptools.findall where the fix for + Python #12885(1) was lost. + + ---------- Footnotes ---------- + + (1) http://bugs.python.org/issue12885 + + +File: setuptools.info, Node: 18 3 1, Next: 18 3, Prev: 18 3 2, Up: History<2> + +9.259 18.3.1 +============ + +07 Sep 2015 + + * Issue #425(1): Correct regression in setuptools.findall. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/425 + + +File: setuptools.info, Node: 18 3, Next: 18 2, Prev: 18 3 1, Up: History<2> + +9.260 18.3 +========== + +06 Sep 2015 + + * BB Pull Request #135(1): Setuptools now allows disabling of the + manipulation of the sys.path during the processing of the + easy-install.pth file. To do so, set the environment variable + ‘SETUPTOOLS_SYS_PATH_TECHNIQUE’ to anything but “rewrite” (consider + “raw”). During any install operation with manipulation disabled, + setuptools packages will be appended to sys.path naturally. + + Future versions may change the default behavior to disable + manipulation. If so, the default behavior can be retained by + setting the variable to “rewrite”. + + * Issue #257(2): ‘easy_install --version’ now shows more detail about + the installation location and Python version. + + * Refactor setuptools.findall in preparation for re-submission back + to distutils. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/135 + + (2) https://github.com/pypa/setuptools/issues/257 + + +File: setuptools.info, Node: 18 2, Next: 18 1, Prev: 18 3, Up: History<2> + +9.261 18.2 +========== + +19 Aug 2015 + + * Issue #412(1): More efficient directory search in ‘find_packages’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/412 + + +File: setuptools.info, Node: 18 1, Next: 18 0 1, Prev: 18 2, Up: History<2> + +9.262 18.1 +========== + +02 Aug 2015 + + * Upgrade to vendored packaging 15.3(1). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging/blob/15.3/CHANGELOG.rst + + +File: setuptools.info, Node: 18 0 1, Next: 18 0, Prev: 18 1, Up: History<2> + +9.263 18.0.1 +============ + +24 Jun 2015 + + * Issue #401(1): Fix failure in test suite. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/401 + + +File: setuptools.info, Node: 18 0, Next: 17 1 1, Prev: 18 0 1, Up: History<2> + +9.264 18.0 +========== + +13 Jun 2015 + + * Dropped support for builds with Pyrex. Only Cython is supported. + + * Issue #288(1): Detect Cython later in the build process, after + ‘setup_requires’ dependencies are resolved. Projects backed by + Cython can now be readily built with a ‘setup_requires’ dependency. + For example: + + ext = setuptools.Extension('mylib', ['src/CythonStuff.pyx', 'src/CStuff.c']) + setuptools.setup( + ... + ext_modules=[ext], + setup_requires=['cython'], + ) + + For compatibility with older versions of setuptools, packagers + should still include ‘src/CythonMod.c’ in the source distributions + or require that Cython be present before building source + distributions. However, for systems with this build of setuptools, + Cython will be downloaded on demand. + + * Issue #396(2): Fixed test failure on OS X. + + * BB Pull Request #136(3): Remove excessive quoting from shebang + headers for Jython. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/288 + + (2) https://github.com/pypa/setuptools/issues/396 + + (3) https://bitbucket.org/pypa/setuptools/pull-request/136 + + +File: setuptools.info, Node: 17 1 1, Next: 17 1, Prev: 18 0, Up: History<2> + +9.265 17.1.1 +============ + +08 Jun 2015 + + * Backed out unintended changes to pkg_resources, restoring removal + of deprecated imp module (ref(1)). + + ---------- Footnotes ---------- + + (1) +https://bitbucket.org/pypa/setuptools/commits/f572ec9563d647fa8d4ffc534f2af8070ea07a8b#comment-1881283 + + +File: setuptools.info, Node: 17 1, Next: 17 0, Prev: 17 1 1, Up: History<2> + +9.266 17.1 +========== + +07 Jun 2015 + + * Issue #380(1): Add support for range operators on environment + marker evaluation. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/380 + + +File: setuptools.info, Node: 17 0, Next: 16 0, Prev: 17 1, Up: History<2> + +9.267 17.0 +========== + +28 May 2015 + + * Issue #378(1): Do not use internal importlib._bootstrap module. + + * Issue #390(2): Disallow console scripts with path separators in the + name. Removes unintended functionality and brings behavior into + parity with pip. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/378 + + (2) https://github.com/pypa/setuptools/issues/390 + + +File: setuptools.info, Node: 16 0, Next: 15 2, Prev: 17 0, Up: History<2> + +9.268 16.0 +========== + +18 May 2015 + + * BB Pull Request #130(1): Better error messages for errors in parsed + requirements. + + * BB Pull Request #133(2): Removed ‘setuptools.tests’ from the + installed packages. + + * BB Pull Request #129(3): Address deprecation warning due to usage + of imp module. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/130 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/133 + + (3) https://bitbucket.org/pypa/setuptools/pull-request/129 + + +File: setuptools.info, Node: 15 2, Next: 15 1, Prev: 16 0, Up: History<2> + +9.269 15.2 +========== + +26 Apr 2015 + + * Issue #373(1): Provisionally expose + ‘pkg_resources._initialize_master_working_set’, allowing for + imperative re-initialization of the master working set. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/373 + + +File: setuptools.info, Node: 15 1, Next: 15 0, Prev: 15 2, Up: History<2> + +9.270 15.1 +========== + +15 Apr 2015 + + * Updated to Packaging 15.1(1) to address Packaging #28(2). + + * Fix ‘setuptools.sandbox._execfile()’ with Python 3.1. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/packaging/blob/15.1/CHANGELOG.rst + + (2) https://github.com/pypa/packaging/issues/28 + + +File: setuptools.info, Node: 15 0, Next: 14 3 1, Prev: 15 1, Up: History<2> + +9.271 15.0 +========== + +03 Apr 2015 + + * BB Pull Request #126(1): DistributionNotFound message now lists the + package or packages that required it. E.g.: + + pkg_resources.DistributionNotFound: The 'colorama>=0.3.1' distribution was not found and is required by smlib.log. + + Note that zc.buildout once dependended on the string rendering of + this message to determine the package that was not found. This + expectation has since been changed, but older versions of buildout + may experience problems. See Buildout #242(2) for details. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/126 + + (2) https://github.com/buildout/buildout/issues/242 + + +File: setuptools.info, Node: 14 3 1, Next: 14 3, Prev: 15 0, Up: History<2> + +9.272 14.3.1 +============ + +20 Mar 2015 + + * Issue #307(1): Removed PEP-440(2) warning during parsing of + versions in ‘pkg_resources.Distribution’. + + * Issue #364(3): Replace deprecated usage with recommended usage of + ‘EntryPoint.load’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/307 + + (2) https://www.python.org/dev/peps/pep-0440/ + + (3) https://github.com/pypa/setuptools/issues/364 + + +File: setuptools.info, Node: 14 3, Next: 14 2, Prev: 14 3 1, Up: History<2> + +9.273 14.3 +========== + +15 Mar 2015 + + * Issue #254(1): When creating temporary egg cache on Unix, use mode + 755 for creating the directory to avoid the subsequent warning if + the directory is group writable. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/254 + + +File: setuptools.info, Node: 14 2, Next: 14 1 1, Prev: 14 3, Up: History<2> + +9.274 14.2 +========== + +15 Mar 2015 + + * Issue #137(1): Update ‘Distribution.hashcmp’ so that Distributions + with None for pyversion or platform can be compared against + Distributions defining those attributes. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/137 + + +File: setuptools.info, Node: 14 1 1, Next: 14 1, Prev: 14 2, Up: History<2> + +9.275 14.1.1 +============ + +14 Mar 2015 + + * Issue #360(1): Removed undesirable behavior from test runs, + preventing write tests and installation to system site packages. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/360 + + +File: setuptools.info, Node: 14 1, Next: 14 0, Prev: 14 1 1, Up: History<2> + +9.276 14.1 +========== + +14 Mar 2015 + + * BB Pull Request #125(1): Add ‘__ne__’ to Requirement class. + + * Various refactoring of easy_install. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/125 + + +File: setuptools.info, Node: 14 0, Next: 13 0 2, Prev: 14 1, Up: History<2> + +9.277 14.0 +========== + +06 Mar 2015 + + * Bootstrap script now accepts ‘--to-dir’ to customize save directory + or allow for re-use of existing repository of setuptools versions. + See BB Pull Request #112(1) for background. + + * Issue #285(2): ‘easy_install’ no longer will default to installing + packages to the “user site packages” directory if it is itself + installed there. Instead, the user must pass ‘--user’ in all cases + to install packages to the user site packages. This behavior now + matches that of “pip install”. To configure an environment to + always install to the user site packages, consider using the + “install-dir” and “scripts-dir” parameters to easy_install through + an appropriate distutils config file. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/112 + + (2) https://github.com/pypa/setuptools/issues/285 + + +File: setuptools.info, Node: 13 0 2, Next: 13 0 1, Prev: 14 0, Up: History<2> + +9.278 13.0.2 +============ + +06 Mar 2015 + + * Issue #359(1): Include pytest.ini in the sdist so invocation of + py.test on the sdist honors the pytest configuration. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/359 + + +File: setuptools.info, Node: 13 0 1, Next: 13 0, Prev: 13 0 2, Up: History<2> + +9.279 13.0.1 +============ + +05 Mar 2015 + +Re-release of 13.0. Intermittent connectivity issues caused the release +process to fail and PyPI uploads no longer accept files for 13.0. + + +File: setuptools.info, Node: 13 0, Next: 12 4, Prev: 13 0 1, Up: History<2> + +9.280 13.0 +========== + +05 Mar 2015 + + * Issue #356(1): Back out BB Pull Request #119(2) as it requires + Setuptools 10 or later as the source during an upgrade. + + * Removed build_py class from setup.py. According to 892f439d216e, + this functionality was added to support upgrades from old + Distribute versions, 0.6.5 and 0.6.6. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/356 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/119 + + +File: setuptools.info, Node: 12 4, Next: 12 3, Prev: 13 0, Up: History<2> + +9.281 12.4 +========== + +04 Mar 2015 + + * BB Pull Request #119(1): Restore writing of ‘setup_requires’ to + metadata (previously added in 8.4 and removed in 9.0). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/119 + + +File: setuptools.info, Node: 12 3, Next: 12 2, Prev: 12 4, Up: History<2> + +9.282 12.3 +========== + +26 Feb 2015 + + * Documentation is now linked using the rst.linker package. + + * Fix ‘setuptools.command.easy_install.extract_wininst_cfg()’ with + Python 2.6 and 2.7. + + * Issue #354(1). Added documentation on building setuptools + documentation. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/354 + + +File: setuptools.info, Node: 12 2, Next: 12 1, Prev: 12 3, Up: History<2> + +9.283 12.2 +========== + +16 Feb 2015 + + * Issue #345(1): Unload all modules under pkg_resources during + ‘ez_setup.use_setuptools()’. + + * Issue #336(2): Removed deprecation from ‘ez_setup.use_setuptools’, + as it is clearly still used by buildout’s bootstrap. ‘ez_setup’ + remains deprecated for use by individual packages. + + * Simplified implementation of ‘ez_setup.use_setuptools’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/345 + + (2) https://github.com/pypa/setuptools/issues/336 + + +File: setuptools.info, Node: 12 1, Next: 12 0 5, Prev: 12 2, Up: History<2> + +9.284 12.1 +========== + +10 Feb 2015 + + * BB Pull Request #118(1): Soften warning for non-normalized versions + in Distribution. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/118 + + +File: setuptools.info, Node: 12 0 5, Next: 12 0 4, Prev: 12 1, Up: History<2> + +9.285 12.0.5 +============ + +26 Jan 2015 + + * Issue #339(1): Correct Attribute reference in + ‘cant_write_to_target’. + + * Issue #336(2): Deprecated ‘ez_setup.use_setuptools’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/339 + + (2) https://github.com/pypa/setuptools/issues/336 + + +File: setuptools.info, Node: 12 0 4, Next: 12 0 3, Prev: 12 0 5, Up: History<2> + +9.286 12.0.4 +============ + +20 Jan 2015 + + * Issue #335(1): Fix script header generation on Windows. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/335 + + +File: setuptools.info, Node: 12 0 3, Next: 12 0 2, Prev: 12 0 4, Up: History<2> + +9.287 12.0.3 +============ + +18 Jan 2015 + + * Fixed incorrect class attribute in ‘install_scripts’. Tests would + be nice. + + +File: setuptools.info, Node: 12 0 2, Next: 12 0 1, Prev: 12 0 3, Up: History<2> + +9.288 12.0.2 +============ + +18 Jan 2015 + + * Issue #331(1): Fixed ‘install_scripts’ command on Windows systems + corrupting the header. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/331 + + +File: setuptools.info, Node: 12 0 1, Next: 12 0, Prev: 12 0 2, Up: History<2> + +9.289 12.0.1 +============ + +16 Jan 2015 + + * Restore ‘setuptools.command.easy_install.sys_executable’ for pbr + compatibility. For the future, tools should construct a + CommandSpec explicitly. + + +File: setuptools.info, Node: 12 0, Next: 11 3 1, Prev: 12 0 1, Up: History<2> + +9.290 12.0 +========== + +16 Jan 2015 + + * Issue #188(1): Setuptools now support multiple entities in the + value for ‘build.executable’, such that an executable of + “/usr/bin/env my-python” may be specified. This means that systems + with a specified executable whose name has spaces in the path must + be updated to escape or quote that value. + + * Deprecated ‘easy_install.ScriptWriter.get_writer’, replaced by + ‘.best()’ with slightly different semantics (no force_windows + flag). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/188 + + +File: setuptools.info, Node: 11 3 1, Next: 11 3, Prev: 12 0, Up: History<2> + +9.291 11.3.1 +============ + +06 Jan 2015 + + * Issue #327(1): Formalize and restore support for any printable + character in an entry point name. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/327 + + +File: setuptools.info, Node: 11 3, Next: 11 2, Prev: 11 3 1, Up: History<2> + +9.292 11.3 +========== + +05 Jan 2015 + + * Expose ‘EntryPoint.resolve’ in place of EntryPoint._load, + implementing the simple, non-requiring load. Deprecated all uses + of ‘EntryPoint._load’ except for calling with no parameters, which + is just a shortcut for ‘ep.require(); ep.resolve();’. + + Apps currently invoking ‘ep.load(require=False)’ should instead do + the following if wanting to avoid the deprecating warning: + + getattr(ep, "resolve", lambda: ep.load(require=False))() + + +File: setuptools.info, Node: 11 2, Next: 11 1, Prev: 11 3, Up: History<2> + +9.293 11.2 +========== + +05 Jan 2015 + + * Pip #2326(1): Report deprecation warning at stacklevel 2 for easier + diagnosis. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/pip/issues/2326 + + +File: setuptools.info, Node: 11 1, Next: 11 0, Prev: 11 2, Up: History<2> + +9.294 11.1 +========== + +04 Jan 2015 + + * Issue #281(1): Since Setuptools 6.1 (Issue #268(2)), a ValueError + would be raised in certain cases where VersionConflict was raised + with two arguments, which occurred in + ‘pkg_resources.WorkingSet.find’. This release adds support for + indicating the dependent packages while maintaining support for a + VersionConflict when no dependent package context is known. New + unit tests now capture the expected interface. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/281 + + (2) https://github.com/pypa/setuptools/issues/268 + + +File: setuptools.info, Node: 11 0, Next: 10 2 1, Prev: 11 1, Up: History<2> + +9.295 11.0 +========== + +02 Jan 2015 + + * Interop #3(1): Upgrade to Packaging 15.0(2); updates to PEP 440(3) + so that >1.7 does not exclude 1.7.1 but does exclude 1.7.0 and + 1.7.0.post1. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/interoperability-peps/issues/3 + + (2) https://github.com/pypa/packaging/blob/15.0/CHANGELOG.rst + + (3) https://www.python.org/dev/peps/pep-0440/ + + +File: setuptools.info, Node: 10 2 1, Next: 10 2, Prev: 11 0, Up: History<2> + +9.296 10.2.1 +============ + +02 Jan 2015 + + * Issue #323(1): Fix regression in entry point name parsing. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/323 + + +File: setuptools.info, Node: 10 2, Next: 10 1, Prev: 10 2 1, Up: History<2> + +9.297 10.2 +========== + +02 Jan 2015 + + * Deprecated use of EntryPoint.load(require=False). Passing a + boolean to a function to select behavior is an anti-pattern. + Instead use ‘Entrypoint._load()’. + + * Substantial refactoring of all unit tests. Tests are now much + leaner and re-use a lot of fixtures and contexts for better clarity + of purpose. + + +File: setuptools.info, Node: 10 1, Next: 10 0 1, Prev: 10 2, Up: History<2> + +9.298 10.1 +========== + +31 Dec 2014 + + * Issue #320(1): Added a compatibility implementation of + ‘sdist._default_revctrl’ so that systems relying on that interface + do not fail (namely, Ubuntu 12.04 and similar Debian releases). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/320 + + +File: setuptools.info, Node: 10 0 1, Next: 10 0, Prev: 10 1, Up: History<2> + +9.299 10.0.1 +============ + +30 Dec 2014 + + * Issue #319(1): Fixed issue installing pure distutils packages. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/319 + + +File: setuptools.info, Node: 10 0, Next: 9 1, Prev: 10 0 1, Up: History<2> + +9.300 10.0 +========== + +30 Dec 2014 + + * Issue #313(1): Removed built-in support for subversion. Projects + wishing to retain support for subversion will need to use a third + party library. The extant implementation is being ported to + setuptools_svn(2). + + * Issue #315(3): Updated setuptools to hide its own loaded modules + during installation of another package. This change will enable + setuptools to upgrade (or downgrade) itself even when its own + metadata and implementation change. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/313 + + (2) https://pypi.org/project/setuptools_svn/ + + (3) https://github.com/pypa/setuptools/issues/315 + + +File: setuptools.info, Node: 9 1, Next: 9 0 1, Prev: 10 0, Up: History<2> + +9.301 9.1 +========= + +29 Dec 2014 + + * Prefer vendored packaging library as recommended(1). + + ---------- Footnotes ---------- + + (1) +https://github.com/pypa/setuptools/commit/170657b68f4b92e7e1bf82f5e19a831f5744af67 + + +File: setuptools.info, Node: 9 0 1, Next: 9 0, Prev: 9 1, Up: History<2> + +9.302 9.0.1 +=========== + +29 Dec 2014 + + * Issue #312(1): Restored presence of pkg_resources API tests + (doctest) to sdist. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/312 + + +File: setuptools.info, Node: 9 0, Next: 8 4, Prev: 9 0 1, Up: History<2> + +9.303 9.0 +========= + +28 Dec 2014 + + * Issue #314(1): Disabled support for ‘setup_requires’ metadata to + avoid issue where Setuptools was unable to upgrade over earlier + versions. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/314 + + +File: setuptools.info, Node: 8 4, Next: 8 3, Prev: 9 0, Up: History<2> + +9.304 8.4 +========= + +26 Dec 2014 + + * BB Pull Request #106(1): Now write ‘setup_requires’ metadata. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/106 + + +File: setuptools.info, Node: 8 3, Next: 8 2 1, Prev: 8 4, Up: History<2> + +9.305 8.3 +========= + +24 Dec 2014 + + * Issue #311(1): Decoupled pkg_resources from setuptools once again. + ‘pkg_resources’ is now a package instead of a module. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/311 + + +File: setuptools.info, Node: 8 2 1, Next: 8 2, Prev: 8 3, Up: History<2> + +9.306 8.2.1 +=========== + +18 Dec 2014 + + * Issue #306(1): Suppress warnings about Version format except in + select scenarios (such as installation). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/306 + + +File: setuptools.info, Node: 8 2, Next: 8 1, Prev: 8 2 1, Up: History<2> + +9.307 8.2 +========= + +18 Dec 2014 + + * BB Pull Request #85(1): Search egg-base when adding egg-info to + manifest. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/85 + + +File: setuptools.info, Node: 8 1, Next: 8 0 4, Prev: 8 2, Up: History<2> + +9.308 8.1 +========= + +18 Dec 2014 + + * Upgrade ‘packaging’ to 14.5, giving preference to “rc” as + designator for release candidates over “c”. + + * PEP-440(1) warnings are now raised as their own class, + ‘pkg_resources.PEP440Warning’, instead of RuntimeWarning. + + * Disabled warnings on empty versions. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440/ + + +File: setuptools.info, Node: 8 0 4, Next: 8 0 3, Prev: 8 1, Up: History<2> + +9.309 8.0.4 +=========== + +15 Dec 2014 + + * Upgrade ‘packaging’ to 14.4, fixing an error where there is a + different result for if 2.0.5 is contained within >2.0dev and + >2.0.dev even though normalization rules should have made them + equal. + + * Issue #296(1): Add warning when a version is parsed as legacy. + This warning will make it easier for developers to recognize + deprecated version numbers. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/296 + + +File: setuptools.info, Node: 8 0 3, Next: 8 0 2, Prev: 8 0 4, Up: History<2> + +9.310 8.0.3 +=========== + +15 Dec 2014 + + * Issue #296(1): Restored support for ‘__hash__’ on parse_version + results. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/296 + + +File: setuptools.info, Node: 8 0 2, Next: 8 0 1, Prev: 8 0 3, Up: History<2> + +9.311 8.0.2 +=========== + +14 Dec 2014 + + * Issue #296(1): Restored support for ‘__getitem__’ and sort + operations on parse_version result. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/296 + + +File: setuptools.info, Node: 8 0 1, Next: 8 0, Prev: 8 0 2, Up: History<2> + +9.312 8.0.1 +=========== + +13 Dec 2014 + + * Issue #296(1): Restore support for iteration over parse_version + result, but deprecated that usage with a warning. Fixes failure + with buildout. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/296 + + +File: setuptools.info, Node: 8 0, Next: 7 0, Prev: 8 0 1, Up: History<2> + +9.313 8.0 +========= + +13 Dec 2014 + + * Implement PEP 440(1) within pkg_resources and setuptools. This + change deprecates some version numbers such that they will no + longer be installable without using the ‘===’ escape hatch. See + the changes to test_resources(2) for specific examples of version + numbers and specifiers that are no longer supported. Setuptools + now “vendors” the packaging(3) library. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0440/ + + (2) +https://bitbucket.org/pypa/setuptools/commits/dcd552da643c4448056de84c73d56da6d70769d5#chg-setuptools/tests/test_resources.py + + (3) https://github.com/pypa/packaging + + +File: setuptools.info, Node: 7 0, Next: 6 1, Prev: 8 0, Up: History<2> + +9.314 7.0 +========= + +19 Oct 2014 + + * Issue #80(1), Issue #209(2): Eggs that are downloaded for + ‘setup_requires’, ‘test_requires’, etc. are now placed in a + ‘./.eggs’ directory instead of directly in the current directory. + This choice of location means the files can be readily managed + (removed, ignored). Additionally, later phases or invocations of + setuptools will not detect the package as already installed and + ignore it for permanent install (See #209(3)). + + This change is indicated as backward-incompatible as installations + that depend on the installation in the current directory will need + to account for the new location. Systems that ignore ‘*.egg’ will + probably need to be adapted to ignore ‘.eggs’. The files will need + to be manually moved or will be retrieved again. Most use cases + will require no attention. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/80 + + (2) https://github.com/pypa/setuptools/issues/209 + + (3) https://github.com/pypa/setuptools/issues/209 + + +File: setuptools.info, Node: 6 1, Next: 6 0 2, Prev: 7 0, Up: History<2> + +9.315 6.1 +========= + +11 Oct 2014 + + * Issue #268(1): When resolving package versions, a VersionConflict + now reports which package previously required the conflicting + version. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/268 + + +File: setuptools.info, Node: 6 0 2, Next: 6 0 1, Prev: 6 1, Up: History<2> + +9.316 6.0.2 +=========== + +29 Sep 2014 + + * Issue #262(1): Fixed regression in pip install due to egg-info + directories being omitted. Re-opens Issue #118(2). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/262 + + (2) https://github.com/pypa/setuptools/issues/118 + + +File: setuptools.info, Node: 6 0 1, Next: 6 0, Prev: 6 0 2, Up: History<2> + +9.317 6.0.1 +=========== + +27 Sep 2014 + + * Issue #259(1): Fixed regression with namespace package handling on + ‘single version, externally managed’ installs. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/259 + + +File: setuptools.info, Node: 6 0, Next: 5 8, Prev: 6 0 1, Up: History<2> + +9.318 6.0 +========= + +27 Sep 2014 + + * Issue #100(1): When building a distribution, Setuptools will no + longer match default files using platform-dependent case + sensitivity, but rather will only match the files if their case + matches exactly. As a result, on Windows and other + case-insensitive file systems, files with names such as + ‘readme.txt’ or ‘README.TXT’ will be omitted from the distribution + and a warning will be issued indicating that ‘README.txt’ was not + found. Other filenames affected are: + + - README.rst + + - README + + - setup.cfg + + - setup.py (or the script name) + + - test/test*.py + + Any users producing distributions with filenames that match those + above case-insensitively, but not case-sensitively, should rename + those files in their repository for better portability. + + * BB Pull Request #72(2): When using + ‘single_version_externally_managed’, the exclusion list now + includes Python 3.2 ‘__pycache__’ entries. + + * BB Pull Request #76(3) and BB Pull Request #78(4): lines in + top_level.txt are now ordered deterministically. + + * Issue #118(5): The egg-info directory is now no longer included in + the list of outputs. + + * Issue #258(6): Setuptools now patches distutils msvc9compiler to + recognize the specially-packaged compiler package for easy + extension module support on Python 2.6, 2.7, and 3.2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/100 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/72 + + (3) https://bitbucket.org/pypa/setuptools/pull-request/76 + + (4) https://bitbucket.org/pypa/setuptools/pull-request/78 + + (5) https://github.com/pypa/setuptools/issues/118 + + (6) https://github.com/pypa/setuptools/issues/258 + + +File: setuptools.info, Node: 5 8, Next: 5 7, Prev: 6 0, Up: History<2> + +9.319 5.8 +========= + +18 Sep 2014 + + * Issue #237(1): ‘pkg_resources’ now uses explicit detection of + Python 2 vs. Python 3, supporting environments where builtins have + been patched to make Python 3 look more like Python 2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/237 + + +File: setuptools.info, Node: 5 7, Next: 5 6, Prev: 5 8, Up: History<2> + +9.320 5.7 +========= + +15 Aug 2014 + + * Issue #240(1): Based on real-world performance measures against + 5.4, zip manifests are now cached in all circumstances. The + ‘PKG_RESOURCES_CACHE_ZIP_MANIFESTS’ environment variable is no + longer relevant. The observed “memory increase” referenced in the + 5.4 release notes and detailed in Issue #154(2) was likely not an + increase over the status quo, but rather only an increase over not + storing the zip info at all. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/240 + + (2) https://github.com/pypa/setuptools/issues/154 + + +File: setuptools.info, Node: 5 6, Next: 5 5 1, Prev: 5 7, Up: History<2> + +9.321 5.6 +========= + +14 Aug 2014 + + * Issue #242(1): Use absolute imports in svn_utils to avoid issues if + the installing package adds an xml module to the path. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/242 + + +File: setuptools.info, Node: 5 5 1, Next: 5 5, Prev: 5 6, Up: History<2> + +9.322 5.5.1 +=========== + +10 Aug 2014 + + * Issue #239(1): Fix typo in 5.5 such that fix did not take. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/239 + + +File: setuptools.info, Node: 5 5, Next: 5 4 2, Prev: 5 5 1, Up: History<2> + +9.323 5.5 +========= + +10 Aug 2014 + + * Issue #239(1): Setuptools now includes the setup_requires directive + on Distribution objects and validates the syntax just like + install_requires and tests_require directives. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/239 + + +File: setuptools.info, Node: 5 4 2, Next: 5 4 1, Prev: 5 5, Up: History<2> + +9.324 5.4.2 +=========== + +01 Aug 2014 + + * Issue #236(1): Corrected regression in execfile implementation for + Python 2.6. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/236 + + +File: setuptools.info, Node: 5 4 1, Next: 5 4, Prev: 5 4 2, Up: History<2> + +9.325 5.4.1 +=========== + +06 Jul 2014 + + * Python #7776(1): (ssl_support) Correct usage of host for validation + when tunneling for HTTPS. + + ---------- Footnotes ---------- + + (1) http://bugs.python.org/issue7776 + + +File: setuptools.info, Node: 5 4, Next: 5 3, Prev: 5 4 1, Up: History<2> + +9.326 5.4 +========= + +05 Jul 2014 + + * Issue #154(1): ‘pkg_resources’ will now cache the zip manifests + rather than re-processing the same file from disk multiple times, + but only if the environment variable + ‘PKG_RESOURCES_CACHE_ZIP_MANIFESTS’ is set. Clients that package + many modules in the same zip file will see some improvement in + startup time by enabling this feature. This feature is not enabled + by default because it causes a substantial increase in memory + usage. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/154 + + +File: setuptools.info, Node: 5 3, Next: 5 2, Prev: 5 4, Up: History<2> + +9.327 5.3 +========= + +28 Jun 2014 + + * Issue #185(1): Make svn tagging work on the new style SVN metadata. + Thanks cazabon! + + * Prune revision control directories (e.g .svn) from base path as + well as sub-directories. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/185 + + +File: setuptools.info, Node: 5 2, Next: 5 1, Prev: 5 3, Up: History<2> + +9.328 5.2 +========= + +22 Jun 2014 + + * Added a Developer Guide(1) to the official documentation. + + * Some code refactoring and cleanup was done with no intended + behavioral changes. + + * During install_egg_info, the generated lines for namespace package + .pth files are now processed even during a dry run. + + ---------- Footnotes ---------- + + (1) https://setuptools.readthedocs.io/en/latest/developer-guide.html + + +File: setuptools.info, Node: 5 1, Next: 5 0 2, Prev: 5 2, Up: History<2> + +9.329 5.1 +========= + +15 Jun 2014 + + * Issue #202(1): Implemented more robust cache invalidation for the + ZipImporter, building on the work in Issue #168(2). Special thanks + to Jurko Gospodnetic and PJE. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/202 + + (2) https://github.com/pypa/setuptools/issues/168 + + +File: setuptools.info, Node: 5 0 2, Next: 5 0 1, Prev: 5 1, Up: History<2> + +9.330 5.0.2 +=========== + +15 Jun 2014 + + * Issue #220(1): Restored script templates. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/220 + + +File: setuptools.info, Node: 5 0 1, Next: 5 0, Prev: 5 0 2, Up: History<2> + +9.331 5.0.1 +=========== + +14 Jun 2014 + + * Renamed script templates to end with .tmpl now that they no longer + need to be processed by 2to3. Fixes spurious syntax errors during + build/install. + + +File: setuptools.info, Node: 5 0, Next: 3 7 1 and 3 8 1 and 4 0 1, Prev: 5 0 1, Up: History<2> + +9.332 5.0 +========= + +14 Jun 2014 + + * Issue #218(1): Re-release of 3.8.1 to signal that it supersedes + 4.x. + + * Incidentally, script templates were updated not to include the + triple-quote escaping. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/218 + + +File: setuptools.info, Node: 3 7 1 and 3 8 1 and 4 0 1, Next: 4 0, Prev: 5 0, Up: History<2> + +9.333 3.7.1 and 3.8.1 and 4.0.1 +=============================== + + * Issue #213(1): Use legacy StringIO behavior for compatibility under + pbr. + + * Issue #218(2): Setuptools 3.8.1 superseded 4.0.1, and 4.x was + removed from the available versions to install. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/213 + + (2) https://github.com/pypa/setuptools/issues/218 + + +File: setuptools.info, Node: 4 0, Next: 3 8, Prev: 3 7 1 and 3 8 1 and 4 0 1, Up: History<2> + +9.334 4.0 +========= + +01 Jun 2014 + + * Issue #210(1): ‘setup.py develop’ now copies scripts in binary mode + rather than text mode, matching the behavior of the ‘install’ + command. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/210 + + +File: setuptools.info, Node: 3 8, Next: 3 7, Prev: 4 0, Up: History<2> + +9.335 3.8 +========= + +01 Jun 2014 + + * Extend Issue #197(1) workaround to include all Python 3 versions + prior to 3.2.2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/197 + + +File: setuptools.info, Node: 3 7, Next: 3 6, Prev: 3 8, Up: History<2> + +9.336 3.7 +========= + +28 May 2014 + + * Issue #193(1): Improved handling of Unicode filenames when building + manifests. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/193 + + +File: setuptools.info, Node: 3 6, Next: 3 5 2, Prev: 3 7, Up: History<2> + +9.337 3.6 +========= + +07 May 2014 + + * Issue #203(1): Honor proxy settings for Powershell downloader in + the bootstrap routine. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/203 + + +File: setuptools.info, Node: 3 5 2, Next: 3 5 1, Prev: 3 6, Up: History<2> + +9.338 3.5.2 +=========== + +07 May 2014 + + * Issue #168(1): More robust handling of replaced zip files and stale + caches. Fixes ZipImportError complaining about a ‘bad local + header’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/168 + + +File: setuptools.info, Node: 3 5 1, Next: 3 5, Prev: 3 5 2, Up: History<2> + +9.339 3.5.1 +=========== + +04 May 2014 + + * Issue #199(1): Restored ‘install._install’ for compatibility with + earlier NumPy versions. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/199 + + +File: setuptools.info, Node: 3 5, Next: 3 4 4, Prev: 3 5 1, Up: History<2> + +9.340 3.5 +========= + +03 May 2014 + + * Issue #195(1): Follow symbolic links in find_packages (restoring + behavior broken in 3.4). + + * Issue #197(2): On Python 3.1, PKG-INFO is now saved in a UTF-8 + encoding instead of ‘sys.getpreferredencoding’ to match the + behavior on Python 2.6-3.4. + + * Issue #192(3): Preferred bootstrap location is now + ‘https://bootstrap.pypa.io/ez_setup.py’ (mirrored from former + location). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/195 + + (2) https://github.com/pypa/setuptools/issues/197 + + (3) https://github.com/pypa/setuptools/issues/192 + + +File: setuptools.info, Node: 3 4 4, Next: 3 4 3, Prev: 3 5, Up: History<2> + +9.341 3.4.4 +=========== + +11 Apr 2014 + + * Issue #184(1): Correct failure where find_package over-matched + packages when directory traversal isn’t short-circuited. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/184 + + +File: setuptools.info, Node: 3 4 3, Next: 3 4 2, Prev: 3 4 4, Up: History<2> + +9.342 3.4.3 +=========== + +07 Apr 2014 + + * Issue #183(1): Really fix test command with Python 3.1. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/183 + + +File: setuptools.info, Node: 3 4 2, Next: 3 4 1, Prev: 3 4 3, Up: History<2> + +9.343 3.4.2 +=========== + +06 Apr 2014 + + * Issue #183(1): Fix additional regression in test command on Python + 3.1. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/183 + + +File: setuptools.info, Node: 3 4 1, Next: 3 4, Prev: 3 4 2, Up: History<2> + +9.344 3.4.1 +=========== + +30 Mar 2014 + + * Issue #180(1): Fix regression in test command not caught by + py.test-run tests. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/180 + + +File: setuptools.info, Node: 3 4, Next: 3 3, Prev: 3 4 1, Up: History<2> + +9.345 3.4 +========= + +30 Mar 2014 + + * Issue #176(1): Add parameter to the test command to support a + custom test runner: –test-runner or -r. + + * Issue #177(2): Now assume most common invocation to install command + on platforms/environments without stack support (issuing a + warning). Setuptools now installs naturally on IronPython. + Behavior on CPython should be unchanged. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/176 + + (2) https://github.com/pypa/setuptools/issues/177 + + +File: setuptools.info, Node: 3 3, Next: 3 2, Prev: 3 4, Up: History<2> + +9.346 3.3 +========= + +16 Mar 2014 + + * Add ‘include’ parameter to ‘setuptools.find_packages()’. + + +File: setuptools.info, Node: 3 2, Next: 3 1, Prev: 3 3, Up: History<2> + +9.347 3.2 +========= + +14 Mar 2014 + + * BB Pull Request #39(1): Add support for C++ targets from Cython + ‘.pyx’ files. + + * Issue #162(2): Update dependency on certifi to 1.0.1. + + * Issue #164(3): Update dependency on wincertstore to 0.2. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/39 + + (2) https://github.com/pypa/setuptools/issues/162 + + (3) https://github.com/pypa/setuptools/issues/164 + + +File: setuptools.info, Node: 3 1, Next: 3 0 2, Prev: 3 2, Up: History<2> + +9.348 3.1 +========= + +08 Mar 2014 + + * Issue #161(1): Restore Features functionality to allow backward + compatibility (for Features) until the uses of that functionality + is sufficiently removed. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/161 + + +File: setuptools.info, Node: 3 0 2, Next: 3 0 1, Prev: 3 1, Up: History<2> + +9.349 3.0.2 +=========== + +06 Mar 2014 + + * Correct typo in previous bugfix. + + +File: setuptools.info, Node: 3 0 1, Next: 3 0, Prev: 3 0 2, Up: History<2> + +9.350 3.0.1 +=========== + +06 Mar 2014 + + * Issue #157(1): Restore support for Python 2.6 in bootstrap script + where ‘zipfile.ZipFile’ does not yet have support for context + managers. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/157 + + +File: setuptools.info, Node: 3 0, Next: 2 2, Prev: 3 0 1, Up: History<2> + +9.351 3.0 +========= + +04 Mar 2014 + + * Issue #125(1): Prevent Subversion support from creating a + ~/.subversion directory just for checking the presence of a + Subversion repository. + + * Issue #12(2): Namespace packages are now imported lazily. That is, + the mere declaration of a namespace package in an egg on ‘sys.path’ + no longer causes it to be imported when ‘pkg_resources’ is + imported. Note that this change means that all of a namespace + package’s ‘__init__.py’ files must include a ‘declare_namespace()’ + call in order to ensure that they will be handled properly at + runtime. In 2.x it was possible to get away without including the + declaration, but only at the cost of forcing namespace packages to + be imported early, which 3.0 no longer does. + + * Issue #148(3): When building (bdist_egg), setuptools no longer adds + ‘__init__.py’ files to namespace packages. Any packages that rely + on this behavior will need to create ‘__init__.py’ files and + include the ‘declare_namespace()’. + + * Issue #7(4): Setuptools itself is now distributed as a zip archive + in addition to tar archive. ez_setup.py now uses zip archive. + This approach avoids the potential security vulnerabilities + presented by use of tar archives in ez_setup.py. It also leverages + the security features added to ZipFile.extract in Python 2.7.4. + + * Issue #65(5): Removed deprecated Features functionality. + + * BB Pull Request #28(6): Remove backport of ‘_bytecode_filenames’ + which is available in Python 2.6 and later, but also has better + compatibility with Python 3 environments. + + * Issue #156(7): Fix spelling of __PYVENV_LAUNCHER__ variable. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/125 + + (2) https://github.com/pypa/setuptools/issues/12 + + (3) https://github.com/pypa/setuptools/issues/148 + + (4) https://github.com/pypa/setuptools/issues/7 + + (5) https://github.com/pypa/setuptools/issues/65 + + (6) https://bitbucket.org/pypa/setuptools/pull-request/28 + + (7) https://github.com/pypa/setuptools/issues/156 + + +File: setuptools.info, Node: 2 2, Next: 2 1 2, Prev: 3 0, Up: History<2> + +9.352 2.2 +========= + +07 Feb 2014 + + * Issue #141(1): Restored fix for allowing setup_requires + dependencies to override installed dependencies during setup. + + * Issue #128(2): Fixed issue where only the first dependency link was + honored in a distribution where multiple dependency links were + supplied. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/141 + + (2) https://github.com/pypa/setuptools/issues/128 + + +File: setuptools.info, Node: 2 1 2, Next: 2 1 1, Prev: 2 2, Up: History<2> + +9.353 2.1.2 +=========== + +05 Feb 2014 + + * Issue #144(1): Read long_description using codecs module to avoid + errors installing on systems where LANG=C. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/144 + + +File: setuptools.info, Node: 2 1 1, Next: 2 1, Prev: 2 1 2, Up: History<2> + +9.354 2.1.1 +=========== + +05 Feb 2014 + + * Issue #139(1): Fix regression in re_finder for CVS repos (and maybe + Git repos as well). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/139 + + +File: setuptools.info, Node: 2 1, Next: 2 0 2, Prev: 2 1 1, Up: History<2> + +9.355 2.1 +========= + +07 Jan 2014 + + * Issue #129(1): Suppress inspection of ‘*.whl’ files when searching + for files in a zip-imported file. + + * Issue #131(2): Fix RuntimeError when constructing an egg fetcher. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/129 + + (2) https://github.com/pypa/setuptools/issues/131 + + +File: setuptools.info, Node: 2 0 2, Next: 2 0 1, Prev: 2 1, Up: History<2> + +9.356 2.0.2 +=========== + +29 Dec 2013 + + * Fix NameError during installation with Python implementations (e.g. + Jython) not containing parser module. + + * Fix NameError in ‘sdist:re_finder’. + + +File: setuptools.info, Node: 2 0 1, Next: 2 0, Prev: 2 0 2, Up: History<2> + +9.357 2.0.1 +=========== + +15 Dec 2013 + + * Issue #124(1): Fixed error in list detection in upload_docs. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/124 + + +File: setuptools.info, Node: 2 0, Next: 1 4 2, Prev: 2 0 1, Up: History<2> + +9.358 2.0 +========= + +07 Dec 2013 + + * Issue #121(1): Exempt lib2to3 pickled grammars from + DirectorySandbox. + + * Issue #41(2): Dropped support for Python 2.4 and Python 2.5. + Clients requiring setuptools for those versions of Python should + use setuptools 1.x. + + * Removed ‘setuptools.command.easy_install.HAS_USER_SITE’. Clients + expecting this boolean variable should use ‘site.ENABLE_USER_SITE’ + instead. + + * Removed ‘pkg_resources.ImpWrapper’. Clients that expected this + class should use ‘pkgutil.ImpImporter’ instead. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/121 + + (2) https://github.com/pypa/setuptools/issues/41 + + +File: setuptools.info, Node: 1 4 2, Next: 1 4 1, Prev: 2 0, Up: History<2> + +9.359 1.4.2 +=========== + +01 Dec 2013 + + * Issue #116(1): Correct TypeError when reading a local package index + on Python 3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/116 + + +File: setuptools.info, Node: 1 4 1, Next: 1 4, Prev: 1 4 2, Up: History<2> + +9.360 1.4.1 +=========== + +23 Nov 2013 + + * Issue #114(1): Use ‘sys.getfilesystemencoding’ for decoding config + in ‘bdist_wininst’ distributions. + + * Issue #105(2) and Issue #113(3): Establish a more robust technique + for determining the terminal encoding: + + 1. Try ``getpreferredencoding`` + 2. If that returns US_ASCII or None, try the encoding from + ``getdefaultlocale``. If that encoding was a "fallback" because Python + could not figure it out from the environment or OS, encoding remains + unresolved. + 3. If the encoding is resolved, then make sure Python actually implements + the encoding. + 4. On the event of an error or unknown codec, revert to fallbacks + (UTF-8 on Darwin, ASCII on everything else). + 5. On the encoding is 'mac-roman' on Darwin, use UTF-8 as 'mac-roman' was + a bug on older Python releases. + + On a side note, it would seem that the encoding only matters for when SVN + does not yet support ``--xml`` and when getting repository and svn version + numbers. The ``--xml`` technique should yield UTF-8 according to some + messages on the SVN mailing lists. So if the version numbers are always + 7-bit ASCII clean, it may be best to only support the file parsing methods + for legacy SVN releases and support for SVN without the subprocess command + would simple go away as support for the older SVNs does. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/114 + + (2) https://github.com/pypa/setuptools/issues/105 + + (3) https://github.com/pypa/setuptools/issues/113 + + +File: setuptools.info, Node: 1 4, Next: 1 3 2, Prev: 1 4 1, Up: History<2> + +9.361 1.4 +========= + +17 Nov 2013 + + * Issue #27(1): ‘easy_install’ will now use credentials from .pypirc + if present for connecting to the package index. + + * BB Pull Request #21(2): Omit unwanted newlines in + ‘package_index._encode_auth’ when the username/password pair length + indicates wrapping. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/27 + + (2) https://bitbucket.org/pypa/setuptools/pull-request/21 + + +File: setuptools.info, Node: 1 3 2, Next: 1 3 1, Prev: 1 4, Up: History<2> + +9.362 1.3.2 +=========== + +09 Nov 2013 + + * Issue #99(1): Fix filename encoding issues in SVN support. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/99 + + +File: setuptools.info, Node: 1 3 1, Next: 1 3, Prev: 1 3 2, Up: History<2> + +9.363 1.3.1 +=========== + +07 Nov 2013 + + * Remove exuberant warning in SVN support when SVN is not used. + + +File: setuptools.info, Node: 1 3, Next: 1 2, Prev: 1 3 1, Up: History<2> + +9.364 1.3 +========= + +03 Nov 2013 + + * Address security vulnerability in SSL match_hostname check as + reported in Python #17997(1). + + * Prefer backports.ssl_match_hostname(2) for backport implementation + if present. + + * Correct NameError in ‘ssl_support’ module (‘socket.error’). + + ---------- Footnotes ---------- + + (1) http://bugs.python.org/issue17997 + + (2) https://pypi.org/project/backports.ssl_match_hostname/ + + +File: setuptools.info, Node: 1 2, Next: 1 1 7, Prev: 1 3, Up: History<2> + +9.365 1.2 +========= + +02 Nov 2013 + + * Issue #26(1): Add support for SVN 1.7. Special thanks to Philip + Thiem for the contribution. + + * Issue #93(2): Wheels are now distributed with every release. Note + that as reported in Issue #108(3), as of Pip 1.4, scripts aren’t + installed properly from wheels. Therefore, if using Pip to install + setuptools from a wheel, the ‘easy_install’ command will not be + available. + + * Setuptools “natural” launcher support, introduced in 1.0, is now + officially supported. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/26 + + (2) https://github.com/pypa/setuptools/issues/93 + + (3) https://github.com/pypa/setuptools/issues/108 + + +File: setuptools.info, Node: 1 1 7, Next: 1 1 6, Prev: 1 2, Up: History<2> + +9.366 1.1.7 +=========== + +11 Apr 2013 + + * Fixed behavior of NameError handling in ‘script template (dev).py’ + (script launcher for ‘develop’ installs). + + * ‘ez_setup.py’ now ensures partial downloads are cleaned up + following a failed download. + + * Distribute #363(1) and Issue #55(2): Skip an sdist test that fails + on locales other than UTF-8. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/363 + + (2) https://github.com/pypa/setuptools/issues/55 + + +File: setuptools.info, Node: 1 1 6, Next: 1 1 5, Prev: 1 1 7, Up: History<2> + +9.367 1.1.6 +=========== + +18 Sep 2013 + + * Distribute #349(1): ‘sandbox.execfile’ now opens the target file in + binary mode, thus honoring a BOM in the file when compiled. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/349 + + +File: setuptools.info, Node: 1 1 5, Next: 1 1 4, Prev: 1 1 6, Up: History<2> + +9.368 1.1.5 +=========== + +12 Sep 2013 + + * Issue #69(1): Second attempt at fix (logic was reversed). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/69 + + +File: setuptools.info, Node: 1 1 4, Next: 1 1 3, Prev: 1 1 5, Up: History<2> + +9.369 1.1.4 +=========== + +07 Sep 2013 + + * Issue #77(1): Fix error in upload command (Python 2.4). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/77 + + +File: setuptools.info, Node: 1 1 3, Next: 1 1 2, Prev: 1 1 4, Up: History<2> + +9.370 1.1.3 +=========== + +06 Sep 2013 + + * Fix NameError in previous patch. + + +File: setuptools.info, Node: 1 1 2, Next: 1 1 1, Prev: 1 1 3, Up: History<2> + +9.371 1.1.2 +=========== + +06 Sep 2013 + + * Issue #69(1): Correct issue where 404 errors are returned for URLs + with fragments in them (such as #egg=). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/69 + + +File: setuptools.info, Node: 1 1 1, Next: 1 1, Prev: 1 1 2, Up: History<2> + +9.372 1.1.1 +=========== + +03 Sep 2013 + + * Issue #75(1): Add ‘--insecure’ option to ez_setup.py to accommodate + environments where a trusted SSL connection cannot be validated. + + * Issue #76(2): Fix AttributeError in upload command with Python 2.4. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/75 + + (2) https://github.com/pypa/setuptools/issues/76 + + +File: setuptools.info, Node: 1 1, Next: 1 0, Prev: 1 1 1, Up: History<2> + +9.373 1.1 +========= + +26 Aug 2013 + + * Issue #71(1) (Distribute #333(2)): EasyInstall now puts less + emphasis on the condition when a host is blocked via + ‘--allow-hosts’. + + * Issue #72(3): Restored Python 2.4 compatibility in ‘ez_setup.py’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/71 + + (2) https://bitbucket.org/tarek/distribute/issue/333 + + (3) https://github.com/pypa/setuptools/issues/72 + + +File: setuptools.info, Node: 1 0, Next: 0 9 8, Prev: 1 1, Up: History<2> + +9.374 1.0 +========= + +17 Aug 2013 + + * Issue #60(1): On Windows, Setuptools supports deferring to another + launcher, such as Vinay Sajip’s pylauncher(2) (included with Python + 3.3) to launch console and GUI scripts and not install its own + launcher executables. This experimental functionality is currently + only enabled if the ‘SETUPTOOLS_LAUNCHER’ environment variable is + set to “natural”. In the future, this behavior may become default, + but only after it has matured and seen substantial adoption. The + ‘SETUPTOOLS_LAUNCHER’ also accepts “executable” to force the + default behavior of creating launcher executables. + + * Issue #63(3): Bootstrap script (ez_setup.py) now prefers + Powershell, curl, or wget for retrieving the Setuptools tarball for + improved security of the install. The script will still fall back + to a simple ‘urlopen’ on platforms that do not have these tools. + + * Issue #65(4): Deprecated the ‘Features’ functionality. + + * Issue #52(5): In ‘VerifyingHTTPSConn’, handle a tunnelled (proxied) + connection. + +* Menu: + +* Backward-Incompatible Changes:: + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/60 + + (2) https://bitbucket.org/pypa/pylauncher + + (3) https://github.com/pypa/setuptools/issues/63 + + (4) https://github.com/pypa/setuptools/issues/65 + + (5) https://github.com/pypa/setuptools/issues/52 + + +File: setuptools.info, Node: Backward-Incompatible Changes, Up: 1 0 + +9.374.1 Backward-Incompatible Changes +------------------------------------- + +This release includes a couple of backward-incompatible changes, but +most if not all users will find 1.0 a drop-in replacement for 0.9. + + * Issue #50(1): Normalized API of environment marker support. + Specifically, removed line number and filename from SyntaxErrors + when returned from ‘pkg_resources.invalid_marker’. Any clients + depending on the specific string representation of exceptions + returned by that function may need to be updated to account for + this change. + + * Issue #50(2): SyntaxErrors generated by + ‘pkg_resources.invalid_marker’ are normalized for + cross-implementation consistency. + + * Removed ‘--ignore-conflicts-at-my-risk’ and ‘--delete-conflicting’ + options to easy_install. These options have been deprecated since + 0.6a11. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/50 + + (2) https://github.com/pypa/setuptools/issues/50 + + +File: setuptools.info, Node: 0 9 8, Next: 0 9 7, Prev: 1 0, Up: History<2> + +9.375 0.9.8 +=========== + +25 Jul 2013 + + * Issue #53(1): Fix NameErrors in ‘_vcs_split_rev_from_url’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/53 + + +File: setuptools.info, Node: 0 9 7, Next: 0 9 6, Prev: 0 9 8, Up: History<2> + +9.376 0.9.7 +=========== + +22 Jul 2013 + + * Issue #49(1): Correct AttributeError on PyPy where a hashlib.HASH + object does not have a ‘.name’ attribute. + + * Issue #34(2): Documentation now refers to bootstrap script in code + repository referenced by bookmark. + + * Add underscore-separated keys to environment markers (markerlib). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/49 + + (2) https://github.com/pypa/setuptools/issues/34 + + +File: setuptools.info, Node: 0 9 6, Next: 0 9 5, Prev: 0 9 7, Up: History<2> + +9.377 0.9.6 +=========== + +17 Jul 2013 + + * Issue #44(1): Test failure on Python 2.4 when MD5 hash doesn’t have + a ‘.name’ attribute. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/44 + + +File: setuptools.info, Node: 0 9 5, Next: 0 9 4, Prev: 0 9 6, Up: History<2> + +9.378 0.9.5 +=========== + +15 Jul 2013 + + * Python #17980(1): Fix security vulnerability in SSL certificate + validation. + + ---------- Footnotes ---------- + + (1) http://bugs.python.org/issue17980 + + +File: setuptools.info, Node: 0 9 4, Next: 0 9 3, Prev: 0 9 5, Up: History<2> + +9.379 0.9.4 +=========== + +15 Jul 2013 + + * Issue #43(1): Fix issue (introduced in 0.9.1) with version + resolution when upgrading over other releases of Setuptools. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/43 + + +File: setuptools.info, Node: 0 9 3, Next: 0 9 2, Prev: 0 9 4, Up: History<2> + +9.380 0.9.3 +=========== + +15 Jul 2013 + + * Issue #42(1): Fix new ‘AttributeError’ introduced in last fix. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/42 + + +File: setuptools.info, Node: 0 9 2, Next: 0 9 1, Prev: 0 9 3, Up: History<2> + +9.381 0.9.2 +=========== + +15 Jul 2013 + + * Issue #42(1): Fix regression where blank checksums would trigger an + ‘AttributeError’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/42 + + +File: setuptools.info, Node: 0 9 1, Next: 0 9, Prev: 0 9 2, Up: History<2> + +9.382 0.9.1 +=========== + +13 Jul 2013 + + * Distribute #386(1): Allow other positional and keyword arguments to + os.open. + + * Corrected dependency on certifi mis-referenced in 0.9. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/386 + + +File: setuptools.info, Node: 0 9, Next: 0 8, Prev: 0 9 1, Up: History<2> + +9.383 0.9 +========= + +13 Jul 2013 + + * ‘package_index’ now validates hashes other than MD5 in download + links. + + +File: setuptools.info, Node: 0 8, Next: 0 7 8, Prev: 0 9, Up: History<2> + +9.384 0.8 +========= + +05 Jul 2013 + + * Code base now runs on Python 2.4 - Python 3.3 without Python 2to3 + conversion. + + +File: setuptools.info, Node: 0 7 8, Next: 0 7 7, Prev: 0 8, Up: History<2> + +9.385 0.7.8 +=========== + +04 Jul 2013 + + * Distribute #375(1): Yet another fix for yet another regression. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/375 + + +File: setuptools.info, Node: 0 7 7, Next: 0 7 6, Prev: 0 7 8, Up: History<2> + +9.386 0.7.7 +=========== + +02 Jul 2013 + + * Distribute #375(1): Repair AttributeError created in last release + (redo). + + * Issue #30(2): Added test for get_cache_path. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/375 + + (2) https://github.com/pypa/setuptools/issues/30 + + +File: setuptools.info, Node: 0 7 6, Next: 0 7 5, Prev: 0 7 7, Up: History<2> + +9.387 0.7.6 +=========== + +02 Jul 2013 + + * Distribute #375(1): Repair AttributeError created in last release. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/375 + + +File: setuptools.info, Node: 0 7 5, Next: 0 7 4, Prev: 0 7 6, Up: History<2> + +9.388 0.7.5 +=========== + +29 Jun 2013 + + * Issue #21(1): Restore Python 2.4 compatibility in + ‘test_easy_install’. + + * Distribute #375(2): Merged additional warning from Distribute + 0.6.46. + + * Now honor the environment variable + ‘SETUPTOOLS_DISABLE_VERSIONED_EASY_INSTALL_SCRIPT’ in addition to + the now deprecated + ‘DISTRIBUTE_DISABLE_VERSIONED_EASY_INSTALL_SCRIPT’. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/21 + + (2) https://bitbucket.org/tarek/distribute/issue/375 + + +File: setuptools.info, Node: 0 7 4, Next: 0 7 3, Prev: 0 7 5, Up: History<2> + +9.389 0.7.4 +=========== + +19 Jun 2013 + + * Issue #20(1): Fix comparison of parsed SVN version on Python 3. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/20 + + +File: setuptools.info, Node: 0 7 3, Next: 0 7 2, Prev: 0 7 4, Up: History<2> + +9.390 0.7.3 +=========== + +18 Jun 2013 + + * Issue #1(1): Disable installation of Windows-specific files on + non-Windows systems. + + * Use new sysconfig module with Python 2.7 or >=3.2. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/1 + + +File: setuptools.info, Node: 0 7 2, Next: 0 7 1, Prev: 0 7 3, Up: History<2> + +9.391 0.7.2 +=========== + +09 Jun 2013 + + * Issue #14(1): Use markerlib when the ‘parser’ module is not + available. + + * Issue #10(2): ‘ez_setup.py’ now uses HTTPS to download setuptools + from PyPI. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/14 + + (2) https://github.com/pypa/setuptools/issues/10 + + +File: setuptools.info, Node: 0 7 1, Next: 0 7, Prev: 0 7 2, Up: History<2> + +9.392 0.7.1 +=========== + +03 Jun 2013 + + * Fix NameError (Issue #3(1)) again - broken in bad merge. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/3 + + +File: setuptools.info, Node: 0 7, Next: 0 7b4, Prev: 0 7 1, Up: History<2> + +9.393 0.7 +========= + +02 Jun 2013 + + * Merged Setuptools and Distribute. See docs/merge.txt for details. + +Added several features that were slated for setuptools 0.6c12: + + * Index URL now defaults to HTTPS. + + * Added experimental environment marker support. Now clients may + designate a PEP-426(1) environment marker for “extra” dependencies. + Setuptools uses this feature in ‘setup.py’ for optional SSL and + certificate validation support on older platforms. Based on + Distutils-SIG discussions, the syntax is somewhat tentative. There + should probably be a PEP with a firmer spec before the feature + should be considered suitable for use. + + * Added support for SSL certificate validation when installing + packages from an HTTPS service. + + ---------- Footnotes ---------- + + (1) https://www.python.org/dev/peps/pep-0426/ + + +File: setuptools.info, Node: 0 7b4, Next: 0 6 49, Prev: 0 7, Up: History<2> + +9.394 0.7b4 +=========== + + * Issue #3(1): Fixed NameError in SSL support. + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/3 + + +File: setuptools.info, Node: 0 6 49, Next: 0 6 48, Prev: 0 7b4, Up: History<2> + +9.395 0.6.49 +============ + +04 Jul 2013 + + * Move warning check in ‘get_cache_path’ to follow the directory + creation to avoid errors when the cache path does not yet exist. + Fixes the error reported in Distribute #375(1). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/375 + + +File: setuptools.info, Node: 0 6 48, Next: 0 6 47, Prev: 0 6 49, Up: History<2> + +9.396 0.6.48 +============ + +02 Jul 2013 + + * Correct AttributeError in ‘ResourceManager.get_cache_path’ + introduced in 0.6.46 (redo). + + +File: setuptools.info, Node: 0 6 47, Next: 0 6 46, Prev: 0 6 48, Up: History<2> + +9.397 0.6.47 +============ + +02 Jul 2013 + + * Correct AttributeError in ‘ResourceManager.get_cache_path’ + introduced in 0.6.46. + + +File: setuptools.info, Node: 0 6 46, Next: 0 6 45, Prev: 0 6 47, Up: History<2> + +9.398 0.6.46 +============ + +29 Jun 2013 + + * Distribute #375(1): Issue a warning if the PYTHON_EGG_CACHE or + otherwise customized egg cache location specifies a directory + that’s group- or world-writable. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/375 + + +File: setuptools.info, Node: 0 6 45, Next: 0 6 44, Prev: 0 6 46, Up: History<2> + +9.399 0.6.45 +============ + +29 May 2013 + + * Distribute #379(1): ‘distribute_setup.py’ now traps VersionConflict + as well, restoring ability to upgrade from an older setuptools + version. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/379 + + +File: setuptools.info, Node: 0 6 44, Next: 0 6 43, Prev: 0 6 45, Up: History<2> + +9.400 0.6.44 +============ + +28 May 2013 + + * ‘distribute_setup.py’ has been updated to allow Setuptools 0.7 to + satisfy use_setuptools. + + +File: setuptools.info, Node: 0 6 43, Next: 0 6 42, Prev: 0 6 44, Up: History<2> + +9.401 0.6.43 +============ + +24 May 2013 + + * Distribute #378(1): Restore support for Python 2.4 Syntax + (regression in 0.6.42). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/378 + + +File: setuptools.info, Node: 0 6 42, Next: 0 6 41, Prev: 0 6 43, Up: History<2> + +9.402 0.6.42 +============ + +24 May 2013 + + * External links finder no longer yields duplicate links. + + * Distribute #337(1): Moved site.py to setuptools/site-patch.py + (graft of very old patch from setuptools trunk which inspired PR + #31(2)). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/337 + + (2) https://github.com/pypa/setuptools/issues/31 + + +File: setuptools.info, Node: 0 6 41, Next: 0 6 40, Prev: 0 6 42, Up: History<2> + +9.403 0.6.41 +============ + +24 May 2013 + + * Distribute #27(1): Use public api for loading resources from zip + files rather than the private method ‘_zip_directory_cache’. + + * Added a new function ‘easy_install.get_win_launcher’ which may be + used by third-party libraries such as buildout to get a suitable + script launcher. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/27 + + +File: setuptools.info, Node: 0 6 40, Next: 0 6 39, Prev: 0 6 41, Up: History<2> + +9.404 0.6.40 +============ + +14 May 2013 + + * Distribute #376(1): brought back cli.exe and gui.exe that were + deleted in the previous release. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/376 + + +File: setuptools.info, Node: 0 6 39, Next: 0 6 38, Prev: 0 6 40, Up: History<2> + +9.405 0.6.39 +============ + +12 May 2013 + + * Add support for console launchers on ARM platforms. + + * Fix possible issue in GUI launchers where the subsystem was not + supplied to the linker. + + * Launcher build script now refactored for robustness. + + * Distribute #375(1): Resources extracted from a zip egg to the file + system now also check the contents of the file against the zip + contents during each invocation of get_resource_filename. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/375 + + +File: setuptools.info, Node: 0 6 38, Next: 0 6 37, Prev: 0 6 39, Up: History<2> + +9.406 0.6.38 +============ + +05 May 2013 + + * Distribute #371(1): The launcher manifest file is now installed + properly. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/371 + + +File: setuptools.info, Node: 0 6 37, Next: 0 6 36, Prev: 0 6 38, Up: History<2> + +9.407 0.6.37 +============ + +04 May 2013 + + * Distribute #143(1): Launcher scripts, including easy_install + itself, are now accompanied by a manifest on 32-bit Windows + environments to avoid the Installer Detection Technology and thus + undesirable UAC elevation described in this Microsoft article(2). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/143 + + (2) +http://technet.microsoft.com/en-us/library/cc709628%28WS.10%29.aspx + + +File: setuptools.info, Node: 0 6 36, Next: 0 6 35, Prev: 0 6 37, Up: History<2> + +9.408 0.6.36 +============ + +05 Apr 2013 + + * BB Pull Request #35(1): In Buildout #64(2), it was reported that + under Python 3, installation of distutils scripts could attempt to + copy the ‘__pycache__’ directory as a file, causing an error, + apparently only under Windows. Easy_install now skips all + directories when processing metadata scripts. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/35 + + (2) https://github.com/buildout/buildout/issues/64 + + +File: setuptools.info, Node: 0 6 35, Next: 0 6 34, Prev: 0 6 36, Up: History<2> + +9.409 0.6.35 +============ + +16 Feb 2013 + +Note this release is backward-incompatible with distribute 0.6.23-0.6.34 +in how it parses version numbers. + + * Distribute #278(1): Restored compatibility with distribute 0.6.22 + and setuptools 0.6. Updated the documentation to match more + closely with the version parsing as intended in setuptools 0.6. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/278 + + +File: setuptools.info, Node: 0 6 34, Next: 0 6 33, Prev: 0 6 35, Up: History<2> + +9.410 0.6.34 +============ + +30 Dec 2012 + + * Distribute #341(1): 0.6.33 fails to build under Python 2.4. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/341 + + +File: setuptools.info, Node: 0 6 33, Next: 0 6 32, Prev: 0 6 34, Up: History<2> + +9.411 0.6.33 +============ + +29 Dec 2012 + + * Fix 2 errors with Jython 2.5. + + * Fix 1 failure with Jython 2.5 and 2.7. + + * Disable workaround for Jython scripts on Linux systems. + + * Distribute #336(1): ‘setup.py’ no longer masks failure exit code + when tests fail. + + * Fix issue in pkg_resources where try/except around a + platform-dependent import would trigger hook load failures on + Mercurial. See pull request 32 for details. + + * Distribute #341(2): Fix a ResourceWarning. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/336 + + (2) https://bitbucket.org/tarek/distribute/issue/341 + + +File: setuptools.info, Node: 0 6 32, Next: 0 6 31, Prev: 0 6 33, Up: History<2> + +9.412 0.6.32 +============ + +26 Nov 2012 + + * Fix test suite with Python 2.6. + + * Fix some DeprecationWarnings and ResourceWarnings. + + * Distribute #335(1): Backed out ‘setup_requires’ superceding + installed requirements until regression can be addressed. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/335 + + +File: setuptools.info, Node: 0 6 31, Next: 0 6 30, Prev: 0 6 32, Up: History<2> + +9.413 0.6.31 +============ + +24 Nov 2012 + + * Distribute #303(1): Make sure the manifest only ever contains UTF-8 + in Python 3. + + * Distribute #329(2): Properly close files created by tests for + compatibility with Jython. + + * Work around Jython #1980(3) and Jython #1981(4). + + * Distribute #334(5): Provide workaround for packages that reference + ‘sys.__stdout__’ such as numpy does. This change should address + virtualenv ‘#359(6) + <‘https://github.com/pypa/virtualenv/issues/359’>‘_ as long as the + system encoding is UTF-8 or the IO encoding is specified in the + environment, i.e.: + + PYTHONIOENCODING=utf8 pip install numpy + + * Fix for encoding issue when installing from Windows executable on + Python 3. + + * Distribute #323(7): Allow ‘setup_requires’ requirements to + supercede installed requirements. Added some new keyword arguments + to existing pkg_resources methods. Also had to updated how + __path__ is handled for namespace packages to ensure that when a + new egg distribution containing a namespace package is placed on + sys.path, the entries in __path__ are found in the same order they + would have been in had that egg been on the path when pkg_resources + was first imported. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/303 + + (2) https://bitbucket.org/tarek/distribute/issue/329 + + (3) http://bugs.jython.org/issue1980 + + (4) http://bugs.jython.org/issue1981 + + (5) https://bitbucket.org/tarek/distribute/issue/334 + + (6) https://github.com/pypa/setuptools/issues/359 + + (7) https://bitbucket.org/tarek/distribute/issue/323 + + +File: setuptools.info, Node: 0 6 30, Next: 0 6 29, Prev: 0 6 31, Up: History<2> + +9.414 0.6.30 +============ + +22 Oct 2012 + + * Distribute #328(1): Clean up temporary directories in + distribute_setup.py. + + * Fix fatal bug in distribute_setup.py. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/328 + + +File: setuptools.info, Node: 0 6 29, Next: 0 6 28, Prev: 0 6 30, Up: History<2> + +9.415 0.6.29 +============ + +21 Oct 2012 + + * BB Pull Request #14(1): Honor file permissions in zip files. + + * Distribute #327(2): Merged pull request #24(3) to fix a dependency + problem with pip. + + * Merged pull request #23(4) to fix + ‘https://github.com/pypa/virtualenv/issues/301’. + + * If Sphinx is installed, the ‘upload_docs’ command now runs + ‘build_sphinx’ to produce uploadable documentation. + + * Distribute #326(5): ‘upload_docs’ provided mangled auth credentials + under Python 3. + + * Distribute #320(6): Fix check for “createable” in + distribute_setup.py. + + * Distribute #305(7): Remove a warning that was triggered during + normal operations. + + * Distribute #311(8): Print metadata in UTF-8 independent of + platform. + + * Distribute #303(9): Read manifest file with UTF-8 encoding under + Python 3. + + * Distribute #301(10): Allow to run tests of namespace packages when + using 2to3. + + * Distribute #304(11): Prevent import loop in site.py under Python + 3.3. + + * Distribute #283(12): Reenable scanning of ‘*.pyc’ / ‘*.pyo’ files + on Python 3.3. + + * Distribute #299(13): The develop command didn’t work on Python 3, + when using 2to3, as the egg link would go to the Python 2 source. + Linking to the 2to3’d code in build/lib makes it work, although you + will have to rebuild the module before testing it. + + * Distribute #306(14): Even if 2to3 is used, we build in-place under + Python 2. + + * Distribute #307(15): Prints the full path when .svn/entries is + broken. + + * Distribute #313(16): Support for sdist subcommands (Python 2.7) + + * Distribute #314(17): test_local_index() would fail an OS X. + + * Distribute #310(18): Non-ascii characters in a namespace + __init__.py causes errors. + + * Distribute #218(19): Improved documentation on behavior of + ‘package_data’ and ‘include_package_data’. Files indicated by + ‘package_data’ are now included in the manifest. + + * ‘distribute_setup.py’ now allows a ‘--download-base’ argument for + retrieving distribute from a specified location. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/pypa/setuptools/pull-request/14 + + (2) https://bitbucket.org/tarek/distribute/issue/327 + + (3) https://github.com/pypa/setuptools/issues/24 + + (4) https://github.com/pypa/setuptools/issues/23 + + (5) https://bitbucket.org/tarek/distribute/issue/326 + + (6) https://bitbucket.org/tarek/distribute/issue/320 + + (7) https://bitbucket.org/tarek/distribute/issue/305 + + (8) https://bitbucket.org/tarek/distribute/issue/311 + + (9) https://bitbucket.org/tarek/distribute/issue/303 + + (10) https://bitbucket.org/tarek/distribute/issue/301 + + (11) https://bitbucket.org/tarek/distribute/issue/304 + + (12) https://bitbucket.org/tarek/distribute/issue/283 + + (13) https://bitbucket.org/tarek/distribute/issue/299 + + (14) https://bitbucket.org/tarek/distribute/issue/306 + + (15) https://bitbucket.org/tarek/distribute/issue/307 + + (16) https://bitbucket.org/tarek/distribute/issue/313 + + (17) https://bitbucket.org/tarek/distribute/issue/314 + + (18) https://bitbucket.org/tarek/distribute/issue/310 + + (19) https://bitbucket.org/tarek/distribute/issue/218 + + +File: setuptools.info, Node: 0 6 28, Next: 0 6 27, Prev: 0 6 29, Up: History<2> + +9.416 0.6.28 +============ + +22 Jul 2012 + + * Distribute #294(1): setup.py can now be invoked from any directory. + + * Scripts are now installed honoring the umask. + + * Added support for .dist-info directories. + + * Distribute #283(2): Fix and disable scanning of ‘*.pyc’ / ‘*.pyo’ + files on Python 3.3. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/294 + + (2) https://bitbucket.org/tarek/distribute/issue/283 + + +File: setuptools.info, Node: 0 6 27, Next: 0 6 26, Prev: 0 6 28, Up: History<2> + +9.417 0.6.27 +============ + +18 May 2012 + + * Support current snapshots of CPython 3.3. + + * Distribute now recognizes README.rst as a standard, default readme + file. + + * Exclude ‘encodings’ modules when removing modules from sys.modules. + Workaround for #285(1). + + * Distribute #231(2): Don’t fiddle with system python when used with + buildout (bootstrap.py) + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/285 + + (2) https://bitbucket.org/tarek/distribute/issue/231 + + +File: setuptools.info, Node: 0 6 26, Next: 0 6 25, Prev: 0 6 27, Up: History<2> + +9.418 0.6.26 +============ + +08 Apr 2012 + + * Distribute #183(1): Symlinked files are now extracted from source + distributions. + + * Distribute #227(2): Easy_install fetch parameters are now passed + during the installation of a source distribution; now fulfillment + of setup_requires dependencies will honor the parameters passed to + easy_install. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/183 + + (2) https://bitbucket.org/tarek/distribute/issue/227 + + +File: setuptools.info, Node: 0 6 25, Next: 0 6 24, Prev: 0 6 26, Up: History<2> + +9.419 0.6.25 +============ + +08 Feb 2012 + + * Distribute #258(1): Workaround a cache issue + + * Distribute #260(2): distribute_setup.py now accepts the –user + parameter for Python 2.6 and later. + + * Distribute #262(3): package_index.open_with_auth no longer throws + LookupError on Python 3. + + * Distribute #269(4): AttributeError when an exception occurs reading + Manifest.in on late releases of Python. + + * Distribute #272(5): Prevent TypeError when namespace package names + are unicode and single-install-externally-managed is used. Also + fixes PIP issue 449. + + * Distribute #273(6): Legacy script launchers now install with + Python2/3 support. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/258 + + (2) https://bitbucket.org/tarek/distribute/issue/260 + + (3) https://bitbucket.org/tarek/distribute/issue/262 + + (4) https://bitbucket.org/tarek/distribute/issue/269 + + (5) https://bitbucket.org/tarek/distribute/issue/272 + + (6) https://bitbucket.org/tarek/distribute/issue/273 + + +File: setuptools.info, Node: 0 6 24, Next: 0 6 23, Prev: 0 6 25, Up: History<2> + +9.420 0.6.24 +============ + +14 Oct 2011 + + * Distribute #249(1): Added options to exclude 2to3 fixers + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/249 + + +File: setuptools.info, Node: 0 6 23, Next: 0 6 21, Prev: 0 6 24, Up: History<2> + +9.421 0.6.23 +============ + +22 Sep 2011 + + * Distribute #244(1): Fixed a test + + * Distribute #243(2): Fixed a test + + * Distribute #239(3): Fixed a test + + * Distribute #240(4): Fixed a test + + * Distribute #241(5): Fixed a test + + * Distribute #237(6): Fixed a test + + * Distribute #238(7): easy_install now uses 64bit executable wrappers + on 64bit Python + + * Distribute #208(8): Fixed parsed_versions, it now honors + post-releases as noted in the documentation + + * Distribute #207(9): Windows cli and gui wrappers pass CTRL-C to + child python process + + * Distribute #227(10): easy_install now passes its arguments to + setup.py bdist_egg + + * Distribute #225(11): Fixed a NameError on Python 2.5, 2.4 + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/244 + + (2) https://bitbucket.org/tarek/distribute/issue/243 + + (3) https://bitbucket.org/tarek/distribute/issue/239 + + (4) https://bitbucket.org/tarek/distribute/issue/240 + + (5) https://bitbucket.org/tarek/distribute/issue/241 + + (6) https://bitbucket.org/tarek/distribute/issue/237 + + (7) https://bitbucket.org/tarek/distribute/issue/238 + + (8) https://bitbucket.org/tarek/distribute/issue/208 + + (9) https://bitbucket.org/tarek/distribute/issue/207 + + (10) https://bitbucket.org/tarek/distribute/issue/227 + + (11) https://bitbucket.org/tarek/distribute/issue/225 + + +File: setuptools.info, Node: 0 6 21, Next: 0 6 20, Prev: 0 6 23, Up: History<2> + +9.422 0.6.21 +============ + +20 Aug 2011 + + * Distribute #225(1): FIxed a regression on py2.4 + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/225 + + +File: setuptools.info, Node: 0 6 20, Next: 0 6 19, Prev: 0 6 21, Up: History<2> + +9.423 0.6.20 +============ + +18 Aug 2011 + + * Distribute #135(1): Include url in warning when processing URLs in + package_index. + + * Distribute #212(2): Fix issue where easy_instal fails on Python 3 + on windows installer. + + * Distribute #213(3): Fix typo in documentation. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/135 + + (2) https://bitbucket.org/tarek/distribute/issue/212 + + (3) https://bitbucket.org/tarek/distribute/issue/213 + + +File: setuptools.info, Node: 0 6 19, Next: 0 6 18, Prev: 0 6 20, Up: History<2> + +9.424 0.6.19 +============ + +02 Jun 2011 + + * Distribute #206(1): AttributeError: ‘HTTPMessage’ object has no + attribute ‘getheaders’ + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/206 + + +File: setuptools.info, Node: 0 6 18, Next: 0 6 17, Prev: 0 6 19, Up: History<2> + +9.425 0.6.18 +============ + +01 Jun 2011 + + * Distribute #210(1): Fixed a regression introduced by Distribute + #204(2) fix. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/210 + + (2) https://bitbucket.org/tarek/distribute/issue/204 + + +File: setuptools.info, Node: 0 6 17, Next: 0 6 16, Prev: 0 6 18, Up: History<2> + +9.426 0.6.17 +============ + +30 May 2011 + + * Support ‘DISTRIBUTE_DISABLE_VERSIONED_EASY_INSTALL_SCRIPT’ + environment variable to allow to disable installation of + easy_install-${version} script. + + * Support Python >=3.1.4 and >=3.2.1. + + * Distribute #204(1): Don’t try to import the parent of a namespace + package in declare_namespace + + * Distribute #196(2): Tolerate responses with multiple Content-Length + headers + + * Distribute #205(3): Sandboxing doesn’t preserve working_set. Leads + to setup_requires problems. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/204 + + (2) https://bitbucket.org/tarek/distribute/issue/196 + + (3) https://bitbucket.org/tarek/distribute/issue/205 + + +File: setuptools.info, Node: 0 6 16, Next: 0 6 15, Prev: 0 6 17, Up: History<2> + +9.427 0.6.16 +============ + +28 Apr 2011 + + * Builds sdist gztar even on Windows (avoiding Distribute #193(1)). + + * Distribute #192(2): Fixed metadata omitted on Windows when + package_dir specified with forward-slash. + + * Distribute #195(3): Cython build support. + + * Distribute #200(4): Issues with recognizing 64-bit packages on + Windows. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/193 + + (2) https://bitbucket.org/tarek/distribute/issue/192 + + (3) https://bitbucket.org/tarek/distribute/issue/195 + + (4) https://bitbucket.org/tarek/distribute/issue/200 + + +File: setuptools.info, Node: 0 6 15, Next: 0 6 14, Prev: 0 6 16, Up: History<2> + +9.428 0.6.15 +============ + +12 Mar 2011 + + * Fixed typo in bdist_egg + + * Several issues under Python 3 has been solved. + + * Distribute #146(1): Fixed missing DLL files after easy_install of + windows exe package. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/146 + + +File: setuptools.info, Node: 0 6 14, Next: 0 6 13, Prev: 0 6 15, Up: History<2> + +9.429 0.6.14 +============ + +15 Jul 2010 + + * Distribute #170(1): Fixed unittest failure. Thanks to Toshio. + + * Distribute #171(2): Fixed race condition in unittests cause + deadlocks in test suite. + + * Distribute #143(3): Fixed a lookup issue with easy_install. Thanks + to David and Zooko. + + * Distribute #174(4): Fixed the edit mode when its used with + setuptools itself + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/170 + + (2) https://bitbucket.org/tarek/distribute/issue/171 + + (3) https://bitbucket.org/tarek/distribute/issue/143 + + (4) https://bitbucket.org/tarek/distribute/issue/174 + + +File: setuptools.info, Node: 0 6 13, Next: 0 6 12, Prev: 0 6 14, Up: History<2> + +9.430 0.6.13 +============ + +31 May 2010 + + * Distribute #160(1): 2.7 gives ValueError(“Invalid IPv6 URL”) + + * Distribute #150(2): Fixed using ~/.local even in a + –no-site-packages virtualenv + + * Distribute #163(3): scan index links before external links, and + don’t use the md5 when comparing two distributions + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/160 + + (2) https://bitbucket.org/tarek/distribute/issue/150 + + (3) https://bitbucket.org/tarek/distribute/issue/163 + + +File: setuptools.info, Node: 0 6 12, Next: 0 6 11, Prev: 0 6 13, Up: History<2> + +9.431 0.6.12 +============ + +06 May 2010 + + * Distribute #149(1): Fixed various failures on 2.3/2.4 + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/149 + + +File: setuptools.info, Node: 0 6 11, Next: 0 6 10, Prev: 0 6 12, Up: History<2> + +9.432 0.6.11 +============ + +06 May 2010 + + * Found another case of SandboxViolation - fixed + + * Distribute #15(1) and Distribute #48(2): Introduced a socket + timeout of 15 seconds on url openings + + * Added indexsidebar.html into MANIFEST.in + + * Distribute #108(3): Fixed TypeError with Python3.1 + + * Distribute #121(4): Fixed –help install command trying to actually + install. + + * Distribute #112(5): Added an os.makedirs so that Tarek’s solution + will work. + + * Distribute #133(6): Added –no-find-links to easy_install + + * Added easy_install –user + + * Distribute #100(7): Fixed develop –user not taking ‘.’ in + PYTHONPATH into account + + * Distribute #134(8): removed spurious UserWarnings. Patch by + VanLindberg + + * Distribute #138(9): cant_write_to_target error when setup_requires + is used. + + * Distribute #147(10): respect the sys.dont_write_bytecode flag + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/15 + + (2) https://bitbucket.org/tarek/distribute/issue/48 + + (3) https://bitbucket.org/tarek/distribute/issue/108 + + (4) https://bitbucket.org/tarek/distribute/issue/121 + + (5) https://bitbucket.org/tarek/distribute/issue/112 + + (6) https://bitbucket.org/tarek/distribute/issue/133 + + (7) https://bitbucket.org/tarek/distribute/issue/100 + + (8) https://bitbucket.org/tarek/distribute/issue/134 + + (9) https://bitbucket.org/tarek/distribute/issue/138 + + (10) https://bitbucket.org/tarek/distribute/issue/147 + + +File: setuptools.info, Node: 0 6 10, Next: 0 6 9, Prev: 0 6 11, Up: History<2> + +9.433 0.6.10 +============ + +12 Dec 2009 + + * Reverted change made for the DistributionNotFound exception because + zc.buildout uses the exception message to get the name of the + distribution. + + +File: setuptools.info, Node: 0 6 9, Next: 0 6 8, Prev: 0 6 10, Up: History<2> + +9.434 0.6.9 +=========== + +12 Dec 2009 + + * Distribute #90(1): unknown setuptools version can be added in the + working set + + * Distribute #87(2): setupt.py doesn’t try to convert + distribute_setup.py anymore Initial Patch by arfrever. + + * Distribute #89(3): added a side bar with a download link to the + doc. + + * Distribute #86(4): fixed missing sentence in pkg_resources doc. + + * Added a nicer error message when a DistributionNotFound is raised. + + * Distribute #80(5): test_develop now works with Python 3.1 + + * Distribute #93(6): upload_docs now works if there is an empty + sub-directory. + + * Distribute #70(7): exec bit on non-exec files + + * Distribute #99(8): now the standalone easy_install command doesn’t + uses a “setup.cfg” if any exists in the working directory. It will + use it only if triggered by ‘install_requires’ from a setup.py call + (install, develop, etc). + + * Distribute #101(9): Allowing ‘os.devnull’ in Sandbox + + * Distribute #92(10): Fixed the “no eggs” found error with MacPort + (platform.mac_ver() fails) + + * Distribute #103(11): test_get_script_header_jython_workaround not + run anymore under py3 with C or POSIX local. Contributed by + Arfrever. + + * Distribute #104(12): remvoved the assertion when the installation + fails, with a nicer message for the end user. + + * Distribute #100(13): making sure there’s no SandboxViolation when + the setup script patches setuptools. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/90 + + (2) https://bitbucket.org/tarek/distribute/issue/87 + + (3) https://bitbucket.org/tarek/distribute/issue/89 + + (4) https://bitbucket.org/tarek/distribute/issue/86 + + (5) https://bitbucket.org/tarek/distribute/issue/80 + + (6) https://bitbucket.org/tarek/distribute/issue/93 + + (7) https://bitbucket.org/tarek/distribute/issue/70 + + (8) https://bitbucket.org/tarek/distribute/issue/99 + + (9) https://bitbucket.org/tarek/distribute/issue/101 + + (10) https://bitbucket.org/tarek/distribute/issue/92 + + (11) https://bitbucket.org/tarek/distribute/issue/103 + + (12) https://bitbucket.org/tarek/distribute/issue/104 + + (13) https://bitbucket.org/tarek/distribute/issue/100 + + +File: setuptools.info, Node: 0 6 8, Next: 0 6 7, Prev: 0 6 9, Up: History<2> + +9.435 0.6.8 +=========== + +01 Nov 2009 + + * Added “check_packages” in dist. (added in Setuptools 0.6c11) + + * Fixed the DONT_PATCH_SETUPTOOLS state. + + +File: setuptools.info, Node: 0 6 7, Next: 0 6 6, Prev: 0 6 8, Up: History<2> + +9.436 0.6.7 +=========== + +01 Nov 2009 + + * Distribute #58(1): Added –user support to the develop command + + * Distribute #11(2): Generated scripts now wrap their call to the + script entry point in the standard “if name == ‘main’” + + * Added the ‘DONT_PATCH_SETUPTOOLS’ environment variable, so + virtualenv can drive an installation that doesn’t patch a global + setuptools. + + * Reviewed unladen-swallow specific change from + ‘http://code.google.com/p/unladen-swallow/source/detail?spec=svn875&r=719’ + and determined that it no longer applies. Distribute should work + fine with Unladen Swallow 2009Q3. + + * Distribute #21(3): Allow PackageIndex.open_url to gracefully handle + all cases of a httplib.HTTPException instead of just InvalidURL and + BadStatusLine. + + * Removed virtual-python.py from this distribution and updated + documentation to point to the actively maintained virtualenv + instead. + + * Distribute #64(4): use_setuptools no longer rebuilds the distribute + egg every time it is run + + * use_setuptools now properly respects the requested version + + * use_setuptools will no longer try to import a distribute egg for + the wrong Python version + + * Distribute #74(5): no_fake should be True by default. + + * Distribute #72(6): avoid a bootstrapping issue with easy_install -U + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/58 + + (2) https://bitbucket.org/tarek/distribute/issue/11 + + (3) https://bitbucket.org/tarek/distribute/issue/21 + + (4) https://bitbucket.org/tarek/distribute/issue/64 + + (5) https://bitbucket.org/tarek/distribute/issue/74 + + (6) https://bitbucket.org/tarek/distribute/issue/72 + + +File: setuptools.info, Node: 0 6 6, Next: 0 6 5, Prev: 0 6 7, Up: History<2> + +9.437 0.6.6 +=========== + +15 Oct 2009 + + * Unified the bootstrap file so it works on both py2.x and py3k + without 2to3 (patch by Holger Krekel) + + +File: setuptools.info, Node: 0 6 5, Next: 0 6 4, Prev: 0 6 6, Up: History<2> + +9.438 0.6.5 +=========== + +15 Oct 2009 + + * Distribute #65(1): cli.exe and gui.exe are now generated at build + time, depending on the platform in use. + + * Distribute #67(2): Fixed doc typo (PEP 381(3)/PEP 382(4)). + + * Distribute no longer shadows setuptools if we require a 0.7-series + setuptools. And an error is raised when installing a 0.7 + setuptools with distribute. + + * When run from within buildout, no attempt is made to modify an + existing setuptools egg, whether in a shared egg directory or a + system setuptools. + + * Fixed a hole in sandboxing allowing builtin file to write outside + of the sandbox. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/65 + + (2) https://bitbucket.org/tarek/distribute/issue/67 + + (3) https://www.python.org/dev/peps/pep-0381/ + + (4) https://www.python.org/dev/peps/pep-0382/ + + +File: setuptools.info, Node: 0 6 4, Next: 0 6 3, Prev: 0 6 5, Up: History<2> + +9.439 0.6.4 +=========== + +10 Oct 2009 + + * Added the generation of ‘distribute_setup_3k.py’ during the + release. This closes Distribute #52(1). + + * Added an upload_docs command to easily upload project documentation + to PyPI’s ‘https://pythonhosted.org’. This close issue Distribute + #56(2). + + * Fixed a bootstrap bug on the use_setuptools() API. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/52 + + (2) https://bitbucket.org/tarek/distribute/issue/56 + + +File: setuptools.info, Node: 0 6 3, Next: 0 6 2, Prev: 0 6 4, Up: History<2> + +9.440 0.6.3 +=========== + +27 Sep 2009 + +* Menu: + +* setuptools:: +* bootstrapping:: + + +File: setuptools.info, Node: setuptools, Next: bootstrapping, Up: 0 6 3 + +9.440.1 setuptools +------------------ + + * Fixed a bunch of calls to file() that caused crashes on Python 3. + + +File: setuptools.info, Node: bootstrapping, Prev: setuptools, Up: 0 6 3 + +9.440.2 bootstrapping +--------------------- + + * Fixed a bug in sorting that caused bootstrap to fail on Python 3. + + +File: setuptools.info, Node: 0 6 2, Next: 0 6 1, Prev: 0 6 3, Up: History<2> + +9.441 0.6.2 +=========== + +26 Sep 2009 + +* Menu: + +* setuptools: setuptools<2>. +* bootstrapping: bootstrapping<2>. + + +File: setuptools.info, Node: setuptools<2>, Next: bootstrapping<2>, Up: 0 6 2 + +9.441.1 setuptools +------------------ + + * Added Python 3 support; see docs/python3.txt. This closes Old + Setuptools #39(1). + + * Added option to run 2to3 automatically when installing on Python 3. + This closes issue Distribute #31(2). + + * Fixed invalid usage of requirement.parse, that broke develop -d. + This closes Old Setuptools #44(3). + + * Fixed script launcher for 64-bit Windows. This closes Old + Setuptools #2(4). + + * KeyError when compiling extensions. This closes Old Setuptools + #41(5). + + ---------- Footnotes ---------- + + (1) http://bugs.python.org/setuptools/issue39 + + (2) https://bitbucket.org/tarek/distribute/issue/31 + + (3) http://bugs.python.org/setuptools/issue44 + + (4) http://bugs.python.org/setuptools/issue2 + + (5) http://bugs.python.org/setuptools/issue41 + + +File: setuptools.info, Node: bootstrapping<2>, Prev: setuptools<2>, Up: 0 6 2 + +9.441.2 bootstrapping +--------------------- + + * Fixed bootstrap not working on Windows. This closes issue + Distribute #49(1). + + * Fixed 2.6 dependencies. This closes issue Distribute #50(2). + + * Make sure setuptools is patched when running through easy_install + This closes Old Setuptools #40(3). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/49 + + (2) https://bitbucket.org/tarek/distribute/issue/50 + + (3) http://bugs.python.org/setuptools/issue40 + + +File: setuptools.info, Node: 0 6 1, Next: 0 6, Prev: 0 6 2, Up: History<2> + +9.442 0.6.1 +=========== + +08 Sep 2009 + +* Menu: + +* setuptools: setuptools<3>. +* bootstrapping: bootstrapping<3>. + + +File: setuptools.info, Node: setuptools<3>, Next: bootstrapping<3>, Up: 0 6 1 + +9.442.1 setuptools +------------------ + + * package_index.urlopen now catches BadStatusLine and malformed url + errors. This closes Distribute #16(1) and Distribute #18(2). + + * zip_ok is now False by default. This closes Old Setuptools #33(3). + + * Fixed invalid URL error catching. Old Setuptools #20(4). + + * Fixed invalid bootstraping with easy_install installation + (Distribute #40(5)). Thanks to Florian Schulze for the help. + + * Removed buildout/bootstrap.py. A new repository will create a + specific bootstrap.py script. + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/16 + + (2) https://bitbucket.org/tarek/distribute/issue/18 + + (3) http://bugs.python.org/setuptools/issue33 + + (4) http://bugs.python.org/setuptools/issue20 + + (5) https://bitbucket.org/tarek/distribute/issue/40 + + +File: setuptools.info, Node: bootstrapping<3>, Prev: setuptools<3>, Up: 0 6 1 + +9.442.2 bootstrapping +--------------------- + + * The boostrap process leave setuptools alone if detected in the + system and –root or –prefix is provided, but is not in the same + location. This closes Distribute #10(1). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/10 + + +File: setuptools.info, Node: 0 6, Next: 0 6c9, Prev: 0 6 1, Up: History<2> + +9.443 0.6 +========= + +09 Aug 2009 + +* Menu: + +* setuptools: setuptools<4>. +* pkg_resources:: +* easy_install:: + + +File: setuptools.info, Node: setuptools<4>, Next: pkg_resources, Up: 0 6 + +9.443.1 setuptools +------------------ + + * Packages required at build time where not fully present at install + time. This closes Distribute #12(1). + + * Protected against failures in tarfile extraction. This closes + Distribute #10(2). + + * Made Jython api_tests.txt doctest compatible. This closes + Distribute #7(3). + + * sandbox.py replaced builtin type file with builtin function open. + This closes Distribute #6(4). + + * Immediately close all file handles. This closes Distribute #3(5). + + * Added compatibility with Subversion 1.6. This references + Distribute #1(6). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/12 + + (2) https://bitbucket.org/tarek/distribute/issue/10 + + (3) https://bitbucket.org/tarek/distribute/issue/7 + + (4) https://bitbucket.org/tarek/distribute/issue/6 + + (5) https://bitbucket.org/tarek/distribute/issue/3 + + (6) https://bitbucket.org/tarek/distribute/issue/1 + + +File: setuptools.info, Node: pkg_resources, Next: easy_install, Prev: setuptools<4>, Up: 0 6 + +9.443.2 pkg_resources +--------------------- + + * Avoid a call to /usr/bin/sw_vers on OSX and use the official + platform API instead. Based on a patch from ronaldoussoren. This + closes issue #5(1). + + * Fixed a SandboxViolation for mkdir that could occur in certain + cases. This closes Distribute #13(2). + + * Allow to find_on_path on systems with tight permissions to fail + gracefully. This closes Distribute #9(3). + + * Corrected inconsistency between documentation and code of + add_entry. This closes Distribute #8(4). + + * Immediately close all file handles. This closes Distribute #3(5). + + ---------- Footnotes ---------- + + (1) https://github.com/pypa/setuptools/issues/5 + + (2) https://bitbucket.org/tarek/distribute/issue/13 + + (3) https://bitbucket.org/tarek/distribute/issue/9 + + (4) https://bitbucket.org/tarek/distribute/issue/8 + + (5) https://bitbucket.org/tarek/distribute/issue/3 + + +File: setuptools.info, Node: easy_install, Prev: pkg_resources, Up: 0 6 + +9.443.3 easy_install +-------------------- + + * Immediately close all file handles. This closes Distribute #3(1). + + ---------- Footnotes ---------- + + (1) https://bitbucket.org/tarek/distribute/issue/3 + + +File: setuptools.info, Node: 0 6c9, Next: 0 6c7, Prev: 0 6, Up: History<2> + +9.444 0.6c9 +=========== + + * Fixed a missing files problem when using Windows source + distributions on non-Windows platforms, due to distutils not + handling manifest file line endings correctly. + + * Updated Pyrex support to work with Pyrex 0.9.6 and higher. + + * Minor changes for Jython compatibility, including skipping + tests that can’t work on Jython. + + * Fixed not installing eggs in ‘install_requires’ if they were + also used for ‘setup_requires’ or ‘tests_require’. + + * Fixed not fetching eggs in ‘install_requires’ when running + tests. + + * Allow ‘ez_setup.use_setuptools()’ to upgrade existing + setuptools installations when called from a standalone + ‘setup.py’. + + * Added a warning if a namespace package is declared, but its + parent package is not also declared as a namespace. + + * Support Subversion 1.5 + + * Removed use of deprecated ‘md5’ module if ‘hashlib’ is + available + + * Fixed ‘bdist_wininst upload’ trying to upload the ‘.exe’ twice + + * Fixed ‘bdist_egg’ putting a ‘native_libs.txt’ in the source + package’s ‘.egg-info’, when it should only be in the built + egg’s ‘EGG-INFO’. + + * Ensure that _full_name is set on all shared libs before + extensions are checked for shared lib usage. (Fixes a bug in + the experimental shared library build support.) + + * Fix to allow unpacked eggs containing native libraries to fail + more gracefully under Google App Engine (with an ‘ImportError’ + loading the C-based module, instead of getting a ‘NameError’). + + * Fixed ‘win32.exe’ support for .pth files, so unnecessary + directory nesting is flattened out in the resulting egg. + (There was a case-sensitivity problem that affected some + distributions, notably ‘pywin32’.) + + * Prevent ‘--help-commands’ and other junk from showing under + Python 2.5 when running ‘easy_install --help’. + + * Fixed GUI scripts sometimes not executing on Windows + + * Fixed not picking up dependency links from recursive + dependencies. + + * Only make ‘.py’, ‘.dll’ and ‘.so’ files executable when + unpacking eggs + + * Changes for Jython compatibility + + * Improved error message when a requirement is also a directory + name, but the specified directory is not a source package. + + * Fixed ‘--allow-hosts’ option blocking ‘file:’ URLs + + * Fixed HTTP SVN detection failing when the page title included + a project name (e.g. on SourceForge-hosted SVN) + + * Fix Jython script installation to handle ‘#!’ lines better + when ‘sys.executable’ is a script. + + * Removed use of deprecated ‘md5’ module if ‘hashlib’ is + available + + * Keep site directories (e.g. ‘site-packages’) from being + included in ‘.pth’ files. + + +File: setuptools.info, Node: 0 6c7, Next: 0 6c6, Prev: 0 6c9, Up: History<2> + +9.445 0.6c7 +=========== + + * Fixed ‘distutils.filelist.findall()’ crashing on broken + symlinks, and ‘egg_info’ command failing on new, uncommitted + SVN directories. + + * Fix import problems with nested namespace packages installed + via ‘--root’ or ‘--single-version-externally-managed’, due to + the parent package not having the child package as an + attribute. + + * ‘ftp:’ download URLs now work correctly. + + * The default ‘--index-url’ is now + ‘https://pypi.python.org/simple’, to use the Python Package + Index’s new simpler (and faster!) REST API. + + +File: setuptools.info, Node: 0 6c6, Next: 0 6c5, Prev: 0 6c7, Up: History<2> + +9.446 0.6c6 +=========== + + * Added ‘--egg-path’ option to ‘develop’ command, allowing you + to force ‘.egg-link’ files to use relative paths (allowing + them to be shared across platforms on a networked drive). + + * Fix not building binary RPMs correctly. + + * Fix “eggsecutables” (such as setuptools’ own egg) only being + runnable with bash-compatible shells. + + * Fix ‘#!’ parsing problems in Windows ‘.exe’ script wrappers, + when there was whitespace inside a quoted argument or at the + end of the ‘#!’ line (a regression introduced in 0.6c4). + + * Fix ‘test’ command possibly failing if an older version of the + project being tested was installed on ‘sys.path’ ahead of the + test source directory. + + * Fix ‘find_packages()’ treating ‘ez_setup’ and directories with + ‘.’ in their names as packages. + + * EasyInstall no longer aborts the installation process if a URL + it wants to retrieve can’t be downloaded, unless the URL is an + actual package download. Instead, it issues a warning and + tries to keep going. + + * Fixed distutils-style scripts originally built on Windows + having their line endings doubled when installed on any + platform. + + * Added ‘--local-snapshots-ok’ flag, to allow building eggs from + projects installed using ‘setup.py develop’. + + * Fixed not HTML-decoding URLs scraped from web pages + + +File: setuptools.info, Node: 0 6c5, Next: 0 6c4, Prev: 0 6c6, Up: History<2> + +9.447 0.6c5 +=========== + + * Fix uploaded ‘bdist_rpm’ packages being described as + ‘bdist_egg’ packages under Python versions less than 2.5. + + * Fix uploaded ‘bdist_wininst’ packages being described as + suitable for “any” version by Python 2.5, even if a + ‘--target-version’ was specified. + + * Fixed ‘.dll’ files on Cygwin not having executable permissions + when an egg is installed unzipped. + + +File: setuptools.info, Node: 0 6c4, Next: 0 6c3, Prev: 0 6c5, Up: History<2> + +9.448 0.6c4 +=========== + + * Overhauled Windows script wrapping to support ‘bdist_wininst’ + better. Scripts installed with ‘bdist_wininst’ will always + use ‘#!python.exe’ or ‘#!pythonw.exe’ as the executable name + (even when built on non-Windows platforms!), and the wrappers + will look for the executable in the script’s parent directory + (which should find the right version of Python). + + * Fix ‘upload’ command not uploading files built by ‘bdist_rpm’ + or ‘bdist_wininst’ under Python 2.3 and 2.4. + + * Add support for “eggsecutable” headers: a ‘#!/bin/sh’ script + that is prepended to an ‘.egg’ file to allow it to be run as a + script on Unix-ish platforms. (This is mainly so that + setuptools itself can have a single-file installer on Unix, + without doing multiple downloads, dealing with firewalls, + etc.) + + * Fix problem with empty revision numbers in Subversion 1.4 + ‘entries’ files + + * Use cross-platform relative paths in ‘easy-install.pth’ when + doing ‘develop’ and the source directory is a subdirectory of + the installation target directory. + + * Fix a problem installing eggs with a system packaging tool if + the project contained an implicit namespace package; for + example if the ‘setup()’ listed a namespace package ‘foo.bar’ + without explicitly listing ‘foo’ as a namespace package. + + * Added support for HTTP “Basic” authentication using + ‘http://user:pass@host’ URLs. If a password-protected page + contains links to the same host (and protocol), those links + will inherit the credentials used to access the original page. + + * Removed all special support for Sourceforge mirrors, as + Sourceforge’s mirror system now works well for non-browser + downloads. + + * Fixed not recognizing ‘win32.exe’ installers that included a + custom bitmap. + + * Fixed not allowing ‘os.open()’ of paths outside the sandbox, + even if they are opened read-only (e.g. reading + ‘/dev/urandom’ for random numbers, as is done by + ‘os.urandom()’ on some platforms). + + * Fixed a problem with ‘.pth’ testing on Windows when + ‘sys.executable’ has a space in it (e.g., the user installed + Python to a ‘Program Files’ directory). + + +File: setuptools.info, Node: 0 6c3, Next: 0 6c2, Prev: 0 6c4, Up: History<2> + +9.449 0.6c3 +=========== + + * Fixed breakages caused by Subversion 1.4’s new “working copy” + format + + * You can once again use “python -m easy_install” with Python + 2.4 and above. + + * Python 2.5 compatibility fixes added. + + +File: setuptools.info, Node: 0 6c2, Next: 0 6c1, Prev: 0 6c3, Up: History<2> + +9.450 0.6c2 +=========== + + * The ‘ez_setup’ module displays the conflicting version of + setuptools (and its installation location) when a script + requests a version that’s not available. + + * Running ‘setup.py develop’ on a setuptools-using project will + now install setuptools if needed, instead of only downloading + the egg. + + * Windows script wrappers now support quoted arguments and + arguments containing spaces. (Patch contributed by Jim + Fulton.) + + * The ‘ez_setup.py’ script now actually works when you put a + setuptools ‘.egg’ alongside it for bootstrapping an offline + machine. + + * A writable installation directory on ‘sys.path’ is no longer + required to download and extract a source distribution using + ‘--editable’. + + * Generated scripts now use ‘-x’ on the ‘#!’ line when + ‘sys.executable’ contains non-ASCII characters, to prevent + deprecation warnings about an unspecified encoding when the + script is run. + + +File: setuptools.info, Node: 0 6c1, Next: 0 6b4, Prev: 0 6c2, Up: History<2> + +9.451 0.6c1 +=========== + + * Fixed ‘AttributeError’ when trying to download a + ‘setup_requires’ dependency when a distribution lacks a + ‘dependency_links’ setting. + + * Made ‘zip-safe’ and ‘not-zip-safe’ flag files contain a single + byte, so as to play better with packaging tools that complain + about zero-length files. + + * Made ‘setup.py develop’ respect the ‘--no-deps’ option, which + it previously was ignoring. + + * Support ‘extra_path’ option to ‘setup()’ when ‘install’ is run + in backward-compatibility mode. + + * Source distributions now always include a ‘setup.cfg’ file + that explicitly sets ‘egg_info’ options such that they produce + an identical version number to the source distribution’s + version number. (Previously, the default version number could + be different due to the use of ‘--tag-date’, or if the version + was overridden on the command line that built the source + distribution.) + + * EasyInstall now includes setuptools version information in the + ‘User-Agent’ string sent to websites it visits. + + +File: setuptools.info, Node: 0 6b4, Next: 0 6b3, Prev: 0 6c1, Up: History<2> + +9.452 0.6b4 +=========== + + * Fix ‘register’ not obeying name/version set by ‘egg_info’ + command, if ‘egg_info’ wasn’t explicitly run first on the same + command line. + + * Added ‘--no-date’ and ‘--no-svn-revision’ options to + ‘egg_info’ command, to allow suppressing tags configured in + ‘setup.cfg’. + + * Fixed redundant warnings about missing ‘README’ file(s); it + should now appear only if you are actually a source + distribution. + + * Fix creating Python wrappers for non-Python scripts + + * Fix ‘ftp://’ directory listing URLs from causing a crash when + used in the “Home page” or “Download URL” slots on PyPI. + + * Fix ‘sys.path_importer_cache’ not being updated when an + existing zipfile or directory is deleted/overwritten. + + * Fix not recognizing HTML 404 pages from package indexes. + + * Allow ‘file://’ URLs to be used as a package index. URLs that + refer to directories will use an internally-generated + directory listing if there is no ‘index.html’ file in the + directory. + + * Allow external links in a package index to be specified using + ‘rel="homepage"’ or ‘rel="download"’, without needing the old + PyPI-specific visible markup. + + * Suppressed warning message about possibly-misspelled project + name, if an egg or link for that project name has already been + seen. + + +File: setuptools.info, Node: 0 6b3, Next: 0 6b2, Prev: 0 6b4, Up: History<2> + +9.453 0.6b3 +=========== + + * Fix ‘bdist_egg’ not including files in subdirectories of + ‘.egg-info’. + + * Allow ‘.py’ files found by the ‘include_package_data’ option + to be automatically included. Remove duplicate data file + matches if both ‘include_package_data’ and ‘package_data’ are + used to refer to the same files. + + * Fix local ‘--find-links’ eggs not being copied except with + ‘--always-copy’. + + * Fix sometimes not detecting local packages installed outside + of “site” directories. + + * Fix mysterious errors during initial ‘setuptools’ install, + caused by ‘ez_setup’ trying to run ‘easy_install’ twice, due + to a code fallthru after deleting the egg from which it’s + running. + + +File: setuptools.info, Node: 0 6b2, Next: 0 6b1, Prev: 0 6b3, Up: History<2> + +9.454 0.6b2 +=========== + + * Don’t install or update a ‘site.py’ patch when installing to a + ‘PYTHONPATH’ directory with ‘--multi-version’, unless an + ‘easy-install.pth’ file is already in use there. + + * Construct ‘.pth’ file paths in such a way that installing an + egg whose name begins with ‘import’ doesn’t cause a syntax + error. + + * Fixed a bogus warning message that wasn’t updated since the + 0.5 versions. + + +File: setuptools.info, Node: 0 6b1, Next: 0 6a11, Prev: 0 6b2, Up: History<2> + +9.455 0.6b1 +=========== + + * Strip ‘module’ from the end of compiled extension modules when + computing the name of a ‘.py’ loader/wrapper. (Python’s + import machinery ignores this suffix when searching for an + extension module.) + + * Better ambiguity management: accept ‘#egg’ name/version even + if processing what appears to be a correctly-named distutils + file, and ignore ‘.egg’ files with no ‘-’, since valid Python + ‘.egg’ files always have a version number (but Scheme eggs + often don’t). + + * Support ‘file://’ links to directories in ‘--find-links’, so + that easy_install can build packages from local source + checkouts. + + * Added automatic retry for Sourceforge mirrors. The new + download process is to first just try dl.sourceforge.net, then + randomly select mirror IPs and remove ones that fail, until + something works. The removed IPs stay removed for the + remainder of the run. + + * Ignore bdist_dumb distributions when looking at download URLs. + + +File: setuptools.info, Node: 0 6a11, Next: 0 6a10, Prev: 0 6b1, Up: History<2> + +9.456 0.6a11 +============ + + * Added ‘test_loader’ keyword to support custom test loaders + + * Added ‘setuptools.file_finders’ entry point group to allow + implementing revision control plugins. + + * Added ‘--identity’ option to ‘upload’ command. + + * Added ‘dependency_links’ to allow specifying URLs for + ‘--find-links’. + + * Enhanced test loader to scan packages as well as modules, and + call ‘additional_tests()’ if present to get non-unittest + tests. + + * Support namespace packages in conjunction with system + packagers, by omitting the installation of any ‘__init__.py’ + files for namespace packages, and adding a special ‘.pth’ file + to create a working package in ‘sys.modules’. + + * Made ‘--single-version-externally-managed’ automatic when + ‘--root’ is used, so that most system packagers won’t require + special support for setuptools. + + * Fixed ‘setup_requires’, ‘tests_require’, etc. not using + ‘setup.cfg’ or other configuration files for their option + defaults when installing, and also made the install use + ‘--multi-version’ mode so that the project directory doesn’t + need to support .pth files. + + * ‘MANIFEST.in’ is now forcibly closed when any errors occur + while reading it. Previously, the file could be left open and + the actual error would be masked by problems trying to remove + the open file on Windows systems. + + * Process ‘dependency_links.txt’ if found in a distribution, by + adding the URLs to the list for scanning. + + * Use relative paths in ‘.pth’ files when eggs are being + installed to the same directory as the ‘.pth’ file. This + maximizes portability of the target directory when building + applications that contain eggs. + + * Added ‘easy_install-N.N’ script(s) for convenience when using + multiple Python versions. + + * Added automatic handling of installation conflicts. Eggs are + now shifted to the front of sys.path, in an order consistent + with where they came from, making EasyInstall seamlessly + co-operate with system package managers. + + The ‘--delete-conflicting’ and ‘--ignore-conflicts-at-my-risk’ + options are now no longer necessary, and will generate + warnings at the end of a run if you use them. + + * Don’t recursively traverse subdirectories given to + ‘--find-links’. + + +File: setuptools.info, Node: 0 6a10, Next: 0 6a9, Prev: 0 6a11, Up: History<2> + +9.457 0.6a10 +============ + + * Fixed the ‘develop’ command ignoring ‘--find-links’. + + * Added exhaustive testing of the install directory, including a + spawn test for ‘.pth’ file support, and directory + writability/existence checks. This should virtually eliminate + the need to set or configure ‘--site-dirs’. + + * Added ‘--prefix’ option for more do-what-I-mean-ishness in the + absence of RTFM-ing. :) + + * Enhanced ‘PYTHONPATH’ support so that you don’t have to put + any eggs on it manually to make it work. ‘--multi-version’ is + no longer a silent default; you must explicitly use it if + installing to a non-PYTHONPATH, non-“site” directory. + + * Expand ‘$variables’ used in the ‘--site-dirs’, + ‘--build-directory’, ‘--install-dir’, and ‘--script-dir’ + options, whether on the command line or in configuration + files. + + * Improved SourceForge mirror processing to work faster and be + less affected by transient HTML changes made by SourceForge. + + * PyPI searches now use the exact spelling of requirements + specified on the command line or in a project’s + ‘install_requires’. Previously, a normalized form of the name + was used, which could lead to unnecessary full-index searches + when a project’s name had an underscore (‘_’) in it. + + * EasyInstall can now download bare ‘.py’ files and wrap them in + an egg, as long as you include an ‘#egg=name-version’ suffix + on the URL, or if the ‘.py’ file is listed as the “Download + URL” on the project’s PyPI page. This allows third parties to + “package” trivial Python modules just by linking to them (e.g. + from within their own PyPI page or download links page). + + * The ‘--always-copy’ option now skips “system” and + “development” eggs since they can’t be reliably copied. Note + that this may cause EasyInstall to choose an older version of + a package than what you expected, or it may cause downloading + and installation of a fresh version of what’s already + installed. + + * The ‘--find-links’ option previously scanned all supplied URLs + and directories as early as possible, but now only directories + and direct archive links are scanned immediately. URLs are + not retrieved unless a package search was already going to go + online due to a package not being available locally, or due to + the use of the ‘--update’ or ‘-U’ option. + + * Fixed the annoying ‘--help-commands’ wart. + + +File: setuptools.info, Node: 0 6a9, Next: 0 6a8, Prev: 0 6a10, Up: History<2> + +9.458 0.6a9 +=========== + + * The ‘sdist’ command no longer uses the traditional ‘MANIFEST’ + file to create source distributions. ‘MANIFEST.in’ is still + read and processed, as are the standard defaults and pruning. + But the manifest is built inside the project’s ‘.egg-info’ + directory as ‘SOURCES.txt’, and it is rebuilt every time the + ‘egg_info’ command is run. + + * Added the ‘include_package_data’ keyword to ‘setup()’, + allowing you to automatically include any package data listed + in revision control or ‘MANIFEST.in’ + + * Added the ‘exclude_package_data’ keyword to ‘setup()’, + allowing you to trim back files included via the + ‘package_data’ and ‘include_package_data’ options. + + * Fixed ‘--tag-svn-revision’ not working when run from a source + distribution. + + * Added warning for namespace packages with missing + ‘declare_namespace()’ + + * Added ‘tests_require’ keyword to ‘setup()’, so that e.g. + packages requiring ‘nose’ to run unit tests can make this + dependency optional unless the ‘test’ command is run. + + * Made all commands that use ‘easy_install’ respect its + configuration options, as this was causing some problems with + ‘setup.py install’. + + * Added an ‘unpack_directory()’ driver to + ‘setuptools.archive_util’, so that you can process a directory + tree through a processing filter as if it were a zipfile or + tarfile. + + * Added an internal ‘install_egg_info’ command to use as part of + old-style ‘install’ operations, that installs an ‘.egg-info’ + directory with the package. + + * Added a ‘--single-version-externally-managed’ option to the + ‘install’ command so that you can more easily wrap a “flat” + egg in a system package. + + * Enhanced ‘bdist_rpm’ so that it installs single-version eggs + that don’t rely on a ‘.pth’ file. The ‘--no-egg’ option has + been removed, since all RPMs are now built in a more + backwards-compatible format. + + * Support full roundtrip translation of eggs to and from + ‘bdist_wininst’ format. Running ‘bdist_wininst’ on a + setuptools-based package wraps the egg in an .exe that will + safely install it as an egg (i.e., with metadata and + entry-point wrapper scripts), and ‘easy_install’ can turn the + .exe back into an ‘.egg’ file or directory and install it as + such. + + * Fixed ‘.pth’ file processing picking up nested eggs (i.e. + ones inside “baskets”) when they weren’t explicitly listed in + the ‘.pth’ file. + + * If more than one URL appears to describe the exact same + distribution, prefer the shortest one. This helps to avoid + “table of contents” CGI URLs like the ones on effbot.org. + + * Quote arguments to python.exe (including python’s path) to + avoid problems when Python (or a script) is installed in a + directory whose name contains spaces on Windows. + + * Support full roundtrip translation of eggs to and from + ‘bdist_wininst’ format. Running ‘bdist_wininst’ on a + setuptools-based package wraps the egg in an .exe that will + safely install it as an egg (i.e., with metadata and + entry-point wrapper scripts), and ‘easy_install’ can turn the + .exe back into an ‘.egg’ file or directory and install it as + such. + + +File: setuptools.info, Node: 0 6a8, Next: 0 6a7, Prev: 0 6a9, Up: History<2> + +9.459 0.6a8 +=========== + + * Fixed some problems building extensions when Pyrex was + installed, especially with Python 2.4 and/or packages using + SWIG. + + * Made ‘develop’ command accept all the same options as + ‘easy_install’, and use the ‘easy_install’ command’s + configuration settings as defaults. + + * Made ‘egg_info --tag-svn-revision’ fall back to extracting the + revision number from ‘PKG-INFO’ in case it is being run on a + source distribution of a snapshot taken from a + Subversion-based project. + + * Automatically detect ‘.dll’, ‘.so’ and ‘.dylib’ files that are + being installed as data, adding them to ‘native_libs.txt’ + automatically. + + * Fixed some problems with fresh checkouts of projects that + don’t include ‘.egg-info/PKG-INFO’ under revision control and + put the project’s source code directly in the project + directory. If such a package had any requirements that get + processed before the ‘egg_info’ command can be run, the setup + scripts would fail with a “Missing ‘Version:’ header and/or + PKG-INFO file” error, because the egg runtime interpreted the + unbuilt metadata in a directory on ‘sys.path’ (i.e. the + current directory) as being a corrupted egg. Setuptools now + monkeypatches the distribution metadata cache to pretend that + the egg has valid version information, until it has a chance + to make it actually be so (via the ‘egg_info’ command). + + * Update for changed SourceForge mirror format + + * Fixed not installing dependencies for some packages fetched + via Subversion + + * Fixed dependency installation with ‘--always-copy’ not using + the same dependency resolution procedure as other operations. + + * Fixed not fully removing temporary directories on Windows, if + a Subversion checkout left read-only files behind + + * Fixed some problems building extensions when Pyrex was + installed, especially with Python 2.4 and/or packages using + SWIG. + + +File: setuptools.info, Node: 0 6a7, Next: 0 6a6, Prev: 0 6a8, Up: History<2> + +9.460 0.6a7 +=========== + + * Fixed not being able to install Windows script wrappers using + Python 2.3 + + +File: setuptools.info, Node: 0 6a6, Next: 0 6a5, Prev: 0 6a7, Up: History<2> + +9.461 0.6a6 +=========== + + * Added support for “traditional” PYTHONPATH-based non-root + installation, and also the convenient ‘virtual-python.py’ + script, based on a contribution by Ian Bicking. The + setuptools egg now contains a hacked ‘site’ module that makes + the PYTHONPATH-based approach work with .pth files, so that + you can get the full EasyInstall feature set on such + installations. + + * Added ‘--no-deps’ and ‘--allow-hosts’ options. + + * Improved Windows ‘.exe’ script wrappers so that the script can + have the same name as a module without confusing Python. + + * Changed dependency processing so that it’s breadth-first, + allowing a depender’s preferences to override those of a + dependee, to prevent conflicts when a lower version is + acceptable to the dependee, but not the depender. Also, + ensure that currently installed/selected packages aren’t given + precedence over ones desired by a package being installed, + which could cause conflict errors. + + +File: setuptools.info, Node: 0 6a5, Next: 0 6a3, Prev: 0 6a6, Up: History<2> + +9.462 0.6a5 +=========== + + * Fixed missing gui/cli .exe files in distribution. Fixed bugs + in tests. + + +File: setuptools.info, Node: 0 6a3, Next: 0 6a2, Prev: 0 6a5, Up: History<2> + +9.463 0.6a3 +=========== + + * Added ‘gui_scripts’ entry point group to allow installing GUI + scripts on Windows and other platforms. (The special handling + is only for Windows; other platforms are treated the same as + for ‘console_scripts’.) + + * Improved error message when trying to use old ways of running + ‘easy_install’. Removed the ability to run via ‘python -m’ or + by running ‘easy_install.py’; ‘easy_install’ is the command to + run on all supported platforms. + + * Improved wrapper script generation and runtime initialization + so that a VersionConflict doesn’t occur if you later install a + competing version of a needed package as the default version + of that package. + + * Fixed a problem parsing version numbers in ‘#egg=’ links. + + +File: setuptools.info, Node: 0 6a2, Next: 0 6a1, Prev: 0 6a3, Up: History<2> + +9.464 0.6a2 +=========== + + * Added ‘console_scripts’ entry point group to allow installing + scripts without the need to create separate script files. On + Windows, console scripts get an ‘.exe’ wrapper so you can just + type their name. On other platforms, the scripts are written + without a file extension. + + * EasyInstall can now install “console_scripts” defined by + packages that use ‘setuptools’ and define appropriate entry + points. On Windows, console scripts get an ‘.exe’ wrapper so + you can just type their name. On other platforms, the scripts + are installed without a file extension. + + * Using ‘python -m easy_install’ or running ‘easy_install.py’ is + now DEPRECATED, since an ‘easy_install’ wrapper is now + available on all platforms. + + +File: setuptools.info, Node: 0 6a1, Next: 0 5a12, Prev: 0 6a2, Up: History<2> + +9.465 0.6a1 +=========== + + * Added support for building “old-style” RPMs that don’t install + an egg for the target package, using a ‘--no-egg’ option. + + * The ‘build_ext’ command now works better when using the + ‘--inplace’ option and multiple Python versions. It now makes + sure that all extensions match the current Python version, + even if newer copies were built for a different Python + version. + + * The ‘upload’ command no longer attaches an extra ‘.zip’ when + uploading eggs, as PyPI now supports egg uploads without + trickery. + + * The ‘ez_setup’ script/module now displays a warning before + downloading the setuptools egg, and attempts to check the + downloaded egg against an internal MD5 checksum table. + + * Fixed the ‘--tag-svn-revision’ option of ‘egg_info’ not + finding the latest revision number; it was using the revision + number of the directory containing ‘setup.py’, not the highest + revision number in the project. + + * Added ‘eager_resources’ setup argument + + * The ‘sdist’ command now recognizes Subversion “deleted file” + entries and does not include them in source distributions. + + * ‘setuptools’ now embeds itself more thoroughly into the + distutils, so that other distutils extensions (e.g. py2exe, + py2app) will subclass setuptools’ versions of things, rather + than the native distutils ones. + + * Added ‘entry_points’ and ‘setup_requires’ arguments to + ‘setup()’; ‘setup_requires’ allows you to automatically find + and download packages that are needed in order to `build' your + project (as opposed to running it). + + * ‘setuptools’ now finds its commands, ‘setup()’ argument + validators, and metadata writers using entry points, so that + they can be extended by third-party packages. See Creating + distutils Extensions(1) for more details. + + * The vestigial ‘depends’ command has been removed. It was + never finished or documented, and never would have worked + without EasyInstall - which it pre-dated and was never + compatible with. + + * EasyInstall now does MD5 validation of downloads from PyPI, or + from any link that has an “#md5=…” trailer with a 32-digit + lowercase hex md5 digest. + + * EasyInstall now handles symlinks in target directories by + removing the link, rather than attempting to overwrite the + link’s destination. This makes it easier to set up an + alternate Python “home” directory (as described in the + Non-Root Installation section of the docs). + + * Added support for handling MacOS platform information in + ‘.egg’ filenames, based on a contribution by Kevin Dangoor. + You may wish to delete and reinstall any eggs whose filename + includes “darwin” and “Power_Macintosh”, because the format + for this platform information has changed so that minor OS X + upgrades (such as 10.4.1 to 10.4.2) do not cause eggs built + with a previous OS version to become obsolete. + + * easy_install’s dependency processing algorithms have changed. + When using ‘--always-copy’, it now ensures that dependencies + are copied too. When not using ‘--always-copy’, it tries to + use a single resolution loop, rather than recursing. + + * Fixed installing extra ‘.pyc’ or ‘.pyo’ files for scripts with + ‘.py’ extensions. + + * Added ‘--site-dirs’ option to allow adding custom “site” + directories. Made ‘easy-install.pth’ work in + platform-specific alternate site directories (e.g. + ‘~/Library/Python/2.x/site-packages’ on Mac OS X). + + * If you manually delete the current version of a package, the + next run of EasyInstall against the target directory will now + remove the stray entry from the ‘easy-install.pth’ file. + + * EasyInstall now recognizes URLs with a ‘#egg=project_name’ + fragment ID as pointing to the named project’s source + checkout. Such URLs have a lower match precedence than any + other kind of distribution, so they’ll only be used if they + have a higher version number than any other available + distribution, or if you use the ‘--editable’ option. The + ‘#egg’ fragment can contain a version if it’s formatted as + ‘#egg=proj-ver’, where ‘proj’ is the project name, and ‘ver’ + is the version number. You `must' use the format for these + values that the ‘bdist_egg’ command uses; i.e., all + non-alphanumeric runs must be condensed to single underscore + characters. + + * Added the ‘--editable’ option; see Editing and Viewing Source + Packages in the docs. Also, slightly changed the behavior of + the ‘--build-directory’ option. + + * Fixed the setup script sandbox facility not recognizing + certain paths as valid on case-insensitive platforms. + + ---------- Footnotes ---------- + + (1) +https://setuptools.readthedocs.io/en/latest/setuptools.html#creating-distutils-extensions + + +File: setuptools.info, Node: 0 5a12, Next: 0 5a11, Prev: 0 6a1, Up: History<2> + +9.466 0.5a12 +============ + + * The zip-safety scanner now checks for modules that might be + used with ‘python -m’, and marks them as unsafe for zipping, + since Python 2.4 can’t handle ‘-m’ on zipped modules. + + * Fix ‘python -m easy_install’ not working due to setuptools + being installed as a zipfile. Update safety scanner to check + for modules that might be used as ‘python -m’ scripts. + + * Misc. fixes for win32.exe support, including changes to + support Python 2.4’s changed ‘bdist_wininst’ format. + + +File: setuptools.info, Node: 0 5a11, Next: 0 5a10, Prev: 0 5a12, Up: History<2> + +9.467 0.5a11 +============ + + * Fix breakage of the “develop” command that was caused by the + addition of ‘--always-unzip’ to the ‘easy_install’ command. + + +File: setuptools.info, Node: 0 5a10, Next: 0 5a9, Prev: 0 5a11, Up: History<2> + +9.468 0.5a10 +============ + + * Put the ‘easy_install’ module back in as a module, as it’s + needed for ‘python -m’ to run it! + + * Allow ‘--find-links/-f’ to accept local directories or + filenames as well as URLs. + + +File: setuptools.info, Node: 0 5a9, Next: 0 5a8, Prev: 0 5a10, Up: History<2> + +9.469 0.5a9 +=========== + + * Include ‘svn:externals’ directories in source distributions as + well as normal subversion-controlled files and directories. + + * Added ‘exclude=patternlist’ option to + ‘setuptools.find_packages()’ + + * Changed –tag-svn-revision to include an “r” in front of the + revision number for better readability. + + * Added ability to build eggs without including source files + (except for any scripts, of course), using the + ‘--exclude-source-files’ option to ‘bdist_egg’. + + * ‘setup.py install’ now automatically detects when an + “unmanaged” package or module is going to be on ‘sys.path’ + ahead of a package being installed, thereby preventing the + newer version from being imported. If this occurs, a warning + message is output to ‘sys.stderr’, but installation proceeds + anyway. The warning message informs the user what files or + directories need deleting, and advises them they can also use + EasyInstall (with the ‘--delete-conflicting’ option) to do it + automatically. + + * The ‘egg_info’ command now adds a ‘top_level.txt’ file to the + metadata directory that lists all top-level modules and + packages in the distribution. This is used by the + ‘easy_install’ command to find possibly-conflicting + “unmanaged” packages when installing the distribution. + + * Added ‘zip_safe’ and ‘namespace_packages’ arguments to + ‘setup()’. Added package analysis to determine zip-safety if + the ‘zip_safe’ flag is not given, and advise the author + regarding what code might need changing. + + * Fixed the swapped ‘-d’ and ‘-b’ options of ‘bdist_egg’. + + * EasyInstall now automatically detects when an “unmanaged” + package or module is going to be on ‘sys.path’ ahead of a + package you’re installing, thereby preventing the newer + version from being imported. By default, it will abort + installation to alert you of the problem, but there are also + new options (‘--delete-conflicting’ and + ‘--ignore-conflicts-at-my-risk’) available to change the + default behavior. (Note: this new feature doesn’t take effect + for egg files that were built with older ‘setuptools’ + versions, because they lack the new metadata file required to + implement it.) + + * The ‘easy_install’ distutils command now uses ‘DistutilsError’ + as its base error type for errors that should just issue a + message to stderr and exit the program without a traceback. + + * EasyInstall can now be given a path to a directory containing + a setup script, and it will attempt to build and install the + package there. + + * EasyInstall now performs a safety analysis on module contents + to determine whether a package is likely to run in zipped + form, and displays information about what modules may be doing + introspection that would break when running as a zipfile. + + * Added the ‘--always-unzip/-Z’ option, to force unzipping of + packages that would ordinarily be considered safe to unzip, + and changed the meaning of ‘--zip-ok/-z’ to “always leave + everything zipped”. + + +File: setuptools.info, Node: 0 5a8, Next: 0 5a7, Prev: 0 5a9, Up: History<2> + +9.470 0.5a8 +=========== + + * The “egg_info” command now always sets the distribution + metadata to “safe” forms of the distribution name and version, + so that distribution files will be generated with parseable + names (i.e., ones that don’t include ‘-‘ in the name or + version). Also, this means that if you use the various + ‘--tag’ options of “egg_info”, any distributions generated + will use the tags in the version, not just egg distributions. + + * Added support for defining command aliases in distutils + configuration files, under the “[aliases]” section. To + prevent recursion and to allow aliases to call the command of + the same name, a given alias can be expanded only once per + command-line invocation. You can define new aliases with the + “alias” command, either for the local, global, or per-user + configuration. + + * Added “rotate” command to delete old distribution files, given + a set of patterns to match and the number of files to keep. + (Keeps the most recently-modified distribution files matching + each pattern.) + + * Added “saveopts” command that saves all command-line options + for the current invocation to the local, global, or per-user + configuration file. Useful for setting defaults without + having to hand-edit a configuration file. + + * Added a “setopt” command that sets a single option in a + specified distutils configuration file. + + * There is now a separate documentation page for setuptools; + revision history that’s not specific to EasyInstall has been + moved to that page. + + +File: setuptools.info, Node: 0 5a7, Next: 0 5a6, Prev: 0 5a8, Up: History<2> + +9.471 0.5a7 +=========== + + * Added “upload” support for egg and source distributions, + including a bug fix for “upload” and a temporary workaround + for lack of .egg support in PyPI. + + +File: setuptools.info, Node: 0 5a6, Next: 0 5a5, Prev: 0 5a7, Up: History<2> + +9.472 0.5a6 +=========== + + * Beefed up the “sdist” command so that if you don’t have a + MANIFEST.in, it will include all files under revision control + (CVS or Subversion) in the current directory, and it will + regenerate the list every time you create a source + distribution, not just when you tell it to. This should make + the default “do what you mean” more often than the distutils’ + default behavior did, while still retaining the old behavior + in the presence of MANIFEST.in. + + * Fixed the “develop” command always updating .pth files, even + if you specified ‘-n’ or ‘--dry-run’. + + * Slightly changed the format of the generated version when you + use ‘--tag-build’ on the “egg_info” command, so that you can + make tagged revisions compare `lower' than the version + specified in setup.py (e.g. by using ‘--tag-build=dev’). + + +File: setuptools.info, Node: 0 5a5, Next: 0 5a4, Prev: 0 5a6, Up: History<2> + +9.473 0.5a5 +=========== + + * Added ‘develop’ command to ‘setuptools’-based packages. This + command installs an ‘.egg-link’ pointing to the package’s + source directory, and script wrappers that ‘execfile()’ the + source versions of the package’s scripts. This lets you put + your development checkout(s) on sys.path without having to + actually install them. (To uninstall the link, use use + ‘setup.py develop --uninstall’.) + + * Added ‘egg_info’ command to ‘setuptools’-based packages. This + command just creates or updates the “projectname.egg-info” + directory, without building an egg. (It’s used by the + ‘bdist_egg’, ‘test’, and ‘develop’ commands.) + + * Enhanced the ‘test’ command so that it doesn’t install the + package, but instead builds any C extensions in-place, updates + the ‘.egg-info’ metadata, adds the source directory to + ‘sys.path’, and runs the tests directly on the source. This + avoids an “unmanaged” installation of the package to + ‘site-packages’ or elsewhere. + + * Made ‘easy_install’ a standard ‘setuptools’ command, moving it + from the ‘easy_install’ module to + ‘setuptools.command.easy_install’. Note that if you were + importing or extending it, you must now change your imports + accordingly. ‘easy_install.py’ is still installed as a + script, but not as a module. + + +File: setuptools.info, Node: 0 5a4, Next: 0 5a3, Prev: 0 5a5, Up: History<2> + +9.474 0.5a4 +=========== + + * Setup scripts using setuptools can now list their dependencies + directly in the setup.py file, without having to manually + create a ‘depends.txt’ file. The ‘install_requires’ and + ‘extras_require’ arguments to ‘setup()’ are used to create a + dependencies file automatically. If you are manually creating + ‘depends.txt’ right now, please switch to using these setup + arguments as soon as practical, because ‘depends.txt’ support + will be removed in the 0.6 release cycle. For documentation + on the new arguments, see the ‘setuptools.dist.Distribution’ + class. + + * Setup scripts using setuptools now always install using + ‘easy_install’ internally, for ease of uninstallation and + upgrading. + + * Added ‘--always-copy/-a’ option to always copy needed packages + to the installation directory, even if they’re already present + elsewhere on sys.path. (In previous versions, this was the + default behavior, but now you must request it.) + + * Added ‘--upgrade/-U’ option to force checking PyPI for latest + available version(s) of all packages requested by name and + version, even if a matching version is available locally. + + * Added automatic installation of dependencies declared by a + distribution being installed. These dependencies must be + listed in the distribution’s ‘EGG-INFO’ directory, so the + distribution has to have declared its dependencies by using + setuptools. If a package has requirements it didn’t declare, + you’ll still have to deal with them yourself. (E.g., by + asking EasyInstall to find and install them.) + + * Added the ‘--record’ option to ‘easy_install’ for the benefit + of tools that run ‘setup.py install --record=filename’ on + behalf of another packaging system.) + + +File: setuptools.info, Node: 0 5a3, Next: 0 5a2, Prev: 0 5a4, Up: History<2> + +9.475 0.5a3 +=========== + + * Fixed not setting script permissions to allow execution. + + * Improved sandboxing so that setup scripts that want a + temporary directory (e.g. pychecker) can still run in the + sandbox. + + +File: setuptools.info, Node: 0 5a2, Next: 0 5a1, Prev: 0 5a3, Up: History<2> + +9.476 0.5a2 +=========== + + * Fix stupid stupid refactoring-at-the-last-minute typos. :( + + +File: setuptools.info, Node: 0 5a1, Next: 0 4a4, Prev: 0 5a2, Up: History<2> + +9.477 0.5a1 +=========== + + * Added support for “self-installation” bootstrapping. Packages + can now include ‘ez_setup.py’ in their source distribution, + and add the following to their ‘setup.py’, in order to + automatically bootstrap installation of setuptools as part of + their setup process: + + from ez_setup import use_setuptools + use_setuptools() + + from setuptools import setup + # etc... + + * Added support for converting ‘.win32.exe’ installers to eggs + on the fly. EasyInstall will now recognize such files by name + and install them. + + * Fixed a problem with picking the “best” version to install + (versions were being sorted as strings, rather than as parsed + values) + + +File: setuptools.info, Node: 0 4a4, Next: 0 4a3, Prev: 0 5a1, Up: History<2> + +9.478 0.4a4 +=========== + + * Added support for the distutils “verbose/quiet” and “dry-run” + options, as well as the “optimize” flag. + + * Support downloading packages that were uploaded to PyPI (by + scanning all links on package pages, not just the + homepage/download links). + + +File: setuptools.info, Node: 0 4a3, Next: 0 4a2, Prev: 0 4a4, Up: History<2> + +9.479 0.4a3 +=========== + + * Add progress messages to the search/download process so that + you can tell what URLs it’s reading to find download links. + (Hopefully, this will help people report out-of-date and + broken links to package authors, and to tell when they’ve + asked for a package that doesn’t exist.) + + +File: setuptools.info, Node: 0 4a2, Next: 0 4a1, Prev: 0 4a3, Up: History<2> + +9.480 0.4a2 +=========== + + * Added ‘ez_setup.py’ installer/bootstrap script to make initial + setuptools installation easier, and to allow distributions + using setuptools to avoid having to include setuptools in + their source distribution. + + * All downloads are now managed by the ‘PackageIndex’ class + (which is now subclassable and replaceable), so that embedders + can more easily override download logic, give download + progress reports, etc. The class has also been moved to the + new ‘setuptools.package_index’ module. + + * The ‘Installer’ class no longer handles downloading, manages a + temporary directory, or tracks the ‘zip_ok’ option. + Downloading is now handled by ‘PackageIndex’, and ‘Installer’ + has become an ‘easy_install’ command class based on + ‘setuptools.Command’. + + * There is a new ‘setuptools.sandbox.run_setup()’ API to invoke + a setup script in a directory sandbox, and a new + ‘setuptools.archive_util’ module with an ‘unpack_archive()’ + API. These were split out of EasyInstall to allow reuse by + other tools and applications. + + * ‘setuptools.Command’ now supports reinitializing commands + using keyword arguments to set/reset options. Also, ‘Command’ + subclasses can now set their ‘command_consumes_arguments’ + attribute to ‘True’ in order to receive an ‘args’ option + containing the rest of the command line. + + * Added support for installing scripts + + * Added support for setting options via distutils configuration + files, and using distutils’ default options as a basis for + EasyInstall’s defaults. + + * Renamed ‘--scan-url/-s’ to ‘--find-links/-f’ to free up ‘-s’ + for the script installation directory option. + + * Use ‘urllib2’ instead of ‘urllib’, to allow use of ‘https:’ + URLs if Python includes SSL support. + + +File: setuptools.info, Node: 0 4a1, Next: 0 3a4, Prev: 0 4a2, Up: History<2> + +9.481 0.4a1 +=========== + + * Added ‘--scan-url’ and ‘--index-url’ options, to scan download + pages and search PyPI for needed packages. + + +File: setuptools.info, Node: 0 3a4, Next: 0 3a3, Prev: 0 4a1, Up: History<2> + +9.482 0.3a4 +=========== + + * Restrict ‘--build-directory=DIR/-b DIR’ option to only be used + with single URL installs, to avoid running the wrong setup.py. + + +File: setuptools.info, Node: 0 3a3, Next: 0 3a2, Prev: 0 3a4, Up: History<2> + +9.483 0.3a3 +=========== + + * Added ‘--build-directory=DIR/-b DIR’ option. + + * Added “installation report” that explains how to use + ‘require()’ when doing a multiversion install or alternate + installation directory. + + * Added SourceForge mirror auto-select (Contributed by Ian + Bicking) + + * Added “sandboxing” that stops a setup script from running if + it attempts to write to the filesystem outside of the build + area + + * Added more workarounds for packages with quirky ‘install_data’ + hacks + + +File: setuptools.info, Node: 0 3a2, Next: 0 3a1, Prev: 0 3a3, Up: History<2> + +9.484 0.3a2 +=========== + + * Added new options to ‘bdist_egg’ to allow tagging the egg’s + version number with a subversion revision number, the current + date, or an explicit tag value. Run ‘setup.py bdist_egg + --help’ to get more information. + + * Added subversion download support for ‘svn:’ and ‘svn+’ URLs, + as well as automatic recognition of HTTP subversion URLs + (Contributed by Ian Bicking) + + * Misc. bug fixes + + +File: setuptools.info, Node: 0 3a1, Prev: 0 3a2, Up: History<2> + +9.485 0.3a1 +=========== + + * Initial release. + + +File: setuptools.info, Node: Credits, Next: Index, Prev: History<2>, Up: Top + +10 Credits +********** + + * The original design for the ‘.egg’ format and the ‘pkg_resources’ + API was co-created by Phillip Eby and Bob Ippolito. Bob also + implemented the first version of ‘pkg_resources’, and supplied the + macOS operating system version compatibility algorithm. + + * Ian Bicking implemented many early “creature comfort” features of + easy_install, including support for downloading via Sourceforge and + Subversion repositories. Ian’s comments on the Web-SIG about WSGI + application deployment also inspired the concept of “entry points” + in eggs, and he has given talks at PyCon and elsewhere to inform + and educate the community about eggs and setuptools. + + * Jim Fulton contributed time and effort to build automated tests of + various aspects of ‘easy_install’, and supplied the doctests for + the command-line ‘.exe’ wrappers on Windows. + + * Phillip J. Eby is the seminal author of setuptools, and first + proposed the idea of an importable binary distribution format for + Python application plug-ins. + + * Significant parts of the implementation of setuptools were funded + by the Open Source Applications Foundation, to provide a plug-in + infrastructure for the Chandler PIM application. In addition, many + OSAF staffers (such as Mike “Code Bear” Taylor) contributed their + time and stress as guinea pigs for the use of eggs and setuptools, + even before eggs were “cool”. (Thanks, guys!) + + * Tarek Ziadé is the principal author of the Distribute fork, which + re-invigorated the community on the project, encouraged renewed + innovation, and addressed many defects. + + * Jason R. Coombs performed the merge with Distribute, maintaining + the project for several years in coordination with the Python + Packaging Authority (PyPA). + + +File: setuptools.info, Node: Index, Prev: Credits, Up: Top + +Index +***** + +[index] +* Menu: + +* Python Enhancement Proposals; PEP 370: Command-Line Options. + (line 100) +* Python Enhancement Proposals; PEP 517: Configuring setup using setup cfg files. + (line 9) +* Python Enhancement Proposals; PEP 517 <1>: setup cfg-only projects. + (line 8) +* Python Enhancement Proposals; PEP 517 <2>: setup cfg-only projects. + (line 12) +* Python Enhancement Proposals; PEP 517 <3>: setup cfg-only projects. + (line 13) +* Python Enhancement Proposals; PEP 517 <4>: setup cfg-only projects. + (line 18) +* Python Enhancement Proposals; PEP 517 <5>: setup cfg-only projects. + (line 23) +* Python Enhancement Proposals; PEP 517 <6>: setup cfg-only projects. + (line 33) +* Python Enhancement Proposals; PEP 517 <7>: setup cfg-only projects. + (line 36) +* Python Enhancement Proposals; PEP 517#build-requirements: Python packaging at a glance. + (line 13) +* Python Enhancement Proposals; PEP 518: setup cfg-only projects. + (line 18) + + + +Tag Table: +Node: Top367 +Ref: index doc615 +Ref: 0615 +Node: Building and Distributing Packages with Setuptools22415 +Ref: userguide/index doc22550 +Ref: 122550 +Ref: userguide/index building-and-distributing-packages-with-setuptools22550 +Ref: 222550 +Ref: userguide/index documentation22550 +Ref: 322550 +Node: Transition to PEP51723037 +Ref: userguide/index transition-to-pep51723148 +Ref: 423148 +Node: setuptools Quickstart24269 +Ref: userguide/quickstart doc24399 +Ref: 624399 +Ref: userguide/quickstart setuptools-quickstart24399 +Ref: 724399 +Node: Installation24844 +Ref: userguide/quickstart installation24955 +Ref: 824955 +Node: Python packaging at a glance25088 +Ref: userguide/quickstart python-packaging-at-a-glance25217 +Ref: 925217 +Ref: Python packaging at a glance-Footnote-125977 +Ref: Python packaging at a glance-Footnote-226045 +Node: Basic Use26080 +Ref: userguide/quickstart basic-use26224 +Ref: a26224 +Ref: Basic Use-Footnote-127806 +Node: Automatic package discovery27847 +Ref: userguide/quickstart automatic-package-discovery28005 +Ref: b28005 +Node: Entry points and automatic script creation29156 +Ref: userguide/quickstart entry-points-and-automatic-script-creation29326 +Ref: d29326 +Node: Dependency management30140 +Ref: userguide/quickstart dependency-management30303 +Ref: f30303 +Node: Including Data Files31432 +Ref: userguide/quickstart id131569 +Ref: 1131569 +Ref: userguide/quickstart including-data-files31569 +Ref: 1231569 +Node: Development mode32179 +Ref: userguide/quickstart development-mode32325 +Ref: 1432325 +Node: Uploading your package to PyPI32997 +Ref: userguide/quickstart uploading-your-package-to-pypi33163 +Ref: 1533163 +Ref: Uploading your package to PyPI-Footnote-133482 +Node: Transitioning from setup py to setup cfg33522 +Ref: userguide/quickstart transitioning-from-setup-py-to-setup-cfg33701 +Ref: 1633701 +Node: Resources on Python packaging34195 +Ref: userguide/quickstart resources-on-python-packaging34335 +Ref: 1734335 +Node: Package Discovery and Namespace Package34512 +Ref: userguide/package_discovery doc34663 +Ref: 1834663 +Ref: userguide/package_discovery package-discovery34663 +Ref: c34663 +Ref: userguide/package_discovery package-discovery-and-namespace-package34663 +Ref: 1934663 +Node: Using find or find_packages36066 +Ref: userguide/package_discovery using-find-or-find-packages36229 +Ref: 1b36229 +Node: Using find_namespace or find_namespace_packages37302 +Ref: userguide/package_discovery namespace-packages37499 +Ref: 1c37499 +Ref: userguide/package_discovery using-find-namespace-or-find-namespace-packages37499 +Ref: 1d37499 +Node: Legacy Namespace Packages39655 +Ref: userguide/package_discovery legacy-namespace-packages39816 +Ref: 1e39816 +Ref: Legacy Namespace Packages-Footnote-140601 +Ref: Legacy Namespace Packages-Footnote-240651 +Node: pkg_resource style namespace package40725 +Ref: userguide/package_discovery pkg-resource-style-namespace-package40867 +Ref: 1f40867 +Node: pkgutil style namespace package41764 +Ref: userguide/package_discovery pkgutil-style-namespace-package41906 +Ref: 2041906 +Node: Entry Points42319 +Ref: userguide/entry_point doc42486 +Ref: e42486 +Ref: userguide/entry_point entry-points42486 +Ref: 2142486 +Ref: userguide/entry_point id142486 +Ref: 2242486 +Node: Console Scripts42830 +Ref: userguide/entry_point console-scripts42927 +Ref: 2342927 +Ref: Console Scripts-Footnote-144615 +Node: Advertising Behavior44668 +Ref: userguide/entry_point advertising-behavior44795 +Ref: 2444795 +Ref: userguide/entry_point dynamic-discovery-of-services-and-plugins44795 +Ref: 2544795 +Ref: Advertising Behavior-Footnote-146935 +Ref: Advertising Behavior-Footnote-246998 +Ref: Advertising Behavior-Footnote-347064 +Node: Dependency Management47116 +Ref: userguide/entry_point dependency-management47219 +Ref: 2647219 +Node: Dependencies Management in Setuptools48070 +Ref: userguide/dependency_management doc48216 +Ref: 1048216 +Ref: userguide/dependency_management dependencies-management-in-setuptools48216 +Ref: 2748216 +Ref: Dependencies Management in Setuptools-Footnote-148731 +Node: Build system requirement48781 +Ref: userguide/dependency_management build-system-requirement48921 +Ref: 2848921 +Node: Package requirement49022 +Ref: userguide/dependency_management package-requirement49106 +Ref: 2949106 +Node: Declaring required dependency50070 +Ref: userguide/dependency_management declaring-dependencies50240 +Ref: 2b50240 +Ref: userguide/dependency_management declaring-required-dependency50240 +Ref: 2c50240 +Node: Platform specific dependencies51217 +Ref: userguide/dependency_management platform-specific-dependencies51360 +Ref: 2d51360 +Ref: Platform specific dependencies-Footnote-152614 +Node: Dependencies that aren’t in PyPI52664 +Ref: userguide/dependency_management dependencies-that-aren-t-in-pypi52807 +Ref: 2e52807 +Node: Optional dependencies55587 +Ref: userguide/dependency_management optional-dependencies55751 +Ref: 2f55751 +Node: Python requirement58781 +Ref: userguide/dependency_management python-requirement58907 +Ref: 3058907 +Node: Data Files Support59157 +Ref: userguide/datafiles doc59313 +Ref: 1359313 +Ref: userguide/datafiles data-files-support59313 +Ref: 3159313 +Ref: Data Files Support-Footnote-165829 +Node: Accessing Data Files at Runtime65915 +Ref: userguide/datafiles accessing-data-files-at-runtime66036 +Ref: 3366036 +Ref: userguide/datafiles id266036 +Ref: 3466036 +Ref: Accessing Data Files at Runtime-Footnote-166883 +Node: Non-Package Data Files66968 +Ref: userguide/datafiles importlib-resources67089 +Ref: 3667089 +Ref: userguide/datafiles non-package-data-files67089 +Ref: 3767089 +Ref: Non-Package Data Files-Footnote-167639 +Node: “Development Mode”67760 +Ref: userguide/development_mode doc67935 +Ref: 3867935 +Ref: userguide/development_mode development-mode67935 +Ref: 3967935 +Node: Tagging and “Daily Build” or “Snapshot” Releases71275 +Ref: userguide/distribution doc71463 +Ref: 3b71463 +Ref: userguide/distribution tagging-and-daily-build-or-snapshot-releases71463 +Ref: 3c71463 +Node: Generating Source Distributions74524 +Ref: userguide/distribution generating-source-distributions74734 +Ref: 4174734 +Node: Making “Official” Non-Snapshot Releases76624 +Ref: userguide/distribution making-official-non-snapshot-releases76739 +Ref: 4276739 +Node: Distributing Extensions compiled with Cython78161 +Ref: userguide/distribution distributing-extensions-compiled-with-cython78350 +Ref: 4378350 +Node: Specifying Your Project’s Version79935 +Ref: userguide/distribution id180122 +Ref: 4480122 +Ref: userguide/distribution specifying-your-project-s-version80122 +Ref: 3e80122 +Node: Creating distutils Extensions85080 +Ref: userguide/extension doc85262 +Ref: 4585262 +Ref: userguide/extension creating-distutils-extensions85262 +Ref: 4685262 +Ref: userguide/extension id185262 +Ref: 4785262 +Node: Adding Commands86205 +Ref: userguide/extension adding-commands86321 +Ref: 4886321 +Node: Adding setup Arguments87585 +Ref: userguide/extension adding-setup-arguments87742 +Ref: 4987742 +Node: Customizing Distribution Options90361 +Ref: userguide/extension customizing-distribution-options90528 +Ref: 4a90528 +Node: Adding new EGG-INFO Files91395 +Ref: userguide/extension adding-new-egg-info-files91583 +Ref: 4b91583 +Ref: userguide/extension id291583 +Ref: 4c91583 +Node: Adding Support for Revision Control Systems94085 +Ref: userguide/extension adding-support-for-revision-control-systems94232 +Ref: 3294232 +Ref: userguide/extension id394232 +Ref: 4d94232 +Ref: Adding Support for Revision Control Systems-Footnote-197167 +Ref: Adding Support for Revision Control Systems-Footnote-297216 +Node: Configuring setup using setup cfg files97265 +Ref: userguide/declarative_config doc97442 +Ref: 4e97442 +Ref: userguide/declarative_config configuring-setup-using-setup-cfg-files97442 +Ref: 4f97442 +Ref: userguide/declarative_config declarative-config97442 +Ref: 5097442 +Ref: Configuring setup using setup cfg files-Footnote-1100442 +Node: Using a src/ layout100491 +Ref: userguide/declarative_config using-a-src-layout100616 +Ref: 51100616 +Node: Specifying values101349 +Ref: userguide/declarative_config specifying-values101474 +Ref: 52101474 +Node: Metadata102746 +Ref: userguide/declarative_config metadata102828 +Ref: 53102828 +Node: Options108439 +Ref: userguide/declarative_config options108521 +Ref: 54108521 +Node: New and Changed setup Keywords113055 +Ref: userguide/keywords doc113220 +Ref: 1a113220 +Ref: userguide/keywords new-and-changed-setup-keywords113220 +Ref: 55113220 +Ref: userguide/keywords test-loader119892 +Ref: 57119892 +Node: Command Reference123156 +Ref: userguide/commands doc123337 +Ref: 5a123337 +Ref: userguide/commands command-reference123337 +Ref: 5b123337 +Node: alias - Define shortcuts for commonly used commands123908 +Ref: userguide/commands alias124073 +Ref: 40124073 +Ref: userguide/commands alias-define-shortcuts-for-commonly-used-commands124073 +Ref: 5c124073 +Node: bdist_egg - Create a Python Egg for the project127121 +Ref: userguide/commands bdist-egg-create-a-python-egg-for-the-project127356 +Ref: 5f127356 +Node: develop - Deploy the project source in “Development Mode”131349 +Ref: userguide/commands develop131582 +Ref: 3a131582 +Ref: userguide/commands develop-deploy-the-project-source-in-development-mode131582 +Ref: 60131582 +Node: egg_info - Create egg metadata and set build tags138553 +Ref: userguide/commands egg-info138782 +Ref: 3d138782 +Ref: userguide/commands egg-info-create-egg-metadata-and-set-build-tags138782 +Ref: 61138782 +Node: Release Tagging Options140219 +Ref: userguide/commands release-tagging-options140363 +Ref: 62140363 +Node: Other egg_info Options142521 +Ref: userguide/commands other-egg-info-options142691 +Ref: 63142691 +Node: egg_info Examples143364 +Ref: userguide/commands egg-info-examples143502 +Ref: 64143502 +Node: rotate - Delete outdated distribution files143966 +Ref: userguide/commands rotate144186 +Ref: 3f144186 +Ref: userguide/commands rotate-delete-outdated-distribution-files144186 +Ref: 65144186 +Node: saveopts - Save used options to a configuration file146091 +Ref: userguide/commands saveopts146324 +Ref: 5e146324 +Ref: userguide/commands saveopts-save-used-options-to-a-configuration-file146324 +Ref: 66146324 +Node: Configuration File Options147929 +Ref: userguide/commands configuration-file-options148048 +Ref: 5d148048 +Node: setopt - Set a distutils or setuptools option in a config file149367 +Ref: userguide/commands setopt149602 +Ref: 67149602 +Ref: userguide/commands setopt-set-a-distutils-or-setuptools-option-in-a-config-file149602 +Ref: 68149602 +Node: test - Build package and run a unittest suite150985 +Ref: userguide/commands test151223 +Ref: 56151223 +Ref: userguide/commands test-build-package-and-run-a-unittest-suite151223 +Ref: 69151223 +Ref: test - Build package and run a unittest suite-Footnote-1154185 +Node: upload - Upload source and/or egg distributions to PyPI154220 +Ref: userguide/commands upload154387 +Ref: 6a154387 +Ref: userguide/commands upload-upload-source-and-or-egg-distributions-to-pypi154387 +Ref: 6b154387 +Ref: upload - Upload source and/or egg distributions to PyPI-Footnote-1154910 +Ref: upload - Upload source and/or egg distributions to PyPI-Footnote-2154943 +Node: Using setuptools to package and distribute your project155047 +Ref: userguide/functionalities_rewrite doc155227 +Ref: 6c155227 +Ref: userguide/functionalities_rewrite using-setuptools-to-package-and-distribute-your-project155227 +Ref: 6d155227 +Node: Automatic Resource Extraction155527 +Ref: userguide/miscellaneous doc155718 +Ref: 6e155718 +Ref: userguide/miscellaneous automatic-resource-extraction155718 +Ref: 58155718 +Ref: userguide/miscellaneous id1155718 +Ref: 6f155718 +Node: Defining Additional Metadata157957 +Ref: userguide/miscellaneous defining-additional-metadata158118 +Ref: 70158118 +Node: Setting the zip_safe flag158928 +Ref: userguide/miscellaneous setting-the-zip-safe-flag159051 +Ref: 71159051 +Node: Build System Support161810 +Ref: build_meta doc161999 +Ref: 5161999 +Ref: build_meta build-system-support161999 +Ref: 72161999 +Node: What is it?162091 +Ref: build_meta what-is-it162186 +Ref: 73162186 +Ref: What is it?-Footnote-1163895 +Ref: What is it?-Footnote-2163940 +Node: How to use it?163990 +Ref: build_meta how-to-use-it164085 +Ref: 74164085 +Ref: How to use it?-Footnote-1165455 +Ref: How to use it?-Footnote-2165517 +Node: Package Discovery and Resource Access using pkg_resources165557 +Ref: pkg_resources doc165704 +Ref: 75165704 +Ref: pkg_resources package-discovery-and-resource-access-using-pkg-resources165704 +Ref: 76165704 +Node: Overview166350 +Ref: pkg_resources overview166478 +Ref: 77166478 +Ref: Overview-Footnote-1170904 +Node: API Reference170982 +Ref: pkg_resources api-reference171110 +Ref: 78171110 +Node: Namespace Package Support171426 +Ref: pkg_resources namespace-package-support171532 +Ref: 79171532 +Node: WorkingSet Objects174056 +Ref: pkg_resources workingset-objects174190 +Ref: 7b174190 +Node: Basic WorkingSet Methods176726 +Ref: pkg_resources basic-workingset-methods176851 +Ref: 7c176851 +Node: WorkingSet Methods and Attributes180619 +Ref: pkg_resources workingset-methods-and-attributes180783 +Ref: 80180783 +Node: Receiving Change Notifications184950 +Ref: pkg_resources receiving-change-notifications185106 +Ref: 82185106 +Node: Locating Plugins186449 +Ref: pkg_resources locating-plugins186563 +Ref: 83186563 +Node: Environment Objects190673 +Ref: pkg_resources environment-objects190801 +Ref: 81190801 +Node: Requirement Objects196336 +Ref: pkg_resources requirement-objects196461 +Ref: 84196461 +Node: Requirements Parsing196840 +Ref: pkg_resources requirements-parsing196963 +Ref: 7d196963 +Node: Requirement Methods and Attributes199298 +Ref: pkg_resources requirement-methods-and-attributes199421 +Ref: 85199421 +Node: Entry Points<2>202078 +Ref: pkg_resources entry-points202204 +Ref: 7f202204 +Node: Convenience API204711 +Ref: pkg_resources convenience-api204811 +Ref: 86204811 +Node: Creating and Parsing207221 +Ref: pkg_resources creating-and-parsing207348 +Ref: 87207348 +Node: EntryPoint Objects210392 +Ref: pkg_resources entrypoint-objects210495 +Ref: 88210495 +Node: Distribution Objects211979 +Ref: pkg_resources distribution-objects212105 +Ref: 89212105 +Node: Getting or Creating Distributions212593 +Ref: pkg_resources getting-or-creating-distributions212719 +Ref: 8a212719 +Node: Distribution Attributes216620 +Ref: pkg_resources distribution-attributes216775 +Ref: 8d216775 +Node: Distribution Methods220522 +Ref: pkg_resources distribution-methods220635 +Ref: 90220635 +Node: ResourceManager API225533 +Ref: pkg_resources id1225656 +Ref: 91225656 +Ref: pkg_resources resourcemanager-api225656 +Ref: 35225656 +Node: Basic Resource Access226588 +Ref: pkg_resources basic-resource-access226697 +Ref: 92226697 +Node: Resource Extraction229834 +Ref: pkg_resources resource-extraction229976 +Ref: 93229976 +Node: “Provider” Interface232647 +Ref: pkg_resources provider-interface232759 +Ref: 94232759 +Node: Metadata API235052 +Ref: pkg_resources metadata-api235165 +Ref: 7e235165 +Node: IMetadataProvider Methods237259 +Ref: pkg_resources imetadataprovider-methods237337 +Ref: 8c237337 +Node: Exceptions238661 +Ref: pkg_resources exceptions238782 +Ref: 96238782 +Node: Supporting Custom Importers240205 +Ref: pkg_resources supporting-custom-importers240331 +Ref: 7a240331 +Node: IResourceProvider242930 +Ref: pkg_resources iresourceprovider243051 +Ref: 8b243051 +Node: Built-in Resource Providers244880 +Ref: pkg_resources built-in-resource-providers245001 +Ref: 97245001 +Node: Utility Functions248068 +Ref: pkg_resources utility-functions248175 +Ref: 98248175 +Node: Parsing Utilities248556 +Ref: pkg_resources parsing-utilities248658 +Ref: 8e248658 +Ref: pkg_resources yield-lines249212 +Ref: 95249212 +Node: Platform Utilities253345 +Ref: pkg_resources platform-utilities253473 +Ref: 8f253473 +Node: PEP 302 Utilities255688 +Ref: pkg_resources pep-302-utilities255818 +Ref: 99255818 +Node: File/Path Utilities255962 +Ref: pkg_resources file-path-utilities256081 +Ref: 9a256081 +Node: History256902 +Ref: pkg_resources history256995 +Ref: 9b256995 +Node: Keywords270832 +Ref: references/keywords doc270966 +Ref: 9c270966 +Ref: references/keywords keywords270966 +Ref: 9d270966 +Ref: references/keywords test-loader282821 +Ref: 9e282821 +Ref: Keywords-Footnote-1286193 +Ref: Keywords-Footnote-2286243 +Node: Roadmap286292 +Ref: roadmap doc286422 +Ref: 9f286422 +Ref: roadmap roadmap286422 +Ref: a0286422 +Ref: Roadmap-Footnote-1286568 +Node: Building and Distributing Packages with Setuptools<2>286622 +Ref: setuptools doc286769 +Ref: a1286769 +Ref: setuptools building-and-distributing-packages-with-setuptools286769 +Ref: a2286769 +Ref: Building and Distributing Packages with Setuptools<2>-Footnote-1289138 +Node: Developer’s Guide289197 +Ref: setuptools developer-s-guide289310 +Ref: a3289310 +Node: TRANSITIONAL NOTE289389 +Ref: setuptools transitional-note289466 +Ref: a4289466 +Node: setup cfg-only projects290789 +Ref: setuptools setup-cfg-only-projects290896 +Ref: a5290896 +Ref: setup cfg-only projects-Footnote-1292299 +Ref: setup cfg-only projects-Footnote-2292348 +Ref: setup cfg-only projects-Footnote-3292397 +Ref: setup cfg-only projects-Footnote-4292446 +Ref: setup cfg-only projects-Footnote-5292495 +Ref: setup cfg-only projects-Footnote-6292544 +Ref: setup cfg-only projects-Footnote-7292593 +Ref: setup cfg-only projects-Footnote-8292642 +Node: Configuration API292691 +Ref: setuptools configuration-api292835 +Ref: a6292835 +Node: Mailing List and Bug Tracker293857 +Ref: setuptools mailing-list-and-bug-tracker293969 +Ref: a7293969 +Ref: Mailing List and Bug Tracker-Footnote-1294339 +Ref: Mailing List and Bug Tracker-Footnote-2294395 +Node: Development on Setuptools294439 +Ref: development/index doc294633 +Ref: a8294633 +Ref: development/index development-on-setuptools294633 +Ref: a9294633 +Ref: development/index setuptools-bug-tracker294633 +Ref: aa294633 +Node: Developer’s Guide for Setuptools296088 +Ref: development/developer-guide doc296212 +Ref: ab296212 +Ref: development/developer-guide developer-s-guide-for-setuptools296212 +Ref: ac296212 +Node: Recommended Reading296586 +Ref: development/developer-guide recommended-reading296707 +Ref: ad296707 +Ref: Recommended Reading-Footnote-1297098 +Node: Project Management297165 +Ref: development/developer-guide project-management297312 +Ref: ae297312 +Ref: Project Management-Footnote-1297899 +Ref: Project Management-Footnote-2297942 +Ref: Project Management-Footnote-3298015 +Node: Authoring Tickets298057 +Ref: development/developer-guide authoring-tickets298206 +Ref: af298206 +Node: Making a pull request299312 +Ref: development/developer-guide making-a-pull-request299462 +Ref: b0299462 +Node: Adding change notes with your PRs299928 +Ref: development/developer-guide adding-change-notes-with-your-prs300071 +Ref: b1300071 +Ref: development/developer-guide id1300071 +Ref: b2300071 +Ref: Adding change notes with your PRs-Footnote-1300998 +Node: Alright! So how to add a news fragment?301076 +Ref: development/developer-guide alright-so-how-to-add-a-news-fragment301287 +Ref: b3301287 +Ref: Alright! So how to add a news fragment?-Footnote-1303484 +Node: Examples for adding changelog entries to your Pull Requests303528 +Ref: development/developer-guide examples-for-adding-changelog-entries-to-your-pull-requests303697 +Ref: b4303697 +Node: Auto-Merge Requests304303 +Ref: development/developer-guide auto-merge-requests304443 +Ref: b5304443 +Ref: development/developer-guide towncrier-philosophy304443 +Ref: b6304443 +Node: Testing304769 +Ref: development/developer-guide testing304907 +Ref: b7304907 +Node: Semantic Versioning305179 +Ref: development/developer-guide semantic-versioning305320 +Ref: b8305320 +Node: Building Documentation305407 +Ref: development/developer-guide building-documentation305562 +Ref: b9305562 +Ref: Building Documentation-Footnote-1305844 +Ref: Building Documentation-Footnote-2305889 +Node: Vendored Dependencies305942 +Ref: development/developer-guide published-documentation306069 +Ref: ba306069 +Ref: development/developer-guide vendored-dependencies306069 +Ref: bb306069 +Ref: Vendored Dependencies-Footnote-1306650 +Node: Release Process306704 +Ref: development/releases doc306828 +Ref: bc306828 +Ref: development/releases release-process306828 +Ref: bd306828 +Node: Release Frequency307216 +Ref: development/releases release-frequency307314 +Ref: be307314 +Node: Release Managers308292 +Ref: development/releases release-managers308390 +Ref: bf308390 +Node: Guides on backward compatibility & deprecated practice308525 +Ref: deprecated/index doc308676 +Ref: 2a308676 +Ref: deprecated/index guides-on-backward-compatibility-deprecated-practice308676 +Ref: c0308676 +Node: Supporting both Python 2 and Python 3 with Setuptools309462 +Ref: deprecated/python3 doc309656 +Ref: 59309656 +Ref: deprecated/python3 supporting-both-python-2-and-python-3-with-setuptools309656 +Ref: c1309656 +Ref: Supporting both Python 2 and Python 3 with Setuptools-Footnote-1310381 +Ref: Supporting both Python 2 and Python 3 with Setuptools-Footnote-2310419 +Node: Using 2to3310460 +Ref: deprecated/python3 using-2to3310602 +Ref: c2310602 +Node: Differential conversion312551 +Ref: deprecated/python3 differential-conversion312625 +Ref: c3312625 +Node: Distributing Python 3 modules313446 +Ref: deprecated/python3 distributing-python-3-modules313614 +Ref: c4313614 +Node: Advanced features314070 +Ref: deprecated/python3 advanced-features314219 +Ref: c5314219 +Node: The Internal Structure of Python Eggs314429 +Ref: deprecated/python_eggs doc314644 +Ref: c6314644 +Ref: deprecated/python_eggs the-internal-structure-of-python-eggs314644 +Ref: c7314644 +Node: Eggs and their Formats314878 +Ref: deprecated/python_eggs eggs-and-their-formats315004 +Ref: c8315004 +Node: Code and Resources318188 +Ref: deprecated/python_eggs code-and-resources318294 +Ref: c9318294 +Node: Project Metadata319082 +Ref: deprecated/python_eggs project-metadata319223 +Ref: ca319223 +Node: Filename-Embedded Metadata321015 +Ref: deprecated/python_eggs filename-embedded-metadata321147 +Ref: cc321147 +Node: Egg Links323662 +Ref: deprecated/python_eggs egg-links323769 +Ref: cd323769 +Node: Standard Metadata325841 +Ref: deprecated/python_eggs standard-metadata326006 +Ref: cb326006 +Node: txt File Formats327581 +Ref: deprecated/python_eggs txt-file-formats327683 +Ref: ce327683 +Node: Dependency Metadata328856 +Ref: deprecated/python_eggs dependency-metadata329020 +Ref: cf329020 +Node: requires txt329290 +Ref: deprecated/python_eggs requires-txt329389 +Ref: d0329389 +Node: setup_requires txt330435 +Ref: deprecated/python_eggs setup-requires-txt330563 +Ref: d1330563 +Node: dependency_links txt330760 +Ref: deprecated/python_eggs dependency-links-txt330915 +Ref: d2330915 +Node: depends txt – Obsolete do not create!331273 +Ref: deprecated/python_eggs depends-txt-obsolete-do-not-create331401 +Ref: d3331401 +Node: namespace_packages txt – Namespace Package Metadata332008 +Ref: deprecated/python_eggs namespace-packages-txt-namespace-package-metadata332210 +Ref: d4332210 +Node: entry_points txt – “Entry Point”/Plugin Metadata332571 +Ref: deprecated/python_eggs entry-points-txt-entry-point-plugin-metadata332778 +Ref: d5332778 +Node: The scripts Subdirectory333650 +Ref: deprecated/python_eggs the-scripts-subdirectory333824 +Ref: d6333824 +Node: Zip Support Metadata334781 +Ref: deprecated/python_eggs zip-support-metadata334947 +Ref: d9334947 +Node: native_libs txt335128 +Ref: deprecated/python_eggs native-libs-txt335232 +Ref: da335232 +Node: eager_resources txt335897 +Ref: deprecated/python_eggs eager-resources-txt336035 +Ref: dc336035 +Node: zip-safe and not-zip-safe336653 +Ref: deprecated/python_eggs zip-safe-and-not-zip-safe336767 +Ref: dd336767 +Node: top_level txt – Conflict Management Metadata337816 +Ref: deprecated/python_eggs top-level-txt-conflict-management-metadata337995 +Ref: de337995 +Node: SOURCES txt – Source Files Manifest338804 +Ref: deprecated/python_eggs sources-txt-source-files-manifest338954 +Ref: df338954 +Node: Other Technical Considerations340189 +Ref: deprecated/python_eggs other-technical-considerations340323 +Ref: e0340323 +Node: Zip File Issues340472 +Ref: deprecated/python_eggs zip-file-issues340606 +Ref: db340606 +Node: The Extraction Process341840 +Ref: deprecated/python_eggs the-extraction-process341952 +Ref: e1341952 +Node: Extension Import Wrappers343133 +Ref: deprecated/python_eggs extension-import-wrappers343245 +Ref: e2343245 +Node: Installation and Path Management Issues344580 +Ref: deprecated/python_eggs installation-and-path-management-issues344714 +Ref: d8344714 +Node: Script Wrappers349270 +Ref: deprecated/python_eggs script-wrappers349365 +Ref: d7349365 +Node: Easy Install351737 +Ref: deprecated/easy_install doc351921 +Ref: e3351921 +Ref: deprecated/easy_install easy-install351921 +Ref: e4351921 +Ref: Easy Install-Footnote-1353191 +Node: Using “Easy Install”353247 +Ref: deprecated/easy_install using-easy-install353349 +Ref: e5353349 +Node: Installing “Easy Install”353732 +Ref: deprecated/easy_install installation-instructions353871 +Ref: e6353871 +Ref: deprecated/easy_install installing-easy-install353871 +Ref: e7353871 +Ref: Installing “Easy Install”-Footnote-1355301 +Ref: Installing “Easy Install”-Footnote-2355346 +Node: Troubleshooting355377 +Ref: deprecated/easy_install troubleshooting355484 +Ref: eb355484 +Node: Windows Notes356373 +Ref: deprecated/easy_install windows-notes356480 +Ref: ec356480 +Node: Downloading and Installing a Package356831 +Ref: deprecated/easy_install downloading-and-installing-a-package356998 +Ref: ee356998 +Ref: Downloading and Installing a Package-Footnote-1360432 +Node: Upgrading a Package360491 +Ref: deprecated/easy_install upgrading-a-package360656 +Ref: f1360656 +Node: Changing the Active Version362592 +Ref: deprecated/easy_install changing-the-active-version362742 +Ref: f3362742 +Node: Uninstalling Packages363810 +Ref: deprecated/easy_install uninstalling-packages363957 +Ref: f2363957 +Node: Managing Scripts364611 +Ref: deprecated/easy_install managing-scripts364756 +Ref: f4364756 +Ref: Managing Scripts-Footnote-1366587 +Node: Executables and Launchers366619 +Ref: deprecated/easy_install executables-and-launchers366760 +Ref: ed366760 +Node: Windows Executable Launcher368152 +Ref: deprecated/easy_install windows-executable-launcher368277 +Ref: f5368277 +Node: Natural Script Launcher368775 +Ref: deprecated/easy_install natural-script-launcher368900 +Ref: f6368900 +Ref: Natural Script Launcher-Footnote-1369909 +Node: Tips & Techniques369955 +Ref: deprecated/easy_install tips-techniques370104 +Ref: f7370104 +Node: Multiple Python Versions370360 +Ref: deprecated/easy_install multiple-python-versions370491 +Ref: f8370491 +Node: Restricting Downloads with --allow-hosts371060 +Ref: deprecated/easy_install restricting-downloads-with-allow-hosts371235 +Ref: e9371235 +Node: Installing on Un-networked Machines372141 +Ref: deprecated/easy_install installing-on-un-networked-machines372328 +Ref: f9372328 +Node: Packaging Others’ Projects As Eggs373914 +Ref: deprecated/easy_install packaging-others-projects-as-eggs374092 +Ref: fa374092 +Node: Creating your own Package Index375193 +Ref: deprecated/easy_install creating-your-own-package-index375327 +Ref: fb375327 +Node: Password-Protected Sites376573 +Ref: deprecated/easy_install password-protected-sites376721 +Ref: fd376721 +Node: Using pypirc Credentials377240 +Ref: deprecated/easy_install using-pypirc-credentials377362 +Ref: fe377362 +Node: Controlling Build Options378021 +Ref: deprecated/easy_install controlling-build-options378155 +Ref: ff378155 +Node: Editing and Viewing Source Packages379091 +Ref: deprecated/easy_install editing-and-viewing-source-packages379269 +Ref: ef379269 +Node: Dealing with Installation Conflicts381362 +Ref: deprecated/easy_install dealing-with-installation-conflicts381538 +Ref: 100381538 +Node: Compressed Installation383597 +Ref: deprecated/easy_install compressed-installation383729 +Ref: 101383729 +Node: Reference Manual386059 +Ref: deprecated/easy_install reference-manual386161 +Ref: 102386161 +Node: Configuration Files386329 +Ref: deprecated/easy_install configuration-files386434 +Ref: f0386434 +Ref: Configuration Files-Footnote-1388421 +Node: Command-Line Options388490 +Ref: deprecated/easy_install command-line-options388633 +Ref: ea388633 +Ref: Command-Line Options-Footnote-1404837 +Node: Custom Installation Locations404886 +Ref: deprecated/easy_install custom-installation-locations405033 +Ref: e8405033 +Ref: deprecated/easy_install non-root-installation405033 +Ref: 103405033 +Ref: Custom Installation Locations-Footnote-1406241 +Node: Use the “–user” option406575 +Ref: deprecated/easy_install pep-370406745 +Ref: 107406745 +Ref: deprecated/easy_install use-the-user-option406745 +Ref: 104406745 +Ref: Use the “–user” option-Footnote-1407332 +Node: Use the “–user” option and customize “PYTHONUSERBASE”407497 +Ref: deprecated/easy_install use-the-user-option-and-customize-pythonuserbase407696 +Ref: 105407696 +Node: Use “virtualenv”408168 +Ref: deprecated/easy_install use-virtualenv408330 +Ref: 106408330 +Ref: Use “virtualenv”-Footnote-1409040 +Node: Package Index “API”409085 +Ref: deprecated/easy_install package-index-api409203 +Ref: fc409203 +Ref: deprecated/easy_install virtualenv409203 +Ref: 108409203 +Node: Porting from Distutils412197 +Ref: deprecated/distutils-legacy doc412370 +Ref: 109412370 +Ref: deprecated/distutils-legacy porting-from-distutils412370 +Ref: 10a412370 +Ref: Porting from Distutils-Footnote-1412920 +Node: Prefer Setuptools412982 +Ref: deprecated/distutils-legacy prefer-setuptools413062 +Ref: 10b413062 +Ref: Prefer Setuptools-Footnote-1413971 +Node: “Eggsecutable” Scripts414022 +Ref: deprecated/functionalities doc414174 +Ref: 10c414174 +Ref: deprecated/functionalities eggsecutable-scripts414174 +Ref: 10d414174 +Node: History<2>415686 +Ref: history doc415819 +Ref: 10e415819 +Ref: history changes415819 +Ref: 10f415819 +Ref: history history415819 +Ref: 110415819 +Node: v50 3 2424446 +Ref: history v50-3-2424520 +Ref: 111424520 +Node: Documentation changes424603 +Ref: history documentation-changes424685 +Ref: 112424685 +Ref: Documentation changes-Footnote-1425167 +Ref: Documentation changes-Footnote-2425222 +Ref: Documentation changes-Footnote-3425267 +Ref: Documentation changes-Footnote-4425322 +Ref: Documentation changes-Footnote-5425367 +Ref: Documentation changes-Footnote-6425422 +Node: Misc425467 +Ref: history misc425549 +Ref: 113425549 +Ref: Misc-Footnote-1425878 +Ref: Misc-Footnote-2425933 +Ref: Misc-Footnote-3425978 +Ref: Misc-Footnote-4426033 +Ref: Misc-Footnote-5426088 +Node: v50 3 1426133 +Ref: history v50-3-1426223 +Ref: 114426223 +Node: Documentation changes<2>426339 +Ref: history id7426427 +Ref: 115426427 +Ref: Documentation changes<2>-Footnote-1426949 +Ref: Documentation changes<2>-Footnote-2427004 +Ref: Documentation changes<2>-Footnote-3427059 +Ref: Documentation changes<2>-Footnote-4427114 +Ref: Documentation changes<2>-Footnote-5427169 +Ref: Documentation changes<2>-Footnote-6427224 +Ref: Documentation changes<2>-Footnote-7427269 +Ref: Documentation changes<2>-Footnote-8427324 +Node: Misc<2>427369 +Ref: history id14427457 +Ref: 116427457 +Ref: Misc<2>-Footnote-1427666 +Ref: Misc<2>-Footnote-2427721 +Node: v50 3 0427776 +Ref: history v50-3-0427866 +Ref: 117427866 +Node: Changes427926 +Ref: history id17427981 +Ref: 118427981 +Ref: Changes-Footnote-1428157 +Ref: Changes-Footnote-2428212 +Node: v50 2 0428264 +Ref: history v50-2-0428354 +Ref: 119428354 +Node: Changes<2>428425 +Ref: history id19428483 +Ref: 11a428483 +Ref: Changes<2>-Footnote-1428942 +Ref: Changes<2>-Footnote-2428997 +Node: v50 1 0429052 +Ref: history v50-1-0429142 +Ref: 11b429142 +Node: Changes<3>429213 +Ref: history id22429271 +Ref: 11c429271 +Ref: Changes<3>-Footnote-1429782 +Node: v50 0 3429837 +Ref: history v50-0-3429927 +Ref: 11d429927 +Node: Misc<3>429992 +Ref: history id24430047 +Ref: 11e430047 +Ref: Misc<3>-Footnote-1430211 +Ref: Misc<3>-Footnote-2430266 +Node: v50 0 2430317 +Ref: history v50-0-2430407 +Ref: 11f430407 +Node: Misc<4>430472 +Ref: history id26430527 +Ref: 120430527 +Ref: Misc<4>-Footnote-1430690 +Ref: Misc<4>-Footnote-2430745 +Node: v50 0 1430787 +Ref: history v50-0-1430877 +Ref: 121430877 +Node: Misc<5>430942 +Ref: history id28430997 +Ref: 122430997 +Ref: Misc<5>-Footnote-1431441 +Ref: Misc<5>-Footnote-2431496 +Ref: Misc<5>-Footnote-3431551 +Node: v50 0 0431606 +Ref: history v50-0-0431696 +Ref: 123431696 +Node: Breaking Changes431788 +Ref: history breaking-changes431871 +Ref: 124431871 +Ref: Breaking Changes-Footnote-1432454 +Node: Changes<4>432509 +Ref: history id33432592 +Ref: 125432592 +Ref: Changes<4>-Footnote-1432719 +Node: v49 6 0432774 +Ref: history v49-6-0432864 +Ref: 126432864 +Node: Changes<5>432937 +Ref: history id35432995 +Ref: 127432995 +Ref: Changes<5>-Footnote-1433221 +Node: v49 5 0433276 +Ref: history v49-5-0433366 +Ref: 128433366 +Node: Changes<6>433439 +Ref: history id37433497 +Ref: 129433497 +Ref: Changes<6>-Footnote-1433762 +Ref: Changes<6>-Footnote-2433817 +Node: v49 4 0433867 +Ref: history v49-4-0433957 +Ref: 12a433957 +Node: Changes<7>434030 +Ref: history id39434088 +Ref: 12b434088 +Ref: Changes<7>-Footnote-1434215 +Node: v49 3 2434270 +Ref: history v49-3-2434360 +Ref: 12c434360 +Node: Documentation changes<3>434478 +Ref: history id41434566 +Ref: 12d434566 +Ref: Documentation changes<3>-Footnote-1434731 +Node: Misc<6>434786 +Ref: history id43434874 +Ref: 12e434874 +Ref: Misc<6>-Footnote-1435027 +Node: v49 3 1435082 +Ref: history v49-3-1435172 +Ref: 12f435172 +Node: Changes<8>435245 +Ref: history id45435303 +Ref: 130435303 +Ref: Changes<8>-Footnote-1435513 +Node: v49 3 0435568 +Ref: history v49-3-0435658 +Ref: 131435658 +Node: Changes<9>435731 +Ref: history id47435789 +Ref: 132435789 +Ref: Changes<9>-Footnote-1436219 +Node: v49 2 1436274 +Ref: history v49-2-1436364 +Ref: 133436364 +Node: Misc<7>436431 +Ref: history id49436486 +Ref: 134436486 +Ref: Misc<7>-Footnote-1436631 +Node: v49 2 0436686 +Ref: history v49-2-0436776 +Ref: 135436776 +Node: Changes<10>436850 +Ref: history id51436909 +Ref: 136436909 +Ref: Changes<10>-Footnote-1437175 +Node: v49 1 3437230 +Ref: history v49-1-3437320 +Ref: 137437320 +Node: Misc<8>437387 +Ref: history id53437442 +Ref: 138437442 +Ref: Misc<8>-Footnote-1437666 +Ref: Misc<8>-Footnote-2437721 +Node: v49 1 2437776 +Ref: history v49-1-2437866 +Ref: 139437866 +Node: Changes<11>437940 +Ref: history id56437999 +Ref: 13a437999 +Ref: Changes<11>-Footnote-1438450 +Node: v49 1 1438505 +Ref: history v49-1-1438595 +Ref: 13b438595 +Node: Misc<9>438662 +Ref: history id58438717 +Ref: 13c438717 +Ref: Misc<9>-Footnote-1438868 +Node: v49 0 1438923 +Ref: history v49-0-1439013 +Ref: 13d439013 +Node: Misc<10>439081 +Ref: history id60439137 +Ref: 13e439137 +Ref: Misc<10>-Footnote-1439369 +Ref: Misc<10>-Footnote-2439424 +Node: v49 1 0439475 +Ref: history v49-1-0439565 +Ref: 13f439565 +Node: Changes<12>439639 +Ref: history id62439698 +Ref: 140439698 +Ref: Changes<12>-Footnote-1439859 +Node: v49 0 0439914 +Ref: history v49-0-0440004 +Ref: 141440004 +Node: Breaking Changes<2>440137 +Ref: history id64440224 +Ref: 142440224 +Ref: Breaking Changes<2>-Footnote-1440630 +Ref: Breaking Changes<2>-Footnote-2440685 +Node: Changes<13>440740 +Ref: history id67440844 +Ref: 143440844 +Ref: Changes<13>-Footnote-1441209 +Ref: Changes<13>-Footnote-2441264 +Node: Misc<11>441319 +Ref: history id70441395 +Ref: 144441395 +Ref: Misc<11>-Footnote-1441574 +Node: v48 0 0441629 +Ref: history v48-0-0441719 +Ref: 145441719 +Node: Breaking Changes<3>441810 +Ref: history id72441877 +Ref: 146441877 +Ref: Breaking Changes<3>-Footnote-1442993 +Node: v47 3 2443048 +Ref: history v47-3-2443138 +Ref: 147443138 +Node: Misc<12>443206 +Ref: history id74443262 +Ref: 148443262 +Ref: Misc<12>-Footnote-1443423 +Node: v47 3 1443478 +Ref: history v47-3-1443568 +Ref: 149443568 +Node: Misc<13>443636 +Ref: history id76443692 +Ref: 14a443692 +Ref: Misc<13>-Footnote-1443971 +Ref: Misc<13>-Footnote-2444026 +Node: v47 3 0444081 +Ref: history v47-3-0444171 +Ref: 14b444171 +Node: Changes<14>444263 +Ref: history id79444339 +Ref: 14c444339 +Ref: Changes<14>-Footnote-1444591 +Node: Misc<14>444646 +Ref: history id81444722 +Ref: 14d444722 +Ref: Misc<14>-Footnote-1444879 +Node: v47 2 0444934 +Ref: history v47-2-0445024 +Ref: 14e445024 +Node: Changes<15>445098 +Ref: history id83445157 +Ref: 14f445157 +Ref: Changes<15>-Footnote-1445332 +Node: v47 1 1445387 +Ref: history v47-1-1445477 +Ref: 150445477 +Node: Documentation changes<4>445649 +Ref: history id85445762 +Ref: 151445762 +Ref: Documentation changes<4>-Footnote-1445919 +Node: Incorporate changes from v44 1 1445974 +Ref: history incorporate-changes-from-v44-1-1446087 +Ref: 152446087 +Ref: Incorporate changes from v44 1 1-Footnote-1446372 +Node: v44 1 1446427 +Ref: history v44-1-1446517 +Ref: 153446517 +Node: Misc<15>446585 +Ref: history id88446641 +Ref: 154446641 +Ref: Misc<15>-Footnote-1446868 +Node: v47 1 0446923 +Ref: history v47-1-0447013 +Ref: 155447013 +Node: Changes<16>447087 +Ref: history id90447146 +Ref: 156447146 +Ref: Changes<16>-Footnote-1447366 +Node: v47 0 0447421 +Ref: history v47-0-0447511 +Ref: 157447511 +Node: Breaking Changes<4>447626 +Ref: history id92447713 +Ref: 158447713 +Ref: Breaking Changes<4>-Footnote-1447957 +Node: Changes<17>448012 +Ref: history id94448099 +Ref: 159448099 +Ref: Changes<17>-Footnote-1448257 +Node: v46 4 0448312 +Ref: history v46-4-0448402 +Ref: 15a448402 +Node: Changes<18>448476 +Ref: history id96448535 +Ref: 15b448535 +Ref: Changes<18>-Footnote-1448913 +Node: v46 3 1448968 +Ref: history v46-3-1449058 +Ref: 15c449058 +Node: v46 3 0449123 +Ref: history v46-3-0449213 +Ref: 15d449213 +Node: Changes<19>449305 +Ref: history id98449381 +Ref: 15e449381 +Ref: Changes<19>-Footnote-1449639 +Ref: Changes<19>-Footnote-2449694 +Node: Misc<16>449748 +Ref: history id101449824 +Ref: 15f449824 +Ref: Misc<16>-Footnote-1450104 +Ref: Misc<16>-Footnote-2450159 +Node: v46 2 0450214 +Ref: history v46-2-0450304 +Ref: 160450304 +Node: Changes<20>450447 +Ref: history id104450539 +Ref: 161450539 +Ref: Changes<20>-Footnote-1451073 +Ref: Changes<20>-Footnote-2451128 +Ref: Changes<20>-Footnote-3451183 +Ref: Changes<20>-Footnote-4451238 +Node: Documentation changes<5>451293 +Ref: history id109451402 +Ref: 162451402 +Ref: Documentation changes<5>-Footnote-1451592 +Node: Misc<17>451647 +Ref: history id111451736 +Ref: 163451736 +Ref: Misc<17>-Footnote-1451969 +Ref: Misc<17>-Footnote-2452024 +Node: v46 1 3452067 +Ref: history v46-1-3452157 +Ref: 164452157 +Node: v46 1 2452222 +Ref: history v46-1-2452312 +Ref: 165452312 +Node: Misc<18>452380 +Ref: history id113452436 +Ref: 166452436 +Ref: Misc<18>-Footnote-1452570 +Node: v46 1 1452625 +Ref: history v46-1-1452715 +Ref: 167452715 +Node: v46 1 0452780 +Ref: history v46-1-0452870 +Ref: 168452870 +Node: Changes<21>453015 +Ref: history id115453115 +Ref: 169453115 +Ref: Changes<21>-Footnote-1453640 +Ref: Changes<21>-Footnote-2453694 +Ref: Changes<21>-Footnote-3453749 +Ref: Changes<21>-Footnote-4453804 +Ref: Changes<21>-Footnote-5453859 +Node: Incorporate changes from v44 1 0453909 +Ref: history incorporate-changes-from-v44-1-0454009 +Ref: 16a454009 +Ref: Incorporate changes from v44 1 0-Footnote-1454457 +Ref: Incorporate changes from v44 1 0-Footnote-2454512 +Ref: Incorporate changes from v44 1 0-Footnote-3454567 +Node: v44 1 0454622 +Ref: history v44-1-0454712 +Ref: 16b454712 +Node: Changes<22>454786 +Ref: history id123454845 +Ref: 16c454845 +Ref: Changes<22>-Footnote-1455241 +Ref: Changes<22>-Footnote-2455296 +Ref: Changes<22>-Footnote-3455351 +Node: v46 0 0455406 +Ref: history v46-0-0455496 +Ref: 16d455496 +Node: Breaking Changes<5>455680 +Ref: history id127455767 +Ref: 16e455767 +Ref: Breaking Changes<5>-Footnote-1455918 +Node: Changes<23>455971 +Ref: history id129456091 +Ref: 16f456091 +Ref: Changes<23>-Footnote-1456415 +Ref: Changes<23>-Footnote-2456470 +Node: Documentation changes<6>456525 +Ref: history id132456634 +Ref: 170456634 +Ref: Documentation changes<6>-Footnote-1456795 +Node: Misc<19>456850 +Ref: history id134456939 +Ref: 171456939 +Ref: Misc<19>-Footnote-1457097 +Node: v45 3 0457152 +Ref: history v45-3-0457242 +Ref: 172457242 +Node: Changes<24>457316 +Ref: history id136457375 +Ref: 173457375 +Ref: Changes<24>-Footnote-1457592 +Ref: Changes<24>-Footnote-2457647 +Node: v45 2 0457702 +Ref: history v45-2-0457792 +Ref: 174457792 +Node: Changes<25>457884 +Ref: history id139457960 +Ref: 175457960 +Ref: Changes<25>-Footnote-1458711 +Ref: Changes<25>-Footnote-2458766 +Ref: Changes<25>-Footnote-3458821 +Ref: Changes<25>-Footnote-4458871 +Ref: Changes<25>-Footnote-5458926 +Node: Misc<20>458981 +Ref: history id144459057 +Ref: 176459057 +Ref: Misc<20>-Footnote-1459211 +Node: v45 1 0459266 +Ref: history v45-1-0459356 +Ref: 177459356 +Node: Changes<26>459430 +Ref: history id146459489 +Ref: 178459489 +Ref: Changes<26>-Footnote-1459851 +Ref: Changes<26>-Footnote-2459906 +Ref: Changes<26>-Footnote-3459961 +Node: v45 0 0460016 +Ref: history v45-0-0460106 +Ref: 179460106 +Node: Breaking Changes<6>460221 +Ref: history id150460308 +Ref: 17a460308 +Ref: Breaking Changes<6>-Footnote-1460576 +Node: Changes<27>460631 +Ref: history id152460718 +Ref: 17b460718 +Ref: Changes<27>-Footnote-1460855 +Node: v44 0 0460910 +Ref: history v44-0-0461000 +Ref: 17c461000 +Node: Breaking Changes<7>461091 +Ref: history id154461158 +Ref: 17d461158 +Ref: Breaking Changes<7>-Footnote-1461288 +Node: v43 0 0461343 +Ref: history v43-0-0461433 +Ref: 17e461433 +Node: Breaking Changes<8>461548 +Ref: history id156461635 +Ref: 17f461635 +Ref: Breaking Changes<8>-Footnote-1462014 +Node: Changes<28>462069 +Ref: history id158462156 +Ref: 180462156 +Ref: Changes<28>-Footnote-1462537 +Ref: Changes<28>-Footnote-2462592 +Node: v42 0 2462642 +Ref: history v42-0-2462732 +Ref: 181462732 +Node: Changes<29>462806 +Ref: history id161462865 +Ref: 182462865 +Ref: Changes<29>-Footnote-1463162 +Ref: Changes<29>-Footnote-2463217 +Node: v42 0 1463272 +Ref: history v42-0-1463362 +Ref: 183463362 +Node: Changes<30>463436 +Ref: history id164463495 +Ref: 184463495 +Ref: Changes<30>-Footnote-1463632 +Node: v42 0 0463687 +Ref: history v42-0-0463777 +Ref: 185463777 +Node: Breaking Changes<9>463892 +Ref: history id166463979 +Ref: 186463979 +Ref: Breaking Changes<9>-Footnote-1464927 +Ref: Breaking Changes<9>-Footnote-2464982 +Ref: Breaking Changes<9>-Footnote-3465037 +Ref: Breaking Changes<9>-Footnote-4465076 +Ref: Breaking Changes<9>-Footnote-5465126 +Ref: Breaking Changes<9>-Footnote-6465176 +Ref: Breaking Changes<9>-Footnote-7465231 +Node: Changes<31>465264 +Ref: history id171465351 +Ref: 187465351 +Ref: Changes<31>-Footnote-1466048 +Ref: Changes<31>-Footnote-2466103 +Ref: Changes<31>-Footnote-3466158 +Ref: Changes<31>-Footnote-4466213 +Ref: Changes<31>-Footnote-5466268 +Node: v41 6 0466316 +Ref: history v41-6-0466406 +Ref: 188466406 +Node: Changes<32>466480 +Ref: history id176466539 +Ref: 189466539 +Ref: Changes<32>-Footnote-1466726 +Node: v41 5 1466780 +Ref: history v41-5-1466870 +Ref: 18a466870 +Node: Changes<33>466944 +Ref: history id178467003 +Ref: 18b467003 +Ref: Changes<33>-Footnote-1467167 +Node: v41 5 0467222 +Ref: history v41-5-0467312 +Ref: 18c467312 +Node: Changes<34>467455 +Ref: history id180467547 +Ref: 18d467547 +Ref: Changes<34>-Footnote-1468021 +Ref: Changes<34>-Footnote-2468076 +Ref: Changes<34>-Footnote-3468131 +Ref: Changes<34>-Footnote-4468181 +Ref: Changes<34>-Footnote-5468236 +Node: Documentation changes<7>468291 +Ref: history id185468400 +Ref: 18e468400 +Ref: Documentation changes<7>-Footnote-1468980 +Ref: Documentation changes<7>-Footnote-2469035 +Ref: Documentation changes<7>-Footnote-3469090 +Ref: Documentation changes<7>-Footnote-4469145 +Node: Misc<21>469200 +Ref: history id190469289 +Ref: 18f469289 +Ref: Misc<21>-Footnote-1469418 +Node: v41 4 0469473 +Ref: history v41-4-0469563 +Ref: 190469563 +Node: Changes<35>469637 +Ref: history id192469696 +Ref: 191469696 +Ref: Changes<35>-Footnote-1469880 +Node: v41 3 0469935 +Ref: history v41-3-0470025 +Ref: 192470025 +Node: Changes<36>470117 +Ref: history id194470193 +Ref: 193470193 +Ref: Changes<36>-Footnote-1470479 +Node: Misc<22>470534 +Ref: history id196470610 +Ref: 194470610 +Ref: Misc<22>-Footnote-1470786 +Node: v41 2 0470841 +Ref: history v41-2-0470931 +Ref: 195470931 +Node: Changes<37>471023 +Ref: history id198471099 +Ref: 196471099 +Ref: Changes<37>-Footnote-1471235 +Node: Misc<23>471289 +Ref: history id200471365 +Ref: 197471365 +Ref: Misc<23>-Footnote-1471567 +Node: v41 1 0471622 +Ref: history v41-1-0471712 +Ref: 198471712 +Node: Misc<24>471831 +Ref: history id202471920 +Ref: 199471920 +Ref: Misc<24>-Footnote-1472922 +Ref: Misc<24>-Footnote-2472977 +Ref: Misc<24>-Footnote-3473032 +Ref: Misc<24>-Footnote-4473082 +Ref: Misc<24>-Footnote-5473137 +Ref: Misc<24>-Footnote-6473187 +Ref: Misc<24>-Footnote-7473242 +Ref: Misc<24>-Footnote-8473297 +Ref: Misc<24>-Footnote-9473352 +Node: Documentation changes<8>473407 +Ref: history id212473496 +Ref: 19a473496 +Ref: Documentation changes<8>-Footnote-1473662 +Node: v41 0 1473717 +Ref: history v41-0-1473807 +Ref: 19b473807 +Node: Changes<38>473881 +Ref: history id214473940 +Ref: 19c473940 +Ref: Changes<38>-Footnote-1474411 +Ref: Changes<38>-Footnote-2474466 +Ref: Changes<38>-Footnote-3474516 +Ref: Changes<38>-Footnote-4474571 +Node: v41 0 0474626 +Ref: history v41-0-0474716 +Ref: 19d474716 +Node: Breaking Changes<10>474808 +Ref: history id219474876 +Ref: 19e474876 +Ref: Breaking Changes<10>-Footnote-1475468 +Node: v40 9 0475523 +Ref: history v40-9-0475613 +Ref: 19f475613 +Node: Changes<39>475738 +Ref: history id221475830 +Ref: 1a0475830 +Ref: Changes<39>-Footnote-1476552 +Ref: Changes<39>-Footnote-2476607 +Ref: Changes<39>-Footnote-3476657 +Ref: Changes<39>-Footnote-4476712 +Node: Documentation changes<9>476767 +Ref: history id226476859 +Ref: 1a1476859 +Ref: Documentation changes<9>-Footnote-1477057 +Node: v40 8 0477112 +Ref: history v40-8-0477202 +Ref: 1a2477202 +Node: Changes<40>477276 +Ref: history id228477335 +Ref: 1a3477335 +Ref: Changes<40>-Footnote-1478222 +Ref: Changes<40>-Footnote-2478277 +Ref: Changes<40>-Footnote-3478327 +Ref: Changes<40>-Footnote-4478382 +Node: v40 7 3478437 +Ref: history v40-7-3478527 +Ref: 1a4478527 +Node: Changes<41>478601 +Ref: history id233478660 +Ref: 1a5478660 +Ref: Changes<41>-Footnote-1478998 +Ref: Changes<41>-Footnote-2479053 +Ref: Changes<41>-Footnote-3479108 +Ref: Changes<41>-Footnote-4479163 +Ref: Changes<41>-Footnote-5479218 +Node: v40 7 2479273 +Ref: history v40-7-2479363 +Ref: 1a6479363 +Node: Changes<42>479437 +Ref: history id239479496 +Ref: 1a7479496 +Ref: Changes<42>-Footnote-1479626 +Node: v40 7 1479681 +Ref: history v40-7-1479771 +Ref: 1a8479771 +Node: Changes<43>479845 +Ref: history id241479904 +Ref: 1a9479904 +Ref: Changes<43>-Footnote-1480103 +Node: v40 7 0480158 +Ref: history v40-7-0480248 +Ref: 1aa480248 +Node: Breaking Changes<11>480364 +Ref: history id243480452 +Ref: 1ab480452 +Ref: Breaking Changes<11>-Footnote-1480654 +Node: Changes<44>480709 +Ref: history id245480797 +Ref: 1ac480797 +Ref: Changes<44>-Footnote-1481314 +Ref: Changes<44>-Footnote-2481369 +Ref: Changes<44>-Footnote-3481424 +Ref: Changes<44>-Footnote-4481479 +Ref: Changes<44>-Footnote-5481534 +Ref: Changes<44>-Footnote-6481576 +Ref: Changes<44>-Footnote-7481631 +Ref: Changes<44>-Footnote-8481686 +Node: v40 6 3481736 +Ref: history v40-6-3481826 +Ref: 1ad481826 +Node: Changes<45>481900 +Ref: history id253481959 +Ref: 1ae481959 +Ref: Changes<45>-Footnote-1482134 +Ref: Changes<45>-Footnote-2482189 +Node: v40 6 2482239 +Ref: history v40-6-2482329 +Ref: 1af482329 +Node: Changes<46>482403 +Ref: history id256482462 +Ref: 1b0482462 +Ref: Changes<46>-Footnote-1482627 +Node: v40 6 1482682 +Ref: history v40-6-1482772 +Ref: 1b1482772 +Node: Changes<47>482846 +Ref: history id258482905 +Ref: 1b2482905 +Ref: Changes<47>-Footnote-1483113 +Node: v40 6 0483168 +Ref: history v40-6-0483258 +Ref: 1b3483258 +Node: Deprecations483419 +Ref: history deprecations483499 +Ref: 1b4483499 +Ref: Deprecations-Footnote-1483666 +Node: Changes<48>483721 +Ref: history id261483835 +Ref: 1b5483835 +Ref: Changes<48>-Footnote-1484727 +Ref: Changes<48>-Footnote-2484782 +Ref: Changes<48>-Footnote-3484837 +Ref: Changes<48>-Footnote-4484892 +Node: Documentation changes<10>484947 +Ref: history id266485057 +Ref: 1b6485057 +Ref: Documentation changes<10>-Footnote-1485903 +Ref: Documentation changes<10>-Footnote-2485958 +Ref: Documentation changes<10>-Footnote-3486013 +Ref: Documentation changes<10>-Footnote-4486068 +Ref: Documentation changes<10>-Footnote-5486123 +Ref: Documentation changes<10>-Footnote-6486178 +Ref: Documentation changes<10>-Footnote-7486233 +Ref: Documentation changes<10>-Footnote-8486288 +Node: Misc<25>486343 +Ref: history id275486433 +Ref: 1b7486433 +Ref: Misc<25>-Footnote-1486731 +Ref: Misc<25>-Footnote-2486786 +Node: v40 5 0486841 +Ref: history v40-5-0486931 +Ref: 1b8486931 +Node: Changes<49>487057 +Ref: history id278487150 +Ref: 1b9487150 +Ref: Changes<49>-Footnote-1487566 +Ref: Changes<49>-Footnote-2487621 +Ref: Changes<49>-Footnote-3487676 +Ref: Changes<49>-Footnote-4487731 +Node: Documentation changes<11>487786 +Ref: history id283487879 +Ref: 1ba487879 +Ref: Documentation changes<11>-Footnote-1488059 +Node: v40 4 3488114 +Ref: history v40-4-3488204 +Ref: 1bb488204 +Node: Changes<50>488278 +Ref: history id285488337 +Ref: 1bc488337 +Ref: Changes<50>-Footnote-1488471 +Node: v40 4 2488526 +Ref: history v40-4-2488616 +Ref: 1bd488616 +Node: Misc<26>488684 +Ref: history id287488740 +Ref: 1be488740 +Ref: Misc<26>-Footnote-1488844 +Node: v40 4 1488899 +Ref: history v40-4-1488989 +Ref: 1bf488989 +Node: Changes<51>489063 +Ref: history id289489122 +Ref: 1c0489122 +Ref: Changes<51>-Footnote-1489239 +Node: v40 4 0489294 +Ref: history v40-4-0489384 +Ref: 1c1489384 +Node: Changes<52>489458 +Ref: history id291489517 +Ref: 1c2489517 +Ref: Changes<52>-Footnote-1489908 +Node: v40 3 0489963 +Ref: history v40-3-0490053 +Ref: 1c3490053 +Node: Changes<53>490145 +Ref: history id293490221 +Ref: 1c4490221 +Ref: Changes<53>-Footnote-1490721 +Ref: Changes<53>-Footnote-2490776 +Ref: Changes<53>-Footnote-3490831 +Ref: Changes<53>-Footnote-4490886 +Node: Misc<27>490941 +Ref: history id298491017 +Ref: 1c5491017 +Ref: Misc<27>-Footnote-1491134 +Node: v40 2 0491189 +Ref: history v40-2-0491279 +Ref: 1c6491279 +Node: Changes<54>491353 +Ref: history id300491412 +Ref: 1c7491412 +Ref: Changes<54>-Footnote-1491551 +Ref: Changes<54>-Footnote-2491606 +Node: v40 1 1491656 +Ref: history v40-1-1491746 +Ref: 1c8491746 +Node: Changes<55>491820 +Ref: history id303491879 +Ref: 1c9491879 +Ref: Changes<55>-Footnote-1492032 +Node: v40 1 0492087 +Ref: history v40-1-0492177 +Ref: 1ca492177 +Node: Changes<56>492269 +Ref: history id305492345 +Ref: 1cb492345 +Ref: Changes<56>-Footnote-1493127 +Ref: Changes<56>-Footnote-2493182 +Ref: Changes<56>-Footnote-3493237 +Ref: Changes<56>-Footnote-4493287 +Ref: Changes<56>-Footnote-5493342 +Ref: Changes<56>-Footnote-6493397 +Ref: Changes<56>-Footnote-7493452 +Ref: Changes<56>-Footnote-8493507 +Ref: Changes<56>-Footnote-9493562 +Ref: Changes<56>-Footnote-10493617 +Node: Misc<28>493673 +Ref: history id315493749 +Ref: 1cc493749 +Ref: Misc<28>-Footnote-1493906 +Node: v40 0 0493961 +Ref: history v40-0-0494051 +Ref: 1cd494051 +Node: Breaking Changes<12>494237 +Ref: history id317494325 +Ref: 1ce494325 +Ref: Breaking Changes<12>-Footnote-1494455 +Node: Changes<57>494510 +Ref: history id319494632 +Ref: 1cf494632 +Ref: Changes<57>-Footnote-1494884 +Ref: Changes<57>-Footnote-2494939 +Node: Documentation changes<12>494994 +Ref: history id322495104 +Ref: 1d0495104 +Ref: Documentation changes<12>-Footnote-1495382 +Ref: Documentation changes<12>-Footnote-2495437 +Ref: Documentation changes<12>-Footnote-3495492 +Node: Misc<29>495547 +Ref: history id326495637 +Ref: 1d1495637 +Ref: Misc<29>-Footnote-1495816 +Ref: Misc<29>-Footnote-2495871 +Node: v39 2 0495921 +Ref: history v39-2-0496011 +Ref: 1d2496011 +Node: Changes<58>496155 +Ref: history id329496248 +Ref: 1d3496248 +Ref: Changes<58>-Footnote-1496841 +Ref: Changes<58>-Footnote-2496896 +Ref: Changes<58>-Footnote-3496946 +Ref: Changes<58>-Footnote-4497001 +Ref: Changes<58>-Footnote-5497056 +Node: Documentation changes<13>497111 +Ref: history id334497221 +Ref: 1d4497221 +Ref: Documentation changes<13>-Footnote-1497608 +Ref: Documentation changes<13>-Footnote-2497663 +Ref: Documentation changes<13>-Footnote-3497718 +Ref: Documentation changes<13>-Footnote-4497773 +Node: Misc<30>497828 +Ref: history id339497918 +Ref: 1d5497918 +Ref: Misc<30>-Footnote-1498694 +Ref: Misc<30>-Footnote-2498749 +Ref: Misc<30>-Footnote-3498804 +Ref: Misc<30>-Footnote-4498859 +Ref: Misc<30>-Footnote-5498914 +Ref: Misc<30>-Footnote-6498969 +Ref: Misc<30>-Footnote-7499024 +Ref: Misc<30>-Footnote-8499074 +Node: v39 1 0499129 +Ref: history v39-1-0499219 +Ref: 1d6499219 +Ref: v39 1 0-Footnote-1499587 +Ref: v39 1 0-Footnote-2499642 +Ref: v39 1 0-Footnote-3499697 +Node: v39 0 1499752 +Ref: history v39-0-1499842 +Ref: 1d7499842 +Ref: v39 0 1-Footnote-1499999 +Node: v39 0 0500054 +Ref: history v39-0-0500144 +Ref: 1d8500144 +Ref: v39 0 0-Footnote-1500728 +Ref: v39 0 0-Footnote-2500783 +Node: v38 7 0500837 +Ref: history v38-7-0500927 +Ref: 1d9500927 +Ref: v38 7 0-Footnote-1501059 +Node: v38 6 1501114 +Ref: history v38-6-1501204 +Ref: 1da501204 +Ref: v38 6 1-Footnote-1501407 +Node: v38 6 0501462 +Ref: history v38-6-0501552 +Ref: 1db501552 +Ref: v38 6 0-Footnote-1501687 +Ref: v38 6 0-Footnote-2501742 +Node: v38 5 2501792 +Ref: history v38-5-2501882 +Ref: 1dc501882 +Ref: v38 5 2-Footnote-1502076 +Ref: v38 5 2-Footnote-2502131 +Node: v38 5 1502181 +Ref: history v38-5-1502271 +Ref: 1dd502271 +Ref: v38 5 1-Footnote-1502436 +Node: v38 5 0502491 +Ref: history v38-5-0502581 +Ref: 1de502581 +Ref: v38 5 0-Footnote-1502847 +Ref: v38 5 0-Footnote-2502902 +Node: v38 4 1502957 +Ref: history v38-4-1503047 +Ref: 1df503047 +Ref: v38 4 1-Footnote-1503195 +Node: v38 4 0503250 +Ref: history v38-4-0503340 +Ref: 1e0503340 +Ref: v38 4 0-Footnote-1503489 +Node: v38 3 0503544 +Ref: history v38-3-0503634 +Ref: 1e1503634 +Ref: v38 3 0-Footnote-1503900 +Ref: v38 3 0-Footnote-2503955 +Ref: v38 3 0-Footnote-3504005 +Node: v38 2 5504060 +Ref: history v38-2-5504150 +Ref: 1e2504150 +Ref: v38 2 5-Footnote-1504298 +Node: v38 2 4504353 +Ref: history v38-2-4504443 +Ref: 1e3504443 +Ref: v38 2 4-Footnote-1504593 +Node: v38 2 3504648 +Ref: history v38-2-3504738 +Ref: 1e4504738 +Node: v38 2 2504814 +Ref: history v38-2-2504904 +Ref: 1e5504904 +Ref: v38 2 2-Footnote-1505066 +Node: v38 2 1505121 +Ref: history v38-2-1505211 +Ref: 1e6505211 +Ref: v38 2 1-Footnote-1505372 +Node: v38 2 0505427 +Ref: history v38-2-0505517 +Ref: 1e7505517 +Ref: v38 2 0-Footnote-1505715 +Node: v38 1 0505770 +Ref: history v38-1-0505860 +Ref: 1e8505860 +Ref: v38 1 0-Footnote-1506034 +Node: v38 0 0506089 +Ref: history v38-0-0506179 +Ref: 1e9506179 +Ref: v38 0 0-Footnote-1506435 +Node: v37 0 0506489 +Ref: history v37-0-0506579 +Ref: 1ea506579 +Ref: v37 0 0-Footnote-1506765 +Node: v36 8 0506819 +Ref: history v36-8-0506909 +Ref: 1eb506909 +Ref: v36 8 0-Footnote-1507078 +Node: v36 7 3507133 +Ref: history v36-7-3507223 +Ref: 1ec507223 +Ref: v36 7 3-Footnote-1507355 +Node: v36 7 2507410 +Ref: history v36-7-2507500 +Ref: 1ed507500 +Ref: v36 7 2-Footnote-1507637 +Node: v36 7 1507691 +Ref: history v36-7-1507781 +Ref: 1ee507781 +Ref: v36 7 1-Footnote-1507950 +Node: v36 7 0508005 +Ref: history v36-7-0508095 +Ref: 1ef508095 +Ref: v36 7 0-Footnote-1508244 +Node: v36 6 1508299 +Ref: history v36-6-1508389 +Ref: 1f0508389 +Ref: v36 6 1-Footnote-1508667 +Ref: v36 6 1-Footnote-2508722 +Node: v36 6 0508776 +Ref: history v36-6-0508866 +Ref: 1f1508866 +Ref: v36 6 0-Footnote-1509157 +Ref: v36 6 0-Footnote-2509212 +Ref: v36 6 0-Footnote-3509262 +Node: v36 5 0509317 +Ref: history v36-5-0509407 +Ref: 1f2509407 +Ref: v36 5 0-Footnote-1509775 +Ref: v36 5 0-Footnote-2509829 +Node: v36 4 0509884 +Ref: history v36-4-0509974 +Ref: 1f3509974 +Ref: v36 4 0-Footnote-1510914 +Ref: v36 4 0-Footnote-2510969 +Ref: v36 4 0-Footnote-3511048 +Ref: v36 4 0-Footnote-4511103 +Ref: v36 4 0-Footnote-5511157 +Ref: v36 4 0-Footnote-6511212 +Ref: v36 4 0-Footnote-7511267 +Ref: v36 4 0-Footnote-8511322 +Node: v36 3 0511377 +Ref: history v36-3-0511467 +Ref: 1f4511467 +Ref: v36 3 0-Footnote-1511678 +Node: v36 2 7511733 +Ref: history v36-2-7511823 +Ref: 1f5511823 +Ref: v36 2 7-Footnote-1512047 +Ref: v36 2 7-Footnote-2512102 +Node: v36 2 6512157 +Ref: history v36-2-6512247 +Ref: 1f6512247 +Ref: v36 2 6-Footnote-1512417 +Node: v36 2 5512471 +Ref: history v36-2-5512561 +Ref: 1f7512561 +Ref: v36 2 5-Footnote-1512795 +Ref: v36 2 5-Footnote-2512850 +Ref: v36 2 5-Footnote-3512905 +Ref: v36 2 5-Footnote-4512960 +Node: v36 2 4513015 +Ref: history v36-2-4513105 +Ref: 1f8513105 +Ref: v36 2 4-Footnote-1513301 +Node: v36 2 3513356 +Ref: history v36-2-3513446 +Ref: 1f9513446 +Ref: v36 2 3-Footnote-1513575 +Node: v36 2 2513630 +Ref: history v36-2-2513720 +Ref: 1fa513720 +Ref: v36 2 2-Footnote-1513917 +Node: v36 2 1513972 +Ref: history v36-2-1514062 +Ref: 1fb514062 +Ref: v36 2 1-Footnote-1514243 +Ref: v36 2 1-Footnote-2514298 +Node: v36 2 0514353 +Ref: history v36-2-0514443 +Ref: 1fc514443 +Ref: v36 2 0-Footnote-1514962 +Ref: v36 2 0-Footnote-2515017 +Node: v36 1 1515072 +Ref: history v36-1-1515162 +Ref: 1fd515162 +Ref: v36 1 1-Footnote-1515411 +Ref: v36 1 1-Footnote-2515466 +Node: v36 1 0515521 +Ref: history v36-1-0515611 +Ref: 1fe515611 +Ref: v36 1 0-Footnote-1516114 +Ref: v36 1 0-Footnote-2516169 +Ref: v36 1 0-Footnote-3516206 +Node: v36 0 1516318 +Ref: history v36-0-1516408 +Ref: 1ff516408 +Ref: v36 0 1-Footnote-1516645 +Node: v36 0 0516700 +Ref: history v36-0-0516790 +Ref: 200516790 +Ref: v36 0 0-Footnote-1517225 +Node: v35 0 2517279 +Ref: history v35-0-2517369 +Ref: 201517369 +Ref: v35 0 2-Footnote-1517565 +Ref: v35 0 2-Footnote-2517620 +Ref: v35 0 2-Footnote-3517675 +Node: v35 0 1517716 +Ref: history v35-0-1517806 +Ref: 202517806 +Ref: v35 0 1-Footnote-1518149 +Ref: v35 0 1-Footnote-2518203 +Ref: v35 0 1-Footnote-3518258 +Ref: v35 0 1-Footnote-4518313 +Node: v35 0 0518367 +Ref: history v35-0-0518457 +Ref: 203518457 +Ref: v35 0 0-Footnote-1519057 +Node: v34 4 1519111 +Ref: history v34-4-1519201 +Ref: 204519201 +Ref: v34 4 1-Footnote-1519633 +Ref: v34 4 1-Footnote-2519688 +Ref: v34 4 1-Footnote-3519743 +Node: v34 4 0519797 +Ref: history v34-4-0519887 +Ref: 205519887 +Ref: v34 4 0-Footnote-1520276 +Ref: v34 4 0-Footnote-2520330 +Ref: v34 4 0-Footnote-3520384 +Node: v34 3 3520439 +Ref: history v34-3-3520529 +Ref: 206520529 +Ref: v34 3 3-Footnote-1520788 +Ref: v34 3 3-Footnote-2520842 +Node: v34 3 2520896 +Ref: history v34-3-2520986 +Ref: 207520986 +Ref: v34 3 2-Footnote-1521182 +Node: v34 3 1521236 +Ref: history v34-3-1521326 +Ref: 208521326 +Ref: v34 3 1-Footnote-1521585 +Ref: v34 3 1-Footnote-2521639 +Node: v34 3 0521693 +Ref: history v34-3-0521783 +Ref: 209521783 +Ref: v34 3 0-Footnote-1522077 +Ref: v34 3 0-Footnote-2522131 +Node: v34 2 0522185 +Ref: history v34-2-0522275 +Ref: 20a522275 +Ref: v34 2 0-Footnote-1522621 +Ref: v34 2 0-Footnote-2522675 +Ref: v34 2 0-Footnote-3522729 +Node: v34 1 1522779 +Ref: history v34-1-1522869 +Ref: 20b522869 +Ref: v34 1 1-Footnote-1523045 +Ref: v34 1 1-Footnote-2523099 +Node: v34 1 0523153 +Ref: history v34-1-0523243 +Ref: 20c523243 +Ref: v34 1 0-Footnote-1523444 +Node: v34 0 3523498 +Ref: history v34-0-3523588 +Ref: 20d523588 +Ref: v34 0 3-Footnote-1523816 +Node: v34 0 2523870 +Ref: history v34-0-2523960 +Ref: 20e523960 +Ref: v34 0 2-Footnote-1524194 +Ref: v34 0 2-Footnote-2524248 +Node: v34 0 1524302 +Ref: history v34-0-1524392 +Ref: 20f524392 +Ref: v34 0 1-Footnote-1524511 +Node: v34 0 0524565 +Ref: history v34-0-0524655 +Ref: 210524655 +Ref: v34 0 0-Footnote-1526296 +Ref: v34 0 0-Footnote-2526350 +Ref: v34 0 0-Footnote-3526404 +Node: v33 1 1526459 +Ref: history v33-1-1526549 +Ref: 211526549 +Ref: v33 1 1-Footnote-1526714 +Node: v33 1 0526768 +Ref: history v33-1-0526858 +Ref: 212526858 +Ref: v33 1 0-Footnote-1527237 +Node: v33 0 0527286 +Ref: history v33-0-0527376 +Ref: 213527376 +Ref: v33 0 0-Footnote-1527673 +Ref: v33 0 0-Footnote-2527727 +Node: v32 3 1527785 +Ref: history v32-3-1527875 +Ref: 214527875 +Ref: v32 3 1-Footnote-1528049 +Node: v32 3 0528103 +Ref: history v32-3-0528193 +Ref: 215528193 +Ref: v32 3 0-Footnote-1528387 +Node: v32 2 0528441 +Ref: history v32-2-0528531 +Ref: 216528531 +Ref: v32 2 0-Footnote-1528766 +Ref: v32 2 0-Footnote-2528820 +Node: v32 1 3528872 +Ref: history v32-1-3528962 +Ref: 217528962 +Ref: v32 1 3-Footnote-1529156 +Node: v32 1 2529210 +Ref: history v32-1-2529300 +Ref: 218529300 +Ref: v32 1 2-Footnote-1529491 +Node: v32 1 1529545 +Ref: history v32-1-1529635 +Ref: 219529635 +Ref: v32 1 1-Footnote-1530069 +Ref: v32 1 1-Footnote-2530123 +Node: v32 1 0530177 +Ref: history v32-1-0530267 +Ref: 21a530267 +Ref: v32 1 0-Footnote-1530473 +Node: v32 0 0530527 +Ref: history v32-0-0530617 +Ref: 21b530617 +Ref: v32 0 0-Footnote-1530953 +Ref: v32 0 0-Footnote-2531007 +Ref: v32 0 0-Footnote-3531061 +Ref: v32 0 0-Footnote-4531115 +Node: v31 0 1531169 +Ref: history v31-0-1531259 +Ref: 21c531259 +Ref: v31 0 1-Footnote-1531514 +Node: v31 0 0531568 +Ref: history v31-0-0531658 +Ref: 21d531658 +Ref: v31 0 0-Footnote-1532258 +Ref: v31 0 0-Footnote-2532312 +Ref: v31 0 0-Footnote-3532366 +Ref: v31 0 0-Footnote-4532421 +Node: v30 4 0532475 +Ref: history v30-4-0532565 +Ref: 21e532565 +Ref: v30 4 0-Footnote-1533101 +Node: v30 3 0533155 +Ref: history v30-3-0533245 +Ref: 21f533245 +Ref: v30 3 0-Footnote-1533424 +Ref: v30 3 0-Footnote-2533478 +Ref: v30 3 0-Footnote-3533532 +Node: v30 2 1533641 +Ref: history v30-2-1533731 +Ref: 220533731 +Ref: v30 2 1-Footnote-1533908 +Node: v30 2 0533962 +Ref: history v30-2-0534052 +Ref: 221534052 +Ref: v30 2 0-Footnote-1534181 +Ref: v30 2 0-Footnote-2534235 +Node: v30 1 0534301 +Ref: history v30-1-0534391 +Ref: 222534391 +Ref: v30 1 0-Footnote-1534726 +Ref: v30 1 0-Footnote-2534780 +Node: v30 0 0534834 +Ref: history v30-0-0534924 +Ref: 223534924 +Ref: v30 0 0-Footnote-1535337 +Ref: v30 0 0-Footnote-2535391 +Ref: v30 0 0-Footnote-3535445 +Ref: v30 0 0-Footnote-4535499 +Node: v29 0 1535553 +Ref: history v29-0-1535643 +Ref: 224535643 +Ref: v29 0 1-Footnote-1536037 +Node: v29 0 0536091 +Ref: history v29-0-0536181 +Ref: 225536181 +Ref: v29 0 0-Footnote-1536409 +Ref: v29 0 0-Footnote-2536463 +Node: v28 8 0536520 +Ref: history v28-8-0536610 +Ref: 226536610 +Ref: v28 8 0-Footnote-1536995 +Ref: v28 8 0-Footnote-2537049 +Ref: v28 8 0-Footnote-3537103 +Node: v28 7 1537157 +Ref: history v28-7-1537247 +Ref: 227537247 +Ref: v28 7 1-Footnote-1537491 +Ref: v28 7 1-Footnote-2537545 +Ref: v28 7 1-Footnote-3537599 +Node: v28 7 0537653 +Ref: history v28-7-0537743 +Ref: 228537743 +Ref: v28 7 0-Footnote-1538138 +Ref: v28 7 0-Footnote-2538192 +Ref: v28 7 0-Footnote-3538246 +Ref: v28 7 0-Footnote-4538300 +Ref: v28 7 0-Footnote-5538354 +Node: v28 6 1538408 +Ref: history v28-6-1538498 +Ref: 229538498 +Ref: v28 6 1-Footnote-1538630 +Node: v28 6 0538684 +Ref: history v28-6-0538774 +Ref: 22a538774 +Ref: v28 6 0-Footnote-1539116 +Node: v28 5 0539170 +Ref: history v28-5-0539260 +Ref: 22b539260 +Ref: v28 5 0-Footnote-1539763 +Ref: v28 5 0-Footnote-2539817 +Ref: v28 5 0-Footnote-3539871 +Ref: v28 5 0-Footnote-4539925 +Node: v28 4 0539979 +Ref: history v28-4-0540069 +Ref: 22c540069 +Ref: v28 4 0-Footnote-1540635 +Ref: v28 4 0-Footnote-2540689 +Ref: v28 4 0-Footnote-3540739 +Ref: v28 4 0-Footnote-4540793 +Ref: v28 4 0-Footnote-5540847 +Ref: v28 4 0-Footnote-6540901 +Node: v28 3 0540955 +Ref: history v28-3-0541045 +Ref: 22d541045 +Ref: v28 3 0-Footnote-1541385 +Ref: v28 3 0-Footnote-2541439 +Ref: v28 3 0-Footnote-3541493 +Ref: v28 3 0-Footnote-4541543 +Node: v28 1 0541598 +Ref: history v28-1-0541688 +Ref: 22e541688 +Ref: v28 1 0-Footnote-1541808 +Node: v28 0 0541862 +Ref: history v28-0-0541952 +Ref: 22f541952 +Ref: v28 0 0-Footnote-1542530 +Ref: v28 0 0-Footnote-2542584 +Ref: v28 0 0-Footnote-3542638 +Node: v27 3 1542692 +Ref: history v27-3-1542782 +Ref: 230542782 +Ref: v27 3 1-Footnote-1543165 +Node: v27 3 0543219 +Ref: history v27-3-0543309 +Ref: 231543309 +Ref: v27 3 0-Footnote-1543613 +Ref: v27 3 0-Footnote-2543667 +Ref: v27 3 0-Footnote-3543717 +Node: v27 2 0543771 +Ref: history v27-2-0543861 +Ref: 232543861 +Ref: v27 2 0-Footnote-1544104 +Ref: v27 2 0-Footnote-2544158 +Node: v27 1 2544212 +Ref: history v27-1-2544302 +Ref: 233544302 +Ref: v27 1 2-Footnote-1544428 +Ref: v27 1 2-Footnote-2544482 +Node: v27 1 1544536 +Ref: history v27-1-1544626 +Ref: 234544626 +Ref: v27 1 1-Footnote-1544744 +Node: v27 1 0544798 +Ref: history v27-1-0544888 +Ref: 235544888 +Node: v27 0 0545039 +Ref: history v27-0-0545129 +Ref: 236545129 +Ref: v27 0 0-Footnote-1545944 +Ref: v27 0 0-Footnote-2545998 +Ref: v27 0 0-Footnote-3546052 +Node: v26 1 1546106 +Ref: history v26-1-1546196 +Ref: 237546196 +Node: v26 1 0546426 +Ref: history v26-1-0546516 +Ref: 238546516 +Ref: v26 1 0-Footnote-1546783 +Ref: v26 1 0-Footnote-2546837 +Node: v26 0 0546878 +Ref: history v26-0-0546968 +Ref: 239546968 +Ref: v26 0 0-Footnote-1547558 +Ref: v26 0 0-Footnote-2547612 +Ref: v26 0 0-Footnote-3547654 +Ref: v26 0 0-Footnote-4547708 +Ref: v26 0 0-Footnote-5547762 +Ref: v26 0 0-Footnote-6547816 +Node: v25 4 0547870 +Ref: history v25-4-0547960 +Ref: 23a547960 +Node: v25 3 0548519 +Ref: history v25-3-0548609 +Ref: 23b548609 +Ref: v25 3 0-Footnote-1549040 +Ref: v25 3 0-Footnote-2549094 +Ref: v25 3 0-Footnote-3549148 +Ref: v25 3 0-Footnote-4549202 +Ref: v25 3 0-Footnote-5549256 +Ref: v25 3 0-Footnote-6549310 +Ref: v25 3 0-Footnote-7549364 +Ref: v25 3 0-Footnote-8549418 +Ref: v25 3 0-Footnote-9549472 +Ref: v25 3 0-Footnote-10549526 +Node: v25 2 0549581 +Ref: history v25-2-0549671 +Ref: 23c549671 +Ref: v25 2 0-Footnote-1549860 +Ref: v25 2 0-Footnote-2549914 +Node: v25 1 6549968 +Ref: history v25-1-6550058 +Ref: 23d550058 +Ref: v25 1 6-Footnote-1550274 +Node: v25 1 5550328 +Ref: history v25-1-5550418 +Ref: 23e550418 +Ref: v25 1 5-Footnote-1550569 +Ref: v25 1 5-Footnote-2550623 +Node: v25 1 4550677 +Ref: history v25-1-4550767 +Ref: 23f550767 +Ref: v25 1 4-Footnote-1551033 +Ref: v25 1 4-Footnote-2551087 +Ref: v25 1 4-Footnote-3551141 +Ref: v25 1 4-Footnote-4551195 +Node: v25 1 3551249 +Ref: history v25-1-3551339 +Ref: 240551339 +Ref: v25 1 3-Footnote-1551571 +Ref: v25 1 3-Footnote-2551625 +Ref: v25 1 3-Footnote-3551679 +Ref: v25 1 3-Footnote-4551733 +Ref: v25 1 3-Footnote-5551787 +Node: v25 1 2551841 +Ref: history v25-1-2551931 +Ref: 241551931 +Ref: v25 1 2-Footnote-1552370 +Ref: v25 1 2-Footnote-2552424 +Ref: v25 1 2-Footnote-3552478 +Node: v25 1 1552532 +Ref: history v25-1-1552622 +Ref: 242552622 +Ref: v25 1 1-Footnote-1552846 +Ref: v25 1 1-Footnote-2552900 +Node: v25 1 0552954 +Ref: history v25-1-0553044 +Ref: 243553044 +Ref: v25 1 0-Footnote-1553424 +Node: v25 0 2553478 +Ref: history v25-0-2553568 +Ref: 244553568 +Ref: v25 0 2-Footnote-1553742 +Node: v25 0 1553796 +Ref: history v25-0-1553886 +Ref: 245553886 +Ref: v25 0 1-Footnote-1554197 +Ref: v25 0 1-Footnote-2554251 +Ref: v25 0 1-Footnote-3554305 +Ref: v25 0 1-Footnote-4554359 +Ref: v25 0 1-Footnote-5554413 +Node: v25 0 0554467 +Ref: history v25-0-0554557 +Ref: 246554557 +Ref: v25 0 0-Footnote-1555394 +Node: v24 3 1555448 +Ref: history v24-3-1555538 +Ref: 247555538 +Ref: v24 3 1-Footnote-1555871 +Ref: v24 3 1-Footnote-2555925 +Ref: v24 3 1-Footnote-3555979 +Ref: v24 3 1-Footnote-4556033 +Node: v24 3 0556087 +Ref: history v24-3-0556177 +Ref: 248556177 +Ref: v24 3 0-Footnote-1556474 +Node: v24 2 1556528 +Ref: history v24-2-1556618 +Ref: 249556618 +Ref: v24 2 1-Footnote-1556787 +Node: v24 2 0556841 +Ref: history v24-2-0556931 +Ref: 24a556931 +Ref: v24 2 0-Footnote-1557071 +Node: v24 1 1557125 +Ref: history v24-1-1557215 +Ref: 24b557215 +Ref: v24 1 1-Footnote-1557351 +Ref: v24 1 1-Footnote-2557405 +Ref: v24 1 1-Footnote-3557459 +Node: v24 1 0557513 +Ref: history v24-1-0557603 +Ref: 24c557603 +Ref: v24 1 0-Footnote-1557911 +Ref: v24 1 0-Footnote-2557965 +Ref: v24 1 0-Footnote-3558019 +Ref: v24 1 0-Footnote-4558073 +Ref: v24 1 0-Footnote-5558127 +Ref: v24 1 0-Footnote-6558181 +Ref: v24 1 0-Footnote-7558235 +Ref: v24 1 0-Footnote-8558289 +Node: v24 0 3558343 +Ref: history v24-0-3558433 +Ref: 24d558433 +Ref: v24 0 3-Footnote-1558675 +Ref: v24 0 3-Footnote-2558729 +Ref: v24 0 3-Footnote-3558783 +Ref: v24 0 3-Footnote-4558837 +Ref: v24 0 3-Footnote-5558891 +Ref: v24 0 3-Footnote-6558945 +Ref: v24 0 3-Footnote-7558999 +Ref: v24 0 3-Footnote-8559053 +Node: v24 0 2559107 +Ref: history v24-0-2559197 +Ref: 24e559197 +Node: v24 0 1559348 +Ref: history v24-0-1559438 +Ref: 24f559438 +Ref: v24 0 1-Footnote-1559610 +Ref: v24 0 1-Footnote-2559664 +Node: v24 0 0559718 +Ref: history v24-0-0559808 +Ref: 250559808 +Ref: v24 0 0-Footnote-1560221 +Node: v23 2 1560275 +Ref: history v23-2-1560365 +Ref: 251560365 +Ref: v23 2 1-Footnote-1560585 +Node: v23 1 0560639 +Ref: history v23-1-0560729 +Ref: 252560729 +Ref: v23 1 0-Footnote-1560877 +Node: v23 0 0560931 +Ref: history v23-0-0561021 +Ref: 253561021 +Ref: v23 0 0-Footnote-1561407 +Ref: v23 0 0-Footnote-2561461 +Node: v22 0 5561515 +Ref: history v22-0-5561605 +Ref: 254561605 +Ref: v22 0 5-Footnote-1561792 +Node: v22 0 4561846 +Ref: history v22-0-4561936 +Ref: 255561936 +Ref: v22 0 4-Footnote-1562104 +Node: v22 0 3562158 +Ref: history v22-0-3562248 +Ref: 256562248 +Ref: v22 0 3-Footnote-1562444 +Node: v22 0 2562498 +Ref: history v22-0-2562588 +Ref: 257562588 +Ref: v22 0 2-Footnote-1562731 +Node: v22 0 1562785 +Ref: history v22-0-1562875 +Ref: 258562875 +Ref: v22 0 1-Footnote-1563102 +Node: v22 0 0563156 +Ref: history v22-0-0563246 +Ref: 259563246 +Ref: v22 0 0-Footnote-1563537 +Ref: v22 0 0-Footnote-2563591 +Node: v21 2 2563649 +Ref: history v21-2-2563739 +Ref: 25a563739 +Node: v21 2 1563822 +Ref: history v21-2-1563912 +Ref: 25b563912 +Ref: v21 2 1-Footnote-1564063 +Node: v21 2 0564117 +Ref: history v21-2-0564207 +Ref: 25c564207 +Ref: v21 2 0-Footnote-1564393 +Node: v21 1 0564447 +Ref: history v21-1-0564537 +Ref: 25d564537 +Ref: v21 1 0-Footnote-1564855 +Node: v21 0 0564909 +Ref: history v21-0-0565000 +Ref: 25e565000 +Node: v20 10 0565278 +Ref: history v20-10-0565369 +Ref: 25f565369 +Ref: v20 10 0-Footnote-1565961 +Ref: v20 10 0-Footnote-2566015 +Ref: v20 10 0-Footnote-3566069 +Ref: v20 10 0-Footnote-4566135 +Node: v20 9 0566189 +Ref: history v20-9-0566280 +Ref: 260566280 +Ref: v20 9 0-Footnote-1566482 +Ref: v20 9 0-Footnote-2566536 +Node: v20 8 1566590 +Ref: history v20-8-1566680 +Ref: 261566680 +Ref: v20 8 1-Footnote-1566877 +Node: v20 8 0566931 +Ref: history v20-8-0567021 +Ref: 262567021 +Ref: v20 8 0-Footnote-1567243 +Node: v20 7 0567297 +Ref: history v20-7-0567387 +Ref: 263567387 +Ref: v20 7 0-Footnote-1567846 +Ref: v20 7 0-Footnote-2567900 +Ref: v20 7 0-Footnote-3567954 +Node: v20 6 8568008 +Ref: history v20-6-8568098 +Ref: 264568098 +Ref: v20 6 8-Footnote-1568290 +Node: v20 6 7568344 +Ref: history v20-6-7568434 +Ref: 265568434 +Ref: v20 6 7-Footnote-1568600 +Node: v20 6 6568654 +Ref: history v20-6-6568744 +Ref: 266568744 +Ref: v20 6 6-Footnote-1568933 +Ref: v20 6 6-Footnote-2568987 +Ref: v20 6 6-Footnote-3569037 +Node: v20 6 0569103 +Ref: history v20-6-0569190 +Ref: 267569190 +Ref: v20 6 0-Footnote-1569634 +Ref: v20 6 0-Footnote-2569681 +Node: 20 5569708 +Ref: history id610569792 +Ref: 268569792 +Ref: 20 5-Footnote-1570090 +Ref: 20 5-Footnote-2570153 +Node: 20 4570207 +Ref: history id612570290 +Ref: 269570290 +Ref: 20 4-Footnote-1570812 +Ref: 20 4-Footnote-2570866 +Ref: 20 4-Footnote-3570909 +Node: 20 3 1570955 +Ref: history id613571038 +Ref: 26a571038 +Ref: 20 3 1-Footnote-1571331 +Ref: 20 3 1-Footnote-2571385 +Node: 20 3571448 +Ref: history id614571533 +Ref: 26b571533 +Ref: 20 3-Footnote-1571959 +Ref: 20 3-Footnote-2572022 +Node: 20 2 2572085 +Ref: history id616572170 +Ref: 26c572170 +Ref: 20 2 2-Footnote-1572365 +Node: 20 2 1572419 +Ref: history id617572504 +Ref: 26d572504 +Ref: 20 2 1-Footnote-1572681 +Ref: 20 2 1-Footnote-2572735 +Node: 20 2572801 +Ref: history id618572886 +Ref: 26e572886 +Ref: 20 2-Footnote-1573761 +Ref: 20 2-Footnote-2573824 +Ref: 20 2-Footnote-3573874 +Ref: 20 2-Footnote-4573924 +Ref: 20 2-Footnote-5573990 +Ref: 20 2-Footnote-6574040 +Ref: 20 2-Footnote-7574094 +Ref: 20 2-Footnote-8574157 +Ref: 20 2-Footnote-9574220 +Ref: 20 2-Footnote-10574283 +Ref: 20 2-Footnote-11574334 +Node: 20 1 1574389 +Ref: history id624574472 +Ref: 26f574472 +Node: 20 1574603 +Ref: history id625574686 +Ref: 270574686 +Ref: 20 1-Footnote-1574876 +Node: 20 0574999 +Ref: history id626575080 +Ref: 271575080 +Ref: 20 0-Footnote-1575358 +Node: 19 7575412 +Ref: history id627575495 +Ref: 272575495 +Ref: 19 7-Footnote-1575683 +Ref: 19 7-Footnote-2575775 +Ref: 19 7-Footnote-3575867 +Node: 19 6 2575959 +Ref: history id628576044 +Ref: 273576044 +Ref: 19 6 2-Footnote-1576265 +Node: 19 6 1576319 +Ref: history id629576404 +Ref: 274576404 +Ref: 19 6 1-Footnote-1576737 +Node: 19 6576791 +Ref: history id630576874 +Ref: 275576874 +Ref: 19 6-Footnote-1577682 +Node: 19 5577736 +Ref: history id631577819 +Ref: 276577819 +Ref: 19 5-Footnote-1578127 +Ref: 19 5-Footnote-2578181 +Ref: 19 5-Footnote-3578235 +Node: 19 4 1578298 +Ref: history id632578381 +Ref: 277578381 +Ref: 19 4 1-Footnote-1578607 +Node: 19 4578661 +Ref: history id634578744 +Ref: 278578744 +Ref: 19 4-Footnote-1579296 +Ref: 19 4-Footnote-2579350 +Ref: 19 4-Footnote-3579407 +Ref: 19 4-Footnote-4579461 +Ref: 19 4-Footnote-5579515 +Node: 19 3579578 +Ref: history id635579659 +Ref: 279579659 +Ref: 19 3-Footnote-1579913 +Node: 19 2579967 +Ref: history id636580050 +Ref: 27a580050 +Ref: 19 2-Footnote-1580292 +Ref: 19 2-Footnote-2580355 +Node: 19 1 1580418 +Ref: history id637580501 +Ref: 27b580501 +Ref: 19 1 1-Footnote-1580877 +Ref: 19 1 1-Footnote-2580931 +Node: 19 1580985 +Ref: history id638581068 +Ref: 27c581068 +Ref: 19 1-Footnote-1581401 +Ref: 19 1-Footnote-2581455 +Node: 19 0581509 +Ref: history id639581592 +Ref: 27d581592 +Ref: 19 0-Footnote-1581796 +Node: 18 8 1581850 +Ref: history id640581933 +Ref: 27e581933 +Ref: 18 8 1-Footnote-1582223 +Node: 18 8582277 +Ref: history id641582362 +Ref: 27f582362 +Ref: 18 8-Footnote-1582773 +Ref: 18 8-Footnote-2582827 +Ref: 18 8-Footnote-3582881 +Node: 18 7 1582935 +Ref: history id642583018 +Ref: 280583018 +Ref: 18 7 1-Footnote-1583201 +Ref: 18 7 1-Footnote-2583255 +Node: 18 7583309 +Ref: history id644583394 +Ref: 281583394 +Ref: 18 7-Footnote-1584144 +Ref: 18 7-Footnote-2584207 +Ref: 18 7-Footnote-3584270 +Ref: 18 7-Footnote-4584324 +Ref: 18 7-Footnote-5584387 +Ref: 18 7-Footnote-6584437 +Ref: 18 7-Footnote-7584491 +Node: 18 6 1584554 +Ref: history id646584637 +Ref: 282584637 +Ref: 18 6 1-Footnote-1584818 +Node: 18 6584872 +Ref: history id647584955 +Ref: 283584955 +Ref: 18 6-Footnote-1585201 +Node: 18 5585255 +Ref: history id648585336 +Ref: 284585336 +Ref: 18 5-Footnote-1585707 +Ref: 18 5-Footnote-2585799 +Node: 18 4585891 +Ref: history id649585974 +Ref: 285585974 +Ref: 18 4-Footnote-1586149 +Node: 18 3 2586203 +Ref: history id650586288 +Ref: 286586288 +Ref: 18 3 2-Footnote-1586469 +Node: 18 3 1586511 +Ref: history id651586596 +Ref: 287586596 +Ref: 18 3 1-Footnote-1586735 +Node: 18 3586789 +Ref: history id652586872 +Ref: 288586872 +Ref: 18 3-Footnote-1587762 +Ref: 18 3-Footnote-2587825 +Node: 18 2587879 +Ref: history id653587960 +Ref: 289587960 +Ref: 18 2-Footnote-1588109 +Node: 18 1588163 +Ref: history id654588246 +Ref: 28a588246 +Ref: 18 1-Footnote-1588363 +Node: 18 0 1588429 +Ref: history id655588512 +Ref: 28b588512 +Ref: 18 0 1-Footnote-1588636 +Node: 18 0588690 +Ref: history id656588775 +Ref: 28c588775 +Ref: 18 0-Footnote-1589841 +Ref: 18 0-Footnote-2589895 +Ref: 18 0-Footnote-3589949 +Node: 17 1 1590012 +Ref: history id657590095 +Ref: 28d590095 +Ref: 17 1 1-Footnote-1590283 +Node: 17 1590395 +Ref: history id658590478 +Ref: 28e590478 +Ref: 17 1-Footnote-1590642 +Node: 17 0590696 +Ref: history id659590777 +Ref: 28f590777 +Ref: 17 0-Footnote-1591085 +Ref: 17 0-Footnote-2591139 +Node: 16 0591193 +Ref: history id660591274 +Ref: 290591274 +Ref: 16 0-Footnote-1591627 +Ref: 16 0-Footnote-2591690 +Ref: 16 0-Footnote-3591753 +Node: 15 2591816 +Ref: history id661591897 +Ref: 291591897 +Ref: 15 2-Footnote-1592142 +Node: 15 1592196 +Ref: history id662592277 +Ref: 292592277 +Ref: 15 1-Footnote-1592477 +Ref: 15 1-Footnote-2592543 +Node: 15 0592595 +Ref: history id663592678 +Ref: 293592678 +Ref: 15 0-Footnote-1593278 +Ref: 15 0-Footnote-2593341 +Node: 14 3 1593397 +Ref: history id664593480 +Ref: 294593480 +Ref: 14 3 1-Footnote-1593773 +Ref: 14 3 1-Footnote-2593827 +Ref: 14 3 1-Footnote-3593877 +Node: 14 3593931 +Ref: history id666594014 +Ref: 295594014 +Ref: 14 3-Footnote-1594268 +Node: 14 2594322 +Ref: history id667594405 +Ref: 296594405 +Ref: 14 2-Footnote-1594665 +Node: 14 1 1594719 +Ref: history id668594802 +Ref: 297594802 +Ref: 14 1 1-Footnote-1595014 +Node: 14 1595068 +Ref: history id669595151 +Ref: 298595151 +Ref: 14 1-Footnote-1595336 +Node: 14 0595399 +Ref: history id670595482 +Ref: 299595482 +Ref: 14 0-Footnote-1596311 +Ref: 14 0-Footnote-2596374 +Node: 13 0 2596428 +Ref: history id671596513 +Ref: 29a596513 +Ref: 13 0 2-Footnote-1596718 +Node: 13 0 1596772 +Ref: history id672596857 +Ref: 29b596857 +Node: 13 0597037 +Ref: history id673597120 +Ref: 29c597120 +Ref: 13 0-Footnote-1597500 +Ref: 13 0-Footnote-2597554 +Node: 12 4597617 +Ref: history id674597698 +Ref: 29d597698 +Ref: 12 4-Footnote-1597904 +Node: 12 3597967 +Ref: history id676598048 +Ref: 29e598048 +Ref: 12 3-Footnote-1598369 +Node: 12 2598423 +Ref: history id677598504 +Ref: 29f598504 +Ref: 12 2-Footnote-1598956 +Ref: 12 2-Footnote-2599010 +Node: 12 1599064 +Ref: history id678599147 +Ref: 2a0599147 +Ref: 12 1-Footnote-1599315 +Node: 12 0 5599378 +Ref: history id679599463 +Ref: 2a1599463 +Ref: 12 0 5-Footnote-1599687 +Ref: 12 0 5-Footnote-2599741 +Node: 12 0 4599795 +Ref: history id681599882 +Ref: 2a2599882 +Ref: 12 0 4-Footnote-1600020 +Node: 12 0 3600074 +Ref: history id682600161 +Ref: 2a3600161 +Node: 12 0 2600292 +Ref: history id683600379 +Ref: 2a4600379 +Ref: 12 0 2-Footnote-1600559 +Node: 12 0 1600613 +Ref: history id684600698 +Ref: 2a5600698 +Node: 12 0600904 +Ref: history id685600989 +Ref: 2a6600989 +Ref: 12 0-Footnote-1601548 +Node: 11 3 1601602 +Ref: history id686601685 +Ref: 2a7601685 +Ref: 11 3 1-Footnote-1601869 +Node: 11 3601923 +Ref: history id687602006 +Ref: 2a8602006 +Node: 11 2602529 +Ref: history id688602610 +Ref: 2a9602610 +Ref: 11 2-Footnote-1602772 +Node: 11 1602820 +Ref: history id689602901 +Ref: 2aa602901 +Ref: 11 1-Footnote-1603425 +Ref: 11 1-Footnote-2603479 +Node: 11 0603533 +Ref: history id690603616 +Ref: 2ab603616 +Ref: 11 0-Footnote-1603847 +Ref: 11 0-Footnote-2603910 +Ref: 11 0-Footnote-3603976 +Node: 10 2 1604026 +Ref: history id692604109 +Ref: 2ac604109 +Ref: 10 2 1-Footnote-1604250 +Node: 10 2604304 +Ref: history id693604387 +Ref: 2ad604387 +Node: 10 1604758 +Ref: history id694604841 +Ref: 2ae604841 +Ref: 10 1-Footnote-1605119 +Node: 10 0 1605173 +Ref: history id695605256 +Ref: 2af605256 +Ref: 10 0 1-Footnote-1605401 +Node: 10 0605455 +Ref: history id696605537 +Ref: 2b0605537 +Ref: 10 0-Footnote-1606091 +Ref: 10 0-Footnote-2606145 +Ref: 10 0-Footnote-3606194 +Node: 9 1606248 +Ref: history id697606329 +Ref: 2b1606329 +Ref: 9 1-Footnote-1606458 +Node: 9 0 1606550 +Ref: history id698606630 +Ref: 2b2606630 +Ref: 9 0 1-Footnote-1606795 +Node: 9 0606849 +Ref: history id699606929 +Ref: 2b3606929 +Ref: 9 0-Footnote-1607158 +Node: 8 4607212 +Ref: history id700607290 +Ref: 2b4607290 +Ref: 8 4-Footnote-1607432 +Node: 8 3607495 +Ref: history id701607575 +Ref: 2b5607575 +Ref: 8 3-Footnote-1607781 +Node: 8 2 1607835 +Ref: history id702607915 +Ref: 2b6607915 +Ref: 8 2 1-Footnote-1608105 +Node: 8 2608159 +Ref: history id703608239 +Ref: 2b7608239 +Ref: 8 2-Footnote-1608394 +Node: 8 1608456 +Ref: history id704608536 +Ref: 2b8608536 +Ref: 8 1-Footnote-1608902 +Node: 8 0 4608952 +Ref: history id706609034 +Ref: 2b9609034 +Ref: 8 0 4-Footnote-1609495 +Node: 8 0 3609549 +Ref: history id707609633 +Ref: 2ba609633 +Ref: 8 0 3-Footnote-1609795 +Node: 8 0 2609849 +Ref: history id709609933 +Ref: 2bb609933 +Ref: 8 0 2-Footnote-1610117 +Node: 8 0 1610171 +Ref: history id711610253 +Ref: 2bc610253 +Ref: 8 0 1-Footnote-1610487 +Node: 8 0610541 +Ref: history id713610621 +Ref: 2bd610621 +Ref: 8 0-Footnote-1611092 +Ref: 8 0-Footnote-2611142 +Ref: 8 0-Footnote-3611277 +Node: 7 0611319 +Ref: history id715611397 +Ref: 2be611397 +Ref: 7 0-Footnote-1612340 +Ref: 7 0-Footnote-2612393 +Ref: 7 0-Footnote-3612447 +Node: 6 1612501 +Ref: history id717612581 +Ref: 2bf612581 +Ref: 6 1-Footnote-1612804 +Node: 6 0 2612858 +Ref: history id719612940 +Ref: 2c0612940 +Ref: 6 0 2-Footnote-1613140 +Ref: 6 0 2-Footnote-2613194 +Node: 6 0 1613248 +Ref: history id721613330 +Ref: 2c1613330 +Ref: 6 0 1-Footnote-1613533 +Node: 6 0613587 +Ref: history id722613667 +Ref: 2c2613667 +Ref: 6 0-Footnote-1615185 +Ref: 6 0-Footnote-2615239 +Ref: 6 0-Footnote-3615301 +Ref: 6 0-Footnote-4615363 +Ref: 6 0-Footnote-5615425 +Ref: 6 0-Footnote-6615479 +Node: 5 8615533 +Ref: history id724615611 +Ref: 2c3615611 +Ref: 5 8-Footnote-1615886 +Node: 5 7615940 +Ref: history id725616018 +Ref: 2c4616018 +Ref: 5 7-Footnote-1616549 +Ref: 5 7-Footnote-2616603 +Node: 5 6616657 +Ref: history id726616737 +Ref: 2c5616737 +Ref: 5 6-Footnote-1616941 +Node: 5 5 1616995 +Ref: history id727617075 +Ref: 2c6617075 +Ref: 5 5 1-Footnote-1617214 +Node: 5 5617268 +Ref: history id728617350 +Ref: 2c7617350 +Ref: 5 5-Footnote-1617610 +Node: 5 4 2617664 +Ref: history id730617746 +Ref: 2c8617746 +Ref: 5 4 2-Footnote-1617910 +Node: 5 4 1617964 +Ref: history id731618046 +Ref: 2c9618046 +Ref: 5 4 1-Footnote-1618225 +Node: 5 4618266 +Ref: history id732618346 +Ref: 2ca618346 +Ref: 5 4-Footnote-1618898 +Node: 5 3618952 +Ref: history id734619030 +Ref: 2cb619030 +Ref: 5 3-Footnote-1619295 +Node: 5 2619349 +Ref: history id735619427 +Ref: 2cc619427 +Ref: 5 2-Footnote-1619782 +Node: 5 1619855 +Ref: history id736619935 +Ref: 2cd619935 +Ref: 5 1-Footnote-1620185 +Ref: 5 1-Footnote-2620239 +Node: 5 0 2620293 +Ref: history id737620375 +Ref: 2ce620375 +Ref: 5 0 2-Footnote-1620497 +Node: 5 0 1620551 +Ref: history id738620633 +Ref: 2cf620633 +Node: 5 0620836 +Ref: history id739620938 +Ref: 2d0620938 +Ref: 5 0-Footnote-1621185 +Node: 3 7 1 and 3 8 1 and 4 0 1621239 +Ref: history and-3-8-1-and-4-0-1621339 +Ref: 2d1621339 +Ref: 3 7 1 and 3 8 1 and 4 0 1-Footnote-1621645 +Ref: 3 7 1 and 3 8 1 and 4 0 1-Footnote-2621699 +Node: 4 0621753 +Ref: history id741621853 +Ref: 2d2621853 +Ref: 4 0-Footnote-1622086 +Node: 3 8622140 +Ref: history id742622218 +Ref: 2d3622218 +Ref: 3 8-Footnote-1622380 +Node: 3 7622434 +Ref: history id743622512 +Ref: 2d4622512 +Ref: 3 7-Footnote-1622672 +Node: 3 6622726 +Ref: history id744622806 +Ref: 2d5622806 +Ref: 3 6-Footnote-1622975 +Node: 3 5 2623029 +Ref: history id745623111 +Ref: 2d6623111 +Ref: 3 5 2-Footnote-1623343 +Node: 3 5 1623397 +Ref: history id747623479 +Ref: 2d7623479 +Ref: 3 5 1-Footnote-1623658 +Node: 3 5623712 +Ref: history id748623794 +Ref: 2d8623794 +Ref: 3 5-Footnote-1624280 +Ref: 3 5-Footnote-2624334 +Ref: 3 5-Footnote-3624388 +Node: 3 4 4624442 +Ref: history id750624524 +Ref: 2d9624524 +Ref: 3 4 4-Footnote-1624731 +Node: 3 4 3624785 +Ref: history id751624869 +Ref: 2da624869 +Ref: 3 4 3-Footnote-1625005 +Node: 3 4 2625059 +Ref: history id752625143 +Ref: 2db625143 +Ref: 3 4 2-Footnote-1625300 +Node: 3 4 1625354 +Ref: history id754625436 +Ref: 2dc625436 +Ref: 3 4 1-Footnote-1625600 +Node: 3 4625654 +Ref: history id755625734 +Ref: 2dd625734 +Ref: 3 4-Footnote-1626168 +Ref: 3 4-Footnote-2626222 +Node: 3 3626276 +Ref: history id756626354 +Ref: 2de626354 +Node: 3 2626459 +Ref: history id757626537 +Ref: 2df626537 +Ref: 3 2-Footnote-1626823 +Ref: 3 2-Footnote-2626885 +Ref: 3 2-Footnote-3626939 +Node: 3 1626993 +Ref: history id758627073 +Ref: 2e0627073 +Ref: 3 1-Footnote-1627314 +Node: 3 0 2627368 +Ref: history id759627450 +Ref: 2e1627450 +Node: 3 0 1627527 +Ref: history id760627609 +Ref: 2e2627609 +Ref: 3 0 1-Footnote-1627841 +Node: 3 0627895 +Ref: history id761627975 +Ref: 2e3627975 +Ref: 3 0-Footnote-1629765 +Ref: 3 0-Footnote-2629819 +Ref: 3 0-Footnote-3629872 +Ref: 3 0-Footnote-4629926 +Ref: 3 0-Footnote-5629978 +Ref: 3 0-Footnote-6630031 +Ref: 3 0-Footnote-7630093 +Node: 2 2630147 +Ref: history id762630227 +Ref: 2e4630227 +Ref: 2 2-Footnote-1630583 +Ref: 2 2-Footnote-2630637 +Node: 2 1 2630691 +Ref: history id764630773 +Ref: 2e5630773 +Ref: 2 1 2-Footnote-1630967 +Node: 2 1 1631021 +Ref: history id765631103 +Ref: 2e6631103 +Ref: 2 1 1-Footnote-1631276 +Node: 2 1631330 +Ref: history id767631412 +Ref: 2e7631412 +Ref: 2 1-Footnote-1631670 +Ref: 2 1-Footnote-2631724 +Node: 2 0 2631778 +Ref: history id768631860 +Ref: 2e8631860 +Node: 2 0 1632061 +Ref: history id769632143 +Ref: 2e9632143 +Ref: 2 0 1-Footnote-1632284 +Node: 2 0632338 +Ref: history id770632420 +Ref: 2ea632420 +Ref: 2 0-Footnote-1633030 +Ref: 2 0-Footnote-2633084 +Node: 1 4 2633137 +Ref: history id771633219 +Ref: 2eb633219 +Ref: 1 4 2-Footnote-1633385 +Node: 1 4 1633439 +Ref: history id772633521 +Ref: 2ec633521 +Ref: 1 4 1-Footnote-1635078 +Ref: 1 4 1-Footnote-2635132 +Ref: 1 4 1-Footnote-3635186 +Node: 1 4635240 +Ref: history id773635322 +Ref: 2ed635322 +Ref: 1 4-Footnote-1635680 +Ref: 1 4-Footnote-2635733 +Node: 1 3 2635795 +Ref: history id774635877 +Ref: 2ee635877 +Ref: 1 3 2-Footnote-1636016 +Node: 1 3 1636069 +Ref: history id775636151 +Ref: 2ef636151 +Node: 1 3636257 +Ref: history id776636337 +Ref: 2f0636337 +Ref: 1 3-Footnote-1636674 +Ref: 1 3-Footnote-2636716 +Node: 1 2636779 +Ref: history id777636859 +Ref: 2f1636859 +Ref: 1 2-Footnote-1637442 +Ref: 1 2-Footnote-2637495 +Ref: 1 2-Footnote-3637548 +Node: 1 1 7637602 +Ref: history id778637684 +Ref: 2f2637684 +Ref: 1 1 7-Footnote-1638096 +Ref: 1 1 7-Footnote-2638153 +Node: 1 1 6638206 +Ref: history id779638290 +Ref: 2f3638290 +Ref: 1 1 6-Footnote-1638507 +Node: 1 1 5638564 +Ref: history id780638648 +Ref: 2f4638648 +Ref: 1 1 5-Footnote-1638786 +Node: 1 1 4638839 +Ref: history id781638923 +Ref: 2f5638923 +Ref: 1 1 4-Footnote-1639059 +Node: 1 1 3639112 +Ref: history id782639196 +Ref: 2f6639196 +Node: 1 1 2639273 +Ref: history id783639357 +Ref: 2f7639357 +Ref: 1 1 2-Footnote-1639549 +Node: 1 1 1639602 +Ref: history id785639684 +Ref: 2f8639684 +Ref: 1 1 1-Footnote-1639980 +Ref: 1 1 1-Footnote-2640033 +Node: 1 1640086 +Ref: history id786640166 +Ref: 2f9640166 +Ref: 1 1-Footnote-1640463 +Ref: 1 1-Footnote-2640516 +Ref: 1 1-Footnote-3640573 +Node: 1 0640626 +Ref: history id787640706 +Ref: 2fa640706 +Ref: 1 0-Footnote-1641904 +Ref: 1 0-Footnote-2641957 +Ref: 1 0-Footnote-3642003 +Ref: 1 0-Footnote-4642056 +Ref: 1 0-Footnote-5642109 +Node: Backward-Incompatible Changes642162 +Ref: history backward-incompatible-changes642235 +Ref: 2fb642235 +Ref: Backward-Incompatible Changes-Footnote-1643158 +Ref: Backward-Incompatible Changes-Footnote-2643211 +Node: 0 9 8643264 +Ref: history id790643346 +Ref: 2fc643346 +Ref: 0 9 8-Footnote-1643489 +Node: 0 9 7643542 +Ref: history id791643626 +Ref: 2fd643626 +Ref: 0 9 7-Footnote-1644008 +Ref: 0 9 7-Footnote-2644061 +Node: 0 9 6644114 +Ref: history id792644198 +Ref: 2fe644198 +Ref: 0 9 6-Footnote-1644378 +Node: 0 9 5644431 +Ref: history id793644515 +Ref: 2ff644515 +Ref: 0 9 5-Footnote-1644676 +Node: 0 9 4644718 +Ref: history id794644802 +Ref: 300644802 +Ref: 0 9 4-Footnote-1645007 +Node: 0 9 3645060 +Ref: history id795645144 +Ref: 301645144 +Ref: 0 9 3-Footnote-1645291 +Node: 0 9 2645344 +Ref: history id796645428 +Ref: 302645428 +Ref: 0 9 2-Footnote-1645603 +Node: 0 9 1645656 +Ref: history id798645738 +Ref: 303645738 +Ref: 0 9 1-Footnote-1645961 +Node: 0 9646018 +Ref: history id799646098 +Ref: 304646098 +Node: 0 8646218 +Ref: history id800646298 +Ref: 305646298 +Node: 0 7 8646421 +Ref: history id801646503 +Ref: 306646503 +Ref: 0 7 8-Footnote-1646647 +Node: 0 7 7646704 +Ref: history id802646788 +Ref: 307646788 +Ref: 0 7 7-Footnote-1646998 +Ref: 0 7 7-Footnote-2647055 +Node: 0 7 6647108 +Ref: history id804647192 +Ref: 308647192 +Ref: 0 7 6-Footnote-1647339 +Node: 0 7 5647396 +Ref: history id806647480 +Ref: 309647480 +Ref: 0 7 5-Footnote-1647922 +Ref: 0 7 5-Footnote-2647975 +Node: 0 7 4648032 +Ref: history id808648116 +Ref: 30a648116 +Ref: 0 7 4-Footnote-1648260 +Node: 0 7 3648313 +Ref: history id809648397 +Ref: 30b648397 +Ref: 0 7 3-Footnote-1648623 +Node: 0 7 2648675 +Ref: history id810648759 +Ref: 30c648759 +Ref: 0 7 2-Footnote-1649011 +Ref: 0 7 2-Footnote-2649064 +Node: 0 7 1649117 +Ref: history id811649199 +Ref: 30d649199 +Ref: 0 7 1-Footnote-1649336 +Node: 0 7649388 +Ref: history id812649470 +Ref: 30e649470 +Ref: 0 7-Footnote-1650291 +Node: 0 7b4650341 +Ref: history b4650424 +Ref: 30f650424 +Ref: 0 7b4-Footnote-1650536 +Node: 0 6 49650588 +Ref: history id815650674 +Ref: 310650674 +Ref: 0 6 49-Footnote-1650946 +Node: 0 6 48651003 +Ref: history id817651090 +Ref: 311651090 +Node: 0 6 47651233 +Ref: history id818651320 +Ref: 312651320 +Node: 0 6 46651456 +Ref: history id819651543 +Ref: 313651543 +Ref: 0 6 46-Footnote-1651795 +Node: 0 6 45651852 +Ref: history id821651939 +Ref: 314651939 +Ref: 0 6 45-Footnote-1652175 +Node: 0 6 44652232 +Ref: history id822652319 +Ref: 315652319 +Node: 0 6 43652464 +Ref: history id823652551 +Ref: 316652551 +Ref: 0 6 43-Footnote-1652720 +Node: 0 6 42652777 +Ref: history id824652864 +Ref: 317652864 +Ref: 0 6 42-Footnote-1653154 +Ref: 0 6 42-Footnote-2653211 +Node: 0 6 41653264 +Ref: history id826653351 +Ref: 318653351 +Ref: 0 6 41-Footnote-1653736 +Node: 0 6 40653792 +Ref: history id827653879 +Ref: 319653879 +Ref: 0 6 40-Footnote-1654062 +Node: 0 6 39654119 +Ref: history id828654206 +Ref: 31a654206 +Ref: 0 6 39-Footnote-1654702 +Node: 0 6 38654759 +Ref: history id830654846 +Ref: 31b654846 +Ref: 0 6 38-Footnote-1655007 +Node: 0 6 37655064 +Ref: history id831655151 +Ref: 31c655151 +Ref: 0 6 37-Footnote-1655501 +Ref: 0 6 37-Footnote-2655558 +Node: 0 6 36655635 +Ref: history id832655722 +Ref: 31d655722 +Ref: 0 6 36-Footnote-1656127 +Ref: 0 6 36-Footnote-2656189 +Node: 0 6 35656244 +Ref: history id833656331 +Ref: 31e656331 +Ref: 0 6 35-Footnote-1656722 +Node: 0 6 34656779 +Ref: history id834656866 +Ref: 31f656866 +Ref: 0 6 34-Footnote-1657008 +Node: 0 6 33657065 +Ref: history id835657152 +Ref: 320657152 +Ref: 0 6 33-Footnote-1657693 +Ref: 0 6 33-Footnote-2657750 +Node: 0 6 32657807 +Ref: history id837657894 +Ref: 321657894 +Ref: 0 6 32-Footnote-1658198 +Node: 0 6 31658255 +Ref: history id838658342 +Ref: 322658342 +Ref: 0 6 31-Footnote-1659665 +Ref: 0 6 31-Footnote-2659722 +Ref: 0 6 31-Footnote-3659779 +Ref: 0 6 31-Footnote-4659820 +Ref: 0 6 31-Footnote-5659861 +Ref: 0 6 31-Footnote-6659918 +Ref: 0 6 31-Footnote-7659972 +Node: 0 6 30660029 +Ref: history id840660116 +Ref: 323660116 +Ref: 0 6 30-Footnote-1660322 +Node: 0 6 29660379 +Ref: history id841660466 +Ref: 324660466 +Ref: 0 6 29-Footnote-1662658 +Ref: 0 6 29-Footnote-2662720 +Ref: 0 6 29-Footnote-3662777 +Ref: 0 6 29-Footnote-4662830 +Ref: 0 6 29-Footnote-5662883 +Ref: 0 6 29-Footnote-6662940 +Ref: 0 6 29-Footnote-7662997 +Ref: 0 6 29-Footnote-8663054 +Ref: 0 6 29-Footnote-9663111 +Ref: 0 6 29-Footnote-10663168 +Ref: 0 6 29-Footnote-11663226 +Ref: 0 6 29-Footnote-12663284 +Ref: 0 6 29-Footnote-13663342 +Ref: 0 6 29-Footnote-14663400 +Ref: 0 6 29-Footnote-15663458 +Ref: 0 6 29-Footnote-16663516 +Ref: 0 6 29-Footnote-17663574 +Ref: 0 6 29-Footnote-18663632 +Ref: 0 6 29-Footnote-19663690 +Node: 0 6 28663748 +Ref: history id845663835 +Ref: 325663835 +Ref: 0 6 28-Footnote-1664191 +Ref: 0 6 28-Footnote-2664248 +Node: 0 6 27664305 +Ref: history id847664392 +Ref: 326664392 +Ref: 0 6 27-Footnote-1664811 +Ref: 0 6 27-Footnote-2664865 +Node: 0 6 26664922 +Ref: history id849665009 +Ref: 327665009 +Ref: 0 6 26-Footnote-1665410 +Ref: 0 6 26-Footnote-2665467 +Node: 0 6 25665524 +Ref: history id850665611 +Ref: 328665611 +Ref: 0 6 25-Footnote-1666330 +Ref: 0 6 25-Footnote-2666387 +Ref: 0 6 25-Footnote-3666444 +Ref: 0 6 25-Footnote-4666501 +Ref: 0 6 25-Footnote-5666558 +Ref: 0 6 25-Footnote-6666615 +Node: 0 6 24666672 +Ref: history id851666759 +Ref: 329666759 +Ref: 0 6 24-Footnote-1666898 +Node: 0 6 23666955 +Ref: history id852667042 +Ref: 32a667042 +Ref: 0 6 23-Footnote-1667811 +Ref: 0 6 23-Footnote-2667868 +Ref: 0 6 23-Footnote-3667925 +Ref: 0 6 23-Footnote-4667982 +Ref: 0 6 23-Footnote-5668039 +Ref: 0 6 23-Footnote-6668096 +Ref: 0 6 23-Footnote-7668153 +Ref: 0 6 23-Footnote-8668210 +Ref: 0 6 23-Footnote-9668267 +Ref: 0 6 23-Footnote-10668324 +Ref: 0 6 23-Footnote-11668382 +Node: 0 6 21668440 +Ref: history id854668527 +Ref: 32b668527 +Ref: 0 6 21-Footnote-1668657 +Node: 0 6 20668714 +Ref: history id856668801 +Ref: 32c668801 +Ref: 0 6 20-Footnote-1669122 +Ref: 0 6 20-Footnote-2669179 +Ref: 0 6 20-Footnote-3669236 +Node: 0 6 19669293 +Ref: history id857669380 +Ref: 32d669380 +Ref: 0 6 19-Footnote-1669562 +Node: 0 6 18669619 +Ref: history id858669706 +Ref: 32e669706 +Ref: 0 6 18-Footnote-1669870 +Ref: 0 6 18-Footnote-2669927 +Node: 0 6 17669984 +Ref: history id859670071 +Ref: 32f670071 +Ref: 0 6 17-Footnote-1670661 +Ref: 0 6 17-Footnote-2670718 +Ref: 0 6 17-Footnote-3670775 +Node: 0 6 16670832 +Ref: history id861670919 +Ref: 330670919 +Ref: 0 6 16-Footnote-1671310 +Ref: 0 6 16-Footnote-2671367 +Ref: 0 6 16-Footnote-3671424 +Ref: 0 6 16-Footnote-4671481 +Node: 0 6 15671538 +Ref: history id862671625 +Ref: 331671625 +Ref: 0 6 15-Footnote-1671882 +Node: 0 6 14671939 +Ref: history id863672026 +Ref: 332672026 +Ref: 0 6 14-Footnote-1672454 +Ref: 0 6 14-Footnote-2672511 +Ref: 0 6 14-Footnote-3672568 +Ref: 0 6 14-Footnote-4672625 +Node: 0 6 13672682 +Ref: history id865672769 +Ref: 333672769 +Ref: 0 6 13-Footnote-1673137 +Ref: 0 6 13-Footnote-2673194 +Ref: 0 6 13-Footnote-3673251 +Node: 0 6 12673308 +Ref: history id866673395 +Ref: 334673395 +Ref: 0 6 12-Footnote-1673531 +Node: 0 6 11673588 +Ref: history id867673675 +Ref: 335673675 +Ref: 0 6 11-Footnote-1674633 +Ref: 0 6 11-Footnote-2674689 +Ref: 0 6 11-Footnote-3674745 +Ref: 0 6 11-Footnote-4674802 +Ref: 0 6 11-Footnote-5674859 +Ref: 0 6 11-Footnote-6674916 +Ref: 0 6 11-Footnote-7674973 +Ref: 0 6 11-Footnote-8675030 +Ref: 0 6 11-Footnote-9675087 +Ref: 0 6 11-Footnote-10675144 +Node: 0 6 10675202 +Ref: history id868675288 +Ref: 336675288 +Node: 0 6 9675488 +Ref: history id869675573 +Ref: 337675573 +Ref: 0 6 9-Footnote-1677104 +Ref: 0 6 9-Footnote-2677160 +Ref: 0 6 9-Footnote-3677216 +Ref: 0 6 9-Footnote-4677272 +Ref: 0 6 9-Footnote-5677328 +Ref: 0 6 9-Footnote-6677384 +Ref: 0 6 9-Footnote-7677440 +Ref: 0 6 9-Footnote-8677496 +Ref: 0 6 9-Footnote-9677552 +Ref: 0 6 9-Footnote-10677609 +Ref: 0 6 9-Footnote-11677666 +Ref: 0 6 9-Footnote-12677724 +Ref: 0 6 9-Footnote-13677782 +Node: 0 6 8677840 +Ref: history id871677924 +Ref: 338677924 +Node: 0 6 7678079 +Ref: history id872678163 +Ref: 339678163 +Ref: 0 6 7-Footnote-1679567 +Ref: 0 6 7-Footnote-2679623 +Ref: 0 6 7-Footnote-3679679 +Ref: 0 6 7-Footnote-4679735 +Ref: 0 6 7-Footnote-5679791 +Ref: 0 6 7-Footnote-6679847 +Node: 0 6 6679903 +Ref: history id873679987 +Ref: 33a679987 +Node: 0 6 5680136 +Ref: history id874680220 +Ref: 33b680220 +Ref: 0 6 5-Footnote-1680900 +Ref: 0 6 5-Footnote-2680956 +Ref: 0 6 5-Footnote-3681012 +Ref: 0 6 5-Footnote-4681062 +Node: 0 6 4681112 +Ref: history id875681196 +Ref: 33c681196 +Ref: 0 6 4-Footnote-1681608 +Ref: 0 6 4-Footnote-2681664 +Node: 0 6 3681720 +Ref: history id876681804 +Ref: 33d681804 +Node: setuptools681885 +Ref: history setuptools681963 +Ref: 33e681963 +Node: bootstrapping682074 +Ref: history bootstrapping682152 +Ref: 33f682152 +Node: 0 6 2682269 +Ref: history id877682353 +Ref: 340682353 +Node: setuptools<2>682465 +Ref: history id878682549 +Ref: 341682549 +Ref: setuptools<2>-Footnote-1683115 +Ref: setuptools<2>-Footnote-2683165 +Ref: setuptools<2>-Footnote-3683221 +Ref: setuptools<2>-Footnote-4683271 +Ref: setuptools<2>-Footnote-5683320 +Node: bootstrapping<2>683370 +Ref: history id879683454 +Ref: 342683454 +Ref: bootstrapping<2>-Footnote-1683804 +Ref: bootstrapping<2>-Footnote-2683860 +Ref: bootstrapping<2>-Footnote-3683916 +Node: 0 6 1683966 +Ref: history id880684048 +Ref: 343684048 +Node: setuptools<3>684160 +Ref: history id881684244 +Ref: 344684244 +Ref: setuptools<3>-Footnote-1684831 +Ref: setuptools<3>-Footnote-2684887 +Ref: setuptools<3>-Footnote-3684943 +Ref: setuptools<3>-Footnote-4684993 +Ref: setuptools<3>-Footnote-5685043 +Node: bootstrapping<3>685099 +Ref: history id882685183 +Ref: 345685183 +Ref: bootstrapping<3>-Footnote-1685453 +Node: 0 6685509 +Ref: history id883685591 +Ref: 346685591 +Node: setuptools<4>685699 +Ref: history id884685778 +Ref: 347685778 +Ref: setuptools<4>-Footnote-1686417 +Ref: setuptools<4>-Footnote-2686473 +Ref: setuptools<4>-Footnote-3686529 +Ref: setuptools<4>-Footnote-4686584 +Ref: setuptools<4>-Footnote-5686639 +Ref: setuptools<4>-Footnote-6686694 +Node: pkg_resources686749 +Ref: history pkg-resources686849 +Ref: 348686849 +Ref: pkg_resources-Footnote-1687509 +Ref: pkg_resources-Footnote-2687561 +Ref: pkg_resources-Footnote-3687617 +Ref: pkg_resources-Footnote-4687672 +Ref: pkg_resources-Footnote-5687727 +Node: easy_install687782 +Ref: history easy-install687860 +Ref: 349687860 +Ref: easy_install-Footnote-1688012 +Node: 0 6c9688067 +Ref: history c9688149 +Ref: 34a688149 +Node: 0 6c7691270 +Ref: history c7691354 +Ref: 34b691354 +Node: 0 6c6692024 +Ref: history c6692108 +Ref: 34c692108 +Node: 0 6c5693672 +Ref: history c5693756 +Ref: 34d693756 +Node: 0 6c4694230 +Ref: history c4694314 +Ref: 34e694314 +Node: 0 6c3696864 +Ref: history c3696948 +Ref: 34f696948 +Node: 0 6c2697218 +Ref: history c2697302 +Ref: 350697302 +Node: 0 6c1698423 +Ref: history c1698507 +Ref: 351698507 +Node: 0 6b4699742 +Ref: history id889699826 +Ref: 352699826 +Node: 0 6b3701372 +Ref: history b3701456 +Ref: 353701456 +Node: 0 6b2702307 +Ref: history b2702391 +Ref: 354702391 +Node: 0 6b1702899 +Ref: history b1702984 +Ref: 355702984 +Node: 0 6a11704131 +Ref: history a11704217 +Ref: 356704217 +Node: 0 6a10706884 +Ref: history a10706970 +Ref: 357706970 +Node: 0 6a9709771 +Ref: history a9709856 +Ref: 358709856 +Node: 0 6a8713618 +Ref: history a8713702 +Ref: 359713702 +Node: 0 6a7715945 +Ref: history a7716029 +Ref: 35a716029 +Node: 0 6a6716148 +Ref: history a6716232 +Ref: 35b716232 +Node: 0 6a5717371 +Ref: history a5717455 +Ref: 35c717455 +Node: 0 6a3717573 +Ref: history a3717657 +Ref: 35d717657 +Node: 0 6a2718537 +Ref: history a2718621 +Ref: 35e718621 +Node: 0 6a1719514 +Ref: history a1719599 +Ref: 35f719599 +Ref: 0 6a1-Footnote-1724998 +Node: 0 5a12725097 +Ref: history a12725183 +Ref: 360725183 +Node: 0 5a11725778 +Ref: history id890725865 +Ref: 361725865 +Node: 0 5a10726046 +Ref: history id891726132 +Ref: 362726132 +Node: 0 5a9726390 +Ref: history id892726475 +Ref: 363726475 +Node: 0 5a8729999 +Ref: history id893730083 +Ref: 364730083 +Node: 0 5a7731866 +Ref: history id894731950 +Ref: 365731950 +Node: 0 5a6732165 +Ref: history id895732249 +Ref: 366732249 +Node: 0 5a5733234 +Ref: history id896733318 +Ref: 367733318 +Node: 0 5a4734902 +Ref: history a4734986 +Ref: 368734986 +Node: 0 5a3737029 +Ref: history id897737113 +Ref: 369737113 +Node: 0 5a2737359 +Ref: history id898737443 +Ref: 36a737443 +Node: 0 5a1737539 +Ref: history id899737623 +Ref: 36b737623 +Node: 0 4a4738466 +Ref: history id900738550 +Ref: 36c738550 +Node: 0 4a3738878 +Ref: history id901738962 +Ref: 36d738962 +Node: 0 4a2739321 +Ref: history id902739405 +Ref: 36e739405 +Node: 0 4a1741512 +Ref: history id903741596 +Ref: 36f741596 +Node: 0 3a4741756 +Ref: history id904741840 +Ref: 370741840 +Node: 0 3a3742016 +Ref: history id905742100 +Ref: 371742100 +Node: 0 3a2742702 +Ref: history id906742786 +Ref: 372742786 +Node: 0 3a1743289 +Ref: history id907743359 +Ref: 373743359 +Node: Credits743412 +Ref: history credits743496 +Ref: 374743496 +Node: Index745380 + +End Tag Table + + +Local Variables: +coding: utf-8 +End: diff --git a/assets/info/sicp.info b/assets/info/sicp.info new file mode 100644 index 00000000..beb889f7 --- /dev/null +++ b/assets/info/sicp.info @@ -0,0 +1,33394 @@ +This is sicp.info, produced by makeinfo version 4.8 from sicp.texi. + +INFO-DIR-SECTION The Algorithmic Language Scheme +START-INFO-DIR-ENTRY +* SICP: (sicp). Structure and Interpretation of Computer Programs +END-INFO-DIR-ENTRY + + +File: sicp.info, Node: Top, Next: UTF, Prev: (dir), Up: (dir) + +Structure and Interpretation of Computer Programs +================================================= + +Second Edition +by Harold Abelson and Gerald Jay Sussman, with Julie Sussman +foreword by Alan J. Perlis +(C) 1996 Massachusetts Institute of Technology + +Unofficial Texinfo Format version 2.neilvandyke4 (January 10, 2007) + +* Menu: + +* UTF:: Unofficial Texinfo Format +* Dedication:: Dedication +* Foreword:: Foreword +* Preface:: Preface to the Second Edition +* Preface 1e:: Preface to the First Edition +* Acknowledgements:: Acknowledgements +* Chapter 1:: Building Abstractions with Procedures +* Chapter 2:: Building Abstractions with Data +* Chapter 3:: Modularity, Objects, and State +* Chapter 4:: Metalinguistic Abstraction +* Chapter 5:: Computing with Register Machines +* References:: References +* Index:: Index + + --- The Detailed Node Listing --- + +Programming in Lisp + +* 1-1:: The Elements of Programming +* 1-2:: Procedures and the Processes They Generate +* 1-3:: Formulating Abstractions with Higher-Order Procedures + +The Elements of Programming + +* 1-1-1:: Expressions +* 1-1-2:: Naming and the Environment +* 1-1-3:: Evaluating Combinations +* 1-1-4:: Compound Procedures +* 1-1-5:: The Substitution Model for Procedure Application +* 1-1-6:: Conditional Expressions and Predicates +* 1-1-7:: Example: Square Roots by Newton's Method +* 1-1-8:: Procedures as Black-Box Abstractions + +Procedures and the Processes They Generate + +* 1-2-1:: Linear Recursion and Iteration +* 1-2-2:: Tree Recursion +* 1-2-3:: Orders of Growth +* 1-2-4:: Exponentiation +* 1-2-5:: Greatest Common Divisors +* 1-2-6:: Example: Testing for Primality + +Formulating Abstractions with Higher-Order Procedures + +* 1-3-1:: Procedures as Arguments +* 1-3-2:: Constructing Procedures Using `Lambda' +* 1-3-3:: Procedures as General Methods +* 1-3-4:: Procedures as Returned Values + +Building Abstractions with Data + +* 2-1:: Introduction to Data Abstraction +* 2-2:: Hierarchical Data and the Closure Property +* 2-3:: Symbolic Data +* 2-4:: Multiple Representations for Abstract Data +* 2-5:: Systems with Generic Operations + +Introduction to Data Abstraction + +* 2-1-1:: Example: Arithmetic Operations for Rational Numbers +* 2-1-2:: Abstraction Barriers +* 2-1-3:: What Is Meant by Data? +* 2-1-4:: Extended Exercise: Interval Arithmetic + +Hierarchical Data and the Closure Property + +* 2-2-1:: Representing Sequences +* 2-2-2:: Hierarchical Structures +* 2-2-3:: Sequences as Conventional Interfaces +* 2-2-4:: Example: A Picture Language + +Symbolic Data + +* 2-3-1:: Quotation +* 2-3-2:: Example: Symbolic Differentiation +* 2-3-3:: Example: Representing Sets +* 2-3-4:: Example: Huffman Encoding Trees + +Multiple Representations for Abstract Data + +* 2-4-1:: Representations for Complex Numbers +* 2-4-2:: Tagged data +* 2-4-3:: Data-Directed Programming and Additivity + +Systems with Generic Operations + +* 2-5-1:: Generic Arithmetic Operations +* 2-5-2:: Combining Data of Different Types +* 2-5-3:: Example: Symbolic Algebra + +Modularity, Objects, and State + +* 3-1:: Assignment and Local State +* 3-2:: The Environment Model of Evaluation +* 3-3:: Modeling with Mutable Data +* 3-4:: Concurrency: Time Is of the Essence +* 3-5:: Streams + +Assignment and Local State + +* 3-1-1:: Local State Variables +* 3-1-2:: The Benefits of Introducing Assignment +* 3-1-3:: The Costs of Introducing Assignment + +The Environment Model of Evaluation + +* 3-2-1:: The Rules for Evaluation +* 3-2-2:: Applying Simple Procedures +* 3-2-3:: Frames as the Repository of Local State +* 3-2-4:: Internal Definitions + +Modeling with Mutable Data + +* 3-3-1:: Mutable List Structure +* 3-3-2:: Representing Queues +* 3-3-3:: Representing Tables +* 3-3-4:: A Simulator for Digital Circuits +* 3-3-5:: Propagation of Constraints + +Concurrency: Time Is of the Essence + +* 3-4-1:: The Nature of Time in Concurrent Systems +* 3-4-2:: Mechanisms for Controlling Concurrency + +Streams + +* 3-5-1:: Streams Are Delayed Lists +* 3-5-2:: Infinite Streams +* 3-5-3:: Exploiting the Stream Paradigm +* 3-5-4:: Streams and Delayed Evaluation +* 3-5-5:: Modularity of Functional Programs and Modularity of + Objects + +Metalinguistic Abstraction + +* 4-1:: The Metacircular Evaluator +* 4-2:: Variations on a Scheme -- Lazy Evaluation +* 4-3:: Variations on a Scheme -- Nondeterministic Computing +* 4-4:: Logic Programming + +The Metacircular Evaluator + +* 4-1-1:: The Core of the Evaluator +* 4-1-2:: Representing Expressions +* 4-1-3:: Evaluator Data Structures +* 4-1-4:: Running the Evaluator as a Program +* 4-1-5:: Data as Programs +* 4-1-6:: Internal Definitions +* 4-1-7:: Separating Syntactic Analysis from Execution + +Variations on a Scheme -- Lazy Evaluation + +* 4-2-1:: Normal Order and Applicative Order +* 4-2-2:: An Interpreter with Lazy Evaluation +* 4-2-3:: Streams as Lazy Lists + +Variations on a Scheme -- Nondeterministic Computing + +* 4-3-1:: Amb and Search +* 4-3-2:: Examples of Nondeterministic Programs +* 4-3-3:: Implementing the `Amb' Evaluator + +Logic Programming + +* 4-4-1:: Deductive Information Retrieval +* 4-4-2:: How the Query System Works +* 4-4-3:: Is Logic Programming Mathematical Logic? +* 4-4-4:: Implementing the Query System + +Implementing the Query System + +* 4-4-4-1:: The Driver Loop and Instantiation +* 4-4-4-2:: The Evaluator +* 4-4-4-3:: Finding Assertions by Pattern Matching +* 4-4-4-4:: Rules and Unification +* 4-4-4-5:: Maintaining the Data Base +* 4-4-4-6:: Stream Operations +* 4-4-4-7:: Query Syntax Procedures +* 4-4-4-8:: Frames and Bindings + +Computing with Register Machines + +* 5-1:: Designing Register Machines +* 5-2:: A Register-Machine Simulator +* 5-3:: Storage Allocation and Garbage Collection +* 5-4:: The Explicit-Control Evaluator +* 5-5:: Compilation + +Designing Register Machines + +* 5-1-1:: A Language for Describing Register Machines +* 5-1-2:: Abstraction in Machine Design +* 5-1-3:: Subroutines +* 5-1-4:: Using a Stack to Implement Recursion +* 5-1-5:: Instruction Summary + +A Register-Machine Simulator + +* 5-2-1:: The Machine Model +* 5-2-2:: The Assembler +* 5-2-3:: Generating Execution Procedures for Instructions +* 5-2-4:: Monitoring Machine Performance + +Storage Allocation and Garbage Collection + +* 5-3-1:: Memory as Vectors +* 5-3-2:: Maintaining the Illusion of Infinite Memory + +Registers and operations + +* 5-4-1:: The Core of the Explicit-Control Evaluator +* 5-4-2:: Sequence Evaluation and Tail Recursion +* 5-4-3:: Conditionals, Assignments, and Definitions +* 5-4-4:: Running the Evaluator + +An overview of the compiler + +* 5-5-1:: Structure of the Compiler +* 5-5-2:: Compiling Expressions +* 5-5-3:: Compiling Combinations +* 5-5-4:: Combining Instruction Sequences +* 5-5-5:: An Example of Compiled Code +* 5-5-6:: Lexical Addressing +* 5-5-7:: Interfacing Compiled Code to the Evaluator + + +File: sicp.info, Node: UTF, Next: Dedication, Prev: Top, Up: Top + +Unofficial Texinfo Format +************************* + +This is the second edition SICP book, from Unofficial Texinfo Format. + + You are probably reading it in an Info hypertext browser, such as +the Info mode of Emacs. You might alternatively be reading it +TeX-formatted on your screen or printer, though that would be silly. +And, if printed, expensive. + + The freely-distributed official HTML-and-GIF format was first +converted personually to Unofficial Texinfo Format (UTF) version 1 by +Lyssa Ayth during a long Emacs lovefest weekend in April, 2001. + + The UTF is easier to search than the HTML format. It is also much +more accessible to people running on modest computers, such as donated +'386-based PCs. A 386 can, in theory, run Linux, Emacs, and a Scheme +interpreter simultaneously, but most 386s probably can't also run both +Netscape and the necessary X Window System without prematurely +introducing budding young underfunded hackers to the concept of "thrashing". +UTF can also fit uncompressed on a 1.44MB floppy diskette, which may +come in handy for installing UTF on PCs that do not have Internet or +LAN access. + + The Texinfo conversion has been a straight transliteration, to the +extent possible. Like the TeX-to-HTML conversion, this was not without +some introduction of breakage. In the case of Unofficial Texinfo +Format, figures have suffered an amateurish resurrection of the lost +art of ASCII art. Also, it's quite possible that some errors of +ambiguity were introduced during the conversion of some of the copious +superscripts (`^') and subscripts (`_'). Divining _which_ has been +left as an exercise to the reader. But at least we don't put our brave +astronauts at risk by encoding the _greater-than-or-equal_ symbol as +`>'. + + If you modify `sicp.texi' to correct errors or improve the ASCII +art, then update the `@set utfversion 2.neilvandyke4' line to reflect +your delta. For example, if you started with Lytha's version `1', and +your name is Bob, then you could name your successive versions +`1.bob1', `1.bob2', ... `1.bobn'. Also update `utfversiondate'. If +you want to distribute your version on the Web, then embedding the +string "sicp.texi" somewhere in the file or Web page will make it +easier for people to find with Web search engines. + + It is believed that the Unofficial Texinfo Format is in keeping with +the spirit of the graciously freely-distributed HTML version. But you +never know when someone's armada of lawyers might need something to do, +and get their shorts all in a knot over some benign little thing, so +think twice before you use your full name or distribute Info, DVI, +PostScript, or PDF formats that might embed your account or machine +name. + +Peath, + +Lytha Ayth + + Addendum: See also the SICP video lectures by Abelson and Sussman: +`http://www.swiss.ai.mit.edu/classes/6.001/abelson-sussman-lectures/' + + +File: sicp.info, Node: Dedication, Next: Foreword, Prev: UTF, Up: Top + +Dedication +********** + +This book is dedicated, in respect and admiration, to the spirit that +lives in the computer. + + "I think that it's extraordinarily important that we in computer + science keep fun in computing. When it started out, it was an + awful lot of fun. Of course, the paying customers got shafted + every now and then, and after a while we began to take their + complaints seriously. We began to feel as if we really were + responsible for the successful, error-free perfect use of these + machines. I don't think we are. I think we're responsible for + stretching them, setting them off in new directions, and keeping + fun in the house. I hope the field of computer science never + loses its sense of fun. Above all, I hope we don't become + missionaries. Don't feel as if you're Bible salesmen. The world + has too many of those already. What you know about computing + other people will learn. Don't feel as if the key to successful + computing is only in your hands. What's in your hands, I think + and hope, is intelligence: the ability to see the machine as more + than when you were first led up to it, that you can make it more." + + --Alan J. Perlis (April 1, 1922 February 7, 1990) + + +File: sicp.info, Node: Foreword, Next: Preface, Prev: Dedication, Up: Top + +Foreword +******** + +Educators, generals, dieticians, psychologists, and parents program. +Armies, students, and some societies are programmed. An assault on +large problems employs a succession of programs, most of which spring +into existence en route. These programs are rife with issues that +appear to be particular to the problem at hand. To appreciate +programming as an intellectual activity in its own right you must turn +to computer programming; you must read and write computer +programs--many of them. It doesn't matter much what the programs are +about or what applications they serve. What does matter is how well +they perform and how smoothly they fit with other programs in the +creation of still greater programs. The programmer must seek both +perfection of part and adequacy of collection. In this book the use of +"program" is focused on the creation, execution, and study of programs +written in a dialect of Lisp for execution on a digital computer. +Using Lisp we restrict or limit not what we may program, but only the +notation for our program descriptions. + + Our traffic with the subject matter of this book involves us with +three foci of phenomena: the human mind, collections of computer +programs, and the computer. Every computer program is a model, hatched +in the mind, of a real or mental process. These processes, arising +from human experience and thought, are huge in number, intricate in +detail, and at any time only partially understood. They are modeled to +our permanent satisfaction rarely by our computer programs. Thus even +though our programs are carefully handcrafted discrete collections of +symbols, mosaics of interlocking functions, they continually evolve: we +change them as our perception of the model deepens, enlarges, +generalizes until the model ultimately attains a metastable place +within still another model with which we struggle. The source of the +exhilaration associated with computer programming is the continual +unfolding within the mind and on the computer of mechanisms expressed +as programs and the explosion of perception they generate. If art +interprets our dreams, the computer executes them in the guise of +programs! + + For all its power, the computer is a harsh taskmaster. Its programs +must be correct, and what we wish to say must be said accurately in +every detail. As in every other symbolic activity, we become convinced +of program truth through argument. Lisp itself can be assigned a +semantics (another model, by the way), and if a program's function can +be specified, say, in the predicate calculus, the proof methods of +logic can be used to make an acceptable correctness argument. +Unfortunately, as programs get large and complicated, as they almost +always do, the adequacy, consistency, and correctness of the +specifications themselves become open to doubt, so that complete formal +arguments of correctness seldom accompany large programs. Since large +programs grow from small ones, it is crucial that we develop an arsenal +of standard program structures of whose correctness we have become +sure--we call them idioms--and learn to combine them into larger +structures using organizational techniques of proven value. These +techniques are treated at length in this book, and understanding them +is essential to participation in the Promethean enterprise called +programming. More than anything else, the uncovering and mastery of +powerful organizational techniques accelerates our ability to create +large, significant programs. Conversely, since writing large programs +is very taxing, we are stimulated to invent new methods of reducing the +mass of function and detail to be fitted into large programs. + + Unlike programs, computers must obey the laws of physics. If they +wish to perform rapidly--a few nanoseconds per state change--they must +transmit electrons only small distances (at most 11 over 2 feet). The +heat generated by the huge number of devices so concentrated in space +has to be removed. An exquisite engineering art has been developed +balancing between multiplicity of function and density of devices. In +any event, hardware always operates at a level more primitive than that +at which we care to program. The processes that transform our Lisp +programs to "machine" programs are themselves abstract models which we +program. Their study and creation give a great deal of insight into +the organizational programs associated with programming arbitrary +models. Of course the computer itself can be so modeled. Think of it: +the behavior of the smallest physical switching element is modeled by +quantum mechanics described by differential equations whose detailed +behavior is captured by numerical approximations represented in +computer programs executing on computers composed of ...! + + It is not merely a matter of tactical convenience to separately +identify the three foci. Even though, as they say, it's all in the +head, this logical separation induces an acceleration of symbolic +traffic between these foci whose richness, vitality, and potential is +exceeded in human experience only by the evolution of life itself. At +best, relationships between the foci are metastable. The computers are +never large enough or fast enough. Each breakthrough in hardware +technology leads to more massive programming enterprises, new +organizational principles, and an enrichment of abstract models. Every +reader should ask himself periodically "Toward what end, toward what +end?"--but do not ask it too often lest you pass up the fun of +programming for the constipation of bittersweet philosophy. + + Among the programs we write, some (but never enough) perform a +precise mathematical function such as sorting or finding the maximum of +a sequence of numbers, determining primality, or finding the square +root. We call such programs algorithms, and a great deal is known of +their optimal behavior, particularly with respect to the two important +parameters of execution time and data storage requirements. A +programmer should acquire good algorithms and idioms. Even though some +programs resist precise specifications, it is the responsibility of the +programmer to estimate, and always to attempt to improve, their +performance. + + Lisp is a survivor, having been in use for about a quarter of a +century. Among the active programming languages only Fortran has had a +longer life. Both languages have supported the programming needs of +important areas of application, Fortran for scientific and engineering +computation and Lisp for artificial intelligence. These two areas +continue to be important, and their programmers are so devoted to these +two languages that Lisp and Fortran may well continue in active use for +at least another quarter-century. + + Lisp changes. The Scheme dialect used in this text has evolved from +the original Lisp and differs from the latter in several important +ways, including static scoping for variable binding and permitting +functions to yield functions as values. In its semantic structure +Scheme is as closely akin to Algol 60 as to early Lisps. Algol 60, +never to be an active language again, lives on in the genes of Scheme +and Pascal. It would be difficult to find two languages that are the +communicating coin of two more different cultures than those gathered +around these two languages. Pascal is for building pyramids--imposing, +breathtaking, static structures built by armies pushing heavy blocks +into place. Lisp is for building organisms--imposing, breathtaking, +dynamic structures built by squads fitting fluctuating myriads of +simpler organisms into place. The organizing principles used are the +same in both cases, except for one extraordinarily important +difference: The discretionary exportable functionality entrusted to the +individual Lisp programmer is more than an order of magnitude greater +than that to be found within Pascal enterprises. Lisp programs inflate +libraries with functions whose utility transcends the application that +produced them. The list, Lisp's native data structure, is largely +responsible for such growth of utility. The simple structure and +natural applicability of lists are reflected in functions that are +amazingly nonidiosyncratic. In Pascal the plethora of declarable data +structures induces a specialization within functions that inhibits and +penalizes casual cooperation. It is better to have 100 functions +operate on one data structure than to have 10 functions operate on 10 +data structures. As a result the pyramid must stand unchanged for a +millennium; the organism must evolve or perish. + + To illustrate this difference, compare the treatment of material and +exercises within this book with that in any first-course text using +Pascal. Do not labor under the illusion that this is a text digestible +at MIT only, peculiar to the breed found there. It is precisely what a +serious book on programming Lisp must be, no matter who the student is +or where it is used. + + Note that this is a text about programming, unlike most Lisp books, +which are used as a preparation for work in artificial intelligence. +After all, the critical programming concerns of software engineering +and artificial intelligence tend to coalesce as the systems under +investigation become larger. This explains why there is such growing +interest in Lisp outside of artificial intelligence. + + As one would expect from its goals, artificial intelligence research +generates many significant programming problems. In other programming +cultures this spate of problems spawns new languages. Indeed, in any +very large programming task a useful organizing principle is to control +and isolate traffic within the task modules via the invention of +language. These languages tend to become less primitive as one +approaches the boundaries of the system where we humans interact most +often. As a result, such systems contain complex language-processing +functions replicated many times. Lisp has such a simple syntax and +semantics that parsing can be treated as an elementary task. Thus +parsing technology plays almost no role in Lisp programs, and the +construction of language processors is rarely an impediment to the rate +of growth and change of large Lisp systems. Finally, it is this very +simplicity of syntax and semantics that is responsible for the burden +and freedom borne by all Lisp programmers. No Lisp program of any size +beyond a few lines can be written without being saturated with +discretionary functions. Invent and fit; have fits and reinvent! We +toast the Lisp programmer who pens his thoughts within nests of +parentheses. + +Alan J. Perlis +New Haven, Connecticut + + +File: sicp.info, Node: Preface, Next: Preface 1e, Prev: Foreword, Up: Top + +Preface to the Second Edition +***************************** + + Is it possible that software is not like anything else, that it is + meant to be discarded: that the whole point is to always see it as + a soap bubble? + + --Alan J. Perlis + + The material in this book has been the basis of MIT's entry-level +computer science subject since 1980. We had been teaching this +material for four years when the first edition was published, and +twelve more years have elapsed until the appearance of this second +edition. We are pleased that our work has been widely adopted and +incorporated into other texts. We have seen our students take the +ideas and programs in this book and build them in as the core of new +computer systems and languages. In literal realization of an ancient +Talmudic pun, our students have become our builders. We are lucky to +have such capable students and such accomplished builders. + + In preparing this edition, we have incorporated hundreds of +clarifications suggested by our own teaching experience and the +comments of colleagues at MIT and elsewhere. We have redesigned most +of the major programming systems in the book, including the +generic-arithmetic system, the interpreters, the register-machine +simulator, and the compiler; and we have rewritten all the program +examples to ensure that any Scheme implementation conforming to the +IEEE Scheme standard (IEEE 1990) will be able to run the code. + + This edition emphasizes several new themes. The most important of +these is the central role played by different approaches to dealing +with time in computational models: objects with state, concurrent +programming, functional programming, lazy evaluation, and +nondeterministic programming. We have included new sections on +concurrency and nondeterminism, and we have tried to integrate this +theme throughout the book. + + The first edition of the book closely followed the syllabus of our +MIT one-semester subject. With all the new material in the second +edition, it will not be possible to cover everything in a single +semester, so the instructor will have to pick and choose. In our own +teaching, we sometimes skip the section on logic programming (section +*Note 4-4::), we have students use the register-machine simulator but +we do not cover its implementation (section *Note 5-2::), and we give +only a cursory overview of the compiler (section *Note 5-5::). Even +so, this is still an intense course. Some instructors may wish to +cover only the first three or four chapters, leaving the other material +for subsequent courses. + + The World-Wide-Web site `http://mitpress.mit.edu/sicp/' provides +support for users of this book. This includes programs from the book, +sample programming assignments, supplementary materials, and +downloadable implementations of the Scheme dialect of Lisp. + + +File: sicp.info, Node: Preface 1e, Next: Acknowledgements, Prev: Preface, Up: Top + +Preface to the First Edition +**************************** + + A computer is like a violin. You can imagine a novice trying + first a phonograph and then a violin. The latter, he says, sounds + terrible. That is the argument we have heard from our humanists + and most of our computer scientists. Computer programs are good, + they say, for particular purposes, but they aren't flexible. + Neither is a violin, or a typewriter, until you learn how to use + it. + + --Marvin Minsky, "Why Programming Is a Good Medium for Expressing + Poorly-Understood and Sloppily-Formulated Ideas" + + "The Structure and Interpretation of Computer Programs" is the +entry-level subject in computer science at the Massachusetts Institute +of Technology. It is required of all students at MIT who major in +electrical engineering or in computer science, as one-fourth of the +"common core curriculum," which also includes two subjects on circuits +and linear systems and a subject on the design of digital systems. We +have been involved in the development of this subject since 1978, and +we have taught this material in its present form since the fall of 1980 +to between 600 and 700 students each year. Most of these students have +had little or no prior formal training in computation, although many +have played with computers a bit and a few have had extensive +programming or hardware-design experience. + + Our design of this introductory computer-science subject reflects +two major concerns. First, we want to establish the idea that a +computer language is not just a way of getting a computer to perform +operations but rather that it is a novel formal medium for expressing +ideas about methodology. Thus, programs must be written for people to +read, and only incidentally for machines to execute. Second, we +believe that the essential material to be addressed by a subject at +this level is not the syntax of particular programming-language +constructs, nor clever algorithms for computing particular functions +efficiently, nor even the mathematical analysis of algorithms and the +foundations of computing, but rather the techniques used to control the +intellectual complexity of large software systems. + + Our goal is that students who complete this subject should have a +good feel for the elements of style and the aesthetics of programming. +They should have command of the major techniques for controlling +complexity in a large system. They should be capable of reading a +50-page-long program, if it is written in an exemplary style. They +should know what not to read, and what they need not understand at any +moment. They should feel secure about modifying a program, retaining +the spirit and style of the original author. + + These skills are by no means unique to computer programming. The +techniques we teach and draw upon are common to all of engineering +design. We control complexity by building abstractions that hide +details when appropriate. We control complexity by establishing +conventional interfaces that enable us to construct systems by +combining standard, well-understood pieces in a "mix and match" way. +We control complexity by establishing new languages for describing a +design, each of which emphasizes particular aspects of the design and +deemphasizes others. + + Underlying our approach to this subject is our conviction that +"computer science" is not a science and that its significance has +little to do with computers. The computer revolution is a revolution +in the way we think and in the way we express what we think. The +essence of this change is the emergence of what might best be called "procedural +epistemology"--the study of the structure of knowledge from an +imperative point of view, as opposed to the more declarative point of +view taken by classical mathematical subjects. Mathematics provides a +framework for dealing precisely with notions of "what is." Computation +provides a framework for dealing precisely with notions of "how to." + + In teaching our material we use a dialect of the programming +language Lisp. We never formally teach the language, because we don't +have to. We just use it, and students pick it up in a few days. This +is one great advantage of Lisp-like languages: They have very few ways +of forming compound expressions, and almost no syntactic structure. +All of the formal properties can be covered in an hour, like the rules +of chess. After a short time we forget about syntactic details of the +language (because there are none) and get on with the real +issues--figuring out what we want to compute, how we will decompose +problems into manageable parts, and how we will work on the parts. +Another advantage of Lisp is that it supports (but does not enforce) +more of the large-scale strategies for modular decomposition of +programs than any other language we know. We can make procedural and +data abstractions, we can use higher-order functions to capture common +patterns of usage, we can model local state using assignment and data +mutation, we can link parts of a program with streams and delayed +evaluation, and we can easily implement embedded languages. All of +this is embedded in an interactive environment with excellent support +for incremental program design, construction, testing, and debugging. +We thank all the generations of Lisp wizards, starting with John +McCarthy, who have fashioned a fine tool of unprecedented power and +elegance. + + Scheme, the dialect of Lisp that we use, is an attempt to bring +together the power and elegance of Lisp and Algol. From Lisp we take +the metalinguistic power that derives from the simple syntax, the +uniform representation of programs as data objects, and the +garbage-collected heap-allocated data. From Algol we take lexical +scoping and block structure, which are gifts from the pioneers of +programming-language design who were on the Algol committee. We wish +to cite John Reynolds and Peter Landin for their insights into the +relationship of Church's [lambda] calculus to the structure of +programming languages. We also recognize our debt to the +mathematicians who scouted out this territory decades before computers +appeared on the scene. These pioneers include Alonzo Church, Barkley +Rosser, Stephen Kleene, and Haskell Curry. + + +File: sicp.info, Node: Acknowledgements, Next: Chapter 1, Prev: Preface 1e, Up: Top + +Acknowledgements +**************** + +We would like to thank the many people who have helped us develop this +book and this curriculum. + + Our subject is a clear intellectual descendant of "6.231," a +wonderful subject on programming linguistics and the [lambda] calculus +taught at MIT in the late 1960s by Jack Wozencraft and Arthur Evans, Jr. + + We owe a great debt to Robert Fano, who reorganized MIT's +introductory curriculum in electrical engineering and computer science +to emphasize the principles of engineering design. He led us in +starting out on this enterprise and wrote the first set of subject +notes from which this book evolved. + + Much of the style and aesthetics of programming that we try to teach +were developed in conjunction with Guy Lewis Steele Jr., who +collaborated with Gerald Jay Sussman in the initial development of the +Scheme language. In addition, David Turner, Peter Henderson, Dan +Friedman, David Wise, and Will Clinger have taught us many of the +techniques of the functional programming community that appear in this +book. + + Joel Moses taught us about structuring large systems. His +experience with the Macsyma system for symbolic computation provided +the insight that one should avoid complexities of control and +concentrate on organizing the data to reflect the real structure of the +world being modeled. + + Marvin Minsky and Seymour Papert formed many of our attitudes about +programming and its place in our intellectual lives. To them we owe +the understanding that computation provides a means of expression for +exploring ideas that would otherwise be too complex to deal with +precisely. They emphasize that a student's ability to write and modify +programs provides a powerful medium in which exploring becomes a +natural activity. + + We also strongly agree with Alan Perlis that programming is lots of +fun and we had better be careful to support the joy of programming. +Part of this joy derives from observing great masters at work. We are +fortunate to have been apprentice programmers at the feet of Bill +Gosper and Richard Greenblatt. + + It is difficult to identify all the people who have contributed to +the development of our curriculum. We thank all the lecturers, +recitation instructors, and tutors who have worked with us over the +past fifteen years and put in many extra hours on our subject, +especially Bill Siebert, Albert Meyer, Joe Stoy, Randy Davis, Louis +Braida, Eric Grimson, Rod Brooks, Lynn Stein, and Peter Szolovits. We +would like to specially acknowledge the outstanding teaching +contributions of Franklyn Turbak, now at Wellesley; his work in +undergraduate instruction set a standard that we can all aspire to. We +are grateful to Jerry Saltzer and Jim Miller for helping us grapple +with the mysteries of concurrency, and to Peter Szolovits and David +McAllester for their contributions to the exposition of +nondeterministic evaluation in *Note Chapter 4::. + + Many people have put in significant effort presenting this material +at other universities. Some of the people we have worked closely with +are Jacob Katzenelson at the Technion, Hardy Mayer at the University of +California at Irvine, Joe Stoy at Oxford, Elisha Sacks at Purdue, and +Jan Komorowski at the Norwegian University of Science and Technology. +We are exceptionally proud of our colleagues who have received major +teaching awards for their adaptations of this subject at other +universities, including Kenneth Yip at Yale, Brian Harvey at the +University of California at Berkeley, and Dan Huttenlocher at Cornell. + + Al Moye' arranged for us to teach this material to engineers at +Hewlett-Packard, and for the production of videotapes of these +lectures. We would like to thank the talented instructors--in +particular Jim Miller, Bill Siebert, and Mike Eisenberg--who have +designed continuing education courses incorporating these tapes and +taught them at universities and industry all over the world. + + Many educators in other countries have put in significant work +translating the first edition. Michel Briand, Pierre Chamard, and +Andre' Pic produced a French edition; Susanne Daniels-Herold produced a +German edition; and Fumio Motoyoshi produced a Japanese edition. We do +not know who produced the Chinese edition, but we consider it an honor +to have been selected as the subject of an "unauthorized" translation. + + It is hard to enumerate all the people who have made technical +contributions to the development of the Scheme systems we use for +instructional purposes. In addition to Guy Steele, principal wizards +have included Chris Hanson, Joe Bowbeer, Jim Miller, Guillermo Rozas, +and Stephen Adams. Others who have put in significant time are Richard +Stallman, Alan Bawden, Kent Pitman, Jon Taft, Neil Mayle, John Lamping, +Gwyn Osnos, Tracy Larrabee, George Carrette, Soma Chaudhuri, Bill +Chiarchiaro, Steven Kirsch, Leigh Klotz, Wayne Noss, Todd Cass, Patrick +O'Donnell, Kevin Theobald, Daniel Weise, Kenneth Sinclair, Anthony +Courtemanche, Henry M. Wu, Andrew Berlin, and Ruth Shyu. + + Beyond the MIT implementation, we would like to thank the many people +who worked on the IEEE Scheme standard, including William Clinger and +Jonathan Rees, who edited the R^4RS, and Chris Haynes, David Bartley, +Chris Hanson, and Jim Miller, who prepared the IEEE standard. + + Dan Friedman has been a long-time leader of the Scheme community. +The community's broader work goes beyond issues of language design to +encompass significant educational innovations, such as the high-school +curriculum based on EdScheme by Schemer's Inc., and the wonderful books +by Mike Eisenberg and by Brian Harvey and Matthew Wright. + + We appreciate the work of those who contributed to making this a +real book, especially Terry Ehling, Larry Cohen, and Paul Bethge at the +MIT Press. Ella Mazel found the wonderful cover image. For the second +edition we are particularly grateful to Bernard and Ella Mazel for help +with the book design, and to David Jones, TeX wizard extraordinaire. +We also are indebted to those readers who made penetrating comments on +the new draft: Jacob Katzenelson, Hardy Mayer, Jim Miller, and +especially Brian Harvey, who did unto this book as Julie did unto his +book `Simply Scheme'. + + Finally, we would like to acknowledge the support of the +organizations that have encouraged this work over the years, including +suppport from Hewlett-Packard, made possible by Ira Goldstein and Joel +Birnbaum, and support from DARPA, made possible by Bob Kahn. + + +File: sicp.info, Node: Chapter 1, Next: Chapter 2, Prev: Acknowledgements, Up: Top + +1 Building Abstractions with Procedures +*************************************** + + The acts of the mind, wherein it exerts its power over simple + ideas, are chiefly these three: 1. Combining several simple ideas + into one compound one, and thus all complex ideas are made. 2. + The second is bringing two ideas, whether simple or complex, + together, and setting them by one another so as to take a view of + them at once, without uniting them into one, by which it gets all + its ideas of relations. 3. The third is separating them from all + other ideas that accompany them in their real existence: this is + called abstraction, and thus all its general ideas are made. + + --John Locke, _An Essay Concerning Human Understanding_ (1690) + + We are about to study the idea of a "computational process". +Computational processes are abstract beings that inhabit computers. As +they evolve, processes manipulate other abstract things called "data". +The evolution of a process is directed by a pattern of rules called a "program". +People create programs to direct processes. In effect, we conjure the +spirits of the computer with our spells. + + A computational process is indeed much like a sorcerer's idea of a +spirit. It cannot be seen or touched. It is not composed of matter at +all. However, it is very real. It can perform intellectual work. It +can answer questions. It can affect the world by disbursing money at a +bank or by controlling a robot arm in a factory. The programs we use +to conjure processes are like a sorcerer's spells. They are carefully +composed from symbolic expressions in arcane and esoteric "programming +languages" that prescribe the tasks we want our processes to perform. + + A computational process, in a correctly working computer, executes +programs precisely and accurately. Thus, like the sorcerer's +apprentice, novice programmers must learn to understand and to +anticipate the consequences of their conjuring. Even small errors +(usually called "bugs" or "glitches") in programs can have complex and +unanticipated consequences. + + Fortunately, learning to program is considerably less dangerous than +learning sorcery, because the spirits we deal with are conveniently +contained in a secure way. Real-world programming, however, requires +care, expertise, and wisdom. A small bug in a computer-aided design +program, for example, can lead to the catastrophic collapse of an +airplane or a dam or the self-destruction of an industrial robot. + + Master software engineers have the ability to organize programs so +that they can be reasonably sure that the resulting processes will +perform the tasks intended. They can visualize the behavior of their +systems in advance. They know how to structure programs so that +unanticipated problems do not lead to catastrophic consequences, and +when problems do arise, they can "debug" their programs. Well-designed +computational systems, like well-designed automobiles or nuclear +reactors, are designed in a modular manner, so that the parts can be +constructed, replaced, and debugged separately. + +Programming in Lisp +................... + +We need an appropriate language for describing processes, and we will +use for this purpose the programming language Lisp. Just as our +everyday thoughts are usually expressed in our natural language (such +as English, French, or Japanese), and descriptions of quantitative +phenomena are expressed with mathematical notations, our procedural +thoughts will be expressed in Lisp. Lisp was invented in the late +1950s as a formalism for reasoning about the use of certain kinds of +logical expressions, called "recursion equations", as a model for +computation. The language was conceived by John McCarthy and is based +on his paper "Recursive Functions of Symbolic Expressions and Their +Computation by Machine" (McCarthy 1960). + + Despite its inception as a mathematical formalism, Lisp is a +practical programming language. A Lisp "interpreter" is a machine that +carries out processes described in the Lisp language. The first Lisp +interpreter was implemented by McCarthy with the help of colleagues and +students in the Artificial Intelligence Group of the MIT Research +Laboratory of Electronics and in the MIT Computation Center.(1) Lisp, +whose name is an acronym for LISt Processing, was designed to provide +symbol-manipulating capabilities for attacking programming problems +such as the symbolic differentiation and integration of algebraic +expressions. It included for this purpose new data objects known as +atoms and lists, which most strikingly set it apart from all other +languages of the period. + + Lisp was not the product of a concerted design effort. Instead, it +evolved informally in an experimental manner in response to users' +needs and to pragmatic implementation considerations. Lisp's informal +evolution has continued through the years, and the community of Lisp +users has traditionally resisted attempts to promulgate any "official" +definition of the language. This evolution, together with the +flexibility and elegance of the initial conception, has enabled Lisp, +which is the second oldest language in widespread use today (only +Fortran is older), to continually adapt to encompass the most modern +ideas about program design. Thus, Lisp is by now a family of dialects, +which, while sharing most of the original features, may differ from one +another in significant ways. The dialect of Lisp used in this book is +called Scheme.(2) + + Because of its experimental character and its emphasis on symbol +manipulation, Lisp was at first very inefficient for numerical +computations, at least in comparison with Fortran. Over the years, +however, Lisp compilers have been developed that translate programs +into machine code that can perform numerical computations reasonably +efficiently. And for special applications, Lisp has been used with +great effectiveness.(3) Although Lisp has not yet overcome its old +reputation as hopelessly inefficient, Lisp is now used in many +applications where efficiency is not the central concern. For example, +Lisp has become a language of choice for operating-system shell +languages and for extension languages for editors and computer-aided +design systems. + + If Lisp is not a mainstream language, why are we using it as the +framework for our discussion of programming? Because the language +possesses unique features that make it an excellent medium for studying +important programming constructs and data structures and for relating +them to the linguistic features that support them. The most +significant of these features is the fact that Lisp descriptions of +processes, called "procedures", can themselves be represented and +manipulated as Lisp data. The importance of this is that there are +powerful program-design techniques that rely on the ability to blur the +traditional distinction between "passive" data and "active" processes. +As we shall discover, Lisp's flexibility in handling procedures as data +makes it one of the most convenient languages in existence for +exploring these techniques. The ability to represent procedures as +data also makes Lisp an excellent language for writing programs that +must manipulate other programs as data, such as the interpreters and +compilers that support computer languages. Above and beyond these +considerations, programming in Lisp is great fun. + +* Menu: + +* 1-1:: The Elements of Programming +* 1-2:: Procedures and the Processes They Generate +* 1-3:: Formulating Abstractions with Higher-Order Procedures + + ---------- Footnotes ---------- + + (1) The `Lisp 1 Programmer's Manual' appeared in 1960, and the `Lisp +1.5 Programmer's Manual' (McCarthy 1965) was published in 1962. The +early history of Lisp is described in McCarthy 1978. + + (2) The two dialects in which most major Lisp programs of the 1970s +were written are MacLisp (Moon 1978; Pitman 1983), developed at the MIT +Project MAC, and Interlisp (Teitelman 1974), developed at Bolt Beranek +and Newman Inc. and the Xerox Palo Alto Research Center. Portable +Standard Lisp (Hearn 1969; Griss 1981) was a Lisp dialect designed to +be easily portable between different machines. MacLisp spawned a +number of subdialects, such as Franz Lisp, which was developed at the +University of California at Berkeley, and Zetalisp (Moon 1981), which +was based on a special-purpose processor designed at the MIT Artificial +Intelligence Laboratory to run Lisp very efficiently. The Lisp dialect +used in this book, called Scheme (Steele 1975), was invented in 1975 by +Guy Lewis Steele Jr. and Gerald Jay Sussman of the MIT Artificial +Intelligence Laboratory and later reimplemented for instructional use +at MIT. Scheme became an IEEE standard in 1990 (IEEE 1990). The +Common Lisp dialect (Steele 1982, Steele 1990) was developed by the +Lisp community to combine features from the earlier Lisp dialects to +make an industrial standard for Lisp. Common Lisp became an ANSI +standard in 1994 (ANSI 1994). + + (3) One such special application was a breakthrough computation of +scientific importance--an integration of the motion of the Solar System +that extended previous results by nearly two orders of magnitude, and +demonstrated that the dynamics of the Solar System is chaotic. This +computation was made possible by new integration algorithms, a +special-purpose compiler, and a special-purpose computer all +implemented with the aid of software tools written in Lisp (Abelson et +al. 1992; Sussman and Wisdom 1992). + + +File: sicp.info, Node: 1-1, Next: 1-2, Prev: Chapter 1, Up: Chapter 1 + +1.1 The Elements of Programming +=============================== + +A powerful programming language is more than just a means for +instructing a computer to perform tasks. The language also serves as a +framework within which we organize our ideas about processes. Thus, +when we describe a language, we should pay particular attention to the +means that the language provides for combining simple ideas to form +more complex ideas. Every powerful language has three mechanisms for +accomplishing this: + +"primitive expressions" + which represent the simplest entities the language is concerned + with, + +"means of combination" + by which compound elements are built from simpler ones, and + +"means of abstraction" + by which compound elements can be named and manipulated as units. + + + In programming, we deal with two kinds of elements: procedures and +data. (Later we will discover that they are really not so distinct.) +Informally, data is "stuff" that we want to manipulate, and procedures +are descriptions of the rules for manipulating the data. Thus, any +powerful programming language should be able to describe primitive data +and primitive procedures and should have methods for combining and +abstracting procedures and data. + + In this chapter we will deal only with simple numerical data so that +we can focus on the rules for building procedures.(1) In later chapters +we will see that these same rules allow us to build procedures to +manipulate compound data as well. + +* Menu: + +* 1-1-1:: Expressions +* 1-1-2:: Naming and the Environment +* 1-1-3:: Evaluating Combinations +* 1-1-4:: Compound Procedures +* 1-1-5:: The Substitution Model for Procedure Application +* 1-1-6:: Conditional Expressions and Predicates +* 1-1-7:: Example: Square Roots by Newton's Method +* 1-1-8:: Procedures as Black-Box Abstractions + + ---------- Footnotes ---------- + + (1) The characterization of numbers as "simple data" is a barefaced +bluff. In fact, the treatment of numbers is one of the trickiest and +most confusing aspects of any programming language. Some typical +issues involved are these: Some computer systems distinguish "integers", +such as 2, from "real numbers", such as 2.71. Is the real number 2.00 +different from the integer 2? Are the arithmetic operations used for +integers the same as the operations used for real numbers? Does 6 +divided by 2 produce 3, or 3.0? How large a number can we represent? +How many decimal places of accuracy can we represent? Is the range of +integers the same as the range of real numbers? Above and beyond these +questions, of course, lies a collection of issues concerning roundoff +and truncation errors - the entire science of numerical analysis. +Since our focus in this book is on large-scale program design rather +than on numerical techniques, we are going to ignore these problems. +The numerical examples in this chapter will exhibit the usual roundoff +behavior that one observes when using arithmetic operations that +preserve a limited number of decimal places of accuracy in noninteger +operations. + + +File: sicp.info, Node: 1-1-1, Next: 1-1-2, Prev: 1-1, Up: 1-1 + +1.1.1 Expressions +----------------- + +One easy way to get started at programming is to examine some typical +interactions with an interpreter for the Scheme dialect of Lisp. +Imagine that you are sitting at a computer terminal. You type an "expression", +and the interpreter responds by displaying the result of its "evaluating" +that expression. + + One kind of primitive expression you might type is a number. (More +precisely, the expression that you type consists of the numerals that +represent the number in base 10.) If you present Lisp with a number + + 486 + +the interpreter will respond by printing (1) + + 486 + + Expressions representing numbers may be combined with an expression +representing a primitive procedure (such as `+' or `*') to form a +compound expression that represents the application of the procedure to +those numbers. For example: + + (+ 137 349) + 486 + + (- 1000 334) + 666 + + (* 5 99) + 495 + + (/ 10 5) + 2 + + (+ 2.7 10) + 12.7 + + Expressions such as these, formed by delimiting a list of +expressions within parentheses in order to denote procedure +application, are called "combinations". The leftmost element in the +list is called the "operator", and the other elements are called "operands". +The value of a combination is obtained by applying the procedure +specified by the operator to the "arguments" that are the values of the +operands. + + The convention of placing the operator to the left of the operands +is known as "prefix notation", and it may be somewhat confusing at +first because it departs significantly from the customary mathematical +convention. Prefix notation has several advantages, however. One of +them is that it can accommodate procedures that may take an arbitrary +number of arguments, as in the following examples: + + (+ 21 35 12 7) + 75 + + (* 25 4 12) + 1200 + + No ambiguity can arise, because the operator is always the leftmost +element and the entire combination is delimited by the parentheses. + + A second advantage of prefix notation is that it extends in a +straightforward way to allow combinations to be nested, that is, to +have combinations whose elements are themselves combinations: + + (+ (* 3 5) (- 10 6)) + 19 + + There is no limit (in principle) to the depth of such nesting and to +the overall complexity of the expressions that the Lisp interpreter can +evaluate. It is we humans who get confused by still relatively simple +expressions such as + + (+ (* 3 (+ (* 2 4) (+ 3 5))) (+ (- 10 7) 6)) + +which the interpreter would readily evaluate to be 57. We can help +ourselves by writing such an expression in the form + + (+ (* 3 + (+ (* 2 4) + (+ 3 5))) + (+ (- 10 7) + 6)) + +following a formatting convention known as "pretty-printing", in which +each long combination is written so that the operands are aligned +vertically. The resulting indentations display clearly the structure +of the expression.(2) + + Even with complex expressions, the interpreter always operates in +the same basic cycle: It reads an expression from the terminal, +evaluates the expression, and prints the result. This mode of +operation is often expressed by saying that the interpreter runs in a "read-eval-print +loop". Observe in particular that it is not necessary to explicitly +instruct the interpreter to print the value of the expression.(3) + + ---------- Footnotes ---------- + + (1) Throughout this book, when we wish to emphasize the distinction +between the input typed by the user and the response printed by the +interpreter, we will show the latter in slanted characters. + + (2) Lisp systems typically provide features to aid the user in +formatting expressions. Two especially useful features are one that +automatically indents to the proper pretty-print position whenever a +new line is started and one that highlights the matching left +parenthesis whenever a right parenthesis is typed. + + (3) Lisp obeys the convention that every expression has a value. +This convention, together with the old reputation of Lisp as an +inefficient language, is the source of the quip by Alan Perlis +(paraphrasing Oscar Wilde) that "Lisp programmers know the value of +everything but the cost of nothing." + + +File: sicp.info, Node: 1-1-2, Next: 1-1-3, Prev: 1-1-1, Up: 1-1 + +1.1.2 Naming and the Environment +-------------------------------- + +A critical aspect of a programming language is the means it provides +for using names to refer to computational objects. We say that the +name identifies a "variable" whose "value" is the object. + + In the Scheme dialect of Lisp, we name things with `define'. Typing + + (define size 2) + +causes the interpreter to associate the value 2 with the name +`size'.(1) Once the name `size' has been associated with the number 2, +we can refer to the value 2 by name: + + size + 2 + + (* 5 size) + 10 + + Here are further examples of the use of `define': + + (define pi 3.14159) + + (define radius 10) + + (* pi (* radius radius)) + 314.159 + + (define circumference (* 2 pi radius)) + + circumference + 62.8318 + + `Define' is our language's simplest means of abstraction, for it +allows us to use simple names to refer to the results of compound +operations, such as the `circumference' computed above. In general, +computational objects may have very complex structures, and it would be +extremely inconvenient to have to remember and repeat their details +each time we want to use them. Indeed, complex programs are +constructed by building, step by step, computational objects of +increasing complexity. The interpreter makes this step-by-step program +construction particularly convenient because name-object associations +can be created incrementally in successive interactions. This feature +encourages the incremental development and testing of programs and is +largely responsible for the fact that a Lisp program usually consists +of a large number of relatively simple procedures. + + It should be clear that the possibility of associating values with +symbols and later retrieving them means that the interpreter must +maintain some sort of memory that keeps track of the name-object pairs. +This memory is called the "environment" (more precisely the "global +environment", since we will see later that a computation may involve a +number of different environments).(2) + + ---------- Footnotes ---------- + + (1) In this book, we do not show the interpreter's response to +evaluating definitions, since this is highly implementation-dependent. + + (2) *Note Chapter 3:: will show that this notion of environment is +crucial, both for understanding how the interpreter works and for +implementing interpreters. + + +File: sicp.info, Node: 1-1-3, Next: 1-1-4, Prev: 1-1-2, Up: 1-1 + +1.1.3 Evaluating Combinations +----------------------------- + +One of our goals in this chapter is to isolate issues about thinking +procedurally. As a case in point, let us consider that, in evaluating +combinations, the interpreter is itself following a procedure. + + To evaluate a combination, do the following: + + 1. Evaluate the subexpressions of the combination. + + 2. Apply the procedure that is the value of the leftmost + subexpression (the operator) to the arguments that are the + values of the other subexpressions (the operands). + + + Even this simple rule illustrates some important points about +processes in general. First, observe that the first step dictates that +in order to accomplish the evaluation process for a combination we must +first perform the evaluation process on each element of the +combination. Thus, the evaluation rule is "recursive" in nature; that +is, it includes, as one of its steps, the need to invoke the rule +itself.(1) + + Notice how succinctly the idea of recursion can be used to express +what, in the case of a deeply nested combination, would otherwise be +viewed as a rather complicated process. For example, evaluating + + (* (+ 2 (* 4 6)) + (+ 3 5 7)) + +requires that the evaluation rule be applied to four different +combinations. We can obtain a picture of this process by representing +the combination in the form of a tree, as shown in *Note Figure 1-1::. +Each combination is represented by a node with branches corresponding +to the operator and the operands of the combination stemming from it. +The terminal nodes (that is, nodes with no branches stemming from them) +represent either operators or numbers. Viewing evaluation in terms of +the tree, we can imagine that the values of the operands percolate +upward, starting from the terminal nodes and then combining at higher +and higher levels. In general, we shall see that recursion is a very +powerful technique for dealing with hierarchical, treelike objects. In +fact, the "percolate values upward" form of the evaluation rule is an +example of a general kind of process known as "tree accumulation". + + *Figure 1.1:* Tree representation, showing the value of each + subcombination. + + 390 + /|\____________ + / | \ + * 26 15 + /|\ | + / | \ // \\ + + 2 24 / | | \ + /|\ + 3 5 7 + / | \ + * 4 6 + + Next, observe that the repeated application of the first step brings +us to the point where we need to evaluate, not combinations, but +primitive expressions such as numerals, built-in operators, or other +names. We take care of the primitive cases by stipulating that + + * the values of numerals are the numbers that they name, + + * the values of built-in operators are the machine instruction + sequences that carry out the corresponding operations, and + + * the values of other names are the objects associated with those + names in the environment. + + + We may regard the second rule as a special case of the third one by +stipulating that symbols such as `+' and `*' are also included in the +global environment, and are associated with the sequences of machine +instructions that are their "values." The key point to notice is the +role of the environment in determining the meaning of the symbols in +expressions. In an interactive language such as Lisp, it is +meaningless to speak of the value of an expression such as `(+ x 1)' +without specifying any information about the environment that would +provide a meaning for the symbol `x' (or even for the symbol `+'). As +we shall see in *Note Chapter 3::, the general notion of the +environment as providing a context in which evaluation takes place will +play an important role in our understanding of program execution. + + Notice that the evaluation rule given above does not handle +definitions. For instance, evaluating `(define x 3)' does not apply +`define' to two arguments, one of which is the value of the symbol `x' +and the other of which is 3, since the purpose of the `define' is +precisely to associate `x' with a value. (That is, `(define x 3)' is +not a combination.) + + Such exceptions to the general evaluation rule are called forms +"special forms". `Define' is the only example of a special form that +we have seen so far, but we will meet others shortly. Each special +form has its own evaluation rule. The various kinds of expressions +(each with its associated evaluation rule) constitute the syntax of the +programming language. In comparison with most other programming +languages, Lisp has a very simple syntax; that is, the evaluation rule +for expressions can be described by a simple general rule together with +specialized rules for a small number of special forms.(2) + + ---------- Footnotes ---------- + + (1) It may seem strange that the evaluation rule says, as part of +the first step, that we should evaluate the leftmost element of a +combination, since at this point that can only be an operator such as +`+' or `*' representing a built-in primitive procedure such as addition +or multiplication. We will see later that it is useful to be able to +work with combinations whose operators are themselves compound +expressions. + + (2) Special syntactic forms that are simply convenient alternative +surface structures for things that can be written in more uniform ways +are sometimes called "syntactic sugar", to use a phrase coined by Peter +Landin. In comparison with users of other languages, Lisp programmers, +as a rule, are less concerned with matters of syntax. (By contrast, +examine any Pascal manual and notice how much of it is devoted to +descriptions of syntax.) This disdain for syntax is due partly to the +flexibility of Lisp, which makes it easy to change surface syntax, and +partly to the observation that many "convenient" syntactic constructs, +which make the language less uniform, end up causing more trouble than +they are worth when programs become large and complex. In the words of +Alan Perlis, "Syntactic sugar causes cancer of the semicolon." + + +File: sicp.info, Node: 1-1-4, Next: 1-1-5, Prev: 1-1-3, Up: 1-1 + +1.1.4 Compound Procedures +------------------------- + +We have identified in Lisp some of the elements that must appear in any +powerful programming language: + + * Numbers and arithmetic operations are primitive data and + procedures. + + * Nesting of combinations provides a means of combining operations. + + * Definitions that associate names with values provide a limited + means of abstraction. + + + Now we will learn about "procedure definitions", a much more powerful +abstraction technique by which a compound operation can be given a name +and then referred to as a unit. + + We begin by examining how to express the idea of "squaring." We +might say, "To square something, multiply it by itself." This is +expressed in our language as + + (define (square x) (* x x)) + + We can understand this in the following way: + + (define (square x) (* x x)) + | | | | | | + To square something, multiply it by itself. + + We have here a "compound procedure", which has been given the name +`square'. The procedure represents the operation of multiplying +something by itself. The thing to be multiplied is given a local name, +`x', which plays the same role that a pronoun plays in natural +language. Evaluating the definition creates this compound procedure +and associates it with the name `square'.(1) + + The general form of a procedure definition is + + (define ( ) ) + + The is a symbol to be associated with the procedure +definition in the environment.(2) The are the names +used within the body of the procedure to refer to the corresponding +arguments of the procedure. The is an expression that will +yield the value of the procedure application when the formal parameters +are replaced by the actual arguments to which the procedure is +applied.(3) The and the are grouped within +parentheses, just as they would be in an actual call to the procedure +being defined. + + Having defined `square', we can now use it: + + (square 21) + 441 + + (square (+ 2 5)) + 49 + + (square (square 3)) + 81 + + We can also use `square' as a building block in defining other +procedures. For example, x^2 + y^2 can be expressed as + + (+ (square x) (square y)) + + We can easily define a procedure `sum-of-squares' that, given any two +numbers as arguments, produces the sum of their squares: + + (define (sum-of-squares x y) + (+ (square x) (square y))) + (sum-of-squares 3 4) + 25 + + Now we can use `sum-of-squares' as a building block in constructing +further procedures: + + (define (f a) + (sum-of-squares (+ a 1) (* a 2))) + + (f 5) + 136 + + Compound procedures are used in exactly the same way as primitive +procedures. Indeed, one could not tell by looking at the definition of +`sum-of-squares' given above whether `square' was built into the +interpreter, like `+' and `*', or defined as a compound procedure. + + ---------- Footnotes ---------- + + (1) Observe that there are two different operations being combined +here: we are creating the procedure, and we are giving it the name +`square'. It is possible, indeed important, to be able to separate +these two notions--to create procedures without naming them, and to +give names to procedures that have already been created. We will see +how to do this in section *Note 1-3-2::. + + (2) Throughout this book, we will describe the general syntax of +expressions by using italic symbols delimited by angle brackets--e.g., +--to denote the "slots" in the expression to be filled in when +such an expression is actually used. + + (3) More generally, the body of the procedure can be a sequence of +expressions. In this case, the interpreter evaluates each expression +in the sequence in turn and returns the value of the final expression +as the value of the procedure application. + + +File: sicp.info, Node: 1-1-5, Next: 1-1-6, Prev: 1-1-4, Up: 1-1 + +1.1.5 The Substitution Model for Procedure Application +------------------------------------------------------ + +To evaluate a combination whose operator names a compound procedure, the +interpreter follows much the same process as for combinations whose +operators name primitive procedures, which we described in section +*Note 1-1-3::. That is, the interpreter evaluates the elements of the +combination and applies the procedure (which is the value of the +operator of the combination) to the arguments (which are the values of +the operands of the combination). + + We can assume that the mechanism for applying primitive procedures +to arguments is built into the interpreter. For compound procedures, +the application process is as follows: + + To apply a compound procedure to arguments, evaluate the body of + the procedure with each formal parameter replaced by the + corresponding argument. + + To illustrate this process, let's evaluate the combination + + (f 5) + +where `f' is the procedure defined in section *Note 1-1-4::. We begin +by retrieving the body of `f': + + (sum-of-squares (+ a 1) (* a 2)) + + Then we replace the formal parameter `a' by the argument 5: + + (sum-of-squares (+ 5 1) (* 5 2)) + + Thus the problem reduces to the evaluation of a combination with two +operands and an operator `sum-of-squares'. Evaluating this combination +involves three subproblems. We must evaluate the operator to get the +procedure to be applied, and we must evaluate the operands to get the +arguments. Now `(+ 5 1)' produces 6 and `(* 5 2)' produces 10, so we +must apply the `sum-of-squares' procedure to 6 and 10. These values +are substituted for the formal parameters `x' and `y' in the body of +`sum-of-squares', reducing the expression to + + (+ (square 6) (square 10)) + + If we use the definition of `square', this reduces to + + (+ (* 6 6) (* 10 10)) + +which reduces by multiplication to + + (+ 36 100) + +and finally to + + 136 + + The process we have just described is called the "substitution model" +for procedure application. It can be taken as a model that determines +the "meaning" of procedure application, insofar as the procedures in +this chapter are concerned. However, there are two points that should +be stressed: + + * The purpose of the substitution is to help us think about procedure + application, not to provide a description of how the interpreter + really works. Typical interpreters do not evaluate procedure + applications by manipulating the text of a procedure to substitute + values for the formal parameters. In practice, the "substitution" + is accomplished by using a local environment for the formal + parameters. We will discuss this more fully in *Note Chapter 3:: + and *Note Chapter 4:: when we examine the implementation of an + interpreter in detail. + + * Over the course of this book, we will present a sequence of + increasingly elaborate models of how interpreters work, + culminating with a complete implementation of an interpreter and + compiler in *Note Chapter 5::. The substitution model is only the + first of these models--a way to get started thinking formally + about the evaluation process. In general, when modeling phenomena + in science and engineering, we begin with simplified, incomplete + models. As we examine things in greater detail, these simple + models become inadequate and must be replaced by more refined + models. The substitution model is no exception. In particular, + when we address in *Note Chapter 3:: the use of procedures with + "mutable data," we will see that the substitution model breaks + down and must be replaced by a more complicated model of procedure + application.(1) + + +Applicative order versus normal order +..................................... + +According to the description of evaluation given in section *Note +1-1-3::, the interpreter first evaluates the operator and operands and +then applies the resulting procedure to the resulting arguments. This +is not the only way to perform evaluation. An alternative evaluation +model would not evaluate the operands until their values were needed. +Instead it would first substitute operand expressions for parameters +until it obtained an expression involving only primitive operators, and +would then perform the evaluation. If we used this method, the +evaluation of `(f 5)' would proceed according to the sequence of +expansions + + (sum-of-squares (+ 5 1) (* 5 2)) + + (+ (square (+ 5 1)) (square (* 5 2)) ) + + (+ (* (+ 5 1) (+ 5 1)) (* (* 5 2) (* 5 2))) + +followed by the reductions + + (+ (* 6 6) (* 10 10)) + + (+ 36 100) + + 136 + + This gives the same answer as our previous evaluation model, but the +process is different. In particular, the evaluations of `(+ 5 1)' and +`(* 5 2)' are each performed twice here, corresponding to the reduction +of the expression `(* x x)' with `x' replaced respectively by `(+ 5 1)' +and `(* 5 2)'. + + This alternative "fully expand and then reduce" evaluation method is +known as "normal-order evaluation", in contrast to the "evaluate the +arguments and then apply" method that the interpreter actually uses, +which is called "applicative-order evaluation". It can be shown that, +for procedure applications that can be modeled using substitution +(including all the procedures in the first two chapters of this book) +and that yield legitimate values, normal-order and applicative-order +evaluation produce the same value. (See *Note Exercise 1-5:: for an +instance of an "illegitimate" value where normal-order and +applicative-order evaluation do not give the same result.) + + Lisp uses applicative-order evaluation, partly because of the +additional efficiency obtained from avoiding multiple evaluations of +expressions such as those illustrated with `(+ 5 1)' and `(* 5 2)' +above and, more significantly, because normal-order evaluation becomes +much more complicated to deal with when we leave the realm of +procedures that can be modeled by substitution. On the other hand, +normal-order evaluation can be an extremely valuable tool, and we will +investigate some of its implications in *Note Chapter 3:: and *Note +Chapter 4::.(2) + + ---------- Footnotes ---------- + + (1) Despite the simplicity of the substitution idea, it turns out to +be surprisingly complicated to give a rigorous mathematical definition +of the substitution process. The problem arises from the possibility of +confusion between the names used for the formal parameters of a +procedure and the (possibly identical) names used in the expressions to +which the procedure may be applied. Indeed, there is a long history of +erroneous definitions of "substitution" in the literature of logic and +programming semantics. See Stoy 1977 for a careful discussion of +substitution. + + (2) In *Note Chapter 3:: we will introduce "stream processing", +which is a way of handling apparently "infinite" data structures by +incorporating a limited form of normal-order evaluation. In section +*Note 4-2:: we will modify the Scheme interpreter to produce a +normal-order variant of Scheme. + + +File: sicp.info, Node: 1-1-6, Next: 1-1-7, Prev: 1-1-5, Up: 1-1 + +1.1.6 Conditional Expressions and Predicates +-------------------------------------------- + +The expressive power of the class of procedures that we can define at +this point is very limited, because we have no way to make tests and to +perform different operations depending on the result of a test. For +instance, we cannot define a procedure that computes the absolute value +of a number by testing whether the number is positive, negative, or +zero and taking different actions in the different cases according to +the rule + + / + | x if x > 0 + |x| = < 0 if x = 0 + | -x if x < 0 + \ + + This construct is called a "case analysis", and there is a special +form in Lisp for notating such a case analysis. It is called `cond' +(which stands for "conditional"), and it is used as follows: + + (define (abs x) + (cond ((> x 0) x) + ((= x 0) 0) + ((< x 0) (- x)))) + + The general form of a conditional expression is + + (cond ( ) + ( ) + ... + ( )) + +consisting of the symbol `cond' followed by parenthesized pairs of +expressions + + (

    ) + +called "clauses". The first expression in each pair is a "predicate"--that +is, an expression whose value is interpreted as either true or false.(1) + + Conditional expressions are evaluated as follows. The predicate + is evaluated first. If its value is false, then is +evaluated. If 's value is also false, then is evaluated. +This process continues until a predicate is found whose value is true, +in which case the interpreter returns the value of the corresponding expression +"consequent expression" of the clause as the value of the +conditional expression. If none of the

    's is found to be true, the +value of the `cond' is undefined. + + The word "predicate" is used for procedures that return true or +false, as well as for expressions that evaluate to true or false. The +absolute-value procedure `abs' makes use of the primitive predicates +`>', `<', and `='.(2) These take two numbers as arguments and test +whether the first number is, respectively, greater than, less than, or +equal to the second number, returning true or false accordingly. + + Another way to write the absolute-value procedure is + + (define (abs x) + (cond ((< x 0) (- x)) + (else x))) + +which could be expressed in English as "If x is less than zero return - +x; otherwise return x." `Else' is a special symbol that can be used in +place of the

    in the final clause of a `cond'. This causes the +`cond' to return as its value the value of the corresponding +whenever all previous clauses have been bypassed. In fact, any +expression that always evaluates to a true value could be used as the +

    here. + + Here is yet another way to write the absolute-value procedure: + + (define (abs x) + (if (< x 0) + (- x) + x)) + + This uses the special form `if', a restricted type of conditional +that can be used when there are precisely two cases in the case +analysis. The general form of an `if' expression is + + (if ) + + To evaluate an `if' expression, the interpreter starts by evaluating +the part of the expression. If the evaluates +to a true value, the interpreter then evaluates the and +returns its value. Otherwise it evaluates the and returns +its value.(3) + + In addition to primitive predicates such as `<', `=', and `>', there +are logical composition operations, which enable us to construct +compound predicates. The three most frequently used are these: + + * `(and ... )' + + The interpreter evaluates the expressions one at a time, in + left-to-right order. If any evaluates to false, the value of + the `and' expression is false, and the rest of the 's are not + evaluated. If all 's evaluate to true values, the value of the + `and' expression is the value of the last one. + + * `(or ... )' + + The interpreter evaluates the expressions one at a time, in + left-to-right order. If any evaluates to a true value, that + value is returned as the value of the `or' expression, and the + rest of the 's are not evaluated. If all 's evaluate to + false, the value of the `or' expression is false. + + * `(not )' + + The value of a `not' expression is true when the expression + evaluates to false, and false otherwise. + + + Notice that `and' and `or' are special forms, not procedures, because +the subexpressions are not necessarily all evaluated. `Not' is an +ordinary procedure. + + As an example of how these are used, the condition that a number x +be in the range 5 < x < 10 may be expressed as + + (and (> x 5) (< x 10)) + + As another example, we can define a predicate to test whether one +number is greater than or equal to another as + + (define (>= x y) + (or (> x y) (= x y))) + +or alternatively as + + (define (>= x y) + (not (< x y))) + + *Exercise 1.1:* Below is a sequence of expressions. What is the + result printed by the interpreter in response to each expression? + Assume that the sequence is to be evaluated in the order in which + it is presented. + + 10 + + (+ 5 3 4) + + (- 9 1) + + (/ 6 2) + + (+ (* 2 4) (- 4 6)) + + (define a 3) + + (define b (+ a 1)) + + (+ a b (* a b)) + + (= a b) + + (if (and (> b a) (< b (* a b))) + b + a) + + (cond ((= a 4) 6) + ((= b 4) (+ 6 7 a)) + (else 25)) + + (+ 2 (if (> b a) b a)) + + (* (cond ((> a b) a) + ((< a b) b) + (else -1)) + (+ a 1)) + + *Exercise 1.2:* Translate the following expression into prefix + form. + + 5 + 4 + (2 - (3 - (6 + 4/5))) + ----------------------------- + 3(6 - 2)(2 - 7) + + *Exercise 1.3:* Define a procedure that takes three numbers as + arguments and returns the sum of the squares of the two larger + numbers. + + *Exercise 1.4:* Observe that our model of evaluation allows for + combinations whose operators are compound expressions. Use this + observation to describe the behavior of the following procedure: + + (define (a-plus-abs-b a b) + ((if (> b 0) + -) a b)) + + *Exercise 1.5:* Ben Bitdiddle has invented a test to determine + whether the interpreter he is faced with is using + applicative-order evaluation or normal-order evaluation. He + defines the following two procedures: + + (define (p) (p)) + + (define (test x y) + (if (= x 0) + 0 + y)) + + Then he evaluates the expression + + (test 0 (p)) + + What behavior will Ben observe with an interpreter that uses + applicative-order evaluation? What behavior will he observe with + an interpreter that uses normal-order evaluation? Explain your + answer. (Assume that the evaluation rule for the special form + `if' is the same whether the interpreter is using normal or + applicative order: The predicate expression is evaluated first, + and the result determines whether to evaluate the consequent or + the alternative expression.) + + ---------- Footnotes ---------- + + (1) "Interpreted as either true or false" means this: In Scheme, +there are two distinguished values that are denoted by the constants +`#t' and `#f'. When the interpreter checks a predicate's value, it +interprets `#f' as false. Any other value is treated as true. (Thus, +providing `#t' is logically unnecessary, but it is convenient.) In +this book we will use names `true' and `false', which are associated +with the values `#t' and `#f' respectively. + + (2) `Abs' also uses the "minus" operator `-', which, when used with +a single operand, as in `(- x)', indicates negation. + + (3) A minor difference between `if' and `cond' is that the part +of each `cond' clause may be a sequence of expressions. If the +corresponding

    is found to be true, the expressions are +evaluated in sequence and the value of the final expression in the +sequence is returned as the value of the `cond'. In an `if' +expression, however, the and must be single +expressions. + + +File: sicp.info, Node: 1-1-7, Next: 1-1-8, Prev: 1-1-6, Up: 1-1 + +1.1.7 Example: Square Roots by Newton's Method +---------------------------------------------- + +Procedures, as introduced above, are much like ordinary mathematical +functions. They specify a value that is determined by one or more +parameters. But there is an important difference between mathematical +functions and computer procedures. Procedures must be effective. + + As a case in point, consider the problem of computing square roots. +We can define the square-root function as + + sqrt(x) = the y such that y >= 0 and y^2 = x + + This describes a perfectly legitimate mathematical function. We +could use it to recognize whether one number is the square root of +another, or to derive facts about square roots in general. On the +other hand, the definition does not describe a procedure. Indeed, it +tells us almost nothing about how to actually find the square root of a +given number. It will not help matters to rephrase this definition in +pseudo-Lisp: + + (define (sqrt x) + (the y (and (>= y 0) + (= (square y) x)))) + + This only begs the question. + + The contrast between function and procedure is a reflection of the +general distinction between describing properties of things and +describing how to do things, or, as it is sometimes referred to, the +distinction between declarative knowledge and imperative knowledge. In +mathematics we are usually concerned with declarative (what is) +descriptions, whereas in computer science we are usually concerned with +imperative (how to) descriptions.(1) + + How does one compute square roots? The most common way is to use +Newton's method of successive approximations, which says that whenever +we have a guess y for the value of the square root of a number x, we +can perform a simple manipulation to get a better guess (one closer to +the actual square root) by averaging y with x/y.(2) For example, we can +compute the square root of 2 as follows. Suppose our initial guess is +1: + + Guess Quotient Average + 1 (2/1) = 2 ((2 + 1)/2) = 1.5 + 1.5 (2/1.5) = 1.3333 ((1.3333 + 1.5)/2) = 1.4167 + 1.4167 (2/1.4167) = 1.4118 ((1.4167 + 1.4118)/2) = 1.4142 + 1.4142 ... ... + +Continuing this process, we obtain better and better approximations to +the square root. + + Now let's formalize the process in terms of procedures. We start +with a value for the radicand (the number whose square root we are +trying to compute) and a value for the guess. If the guess is good +enough for our purposes, we are done; if not, we must repeat the +process with an improved guess. We write this basic strategy as a +procedure: + + (define (sqrt-iter guess x) + (if (good-enough? guess x) + guess + (sqrt-iter (improve guess x) + x))) + + A guess is improved by averaging it with the quotient of the +radicand and the old guess: + + (define (improve guess x) + (average guess (/ x guess))) + +where + + (define (average x y) + (/ (+ x y) 2)) + + We also have to say what we mean by "good enough." The following +will do for illustration, but it is not really a very good test. (See +exercise *Note Exercise 1-7::.) The idea is to improve the answer +until it is close enough so that its square differs from the radicand +by less than a predetermined tolerance (here 0.001):(3) + + (define (good-enough? guess x) + (< (abs (- (square guess) x)) 0.001)) + + Finally, we need a way to get started. For instance, we can always +guess that the square root of any number is 1:(4) + + (define (sqrt x) + (sqrt-iter 1.0 x)) + + If we type these definitions to the interpreter, we can use `sqrt' +just as we can use any procedure: + + (sqrt 9) + 3.00009155413138 + + (sqrt (+ 100 37)) + 11.704699917758145 + + (sqrt (+ (sqrt 2) (sqrt 3))) + 1.7739279023207892 + + (square (sqrt 1000)) + 1000.000369924366 + + The `sqrt' program also illustrates that the simple procedural +language we have introduced so far is sufficient for writing any purely +numerical program that one could write in, say, C or Pascal. This +might seem surprising, since we have not included in our language any +iterative (looping) constructs that direct the computer to do something +over and over again. `Sqrt-iter', on the other hand, demonstrates how +iteration can be accomplished using no special construct other than the +ordinary ability to call a procedure.(5) + + *Exercise 1.6:* Alyssa P. Hacker doesn't see why `if' needs to be + provided as a special form. "Why can't I just define it as an + ordinary procedure in terms of `cond'?" she asks. Alyssa's friend + Eva Lu Ator claims this can indeed be done, and she defines a new + version of `if': + + (define (new-if predicate then-clause else-clause) + (cond (predicate then-clause) + (else else-clause))) + + Eva demonstrates the program for Alyssa: + + (new-if (= 2 3) 0 5) + 5 + + (new-if (= 1 1) 0 5) + 0 + + Delighted, Alyssa uses `new-if' to rewrite the square-root program: + + (define (sqrt-iter guess x) + (new-if (good-enough? guess x) + guess + (sqrt-iter (improve guess x) + x))) + + What happens when Alyssa attempts to use this to compute square + roots? Explain. + + *Exercise 1.7:* The `good-enough?' test used in computing square + roots will not be very effective for finding the square roots of + very small numbers. Also, in real computers, arithmetic operations + are almost always performed with limited precision. This makes + our test inadequate for very large numbers. Explain these + statements, with examples showing how the test fails for small and + large numbers. An alternative strategy for implementing + `good-enough?' is to watch how `guess' changes from one iteration + to the next and to stop when the change is a very small fraction + of the guess. Design a square-root procedure that uses this kind + of end test. Does this work better for small and large numbers? + + *Exercise 1.8:* Newton's method for cube roots is based on the + fact that if y is an approximation to the cube root of x, then a + better approximation is given by the value + + x/y^2 + 2y + ---------- + 3 + + Use this formula to implement a cube-root procedure analogous to + the square-root procedure. (In section *Note 1-3-4:: we will see + how to implement Newton's method in general as an abstraction of + these square-root and cube-root procedures.) + + ---------- Footnotes ---------- + + (1) Declarative and imperative descriptions are intimately related, +as indeed are mathematics and computer science. For instance, to say +that the answer produced by a program is "correct" is to make a +declarative statement about the program. There is a large amount of +research aimed at establishing techniques for proving that programs are +correct, and much of the technical difficulty of this subject has to do +with negotiating the transition between imperative statements (from +which programs are constructed) and declarative statements (which can be +used to deduce things). In a related vein, an important current area in +programming-language design is the exploration of so-called very +high-level languages, in which one actually programs in terms of +declarative statements. The idea is to make interpreters sophisticated +enough so that, given "what is" knowledge specified by the programmer, +they can generate "how to" knowledge automatically. This cannot be +done in general, but there are important areas where progress has been +made. We shall revisit this idea in *Note Chapter 4::. + + (2) This square-root algorithm is actually a special case of +Newton's method, which is a general technique for finding roots of +equations. The square-root algorithm itself was developed by Heron of +Alexandria in the first century A.D. We will see how to express the +general Newton's method as a Lisp procedure in section *Note 1-3-4::. + + (3) We will usually give predicates names ending with question +marks, to help us remember that they are predicates. This is just a +stylistic convention. As far as the interpreter is concerned, the +question mark is just an ordinary character. + + (4) Observe that we express our initial guess as 1.0 rather than 1. +This would not make any difference in many Lisp implementations. MIT +Scheme, however, distinguishes between exact integers and decimal +values, and dividing two integers produces a rational number rather +than a decimal. For example, dividing 10 by 6 yields 5/3, while +dividing 10.0 by 6.0 yields 1.6666666666666667. (We will learn how to +implement arithmetic on rational numbers in section *Note 2-1-1::.) If +we start with an initial guess of 1 in our square-root program, and x +is an exact integer, all subsequent values produced in the square-root +computation will be rational numbers rather than decimals. Mixed +operations on rational numbers and decimals always yield decimals, so +starting with an initial guess of 1.0 forces all subsequent values to +be decimals. + + (5) Readers who are worried about the efficiency issues involved in +using procedure calls to implement iteration should note the remarks on +"tail recursion" in section *Note 1-2-1::. + + +File: sicp.info, Node: 1-1-8, Prev: 1-1-7, Up: 1-1 + +1.1.8 Procedures as Black-Box Abstractions +------------------------------------------ + +`Sqrt' is our first example of a process defined by a set of mutually +defined procedures. Notice that the definition of `sqrt-iter' is "recursive"; +that is, the procedure is defined in terms of itself. The idea of +being able to define a procedure in terms of itself may be disturbing; +it may seem unclear how such a "circular" definition could make sense +at all, much less specify a well-defined process to be carried out by a +computer. This will be addressed more carefully in section *Note +1-2::. But first let's consider some other important points +illustrated by the `sqrt' example. + + Observe that the problem of computing square roots breaks up +naturally into a number of subproblems: how to tell whether a guess is +good enough, how to improve a guess, and so on. Each of these tasks is +accomplished by a separate procedure. The entire `sqrt' program can be +viewed as a cluster of procedures (shown in *Note Figure 1-2::) that +mirrors the decomposition of the problem into subproblems. + + *Figure 1.2:* Procedural decomposition of the `sqrt' program. + + sqrt + | + sqrt-iter + / \ + good-enough improve + / \ | + square abs average + + The importance of this decomposition strategy is not simply that one +is dividing the program into parts. After all, we could take any large +program and divide it into parts--the first ten lines, the next ten +lines, the next ten lines, and so on. Rather, it is crucial that each +procedure accomplishes an identifiable task that can be used as a +module in defining other procedures. For example, when we define the +`good-enough?' procedure in terms of `square', we are able to regard +the `square' procedure as a "black box." We are not at that moment +concerned with _how_ the procedure computes its result, only with the +fact that it computes the square. The details of how the square is +computed can be suppressed, to be considered at a later time. Indeed, +as far as the `good-enough?' procedure is concerned, `square' is not +quite a procedure but rather an abstraction of a procedure, a so-called "procedural +abstraction". At this level of abstraction, any procedure that +computes the square is equally good. + + Thus, considering only the values they return, the following two +procedures for squaring a number should be indistinguishable. Each +takes a numerical argument and produces the square of that number as +the value.(1) + + (define (square x) (* x x)) + + (define (square x) + (exp (double (log x)))) + + (define (double x) (+ x x)) + + So a procedure definition should be able to suppress detail. The +users of the procedure may not have written the procedure themselves, +but may have obtained it from another programmer as a black box. A +user should not need to know how the procedure is implemented in order +to use it. + +Local names +........... + +One detail of a procedure's implementation that should not matter to +the user of the procedure is the implementer's choice of names for the +procedure's formal parameters. Thus, the following procedures should +not be distinguishable: + + (define (square x) (* x x)) + + (define (square y) (* y y)) + + This principle--that the meaning of a procedure should be +independent of the parameter names used by its author--seems on the +surface to be self-evident, but its consequences are profound. The +simplest consequence is that the parameter names of a procedure must be +local to the body of the procedure. For example, we used `square' in +the definition of `good-enough?' in our square-root procedure: + + (define (good-enough? guess x) + (< (abs (- (square guess) x)) 0.001)) + + The intention of the author of `good-enough?' is to determine if the +square of the first argument is within a given tolerance of the second +argument. We see that the author of `good-enough?' used the name +`guess' to refer to the first argument and `x' to refer to the second +argument. The argument of `square' is `guess'. If the author of +`square' used `x' (as above) to refer to that argument, we see that the +`x' in `good-enough?' must be a different `x' than the one in `square'. +Running the procedure `square' must not affect the value of `x' that +is used by `good-enough?', because that value of `x' may be needed by +`good-enough?' after `square' is done computing. + + If the parameters were not local to the bodies of their respective +procedures, then the parameter `x' in `square' could be confused with +the parameter `x' in `good-enough?', and the behavior of `good-enough?' +would depend upon which version of `square' we used. Thus, `square' +would not be the black box we desired. + + A formal parameter of a procedure has a very special role in the +procedure definition, in that it doesn't matter what name the formal +parameter has. Such a name is called a "bound variable", and we say +that the procedure definition "binds" its formal parameters. The +meaning of a procedure definition is unchanged if a bound variable is +consistently renamed throughout the definition.(2) If a variable is +not bound, we say that it is "free". The set of expressions for which +a binding defines a name is called the "scope" of that name. In a +procedure definition, the bound variables declared as the formal +parameters of the procedure have the body of the procedure as their +scope. + + In the definition of `good-enough?' above, `guess' and `x' are bound +variables but `<', `-', `abs', and `square' are free. The meaning of +`good-enough?' should be independent of the names we choose for `guess' +and `x' so long as they are distinct and different from `<', `-', +`abs', and `square'. (If we renamed `guess' to `abs' we would have +introduced a bug by "capturing" the variable `abs'. It would have +changed from free to bound.) The meaning of `good-enough?' is not +independent of the names of its free variables, however. It surely +depends upon the fact (external to this definition) that the symbol +`abs' names a procedure for computing the absolute value of a number. +`Good-enough?' will compute a different function if we substitute `cos' +for `abs' in its definition. + +Internal definitions and block structure +........................................ + +We have one kind of name isolation available to us so far: The formal +parameters of a procedure are local to the body of the procedure. The +square-root program illustrates another way in which we would like to +control the use of names. The existing program consists of separate +procedures: + + (define (sqrt x) + (sqrt-iter 1.0 x)) + + (define (sqrt-iter guess x) + (if (good-enough? guess x) + guess + (sqrt-iter (improve guess x) x))) + + (define (good-enough? guess x) + (< (abs (- (square guess) x)) 0.001)) + + (define (improve guess x) + (average guess (/ x guess))) + + The problem with this program is that the only procedure that is +important to users of `sqrt' is `sqrt'. The other procedures +(`sqrt-iter', `good-enough?', and `improve') only clutter up their +minds. They may not define any other procedure called `good-enough?' +as part of another program to work together with the square-root +program, because `sqrt' needs it. The problem is especially severe in +the construction of large systems by many separate programmers. For +example, in the construction of a large library of numerical +procedures, many numerical functions are computed as successive +approximations and thus might have procedures named `good-enough?' and +`improve' as auxiliary procedures. We would like to localize the +subprocedures, hiding them inside `sqrt' so that `sqrt' could coexist +with other successive approximations, each having its own private +`good-enough?' procedure. To make this possible, we allow a procedure +to have internal definitions that are local to that procedure. For +example, in the square-root problem we can write + + (define (sqrt x) + (define (good-enough? guess x) + (< (abs (- (square guess) x)) 0.001)) + (define (improve guess x) + (average guess (/ x guess))) + (define (sqrt-iter guess x) + (if (good-enough? guess x) + guess + (sqrt-iter (improve guess x) x))) + (sqrt-iter 1.0 x)) + + Such nesting of definitions, called "block structure", is basically +the right solution to the simplest name-packaging problem. But there +is a better idea lurking here. In addition to internalizing the +definitions of the auxiliary procedures, we can simplify them. Since +`x' is bound in the definition of `sqrt', the procedures +`good-enough?', `improve', and `sqrt-iter', which are defined +internally to `sqrt', are in the scope of `x'. Thus, it is not +necessary to pass `x' explicitly to each of these procedures. Instead, +we allow `x' to be a free variable in the internal definitions, as +shown below. Then `x' gets its value from the argument with which the +enclosing procedure `sqrt' is called. This discipline is called "lexical +scoping".(3) + + (define (sqrt x) + (define (good-enough? guess) + (< (abs (- (square guess) x)) 0.001)) + (define (improve guess) + (average guess (/ x guess))) + (define (sqrt-iter guess) + (if (good-enough? guess) + guess + (sqrt-iter (improve guess)))) + (sqrt-iter 1.0)) + + We will use block structure extensively to help us break up large +programs into tractable pieces.(4) The idea of block structure +originated with the programming language Algol 60. It appears in most +advanced programming languages and is an important tool for helping to +organize the construction of large programs. + + ---------- Footnotes ---------- + + (1) It is not even clear which of these procedures is a more +efficient implementation. This depends upon the hardware available. +There are machines for which the "obvious" implementation is the less +efficient one. Consider a machine that has extensive tables of +logarithms and antilogarithms stored in a very efficient manner. + + (2) The concept of consistent renaming is actually subtle and +difficult to define formally. Famous logicians have made embarrassing +errors here. + + (3) [Footnote 28] Lexical scoping dictates that free variables in a +procedure are taken to refer to bindings made by enclosing procedure +definitions; that is, they are looked up in the environment in which the +procedure was defined. We will see how this works in detail in chapter +3 when we study environments and the detailed behavior of the +interpreter. + + (4) Embedded definitions must come first in a procedure body. The +management is not responsible for the consequences of running programs +that intertwine definition and use. + + +File: sicp.info, Node: 1-2, Next: 1-3, Prev: 1-1, Up: Chapter 1 + +1.2 Procedures and the Processes They Generate +============================================== + +We have now considered the elements of programming: We have used +primitive arithmetic operations, we have combined these operations, and +we have abstracted these composite operations by defining them as +compound procedures. But that is not enough to enable us to say that +we know how to program. Our situation is analogous to that of someone +who has learned the rules for how the pieces move in chess but knows +nothing of typical openings, tactics, or strategy. Like the novice +chess player, we don't yet know the common patterns of usage in the +domain. We lack the knowledge of which moves are worth making (which +procedures are worth defining). We lack the experience to predict the +consequences of making a move (executing a procedure). + + The ability to visualize the consequences of the actions under +consideration is crucial to becoming an expert programmer, just as it +is in any synthetic, creative activity. In becoming an expert +photographer, for example, one must learn how to look at a scene and +know how dark each region will appear on a print for each possible +choice of exposure and development conditions. Only then can one +reason backward, planning framing, lighting, exposure, and development +to obtain the desired effects. So it is with programming, where we are +planning the course of action to be taken by a process and where we +control the process by means of a program. To become experts, we must +learn to visualize the processes generated by various types of +procedures. Only after we have developed such a skill can we learn to +reliably construct programs that exhibit the desired behavior. + + A procedure is a pattern for the "local evolution" of a computational +process. It specifies how each stage of the process is built upon the +previous stage. We would like to be able to make statements about the +overall, or "global", behavior of a process whose local evolution has +been specified by a procedure. This is very difficult to do in +general, but we can at least try to describe some typical patterns of +process evolution. + + In this section we will examine some common "shapes" for processes +generated by simple procedures. We will also investigate the rates at +which these processes consume the important computational resources of +time and space. The procedures we will consider are very simple. +Their role is like that played by test patterns in photography: as +oversimplified prototypical patterns, rather than practical examples in +their own right. + +* Menu: + +* 1-2-1:: Linear Recursion and Iteration +* 1-2-2:: Tree Recursion +* 1-2-3:: Orders of Growth +* 1-2-4:: Exponentiation +* 1-2-5:: Greatest Common Divisors +* 1-2-6:: Example: Testing for Primality + + +File: sicp.info, Node: 1-2-1, Next: 1-2-2, Prev: 1-2, Up: 1-2 + +1.2.1 Linear Recursion and Iteration +------------------------------------ + + *Figure 1.3:* A linear recursive process for computing 6!. + + (factorial 6) ------------------------. + (* 6 (factorial 5)) | + (* 6 (* 5 (factorial 4))) | + (* 6 (* 5 (* 4 (factorial 3)))) | + (* 6 (* 5 (* 4 (* 3 (factorial 2))))) | + (* 6 (* 5 (* 4 (* 3 (* 2 (factorial 1)))))) | + (* 6 (* 5 (* 4 (* 3 (* 2 1))))) | + (* 6 (* 5 (* 4 (* 3 2)))) | + (* 6 (* 5 (* 4 6))) | + (* 6 (* 5 24)) | + (* 6 120) | + 720 <-------------------------------' + +We begin by considering the factorial function, defined by + + n! = n * (n - 1) * (n - 2) ... 3 * 2 * 1 + + There are many ways to compute factorials. One way is to make use +of the observation that n! is equal to n times (n - 1)! for any positive +integer n: + + n! = n * [(n - 1) * (n - 2) ... 3 * 2 * 1] = n * (n - 1)! + + Thus, we can compute n! by computing (n - 1)! and multiplying the +result by n. If we add the stipulation that 1! is equal to 1, this +observation translates directly into a procedure: + + (define (factorial n) + (if (= n 1) + 1 + (* n (factorial (- n 1))))) + + We can use the substitution model of section *Note 1-1-5:: to watch +this procedure in action computing 6!, as shown in *Note Figure 1-3::. + + Now let's take a different perspective on computing factorials. We +could describe a rule for computing n! by specifying that we first +multiply 1 by 2, then multiply the result by 3, then by 4, and so on +until we reach n. More formally, we maintain a running product, +together with a counter that counts from 1 up to n. We can describe +the computation by saying that the counter and the product +simultaneously change from one step to the next according to the rule + + product <- counter ... product + + counter <- counter + 1 + +and stipulating that n! is the value of the product when the counter +exceeds n. + + *Figure 1.4:* A linear iterative process for computing 6!. + + (factorial 6) -----. + (fact-iter 1 1 6) | + (fact-iter 1 2 6) | + (fact-iter 2 3 6) | + (fact-iter 6 4 6) | + (fact-iter 24 5 6) | + (fact-iter 120 6 6) | + (fact-iter 720 7 6) V + 720 + + Once again, we can recast our description as a procedure for +computing factorials:(1) + + (define (factorial n) + (fact-iter 1 1 n)) + + (define (fact-iter product counter max-count) + (if (> counter max-count) + product + (fact-iter (* counter product) + (+ counter 1) + max-count))) + + As before, we can use the substitution model to visualize the +process of computing 6!, as shown in *Note Figure 1-4::. + + Compare the two processes. From one point of view, they seem hardly +different at all. Both compute the same mathematical function on the +same domain, and each requires a number of steps proportional to n to +compute n!. Indeed, both processes even carry out the same sequence of +multiplications, obtaining the same sequence of partial products. On +the other hand, when we consider the "shapes" of the two processes, we +find that they evolve quite differently. + + Consider the first process. The substitution model reveals a shape +of expansion followed by contraction, indicated by the arrow in *Note +Figure 1-3::. The expansion occurs as the process builds up a chain of operations +"deferred operations" (in this case, a chain of multiplications). The +contraction occurs as the operations are actually performed. This type +of process, characterized by a chain of deferred operations, is called +a "recursive process". Carrying out this process requires that the +interpreter keep track of the operations to be performed later on. In +the computation of n!, the length of the chain of deferred +multiplications, and hence the amount of information needed to keep +track of it, grows linearly with n (is proportional to n), just like +the number of steps. Such a process is called a "linear recursive +process". + + By contrast, the second process does not grow and shrink. At each +step, all we need to keep track of, for any n, are the current values +of the variables `product', `counter', and `max-count'. We call this an "iterative +process". In general, an iterative process is one whose state can be +summarized by a fixed number of "state variables", together with a +fixed rule that describes how the state variables should be updated as +the process moves from state to state and an (optional) end test that +specifies conditions under which the process should terminate. In +computing n!, the number of steps required grows linearly with n. Such +a process is called a "linear iterative process". + + The contrast between the two processes can be seen in another way. +In the iterative case, the program variables provide a complete +description of the state of the process at any point. If we stopped +the computation between steps, all we would need to do to resume the +computation is to supply the interpreter with the values of the three +program variables. Not so with the recursive process. In this case +there is some additional "hidden" information, maintained by the +interpreter and not contained in the program variables, which indicates +"where the process is" in negotiating the chain of deferred operations. +The longer the chain, the more information must be maintained.(2) + + In contrasting iteration and recursion, we must be careful not to +confuse the notion of a recursive "process" with the notion of a +recursive "procedure". When we describe a procedure as recursive, we +are referring to the syntactic fact that the procedure definition +refers (either directly or indirectly) to the procedure itself. But +when we describe a process as following a pattern that is, say, +linearly recursive, we are speaking about how the process evolves, not +about the syntax of how a procedure is written. It may seem disturbing +that we refer to a recursive procedure such as `fact-iter' as +generating an iterative process. However, the process really is +iterative: Its state is captured completely by its three state +variables, and an interpreter need keep track of only three variables +in order to execute the process. + + One reason that the distinction between process and procedure may be +confusing is that most implementations of common languages (including +Ada, Pascal, and C) are designed in such a way that the interpretation +of any recursive procedure consumes an amount of memory that grows with +the number of procedure calls, even when the process described is, in +principle, iterative. As a consequence, these languages can describe +iterative processes only by resorting to special-purpose "looping +constructs" such as `do', `repeat', `until', `for', and `while'. The +implementation of Scheme we shall consider in *Note Chapter 5:: does +not share this defect. It will execute an iterative process in +constant space, even if the iterative process is described by a +recursive procedure. An implementation with this property is called "tail-recursive". +With a tail-recursive implementation, iteration can be expressed using +the ordinary procedure call mechanism, so that special iteration +constructs are useful only as syntactic sugar.(3) + + *Exercise 1.9:* Each of the following two procedures defines a + method for adding two positive integers in terms of the procedures + `inc', which increments its argument by 1, and `dec', which + decrements its argument by 1. + + (define (+ a b) + (if (= a 0) + b + (inc (+ (dec a) b)))) + + (define (+ a b) + (if (= a 0) + b + (+ (dec a) (inc b)))) + + Using the substitution model, illustrate the process generated by + each procedure in evaluating `(+ 4 5)'. Are these processes + iterative or recursive? + + *Exercise 1.10:* The following procedure computes a mathematical + function called Ackermann's function. + + (define (A x y) + (cond ((= y 0) 0) + ((= x 0) (* 2 y)) + ((= y 1) 2) + (else (A (- x 1) + (A x (- y 1)))))) + + What are the values of the following expressions? + + (A 1 10) + + (A 2 4) + + (A 3 3) + + Consider the following procedures, where `A' is the procedure + defined above: + + (define (f n) (A 0 n)) + + (define (g n) (A 1 n)) + + (define (h n) (A 2 n)) + + (define (k n) (* 5 n n)) + + Give concise mathematical definitions for the functions computed + by the procedures `f', `g', and `h' for positive integer values of + n. For example, `(k n)' computes 5n^2. + + ---------- Footnotes ---------- + + (1) In a real program we would probably use the block structure +introduced in the last section to hide the definition of `fact-iter': + + (define (factorial n) + (define (iter product counter) + (if (> counter n) + product + (iter (* counter product) + (+ counter 1)))) + (iter 1 1)) + + We avoided doing this here so as to minimize the number of things to +think about at once. + + (2) When we discuss the implementation of procedures on register +machines in *Note Chapter 5::, we will see that any iterative process +can be realized "in hardware" as a machine that has a fixed set of +registers and no auxiliary memory. In contrast, realizing a recursive +process requires a machine that uses an auxiliary data structure known +as a "stack". + + (3) Tail recursion has long been known as a compiler optimization +trick. A coherent semantic basis for tail recursion was provided by +Carl Hewitt (1977), who explained it in terms of the "message-passing" +model of computation that we shall discuss in *Note Chapter 3::. +Inspired by this, Gerald Jay Sussman and Guy Lewis Steele Jr. (see +Steele 1975) constructed a tail-recursive interpreter for Scheme. +Steele later showed how tail recursion is a consequence of the natural +way to compile procedure calls (Steele 1977). The IEEE standard for +Scheme requires that Scheme implementations be tail-recursive. + + +File: sicp.info, Node: 1-2-2, Next: 1-2-3, Prev: 1-2-1, Up: 1-2 + +1.2.2 Tree Recursion +-------------------- + +Another common pattern of computation is called "tree recursion". As +an example, consider computing the sequence of Fibonacci numbers, in +which each number is the sum of the preceding two: + + 0, 1, 1, 2, 3, 4, 8, 13, 21, ... + + In general, the Fibonacci numbers can be defined by the rule + + / + | 0 if n = 0 + Fib(n) = < 1 if n = 1 + | Fib(n - 1) + Fib(n - 2) otherwise + \ + + We can immediately translate this definition into a recursive +procedure for computing Fibonacci numbers: + + (define (fib n) + (cond ((= n 0) 0) + ((= n 1) 1) + (else (+ (fib (- n 1)) + (fib (- n 2)))))) + + *Figure 1.5:* The tree-recursive process generated in computing + `(fib 5)'. + + ..<............ fib5 <.......... + ... ___________/ \___________ . + ... / . ..... \ . + .. fib4 . . . . . fib3 . + .. ____/. \____ .. . __/ \__ . + .. / . . .. \ . .. / . . \ . + .. fib3 . . fib2 . . fib2 . . fib1 . + .. / . \ . . / \ . . / \ ... . | . + .. / . . \ . . / . \ . . / . \ . . 1 . + . fib2 . . fib1. .fib1 . fib0 . .fib1. . fib0 . . . + . / \ . . | . . | . . | . . | . . | . .> + V / . \ . 1 . . 1 . . 0 . . 1 . . 0 .. + . fib1 .. fib0.. . . . . . V . .. . + . | . . | . .> .>. . . ..>. .> + . 1 . . 0 . + . . . . + .>. .. + + Consider the pattern of this computation. To compute `(fib 5)', we +compute `(fib 4)' and `(fib 3)'. To compute `(fib 4)', we compute +`(fib 3)' and `(fib 2)'. In general, the evolved process looks like a +tree, as shown in *Note Figure 1-5::. Notice that the branches split +into two at each level (except at the bottom); this reflects the fact +that the `fib' procedure calls itself twice each time it is invoked. + + This procedure is instructive as a prototypical tree recursion, but +it is a terrible way to compute Fibonacci numbers because it does so +much redundant computation. Notice in *Note Figure 1-5:: that the +entire computation of `(fib 3)'--almost half the work--is duplicated. +In fact, it is not hard to show that the number of times the procedure +will compute `(fib 1)' or `(fib 0)' (the number of leaves in the above +tree, in general) is precisely _Fib_(n + 1). To get an idea of how bad +this is, one can show that the value of _Fib_(n) grows exponentially +with n. More precisely (see *Note Exercise 1-13::), _Fib_(n) is the +closest integer to [phi]^n /[sqrt](5), where + + [phi] = (1 + [sqrt]5)/2 ~= 1.6180 + +is the "golden ratio", which satisfies the equation + + [phi]^2 = [phi] + 1 + + Thus, the process uses a number of steps that grows exponentially +with the input. On the other hand, the space required grows only +linearly with the input, because we need keep track only of which nodes +are above us in the tree at any point in the computation. In general, +the number of steps required by a tree-recursive process will be +proportional to the number of nodes in the tree, while the space +required will be proportional to the maximum depth of the tree. + + We can also formulate an iterative process for computing the +Fibonacci numbers. The idea is to use a pair of integers a and b, +initialized to _Fib_(1) = 1 and _Fib_(0) = 0, and to repeatedly apply +the simultaneous transformations + + a <- a + b + b <- a + +It is not hard to show that, after applying this transformation n times, +a and b will be equal, respectively, to _Fib_(n + 1) and _Fib_(n). +Thus, we can compute Fibonacci numbers iteratively using the procedure + + (define (fib n) + (fib-iter 1 0 n)) + + (define (fib-iter a b count) + (if (= count 0) + b + (fib-iter (+ a b) a (- count 1)))) + + This second method for computing _Fib_(n) is a linear iteration. The +difference in number of steps required by the two methods--one linear in +n, one growing as fast as _Fib_(n) itself--is enormous, even for small +inputs. + + One should not conclude from this that tree-recursive processes are +useless. When we consider processes that operate on hierarchically +structured data rather than numbers, we will find that tree recursion +is a natural and powerful tool.(1) But even in numerical operations, +tree-recursive processes can be useful in helping us to understand and +design programs. For instance, although the first `fib' procedure is +much less efficient than the second one, it is more straightforward, +being little more than a translation into Lisp of the definition of the +Fibonacci sequence. To formulate the iterative algorithm required +noticing that the computation could be recast as an iteration with +three state variables. + +Example: Counting change +........................ + +It takes only a bit of cleverness to come up with the iterative +Fibonacci algorithm. In contrast, consider the following problem: How +many different ways can we make change of $ 1.00, given half-dollars, +quarters, dimes, nickels, and pennies? More generally, can we write a +procedure to compute the number of ways to change any given amount of +money? + + This problem has a simple solution as a recursive procedure. +Suppose we think of the types of coins available as arranged in some +order. Then the following relation holds: + + The number of ways to change amount a using n kinds of coins equals + + * the number of ways to change amount a using all but the first kind + of coin, plus + + * the number of ways to change amount a - d using all n kinds of + coins, where d is the denomination of the first kind of coin. + + + To see why this is true, observe that the ways to make change can be +divided into two groups: those that do not use any of the first kind of +coin, and those that do. Therefore, the total number of ways to make +change for some amount is equal to the number of ways to make change +for the amount without using any of the first kind of coin, plus the +number of ways to make change assuming that we do use the first kind of +coin. But the latter number is equal to the number of ways to make +change for the amount that remains after using a coin of the first kind. + + Thus, we can recursively reduce the problem of changing a given +amount to the problem of changing smaller amounts using fewer kinds of +coins. Consider this reduction rule carefully, and convince yourself +that we can use it to describe an algorithm if we specify the following +degenerate cases:(2) + + * If a is exactly 0, we should count that as 1 way to make change. + + * If a is less than 0, we should count that as 0 ways to make change. + + * If n is 0, we should count that as 0 ways to make change. + + + We can easily translate this description into a recursive procedure: + + (define (count-change amount) + (cc amount 5)) + + (define (cc amount kinds-of-coins) + (cond ((= amount 0) 1) + ((or (< amount 0) (= kinds-of-coins 0)) 0) + (else (+ (cc amount + (- kinds-of-coins 1)) + (cc (- amount + (first-denomination kinds-of-coins)) + kinds-of-coins))))) + + (define (first-denomination kinds-of-coins) + (cond ((= kinds-of-coins 1) 1) + ((= kinds-of-coins 2) 5) + ((= kinds-of-coins 3) 10) + ((= kinds-of-coins 4) 25) + ((= kinds-of-coins 5) 50))) + + (The `first-denomination' procedure takes as input the number of +kinds of coins available and returns the denomination of the first +kind. Here we are thinking of the coins as arranged in order from +largest to smallest, but any order would do as well.) We can now +answer our original question about changing a dollar: + + (count-change 100) + 292 + + `Count-change' generates a tree-recursive process with redundancies +similar to those in our first implementation of `fib'. (It will take +quite a while for that 292 to be computed.) On the other hand, it is +not obvious how to design a better algorithm for computing the result, +and we leave this problem as a challenge. The observation that a +tree-recursive process may be highly inefficient but often easy to +specify and understand has led people to propose that one could get the +best of both worlds by designing a "smart compiler" that could +transform tree-recursive procedures into more efficient procedures that +compute the same result.(3) + + *Exercise 1.11:* A function f is defined by the rule that f(n) = n + if n<3 and f(n) = f(n - 1) + 2f(n - 2) + 3f(n - 3) if n>= 3. + Write a procedure that computes f by means of a recursive process. + Write a procedure that computes f by means of an iterative + process. + + *Exercise 1.12:* The following pattern of numbers is called "Pascal's + triangle". + + 1 + 1 1 + 1 2 1 + 1 3 3 1 + 1 4 6 4 1 + + The numbers at the edge of the triangle are all 1, and each number + inside the triangle is the sum of the two numbers above it.(4) + Write a procedure that computes elements of Pascal's triangle by + means of a recursive process. + + *Exercise 1.13:* Prove that _Fib_(n) is the closest integer to + [phi]^n/[sqrt](5), where [phi] = (1 + [sqrt](5))/2. Hint: Let + [illegiblesymbol] = (1 - [sqrt](5))/2. Use induction and the + definition of the Fibonacci numbers (see section *Note 1-2-2::) to + prove that _Fib_(n) = ([phi]^n - [illegiblesymbol]^n)/[sqrt](5). + + ---------- Footnotes ---------- + + (1) An example of this was hinted at in section *Note 1-1-3::: The +interpreter itself evaluates expressions using a tree-recursive process. + + (2) For example, work through in detail how the reduction rule +applies to the problem of making change for 10 cents using pennies and +nickels. + + (3) One approach to coping with redundant computations is to arrange +matters so that we automatically construct a table of values as they +are computed. Each time we are asked to apply the procedure to some +argument, we first look to see if the value is already stored in the +table, in which case we avoid performing the redundant computation. +This strategy, known as "tabulation" or "memoization", can be +implemented in a straightforward way. Tabulation can sometimes be used +to transform processes that require an exponential number of steps +(such as `count-change') into processes whose space and time +requirements grow linearly with the input. See *Note Exercise 3-27::. + + (4) The elements of Pascal's triangle are called the "binomial +coefficients", because the nth row consists of the coefficients of the +terms in the expansion of (x + y)^n. This pattern for computing the +coefficients appeared in Blaise Pascal's 1653 seminal work on +probability theory, `Traite' du triangle arithme'tique'. According to +Knuth (1973), the same pattern appears in the `Szu-yuen Yu"-chien' +("The Precious Mirror of the Four Elements"), published by the Chinese +mathematician Chu Shih-chieh in 1303, in the works of the +twelfth-century Persian poet and mathematician Omar Khayyam, and in the +works of the twelfth-century Hindu mathematician Bha'scara A'cha'rya. + + +File: sicp.info, Node: 1-2-3, Next: 1-2-4, Prev: 1-2-2, Up: 1-2 + +1.2.3 Orders of Growth +---------------------- + +The previous examples illustrate that processes can differ considerably +in the rates at which they consume computational resources. One +convenient way to describe this difference is to use the notion of "order +of growth" to obtain a gross measure of the resources required by a +process as the inputs become larger. + + Let n be a parameter that measures the size of the problem, and let +R(n) be the amount of resources the process requires for a problem of +size n. In our previous examples we took n to be the number for which +a given function is to be computed, but there are other possibilities. +For instance, if our goal is to compute an approximation to the square +root of a number, we might take n to be the number of digits accuracy +required. For matrix multiplication we might take n to be the number +of rows in the matrices. In general there are a number of properties +of the problem with respect to which it will be desirable to analyze a +given process. Similarly, R(n) might measure the number of internal +storage registers used, the number of elementary machine operations +performed, and so on. In computers that do only a fixed number of +operations at a time, the time required will be proportional to the +number of elementary machine operations performed. + + We say that R(n) has order of growth [theta](f(n)), written R(n) = +[theta](f(n)) (pronounced "theta of f(n)"), if there are positive +constants k_1 and k_2 independent of n such that + + k_1 f(n) <= R(n) <= k_2 f(n) + +for any sufficiently large value of n. (In other words, for large n, +the value R(n) is sandwiched between k_1f(n) and k_2f(n).) + + For instance, with the linear recursive process for computing +factorial described in section *Note 1-2-1:: the number of steps grows +proportionally to the input n. Thus, the steps required for this +process grows as [theta](n). We also saw that the space required grows +as [theta](n). For the iterative factorial, the number of steps is +still [theta](n) but the space is [theta](1)--that is, constant.(1) The +tree-recursive Fibonacci computation requires [theta]([phi]^n) steps +and space [theta](n), where [phi] is the golden ratio described in +section *Note 1-2-2::. + + Orders of growth provide only a crude description of the behavior of +a process. For example, a process requiring n^2 steps and a process +requiring 1000n^2 steps and a process requiring 3n^2 + 10n + 17 steps +all have [theta](n^2) order of growth. On the other hand, order of +growth provides a useful indication of how we may expect the behavior +of the process to change as we change the size of the problem. For a +[theta](n) (linear) process, doubling the size will roughly double the +amount of resources used. For an exponential process, each increment +in problem size will multiply the resource utilization by a constant +factor. In the remainder of section *Note 1-2:: we will examine two +algorithms whose order of growth is logarithmic, so that doubling the +problem size increases the resource requirement by a constant amount. + + *Exercise 1.14:* Draw the tree illustrating the process generated + by the `count-change' procedure of section *Note 1-2-2:: in making + change for 11 cents. What are the orders of growth of the space + and number of steps used by this process as the amount to be + changed increases? + + *Exercise 1.15:* The sine of an angle (specified in radians) can + be computed by making use of the approximation `sin' xapprox x if + x is sufficiently small, and the trigonometric identity + + x x + sin x = 3 sin --- - 4 sin^3 --- + 3 3 + + to reduce the size of the argument of `sin'. (For purposes of this + exercise an angle is considered "sufficiently small" if its + magnitude is not greater than 0.1 radians.) These ideas are + incorporated in the following procedures: + + (define (cube x) (* x x x)) + + (define (p x) (- (* 3 x) (* 4 (cube x)))) + + (define (sine angle) + (if (not (> (abs angle) 0.1)) + angle + (p (sine (/ angle 3.0))))) + + a. How many times is the procedure `p' applied when `(sine + 12.15)' is evaluated? + + b. What is the order of growth in space and number of steps (as + a function of a) used by the process generated by the `sine' + procedure when `(sine a)' is evaluated? + + + ---------- Footnotes ---------- + + (1) These statements mask a great deal of oversimplification. For +instance, if we count process steps as "machine operations" we are +making the assumption that the number of machine operations needed to +perform, say, a multiplication is independent of the size of the +numbers to be multiplied, which is false if the numbers are +sufficiently large. Similar remarks hold for the estimates of space. +Like the design and description of a process, the analysis of a process +can be carried out at various levels of abstraction. + + +File: sicp.info, Node: 1-2-4, Next: 1-2-5, Prev: 1-2-3, Up: 1-2 + +1.2.4 Exponentiation +-------------------- + +Consider the problem of computing the exponential of a given number. +We would like a procedure that takes as arguments a base b and a +positive integer exponent n and computes b^n. One way to do this is +via the recursive definition + + b^n = b * b^(n - 1) + b^0 = 1 + +which translates readily into the procedure + + (define (expt b n) + (if (= n 0) + 1 + (* b (expt b (- n 1))))) + + This is a linear recursive process, which requires [theta](n) steps +and [theta](n) space. Just as with factorial, we can readily formulate +an equivalent linear iteration: + + (define (expt b n) + (expt-iter b n 1)) + + (define (expt-iter b counter product) + (if (= counter 0) + product + (expt-iter b + (- counter 1) + (* b product)))) + + This version requires [theta](n) steps and [theta](1) space. + + We can compute exponentials in fewer steps by using successive +squaring. For instance, rather than computing b^8 as + + b * (b * (b * (b * (b * (b * (b * b)))))) + +we can compute it using three multiplications: + + b^2 = b * b + b^4 = b^2 * b^2 + b^8 = b^4 * b^4 + + This method works fine for exponents that are powers of 2. We can +also take advantage of successive squaring in computing exponentials in +general if we use the rule + + b^n = (b^(b/2))^2 if n is even + b^n = b * b^(n - 1) if n is odd + + We can express this method as a procedure: + + (define (fast-expt b n) + (cond ((= n 0) 1) + ((even? n) (square (fast-expt b (/ n 2)))) + (else (* b (fast-expt b (- n 1)))))) + +where the predicate to test whether an integer is even is defined in +terms of the primitive procedure `remainder' by + + (define (even? n) + (= (remainder n 2) 0)) + + The process evolved by `fast-expt' grows logarithmically with n in +both space and number of steps. To see this, observe that computing +b^(2n) using `fast-expt' requires only one more multiplication than +computing b^n. The size of the exponent we can compute therefore +doubles (approximately) with every new multiplication we are allowed. +Thus, the number of multiplications required for an exponent of n grows +about as fast as the logarithm of n to the base 2. The process has +[theta](`log' n) growth.(1) + + The difference between [theta](`log' n) growth and [theta](n) growth +becomes striking as n becomes large. For example, `fast-expt' for n = +1000 requires only 14 multiplications.(2) It is also possible to use +the idea of successive squaring to devise an iterative algorithm that +computes exponentials with a logarithmic number of steps (see *Note +Exercise 1-16::), although, as is often the case with iterative +algorithms, this is not written down so straightforwardly as the +recursive algorithm.(3) + + *Exercise 1.16:* Design a procedure that evolves an iterative + exponentiation process that uses successive squaring and uses a + logarithmic number of steps, as does `fast-expt'. (Hint: Using the + observation that (b^(n/2))^2 = (b^2)^(n/2), keep, along with the + exponent n and the base b, an additional state variable a, and + define the state transformation in such a way that the product a + b^n is unchanged from state to state. At the beginning of the + process a is taken to be 1, and the answer is given by the value + of a at the end of the process. In general, the technique of + defining an "invariant quantity" that remains unchanged from state + to state is a powerful way to think about the design of iterative + algorithms.) + + *Exercise 1.17:* The exponentiation algorithms in this section are + based on performing exponentiation by means of repeated + multiplication. In a similar way, one can perform integer + multiplication by means of repeated addition. The following + multiplication procedure (in which it is assumed that our language + can only add, not multiply) is analogous to the `expt' procedure: + + (define (* a b) + (if (= b 0) + 0 + (+ a (* a (- b 1))))) + + This algorithm takes a number of steps that is linear in `b'. Now + suppose we include, together with addition, operations `double', + which doubles an integer, and `halve', which divides an (even) + integer by 2. Using these, design a multiplication procedure + analogous to `fast-expt' that uses a logarithmic number of steps. + + *Exercise 1.18:* Using the results of *Note Exercise 1-16:: and + *Note Exercise 1-17::, devise a procedure that generates an + iterative process for multiplying two integers in terms of adding, + doubling, and halving and uses a logarithmic number of steps.(4) + + *Exercise 1.19:* There is a clever algorithm for computing the + Fibonacci numbers in a logarithmic number of steps. Recall the + transformation of the state variables a and b in the `fib-iter' + process of section *Note 1-2-2::: a <- a + b and b <- a. Call + this transformation T, and observe that applying T over and over + again n times, starting with 1 and 0, produces the pair _Fib_(n + + 1) and _Fib_(n). In other words, the Fibonacci numbers are + produced by applying T^n, the nth power of the transformation T, + starting with the pair (1,0). Now consider T to be the special + case of p = 0 and q = 1 in a family of transformations T_(pq), + where T_(pq) transforms the pair (a,b) according to a <- bq + aq + + ap and b <- bp + aq. Show that if we apply such a transformation + T_(pq) twice, the effect is the same as using a single + transformation T_(p'q') of the same form, and compute p' and q' in + terms of p and q. This gives us an explicit way to square these + transformations, and thus we can compute T^n using successive + squaring, as in the `fast-expt' procedure. Put this all together + to complete the following procedure, which runs in a logarithmic + number of steps:(5) + + (define (fib n) + (fib-iter 1 0 0 1 n)) + + (define (fib-iter a b p q count) + (cond ((= count 0) b) + ((even? count) + (fib-iter a + b + ; compute p' + ; compute q' + (/ count 2))) + (else (fib-iter (+ (* b q) (* a q) (* a p)) + (+ (* b p) (* a q)) + p + q + (- count 1))))) + + ---------- Footnotes ---------- + + (1) More precisely, the number of multiplications required is equal +to 1 less than the log base 2 of n plus the number of ones in the +binary representation of n. This total is always less than twice the +log base 2 of n. The arbitrary constants k_1 and k_2 in the definition +of order notation imply that, for a logarithmic process, the base to +which logarithms are taken does not matter, so all such processes are +described as [theta](`log' n). + + (2) You may wonder why anyone would care about raising numbers to +the 1000th power. See section *Note 1-2-6::. + + (3) This iterative algorithm is ancient. It appears in the +`Chandah-sutra' by A'cha'rya Pingala, written before 200 B.C. See Knuth +1981, section 4.6.3, for a full discussion and analysis of this and +other methods of exponentiation. + + (4) This algorithm, which is sometimes known as the "Russian peasant +method" of multiplication, is ancient. Examples of its use are found +in the Rhind Papyrus, one of the two oldest mathematical documents in +existence, written about 1700 B.C. (and copied from an even older +document) by an Egyptian scribe named A'h-mose. + + (5) This exercise was suggested to us by Joe Stoy, based on an +example in Kaldewaij 1990. + + +File: sicp.info, Node: 1-2-5, Next: 1-2-6, Prev: 1-2-4, Up: 1-2 + +1.2.5 Greatest Common Divisors +------------------------------ + +The greatest common divisor (GCD) of two integers a and b is defined to +be the largest integer that divides both a and b with no remainder. +For example, the GCD of 16 and 28 is 4. In *Note Chapter 2::, when we +investigate how to implement rational-number arithmetic, we will need +to be able to compute GCDs in order to reduce rational numbers to +lowest terms. (To reduce a rational number to lowest terms, we must +divide both the numerator and the denominator by their GCD. For +example, 16/28 reduces to 4/7.) One way to find the GCD of two +integers is to factor them and search for common factors, but there is +a famous algorithm that is much more efficient. + + The idea of the algorithm is based on the observation that, if r is +the remainder when a is divided by b, then the common divisors of a and +b are precisely the same as the common divisors of b and r. Thus, we +can use the equation + + GCD(a,b) = GCD(b,r) + +to successively reduce the problem of computing a GCD to the problem of +computing the GCD of smaller and smaller pairs of integers. For +example, + + GCD(206,40) = GCD(40,6) + = GCD(6,4) + = GCD(4,2) + = GCD(2,0) + = 2 + +reduces GCD(206,40) to GCD(2,0), which is 2. It is possible to show +that starting with any two positive integers and performing repeated +reductions will always eventually produce a pair where the second +number is 0. Then the GCD is the other number in the pair. This +method for computing the GCD is known as Algorithm "Euclid's +Algorithm".(1) + + It is easy to express Euclid's Algorithm as a procedure: + + (define (gcd a b) + (if (= b 0) + a + (gcd b (remainder a b)))) + + This generates an iterative process, whose number of steps grows as +the logarithm of the numbers involved. + + The fact that the number of steps required by Euclid's Algorithm has +logarithmic growth bears an interesting relation to the Fibonacci +numbers: + + *Lame''s Theorem:* If Euclid's Algorithm requires k steps to + compute the GCD of some pair, then the smaller number in the pair + must be greater than or equal to the kth Fibonacci number.(2) + + We can use this theorem to get an order-of-growth estimate for +Euclid's Algorithm. Let n be the smaller of the two inputs to the +procedure. If the process takes k steps, then we must have n>= _Fib_(k) +approx [phi]^k/[sqrt](5). Therefore the number of steps k grows as the +logarithm (to the base [phi]) of n. Hence, the order of growth is +[theta](`log' n). + + *Exercise 1.20:* The process that a procedure generates is of + course dependent on the rules used by the interpreter. As an + example, consider the iterative `gcd' procedure given above. + Suppose we were to interpret this procedure using normal-order + evaluation, as discussed in section *Note 1-1-5::. (The + normal-order-evaluation rule for `if' is described in *Note + Exercise 1-5::.) Using the substitution method (for normal + order), illustrate the process generated in evaluating `(gcd 206 + 40)' and indicate the `remainder' operations that are actually + performed. How many `remainder' operations are actually performed + in the normal-order evaluation of `(gcd 206 40)'? In the + applicative-order evaluation? + + ---------- Footnotes ---------- + + (1) Euclid's Algorithm is so called because it appears in Euclid's +`Elements' (Book 7, ca. 300 B.C.). According to Knuth (1973), it can +be considered the oldest known nontrivial algorithm. The ancient +Egyptian method of multiplication (*Note Exercise 1-18::) is surely +older, but, as Knuth explains, Euclid's algorithm is the oldest known +to have been presented as a general algorithm, rather than as a set of +illustrative examples. + + (2) This theorem was proved in 1845 by Gabriel Lame', a French +mathematician and engineer known chiefly for his contributions to +mathematical physics. To prove the theorem, we consider pairs (a_k +,b_k), where a_k>= b_k, for which Euclid's Algorithm terminates in k +steps. The proof is based on the claim that, if (a_(k+1), b_(k+1)) -> +(a_k, b_k) -> (a_(k-1), b_(k-1)) are three successive pairs in the +reduction process, then we must have b_(k+1)>= b_k + b_(k-1). To +verify the claim, consider that a reduction step is defined by applying +the transformation a_(k-1) = b_k, b_(k-1) = remainder of a_k divided by +b_k. The second equation means that a_k = qb_k + b_(k-1) for some +positive integer q. And since q must be at least 1 we have a_k = qb_k ++ b_(k-1) >= b_k + b_(k-1). But in the previous reduction step we have +b_(k+1) = a_k. Therefore, b_(k+1) = a_k>= b_k + b_(k-1). This verifies +the claim. Now we can prove the theorem by induction on k, the number +of steps that the algorithm requires to terminate. The result is true +for k = 1, since this merely requires that b be at least as large as +_Fib_(1) = 1. Now, assume that the result is true for all integers +less than or equal to k and establish the result for k + 1. Let +(a_(k+1), b_(k+1)) -> (a_k, b_k) -> (a_k-1), b_(k-1)) be successive +pairs in the reduction process. By our induction hypotheses, we have +b_(k-1) >= _Fib_(k - 1) and b_k >= _Fib_(k). Thus, applying the claim +we just proved together with the definition of the Fibonacci numbers +gives b_(k+1) >= b_k + b_(k-1) >= _Fib_(k) + _Fib_(k - 1) = _Fib_(k + +1), which completes the proof of Lame''s Theorem. + + +File: sicp.info, Node: 1-2-6, Prev: 1-2-5, Up: 1-2 + +1.2.6 Example: Testing for Primality +------------------------------------ + +This section describes two methods for checking the primality of an +integer n, one with order of growth [theta](_[sqrt]_(n)), and a +"probabilistic" algorithm with order of growth [theta](`log' n). The +exercises at the end of this section suggest programming projects based +on these algorithms. + +Searching for divisors +...................... + +Since ancient times, mathematicians have been fascinated by problems +concerning prime numbers, and many people have worked on the problem of +determining ways to test if numbers are prime. One way to test if a +number is prime is to find the number's divisors. The following +program finds the smallest integral divisor (greater than 1) of a given +number n. It does this in a straightforward way, by testing n for +divisibility by successive integers starting with 2. + + (define (smallest-divisor n) + (find-divisor n 2)) + + (define (find-divisor n test-divisor) + (cond ((> (square test-divisor) n) n) + ((divides? test-divisor n) test-divisor) + (else (find-divisor n (+ test-divisor 1))))) + + (define (divides? a b) + (= (remainder b a) 0)) + + We can test whether a number is prime as follows: n is prime if and +only if n is its own smallest divisor. + + (define (prime? n) + (= n (smallest-divisor n))) + + The end test for `find-divisor' is based on the fact that if n is not +prime it must have a divisor less than or equal to _[sqrt]_(n).(1) +This means that the algorithm need only test divisors between 1 and +_[sqrt]_(n). Consequently, the number of steps required to identify n +as prime will have order of growth [theta](_[sqrt]_(n)). + +The Fermat test +............... + +The [theta](`log' n) primality test is based on a result from number +theory known as Fermat's Little Theorem.(2) + + *Fermat's Little Theorem:* If n is a prime number and a is any + positive integer less than n, then a raised to the nth power is + congruent to a modulo n. + + (Two numbers are said to be "congruent modulo" n if they both have +the same remainder when divided by n. The remainder of a number a when +divided by n is also referred to as the "remainder of" a "modulo" n, or +simply as a "modulo" n.) + + If n is not prime, then, in general, most of the numbers a< n will +not satisfy the above relation. This leads to the following algorithm +for testing primality: Given a number n, pick a random number a < n and +compute the remainder of a^n modulo n. If the result is not equal to +a, then n is certainly not prime. If it is a, then chances are good +that n is prime. Now pick another random number a and test it with the +same method. If it also satisfies the equation, then we can be even +more confident that n is prime. By trying more and more values of a, +we can increase our confidence in the result. This algorithm is known +as the Fermat test. + + To implement the Fermat test, we need a procedure that computes the +exponential of a number modulo another number: + + (define (expmod base exp m) + (cond ((= exp 0) 1) + ((even? exp) + (remainder (square (expmod base (/ exp 2) m)) + m)) + (else + (remainder (* base (expmod base (- exp 1) m)) + m)))) + + This is very similar to the `fast-expt' procedure of section *Note +1-2-4::. It uses successive squaring, so that the number of steps +grows logarithmically with the exponent.(3) + + The Fermat test is performed by choosing at random a number a +between 1 and n - 1 inclusive and checking whether the remainder modulo +n of the nth power of a is equal to a. The random number a is chosen +using the procedure `random', which we assume is included as a primitive +in Scheme. `Random' returns a nonnegative integer less than its integer +input. Hence, to obtain a random number between 1 and n - 1, we call +`random' with an input of n - 1 and add 1 to the result: + + (define (fermat-test n) + (define (try-it a) + (= (expmod a n n) a)) + (try-it (+ 1 (random (- n 1))))) + + The following procedure runs the test a given number of times, as +specified by a parameter. Its value is true if the test succeeds every +time, and false otherwise. + + (define (fast-prime? n times) + (cond ((= times 0) true) + ((fermat-test n) (fast-prime? n (- times 1))) + (else false))) + +Probabilistic methods +..................... + +The Fermat test differs in character from most familiar algorithms, in +which one computes an answer that is guaranteed to be correct. Here, +the answer obtained is only probably correct. More precisely, if n +ever fails the Fermat test, we can be certain that n is not prime. But +the fact that n passes the test, while an extremely strong indication, +is still not a guarantee that n is prime. What we would like to say is +that for any number n, if we perform the test enough times and find +that n always passes the test, then the probability of error in our +primality test can be made as small as we like. + + Unfortunately, this assertion is not quite correct. There do exist +numbers that fool the Fermat test: numbers n that are not prime and yet +have the property that a^n is congruent to a modulo n for all integers +a < n. Such numbers are extremely rare, so the Fermat test is quite +reliable in practice.(4) + + There are variations of the Fermat test that cannot be fooled. In +these tests, as with the Fermat method, one tests the primality of an +integer n by choosing a random integer a a b) + 0 + (+ a (sum-integers (+ a 1) b)))) + + The second computes the sum of the cubes of the integers in the +given range: + + (define (sum-cubes a b) + (if (> a b) + 0 + (+ (cube a) (sum-cubes (+ a 1) b)))) + + The third computes the sum of a sequence of terms in the series + + 1 1 1 + ----- + ----- + ------ + ... + 1 * 3 5 * 7 9 * 11 + +which converges to [pi]/8 (very slowly):(1) + + (define (pi-sum a b) + (if (> a b) + 0 + (+ (/ 1.0 (* a (+ a 2))) (pi-sum (+ a 4) b)))) + + These three procedures clearly share a common underlying pattern. +They are for the most part identical, differing only in the name of the +procedure, the function of `a' used to compute the term to be added, +and the function that provides the next value of `a'. We could +generate each of the procedures by filling in slots in the same +template: + + (define ( a b) + (if (> a b) + 0 + (+ ( a) + ( ( a) b)))) + + The presence of such a common pattern is strong evidence that there +is a useful abstraction waiting to be brought to the surface. Indeed, +mathematicians long ago identified the abstraction of "summation of a +series" and invented "sigma notation," for example + + b + --- + > f(n) = f(a) + ... + f(b) + --- + n=a + +to express this concept. The power of sigma notation is that it allows +mathematicians to deal with the concept of summation itself rather than +only with particular sums--for example, to formulate general results +about sums that are independent of the particular series being summed. + + Similarly, as program designers, we would like our language to be +powerful enough so that we can write a procedure that expresses the +concept of summation itself rather than only procedures that compute +particular sums. We can do so readily in our procedural language by +taking the common template shown above and transforming the "slots" +into formal parameters: + + (define (sum term a next b) + (if (> a b) + 0 + (+ (term a) + (sum term (next a) next b)))) + + Notice that `sum' takes as its arguments the lower and upper bounds +`a' and `b' together with the procedures `term' and `next'. We can use +`sum' just as we would any procedure. For example, we can use it +(along with a procedure `inc' that increments its argument by 1) to +define `sum-cubes': + + (define (inc n) (+ n 1)) + + (define (sum-cubes a b) + (sum cube a inc b)) + + Using this, we can compute the sum of the cubes of the integers from +1 to 10: + + (sum-cubes 1 10) + 3025 + + With the aid of an identity procedure to compute the term, we can +define `sum-integers' in terms of `sum': + + (define (identity x) x) + + (define (sum-integers a b) + (sum identity a inc b)) + + Then we can add up the integers from 1 to 10: + + (sum-integers 1 10) + 55 + + We can also define `pi-sum' in the same way:(2) + + (define (pi-sum a b) + (define (pi-term x) + (/ 1.0 (* x (+ x 2)))) + (define (pi-next x) + (+ x 4)) + (sum pi-term a pi-next b)) + + Using these procedures, we can compute an approximation to [pi]: + + (* 8 (pi-sum 1 1000)) + 3.139592655589783 + + Once we have `sum', we can use it as a building block in formulating +further concepts. For instance, the definite integral of a function f +between the limits a and b can be approximated numerically using the +formula + + /b / / dx \ / dx \ / dx \ \ + | f = | f| a + -- | + f| a + dx + -- | + f| a + 2dx + -- | + ...| dx + /a \ \ 2 / \ 2 / \ 2 / / + +for small values of dx. We can express this directly as a procedure: + + (define (integral f a b dx) + (define (add-dx x) (+ x dx)) + (* (sum f (+ a (/ dx 2.0)) add-dx b) + dx)) + + (integral cube 0 1 0.01) + .24998750000000042 + + (integral cube 0 1 0.001) + .249999875000001 + +(The exact value of the integral of `cube' between 0 and 1 is 1/4.) + + *Exercise 1.29:* Simpson's Rule is a more accurate method of + numerical integration than the method illustrated above. Using + Simpson's Rule, the integral of a function f between a and b is + approximated as + + h + - (y_0 + 4y_1 + 2y_2 + 4y_3 + 2y_4 + ... + 2y_(n-2) + 4y_(n-1) + y_n) + 3 + + where h = (b - a)/n, for some even integer n, and y_k = f(a + kh). + (Increasing n increases the accuracy of the approximation.) + Define a procedure that takes as arguments f, a, b, and n and + returns the value of the integral, computed using Simpson's Rule. + Use your procedure to integrate `cube' between 0 and 1 (with n = + 100 and n = 1000), and compare the results to those of the + `integral' procedure shown above. + + *Exercise 1.30:* The `sum' procedure above generates a linear + recursion. The procedure can be rewritten so that the sum is + performed iteratively. Show how to do this by filling in the + missing expressions in the following definition: + + (define (sum term a next b) + (define (iter a result) + (if + + (iter ))) + (iter )) + + *Exercise 1.31:* + a. The `sum' procedure is only the simplest of a vast number of + similar abstractions that can be captured as higher-order + procedures.(3) Write an analogous procedure called `product' + that returns the product of the values of a function at + points over a given range. Show how to define `factorial' in + terms of `product'. Also use `product' to compute + approximations to [pi] using the formula(4) + + pi 2 * 4 * 4 * 6 * 6 * 8 ... + -- = ------------------------- + 4 3 * 3 * 5 * 5 * 7 * 7 ... + + b. If your `product' procedure generates a recursive process, + write one that generates an iterative process. If it + generates an iterative process, write one that generates a + recursive process. + + + *Exercise 1.32:* + a. Show that `sum' and `product' (*Note Exercise 1-31::) are + both special cases of a still more general notion called + `accumulate' that combines a collection of terms, using some + general accumulation function: + + (accumulate combiner null-value term a next b) + + `Accumulate' takes as arguments the same term and range + specifications as `sum' and `product', together with a + `combiner' procedure (of two arguments) that specifies how + the current term is to be combined with the accumulation of + the preceding terms and a `null-value' that specifies what + base value to use when the terms run out. Write `accumulate' + and show how `sum' and `product' can both be defined as + simple calls to `accumulate'. + + b. If your `accumulate' procedure generates a recursive process, + write one that generates an iterative process. If it + generates an iterative process, write one that generates a + recursive process. + + + *Exercise 1.33:* You can obtain an even more general version of + `accumulate' (*Note Exercise 1-32::) by introducing the notion of + a "filter" on the terms to be combined. That is, combine only + those terms derived from values in the range that satisfy a + specified condition. The resulting `filtered-accumulate' + abstraction takes the same arguments as accumulate, together with + an additional predicate of one argument that specifies the filter. + Write `filtered-accumulate' as a procedure. Show how to express + the following using `filtered-accumulate': + + a. the sum of the squares of the prime numbers in the interval a + to b (assuming that you have a `prime?' predicate already + written) + + b. the product of all the positive integers less than n that are + relatively prime to n (i.e., all positive integers i < n such + that GCD(i,n) = 1). + + + ---------- Footnotes ---------- + + (1) This series, usually written in the equivalent form ([pi]/4) = 1 +- (1/3) + (1/5) - (1/7) + ..., is due to Leibniz. We'll see how to use +this as the basis for some fancy numerical tricks in section *Note +3-5-3::. + + (2) Notice that we have used block structure (section *Note 1-1-8::) +to embed the definitions of `pi-next' and `pi-term' within `pi-sum', +since these procedures are unlikely to be useful for any other purpose. +We will see how to get rid of them altogether in section *Note 1-3-2::. + + (3) The intent of *Note Exercise 1-31:: through *Note Exercise +1-33:: is to demonstrate the expressive power that is attained by using +an appropriate abstraction to consolidate many seemingly disparate +operations. However, though accumulation and filtering are elegant +ideas, our hands are somewhat tied in using them at this point since we +do not yet have data structures to provide suitable means of +combination for these abstractions. We will return to these ideas in +section *Note 2-2-3:: when we show how to use "sequences" as interfaces +for combining filters and accumulators to build even more powerful +abstractions. We will see there how these methods really come into +their own as a powerful and elegant approach to designing programs. + + (4) This formula was discovered by the seventeenth-century English +mathematician John Wallis. + + +File: sicp.info, Node: 1-3-2, Next: 1-3-3, Prev: 1-3-1, Up: 1-3 + +1.3.2 Constructing Procedures Using `Lambda' +-------------------------------------------- + +In using `sum' as in section *Note 1-3-1::, it seems terribly awkward to +have to define trivial procedures such as `pi-term' and `pi-next' just +so we can use them as arguments to our higher-order procedure. Rather +than define `pi-next' and `pi-term', it would be more convenient to +have a way to directly specify "the procedure that returns its input +incremented by 4" and "the procedure that returns the reciprocal of its +input times its input plus 2." We can do this by introducing the +special form `lambda', which creates procedures. Using `lambda' we can +describe what we want as + + (lambda (x) (+ x 4)) + +and + + (lambda (x) (/ 1.0 (* x (+ x 2)))) + + Then our `pi-sum' procedure can be expressed without defining any +auxiliary procedures as + + (define (pi-sum a b) + (sum (lambda (x) (/ 1.0 (* x (+ x 2)))) + a + (lambda (x) (+ x 4)) + b)) + + Again using `lambda', we can write the `integral' procedure without +having to define the auxiliary procedure `add-dx': + + (define (integral f a b dx) + (* (sum f + (+ a (/ dx 2.0)) + (lambda (x) (+ x dx)) + b) + dx)) + + In general, `lambda' is used to create procedures in the same way as +`define', except that no name is specified for the procedure: + + (lambda () ) + + The resulting procedure is just as much a procedure as one that is +created using `define'. The only difference is that it has not been +associated with any name in the environment. In fact, + + (define (plus4 x) (+ x 4)) + +is equivalent to + + (define plus4 (lambda (x) (+ x 4))) + + We can read a `lambda' expression as follows: + + (lambda (x) (+ x 4)) + | | | | | + the procedure of an argument x that adds x and 4 + + Like any expression that has a procedure as its value, a `lambda' +expression can be used as the operator in a combination such as + + ((lambda (x y z) (+ x y (square z))) 1 2 3) + 12 + +or, more generally, in any context where we would normally use a +procedure name.(1) + +Using `let' to create local variables +..................................... + +Another use of `lambda' is in creating local variables. We often need +local variables in our procedures other than those that have been bound +as formal parameters. For example, suppose we wish to compute the +function + + f(x,y) = x(1 + xy)^2 + y(1 - y) + (1 + xy)(1 - y) + +which we could also express as + + a = 1 + xy + b = 1 - y + f(x,y) = xa^2 + yb + ab + + In writing a procedure to compute f, we would like to include as +local variables not only x and y but also the names of intermediate +quantities like a and b. One way to accomplish this is to use an +auxiliary procedure to bind the local variables: + + (define (f x y) + (define (f-helper a b) + (+ (* x (square a)) + (* y b) + (* a b))) + (f-helper (+ 1 (* x y)) + (- 1 y))) + + Of course, we could use a `lambda' expression to specify an anonymous +procedure for binding our local variables. The body of `f' then +becomes a single call to that procedure: + + (define (f x y) + ((lambda (a b) + (+ (* x (square a)) + (* y b) + (* a b))) + (+ 1 (* x y)) + (- 1 y))) + + This construct is so useful that there is a special form called +`let' to make its use more convenient. Using `let', the `f' procedure +could be written as + + (define (f x y) + (let ((a (+ 1 (* x y))) + (b (- 1 y))) + (+ (* x (square a)) + (* y b) + (* a b)))) + + The general form of a `let' expression is + + (let (( ) + ( ) + ... + ( )) + ) + +which can be thought of as saying + + let have the value and + have the value and + ... + have the value + in + + The first part of the `let' expression is a list of name-expression +pairs. When the `let' is evaluated, each name is associated with the +value of the corresponding expression. The body of the `let' is +evaluated with these names bound as local variables. The way this +happens is that the `let' expression is interpreted as an alternate +syntax for + + ((lambda ( ... ) + ) + + ... + ) + + No new mechanism is required in the interpreter in order to provide +local variables. A `let' expression is simply syntactic sugar for the +underlying `lambda' application. + + We can see from this equivalence that the scope of a variable +specified by a `let' expression is the body of the `let'. This implies +that: + + * `Let' allows one to bind variables as locally as possible to where + they are to be used. For example, if the value of `x' is 5, the + value of the expression + + (+ (let ((x 3)) + (+ x (* x 10))) + x) + + is 38. Here, the `x' in the body of the `let' is 3, so the value + of the `let' expression is 33. On the other hand, the `x' that is + the second argument to the outermost `+' is still 5. + + * The variables' values are computed outside the `let'. This + matters when the expressions that provide the values for the local + variables depend upon variables having the same names as the local + variables themselves. For example, if the value of `x' is 2, the + expression + + (let ((x 3) + (y (+ x 2))) + (* x y)) + + will have the value 12 because, inside the body of the `let', `x' + will be 3 and `y' will be 4 (which is the outer `x' plus 2). + + + Sometimes we can use internal definitions to get the same effect as +with `let'. For example, we could have defined the procedure `f' above +as + + (define (f x y) + (define a (+ 1 (* x y))) + (define b (- 1 y)) + (+ (* x (square a)) + (* y b) + (* a b))) + + We prefer, however, to use `let' in situations like this and to use +internal `define' only for internal procedures.(2) + + *Exercise 1.34:* Suppose we define the procedure + + (define (f g) + (g 2)) + + Then we have + + (f square) + 4 + + (f (lambda (z) (* z (+ z 1)))) + 6 + + What happens if we (perversely) ask the interpreter to evaluate + the combination `(f f)'? Explain. + + ---------- Footnotes ---------- + + (1) It would be clearer and less intimidating to people learning +Lisp if a name more obvious than `lambda', such as `make-procedure', +were used. But the convention is firmly entrenched. The notation is +adopted from the [lambda] calculus, a mathematical formalism introduced +by the mathematical logician Alonzo Church (1941). Church developed +the [lambda] calculus to provide a rigorous foundation for studying the +notions of function and function application. The [lambda] calculus +has become a basic tool for mathematical investigations of the +semantics of programming languages. + + (2) Understanding internal definitions well enough to be sure a +program means what we intend it to mean requires a more elaborate model +of the evaluation process than we have presented in this chapter. The +subtleties do not arise with internal definitions of procedures, +however. We will return to this issue in section *Note 4-1-6::, after +we learn more about evaluation. + + +File: sicp.info, Node: 1-3-3, Next: 1-3-4, Prev: 1-3-2, Up: 1-3 + +1.3.3 Procedures as General Methods +----------------------------------- + +We introduced compound procedures in section *Note 1-1-4:: as a +mechanism for abstracting patterns of numerical operations so as to +make them independent of the particular numbers involved. With +higher-order procedures, such as the `integral' procedure of section +*Note 1-3-1::, we began to see a more powerful kind of abstraction: +procedures used to express general methods of computation, independent +of the particular functions involved. In this section we discuss two +more elaborate examples--general methods for finding zeros and fixed +points of functions--and show how these methods can be expressed +directly as procedures. + +Finding roots of equations by the half-interval method +...................................................... + +The "half-interval method" is a simple but powerful technique for +finding roots of an equation f(x) = 0, where f is a continuous +function. The idea is that, if we are given points a and b such that +f(a) < 0 < f(b), then f must have at least one zero between a and b. +To locate a zero, let x be the average of a and b and compute f(x). If +f(x) > 0, then f must have a zero between a and x. If f(x) < 0, then f +must have a zero between x and b. Continuing in this way, we can +identify smaller and smaller intervals on which f must have a zero. +When we reach a point where the interval is small enough, the process +stops. Since the interval of uncertainty is reduced by half at each +step of the process, the number of steps required grows as +[theta](`log'( L/T)), where L is the length of the original interval +and T is the error tolerance (that is, the size of the interval we will +consider "small enough"). Here is a procedure that implements this +strategy: + + (define (search f neg-point pos-point) + (let ((midpoint (average neg-point pos-point))) + (if (close-enough? neg-point pos-point) + midpoint + (let ((test-value (f midpoint))) + (cond ((positive? test-value) + (search f neg-point midpoint)) + ((negative? test-value) + (search f midpoint pos-point)) + (else midpoint)))))) + + We assume that we are initially given the function f together with +points at which its values are negative and positive. We first compute +the midpoint of the two given points. Next we check to see if the +given interval is small enough, and if so we simply return the midpoint +as our answer. Otherwise, we compute as a test value the value of f at +the midpoint. If the test value is positive, then we continue the +process with a new interval running from the original negative point to +the midpoint. If the test value is negative, we continue with the +interval from the midpoint to the positive point. Finally, there is +the possibility that the test value is 0, in which case the midpoint is +itself the root we are searching for. + + To test whether the endpoints are "close enough" we can use a +procedure similar to the one used in section *Note 1-1-7:: for +computing square roots:(1) + + (define (close-enough? x y) + (< (abs (- x y)) 0.001)) + + `Search' is awkward to use directly, because we can accidentally +give it points at which f's values do not have the required sign, in +which case we get a wrong answer. Instead we will use `search' via the +following procedure, which checks to see which of the endpoints has a +negative function value and which has a positive value, and calls the +`search' procedure accordingly. If the function has the same sign on +the two given points, the half-interval method cannot be used, in which +case the procedure signals an error.(2) + + (define (half-interval-method f a b) + (let ((a-value (f a)) + (b-value (f b))) + (cond ((and (negative? a-value) (positive? b-value)) + (search f a b)) + ((and (negative? b-value) (positive? a-value)) + (search f b a)) + (else + (error "Values are not of opposite sign" a b))))) + + The following example uses the half-interval method to approximate +[pi] as the root between 2 and 4 of `sin' x = 0: + + (half-interval-method sin 2.0 4.0) + 3.14111328125 + + Here is another example, using the half-interval method to search +for a root of the equation x^3 - 2x - 3 = 0 between 1 and 2: + + (half-interval-method (lambda (x) (- (* x x x) (* 2 x) 3)) + 1.0 + 2.0) + 1.89306640625 + +Finding fixed points of functions +................................. + +A number x is called a "fixed point" of a function f if x satisfies the +equation f(x) = x. For some functions f we can locate a fixed point by +beginning with an initial guess and applying f repeatedly, + + f(x), f(f(x), (f(f(f(x)))) + +until the value does not change very much. Using this idea, we can +devise a procedure `fixed-point' that takes as inputs a function and an +initial guess and produces an approximation to a fixed point of the +function. We apply the function repeatedly until we find two +successive values whose difference is less than some prescribed +tolerance: + + (define tolerance 0.00001) + + (define (fixed-point f first-guess) + (define (close-enough? v1 v2) + (< (abs (- v1 v2)) tolerance)) + (define (try guess) + (let ((next (f guess))) + (if (close-enough? guess next) + next + (try next)))) + (try first-guess)) + + For example, we can use this method to approximate the fixed point +of the cosine function, starting with 1 as an initial approximation:(3) + + (fixed-point cos 1.0) + .7390822985224023 + + Similarly, we can find a solution to the equation y = `sin' y + +`cos' y: + + (fixed-point (lambda (y) (+ (sin y) (cos y))) + 1.0) + 1.2587315962971173 + + The fixed-point process is reminiscent of the process we used for +finding square roots in section *Note 1-1-7::. Both are based on the +idea of repeatedly improving a guess until the result satisfies some +criterion. In fact, we can readily formulate the square-root +computation as a fixed-point search. Computing the square root of some +number x requires finding a y such that y^2 = x. Putting this equation +into the equivalent form y = x/y, we recognize that we are looking for +a fixed point of the function(4) y |-> x/y, and we can therefore try to +compute square roots as + + (define (sqrt x) + (fixed-point (lambda (y) (/ x y)) + 1.0)) + + Unfortunately, this fixed-point search does not converge. Consider +an initial guess y_1. The next guess is y_2 = x/y_1 and the next guess +is y_3 = x/y_2 = x/(x/y_1) = y_1. This results in an infinite loop in +which the two guesses y_1 and y_2 repeat over and over, oscillating +about the answer. + + One way to control such oscillations is to prevent the guesses from +changing so much. Since the answer is always between our guess y and +x/y, we can make a new guess that is not as far from y as x/y by +averaging y with x/y, so that the next guess after y is (1/2)(y + x/y) +instead of x/y. The process of making such a sequence of guesses is +simply the process of looking for a fixed point of y |-> (1/2)(y + x/y): + + (define (sqrt x) + (fixed-point (lambda (y) (average y (/ x y))) + 1.0)) + + (Note that y = (1/2)(y + x/y) is a simple transformation of the +equation y = x/y; to derive it, add y to both sides of the equation and +divide by 2.) + + With this modification, the square-root procedure works. In fact, +if we unravel the definitions, we can see that the sequence of +approximations to the square root generated here is precisely the same +as the one generated by our original square-root procedure of section +*Note 1-1-7::. This approach of averaging successive approximations to +a solution, a technique we that we call "average damping", often aids +the convergence of fixed-point searches. + + *Exercise 1.35:* Show that the golden ratio [phi] (section *Note + 1-2-2::) is a fixed point of the transformation x |-> 1 + 1/x, and + use this fact to compute [phi] by means of the `fixed-point' + procedure. + + *Exercise 1.36:* Modify `fixed-point' so that it prints the + sequence of approximations it generates, using the `newline' and + `display' primitives shown in *Note Exercise 1-22::. Then find a + solution to x^x = 1000 by finding a fixed point of x |-> + `log'(1000)/`log'(x). (Use Scheme's primitive `log' procedure, + which computes natural logarithms.) Compare the number of steps + this takes with and without average damping. (Note that you + cannot start `fixed-point' with a guess of 1, as this would cause + division by `log'(1) = 0.) + + *Exercise 1.37:* + a. An infinite "continued fraction" is an expression of the form + + N_1 + f = --------------------- + N_2 + D_1 + --------------- + N_3 + D_2 + --------- + D_3 + ... + + As an example, one can show that the infinite continued + fraction expansion with the n_i and the D_i all equal to 1 + produces 1/[phi], where [phi] is the golden ratio (described + in section *Note 1-2-2::). One way to approximate an + infinite continued fraction is to truncate the expansion + after a given number of terms. Such a truncation--a + so-called finite continued fraction "k-term finite continued + fraction"--has the form + + N_1 + ----------------- + N_2 + D_1 + ----------- + ... N_K + + ----- + D_K + + Suppose that `n' and `d' are procedures of one argument (the + term index i) that return the n_i and D_i of the terms of the + continued fraction. Define a procedure `cont-frac' such that + evaluating `(cont-frac n d k)' computes the value of the + k-term finite continued fraction. Check your procedure by + approximating 1/[phi] using + + (cont-frac (lambda (i) 1.0) + (lambda (i) 1.0) + k) + + for successive values of `k'. How large must you make `k' in + order to get an approximation that is accurate to 4 decimal + places? + + b. If your `cont-frac' procedure generates a recursive process, + write one that generates an iterative process. If it + generates an iterative process, write one that generates a + recursive process. + + + *Exercise 1.38:* In 1737, the Swiss mathematician Leonhard Euler + published a memoir `De Fractionibus Continuis', which included a + continued fraction expansion for e - 2, where e is the base of the + natural logarithms. In this fraction, the n_i are all 1, and the + D_i are successively 1, 2, 1, 1, 4, 1, 1, 6, 1, 1, 8, .... Write + a program that uses your `cont-frac' procedure from *Note Exercise + 1-37:: to approximate e, based on Euler's expansion. + + *Exercise 1.39:* A continued fraction representation of the + tangent function was published in 1770 by the German mathematician + J.H. Lambert: + + x + tan x = --------------- + x^2 + 1 - ----------- + x^2 + 3 - ------- + 5 - ... + + where x is in radians. Define a procedure `(tan-cf x k)' that + computes an approximation to the tangent function based on + Lambert's formula. `K' specifies the number of terms to compute, + as in *Note Exercise 1-37::. + + ---------- Footnotes ---------- + + (1) We have used 0.001 as a representative "small" number to +indicate a tolerance for the acceptable error in a calculation. The +appropriate tolerance for a real calculation depends upon the problem +to be solved and the limitations of the computer and the algorithm. +This is often a very subtle consideration, requiring help from a +numerical analyst or some other kind of magician. + + (2) This can be accomplished using `error', which takes as arguments +a number of items that are printed as error messages. + + (3) Try this during a boring lecture: Set your calculator to radians +mode and then repeatedly press the `cos' button until you obtain the +fixed point. + + (4) |-> (pronounced "maps to") is the mathematician's way of writing +`lambda'. y |-> x/y means `(lambda(y) (/ x y))', that is, the function +whose value at y is x/y. + + +File: sicp.info, Node: 1-3-4, Prev: 1-3-3, Up: 1-3 + +1.3.4 Procedures as Returned Values +----------------------------------- + +The above examples demonstrate how the ability to pass procedures as +arguments significantly enhances the expressive power of our +programming language. We can achieve even more expressive power by +creating procedures whose returned values are themselves procedures. + + We can illustrate this idea by looking again at the fixed-point +example described at the end of section *Note 1-3-3::. We formulated a +new version of the square-root procedure as a fixed-point search, +starting with the observation that [sqrt]x is a fixed-point of the +function y |-> x/y. Then we used average damping to make the +approximations converge. Average damping is a useful general technique +in itself. Namely, given a function f, we consider the function whose +value at x is equal to the average of x and f(x). + + We can express the idea of average damping by means of the following +procedure: + + (define (average-damp f) + (lambda (x) (average x (f x)))) + + `Average-damp' is a procedure that takes as its argument a procedure +`f' and returns as its value a procedure (produced by the `lambda') +that, when applied to a number `x', produces the average of `x' and `(f +x)'. For example, applying `average-damp' to the `square' procedure +produces a procedure whose value at some number x is the average of x +and x^2. Applying this resulting procedure to 10 returns the average +of 10 and 100, or 55:(1) + + ((average-damp square) 10) + 55 + + Using `average-damp', we can reformulate the square-root procedure as +follows: + + (define (sqrt x) + (fixed-point (average-damp (lambda (y) (/ x y))) + 1.0)) + + Notice how this formulation makes explicit the three ideas in the +method: fixed-point search, average damping, and the function y |-> x/y. +It is instructive to compare this formulation of the square-root method +with the original version given in section *Note 1-1-7::. Bear in mind +that these procedures express the same process, and notice how much +clearer the idea becomes when we express the process in terms of these +abstractions. In general, there are many ways to formulate a process +as a procedure. Experienced programmers know how to choose procedural +formulations that are particularly perspicuous, and where useful +elements of the process are exposed as separate entities that can be +reused in other applications. As a simple example of reuse, notice +that the cube root of x is a fixed point of the function y |-> x/y^2, +so we can immediately generalize our square-root procedure to one that +extracts cube roots:(2) + + (define (cube-root x) + (fixed-point (average-damp (lambda (y) (/ x (square y)))) + 1.0)) + +Newton's method +............... + +When we first introduced the square-root procedure, in section *Note +1-1-7::, we mentioned that this was a special case of "Newton's +method". If x |-> g(x) is a differentiable function, then a solution +of the equation g(x) = 0 is a fixed point of the function x |-> f(x) +where + + g(x) + f(x) = x - ----- + Dg(x) + +and Dg(x) is the derivative of g evaluated at x. Newton's method is +the use of the fixed-point method we saw above to approximate a +solution of the equation by finding a fixed point of the function f.(3) + + For many functions g and for sufficiently good initial guesses for x, +Newton's method converges very rapidly to a solution of g(x) = 0.(4) + + In order to implement Newton's method as a procedure, we must first +express the idea of derivative. Note that "derivative," like average +damping, is something that transforms a function into another function. +For instance, the derivative of the function x |-> x^3 is the function +x |-> 3x^2. In general, if g is a function and dx is a small number, +then the derivative Dg of g is the function whose value at any number x +is given (in the limit of small dx) by + + g(x + dx) - g(x) + Dg(c) = ---------------- + dx + +Thus, we can express the idea of derivative (taking dx to be, say, +0.00001) as the procedure + + (define (deriv g) + (lambda (x) + (/ (- (g (+ x dx)) (g x)) + dx))) + +along with the definition + + (define dx 0.00001) + + Like `average-damp', `deriv' is a procedure that takes a procedure as +argument and returns a procedure as value. For example, to approximate +the derivative of x |-> x^3 at 5 (whose exact value is 75) we can +evaluate + + (define (cube x) (* x x x)) + + ((deriv cube) 5) + 75.00014999664018 + + With the aid of `deriv', we can express Newton's method as a +fixed-point process: + + (define (newton-transform g) + (lambda (x) + (- x (/ (g x) ((deriv g) x))))) + + (define (newtons-method g guess) + (fixed-point (newton-transform g) guess)) + + The `newton-transform' procedure expresses the formula at the +beginning of this section, and `newtons-method' is readily defined in +terms of this. It takes as arguments a procedure that computes the +function for which we want to find a zero, together with an initial +guess. For instance, to find the square root of x, we can use Newton's +method to find a zero of the function y |-> y^2 - x starting with an +initial guess of 1.(5) + + This provides yet another form of the square-root procedure: + + (define (sqrt x) + (newtons-method (lambda (y) (- (square y) x)) + 1.0)) + +Abstractions and first-class procedures +....................................... + +We've seen two ways to express the square-root computation as an +instance of a more general method, once as a fixed-point search and +once using Newton's method. Since Newton's method was itself expressed +as a fixed-point process, we actually saw two ways to compute square +roots as fixed points. Each method begins with a function and finds a +fixed point of some transformation of the function. We can express +this general idea itself as a procedure: + + (define (fixed-point-of-transform g transform guess) + (fixed-point (transform g) guess)) + + This very general procedure takes as its arguments a procedure `g' +that computes some function, a procedure that transforms `g', and an +initial guess. The returned result is a fixed point of the transformed +function. + + Using this abstraction, we can recast the first square-root +computation from this section (where we look for a fixed point of the +average-damped version of y |-> x/y) as an instance of this general +method: + + (define (sqrt x) + (fixed-point-of-transform (lambda (y) (/ x y)) + average-damp + 1.0)) + + Similarly, we can express the second square-root computation from +this section (an instance of Newton's method that finds a fixed point +of the Newton transform of y |-> y^2 - x) as + + (define (sqrt x) + (fixed-point-of-transform (lambda (y) (- (square y) x)) + newton-transform + 1.0)) + + We began section *Note 1-3:: with the observation that compound +procedures are a crucial abstraction mechanism, because they permit us +to express general methods of computing as explicit elements in our +programming language. Now we've seen how higher-order procedures +permit us to manipulate these general methods to create further +abstractions. + + As programmers, we should be alert to opportunities to identify the +underlying abstractions in our programs and to build upon them and +generalize them to create more powerful abstractions. This is not to +say that one should always write programs in the most abstract way +possible; expert programmers know how to choose the level of +abstraction appropriate to their task. But it is important to be able +to think in terms of these abstractions, so that we can be ready to +apply them in new contexts. The significance of higher-order +procedures is that they enable us to represent these abstractions +explicitly as elements in our programming language, so that they can be +handled just like other computational elements. + + In general, programming languages impose restrictions on the ways in +which computational elements can be manipulated. Elements with the +fewest restrictions are said to have "first-class" status. Some of the +"rights and privileges" of first-class elements are:(6) + + * They may be named by variables. + + * They may be passed as arguments to procedures. + + * They may be returned as the results of procedures. + + * They may be included in data structures.(7) + + + Lisp, unlike other common programming languages, awards procedures +full first-class status. This poses challenges for efficient +implementation, but the resulting gain in expressive power is +enormous.(8) + + *Exercise 1.40:* Define a procedure `cubic' that can be used + together with the `newtons-method' procedure in expressions of the + form + + (newtons-method (cubic a b c) 1) + + to approximate zeros of the cubic x^3 + ax^2 + bx + c. + + *Exercise 1.41:* Define a procedure `double' that takes a + procedure of one argument as argument and returns a procedure that + applies the original procedure twice. For example, if `inc' is a + procedure that adds 1 to its argument, then `(double inc)' should + be a procedure that adds 2. What value is returned by + + (((double (double double)) inc) 5) + + *Exercise 1.42:* Let f and g be two one-argument functions. The "composition" + f after g is defined to be the function x |-> f(g(x)). Define a + procedure `compose' that implements composition. For example, if + `inc' is a procedure that adds 1 to its argument, + + ((compose square inc) 6) + 49 + + *Exercise 1.43:* If f is a numerical function and n is a positive + integer, then we can form the nth repeated application of f, which + is defined to be the function whose value at x is + f(f(...(f(x))...)). For example, if f is the function x |-> x + + 1, then the nth repeated application of f is the function x |-> x + + n. If f is the operation of squaring a number, then the nth + repeated application of f is the function that raises its argument + to the 2^nth power. Write a procedure that takes as inputs a + procedure that computes f and a positive integer n and returns the + procedure that computes the nth repeated application of f. Your + procedure should be able to be used as follows: + + ((repeated square 2) 5) + 625 + + Hint: You may find it convenient to use `compose' from *Note + Exercise 1-42::. + + *Exercise 1.44:* The idea of "smoothing" a function is an + important concept in signal processing. If f is a function and dx + is some small number, then the smoothed version of f is the + function whose value at a point x is the average of f(x - dx), + f(x), and f(x + dx). Write a procedure `smooth' that takes as + input a procedure that computes f and returns a procedure that + computes the smoothed f. It is sometimes valuable to repeatedly + smooth a function (that is, smooth the smoothed function, and so + on) to obtained the "n-fold smoothed function". Show how to + generate the n-fold smoothed function of any given function using + `smooth' and `repeated' from *Note Exercise 1-43::. + + *Exercise 1.45:* We saw in section *Note 1-3-3:: that attempting + to compute square roots by naively finding a fixed point of y |-> + x/y does not converge, and that this can be fixed by average + damping. The same method works for finding cube roots as fixed + points of the average-damped y |-> x/y^2. Unfortunately, the + process does not work for fourth roots--a single average damp is + not enough to make a fixed-point search for y |-> x/y^3 converge. + On the other hand, if we average damp twice (i.e., use the average + damp of the average damp of y |-> x/y^3) the fixed-point search + does converge. Do some experiments to determine how many average + damps are required to compute nth roots as a fixed-point search + based upon repeated average damping of y |-> x/y^(n-1). Use this + to implement a simple procedure for computing nth roots using + `fixed-point', `average-damp', and the `repeated' procedure of + *Note Exercise 1-43::. Assume that any arithmetic operations you + need are available as primitives. + + *Exercise 1.46:* Several of the numerical methods described in + this chapter are instances of an extremely general computational + strategy known as "iterative improvement". Iterative improvement + says that, to compute something, we start with an initial guess + for the answer, test if the guess is good enough, and otherwise + improve the guess and continue the process using the improved + guess as the new guess. Write a procedure `iterative-improve' + that takes two procedures as arguments: a method for telling + whether a guess is good enough and a method for improving a guess. + `Iterative-improve' should return as its value a procedure that + takes a guess as argument and keeps improving the guess until it + is good enough. Rewrite the `sqrt' procedure of section *Note + 1-1-7:: and the `fixed-point' procedure of section *Note 1-3-3:: + in terms of `iterative-improve'. + + ---------- Footnotes ---------- + + (1) Observe that this is a combination whose operator is itself a +combination. *Note Exercise 1-4:: already demonstrated the ability to +form such combinations, but that was only a toy example. Here we begin +to see the real need for such combinations--when applying a procedure +that is obtained as the value returned by a higher-order procedure. + + (2) See *Note Exercise 1-45:: for a further generalization. + + (3) Elementary calculus books usually describe Newton's method in +terms of the sequence of approximations x_(n+1) = x_n - g(x_n)/Dg(x_n). +Having language for talking about processes and using the idea of +fixed points simplifies the description of the method. + + (4) Newton's method does not always converge to an answer, but it can +be shown that in favorable cases each iteration doubles the +number-of-digits accuracy of the approximation to the solution. In +such cases, Newton's method will converge much more rapidly than the +half-interval method. + + (5) For finding square roots, Newton's method converges rapidly to +the correct solution from any starting point. + + (6) The notion of first-class status of programming-language +elements is due to the British computer scientist Christopher Strachey +(1916-1975). + + (7) We'll see examples of this after we introduce data structures in +*Note Chapter 2::. + + (8) The major implementation cost of first-class procedures is that +allowing procedures to be returned as values requires reserving storage +for a procedure's free variables even while the procedure is not +executing. In the Scheme implementation we will study in section *Note +4-1::, these variables are stored in the procedure's environment. + + +File: sicp.info, Node: Chapter 2, Next: Chapter 3, Prev: Chapter 1, Up: Top + +2 Building Abstractions with Data +********************************* + + We now come to the decisive step of mathematical abstraction: we + forget about what the symbols stand for. ...[The mathematician] + need not be idle; there are many operations which he may carry out + with these symbols, without ever having to look at the things they + stand for. + + --Hermann Weyl, `The Mathematical Way of Thinking' + + We concentrated in *Note Chapter 1:: on computational processes and +on the role of procedures in program design. We saw how to use +primitive data (numbers) and primitive operations (arithmetic +operations), how to combine procedures to form compound procedures +through composition, conditionals, and the use of parameters, and how +to abstract procedures by using `define'. We saw that a procedure can +be regarded as a pattern for the local evolution of a process, and we +classified, reasoned about, and performed simple algorithmic analyses of +some common patterns for processes as embodied in procedures. We also +saw that higher-order procedures enhance the power of our language by +enabling us to manipulate, and thereby to reason in terms of, general +methods of computation. This is much of the essence of programming. + + In this chapter we are going to look at more complex data. All the +procedures in *Note Chapter 1:: operate on simple numerical data, and +simple data are not sufficient for many of the problems we wish to +address using computation. Programs are typically designed to model +complex phenomena, and more often than not one must construct +computational objects that have several parts in order to model +real-world phenomena that have several aspects. Thus, whereas our +focus in *Note Chapter 1:: was on building abstractions by combining +procedures to form compound procedures, we turn in this chapter to +another key aspect of any programming language: the means it provides +for building abstractions by combining data objects to form "compound +data". + + Why do we want compound data in a programming language? For the +same reasons that we want compound procedures: to elevate the +conceptual level at which we can design our programs, to increase the +modularity of our designs, and to enhance the expressive power of our +language. Just as the ability to define procedures enables us to deal +with processes at a higher conceptual level than that of the primitive +operations of the language, the ability to construct compound data +objects enables us to deal with data at a higher conceptual level than +that of the primitive data objects of the language. + + Consider the task of designing a system to perform arithmetic with +rational numbers. We could imagine an operation `add-rat' that takes +two rational numbers and produces their sum. In terms of simple data, +a rational number can be thought of as two integers: a numerator and a +denominator. Thus, we could design a program in which each rational +number would be represented by two integers (a numerator and a +denominator) and where `add-rat' would be implemented by two procedures +(one producing the numerator of the sum and one producing the +denominator). But this would be awkward, because we would then need to +explicitly keep track of which numerators corresponded to which +denominators. In a system intended to perform many operations on many +rational numbers, such bookkeeping details would clutter the programs +substantially, to say nothing of what they would do to our minds. It +would be much better if we could "glue together" a numerator and +denominator to form a pair--a "compound data object"--that our programs +could manipulate in a way that would be consistent with regarding a +rational number as a single conceptual unit. + + The use of compound data also enables us to increase the modularity +of our programs. If we can manipulate rational numbers directly as +objects in their own right, then we can separate the part of our +program that deals with rational numbers per se from the details of how +rational numbers may be represented as pairs of integers. The general +technique of isolating the parts of a program that deal with how data +objects are represented from the parts of a program that deal with how +data objects are used is a powerful design methodology called "data +abstraction". We will see how data abstraction makes programs much +easier to design, maintain, and modify. + + The use of compound data leads to a real increase in the expressive +power of our programming language. Consider the idea of forming a +"linear combination" ax + by. We might like to write a procedure that +would accept a, b, x, and y as arguments and return the value of ax + +by. This presents no difficulty if the arguments are to be numbers, +because we can readily define the procedure + + (define (linear-combination a b x y) + (+ (* a x) (* b y))) + + But suppose we are not concerned only with numbers. Suppose we +would like to express, in procedural terms, the idea that one can form +linear combinations whenever addition and multiplication are +defined--for rational numbers, complex numbers, polynomials, or +whatever. We could express this as a procedure of the form + + (define (linear-combination a b x y) + (add (mul a x) (mul b y))) + +where `add' and `mul' are not the primitive procedures `+' and `*' but +rather more complex things that will perform the appropriate operations +for whatever kinds of data we pass in as the arguments `a', `b', `x', +and `y'. The key point is that the only thing `linear-combination' +should need to know about `a', `b', `x', and `y' is that the procedures +`add' and `mul' will perform the appropriate manipulations. From the +perspective of the procedure `linear-combination', it is irrelevant +what `a', `b', `x', and `y' are and even more irrelevant how they might +happen to be represented in terms of more primitive data. This same +example shows why it is important that our programming language provide +the ability to manipulate compound objects directly: Without this, +there is no way for a procedure such as `linear-combination' to pass +its arguments along to `add' and `mul' without having to know their +detailed structure.(1) + + We begin this chapter by implementing the rational-number arithmetic +system mentioned above. This will form the background for our +discussion of compound data and data abstraction. As with compound +procedures, the main issue to be addressed is that of abstraction as a +technique for coping with complexity, and we will see how data +abstraction enables us to erect suitable "abstraction barriers" between +different parts of a program. + + We will see that the key to forming compound data is that a +programming language should provide some kind of "glue" so that data +objects can be combined to form more complex data objects. There are +many possible kinds of glue. Indeed, we will discover how to form +compound data using no special "data" operations at all, only +procedures. This will further blur the distinction between "procedure" +and "data," which was already becoming tenuous toward the end of *Note +Chapter 1::. We will also explore some conventional techniques for +representing sequences and trees. One key idea in dealing with +compound data is the notion of "closure"--that the glue we use for +combining data objects should allow us to combine not only primitive +data objects, but compound data objects as well. Another key idea is +that compound data objects can serve as "conventional interfaces" for +combining program modules in mix-and-match ways. We illustrate some of +these ideas by presenting a simple graphics language that exploits +closure. + + We will then augment the representational power of our language by +introducing "symbolic expressions"--data whose elementary parts can be +arbitrary symbols rather than only numbers. We explore various +alternatives for representing sets of objects. We will find that, just +as a given numerical function can be computed by many different +computational processes, there are many ways in which a given data +structure can be represented in terms of simpler objects, and the +choice of representation can have significant impact on the time and +space requirements of processes that manipulate the data. We will +investigate these ideas in the context of symbolic differentiation, the +representation of sets, and the encoding of information. + + Next we will take up the problem of working with data that may be +represented differently by different parts of a program. This leads to +the need to implement "generic operations", which must handle many +different types of data. Maintaining modularity in the presence of +generic operations requires more powerful abstraction barriers than can +be erected with simple data abstraction alone. In particular, we +introduce programming "data-directed programming" as a technique that +allows individual data representations to be designed in isolation and +then combined "additively" (i.e., without modification). To illustrate +the power of this approach to system design, we close the chapter by +applying what we have learned to the implementation of a package for +performing symbolic arithmetic on polynomials, in which the +coefficients of the polynomials can be integers, rational numbers, +complex numbers, and even other polynomials. + +* Menu: + +* 2-1:: Introduction to Data Abstraction +* 2-2:: Hierarchical Data and the Closure Property +* 2-3:: Symbolic Data +* 2-4:: Multiple Representations for Abstract Data +* 2-5:: Systems with Generic Operations + + ---------- Footnotes ---------- + + (1) The ability to directly manipulate procedures provides an +analogous increase in the expressive power of a programming language. +For example, in section *Note 1-3-1:: we introduced the `sum' +procedure, which takes a procedure `term' as an argument and computes +the sum of the values of `term' over some specified interval. In order +to define `sum', it is crucial that we be able to speak of a procedure +such as `term' as an entity in its own right, without regard for how +`term' might be expressed with more primitive operations. Indeed, if +we did not have the notion of "a procedure," it is doubtful that we +would ever even think of the possibility of defining an operation such +as `sum'. Moreover, insofar as performing the summation is concerned, +the details of how `term' may be constructed from more primitive +operations are irrelevant. + + +File: sicp.info, Node: 2-1, Next: 2-2, Prev: Chapter 2, Up: Chapter 2 + +2.1 Introduction to Data Abstraction +==================================== + +In section *Note 1-1-8::, we noted that a procedure used as an element +in creating a more complex procedure could be regarded not only as a +collection of particular operations but also as a procedural +abstraction. That is, the details of how the procedure was implemented +could be suppressed, and the particular procedure itself could be +replaced by any other procedure with the same overall behavior. In +other words, we could make an abstraction that would separate the way +the procedure would be used from the details of how the procedure would +be implemented in terms of more primitive procedures. The analogous +notion for compound data is called "data abstraction". Data +abstraction is a methodology that enables us to isolate how a compound +data object is used from the details of how it is constructed from more +primitive data objects. + + The basic idea of data abstraction is to structure the programs that +are to use compound data objects so that they operate on "abstract +data." That is, our programs should use data in such a way as to make +no assumptions about the data that are not strictly necessary for +performing the task at hand. At the same time, a "concrete" data +representation is defined independent of the programs that use the +data. The interface between these two parts of our system will be a +set of procedures, called "selectors" and "constructors", that +implement the abstract data in terms of the concrete representation. To +illustrate this technique, we will consider how to design a set of +procedures for manipulating rational numbers. + +* Menu: + +* 2-1-1:: Example: Arithmetic Operations for Rational Numbers +* 2-1-2:: Abstraction Barriers +* 2-1-3:: What Is Meant by Data? +* 2-1-4:: Extended Exercise: Interval Arithmetic + + +File: sicp.info, Node: 2-1-1, Next: 2-1-2, Prev: 2-1, Up: 2-1 + +2.1.1 Example: Arithmetic Operations for Rational Numbers +--------------------------------------------------------- + +Suppose we want to do arithmetic with rational numbers. We want to be +able to add, subtract, multiply, and divide them and to test whether +two rational numbers are equal. + + Let us begin by assuming that we already have a way of constructing +a rational number from a numerator and a denominator. We also assume +that, given a rational number, we have a way of extracting (or +selecting) its numerator and its denominator. Let us further assume +that the constructor and selectors are available as procedures: + + * `(make-rat )' returns therational number whose numerator is + the integer `' and whose denominator is the integer `'. + + * `(numer )' returns the numerator of the rational number `'. + + * `(denom )' returns the denominator of the rational number `'. + + + We are using here a powerful strategy of synthesis: "wishful +thinking". We haven't yet said how a rational number is represented, +or how the procedures `numer', `denom', and `make-rat' should be +implemented. Even so, if we did have these three procedures, we could +then add, subtract, multiply, divide, and test equality by using the +following relations: + + n_1 n_2 n_1 d_2 + n_2 d_1 + --- + --- = ----------------- + d_1 d_2 d_1 d_2 + + n_1 n_2 n_1 d_2 - n_2 d_1 + --- - --- = ----------------- + d_1 d_2 d_1 d_2 + + n_1 n_2 n_1 n_2 + --- * --- = ------- + d_1 d_2 d_1 d_2 + + n_1 / d_1 n_1 d_2 + --------- = ------- + n_2 / d_2 d_1 n_2 + + n_1 n_2 + --- = --- if and only if n_1 d_2 = n_2 d_1 + d_1 d_2 + +We can express these rules as procedures: + + (define (add-rat x y) + (make-rat (+ (* (numer x) (denom y)) + (* (numer y) (denom x))) + (* (denom x) (denom y)))) + + (define (sub-rat x y) + (make-rat (- (* (numer x) (denom y)) + (* (numer y) (denom x))) + (* (denom x) (denom y)))) + + (define (mul-rat x y) + (make-rat (* (numer x) (numer y)) + (* (denom x) (denom y)))) + + (define (div-rat x y) + (make-rat (* (numer x) (denom y)) + (* (denom x) (numer y)))) + + (define (equal-rat? x y) + (= (* (numer x) (denom y)) + (* (numer y) (denom x)))) + + Now we have the operations on rational numbers defined in terms of +the selector and constructor procedures `numer', `denom', and +`make-rat'. But we haven't yet defined these. What we need is some +way to glue together a numerator and a denominator to form a rational +number. + +Pairs +..... + +To enable us to implement the concrete level of our data abstraction, +our language provides a compound structure called a "pair", which can be +constructed with the primitive procedure `cons'. This procedure takes +two arguments and returns a compound data object that contains the two +arguments as parts. Given a pair, we can extract the parts using the +primitive procedures `car' and `cdr'.(1) Thus, we can use `cons', +`car', and `cdr' as follows: + + (define x (cons 1 2)) + + (car x) + 1 + + (cdr x) + 2 + + Notice that a pair is a data object that can be given a name and +manipulated, just like a primitive data object. Moreover, `cons' can +be used to form pairs whose elements are pairs, and so on: + + (define x (cons 1 2)) + + (define y (cons 3 4)) + + (define z (cons x y)) + + (car (car z)) + 1 + + (car (cdr z)) + 3 + + In section *Note 2-2:: we will see how this ability to combine pairs +means that pairs can be used as general-purpose building blocks to +create all sorts of complex data structures. The single compound-data +primitive "pair", implemented by the procedures `cons', `car', and +`cdr', is the only glue we need. Data objects constructed from pairs +are called "list-structured" data. + +Representing rational numbers +............................. + +Pairs offer a natural way to complete the rational-number system. +Simply represent a rational number as a pair of two integers: a +numerator and a denominator. Then `make-rat', `numer', and `denom' are +readily implemented as follows:(2) + + (define (make-rat n d) (cons n d)) + + (define (numer x) (car x)) + + (define (denom x) (cdr x)) + + Also, in order to display the results of our computations, we can +print rational numbers by printing the numerator, a slash, and the +denominator:(3) + + (define (print-rat x) + (newline) + (display (numer x)) + (display "/") + (display (denom x))) + + Now we can try our rational-number procedures: + + (define one-half (make-rat 1 2)) + + (print-rat one-half) + 1/2 + + (define one-third (make-rat 1 3)) + + (print-rat (add-rat one-half one-third)) + 5/6 + + (print-rat (mul-rat one-half one-third)) + 1/6 + + (print-rat (add-rat one-third one-third)) + 6/9 + + As the final example shows, our rational-number implementation does +not reduce rational numbers to lowest terms. We can remedy this by +changing `make-rat'. If we have a `gcd' procedure like the one in +section *Note 1-2-5:: that produces the greatest common divisor of two +integers, we can use `gcd' to reduce the numerator and the denominator +to lowest terms before constructing the pair: + + (define (make-rat n d) + (let ((g (gcd n d))) + (cons (/ n g) (/ d g)))) + + Now we have + + (print-rat (add-rat one-third one-third)) + 2/3 + +as desired. This modification was accomplished by changing the +constructor `make-rat' without changing any of the procedures (such as +`add-rat' and `mul-rat') that implement the actual operations. + + *Exercise 2.1:* Define a better version of `make-rat' that handles + both positive and negative arguments. `Make-rat' should normalize + the sign so that if the rational number is positive, both the + numerator and denominator are positive, and if the rational number + is negative, only the numerator is negative. + + ---------- Footnotes ---------- + + (1) The name `cons' stands for "construct." The names `car' and +`cdr' derive from the original implementation of Lisp on the IBM 704. +That machine had an addressing scheme that allowed one to reference the +"address" and "decrement" parts of a memory location. `Car' stands for +"Contents of Address part of Register" and `cdr' (pronounced +"could-er") stands for "Contents of Decrement part of Register." + + (2) Another way to define the selectors and constructor is + + (define make-rat cons) + (define numer car) + (define denom cdr) + + The first definition associates the name `make-rat' with the value +of the expression `cons', which is the primitive procedure that +constructs pairs. Thus `make-rat' and `cons' are names for the same +primitive constructor. + + Defining selectors and constructors in this way is efficient: +Instead of `make-rat' _calling_ `cons', `make-rat' _is_ `cons', so +there is only one procedure called, not two, when `make-rat' is called. +On the other hand, doing this defeats debugging aids that trace +procedure calls or put breakpoints on procedure calls: You may want to +watch `make-rat' being called, but you certainly don't want to watch +every call to `cons'. + + We have chosen not to use this style of definition in this book. + + (3) `Display' is the Scheme primitive for printing data. The Scheme +primitive `newline' starts a new line for printing. Neither of these +procedures returns a useful value, so in the uses of `print-rat' below, +we show only what `print-rat' prints, not what the interpreter prints +as the value returned by `print-rat'. + + +File: sicp.info, Node: 2-1-2, Next: 2-1-3, Prev: 2-1-1, Up: 2-1 + +2.1.2 Abstraction Barriers +-------------------------- + +Before continuing with more examples of compound data and data +abstraction, let us consider some of the issues raised by the +rational-number example. We defined the rational-number operations in +terms of a constructor `make-rat' and selectors `numer' and `denom'. +In general, the underlying idea of data abstraction is to identify for +each type of data object a basic set of operations in terms of which +all manipulations of data objects of that type will be expressed, and +then to use only those operations in manipulating the data. + + We can envision the structure of the rational-number system as shown +in figure *Note Figure 2-1::. The horizontal lines represent barriers +"abstraction barriers" that isolate different "levels" of the system. +At each level, the barrier separates the programs (above) that use the +data abstraction from the programs (below) that implement the data +abstraction. Programs that use rational numbers manipulate them solely +in terms of the procedures supplied "for public use" by the +rational-number package: `add-rat', `sub-rat', `mul-rat', `div-rat', +and `equal-rat?'. These, in turn, are implemented solely in terms of +the constructor and selectors `make-rat', `numer', and `denom', which +themselves are implemented in terms of pairs. The details of how pairs +are implemented are irrelevant to the rest of the rational-number +package so long as pairs can be manipulated by the use of `cons', +`car', and `cdr'. In effect, procedures at each level are the +interfaces that define the abstraction barriers and connect the +different levels. + + *Figure 2.1:* Data-abstraction barriers in the rational-number + package. + + +------------------------------------+ + --------| Programs that use rational numbers |-------- + +------------------------------------+ + Rational numbers in promblem domain + +---------------------------+ + ------------| add-rat sub-rat ... |------------- + +---------------------------+ + Rational numbers as numerators and denominators + +------------------------+ + --------------| make-rat numer denom |-------------- + +------------------------+ + Rational numbers as pairs + +----------------+ + ------------------| cons car cdr |------------------ + +----------------+ + However pairs are implemented + + This simple idea has many advantages. One advantage is that it +makes programs much easier to maintain and to modify. Any complex data +structure can be represented in a variety of ways with the primitive +data structures provided by a programming language. Of course, the +choice of representation influences the programs that operate on it; +thus, if the representation were to be changed at some later time, all +such programs might have to be modified accordingly. This task could +be time-consuming and expensive in the case of large programs unless +the dependence on the representation were to be confined by design to a +very few program modules. + + For example, an alternate way to address the problem of reducing +rational numbers to lowest terms is to perform the reduction whenever +we access the parts of a rational number, rather than when we construct +it. This leads to different constructor and selector procedures: + + (define (make-rat n d) + (cons n d)) + + (define (numer x) + (let ((g (gcd (car x) (cdr x)))) + (/ (car x) g))) + + (define (denom x) + (let ((g (gcd (car x) (cdr x)))) + (/ (cdr x) g))) + + The difference between this implementation and the previous one lies +in when we compute the `gcd'. If in our typical use of rational +numbers we access the numerators and denominators of the same rational +numbers many times, it would be preferable to compute the `gcd' when +the rational numbers are constructed. If not, we may be better off +waiting until access time to compute the `gcd'. In any case, when we +change from one representation to the other, the procedures `add-rat', +`sub-rat', and so on do not have to be modified at all. + + Constraining the dependence on the representation to a few interface +procedures helps us design programs as well as modify them, because it +allows us to maintain the flexibility to consider alternate +implementations. To continue with our simple example, suppose we are +designing a rational-number package and we can't decide initially +whether to perform the `gcd' at construction time or at selection time. +The data-abstraction methodology gives us a way to defer that decision +without losing the ability to make progress on the rest of the system. + + *Exercise 2.2:* Consider the problem of representing line segments + in a plane. Each segment is represented as a pair of points: a + starting point and an ending point. Define a constructor + `make-segment' and selectors `start-segment' and `end-segment' + that define the representation of segments in terms of points. + Furthermore, a point can be represented as a pair of numbers: the + x coordinate and the y coordinate. Accordingly, specify a + constructor `make-point' and selectors `x-point' and `y-point' + that define this representation. Finally, using your selectors + and constructors, define a procedure `midpoint-segment' that takes + a line segment as argument and returns its midpoint (the point + whose coordinates are the average of the coordinates of the + endpoints). To try your procedures, you'll need a way to print + points: + + (define (print-point p) + (newline) + (display "(") + (display (x-point p)) + (display ",") + (display (y-point p)) + (display ")")) + + *Exercise 2.3:* Implement a representation for rectangles in a + plane. (Hint: You may want to make use of *Note Exercise 2-2::.) + In terms of your constructors and selectors, create procedures + that compute the perimeter and the area of a given rectangle. Now + implement a different representation for rectangles. Can you + design your system with suitable abstraction barriers, so that the + same perimeter and area procedures will work using either + representation? + + +File: sicp.info, Node: 2-1-3, Next: 2-1-4, Prev: 2-1-2, Up: 2-1 + +2.1.3 What Is Meant by Data? +---------------------------- + +We began the rational-number implementation in section *Note 2-1-1:: by +implementing the rational-number operations `add-rat', `sub-rat', and +so on in terms of three unspecified procedures: `make-rat', `numer', +and `denom'. At that point, we could think of the operations as being +defined in terms of data objects--numerators, denominators, and rational +numbers--whose behavior was specified by the latter three procedures. + + But exactly what is meant by "data"? It is not enough to say +"whatever is implemented by the given selectors and constructors." +Clearly, not every arbitrary set of three procedures can serve as an +appropriate basis for the rational-number implementation. We need to +guarantee that, if we construct a rational number `x' from a pair of +integers `n' and `d', then extracting the `numer' and the `denom' of +`x' and dividing them should yield the same result as dividing `n' by +`d'. In other words, `make-rat', `numer', and `denom' must satisfy the +condition that, for any integer `n' and any non-zero integer `d', if +`x' is (`make-rat n d'), then + + (numer x) n + --------- = --- + (denom x) d + + In fact, this is the only condition `make-rat', `numer', and `denom' +must fulfill in order to form a suitable basis for a rational-number +representation. In general, we can think of data as defined by some +collection of selectors and constructors, together with specified +conditions that these procedures must fulfill in order to be a valid +representation.(1) + + This point of view can serve to define not only "high-level" data +objects, such as rational numbers, but lower-level objects as well. +Consider the notion of a pair, which we used in order to define our +rational numbers. We never actually said what a pair was, only that +the language supplied procedures `cons', `car', and `cdr' for operating +on pairs. But the only thing we need to know about these three +operations is that if we glue two objects together using `cons' we can +retrieve the objects using `car' and `cdr'. That is, the operations +satisfy the condition that, for any objects `x' and `y', if `z' is +`(cons x y)' then `(car z)' is `x' and `(cdr z)' is `y'. Indeed, we +mentioned that these three procedures are included as primitives in our +language. However, any triple of procedures that satisfies the above +condition can be used as the basis for implementing pairs. This point +is illustrated strikingly by the fact that we could implement `cons', +`car', and `cdr' without using any data structures at all but only +using procedures. Here are the definitions: + + (define (cons x y) + (define (dispatch m) + (cond ((= m 0) x) + ((= m 1) y) + (else (error "Argument not 0 or 1 -- CONS" m)))) + dispatch) + + (define (car z) (z 0)) + + (define (cdr z) (z 1)) + + This use of procedures corresponds to nothing like our intuitive +notion of what data should be. Nevertheless, all we need to do to show +that this is a valid way to represent pairs is to verify that these +procedures satisfy the condition given above. + + The subtle point to notice is that the value returned by `(cons x +y)' is a procedure--namely the internally defined procedure `dispatch', +which takes one argument and returns either `x' or `y' depending on +whether the argument is 0 or 1. Correspondingly, `(car z)' is defined +to apply `z' to 0. Hence, if `z' is the procedure formed by `(cons x +y)', then `z' applied to 0 will yield `x'. Thus, we have shown that +`(car (cons x y))' yields `x', as desired. Similarly, `(cdr (cons x +y))' applies the procedure returned by `(cons x y)' to 1, which returns +`y'. Therefore, this procedural implementation of pairs is a valid +implementation, and if we access pairs using only `cons', `car', and +`cdr' we cannot distinguish this implementation from one that uses +"real" data structures. + + The point of exhibiting the procedural representation of pairs is +not that our language works this way (Scheme, and Lisp systems in +general, implement pairs directly, for efficiency reasons) but that it +could work this way. The procedural representation, although obscure, +is a perfectly adequate way to represent pairs, since it fulfills the +only conditions that pairs need to fulfill. This example also +demonstrates that the ability to manipulate procedures as objects +automatically provides the ability to represent compound data. This +may seem a curiosity now, but procedural representations of data will +play a central role in our programming repertoire. This style of +programming is often called "message passing", and we will be using it +as a basic tool in *Note Chapter 3:: when we address the issues of +modeling and simulation. + + *Exercise 2.4:* Here is an alternative procedural representation + of pairs. For this representation, verify that `(car (cons x y))' + yields `x' for any objects `x' and `y'. + + (define (cons x y) + (lambda (m) (m x y))) + + (define (car z) + (z (lambda (p q) p))) + + What is the corresponding definition of `cdr'? (Hint: To verify + that this works, make use of the substitution model of section + *Note 1-1-5::.) + + *Exercise 2.5:* Show that we can represent pairs of nonnegative + integers using only numbers and arithmetic operations if we + represent the pair a and b as the integer that is the product 2^a + 3^b. Give the corresponding definitions of the procedures `cons', + `car', and `cdr'. + + *Exercise 2.6:* In case representing pairs as procedures wasn't + mind-boggling enough, consider that, in a language that can + manipulate procedures, we can get by without numbers (at least + insofar as nonnegative integers are concerned) by implementing 0 + and the operation of adding 1 as + + (define zero (lambda (f) (lambda (x) x))) + + (define (add-1 n) + (lambda (f) (lambda (x) (f ((n f) x))))) + + This representation is known as "Church numerals", after its + inventor, Alonzo Church, the logician who invented the [lambda] + calculus. + + Define `one' and `two' directly (not in terms of `zero' and + `add-1'). (Hint: Use substitution to evaluate `(add-1 zero)'). + Give a direct definition of the addition procedure `+' (not in + terms of repeated application of `add-1'). + + ---------- Footnotes ---------- + + (1) Surprisingly, this idea is very difficult to formulate +rigorously. There are two approaches to giving such a formulation. One, +pioneered by C. A. R. Hoare (1972), is known as the method of models +"abstract models". It formalizes the "procedures plus conditions" +specification as outlined in the rational-number example above. Note +that the condition on the rational-number representation was stated in +terms of facts about integers (equality and division). In general, +abstract models define new kinds of data objects in terms of previously +defined types of data objects. Assertions about data objects can +therefore be checked by reducing them to assertions about previously +defined data objects. Another approach, introduced by Zilles at MIT, +by Goguen, Thatcher, Wagner, and Wright at IBM (see Thatcher, Wagner, +and Wright 1978), and by Guttag at Toronto (see Guttag 1977), is called "algebraic +specification". It regards the "procedures" as elements of an abstract +algebraic system whose behavior is specified by axioms that correspond +to our "conditions," and uses the techniques of abstract algebra to +check assertions about data objects. Both methods are surveyed in the +paper by Liskov and Zilles (1975). + + +File: sicp.info, Node: 2-1-4, Prev: 2-1-3, Up: 2-1 + +2.1.4 Extended Exercise: Interval Arithmetic +-------------------------------------------- + +Alyssa P. Hacker is designing a system to help people solve engineering +problems. One feature she wants to provide in her system is the +ability to manipulate inexact quantities (such as measured parameters +of physical devices) with known precision, so that when computations +are done with such approximate quantities the results will be numbers +of known precision. + + Electrical engineers will be using Alyssa's system to compute +electrical quantities. It is sometimes necessary for them to compute +the value of a parallel equivalent resistance R_p of two resistors R_1 +and R_2 using the formula + + 1 + R_p = ------------- + 1/R_1 + 1/R_2 + + Resistance values are usually known only up to some tolerance +guaranteed by the manufacturer of the resistor. For example, if you +buy a resistor labeled "6.8 ohms with 10% tolerance" you can only be +sure that the resistor has a resistance between 6.8 - 0.68 = 6.12 and +6.8 + 0.68 = 7.48 ohms. Thus, if you have a 6.8-ohm 10% resistor in +parallel with a 4.7-ohm 5% resistor, the resistance of the combination +can range from about 2.58 ohms (if the two resistors are at the lower +bounds) to about 2.97 ohms (if the two resistors are at the upper +bounds). + + Alyssa's idea is to implement "interval arithmetic" as a set of +arithmetic operations for combining "intervals" (objects that represent +the range of possible values of an inexact quantity). The result of +adding, subtracting, multiplying, or dividing two intervals is itself +an interval, representing the range of the result. + + Alyssa postulates the existence of an abstract object called an +"interval" that has two endpoints: a lower bound and an upper bound. +She also presumes that, given the endpoints of an interval, she can +construct the interval using the data constructor `make-interval'. +Alyssa first writes a procedure for adding two intervals. She reasons +that the minimum value the sum could be is the sum of the two lower +bounds and the maximum value it could be is the sum of the two upper +bounds: + + (define (add-interval x y) + (make-interval (+ (lower-bound x) (lower-bound y)) + (+ (upper-bound x) (upper-bound y)))) + + Alyssa also works out the product of two intervals by finding the +minimum and the maximum of the products of the bounds and using them as +the bounds of the resulting interval. (`Min' and `max' are primitives +that find the minimum or maximum of any number of arguments.) + + (define (mul-interval x y) + (let ((p1 (* (lower-bound x) (lower-bound y))) + (p2 (* (lower-bound x) (upper-bound y))) + (p3 (* (upper-bound x) (lower-bound y))) + (p4 (* (upper-bound x) (upper-bound y)))) + (make-interval (min p1 p2 p3 p4) + (max p1 p2 p3 p4)))) + + To divide two intervals, Alyssa multiplies the first by the +reciprocal of the second. Note that the bounds of the reciprocal +interval are the reciprocal of the upper bound and the reciprocal of +the lower bound, in that order. + + (define (div-interval x y) + (mul-interval x + (make-interval (/ 1.0 (upper-bound y)) + (/ 1.0 (lower-bound y))))) + + *Exercise 2.7:* Alyssa's program is incomplete because she has not + specified the implementation of the interval abstraction. Here is + a definition of the interval constructor: + + (define (make-interval a b) (cons a b)) + + Define selectors `upper-bound' and `lower-bound' to complete the + implementation. + + *Exercise 2.8:* Using reasoning analogous to Alyssa's, describe + how the difference of two intervals may be computed. Define a + corresponding subtraction procedure, called `sub-interval'. + + *Exercise 2.9:* The "width" of an interval is half of the + difference between its upper and lower bounds. The width is a + measure of the uncertainty of the number specified by the + interval. For some arithmetic operations the width of the result + of combining two intervals is a function only of the widths of the + argument intervals, whereas for others the width of the + combination is not a function of the widths of the argument + intervals. Show that the width of the sum (or difference) of two + intervals is a function only of the widths of the intervals being + added (or subtracted). Give examples to show that this is not + true for multiplication or division. + + *Exercise 2.10:* Ben Bitdiddle, an expert systems programmer, + looks over Alyssa's shoulder and comments that it is not clear what + it means to divide by an interval that spans zero. Modify + Alyssa's code to check for this condition and to signal an error + if it occurs. + + *Exercise 2.11:* In passing, Ben also cryptically comments: "By + testing the signs of the endpoints of the intervals, it is + possible to break `mul-interval' into nine cases, only one of which + requires more than two multiplications." Rewrite this procedure + using Ben's suggestion. + + After debugging her program, Alyssa shows it to a potential user, + who complains that her program solves the wrong problem. He wants + a program that can deal with numbers represented as a center value + and an additive tolerance; for example, he wants to work with + intervals such as 3.5 +/- 0.15 rather than [3.35, 3.65]. Alyssa + returns to her desk and fixes this problem by supplying an + alternate constructor and alternate selectors: + + (define (make-center-width c w) + (make-interval (- c w) (+ c w))) + + (define (center i) + (/ (+ (lower-bound i) (upper-bound i)) 2)) + + (define (width i) + (/ (- (upper-bound i) (lower-bound i)) 2)) + + Unfortunately, most of Alyssa's users are engineers. Real + engineering situations usually involve measurements with only a + small uncertainty, measured as the ratio of the width of the + interval to the midpoint of the interval. Engineers usually + specify percentage tolerances on the parameters of devices, as in + the resistor specifications given earlier. + + *Exercise 2.12:* Define a constructor `make-center-percent' that + takes a center and a percentage tolerance and produces the desired + interval. You must also define a selector `percent' that produces + the percentage tolerance for a given interval. The `center' + selector is the same as the one shown above. + + *Exercise 2.13:* Show that under the assumption of small + percentage tolerances there is a simple formula for the approximate + percentage tolerance of the product of two intervals in terms of + the tolerances of the factors. You may simplify the problem by + assuming that all numbers are positive. + + After considerable work, Alyssa P. Hacker delivers her finished + system. Several years later, after she has forgotten all about + it, she gets a frenzied call from an irate user, Lem E. Tweakit. + It seems that Lem has noticed that the formula for parallel + resistors can be written in two algebraically equivalent ways: + + R_1 R_2 + --------- + R_1 + R_2 + + and + + 1 + ------------- + 1/R_1 + 1/R_2 + + He has written the following two programs, each of which computes + the parallel-resistors formula differently: + + (define (par1 r1 r2) + (div-interval (mul-interval r1 r2) + (add-interval r1 r2))) + + (define (par2 r1 r2) + (let ((one (make-interval 1 1))) + (div-interval one + (add-interval (div-interval one r1) + (div-interval one r2))))) + + Lem complains that Alyssa's program gives different answers for + the two ways of computing. This is a serious complaint. + + *Exercise 2.14:* Demonstrate that Lem is right. Investigate the + behavior of the system on a variety of arithmetic expressions. + Make some intervals A and B, and use them in computing the + expressions A/A and A/B. You will get the most insight by using + intervals whose width is a small percentage of the center value. + Examine the results of the computation in center-percent form (see + *Note Exercise 2-12::). + + *Exercise 2.15:* Eva Lu Ator, another user, has also noticed the + different intervals computed by different but algebraically + equivalent expressions. She says that a formula to compute with + intervals using Alyssa's system will produce tighter error bounds + if it can be written in such a form that no variable that + represents an uncertain number is repeated. Thus, she says, + `par2' is a "better" program for parallel resistances than `par1'. + Is she right? Why? + + *Exercise 2.16:* Explain, in general, why equivalent algebraic + expressions may lead to different answers. Can you devise an + interval-arithmetic package that does not have this shortcoming, + or is this task impossible? (Warning: This problem is very + difficult.) + + +File: sicp.info, Node: 2-2, Next: 2-3, Prev: 2-1, Up: Chapter 2 + +2.2 Hierarchical Data and the Closure Property +============================================== + +As we have seen, pairs provide a primitive "glue" that we can use to +construct compound data objects. *Note Figure 2-2:: shows a standard +way to visualize a pair--in this case, the pair formed by `(cons 1 2)'. +In this representation, which is called "box-and-pointer notation", +each object is shown as a "pointer" to a box. The box for a primitive +object contains a representation of the object. For example, the box +for a number contains a numeral. The box for a pair is actually a +double box, the left part containing (a pointer to) the `car' of the +pair and the right part containing the `cdr'. + + *Figure 2.2:* Box-and-pointer representation of `(cons 1 2)'. + + +---+---+ +---+ + ---->| * | *-+---->| 2 | + +-|-+---+ +---+ + | + V + +---+ + | 1 | + +---+ + + We have already seen that `cons' can be used to combine not only +numbers but pairs as well. (You made use of this fact, or should have, +in doing *Note Exercise 2-2:: and *Note Exercise 2-3::.) As a +consequence, pairs provide a universal building block from which we can +construct all sorts of data structures. *Note Figure 2-3:: shows two +ways to use pairs to combine the numbers 1, 2, 3, and 4. + + *Figure 2.3:* Two ways to combine 1, 2, 3, and 4 using pairs. + + +---+---+ +---+---+ +---+---+ +---+ + ---->| * | *-+---->| * | * | ---->| * | *-+---->| 4 | + +-|-+---+ +-|-+-|-+ +-|-+---+ +---+ + | | | | + V V V V + +---+---+ +---+ +---+ +---+---+ +---+---+ + | * | * | | 3 | | 4 | | * | *-+---->| * | * | + +-|-+-|-+ +---+ +---+ +-|-+---+ +-|-+-|-+ + | | | | | + V V V V V + +---+ +---+ +---+ +---+ +---+ + | 1 | | 2 | | 1 | | 2 | | 3 | + +---+ +---+ +---+ +---+ +---+ + + (cons (cons 1 2) (cons (cons 1 + (cons 3 4)) (cons 2 3)) + 4) + + The ability to create pairs whose elements are pairs is the essence +of list structure's importance as a representational tool. We refer to +this ability as the "closure property" of `cons'. In general, an +operation for combining data objects satisfies the closure property if +the results of combining things with that operation can themselves be +combined using the same operation.(1) Closure is the key to power in +any means of combination because it permits us to create "hierarchical" +structures--structures made up of parts, which themselves are made up +of parts, and so on. + + From the outset of *Note Chapter 1::, we've made essential use of +closure in dealing with procedures, because all but the very simplest +programs rely on the fact that the elements of a combination can +themselves be combinations. In this section, we take up the +consequences of closure for compound data. We describe some +conventional techniques for using pairs to represent sequences and +trees, and we exhibit a graphics language that illustrates closure in a +vivid way.(2) + +* Menu: + +* 2-2-1:: Representing Sequences +* 2-2-2:: Hierarchical Structures +* 2-2-3:: Sequences as Conventional Interfaces +* 2-2-4:: Example: A Picture Language + + ---------- Footnotes ---------- + + (1) The use of the word "closure" here comes from abstract algebra, +where a set of elements is said to be closed under an operation if +applying the operation to elements in the set produces an element that +is again an element of the set. The Lisp community also +(unfortunately) uses the word "closure" to describe a totally unrelated +concept: A closure is an implementation technique for representing +procedures with free variables. We do not use the word "closure" in +this second sense in this book. + + (2) The notion that a means of combination should satisfy closure is +a straightforward idea. Unfortunately, the data combiners provided in +many popular programming languages do not satisfy closure, or make +closure cumbersome to exploit. In Fortran or Basic, one typically +combines data elements by assembling them into arrays--but one cannot +form arrays whose elements are themselves arrays. Pascal and C admit +structures whose elements are structures. However, this requires that +the programmer manipulate pointers explicitly, and adhere to the +restriction that each field of a structure can contain only elements of +a prespecified form. Unlike Lisp with its pairs, these languages have +no built-in general-purpose glue that makes it easy to manipulate +compound data in a uniform way. This limitation lies behind Alan +Perlis's comment in his foreword to this book: "In Pascal the plethora +of declarable data structures induces a specialization within functions +that inhibits and penalizes casual cooperation. It is better to have +100 functions operate on one data structure than to have 10 functions +operate on 10 data structures." + + +File: sicp.info, Node: 2-2-1, Next: 2-2-2, Prev: 2-2, Up: 2-2 + +2.2.1 Representing Sequences +---------------------------- + + *Figure 2.4:* The sequence 1, 2, 3, 4 represented as a chain of + pairs. + + +---+---+ +---+---+ +---+---+ +---+---+ + ---->| * | *-+---->| * | *-+---->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ +-|-+---+ + | | | | + V V V V + +---+ +---+ +---+ +---+ + | 1 | | 2 | | 3 | | 4 | + +---+ +---+ +---+ +---+ + +One of the useful structures we can build with pairs is a "sequence"--an +ordered collection of data objects. There are, of course, many ways to +represent sequences in terms of pairs. One particularly +straightforward representation is illustrated in *Note Figure 2-4::, +where the sequence 1, 2, 3, 4 is represented as a chain of pairs. The +`car' of each pair is the corresponding item in the chain, and the +`cdr' of the pair is the next pair in the chain. The `cdr' of the +final pair signals the end of the sequence by pointing to a +distinguished value that is not a pair, represented in box-and-pointer +diagrams as a diagonal line and in programs as the value of the +variable `nil'. The entire sequence is constructed by nested `cons' +operations: + + (cons 1 + (cons 2 + (cons 3 + (cons 4 nil)))) + + Such a sequence of pairs, formed by nested `cons'es, is called a "list", +and Scheme provides a primitive called `list' to help in constructing +lists.(1) The above sequence could be produced by `(list 1 2 3 4)'. +In general, + + (list ... ) + +is equivalent to + + (cons + (cons + (cons ... + (cons + nil) + ...))) + + Lisp systems conventionally print lists by printing the sequence of +elements, enclosed in parentheses. Thus, the data object in *Note +Figure 2-4:: is printed as `(1 2 3 4)': + + (define one-through-four (list 1 2 3 4)) + + one-through-four + (1 2 3 4) + + Be careful not to confuse the expression `(list 1 2 3 4)' with the +list `(1 2 3 4)', which is the result obtained when the expression is +evaluated. Attempting to evaluate the expression `(1 2 3 4)' will +signal an error when the interpreter tries to apply the procedure `1' to +arguments `2', `3', and `4'. + + We can think of `car' as selecting the first item in the list, and of +`cdr' as selecting the sublist consisting of all but the first item. +Nested applications of `car' and `cdr' can be used to extract the +second, third, and subsequent items in the list.(2) The constructor +`cons' makes a list like the original one, but with an additional item +at the beginning. + + (car one-through-four) + 1 + + (cdr one-through-four) + (2 3 4) + + (car (cdr one-through-four)) + 2 + + (cons 10 one-through-four) + (10 1 2 3 4) + + (cons 5 one-through-four) + (5 1 2 3 4) + + The value of `nil', used to terminate the chain of pairs, can be +thought of as a sequence of no elements, the "empty list". The word "nil" +is a contraction of the Latin word _nihil_, which means "nothing."(3) + +List operations +............... + +The use of pairs to represent sequences of elements as lists is +accompanied by conventional programming techniques for manipulating +lists by successively "`cdr'ing down" the lists. For example, the +procedure `list-ref' takes as arguments a list and a number n and +returns the nth item of the list. It is customary to number the +elements of the list beginning with 0. The method for computing +`list-ref' is the following: + + * For n = 0, `list-ref' should return the `car' of the list. + + * Otherwise, `list-ref' should return the (n - 1)st item of the + `cdr' of the list. + + + (define (list-ref items n) + (if (= n 0) + (car items) + (list-ref (cdr items) (- n 1)))) + + (define squares (list 1 4 9 16 25)) + + (list-ref squares 3) + 16 + + Often we `cdr' down the whole list. To aid in this, Scheme includes +a primitive predicate `null?', which tests whether its argument is the +empty list. The procedure `length', which returns the number of items +in a list, illustrates this typical pattern of use: + + (define (length items) + (if (null? items) + 0 + (+ 1 (length (cdr items))))) + + (define odds (list 1 3 5 7)) + + (length odds) + 4 + + The `length' procedure implements a simple recursive plan. The +reduction step is: + + * The `length' of any list is 1 plus the `length' of the `cdr' of + the list. + + + This is applied successively until we reach the base case: + + * The `length' of the empty list is 0. + + + We could also compute `length' in an iterative style: + + (define (length items) + (define (length-iter a count) + (if (null? a) + count + (length-iter (cdr a) (+ 1 count)))) + (length-iter items 0)) + + Another conventional programming technique is to "`cons' up" an +answer list while `cdr'ing down a list, as in the procedure `append', +which takes two lists as arguments and combines their elements to make +a new list: + + (append squares odds) + (1 4 9 16 25 1 3 5 7) + + (append odds squares) + (1 3 5 7 1 4 9 16 25) + + `Append' is also implemented using a recursive plan. To `append' +lists `list1' and `list2', do the following: + + * If `list1' is the empty list, then the result is just `list2'. + + * Otherwise, `append' the `cdr' of `list1' and `list2', and `cons' + the `car' of `list1' onto the result: + + + (define (append list1 list2) + (if (null? list1) + list2 + (cons (car list1) (append (cdr list1) list2)))) + + *Exercise 2.17:* Define a procedure `last-pair' that returns the + list that contains only the last element of a given (nonempty) + list: + + (last-pair (list 23 72 149 34)) + (34) + + *Exercise 2.18:* Define a procedure `reverse' that takes a list as + argument and returns a list of the same elements in reverse order: + + (reverse (list 1 4 9 16 25)) + (25 16 9 4 1) + + *Exercise 2.19:* Consider the change-counting program of section + *Note 1-2-2::. It would be nice to be able to easily change the + currency used by the program, so that we could compute the number + of ways to change a British pound, for example. As the program is + written, the knowledge of the currency is distributed partly into + the procedure `first-denomination' and partly into the procedure + `count-change' (which knows that there are five kinds of U.S. + coins). It would be nicer to be able to supply a list of coins to + be used for making change. + + We want to rewrite the procedure `cc' so that its second argument + is a list of the values of the coins to use rather than an integer + specifying which coins to use. We could then have lists that + defined each kind of currency: + + (define us-coins (list 50 25 10 5 1)) + + (define uk-coins (list 100 50 20 10 5 2 1 0.5)) + + We could then call `cc' as follows: + + (cc 100 us-coins) + 292 + + To do this will require changing the program `cc' somewhat. It + will still have the same form, but it will access its second + argument differently, as follows: + + (define (cc amount coin-values) + (cond ((= amount 0) 1) + ((or (< amount 0) (no-more? coin-values)) 0) + (else + (+ (cc amount + (except-first-denomination coin-values)) + (cc (- amount + (first-denomination coin-values)) + coin-values))))) + + Define the procedures `first-denomination', + `except-first-denomination', and `no-more?' in terms of primitive + operations on list structures. Does the order of the list + `coin-values' affect the answer produced by `cc'? Why or why not? + + *Exercise 2.20:* The procedures `+', `*', and `list' take + arbitrary numbers of arguments. One way to define such procedures + is to use `define' with notation "dotted-tail notation". In a + procedure definition, a parameter list that has a dot before the + last parameter name indicates that, when the procedure is called, + the initial parameters (if any) will have as values the initial + arguments, as usual, but the final parameter's value will be a "list" + of any remaining arguments. For instance, given the definition + + (define (f x y . z) ) + + the procedure `f' can be called with two or more arguments. If we + evaluate + + (f 1 2 3 4 5 6) + + then in the body of `f', `x' will be 1, `y' will be 2, and `z' + will be the list `(3 4 5 6)'. Given the definition + + (define (g . w) ) + + the procedure `g' can be called with zero or more arguments. If we + evaluate + + (g 1 2 3 4 5 6) + + then in the body of `g', `w' will be the list `(1 2 3 4 5 6)'.(4) + + Use this notation to write a procedure `same-parity' that takes + one or more integers and returns a list of all the arguments that + have the same even-odd parity as the first argument. For example, + + (same-parity 1 2 3 4 5 6 7) + (1 3 5 7) + + (same-parity 2 3 4 5 6 7) + (2 4 6) + +Mapping over lists +.................. + +One extremely useful operation is to apply some transformation to each +element in a list and generate the list of results. For instance, the +following procedure scales each number in a list by a given factor: + + (define (scale-list items factor) + (if (null? items) + nil + (cons (* (car items) factor) + (scale-list (cdr items) factor)))) + + (scale-list (list 1 2 3 4 5) 10) + (10 20 30 40 50) + + We can abstract this general idea and capture it as a common pattern +expressed as a higher-order procedure, just as in section *Note 1-3::. +The higher-order procedure here is called `map'. `Map' takes as +arguments a procedure of one argument and a list, and returns a list of +the results produced by applying the procedure to each element in the +list:(5) + + (define (map proc items) + (if (null? items) + nil + (cons (proc (car items)) + (map proc (cdr items))))) + + (map abs (list -10 2.5 -11.6 17)) + (10 2.5 11.6 17) + + (map (lambda (x) (* x x)) + (list 1 2 3 4)) + (1 4 9 16) + + Now we can give a new definition of `scale-list' in terms of `map': + + (define (scale-list items factor) + (map (lambda (x) (* x factor)) + items)) + + `Map' is an important construct, not only because it captures a +common pattern, but because it establishes a higher level of +abstraction in dealing with lists. In the original definition of +`scale-list', the recursive structure of the program draws attention to +the element-by-element processing of the list. Defining `scale-list' +in terms of `map' suppresses that level of detail and emphasizes that +scaling transforms a list of elements to a list of results. The +difference between the two definitions is not that the computer is +performing a different process (it isn't) but that we think about the +process differently. In effect, `map' helps establish an abstraction +barrier that isolates the implementation of procedures that transform +lists from the details of how the elements of the list are extracted +and combined. Like the barriers shown in *Note Figure 2-1::, this +abstraction gives us the flexibility to change the low-level details of +how sequences are implemented, while preserving the conceptual +framework of operations that transform sequences to sequences. Section +*Note 2-2-3:: expands on this use of sequences as a framework for +organizing programs. + + *Exercise 2.21:* The procedure `square-list' takes a list of + numbers as argument and returns a list of the squares of those + numbers. + + (square-list (list 1 2 3 4)) + (1 4 9 16) + + Here are two different definitions of `square-list'. Complete + both of them by filling in the missing expressions: + + (define (square-list items) + (if (null? items) + nil + (cons ))) + + (define (square-list items) + (map )) + + *Exercise 2.22:* Louis Reasoner tries to rewrite the first + `square-list' procedure of *Note Exercise 2-21:: so that it + evolves an iterative process: + + (define (square-list items) + (define (iter things answer) + (if (null? things) + answer + (iter (cdr things) + (cons (square (car things)) + answer)))) + (iter items nil)) + + Unfortunately, defining `square-list' this way produces the answer + list in the reverse order of the one desired. Why? + + Louis then tries to fix his bug by interchanging the arguments to + `cons': + + (define (square-list items) + (define (iter things answer) + (if (null? things) + answer + (iter (cdr things) + (cons answer + (square (car things)))))) + (iter items nil)) + + This doesn't work either. Explain. + + *Exercise 2.23:* The procedure `for-each' is similar to `map'. It + takes as arguments a procedure and a list of elements. However, + rather than forming a list of the results, `for-each' just applies + the procedure to each of the elements in turn, from left to right. + The values returned by applying the procedure to the elements are + not used at all--`for-each' is used with procedures that perform + an action, such as printing. For example, + + (for-each (lambda (x) (newline) (display x)) + (list 57 321 88)) + 57 + 321 + 88 + + The value returned by the call to `for-each' (not illustrated + above) can be something arbitrary, such as true. Give an + implementation of `for-each'. + + ---------- Footnotes ---------- + + (1) In this book, we use "list" to mean a chain of pairs terminated +by the end-of-list marker. In contrast, the term "list structure" +refers to any data structure made out of pairs, not just to lists. + + (2) Since nested applications of `car' and `cdr' are cumbersome to +write, Lisp dialects provide abbreviations for them--for instance, + + (cadr (ARG)) = (car (cdr (ARG))) + + The names of all such procedures start with `c' and end with `r'. +Each `a' between them stands for a `car' operation and each `d' for a +`cdr' operation, to be applied in the same order in which they appear +in the name. The names `car' and `cdr' persist because simple +combinations like `cadr' are pronounceable. + + (3) It's remarkable how much energy in the standardization of Lisp +dialects has been dissipated in arguments that are literally over +nothing: Should `nil' be an ordinary name? Should the value of `nil' +be a symbol? Should it be a list? Should it be a pair? In Scheme, +`nil' is an ordinary name, which we use in this section as a variable +whose value is the end-of-list marker (just as `true' is an ordinary +variable that has a true value). Other dialects of Lisp, including +Common Lisp, treat `nil' as a special symbol. The authors of this +book, who have endured too many language standardization brawls, would +like to avoid the entire issue. Once we have introduced quotation in +section *Note 2-3::, we will denote the empty list as `'()' and +dispense with the variable `nil' entirely. + + (4) To define `f' and `g' using `lambda' we would write + + (define f (lambda (x y . z) )) + (define g (lambda w )) + + (5) [Footnote 12] Scheme standardly provides a `map' procedure that +is more general than the one described here. This more general `map' +takes a procedure of n arguments, together with n lists, and applies +the procedure to all the first elements of the lists, all the second +elements of the lists, and so on, returning a list of the results. For +example: + + (map + (list 1 2 3) (list 40 50 60) (list 700 800 900)) + (741 852 963) + + (map (lambda (x y) (+ x (* 2 y))) + (list 1 2 3) + (list 4 5 6)) + (9 12 15) + + +File: sicp.info, Node: 2-2-2, Next: 2-2-3, Prev: 2-2-1, Up: 2-2 + +2.2.2 Hierarchical Structures +----------------------------- + +The representation of sequences in terms of lists generalizes naturally +to represent sequences whose elements may themselves be sequences. For +example, we can regard the object `((1 2) 3 4)' constructed by + + (cons (list 1 2) (list 3 4)) + +as a list of three items, the first of which is itself a list, `(1 2)'. +Indeed, this is suggested by the form in which the result is printed by +the interpreter. *Note Figure 2-5:: shows the representation of this +structure in terms of pairs. + + *Figure 2.5:* Structure formed by `(cons (list 1 2) (list 3 4))'. + + (3 4) + | + V + ((1 2) 3 4) +---+---+ +---+---+ +---+---+ + ---->| * | *-+----------------->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ + | | | + V V V + (1 2) +---+---+ +---+---+ +---+ +---+ + ---->| * | *-+---->| * | / | | 3 | | 4 | + +-|-+---+ +-|-+---+ +---+ +---+ + | | + V V + +---+ +---+ + | 1 | | 2 | + +---+ +---+ + + Another way to think of sequences whose elements are sequences is as "trees". +The elements of the sequence are the branches of the tree, and +elements that are themselves sequences are subtrees. *Note Figure 2-6:: +shows the structure in *Note Figure 2-5:: viewed as a tree. + + *Figure 2.6:* The list structure in *Note Figure 2-5:: viewed as a + tree. + + ((1 2) 3 4) + /\\ + / | \ + (1 2) 3 4 + / \ + 1 2 + + Recursion is a natural tool for dealing with tree structures, since +we can often reduce operations on trees to operations on their +branches, which reduce in turn to operations on the branches of the +branches, and so on, until we reach the leaves of the tree. As an +example, compare the `length' procedure of section *Note 2-2-1:: with +the `count-leaves' procedure, which returns the total number of leaves +of a tree: + + (define x (cons (list 1 2) (list 3 4))) + + (length x) + 3 + + (count-leaves x) + 4 + + (list x x) + (((1 2) 3 4) ((1 2) 3 4)) + + (length (list x x)) + 2 + + (count-leaves (list x x)) + 8 + + To implement `count-leaves', recall the recursive plan for computing +`length': + + * `Length' of a list `x' is 1 plus `length' of the `cdr' of `x'. + + * `Length' of the empty list is 0. + + + `Count-leaves' is similar. The value for the empty list is the same: + + * `Count-leaves' of the empty list is 0. + + + But in the reduction step, where we strip off the `car' of the list, +we must take into account that the `car' may itself be a tree whose +leaves we need to count. Thus, the appropriate reduction step is + + * `Count-leaves' of a tree `x' is `count-leaves' of the `car' of `x' + plus `count-leaves' of the `cdr' of `x'. + + + Finally, by taking `car's we reach actual leaves, so we need another +base case: + + * `Count-leaves' of a leaf is 1. + + + To aid in writing recursive procedures on trees, Scheme provides the +primitive predicate `pair?', which tests whether its argument is a +pair. Here is the complete procedure:(1) + + (define (count-leaves x) + (cond ((null? x) 0) + ((not (pair? x)) 1) + (else (+ (count-leaves (car x)) + (count-leaves (cdr x)))))) + + *Exercise 2.24:* Suppose we evaluate the expression `(list 1 (list + 2 (list 3 4)))'. Give the result printed by the interpreter, the + corresponding box-and-pointer structure, and the interpretation of + this as a tree (as in *Note Figure 2-6::). + + *Exercise 2.25:* Give combinations of `car's and `cdr's that will + pick 7 from each of the following lists: + + (1 3 (5 7) 9) + + ((7)) + + (1 (2 (3 (4 (5 (6 7)))))) + + *Exercise 2.26:* Suppose we define `x' and `y' to be two lists: + + (define x (list 1 2 3)) + + (define y (list 4 5 6)) + + What result is printed by the interpreter in response to + evaluating each of the following expressions: + + (append x y) + + (cons x y) + + (list x y) + + *Exercise 2.27:* Modify your `reverse' procedure of *Note Exercise + 2-18:: to produce a `deep-reverse' procedure that takes a list as + argument and returns as its value the list with its elements + reversed and with all sublists deep-reversed as well. For example, + + (define x (list (list 1 2) (list 3 4))) + + x + ((1 2) (3 4)) + + (reverse x) + ((3 4) (1 2)) + + (deep-reverse x) + ((4 3) (2 1)) + + *Exercise 2.28:* Write a procedure `fringe' that takes as argument + a tree (represented as a list) and returns a list whose elements + are all the leaves of the tree arranged in left-to-right order. + For example, + + (define x (list (list 1 2) (list 3 4))) + + (fringe x) + (1 2 3 4) + + (fringe (list x x)) + (1 2 3 4 1 2 3 4) + + *Exercise 2.29:* A binary mobile consists of two branches, a left + branch and a right branch. Each branch is a rod of a certain + length, from which hangs either a weight or another binary mobile. + We can represent a binary mobile using compound data by + constructing it from two branches (for example, using `list'): + + (define (make-mobile left right) + (list left right)) + + A branch is constructed from a `length' (which must be a number) + together with a `structure', which may be either a number + (representing a simple weight) or another mobile: + + (define (make-branch length structure) + (list length structure)) + + a. Write the corresponding selectors `left-branch' and + `right-branch', which return the branches of a mobile, and + `branch-length' and `branch-structure', which return the + components of a branch. + + b. Using your selectors, define a procedure `total-weight' that + returns the total weight of a mobile. + + c. A mobile is said to be "balanced" if the torque applied by + its top-left branch is equal to that applied by its top-right + branch (that is, if the length of the left rod multiplied by + the weight hanging from that rod is equal to the + corresponding product for the right side) and if each of the + submobiles hanging off its branches is balanced. Design a + predicate that tests whether a binary mobile is balanced. + + d. Suppose we change the representation of mobiles so that the + constructors are + + (define (make-mobile left right) + (cons left right)) + + (define (make-branch length structure) + (cons length structure)) + + How much do you need to change your programs to convert to + the new representation? + + +Mapping over trees +.................. + +Just as `map' is a powerful abstraction for dealing with sequences, +`map' together with recursion is a powerful abstraction for dealing with +trees. For instance, the `scale-tree' procedure, analogous to +`scale-list' of section *Note 2-2-1::, takes as arguments a numeric +factor and a tree whose leaves are numbers. It returns a tree of the +same shape, where each number is multiplied by the factor. The +recursive plan for `scale-tree' is similar to the one for +`count-leaves': + + (define (scale-tree tree factor) + (cond ((null? tree) nil) + ((not (pair? tree)) (* tree factor)) + (else (cons (scale-tree (car tree) factor) + (scale-tree (cdr tree) factor))))) + + (scale-tree (list 1 (list 2 (list 3 4) 5) (list 6 7)) + 10) + (10 (20 (30 40) 50) (60 70)) + + Another way to implement `scale-tree' is to regard the tree as a +sequence of sub-trees and use `map'. We map over the sequence, scaling +each sub-tree in turn, and return the list of results. In the base +case, where the tree is a leaf, we simply multiply by the factor: + + (define (scale-tree tree factor) + (map (lambda (sub-tree) + (if (pair? sub-tree) + (scale-tree sub-tree factor) + (* sub-tree factor))) + tree)) + + Many tree operations can be implemented by similar combinations of +sequence operations and recursion. + + *Exercise 2.30:* Define a procedure `square-tree' analogous to the + `square-list' procedure of *Note Exercise 2-21::. That is, + `square-list' should behave as follows: + + (square-tree + (list 1 + (list 2 (list 3 4) 5) + (list 6 7))) + (1 (4 (9 16) 25) (36 49)) + + Define `square-tree' both directly (i.e., without using any + higher-order procedures) and also by using `map' and recursion. + + *Exercise 2.31:* Abstract your answer to *Note Exercise 2-30:: to + produce a procedure `tree-map' with the property that + `square-tree' could be defined as + + (define (square-tree tree) (tree-map square tree)) + + *Exercise 2.32:* We can represent a set as a list of distinct + elements, and we can represent the set of all subsets of the set as + a list of lists. For example, if the set is `(1 2 3)', then the + set of all subsets is `(() (3) (2) (2 3) (1) (1 3) (1 2) (1 2 + 3))'. Complete the following definition of a procedure that + generates the set of subsets of a set and give a clear explanation + of why it works: + + (define (subsets s) + (if (null? s) + (list nil) + (let ((rest (subsets (cdr s)))) + (append rest (map rest))))) + + ---------- Footnotes ---------- + + (1) The order of the first two clauses in the `cond' matters, since +the empty list satisfies `null?' and also is not a pair. + + +File: sicp.info, Node: 2-2-3, Next: 2-2-4, Prev: 2-2-2, Up: 2-2 + +2.2.3 Sequences as Conventional Interfaces +------------------------------------------ + +In working with compound data, we've stressed how data abstraction +permits us to design programs without becoming enmeshed in the details +of data representations, and how abstraction preserves for us the +flexibility to experiment with alternative representations. In this +section, we introduce another powerful design principle for working +with data structures--the use of "conventional interfaces". + + In section *Note 1-3:: we saw how program abstractions, implemented +as higher-order procedures, can capture common patterns in programs +that deal with numerical data. Our ability to formulate analogous +operations for working with compound data depends crucially on the +style in which we manipulate our data structures. Consider, for +example, the following procedure, analogous to the `count-leaves' +procedure of section *Note 2-2-2::, which takes a tree as argument and +computes the sum of the squares of the leaves that are odd: + + (define (sum-odd-squares tree) + (cond ((null? tree) 0) + ((not (pair? tree)) + (if (odd? tree) (square tree) 0)) + (else (+ (sum-odd-squares (car tree)) + (sum-odd-squares (cdr tree)))))) + + On the surface, this procedure is very different from the following +one, which constructs a list of all the even Fibonacci numbers +_Fib_(k), where k is less than or equal to a given integer n: + + (define (even-fibs n) + (define (next k) + (if (> k n) + nil + (let ((f (fib k))) + (if (even? f) + (cons f (next (+ k 1))) + (next (+ k 1)))))) + (next 0)) + + Despite the fact that these two procedures are structurally very +different, a more abstract description of the two computations reveals +a great deal of similarity. The first program + + * enumerates the leaves of a tree; + + * filters them, selecting the odd ones; + + * squares each of the selected ones; and + + * accumulates the results using `+', starting with 0. + + + The second program + + * enumerates the integers from 0 to n; + + * computes the Fibonacci number for each integer; + + * filters them, selecting the even ones; and + + * accumulates the results using `cons', starting with the empty + list. + + + A signal-processing engineer would find it natural to conceptualize +these processes in terms of signals flowing through a cascade of +stages, each of which implements part of the program plan, as shown in +*Note Figure 2-7::. In `sum-odd-squares', we begin with an "enumerator", +which generates a "signal" consisting of the leaves of a given tree. +This signal is passed through a "filter", which eliminates all but the +odd elements. The resulting signal is in turn passed through a "map", +which is a "transducer" that applies the `square' procedure to each +element. The output of the map is then fed to an "accumulator", which +combines the elements using `+', starting from an initial 0. The plan +for `even-fibs' is analogous. + + *Figure 2.7:* The signal-flow plans for the procedures + `sum-odd-squares' (top) and `even-fibs' (bottom) reveal the + commonality between the two programs. + + +-------------+ +-------------+ +-------------+ +-------------+ + | enumerate: |-->| filter: |-->| map: |-->| accumulate: | + | tree leaves | | odd? | | square | | +, 0 | + +-------------+ +-------------+ +-------------+ +-------------+ + + +-------------+ +-------------+ +-------------+ +-------------+ + | enumerate: |-->| map: |-->| filter: |-->| accumulate: | + | integers | | fib | | even? | | cons, () | + +-------------+ +-------------+ +-------------+ +-------------+ + + Unfortunately, the two procedure definitions above fail to exhibit +this signal-flow structure. For instance, if we examine the +`sum-odd-squares' procedure, we find that the enumeration is +implemented partly by the `null?' and `pair?' tests and partly by the +tree-recursive structure of the procedure. Similarly, the accumulation +is found partly in the tests and partly in the addition used in the +recursion. In general, there are no distinct parts of either procedure +that correspond to the elements in the signal-flow description. Our +two procedures decompose the computations in a different way, spreading +the enumeration over the program and mingling it with the map, the +filter, and the accumulation. If we could organize our programs to +make the signal-flow structure manifest in the procedures we write, this +would increase the conceptual clarity of the resulting code. + +Sequence Operations +................... + +The key to organizing programs so as to more clearly reflect the +signal-flow structure is to concentrate on the "signals" that flow from +one stage in the process to the next. If we represent these signals as +lists, then we can use list operations to implement the processing at +each of the stages. For instance, we can implement the mapping stages +of the signal-flow diagrams using the `map' procedure from section +*Note 2-2-1::: + + (map square (list 1 2 3 4 5)) + (1 4 9 16 25) + + Filtering a sequence to select only those elements that satisfy a +given predicate is accomplished by + + (define (filter predicate sequence) + (cond ((null? sequence) nil) + ((predicate (car sequence)) + (cons (car sequence) + (filter predicate (cdr sequence)))) + (else (filter predicate (cdr sequence))))) + + For example, + + (filter odd? (list 1 2 3 4 5)) + (1 3 5) + + Accumulations can be implemented by + + (define (accumulate op initial sequence) + (if (null? sequence) + initial + (op (car sequence) + (accumulate op initial (cdr sequence))))) + + (accumulate + 0 (list 1 2 3 4 5)) + 15 + + (accumulate * 1 (list 1 2 3 4 5)) + 120 + + (accumulate cons nil (list 1 2 3 4 5)) + (1 2 3 4 5) + + All that remains to implement signal-flow diagrams is to enumerate +the sequence of elements to be processed. For `even-fibs', we need to +generate the sequence of integers in a given range, which we can do as +follows: + + (define (enumerate-interval low high) + (if (> low high) + nil + (cons low (enumerate-interval (+ low 1) high)))) + + (enumerate-interval 2 7) + (2 3 4 5 6 7) + + To enumerate the leaves of a tree, we can use(1) + + (define (enumerate-tree tree) + (cond ((null? tree) nil) + ((not (pair? tree)) (list tree)) + (else (append (enumerate-tree (car tree)) + (enumerate-tree (cdr tree)))))) + + (enumerate-tree (list 1 (list 2 (list 3 4)) 5)) + (1 2 3 4 5) + + Now we can reformulate `sum-odd-squares' and `even-fibs' as in the +signal-flow diagrams. For `sum-odd-squares', we enumerate the sequence +of leaves of the tree, filter this to keep only the odd numbers in the +sequence, square each element, and sum the results: + + (define (sum-odd-squares tree) + (accumulate + + 0 + (map square + (filter odd? + (enumerate-tree tree))))) + + For `even-fibs', we enumerate the integers from 0 to n, generate the +Fibonacci number for each of these integers, filter the resulting +sequence to keep only the even elements, and accumulate the results +into a list: + + (define (even-fibs n) + (accumulate cons + nil + (filter even? + (map fib + (enumerate-interval 0 n))))) + + The value of expressing programs as sequence operations is that this +helps us make program designs that are modular, that is, designs that +are constructed by combining relatively independent pieces. We can +encourage modular design by providing a library of standard components +together with a conventional interface for connecting the components in +flexible ways. + + Modular construction is a powerful strategy for controlling +complexity in engineering design. In real signal-processing +applications, for example, designers regularly build systems by +cascading elements selected from standardized families of filters and +transducers. Similarly, sequence operations provide a library of +standard program elements that we can mix and match. For instance, we +can reuse pieces from the `sum-odd-squares' and `even-fibs' procedures +in a program that constructs a list of the squares of the first n + 1 +Fibonacci numbers: + + (define (list-fib-squares n) + (accumulate cons + nil + (map square + (map fib + (enumerate-interval 0 n))))) + + (list-fib-squares 10) + (0 1 1 4 9 25 64 169 441 1156 3025) + + We can rearrange the pieces and use them in computing the product of +the odd integers in a sequence: + + (define (product-of-squares-of-odd-elements sequence) + (accumulate * + 1 + (map square + (filter odd? sequence)))) + + (product-of-squares-of-odd-elements (list 1 2 3 4 5)) + 225 + + We can also formulate conventional data-processing applications in +terms of sequence operations. Suppose we have a sequence of personnel +records and we want to find the salary of the highest-paid programmer. +Assume that we have a selector `salary' that returns the salary of a +record, and a predicate `programmer?' that tests if a record is for a +programmer. Then we can write + + (define (salary-of-highest-paid-programmer records) + (accumulate max + 0 + (map salary + (filter programmer? records)))) + + These examples give just a hint of the vast range of operations that +can be expressed as sequence operations.(2) + + Sequences, implemented here as lists, serve as a conventional +interface that permits us to combine processing modules. Additionally, +when we uniformly represent structures as sequences, we have localized +the data-structure dependencies in our programs to a small number of +sequence operations. By changing these, we can experiment with +alternative representations of sequences, while leaving the overall +design of our programs intact. We will exploit this capability in +section *Note 3-5::, when we generalize the sequence-processing +paradigm to admit infinite sequences. + + *Exercise 2.33:* Fill in the missing expressions to complete the + following definitions of some basic list-manipulation operations + as accumulations: + + (define (map p sequence) + (accumulate (lambda (x y) ) nil sequence)) + + (define (append seq1 seq2) + (accumulate cons )) + + (define (length sequence) + (accumulate 0 sequence)) + + *Exercise 2.34:* Evaluating a polynomial in x at a given value of + x can be formulated as an accumulation. We evaluate the polynomial + + a_n r^n | a_(n-1) r^(n-1) + ... + a_1 r + a_0 + + using a well-known algorithm called "Horner's rule", which + structures the computation as + + (... (a_n r + a_(n-1)) r + ... + a_1) r + a_0 + + In other words, we start with a_n, multiply by x, add a_(n-1), + multiply by x, and so on, until we reach a_0.(3) + + Fill in the following template to produce a procedure that + evaluates a polynomial using Horner's rule. Assume that the + coefficients of the polynomial are arranged in a sequence, from + a_0 through a_n. + + (define (horner-eval x coefficient-sequence) + (accumulate (lambda (this-coeff higher-terms) ) + 0 + coefficient-sequence)) + + For example, to compute 1 + 3x + 5x^3 + x^(5) at x = 2 you would + evaluate + + (horner-eval 2 (list 1 3 0 5 0 1)) + + *Exercise 2.35:* Redefine `count-leaves' from section *Note + 2-2-2:: as an accumulation: + + (define (count-leaves t) + (accumulate (map ))) + + *Exercise 2.36:* The procedure `accumulate-n' is similar to + `accumulate' except that it takes as its third argument a sequence + of sequences, which are all assumed to have the same number of + elements. It applies the designated accumulation procedure to + combine all the first elements of the sequences, all the second + elements of the sequences, and so on, and returns a sequence of + the results. For instance, if `s' is a sequence containing four + sequences, `((1 2 3) (4 5 6) (7 8 9) (10 11 12)),' then the value + of `(accumulate-n + 0 s)' should be the sequence `(22 26 30)'. + Fill in the missing expressions in the following definition of + `accumulate-n': + + (define (accumulate-n op init seqs) + (if (null? (car seqs)) + nil + (cons (accumulate op init ) + (accumulate-n op init )))) + +Exercise 2.37 +............. + +Suppose we represent vectors v = (v_i) as sequences of numbers, and +matrices m = (m_(ij)) as sequences of vectors (the rows of the matrix). +For example, the matrix + + +- -+ + | 1 2 3 4 | + | 4 5 6 6 | + | 6 7 8 9 | + +- -+ + +is represented as the sequence `((1 2 3 4) (4 5 6 6) (6 7 8 9))'. With +this representation, we can use sequence operations to concisely +express the basic matrix and vector operations. These operations +(which are described in any book on matrix algebra) are the following: + + __ + (dot-product v w) returns the sum >_i v_i w_i + + (matrix-*-vector m v) returns the vector t, + __ + where t_i = >_j m_(ij) v_j + + (matrix-*-matrix m n) returns the matrix p, + __ + where p_(ij) = >_k m_(ik) n_(kj) + + (transpose m) returns the matrix n, + where n_(ij) = m_(ji) + + We can define the dot product as(4) + + (define (dot-product v w) + (accumulate + 0 (map * v w))) + + Fill in the missing expressions in the following procedures for +computing the other matrix operations. (The procedure `accumulate-n' +is defined in *Note Exercise 2-36::.) + + (define (matrix-*-vector m v) + (map m)) + + (define (transpose mat) + (accumulate-n mat)) + + (define (matrix-*-matrix m n) + (let ((cols (transpose n))) + (map m))) + + *Exercise 2.38:* The `accumulate' procedure is also known as + `fold-right', because it combines the first element of the + sequence with the result of combining all the elements to the + right. There is also a `fold-left', which is similar to + `fold-right', except that it combines elements working in the + opposite direction: + + (define (fold-left op initial sequence) + (define (iter result rest) + (if (null? rest) + result + (iter (op result (car rest)) + (cdr rest)))) + (iter initial sequence)) + + What are the values of + + (fold-right / 1 (list 1 2 3)) + + (fold-left / 1 (list 1 2 3)) + + (fold-right list nil (list 1 2 3)) + + (fold-left list nil (list 1 2 3)) + + Give a property that `op' should satisfy to guarantee that + `fold-right' and `fold-left' will produce the same values for any + sequence. + + *Exercise 2.39:* Complete the following definitions of `reverse' + (*Note Exercise 2-18::) in terms of `fold-right' and `fold-left' + from *Note Exercise 2-38::: + + (define (reverse sequence) + (fold-right (lambda (x y) ) nil sequence)) + + (define (reverse sequence) + (fold-left (lambda (x y) ) nil sequence)) + +Nested Mappings +............... + +We can extend the sequence paradigm to include many computations that +are commonly expressed using nested loops.(5) Consider this problem: +Given a positive integer n, find all ordered pairs of distinct positive +integers i and j, where 1 <= j< i<= n, such that i + j is prime. For +example, if n is 6, then the pairs are the following: + + i | 2 3 4 4 5 6 6 + j | 1 2 1 3 2 1 5 + ------+--------------- + i + j | 3 5 5 7 7 7 11 + + A natural way to organize this computation is to generate the +sequence of all ordered pairs of positive integers less than or equal +to n, filter to select those pairs whose sum is prime, and then, for +each pair (i, j) that passes through the filter, produce the triple +(i,j,i + j). + + Here is a way to generate the sequence of pairs: For each integer i +<= n, enumerate the integers jpainter segment-list) + (lambda (frame) + (for-each + (lambda (segment) + (draw-line + ((frame-coord-map frame) (start-segment segment)) + ((frame-coord-map frame) (end-segment segment)))) + segment-list))) + + The segments are given using coordinates with respect to the unit +square. For each segment in the list, the painter transforms the +segment endpoints with the frame coordinate map and draws a line +between the transformed points. + + Representing painters as procedures erects a powerful abstraction +barrier in the picture language. We can create and intermix all sorts +of primitive painters, based on a variety of graphics capabilities. The +details of their implementation do not matter. Any procedure can serve +as a painter, provided that it takes a frame as argument and draws +something scaled to fit the frame.(7) + + *Exercise 2.48:* A directed line segment in the plane can be + represented as a pair of vectors--the vector running from the + origin to the start-point of the segment, and the vector running + from the origin to the end-point of the segment. Use your vector + representation from *Note Exercise 2-46:: to define a + representation for segments with a constructor `make-segment' and + selectors `start-segment' and `end-segment'. + + *Exercise 2.49:* Use `segments->painter' to define the following + primitive painters: + + a. The painter that draws the outline of the designated frame. + + b. The painter that draws an "X" by connecting opposite corners + of the frame. + + c. The painter that draws a diamond shape by connecting the + midpoints of the sides of the frame. + + d. The `wave' painter. + + +Transforming and combining painters +................................... + +An operation on painters (such as `flip-vert' or `beside') works by +creating a painter that invokes the original painters with respect to +frames derived from the argument frame. Thus, for example, `flip-vert' +doesn't have to know how a painter works in order to flip it--it just +has to know how to turn a frame upside down: The flipped painter just +uses the original painter, but in the inverted frame. + + Painter operations are based on the procedure `transform-painter', +which takes as arguments a painter and information on how to transform +a frame and produces a new painter. The transformed painter, when +called on a frame, transforms the frame and calls the original painter +on the transformed frame. The arguments to `transform-painter' are +points (represented as vectors) that specify the corners of the new +frame: When mapped into the frame, the first point specifies the new +frame's origin and the other two specify the ends of its edge vectors. +Thus, arguments within the unit square specify a frame contained within +the original frame. + + (define (transform-painter painter origin corner1 corner2) + (lambda (frame) + (let ((m (frame-coord-map frame))) + (let ((new-origin (m origin))) + (painter + (make-frame new-origin + (sub-vect (m corner1) new-origin) + (sub-vect (m corner2) new-origin))))))) + + Here's how to flip painter images vertically: + + (define (flip-vert painter) + (transform-painter painter + (make-vect 0.0 1.0) ; new `origin' + (make-vect 1.0 1.0) ; new end of `edge1' + (make-vect 0.0 0.0))) ; new end of `edge2' + + Using `transform-painter', we can easily define new transformations. +For example, we can define a painter that shrinks its image to the +upper-right quarter of the frame it is given: + + (define (shrink-to-upper-right painter) + (transform-painter painter + (make-vect 0.5 0.5) + (make-vect 1.0 0.5) + (make-vect 0.5 1.0))) + + Other transformations rotate images counterclockwise by 90 degrees(8) + + (define (rotate90 painter) + (transform-painter painter + (make-vect 1.0 0.0) + (make-vect 1.0 1.0) + (make-vect 0.0 0.0))) + +or squash images towards the center of the frame:(9) + + (define (squash-inwards painter) + (transform-painter painter + (make-vect 0.0 0.0) + (make-vect 0.65 0.35) + (make-vect 0.35 0.65))) + + Frame transformation is also the key to defining means of combining +two or more painters. The `beside' procedure, for example, takes two +painters, transforms them to paint in the left and right halves of an +argument frame respectively, and produces a new, compound painter. +When the compound painter is given a frame, it calls the first +transformed painter to paint in the left half of the frame and calls +the second transformed painter to paint in the right half of the frame: + + (define (beside painter1 painter2) + (let ((split-point (make-vect 0.5 0.0))) + (let ((paint-left + (transform-painter painter1 + (make-vect 0.0 0.0) + split-point + (make-vect 0.0 1.0))) + (paint-right + (transform-painter painter2 + split-point + (make-vect 1.0 0.0) + (make-vect 0.5 1.0)))) + (lambda (frame) + (paint-left frame) + (paint-right frame))))) + + Observe how the painter data abstraction, and in particular the +representation of painters as procedures, makes `beside' easy to +implement. The `beside' procedure need not know anything about the +details of the component painters other than that each painter will +draw something in its designated frame. + + *Exercise 2.50:* Define the transformation `flip-horiz', which + flips painters horizontally, and transformations that rotate + painters counterclockwise by 180 degrees and 270 degrees. + + *Exercise 2.51:* Define the `below' operation for painters. + `Below' takes two painters as arguments. The resulting painter, + given a frame, draws with the first painter in the bottom of the + frame and with the second painter in the top. Define `below' in + two different ways--first by writing a procedure that is analogous + to the `beside' procedure given above, and again in terms of + `beside' and suitable rotation operations (from *Note Exercise + 2-50::). + +Levels of language for robust design +.................................... + +The picture language exercises some of the critical ideas we've +introduced about abstraction with procedures and data. The fundamental +data abstractions, painters, are implemented using procedural +representations, which enables the language to handle different basic +drawing capabilities in a uniform way. The means of combination +satisfy the closure property, which permits us to easily build up +complex designs. Finally, all the tools for abstracting procedures are +available to us for abstracting means of combination for painters. + + We have also obtained a glimpse of another crucial idea about +languages and program design. This is the approach of "stratified +design", the notion that a complex system should be structured as a +sequence of levels that are described using a sequence of languages. +Each level is constructed by combining parts that are regarded as +primitive at that level, and the parts constructed at each level are +used as primitives at the next level. The language used at each level +of a stratified design has primitives, means of combination, and means +of abstraction appropriate to that level of detail. + + Stratified design pervades the engineering of complex systems. For +example, in computer engineering, resistors and transistors are +combined (and described using a language of analog circuits) to produce +parts such as and-gates and or-gates, which form the primitives of a +language for digital-circuit design.(10) These parts are combined to +build processors, bus structures, and memory systems, which are in turn +combined to form computers, using languages appropriate to computer +architecture. Computers are combined to form distributed systems, using +languages appropriate for describing network interconnections, and so +on. + + As a tiny example of stratification, our picture language uses +primitive elements (primitive painters) that are created using a +language that specifies points and lines to provide the lists of line +segments for `segments->painter', or the shading details for a painter +like `rogers'. The bulk of our description of the picture language +focused on combining these primitives, using geometric combiners such +as `beside' and `below'. We also worked at a higher level, regarding +`beside' and `below' as primitives to be manipulated in a language +whose operations, such as `square-of-four', capture common patterns of +combining geometric combiners. + + Stratified design helps make programs "robust", that is, it makes it +likely that small changes in a specification will require +correspondingly small changes in the program. For instance, suppose we +wanted to change the image based on `wave' shown in *Note Figure 2-9::. +We could work at the lowest level to change the detailed appearance of +the `wave' element; we could work at the middle level to change the way +`corner-split' replicates the `wave'; we could work at the highest +level to change how `square-limit' arranges the four copies of the +corner. In general, each level of a stratified design provides a +different vocabulary for expressing the characteristics of the system, +and a different kind of ability to change it. + + *Exercise 2.52:* Make changes to the square limit of `wave' shown + in *Note Figure 2-9:: by working at each of the levels described + above. In particular: + + a. Add some segments to the primitive `wave' painter of *Note + Exercise 2-49:: (to add a smile, for example). + + b. Change the pattern constructed by `corner-split' (for + example, by using only one copy of the `up-split' and + `right-split' images instead of two). + + c. Modify the version of `square-limit' that uses + `square-of-four' so as to assemble the corners in a different + pattern. (For example, you might make the big Mr. Rogers + look outward from each corner of the square.) + + + ---------- Footnotes ---------- + + (1) The picture language is based on the language Peter Henderson +created to construct images like M.C. Escher's "Square Limit" woodcut +(see Henderson 1982). The woodcut incorporates a repeated scaled +pattern, similar to the arrangements drawn using the `square-limit' +procedure in this section. + + (2) William Barton Rogers (1804-1882) was the founder and first +president of MIT. A geologist and talented teacher, he taught at +William and Mary College and at the University of Virginia. In 1859 he +moved to Boston, where he had more time for research, worked on a plan +for establishing a "polytechnic institute," and served as +Massachusetts's first State Inspector of Gas Meters. + + When MIT was established in 1861, Rogers was elected its first +president. Rogers espoused an ideal of "useful learning" that was +different from the university education of the time, with its +overemphasis on the classics, which, as he wrote, "stand in the way of +the broader, higher and more practical instruction and discipline of +the natural and social sciences." This education was likewise to be +different from narrow trade-school education. In Rogers's words: + + The world-enforced distinction between the practical and the + scientific worker is utterly futile, and the whole experience of + modern times has demonstrated its utter worthlessness. + + Rogers served as president of MIT until 1870, when he resigned due to +ill health. In 1878 the second president of MIT, John Runkle, resigned +under the pressure of a financial crisis brought on by the Panic of +1873 and strain of fighting off attempts by Harvard to take over MIT. +Rogers returned to hold the office of president until 1881. + + Rogers collapsed and died while addressing MIT's graduating class at +the commencement exercises of 1882. Runkle quoted Rogers's last words +in a memorial address delivered that same year: + + "As I stand here today and see what the Institute is, ... I call + to mind the beginnings of science. I remember one hundred and + fifty years ago Stephen Hales published a pamphlet on the subject + of illuminating gas, in which he stated that his researches had + demonstrated that 128 grains of bituminous coal - " "Bituminous + coal," these were his last words on earth. Here he bent forward, + as if consulting some notes on the table before him, then slowly + regaining an erect position, threw up his hands, and was + translated from the scene of his earthly labors and triumphs to + "the tomorrow of death," where the mysteries of life are solved, + and the disembodied spirit finds unending satisfaction in + contemplating the new and still unfathomable mysteries of the + infinite future. + + In the words of Francis A. Walker (MIT's third president): + + All his life he had borne himself most faithfully and heroically, + and he died as so good a knight would surely have wished, in + harness, at his post, and in the very part and act of public duty. + + (3) Equivalently, we could write + + (define flipped-pairs + (square-of-four identity flip-vert identity flip-vert)) + + (4) `Rotate180' rotates a painter by 180 degrees (see *Note Exercise +2-50::). Instead of `rotate180' we could say `(compose flip-vert +flip-horiz)', using the `compose' procedure from *Note Exercise 1-42::. + + (5) `Frame-coord-map' uses the vector operations described in *Note +Exercise 2-46:: below, which we assume have been implemented using some +representation for vectors. Because of data abstraction, it doesn't +matter what this vector representation is, so long as the vector +operations behave correctly. + + (6) `Segments->painter' uses the representation for line segments +described in *Note Exercise 2-48:: below. It also uses the `for-each' +procedure described in *Note Exercise 2-23::. + + (7) For example, the `rogers' painter of *Note Figure 2-11:: was +constructed from a gray-level image. For each point in a given frame, +the `rogers' painter determines the point in the image that is mapped +to it under the frame coordinate map, and shades it accordingly. By +allowing different types of painters, we are capitalizing on the +abstract data idea discussed in section *Note 2-1-3::, where we argued +that a rational-number representation could be anything at all that +satisfies an appropriate condition. Here we're using the fact that a +painter can be implemented in any way at all, so long as it draws +something in the designated frame. Section *Note 2-1-3:: also showed +how pairs could be implemented as procedures. Painters are our second +example of a procedural representation for data. + + (8) `Rotate90' is a pure rotation only for square frames, because it +also stretches and shrinks the image to fit into the rotated frame. + + (9) The diamond-shaped images in figures *Note Figure 2-10:: and +*Note Figure 2-11:: were created with `squash-inwards' applied to +`wave' and `rogers'. + + (10) Section *Note 3-3-4:: describes one such language. + + +File: sicp.info, Node: 2-3, Next: 2-4, Prev: 2-2, Up: Chapter 2 + +2.3 Symbolic Data +================= + +All the compound data objects we have used so far were constructed +ultimately from numbers. In this section we extend the +representational capability of our language by introducing the ability +to work with arbitrary symbols as data. + +* Menu: + +* 2-3-1:: Quotation +* 2-3-2:: Example: Symbolic Differentiation +* 2-3-3:: Example: Representing Sets +* 2-3-4:: Example: Huffman Encoding Trees + + +File: sicp.info, Node: 2-3-1, Next: 2-3-2, Prev: 2-3, Up: 2-3 + +2.3.1 Quotation +--------------- + +If we can form compound data using symbols, we can have lists such as + + (a b c d) + (23 45 17) + ((Norah 12) (Molly 9) (Anna 7) (Lauren 6) (Charlotte 4)) + + Lists containing symbols can look just like the expressions of our +language: + + (* (+ 23 45) (+ x 9)) + + (define (fact n) (if (= n 1) 1 (* n (fact (- n 1))))) + + In order to manipulate symbols we need a new element in our +language: the ability to "quote" a data object. Suppose we want to +construct the list `(a b)'. We can't accomplish this with `(list a +b)', because this expression constructs a list of the "values" of `a' +and `b' rather than the symbols themselves. This issue is well known +in the context of natural languages, where words and sentences may be +regarded either as semantic entities or as character strings (syntactic +entities). The common practice in natural languages is to use +quotation marks to indicate that a word or a sentence is to be treated +literally as a string of characters. For instance, the first letter of +"John" is clearly "J." If we tell somebody "say your name aloud," we +expect to hear that person's name. However, if we tell somebody "say +`your name' aloud," we expect to hear the words "your name." Note that +we are forced to nest quotation marks to describe what somebody else +might say.(1) + + We can follow this same practice to identify lists and symbols that +are to be treated as data objects rather than as expressions to be +evaluated. However, our format for quoting differs from that of +natural languages in that we place a quotation mark (traditionally, the +single quote symbol `'') only at the beginning of the object to be +quoted. We can get away with this in Scheme syntax because we rely on +blanks and parentheses to delimit objects. Thus, the meaning of the +single quote character is to quote the next object.(2) + + Now we can distinguish between symbols and their values: + + (define a 1) + + (define b 2) + + (list a b) + (1 2) + + (list 'a 'b) + (a b) + + (list 'a b) + (a 2) + + Quotation also allows us to type in compound objects, using the +conventional printed representation for lists:(3) + + (car '(a b c)) + a + + (cdr '(a b c)) + (b c) + + In keeping with this, we can obtain the empty list by evaluating +`'()', and thus dispense with the variable `nil'. + + One additional primitive used in manipulating symbols is `eq?', which +takes two symbols as arguments and tests whether they are the same.(4) +Using `eq?', we can implement a useful procedure called `memq'. This +takes two arguments, a symbol and a list. If the symbol is not +contained in the list (i.e., is not `eq?' to any item in the list), +then `memq' returns false. Otherwise, it returns the sublist of the +list beginning with the first occurrence of the symbol: + + (define (memq item x) + (cond ((null? x) false) + ((eq? item (car x)) x) + (else (memq item (cdr x))))) + + For example, the value of + + (memq 'apple '(pear banana prune)) + +is false, whereas the value of + + (memq 'apple '(x (apple sauce) y apple pear)) + +is `(apple pear)'. + + *Exercise 2.53:* What would the interpreter print in response to + evaluating each of the following expressions? + + (list 'a 'b 'c) + + (list (list 'george)) + + (cdr '((x1 x2) (y1 y2))) + + (cadr '((x1 x2) (y1 y2))) + + (pair? (car '(a short list))) + + (memq 'red '((red shoes) (blue socks))) + + (memq 'red '(red shoes blue socks)) + + *Exercise 2.54:* Two lists are said to be `equal?' if they contain + equal elements arranged in the same order. For example, + + (equal? '(this is a list) '(this is a list)) + + is true, but + + (equal? '(this is a list) '(this (is a) list)) + + is false. To be more precise, we can define `equal?' recursively + in terms of the basic `eq?' equality of symbols by saying that `a' + and `b' are `equal?' if they are both symbols and the symbols are + `eq?', or if they are both lists such that `(car a)' is `equal?' + to `(car b)' and `(cdr a)' is `equal?' to `(cdr b)'. Using this + idea, implement `equal?' as a procedure.(5) + + *Exercise 2.55:* Eva Lu Ator types to the interpreter the + expression + + (car ''abracadabra) + + To her surprise, the interpreter prints back `quote'. Explain. + + ---------- Footnotes ---------- + + (1) Allowing quotation in a language wreaks havoc with the ability +to reason about the language in simple terms, because it destroys the +notion that equals can be substituted for equals. For example, three +is one plus two, but the word "three" is not the phrase "one plus two." +Quotation is powerful because it gives us a way to build expressions +that manipulate other expressions (as we will see when we write an +interpreter in *Note Chapter 4::). But allowing statements in a +language that talk about other statements in that language makes it +very difficult to maintain any coherent principle of what "equals can +be substituted for equals" should mean. For example, if we know that +the evening star is the morning star, then from the statement "the +evening star is Venus" we can deduce "the morning star is Venus." +However, given that "John knows that the evening star is Venus" we +cannot infer that "John knows that the morning star is Venus." + + (2) The single quote is different from the double quote we have been +using to enclose character strings to be printed. Whereas the single +quote can be used to denote lists or symbols, the double quote is used +only with character strings. In this book, the only use for character +strings is as items to be printed. + + (3) Strictly, our use of the quotation mark violates the general +rule that all compound expressions in our language should be delimited +by parentheses and look like lists. We can recover this consistency by +introducing a special form `quote', which serves the same purpose as +the quotation mark. Thus, we would type `(quote a)' instead of `'a', +and we would type `(quote (a b c))' instead of `'(a b c)'. This is +precisely how the interpreter works. The quotation mark is just a +single-character abbreviation for wrapping the next complete expression +with `quote' to form `(quote )'. This is important because +it maintains the principle that any expression seen by the interpreter +can be manipulated as a data object. For instance, we could construct +the expression `(car '(a b c))', which is the same as `(car (quote (a b +c)))', by evaluating `(list 'car (list 'quote '(a b c)))'. + + (4) We can consider two symbols to be "the same" if they consist of +the same characters in the same order. Such a definition skirts a deep +issue that we are not yet ready to address: the meaning of "sameness" +in a programming language. We will return to this in *Note Chapter 3:: +(section *Note 3-1-3::). + + (5) In practice, programmers use `equal?' to compare lists that +contain numbers as well as symbols. Numbers are not considered to be +symbols. The question of whether two numerically equal numbers (as +tested by `=') are also `eq?' is highly implementation-dependent. A +better definition of `equal?' (such as the one that comes as a +primitive in Scheme) would also stipulate that if `a' and `b' are both +numbers, then `a' and `b' are `equal?' if they are numerically equal. + + +File: sicp.info, Node: 2-3-2, Next: 2-3-3, Prev: 2-3-1, Up: 2-3 + +2.3.2 Example: Symbolic Differentiation +--------------------------------------- + +As an illustration of symbol manipulation and a further illustration of +data abstraction, consider the design of a procedure that performs +symbolic differentiation of algebraic expressions. We would like the +procedure to take as arguments an algebraic expression and a variable +and to return the derivative of the expression with respect to the +variable. For example, if the arguments to the procedure are ax^2 + bx ++ c and x, the procedure should return 2ax + b. Symbolic +differentiation is of special historical significance in Lisp. It was +one of the motivating examples behind the development of a computer +language for symbol manipulation. Furthermore, it marked the beginning +of the line of research that led to the development of powerful systems +for symbolic mathematical work, which are currently being used by a +growing number of applied mathematicians and physicists. + + In developing the symbolic-differentiation program, we will follow +the same strategy of data abstraction that we followed in developing +the rational-number system of section *Note 2-1-1::. That is, we will +first define a differentiation algorithm that operates on abstract +objects such as "sums," "products," and "variables" without worrying +about how these are to be represented. Only afterward will we address +the representation problem. + +The differentiation program with abstract data +.............................................. + +In order to keep things simple, we will consider a very simple +symbolic-differentiation program that handles expressions that are +built up using only the operations of addition and multiplication with +two arguments. Differentiation of any such expression can be carried +out by applying the following reduction rules: + + dc + -- = 0 for c a constant, or a variable different from x + dx + + dx + -- = 1 + dx + + d(u + v) du dv + -------- = -- + -- + dx dx dx + + d(uv) / dv \ / du \ + ----- = u | -- | + v | -- | + dx \ dx / \ dx / + + Observe that the latter two rules are recursive in nature. That is, +to obtain the derivative of a sum we first find the derivatives of the +terms and add them. Each of the terms may in turn be an expression +that needs to be decomposed. Decomposing into smaller and smaller +pieces will eventually produce pieces that are either constants or +variables, whose derivatives will be either 0 or 1. + + To embody these rules in a procedure we indulge in a little wishful +thinking, as we did in designing the rational-number implementation. +If we had a means for representing algebraic expressions, we should be +able to tell whether an expression is a sum, a product, a constant, or +a variable. We should be able to extract the parts of an expression. +For a sum, for example we want to be able to extract the addend (first +term) and the augend (second term). We should also be able to +construct expressions from parts. Let us assume that we already have +procedures to implement the following selectors, constructors, and +predicates: + + (variable? e) Is `e' a variable? + (same-variable? v1 v2) Are `v1' and `v2' the same variable? + (sum? e) Is `e' a sum? + (addend e) Addend of the sum `e'. + (augend e) Augend of the sum `e'. + (make-sum a1 a2) Construct the sum of `a1' and `a2'. + (product? e) Is `e' a product? + (multiplier e) Multiplier of the product `e'. + (multiplicand e) Multiplicand of the product `e'. + (make-product m1 m2) Construct the product of `m1' and `m2'. + + Using these, and the primitive predicate `number?', which identifies +numbers, we can express the differentiation rules as the following +procedure: + + (define (deriv exp var) + (cond ((number? exp) 0) + ((variable? exp) + (if (same-variable? exp var) 1 0)) + ((sum? exp) + (make-sum (deriv (addend exp) var) + (deriv (augend exp) var))) + ((product? exp) + (make-sum + (make-product (multiplier exp) + (deriv (multiplicand exp) var)) + (make-product (deriv (multiplier exp) var) + (multiplicand exp)))) + (else + (error "unknown expression type -- DERIV" exp)))) + + This `deriv' procedure incorporates the complete differentiation +algorithm. Since it is expressed in terms of abstract data, it will +work no matter how we choose to represent algebraic expressions, as +long as we design a proper set of selectors and constructors. This is +the issue we must address next. + +Representing algebraic expressions +.................................. + +We can imagine many ways to use list structure to represent algebraic +expressions. For example, we could use lists of symbols that mirror +the usual algebraic notation, representing ax + b as the list `(a * x + +b)' . However, one especially straightforward choice is to use the same +parenthesized prefix notation that Lisp uses for combinations; that is, +to represent ax + b as `(+ (* a x) b)'. Then our data representation +for the differentiation problem is as follows: + + * The variables are symbols. They are identified by the primitive + predicate `symbol?': + + (define (variable? x) (symbol? x)) + + * Two variables are the same if the symbols representing them are + `eq?': + + (define (same-variable? v1 v2) + (and (variable? v1) (variable? v2) (eq? v1 v2))) + + * Sums and products are constructed as lists: + + (define (make-sum a1 a2) (list '+ a1 a2)) + + (define (make-product m1 m2) (list '* m1 m2)) + + * A sum is a list whose first element is the symbol `+': + + (define (sum? x) + (and (pair? x) (eq? (car x) '+))) + + * The addend is the second item of the sum list: + + (define (addend s) (cadr s)) + + * The augend is the third item of the sum list: + + (define (augend s) (caddr s)) + + * A product is a list whose first element is the symbol `*': + + (define (product? x) + (and (pair? x) (eq? (car x) '*))) + + * The multiplier is the second item of the product list: + + (define (multiplier p) (cadr p)) + + * The multiplicand is the third item of the product list: + + (define (multiplicand p) (caddr p)) + + + Thus, we need only combine these with the algorithm as embodied by +`deriv' in order to have a working symbolic-differentiation program. +Let us look at some examples of its behavior: + + (deriv '(+ x 3) 'x) + (+ 1 0) + + (deriv '(* x y) 'x) + (+ (* x 0) (* 1 y)) + + (deriv '(* (* x y) (+ x 3)) 'x) + (+ (* (* x y) (+ 1 0)) + (* (+ (* x 0) (* 1 y)) + (+ x 3))) + + The program produces answers that are correct; however, they are +unsimplified. It is true that + + d(xy) + ----- = x * 0 + 1 * y + dx + +but we would like the program to know that x * 0 = 0, 1 * y = y, and 0 ++ y = y. The answer for the second example should have been simply +`y'. As the third example shows, this becomes a serious issue when the +expressions are complex. + + Our difficulty is much like the one we encountered with the +rational-number implementation: we haven't reduced answers to simplest +form. To accomplish the rational-number reduction, we needed to change +only the constructors and the selectors of the implementation. We can +adopt a similar strategy here. We won't change `deriv' at all. +Instead, we will change `make-sum' so that if both summands are +numbers, `make-sum' will add them and return their sum. Also, if one +of the summands is 0, then `make-sum' will return the other summand. + + (define (make-sum a1 a2) + (cond ((=number? a1 0) a2) + ((=number? a2 0) a1) + ((and (number? a1) (number? a2)) (+ a1 a2)) + (else (list '+ a1 a2)))) + + This uses the procedure `=number?', which checks whether an +expression is equal to a given number: + + (define (=number? exp num) + (and (number? exp) (= exp num))) + + Similarly, we will change `make-product' to build in the rules that 0 +times anything is 0 and 1 times anything is the thing itself: + + (define (make-product m1 m2) + (cond ((or (=number? m1 0) (=number? m2 0)) 0) + ((=number? m1 1) m2) + ((=number? m2 1) m1) + ((and (number? m1) (number? m2)) (* m1 m2)) + (else (list '* m1 m2)))) + + Here is how this version works on our three examples: + + (deriv '(+ x 3) 'x) + 1 + + (deriv '(* x y) 'x) + y + + (deriv '(* (* x y) (+ x 3)) 'x) + (+ (* x y) (* y (+ x 3))) + + Although this is quite an improvement, the third example shows that +there is still a long way to go before we get a program that puts +expressions into a form that we might agree is "simplest." The problem +of algebraic simplification is complex because, among other reasons, a +form that may be simplest for one purpose may not be for another. + + *Exercise 2.56:* Show how to extend the basic differentiator to + handle more kinds of expressions. For instance, implement the + differentiation rule + + n_1 n_2 + --- = --- if and only if n_1 d_2 = n_2 d_1 + d_1 d_2 + + by adding a new clause to the `deriv' program and defining + appropriate procedures `exponentiation?', `base', `exponent', and + `make-exponentiation'. (You may use the symbol `**' to denote + exponentiation.) Build in the rules that anything raised to the + power 0 is 1 and anything raised to the power 1 is the thing + itself. + + *Exercise 2.57:* Extend the differentiation program to handle sums + and products of arbitrary numbers of (two or more) terms. Then + the last example above could be expressed as + + (deriv '(* x y (+ x 3)) 'x) + + Try to do this by changing only the representation for sums and + products, without changing the `deriv' procedure at all. For + example, the `addend' of a sum would be the first term, and the + `augend' would be the sum of the rest of the terms. + + *Exercise 2.58:* Suppose we want to modify the differentiation + program so that it works with ordinary mathematical notation, in + which `+' and `*' are infix rather than prefix operators. Since + the differentiation program is defined in terms of abstract data, + we can modify it to work with different representations of + expressions solely by changing the predicates, selectors, and + constructors that define the representation of the algebraic + expressions on which the differentiator is to operate. + + a. Show how to do this in order to differentiate algebraic + expressions presented in infix form, such as `(x + (3 * (x + + (y + 2))))'. To simplify the task, assume that `+' and `*' + always take two arguments and that expressions are fully + parenthesized. + + b. The problem becomes substantially harder if we allow standard + algebraic notation, such as `(x + 3 * (x + y + 2))', which + drops unnecessary parentheses and assumes that multiplication + is done before addition. Can you design appropriate + predicates, selectors, and constructors for this notation + such that our derivative program still works? + + + +File: sicp.info, Node: 2-3-3, Next: 2-3-4, Prev: 2-3-2, Up: 2-3 + +2.3.3 Example: Representing Sets +-------------------------------- + +In the previous examples we built representations for two kinds of +compound data objects: rational numbers and algebraic expressions. In +one of these examples we had the choice of simplifying (reducing) the +expressions at either construction time or selection time, but other +than that the choice of a representation for these structures in terms +of lists was straightforward. When we turn to the representation of +sets, the choice of a representation is not so obvious. Indeed, there +are a number of possible representations, and they differ significantly +from one another in several ways. + + Informally, a set is simply a collection of distinct objects. To +give a more precise definition we can employ the method of data +abstraction. That is, we define "set" by specifying the operations +that are to be used on sets. These are `union-set', +`intersection-set', `element-of-set?', and `adjoin-set'. +`Element-of-set?' is a predicate that determines whether a given +element is a member of a set. `Adjoin-set' takes an object and a set +as arguments and returns a set that contains the elements of the +original set and also the adjoined element. `Union-set' computes the +union of two sets, which is the set containing each element that +appears in either argument. `Intersection-set' computes the +intersection of two sets, which is the set containing only elements +that appear in both arguments. From the viewpoint of data abstraction, +we are free to design any representation that implements these +operations in a way consistent with the interpretations given above.(1) + +Sets as unordered lists +....................... + +One way to represent a set is as a list of its elements in which no +element appears more than once. The empty set is represented by the +empty list. In this representation, `element-of-set?' is similar to +the procedure `memq' of section *Note 2-3-1::. It uses `equal?' +instead of `eq?' so that the set elements need not be symbols: + + (define (element-of-set? x set) + (cond ((null? set) false) + ((equal? x (car set)) true) + (else (element-of-set? x (cdr set))))) + + Using this, we can write `adjoin-set'. If the object to be adjoined +is already in the set, we just return the set. Otherwise, we use +`cons' to add the object to the list that represents the set: + + (define (adjoin-set x set) + (if (element-of-set? x set) + set + (cons x set))) + + For `intersection-set' we can use a recursive strategy. If we know +how to form the intersection of `set2' and the `cdr' of `set1', we only +need to decide whether to include the `car' of `set1' in this. But +this depends on whether `(car set1)' is also in `set2'. Here is the +resulting procedure: + + (define (intersection-set set1 set2) + (cond ((or (null? set1) (null? set2)) '()) + ((element-of-set? (car set1) set2) + (cons (car set1) + (intersection-set (cdr set1) set2))) + (else (intersection-set (cdr set1) set2)))) + + In designing a representation, one of the issues we should be +concerned with is efficiency. Consider the number of steps required by +our set operations. Since they all use `element-of-set?', the speed of +this operation has a major impact on the efficiency of the set +implementation as a whole. Now, in order to check whether an object is +a member of a set, `element-of-set?' may have to scan the entire set. +(In the worst case, the object turns out not to be in the set.) Hence, +if the set has n elements, `element-of-set?' might take up to n steps. +Thus, the number of steps required grows as [theta](n). The number of +steps required by `adjoin-set', which uses this operation, also grows +as [theta](n). For `intersection-set', which does an `element-of-set?' +check for each element of `set1', the number of steps required grows as +the product of the sizes of the sets involved, or [theta](n^2) for two +sets of size n. The same will be true of `union-set'. + + *Exercise 2.59:* Implement the `union-set' operation for the + unordered-list representation of sets. + + *Exercise 2.60:* We specified that a set would be represented as a + list with no duplicates. Now suppose we allow duplicates. For + instance, the set {1,2,3} could be represented as the list `(2 3 2 + 1 3 2 2)'. Design procedures `element-of-set?', `adjoin-set', + `union-set', and `intersection-set' that operate on this + representation. How does the efficiency of each compare with the + corresponding procedure for the non-duplicate representation? Are + there applications for which you would use this representation in + preference to the non-duplicate one? + +Sets as ordered lists +..................... + +One way to speed up our set operations is to change the representation +so that the set elements are listed in increasing order. To do this, +we need some way to compare two objects so that we can say which is +bigger. For example, we could compare symbols lexicographically, or we +could agree on some method for assigning a unique number to an object +and then compare the elements by comparing the corresponding numbers. +To keep our discussion simple, we will consider only the case where the +set elements are numbers, so that we can compare elements using `>' and +`<'. We will represent a set of numbers by listing its elements in +increasing order. Whereas our first representation above allowed us to +represent the set {1,3,6,10} by listing the elements in any order, our +new representation allows only the list `(1 3 6 10)'. + + One advantage of ordering shows up in `element-of-set?': In checking +for the presence of an item, we no longer have to scan the entire set. +If we reach a set element that is larger than the item we are looking +for, then we know that the item is not in the set: + + (define (element-of-set? x set) + (cond ((null? set) false) + ((= x (car set)) true) + ((< x (car set)) false) + (else (element-of-set? x (cdr set))))) + + How many steps does this save? In the worst case, the item we are +looking for may be the largest one in the set, so the number of steps +is the same as for the unordered representation. On the other hand, if +we search for items of many different sizes we can expect that +sometimes we will be able to stop searching at a point near the +beginning of the list and that other times we will still need to +examine most of the list. On the average we should expect to have to +examine about half of the items in the set. Thus, the average number +of steps required will be about n/2. This is still [theta](n) growth, +but it does save us, on the average, a factor of 2 in number of steps +over the previous implementation. + + We obtain a more impressive speedup with `intersection-set'. In the +unordered representation this operation required [theta](n^2) steps, +because we performed a complete scan of `set2' for each element of +`set1'. But with the ordered representation, we can use a more clever +method. Begin by comparing the initial elements, `x1' and `x2', of the +two sets. If `x1' equals `x2', then that gives an element of the +intersection, and the rest of the intersection is the intersection of +the `cdr's of the two sets. Suppose, however, that `x1' is less than +`x2'. Since `x2' is the smallest element in `set2', we can immediately +conclude that `x1' cannot appear anywhere in `set2' and hence is not in +the intersection. Hence, the intersection is equal to the intersection +of `set2' with the `cdr' of `set1'. Similarly, if `x2' is less than +`x1', then the intersection is given by the intersection of `set1' with +the `cdr' of `set2'. Here is the procedure: + + (define (intersection-set set1 set2) + (if (or (null? set1) (null? set2)) + '() + (let ((x1 (car set1)) (x2 (car set2))) + (cond ((= x1 x2) + (cons x1 + (intersection-set (cdr set1) + (cdr set2)))) + ((< x1 x2) + (intersection-set (cdr set1) set2)) + ((< x2 x1) + (intersection-set set1 (cdr set2))))))) + + To estimate the number of steps required by this process, observe +that at each step we reduce the intersection problem to computing +intersections of smaller sets--removing the first element from `set1' +or `set2' or both. Thus, the number of steps required is at most the +sum of the sizes of `set1' and `set2', rather than the product of the +sizes as with the unordered representation. This is [theta](n) growth +rather than [theta](n^2)--a considerable speedup, even for sets of +moderate size. + + *Exercise 2.61:* Give an implementation of `adjoin-set' using the + ordered representation. By analogy with `element-of-set?' show + how to take advantage of the ordering to produce a procedure that + requires on the average about half as many steps as with the + unordered representation. + + *Exercise 2.62:* Give a [theta](n) implementation of `union-set' + for sets represented as ordered lists. + +Sets as binary trees +.................... + +We can do better than the ordered-list representation by arranging the +set elements in the form of a tree. Each node of the tree holds one +element of the set, called the "entry" at that node, and a link to each +of two other (possibly empty) nodes. The "left" link points to +elements smaller than the one at the node, and the "right" link to +elements greater than the one at the node. *Note Figure 2-16:: shows +some trees that represent the set {1,3,5,7,9,11}. The same set may be +represented by a tree in a number of different ways. The only thing we +require for a valid representation is that all elements in the left +subtree be smaller than the node entry and that all elements in the +right subtree be larger. + + *Figure 2.16:* Various binary trees that represent the set + {1,3,5,7,9,11}. + + 7 3 5 + /\ /\ /\ + 3 9 1 7 3 9 + /\ \ /\ / /\ + 1 5 11 5 9 1 7 11 + \ + 11 + + The advantage of the tree representation is this: Suppose we want to +check whether a number x is contained in a set. We begin by comparing +x with the entry in the top node. If x is less than this, we know that +we need only search the left subtree; if x is greater, we need only +search the right subtree. Now, if the tree is "balanced," each of +these subtrees will be about half the size of the original. Thus, in +one step we have reduced the problem of searching a tree of size n to +searching a tree of size n/2. Since the size of the tree is halved at +each step, we should expect that the number of steps needed to search a +tree of size n grows as [theta](`log' n).(2) For large sets, this will +be a significant speedup over the previous representations. + + We can represent trees by using lists. Each node will be a list of +three items: the entry at the node, the left subtree, and the right +subtree. A left or a right subtree of the empty list will indicate +that there is no subtree connected there. We can describe this +representation by the following procedures:(3) + + (define (entry tree) (car tree)) + + (define (left-branch tree) (cadr tree)) + + (define (right-branch tree) (caddr tree)) + + (define (make-tree entry left right) + (list entry left right)) + + Now we can write the `element-of-set?' procedure using the strategy +described above: + + (define (element-of-set? x set) + (cond ((null? set) false) + ((= x (entry set)) true) + ((< x (entry set)) + (element-of-set? x (left-branch set))) + ((> x (entry set)) + (element-of-set? x (right-branch set))))) + + Adjoining an item to a set is implemented similarly and also requires +[theta](`log' n) steps. To adjoin an item `x', we compare `x' with the +node entry to determine whether `x' should be added to the right or to +the left branch, and having adjoined `x' to the appropriate branch we +piece this newly constructed branch together with the original entry +and the other branch. If `x' is equal to the entry, we just return the +node. If we are asked to adjoin `x' to an empty tree, we generate a +tree that has `x' as the entry and empty right and left branches. Here +is the procedure: + + (define (adjoin-set x set) + (cond ((null? set) (make-tree x '() '())) + ((= x (entry set)) set) + ((< x (entry set)) + (make-tree (entry set) + (adjoin-set x (left-branch set)) + (right-branch set))) + ((> x (entry set)) + (make-tree (entry set) + (left-branch set) + (adjoin-set x (right-branch set)))))) + + The above claim that searching the tree can be performed in a +logarithmic number of steps rests on the assumption that the tree is +"balanced," i.e., that the left and the right subtree of every tree +have approximately the same number of elements, so that each subtree +contains about half the elements of its parent. But how can we be +certain that the trees we construct will be balanced? Even if we start +with a balanced tree, adding elements with `adjoin-set' may produce an +unbalanced result. Since the position of a newly adjoined element +depends on how the element compares with the items already in the set, +we can expect that if we add elements "randomly" the tree will tend to +be balanced on the average. But this is not a guarantee. For example, +if we start with an empty set and adjoin the numbers 1 through 7 in +sequence we end up with the highly unbalanced tree shown in *Note +Figure 2-17::. In this tree all the left subtrees are empty, so it has +no advantage over a simple ordered list. One way to solve this problem +is to define an operation that transforms an arbitrary tree into a +balanced tree with the same elements. Then we can perform this +transformation after every few `adjoin-set' operations to keep our set +in balance. There are also other ways to solve this problem, most of +which involve designing new data structures for which searching and +insertion both can be done in [theta](`log' n) steps.(4) + + *Figure 2.17:* Unbalanced tree produced by adjoining 1 through 7 + in sequence. + + 1 + \ + 2 + \ + 4 + \ + 5 + \ + 6 + \ + 7 + + *Exercise 2.63:* Each of the following two procedures converts a + binary tree to a list. + + (define (tree->list-1 tree) + (if (null? tree) + '() + (append (tree->list-1 (left-branch tree)) + (cons (entry tree) + (tree->list-1 (right-branch tree)))))) + + (define (tree->list-2 tree) + (define (copy-to-list tree result-list) + (if (null? tree) + result-list + (copy-to-list (left-branch tree) + (cons (entry tree) + (copy-to-list (right-branch tree) + result-list))))) + (copy-to-list tree '())) + + a. Do the two procedures produce the same result for every tree? + If not, how do the results differ? What lists do the two + procedures produce for the trees in *Note Figure 2-16::? + + b. Do the two procedures have the same order of growth in the + number of steps required to convert a balanced tree with n + elements to a list? If not, which one grows more slowly? + + + *Exercise 2.64:* The following procedure `list->tree' converts an + ordered list to a balanced binary tree. The helper procedure + `partial-tree' takes as arguments an integer n and list of at + least n elements and constructs a balanced tree containing the + first n elements of the list. The result returned by + `partial-tree' is a pair (formed with `cons') whose `car' is the + constructed tree and whose `cdr' is the list of elements not + included in the tree. + + (define (list->tree elements) + (car (partial-tree elements (length elements)))) + + (define (partial-tree elts n) + (if (= n 0) + (cons '() elts) + (let ((left-size (quotient (- n 1) 2))) + (let ((left-result (partial-tree elts left-size))) + (let ((left-tree (car left-result)) + (non-left-elts (cdr left-result)) + (right-size (- n (+ left-size 1)))) + (let ((this-entry (car non-left-elts)) + (right-result (partial-tree (cdr non-left-elts) + right-size))) + (let ((right-tree (car right-result)) + (remaining-elts (cdr right-result))) + (cons (make-tree this-entry left-tree right-tree) + remaining-elts)))))))) + + a. Write a short paragraph explaining as clearly as you can how + `partial-tree' works. Draw the tree produced by `list->tree' + for the list `(1 3 5 7 9 11)'. + + b. What is the order of growth in the number of steps required by + `list->tree' to convert a list of n elements? + + + *Exercise 2.65:* Use the results of *Note Exercise 2-63:: and + *Note Exercise 2-64:: to give [theta](n) implementations of + `union-set' and `intersection-set' for sets implemented as + (balanced) binary trees.(5) + +Sets and information retrieval +.............................. + +We have examined options for using lists to represent sets and have +seen how the choice of representation for a data object can have a +large impact on the performance of the programs that use the data. +Another reason for concentrating on sets is that the techniques +discussed here appear again and again in applications involving +information retrieval. + + Consider a data base containing a large number of individual +records, such as the personnel files for a company or the transactions +in an accounting system. A typical data-management system spends a +large amount of time accessing or modifying the data in the records and +therefore requires an efficient method for accessing records. This is +done by identifying a part of each record to serve as an identifying "key". +A key can be anything that uniquely identifies the record. For a +personnel file, it might be an employee's ID number. For an accounting +system, it might be a transaction number. Whatever the key is, when we +define the record as a data structure we should include a `key' +selector procedure that retrieves the key associated with a given +record. + + Now we represent the data base as a set of records. To locate the +record with a given key we use a procedure `lookup', which takes as +arguments a key and a data base and which returns the record that has +that key, or false if there is no such record. `Lookup' is implemented +in almost the same way as `element-of-set?'. For example, if the set +of records is implemented as an unordered list, we could use + + (define (lookup given-key set-of-records) + (cond ((null? set-of-records) false) + ((equal? given-key (key (car set-of-records))) + (car set-of-records)) + (else (lookup given-key (cdr set-of-records))))) + + Of course, there are better ways to represent large sets than as +unordered lists. Information-retrieval systems in which records have +to be "randomly accessed" are typically implemented by a tree-based +method, such as the binary-tree representation discussed previously. +In designing such a system the methodology of data abstraction can be a +great help. The designer can create an initial implementation using a +simple, straightforward representation such as unordered lists. This +will be unsuitable for the eventual system, but it can be useful in +providing a "quick and dirty" data base with which to test the rest of +the system. Later on, the data representation can be modified to be +more sophisticated. If the data base is accessed in terms of abstract +selectors and constructors, this change in representation will not +require any changes to the rest of the system. + + *Exercise 2.66:* Implement the `lookup' procedure for the case + where the set of records is structured as a binary tree, ordered + by the numerical values of the keys. + + ---------- Footnotes ---------- + + (1) If we want to be more formal, we can specify "consistent with +the interpretations given above" to mean that the operations satisfy a +collection of rules such as these: + + * For any set `S' and any object `x', `(element-of-set? x + (adjoin-set x S))' is true (informally: "Adjoining an object to a + set produces a set that contains the object"). + + * For any sets `S' and `T' and any object `x', `(element-of-set? x + (union-set S T))' is equal to `(or (element-of-set? x S) + (element-of-set? x T))' (informally: "The elements of `(union S + T)' are the elements that are in `S' or in `T'"). + + * For any object `x', `(element-of-set? x '())' is false + (informally: "No object is an element of the empty set"). + + + (2) Halving the size of the problem at each step is the +distinguishing characteristic of logarithmic growth, as we saw with the +fast-exponentiation algorithm of section *Note 1-2-4:: and the +half-interval search method of section *Note 1-3-3::. + + (3) We are representing sets in terms of trees, and trees in terms +of lists--in effect, a data abstraction built upon a data abstraction. +We can regard the procedures `entry', `left-branch', `right-branch', +and `make-tree' as a way of isolating the abstraction of a "binary +tree" from the particular way we might wish to represent such a tree in +terms of list structure. + + (4) Examples of such structures include "B-trees" and "red-black +trees". There is a large literature on data structures devoted to this +problem. See Cormen, Leiserson, and Rivest 1990. + + (5) *Note Exercise 2-63:: through *Note Exercise 2-65:: are due to +Paul Hilfinger. + + +File: sicp.info, Node: 2-3-4, Prev: 2-3-3, Up: 2-3 + +2.3.4 Example: Huffman Encoding Trees +------------------------------------- + +This section provides practice in the use of list structure and data +abstraction to manipulate sets and trees. The application is to +methods for representing data as sequences of ones and zeros (bits). +For example, the ASCII standard code used to represent text in +computers encodes each character as a sequence of seven bits. Using +seven bits allows us to distinguish 2^(7), or 128, possible different +characters. In general, if we want to distinguish n different symbols, +we will need to use `log'_2 n bits per symbol. If all our messages are +made up of the eight symbols A, B, C, D, E, F, G, and H, we can choose +a code with three bits per character, for example + + A 000 C 010 E 100 G 110 + B 001 D 011 F 101 H 111 + +With this code, the message + + BACADAEAFABBAAAGAH + +is encoded as the string of 54 bits + + 001000010000011000100000101000001001000000000110000111 + + Codes such as ASCII and the A-through-H code above are known as "fixed-length" +codes, because they represent each symbol in the message with the same +number of bits. It is sometimes advantageous to use "variable-length" +codes, in which different symbols may be represented by different +numbers of bits. For example, Morse code does not use the same number +of dots and dashes for each letter of the alphabet. In particular, E, +the most frequent letter, is represented by a single dot. In general, +if our messages are such that some symbols appear very frequently and +some very rarely, we can encode data more efficiently (i.e., using +fewer bits per message) if we assign shorter codes to the frequent +symbols. Consider the following alternative code for the letters A +through H: + + A 0 C 1010 E 1100 G 1110 + B 100 D 1011 F 1101 H 1111 + +With this code, the same message as above is encoded as the string + + 100010100101101100011010100100000111001111 + + This string contains 42 bits, so it saves more than 20% in space in +comparison with the fixed-length code shown above. + + One of the difficulties of using a variable-length code is knowing +when you have reached the end of a symbol in reading a sequence of +zeros and ones. Morse code solves this problem by using a special "separator +code" (in this case, a pause) after the sequence of dots and dashes for +each letter. Another solution is to design the code in such a way that +no complete code for any symbol is the beginning (or "prefix") of the +code for another symbol. Such a code is called a "prefix code". In +the example above, A is encoded by 0 and B is encoded by 100, so no +other symbol can have a code that begins with 0 or with 100. + + In general, we can attain significant savings if we use +variable-length prefix codes that take advantage of the relative +frequencies of the symbols in the messages to be encoded. One +particular scheme for doing this is called the Huffman encoding method, +after its discoverer, David Huffman. A Huffman code can be represented +as a binary tree whose leaves are the symbols that are encoded. At +each non-leaf node of the tree there is a set containing all the +symbols in the leaves that lie below the node. In addition, each +symbol at a leaf is assigned a weight (which is its relative +frequency), and each non-leaf node contains a weight that is the sum of +all the weights of the leaves lying below it. The weights are not used +in the encoding or the decoding process. We will see below how they +are used to help construct the tree. + + *Figure 2.18:* A Huffman encoding tree. + + {A B C D E F G H} 17 + * + / \ + / \ + A 8 * {B C D E F G H} 9 + __________/ \_____________ + / \ + {B C D} 5 * * {E F G H} 4 + / \ ___/ \___ + / \ / \ + B 3 * {C D} 2 {E F} 2 * * {G H} 2 + / \ / \ / \ + / \ / \ / \ + C 1 D 1 E 1 F 1 G 1 H 1 + + *Note Figure 2-18:: shows the Huffman tree for the A-through-H code +given above. The weights at the leaves indicate that the tree was +designed for messages in which A appears with relative frequency 8, B +with relative frequency 3, and the other letters each with relative +frequency 1. + + Given a Huffman tree, we can find the encoding of any symbol by +starting at the root and moving down until we reach the leaf that holds +the symbol. Each time we move down a left branch we add a 0 to the +code, and each time we move down a right branch we add a 1. (We decide +which branch to follow by testing to see which branch either is the +leaf node for the symbol or contains the symbol in its set.) For +example, starting from the root of the tree in *Note Figure 2-18::, we +arrive at the leaf for D by following a right branch, then a left +branch, then a right branch, then a right branch; hence, the code for D +is 1011. + + To decode a bit sequence using a Huffman tree, we begin at the root +and use the successive zeros and ones of the bit sequence to determine +whether to move down the left or the right branch. Each time we come +to a leaf, we have generated a new symbol in the message, at which +point we start over from the root of the tree to find the next symbol. +For example, suppose we are given the tree above and the sequence +10001010. Starting at the root, we move down the right branch, (since +the first bit of the string is 1), then down the left branch (since the +second bit is 0), then down the left branch (since the third bit is +also 0). This brings us to the leaf for B, so the first symbol of the +decoded message is B. Now we start again at the root, and we make a +left move because the next bit in the string is 0. This brings us to +the leaf for A. Then we start again at the root with the rest of the +string 1010, so we move right, left, right, left and reach C. Thus, +the entire message is BAC. + +Generating Huffman trees +........................ + +Given an "alphabet" of symbols and their relative frequencies, how do we +construct the "best" code? (In other words, which tree will encode +messages with the fewest bits?) Huffman gave an algorithm for doing +this and showed that the resulting code is indeed the best +variable-length code for messages where the relative frequency of the +symbols matches the frequencies with which the code was constructed. +We will not prove this optimality of Huffman codes here, but we will +show how Huffman trees are constructed.(1) + + The algorithm for generating a Huffman tree is very simple. The idea +is to arrange the tree so that the symbols with the lowest frequency +appear farthest away from the root. Begin with the set of leaf nodes, +containing symbols and their frequencies, as determined by the initial +data from which the code is to be constructed. Now find two leaves with +the lowest weights and merge them to produce a node that has these two +nodes as its left and right branches. The weight of the new node is the +sum of the two weights. Remove the two leaves from the original set and +replace them by this new node. Now continue this process. At each step, +merge two nodes with the smallest weights, removing them from the set +and replacing them with a node that has these two as its left and right +branches. The process stops when there is only one node left, which is +the root of the entire tree. Here is how the Huffman tree of *Note +Figure 2-18:: was generated: + + Initial leaves {(A 8) (B 3) (C 1) (D 1) (E 1) (F 1) (G 1) (H 1)} + Merge {(A 8) (B 3) ({C D} 2) (E 1) (F 1) (G 1) (H 1)} + Merge {(A 8) (B 3) ({C D} 2) ({E F} 2) (G 1) (H 1)} + Merge {(A 8) (B 3) ({C D} 2) ({E F} 2) ({G H} 2)} + Merge {(A 8) (B 3) ({C D} 2) ({E F G H} 4)} + Merge {(A 8) ({B C D} 5) ({E F G H} 4)} + Merge {(A 8) ({B C D E F G H} 9)} + Final merge {({A B C D E F G H} 17)} + + The algorithm does not always specify a unique tree, because there +may not be unique smallest-weight nodes at each step. Also, the choice +of the order in which the two nodes are merged (i.e., which will be the +right branch and which will be the left branch) is arbitrary. + +Representing Huffman trees +.......................... + +In the exercises below we will work with a system that uses Huffman +trees to encode and decode messages and generates Huffman trees +according to the algorithm outlined above. We will begin by discussing +how trees are represented. + + Leaves of the tree are represented by a list consisting of the symbol +`leaf', the symbol at the leaf, and the weight: + + (define (make-leaf symbol weight) + (list 'leaf symbol weight)) + + (define (leaf? object) + (eq? (car object) 'leaf)) + + (define (symbol-leaf x) (cadr x)) + + (define (weight-leaf x) (caddr x)) + + A general tree will be a list of a left branch, a right branch, a +set of symbols, and a weight. The set of symbols will be simply a list +of the symbols, rather than some more sophisticated set representation. +When we make a tree by merging two nodes, we obtain the weight of the +tree as the sum of the weights of the nodes, and the set of symbols as +the union of the sets of symbols for the nodes. Since our symbol sets +are represented as lists, we can form the union by using the `append' +procedure we defined in section *Note 2-2-1::: + + (define (make-code-tree left right) + (list left + right + (append (symbols left) (symbols right)) + (+ (weight left) (weight right)))) + + If we make a tree in this way, we have the following selectors: + + (define (left-branch tree) (car tree)) + + (define (right-branch tree) (cadr tree)) + + (define (symbols tree) + (if (leaf? tree) + (list (symbol-leaf tree)) + (caddr tree))) + + (define (weight tree) + (if (leaf? tree) + (weight-leaf tree) + (cadddr tree))) + + The procedures `symbols' and `weight' must do something slightly +different depending on whether they are called with a leaf or a general +tree. These are simple examples of "generic procedures" (procedures +that can handle more than one kind of data), which we will have much +more to say about in sections *Note 2-4:: and *Note 2-5::. + +The decoding procedure +...................... + +The following procedure implements the decoding algorithm. It takes as +arguments a list of zeros and ones, together with a Huffman tree. + + (define (decode bits tree) + (define (decode-1 bits current-branch) + (if (null? bits) + '() + (let ((next-branch + (choose-branch (car bits) current-branch))) + (if (leaf? next-branch) + (cons (symbol-leaf next-branch) + (decode-1 (cdr bits) tree)) + (decode-1 (cdr bits) next-branch))))) + (decode-1 bits tree)) + + (define (choose-branch bit branch) + (cond ((= bit 0) (left-branch branch)) + ((= bit 1) (right-branch branch)) + (else (error "bad bit -- CHOOSE-BRANCH" bit)))) + + The procedure `decode-1' takes two arguments: the list of remaining +bits and the current position in the tree. It keeps moving "down" the +tree, choosing a left or a right branch according to whether the next +bit in the list is a zero or a one. (This is done with the procedure +`choose-branch'.) When it reaches a leaf, it returns the symbol at +that leaf as the next symbol in the message by `cons'ing it onto the +result of decoding the rest of the message, starting at the root of the +tree. Note the error check in the final clause of `choose-branch', +which complains if the procedure finds something other than a zero or a +one in the input data. + +Sets of weighted elements +......................... + +In our representation of trees, each non-leaf node contains a set of +symbols, which we have represented as a simple list. However, the +tree-generating algorithm discussed above requires that we also work +with sets of leaves and trees, successively merging the two smallest +items. Since we will be required to repeatedly find the smallest item +in a set, it is convenient to use an ordered representation for this +kind of set. + + We will represent a set of leaves and trees as a list of elements, +arranged in increasing order of weight. The following `adjoin-set' +procedure for constructing sets is similar to the one described in +*Note Exercise 2-61::; however, items are compared by their weights, +and the element being added to the set is never already in it. + + (define (adjoin-set x set) + (cond ((null? set) (list x)) + ((< (weight x) (weight (car set))) (cons x set)) + (else (cons (car set) + (adjoin-set x (cdr set)))))) + + The following procedure takes a list of symbol-frequency pairs such +as `((A 4) (B 2) (C 1) (D 1))' and constructs an initial ordered set of +leaves, ready to be merged according to the Huffman algorithm: + + (define (make-leaf-set pairs) + (if (null? pairs) + '() + (let ((pair (car pairs))) + (adjoin-set (make-leaf (car pair) ; symbol + (cadr pair)) ; frequency + (make-leaf-set (cdr pairs)))))) + + *Exercise 2.67:* Define an encoding tree and a sample message: + + (define sample-tree + (make-code-tree (make-leaf 'A 4) + (make-code-tree + (make-leaf 'B 2) + (make-code-tree (make-leaf 'D 1) + (make-leaf 'C 1))))) + + (define sample-message '(0 1 1 0 0 1 0 1 0 1 1 1 0)) + + Use the `decode' procedure to decode the message, and give the + result. + + *Exercise 2.68:* The `encode' procedure takes as arguments a + message and a tree and produces the list of bits that gives the + encoded message. + + (define (encode message tree) + (if (null? message) + '() + (append (encode-symbol (car message) tree) + (encode (cdr message) tree)))) + + `Encode-symbol' is a procedure, which you must write, that returns + the list of bits that encodes a given symbol according to a given + tree. You should design `encode-symbol' so that it signals an + error if the symbol is not in the tree at all. Test your + procedure by encoding the result you obtained in *Note Exercise + 2-67:: with the sample tree and seeing whether it is the same as + the original sample message. + + *Exercise 2.69:* The following procedure takes as its argument a + list of symbol-frequency pairs (where no symbol appears in more + than one pair) and generates a Huffman encoding tree according to + the Huffman algorithm. + + (define (generate-huffman-tree pairs) + (successive-merge (make-leaf-set pairs))) + + `Make-leaf-set' is the procedure given above that transforms the + list of pairs into an ordered set of leaves. `Successive-merge' + is the procedure you must write, using `make-code-tree' to + successively merge the smallest-weight elements of the set until + there is only one element left, which is the desired Huffman tree. + (This procedure is slightly tricky, but not really complicated. + If you find yourself designing a complex procedure, then you are + almost certainly doing something wrong. You can take significant + advantage of the fact that we are using an ordered set + representation.) + + *Exercise 2.70:* The following eight-symbol alphabet with + associated relative frequencies was designed to efficiently encode + the lyrics of 1950s rock songs. (Note that the "symbols" of an + "alphabet" need not be individual letters.) + + A 2 NA 16 + BOOM 1 SHA 3 + GET 2 YIP 9 + JOB 2 WAH 1 + + Use `generate-huffman-tree' (*Note Exercise 2-69::) to generate a + corresponding Huffman tree, and use `encode' (*Note Exercise + 2-68::) to encode the following message: + + Get a job + + Sha na na na na na na na na + + Get a job + + Sha na na na na na na na na + + Wah yip yip yip yip yip yip yip yip yip + + Sha boom + + How many bits are required for the encoding? What is the smallest + number of bits that would be needed to encode this song if we used + a fixed-length code for the eight-symbol alphabet? + + *Exercise 2.71:* Suppose we have a Huffman tree for an alphabet of + n symbols, and that the relative frequencies of the symbols are 1, + 2, 4, ..., 2^(n-1). Sketch the tree for n=5; for n=10. In such a + tree (for general n) how may bits are required to encode the most + frequent symbol? the least frequent symbol? + + *Exercise 2.72:* Consider the encoding procedure that you designed + in *Note Exercise 2-68::. What is the order of growth in the + number of steps needed to encode a symbol? Be sure to include the + number of steps needed to search the symbol list at each node + encountered. To answer this question in general is difficult. + Consider the special case where the relative frequencies of the n + symbols are as described in *Note Exercise 2-71::, and give the + order of growth (as a function of n) of the number of steps needed + to encode the most frequent and least frequent symbols in the + alphabet. + + ---------- Footnotes ---------- + + (1) See Hamming 1980 for a discussion of the mathematical properties +of Huffman codes. + + +File: sicp.info, Node: 2-4, Next: 2-5, Prev: 2-3, Up: Chapter 2 + +2.4 Multiple Representations for Abstract Data +============================================== + +We have introduced data abstraction, a methodology for structuring +systems in such a way that much of a program can be specified +independent of the choices involved in implementing the data objects +that the program manipulates. For example, we saw in section *Note +2-1-1:: how to separate the task of designing a program that uses +rational numbers from the task of implementing rational numbers in +terms of the computer language's primitive mechanisms for constructing +compound data. The key idea was to erect an abstraction barrier - in +this case, the selectors and constructors for rational numbers +(`make-rat', `numer', `denom')--that isolates the way rational numbers +are used from their underlying representation in terms of list +structure. A similar abstraction barrier isolates the details of the +procedures that perform rational arithmetic (`add-rat', `sub-rat', +`mul-rat', and `div-rat') from the "higher-level" procedures that use +rational numbers. The resulting program has the structure shown in +*Note Figure 2-1::. + + These data-abstraction barriers are powerful tools for controlling +complexity. By isolating the underlying representations of data +objects, we can divide the task of designing a large program into +smaller tasks that can be performed separately. But this kind of data +abstraction is not yet powerful enough, because it may not always make +sense to speak of "the underlying representation" for a data object. + + For one thing, there might be more than one useful representation +for a data object, and we might like to design systems that can deal +with multiple representations. To take a simple example, complex +numbers may be represented in two almost equivalent ways: in +rectangular form (real and imaginary parts) and in polar form +(magnitude and angle). Sometimes rectangular form is more appropriate +and sometimes polar form is more appropriate. Indeed, it is perfectly +plausible to imagine a system in which complex numbers are represented +in both ways, and in which the procedures for manipulating complex +numbers work with either representation. + + More importantly, programming systems are often designed by many +people working over extended periods of time, subject to requirements +that change over time. In such an environment, it is simply not +possible for everyone to agree in advance on choices of data +representation. So in addition to the data-abstraction barriers that +isolate representation from use, we need abstraction barriers that +isolate different design choices from each other and permit different +choices to coexist in a single program. Furthermore, since large +programs are often created by combining pre-existing modules that were +designed in isolation, we need conventions that permit programmers to +incorporate modules into larger systems "additively", that is, without +having to redesign or reimplement these modules. + + In this section, we will learn how to cope with data that may be +represented in different ways by different parts of a program. This +requires constructing "generic procedures"--procedures that can operate +on data that may be represented in more than one way. Our main +technique for building generic procedures will be to work in terms of +data objects that have tags "type tags", that is, data objects that +include explicit information about how they are to be processed. We +will also discuss "data-directed" programming, a powerful and +convenient implementation strategy for additively assembling systems +with generic operations. + + We begin with the simple complex-number example. We will see how +type tags and data-directed style enable us to design separate +rectangular and polar representations for complex numbers while +maintaining the notion of an abstract "complex-number" data object. We +will accomplish this by defining arithmetic procedures for complex +numbers (`add-complex', `sub-complex', `mul-complex', and +`div-complex') in terms of generic selectors that access parts of a +complex number independent of how the number is represented. The +resulting complex-number system, as shown in *Note Figure 2-19::, +contains two different kinds of abstraction barriers. The "horizontal" +abstraction barriers play the same role as the ones in *Note Figure +2-1::. They isolate "higher-level" operations from "lower-level" +representations. In addition, there is a "vertical" barrier that gives +us the ability to separately design and install alternative +representations. + + *Figure 2.19:* Data-abstraction barriers in the complex-number + system. + + Programs that use complex numbers + +-------------------------------------------------+ + --| add-complex sub-complex mul-complex div-complex |-- + +-------------------------------------------------+ + Complex arithmetic package + ---------------------------+--------------------------- + Rectangular | Polar + representation | representation + ---------------------------+--------------------------- + List structure and primitive machine arithmetic + + In section *Note 2-5:: we will show how to use type tags and +data-directed style to develop a generic arithmetic package. This +provides procedures (`add', `mul', and so on) that can be used to +manipulate all sorts of "numbers" and can be easily extended when a new +kind of number is needed. In section *Note 2-5-3::, we'll show how to +use generic arithmetic in a system that performs symbolic algebra. + +* Menu: + +* 2-4-1:: Representations for Complex Numbers +* 2-4-2:: Tagged data +* 2-4-3:: Data-Directed Programming and Additivity + + +File: sicp.info, Node: 2-4-1, Next: 2-4-2, Prev: 2-4, Up: 2-4 + +2.4.1 Representations for Complex Numbers +----------------------------------------- + +We will develop a system that performs arithmetic operations on complex +numbers as a simple but unrealistic example of a program that uses +generic operations. We begin by discussing two plausible +representations for complex numbers as ordered pairs: rectangular form +(real part and imaginary part) and polar form (magnitude and angle).(1) +Section *Note 2-4-2:: will show how both representations can be made +to coexist in a single system through the use of type tags and generic +operations. + + Like rational numbers, complex numbers are naturally represented as +ordered pairs. The set of complex numbers can be thought of as a +two-dimensional space with two orthogonal axes, the "real" axis and the +"imaginary" axis. (See *Note Figure 2-20::.) From this point of view, +the complex number z = x + iy (where i^2 = - 1) can be thought of as +the point in the plane whose real coordinate is x and whose imaginary +coordinate is y. Addition of complex numbers reduces in this +representation to addition of coordinates: + + Real-part(z_1 + z_2) = Real-part(z_1) + Real-part(z_2) + + Imaginary-part(z_1 + z_2) = Imaginary-part(z_1) + Imaginary-part(z_2) + + When multiplying complex numbers, it is more natural to think in +terms of representing a complex number in polar form, as a magnitude +and an angle (r and A in *Note Figure 2-20::). The product of two +complex numbers is the vector obtained by stretching one complex number +by the length of the other and then rotating it through the angle of +the other: + + Magnitude(z_1 * z_2) = Magnitude(z_1) * Magnitude(z_2) + + Angle(z_1 * z_2) = Angle(z_1) + Angle(z_2) + + *Figure 2.20:* Complex numbers as points in the plane. + + Imaginary + ^ + | + y |.........................* z = x + ?y = r e^(?A) + | __-- . + | __-- . + | r __-- . + | __-- . + | __-- \ . + |__-- A | . + ----+----------+-------------------> Real + x + + Thus, there are two different representations for complex numbers, +which are appropriate for different operations. Yet, from the +viewpoint of someone writing a program that uses complex numbers, the +principle of data abstraction suggests that all the operations for +manipulating complex numbers should be available regardless of which +representation is used by the computer. For example, it is often +useful to be able to find the magnitude of a complex number that is +specified by rectangular coordinates. Similarly, it is often useful to +be able to determine the real part of a complex number that is +specified by polar coordinates. + + To design such a system, we can follow the same data-abstraction +strategy we followed in designing the rational-number package in +section *Note 2-1-1::. Assume that the operations on complex numbers +are implemented in terms of four selectors: `real-part', `imag-part', +`magnitude', and `angle'. Also assume that we have two procedures for +constructing complex numbers: `make-from-real-imag' returns a complex +number with specified real and imaginary parts, and `make-from-mag-ang' +returns a complex number with specified magnitude and angle. These +procedures have the property that, for any complex number `z', both + + (make-from-real-imag (real-part z) (imag-part z)) + +and + + (make-from-mag-ang (magnitude z) (angle z)) + +produce complex numbers that are equal to `z'. + + Using these constructors and selectors, we can implement arithmetic +on complex numbers using the "abstract data" specified by the +constructors and selectors, just as we did for rational numbers in +section *Note 2-1-1::. As shown in the formulas above, we can add and +subtract complex numbers in terms of real and imaginary parts while +multiplying and dividing complex numbers in terms of magnitudes and +angles: + + (define (add-complex z1 z2) + (make-from-real-imag (+ (real-part z1) (real-part z2)) + (+ (imag-part z1) (imag-part z2)))) + + (define (sub-complex z1 z2) + (make-from-real-imag (- (real-part z1) (real-part z2)) + (- (imag-part z1) (imag-part z2)))) + + (define (mul-complex z1 z2) + (make-from-mag-ang (* (magnitude z1) (magnitude z2)) + (+ (angle z1) (angle z2)))) + + (define (div-complex z1 z2) + (make-from-mag-ang (/ (magnitude z1) (magnitude z2)) + (- (angle z1) (angle z2)))) + + To complete the complex-number package, we must choose a +representation and we must implement the constructors and selectors in +terms of primitive numbers and primitive list structure. There are two +obvious ways to do this: We can represent a complex number in +"rectangular form" as a pair (real part, imaginary part) or in "polar +form" as a pair (magnitude, angle). Which shall we choose? + + In order to make the different choices concrete, imagine that there +are two programmers, Ben Bitdiddle and Alyssa P. Hacker, who are +independently designing representations for the complex-number system. +Ben chooses to represent complex numbers in rectangular form. With +this choice, selecting the real and imaginary parts of a complex number +is straightforward, as is constructing a complex number with given real +and imaginary parts. To find the magnitude and the angle, or to +construct a complex number with a given magnitude and angle, he uses +the trigonometric relations + + __________ + x = r cos A r = ./ x^2 + y^2 + + y = r sin A A = arctan(y,x) + +which relate the real and imaginary parts (x, y) to the magnitude and +the angle (r, A).(2) Ben's representation is therefore given by the +following selectors and constructors: + + (define (real-part z) (car z)) + + (define (imag-part z) (cdr z)) + + (define (magnitude z) + (sqrt (+ (square (real-part z)) (square (imag-part z))))) + + (define (angle z) + (atan (imag-part z) (real-part z))) + + (define (make-from-real-imag x y) (cons x y)) + + (define (make-from-mag-ang r a) + (cons (* r (cos a)) (* r (sin a)))) + + Alyssa, in contrast, chooses to represent complex numbers in polar +form. For her, selecting the magnitude and angle is straightforward, +but she has to use the trigonometric relations to obtain the real and +imaginary parts. Alyssa's representation is: + + (define (real-part z) + (* (magnitude z) (cos (angle z)))) + + (define (imag-part z) + (* (magnitude z) (sin (angle z)))) + + (define (magnitude z) (car z)) + + (define (angle z) (cdr z)) + + (define (make-from-real-imag x y) + (cons (sqrt (+ (square x) (square y))) + (atan y x))) + + (define (make-from-mag-ang r a) (cons r a)) + + The discipline of data abstraction ensures that the same +implementation of `add-complex', `sub-complex', `mul-complex', and +`div-complex' will work with either Ben's representation or Alyssa's +representation. + + ---------- Footnotes ---------- + + (1) In actual computational systems, rectangular form is preferable +to polar form most of the time because of roundoff errors in conversion +between rectangular and polar form. This is why the complex-number +example is unrealistic. Nevertheless, it provides a clear illustration +of the design of a system using generic operations and a good +introduction to the more substantial systems to be developed later in +this chapter. + + (2) The arctangent function referred to here, computed by Scheme's +`atan' procedure, is defined so as to take two arguments y and x and to +return the angle whose tangent is y/x. The signs of the arguments +determine the quadrant of the angle. + + +File: sicp.info, Node: 2-4-2, Next: 2-4-3, Prev: 2-4-1, Up: 2-4 + +2.4.2 Tagged data +----------------- + +One way to view data abstraction is as an application of the "principle +of least commitment." In implementing the complex-number system in +section *Note 2-4-1::, we can use either Ben's rectangular +representation or Alyssa's polar representation. The abstraction +barrier formed by the selectors and constructors permits us to defer to +the last possible moment the choice of a concrete representation for +our data objects and thus retain maximum flexibility in our system +design. + + The principle of least commitment can be carried to even further +extremes. If we desire, we can maintain the ambiguity of +representation even _after_ we have designed the selectors and +constructors, and elect to use both Ben's representation _and_ Alyssa's +representation. If both representations are included in a single +system, however, we will need some way to distinguish data in polar +form from data in rectangular form. Otherwise, if we were asked, for +instance, to find the `magnitude' of the pair (3,4), we wouldn't know +whether to answer 5 (interpreting the number in rectangular form) or 3 +(interpreting the number in polar form). A straightforward way to +accomplish this distinction is to include a "type tag"--the symbol +`rectangular' or `polar'--as part of each complex number. Then when we +need to manipulate a complex number we can use the tag to decide which +selector to apply. + + In order to manipulate tagged data, we will assume that we have +procedures `type-tag' and `contents' that extract from a data object +the tag and the actual contents (the polar or rectangular coordinates, +in the case of a complex number). We will also postulate a procedure +`attach-tag' that takes a tag and contents and produces a tagged data +object. A straightforward way to implement this is to use ordinary +list structure: + + (define (attach-tag type-tag contents) + (cons type-tag contents)) + + (define (type-tag datum) + (if (pair? datum) + (car datum) + (error "Bad tagged datum -- TYPE-TAG" datum))) + + (define (contents datum) + (if (pair? datum) + (cdr datum) + (error "Bad tagged datum -- CONTENTS" datum))) + + Using these procedures, we can define predicates `rectangular?' and +`polar?', which recognize polar and rectangular numbers, respectively: + + (define (rectangular? z) + (eq? (type-tag z) 'rectangular)) + + (define (polar? z) + (eq? (type-tag z) 'polar)) + + With type tags, Ben and Alyssa can now modify their code so that +their two different representations can coexist in the same system. +Whenever Ben constructs a complex number, he tags it as rectangular. +Whenever Alyssa constructs a complex number, she tags it as polar. In +addition, Ben and Alyssa must make sure that the names of their +procedures do not conflict. One way to do this is for Ben to append +the suffix `rectangular' to the name of each of his representation +procedures and for Alyssa to append `polar' to the names of hers. Here +is Ben's revised rectangular representation from section *Note 2-4-1::: + + (define (real-part-rectangular z) (car z)) + + (define (imag-part-rectangular z) (cdr z)) + + (define (magnitude-rectangular z) + (sqrt (+ (square (real-part-rectangular z)) + (square (imag-part-rectangular z))))) + + (define (angle-rectangular z) + (atan (imag-part-rectangular z) + (real-part-rectangular z))) + + (define (make-from-real-imag-rectangular x y) + (attach-tag 'rectangular (cons x y))) + + (define (make-from-mag-ang-rectangular r a) + (attach-tag 'rectangular + (cons (* r (cos a)) (* r (sin a))))) + +and here is Alyssa's revised polar representation: + + (define (real-part-polar z) + (* (magnitude-polar z) (cos (angle-polar z)))) + + (define (imag-part-polar z) + (* (magnitude-polar z) (sin (angle-polar z)))) + + (define (magnitude-polar z) (car z)) + + (define (angle-polar z) (cdr z)) + + (define (make-from-real-imag-polar x y) + (attach-tag 'polar + (cons (sqrt (+ (square x) (square y))) + (atan y x)))) + + (define (make-from-mag-ang-polar r a) + (attach-tag 'polar (cons r a))) + + Each generic selector is implemented as a procedure that checks the +tag of its argument and calls the appropriate procedure for handling +data of that type. For example, to obtain the real part of a complex +number, `real-part' examines the tag to determine whether to use Ben's +`real-part-rectangular' or Alyssa's `real-part-polar'. In either case, +we use `contents' to extract the bare, untagged datum and send this to +the rectangular or polar procedure as required: + + (define (real-part z) + (cond ((rectangular? z) + (real-part-rectangular (contents z))) + ((polar? z) + (real-part-polar (contents z))) + (else (error "Unknown type -- REAL-PART" z)))) + + (define (imag-part z) + (cond ((rectangular? z) + (imag-part-rectangular (contents z))) + ((polar? z) + (imag-part-polar (contents z))) + (else (error "Unknown type -- IMAG-PART" z)))) + + (define (magnitude z) + (cond ((rectangular? z) + (magnitude-rectangular (contents z))) + ((polar? z) + (magnitude-polar (contents z))) + (else (error "Unknown type -- MAGNITUDE" z)))) + + (define (angle z) + (cond ((rectangular? z) + (angle-rectangular (contents z))) + ((polar? z) + (angle-polar (contents z))) + (else (error "Unknown type -- ANGLE" z)))) + + To implement the complex-number arithmetic operations, we can use +the same procedures `add-complex', `sub-complex', `mul-complex', and +`div-complex' from section *Note 2-4-1::, because the selectors they +call are generic, and so will work with either representation. For +example, the procedure `add-complex' is still + + (define (add-complex z1 z2) + (make-from-real-imag (+ (real-part z1) (real-part z2)) + (+ (imag-part z1) (imag-part z2)))) + + Finally, we must choose whether to construct complex numbers using +Ben's representation or Alyssa's representation. One reasonable choice +is to construct rectangular numbers whenever we have real and imaginary +parts and to construct polar numbers whenever we have magnitudes and +angles: + + (define (make-from-real-imag x y) + (make-from-real-imag-rectangular x y)) + + (define (make-from-mag-ang r a) + (make-from-mag-ang-polar r a)) + + *Figure 2.21:* Structure of the generic complex-arithmetic system. + + +--------------------------------------------------+ + ----| add-complex sub-complex mul-complex- div-complex |---- + +--------------------------------------------------+ + Complex arithmetic package + +-----------------------+ + | real-part imag-part | + -----------------| |------------------ + | magnitude angle | + +-----------+-----------+ + Rectangular | Polar + representation | representation + -----------------------------+------------------------------ + List structure and primitive machine arithmetic + + The resulting complex-number system has the structure shown in *Note +Figure 2-21::. The system has been decomposed into three relatively +independent parts: the complex-number-arithmetic operations, Alyssa's +polar implementation, and Ben's rectangular implementation. The polar +and rectangular implementations could have been written by Ben and +Alyssa working separately, and both of these can be used as underlying +representations by a third programmer implementing the +complex-arithmetic procedures in terms of the abstract +constructor/selector interface. + + Since each data object is tagged with its type, the selectors +operate on the data in a generic manner. That is, each selector is +defined to have a behavior that depends upon the particular type of +data it is applied to. Notice the general mechanism for interfacing +the separate representations: Within a given representation +implementation (say, Alyssa's polar package) a complex number is an +untyped pair (magnitude, angle). When a generic selector operates on a +number of `polar' type, it strips off the tag and passes the contents on +to Alyssa's code. Conversely, when Alyssa constructs a number for +general use, she tags it with a type so that it can be appropriately +recognized by the higher-level procedures. This discipline of +stripping off and attaching tags as data objects are passed from level +to level can be an important organizational strategy, as we shall see +in section *Note 2-5::. + + +File: sicp.info, Node: 2-4-3, Prev: 2-4-2, Up: 2-4 + +2.4.3 Data-Directed Programming and Additivity +---------------------------------------------- + +The general strategy of checking the type of a datum and calling an +appropriate procedure is called "dispatching on type". This is a +powerful strategy for obtaining modularity in system design. Oh the +other hand, implementing the dispatch as in section *Note 2-4-2:: has +two significant weaknesses. One weakness is that the generic interface +procedures (`real-part', `imag-part', `magnitude', and `angle') must +know about all the different representations. For instance, suppose we +wanted to incorporate a new representation for complex numbers into our +complex-number system. We would need to identify this new +representation with a type, and then add a clause to each of the +generic interface procedures to check for the new type and apply the +appropriate selector for that representation. + + Another weakness of the technique is that even though the individual +representations can be designed separately, we must guarantee that no +two procedures in the entire system have the same name. This is why +Ben and Alyssa had to change the names of their original procedures +from section *Note 2-4-1::. + + The issue underlying both of these weaknesses is that the technique +for implementing generic interfaces is not "additive". The person +implementing the generic selector procedures must modify those +procedures each time a new representation is installed, and the people +interfacing the individual representations must modify their code to +avoid name conflicts. In each of these cases, the changes that must be +made to the code are straightforward, but they must be made +nonetheless, and this is a source of inconvenience and error. This is +not much of a problem for the complex-number system as it stands, but +suppose there were not two but hundreds of different representations +for complex numbers. And suppose that there were many generic +selectors to be maintained in the abstract-data interface. Suppose, in +fact, that no one programmer knew all the interface procedures or all +the representations. The problem is real and must be addressed in such +programs as large-scale data-base-management systems. + + What we need is a means for modularizing the system design even +further. This is provided by the programming technique known as programming +"data-directed programming". To understand how data-directed +programming works, begin with the observation that whenever we deal +with a set of generic operations that are common to a set of different +types we are, in effect, dealing with a two-dimensional table that +contains the possible operations on one axis and the possible types on +the other axis. The entries in the table are the procedures that +implement each operation for each type of argument presented. In the +complex-number system developed in the previous section, the +correspondence between operation name, data type, and actual procedure +was spread out among the various conditional clauses in the generic +interface procedures. But the same information could have been +organized in a table, as shown in *Note Figure 2-22::. + + Data-directed programming is the technique of designing programs to +work with such a table directly. Previously, we implemented the +mechanism that interfaces the complex-arithmetic code with the two +representation packages as a set of procedures that each perform an +explicit dispatch on type. Here we will implement the interface as a +single procedure that looks up the combination of the operation name +and argument type in the table to find the correct procedure to apply, +and then applies it to the contents of the argument. If we do this, +then to add a new representation package to the system we need not +change any existing procedures; we need only add new entries to the +table. + + *Figure 2.22:* Table of operations for the complex-number system. + + | Types + Operations | Polar | Rectangular + ===========+=================+====================== + real-part | real-part-polar | real-part-rectangular + imag-part | imag-part-polar | imag-part-rectangular + magnitude | magnitude-polar | magnitude-rectangular + angle | angle-polar | angle-rectangular + + To implement this plan, assume that we have two procedures, `put' and +`get', for manipulating the operation-and-type table: + + * `(put )' installs the `' in the table, + indexed by the `' and the `'. + + * `(get )' looks up the `', `' entry in the + table and returns the item found there. If no item is found, + `get' returns false. + + + For now, we can assume that `put' and `get' are included in our +language. In *Note Chapter 3:: (section *Note 3-3-3::, *Note Exercise +3-24::) we will see how to implement these and other operations for +manipulating tables. + + Here is how data-directed programming can be used in the +complex-number system. Ben, who developed the rectangular +representation, implements his code just as he did originally. He +defines a collection of procedures, or a "package", and interfaces +these to the rest of the system by adding entries to the table that +tell the system how to operate on rectangular numbers. This is +accomplished by calling the following procedure: + + (define (install-rectangular-package) + ;; internal procedures + (define (real-part z) (car z)) + (define (imag-part z) (cdr z)) + (define (make-from-real-imag x y) (cons x y)) + (define (magnitude z) + (sqrt (+ (square (real-part z)) + (square (imag-part z))))) + (define (angle z) + (atan (imag-part z) (real-part z))) + (define (make-from-mag-ang r a) + (cons (* r (cos a)) (* r (sin a)))) + + ;; interface to the rest of the system + (define (tag x) (attach-tag 'rectangular x)) + (put 'real-part '(rectangular) real-part) + (put 'imag-part '(rectangular) imag-part) + (put 'magnitude '(rectangular) magnitude) + (put 'angle '(rectangular) angle) + (put 'make-from-real-imag 'rectangular + (lambda (x y) (tag (make-from-real-imag x y)))) + (put 'make-from-mag-ang 'rectangular + (lambda (r a) (tag (make-from-mag-ang r a)))) + 'done) + + Notice that the internal procedures here are the same procedures +from section *Note 2-4-1:: that Ben wrote when he was working in +isolation. No changes are necessary in order to interface them to the +rest of the system. Moreover, since these procedure definitions are +internal to the installation procedure, Ben needn't worry about name +conflicts with other procedures outside the rectangular package. To +interface these to the rest of the system, Ben installs his `real-part' +procedure under the operation name `real-part' and the type +`(rectangular)', and similarly for the other selectors.(1) The +interface also defines the constructors to be used by the external +system.(2) These are identical to Ben's internally defined +constructors, except that they attach the tag. + + Alyssa's polar package is analogous: + + (define (install-polar-package) + ;; internal procedures + (define (magnitude z) (car z)) + (define (angle z) (cdr z)) + (define (make-from-mag-ang r a) (cons r a)) + (define (real-part z) + (* (magnitude z) (cos (angle z)))) + (define (imag-part z) + (* (magnitude z) (sin (angle z)))) + (define (make-from-real-imag x y) + (cons (sqrt (+ (square x) (square y))) + (atan y x))) + + ;; interface to the rest of the system + (define (tag x) (attach-tag 'polar x)) + (put 'real-part '(polar) real-part) + (put 'imag-part '(polar) imag-part) + (put 'magnitude '(polar) magnitude) + (put 'angle '(polar) angle) + (put 'make-from-real-imag 'polar + (lambda (x y) (tag (make-from-real-imag x y)))) + (put 'make-from-mag-ang 'polar + (lambda (r a) (tag (make-from-mag-ang r a)))) + 'done) + + Even though Ben and Alyssa both still use their original procedures +defined with the same names as each other's (e.g., `real-part'), these +definitions are now internal to different procedures (see section *Note +1-1-8::), so there is no name conflict. + + The complex-arithmetic selectors access the table by means of a +general "operation" procedure called `apply-generic', which applies a +generic operation to some arguments. `Apply-generic' looks in the +table under the name of the operation and the types of the arguments +and applies the resulting procedure if one is present:(3) + + (define (apply-generic op . args) + (let ((type-tags (map type-tag args))) + (let ((proc (get op type-tags))) + (if proc + (apply proc (map contents args)) + (error + "No method for these types -- APPLY-GENERIC" + (list op type-tags)))))) + + Using `apply-generic', we can define our generic selectors as +follows: + + (define (real-part z) (apply-generic 'real-part z)) + (define (imag-part z) (apply-generic 'imag-part z)) + (define (magnitude z) (apply-generic 'magnitude z)) + (define (angle z) (apply-generic 'angle z)) + + Observe that these do not change at all if a new representation is +added to the system. + + We can also extract from the table the constructors to be used by +the programs external to the packages in making complex numbers from +real and imaginary parts and from magnitudes and angles. As in section +*Note 2-4-2::, we construct rectangular numbers whenever we have real +and imaginary parts, and polar numbers whenever we have magnitudes and +angles: + + (define (make-from-real-imag x y) + ((get 'make-from-real-imag 'rectangular) x y)) + + (define (make-from-mag-ang r a) + ((get 'make-from-mag-ang 'polar) r a)) + + *Exercise 2.73:* Section *Note 2-3-2:: described a program that + performs symbolic differentiation: + + (define (deriv exp var) + (cond ((number? exp) 0) + ((variable? exp) (if (same-variable? exp var) 1 0)) + ((sum? exp) + (make-sum (deriv (addend exp) var) + (deriv (augend exp) var))) + ((product? exp) + (make-sum + (make-product (multiplier exp) + (deriv (multiplicand exp) var)) + (make-product (deriv (multiplier exp) var) + (multiplicand exp)))) + + (else (error "unknown expression type -- DERIV" exp)))) + + We can regard this program as performing a dispatch on the type of + the expression to be differentiated. In this situation the "type + tag" of the datum is the algebraic operator symbol (such as `+') + and the operation being performed is `deriv'. We can transform + this program into data-directed style by rewriting the basic + derivative procedure as + + (define (deriv exp var) + (cond ((number? exp) 0) + ((variable? exp) (if (same-variable? exp var) 1 0)) + (else ((get 'deriv (operator exp)) (operands exp) + var)))) + + (define (operator exp) (car exp)) + + (define (operands exp) (cdr exp)) + + a. Explain what was done above. Why can't we assimilate the + predicates `number?' and `same-variable?' into the + data-directed dispatch? + + b. Write the procedures for derivatives of sums and products, + and the auxiliary code required to install them in the table + used by the program above. + + c. Choose any additional differentiation rule that you like, + such as the one for exponents (*Note Exercise 2-56::), and + install it in this data-directed system. + + d. In this simple algebraic manipulator the type of an + expression is the algebraic operator that binds it together. + Suppose, however, we indexed the procedures in the opposite + way, so that the dispatch line in `deriv' looked like + + ((get (operator exp) 'deriv) (operands exp) var) + + What corresponding changes to the derivative system are + required? + + + *Exercise 2.74:* Insatiable Enterprises, Inc., is a highly + decentralized conglomerate company consisting of a large number of + independent divisions located all over the world. The company's + computer facilities have just been interconnected by means of a + clever network-interfacing scheme that makes the entire network + appear to any user to be a single computer. Insatiable's + president, in her first attempt to exploit the ability of the + network to extract administrative information from division files, + is dismayed to discover that, although all the division files have + been implemented as data structures in Scheme, the particular data + structure used varies from division to division. A meeting of + division managers is hastily called to search for a strategy to + integrate the files that will satisfy headquarters' needs while + preserving the existing autonomy of the divisions. + + Show how such a strategy can be implemented with data-directed + programming. As an example, suppose that each division's + personnel records consist of a single file, which contains a set + of records keyed on employees' names. The structure of the set + varies from division to division. Furthermore, each employee's + record is itself a set (structured differently from division to + division) that contains information keyed under identifiers such + as `address' and `salary'. In particular: + + a. Implement for headquarters a `get-record' procedure that + retrieves a specified employee's record from a specified + personnel file. The procedure should be applicable to any + division's file. Explain how the individual divisions' files + should be structured. In particular, what type information + must be supplied? + + b. Implement for headquarters a `get-salary' procedure that + returns the salary information from a given employee's record + from any division's personnel file. How should the record be + structured in order to make this operation work? + + c. Implement for headquarters a `find-employee-record' + procedure. This should search all the divisions' files for + the record of a given employee and return the record. Assume + that this procedure takes as arguments an employee's name and + a list of all the divisions' files. + + d. When Insatiable takes over a new company, what changes must + be made in order to incorporate the new personnel information + into the central system? + + +Message passing +............... + +The key idea of data-directed programming is to handle generic +operations in programs by dealing explicitly with operation-and-type +tables, such as the table in *Note Figure 2-22::. The style of +programming we used in section *Note 2-4-2:: organized the required +dispatching on type by having each operation take care of its own +dispatching. In effect, this decomposes the operation-and-type table +into rows, with each generic operation procedure representing a row of +the table. + + An alternative implementation strategy is to decompose the table +into columns and, instead of using "intelligent operations" that +dispatch on data types, to work with "intelligent data objects" that +dispatch on operation names. We can do this by arranging things so +that a data object, such as a rectangular number, is represented as a +procedure that takes as input the required operation name and performs +the operation indicated. In such a discipline, `make-from-real-imag' +could be written as + + (define (make-from-real-imag x y) + (define (dispatch op) + (cond ((eq? op 'real-part) x) + ((eq? op 'imag-part) y) + ((eq? op 'magnitude) + (sqrt (+ (square x) (square y)))) + ((eq? op 'angle) (atan y x)) + (else + (error "Unknown op -- MAKE-FROM-REAL-IMAG" op)))) + dispatch) + + The corresponding `apply-generic' procedure, which applies a generic +operation to an argument, now simply feeds the operation's name to the +data object and lets the object do the work:(4) + + (define (apply-generic op arg) (arg op)) + + Note that the value returned by `make-from-real-imag' is a +procedure--the internal `dispatch' procedure. This is the procedure +that is invoked when `apply-generic' requests an operation to be +performed. + + This style of programming is called "message passing". The name +comes from the image that a data object is an entity that receives the +requested operation name as a "message." We have already seen an +example of message passing in section *Note 2-1-3::, where we saw how +`cons', `car', and `cdr' could be defined with no data objects but only +procedures. Here we see that message passing is not a mathematical +trick but a useful technique for organizing systems with generic +operations. In the remainder of this chapter we will continue to use +data-directed programming, rather than message passing, to discuss +generic arithmetic operations. In *Note Chapter 3:: we will return to +message passing, and we will see that it can be a powerful tool for +structuring simulation programs. + + *Exercise 2.75:* Implement the constructor `make-from-mag-ang' in + message-passing style. This procedure should be analogous to the + `make-from-real-imag' procedure given above. + + *Exercise 2.76:* As a large system with generic operations + evolves, new types of data objects or new operations may be needed. + For each of the three strategies--generic operations with explicit + dispatch, data-directed style, and message-passing-style--describe + the changes that must be made to a system in order to add new + types or new operations. Which organization would be most + appropriate for a system in which new types must often be added? + Which would be most appropriate for a system in which new + operations must often be added? + + ---------- Footnotes ---------- + + (1) We use the list `(rectangular)' rather than the symbol +`rectangular' to allow for the possibility of operations with multiple +arguments, not all of the same type. + + (2) The type the constructors are installed under needn't be a list +because a constructor is always used to make an object of one +particular type. + + (3) `Apply-generic' uses the dotted-tail notation described in *Note +Exercise 2-20::, because different generic operations may take +different numbers of arguments. In `apply-generic', `op' has as its +value the first argument to `apply-generic' and `args' has as its value +a list of the remaining arguments. + + `Apply-generic' also uses the primitive procedure `apply', which +takes two arguments, a procedure and a list. `Apply' applies the +procedure, using the elements in the list as arguments. For example, + + (apply + (list 1 2 3 4)) + +returns 10. + + (4) One limitation of this organization is it permits only generic +procedures of one argument. + + +File: sicp.info, Node: 2-5, Prev: 2-4, Up: Chapter 2 + +2.5 Systems with Generic Operations +=================================== + +In the previous section, we saw how to design systems in which data +objects can be represented in more than one way. The key idea is to +link the code that specifies the data operations to the several +representations by means of generic interface procedures. Now we will +see how to use this same idea not only to define operations that are +generic over different representations but also to define operations +that are generic over different kinds of arguments. We have already +seen several different packages of arithmetic operations: the primitive +arithmetic (`+', `-', `*', `/') built into our language, the +rational-number arithmetic (`add-rat', `sub-rat', `mul-rat', `div-rat') +of section *Note 2-1-1::, and the complex-number arithmetic that we +implemented in section *Note 2-4-3::. We will now use data-directed +techniques to construct a package of arithmetic operations that +incorporates all the arithmetic packages we have already constructed. + + *Note Figure 2-23:: shows the structure of the system we shall +build. Notice the abstraction barriers. From the perspective of +someone using "numbers," there is a single procedure `add' that +operates on whatever numbers are supplied. `Add' is part of a generic +interface that allows the separate ordinary-arithmetic, +rational-arithmetic, and complex-arithmetic packages to be accessed +uniformly by programs that use numbers. Any individual arithmetic +package (such as the complex package) may itself be accessed through +generic procedures (such as `add-complex') that combine packages +designed for different representations (such as rectangular and polar). +Moreover, the structure of the system is additive, so that one can +design the individual arithmetic packages separately and combine them +to produce a generic arithmetic system. + + *Figure 2.23:* Generic arithmetic system. + + Programs that use numbers + +-----------------+ + ---------------------------| add sub mul div |------------------- + +-----------------+ + Generic arithmetic package + +-----------------+ +-------------------------+ + | add-rat sub-rat | | add-complex sub-complex | +---------+ + -| |-+-| |-+-| + - * / |- + | mul-rat div-rat | | | mul-complex div-complex | | +---------+ + +-----------------+ | +-------------------------+ | + Rational | Complex artithmetic | Ordinary + arithmetic +--------------+--------------+ arithmetic + | Rectangular | Polar | + ---------------------+--------------+--------------+------------- + +* Menu: + +* 2-5-1:: Generic Arithmetic Operations +* 2-5-2:: Combining Data of Different Types +* 2-5-3:: Example: Symbolic Algebra + + +File: sicp.info, Node: 2-5-1, Next: 2-5-2, Prev: 2-5, Up: 2-5 + +2.5.1 Generic Arithmetic Operations +----------------------------------- + +The task of designing generic arithmetic operations is analogous to +that of designing the generic complex-number operations. We would +like, for instance, to have a generic addition procedure `add' that +acts like ordinary primitive addition `+' on ordinary numbers, like +`add-rat' on rational numbers, and like `add-complex' on complex +numbers. We can implement `add', and the other generic arithmetic +operations, by following the same strategy we used in section *Note +2-4-3:: to implement the generic selectors for complex numbers. We +will attach a type tag to each kind of number and cause the generic +procedure to dispatch to an appropriate package according to the data +type of its arguments. + + The generic arithmetic procedures are defined as follows: + + (define (add x y) (apply-generic 'add x y)) + (define (sub x y) (apply-generic 'sub x y)) + (define (mul x y) (apply-generic 'mul x y)) + (define (div x y) (apply-generic 'div x y)) + + We begin by installing a package for handling "ordinary" numbers, +that is, the primitive numbers of our language. We will tag these with +the symbol `scheme-number'. The arithmetic operations in this package +are the primitive arithmetic procedures (so there is no need to define +extra procedures to handle the untagged numbers). Since these +operations each take two arguments, they are installed in the table +keyed by the list `(scheme-number scheme-number)': + + (define (install-scheme-number-package) + (define (tag x) + (attach-tag 'scheme-number x)) + (put 'add '(scheme-number scheme-number) + (lambda (x y) (tag (+ x y)))) + (put 'sub '(scheme-number scheme-number) + (lambda (x y) (tag (- x y)))) + (put 'mul '(scheme-number scheme-number) + (lambda (x y) (tag (* x y)))) + (put 'div '(scheme-number scheme-number) + (lambda (x y) (tag (/ x y)))) + (put 'make 'scheme-number + (lambda (x) (tag x))) + 'done) + + Users of the Scheme-number package will create (tagged) ordinary +numbers by means of the procedure: + + (define (make-scheme-number n) + ((get 'make 'scheme-number) n)) + + Now that the framework of the generic arithmetic system is in place, +we can readily include new kinds of numbers. Here is a package that +performs rational arithmetic. Notice that, as a benefit of additivity, +we can use without modification the rational-number code from section +*Note 2-1-1:: as the internal procedures in the package: + + (define (install-rational-package) + ;; internal procedures + (define (numer x) (car x)) + (define (denom x) (cdr x)) + (define (make-rat n d) + (let ((g (gcd n d))) + (cons (/ n g) (/ d g)))) + (define (add-rat x y) + (make-rat (+ (* (numer x) (denom y)) + (* (numer y) (denom x))) + (* (denom x) (denom y)))) + (define (sub-rat x y) + (make-rat (- (* (numer x) (denom y)) + (* (numer y) (denom x))) + (* (denom x) (denom y)))) + (define (mul-rat x y) + (make-rat (* (numer x) (numer y)) + (* (denom x) (denom y)))) + (define (div-rat x y) + (make-rat (* (numer x) (denom y)) + (* (denom x) (numer y)))) + + ;; interface to rest of the system + (define (tag x) (attach-tag 'rational x)) + (put 'add '(rational rational) + (lambda (x y) (tag (add-rat x y)))) + (put 'sub '(rational rational) + (lambda (x y) (tag (sub-rat x y)))) + (put 'mul '(rational rational) + (lambda (x y) (tag (mul-rat x y)))) + (put 'div '(rational rational) + (lambda (x y) (tag (div-rat x y)))) + + (put 'make 'rational + (lambda (n d) (tag (make-rat n d)))) + 'done) + + (define (make-rational n d) + ((get 'make 'rational) n d)) + + We can install a similar package to handle complex numbers, using +the tag `complex'. In creating the package, we extract from the table +the operations `make-from-real-imag' and `make-from-mag-ang' that were +defined by the rectangular and polar packages. Additivity permits us +to use, as the internal operations, the same `add-complex', +`sub-complex', `mul-complex', and `div-complex' procedures from section +*Note 2-4-1::. + + (define (install-complex-package) + ;; imported procedures from rectangular and polar packages + (define (make-from-real-imag x y) + ((get 'make-from-real-imag 'rectangular) x y)) + (define (make-from-mag-ang r a) + ((get 'make-from-mag-ang 'polar) r a)) + + ;; internal procedures + (define (add-complex z1 z2) + (make-from-real-imag (+ (real-part z1) (real-part z2)) + (+ (imag-part z1) (imag-part z2)))) + (define (sub-complex z1 z2) + (make-from-real-imag (- (real-part z1) (real-part z2)) + (- (imag-part z1) (imag-part z2)))) + (define (mul-complex z1 z2) + (make-from-mag-ang (* (magnitude z1) (magnitude z2)) + (+ (angle z1) (angle z2)))) + (define (div-complex z1 z2) + (make-from-mag-ang (/ (magnitude z1) (magnitude z2)) + (- (angle z1) (angle z2)))) + + ;; interface to rest of the system + (define (tag z) (attach-tag 'complex z)) + (put 'add '(complex complex) + (lambda (z1 z2) (tag (add-complex z1 z2)))) + (put 'sub '(complex complex) + (lambda (z1 z2) (tag (sub-complex z1 z2)))) + (put 'mul '(complex complex) + (lambda (z1 z2) (tag (mul-complex z1 z2)))) + (put 'div '(complex complex) + (lambda (z1 z2) (tag (div-complex z1 z2)))) + (put 'make-from-real-imag 'complex + (lambda (x y) (tag (make-from-real-imag x y)))) + (put 'make-from-mag-ang 'complex + (lambda (r a) (tag (make-from-mag-ang r a)))) + 'done) + + Programs outside the complex-number package can construct complex +numbers either from real and imaginary parts or from magnitudes and +angles. Notice how the underlying procedures, originally defined in +the rectangular and polar packages, are exported to the complex +package, and exported from there to the outside world. + + (define (make-complex-from-real-imag x y) + ((get 'make-from-real-imag 'complex) x y)) + + (define (make-complex-from-mag-ang r a) + ((get 'make-from-mag-ang 'complex) r a)) + + What we have here is a two-level tag system. A typical complex +number, such as 3 + 4i in rectangular form, would be represented as +shown in *Note Figure 2-24::. The outer tag (`complex') is used to +direct the number to the complex package. Once within the complex +package, the next tag (`rectangular') is used to direct the number to +the rectangular package. In a large and complicated system there might +be many levels, each interfaced with the next by means of generic +operations. As a data object is passed "downward," the outer tag that +is used to direct it to the appropriate package is stripped off (by +applying `contents') and the next level of tag (if any) becomes visible +to be used for further dispatching. + + *Figure 2.24:* Representation of 3 + 4i in rectangular form. + + +---+---+ +---+---+ +---+---+ + ---->| * | *-+---->| * | *-+---->| * | * | + +-|-+---+ +-|-+---+ +-|-+-|-+ + | | | | + V V V V + +---------+ +-------------+ +---+ +---+ + | complex | | rectangular | | 3 | | 4 | + +---------+ +-------------+ +---+ +---+ + + In the above packages, we used `add-rat', `add-complex', and the +other arithmetic procedures exactly as originally written. Once these +definitions are internal to different installation procedures, however, +they no longer need names that are distinct from each other: we could +simply name them `add', `sub', `mul', and `div' in both packages. + + *Exercise 2.77:* Louis Reasoner tries to evaluate the expression + `(magnitude z)' where `z' is the object shown in *Note Figure + 2-24::. To his surprise, instead of the answer 5 he gets an error + message from `apply-generic', saying there is no method for the + operation `magnitude' on the types `(complex)'. He shows this + interaction to Alyssa P. Hacker, who says "The problem is that the + complex-number selectors were never defined for `complex' numbers, + just for `polar' and `rectangular' numbers. All you have to do to + make this work is add the following to the `complex' package:" + + (put 'real-part '(complex) real-part) + (put 'imag-part '(complex) imag-part) + (put 'magnitude '(complex) magnitude) + (put 'angle '(complex) angle) + + Describe in detail why this works. As an example, trace through + all the procedures called in evaluating the expression `(magnitude + z)' where `z' is the object shown in *Note Figure 2-24::. In + particular, how many times is `apply-generic' invoked? What + procedure is dispatched to in each case? + + *Exercise 2.78:* The internal procedures in the `scheme-number' + package are essentially nothing more than calls to the primitive + procedures `+', `-', etc. It was not possible to use the + primitives of the language directly because our type-tag system + requires that each data object have a type attached to it. In + fact, however, all Lisp implementations do have a type system, + which they use internally. Primitive predicates such as `symbol?' + and `number?' determine whether data objects have particular + types. Modify the definitions of `type-tag', `contents', and + `attach-tag' from section *Note 2-4-2:: so that our generic system + takes advantage of Scheme's internal type system. That is to say, + the system should work as before except that ordinary numbers + should be represented simply as Scheme numbers rather than as + pairs whose `car' is the symbol `scheme-number'. + + *Exercise 2.79:* Define a generic equality predicate `equ?' that + tests the equality of two numbers, and install it in the generic + arithmetic package. This operation should work for ordinary + numbers, rational numbers, and complex numbers. + + *Exercise 2.80:* Define a generic predicate `=zero?' that tests if + its argument is zero, and install it in the generic arithmetic + package. This operation should work for ordinary numbers, rational + numbers, and complex numbers. + + +File: sicp.info, Node: 2-5-2, Next: 2-5-3, Prev: 2-5-1, Up: 2-5 + +2.5.2 Combining Data of Different Types +--------------------------------------- + +We have seen how to define a unified arithmetic system that encompasses +ordinary numbers, complex numbers, rational numbers, and any other type +of number we might decide to invent, but we have ignored an important +issue. The operations we have defined so far treat the different data +types as being completely independent. Thus, there are separate +packages for adding, say, two ordinary numbers, or two complex numbers. +What we have not yet considered is the fact that it is meaningful to +define operations that cross the type boundaries, such as the addition +of a complex number to an ordinary number. We have gone to great pains +to introduce barriers between parts of our programs so that they can be +developed and understood separately. We would like to introduce the +cross-type operations in some carefully controlled way, so that we can +support them without seriously violating our module boundaries. + + One way to handle cross-type operations is to design a different +procedure for each possible combination of types for which the +operation is valid. For example, we could extend the complex-number +package so that it provides a procedure for adding complex numbers to +ordinary numbers and installs this in the table using the tag `(complex +scheme-number)':(1) + + ;; to be included in the complex package + (define (add-complex-to-schemenum z x) + (make-from-real-imag (+ (real-part z) x) + (imag-part z))) + + (put 'add '(complex scheme-number) + (lambda (z x) (tag (add-complex-to-schemenum z x)))) + + This technique works, but it is cumbersome. With such a system, the +cost of introducing a new type is not just the construction of the +package of procedures for that type but also the construction and +installation of the procedures that implement the cross-type +operations. This can easily be much more code than is needed to define +the operations on the type itself. The method also undermines our +ability to combine separate packages additively, or least to limit the +extent to which the implementors of the individual packages need to +take account of other packages. For instance, in the example above, it +seems reasonable that handling mixed operations on complex numbers and +ordinary numbers should be the responsibility of the complex-number +package. Combining rational numbers and complex numbers, however, +might be done by the complex package, by the rational package, or by +some third package that uses operations extracted from these two +packages. Formulating coherent policies on the division of +responsibility among packages can be an overwhelming task in designing +systems with many packages and many cross-type operations. + +Coercion +........ + +In the general situation of completely unrelated operations acting on +completely unrelated types, implementing explicit cross-type operations, +cumbersome though it may be, is the best that one can hope for. +Fortunately, we can usually do better by taking advantage of additional +structure that may be latent in our type system. Often the different +data types are not completely independent, and there may be ways by +which objects of one type may be viewed as being of another type. This +process is called "coercion". For example, if we are asked to +arithmetically combine an ordinary number with a complex number, we can +view the ordinary number as a complex number whose imaginary part is +zero. This transforms the problem to that of combining two complex +numbers, which can be handled in the ordinary way by the +complex-arithmetic package. + + In general, we can implement this idea by designing coercion +procedures that transform an object of one type into an equivalent +object of another type. Here is a typical coercion procedure, which +transforms a given ordinary number to a complex number with that real +part and zero imaginary part: + + (define (scheme-number->complex n) + (make-complex-from-real-imag (contents n) 0)) + + We install these coercion procedures in a special coercion table, +indexed under the names of the two types: + + (put-coercion 'scheme-number 'complex scheme-number->complex) + + (We assume that there are `put-coercion' and `get-coercion' +procedures available for manipulating this table.) Generally some of +the slots in the table will be empty, because it is not generally +possible to coerce an arbitrary data object of each type into all other +types. For example, there is no way to coerce an arbitrary complex +number to an ordinary number, so there will be no general +`complex->scheme-number' procedure included in the table. + + Once the coercion table has been set up, we can handle coercion in a +uniform manner by modifying the `apply-generic' procedure of section +*Note 2-4-3::. When asked to apply an operation, we first check +whether the operation is defined for the arguments' types, just as +before. If so, we dispatch to the procedure found in the +operation-and-type table. Otherwise, we try coercion. For simplicity, +we consider only the case where there are two arguments.(2) We check +the coercion table to see if objects of the first type can be coerced +to the second type. If so, we coerce the first argument and try the +operation again. If objects of the first type cannot in general be +coerced to the second type, we try the coercion the other way around to +see if there is a way to coerce the second argument to the type of the +first argument. Finally, if there is no known way to coerce either +type to the other type, we give up. Here is the procedure: + + (define (apply-generic op . args) + (let ((type-tags (map type-tag args))) + (let ((proc (get op type-tags))) + (if proc + (apply proc (map contents args)) + (if (= (length args) 2) + (let ((type1 (car type-tags)) + (type2 (cadr type-tags)) + (a1 (car args)) + (a2 (cadr args))) + (let ((t1->t2 (get-coercion type1 type2)) + (t2->t1 (get-coercion type2 type1))) + (cond (t1->t2 + (apply-generic op (t1->t2 a1) a2)) + (t2->t1 + (apply-generic op a1 (t2->t1 a2))) + (else + (error "No method for these types" + (list op type-tags)))))) + (error "No method for these types" + (list op type-tags))))))) + + This coercion scheme has many advantages over the method of defining +explicit cross-type operations, as outlined above. Although we still +need to write coercion procedures to relate the types (possibly n^2 +procedures for a system with n types), we need to write only one +procedure for each pair of types rather than a different procedure for +each collection of types and each generic operation.(3) What we are +counting on here is the fact that the appropriate transformation +between types depends only on the types themselves, not on the +operation to be applied. + + On the other hand, there may be applications for which our coercion +scheme is not general enough. Even when neither of the objects to be +combined can be converted to the type of the other it may still be +possible to perform the operation by converting both objects to a third +type. In order to deal with such complexity and still preserve +modularity in our programs, it is usually necessary to build systems +that take advantage of still further structure in the relations among +types, as we discuss next. + +Hierarchies of types +.................... + +The coercion scheme presented above relied on the existence of natural +relations between pairs of types. Often there is more "global" +structure in how the different types relate to each other. For +instance, suppose we are building a generic arithmetic system to handle +integers, rational numbers, real numbers, and complex numbers. In such +a system, it is quite natural to regard an integer as a special kind of +rational number, which is in turn a special kind of real number, which +is in turn a special kind of complex number. What we actually have is +a so-called "hierarchy of types", in which, for example, integers are a "subtype" +of rational numbers (i.e., any operation that can be applied to a +rational number can automatically be applied to an integer). +Conversely, we say that rational numbers form a "supertype" of +integers. The particular hierarchy we have here is of a very simple +kind, in which each type has at most one supertype and at most one +subtype. Such a structure, called a "tower", is illustrated in *Note +Figure 2-25::. + + *Figure 2.25:* A tower of types. + + complex + ^ + | + real + ^ + | + rational + ^ + | + integer + + If we have a tower structure, then we can greatly simplify the +problem of adding a new type to the hierarchy, for we need only specify +how the new type is embedded in the next supertype above it and how it +is the supertype of the type below it. For example, if we want to add +an integer to a complex number, we need not explicitly define a special +coercion procedure `integer->complex'. Instead, we define how an +integer can be transformed into a rational number, how a rational +number is transformed into a real number, and how a real number is +transformed into a complex number. We then allow the system to +transform the integer into a complex number through these steps and +then add the two complex numbers. + + We can redesign our `apply-generic' procedure in the following way: +For each type, we need to supply a `raise' procedure, which "raises" +objects of that type one level in the tower. Then when the system is +required to operate on objects of different types it can successively +raise the lower types until all the objects are at the same level in +the tower. (*Note Exercise 2-83:: and *Note Exercise 2-84:: concern +the details of implementing such a strategy.) + + Another advantage of a tower is that we can easily implement the +notion that every type "inherits" all operations defined on a +supertype. For instance, if we do not supply a special procedure for +finding the real part of an integer, we should nevertheless expect that +`real-part' will be defined for integers by virtue of the fact that +integers are a subtype of complex numbers. In a tower, we can arrange +for this to happen in a uniform way by modifying `apply-generic'. If +the required operation is not directly defined for the type of the +object given, we raise the object to its supertype and try again. We +thus crawl up the tower, transforming our argument as we go, until we +either find a level at which the desired operation can be performed or +hit the top (in which case we give up). + + Yet another advantage of a tower over a more general hierarchy is +that it gives us a simple way to "lower" a data object to the simplest +representation. For example, if we add 2 + 3i to 4 - 3i, it would be +nice to obtain the answer as the integer 6 rather than as the complex +number 6 + 0i. *Note Exercise 2-85:: discusses a way to implement such +a lowering operation. (The trick is that we need a general way to +distinguish those objects that can be lowered, such as 6 + 0i, from +those that cannot, such as 6 + 2i.) + + *Figure 2.26:* Relations among types of geometric figures. + + polygon + / \ + / \ + triangle quadrilateral + / \ / \ + / \ / \ + isosceles right trapezoid kite + triangle triangle | | + | \ | | | + | \ | | | + equilateral isosceles parallelogram | + triangle right | \ | + triangle | \ | + rectangle rhombus + \ / + \ / + square + +Inadequacies of hierarchies +........................... + +If the data types in our system can be naturally arranged in a tower, +this greatly simplifies the problems of dealing with generic operations +on different types, as we have seen. Unfortunately, this is usually +not the case. *Note Figure 2-26:: illustrates a more complex +arrangement of mixed types, this one showing relations among different +types of geometric figures. We see that, in general, a type may have +more than one subtype. Triangles and quadrilaterals, for instance, are +both subtypes of polygons. In addition, a type may have more than one +supertype. For example, an isosceles right triangle may be regarded +either as an isosceles triangle or as a right triangle. This +multiple-supertypes issue is particularly thorny, since it means that +there is no unique way to "raise" a type in the hierarchy. Finding the +"correct" supertype in which to apply an operation to an object may +involve considerable searching through the entire type network on the +part of a procedure such as `apply-generic'. Since there generally are +multiple subtypes for a type, there is a similar problem in coercing a +value "down" the type hierarchy. Dealing with large numbers of +interrelated types while still preserving modularity in the design of +large systems is very difficult, and is an area of much current +research.(4) + + *Exercise 2.81:* Louis Reasoner has noticed that `apply-generic' + may try to coerce the arguments to each other's type even if they + already have the same type. Therefore, he reasons, we need to put + procedures in the coercion table to "coerce" arguments of each + type to their own type. For example, in addition to the + `scheme-number->complex' coercion shown above, he would do: + + (define (scheme-number->scheme-number n) n) + (define (complex->complex z) z) + (put-coercion 'scheme-number 'scheme-number + scheme-number->scheme-number) + (put-coercion 'complex 'complex complex->complex) + + a. With Louis's coercion procedures installed, what happens if + `apply-generic' is called with two arguments of type + `scheme-number' or two arguments of type `complex' for an + operation that is not found in the table for those types? + For example, assume that we've defined a generic + exponentiation operation: + + (define (exp x y) (apply-generic 'exp x y)) + + and have put a procedure for exponentiation in the + Scheme-number package but not in any other package: + + ;; following added to Scheme-number package + (put 'exp '(scheme-number scheme-number) + (lambda (x y) (tag (expt x y)))) ; using primitive `expt' + + What happens if we call `exp' with two complex numbers as + arguments? + + b. Is Louis correct that something had to be done about coercion + with arguments of the same type, or does `apply-generic' work + correctly as is? + + c. Modify `apply-generic' so that it doesn't try coercion if the + two arguments have the same type. + + + *Exercise 2.82:* Show how to generalize `apply-generic' to handle + coercion in the general case of multiple arguments. One strategy + is to attempt to coerce all the arguments to the type of the first + argument, then to the type of the second argument, and so on. + Give an example of a situation where this strategy (and likewise + the two-argument version given above) is not sufficiently general. + (Hint: Consider the case where there are some suitable mixed-type + operations present in the table that will not be tried.) + + *Exercise 2.83:* Suppose you are designing a generic arithmetic + system for dealing with the tower of types shown in *Note Figure + 2-25::: integer, rational, real, complex. For each type (except + complex), design a procedure that raises objects of that type one + level in the tower. Show how to install a generic `raise' + operation that will work for each type (except complex). + + *Exercise 2.84:* Using the `raise' operation of *Note Exercise + 2-83::, modify the `apply-generic' procedure so that it coerces + its arguments to have the same type by the method of successive + raising, as discussed in this section. You will need to devise a + way to test which of two types is higher in the tower. Do this in + a manner that is "compatible" with the rest of the system and will + not lead to problems in adding new levels to the tower. + + *Exercise 2.85:* This section mentioned a method for "simplifying" + a data object by lowering it in the tower of types as far as + possible. Design a procedure `drop' that accomplishes this for the + tower described in *Note Exercise 2-83::. The key is to decide, + in some general way, whether an object can be lowered. For + example, the complex number 1.5 + 0i can be lowered as far as + `real', the complex number 1 + 0i can be lowered as far as + `integer', and the complex number 2 + 3i cannot be lowered at all. + Here is a plan for determining whether an object can be lowered: + Begin by defining a generic operation `project' that "pushes" an + object down in the tower. For example, projecting a complex + number would involve throwing away the imaginary part. Then a + number can be dropped if, when we `project' it and `raise' the + result back to the type we started with, we end up with something + equal to what we started with. Show how to implement this idea in + detail, by writing a `drop' procedure that drops an object as far + as possible. You will need to design the various projection + operations(5) and install `project' as a generic operation in the + system. You will also need to make use of a generic equality + predicate, such as described in *Note Exercise 2-79::. Finally, + use `drop' to rewrite `apply-generic' from *Note Exercise 2-84:: + so that it "simplifies" its answers. + + *Exercise 2.86:* Suppose we want to handle complex numbers whose + real parts, imaginary parts, magnitudes, and angles can be either + ordinary numbers, rational numbers, or other numbers we might wish + to add to the system. Describe and implement the changes to the + system needed to accommodate this. You will have to define + operations such as `sine' and `cosine' that are generic over + ordinary numbers and rational numbers. + + ---------- Footnotes ---------- + + (1) We also have to supply an almost identical procedure to handle +the types `(scheme-number complex)'. + + (2) See *Note Exercise 2-82:: for generalizations. + + (3) If we are clever, we can usually get by with fewer than n^2 +coercion procedures. For instance, if we know how to convert from type +1 to type 2 and from type 2 to type 3, then we can use this knowledge to +convert from type 1 to type 3. This can greatly decrease the number of +coercion procedures we need to supply explicitly when we add a new type +to the system. If we are willing to build the required amount of +sophistication into our system, we can have it search the "graph" of +relations among types and automatically generate those coercion +procedures that can be inferred from the ones that are supplied +explicitly. + + (4) This statement, which also appears in the first edition of this +book, is just as true now as it was when we wrote it twelve years ago. +Developing a useful, general framework for expressing the relations +among different types of entities (what philosophers call "ontology") +seems intractably difficult. The main difference between the confusion +that existed ten years ago and the confusion that exists now is that +now a variety of inadequate ontological theories have been embodied in +a plethora of correspondingly inadequate programming languages. For +example, much of the complexity of object-oriented programming +languages--and the subtle and confusing differences among contemporary +object-oriented languages--centers on the treatment of generic +operations on interrelated types. Our own discussion of computational +objects in *Note Chapter 3:: avoids these issues entirely. Readers +familiar with object-oriented programming will notice that we have much +to say in *Note Chapter 3:: about local state, but we do not even +mention "classes" or "inheritance." In fact, we suspect that these +problems cannot be adequately addressed in terms of computer-language +design alone, without also drawing on work in knowledge representation +and automated reasoning. + + (5) A real number can be projected to an integer using the `round' +primitive, which returns the closest integer to its argument. + + +File: sicp.info, Node: 2-5-3, Prev: 2-5-2, Up: 2-5 + +2.5.3 Example: Symbolic Algebra +------------------------------- + +The manipulation of symbolic algebraic expressions is a complex process +that illustrates many of the hardest problems that occur in the design +of large-scale systems. An algebraic expression, in general, can be +viewed as a hierarchical structure, a tree of operators applied to +operands. We can construct algebraic expressions by starting with a +set of primitive objects, such as constants and variables, and +combining these by means of algebraic operators, such as addition and +multiplication. As in other languages, we form abstractions that +enable us to refer to compound objects in simple terms. Typical +abstractions in symbolic algebra are ideas such as linear combination, +polynomial, rational function, or trigonometric function. We can +regard these as compound "types," which are often useful for directing +the processing of expressions. For example, we could describe the +expression + + x^2 sin (y^2 + 1) + r cos 2y + cos(y^3 - 2y^2) + +as a polynomial in x with coefficients that are trigonometric functions +of polynomials in y whose coefficients are integers. + + We will not attempt to develop a complete algebraic-manipulation +system here. Such systems are exceedingly complex programs, embodying +deep algebraic knowledge and elegant algorithms. What we will do is +look at a simple but important part of algebraic manipulation: the +arithmetic of polynomials. We will illustrate the kinds of decisions +the designer of such a system faces, and how to apply the ideas of +abstract data and generic operations to help organize this effort. + +Arithmetic on polynomials +......................... + +Our first task in designing a system for performing arithmetic on +polynomials is to decide just what a polynomial is. Polynomials are +normally defined relative to certain variables (the "indeterminates" of +the polynomial). For simplicity, we will restrict ourselves to +polynomials having just one indeterminate ("univariate +polynomials").(1) We will define a polynomial to be a sum of terms, +each of which is either a coefficient, a power of the indeterminate, or +a product of a coefficient and a power of the indeterminate. A +coefficient is defined as an algebraic expression that is not dependent +upon the indeterminate of the polynomial. For example, + + 5x^2 + 3r + 7 + +is a simple polynomial in x, and + + (y^2 + 1)r^3 + (2y)x + 1 + +is a polynomial in x whose coefficients are polynomials in y. + + Already we are skirting some thorny issues. Is the first of these +polynomials the same as the polynomial 5y^2 + 3y + 7, or not? A +reasonable answer might be "yes, if we are considering a polynomial +purely as a mathematical function, but no, if we are considering a +polynomial to be a syntactic form." The second polynomial is +algebraically equivalent to a polynomial in y whose coefficients are +polynomials in x. Should our system recognize this, or not? +Furthermore, there are other ways to represent a polynomial--for +example, as a product of factors, or (for a univariate polynomial) as +the set of roots, or as a listing of the values of the polynomial at a +specified set of points.(2) We can finesse these questions by deciding +that in our algebraic-manipulation system a "polynomial" will be a +particular syntactic form, not its underlying mathematical meaning. + + Now we must consider how to go about doing arithmetic on +polynomials. In this simple system, we will consider only addition and +multiplication. Moreover, we will insist that two polynomials to be +combined must have the same indeterminate. + + We will approach the design of our system by following the familiar +discipline of data abstraction. We will represent polynomials using a +data structure called a "poly", which consists of a variable and a +collection of terms. We assume that we have selectors `variable' and +`term-list' that extract those parts from a poly and a constructor +`make-poly' that assembles a poly from a given variable and a term +list. A variable will be just a symbol, so we can use the +`same-variable?' procedure of section *Note 2-3-2:: to compare +variables. The following procedures define addition and multiplication +of polys: + + (define (add-poly p1 p2) + (if (same-variable? (variable p1) (variable p2)) + (make-poly (variable p1) + (add-terms (term-list p1) + (term-list p2))) + (error "Polys not in same var -- ADD-POLY" + (list p1 p2)))) + + (define (mul-poly p1 p2) + (if (same-variable? (variable p1) (variable p2)) + (make-poly (variable p1) + (mul-terms (term-list p1) + (term-list p2))) + (error "Polys not in same var -- MUL-POLY" + (list p1 p2)))) + + To incorporate polynomials into our generic arithmetic system, we +need to supply them with type tags. We'll use the tag `polynomial', +and install appropriate operations on tagged polynomials in the +operation table. We'll embed all our code in an installation procedure +for the polynomial package, similar to the ones in section *Note +2-5-1::: + + (define (install-polynomial-package) + ;; internal procedures + ;; representation of poly + (define (make-poly variable term-list) + (cons variable term-list)) + (define (variable p) (car p)) + (define (term-list p) (cdr p)) + <_procedures `same-variable?' and `variable?' from section 2.3.2_> + + ;; representation of terms and term lists + <_procedures `adjoin-term' ... `coeff' from text below_> + + ;; continued on next page + + (define (add-poly p1 p2) ...) + <_procedures used by `add-poly'_> + (define (mul-poly p1 p2) ...) + <_procedures used by `mul-poly'_> + + ;; interface to rest of the system + (define (tag p) (attach-tag 'polynomial p)) + (put 'add '(polynomial polynomial) + (lambda (p1 p2) (tag (add-poly p1 p2)))) + (put 'mul '(polynomial polynomial) + (lambda (p1 p2) (tag (mul-poly p1 p2)))) + (put 'make 'polynomial + (lambda (var terms) (tag (make-poly var terms)))) + 'done) + + Polynomial addition is performed termwise. Terms of the same order +(i.e., with the same power of the indeterminate) must be combined. +This is done by forming a new term of the same order whose coefficient +is the sum of the coefficients of the addends. Terms in one addend for +which there are no terms of the same order in the other addend are +simply accumulated into the sum polynomial being constructed. + + In order to manipulate term lists, we will assume that we have a +constructor `the-empty-termlist' that returns an empty term list and a +constructor `adjoin-term' that adjoins a new term to a term list. We +will also assume that we have a predicate `empty-termlist?' that tells +if a given term list is empty, a selector `first-term' that extracts +the highest-order term from a term list, and a selector `rest-terms' +that returns all but the highest-order term. To manipulate terms, we +will suppose that we have a constructor `make-term' that constructs a +term with given order and coefficient, and selectors `order' and +`coeff' that return, respectively, the order and the coefficient of the +term. These operations allow us to consider both terms and term lists +as data abstractions, whose concrete representations we can worry about +separately. + + Here is the procedure that constructs the term list for the sum of +two polynomials:(3) + + (define (add-terms L1 L2) + (cond ((empty-termlist? L1) L2) + ((empty-termlist? L2) L1) + (else + (let ((t1 (first-term L1)) (t2 (first-term L2))) + (cond ((> (order t1) (order t2)) + (adjoin-term + t1 (add-terms (rest-terms L1) L2))) + ((< (order t1) (order t2)) + (adjoin-term + t2 (add-terms L1 (rest-terms L2)))) + (else + (adjoin-term + (make-term (order t1) + (add (coeff t1) (coeff t2))) + (add-terms (rest-terms L1) + (rest-terms L2))))))))) + + The most important point to note here is that we used the generic +addition procedure `add' to add together the coefficients of the terms +being combined. This has powerful consequences, as we will see below. + + In order to multiply two term lists, we multiply each term of the +first list by all the terms of the other list, repeatedly using +`mul-term-by-all-terms', which multiplies a given term by all terms in +a given term list. The resulting term lists (one for each term of the +first list) are accumulated into a sum. Multiplying two terms forms a +term whose order is the sum of the orders of the factors and whose +coefficient is the product of the coefficients of the factors: + + (define (mul-terms L1 L2) + (if (empty-termlist? L1) + (the-empty-termlist) + (add-terms (mul-term-by-all-terms (first-term L1) L2) + (mul-terms (rest-terms L1) L2)))) + + (define (mul-term-by-all-terms t1 L) + (if (empty-termlist? L) + (the-empty-termlist) + (let ((t2 (first-term L))) + (adjoin-term + (make-term (+ (order t1) (order t2)) + (mul (coeff t1) (coeff t2))) + (mul-term-by-all-terms t1 (rest-terms L)))))) + + This is really all there is to polynomial addition and +multiplication. Notice that, since we operate on terms using the +generic procedures `add' and `mul', our polynomial package is +automatically able to handle any type of coefficient that is known +about by the generic arithmetic package. If we include a coercion +mechanism such as one of those discussed in section *Note 2-5-2::, then +we also are automatically able to handle operations on polynomials of +different coefficient types, such as + + / 2 \ + [3x^2 + (2 + 3i)x + 7] * | x^4 + --- x^2 + (5 + 3i) | + \ 3 / + + Because we installed the polynomial addition and multiplication +procedures `add-poly' and `mul-poly' in the generic arithmetic system +as the `add' and `mul' operations for type `polynomial', our system is +also automatically able to handle polynomial operations such as + + [(y + 1)x^2 + (y^2 + 1)x + (y - 1)] * [(y - 2)x + (y^3 + 7)] + + The reason is that when the system tries to combine coefficients, it +will dispatch through `add' and `mul'. Since the coefficients are +themselves polynomials (in y), these will be combined using `add-poly' +and `mul-poly'. The result is a kind of "data-directed recursion" in +which, for example, a call to `mul-poly' will result in recursive calls +to `mul-poly' in order to multiply the coefficients. If the +coefficients of the coefficients were themselves polynomials (as might +be used to represent polynomials in three variables), the data +direction would ensure that the system would follow through another +level of recursive calls, and so on through as many levels as the +structure of the data dictates.(4) + +Representing term lists +....................... + +Finally, we must confront the job of implementing a good representation +for term lists. A term list is, in effect, a set of coefficients keyed +by the order of the term. Hence, any of the methods for representing +sets, as discussed in section *Note 2-3-3::, can be applied to this +task. On the other hand, our procedures `add-terms' and `mul-terms' +always access term lists sequentially from highest to lowest order. +Thus, we will use some kind of ordered list representation. + + How should we structure the list that represents a term list? One +consideration is the "density" of the polynomials we intend to +manipulate. A polynomial is said to be "dense" if it has nonzero +coefficients in terms of most orders. If it has many zero terms it is +said to be "sparse". For example, + + A : x^5 + 2x^4 + 3x^2 - 2x - 5 + +is a dense polynomial, whereas + + B : x^100 + 2x^2 + 1 + +is sparse. + + The term lists of dense polynomials are most efficiently represented +as lists of the coefficients. For example, A above would be nicely +represented as `(1 2 0 3 -2 -5)'. The order of a term in this +representation is the length of the sublist beginning with that term's +coefficient, decremented by 1.(5) This would be a terrible +representation for a sparse polynomial such as B: There would be a +giant list of zeros punctuated by a few lonely nonzero terms. A more +reasonable representation of the term list of a sparse polynomial is as +a list of the nonzero terms, where each term is a list containing the +order of the term and the coefficient for that order. In such a +scheme, polynomial B is efficiently represented as `((100 1) (2 2) (0 +1))'. As most polynomial manipulations are performed on sparse +polynomials, we will use this method. We will assume that term lists +are represented as lists of terms, arranged from highest-order to +lowest-order term. Once we have made this decision, implementing the +selectors and constructors for terms and term lists is +straightforward:(6) + + (define (adjoin-term term term-list) + (if (=zero? (coeff term)) + term-list + (cons term term-list))) + + (define (the-empty-termlist) '()) + (define (first-term term-list) (car term-list)) + (define (rest-terms term-list) (cdr term-list)) + (define (empty-termlist? term-list) (null? term-list)) + + (define (make-term order coeff) (list order coeff)) + (define (order term) (car term)) + (define (coeff term) (cadr term)) + +where `=zero?' is as defined in *Note Exercise 2-80::. (See also *Note +Exercise 2-87:: below.) + + Users of the polynomial package will create (tagged) polynomials by +means of the procedure: + + (define (make-polynomial var terms) + ((get 'make 'polynomial) var terms)) + + *Exercise 2.87:* Install `=zero?' for polynomials in the generic + arithmetic package. This will allow `adjoin-term' to work for + polynomials with coefficients that are themselves polynomials. + + *Exercise 2.88:* Extend the polynomial system to include + subtraction of polynomials. (Hint: You may find it helpful to + define a generic negation operation.) + + *Exercise 2.89:* Define procedures that implement the term-list + representation described above as appropriate for dense + polynomials. + + *Exercise 2.90:* Suppose we want to have a polynomial system that + is efficient for both sparse and dense polynomials. One way to do + this is to allow both kinds of term-list representations in our + system. The situation is analogous to the complex-number example + of section *Note 2-4::, where we allowed both rectangular and + polar representations. To do this we must distinguish different + types of term lists and make the operations on term lists generic. + Redesign the polynomial system to implement this generalization. + This is a major effort, not a local change. + + *Exercise 2.91:* A univariate polynomial can be divided by another + one to produce a polynomial quotient and a polynomial remainder. + For example, + + x^5 - 1 + ------- = x^3 + x, remainder x - 1 + x^2 - 1 + + Division can be performed via long division. That is, divide the + highest-order term of the dividend by the highest-order term of + the divisor. The result is the first term of the quotient. Next, + multiply the result by the divisor, subtract that from the + dividend, and produce the rest of the answer by recursively + dividing the difference by the divisor. Stop when the order of the + divisor exceeds the order of the dividend and declare the dividend + to be the remainder. Also, if the dividend ever becomes zero, + return zero as both quotient and remainder. + + We can design a `div-poly' procedure on the model of `add-poly' and + `mul-poly'. The procedure checks to see if the two polys have the + same variable. If so, `div-poly' strips off the variable and + passes the problem to `div-terms', which performs the division + operation on term lists. `Div-poly' finally reattaches the + variable to the result supplied by `div-terms'. It is convenient + to design `div-terms' to compute both the quotient and the + remainder of a division. `Div-terms' can take two term lists as + arguments and return a list of the quotient term list and the + remainder term list. + + Complete the following definition of `div-terms' by filling in the + missing expressions. Use this to implement `div-poly', which + takes two polys as arguments and returns a list of the quotient + and remainder polys. + + (define (div-terms L1 L2) + (if (empty-termlist? L1) + (list (the-empty-termlist) (the-empty-termlist)) + (let ((t1 (first-term L1)) + (t2 (first-term L2))) + (if (> (order t2) (order t1)) + (list (the-empty-termlist) L1) + (let ((new-c (div (coeff t1) (coeff t2))) + (new-o (- (order t1) (order t2)))) + (let ((rest-of-result + + )) + + )))))) + +Hierarchies of types in symbolic algebra +........................................ + +Our polynomial system illustrates how objects of one type (polynomials) +may in fact be complex objects that have objects of many different +types as parts. This poses no real difficulty in defining generic +operations. We need only install appropriate generic operations for +performing the necessary manipulations of the parts of the compound +types. In fact, we saw that polynomials form a kind of "recursive data +abstraction," in that parts of a polynomial may themselves be +polynomials. Our generic operations and our data-directed programming +style can handle this complication without much trouble. + + On the other hand, polynomial algebra is a system for which the data +types cannot be naturally arranged in a tower. For instance, it is +possible to have polynomials in x whose coefficients are polynomials in +y. It is also possible to have polynomials in y whose coefficients are +polynomials in x. Neither of these types is "above" the other in any +natural way, yet it is often necessary to add together elements from +each set. There are several ways to do this. One possibility is to +convert one polynomial to the type of the other by expanding and +rearranging terms so that both polynomials have the same principal +variable. One can impose a towerlike structure on this by ordering the +variables and thus always converting any polynomial to a "canonical +form" with the highest-priority variable dominant and the +lower-priority variables buried in the coefficients. This strategy +works fairly well, except that the conversion may expand a polynomial +unnecessarily, making it hard to read and perhaps less efficient to +work with. The tower strategy is certainly not natural for this domain +or for any domain where the user can invent new types dynamically using +old types in various combining forms, such as trigonometric functions, +power series, and integrals. + + It should not be surprising that controlling coercion is a serious +problem in the design of large-scale algebraic-manipulation systems. +Much of the complexity of such systems is concerned with relationships +among diverse types. Indeed, it is fair to say that we do not yet +completely understand coercion. In fact, we do not yet completely +understand the concept of a data type. Nevertheless, what we know +provides us with powerful structuring and modularity principles to +support the design of large systems. + + *Exercise 2.92:* By imposing an ordering on variables, extend the + polynomial package so that addition and multiplication of + polynomials works for polynomials in different variables. (This + is not easy!) + +Extended exercise: Rational functions +..................................... + +We can extend our generic arithmetic system to include functions +"rational functions". These are "fractions" whose numerator and +denominator are polynomials, such as + + x + 1 + ------- + x^3 - 1 + + The system should be able to add, subtract, multiply, and divide +rational functions, and to perform such computations as + + x + 1 x x^3 + 2x^2 + 3x + 1 + ------- + ------- = ------------------- + x^3 - 1 x^2 - 1 x^4 + x^3 - x - 1 + +(Here the sum has been simplified by removing common factors. Ordinary +"cross multiplication" would have produced a fourth-degree polynomial +over a fifth-degree polynomial.) + + If we modify our rational-arithmetic package so that it uses generic +operations, then it will do what we want, except for the problem of +reducing fractions to lowest terms. + + *Exercise 2.93:* Modify the rational-arithmetic package to use + generic operations, but change `make-rat' so that it does not + attempt to reduce fractions to lowest terms. Test your system by + calling `make-rational' on two polynomials to produce a rational + function + + (define p1 (make-polynomial 'x '((2 1)(0 1)))) + (define p2 (make-polynomial 'x '((3 1)(0 1)))) + (define rf (make-rational p2 p1)) + + Now add `rf' to itself, using `add'. You will observe that this + addition procedure does not reduce fractions to lowest terms. + + We can reduce polynomial fractions to lowest terms using the same + idea we used with integers: modifying `make-rat' to divide both + the numerator and the denominator by their greatest common + divisor. The notion of "greatest common divisor" makes sense for + polynomials. In fact, we can compute the GCD of two polynomials + using essentially the same Euclid's Algorithm that works for + integers.(7) The integer version is + + (define (gcd a b) + (if (= b 0) + a + (gcd b (remainder a b)))) + + Using this, we could make the obvious modification to define a GCD + operation that works on term lists: + + (define (gcd-terms a b) + (if (empty-termlist? b) + a + (gcd-terms b (remainder-terms a b)))) + + where `remainder-terms' picks out the remainder component of the + list returned by the term-list division operation `div-terms' that + was implemented in *Note Exercise 2-91::. + + *Exercise 2.94:* Using `div-terms', implement the procedure + `remainder-terms' and use this to define `gcd-terms' as above. + Now write a procedure `gcd-poly' that computes the polynomial GCD + of two polys. (The procedure should signal an error if the two + polys are not in the same variable.) Install in the system a + generic operation `greatest-common-divisor' that reduces to + `gcd-poly' for polynomials and to ordinary `gcd' for ordinary + numbers. As a test, try + + (define p1 (make-polynomial 'x '((4 1) (3 -1) (2 -2) (1 2)))) + (define p2 (make-polynomial 'x '((3 1) (1 -1)))) + (greatest-common-divisor p1 p2) + + and check your result by hand. + + *Exercise 2.95:* Define P_1, P_2, and P_3 to be the polynomials + + P_1 : x^2 - 2x + 1 + + P_2 : 11x^2 + 7 + + P_3 : 13x + 5 + + Now define Q_1 to be the product of P_1 and P_2 and Q_2 to be the + product of P_1 and P_3, and use `greatest-common-divisor' (*Note + Exercise 2-94::) to compute the GCD of Q_1 and Q_2. Note that the + answer is not the same as P_1. This example introduces noninteger + operations into the computation, causing difficulties with the GCD + algorithm.(8) To understand what is happening, try tracing + `gcd-terms' while computing the GCD or try performing the division + by hand. + + We can solve the problem exhibited in *Note Exercise 2-95:: if we + use the following modification of the GCD algorithm (which really + works only in the case of polynomials with integer coefficients). + Before performing any polynomial division in the GCD computation, + we multiply the dividend by an integer constant factor, chosen to + guarantee that no fractions will arise during the division + process. Our answer will thus differ from the actual GCD by an + integer constant factor, but this does not matter in the case of + reducing rational functions to lowest terms; the GCD will be used + to divide both the numerator and denominator, so the integer + constant factor will cancel out. + + More precisely, if P and Q are polynomials, let O_1 be the order of + P (i.e., the order of the largest term of P) and let O_2 be the + order of Q. Let c be the leading coefficient of Q. Then it can be + shown that, if we multiply P by the "integerizing factor" c^(1+O_1 + -O_2), the resulting polynomial can be divided by Q by using the + `div-terms' algorithm without introducing any fractions. The + operation of multiplying the dividend by this constant and then + dividing is sometimes called the "pseudodivision" of P by Q. The + remainder of the division is called the "pseudoremainder". + + *Exercise 2.96:* + a. Implement the procedure `pseudoremainder-terms', which is + just like `remainder-terms' except that it multiplies the + dividend by the integerizing factor described above before + calling `div-terms'. Modify `gcd-terms' to use + `pseudoremainder-terms', and verify that + `greatest-common-divisor' now produces an answer with integer + coefficients on the example in *Note Exercise 2-95::. + + b. The GCD now has integer coefficients, but they are larger + than those of P_1. Modify `gcd-terms' so that it removes + common factors from the coefficients of the answer by + dividing all the coefficients by their (integer) greatest + common divisor. + + + Thus, here is how to reduce a rational function to lowest terms: + + * Compute the GCD of the numerator and denominator, using the + version of `gcd-terms' from *Note Exercise 2-96::. + + * When you obtain the GCD, multiply both numerator and + denominator by the same integerizing factor before dividing + through by the GCD, so that division by the GCD will not + introduce any noninteger coefficients. As the factor you can + use the leading coefficient of the GCD raised to the power 1 + + O_1 - O_2, where O_2 is the order of the GCD and O_1 is the + maximum of the orders of the numerator and denominator. This + will ensure that dividing the numerator and denominator by + the GCD will not introduce any fractions. + + * The result of this operation will be a numerator and + denominator with integer coefficients. The coefficients will + normally be very large because of all of the integerizing + factors, so the last step is to remove the redundant factors + by computing the (integer) greatest common divisor of all the + coefficients of the numerator and the denominator and + dividing through by this factor. + + + *Exercise 2.97:* + a. Implement this algorithm as a procedure `reduce-terms' that + takes two term lists `n' and `d' as arguments and returns a + list `nn', `dd', which are `n' and `d' reduced to lowest + terms via the algorithm given above. Also write a procedure + `reduce-poly', analogous to `add-poly', that checks to see if + the two polys have the same variable. If so, `reduce-poly' + strips off the variable and passes the problem to + `reduce-terms', then reattaches the variable to the two term + lists supplied by `reduce-terms'. + + b. Define a procedure analogous to `reduce-terms' that does what + the original `make-rat' did for integers: + + (define (reduce-integers n d) + (let ((g (gcd n d))) + (list (/ n g) (/ d g)))) + + and define `reduce' as a generic operation that calls + `apply-generic' to dispatch to either `reduce-poly' (for + `polynomial' arguments) or `reduce-integers' (for + `scheme-number' arguments). You can now easily make the + rational-arithmetic package reduce fractions to lowest terms + by having `make-rat' call `reduce' before combining the given + numerator and denominator to form a rational number. The + system now handles rational expressions in either integers or + polynomials. To test your program, try the example at the + beginning of this extended exercise: + + (define p1 (make-polynomial 'x '((1 1)(0 1)))) + (define p2 (make-polynomial 'x '((3 1)(0 -1)))) + (define p3 (make-polynomial 'x '((1 1)))) + (define p4 (make-polynomial 'x '((2 1)(0 -1)))) + + (define rf1 (make-rational p1 p2)) + (define rf2 (make-rational p3 p4)) + + (add rf1 rf2) + + See if you get the correct answer, correctly reduced to + lowest terms. + + The GCD computation is at the heart of any system that does + operations on rational functions. The algorithm used above, + although mathematically straightforward, is extremely slow. + The slowness is due partly to the large number of division + operations and partly to the enormous size of the + intermediate coefficients generated by the pseudodivisions. + One of the active areas in the development of + algebraic-manipulation systems is the design of better + algorithms for computing polynomial GCDs.(9) + + + ---------- Footnotes ---------- + + (1) On the other hand, we will allow polynomials whose coefficients +are themselves polynomials in other variables. This will give us +essentially the same representational power as a full multivariate +system, although it does lead to coercion problems, as discussed below. + + (2) For univariate polynomials, giving the value of a polynomial at +a given set of points can be a particularly good representation. This +makes polynomial arithmetic extremely simple. To obtain, for example, +the sum of two polynomials represented in this way, we need only add +the values of the polynomials at corresponding points. To transform +back to a more familiar representation, we can use the Lagrange +interpolation formula, which shows how to recover the coefficients of a +polynomial of degree n given the values of the polynomial at n + 1 +points. + + (3) This operation is very much like the ordered `union-set' +operation we developed in exercise *Note Exercise 2-62::. In fact, if +we think of the terms of the polynomial as a set ordered according to +the power of the indeterminate, then the program that produces the term +list for a sum is almost identical to `union-set'. + + (4) To make this work completely smoothly, we should also add to our +generic arithmetic system the ability to coerce a "number" to a +polynomial by regarding it as a polynomial of degree zero whose +coefficient is the number. This is necessary if we are going to +perform operations such as + + [x^2 + (y + 1)x + 5] + [x^2 + 2x + 1] + +which requires adding the coefficient y + 1 to the coefficient 2. + + (5) In these polynomial examples, we assume that we have implemented +the generic arithmetic system using the type mechanism suggested in +*Note Exercise 2-78::. Thus, coefficients that are ordinary numbers +will be represented as the numbers themselves rather than as pairs +whose `car' is the symbol `scheme-number'. + + (6) Although we are assuming that term lists are ordered, we have +implemented `adjoin-term' to simply `cons' the new term onto the +existing term list. We can get away with this so long as we guarantee +that the procedures (such as `add-terms') that use `adjoin-term' always +call it with a higher-order term than appears in the list. If we did +not want to make such a guarantee, we could have implemented +`adjoin-term' to be similar to the `adjoin-set' constructor for the +ordered-list representation of sets (*Note Exercise 2-61::). + + (7) The fact that Euclid's Algorithm works for polynomials is +formalized in algebra by saying that polynomials form a kind of +algebraic domain called a "Euclidean ring". A Euclidean ring is a +domain that admits addition, subtraction, and commutative +multiplication, together with a way of assigning to each element x of +the ring a positive integer "measure" m(x) with the properties that +m(xy)>= m(x) for any nonzero x and y and that, given any x and y, there +exists a q such that y = qx + r and either r = 0 or m(r)< m(x). From +an abstract point of view, this is what is needed to prove that +Euclid's Algorithm works. For the domain of integers, the measure m of +an integer is the absolute value of the integer itself. For the domain +of polynomials, the measure of a polynomial is its degree. + + (8) In an implementation like MIT Scheme, this produces a polynomial +that is indeed a divisor of Q_1 and Q_2, but with rational +coefficients. In many other Scheme systems, in which division of +integers can produce limited-precision decimal numbers, we may fail to +get a valid divisor. + + (9) One extremely efficient and elegant method for computing +polynomial GCDs was discovered by Richard Zippel (1979). The method is +a probabilistic algorithm, as is the fast test for primality that we +discussed in *Note Chapter 1::. Zippel's book (1993) describes this +method, together with other ways to compute polynomial GCDs. + + +File: sicp.info, Node: Chapter 3, Next: Chapter 4, Prev: Chapter 2, Up: Top + +3 Modularity, Objects, and State +******************************** + + [greek not included here] + + (Even while it changes, it stands still.) + + --Heraclitus + + Plus c,a change, plus c'est la me*me chose. + + --Alphonse Karr + + The preceding chapters introduced the basic elements from which +programs are made. We saw how primitive procedures and primitive data +are combined to construct compound entities, and we learned that +abstraction is vital in helping us to cope with the complexity of large +systems. But these tools are not sufficient for designing programs. +Effective program synthesis also requires organizational principles +that can guide us in formulating the overall design of a program. In +particular, we need strategies to help us structure large systems so +that they will be "modular", that is, so that they can be divided +"naturally" into coherent parts that can be separately developed and +maintained. + + One powerful design strategy, which is particularly appropriate to +the construction of programs for modeling physical systems, is to base +the structure of our programs on the structure of the system being +modeled. For each object in the system, we construct a corresponding +computational object. For each system action, we define a symbolic +operation in our computational model. Our hope in using this strategy +is that extending the model to accommodate new objects or new actions +will require no strategic changes to the program, only the addition of +the new symbolic analogs of those objects or actions. If we have been +successful in our system organization, then to add a new feature or +debug an old one we will have to work on only a localized part of the +system. + + To a large extent, then, the way we organize a large program is +dictated by our perception of the system to be modeled. In this +chapter we will investigate two prominent organizational strategies +arising from two rather different "world views" of the structure of +systems. The first organizational strategy concentrates on "objects", +viewing a large system as a collection of distinct objects whose +behaviors may change over time. An alternative organizational strategy +concentrates on the "streams" of information that flow in the system, +much as an electrical engineer views a signal-processing system. + + Both the object-based approach and the stream-processing approach +raise significant linguistic issues in programming. With objects, we +must be concerned with how a computational object can change and yet +maintain its identity. This will force us to abandon our old +substitution model of computation (section *Note 1-1-5::) in favor of a +more mechanistic but less theoretically tractable "environment model" +of computation. The difficulties of dealing with objects, change, and +identity are a fundamental consequence of the need to grapple with time +in our computational models. These difficulties become even greater +when we allow the possibility of concurrent execution of programs. The +stream approach can be most fully exploited when we decouple simulated +time in our model from the order of the events that take place in the +computer during evaluation. We will accomplish this using a technique +known as "delayed evaluation". + +* Menu: + +* 3-1:: Assignment and Local State +* 3-2:: The Environment Model of Evaluation +* 3-3:: Modeling with Mutable Data +* 3-4:: Concurrency: Time Is of the Essence +* 3-5:: Streams + + +File: sicp.info, Node: 3-1, Next: 3-2, Prev: Chapter 3, Up: Chapter 3 + +3.1 Assignment and Local State +============================== + +We ordinarily view the world as populated by independent objects, each +of which has a state that changes over time. An object is said to +"have state" if its behavior is influenced by its history. A bank +account, for example, has state in that the answer to the question "Can +I withdraw $100?" depends upon the history of deposit and withdrawal +transactions. We can characterize an object's state by one or more "state +variables", which among them maintain enough information about history +to determine the object's current behavior. In a simple banking +system, we could characterize the state of an account by a current +balance rather than by remembering the entire history of account +transactions. + + In a system composed of many objects, the objects are rarely +completely independent. Each may influence the states of others +through interactions, which serve to couple the state variables of one +object to those of other objects. Indeed, the view that a system is +composed of separate objects is most useful when the state variables of +the system can be grouped into closely coupled subsystems that are only +loosely coupled to other subsystems. + + This view of a system can be a powerful framework for organizing +computational models of the system. For such a model to be modular, it +should be decomposed into computational objects that model the actual +objects in the system. Each computational object must have its own "local +state variables" describing the actual object's state. Since the +states of objects in the system being modeled change over time, the +state variables of the corresponding computational objects must also +change. If we choose to model the flow of time in the system by the +elapsed time in the computer, then we must have a way to construct +computational objects whose behaviors change as our programs run. In +particular, if we wish to model state variables by ordinary symbolic +names in the programming language, then the language must provide an operator +"assignment operator" to enable us to change the value associated with +a name. + +* Menu: + +* 3-1-1:: Local State Variables +* 3-1-2:: The Benefits of Introducing Assignment +* 3-1-3:: The Costs of Introducing Assignment + + +File: sicp.info, Node: 3-1-1, Next: 3-1-2, Prev: 3-1, Up: 3-1 + +3.1.1 Local State Variables +--------------------------- + +To illustrate what we mean by having a computational object with +time-varying state, let us model the situation of withdrawing money +from a bank account. We will do this using a procedure `withdraw', +which takes as argument an `amount' to be withdrawn. If there is +enough money in the account to accommodate the withdrawal, then +`withdraw' should return the balance remaining after the withdrawal. +Otherwise, `withdraw' should return the message _Insufficient funds_. +For example, if we begin with $100 in the account, we should obtain the +following sequence of responses using `withdraw': + + (withdraw 25) + 75 + + (withdraw 25) + 50 + + (withdraw 60) + "Insufficient funds" + + (withdraw 15) + 35 + + Observe that the expression `(withdraw 25)', evaluated twice, yields +different values. This is a new kind of behavior for a procedure. +Until now, all our procedures could be viewed as specifications for +computing mathematical functions. A call to a procedure computed the +value of the function applied to the given arguments, and two calls to +the same procedure with the same arguments always produced the same +result.(1) + + To implement `withdraw', we can use a variable `balance' to indicate +the balance of money in the account and define `withdraw' as a procedure +that accesses `balance'. The `withdraw' procedure checks to see if +`balance' is at least as large as the requested `amount'. If so, +`withdraw' decrements `balance' by `amount' and returns the new value +of `balance'. Otherwise, `withdraw' returns the _Insufficient funds_ +message. Here are the definitions of `balance' and `withdraw': + + (define balance 100) + + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + + Decrementing `balance' is accomplished by the expression + + (set! balance (- balance amount)) + + This uses the `set!' special form, whose syntax is + + (set! ) + + Here is a symbol and is any expression. `Set!' +changes so that its value is the result obtained by evaluating +. In the case at hand, we are changing `balance' so that +its new value will be the result of subtracting `amount' from the +previous value of `balance'.(2) + + `Withdraw' also uses the `begin' special form to cause two +expressions to be evaluated in the case where the `if' test is true: +first decrementing `balance' and then returning the value of `balance'. +In general, evaluating the expression + + (begin ... ) + +causes the expressions through to be evaluated in +sequence and the value of the final expression to be returned +as the value of the entire `begin' form.(3) + + Although `withdraw' works as desired, the variable `balance' presents +a problem. As specified above, `balance' is a name defined in the +global environment and is freely accessible to be examined or modified +by any procedure. It would be much better if we could somehow make +`balance' internal to `withdraw', so that `withdraw' would be the only +procedure that could access `balance' directly and any other procedure +could access `balance' only indirectly (through calls to `withdraw'). +This would more accurately model the notion that `balance' is a local +state variable used by `withdraw' to keep track of the state of the +account. + + We can make `balance' internal to `withdraw' by rewriting the +definition as follows: + + (define new-withdraw + (let ((balance 100)) + (lambda (amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")))) + + What we have done here is use `let' to establish an environment with +a local variable `balance', bound to the initial value 100. Within this +local environment, we use `lambda' to create a procedure that takes +`amount' as an argument and behaves like our previous `withdraw' +procedure. This procedure--returned as the result of evaluating the +`let' expression--is `new-withdraw', which behaves in precisely the +same way as `withdraw' but whose variable `balance' is not accessible +by any other procedure.(4) + + Combining `set!' with local variables is the general programming +technique we will use for constructing computational objects with local +state. Unfortunately, using this technique raises a serious problem: +When we first introduced procedures, we also introduced the +substitution model of evaluation (section *Note 1-1-5::) to provide an +interpretation of what procedure application means. We said that +applying a procedure should be interpreted as evaluating the body of +the procedure with the formal parameters replaced by their values. The +trouble is that, as soon as we introduce assignment into our language, +substitution is no longer an adequate model of procedure application. +(We will see why this is so in section *Note 3-1-3::.) As a +consequence, we technically have at this point no way to understand why +the `new-withdraw' procedure behaves as claimed above. In order to +really understand a procedure such as `new-withdraw', we will need to +develop a new model of procedure application. In section *Note 3-2:: +we will introduce such a model, together with an explanation of `set!' +and local variables. First, however, we examine some variations on the +theme established by `new-withdraw'. + + The following procedure, `make-withdraw', creates "withdrawal +processors." The formal parameter `balance' in `make-withdraw' +specifies the initial amount of money in the account.(5) + + (define (make-withdraw balance) + (lambda (amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds"))) + + `Make-withdraw' can be used as follows to create two objects `W1' and +`W2': + + (define W1 (make-withdraw 100)) + (define W2 (make-withdraw 100)) + + (W1 50) + 50 + + (W2 70) + 30 + + (W2 40) + "Insufficient funds" + + (W1 40) + 10 + + Observe that `W1' and `W2' are completely independent objects, each +with its own local state variable `balance'. Withdrawals from one do +not affect the other. + + We can also create objects that handle deposits as well as +withdrawals, and thus we can represent simple bank accounts. Here is a +procedure that returns a "bank-account object" with a specified initial +balance: + + (define (make-account balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + (define (dispatch m) + (cond ((eq? m 'withdraw) withdraw) + ((eq? m 'deposit) deposit) + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch) + + Each call to `make-account' sets up an environment with a local state +variable `balance'. Within this environment, `make-account' defines +procedures `deposit' and `withdraw' that access `balance' and an +additional procedure `dispatch' that takes a "message" as input and +returns one of the two local procedures. The `dispatch' procedure +itself is returned as the value that represents the bank-account +object. This is precisely the "message-passing" style of programming +that we saw in section *Note 2-4-3::, although here we are using it in +conjunction with the ability to modify local variables. + + `Make-account' can be used as follows: + + (define acc (make-account 100)) + + ((acc 'withdraw) 50) + 50 + + ((acc 'withdraw) 60) + "Insufficient funds" + + ((acc 'deposit) 40) + 90 + + ((acc 'withdraw) 60) + 30 + + Each call to `acc' returns the locally defined `deposit' or +`withdraw' procedure, which is then applied to the specified `amount'. +As was the case with `make-withdraw', another call to `make-account' + + (define acc2 (make-account 100)) + +will produce a completely separate account object, which maintains its +own local `balance'. + + *Exercise 3.1:* An "accumulator" is a procedure that is called + repeatedly with a single numeric argument and accumulates its + arguments into a sum. Each time it is called, it returns the + currently accumulated sum. Write a procedure `make-accumulator' + that generates accumulators, each maintaining an independent sum. + The input to `make-accumulator' should specify the initial value + of the sum; for example + + (define A (make-accumulator 5)) + + (A 10) + 15 + + (A 10) + 25 + + *Exercise 3.2:* In software-testing applications, it is useful to + be able to count the number of times a given procedure is called + during the course of a computation. Write a procedure + `make-monitored' that takes as input a procedure, `f', that itself + takes one input. The result returned by `make-monitored' is a + third procedure, say `mf', that keeps track of the number of times + it has been called by maintaining an internal counter. If the + input to `mf' is the special symbol `how-many-calls?', then `mf' + returns the value of the counter. If the input is the special + symbol `reset-count', then `mf' resets the counter to zero. For + any other input, `mf' returns the result of calling `f' on that + input and increments the counter. For instance, we could make a + monitored version of the `sqrt' procedure: + + (define s (make-monitored sqrt)) + + (s 100) + 10 + + (s 'how-many-calls?) + 1 + + *Exercise 3.3:* Modify the `make-account' procedure so that it + creates password-protected accounts. That is, `make-account' + should take a symbol as an additional argument, as in + + (define acc (make-account 100 'secret-password)) + + The resulting account object should process a request only if it + is accompanied by the password with which the account was created, + and should otherwise return a complaint: + + ((acc 'secret-password 'withdraw) 40) + 60 + + ((acc 'some-other-password 'deposit) 50) + "Incorrect password" + + *Exercise 3.4:* Modify the `make-account' procedure of *Note + Exercise 3-3:: by adding another local state variable so that, if + an account is accessed more than seven consecutive times with an + incorrect password, it invokes the procedure `call-the-cops'. + + ---------- Footnotes ---------- + + (1) Actually, this is not quite true. One exception was the +random-number generator in section *Note 1-2-6::. Another exception +involved the operation/type tables we introduced in section *Note +2-4-3::, where the values of two calls to `get' with the same arguments +depended on intervening calls to `put'. On the other hand, until we +introduce assignment, we have no way to create such procedures +ourselves. + + (2) The value of a `set!' expression is implementation-dependent. +`Set!' should be used only for its effect, not for its value. + + The name `set!' reflects a naming convention used in Scheme: +Operations that change the values of variables (or that change data +structures, as we will see in section *Note 3-3::) are given names that +end with an exclamation point. This is similar to the convention of +designating predicates by names that end with a question mark. + + (3) We have already used `begin' implicitly in our programs, because +in Scheme the body of a procedure can be a sequence of expressions. +Also, the part of each clause in a `cond' expression can +be a sequence of expressions rather than a single expression. + + (4) In programming-language jargon, the variable `balance' is said +to be "encapsulated" within the `new-withdraw' procedure. +Encapsulation reflects the general system-design principle known as the "hiding +principle": One can make a system more modular and robust by protecting +parts of the system from each other; that is, by providing information +access only to those parts of the system that have a "need to know." + + (5) In contrast with `new-withdraw' above, we do not have to use +`let' to make `balance' a local variable, since formal parameters are +already local. This will be clearer after the discussion of the +environment model of evaluation in section *Note 3-2::. (See also +*Note Exercise 3-10::.) + + +File: sicp.info, Node: 3-1-2, Next: 3-1-3, Prev: 3-1-1, Up: 3-1 + +3.1.2 The Benefits of Introducing Assignment +-------------------------------------------- + +As we shall see, introducing assignment into our programming language +leads us into a thicket of difficult conceptual issues. Nevertheless, +viewing systems as collections of objects with local state is a +powerful technique for maintaining a modular design. As a simple +example, consider the design of a procedure `rand' that, whenever it is +called, returns an integer chosen at random. + + It is not at all clear what is meant by "chosen at random." What we +presumably want is for successive calls to `rand' to produce a sequence +of numbers that has statistical properties of uniform distribution. We +will not discuss methods for generating suitable sequences here. +Rather, let us assume that we have a procedure `rand-update' that has +the property that if we start with a given number x_1 and form + + x_2 = (rand-update x_1) + x_3 = (rand-update x_2) + +then the sequence of values x_1, x_2, x_3, ..., will have the desired +statistical properties.(1) + + We can implement `rand' as a procedure with a local state variable +`x' that is initialized to some fixed value `random-init'. Each call +to `rand' computes `rand-update' of the current value of `x', returns +this as the random number, and also stores this as the new value of `x'. + + (define rand + (let ((x random-init)) + (lambda () + (set! x (rand-update x)) + x))) + + Of course, we could generate the same sequence of random numbers +without using assignment by simply calling `rand-update' directly. +However, this would mean that any part of our program that used random +numbers would have to explicitly remember the current value of `x' to +be passed as an argument to `rand-update'. To realize what an +annoyance this would be, consider using random numbers to implement a +technique called simulation "Monte Carlo simulation". + + The Monte Carlo method consists of choosing sample experiments at +random from a large set and then making deductions on the basis of the +probabilities estimated from tabulating the results of those +experiments. For example, we can approximate [pi] using the fact that +6/[pi]^2 is the probability that two integers chosen at random will +have no factors in common; that is, that their greatest common divisor +will be 1.(2) To obtain the approximation to [pi], we perform a large +number of experiments. In each experiment we choose two integers at +random and perform a test to see if their GCD is 1. The fraction of +times that the test is passed gives us our estimate of 6/[pi]^2, and +from this we obtain our approximation to [pi]. + + The heart of our program is a procedure `monte-carlo', which takes as +arguments the number of times to try an experiment, together with the +experiment, represented as a no-argument procedure that will return +either true or false each time it is run. `Monte-carlo' runs the +experiment for the designated number of trials and returns a number +telling the fraction of the trials in which the experiment was found to +be true. + + (define (estimate-pi trials) + (sqrt (/ 6 (monte-carlo trials cesaro-test)))) + + (define (cesaro-test) + (= (gcd (rand) (rand)) 1)) + + (define (monte-carlo trials experiment) + (define (iter trials-remaining trials-passed) + (cond ((= trials-remaining 0) + (/ trials-passed trials)) + ((experiment) + (iter (- trials-remaining 1) (+ trials-passed 1))) + (else + (iter (- trials-remaining 1) trials-passed)))) + (iter trials 0)) + + Now let us try the same computation using `rand-update' directly +rather than `rand', the way we would be forced to proceed if we did not +use assignment to model local state: + + (define (estimate-pi trials) + (sqrt (/ 6 (random-gcd-test trials random-init)))) + + (define (random-gcd-test trials initial-x) + (define (iter trials-remaining trials-passed x) + (let ((x1 (rand-update x))) + (let ((x2 (rand-update x1))) + (cond ((= trials-remaining 0) + (/ trials-passed trials)) + ((= (gcd x1 x2) 1) + (iter (- trials-remaining 1) + (+ trials-passed 1) + x2)) + (else + (iter (- trials-remaining 1) + trials-passed + x2)))))) + (iter trials 0 initial-x)) + + While the program is still simple, it betrays some painful breaches +of modularity. In our first version of the program, using `rand', we +can express the Monte Carlo method directly as a general `monte-carlo' +procedure that takes as an argument an arbitrary `experiment' procedure. +In our second version of the program, with no local state for the +random-number generator, `random-gcd-test' must explicitly manipulate +the random numbers `x1' and `x2' and recycle `x2' through the iterative +loop as the new input to `rand-update'. This explicit handling of the +random numbers intertwines the structure of accumulating test results +with the fact that our particular experiment uses two random numbers, +whereas other Monte Carlo experiments might use one random number or +three. Even the top-level procedure `estimate-pi' has to be concerned +with supplying an initial random number. The fact that the +random-number generator's insides are leaking out into other parts of +the program makes it difficult for us to isolate the Monte Carlo idea +so that it can be applied to other tasks. In the first version of the +program, assignment encapsulates the state of the random-number +generator within the `rand' procedure, so that the details of +random-number generation remain independent of the rest of the program. + + The general phenomenon illustrated by the Monte Carlo example is +this: From the point of view of one part of a complex process, the +other parts appear to change with time. They have hidden time-varying +local state. If we wish to write computer programs whose structure +reflects this decomposition, we make computational objects (such as +bank accounts and random-number generators) whose behavior changes with +time. We model state with local state variables, and we model the +changes of state with assignments to those variables. + + It is tempting to conclude this discussion by saying that, by +introducing assignment and the technique of hiding state in local +variables, we are able to structure systems in a more modular fashion +than if all state had to be manipulated explicitly, by passing +additional parameters. Unfortunately, as we shall see, the story is +not so simple. + + *Exercise 3.5:* "Monte Carlo integration" is a method of + estimating definite integrals by means of Monte Carlo simulation. + Consider computing the area of a region of space described by a + predicate P(x, y) that is true for points (x, y) in the region and + false for points not in the region. For example, the region + contained within a circle of radius 3 centered at (5, 7) is + described by the predicate that tests whether (x - 5)^2 + (y - + 7)^2 <= 3^2. To estimate the area of the region described by such + a predicate, begin by choosing a rectangle that contains the + region. For example, a rectangle with diagonally opposite corners + at (2, 4) and (8, 10) contains the circle above. The desired + integral is the area of that portion of the rectangle that lies in + the region. We can estimate the integral by picking, at random, + points (x,y) that lie in the rectangle, and testing P(x, y) for + each point to determine whether the point lies in the region. If + we try this with many points, then the fraction of points that + fall in the region should give an estimate of the proportion of + the rectangle that lies in the region. Hence, multiplying this + fraction by the area of the entire rectangle should produce an + estimate of the integral. + + Implement Monte Carlo integration as a procedure + `estimate-integral' that takes as arguments a predicate `P', upper + and lower bounds `x1', `x2', `y1', and `y2' for the rectangle, and + the number of trials to perform in order to produce the estimate. + Your procedure should use the same `monte-carlo' procedure that + was used above to estimate [pi]. Use your `estimate-integral' to + produce an estimate of [pi] by measuring the area of a unit circle. + + You will find it useful to have a procedure that returns a number + chosen at random from a given range. The following + `random-in-range' procedure implements this in terms of the + `random' procedure used in section *Note 1-2-6::, which returns a + nonnegative number less than its input.(3) + + (define (random-in-range low high) + (let ((range (- high low))) + (+ low (random range)))) + + *Exercise 3.6:* It is useful to be able to reset a random-number + generator to produce a sequence starting from a given value. + Design a new `rand' procedure that is called with an argument that + is either the symbol `generate' or the symbol `reset' and behaves + as follows: `(rand 'generate)' produces a new random number; + `((rand 'reset) )' resets the internal state variable + to the designated . Thus, by resetting the state, one + can generate repeatable sequences. These are very handy to have + when testing and debugging programs that use random numbers. + + ---------- Footnotes ---------- + + (1) One common way to implement `rand-update' is to use the rule +that x is updated to ax + b modulo m, where a, b, and m are +appropriately chosen integers. Chapter 3 of Knuth 1981 includes an +extensive discussion of techniques for generating sequences of random +numbers and establishing their statistical properties. Notice that the +`rand-update' procedure computes a mathematical function: Given the +same input twice, it produces the same output. Therefore, the number +sequence produced by `rand-update' certainly is not "random," if by +"random" we insist that each number in the sequence is unrelated to the +preceding number. The relation between "real randomness" and so-called "pseudo-random" +sequences, which are produced by well-determined computations and yet +have suitable statistical properties, is a complex question involving +difficult issues in mathematics and philosophy. Kolmogorov, +Solomonoff, and Chaitin have made great progress in clarifying these +issues; a discussion can be found in Chaitin 1975. + + (2) This theorem is due to E. Cesa`ro. See section 4.5.2 of Knuth +1981 for a discussion and a proof. + + (3) MIT Scheme provides such a procedure. If `random' is given an +exact integer (as in section *Note 1-2-6::) it returns an exact +integer, but if it is given a decimal value (as in this exercise) it +returns a decimal value. + + +File: sicp.info, Node: 3-1-3, Prev: 3-1-2, Up: 3-1 + +3.1.3 The Costs of Introducing Assignment +----------------------------------------- + +As we have seen, the `set!' operation enables us to model objects that +have local state. However, this advantage comes at a price. Our +programming language can no longer be interpreted in terms of the +substitution model of procedure application that we introduced in +section *Note 1-1-5::. Moreover, no simple model with "nice" +mathematical properties can be an adequate framework for dealing with +objects and assignment in programming languages. + + So long as we do not use assignments, two evaluations of the same +procedure with the same arguments will produce the same result, so that +procedures can be viewed as computing mathematical functions. +Programming without any use of assignments, as we did throughout the +first two chapters of this book, is accordingly known as "functional +programming". + + To understand how assignment complicates matters, consider a +simplified version of the `make-withdraw' procedure of section *Note +3-1-1:: that does not bother to check for an insufficient amount: + + (define (make-simplified-withdraw balance) + (lambda (amount) + (set! balance (- balance amount)) + balance)) + + (define W (make-simplified-withdraw 25)) + + (W 20) + 5 + + (W 10) + - 5 + + Compare this procedure with the following `make-decrementer' +procedure, which does not use `set!': + + (define (make-decrementer balance) + (lambda (amount) + (- balance amount))) + + `Make-decrementer' returns a procedure that subtracts its input from +a designated amount `balance', but there is no accumulated effect over +successive calls, as with `make-simplified-withdraw': + + (define D (make-decrementer 25)) + + (D 20) + 5 + + (D 10) + 15 + + We can use the substitution model to explain how `make-decrementer' +works. For instance, let us analyze the evaluation of the expression + + ((make-decrementer 25) 20) + + We first simplify the operator of the combination by substituting 25 +for `balance' in the body of `make-decrementer'. This reduces the +expression to + + ((lambda (amount) (- 25 amount)) 20) + + Now we apply the operator by substituting 20 for `amount' in the +body of the `lambda' expression: + + (- 25 20) + + The final answer is 5. + + Observe, however, what happens if we attempt a similar substitution +analysis with `make-simplified-withdraw': + + ((make-simplified-withdraw 25) 20) + + We first simplify the operator by substituting 25 for `balance' in +the body of `make-simplified-withdraw'. This reduces the expression +to(1) + + ((lambda (amount) (set! balance (- 25 amount)) 25) 20) + + Now we apply the operator by substituting 20 for `amount' in the +body of the `lambda' expression: + + (set! balance (- 25 20)) 25 + + If we adhered to the substitution model, we would have to say that +the meaning of the procedure application is to first set `balance' to 5 +and then return 25 as the value of the expression. This gets the wrong +answer. In order to get the correct answer, we would have to somehow +distinguish the first occurrence of `balance' (before the effect of the +`set!') from the second occurrence of `balance' (after the effect of +the `set!'), and the substitution model cannot do this. + + The trouble here is that substitution is based ultimately on the +notion that the symbols in our language are essentially names for +values. But as soon as we introduce `set!' and the idea that the value +of a variable can change, a variable can no longer be simply a name. +Now a variable somehow refers to a place where a value can be stored, +and the value stored at this place can change. In section *Note 3-2:: +we will see how environments play this role of "place" in our +computational model. + +Sameness and change +................... + +The issue surfacing here is more profound than the mere breakdown of a +particular model of computation. As soon as we introduce change into +our computational models, many notions that were previously +straightforward become problematical. Consider the concept of two +things being "the same." + + Suppose we call `make-decrementer' twice with the same argument to +create two procedures: + + (define D1 (make-decrementer 25)) + + (define D2 (make-decrementer 25)) + + Are `D1' and `D2' the same? An acceptable answer is yes, because +`D1' and `D2' have the same computational behavior--each is a procedure +that subtracts its input from 25. In fact, `D1' could be substituted +for `D2' in any computation without changing the result. + + Contrast this with making two calls to `make-simplified-withdraw': + + (define W1 (make-simplified-withdraw 25)) + + (define W2 (make-simplified-withdraw 25)) + + Are `W1' and `W2' the same? Surely not, because calls to `W1' and +`W2' have distinct effects, as shown by the following sequence of +interactions: + + (W1 20) + 5 + + (W1 20) + - 15 + + (W2 20) + 5 + + Even though `W1' and `W2' are "equal" in the sense that they are +both created by evaluating the same expression, +`(make-simplified-withdraw 25)', it is not true that `W1' could be +substituted for `W2' in any expression without changing the result of +evaluating the expression. + + A language that supports the concept that "equals can be substituted +for equals" in an expresssion without changing the value of the +expression is said to be "referentially transparent". Referential +transparency is violated when we include `set!' in our computer +language. This makes it tricky to determine when we can simplify +expressions by substituting equivalent expressions. Consequently, +reasoning about programs that use assignment becomes drastically more +difficult. + + Once we forgo referential transparency, the notion of what it means +for computational objects to be "the same" becomes difficult to capture +in a formal way. Indeed, the meaning of "same" in the real world that +our programs model is hardly clear in itself. In general, we can +determine that two apparently identical objects are indeed "the same +one" only by modifying one object and then observing whether the other +object has changed in the same way. But how can we tell if an object +has "changed" other than by observing the "same" object twice and +seeing whether some property of the object differs from one observation +to the next? Thus, we cannot determine "change" without some _a +priori_ notion of "sameness," and we cannot determine sameness without +observing the effects of change. + + As an example of how this issue arises in programming, consider the +situation where Peter and Paul have a bank account with $100 in it. +There is a substantial difference between modeling this as + + (define peter-acc (make-account 100)) + (define paul-acc (make-account 100)) + +and modeling it as + + (define peter-acc (make-account 100)) + (define paul-acc peter-acc) + + In the first situation, the two bank accounts are distinct. +Transactions made by Peter will not affect Paul's account, and vice +versa. In the second situation, however, we have defined `paul-acc' to +be _the same thing_ as `peter-acc'. In effect, Peter and Paul now have +a joint bank account, and if Peter makes a withdrawal from `peter-acc' +Paul will observe less money in `paul-acc'. These two similar but +distinct situations can cause confusion in building computational +models. With the shared account, in particular, it can be especially +confusing that there is one object (the bank account) that has two +different names (`peter-acc' and `paul-acc'); if we are searching for +all the places in our program where `paul-acc' can be changed, we must +remember to look also at things that change `peter-acc'.(2) + + With reference to the above remarks on "sameness" and "change," +observe that if Peter and Paul could only examine their bank balances, +and could not perform operations that changed the balance, then the +issue of whether the two accounts are distinct would be moot. In +general, so long as we never modify data objects, we can regard a +compound data object to be precisely the totality of its pieces. For +example, a rational number is determined by giving its numerator and +its denominator. But this view is no longer valid in the presence of +change, where a compound data object has an "identity" that is +something different from the pieces of which it is composed. A bank +account is still "the same" bank account even if we change the balance +by making a withdrawal; conversely, we could have two different bank +accounts with the same state information. This complication is a +consequence, not of our programming language, but of our perception of +a bank account as an object. We do not, for example, ordinarily regard +a rational number as a changeable object with identity, such that we +could change the numerator and still have "the same" rational number. + +Pitfalls of imperative programming +.................................. + +In contrast to functional programming, programming that makes extensive +use of assignment is known as "imperative programming". In addition to +raising complications about computational models, programs written in +imperative style are susceptible to bugs that cannot occur in functional +programs. For example, recall the iterative factorial program from +section *Note 1-2-1::: + + (define (factorial n) + (define (iter product counter) + (if (> counter n) + product + (iter (* counter product) + (+ counter 1)))) + (iter 1 1)) + + Instead of passing arguments in the internal iterative loop, we +could adopt a more imperative style by using explicit assignment to +update the values of the variables `product' and `counter': + + (define (factorial n) + (let ((product 1) + (counter 1)) + (define (iter) + (if (> counter n) + product + (begin (set! product (* counter product)) + (set! counter (+ counter 1)) + (iter)))) + (iter))) + + This does not change the results produced by the program, but it +does introduce a subtle trap. How do we decide the order of the +assignments? As it happens, the program is correct as written. But +writing the assignments in the opposite order + + (set! counter (+ counter 1)) + (set! product (* counter product)) + +would have produced a different, incorrect result. In general, +programming with assignment forces us to carefully consider the +relative orders of the assignments to make sure that each statement is +using the correct version of the variables that have been changed. +This issue simply does not arise in functional programs.(3) + + The complexity of imperative programs becomes even worse if we +consider applications in which several processes execute concurrently. +We will return to this in section *Note 3-4::. First, however, we will +address the issue of providing a computational model for expressions +that involve assignment, and explore the uses of objects with local +state in designing simulations. + + *Exercise 3.7:* Consider the bank account objects created by + `make-account', with the password modification described in *Note + Exercise 3-3::. Suppose that our banking system requires the + ability to make joint accounts. Define a procedure `make-joint' + that accomplishes this. `Make-joint' should take three arguments. + The first is a password-protected account. The second argument + must match the password with which the account was defined in + order for the `make-joint' operation to proceed. The third + argument is a new password. `Make-joint' is to create an + additional access to the original account using the new password. + For example, if `peter-acc' is a bank account with password + `open-sesame', then + + (define paul-acc + (make-joint peter-acc 'open-sesame 'rosebud)) + + will allow one to make transactions on `peter-acc' using the name + `paul-acc' and the password `rosebud'. You may wish to modify your + solution to *Note Exercise 3-3:: to accommodate this new feature + + *Exercise 3.8:* When we defined the evaluation model in section + *Note 1-1-3::, we said that the first step in evaluating an + expression is to evaluate its subexpressions. But we never + specified the order in which the subexpressions should be + evaluated (e.g., left to right or right to left). When we + introduce assignment, the order in which the arguments to a + procedure are evaluated can make a difference to the result. + Define a simple procedure `f' such that evaluating `(+ (f 0) (f + 1))' will return 0 if the arguments to `+' are evaluated from left + to right but will return 1 if the arguments are evaluated from + right to left. + + ---------- Footnotes ---------- + + (1) We don't substitute for the occurrence of `balance' in the +`set!' expression because the in a `set!' is not evaluated. If +we did substitute for it, we would get `(set! 25 (- 25 amount))', which +makes no sense. + + (2) The phenomenon of a single computational object being accessed +by more than one name is known as "aliasing". The joint bank account +situation illustrates a very simple example of an alias. In section +*Note 3-3:: we will see much more complex examples, such as "distinct" +compound data structures that share parts. Bugs can occur in our +programs if we forget that a change to an object may also, as a "side +effect," change a "different" object because the two "different" +objects are actually a single object appearing under different aliases. +These so-called "side-effect bugs" are so difficult to locate and to +analyze that some people have proposed that programming languages be +designed in such a way as to not allow side effects or aliasing +(Lampson et al. 1981; Morris, Schmidt, and Wadler 1980). + + (3) In view of this, it is ironic that introductory programming is +most often taught in a highly imperative style. This may be a vestige +of a belief, common throughout the 1960s and 1970s, that programs that +call procedures must inherently be less efficient than programs that +perform assignments. (Steele (1977) debunks this argument.) +Alternatively it may reflect a view that step-by-step assignment is +easier for beginners to visualize than procedure call. Whatever the +reason, it often saddles beginning programmers with "should I set this +variable before or after that one" concerns that can complicate +programming and obscure the important ideas. + + +File: sicp.info, Node: 3-2, Next: 3-3, Prev: 3-1, Up: Chapter 3 + +3.2 The Environment Model of Evaluation +======================================= + +When we introduced compound procedures in *Note Chapter 1::, we used the +substitution model of evaluation (section *Note 1-1-5::) to define what +is meant by applying a procedure to arguments: + + * To apply a compound procedure to arguments, evaluate the body of + the procedure with each formal parameter replaced by the + corresponding argument. + + + Once we admit assignment into our programming language, such a +definition is no longer adequate. In particular, section *Note 3-1-3:: +argued that, in the presence of assignment, a variable can no longer be +considered to be merely a name for a value. Rather, a variable must +somehow designate a "place" in which values can be stored. In our new +model of evaluation, these places will be maintained in structures +called "environments". + + An environment is a sequence of "frames". Each frame is a table +(possibly empty) of "bindings", which associate variable names with +their corresponding values. (A single frame may contain at most one +binding for any variable.) Each frame also has a pointer to its environment +"enclosing environment", unless, for the purposes of discussion, the +frame is considered to be "global". The "value of a variable" with +respect to an environment is the value given by the binding of the +variable in the first frame in the environment that contains a binding +for that variable. If no frame in the sequence specifies a binding for +the variable, then the variable is said to be "unbound" in the +environment. + + *Figure 3.1:* A simple environment structure. + + +--------+ + | I | + | x: 3 | + | y: 5 | + +--------+ + ^ ^ + | | + C | | D + +---------+ | | +----------+ + | II | | | | III | + | z: 6 +---+ +---+ m: 1 | + | x: 7 | | y: 2 | + +---------+ +----------+ + + *Note Figure 3-1:: shows a simple environment structure consisting +of three frames, labeled I, II, and III. In the diagram, A, B, C, and +D are pointers to environments. C and D point to the same environment. +The variables `z' and `x' are bound in frame II, while `y' and `x' are +bound in frame I. The value of `x' in environment D is 3. The value +of `x' with respect to environment B is also 3. This is determined as +follows: We examine the first frame in the sequence (frame III) and do +not find a binding for `x', so we proceed to the enclosing environment +D and find the binding in frame I. On the other hand, the value of `x' +in environment A is 7, because the first frame in the sequence (frame +II) contains a binding of `x' to 7. With respect to environment A, the +binding of `x' to 7 in frame II is said to "shadow" the binding of `x' +to 3 in frame I. + + The environment is crucial to the evaluation process, because it +determines the context in which an expression should be evaluated. +Indeed, one could say that expressions in a programming language do +not, in themselves, have any meaning. Rather, an expression acquires a +meaning only with respect to some environment in which it is evaluated. +Even the interpretation of an expression as straightforward as `(+ 1 +1)' depends on an understanding that one is operating in a context in +which `+' is the symbol for addition. Thus, in our model of evaluation +we will always speak of evaluating an expression with respect to some +environment. To describe interactions with the interpreter, we will +suppose that there is a global environment, consisting of a single frame +(with no enclosing environment) that includes values for the symbols +associated with the primitive procedures. For example, the idea that +`+' is the symbol for addition is captured by saying that the symbol +`+' is bound in the global environment to the primitive addition +procedure. + +* Menu: + +* 3-2-1:: The Rules for Evaluation +* 3-2-2:: Applying Simple Procedures +* 3-2-3:: Frames as the Repository of Local State +* 3-2-4:: Internal Definitions + + +File: sicp.info, Node: 3-2-1, Next: 3-2-2, Prev: 3-2, Up: 3-2 + +3.2.1 The Rules for Evaluation +------------------------------ + +The overall specification of how the interpreter evaluates a combination +remains the same as when we first introduced it in section *Note +1-1-3::: + + * To evaluate a combination: + + + 1. Evaluate the subexpressions of the combination.(1) + + 2. Apply the value of the operator subexpression to the values of the + operand subexpressions. + + + The environment model of evaluation replaces the substitution model +in specifying what it means to apply a compound procedure to arguments. + + In the environment model of evaluation, a procedure is always a pair +consisting of some code and a pointer to an environment. Procedures +are created in one way only: by evaluating a `lambda' expression. This +produces a procedure whose code is obtained from the text of the +`lambda' expression and whose environment is the environment in which +the `lambda' expression was evaluated to produce the procedure. For +example, consider the procedure definition + + (define (square x) + (* x x)) + +evaluated in the global environment. The procedure definition syntax +is just syntactic sugar for an underlying implicit `lambda' expression. +It would have been equivalent to have used + + (define square + (lambda (x) (* x x))) + +which evaluates `(lambda (x) (* x x))' and binds `square' to the +resulting value, all in the global environment. + + *Note Figure 3-2:: shows the result of evaluating this `define' +expression. The procedure object is a pair whose code specifies that +the procedure has one formal parameter, namely `x', and a procedure +body `(* x x)'. The environment part of the procedure is a pointer to +the global environment, since that is the environment in which the +`lambda' expression was evaluated to produce the procedure. A new +binding, which associates the procedure object with the symbol +`square', has been added to the global frame. In general, `define' +creates definitions by adding bindings to frames. + + *Figure 3.2:* Environment structure produced by evaluating + `(define (square x) (* x x))' in the global environment. + + +----------------------+ + | other variables | + global --->| | + env | square: --+ | + +-----------|----------+ + | ^ + (define (square x) | | + (* x x)) V | + .---.---. | + | O | O-+---+ + `-|-^---' + | + V + parameters: x + body: (* x x) + + Now that we have seen how procedures are created, we can describe how +procedures are applied. The environment model specifies: To apply a +procedure to arguments, create a new environment containing a frame +that binds the parameters to the values of the arguments. The +enclosing environment of this frame is the environment specified by the +procedure. Now, within this new environment, evaluate the procedure +body. + + To show how this rule is followed, *Note Figure 3-3:: illustrates +the environment structure created by evaluating the expression `(square +5)' in the global environment, where `square' is the procedure +generated in *Note Figure 3-2::. Applying the procedure results in the +creation of a new environment, labeled E1 in the figure, that begins +with a frame in which `x', the formal parameter for the procedure, is +bound to the argument 5. The pointer leading upward from this frame +shows that the frame's enclosing environment is the global environment. +The global environment is chosen here, because this is the environment +that is indicated as part of the `square' procedure object. Within E1, +we evaluate the body of the procedure, `(* x x)'. Since the value of +`x' in E1 is 5, the result is `(* 5 5)', or 25. + + *Figure 3.3:* Environment created by evaluating `(square 5)' in + the global environment. + + +------------------------------------+ + | other variables | + global -->| | + env | square: --+ | + +-----------|---------------------+--+ + | ^ ^ + (square 5) | | | + V | | + .---.---. | +---+--+ + | O | O-+---+ E1 -->| x: 5 | + `-|-^---' +------+ + | + V + parameters: x + body: (* x x) + + The environment model of procedure application can be summarized by +two rules: + + * A procedure object is applied to a set of arguments by + constructing a frame, binding the formal parameters of the + procedure to the arguments of the call, and then evaluating the + body of the procedure in the context of the new environment + constructed. The new frame has as its enclosing environment the + environment part of the procedure object being applied. + + * A procedure is created by evaluating a `lambda' expression + relative to a given environment. The resulting procedure object + is a pair consisting of the text of the `lambda' expression and a + pointer to the environment in which the procedure was created. + + + We also specify that defining a symbol using `define' creates a +binding in the current environment frame and assigns to the symbol the +indicated value.(2) Finally, we specify the behavior of `set!', the +operation that forced us to introduce the environment model in the +first place. Evaluating the expression `(set! )' in +some environment locates the binding of the variable in the environment +and changes that binding to indicate the new value. That is, one finds +the first frame in the environment that contains a binding for the +variable and modifies that frame. If the variable is unbound in the +environment, then `set!' signals an error. + + These evaluation rules, though considerably more complex than the +substitution model, are still reasonably straightforward. Moreover, +the evaluation model, though abstract, provides a correct description +of how the interpreter evaluates expressions. In *Note Chapter 4:: we +shall see how this model can serve as a blueprint for implementing a +working interpreter. The following sections elaborate the details of +the model by analyzing some illustrative programs. + + ---------- Footnotes ---------- + + (1) ssignment introduces a subtlety into step 1 of the evaluation +rule. As shown in *Note Exercise 3-8::, the presence of assignment +allows us to write expressions that will produce different values +depending on the order in which the subexpressions in a combination are +evaluated. Thus, to be precise, we should specify an evaluation order +in step 1 (e.g., left to right or right to left). However, this order +should always be considered to be an implementation detail, and one +should never write programs that depend on some particular order. For +instance, a sophisticated compiler might optimize a program by varying +the order in which subexpressions are evaluated. + + (2) If there is already a binding for the variable in the current +frame, then the binding is changed. This is convenient because it +allows redefinition of symbols; however, it also means that `define' +can be used to change values, and this brings up the issues of +assignment without explicitly using `set!'. Because of this, some +people prefer redefinitions of existing symbols to signal errors or +warnings. + + +File: sicp.info, Node: 3-2-2, Next: 3-2-3, Prev: 3-2-1, Up: 3-2 + +3.2.2 Applying Simple Procedures +-------------------------------- + +When we introduced the substitution model in section *Note 1-1-5:: we +showed how the combination `(f 5)' evaluates to 136, given the +following procedure definitions: + + (define (square x) + (* x x)) + + (define (sum-of-squares x y) + (+ (square x) (square y))) + + (define (f a) + (sum-of-squares (+ a 1) (* a 2))) + + We can analyze the same example using the environment model. *Note +Figure 3-4:: shows the three procedure objects created by evaluating +the definitions of `f', `square', and `sum-of-squares' in the global +environment. Each procedure object consists of some code, together +with a pointer to the global environment. + + *Figure 3.4:* Procedure objects in the global frame. + + +--------------------------------------------+ + | sum-of-squares: | + global -->| square: | + env | f: --+ | + +------|--------------+--------------+-------+ + | ^ | ^ | ^ + | | | | | | + V | V | V | + .---.---. | .---.---. | .---.---. | + | O | O-+-+ | O | O-+-+ | O | O-+-+ + `-|-^---' `-|-^---' `-|-^---' + | | | + V V V + parameters: a parameters: x parameters: x, y + body: (sum-of-squares body: (* x x) body: (+ (square x) + (+ a 1) (square y)) + (* a 2)) + + In *Note Figure 3-5:: we see the environment structure created by +evaluating the expression `(f 5)'. The call to `f' creates a new +environment E1 beginning with a frame in which `a', the formal +parameter of `f', is bound to the argument 5. In E1, we evaluate the +body of `f': + + (sum-of-squares (+ a 1) (* a 2)) + + *Figure 3.5:* Environments created by evaluating `(f 5)' using the + procedures in *Note Figure 3-4::. + + +-----------------------------------------------------+ + global -->| | + env +-----------------------------------------------------+ + ^ ^ ^ ^ + (f 5) | | | | + +------+ +-------+ +------+ +-------+ + E1 -->| a: 5 | E2 ->| x: 6 | E3 -->| x: 6 | E4 -->| x: 10 | + | | | y: 10 | | | | | + +------+ +-------+ +------+ +-------+ + (sum-of-squares (+ (square x) (* x x) (* x x) + (+ a 1) (square u)) + (+ a 2)) + + To evaluate this combination, we first evaluate the subexpressions. +The first subexpression, `sum-of-squares', has a value that is a +procedure object. (Notice how this value is found: We first look in +the first frame of E1, which contains no binding for `sum-of-squares'. +Then we proceed to the enclosing environment, i.e. the global +environment, and find the binding shown in *Note Figure 3-4::.) The +other two subexpressions are evaluated by applying the primitive +operations `+' and `*' to evaluate the two combinations `(+ a 1)' and +`(* a 2)' to obtain 6 and 10, respectively. + + Now we apply the procedure object `sum-of-squares' to the arguments +6 and 10. This results in a new environment E2 in which the formal +parameters `x' and `y' are bound to the arguments. Within E2 we +evaluate the combination `(+ (square x) (square y))'. This leads us to +evaluate `(square x)', where `square' is found in the global frame and +`x' is 6. Once again, we set up a new environment, E3, in which `x' is +bound to 6, and within this we evaluate the body of `square', which is +`(* x x)'. Also as part of applying `sum-of-squares', we must evaluate +the subexpression `(square y)', where `y' is 10. This second call to +`square' creates another environment, E4, in which `x', the formal +parameter of `square', is bound to 10. And within E4 we must evaluate +`(* x x)'. + + The important point to observe is that each call to `square' creates +a new environment containing a binding for `x'. We can see here how the +different frames serve to keep separate the different local variables +all named `x'. Notice that each frame created by `square' points to +the global environment, since this is the environment indicated by the +`square' procedure object. + + After the subexpressions are evaluated, the results are returned. +The values generated by the two calls to `square' are added by +`sum-of-squares', and this result is returned by `f'. Since our focus +here is on the environment structures, we will not dwell on how these +returned values are passed from call to call; however, this is also an +important aspect of the evaluation process, and we will return to it in +detail in *Note Chapter 5::. + + *Exercise 3.9:* In section *Note 1-2-1:: we used the substitution + model to analyze two procedures for computing factorials, a + recursive version + + (define (factorial n) + (if (= n 1) + 1 + (* n (factorial (- n 1))))) + + and an iterative version + + (define (factorial n) + (fact-iter 1 1 n)) + + (define (fact-iter product counter max-count) + (if (> counter max-count) + product + (fact-iter (* counter product) + (+ counter 1) + max-count))) + + Show the environment structures created by evaluating `(factorial + 6)' using each version of the `factorial' procedure.(1) + + ---------- Footnotes ---------- + + (1) The environment model will not clarify our claim in section +*Note 1-2-1:: that the interpreter can execute a procedure such as +`fact-iter' in a constant amount of space using tail recursion. We +will discuss tail recursion when we deal with the control structure of +the interpreter in section *Note 5-4::. + + +File: sicp.info, Node: 3-2-3, Next: 3-2-4, Prev: 3-2-2, Up: 3-2 + +3.2.3 Frames as the Repository of Local State +--------------------------------------------- + +We can turn to the environment model to see how procedures and +assignment can be used to represent objects with local state. As an +example, consider the "withdrawal processor" from section *Note 3-1-1:: +created by calling the procedure + + (define (make-withdraw balance) + (lambda (amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds"))) + + Let us describe the evaluation of + + (define W1 (make-withdraw 100)) + +followed by + + (W1 50) + 50 + + *Note Figure 3-6:: shows the result of defining the `make-withdraw' +procedure in the global environment. This produces a procedure object +that contains a pointer to the global environment. So far, this is no +different from the examples we have already seen, except that the body +of the procedure is itself a `lambda' expression. + + *Figure 3.6:* Result of defining `make-withdraw' in the global + environment. + + +---------------------------+ + global -->| make-withdraw: --+ | + env +------------------|--------+ + | ^ + V | + .---.---. | + | O | O-+--+ + `-|-^---' + | + V + parameters: balance + body: (lambda (amount) + (if (>= balance amount) + (begin (set! balance + (- balance amount)) + balance) + "Insufficient funds")) + + The interesting part of the computation happens when we apply the +procedure `make-withdraw' to an argument: + + (define W1 (make-withdraw 100)) + + We begin, as usual, by setting up an environment E1 in which the +formal parameter `balance' is bound to the argument 100. Within this +environment, we evaluate the body of `make-withdraw', namely the +`lambda' expression. This constructs a new procedure object, whose code +is as specified by the `lambda' and whose environment is E1, the +environment in which the `lambda' was evaluated to produce the +procedure. The resulting procedure object is the value returned by the +call to `make-withdraw'. This is bound to `W1' in the global +environment, since the `define' itself is being evaluated in the global +environment. *Note Figure 3-7:: shows the resulting environment +structure. + + *Figure 3.7:* Result of evaluating `(define W1 (make-withdraw + 100))'. + + +-----------------------------------------------+ + | make-withdraw: -----------------------+ | + global -->| | | + | W1: --+ | | + +-------|-------------------------------|-------+ + | ^ | ^ + | | V | + | +-------+------+ .---.---. | + | E1 -->| balance: 100 | | O | O-+-+ + | +--------------+ `-|-^---' + V ^ | + .---.---. | V + +-+-O | O-+------------+ parameters: balance + | `---^---' body: ... + V + parameters: amount + body: (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds") + + Now we can analyze what happens when `W1' is applied to an argument: + + (W1 50) + 50 + + We begin by constructing a frame in which `amount', the formal +parameter of `W1', is bound to the argument 50. The crucial point to +observe is that this frame has as its enclosing environment not the +global environment, but rather the environment E1, because this is the +environment that is specified by the `W1' procedure object. Within +this new environment, we evaluate the body of the procedure: + + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds") + + The resulting environment structure is shown in *Note Figure 3-8::. +The expression being evaluated references both `amount' and `balance'. +`Amount' will be found in the first frame in the environment, while +`balance' will be found by following the enclosing-environment pointer +to E1. + + *Figure 3.8:* Environments created by applying the procedure + object `W1'. + + +---------------------------------------------------+ + | make-withdraw: ... | + global -->| | + env | W1: --+ | + +-------|-------------------------------------------+ + | ^ + | | + | +-------+------+ Here is the balance + | E1 -->| balance: 100 | that will be changed + | +--------------+ by the set!. + V ^ ^ + .---.---. | +----+ + | O | O-+-----------+ | + `-|-^---' +------+-----+ + | | amount: 50 | + V +------------+ + parameters: amount (if (>= balance amount) + body: ... (begin (set! balance + (- balance amount)) + balance) + "Insufficient funds") + + When the `set!' is executed, the binding of `balance' in E1 is +changed. At the completion of the call to `W1', `balance' is 50, and +the frame that contains `balance' is still pointed to by the procedure +object `W1'. The frame that binds `amount' (in which we executed the +code that changed `balance') is no longer relevant, since the procedure +call that constructed it has terminated, and there are no pointers to +that frame from other parts of the environment. The next time `W1' is +called, this will build a new frame that binds `amount' and whose +enclosing environment is E1. We see that E1 serves as the "place" that +holds the local state variable for the procedure object `W1'. *Note +Figure 3-9:: shows the situation after the call to `W1'. + + *Figure 3.9:* Environments after the call to `W1'. + + +------------------------------------+ + | make-withdraw: ... | + global --->| | + env | W1: --+ | + +-------|----------------------------+ + | ^ + | | + | +------+------+ + | E1 --->| balance: 50 | + | +-------------+ + V ^ + .---.---. | + | O | O-+---------------+ + `-|-^---' + | + V + parameters: amount + body: ... + + Observe what happens when we create a second "withdraw" object by +making another call to `make-withdraw': + + (define W2 (make-withdraw 100)) + + This produces the environment structure of *Note Figure 3-10::, +which shows that `W2' is a procedure object, that is, a pair with some +code and an environment. The environment E2 for `W2' was created by +the call to `make-withdraw'. It contains a frame with its own local +binding for `balance'. On the other hand, `W1' and `W2' have the same +code: the code specified by the `lambda' expression in the body of +`make-withdraw'.(1) We see here why `W1' and `W2' behave as independent +objects. Calls to `W1' reference the state variable `balance' stored +in E1, whereas calls to `W2' reference the `balance' stored in E2. +Thus, changes to the local state of one object do not affect the other +object. + + *Figure 3.10:* Using `(define W2 (make-withdraw 100))' to create a + second object. + + +-------------------------------------------------+ + | make-withdraw: ... | + global ->| W2: ---------------------------+ | + env | W1: --+ | | + +-------|------------------------|----------------+ + | ^ | ^ + | | | | + | +------+------+ | +------+-------+ + | E1 ->| balance: 50 | | E2 ->| balance: 100 | + | +-------------+ | +--------------+ + V ^ V ^ + .---.---. | .---.---. | + | O | O-+----------+ | O | O-+----------+ + `-|-^---' `-|-^---' + | +----------------------+ + V V + parameters: amount + body: ... + + *Exercise 3.10:* In the `make-withdraw' procedure, the local + variable `balance' is created as a parameter of `make-withdraw'. + We could also create the local state variable explicitly, using + `let', as follows: + + (define (make-withdraw initial-amount) + (let ((balance initial-amount)) + (lambda (amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")))) + + Recall from section *Note 1-3-2:: that `let' is simply syntactic + sugar for a procedure call: + + (let (( )) ) + + is interpreted as an alternate syntax for + + ((lambda () ) ) + + Use the environment model to analyze this alternate version of + `make-withdraw', drawing figures like the ones above to illustrate + the interactions + + (define W1 (make-withdraw 100)) + + (W1 50) + + (define W2 (make-withdraw 100)) + + Show that the two versions of `make-withdraw' create objects with + the same behavior. How do the environment structures differ for + the two versions? + + ---------- Footnotes ---------- + + (1) Whether `W1' and `W2' share the same physical code stored in the +computer, or whether they each keep a copy of the code, is a detail of +the implementation. For the interpreter we implement in *Note Chapter +4::, the code is in fact shared. + + +File: sicp.info, Node: 3-2-4, Prev: 3-2-3, Up: 3-2 + +3.2.4 Internal Definitions +-------------------------- + +Section *Note 1-1-8:: introduced the idea that procedures can have +internal definitions, thus leading to a block structure as in the +following procedure to compute square roots: + + (define (sqrt x) + (define (good-enough? guess) + (< (abs (- (square guess) x)) 0.001)) + (define (improve guess) + (average guess (/ x guess))) + (define (sqrt-iter guess) + (if (good-enough? guess) + guess + (sqrt-iter (improve guess)))) + (sqrt-iter 1.0)) + + Now we can use the environment model to see why these internal +definitions behave as desired. *Note Figure 3-11:: shows the point in +the evaluation of the expression `(sqrt 2)' where the internal +procedure `good-enough?' has been called for the first time with +`guess' equal to 1. + + *Figure 3.11:* `Sqrt' procedure with internal definitions. + + +--------------------------------------------------+ + global -->| sqrt: --+ | + env | | | + +---------|----------------------------------------+ + V ^ ^ + .---.---. | | + +----------+-O | O-+---+ +----------+------------+ + | `---^---' | x: 2 | + V E1 -->| good-enough?: -+ | + parameters: x | improve: ... | | + body: (define good-enough? ...) | sqrt-iter: ... | | + (define improve ...) +----------------|------+ + (define sqrt-iter ...) ^ ^ | ^ + (sqrt-iter 1.0) | | V | + +---------++ | .---.---. | + E2 -->| guess: 1 | | | O | O-+-+ + +----------+ | `-|-^---' + call to sqrt-iter | | + | V + +---------++ parameters: guess + E3 -->| guess: 1 | body: (< (abs ...) + +----------+ ...) + call to good-enough? + + Observe the structure of the environment. `Sqrt' is a symbol in the +global environment that is bound to a procedure object whose associated +environment is the global environment. When `sqrt' was called, a new +environment E1 was formed, subordinate to the global environment, in +which the parameter `x' is bound to 2. The body of `sqrt' was then +evaluated in E1. Since the first expression in the body of `sqrt' is + + (define (good-enough? guess) + (< (abs (- (square guess) x)) 0.001)) + +evaluating this expression defined the procedure `good-enough?' in the +environment E1. To be more precise, the symbol `good-enough?' was added +to the first frame of E1, bound to a procedure object whose associated +environment is E1. Similarly, `improve' and `sqrt-iter' were defined +as procedures in E1. For conciseness, *Note Figure 3-11:: shows only +the procedure object for `good-enough?'. + + After the local procedures were defined, the expression `(sqrt-iter +1.0)' was evaluated, still in environment E1. So the procedure object +bound to `sqrt-iter' in E1 was called with 1 as an argument. This +created an environment E2 in which `guess', the parameter of +`sqrt-iter', is bound to 1. `Sqrt-iter' in turn called `good-enough?' +with the value of `guess' (from E2) as the argument for `good-enough?'. +This set up another environment, E3, in which `guess' (the parameter of +`good-enough?') is bound to 1. Although `sqrt-iter' and `good-enough?' +both have a parameter named `guess', these are two distinct local +variables located in different frames. Also, E2 and E3 both have E1 as +their enclosing environment, because the `sqrt-iter' and `good-enough?' +procedures both have E1 as their environment part. One consequence of +this is that the symbol `x' that appears in the body of `good-enough?' +will reference the binding of `x' that appears in E1, namely the value +of `x' with which the original `sqrt' procedure was called. + + The environment model thus explains the two key properties that make +local procedure definitions a useful technique for modularizing +programs: + + * The names of the local procedures do not interfere with names + external to the enclosing procedure, because the local procedure + names will be bound in the frame that the procedure creates when + it is run, rather than being bound in the global environment. + + * The local procedures can access the arguments of the enclosing + procedure, simply by using parameter names as free variables. + This is because the body of the local procedure is evaluated in an + environment that is subordinate to the evaluation environment for + the enclosing procedure. + + + *Exercise 3.11:* In section *Note 3-2-3:: we saw how the + environment model described the behavior of procedures with local + state. Now we have seen how internal definitions work. A typical + message-passing procedure contains both of these aspects. + Consider the bank account procedure of section *Note 3-1-1::: + + (define (make-account balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + (define (dispatch m) + (cond ((eq? m 'withdraw) withdraw) + ((eq? m 'deposit) deposit) + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch) + + Show the environment structure generated by the sequence of + interactions + + (define acc (make-account 50)) + + ((acc 'deposit) 40) + 90 + + ((acc 'withdraw) 60) + 30 + + Where is the local state for `acc' kept? Suppose we define another + account + + (define acc2 (make-account 100)) + + How are the local states for the two accounts kept distinct? + Which parts of the environment structure are shared between `acc' + and `acc2'? + + +File: sicp.info, Node: 3-3, Next: 3-4, Prev: 3-2, Up: Chapter 3 + +3.3 Modeling with Mutable Data +============================== + +Chapter 2 dealt with compound data as a means for constructing +computational objects that have several parts, in order to model +real-world objects that have several aspects. In that chapter we +introduced the discipline of data abstraction, according to which data +structures are specified in terms of constructors, which create data +objects, and selectors, which access the parts of compound data +objects. But we now know that there is another aspect of data that +*Note Chapter 2:: did not address. The desire to model systems +composed of objects that have changing state leads us to the need to +modify compound data objects, as well as to construct and select from +them. In order to model compound objects with changing state, we will +design data abstractions to include, in addition to selectors and +constructors, operations called "mutators", which modify data objects. +For instance, modeling a banking system requires us to change account +balances. Thus, a data structure for representing bank accounts might +admit an operation + + (set-balance! ) + +that changes the balance of the designated account to the designated +new value. Data objects for which mutators are defined are known as objects +"mutable data objects". + + *Note Chapter 2:: introduced pairs as a general-purpose "glue" for +synthesizing compound data. We begin this section by defining basic +mutators for pairs, so that pairs can serve as building blocks for +constructing mutable data objects. These mutators greatly enhance the +representational power of pairs, enabling us to build data structures +other than the sequences and trees that we worked with in section *Note +2-2::. We also present some examples of simulations in which complex +systems are modeled as collections of objects with local state. + +* Menu: + +* 3-3-1:: Mutable List Structure +* 3-3-2:: Representing Queues +* 3-3-3:: Representing Tables +* 3-3-4:: A Simulator for Digital Circuits +* 3-3-5:: Propagation of Constraints + + +File: sicp.info, Node: 3-3-1, Next: 3-3-2, Prev: 3-3, Up: 3-3 + +3.3.1 Mutable List Structure +---------------------------- + +The basic operations on pairs--`cons', `car', and `cdr'--can be used to +construct list structure and to select parts from list structure, but +they are incapable of modifying list structure. The same is true of the +list operations we have used so far, such as `append' and `list', since +these can be defined in terms of `cons', `car', and `cdr'. To modify +list structures we need new operations. + + *Figure 3.12:* Lists `x': `((a b) c d)' and `y': `(e f)'. + + +---+---+ +---+---+ +---+---+ + x -->| * | *-+---->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ + | V V + | +---+ +---+ + | | c | | d | + | +---+ +---+ + | +---+---+ +---+---+ + +---------->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ + V V + +---+ +---+ + | a | | b | + +---+ +---+ + +---+---+ +---+---+ + y -->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ + V V + +---+ +---+ + | e | | f | + +---+ +---+ + + *Figure 3.13:* Effect of `(set-car! x y)' on the lists in *Note + Figure 3-12::. + + +---+---+ +---+---+ +---+---+ + x -->| * | *-+---->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ + | V V + | +---+ +---+ + | | c | | d | + | +---+ +---+ + | +---+---+ +---+---+ + | | * | *-+---->| * | / | + | +-|-+---+ +-|-+---+ + | V V + | +---+ +---+ + | | a | | b | + | +---+ +---+ + +---------->+---+---+ +---+---+ + | * | *-+---->| * | / | + y -->+-|-+---+ +-|-+---+ + V V + +---+ +---+ + | e | | f | + +---+ +---+ + + *Figure 3.14:* Effect of `(define z (cons y (cdr x)))' on the + lists in *Note Figure 3-12::. + + +---+---+ +---+---+ +---+---+ + x -->| * | *-+---->| * | *-+---->| * | / | + +-|-+---+ +-->+-|-+---+ +-|-+---+ + | | V V + | | +---+ +---+ + | | | c | | d | + | | +---+ +---+ + | | +---+---+ +---+---+ + +-------+-->| * | *-+---->| * | / | + | +-|-+---+ +-|-+---+ + +---+---+ | V V + z -->| * | *-+-+ +---+ +---+ + +-|-+---+ | a | | b | + | +---+ +---+ + +---------->+---+---+ +---+---+ + | * | *-+---->| * | / | + y -->+-|-+---+ +-|-+---+ + V V + +---+ +---+ + | e | | f | + +---+ +---+ + + *Figure 3.15:* Effect of `(set-cdr! x y)' on the lists in *Note + Figure 3-12::. + + +---+---+ +---+---+ +---+---+ + x -->| * | * | | * | *-+---->| * | / | + +-|-+-|-+ +-|-+---+ +-|-+---+ + | | V V + | | +---+ +---+ + | | | c | | d | + | | +---+ +---+ + | | +---+---+ +---+---+ + +---+------>| * | *-+---->| * | / | + | +-|-+---+ +-|-+---+ + | V V + | +---+ +---+ + | | a | | b | + | +---+ +---+ + +------>+---+---+ +---+---+ + | * | *-+---->| * | / | + y -->+-|-+---+ +-|-+---+ + V V + +---+ +---+ + | e | | f | + +---+ +---+ + + The primitive mutators for pairs are `set-car!' and `set-cdr!'. +`Set-car!' takes two arguments, the first of which must be a pair. It +modifies this pair, replacing the `car' pointer by a pointer to the +second argument of `set-car!'.(1) + + As an example, suppose that `x' is bound to the list `((a b) c d)' +and `y' to the list `(e f)' as illustrated in *Note Figure 3-12::. +Evaluating the expression ` (set-car! x y)' modifies the pair to which +`x' is bound, replacing its `car' by the value of `y'. The result of +the operation is shown in *Note Figure 3-13::. The structure `x' has +been modified and would now be printed as `((e f) c d)'. The pairs +representing the list `(a b)', identified by the pointer that was +replaced, are now detached from the original structure.(2) + + Compare *Note Figure 3-13:: with *Note Figure 3-14::, which +illustrates the result of executing `(define z (cons y (cdr x)))' with +`x' and `y' bound to the original lists of *Note Figure 3-12::. The +variable `z' is now bound to a new pair created by the `cons' +operation; the list to which `x' is bound is unchanged. + + The `set-cdr!' operation is similar to `set-car!'. The only +difference is that the `cdr' pointer of the pair, rather than the `car' +pointer, is replaced. The effect of executing `(set-cdr! x y)' on the +lists of *Note Figure 3-12:: is shown in *Note Figure 3-15::. Here the +`cdr' pointer of `x' has been replaced by the pointer to `(e f)'. +Also, the list `(c d)', which used to be the `cdr' of `x', is now +detached from the structure. + + `Cons' builds new list structure by creating new pairs, while +`set-car!' and `set-cdr!' modify existing pairs. Indeed, we could +implement `cons' in terms of the two mutators, together with a procedure +`get-new-pair', which returns a new pair that is not part of any +existing list structure. We obtain the new pair, set its `car' and +`cdr' pointers to the designated objects, and return the new pair as +the result of the `cons'.(3) + + (define (cons x y) + (let ((new (get-new-pair))) + (set-car! new x) + (set-cdr! new y) + new)) + + *Exercise 3.12:* The following procedure for appending lists was + introduced in section *Note 2-2-1::: + + (define (append x y) + (if (null? x) + y + (cons (car x) (append (cdr x) y)))) + + `Append' forms a new list by successively `cons'ing the elements of + `x' onto `y'. The procedure `append!' is similar to `append', but + it is a mutator rather than a constructor. It appends the lists + by splicing them together, modifying the final pair of `x' so that + its `cdr' is now `y'. (It is an error to call `append!' with an + empty `x'.) + + (define (append! x y) + (set-cdr! (last-pair x) y) + x) + + Here `last-pair' is a procedure that returns the last pair in its + argument: + + (define (last-pair x) + (if (null? (cdr x)) + x + (last-pair (cdr x)))) + + Consider the interaction + + (define x (list 'a 'b)) + + (define y (list 'c 'd)) + + (define z (append x y)) + + z + (a b c d) + + (cdr x) + + + (define w (append! x y)) + + w + (a b c d) + + (cdr x) + + + What are the missing s? Draw box-and-pointer diagrams to + explain your answer. + + *Exercise 3.13:* Consider the following `make-cycle' procedure, + which uses the `last-pair' procedure defined in *Note Exercise + 3-12::: + + (define (make-cycle x) + (set-cdr! (last-pair x) x) + x) + + Draw a box-and-pointer diagram that shows the structure `z' + created by + + (define z (make-cycle (list 'a 'b 'c))) + + What happens if we try to compute `(last-pair z)'? + + *Exercise 3.14:* The following procedure is quite useful, although + obscure: + + (define (mystery x) + (define (loop x y) + (if (null? x) + y + (let ((temp (cdr x))) + (set-cdr! x y) + (loop temp x)))) + (loop x '())) + + `Loop' uses the "temporary" variable `temp' to hold the old value + of the `cdr' of `x', since the `set-cdr!' on the next line + destroys the `cdr'. Explain what `mystery' does in general. + Suppose `v' is defined by `(define v (list 'a 'b 'c 'd))'. Draw the + box-and-pointer diagram that represents the list to which `v' is + bound. Suppose that we now evaluate `(define w (mystery v))'. Draw + box-and-pointer diagrams that show the structures `v' and `w' after + evaluating this expression. What would be printed as the values + of `v' and `w'? + +Sharing and identity +.................... + +We mentioned in section *Note 3-1-3:: the theoretical issues of +"sameness" and "change" raised by the introduction of assignment. +These issues arise in practice when individual pairs are "shared" among +different data objects. For example, consider the structure formed by + + (define x (list 'a 'b)) + (define z1 (cons x x)) + + As shown in *Note Figure 3-16::, `z1' is a pair whose `car' and +`cdr' both point to the same pair `x'. This sharing of `x' by the +`car' and `cdr' of `z1' is a consequence of the straightforward way in +which `cons' is implemented. In general, using `cons' to construct +lists will result in an interlinked structure of pairs in which many +individual pairs are shared by many different structures. + + *Figure 3.16:* The list `z1' formed by `(cons x x)'. + + +---+---+ + z1 -->| * | * | + +-|-+-|-+ + V V + +---+---+ +---+---+ + x -->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ + V V + +---+ +---+ + | a | | b | + +---+ +---+ + + *Figure 3.17:* The list `z2' formed by `(cons (list 'a 'b) (list + 'a 'b))'. + + +---+---+ +---+---+ +---+---+ + z2 -->| * | *-+---->| * | *-+---->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ + | V V + | +---+ +---+ + | | a | | b | + | +---+ +---+ + | ^ ^ + | | | + | +-|-+---+ +-|-+---+ + +---------->| * | *-+---->| * | / | + +---+---+ +---+---+ + + In contrast to *Note Figure 3-16::, *Note Figure 3-17:: shows the +structure created by + + (define z2 (cons (list 'a 'b) (list 'a 'b))) + + In this structure, the pairs in the two `(a b)' lists are distinct, +although the actual symbols are shared.(4) + + When thought of as a list, `z1' and `z2' both represent "the same" +list, `((a b) a b)'. In general, sharing is completely undetectable if +we operate on lists using only `cons', `car', and `cdr'. However, if +we allow mutators on list structure, sharing becomes significant. As an +example of the difference that sharing can make, consider the following +procedure, which modifies the `car' of the structure to which it is +applied: + + (define (set-to-wow! x) + (set-car! (car x) 'wow) + x) + + Even though `z1' and `z2' are "the same" structure, applying +`set-to-wow!' to them yields different results. With `z1', altering +the `car' also changes the `cdr', because in `z1' the `car' and the +`cdr' are the same pair. With `z2', the `car' and `cdr' are distinct, +so `set-to-wow!' modifies only the `car': + + z1 + ((a b) a b) + + (set-to-wow! z1) + ((wow b) wow b) + + z2 + ((a b) a b) + + (set-to-wow! z2) + ((wow b) a b) + + One way to detect sharing in list structures is to use the predicate +`eq?', which we introduced in section *Note 2-3-1:: as a way to test +whether two symbols are equal. More generally, `(eq? x y)' tests +whether `x' and `y' are the same object (that is, whether `x' and `y' +are equal as pointers). Thus, with `z1' and `z2' as defined in figures +*Note Figure 3-16:: and *Note Figure 3-17::, `(eq? (car z1) (cdr z1))' +is true and `(eq? (car z2) (cdr z2))' is false. + + As will be seen in the following sections, we can exploit sharing to +greatly extend the repertoire of data structures that can be +represented by pairs. On the other hand, sharing can also be +dangerous, since modifications made to structures will also affect +other structures that happen to share the modified parts. The mutation +operations `set-car!' and `set-cdr!' should be used with care; unless +we have a good understanding of how our data objects are shared, +mutation can have unanticipated results.(5) + + *Exercise 3.15:* Draw box-and-pointer diagrams to explain the + effect of `set-to-wow!' on the structures `z1' and `z2' above. + + *Exercise 3.16:* Ben Bitdiddle decides to write a procedure to + count the number of pairs in any list structure. "It's easy," he + reasons. "The number of pairs in any structure is the number in + the `car' plus the number in the `cdr' plus one more to count the + current pair." So Ben writes the following procedure: + + (define (count-pairs x) + (if (not (pair? x)) + 0 + (+ (count-pairs (car x)) + (count-pairs (cdr x)) + 1))) + + Show that this procedure is not correct. In particular, draw + box-and-pointer diagrams representing list structures made up of + exactly three pairs for which Ben's procedure would return 3; + return 4; return 7; never return at all. + + *Exercise 3.17:* Devise a correct version of the `count-pairs' + procedure of *Note Exercise 3-16:: that returns the number of + distinct pairs in any structure. (Hint: Traverse the structure, + maintaining an auxiliary data structure that is used to keep track + of which pairs have already been counted.) + + *Exercise 3.18:* Write a procedure that examines a list and + determines whether it contains a cycle, that is, whether a program + that tried to find the end of the list by taking successive `cdr's + would go into an infinite loop. *Note Exercise 3-13:: constructed + such lists. + + *Exercise 3.19:* Redo *Note Exercise 3-18:: using an algorithm + that takes only a constant amount of space. (This requires a very + clever idea.) + +Mutation is just assignment +........................... + +When we introduced compound data, we observed in section *Note 2-1-3:: +that pairs can be represented purely in terms of procedures: + + (define (cons x y) + (define (dispatch m) + (cond ((eq? m 'car) x) + ((eq? m 'cdr) y) + (else (error "Undefined operation -- CONS" m)))) + dispatch) + + (define (car z) (z 'car)) + + (define (cdr z) (z 'cdr)) + + The same observation is true for mutable data. We can implement +mutable data objects as procedures using assignment and local state. +For instance, we can extend the above pair implementation to handle +`set-car!' and `set-cdr!' in a manner analogous to the way we +implemented bank accounts using `make-account' in section *Note 3-1-1::: + + (define (cons x y) + (define (set-x! v) (set! x v)) + (define (set-y! v) (set! y v)) + (define (dispatch m) + (cond ((eq? m 'car) x) + ((eq? m 'cdr) y) + ((eq? m 'set-car!) set-x!) + ((eq? m 'set-cdr!) set-y!) + (else (error "Undefined operation -- CONS" m)))) + dispatch) + + (define (car z) (z 'car)) + + (define (cdr z) (z 'cdr)) + + (define (set-car! z new-value) + ((z 'set-car!) new-value) + z) + + (define (set-cdr! z new-value) + ((z 'set-cdr!) new-value) + z) + + Assignment is all that is needed, theoretically, to account for the +behavior of mutable data. As soon as we admit `set!' to our language, +we raise all the issues, not only of assignment, but of mutable data in +general.(6) + + *Exercise 3.20:* Draw environment diagrams to illustrate the + evaluation of the sequence of expressions + + (define x (cons 1 2)) + (define z (cons x x)) + (set-car! (cdr z) 17) + + (car x) + 17 + + using the procedural implementation of pairs given above. (Compare + *Note Exercise 3-11::.) + + ---------- Footnotes ---------- + + (1) `Set-car!' and `set-cdr!' return implementation-dependent +values. Like `set!', they should be used only for their effect. + + (2) We see from this that mutation operations on lists can create +"garbage" that is not part of any accessible structure. We will see in +section *Note 5-3-2:: that Lisp memory-management systems include a "garbage +collector", which identifies and recycles the memory space used by +unneeded pairs. + + (3) `Get-new-pair' is one of the operations that must be implemented +as part of the memory management required by a Lisp implementation. We +will discuss this in section *Note 5-3-1::. + + (4) The two pairs are distinct because each call to `cons' returns a +new pair. The symbols are shared; in Scheme there is a unique symbol +with any given name. Since Scheme provides no way to mutate a symbol, +this sharing is undetectable. Note also that the sharing is what +enables us to compare symbols using `eq?', which simply checks equality +of pointers. + + (5) The subtleties of dealing with sharing of mutable data objects +reflect the underlying issues of "sameness" and "change" that were +raised in section *Note 3-1-3::. We mentioned there that admitting +change to our language requires that a compound object must have an +"identity" that is something different from the pieces from which it is +composed. In Lisp, we consider this "identity" to be the quality that +is tested by `eq?', i.e., by equality of pointers. Since in most Lisp +implementations a pointer is essentially a memory address, we are +"solving the problem" of defining the identity of objects by +stipulating that a data object "itself" is the information stored in +some particular set of memory locations in the computer. This suffices +for simple Lisp programs, but is hardly a general way to resolve the +issue of "sameness" in computational models. + + (6) On the other hand, from the viewpoint of implementation, +assignment requires us to modify the environment, which is itself a +mutable data structure. Thus, assignment and mutation are equipotent: +Each can be implemented in terms of the other. + + +File: sicp.info, Node: 3-3-2, Next: 3-3-3, Prev: 3-3-1, Up: 3-3 + +3.3.2 Representing Queues +------------------------- + +The mutators `set-car!' and `set-cdr!' enable us to use pairs to +construct data structures that cannot be built with `cons', `car', and +`cdr' alone. This section shows how to use pairs to represent a data +structure called a queue. Section *Note 3-3-3:: will show how to +represent data structures called tables. + + A "queue" is a sequence in which items are inserted at one end +(called the "rear" of the queue) and deleted from the other end (the "front"). +*Note Figure 3-18:: shows an initially empty queue in which the items +`a' and `b' are inserted. Then `a' is removed, `c' and `d' are +inserted, and `b' is removed. Because items are always removed in the +order in which they are inserted, a queue is sometimes called a "FIFO" +(first in, first out) buffer. + + *Figure 3.18:* Queue operations. + + Operation Resulting Queue + (define q (make-queue)) + (insert-queue! q 'a) a + (insert-queue! q 'b) a b + (delete-queue! q) b + (insert-queue! q 'c) b c + (insert-queue! q 'd) b c d + (delete-queue! q) c d + + In terms of data abstraction, we can regard a queue as defined by +the following set of operations: + + * a constructor: `(make-queue)' returns an empty queue (a queue + containing no items). + + * two selectors: + + (empty-queue? ) + + tests if the queue is empty. + + (front-queue ) + + returns the object at the front of the queue, signaling an error + if the queue is empty; it does not modify the queue. + + * two mutators: + + (insert-queue! ) + + inserts the item at the rear of the queue and returns the modified + queue as its value. + + (delete-queue! ) + + removes the item at the front of the queue and returns the + modified queue as its value, signaling an error if the queue is + empty before the deletion. + + + Because a queue is a sequence of items, we could certainly represent +it as an ordinary list; the front of the queue would be the `car' of +the list, inserting an item in the queue would amount to appending a +new element at the end of the list, and deleting an item from the queue +would just be taking the `cdr' of the list. However, this +representation is inefficient, because in order to insert an item we +must scan the list until we reach the end. Since the only method we +have for scanning a list is by successive `cdr' operations, this +scanning requires [theta](n) steps for a list of n items. A simple +modification to the list representation overcomes this disadvantage by +allowing the queue operations to be implemented so that they require +[theta](1) steps; that is, so that the number of steps needed is +independent of the length of the queue. + + The difficulty with the list representation arises from the need to +scan to find the end of the list. The reason we need to scan is that, +although the standard way of representing a list as a chain of pairs +readily provides us with a pointer to the beginning of the list, it +gives us no easily accessible pointer to the end. The modification +that avoids the drawback is to represent the queue as a list, together +with an additional pointer that indicates the final pair in the list. +That way, when we go to insert an item, we can consult the rear pointer +and so avoid scanning the list. + + A queue is represented, then, as a pair of pointers, `front-ptr' and +`rear-ptr', which indicate, respectively, the first and last pairs in an +ordinary list. Since we would like the queue to be an identifiable +object, we can use `cons' to combine the two pointers. Thus, the queue +itself will be the `cons' of the two pointers. *Note Figure 3-19:: +illustrates this representation. + + *Figure 3.19:* Implementation of a queue as a list with front and + rear pointers. + + +---+---+ + q -->| * | *-+-------------------+ + +-|-+---+ | + | | + | front-ptr | rear-ptr + V V + +---+---+ +---+---+ +---+---+ + | * | *-+--->| * | *-+--->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ + V V V + +---+ +---+ +---+ + | a | | b | | c | + +---+ +---+ +---+ + + To define the queue operations we use the following procedures, +which enable us to select and to modify the front and rear pointers of +a queue: + + (define (front-ptr queue) (car queue)) + + (define (rear-ptr queue) (cdr queue)) + + (define (set-front-ptr! queue item) (set-car! queue item)) + + (define (set-rear-ptr! queue item) (set-cdr! queue item)) + + Now we can implement the actual queue operations. We will consider +a queue to be empty if its front pointer is the empty list: + + (define (empty-queue? queue) (null? (front-ptr queue))) + + The `make-queue' constructor returns, as an initially empty queue, a +pair whose `car' and `cdr' are both the empty list: + + (define (make-queue) (cons '() '())) + + To select the item at the front of the queue, we return the `car' of +the pair indicated by the front pointer: + + (define (front-queue queue) + (if (empty-queue? queue) + (error "FRONT called with an empty queue" queue) + (car (front-ptr queue)))) + + To insert an item in a queue, we follow the method whose result is +indicated in *Note Figure 3-20::. We first create a new pair whose +`car' is the item to be inserted and whose `cdr' is the empty list. If +the queue was initially empty, we set the front and rear pointers of +the queue to this new pair. Otherwise, we modify the final pair in the +queue to point to the new pair, and also set the rear pointer to the +new pair. + + *Figure 3.20:* Result of using `(insert-queue! q 'd)' on the + queue of *Note Figure 3-19::. + + +---+---+ + q -->| * | *-+--------------------------------+ + +-|-+---+ | + | | + | front-ptr | rear-ptr + V V + +---+---+ +---+---+ +---+---+ +---+---+ + | * | *-+--->| * | *-+--->| * | *-+--->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ +-|-+---+ + V V V V + +---+ +---+ +---+ +---+ + | a | | b | | c | | d | + +---+ +---+ +---+ +---+ + + (define (insert-queue! queue item) + (let ((new-pair (cons item '()))) + (cond ((empty-queue? queue) + (set-front-ptr! queue new-pair) + (set-rear-ptr! queue new-pair) + queue) + (else + (set-cdr! (rear-ptr queue) new-pair) + (set-rear-ptr! queue new-pair) + queue)))) + + To delete the item at the front of the queue, we merely modify the +front pointer so that it now points at the second item in the queue, +which can be found by following the `cdr' pointer of the first item +(see *Note Figure 3-21::):(1) + + *Figure 3.21:* Result of using `(delete-queue! q)' on the queue + of *Note Figure 3-20::. + + +---+---+ + q -->| * | *-+--------------------------------+ + +-|-+---+ | + +------------+ | + front-ptr | | rear-ptr + V V + +---+---+ +---+---+ +---+---+ +---+---+ + | * | *-+--->| * | *-+--->| * | *-+--->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ +-|-+---+ + V V V V + +---+ +---+ +---+ +---+ + | a | | b | | c | | d | + +---+ +---+ +---+ +---+ + + (define (delete-queue! queue) + (cond ((empty-queue? queue) + (error "DELETE! called with an empty queue" queue)) + (else + (set-front-ptr! queue (cdr (front-ptr queue))) + queue))) + + *Exercise 3.21:* Ben Bitdiddle decides to test the queue + implementation described above. He types in the procedures to the + Lisp interpreter and proceeds to try them out: + + (define q1 (make-queue)) + + (insert-queue! q1 'a) + ((a) a) + + (insert-queue! q1 'b) + ((a b) b) + + (delete-queue! q1) + ((b) b) + + (delete-queue! q1) + (() b) + + "It's all wrong!" he complains. "The interpreter's response shows + that the last item is inserted into the queue twice. And when I + delete both items, the second `b' is still there, so the queue + isn't empty, even though it's supposed to be." Eva Lu Ator + suggests that Ben has misunderstood what is happening. "It's not + that the items are going into the queue twice," she explains. + "It's just that the standard Lisp printer doesn't know how to make + sense of the queue representation. If you want to see the queue + printed correctly, you'll have to define your own print procedure + for queues." Explain what Eva Lu is talking about. In particular, + show why Ben's examples produce the printed results that they do. + Define a procedure `print-queue' that takes a queue as input and + prints the sequence of items in the queue. + + *Exercise 3.22:* Instead of representing a queue as a pair of + pointers, we can build a queue as a procedure with local state. + The local state will consist of pointers to the beginning and the + end of an ordinary list. Thus, the `make-queue' procedure will + have the form + + (define (make-queue) + (let ((front-ptr ... ) + (rear-ptr ... )) + + (define (dispatch m) ...) + dispatch)) + + Complete the definition of `make-queue' and provide + implementations of the queue operations using this representation. + + *Exercise 3.23:* A "deque" ("double-ended queue") is a sequence in + which items can be inserted and deleted at either the front or the + rear. Operations on deques are the constructor `make-deque', the + predicate `empty-deque?', selectors `front-deque' and + `rear-deque', and mutators `front-insert-deque!', + `rear-insert-deque!', `front-delete-deque!', and + `rear-delete-deque!'. Show how to represent deques using pairs, + and give implementations of the operations.(2) All operations + should be accomplished in [theta](1) steps. + + ---------- Footnotes ---------- + + (1) If the first item is the final item in the queue, the front +pointer will be the empty list after the deletion, which will mark the +queue as empty; we needn't worry about updating the rear pointer, which +will still point to the deleted item, because `empty-queue?' looks only +at the front pointer. + + (2) Be careful not to make the interpreter try to print a structure +that contains cycles. (See *Note Exercise 3-13::.) + + +File: sicp.info, Node: 3-3-3, Next: 3-3-4, Prev: 3-3-2, Up: 3-3 + +3.3.3 Representing Tables +------------------------- + +When we studied various ways of representing sets in *Note Chapter 2::, +we mentioned in section *Note 2-3-3:: the task of maintaining a table +of records indexed by identifying keys. In the implementation of +data-directed programming in section *Note 2-4-3::, we made extensive +use of two-dimensional tables, in which information is stored and +retrieved using two keys. Here we see how to build tables as mutable +list structures. + + We first consider a one-dimensional table, in which each value is +stored under a single key. We implement the table as a list of +records, each of which is implemented as a pair consisting of a key and +the associated value. The records are glued together to form a list by +pairs whose `car's point to successive records. These gluing pairs are +called the "backbone" of the table. In order to have a place that we +can change when we add a new record to the table, we build the table as +a "headed list". A headed list has a special backbone pair at the +beginning, which holds a dummy "record"--in this case the arbitrarily +chosen symbol `*table*'. *Note Figure 3-22:: shows the box-and-pointer +diagram for the table + + a: 1 + b: 2 + c: 3 + + *Figure 3.22:* A table represented as a headed list. + + +---+---+ +---+---+ +---+---+ +---+---+ + | * | *-+--->| * | *-+--->| * | *-+--->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ +-|-+---+ + | | | | + V V V V + +---------+ +---+---+ +---+---+ +---+---+ + | *table* | | * | * | | * | * | | * | * | + +---------+ +-|-+-|-+ +-|-+-|-+ +-|-+-|-+ + | | | | | | + V V V V V V + +---+ +---+ +---+ +---+ +---+ +---+ + | a | | 1 | | b | | 2 | | c | | 3 | + +---+ +---+ +---+ +---+ +---+ +---+ + + To extract information from a table we use the `lookup' procedure, +which takes a key as argument and returns the associated value (or +false if there is no value stored under that key). `Lookup' is defined +in terms of the `assoc' operation, which expects a key and a list of +records as arguments. Note that `assoc' never sees the dummy record. +`Assoc' returns the record that has the given key as its `car'.(1) +`Lookup' then checks to see that the resulting record returned by +`assoc' is not false, and returns the value (the `cdr') of the record. + + (define (lookup key table) + (let ((record (assoc key (cdr table)))) + (if record + (cdr record) + false))) + + (define (assoc key records) + (cond ((null? records) false) + ((equal? key (caar records)) (car records)) + (else (assoc key (cdr records))))) + + To insert a value in a table under a specified key, we first use +`assoc' to see if there is already a record in the table with this key. +If not, we form a new record by `cons'ing the key with the value, and +insert this at the head of the table's list of records, after the dummy +record. If there already is a record with this key, we set the `cdr' +of this record to the designated new value. The header of the table +provides us with a fixed location to modify in order to insert the new +record.(2) + + (define (insert! key value table) + (let ((record (assoc key (cdr table)))) + (if record + (set-cdr! record value) + (set-cdr! table + (cons (cons key value) (cdr table))))) + 'ok) + + To construct a new table, we simply create a list containing the +symbol `*table*': + + (define (make-table) + (list '*table*)) + +Two-dimensional tables +...................... + +In a two-dimensional table, each value is indexed by two keys. We can +construct such a table as a one-dimensional table in which each key +identifies a subtable. *Note Figure 3-23:: shows the box-and-pointer +diagram for the table + + math: + +: 43 + -: 45 + *: 42 + letters: + a: 97 + b: 98 + +which has two subtables. (The subtables don't need a special header +symbol, since the key that identifies the subtable serves this purpose.) + + *Figure 3.23:* A two-dimensional table. + + table + | + V + +---+---+ +---+---+ +---+---+ + | * | *-+-->| * | *-+-->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ + V | V + +-------+ | +---+---+ +---+---+ +---+---+ + |*table*| | | * | *-+-->| * | *-+-->| * | / | + +-------+ | +-|-+---+ +-|-+---+ +-|-+---+ + | V V V + | +-------+ +---+---+ +---+---+ + | |letters| | * | * | | * | * | + | +-------+ +-|-+-|-+ +-|-+-|-+ + | V V V V + | +---+ +---+ +---+ +---+ + | | a | | 97| | b | | 98| + | +---+ +---+ +---+ +---+ + V + +---+---+ +---+---+ +---+---+ +---+---+ + | * | *-+-->| * | *-+-->| * | *-+-->| * | / | + +-|-+---+ +-|-+---+ +-|-+---+ +-|-+---+ + V V V V + +------+ +---+---+ +---+---+ +---+---+ + | math | | * | * | | * | * | | * | * | + +------+ +-|-+-|-+ +-|-+-|-+ +-|-+-|-+ + V V V V V V + +---+ +---+ +---+ +---+ +---+ +---+ + | + | | 43| | - | | 45| | * | | 42| + +---+ +---+ +---+ +---+ +---+ +---+ + + When we look up an item, we use the first key to identify the +correct subtable. Then we use the second key to identify the record +within the subtable. + + (define (lookup key-1 key-2 table) + (let ((subtable (assoc key-1 (cdr table)))) + (if subtable + (let ((record (assoc key-2 (cdr subtable)))) + (if record + (cdr record) + false)) + false))) + + To insert a new item under a pair of keys, we use `assoc' to see if +there is a subtable stored under the first key. If not, we build a new +subtable containing the single record (`key-2', `value') and insert it +into the table under the first key. If a subtable already exists for +the first key, we insert the new record into this subtable, using the +insertion method for one-dimensional tables described above: + + (define (insert! key-1 key-2 value table) + (let ((subtable (assoc key-1 (cdr table)))) + (if subtable + (let ((record (assoc key-2 (cdr subtable)))) + (if record + (set-cdr! record value) + (set-cdr! subtable + (cons (cons key-2 value) + (cdr subtable))))) + (set-cdr! table + (cons (list key-1 + (cons key-2 value)) + (cdr table))))) + 'ok) + +Creating local tables +..................... + +The `lookup' and `insert!' operations defined above take the table as +an argument. This enables us to use programs that access more than one +table. Another way to deal with multiple tables is to have separate +`lookup' and `insert!' procedures for each table. We can do this by +representing a table procedurally, as an object that maintains an +internal table as part of its local state. When sent an appropriate +message, this "table object" supplies the procedure with which to +operate on the internal table. Here is a generator for two-dimensional +tables represented in this fashion: + + (define (make-table) + (let ((local-table (list '*table*))) + (define (lookup key-1 key-2) + (let ((subtable (assoc key-1 (cdr local-table)))) + (if subtable + (let ((record (assoc key-2 (cdr subtable)))) + (if record + (cdr record) + false)) + false))) + (define (insert! key-1 key-2 value) + (let ((subtable (assoc key-1 (cdr local-table)))) + (if subtable + (let ((record (assoc key-2 (cdr subtable)))) + (if record + (set-cdr! record value) + (set-cdr! subtable + (cons (cons key-2 value) + (cdr subtable))))) + (set-cdr! local-table + (cons (list key-1 + (cons key-2 value)) + (cdr local-table))))) + 'ok) + (define (dispatch m) + (cond ((eq? m 'lookup-proc) lookup) + ((eq? m 'insert-proc!) insert!) + (else (error "Unknown operation -- TABLE" m)))) + dispatch)) + + Using `make-table', we could implement the `get' and `put' +operations used in section *Note 2-4-3:: for data-directed programming, +as follows: + + (define operation-table (make-table)) + (define get (operation-table 'lookup-proc)) + (define put (operation-table 'insert-proc!)) + + `Get' takes as arguments two keys, and `put' takes as arguments two +keys and a value. Both operations access the same local table, which is +encapsulated within the object created by the call to `make-table'. + + *Exercise 3.24:* In the table implementations above, the keys are + tested for equality using `equal?' (called by `assoc'). This is + not always the appropriate test. For instance, we might have a + table with numeric keys in which we don't need an exact match to + the number we're looking up, but only a number within some + tolerance of it. Design a table constructor `make-table' that + takes as an argument a `same-key?' procedure that will be used to + test "equality" of keys. `Make-table' should return a `dispatch' + procedure that can be used to access appropriate `lookup' and + `insert!' procedures for a local table. + + *Exercise 3.25:* Generalizing one- and two-dimensional tables, + show how to implement a table in which values are stored under an + arbitrary number of keys and different values may be stored under + different numbers of keys. The `lookup' and `insert!' procedures + should take as input a list of keys used to access the table. + + *Exercise 3.26:* To search a table as implemented above, one needs + to scan through the list of records. This is basically the + unordered list representation of section *Note 2-3-3::. For large + tables, it may be more efficient to structure the table in a + different manner. Describe a table implementation where the (key, + value) records are organized using a binary tree, assuming that + keys can be ordered in some way (e.g., numerically or + alphabetically). (Compare *Note Exercise 2-66:: of *Note Chapter + 2::.) + + *Exercise 3.27:* "Memoization" (also called "tabulation") is a + technique that enables a procedure to record, in a local table, + values that have previously been computed. This technique can + make a vast difference in the performance of a program. A memoized + procedure maintains a table in which values of previous calls are + stored using as keys the arguments that produced the values. When + the memoized procedure is asked to compute a value, it first + checks the table to see if the value is already there and, if so, + just returns that value. Otherwise, it computes the new value in + the ordinary way and stores this in the table. As an example of + memoization, recall from section *Note 1-2-2:: the exponential + process for computing Fibonacci numbers: + + (define (fib n) + (cond ((= n 0) 0) + ((= n 1) 1) + (else (+ (fib (- n 1)) + (fib (- n 2)))))) + + The memoized version of the same procedure is + + (define memo-fib + (memoize (lambda (n) + (cond ((= n 0) 0) + ((= n 1) 1) + (else (+ (memo-fib (- n 1)) + (memo-fib (- n 2)))))))) + + where the memoizer is defined as + + (define (memoize f) + (let ((table (make-table))) + (lambda (x) + (let ((previously-computed-result (lookup x table))) + (or previously-computed-result + (let ((result (f x))) + (insert! x result table) + result)))))) + + Draw an environment diagram to analyze the computation of + `(memo-fib 3)'. Explain why `memo-fib' computes the nth Fibonacci + number in a number of steps proportional to n. Would the scheme + still work if we had simply defined `memo-fib' to be `(memoize + fib)'? + + ---------- Footnotes ---------- + + (1) Because `assoc' uses `equal?', it can recognize keys that are +symbols, numbers, or list structure. + + (2) Thus, the first backbone pair is the object that represents the +table "itself"; that is, a pointer to the table is a pointer to this +pair. This same backbone pair always starts the table. If we did not +arrange things in this way, `insert!' would have to return a new value +for the start of the table when it added a new record. + + +File: sicp.info, Node: 3-3-4, Next: 3-3-5, Prev: 3-3-3, Up: 3-3 + +3.3.4 A Simulator for Digital Circuits +-------------------------------------- + +Designing complex digital systems, such as computers, is an important +engineering activity. Digital systems are constructed by +interconnecting simple elements. Although the behavior of these +individual elements is simple, networks of them can have very complex +behavior. Computer simulation of proposed circuit designs is an +important tool used by digital systems engineers. In this section we +design a system for performing digital logic simulations. This system +typifies a kind of program called an "event-driven simulation", in +which actions ("events") trigger further events that happen at a later +time, which in turn trigger more events, and so so. + + Our computational model of a circuit will be composed of objects that +correspond to the elementary components from which the circuit is +constructed. There are "wires", which carry "digital signals". A +digital signal may at any moment have only one of two possible values, +0 and 1. There are also various types of digital "function boxes", +which connect wires carrying input signals to other output wires. Such +boxes produce output signals computed from their input signals. The +output signal is delayed by a time that depends on the type of the +function box. For example, an "inverter" is a primitive function box +that inverts its input. If the input signal to an inverter changes to +0, then one inverter-delay later the inverter will change its output +signal to 1. If the input signal to an inverter changes to 1, then one +inverter-delay later the inverter will change its output signal to 0. +We draw an inverter symbolically as in *Note Figure 3-24::. An "and-gate", +also shown in *Note Figure 3-24::, is a primitive function box with two +inputs and one output. It drives its output signal to a value that is +the "logical and" of the inputs. That is, if both of its input signals +become 1, then one and-gate-delay time later the and-gate will force +its output signal to be 1; otherwise the output will be 0. An "or-gate" +is a similar two-input primitive function box that drives its output +signal to a value that is the "logical or" of the inputs. That is, the +output will become 1 if at least one of the input signals is 1; +otherwise the output will become 0. + + *Figure 3.24:* Primitive functions in the digital logic simulator. + + __ ___ + |\ --| \ --\ \ + --| >o-- | )-- ) >-- + |/ --|__/ --/__/ + + Inverter And-gate Or-gate + + We can connect primitive functions together to construct more complex +functions. To accomplish this we wire the outputs of some function +boxes to the inputs of other function boxes. For example, the "half-adder" +circuit shown in *Note Figure 3-25:: consists of an or-gate, two +and-gates, and an inverter. It takes two input signals, A and B, and +has two output signals, S and C. S will become 1 whenever precisely +one of A and B is 1, and C will become 1 whenever A and B are both 1. +We can see from the figure that, because of the delays involved, the +outputs may be generated at different times. Many of the difficulties +in the design of digital circuits arise from this fact. + + *Figure 3.25:* A half-adder circuit. + + +--------------------------------------+ + | ____ | + A --------*---\ \ D ___ | + | | > >---------------| \ | + | +--|---/___/ | )----- S + | | | |\ E +--|___/ | + | | | +--| >o---+ | + | | | ___ | |/ | + | | +---| \ | | + | | | )--*----------------------- C + B -----*------|___/ | + | | + +--------------------------------------+ + + We will now build a program for modeling the digital logic circuits +we wish to study. The program will construct computational objects +modeling the wires, which will "hold" the signals. Function boxes will +be modeled by procedures that enforce the correct relationships among +the signals. + + One basic element of our simulation will be a procedure `make-wire', +which constructs wires. For example, we can construct six wires as +follows: + + (define a (make-wire)) + (define b (make-wire)) + (define c (make-wire)) + + (define d (make-wire)) + (define e (make-wire)) + (define s (make-wire)) + + We attach a function box to a set of wires by calling a procedure +that constructs that kind of box. The arguments to the constructor +procedure are the wires to be attached to the box. For example, given +that we can construct and-gates, or-gates, and inverters, we can wire +together the half-adder shown in *Note Figure 3-25::: + + (or-gate a b d) + ok + + (and-gate a b c) + ok + + (inverter c e) + ok + + (and-gate d e s) + ok + + Better yet, we can explicitly name this operation by defining a +procedure `half-adder' that constructs this circuit, given the four +external wires to be attached to the half-adder: + + (define (half-adder a b s c) + (let ((d (make-wire)) (e (make-wire))) + (or-gate a b d) + (and-gate a b c) + (inverter c e) + (and-gate d e s) + 'ok)) + + The advantage of making this definition is that we can use +`half-adder' itself as a building block in creating more complex +circuits. *Note Figure 3-26::, for example, shows a "full-adder" +composed of two half-adders and an or-gate.(1) We can construct a +full-adder as follows: + + (define (full-adder a b c-in sum c-out) + (let ((s (make-wire)) + (c1 (make-wire)) + (c2 (make-wire))) + (half-adder b c-in s c1) + (half-adder a s sum c2) + (or-gate c1 c2 c-out) + 'ok)) + + Having defined `full-adder' as a procedure, we can now use it as a +building block for creating still more complex circuits. (For example, +see *Note Exercise 3-30::.) + + *Figure 3.26:* A full-adder circuit. + + +----------------------------------+ + | +-------+ | + A -----------------+ half +-------------- SUM + | +-------+ | adder | ____ | + B -----+ half +---+ +---\ \ | + | | adder | +-------+ >or >----- Cout + C -----+ +---------------/___/ | + | +-------+ | + +----------------------------------+ + + In essence, our simulator provides us with the tools to construct a +language of circuits. If we adopt the general perspective on languages +with which we approached the study of Lisp in section *Note 1-1::, we +can say that the primitive function boxes form the primitive elements +of the language, that wiring boxes together provides a means of +combination, and that specifying wiring patterns as procedures serves +as a means of abstraction. + +Primitive function boxes +........................ + +The primitive function boxes implement the "forces" by which a change +in the signal on one wire influences the signals on other wires. To +build function boxes, we use the following operations on wires: + + * (get-signal ) + + returns the current value of the signal on the wire. + + * (set-signal! ) + + changes the value of the signal on the wire to the new value. + + * (add-action! ) + + asserts that the designated procedure should be run whenever the + signal on the wire changes value. Such procedures are the + vehicles by which changes in the signal value on the wire are + communicated to other wires. + + + In addition, we will make use of a procedure `after-delay' that +takes a time delay and a procedure to be run and executes the given +procedure after the given delay. + + Using these procedures, we can define the primitive digital logic +functions. To connect an input to an output through an inverter, we +use `add-action!' to associate with the input wire a procedure that +will be run whenever the signal on the input wire changes value. The +procedure computes the `logical-not' of the input signal, and then, +after one `inverter-delay', sets the output signal to be this new value: + + (define (inverter input output) + (define (invert-input) + (let ((new-value (logical-not (get-signal input)))) + (after-delay inverter-delay + (lambda () + (set-signal! output new-value))))) + (add-action! input invert-input) + 'ok) + + (define (logical-not s) + (cond ((= s 0) 1) + ((= s 1) 0) + (else (error "Invalid signal" s)))) + + An and-gate is a little more complex. The action procedure must be +run if either of the inputs to the gate changes. It computes the +`logical-and' (using a procedure analogous to `logical-not') of the +values of the signals on the input wires and sets up a change to the +new value to occur on the output wire after one `and-gate-delay'. + + (define (and-gate a1 a2 output) + (define (and-action-procedure) + (let ((new-value + (logical-and (get-signal a1) (get-signal a2)))) + (after-delay and-gate-delay + (lambda () + (set-signal! output new-value))))) + (add-action! a1 and-action-procedure) + (add-action! a2 and-action-procedure) + 'ok) + + *Exercise 3.28:* Define an or-gate as a primitive function box. + Your `or-gate' constructor should be similar to `and-gate'. + + *Exercise 3.29:* Another way to construct an or-gate is as a + compound digital logic device, built from and-gates and inverters. + Define a procedure `or-gate' that accomplishes this. What is the + delay time of the or-gate in terms of `and-gate-delay' and + `inverter-delay'? + + *Exercise 3.30:* *Note Figure 3-27:: shows a "ripple-carry adder" + formed by stringing together n full-adders. This is the simplest + form of parallel adder for adding two n-bit binary numbers. The + inputs A_1, A_2, A_3, ..., A_n and B_1, B_2, B_3, ..., B_n are the + two binary numbers to be added (each A_k and B_k is a 0 or a 1). + The circuit generates S_1, S_2, S_3, ..., S_n, the n bits of the + sum, and C, the carry from the addition. Write a procedure + `ripple-carry-adder' that generates this circuit. The procedure + should take as arguments three lists of n wires each--the A_k, the + B_k, and the S_k--and also another wire C. The major drawback of + the ripple-carry adder is the need to wait for the carry signals + to propagate. What is the delay needed to obtain the complete + output from an n-bit ripple-carry adder, expressed in terms of the + delays for and-gates, or-gates, and inverters? + + *Figure 3.27:* A ripple-carry adder for n-bit numbers. + + : : : + : A_1 B_1 C_1 A_2 B_2 C_2 A_3 B_3 C_3: : A_n B_n C_n=0 + : | | +---+ | | +---+ | | +----- : | | +- + | | | | | | | | | | | | : : | | | + : ++---+---++ | ++---+---++ | ++---+---++ : : ++---+---++ + : | FA | | | FA | | | FA | : : | FA | + : +--+---+--+ | +--+---+--+ | +--+---+--+ : : +--+---+--+ + : | | | | | | | | : : | | + C ------+ | +-----+ | +-----+ | : ------+ | + : | C_1 | C_2 | : :C_(n-1) | + : | | | : : | + S_1 S_2 S_3 S_n + +Representing wires +.................. + +A wire in our simulation will be a computational object with two local +state variables: a `signal-value' (initially taken to be 0) and a +collection of `action-procedures' to be run when the signal changes +value. We implement the wire, using message-passing style, as a +collection of local procedures together with a `dispatch' procedure +that selects the appropriate local operation, just as we did with the +simple bank-account object in section *Note 3-1-1::: + + (define (make-wire) + (let ((signal-value 0) (action-procedures '())) + (define (set-my-signal! new-value) + (if (not (= signal-value new-value)) + (begin (set! signal-value new-value) + (call-each action-procedures)) + 'done)) + + (define (accept-action-procedure! proc) + (set! action-procedures (cons proc action-procedures)) + (proc)) + + (define (dispatch m) + (cond ((eq? m 'get-signal) signal-value) + ((eq? m 'set-signal!) set-my-signal!) + ((eq? m 'add-action!) accept-action-procedure!) + (else (error "Unknown operation -- WIRE" m)))) + dispatch)) + + The local procedure `set-my-signal!' tests whether the new signal +value changes the signal on the wire. If so, it runs each of the +action procedures, using the following procedure `call-each', which +calls each of the items in a list of no-argument procedures: + + (define (call-each procedures) + (if (null? procedures) + 'done + (begin + ((car procedures)) + (call-each (cdr procedures))))) + + The local procedure `accept-action-procedure!' adds the given +procedure to the list of procedures to be run, and then runs the new +procedure once. (See *Note Exercise 3-31::.) + + With the local `dispatch' procedure set up as specified, we can +provide the following procedures to access the local operations on +wires:(2) + + (define (get-signal wire) + (wire 'get-signal)) + + (define (set-signal! wire new-value) + ((wire 'set-signal!) new-value)) + + (define (add-action! wire action-procedure) + ((wire 'add-action!) action-procedure)) + + Wires, which have time-varying signals and may be incrementally +attached to devices, are typical of mutable objects. We have modeled +them as procedures with local state variables that are modified by +assignment. When a new wire is created, a new set of state variables +is allocated (by the `let' expression in `make-wire') and a new +`dispatch' procedure is constructed and returned, capturing the +environment with the new state variables. + + The wires are shared among the various devices that have been +connected to them. Thus, a change made by an interaction with one +device will affect all the other devices attached to the wire. The +wire communicates the change to its neighbors by calling the action +procedures provided to it when the connections were established. + +The agenda +.......... + +The only thing needed to complete the simulator is `after-delay'. The +idea here is that we maintain a data structure, called an "agenda", +that contains a schedule of things to do. The following operations are +defined for agendas: + + * `(make-agenda)' returns a new empty agenda. + + * `(empty-agenda? )' is true if the specified agenda is + empty. + + * `(first-agenda-item )' returns the first item on the + agenda. + + * `(remove-first-agenda-item! )' modifies the agenda by + removing the first item. + + * `(add-to-agenda!

    must be a procedure of no arguments. `Parallel-execute' +creates a separate process for each

    , which applies

    (to no +arguments). These processes all run concurrently.(1) + + As an example of how this is used, consider + + (define x 10) + + (parallel-execute (lambda () (set! x (* x x))) + (lambda () (set! x (+ x 1)))) + + This creates two concurrent processes--P_1, which sets `x' to `x' +times `x', and P_2, which increments `x'. After execution is complete, +`x' will be left with one of five possible values, depending on the +interleaving of the events of P_1 and P_2: + + 101: P_1 sets `x' to 100 and then P_2 increments + `x' to 101. + 121: P_2 increments `x' to 11 and then P_1 sets + `x' to `x' times `x'. + 110: P_2 changes `x' from 10 to 11 between the two + times that P_1 accesses the value of `x' during + the evaluation of `(* x x)'. + 11: P_2 accesses `x', then P_1 sets `x' to + 100, then P_2 sets `x'. + 100: P_1 accesses `x' (twice), then P_2 sets + `x' to 11, then P_1 sets `x'. + + We can constrain the concurrency by using serialized procedures, +which are created by "serializers". Serializers are constructed by +`make-serializer', whose implementation is given below. A serializer +takes a procedure as argument and returns a serialized procedure that +behaves like the original procedure. All calls to a given serializer +return serialized procedures in the same set. + + Thus, in contrast to the example above, executing + + (define x 10) + + (define s (make-serializer)) + + (parallel-execute (s (lambda () (set! x (* x x)))) + (s (lambda () (set! x (+ x 1))))) + +can produce only two possible values for `x', 101 or 121. The other +possibilities are eliminated, because the execution of P_1 and P_2 +cannot be interleaved. + + Here is a version of the `make-account' procedure from section *Note +3-1-1::, where the deposits and withdrawals have been serialized: + + (define (make-account balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + (let ((protected (make-serializer))) + (define (dispatch m) + (cond ((eq? m 'withdraw) (protected withdraw)) + ((eq? m 'deposit) (protected deposit)) + ((eq? m 'balance) balance) + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch)) + + With this implementation, two processes cannot be withdrawing from or +depositing into a single account concurrently. This eliminates the +source of the error illustrated in *Note Figure 3-29::, where Peter +changes the account balance between the times when Paul accesses the +balance to compute the new value and when Paul actually performs the +assignment. On the other hand, each account has its own serializer, so +that deposits and withdrawals for different accounts can proceed +concurrently. + + *Exercise 3.39:* Which of the five possibilities in the parallel + execution shown above remain if we instead serialize execution as + follows: + + (define x 10) + + (define s (make-serializer)) + + (parallel-execute (lambda () (set! x ((s (lambda () (* x x)))))) + (s (lambda () (set! x (+ x 1))))) + + *Exercise 3.40:* Give all possible values of `x' that can result + from executing + + (define x 10) + + (parallel-execute (lambda () (set! x (* x x))) + (lambda () (set! x (* x x x)))) + + Which of these possibilities remain if we instead use serialized + procedures: + + (define x 10) + + (define s (make-serializer)) + + (parallel-execute (s (lambda () (set! x (* x x)))) + (s (lambda () (set! x (* x x x))))) + + *Exercise 3.41:* Ben Bitdiddle worries that it would be better to + implement the bank account as follows (where the commented line + has been changed): + + (define (make-account balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + ;; continued on next page + + (let ((protected (make-serializer))) + (define (dispatch m) + (cond ((eq? m 'withdraw) (protected withdraw)) + ((eq? m 'deposit) (protected deposit)) + ((eq? m 'balance) + ((protected (lambda () balance)))) ; serialized + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch)) + + because allowing unserialized access to the bank balance can + result in anomalous behavior. Do you agree? Is there any + scenario that demonstrates Ben's concern? + + *Exercise 3.42:* Ben Bitdiddle suggests that it's a waste of time + to create a new serialized procedure in response to every + `withdraw' and `deposit' message. He says that `make-account' + could be changed so that the calls to `protected' are done outside + the `dispatch' procedure. That is, an account would return the + same serialized procedure (which was created at the same time as + the account) each time it is asked for a withdrawal procedure. + + (define (make-account balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + (let ((protected (make-serializer))) + (let ((protected-withdraw (protected withdraw)) + (protected-deposit (protected deposit))) + (define (dispatch m) + (cond ((eq? m 'withdraw) protected-withdraw) + ((eq? m 'deposit) protected-deposit) + ((eq? m 'balance) balance) + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch))) + + Is this a safe change to make? In particular, is there any + difference in what concurrency is allowed by these two versions of + `make-account' ? + +Complexity of using multiple shared resources +............................................. + +Serializers provide a powerful abstraction that helps isolate the +complexities of concurrent programs so that they can be dealt with +carefully and (hopefully) correctly. However, while using serializers +is relatively straightforward when there is only a single shared +resource (such as a single bank account), concurrent programming can be +treacherously difficult when there are multiple shared resources. + + To illustrate one of the difficulties that can arise, suppose we +wish to swap the balances in two bank accounts. We access each account +to find the balance, compute the difference between the balances, +withdraw this difference from one account, and deposit it in the other +account. We could implement this as follows:(2) + + (define (exchange account1 account2) + (let ((difference (- (account1 'balance) + (account2 'balance)))) + ((account1 'withdraw) difference) + ((account2 'deposit) difference))) + + This procedure works well when only a single process is trying to do +the exchange. Suppose, however, that Peter and Paul both have access +to accounts a1, a2, and a3, and that Peter exchanges a1 and a2 while +Paul concurrently exchanges a1 and a3. Even with account deposits and +withdrawals serialized for individual accounts (as in the `make-account' +procedure shown above in this section), `exchange' can still produce +incorrect results. For example, Peter might compute the difference in +the balances for a1 and a2, but then Paul might change the balance in +a1 before Peter is able to complete the exchange.(3) For correct +behavior, we must arrange for the `exchange' procedure to lock out any +other concurrent accesses to the accounts during the entire time of the +exchange. + + One way we can accomplish this is by using both accounts' +serializers to serialize the entire `exchange' procedure. To do this, +we will arrange for access to an account's serializer. Note that we +are deliberately breaking the modularity of the bank-account object by +exposing the serializer. The following version of `make-account' is +identical to the original version given in section *Note 3-1-1::, +except that a serializer is provided to protect the balance variable, +and the serializer is exported via message passing: + + (define (make-account-and-serializer balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + (let ((balance-serializer (make-serializer))) + (define (dispatch m) + (cond ((eq? m 'withdraw) withdraw) + ((eq? m 'deposit) deposit) + ((eq? m 'balance) balance) + ((eq? m 'serializer) balance-serializer) + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch)) + + We can use this to do serialized deposits and withdrawals. However, +unlike our earlier serialized account, it is now the responsibility of +each user of bank-account objects to explicitly manage the +serialization, for example as follows:(4) + + (define (deposit account amount) + (let ((s (account 'serializer)) + (d (account 'deposit))) + ((s d) amount))) + + Exporting the serializer in this way gives us enough flexibility to +implement a serialized exchange program. We simply serialize the +original `exchange' procedure with the serializers for both accounts: + + (define (serialized-exchange account1 account2) + (let ((serializer1 (account1 'serializer)) + (serializer2 (account2 'serializer))) + ((serializer1 (serializer2 exchange)) + account1 + account2))) + + *Exercise 3.43:* Suppose that the balances in three accounts start + out as $10, $20, and $30, and that multiple processes run, + exchanging the balances in the accounts. Argue that if the + processes are run sequentially, after any number of concurrent + exchanges, the account balances should be $10, $20, and $30 in + some order. Draw a timing diagram like the one in *Note Figure + 3-29:: to show how this condition can be violated if the exchanges + are implemented using the first version of the account-exchange + program in this section. On the other hand, argue that even with + this `exchange' program, the sum of the balances in the accounts + will be preserved. Draw a timing diagram to show how even this + condition would be violated if we did not serialize the + transactions on individual accounts. + + *Exercise 3.44:* Consider the problem of transferring an amount + from one account to another. Ben Bitdiddle claims that this can + be accomplished with the following procedure, even if there are + multiple people concurrently transferring money among multiple + accounts, using any account mechanism that serializes deposit and + withdrawal transactions, for example, the version of + `make-account' in the text above. + + (define (transfer from-account to-account amount) + ((from-account 'withdraw) amount) + ((to-account 'deposit) amount)) + + Louis Reasoner claims that there is a problem here, and that we + need to use a more sophisticated method, such as the one required + for dealing with the exchange problem. Is Louis right? If not, + what is the essential difference between the transfer problem and + the exchange problem? (You should assume that the balance in + `from-account' is at least `amount'.) + + *Exercise 3.45:* Louis Reasoner thinks our bank-account system is + unnecessarily complex and error-prone now that deposits and + withdrawals aren't automatically serialized. He suggests that + `make-account-and-serializer' should have exported the serializer + (for use by such procedures as `serialized-exchange') in addition + to (rather than instead of) using it to serialize accounts and + deposits as `make-account' did. He proposes to redefine accounts + as follows: + + (define (make-account-and-serializer balance) + (define (withdraw amount) + (if (>= balance amount) + (begin (set! balance (- balance amount)) + balance) + "Insufficient funds")) + (define (deposit amount) + (set! balance (+ balance amount)) + balance) + (let ((balance-serializer (make-serializer))) + (define (dispatch m) + (cond ((eq? m 'withdraw) (balance-serializer withdraw)) + ((eq? m 'deposit) (balance-serializer deposit)) + ((eq? m 'balance) balance) + ((eq? m 'serializer) balance-serializer) + (else (error "Unknown request -- MAKE-ACCOUNT" + m)))) + dispatch)) + + Then deposits are handled as with the original `make-account': + + (define (deposit account amount) + ((account 'deposit) amount)) + + Explain what is wrong with Louis's reasoning. In particular, + consider what happens when `serialized-exchange' is called. + +Implementing serializers +........................ + +We implement serializers in terms of a more primitive synchronization +mechanism called a "mutex". A mutex is an object that supports two +operations--the mutex can be "acquired", and the mutex can be "released". +Once a mutex has been acquired, no other acquire operations on that +mutex may proceed until the mutex is released.(5) In our +implementation, each serializer has an associated mutex. Given a +procedure `p', the serializer returns a procedure that acquires the +mutex, runs `p', and then releases the mutex. This ensures that only +one of the procedures produced by the serializer can be running at +once, which is precisely the serialization property that we need to +guarantee. + + (define (make-serializer) + (let ((mutex (make-mutex))) + (lambda (p) + (define (serialized-p . args) + (mutex 'acquire) + (let ((val (apply p args))) + (mutex 'release) + val)) + serialized-p))) + + The mutex is a mutable object (here we'll use a one-element list, +which we'll refer to as a "cell") that can hold the value true or +false. When the value is false, the mutex is available to be acquired. +When the value is true, the mutex is unavailable, and any process that +attempts to acquire the mutex must wait. + + Our mutex constructor `make-mutex' begins by initializing the cell +contents to false. To acquire the mutex, we test the cell. If the +mutex is available, we set the cell contents to true and proceed. +Otherwise, we wait in a loop, attempting to acquire over and over +again, until we find that the mutex is available.(6) To release the +mutex, we set the cell contents to false. + + (define (make-mutex) + (let ((cell (list false))) + (define (the-mutex m) + (cond ((eq? m 'acquire) + (if (test-and-set! cell) + (the-mutex 'acquire))) ; retry + ((eq? m 'release) (clear! cell)))) + the-mutex)) + + (define (clear! cell) + (set-car! cell false)) + + `Test-and-set!' tests the cell and returns the result of the test. +In addition, if the test was false, `test-and-set!' sets the cell +contents to true before returning false. We can express this behavior +as the following procedure: + + (define (test-and-set! cell) + (if (car cell) + true + (begin (set-car! cell true) + false))) + + However, this implementation of `test-and-set!' does not suffice as +it stands. There is a crucial subtlety here, which is the essential +place where concurrency control enters the system: The `test-and-set!' +operation must be performed "atomically". That is, we must guarantee +that, once a process has tested the cell and found it to be false, the +cell contents will actually be set to true before any other process can +test the cell. If we do not make this guarantee, then the mutex can +fail in a way similar to the bank-account failure in *Note Figure +3-29::. (See *Note Exercise 3-46::.) + + The actual implementation of `test-and-set!' depends on the details +of how our system runs concurrent processes. For example, we might be +executing concurrent processes on a sequential processor using a +time-slicing mechanism that cycles through the processes, permitting +each process to run for a short time before interrupting it and moving +on to the next process. In that case, `test-and-set!' can work by +disabling time slicing during the testing and setting.(7) +Alternatively, multiprocessing computers provide instructions that +support atomic operations directly in hardware.(8) + + *Exercise 3.46:* Suppose that we implement `test-and-set!' using + an ordinary procedure as shown in the text, without attempting to + make the operation atomic. Draw a timing diagram like the one in + *Note Figure 3-29:: to demonstrate how the mutex implementation + can fail by allowing two processes to acquire the mutex at the + same time. + + *Exercise 3.47:* A semaphore (of size n) is a generalization of a + mutex. Like a mutex, a semaphore supports acquire and release + operations, but it is more general in that up to n processes can + acquire it concurrently. Additional processes that attempt to + acquire the semaphore must wait for release operations. Give + implementations of semaphores + + a. in terms of mutexes + + b. in terms of atomic `test-and-set!' operations. + + +Deadlock +........ + +Now that we have seen how to implement serializers, we can see that +account exchanging still has a problem, even with the +`serialized-exchange' procedure above. Imagine that Peter attempts to +exchange a1 with a2 while Paul concurrently attempts to exchange a2 +with a1. Suppose that Peter's process reaches the point where it has +entered a serialized procedure protecting a1 and, just after that, +Paul's process enters a serialized procedure protecting a2. Now Peter +cannot proceed (to enter a serialized procedure protecting a2) until +Paul exits the serialized procedure protecting a2. Similarly, Paul +cannot proceed until Peter exits the serialized procedure protecting +a1. Each process is stalled forever, waiting for the other. This +situation is called a "deadlock". Deadlock is always a danger in +systems that provide concurrent access to multiple shared resources. + + One way to avoid the deadlock in this situation is to give each +account a unique identification number and rewrite +`serialized-exchange' so that a process will always attempt to enter a +procedure protecting the lowest-numbered account first. Although this +method works well for the exchange problem, there are other situations +that require more sophisticated deadlock-avoidance techniques, or where +deadlock cannot be avoided at all. (See *Note Exercise 3-48:: and +*Note Exercise 3-49::.)(9) + + *Exercise 3.48:* Explain in detail why the deadlock-avoidance + method described above, (i.e., the accounts are numbered, and each + process attempts to acquire the smaller-numbered account first) + avoids deadlock in the exchange problem. Rewrite + `serialized-exchange' to incorporate this idea. (You will also + need to modify `make-account' so that each account is created with + a number, which can be accessed by sending an appropriate message.) + + *Exercise 3.49:* Give a scenario where the deadlock-avoidance + mechanism described above does not work. (Hint: In the exchange + problem, each process knows in advance which accounts it will need + to get access to. Consider a situation where a process must get + access to some shared resources before it can know which + additional shared resources it will require.) + +Concurrency, time, and communication +.................................... + +We've seen how programming concurrent systems requires controlling the +ordering of events when different processes access shared state, and +we've seen how to achieve this control through judicious use of +serializers. But the problems of concurrency lie deeper than this, +because, from a fundamental point of view, it's not always clear what +is meant by "shared state." + + Mechanisms such as `test-and-set!' require processes to examine a +global shared flag at arbitrary times. This is problematic and +inefficient to implement in modern high-speed processors, where due to +optimization techniques such as pipelining and cached memory, the +contents of memory may not be in a consistent state at every instant. +In contemporary multiprocessing systems, therefore, the serializer +paradigm is being supplanted by new approaches to concurrency +control.(10) + + The problematic aspects of shared state also arise in large, +distributed systems. For instance, imagine a distributed banking +system where individual branch banks maintain local values for bank +balances and periodically compare these with values maintained by other +branches. In such a system the value of "the account balance" would be +undetermined, except right after synchronization. If Peter deposits +money in an account he holds jointly with Paul, when should we say that +the account balance has changed--when the balance in the local branch +changes, or not until after the synchronization? And if Paul accesses +the account from a different branch, what are the reasonable +constraints to place on the banking system such that the behavior is +"correct"? The only thing that might matter for correctness is the +behavior observed by Peter and Paul individually and the "state" of the +account immediately after synchronization. Questions about the "real" +account balance or the order of events between synchronizations may be +irrelevant or meaningless.(11) + + The basic phenomenon here is that synchronizing different processes, +establishing shared state, or imposing an order on events requires +communication among the processes. In essence, any notion of time in +concurrency control must be intimately tied to communication.(12) It +is intriguing that a similar connection between time and communication +also arises in the Theory of Relativity, where the speed of light (the +fastest signal that can be used to synchronize events) is a fundamental +constant relating time and space. The complexities we encounter in +dealing with time and state in our computational models may in fact +mirror a fundamental complexity of the physical universe. + + ---------- Footnotes ---------- + + (1) `Parallel-execute' is not part of standard Scheme, but it can be +implemented in MIT Scheme. In our implementation, the new concurrent +processes also run concurrently with the original Scheme process. +Also, in our implementation, the value returned by `parallel-execute' +is a special control object that can be used to halt the newly created +processes. + + (2) We have simplified `exchange' by exploiting the fact that our +`deposit' message accepts negative amounts. (This is a serious bug in +our banking system!) + + (3) If the account balances start out as $10, $20, and $30, then +after any number of concurrent exchanges, the balances should still be +$10, $20, and $30 in some order. Serializing the deposits to +individual accounts is not sufficient to guarantee this. See *Note +Exercise 3-43::. + + (4) *Note Exercise 3-45:: investigates why deposits and withdrawals +are no longer automatically serialized by the account. + + (5) The term "mutex" is an abbreviation for "mutual exclusion". The +general problem of arranging a mechanism that permits concurrent +processes to safely share resources is called the mutual exclusion +problem. Our mutex is a simple variant of the "semaphore" mechanism +(see *Note Exercise 3-47::), which was introduced in the "THE" +Multiprogramming System developed at the Technological University of +Eindhoven and named for the university's initials in Dutch (Dijkstra +1968a). The acquire and release operations were originally called P +and V, from the Dutch words _passeren_ (to pass) and _vrijgeven_ (to +release), in reference to the semaphores used on railroad systems. +Dijkstra's classic exposition (1968b) was one of the first to clearly +present the issues of concurrency control, and showed how to use +semaphores to handle a variety of concurrency problems. + + (6) In most time-shared operating systems, processes that are +blocked by a mutex do not waste time "busy-waiting" as above. Instead, +the system schedules another process to run while the first is waiting, +and the blocked process is awakened when the mutex becomes available. + + (7) In MIT Scheme for a single processor, which uses a time-slicing +model, `test-and-set!' can be implemented as follows: + + (define (test-and-set! cell) + (without-interrupts + (lambda () + (if (car cell) + true + (begin (set-car! cell true) + false))))) + + `Without-interrupts' disables time-slicing interrupts while its +procedure argument is being executed. + + (8) There are many variants of such instructions--including +test-and-set, test-and-clear, swap, compare-and-exchange, load-reserve, +and store-conditional--whose design must be carefully matched to the +machine's processor-memory interface. One issue that arises here is to +determine what happens if two processes attempt to acquire the same +resource at exactly the same time by using such an instruction. This +requires some mechanism for making a decision about which process gets +control. Such a mechanism is called an "arbiter". Arbiters usually +boil down to some sort of hardware device. Unfortunately, it is +possible to prove that one cannot physically construct a fair arbiter +that works 100% of the time unless one allows the arbiter an +arbitrarily long time to make its decision. The fundamental phenomenon +here was originally observed by the fourteenth-century French +philosopher Jean Buridan in his commentary on Aristotle's De caelo. +Buridan argued that a perfectly rational dog placed between two equally +attractive sources of food will starve to death, because it is +incapable of deciding which to go to first. + + (9) The general technique for avoiding deadlock by numbering the +shared resources and acquiring them in order is due to Havender (1968). +Situations where deadlock cannot be avoided require "deadlock-recovery" +methods, which entail having processes "back out" of the deadlocked +state and try again. Deadlock-recovery mechanisms are widely used in +database management systems, a topic that is treated in detail in Gray +and Reuter 1993. + + (10) One such alternative to serialization is called "barrier +synchronization". The programmer permits concurrent processes to +execute as they please, but establishes certain synchronization points +("barriers") through which no process can proceed until all the +processes have reached the barrier. Modern processors provide machine +instructions that permit programmers to establish synchronization +points at places where consistency is required. The PowerPC^( TM), for +example, includes for this purpose two instructions called SYNC and +EIEIO (Enforced In-order Execution of Input/Output). + + (11) This may seem like a strange point of view, but there are +systems that work this way. International charges to credit-card +accounts, for example, are normally cleared on a per-country basis, and +the charges made in different countries are periodically reconciled. +Thus the account balance may be different in different countries. + + (12) For distributed systems, this perspective was pursued by +Lamport (1978), who showed how to use communication to establish +"global clocks" that can be used to establish orderings on events in +distributed systems. + + +File: sicp.info, Node: 3-5, Prev: 3-4, Up: Chapter 3 + +3.5 Streams +=========== + +We've gained a good understanding of assignment as a tool in modeling, +as well as an appreciation of the complex problems that assignment +raises. It is time to ask whether we could have gone about things in a +different way, so as to avoid some of these problems. In this section, +we explore an alternative approach to modeling state, based on data +structures called "streams". As we shall see, streams can mitigate +some of the complexity of modeling state. + + Let's step back and review where this complexity comes from. In an +attempt to model real-world phenomena, we made some apparently +reasonable decisions: We modeled real-world objects with local state by +computational objects with local variables. We identified time +variation in the real world with time variation in the computer. We +implemented the time variation of the states of the model objects in +the computer with assignments to the local variables of the model +objects. + + Is there another approach? Can we avoid identifying time in the +computer with time in the modeled world? Must we make the model change +with time in order to model phenomena in a changing world? Think about +the issue in terms of mathematical functions. We can describe the +time-varying behavior of a quantity x as a function of time x(t). If +we concentrate on x instant by instant, we think of it as a changing +quantity. Yet if we concentrate on the entire time history of values, +we do not emphasize change--the function itself does not change.(1) + + If time is measured in discrete steps, then we can model a time +function as a (possibly infinite) sequence. In this section, we will +see how to model change in terms of sequences that represent the time +histories of the systems being modeled. To accomplish this, we +introduce new data structures called "streams". From an abstract point +of view, a stream is simply a sequence. However, we will find that the +straightforward implementation of streams as lists (as in section *Note +2-2-1::) doesn't fully reveal the power of stream processing. As an +alternative, we introduce the technique of "delayed evaluation", which +enables us to represent very large (even infinite) sequences as streams. + + Stream processing lets us model systems that have state without ever +using assignment or mutable data. This has important implications, +both theoretical and practical, because we can build models that avoid +the drawbacks inherent in introducing assignment. On the other hand, +the stream framework raises difficulties of its own, and the question +of which modeling technique leads to more modular and more easily +maintained systems remains open. + +* Menu: + +* 3-5-1:: Streams Are Delayed Lists +* 3-5-2:: Infinite Streams +* 3-5-3:: Exploiting the Stream Paradigm +* 3-5-4:: Streams and Delayed Evaluation +* 3-5-5:: Modularity of Functional Programs and Modularity of + Objects + + ---------- Footnotes ---------- + + (1) Physicists sometimes adopt this view by introducing the "world +lines" of particles as a device for reasoning about motion. We've also +already mentioned (section *Note 2-2-3::) that this is the natural way +to think about signal-processing systems. We will explore applications +of streams to signal processing in section *Note 3-5-3::. + + +File: sicp.info, Node: 3-5-1, Next: 3-5-2, Prev: 3-5, Up: 3-5 + +3.5.1 Streams Are Delayed Lists +------------------------------- + +As we saw in section *Note 2-2-3::, sequences can serve as standard +interfaces for combining program modules. We formulated powerful +abstractions for manipulating sequences, such as `map', `filter', and +`accumulate', that capture a wide variety of operations in a manner that +is both succinct and elegant. + + Unfortunately, if we represent sequences as lists, this elegance is +bought at the price of severe inefficiency with respect to both the +time and space required by our computations. When we represent +manipulations on sequences as transformations of lists, our programs +must construct and copy data structures (which may be huge) at every +step of a process. + + To see why this is true, let us compare two programs for computing +the sum of all the prime numbers in an interval. The first program is +written in standard iterative style:(1) + + (define (sum-primes a b) + (define (iter count accum) + (cond ((> count b) accum) + ((prime? count) (iter (+ count 1) (+ count accum))) + (else (iter (+ count 1) accum)))) + (iter a 0)) + + The second program performs the same computation using the sequence +operations of section *Note 2-2-3::: + + (define (sum-primes a b) + (accumulate + + 0 + (filter prime? (enumerate-interval a b)))) + + In carrying out the computation, the first program needs to store +only the sum being accumulated. In contrast, the filter in the second +program cannot do any testing until `enumerate-interval' has +constructed a complete list of the numbers in the interval. The filter +generates another list, which in turn is passed to `accumulate' before +being collapsed to form a sum. Such large intermediate storage is not +needed by the first program, which we can think of as enumerating the +interval incrementally, adding each prime to the sum as it is generated. + + The inefficiency in using lists becomes painfully apparent if we use +the sequence paradigm to compute the second prime in the interval from +10,000 to 1,000,000 by evaluating the expression + + (car (cdr (filter prime? + (enumerate-interval 10000 1000000)))) + + This expression does find the second prime, but the computational +overhead is outrageous. We construct a list of almost a million +integers, filter this list by testing each element for primality, and +then ignore almost all of the result. In a more traditional +programming style, we would interleave the enumeration and the +filtering, and stop when we reached the second prime. + + Streams are a clever idea that allows one to use sequence +manipulations without incurring the costs of manipulating sequences as +lists. With streams we can achieve the best of both worlds: We can +formulate programs elegantly as sequence manipulations, while attaining +the efficiency of incremental computation. The basic idea is to +arrange to construct a stream only partially, and to pass the partial +construction to the program that consumes the stream. If the consumer +attempts to access a part of the stream that has not yet been +constructed, the stream will automatically construct just enough more +of itself to produce the required part, thus preserving the illusion +that the entire stream exists. In other words, although we will write +programs as if we were processing complete sequences, we design our +stream implementation to automatically and transparently interleave the +construction of the stream with its use. + + On the surface, streams are just lists with different names for the +procedures that manipulate them. There is a constructor, +`cons-stream', and two selectors, `stream-car' and `stream-cdr', which +satisfy the constraints + + (stream-car (cons-stream x y)) = x + (stream-cdr (cons-stream x y)) = y + + There is a distinguishable object, `the-empty-stream', which cannot +be the result of any `cons-stream' operation, and which can be +identified with the predicate `stream-null?'.(2) Thus we can make and +use streams, in just the same way as we can make and use lists, to +represent aggregate data arranged in a sequence. In particular, we can +build stream analogs of the list operations from *Note Chapter 2::, +such as `list-ref', `map', and `for-each':(3) + + (define (stream-ref s n) + (if (= n 0) + (stream-car s) + (stream-ref (stream-cdr s) (- n 1)))) + + (define (stream-map proc s) + (if (stream-null? s) + the-empty-stream + (cons-stream (proc (stream-car s)) + (stream-map proc (stream-cdr s))))) + + (define (stream-for-each proc s) + (if (stream-null? s) + 'done + (begin (proc (stream-car s)) + (stream-for-each proc (stream-cdr s))))) + + `Stream-for-each' is useful for viewing streams: + + (define (display-stream s) + (stream-for-each display-line s)) + + (define (display-line x) + (newline) + (display x)) + + To make the stream implementation automatically and transparently +interleave the construction of a stream with its use, we will arrange +for the `cdr' of a stream to be evaluated when it is accessed by the +`stream-cdr' procedure rather than when the stream is constructed by +`cons-stream'. This implementation choice is reminiscent of our +discussion of rational numbers in section *Note 2-1-2::, where we saw +that we can choose to implement rational numbers so that the reduction +of numerator and denominator to lowest terms is performed either at +construction time or at selection time. The two rational-number +implementations produce the same data abstraction, but the choice has +an effect on efficiency. There is a similar relationship between +streams and ordinary lists. As a data abstraction, streams are the +same as lists. The difference is the time at which the elements are +evaluated. With ordinary lists, both the `car' and the `cdr' are +evaluated at construction time. With streams, the `cdr' is evaluated +at selection time. + + Our implementation of streams will be based on a special form called +`delay'. Evaluating `(delay )' does not evaluate the expression +, but rather returns a so-called object "delayed object", which we +can think of as a "promise" to evaluate at some future time. As +a companion to `delay', there is a procedure called `force' that takes +a delayed object as argument and performs the evaluation--in effect, +forcing the `delay' to fulfill its promise. We will see below how +`delay' and `force' can be implemented, but first let us use these to +construct streams. + + `Cons-stream' is a special form defined so that + + (cons-stream ) + +is equivalent to + + (cons (delay )) + + What this means is that we will construct streams using pairs. +However, rather than placing the value of the rest of the stream into +the `cdr' of the pair we will put there a promise to compute the rest +if it is ever requested. `Stream-car' and `stream-cdr' can now be +defined as procedures: + + (define (stream-car stream) (car stream)) + + (define (stream-cdr stream) (force (cdr stream))) + + `Stream-car' selects the `car' of the pair; `stream-cdr' selects the +`cdr' of the pair and evaluates the delayed expression found there to +obtain the rest of the stream.(4) + +The stream implementation in action +................................... + +To see how this implementation behaves, let us analyze the "outrageous" +prime computation we saw above, reformulated in terms of streams: + + (stream-car + (stream-cdr + (stream-filter prime? + (stream-enumerate-interval 10000 1000000)))) + + We will see that it does indeed work efficiently. + + We begin by calling `stream-enumerate-interval' with the arguments +10,000 and 1,000,000. `Stream-enumerate-interval' is the stream analog +of `enumerate-interval' (section *Note 2-2-3::): + + (define (stream-enumerate-interval low high) + (if (> low high) + the-empty-stream + (cons-stream + low + (stream-enumerate-interval (+ low 1) high)))) + +and thus the result returned by `stream-enumerate-interval', formed by +the `cons-stream', is(5) + + (cons 10000 + (delay (stream-enumerate-interval 10001 1000000))) + + That is, `stream-enumerate-interval' returns a stream represented as +a pair whose `car' is 10,000 and whose `cdr' is a promise to enumerate +more of the interval if so requested. This stream is now filtered for +primes, using the stream analog of the `filter' procedure (section +*Note 2-2-3::): + + (define (stream-filter pred stream) + (cond ((stream-null? stream) the-empty-stream) + ((pred (stream-car stream)) + (cons-stream (stream-car stream) + (stream-filter pred + (stream-cdr stream)))) + (else (stream-filter pred (stream-cdr stream))))) + + `Stream-filter' tests the `stream-car' of the stream (the `car' of +the pair, which is 10,000). Since this is not prime, `stream-filter' +examines the `stream-cdr' of its input stream. The call to +`stream-cdr' forces evaluation of the delayed +`stream-enumerate-interval', which now returns + + (cons 10001 + (delay (stream-enumerate-interval 10002 1000000))) + + `Stream-filter' now looks at the `stream-car' of this stream, 10,001, +sees that this is not prime either, forces another `stream-cdr', and so +on, until `stream-enumerate-interval' yields the prime 10,007, whereupon +`stream-filter', according to its definition, returns + + (cons-stream (stream-car stream) + (stream-filter pred (stream-cdr stream))) + +which in this case is + + (cons 10007 + (delay + (stream-filter + prime? + (cons 10008 + (delay + (stream-enumerate-interval 10009 + 1000000)))))) + + This result is now passed to `stream-cdr' in our original expression. +This forces the delayed `stream-filter', which in turn keeps forcing the +delayed `stream-enumerate-interval' until it finds the next prime, which +is 10,009. Finally, the result passed to `stream-car' in our original +expression is + + (cons 10009 + (delay + (stream-filter + prime? + (cons 10010 + (delay + (stream-enumerate-interval 10011 + 1000000)))))) + + `Stream-car' returns 10,009, and the computation is complete. Only +as many integers were tested for primality as were necessary to find +the second prime, and the interval was enumerated only as far as was +necessary to feed the prime filter. + + In general, we can think of delayed evaluation as "demand-driven" +programming, whereby each stage in the stream process is activated only +enough to satisfy the next stage. What we have done is to decouple the +actual order of events in the computation from the apparent structure +of our procedures. We write procedures as if the streams existed "all +at once" when, in reality, the computation is performed incrementally, +as in traditional programming styles. + +Implementing `delay' and `force' +................................ + +Although `delay' and `force' may seem like mysterious operations, their +implementation is really quite straightforward. `Delay' must package +an expression so that it can be evaluated later on demand, and we can +accomplish this simply by treating the expression as the body of a +procedure. `Delay' can be a special form such that + + (delay ) + +is syntactic sugar for + + (lambda () ) + + `Force' simply calls the procedure (of no arguments) produced by +`delay', so we can implement `force' as a procedure: + + (define (force delayed-object) + (delayed-object)) + + This implementation suffices for `delay' and `force' to work as +advertised, but there is an important optimization that we can include. +In many applications, we end up forcing the same delayed object many +times. This can lead to serious inefficiency in recursive programs +involving streams. (See *Note Exercise 3-57::.) The solution is to +build delayed objects so that the first time they are forced, they +store the value that is computed. Subsequent forcings will simply +return the stored value without repeating the computation. In other +words, we implement `delay' as a special-purpose memoized procedure +similar to the one described in *Note Exercise 3-27::. One way to +accomplish this is to use the following procedure, which takes as +argument a procedure (of no arguments) and returns a memoized version +of the procedure. The first time the memoized procedure is run, it +saves the computed result. On subsequent evaluations, it simply +returns the result. + + (define (memo-proc proc) + (let ((already-run? false) (result false)) + (lambda () + (if (not already-run?) + (begin (set! result (proc)) + (set! already-run? true) + result) + result)))) + + `Delay' is then defined so that `(delay )' is equivalent to + + (memo-proc (lambda () )) + +and `force' is as defined previously.(6) + + *Exercise 3.50:* Complete the following definition, which + generalizes `stream-map' to allow procedures that take multiple + arguments, analogous to `map' in section *Note 2-2-3::, footnote + *Note Footnote 12::. + + (define (stream-map proc . argstreams) + (if ( (car argstreams)) + the-empty-stream + ( + (apply proc (map argstreams)) + (apply stream-map + (cons proc (map argstreams)))))) + + *Exercise 3.51:* In order to take a closer look at delayed + evaluation, we will use the following procedure, which simply + returns its argument after printing it: + + (define (show x) + (display-line x) + x) + + What does the interpreter print in response to evaluating each + expression in the following sequence?(7) + + (define x (stream-map show (stream-enumerate-interval 0 10))) + + (stream-ref x 5) + + (stream-ref x 7) + + *Exercise 3.52:* Consider the sequence of expressions + + (define sum 0) + + (define (accum x) + (set! sum (+ x sum)) + sum) + + (define seq (stream-map accum (stream-enumerate-interval 1 20))) + (define y (stream-filter even? seq)) + (define z (stream-filter (lambda (x) (= (remainder x 5) 0)) + seq)) + + (stream-ref y 7) + + (display-stream z) + + What is the value of `sum' after each of the above expressions is + evaluated? What is the printed response to evaluating the + `stream-ref' and `display-stream' expressions? Would these + responses differ if we had implemented `(delay )' simply as + `(lambda () )' without using the optimization provided by + `memo-proc'? Explain + + ---------- Footnotes ---------- + + (1) Assume that we have a predicate `prime?' (e.g., as in section +*Note 1-2-6::) that tests for primality. + + (2) In the MIT implementation, `the-empty-stream' is the same as the +empty list `'()', and `stream-null?' is the same as `null?'. + + (3) This should bother you. The fact that we are defining such +similar procedures for streams and lists indicates that we are missing +some underlying abstraction. Unfortunately, in order to exploit this +abstraction, we will need to exert finer control over the process of +evaluation than we can at present. We will discuss this point further +at the end of section *Note 3-5-4::. In section *Note 4-2::, we'll +develop a framework that unifies lists and streams. + + (4) Although `stream-car' and `stream-cdr' can be defined as +procedures, `cons-stream' must be a special form. If `cons-stream' +were a procedure, then, according to our model of evaluation, +evaluating `(cons-stream )' would automatically cause to be +evaluated, which is precisely what we do not want to happen. For the +same reason, `delay' must be a special form, though `force' can be an +ordinary procedure. + + (5) The numbers shown here do not really appear in the delayed +expression. What actually appears is the original expression, in an +environment in which the variables are bound to the appropriate numbers. +For example, `(+ low 1)' with `low' bound to 10,000 actually appears +where `10001' is shown. + + (6) There are many possible implementations of streams other than +the one described in this section. Delayed evaluation, which is the +key to making streams practical, was inherent in Algol 60's "call-by-name" +parameter-passing method. The use of this mechanism to implement +streams was first described by Landin (1965). Delayed evaluation for +streams was introduced into Lisp by Friedman and Wise (1976). In their +implementation, `cons' always delays evaluating its arguments, so that +lists automatically behave as streams. The memoizing optimization is +also known as "call-by-need". The Algol community would refer to our +original delayed objects as "call-by-name thunks" and to the optimized +versions as "call-by-need thunks". + + (7) Exercises such as *Note Exercise 3-51:: and *Note Exercise +3-52:: are valuable for testing our understanding of how `delay' works. +On the other hand, intermixing delayed evaluation with printing--and, +even worse, with assignment--is extremely confusing, and instructors of +courses on computer languages have traditionally tormented their +students with examination questions such as the ones in this section. +Needless to say, writing programs that depend on such subtleties is +odious programming style. Part of the power of stream processing is +that it lets us ignore the order in which events actually happen in our +programs. Unfortunately, this is precisely what we cannot afford to do +in the presence of assignment, which forces us to be concerned with +time and change. + + +File: sicp.info, Node: 3-5-2, Next: 3-5-3, Prev: 3-5-1, Up: 3-5 + +3.5.2 Infinite Streams +---------------------- + +We have seen how to support the illusion of manipulating streams as +complete entities even though, in actuality, we compute only as much of +the stream as we need to access. We can exploit this technique to +represent sequences efficiently as streams, even if the sequences are +very long. What is more striking, we can use streams to represent +sequences that are infinitely long. For instance, consider the +following definition of the stream of positive integers: + + (define (integers-starting-from n) + (cons-stream n (integers-starting-from (+ n 1)))) + + (define integers (integers-starting-from 1)) + + This makes sense because `integers' will be a pair whose `car' is 1 +and whose `cdr' is a promise to produce the integers beginning with 2. +This is an infinitely long stream, but in any given time we can examine +only a finite portion of it. Thus, our programs will never know that +the entire infinite stream is not there. + + Using `integers' we can define other infinite streams, such as the +stream of integers that are not divisible by 7: + + (define (divisible? x y) (= (remainder x y) 0)) + + (define no-sevens + (stream-filter (lambda (x) (not (divisible? x 7))) + integers)) + + Then we can find integers not divisible by 7 simply by accessing +elements of this stream: + + (stream-ref no-sevens 100) + 117 + + In analogy with `integers', we can define the infinite stream of +Fibonacci numbers: + + (define (fibgen a b) + (cons-stream a (fibgen b (+ a b)))) + + (define fibs (fibgen 0 1)) + + `Fibs' is a pair whose `car' is 0 and whose `cdr' is a promise to +evaluate `(fibgen 1 1)'. When we evaluate this delayed `(fibgen 1 1)', +it will produce a pair whose `car' is 1 and whose `cdr' is a promise to +evaluate `(fibgen 1 2)', and so on. + + For a look at a more exciting infinite stream, we can generalize the +`no-sevens' example to construct the infinite stream of prime numbers, +using a method known as the Eratosthenes "sieve of Eratosthenes".(1) We +start with the integers beginning with 2, which is the first prime. To +get the rest of the primes, we start by filtering the multiples of 2 +from the rest of the integers. This leaves a stream beginning with 3, +which is the next prime. Now we filter the multiples of 3 from the +rest of this stream. This leaves a stream beginning with 5, which is +the next prime, and so on. In other words, we construct the primes by +a sieving process, described as follows: To sieve a stream `S', form a +stream whose first element is the first element of `S' and the rest of +which is obtained by filtering all multiples of the first element of +`S' out of the rest of `S' and sieving the result. This process is +readily described in terms of stream operations: + + (define (sieve stream) + (cons-stream + (stream-car stream) + (sieve (stream-filter + (lambda (x) + (not (divisible? x (stream-car stream)))) + (stream-cdr stream))))) + + (define primes (sieve (integers-starting-from 2))) + + Now to find a particular prime we need only ask for it: + + (stream-ref primes 50) + 233 + + It is interesting to contemplate the signal-processing system set up +by `sieve', shown in the "Henderson diagram" in *Note Figure 3-31::.(2) +The input stream feeds into an "un`cons'er" that separates the first +element of the stream from the rest of the stream. The first element +is used to construct a divisibility filter, through which the rest is +passed, and the output of the filter is fed to another sieve box. Then +the original first element is `cons'ed onto the output of the internal +sieve to form the output stream. Thus, not only is the stream +infinite, but the signal processor is also infinite, because the sieve +contains a sieve within it. + + *Figure 3.31:* The prime sieve viewed as a signal-processing + system. + + +---------------------------------------------------------------+ + | sieve | + | | + | __/| |\__ | + | __/car|........................................| \__ | + | _/ | : | \_ | + ----><_ | V | cons _>----> + | \__ | +------------+ +------------+ | __/ | + | \cdr|--->| filter: | | sieve |--->| __/ | + | \| | |--->| | |/ | + | | not | | | | + | | divisible? | | | | + | +------------+ +------------+ | + +---------------------------------------------------------------+ + +Defining streams implicitly +........................... + +The `integers' and `fibs' streams above were defined by specifying +"generating" procedures that explicitly compute the stream elements one +by one. An alternative way to specify streams is to take advantage of +delayed evaluation to define streams implicitly. For example, the +following expression defines the stream `ones' to be an infinite stream +of ones: + + (define ones (cons-stream 1 ones)) + + This works much like the definition of a recursive procedure: `ones' +is a pair whose `car' is 1 and whose `cdr' is a promise to evaluate +`ones'. Evaluating the `cdr' gives us again a 1 and a promise to +evaluate `ones', and so on. + + We can do more interesting things by manipulating streams with +operations such as `add-streams', which produces the elementwise sum of +two given streams:(3) + + (define (add-streams s1 s2) + (stream-map + s1 s2)) + + Now we can define the integers as follows: + + (define integers (cons-stream 1 (add-streams ones integers))) + + This defines `integers' to be a stream whose first element is 1 and +the rest of which is the sum of `ones' and `integers'. Thus, the second +element of `integers' is 1 plus the first element of `integers', or 2; +the third element of `integers' is 1 plus the second element of +`integers', or 3; and so on. This definition works because, at any +point, enough of the `integers' stream has been generated so that we +can feed it back into the definition to produce the next integer. + + We can define the Fibonacci numbers in the same style: + + (define fibs + (cons-stream 0 + (cons-stream 1 + (add-streams (stream-cdr fibs) + fibs)))) + + This definition says that `fibs' is a stream beginning with 0 and 1, +such that the rest of the stream can be generated by adding `fibs' to +itself shifted by one place: + + 1 1 2 3 5 8 13 21 ... = `(stream-cdr fibs)' + 0 1 1 2 3 5 8 13 ... = `fibs' + 0 1 1 2 3 5 8 13 21 34 ... = `fibs' + + `Scale-stream' is another useful procedure in formulating such stream +definitions. This multiplies each item in a stream by a given constant: + + (define (scale-stream stream factor) + (stream-map (lambda (x) (* x factor)) stream)) + + For example, + + (define double (cons-stream 1 (scale-stream double 2))) + +produces the stream of powers of 2: 1, 2, 4, 8, 16, 32, .... + + An alternate definition of the stream of primes can be given by +starting with the integers and filtering them by testing for primality. +We will need the first prime, 2, to get started: + + (define primes + (cons-stream + 2 + (stream-filter prime? (integers-starting-from 3)))) + + This definition is not so straightforward as it appears, because we +will test whether a number n is prime by checking whether n is +divisible by a prime (not by just any integer) less than or equal to +_[sqrt]_(n): + + (define (prime? n) + (define (iter ps) + (cond ((> (square (stream-car ps)) n) true) + ((divisible? n (stream-car ps)) false) + (else (iter (stream-cdr ps))))) + (iter primes)) + + This is a recursive definition, since `primes' is defined in terms +of the `prime?' predicate, which itself uses the `primes' stream. The +reason this procedure works is that, at any point, enough of the +`primes' stream has been generated to test the primality of the numbers +we need to check next. That is, for every n we test for primality, +either n is not prime (in which case there is a prime already generated +that divides it) or n is prime (in which case there is a prime already +generated--i.e., a prime less than n--that is greater than +_[sqrt]_(n)).(4) + + *Exercise 3.53:* Without running the program, describe the + elements of the stream defined by + + (define s (cons-stream 1 (add-streams s s))) + + *Exercise 3.54:* Define a procedure `mul-streams', analogous to + `add-streams', that produces the elementwise product of its two + input streams. Use this together with the stream of `integers' to + complete the following definition of the stream whose nth element + (counting from 0) is n + 1 factorial: + + (define factorials (cons-stream 1 (mul-streams ))) + + *Exercise 3.55:* Define a procedure `partial-sums' that takes as + argument a stream S and returns the stream whose elements are S_0, + S_0 + S_1, S_0 + S_1 + S_2, .... For example, `(partial-sums + integers)' should be the stream 1, 3, 6, 10, 15, .... + + *Exercise 3.56:* A famous problem, first raised by R. Hamming, is + to enumerate, in ascending order with no repetitions, all positive + integers with no prime factors other than 2, 3, or 5. One obvious + way to do this is to simply test each integer in turn to see + whether it has any factors other than 2, 3, and 5. But this is + very inefficient, since, as the integers get larger, fewer and + fewer of them fit the requirement. As an alternative, let us call + the required stream of numbers `S' and notice the following facts + about it. + + * `S' begins with 1. + + * The elements of `(scale-stream S 2)' are also elements of `S'. + + * The same is true for `(scale-stream S 3)' and `(scale-stream + 5 S)'. + + * These are all the elements of `S'. + + + Now all we have to do is combine elements from these sources. For + this we define a procedure `merge' that combines two ordered + streams into one ordered result stream, eliminating repetitions: + + (define (merge s1 s2) + (cond ((stream-null? s1) s2) + ((stream-null? s2) s1) + (else + (let ((s1car (stream-car s1)) + (s2car (stream-car s2))) + (cond ((< s1car s2car) + (cons-stream s1car (merge (stream-cdr s1) s2))) + ((> s1car s2car) + (cons-stream s2car (merge s1 (stream-cdr s2)))) + (else + (cons-stream s1car + (merge (stream-cdr s1) + (stream-cdr s2))))))))) + + Then the required stream may be constructed with `merge', as + follows: + + (define S (cons-stream 1 (merge ))) + + Fill in the missing expressions in the places marked above. + + *Exercise 3.57:* How many additions are performed when we compute + the nth Fibonacci number using the definition of `fibs' based on + the `add-streams' procedure? Show that the number of additions + would be exponentially greater if we had implemented `(delay + )' simply as `(lambda () )', without using the + optimization provided by the `memo-proc' procedure described in + section *Note 3-5-1::.(5) + + *Exercise 3.58:* Give an interpretation of the stream computed by + the following procedure: + + (define (expand num den radix) + (cons-stream + (quotient (* num radix) den) + (expand (remainder (* num radix) den) den radix))) + + (`Quotient' is a primitive that returns the integer quotient of two + integers.) What are the successive elements produced by `(expand + 1 7 10)'? What is produced by `(expand 3 8 10)'? + + *Exercise 3.59:* In section *Note 2-5-3:: we saw how to implement + a polynomial arithmetic system representing polynomials as lists + of terms. In a similar way, we can work with "power series", such + as + + x^2 x^3 x^4 + e^x = 1 + x + ----- + ----- + --------- + ... + 2 3 * 2 4 * 3 * 2 + + x^2 x^4 + cos x = 1 - ----- + --------- - ... + 2 4 * 3 * 2 + + x^3 x^5 + sin x = x - ----- + ------------- - ... + 3 * 2 5 * 4 * 3 * 2 + + represented as infinite streams. We will represent the series a_0 + + a_1 x + a_2 x^2 + a_3 x^3 + ... as the stream whose elements are + the coefficients a_0, a_1, a_2, a_3, .... + + a. The integral of the series a_0 + a_1 x + a_2 x^2 + a_3 x^3 + + ... is the series + + 1 1 1 + c + a_0 x + --- x_1 r^2 + --- a_2 r^3 + --- a_3 r^4 + ... + 2 3 4 + + where c is any constant. Define a procedure + `integrate-series' that takes as input a stream a_0, a_1, + a_2, ... representing a power series and returns the stream + a_0, (1/2)a_1, (1/3)a_2, ... of coefficients of the + non-constant terms of the integral of the series. (Since the + result has no constant term, it doesn't represent a power + series; when we use `integrate-series', we will `cons' on the + appropriate constant.) + + b. The function x |-> e^x is its own derivative. This implies + that e^x and the integral of e^x are the same series, except + for the constant term, which is e^0 = 1. Accordingly, we can + generate the series for e^x as + + (define exp-series + (cons-stream 1 (integrate-series exp-series))) + + Show how to generate the series for sine and cosine, starting + from the facts that the derivative of sine is cosine and the + derivative of cosine is the negative of sine: + + (define cosine-series + (cons-stream 1 )) + + (define sine-series + (cons-stream 0 )) + + *Exercise 3.60:* With power series represented as streams of + coefficients as in *Note Exercise 3-59::, adding series is + implemented by `add-streams'. Complete the definition of the + following procedure for multiplying series: + + (define (mul-series s1 s2) + (cons-stream (add-streams ))) + + You can test your procedure by verifying that sin^2 x + cos^2 x = + 1, using the series from *Note Exercise 3-59::. + + *Exercise 3.61:* Let S be a power series (*Note Exercise 3-59::) + whose constant term is 1. Suppose we want to find the power + series 1/S, that is, the series X such that S * X = 1. Write S = + 1 + S_R where S_R is the part of S after the constant term. Then + we can solve for X as follows: + + S * X = 1 + (1 + S_R) * X = 1 + X + S_R * X = 1 + X = 1 - S_R * X + + In other words, X is the power series whose constant term is 1 and + whose higher-order terms are given by the negative of S_R times X. + Use this idea to write a procedure `invert-unit-series' that + computes 1/S for a power series S with constant term 1. You will + need to use `mul-series' from *Note Exercise 3-60::. + + *Exercise 3.62:* Use the results of *Note Exercise 3-60:: and + *Note Exercise 3-61:: to define a procedure `div-series' that + divides two power series. `Div-series' should work for any two + series, provided that the denominator series begins with a nonzero + constant term. (If the denominator has a zero constant term, then + `div-series' should signal an error.) Show how to use + `div-series' together with the result of *Note Exercise 3-59:: to + generate the power series for tangent. + + ---------- Footnotes ---------- + + (1) Eratosthenes, a third-century B.C. Alexandrian Greek +philosopher, is famous for giving the first accurate estimate of the +circumference of the Earth, which he computed by observing shadows cast +at noon on the day of the summer solstice. Eratosthenes's sieve method, +although ancient, has formed the basis for special-purpose hardware +"sieves" that, until recently, were the most powerful tools in +existence for locating large primes. Since the 70s, however, these +methods have been superseded by outgrowths of the probabilistic +techniques discussed in section *Note 1-2-6::. + + (2) We have named these figures after Peter Henderson, who was the +first person to show us diagrams of this sort as a way of thinking +about stream processing. Each solid line represents a stream of values +being transmitted. The dashed line from the `car' to the `cons' and +the `filter' indicates that this is a single value rather than a stream. + + (3) This uses the generalized version of `stream-map' from *Note +Exercise 3-50::. + + (4) This last point is very subtle and relies on the fact that +p_(n+1) <= p_n^2. (Here, p_k denotes the kth prime.) Estimates such +as these are very difficult to establish. The ancient proof by Euclid +that there are an infinite number of primes shows that p_(n+1)<= p_1 +p_2...p_n + 1, and no substantially better result was proved until +1851, when the Russian mathematician P. L. Chebyshev established that +p_(n+1)<= 2p_n for all n. This result, originally conjectured in 1845, +is known as hypothesis "Bertrand's hypothesis". A proof can be found +in section 22.3 of Hardy and Wright 1960. + + (5) This exercise shows how call-by-need is closely related to +ordinary memoization as described in *Note Exercise 3-27::. In that +exercise, we used assignment to explicitly construct a local table. +Our call-by-need stream optimization effectively constructs such a +table automatically, storing values in the previously forced parts of +the stream. + + +File: sicp.info, Node: 3-5-3, Next: 3-5-4, Prev: 3-5-2, Up: 3-5 + +3.5.3 Exploiting the Stream Paradigm +------------------------------------ + +Streams with delayed evaluation can be a powerful modeling tool, +providing many of the benefits of local state and assignment. +Moreover, they avoid some of the theoretical tangles that accompany the +introduction of assignment into a programming language. + + The stream approach can be illuminating because it allows us to +build systems with different module boundaries than systems organized +around assignment to state variables. For example, we can think of an +entire time series (or signal) as a focus of interest, rather than the +values of the state variables at individual moments. This makes it +convenient to combine and compare components of state from different +moments. + +Formulating iterations as stream processes +.......................................... + +In section *Note 1-2-1::, we introduced iterative processes, which +proceed by updating state variables. We know now that we can represent +state as a "timeless" stream of values rather than as a set of +variables to be updated. Let's adopt this perspective in revisiting +the square-root procedure from section *Note 1-1-7::. Recall that the +idea is to generate a sequence of better and better guesses for the +square root of x by applying over and over again the procedure that +improves guesses: + + (define (sqrt-improve guess x) + (average guess (/ x guess))) + + In our original `sqrt' procedure, we made these guesses be the +successive values of a state variable. Instead we can generate the +infinite stream of guesses, starting with an initial guess of 1:(1) + + (define (sqrt-stream x) + (define guesses + (cons-stream 1.0 + (stream-map (lambda (guess) + (sqrt-improve guess x)) + guesses))) + guesses) + + (display-stream (sqrt-stream 2)) + 1. + 1.5 + 1.4166666666666665 + 1.4142156862745097 + 1.4142135623746899 + ... + + We can generate more and more terms of the stream to get better and +better guesses. If we like, we can write a procedure that keeps +generating terms until the answer is good enough. (See *Note Exercise +3-64::.) + + Another iteration that we can treat in the same way is to generate an +approximation to [pi], based upon the alternating series that we saw in +section *Note 1-3-1::: + + [pi] 1 1 1 + ---- = 1 - --- + --- - --- + ... + 4 3 5 7 + + We first generate the stream of summands of the series (the +reciprocals of the odd integers, with alternating signs). Then we take +the stream of sums of more and more terms (using the `partial-sums' +procedure of *Note Exercise 3-55::) and scale the result by 4: + + (define (pi-summands n) + (cons-stream (/ 1.0 n) + (stream-map - (pi-summands (+ n 2))))) + + (define pi-stream + (scale-stream (partial-sums (pi-summands 1)) 4)) + + (display-stream pi-stream) + 4. + 2.666666666666667 + 3.466666666666667 + 2.8952380952380956 + 3.3396825396825403 + 2.9760461760461765 + 3.2837384837384844 + 3.017071817071818 + ... + + This gives us a stream of better and better approximations to [pi], +although the approximations converge rather slowly. Eight terms of the +sequence bound the value of [pi] between 3.284 and 3.017. + + So far, our use of the stream of states approach is not much +different from updating state variables. But streams give us an +opportunity to do some interesting tricks. For example, we can +transform a stream with a "sequence accelerator" that converts a +sequence of approximations to a new sequence that converges to the same +value as the original, only faster. + + One such accelerator, due to the eighteenth-century Swiss +mathematician Leonhard Euler, works well with sequences that are +partial sums of alternating series (series of terms with alternating +signs). In Euler's technique, if S_n is the nth term of the original +sum sequence, then the accelerated sequence has terms + + (S_(n+1) - S_n)^2 + S_(n+1) - ------------------------ + S_(n-1) - 2S_n + S_(n+1) + + Thus, if the original sequence is represented as a stream of values, +the transformed sequence is given by + + (define (euler-transform s) + (let ((s0 (stream-ref s 0)) ; S_(n-1) + (s1 (stream-ref s 1)) ; S_n + (s2 (stream-ref s 2))) ; S_(n+1) + (cons-stream (- s2 (/ (square (- s2 s1)) + (+ s0 (* -2 s1) s2))) + (euler-transform (stream-cdr s))))) + + We can demonstrate Euler acceleration with our sequence of +approximations to [pi]: + + (display-stream (euler-transform pi-stream)) + 3.166666666666667 + 3.1333333333333337 + 3.1452380952380956 + 3.13968253968254 + 3.1427128427128435 + 3.1408813408813416 + 3.142071817071818 + 3.1412548236077655 + ... + + Even better, we can accelerate the accelerated sequence, and +recursively accelerate that, and so on. Namely, we create a stream of +streams (a structure we'll call a "tableau") in which each stream is +the transform of the preceding one: + + (define (make-tableau transform s) + (cons-stream s + (make-tableau transform + (transform s)))) + + The tableau has the form + + s_00 s_01 s_02 s_03 s_04 ... + s_10 s_11 s_12 s_13 ... + s_20 s_21 s_22 ... + ... + + Finally, we form a sequence by taking the first term in each row of +the tableau: + + (define (accelerated-sequence transform s) + (stream-map stream-car + (make-tableau transform s))) + + We can demonstrate this kind of "super-acceleration" of the [pi] +sequence: + + (display-stream (accelerated-sequence euler-transform + pi-stream)) + 4. + 3.166666666666667 + 3.142105263157895 + 3.141599357319005 + 3.1415927140337785 + 3.1415926539752927 + 3.1415926535911765 + 3.141592653589778 + ... + + The result is impressive. Taking eight terms of the sequence yields +the correct value of [pi] to 14 decimal places. If we had used only the +original [pi] sequence, we would need to compute on the order of 10^13 +terms (i.e., expanding the series far enough so that the individual +terms are less then 10^(-13)) to get that much accuracy! + + We could have implemented these acceleration techniques without +using streams. But the stream formulation is particularly elegant and +convenient because the entire sequence of states is available to us as +a data structure that can be manipulated with a uniform set of +operations. + + *Exercise 3.63:* Louis Reasoner asks why the `sqrt-stream' + procedure was not written in the following more straightforward + way, without the local variable `guesses': + + (define (sqrt-stream x) + (cons-stream 1.0 + (stream-map (lambda (guess) + (sqrt-improve guess x)) + (sqrt-stream x)))) + + Alyssa P. Hacker replies that this version of the procedure is + considerably less efficient because it performs redundant + computation. Explain Alyssa's answer. Would the two versions + still differ in efficiency if our implementation of `delay' used + only `(lambda () )' without using the optimization provided + by `memo-proc' (section *Note 3-5-1::)? + + *Exercise 3.64:* Write a procedure `stream-limit' that takes as + arguments a stream and a number (the tolerance). It should + examine the stream until it finds two successive elements that + differ in absolute value by less than the tolerance, and return + the second of the two elements. Using this, we could compute + square roots up to a given tolerance by + + (define (sqrt x tolerance) + (stream-limit (sqrt-stream x) tolerance)) + + *Exercise 3.65:* Use the series + + 1 1 1 + ln 2 = 1 - --- + --- - --- + ... + 2 3 4 + + to compute three sequences of approximations to the natural + logarithm of 2, in the same way we did above for [pi]. How + rapidly do these sequences converge? + +Infinite streams of pairs +......................... + +In section *Note 2-2-3::, we saw how the sequence paradigm handles +traditional nested loops as processes defined on sequences of pairs. +If we generalize this technique to infinite streams, then we can write +programs that are not easily represented as loops, because the +"looping" must range over an infinite set. + + For example, suppose we want to generalize the `prime-sum-pairs' +procedure of section *Note 2-2-3:: to produce the stream of pairs of +_all_ integers (i,j) with i <= j such that i + j is prime. If +`int-pairs' is the sequence of all pairs of integers (i,j) with i <= j, +then our required stream is simply(2) + + (stream-filter (lambda (pair) + (prime? (+ (car pair) (cadr pair)))) + int-pairs) + + Our problem, then, is to produce the stream `int-pairs'. More +generally, suppose we have two streams S = (S_i) and T = (T_j), and +imagine the infinite rectangular array + + (S_0, T_0) (S_0, T_1) (S_0, T_2) ... + (S_1, T_0) (S_1, T_1) (S_1, T_2) ... + (S_2, T_0) (S_2, T_1) (S_2, T_2) ... + ... + + We wish to generate a stream that contains all the pairs in the +array that lie on or above the diagonal, i.e., the pairs + + (S_0, T_0) (S_0, T_1) (S_0, T_2) ... + (S_1, T_1) (S_1, T_2) ... + (S_2, T_2) ... + ... + +(If we take both S and T to be the stream of integers, then this will +be our desired stream `int-pairs'.) + + Call the general stream of pairs `(pairs S T)', and consider it to be +composed of three parts: the pair (S_0,T_0), the rest of the pairs in +the first row, and the remaining pairs:(3) + + (S_0, T_0) | (S_0, T_1) (S_0, T_2) ... + -----------+----------------------------- + | (S_1, T_1) (S_1, T_2) ... + | (S_2, T_2) ... + | ... + + Observe that the third piece in this decomposition (pairs that are +not in the first row) is (recursively) the pairs formed from +`(stream-cdr S)' and `(stream-cdr T)'. Also note that the second piece +(the rest of the first row) is + + (stream-map (lambda (x) (list (stream-car s) x)) + (stream-cdr t)) + + Thus we can form our stream of pairs as follows: + + (define (pairs s t) + (cons-stream + (list (stream-car s) (stream-car t)) + ( + (stream-map (lambda (x) (list (stream-car s) x)) + (stream-cdr t)) + (pairs (stream-cdr s) (stream-cdr t))))) + + In order to complete the procedure, we must choose some way to +combine the two inner streams. One idea is to use the stream analog of +the `append' procedure from section *Note 2-2-1::: + + (define (stream-append s1 s2) + (if (stream-null? s1) + s2 + (cons-stream (stream-car s1) + (stream-append (stream-cdr s1) s2)))) + + This is unsuitable for infinite streams, however, because it takes +all the elements from the first stream before incorporating the second +stream. In particular, if we try to generate all pairs of positive +integers using + + (pairs integers integers) + +our stream of results will first try to run through all pairs with the +first integer equal to 1, and hence will never produce pairs with any +other value of the first integer. + + To handle infinite streams, we need to devise an order of +combination that ensures that every element will eventually be reached +if we let our program run long enough. An elegant way to accomplish +this is with the following `interleave' procedure:(4) + + (define (interleave s1 s2) + (if (stream-null? s1) + s2 + (cons-stream (stream-car s1) + (interleave s2 (stream-cdr s1))))) + + Since `interleave' takes elements alternately from the two streams, +every element of the second stream will eventually find its way into +the interleaved stream, even if the first stream is infinite. + + We can thus generate the required stream of pairs as + + (define (pairs s t) + (cons-stream + (list (stream-car s) (stream-car t)) + (interleave + (stream-map (lambda (x) (list (stream-car s) x)) + (stream-cdr t)) + (pairs (stream-cdr s) (stream-cdr t))))) + + *Exercise 3.66:* Examine the stream `(pairs integers integers)'. + Can you make any general comments about the order in which the + pairs are placed into the stream? For example, about how many + pairs precede the pair (1,100)? the pair (99,100)? the pair + (100,100)? (If you can make precise mathematical statements here, + all the better. But feel free to give more qualitative answers if + you find yourself getting bogged down.) + + *Exercise 3.67:* Modify the `pairs' procedure so that `(pairs + integers integers)' will produce the stream of _all_ pairs of + integers (i,j) (without the condition i <= j). Hint: You will + need to mix in an additional stream. + + *Exercise 3.68:* Louis Reasoner thinks that building a stream of + pairs from three parts is unnecessarily complicated. Instead of + separating the pair (S_0,T_0) from the rest of the pairs in the + first row, he proposes to work with the whole first row, as + follows: + + (define (pairs s t) + (interleave + (stream-map (lambda (x) (list (stream-car s) x)) + t) + (pairs (stream-cdr s) (stream-cdr t)))) + + Does this work? Consider what happens if we evaluate `(pairs + integers integers)' using Louis's definition of `pairs'. + + *Exercise 3.69:* Write a procedure `triples' that takes three + infinite streams, S, T, and U, and produces the stream of triples + (S_i,T_j,U_k) such that i <= j <= k. Use `triples' to generate + the stream of all Pythagorean triples of positive integers, i.e., + the triples (i,j,k) such that i <= j and i^2 + j^2 = k^2. + + *Exercise 3.70:* It would be nice to be able to generate streams + in which the pairs appear in some useful order, rather than in the + order that results from an _ad hoc_ interleaving process. We can + use a technique similar to the `merge' procedure of *Note Exercise + 3-56::, if we define a way to say that one pair of integers is + "less than" another. One way to do this is to define a "weighting + function" W(i,j) and stipulate that (i_1,j_1) is less than + (i_2,j_2) if W(i_1,j_1) < W(i_2,j_2). Write a procedure + `merge-weighted' that is like `merge', except that + `merge-weighted' takes an additional argument `weight', which is a + procedure that computes the weight of a pair, and is used to + determine the order in which elements should appear in the + resulting merged stream.(5) Using this, generalize `pairs' to a + procedure `weighted-pairs' that takes two streams, together with a + procedure that computes a weighting function, and generates the + stream of pairs, ordered according to weight. Use your procedure + to generate + + a. the stream of all pairs of positive integers (i,j) with i <= j + ordered according to the sum i + j + + b. the stream of all pairs of positive integers (i,j) with i <= + j, where neither i nor j is divisible by 2, 3, or 5, and the + pairs are ordered according to the sum 2 i + 3 j + 5 i j. + + + *Exercise 3.71:* Numbers that can be expressed as the sum of two + cubes in more than one way are sometimes called "Ramanujan + numbers", in honor of the mathematician Srinivasa Ramanujan.(6) + Ordered streams of pairs provide an elegant solution to the + problem of computing these numbers. To find a number that can be + written as the sum of two cubes in two different ways, we need + only generate the stream of pairs of integers (i,j) weighted + according to the sum i^3 + j^3 (see *Note Exercise 3-70::), then + search the stream for two consecutive pairs with the same weight. + Write a procedure to generate the Ramanujan numbers. The first + such number is 1,729. What are the next five? + + *Exercise 3.72:* In a similar way to *Note Exercise 3-71:: + generate a stream of all numbers that can be written as the sum of + two squares in three different ways (showing how they can be so + written). + +Streams as signals +.................. + +We began our discussion of streams by describing them as computational +analogs of the "signals" in signal-processing systems. In fact, we can +use streams to model signal-processing systems in a very direct way, +representing the values of a signal at successive time intervals as +consecutive elements of a stream. For instance, we can implement an "integrator" +or "summer" that, for an input stream x = (x_i), an initial value C, +and a small increment dt, accumulates the sum + + i + --- + S_i = C + > x_j dt + --- + j=1 + +and returns the stream of values S = (S_i). The following `integral' +procedure is reminiscent of the "implicit style" definition of the +stream of integers (section *Note 3-5-2::): + + (define (integral integrand initial-value dt) + (define int + (cons-stream initial-value + (add-streams (scale-stream integrand dt) + int))) + int) + + *Note Figure 3-32:: is a picture of a signal-processing system that +corresponds to the `integral' procedure. The input stream is scaled by +dt and passed through an adder, whose output is passed back through the +same adder. The self-reference in the definition of `int' is reflected +in the figure by the feedback loop that connects the output of the +adder to one of the inputs. + + *Figure 3.32:* The `integral' procedure viewed as a + signal-processing system. + + initial-value + | + +-----------+ | |\__ + input | | |\__ +-->| \_ integral + ------>| scale: dt +----->| \_ |cons_>--*-------> + | | | add_>---->| __/ | + +-----------+ +-->| __/ |/ | + | |/ | + | | + +------------------------+ + + *Exercise 3.73:* We can model electrical circuits using streams to + represent the values of currents or voltages at a sequence of + times. For instance, suppose we have an "RC circuit" consisting + of a resistor of resistance R and a capacitor of capacitance C in + series. The voltage response v of the circuit to an injected + current i is determined by the formula in *Note Figure 3-33::, + whose structure is shown by the accompanying signal-flow diagram. + + Write a procedure `RC' that models this circuit. `RC' should take + as inputs the values of R, C, and dt and should return a procedure + that takes as inputs a stream representing the current i and an + initial value for the capacitor voltage v_0 and produces as output + the stream of voltages v. For example, you should be able to use + `RC' to model an RC circuit with R = 5 ohms, C = 1 farad, and a + 0.5-second time step by evaluating `(define RC1 (RC 5 1 0.5))'. + This defines `RC1' as a procedure that takes a stream representing + the time sequence of currents and an initial capacitor voltage and + produces the output stream of voltages. + + *Figure 3.33:* An RC circuit and the associated signal-flow + diagram. + + + - + ->----'\/\/\,---| |--- + i C + + / t + | i + v = v + | dt + R i + 0 | + / 0 + + +--------------+ + +-->| scale: R |---------------------+ |\_ + | +--------------+ | | \_ + | +-->| \ v + i | +--------------+ +------------+ | add >---> + ----+-->| scale: 1/C |---->| integral |----->| _/ + +--------------+ +------------+ | _/ + |/ + + *Exercise 3.74:* Alyssa P. Hacker is designing a system to process + signals coming from physical sensors. One important feature she + wishes to produce is a signal that describes the "zero crossings" + of the input signal. That is, the resulting signal should be + 1 + whenever the input signal changes from negative to positive, - 1 + whenever the input signal changes from positive to negative, and 0 + otherwise. (Assume that the sign of a 0 input is positive.) For + example, a typical input signal with its associated zero-crossing + signal would be + + ... 1 2 1.5 1 0.5 -0.1 -2 -3 -2 -0.5 0.2 3 4 ... + ... 0 0 0 0 0 -1 0 0 0 0 1 0 0 ... + + In Alyssa's system, the signal from the sensor is represented as a + stream `sense-data' and the stream `zero-crossings' is the + corresponding stream of zero crossings. Alyssa first writes a + procedure `sign-change-detector' that takes two values as + arguments and compares the signs of the values to produce an + appropriate 0, 1, or - 1. She then constructs her zero-crossing + stream as follows: + + (define (make-zero-crossings input-stream last-value) + (cons-stream + (sign-change-detector (stream-car input-stream) last-value) + (make-zero-crossings (stream-cdr input-stream) + (stream-car input-stream)))) + + (define zero-crossings (make-zero-crossings sense-data 0)) + + Alyssa's boss, Eva Lu Ator, walks by and suggests that this + program is approximately equivalent to the following one, which + uses the generalized version of `stream-map' from *Note Exercise + 3-50::: + + (define zero-crossings + (stream-map sign-change-detector sense-data )) + + Complete the program by supplying the indicated . + + *Exercise 3.75:* Unfortunately, Alyssa's zero-crossing detector in + *Note Exercise 3-74:: proves to be insufficient, because the noisy + signal from the sensor leads to spurious zero crossings. Lem E. + Tweakit, a hardware specialist, suggests that Alyssa smooth the + signal to filter out the noise before extracting the zero + crossings. Alyssa takes his advice and decides to extract the + zero crossings from the signal constructed by averaging each value + of the sense data with the previous value. She explains the + problem to her assistant, Louis Reasoner, who attempts to + implement the idea, altering Alyssa's program as follows: + + (define (make-zero-crossings input-stream last-value) + (let ((avpt (/ (+ (stream-car input-stream) last-value) 2))) + (cons-stream (sign-change-detector avpt last-value) + (make-zero-crossings (stream-cdr input-stream) + avpt)))) + + This does not correctly implement Alyssa's plan. Find the bug + that Louis has installed and fix it without changing the structure + of the program. (Hint: You will need to increase the number of + arguments to `make-zero-crossings'.) + + *Exercise 3.76:* Eva Lu Ator has a criticism of Louis's approach + in *Note Exercise 3-75::. The program he wrote is not modular, + because it intermixes the operation of smoothing with the + zero-crossing extraction. For example, the extractor should not + have to be changed if Alyssa finds a better way to condition her + input signal. Help Louis by writing a procedure `smooth' that + takes a stream as input and produces a stream in which each + element is the average of two successive input stream elements. + Then use `smooth' as a component to implement the zero-crossing + detector in a more modular style. + + ---------- Footnotes ---------- + + (1) We can't use `let' to bind the local variable `guesses', because +the value of `guesses' depends on `guesses' itself. *Note Exercise +3-63:: addresses why we want a local variable here. + + (2) As in section *Note 2-2-3::, we represent a pair of integers as +a list rather than a Lisp pair. + + (3) See *Note Exercise 3-68:: for some insight into why we chose +this decomposition. + + (4) The precise statement of the required property on the order of +combination is as follows: There should be a function f of two +arguments such that the pair corresponding to element i of the first +stream and element j of the second stream will appear as element number +f(i,j) of the output stream. The trick of using `interleave' to +accomplish this was shown to us by David Turner, who employed it in the +language KRC (Turner 1981). + + (5) We will require that the weighting function be such that the +weight of a pair increases as we move out along a row or down along a +column of the array of pairs. + + (6) To quote from G. H. Hardy's obituary of Ramanujan (Hardy 1921): +"It was Mr. Littlewood (I believe) who remarked that `every positive +integer was one of his friends.' I remember once going to see him when +he was lying ill at Putney. I had ridden in taxi-cab No. 1729, and +remarked that the number seemed to me a rather dull one, and that I +hoped it was not an unfavorable omen. `No,' he replied, `it is a very +interesting number; it is the smallest number expressible as the sum of +two cubes in two different ways.' " The trick of using weighted pairs +to generate the Ramanujan numbers was shown to us by Charles Leiserson. + + +File: sicp.info, Node: 3-5-4, Next: 3-5-5, Prev: 3-5-3, Up: 3-5 + +3.5.4 Streams and Delayed Evaluation +------------------------------------ + +The `integral' procedure at the end of the preceding section shows how +we can use streams to model signal-processing systems that contain +feedback loops. The feedback loop for the adder shown in *Note Figure +3-32:: is modeled by the fact that `integral''s internal stream `int' +is defined in terms of itself: + + (define int + (cons-stream initial-value + (add-streams (scale-stream integrand dt) + int))) + + The interpreter's ability to deal with such an implicit definition +depends on the `delay' that is incorporated into `cons-stream'. +Without this `delay', the interpreter could not construct `int' before +evaluating both arguments to `cons-stream', which would require that +`int' already be defined. In general, `delay' is crucial for using +streams to model signal-processing systems that contain loops. Without +`delay', our models would have to be formulated so that the inputs to +any signal-processing component would be fully evaluated before the +output could be produced. This would outlaw loops. + + Unfortunately, stream models of systems with loops may require uses +of `delay' beyond the "hidden" `delay' supplied by `cons-stream'. For +instance, *Note Figure 3-34:: shows a signal-processing system for +solving the differential equation dy/dt = f(y) where f is a given +function. The figure shows a mapping component, which applies f to its +input signal, linked in a feedback loop to an integrator in a manner +very similar to that of the analog computer circuits that are actually +used to solve such equations. + + *Figure 3.34:* An "analog computer circuit" that solves the + equation dy/dt = f(y). + + y_0 + | + V + +----------+ dy +----------+ y + +-->| map: f +------>| integral +--*-----> + | +----------+ +----------+ | + | | + +------------------------------------+ + + Assuming we are given an initial value y_0 for y, we could try to +model this system using the procedure + + (define (solve f y0 dt) + (define y (integral dy y0 dt)) + (define dy (stream-map f y)) + y) + + This procedure does not work, because in the first line of `solve' +the call to `integral' requires that the input `dy' be defined, which +does not happen until the second line of `solve'. + + On the other hand, the intent of our definition does make sense, +because we can, in principle, begin to generate the `y' stream without +knowing `dy'. Indeed, `integral' and many other stream operations have +properties similar to those of `cons-stream', in that we can generate +part of the answer given only partial information about the arguments. +For `integral', the first element of the output stream is the specified +`initial-value'. Thus, we can generate the first element of the output +stream without evaluating the integrand `dy'. Once we know the first +element of `y', the `stream-map' in the second line of `solve' can +begin working to generate the first element of `dy', which will produce +the next element of `y', and so on. + + To take advantage of this idea, we will redefine `integral' to +expect the integrand stream to be a "delayed argument". `Integral' will +`force' the integrand to be evaluated only when it is required to +generate more than the first element of the output stream: + + (define (integral delayed-integrand initial-value dt) + (define int + (cons-stream initial-value + (let ((integrand (force delayed-integrand))) + (add-streams (scale-stream integrand dt) + int)))) + int) + + Now we can implement our `solve' procedure by delaying the +evaluation of `dy' in the definition of `y':(1) + + (define (solve f y0 dt) + (define y (integral (delay dy) y0 dt)) + (define dy (stream-map f y)) + y) + + In general, every caller of `integral' must now `delay' the integrand +argument. We can demonstrate that the `solve' procedure works by +approximating eapprox 2.718 by computing the value at y = 1 of the +solution to the differential equation dy/dt = y with initial condition +y(0) = 1: + + (stream-ref (solve (lambda (y) y) 1 0.001) 1000) + 2.716924 + + *Exercise 3.77:* The `integral' procedure used above was analogous + to the "implicit" definition of the infinite stream of integers in + section *Note 3-5-2::. Alternatively, we can give a definition of + `integral' that is more like `integers-starting-from' (also in + section *Note 3-5-2::): + + (define (integral integrand initial-value dt) + (cons-stream initial-value + (if (stream-null? integrand) + the-empty-stream + (integral (stream-cdr integrand) + (+ (* dt (stream-car integrand)) + initial-value) + dt)))) + + When used in systems with loops, this procedure has the same + problem as does our original version of `integral'. Modify the + procedure so that it expects the `integrand' as a delayed argument + and hence can be used in the `solve' procedure shown above. + + *Figure 3.35:* Signal-flow diagram for the solution to a + second-order linear differential equation. + + dy_0 y_0 + | | + V V + ddy +----------+ dy +----------+ y + +--------->| integral +-----*--+ integral +--*---> + | +----------+ | +----------+ | + | | | + | +----------+ | | + | __/|<--+ scale: a |<--+ | + | _/ | +----------+ | + +--<_add | | + \__ | +----------+ | + \|<--+ scale: b |<-------------------+ + +----------+ + + *Exercise 3.78:* Consider the problem of designing a + signal-processing system to study the homogeneous second-order + linear differential equation + + d^2 y d y + ----- - a ----- - by = 0 + d t^2 d t + + The output stream, modeling y, is generated by a network that + contains a loop. This is because the value of d^2y/dt^2 depends + upon the values of y and dy/dt and both of these are determined by + integrating d^2y/dt^2. The diagram we would like to encode is + shown in *Note Figure 3-35::. Write a procedure `solve-2nd' that + takes as arguments the constants a, b, and dt and the initial + values y_0 and dy_0 for y and dy/dt and generates the stream of + successive values of y. + + *Exercise 3.79:* Generalize the `solve-2nd' procedure of *Note + Exercise 3-78:: so that it can be used to solve general + second-order differential equations d^2 y/dt^2 = f(dy/dt, y). + + *Exercise 3.80:* A "series RLC circuit" consists of a resistor, a + capacitor, and an inductor connected in series, as shown in *Note + Figure 3-36::. If R, L, and C are the resistance, inductance, and + capacitance, then the relations between voltage (v) and current + (i) for the three components are described by the equations + + v_R = i_R R + + d_(i L) + v_L = L --------- + d t + + d v_C + i_C = C ------- + d t + + and the circuit connections dictate the relations + + i_R = i_L = -i_C + + v_C = v_L + v_R + + Combining these equations shows that the state of the circuit + (summarized by v_C, the voltage across the capacitor, and i_L, the + current in the inductor) is described by the pair of differential + equations + + d v_C i_L + ----- = - --- + d t C + + d i_L 1 R + ----- = --- v_C - --- i_L + d t L L + + The signal-flow diagram representing this system of differential + equations is shown in *Note Figure 3-37::. + + *Figure 3.36:* A series RLC circuit. + + v_R - + i_R + +--->----'\/\/\,--------+ + | | i_L + \|/ R \|/ + + | i_C |_ + + -+- __) + v_C -+- C (_) v_L + | __) + - | | - + +-----------------------+ + + *Figure 3.37:* A signal-flow diagram for the solution to a series + RLC circuit. + + +-------------+ + +----------------+ scale: l/L |<--+ + | +-------------+ | + | | + | +-------------+ | v_C + | dv_C +-->| integral +---*------> + | | +-------------+ + | | ^ + | | | v_(C_0) + | | + | | +-------------+ + | +---+ scale: -l/C |<--+ + | +-------------+ | + | |\__ | + +->| \_ di_L +-------------+ | i_L + | add_>------>| integral +---*------> + +->| __/ +-------------+ | + | |/ ^ | + | | i_(L_0) | + | | + | +-------------+ | + +----------------+ scale: -R/L |<--+ + +-------------+ + + Write a procedure `RLC' that takes as arguments the parameters R, L, +and C of the circuit and the time increment dt. In a manner similar to +that of the `RC' procedure of *Note Exercise 3-73::, `RLC' should +produce a procedure that takes the initial values of the state +variables, v_(C_0) and i_(L_0), and produces a pair (using `cons') of +the streams of states v_C and i_L. Using `RLC', generate the pair of +streams that models the behavior of a series RLC circuit with R = 1 +ohm, C = 0.2 farad, L = 1 henry, dt = 0.1 second, and initial values +i_(L_0) = 0 amps and v_(C_0) = 10 volts. + +Normal-order evaluation +....................... + +The examples in this section illustrate how the explicit use of `delay' +and `force' provides great programming flexibility, but the same +examples also show how this can make our programs more complex. Our +new `integral' procedure, for instance, gives us the power to model +systems with loops, but we must now remember that `integral' should be +called with a delayed integrand, and every procedure that uses +`integral' must be aware of this. In effect, we have created two +classes of procedures: ordinary procedures and procedures that take +delayed arguments. In general, creating separate classes of procedures +forces us to create separate classes of higher-order procedures as +well.(2) + + One way to avoid the need for two different classes of procedures is +to make all procedures take delayed arguments. We could adopt a model +of evaluation in which all arguments to procedures are automatically +delayed and arguments are forced only when they are actually needed +(for example, when they are required by a primitive operation). This +would transform our language to use normal-order evaluation, which we +first described when we introduced the substitution model for +evaluation in section *Note 1-1-5::. Converting to normal-order +evaluation provides a uniform and elegant way to simplify the use of +delayed evaluation, and this would be a natural strategy to adopt if we +were concerned only with stream processing. In section *Note 4-2::, +after we have studied the evaluator, we will see how to transform our +language in just this way. Unfortunately, including delays in +procedure calls wreaks havoc with our ability to design programs that +depend on the order of events, such as programs that use assignment, +mutate data, or perform input or output. Even the single `delay' in +`cons-stream' can cause great confusion, as illustrated by *Note +Exercise 3-51:: and *Note Exercise 3-52::. As far as anyone knows, +mutability and delayed evaluation do not mix well in programming +languages, and devising ways to deal with both of these at once is an +active area of research. + + ---------- Footnotes ---------- + + (1) This procedure is not guaranteed to work in all Scheme +implementations, although for any implementation there is a simple +variation that will work. The problem has to do with subtle +differences in the ways that Scheme implementations handle internal +definitions. (See section *Note 4-1-6::.) + + (2) This is a small reflection, in Lisp, of the difficulties that +conventional strongly typed languages such as Pascal have in coping with +higher-order procedures. In such languages, the programmer must +specify the data types of the arguments and the result of each +procedure: number, logical value, sequence, and so on. Consequently, +we could not express an abstraction such as "map a given procedure +`proc' over all the elements in a sequence" by a single higher-order +procedure such as `stream-map'. Rather, we would need a different +mapping procedure for each different combination of argument and result +data types that might be specified for a `proc'. Maintaining a +practical notion of "data type" in the presence of higher-order +procedures raises many difficult issues. One way of dealing with this +problem is illustrated by the language ML (Gordon, Milner, and +Wadsworth 1979), whose "polymorphic data types" include templates for +higher-order transformations between data types. Moreover, data types +for most procedures in ML are never explicitly declared by the +programmer. Instead, ML includes a "type-inferencing" mechanism that +uses information in the environment to deduce the data types for newly +defined procedures. + + +File: sicp.info, Node: 3-5-5, Prev: 3-5-4, Up: 3-5 + +3.5.5 Modularity of Functional Programs and Modularity of Objects +----------------------------------------------------------------- + +As we saw in section *Note 3-1-2::, one of the major benefits of +introducing assignment is that we can increase the modularity of our +systems by encapsulating, or "hiding," parts of the state of a large +system within local variables. Stream models can provide an equivalent +modularity without the use of assignment. As an illustration, we can +reimplement the Monte Carlo estimation of [pi], which we examined in +section *Note 3-1-2::, from a stream-processing point of view. + + The key modularity issue was that we wished to hide the internal +state of a random-number generator from programs that used random +numbers. We began with a procedure `rand-update', whose successive +values furnished our supply of random numbers, and used this to produce +a random-number generator: + + (define rand + (let ((x random-init)) + (lambda () + (set! x (rand-update x)) + x))) + + In the stream formulation there is no random-number generator _per +se_, just a stream of random numbers produced by successive calls to +`rand-update': + + (define random-numbers + (cons-stream random-init + (stream-map rand-update random-numbers))) + + We use this to construct the stream of outcomes of the Cesa`ro +experiment performed on consecutive pairs in the `random-numbers' +stream: + + (define cesaro-stream + (map-successive-pairs (lambda (r1 r2) (= (gcd r1 r2) 1)) + random-numbers)) + + (define (map-successive-pairs f s) + (cons-stream + (f (stream-car s) (stream-car (stream-cdr s))) + (map-successive-pairs f (stream-cdr (stream-cdr s))))) + + The `cesaro-stream' is now fed to a `monte-carlo' procedure, which +produces a stream of estimates of probabilities. The results are then +converted into a stream of estimates of [pi]. This version of the +program doesn't need a parameter telling how many trials to perform. +Better estimates of [pi] (from performing more experiments) are +obtained by looking farther into the `pi' stream: + + (define (monte-carlo experiment-stream passed failed) + (define (next passed failed) + (cons-stream + (/ passed (+ passed failed)) + (monte-carlo + (stream-cdr experiment-stream) passed failed))) + (if (stream-car experiment-stream) + (next (+ passed 1) failed) + (next passed (+ failed 1)))) + + (define pi + (stream-map (lambda (p) (sqrt (/ 6 p))) + (monte-carlo cesaro-stream 0 0))) + + There is considerable modularity in this approach, because we still +can formulate a general `monte-carlo' procedure that can deal with +arbitrary experiments. Yet there is no assignment or local state. + + *Exercise 3.81:* *Note Exercise 3-6:: discussed generalizing the + random-number generator to allow one to reset the random-number + sequence so as to produce repeatable sequences of "random" + numbers. Produce a stream formulation of this same generator that + operates on an input stream of requests to `generate' a new random + number or to `reset' the sequence to a specified value and that + produces the desired stream of random numbers. Don't use + assignment in your solution. + + *Exercise 3.82:* Redo *Note Exercise 3-5:: on Monte Carlo + integration in terms of streams. The stream version of + `estimate-integral' will not have an argument telling how many + trials to perform. Instead, it will produce a stream of estimates + based on successively more trials. + +A functional-programming view of time +..................................... + +Let us now return to the issues of objects and state that were raised +at the beginning of this chapter and examine them in a new light. We +introduced assignment and mutable objects to provide a mechanism for +modular construction of programs that model systems with state. We +constructed computational objects with local state variables and used +assignment to modify these variables. We modeled the temporal behavior +of the objects in the world by the temporal behavior of the +corresponding computational objects. + + Now we have seen that streams provide an alternative way to model +objects with local state. We can model a changing quantity, such as +the local state of some object, using a stream that represents the time +history of successive states. In essence, we represent time +explicitly, using streams, so that we decouple time in our simulated +world from the sequence of events that take place during evaluation. +Indeed, because of the presence of `delay' there may be little relation +between simulated time in the model and the order of events during the +evaluation. + + In order to contrast these two approaches to modeling, let us +reconsider the implementation of a "withdrawal processor" that monitors +the balance in a bank account. In section *Note 3-1-3:: we implemented +a simplified version of such a processor: + + (define (make-simplified-withdraw balance) + (lambda (amount) + (set! balance (- balance amount)) + balance)) + + Calls to `make-simplified-withdraw' produce computational objects, +each with a local state variable `balance' that is decremented by +successive calls to the object. The object takes an `amount' as an +argument and returns the new balance. We can imagine the user of a +bank account typing a sequence of inputs to such an object and +observing the sequence of returned values shown on a display screen. + + Alternatively, we can model a withdrawal processor as a procedure +that takes as input a balance and a stream of amounts to withdraw and +produces the stream of successive balances in the account: + + (define (stream-withdraw balance amount-stream) + (cons-stream + balance + (stream-withdraw (- balance (stream-car amount-stream)) + (stream-cdr amount-stream)))) + + `Stream-withdraw' implements a well-defined mathematical function +whose output is fully determined by its input. Suppose, however, that +the input `amount-stream' is the stream of successive values typed by +the user and that the resulting stream of balances is displayed. Then, +from the perspective of the user who is typing values and watching +results, the stream process has the same behavior as the object created +by `make-simplified-withdraw'. However, with the stream version, there +is no assignment, no local state variable, and consequently none of the +theoretical difficulties that we encountered in section *Note 3-1-3::. +Yet the system has state! + + This is really remarkable. Even though `stream-withdraw' implements +a well-defined mathematical function whose behavior does not change, +the user's perception here is one of interacting with a system that has +a changing state. One way to resolve this paradox is to realize that +it is the user's temporal existence that imposes state on the system. +If the user could step back from the interaction and think in terms of +streams of balances rather than individual transactions, the system +would appear stateless.(1) + + From the point of view of one part of a complex process, the other +parts appear to change with time. They have hidden time-varying local +state. If we wish to write programs that model this kind of natural +decomposition in our world (as we see it from our viewpoint as a part +of that world) with structures in our computer, we make computational +objects that are not functional--they must change with time. We model +state with local state variables, and we model the changes of state +with assignments to those variables. By doing this we make the time of +execution of a computation model time in the world that we are part of, +and thus we get "objects" in our computer. + + Modeling with objects is powerful and intuitive, largely because +this matches the perception of interacting with a world of which we are +part. However, as we've seen repeatedly throughout this chapter, these +models raise thorny problems of constraining the order of events and of +synchronizing multiple processes. The possibility of avoiding these +problems has stimulated the development of "functional programming +languages", which do not include any provision for assignment or +mutable data. In such a language, all procedures implement +well-defined mathematical functions of their arguments, whose behavior +does not change. The functional approach is extremely attractive for +dealing with concurrent systems.(2) + + On the other hand, if we look closely, we can see time-related +problems creeping into functional models as well. One particularly +troublesome area arises when we wish to design interactive systems, +especially ones that model interactions between independent entities. +For instance, consider once more the implementation a banking system +that permits joint bank accounts. In a conventional system using +assignment and objects, we would model the fact that Peter and Paul +share an account by having both Peter and Paul send their transaction +requests to the same bank-account object, as we saw in section *Note +3-1-3::. From the stream point of view, where there are no "objects" +_per se_, we have already indicated that a bank account can be modeled +as a process that operates on a stream of transaction requests to +produce a stream of responses. Accordingly, we could model the fact +that Peter and Paul have a joint bank account by merging Peter's stream +of transaction requests with Paul's stream of requests and feeding the +result to the bank-account stream process, as shown in *Note Figure +3-38::. + + *Figure 3.38:* A joint bank account, modeled by merging two + streams of transaction requests. + + Peter's requests +---------+ +---------+ + ------------------>| | | | + Paul's requests | merge |---->| bank |----> + ------------------>| | | account | + +---------+ +---------+ + + The trouble with this formulation is in the notion of "merge". It +will not do to merge the two streams by simply taking alternately one +request from Peter and one request from Paul. Suppose Paul accesses the +account only very rarely. We could hardly force Peter to wait for Paul +to access the account before he could issue a second transaction. +However such a merge is implemented, it must interleave the two +transaction streams in some way that is constrained by "real time" as +perceived by Peter and Paul, in the sense that, if Peter and Paul meet, +they can agree that certain transactions were processed before the +meeting, and other transactions were processed after the meeting.(3) +This is precisely the same constraint that we had to deal with in +section *Note 3-4-1::, where we found the need to introduce explicit +synchronization to ensure a "correct" order of events in concurrent +processing of objects with state. Thus, in an attempt to support the +functional style, the need to merge inputs from different agents +reintroduces the same problems that the functional style was meant to +eliminate. + + We began this chapter with the goal of building computational models +whose structure matches our perception of the real world we are trying +to model. We can model the world as a collection of separate, +time-bound, interacting objects with state, or we can model the world +as a single, timeless, stateless unity. Each view has powerful +advantages, but neither view alone is completely satisfactory. A grand +unification has yet to emerge.(4) + + ---------- Footnotes ---------- + + (1) Similarly in physics, when we observe a moving particle, we say +that the position (state) of the particle is changing. However, from +the perspective of the particle's world line in space-time there is no +change involved. + + (2) John Backus, the inventor of Fortran, gave high visibility to +functional programming when he was awarded the ACM Turing award in +1978. His acceptance speech (Backus 1978) strongly advocated the +functional approach. A good overview of functional programming is +given in Henderson 1980 and in Darlington, Henderson, and Turner 1982. + + (3) Observe that, for any two streams, there is in general more than +one acceptable order of interleaving. Thus, technically, "merge" is a +relation rather than a function--the answer is not a deterministic +function of the inputs. We already mentioned (*Note Footnote 39::) +that nondeterminism is essential when dealing with concurrency. The +merge relation illustrates the same essential nondeterminism, from the +functional perspective. In section *Note 4-3::, we will look at +nondeterminism from yet another point of view. + + (4) The object model approximates the world by dividing it into +separate pieces. The functional model does not modularize along object +boundaries. The object model is useful when the unshared state of the +"objects" is much larger than the state that they share. An example of +a place where the object viewpoint fails is quantum mechanics, where +thinking of things as individual particles leads to paradoxes and +confusions. Unifying the object view with the functional view may have +little to do with programming, but rather with fundamental +epistemological issues. + + +File: sicp.info, Node: Chapter 4, Next: Chapter 5, Prev: Chapter 3, Up: Top + +4 Metalinguistic Abstraction +**************************** + + ... It's in words that the magic is--Abracadabra, Open Sesame, and + the rest--but the magic words in one story aren't magical in the + next. The real magic is to understand which words work, and when, + and for what; the trick is to learn the trick. + + ... And those words are made from the letters of our alphabet: a + couple-dozen squiggles we can draw with the pen. This is the key! + And the treasure, too, if we can only get our hands on it! It's + as if--as if the key to the treasure _is_ the treasure! + + --John Barth, `Chimera' + + In our study of program design, we have seen that expert programmers +control the complexity of their designs with the same general +techniques used by designers of all complex systems. They combine +primitive elements to form compound objects, they abstract compound +objects to form higher-level building blocks, and they preserve +modularity by adopting appropriate large-scale views of system +structure. In illustrating these techniques, we have used Lisp as a +language for describing processes and for constructing computational +data objects and processes to model complex phenomena in the real +world. However, as we confront increasingly complex problems, we will +find that Lisp, or indeed any fixed programming language, is not +sufficient for our needs. We must constantly turn to new languages in +order to express our ideas more effectively. Establishing new +languages is a powerful strategy for controlling complexity in +engineering design; we can often enhance our ability to deal with a +complex problem by adopting a new language that enables us to describe +(and hence to think about) the problem in a different way, using +primitives, means of combination, and means of abstraction that are +particularly well suited to the problem at hand.(1) + + Programming is endowed with a multitude of languages. There are +physical languages, such as the machine languages for particular +computers. These languages are concerned with the representation of +data and control in terms of individual bits of storage and primitive +machine instructions. The machine-language programmer is concerned +with using the given hardware to erect systems and utilities for the +efficient implementation of resource-limited computations. High-level +languages, erected on a machine-language substrate, hide concerns about +the representation of data as collections of bits and the +representation of programs as sequences of primitive instructions. +These languages have means of combination and abstraction, such as +procedure definition, that are appropriate to the larger-scale +organization of systems. + + "Metalinguistic abstraction"--establishing new languages--plays an +important role in all branches of engineering design. It is +particularly important to computer programming, because in programming +not only can we formulate new languages but we can also implement these +languages by constructing evaluators. An "evaluator" (or "interpreter") +for a programming language is a procedure that, when applied to an +expression of the language, performs the actions required to evaluate +that expression. + + It is no exaggeration to regard this as the most fundamental idea in +programming: + + The evaluator, which determines the meaning of expressions in a + programming language, is just another program. + + To appreciate this point is to change our images of ourselves as +programmers. We come to see ourselves as designers of languages, +rather than only users of languages designed by others. + + In fact, we can regard almost any program as the evaluator for some +language. For instance, the polynomial manipulation system of section +*Note 2-5-3:: embodies the rules of polynomial arithmetic and +implements them in terms of operations on list-structured data. If we +augment this system with procedures to read and print polynomial +expressions, we have the core of a special-purpose language for dealing +with problems in symbolic mathematics. The digital-logic simulator of +section *Note 3-3-4:: and the constraint propagator of section *Note +3-3-5:: are legitimate languages in their own right, each with its own +primitives, means of combination, and means of abstraction. Seen from +this perspective, the technology for coping with large-scale computer +systems merges with the technology for building new computer languages, +and computer science itself becomes no more (and no less) than the +discipline of constructing appropriate descriptive languages. + + We now embark on a tour of the technology by which languages are +established in terms of other languages. In this chapter we shall use +Lisp as a base, implementing evaluators as Lisp procedures. Lisp is +particularly well suited to this task, because of its ability to +represent and manipulate symbolic expressions. We will take the first +step in understanding how languages are implemented by building an +evaluator for Lisp itself. The language implemented by our evaluator +will be a subset of the Scheme dialect of Lisp that we use in this +book. Although the evaluator described in this chapter is written for a +particular dialect of Lisp, it contains the essential structure of an +evaluator for any expression-oriented language designed for writing +programs for a sequential machine. (In fact, most language processors +contain, deep within them, a little "Lisp" evaluator.) The evaluator +has been simplified for the purposes of illustration and discussion, +and some features have been left out that would be important to include +in a production-quality Lisp system. Nevertheless, this simple +evaluator is adequate to execute most of the programs in this book.(2) + + An important advantage of making the evaluator accessible as a Lisp +program is that we can implement alternative evaluation rules by +describing these as modifications to the evaluator program. One place +where we can use this power to good effect is to gain extra control +over the ways in which computational models embody the notion of time, +which was so central to the discussion in *Note Chapter 3::. There, we +mitigated some of the complexities of state and assignment by using +streams to decouple the representation of time in the world from time +in the computer. Our stream programs, however, were sometimes +cumbersome, because they were constrained by the applicative-order +evaluation of Scheme. In section *Note 4-2::, we'll change the +underlying language to provide for a more elegant approach, by +modifying the evaluator to provide for "normal-order evaluation". + + Section *Note 4-3:: implements a more ambitious linguistic change, +whereby expressions have many values, rather than just a single value. +In this language of "nondeterministic computing", it is natural to +express processes that generate all possible values for expressions and +then search for those values that satisfy certain constraints. In +terms of models of computation and time, this is like having time +branch into a set of "possible futures" and then searching for +appropriate time lines. With our nondeterministic evaluator, keeping +track of multiple values and performing searches are handled +automatically by the underlying mechanism of the language. + + In section *Note 4-4:: we implement a "logic-programming" language in +which knowledge is expressed in terms of relations, rather than in +terms of computations with inputs and outputs. Even though this makes +the language drastically different from Lisp, or indeed from any +conventional language, we will see that the logic-programming evaluator +shares the essential structure of the Lisp evaluator. + +* Menu: + +* 4-1:: The Metacircular Evaluator +* 4-2:: Variations on a Scheme -- Lazy Evaluation +* 4-3:: Variations on a Scheme -- Nondeterministic Computing +* 4-4:: Logic Programming + + ---------- Footnotes ---------- + + (1) The same idea is pervasive throughout all of engineering. For +example, electrical engineers use many different languages for +describing circuits. Two of these are the language of electrical "networks" +and the language of electrical "systems". The network language +emphasizes the physical modeling of devices in terms of discrete +electrical elements. The primitive objects of the network language are +primitive electrical components such as resistors, capacitors, +inductors, and transistors, which are characterized in terms of +physical variables called voltage and current. When describing +circuits in the network language, the engineer is concerned with the +physical characteristics of a design. In contrast, the primitive +objects of the system language are signal-processing modules such as +filters and amplifiers. Only the functional behavior of the modules is +relevant, and signals are manipulated without concern for their +physical realization as voltages and currents. The system language is +erected on the network language, in the sense that the elements of +signal-processing systems are constructed from electrical networks. +Here, however, the concerns are with the large-scale organization of +electrical devices to solve a given application problem; the physical +feasibility of the parts is assumed. This layered collection of +languages is another example of the stratified design technique +illustrated by the picture language of section *Note 2-2-4::. + + (2) The most important features that our evaluator leaves out are +mechanisms for handling errors and supporting debugging. For a more +extensive discussion of evaluators, see Friedman, Wand, and Haynes +1992, which gives an exposition of programming languages that proceeds +via a sequence of evaluators written in Scheme. + + +File: sicp.info, Node: 4-1, Next: 4-2, Prev: Chapter 4, Up: Chapter 4 + +4.1 The Metacircular Evaluator +============================== + +Our evaluator for Lisp will be implemented as a Lisp program. It may +seem circular to think about evaluating Lisp programs using an +evaluator that is itself implemented in Lisp. However, evaluation is a +process, so it is appropriate to describe the evaluation process using +Lisp, which, after all, is our tool for describing processes.(1) An +evaluator that is written in the same language that it evaluates is +said to be "metacircular". + + The metacircular evaluator is essentially a Scheme formulation of the +environment model of evaluation described in section *Note 3-2::. +Recall that the model has two basic parts: + + 1. To evaluate a combination (a compound expression other than a + special form), evaluate the subexpressions and then apply the + value of the operator subexpression to the values of the operand + subexpressions. + + 2. To apply a compound procedure to a set of arguments, evaluate the + body of the procedure in a new environment. To construct this + environment, extend the environment part of the procedure object + by a frame in which the formal parameters of the procedure are + bound to the arguments to which the procedure is applied. + + + These two rules describe the essence of the evaluation process, a +basic cycle in which expressions to be evaluated in environments are +reduced to procedures to be applied to arguments, which in turn are +reduced to new expressions to be evaluated in new environments, and so +on, until we get down to symbols, whose values are looked up in the +environment, and to primitive procedures, which are applied directly +(see *Note Figure 4-1::).(2) This evaluation cycle will be embodied by +the interplay between the two critical procedures in the evaluator, +`eval' and `apply', which are described in section *Note 4-1-1:: (see +*Note Figure 4-1::). + + The implementation of the evaluator will depend upon procedures that +define the "syntax" of the expressions to be evaluated. We will use +data abstraction to make the evaluator independent of the +representation of the language. For example, rather than committing to +a choice that an assignment is to be represented by a list beginning +with the symbol `set!' we use an abstract predicate `assignment?' to +test for an assignment, and we use abstract selectors +`assignment-variable' and `assignment-value' to access the parts of an +assignment. Implementation of expressions will be described in detail +in section *Note 4-1-2::. There are also operations, described in +section *Note 4-1-3::, that specify the representation of procedures +and environments. For example, `make-procedure' constructs compound +procedures, `lookup-variable-value' accesses the values of variables, +and `apply-primitive-procedure' applies a primitive procedure to a +given list of arguments. + +* Menu: + +* 4-1-1:: The Core of the Evaluator +* 4-1-2:: Representing Expressions +* 4-1-3:: Evaluator Data Structures +* 4-1-4:: Running the Evaluator as a Program +* 4-1-5:: Data as Programs +* 4-1-6:: Internal Definitions +* 4-1-7:: Separating Syntactic Analysis from Execution + + ---------- Footnotes ---------- + + (1) Even so, there will remain important aspects of the evaluation +process that are not elucidated by our evaluator. The most important +of these are the detailed mechanisms by which procedures call other +procedures and return values to their callers. We will address these +issues in *Note Chapter 5::, where we take a closer look at the +evaluation process by implementing the evaluator as a simple register +machine. + + (2) If we grant ourselves the ability to apply primitives, then what +remains for us to implement in the evaluator? The job of the evaluator +is not to specify the primitives of the language, but rather to provide +the connective tissue--the means of combination and the means of +abstraction--that binds a collection of primitives to form a language. +Specifically: + + * The evaluator enables us to deal with nested expressions. For + example, although simply applying primitives would suffice for + evaluating the expression `(+ 1 6)', it is not adequate for + handling `(+ 1 (* 2 3))'. As far as the primitive procedure `+' + is concerned, its arguments must be numbers, and it would choke if + we passed it the expression `(* 2 3)' as an argument. One + important role of the evaluator is to choreograph procedure + composition so that `(* 2 3)' is reduced to 6 before being passed + as an argument to `+'. + + * The evaluator allows us to use variables. For example, the + primitive procedure for addition has no way to deal with + expressions such as `(+ x 1)'. We need an evaluator to keep track + of variables and obtain their values before invoking the primitive + procedures. + + * The evaluator allows us to define compound procedures. This + involves keeping track of procedure definitions, knowing how to + use these definitions in evaluating expressions, and providing a + mechanism that enables procedures to accept arguments. + + * The evaluator provides the special forms, which must be evaluated + differently from procedure calls. + + + +File: sicp.info, Node: 4-1-1, Next: 4-1-2, Prev: 4-1, Up: 4-1 + +4.1.1 The Core of the Evaluator +------------------------------- + + *Figure 4.1:* The `eval'-`apply' cycle exposes the essence of a + computer language. + + .,ad88888888baa, + _ ,d8P""" ""9888ba. _ + / .a8" ,ad88888888888a |\ + / aP' ,88888888888888888a \ + / ,8" ,88888888888888888888, \ + | ,8' (888888888888888888888, | + / ,8' `8888888888888888888888 \ + | 8) `888888888888888888888, | + Procedure, | 8 "88888 Apply 8888888) | Expression + Arguments | 8 Eval `888888888888888888) | Environment + | 8) "8888888888888888 | + \ (b "88888888888888' / + | `8, 8888888888888) | + \ "8a ,888888888888) / + \ V8, d88888888888" / + _\| `8b, ,d8888888888P' _/ + `V8a, ,ad8888888888P' + ""88888888888888888P" + """""""""""" + + [graphic by Normand Veillux, modified] + +The evaluation process can be described as the interplay between two +procedures: `eval' and `apply'. + +Eval +.... + +`Eval' takes as arguments an expression and an environment. It +classifies the expression and directs its evaluation. `Eval' is +structured as a case analysis of the syntactic type of the expression +to be evaluated. In order to keep the procedure general, we express +the determination of the type of an expression abstractly, making no +commitment to any particular representation for the various types of +expressions. Each type of expression has a predicate that tests for it +and an abstract means for selecting its parts. This "abstract syntax" +makes it easy to see how we can change the syntax of the language by +using the same evaluator, but with a different collection of syntax +procedures. + +Primitive expressions + + * For self-evaluating expressions, such as numbers, `eval' returns + the expression itself. + + * `Eval' must look up variables in the environment to find their + values. + + +Special forms + + * For quoted expressions, `eval' returns the expression that was + quoted. + + * An assignment to (or a definition of) a variable must recursively + call `eval' to compute the new value to be associated with the + variable. The environment must be modified to change (or create) + the binding of the variable. + + * An `if' expression requires special processing of its parts, so as + to evaluate the consequent if the predicate is true, and otherwise + to evaluate the alternative. + + * A `lambda' expression must be transformed into an applicable + procedure by packaging together the parameters and body specified + by the `lambda' expression with the environment of the evaluation. + + * A `begin' expression requires evaluating its sequence of + expressions in the order in which they appear. + + * A case analysis (`cond') is transformed into a nest of `if' + expressions and then evaluated. + + +Combinations + + * For a procedure application, `eval' must recursively evaluate the + operator part and the operands of the combination. The resulting + procedure and arguments are passed to `apply', which handles the + actual procedure application. + + + Here is the definition of `eval': + + (define (eval exp env) + (cond ((self-evaluating? exp) exp) + ((variable? exp) (lookup-variable-value exp env)) + ((quoted? exp) (text-of-quotation exp)) + ((assignment? exp) (eval-assignment exp env)) + ((definition? exp) (eval-definition exp env)) + ((if? exp) (eval-if exp env)) + ((lambda? exp) + (make-procedure (lambda-parameters exp) + (lambda-body exp) + env)) + ((begin? exp) + (eval-sequence (begin-actions exp) env)) + ((cond? exp) (eval (cond->if exp) env)) + ((application? exp) + (apply (eval (operator exp) env) + (list-of-values (operands exp) env))) + (else + (error "Unknown expression type -- EVAL" exp)))) + + For clarity, `eval' has been implemented as a case analysis using +`cond'. The disadvantage of this is that our procedure handles only a +few distinguishable types of expressions, and no new ones can be +defined without editing the definition of `eval'. In most Lisp +implementations, dispatching on the type of an expression is done in a +data-directed style. This allows a user to add new types of +expressions that `eval' can distinguish, without modifying the +definition of `eval' itself. (See *Note Exercise 4-3::.) + +Apply +..... + +`Apply' takes two arguments, a procedure and a list of arguments to +which the procedure should be applied. `Apply' classifies procedures +into two kinds: It calls `apply-primitive-procedure' to apply +primitives; it applies compound procedures by sequentially evaluating +the expressions that make up the body of the procedure. The +environment for the evaluation of the body of a compound procedure is +constructed by extending the base environment carried by the procedure +to include a frame that binds the parameters of the procedure to the +arguments to which the procedure is to be applied. Here is the +definition of `apply': + + (define (apply procedure arguments) + (cond ((primitive-procedure? procedure) + (apply-primitive-procedure procedure arguments)) + ((compound-procedure? procedure) + (eval-sequence + (procedure-body procedure) + (extend-environment + (procedure-parameters procedure) + arguments + (procedure-environment procedure)))) + (else + (error + "Unknown procedure type -- APPLY" procedure)))) + +Procedure arguments +................... + +When `eval' processes a procedure application, it uses `list-of-values' +to produce the list of arguments to which the procedure is to be +applied. `List-of-values' takes as an argument the operands of the +combination. It evaluates each operand and returns a list of the +corresponding values:(1) + + (define (list-of-values exps env) + (if (no-operands? exps) + '() + (cons (eval (first-operand exps) env) + (list-of-values (rest-operands exps) env)))) + +Conditionals +............ + +`Eval-if' evaluates the predicate part of an `if' expression in the +given environment. If the result is true, `eval-if' evaluates the +consequent, otherwise it evaluates the alternative: + + (define (eval-if exp env) + (if (true? (eval (if-predicate exp) env)) + (eval (if-consequent exp) env) + (eval (if-alternative exp) env))) + + The use of `true?' in `eval-if' highlights the issue of the +connection between an implemented language and an implementation +language. The `if-predicate' is evaluated in the language being +implemented and thus yields a value in that language. The interpreter +predicate `true?' translates that value into a value that can be tested +by the `if' in the implementation language: The metacircular +representation of truth might not be the same as that of the underlying +Scheme.(2) + +Sequences +......... + +`Eval-sequence' is used by `apply' to evaluate the sequence of +expressions in a procedure body and by `eval' to evaluate the sequence +of expressions in a `begin' expression. It takes as arguments a +sequence of expressions and an environment, and evaluates the +expressions in the order in which they occur. The value returned is +the value of the final expression. + + (define (eval-sequence exps env) + (cond ((last-exp? exps) (eval (first-exp exps) env)) + (else (eval (first-exp exps) env) + (eval-sequence (rest-exps exps) env)))) + +Assignments and definitions +........................... + +The following procedure handles assignments to variables. It calls +`eval' to find the value to be assigned and transmits the variable and +the resulting value to `set-variable-value!' to be installed in the +designated environment. + + (define (eval-assignment exp env) + (set-variable-value! (assignment-variable exp) + (eval (assignment-value exp) env) + env) + 'ok) + +Definitions of variables are handled in a similar manner.(3) + + (define (eval-definition exp env) + (define-variable! (definition-variable exp) + (eval (definition-value exp) env) + env) + 'ok) + + We have chosen here to return the symbol `ok' as the value of an +assignment or a definition.(4) + + *Exercise 4.1:* Notice that we cannot tell whether the + metacircular evaluator evaluates operands from left to right or + from right to left. Its evaluation order is inherited from the + underlying Lisp: If the arguments to `cons' in `list-of-values' + are evaluated from left to right, then `list-of-values' will + evaluate operands from left to right; and if the arguments to + `cons' are evaluated from right to left, then `list-of-values' + will evaluate operands from right to left. + + Write a version of `list-of-values' that evaluates operands from + left to right regardless of the order of evaluation in the + underlying Lisp. Also write a version of `list-of-values' that + evaluates operands from right to left. + + ---------- Footnotes ---------- + + (1) We could have simplified the `application?' clause in `eval' by +using `map' (and stipulating that `operands' returns a list) rather +than writing an explicit `list-of-values' procedure. We chose not to +use `map' here to emphasize the fact that the evaluator can be +implemented without any use of higher-order procedures (and thus could +be written in a language that doesn't have higher-order procedures), +even though the language that it supports will include higher-order +procedures. + + (2) In this case, the language being implemented and the +implementation language are the same. Contemplation of the meaning of +`true?' here yields expansion of consciousness without the abuse of +substance. + + (3) This implementation of `define' ignores a subtle issue in the +handling of internal definitions, although it works correctly in most +cases. We will see what the problem is and how to solve it in section +*Note 4-1-6::. + + (4) As we said when we introduced `define' and `set!', these values +are implementation-dependent in Scheme--that is, the implementor can +choose what value to return. + + +File: sicp.info, Node: 4-1-2, Next: 4-1-3, Prev: 4-1-1, Up: 4-1 + +4.1.2 Representing Expressions +------------------------------ + +The evaluator is reminiscent of the symbolic differentiation program +discussed in section *Note 2-3-2::. Both programs operate on symbolic +expressions. In both programs, the result of operating on a compound +expression is determined by operating recursively on the pieces of the +expression and combining the results in a way that depends on the type +of the expression. In both programs we used data abstraction to +decouple the general rules of operation from the details of how +expressions are represented. In the differentiation program this meant +that the same differentiation procedure could deal with algebraic +expressions in prefix form, in infix form, or in some other form. For +the evaluator, this means that the syntax of the language being +evaluated is determined solely by the procedures that classify and +extract pieces of expressions. + + Here is the specification of the syntax of our language: + + * The only self-evaluating items are numbers and strings: + + (define (self-evaluating? exp) + (cond ((number? exp) true) + ((string? exp) true) + (else false))) + + * Variables are represented by symbols: + + (define (variable? exp) (symbol? exp)) + + * Quotations have the form `(quote )':(1) + + (define (quoted? exp) + (tagged-list? exp 'quote)) + + (define (text-of-quotation exp) (cadr exp)) + + `Quoted?' is defined in terms of the procedure `tagged-list?', + which identifies lists beginning with a designated symbol: + + (define (tagged-list? exp tag) + (if (pair? exp) + (eq? (car exp) tag) + false)) + + * Assignments have the form `(set! )': + + (define (assignment? exp) + (tagged-list? exp 'set!)) + + (define (assignment-variable exp) (cadr exp)) + + (define (assignment-value exp) (caddr exp)) + + * Definitions have the form + + (define ) + + or the form + + (define ( ... ) + ) + + The latter form (standard procedure definition) is syntactic sugar + for + + (define + (lambda ( ... ) + )) + + The corresponding syntax procedures are the following: + + (define (definition? exp) + (tagged-list? exp 'define)) + + (define (definition-variable exp) + (if (symbol? (cadr exp)) + (cadr exp) + (caadr exp))) + + (define (definition-value exp) + (if (symbol? (cadr exp)) + (caddr exp) + (make-lambda (cdadr exp) ; formal parameters + (cddr exp)))) ; body + + * `Lambda' expressions are lists that begin with the symbol `lambda': + + (define (lambda? exp) (tagged-list? exp 'lambda)) + + (define (lambda-parameters exp) (cadr exp)) + + (define (lambda-body exp) (cddr exp)) + + We also provide a constructor for `lambda' expressions, which is + used by `definition-value', above: + + (define (make-lambda parameters body) + (cons 'lambda (cons parameters body))) + + * Conditionals begin with `if' and have a predicate, a consequent, + and an (optional) alternative. If the expression has no + alternative part, we provide `false' as the alternative.(2) + + (define (if? exp) (tagged-list? exp 'if)) + + (define (if-predicate exp) (cadr exp)) + + (define (if-consequent exp) (caddr exp)) + + (define (if-alternative exp) + (if (not (null? (cdddr exp))) + (cadddr exp) + 'false)) + + We also provide a constructor for `if' expressions, to be used by + `cond->if' to transform `cond' expressions into `if' expressions: + + (define (make-if predicate consequent alternative) + (list 'if predicate consequent alternative)) + + * `Begin' packages a sequence of expressions into a single + expression. We include syntax operations on `begin' expressions + to extract the actual sequence from the `begin' expression, as + well as selectors that return the first expression and the rest of + the expressions in the sequence.(3) + + (define (begin? exp) (tagged-list? exp 'begin)) + + (define (begin-actions exp) (cdr exp)) + + (define (last-exp? seq) (null? (cdr seq))) + + (define (first-exp seq) (car seq)) + + (define (rest-exps seq) (cdr seq)) + + We also include a constructor `sequence->exp' (for use by + `cond->if') that transforms a sequence into a single expression, + using `begin' if necessary: + + (define (sequence->exp seq) + (cond ((null? seq) seq) + ((last-exp? seq) (first-exp seq)) + (else (make-begin seq)))) + + (define (make-begin seq) (cons 'begin seq)) + + * A procedure application is any compound expression that is not one + of the above expression types. The `car' of the expression is the + operator, and the `cdr' is the list of operands: + + (define (application? exp) (pair? exp)) + + (define (operator exp) (car exp)) + + (define (operands exp) (cdr exp)) + + (define (no-operands? ops) (null? ops)) + + (define (first-operand ops) (car ops)) + + (define (rest-operands ops) (cdr ops)) + + +Derived expressions +................... + +Some special forms in our language can be defined in terms of +expressions involving other special forms, rather than being +implemented directly. One example is `cond', which can be implemented +as a nest of `if' expressions. For example, we can reduce the problem +of evaluating the expression + + (cond ((> x 0) x) + ((= x 0) (display 'zero) 0) + (else (- x))) + +to the problem of evaluating the following expression involving `if' and +`begin' expressions: + + (if (> x 0) + x + (if (= x 0) + (begin (display 'zero) + 0) + (- x))) + + Implementing the evaluation of `cond' in this way simplifies the +evaluator because it reduces the number of special forms for which the +evaluation process must be explicitly specified. + + We include syntax procedures that extract the parts of a `cond' +expression, and a procedure `cond->if' that transforms `cond' +expressions into `if' expressions. A case analysis begins with `cond' +and has a list of predicate-action clauses. A clause is an `else' +clause if its predicate is the symbol `else'.(4) + + (define (cond? exp) (tagged-list? exp 'cond)) + + (define (cond-clauses exp) (cdr exp)) + + (define (cond-else-clause? clause) + (eq? (cond-predicate clause) 'else)) + + (define (cond-predicate clause) (car clause)) + + (define (cond-actions clause) (cdr clause)) + + (define (cond->if exp) + (expand-clauses (cond-clauses exp))) + + (define (expand-clauses clauses) + (if (null? clauses) + 'false ; no `else' clause + (let ((first (car clauses)) + (rest (cdr clauses))) + (if (cond-else-clause? first) + (if (null? rest) + (sequence->exp (cond-actions first)) + (error "ELSE clause isn't last -- COND->IF" + clauses)) + (make-if (cond-predicate first) + (sequence->exp (cond-actions first)) + (expand-clauses rest)))))) + + Expressions (such as `cond') that we choose to implement as syntactic +transformations are called "derived expressions". `Let' expressions +are also derived expressions (see *Note Exercise 4-6::).(5) + + *Exercise 4.2:* Louis Reasoner plans to reorder the `cond' clauses + in `eval' so that the clause for procedure applications appears + before the clause for assignments. He argues that this will make + the interpreter more efficient: Since programs usually contain more + applications than assignments, definitions, and so on, his + modified `eval' will usually check fewer clauses than the original + `eval' before identifying the type of an expression. + + a. What is wrong with Louis's plan? (Hint: What will Louis's + evaluator do with the expression `(define x 3)'?) + + b. Louis is upset that his plan didn't work. He is willing to + go to any lengths to make his evaluator recognize procedure + applications before it checks for most other kinds of + expressions. Help him by changing the syntax of the + evaluated language so that procedure applications start with + `call'. For example, instead of `(factorial 3)' we will now + have to write `(call factorial 3)' and instead of `(+ 1 2)' + we will have to write `(call + 1 2)'. + + + *Exercise 4.3:* Rewrite `eval' so that the dispatch is done in + data-directed style. Compare this with the data-directed + differentiation procedure of *Note Exercise 2-73::. (You may use + the `car' of a compound expression as the type of the expression, + as is appropriate for the syntax implemented in this section.) + + *Exercise 4.4:* Recall the definitions of the special forms `and' + and `or' from *Note Chapter 1::: + + * `and': The expressions are evaluated from left to right. If + any expression evaluates to false, false is returned; any + remaining expressions are not evaluated. If all the + expressions evaluate to true values, the value of the last + expression is returned. If there are no expressions then + true is returned. + + * `or': The expressions are evaluated from left to right. If + any expression evaluates to a true value, that value is + returned; any remaining expressions are not evaluated. If + all expressions evaluate to false, or if there are no + expressions, then false is returned. + + + Install `and' and `or' as new special forms for the evaluator by + defining appropriate syntax procedures and evaluation procedures + `eval-and' and `eval-or'. Alternatively, show how to implement + `and' and `or' as derived expressions. + + *Exercise 4.5:* Scheme allows an additional syntax for `cond' + clauses, `( => )'. If evaluates to a true + value, then is evaluated. Its value must be a + procedure of one argument; this procedure is then invoked on the + value of the , and the result is returned as the value of + the `cond' expression. For example + + (cond ((assoc 'b '((a 1) (b 2))) => cadr) + (else false)) + + returns 2. Modify the handling of `cond' so that it supports this + extended syntax. + + *Exercise 4.6:* `Let' expressions are derived expressions, because + + (let (( ) ... ( )) + ) + + is equivalent to + + ((lambda ( ... ) + ) + + ... + ) + + Implement a syntactic transformation `let->combination' that + reduces evaluating `let' expressions to evaluating combinations of + the type shown above, and add the appropriate clause to `eval' to + handle `let' expressions. + + *Exercise 4.7:* `Let*' is similar to `let', except that the + bindings of the `let' variables are performed sequentially from + left to right, and each binding is made in an environment in which + all of the preceding bindings are visible. For example + + (let* ((x 3) + (y (+ x 2)) + (z (+ x y 5))) + (* x z)) + + returns 39. Explain how a `let*' expression can be rewritten as a + set of nested `let' expressions, and write a procedure + `let*->nested-lets' that performs this transformation. If we have + already implemented `let' (*Note Exercise 4-6::) and we want to + extend the evaluator to handle `let*', is it sufficient to add a + clause to `eval' whose action is + + (eval (let*->nested-lets exp) env) + + or must we explicitly expand `let*' in terms of non-derived + expressions? + + *Exercise 4.8:* "Named `let'" is a variant of `let' that has the + form + + (let ) + + The and are just as in ordinary `let', except + that is bound within to a procedure whose body is + and whose parameters are the variables in the . + Thus, one can repeatedly execute the by invoking the + procedure named . For example, the iterative Fibonacci + procedure (section *Note 1-2-2::) can be rewritten using named + `let' as follows: + + (define (fib n) + (let fib-iter ((a 1) + (b 0) + (count n)) + (if (= count 0) + b + (fib-iter (+ a b) a (- count 1))))) + + Modify `let->combination' of *Note Exercise 4-6:: to also support + named `let'. + + *Exercise 4.9:* Many languages support a variety of iteration + constructs, such as `do', `for', `while', and `until'. In Scheme, + iterative processes can be expressed in terms of ordinary + procedure calls, so special iteration constructs provide no + essential gain in computational power. On the other hand, such + constructs are often convenient. Design some iteration + constructs, give examples of their use, and show how to implement + them as derived expressions. + + *Exercise 4.10:* By using data abstraction, we were able to write + an `eval' procedure that is independent of the particular syntax + of the language to be evaluated. To illustrate this, design and + implement a new syntax for Scheme by modifying the procedures in + this section, without changing `eval' or `apply'. + + ---------- Footnotes ---------- + + (1) As mentioned in section *Note 2-3-1::, the evaluator sees a +quoted expression as a list beginning with `quote', even if the +expression is typed with the quotation mark. For example, the +expression `'a' would be seen by the evaluator as `(quote a)'. See +*Note Exercise 2-55::. + + (2) The value of an `if' expression when the predicate is false and +there is no alternative is unspecified in Scheme; we have chosen here +to make it false. We will support the use of the variables `true' and +`false' in expressions to be evaluated by binding them in the global +environment. See section *Note 4-1-4::. + + (3) These selectors for a list of expressions--and the corresponding +ones for a list of operands--are not intended as a data abstraction. +They are introduced as mnemonic names for the basic list operations in +order to make it easier to understand the explicit-control evaluator in +section *Note 5-4::. + + (4) The value of a `cond' expression when all the predicates are +false and there is no `else' clause is unspecified in Scheme; we have +chosen here to make it false. + + (5) Practical Lisp systems provide a mechanism that allows a user to +add new derived expressions and specify their implementation as +syntactic transformations without modifying the evaluator. Such a +user-defined transformation is called a "macro". Although it is easy +to add an elementary mechanism for defining macros, the resulting +language has subtle name-conflict problems. There has been much +research on mechanisms for macro definition that do not cause these +difficulties. See, for example, Kohlbecker 1986, Clinger and Rees +1991, and Hanson 1991. + + +File: sicp.info, Node: 4-1-3, Next: 4-1-4, Prev: 4-1-2, Up: 4-1 + +4.1.3 Evaluator Data Structures +------------------------------- + +In addition to defining the external syntax of expressions, the +evaluator implementation must also define the data structures that the +evaluator manipulates internally, as part of the execution of a +program, such as the representation of procedures and environments and +the representation of true and false. + +Testing of predicates +..................... + +For conditionals, we accept anything to be true that is not the explicit +`false' object. + + (define (true? x) + (not (eq? x false))) + + (define (false? x) + (eq? x false)) + +Representing procedures +....................... + +To handle primitives, we assume that we have available the following +procedures: + + * `(apply-primitive-procedure )' + + applies the given primitive procedure to the argument values in + the list and returns the result of the application. + + * `(primitive-procedure? )' + + tests whether is a primitive procedure. + + + These mechanisms for handling primitives are further described in +section *Note 4-1-4::. + + Compound procedures are constructed from parameters, procedure +bodies, and environments using the constructor `make-procedure': + + (define (make-procedure parameters body env) + (list 'procedure parameters body env)) + + (define (compound-procedure? p) + (tagged-list? p 'procedure)) + + (define (procedure-parameters p) (cadr p)) + + (define (procedure-body p) (caddr p)) + + (define (procedure-environment p) (cadddr p)) + +Operations on Environments +.......................... + +The evaluator needs operations for manipulating environments. As +explained in section *Note 3-2::, an environment is a sequence of +frames, where each frame is a table of bindings that associate +variables with their corresponding values. We use the following +operations for manipulating environments: + + * `(lookup-variable-value )' + + returns the value that is bound to the symbol in the + environment , or signals an error if the variable is unbound. + + * `(extend-environment )' + + returns a new environment, consisting of a new frame in which the + symbols in the list are bound to the corresponding + elements in the list , where the enclosing environment is + the environment . + + * `(define-variable! )' + + adds to the first frame in the environment a new binding that + associates the variable with the value . + + * `(set-variable-value! )' + + changes the binding of the variable in the environment + so that the variable is now bound to the value , or signals + an error if the variable is unbound. + + + To implement these operations we represent an environment as a list +of frames. The enclosing environment of an environment is the `cdr' of +the list. The empty environment is simply the empty list. + + (define (enclosing-environment env) (cdr env)) + + (define (first-frame env) (car env)) + + (define the-empty-environment '()) + + Each frame of an environment is represented as a pair of lists: a +list of the variables bound in that frame and a list of the associated +values.(1) + + (define (make-frame variables values) + (cons variables values)) + + (define (frame-variables frame) (car frame)) + + (define (frame-values frame) (cdr frame)) + + (define (add-binding-to-frame! var val frame) + (set-car! frame (cons var (car frame))) + (set-cdr! frame (cons val (cdr frame)))) + + To extend an environment by a new frame that associates variables +with values, we make a frame consisting of the list of variables and +the list of values, and we adjoin this to the environment. We signal +an error if the number of variables does not match the number of values. + + (define (extend-environment vars vals base-env) + (if (= (length vars) (length vals)) + (cons (make-frame vars vals) base-env) + (if (< (length vars) (length vals)) + (error "Too many arguments supplied" vars vals) + (error "Too few arguments supplied" vars vals)))) + + To look up a variable in an environment, we scan the list of +variables in the first frame. If we find the desired variable, we +return the corresponding element in the list of values. If we do not +find the variable in the current frame, we search the enclosing +environment, and so on. If we reach the empty environment, we signal +an "unbound variable" error. + + (define (lookup-variable-value var env) + (define (env-loop env) + (define (scan vars vals) + (cond ((null? vars) + (env-loop (enclosing-environment env))) + ((eq? var (car vars)) + (car vals)) + (else (scan (cdr vars) (cdr vals))))) + (if (eq? env the-empty-environment) + (error "Unbound variable" var) + (let ((frame (first-frame env))) + (scan (frame-variables frame) + (frame-values frame))))) + (env-loop env)) + + To set a variable to a new value in a specified environment, we scan +for the variable, just as in `lookup-variable-value', and change the +corresponding value when we find it. + + (define (set-variable-value! var val env) + (define (env-loop env) + (define (scan vars vals) + (cond ((null? vars) + (env-loop (enclosing-environment env))) + ((eq? var (car vars)) + (set-car! vals val)) + (else (scan (cdr vars) (cdr vals))))) + (if (eq? env the-empty-environment) + (error "Unbound variable -- SET!" var) + (let ((frame (first-frame env))) + (scan (frame-variables frame) + (frame-values frame))))) + (env-loop env)) + + To define a variable, we search the first frame for a binding for +the variable, and change the binding if it exists (just as in +`set-variable-value!'). If no such binding exists, we adjoin one to +the first frame. + + (define (define-variable! var val env) + (let ((frame (first-frame env))) + (define (scan vars vals) + (cond ((null? vars) + (add-binding-to-frame! var val frame)) + ((eq? var (car vars)) + (set-car! vals val)) + (else (scan (cdr vars) (cdr vals))))) + (scan (frame-variables frame) + (frame-values frame)))) + + The method described here is only one of many plausible ways to +represent environments. Since we used data abstraction to isolate the +rest of the evaluator from the detailed choice of representation, we +could change the environment representation if we wanted to. (See +*Note Exercise 4-11::.) In a production-quality Lisp system, the speed +of the evaluator's environment operations--especially that of variable +lookup--has a major impact on the performance of the system. The +representation described here, although conceptually simple, is not +efficient and would not ordinarily be used in a production system.(2) + + *Exercise 4.11:* Instead of representing a frame as a pair of + lists, we can represent a frame as a list of bindings, where each + binding is a name-value pair. Rewrite the environment operations + to use this alternative representation. + + *Exercise 4.12:* The procedures `set-variable-value!', + `define-variable!', and `lookup-variable-value' can be expressed + in terms of more abstract procedures for traversing the + environment structure. Define abstractions that capture the + common patterns and redefine the three procedures in terms of these + abstractions. + + *Exercise 4.13:* Scheme allows us to create new bindings for + variables by means of `define', but provides no way to get rid of + bindings. Implement for the evaluator a special form + `make-unbound!' that removes the binding of a given symbol from the + environment in which the `make-unbound!' expression is evaluated. + This problem is not completely specified. For example, should we + remove only the binding in the first frame of the environment? + Complete the specification and justify any choices you make. + + ---------- Footnotes ---------- + + (1) Frames are not really a data abstraction in the following code: +`Set-variable-value!' and `define-variable!' use `set-car!' to +directly modify the values in a frame. The purpose of the frame +procedures is to make the environment-manipulation procedures easy to +read. + + (2) The drawback of this representation (as well as the variant in +*Note Exercise 4-11::) is that the evaluator may have to search through +many frames in order to find the binding for a given variable. (Such +an approach is referred to as "deep binding".) One way to avoid this +inefficiency is to make use of a strategy called "lexical addressing", +which will be discussed in section *Note 5-5-6::. + + +File: sicp.info, Node: 4-1-4, Next: 4-1-5, Prev: 4-1-3, Up: 4-1 + +4.1.4 Running the Evaluator as a Program +---------------------------------------- + +Given the evaluator, we have in our hands a description (expressed in +Lisp) of the process by which Lisp expressions are evaluated. One +advantage of expressing the evaluator as a program is that we can run +the program. This gives us, running within Lisp, a working model of +how Lisp itself evaluates expressions. This can serve as a framework +for experimenting with evaluation rules, as we shall do later in this +chapter. + + Our evaluator program reduces expressions ultimately to the +application of primitive procedures. Therefore, all that we need to +run the evaluator is to create a mechanism that calls on the underlying +Lisp system to model the application of primitive procedures. + + There must be a binding for each primitive procedure name, so that +when `eval' evaluates the operator of an application of a primitive, it +will find an object to pass to `apply'. We thus set up a global +environment that associates unique objects with the names of the +primitive procedures that can appear in the expressions we will be +evaluating. The global environment also includes bindings for the +symbols `true' and `false', so that they can be used as variables in +expressions to be evaluated. + + (define (setup-environment) + (let ((initial-env + (extend-environment (primitive-procedure-names) + (primitive-procedure-objects) + the-empty-environment))) + (define-variable! 'true true initial-env) + (define-variable! 'false false initial-env) + initial-env)) + + (define the-global-environment (setup-environment)) + + It does not matter how we represent the primitive procedure objects, +so long as `apply' can identify and apply them by using the procedures +`primitive-procedure?' and `apply-primitive-procedure'. We have chosen +to represent a primitive procedure as a list beginning with the symbol +`primitive' and containing a procedure in the underlying Lisp that +implements that primitive. + + (define (primitive-procedure? proc) + (tagged-list? proc 'primitive)) + + (define (primitive-implementation proc) (cadr proc)) + + `Setup-environment' will get the primitive names and implementation +procedures from a list:(1) + + (define primitive-procedures + (list (list 'car car) + (list 'cdr cdr) + (list 'cons cons) + (list 'null? null?) + + )) + + (define (primitive-procedure-names) + (map car + primitive-procedures)) + + (define (primitive-procedure-objects) + (map (lambda (proc) (list 'primitive (cadr proc))) + primitive-procedures)) + + To apply a primitive procedure, we simply apply the implementation +procedure to the arguments, using the underlying Lisp system:(2) + + (define (apply-primitive-procedure proc args) + (apply-in-underlying-scheme + (primitive-implementation proc) args)) + + For convenience in running the metacircular evaluator, we provide a "driver +loop" that models the read-eval-print loop of the underlying Lisp +system. It prints a "prompt", reads an input expression, evaluates +this expression in the global environment, and prints the result. We +precede each printed result by an "output prompt" so as to distinguish +the value of the expression from other output that may be printed.(3) + + (define input-prompt ";;; M-Eval input:") + (define output-prompt ";;; M-Eval value:") + + (define (driver-loop) + (prompt-for-input input-prompt) + (let ((input (read))) + (let ((output (eval input the-global-environment))) + (announce-output output-prompt) + (user-print output))) + (driver-loop)) + + (define (prompt-for-input string) + (newline) (newline) (display string) (newline)) + + (define (announce-output string) + (newline) (display string) (newline)) + + We use a special printing procedure, `user-print', to avoid printing +the environment part of a compound procedure, which may be a very long +list (or may even contain cycles). + + (define (user-print object) + (if (compound-procedure? object) + (display (list 'compound-procedure + (procedure-parameters object) + (procedure-body object) + ')) + (display object))) + + Now all we need to do to run the evaluator is to initialize the +global environment and start the driver loop. Here is a sample +interaction: + + (define the-global-environment (setup-environment)) + + (driver-loop) + + ;;; M-Eval input: + (define (append x y) + (if (null? x) + y + (cons (car x) + (append (cdr x) y)))) + ;;; M-Eval value: + ok + + ;;; M-Eval input: + (append '(a b c) '(d e f)) + ;;; M-Eval value: + (a b c d e f) + + *Exercise 4.14:* Eva Lu Ator and Louis Reasoner are each + experimenting with the metacircular evaluator. Eva types in the + definition of `map', and runs some test programs that use it. + They work fine. Louis, in contrast, has installed the system + version of `map' as a primitive for the metacircular evaluator. + When he tries it, things go terribly wrong. Explain why Louis's + `map' fails even though Eva's works. + + ---------- Footnotes ---------- + + (1) Any procedure defined in the underlying Lisp can be used as a +primitive for the metacircular evaluator. The name of a primitive +installed in the evaluator need not be the same as the name of its +implementation in the underlying Lisp; the names are the same here +because the metacircular evaluator implements Scheme itself. Thus, for +example, we could put `(list 'first car)' or `(list 'square (lambda (x) +(* x x)))' in the list of `primitive-procedures'. + + (2) `Apply-in-underlying-scheme' is the `apply' procedure we have +used in earlier chapters. The metacircular evaluator's `apply' +procedure (section *Note 4-1-1::) models the working of this primitive. +Having two different things called `apply' leads to a technical +problem in running the metacircular evaluator, because defining the +metacircular evaluator's `apply' will mask the definition of the +primitive. One way around this is to rename the metacircular `apply' to +avoid conflict with the name of the primitive procedure. We have +assumed instead that we have saved a reference to the underlying +`apply' by doing + + (define apply-in-underlying-scheme apply) + +before defining the metacircular `apply'. This allows us to access the +original version of `apply' under a different name. + + (3) The primitive procedure `read' waits for input from the user, +and returns the next complete expression that is typed. For example, +if the user types `(+ 23 x)', `read' returns a three-element list +containing the symbol `+', the number 23, and the symbol `x'. If the +user types `'x', `read' returns a two-element list containing the +symbol `quote' and the symbol `x'. + + +File: sicp.info, Node: 4-1-5, Next: 4-1-6, Prev: 4-1-4, Up: 4-1 + +4.1.5 Data as Programs +---------------------- + +In thinking about a Lisp program that evaluates Lisp expressions, an +analogy might be helpful. One operational view of the meaning of a +program is that a program is a description of an abstract (perhaps +infinitely large) machine. For example, consider the familiar program +to compute factorials: + + (define (factorial n) + (if (= n 1) + 1 + (* (factorial (- n 1)) n))) + + We may regard this program as the description of a machine +containing parts that decrement, multiply, and test for equality, +together with a two-position switch and another factorial machine. (The +factorial machine is infinite because it contains another factorial +machine within it.) *Note Figure 4-2:: is a flow diagram for the +factorial machine, showing how the parts are wired together. + + *Figure 4.2:* The factorial program, viewed as an abstract machine. + + +-----------------------------------+ + | factorial |1 | + | |1 V | + | | +-----+ | + | V | # | | + | +-----+ | | | + 6 --------*-----| = |------->| #-+-----> 720 + | | +-----+ | / | | + | | | # | | + | | +-----+ | + | | ^ | + | | | | + | | +--+--+ | + | *------------------->| * | | + | | +-----+ | + | V ^ | + | +-----+ +-----------+ | | + | | - +--->| factorial +---+ | + | +-----+ +-----------+ | + | ^ | + | |1 | + +-----------------------------------+ + + In a similar way, we can regard the evaluator as a very special +machine that takes as input a description of a machine. Given this +input, the evaluator configures itself to emulate the machine +described. For example, if we feed our evaluator the definition of +`factorial', as shown in *Note Figure 4-3::, the evaluator will be able +to compute factorials. + + *Figure 4.3:* The evaluator emulating a factorial machine. + + +--------+ + 6 ---->| eval |----> 720 + +--------+ + / + . . . / . . . + . . . ../. . . + . .. + . (define (factorial n) . . . + . (if (= n 1) . . + . 1 . + . (* (factorial (- n 1)) n))) . + . . . . . + . . . . . . . . . . . . + . .. + + From this perspective, our evaluator is seen to be a machine +"universal machine". It mimics other machines when these are described +as Lisp programs.(1) This is striking. Try to imagine an analogous +evaluator for electrical circuits. This would be a circuit that takes +as input a signal encoding the plans for some other circuit, such as a +filter. Given this input, the circuit evaluator would then behave like +a filter with the same description. Such a universal electrical +circuit is almost unimaginably complex. It is remarkable that the +program evaluator is a rather simple program.(2) + + Another striking aspect of the evaluator is that it acts as a bridge +between the data objects that are manipulated by our programming +language and the programming language itself. Imagine that the +evaluator program (implemented in Lisp) is running, and that a user is +typing expressions to the evaluator and observing the results. From +the perspective of the user, an input expression such as `(* x x)' is +an expression in the programming language, which the evaluator should +execute. From the perspective of the evaluator, however, the +expression is simply a list (in this case, a list of three symbols: `*', +`x', and `x') that is to be manipulated according to a well-defined set +of rules. + + That the user's programs are the evaluator's data need not be a +source of confusion. In fact, it is sometimes convenient to ignore +this distinction, and to give the user the ability to explicitly +evaluate a data object as a Lisp expression, by making `eval' available +for use in programs. Many Lisp dialects provide a primitive `eval' +procedure that takes as arguments an expression and an environment and +evaluates the expression relative to the environment.(3) Thus, + + (eval '(* 5 5) user-initial-environment) + +and + + (eval (cons '* (list 5 5)) user-initial-environment) + +will both return 25.(4) + + *Exercise 4.15:* Given a one-argument procedure `p' and an object + `a', `p' is said to "halt" on `a' if evaluating the expression `(p + a)' returns a value (as opposed to terminating with an error + message or running forever). Show that it is impossible to write + a procedure `halts?' that correctly determines whether `p' halts + on `a' for any procedure `p' and object `a'. Use the following + reasoning: If you had such a procedure `halts?', you could + implement the following program: + + (define (run-forever) (run-forever)) + + (define (try p) + (if (halts? p p) + (run-forever) + 'halted)) + + Now consider evaluating the expression `(try try)' and show that + any possible outcome (either halting or running forever) violates + the intended behavior of `halts?'.(5) + + ---------- Footnotes ---------- + + (1) The fact that the machines are described in Lisp is inessential. +If we give our evaluator a Lisp program that behaves as an evaluator +for some other language, say C, the Lisp evaluator will emulate the C +evaluator, which in turn can emulate any machine described as a C +program. Similarly, writing a Lisp evaluator in C produces a C program +that can execute any Lisp program. The deep idea here is that any +evaluator can emulate any other. Thus, the notion of "what can in +principle be computed" (ignoring practicalities of time and memory +required) is independent of the language or the computer, and instead +reflects an underlying notion of "computability". This was first +demonstrated in a clear way by Alan M. Turing (1912-1954), whose 1936 +paper laid the foundations for theoretical computer science. In the +paper, Turing presented a simple computational model--now known as a "Turing +machine"--and argued that any "effective process" can be formulated as +a program for such a machine. (This argument is known as the "Church-Turing +thesis".) Turing then implemented a universal machine, i.e., a Turing +machine that behaves as an evaluator for Turing-machine programs. He +used this framework to demonstrate that there are well-posed problems +that cannot be computed by Turing machines (see *Note Exercise 4-15::), +and so by implication cannot be formulated as "effective processes." +Turing went on to make fundamental contributions to practical computer +science as well. For example, he invented the idea of structuring +programs using general-purpose subroutines. See Hodges 1983 for a +biography of Turing. + + (2) Some people find it counterintuitive that an evaluator, which is +implemented by a relatively simple procedure, can emulate programs that +are more complex than the evaluator itself. The existence of a +universal evaluator machine is a deep and wonderful property of +computation. theory "Recursion theory", a branch of mathematical +logic, is concerned with logical limits of computation. Douglas +Hofstadter's beautiful book `Go"del, Escher, Bach' (1979) explores some +of these ideas. + + (3) Warning: This `eval' primitive is not identical to the `eval' +procedure we implemented in section *Note 4-1-1::, because it uses +_actual_ Scheme environments rather than the sample environment +structures we built in section *Note 4-1-3::. These actual +environments cannot be manipulated by the user as ordinary lists; they +must be accessed via `eval' or other special operations. Similarly, +the `apply' primitive we saw earlier is not identical to the +metacircular `apply', because it uses actual Scheme procedures rather +than the procedure objects we constructed in sections *Note 4-1-3:: and +*Note 4-1-4::. + + (4) The MIT implementation of Scheme includes `eval', as well as a +symbol `user-initial-environment' that is bound to the initial +environment in which the user's input expressions are evaluated. + + (5) Although we stipulated that `halts?' is given a procedure +object, notice that this reasoning still applies even if `halts?' can +gain access to the procedure's text and its environment. This is +Turing's celebrated "Halting Theorem", which gave the first clear +example of a "non-computable" problem, i.e., a well-posed task that +cannot be carried out as a computational procedure. + + +File: sicp.info, Node: 4-1-6, Next: 4-1-7, Prev: 4-1-5, Up: 4-1 + +4.1.6 Internal Definitions +-------------------------- + +Our environment model of evaluation and our metacircular evaluator +execute definitions in sequence, extending the environment frame one +definition at a time. This is particularly convenient for interactive +program development, in which the programmer needs to freely mix the +application of procedures with the definition of new procedures. +However, if we think carefully about the internal definitions used to +implement block structure (introduced in section *Note 1-1-8::), we +will find that name-by-name extension of the environment may not be the +best way to define local variables. + + Consider a procedure with internal definitions, such as + + (define (f x) + (define (even? n) + (if (= n 0) + true + (odd? (- n 1)))) + (define (odd? n) + (if (= n 0) + false + (even? (- n 1)))) + ) + + Our intention here is that the name `odd?' in the body of the +procedure `even?' should refer to the procedure `odd?' that is defined +after `even?'. The scope of the name `odd?' is the entire body of `f', +not just the portion of the body of `f' starting at the point where the +`define' for `odd?' occurs. Indeed, when we consider that `odd?' is +itself defined in terms of `even?'--so that `even?' and `odd?' are +mutually recursive procedures--we see that the only satisfactory +interpretation of the two `define's is to regard them as if the names +`even?' and `odd?' were being added to the environment simultaneously. +More generally, in block structure, the scope of a local name is the +entire procedure body in which the `define' is evaluated. + + As it happens, our interpreter will evaluate calls to `f' correctly, +but for an "accidental" reason: Since the definitions of the internal +procedures come first, no calls to these procedures will be evaluated +until all of them have been defined. Hence, `odd?' will have been +defined by the time `even?' is executed. In fact, our sequential +evaluation mechanism will give the same result as a mechanism that +directly implements simultaneous definition for any procedure in which +the internal definitions come first in a body and evaluation of the +value expressions for the defined variables doesn't actually use any of +the defined variables. (For an example of a procedure that doesn't +obey these restrictions, so that sequential definition isn't equivalent +to simultaneous definition, see *Note Exercise 4-19::.)(1) + + There is, however, a simple way to treat definitions so that +internally defined names have truly simultaneous scope--just create all +local variables that will be in the current environment before +evaluating any of the value expressions. One way to do this is by a +syntax transformation on `lambda' expressions. Before evaluating the +body of a `lambda' expression, we "scan out" and eliminate all the +internal definitions in the body. The internally defined variables +will be created with a `let' and then set to their values by +assignment. For example, the procedure + + (lambda + (define u ) + (define v ) + ) + +would be transformed into + + (lambda + (let ((u '*unassigned*) + (v '*unassigned*)) + (set! u ) + (set! v ) + )) + +where `*unassigned*' is a special symbol that causes looking up a +variable to signal an error if an attempt is made to use the value of +the not-yet-assigned variable. + + An alternative strategy for scanning out internal definitions is +shown in *Note Exercise 4-18::. Unlike the transformation shown above, +this enforces the restriction that the defined variables' values can be +evaluated without using any of the variables' values.(2) + + *Exercise 4.16:* In this exercise we implement the method just + described for interpreting internal definitions. We assume that + the evaluator supports `let' (see *Note Exercise 4-6::). + + a. Change `lookup-variable-value' (section *Note 4-1-3::) to + signal an error if the value it finds is the symbol + `*unassigned*'. + + b. Write a procedure `scan-out-defines' that takes a procedure + body and returns an equivalent one that has no internal + definitions, by making the transformation described above. + + c. Install `scan-out-defines' in the interpreter, either in + `make-procedure' or in `procedure-body' (see section *Note + 4-1-3::). Which place is better? Why? + + + *Exercise 4.17:* Draw diagrams of the environment in effect when + evaluating the expression in the procedure in the text, + comparing how this will be structured when definitions are + interpreted sequentially with how it will be structured if + definitions are scanned out as described. Why is there an extra + frame in the transformed program? Explain why this difference in + environment structure can never make a difference in the behavior + of a correct program. Design a way to make the interpreter + implement the "simultaneous" scope rule for internal definitions + without constructing the extra frame. + + *Exercise 4.18:* Consider an alternative strategy for scanning out + definitions that translates the example in the text to + + (lambda + (let ((u '*unassigned*) + (v '*unassigned*)) + (let ((a ) + (b )) + (set! u a) + (set! v b)) + )) + + Here `a' and `b' are meant to represent new variable names, created + by the interpreter, that do not appear in the user's program. + Consider the `solve' procedure from section *Note 3-5-4::: + + (define (solve f y0 dt) + (define y (integral (delay dy) y0 dt)) + (define dy (stream-map f y)) + y) + + Will this procedure work if internal definitions are scanned out + as shown in this exercise? What if they are scanned out as shown + in the text? Explain. + + *Exercise 4.19:* Ben Bitdiddle, Alyssa P. Hacker, and Eva Lu Ator + are arguing about the desired result of evaluating the expression + + (let ((a 1)) + (define (f x) + (define b (+ a x)) + (define a 5) + (+ a b)) + (f 10)) + + Ben asserts that the result should be obtained using the + sequential rule for `define': `b' is defined to be 11, then `a' is + defined to be 5, so the result is 16. Alyssa objects that mutual + recursion requires the simultaneous scope rule for internal + procedure definitions, and that it is unreasonable to treat + procedure names differently from other names. Thus, she argues + for the mechanism implemented in *Note Exercise 4-16::. This + would lead to `a' being unassigned at the time that the value for + `b' is to be computed. Hence, in Alyssa's view the procedure + should produce an error. Eva has a third opinion. She says that + if the definitions of `a' and `b' are truly meant to be + simultaneous, then the value 5 for `a' should be used in + evaluating `b'. Hence, in Eva's view `a' should be 5, `b' should + be 15, and the result should be 20. Which (if any) of these + viewpoints do you support? Can you devise a way to implement + internal definitions so that they behave as Eva prefers?(3) + + *Exercise 4.20:* Because internal definitions look sequential but + are actually simultaneous, some people prefer to avoid them + entirely, and use the special form `letrec' instead. `Letrec' + looks like `let', so it is not surprising that the variables it + binds are bound simultaneously and have the same scope as each + other. The sample procedure `f' above can be written without + internal definitions, but with exactly the same meaning, as + + (define (f x) + (letrec ((even? + (lambda (n) + (if (= n 0) + true + (odd? (- n 1))))) + (odd? + (lambda (n) + (if (= n 0) + false + (even? (- n 1)))))) + )) + + `Letrec' expressions, which have the form + + (letrec (( ) ... ( )) + ) + + are a variation on `let' in which the expressions that + provide the initial values for the variables are evaluated + in an environment that includes all the `letrec' bindings. This + permits recursion in the bindings, such as the mutual recursion of + `even?' and `odd?' in the example above, or the evaluation of 10 + factorial with + + (letrec ((fact + (lambda (n) + (if (= n 1) + 1 + (* n (fact (- n 1))))))) + (fact 10)) + + a. Implement `letrec' as a derived expression, by transforming a + `letrec' expression into a `let' expression as shown in the + text above or in *Note Exercise 4-18::. That is, the + `letrec' variables should be created with a `let' and then be + assigned their values with `set!'. + + b. Louis Reasoner is confused by all this fuss about internal + definitions. The way he sees it, if you don't like to use + `define' inside a procedure, you can just use `let'. + Illustrate what is loose about his reasoning by drawing an + environment diagram that shows the environment in which the + is evaluated during evaluation of the + expression `(f 5)', with `f' defined as in this exercise. + Draw an environment diagram for the same evaluation, but with + `let' in place of `letrec' in the definition of `f'. + + + *Exercise 4.21:* Amazingly, Louis's intuition in *Note Exercise + 4-20:: is correct. It is indeed possible to specify recursive + procedures without using `letrec' (or even `define'), although the + method for accomplishing this is much more subtle than Louis + imagined. The following expression computes 10 factorial by + applying a recursive factorial procedure:(4) + + ((lambda (n) + ((lambda (fact) + (fact fact n)) + (lambda (ft k) + (if (= k 1) + 1 + (* k (ft ft (- k 1))))))) + 10) + + a. Check (by evaluating the expression) that this really does + compute factorials. Devise an analogous expression for + computing Fibonacci numbers. + + b. Consider the following procedure, which includes mutually + recursive internal definitions: + + (define (f x) + (define (even? n) + (if (= n 0) + true + (odd? (- n 1)))) + (define (odd? n) + (if (= n 0) + false + (even? (- n 1)))) + (even? x)) + + Fill in the missing expressions to complete an alternative + definition of `f', which uses neither internal definitions + nor `letrec': + + (define (f x) + ((lambda (even? odd?) + (even? even? odd? x)) + (lambda (ev? od? n) + (if (= n 0) true (od? ))) + (lambda (ev? od? n) + (if (= n 0) false (ev? ))))) + + ---------- Footnotes ---------- + + (1) Wanting programs to not depend on this evaluation mechanism is +the reason for the "management is not responsible" remark in *Note +Footnote 28:: of *Note Chapter 1::. By insisting that internal +definitions come first and do not use each other while the definitions +are being evaluated, the IEEE standard for Scheme leaves implementors +some choice in the mechanism used to evaluate these definitions. The +choice of one evaluation rule rather than another here may seem like a +small issue, affecting only the interpretation of "badly formed" +programs. However, we will see in section *Note 5-5-6:: that moving to +a model of simultaneous scoping for internal definitions avoids some +nasty difficulties that would otherwise arise in implementing a +compiler. + + (2) The IEEE standard for Scheme allows for different implementation +strategies by specifying that it is up to the programmer to obey this +restriction, not up to the implementation to enforce it. Some Scheme +implementations, including MIT Scheme, use the transformation shown +above. Thus, some programs that don't obey this restriction will in +fact run in such implementations. + + (3) The MIT implementors of Scheme support Alyssa on the following +grounds: Eva is in principle correct - the definitions should be +regarded as simultaneous. But it seems difficult to implement a +general, efficient mechanism that does what Eva requires. In the +absence of such a mechanism, it is better to generate an error in the +difficult cases of simultaneous definitions (Alyssa's notion) than to +produce an incorrect answer (as Ben would have it). + + (4) This example illustrates a programming trick for formulating +recursive procedures without using `define'. The most general trick of +this sort is the Y "operator", which can be used to give a "pure +[lambda]-calculus" implementation of recursion. (See Stoy 1977 for +details on the [lambda] calculus, and Gabriel 1988 for an exposition of +the Y operator in Scheme.) + + +File: sicp.info, Node: 4-1-7, Prev: 4-1-6, Up: 4-1 + +4.1.7 Separating Syntactic Analysis from Execution +-------------------------------------------------- + +The evaluator implemented above is simple, but it is very inefficient, +because the syntactic analysis of expressions is interleaved with their +execution. Thus if a program is executed many times, its syntax is +analyzed many times. Consider, for example, evaluating `(factorial 4)' +using the following definition of `factorial': + + (define (factorial n) + (if (= n 1) + 1 + (* (factorial (- n 1)) n))) + + Each time `factorial' is called, the evaluator must determine that +the body is an `if' expression and extract the predicate. Only then +can it evaluate the predicate and dispatch on its value. Each time it +evaluates the expression `(* (factorial (- n 1)) n)', or the +subexpressions `(factorial (- n 1))' and `(- n 1)', the evaluator must +perform the case analysis in `eval' to determine that the expression is +an application, and must extract its operator and operands. This +analysis is expensive. Performing it repeatedly is wasteful. + + We can transform the evaluator to be significantly more efficient by +arranging things so that syntactic analysis is performed only once.(1) +We split `eval', which takes an expression and an environment, into two +parts. The procedure `analyze' takes only the expression. It performs +the syntactic analysis and returns a new procedure, the "execution +procedure", that encapsulates the work to be done in executing the +analyzed expression. The execution procedure takes an environment as +its argument and completes the evaluation. This saves work because +`analyze' will be called only once on an expression, while the +execution procedure may be called many times. + + With the separation into analysis and execution, `eval' now becomes + + (define (eval exp env) + ((analyze exp) env)) + + The result of calling `analyze' is the execution procedure to be +applied to the environment. The `analyze' procedure is the same case +analysis as performed by the original `eval' of section *Note 4-1-1::, +except that the procedures to which we dispatch perform only analysis, +not full evaluation: + + (define (analyze exp) + (cond ((self-evaluating? exp) + (analyze-self-evaluating exp)) + ((quoted? exp) (analyze-quoted exp)) + ((variable? exp) (analyze-variable exp)) + ((assignment? exp) (analyze-assignment exp)) + ((definition? exp) (analyze-definition exp)) + ((if? exp) (analyze-if exp)) + ((lambda? exp) (analyze-lambda exp)) + ((begin? exp) (analyze-sequence (begin-actions exp))) + ((cond? exp) (analyze (cond->if exp))) + ((application? exp) (analyze-application exp)) + (else + (error "Unknown expression type -- ANALYZE" exp)))) + + Here is the simplest syntactic analysis procedure, which handles +self-evaluating expressions. It returns an execution procedure that +ignores its environment argument and just returns the expression: + + (define (analyze-self-evaluating exp) + (lambda (env) exp)) + + For a quoted expression, we can gain a little efficiency by +extracting the text of the quotation only once, in the analysis phase, +rather than in the execution phase. + + (define (analyze-quoted exp) + (let ((qval (text-of-quotation exp))) + (lambda (env) qval))) + + Looking up a variable value must still be done in the execution +phase, since this depends upon knowing the environment.(2) + + (define (analyze-variable exp) + (lambda (env) (lookup-variable-value exp env))) + + `Analyze-assignment' also must defer actually setting the variable +until the execution, when the environment has been supplied. However, +the fact that the `assignment-value' expression can be analyzed +(recursively) during analysis is a major gain in efficiency, because +the `assignment-value' expression will now be analyzed only once. The +same holds true for definitions. + + (define (analyze-assignment exp) + (let ((var (assignment-variable exp)) + (vproc (analyze (assignment-value exp)))) + (lambda (env) + (set-variable-value! var (vproc env) env) + 'ok))) + + (define (analyze-definition exp) + (let ((var (definition-variable exp)) + (vproc (analyze (definition-value exp)))) + (lambda (env) + (define-variable! var (vproc env) env) + 'ok))) + + For `if' expressions, we extract and analyze the predicate, +consequent, and alternative at analysis time. + + (define (analyze-if exp) + (let ((pproc (analyze (if-predicate exp))) + (cproc (analyze (if-consequent exp))) + (aproc (analyze (if-alternative exp)))) + (lambda (env) + (if (true? (pproc env)) + (cproc env) + (aproc env))))) + + Analyzing a `lambda' expression also achieves a major gain in +efficiency: We analyze the `lambda' body only once, even though +procedures resulting from evaluation of the `lambda' may be applied +many times. + + (define (analyze-lambda exp) + (let ((vars (lambda-parameters exp)) + (bproc (analyze-sequence (lambda-body exp)))) + (lambda (env) (make-procedure vars bproc env)))) + + Analysis of a sequence of expressions (as in a `begin' or the body +of a `lambda' expression) is more involved.(3) Each expression in the +sequence is analyzed, yielding an execution procedure. These execution +procedures are combined to produce an execution procedure that takes an +environment as argument and sequentially calls each individual execution +procedure with the environment as argument. + + (define (analyze-sequence exps) + (define (sequentially proc1 proc2) + (lambda (env) (proc1 env) (proc2 env))) + (define (loop first-proc rest-procs) + (if (null? rest-procs) + first-proc + (loop (sequentially first-proc (car rest-procs)) + (cdr rest-procs)))) + (let ((procs (map analyze exps))) + (if (null? procs) + (error "Empty sequence -- ANALYZE")) + (loop (car procs) (cdr procs)))) + + To analyze an application, we analyze the operator and operands and +construct an execution procedure that calls the operator execution +procedure (to obtain the actual procedure to be applied) and the +operand execution procedures (to obtain the actual arguments). We then +pass these to `execute-application', which is the analog of `apply' in +section *Note 4-1-1::. `Execute-application' differs from `apply' in +that the procedure body for a compound procedure has already been +analyzed, so there is no need to do further analysis. Instead, we just +call the execution procedure for the body on the extended environment. + + (define (analyze-application exp) + (let ((fproc (analyze (operator exp))) + (aprocs (map analyze (operands exp)))) + (lambda (env) + (execute-application (fproc env) + (map (lambda (aproc) (aproc env)) + aprocs))))) + + (define (execute-application proc args) + (cond ((primitive-procedure? proc) + (apply-primitive-procedure proc args)) + ((compound-procedure? proc) + ((procedure-body proc) + (extend-environment (procedure-parameters proc) + args + (procedure-environment proc)))) + (else + (error + "Unknown procedure type -- EXECUTE-APPLICATION" + proc)))) + + Our new evaluator uses the same data structures, syntax procedures, +and run-time support procedures as in sections *Note 4-1-2::, *Note +4-1-3::, and *Note 4-1-4::. + + *Exercise 4.22:* Extend the evaluator in this section to support + the special form `let'. (See *Note Exercise 4-6::.) + + *Exercise 4.23:* Alyssa P. Hacker doesn't understand why + `analyze-sequence' needs to be so complicated. All the other + analysis procedures are straightforward transformations of the + corresponding evaluation procedures (or `eval' clauses) in section + *Note 4-1-1::. She expected `analyze-sequence' to look like this: + + (define (analyze-sequence exps) + (define (execute-sequence procs env) + (cond ((null? (cdr procs)) ((car procs) env)) + (else ((car procs) env) + (execute-sequence (cdr procs) env)))) + (let ((procs (map analyze exps))) + (if (null? procs) + (error "Empty sequence -- ANALYZE")) + (lambda (env) (execute-sequence procs env)))) + + Eva Lu Ator explains to Alyssa that the version in the text does + more of the work of evaluating a sequence at analysis time. + Alyssa's sequence-execution procedure, rather than having the + calls to the individual execution procedures built in, loops + through the procedures in order to call them: In effect, although + the individual expressions in the sequence have been analyzed, the + sequence itself has not been. + + Compare the two versions of `analyze-sequence'. For example, + consider the common case (typical of procedure bodies) where the + sequence has just one expression. What work will the execution + procedure produced by Alyssa's program do? What about the + execution procedure produced by the program in the text above? + How do the two versions compare for a sequence with two + expressions? + + *Exercise 4.24:* Design and carry out some experiments to compare + the speed of the original metacircular evaluator with the version + in this section. Use your results to estimate the fraction of time + that is spent in analysis versus execution for various procedures. + + ---------- Footnotes ---------- + + (1) This technique is an integral part of the compilation process, +which we shall discuss in *Note Chapter 5::. Jonathan Rees wrote a +Scheme interpreter like this in about 1982 for the T project (Rees and +Adams 1982). Marc Feeley (1986) (see also Feeley and Lapalme 1987) +independently invented this technique in his master's thesis. + + (2) There is, however, an important part of the variable search that +_can_ be done as part of the syntactic analysis. As we will show in +section *Note 5-5-6::, one can determine the position in the +environment structure where the value of the variable will be found, +thus obviating the need to scan the environment for the entry that +matches the variable. + + (3) See *Note Exercise 4-23:: for some insight into the processing +of sequences. + + +File: sicp.info, Node: 4-2, Next: 4-3, Prev: 4-1, Up: Chapter 4 + +4.2 Variations on a Scheme - Lazy Evaluation +============================================ + +Now that we have an evaluator expressed as a Lisp program, we can +experiment with alternative choices in language design simply by +modifying the evaluator. Indeed, new languages are often invented by +first writing an evaluator that embeds the new language within an +existing high-level language. For example, if we wish to discuss some +aspect of a proposed modification to Lisp with another member of the +Lisp community, we can supply an evaluator that embodies the change. +The recipient can then experiment with the new evaluator and send back +comments as further modifications. Not only does the high-level +implementation base make it easier to test and debug the evaluator; in +addition, the embedding enables the designer to snarf(1) features from +the underlying language, just as our embedded Lisp evaluator uses +primitives and control structure from the underlying Lisp. Only later +(if ever) need the designer go to the trouble of building a complete +implementation in a low-level language or in hardware. In this section +and the next we explore some variations on Scheme that provide +significant additional expressive power. + +* Menu: + +* 4-2-1:: Normal Order and Applicative Order +* 4-2-2:: An Interpreter with Lazy Evaluation +* 4-2-3:: Streams as Lazy Lists + + ---------- Footnotes ---------- + + (1) Snarf: "To grab, especially a large document or file for the +purpose of using it either with or without the owner's permission." +Snarf down: "To snarf, sometimes with the connotation of absorbing, +processing, or understanding." (These definitions were snarfed from +Steele et al. 1983. See also Raymond 1993.) + + +File: sicp.info, Node: 4-2-1, Next: 4-2-2, Prev: 4-2, Up: 4-2 + +4.2.1 Normal Order and Applicative Order +---------------------------------------- + +In section *Note 1-1::, where we began our discussion of models of +evaluation, we noted that Scheme is an "applicative-order" language, +namely, that all the arguments to Scheme procedures are evaluated when +the procedure is applied. In contrast, "normal-order" languages delay +evaluation of procedure arguments until the actual argument values are +needed. Delaying evaluation of procedure arguments until the last +possible moment (e.g., until they are required by a primitive +operation) is called evaluation "lazy evaluation".(1) Consider the +procedure + + (define (try a b) + (if (= a 0) 1 b)) + + Evaluating `(try 0 (/ 1 0))' generates an error in Scheme. With lazy +evaluation, there would be no error. Evaluating the expression would +return 1, because the argument `(/ 1 0)' would never be evaluated. + + An example that exploits lazy evaluation is the definition of a +procedure `unless' + + (define (unless condition usual-value exceptional-value) + (if condition exceptional-value usual-value)) + +that can be used in expressions such as + + (unless (= b 0) + (/ a b) + (begin (display "exception: returning 0") + 0)) + + This won't work in an applicative-order language because both the +usual value and the exceptional value will be evaluated before `unless' +is called (compare *Note Exercise 1-6::). An advantage of lazy +evaluation is that some procedures, such as `unless', can do useful +computation even if evaluation of some of their arguments would produce +errors or would not terminate. + + If the body of a procedure is entered before an argument has been +evaluated we say that the procedure is "non-strict" in that argument. +If the argument is evaluated before the body of the procedure is +entered we say that the procedure is "strict" in that argument.(2) In +a purely applicative-order language, all procedures are strict in each +argument. In a purely normal-order language, all compound procedures +are non-strict in each argument, and primitive procedures may be either +strict or non-strict. There are also languages (see *Note Exercise +4-31::) that give programmers detailed control over the strictness of +the procedures they define. + + A striking example of a procedure that can usefully be made +non-strict is `cons' (or, in general, almost any constructor for data +structures). One can do useful computation, combining elements to form +data structures and operating on the resulting data structures, even if +the values of the elements are not known. It makes perfect sense, for +instance, to compute the length of a list without knowing the values of +the individual elements in the list. We will exploit this idea in +section *Note 4-2-3:: to implement the streams of *Note Chapter 3:: as +lists formed of non-strict `cons' pairs. + + *Exercise 4.25:* Suppose that (in ordinary applicative-order + Scheme) we define `unless' as shown above and then define + `factorial' in terms of `unless' as + + (define (factorial n) + (unless (= n 1) + (* n (factorial (- n 1))) + 1)) + + What happens if we attempt to evaluate `(factorial 5)'? Will our + definitions work in a normal-order language? + + *Exercise 4.26:* Ben Bitdiddle and Alyssa P. Hacker disagree over + the importance of lazy evaluation for implementing things such as + `unless'. Ben points out that it's possible to implement `unless' + in applicative order as a special form. Alyssa counters that, if + one did that, `unless' would be merely syntax, not a procedure + that could be used in conjunction with higher-order procedures. + Fill in the details on both sides of the argument. Show how to + implement `unless' as a derived expression (like `cond' or `let'), + and give an example of a situation where it might be useful to + have `unless' available as a procedure, rather than as a special + form. + + ---------- Footnotes ---------- + + (1) The difference between the "lazy" terminology and the +"normal-order" terminology is somewhat fuzzy. Generally, "lazy" refers +to the mechanisms of particular evaluators, while "normal-order" refers +to the semantics of languages, independent of any particular evaluation +strategy. But this is not a hard-and-fast distinction, and the two +terminologies are often used interchangeably. + + (2) The "strict" versus "non-strict" terminology means essentially +the same thing as "applicative-order" versus "normal-order," except +that it refers to individual procedures and arguments rather than to +the language as a whole. At a conference on programming languages you +might hear someone say, "The normal-order language Hassle has certain +strict primitives. Other procedures take their arguments by lazy +evaluation." + + +File: sicp.info, Node: 4-2-2, Next: 4-2-3, Prev: 4-2-1, Up: 4-2 + +4.2.2 An Interpreter with Lazy Evaluation +----------------------------------------- + +In this section we will implement a normal-order language that is the +same as Scheme except that compound procedures are non-strict in each +argument. Primitive procedures will still be strict. It is not +difficult to modify the evaluator of section *Note 4-1-1:: so that the +language it interprets behaves this way. Almost all the required +changes center around procedure application. + + The basic idea is that, when applying a procedure, the interpreter +must determine which arguments are to be evaluated and which are to be +delayed. The delayed arguments are not evaluated; instead, they are +transformed into objects called "thunks".(1) The thunk must contain the +information required to produce the value of the argument when it is +needed, as if it had been evaluated at the time of the application. +Thus, the thunk must contain the argument expression and the +environment in which the procedure application is being evaluated. + + The process of evaluating the expression in a thunk is called "forcing".(2) +In general, a thunk will be forced only when its value is needed: when +it is passed to a primitive procedure that will use the value of the +thunk; when it is the value of a predicate of a conditional; and when +it is the value of an operator that is about to be applied as a +procedure. One design choice we have available is whether or not to "memoize" +thunks, as we did with delayed objects in section *Note 3-5-1::. With +memoization, the first time a thunk is forced, it stores the value that +is computed. Subsequent forcings simply return the stored value +without repeating the computation. We'll make our interpreter memoize, +because this is more efficient for many applications. There are tricky +considerations here, however.(3) + +Modifying the evaluator +....................... + +The main difference between the lazy evaluator and the one in section +*Note 4-1:: is in the handling of procedure applications in `eval' and +`apply'. + + The `application?' clause of `eval' becomes + + ((application? exp) + (apply (actual-value (operator exp) env) + (operands exp) + env)) + + This is almost the same as the `application?' clause of `eval' in +section *Note 4-1-1::. For lazy evaluation, however, we call `apply' +with the operand expressions, rather than the arguments produced by +evaluating them. Since we will need the environment to construct +thunks if the arguments are to be delayed, we must pass this as well. +We still evaluate the operator, because `apply' needs the actual +procedure to be applied in order to dispatch on its type (primitive +versus compound) and apply it. + + Whenever we need the actual value of an expression, we use + + (define (actual-value exp env) + (force-it (eval exp env))) + +instead of just `eval', so that if the expression's value is a thunk, it +will be forced. + + Our new version of `apply' is also almost the same as the version in +section *Note 4-1-1::. The difference is that `eval' has passed in +unevaluated operand expressions: For primitive procedures (which are +strict), we evaluate all the arguments before applying the primitive; +for compound procedures (which are non-strict) we delay all the +arguments before applying the procedure. + + (define (apply procedure arguments env) + (cond ((primitive-procedure? procedure) + (apply-primitive-procedure + procedure + (list-of-arg-values arguments env))) ; changed + ((compound-procedure? procedure) + (eval-sequence + (procedure-body procedure) + (extend-environment + (procedure-parameters procedure) + (list-of-delayed-args arguments env) ; changed + (procedure-environment procedure)))) + (else + (error + "Unknown procedure type -- APPLY" procedure)))) + + The procedures that process the arguments are just like +`list-of-values' from section *Note 4-1-1::, except that +`list-of-delayed-args' delays the arguments instead of evaluating them, +and `list-of-arg-values' uses `actual-value' instead of `eval': + + (define (list-of-arg-values exps env) + (if (no-operands? exps) + '() + (cons (actual-value (first-operand exps) env) + (list-of-arg-values (rest-operands exps) + env)))) + + (define (list-of-delayed-args exps env) + (if (no-operands? exps) + '() + (cons (delay-it (first-operand exps) env) + (list-of-delayed-args (rest-operands exps) + env)))) + + The other place we must change the evaluator is in the handling of +`if', where we must use `actual-value' instead of `eval' to get the +value of the predicate expression before testing whether it is true or +false: + + (define (eval-if exp env) + (if (true? (actual-value (if-predicate exp) env)) + (eval (if-consequent exp) env) + (eval (if-alternative exp) env))) + + Finally, we must change the `driver-loop' procedure (section *Note +4-1-4::) to use `actual-value' instead of `eval', so that if a delayed +value is propagated back to the read-eval-print loop, it will be forced +before being printed. We also change the prompts to indicate that this +is the lazy evaluator: + + (define input-prompt ";;; L-Eval input:") + (define output-prompt ";;; L-Eval value:") + + (define (driver-loop) + (prompt-for-input input-prompt) + (let ((input (read))) + (let ((output + (actual-value input the-global-environment))) + (announce-output output-prompt) + (user-print output))) + (driver-loop)) + + With these changes made, we can start the evaluator and test it. The +successful evaluation of the `try' expression discussed in section +*Note 4-2-1:: indicates that the interpreter is performing lazy +evaluation: + + (define the-global-environment (setup-environment)) + + (driver-loop) + + ;;; L-Eval input: + (define (try a b) + (if (= a 0) 1 b)) + ;;; L-Eval value: + ok + + ;;; L-Eval input: + (try 0 (/ 1 0)) + ;;; L-Eval value: + 1 + +Representing thunks +................... + +Our evaluator must arrange to create thunks when procedures are applied +to arguments and to force these thunks later. A thunk must package an +expression together with the environment, so that the argument can be +produced later. To force the thunk, we simply extract the expression +and environment from the thunk and evaluate the expression in the +environment. We use `actual-value' rather than `eval' so that in case +the value of the expression is itself a thunk, we will force that, and +so on, until we reach something that is not a thunk: + + (define (force-it obj) + (if (thunk? obj) + (actual-value (thunk-exp obj) (thunk-env obj)) + obj)) + + One easy way to package an expression with an environment is to make +a list containing the expression and the environment. Thus, we create +a thunk as follows: + + (define (delay-it exp env) + (list 'thunk exp env)) + + (define (thunk? obj) + (tagged-list? obj 'thunk)) + + (define (thunk-exp thunk) (cadr thunk)) + + (define (thunk-env thunk) (caddr thunk)) + + Actually, what we want for our interpreter is not quite this, but +rather thunks that have been memoized. When a thunk is forced, we will +turn it into an evaluated thunk by replacing the stored expression with +its value and changing the `thunk' tag so that it can be recognized as +already evaluated.(4) + + (define (evaluated-thunk? obj) + (tagged-list? obj 'evaluated-thunk)) + + (define (thunk-value evaluated-thunk) (cadr evaluated-thunk)) + + (define (force-it obj) + (cond ((thunk? obj) + (let ((result (actual-value + (thunk-exp obj) + (thunk-env obj)))) + (set-car! obj 'evaluated-thunk) + (set-car! (cdr obj) result) ; replace `exp' with its value + (set-cdr! (cdr obj) '()) ; forget unneeded `env' + result)) + ((evaluated-thunk? obj) + (thunk-value obj)) + (else obj))) + + Notice that the same `delay-it' procedure works both with and without +memoization. + + *Exercise 4.27:* Suppose we type in the following definitions to + the lazy evaluator: + + (define count 0) + + (define (id x) + (set! count (+ count 1)) + x) + + Give the missing values in the following sequence of interactions, + and explain your answers.(5) + + (define w (id (id 10))) + + ;;; L-Eval input: + count + ;;; L-Eval value: + + + ;;; L-Eval input: + w + ;;; L-Eval value: + + + ;;; L-Eval input: + count + ;;; L-Eval value: + + + *Exercise 4.28:* `Eval' uses `actual-value' rather than `eval' to + evaluate the operator before passing it to `apply', in order to + force the value of the operator. Give an example that + demonstrates the need for this forcing. + + *Exercise 4.29:* Exhibit a program that you would expect to run + much more slowly without memoization than with memoization. Also, + consider the following interaction, where the `id' procedure is + defined as in *Note Exercise 4-27:: and `count' starts at 0: + + (define (square x) + (* x x)) + + ;;; L-Eval input: + (square (id 10)) + ;;; L-Eval value: + + + ;;; L-Eval input: + count + ;;; L-Eval value: + + + Give the responses both when the evaluator memoizes and when it + does not. + + *Exercise 4.30:* Cy D. Fect, a reformed C programmer, is worried + that some side effects may never take place, because the lazy + evaluator doesn't force the expressions in a sequence. Since the + value of an expression in a sequence other than the last one is + not used (the expression is there only for its effect, such as + assigning to a variable or printing), there can be no subsequent + use of this value (e.g., as an argument to a primitive procedure) + that will cause it to be forced. Cy thus thinks that when + evaluating sequences, we must force all expressions in the + sequence except the final one. He proposes to modify + `eval-sequence' from section *Note 4-1-1:: to use `actual-value' + rather than `eval': + + (define (eval-sequence exps env) + (cond ((last-exp? exps) (eval (first-exp exps) env)) + (else (actual-value (first-exp exps) env) + (eval-sequence (rest-exps exps) env)))) + + a. Ben Bitdiddle thinks Cy is wrong. He shows Cy the `for-each' + procedure described in *Note Exercise 2-23::, which gives an + important example of a sequence with side effects: + + (define (for-each proc items) + (if (null? items) + 'done + (begin (proc (car items)) + (for-each proc (cdr items))))) + + He claims that the evaluator in the text (with the original + `eval-sequence') handles this correctly: + + ;;; L-Eval input: + (for-each (lambda (x) (newline) (display x)) + (list 57 321 88)) + 57 + 321 + 88 + ;;; L-Eval value: + done + + Explain why Ben is right about the behavior of `for-each'. + + b. Cy agrees that Ben is right about the `for-each' example, but + says that that's not the kind of program he was thinking + about when he proposed his change to `eval-sequence'. He + defines the following two procedures in the lazy evaluator: + + (define (p1 x) + (set! x (cons x '(2))) + x) + + (define (p2 x) + (define (p e) + e + x) + (p (set! x (cons x '(2))))) + + What are the values of `(p1 1)' and `(p2 1)' with the original + `eval-sequence'? What would the values be with Cy's proposed + change to `eval-sequence'? + + c. Cy also points out that changing `eval-sequence' as he + proposes does not affect the behavior of the example in part + a. Explain why this is true. + + d. How do you think sequences ought to be treated in the lazy + evaluator? Do you like Cy's approach, the approach in the + text, or some other approach? + + + *Exercise 4.31:* The approach taken in this section is somewhat + unpleasant, because it makes an incompatible change to Scheme. It + might be nicer to implement lazy evaluation as an "upward-compatible + extension", that is, so that ordinary Scheme programs will work as + before. We can do this by extending the syntax of procedure + declarations to let the user control whether or not arguments are + to be delayed. While we're at it, we may as well also give the + user the choice between delaying with and without memoization. + For example, the definition + + (define (f a (b lazy) c (d lazy-memo)) + ...) + + would define `f' to be a procedure of four arguments, where the + first and third arguments are evaluated when the procedure is + called, the second argument is delayed, and the fourth argument is + both delayed and memoized. Thus, ordinary procedure definitions + will produce the same behavior as ordinary Scheme, while adding + the `lazy-memo' declaration to each parameter of every compound + procedure will produce the behavior of the lazy evaluator defined + in this section. Design and implement the changes required to + produce such an extension to Scheme. You will have to implement + new syntax procedures to handle the new syntax for `define'. You + must also arrange for `eval' or `apply' to determine when + arguments are to be delayed, and to force or delay arguments + accordingly, and you must arrange for forcing to memoize or not, + as appropriate. + + ---------- Footnotes ---------- + + (1) The word "thunk" was invented by an informal working group that +was discussing the implementation of call-by-name in Algol 60. They +observed that most of the analysis of ("thinking about") the expression +could be done at compile time; thus, at run time, the expression would +already have been "thunk" about (Ingerman et al. 1960). + + (2) This is analogous to the use of `force' on the delayed objects +that were introduced in *Note Chapter 3:: to represent streams. The +critical difference between what we are doing here and what we did in +*Note Chapter 3:: is that we are building delaying and forcing into the +evaluator, and thus making this uniform and automatic throughout the +language. + + (3) Lazy evaluation combined with memoization is sometimes referred +to as "call-by-need" argument passing, in contrast to "call-by-name" +argument passing. (Call-by-name, introduced in Algol 60, is similar to +non-memoized lazy evaluation.) As language designers, we can build our +evaluator to memoize, not to memoize, or leave this an option for +programmers (*Note Exercise 4-31::). As you might expect from *Note +Chapter 3::, these choices raise issues that become both subtle and +confusing in the presence of assignments. (See *Note Exercise 4-27:: +and *Note Exercise 4-29::.) An excellent article by Clinger (1982) +attempts to clarify the multiple dimensions of confusion that arise +here. + + (4) Notice that we also erase the `env' from the thunk once the +expression's value has been computed. This makes no difference in the +values returned by the interpreter. It does help save space, however, +because removing the reference from the thunk to the `env' once it is +no longer needed allows this structure to be "garbage-collected" and +its space recycled, as we will discuss in section *Note 5-3::. + + Similarly, we could have allowed unneeded environments in the +memoized delayed objects of section *Note 3-5-1:: to be +garbage-collected, by having `memo-proc' do something like `(set! proc +'())' to discard the procedure `proc' (which includes the environment +in which the `delay' was evaluated) after storing its value. + + (5) This exercise demonstrates that the interaction between lazy +evaluation and side effects can be very confusing. This is just what +you might expect from the discussion in *Note Chapter 3::. + + +File: sicp.info, Node: 4-2-3, Prev: 4-2-2, Up: 4-2 + +4.2.3 Streams as Lazy Lists +--------------------------- + +In section *Note 3-5-1::, we showed how to implement streams as delayed +lists. We introduced special forms `delay' and `cons-stream', which +allowed us to construct a "promise" to compute the `cdr' of a stream, +without actually fulfilling that promise until later. We could use +this general technique of introducing special forms whenever we need +more control over the evaluation process, but this is awkward. For one +thing, a special form is not a first-class object like a procedure, so +we cannot use it together with higher-order procedures.(1) +Additionally, we were forced to create streams as a new kind of data +object similar but not identical to lists, and this required us to +reimplement many ordinary list operations (`map', `append', and so on) +for use with streams. + + With lazy evaluation, streams and lists can be identical, so there +is no need for special forms or for separate list and stream +operations. All we need to do is to arrange matters so that `cons' is +non-strict. One way to accomplish this is to extend the lazy evaluator +to allow for non-strict primitives, and to implement `cons' as one of +these. An easier way is to recall (section *Note 2-1-3::) that there +is no fundamental need to implement `cons' as a primitive at all. +Instead, we can represent pairs as procedures:(2) + + (define (cons x y) + (lambda (m) (m x y))) + + (define (car z) + (z (lambda (p q) p))) + + (define (cdr z) + (z (lambda (p q) q))) + + In terms of these basic operations, the standard definitions of the +list operations will work with infinite lists (streams) as well as +finite ones, and the stream operations can be implemented as list +operations. Here are some examples: + + (define (list-ref items n) + (if (= n 0) + (car items) + (list-ref (cdr items) (- n 1)))) + + (define (map proc items) + (if (null? items) + '() + (cons (proc (car items)) + (map proc (cdr items))))) + + (define (scale-list items factor) + (map (lambda (x) (* x factor)) + items)) + + (define (add-lists list1 list2) + (cond ((null? list1) list2) + ((null? list2) list1) + (else (cons (+ (car list1) (car list2)) + (add-lists (cdr list1) (cdr list2)))))) + + (define ones (cons 1 ones)) + + (define integers (cons 1 (add-lists ones integers))) + + ;;; L-Eval input: + (list-ref integers 17) + ;;; L-Eval value: + 18 + + Note that these lazy lists are even lazier than the streams of *Note +Chapter 3::: The `car' of the list, as well as the `cdr', is +delayed.(3) In fact, even accessing the `car' or `cdr' of a lazy pair +need not force the value of a list element. The value will be forced +only when it is really needed - e.g., for use as the argument of a +primitive, or to be printed as an answer. + + Lazy pairs also help with the problem that arose with streams in +section *Note 3-5-4::, where we found that formulating stream models of +systems with loops may require us to sprinkle our programs with +explicit `delay' operations, beyond the ones supplied by `cons-stream'. +With lazy evaluation, all arguments to procedures are delayed +uniformly. For instance, we can implement procedures to integrate +lists and solve differential equations as we originally intended in +section *Note 3-5-4::: + + (define (integral integrand initial-value dt) + (define int + (cons initial-value + (add-lists (scale-list integrand dt) + int))) + int) + + (define (solve f y0 dt) + (define y (integral dy y0 dt)) + (define dy (map f y)) + y) + + ;;; L-Eval input: + (list-ref (solve (lambda (x) x) 1 0.001) 1000) + ;;; L-Eval value: + 2.716924 + + *Exercise 4.32:* Give some examples that illustrate the difference + between the streams of *Note Chapter 3:: and the "lazier" lazy + lists described in this section. How can you take advantage of + this extra laziness? + + *Exercise 4.33:* Ben Bitdiddle tests the lazy list implementation + given above by evaluating the expression + + (car '(a b c)) + + To his surprise, this produces an error. After some thought, he + realizes that the "lists" obtained by reading in quoted + expressions are different from the lists manipulated by the new + definitions of `cons', `car', and `cdr'. Modify the evaluator's + treatment of quoted expressions so that quoted lists typed at the + driver loop will produce true lazy lists. + + *Exercise 4.34:* Modify the driver loop for the evaluator so that + lazy pairs and lists will print in some reasonable way. (What are + you going to do about infinite lists?) You may also need to modify + the representation of lazy pairs so that the evaluator can + identify them in order to print them. + + ---------- Footnotes ---------- + + (1) This is precisely the issue with the `unless' procedure, as in +*Note Exercise 4-26::. + + (2) This is the procedural representation described in *Note +Exercise 2-4::. Essentially any procedural representation (e.g., a +message-passing implementation) would do as well. Notice that we can +install these definitions in the lazy evaluator simply by typing them +at the driver loop. If we had originally included `cons', `car', and +`cdr' as primitives in the global environment, they will be redefined. +(Also see *Note Exercise 4-33:: and *Note Exercise 4-34::.) + + (3) This permits us to create delayed versions of more general kinds +of list structures, not just sequences. Hughes 1990 discusses some +applications of "lazy trees." + + +File: sicp.info, Node: 4-3, Next: 4-4, Prev: 4-2, Up: Chapter 4 + +4.3 Variations on a Scheme - Nondeterministic Computing +======================================================= + +In this section, we extend the Scheme evaluator to support a programming +paradigm called "nondeterministic computing" by building into the +evaluator a facility to support automatic search. This is a much more +profound change to the language than the introduction of lazy +evaluation in section *Note 4-2::. + + Nondeterministic computing, like stream processing, is useful for +"generate and test" applications. Consider the task of starting with +two lists of positive integers and finding a pair of integers--one from +the first list and one from the second list--whose sum is prime. We +saw how to handle this with finite sequence operations in section *Note +2-2-3:: and with infinite streams in section *Note 3-5-3::. Our +approach was to generate the sequence of all possible pairs and filter +these to select the pairs whose sum is prime. Whether we actually +generate the entire sequence of pairs first as in *Note Chapter 2::, or +interleave the generating and filtering as in *Note Chapter 3::, is +immaterial to the essential image of how the computation is organized. + + The nondeterministic approach evokes a different image. Imagine +simply that we choose (in some way) a number from the first list and a +number from the second list and require (using some mechanism) that +their sum be prime. This is expressed by following procedure: + + (define (prime-sum-pair list1 list2) + (let ((a (an-element-of list1)) + (b (an-element-of list2))) + (require (prime? (+ a b))) + (list a b))) + + It might seem as if this procedure merely restates the problem, +rather than specifying a way to solve it. Nevertheless, this is a +legitimate nondeterministic program.(1) + + The key idea here is that expressions in a nondeterministic language +can have more than one possible value. For instance, `an-element-of' +might return any element of the given list. Our nondeterministic +program evaluator will work by automatically choosing a possible value +and keeping track of the choice. If a subsequent requirement is not +met, the evaluator will try a different choice, and it will keep trying +new choices until the evaluation succeeds, or until we run out of +choices. Just as the lazy evaluator freed the programmer from the +details of how values are delayed and forced, the nondeterministic +program evaluator will free the programmer from the details of how +choices are made. + + It is instructive to contrast the different images of time evoked by +nondeterministic evaluation and stream processing. Stream processing +uses lazy evaluation to decouple the time when the stream of possible +answers is assembled from the time when the actual stream elements are +produced. The evaluator supports the illusion that all the possible +answers are laid out before us in a timeless sequence. With +nondeterministic evaluation, an expression represents the exploration +of a set of possible worlds, each determined by a set of choices. Some +of the possible worlds lead to dead ends, while others have useful +values. The nondeterministic program evaluator supports the illusion +that time branches, and that our programs have different possible +execution histories. When we reach a dead end, we can revisit a +previous choice point and proceed along a different branch. + + The nondeterministic program evaluator implemented below is called +the `amb' evaluator because it is based on a new special form called +`amb'. We can type the above definition of `prime-sum-pair' at the +`amb' evaluator driver loop (along with definitions of `prime?', +`an-element-of', and `require') and run the procedure as follows: + + ;;; Amb-Eval input: + (prime-sum-pair '(1 3 5 8) '(20 35 110)) + ;;; Starting a new problem + ;;; Amb-Eval value: + (3 20) + + The value returned was obtained after the evaluator repeatedly chose +elements from each of the lists, until a successful choice was made. + + Section *Note 4-3-1:: introduces `amb' and explains how it supports +nondeterminism through the evaluator's automatic search mechanism. +Section *Note 4-3-2:: presents examples of nondeterministic programs, +and section *Note 4-3-3:: gives the details of how to implement the +`amb' evaluator by modifying the ordinary Scheme evaluator. + +* Menu: + +* 4-3-1:: Amb and Search +* 4-3-2:: Examples of Nondeterministic Programs +* 4-3-3:: Implementing the `Amb' Evaluator + + ---------- Footnotes ---------- + + (1) We assume that we have previously defined a procedure `prime?' +that tests whether numbers are prime. Even with `prime?' defined, the +`prime-sum-pair' procedure may look suspiciously like the unhelpful +"pseudo-Lisp" attempt to define the square-root function, which we +described at the beginning of section *Note 1-1-7::. In fact, a +square-root procedure along those lines can actually be formulated as a +nondeterministic program. By incorporating a search mechanism into the +evaluator, we are eroding the distinction between purely declarative +descriptions and imperative specifications of how to compute answers. +We'll go even farther in this direction in section *Note 4-4::. + + +File: sicp.info, Node: 4-3-1, Next: 4-3-2, Prev: 4-3, Up: 4-3 + +4.3.1 Amb and Search +-------------------- + +To extend Scheme to support nondeterminism, we introduce a new special +form called `amb'.(1) The expression + + (amb ... ) + +returns the value of one of the n expressions "ambiguously." For +example, the expression + + (list (amb 1 2 3) (amb 'a 'b)) + +can have six possible values: + + `(1 a)' `(1 b)' `(2 a)' `(2 b)' `(3 a)' `(3 b)' + + `Amb' with a single choice produces an ordinary (single) value. + + `Amb' with no choices--the expression `(amb)'--is an expression with +no acceptable values. Operationally, we can think of `(amb)' as an +expression that when evaluated causes the computation to "fail": The +computation aborts and no value is produced. Using this idea, we can +express the requirement that a particular predicate expression `p' must +be true as follows: + + (define (require p) + (if (not p) (amb))) + + With `amb' and `require', we can implement the `an-element-of' +procedure used above: + + (define (an-element-of items) + (require (not (null? items))) + (amb (car items) (an-element-of (cdr items)))) + + `An-element-of' fails if the list is empty. Otherwise it ambiguously +returns either the first element of the list or an element chosen from +the rest of the list. + + We can also express infinite ranges of choices. The following +procedure potentially returns any integer greater than or equal to some +given n: + + (define (an-integer-starting-from n) + (amb n (an-integer-starting-from (+ n 1)))) + + This is like the stream procedure `integers-starting-from' described +in section *Note 3-5-2::, but with an important difference: The stream +procedure returns an object that represents the sequence of all +integers beginning with n, whereas the `amb' procedure returns a single +integer.(2) + + Abstractly, we can imagine that evaluating an `amb' expression +causes time to split into branches, where the computation continues on +each branch with one of the possible values of the expression. We say +that `amb' represents a "nondeterministic choice point". If we had a +machine with a sufficient number of processors that could be +dynamically allocated, we could implement the search in a +straightforward way. Execution would proceed as in a sequential +machine, until an `amb' expression is encountered. At this point, more +processors would be allocated and initialized to continue all of the +parallel executions implied by the choice. Each processor would proceed +sequentially as if it were the only choice, until it either terminates +by encountering a failure, or it further subdivides, or it finishes.(3) + + On the other hand, if we have a machine that can execute only one +process (or a few concurrent processes), we must consider the +alternatives sequentially. One could imagine modifying an evaluator to +pick at random a branch to follow whenever it encounters a choice +point. Random choice, however, can easily lead to failing values. We +might try running the evaluator over and over, making random choices +and hoping to find a non-failing value, but it is better to "systematically +search" all possible execution paths. The `amb' evaluator that we will +develop and work with in this section implements a systematic search as +follows: When the evaluator encounters an application of `amb', it +initially selects the first alternative. This selection may itself +lead to a further choice. The evaluator will always initially choose +the first alternative at each choice point. If a choice results in a +failure, then the evaluator automagically(4) "backtracks" to the most +recent choice point and tries the next alternative. If it runs out of +alternatives at any choice point, the evaluator will back up to the +previous choice point and resume from there. This process leads to a +search strategy known as "depth-first search" or backtracking +"chronological backtracking".(5) + +Driver loop +........... + +The driver loop for the `amb' evaluator has some unusual properties. It +reads an expression and prints the value of the first non-failing +execution, as in the `prime-sum-pair' example shown above. If we want +to see the value of the next successful execution, we can ask the +interpreter to backtrack and attempt to generate a second non-failing +execution. This is signaled by typing the symbol `try-again'. If any +expression except `try-again' is given, the interpreter will start a +new problem, discarding the unexplored alternatives in the previous +problem. Here is a sample interaction: + + ;;; Amb-Eval input: + (prime-sum-pair '(1 3 5 8) '(20 35 110)) + ;;; Starting a new problem + ;;; Amb-Eval value: + (3 20) + + ;;; Amb-Eval input: + try-again + ;;; Amb-Eval value: + (3 110) + + ;;; Amb-Eval input: + try-again + ;;; Amb-Eval value: + (8 35) + + ;;; Amb-Eval input: + try-again + ;;; There are no more values of + (prime-sum-pair (quote (1 3 5 8)) (quote (20 35 110))) + + ;;; Amb-Eval input: + (prime-sum-pair '(19 27 30) '(11 36 58)) + ;;; Starting a new problem + ;;; Amb-Eval value: + (30 11) + + *Exercise 4.35:* Write a procedure `an-integer-between' that + returns an integer between two given bounds. This can be used to + implement a procedure that finds Pythagorean triples, i.e., + triples of integers (i,j,k) between the given bounds such that i + <= j and i^2 + j^2 = k^2, as follows: + + (define (a-pythagorean-triple-between low high) + (let ((i (an-integer-between low high))) + (let ((j (an-integer-between i high))) + (let ((k (an-integer-between j high))) + (require (= (+ (* i i) (* j j)) (* k k))) + (list i j k))))) + + *Exercise 4.36:* *Note Exercise 3-69:: discussed how to generate + the stream of _all_ Pythagorean triples, with no upper bound on + the size of the integers to be searched. Explain why simply + replacing `an-integer-between' by `an-integer-starting-from' in + the procedure in *Note Exercise 4-35:: is not an adequate way to + generate arbitrary Pythagorean triples. Write a procedure that + actually will accomplish this. (That is, write a procedure for + which repeatedly typing `try-again' would in principle eventually + generate all Pythagorean triples.) + + *Exercise 4.37:* Ben Bitdiddle claims that the following method + for generating Pythagorean triples is more efficient than the one + in *Note Exercise 4-35::. Is he correct? (Hint: Consider the + number of possibilities that must be explored.) + + (define (a-pythagorean-triple-between low high) + (let ((i (an-integer-between low high)) + (hsq (* high high))) + (let ((j (an-integer-between i high))) + (let ((ksq (+ (* i i) (* j j)))) + (require (>= hsq ksq)) + (let ((k (sqrt ksq))) + (require (integer? k)) + (list i j k)))))) + + ---------- Footnotes ---------- + + (1) The idea of `amb' for nondeterministic programming was first +described in 1961 by John McCarthy (see McCarthy 1967). + + (2) In actuality, the distinction between nondeterministically +returning a single choice and returning all choices depends somewhat on +our point of view. From the perspective of the code that uses the +value, the nondeterministic choice returns a single value. From the +perspective of the programmer designing the code, the nondeterministic +choice potentially returns all possible values, and the computation +branches so that each value is investigated separately. + + (3) One might object that this is a hopelessly inefficient +mechanism. It might require millions of processors to solve some +easily stated problem this way, and most of the time most of those +processors would be idle. This objection should be taken in the +context of history. Memory used to be considered just such an +expensive commodity. In 1964 a megabyte of RAM cost about $400,000. +Now every personal computer has many megabytes of RAM, and most of the +time most of that RAM is unused. It is hard to underestimate the cost +of mass-produced electronics. + + (4) Automagically: "Automatically, but in a way which, for some +reason (typically because it is too complicated, or too ugly, or +perhaps even too trivial), the speaker doesn't feel like explaining." +(Steele 1983, Raymond 1993) + + (5) [Footnote 4.47] The integration of automatic search strategies +into programming languages has had a long and checkered history. The +first suggestions that nondeterministic algorithms might be elegantly +encoded in a programming language with search and automatic +backtracking came from Robert Floyd (1967). Carl Hewitt (1969) +invented a programming language called Planner that explicitly +supported automatic chronological backtracking, providing for a +built-in depth-first search strategy. Sussman, Winograd, and Charniak +(1971) implemented a subset of this language, called MicroPlanner, +which was used to support work in problem solving and robot planning. +Similar ideas, arising from logic and theorem proving, led to the +genesis in Edinburgh and Marseille of the elegant language Prolog +(which we will discuss in section *Note 4-4::). After sufficient +frustration with automatic search, McDermott and Sussman (1972) +developed a language called Conniver, which included mechanisms for +placing the search strategy under programmer control. This proved +unwieldy, however, and Sussman and Stallman (1975) found a more +tractable approach while investigating methods of symbolic analysis for +electrical circuits. They developed a non-chronological backtracking +scheme that was based on tracing out the logical dependencies +connecting facts, a technique that has come to be known as "dependency-directed +backtracking". Although their method was complex, it produced +reasonably efficient programs because it did little redundant search. +Doyle (1979) and McAllester (1978, 1980) generalized and clarified the +methods of Stallman and Sussman, developing a new paradigm for +formulating search that is now called "truth maintenance". Modern +problem-solving systems all use some form of truth-maintenance system +as a substrate. See Forbus and deKleer 1993 for a discussion of +elegant ways to build truth-maintenance systems and applications using +truth maintenance. Zabih, McAllester, and Chapman 1987 describes a +nondeterministic extension to Scheme that is based on `amb'; it is +similar to the interpreter described in this section, but more +sophisticated, because it uses dependency-directed backtracking rather +than chronological backtracking. Winston 1992 gives an introduction to +both kinds of backtracking. + + +File: sicp.info, Node: 4-3-2, Next: 4-3-3, Prev: 4-3-1, Up: 4-3 + +4.3.2 Examples of Nondeterministic Programs +------------------------------------------- + +Section *Note 4-3-3:: describes the implementation of the `amb' +evaluator. First, however, we give some examples of how it can be +used. The advantage of nondeterministic programming is that we can +suppress the details of how search is carried out, thereby expressing +our programs at a higher level of abstraction. + +Logic Puzzles +............. + +The following puzzle (taken from Dinesman 1968) is typical of a large +class of simple logic puzzles: + + Baker, Cooper, Fletcher, Miller, and Smith live on different + floors of an apartment house that contains only five floors. + Baker does not live on the top floor. Cooper does not live on the + bottom floor. Fletcher does not live on either the top or the + bottom floor. Miller lives on a higher floor than does Cooper. + Smith does not live on a floor adjacent to Fletcher's. Fletcher + does not live on a floor adjacent to Cooper's. Where does + everyone live? + + We can determine who lives on each floor in a straightforward way by +enumerating all the possibilities and imposing the given +restrictions:(1) + + (define (multiple-dwelling) + (let ((baker (amb 1 2 3 4 5)) + (cooper (amb 1 2 3 4 5)) + (fletcher (amb 1 2 3 4 5)) + (miller (amb 1 2 3 4 5)) + (smith (amb 1 2 3 4 5))) + (require + (distinct? (list baker cooper fletcher miller smith))) + (require (not (= baker 5))) + (require (not (= cooper 1))) + (require (not (= fletcher 5))) + (require (not (= fletcher 1))) + (require (> miller cooper)) + (require (not (= (abs (- smith fletcher)) 1))) + (require (not (= (abs (- fletcher cooper)) 1))) + (list (list 'baker baker) + (list 'cooper cooper) + (list 'fletcher fletcher) + (list 'miller miller) + (list 'smith smith)))) + + Evaluating the expression `(multiple-dwelling)' produces the result + + ((baker 3) (cooper 2) (fletcher 4) (miller 5) (smith 1)) + + Although this simple procedure works, it is very slow. *Note +Exercise 4-39:: and *Note Exercise 4-40:: discuss some possible +improvements. + + *Exercise 4.38:* Modify the multiple-dwelling procedure to omit + the requirement that Smith and Fletcher do not live on adjacent + floors. How many solutions are there to this modified puzzle? + + *Exercise 4.39:* Does the order of the restrictions in the + multiple-dwelling procedure affect the answer? Does it affect the + time to find an answer? If you think it matters, demonstrate a + faster program obtained from the given one by reordering the + restrictions. If you think it does not matter, argue your case. + + *Exercise 4.40:* In the multiple dwelling problem, how many sets + of assignments are there of people to floors, both before and + after the requirement that floor assignments be distinct? It is + very inefficient to generate all possible assignments of people to + floors and then leave it to backtracking to eliminate them. For + example, most of the restrictions depend on only one or two of the + person-floor variables, and can thus be imposed before floors have + been selected for all the people. Write and demonstrate a much + more efficient nondeterministic procedure that solves this problem + based upon generating only those possibilities that are not already + ruled out by previous restrictions. (Hint: This will require a + nest of `let' expressions.) + + *Exercise 4.41:* Write an ordinary Scheme program to solve the + multiple dwelling puzzle. + + *Exercise 4.42:* Solve the following "Liars" puzzle (from Phillips + 1934): + + Five schoolgirls sat for an examination. Their parents--so they + thought--showed an undue degree of interest in the result. They + therefore agreed that, in writing home about the examination, each + girl should make one true statement and one untrue one. The + following are the relevant passages from their letters: + + * Betty: "Kitty was second in the examination. I was only + third." + + * Ethel: "You'll be glad to hear that I was on top. Joan was + second." + + * Joan: "I was third, and poor old Ethel was bottom." + + * Kitty: "I came out second. Mary was only fourth." + + * Mary: "I was fourth. Top place was taken by Betty." + + + What in fact was the order in which the five girls were placed? + + *Exercise 4.43:* Use the `amb' evaluator to solve the following + puzzle:(2) + + Mary Ann Moore's father has a yacht and so has each of his + four friends: Colonel Downing, Mr. Hall, Sir Barnacle Hood, + and Dr. Parker. Each of the five also has one daughter and + each has named his yacht after a daughter of one of the + others. Sir Barnacle's yacht is the Gabrielle, Mr. Moore + owns the Lorna; Mr. Hall the Rosalind. The Melissa, owned by + Colonel Downing, is named after Sir Barnacle's daughter. + Gabrielle's father owns the yacht that is named after Dr. + Parker's daughter. Who is Lorna's father? + + Try to write the program so that it runs efficiently (see *Note + Exercise 4-40::). Also determine how many solutions there are if + we are not told that Mary Ann's last name is Moore. + + *Exercise 4.44:* *Note Exercise 2-42:: described the "eight-queens + puzzle" of placing queens on a chessboard so that no two attack + each other. Write a nondeterministic program to solve this puzzle. + +Parsing natural language +........................ + +Programs designed to accept natural language as input usually start by +attempting to "parse" the input, that is, to match the input against +some grammatical structure. For example, we might try to recognize +simple sentences consisting of an article followed by a noun followed +by a verb, such as "The cat eats." To accomplish such an analysis, we +must be able to identify the parts of speech of individual words. We +could start with some lists that classify various words:(3) + + (define nouns '(noun student professor cat class)) + + (define verbs '(verb studies lectures eats sleeps)) + + (define articles '(article the a)) + + We also need a "grammar", that is, a set of rules describing how +grammatical elements are composed from simpler elements. A very simple +grammar might stipulate that a sentence always consists of two +pieces--a noun phrase followed by a verb--and that a noun phrase +consists of an article followed by a noun. With this grammar, the +sentence "The cat eats" is parsed as follows: + + (sentence (noun-phrase (article the) (noun cat)) + (verb eats)) + + We can generate such a parse with a simple program that has separate +procedures for each of the grammatical rules. To parse a sentence, we +identify its two constituent pieces and return a list of these two +elements, tagged with the symbol `sentence': + + (define (parse-sentence) + (list 'sentence + (parse-noun-phrase) + (parse-word verbs))) + + A noun phrase, similarly, is parsed by finding an article followed +by a noun: + + (define (parse-noun-phrase) + (list 'noun-phrase + (parse-word articles) + (parse-word nouns))) + + At the lowest level, parsing boils down to repeatedly checking that +the next unparsed word is a member of the list of words for the +required part of speech. To implement this, we maintain a global +variable `*unparsed*', which is the input that has not yet been parsed. +Each time we check a word, we require that `*unparsed*' must be +non-empty and that it should begin with a word from the designated +list. If so, we remove that word from `*unparsed*' and return the word +together with its part of speech (which is found at the head of the +list):(4) + + (define (parse-word word-list) + (require (not (null? *unparsed*))) + (require (memq (car *unparsed*) (cdr word-list))) + (let ((found-word (car *unparsed*))) + (set! *unparsed* (cdr *unparsed*)) + (list (car word-list) found-word))) + + To start the parsing, all we need to do is set `*unparsed*' to be the +entire input, try to parse a sentence, and check that nothing is left +over: + + (define *unparsed* '()) + + (define (parse input) + (set! *unparsed* input) + (let ((sent (parse-sentence))) + (require (null? *unparsed*)) + sent)) + + We can now try the parser and verify that it works for our simple +test sentence: + + ;;; Amb-Eval input: + (parse '(the cat eats)) + ;;; Starting a new problem + ;;; Amb-Eval value: + (sentence (noun-phrase (article the) (noun cat)) (verb eats)) + + The `amb' evaluator is useful here because it is convenient to +express the parsing constraints with the aid of `require'. Automatic +search and backtracking really pay off, however, when we consider more +complex grammars where there are choices for how the units can be +decomposed. + + Let's add to our grammar a list of prepositions: + + (define prepositions '(prep for to in by with)) + +and define a prepositional phrase (e.g., "for the cat") to be a +preposition followed by a noun phrase: + + (define (parse-prepositional-phrase) + (list 'prep-phrase + (parse-word prepositions) + (parse-noun-phrase))) + + Now we can define a sentence to be a noun phrase followed by a verb +phrase, where a verb phrase can be either a verb or a verb phrase +extended by a prepositional phrase:(5) + + (define (parse-sentence) + (list 'sentence + (parse-noun-phrase) + (parse-verb-phrase))) + + (define (parse-verb-phrase) + (define (maybe-extend verb-phrase) + (amb verb-phrase + (maybe-extend (list 'verb-phrase + verb-phrase + (parse-prepositional-phrase))))) + (maybe-extend (parse-word verbs))) + + While we're at it, we can also elaborate the definition of noun +phrases to permit such things as "a cat in the class." What we used to +call a noun phrase, we'll now call a simple noun phrase, and a noun +phrase will now be either a simple noun phrase or a noun phrase +extended by a prepositional phrase: + + (define (parse-simple-noun-phrase) + (list 'simple-noun-phrase + (parse-word articles) + (parse-word nouns))) + + (define (parse-noun-phrase) + (define (maybe-extend noun-phrase) + (amb noun-phrase + (maybe-extend (list 'noun-phrase + noun-phrase + (parse-prepositional-phrase))))) + (maybe-extend (parse-simple-noun-phrase))) + + Our new grammar lets us parse more complex sentences. For example + + (parse '(the student with the cat sleeps in the class)) + +produces + + (sentence + (noun-phrase + (simple-noun-phrase (article the) (noun student)) + (prep-phrase (prep with) + (simple-noun-phrase + (article the) (noun cat)))) + (verb-phrase + (verb sleeps) + (prep-phrase (prep in) + (simple-noun-phrase + (article the) (noun class))))) + + Observe that a given input may have more than one legal parse. In +the sentence "The professor lectures to the student with the cat," it +may be that the professor is lecturing with the cat, or that the +student has the cat. Our nondeterministic program finds both +possibilities: + + (parse '(the professor lectures to the student with the cat)) + +produces + + (sentence + (simple-noun-phrase (article the) (noun professor)) + (verb-phrase + (verb-phrase + (verb lectures) + (prep-phrase (prep to) + (simple-noun-phrase + (article the) (noun student)))) + (prep-phrase (prep with) + (simple-noun-phrase + (article the) (noun cat))))) + + Asking the evaluator to try again yields + + (sentence + (simple-noun-phrase (article the) (noun professor)) + (verb-phrase + (verb lectures) + (prep-phrase (prep to) + (noun-phrase + (simple-noun-phrase + (article the) (noun student)) + (prep-phrase (prep with) + (simple-noun-phrase + (article the) (noun cat))))))) + + *Exercise 4.45:* With the grammar given above, the following + sentence can be parsed in five different ways: "The professor + lectures to the student in the class with the cat." Give the five + parses and explain the differences in shades of meaning among them. + + *Exercise 4.46:* The evaluators in sections *Note 4-1:: and *Note + 4-2:: do not determine what order operands are evaluated in. We + will see that the `amb' evaluator evaluates them from left to + right. Explain why our parsing program wouldn't work if the + operands were evaluated in some other order. + + *Exercise 4.47:* Louis Reasoner suggests that, since a verb phrase + is either a verb or a verb phrase followed by a prepositional + phrase, it would be much more straightforward to define the + procedure `parse-verb-phrase' as follows (and similarly for noun + phrases): + + (define (parse-verb-phrase) + (amb (parse-word verbs) + (list 'verb-phrase + (parse-verb-phrase) + (parse-prepositional-phrase)))) + + Does this work? Does the program's behavior change if we + interchange the order of expressions in the `amb' + + *Exercise 4.48:* Extend the grammar given above to handle more + complex sentences. For example, you could extend noun phrases and + verb phrases to include adjectives and adverbs, or you could + handle compound sentences.(6) + + *Exercise 4.49:* Alyssa P. Hacker is more interested in generating + interesting sentences than in parsing them. She reasons that by + simply changing the procedure `parse-word' so that it ignores the + "input sentence" and instead always succeeds and generates an + appropriate word, we can use the programs we had built for parsing + to do generation instead. Implement Alyssa's idea, and show the + first half-dozen or so sentences generated.(7) + + ---------- Footnotes ---------- + + (1) Our program uses the following procedure to determine if the +elements of a list are distinct: + + (define (distinct? items) + (cond ((null? items) true) + ((null? (cdr items)) true) + ((member (car items) (cdr items)) false) + (else (distinct? (cdr items))))) + + `Member' is like `memq' except that it uses `equal?' instead of +`eq?' to test for equality. + + (2) This is taken from a booklet called "Problematical Recreations," +published in the 1960s by Litton Industries, where it is attributed to +the `Kansas State Engineer'. + + (3) Here we use the convention that the first element of each list +designates the part of speech for the rest of the words in the list. + + (4) Notice that `parse-word' uses `set!' to modify the unparsed +input list. For this to work, our `amb' evaluator must undo the +effects of `set!' operations when it backtracks. + + (5) Observe that this definition is recursive--a verb may be +followed by any number of prepositional phrases. + + (6) This kind of grammar can become arbitrarily complex, but it is +only a toy as far as real language understanding is concerned. Real +natural-language understanding by computer requires an elaborate +mixture of syntactic analysis and interpretation of meaning. On the +other hand, even toy parsers can be useful in supporting flexible +command languages for programs such as information-retrieval systems. +Winston 1992 discusses computational approaches to real language +understanding and also the applications of simple grammars to command +languages. + + (7) Although Alyssa's idea works just fine (and is surprisingly +simple), the sentences that it generates are a bit boring--they don't +sample the possible sentences of this language in a very interesting +way. In fact, the grammar is highly recursive in many places, and +Alyssa's technique "falls into" one of these recursions and gets stuck. +See *Note Exercise 4-50:: for a way to deal with this. + + +File: sicp.info, Node: 4-3-3, Prev: 4-3-2, Up: 4-3 + +4.3.3 Implementing the `Amb' Evaluator +-------------------------------------- + +The evaluation of an ordinary Scheme expression may return a value, may +never terminate, or may signal an error. In nondeterministic Scheme +the evaluation of an expression may in addition result in the discovery +of a dead end, in which case evaluation must backtrack to a previous +choice point. The interpretation of nondeterministic Scheme is +complicated by this extra case. + + We will construct the `amb' evaluator for nondeterministic Scheme by +modifying the analyzing evaluator of section *Note 4-1-7::.(1) As in +the analyzing evaluator, evaluation of an expression is accomplished by +calling an execution procedure produced by analysis of that expression. +The difference between the interpretation of ordinary Scheme and the +interpretation of nondeterministic Scheme will be entirely in the +execution procedures. + +Execution procedures and continuations +...................................... + +Recall that the execution procedures for the ordinary evaluator take one +argument: the environment of execution. In contrast, the execution +procedures in the `amb' evaluator take three arguments: the +environment, and two procedures called "continuation procedures". The +evaluation of an expression will finish by calling one of these two +continuations: If the evaluation results in a value, the "success +continuation" is called with that value; if the evaluation results in +the discovery of a dead end, the "failure continuation" is called. +Constructing and calling appropriate continuations is the mechanism by +which the nondeterministic evaluator implements backtracking. + + It is the job of the success continuation to receive a value and +proceed with the computation. Along with that value, the success +continuation is passed another failure continuation, which is to be +called subsequently if the use of that value leads to a dead end. + + It is the job of the failure continuation to try another branch of +the nondeterministic process. The essence of the nondeterministic +language is in the fact that expressions may represent choices among +alternatives. The evaluation of such an expression must proceed with +one of the indicated alternative choices, even though it is not known +in advance which choices will lead to acceptable results. To deal with +this, the evaluator picks one of the alternatives and passes this value +to the success continuation. Together with this value, the evaluator +constructs and passes along a failure continuation that can be called +later to choose a different alternative. + + A failure is triggered during evaluation (that is, a failure +continuation is called) when a user program explicitly rejects the +current line of attack (for example, a call to `require' may result in +execution of `(amb)', an expression that always fails--see section +*Note 4-3-1::). The failure continuation in hand at that point will +cause the most recent choice point to choose another alternative. If +there are no more alternatives to be considered at that choice point, a +failure at an earlier choice point is triggered, and so on. Failure +continuations are also invoked by the driver loop in response to a +`try-again' request, to find another value of the expression. + + In addition, if a side-effect operation (such as assignment to a +variable) occurs on a branch of the process resulting from a choice, it +may be necessary, when the process finds a dead end, to undo the side +effect before making a new choice. This is accomplished by having the +side-effect operation produce a failure continuation that undoes the +side effect and propagates the failure. + + In summary, failure continuations are constructed by + + * `amb' expressions--to provide a mechanism to make alternative + choices if the current choice made by the `amb' expression leads + to a dead end; + + * the top-level driver--to provide a mechanism to report failure + when the choices are exhausted; + + * assignments--to intercept failures and undo assignments during + backtracking. + + + Failures are initiated only when a dead end is encountered. This +occurs + + * if the user program executes `(amb)'; + + * if the user types `try-again' at the top-level driver. + + + Failure continuations are also called during processing of a failure: + + * When the failure continuation created by an assignment finishes + undoing a side effect, it calls the failure continuation it + intercepted, in order to propagate the failure back to the choice + point that led to this assignment or to the top level. + + * When the failure continuation for an `amb' runs out of choices, it + calls the failure continuation that was originally given to the + `amb', in order to propagate the failure back to the previous + choice point or to the top level. + + +Structure of the evaluator +.......................... + +The syntax- and data-representation procedures for the `amb' evaluator, +and also the basic `analyze' procedure, are identical to those in the +evaluator of section *Note 4-1-7::, except for the fact that we need +additional syntax procedures to recognize the `amb' special form:(2) + + (define (amb? exp) (tagged-list? exp 'amb)) + + (define (amb-choices exp) (cdr exp)) + + We must also add to the dispatch in `analyze' a clause that will +recognize this special form and generate an appropriate execution +procedure: + + ((amb? exp) (analyze-amb exp)) + + The top-level procedure `ambeval' (similar to the version of `eval' +given in section *Note 4-1-7::) analyzes the given expression and +applies the resulting execution procedure to the given environment, +together with two given continuations: + + (define (ambeval exp env succeed fail) + ((analyze exp) env succeed fail)) + + A success continuation is a procedure of two arguments: the value +just obtained and another failure continuation to be used if that value +leads to a subsequent failure. A failure continuation is a procedure of +no arguments. So the general form of an execution procedure is + + (lambda (env succeed fail) + ;; `succeed' is `(lambda (value fail) ...)' + ;; `fail' is `(lambda () ...)' + ...) + + For example, executing + + (ambeval + the-global-environment + (lambda (value fail) value) + (lambda () 'failed)) + +will attempt to evaluate the given expression and will return either the +expression's value (if the evaluation succeeds) or the symbol `failed' +(if the evaluation fails). The call to `ambeval' in the driver loop +shown below uses much more complicated continuation procedures, which +continue the loop and support the `try-again' request. + + Most of the complexity of the `amb' evaluator results from the +mechanics of passing the continuations around as the execution +procedures call each other. In going through the following code, you +should compare each of the execution procedures with the corresponding +procedure for the ordinary evaluator given in section *Note 4-1-7::. + +Simple expressions +.................. + +The execution procedures for the simplest kinds of expressions are +essentially the same as those for the ordinary evaluator, except for +the need to manage the continuations. The execution procedures simply +succeed with the value of the expression, passing along the failure +continuation that was passed to them. + + (define (analyze-self-evaluating exp) + (lambda (env succeed fail) + (succeed exp fail))) + + (define (analyze-quoted exp) + (let ((qval (text-of-quotation exp))) + (lambda (env succeed fail) + (succeed qval fail)))) + + (define (analyze-variable exp) + (lambda (env succeed fail) + (succeed (lookup-variable-value exp env) + fail))) + + (define (analyze-lambda exp) + (let ((vars (lambda-parameters exp)) + (bproc (analyze-sequence (lambda-body exp)))) + (lambda (env succeed fail) + (succeed (make-procedure vars bproc env) + fail)))) + + Notice that looking up a variable always "succeeds." If +`lookup-variable-value' fails to find the variable, it signals an error, +as usual. Such a "failure" indicates a program bug--a reference to an +unbound variable; it is not an indication that we should try another +nondeterministic choice instead of the one that is currently being +tried. + +Conditionals and sequences +.......................... + +Conditionals are also handled in a similar way as in the ordinary +evaluator. The execution procedure generated by `analyze-if' invokes +the predicate execution procedure `pproc' with a success continuation +that checks whether the predicate value is true and goes on to execute +either the consequent or the alternative. If the execution of `pproc' +fails, the original failure continuation for the `if' expression is +called. + + (define (analyze-if exp) + (let ((pproc (analyze (if-predicate exp))) + (cproc (analyze (if-consequent exp))) + (aproc (analyze (if-alternative exp)))) + (lambda (env succeed fail) + (pproc env + ;; success continuation for evaluating the predicate + ;; to obtain `pred-value' + (lambda (pred-value fail2) + (if (true? pred-value) + (cproc env succeed fail2) + (aproc env succeed fail2))) + ;; failure continuation for evaluating the predicate + fail)))) + + Sequences are also handled in the same way as in the previous +evaluator, except for the machinations in the subprocedure +`sequentially' that are required for passing the continuations. +Namely, to sequentially execute `a' and then `b', we call `a' with a +success continuation that calls `b'. + + (define (analyze-sequence exps) + (define (sequentially a b) + (lambda (env succeed fail) + (a env + ;; success continuation for calling `a' + (lambda (a-value fail2) + (b env succeed fail2)) + ;; failure continuation for calling `a' + fail))) + (define (loop first-proc rest-procs) + (if (null? rest-procs) + first-proc + (loop (sequentially first-proc (car rest-procs)) + (cdr rest-procs)))) + (let ((procs (map analyze exps))) + (if (null? procs) + (error "Empty sequence -- ANALYZE")) + (loop (car procs) (cdr procs)))) + +Definitions and assignments +........................... + +Definitions are another case where we must go to some trouble to manage +the continuations, because it is necessary to evaluate the +definition-value expression before actually defining the new variable. +To accomplish this, the definition-value execution procedure `vproc' is +called with the environment, a success continuation, and the failure +continuation. If the execution of `vproc' succeeds, obtaining a value +`val' for the defined variable, the variable is defined and the success +is propagated: + + (define (analyze-definition exp) + (let ((var (definition-variable exp)) + (vproc (analyze (definition-value exp)))) + (lambda (env succeed fail) + (vproc env + (lambda (val fail2) + (define-variable! var val env) + (succeed 'ok fail2)) + fail)))) + + Assignments are more interesting. This is the first place where we +really use the continuations, rather than just passing them around. +The execution procedure for assignments starts out like the one for +definitions. It first attempts to obtain the new value to be assigned +to the variable. If this evaluation of `vproc' fails, the assignment +fails. + + If `vproc' succeeds, however, and we go on to make the assignment, +we must consider the possibility that this branch of the computation +might later fail, which will require us to backtrack out of the +assignment. Thus, we must arrange to undo the assignment as part of +the backtracking process.(3) + + This is accomplished by giving `vproc' a success continuation +(marked with the comment "*1*" below) that saves the old value of the +variable before assigning the new value to the variable and proceeding +from the assignment. The failure continuation that is passed along +with the value of the assignment (marked with the comment "*2*" below) +restores the old value of the variable before continuing the failure. +That is, a successful assignment provides a failure continuation that +will intercept a subsequent failure; whatever failure would otherwise +have called `fail2' calls this procedure instead, to undo the +assignment before actually calling `fail2'. + + (define (analyze-assignment exp) + (let ((var (assignment-variable exp)) + (vproc (analyze (assignment-value exp)))) + (lambda (env succeed fail) + (vproc env + (lambda (val fail2) ; *1* + (let ((old-value + (lookup-variable-value var env))) + (set-variable-value! var val env) + (succeed 'ok + (lambda () ; *2* + (set-variable-value! var + old-value + env) + (fail2))))) + fail)))) + +Procedure applications +...................... + +The execution procedure for applications contains no new ideas except +for the technical complexity of managing the continuations. This +complexity arises in `analyze-application', due to the need to keep +track of the success and failure continuations as we evaluate the +operands. We use a procedure `get-args' to evaluate the list of +operands, rather than a simple `map' as in the ordinary evaluator. + + (define (analyze-application exp) + (let ((fproc (analyze (operator exp))) + (aprocs (map analyze (operands exp)))) + (lambda (env succeed fail) + (fproc env + (lambda (proc fail2) + (get-args aprocs + env + (lambda (args fail3) + (execute-application + proc args succeed fail3)) + fail2)) + fail)))) + + In `get-args', notice how `cdr'ing down the list of `aproc' +execution procedures and `cons'ing up the resulting list of `args' is +accomplished by calling each `aproc' in the list with a success +continuation that recursively calls `get-args'. Each of these recursive +calls to `get-args' has a success continuation whose value is the +`cons' of the newly obtained argument onto the list of accumulated +arguments: + + (define (get-args aprocs env succeed fail) + (if (null? aprocs) + (succeed '() fail) + ((car aprocs) env + ;; success continuation for this `aproc' + (lambda (arg fail2) + (get-args (cdr aprocs) + env + ;; success continuation for recursive + ;; call to `get-args' + (lambda (args fail3) + (succeed (cons arg args) + fail3)) + fail2)) + fail))) + + The actual procedure application, which is performed by +`execute-application', is accomplished in the same way as for the +ordinary evaluator, except for the need to manage the continuations. + + (define (execute-application proc args succeed fail) + (cond ((primitive-procedure? proc) + (succeed (apply-primitive-procedure proc args) + fail)) + ((compound-procedure? proc) + ((procedure-body proc) + (extend-environment (procedure-parameters proc) + args + (procedure-environment proc)) + succeed + fail)) + (else + (error + "Unknown procedure type -- EXECUTE-APPLICATION" + proc)))) + +Evaluating `amb' expressions +............................ + +The `amb' special form is the key element in the nondeterministic +language. Here we see the essence of the interpretation process and +the reason for keeping track of the continuations. The execution +procedure for `amb' defines a loop `try-next' that cycles through the +execution procedures for all the possible values of the `amb' +expression. Each execution procedure is called with a failure +continuation that will try the next one. When there are no more +alternatives to try, the entire `amb' expression fails. + + (define (analyze-amb exp) + (let ((cprocs (map analyze (amb-choices exp)))) + (lambda (env succeed fail) + (define (try-next choices) + (if (null? choices) + (fail) + ((car choices) env + succeed + (lambda () + (try-next (cdr choices)))))) + (try-next cprocs)))) + +Driver loop +........... + +The driver loop for the `amb' evaluator is complex, due to the mechanism +that permits the user to try again in evaluating an expression. The +driver uses a procedure called `internal-loop', which takes as argument +a procedure `try-again'. The intent is that calling `try-again' should +go on to the next untried alternative in the nondeterministic +evaluation. `Internal-loop' either calls `try-again' in response to +the user typing `try-again' at the driver loop, or else starts a new +evaluation by calling `ambeval'. + + The failure continuation for this call to `ambeval' informs the user +that there are no more values and re-invokes the driver loop. + + The success continuation for the call to `ambeval' is more subtle. +We print the obtained value and then invoke the internal loop again +with a `try-again' procedure that will be able to try the next +alternative. This `next-alternative' procedure is the second argument +that was passed to the success continuation. Ordinarily, we think of +this second argument as a failure continuation to be used if the +current evaluation branch later fails. In this case, however, we have +completed a successful evaluation, so we can invoke the "failure" +alternative branch in order to search for additional successful +evaluations. + + (define input-prompt ";;; Amb-Eval input:") + (define output-prompt ";;; Amb-Eval value:") + + (define (driver-loop) + (define (internal-loop try-again) + (prompt-for-input input-prompt) + (let ((input (read))) + (if (eq? input 'try-again) + (try-again) + (begin + (newline) + (display ";;; Starting a new problem ") + (ambeval input + the-global-environment + ;; `ambeval' success + (lambda (val next-alternative) + (announce-output output-prompt) + (user-print val) + (internal-loop next-alternative)) + ;; `ambeval' failure + (lambda () + (announce-output + ";;; There are no more values of") + (user-print input) + (driver-loop))))))) + (internal-loop + (lambda () + (newline) + (display ";;; There is no current problem") + (driver-loop)))) + + The initial call to `internal-loop' uses a `try-again' procedure that +complains that there is no current problem and restarts the driver +loop. This is the behavior that will happen if the user types +`try-again' when there is no evaluation in progress. + + *Exercise 4.50:* Implement a new special form `ramb' that is like + `amb' except that it searches alternatives in a random order, + rather than from left to right. Show how this can help with + Alyssa's problem in *Note Exercise 4-49::. + + *Exercise 4.51:* Implement a new kind of assignment called + `permanent-set!' that is not undone upon failure. For example, we + can choose two distinct elements from a list and count the number + of trials required to make a successful choice as follows: + + (define count 0) + + (let ((x (an-element-of '(a b c))) + (y (an-element-of '(a b c)))) + (permanent-set! count (+ count 1)) + (require (not (eq? x y))) + (list x y count)) + ;;; Starting a new problem + ;;; Amb-Eval value: + (a b 2) + + ;;; Amb-Eval input: + try-again + ;;; Amb-Eval value: + (a c 3) + + What values would have been displayed if we had used `set!' here + rather than `permanent-set!' ? + + *Exercise 4.52:* Implement a new construct called `if-fail' that + permits the user to catch the failure of an expression. `If-fail' + takes two expressions. It evaluates the first expression as usual + and returns as usual if the evaluation succeeds. If the evaluation + fails, however, the value of the second expression is returned, as + in the following example: + + ;;; Amb-Eval input: + (if-fail (let ((x (an-element-of '(1 3 5)))) + (require (even? x)) + x) + 'all-odd) + ;;; Starting a new problem + ;;; Amb-Eval value: + all-odd + + ;;; Amb-Eval input: + (if-fail (let ((x (an-element-of '(1 3 5 8)))) + (require (even? x)) + x) + 'all-odd) + ;;; Starting a new problem + ;;; Amb-Eval value: + 8 + + *Exercise 4.53:* With `permanent-set!' as described in *Note + Exercise 4-51:: and `if-fail' as in *Note Exercise 4-52::, what + will be the result of evaluating + + (let ((pairs '())) + (if-fail (let ((p (prime-sum-pair '(1 3 5 8) '(20 35 110)))) + (permanent-set! pairs (cons p pairs)) + (amb)) + pairs)) + + *Exercise 4.54:* If we had not realized that `require' could be + implemented as an ordinary procedure that uses `amb', to be + defined by the user as part of a nondeterministic program, we + would have had to implement it as a special form. This would + require syntax procedures + + (define (require? exp) (tagged-list? exp 'require)) + + (define (require-predicate exp) (cadr exp)) + + and a new clause in the dispatch in `analyze' + + ((require? exp) (analyze-require exp)) + + as well the procedure `analyze-require' that handles `require' + expressions. Complete the following definition of + `analyze-require'. + + (define (analyze-require exp) + (let ((pproc (analyze (require-predicate exp)))) + (lambda (env succeed fail) + (pproc env + (lambda (pred-value fail2) + (if + + (succeed 'ok fail2))) + fail)))) + + ---------- Footnotes ---------- + + (1) We chose to implement the lazy evaluator in section *Note 4-2:: +as a modification of the ordinary metacircular evaluator of section +*Note 4-1-1::. In contrast, we will base the `amb' evaluator on the +analyzing evaluator of section *Note 4-1-7::, because the execution +procedures in that evaluator provide a convenient framework for +implementing backtracking. + + (2) We assume that the evaluator supports `let' (see *Note Exercise +4-22::), which we have used in our nondeterministic programs. + + (3) We didn't worry about undoing definitions, since we can assume +that internal definitions are scanned out (section *Note 4-1-6::). + + +File: sicp.info, Node: 4-4, Prev: 4-3, Up: Chapter 4 + +4.4 Logic Programming +===================== + +In *Note Chapter 1:: we stressed that computer science deals with +imperative (how to) knowledge, whereas mathematics deals with +declarative (what is) knowledge. Indeed, programming languages require +that the programmer express knowledge in a form that indicates the +step-by-step methods for solving particular problems. On the other +hand, high-level languages provide, as part of the language +implementation, a substantial amount of methodological knowledge that +frees the user from concern with numerous details of how a specified +computation will progress. + + Most programming languages, including Lisp, are organized around +computing the values of mathematical functions. Expression-oriented +languages (such as Lisp, Fortran, and Algol) capitalize on the "pun" +that an expression that describes the value of a function may also be +interpreted as a means of computing that value. Because of this, most +programming languages are strongly biased toward unidirectional +computations (computations with well-defined inputs and outputs). +There are, however, radically different programming languages that +relax this bias. We saw one such example in section *Note 3-3-5::, +where the objects of computation were arithmetic constraints. In a +constraint system the direction and the order of computation are not so +well specified; in carrying out a computation the system must therefore +provide more detailed "how to" knowledge than would be the case with an +ordinary arithmetic computation. This does not mean, however, that the +user is released altogether from the responsibility of providing +imperative knowledge. There are many constraint networks that +implement the same set of constraints, and the user must choose from +the set of mathematically equivalent networks a suitable network to +specify a particular computation. + + The nondeterministic program evaluator of section *Note 4-3:: also +moves away from the view that programming is about constructing +algorithms for computing unidirectional functions. In a +nondeterministic language, expressions can have more than one value, +and, as a result, the computation is dealing with relations rather than +with single-valued functions. Logic programming extends this idea by +combining a relational vision of programming with a powerful kind of +symbolic pattern matching called "unification".(1) + + This approach, when it works, can be a very powerful way to write +programs. Part of the power comes from the fact that a single "what +is" fact can be used to solve a number of different problems that would +have different "how to" components. As an example, consider the +`append' operation, which takes two lists as arguments and combines +their elements to form a single list. In a procedural language such as +Lisp, we could define `append' in terms of the basic list constructor +`cons', as we did in section *Note 2-2-1::: + + (define (append x y) + (if (null? x) + y + (cons (car x) (append (cdr x) y)))) + + This procedure can be regarded as a translation into Lisp of the +following two rules, the first of which covers the case where the first +list is empty and the second of which handles the case of a nonempty +list, which is a `cons' of two parts: + + * For any list `y', the empty list and `y' `append' to form `y'. + + * For any `u', `v', `y', and `z', `(cons u v)' and `y' `append' to + form `(cons u z)' if `v' and `y' `append' to form `z'.(2) + + + Using the `append' procedure, we can answer questions such as + + Find the `append' of `(a b)' and `(c d)'. + + But the same two rules are also sufficient for answering the +following sorts of questions, which the procedure can't answer: + + Find a list `y' that `append's with `(a b)' to produce `(a b c d)'. + + Find all `x' and `y' that `append' to form `(a b c d)'. + + In a logic programming language, the programmer writes an `append' +"procedure" by stating the two rules about `append' given above. "How +to" knowledge is provided automatically by the interpreter to allow this +single pair of rules to be used to answer all three types of questions +about `append'.(3) + + Contemporary logic programming languages (including the one we +implement here) have substantial deficiencies, in that their general +"how to" methods can lead them into spurious infinite loops or other +undesirable behavior. Logic programming is an active field of research +in computer science.(4) + + Earlier in this chapter we explored the technology of implementing +interpreters and described the elements that are essential to an +interpreter for a Lisp-like language (indeed, to an interpreter for any +conventional language). Now we will apply these ideas to discuss an +interpreter for a logic programming language. We call this language +the "query language", because it is very useful for retrieving +information from data bases by formulating "queries", or questions, +expressed in the language. Even though the query language is very +different from Lisp, we will find it convenient to describe the +language in terms of the same general framework we have been using all +along: as a collection of primitive elements, together with means of +combination that enable us to combine simple elements to create more +complex elements and means of abstraction that enable us to regard +complex elements as single conceptual units. An interpreter for a +logic programming language is considerably more complex than an +interpreter for a language like Lisp. Nevertheless, we will see that +our query-language interpreter contains many of the same elements found +in the interpreter of section *Note 4-1::. In particular, there will +be an "eval" part that classifies expressions according to type and an +"apply" part that implements the language's abstraction mechanism +(procedures in the case of Lisp, and "rules" in the case of logic +programming). Also, a central role is played in the implementation by +a frame data structure, which determines the correspondence between +symbols and their associated values. One additional interesting aspect +of our query-language implementation is that we make substantial use of +streams, which were introduced in *Note Chapter 3::. + +* Menu: + +* 4-4-1:: Deductive Information Retrieval +* 4-4-2:: How the Query System Works +* 4-4-3:: Is Logic Programming Mathematical Logic? +* 4-4-4:: Implementing the Query System + + ---------- Footnotes ---------- + + (1) Logic programming has grown out of a long history of research in +automatic theorem proving. Early theorem-proving programs could +accomplish very little, because they exhaustively searched the space of +possible proofs. The major breakthrough that made such a search +plausible was the discovery in the early 1960s of the "unification +algorithm" and the principle "resolution principle" (Robinson 1965). +Resolution was used, for example, by Green and Raphael (1968) (see also +Green 1969) as the basis for a deductive question-answering system. +During most of this period, researchers concentrated on algorithms that +are guaranteed to find a proof if one exists. Such algorithms were +difficult to control and to direct toward a proof. Hewitt (1969) +recognized the possibility of merging the control structure of a +programming language with the operations of a logic-manipulation system, +leading to the work in automatic search mentioned in section *Note +4-3-1:: (footnote *Note Footnote 4-47::). At the same time that this +was being done, Colmerauer, in Marseille, was developing rule-based +systems for manipulating natural language (see Colmerauer et al. 1973). +He invented a programming language called Prolog for representing +those rules. Kowalski (1973; 1979), in Edinburgh, recognized that +execution of a Prolog program could be interpreted as proving theorems +(using a proof technique called linear Horn-clause resolution). The +merging of the last two strands led to the logic-programming movement. +Thus, in assigning credit for the development of logic programming, the +French can point to Prolog's genesis at the University of Marseille, +while the British can highlight the work at the University of +Edinburgh. According to people at MIT, logic programming was developed +by these groups in an attempt to figure out what Hewitt was talking +about in his brilliant but impenetrable Ph.D. thesis. For a history of +logic programming, see Robinson 1983. + + (2) To see the correspondence between the rules and the procedure, +let `x' in the procedure (where `x' is nonempty) correspond to `(cons u +v)' in the rule. Then `z' in the rule corresponds to the `append' of +`(cdr x)' and `y'. + + (3) This certainly does not relieve the user of the entire problem +of how to compute the answer. There are many different mathematically +equivalent sets of rules for formulating the `append' relation, only +some of which can be turned into effective devices for computing in any +direction. In addition, sometimes "what is" information gives no clue +"how to" compute an answer. For example, consider the problem of +computing the y such that y^2 = x. + + (4) Interest in logic programming peaked during the early 80s when +the Japanese government began an ambitious project aimed at building +superfast computers optimized to run logic programming languages. The +speed of such computers was to be measured in LIPS (Logical Inferences +Per Second) rather than the usual FLOPS (FLoating-point Operations Per +Second). Although the project succeeded in developing hardware and +software as originally planned, the international computer industry +moved in a different direction. See Feigenbaum and Shrobe 1993 for an +overview evaluation of the Japanese project. The logic programming +community has also moved on to consider relational programming based on +techniques other than simple pattern matching, such as the ability to +deal with numerical constraints such as the ones illustrated in the +constraint-propagation system of section *Note 3-3-5::. + + +File: sicp.info, Node: 4-4-1, Next: 4-4-2, Prev: 4-4, Up: 4-4 + +4.4.1 Deductive Information Retrieval +------------------------------------- + +Logic programming excels in providing interfaces to data bases for +information retrieval. The query language we shall implement in this +chapter is designed to be used in this way. + + In order to illustrate what the query system does, we will show how +it can be used to manage the data base of personnel records for +Microshaft, a thriving high-technology company in the Boston area. The +language provides pattern-directed access to personnel information and +can also take advantage of general rules in order to make logical +deductions. + +A sample data base +.................. + +The personnel data base for Microshaft contains "assertions" about +company personnel. Here is the information about Ben Bitdiddle, the +resident computer wizard: + + (address (Bitdiddle Ben) (Slumerville (Ridge Road) 10)) + (job (Bitdiddle Ben) (computer wizard)) + (salary (Bitdiddle Ben) 60000) + + Each assertion is a list (in this case a triple) whose elements can +themselves be lists. + + As resident wizard, Ben is in charge of the company's computer +division, and he supervises two programmers and one technician. Here +is the information about them: + + (address (Hacker Alyssa P) (Cambridge (Mass Ave) 78)) + (job (Hacker Alyssa P) (computer programmer)) + (salary (Hacker Alyssa P) 40000) + (supervisor (Hacker Alyssa P) (Bitdiddle Ben)) + + (address (Fect Cy D) (Cambridge (Ames Street) 3)) + (job (Fect Cy D) (computer programmer)) + (salary (Fect Cy D) 35000) + (supervisor (Fect Cy D) (Bitdiddle Ben)) + + (address (Tweakit Lem E) (Boston (Bay State Road) 22)) + (job (Tweakit Lem E) (computer technician)) + (salary (Tweakit Lem E) 25000) + (supervisor (Tweakit Lem E) (Bitdiddle Ben)) + + There is also a programmer trainee, who is supervised by Alyssa: + + (address (Reasoner Louis) (Slumerville (Pine Tree Road) 80)) + (job (Reasoner Louis) (computer programmer trainee)) + (salary (Reasoner Louis) 30000) + (supervisor (Reasoner Louis) (Hacker Alyssa P)) + + All of these people are in the computer division, as indicated by +the word `computer' as the first item in their job descriptions. + + Ben is a high-level employee. His supervisor is the company's big +wheel himself: + + (supervisor (Bitdiddle Ben) (Warbucks Oliver)) + + (address (Warbucks Oliver) (Swellesley (Top Heap Road))) + (job (Warbucks Oliver) (administration big wheel)) + (salary (Warbucks Oliver) 150000) + + Besides the computer division supervised by Ben, the company has an +accounting division, consisting of a chief accountant and his assistant: + + (address (Scrooge Eben) (Weston (Shady Lane) 10)) + (job (Scrooge Eben) (accounting chief accountant)) + (salary (Scrooge Eben) 75000) + (supervisor (Scrooge Eben) (Warbucks Oliver)) + + (address (Cratchet Robert) (Allston (N Harvard Street) 16)) + (job (Cratchet Robert) (accounting scrivener)) + (salary (Cratchet Robert) 18000) + (supervisor (Cratchet Robert) (Scrooge Eben)) + + There is also a secretary for the big wheel: + + (address (Aull DeWitt) (Slumerville (Onion Square) 5)) + (job (Aull DeWitt) (administration secretary)) + (salary (Aull DeWitt) 25000) + (supervisor (Aull DeWitt) (Warbucks Oliver)) + + The data base also contains assertions about which kinds of jobs can +be done by people holding other kinds of jobs. For instance, a +computer wizard can do the jobs of both a computer programmer and a +computer technician: + + (can-do-job (computer wizard) (computer programmer)) + (can-do-job (computer wizard) (computer technician)) + + A computer programmer could fill in for a trainee: + + (can-do-job (computer programmer) + (computer programmer trainee)) + + Also, as is well known, + + (can-do-job (administration secretary) + (administration big wheel)) + +Simple queries +.............. + +The query language allows users to retrieve information from the data +base by posing queries in response to the system's prompt. For +example, to find all computer programmers one can say + + ;;; Query input: + (job ?x (computer programmer)) + + The system will respond with the following items: + + ;;; Query results: + (job (Hacker Alyssa P) (computer programmer)) + (job (Fect Cy D) (computer programmer)) + + The input query specifies that we are looking for entries in the +data base that match a certain "pattern". In this example, the pattern +specifies entries consisting of three items, of which the first is the +literal symbol `job', the second can be anything, and the third is the +literal list `(computer programmer)'. The "anything" that can be the +second item in the matching list is specified by a "pattern variable", +`?x'. The general form of a pattern variable is a symbol, taken to be +the name of the variable, preceded by a question mark. We will see +below why it is useful to specify names for pattern variables rather +than just putting `?' into patterns to represent "anything." The +system responds to a simple query by showing all entries in the data +base that match the specified pattern. + + A pattern can have more than one variable. For example, the query + + (address ?x ?y) + +will list all the employees' addresses. + + A pattern can have no variables, in which case the query simply +determines whether that pattern is an entry in the data base. If so, +there will be one match; if not, there will be no matches. + + The same pattern variable can appear more than once in a query, +specifying that the same "anything" must appear in each position. This +is why variables have names. For example, + + (supervisor ?x ?x) + +finds all people who supervise themselves (though there are no such +assertions in our sample data base). + + The query + + (job ?x (computer ?type)) + +matches all job entries whose third item is a two-element list whose +first item is `computer': + + (job (Bitdiddle Ben) (computer wizard)) + (job (Hacker Alyssa P) (computer programmer)) + (job (Fect Cy D) (computer programmer)) + (job (Tweakit Lem E) (computer technician)) + + This same pattern does _not_ match + + (job (Reasoner Louis) (computer programmer trainee)) + +because the third item in the entry is a list of three elements, and the +pattern's third item specifies that there should be two elements. If +we wanted to change the pattern so that the third item could be any +list beginning with `computer', we could specify(1) + + (job ?x (computer . ?type)) + + For example, + + (computer . ?type) + +matches the data + + (computer programmer trainee) + +with `?type' as the list `(programmer trainee)'. It also matches the +data + + (computer programmer) + +with `?type' as the list `(programmer)', and matches the data + + (computer) + +with `?type' as the empty list `()'. + + We can describe the query language's processing of simple queries as +follows: + + * The system finds all assignments to variables in the query pattern + that "satisfy" the pattern--that is, all sets of values for the + variables such that if the pattern variables are "instantiated + with" (replaced by) the values, the result is in the data base. + + * The system responds to the query by listing all instantiations of + the query pattern with the variable assignments that satisfy it. + + + Note that if the pattern has no variables, the query reduces to a +determination of whether that pattern is in the data base. If so, the +empty assignment, which assigns no values to variables, satisfies that +pattern for that data base. + + *Exercise 4.55:* Give simple queries that retrieve the following + information from the data base: + + 1. all people supervised by Ben Bitdiddle; + + 2. the names and jobs of all people in the accounting division; + + 3. the names and addresses of all people who live in Slumerville. + + +Compound queries +................ + +Simple queries form the primitive operations of the query language. In +order to form compound operations, the query language provides means of +combination. One thing that makes the query language a logic +programming language is that the means of combination mirror the means +of combination used in forming logical expressions: `and', `or', and +`not'. (Here `and', `or', and `not' are not the Lisp primitives, but +rather operations built into the query language.) + + We can use `and' as follows to find the addresses of all the computer +programmers: + + (and (job ?person (computer programmer)) + (address ?person ?where)) + + The resulting output is + + (and (job (Hacker Alyssa P) (computer programmer)) + (address (Hacker Alyssa P) (Cambridge (Mass Ave) 78))) + + (and (job (Fect Cy D) (computer programmer)) + (address (Fect Cy D) (Cambridge (Ames Street) 3))) + + In general, + + (and ... ) + +is satisfied by all sets of values for the pattern variables that +simultaneously satisfy ... . + + As for simple queries, the system processes a compound query by +finding all assignments to the pattern variables that satisfy the +query, then displaying instantiations of the query with those values. + + Another means of constructing compound queries is through `or'. For +example, + + (or (supervisor ?x (Bitdiddle Ben)) + (supervisor ?x (Hacker Alyssa P))) + +will find all employees supervised by Ben Bitdiddle or Alyssa P. +Hacker: + + (or (supervisor (Hacker Alyssa P) (Bitdiddle Ben)) + (supervisor (Hacker Alyssa P) (Hacker Alyssa P))) + + (or (supervisor (Fect Cy D) (Bitdiddle Ben)) + (supervisor (Fect Cy D) (Hacker Alyssa P))) + + (or (supervisor (Tweakit Lem E) (Bitdiddle Ben)) + (supervisor (Tweakit Lem E) (Hacker Alyssa P))) + + (or (supervisor (Reasoner Louis) (Bitdiddle Ben)) + (supervisor (Reasoner Louis) (Hacker Alyssa P))) + + In general, + + (or ... ) + +is satisfied by all sets of values for the pattern variables that +satisfy at least one of ... . + + Compound queries can also be formed with `not'. For example, + + (and (supervisor ?x (Bitdiddle Ben)) + (not (job ?x (computer programmer)))) + +finds all people supervised by Ben Bitdiddle who are not computer +programmers. In general, + + (not ) + +is satisfied by all assignments to the pattern variables that do not +satisfy .(2) + + The final combining form is called `lisp-value'. When `lisp-value' +is the first element of a pattern, it specifies that the next element +is a Lisp predicate to be applied to the rest of the (instantiated) +elements as arguments. In general, + + (lisp-value ... ) + +will be satisfied by assignments to the pattern variables for which the + applied to the instantiated ... is true. +For example, to find all people whose salary is greater than $30,000 we +could write(3) + + (and (salary ?person ?amount) + (lisp-value > ?amount 30000)) + + *Exercise 4.56:* Formulate compound queries that retrieve the + following information: + + a. the names of all people who are supervised by Ben Bitdiddle, + together with their addresses; + + b. all people whose salary is less than Ben Bitdiddle's, + together with their salary and Ben Bitdiddle's salary; + + c. all people who are supervised by someone who is not in the + computer division, together with the supervisor's name and + job. + + +Rules +..... + +In addition to primitive queries and compound queries, the query +language provides means for abstracting queries. These are given by "rules". +The rule + + (rule (lives-near ?person-1 ?person-2) + (and (address ?person-1 (?town . ?rest-1)) + (address ?person-2 (?town . ?rest-2)) + (not (same ?person-1 ?person-2)))) + +specifies that two people live near each other if they live in the same +town. The final `not' clause prevents the rule from saying that all +people live near themselves. The `same' relation is defined by a very +simple rule:(4) + + (rule (same ?x ?x)) + + The following rule declares that a person is a "wheel" in an +organization if he supervises someone who is in turn a supervisor: + + (rule (wheel ?person) + (and (supervisor ?middle-manager ?person) + (supervisor ?x ?middle-manager))) + + The general form of a rule is + + (rule ) + +where is a pattern and is any query.(5) We can +think of a rule as representing a large (even infinite) set of +assertions, namely all instantiations of the rule conclusion with +variable assignments that satisfy the rule body. When we described +simple queries (patterns), we said that an assignment to variables +satisfies a pattern if the instantiated pattern is in the data base. +But the pattern needn't be explicitly in the data base as an assertion. +It can be an implicit assertion implied by a rule. For example, the +query + + (lives-near ?x (Bitdiddle Ben)) + +results in + + (lives-near (Reasoner Louis) (Bitdiddle Ben)) + (lives-near (Aull DeWitt) (Bitdiddle Ben)) + + To find all computer programmers who live near Ben Bitdiddle, we can +ask + + (and (job ?x (computer programmer)) + (lives-near ?x (Bitdiddle Ben))) + + As in the case of compound procedures, rules can be used as parts of +other rules (as we saw with the `lives-near' rule above) or even be +defined recursively. For instance, the rule + + (rule (outranked-by ?staff-person ?boss) + (or (supervisor ?staff-person ?boss) + (and (supervisor ?staff-person ?middle-manager) + (outranked-by ?middle-manager ?boss)))) + +says that a staff person is outranked by a boss in the organization if +the boss is the person's supervisor or (recursively) if the person's +supervisor is outranked by the boss. + + *Exercise 4.57:* Define a rule that says that person 1 can replace + person 2 if either person 1 does the same job as person 2 or + someone who does person 1's job can also do person 2's job, and if + person 1 and person 2 are not the same person. Using your rule, + give queries that find the following: + + a. all people who can replace Cy D. Fect; + + b. all people who can replace someone who is being paid more + than they are, together with the two salaries. + + + *Exercise 4.58:* Define a rule that says that a person is a "big + shot" in a division if the person works in the division but does + not have a supervisor who works in the division. + + *Exercise 4.59:* Ben Bitdiddle has missed one meeting too many. + Fearing that his habit of forgetting meetings could cost him his + job, Ben decides to do something about it. He adds all the weekly + meetings of the firm to the Microshaft data base by asserting the + following: + + (meeting accounting (Monday 9am)) + (meeting administration (Monday 10am)) + (meeting computer (Wednesday 3pm)) + (meeting administration (Friday 1pm)) + + Each of the above assertions is for a meeting of an entire + division. Ben also adds an entry for the company-wide meeting + that spans all the divisions. All of the company's employees + attend this meeting. + + (meeting whole-company (Wednesday 4pm)) + + a. On Friday morning, Ben wants to query the data base for all + the meetings that occur that day. What query should he use? + + b. Alyssa P. Hacker is unimpressed. She thinks it would be much + more useful to be able to ask for her meetings by specifying + her name. So she designs a rule that says that a person's + meetings include all `whole-company' meetings plus all + meetings of that person's division. Fill in the body of + Alyssa's rule. + + (rule (meeting-time ?person ?day-and-time) + ) + + c. Alyssa arrives at work on Wednesday morning and wonders what + meetings she has to attend that day. Having defined the + above rule, what query should she make to find this out? + + + *Exercise 4.60:* By giving the query + + (lives-near ?person (Hacker Alyssa P)) + + Alyssa P. Hacker is able to find people who live near her, with + whom she can ride to work. On the other hand, when she tries to + find all pairs of people who live near each other by querying + + (lives-near ?person-1 ?person-2) + + she notices that each pair of people who live near each other is + listed twice; for example, + + (lives-near (Hacker Alyssa P) (Fect Cy D)) + (lives-near (Fect Cy D) (Hacker Alyssa P)) + + Why does this happen? Is there a way to find a list of people who + live near each other, in which each pair appears only once? + Explain. + +Logic as programs +................. + +We can regard a rule as a kind of logical implication: _If_ an +assignment of values to pattern variables satisfies the body, _then_ it +satisfies the conclusion. Consequently, we can regard the query +language as having the ability to perform "logical deductions" based +upon the rules. As an example, consider the `append' operation +described at the beginning of section *Note 4-4::. As we said, +`append' can be characterized by the following two rules: + + * For any list `y', the empty list and `y' `append' to form `y'. + + * For any `u', `v', `y', and `z', `(cons u v)' and `y' `append' to + form `(cons u z)' if `v' and `y' `append' to form `z'. + + + To express this in our query language, we define two rules for a +relation + + (append-to-form x y z) + +which we can interpret to mean "`x' and `y' `append' to form `z'": + + (rule (append-to-form () ?y ?y)) + + (rule (append-to-form (?u . ?v) ?y (?u . ?z)) + (append-to-form ?v ?y ?z)) + + The first rule has no body, which means that the conclusion holds +for any value of `?y'. Note how the second rule makes use of +dotted-tail notation to name the `car' and `cdr' of a list. + + Given these two rules, we can formulate queries that compute the +`append' of two lists: + + ;;; Query input: + (append-to-form (a b) (c d) ?z) + ;;; Query results: + (append-to-form (a b) (c d) (a b c d)) + + What is more striking, we can use the same rules to ask the question +"Which list, when `append'ed to `(a b)', yields `(a b c d)'?" This is +done as follows: + + ;;; Query input: + (append-to-form (a b) ?y (a b c d)) + ;;; Query results: + (append-to-form (a b) (c d) (a b c d)) + + We can also ask for all pairs of lists that `append' to form `(a b c +d)': + + ;;; Query input: + (append-to-form ?x ?y (a b c d)) + ;;; Query results: + (append-to-form () (a b c d) (a b c d)) + (append-to-form (a) (b c d) (a b c d)) + (append-to-form (a b) (c d) (a b c d)) + (append-to-form (a b c) (d) (a b c d)) + (append-to-form (a b c d) () (a b c d)) + + The query system may seem to exhibit quite a bit of intelligence in +using the rules to deduce the answers to the queries above. Actually, +as we will see in the next section, the system is following a +well-determined algorithm in unraveling the rules. Unfortunately, +although the system works impressively in the `append' case, the +general methods may break down in more complex cases, as we will see in +section *Note 4-4-3::. + + *Exercise 4.61:* The following rules implement a `next-to' + relation that finds adjacent elements of a list: + + (rule (?x next-to ?y in (?x ?y . ?u))) + + (rule (?x next-to ?y in (?v . ?z)) + (?x next-to ?y in ?z)) + + What will the response be to the following queries? + + (?x next-to ?y in (1 (2 3) 4)) + + (?x next-to 1 in (2 1 3 1)) + + *Exercise 4.62:* Define rules to implement the `last-pair' + operation of *Note Exercise 2-17::, which returns a list + containing the last element of a nonempty list. Check your rules + on queries such as `(last-pair (3) ?x)', `(last-pair (1 2 3) ?x)', + and `(last-pair (2 ?x) (3))'. Do your rules work correctly on + queries such as `(last-pair ?x (3))' ? + + *Exercise 4.63:* The following data base (see Genesis 4) traces + the genealogy of the descendants of Ada back to Adam, by way of + Cain: + + (son Adam Cain) + (son Cain Enoch) + (son Enoch Irad) + (son Irad Mehujael) + (son Mehujael Methushael) + (son Methushael Lamech) + (wife Lamech Ada) + (son Ada Jabal) + (son Ada Jubal) + + Formulate rules such as "If S is the son of f, and f is the son of + G, then S is the grandson of G" and "If W is the wife of M, and S + is the son of W, then S is the son of M" (which was supposedly + more true in biblical times than today) that will enable the query + system to find the grandson of Cain; the sons of Lamech; the + grandsons of Methushael. (See *Note Exercise 4-69:: for some + rules to deduce more complicated relationships.) + + ---------- Footnotes ---------- + + (1) This uses the dotted-tail notation introduced in *Note Exercise +2-20::. + + (2) Actually, this description of `not' is valid only for simple +cases. The real behavior of `not' is more complex. We will examine +`not''s peculiarities in sections *Note 4-4-2:: and *Note 4-4-3::. + + (3) `Lisp-value' should be used only to perform an operation not +provided in the query language. In particular, it should not be used +to test equality (since that is what the matching in the query language +is designed to do) or inequality (since that can be done with the +`same' rule shown below). + + (4) Notice that we do not need `same' in order to make two things be +the same: We just use the same pattern variable for each--in effect, we +have one thing instead of two things in the first place. For example, +see `?town' in the `lives-near' rule and `?middle-manager' in the +`wheel' rule below. `Same' is useful when we want to force two things +to be different, such as `?person-1' and `?person-2' in the +`lives-near' rule. Although using the same pattern variable in two +parts of a query forces the same value to appear in both places, using +different pattern variables does not force different values to appear. +(The values assigned to different pattern variables may be the same or +different.) + + (5) We will also allow rules without bodies, as in `same', and we +will interpret such a rule to mean that the rule conclusion is +satisfied by any values of the variables. + + +File: sicp.info, Node: 4-4-2, Next: 4-4-3, Prev: 4-4-1, Up: 4-4 + +4.4.2 How the Query System Works +-------------------------------- + +In section *Note 4-4-4:: we will present an implementation of the query +interpreter as a collection of procedures. In this section we give an +overview that explains the general structure of the system independent +of low-level implementation details. After describing the +implementation of the interpreter, we will be in a position to +understand some of its limitations and some of the subtle ways in which +the query language's logical operations differ from the operations of +mathematical logic. + + It should be apparent that the query evaluator must perform some +kind of search in order to match queries against facts and rules in the +data base. One way to do this would be to implement the query system +as a nondeterministic program, using the `amb' evaluator of section +*Note 4-3:: (see *Note Exercise 4-78::). Another possibility is to +manage the search with the aid of streams. Our implementation follows +this second approach. + + The query system is organized around two central operations called "pattern +matching" and "unification". We first describe pattern matching and +explain how this operation, together with the organization of +information in terms of streams of frames, enables us to implement both +simple and compound queries. We next discuss unification, a +generalization of pattern matching needed to implement rules. Finally, +we show how the entire query interpreter fits together through a +procedure that classifies expressions in a manner analogous to the way +`eval' classifies expressions for the interpreter described in section +*Note 4-1::. + +Pattern matching +................ + +A "pattern matcher" is a program that tests whether some datum fits a +specified pattern. For example, the data list `((a b) c (a b))' matches +the pattern `(?x c ?x)' with the pattern variable `?x' bound to `(a +b)'. The same data list matches the pattern `(?x ?y ?z)' with `?x' and +`?z' both bound to `(a b)' and `?y' bound to `c'. It also matches the +pattern `((?x ?y) c (?x ?y))' with `?x' bound to `a' and `?y' bound to +`b'. However, it does not match the pattern `(?x a ?y)', since that +pattern specifies a list whose second element is the symbol `a'. + + The pattern matcher used by the query system takes as inputs a +pattern, a datum, and a "frame" that specifies bindings for various +pattern variables. It checks whether the datum matches the pattern in +a way that is consistent with the bindings already in the frame. If +so, it returns the given frame augmented by any bindings that may have +been determined by the match. Otherwise, it indicates that the match +has failed. + + For example, using the pattern `(?x ?y ?x)' to match `(a b a)' given +an empty frame will return a frame specifying that `?x' is bound to `a' +and `?y' is bound to `b'. Trying the match with the same pattern, the +same datum, and a frame specifying that `?y' is bound to `a' will fail. +Trying the match with the same pattern, the same datum, and a frame in +which `?y' is bound to `b' and `?x' is unbound will return the given +frame augmented by a binding of `?x' to `a'. + + The pattern matcher is all the mechanism that is needed to process +simple queries that don't involve rules. For instance, to process the +query + + (job ?x (computer programmer)) + +we scan through all assertions in the data base and select those that +match the pattern with respect to an initially empty frame. For each +match we find, we use the frame returned by the match to instantiate +the pattern with a value for `?x'. + +Streams of frames +................. + +The testing of patterns against frames is organized through the use of +streams. Given a single frame, the matching process runs through the +data-base entries one by one. For each data-base entry, the matcher +generates either a special symbol indicating that the match has failed +or an extension to the frame. The results for all the data-base +entries are collected into a stream, which is passed through a filter +to weed out the failures. The result is a stream of all the frames +that extend the given frame via a match to some assertion in the data +base.(1) + + In our system, a query takes an input stream of frames and performs +the above matching operation for every frame in the stream, as +indicated in *Note Figure 4-4::. That is, for each frame in the input +stream, the query generates a new stream consisting of all extensions +to that frame by matches to assertions in the data base. All these +streams are then combined to form one huge stream, which contains all +possible extensions of every frame in the input stream. This stream is +the output of the query. + + *Figure 4.4:* A query processes a stream of frames. + + output stream + input stream +-------------+ of frames, + of frames | query | filtered and extended + ---------------->| +-------------------------> + | (job ?x ?y) | + +-------------+ + ^ + | + stream of assertions + from data base + + To answer a simple query, we use the query with an input stream +consisting of a single empty frame. The resulting output stream +contains all extensions to the empty frame (that is, all answers to our +query). This stream of frames is then used to generate a stream of +copies of the original query pattern with the variables instantiated by +the values in each frame, and this is the stream that is finally +printed. + +Compound queries +................ + +The real elegance of the stream-of-frames implementation is evident +when we deal with compound queries. The processing of compound queries +makes use of the ability of our matcher to demand that a match be +consistent with a specified frame. For example, to handle the `and' of +two queries, such as + + (and (can-do-job ?x (computer programmer trainee)) + (job ?person ?x)) + +(informally, "Find all people who can do the job of a computer +programmer trainee"), we first find all entries that match the pattern + + (can-do-job ?x (computer programmer trainee)) + + This produces a stream of frames, each of which contains a binding +for `?x'. Then for each frame in the stream we find all entries that +match + + (job ?person ?x) + +in a way that is consistent with the given binding for `?x'. Each such +match will produce a frame containing bindings for `?x' and `?person'. +The `and' of two queries can be viewed as a series combination of the +two component queries, as shown in *Note Figure 4-5::. The frames that +pass through the first query filter are filtered and further extended +by the second query. + + *Figure 4.5:* The `and' combination of two queries is produced by + operating on the stream of frames in series. + + +----------------------+ + | (and A B) | + input stream | | output stream + of frames | +---+ +---+ | of frames + ------------------->| A +------>| B +--------------------> + | +---+ +---+ | + | ^ ^ | + | | | | + | +-----*-----+ | + +-----------|----------+ + | + data base + + *Note Figure 4-6:: shows the analogous method for computing the `or' +of two queries as a parallel combination of the two component queries. +The input stream of frames is extended separately by each query. The +two resulting streams are then merged to produce the final output +stream. + + *Figure 4.6:* The `or' combination of two queries is produced by + operating on the stream of frames in parallel and merging the + results. + + +---------------------------+ + | (or A B) | + | +---+ | + input | +->| A |------------+ | output + stream of | | +---+ V | stream of + frames | | ^ +-------+ | frames + -------------* | | merge +---------------> + | | | +-------+ | + | | | ^ | + | | | +---+ | | + | +------->| B +------+ | + | | +---+ | + | | ^ | + | | | | + | +--*--+ | + +---------|-----------------+ + | + data base + + Even from this high-level description, it is apparent that the +processing of compound queries can be slow. For example, since a query +may produce more than one output frame for each input frame, and each +query in an `and' gets its input frames from the previous query, an +`and' query could, in the worst case, have to perform a number of +matches that is exponential in the number of queries (see *Note +Exercise 4-76::).(2) Though systems for handling only simple queries +are quite practical, dealing with complex queries is extremely +difficult.(3) + + From the stream-of-frames viewpoint, the `not' of some query acts as +a filter that removes all frames for which the query can be satisfied. +For instance, given the pattern + + (not (job ?x (computer programmer))) + +we attempt, for each frame in the input stream, to produce extension +frames that satisfy `(job ?x (computer programmer))'. We remove from +the input stream all frames for which such extensions exist. The +result is a stream consisting of only those frames in which the binding +for `?x' does not satisfy `(job ?x (computer programmer))'. For +example, in processing the query + + (and (supervisor ?x ?y) + (not (job ?x (computer programmer)))) + +the first clause will generate frames with bindings for `?x' and `?y'. +The `not' clause will then filter these by removing all frames in which +the binding for `?x' satisfies the restriction that `?x' is a computer +programmer.(4) + + The `lisp-value' special form is implemented as a similar filter on +frame streams. We use each frame in the stream to instantiate any +variables in the pattern, then apply the Lisp predicate. We remove +from the input stream all frames for which the predicate fails. + +Unification +........... + +In order to handle rules in the query language, we must be able to find +the rules whose conclusions match a given query pattern. Rule +conclusions are like assertions except that they can contain variables, +so we will need a generalization of pattern matching--called "unification"--in +which both the "pattern" and the "datum" may contain variables. + + A unifier takes two patterns, each containing constants and +variables, and determines whether it is possible to assign values to +the variables that will make the two patterns equal. If so, it returns +a frame containing these bindings. For example, unifying `(?x a ?y)' +and `(?y ?z a)' will specify a frame in which `?x', `?y', and `?z' must +all be bound to `a'. On the other hand, unifying `(?x ?y a)' and `(?x +b ?y)' will fail, because there is no value for `?y' that can make the +two patterns equal. (For the second elements of the patterns to be +equal, `?y' would have to be `b'; however, for the third elements to be +equal, `?y' would have to be `a'.) The unifier used in the query +system, like the pattern matcher, takes a frame as input and performs +unifications that are consistent with this frame. + + The unification algorithm is the most technically difficult part of +the query system. With complex patterns, performing unification may +seem to require deduction. To unify `(?x ?x)' and `((a ?y c) (a b +?z))', for example, the algorithm must infer that `?x' should be `(a b +c)', `?y' should be `b', and `?z' should be `c'. We may think of this +process as solving a set of equations among the pattern components. In +general, these are simultaneous equations, which may require substantial +manipulation to solve.(5) For example, unifying `(?x ?x)' and `((a ?y +c) (a b ?z))' may be thought of as specifying the simultaneous equations + + ?x = (a ?y c) + ?x = (a b ?z) + + These equations imply that + + (a ?y c) = (a b ?z) + +which in turn implies that + + a = a, ?y = b, c = ?z, + +and hence that + + ?x = (a b c) + + In a successful pattern match, all pattern variables become bound, +and the values to which they are bound contain only constants. This is +also true of all the examples of unification we have seen so far. In +general, however, a successful unification may not completely determine +the variable values; some variables may remain unbound and others may +be bound to values that contain variables. + + Consider the unification of `(?x a)' and `((b ?y) ?z)'. We can +deduce that `?x = (b ?y)' and `a = ?z', but we cannot further solve for +`?x' or `?y'. The unification doesn't fail, since it is certainly +possible to make the two patterns equal by assigning values to `?x' and +`?y'. Since this match in no way restricts the values `?y' can take +on, no binding for `?y' is put into the result frame. The match does, +however, restrict the value of `?x'. Whatever value `?y' has, `?x' +must be `(b ?y)'. A binding of `?x' to the pattern `(b ?y)' is thus +put into the frame. If a value for `?y' is later determined and added +to the frame (by a pattern match or unification that is required to be +consistent with this frame), the previously bound `?x' will refer to +this value.(6) + +Applying rules +.............. + +Unification is the key to the component of the query system that makes +inferences from rules. To see how this is accomplished, consider +processing a query that involves applying a rule, such as + + (lives-near ?x (Hacker Alyssa P)) + + To process this query, we first use the ordinary pattern-match +procedure described above to see if there are any assertions in the +data base that match this pattern. (There will not be any in this +case, since our data base includes no direct assertions about who lives +near whom.) The next step is to attempt to unify the query pattern +with the conclusion of each rule. We find that the pattern unifies +with the conclusion of the rule + + (rule (lives-near ?person-1 ?person-2) + (and (address ?person-1 (?town . ?rest-1)) + (address ?person-2 (?town . ?rest-2)) + (not (same ?person-1 ?person-2)))) + +resulting in a frame specifying that `?person-2' is bound to `(Hacker +Alyssa P)' and that `?x' should be bound to (have the same value as) +`?person-1'. Now, relative to this frame, we evaluate the compound +query given by the body of the rule. Successful matches will extend +this frame by providing a binding for `?person-1', and consequently a +value for `?x', which we can use to instantiate the original query +pattern. + + In general, the query evaluator uses the following method to apply a +rule when trying to establish a query pattern in a frame that specifies +bindings for some of the pattern variables: + + * Unify the query with the conclusion of the rule to form, if + successful, an extension of the original frame. + + * Relative to the extended frame, evaluate the query formed by the + body of the rule. + + + Notice how similar this is to the method for applying a procedure in +the `eval'/`apply' evaluator for Lisp: + + * Bind the procedure's parameters to its arguments to form a frame + that extends the original procedure environment. + + * Relative to the extended environment, evaluate the expression + formed by the body of the procedure. + + + The similarity between the two evaluators should come as no +surprise. Just as procedure definitions are the means of abstraction +in Lisp, rule definitions are the means of abstraction in the query +language. In each case, we unwind the abstraction by creating +appropriate bindings and evaluating the rule or procedure body relative +to these. + +Simple queries +.............. + +We saw earlier in this section how to evaluate simple queries in the +absence of rules. Now that we have seen how to apply rules, we can +describe how to evaluate simple queries by using both rules and +assertions. + + Given the query pattern and a stream of frames, we produce, for each +frame in the input stream, two streams: + + * a stream of extended frames obtained by matching the pattern + against all assertions in the data base (using the pattern + matcher), and + + * a stream of extended frames obtained by applying all possible + rules (using the unifier).(7) + + + Appending these two streams produces a stream that consists of all +the ways that the given pattern can be satisfied consistent with the +original frame. These streams (one for each frame in the input stream) +are now all combined to form one large stream, which therefore consists +of all the ways that any of the frames in the original input stream can +be extended to produce a match with the given pattern. + +The query evaluator and the driver loop +....................................... + +Despite the complexity of the underlying matching operations, the +system is organized much like an evaluator for any language. The +procedure that coordinates the matching operations is called `qeval', +and it plays a role analogous to that of the `eval' procedure for Lisp. +`Qeval' takes as inputs a query and a stream of frames. Its output is +a stream of frames, corresponding to successful matches to the query +pattern, that extend some frame in the input stream, as indicated in +*Note Figure 4-4::. Like `eval', `qeval' classifies the different +types of expressions (queries) and dispatches to an appropriate +procedure for each. There is a procedure for each special form (`and', +`or', `not', and `lisp-value') and one for simple queries. + + The driver loop, which is analogous to the `driver-loop' procedure +for the other evaluators in this chapter, reads queries from the +terminal. For each query, it calls `qeval' with the query and a stream +that consists of a single empty frame. This will produce the stream of +all possible matches (all possible extensions to the empty frame). For +each frame in the resulting stream, it instantiates the original query +using the values of the variables found in the frame. This stream of +instantiated queries is then printed.(8) + + The driver also checks for the special command `assert!', which +signals that the input is not a query but rather an assertion or rule +to be added to the data base. For instance, + + (assert! (job (Bitdiddle Ben) (computer wizard))) + + (assert! (rule (wheel ?person) + (and (supervisor ?middle-manager ?person) + (supervisor ?x ?middle-manager)))) + + ---------- Footnotes ---------- + + (1) Because matching is generally very expensive, we would like to +avoid applying the full matcher to every element of the data base. +This is usually arranged by breaking up the process into a fast, coarse +match and the final match. The coarse match filters the data base to +produce a small set of candidates for the final match. With care, we +can arrange our data base so that some of the work of coarse matching +can be done when the data base is constructed rather then when we want +to select the candidates. This is called "indexing" the data base. +There is a vast technology built around data-base-indexing schemes. +Our implementation, described in section *Note 4-4-4::, contains a +simple-minded form of such an optimization. + + (2) But this kind of exponential explosion is not common in `and' +queries because the added conditions tend to reduce rather than expand +the number of frames produced. + + (3) There is a large literature on data-base-management systems that +is concerned with how to handle complex queries efficiently. + + (4) There is a subtle difference between this filter implementation +of `not' and the usual meaning of `not' in mathematical logic. See +section *Note 4-4-3::. + + (5) In one-sided pattern matching, all the equations that contain +pattern variables are explicit and already solved for the unknown (the +pattern variable). + + (6) Another way to think of unification is that it generates the +most general pattern that is a specialization of the two input +patterns. That is, the unification of `(?x a)' and `((b ?y) ?z)' is +`((b ?y) a)', and the unification of `(?x a ?y)' and `(?y ?z a)', +discussed above, is `(a a a)'. For our implementation, it is more +convenient to think of the result of unification as a frame rather than +a pattern. + + (7) Since unification is a generalization of matching, we could +simplify the system by using the unifier to produce both streams. +Treating the easy case with the simple matcher, however, illustrates +how matching (as opposed to full-blown unification) can be useful in +its own right. + + (8) The reason we use streams (rather than lists) of frames is that +the recursive application of rules can generate infinite numbers of +values that satisfy a query. The delayed evaluation embodied in +streams is crucial here: The system will print responses one by one as +they are generated, regardless of whether there are a finite or +infinite number of responses. + + +File: sicp.info, Node: 4-4-3, Next: 4-4-4, Prev: 4-4-2, Up: 4-4 + +4.4.3 Is Logic Programming Mathematical Logic? +---------------------------------------------- + +The means of combination used in the query language may at first seem +identical to the operations `and', `or', and `not' of mathematical +logic, and the application of query-language rules is in fact +accomplished through a legitimate method of inference.(1) This +identification of the query language with mathematical logic is not +really valid, though, because the query language provides a structure +"control structure" that interprets the logical statements +procedurally. We can often take advantage of this control structure. +For example, to find all of the supervisors of programmers we could +formulate a query in either of two logically equivalent forms: + + (and (job ?x (computer programmer)) + (supervisor ?x ?y)) + +or + + (and (supervisor ?x ?y) + (job ?x (computer programmer))) + + If a company has many more supervisors than programmers (the usual +case), it is better to use the first form rather than the second +because the data base must be scanned for each intermediate result +(frame) produced by the first clause of the `and'. + + The aim of logic programming is to provide the programmer with +techniques for decomposing a computational problem into two separate +problems: "what" is to be computed, and "how" this should be computed. +This is accomplished by selecting a subset of the statements of +mathematical logic that is powerful enough to be able to describe +anything one might want to compute, yet weak enough to have a +controllable procedural interpretation. The intention here is that, on +the one hand, a program specified in a logic programming language +should be an effective program that can be carried out by a computer. +Control ("how" to compute) is effected by using the order of evaluation +of the language. We should be able to arrange the order of clauses and +the order of subgoals within each clause so that the computation is +done in an order deemed to be effective and efficient. At the same +time, we should be able to view the result of the computation ("what" +to compute) as a simple consequence of the laws of logic. + + Our query language can be regarded as just such a procedurally +interpretable subset of mathematical logic. An assertion represents a +simple fact (an atomic proposition). A rule represents the implication +that the rule conclusion holds for those cases where the rule body +holds. A rule has a natural procedural interpretation: To establish +the conclusion of the rule, establish the body of the rule. Rules, +therefore, specify computations. However, because rules can also be +regarded as statements of mathematical logic, we can justify any +"inference" accomplished by a logic program by asserting that the same +result could be obtained by working entirely within mathematical +logic.(2) + +Infinite loops +.............. + +A consequence of the procedural interpretation of logic programs is +that it is possible to construct hopelessly inefficient programs for +solving certain problems. An extreme case of inefficiency occurs when +the system falls into infinite loops in making deductions. As a simple +example, suppose we are setting up a data base of famous marriages, +including + + (assert! (married Minnie Mickey)) + + If we now ask + + (married Mickey ?who) + +we will get no response, because the system doesn't know that if A is +married to B, then B is married to A. So we assert the rule + + (assert! (rule (married ?x ?y) + (married ?y ?x))) + +and again query + + (married Mickey ?who) + + Unfortunately, this will drive the system into an infinite loop, as +follows: + + * The system finds that the `married' rule is applicable; that is, + the rule conclusion `(married ?x ?y)' successfully unifies with + the query pattern `(married Mickey ?who)' to produce a frame in + which `?x' is bound to `Mickey' and `?y' is bound to `?who'. So + the interpreter proceeds to evaluate the rule body `(married ?y + ?x)' in this frame--in effect, to process the query `(married ?who + Mickey)'. + + * One answer appears directly as an assertion in the data base: + `(married Minnie Mickey)'. + + * The `married' rule is also applicable, so the interpreter again + evaluates the rule body, which this time is equivalent to + `(married Mickey ?who)'. + + + The system is now in an infinite loop. Indeed, whether the system +will find the simple answer `(married Minnie Mickey)' before it goes +into the loop depends on implementation details concerning the order in +which the system checks the items in the data base. This is a very +simple example of the kinds of loops that can occur. Collections of +interrelated rules can lead to loops that are much harder to +anticipate, and the appearance of a loop can depend on the order of +clauses in an `and' (see *Note Exercise 4-64::) or on low-level details +concerning the order in which the system processes queries.(3) + +Problems with `not' +................... + +Another quirk in the query system concerns `not'. Given the data base +of section *Note 4-4-1::, consider the following two queries: + + (and (supervisor ?x ?y) + (not (job ?x (computer programmer)))) + + (and (not (job ?x (computer programmer))) + (supervisor ?x ?y)) + + These two queries do not produce the same result. The first query +begins by finding all entries in the data base that match `(supervisor +?x ?y)', and then filters the resulting frames by removing the ones in +which the value of `?x' satisfies `(job ?x (computer programmer))'. +The second query begins by filtering the incoming frames to remove +those that can satisfy `(job ?x (computer programmer))'. Since the +only incoming frame is empty, it checks the data base to see if there +are any patterns that satisfy `(job ?x (computer programmer))'. Since +there generally are entries of this form, the `not' clause filters out +the empty frame and returns an empty stream of frames. Consequently, +the entire compound query returns an empty stream. + + The trouble is that our implementation of `not' really is meant to +serve as a filter on values for the variables. If a `not' clause is +processed with a frame in which some of the variables remain unbound +(as does `?x' in the example above), the system will produce unexpected +results. Similar problems occur with the use of `lisp-value'--the Lisp +predicate can't work if some of its arguments are unbound. See *Note +Exercise 4-77::. + + There is also a much more serious way in which the `not' of the query +language differs from the `not' of mathematical logic. In logic, we +interpret the statement "not P" to mean that P is not true. In the +query system, however, "not P" means that P is not deducible from the +knowledge in the data base. For example, given the personnel data base +of section *Note 4-4-1::, the system would happily deduce all sorts of +`not' statements, such as that Ben Bitdiddle is not a baseball fan, +that it is not raining outside, and that 2 + 2 is not 4.(4) In other +words, the `not' of logic programming languages reflects the so-called "closed +world assumption" that all relevant information has been included in +the data base.(5) + + *Exercise 4.64:* Louis Reasoner mistakenly deletes the + `outranked-by' rule (section *Note 4-4-1::) from the data base. + When he realizes this, he quickly reinstalls it. Unfortunately, + he makes a slight change in the rule, and types it in as + + (rule (outranked-by ?staff-person ?boss) + (or (supervisor ?staff-person ?boss) + (and (outranked-by ?middle-manager ?boss) + (supervisor ?staff-person ?middle-manager)))) + + Just after Louis types this information into the system, DeWitt + Aull comes by to find out who outranks Ben Bitdiddle. He issues + the query + + (outranked-by (Bitdiddle Ben) ?who) + + After answering, the system goes into an infinite loop. Explain + why. + + *Exercise 4.65:* Cy D. Fect, looking forward to the day when he + will rise in the organization, gives a query to find all the + wheels (using the `wheel' rule of section *Note 4-4-1::): + + (wheel ?who) + + To his surprise, the system responds + + ;;; Query results: + (wheel (Warbucks Oliver)) + (wheel (Bitdiddle Ben)) + (wheel (Warbucks Oliver)) + (wheel (Warbucks Oliver)) + (wheel (Warbucks Oliver)) + + Why is Oliver Warbucks listed four times? + + *Exercise 4.66:* Ben has been generalizing the query system to + provide statistics about the company. For example, to find the + total salaries of all the computer programmers one will be able to + say + + (sum ?amount + (and (job ?x (computer programmer)) + (salary ?x ?amount))) + + In general, Ben's new system allows expressions of the form + + (accumulation-function + ) + + where `accumulation-function' can be things like `sum', `average', + or `maximum'. Ben reasons that it should be a cinch to implement + this. He will simply feed the query pattern to `qeval'. This + will produce a stream of frames. He will then pass this stream + through a mapping function that extracts the value of the + designated variable from each frame in the stream and feed the + resulting stream of values to the accumulation function. Just as + Ben completes the implementation and is about to try it out, Cy + walks by, still puzzling over the `wheel' query result in exercise + *Note Exercise 4-65::. When Cy shows Ben the system's response, + Ben groans, "Oh, no, my simple accumulation scheme won't work!" + + What has Ben just realized? Outline a method he can use to + salvage the situation. + + *Exercise 4.67:* Devise a way to install a loop detector in the + query system so as to avoid the kinds of simple loops illustrated + in the text and in *Note Exercise 4-64::. The general idea is that + the system should maintain some sort of history of its current + chain of deductions and should not begin processing a query that + it is already working on. Describe what kind of information + (patterns and frames) is included in this history, and how the + check should be made. (After you study the details of the + query-system implementation in section *Note 4-4-4::, you may want + to modify the system to include your loop detector.) + + *Exercise 4.68:* Define rules to implement the `reverse' operation + of *Note Exercise 2-18::, which returns a list containing the same + elements as a given list in reverse order. (Hint: Use + `append-to-form'.) Can your rules answer both `(reverse (1 2 3) + ?x)' and `(reverse ?x (1 2 3))' ? + + *Exercise 4.69:* Beginning with the data base and the rules you + formulated in *Note Exercise 4-63::, devise a rule for adding + "greats" to a grandson relationship. This should enable the system + to deduce that Irad is the great-grandson of Adam, or that Jabal + and Jubal are the great-great-great-great-great-grandsons of Adam. + (Hint: Represent the fact about Irad, for example, as `((great + grandson) Adam Irad)'. Write rules that determine if a list ends + in the word `grandson'. Use this to express a rule that allows + one to derive the relationship `((great . ?rel) ?x ?y)', where + `?rel' is a list ending in `grandson'.) Check your rules on + queries such as `((great grandson) ?g ?ggs)' and `(?relationship + Adam Irad)'. + + ---------- Footnotes ---------- + + (1) That a particular method of inference is legitimate is not a +trivial assertion. One must prove that if one starts with true +premises, only true conclusions can be derived. The method of +inference represented by rule applications is "modus ponens", the +familiar method of inference that says that if A is true and _A implies +B_ is true, then we may conclude that B is true. + + (2) We must qualify this statement by agreeing that, in speaking of +the "inference" accomplished by a logic program, we assume that the +computation terminates. Unfortunately, even this qualified statement +is false for our implementation of the query language (and also false +for programs in Prolog and most other current logic programming +languages) because of our use of `not' and `lisp-value'. As we will +describe below, the `not' implemented in the query language is not +always consistent with the `not' of mathematical logic, and +`lisp-value' introduces additional complications. We could implement a +language consistent with mathematical logic by simply removing `not' +and `lisp-value' from the language and agreeing to write programs using +only simple queries, `and', and `or'. However, this would greatly +restrict the expressive power of the language. One of the major +concerns of research in logic programming is to find ways to achieve +more consistency with mathematical logic without unduly sacrificing +expressive power. + + (3) This is not a problem of the logic but one of the procedural +interpretation of the logic provided by our interpreter. We could +write an interpreter that would not fall into a loop here. For +example, we could enumerate all the proofs derivable from our +assertions and our rules in a breadth-first rather than a depth-first +order. However, such a system makes it more difficult to take +advantage of the order of deductions in our programs. One attempt to +build sophisticated control into such a program is described in deKleer +et al. 1977. Another technique, which does not lead to such serious +control problems, is to put in special knowledge, such as detectors for +particular kinds of loops (*Note Exercise 4-67::). However, there can +be no general scheme for reliably preventing a system from going down +infinite paths in performing deductions. Imagine a diabolical rule of +the form "To show P(x) is true, show that P(f(x)) is true," for some +suitably chosen function f. + + (4) Consider the query `(not (baseball-fan (Bitdiddle Ben)))'. The +system finds that `(baseball-fan (Bitdiddle Ben))' is not in the data +base, so the empty frame does not satisfy the pattern and is not +filtered out of the initial stream of frames. The result of the query +is thus the empty frame, which is used to instantiate the input query +to produce `(not (baseball-fan (Bitdiddle Ben)))'. + + (5) A discussion and justification of this treatment of `not' can be +found in the article by Clark (1978). + + +File: sicp.info, Node: 4-4-4, Prev: 4-4-3, Up: 4-4 + +4.4.4 Implementing the Query System +----------------------------------- + +Section *Note 4-4-2:: described how the query system works. Now we fill +in the details by presenting a complete implementation of the system. + +* Menu: + +* 4-4-4-1:: The Driver Loop and Instantiation +* 4-4-4-2:: The Evaluator +* 4-4-4-3:: Finding Assertions by Pattern Matching +* 4-4-4-4:: Rules and Unification +* 4-4-4-5:: Maintaining the Data Base +* 4-4-4-6:: Stream Operations +* 4-4-4-7:: Query Syntax Procedures +* 4-4-4-8:: Frames and Bindings + + +File: sicp.info, Node: 4-4-4-1, Next: 4-4-4-2, Prev: 4-4-4, Up: 4-4-4 + +4.4.4.1 The Driver Loop and Instantiation +......................................... + +The driver loop for the query system repeatedly reads input +expressions. If the expression is a rule or assertion to be added to +the data base, then the information is added. Otherwise the expression +is assumed to be a query. The driver passes this query to the +evaluator `qeval' together with an initial frame stream consisting of a +single empty frame. The result of the evaluation is a stream of frames +generated by satisfying the query with variable values found in the +data base. These frames are used to form a new stream consisting of +copies of the original query in which the variables are instantiated +with values supplied by the stream of frames, and this final stream is +printed at the terminal: + + (define input-prompt ";;; Query input:") + (define output-prompt ";;; Query results:") + + (define (query-driver-loop) + (prompt-for-input input-prompt) + (let ((q (query-syntax-process (read)))) + (cond ((assertion-to-be-added? q) + (add-rule-or-assertion! (add-assertion-body q)) + (newline) + (display "Assertion added to data base.") + (query-driver-loop)) + (else + (newline) + (display output-prompt) + (display-stream + (stream-map + (lambda (frame) + (instantiate q + frame + (lambda (v f) + (contract-question-mark v)))) + (qeval q (singleton-stream '())))) + (query-driver-loop))))) + + Here, as in the other evaluators in this chapter, we use an abstract +syntax for the expressions of the query language. The implementation +of the expression syntax, including the predicate +`assertion-to-be-added?' and the selector `add-assertion-body', is +given in section *Note 4-4-4-7::. `Add-rule-or-assertion!' is defined +in section *Note 4-4-4-5::. + + Before doing any processing on an input expression, the driver loop +transforms it syntactically into a form that makes the processing more +efficient. This involves changing the representation of pattern +variables. When the query is instantiated, any variables that remain +unbound are transformed back to the input representation before being +printed. These transformations are performed by the two procedures +`query-syntax-process' and `contract-question-mark' (section *Note +4-4-4-7::). + + To instantiate an expression, we copy it, replacing any variables in +the expression by their values in a given frame. The values are +themselves instantiated, since they could contain variables (for +example, if `?x' in `exp' is bound to `?y' as the result of unification +and `?y' is in turn bound to 5). The action to take if a variable +cannot be instantiated is given by a procedural argument to +`instantiate'. + + (define (instantiate exp frame unbound-var-handler) + (define (copy exp) + (cond ((var? exp) + (let ((binding (binding-in-frame exp frame))) + (if binding + (copy (binding-value binding)) + (unbound-var-handler exp frame)))) + ((pair? exp) + (cons (copy (car exp)) (copy (cdr exp)))) + (else exp))) + (copy exp)) + + The procedures that manipulate bindings are defined in section *Note +4-4-4-8::. + + +File: sicp.info, Node: 4-4-4-2, Next: 4-4-4-3, Prev: 4-4-4-1, Up: 4-4-4 + +4.4.4.2 The Evaluator +..................... + +The `qeval' procedure, called by the `query-driver-loop', is the basic +evaluator of the query system. It takes as inputs a query and a stream +of frames, and it returns a stream of extended frames. It identifies +special forms by a data-directed dispatch using `get' and `put', just +as we did in implementing generic operations in *Note Chapter 2::. Any +query that is not identified as a special form is assumed to be a +simple query, to be processed by `simple-query'. + + (define (qeval query frame-stream) + (let ((qproc (get (type query) 'qeval))) + (if qproc + (qproc (contents query) frame-stream) + (simple-query query frame-stream)))) + + `Type' and `contents', defined in section *Note 4-4-4-7::, implement +the abstract syntax of the special forms. + +Simple queries +.............. + +The `simple-query' procedure handles simple queries. It takes as +arguments a simple query (a pattern) together with a stream of frames, +and it returns the stream formed by extending each frame by all +data-base matches of the query. + + (define (simple-query query-pattern frame-stream) + (stream-flatmap + (lambda (frame) + (stream-append-delayed + (find-assertions query-pattern frame) + (delay (apply-rules query-pattern frame)))) + frame-stream)) + + For each frame in the input stream, we use `find-assertions' (section +*Note 4-4-4-3::) to match the pattern against all assertions in the +data base, producing a stream of extended frames, and we use +`apply-rules' (section *Note 4-4-4-4::) to apply all possible rules, +producing another stream of extended frames. These two streams are +combined (using `stream-append-delayed', section *Note 4-4-4-6::) to +make a stream of all the ways that the given pattern can be satisfied +consistent with the original frame (see *Note Exercise 4-71::). The +streams for the individual input frames are combined using +`stream-flatmap' (section *Note 4-4-4-6::) to form one large stream of +all the ways that any of the frames in the original input stream can be +extended to produce a match with the given pattern. + +Compound queries +................ + +`And' queries are handled as illustrated in *Note Figure 4-5:: by the +`conjoin' procedure. `Conjoin' takes as inputs the conjuncts and the +frame stream and returns the stream of extended frames. First, +`conjoin' processes the stream of frames to find the stream of all +possible frame extensions that satisfy the first query in the +conjunction. Then, using this as the new frame stream, it recursively +applies `conjoin' to the rest of the queries. + + (define (conjoin conjuncts frame-stream) + (if (empty-conjunction? conjuncts) + frame-stream + (conjoin (rest-conjuncts conjuncts) + (qeval (first-conjunct conjuncts) + frame-stream)))) + + The expression + + (put 'and 'qeval conjoin) + +sets up `qeval' to dispatch to `conjoin' when an `and' form is +encountered. + + `Or' queries are handled similarly, as shown in *Note Figure 4-6::. +The output streams for the various disjuncts of the `or' are computed +separately and merged using the `interleave-delayed' procedure from +section *Note 4-4-4-6::. (See *Note Exercise 4-71:: and *Note Exercise +4-72::.) + + (define (disjoin disjuncts frame-stream) + (if (empty-disjunction? disjuncts) + the-empty-stream + (interleave-delayed + (qeval (first-disjunct disjuncts) frame-stream) + (delay (disjoin (rest-disjuncts disjuncts) + frame-stream))))) + + (put 'or 'qeval disjoin) + + The predicates and selectors for the syntax of conjuncts and +disjuncts are given in section *Note 4-4-4-7::. + +Filters +....... + +`Not' is handled by the method outlined in section *Note 4-4-2::. We +attempt to extend each frame in the input stream to satisfy the query +being negated, and we include a given frame in the output stream only +if it cannot be extended. + + (define (negate operands frame-stream) + (stream-flatmap + (lambda (frame) + (if (stream-null? (qeval (negated-query operands) + (singleton-stream frame))) + (singleton-stream frame) + the-empty-stream)) + frame-stream)) + + (put 'not 'qeval negate) + + `Lisp-value' is a filter similar to `not'. Each frame in the stream +is used to instantiate the variables in the pattern, the indicated +predicate is applied, and the frames for which the predicate returns +false are filtered out of the input stream. An error results if there +are unbound pattern variables. + + (define (lisp-value call frame-stream) + (stream-flatmap + (lambda (frame) + (if (execute + (instantiate + call + frame + (lambda (v f) + (error "Unknown pat var -- LISP-VALUE" v)))) + (singleton-stream frame) + the-empty-stream)) + frame-stream)) + + (put 'lisp-value 'qeval lisp-value) + + `Execute', which applies the predicate to the arguments, must `eval' +the predicate expression to get the procedure to apply. However, it +must not evaluate the arguments, since they are already the actual +arguments, not expressions whose evaluation (in Lisp) will produce the +arguments. Note that `execute' is implemented using `eval' and `apply' +from the underlying Lisp system. + + (define (execute exp) + (apply (eval (predicate exp) user-initial-environment) + (args exp))) + + The `always-true' special form provides for a query that is always +satisfied. It ignores its contents (normally empty) and simply passes +through all the frames in the input stream. `Always-true' is used by +the `rule-body' selector (section *Note 4-4-4-7::) to provide bodies +for rules that were defined without bodies (that is, rules whose +conclusions are always satisfied). + + (define (always-true ignore frame-stream) frame-stream) + + (put 'always-true 'qeval always-true) + + The selectors that define the syntax of `not' and `lisp-value' are +given in section *Note 4-4-4-7::. + + +File: sicp.info, Node: 4-4-4-3, Next: 4-4-4-4, Prev: 4-4-4-2, Up: 4-4-4 + +4.4.4.3 Finding Assertions by Pattern Matching +.............................................. + +`Find-assertions', called by `simple-query' (section *Note 4-4-4-2::), +takes as input a pattern and a frame. It returns a stream of frames, +each extending the given one by a data-base match of the given pattern. +It uses `fetch-assertions' (section *Note 4-4-4-5::) to get a stream +of all the assertions in the data base that should be checked for a +match against the pattern and the frame. The reason for +`fetch-assertions' here is that we can often apply simple tests that +will eliminate many of the entries in the data base from the pool of +candidates for a successful match. The system would still work if we +eliminated `fetch-assertions' and simply checked a stream of all +assertions in the data base, but the computation would be less efficient +because we would need to make many more calls to the matcher. + + (define (find-assertions pattern frame) + (stream-flatmap (lambda (datum) + (check-an-assertion datum pattern frame)) + (fetch-assertions pattern frame))) + + `Check-an-assertion' takes as arguments a pattern, a data object +(assertion), and a frame and returns either a one-element stream +containing the extended frame or `the-empty-stream' if the match fails. + + (define (check-an-assertion assertion query-pat query-frame) + (let ((match-result + (pattern-match query-pat assertion query-frame))) + (if (eq? match-result 'failed) + the-empty-stream + (singleton-stream match-result)))) + + The basic pattern matcher returns either the symbol `failed' or an +extension of the given frame. The basic idea of the matcher is to +check the pattern against the data, element by element, accumulating +bindings for the pattern variables. If the pattern and the data object +are the same, the match succeeds and we return the frame of bindings +accumulated so far. Otherwise, if the pattern is a variable we extend +the current frame by binding the variable to the data, so long as this +is consistent with the bindings already in the frame. If the pattern +and the data are both pairs, we (recursively) match the `car' of the +pattern against the `car' of the data to produce a frame; in this frame +we then match the `cdr' of the pattern against the `cdr' of the data. +If none of these cases are applicable, the match fails and we return +the symbol `failed'. + + (define (pattern-match pat dat frame) + (cond ((eq? frame 'failed) 'failed) + ((equal? pat dat) frame) + ((var? pat) (extend-if-consistent pat dat frame)) + ((and (pair? pat) (pair? dat)) + (pattern-match (cdr pat) + (cdr dat) + (pattern-match (car pat) + (car dat) + frame))) + (else 'failed))) + + Here is the procedure that extends a frame by adding a new binding, +if this is consistent with the bindings already in the frame: + + (define (extend-if-consistent var dat frame) + (let ((binding (binding-in-frame var frame))) + (if binding + (pattern-match (binding-value binding) dat frame) + (extend var dat frame)))) + + If there is no binding for the variable in the frame, we simply add +the binding of the variable to the data. Otherwise we match, in the +frame, the data against the value of the variable in the frame. If the +stored value contains only constants, as it must if it was stored +during pattern matching by `extend-if-consistent', then the match +simply tests whether the stored and new values are the same. If so, it +returns the unmodified frame; if not, it returns a failure indication. +The stored value may, however, contain pattern variables if it was +stored during unification (see section *Note 4-4-4-4::). The recursive +match of the stored pattern against the new data will add or check +bindings for the variables in this pattern. For example, suppose we +have a frame in which `?x' is bound to `(f ?y)' and `?y' is unbound, +and we wish to augment this frame by a binding of `?x' to `(f b)'. We +look up `?x' and find that it is bound to `(f ?y)'. This leads us to +match `(f ?y)' against the proposed new value `(f b)' in the same +frame. Eventually this match extends the frame by adding a binding of +`?y' to `b'. `?X' remains bound to `(f ?y)'. We never modify a stored +binding and we never store more than one binding for a given variable. + + The procedures used by `extend-if-consistent' to manipulate bindings +are defined in section *Note 4-4-4-8::. + +Patterns with dotted tails +.......................... + +If a pattern contains a dot followed by a pattern variable, the pattern +variable matches the rest of the data list (rather than the next +element of the data list), just as one would expect with the +dotted-tail notation described in *Note Exercise 2-20::. Although the +pattern matcher we have just implemented doesn't look for dots, it does +behave as we want. This is because the Lisp `read' primitive, which is +used by `query-driver-loop' to read the query and represent it as a +list structure, treats dots in a special way. + + When `read' sees a dot, instead of making the next item be the next +element of a list (the `car' of a `cons' whose `cdr' will be the rest +of the list) it makes the next item be the `cdr' of the list structure. +For example, the list structure produced by `read' for the pattern +`(computer ?type)' could be constructed by evaluating the expression +`(cons 'computer (cons '?type '()))', and that for `(computer . ?type)' +could be constructed by evaluating the expression `(cons 'computer +'?type)'. + + Thus, as `pattern-match' recursively compares `car's and `cdr's of a +data list and a pattern that had a dot, it eventually matches the +variable after the dot (which is a `cdr' of the pattern) against a +sublist of the data list, binding the variable to that list. For +example, matching the pattern `(computer . ?type)' against `(computer +programmer trainee)' will match `?type' against the list `(programmer +trainee)'. + + +File: sicp.info, Node: 4-4-4-4, Next: 4-4-4-5, Prev: 4-4-4-3, Up: 4-4-4 + +4.4.4.4 Rules and Unification +............................. + +`Apply-rules' is the rule analog of `find-assertions' (section *Note +4-4-4-3::). It takes as input a pattern and a frame, and it forms a +stream of extension frames by applying rules from the data base. +`Stream-flatmap' maps `apply-a-rule' down the stream of possibly +applicable rules (selected by `fetch-rules', section *Note 4-4-4-5::) +and combines the resulting streams of frames. + + (define (apply-rules pattern frame) + (stream-flatmap (lambda (rule) + (apply-a-rule rule pattern frame)) + (fetch-rules pattern frame))) + + `Apply-a-rule' applies rules using the method outlined in section +*Note 4-4-2::. It first augments its argument frame by unifying the +rule conclusion with the pattern in the given frame. If this succeeds, +it evaluates the rule body in this new frame. + + Before any of this happens, however, the program renames all the +variables in the rule with unique new names. The reason for this is to +prevent the variables for different rule applications from becoming +confused with each other. For instance, if two rules both use a +variable named `?x', then each one may add a binding for `?x' to the +frame when it is applied. These two `?x''s have nothing to do with +each other, and we should not be fooled into thinking that the two +bindings must be consistent. Rather than rename variables, we could +devise a more clever environment structure; however, the renaming +approach we have chosen here is the most straightforward, even if not +the most efficient. (See *Note Exercise 4-79::.) Here is the +`apply-a-rule' procedure: + + (define (apply-a-rule rule query-pattern query-frame) + (let ((clean-rule (rename-variables-in rule))) + (let ((unify-result + (unify-match query-pattern + (conclusion clean-rule) + query-frame))) + (if (eq? unify-result 'failed) + the-empty-stream + (qeval (rule-body clean-rule) + (singleton-stream unify-result)))))) + + The selectors `rule-body' and `conclusion' that extract parts of a +rule are defined in section *Note 4-4-4-7::. + + We generate unique variable names by associating a unique identifier +(such as a number) with each rule application and combining this +identifier with the original variable names. For example, if the +rule-application identifier is 7, we might change each `?x' in the rule +to `?x-7' and each `?y' in the rule to `?y-7'. (`Make-new-variable' and +`new-rule-application-id' are included with the syntax procedures in +section *Note 4-4-4-7::.) + + (define (rename-variables-in rule) + (let ((rule-application-id (new-rule-application-id))) + (define (tree-walk exp) + (cond ((var? exp) + (make-new-variable exp rule-application-id)) + ((pair? exp) + (cons (tree-walk (car exp)) + (tree-walk (cdr exp)))) + (else exp))) + (tree-walk rule))) + + The unification algorithm is implemented as a procedure that takes +as inputs two patterns and a frame and returns either the extended +frame or the symbol `failed'. The unifier is like the pattern matcher +except that it is symmetrical--variables are allowed on both sides of +the match. `Unify-match' is basically the same as `pattern-match', +except that there is extra code (marked "`***'" below) to handle the +case where the object on the right side of the match is a variable. + + (define (unify-match p1 p2 frame) + (cond ((eq? frame 'failed) 'failed) + ((equal? p1 p2) frame) + ((var? p1) (extend-if-possible p1 p2 frame)) + ((var? p2) (extend-if-possible p2 p1 frame)) ; *** + ((and (pair? p1) (pair? p2)) + (unify-match (cdr p1) + (cdr p2) + (unify-match (car p1) + (car p2) + frame))) + (else 'failed))) + + In unification, as in one-sided pattern matching, we want to accept +a proposed extension of the frame only if it is consistent with +existing bindings. The procedure `extend-if-possible' used in +unification is the same as the `extend-if-consistent' used in pattern +matching except for two special checks, marked "`***'" in the program +below. In the first case, if the variable we are trying to match is +not bound, but the value we are trying to match it with is itself a +(different) variable, it is necessary to check to see if the value is +bound, and if so, to match its value. If both parties to the match are +unbound, we may bind either to the other. + + The second check deals with attempts to bind a variable to a pattern +that includes that variable. Such a situation can occur whenever a +variable is repeated in both patterns. Consider, for example, unifying +the two patterns `(?x ?x)' and `(?y )' in a +frame where both `?x' and `?y' are unbound. First `?x' is matched +against `?y', making a binding of `?x' to `?y'. Next, the same `?x' is +matched against the given expression involving `?y'. Since `?x' is +already bound to `?y', this results in matching `?y' against the +expression. If we think of the unifier as finding a set of values for +the pattern variables that make the patterns the same, then these +patterns imply instructions to find a `?y' such that `?y' is equal to +the expression involving `?y'. There is no general method for solving +such equations, so we reject such bindings; these cases are recognized +by the predicate `depends-on?'.(1) On the other hand, we do not want +to reject attempts to bind a variable to itself. For example, consider +unifying `(?x ?x)' and `(?y ?y)'. The second attempt to bind `?x' to +`?y' matches `?y' (the stored value of `?x') against `?y' (the new +value of `?x'). This is taken care of by the `equal?' clause of +`unify-match'. + + (define (extend-if-possible var val frame) + (let ((binding (binding-in-frame var frame))) + (cond (binding + (unify-match + (binding-value binding) val frame)) + ((var? val) ; *** + (let ((binding (binding-in-frame val frame))) + (if binding + (unify-match + var (binding-value binding) frame) + (extend var val frame)))) + ((depends-on? val var frame) ; *** + 'failed) + (else (extend var val frame))))) + + `Depends-on?' is a predicate that tests whether an expression +proposed to be the value of a pattern variable depends on the variable. +This must be done relative to the current frame because the expression +may contain occurrences of a variable that already has a value that +depends on our test variable. The structure of `depends-on?' is a +simple recursive tree walk in which we substitute for the values of +variables whenever necessary. + + (define (depends-on? exp var frame) + (define (tree-walk e) + (cond ((var? e) + (if (equal? var e) + true + (let ((b (binding-in-frame e frame))) + (if b + (tree-walk (binding-value b)) + false)))) + ((pair? e) + (or (tree-walk (car e)) + (tree-walk (cdr e)))) + (else false))) + (tree-walk exp)) + + ---------- Footnotes ---------- + + (1) In general, unifying `?y' with an expression involving `?y' +would require our being able to find a fixed point of the equation `?y' += . It is sometimes possible to +syntactically form an expression that appears to be the solution. For +example, `?y' = `(f ?y)' seems to have the fixed point `(f (f (f ... +)))', which we can produce by beginning with the expression `(f ?y)' +and repeatedly substituting `(f ?y)' for `?y'. Unfortunately, not +every such equation has a meaningful fixed point. The issues that +arise here are similar to the issues of manipulating infinite series in +mathematics. For example, we know that 2 is the solution to the +equation y = 1 + y/2. Beginning with the expression 1 + y/2 and +repeatedly substituting 1 + y/2 for y gives + + 2 = y = 1 + y/2 = 1 + (1 + y/2)/2 = 1 + 1/2 + y/4 = ... + +which leads to + + 2 = 1 + 1/2 + 1/4 + 1/8 + ... + +However, if we try the same manipulation beginning with the observation +that - 1 is the solution to the equation y = 1 + 2y, we obtain + + -1 = y = 1 + 2y = 1 + 2(1 + 2y) = 1 + 2 + 4y = ... + +which leads to + + -1 = 1 + 2 + 4 + 8 + ... + +Although the formal manipulations used in deriving these two equations +are identical, the first result is a valid assertion about infinite +series but the second is not. Similarly, for our unification results, +reasoning with an arbitrary syntactically constructed expression may +lead to errors. + + +File: sicp.info, Node: 4-4-4-5, Next: 4-4-4-6, Prev: 4-4-4-4, Up: 4-4-4 + +4.4.4.5 Maintaining the Data Base +................................. + +One important problem in designing logic programming languages is that +of arranging things so that as few irrelevant data-base entries as +possible will be examined in checking a given pattern. In our system, +in addition to storing all assertions in one big stream, we store all +assertions whose `car's are constant symbols in separate streams, in a +table indexed by the symbol. To fetch an assertion that may match a +pattern, we first check to see if the `car' of the pattern is a +constant symbol. If so, we return (to be tested using the matcher) all +the stored assertions that have the same `car'. If the pattern's `car' +is not a constant symbol, we return all the stored assertions. +Cleverer methods could also take advantage of information in the frame, +or try also to optimize the case where the `car' of the pattern is not +a constant symbol. We avoid building our criteria for indexing (using +the `car', handling only the case of constant symbols) into the program; +instead we call on predicates and selectors that embody our criteria. + + (define THE-ASSERTIONS the-empty-stream) + + (define (fetch-assertions pattern frame) + (if (use-index? pattern) + (get-indexed-assertions pattern) + (get-all-assertions))) + + (define (get-all-assertions) THE-ASSERTIONS) + + (define (get-indexed-assertions pattern) + (get-stream (index-key-of pattern) 'assertion-stream)) + + `Get-stream' looks up a stream in the table and returns an empty +stream if nothing is stored there. + + (define (get-stream key1 key2) + (let ((s (get key1 key2))) + (if s s the-empty-stream))) + + Rules are stored similarly, using the `car' of the rule conclusion. +Rule conclusions are arbitrary patterns, however, so they differ from +assertions in that they can contain variables. A pattern whose `car' +is a constant symbol can match rules whose conclusions start with a +variable as well as rules whose conclusions have the same `car'. Thus, +when fetching rules that might match a pattern whose `car' is a +constant symbol we fetch all rules whose conclusions start with a +variable as well as those whose conclusions have the same `car' as the +pattern. For this purpose we store all rules whose conclusions start +with a variable in a separate stream in our table, indexed by the +symbol `?'. + + (define THE-RULES the-empty-stream) + + (define (fetch-rules pattern frame) + (if (use-index? pattern) + (get-indexed-rules pattern) + (get-all-rules))) + + (define (get-all-rules) THE-RULES) + + (define (get-indexed-rules pattern) + (stream-append + (get-stream (index-key-of pattern) 'rule-stream) + (get-stream '? 'rule-stream))) + + `Add-rule-or-assertion!' is used by `query-driver-loop' to add +assertions and rules to the data base. Each item is stored in the +index, if appropriate, and in a stream of all assertions or rules in +the data base. + + (define (add-rule-or-assertion! assertion) + (if (rule? assertion) + (add-rule! assertion) + (add-assertion! assertion))) + + (define (add-assertion! assertion) + (store-assertion-in-index assertion) + (let ((old-assertions THE-ASSERTIONS)) + (set! THE-ASSERTIONS + (cons-stream assertion old-assertions)) + 'ok)) + + (define (add-rule! rule) + (store-rule-in-index rule) + (let ((old-rules THE-RULES)) + (set! THE-RULES (cons-stream rule old-rules)) + 'ok)) + + To actually store an assertion or a rule, we check to see if it can +be indexed. If so, we store it in the appropriate stream. + + (define (store-assertion-in-index assertion) + (if (indexable? assertion) + (let ((key (index-key-of assertion))) + (let ((current-assertion-stream + (get-stream key 'assertion-stream))) + (put key + 'assertion-stream + (cons-stream assertion + current-assertion-stream)))))) + + (define (store-rule-in-index rule) + (let ((pattern (conclusion rule))) + (if (indexable? pattern) + (let ((key (index-key-of pattern))) + (let ((current-rule-stream + (get-stream key 'rule-stream))) + (put key + 'rule-stream + (cons-stream rule + current-rule-stream))))))) + + The following procedures define how the data-base index is used. A +pattern (an assertion or a rule conclusion) will be stored in the table +if it starts with a variable or a constant symbol. + + (define (indexable? pat) + (or (constant-symbol? (car pat)) + (var? (car pat)))) + + The key under which a pattern is stored in the table is either `?' +(if it starts with a variable) or the constant symbol with which it +starts. + + (define (index-key-of pat) + (let ((key (car pat))) + (if (var? key) '? key))) + + The index will be used to retrieve items that might match a pattern +if the pattern starts with a constant symbol. + + (define (use-index? pat) + (constant-symbol? (car pat))) + + *Exercise 4.70:* What is the purpose of the `let' bindings in the + procedures `add-assertion!' and `add-rule!' ? What would be wrong + with the following implementation of `add-assertion!' ? Hint: + Recall the definition of the infinite stream of ones in section + *Note 3-5-2::: `(define ones (cons-stream 1 ones))'. + + (define (add-assertion! assertion) + (store-assertion-in-index assertion) + (set! THE-ASSERTIONS + (cons-stream assertion THE-ASSERTIONS)) + 'ok) + + +File: sicp.info, Node: 4-4-4-6, Next: 4-4-4-7, Prev: 4-4-4-5, Up: 4-4-4 + +4.4.4.6 Stream Operations +......................... + +The query system uses a few stream operations that were not presented in +*Note Chapter 3::. + + `Stream-append-delayed' and `interleave-delayed' are just like +`stream-append' and `interleave' (section *Note 3-5-3::), except that +they take a delayed argument (like the `integral' procedure in section +*Note 3-5-4::). This postpones looping in some cases (see *Note +Exercise 4-71::). + + (define (stream-append-delayed s1 delayed-s2) + (if (stream-null? s1) + (force delayed-s2) + (cons-stream + (stream-car s1) + (stream-append-delayed (stream-cdr s1) delayed-s2)))) + + (define (interleave-delayed s1 delayed-s2) + (if (stream-null? s1) + (force delayed-s2) + (cons-stream + (stream-car s1) + (interleave-delayed (force delayed-s2) + (delay (stream-cdr s1)))))) + + `Stream-flatmap', which is used throughout the query evaluator to +map a procedure over a stream of frames and combine the resulting +streams of frames, is the stream analog of the `flatmap' procedure +introduced for ordinary lists in section *Note 2-2-3::. Unlike +ordinary `flatmap', however, we accumulate the streams with an +interleaving process, rather than simply appending them (see *Note +Exercise 4-72:: and *Note Exercise 4-73::). + + (define (stream-flatmap proc s) + (flatten-stream (stream-map proc s))) + + (define (flatten-stream stream) + (if (stream-null? stream) + the-empty-stream + (interleave-delayed + (stream-car stream) + (delay (flatten-stream (stream-cdr stream)))))) + + The evaluator also uses the following simple procedure to generate a +stream consisting of a single element: + + (define (singleton-stream x) + (cons-stream x the-empty-stream)) + + +File: sicp.info, Node: 4-4-4-7, Next: 4-4-4-8, Prev: 4-4-4-6, Up: 4-4-4 + +4.4.4.7 Query Syntax Procedures +............................... + +`Type' and `contents', used by `qeval' (section *Note 4-4-4-2::), +specify that a special form is identified by the symbol in its `car'. +They are the same as the `type-tag' and `contents' procedures in +section *Note 2-4-2::, except for the error message. + + (define (type exp) + (if (pair? exp) + (car exp) + (error "Unknown expression TYPE" exp))) + + (define (contents exp) + (if (pair? exp) + (cdr exp) + (error "Unknown expression CONTENTS" exp))) + + The following procedures, used by `query-driver-loop' (in section +*Note 4-4-4-1::), specify that rules and assertions are added to the +data base by expressions of the form `(assert! )': + + (define (assertion-to-be-added? exp) + (eq? (type exp) 'assert!)) + + (define (add-assertion-body exp) + (car (contents exp))) + + Here are the syntax definitions for the `and', `or', `not', and +`lisp-value' special forms (section *Note 4-4-4-2::): + + (define (empty-conjunction? exps) (null? exps)) + (define (first-conjunct exps) (car exps)) + (define (rest-conjuncts exps) (cdr exps)) + + (define (empty-disjunction? exps) (null? exps)) + (define (first-disjunct exps) (car exps)) + (define (rest-disjuncts exps) (cdr exps)) + + (define (negated-query exps) (car exps)) + + (define (predicate exps) (car exps)) + (define (args exps) (cdr exps)) + + The following three procedures define the syntax of rules: + + (define (rule? statement) + (tagged-list? statement 'rule)) + + (define (conclusion rule) (cadr rule)) + + (define (rule-body rule) + (if (null? (cddr rule)) + '(always-true) + (caddr rule))) + + `Query-driver-loop' (section *Note 4-4-4-1::) calls +`query-syntax-process' to transform pattern variables in the expression, +which have the form `?symbol', into the internal format `(? symbol)'. +That is to say, a pattern such as `(job ?x ?y)' is actually represented +internally by the system as `(job (? x) (? y))'. This increases the +efficiency of query processing, since it means that the system can +check to see if an expression is a pattern variable by checking whether +the `car' of the expression is the symbol `?', rather than having to +extract characters from the symbol. The syntax transformation is +accomplished by the following procedure:(1) + + (define (query-syntax-process exp) + (map-over-symbols expand-question-mark exp)) + + (define (map-over-symbols proc exp) + (cond ((pair? exp) + (cons (map-over-symbols proc (car exp)) + (map-over-symbols proc (cdr exp)))) + ((symbol? exp) (proc exp)) + (else exp))) + + (define (expand-question-mark symbol) + (let ((chars (symbol->string symbol))) + (if (string=? (substring chars 0 1) "?") + (list '? + (string->symbol + (substring chars 1 (string-length chars)))) + symbol))) + + Once the variables are transformed in this way, the variables in a +pattern are lists starting with `?', and the constant symbols (which +need to be recognized for data-base indexing, section *Note 4-4-4-5::) +are just the symbols. + + (define (var? exp) + (tagged-list? exp '?)) + + (define (constant-symbol? exp) (symbol? exp)) + + Unique variables are constructed during rule application (in section +*Note 4-4-4-4::) by means of the following procedures. The unique +identifier for a rule application is a number, which is incremented +each time a rule is applied. + + (define rule-counter 0) + + (define (new-rule-application-id) + (set! rule-counter (+ 1 rule-counter)) + rule-counter) + + (define (make-new-variable var rule-application-id) + (cons '? (cons rule-application-id (cdr var)))) + + When `query-driver-loop' instantiates the query to print the answer, +it converts any unbound pattern variables back to the right form for +printing, using + + (define (contract-question-mark variable) + (string->symbol + (string-append "?" + (if (number? (cadr variable)) + (string-append (symbol->string (caddr variable)) + "-" + (number->string (cadr variable))) + (symbol->string (cadr variable)))))) + + ---------- Footnotes ---------- + + (1) Most Lisp systems give the user the ability to modify the +ordinary `read' procedure to perform such transformations by defining "reader +macro characters". Quoted expressions are already handled in this way: +The reader automatically translates `'expression' into `(quote +expression)' before the evaluator sees it. We could arrange for +`?expression' to be transformed into `(? expression)' in the same way; +however, for the sake of clarity we have included the transformation +procedure here explicitly. + + `Expand-question-mark' and `contract-question-mark' use several +procedures with `string' in their names. These are Scheme primitives. + + +File: sicp.info, Node: 4-4-4-8, Prev: 4-4-4-7, Up: 4-4-4 + +4.4.4.8 Frames and Bindings +........................... + +Frames are represented as lists of bindings, which are variable-value +pairs: + + (define (make-binding variable value) + (cons variable value)) + + (define (binding-variable binding) + (car binding)) + + (define (binding-value binding) + (cdr binding)) + + (define (binding-in-frame variable frame) + (assoc variable frame)) + + (define (extend variable value frame) + (cons (make-binding variable value) frame)) + + *Exercise 4.71:* Louis Reasoner wonders why the `simple-query' and + `disjoin' procedures (section *Note 4-4-4-2::) are implemented + using explicit `delay' operations, rather than being defined as + follows: + + (define (simple-query query-pattern frame-stream) + (stream-flatmap + (lambda (frame) + (stream-append (find-assertions query-pattern frame) + (apply-rules query-pattern frame))) + frame-stream)) + + (define (disjoin disjuncts frame-stream) + (if (empty-disjunction? disjuncts) + the-empty-stream + (interleave + (qeval (first-disjunct disjuncts) frame-stream) + (disjoin (rest-disjuncts disjuncts) frame-stream)))) + + Can you give examples of queries where these simpler definitions + would lead to undesirable behavior? + + *Exercise 4.72:* Why do `disjoin' and `stream-flatmap' interleave + the streams rather than simply append them? Give examples that + illustrate why interleaving works better. (Hint: Why did we use + `interleave' in section *Note 3-5-3::?) + + *Exercise 4.73:* Why does `flatten-stream' use `delay' explicitly? + What would be wrong with defining it as follows: + + (define (flatten-stream stream) + (if (stream-null? stream) + the-empty-stream + (interleave + (stream-car stream) + (flatten-stream (stream-cdr stream))))) + + *Exercise 4.74:* Alyssa P. Hacker proposes to use a simpler + version of `stream-flatmap' in `negate', `lisp-value', and + `find-assertions'. She observes that the procedure that is mapped + over the frame stream in these cases always produces either the + empty stream or a singleton stream, so no interleaving is needed + when combining these streams. + + a. Fill in the missing expressions in Alyssa's program. + + (define (simple-stream-flatmap proc s) + (simple-flatten (stream-map proc s))) + + (define (simple-flatten stream) + (stream-map + (stream-filter stream))) + + b. Does the query system's behavior change if we change it in + this way? + + + *Exercise 4.75:* Implement for the query language a new special + form called `unique'. `Unique' should succeed if there is + precisely one item in the data base satisfying a specified query. + For example, + + (unique (job ?x (computer wizard))) + + should print the one-item stream + + (unique (job (Bitdiddle Ben) (computer wizard))) + + since Ben is the only computer wizard, and + + (unique (job ?x (computer programmer))) + + should print the empty stream, since there is more than one + computer programmer. Moreover, + + (and (job ?x ?j) (unique (job ?anyone ?j))) + + should list all the jobs that are filled by only one person, and + the people who fill them. + + There are two parts to implementing `unique'. The first is to + write a procedure that handles this special form, and the second + is to make `qeval' dispatch to that procedure. The second part is + trivial, since `qeval' does its dispatching in a data-directed + way. If your procedure is called `uniquely-asserted', all you + need to do is + + (put 'unique 'qeval uniquely-asserted) + + and `qeval' will dispatch to this procedure for every query whose + `type' (`car') is the symbol `unique'. + + The real problem is to write the procedure `uniquely-asserted'. + This should take as input the `contents' (`cdr') of the `unique' + query, together with a stream of frames. For each frame in the + stream, it should use `qeval' to find the stream of all extensions + to the frame that satisfy the given query. Any stream that does + not have exactly one item in it should be eliminated. The + remaining streams should be passed back to be accumulated into one + big stream that is the result of the `unique' query. This is + similar to the implementation of the `not' special form. + + Test your implementation by forming a query that lists all people + who supervise precisely one person. + + *Exercise 4.76:* Our implementation of `and' as a series + combination of queries (*Note Figure 4-5::) is elegant, but it is + inefficient because in processing the second query of the `and' we + must scan the data base for each frame produced by the first + query. If the data base has n elements, and a typical query + produces a number of output frames proportional to n (say n/k), + then scanning the data base for each frame produced by the first + query will require n^2/k calls to the pattern matcher. Another + approach would be to process the two clauses of the `and' + separately, then look for all pairs of output frames that are + compatible. If each query produces n/k output frames, then this + means that we must perform n^2/k^2 compatibility checks--a factor + of k fewer than the number of matches required in our current + method. + + Devise an implementation of `and' that uses this strategy. You + must implement a procedure that takes two frames as inputs, checks + whether the bindings in the frames are compatible, and, if so, + produces a frame that merges the two sets of bindings. This + operation is similar to unification. + + *Exercise 4.77:* In section *Note 4-4-3:: we saw that `not' and + `lisp-value' can cause the query language to give "wrong" answers + if these filtering operations are applied to frames in which + variables are unbound. Devise a way to fix this shortcoming. One + idea is to perform the filtering in a "delayed" manner by + appending to the frame a "promise" to filter that is fulfilled + only when enough variables have been bound to make the operation + possible. We could wait to perform filtering until all other + operations have been performed. However, for efficiency's sake, we + would like to perform filtering as soon as possible so as to cut + down on the number of intermediate frames generated. + + *Exercise 4.78:* Redesign the query language as a nondeterministic + program to be implemented using the evaluator of section *Note + 4-3::, rather than as a stream process. In this approach, each + query will produce a single answer (rather than the stream of all + answers) and the user can type `try-again' to see more answers. + You should find that much of the mechanism we built in this + section is subsumed by nondeterministic search and backtracking. + You will probably also find, however, that your new query language + has subtle differences in behavior from the one implemented here. + Can you find examples that illustrate this difference? + + *Exercise 4.79:* When we implemented the Lisp evaluator in section + *Note 4-1::, we saw how to use local environments to avoid name + conflicts between the parameters of procedures. For example, in + evaluating + + (define (square x) + (* x x)) + + (define (sum-of-squares x y) + (+ (square x) (square y))) + + (sum-of-squares 3 4) + + there is no confusion between the `x' in `square' and the `x' in + `sum-of-squares', because we evaluate the body of each procedure + in an environment that is specially constructed to contain + bindings for the local variables. In the query system, we used a + different strategy to avoid name conflicts in applying rules. + Each time we apply a rule we rename the variables with new names + that are guaranteed to be unique. The analogous strategy for the + Lisp evaluator would be to do away with local environments and + simply rename the variables in the body of a procedure each time + we apply the procedure. + + Implement for the query language a rule-application method that + uses environments rather than renaming. See if you can build on + your environment structure to create constructs in the query + language for dealing with large systems, such as the rule analog + of block-structured procedures. Can you relate any of this to the + problem of making deductions in a context (e.g., "If I supposed + that P were true, then I would be able to deduce A and B.") as a + method of problem solving? (This problem is open-ended. A good + answer is probably worth a Ph.D.) + + +File: sicp.info, Node: Chapter 5, Next: References, Prev: Chapter 4, Up: Top + +5 Computing with Register Machines +********************************** + + My aim is to show that the heavenly machine is not a kind of + divine, live being, but a kind of clockwork (and he who believes + that a clock has soul attributes the maker's glory to the work), + insofar as nearly all the manifold motions are caused by a most + simple and material force, just as all motions of the clock are + caused by a single weight. + + Johannes Kepler (letter to Herwart von Hohenburg, 1605) + + We began this book by studying processes and by describing processes +in terms of procedures written in Lisp. To explain the meanings of +these procedures, we used a succession of models of evaluation: the +substitution model of *Note Chapter 1::, the environment model of *Note +Chapter 3::, and the metacircular evaluator of *Note Chapter 4::. Our +examination of the metacircular evaluator, in particular, dispelled +much of the mystery of how Lisp-like languages are interpreted. But +even the metacircular evaluator leaves important questions unanswered, +because it fails to elucidate the mechanisms of control in a Lisp +system. For instance, the evaluator does not explain how the +evaluation of a subexpression manages to return a value to the +expression that uses this value, nor does the evaluator explain how +some recursive procedures generate iterative processes (that is, are +evaluated using constant space) whereas other recursive procedures +generate recursive processes. These questions remain unanswered +because the metacircular evaluator is itself a Lisp program and hence +inherits the control structure of the underlying Lisp system. In order +to provide a more complete description of the control structure of the +Lisp evaluator, we must work at a more primitive level than Lisp itself. + + In this chapter we will describe processes in terms of the +step-by-step operation of a traditional computer. Such a computer, or machine +"register machine", sequentially executes "instructions" that +manipulate the contents of a fixed set of storage elements called "registers". +A typical register-machine instruction applies a primitive operation +to the contents of some registers and assigns the result to another +register. Our descriptions of processes executed by register machines +will look very much like "machine-language" programs for traditional +computers. However, instead of focusing on the machine language of any +particular computer, we will examine several Lisp procedures and design +a specific register machine to execute each procedure. Thus, we will +approach our task from the perspective of a hardware architect rather +than that of a machine-language computer programmer. In designing +register machines, we will develop mechanisms for implementing +important programming constructs such as recursion. We will also +present a language for describing designs for register machines. In +section *Note 5-2:: we will implement a Lisp program that uses these +descriptions to simulate the machines we design. + + Most of the primitive operations of our register machines are very +simple. For example, an operation might add the numbers fetched from +two registers, producing a result to be stored into a third register. +Such an operation can be performed by easily described hardware. In +order to deal with list structure, however, we will also use the memory +operations `car', `cdr', and `cons', which require an elaborate +storage-allocation mechanism. In section *Note 5-3:: we study their +implementation in terms of more elementary operations. + + In section *Note 5-4::, after we have accumulated experience +formulating simple procedures as register machines, we will design a +machine that carries out the algorithm described by the metacircular +evaluator of section *Note 4-1::. This will fill in the gap in our +understanding of how Scheme expressions are interpreted, by providing +an explicit model for the mechanisms of control in the evaluator. In +section *Note 5-5:: we will study a simple compiler that translates +Scheme programs into sequences of instructions that can be executed +directly with the registers and operations of the evaluator register +machine. + +* Menu: + +* 5-1:: Designing Register Machines +* 5-2:: A Register-Machine Simulator +* 5-3:: Storage Allocation and Garbage Collection +* 5-4:: The Explicit-Control Evaluator +* 5-5:: Compilation + + +File: sicp.info, Node: 5-1, Next: 5-2, Prev: Chapter 5, Up: Chapter 5 + +5.1 Designing Register Machines +=============================== + +To design a register machine, we must design its "data paths" +(registers and operations) and the "controller" that sequences these +operations. To illustrate the design of a simple register machine, let +us examine Euclid's Algorithm, which is used to compute the greatest +common divisor (GCD) of two integers. As we saw in section *Note +1-2-5::, Euclid's Algorithm can be carried out by an iterative process, +as specified by the following procedure: + + (define (gcd a b) + (if (= b 0) + a + (gcd b (remainder a b)))) + + A machine to carry out this algorithm must keep track of two +numbers, a and b, so let us assume that these numbers are stored in two +registers with those names. The basic operations required are testing +whether the contents of register `b' is zero and computing the +remainder of the contents of register `a' divided by the contents of +register `b'. The remainder operation is a complex process, but assume +for the moment that we have a primitive device that computes +remainders. On each cycle of the GCD algorithm, the contents of +register `a' must be replaced by the contents of register `b', and the +contents of `b' must be replaced by the remainder of the old contents +of `a' divided by the old contents of `b'. It would be convenient if +these replacements could be done simultaneously, but in our model of +register machines we will assume that only one register can be assigned +a new value at each step. To accomplish the replacements, our machine +will use a third "temporary" register, which we call `t'. (First the +remainder will be placed in `t', then the contents of `b' will be +placed in `a', and finally the remainder stored in `t' will be placed +in `b'.) + + We can illustrate the registers and operations required for this +machine by using the data-path diagram shown in *Note Figure 5-1::. In +this diagram, the registers (`a', `b', and `t') are represented by +rectangles. Each way to assign a value to a register is indicated by +an arrow with an `X' behind the head, pointing from the source of data +to the register. We can think of the `X' as a button that, when +pushed, allows the value at the source to "flow" into the designated +register. The label next to each button is the name we will use to +refer to the button. The names are arbitrary, and can be chosen to +have mnemonic value (for example, `a<-b' denotes pushing the button +that assigns the contents of register `b' to register `a'). The source +of data for a register can be another register (as in the `a<-b' +assignment), an operation result (as in the `t<-r' assignment), or a +constant (a built-in value that cannot be changed, represented in a +data-path diagram by a triangle containing the constant). + + An operation that computes a value from constants and the contents +of registers is represented in a data-path diagram by a trapezoid +containing a name for the operation. For example, the box marked `rem' +in *Note Figure 5-1:: represents an operation that computes the +remainder of the contents of the registers `a' and `b' to which it is +attached. Arrows (without buttons) point from the input registers and +constants to the box, and arrows connect the operation's output value +to registers. A test is represented by a circle containing a name for +the test. For example, our GCD machine has an operation that tests +whether the contents of register `b' is zero. A test also has arrows +from its input registers and constants, but it has no output arrows; +its value is used by the controller rather than by the data paths. +Overall, the data-path diagram shows the registers and operations that +are required for the machine and how they must be connected. If we +view the arrows as wires and the `X' buttons as switches, the data-path +diagram is very like the wiring diagram for a machine that could be +constructed from electrical components. + + *Figure 5.1:* Data paths for a GCD machine. + + ___ + +-----+ +-----+ / \ + | a |<--(X)----| b +--->| = | + +--+--+ a<-b +-+---+ \___/ + | | ^ ^ + +------+ +----+ | | + | | (X) b<-t | + .--+---+--. | / \ + \ rem / | / O \ + \_____/ | +-----+ + | | + (X) t<-r | + | | + V | + +-----+ | + | t +------+ + +-----+ + + In order for the data paths to actually compute GCDs, the buttons +must be pushed in the correct sequence. We will describe this sequence +in terms of a controller diagram, as illustrated in *Note Figure 5-2::. +The elements of the controller diagram indicate how the data-path +components should be operated. The rectangular boxes in the controller +diagram identify data-path buttons to be pushed, and the arrows +describe the sequencing from one step to the next. The diamond in the +diagram represents a decision. One of the two sequencing arrows will +be followed, depending on the value of the data-path test identified in +the diamond. We can interpret the controller in terms of a physical +analogy: Think of the diagram as a maze in which a marble is rolling. +When the marble rolls into a box, it pushes the data-path button that +is named by the box. When the marble rolls into a decision node (such +as the test for `b' = 0), it leaves the node on the path determined by +the result of the indicated test. Taken together, the data paths and +the controller completely describe a machine for computing GCDs. We +start the controller (the rolling marble) at the place marked `start', +after placing numbers in registers `a' and `b'. When the controller +reaches `done', we will find the value of the GCD in register `a'. + + *Figure 5.2:* Controller for a GCD machine. + + start + | + V + / \ yes + +--->< = >-----> done + | \ / + | | no + | V + | +------+ + | | t<-r | + | +---+--+ + | | + | V + | +------+ + | | a<-b | + | +---+--+ + | | + | V + | +------+ + +--+ b<-t | + +------+ + + *Exercise 5.1:* Design a register machine to compute factorials + using the iterative algorithm specified by the following + procedure. Draw data-path and controller diagrams for this + machine. + + (define (factorial n) + (define (iter product counter) + (if (> counter n) + product + (iter (* counter product) + (+ counter 1)))) + (iter 1 1)) + +* Menu: + +* 5-1-1:: A Language for Describing Register Machines +* 5-1-2:: Abstraction in Machine Design +* 5-1-3:: Subroutines +* 5-1-4:: Using a Stack to Implement Recursion +* 5-1-5:: Instruction Summary + + +File: sicp.info, Node: 5-1-1, Next: 5-1-2, Prev: 5-1, Up: 5-1 + +5.1.1 A Language for Describing Register Machines +------------------------------------------------- + +Data-path and controller diagrams are adequate for representing simple +machines such as GCD, but they are unwieldy for describing large +machines such as a Lisp interpreter. To make it possible to deal with +complex machines, we will create a language that presents, in textual +form, all the information given by the data-path and controller +diagrams. We will start with a notation that directly mirrors the +diagrams. + + We define the data paths of a machine by describing the registers +and the operations. To describe a register, we give it a name and +specify the buttons that control assignment to it. We give each of +these buttons a name and specify the source of the data that enters the +register under the button's control. (The source is a register, a +constant, or an operation.) To describe an operation, we give it a +name and specify its inputs (registers or constants). + + We define the controller of a machine as a sequence of "instructions" +together with "labels" that identify "entry points" in the sequence. An +instruction is one of the following: + + * The name of a data-path button to push to assign a value to a + register. (This corresponds to a box in the controller diagram.) + + * A `test' instruction, that performs a specified test. + + * A conditional branch (`branch' instruction) to a location + indicated by a controller label, based on the result of the + previous test. (The test and branch together correspond to a + diamond in the controller diagram.) If the test is false, the + controller should continue with the next instruction in the + sequence. Otherwise, the controller should continue with the + instruction after the label. + + * An unconditional branch (`goto' instruction) naming a controller + label at which to continue execution. + + + The machine starts at the beginning of the controller instruction +sequence and stops when execution reaches the end of the sequence. +Except when a branch changes the flow of control, instructions are +executed in the order in which they are listed. + + *Figure 5.3:* A specification of the GCD machine. + + (data-paths + (registers + ((name a) + (buttons ((name a<-b) (source (register b))))) + ((name b) + (buttons ((name b<-t) (source (register t))))) + ((name t) + (buttons ((name t<-r) (source (operation rem)))))) + + (operations + ((name rem) + (inputs (register a) (register b))) + ((name =) + (inputs (register b) (constant 0))))) + + (controller + test-b ; label + (test =) ; test + (branch (label gcd-done)) ; conditional branch + (t<-r) ; button push + (a<-b) ; button push + (b<-t) ; button push + (goto (label test-b)) ; unconditional branch + gcd-done) ; label + + + *Note Figure 5-3:: shows the GCD machine described in this way. This +example only hints at the generality of these descriptions, since the +GCD machine is a very simple case: Each register has only one button, +and each button and test is used only once in the controller. + + Unfortunately, it is difficult to read such a description. In order +to understand the controller instructions we must constantly refer back +to the definitions of the button names and the operation names, and to +understand what the buttons do we may have to refer to the definitions +of the operation names. We will thus transform our notation to combine +the information from the data-path and controller descriptions so that +we see it all together. + + To obtain this form of description, we will replace the arbitrary +button and operation names by the definitions of their behavior. That +is, instead of saying (in the controller) "Push button `t<-r'" and +separately saying (in the data paths) "Button `t<-r' assigns the value +of the `rem' operation to register `t'" and "The `rem' operation's +inputs are the contents of registers `a' and `b'," we will say (in the +controller) "Push the button that assigns to register `t' the value of +the `rem' operation on the contents of registers `a' and `b'." +Similarly, instead of saying (in the controller) "Perform the `=' test" +and separately saying (in the data paths) "The `=' test operates on the +contents of register `b' and the constant 0," we will say "Perform the +`=' test on the contents of register `b' and the constant 0." We will +omit the data-path description, leaving only the controller sequence. +Thus, the GCD machine is described as follows: + + (controller + test-b + (test (op =) (reg b) (const 0)) + (branch (label gcd-done)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label test-b)) + gcd-done) + + This form of description is easier to read than the kind illustrated +in *Note Figure 5-3::, but it also has disadvantages: + + * It is more verbose for large machines, because complete + descriptions of the data-path elements are repeated whenever the + elements are mentioned in the controller instruction sequence. + (This is not a problem in the GCD example, because each operation + and button is used only once.) Moreover, repeating the data-path + descriptions obscures the actual data-path structure of the + machine; it is not obvious for a large machine how many registers, + operations, and buttons there are and how they are interconnected. + + * Because the controller instructions in a machine definition look + like Lisp expressions, it is easy to forget that they are not + arbitrary Lisp expressions. They can notate only legal machine + operations. For example, operations can operate directly only on + constants and the contents of registers, not on the results of + other operations. + + + In spite of these disadvantages, we will use this register-machine +language throughout this chapter, because we will be more concerned +with understanding controllers than with understanding the elements and +connections in data paths. We should keep in mind, however, that +data-path design is crucial in designing real machines. + + *Exercise 5.2:* Use the register-machine language to describe the + iterative factorial machine of *Note Exercise 5-1::. + +Actions +....... + +Let us modify the GCD machine so that we can type in the numbers whose +GCD we want and get the answer printed at our terminal. We will not +discuss how to make a machine that can read and print, but will assume +(as we do when we use `read' and `display' in Scheme) that they are +available as primitive operations.(1) + + `Read' is like the operations we have been using in that it produces +a value that can be stored in a register. But `read' does not take +inputs from any registers; its value depends on something that happens +outside the parts of the machine we are designing. We will allow our +machine's operations to have such behavior, and thus will draw and +notate the use of `read' just as we do any other operation that +computes a value. + + `Print', on the other hand, differs from the operations we have been +using in a fundamental way: It does not produce an output value to be +stored in a register. Though it has an effect, this effect is not on a +part of the machine we are designing. We will refer to this kind of +operation as an "action". We will represent an action in a data-path +diagram just as we represent an operation that computes a value--as a +trapezoid that contains the name of the action. Arrows point to the +action box from any inputs (registers or constants). We also associate +a button with the action. Pushing the button makes the action happen. +To make a controller push an action button we use a new kind of +instruction called `perform'. Thus, the action of printing the +contents of register `a' is represented in a controller sequence by the +instruction + + (perform (op print) (reg a)) + + *Note Figure 5-4:: shows the data paths and controller for the new +GCD machine. Instead of having the machine stop after printing the +answer, we have made it start over, so that it repeatedly reads a pair +of numbers, computes their GCD, and prints the result. This structure +is like the driver loops we used in the interpreters of *Note Chapter +4::. + + *Figure 5.4:* A GCD machine that reads inputs and prints results. + + .--------. + \ read / + \____/ + | + +-------*------+ + | | + a<-rd (X) (X) b<-rd + | | + V V ___ + +-----+ +-----+ / \ + | a |<--(X)--+ b +----->| = | + +-+-+-+ a<-b +-+---+ \___/ + | | | ^ ^ + +--+ +----+ +--+ | | + | | | (X) b<-t / \ + V V V | / O \ + .---------. .---------. | /_____\ + --(X)->\ print / \ rem / | + P \_____/ \_____/ | + | | + (X) t<-r | + | | + V | + +-----+ | + | t +----+ + +-----+ + + (controller + gcd-loop + (assign a (op read)) + (assign b (op read)) + test-b + (test (op =) (reg b) (const 0)) + (branch (label gcd-done)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label test-b)) + gcd-done + (perform (op print) (reg a)) + (goto (label gcd-loop))) + + + ---------- Footnotes ---------- + + (1) This assumption glosses over a great deal of complexity. +Usually a large portion of the implementation of a Lisp system is +dedicated to making reading and printing work. + + +File: sicp.info, Node: 5-1-2, Next: 5-1-3, Prev: 5-1-1, Up: 5-1 + +5.1.2 Abstraction in Machine Design +----------------------------------- + +We will often define a machine to include "primitive" operations that +are actually very complex. For example, in sections *Note 5-4:: and +*Note 5-5:: we will treat Scheme's environment manipulations as +primitive. Such abstraction is valuable because it allows us to ignore +the details of parts of a machine so that we can concentrate on other +aspects of the design. The fact that we have swept a lot of complexity +under the rug, however, does not mean that a machine design is +unrealistic. We can always replace the complex "primitives" by simpler +primitive operations. + + Consider the GCD machine. The machine has an instruction that +computes the remainder of the contents of registers `a' and `b' and +assigns the result to register `t'. If we want to construct the GCD +machine without using a primitive remainder operation, we must specify +how to compute remainders in terms of simpler operations, such as +subtraction. Indeed, we can write a Scheme procedure that finds +remainders in this way: + + (define (remainder n d) + (if (< n d) + n + (remainder (- n d) d))) + + We can thus replace the remainder operation in the GCD machine's data +paths with a subtraction operation and a comparison test. *Note Figure +5-5:: shows the data paths and controller for the elaborated machine. +The instruction + + *Figure 5.5:* Data paths and controller for the elaborated GCD + machine. + + ___ + +-----+ +-----+ / \ + | a |<--(X)---+ b +-------*-->| = | + +--+--+ a<-b +-+---+ | \___/ + | | ^ | + (X) t<-a | | | + | | (X) b<-t | + V | | _V_ + +-----+ | | / \ + | t +-------*---|--*-----| < | + +-----+ | | \___/ + ^ V V + | --------- + (X) t<-d \ - / + | --+-- + | | + +------------+ + + + start + | + V + / \ yes +-------+ + +->< = >----> done | t<-d |<--+ + | \ / +---+---+ | + | | no | | + | | V | + | | +------+ / \ no | + | +-->| t<-a +------->< < >-----+ + | +------+ \ / + | | yes + | +-------------------+ + | V + | +-------+ + | | a<-b | + | +---+---+ + | | + | V + | +-------+ + +--+ b<-t | + +-------+ + + (assign t (op rem) (reg a) (reg b)) + +in the GCD controller definition is replaced by a sequence of +instructions that contains a loop, as shown in *Note Figure 5-6::. + + *Figure 5.6:* Controller instruction sequence for the GCD machine + in *Note Figure 5-5::. + + (controller + test-b + (test (op =) (reg b) (const 0)) + (branch (label gcd-done)) + (assign t (reg a)) + rem-loop + (test (op <) (reg t) (reg b)) + (branch (label rem-done)) + (assign t (op -) (reg t) (reg b)) + (goto (label rem-loop)) + rem-done + (assign a (reg b)) + (assign b (reg t)) + (goto (label test-b)) + gcd-done) + + + *Exercise 5.3:* Design a machine to compute square roots using + Newton's method, as described in section *Note 1-1-7::: + + (define (sqrt x) + (define (good-enough? guess) + (< (abs (- (square guess) x)) 0.001)) + (define (improve guess) + (average guess (/ x guess))) + (define (sqrt-iter guess) + (if (good-enough? guess) + guess + (sqrt-iter (improve guess)))) + (sqrt-iter 1.0)) + + Begin by assuming that `good-enough?' and `improve' operations are + available as primitives. Then show how to expand these in terms + of arithmetic operations. Describe each version of the `sqrt' + machine design by drawing a data-path diagram and writing a + controller definition in the register-machine language. + + +File: sicp.info, Node: 5-1-3, Next: 5-1-4, Prev: 5-1-2, Up: 5-1 + +5.1.3 Subroutines +----------------- + +When designing a machine to perform a computation, we would often +prefer to arrange for components to be shared by different parts of the +computation rather than duplicate the components. Consider a machine +that includes two GCD computations--one that finds the GCD of the +contents of registers `a' and `b' and one that finds the GCD of the +contents of registers `c' and `d'. We might start by assuming we have +a primitive `gcd' operation, then expand the two instances of `gcd' in +terms of more primitive operations. *Note Figure 5-7:: shows just the +GCD portions of the resulting machine's data paths, without showing how +they connect to the rest of the machine. The figure also shows the +corresponding portions of the machine's controller sequence. + + *Figure 5.7:* Portions of the data paths and controller sequence + for a machine with two GCD computations. + + ___ ___ + +-----+ +-----+ / \ +-----+ +-----+ / \ + | a |<-(X)---+ b |--->| = | | c |<-(X)---+ d |--->| = | + +--+--+ a<-b ++----+ \___/ +--+--+ c<-d ++----+ \___/ + | | ^ ^ | | ^ ^ + `----. .---' | | `----. .---' | | + V V (X) b<-t | V V (X) d<-t | + ------- | / \ ------- | / \ + \ rem / | /_0_\ \ rem / | /_0_\ + --+-- | --+-- | + | | | | + (X) t<-r | (X) s<-r | + | | | | + V | V | + +-----+ | +-----+ | + | t +-----' | s +-----' + +-----+ +-----+ + + gcd-1 + (test (op =) (reg b) (const 0)) + (branch (label after-gcd-1)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label gcd-1)) + after-gcd-1 + ... + gcd-2 + (test (op =) (reg d) (const 0)) + (branch (label after-gcd-2)) + (assign s (op rem) (reg c) (reg d)) + (assign c (reg d)) + (assign d (reg s)) + (goto (label gcd-2)) + after-gcd-2 + + + This machine has two remainder operation boxes and two boxes for +testing equality. If the duplicated components are complicated, as is +the remainder box, this will not be an economical way to build the +machine. We can avoid duplicating the data-path components by using +the same components for both GCD computations, provided that doing so +will not affect the rest of the larger machine's computation. If the +values in registers `a' and `b' are not needed by the time the +controller gets to `gcd-2' (or if these values can be moved to other +registers for safekeeping), we can change the machine so that it uses +registers `a' and `b', rather than registers `c' and `d', in computing +the second GCD as well as the first. If we do this, we obtain the +controller sequence shown in *Note Figure 5-8::. + + We have removed the duplicate data-path components (so that the data +paths are again as in *Note Figure 5-1::), but the controller now has +two GCD sequences that differ only in their entry-point labels. It +would be better to replace these two sequences by branches to a single +sequence--a `gcd' "subroutine"--at the end of which we branch back to +the correct place in the main instruction sequence. We can accomplish +this as follows: Before branching to `gcd', we place a distinguishing +value (such as 0 or 1) into a special register, `continue'. At the end +of the `gcd' subroutine we return either to `after-gcd-1' or to +`after-gcd-2', depending on the value of the `continue' register. +*Note Figure 5-9:: shows the relevant portion of the resulting +controller sequence, which includes only a single copy of the `gcd' +instructions. + + *Figure 5.8:* Portions of the controller sequence for a machine + that uses the same data-path components for two different GCD + computations. + + gcd-1 + (test (op =) (reg b) (const 0)) + (branch (label after-gcd-1)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label gcd-1)) + after-gcd-1 + ... + gcd-2 + (test (op =) (reg b) (const 0)) + (branch (label after-gcd-2)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label gcd-2)) + after-gcd-2 + + + *Figure 5.9:* Using a `continue' register to avoid the duplicate + controller sequence in *Note Figure 5-8::. + + gcd + (test (op =) (reg b) (const 0)) + (branch (label gcd-done)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label gcd)) + gcd-done + (test (op =) (reg continue) (const 0)) + (branch (label after-gcd-1)) + (goto (label after-gcd-2)) + ... + ;; Before branching to `gcd' from the first place where + ;; it is needed, we place 0 in the `continue' register + (assign continue (const 0)) + (goto (label gcd)) + after-gcd-1 + ... + ;; Before the second use of `gcd', we place 1 in the `continue' register + (assign continue (const 1)) + (goto (label gcd)) + after-gcd-2 + + + *Figure 5.10:* Assigning labels to the `continue' register + simplifies and generalizes the strategy shown in *Note Figure + 5-9::. + + gcd + (test (op =) (reg b) (const 0)) + (branch (label gcd-done)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label gcd)) + gcd-done + (goto (reg continue)) + ... + ;; Before calling `gcd', we assign to `continue' + ;; the label to which `gcd' should return. + (assign continue (label after-gcd-1)) + (goto (label gcd)) + after-gcd-1 + ... + ;; Here is the second call to `gcd', with a different continuation. + (assign continue (label after-gcd-2)) + (goto (label gcd)) + after-gcd-2 + + + This is a reasonable approach for handling small problems, but it +would be awkward if there were many instances of GCD computations in the +controller sequence. To decide where to continue executing after the +`gcd' subroutine, we would need tests in the data paths and branch +instructions in the controller for all the places that use `gcd'. A +more powerful method for implementing subroutines is to have the +`continue' register hold the label of the entry point in the controller +sequence at which execution should continue when the subroutine is +finished. Implementing this strategy requires a new kind of connection +between the data paths and the controller of a register machine: There +must be a way to assign to a register a label in the controller +sequence in such a way that this value can be fetched from the register +and used to continue execution at the designated entry point. + + To reflect this ability, we will extend the `assign' instruction of +the register-machine language to allow a register to be assigned as +value a label from the controller sequence (as a special kind of +constant). We will also extend the `goto' instruction to allow +execution to continue at the entry point described by the contents of a +register rather than only at an entry point described by a constant +label. Using these new constructs we can terminate the `gcd' +subroutine with a branch to the location stored in the `continue' +register. This leads to the controller sequence shown in *Note Figure +5-10::. + + A machine with more than one subroutine could use multiple +continuation registers (e.g., `gcd-continue', `factorial-continue') or +we could have all subroutines share a single `continue' register. +Sharing is more economical, but we must be careful if we have a +subroutine (`sub1') that calls another subroutine (`sub2'). Unless +`sub1' saves the contents of `continue' in some other register before +setting up `continue' for the call to `sub2', `sub1' will not know +where to go when it is finished. The mechanism developed in the next +section to handle recursion also provides a better solution to this +problem of nested subroutine calls. + + +File: sicp.info, Node: 5-1-4, Next: 5-1-5, Prev: 5-1-3, Up: 5-1 + +5.1.4 Using a Stack to Implement Recursion +------------------------------------------ + +With the ideas illustrated so far, we can implement any iterative +process by specifying a register machine that has a register +corresponding to each state variable of the process. The machine +repeatedly executes a controller loop, changing the contents of the +registers, until some termination condition is satisfied. At each +point in the controller sequence, the state of the machine +(representing the state of the iterative process) is completely +determined by the contents of the registers (the values of the state +variables). + + Implementing recursive processes, however, requires an additional +mechanism. Consider the following recursive method for computing +factorials, which we first examined in section *Note 1-2-1::: + + (define (factorial n) + (if (= n 1) + 1 + (* (factorial (- n 1)) n))) + + As we see from the procedure, computing n! requires computing (n - +1)!. Our GCD machine, modeled on the procedure + + (define (gcd a b) + (if (= b 0) + a + (gcd b (remainder a b)))) + +similarly had to compute another GCD. But there is an important +difference between the `gcd' procedure, which reduces the original +computation to a new GCD computation, and `factorial', which requires +computing another factorial as a subproblem. In GCD, the answer to the +new GCD computation is the answer to the original problem. To compute +the next GCD, we simply place the new arguments in the input registers +of the GCD machine and reuse the machine's data paths by executing the +same controller sequence. When the machine is finished solving the +final GCD problem, it has completed the entire computation. + + In the case of factorial (or any recursive process) the answer to +the new factorial subproblem is not the answer to the original problem. +The value obtained for (n - 1)! must be multiplied by n to get the +final answer. If we try to imitate the GCD design, and solve the +factorial subproblem by decrementing the `n' register and rerunning the +factorial machine, we will no longer have available the old value of +`n' by which to multiply the result. We thus need a second factorial +machine to work on the subproblem. This second factorial computation +itself has a factorial subproblem, which requires a third factorial +machine, and so on. Since each factorial machine contains another +factorial machine within it, the total machine contains an infinite +nest of similar machines and hence cannot be constructed from a fixed, +finite number of parts. + + Nevertheless, we can implement the factorial process as a register +machine if we can arrange to use the same components for each nested +instance of the machine. Specifically, the machine that computes n! +should use the same components to work on the subproblem of computing +(n - 1)!, on the subproblem for (n - 2)!, and so on. This is plausible +because, although the factorial process dictates that an unbounded +number of copies of the same machine are needed to perform a +computation, only one of these copies needs to be active at any given +time. When the machine encounters a recursive subproblem, it can +suspend work on the main problem, reuse the same physical parts to work +on the subproblem, then continue the suspended computation. + + In the subproblem, the contents of the registers will be different +than they were in the main problem. (In this case the `n' register is +decremented.) In order to be able to continue the suspended +computation, the machine must save the contents of any registers that +will be needed after the subproblem is solved so that these can be +restored to continue the suspended computation. In the case of +factorial, we will save the old value of `n', to be restored when we +are finished computing the factorial of the decremented `n' register.(1) + + Since there is no _a priori_ limit on the depth of nested recursive +calls, we may need to save an arbitrary number of register values. +These values must be restored in the reverse of the order in which they +were saved, since in a nest of recursions the last subproblem to be +entered is the first to be finished. This dictates the use of a "stack", +or "last in, first out" data structure, to save register values. We +can extend the register-machine language to include a stack by adding +two kinds of instructions: Values are placed on the stack using a +`save' instruction and restored from the stack using a `restore' +instruction. After a sequence of values has been `save'd on the stack, +a sequence of `restore's will retrieve these values in reverse order.(2) + + With the aid of the stack, we can reuse a single copy of the +factorial machine's data paths for each factorial subproblem. There is +a similar design issue in reusing the controller sequence that operates +the data paths. To reexecute the factorial computation, the controller +cannot simply loop back to the beginning, as with an iterative process, +because after solving the (n - 1)! subproblem the machine must still +multiply the result by n. The controller must suspend its computation +of n!, solve the (n - 1)! subproblem, then continue its computation of +n!. This view of the factorial computation suggests the use of the +subroutine mechanism described in section *Note 5-1-3::, which has the +controller use a `continue' register to transfer to the part of the +sequence that solves a subproblem and then continue where it left off +on the main problem. We can thus make a factorial subroutine that +returns to the entry point stored in the `continue' register. Around +each subroutine call, we save and restore `continue' just as we do the +`n' register, since each "level" of the factorial computation will use +the same `continue' register. That is, the factorial subroutine must +put a new value in `continue' when it calls itself for a subproblem, +but it will need the old value in order to return to the place that +called it to solve a subproblem. + + *Note Figure 5-11:: shows the data paths and controller for a +machine that implements the recursive `factorial' procedure. The +machine has a stack and three registers, called `n', `val', and +`continue'. To simplify the data-path diagram, we have not named the +register-assignment buttons, only the stack-operation buttons (`sc' and +`sn' to save registers, `rc' and `rn' to restore registers). To +operate the machine, we put in register `n' the number whose factorial +we wish to compute and start the machine. When the machine reaches +`fact-done', the computation is finished and the answer will be found +in the `val' register. In the controller sequence, `n' and `continue' +are saved before each recursive call and restored upon return from the +call. Returning from a call is accomplished by branching to the +location stored in `continue'. `Continue' is initialized when the +machine starts so that the last return will go to `fact-done'. The +`val' register, which holds the result of the factorial computation, is +not saved before the recursive call, because the old contents of `val' +is not useful after the subroutine returns. Only the new value, which +is the value produced by the subcomputation, is needed. + + Although in principle the factorial computation requires an infinite +machine, the machine in *Note Figure 5-11:: is actually finite except +for the stack, which is potentially unbounded. Any particular physical +implementation of a stack, however, will be of finite size, and this +will limit the depth of recursive calls that can be handled by the +machine. This implementation of factorial illustrates the general +strategy for realizing recursive algorithms as ordinary register +machines augmented by stacks. When a recursive subproblem is +encountered, we save on the stack the registers whose current values +will be required after the subproblem is solved, solve the recursive +subproblem, then restore the saved registers and continue execution on +the main problem. The `continue' register must always be saved. +Whether there are other registers that need to be saved depends on the +particular machine, since not all recursive computations need the +original values of registers that are modified during solution of the +subproblem (see *Note Exercise 5-4::). + +A double recursion +.................. + +Let us examine a more complex recursive process, the tree-recursive +computation of the Fibonacci numbers, which we introduced in section +*Note 1-2-2::: + + (define (fib n) + (if (< n 2) + n + (+ (fib (- n 1)) (fib (- n 2))))) + + Just as with factorial, we can implement the recursive Fibonacci +computation as a register machine with registers `n', `val', and +`continue'. The machine is more complex than the one for factorial, +because there are two places in the controller sequence where we need +to perform recursive calls--once to compute Fib(n - 1) and once to +compute Fib(n - 2). To set up for each of these calls, we save the +registers whose values will be needed later, set the `n' register to +the number whose Fib we need to compute recursively (n - 1 or n - 2), +and assign to `continue' the entry point in the main sequence to which +to return (`afterfib-n-1' or `afterfib-n-2', respectively). We then go +to `fib-loop'. When we return from the recursive call, the answer is +in `val'. *Note Figure 5-12:: shows the controller sequence for this +machine. + + *Figure 5.11:* A recursive factorial machine. + + ___ + / \ + +----------*-----------| = | + | | \___/ + (X) | ^ + | | | + V | +---+---+ sn +-------+ + +-------+ | | +---(X)-->| | + | val |<-(X)-|----------+ n | | stack | + +-----+-+ | | |<--(X)---+ | + ^ | | +-------+ rn +-+-----+ + | | | ^ | ^ + (X) | | | | | + | | +----|--------* (X) | (X) sc + | | | | | | rc (X) | + | | | *----. | | | | + | V V | V V | V | + | ------- | ------- | +------+-+ + | \ * / | \ - / | |continue+--> controller + | --+-- | --+-- | +--------+ + | | | | | ^ ^ + +-----+ | +-----+ | | + | (X) (X) + | | | + / \ after- / \ / \ fact- + /_1_\ fact /___\ /___\ done + + (controller + (assign continue (label fact-done)) ; set up final return address + fact-loop + (test (op =) (reg n) (const 1)) + (branch (label base-case)) + ;; Set up for the recursive call by saving `n' and `continue'. + ;; Set up `continue' so that the computation will continue + ;; at `after-fact' when the subroutine returns. + (save continue) + (save n) + (assign n (op -) (reg n) (const 1)) + (assign continue (label after-fact)) + (goto (label fact-loop)) + after-fact + (restore n) + (restore continue) + (assign val (op *) (reg n) (reg val)) ; `val' now contains n(n - 1)! + (goto (reg continue)) ; return to caller + base-case + (assign val (const 1)) ; base case: 1! = 1 + (goto (reg continue)) ; return to caller + fact-done) + + + *Figure 5.12:* Controller for a machine to compute Fibonacci + numbers. + + (controller + (assign continue (label fib-done)) + fib-loop + (test (op <) (reg n) (const 2)) + (branch (label immediate-answer)) + ;; set up to compute _Fib_(n - 1) + (save continue) + (assign continue (label afterfib-n-1)) + (save n) ; save old value of `n' + (assign n (op -) (reg n) (const 1)); clobber `n' to n - 1 + (goto (label fib-loop)) ; perform recursive call + afterfib-n-1 ; upon return, `val' contains _Fib_(n - 1) + (restore n) + (restore continue) + ;; set up to compute _Fib_(n - 2) + (assign n (op -) (reg n) (const 2)) + (save continue) + (assign continue (label afterfib-n-2)) + (save val) ; save _Fib_(n - 1) + (goto (label fib-loop)) + afterfib-n-2 ; upon return, `val' contains _Fib_(n - 2) + (assign n (reg val)) ; `n' now contains _Fib_(n - 2) + (restore val) ; `val' now contains _Fib_(n - 1) + (restore continue) + (assign val ; _Fib_(n - 1) + _Fib_(n - 2) + (op +) (reg val) (reg n)) + (goto (reg continue)) ; return to caller, answer is in `val' + immediate-answer + (assign val (reg n)) ; base case: _Fib_(n) = n + (goto (reg continue)) + fib-done) + + + *Exercise 5.4:* Specify register machines that implement each of + the following procedures. For each machine, write a controller + instruction sequence and draw a diagram showing the data paths. + + a. Recursive exponentiation: + + (define (expt b n) + (if (= n 0) + 1 + (* b (expt b (- n 1))))) + + b. Iterative exponentiation: + + (define (expt b n) + (define (expt-iter counter product) + (if (= counter 0) + product + (expt-iter (- counter 1) (* b product)))) + (expt-iter n 1)) + + + *Exercise 5.5:* Hand-simulate the factorial and Fibonacci + machines, using some nontrivial input (requiring execution of at + least one recursive call). Show the contents of the stack at each + significant point in the execution. + + *Exercise 5.6:* Ben Bitdiddle observes that the Fibonacci + machine's controller sequence has an extra `save' and an extra + `restore', which can be removed to make a faster machine. Where + are these instructions? + + ---------- Footnotes ---------- + + (1) One might argue that we don't need to save the old `n'; after we +decrement it and solve the subproblem, we could simply increment it to +recover the old value. Although this strategy works for factorial, it +cannot work in general, since the old value of a register cannot always +be computed from the new one. + + (2) In section *Note 5-3:: we will see how to implement a stack in +terms of more primitive operations. + + +File: sicp.info, Node: 5-1-5, Prev: 5-1-4, Up: 5-1 + +5.1.5 Instruction Summary +------------------------- + +A controller instruction in our register-machine language has one of the +following forms, where each is either `(reg )' +or `(const )'. These instructions were introduced in +section *Note 5-1-1::: + + (assign (reg )) + + (assign (const )) + + (assign (op ) ... ) + + (perform (op ) ... ) + + (test (op ) ... ) + + (branch (label )) + + (goto (label )) + + The use of registers to hold labels was introduced in section *Note +5-1-3::: + + (assign (label )) + + (goto (reg )) + + Instructions to use the stack were introduced in section *Note +5-1-4::: + + (save ) + + (restore ) + + The only kind of we have seen so far is a number, +but later we will use strings, symbols, and lists. For example, +`(const "abc")' is the string `"abc"', +`(const abc)' is the symbol `abc', +`(const (a b c))' is the list `(a b c)', +and `(const ())' is the empty list. + + +File: sicp.info, Node: 5-2, Next: 5-3, Prev: 5-1, Up: Chapter 5 + +5.2 A Register-Machine Simulator +================================ + +In order to gain a good understanding of the design of register +machines, we must test the machines we design to see if they perform as +expected. One way to test a design is to hand-simulate the operation +of the controller, as in *Note Exercise 5-5::. But this is extremely +tedious for all but the simplest machines. In this section we +construct a simulator for machines described in the register-machine +language. The simulator is a Scheme program with four interface +procedures. The first uses a description of a register machine to +construct a model of the machine (a data structure whose parts +correspond to the parts of the machine to be simulated), and the other +three allow us to simulate the machine by manipulating the model: + + (make-machine ) + + constructs and returns a model of the machine with the given + registers, operations, and controller. + + (set-register-contents! ) + + stores a value in a simulated register in the given machine. + + (get-register-contents ) + + returns the contents of a simulated register in the given machine. + + (start ) + + simulates the execution of the given machine, starting from the + beginning of the controller sequence and stopping when it reaches + the end of the sequence. + + As an example of how these procedures are used, we can define +`gcd-machine' to be a model of the GCD machine of section *Note 5-1-1:: +as follows: + + (define gcd-machine + (make-machine + '(a b t) + (list (list 'rem remainder) (list '= =)) + '(test-b + (test (op =) (reg b) (const 0)) + (branch (label gcd-done)) + (assign t (op rem) (reg a) (reg b)) + (assign a (reg b)) + (assign b (reg t)) + (goto (label test-b)) + gcd-done))) + + The first argument to `make-machine' is a list of register names. +The next argument is a table (a list of two-element lists) that pairs +each operation name with a Scheme procedure that implements the +operation (that is, produces the same output value given the same input +values). The last argument specifies the controller as a list of +labels and machine instructions, as in section *Note 5-1::. + + To compute GCDs with this machine, we set the input registers, start +the machine, and examine the result when the simulation terminates: + + (set-register-contents! gcd-machine 'a 206) + done + + (set-register-contents! gcd-machine 'b 40) + done + + (start gcd-machine) + done + + (get-register-contents gcd-machine 'a) + 2 + + This computation will run much more slowly than a `gcd' procedure +written in Scheme, because we will simulate low-level machine +instructions, such as `assign', by much more complex operations. + + *Exercise 5.7:* Use the simulator to test the machines you + designed in *Note Exercise 5-4::. + +* Menu: + +* 5-2-1:: The Machine Model +* 5-2-2:: The Assembler +* 5-2-3:: Generating Execution Procedures for Instructions +* 5-2-4:: Monitoring Machine Performance + + +File: sicp.info, Node: 5-2-1, Next: 5-2-2, Prev: 5-2, Up: 5-2 + +5.2.1 The Machine Model +----------------------- + +The machine model generated by `make-machine' is represented as a +procedure with local state using the message-passing techniques +developed in *Note Chapter 3::. To build this model, `make-machine' +begins by calling the procedure `make-new-machine' to construct the +parts of the machine model that are common to all register machines. +This basic machine model constructed by `make-new-machine' is +essentially a container for some registers and a stack, together with +an execution mechanism that processes the controller instructions one +by one. + + `Make-machine' then extends this basic model (by sending it +messages) to include the registers, operations, and controller of the +particular machine being defined. First it allocates a register in the +new machine for each of the supplied register names and installs the +designated operations in the machine. Then it uses an "assembler" +(described below in section *Note 5-2-2::) to transform the controller +list into instructions for the new machine and installs these as the +machine's instruction sequence. `Make-machine' returns as its value +the modified machine model. + + (define (make-machine register-names ops controller-text) + (let ((machine (make-new-machine))) + (for-each (lambda (register-name) + ((machine 'allocate-register) register-name)) + register-names) + ((machine 'install-operations) ops) + ((machine 'install-instruction-sequence) + (assemble controller-text machine)) + machine)) + +Registers +......... + +We will represent a register as a procedure with local state, as in +*Note Chapter 3::. The procedure `make-register' creates a register +that holds a value that can be accessed or changed: + + (define (make-register name) + (let ((contents '*unassigned*)) + (define (dispatch message) + (cond ((eq? message 'get) contents) + ((eq? message 'set) + (lambda (value) (set! contents value))) + (else + (error "Unknown request -- REGISTER" message)))) + dispatch)) + + The following procedures are used to access registers: + + (define (get-contents register) + (register 'get)) + + (define (set-contents! register value) + ((register 'set) value)) + +The stack +......... + +We can also represent a stack as a procedure with local state. The +procedure `make-stack' creates a stack whose local state consists of a +list of the items on the stack. A stack accepts requests to `push' an +item onto the stack, to `pop' the top item off the stack and return it, +and to `initialize' the stack to empty. + + (define (make-stack) + (let ((s '())) + (define (push x) + (set! s (cons x s))) + (define (pop) + (if (null? s) + (error "Empty stack -- POP") + (let ((top (car s))) + (set! s (cdr s)) + top))) + (define (initialize) + (set! s '()) + 'done) + (define (dispatch message) + (cond ((eq? message 'push) push) + ((eq? message 'pop) (pop)) + ((eq? message 'initialize) (initialize)) + (else (error "Unknown request -- STACK" + message)))) + dispatch)) + + The following procedures are used to access stacks: + + (define (pop stack) + (stack 'pop)) + + (define (push stack value) + ((stack 'push) value)) + +The basic machine +................. + +The `make-new-machine' procedure, shown in *Note Figure 5-13::, +constructs an object whose local state consists of a stack, an +initially empty instruction sequence, a list of operations that +initially contains an operation to initialize the stack, and a "register +table" that initially contains two registers, named `flag' and `pc' +(for "program counter"). The internal procedure `allocate-register' +adds new entries to the register table, and the internal procedure +`lookup-register' looks up registers in the table. + + The `flag' register is used to control branching in the simulated +machine. `Test' instructions set the contents of `flag' to the result +of the test (true or false). `Branch' instructions decide whether or +not to branch by examining the contents of `flag'. + + The `pc' register determines the sequencing of instructions as the +machine runs. This sequencing is implemented by the internal procedure +`execute'. In the simulation model, each machine instruction is a data +structure that includes a procedure of no arguments, called the procedure +"instruction execution procedure", such that calling this procedure +simulates executing the instruction. As the simulation runs, `pc' +points to the place in the instruction sequence beginning with the next +instruction to be executed. `Execute' gets that instruction, executes +it by calling the instruction execution procedure, and repeats this +cycle until there are no more instructions to execute (i.e., until `pc' +points to the end of the instruction sequence). + + *Figure 5.13:* The `make-new-machine' procedure, which implements + the basic machine model. + + (define (make-new-machine) + (let ((pc (make-register 'pc)) + (flag (make-register 'flag)) + (stack (make-stack)) + (the-instruction-sequence '())) + (let ((the-ops + (list (list 'initialize-stack + (lambda () (stack 'initialize))))) + (register-table + (list (list 'pc pc) (list 'flag flag)))) + (define (allocate-register name) + (if (assoc name register-table) + (error "Multiply defined register: " name) + (set! register-table + (cons (list name (make-register name)) + register-table))) + 'register-allocated) + (define (lookup-register name) + (let ((val (assoc name register-table))) + (if val + (cadr val) + (error "Unknown register:" name)))) + (define (execute) + (let ((insts (get-contents pc))) + (if (null? insts) + 'done + (begin + ((instruction-execution-proc (car insts))) + (execute))))) + (define (dispatch message) + (cond ((eq? message 'start) + (set-contents! pc the-instruction-sequence) + (execute)) + ((eq? message 'install-instruction-sequence) + (lambda (seq) (set! the-instruction-sequence seq))) + ((eq? message 'allocate-register) allocate-register) + ((eq? message 'get-register) lookup-register) + ((eq? message 'install-operations) + (lambda (ops) (set! the-ops (append the-ops ops)))) + ((eq? message 'stack) stack) + ((eq? message 'operations) the-ops) + (else (error "Unknown request -- MACHINE" message)))) + dispatch))) + + As part of its operation, each instruction execution procedure +modifies `pc' to indicate the next instruction to be executed. +`Branch' and `goto' instructions change `pc' to point to the new +destination. All other instructions simply advance `pc', making it +point to the next instruction in the sequence. Observe that each call +to `execute' calls `execute' again, but this does not produce an +infinite loop because running the instruction execution procedure +changes the contents of `pc'. + + `Make-new-machine' returns a `dispatch' procedure that implements +message-passing access to the internal state. Notice that starting the +machine is accomplished by setting `pc' to the beginning of the +instruction sequence and calling `execute'. + + For convenience, we provide an alternate procedural interface to a +machine's `start' operation, as well as procedures to set and examine +register contents, as specified at the beginning of section *Note 5-2::: + + (define (start machine) + (machine 'start)) + + (define (get-register-contents machine register-name) + (get-contents (get-register machine register-name))) + + (define (set-register-contents! machine register-name value) + (set-contents! (get-register machine register-name) value) + 'done) + + These procedures (and many procedures in sections *Note 5-2-2:: and +*Note 5-2-3::) use the following to look up the register with a given +name in a given machine: + + (define (get-register machine reg-name) + ((machine 'get-register) reg-name)) + + +File: sicp.info, Node: 5-2-2, Next: 5-2-3, Prev: 5-2-1, Up: 5-2 + +5.2.2 The Assembler +------------------- + +The assembler transforms the sequence of controller expressions for a +machine into a corresponding list of machine instructions, each with +its execution procedure. Overall, the assembler is much like the +evaluators we studied in *Note Chapter 4::--there is an input language +(in this case, the register-machine language) and we must perform an +appropriate action for each type of expression in the language. + + The technique of producing an execution procedure for each +instruction is just what we used in section *Note 4-1-7:: to speed up +the evaluator by separating analysis from runtime execution. As we saw +in *Note Chapter 4::, much useful analysis of Scheme expressions could +be performed without knowing the actual values of variables. Here, +analogously, much useful analysis of register-machine-language +expressions can be performed without knowing the actual contents of +machine registers. For example, we can replace references to registers +by pointers to the register objects, and we can replace references to +labels by pointers to the place in the instruction sequence that the +label designates. + + Before it can generate the instruction execution procedures, the +assembler must know what all the labels refer to, so it begins by +scanning the controller text to separate the labels from the +instructions. As it scans the text, it constructs both a list of +instructions and a table that associates each label with a pointer into +that list. Then the assembler augments the instruction list by +inserting the execution procedure for each instruction. + + The `assemble' procedure is the main entry to the assembler. It +takes the controller text and the machine model as arguments and +returns the instruction sequence to be stored in the model. `Assemble' +calls `extract-labels' to build the initial instruction list and label +table from the supplied controller text. The second argument to +`extract-labels' is a procedure to be called to process these results: +This procedure uses `update-insts!' to generate the instruction +execution procedures and insert them into the instruction list, and +returns the modified list. + + (define (assemble controller-text machine) + (extract-labels controller-text + (lambda (insts labels) + (update-insts! insts labels machine) + insts))) + + `Extract-labels' takes as arguments a list `text' (the sequence of +controller instruction expressions) and a `receive' procedure. +`Receive' will be called with two values: (1) a list `insts' of +instruction data structures, each containing an instruction from `text'; +and (2) a table called `labels', which associates each label from +`text' with the position in the list `insts' that the label designates. + + (define (extract-labels text receive) + (if (null? text) + (receive '() '()) + (extract-labels (cdr text) + (lambda (insts labels) + (let ((next-inst (car text))) + (if (symbol? next-inst) + (receive insts + (cons (make-label-entry next-inst + insts) + labels)) + (receive (cons (make-instruction next-inst) + insts) + labels))))))) + + `Extract-labels' works by sequentially scanning the elements of the +`text' and accumulating the `insts' and the `labels'. If an element is +a symbol (and thus a label) an appropriate entry is added to the +`labels' table. Otherwise the element is accumulated onto the `insts' +list.(1) + + `Update-insts!' modifies the instruction list, which initially +contains only the text of the instructions, to include the +corresponding execution procedures: + + (define (update-insts! insts labels machine) + (let ((pc (get-register machine 'pc)) + (flag (get-register machine 'flag)) + (stack (machine 'stack)) + (ops (machine 'operations))) + (for-each + (lambda (inst) + (set-instruction-execution-proc! + inst + (make-execution-procedure + (instruction-text inst) labels machine + pc flag stack ops))) + insts))) + + The machine instruction data structure simply pairs the instruction +text with the corresponding execution procedure. The execution +procedure is not yet available when `extract-labels' constructs the +instruction, and is inserted later by `update-insts!'. + + (define (make-instruction text) + (cons text '())) + + (define (instruction-text inst) + (car inst)) + + (define (instruction-execution-proc inst) + (cdr inst)) + + (define (set-instruction-execution-proc! inst proc) + (set-cdr! inst proc)) + + The instruction text is not used by our simulator, but it is handy +to keep around for debugging (see *Note Exercise 5-16::). + + Elements of the label table are pairs: + + (define (make-label-entry label-name insts) + (cons label-name insts)) + + Entries will be looked up in the table with + + (define (lookup-label labels label-name) + (let ((val (assoc label-name labels))) + (if val + (cdr val) + (error "Undefined label -- ASSEMBLE" label-name)))) + + *Exercise 5.8:* The following register-machine code is ambiguous, + because the label `here' is defined more than once: + + start + (goto (label here)) + here + (assign a (const 3)) + (goto (label there)) + here + (assign a (const 4)) + (goto (label there)) + there + + With the simulator as written, what will the contents of register + `a' be when control reaches `there'? Modify the `extract-labels' + procedure so that the assembler will signal an error if the same + label name is used to indicate two different locations. + + ---------- Footnotes ---------- + + (1) Using the `receive' procedure here is a way to get +`extract-labels' to effectively return two values--`labels' and +`insts'--without explicitly making a compound data structure to hold +them. An alternative implementation, which returns an explicit pair of +values, is + + (define (extract-labels text) + (if (null? text) + (cons '() '()) + (let ((result (extract-labels (cdr text)))) + (let ((insts (car result)) (labels (cdr result))) + (let ((next-inst (car text))) + (if (symbol? next-inst) + (cons insts + (cons (make-label-entry next-inst insts) labels)) + (cons (cons (make-instruction next-inst) insts) + labels))))))) + +which would be called by `assemble' as follows: + + (define (assemble controller-text machine) + (let ((result (extract-labels controller-text))) + (let ((insts (car result)) (labels (cdr result))) + (update-insts! insts labels machine) + insts))) + + You can consider our use of `receive' as demonstrating an elegant +way to return multiple values, or simply an excuse to show off a +programming trick. An argument like `receive' that is the next +procedure to be invoked is called a "continuation." Recall that we +also used continuations to implement the backtracking control structure +in the `amb' evaluator in section *Note 4-3-3::. + + +File: sicp.info, Node: 5-2-3, Next: 5-2-4, Prev: 5-2-2, Up: 5-2 + +5.2.3 Generating Execution Procedures for Instructions +------------------------------------------------------ + +The assembler calls `make-execution-procedure' to generate the execution +procedure for an instruction. Like the `analyze' procedure in the +evaluator of section *Note 4-1-7::, this dispatches on the type of +instruction to generate the appropriate execution procedure. + + (define (make-execution-procedure inst labels machine + pc flag stack ops) + (cond ((eq? (car inst) 'assign) + (make-assign inst machine labels ops pc)) + ((eq? (car inst) 'test) + (make-test inst machine labels ops flag pc)) + ((eq? (car inst) 'branch) + (make-branch inst machine labels flag pc)) + ((eq? (car inst) 'goto) + (make-goto inst machine labels pc)) + ((eq? (car inst) 'save) + (make-save inst machine stack pc)) + ((eq? (car inst) 'restore) + (make-restore inst machine stack pc)) + ((eq? (car inst) 'perform) + (make-perform inst machine labels ops pc)) + (else (error "Unknown instruction type -- ASSEMBLE" + inst)))) + + For each type of instruction in the register-machine language, there +is a generator that builds an appropriate execution procedure. The +details of these procedures determine both the syntax and meaning of +the individual instructions in the register-machine language. We use +data abstraction to isolate the detailed syntax of register-machine +expressions from the general execution mechanism, as we did for +evaluators in section *Note 4-1-2::, by using syntax procedures to +extract and classify the parts of an instruction. + +`Assign' instructions +..................... + +The `make-assign' procedure handles `assign' instructions: + + (define (make-assign inst machine labels operations pc) + (let ((target + (get-register machine (assign-reg-name inst))) + (value-exp (assign-value-exp inst))) + (let ((value-proc + (if (operation-exp? value-exp) + (make-operation-exp + value-exp machine labels operations) + (make-primitive-exp + (car value-exp) machine labels)))) + (lambda () ; execution procedure for `assign' + (set-contents! target (value-proc)) + (advance-pc pc))))) + + `Make-assign' extracts the target register name (the second element +of the instruction) and the value expression (the rest of the list that +forms the instruction) from the `assign' instruction using the selectors + + (define (assign-reg-name assign-instruction) + (cadr assign-instruction)) + + (define (assign-value-exp assign-instruction) + (cddr assign-instruction)) + + The register name is looked up with `get-register' to produce the +target register object. The value expression is passed to +`make-operation-exp' if the value is the result of an operation, and to +`make-primitive-exp' otherwise. These procedures (shown below) parse +the value expression and produce an execution procedure for the value. +This is a procedure of no arguments, called `value-proc', which will be +evaluated during the simulation to produce the actual value to be +assigned to the register. Notice that the work of looking up the +register name and parsing the value expression is performed just once, +at assembly time, not every time the instruction is simulated. This +saving of work is the reason we use execution procedures, and +corresponds directly to the saving in work we obtained by separating +program analysis from execution in the evaluator of section *Note +4-1-7::. + + The result returned by `make-assign' is the execution procedure for +the `assign' instruction. When this procedure is called (by the machine +model's `execute' procedure), it sets the contents of the target +register to the result obtained by executing `value-proc'. Then it +advances the `pc' to the next instruction by running the procedure + + (define (advance-pc pc) + (set-contents! pc (cdr (get-contents pc)))) + + `Advance-pc' is the normal termination for all instructions except +`branch' and `goto'. + +`Test', `branch', and `goto' instructions +......................................... + +`Make-test' handles `test' instructions in a similar way. It extracts +the expression that specifies the condition to be tested and generates +an execution procedure for it. At simulation time, the procedure for +the condition is called, the result is assigned to the `flag' register, +and the `pc' is advanced: + + (define (make-test inst machine labels operations flag pc) + (let ((condition (test-condition inst))) + (if (operation-exp? condition) + (let ((condition-proc + (make-operation-exp + condition machine labels operations))) + (lambda () + (set-contents! flag (condition-proc)) + (advance-pc pc))) + (error "Bad TEST instruction -- ASSEMBLE" inst)))) + + (define (test-condition test-instruction) + (cdr test-instruction)) + + The execution procedure for a `branch' instruction checks the +contents of the `flag' register and either sets the contents of the +`pc' to the branch destination (if the branch is taken) or else just +advances the `pc' (if the branch is not taken). Notice that the +indicated destination in a `branch' instruction must be a label, and +the `make-branch' procedure enforces this. Notice also that the label +is looked up at assembly time, not each time the `branch' instruction +is simulated. + + (define (make-branch inst machine labels flag pc) + (let ((dest (branch-dest inst))) + (if (label-exp? dest) + (let ((insts + (lookup-label labels (label-exp-label dest)))) + (lambda () + (if (get-contents flag) + (set-contents! pc insts) + (advance-pc pc)))) + (error "Bad BRANCH instruction -- ASSEMBLE" inst)))) + + (define (branch-dest branch-instruction) + (cadr branch-instruction)) + + A `goto' instruction is similar to a branch, except that the +destination may be specified either as a label or as a register, and +there is no condition to check--the `pc' is always set to the new +destination. + + (define (make-goto inst machine labels pc) + (let ((dest (goto-dest inst))) + (cond ((label-exp? dest) + (let ((insts + (lookup-label labels + (label-exp-label dest)))) + (lambda () (set-contents! pc insts)))) + ((register-exp? dest) + (let ((reg + (get-register machine + (register-exp-reg dest)))) + (lambda () + (set-contents! pc (get-contents reg))))) + (else (error "Bad GOTO instruction -- ASSEMBLE" + inst))))) + + (define (goto-dest goto-instruction) + (cadr goto-instruction)) + +Other instructions +.................. + +The stack instructions `save' and `restore' simply use the stack with +the designated register and advance the `pc': + + (define (make-save inst machine stack pc) + (let ((reg (get-register machine + (stack-inst-reg-name inst)))) + (lambda () + (push stack (get-contents reg)) + (advance-pc pc)))) + + (define (make-restore inst machine stack pc) + (let ((reg (get-register machine + (stack-inst-reg-name inst)))) + (lambda () + (set-contents! reg (pop stack)) + (advance-pc pc)))) + + (define (stack-inst-reg-name stack-instruction) + (cadr stack-instruction)) + + The final instruction type, handled by `make-perform', generates an +execution procedure for the action to be performed. At simulation +time, the action procedure is executed and the `pc' advanced. + + (define (make-perform inst machine labels operations pc) + (let ((action (perform-action inst))) + (if (operation-exp? action) + (let ((action-proc + (make-operation-exp + action machine labels operations))) + (lambda () + (action-proc) + (advance-pc pc))) + (error "Bad PERFORM instruction -- ASSEMBLE" inst)))) + + (define (perform-action inst) (cdr inst)) + +Execution procedures for subexpressions +....................................... + +The value of a `reg', `label', or `const' expression may be needed for +assignment to a register (`make-assign') or for input to an operation +(`make-operation-exp', below). The following procedure generates +execution procedures to produce values for these expressions during the +simulation: + + (define (make-primitive-exp exp machine labels) + (cond ((constant-exp? exp) + (let ((c (constant-exp-value exp))) + (lambda () c))) + ((label-exp? exp) + (let ((insts + (lookup-label labels + (label-exp-label exp)))) + (lambda () insts))) + ((register-exp? exp) + (let ((r (get-register machine + (register-exp-reg exp)))) + (lambda () (get-contents r)))) + (else + (error "Unknown expression type -- ASSEMBLE" exp)))) + + The syntax of `reg', `label', and `const' expressions is determined +by + + (define (register-exp? exp) (tagged-list? exp 'reg)) + + (define (register-exp-reg exp) (cadr exp)) + + (define (constant-exp? exp) (tagged-list? exp 'const)) + + (define (constant-exp-value exp) (cadr exp)) + + (define (label-exp? exp) (tagged-list? exp 'label)) + + (define (label-exp-label exp) (cadr exp)) + + `Assign', `perform', and `test' instructions may include the +application of a machine operation (specified by an `op' expression) to +some operands (specified by `reg' and `const' expressions). The +following procedure produces an execution procedure for an "operation +expression"--a list containing the operation and operand expressions +from the instruction: + + (define (make-operation-exp exp machine labels operations) + (let ((op (lookup-prim (operation-exp-op exp) operations)) + (aprocs + (map (lambda (e) + (make-primitive-exp e machine labels)) + (operation-exp-operands exp)))) + (lambda () + (apply op (map (lambda (p) (p)) aprocs))))) + + The syntax of operation expressions is determined by + + (define (operation-exp? exp) + (and (pair? exp) (tagged-list? (car exp) 'op))) + + (define (operation-exp-op operation-exp) + (cadr (car operation-exp))) + + (define (operation-exp-operands operation-exp) + (cdr operation-exp)) + + Observe that the treatment of operation expressions is very much +like the treatment of procedure applications by the +`analyze-application' procedure in the evaluator of section *Note +4-1-7:: in that we generate an execution procedure for each operand. +At simulation time, we call the operand procedures and apply the Scheme +procedure that simulates the operation to the resulting values. The +simulation procedure is found by looking up the operation name in the +operation table for the machine: + + (define (lookup-prim symbol operations) + (let ((val (assoc symbol operations))) + (if val + (cadr val) + (error "Unknown operation -- ASSEMBLE" symbol)))) + + *Exercise 5.9:* The treatment of machine operations above permits + them to operate on labels as well as on constants and the contents + of registers. Modify the expression-processing procedures to + enforce the condition that operations can be used only with + registers and constants. + + *Exercise 5.10:* Design a new syntax for register-machine + instructions and modify the simulator to use your new syntax. Can + you implement your new syntax without changing any part of the + simulator except the syntax procedures in this section? + + *Exercise 5.11:* When we introduced `save' and `restore' in + section *Note 5-1-4::, we didn't specify what would happen if you + tried to restore a register that was not the last one saved, as in + the sequence + + (save y) + (save x) + (restore y) + + There are several reasonable possibilities for the meaning of + `restore': + + a. `(restore y)' puts into `y' the last value saved on the stack, + regardless of what register that value came from. This is + the way our simulator behaves. Show how to take advantage of + this behavior to eliminate one instruction from the Fibonacci + machine of section *Note 5-1-4:: (*Note Figure 5-12::). + + b. `(restore y)' puts into `y' the last value saved on the + stack, but only if that value was saved from `y'; otherwise, + it signals an error. Modify the simulator to behave this + way. You will have to change `save' to put the register name + on the stack along with the value. + + c. `(restore y)' puts into `y' the last value saved from `y' + regardless of what other registers were saved after `y' and + not restored. Modify the simulator to behave this way. You + will have to associate a separate stack with each register. + You should make the `initialize-stack' operation initialize + all the register stacks. + + + *Exercise 5.12:* The simulator can be used to help determine the + data paths required for implementing a machine with a given + controller. Extend the assembler to store the following + information in the machine model: + + * a list of all instructions, with duplicates removed, sorted + by instruction type (`assign', `goto', and so on); + + * a list (without duplicates) of the registers used to hold + entry points (these are the registers referenced by `goto' + instructions); + + * a list (without duplicates) of the registers that are `save'd + or `restore'd; + + * for each register, a list (without duplicates) of the sources + from which it is assigned (for example, the sources for + register `val' in the factorial machine of *Note Figure + 5-11:: are `(const 1)' and `((op *) (reg n) (reg val))'). + + + Extend the message-passing interface to the machine to provide + access to this new information. To test your analyzer, define the + Fibonacci machine from *Note Figure 5-12:: and examine the lists + you constructed. + + *Exercise 5.13:* Modify the simulator so that it uses the + controller sequence to determine what registers the machine has + rather than requiring a list of registers as an argument to + `make-machine'. Instead of pre-allocating the registers in + `make-machine', you can allocate them one at a time when they are + first seen during assembly of the instructions. + + +File: sicp.info, Node: 5-2-4, Prev: 5-2-3, Up: 5-2 + +5.2.4 Monitoring Machine Performance +------------------------------------ + +Simulation is useful not only for verifying the correctness of a +proposed machine design but also for measuring the machine's +performance. For example, we can install in our simulation program a +"meter" that measures the number of stack operations used in a +computation. To do this, we modify our simulated stack to keep track +of the number of times registers are saved on the stack and the maximum +depth reached by the stack, and add a message to the stack's interface +that prints the statistics, as shown below. We also add an operation +to the basic machine model to print the stack statistics, by +initializing `the-ops' in `make-new-machine' to + + (list (list 'initialize-stack + (lambda () (stack 'initialize))) + (list 'print-stack-statistics + (lambda () (stack 'print-statistics)))) + + Here is the new version of `make-stack': + + (define (make-stack) + (let ((s '()) + (number-pushes 0) + (max-depth 0) + (current-depth 0)) + (define (push x) + (set! s (cons x s)) + (set! number-pushes (+ 1 number-pushes)) + (set! current-depth (+ 1 current-depth)) + (set! max-depth (max current-depth max-depth))) + (define (pop) + (if (null? s) + (error "Empty stack -- POP") + (let ((top (car s))) + (set! s (cdr s)) + (set! current-depth (- current-depth 1)) + top))) + (define (initialize) + (set! s '()) + (set! number-pushes 0) + (set! max-depth 0) + (set! current-depth 0) + 'done) + (define (print-statistics) + (newline) + (display (list 'total-pushes '= number-pushes + 'maximum-depth '= max-depth))) + (define (dispatch message) + (cond ((eq? message 'push) push) + ((eq? message 'pop) (pop)) + ((eq? message 'initialize) (initialize)) + ((eq? message 'print-statistics) + (print-statistics)) + (else + (error "Unknown request -- STACK" message)))) + dispatch)) + + *Note Exercise 5-15:: through *Note Exercise 5-19:: describe other +useful monitoring and debugging features that can be added to the +register-machine simulator. + + *Exercise 5.14:* Measure the number of pushes and the maximum + stack depth required to compute n! for various small values of n + using the factorial machine shown in *Note Figure 5-11::. From + your data determine formulas in terms of n for the total number of + push operations and the maximum stack depth used in computing n! + for any n > 1. Note that each of these is a linear function of n + and is thus determined by two constants. In order to get the + statistics printed, you will have to augment the factorial machine + with instructions to initialize the stack and print the + statistics. You may want to also modify the machine so that it + repeatedly reads a value for n, computes the factorial, and prints + the result (as we did for the GCD machine in *Note Figure 5-4::), + so that you will not have to repeatedly invoke + `get-register-contents', `set-register-contents!', and `start'. + + *Exercise 5.15:* Add counting "instruction counting" to the + register machine simulation. That is, have the machine model keep + track of the number of instructions executed. Extend the machine + model's interface to accept a new message that prints the value of + the instruction count and resets the count to zero. + + *Exercise 5.16:* Augment the simulator to provide for "instruction + tracing". That is, before each instruction is executed, the + simulator should print the text of the instruction. Make the + machine model accept `trace-on' and `trace-off' messages to turn + tracing on and off. + + *Exercise 5.17:* Extend the instruction tracing of *Note Exercise + 5-16:: so that before printing an instruction, the simulator + prints any labels that immediately precede that instruction in the + controller sequence. Be careful to do this in a way that does not + interfere with instruction counting (*Note Exercise 5-15::). You + will have to make the simulator retain the necessary label + information. + + *Exercise 5.18:* Modify the `make-register' procedure of section + *Note 5-2-1:: so that registers can be traced. Registers should + accept messages that turn tracing on and off. When a register is + traced, assigning a value to the register should print the name of + the register, the old contents of the register, and the new + contents being assigned. Extend the interface to the machine + model to permit you to turn tracing on and off for designated + machine registers. + + *Exercise 5.19:* Alyssa P. Hacker wants a "breakpoint" feature in + the simulator to help her debug her machine designs. You have + been hired to install this feature for her. She wants to be able + to specify a place in the controller sequence where the simulator + will stop and allow her to examine the state of the machine. You + are to implement a procedure + + (set-breakpoint

    ' strings are completed. + + -n PLUGIN + Complete aliases from PLUGIN. + + + The following plugins are available by default: _email-ldap (see + the filter style), _email-local (completes USER@HOSTNAME Unix + addresses), _email-mail (completes aliases from ~/.mailrc), + _email-mush, _email-mutt, and _email-pine. + + Addresses from the _email-FOO plugin are added under the tag + `email-FOO'. + + _Writing plugins_ + + Plugins are written as separate functions with names starting with + `_email-'. They are invoked with the -c option and compadd + options. They should either do their own completion or set the + $reply array to a list of `ALIAS:ADDRESS' elements and return 300. + New plugins will be picked up and run automatically. + +_files + The function _files is a wrapper around _path_files. It supports + all of the same functionality, with some enhancements -- notably, + it respects the list-dirs-first style, and it allows users to + override the behaviour of the -g and -/ options with the + file-patterns style. _files should therefore be preferred over + _path_files in most cases. + + This function accepts the full set of options allowed by + _path_files, described below. + +_gnu_generic + This function is a simple wrapper around the _arguments function + described above. It can be used to determine automatically the + long options understood by commands that produce a list when + passed the option `--help'. It is intended to be used as a + top-level completion function in its own right. For example, to + enable option completion for the commands foo and bar, use + + + compdef _gnu_generic foo bar + + after the call to compinit. + + The completion system as supplied is conservative in its use of + this function, since it is important to be sure the command + understands the option `--help'. + +_guard [ OPTIONS ] PATTERN DESCR + This function displays DESCR if PATTERN matches the string to be + completed. It is intended to be used in the ACTION for the + specifications passed to _arguments and similar functions. + + The return status is zero if the message was displayed and the + word to complete is not empty, and non-zero otherwise. + + The PATTERN may be preceded by any of the options understood by + compadd that are passed down from _description, namely -M, -J, -V, + -1, -2, -n, -F and -X. All of these options will be ignored. + This fits in conveniently with the argument-passing conventions of + actions for _arguments. + + As an example, consider a command taking the options -n and -none, + where -n must be followed by a numeric value in the same word. By + using: + + + _arguments '-n-: :_guard "[0-9]#" "numeric value"' '-none' + + _arguments can be made to both display the message `numeric value' + and complete options after `-n'. If the `-n' is already + followed by one or more digits (the pattern passed to _guard) only + the message will be displayed; if the `-n' is followed by another + character, only options are completed. + +_message [ -r12 ] [ -VJ GROUP ] DESCR +_message -e [ TAG ] DESCR + The DESCR is used in the same way as the third argument to the + _description function, except that the resulting string will + always be shown whether or not matches were generated. This is + useful for displaying a help message in places where no + completions can be generated. + + The format style is examined with the messages tag to find a + message; the usual tag, descriptions, is used only if the style is + not set with the former. + + If the -r option is given, no style is used; the DESCR is taken + literally as the string to display. This is most useful when the + DESCR comes from a pre-processed argument list which already + contains an expanded description. Note that this option does not + disable the `%'-sequence parsing done by compadd. + + The -12VJ options and the GROUP are passed to compadd and hence + determine the group the message string is added to. + + The second -e form gives a description for completions with the tag + TAG to be shown even if there are no matches for that tag. This + form is called by _arguments in the event that there is no action + for an option specification. The tag can be omitted and if so the + tag is taken from the parameter $curtag; this is maintained by the + completion system and so is usually correct. Note that if there + are no matches at the time this function is called, + compstate[insert] is cleared, so additional matches generated + later are not inserted on the command line. + +_multi_parts [ -i ] SEP ARRAY + The argument SEP is a separator character. The ARRAY may be + either the name of an array parameter or a literal array in the + form `(foo bar)', a parenthesised list of words separated by + whitespace. The possible completions are the strings from the + array. However, each chunk delimited by SEP will be completed + separately. For example, the _tar function uses `_multi_parts / + PATHARRAY' to complete partial file paths from the given array of + complete file paths. + + The -i option causes _multi_parts to insert a unique match even if + that requires multiple separators to be inserted. This is not + usually the expected behaviour with filenames, but certain other + types of completion, for example those with a fixed set of + possibilities, may be more suited to this form. + + Like other utility functions, this function accepts the `-V', + `-J', `-1', `-2', `-n', `-f', `-X', `-M', `-P', `-S', `-r', `-R', + and `-q' options and passes them to the compadd builtin. + +_next_label [ -x ] [ -12VJ ] TAG NAME DESCR [ OPTION ... ] + This function is used to implement the loop over different tag + labels for a particular tag as described above for the tag-order + style. On each call it checks to see if there are any more tag + labels; if there is it returns status zero, otherwise non-zero. + As this function requires a current tag to be set, it must always + follow a call to _tags or _requested. + + The -x12VJ options and the first three arguments are passed to the + _description function. Where appropriate the TAG will be replaced + by a tag label in this call. Any description given in the + tag-order style is preferred to the DESCR passed to _next_label. + + The OPTIONs given after the DESCR are set in the parameter given + by NAME, and hence are to be passed to compadd or whatever + function is called to add the matches. + + Here is a typical use of this function for the tag foo. The call + to _requested determines if tag foo is required at all; the loop + over _next_label handles any labels defined for the tag in the + tag-order style. + + + local expl ret=1 + ... + if _requested foo; then + ... + while _next_label foo expl '...'; do + compadd "$expl[@]" ... && ret=0 + done + ... + fi + return ret + +_normal [ -P | -p PRECOMMAND ] + This is the standard function called to handle completion outside + any special -CONTEXT-. It is called both to complete the command + word and also the arguments for a command. In the second case, + _normal looks for a special completion for that command, and if + there is none it uses the completion for the -default- context. + + A second use is to reexamine the command line specified by the + $words array and the $CURRENT parameter after those have been + modified. For example, the function _precommand, which completes + after precommand specifiers such as nohup, removes the first word + from the words array, decrements the CURRENT parameter, then calls + `_normal -p $service'. The effect is that `nohup CMD ...' is + treated in the same way as `CMD ...'. + + + -P + Reset the list of precommands. This option should be used if + completing a command line which allows internal commands + (e.g. builtins and functions) regardless of prior precommands + (e.g. `zsh -c'). + + -p PRECOMMAND + Append PRECOMMAND to the list of precommands. This option + should be used in nearly all cases in which -P is not + applicable. + + + If the command name matches one of the patterns given by one of the + options -p or -P to compdef, the corresponding completion function + is called and then the parameter _compskip is checked. If it is + set completion is terminated at that point even if no matches have + been found. This is the same effect as in the -first- context. + +_options + This can be used to complete the names of shell options. It + provides a matcher specification that ignores a leading `no', + ignores underscores and allows upper-case letters to match their + lower-case counterparts (for example, `glob', `noglob', `NO_GLOB' + are all completed). Any arguments are propagated to the compadd + builtin. + +_options_set and _options_unset + These functions complete only set or unset options, with the same + matching specification used in the _options function. + + Note that you need to uncomment a few lines in the _main_complete + function for these functions to work properly. The lines in + question are used to store the option settings in effect before + the completion widget locally sets the options it needs. Hence + these functions are not generally used by the completion system. + +_parameters + This is used to complete the names of shell parameters. + + The option `-g PATTERN' limits the completion to parameters whose + type matches the PATTERN. The type of a parameter is that shown + by `print ${(t)PARAM}', hence judicious use of `*' in PATTERN is + probably necessary. + + All other arguments are passed to the compadd builtin. + +_path_files + This function is used throughout the completion system to complete + filenames. It allows completion of partial paths. For example, + the string `/u/i/s/sig' may be completed to + `/usr/include/sys/signal.h'. + + The options accepted by both _path_files and _files are: + + + -f + Complete all filenames. This is the default. + + -/ + Specifies that only directories should be completed. + + -g PATTERN + Specifies that only files matching the PATTERN should be + completed. + + -W PATHS + Specifies path prefixes that are to be prepended to the + string from the command line to generate the filenames but + that should not be inserted as completions nor shown in + completion listings. Here, PATHS may be the name of an array + parameter, a literal list of paths enclosed in parentheses or + an absolute pathname. + + -F IGNORED-FILES + This behaves as for the corresponding option to the compadd + builtin. It gives direct control over which filenames should + be ignored. If the option is not present, the + ignored-patterns style is used. + + + Both _path_files and _files also accept the following options + which are passed to compadd: `-J', `-V', `-1', `-2', `-n', `-X', + `-M', `-P', `-S', `-q', `-r', and `-R'. + + Finally, the _path_files function uses the styles expand, + ambiguous, special-dirs, list-suffixes and file-sort described + above. + +_pick_variant [ -b BUILTIN-LABEL ] [ -c COMMAND ] [ -r NAME ] + LABEL=PATTERN ... LABEL [ ARG ... ] + This function is used to resolve situations where a single command + name requires more than one type of handling, either because it + has more than one variant or because there is a name clash between + two different commands. + + The command to run is taken from the first element of the array + words unless this is overridden by the option -c. This command is + run and its output is compared with a series of patterns. + Arguments to be passed to the command can be specified at the end + after all the other arguments. The patterns to try in order are + given by the arguments LABEL=PATTERN; if the output of `COMMAND ARG + ...' contains PATTERN, then LABEL is selected as the label for the + command variant. If none of the patterns match, the final command + label is selected and status 1 is returned. + + If the `-b BUILTIN-LABEL' is given, the command is tested to see + if it is provided as a shell builtin, possibly autoloaded; if so, + the label BUILTIN-LABEL is selected as the label for the variant. + + If the `-r NAME' is given, the LABEL picked is stored in the + parameter named NAME. + + The results are also cached in the _cmd_variant associative array + indexed by the name of the command run. + +_regex_arguments NAME SPEC ... + This function generates a completion function NAME which matches + the specifications SPECs, a set of regular expressions as + described below. After running _regex_arguments, the function + NAME should be called as a normal completion function. The + pattern to be matched is given by the contents of the words array + up to the current cursor position joined together with null + characters; no quotation is applied. + + The arguments are grouped as sets of alternatives separated by `|', + which are tried one after the other until one matches. Each + alternative consists of a one or more specifications which are + tried left to right, with each pattern matched being stripped in + turn from the command line being tested, until all of the group + succeeds or until one fails; in the latter case, the next + alternative is tried. This structure can be repeated to arbitrary + depth by using parentheses; matching proceeds from inside to + outside. + + A special procedure is applied if no test succeeds but the + remaining command line string contains no null character (implying + the remaining word is the one for which completions are to be + generated). The completion target is restricted to the remaining + word and any ACTIONs for the corresponding patterns are executed. + In this case, nothing is stripped from the command line string. + The order of evaluation of the ACTIONs can be determined by the + tag-order style; the various formats supported by _alternative can + be used in ACTION. The DESCR is used for setting up the array + parameter expl. + + Specification arguments take one of following forms, in which + metacharacters such as `(', `)', `#' and `|' should be quoted. + + + /PATTERN/ [%LOOKAHEAD%] [-GUARD] [:TAG:DESCR:ACTION] + This is a single primitive component. The function tests + whether the combined pattern `(#b)((#B)PATTERN)LOOKAHEAD*' + matches the command line string. If so, `GUARD' is evaluated + and its return status is examined to determine if the test + has succeeded. The PATTERN string `[]' is guaranteed never + to match. The LOOKAHEAD is not stripped from the command + line before the next pattern is examined. + + The argument starting with : is used in the same manner as an + argument to _alternative. + + A component is used as follows: PATTERN is tested to see if + the component already exists on the command line. If it + does, any following specifications are examined to find + something to complete. If a component is reached but no such + pattern exists yet on the command line, the string containing + the ACTION is used to generate matches to insert at that + point. + + /PATTERN/+ [%LOOKAHEAD%] [-GUARD] [:TAG:DESCR:ACTION] + This is similar to `/PATTERN/ ...' but the left part of the + command line string (i.e. the part already matched by + previous patterns) is also considered part of the completion + target. + + /PATTERN/- [%LOOKAHEAD%] [-GUARD] [:TAG:DESCR:ACTION] + This is similar to `/PATTERN/ ...' but the ACTIONs of the + current and previously matched patterns are ignored even if + the following `PATTERN' matches the empty string. + + ( SPEC ) + Parentheses may be used to groups SPECs; note each parenthesis + is a single argument to _regex_arguments. + + SPEC # + This allows any number of repetitions of SPEC. + + SPEC SPEC + The two SPECs are to be matched one after the other as + described above. + + SPEC | SPEC + Either of the two SPECs can be matched. + + + The function _regex_words can be used as a helper function to + generate matches for a set of alternative words possibly with + their own arguments as a command line argument. + + Examples: + + + _regex_arguments _tst /$'[^\0]#\0'/ \ + /$'[^\0]#\0'/ :'compadd aaa' + + This generates a function _tst that completes aaa as its only + argument. The TAG and DESCRIPTION for the action have been + omitted for brevity (this works but is not recommended in normal + use). The first component matches the command word, which is + arbitrary; the second matches any argument. As the argument is + also arbitrary, any following component would not depend on aaa + being present. + + + _regex_arguments _tst /$'[^\0]#\0'/ \ + /$'aaa\0'/ :'compadd aaa' + + This is a more typical use; it is similar, but any following + patterns would only match if aaa was present as the first argument. + + + _regex_arguments _tst /$'[^\0]#\0'/ \( \ + /$'aaa\0'/ :'compadd aaa' \ + /$'bbb\0'/ :'compadd bbb' \) \# + + In this example, an indefinite number of command arguments may be + completed. Odd arguments are completed as aaa and even arguments + as bbb. Completion fails unless the set of aaa and bbb arguments + before the current one is matched correctly. + + + _regex_arguments _tst /$'[^\0]#\0'/ \ + \( /$'aaa\0'/ :'compadd aaa' \| \ + /$'bbb\0'/ :'compadd bbb' \) \# + + This is similar, but either aaa or bbb may be completed for any + argument. In this case _regex_words could be used to generate a + suitable expression for the arguments. + + + +_regex_words TAG DESCRIPTION SPEC ... + This function can be used to generate arguments for the + _regex_arguments command which may be inserted at any point where + a set of rules is expected. The TAG and DESCRIPTION give a + standard tag and description pertaining to the current context. + Each SPEC contains two or three arguments separated by a colon: + note that there is no leading colon in this case. + + Each SPEC gives one of a set of words that may be completed at + this point, together with arguments. It is thus roughly + equivalent to the _arguments function when used in normal + (non-regex) completion. + + The part of the SPEC before the first colon is the word to be + completed. This may contain a *; the entire word, before and after + the * is completed, but only the text before the * is required for + the context to be matched, so that further arguments may be + completed after the abbreviated form. + + The second part of SPEC is a description for the word being + completed. + + The optional third part of the SPEC describes how words following + the one being completed are themselves to be completed. It will be + evaluated in order to avoid problems with quoting. This means that + typically it contains a reference to an array containing previously + generated regex arguments. + + The option -t TERM specifies a terminator for the word instead of + the usual space. This is handled as an auto-removable suffix in + the manner of the option -s SEP to _values. + + The result of the processing by _regex_words is placed in the array + reply, which should be made local to the calling function. If the + set of words and arguments may be matched repeatedly, a # should + be appended to the generated array at that point. + + For example: + + + local -a reply + _regex_words mydb-commands 'mydb commands' \ + 'add:add an entry to mydb:$mydb_add_cmds' \ + 'show:show entries in mydb' + _regex_arguments _mydb "$reply[@]" + _mydb "$@" + + This shows a completion function for a command mydb which takes + two command arguments, add and show. show takes no arguments, + while the arguments for add have already been prepared in an array + mydb_add_cmds, quite possibly by a previous call to _regex_words. + +_requested [ -x ] [ -12VJ ] TAG [ NAME DESCR [ COMMAND [ ARG ... ] ] + This function is called to decide whether a tag already registered + by a call to _tags (see below) has been requested by the user and + hence completion should be performed for it. It returns status + zero if the tag is requested and non-zero otherwise. The function + is typically used as part of a loop over different tags as follows: + + + _tags foo bar baz + while _tags; do + if _requested foo; then + ... # perform completion for foo + fi + ... # test the tags bar and baz in the same way + ... # exit loop if matches were generated + done + + Note that the test for whether matches were generated is not + performed until the end of the _tags loop. This is so that the + user can set the tag-order style to specify a set of tags to be + completed at the same time. + + If NAME and DESCR are given, _requested calls the _description + function with these arguments together with the options passed to + _requested. + + If COMMAND is given, the _all_labels function will be called + immediately with the same arguments. In simple cases this makes it + possible to perform the test for the tag and the matching in one + go. For example: + + + local expl ret=1 + _tags foo bar baz + while _tags; do + _requested foo expl 'description' \ + compadd foobar foobaz && ret=0 + ... + (( ret )) || break + done + + If the COMMAND is not compadd, it must nevertheless be prepared to + handle the same options. + +_retrieve_cache CACHE_IDENTIFIER + This function retrieves completion information from the file given + by CACHE_IDENTIFIER, stored in a directory specified by the + cache-path style which defaults to ~/.zcompcache. The return + status is zero if retrieval was successful. It will only attempt + retrieval if the use-cache style is set, so you can call this + function without worrying about whether the user wanted to use the + caching layer. + + See _store_cache below for more details. + +_sep_parts + This function is passed alternating arrays and separators as + arguments. The arrays specify completions for parts of strings to + be separated by the separators. The arrays may be the names of + array parameters or a quoted list of words in parentheses. For + example, with the array `hosts=(ftp news)' the call `_sep_parts + '(foo bar)' @ hosts' will complete the string `f' to `foo' and + the string `b@n' to `bar@news'. + + This function accepts the compadd options `-V', `-J', `-1', `-2', + `-n', `-X', `-M', `-P', `-S', `-r', `-R', and `-q' and passes them + on to the compadd builtin used to add the matches. + +_sequence [ -s SEP ] [ -n MAX ] [ -d ] FUNCTION [ - ] ... + This function is a wrapper to other functions for completing items + in a separated list. The same function is used to complete each + item in the list. The separator is specified with the -s option. + If -s is omitted it will use `,'. Duplicate values are not matched + unless -d is specified. If there is a fixed or maximum number of + items in the list, this can be specified with the -n option. + + Common compadd options are passed on to the function. It is + possible to use compadd directly with _sequence, though _values may + be more appropriate in this situation. + +_setup TAG [ GROUP ] + This function sets up the special parameters used by the + completion system appropriately for the TAG given as the first + argument. It uses the styles list-colors, list-packed, + list-rows-first, last-prompt, accept-exact, menu and force-list. + + The optional GROUP supplies the name of the group in which the + matches will be placed. If it is not given, the TAG is used as + the group name. + + This function is called automatically from _description and hence + is not normally called explicitly. + +_store_cache CACHE_IDENTIFIER PARAM ... + This function, together with _retrieve_cache and _cache_invalid, + implements a caching layer which can be used in any completion + function. Data obtained by costly operations are stored in + parameters; this function then dumps the values of those + parameters to a file. The data can then be retrieved quickly from + that file via _retrieve_cache, even in different instances of the + shell. + + The CACHE_IDENTIFIER specifies the file which the data should be + dumped to. The file is stored in a directory specified by the + cache-path style which defaults to ~/.zcompcache. The remaining + PARAMs arguments are the parameters to dump to the file. + + The return status is zero if storage was successful. The function + will only attempt storage if the use-cache style is set, so you can + call this function without worrying about whether the user wanted + to use the caching layer. + + The completion function may avoid calling _retrieve_cache when it + already has the completion data available as parameters. However, + in that case it should call _cache_invalid to check whether the + data in the parameters and in the cache are still valid. + + See the _perl_modules completion function for a simple example of + the usage of the caching layer. + +_tags [ [ -C NAME ] TAG ... ] + If called with arguments, these are taken to be the names of tags + valid for completions in the current context. These tags are + stored internally and sorted by using the tag-order style. + + Next, _tags is called repeatedly without arguments from the same + completion function. This successively selects the first, second, + etc. set of tags requested by the user. The return status is zero + if at least one of the tags is requested and non-zero otherwise. + To test if a particular tag is to be tried, the _requested + function should be called (see above). + + If `-C NAME' is given, NAME is temporarily stored in the ARGUMENT + field (the fifth) of the context in the curcontext parameter + during the call to _tags; the field is restored on exit. This + allows _tags to use a more specific context without having to + change and reset the curcontext parameter (which has the same + effect). + +_tilde_files + Like _files, but resolve leading tildes according to the rules of + filename expansion, so the suggested completions don't start with + a `~' even if the filename on the command-line does. + +_values [ -O NAME ] [ -s SEP ] [ -S SEP ] [ -wC ] DESC SPEC ... + This is used to complete arbitrary keywords (values) and their + arguments, or lists of such combinations. + + If the first argument is the option `-O NAME', it will be used in + the same way as by the _arguments function. In other words, the + elements of the NAME array will be passed to compadd when + executing an action. + + If the first argument (or the first argument after `-O NAME') is + `-s', the next argument is used as the character that separates + multiple values. This character is automatically added after each + value in an auto-removable fashion (see below); all values + completed by `_values -s' appear in the same word on the command + line, unlike completion using _arguments. If this option is not + present, only a single value will be completed per word. + + Normally, _values will only use the current word to determine + which values are already present on the command line and hence are + not to be completed again. If the -w option is given, other + arguments are examined as well. + + The first non-option argument, DESC, is used as a string to print + as a description before listing the values. + + All other arguments describe the possible values and their + arguments in the same format used for the description of options by + the _arguments function (see above). The only differences are that + no minus or plus sign is required at the beginning, values can + have only one argument, and the forms of action beginning with an + equal sign are not supported. + + The character separating a value from its argument can be set + using the option -S (like -s, followed by the character to use as + the separator in the next argument). By default the equals sign + will be used as the separator between values and arguments. + + Example: + + + _values -s , 'description' \ + '*foo[bar]' \ + '(two)*one[number]:first count:' \ + 'two[another number]::second count:(1 2 3)' + + This describes three possible values: `foo', `one', and `two'. + The first is described as `bar', takes no argument and may appear + more than once. The second is described as `number', may appear + more than once, and takes one mandatory argument described as + `first count'; no action is specified, so it will not be + completed. The `(two)' at the beginning says that if the value + `one' is on the line, the value `two' will no longer be considered + a possible completion. Finally, the last value (`two') is + described as `another number' and takes an optional argument + described as `second count' for which the completions (to appear + after an `=') are `1', `2', and `3'. The _values function will + complete lists of these values separated by commas. + + Like _arguments, this function temporarily adds another context + name component to the arguments element (the fifth) of the current + context while executing the ACTION. Here this name is just the + name of the value for which the argument is completed. + + The style verbose is used to decide if the descriptions for the + values (but not those for the arguments) should be printed. + + The associative array val_args is used to report values and their + arguments; this works similarly to the opt_args associative array + used by _arguments. Hence the function calling _values should + declare the local parameters state, state_descr, line, context and + val_args: + + + local context state state_descr line + typeset -A val_args + + when using an action of the form `->STRING'. With this function + the context parameter will be set to the name of the value whose + argument is to be completed. Note that for _values, the state and + state_descr are scalars rather than arrays. Only a single + matching state is returned. + + Note also that _values normally adds the character used as the + separator between values as an auto-removable suffix (similar to a + `/' after a directory). However, this is not possible for a + `->STRING' action as the matches for the argument are generated by + the calling function. To get the usual behaviour, the calling + function can add the separator X as a suffix by passing the + options `-qS X' either directly or indirectly to compadd. + + The option -C is treated in the same way as it is by _arguments. + In that case the parameter curcontext should be made local instead + of context (as described above). + +_wanted [ -x ] [ -C NAME ] [ -12VJ ] TAG NAME DESCR COMMAND [ ARG ...] + In many contexts, completion can only generate one particular set + of matches, usually corresponding to a single tag. However, it is + still necessary to decide whether the user requires matches of + this type. This function is useful in such a case. + + The arguments to _wanted are the same as those to _requested, i.e. + arguments to be passed to _description. However, in this case the + COMMAND is not optional; all the processing of tags, including + the loop over both tags and tag labels and the generation of + matches, is carried out automatically by _wanted. + + Hence to offer only one tag and immediately add the corresponding + matches with the given description: + + + local expl + _wanted tag expl 'description' \ + compadd matches... + + Note that, as for _requested, the COMMAND must be able to accept + options to be passed down to compadd. + + Like _tags this function supports the -C option to give a + different name for the argument context field. The -x option has + the same meaning as for _description. + +_widgets [ -g PATTERN ] + This function completes names of zle widgets (see *Note Zle + Widgets::). The PATTERN, if present, is matched against values of + the $widgets special parameter, documented in *Note The + zsh/zleparameter Module::. + + + + +File: zsh.info, Node: Completion System Variables, Next: Completion Directories, Prev: Completion Functions, Up: Completion System + +20.7 Completion System Variables +================================ + + + +There are some standard variables, initialised by the _main_complete +function and then used from other functions. + +The standard variables are: + + +_comp_caller_options + The completion system uses setopt to set a number of options. This + allows functions to be written without concern for compatibility + with every possible combination of user options. However, + sometimes completion needs to know what the user's option + preferences are. These are saved in the _comp_caller_options + associative array. Option names, spelled in lowercase without + underscores, are mapped to one or other of the strings `on' and + `off'. + + + +_comp_priv_prefix + Completion functions such as _sudo can set the _comp_priv_prefix + array to a command prefix that may then be used by _call_program to + match the privileges when calling programs to generate matches. + + +Two more features are offered by the _main_complete function. The +arrays compprefuncs and comppostfuncs may contain names of functions +that are to be called immediately before or after completion has been +tried. A function will only be called once unless it explicitly +reinserts itself into the array. + + + + +File: zsh.info, Node: Completion Directories, Prev: Completion System Variables, Up: Completion System + +20.8 Completion Directories +=========================== + + + +In the source distribution, the files are contained in various +subdirectories of the Completion directory. They may have been +installed in the same structure, or into one single function directory. +The following is a description of the files found in the original +directory structure. If you wish to alter an installed file, you will +need to copy it to some directory which appears earlier in your fpath +than the standard directory where it appears. + + +Base + The core functions and special completion widgets automatically + bound to keys. You will certainly need most of these, though will + probably not need to alter them. Many of these are documented + above. + +Zsh + Functions for completing arguments of shell builtin commands and + utility functions for this. Some of these are also used by + functions from the Unix directory. + +Unix + Functions for completing arguments of external commands and suites + of commands. They may need modifying for your system, although in + many cases some attempt is made to decide which version of a + command is present. For example, completion for the mount command + tries to determine the system it is running on, while completion + for many other utilities try to decide whether the GNU version of + the command is in use, and hence whether the --help option is + supported. + +X, AIX, BSD, ... + Completion and utility function for commands available only on + some systems. These are not arranged hierarchically, so, for + example, both the Linux and Debian directories, as well as the X + directory, may be useful on your system. + + + +File: zsh.info, Node: Completion Using compctl, Next: Zsh Modules, Prev: Completion System, Up: Top + +21 Completion Using compctl +*************************** + + + +21.1 Types of completion +======================== + +This version of zsh has two ways of performing completion of words on +the command line. New users of the shell may prefer to use the newer +and more powerful system based on shell functions; this is described in +*Note Completion System::, and the basic shell mechanisms which support +it are described in *Note Completion Widgets::. This chapter describes +the older compctl command. + +21.2 Description +================ + + + +compctl [ -CDT ] OPTIONS [ COMMAND ... ] + +compctl [ -CDT ] OPTIONS [ -x PATTERN OPTIONS - ... -- ] + + [ + OPTIONS [ -x ... -- ] ... [+] ] [ COMMAND ... ] + +compctl -M MATCH-SPECS ... + +compctl -L [ -CDTM ] [ COMMAND ... ] + +compctl + COMMAND ... + + +Control the editor's completion behavior according to the supplied set +of OPTIONS. Various editing commands, notably expand-or-complete-word, +usually bound to tab, will attempt to complete a word typed by the +user, while others, notably delete-char-or-list, usually bound to ^D in +EMACS editing mode, list the possibilities; compctl controls what those +possibilities are. They may for example be filenames (the most common +case, and hence the default), shell variables, or words from a +user-specified list. + +* Menu: + +* Command Flags:: +* Option Flags:: +* Alternative Completion:: +* Extended Completion:: +* Example:: + + + +File: zsh.info, Node: Command Flags, Next: Option Flags, Up: Completion Using compctl + +21.3 Command Flags +================== + +Completion of the arguments of a command may be different for each +command or may use the default. The behavior when completing the +command word itself may also be separately specified. These correspond +to the following flags and arguments, all of which (except for -L) may +be combined with any combination of the OPTIONS described subsequently +in *Note Option Flags::: + + +COMMAND ... + controls completion for the named commands, which must be listed + last on the command line. If completion is attempted for a + command with a pathname containing slashes and no completion + definition is found, the search is retried with the last pathname + component. If the command starts with a =, completion is tried + with the pathname of the command. + + Any of the COMMAND strings may be patterns of the form normally + used for filename generation. These should be quoted to protect + them from immediate expansion; for example the command string + 'foo*' arranges for completion of the words of any command + beginning with foo. When completion is attempted, all pattern + completions are tried in the reverse order of their definition + until one matches. By default, completion then proceeds as + normal, i.e. the shell will try to generate more matches for the + specific command on the command line; this can be overridden by + including -tn in the flags for the pattern completion. + + Note that aliases are expanded before the command name is + determined unless the COMPLETE_ALIASES option is set. Commands + may not be combined with the -C, -D or -T flags. + +-C + controls completion when the command word itself is being + completed. If no compctl -C command has been issued, the names + of any executable command (whether in the path or specific to the + shell, such as aliases or functions) are completed. + +-D + controls default completion behavior for the arguments of commands + not assigned any special behavior. If no compctl -D command has + been issued, filenames are completed. + +-T + supplies completion flags to be used before any other processing is + done, even before processing for compctls defined for specific + commands. This is especially useful when combined with extended + completion (the -x flag, see *Note Extended Completion:: below). + Using this flag you can define default behavior which will apply + to all commands without exception, or you can alter the standard + behavior for all commands. For example, if your access to the + user database is too slow and/or it contains too many users (so + that completion after `~' is too slow to be usable), you can use + + + compctl -T -x 's[~] C[0,[^/]#]' -k friends -S/ -tn + + to complete the strings in the array friends after a `~'. The + C[...] argument is necessary so that this form of ~-completion is + not tried after the directory name is finished. + +-L + lists the existing completion behavior in a manner suitable for + putting into a start-up script; the existing behavior is not + changed. Any combination of the above forms, or the -M flag + (which must follow the -L flag), may be specified, otherwise all + defined completions are listed. Any other flags supplied are + ignored. + +_no argument_ + If no argument is given, compctl lists all defined completions in + an abbreviated form; with a list of OPTIONS, all completions with + those flags set (not counting extended completion) are listed. + + +If the + flag is alone and followed immediately by the COMMAND list, +the completion behavior for all the commands in the list is reset to +the default. In other words, completion will subsequently use the +options specified by the -D flag. + +The form with -M as the first and only option defines global matching +specifications (see *Note Completion Matching Control::). The match +specifications given will be used for every completion attempt (only +when using compctl, not with the new completion system) and are tried +in the order in which they are defined until one generates at least one +match. E.g.: + + + compctl -M '' 'm:{a-zA-Z}={A-Za-z}' + +This will first try completion without any global match specifications +(the empty string) and, if that generates no matches, will try case +insensitive completion. + + + + +File: zsh.info, Node: Option Flags, Next: Alternative Completion, Prev: Command Flags, Up: Completion Using compctl + +21.4 Option Flags +================= + + +[ -fcFBdeaRGovNAIOPZEnbjrzu/12 ] + +[ -k ARRAY ] [ -g GLOBSTRING ] [ -s SUBSTSTRING ] + +[ -K FUNCTION ] + +[ -Q ] [ -P PREFIX ] [ -S SUFFIX ] + +[ -W FILE-PREFIX ] [ -H NUM PATTERN ] + +[ -q ] [ -X EXPLANATION ] [ -Y EXPLANATION ] + +[ -y FUNC-OR-VAR ] [ -l CMD ] [ -h CMD ] [ -U ] + +[ -t CONTINUE ] [ -J NAME ] [ -V NAME ] + +[ -M MATCH-SPEC ] + + +The remaining OPTIONS specify the type of command arguments to look for +during completion. Any combination of these flags may be specified; +the result is a sorted list of all the possibilities. The options are +as follows. + +* Menu: + +* Simple Flags:: +* Flags with Arguments:: +* Control Flags:: + + + +File: zsh.info, Node: Simple Flags, Next: Flags with Arguments, Up: Option Flags + +21.4.1 Simple Flags +------------------- + +These produce completion lists made up by the shell itself: + + +-f + Filenames and file system paths. + +-/ + Just file system paths. + +-c + Command names, including aliases, shell functions, builtins and + reserved words. + +-F + Function names. + +-B + Names of builtin commands. + +-m + Names of external commands. + +-w + Reserved words. + +-a + Alias names. + +-R + Names of regular (non-global) aliases. + +-G + Names of global aliases. + +-d + This can be combined with -F, -B, -w, -a, -R and -G to get names + of disabled functions, builtins, reserved words or aliases. + +-e + This option (to show enabled commands) is in effect by default, but + may be combined with -d; -de in combination with -F, -B, -w, -a, + -R and -G will complete names of functions, builtins, reserved + words or aliases whether or not they are disabled. + +-o + Names of shell options (see *Note Options::). + +-v + Names of any variable defined in the shell. + +-N + Names of scalar (non-array) parameters. + +-A + Array names. + +-I + Names of integer variables. + +-O + Names of read-only variables. + +-p + Names of parameters used by the shell (including special + parameters). + +-Z + Names of shell special parameters. + +-E + Names of environment variables. + +-n + Named directories. + +-b + Key binding names. + +-j + Job names: the first word of the job leader's command line. This + is useful with the kill builtin. + +-r + Names of running jobs. + +-z + Names of suspended jobs. + +-u + User names. + + + + +File: zsh.info, Node: Flags with Arguments, Next: Control Flags, Prev: Simple Flags, Up: Option Flags + +21.4.2 Flags with Arguments +--------------------------- + +These have user supplied arguments to determine how the list of +completions is to be made up: + + +-k ARRAY + Names taken from the elements of $ARRAY (note that the `$' does + not appear on the command line). Alternatively, the argument + ARRAY itself may be a set of space- or comma-separated values in + parentheses, in which any delimiter may be escaped with a + backslash; in this case the argument should be quoted. For + example, + + + compctl -k "(cputime filesize datasize stacksize + coredumpsize resident descriptors)" limit + +-g GLOBSTRING + The GLOBSTRING is expanded using filename globbing; it should be + quoted to protect it from immediate expansion. The resulting + filenames are taken as the possible completions. Use `*(/)' + instead of `*/' for directories. The fignore special parameter is + not applied to the resulting files. More than one pattern may be + given separated by blanks. (Note that brace expansion is _not_ + part of globbing. Use the syntax `(either|or)' to match + alternatives.) + +-s SUBSTSTRING + The SUBSTSTRING is split into words and these words are than + expanded using all shell expansion mechanisms (see *Note + Expansion::). The resulting words are taken as possible + completions. The fignore special parameter is not applied to the + resulting files. Note that -g is faster for filenames. + +-K FUNCTION + Call the given function to get the completions. Unless the name + starts with an underscore, the function is passed two arguments: + the prefix and the suffix of the word on which completion is to be + attempted, in other words those characters before the cursor + position, and those from the cursor position onwards. The whole + command line can be accessed with the -c and -l flags of the read + builtin. The function should set the variable reply to an array + containing the completions (one completion per element); note that + reply should not be made local to the function. From such a + function the command line can be accessed with the -c and -l flags + to the read builtin. For example, + + + function whoson { reply=(`users`); } + compctl -K whoson talk + + completes only logged-on users after `talk'. Note that `whoson' + must return an array, so `reply=`users`' would be incorrect. + +-H NUM PATTERN + The possible completions are taken from the last NUM history + lines. Only words matching PATTERN are taken. If NUM is zero or + negative the whole history is searched and if PATTERN is the empty + string all words are taken (as with `*'). A typical use is + + + compctl -D -f + -H 0 '' + + which forces completion to look back in the history list for a + word if no filename matches. + + + + +File: zsh.info, Node: Control Flags, Prev: Flags with Arguments, Up: Option Flags + +21.4.3 Control Flags +-------------------- + +These do not directly specify types of name to be completed, but +manipulate the options that do: + + +-Q + This instructs the shell not to quote any metacharacters in the + possible completions. Normally the results of a completion are + inserted into the command line with any metacharacters quoted so + that they are interpreted as normal characters. This is + appropriate for filenames and ordinary strings. However, for + special effects, such as inserting a backquoted expression from a + completion array (-k) so that the expression will not be evaluated + until the complete line is executed, this option must be used. + +-P PREFIX + The PREFIX is inserted just before the completed string; any + initial part already typed will be completed and the whole PREFIX + ignored for completion purposes. For example, + + + compctl -j -P "%" kill + + inserts a `%' after the kill command and then completes job names. + +-S SUFFIX + When a completion is found the SUFFIX is inserted after the + completed string. In the case of menu completion the suffix is + inserted immediately, but it is still possible to cycle through the + list of completions by repeatedly hitting the same key. + +-W FILE-PREFIX + With directory FILE-PREFIX: for command, file, directory and + globbing completion (options -c, -f, -/, -g), the file prefix is + implicitly added in front of the completion. For example, + + + compctl -/ -W ~/Mail maildirs + + completes any subdirectories to any depth beneath the directory + ~/Mail, although that prefix does not appear on the command line. + The FILE-PREFIX may also be of the form accepted by the -k flag, + i.e. the name of an array or a literal list in parenthesis. In + this case all the directories in the list will be searched for + possible completions. + +-q + If used with a suffix as specified by the -S option, this causes + the suffix to be removed if the next character typed is a blank or + does not insert anything or if the suffix consists of only one + character and the next character typed is the same character; this + the same rule used for the AUTO_REMOVE_SLASH option. The option + is most useful for list separators (comma, colon, etc.). + +-l CMD + This option restricts the range of command line words that are + considered to be arguments. If combined with one of the extended + completion patterns `p[...]', `r[...]', or `R[...]' (see *Note + Extended Completion:: below) the range is restricted to the range + of arguments specified in the brackets. Completion is then + performed as if these had been given as arguments to the CMD + supplied with the option. If the CMD string is empty the first + word in the range is instead taken as the command name, and + command name completion performed on the first word in the range. + For example, + + + compctl -x 'r[-exec,;]' -l '' -- find + + completes arguments between `-exec' and the following `;' (or the + end of the command line if there is no such string) as if they were + a separate command line. + +-h CMD + Normally zsh completes quoted strings as a whole. With this option, + completion can be done separately on different parts of such + strings. It works like the -l option but makes the completion code + work on the parts of the current word that are separated by + spaces. These parts are completed as if they were arguments to the + given CMD. If CMD is the empty string, the first part is completed + as a command name, as with -l. + +-U + Use the whole list of possible completions, whether or not they + actually match the word on the command line. The word typed so far + will be deleted. This is most useful with a function (given by the + -K option) which can examine the word components passed to it (or + via the read builtin's -c and -l flags) and use its own criteria + to decide what matches. If there is no completion, the original + word is retained. Since the produced possible completions seldom + have interesting common prefixes and suffixes, menu completion is + started immediately if AUTO_MENU is set and this flag is used. + +-y FUNC-OR-VAR + The list provided by FUNC-OR-VAR is displayed instead of the list + of completions whenever a listing is required; the actual + completions to be inserted are not affected. It can be provided + in two ways. Firstly, if FUNC-OR-VAR begins with a $ it defines a + variable, or if it begins with a left parenthesis a literal array, + which contains the list. A variable may have been set by a call + to a function using the -K option. Otherwise it contains the name + of a function which will be executed to create the list. The + function will be passed as an argument list all matching + completions, including prefixes and suffixes expanded in full, and + should set the array reply to the result. In both cases, the + display list will only be retrieved after a complete list of + matches has been created. + + Note that the returned list does not have to correspond, even in + length, to the original set of matches, and may be passed as a + scalar instead of an array. No special formatting of characters is + performed on the output in this case; in particular, newlines are + printed literally and if they appear output in columns is + suppressed. + +-X EXPLANATION + Print EXPLANATION when trying completion on the current set of + options. A `%n' in this string is replaced by the number of + matches that were added for this explanation string. The + explanation only appears if completion was tried and there was no + unique match, or when listing completions. Explanation strings + will be listed together with the matches of the group specified + together with the -X option (using the -J or -V option). If the + same explanation string is given to multiple -X options, the + string appears only once (for each group) and the number of + matches shown for the `%n' is the total number of all matches for + each of these uses. In any case, the explanation string will only + be shown if there was at least one match added for the explanation + string. + + The sequences %B, %b, %S, %s, %U, and %u specify output attributes + (bold, standout, and underline), %F, %f, %K, %k specify foreground + and background colours, and %{...%} can be used to include literal + escape sequences as in prompts. + +-Y EXPLANATION + Identical to -X, except that the EXPLANATION first undergoes + expansion following the usual rules for strings in double quotes. + The expansion will be carried out after any functions are called + for the -K or -y options, allowing them to set variables. + +-t CONTINUE + The CONTINUE-string contains a character that specifies which set + of completion flags should be used next. It is useful: + + (i) With -T, or when trying a list of pattern completions, when + compctl would usually continue with ordinary processing after + finding matches; this can be suppressed with `-tn'. + + (ii) With a list of alternatives separated by +, when compctl + would normally stop when one of the alternatives generates + matches. It can be forced to consider the next set of completions + by adding `-t+' to the flags of the alternative before the `+'. + + (iii) In an extended completion list (see below), when compctl + would normally continue until a set of conditions succeeded, then + use only the immediately following flags. With `-t-', compctl will + continue trying extended completions after the next `-'; with + `-tx' it will attempt completion with the default flags, in other + words those before the `-x'. + +-J NAME + This gives the name of the group the matches should be placed in. + Groups are listed and sorted separately; likewise, menu completion + will offer the matches in the groups in the order in which the + groups were defined. If no group name is explicitly given, the + matches are stored in a group named default. The first time a + group name is encountered, a group with that name is created. + After that all matches with the same group name are stored in that + group. + + This can be useful with non-exclusive alternative completions. For + example, in + + + compctl -f -J files -t+ + -v -J variables foo + + both files and variables are possible completions, as the -t+ + forces both sets of alternatives before and after the + to be + considered at once. Because of the -J options, however, all files + are listed before all variables. + +-V NAME + Like -J, but matches within the group will not be sorted in + listings nor in menu completion. These unsorted groups are in a + different name space from the sorted ones, so groups defined as -J + files and -V files are distinct. + +-1 + If given together with the -V option, makes only consecutive + duplicates in the group be removed. Note that groups with and + without this flag are in different name spaces. + +-2 + If given together with the -J or -V option, makes all duplicates + be kept. Again, groups with and without this flag are in different + name spaces. + +-M MATCH-SPEC + This defines additional matching control specifications that + should be used only when testing words for the list of flags this + flag appears in. The format of the MATCH-SPEC string is described + in *Note Completion Matching Control::. + + + + +File: zsh.info, Node: Alternative Completion, Next: Extended Completion, Prev: Option Flags, Up: Completion Using compctl + +21.5 Alternative Completion +=========================== + + +compctl [ -CDT ] OPTIONS + OPTIONS [ + ... ] [ + ] COMMAND ... + + +The form with `+' specifies alternative options. Completion is tried +with the options before the first `+'. If this produces no matches +completion is tried with the flags after the `+' and so on. If there +are no flags after the last `+' and a match has not been found up to +that point, default completion is tried. If the list of flags contains +a -t with a + character, the next list of flags is used even if the +current list produced matches. + + + + +File: zsh.info, Node: Extended Completion, Next: Example, Prev: Alternative Completion, Up: Completion Using compctl + +Additional options are available that restrict completion to some part +of the command line; this is referred to as `extended completion'. + + + +21.6 Extended Completion +======================== + + +compctl [ -CDT ] OPTIONS -x PATTERN OPTIONS - ... -- + + [ COMMAND ... ] + +compctl [ -CDT ] OPTIONS [ -x PATTERN OPTIONS - ... -- ] + + [ + OPTIONS [ -x ... -- ] ... [+] ] [ COMMAND ... ] + + +The form with `-x' specifies extended completion for the commands +given; as shown, it may be combined with alternative completion using +`+'. Each PATTERN is examined in turn; when a match is found, the +corresponding OPTIONS, as described in *Note Option Flags:: above, are +used to generate possible completions. If no PATTERN matches, the +OPTIONS given before the -x are used. + +Note that each pattern should be supplied as a single argument and +should be quoted to prevent expansion of metacharacters by the shell. + +A PATTERN is built of sub-patterns separated by commas; it matches if +at least one of these sub-patterns matches (they are `or'ed). These +sub-patterns are in turn composed of other sub-patterns separated by +white spaces which match if all of the sub-patterns match (they are +`and'ed). An element of the sub-patterns is of the form `C[...][...]', +where the pairs of brackets may be repeated as often as necessary, and +matches if any of the sets of brackets match (an `or'). The example +below makes this clearer. + +The elements may be any of the following: + + +s[STRING]... + Matches if the current word on the command line starts with one of + the strings given in brackets. The STRING is not removed and is + not part of the completion. + +S[STRING]... + Like s[STRING] except that the STRING is part of the completion. + +p[FROM,TO]... + Matches if the number of the current word is between one of the + FROM and TO pairs inclusive. The comma and TO are optional; TO + defaults to the same value as FROM. The numbers may be negative: + -N refers to the N'th last word on the line. + +c[OFFSET,STRING]... + Matches if the STRING matches the word offset by OFFSET from the + current word position. Usually OFFSET will be negative. + +C[OFFSET,PATTERN]... + Like c but using pattern matching instead. + +w[INDEX,STRING]... + Matches if the word in position INDEX is equal to the + corresponding STRING. Note that the word count is made after any + alias expansion. + +W[INDEX,PATTERN]... + Like w but using pattern matching instead. + +n[INDEX,STRING]... + Matches if the current word contains STRING. Anything up to and + including the INDEXth occurrence of this string will not be + considered part of the completion, but the rest will. INDEX may + be negative to count from the end: in most cases, INDEX will be 1 + or -1. For example, + + + compctl -s '`users`' -x 'n[1,@]' -k hosts -- talk + + will usually complete usernames, but if you insert an @ after the + name, names from the array HOSTS (assumed to contain hostnames, + though you must make the array yourself) will be completed. Other + commands such as rcp can be handled similarly. + +N[INDEX,STRING]... + Like n except that the string will be taken as a character class. + Anything up to and including the INDEXth occurrence of any of the + characters in STRING will not be considered part of the completion. + +m[MIN,MAX]... + Matches if the total number of words lies between MIN and MAX + inclusive. + +r[STR1,STR2]... + Matches if the cursor is after a word with prefix STR1. If there + is also a word with prefix STR2 on the command line after the one + matched by STR1 it matches only if the cursor is before this word. + If the comma and STR2 are omitted, it matches if the cursor is + after a word with prefix STR1. + +R[STR1,STR2]... + Like r but using pattern matching instead. + +q[STR]... + Matches the word currently being completed is in single quotes and + the STR begins with the letter `s', or if completion is done in + double quotes and STR starts with the letter `d', or if completion + is done in backticks and STR starts with a `b'. + + + + +File: zsh.info, Node: Example, Prev: Extended Completion, Up: Completion Using compctl + +21.7 Example +============ + + + compctl -u -x 's[+] c[-1,-f],s[-f+]' \ + -g '~/Mail/*(:t)' - 's[-f],c[-1,-f]' -f -- mail + +This is to be interpreted as follows: + +If the current command is mail, then + + + + if ((the current word begins with + and the previous word is -f) + or (the current word begins with -f+)), then complete the + non-directory part (the `:t' glob modifier) of files in the + directory ~/Mail; else + + if the current word begins with -f or the previous word was -f, + then complete any file; else + + complete user names. + + + +File: zsh.info, Node: Zsh Modules, Next: Calendar Function System, Prev: Completion Using compctl, Up: Top + +22 Zsh Modules +************** + + + +22.1 Description +================ + +Some optional parts of zsh are in modules, separate from the core of +the shell. Each of these modules may be linked in to the shell at +build time, or can be dynamically linked while the shell is running if +the installation supports this feature. Modules are linked at runtime +with the zmodload command, see *Note Shell Builtin Commands::. + +The modules that are bundled with the zsh distribution are: + + +zsh/attr + Builtins for manipulating extended attributes (xattr). + +zsh/cap + Builtins for manipulating POSIX.1e (POSIX.6) capability + (privilege) sets. + +zsh/clone + A builtin that can clone a running shell onto another terminal. + +zsh/compctl + The compctl builtin for controlling completion. + +zsh/complete + The basic completion code. + +zsh/complist + Completion listing extensions. + +zsh/computil + A module with utility builtins needed for the shell function based + completion system. + +zsh/curses + curses windowing commands + +zsh/datetime + Some date/time commands and parameters. + +zsh/db/gdbm + Builtins for managing associative array parameters tied to GDBM + databases. + +zsh/deltochar + A ZLE function duplicating EMACS' zap-to-char. + +zsh/example + An example of how to write a module. + +zsh/files + Some basic file manipulation commands as builtins. + +zsh/langinfo + Interface to locale information. + +zsh/mapfile + Access to external files via a special associative array. + +zsh/mathfunc + Standard scientific functions for use in mathematical evaluations. + +zsh/nearcolor + Map colours to the nearest colour in the available palette. + +zsh/newuser + Arrange for files for new users to be installed. + +zsh/parameter + Access to internal hash tables via special associative arrays. + +zsh/pcre + Interface to the PCRE library. + +zsh/param/private + Builtins for managing private-scoped parameters in function + context. + +zsh/regex + Interface to the POSIX regex library. + +zsh/sched + A builtin that provides a timed execution facility within the + shell. + +zsh/net/socket + Manipulation of Unix domain sockets + +zsh/stat + A builtin command interface to the stat system call. + +zsh/system + A builtin interface to various low-level system features. + +zsh/net/tcp + Manipulation of TCP sockets + +zsh/termcap + Interface to the termcap database. + +zsh/terminfo + Interface to the terminfo database. + +zsh/zftp + A builtin FTP client. + +zsh/zle + The Zsh Line Editor, including the bindkey and vared builtins. + +zsh/zleparameter + Access to internals of the Zsh Line Editor via parameters. + +zsh/zprof + A module allowing profiling for shell functions. + +zsh/zpty + A builtin for starting a command in a pseudo-terminal. + +zsh/zselect + Block and return when file descriptors are ready. + +zsh/zutil + Some utility builtins, e.g. the one for supporting configuration + via styles. + + +* Menu: + +* The zsh/attr Module:: +* The zsh/cap Module:: +* The zsh/clone Module:: +* The zsh/compctl Module:: +* The zsh/complete Module:: +* The zsh/complist Module:: +* The zsh/computil Module:: +* The zsh/curses Module:: +* The zsh/datetime Module:: +* The zsh/db/gdbm Module:: +* The zsh/deltochar Module:: +* The zsh/example Module:: +* The zsh/files Module:: +* The zsh/langinfo Module:: +* The zsh/mapfile Module:: +* The zsh/mathfunc Module:: +* The zsh/nearcolor Module:: +* The zsh/newuser Module:: +* The zsh/parameter Module:: +* The zsh/pcre Module:: +* The zsh/param/private Module:: +* The zsh/regex Module:: +* The zsh/sched Module:: +* The zsh/net/socket Module:: +* The zsh/stat Module:: +* The zsh/system Module:: +* The zsh/net/tcp Module:: +* The zsh/termcap Module:: +* The zsh/terminfo Module:: +* The zsh/zftp Module:: +* The zsh/zle Module:: +* The zsh/zleparameter Module:: +* The zsh/zprof Module:: +* The zsh/zpty Module:: +* The zsh/zselect Module:: +* The zsh/zutil Module:: + + +File: zsh.info, Node: The zsh/attr Module, Next: The zsh/cap Module, Up: Zsh Modules + +22.2 The zsh/attr Module +======================== + + + + The zsh/attr module is used for manipulating extended attributes. +The -h option causes all commands to operate on symbolic links instead +of their targets. The builtins in this module are: + + +zgetattr [ -h ] FILENAME ATTRIBUTE [ PARAMETER ] + Get the extended attribute ATTRIBUTE from the specified FILENAME. + If the optional argument PARAMETER is given, the attribute is set + on that parameter instead of being printed to stdout. + +zsetattr [ -h ] FILENAME ATTRIBUTE VALUE + Set the extended attribute ATTRIBUTE on the specified FILENAME to + VALUE. + +zdelattr [ -h ] FILENAME ATTRIBUTE + Remove the extended attribute ATTRIBUTE from the specified + FILENAME. + +zlistattr [ -h ] FILENAME [ PARAMETER ] + List the extended attributes currently set on the specified + FILENAME. If the optional argument PARAMETER is given, the list of + attributes is set on that parameter instead of being printed to + stdout. + + +zgetattr and zlistattr allocate memory dynamically. If the attribute +or list of attributes grows between the allocation and the call to get +them, they return 2. On all other errors, 1 is returned. This allows +the calling function to check for this case and retry. + + +File: zsh.info, Node: The zsh/cap Module, Next: The zsh/clone Module, Prev: The zsh/attr Module, Up: Zsh Modules + +22.3 The zsh/cap Module +======================= + + + + The zsh/cap module is used for manipulating POSIX.1e (POSIX.6) +capability sets. If the operating system does not support this +interface, the builtins defined by this module will do nothing. The +builtins in this module are: + + +cap [ CAPABILITIES ] + Change the shell's process capability sets to the specified + CAPABILITIES, otherwise display the shell's current capabilities. + +getcap FILENAME ... + This is a built-in implementation of the POSIX standard utility. + It displays the capability sets on each specified FILENAME. + +setcap CAPABILITIES FILENAME ... + This is a built-in implementation of the POSIX standard utility. + It sets the capability sets on each specified FILENAME to the + specified CAPABILITIES. + + + +File: zsh.info, Node: The zsh/clone Module, Next: The zsh/compctl Module, Prev: The zsh/cap Module, Up: Zsh Modules + +22.4 The zsh/clone Module +========================= + + + + The zsh/clone module makes available one builtin command: + + +clone TTY + Creates a forked instance of the current shell, attached to the + specified TTY. In the new shell, the PID, PPID and TTY special + parameters are changed appropriately. $! is set to zero in the new + shell, and to the new shell's PID in the original shell. + + The return status of the builtin is zero in both shells if + successful, and non-zero on error. + + The target of clone should be an unused terminal, such as an + unused virtual console or a virtual terminal created by + + + xterm -e sh -c 'trap : INT QUIT TSTP; tty; + while :; do sleep 100000000; done' + + Some words of explanation are warranted about this long xterm + command line: when doing clone on a pseudo-terminal, some other + session ("session" meant as a unix session group, or SID) is + already owning the terminal. Hence the cloned zsh cannot acquire + the pseudo-terminal as a controlling tty. That means two things: + + + * the job control signals will go to the sh-started-by-xterm + process group (that's why we disable INT QUIT and TSTP with + trap; otherwise the while loop could get suspended or killed) + + * the cloned shell will have job control disabled, and the job + control keys (control-C, control-\ and control-Z) will not + work. + + This does not apply when cloning to an _unused_ vc. + + Cloning to a used (and unprepared) terminal will result in two + processes reading simultaneously from the same terminal, with + input bytes going randomly to either process. + + clone is mostly useful as a shell built-in replacement for openvt. + + + +File: zsh.info, Node: The zsh/compctl Module, Next: The zsh/complete Module, Prev: The zsh/clone Module, Up: Zsh Modules + +22.5 The zsh/compctl Module +=========================== + + + + The zsh/compctl module makes available two builtin commands. compctl, +is the old, deprecated way to control completions for ZLE. See *Note +Completion Using compctl::. The other builtin command, compcall can be +used in user-defined completion widgets, see *Note Completion Widgets::. + + +File: zsh.info, Node: The zsh/complete Module, Next: The zsh/complist Module, Prev: The zsh/compctl Module, Up: Zsh Modules + +22.6 The zsh/complete Module +============================ + + + + The zsh/complete module makes available several builtin commands +which can be used in user-defined completion widgets, see *Note +Completion Widgets::. + + +File: zsh.info, Node: The zsh/complist Module, Next: The zsh/computil Module, Prev: The zsh/complete Module, Up: Zsh Modules + +22.7 The zsh/complist Module +============================ + + + + The zsh/complist module offers three extensions to completion +listings: the ability to highlight matches in such a list, the ability +to scroll through long lists and a different style of menu completion. + + + +22.7.1 Colored completion listings +---------------------------------- + +Whenever one of the parameters ZLS_COLORS or ZLS_COLOURS is set and the +zsh/complist module is loaded or linked into the shell, completion +lists will be colored. Note, however, that complist will not +automatically be loaded if it is not linked in: on systems with +dynamic loading, `zmodload zsh/complist' is required. + +The parameters ZLS_COLORS and ZLS_COLOURS describe how matches are +highlighted. To turn on highlighting an empty value suffices, in which +case all the default values given below will be used. The format of +the value of these parameters is the same as used by the GNU version of +the ls command: a colon-separated list of specifications of the form +`NAME=VALUE'. The NAME may be one of the following strings, most of +which specify file types for which the VALUE will be used. The strings +and their default values are: + + +no 0 + for normal text (i.e. when displaying something other than a + matched file) + +fi 0 + for regular files + +di 32 + for directories + +ln 36 + for symbolic links. If this has the special value target, + symbolic links are dereferenced and the target file used to + determine the display format. + +pi 31 + for named pipes (FIFOs) + +so 33 + for sockets + +bd 44;37 + for block devices + +cd 44;37 + for character devices + +or NONE + for a symlink to nonexistent file (default is the value defined + for ln) + +mi NONE + for a non-existent file (default is the value defined for fi); + this code is currently not used + +su 37;41 + for files with setuid bit set + +sg 30;43 + for files with setgid bit set + +tw 30;42 + for world writable directories with sticky bit set + +ow 34;43 + for world writable directories without sticky bit set + +sa NONE + for files with an associated suffix alias; this is only tested + after specific suffixes, as described below + +st 37;44 + for directories with sticky bit set but not world writable + +ex 35 + for executable files + +lc \e[ + for the left code (see below) + +rc m + for the right code + +tc 0 + for the character indicating the file type printed after + filenames if the LIST_TYPES option is set + +sp 0 + for the spaces printed after matches to align the next column + +ec NONE + for the end code + + +Apart from these strings, the NAME may also be an asterisk (`*') +followed by any string. The VALUE given for such a string will be used +for all files whose name ends with the string. The NAME may also be an +equals sign (`=') followed by a pattern; the EXTENDED_GLOB option will +be turned on for evaluation of the pattern. The VALUE given for this +pattern will be used for all matches (not just filenames) whose display +string are matched by the pattern. Definitions for the form with the +leading equal sign take precedence over the values defined for file +types, which in turn take precedence over the form with the leading +asterisk (file extensions). + +The leading-equals form also allows different parts of the displayed +strings to be colored differently. For this, the pattern has to use the +`(#b)' globbing flag and pairs of parentheses surrounding the parts of +the strings that are to be colored differently. In this case the VALUE +may consist of more than one color code separated by equal signs. The +first code will be used for all parts for which no explicit code is +specified and the following codes will be used for the parts matched by +the sub-patterns in parentheses. For example, the specification +`=(#b)(?)*(?)=0=3=7' will be used for all matches which are at least +two characters long and will use the code `3' for the first character, +`7' for the last character and `0' for the rest. + +All three forms of NAME may be preceded by a pattern in parentheses. +If this is given, the VALUE will be used only for matches in groups +whose names are matched by the pattern given in the parentheses. For +example, `(g*)m*=43' highlights all matches beginning with `m' in +groups whose names begin with `g' using the color code `43'. In case +of the `lc', `rc', and `ec' codes, the group pattern is ignored. + +Note also that all patterns are tried in the order in which they appear +in the parameter value until the first one matches which is then used. +Patterns may be matched against completions, descriptions (possibly +with spaces appended for padding), or lines consisting of a completion +followed by a description. For consistent coloring it may be necessary +to use more than one pattern or a pattern with backreferences. + +When printing a match, the code prints the value of lc, the value for +the file-type or the last matching specification with a `*', the value +of rc, the string to display for the match itself, and then the value +of ec if that is defined or the values of lc, no, and rc if ec is not +defined. + +The default values are ISO 6429 (ANSI) compliant and can be used on +vt100 compatible terminals such as xterms. On monochrome terminals the +default values will have no visible effect. The colors function from +the contribution can be used to get associative arrays containing the +codes for ANSI terminals (see *Note Other Functions::). For example, +after loading colors, one could use `$color[red]' to get the code for +foreground color red and `$color[bg-green]' for the code for background +color green. + +If the completion system invoked by compinit is used, these parameters +should not be set directly because the system controls them itself. +Instead, the list-colors style should be used (see *Note Completion +System Configuration::). + + + +22.7.2 Scrolling in completion listings +--------------------------------------- + +To enable scrolling through a completion list, the LISTPROMPT parameter +must be set. Its value will be used as the prompt; if it is the empty +string, a default prompt will be used. The value may contain escapes +of the form `%x'. It supports the escapes `%B', `%b', `%S', `%s', +`%U', `%u', `%F', `%f', `%K', `%k' and `%{...%}' used also in shell +prompts as well as three pairs of additional sequences: a `%l' or `%L' +is replaced by the number of the last line shown and the total number +of lines in the form `NUMBER/TOTAL'; a `%m' or `%M' is replaced with +the number of the last match shown and the total number of matches; and +`%p' or `%P' is replaced with `Top', `Bottom' or the position of the +first line shown in percent of the total number of lines, respectively. +In each of these cases the form with the uppercase letter will be +replaced with a string of fixed width, padded to the right with spaces, +while the lowercase form will not be padded. + +If the parameter LISTPROMPT is set, the completion code will not ask if +the list should be shown. Instead it immediately starts displaying the +list, stopping after the first screenful, showing the prompt at the +bottom, waiting for a keypress after temporarily switching to the +listscroll keymap. Some of the zle functions have a special meaning +while scrolling lists: + + +send-break + stops listing discarding the key pressed + +accept-line, down-history, down-line-or-history +down-line-or-search, vi-down-line-or-history + scrolls forward one line + +complete-word, menu-complete, expand-or-complete +expand-or-complete-prefix, menu-complete-or-expand + scrolls forward one screenful + +accept-search + stop listing but take no other action + + +Every other character stops listing and immediately processes the key +as usual. Any key that is not bound in the listscroll keymap or that +is bound to undefined-key is looked up in the keymap currently selected. + +As for the ZLS_COLORS and ZLS_COLOURS parameters, LISTPROMPT should not +be set directly when using the shell function based completion system. +Instead, the list-prompt style should be used. + + + +22.7.3 Menu selection +--------------------- + +The zsh/complist module also offers an alternative style of selecting +matches from a list, called menu selection, which can be used if the +shell is set up to return to the last prompt after showing a completion +list (see the ALWAYS_LAST_PROMPT option in *Note Options::). + +Menu selection can be invoked directly by the widget menu-select +defined by this module. This is a standard ZLE widget that can be +bound to a key in the usual way as described in *Note Zsh Line Editor::. + +Alternatively, the parameter MENUSELECT can be set to an integer, which +gives the minimum number of matches that must be present before menu +selection is automatically turned on. This second method requires that +menu completion be started, either directly from a widget such as +menu-complete, or due to one of the options MENU_COMPLETE or AUTO_MENU +being set. If MENUSELECT is set, but is 0, 1 or empty, menu selection +will always be started during an ambiguous menu completion. + +When using the completion system based on shell functions, the +MENUSELECT parameter should not be used (like the ZLS_COLORS and +ZLS_COLOURS parameters described above). Instead, the menu style +should be used with the select=... keyword. + +After menu selection is started, the matches will be listed. If there +are more matches than fit on the screen, only the first screenful is +shown. The matches to insert into the command line can be selected +from this list. In the list one match is highlighted using the value +for ma from the ZLS_COLORS or ZLS_COLOURS parameter. The default value +for this is `7' which forces the selected match to be highlighted using +standout mode on a vt100-compatible terminal. If neither ZLS_COLORS +nor ZLS_COLOURS is set, the same terminal control sequence as for the +`%S' escape in prompts is used. + +If there are more matches than fit on the screen and the parameter +MENUPROMPT is set, its value will be shown below the matches. It +supports the same escape sequences as LISTPROMPT, but the number of the +match or line shown will be that of the one where the mark is placed. +If its value is the empty string, a default prompt will be used. + +The MENUSCROLL parameter can be used to specify how the list is +scrolled. If the parameter is unset, this is done line by line, if it +is set to `0' (zero), the list will scroll half the number of lines of +the screen. If the value is positive, it gives the number of lines to +scroll and if it is negative, the list will be scrolled the number of +lines of the screen minus the (absolute) value. + +As for the ZLS_COLORS, ZLS_COLOURS and LISTPROMPT parameters, neither +MENUPROMPT nor MENUSCROLL should be set directly when using the shell +function based completion system. Instead, the select-prompt and +select-scroll styles should be used. + +The completion code sometimes decides not to show all of the matches in +the list. These hidden matches are either matches for which the +completion function which added them explicitly requested that they not +appear in the list (using the -n option of the compadd builtin command) +or they are matches which duplicate a string already in the list +(because they differ only in things like prefixes or suffixes that are +not displayed). In the list used for menu selection, however, even +these matches are shown so that it is possible to select them. To +highlight such matches the hi and du capabilities in the ZLS_COLORS and +ZLS_COLOURS parameters are supported for hidden matches of the first +and second kind, respectively. + +Selecting matches is done by moving the mark around using the zle +movement functions. When not all matches can be shown on the screen at +the same time, the list will scroll up and down when crossing the top or +bottom line. The following zle functions have special meaning during +menu selection. Note that the following always perform the same task +within the menu selection map and cannot be replaced by user defined +widgets, nor can the set of functions be extended: + + +accept-line, accept-search + accept the current match and leave menu selection (but do not + cause the command line to be accepted) + +send-break + leaves menu selection and restores the previous contents of the + command line + +redisplay, clear-screen + execute their normal function without leaving menu selection + +accept-and-hold, accept-and-menu-complete + accept the currently inserted match and continue selection + allowing to select the next match to insert into the line + +accept-and-infer-next-history + accepts the current match and then tries completion with menu + selection again; in the case of files this allows one to select a + directory and immediately attempt to complete files in it; if + there are no matches, a message is shown and one can use undo to + go back to completion on the previous level, every other key + leaves menu selection (including the other zle functions which are + otherwise special during menu selection) + +undo + removes matches inserted during the menu selection by one of the + three functions before + +down-history, down-line-or-history +vi-down-line-or-history, down-line-or-search + moves the mark one line down + +up-history, up-line-or-history +vi-up-line-or-history, up-line-or-search + moves the mark one line up + +forward-char, vi-forward-char + moves the mark one column right + +backward-char, vi-backward-char + moves the mark one column left + +forward-word, vi-forward-word +vi-forward-word-end, emacs-forward-word + moves the mark one screenful down + +backward-word, vi-backward-word, emacs-backward-word + moves the mark one screenful up + +vi-forward-blank-word, vi-forward-blank-word-end + moves the mark to the first line of the next group of matches + +vi-backward-blank-word + moves the mark to the last line of the previous group of matches + +beginning-of-history + moves the mark to the first line + +end-of-history + moves the mark to the last line + +beginning-of-buffer-or-history, beginning-of-line +beginning-of-line-hist, vi-beginning-of-line + moves the mark to the leftmost column + +end-of-buffer-or-history, end-of-line +end-of-line-hist, vi-end-of-line + moves the mark to the rightmost column + +complete-word, menu-complete, expand-or-complete +expand-or-complete-prefix, menu-expand-or-complete + moves the mark to the next match + +reverse-menu-complete + moves the mark to the previous match + +vi-insert + this toggles between normal and interactive mode; in interactive + mode the keys bound to self-insert and self-insert-unmeta insert + into the command line as in normal editing mode but without leaving + menu selection; after each character completion is tried again and + the list changes to contain only the new matches; the completion + widgets make the longest unambiguous string be inserted in the + command line and undo and backward-delete-char go back to the + previous set of matches + +history-incremental-search-forward +history-incremental-search-backward + this starts incremental searches in the list of completions + displayed; in this mode, accept-line only leaves incremental + search, going back to the normal menu selection mode + + +All movement functions wrap around at the edges; any other zle function +not listed leaves menu selection and executes that function. It is +possible to make widgets in the above list do the same by using the +form of the widget with a `.' in front. For example, the widget +`.accept-line' has the effect of leaving menu selection and accepting +the entire command line. + +During this selection the widget uses the keymap menuselect. Any key +that is not defined in this keymap or that is bound to undefined-key is +looked up in the keymap currently selected. This is used to ensure +that the most important keys used during selection (namely the cursor +keys, return, and TAB) have sensible defaults. However, keys in the +menuselect keymap can be modified directly using the bindkey builtin +command (see *Note The zsh/zle Module::). For example, to make the +return key leave menu selection without accepting the match currently +selected one could call + + + bindkey -M menuselect '^M' send-break + +after loading the zsh/complist module. + + +File: zsh.info, Node: The zsh/computil Module, Next: The zsh/curses Module, Prev: The zsh/complist Module, Up: Zsh Modules + +22.8 The zsh/computil Module +============================ + + + + The zsh/computil module adds several builtin commands that are used +by some of the completion functions in the completion system based on +shell functions (see *Note Completion System:: ). Except for compquote +these builtin commands are very specialised and thus not very +interesting when writing your own completion functions. In summary, +these builtin commands are: + + +comparguments + This is used by the _arguments function to do the argument and + command line parsing. Like compdescribe it has an option -i to do + the parsing and initialize some internal state and various options + to access the state information to decide what should be completed. + +compdescribe + This is used by the _describe function to build the displays for + the matches and to get the strings to add as matches with their + options. On the first call one of the options -i or -I should be + supplied as the first argument. In the first case, display + strings without the descriptions will be generated, in the second + case, the string used to separate the matches from their + descriptions must be given as the second argument and the + descriptions (if any) will be shown. All other arguments are like + the definition arguments to _describe itself. + + Once compdescribe has been called with either the -i or the -I + option, it can be repeatedly called with the -g option and the + names of four parameters as its arguments. This will step through + the different sets of matches and store the value of + compstate[list] in the first scalar, the options for compadd in + the second array, the matches in the third array, and the strings + to be displayed in the completion listing in the fourth array. + The arrays may then be directly given to compadd to register the + matches with the completion code. + +compfiles + Used by the _path_files function to optimize complex recursive + filename generation (globbing). It does three things. With the + -p and -P options it builds the glob patterns to use, including + the paths already handled and trying to optimize the patterns with + respect to the prefix and suffix from the line and the match + specification currently used. The -i option does the directory + tests for the ignore-parents style and the -r option tests if a + component for some of the matches are equal to the string on the + line and removes all other matches if that is true. + +compgroups + Used by the _tags function to implement the internals of the + group-order style. This only takes its arguments as names of + completion groups and creates the groups for it (all six types: + sorted and unsorted, both without removing duplicates, with + removing all duplicates and with removing consecutive duplicates). + +compquote [ -p ] NAMES ... + There may be reasons to write completion functions that have to add + the matches using the -Q option to compadd and perform quoting + themselves. Instead of interpreting the first character of the + all_quotes key of the compstate special association and using the + q flag for parameter expansions, one can use this builtin command. + The arguments are the names of scalar or array parameters and the + values of these parameters are quoted as needed for the innermost + quoting level. If the -p option is given, quoting is done as if + there is some prefix before the values of the parameters, so that + a leading equal sign will not be quoted. + + The return status is non-zero in case of an error and zero + otherwise. + +comptags +comptry + These implement the internals of the tags mechanism. + +compvalues + Like comparguments, but for the _values function. + + + +File: zsh.info, Node: The zsh/curses Module, Next: The zsh/datetime Module, Prev: The zsh/computil Module, Up: Zsh Modules + +22.9 The zsh/curses Module +========================== + + + + The zsh/curses module makes available one builtin command and +various parameters. + + + +22.9.1 Builtin +-------------- + + +zcurses init +zcurses end +zcurses addwin TARGETWIN NLINES NCOLS BEGIN_Y BEGIN_X [ PARENTWIN ] +zcurses delwin TARGETWIN +zcurses refresh [ TARGETWIN ... ] +zcurses touch TARGETWIN ... +zcurses move TARGETWIN NEW_Y NEW_X +zcurses clear TARGETWIN [ redraw | eol | bot ] +zcurses position TARGETWIN ARRAY +zcurses char TARGETWIN CHARACTER +zcurses string TARGETWIN STRING +zcurses border TARGETWIN BORDER +zcurses attr TARGETWIN [ [+|-]ATTRIBUTE | FG_COL/BG_COL ] [...] +zcurses bg TARGETWIN [ [+|-]ATTRIBUTE | FG_COL/BG_COL | @CHAR ] [...] +zcurses scroll TARGETWIN [ on | off | [+|-]LINES ] +zcurses input TARGETWIN [ PARAM [ KPARAM [ MPARAM ] ] ] +zcurses mouse [ delay NUM | [+|-]motion ] +zcurses timeout TARGETWIN INTVAL +zcurses querychar TARGETWIN [ PARAM ] +zcurses resize HEIGHT WIDTH [ endwin | nosave | endwin_nosave ] + Manipulate curses windows. All uses of this command should be + bracketed by `zcurses init' to initialise use of curses, and + `zcurses end' to end it; omitting `zcurses end' can cause the + terminal to be in an unwanted state. + + The subcommand addwin creates a window with NLINES lines and NCOLS + columns. Its upper left corner will be placed at row BEGIN_Y and + column BEGIN_X of the screen. TARGETWIN is a string and refers to + the name of a window that is not currently assigned. Note in + particular the curses convention that vertical values appear + before horizontal values. + + If addwin is given an existing window as the final argument, the + new window is created as a subwindow of PARENTWIN. This differs + from an ordinary new window in that the memory of the window + contents is shared with the parent's memory. Subwindows must be + deleted before their parent. Note that the coordinates of + subwindows are relative to the screen, not the parent, as with + other windows. + + Use the subcommand delwin to delete a window created with addwin. + Note that end does _not_ implicitly delete windows, and that + delwin does not erase the screen image of the window. + + The window corresponding to the full visible screen is called + stdscr; it always exists after `zcurses init' and cannot be delete + with delwin. + + The subcommand refresh will refresh window TARGETWIN; this is + necessary to make any pending changes (such as characters you have + prepared for output with char) visible on the screen. refresh + without an argument causes the screen to be cleared and redrawn. + If multiple windows are given, the screen is updated once at the + end. + + The subcommand touch marks the TARGETWINs listed as changed. This + is necessary before refreshing windows if a window that was in + front of another window (which may be stdscr) is deleted. + + The subcommand move moves the cursor position in TARGETWIN to new + coordinates NEW_Y and NEW_X. Note that the subcommand string (but + not the subcommand char) advances the cursor position over the + characters added. + + The subcommand clear erases the contents of TARGETWIN. One (and + no more than one) of three options may be specified. With the + option redraw, in addition the next refresh of TARGETWIN will + cause the screen to be cleared and repainted. With the option + eol, TARGETWIN is only cleared to the end of the current cursor + line. With the option bot, TARGETWIN is cleared to the end of the + window, i.e everything to the right and below the cursor is + cleared. + + The subcommand position writes various positions associated with + TARGETWIN into the array named ARRAY. These are, in order: + - + The y and x coordinates of the cursor relative to the top left + of TARGETWIN + + - + The y and x coordinates of the top left of TARGETWIN on the + screen + + - + The size of TARGETWIN in y and x dimensions. + + Outputting characters and strings are achieved by char and string + respectively. + + To draw a border around window TARGETWIN, use border. Note that + the border is not subsequently handled specially: in other words, + the border is simply a set of characters output at the edge of the + window. Hence it can be overwritten, can scroll off the window, + etc. + + The subcommand attr will set TARGETWIN's attributes or + foreground/background color pair for any successive character + output. Each ATTRIBUTE given on the line may be prepended by a + + to set or a - to unset that attribute; + is assumed if absent. The + attributes supported are blink, bold, dim, reverse, standout, and + underline. + + Each FG_COL/BG_COL attribute (to be read as `FG_COL on BG_COL') + sets the foreground and background color for character output. + The color default is sometimes available (in particular if the + library is ncurses), specifying the foreground or background color + with which the terminal started. The color pair default/default + is always available. To use more than the 8 named colors (red, + green, etc.) construct the FG_COL/BG_COL pairs where FG_COL and + BG_COL are decimal integers, e.g 128/200. The maximum color value + is 254 if the terminal supports 256 colors. + + bg overrides the color and other attributes of all characters in + the window. Its usual use is to set the background initially, but + it will overwrite the attributes of any characters at the time + when it is called. In addition to the arguments allowed with + attr, an argument @CHAR specifies a character to be shown in + otherwise blank areas of the window. Owing to limitations of + curses this cannot be a multibyte character (use of ASCII + characters only is recommended). As the specified set of + attributes override the existing background, turning attributes + off in the arguments is not useful, though this does not cause an + error. + + The subcommand scroll can be used with on or off to enabled or + disable scrolling of a window when the cursor would otherwise move + below the window due to typing or output. It can also be used + with a positive or negative integer to scroll the window up or + down the given number of lines without changing the current cursor + position (which therefore appears to move in the opposite + direction relative to the window). In the second case, if + scrolling is off it is temporarily turned on to allow the window + to be scrolled. + + The subcommand input reads a single character from the window + without echoing it back. If PARAM is supplied the character is + assigned to the parameter PARAM, else it is assigned to the + parameter REPLY. + + If both PARAM and KPARAM are supplied, the key is read in `keypad' + mode. In this mode special keys such as function keys and arrow + keys return the name of the key in the parameter KPARAM. The key + names are the macros defined in the curses.h or ncurses.h with the + prefix `KEY_' removed; see also the description of the parameter + zcurses_keycodes below. Other keys cause a value to be set in + PARAM as before. On a successful return only one of PARAM or + KPARAM contains a non-empty string; the other is set to an empty + string. + + If MPARAM is also supplied, input attempts to handle mouse input. + This is only available with the ncurses library; mouse handling + can be detected by checking for the exit status of `zcurses mouse' + with no arguments. If a mouse button is clicked (or double- or + triple-clicked, or pressed or released with a configurable delay + from being clicked) then kparam is set to the string MOUSE, and + MPARAM is set to an array consisting of the following elements: + - + An identifier to discriminate different input devices; this + is only rarely useful. + + - + The x, y and z coordinates of the mouse click relative to the + full screen, as three elements in that order (i.e. the y + coordinate is, unusually, after the x coordinate). The z + coordinate is only available for a few unusual input devices + and is otherwise set to zero. + + - + Any events that occurred as separate items; usually there + will be just one. An event consists of PRESSED, RELEASED, + CLICKED, DOUBLE_CLICKED or TRIPLE_CLICKED followed + immediately (in the same element) by the number of the button. + + - + If the shift key was pressed, the string SHIFT. + + - + If the control key was pressed, the string CTRL. + + - + If the alt key was pressed, the string ALT. + + Not all mouse events may be passed through to the terminal window; + most terminal emulators handle some mouse events themselves. Note + that the ncurses manual implies that using input both with and + without mouse handling may cause the mouse cursor to appear and + disappear. + + The subcommand mouse can be used to configure the use of the mouse. + There is no window argument; mouse options are global. `zcurses + mouse' with no arguments returns status 0 if mouse handling is + possible, else status 1. Otherwise, the possible arguments (which + may be combined on the same command line) are as follows. delay + NUM sets the maximum delay in milliseconds between press and + release events to be considered as a click; the value 0 disables + click resolution, and the default is one sixth of a second. + motion proceeded by an optional `+' (the default) or - turns on or + off reporting of mouse motion in addition to clicks, presses and + releases, which are always reported. However, it appears reports + for mouse motion are not currently implemented. + + The subcommand timeout specifies a timeout value for input from + TARGETWIN. If INTVAL is negative, `zcurses input' waits + indefinitely for a character to be typed; this is the default. If + INTVAL is zero, `zcurses input' returns immediately; if there is + typeahead it is returned, else no input is done and status 1 is + returned. If INTVAL is positive, `zcurses input' waits INTVAL + milliseconds for input and if there is none at the end of that + period returns status 1. + + The subcommand querychar queries the character at the current + cursor position. The return values are stored in the array named + PARAM if supplied, else in the array reply. The first value is + the character (which may be a multibyte character if the system + supports them); the second is the color pair in the usual + FG_COL/BG_COL notation, or 0 if color is not supported. Any + attributes other than color that apply to the character, as set + with the subcommand attr, appear as additional elements. + + The subcommand resize resizes stdscr and all windows to given + dimensions (windows that stick out from the new dimensions are + resized down). The underlying curses extension (resize_term call) + can be unavailable. To verify, zeroes can be used for HEIGHT and + WIDTH. If the result of the subcommand is 0, resize_term is + available (2 otherwise). Tests show that resizing can be normally + accomplished by calling zcurses end and zcurses refresh. The + resize subcommand is provided for versatility. Multiple system + configurations have been checked and zcurses end and zcurses + refresh are still needed for correct terminal state after resize. + To invoke them with resize, use ENDWIN argument. Using NOSAVE + argument will cause new terminal state to not be saved internally + by zcurses. This is also provided for versatility and should + normally be not needed. + + + +22.9.2 Parameters +----------------- + + +ZCURSES_COLORS + Readonly integer. The maximum number of colors the terminal + supports. This value is initialised by the curses library and is + not available until the first time zcurses init is run. + +ZCURSES_COLOR_PAIRS + Readonly integer. The maximum number of color pairs FG_COL/BG_COL + that may be defined in `zcurses attr' commands; note this limit + applies to all color pairs that have been used whether or not they + are currently active. This value is initialised by the curses + library and is not available until the first time zcurses init is + run. + +zcurses_attrs + Readonly array. The attributes supported by zsh/curses; available + as soon as the module is loaded. + +zcurses_colors + Readonly array. The colors supported by zsh/curses; available as + soon as the module is loaded. + +zcurses_keycodes + Readonly array. The values that may be returned in the second + parameter supplied to `zcurses input' in the order in which they + are defined internally by curses. Not all function keys are + listed, only F0; curses reserves space for F0 up to F63. + +zcurses_windows + Readonly array. The current list of windows, i.e. all windows that + have been created with `zcurses addwin' and not removed with + `zcurses delwin'. + + + +File: zsh.info, Node: The zsh/datetime Module, Next: The zsh/db/gdbm Module, Prev: The zsh/curses Module, Up: Zsh Modules + +22.10 The zsh/datetime Module +============================= + + + + The zsh/datetime module makes available one builtin command: + + +strftime [ -s SCALAR ] FORMAT [ EPOCHTIME [ NANOSECONDS ] ] +strftime -r [ -q ] [ -s SCALAR ] FORMAT TIMESTRING + Output the date in the FORMAT specified. With no EPOCHTIME, the + current system date/time is used; optionally, EPOCHTIME may be + used to specify the number of seconds since the epoch, and + NANOSECONDS may additionally be used to specify the number of + nanoseconds past the second (otherwise that number is assumed to + be 0). See man page strftime(3) for details. The zsh extensions + described in *Note Prompt Expansion:: are also available. + + + -q + Run quietly; suppress printing of all error messages + described below. Errors for invalid EPOCHTIME values are + always printed. + + -r + With the option -r (reverse), use FORMAT to parse the input + string TIMESTRING and output the number of seconds since the + epoch at which the time occurred. The parsing is implemented + by the system function strptime; see man page strptime(3). + This means that zsh format extensions are not available, but + for reverse lookup they are not required. + + In most implementations of strftime any timezone in the + TIMESTRING is ignored and the local timezone declared by the + TZ environment variable is used; other parameters are set to + zero if not present. + + If TIMESTRING does not match FORMAT the command returns + status 1 and prints an error message. If TIMESTRING matches + FORMAT but not all characters in TIMESTRING were used, the + conversion succeeds but also prints an error message. + + If either of the system functions strptime or mktime is not + available, status 2 is returned and an error message is + printed. + + -s SCALAR + Assign the date string (or epoch time in seconds if -r is + given) to SCALAR instead of printing it. + + + Note that depending on the system's declared integral time type, + strftime may produce incorrect results for epoch times greater than + 2147483647 which corresponds to 2038-01-19 03:14:07 +0000. + + +The zsh/datetime module makes available several parameters; all are +readonly: + + +EPOCHREALTIME + A floating point value representing the number of seconds since + the epoch. The notional accuracy is to nanoseconds if the + clock_gettime call is available and to microseconds otherwise, but + in practice the range of double precision floating point and shell + scheduling latencies may be significant effects. + +EPOCHSECONDS + An integer value representing the number of seconds since the + epoch. + +epochtime + An array value containing the number of seconds since the epoch in + the first element and the remainder of the time since the epoch in + nanoseconds in the second element. To ensure the two elements are + consistent the array should be copied or otherwise referenced as a + single substitution before the values are used. The following + idiom may be used: + + + for secs nsecs in $epochtime; do + ... + done + + + + +File: zsh.info, Node: The zsh/db/gdbm Module, Next: The zsh/deltochar Module, Prev: The zsh/datetime Module, Up: Zsh Modules + +22.11 The zsh/db/gdbm Module +============================ + + + + The zsh/db/gdbm module is used to create "tied" associative arrays +that interface to database files. If the GDBM interface is not +available, the builtins defined by this module will report an error. +This module is also intended as a prototype for creating additional +database interfaces, so the ztie builtin may move to a more generic +module in the future. + +The builtins in this module are: + + +ztie -d db/gdbm -f FILENAME [ -r ] ARRAYNAME + Open the GDBM database identified by FILENAME and, if successful, + create the associative array ARRAYNAME linked to the file. To + create a local tied array, the parameter must first be declared, + so commands similar to the following would be executed inside a + function scope: + + + local -A sampledb + ztie -d db/gdbm -f sample.gdbm sampledb + + The -r option opens the database file for reading only, creating a + parameter with the readonly attribute. Without this option, using + `ztie' on a file for which the user does not have write permission + is an error. If writable, the database is opened synchronously so + fields changed in ARRAYNAME are immediately written to FILENAME. + + Changes to the file modes FILENAME after it has been opened do not + alter the state of ARRAYNAME, but `typeset -r ARRAYNAME' works as + expected. + +zuntie [ -u ] ARRAYNAME ... + Close the GDBM database associated with each ARRAYNAME and then + unset the parameter. The -u option forces an unset of parameters + made readonly with `ztie -r'. + + This happens automatically if the parameter is explicitly unset or + its local scope (function) ends. Note that a readonly parameter + may not be explicitly unset, so the only way to unset a global + parameter created with `ztie -r' is to use `zuntie -u'. + +zgdbmpath PARAMETERNAME + Put path to database file assigned to PARAMETERNAME into REPLY + scalar. + +zgdbm_tied + Array holding names of all tied parameters. + + +The fields of an associative array tied to GDBM are neither cached nor +otherwise stored in memory, they are read from or written to the +database on each reference. Thus, for example, the values in a +readonly array may be changed by a second writer of the same database +file. + + +File: zsh.info, Node: The zsh/deltochar Module, Next: The zsh/example Module, Prev: The zsh/db/gdbm Module, Up: Zsh Modules + +22.12 The zsh/deltochar Module +============================== + + + + The zsh/deltochar module makes available two ZLE functions: + + +delete-to-char + Read a character from the keyboard, and delete from the cursor + position up to and including the next (or, with repeat count N, + the Nth) instance of that character. Negative repeat counts mean + delete backwards. + +zap-to-char + This behaves like delete-to-char, except that the final occurrence + of the character itself is not deleted. + + + +File: zsh.info, Node: The zsh/example Module, Next: The zsh/files Module, Prev: The zsh/deltochar Module, Up: Zsh Modules + +22.13 The zsh/example Module +============================ + + + + The zsh/example module makes available one builtin command: + + +example [ -flags ] [ ARGS ... ] + Displays the flags and arguments it is invoked with. + + +The purpose of the module is to serve as an example of how to write a +module. + + +File: zsh.info, Node: The zsh/files Module, Next: The zsh/langinfo Module, Prev: The zsh/example Module, Up: Zsh Modules + +22.14 The zsh/files Module +========================== + + + + The zsh/files module makes available some common commands for file +manipulation as builtins; these commands are probably not needed for +many normal situations but can be useful in emergency recovery +situations with constrained resources. The commands do not implement +all features now required by relevant standards committees. + +For all commands, a variant beginning zf_ is also available and loaded +automatically. Using the features capability of zmodload will let you +load only those names you want. Note that it's possible to load only +the builtins with zsh-specific names using the following command: + + + zmodload -m -F zsh/files b:zf_\* + +The commands loaded by default are: + + +chgrp [ -hRs ] GROUP FILENAME ... + Changes group of files specified. This is equivalent to chown with + a USER-SPEC argument of `:GROUP'. + +chmod [ -Rs ] MODE FILENAME ... + Changes mode of files specified. + + The specified MODE must be in octal. + + The -R option causes chmod to recursively descend into directories, + changing the mode of all files in the directory after changing the + mode of the directory itself. + + The -s option is a zsh extension to chmod functionality. It + enables paranoid behaviour, intended to avoid security problems + involving a chmod being tricked into affecting files other than + the ones intended. It will refuse to follow symbolic links, so + that (for example) ``chmod 600 /tmp/foo/passwd'' can't + accidentally chmod /etc/passwd if /tmp/foo happens to be a link to + /etc. It will also check where it is after leaving directories, + so that a recursive chmod of a deep directory tree can't end up + recursively chmoding /usr as a result of directories being moved + up the tree. + +chown [ -hRs ] USER-SPEC FILENAME ... + Changes ownership and group of files specified. + + The USER-SPEC can be in four forms: + + + USER + change owner to USER; do not change group + + USER:: + change owner to USER; do not change group + + USER: + change owner to USER; change group to USER's primary group + + USER:GROUP + change owner to USER; change group to GROUP + + :GROUP + do not change owner; change group to GROUP + + In each case, the `:' may instead be a `.'. The rule is that if + there is a `:' then the separator is `:', otherwise if there is a + `.' then the separator is `.', otherwise there is no separator. + + Each of USER and GROUP may be either a username (or group name, as + appropriate) or a decimal user ID (group ID). Interpretation as a + name takes precedence, if there is an all-numeric username (or + group name). + + If the target is a symbolic link, the -h option causes chown to set + the ownership of the link instead of its target. + + The -R option causes chown to recursively descend into directories, + changing the ownership of all files in the directory after + changing the ownership of the directory itself. + + The -s option is a zsh extension to chown functionality. It + enables paranoid behaviour, intended to avoid security problems + involving a chown being tricked into affecting files other than + the ones intended. It will refuse to follow symbolic links, so + that (for example) ``chown luser /tmp/foo/passwd'' can't + accidentally chown /etc/passwd if /tmp/foo happens to be a link to + /etc. It will also check where it is after leaving directories, + so that a recursive chown of a deep directory tree can't end up + recursively chowning /usr as a result of directories being moved + up the tree. + +ln [ -dfhins ] FILENAME DEST +ln [ -dfhins ] FILENAME ... DIR + Creates hard (or, with -s, symbolic) links. In the first form, the + specified DESTination is created, as a link to the specified + FILENAME. In the second form, each of the FILENAMEs is taken in + turn, and linked to a pathname in the specified DIRectory that has + the same last pathname component. + + Normally, ln will not attempt to create hard links to directories. + This check can be overridden using the -d option. Typically only + the super-user can actually succeed in creating hard links to + directories. This does not apply to symbolic links in any case. + + By default, existing files cannot be replaced by links. The -i + option causes the user to be queried about replacing existing + files. The -f option causes existing files to be silently + deleted, without querying. -f takes precedence. + + The -h and -n options are identical and both exist for + compatibility; either one indicates that if the target is a symlink + then it should not be dereferenced. Typically this is used in + combination with -sf so that if an existing link points to a + directory then it will be removed, instead of followed. If this + option is used with multiple filenames and the target is a + symbolic link pointing to a directory then the result is an error. + +mkdir [ -p ] [ -m MODE ] DIR ... + Creates directories. With the -p option, non-existing parent + directories are first created if necessary, and there will be no + complaint if the directory already exists. The -m option can be + used to specify (in octal) a set of file permissions for the + created directories, otherwise mode 777 modified by the current + umask (see man page umask(2)) is used. + +mv [ -fi ] FILENAME DEST +mv [ -fi ] FILENAME ... DIR + Moves files. In the first form, the specified FILENAME is moved + to the specified DESTination. In the second form, each of the + FILENAMEs is taken in turn, and moved to a pathname in the + specified DIRectory that has the same last pathname component. + + By default, the user will be queried before replacing any file + that the user cannot write to, but writable files will be silently + removed. The -i option causes the user to be queried about + replacing any existing files. The -f option causes any existing + files to be silently deleted, without querying. -f takes + precedence. + + Note that this mv will not move files across devices. Historical + versions of mv, when actual renaming is impossible, fall back on + copying and removing files; if this behaviour is desired, use cp + and rm manually. This may change in a future version. + +rm [ -dfiRrs ] FILENAME ... + Removes files and directories specified. + + Normally, rm will not remove directories (except with the -R or -r + options). The -d option causes rm to try removing directories + with unlink (see man page unlink(2)), the same method used for + files. Typically only the super-user can actually succeed in + unlinking directories in this way. -d takes precedence over -R + and -r. + + By default, the user will be queried before removing any file that + the user cannot write to, but writable files will be silently + removed. The -i option causes the user to be queried about + removing any files. The -f option causes files to be silently + deleted, without querying, and suppresses all error indications. + -f takes precedence. + + The -R and -r options cause rm to recursively descend into + directories, deleting all files in the directory before removing + the directory with the rmdir system call (see man page rmdir(2)). + + The -s option is a zsh extension to rm functionality. It enables + paranoid behaviour, intended to avoid common security problems + involving a root-run rm being tricked into removing files other + than the ones intended. It will refuse to follow symbolic links, + so that (for example) ``rm /tmp/foo/passwd'' can't accidentally + remove /etc/passwd if /tmp/foo happens to be a link to /etc. It + will also check where it is after leaving directories, so that a + recursive removal of a deep directory tree can't end up + recursively removing /usr as a result of directories being moved + up the tree. + +rmdir DIR ... + Removes empty directories specified. + +sync + Calls the system call of the same name (see man page sync(2)), + which flushes dirty buffers to disk. It might return before the + I/O has actually been completed. + + + +File: zsh.info, Node: The zsh/langinfo Module, Next: The zsh/mapfile Module, Prev: The zsh/files Module, Up: Zsh Modules + +22.15 The zsh/langinfo Module +============================= + + + + The zsh/langinfo module makes available one parameter: + + +langinfo + An associative array that maps langinfo elements to their values. + + Your implementation may support a number of the following keys: + + CODESET, D_T_FMT, D_FMT, T_FMT, RADIXCHAR, THOUSEP, YESEXPR, + NOEXPR, CRNCYSTR, ABDAY_{1..7}, DAY_{1..7}, ABMON_{1..12}, + MON_{1..12}, T_FMT_AMPM, AM_STR, PM_STR, ERA, ERA_D_FMT, + ERA_D_T_FMT, ERA_T_FMT, ALT_DIGITS + + + + +File: zsh.info, Node: The zsh/mapfile Module, Next: The zsh/mathfunc Module, Prev: The zsh/langinfo Module, Up: Zsh Modules + +22.16 The zsh/mapfile Module +============================ + + + + The zsh/mapfile module provides one special associative array +parameter of the same name. + + +mapfile + This associative array takes as keys the names of files; the + resulting value is the content of the file. The value is treated + identically to any other text coming from a parameter. The value + may also be assigned to, in which case the file in question is + written (whether or not it originally existed); or an element may + be unset, which will delete the file in question. For example, + `vared mapfile[myfile]' works as expected, editing the file + `myfile'. + + When the array is accessed as a whole, the keys are the names of + files in the current directory, and the values are empty (to save + a huge overhead in memory). Thus ${(k)mapfile} has the same + effect as the glob operator *(D), since files beginning with a dot + are not special. Care must be taken with expressions such as rm + ${(k)mapfile}, which will delete every file in the current + directory without the usual `rm *' test. + + The parameter mapfile may be made read-only; in that case, files + referenced may not be written or deleted. + + A file may conveniently be read into an array as one line per + element with the form `ARRAY=("${(f@)mapfile[FILENAME]}")'. The + double quotes and the `@' are necessary to prevent empty lines + from being removed. Note that if the file ends with a newline, + the shell will split on the final newline, generating an additional + empty field; this can be suppressed by using + `ARRAY=("${(f@)${mapfile[FILENAME]%$'\n'}}")'. + + + +22.16.1 Limitations +------------------- + +Although reading and writing of the file in question is efficiently +handled, zsh's internal memory management may be arbitrarily baroque; +however, mapfile is usually very much more efficient than anything +involving a loop. Note in particular that the whole contents of the +file will always reside physically in memory when accessed (possibly +multiple times, due to standard parameter substitution operations). In +particular, this means handling of sufficiently long files (greater +than the machine's swap space, or than the range of the pointer type) +will be incorrect. + +No errors are printed or flagged for non-existent, unreadable, or +unwritable files, as the parameter mechanism is too low in the shell +execution hierarchy to make this convenient. + +It is unfortunate that the mechanism for loading modules does not yet +allow the user to specify the name of the shell parameter to be given +the special behaviour. + + +File: zsh.info, Node: The zsh/mathfunc Module, Next: The zsh/nearcolor Module, Prev: The zsh/mapfile Module, Up: Zsh Modules + +22.17 The zsh/mathfunc Module +============================= + + + + The zsh/mathfunc module provides standard mathematical functions for +use when evaluating mathematical formulae. The syntax agrees with +normal C and FORTRAN conventions, for example, + + + (( f = sin(0.3) )) + +assigns the sine of 0.3 to the parameter f. + +Most functions take floating point arguments and return a floating point +value. However, any necessary conversions from or to integer type will +be performed automatically by the shell. Apart from atan with a second +argument and the abs, int and float functions, all functions behave as +noted in the manual page for the corresponding C function, except that +any arguments out of range for the function in question will be +detected by the shell and an error reported. + +The following functions take a single floating point argument: acos, +acosh, asin, asinh, atan, atanh, cbrt, ceil, cos, cosh, erf, erfc, exp, +expm1, fabs, floor, gamma, j0, j1, lgamma, log, log10, log1p, log2, +logb, sin, sinh, sqrt, tan, tanh, y0, y1. The atan function can +optionally take a second argument, in which case it behaves like the C +function atan2. The ilogb function takes a single floating point +argument, but returns an integer. + +The function signgam takes no arguments, and returns an integer, which +is the C variable of the same name, as described in man page gamma(3). +Note that it is therefore only useful immediately after a call to gamma +or lgamma. Note also that `signgam()' and `signgam' are distinct +expressions. + +The functions min, max, and sum are defined not in this module but in +the zmathfunc autoloadable function, described in *Note Mathematical +Functions::. + +The following functions take two floating point arguments: copysign, +fmod, hypot, nextafter. + +The following take an integer first argument and a floating point second +argument: jn, yn. + +The following take a floating point first argument and an integer second +argument: ldexp, scalb. + +The function abs does not convert the type of its single argument; it +returns the absolute value of either a floating point number or an +integer. The functions float and int convert their arguments into a +floating point or integer value (by truncation) respectively. + +Note that the C pow function is available in ordinary math evaluation +as the `**' operator and is not provided here. + +The function rand48 is available if your system's mathematical library +has the function erand48(3). It returns a pseudo-random floating point +number between 0 and 1. It takes a single string optional argument. + +If the argument is not present, the random number seed is initialised by +three calls to the rand(3) function -- this produces the same random +numbers as the next three values of $RANDOM. + +If the argument is present, it gives the name of a scalar parameter +where the current random number seed will be stored. On the first +call, the value must contain at least twelve hexadecimal digits (the +remainder of the string is ignored), or the seed will be initialised in +the same manner as for a call to rand48 with no argument. Subsequent +calls to rand48(PARAM) will then maintain the seed in the parameter +PARAM as a string of twelve hexadecimal digits, with no base signifier. +The random number sequences for different parameters are completely +independent, and are also independent from that used by calls to rand48 +with no argument. + +For example, consider + + + print $(( rand48(seed) )) + print $(( rand48() )) + print $(( rand48(seed) )) + +Assuming $seed does not exist, it will be initialised by the first +call. In the second call, the default seed is initialised; note, +however, that because of the properties of rand() there is a +correlation between the seeds used for the two initialisations, so for +more secure uses, you should generate your own 12-byte seed. The third +call returns to the same sequence of random numbers used in the first +call, unaffected by the intervening rand48(). + + +File: zsh.info, Node: The zsh/nearcolor Module, Next: The zsh/newuser Module, Prev: The zsh/mathfunc Module, Up: Zsh Modules + +22.18 The zsh/nearcolor Module +============================== + + + + The zsh/nearcolor module replaces colours specified as hex triplets +with the nearest colour in the 88 or 256 colour palettes that are widely +used by terminal emulators. By default, 24-bit true colour escape codes +are generated when colours are specified using hex triplets. These are +not supported by all terminals. The purpose of this module is to make +it easier to define colour preferences in a form that can work across a +range of terminal emulators. + +Aside from the default colour, the ANSI standard for terminal escape +codes provides for eight colours. The bright attribute brings this to +sixteen. These basic colours are commonly used in terminal applications +due to being widely supported. Expanded 88 and 256 colour palettes are +also common and, while the first sixteen colours vary somewhat between +terminals and configurations, these add a generally consistent and +predictable set of colours. + +In order to use the zsh/nearcolor module, it only needs to be loaded. +Thereafter, whenever a colour is specified using a hex triplet, it will +be compared against each of the available colours and the closest will +be selected. The first sixteen colours are never matched in this +process due to being unpredictable. + +It isn't possible to reliably detect support for true colour in the +terminal emulator. It is therefore recommended to be selective in +loading the zsh/nearcolor module. For example, the following checks the +COLORTERM environment variable: + + + [[ $COLORTERM = *(24bit|truecolor)* ]] || zmodload zsh/nearcolor + +Note that some terminals accept the true color escape codes but map +them internally to a more limited palette in a similar manner to the +zsh/nearcolor module. + + +File: zsh.info, Node: The zsh/newuser Module, Next: The zsh/parameter Module, Prev: The zsh/nearcolor Module, Up: Zsh Modules + +22.19 The zsh/newuser Module +============================ + + + + The zsh/newuser module is loaded at boot if it is available, the RCS +option is set, and the PRIVILEGED option is not set (all three are true +by default). This takes place immediately after commands in the global +zshenv file (typically /etc/zshenv), if any, have been executed. If +the module is not available it is silently ignored by the shell; the +module may safely be removed from $MODULE_PATH by the administrator if +it is not required. + +On loading, the module tests if any of the start-up files .zshenv, +.zprofile, .zshrc or .zlogin exist in the directory given by the +environment variable ZDOTDIR, or the user's home directory if that is +not set. The test is not performed and the module halts processing if +the shell was in an emulation mode (i.e. had been invoked as some other +shell than zsh). + +If none of the start-up files were found, the module then looks for the +file newuser first in a sitewide directory, usually the parent +directory of the site-functions directory, and if that is not found the +module searches in a version-specific directory, usually the parent of +the functions directory containing version-specific functions. (These +directories can be configured when zsh is built using the +-enable-site-scriptdir=DIR and -enable-scriptdir=DIR flags to +configure, respectively; the defaults are PREFIX/share/zsh and +PREFIX/share/zsh/$ZSH_VERSION where the default PREFIX is /usr/local.) + +If the file newuser is found, it is then sourced in the same manner as +a start-up file. The file is expected to contain code to install +start-up files for the user, however any valid shell code will be +executed. + +The zsh/newuser module is then unconditionally unloaded. + +Note that it is possible to achieve exactly the same effect as the +zsh/newuser module by adding code to /etc/zshenv. The module exists +simply to allow the shell to make arrangements for new users without +the need for intervention by package maintainers and system +administrators. + +The script supplied with the module invokes the shell function +zsh-newuser-install. This may be invoked directly by the user even if +the zsh/newuser module is disabled. Note, however, that if the module +is not installed the function will not be installed either. The +function is documented in *Note User Configuration Functions::. + + +File: zsh.info, Node: The zsh/parameter Module, Next: The zsh/pcre Module, Prev: The zsh/newuser Module, Up: Zsh Modules + +22.20 The zsh/parameter Module +============================== + + + + The zsh/parameter module gives access to some of the internal hash +tables used by the shell by defining some special parameters. + + +options + The keys for this associative array are the names of the options + that can be set and unset using the setopt and unsetopt builtins. + The value of each key is either the string on if the option is + currently set, or the string off if the option is unset. Setting + a key to one of these strings is like setting or unsetting the + option, respectively. Unsetting a key in this array is like + setting it to the value off. + +commands + This array gives access to the command hash table. The keys are the + names of external commands, the values are the pathnames of the + files that would be executed when the command would be invoked. + Setting a key in this array defines a new entry in this table in + the same way as with the hash builtin. Unsetting a key as in `unset + "commands[foo]"' removes the entry for the given key from the + command hash table. + +functions + This associative array maps names of enabled functions to their + definitions. Setting a key in it is like defining a function with + the name given by the key and the body given by the value. + Unsetting a key removes the definition for the function named by + the key. + +dis_functions + Like functions but for disabled functions. + +functions_source + This readonly associative array maps names of enabled functions to + the name of the file containing the source of the function. + + For an autoloaded function that has already been loaded, or marked + for autoload with an absolute path, or that has had its path + resolved with `functions -r', this is the file found for + autoloading, resolved to an absolute path. + + For a function defined within the body of a script or sourced file, + this is the name of that file. In this case, this is the exact + path originally used to that file, which may be a relative path. + + For any other function, including any defined at an interactive + prompt or an autoload function whose path has not yet been + resolved, this is the empty string. However, the hash element is + reported as defined just so long as the function is present: the + keys to this hash are the same as those to $functions. + +dis_functions_source + Like functions_source but for disabled functions. + +builtins + This associative array gives information about the builtin commands + currently enabled. The keys are the names of the builtin commands + and the values are either `undefined' for builtin commands that + will automatically be loaded from a module if invoked or `defined' + for builtin commands that are already loaded. + +dis_builtins + Like builtins but for disabled builtin commands. + +reswords + This array contains the enabled reserved words. + +dis_reswords + Like reswords but for disabled reserved words. + +patchars + This array contains the enabled pattern characters. + +dis_patchars + Like patchars but for disabled pattern characters. + +aliases + This maps the names of the regular aliases currently enabled to + their expansions. + +dis_aliases + Like aliases but for disabled regular aliases. + +galiases + Like aliases, but for global aliases. + +dis_galiases + Like galiases but for disabled global aliases. + +saliases + Like raliases, but for suffix aliases. + +dis_saliases + Like saliases but for disabled suffix aliases. + +parameters + The keys in this associative array are the names of the parameters + currently defined. The values are strings describing the type of + the parameter, in the same format used by the t parameter flag, see + *Note Parameter Expansion:: . Setting or unsetting keys in this + array is not possible. + +modules + An associative array giving information about modules. The keys + are the names of the modules loaded, registered to be autoloaded, + or aliased. The value says which state the named module is in and + is one of the strings `loaded', `autoloaded', or `alias:NAME', + where NAME is the name the module is aliased to. + + Setting or unsetting keys in this array is not possible. + +dirstack + A normal array holding the elements of the directory stack. Note + that the output of the dirs builtin command includes one more + directory, the current working directory. + +history + This associative array maps history event numbers to the full + history lines. Although it is presented as an associative array, + the array of all values (${history[@]}) is guaranteed to be + returned in order from most recent to oldest history event, that + is, by decreasing history event number. + +historywords + A special array containing the words stored in the history. These + also appear in most to least recent order. + +jobdirs + This associative array maps job numbers to the directories from + which the job was started (which may not be the current directory + of the job). + + The keys of the associative arrays are usually valid job numbers, + and these are the values output with, for example, ${(k)jobdirs}. + Non-numeric job references may be used when looking up a value; + for example, ${jobdirs[%+]} refers to the current job. + +jobtexts + This associative array maps job numbers to the texts of the + command lines that were used to start the jobs. + + Handling of the keys of the associative array is as described for + jobdirs above. + +jobstates + This associative array gives information about the states of the + jobs currently known. The keys are the job numbers and the values + are strings of the form `JOB-STATE:MARK:PID=STATE...'. The + JOB-STATE gives the state the whole job is currently in, one of + `running', `suspended', or `done'. The MARK is `+' for the current + job, `-' for the previous job and empty otherwise. This is + followed by one `:PID=STATE' for every process in the job. The + PIDs are, of course, the process IDs and the STATE describes the + state of that process. + + Handling of the keys of the associative array is as described for + jobdirs above. + +nameddirs + This associative array maps the names of named directories to the + pathnames they stand for. + +userdirs + This associative array maps user names to the pathnames of their + home directories. + +usergroups + This associative array maps names of system groups of which the + current user is a member to the corresponding group identifiers. + The contents are the same as the groups output by the id command. + +funcfiletrace + This array contains the absolute line numbers and corresponding + file names for the point where the current function, sourced file, + or (if EVAL_LINENO is set) eval command was called. The array is + of the same length as funcsourcetrace and functrace, but differs + from funcsourcetrace in that the line and file are the point of + call, not the point of definition, and differs from functrace in + that all values are absolute line numbers in files, rather than + relative to the start of a function, if any. + +funcsourcetrace + This array contains the file names and line numbers of the points + where the functions, sourced files, and (if EVAL_LINENO is set) + eval commands currently being executed were defined. The line + number is the line where the `function NAME' or `NAME ()' started. + In the case of an autoloaded function the line number is + reported as zero. The format of each element is FILENAME:LINENO. + + For functions autoloaded from a file in native zsh format, where + only the body of the function occurs in the file, or for files + that have been executed by the source or `.' builtins, the trace + information is shown as FILENAME:0, since the entire file is the + definition. The source file name is resolved to an absolute path + when the function is loaded or the path to it otherwise resolved. + + Most users will be interested in the information in the + funcfiletrace array instead. + +funcstack + This array contains the names of the functions, sourced files, and + (if EVAL_LINENO is set) eval commands. currently being executed. + The first element is the name of the function using the parameter. + + The standard shell array zsh_eval_context can be used to determine + the type of shell construct being executed at each depth: note, + however, that is in the opposite order, with the most recent item + last, and it is more detailed, for example including an entry for + toplevel, the main shell code being executed either interactively + or from a script, which is not present in $funcstack. + +functrace + This array contains the names and line numbers of the callers + corresponding to the functions currently being executed. The + format of each element is NAME:LINENO. Callers are also shown for + sourced files; the caller is the point where the source or `.' + command was executed. + + + +File: zsh.info, Node: The zsh/pcre Module, Next: The zsh/param/private Module, Prev: The zsh/parameter Module, Up: Zsh Modules + +22.21 The zsh/pcre Module +========================= + + + + The zsh/pcre module makes some commands available as builtins: + + +pcre_compile [ -aimxs ] PCRE + Compiles a perl-compatible regular expression. + + Option -a will force the pattern to be anchored. Option -i will + compile a case-insensitive pattern. Option -m will compile a + multi-line pattern; that is, ^ and $ will match newlines within + the pattern. Option -x will compile an extended pattern, wherein + whitespace and # comments are ignored. Option -s makes the dot + metacharacter match all characters, including those that indicate + newline. + +pcre_study + Studies the previously-compiled PCRE which may result in faster + matching. + +pcre_match [ -v VAR ] [ -a ARR ] [ -n OFFSET ] [ -b ] STRING + Returns successfully if string matches the previously-compiled + PCRE. + + Upon successful match, if the expression captures substrings + within parentheses, pcre_match will set the array match to those + substrings, unless the -a option is given, in which case it will + set the array ARR. Similarly, the variable MATCH will be set to + the entire matched portion of the string, unless the -v option is + given, in which case the variable VAR will be set. No variables + are altered if there is no successful match. A -n option starts + searching for a match from the byte OFFSET position in STRING. If + the -b option is given, the variable ZPCRE_OP will be set to an + offset pair string, representing the byte offset positions of the + entire matched portion within the STRING. For example, a ZPCRE_OP + set to "32 45" indicates that the matched portion began on byte + offset 32 and ended on byte offset 44. Here, byte offset position + 45 is the position directly after the matched portion. Keep in + mind that the byte position isn't necessarily the same as the + character position when UTF-8 characters are involved. + Consequently, the byte offset positions are only to be relied on + in the context of using them for subsequent searches on STRING, + using an offset position as an argument to the -n option. This is + mostly used to implement the "find all non-overlapping matches" + functionality. + + A simple example of "find all non-overlapping matches": + + + string="The following zip codes: 78884 90210 99513" + pcre_compile -m "\d{5}" + accum=() + pcre_match -b -- $string + while [[ $? -eq 0 ]] do + b=($=ZPCRE_OP) + accum+=$MATCH + pcre_match -b -n $b[2] -- $string + done + print -l $accum + + +The zsh/pcre module makes available the following test condition: + + +EXPR -pcre-match PCRE + Matches a string against a perl-compatible regular expression. + + For example, + + + [[ "$text" -pcre-match ^d+$ ]] && + print text variable contains only "d's". + + If the REMATCH_PCRE option is set, the =~ operator is equivalent to + -pcre-match, and the NO_CASE_MATCH option may be used. Note that + NO_CASE_MATCH never applies to the pcre_match builtin, instead use + the -i switch of pcre_compile. + + + +File: zsh.info, Node: The zsh/param/private Module, Next: The zsh/regex Module, Prev: The zsh/pcre Module, Up: Zsh Modules + +22.22 The zsh/param/private Module +================================== + + + + The zsh/param/private module is used to create parameters whose scope +is limited to the current function body, and _not_ to other functions +called by the current function. + +This module provides a single autoloaded builtin: + +private [ {+|-}AHUahlprtux ] [ {+|-}EFLRZi [ N ] ] [ NAME[=VALUE] ... ] + The private builtin accepts all the same options and arguments as + local (*Note Shell Builtin Commands::) except for the `-T' option. + Tied parameters may not be made private. + + If used at the top level (outside a function scope), private + creates a normal parameter in the same manner as declare or + typeset. A warning about this is printed if WARN_CREATE_GLOBAL is + set (*Note Options::). Used inside a function scope, private + creates a local parameter similar to one declared with local, + except having special properties noted below. + + Special parameters which expose or manipulate internal shell + state, such as ARGC, argv, COLUMNS, LINES, UID, EUID, IFS, PROMPT, + RANDOM, SECONDS, etc., cannot be made private unless the `-h' + option is used to hide the special meaning of the parameter. This + may change in the future. + + +As with other typeset equivalents, private is both a builtin and a +reserved word, so arrays may be assigned with parenthesized word list +NAME=(VALUE...) syntax. However, the reserved word `private' is not +available until zsh/param/private is loaded, so care must be taken with +order of execution and parsing for function definitions which use +private. To compensate for this, the module also adds the option `-P' +to the `local' builtin to declare private parameters. + +For example, this construction fails if zsh/param/private has not yet +been loaded when `bad_declaration' is defined: + bad_declaration() { + zmodload zsh/param/private + private array=( one two three ) + } + +This construction works because local is already a keyword, and the +module is loaded before the statement is executed: + good_declaration() { + zmodload zsh/param/private + local -P array=( one two three ) + } + +The following is usable in scripts but may have trouble with autoload: + zmodload zsh/param/private + iffy_declaration() { + private array=( one two three ) + } + +The private builtin may always be used with scalar assignments and for +declarations without assignments. + +Parameters declared with private have the following properties: + + * Within the function body where it is declared, the parameter + behaves as a local, except as noted above for tied or special + parameters. + + * The type of a parameter declared private cannot be changed in the + scope where it was declared, even if the parameter is unset. Thus + an array cannot be assigned to a private scalar, etc. + + * Within any other function called by the declaring function, the + private parameter does _NOT_ hide other parameters of the same + name, so for example a global parameter of the same name is + visible and may be assigned or unset. This includes calls to + anonymous functions, although that may also change in the future. + + * An exported private remains in the environment of inner scopes but + appears unset for the current shell in those scopes. Generally, + exporting private parameters should be avoided. + +Note that this differs from the static scope defined by compiled +languages derived from C, in that the a new call to the same function +creates a new scope, i.e., the parameter is still associated with the +call stack rather than with the function definition. It differs from +ksh `typeset -S' because the syntax used to define the function has no +bearing on whether the parameter scope is respected. + + +File: zsh.info, Node: The zsh/regex Module, Next: The zsh/sched Module, Prev: The zsh/param/private Module, Up: Zsh Modules + +22.23 The zsh/regex Module +========================== + + + + The zsh/regex module makes available the following test condition: + + +EXPR -regex-match REGEX + Matches a string against a POSIX extended regular expression. On + successful match, matched portion of the string will normally be + placed in the MATCH variable. If there are any capturing + parentheses within the regex, then the match array variable will + contain those. If the match is not successful, then the variables + will not be altered. + + For example, + + + [[ alphabetical -regex-match ^a([^a]+)a([^a]+)a ]] && + print -l $MATCH X $match + + If the option REMATCH_PCRE is not set, then the =~ operator will + automatically load this module as needed and will invoke the + -regex-match operator. + + If BASH_REMATCH is set, then the array BASH_REMATCH will be set + instead of MATCH and match. + + + +File: zsh.info, Node: The zsh/sched Module, Next: The zsh/net/socket Module, Prev: The zsh/regex Module, Up: Zsh Modules + +22.24 The zsh/sched Module +========================== + + + + The zsh/sched module makes available one builtin command and one +parameter. + + +sched [-o] [+]HH:MM[:SS] COMMAND ... +sched [-o] [+]SECONDS COMMAND ... +sched [ -ITEM ] + Make an entry in the scheduled list of commands to execute. The + time may be specified in either absolute or relative time, and + either as hours, minutes and (optionally) seconds separated by a + colon, or seconds alone. An absolute number of seconds indicates + the time since the epoch (1970/01/01 00:00); this is useful in + combination with the features in the zsh/datetime module, see + *Note The zsh/datetime Module::. + + With no arguments, prints the list of scheduled commands. If the + scheduled command has the -o flag set, this is shown at the start + of the command. + + With the argument `-ITEM', removes the given item from the list. + The numbering of the list is continuous and entries are in time + order, so the numbering can change when entries are added or + deleted. + + Commands are executed either immediately before a prompt, or while + the shell's line editor is waiting for input. In the latter case + it is useful to be able to produce output that does not interfere + with the line being edited. Providing the option -o causes the + shell to clear the command line before the event and redraw it + afterwards. This should be used with any scheduled event that + produces visible output to the terminal; it is not needed, for + example, with output that updates a terminal emulator's title bar. + + To effect changes to the editor buffer when an event executes, use + the `zle' command with no arguments to test whether the editor is + active, and if it is, then use `zle WIDGET' to access the editor + via the named WIDGET. + + The sched builtin is not made available by default when the shell + starts in a mode emulating another shell. It can be made available + with the command `zmodload -F zsh/sched b:sched'. + + + +zsh_scheduled_events + A readonly array corresponding to the events scheduled by the + sched builtin. The indices of the array correspond to the numbers + shown when sched is run with no arguments (provided that the + KSH_ARRAYS option is not set). The value of the array consists of + the scheduled time in seconds since the epoch (see *Note The + zsh/datetime Module:: for facilities for using this number), + followed by a colon, followed by any options (which may be empty + but will be preceded by a `-' otherwise), followed by a colon, + followed by the command to be executed. + + The sched builtin should be used for manipulating the events. Note + that this will have an immediate effect on the contents of the + array, so that indices may become invalid. + + + +File: zsh.info, Node: The zsh/net/socket Module, Next: The zsh/stat Module, Prev: The zsh/sched Module, Up: Zsh Modules + +22.25 The zsh/net/socket Module +=============================== + + + + The zsh/net/socket module makes available one builtin command: + + +zsocket [ -altv ] [ -d FD ] [ ARGS ] + zsocket is implemented as a builtin to allow full use of shell + command line editing, file I/O, and job control mechanisms. + + + +22.25.1 Outbound Connections +---------------------------- + + + + +zsocket [ -v ] [ -d FD ] FILENAME + Open a new Unix domain connection to FILENAME. The shell + parameter REPLY will be set to the file descriptor associated with + that connection. Currently, only stream connections are supported. + + If -d is specified, its argument will be taken as the target file + descriptor for the connection. + + In order to elicit more verbose output, use -v. + + File descriptors can be closed with normal shell syntax when no + longer needed, for example: + + + exec {REPLY}>&- + + + +22.25.2 Inbound Connections +--------------------------- + + + + +zsocket -l [ -v ] [ -d FD ] FILENAME + zsocket -l will open a socket listening on FILENAME. The shell + parameter REPLY will be set to the file descriptor associated with + that listener. The file descriptor remains open in subshells and + forked external executables. + + If -d is specified, its argument will be taken as the target file + descriptor for the connection. + + In order to elicit more verbose output, use -v. + +zsocket -a [ -tv ] [ -d TARGETFD ] LISTENFD + zsocket -a will accept an incoming connection to the socket + associated with LISTENFD. The shell parameter REPLY will be set + to the file descriptor associated with the inbound connection. + The file descriptor remains open in subshells and forked external + executables. + + If -d is specified, its argument will be taken as the target file + descriptor for the connection. + + If -t is specified, zsocket will return if no incoming connection + is pending. Otherwise it will wait for one. + + In order to elicit more verbose output, use -v. + + + +File: zsh.info, Node: The zsh/stat Module, Next: The zsh/system Module, Prev: The zsh/net/socket Module, Up: Zsh Modules + +22.26 The zsh/stat Module +========================= + + + + The zsh/stat module makes available one builtin command under two +possible names: + + +zstat [ -gnNolLtTrs ] [ -f FD ] [ -H HASH ] [ -A ARRAY ] [ -F FMT ] + [ +ELEMENT ] [ FILE ... ] +stat ... + The command acts as a front end to the stat system call (see man + page stat(2)). The same command is provided with two names; as + the name stat is often used by an external command it is + recommended that only the zstat form of the command is used. This + can be arranged by loading the module with the command `zmodload + -F zsh/stat b:zstat'. + + If the stat call fails, the appropriate system error message + printed and status 1 is returned. The fields of struct stat give + information about the files provided as arguments to the command. + In addition to those available from the stat call, an extra + element `link' is provided. These elements are: + + + device + The number of the device on which the file resides. + + inode + The unique number of the file on this device (`_inode_' + number). + + mode + The mode of the file; that is, the file's type and access + permissions. With the -s option, this will be returned as a + string corresponding to the first column in the display of + the ls -l command. + + nlink + The number of hard links to the file. + + uid + The user ID of the owner of the file. With the -s option, + this is displayed as a user name. + + gid + The group ID of the file. With the -s option, this is + displayed as a group name. + + rdev + The raw device number. This is only useful for special + devices. + + size + The size of the file in bytes. + + atime + mtime + ctime + The last access, modification and inode change times of the + file, respectively, as the number of seconds since midnight + GMT on 1st January, 1970. With the -s option, these are + printed as strings for the local time zone; the format can be + altered with the -F option, and with the -g option the times + are in GMT. + + blksize + The number of bytes in one allocation block on the device on + which the file resides. + + block + The number of disk blocks used by the file. + + link + If the file is a link and the -L option is in effect, this + contains the name of the file linked to, otherwise it is + empty. Note that if this element is selected (``zstat + +link'') then the -L option is automatically used. + + + A particular element may be selected by including its name + preceded by a `+' in the option list; only one element is allowed. + The element may be shortened to any unique set of leading + characters. Otherwise, all elements will be shown for all files. + + Options: + + + -A ARRAY + Instead of displaying the results on standard output, assign + them to an ARRAY, one struct stat element per array element + for each file in order. In this case neither the name of the + element nor the name of the files appears in ARRAY unless the + -t or -n options were given, respectively. If -t is given, + the element name appears as a prefix to the appropriate array + element; if -n is given, the file name appears as a separate + array element preceding all the others. Other formatting + options are respected. + + -H HASH + Similar to -A, but instead assign the values to HASH. The + keys are the elements listed above. If the -n option is + provided then the name of the file is included in the hash + with key name. + + -f FD + Use the file on file descriptor FD instead of named files; no + list of file names is allowed in this case. + + -F FMT + Supplies a strftime (see man page strftime(3)) string for the + formatting of the time elements. The format string supports + all of the zsh extensions described in *Note Prompt + Expansion::. The -s option is implied. + + -g + Show the time elements in the GMT time zone. The -s option + is implied. + + -l + List the names of the type elements (to standard output or an + array as appropriate) and return immediately; arguments, and + options other than -A, are ignored. + + -L + Perform an lstat (see man page lstat(2)) rather than a stat + system call. In this case, if the file is a link, information + about the link itself rather than the target file is returned. + This option is required to make the link element useful. + It's important to note that this is the exact opposite from + man page ls(1), etc. + + -n + Always show the names of files. Usually these are only shown + when output is to standard output and there is more than one + file in the list. + + -N + Never show the names of files. + + -o + If a raw file mode is printed, show it in octal, which is + more useful for human consumption than the default of + decimal. A leading zero will be printed in this case. Note + that this does not affect whether a raw or formatted file + mode is shown, which is controlled by the -r and -s options, + nor whether a mode is shown at all. + + -r + Print raw data (the default format) alongside string data + (the -s format); the string data appears in parentheses after + the raw data. + + -s + Print mode, uid, gid and the three time elements as strings + instead of numbers. In each case the format is like that of + ls -l. + + -t + Always show the type names for the elements of struct stat. + Usually these are only shown when output is to standard + output and no individual element has been selected. + + -T + Never show the type names of the struct stat elements. + + + + +File: zsh.info, Node: The zsh/system Module, Next: The zsh/net/tcp Module, Prev: The zsh/stat Module, Up: Zsh Modules + +22.27 The zsh/system Module +=========================== + + + + The zsh/system module makes available various builtin commands and +parameters. + + + +22.27.1 Builtins +---------------- + + +syserror [ -e ERRVAR ] [ -p PREFIX ] [ ERRNO | ERRNAME ] + This command prints out the error message associated with ERRNO, a + system error number, followed by a newline to standard error. + + Instead of the error number, a name ERRNAME, for example ENOENT, + may be used. The set of names is the same as the contents of the + array errnos, see below. + + If the string PREFIX is given, it is printed in front of the error + message, with no intervening space. + + If ERRVAR is supplied, the entire message, without a newline, is + assigned to the parameter names ERRVAR and nothing is output. + + A return status of 0 indicates the message was successfully printed + (although it may not be useful if the error number was out of the + system's range), a return status of 1 indicates an error in the + parameters, and a return status of 2 indicates the error name was + not recognised (no message is printed for this). + +sysopen [ -arw ] [ -m PERMISSIONS ] [ -o OPTIONS ] + -u FD FILE + This command opens a file. The -r, -w and -a flags indicate + whether the file should be opened for reading, writing and + appending, respectively. The -m option allows the initial + permissions to use when creating a file to be specified in octal + form. The file descriptor is specified with -u. Either an + explicit file descriptor in the range 0 to 9 can be specified or a + variable name can be given to which the file descriptor number + will be assigned. + + The -o option allows various system specific options to be + specified as a comma-separated list. The following is a list of + possible options. Note that, depending on the system, some may not + be available. + cloexec + mark file to be closed when other programs are executed (else + the file descriptor remains open in subshells and forked + external executables) + + create + creat + create file if it does not exist + + excl + create file, error if it already exists + + noatime + suppress updating of the file atime + + nofollow + fail if FILE is a symbolic link + + sync + request that writes wait until data has been physically + written + + truncate + trunc + truncate file to size 0 + + + To close the file, use one of the following: + + + exec {FD}<&- + exec {FD}>&- + +sysread [ -c COUNTVAR ] [ -i INFD ] [ -o OUTFD ] + [ -s BUFSIZE ] [ -t TIMEOUT ] [ PARAM ] + Perform a single system read from file descriptor INFD, or zero if + that is not given. The result of the read is stored in PARAM or + REPLY if that is not given. If COUNTVAR is given, the number of + bytes read is assigned to the parameter named by COUNTVAR. + + The maximum number of bytes read is BUFSIZE or 8192 if that is not + given, however the command returns as soon as any number of bytes + was successfully read. + + If TIMEOUT is given, it specifies a timeout in seconds, which may + be zero to poll the file descriptor. This is handled by the poll + system call if available, otherwise the select system call if + available. + + If OUTFD is given, an attempt is made to write all the bytes just + read to the file descriptor OUTFD. If this fails, because of a + system error other than EINTR or because of an internal zsh error + during an interrupt, the bytes read but not written are stored in + the parameter named by PARAM if supplied (no default is used in + this case), and the number of bytes read but not written is stored + in the parameter named by COUNTVAR if that is supplied. If it was + successful, COUNTVAR contains the full number of bytes transferred, + as usual, and PARAM is not set. + + The error EINTR (interrupted system call) is handled internally so + that shell interrupts are transparent to the caller. Any other + error causes a return. + + The possible return statuses are + 0 + At least one byte of data was successfully read and, if + appropriate, written. + + 1 + There was an error in the parameters to the command. This is + the only error for which a message is printed to standard + error. + + 2 + There was an error on the read, or on polling the input file + descriptor for a timeout. The parameter ERRNO gives the + error. + + 3 + Data were successfully read, but there was an error writing + them to OUTFD. The parameter ERRNO gives the error. + + 4 + The attempt to read timed out. Note this does not set ERRNO + as this is not a system error. + + 5 + No system error occurred, but zero bytes were read. This + usually indicates end of file. The parameters are set + according to the usual rules; no write to OUTFD is attempted. + + +sysseek [ -u FD ] [ -w start|end|current ] OFFSET + The current file position at which future reads and writes will + take place is adjusted to the specified byte offset. The OFFSET is + evaluated as a math expression. The -u option allows the file + descriptor to be specified. By default the offset is specified + relative to the start or the file but, with the -w option, it is + possible to specify that the offset should be relative to the + current position or the end of the file. + +syswrite [ -c COUNTVAR ] [ -o OUTFD ] DATA + The data (a single string of bytes) are written to the file + descriptor OUTFD, or 1 if that is not given, using the write + system call. Multiple write operations may be used if the first + does not write all the data. + + If COUNTVAR is given, the number of byte written is stored in the + parameter named by COUNTVAR; this may not be the full length of + DATA if an error occurred. + + The error EINTR (interrupted system call) is handled internally by + retrying; otherwise an error causes the command to return. For + example, if the file descriptor is set to non-blocking output, an + error EAGAIN (on some systems, EWOULDBLOCK) may result in the + command returning early. + + The return status may be 0 for success, 1 for an error in the + parameters to the command, or 2 for an error on the write; no + error message is printed in the last case, but the parameter ERRNO + will reflect the error that occurred. + +zsystem flock [ -t TIMEOUT ] [ -f VAR ] [-er] FILE +zsystem flock -u FD_EXPR + The builtin zsystem's subcommand flock performs advisory file + locking (via the man page fcntl(2) system call) over the entire + contents of the given file. This form of locking requires the + processes accessing the file to cooperate; its most obvious use is + between two instances of the shell itself. + + In the first form the named FILE, which must already exist, is + locked by opening a file descriptor to the file and applying a + lock to the file descriptor. The lock terminates when the shell + process that created the lock exits; it is therefore often + convenient to create file locks within subshells, since the lock + is automatically released when the subshell exits. Note that use + of the print builtin with the -u option will, as a side effect, + release the lock, as will redirection to the file in the shell + holding the lock. To work around this use a subshell, e.g. + `(print message) >> FILE'. Status 0 is returned if the lock + succeeds, else status 1. + + In the second form the file descriptor given by the arithmetic + expression FD_EXPR is closed, releasing a lock. The file + descriptor can be queried by using the `-f VAR' form during the + lock; on a successful lock, the shell variable VAR is set to the + file descriptor used for locking. The lock will be released if the + file descriptor is closed by any other means, for example using + `exec {VAR}>&-'; however, the form described here performs a + safety check that the file descriptor is in use for file locking. + + By default the shell waits indefinitely for the lock to succeed. + The option -t TIMEOUT specifies a timeout for the lock in seconds; + currently this must be an integer. The shell will attempt to lock + the file once a second during this period. If the attempt times + out, status 2 is returned. + + If the option -e is given, the file descriptor for the lock is + preserved when the shell uses exec to start a new process; + otherwise it is closed at that point and the lock released. + + If the option -r is given, the lock is only for reading, otherwise + it is for reading and writing. The file descriptor is opened + accordingly. + +zsystem supports SUBCOMMAND + The builtin zsystem's subcommand supports tests whether a given + subcommand is supported. It returns status 0 if so, else status + 1. It operates silently unless there was a syntax error (i.e. the + wrong number of arguments), in which case status 255 is returned. + Status 1 can indicate one of two things: SUBCOMMAND is known but + not supported by the current operating system, or SUBCOMMAND is + not known (possibly because this is an older version of the shell + before it was implemented). + + + +22.27.2 Math Functions +---------------------- + + +systell(FD) + The systell math function returns the current file position for + the file descriptor passed as an argument. + + + +22.27.3 Parameters +------------------ + + +errnos + A readonly array of the names of errors defined on the system. + These are typically macros defined in C by including the system + header file errno.h. The index of each name (assuming the option + KSH_ARRAYS is unset) corresponds to the error number. Error + numbers NUM before the last known error which have no name are + given the name ENUM in the array. + + Note that aliases for errors are not handled; only the canonical + name is used. + +sysparams + A readonly associative array. The keys are: + + + pid + Returns the process ID of the current process, even in + subshells. Compare $$, which returns the process ID of the + main shell process. + + ppid + Returns the process ID of the parent of the current process, + even in subshells. Compare $PPID, which returns the process + ID of the parent of the main shell process. + + procsubstpid + Returns the process ID of the last process started for process + substitution, i.e. the <(...) and >(...) expansions. + + + + +File: zsh.info, Node: The zsh/net/tcp Module, Next: The zsh/termcap Module, Prev: The zsh/system Module, Up: Zsh Modules + +22.28 The zsh/net/tcp Module +============================ + + + + The zsh/net/tcp module makes available one builtin command: + + +ztcp [ -acflLtv ] [ -d FD ] [ ARGS ] + ztcp is implemented as a builtin to allow full use of shell + command line editing, file I/O, and job control mechanisms. + + If ztcp is run with no options, it will output the contents of its + session table. + + If it is run with only the option -L, it will output the contents + of the session table in a format suitable for automatic parsing. + The option is ignored if given with a command to open or close a + session. The output consists of a set of lines, one per session, + each containing the following elements separated by spaces: + + + File descriptor + The file descriptor in use for the connection. For normal + inbound (I) and outbound (O) connections this may be read and + written by the usual shell mechanisms. However, it should + only be close with `ztcp -c'. + + Connection type + A letter indicating how the session was created: + + + Z + A session created with the zftp command. + + L + A connection opened for listening with `ztcp -l'. + + I + An inbound connection accepted with `ztcp -a'. + + O + An outbound connection created with `ztcp HOST ...'. + + + + The local host + This is usually set to an all-zero IP address as the address + of the localhost is irrelevant. + + The local port + This is likely to be zero unless the connection is for + listening. + + The remote host + This is the fully qualified domain name of the peer, if + available, else an IP address. It is an all-zero IP address + for a session opened for listening. + + The remote port + This is zero for a connection opened for listening. + + + + +22.28.1 Outbound Connections +---------------------------- + + + + +ztcp [ -v ] [ -d FD ] HOST [ PORT ] + Open a new TCP connection to HOST. If the PORT is omitted, it + will default to port 23. The connection will be added to the + session table and the shell parameter REPLY will be set to the + file descriptor associated with that connection. + + If -d is specified, its argument will be taken as the target file + descriptor for the connection. + + In order to elicit more verbose output, use -v. + + + +22.28.2 Inbound Connections +--------------------------- + + + + +ztcp -l [ -v ] [ -d FD ] PORT + ztcp -l will open a socket listening on TCP PORT. The socket will + be added to the session table and the shell parameter REPLY will + be set to the file descriptor associated with that listener. + + If -d is specified, its argument will be taken as the target file + descriptor for the connection. + + In order to elicit more verbose output, use -v. + +ztcp -a [ -tv ] [ -d TARGETFD ] LISTENFD + ztcp -a will accept an incoming connection to the port associated + with LISTENFD. The connection will be added to the session table + and the shell parameter REPLY will be set to the file descriptor + associated with the inbound connection. + + If -d is specified, its argument will be taken as the target file + descriptor for the connection. + + If -t is specified, ztcp will return if no incoming connection is + pending. Otherwise it will wait for one. + + In order to elicit more verbose output, use -v. + + + +22.28.3 Closing Connections +--------------------------- + + + + +ztcp -cf [ -v ] [ FD ] +ztcp -c [ -v ] [ FD ] + ztcp -c will close the socket associated with FD. The socket will + be removed from the session table. If FD is not specified, ztcp + will close everything in the session table. + + Normally, sockets registered by zftp (see *Note The zsh/zftp + Module:: ) cannot be closed this way. In order to force such a + socket closed, use -f. + + In order to elicit more verbose output, use -v. + + + +22.28.4 Example +--------------- + +Here is how to create a TCP connection between two instances of zsh. We +need to pick an unassigned port; here we use the randomly chosen 5123. + +On host1, + zmodload zsh/net/tcp + ztcp -l 5123 + listenfd=$REPLY + ztcp -a $listenfd + fd=$REPLY + The second from last command blocks until there is an incoming +connection. + +Now create a connection from host2 (which may, of course, be the same +machine): + zmodload zsh/net/tcp + ztcp host1 5123 + fd=$REPLY + +Now on each host, $fd contains a file descriptor for talking to the +other. For example, on host1: + print This is a message >&$fd + and on host2: + read -r line <&$fd; print -r - $line + prints `This is a message'. + +To tidy up, on host1: + ztcp -c $listenfd + ztcp -c $fd + and on host2 + ztcp -c $fd + + +File: zsh.info, Node: The zsh/termcap Module, Next: The zsh/terminfo Module, Prev: The zsh/net/tcp Module, Up: Zsh Modules + +22.29 The zsh/termcap Module +============================ + + + + The zsh/termcap module makes available one builtin command: + + +echotc CAP [ ARG ... ] + Output the termcap value corresponding to the capability CAP, with + optional arguments. + + +The zsh/termcap module makes available one parameter: + + +termcap + An associative array that maps termcap capability codes to their + values. + + + +File: zsh.info, Node: The zsh/terminfo Module, Next: The zsh/zftp Module, Prev: The zsh/termcap Module, Up: Zsh Modules + +22.30 The zsh/terminfo Module +============================= + + + + The zsh/terminfo module makes available one builtin command: + + +echoti CAP [ ARG ] + Output the terminfo value corresponding to the capability CAP, + instantiated with ARG if applicable. + + +The zsh/terminfo module makes available one parameter: + + +terminfo + An associative array that maps terminfo capability names to their + values. + + + +File: zsh.info, Node: The zsh/zftp Module, Next: The zsh/zle Module, Prev: The zsh/terminfo Module, Up: Zsh Modules + +22.31 The zsh/zftp Module +========================= + + + + The zsh/zftp module makes available one builtin command: + + +zftp SUBCOMMAND [ ARGS ] + The zsh/zftp module is a client for FTP (file transfer protocol). + It is implemented as a builtin to allow full use of shell command + line editing, file I/O, and job control mechanisms. Often, users + will access it via shell functions providing a more powerful + interface; a set is provided with the zsh distribution and is + described in *Note Zftp Function System::. However, the zftp + command is entirely usable in its own right. + + All commands consist of the command name zftp followed by the name + of a subcommand. These are listed below. The return status of + each subcommand is supposed to reflect the success or failure of + the remote operation. See a description of the variable + ZFTP_VERBOSE for more information on how responses from the server + may be printed. + + + +22.31.1 Subcommands +------------------- + + + + +open HOST[:PORT] [ USER [ PASSWORD [ ACCOUNT ] ] ] + Open a new FTP session to HOST, which may be the name of a TCP/IP + connected host or an IP number in the standard dot notation. If + the argument is in the form HOST:PORT, open a connection to TCP + port PORT instead of the standard FTP port 21. This may be the + name of a TCP service or a number: see the description of + ZFTP_PORT below for more information. + + If IPv6 addresses in colon format are used, the HOST should be + surrounded by quoted square brackets to distinguish it from the + PORT, for example '[fe80::203:baff:fe02:8b56]'. For consistency + this is allowed with all forms of HOST. + + Remaining arguments are passed to the login subcommand. Note that + if no arguments beyond HOST are supplied, open will _not_ + automatically call login. If no arguments at all are supplied, + open will use the parameters set by the params subcommand. + + After a successful open, the shell variables ZFTP_HOST, ZFTP_PORT, + ZFTP_IP and ZFTP_SYSTEM are available; see `Variables' below. + +login [ NAME [ PASSWORD [ ACCOUNT ] ] ] +user [ NAME [ PASSWORD [ ACCOUNT ] ] ] + Login the user NAME with parameters PASSWORD and ACCOUNT. Any of + the parameters can be omitted, and will be read from standard + input if needed (NAME is always needed). If standard input is a + terminal, a prompt for each one will be printed on standard error + and PASSWORD will not be echoed. If any of the parameters are not + used, a warning message is printed. + + After a successful login, the shell variables ZFTP_USER, + ZFTP_ACCOUNT and ZFTP_PWD are available; see `Variables' below. + + This command may be re-issued when a user is already logged in, and + the server will first be reinitialized for a new user. + +params [ HOST [ USER [ PASSWORD [ ACCOUNT ] ] ] ] +params - + Store the given parameters for a later open command with no + arguments. Only those given on the command line will be + remembered. If no arguments are given, the parameters currently + set are printed, although the password will appear as a line of + stars; the return status is one if no parameters were set, zero + otherwise. + + Any of the parameters may be specified as a `?', which may need to + be quoted to protect it from shell expansion. In this case, the + appropriate parameter will be read from stdin as with the login + subcommand, including special handling of PASSWORD. If the `?' is + followed by a string, that is used as the prompt for reading the + parameter instead of the default message (any necessary + punctuation and whitespace should be included at the end of the + prompt). The first letter of the parameter (only) may be quoted + with a `\'; hence an argument "\\$word" guarantees that the string + from the shell parameter $word will be treated literally, whether + or not it begins with a `?'. + + If instead a single `-' is given, the existing parameters, if any, + are deleted. In that case, calling open with no arguments will + cause an error. + + The list of parameters is not deleted after a close, however it + will be deleted if the zsh/zftp module is unloaded. + + For example, + + + zftp params ftp.elsewhere.xx juser '?Password for juser: ' + + will store the host ftp.elsewhere.xx and the user juser and then + prompt the user for the corresponding password with the given + prompt. + +test + Test the connection; if the server has reported that it has closed + the connection (maybe due to a timeout), return status 2; if no + connection was open anyway, return status 1; else return status 0. + The test subcommand is silent, apart from messages printed by the + $ZFTP_VERBOSE mechanism, or error messages if the connection + closes. There is no network overhead for this test. + + The test is only supported on systems with either the select(2) or + poll(2) system calls; otherwise the message `not supported on this + system' is printed instead. + + The test subcommand will automatically be called at the start of + any other subcommand for the current session when a connection is + open. + +cd DIRECTORY + Change the remote directory to DIRECTORY. Also alters the shell + variable ZFTP_PWD. + +cdup + Change the remote directory to the one higher in the directory + tree. Note that cd .. will also work correctly on non-UNIX + systems. + +dir [ ARG ... ] + Give a (verbose) listing of the remote directory. The ARGs are + passed directly to the server. The command's behaviour is + implementation dependent, but a UNIX server will typically + interpret ARGs as arguments to the ls command and with no + arguments return the result of `ls -l'. The directory is listed to + standard output. + +ls [ ARG ... ] + Give a (short) listing of the remote directory. With no ARG, + produces a raw list of the files in the directory, one per line. + Otherwise, up to vagaries of the server implementation, behaves + similar to dir. + +type [ TYPE ] + Change the type for the transfer to TYPE, or print the current type + if TYPE is absent. The allowed values are `A' (ASCII), `I' + (Image, i.e. binary), or `B' (a synonym for `I'). + + The FTP default for a transfer is ASCII. However, if zftp finds + that the remote host is a UNIX machine with 8-bit byes, it will + automatically switch to using binary for file transfers upon open. + This can subsequently be overridden. + + The transfer type is only passed to the remote host when a data + connection is established; this command involves no network + overhead. + +ascii + The same as type A. + +binary + The same as type I. + +mode [ S | B ] + Set the mode type to stream (S) or block (B). Stream mode is the + default; block mode is not widely supported. + +remote FILE ... +local [ FILE ... ] + Print the size and last modification time of the remote or local + files. If there is more than one item on the list, the name of the + file is printed first. The first number is the file size, the + second is the last modification time of the file in the format + CCYYMMDDhhmmSS consisting of year, month, date, hour, minutes and + seconds in GMT. Note that this format, including the length, is + guaranteed, so that time strings can be directly compared via the + [[ builtin's < and > operators, even if they are too long to be + represented as integers. + + Not all servers support the commands for retrieving this + information. In that case, the remote command will print nothing + and return status 2, compared with status 1 for a file not found. + + The local command (but not remote) may be used with no arguments, + in which case the information comes from examining file descriptor + zero. This is the same file as seen by a put command with no + further redirection. + +get FILE ... + Retrieve all FILEs from the server, concatenating them and sending + them to standard output. + +put FILE ... + For each FILE, read a file from standard input and send that to + the remote host with the given name. + +append FILE ... + As put, but if the remote FILE already exists, data is appended to + it instead of overwriting it. + +getat FILE POINT +putat FILE POINT +appendat FILE POINT + Versions of get, put and append which will start the transfer at + the given POINT in the remote FILE. This is useful for appending + to an incomplete local file. However, note that this ability is + not universally supported by servers (and is not quite the + behaviour specified by the standard). + +delete FILE ... + Delete the list of files on the server. + +mkdir DIRECTORY + Create a new directory DIRECTORY on the server. + +rmdir DIRECTORY + Delete the directory DIRECTORY on the server. + +rename OLD-NAME NEW-NAME + Rename file OLD-NAME to NEW-NAME on the server. + +site ARG ... + Send a host-specific command to the server. You will probably + only need this if instructed by the server to use it. + +quote ARG ... + Send the raw FTP command sequence to the server. You should be + familiar with the FTP command set as defined in RFC959 before doing + this. Useful commands may include STAT and HELP. Note also the + mechanism for returning messages as described for the variable + ZFTP_VERBOSE below, in particular that all messages from the + control connection are sent to standard error. + +close +quit + Close the current data connection. This unsets the shell + parameters ZFTP_HOST, ZFTP_PORT, ZFTP_IP, ZFTP_SYSTEM, ZFTP_USER, + ZFTP_ACCOUNT, ZFTP_PWD, ZFTP_TYPE and ZFTP_MODE. + +session [ SESSNAME ] + Allows multiple FTP sessions to be used at once. The name of the + session is an arbitrary string of characters; the default session + is called `default'. If this command is called without an + argument, it will list all the current sessions; with an argument, + it will either switch to the existing session called SESSNAME, or + create a new session of that name. + + Each session remembers the status of the connection, the set of + connection-specific shell parameters (the same set as are unset + when a connection closes, as given in the description of close), + and any user parameters specified with the params subcommand. + Changing to a previous session restores those values; changing to + a new session initialises them in the same way as if zftp had just + been loaded. The name of the current session is given by the + parameter ZFTP_SESSION. + +rmsession [ SESSNAME ] + Delete a session; if a name is not given, the current session is + deleted. If the current session is deleted, the earliest existing + session becomes the new current session, otherwise the current + session is not changed. If the session being deleted is the only + one, a new session called `default' is created and becomes the + current session; note that this is a new session even if the + session being deleted is also called `default'. It is recommended + that sessions not be deleted while background commands which use + zftp are still active. + + + +22.31.2 Parameters +------------------ + +The following shell parameters are used by zftp. Currently none of +them are special. + + +ZFTP_TMOUT + Integer. The time in seconds to wait for a network operation to + complete before returning an error. If this is not set when the + module is loaded, it will be given the default value 60. A value + of zero turns off timeouts. If a timeout occurs on the control + connection it will be closed. Use a larger value if this occurs + too frequently. + +ZFTP_IP + Readonly. The IP address of the current connection in dot + notation. + +ZFTP_HOST + Readonly. The hostname of the current remote server. If the host + was opened as an IP number, ZFTP_HOST contains that instead; this + saves the overhead for a name lookup, as IP numbers are most + commonly used when a nameserver is unavailable. + +ZFTP_PORT + Readonly. The number of the remote TCP port to which the + connection is open (even if the port was originally specified as a + named service). Usually this is the standard FTP port, 21. + + In the unlikely event that your system does not have the + appropriate conversion functions, this appears in network byte + order. If your system is little-endian, the port then consists of + two swapped bytes and the standard port will be reported as 5376. + In that case, numeric ports passed to zftp open will also need to + be in this format. + +ZFTP_SYSTEM + Readonly. The system type string returned by the server in + response to an FTP SYST request. The most interesting case is a + string beginning "UNIX Type: L8", which ensures maximum + compatibility with a local UNIX host. + +ZFTP_TYPE + Readonly. The type to be used for data transfers , either `A' or + `I'. Use the type subcommand to change this. + +ZFTP_USER + Readonly. The username currently logged in, if any. + +ZFTP_ACCOUNT + Readonly. The account name of the current user, if any. Most + servers do not require an account name. + +ZFTP_PWD + Readonly. The current directory on the server. + +ZFTP_CODE + Readonly. The three digit code of the last FTP reply from the + server as a string. This can still be read after the connection + is closed, and is not changed when the current session changes. + +ZFTP_REPLY + Readonly. The last line of the last reply sent by the server. + This can still be read after the connection is closed, and is not + changed when the current session changes. + +ZFTP_SESSION + Readonly. The name of the current FTP session; see the + description of the session subcommand. + +ZFTP_PREFS + A string of preferences for altering aspects of zftp's behaviour. + Each preference is a single character. The following are defined: + + + P + Passive: attempt to make the remote server initiate data + transfers. This is slightly more efficient than sendport + mode. If the letter S occurs later in the string, zftp will + use sendport mode if passive mode is not available. + + S + Sendport: initiate transfers by the FTP PORT command. If + this occurs before any P in the string, passive mode will + never be attempted. + + D + Dumb: use only the bare minimum of FTP commands. This + prevents the variables ZFTP_SYSTEM and ZFTP_PWD from being + set, and will mean all connections default to ASCII type. It + may prevent ZFTP_SIZE from being set during a transfer if the + server does not send it anyway (many servers do). + + + If ZFTP_PREFS is not set when zftp is loaded, it will be set to a + default of `PS', i.e. use passive mode if available, otherwise + fall back to sendport mode. + +ZFTP_VERBOSE + A string of digits between 0 and 5 inclusive, specifying which + responses from the server should be printed. All responses go to + standard error. If any of the numbers 1 to 5 appear in the string, + raw responses from the server with reply codes beginning with that + digit will be printed to standard error. The first digit of the + three digit reply code is defined by RFC959 to correspond to: + + + 1. + A positive preliminary reply. + + 2. + A positive completion reply. + + 3. + A positive intermediate reply. + + 4. + A transient negative completion reply. + + 5. + A permanent negative completion reply. + + + It should be noted that, for unknown reasons, the reply `Service + not available', which forces termination of a connection, is + classified as 421, i.e. `transient negative', an interesting + interpretation of the word `transient'. + + The code 0 is special: it indicates that all but the last line of + multiline replies read from the server will be printed to standard + error in a processed format. By convention, servers use this + mechanism for sending information for the user to read. The + appropriate reply code, if it matches the same response, takes + priority. + + If ZFTP_VERBOSE is not set when zftp is loaded, it will be set to + the default value 450, i.e., messages destined for the user and + all errors will be printed. A null string is valid and specifies + that no messages should be printed. + + + +22.31.3 Functions +----------------- + + + + +zftp_chpwd + If this function is set by the user, it is called every time the + directory changes on the server, including when a user is logged + in, or when a connection is closed. In the last case, $ZFTP_PWD + will be unset; otherwise it will reflect the new directory. + +zftp_progress + If this function is set by the user, it will be called during a + get, put or append operation each time sufficient data has been + received from the host. During a get, the data is sent to + standard output, so it is vital that this function should write to + standard error or directly to the terminal, _not_ to standard + output. + + When it is called with a transfer in progress, the following + additional shell parameters are set: + + + ZFTP_FILE + The name of the remote file being transferred from or to. + + ZFTP_TRANSFER + A G for a get operation and a P for a put operation. + + ZFTP_SIZE + The total size of the complete file being transferred: the + same as the first value provided by the remote and local + subcommands for a particular file. If the server cannot + supply this value for a remote file being retrieved, it will + not be set. If input is from a pipe the value may be + incorrect and correspond simply to a full pipe buffer. + + ZFTP_COUNT + The amount of data so far transferred; a number between zero + and $ZFTP_SIZE, if that is set. This number is always + available. + + + The function is initially called with ZFTP_TRANSFER set + appropriately and ZFTP_COUNT set to zero. After the transfer is + finished, the function will be called one more time with + ZFTP_TRANSFER set to GF or PF, in case it wishes to tidy up. It + is otherwise never called twice with the same value of ZFTP_COUNT. + + Sometimes the progress meter may cause disruption. It is up to the + user to decide whether the function should be defined and to use + unfunction when necessary. + + + +22.31.4 Problems +---------------- + + + +A connection may not be opened in the left hand side of a pipe as this +occurs in a subshell and the file information is not updated in the main +shell. In the case of type or mode changes or closing the connection +in a subshell, the information is returned but variables are not +updated until the next call to zftp. Other status changes in subshells +will not be reflected by changes to the variables (but should be +otherwise harmless). + +Deleting sessions while a zftp command is active in the background can +have unexpected effects, even if it does not use the session being +deleted. This is because all shell subprocesses share information on +the state of all connections, and deleting a session changes the +ordering of that information. + +On some operating systems, the control connection is not valid after a +fork(), so that operations in subshells, on the left hand side of a +pipeline, or in the background are not possible, as they should be. +This is presumably a bug in the operating system. + + +File: zsh.info, Node: The zsh/zle Module, Next: The zsh/zleparameter Module, Prev: The zsh/zftp Module, Up: Zsh Modules + +22.32 The zsh/zle Module +======================== + + + + The zsh/zle module contains the Zsh Line Editor. See *Note Zsh Line +Editor::. + + +File: zsh.info, Node: The zsh/zleparameter Module, Next: The zsh/zprof Module, Prev: The zsh/zle Module, Up: Zsh Modules + +22.33 The zsh/zleparameter Module +================================= + + + + The zsh/zleparameter module defines two special parameters that can +be used to access internal information of the Zsh Line Editor (see +*Note Zsh Line Editor::). + + +keymaps + This array contains the names of the keymaps currently defined. + +widgets + This associative array contains one entry per widget. The name of + the widget is the key and the value gives information about the + widget. It is either the string `builtin' for builtin widgets, + a string of the form `user:NAME' for user-defined widgets, + where NAME is the name of the shell function implementing the + widget, a string of the form `completion:TYPE:NAME' for + completion widgets, or a null value if the widget is not yet + fully defined. In the penultimate case, TYPE is the name of the + builtin widget the completion widget imitates in its behavior and + NAME is the name of the shell function implementing the completion + widget. + + + +File: zsh.info, Node: The zsh/zprof Module, Next: The zsh/zpty Module, Prev: The zsh/zleparameter Module, Up: Zsh Modules + +22.34 The zsh/zprof Module +========================== + + + + When loaded, the zsh/zprof causes shell functions to be profiled. +The profiling results can be obtained with the zprof builtin command +made available by this module. There is no way to turn profiling off +other than unloading the module. + + +zprof [ -c ] + Without the -c option, zprof lists profiling results to standard + output. The format is comparable to that of commands like gprof. + + At the top there is a summary listing all functions that were + called at least once. This summary is sorted in decreasing order + of the amount of time spent in each. The lines contain the number + of the function in order, which is used in other parts of the list + in suffixes of the form `[NUM]', then the number of calls made to + the function. The next three columns list the time in + milliseconds spent in the function and its descendants, the average + time in milliseconds spent in the function and its descendants per + call and the percentage of time spent in all shell functions used + in this function and its descendants. The following three columns + give the same information, but counting only the time spent in the + function itself. The final column shows the name of the function. + + After the summary, detailed information about every function that + was invoked is listed, sorted in decreasing order of the amount of + time spent in each function and its descendants. Each of these + entries consists of descriptions for the functions that called the + function described, the function itself, and the functions that + were called from it. The description for the function itself has + the same format as in the summary (and shows the same + information). The other lines don't show the number of the + function at the beginning and have their function named indented to + make it easier to distinguish the line showing the function + described in the section from the surrounding lines. + + The information shown in this case is almost the same as in the + summary, but only refers to the call hierarchy being displayed. + For example, for a calling function the column showing the total + running time lists the time spent in the described function and + its descendants only for the times when it was called from that + particular calling function. Likewise, for a called function, + this columns lists the total time spent in the called function and + its descendants only for the times when it was called from the + function described. + + Also in this case, the column showing the number of calls to a + function also shows a slash and then the total number of + invocations made to the called function. + + As long as the zsh/zprof module is loaded, profiling will be done + and multiple invocations of the zprof builtin command will show the + times and numbers of calls since the module was loaded. With the + -c option, the zprof builtin command will reset its internal + counters and will not show the listing. + + + +File: zsh.info, Node: The zsh/zpty Module, Next: The zsh/zselect Module, Prev: The zsh/zprof Module, Up: Zsh Modules + +22.35 The zsh/zpty Module +========================= + + + + The zsh/zpty module offers one builtin: + + +zpty [ -e ] [ -b ] NAME [ ARG ... ] + The arguments following NAME are concatenated with spaces between, + then executed as a command, as if passed to the eval builtin. The + command runs under a newly assigned pseudo-terminal; this is + useful for running commands non-interactively which expect an + interactive environment. The NAME is not part of the command, but + is used to refer to this command in later calls to zpty. + + With the -e option, the pseudo-terminal is set up so that input + characters are echoed. + + With the -b option, input to and output from the pseudo-terminal + are made non-blocking. + + The shell parameter REPLY is set to the file descriptor assigned to + the master side of the pseudo-terminal. This allows the terminal + to be monitored with ZLE descriptor handlers (see *Note Zle + Builtins::) or manipulated with sysread and syswrite (see *Note + The zsh/system Module::). _Warning_: Use of sysread and syswrite + is _not_ recommended; use zpty -r and zpty -w unless you know + exactly what you are doing. + +zpty -d [ NAME ... ] + The second form, with the -d option, is used to delete commands + previously started, by supplying a list of their NAMEs. If no + NAME is given, all commands are deleted. Deleting a command causes + the HUP signal to be sent to the corresponding process. + +zpty -w [ -n ] NAME [ STRING ... ] + The -w option can be used to send the to command NAME the given + STRINGs as input (separated by spaces). If the -n option is _not_ + given, a newline is added at the end. + + If no STRING is provided, the standard input is copied to the + pseudo-terminal; this may stop before copying the full input if the + pseudo-terminal is non-blocking. The exact input is always copied: + the -n option is not applied. + + Note that the command under the pseudo-terminal sees this input as + if it were typed, so beware when sending special tty driver + characters such as word-erase, line-kill, and end-of-file. + +zpty -r [ -mt ] NAME [ PARAM [ PATTERN ] ] + The -r option can be used to read the output of the command NAME. + With only a NAME argument, the output read is copied to the + standard output. Unless the pseudo-terminal is non-blocking, + copying continues until the command under the pseudo-terminal + exits; when non-blocking, only as much output as is immediately + available is copied. The return status is zero if any output is + copied. + + When also given a PARAM argument, at most one line is read and + stored in the parameter named PARAM. Less than a full line may be + read if the pseudo-terminal is non-blocking. The return status is + zero if at least one character is stored in PARAM. + + If a PATTERN is given as well, output is read until the whole + string read matches the PATTERN, even in the non-blocking case. + The return status is zero if the string read matches the pattern, + or if the command has exited but at least one character could + still be read. If the option -m is present, the return status is + zero only if the pattern matches. As of this writing, a maximum + of one megabyte of output can be consumed this way; if a full + megabyte is read without matching the pattern, the return status + is non-zero. + + In all cases, the return status is non-zero if nothing could be + read, and is 2 if this is because the command has finished. + + If the -r option is combined with the -t option, zpty tests + whether output is available before trying to read. If no output is + available, zpty immediately returns the status 1. When used with + a PATTERN, the behaviour on a failed poll is similar to when the + command has exited: the return value is zero if at least one + character could still be read even if the pattern failed to match. + +zpty -t NAME + The -t option without the -r option can be used to test whether + the command NAME is still running. It returns a zero status if + the command is running and a non-zero value otherwise. + +zpty [ -L ] + The last form, without any arguments, is used to list the commands + currently defined. If the -L option is given, this is done in the + form of calls to the zpty builtin. + + + +File: zsh.info, Node: The zsh/zselect Module, Next: The zsh/zutil Module, Prev: The zsh/zpty Module, Up: Zsh Modules + +22.36 The zsh/zselect Module +============================ + + + + The zsh/zselect module makes available one builtin command: + + +zselect [ -rwe ] [ -t TIMEOUT ] [ -a ARRAY ] [ -A ASSOC ] [ FD ... ] + The zselect builtin is a front-end to the `select' system call, + which blocks until a file descriptor is ready for reading or + writing, or has an error condition, with an optional timeout. If + this is not available on your system, the command prints an error + message and returns status 2 (normal errors return status 1). For + more information, see your systems documentation for man page + select(3). Note there is no connection with the shell builtin of + the same name. + + Arguments and options may be intermingled in any order. Non-option + arguments are file descriptors, which must be decimal integers. By + default, file descriptors are to be tested for reading, i.e. + zselect will return when data is available to be read from the + file descriptor, or more precisely, when a read operation from the + file descriptor will not block. After a -r, -w and -e, the given + file descriptors are to be tested for reading, writing, or error + conditions. These options and an arbitrary list of file + descriptors may be given in any order. + + (The presence of an `error condition' is not well defined in the + documentation for many implementations of the select system call. + According to recent versions of the POSIX specification, it is + really an _exception_ condition, of which the only standard + example is out-of-band data received on a socket. So zsh users + are unlikely to find the -e option useful.) + + The option `-t TIMEOUT' specifies a timeout in hundredths of a + second. This may be zero, in which case the file descriptors will + simply be polled and zselect will return immediately. It is + possible to call zselect with no file descriptors and a non-zero + timeout for use as a finer-grained replacement for `sleep'; note, + however, the return status is always 1 for a timeout. + + The option `-a ARRAY' indicates that ARRAY should be set to + indicate the file descriptor(s) which are ready. If the option is + not given, the array reply will be used for this purpose. The + array will contain a string similar to the arguments for zselect. + For example, + + + zselect -t 0 -r 0 -w 1 + + might return immediately with status 0 and $reply containing `-r 0 + -w 1' to show that both file descriptors are ready for the + requested operations. + + The option `-A ASSOC' indicates that the associative array ASSOC + should be set to indicate the file descriptor(s) which are ready. + This option overrides the option -a, nor will reply be modified. + The keys of assoc are the file descriptors, and the corresponding + values are any of the characters `rwe' to indicate the condition. + + The command returns status 0 if some file descriptors are ready for + reading. If the operation timed out, or a timeout of 0 was given + and no file descriptors were ready, or there was an error, it + returns status 1 and the array will not be set (nor modified in + any way). If there was an error in the select operation the + appropriate error message is printed. + + + +File: zsh.info, Node: The zsh/zutil Module, Prev: The zsh/zselect Module, Up: Zsh Modules + +22.37 The zsh/zutil Module +========================== + + + + The zsh/zutil module only adds some builtins: + + +zstyle [ -L [ METAPATTERN [ STYLE ] ] ] +zstyle [ -e | - | -- ] PATTERN STYLE STRING ... +zstyle -d [ PATTERN [ STYLE ... ] ] +zstyle -g NAME [ PATTERN [ STYLE ] ] +zstyle -{a|b|s} CONTEXT STYLE NAME [ SEP ] +zstyle -{T|t} CONTEXT STYLE [ STRING ... ] +zstyle -m CONTEXT STYLE PATTERN + This builtin command is used to define and lookup styles. Styles + are pairs of names and values, where the values consist of any + number of strings. They are stored together with patterns and + lookup is done by giving a string, called the `_context_', which + is matched against the patterns. The definition stored for the + most specific pattern that matches will be returned. + + A pattern is considered to be more specific than another if it + contains more components (substrings separated by colons) or if + the patterns for the components are more specific, where simple + strings are considered to be more specific than patterns and + complex patterns are considered to be more specific than the + pattern `*'. A `*' in the pattern will match zero or more + characters in the context; colons are not treated specially in + this regard. If two patterns are equally specific, the tie is + broken in favour of the pattern that was defined first. + + _Example_ + + For example, to define your preferred form of precipitation + depending on which city you're in, you might set the following in + your zshrc: + + + zstyle ':weather:europe:*' preferred-precipitation rain + zstyle ':weather:europe:germany:* preferred-precipitation none + zstyle ':weather:europe:germany:*:munich' preferred-precipitation snow + + Then, the fictional `weather' plugin might run under the hood a + command such as + + + zstyle -s ":weather:${continent}:${country}:${county}:${city}" preferred-precipitation REPLY + + in order to retrieve your preference into the scalar variable + $REPLY. + + _Usage_ + + The forms that operate on patterns are the following. + + + zstyle [ -L [ METAPATTERN [ STYLE ] ] ] + Without arguments, lists style definitions. Styles are shown + in alphabetic order and patterns are shown in the order + zstyle will test them. + + If the -L option is given, listing is done in the form of + calls to zstyle. The optional first argument, METAPATTERN, + is a pattern which will be matched against the string + supplied as PATTERN when the style was defined. Note: this + means, for example, `zstyle -L ":completion:*"' will match + any supplied pattern beginning `:completion:', not just + ":completion:*": use ':completion:\*' to match that. The + optional second argument limits the output to a specific + STYLE (not a pattern). -L is not compatible with any other + options. + + zstyle [ - | -- | -e ] PATTERN STYLE STRING ... + Defines the given STYLE for the PATTERN with the STRINGs as + the value. If the -e option is given, the STRINGs will be + concatenated (separated by spaces) and the resulting string + will be evaluated (in the same way as it is done by the eval + builtin command) when the style is looked up. In this case + the parameter `reply' must be assigned to set the strings + returned after the evaluation. Before evaluating the value, + reply is unset, and if it is still unset after the + evaluation, the style is treated as if it were not set. + + zstyle -d [ PATTERN [ STYLE ... ] ] + Delete style definitions. Without arguments all definitions + are deleted, with a PATTERN all definitions for that pattern + are deleted and if any STYLEs are given, then only those + styles are deleted for the PATTERN. + + zstyle -g NAME [ PATTERN [ STYLE ] ] + Retrieve a style definition. The NAME is used as the name of + an array in which the results are stored. Without any further + arguments, all patterns defined are returned. With a PATTERN + the styles defined for that pattern are returned and with + both a PATTERN and a STYLE, the value strings of that + combination is returned. + + + The other forms can be used to look up or test styles for a given + context. + + + zstyle -s CONTEXT STYLE NAME [ SEP ] + The parameter NAME is set to the value of the style + interpreted as a string. If the value contains several + strings they are concatenated with spaces (or with the SEP + string if that is given) between them. + + Return 0 if the style is set, 1 otherwise. + + zstyle -b CONTEXT STYLE NAME + The value is stored in NAME as a boolean, i.e. as the string + `yes' if the value has only one string and that string is + equal to one of `yes', `true', `on', or `1'. If the value is + any other string or has more than one string, the parameter + is set to `no'. + + Return 0 if NAME is set to `yes', 1 otherwise. + + zstyle -a CONTEXT STYLE NAME + The value is stored in NAME as an array. If NAME is declared + as an associative array, the first, third, etc. strings are + used as the keys and the other strings are used as the values. + + Return 0 if the style is set, 1 otherwise. + + zstyle -t CONTEXT STYLE [ STRING ... ] + zstyle -T CONTEXT STYLE [ STRING ... ] + Test the value of a style, i.e. the -t option only returns a + status (sets $?). Without any STRING the return status is + zero if the style is defined for at least one matching + pattern, has only one string in its value, and that is equal + to one of `true', `yes', `on' or `1'. If any STRINGs are + given the status is zero if and only if at least one of the + STRINGs is equal to at least one of the strings in the value. + If the style is defined but doesn't match, the return status + is 1. If the style is not defined, the status is 2. + + The -T option tests the values of the style like -t, but it + returns status zero (rather than 2) if the style is not + defined for any matching pattern. + + zstyle -m CONTEXT STYLE PATTERN + Match a value. Returns status zero if the PATTERN matches at + least one of the strings in the value. + + +zformat -f PARAM FORMAT SPEC ... +zformat -a ARRAY SEP SPEC ... + This builtin provides two different forms of formatting. The first + form is selected with the -f option. In this case the FORMAT + string will be modified by replacing sequences starting with a + percent sign in it with strings from the SPECs. Each SPEC should + be of the form `CHAR:STRING' which will cause every appearance of + the sequence `%CHAR' in FORMAT to be replaced by the STRING. The + `%' sequence may also contain optional minimum and maximum field + width specifications between the `%' and the `CHAR' in the form + `%MIN.MAXc', i.e. the minimum field width is given first and if + the maximum field width is used, it has to be preceded by a dot. + Specifying a minimum field width makes the result be padded with + spaces to the right if the STRING is shorter than the requested + width. Padding to the left can be achieved by giving a negative + minimum field width. If a maximum field width is specified, the + STRING will be truncated after that many characters. After all + `%' sequences for the given SPECs have been processed, the + resulting string is stored in the parameter PARAM. + + The %-escapes also understand ternary expressions in the form used + by prompts. The % is followed by a `(' and then an ordinary + format specifier character as described above. There may be a set + of digits either before or after the `('; these specify a test + number, which defaults to zero. Negative numbers are also + allowed. An arbitrary delimiter character follows the format + specifier, which is followed by a piece of `true' text, the + delimiter character again, a piece of `false' text, and a closing + parenthesis. The complete expression (without the digits) thus + looks like `%(X.TEXT1.TEXT2)', except that the `.' character is + arbitrary. The value given for the format specifier in the + CHAR:STRING expressions is evaluated as a mathematical expression, + and compared with the test number. If they are the same, TEXT1 is + output, else TEXT2 is output. A parenthesis may be escaped in + TEXT2 as %). Either of TEXT1 or TEXT2 may contain nested + %-escapes. + + For example: + + + zformat -f REPLY "The answer is '%3(c.yes.no)'." c:3 + + outputs "The answer is 'yes'." to REPLY since the value for the + format specifier c is 3, agreeing with the digit argument to the + ternary expression. + + The second form, using the -a option, can be used for aligning + strings. Here, the SPECs are of the form `LEFT:RIGHT' where + `LEFT' and `RIGHT' are arbitrary strings. These strings are + modified by replacing the colons by the SEP string and padding the + LEFT strings with spaces to the right so that the SEP strings in + the result (and hence the RIGHT strings after them) are all + aligned if the strings are printed below each other. All strings + without a colon are left unchanged and all strings with an empty + RIGHT string have the trailing colon removed. In both cases the + lengths of the strings are not used to determine how the other + strings are to be aligned. A colon in the LEFT string can be + escaped with a backslash. The resulting strings are stored in the + ARRAY. + +zregexparse + This implements some internals of the _regex_arguments function. + +zparseopts [ -D -E -F -K -M ] [ -a ARRAY ] [ -A ASSOC ] [ - ] SPEC ... + This builtin simplifies the parsing of options in positional + parameters, i.e. the set of arguments given by $*. Each SPEC + describes one option and must be of the form `OPT[=ARRAY]'. If an + option described by OPT is found in the positional parameters it + is copied into the ARRAY specified with the -a option; if the + optional `=ARRAY' is given, it is instead copied into that array, + which should be declared as a normal array and never as an + associative array. + + Note that it is an error to give any SPEC without an `=ARRAY' + unless one of the -a or -A options is used. + + Unless the -E option is given, parsing stops at the first string + that isn't described by one of the SPECs. Even with -E, parsing + always stops at a positional parameter equal to `-' or `--'. See + also -F. + + The OPT description must be one of the following. Any of the + special characters can appear in the option name provided it is + preceded by a backslash. + + + NAME + NAME+ + The NAME is the name of the option without the leading `-'. + To specify a GNU-style long option, one of the usual two + leading `-' must be included in NAME; for example, a `--file' + option is represented by a NAME of `-file'. + + If a `+' appears after NAME, the option is appended to ARRAY + each time it is found in the positional parameters; without + the `+' only the _last_ occurrence of the option is preserved. + + If one of these forms is used, the option takes no argument, + so parsing stops if the next positional parameter does not + also begin with `-' (unless the -E option is used). + + NAME: + NAME:- + NAME:: + If one or two colons are given, the option takes an argument; + with one colon, the argument is mandatory and with two colons + it is optional. The argument is appended to the ARRAY after + the option itself. + + An optional argument is put into the same array element as + the option name (note that this makes empty strings as + arguments indistinguishable). A mandatory argument is added + as a separate element unless the `:-' form is used, in which + case the argument is put into the same element. + + A `+' as described above may appear between the NAME and the + first colon. + + + In all cases, option-arguments must appear either immediately + following the option in the same positional parameter or in the + next one. Even an optional argument may appear in the next + parameter, unless it begins with a `-'. There is no special + handling of `=' as with GNU-style argument parsers; given the SPEC + `-foo:', the positional parameter `--foo=bar' is parsed as `--foo' + with an argument of `=bar'. + + When the names of two options that take no arguments overlap, the + longest one wins, so that parsing for the SPECs `-foo -foobar' + (for example) is unambiguous. However, due to the aforementioned + handling of option-arguments, ambiguities may arise when at least + one overlapping SPEC takes an argument, as in `-foo: -foobar'. In + that case, the last matching SPEC wins. + + The options of zparseopts itself cannot be stacked because, for + example, the stack `-DEK' is indistinguishable from a SPEC for the + GNU-style long option `--DEK'. The options of zparseopts itself + are: + + + -a ARRAY + As described above, this names the default array in which to + store the recognised options. + + -A ASSOC + If this is given, the options and their values are also put + into an associative array with the option names as keys and + the arguments (if any) as the values. + + -D + If this option is given, all options found are removed from + the positional parameters of the calling shell or shell + function, up to but not including any not described by the + SPECs. If the first such parameter is `-' or `--', it is + removed as well. This is similar to using the shift builtin. + + -E + This changes the parsing rules to _not_ stop at the first + string that isn't described by one of the SPECs. It can be + used to test for or (if used together with -D) extract + options and their arguments, ignoring all other options and + arguments that may be in the positional parameters. As + indicated above, parsing still stops at the first `-' or `--' + not described by a SPEC, but it is not removed when used with + -D. + + -F + If this option is given, zparseopts immediately stops at the + first option-like parameter not described by one of the + SPECs, prints an error message, and returns status 1. + Removal (-D) and extraction (-E) are not performed, and + option arrays are not updated. This provides basic + validation for the given options. + + Note that the appearance in the positional parameters of an + option without its required argument always aborts parsing + and returns an error as described above regardless of whether + this option is used. + + -K + With this option, the arrays specified with the -a option and + with the `=ARRAY' forms are kept unchanged when none of the + SPECs for them is used. Otherwise the entire array is + replaced when any of the SPECs is used. Individual elements + of associative arrays specified with the -A option are + preserved by -K. This allows assignment of default values to + arrays before calling zparseopts. + + -M + This changes the assignment rules to implement a map among + equivalent option names. If any SPEC uses the `=ARRAY' form, + the string ARRAY is interpreted as the name of another SPEC, + which is used to choose where to store the values. If no + other SPEC is found, the values are stored as usual. This + changes only the way the values are stored, not the way $* is + parsed, so results may be unpredictable if the `NAME+' + specifier is used inconsistently. + + + For example, + + + set -- -a -bx -c y -cz baz -cend + zparseopts a=foo b:=bar c+:=bar + + will have the effect of + + + foo=(-a) + bar=(-b x -c y -c z) + + The arguments from `baz' on will not be used. + + As an example for the -E option, consider: + + + set -- -a x -b y -c z arg1 arg2 + zparseopts -E -D b:=bar + + will have the effect of + + + bar=(-b y) + set -- -a x -c z arg1 arg2 + + I.e., the option -b and its arguments are taken from the + positional parameters and put into the array bar. + + The -M option can be used like this: + + + set -- -a -bx -c y -cz baz -cend + zparseopts -A bar -M a=foo b+: c:=b + + to have the effect of + + + foo=(-a) + bar=(-a '' -b xyz) + + + +File: zsh.info, Node: Calendar Function System, Next: TCP Function System, Prev: Zsh Modules, Up: Top + +23 Calendar Function System +*************************** + + + +23.1 Description +================ + +The shell is supplied with a series of functions to replace and enhance +the traditional Unix calendar programme, which warns the user of +imminent or future events, details of which are stored in a text file +(typically calendar in the user's home directory). The version +provided here includes a mechanism for alerting the user when an event +is due. + +In addition functions age, before and after are provided that can be +used in a glob qualifier; they allow files to be selected based on +their modification times. + +The format of the calendar file and the dates used there in and in the +age function are described first, then the functions that can be called +to examine and modify the calendar file. + +The functions here depend on the availability of the zsh/datetime +module which is usually installed with the shell. The library function +strptime() must be available; it is present on most recent operating +systems. + + + +* Menu: + +* Calendar File and Date Formats:: +* Calendar System User Functions:: +* Calendar Styles:: +* Calendar Utility Functions:: +* Calendar Bugs:: + + + +File: zsh.info, Node: Calendar File and Date Formats, Next: Calendar System User Functions, Up: Calendar Function System + +23.2 File and Date Formats +========================== + + + +23.2.1 Calendar File Format +--------------------------- + +The calendar file is by default ~/calendar. This can be configured by +the calendar-file style, see *Note Calendar Styles::. The basic format +consists of a series of separate lines, with no indentation, each +including a date and time specification followed by a description of +the event. + +Various enhancements to this format are supported, based on the syntax +of Emacs calendar mode. An indented line indicates a continuation line +that continues the description of the event from the preceding line +(note the date may not be continued in this way). An initial ampersand +(&) is ignored for compatibility. + +An indented line on which the first non-whitespace character is # is +not displayed with the calendar entry, but is still scanned for +information. This can be used to hide information useful to the +calendar system but not to the user, such as the unique identifier used +by calendar_add. + +The Emacs extension that a date with no description may refer to a +number of succeeding events at different times is not supported. + +Unless the done-file style has been altered, any events which have been +processed are appended to the file with the same name as the calendar +file with the suffix .done, hence ~/calendar.done by default. + +An example is shown below. + + + +23.2.2 Date Format +------------------ + +The format of the date and time is designed to allow flexibility without +admitting ambiguity. (The words `date' and `time' are both used in the +documentation below; except where specifically noted this implies a +string that may include both a date and a time specification.) Note +that there is no localization support; month and day names must be in +English and separator characters are fixed. Matching is case +insensitive, and only the first three letters of the names are +significant, although as a special case a form beginning "month" does +not match "Monday". Furthermore, time zones are not handled; all times +are assumed to be local. + +It is recommended that, rather than exploring the intricacies of the +system, users find a date format that is natural to them and stick to +it. This will avoid unexpected effects. Various key facts should be +noted. + + + * In particular, note the confusion between MONTH/DAY/YEAR and + DAY/MONTH/YEAR when the month is numeric; these formats should be + avoided if at all possible. Many alternatives are available. + + * The year must be given in full to avoid confusion, and only years + from 1900 to 2099 inclusive are matched. + +The following give some obvious examples; users finding here a format +they like and not subject to vagaries of style may skip the full +description. As dates and times are matched separately (even though +the time may be embedded in the date), any date format may be mixed +with any format for the time of day provide the separators are clear +(whitespace, colons, commas). + + + 2007/04/03 13:13 + 2007/04/03:13:13 + 2007/04/03 1:13 pm + 3rd April 2007, 13:13 + April 3rd 2007 1:13 p.m. + Apr 3, 2007 13:13 + Tue Apr 03 13:13:00 2007 + 13:13 2007/apr/3 + +More detailed rules follow. + +Times are parsed and extracted before dates. They must use colons to +separate hours and minutes, though a dot is allowed before seconds if +they are present. This limits time formats to the following: + + + * HH:MM[:SS[.FFFFF]] [am|pm|a.m.|p.m.] + + * HH:MM.SS[.FFFFF] [am|pm|a.m.|p.m.] + +Here, square brackets indicate optional elements, possibly with +alternatives. Fractions of a second are recognised but ignored. For +absolute times (the normal format require by the calendar file and the +age, before and after functions) a date is mandatory but a time of day +is not; the time returned is at the start of the date. One variation +is allowed: if a.m. or p.m. or one of their variants is present, an +hour without a minute is allowed, e.g. 3 p.m.. + +Time zones are not handled, though if one is matched following a time +specification it will be removed to allow a surrounding date to be +parsed. This only happens if the format of the timezone is not too +unusual. The following are examples of forms that are understood: + + + +0100 + GMT + GMT-7 + CET+1CDT + +Any part of the timezone that is not numeric must have exactly three +capital letters in the name. + +Dates suffer from the ambiguity between DD/MM/YYYY and MM/DD/YYYY. It +is recommended this form is avoided with purely numeric dates, but use +of ordinals, eg. 3rd/04/2007, will resolve the ambiguity as the ordinal +is always parsed as the day of the month. Years must be four digits +(and the first two must be 19 or 20); 03/04/08 is not recognised. Other +numbers may have leading zeroes, but they are not required. The +following are handled: + + + * YYYY/MM/DD + + * YYYY-MM-DD + + * YYYY/MNM/DD + + * YYYY-MNM-DD + + * DD[th|st|rd] MNM[,] [ YYYY ] + + * MNM DD[th|st|rd][,] [ YYYY ] + + * DD[th|st|rd]/MM[,] YYYY + + * DD[th|st|rd]/MM/YYYY + + * MM/DD[th|st|rd][,] YYYY + + * MM/DD[th|st|rd]/YYYY + +Here, MNM is at least the first three letters of a month name, matched +case-insensitively. The remainder of the month name may appear but its +contents are irrelevant, so janissary, febrile, martial, apricot, +maybe, junta, etc. are happily handled. + +Where the year is shown as optional, the current year is assumed. There +are only two such cases, the form Jun 20 or 14 September (the only two +commonly occurring forms, apart from a "the" in some forms of English, +which isn't currently supported). Such dates will of course become +ambiguous in the future, so should ideally be avoided. + +Times may follow dates with a colon, e.g. 1965/07/12:09:45; this is in +order to provide a format with no whitespace. A comma and whitespace +are allowed, e.g. 1965/07/12, 09:45. Currently the order of these +separators is not checked, so illogical formats such as 1965/07/12, : +,09:45 will also be matched. For simplicity such variations are not +shown in the list above. Otherwise, a time is only recognised as being +associated with a date if there is only whitespace in between, or if the +time was embedded in the date. + +Days of the week are not normally scanned, but will be ignored if they +occur at the start of the date pattern only. However, in contexts +where it is useful to specify dates relative to today, days of the week +with no other date specification may be given. The day is assumed to +be either today or within the past week. Likewise, the words yesterday, +today and tomorrow are handled. All matches are case-insensitive. +Hence if today is Monday, then Sunday is equivalent to yesterday, +Monday is equivalent to today, but Tuesday gives a date six days ago. +This is not generally useful within the calendar file. Dates in this +format may be combined with a time specification; for example Tomorrow, +8 p.m.. + +For example, the standard date format: + + + Fri Aug 18 17:00:48 BST 2006 + +is handled by matching HH:MM:SS and removing it together with the +matched (but unused) time zone. This leaves the following: + + + Fri Aug 18 2006 + +Fri is ignored and the rest is matched according to the standard rules. + + + +23.2.3 Relative Time Format +--------------------------- + +In certain places relative times are handled. Here, a date is not +allowed; instead a combination of various supported periods are +allowed, together with an optional time. The periods must be in order +from most to least significant. + +In some cases, a more accurate calculation is possible when there is an +anchor date: offsets of months or years pick the correct day, rather +than being rounded, and it is possible to pick a particular day in a +month as `(1st Friday)', etc., as described in more detail below. + +Anchors are available in the following cases. If one or two times are +passed to the function calendar, the start time acts an anchor for the +end time when the end time is relative (even if the start time is +implicit). When examining calendar files, the scheduled event being +examined anchors the warning time when it is given explicitly by means +of the WARN keyword; likewise, the scheduled event anchors a repetition +period when given by the RPT keyword, so that specifications such as +RPT 2 months, 3rd Thursday are handled properly. Finally, the -R +argument to calendar_scandate directly provides an anchor for relative +calculations. + +The periods handled, with possible abbreviations are: + + +Years + years, yrs, ys, year, yr, y, yearly. A year is 365.25 days unless + there is an anchor. + +Months + months, mons, mnths, mths, month, mon, mnth, mth, monthly. Note + that m, ms, mn, mns are ambiguous and are _not_ handled. A month + is a period of 30 days rather than a calendar month unless there + is an anchor. + +Weeks + weeks, wks, ws, week, wk, w, weekly + +Days + days, dys, ds, day, dy, d, daily + +Hours + hours, hrs, hs, hour, hr, h, hourly + +Minutes + minutes, mins, minute, min, but _not_ m, ms, mn or mns + +Seconds + seconds, secs, ss, second, sec, s + + +Spaces between the numbers are optional, but are required between items, +although a comma may be used (with or without spaces). + +The forms yearly to hourly allow the number to be omitted; it is +assumed to be 1. For example, 1 d and daily are equivalent. Note that +using those forms with plurals is confusing; 2 yearly is the same as 2 +years, _not_ twice yearly, so it is recommended they only be used +without numbers. + +When an anchor time is present, there is an extension to handle regular +events in the form of the Nth SOMEday of the month. Such a +specification must occur immediately after any year and month +specification, but before any time of day, and must be in the form +N(th|st|rd) DAY, for example 1st Tuesday or 3rd Monday. As in other +places, days are matched case insensitively, must be in English, and +only the first three letters are significant except that a form +beginning `month' does not match `Monday'. No attempt is made to +sanitize the resulting date; attempts to squeeze too many occurrences +into a month will push the day into the next month (but in the obvious +fashion, retaining the correct day of the week). + +Here are some examples: + + + 30 years 3 months 4 days 3:42:41 + 14 days 5 hours + Monthly, 3rd Thursday + 4d,10hr + + +23.2.4 Example +-------------- + +Here is an example calendar file. It uses a consistent date format, as +recommended above. + + + Feb 1, 2006 14:30 Pointless bureaucratic meeting + Mar 27, 2006 11:00 Mutual recrimination and finger pointing + Bring water pistol and waterproofs + Mar 31, 2006 14:00 Very serious managerial pontification + # UID 12C7878A9A50 + Apr 10, 2006 13:30 Even more pointless blame assignment exercise WARN 30 mins + May 18, 2006 16:00 Regular moaning session RPT monthly, 3rd Thursday + +The second entry has a continuation line. The third entry has a +continuation line that will not be shown when the entry is displayed, +but the unique identifier will be used by the calendar_add function when +updating the event. The fourth entry will produce a warning 30 minutes +before the event (to allow you to equip yourself appropriately). The +fifth entry repeats after a month on the 3rd Thursday, i.e. June 15, +2006, at the same time. + + + diff --git a/assets/info/zsh.info-5 b/assets/info/zsh.info-5 new file mode 100644 index 00000000..5720d6b2 --- /dev/null +++ b/assets/info/zsh.info-5 @@ -0,0 +1,7230 @@ +This is zsh.info, produced by makeinfo version 4.8 from ./zsh.texi. + +INFO-DIR-SECTION Utilities +START-INFO-DIR-ENTRY +* ZSH: (zsh). The Z Shell Manual. +END-INFO-DIR-ENTRY + + +File: zsh.info, Node: Calendar System User Functions, Next: Calendar Styles, Prev: Calendar File and Date Formats, Up: Calendar Function System + +23.3 User Functions +=================== + +This section describes functions that are designed to be called +directly by the user. The first part describes those functions +associated with the user's calendar; the second part describes the use +in glob qualifiers. + + + +23.3.1 Calendar system functions +-------------------------------- + + +calendar [ -abdDsv ] [ -C CALFILE ] [ -n NUM ] [ -S SHOWPROG ] + [ [ START ] END ] +calendar -r [ -abdDrsv ] [ -C CALFILE ] [ -n NUM ] [ -S SHOWPROG ] + [ START ] + Show events in the calendar. + + With no arguments, show events from the start of today until the + end of the next working day after today. In other words, if today + is Friday, Saturday, or Sunday, show up to the end of the + following Monday, otherwise show today and tomorrow. + + If END is given, show events from the start of today up to the time + and date given, which is in the format described in the previous + section. Note that if this is a date the time is assumed to be + midnight at the start of the date, so that effectively this shows + all events before the given date. + + END may start with a +, in which case the remainder of the + specification is a relative time format as described in the + previous section indicating the range of time from the start time + that is to be included. + + If START is also given, show events starting from that time and + date. The word now can be used to indicate the current time. + + To implement an alert when events are due, include calendar -s in + your ~/.zshrc file. + + Options: + + + -a + Show all items in the calendar, regardless of the start and + end. + + -b + Brief: don't display continuation lines (i.e. indented lines + following the line with the date/time), just the first line. + + -B LINES + Brief: display at most the first LINES lines of the calendar + entry. `-B 1' is equivalent to `-b'. + + -C CALFILE + Explicitly specify a calendar file instead of the value of + the calendar-file style or the default ~/calendar. + + -d + Move any events that have passed from the calendar file to the + "done" file, as given by the done-file style or the default + which is the calendar file with .done appended. This option + is implied by the -s option. + + -D + Turns off the option -d, even if the -s option is also + present. + + -n NUM, -NUM + Show at least NUM events, if present in the calendar file, + regardless of the start and end. + + -r + Show all the remaining options in the calendar, ignoring the + given END time. The START time is respected; any argument + given is treated as a START time. + + -s + Use the shell's sched command to schedule a timed event that + will warn the user when an event is due. Note that the sched + command only runs if the shell is at an interactive prompt; a + foreground task blocks the scheduled task from running until + it is finished. + + The timed event usually runs the programme calendar_show to + show the event, as described in *Note Calendar Utility + Functions::. + + By default, a warning of the event is shown five minutes + before it is due. The warning period can be configured by + the style warn-time or for a single calendar entry by + including WARN RELTIME in the first line of the entry, where + RELTIME is one of the usual relative time formats. + + A repeated event may be indicated by including RPT RELDATE in + the first line of the entry. After the scheduled event has + been displayed it will be re-entered into the calendar file + at a time RELDATE after the existing event. Note that this + is currently the only use made of the repeat count, so that + it is not possible to query the schedule for a recurrence of + an event in the calendar until the previous event has passed. + + If RPT is used, it is also possible to specify that certain + recurrences of an event are rescheduled or cancelled. This is + done with the OCCURRENCE keyword, followed by whitespace and + the date and time of the occurrence in the regular sequence, + followed by whitespace and either the date and time of the + rescheduled event or the exact string CANCELLED. In this + case the date and time must be in exactly the "date with + local time" format used by the text/calendar MIME type (RFC + 2445),
    T (note the presence of the + literal character T). The first word (the regular + recurrence) may be something other than a proper date/time to + indicate that the event is additional to the normal sequence; + a convention that retains the formatting appearance is + XXXXXXXXTXXXXXX. + + Furthermore, it is useful to record the next regular + recurrence (as then the displayed date may be for a + rescheduled event so cannot be used for calculating the + regular sequence). This is specified by RECURRENCE and a + time or date in the same format. calendar_add adds such an + indication when it encounters a recurring event that does not + include one, based on the headline date/time. + + If calendar_add is used to update occurrences the UID keyword + described there should be present in both the existing entry + and the added occurrence in order to identify recurring event + sequences. + + For example, + + + Thu May 6, 2010 11:00 Informal chat RPT 1 week + # RECURRENCE 20100506T110000 + # OCCURRENCE 20100513T110000 20100513T120000 + # OCCURRENCE 20100520T110000 CANCELLED + + The event that occurs at 11:00 on 13th May 2010 is + rescheduled an hour later. The event that occurs a week + later is cancelled. The occurrences are given on a + continuation line starting with a # character so will not + usually be displayed as part of the event. As elsewhere, no + account of time zones is taken with the times. After the next + event occurs the headline date/time will be `Thu May 13, 2010 + 12:00' while the RECURRENCE date/time will be + `20100513T110000' (note that cancelled and moved events are + not taken account of in the RECURRENCE, which records what + the next regular recurrence is, but they are accounted for in + the headline date/time). + + It is safe to run calendar -s to reschedule an existing event + (if the calendar file has changed, for example), and also to + have it running in multiples instances of the shell since the + calendar file is locked when in use. + + By default, expired events are moved to the "done" file; see + the -d option. Use -D to prevent this. + + -S SHOWPROG + Explicitly specify a programme to be used for showing events + instead of the value of the show-prog style or the default + calendar_show. + + -v + Verbose: show more information about stages of processing. + This is useful for confirming that the function has + successfully parsed the dates in the calendar file. + + +calendar_add [ -BL ] EVENT ... + Adds a single event to the calendar in the appropriate location. + The event can contain multiple lines, as described in *Note + Calendar File and Date Formats::. Using this function ensures + that the calendar file is sorted in date and time order. It also + makes special arrangements for locking the file while it is + altered. The old calendar is left in a file with the suffix .old. + + The option -B indicates that backing up the calendar file will be + handled by the caller and should not be performed by calendar_add. + The option -L indicates that calendar_add does not need to lock + the calendar file as it is already locked. These options will not + usually be needed by users. + + If the style reformat-date is true, the date and time of the new + entry will be rewritten into the standard date format: see the + descriptions of this style and the style date-format. + + The function can use a unique identifier stored with each event to + ensure that updates to existing events are treated correctly. The + entry should contain the word UID, followed by whitespace, + followed by a word consisting entirely of hexadecimal digits of + arbitrary length (all digits are significant, including leading + zeroes). As the UID is not directly useful to the user, it is + convenient to hide it on an indented continuation line starting + with a #, for example: + + + Aug 31, 2007 09:30 Celebrate the end of the holidays + # UID 045B78A0 + + The second line will not be shown by the calendar function. + + It is possible to specify the RPT keyword followed by CANCELLED + instead of a relative time. This causes any matched event or + series of events to be cancelled (the original event does not have + to be marked as recurring in order to be cancelled by this + method). A UID is required in order to match an existing event in + the calendar. + + calendar_add will attempt to manage recurrences and occurrences of + repeating events as described for event scheduling by calendar -s + above. To reschedule or cancel a single event calendar_add should + be called with an entry that includes the correct UID but does + _not_ include the RPT keyword as this is taken to mean the entry + applies to a series of repeating events and hence replaces all + existing information. Each rescheduled or cancelled occurrence + must have an OCCURRENCE keyword in the entry passed to + calendar_add which will be merged into the calendar file. Any + existing reference to the occurrence is replaced. An occurrence + that does not refer to a valid existing event is added as a + one-off occurrence to the same calendar entry. + +calendar_edit + This calls the user's editor to edit the calendar file. If there + are arguments, they are taken as the editor to use (the file name + is appended to the commands); otherwise, the editor is given by the + variable VISUAL, if set, else the variable EDITOR. + + If the calendar scheduler was running, then after editing the file + calendar -s is called to update it. + + This function locks out the calendar system during the edit. + Hence it should be used to edit the calendar file if there is any + possibility of a calendar event occurring meanwhile. Note this + can lead to another shell with calendar functions enabled hanging + waiting for a lock, so it is necessary to quit the editor as soon + as possible. + +calendar_parse CALENDAR-ENTRY + This is the internal function that analyses the parts of a calendar + entry, which is passed as the only argument. The function returns + status 1 if the argument could not be parsed as a calendar entry + and status 2 if the wrong number of arguments were passed; it also + sets the parameter reply to an empty associative array. Otherwise, + it returns status 0 and sets elements of the associative array + reply as follows: + + + time + The time as a string of digits in the same units as + $EPOCHSECONDS + + schedtime + The regularly scheduled time. This may differ from the + actual event time time if this is a recurring event and the + next occurrence has been rescheduled. Then time gives the + actual time and schedtime the time of the regular recurrence + before modification. + + text1 + The text from the line not including the date and time of the + event, but including any WARN or RPT keywords and values. + + warntime + Any warning time given by the WARN keyword as a string of + digits containing the time at which to warn in the same units + as $EPOCHSECONDS. (Note this is an absolute time, not the + relative time passed down.) Not set no WARN keyword and + value were matched. + + warnstr + The raw string matched after the WARN keyword, else unset. + + rpttime + Any recurrence time given by the RPT keyword as a string of + digits containing the time of the recurrence in the same units + as $EPOCHSECONDS. (Note this is an absolute time.) Not set + if no RPT keyword and value were matched. + + schedrpttime + The next regularly scheduled occurrence of a recurring event + before modification. This may differ from rpttime, which is + the actual time of the event that may have been rescheduled + from the regular time. + + rptstr + The raw string matched after the RPT keyword, else unset. + + text2 + The text from the line after removal of the date and any + keywords and values. + + + +calendar_showdate [ -r ] [ -f FMT ] DATE-SPEC ... + The given DATE-SPEC is interpreted and the corresponding date and + time printed. If the initial DATE-SPEC begins with a + or - it is + treated as relative to the current time; DATE-SPECs after the + first are treated as relative to the date calculated so far and a + leading + is optional in that case. This allows one to use the + system as a date calculator. For example, calendar_showdate '+1 + month, 1st Friday' shows the date of the first Friday of next + month. + + With the option -r nothing is printed but the value of the date and + time in seconds since the epoch is stored in the parameter REPLY. + + With the option -f FMT the given date/time conversion format is + passed to strftime; see notes on the date-format style below. + + In order to avoid ambiguity with negative relative date + specifications, options must occur in separate words; in other + words, -r and -f should not be combined in the same word. + +calendar_sort + Sorts the calendar file into date and time order. The old + calendar is left in a file with the suffix .old. + + + +23.3.2 Glob qualifiers +---------------------- + + +age + The function age can be autoloaded and use separately from the + calendar system, although it uses the function calendar_scandate + for date formatting. It requires the zsh/stat builtin, but uses + only the builtin zstat. + + age selects files having a given modification time for use as a + glob qualifier. The format of the date is the same as that + understood by the calendar system, described in *Note Calendar + File and Date Formats::. + + The function can take one or two arguments, which can be supplied + either directly as command or arguments, or separately as shell + parameters. + + + print *(e:age 2006/10/04 2006/10/09:) + + The example above matches all files modified between the start of + those dates. The second argument may alternatively be a relative + time introduced by a +: + + + print *(e:age 2006/10/04 +5d:) + + The example above is equivalent to the previous example. + + In addition to the special use of days of the week, today and + yesterday, times with no date may be specified; these apply to + today. Obviously such uses become problematic around midnight. + + + print *(e-age 12:00 13:30-) + + The example above shows files modified between 12:00 and 13:00 + today. + + + print *(e:age 2006/10/04:) + + The example above matches all files modified on that date. If the + second argument is omitted it is taken to be exactly 24 hours + after the first argument (even if the first argument contains a + time). + + + print *(e-age 2006/10/04:10:15 2006/10/04:10:45-) + + The example above supplies times. Note that whitespace within the + time and date specification must be quoted to ensure age receives + the correct arguments, hence the use of the additional colon to + separate the date and time. + + + AGEREF=2006/10/04:10:15 + AGEREF2=2006/10/04:10:45 + print *(+age) + + This shows the same example before using another form of argument + passing. The dates and times in the parameters AGEREF and AGEREF2 + stay in effect until unset, but will be overridden if any argument + is passed as an explicit argument to age. Any explicit argument + causes both parameters to be ignored. + + Instead of an explicit date and time, it's possible to use the + modification time of a file as the date and time for either + argument by introducing the file name with a colon: + + + print *(e-age :file1-) + + matches all files created on the same day (24 hours starting from + midnight) as file1. + + + print *(e-age :file1 :file2-) + + matches all files modified no earlier than file1 and no later than + file2; precision here is to the nearest second. + +after +before + The functions after and before are simpler versions of age that + take just one argument. The argument is parsed similarly to an + argument of age; if it is not given the variable AGEREF is + consulted. As the names of the functions suggest, a file matches + if its modification time is after or before the time and date + specified. If a time only is given the date is today. + + The two following examples are therefore equivalent: + print *(e-after 12:00-) + print *(e-after today:12:00-) + + + + +File: zsh.info, Node: Calendar Styles, Next: Calendar Utility Functions, Prev: Calendar System User Functions, Up: Calendar Function System + +23.4 Styles +=========== + +The zsh style mechanism using the zstyle command is describe in *Note +The zsh/zutil Module::. This is the same mechanism used in the +completion system. + +The styles below are all examined in the context :datetime:FUNCTION:, +for example :datetime:calendar:. + + +calendar-file + The location of the main calendar. The default is ~/calendar. + +date-format + A strftime format string (see man page strftime(3)) with the zsh + extensions providing various numbers with no leading zero or space + if the number is a single digit as described for the %D{STRING} + prompt format in *Note Prompt Expansion::. + + This is used for outputting dates in calendar, both to support the + -v option and when adding recurring events back to the calendar + file, and in calendar_showdate as the final output format. + + If the style is not set, the default used is similar the standard + system format as output by the date command (also known as `ctime + format'): `%a %b %d %H:%M:%S %Z %Y'. + +done-file + The location of the file to which events which have passed are + appended. The default is the calendar file location with the + suffix .done. The style may be set to an empty string in which + case a "done" file will not be maintained. + +reformat-date + Boolean, used by calendar_add. If it is true, the date and time + of new entries added to the calendar will be reformatted to the + format given by the style date-format or its default. Only the + date and time of the event itself is reformatted; any subsidiary + dates and times such as those associated with repeat and warning + times are left alone. + +show-prog + The programme run by calendar for showing events. It will be + passed the start time and stop time of the events requested in + seconds since the epoch followed by the event text. Note that + calendar -s uses a start time and stop time equal to one another + to indicate alerts for specific events. + + The default is the function calendar_show. + +warn-time + The time before an event at which a warning will be displayed, if + the first line of the event does not include the text EVENT + RELTIME. The default is 5 minutes. + + + + +File: zsh.info, Node: Calendar Utility Functions, Next: Calendar Bugs, Prev: Calendar Styles, Up: Calendar Function System + +23.5 Utility functions +====================== + + +calendar_lockfiles + Attempt to lock the files given in the argument. To prevent + problems with network file locking this is done in an ad hoc + fashion by attempting to create a symbolic link to the file with + the name FILE.lockfile. No other system level functions are used + for locking, i.e. the file can be accessed and modified by any + utility that does not use this mechanism. In particular, the user + is not prevented from editing the calendar file at the same time + unless calendar_edit is used. + + Three attempts are made to lock the file before giving up. If the + module zsh/zselect is available, the times of the attempts are + jittered so that multiple instances of the calling function are + unlikely to retry at the same time. + + The files locked are appended to the array lockfiles, which should + be local to the caller. + + If all files were successfully locked, status zero is returned, + else status one. + + This function may be used as a general file locking function, + although this will only work if only this mechanism is used to + lock files. + +calendar_read + This is a backend used by various other functions to parse the + calendar file, which is passed as the only argument. The array + calendar_entries is set to the list of events in the file; no + pruning is done except that ampersands are removed from the start + of the line. Each entry may contain multiple lines. + +calendar_scandate + This is a generic function to parse dates and times that may be + used separately from the calendar system. The argument is a date + or time specification as described in *Note Calendar File and Date + Formats::. The parameter REPLY is set to the number of seconds + since the epoch corresponding to that date or time. By default, + the date and time may occur anywhere within the given argument. + + Returns status zero if the date and time were successfully parsed, + else one. + + Options: + -a + The date and time are anchored to the start of the argument; + they will not be matched if there is preceding text. + + -A + The date and time are anchored to both the start and end of + the argument; they will not be matched if the is any other + text in the argument. + + -d + Enable additional debugging output. + + -m + Minus. When -R ANCHOR_TIME is also given the relative time is + calculated backwards from ANCHOR_TIME. + + -r + The argument passed is to be parsed as a relative time. + + -R ANCHOR_TIME + The argument passed is to be parsed as a relative time. The + time is relative to ANCHOR_TIME, a time in seconds since the + epoch, and the returned value is the absolute time + corresponding to advancing ANCHOR_TIME by the relative time + given. This allows lengths of months to be correctly taken + into account. If the final day does not exist in the given + month, the last day of the final month is given. For + example, if the anchor time is during 31st January 2007 and + the relative time is 1 month, the final time is the same time + of day during 28th February 2007. + + -s + In addition to setting REPLY, set REPLY2 to the remainder of + the argument after the date and time have been stripped. + This is empty if the option -A was given. + + -t + Allow a time with no date specification. The date is assumed + to be today. The behaviour is unspecified if the iron tongue + of midnight is tolling twelve. + + +calendar_show + The function used by default to display events. It accepts a + start time and end time for events, both in epoch seconds, and an + event description. + + The event is always printed to standard output. If the command + line editor is active (which will usually be the case) the command + line will be redisplayed after the output. + + If the parameter DISPLAY is set and the start and end times are + the same (indicating a scheduled event), the function uses the + command xmessage to display a window with the event details. + + + + +File: zsh.info, Node: Calendar Bugs, Prev: Calendar Utility Functions, Up: Calendar Function System + +23.6 Bugs +========= + +As the system is based entirely on shell functions (with a little +support from the zsh/datetime module) the mechanisms used are not as +robust as those provided by a dedicated calendar utility. Consequently +the user should not rely on the shell for vital alerts. + +There is no calendar_delete function. + +There is no localization support for dates and times, nor any support +for the use of time zones. + +Relative periods of months and years do not take into account the +variable number of days. + +The calendar_show function is currently hardwired to use xmessage for +displaying alerts on X Window System displays. This should be +configurable and ideally integrate better with the desktop. + +calendar_lockfiles hangs the shell while waiting for a lock on a file. +If called from a scheduled task, it should instead reschedule the event +that caused it. + + +File: zsh.info, Node: TCP Function System, Next: Zftp Function System, Prev: Calendar Function System, Up: Top + +24 TCP Function System +********************** + + + +24.1 Description +================ + +A module zsh/net/tcp is provided to provide network I/O over TCP/IP +from within the shell; see its description in *Note Zsh Modules::. +This manual page describes a function suite based on the module. If +the module is installed, the functions are usually installed at the +same time, in which case they will be available for autoloading in the +default function search path. In addition to the zsh/net/tcp module, +the zsh/zselect module is used to implement timeouts on read +operations. For troubleshooting tips, consult the corresponding advice +for the zftp functions described in *Note Zftp Function System::. + +There are functions corresponding to the basic I/O operations open, +close, read and send, named tcp_open etc., as well as a function +tcp_expect for pattern match analysis of data read as input. The +system makes it easy to receive data from and send data to multiple +named sessions at once. In addition, it can be linked with the shell's +line editor in such a way that input data is automatically shown at the +terminal. Other facilities available including logging, filtering and +configurable output prompts. + +To use the system where it is available, it should be enough to +`autoload -U tcp_open' and run tcp_open as documented below to start a +session. The tcp_open function will autoload the remaining functions. + + + +* Menu: + +* TCP Functions:: +* TCP Parameters:: +* TCP Examples:: +* TCP Bugs:: + + + +File: zsh.info, Node: TCP Functions, Next: TCP Parameters, Up: TCP Function System + +24.2 TCP User Functions +======================= + + + +24.2.1 Basic I/O +---------------- + + +tcp_open [ -qz ] HOST PORT [ SESS ] +tcp_open [ -qz ] [ -s SESS | -l SESS[,...] ] ... +tcp_open [ -qz ] [ -a FD | -f FD ] [ SESS ] + Open a new session. In the first and simplest form, open a TCP + connection to host HOST at port PORT; numeric and symbolic forms + are understood for both. + + If SESS is given, this becomes the name of the session which can be + used to refer to multiple different TCP connections. If SESS is + not given, the function will invent a numeric name value (note + this is _not_ the same as the file descriptor to which the session + is attached). It is recommended that session names not include + `funny' characters, where funny characters are not well-defined + but certainly do not include alphanumerics or underscores, and + certainly do include whitespace. + + In the second case, one or more sessions to be opened are given by + name. A single session name is given after -s and a + comma-separated list after -l; both options may be repeated as + many times as necessary. A failure to open any session causes + tcp_open to abort. The host and port are read from the file + .ztcp_sessions in the same directory as the user's zsh + initialisation files, i.e. usually the home directory, but + $ZDOTDIR if that is set. The file consists of lines each giving a + session name and the corresponding host and port, in that order + (note the session name comes first, not last), separated by + whitespace. + + The third form allows passive and fake TCP connections. If the + option -a is used, its argument is a file descriptor open for + listening for connections. No function front-end is provided to + open such a file descriptor, but a call to `ztcp -l PORT' will + create one with the file descriptor stored in the parameter + $REPLY. The listening port can be closed with `ztcp -c FD'. A + call to `tcp_open -a FD' will block until a remote TCP connection + is made to PORT on the local machine. At this point, a session is + created in the usual way and is largely indistinguishable from an + active connection created with one of the first two forms. + + If the option -f is used, its argument is a file descriptor which + is used directly as if it were a TCP session. How well the + remainder of the TCP function system copes with this depends on + what actually underlies this file descriptor. A regular file is + likely to be unusable; a FIFO (pipe) of some sort will work + better, but note that it is not a good idea for two different + sessions to attempt to read from the same FIFO at once. + + If the option -q is given with any of the three forms, tcp_open + will not print informational messages, although it will in any + case exit with an appropriate status. + + If the line editor (zle) is in use, which is typically the case if + the shell is interactive, tcp_open installs a handler inside zle + which will check for new data at the same time as it checks for + keyboard input. This is convenient as the shell consumes no CPU + time while waiting; the test is performed by the operating system. + Giving the option -z to any of the forms of tcp_open prevents the + handler from being installed, so data must be read explicitly. + Note, however, this is not necessary for executing complete sets + of send and read commands from a function, as zle is not active at + this point. Generally speaking, the handler is only active when + the shell is waiting for input at a command prompt or in the vared + builtin. The option has no effect if zle is not active; `[[ -o + zle]]' will test for this. + + The first session to be opened becomes the current session and + subsequent calls to tcp_open do not change it. The current + session is stored in the parameter $TCP_SESS; see below for more + detail about the parameters used by the system. + + The function tcp_on_open, if defined, is called when a session is + opened. See the description below. + +tcp_close [ -qn ] [ -a | -l SESS[,...] | SESS ... ] + Close the named sessions, or the current session if none is given, + or all open sessions if -a is given. The options -l and -s are + both handled for consistency with tcp_open, although the latter is + redundant. + + If the session being closed is the current one, $TCP_SESS is unset, + leaving no current session, even if there are other sessions still + open. + + If the session was opened with tcp_open -f, the file descriptor is + closed so long as it is in the range 0 to 9 accessible directly + from the command line. If the option -n is given, no attempt will + be made to close file descriptors in this case. The -n option is + not used for genuine ztcp session; the file descriptors are always + closed with the session. + + If the option -q is given, no informational messages will be + printed. + +tcp_read [ -bdq ] [ -t TO ] [ -T TO ] + [ -a | -u FD[,...] | -l SESS[,...] | -s SESS ... ] + Perform a read operation on the current session, or on a list of + sessions if any are given with -u, -l or -s, or all open sessions + if the option -a is given. Any of the -u, -l or -s options may be + repeated or mixed together. The -u option specifies a file + descriptor directly (only those managed by this system are + useful), the other two specify sessions as described for tcp_open + above. + + The function checks for new data available on all the sessions + listed. Unless the -b option is given, it will not block waiting + for new data. Any one line of data from any of the available + sessions will be read, stored in the parameter $TCP_LINE, and + displayed to standard output unless $TCP_SILENT contains a + non-empty string. When printed to standard output the string + $TCP_PROMPT will be shown at the start of the line; the default + form for this includes the name of the session being read. See + below for more information on these parameters. In this mode, + tcp_read can be called repeatedly until it returns status 2 which + indicates all pending input from all specified sessions has been + handled. + + With the option -b, equivalent to an infinite timeout, the function + will block until a line is available to read from one of the + specified sessions. However, only a single line is returned. + + The option -d indicates that all pending input should be drained. + In this case tcp_read may process multiple lines in the manner + given above; only the last is stored in $TCP_LINE, but the + complete set is stored in the array $tcp_lines. This is cleared + at the start of each call to tcp_read. + + The options -t and -T specify a timeout in seconds, which may be a + floating point number for increased accuracy. With -t the timeout + is applied before each line read. With -T, the timeout applies to + the overall operation, possibly including multiple read operations + if the option -d is present; without this option, there is no + distinction between -t and -T. + + The function does not print informational messages, but if the + option -q is given, no error message is printed for a non-existent + session. + + A return status of 2 indicates a timeout or no data to read. Any + other non-zero return status indicates some error condition. + + See tcp_log for how to control where data is sent by tcp_read. + +tcp_send [ -cnq ] [ -s SESS | -l SESS[,...] ] DATA ... +tcp_send [ -cnq ] -a DATA ... + Send the supplied data strings to all the specified sessions in + turn. The underlying operation differs little from a `print -r' + to the session's file descriptor, although it attempts to prevent + the shell from dying owing to a SIGPIPE caused by an attempt to + write to a defunct session. + + The option -c causes tcp_send to behave like cat. It reads lines + from standard input until end of input and sends them in turn to + the specified session(s) exactly as if they were given as DATA + arguments to individual tcp_send commands. + + The option -n prevents tcp_send from putting a newline at the end + of the data strings. + + The remaining options all behave as for tcp_read. + + The data arguments are not further processed once they have been + passed to tcp_send; they are simply passed down to print -r. + + If the parameter $TCP_OUTPUT is a non-empty string and logging is + enabled then the data sent to each session will be echoed to the + log file(s) with $TCP_OUTPUT in front where appropriate, much in + the manner of $TCP_PROMPT. + + + +24.2.2 Session Management +------------------------- + + +tcp_alias [ -q ] ALIAS=SESS ... +tcp_alias [ -q ] [ ALIAS ... ] +tcp_alias -d [ -q ] ALIAS ... + This function is not particularly well tested. + + The first form creates an alias for a session name; ALIAS can then + be used to refer to the existing session SESS. As many aliases + may be listed as required. + + The second form lists any aliases specified, or all aliases if + none. + + The third form deletes all the aliases listed. The underlying + sessions are not affected. + + The option -q suppresses an inconsistently chosen subset of error + messages. + +tcp_log [ -asc ] [ -n | -N ] [ LOGFILE ] + With an argument LOGFILE, all future input from tcp_read will be + logged to the named file. Unless -a (append) is given, this file + will first be truncated or created empty. With no arguments, show + the current status of logging. + + With the option -s, per-session logging is enabled. Input from + tcp_read is output to the file LOGFILE.SESS. As the session is + automatically discriminated by the filename, the contents are raw + (no $TCP_PROMPT). The option -a applies as above. Per-session + logging and logging of all data in one file are not mutually + exclusive. + + The option -c closes all logging, both complete and per-session + logs. + + The options -n and -N respectively turn off or restore output of + data read by tcp_read to standard output; hence `tcp_log -cn' turns + off all output by tcp_read. + + The function is purely a convenient front end to setting the + parameters $TCP_LOG, $TCP_LOG_SESS, $TCP_SILENT, which are + described below. + +tcp_rename OLD NEW + Rename session OLD to session NEW. The old name becomes invalid. + +tcp_sess [ SESS [ COMMAND [ ARG ... ] ] ] + With no arguments, list all the open sessions and associated file + descriptors. The current session is marked with a star. For use + in functions, direct access to the parameters $tcp_by_name, + $tcp_by_fd and $TCP_SESS is probably more convenient; see below. + + With a SESS argument, set the current session to SESS. This is + equivalent to changing $TCP_SESS directly. + + With additional arguments, temporarily set the current session + while executing `COMMAND ARG ...'. COMMAND is re-evaluated so as + to expand aliases etc., but the remaining ARGs are passed through + as that appear to tcp_sess. The original session is restored when + tcp_sess exits. + + + +24.2.3 Advanced I/O +------------------- + + +tcp_command SEND-OPTION ... SEND-ARGUMENT ... + This is a convenient front-end to tcp_send. All arguments are + passed to tcp_send, then the function pauses waiting for data. + While data is arriving at least every $TCP_TIMEOUT (default 0.3) + seconds, data is handled and printed out according to the current + settings. Status 0 is always returned. + + This is generally only useful for interactive use, to prevent the + display becoming fragmented by output returned from the + connection. Within a programme or function it is generally better + to handle reading data by a more explicit method. + +tcp_expect [ -q ] [ -p VAR | -P VAR ] [ -t TO | -T TO ] + [ -a | -s SESS | -l SESS[,...] ] PATTERN ... + Wait for input matching any of the given PATTERNs from any of the + specified sessions. Input is ignored until an input line matches + one of the given patterns; at this point status zero is returned, + the matching line is stored in $TCP_LINE, and the full set of + lines read during the call to tcp_expect is stored in the array + $tcp_expect_lines. + + Sessions are specified in the same way as tcp_read: the default is + to use the current session, otherwise the sessions specified by -a, + -s, or -l are used. + + Each PATTERN is a standard zsh extended-globbing pattern; note + that it needs to be quoted to avoid it being expanded immediately + by filename generation. It must match the full line, so to match + a substring there must be a `*' at the start and end. The line + matched against includes the $TCP_PROMPT added by tcp_read. It is + possible to include the globbing flags `#b' or `#m' in the + patterns to make backreferences available in the parameters + $MATCH, $match, etc., as described in the base zsh documentation + on pattern matching. + + Unlike tcp_read, the default behaviour of tcp_expect is to block + indefinitely until the required input is found. This can be + modified by specifying a timeout with -t or -T; these function as + in tcp_read, specifying a per-read or overall timeout, + respectively, in seconds, as an integer or floating-point number. + As tcp_read, the function returns status 2 if a timeout occurs. + + The function returns as soon as any one of the patterns given + match. If the caller needs to know which of the patterns matched, + the option -p VAR can be used; on return, $var is set to the + number of the pattern using ordinary zsh indexing, i.e. the first + is 1, and so on. Note the absence of a `$' in front of VAR. To + avoid clashes, the parameter cannot begin with `_expect'. The + index -1 is used if there is a timeout and 0 if there is no match. + + The option -P VAR works similarly to -p, but instead of numerical + indexes the regular arguments must begin with a prefix followed by + a colon: that prefix is then used as a tag to which VAR is set + when the argument matches. The tag timeout is used if there is a + timeout and the empty string if there is no match. Note it is + acceptable for different arguments to start with the same prefix + if the matches do not need to be distinguished. + + The option -q is passed directly down to tcp_read. + + As all input is done via tcp_read, all the usual rules about + output of lines read apply. One exception is that the parameter + $tcp_lines will only reflect the line actually matched by + tcp_expect; use $tcp_expect_lines for the full set of lines read + during the function call. + +tcp_proxy + This is a simple-minded function to accept a TCP connection and + execute a command with I/O redirected to the connection. Extreme + caution should be taken as there is no security whatsoever and + this can leave your computer open to the world. Ideally, it + should only be used behind a firewall. + + The first argument is a TCP port on which the function will listen. + + The remaining arguments give a command and its arguments to + execute with standard input, standard output and standard error + redirected to the file descriptor on which the TCP session has + been accepted. If no command is given, a new zsh is started. + This gives everyone on your network direct access to your account, + which in many cases will be a bad thing. + + The command is run in the background, so tcp_proxy can then accept + new connections. It continues to accept new connections until + interrupted. + +tcp_spam [ -ertv ] [ -a | -s SESS | -l SESS[,...] ] CMD [ ARG ... ] + Execute `CMD [ ARG ... ]' for each session in turn. Note this + executes the command and arguments; it does not send the command + line as data unless the -t (transmit) option is given. + + The sessions may be selected explicitly with the standard -a, -s or + -l options, or may be chosen implicitly. If none of the three + options is given the rules are: first, if the array $tcp_spam_list + is set, this is taken as the list of sessions, otherwise all + sessions are taken. Second, any sessions given in the array + $tcp_no_spam_list are removed from the list of sessions. + + Normally, any sessions added by the `-a' flag or when all sessions + are chosen implicitly are spammed in alphabetic order; sessions + given by the $tcp_spam_list array or on the command line are + spammed in the order given. The -r flag reverses the order + however it was arrived it. + + The -v flag specifies that a $TCP_PROMPT will be output before each + session. This is output after any modification to TCP_SESS by the + user-defined tcp_on_spam function described below. (Obviously that + function is able to generate its own output.) + + If the option -e is present, the line given as `CMD [ ARG ... ]' + is executed using eval, otherwise it is executed without any + further processing. + +tcp_talk + This is a fairly simple-minded attempt to force input to the line + editor to go straight to the default TCP_SESS. + + An escape string, $TCP_TALK_ESCAPE, default `:', is used to allow + access to normal shell operation. If it is on its own at the + start of the line, or followed only by whitespace, the line editor + returns to normal operation. Otherwise, the string and any + following whitespace are skipped and the remainder of the line + executed as shell input without any change of the line editor's + operating mode. + + The current implementation is somewhat deficient in terms of use + of the command history. For this reason, many users will prefer + to use some form of alternative approach for sending data easily + to the current session. One simple approach is to alias some + special character (such as `%') to `tcp_command --'. + +tcp_wait + The sole argument is an integer or floating point number which + gives the seconds to delay. The shell will do nothing for that + period except wait for input on all TCP sessions by calling + tcp_read -a. This is similar to the interactive behaviour at the + command prompt when zle handlers are installed. + + + +24.2.4 `One-shot' file transfer +------------------------------- + + +tcp_point PORT +tcp_shoot HOST PORT + This pair of functions provide a simple way to transfer a file + between two hosts within the shell. Note, however, that bulk data + transfer is currently done using cat. tcp_point reads any data + arriving at PORT and sends it to standard output; tcp_shoot + connects to PORT on HOST and sends its standard input. Any unused + PORT may be used; the standard mechanism for picking a port is to + think of a random four-digit number above 1024 until one works. + + To transfer a file from host woodcock to host springes, on + springes: + + + tcp_point 8091 >output_file + + and on woodcock: + + + tcp_shoot springes 8091 zsh.report + +You should check the zsh.report file for any sensitive information such +as passwords and delete them by hand before sending the script to the +developers. Also, as the output can be voluminous, it's best to wait +for the developers to ask for this information before sending it. + +You can also use reporter to dump only a subset of the shell state. +This is sometimes useful for creating startup files for the first time. +Most of the output from reporter is far more detailed than usually is +necessary for a startup file, but the aliases, options, and zstyles +states may be useful because they include only changes from the +defaults. The bindings state may be useful if you have created any of +your own keymaps, because reporter arranges to dump the keymap creation +commands as well as the bindings for every keymap. + +As is usual with automated tools, if you create a startup file with +reporter, you should edit the results to remove unnecessary commands. +Note that if you're using the new completion system, you should _not_ +dump the functions state to your startup files with reporter; use the +compdump function instead (see *Note Completion System::). + + +reporter [ STATE ... ] + Print to standard output the indicated subset of the current shell + state. The STATE arguments may be one or more of: + + + all + Output everything listed below. + + aliases + Output alias definitions. + + bindings + Output ZLE key maps and bindings. + + completion + Output old-style compctl commands. New completion is covered + by functions and zstyles. + + functions + Output autoloads and function definitions. + + limits + Output limit commands. + + options + Output setopt commands. + + styles + Same as zstyles. + + variables + Output shell parameter assignments, plus export commands for + any environment variables. + + zstyles + Output zstyle commands. + + If the STATE is omitted, all is assumed. + + With the exception of `all', every STATE can be abbreviated by any + prefix, even a single letter; thus a is the same as aliases, z is + the same as zstyles, etc. + + +26.2.5 Manipulating Hook Functions +---------------------------------- + + + + +add-zsh-hook [ -L | -dD ] [ -Uzk ] HOOK FUNCTION + Several functions are special to the shell, as described in the + section Special Functions, *Note Functions::, in that they are + automatically called at specific points during shell execution. + Each has an associated array consisting of names of functions to be + called at the same point; these are so-called `hook functions'. + The shell function add-zsh-hook provides a simple way of adding or + removing functions from the array. + + HOOK is one of chpwd, periodic, precmd, preexec, zshaddhistory, + zshexit, or zsh_directory_name, the special functions in question. + Note that zsh_directory_name is called in a different way from + the other functions, but may still be manipulated as a hook. + + FUNCTION is name of an ordinary shell function. If no options are + given this will be added to the array of functions to be executed + in the given context. Functions are invoked in the order they + were added. + + If the option -L is given, the current values for the hook arrays + are listed with typeset. + + If the option -d is given, the FUNCTION is removed from the array + of functions to be executed. + + If the option -D is given, the FUNCTION is treated as a pattern + and any matching names of functions are removed from the array of + functions to be executed. + + The options -U, -z and -k are passed as arguments to autoload for + FUNCTION. For functions contributed with zsh, the options -Uz are + appropriate. + +add-zle-hook-widget [ -L | -dD ] [ -Uzk ] HOOK WIDGETNAME + Several widget names are special to the line editor, as described + in the section Special Widgets, *Note Zle Widgets::, in that they + are automatically called at specific points during editing. + Unlike function hooks, these do not use a predefined array of + other names to call at the same point; the shell function + add-zle-hook-widget maintains a similar array and arranges for the + special widget to invoke those additional widgets. + + HOOK is one of isearch-exit, isearch-update, line-pre-redraw, + line-init, line-finish, history-line-set, or keymap-select, + corresponding to each of the special widgets zle-isearch-exit, + etc. The special widget names are also accepted as the HOOK + argument. + + WIDGETNAME is the name of a ZLE widget. If no options are given + this is added to the array of widgets to be invoked in the given + hook context. Widgets are invoked in the order they were added, + with + zle WIDGETNAME -Nw -- "$@" + + Note that this means that the `WIDGET' special parameter tracks the + WIDGETNAME when the widget function is called, rather than tracking + the name of the corresponding special hook widget. + + If the option -d is given, the WIDGETNAME is removed from the + array of widgets to be executed. + + If the option -D is given, the WIDGETNAME is treated as a pattern + and any matching names of widgets are removed from the array. + + If WIDGETNAME does not name an existing widget when added to the + array, it is assumed that a shell function also named WIDGETNAME is + meant to provide the implementation of the widget. This name is + therefore marked for autoloading, and the options -U, -z and -k are + passed as arguments to autoload as with add-zsh-hook. The widget + is also created with `zle -N WIDGETNAME' to cause the + corresponding function to be loaded the first time the hook is + called. + + The arrays of WIDGETNAME are currently maintained in zstyle + contexts, one for each HOOK context, with a style of `widgets'. + If the -L option is given, this set of styles is listed with + `zstyle -L'. This implementation may change, and the special + widgets that refer to the styles are created only if + add-zle-hook-widget is called to add at least one widget, so if + this function is used for any hooks, then all hooks should be + managed only via this function. + + + + +File: zsh.info, Node: Recent Directories, Next: Other Directory Functions, Prev: Utilities, Up: User Contributions + +26.3 Remembering Recent Directories +=================================== + +The function cdr allows you to change the working directory to a +previous working directory from a list maintained automatically. It is +similar in concept to the directory stack controlled by the pushd, popd +and dirs builtins, but is more configurable, and as it stores all +entries in files it is maintained across sessions and (by default) +between terminal emulators in the current session. Duplicates are +automatically removed, so that the list reflects the single most recent +use of each directory. + +Note that the pushd directory stack is not actually modified or used by +cdr unless you configure it to do so as described in the configuration +section below. + + + +26.3.1 Installation +------------------- + +The system works by means of a hook function that is called every time +the directory changes. To install the system, autoload the required +functions and use the add-zsh-hook function described above: + + + autoload -Uz chpwd_recent_dirs cdr add-zsh-hook + add-zsh-hook chpwd chpwd_recent_dirs + +Now every time you change directly interactively, no matter which +command you use, the directory to which you change will be remembered +in most-recent-first order. + + + +26.3.2 Use +---------- + +All direct user interaction is via the cdr function. + +The argument to cdr is a number N corresponding to the Nth most +recently changed-to directory. 1 is the immediately preceding +directory; the current directory is remembered but is not offered as a +destination. Note that if you have multiple windows open 1 may refer +to a directory changed to in another window; you can avoid this by +having per-terminal files for storing directory as described for the +recent-dirs-file style below. + +If you set the recent-dirs-default style described below cdr will +behave the same as cd if given a non-numeric argument, or more than one +argument. The recent directory list is updated just the same however +you change directory. + +If the argument is omitted, 1 is assumed. This is similar to pushd's +behaviour of swapping the two most recent directories on the stack. + +Completion for the argument to cdr is available if compinit has been +run; menu selection is recommended, using: + + + zstyle ':completion:*:*:cdr:*:*' menu selection + +to allow you to cycle through recent directories; the order is +preserved, so the first choice is the most recent directory before the +current one. The verbose style is also recommended to ensure the +directory is shown; this style is on by default so no action is +required unless you have changed it. + + + +26.3.3 Options +-------------- + +The behaviour of cdr may be modified by the following options. + + +-l + lists the numbers and the corresponding directories in abbreviated + form (i.e. with ~ substitution reapplied), one per line. The + directories here are not quoted (this would only be an issue if a + directory name contained a newline). This is used by the + completion system. + +-r + sets the variable reply to the current set of directories. Nothing + is printed and the directory is not changed. + +-e + allows you to edit the list of directories, one per line. The + list can be edited to any extent you like; no sanity checking is + performed. Completion is available. No quoting is necessary + (except for newlines, where I have in any case no sympathy); + directories are in unabbreviated from and contain an absolute + path, i.e. they start with /. Usually the first entry should be + left as the current directory. + +-p 'PATTERN' + Prunes any items in the directory list that match the given + extended glob pattern; the pattern needs to be quoted from + immediate expansion on the command line. The pattern is matched + against each completely expanded file name in the list; the full + string must match, so wildcards at the end (e.g. '*removeme*') are + needed to remove entries with a given substring. + + If output is to a terminal, then the function will print the new + list after pruning and prompt for confirmation by the user. This + output and confirmation step can be skipped by using -P instead of + -p. + + + +26.3.4 Configuration +-------------------- + +Configuration is by means of the styles mechanism that should be +familiar from completion; if not, see the description of the zstyle +command in *Note The zsh/zutil Module::. The context for setting styles +should be ':chpwd:*' in case the meaning of the context is extended in +future, for example: + + + zstyle ':chpwd:*' recent-dirs-max 0 + +sets the value of the recent-dirs-max style to 0. In practice the +style name is specific enough that a context of '*' should be fine. + +An exception is recent-dirs-insert, which is used exclusively by the +completion system and so has the usual completion system context +(':completion:*' if nothing more specific is needed), though again '*' +should be fine in practice. + + +recent-dirs-default + If true, and the command is expecting a recent directory index, and + either there is more than one argument or the argument is not an + integer, then fall through to "cd". This allows the lazy to use + only one command for directory changing. Completion recognises + this, too; see recent-dirs-insert for how to control completion + when this option is in use. + +recent-dirs-file + The file where the list of directories is saved. The default is + ${ZDOTDIR:-$HOME}/.chpwd-recent-dirs, i.e. this is in your home + directory unless you have set the variable ZDOTDIR to point + somewhere else. Directory names are saved in $'...' quoted form, + so each line in the file can be supplied directly to the shell as + an argument. + + The value of this style may be an array. In this case, the first + file in the list will always be used for saving directories while + any other files are left untouched. When reading the recent + directory list, if there are fewer than the maximum number of + entries in the first file, the contents of later files in the + array will be appended with duplicates removed from the list + shown. The contents of the two files are not sorted together, + i.e. all the entries in the first file are shown first. The + special value + can appear in the list to indicate the default + file should be read at that point. This allows effects like the + following: + + + zstyle ':chpwd:*' recent-dirs-file \ + ~/.chpwd-recent-dirs-${TTY##*/} + + + Recent directories are read from a file numbered according to the + terminal. If there are insufficient entries the list is + supplemented from the default file. + + It is possible to use zstyle -e to make the directory configurable + at run time: + + + zstyle -e ':chpwd:*' recent-dirs-file pick-recent-dirs-file + pick-recent-dirs-file() { + if [[ $PWD = ~/text/writing(|/*) ]]; then + reply=(~/.chpwd-recent-dirs-writing) + else + reply=(+) + fi + } + + In this example, if the current directory is ~/text/writing or a + directory under it, then use a special file for saving recent + directories, else use the default. + +recent-dirs-insert + Used by completion. If recent-dirs-default is true, then setting + this to true causes the actual directory, rather than its index, to + be inserted on the command line; this has the same effect as using + the corresponding index, but makes the history clearer and the line + easier to edit. With this setting, if part of an argument was + already typed, normal directory completion rather than recent + directory completion is done; this is because recent directory + completion is expected to be done by cycling through entries menu + fashion. + + If the value of the style is always, then only recent directories + will be completed; in that case, use the cd command when you want + to complete other directories. + + If the value is fallback, recent directories will be tried first, + then normal directory completion is performed if recent directory + completion failed to find a match. + + Finally, if the value is both then both sets of completions are + presented; the usual tag mechanism can be used to distinguish + results, with recent directories tagged as recent-dirs. Note that + the recent directories inserted are abbreviated with directory + names where appropriate. + +recent-dirs-max + The maximum number of directories to save to the file. If this is + zero or negative there is no maximum. The default is 20. Note + this includes the current directory, which isn't offered, so the + highest number of directories you will be offered is one less than + the maximum. + +recent-dirs-prune + This style is an array determining what directories should (or + should not) be added to the recent list. Elements of the array + can include: + + + parent + Prune parents (more accurately, ancestors) from the recent + list. If present, changing directly down by any number of + directories causes the current directory to be overwritten. + For example, changing from ~pws to ~pws/some/other/dir causes + ~pws not to be left on the recent directory stack. This only + applies to direct changes to descendant directories; earlier + directories on the list are not pruned. For example, + changing from ~pws/yet/another to ~pws/some/other/dir does + not cause ~pws to be pruned. + + pattern:PATTERN + Gives a zsh pattern for directories that should not be added + to the recent list (if not already there). This element can + be repeated to add different patterns. For example, + 'pattern:/tmp(|/*)' stops /tmp or its descendants from being + added. The EXTENDED_GLOB option is always turned on for + these patterns. + + +recent-dirs-pushd + If set to true, cdr will use pushd instead of cd to change the + directory, so the directory is saved on the directory stack. As + the directory stack is completely separate from the list of files + saved by the mechanism used in this file there is no obvious + reason to do this. + + + +26.3.5 Use with dynamic directory naming +---------------------------------------- + +It is possible to refer to recent directories using the dynamic +directory name syntax by using the supplied function +zsh_directory_name_cdr a hook: + + + autoload -Uz add-zsh-hook + add-zsh-hook -Uz zsh_directory_name zsh_directory_name_cdr + +When this is done, ~[1] will refer to the most recent directory other +than $PWD, and so on. Completion after ~[... also works. + + + +26.3.6 Details of directory handling +------------------------------------ + +This section is for the curious or confused; most users will not need +to know this information. + +Recent directories are saved to a file immediately and hence are +preserved across sessions. Note currently no file locking is applied: +the list is updated immediately on interactive commands and nowhere else +(unlike history), and it is assumed you are only going to change +directory in one window at once. This is not safe on shared accounts, +but in any case the system has limited utility when someone else is +changing to a different set of directories behind your back. + +To make this a little safer, only directory changes instituted from the +command line, either directly or indirectly through shell function calls +(but not through subshells, evals, traps, completion functions and the +like) are saved. Shell functions should use cd -q or pushd -q to avoid +side effects if the change to the directory is to be invisible at the +command line. See the contents of the function chpwd_recent_dirs for +more details. + + + + +File: zsh.info, Node: Other Directory Functions, Next: Version Control Information, Prev: Recent Directories, Up: User Contributions + +26.4 Abbreviated dynamic references to directories +================================================== + +The dynamic directory naming system is described in the subsection +_Dynamic named directories_ of *Note Filename Expansion::. In this, a +reference to ~[...] is expanded by a function found by the hooks +mechanism. + +The contributed function zsh_directory_name_generic provides a system +allowing the user to refer to directories with only a limited amount of +new code. It supports all three of the standard interfaces for +directory naming: converting from a name to a directory, converting in +the reverse direction to find a short name, and completion of names. + +The main feature of this function is a path-like syntax, combining +abbreviations at multiple levels separated by ":". As an example, +~[g:p:s] might specify: +g + The top level directory for your git area. This first component + has to match, or the function will return indicating another + directory name hook function should be tried. + +p + The name of a project within your git area. + +s + The source area within that project. + + This allows you to collapse references to long hierarchies to a very +compact form, particularly if the hierarchies are similar across +different areas of the disk. + +Name components may be completed: if a description is shown at the top +of the list of completions, it includes the path to which previous +components expand, while the description for an individual completion +shows the path segment it would add. No additional configuration is +needed for this as the completion system is aware of the dynamic +directory name mechanism. + + + +26.4.1 Usage +------------ + +To use the function, first define a wrapper function for your specific +case. We'll assume it's to be autoloaded. This can have any name but +we'll refer to it as zdn_mywrapper. This wrapper function will define +various variables and then call this function with the same arguments +that the wrapper function gets. This configuration is described below. + +Then arrange for the wrapper to be run as a zsh_directory_name hook: + + + autoload -Uz add-zsh-hook zsh_diretory_name_generic zdn_mywrapper + add-zsh-hook -U zsh_directory_name zdn_mywrapper + + +26.4.2 Configuration +-------------------- + +The wrapper function should define a local associative array zdn_top. +Alternatively, this can be set with a style called mapping. The +context for the style is :zdn:WRAPPER-NAME where WRAPPER-NAME is the +function calling zsh_directory_name_generic; for example: + + + zstyle :zdn:zdn_mywrapper: mapping zdn_mywrapper_top + +The keys in this associative array correspond to the first component of +the name. The values are matching directories. They may have an +optional suffix with a slash followed by a colon and the name of a +variable in the same format to give the next component. (The slash +before the colon is to disambiguate the case where a colon is needed in +the path for a drive. There is otherwise no syntax for escaping this, +so path components whose names start with a colon are not supported.) A +special component :default: specifies a variable in the form /:VAR (the +path section is ignored and so is usually empty) that will be used for +the next component if no variable is given for the path. Variables +referred to within zdn_top have the same format as zdn_top itself, but +contain relative paths. + +For example, + + + local -A zdn_top=( + g ~/git + ga ~/alternate/git + gs /scratch/$USER/git/:second2 + :default: /:second1 + ) + +This specifies the behaviour of a directory referred to as ~[g:...] or +~[ga:...] or ~[gs:...]. Later path components are optional; in that +case ~[g] expands to ~/git, and so on. gs expands to +/scratch/$USER/git and uses the associative array second2 to match the +second component; g and ga use the associative array second1 to match +the second component. + +When expanding a name to a directory, if the first component is not g or +ga or gs, it is not an error; the function simply returns 1 so that a +later hook function can be tried. However, matching the first component +commits the function, so if a later component does not match, an error +is printed (though this still does not stop later hooks from being +executed). + +For components after the first, a relative path is expected, but note +that multiple levels may still appear. Here is an example of second1: + + + local -A second1=( + p myproject + s somproject + os otherproject/subproject/:third + ) + +The path as found from zdn_top is extended with the matching directory, +so ~[g:p] becomes ~/git/myproject. The slash between is added +automatically (it's not possible to have a later component modify the +name of a directory already matched). Only os specifies a variable for +a third component, and there's no :default:, so it's an error to use a +name like ~[g:p:x] or ~[ga:s:y] because there's nowhere to look up the +x or y. + +The associative arrays need to be visible within this function; the +generic function therefore uses internal variable names beginning _zdn_ +in order to avoid clashes. Note that the variable reply needs to be +passed back to the shell, so should not be local in the calling +function. + +The function does not test whether directories assembled by component +actually exist; this allows the system to work across automounted file +systems. The error from the command trying to use a non-existent +directory should be sufficient to indicate the problem. + + + +26.4.3 Complete example +----------------------- + +Here is a full fictitious but usable autoloadable definition of the +example function defined by the code above. So ~[gs:p:s] expands to +/scratch/$USER/git/myscratchproject/top/srcdir (with $USER also +expanded). + + + local -A zdn_top=( + g ~/git + ga ~/alternate/git + gs /scratch/$USER/git/:second2 + :default: /:second1 + ) + + local -A second1=( + p myproject + s somproject + os otherproject/subproject/:third + ) + + local -A second2=( + p myscratchproject + s somescratchproject + ) + + local -A third=( + s top/srcdir + d top/documentation + ) + + # autoload not needed if you did this at initialisation... + autoload -Uz zsh_directory_name_generic + zsh_directory_name_generic "$@ + +It is also possible to use global associative arrays, suitably named, +and set the style for the context of your wrapper function to refer to +this. Then your set up code would contain the following: + + + typeset -A zdn_mywrapper_top=(...) + # ... and so on for other associative arrays ... + zstyle ':zdn:zdn_mywrapper:' mapping zdn_mywrapper_top + autoload -Uz add-zsh-hook zsh_directory_name_generic zdn_mywrapper + add-zsh-hook -U zsh_directory_name zdn_mywrapper + +and the function zdn_mywrapper would contain only the following: + + + zsh_directory_name_generic "$@" + + + +File: zsh.info, Node: Version Control Information, Next: Prompt Themes, Prev: Other Directory Functions, Up: User Contributions + +26.5 Gathering information from version control systems +======================================================= + + + +In a lot of cases, it is nice to automatically retrieve information from +version control systems (VCSs), such as subversion, CVS or git, to be +able to provide it to the user; possibly in the user's prompt. So that +you can instantly tell which branch you are currently on, for example. + +In order to do that, you may use the vcs_info function. + +The following VCSs are supported, showing the abbreviated name by which +they are referred to within the system: +Bazaar (bzr) + `https://bazaar.canonical.com/' + +Codeville (cdv) + `http://freecode.com/projects/codeville/' + +Concurrent Versioning System (cvs) + `https://www.nongnu.org/cvs/' + +Darcs (darcs) + `http://darcs.net/' + +Fossil (fossil) + `https://fossil-scm.org/' + +Git (git) + `https://git-scm.com/' + +GNU arch (tla) + `https://www.gnu.org/software/gnu-arch/' + +Mercurial (hg) + `https://www.mercurial-scm.org/' + +Monotone (mtn) + `https://monotone.ca/' + +Perforce (p4) + `https://www.perforce.com/' + +Subversion (svn) + `https://subversion.apache.org/' + +SVK (svk) + `https://svk.bestpractical.com/' + +There is also support for the patch management system quilt +(`https://savannah.nongnu.org/projects/quilt'). See *Note vcs_info +Quilt Support:: below for details. + +To load vcs_info: + + + autoload -Uz vcs_info + +It can be used in any existing prompt, because it does not require any +specific $psvar entries to be available. + + + +* Menu: + +* vcs_info Quickstart:: +* vcs_info Configuration:: +* vcs_info Oddities:: +* vcs_info Quilt Support:: +* vcs_info API:: +* vcs_info Variables:: +* vcs_info Hooks:: +* vcs_info Examples:: + + + +File: zsh.info, Node: vcs_info Quickstart, Next: vcs_info Configuration, Up: Version Control Information + +26.5.1 Quickstart +----------------- + +To get this feature working quickly (including colors), you can do the +following (assuming, you loaded vcs_info properly - see above): + + + zstyle ':vcs_info:*' actionformats \ + '%F{5}(%f%s%F{5})%F{3}-%F{5}[%F{2}%b%F{3}|%F{1}%a%F{5}]%f ' + zstyle ':vcs_info:*' formats \ + '%F{5}(%f%s%F{5})%F{3}-%F{5}[%F{2}%b%F{5}]%f ' + zstyle ':vcs_info:(sv[nk]|bzr):*' branchformat '%b%F{1}:%F{3}%r' + precmd () { vcs_info } + PS1='%F{5}[%F{2}%n%F{5}] %F{3}%3~ ${vcs_info_msg_0_}%f%# ' + +Obviously, the last two lines are there for demonstration. You need to +call vcs_info from your precmd function. Once that is done you need a +_single quoted_ '${vcs_info_msg_0_}' in your prompt. + +To be able to use '${vcs_info_msg_0_}' directly in your prompt like +this, you will need to have the PROMPT_SUBST option enabled. + +Now call the vcs_info_printsys utility from the command line: + + + % vcs_info_printsys + ## list of supported version control backends: + ## disabled systems are prefixed by a hash sign (#) + bzr + cdv + cvs + darcs + fossil + git + hg + mtn + p4 + svk + svn + tla + ## flavours (cannot be used in the enable or disable styles; they + ## are enabled and disabled with their master [git-svn -> git]) + ## they *can* be used in contexts: ':vcs_info:git-svn:*'. + git-p4 + git-svn + hg-git + hg-hgsubversion + hg-hgsvn + +You may not want all of these because there is no point in running the +code to detect systems you do not use. So there is a way to disable +some backends altogether: + + + zstyle ':vcs_info:*' disable bzr cdv darcs mtn svk tla + +You may also pick a few from that list and enable only those: + + + zstyle ':vcs_info:*' enable git cvs svn + +If you rerun vcs_info_printsys after one of these commands, you will +see the backends listed in the disable style (or backends not in the +enable style - if you used that) marked as disabled by a hash sign. +That means the detection of these systems is skipped _completely_. No +wasted time there. + + + + +File: zsh.info, Node: vcs_info Configuration, Next: vcs_info Oddities, Prev: vcs_info Quickstart, Up: Version Control Information + +26.5.2 Configuration +-------------------- + +The vcs_info feature can be configured via zstyle. + +First, the context in which we are working: + :vcs_info:VCS-STRING:USER-CONTEXT:REPO-ROOT-NAME + + +VCS-STRING + is one of: git, git-svn, git-p4, hg, hg-git, hg-hgsubversion, + hg-hgsvn, darcs, bzr, cdv, mtn, svn, cvs, svk, tla, p4 or fossil. + This is followed by `.quilt-QUILT-MODE' in Quilt mode (see *Note + vcs_info Quilt Support:: for details) and by `+HOOK-NAME' while + hooks are active (see *Note vcs_info Hooks:: for details). + + Currently, hooks in quilt mode don't add the `.quilt-QUILT-MODE' + information. This may change in the future. + +USER-CONTEXT + is a freely configurable string, assignable by the user as the + first argument to vcs_info (see its description below). + +REPO-ROOT-NAME + is the name of a repository in which you want a style to match. + So, if you want a setting specific to /usr/src/zsh, with that + being a CVS checkout, you can set REPO-ROOT-NAME to zsh to make it + so. + + +There are three special values for VCS-STRING: The first is named +-init-, that is in effect as long as there was no decision what VCS +backend to use. The second is -preinit-; it is used _before_ vcs_info +is run, when initializing the data exporting variables. The third +special value is formats and is used by the vcs_info_lastmsg for +looking up its styles. + +The initial value of REPO-ROOT-NAME is -all- and it is replaced with +the actual name, as soon as it is known. Only use this part of the +context for defining the formats, actionformats or branchformat styles, +as it is guaranteed that REPO-ROOT-NAME is set up correctly for these +only. For all other styles, just use '*' instead. + +There are two pre-defined values for USER-CONTEXT: +default + the one used if none is specified + +command + used by vcs_info_lastmsg to lookup its styles + +You can of course use ':vcs_info:*' to match all VCSs in all +user-contexts at once. + +This is a description of all styles that are looked up. + + +formats + A list of formats, used when actionformats is not used (which is + most of the time). + +actionformats + A list of formats, used if there is a special action going on in + your current repository; like an interactive rebase or a merge + conflict. + +branchformat + Some backends replace %b in the formats and actionformats styles + above, not only by a branch name but also by a revision number. + This style lets you modify how that string should look. + +nvcsformats + These "formats" are set when we didn't detect a version control + system for the current directory or vcs_info was disabled. This is + useful if you want vcs_info to completely take over the generation + of your prompt. You would do something like + PS1='${vcs_info_msg_0_}' to accomplish that. + +hgrevformat + hg uses both a hash and a revision number to reference a specific + changeset in a repository. With this style you can format the + revision string (see branchformat) to include either or both. It's + only useful when get-revision is true. Note, the full 40-character + revision id is not available (except when using the use-simple + option) because executing hg more than once per prompt is too + slow; you may customize this behavior using hooks. + +max-exports + Defines the maximum number of vcs_info_msg_*_ variables vcs_info + will set. + +enable + A list of backends you want to use. Checked in the -init- context. + If this list contains an item called NONE no backend is used at + all and vcs_info will do nothing. If this list contains ALL, + vcs_info will use all known backends. Only with ALL in enable will + the disable style have any effect. ALL and NONE are case + insensitive. + +disable + A list of VCSs you don't want vcs_info to test for repositories + (checked in the -init- context, too). Only used if enable contains + ALL. + +disable-patterns + A list of patterns that are checked against $PWD. If a pattern + matches, vcs_info will be disabled. This style is checked in the + :vcs_info:-init-:*:-all- context. + + Say, ~/.zsh is a directory under version control, in which you do + not want vcs_info to be active, do: + zstyle ':vcs_info:*' disable-patterns "${(b)HOME}/.zsh(|/*)" + +use-quilt + If enabled, the quilt support code is active in `addon' mode. See + *Note vcs_info Quilt Support:: for details. + +quilt-standalone + If enabled, `standalone' mode detection is attempted if no VCS is + active in a given directory. See *Note vcs_info Quilt Support:: + for details. + +quilt-patch-dir + Overwrite the value of the $QUILT_PATCHES environment variable. See + *Note vcs_info Quilt Support:: for details. + +quiltcommand + When quilt itself is called in quilt support, the value of this + style is used as the command name. + +check-for-changes + If enabled, this style causes the %c and %u format escapes to show + when the working directory has uncommitted changes. The strings + displayed by these escapes can be controlled via the stagedstr and + unstagedstr styles. The only backends that currently support this + option are git, hg, and bzr (the latter two only support unstaged). + + For this style to be evaluated with the hg backend, the + get-revision style needs to be set and the use-simple style needs + to be unset. The latter is the default; the former is not. + + With the bzr backend, _lightweight checkouts_ only honor this + style if the use-server style is set. + + Note, the actions taken if this style is enabled are potentially + expensive (read: they may be slow, depending on how big the + current repository is). Therefore, it is disabled by default. + +check-for-staged-changes + This style is like check-for-changes, but it never checks the + worktree files, only the metadata in the .${vcs} dir. Therefore, + this style initializes only the %c escape (with stagedstr) but not + the %u escape. This style is faster than check-for-changes. + + In the git backend, this style checks for changes in the index. + Other backends do not currently implement this style. + + This style is disabled by default. + +stagedstr + This string will be used in the %c escape if there are staged + changes in the repository. + +unstagedstr + This string will be used in the %u escape if there are unstaged + changes in the repository. + +command + This style causes vcs_info to use the supplied string as the + command to use as the VCS's binary. Note, that setting this in + ':vcs_info:*' is not a good idea. + + If the value of this style is empty (which is the default), the + used binary name is the name of the backend in use (e.g. svn is + used in an svn repository). + + The repo-root-name part in the context is always the default -all- + when this style is looked up. + + For example, this style can be used to use binaries from + non-default installation directories. Assume, git is installed in + /usr/bin but your sysadmin installed a newer version in + /usr/local/bin. Instead of changing the order of your $PATH + parameter, you can do this: + zstyle ':vcs_info:git:*:-all-' command /usr/local/bin/git + +use-server + This is used by the Perforce backend (p4) to decide if it should + contact the Perforce server to find out if a directory is managed + by Perforce. This is the only reliable way of doing this, but runs + the risk of a delay if the server name cannot be found. If the + server (more specifically, the host:port pair describing the + server) cannot be contacted, its name is put into the associative + array vcs_info_p4_dead_servers and is not contacted again during + the session until it is removed by hand. If you do not set this + style, the p4 backend is only usable if you have set the + environment variable P4CONFIG to a file name and have + corresponding files in the root directories of each Perforce + client. See comments in the function VCS_INFO_detect_p4 for more + detail. + + The Bazaar backend (bzr) uses this to permit contacting the server + about lightweight checkouts, see the check-for-changes style. + +use-simple + If there are two different ways of gathering information, you can + select the simpler one by setting this style to true; the default + is to use the not-that-simple code, which is potentially a lot + slower but might be more accurate in all possible cases. This + style is used by the bzr and hg backends. In the case of hg it + will invoke the external hexdump program to parse the binary + dirstate cache file; this method will not return the local + revision number. + +get-revision + If set to true, vcs_info goes the extra mile to figure out the + revision of a repository's work tree (currently for the git and hg + backends, where this kind of information is not always vital). For + git, the hash value of the currently checked out commit is + available via the %i expansion. With hg, the local revision number + and the corresponding global hash are available via %i. + +get-mq + If set to true, the hg backend will look for a Mercurial Queue (mq) + patch directory. Information will be available via the `%m' + replacement. + +get-bookmarks + If set to true, the hg backend will try to get a list of current + bookmarks. They will be available via the `%m' replacement. + + The default is to generate a comma-separated list of all bookmark + names that refer to the currently checked out revision. If a + bookmark is active, its name is suffixed an asterisk and placed + first in the list. + +use-prompt-escapes + Determines if we assume that the assembled string from vcs_info + includes prompt escapes. (Used by vcs_info_lastmsg.) + +debug + Enable debugging output to track possible problems. Currently this + style is only used by vcs_info's hooks system. + +hooks + A list style that defines hook-function names. See *Note vcs_info + Hooks:: below for details. + +patch-format +nopatch-format + This pair of styles format the patch information used by the %m + expando in formats and actionformats for the git and hg backends. + The value is subject to certain %-expansions described below. The + expanded value is made available in the global backend_misc array + as ${backend_misc[patches]} (also if a set-patch-format hook is + used). + +get-unapplied + This boolean style controls whether a backend should attempt to + gather a list of unapplied patches (for example with Mercurial + Queue patches). + + Used by the quilt and hg backends. + + +The default values for these styles in all contexts are: + + +formats + " (%s)-[%b]%u%c-" + +actionformats + " (%s)-[%b|%a]%u%c-" + +branchformat + "%b:%r" (for bzr, svn, svk and hg) + +nvcsformats + "" + +hgrevformat + "%r:%h" + +max-exports + 2 + +enable + ALL + +disable + (empty list) + +disable-patterns + (empty list) + +check-for-changes + false + +check-for-staged-changes + false + +stagedstr + (string: "S") + +unstagedstr + (string: "U") + +command + (empty string) + +use-server + false + +use-simple + false + +get-revision + false + +get-mq + true + +get-bookmarks + false + +use-prompt-escapes + true + +debug + false + +hooks + (empty list) + +use-quilt + false + +quilt-standalone + false + +quilt-patch-dir + empty - use $QUILT_PATCHES + +quiltcommand + quilt + +patch-format + BACKEND DEPENDENT + +nopatch-format + BACKEND DEPENDENT + +get-unapplied + false + +In normal formats and actionformats the following replacements are done: + + +%s + The VCS in use (git, hg, svn, etc.). + +%b + Information about the current branch. + +%a + An identifier that describes the action. Only makes sense in + actionformats. + +%i + The current revision number or identifier. For hg the hgrevformat + style may be used to customize the output. + +%c + The string from the stagedstr style if there are staged changes in + the repository. + +%u + The string from the unstagedstr style if there are unstaged + changes in the repository. + +%R + The base directory of the repository. + +%r + The repository name. If %R is /foo/bar/repoXY, %r is repoXY. + +%S + A subdirectory within a repository. If $PWD is + /foo/bar/repoXY/beer/tasty, %S is beer/tasty. + +%m + A "misc" replacement. It is at the discretion of the backend to + decide what this replacement expands to. + + The hg and git backends use this expando to display patch + information. hg sources patch information from the mq extensions; + git from in-progress rebase and cherry-pick operations and from + the stgit extension. The patch-format and nopatch-format styles + control the generated string. The former is used when at least + one patch from the patch queue has been applied, and the latter + otherwise. + + The hg backend displays bookmark information in this expando (in + addition to mq information). See the get-mq and get-bookmarks + styles. Both of these styles may be enabled at the same time. If + both are enabled, both resulting strings will be shown separated + by a semicolon (that cannot currently be customized). + + The quilt `standalone' backend sets this expando to the same value + as the %Q expando. + +%Q + Quilt series information. When quilt is used (either in `addon' + mode or as a `standalone' backend), this expando is set to quilt + series' patch-format string. The set-patch-format hook and + nopatch-format style are honoured. + + See *Note vcs_info Quilt Support:: below for details. + + +In branchformat these replacements are done: + + +%b + The branch name. + +%r + The current revision number or the hgrevformat style for hg. + +In hgrevformat these replacements are done: + + +%r + The current local revision number. + +%h + The current global revision identifier. + +In patch-format and nopatch-format these replacements are done: + + +%p + The name of the top-most applied patch; may be overridden by the + applied-string hook. + +%u + The number of unapplied patches; may be overridden by the + unapplied-string hook. + +%n + The number of applied patches. + +%c + The number of unapplied patches. + +%a + The number of all patches (%a = %n + %c). + +%g + The names of active mq guards (hg backend). + +%G + The number of active mq guards (hg backend). + +Not all VCS backends have to support all replacements. For nvcsformats +no replacements are performed at all, it is just a string. + + + + +File: zsh.info, Node: vcs_info Oddities, Next: vcs_info Quilt Support, Prev: vcs_info Configuration, Up: Version Control Information + +26.5.3 Oddities +--------------- + +If you want to use the %b (bold off) prompt expansion in formats, which +expands %b itself, use %%b. That will cause the vcs_info expansion to +replace %%b with %b, so that zsh's prompt expansion mechanism can +handle it. Similarly, to hand down %b from branchformat, use %%%%b. +Sorry for this inconvenience, but it cannot be easily avoided. Luckily +we do not clash with a lot of prompt expansions and this only needs to +be done for those. + +When one of the gen-applied-string, gen-unapplied-string, and +set-patch-format hooks is defined, applying %-escaping +(`foo=${foo//'%'/%%}') to the interpolated values for use in the prompt +is the responsibility of those hooks (jointly); when neither of those +hooks is defined, vcs_info handles escaping by itself. We regret this +coupling, but it was required for backwards compatibility. + + + + +File: zsh.info, Node: vcs_info Quilt Support, Next: vcs_info API, Prev: vcs_info Oddities, Up: Version Control Information + +26.5.4 Quilt Support +-------------------- + +`Quilt' is not a version control system, therefore this is not +implemented as a backend. It can help keeping track of a series of +patches. People use it to keep a set of changes they want to use on top +of software packages (which is tightly integrated into the package +build process - the Debian project does this for a large number of +packages). Quilt can also help individual developers keep track of +their own patches on top of real version control systems. + +The vcs_info integration tries to support both ways of using quilt by +having two slightly different modes of operation: `addon' mode and +`standalone' mode). + +Quilt integration is off by default; to enable it, set the use-quilt +style, and add %Q to your formats or actionformats style: + zstyle ':vcs_info:*' use-quilt true + +Styles looked up from the Quilt support code include `.quilt-QUILT-MODE' +in the VCS-STRING part of the context, where QUILT-MODE is either addon +or standalone. Example: +:vcs_info:git.quilt-addon:default:REPO-ROOT-NAME. + +For `addon' mode to become active vcs_info must have already detected a +real version control system controlling the directory. If that is the +case, a directory that holds quilt's patches needs to be found. That +directory is configurable via the `QUILT_PATCHES' environment variable. +If that variable exists its value is used, otherwise the value +`patches' is assumed. The value from $QUILT_PATCHES can be overwritten +using the `quilt-patches' style. (Note: you can use vcs_info to keep +the value of $QUILT_PATCHES correct all the time via the post-quilt +hook). + +When the directory in question is found, quilt is assumed to be active. +To gather more information, vcs_info looks for a directory called `.pc'; +Quilt uses that directory to track its current state. If this directory +does not exist we know that quilt has not done anything to the working +directory (read: no patches have been applied yet). + +If patches are applied, vcs_info will try to find out which. If you want +to know which patches of a series are not yet applied, you need to +activate the get-unapplied style in the appropriate context. + +vcs_info allows for very detailed control over how the gathered +information is presented (see *Note vcs_info Configuration:: and *Note +vcs_info Hooks::), all of which are documented below. Note there are a +number of other patch tracking systems that work on top of a certain +version control system (like stgit for `git', or mq for `hg'); the +configuration for systems like that are generally configured the same +way as the `quilt' support. + +If the `quilt' support is working in `addon' mode, the produced string +is available as a simple format replacement (%Q to be precise), which +can be used in formats and actionformats; see below for details). + +If, on the other hand, the support code is working in `standalone' mode, +vcs_info will pretend as if quilt were an actual version control +system. That means that the version control system identifier (which +otherwise would be something like `svn' or `cvs') will be set to +`-quilt-'. This has implications on the used style context where this +identifier is the second element. vcs_info will have filled in a proper +value for the "repository's" root directory and the string containing +the information about quilt's state will be available as the `misc' +replacement (and %Q for compatibility with `addon' mode). + +What is left to discuss is how `standalone' mode is detected. The +detection itself is a series of searches for directories. You can have +this detection enabled all the time in every directory that is not +otherwise under version control. If you know there is only a limited +set of trees where you would like vcs_info to try and look for Quilt in +`standalone' mode to minimise the amount of searching on every call to +vcs_info, there are a number of ways to do that: + +Essentially, `standalone' mode detection is controlled by a style called +`quilt-standalone'. It is a string style and its value can have +different effects. The simplest values are: `always' to run detection +every time vcs_info is run, and `never' to turn the detection off +entirely. + +If the value of quilt-standalone is something else, it is interpreted +differently. If the value is the name of a scalar variable the value of +that variable is checked and that value is used in the same +`always'/`never' way as described above. + +If the value of quilt-standalone is an array, the elements of that array +are used as directory names under which you want the detection to be +active. + +If quilt-standalone is an associative array, the keys are taken as +directory names under which you want the detection to be active, but +only if the corresponding value is the string `true'. + +Last, but not least, if the value of quilt-standalone is the name of a +function, the function is called without arguments and the return value +decides whether detection should be active. A `0' return value is true; +a non-zero return value is interpreted as false. + +Note, if there is both a function and a variable by the name of +quilt-standalone, the function will take precedence. + + + + +File: zsh.info, Node: vcs_info API, Next: vcs_info Variables, Prev: vcs_info Quilt Support, Up: Version Control Information + +26.5.5 Function Descriptions (Public API) +----------------------------------------- + + +vcs_info [USER-CONTEXT] + The main function, that runs all backends and assembles all data + into ${vcs_info_msg_*_}. This is the function you want to call from + precmd if you want to include up-to-date information in your + prompt (see *Note vcs_info Variables:: below). If an argument is + given, that string will be used instead of default in the + USER-CONTEXT field of the style context. + +vcs_info_hookadd + Statically registers a number of functions to a given hook. The + hook needs to be given as the first argument; what follows is a + list of hook-function names to register to the hook. The `+vi-' + prefix needs to be left out here. See *Note vcs_info Hooks:: below + for details. + +vcs_info_hookdel + Remove hook-functions from a given hook. The hook needs to be + given as the first non-option argument; what follows is a list of + hook-function names to un-register from the hook. If `-a' is used + as the first argument, all occurrences of the functions are + unregistered. Otherwise only the last occurrence is removed (if a + function was registered to a hook more than once). The `+vi-' + prefix needs to be left out here. See *Note vcs_info Hooks:: + below for details. + +vcs_info_lastmsg + Outputs the last ${vcs_info_msg_*_} value. Takes into account the + value of the use-prompt-escapes style in + ':vcs_info:formats:command:-all-'. It also only prints max-exports + values. + +vcs_info_printsys [USER-CONTEXT] + Prints a list of all supported version control systems. Useful to + find out possible contexts (and which of them are enabled) or + values for the disable style. + +vcs_info_setsys + Initializes vcs_info's internal list of available backends. With + this function, you can add support for new VCSs without restarting + the shell. + + +All functions named VCS_INFO_* are for internal use only. + + + + +File: zsh.info, Node: vcs_info Variables, Next: vcs_info Hooks, Prev: vcs_info API, Up: Version Control Information + +26.5.6 Variable Description +--------------------------- + + +${vcs_info_msg_N_} (Note the trailing underscore) + Where N is an integer, e.g., vcs_info_msg_0_. These variables are + the storage for the informational message the last vcs_info call + has assembled. These are strongly connected to the formats, + actionformats and nvcsformats styles described above. Those styles + are lists. The first member of that list gets expanded into + ${vcs_info_msg_0_}, the second into ${vcs_info_msg_1_} and the Nth + into ${vcs_info_msg_N-1_}. (See the max-exports style above.) + + +All variables named VCS_INFO_* are for internal use only. + + + + +File: zsh.info, Node: vcs_info Hooks, Next: vcs_info Examples, Prev: vcs_info Variables, Up: Version Control Information + +26.5.7 Hooks in vcs_info +------------------------ + +Hooks are places in vcs_info where you can run your own code. That code +can communicate with the code that called it and through that, change +the system's behaviour. + +For configuration, hooks change the style context: + :vcs_info:VCS-STRING+HOOK-NAME:USER-CONTEXT:REPO-ROOT-NAME + +To register functions to a hook, you need to list them in the hooks +style in the appropriate context. + +Example: + zstyle ':vcs_info:*+foo:*' hooks bar baz + +This registers functions to the hook `foo' for all backends. In order to +avoid namespace problems, all registered function names are prepended by +a `+vi-', so the actual functions called for the `foo' hook are +`+vi-bar' and `+vi-baz'. + +If you would like to register a function to a hook regardless of the +current context, you may use the vcs_info_hookadd function. To remove a +function that was added like that, the vcs_info_hookdel function can be +used. + +If something seems weird, you can enable the `debug' boolean style in +the proper context and the hook-calling code will print what it tried +to execute and whether the function in question existed. + +When you register more than one function to a hook, all functions are +executed one after another until one function returns non-zero or until +all functions have been called. Context-sensitive hook functions are +executed before statically registered ones (the ones added by +vcs_info_hookadd). + +You may pass data between functions via an associative array, user_data. +For example: + + +vi-git-myfirsthook(){ + user_data[myval]=$myval + } + +vi-git-mysecondhook(){ + # do something with ${user_data[myval]} + } + +There are a number of variables that are special in hook contexts: + + +ret + The return value that the hooks system will return to the caller. + The default is an integer `zero'. If and how a changed ret value + changes the execution of the caller depends on the specific hook. + See the hook documentation below for details. + +hook_com + An associated array which is used for bidirectional communication + from the caller to hook functions. The used keys depend on the + specific hook. + +context + The active context of the hook. Functions that wish to change this + variable should make it local scope first. + +vcs + The current VCS after it was detected. The same values as in the + enable/disable style are used. Available in all hooks except + start-up. + + +Finally, the full list of currently available hooks: + + +start-up + Called after starting vcs_info but before the VCS in this + directory is determined. It can be used to deactivate vcs_info + temporarily if necessary. When ret is set to 1, vcs_info aborts + and does nothing; when set to 2, vcs_info sets up everything as if + no version control were active and exits. + +pre-get-data + Same as start-up but after the VCS was detected. + +gen-hg-bookmark-string + Called in the Mercurial backend when a bookmark string is + generated; the get-revision and get-bookmarks styles must be true. + + This hook gets the names of the Mercurial bookmarks that vcs_info + collected from `hg'. + + If a bookmark is active, the key ${hook_com[hg-active-bookmark]} is + set to its name. The key is otherwise unset. + + When setting ret to non-zero, the string in + ${hook_com[hg-bookmark-string]} will be used in the %m escape in + formats and actionformats and will be available in the global + backend_misc array as ${backend_misc[bookmarks]}. + +gen-applied-string + Called in the git (with stgit or during rebase or merge), and hg + (with mq) backends and in quilt support when the applied-string is + generated; the use-quilt zstyle must be true for quilt (the mq and + stgit backends are active by default). + + This hook gets the names of all applied patches which vcs_info + collected so far in the opposite order, which means that the first + argument is the top-most patch and so forth. + + When setting ret to non-zero, the string in + ${hook_com[applied-string]} will be available as %p in the + patch-format and nopatch-format styles. This hook is, in concert + with set-patch-format, responsible for %-escaping that value for + use in the prompt. (See *Note vcs_info Oddities::.) + +gen-unapplied-string + Called in the git (with stgit or during rebase), and hg (with mq) + backend and in quilt support when the unapplied-string is + generated; the get-unapplied style must be true. + + This hook gets the names of all unapplied patches which vcs_info + collected so far in order, which means that the first argument is + the patch next-in-line to be applied and so forth. + + When setting ret to non-zero, the string in + ${hook_com[unapplied-string]} will be available as %u in the + patch-format and nopatch-format styles. This hook is, in concert + with set-patch-format, responsible for %-escaping that value for + use in the prompt. (See *Note vcs_info Oddities::.) + +gen-mqguards-string + Called in the hg backend when guards-string is generated; the + get-mq style must be true (default). + + This hook gets the names of any active mq guards. + + When setting ret to non-zero, the string in + ${hook_com[guards-string]} will be used in the %g escape in the + patch-format and nopatch-format styles. + +no-vcs + This hooks is called when no version control system was detected. + + The `hook_com' parameter is not used. + +post-backend + Called as soon as the backend has finished collecting information. + + The `hook_com' keys available are as for the set-message hook. + +post-quilt + Called after the quilt support is done. The following information + is passed as arguments to the hook: 1. the quilt-support mode + (`addon' or `standalone'); 2. the directory that contains the + patch series; 3. the directory that holds quilt's status + information (the `.pc' directory) or the string "-nopc-" if that + directory wasn't found. + + The `hook_com' parameter is not used. + +set-branch-format + Called before `branchformat' is set. The only argument to the hook + is the format that is configured at this point. + + The `hook_com' keys considered are `branch' and `revision'. They + are set to the values figured out so far by vcs_info and any + change will be used directly when the actual replacement is done. + + If ret is set to non-zero, the string in + ${hook_com[branch-replace]} will be used unchanged as the `%b' + replacement in the variables set by vcs_info. + +set-hgrev-format + Called before a `hgrevformat' is set. The only argument to the + hook is the format that is configured at this point. + + The `hook_com' keys considered are `hash' and `localrev'. They + are set to the values figured out so far by vcs_info and any + change will be used directly when the actual replacement is done. + + If ret is set to non-zero, the string in ${hook_com[rev-replace]} + will be used unchanged as the `%i' replacement in the variables + set by vcs_info. + +pre-addon-quilt + This hook is used when vcs_info's quilt functionality is active in + "addon" mode (quilt used on top of a real version control system). + It is activated right before any quilt specific action is taken. + + Setting the `ret' variable in this hook to a non-zero value avoids + any quilt specific actions from being run at all. + +set-patch-format + This hook is used to control some of the possible expansions in + patch-format and nopatch-format styles with patch queue systems + such as quilt, mqueue and the like. + + This hook is used in the git, hg and quilt backends. + + The hook allows the control of the %p (${hook_com[applied]}) and %u + (${hook_com[unapplied]}) expansion in all backends that use the + hook. With the mercurial backend, the %g (${hook_com[guards]}) + expansion is controllable in addition to that. + + If ret is set to non-zero, the string in ${hook_com[patch-replace]} + will be used unchanged instead of an expanded format from + patch-format or nopatch-format. + + This hook is, in concert with the gen-applied-string or + gen-unapplied-string hooks if they are defined, responsible for + %-escaping the final patch-format value for use in the prompt. + (See *Note vcs_info Oddities::.) + +set-message + Called each time before a `vcs_info_msg_N_' message is set. It + takes two arguments; the first being the `N' in the message + variable name, the second is the currently configured formats or + actionformats. + + There are a number of `hook_com' keys, that are used here: + `action', `branch', `base', `base-name', `subdir', `staged', + `unstaged', `revision', `misc', `vcs' and one `miscN' entry for + each backend-specific data field (N starting at zero). They are + set to the values figured out so far by vcs_info and any change + will be used directly when the actual replacement is done. + + Since this hook is triggered multiple times (once for each + configured formats or actionformats), each of the `hook_com' keys + mentioned above (except for the miscN entries) has an `_orig' + counterpart, so even if you changed a value to your liking you can + still get the original value in the next run. Changing the `_orig' + values is probably not a good idea. + + If ret is set to non-zero, the string in ${hook_com[message]} will + be used unchanged as the message by vcs_info. + + +If all of this sounds rather confusing, take a look at *Note vcs_info +Examples:: and also in the Misc/vcs_info-examples file in the Zsh +source. They contain some explanatory code. + + + + +File: zsh.info, Node: vcs_info Examples, Prev: vcs_info Hooks, Up: Version Control Information + +26.5.8 Examples +--------------- + +Don't use vcs_info at all (even though it's in your prompt): + zstyle ':vcs_info:*' enable NONE + +Disable the backends for bzr and svk: + zstyle ':vcs_info:*' disable bzr svk + +Disable everything _but_ bzr and svk: + zstyle ':vcs_info:*' enable bzr svk + +Provide a special formats for git: + zstyle ':vcs_info:git:*' formats ' GIT, BABY! [%b]' + zstyle ':vcs_info:git:*' actionformats ' GIT ACTION! [%b|%a]' + +All %x expansion in all sorts of formats (formats, actionformats, +branchformat, you name it) are done using the `zformat' builtin from +the `zsh/zutil' module. That means you can do everything with these %x +items what zformat supports. In particular, if you want something that +is really long to have a fixed width, like a hash in a mercurial +branchformat, you can do this: %12.12i. That'll shrink the 40 character +hash to its 12 leading characters. The form is actually `%MIN.MAXx'. +More is possible. See *Note The zsh/zutil Module:: for details. + +Use the quicker bzr backend + zstyle ':vcs_info:bzr:*' use-simple true + +If you do use use-simple, please report if it does +`the-right-thing[tm]'. + +Display the revision number in yellow for bzr and svn: + zstyle ':vcs_info:(svn|bzr):*' \ + branchformat '%b%{'${fg[yellow]}'%}:%r' + +If you want colors, make sure you enclose the color codes in %{...%} if +you want to use the string provided by vcs_info in prompts. + +Here is how to print the VCS information as a command (not in a prompt): + alias vcsi='vcs_info command; vcs_info_lastmsg' + +This way, you can even define different formats for output via +vcs_info_lastmsg in the ':vcs_info:*:command:*' namespace. + +Now as promised, some code that uses hooks: say, you'd like to replace +the string `svn' by `subversion' in vcs_info's %s formats replacement. + +First, we will tell vcs_info to call a function when populating the +message variables with the gathered information: + zstyle ':vcs_info:*+set-message:*' hooks svn2subversion + +Nothing happens. Which is reasonable, since we didn't define the actual +function yet. To see what the hooks subsystem is trying to do, enable +the `debug' style: + zstyle ':vcs_info:*+*:*' debug true + +That should give you an idea what is going on. Specifically, the +function that we are looking for is `+vi-svn2subversion'. Note, the +`+vi-' prefix. So, everything is in order, just as documented. When you +are done checking out the debugging output, disable it again: + zstyle ':vcs_info:*+*:*' debug false + +Now, let's define the function: + + function +vi-svn2subversion() { + [[ ${hook_com[vcs_orig]} == svn ]] && hook_com[vcs]=subversion + } + +Simple enough. And it could have even been simpler, if only we had +registered our function in a less generic context. If we do it only in +the `svn' backend's context, we don't need to test which the active +backend is: + zstyle ':vcs_info:svn+set-message:*' hooks svn2subversion + + + function +vi-svn2subversion() { + hook_com[vcs]=subversion + } + +And finally a little more elaborate example, that uses a hook to create +a customised bookmark string for the hg backend. + +Again, we start off by registering a function: + zstyle ':vcs_info:hg+gen-hg-bookmark-string:*' hooks hgbookmarks + +And then we define the `+vi-hgbookmarks' function: + + function +vi-hgbookmarks() { + # The default is to connect all bookmark names by + # commas. This mixes things up a little. + # Imagine, there's one type of bookmarks that is + # special to you. Say, because it's *your* work. + # Those bookmarks look always like this: "sh/*" + # (because your initials are sh, for example). + # This makes the bookmarks string use only those + # bookmarks. If there's more than one, it + # concatenates them using commas. + # The bookmarks returned by `hg' are available in + # the function's positional parameters. + local s="${(Mj:,:)@:#sh/*}" + # Now, the communication with the code that calls + # the hook functions is done via the hook_com[] + # hash. The key at which the `gen-hg-bookmark-string' + # hook looks is `hg-bookmark-string'. So: + hook_com[hg-bookmark-string]=$s + # And to signal that we want to use the string we + # just generated, set the special variable `ret' to + # something other than the default zero: + ret=1 + return 0 + } + +Some longer examples and code snippets which might be useful are +available in the examples file located at Misc/vcs_info-examples in the +Zsh source directory. + +This concludes our guided tour through zsh's vcs_info. + + + + +File: zsh.info, Node: Prompt Themes, Next: ZLE Functions, Prev: Version Control Information, Up: User Contributions + +26.6 Prompt Themes +================== + + + +26.6.1 Installation +------------------- + +You should make sure all the functions from the Functions/Prompts +directory of the source distribution are available; they all begin with +the string `prompt_' except for the special function`promptinit'. You +also need the `colors' and `add-zsh-hook' functions from Functions/Misc. +All these functions may already be installed on your system; if not, +you will need to find them and copy them. The directory should appear +as one of the elements of the fpath array (this should already be the +case if they were installed), and at least the function promptinit +should be autoloaded; it will autoload the rest. Finally, to initialize +the use of the system you need to call the promptinit function. The +following code in your .zshrc will arrange for this; assume the +functions are stored in the directory ~/myfns: + + + fpath=(~/myfns $fpath) + autoload -U promptinit + promptinit + + +26.6.2 Theme Selection +---------------------- + +Use the prompt command to select your preferred theme. This command +may be added to your .zshrc following the call to promptinit in order +to start zsh with a theme already selected. + + +prompt [ -c | -l ] +prompt [ -p | -h ] [ THEME ... ] +prompt [ -s ] THEME [ ARG ... ] + Set or examine the prompt theme. With no options and a THEME + argument, the theme with that name is set as the current theme. + The available themes are determined at run time; use the -l option + to see a list. The special THEME `random' selects at random one + of the available themes and sets your prompt to that. + + In some cases the THEME may be modified by one or more arguments, + which should be given after the theme name. See the help for each + theme for descriptions of these arguments. + + Options are: + + + -c + Show the currently selected theme and its parameters, if any. + + -l + List all available prompt themes. + + -p + Preview the theme named by THEME, or all themes if no THEME + is given. + + -h + Show help for the theme named by THEME, or for the prompt + function if no THEME is given. + + -s + Set THEME as the current theme and save state. + +prompt_THEME_setup + Each available THEME has a setup function which is called by the + prompt function to install that theme. This function may define + other functions as necessary to maintain the prompt, including + functions used to preview the prompt or provide help for its use. + You should not normally call a theme's setup function directly. + + + +26.6.3 Utility Themes +--------------------- + + +prompt off + The theme `off' sets all the prompt variables to minimal values + with no special effects. + +prompt default + The theme `default' sets all prompt variables to the same state as + if an interactive zsh was started with no initialization files. + +prompt restore + The special theme `restore' erases all theme settings and sets + prompt variables to their state before the first time the `prompt' + function was run, provided each theme has properly defined its + cleanup (see below). + + Note that you can undo `prompt off' and `prompt default' with + `prompt restore', but a second restore does not undo the first. + + + +26.6.4 Writing Themes +--------------------- + +The first step for adding your own theme is to choose a name for it, +and create a file `prompt_NAME_setup' in a directory in your fpath, +such as ~/myfns in the example above. The file should at minimum +contain assignments for the prompt variables that your theme wishes to +modify. By convention, themes use PS1, PS2, RPS1, etc., rather than +the longer PROMPT and RPROMPT. + +The file is autoloaded as a function in the current shell context, so +it may contain any necessary commands to customize your theme, including +defining additional functions. To make some complex tasks easier, your +setup function may also do any of the following: + + +Assign prompt_opts + The array prompt_opts may be assigned any of "bang", "cr", + "percent", "sp", and/or "subst" as values. The corresponding + setopts (promptbang, etc.) are turned on, all other prompt-related + options are turned off. The prompt_opts array preserves setopts + even beyond the scope of localoptions, should your function need + that. + +Modify precmd and preexec + Use of add-zsh-hook is recommended. The precmd and preexec hooks + are automatically adjusted if the prompt theme changes or is + disabled. + +Declare cleanup + If your function makes any other changes that should be undone + when the theme is disabled, your setup function may call + prompt_cleanup COMMAND + where COMMAND should be suitably quoted. If your theme is ever + disabled or replaced by another, COMMAND is executed with eval. + You may declare more than one such cleanup hook. + +Define preview + Define or autoload a function prompt_NAME_preview to display a + simulated version of your prompt. A simple default previewer is + defined by promptinit for themes that do not define their own. + This preview function is called by `prompt -p'. + +Provide help + Define or autoload a function prompt_NAME_help to display + documentation or help text for your theme. This help function is + called by `prompt -h'. + + + + +File: zsh.info, Node: ZLE Functions, Next: Exception Handling, Prev: Prompt Themes, Up: User Contributions + +26.7 ZLE Functions +================== + + + +26.7.1 Widgets +-------------- + +These functions all implement user-defined ZLE widgets (see *Note Zsh +Line Editor::) which can be bound to keystrokes in interactive shells. +To use them, your .zshrc should contain lines of the form + + + autoload FUNCTION + zle -N FUNCTION + +followed by an appropriate bindkey command to associate the function +with a key sequence. Suggested bindings are described below. + + +bash-style word functions + If you are looking for functions to implement moving over and + editing words in the manner of bash, where only alphanumeric + characters are considered word characters, you can use the + functions described in the next section. The following is + sufficient: + + + autoload -U select-word-style + select-word-style bash + + +forward-word-match, backward-word-match +kill-word-match, backward-kill-word-match +transpose-words-match, capitalize-word-match +up-case-word-match, down-case-word-match +delete-whole-word-match, select-word-match +select-word-style, match-word-context, match-words-by-style + The first eight `-match' functions are drop-in replacements for the + builtin widgets without the suffix. By default they behave in a + similar way. However, by the use of styles and the function + select-word-style, the way words are matched can be altered. + select-word-match is intended to be used as a text object in vi + mode but with custom word styles. For comparison, the widgets + described in *Note Text Objects:: use fixed definitions of words, + compatible with the vim editor. + + The simplest way of configuring the functions is to use + select-word-style, which can either be called as a normal function + with the appropriate argument, or invoked as a user-defined widget + that will prompt for the first character of the word style to be + used. The first time it is invoked, the first eight -match + functions will automatically replace the builtin versions, so they + do not need to be loaded explicitly. + + The word styles available are as follows. Only the first character + is examined. + + + bash + Word characters are alphanumeric characters only. + + normal + As in normal shell operation: word characters are + alphanumeric characters plus any characters present in the + string given by the parameter $WORDCHARS. + + shell + Words are complete shell command arguments, possibly + including complete quoted strings, or any tokens special to + the shell. + + whitespace + Words are any set of characters delimited by whitespace. + + default + Restore the default settings; this is usually the same as + `normal'. + + + All but `default' can be input as an upper case character, which + has the same effect but with subword matching turned on. In this + case, words with upper case characters are treated specially: each + separate run of upper case characters, or an upper case character + followed by any number of other characters, is considered a word. + The style subword-range can supply an alternative character range + to the default `[:upper:]'; the value of the style is treated as + the contents of a `[...]' pattern (note that the outer brackets + should not be supplied, only those surrounding named ranges). + + More control can be obtained using the zstyle command, as + described in *Note The zsh/zutil Module::. Each style is looked + up in the context :zle:WIDGET where WIDGET is the name of the + user-defined widget, not the name of the function implementing it, + so in the case of the definitions supplied by select-word-style the + appropriate contexts are :zle:forward-word, and so on. The + function select-word-style itself always defines styles for the + context `:zle:*' which can be overridden by more specific (longer) + patterns as well as explicit contexts. + + The style word-style specifies the rules to use. This may have the + following values. + + + normal + Use the standard shell rules, i.e. alphanumerics and + $WORDCHARS, unless overridden by the styles word-chars or + word-class. + + specified + Similar to normal, but _only_ the specified characters, and + not also alphanumerics, are considered word characters. + + unspecified + The negation of specified. The given characters are those + which will _not_ be considered part of a word. + + shell + Words are obtained by using the syntactic rules for + generating shell command arguments. In addition, special + tokens which are never command arguments such as `()' are + also treated as words. + + whitespace + Words are whitespace-delimited strings of characters. + + + The first three of those rules usually use $WORDCHARS, but the + value in the parameter can be overridden by the style word-chars, + which works in exactly the same way as $WORDCHARS. In addition, + the style word-class uses character class syntax to group + characters and takes precedence over word-chars if both are set. + The word-class style does not include the surrounding brackets of + the character class; for example, `-:[:alnum:]' is a valid + word-class to include all alphanumerics plus the characters `-' + and `:'. Be careful including `]', `^' and `-' as these are + special inside character classes. + + word-style may also have `-subword' appended to its value to turn + on subword matching, as described above. + + The style skip-chars is mostly useful for transpose-words and + similar functions. If set, it gives a count of characters + starting at the cursor position which will not be considered part + of the word and are treated as space, regardless of what they + actually are. For example, if + + + zstyle ':zle:transpose-words' skip-chars 1 + + has been set, and transpose-words-match is called with the cursor + on the X of fooXbar, where X can be any character, then the + resulting expression is barXfoo. + + Finer grained control can be obtained by setting the style + word-context to an array of pairs of entries. Each pair of + entries consists of a PATTERN and a SUBCONTEXT. The shell + argument the cursor is on is matched against each PATTERN in turn + until one matches; if it does, the context is extended by a colon + and the corresponding SUBCONTEXT. Note that the test is made + against the original word on the line, with no stripping of + quotes. Special handling is done between words: the current + context is examined and if it contains the string between the word + is set to a single space; else if it is contains the string back, + the word before the cursor is considered, else the word after + cursor is considered. Some examples are given below. + + The style skip-whitespace-first is only used with the forward-word + widget. If it is set to true, then forward-word skips any + non-word-characters, followed by any non-word-characters: this is + similar to the behaviour of other word-orientated widgets, and + also that used by other editors, however it differs from the + standard zsh behaviour. When using select-word-style the widget + is set in the context :zle:* to true if the word style is bash and + false otherwise. It may be overridden by setting it in the more + specific context :zle:forward-word*. + + It is possible to create widgets with specific behaviour by + defining a new widget implemented by the appropriate generic + function, then setting a style for the context of the specific + widget. For example, the following defines a widget + backward-kill-space-word using backward-kill-word-match, the + generic widget implementing backward-kill-word behaviour, and + ensures that the new widget always implements space-delimited + behaviour. + + + zle -N backward-kill-space-word backward-kill-word-match + zstyle :zle:backward-kill-space-word word-style space + + The widget backward-kill-space-word can now be bound to a key. + + Here are some further examples of use of the styles, actually + taken from the simplified interface in select-word-style: + + + zstyle ':zle:*' word-style standard + zstyle ':zle:*' word-chars '' + + Implements bash-style word handling for all widgets, i.e. only + alphanumerics are word characters; equivalent to setting the + parameter WORDCHARS empty for the given context. + + + style ':zle:*kill*' word-style space + + Uses space-delimited words for widgets with the word `kill' in the + name. Neither of the styles word-chars nor word-class is used in + this case. + + Here are some examples of use of the word-context style to extend + the context. + + + zstyle ':zle:*' word-context \ + "*/*" filename "[[:space:]]" whitespace + zstyle ':zle:transpose-words:whitespace' word-style shell + zstyle ':zle:transpose-words:filename' word-style normal + zstyle ':zle:transpose-words:filename' word-chars '' + + This provides two different ways of using transpose-words + depending on whether the cursor is on whitespace between words or + on a filename, here any word containing a /. On whitespace, + complete arguments as defined by standard shell rules will be + transposed. In a filename, only alphanumerics will be transposed. + Elsewhere, words will be transposed using the default style for + :zle:transpose-words. + + The word matching and all the handling of zstyle settings is + actually implemented by the function match-words-by-style. This + can be used to create new user-defined widgets. The calling + function should set the local parameter curcontext to :zle:WIDGET, + create the local parameter matched_words and call + match-words-by-style with no arguments. On return, matched_words + will be set to an array with the elements: (1) the start of the + line (2) the word before the cursor (3) any non-word characters + between that word and the cursor (4) any non-word character at the + cursor position plus any remaining non-word characters before the + next word, including all characters specified by the skip-chars + style, (5) the word at or following the cursor (6) any non-word + characters following that word (7) the remainder of the line. Any + of the elements may be an empty string; the calling function + should test for this to decide whether it can perform its function. + + If the variable matched_words is defined by the caller to + match-words-by-style as an associative array (local -A + matched_words), then the seven values given above should be + retrieved from it as elements named start, word-before-cursor, + ws-before-cursor, ws-after-cursor, word-after-cursor, + ws-after-word, and end. In addition the element is-word-start is + 1 if the cursor is on the start of a word or subword, or on white + space before it (the cases can be distinguished by testing the + ws-after-cursor element) and 0 otherwise. This form is + recommended for future compatibility. + + It is possible to pass options with arguments to + match-words-by-style to override the use of styles. The options + are: + -w + WORD-STYLE + + -s + SKIP-CHARS + + -c + WORD-CLASS + + -C + WORD-CHARS + + -r + SUBWORD-RANGE + + For example, match-words-by-style -w shell -c 0 may be used to + extract the command argument around the cursor. + + The word-context style is implemented by the function + match-word-context. This should not usually need to be called + directly. + +bracketed-paste-magic + The bracketed-paste widget (see *Note Miscellaneous:: in *Note Zle + Widgets::) inserts pasted text literally into the editor buffer + rather than interpret it as keystrokes. This disables some common + usages where the self-insert widget is replaced in order to + accomplish some extra processing. An example is the contributed + url-quote-magic widget described below. + + The bracketed-paste-magic widget is meant to replace + bracketed-paste with a wrapper that re-enables these self-insert + actions, and other actions as selected by zstyles. Therefore this + widget is installed with + + autoload -Uz bracketed-paste-magic + zle -N bracketed-paste bracketed-paste-magic + + Other than enabling some widget processing, bracketed-paste-magic + attempts to replicate bracketed-paste as faithfully as possible. + + The following zstyles may be set to control processing of pasted + text. All are looked up in the context `:bracketed-paste-magic'. + + + active-widgets + A list of patterns matching widget names that should be + activated during the paste. All other key sequences are + processed as self-insert-unmeta. The default is `self-*' so + any user-defined widgets named with that prefix are active + along with the builtin self-insert. + + If this style is not set (explicitly deleted) or set to an + empty value, no widgets are active and the pasted text is + inserted literally. If the value includes `undefined-key', + any unknown sequences are discarded from the pasted text. + + inactive-keys + The inverse of active-widgets, a list of key sequences that + always use self-insert-unmeta even when bound to an active + widget. Note that this is a list of literal key sequences, + not patterns. + + paste-init + A list of function names, called in widget context (but not + as widgets). The functions are called in order until one of + them returns a non-zero status. The parameter `PASTED' + contains the initial state of the pasted text. All other ZLE + parameters such as `BUFFER' have their normal values and + side-effects, and full history is available, so for example + paste-init functions may move words from BUFFER into PASTED + to make those words visible to the active-widgets. + + A non-zero return from a paste-init function does _not_ + prevent the paste itself from proceeding. + + Loading bracketed-paste-magic defines backward-extend-paste, a + helper function for use in paste-init. + + + zstyle :bracketed-paste-magic paste-init \ + backward-extend-paste + + When a paste would insert into the middle of a word or append + text to a word already on the line, backward-extend-paste + moves the prefix from LBUFFER into PASTED so that the + active-widgets see the full word so far. This may be useful + with url-quote-magic. + + paste-finish + Another list of function names called in order until one + returns non-zero. These functions are called _after_ the + pasted text has been processed by the active-widgets, but + _before_ it is inserted into `BUFFER'. ZLE parameters have + their normal values and side-effects. + + A non-zero return from a paste-finish function does _not_ + prevent the paste itself from proceeding. + + Loading bracketed-paste-magic also defines quote-paste, a + helper function for use in paste-finish. + + + zstyle :bracketed-paste-magic paste-finish \ + quote-paste + zstyle :bracketed-paste-magic:finish quote-style \ + qqq + + When the pasted text is inserted into BUFFER, it is quoted + per the quote-style value. To forcibly turn off the built-in + numeric prefix quoting of bracketed-paste, use: + + + zstyle :bracketed-paste-magic:finish quote-style \ + none + + + _Important:_ During active-widgets processing of the paste (after + paste-init and before paste-finish), BUFFER starts empty and + history is restricted, so cursor motions, etc., may not pass + outside of the pasted content. Text assigned to BUFFER by the + active widgets is copied back into PASTED before paste-finish. + +copy-earlier-word + This widget works like a combination of insert-last-word and + copy-prev-shell-word. Repeated invocations of the widget retrieve + earlier words on the relevant history line. With a numeric + argument N, insert the Nth word from the history line; N may be + negative to count from the end of the line. + + If insert-last-word has been used to retrieve the last word on a + previous history line, repeated invocations will replace that word + with earlier words from the same line. + + Otherwise, the widget applies to words on the line currently being + edited. The widget style can be set to the name of another widget + that should be called to retrieve words. This widget must accept + the same three arguments as insert-last-word. + +cycle-completion-positions + After inserting an unambiguous string into the command line, the + new function based completion system may know about multiple + places in this string where characters are missing or differ from + at least one of the possible matches. It will then place the + cursor on the position it considers to be the most interesting + one, i.e. the one where one can disambiguate between as many + matches as possible with as little typing as possible. + + This widget allows the cursor to be easily moved to the other + interesting spots. It can be invoked repeatedly to cycle between + all positions reported by the completion system. + +delete-whole-word-match + This is another function which works like the -match functions + described immediately above, i.e. using styles to decide the word + boundaries. However, it is not a replacement for any existing + function. + + The basic behaviour is to delete the word around the cursor. + There is no numeric argument handling; only the single word around + the cursor is considered. If the widget contains the string kill, + the removed text will be placed in the cutbuffer for future + yanking. This can be obtained by defining kill-whole-word-match + as follows: + + + zle -N kill-whole-word-match delete-whole-word-match + + and then binding the widget kill-whole-word-match. + +up-line-or-beginning-search, down-line-or-beginning-search + These widgets are similar to the builtin functions + up-line-or-search and down-line-or-search: if in a multiline + buffer they move up or down within the buffer, otherwise they + search for a history line matching the start of the current line. + In this case, however, they search for a line which matches the + current line up to the current cursor position, in the manner of + history-beginning-search-backward and -forward, rather than the + first word on the line. + +edit-command-line + Edit the command line using your visual editor, as in ksh. + + + bindkey -M vicmd v edit-command-line + +expand-absolute-path + Expand the file name under the cursor to an absolute path, + resolving symbolic links. Where possible, the initial path + segment is turned into a named directory or reference to a user's + home directory. + +history-search-end + This function implements the widgets + history-beginning-search-backward-end and + history-beginning-search-forward-end. These commands work by first + calling the corresponding builtin widget (see *Note History + Control::) and then moving the cursor to the end of the line. The + original cursor position is remembered and restored before calling + the builtin widget a second time, so that the same search is + repeated to look farther through the history. + + Although you autoload only one function, the commands to use it are + slightly different because it implements two widgets. + + + zle -N history-beginning-search-backward-end \ + history-search-end + zle -N history-beginning-search-forward-end \ + history-search-end + bindkey '\e^P' history-beginning-search-backward-end + bindkey '\e^N' history-beginning-search-forward-end + +history-beginning-search-menu + This function implements yet another form of history searching. + The text before the cursor is used to select lines from the + history, as for history-beginning-search-backward except that all + matches are shown in a numbered menu. Typing the appropriate + digits inserts the full history line. Note that leading zeroes + must be typed (they are only shown when necessary for removing + ambiguity). The entire history is searched; there is no + distinction between forwards and backwards. + + With a numeric argument, the search is not anchored to the start of + the line; the string typed by the use may appear anywhere in the + line in the history. + + If the widget name contains `-end' the cursor is moved to the end + of the line inserted. If the widget name contains `-space' any + space in the text typed is treated as a wildcard and can match + anything (hence a leading space is equivalent to giving a numeric + argument). Both forms can be combined, for example: + + + zle -N history-beginning-search-menu-space-end \ + history-beginning-search-menu + +history-pattern-search + The function history-pattern-search implements widgets which prompt + for a pattern with which to search the history backwards or + forwards. The pattern is in the usual zsh format, however the + first character may be ^ to anchor the search to the start of the + line, and the last character may be $ to anchor the search to the + end of the line. If the search was not anchored to the end of the + line the cursor is positioned just after the pattern found. + + The commands to create bindable widgets are similar to those in the + example immediately above: + + + autoload -U history-pattern-search + zle -N history-pattern-search-backward history-pattern-search + zle -N history-pattern-search-forward history-pattern-search + +incarg + Typing the keystrokes for this widget with the cursor placed on or + to the left of an integer causes that integer to be incremented by + one. With a numeric argument, the number is incremented by the + amount of the argument (decremented if the numeric argument is + negative). The shell parameter incarg may be set to change the + default increment to something other than one. + + + bindkey '^X+' incarg + +incremental-complete-word + This allows incremental completion of a word. After starting this + command, a list of completion choices can be shown after every + character you type, which you can delete with ^H or DEL. Pressing + return accepts the completion so far and returns you to normal + editing (that is, the command line is _not_ immediately executed). + You can hit TAB to do normal completion, ^G to abort back to the + state when you started, and ^D to list the matches. + + This works only with the new function based completion system. + + + bindkey '^Xi' incremental-complete-word + +insert-composed-char + This function allows you to compose characters that don't appear + on the keyboard to be inserted into the command line. The command + is followed by two keys corresponding to ASCII characters (there + is no prompt). For accented characters, the two keys are a base + character followed by a code for the accent, while for other + special characters the two characters together form a mnemonic for + the character to be inserted. The two-character codes are a + subset of those given by RFC 1345 (see for example + `http://www.faqs.org/rfcs/rfc1345.html'). + + The function may optionally be followed by up to two characters + which replace one or both of the characters read from the + keyboard; if both characters are supplied, no input is read. For + example, insert-composed-char a: can be used within a widget to + insert an a with umlaut into the command line. This has the + advantages over use of a literal character that it is more + portable. + + For best results zsh should have been built with support for + multibyte characters (configured with -enable-multibyte); however, + the function works for the limited range of characters available + in single-byte character sets such as ISO-8859-1. + + The character is converted into the local representation and + inserted into the command line at the cursor position. (The + conversion is done within the shell, using whatever facilities the + C library provides.) With a numeric argument, the character and + its code are previewed in the status line + + The function may be run outside zle in which case it prints the + character (together with a newline) to standard output. Input is + still read from keystrokes. + + See insert-unicode-char for an alternative way of inserting Unicode + characters using their hexadecimal character number. + + The set of accented characters is reasonably complete up to Unicode + character U+0180, the set of special characters less so. However, + it is very sporadic from that point. Adding new characters is + easy, however; see the function define-composed-chars. Please + send any additions to zsh-workers@zsh.org. + + The codes for the second character when used to accent the first + are as follows. Note that not every character can take every + accent. + ! + Grave. + + ' + Acute. + + > + Circumflex. + + ? + Tilde. (This is not ~ as RFC 1345 does not assume that + character is present on the keyboard.) + + - + Macron. (A horizontal bar over the base character.) + + ( + Breve. (A shallow dish shape over the base character.) + + . + Dot above the base character, or in the case of i no dot, or + in the case of L and l a centered dot. + + : + Diaeresis (Umlaut). + + c + Cedilla. + + _ + Underline, however there are currently no underlined + characters. + + / + Stroke through the base character. + + " + Double acute (only supported on a few letters). + + ; + Ogonek. (A little forward facing hook at the bottom right of + the character.) + + < + Caron. (A little v over the letter.) + + 0 + Circle over the base character. + + 2 + Hook over the base character. + + 9 + Horn over the base character. + + The most common characters from the Arabic, Cyrillic, Greek and + Hebrew alphabets are available; consult RFC 1345 for the + appropriate sequences. In addition, a set of two letter codes not + in RFC 1345 are available for the double-width characters + corresponding to ASCII characters from ! to ~ (0x21 to 0x7e) by + preceding the character with ^, for example ^A for a double-width + A. + + The following other two-character sequences are understood. + + + ASCII characters + These are already present on most keyboards: + <( + Left square bracket + + // + Backslash (solidus) + + )> + Right square bracket + + (! + Left brace (curly bracket) + + !! + Vertical bar (pipe symbol) + + !) + Right brace (curly bracket) + + '? + Tilde + + Special letters + Characters found in various variants of the Latin alphabet: + ss + Eszett (scharfes S) + + D-, d- + Eth + + TH, th + Thorn + + kk + Kra + + 'n + 'n + + NG, ng + Ng + + OI, oi + Oi + + yr + yr + + ED + ezh + + Currency symbols + + Ct + Cent + + Pd + Pound sterling (also lira and others) + + Cu + Currency + + Ye + Yen + + Eu + Euro (N.B. not in RFC 1345) + + Punctuation characters + References to "right" quotes indicate the shape (like a 9 + rather than 6) rather than their grammatical use. (For + example, a "right" low double quote is used to open + quotations in German.) + !I + Inverted exclamation mark + + BB + Broken vertical bar + + SE + Section + + Co + Copyright + + -a + Spanish feminine ordinal indicator + + << + Left guillemet + + -- + Soft hyphen + + Rg + Registered trade mark + + PI + Pilcrow (paragraph) + + -o + Spanish masculine ordinal indicator + + >> + Right guillemet + + ?I + Inverted question mark + + -1 + Hyphen + + -N + En dash + + -M + Em dash + + -3 + Horizontal bar + + :3 + Vertical ellipsis + + .3 + Horizontal midline ellipsis + + !2 + Double vertical line + + =2 + Double low line + + '6 + Left single quote + + '9 + Right single quote + + .9 + "Right" low quote + + 9' + Reversed "right" quote + + "6 + Left double quote + + "9 + Right double quote + + :9 + "Right" low double quote + + 9" + Reversed "right" double quote + + /- + Dagger + + /= + Double dagger + + Mathematical symbols + + DG + Degree + + -2, +-, -+ + - sign, +/- sign, -/+ sign + + 2S + Superscript 2 + + 3S + Superscript 3 + + 1S + Superscript 1 + + My + Micro + + .M + Middle dot + + 14 + Quarter + + 12 + Half + + 34 + Three quarters + + *X + Multiplication + + -: + Division + + %0 + Per mille + + FA, TE, /0 + For all, there exists, empty set + + dP, DE, NB + Partial derivative, delta (increment), del (nabla) + + (-, -) + Element of, contains + + *P, +Z + Product, sum + + *-, Ob, Sb + Asterisk, ring, bullet + + RT, 0(, 00 + Root sign, proportional to, infinity + + Other symbols + + cS, cH, cD, cC + Card suits: spades, hearts, diamonds, clubs + + Md, M8, M2, Mb, Mx, MX + Musical notation: crotchet (quarter note), quaver + (eighth note), semiquavers (sixteenth notes), flag sign, + natural sign, sharp sign + + Fm, Ml + Female, male + + Accents on their own + + '> + Circumflex (same as caret, ^) + + '! + Grave (same as backtick, `) + + ', + Cedilla + + ': + Diaeresis (Umlaut) + + 'm + Macron + + " + Acute + + +insert-files + This function allows you type a file pattern, and see the results + of the expansion at each step. When you hit return, all + expansions are inserted into the command line. + + + bindkey '^Xf' insert-files + +insert-unicode-char + When first executed, the user inputs a set of hexadecimal digits. + This is terminated with another call to insert-unicode-char. The + digits are then turned into the corresponding Unicode character. + For example, if the widget is bound to ^XU, the character sequence + `^XU 4 c ^XU' inserts L (Unicode U+004c). + + See insert-composed-char for a way of inserting characters using a + two-character mnemonic. + +narrow-to-region [ -p PRE ] [ -P POST ] + [ -S STATEPM | -R STATEPM | [ -l LBUFVAR ] [ -r RBUFVAR ] ] + [ -n ] [ START END ] +narrow-to-region-invisible + Narrow the editable portion of the buffer to the region between + the cursor and the mark, which may be in either order. The region + may not be empty. + + narrow-to-region may be used as a widget or called as a function + from a user-defined widget; by default, the text outside the + editable area remains visible. A recursive-edit is performed and + the original widening status is then restored. Various options + and arguments are available when it is called as a function. + + The options -p PRETEXT and -P POSTTEXT may be used to replace the + text before and after the display for the duration of the + function; either or both may be an empty string. + + If the option -n is also given, PRETEXT or POSTTEXT will only be + inserted if there is text before or after the region respectively + which will be made invisible. + + Two numeric arguments may be given which will be used instead of + the cursor and mark positions. + + The option -S STATEPM is used to narrow according to the other + options while saving the original state in the parameter with name + STATEPM, while the option -R STATEPM is used to restore the state + from the parameter; note in both cases the _name_ of the parameter + is required. In the second case, other options and arguments are + irrelevant. When this method is used, no recursive-edit is + performed; the calling widget should call this function with the + option -S, perform its own editing on the command line or pass + control to the user via `zle recursive-edit', then call this + function with the option -R. The argument STATEPM must be a + suitable name for an ordinary parameter, except that parameters + beginning with the prefix _ntr_ are reserved for use within + narrow-to-region. Typically the parameter will be local to the + calling function. + + The options -l LBUFVAR and -r RBUFVAR may be used to specify + parameters where the widget will store the resulting text from the + operation. The parameter LBUFVAR will contain LBUFFER and RBUFVAR + will contain RBUFFER. Neither of these two options may be used + with -S or -R. + + narrow-to-region-invisible is a simple widget which calls + narrow-to-region with arguments which replace any text outside the + region with `...'. It does not take any arguments. + + The display is restored (and the widget returns) upon any zle + command which would usually cause the line to be accepted or + aborted. Hence an additional such command is required to accept + or abort the current line. + + The return status of both widgets is zero if the line was + accepted, else non-zero. + + Here is a trivial example of a widget using this feature. + local state + narrow-to-region -p $'Editing restricted region\n' \ + -P '' -S state + zle recursive-edit + narrow-to-region -R state + +predict-on + This set of functions implements predictive typing using history + search. After predict-on, typing characters causes the editor to + look backward in the history for the first line beginning with + what you have typed so far. After predict-off, editing returns to + normal for the line found. In fact, you often don't even need to + use predict-off, because if the line doesn't match something in + the history, adding a key performs standard completion, and then + inserts itself if no completions were found. However, editing in + the middle of a line is liable to confuse prediction; see the + toggle style below. + + With the function based completion system (which is needed for + this), you should be able to type TAB at almost any point to + advance the cursor to the next ``interesting'' character position + (usually the end of the current word, but sometimes somewhere in + the middle of the word). And of course as soon as the entire line + is what you want, you can accept with return, without needing to + move the cursor to the end first. + + The first time predict-on is used, it creates several additional + widget functions: + + + delete-backward-and-predict + Replaces the backward-delete-char widget. You do not need to + bind this yourself. + + insert-and-predict + Implements predictive typing by replacing the self-insert + widget. You do not need to bind this yourself. + + predict-off + Turns off predictive typing. + + Although you autoload only the predict-on function, it is + necessary to create a keybinding for predict-off as well. + + + zle -N predict-on + zle -N predict-off + bindkey '^X^Z' predict-on + bindkey '^Z' predict-off + +read-from-minibuffer + This is most useful when called as a function from inside a + widget, but will work correctly as a widget in its own right. It + prompts for a value below the current command line; a value may be + input using all of the standard zle operations (and not merely the + restricted set available when executing, for example, + execute-named-cmd). The value is then returned to the calling + function in the parameter $REPLY and the editing buffer restored + to its previous state. If the read was aborted by a keyboard + break (typically ^G), the function returns status 1 and $REPLY is + not set. + + If one argument is supplied to the function it is taken as a + prompt, otherwise `? ' is used. If two arguments are supplied, + they are the prompt and the initial value of $LBUFFER, and if a + third argument is given it is the initial value of $RBUFFER. This + provides a default value and starting cursor placement. Upon + return the entire buffer is the value of $REPLY. + + One option is available: `-k NUM' specifies that NUM characters + are to be read instead of a whole line. The line editor is not + invoked recursively in this case, so depending on the terminal + settings the input may not be visible, and only the input keys are + placed in $REPLY, not the entire buffer. Note that unlike the + read builtin NUM must be given; there is no default. + + The name is a slight misnomer, as in fact the shell's own + minibuffer is not used. Hence it is still possible to call + executed-named-cmd and similar functions while reading a value. + +replace-argument, replace-argument-edit + The function replace-argument can be used to replace a command + line argument in the current command line or, if the current + command line is empty, in the last command line executed (the new + command line is not executed). Arguments are as delimited by + standard shell syntax, + + If a numeric argument is given, that specifies the argument to be + replaced. 0 means the command name, as in history expansion. A + negative numeric argument counts backward from the last word. + + If no numeric argument is given, the current argument is replaced; + this is the last argument if the previous history line is being + used. + + The function prompts for a replacement argument. + + If the widget contains the string edit, for example is defined as + + + zle -N replace-argument-edit replace-argument + + then the function presents the current value of the argument for + editing, otherwise the editing buffer for the replacement is + initially empty. + +replace-string, replace-pattern +replace-string-again, replace-pattern-again + The function replace-string implements three widgets. If defined + under the same name as the function, it prompts for two strings; + the first (source) string will be replaced by the second + everywhere it occurs in the line editing buffer. + + If the widget name contains the word `pattern', for example by + defining the widget using the command `zle -N replace-pattern + replace-string', then the matching is performed using zsh + patterns. All zsh extended globbing patterns can be used in the + source string; note that unlike filename generation the pattern + does not need to match an entire word, nor do glob qualifiers have + any effect. In addition, the replacement string can contain + parameter or command substitutions. Furthermore, a `&' in the + replacement string will be replaced with the matched source + string, and a backquoted digit `\N' will be replaced by the Nth + parenthesised expression matched. The form `\{N}' may be used to + protect the digit from following digits. + + If the widget instead contains the word `regex' (or `regexp'), + then the matching is performed using regular expressions, + respecting the setting of the option RE_MATCH_PCRE (see the + description of the function regexp-replace below). The special + replacement facilities described above for pattern matching are + available. + + By default the previous source or replacement string will not be + offered for editing. However, this feature can be activated by + setting the style edit-previous in the context :zle:WIDGET (for + example, :zle:replace-string) to true. In addition, a positive + numeric argument forces the previous values to be offered, a + negative or zero argument forces them not to be. + + The function replace-string-again can be used to repeat the + previous replacement; no prompting is done. As with + replace-string, if the name of the widget contains the word + `pattern' or `regex', pattern or regular expression matching is + performed, else a literal string replacement. Note that the + previous source and replacement text are the same whether pattern, + regular expression or string matching is used. + + In addition, replace-string shows the previous replacement above + the prompt, so long as there was one during the current session; + if the source string is empty, that replacement will be repeated + without the widget prompting for a replacement string. + + For example, starting from the line: + + + print This line contains fan and fond + + and invoking replace-pattern with the source string `f(?)n' and + the replacement string `c\1r' produces the not very useful line: + + + print This line contains car and cord + + The range of the replacement string can be limited by using the + narrow-to-region-invisible widget. One limitation of the current + version is that undo will cycle through changes to the replacement + and source strings before undoing the replacement itself. + +send-invisible + This is similar to read-from-minibuffer in that it may be called + as a function from a widget or as a widget of its own, and + interactively reads input from the keyboard. However, the input + being typed is concealed and a string of asterisks (`*') is shown + instead. The value is saved in the parameter $INVISIBLE to which + a reference is inserted into the editing buffer at the restored + cursor position. If the read was aborted by a keyboard break + (typically ^G) or another escape from editing such as push-line, + $INVISIBLE is set to empty and the original buffer is restored + unchanged. + + If one argument is supplied to the function it is taken as a + prompt, otherwise `Non-echoed text: ' is used (as in emacs). If a + second and third argument are supplied they are used to begin and + end the reference to $INVISIBLE that is inserted into the buffer. + The default is to open with ${, then INVISIBLE, and close with }, + but many other effects are possible. + +smart-insert-last-word + This function may replace the insert-last-word widget, like so: + + + zle -N insert-last-word smart-insert-last-word + + With a numeric argument, or when passed command line arguments in + a call from another widget, it behaves like insert-last-word, + except that words in comments are ignored when + INTERACTIVE_COMMENTS is set. + + Otherwise, the rightmost ``interesting'' word from the previous + command is found and inserted. The default definition of + ``interesting'' is that the word contains at least one alphabetic + character, slash, or backslash. This definition may be overridden + by use of the match style. The context used to look up the style + is the widget name, so usually the context is :insert-last-word. + However, you can bind this function to different widgets to use + different patterns: + + + zle -N insert-last-assignment smart-insert-last-word + zstyle :insert-last-assignment match '[[:alpha:]][][[:alnum:]]#=*' + bindkey '\e=' insert-last-assignment + + If no interesting word is found and the auto-previous style is set + to a true value, the search continues upward through the history. + When auto-previous is unset or false (the default), the widget + must be invoked repeatedly in order to search earlier history + lines. + +transpose-lines + Only useful with a multi-line editing buffer; the lines here are + lines within the current on-screen buffer, not history lines. The + effect is similar to the function of the same name in Emacs. + + Transpose the current line with the previous line and move the + cursor to the start of the next line. Repeating this (which can + be done by providing a positive numeric argument) has the effect + of moving the line above the cursor down by a number of lines. + + With a negative numeric argument, requires two lines above the + cursor. These two lines are transposed and the cursor moved to the + start of the previous line. Using a numeric argument less than -1 + has the effect of moving the line above the cursor up by minus that + number of lines. + +url-quote-magic + This widget replaces the built-in self-insert to make it easier to + type URLs as command line arguments. As you type, the input + character is analyzed and, if it may need quoting, the current + word is checked for a URI scheme. If one is found and the current + word is not already in quotes, a backslash is inserted before the + input character. + + Styles to control quoting behavior: + + + url-metas + This style is looked up in the context + `:url-quote-magic:SCHEME' (where SCHEME is that of the + current URL, e.g. "ftp"). The value is a string listing the + characters to be treated as globbing metacharacters when + appearing in a URL using that scheme. The default is to + quote all zsh extended globbing characters, excluding '<' and + '>' but including braces (as in brace expansion). See also + url-seps. + + url-seps + Like url-metas, but lists characters that should be + considered command separators, redirections, history + references, etc. The default is to quote the standard set of + shell separators, excluding those that overlap with the + extended globbing characters, but including '<' and '>' and + the first character of $histchars. + + url-globbers + This style is looked up in the context `:url-quote-magic'. + The values form a list of command names that are expected to + do their own globbing on the URL string. This implies that + they are aliased to use the `noglob' modifier. When the + first word on the line matches one of the values _and_ the + URL refers to a local file (see url-local-schema), only the + url-seps characters are quoted; the url-metas are left alone, + allowing them to affect command-line parsing, completion, + etc. The default values are a literal `noglob' plus (when + the zsh/parameter module is available) any commands aliased + to the helper function `urlglobber' or its alias `globurl'. + + url-local-schema + This style is always looked up in the context `:urlglobber', + even though it is used by both url-quote-magic and + urlglobber. The values form a list of URI schema that should + be treated as referring to local files by their real local + path names, as opposed to files which are specified relative + to a web-server-defined document root. The defaults are + "ftp" and "file". + + url-other-schema + Like url-local-schema, but lists all other URI schema upon + which urlglobber and url-quote-magic should act. If the URI + on the command line does not have a scheme appearing either + in this list or in url-local-schema, it is not magically + quoted. The default values are "http", "https", and "ftp". + When a scheme appears both here and in url-local-schema, it + is quoted differently depending on whether the command name + appears in url-globbers. + + + Loading url-quote-magic also defines a helper function `urlglobber' + and aliases `globurl' to `noglob urlglobber'. This function takes + a local URL apart, attempts to pattern-match the local file + portion of the URL path, and then puts the results back into URL + format again. + +vi-pipe + This function reads a movement command from the keyboard and then + prompts for an external command. The part of the buffer covered by + the movement is piped to the external command and then replaced by + the command's output. If the movement command is bound to vi-pipe, + the current line is used. + + The function serves as an example for reading a vi movement command + from within a user-defined widget. + +which-command + This function is a drop-in replacement for the builtin widget + which-command. It has enhanced behaviour, in that it correctly + detects whether or not the command word needs to be expanded as an + alias; if so, it continues tracing the command word from the + expanded alias until it reaches the command that will be executed. + + The style whence is available in the context :zle:$WIDGET; this + may be set to an array to give the command and options that will + be used to investigate the command word found. The default is + whence -c. + +zcalc-auto-insert + This function is useful together with the zcalc function described + in *Note Mathematical Functions::. It should be bound to a key + representing a binary operator such as `+', `-', `*' or `/'. When + running in zcalc, if the key occurs at the start of the line or + immediately following an open parenthesis, the text "ans " is + inserted before the representation of the key itself. This allows + easy use of the answer from the previous calculation in the + current line. The text to be inserted before the symbol typed can + be modified by setting the variable ZCALC_AUTO_INSERT_PREFIX. + + Hence, for example, typing `+12' followed by return adds 12 to the + previous result. + + If zcalc is in RPN mode (-r option) the effect of this binding is + automatically suppressed as operators alone on a line are + meaningful. + + When not in zcalc, the key simply inserts the symbol itself. + + + +26.7.2 Utility Functions +------------------------ + +These functions are useful in constructing widgets. They should be +loaded with `autoload -U FUNCTION' and called as indicated from +user-defined widgets. + + +split-shell-arguments + This function splits the line currently being edited into shell + arguments and whitespace. The result is stored in the array + reply. The array contains all the parts of the line in order, + starting with any whitespace before the first argument, and + finishing with any whitespace after the last argument. Hence (so + long as the option KSH_ARRAYS is not set) whitespace is given by + odd indices in the array and arguments by even indices. Note that + no stripping of quotes is done; joining together all the elements + of reply in order is guaranteed to produce the original line. + + The parameter REPLY is set to the index of the word in reply which + contains the character after the cursor, where the first element + has index 1. The parameter REPLY2 is set to the index of the + character under the cursor in that word, where the first character + has index 1. + + Hence reply, REPLY and REPLY2 should all be made local to the + enclosing function. + + See the function modify-current-argument, described below, for an + example of how to call this function. + +modify-current-argument [ EXPR-USING-$ARG | FUNC ] + This function provides a simple method of allowing user-defined + widgets to modify the command line argument under the cursor (or + immediately to the left of the cursor if the cursor is between + arguments). + + The argument can be an expression which when evaluated operates on + the shell parameter ARG, which will have been set to the command + line argument under the cursor. The expression should be suitably + quoted to prevent it being evaluated too early. + + Alternatively, if the argument does not contain the string ARG, it + is assumed to be a shell function, to which the current command + line argument is passed as the only argument. The function should + set the variable REPLY to the new value for the command line + argument. If the function returns non-zero status, so does the + calling function. + + For example, a user-defined widget containing the following code + converts the characters in the argument under the cursor into all + upper case: + + + modify-current-argument '${(U)ARG}' + + The following strips any quoting from the current word (whether + backslashes or one of the styles of quotes), and replaces it with + single quoting throughout: + + + modify-current-argument '${(qq)${(Q)ARG}}' + + The following performs directory expansion on the command line + argument and replaces it by the absolute path: + + + expand-dir() { + REPLY=${~1} + REPLY=${REPLY:a} + } + modify-current-argument expand-dir + + In practice the function expand-dir would probably not be defined + within the widget where modify-current-argument is called. + + + +26.7.3 Styles +------------- + +The behavior of several of the above widgets can be controlled by the +use of the zstyle mechanism. In particular, widgets that interact with +the completion system pass along their context to any completions that +they invoke. + + +break-keys + This style is used by the incremental-complete-word widget. Its + value should be a pattern, and all keys matching this pattern will + cause the widget to stop incremental completion without the key + having any further effect. Like all styles used directly by + incremental-complete-word, this style is looked up using the + context `:incremental'. + +completer + The incremental-complete-word and insert-and-predict widgets set + up their top-level context name before calling completion. This + allows one to define different sets of completer functions for + normal completion and for these widgets. For example, to use + completion, approximation and correction for normal completion, + completion and correction for incremental completion and only + completion for prediction one could use: + + + zstyle ':completion:*' completer \ + _complete _correct _approximate + zstyle ':completion:incremental:*' completer \ + _complete _correct + zstyle ':completion:predict:*' completer \ + _complete + + It is a good idea to restrict the completers used in prediction, + because they may be automatically invoked as you type. The _list + and _menu completers should never be used with prediction. The + _approximate, _correct, _expand, and _match completers may be + used, but be aware that they may change characters anywhere in the + word behind the cursor, so you need to watch carefully that the + result is what you intended. + +cursor + The insert-and-predict widget uses this style, in the context + `:predict', to decide where to place the cursor after completion + has been tried. Values are: + + + complete + The cursor is left where it was when completion finished, but + only if it is after a character equal to the one just + inserted by the user. If it is after another character, this + value is the same as `key'. + + key + The cursor is left after the Nth occurrence of the character + just inserted, where N is the number of times that character + appeared in the word before completion was attempted. In + short, this has the effect of leaving the cursor after the + character just typed even if the completion code found out + that no other characters need to be inserted at that position. + + + Any other value for this style unconditionally leaves the cursor + at the position where the completion code left it. + +list + When using the incremental-complete-word widget, this style says + if the matches should be listed on every key press (if they fit on + the screen). Use the context prefix `:completion:incremental'. + + The insert-and-predict widget uses this style to decide if the + completion should be shown even if there is only one possible + completion. This is done if the value of this style is the string + always. In this case the context is `:predict' (_not_ + `:completion:predict'). + +match + This style is used by smart-insert-last-word to provide a pattern + (using full EXTENDED_GLOB syntax) that matches an interesting word. + The context is the name of the widget to which + smart-insert-last-word is bound (see above). The default behavior + of smart-insert-last-word is equivalent to: + + + zstyle :insert-last-word match '*[[:alpha:]/\\]*' + + However, you might want to include words that contain spaces: + + + zstyle :insert-last-word match '*[[:alpha:][:space:]/\\]*' + + Or include numbers as long as the word is at least two characters + long: + + + zstyle :insert-last-word match '*([[:digit:]]?|[[:alpha:]/\\])*' + + The above example causes redirections like "2>" to be included. + +prompt + The incremental-complete-word widget shows the value of this style + in the status line during incremental completion. The string + value may contain any of the following substrings in the manner of + the PS1 and other prompt parameters: + + + %c + Replaced by the name of the completer function that generated + the matches (without the leading underscore). + + %l + When the list style is set, replaced by `...' if the list of + matches is too long to fit on the screen and with an empty + string otherwise. If the list style is `false' or not set, + `%l' is always removed. + + %n + Replaced by the number of matches generated. + + %s + Replaced by `-no match-', `-no prefix-', or an empty string + if there is no completion matching the word on the line, if + the matches have no common prefix different from the word on + the line, or if there is such a common prefix, respectively. + + %u + Replaced by the unambiguous part of all matches, if there is + any, and if it is different from the word on the line. + + + Like `break-keys', this uses the `:incremental' context. + +stop-keys + This style is used by the incremental-complete-word widget. Its + value is treated similarly to the one for the break-keys style + (and uses the same context: `:incremental'). However, in this + case all keys matching the pattern given as its value will stop + incremental completion and will then execute their usual function. + +toggle + This boolean style is used by predict-on and its related widgets in + the context `:predict'. If set to one of the standard `true' + values, predictive typing is automatically toggled off in + situations where it is unlikely to be useful, such as when editing + a multi-line buffer or after moving into the middle of a line and + then deleting a character. The default is to leave prediction + turned on until an explicit call to predict-off. + +verbose + This boolean style is used by predict-on and its related widgets in + the context `:predict'. If set to one of the standard `true' + values, these widgets display a message below the prompt when the + predictive state is toggled. This is most useful in combination + with the toggle style. The default does not display these + messages. + +widget + This style is similar to the command style: For widget functions + that use zle to call other widgets, this style can sometimes be + used to override the widget which is called. The context for this + style is the name of the calling widget (_not_ the name of the + calling function, because one function may be bound to multiple + widget names). + + + zstyle :copy-earlier-word widget smart-insert-last-word + + Check the documentation for the calling widget or function to + determine whether the widget style is used. + + + + +File: zsh.info, Node: Exception Handling, Next: MIME Functions, Prev: ZLE Functions, Up: User Contributions + +26.8 Exception Handling +======================= + +Two functions are provided to enable zsh to provide exception handling +in a form that should be familiar from other languages. + + +throw EXCEPTION + The function throw throws the named EXCEPTION. The name is an + arbitrary string and is only used by the throw and catch + functions. An exception is for the most part treated the same as a + shell error, i.e. an unhandled exception will cause the shell to + abort all processing in a function or script and to return to the + top level in an interactive shell. + +catch EXCEPTION-PATTERN + The function catch returns status zero if an exception was thrown + and the pattern EXCEPTION-PATTERN matches its name. Otherwise it + returns status 1. EXCEPTION-PATTERN is a standard shell pattern, + respecting the current setting of the EXTENDED_GLOB option. An + alias catch is also defined to prevent the argument to the + function from matching filenames, so patterns may be used + unquoted. Note that as exceptions are not fundamentally different + from other shell errors it is possible to catch shell errors by + using an empty string as the exception name. The shell variable + CAUGHT is set by catch to the name of the exception caught. It is + possible to rethrow an exception by calling the throw function + again once an exception has been caught. + + +The functions are designed to be used together with the always construct +described in *Note Complex Commands::. This is important as only this +construct provides the required support for exceptions. A typical +example is as follows. + + + { + # "try" block + # ... nested code here calls "throw MyExcept" + } always { + # "always" block + if catch MyExcept; then + print "Caught exception MyExcept" + elif catch ''; then + print "Caught a shell error. Propagating..." + throw '' + fi + # Other exceptions are not handled but may be caught further + # up the call stack. + } + +If all exceptions should be caught, the following idiom might be +preferable. + + + { + # ... nested code here throws an exception + } always { + if catch *; then + case $CAUGHT in + (MyExcept) + print "Caught my own exception" + ;; + (*) + print "Caught some other exception" + ;; + esac + fi + } + +In common with exception handling in other languages, the exception may +be thrown by code deeply nested inside the `try' block. However, note +that it must be thrown inside the current shell, not in a subshell +forked for a pipeline, parenthesised current-shell construct, or some +form of command or process substitution. + +The system internally uses the shell variable EXCEPTION to record the +name of the exception between throwing and catching. One drawback of +this scheme is that if the exception is not handled the variable +EXCEPTION remains set and may be incorrectly recognised as the name of +an exception if a shell error subsequently occurs. Adding unset +EXCEPTION at the start of the outermost layer of any code that uses +exception handling will eliminate this problem. + + + + +File: zsh.info, Node: MIME Functions, Next: Mathematical Functions, Prev: Exception Handling, Up: User Contributions + +26.9 MIME Functions +=================== + +Three functions are available to provide handling of files recognised by +extension, for example to dispatch a file text.ps when executed as a +command to an appropriate viewer. + + +zsh-mime-setup [ -fv ] [ -l [ SUFFIX ... ] ] +zsh-mime-handler [ -l ] COMMAND ARGUMENT ... + These two functions use the files ~/.mime.types and + /etc/mime.types, which associate types and extensions, as well as + ~/.mailcap and /etc/mailcap files, which associate types and the + programs that handle them. These are provided on many systems + with the Multimedia Internet Mail Extensions. + + To enable the system, the function zsh-mime-setup should be + autoloaded and run. This allows files with extensions to be + treated as executable; such files be completed by the function + completion system. The function zsh-mime-handler should not need + to be called by the user. + + The system works by setting up suffix aliases with `alias -s'. + Suffix aliases already installed by the user will not be + overwritten. + + For suffixes defined in lower case, upper case variants will also + automatically be handled (e.g. PDF is automatically handled if + handling for the suffix pdf is defined), but not vice versa. + + Repeated calls to zsh-mime-setup do not override the existing + mapping between suffixes and executable files unless the option -f + is given. Note, however, that this does not override existing + suffix aliases assigned to handlers other than zsh-mime-handler. + + Calling zsh-mime-setup with the option -l lists the existing + mappings without altering them. Suffixes to list (which may + contain pattern characters that should be quoted from immediate + interpretation on the command line) may be given as additional + arguments, otherwise all suffixes are listed. + + Calling zsh-mime-setup with the option -v causes verbose output to + be shown during the setup operation. + + The system respects the mailcap flags needsterminal and + copiousoutput, see man page mailcap(4). + + The functions use the following styles, which are defined with the + zstyle builtin command (*Note The zsh/zutil Module::). They + should be defined before zsh-mime-setup is run. The contexts used + all start with :mime:, with additional components in some cases. + It is recommended that a trailing * (suitably quoted) be appended + to style patterns in case the system is extended in future. Some + examples are given below. + + For files that have multiple suffixes, e.g. .pdf.gz, where the + context includes the suffix it will be looked up starting with the + longest possible suffix until a match for the style is found. For + example, if .pdf.gz produces a match for the handler, that will be + used; otherwise the handler for .gz will be used. Note that, + owing to the way suffix aliases work, it is always required that + there be a handler for the shortest possible suffix, so in this + example .pdf.gz can only be handled if .gz is also handled (though + not necessarily in the same way). Alternatively, if no handling + for .gz on its own is needed, simply adding the command + + + alias -s gz=zsh-mime-handler + + to the initialisation code is sufficient; .gz will not be handled + on its own, but may be in combination with other suffixes. + + + current-shell + If this boolean style is true, the mailcap handler for the + context in question is run using the eval builtin instead of + by starting a new sh process. This is more efficient, but + may not work in the occasional cases where the mailcap + handler uses strict POSIX syntax. + + disown + If this boolean style is true, mailcap handlers started in the + background will be disowned, i.e. not subject to job control + within the parent shell. Such handlers nearly always produce + their own windows, so the only likely harmful side effect of + setting the style is that it becomes harder to kill jobs from + within the shell. + + execute-as-is + This style gives a list of patterns to be matched against + files passed for execution with a handler program. If the + file matches the pattern, the entire command line is executed + in its current form, with no handler. This is useful for + files which might have suffixes but nonetheless be executable + in their own right. If the style is not set, the pattern + *(*) *(/) is used; hence executable files are executed + directly and not passed to a handler, and the option AUTO_CD + may be used to change to directories that happen to have MIME + suffixes. + + execute-never + This style is useful in combination with execute-as-is. It is + set to an array of patterns corresponding to full paths to + files that should never be treated as executable, even if the + file passed to the MIME handler matches execute-as-is. This + is useful for file systems that don't handle execute + permission or that contain executables from another operating + system. For example, if /mnt/windows is a Windows mount, then + + + zstyle ':mime:*' execute-never '/mnt/windows/*' + + will ensure that any files found in that area will be + executed as MIME types even if they are executable. As this + example shows, the complete file name is matched against the + pattern, regardless of how the file was passed to the + handler. The file is resolved to a full path using the :P + modifier described in *Note Modifiers::; this means that + symbolic links are resolved where possible, so that links + into other file systems behave in the correct fashion. + + file-path + Used if the style find-file-in-path is true for the same + context. Set to an array of directories that are used for + searching for the file to be handled; the default is the + command path given by the special parameter path. The shell + option PATH_DIRS is respected; if that is set, the + appropriate path will be searched even if the name of the + file to be handled as it appears on the command line contains + a `/'. The full context is :mime:.SUFFIX:, as described for + the style handler. + + find-file-in-path + If set, allows files whose names do not contain absolute paths + to be searched for in the command path or the path specified + by the file-path style. If the file is not found in the + path, it is looked for locally (whether or not the current + directory is in the path); if it is not found locally, the + handler will abort unless the handle-nonexistent style is + set. Files found in the path are tested as described for the + style execute-as-is. The full context is :mime:.SUFFIX:, as + described for the style handler. + + flags + Defines flags to go with a handler; the context is as for the + handler style, and the format is as for the flags in mailcap. + + handle-nonexistent + By default, arguments that don't correspond to files are not + passed to the MIME handler in order to prevent it from + intercepting commands found in the path that happen to have + suffixes. This style may be set to an array of extended glob + patterns for arguments that will be passed to the handler + even if they don't exist. If it is not explicitly set it + defaults to [[:alpha:]]#:/* which allows URLs to be passed to + the MIME handler even though they don't exist in that format + in the file system. The full context is :mime:.SUFFIX:, as + described for the style handler. + + handler + Specifies a handler for a suffix; the suffix is given by the + context as :mime:.SUFFIX:, and the format of the handler is + exactly that in mailcap. Note in particular the `.' and + trailing colon to distinguish this use of the context. This + overrides any handler specified by the mailcap files. If the + handler requires a terminal, the flags style should be set to + include the word needsterminal, or if the output is to be + displayed through a pager (but not if the handler is itself a + pager), it should include copiousoutput. + + mailcap + A list of files in the format of ~/.mailcap and /etc/mailcap + to be read during setup, replacing the default list which + consists of those two files. The context is :mime:. A + in + the list will be replaced by the default files. + + mailcap-priorities + This style is used to resolve multiple mailcap entries for + the same MIME type. It consists of an array of the following + elements, in descending order of priority; later entries will + be used if earlier entries are unable to resolve the entries + being compared. If none of the tests resolve the entries, + the first entry encountered is retained. + + + files + The order of files (entries in the mailcap style) read. + Earlier files are preferred. (Note this does not + resolve entries in the same file.) + + priority + The priority flag from the mailcap entry. The priority + is an integer from 0 to 9 with the default value being 5. + + flags + The test given by the mailcap-prio-flags option is used + to resolve entries. + + place + Later entries are preferred; as the entries are strictly + ordered, this test always succeeds. + + + Note that as this style is handled during initialisation, the + context is always :mime:, with no discrimination by suffix. + + mailcap-prio-flags + This style is used when the keyword flags is encountered in + the list of tests specified by the mailcap-priorities style. + It should be set to a list of patterns, each of which is + tested against the flags specified in the mailcap entry (in + other words, the sets of assignments found with some entries + in the mailcap file). Earlier patterns in the list are + preferred to later ones, and matched patterns are preferred + to unmatched ones. + + mime-types + A list of files in the format of ~/.mime.types and + /etc/mime.types to be read during setup, replacing the + default list which consists of those two files. The context + is :mime:. A + in the list will be replaced by the default + files. + + never-background + If this boolean style is set, the handler for the given + context is always run in the foreground, even if the flags + provided in the mailcap entry suggest it need not be (for + example, it doesn't require a terminal). + + pager + If set, will be used instead of $PAGER or more to handle + suffixes where the copiousoutput flag is set. The context is + as for handler, i.e. :mime:.SUFFIX: for handling a file with + the given SUFFIX. + + + Examples: + + + zstyle ':mime:*' mailcap ~/.mailcap /usr/local/etc/mailcap + zstyle ':mime:.txt:' handler less %s + zstyle ':mime:.txt:' flags needsterminal + + When zsh-mime-setup is subsequently run, it will look for mailcap + entries in the two files given. Files of suffix .txt will be + handled by running `less FILE.TXT'. The flag needsterminal is set + to show that this program must run attached to a terminal. + + As there are several steps to dispatching a command, the following + should be checked if attempting to execute a file by extension + .EXT does not have the expected effect. + + The command `alias -s EXT' should show `ps=zsh-mime-handler'. If + it shows something else, another suffix alias was already + installed and was not overwritten. If it shows nothing, no + handler was installed: this is most likely because no handler was + found in the .mime.types and mailcap combination for .ext files. + In that case, appropriate handling should be added to + ~/.mime.types and mailcap. + + If the extension is handled by zsh-mime-handler but the file is + not opened correctly, either the handler defined for the type is + incorrect, or the flags associated with it are in appropriate. + Running zsh-mime-setup -l will show the handler and, if there are + any, the flags. A %s in the handler is replaced by the file + (suitably quoted if necessary). Check that the handler program + listed lists and can be run in the way shown. Also check that the + flags needsterminal or copiousoutput are set if the handler needs + to be run under a terminal; the second flag is used if the output + should be sent to a pager. An example of a suitable mailcap entry + for such a program is: + + + text/html; /usr/bin/lynx '%s'; needsterminal + + Running `zsh-mime-handler -l COMMAND LINE' prints the command line + that would be executed, simplified to remove the effect of any + flags, and quoted so that the output can be run as a complete zsh + command line. This is used by the completion system to decide how + to complete after a file handled by zsh-mime-setup. + +pick-web-browser + This function is separate from the two MIME functions described + above and can be assigned directly to a suffix: + + + autoload -U pick-web-browser + alias -s html=pick-web-browser + + It is provided as an intelligent front end to dispatch a web + browser. It may be run as either a function or a shell script. + The status 255 is returned if no browser could be started. + + Various styles are available to customize the choice of browsers: + + + browser-style + The value of the style is an array giving preferences in + decreasing order for the type of browser to use. The values + of elements may be + + + running + Use a GUI browser that is already running when an X + Window display is available. The browsers listed in the + x-browsers style are tried in order until one is found; + if it is, the file will be displayed in that browser, so + the user may need to check whether it has appeared. If + no running browser is found, one is not started. + Browsers other than Firefox, Opera and Konqueror are + assumed to understand the Mozilla syntax for opening a + URL remotely. + + x + Start a new GUI browser when an X Window display is + available. Search for the availability of one of the + browsers listed in the x-browsers style and start the + first one that is found. No check is made for an already + running browser. + + tty + Start a terminal-based browser. Search for the + availability of one of the browsers listed in the + tty-browsers style and start the first one that is found. + + + If the style is not set the default running x tty is used. + + x-browsers + An array in decreasing order of preference of browsers to use + when running under the X Window System. The array consists + of the command name under which to start the browser. They + are looked up in the context :mime: (which may be extended in + future, so appending `*' is recommended). For example, + + + zstyle ':mime:*' x-browsers opera konqueror firefox + + specifies that pick-web-browser should first look for a + running instance of Opera, Konqueror or Firefox, in that + order, and if it fails to find any should attempt to start + Opera. The default is firefox mozilla netscape opera + konqueror. + + tty-browsers + An array similar to x-browsers, except that it gives browsers + to use when no X Window display is available. The default is + elinks links lynx. + + command + If it is set this style is used to pick the command used to + open a page for a browser. The context is + :mime:browser:new:$browser: to start a new browser or + :mime:browser:running:$browser: to open a URL in a browser + already running on the current X display, where $browser is + the value matched in the x-browsers or tty-browsers style. + The escape sequence %b in the style's value will be replaced + by the browser, while %u will be replaced by the URL. If the + style is not set, the default for all new instances is + equivalent to %b %u and the defaults for using running + browsers are equivalent to the values kfmclient openURL %u for + Konqueror, firefox -new-tab %u for Firefox, opera -newpage %u + for Opera, and %b -remote "openUrl(%u)" for all others. + + + + + +File: zsh.info, Node: Mathematical Functions, Next: User Configuration Functions, Prev: MIME Functions, Up: User Contributions + +26.10 Mathematical Functions +============================ + + +zcalc [ -erf ] [ EXPRESSION ... ] + A reasonably powerful calculator based on zsh's arithmetic + evaluation facility. The syntax is similar to that of formulae in + most programming languages; see *Note Arithmetic Evaluation:: for + details. + + Non-programmers should note that, as in many other programming + languages, expressions involving only integers (whether constants + without a `.', variables containing such constants as strings, or + variables declared to be integers) are by default evaluated using + integer arithmetic, which is not how an ordinary desk calculator + operates. To force floating point operation, pass the option -f; + see further notes below. + + If the file ~/.zcalcrc exists it will be sourced inside the + function once it is set up and about to process the command line. + This can be used, for example, to set shell options; emulate -L zsh + and setopt extendedglob are in effect at this point. Any failure + to source the file if it exists is treated as fatal. As with + other initialisation files, the directory $ZDOTDIR is used instead + of $HOME if it is set. + + The mathematical library zsh/mathfunc will be loaded if it is + available; see *Note The zsh/mathfunc Module::. The mathematical + functions correspond to the raw system libraries, so trigonometric + functions are evaluated using radians, and so on. + + Each line typed is evaluated as an expression. The prompt shows a + number, which corresponds to a positional parameter where the + result of that calculation is stored. For example, the result of + the calculation on the line preceded by `4> ' is available as $4. + The last value calculated is available as ans. Full command line + editing, including the history of previous calculations, is + available; the history is saved in the file ~/.zcalc_history. To + exit, enter a blank line or type `:q' on its own (`q' is allowed + for historical compatibility). + + A line ending with a single backslash is treated in the same + fashion as it is in command line editing: the backslash is + removed, the function prompts for more input (the prompt is + preceded by `...' to indicate this), and the lines are combined + into one to get the final result. In addition, if the input so + far contains more open than close parentheses zcalc will prompt + for more input. + + If arguments are given to zcalc on start up, they are used to + prime the first few positional parameters. A visual indication of + this is given when the calculator starts. + + The constants PI (3.14159...) and E (2.71828...) are provided. + Parameter assignment is possible, but note that all parameters + will be put into the global namespace unless the :local special + command is used. The function creates local variables whose names + start with _, so users should avoid doing so. The variables ans + (the last answer) and stack (the stack in RPN mode) may be + referred to directly; stack is an array but elements of it are + numeric. Various other special variables are used locally with + their standard meaning, for example compcontext, match, mbegin, + mend, psvar. + + The output base can be initialised by passing the option `-#BASE', + for example `zcalc -#16' (the `#' may have to be quoted, depending + on the globbing options set). + + If the option `-e' is set, the function runs non-interactively: + the arguments are treated as expressions to be evaluated as if + entered interactively line by line. + + If the option `-f' is set, all numbers are treated as floating + point, hence for example the expression `3/4' evaluates to 0.75 + rather than 0. Options must appear in separate words. + + If the option `-r' is set, RPN (Reverse Polish Notation) mode is + entered. This has various additional properties: + Stack + Evaluated values are maintained in a stack; this is contained + in an array named stack with the most recent value in + ${stack[1]}. + + Operators and functions + If the line entered matches an operator (+, -, *, /, **, ^, | + or &) or a function supplied by the zsh/mathfunc library, the + bottom element or elements of the stack are popped to use as + the argument or arguments. The higher elements of stack + (least recent) are used as earlier arguments. The result is + then pushed into ${stack[1]}. + + Expressions + Other expressions are evaluated normally, printed, and added + to the stack as numeric values. The syntax within + expressions on a single line is normal shell arithmetic (not + RPN). + + Stack listing + If an integer follows the option -r with no space, then on + every evaluation that many elements of the stack, where + available, are printed instead of just the most recent + result. Hence, for example, zcalc -r4 shows $stack[4] to + $stack[1] each time results are printed. + + Duplication: = + The pseudo-operator = causes the most recent element of the + stack to be duplicated onto the stack. + + pop + The pseudo-function pop causes the most recent element of the + stack to be popped. A `>' on its own has the same effect. + + >IDENT + The expression > followed (with no space) by a shell + identifier causes the most recent element of the stack to be + popped and assigned to the variable with that name. The + variable is local to the zcalc function. + + in the standard zcalc prompt) is put on the stack. + + Exchange: xy + The pseudo-function xy causes the most recent two elements of + the stack to be exchanged. `<>' has the same effect. + + + The prompt is configurable via the parameter ZCALCPROMPT, which + undergoes standard prompt expansion. The index of the current + entry is stored locally in the first element of the array psvar, + which can be referred to in ZCALCPROMPT as `%1v'. The default + prompt is `%1v> '. + + The variable ZCALC_ACTIVE is set within the function and can be + tested by nested functions; it has the value rpn if RPN mode is + active, else 1. + + A few special commands are available; these are introduced by a + colon. For backward compatibility, the colon may be omitted for + certain commands. Completion is available if compinit has been + run. + + The output precision may be specified within zcalc by special + commands familiar from many calculators. + :norm + The default output format. It corresponds to the printf %g + specification. Typically this shows six decimal digits. + + :sci DIGITS + Scientific notation, corresponding to the printf %g output + format with the precision given by DIGITS. This produces + either fixed point or exponential notation depending on the + value output. + + :fix DIGITS + Fixed point notation, corresponding to the printf %f output + format with the precision given by DIGITS. + + :eng DIGITS + Exponential notation, corresponding to the printf %E output + format with the precision given by DIGITS. + + :raw + Raw output: this is the default form of the output from a + math evaluation. This may show more precision than the + number actually possesses. + + + Other special commands: + :!LINE... + Execute LINE... as a normal shell command line. Note that it + is executed in the context of the function, i.e. with local + variables. Space is optional after :!. + + :local ARG ... + Declare variables local to the function. Other variables may + be used, too, but they will be taken from or put into the + global scope. + + :function NAME [ BODY ] + Define a mathematical function or (with no BODY) delete it. + :function may be abbreviated to :func or simply :f. The NAME + may contain the same characters as a shell function name. + The function is defined using zmathfuncdef, see below. + + Note that zcalc takes care of all quoting. Hence for example: + + + :f cube $1 * $1 * $1 + + defines a function to cube the sole argument. Functions so + defined, or indeed any functions defined directly or + indirectly using functions -M, are available to execute by + typing only the name on the line in RPN mode; this pops the + appropriate number of arguments off the stack to pass to the + function, i.e. 1 in the case of the example cube function. + If there are optional arguments only the mandatory arguments + are supplied by this means. + + [#BASE] + This is not a special command, rather part of normal + arithmetic syntax; however, when this form appears on a line + by itself the default output radix is set to BASE. Use, for + example, `[#16]' to display hexadecimal output preceded by an + indication of the base, or `[##16]' just to display the raw + number in the given base. Bases themselves are always + specified in decimal. `[#]' restores the normal output format. + Note that setting an output base suppresses floating point + output; use `[#]' to return to normal operation. + + $VAR + Print out the value of var literally; does not affect the + calculation. To use the value of var, omit the leading `$'. + + + See the comments in the function for a few extra tips. + +min(ARG, ...) +max(ARG, ...) +sum(ARG, ...) +zmathfunc + The function zmathfunc defines the three mathematical functions + min, max, and sum. The functions min and max take one or more + arguments. The function sum takes zero or more arguments. + Arguments can be of different types (ints and floats). + + Not to be confused with the zsh/mathfunc module, described in + *Note The zsh/mathfunc Module::. + +zmathfuncdef [ MATHFUNC [ BODY ] ] + A convenient front end to functions -M. + + With two arguments, define a mathematical function named MATHFUNC + which can be used in any form of arithmetic evaluation. BODY is a + mathematical expression to implement the function. It may contain + references to position parameters $1, $2, ... to refer to + mandatory parameters and ${1:-DEFVALUE} ... to refer to optional + parameters. Note that the forms must be strictly adhered to for + the function to calculate the correct number of arguments. The + implementation is held in a shell function named + zsh_math_func_MATHFUNC; usually the user will not need to refer to + the shell function directly. Any existing function of the same + name is silently replaced. + + With one argument, remove the mathematical function MATHFUNC as + well as the shell function implementation. + + With no arguments, list all MATHFUNC functions in a form suitable + for restoring the definition. The functions have not necessarily + been defined by zmathfuncdef. + + + + +File: zsh.info, Node: User Configuration Functions, Next: Other Functions, Prev: Mathematical Functions, Up: User Contributions + +26.11 User Configuration Functions +================================== + +The zsh/newuser module comes with a function to aid in configuring +shell options for new users. If the module is installed, this function +can also be run by hand. It is available even if the module's default +behaviour, namely running the function for a new user logging in without +startup files, is inhibited. + + +zsh-newuser-install [ -f ] + The function presents the user with various options for customizing + their initialization scripts. Currently only ~/.zshrc is handled. + $ZDOTDIR/.zshrc is used instead if the parameter ZDOTDIR is set; + this provides a way for the user to configure a file without + altering an existing .zshrc. + + By default the function exits immediately if it finds any of the + files .zshenv, .zprofile, .zshrc, or .zlogin in the appropriate + directory. The option -f is required in order to force the + function to continue. Note this may happen even if .zshrc itself + does not exist. + + As currently configured, the function will exit immediately if the + user has root privileges; this behaviour cannot be overridden. + + Once activated, the function's behaviour is supposed to be + self-explanatory. Menus are present allowing the user to alter + the value of options and parameters. Suggestions for improvements + are always welcome. + + When the script exits, the user is given the opportunity to save + the new file or not; changes are not irreversible until this + point. However, the script is careful to restrict changes to the + file only to a group marked by the lines `# Lines configured by + zsh-newuser-install' and `# End of lines configured by + zsh-newuser-install'. In addition, the old version of .zshrc is + saved to a file with the suffix .zni appended. + + If the function edits an existing .zshrc, it is up to the user to + ensure that the changes made will take effect. For example, if + control usually returns early from the existing .zshrc the lines + will not be executed; or a later initialization file may override + options or parameters, and so on. The function itself does not + attempt to detect any such conflicts. + + + + +File: zsh.info, Node: Other Functions, Prev: User Configuration Functions, Up: User Contributions + +26.12 Other Functions +===================== + +There are a large number of helpful functions in the Functions/Misc +directory of the zsh distribution. Most are very simple and do not +require documentation here, but a few are worthy of special mention. + + + +26.12.1 Descriptions +-------------------- + + +colors + This function initializes several associative arrays to map color + names to (and from) the ANSI standard eight-color terminal codes. + These are used by the prompt theme system (*Note Prompt Themes::). + You seldom should need to run colors more than once. + + The eight base colors are: black, red, green, yellow, blue, + magenta, cyan, and white. Each of these has codes for foreground + and background. In addition there are seven intensity attributes: + bold, faint, standout, underline, blink, reverse, and conceal. + Finally, there are seven codes used to negate attributes: none + (reset all attributes to the defaults), normal (neither bold nor + faint), no-standout, no-underline, no-blink, no-reverse, and + no-conceal. + + Some terminals do not support all combinations of colors and + intensities. + + The associative arrays are: + + + color + colour + Map all the color names to their integer codes, and integer + codes to the color names. The eight base names map to the + foreground color codes, as do names prefixed with `fg-', such + as `fg-red'. Names prefixed with `bg-', such as `bg-blue', + refer to the background codes. The reverse mapping from code + to color yields base name for foreground codes and the bg- + form for backgrounds. + + Although it is a misnomer to call them `colors', these arrays + also map the other fourteen attributes from names to codes + and codes to names. + + fg + fg_bold + fg_no_bold + Map the eight basic color names to ANSI terminal escape + sequences that set the corresponding foreground text + properties. The fg sequences change the color without + changing the eight intensity attributes. + + bg + bg_bold + bg_no_bold + Map the eight basic color names to ANSI terminal escape + sequences that set the corresponding background properties. + The bg sequences change the color without changing the eight + intensity attributes. + + + In addition, the scalar parameters reset_color and bold_color are + set to the ANSI terminal escapes that turn off all attributes and + turn on bold intensity, respectively. + +fned [ -x NUM ] NAME + Same as zed -f. This function does not appear in the zsh + distribution, but can be created by linking zed to the name fned + in some directory in your fpath. + +is-at-least NEEDED [ PRESENT ] + Perform a greater-than-or-equal-to comparison of two strings + having the format of a zsh version number; that is, a string of + numbers and text with segments separated by dots or dashes. If + the PRESENT string is not provided, $ZSH_VERSION is used. + Segments are paired left-to-right in the two strings with leading + non-number parts ignored. If one string has fewer segments than + the other, the missing segments are considered zero. + + This is useful in startup files to set options and other state + that are not available in all versions of zsh. + + + is-at-least 3.1.6-15 && setopt NO_GLOBAL_RCS + is-at-least 3.1.0 && setopt HIST_REDUCE_BLANKS + is-at-least 2.6-17 || print "You can't use is-at-least here." + +nslookup [ ARG ... ] + This wrapper function for the nslookup command requires the + zsh/zpty module (see *Note The zsh/zpty Module::). It behaves + exactly like the standard nslookup except that it provides + customizable prompts (including a right-side prompt) and + completion of nslookup commands, host names, etc. (if you use the + function-based completion system). Completion styles may be set + with the context prefix `:completion:nslookup'. + + See also the pager, prompt and rprompt styles below. + +regexp-replace VAR REGEXP REPLACE + Use regular expressions to perform a global search and replace + operation on a variable. POSIX extended regular expressions are + used, unless the option RE_MATCH_PCRE has been set, in which case + Perl-compatible regular expressions are used (this requires the + shell to be linked against the pcre library). + + VAR is the name of the variable containing the string to be + matched. The variable will be modified directly by the function. + The variables MATCH, MBEGIN, MEND, match, mbegin, mend should be + avoided as these are used by the regular expression code. + + REGEXP is the regular expression to match against the string. + + REPLACE is the replacement text. This can contain parameter, + command and arithmetic expressions which will be replaced: in + particular, a reference to $MATCH will be replaced by the text + matched by the pattern. + + The return status is 0 if at least one match was performed, else 1. + +run-help CMD + This function is designed to be invoked by the run-help ZLE widget, + in place of the default alias. See `Accessing On-Line Help' + (*Note Utilities::) for setup instructions. + + In the discussion which follows, if CMD is a file system path, it + is first reduced to its rightmost component (the file name). + + Help is first sought by looking for a file named CMD in the + directory named by the HELPDIR parameter. If no file is found, an + assistant function, alias, or command named run-help-CMD is + sought. If found, the assistant is executed with the rest of the + current command line (everything after the command name CMD) as + its arguments. When neither file nor assistant is found, the + external command `man CMD' is run. + + An example assistant for the "ssh" command: + + + run-help-ssh() { + emulate -LR zsh + local -a args + # Delete the "-l username" option + zparseopts -D -E -a args l: + # Delete other options, leaving: host command + args=(${@:#-*}) + if [[ ${#args} -lt 2 ]]; then + man ssh + else + run-help $args[2] + fi + } + + Several of these assistants are provided in the Functions/Misc + directory. These must be autoloaded, or placed as executable + scripts in your search path, in order to be found and used by + run-help. + + + run-help-git + run-help-ip + run-help-openssl + run-help-p4 + run-help-sudo + run-help-svk + run-help-svn + Assistant functions for the git, ip, openssl, p4, sudo, svk, + and svn, commands. + + +tetris + Zsh was once accused of not being as complete as Emacs, because it + lacked a Tetris game. This function was written to refute this + vicious slander. + + This function must be used as a ZLE widget: + + + autoload -U tetris + zle -N tetris + bindkey KEYS tetris + + To start a game, execute the widget by typing the KEYS. Whatever + command line you were editing disappears temporarily, and your + keymap is also temporarily replaced by the Tetris control keys. + The previous editor state is restored when you quit the game (by + pressing `q') or when you lose. + + If you quit in the middle of a game, the next invocation of the + tetris widget will continue where you left off. If you lost, it + will start a new game. + +tetriscurses + This is a port of the above to zcurses. The input handling is + improved a bit so that moving a block sideways doesn't + automatically advance a timestep, and the graphics use unicode + block graphics. + + This version does not save the game state between invocations, and + is not invoked as a widget, but rather as: + + + autoload -U tetriscurses + tetriscurses + +zargs [ OPTION ... -- ] [ INPUT ... ] [ -- COMMAND [ ARG ... ] ] + This function has a similar purpose to GNU xargs. Instead of + reading lines of arguments from the standard input, it takes them + from the command line. This is useful because zsh, especially + with recursive glob operators, often can construct a command line + for a shell function that is longer than can be accepted by an + external command. + + The OPTION list represents options of the zargs command itself, + which are the same as those of xargs. The INPUT list is the + collection of strings (often file names) that become the arguments + of the command, analogous to the standard input of xargs. + Finally, the ARG list consists of those arguments (usually + options) that are passed to the COMMAND each time it runs. The + ARG list precedes the elements from the input list in each run. + If no COMMAND is provided, then no ARG list may be provided, and + in that event the default command is `print' with arguments `-r + --'. + + For example, to get a long ls listing of all non-hidden plain files + in the current directory or its subdirectories: + + + autoload -U zargs + zargs -- **/*(.) -- ls -ld -- + + The first and third occurrences of `--' are used to mark the end + of options for zargs and ls respectively to guard against + filenames starting with `-', while the second is used to separate + the list of files from the command to run (`ls -ld -'). + + The first `--' would also be needed if there was a chance the list + might be empty as in: + + + zargs -r -- ./*.back(#qN) -- rm -f + + In the event that the string `--' is or may be an INPUT, the -e + option may be used to change the end-of-inputs marker. Note that + this does _not_ change the end-of-options marker. For example, to + use `..' as the marker: + + + zargs -e.. -- **/*(.) .. ls -ld -- + + This is a good choice in that example because no plain file can be + named `..', but the best end-marker depends on the circumstances. + + The options -i, -I, -l, -L, and -n differ slightly from their + usage in xargs. There are no input lines for zargs to count, so + -l and -L count through the INPUT list, and -n counts the number + of arguments passed to each execution of COMMAND, _including_ any + ARG list. Also, any time -i or -I is used, each INPUT is + processed separately as if by `-L 1'. + + For details of the other zargs options, see man page xargs(1) (but + note the difference in function between zargs and xargs) or run + zargs with the --help option. + +zed [ -f [ -x NUM ] ] NAME +zed -b + This function uses the ZLE editor to edit a file or function. + + Only one NAME argument is allowed. If the -f option is given, the + name is taken to be that of a function; if the function is marked + for autoloading, zed searches for it in the fpath and loads it. + Note that functions edited this way are installed into the current + shell, but _not_ written back to the autoload file. In this case + the -x option specifies that leading tabs indenting the function + according to syntax should be converted into the given number of + spaces; `-x 2' is consistent with the layout of functions + distributed with the shell. + + Without -f, NAME is the path name of the file to edit, which need + not exist; it is created on write, if necessary. + + While editing, the function sets the main keymap to zed and the vi + command keymap to zed-vicmd. These will be copied from the + existing main and vicmd keymaps if they do not exist the first + time zed is run. They can be used to provide special key bindings + used only in zed. + + If it creates the keymap, zed rebinds the return key to insert a + line break and `^X^W' to accept the edit in the zed keymap, and + binds `ZZ' to accept the edit in the zed-vicmd keymap. + + The bindings alone can be installed by running `zed -b'. This is + suitable for putting into a startup file. Note that, if rerun, + this will overwrite the existing zed and zed-vicmd keymaps. + + Completion is available, and styles may be set with the context + prefix `:completion:zed'. + + A zle widget zed-set-file-name is available. This can be called by + name from within zed using `\ex zed-set-file-name' (note, however, + that because of zed's rebindings you will have to type ^j at the + end instead of the return key), or can be bound to a key in either + of the zed or zed-vicmd keymaps after `zed -b' has been run. When + the widget is called, it prompts for a new name for the file being + edited. When zed exits the file will be written under that name + and the original file will be left alone. The widget has no + effect with `zed -f'. + + While zed-set-file-name is running, zed uses the keymap + zed-normal-keymap, which is linked from the main keymap in effect + at the time zed initialised its bindings. (This is to make the + return key operate normally.) The result is that if the main + keymap has been changed, the widget won't notice. This is not a + concern for most users. + +zcp [ -finqQvwW ] SRCPAT DEST +zln [ -finqQsvwW ] SRCPAT DEST + Same as zmv -C and zmv -L, respectively. These functions do not + appear in the zsh distribution, but can be created by linking zmv + to the names zcp and zln in some directory in your fpath. + +zkbd + See `Keyboard Definition' (*Note Utilities::). + +zmv [ -finqQsvwW ] [ -C | -L | -M | -{p|P} PROGRAM ] [ -o OPTSTRING ] + SRCPAT DEST + Move (usually, rename) files matching the pattern SRCPAT to + corresponding files having names of the form given by DEST, where + SRCPAT contains parentheses surrounding patterns which will be + replaced in turn by $1, $2, ... in DEST. For example, + + + zmv '(*).lis' '$1.txt' + + renames `foo.lis' to `foo.txt', `my.old.stuff.lis' to + `my.old.stuff.txt', and so on. + + The pattern is always treated as an EXTENDED_GLOB pattern. Any + file whose name is not changed by the substitution is simply + ignored. Any error (a substitution resulted in an empty string, + two substitutions gave the same result, the destination was an + existing regular file and -f was not given) causes the entire + function to abort without doing anything. + + In addition to pattern replacement, the variable $f can be + referrred to in the second (replacement) argument. This makes it + possible to use variable substitution to alter the argument; see + examples below. + + Options: + + + -f + Force overwriting of destination files. Not currently passed + down to the mv/cp/ln command due to vagaries of + implementations (but you can use -o-f to do that). + + -i + Interactive: show each line to be executed and ask the user + whether to execute it. `Y' or `y' will execute it, anything + else will skip it. Note that you just need to type one + character. + + -n + No execution: print what would happen, but don't do it. + + -q + Turn bare glob qualifiers off: now assumed by default, so + this has no effect. + + -Q + Force bare glob qualifiers on. Don't turn this on unless you + are actually using glob qualifiers in a pattern. + + -s + Symbolic, passed down to ln; only works with -L. + + -v + Verbose: print each command as it's being executed. + + -w + Pick out wildcard parts of the pattern, as described above, + and implicitly add parentheses for referring to them. + + -W + Just like -w, with the addition of turning wildcards in the + replacement pattern into sequential ${1} .. ${N} references. + + -C + -L + -M + Force cp, ln or mv, respectively, regardless of the name of + the function. + + -p PROGRAM + Call PROGRAM instead of cp, ln or mv. Whatever it does, it + should at least understand the form + PROGRAM -- OLDNAME NEWNAME + where OLDNAME and NEWNAME are filenames generated by zmv. + PROGRAM will be split into words, so might be e.g. the name + of an archive tool plus a copy or rename subcommand. + + -P PROGRAM + As -p PROGRAM, except that PROGRAM does not accept a + following -- to indicate the end of options. In this case + filenames must already be in a sane form for the program in + question. + + -o OPTSTRING + The OPTSTRING is split into words and passed down verbatim to + the cp, ln or mv command called to perform the work. It + should probably begin with a `-'. + + Further examples: + + + zmv -v '(* *)' '${1// /_}' + + For any file in the current directory with at least one space in + the name, replace every space by an underscore and display the + commands executed. + + + zmv -v '* *' '${f// /_}' + + This does exactly the same by referring to the file name stored in + $f. + + For more complete examples and other implementation details, see + the zmv source file, usually located in one of the directories + named in your fpath, or in Functions/Misc/zmv in the zsh + distribution. + +zrecompile + See `Recompiling Functions' (*Note Utilities::). + +zstyle+ CONTEXT STYLE VALUE [ + SUBCONTEXT STYLE VALUE ... ] + This makes defining styles a bit simpler by using a single `+' as a + special token that allows you to append a context name to the + previously used context name. Like this: + + + zstyle+ ':foo:bar' STYLE1 VALUE1 \ + +':baz' STYLE2 VALUE2 \ + +':frob' STYLE3 VALUE3 + + This defines STYLE1 with VALUE1 for the context :foo:bar as usual, + but it also defines STYLE2 with VALUE2 for the context + :foo:bar:baz and STYLE3 with VALUE3 for :foo:bar:frob. Any + SUBCONTEXT may be the empty string to re-use the first context + unchanged. + + + +26.12.2 Styles +-------------- + + +insert-tab + The zed function _sets_ this style in context `:completion:zed:*' + to turn off completion when TAB is typed at the beginning of a + line. You may override this by setting your own value for this + context and style. + +pager + The nslookup function looks up this style in the context + `:nslookup' to determine the program used to display output that + does not fit on a single screen. + +prompt +rprompt + The nslookup function looks up this style in the context + `:nslookup' to set the prompt and the right-side prompt, + respectively. The usual expansions for the PS1 and RPS1 + parameters may be used (see *Note Prompt Expansion::). + + diff --git a/assets/info/zsh.info-6 b/assets/info/zsh.info-6 new file mode 100644 index 00000000..6aca74aa Binary files /dev/null and b/assets/info/zsh.info-6 differ diff --git a/assets/info/zsh.info-7 b/assets/info/zsh.info-7 new file mode 100644 index 00000000..2bd2e004 Binary files /dev/null and b/assets/info/zsh.info-7 differ -- cgit v1.2.3